2024-03-29T07:38:55.108 INFO:root:teuthology version: 0.0.1.dev262+ge691533 2024-03-29T07:38:55.108 DEBUG:teuthology.run:Teuthology command: teuthology --archive /home/teuthworker/archive/vshankar-2024-03-28_11:21:59-fs-wip-vshankar-testing-20240326.105515-reef-testing-default-smithi/7628087 --owner scheduled_vshankar@teuthology --description fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} --verbose --name vshankar-2024-03-28_11:21:59-fs-wip-vshankar-testing-20240326.105515-reef-testing-default-smithi -- /home/teuthworker/archive/vshankar-2024-03-28_11:21:59-fs-wip-vshankar-testing-20240326.105515-reef-testing-default-smithi/7628087/orig.config.yaml 2024-03-29T07:38:55.170 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-29T07:38:55.261 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/vshankar-2024-03-28_11:21:59-fs-wip-vshankar-testing-20240326.105515-reef-testing-default-smithi/7628087 branch: wip-vshankar-testing-20240326.105515-reef description: fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} email: vshankar@redhat.com first_in_suite: false job_id: '7628087' kernel: flavor: default kdb: true sha1: 1df0c7e8a2354c42d738e5454ba83ea7332deac2 last_in_suite: false machine_type: smithi meta: - desc: 'setup ceph/quincy ' name: vshankar-2024-03-28_11:21:59-fs-wip-vshankar-testing-20240326.105515-reef-testing-default-smithi no_nested_subset: false nuke-on-error: true os_type: centos os_version: 8.stream overrides: admin_socket: branch: wip-vshankar-testing-20240326.105515-reef ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: mon pg warn min per osd: 0 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - OSD_DOWN - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding sha1: d0e392d9aee635f2b65768c460c7a151ead7a5e2 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: d0e392d9aee635f2b65768c460c7a151ead7a5e2 kclient: syntax: v1 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-vshankar-testing-20240326.105515-reef sha1: d0e392d9aee635f2b65768c460c7a151ead7a5e2 owner: scheduled_vshankar@teuthology priority: 120 repo: https://git.ceph.com/ceph-ci.git roles: - - host.a - client.0 - osd.0 - osd.1 - osd.2 - - host.b - client.1 - osd.3 - osd.4 - osd.5 seed: 7327 sha1: d0e392d9aee635f2b65768c460c7a151ead7a5e2 sleep_before_teardown: 0 subset: 990/1024 suite: fs suite_branch: wip-vshankar-testing-20240326.105515-reef suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_d0e392d9aee635f2b65768c460c7a151ead7a5e2/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: d0e392d9aee635f2b65768c460c7a151ead7a5e2 targets: smithi037.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBG40xk/CPmL/al8W0cAbek/cWvcY1E1HDpyPXO/7dLFpI017G89+aK9jHEmCulUy1Vc1IwZ6wm57F+V5ShTIQwk= smithi148.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAtrgGk5u5xlk1Xeo4rxeod054uNJvPknXxGcaxLta9d8QZ6W3dYLKJuqZ01Wj4+0brVQe6hw1zwCHoWBZR4FW8= tasks: - pexec: all: - sudo cp /etc/containers/registries.conf /etc/containers/registries.conf.backup - sudo dnf -y module reset container-tools - sudo dnf -y module install container-tools --allowerasing --nobest - sudo cp /etc/containers/registries.conf.backup /etc/containers/registries.conf - install: branch: quincy exclude_packages: - ceph-volume - print: '**** done install task...' - cephadm: cephadm_branch: quincy cephadm_git_url: https://github.com/ceph/ceph conf: osd: osd_class_default_list: '*' osd_class_load_list: '*' image: quay.ceph.io/ceph-ci/ceph:quincy roleless: true - print: '**** done end installing quincy cephadm ...' - cephadm.shell: host.a: - ceph config set mgr mgr/cephadm/use_repo_digest true --force - print: '**** done cephadm.shell ceph config set mgr...' - cephadm.shell: host.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: host.a: - ceph fs volume create cephfs --placement=4 - ceph fs dump - cephadm.shell: host.a: - ceph fs set cephfs max_mds 2 - cephadm.shell: host.a: - ceph fs set cephfs allow_standby_replay false - cephadm.shell: host.a: - ceph fs set cephfs inline_data false - cephadm.shell: host.a: - ceph fs dump - ceph --format=json fs dump | jq -e ".filesystems | length == 1" - while ! ceph --format=json mds versions | jq -e ". | add == 4"; do sleep 1; done - fs.pre_upgrade_save: null - ceph-fuse: null - print: '**** done client' - parallel: - upgrade-tasks - workload-tasks - cephadm.shell: host.a: - ceph fs dump - fs.post_upgrade_checks: null teuthology: fragments_dropped: [] meta: {} postmerge: [] variables: fail_fs: true teuthology_branch: main teuthology_sha1: e691533f9cbb33d85b2187bba20d7102f098636d timestamp: 2024-03-28_11:21:59 tube: smithi upgrade-tasks: sequential: - cephadm.shell: env: - sha1 host.a: - ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force - ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force - ceph config set global log_to_journald false --force - ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1 --daemon-types mgr - while ceph orch upgrade status | jq '.in_progress' | grep true && ! ceph orch upgrade status | jq '.message' | grep Error ; do ceph orch ps ; ceph versions ; ceph orch upgrade status ; sleep 30 ; done - ceph versions | jq -e '.mgr | length == 1' - ceph versions | jq -e '.mgr | keys' | grep $sha1 - ceph versions | jq -e '.overall | length == 2' - ceph orch upgrade check quay.ceph.io/ceph-ci/ceph:$sha1 | jq -e '.up_to_date | length == 2' - ceph orch ps - cephadm.shell: env: - sha1 host.a: - ceph config set mgr mgr/orchestrator/fail_fs true - cephadm.shell: env: - sha1 host.a: - ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force - ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force - ceph config set global log_to_journald false --force - ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1 - cephadm.shell: env: - sha1 host.a: - while ceph orch upgrade status | jq '.in_progress' | grep true && ! ceph orch upgrade status | jq '.message' | grep Error ; do ceph orch ps ; ceph versions ; ceph fs dump; ceph orch upgrade status ; ceph health detail ; sleep 30 ; done - ceph orch ps - ceph orch upgrade status - ceph health detail - ceph versions - echo "wait for servicemap items w/ changing names to refresh" - sleep 60 - ceph orch ps - ceph versions - ceph versions | jq -e '.overall | length == 1' - ceph versions | jq -e '.overall | keys' | grep $sha1 user: vshankar verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 workload-tasks: sequential: - workunit: clients: all: - suites/fsstress.sh 2024-03-29T07:38:55.261 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_d0e392d9aee635f2b65768c460c7a151ead7a5e2/qa; will attempt to use it 2024-03-29T07:38:55.262 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_d0e392d9aee635f2b65768c460c7a151ead7a5e2/qa/tasks 2024-03-29T07:38:55.262 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-03-29T07:38:55.264 INFO:teuthology.task.internal:Checking packages... 2024-03-29T07:38:55.290 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'd0e392d9aee635f2b65768c460c7a151ead7a5e2' 2024-03-29T07:38:55.290 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-03-29T07:38:55.290 INFO:teuthology.packaging:ref: None 2024-03-29T07:38:55.290 INFO:teuthology.packaging:tag: None 2024-03-29T07:38:55.290 INFO:teuthology.packaging:branch: wip-vshankar-testing-20240326.105515-reef 2024-03-29T07:38:55.290 INFO:teuthology.packaging:sha1: d0e392d9aee635f2b65768c460c7a151ead7a5e2 2024-03-29T07:38:55.291 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=wip-vshankar-testing-20240326.105515-reef 2024-03-29T07:38:55.508 INFO:teuthology.task.internal:Found packages for ceph version 18.2.2-1015.gd0e392d9 2024-03-29T07:38:55.509 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-03-29T07:38:55.516 INFO:teuthology.task.internal:no buildpackages task found 2024-03-29T07:38:55.516 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-03-29T07:38:55.540 INFO:teuthology.task.internal:Saving configuration 2024-03-29T07:38:55.560 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-03-29T07:38:55.573 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-29T07:38:55.597 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi037.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-03-28_11:21:59-fs-wip-vshankar-testing-20240326.105515-reef-testing-default-smithi/7628087', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-29 07:33:38.158020', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBG40xk/CPmL/al8W0cAbek/cWvcY1E1HDpyPXO/7dLFpI017G89+aK9jHEmCulUy1Vc1IwZ6wm57F+V5ShTIQwk='} 2024-03-29T07:38:55.622 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi148.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-03-28_11:21:59-fs-wip-vshankar-testing-20240326.105515-reef-testing-default-smithi/7628087', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-29 07:33:38.159370', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAtrgGk5u5xlk1Xeo4rxeod054uNJvPknXxGcaxLta9d8QZ6W3dYLKJuqZ01Wj4+0brVQe6hw1zwCHoWBZR4FW8='} 2024-03-29T07:38:55.622 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-03-29T07:38:55.629 INFO:teuthology.task.internal:roles: ubuntu@smithi037.front.sepia.ceph.com - ['host.a', 'client.0', 'osd.0', 'osd.1', 'osd.2'] 2024-03-29T07:38:55.629 INFO:teuthology.task.internal:roles: ubuntu@smithi148.front.sepia.ceph.com - ['host.b', 'client.1', 'osd.3', 'osd.4', 'osd.5'] 2024-03-29T07:38:55.629 INFO:teuthology.run_tasks:Running task console_log... 2024-03-29T07:38:55.705 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7efe57bed0d0>, signals=[15]) 2024-03-29T07:38:55.705 INFO:teuthology.run_tasks:Running task internal.connect... 2024-03-29T07:38:55.712 INFO:teuthology.task.internal:Opening connections... 2024-03-29T07:38:55.712 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi037.front.sepia.ceph.com 2024-03-29T07:38:55.715 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi037.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-29T07:38:55.790 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi148.front.sepia.ceph.com 2024-03-29T07:38:55.791 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi148.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-29T07:38:55.866 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-03-29T07:38:55.873 DEBUG:teuthology.orchestra.run.smithi037:> uname -m 2024-03-29T07:38:55.898 INFO:teuthology.orchestra.run.smithi037.stdout:x86_64 2024-03-29T07:38:55.899 DEBUG:teuthology.orchestra.run.smithi037:> cat /etc/os-release 2024-03-29T07:38:55.960 INFO:teuthology.orchestra.run.smithi037.stdout:NAME="CentOS Stream" 2024-03-29T07:38:55.961 INFO:teuthology.orchestra.run.smithi037.stdout:VERSION="8" 2024-03-29T07:38:55.961 INFO:teuthology.orchestra.run.smithi037.stdout:ID="centos" 2024-03-29T07:38:55.961 INFO:teuthology.orchestra.run.smithi037.stdout:ID_LIKE="rhel fedora" 2024-03-29T07:38:55.961 INFO:teuthology.orchestra.run.smithi037.stdout:VERSION_ID="8" 2024-03-29T07:38:55.961 INFO:teuthology.orchestra.run.smithi037.stdout:PLATFORM_ID="platform:el8" 2024-03-29T07:38:55.961 INFO:teuthology.orchestra.run.smithi037.stdout:PRETTY_NAME="CentOS Stream 8" 2024-03-29T07:38:55.961 INFO:teuthology.orchestra.run.smithi037.stdout:ANSI_COLOR="0;31" 2024-03-29T07:38:55.961 INFO:teuthology.orchestra.run.smithi037.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2024-03-29T07:38:55.961 INFO:teuthology.orchestra.run.smithi037.stdout:HOME_URL="https://centos.org/" 2024-03-29T07:38:55.961 INFO:teuthology.orchestra.run.smithi037.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-29T07:38:55.961 INFO:teuthology.orchestra.run.smithi037.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2024-03-29T07:38:55.961 INFO:teuthology.orchestra.run.smithi037.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-29T07:38:55.962 INFO:teuthology.lock.ops:Updating smithi037.front.sepia.ceph.com on lock server 2024-03-29T07:38:55.978 DEBUG:teuthology.orchestra.run.smithi148:> uname -m 2024-03-29T07:38:56.000 INFO:teuthology.orchestra.run.smithi148.stdout:x86_64 2024-03-29T07:38:56.000 DEBUG:teuthology.orchestra.run.smithi148:> cat /etc/os-release 2024-03-29T07:38:56.062 INFO:teuthology.orchestra.run.smithi148.stdout:NAME="CentOS Stream" 2024-03-29T07:38:56.063 INFO:teuthology.orchestra.run.smithi148.stdout:VERSION="8" 2024-03-29T07:38:56.063 INFO:teuthology.orchestra.run.smithi148.stdout:ID="centos" 2024-03-29T07:38:56.063 INFO:teuthology.orchestra.run.smithi148.stdout:ID_LIKE="rhel fedora" 2024-03-29T07:38:56.063 INFO:teuthology.orchestra.run.smithi148.stdout:VERSION_ID="8" 2024-03-29T07:38:56.063 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM_ID="platform:el8" 2024-03-29T07:38:56.063 INFO:teuthology.orchestra.run.smithi148.stdout:PRETTY_NAME="CentOS Stream 8" 2024-03-29T07:38:56.063 INFO:teuthology.orchestra.run.smithi148.stdout:ANSI_COLOR="0;31" 2024-03-29T07:38:56.063 INFO:teuthology.orchestra.run.smithi148.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2024-03-29T07:38:56.063 INFO:teuthology.orchestra.run.smithi148.stdout:HOME_URL="https://centos.org/" 2024-03-29T07:38:56.063 INFO:teuthology.orchestra.run.smithi148.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-29T07:38:56.063 INFO:teuthology.orchestra.run.smithi148.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2024-03-29T07:38:56.063 INFO:teuthology.orchestra.run.smithi148.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-29T07:38:56.064 INFO:teuthology.lock.ops:Updating smithi148.front.sepia.ceph.com on lock server 2024-03-29T07:38:56.086 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-03-29T07:38:56.095 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-03-29T07:38:56.111 INFO:teuthology.task.internal:Checking for old test directory... 2024-03-29T07:38:56.111 DEBUG:teuthology.orchestra.run.smithi037:> test '!' -e /home/ubuntu/cephtest 2024-03-29T07:38:56.114 DEBUG:teuthology.orchestra.run.smithi148:> test '!' -e /home/ubuntu/cephtest 2024-03-29T07:38:56.135 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-03-29T07:38:56.141 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-03-29T07:38:56.141 DEBUG:teuthology.orchestra.run.smithi037:> test -z $(ls -A /var/lib/ceph) 2024-03-29T07:38:56.176 DEBUG:teuthology.orchestra.run.smithi148:> test -z $(ls -A /var/lib/ceph) 2024-03-29T07:38:56.207 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-03-29T07:38:56.264 INFO:teuthology.run_tasks:Running task kernel... 2024-03-29T07:38:56.280 INFO:teuthology.task.kernel:normalize config orig: {'flavor': 'default', 'kdb': True, 'sha1': '1df0c7e8a2354c42d738e5454ba83ea7332deac2'} 2024-03-29T07:38:56.280 INFO:teuthology.task.kernel:config {'host.a': {'flavor': 'default', 'kdb': True, 'sha1': '1df0c7e8a2354c42d738e5454ba83ea7332deac2'}, 'host.b': {'flavor': 'default', 'kdb': True, 'sha1': '1df0c7e8a2354c42d738e5454ba83ea7332deac2'}}, timeout 300 2024-03-29T07:38:56.280 DEBUG:teuthology.orchestra.run.smithi037:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-29T07:38:56.281 DEBUG:teuthology.orchestra.run.smithi148:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-29T07:38:56.301 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-29T07:38:56.302 DEBUG:teuthology.task.kernel:sha1 for host.b is 1df0c7e8a2354c42d738e5454ba83ea7332deac2 2024-03-29T07:38:56.302 INFO:teuthology.task.kernel:Checking kernel version of host.b, want "1df0c7e8a2354c42d738e5454ba83ea7332deac2"... 2024-03-29T07:38:56.302 DEBUG:teuthology.orchestra.run.smithi148:> uname -r 2024-03-29T07:38:56.302 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-29T07:38:56.303 DEBUG:teuthology.task.kernel:sha1 for host.a is 1df0c7e8a2354c42d738e5454ba83ea7332deac2 2024-03-29T07:38:56.303 INFO:teuthology.task.kernel:Checking kernel version of host.a, want "1df0c7e8a2354c42d738e5454ba83ea7332deac2"... 2024-03-29T07:38:56.303 DEBUG:teuthology.orchestra.run.smithi037:> uname -r 2024-03-29T07:38:56.363 INFO:teuthology.orchestra.run.smithi148.stdout:4.18.0-489.el8.x86_64 2024-03-29T07:38:56.363 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-489.el8.x86_64 vs 1df0c7e8a2354c42d738e5454ba83ea7332deac2 2024-03-29T07:38:56.364 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-03-29T07:38:56.364 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F8%2Fx86_64&sha1=1df0c7e8a2354c42d738e5454ba83ea7332deac2 2024-03-29T07:38:56.367 INFO:teuthology.orchestra.run.smithi037.stdout:4.18.0-489.el8.x86_64 2024-03-29T07:38:56.368 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-489.el8.x86_64 vs 1df0c7e8a2354c42d738e5454ba83ea7332deac2 2024-03-29T07:38:56.368 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-03-29T07:38:56.368 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F8%2Fx86_64&sha1=1df0c7e8a2354c42d738e5454ba83ea7332deac2 2024-03-29T07:38:56.588 DEBUG:teuthology.orchestra.run.smithi037:> sudo yum upgrade -y linux-firmware 2024-03-29T07:38:56.589 DEBUG:teuthology.orchestra.run.smithi148:> sudo yum upgrade -y linux-firmware 2024-03-29T07:39:00.470 INFO:teuthology.orchestra.run.smithi148.stdout:CentOS Stream 8 - AppStream 19 MB/s | 28 MB 00:01 2024-03-29T07:39:00.504 INFO:teuthology.orchestra.run.smithi037.stdout:CentOS Stream 8 - AppStream 20 MB/s | 28 MB 00:01 2024-03-29T07:39:07.537 INFO:teuthology.orchestra.run.smithi148.stdout:CentOS Stream 8 - BaseOS 5.1 MB/s | 10 MB 00:02 2024-03-29T07:39:08.788 INFO:teuthology.orchestra.run.smithi037.stdout:CentOS Stream 8 - BaseOS 3.4 MB/s | 10 MB 00:03 2024-03-29T07:39:10.703 INFO:teuthology.orchestra.run.smithi148.stdout:CentOS Stream 8 - Extras 11 kB/s | 18 kB 00:01 2024-03-29T07:39:10.905 INFO:teuthology.orchestra.run.smithi037.stdout:CentOS Stream 8 - Extras 42 kB/s | 18 kB 00:00 2024-03-29T07:39:11.012 INFO:teuthology.orchestra.run.smithi148.stdout:CentOS Stream 8 - Extras common packages 38 kB/s | 7.5 kB 00:00 2024-03-29T07:39:11.255 INFO:teuthology.orchestra.run.smithi148.stdout:Copr repo for python3-asyncssh owned by ceph 23 kB/s | 3.5 kB 00:00 2024-03-29T07:39:11.639 INFO:teuthology.orchestra.run.smithi037.stdout:CentOS Stream 8 - Extras common packages 12 kB/s | 7.5 kB 00:00 2024-03-29T07:39:11.899 INFO:teuthology.orchestra.run.smithi037.stdout:Copr repo for python3-asyncssh owned by ceph 22 kB/s | 3.5 kB 00:00 2024-03-29T07:39:12.574 INFO:teuthology.orchestra.run.smithi037.stdout:Extra Packages for Enterprise Linux 28 MB/s | 16 MB 00:00 2024-03-29T07:39:13.955 INFO:teuthology.orchestra.run.smithi148.stdout:Extra Packages for Enterprise Linux 6.2 MB/s | 16 MB 00:02 2024-03-29T07:39:16.516 INFO:teuthology.orchestra.run.smithi037.stdout:lab-extras 411 kB/s | 24 kB 00:00 2024-03-29T07:39:17.780 INFO:teuthology.orchestra.run.smithi148.stdout:lab-extras 374 kB/s | 24 kB 00:00 2024-03-29T07:39:19.901 INFO:teuthology.orchestra.run.smithi037.stdout:Dependencies resolved. 2024-03-29T07:39:19.902 INFO:teuthology.orchestra.run.smithi037.stdout:================================================================================ 2024-03-29T07:39:19.902 INFO:teuthology.orchestra.run.smithi037.stdout: Package Arch Version Repo Size 2024-03-29T07:39:19.902 INFO:teuthology.orchestra.run.smithi037.stdout:================================================================================ 2024-03-29T07:39:19.902 INFO:teuthology.orchestra.run.smithi037.stdout:Upgrading: 2024-03-29T07:39:19.902 INFO:teuthology.orchestra.run.smithi037.stdout: linux-firmware noarch 20240111-121.gitb3132c18.el8 baseos 355 M 2024-03-29T07:39:19.902 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T07:39:19.902 INFO:teuthology.orchestra.run.smithi037.stdout:Transaction Summary 2024-03-29T07:39:19.902 INFO:teuthology.orchestra.run.smithi037.stdout:================================================================================ 2024-03-29T07:39:19.902 INFO:teuthology.orchestra.run.smithi037.stdout:Upgrade 1 Package 2024-03-29T07:39:19.902 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T07:39:19.903 INFO:teuthology.orchestra.run.smithi037.stdout:Total download size: 355 M 2024-03-29T07:39:19.903 INFO:teuthology.orchestra.run.smithi037.stdout:Downloading Packages: 2024-03-29T07:39:20.982 INFO:teuthology.orchestra.run.smithi148.stdout:Dependencies resolved. 2024-03-29T07:39:20.982 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-03-29T07:39:20.982 INFO:teuthology.orchestra.run.smithi148.stdout: Package Arch Version Repo Size 2024-03-29T07:39:20.982 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-03-29T07:39:20.982 INFO:teuthology.orchestra.run.smithi148.stdout:Upgrading: 2024-03-29T07:39:20.982 INFO:teuthology.orchestra.run.smithi148.stdout: linux-firmware noarch 20240111-121.gitb3132c18.el8 baseos 355 M 2024-03-29T07:39:20.983 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T07:39:20.983 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction Summary 2024-03-29T07:39:20.983 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-03-29T07:39:20.983 INFO:teuthology.orchestra.run.smithi148.stdout:Upgrade 1 Package 2024-03-29T07:39:20.983 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T07:39:20.983 INFO:teuthology.orchestra.run.smithi148.stdout:Total download size: 355 M 2024-03-29T07:39:20.983 INFO:teuthology.orchestra.run.smithi148.stdout:Downloading Packages: 2024-03-29T07:39:29.675 INFO:teuthology.orchestra.run.smithi037.stdout:linux-firmware-20240111-121.gitb3132c18.el8.noa 37 MB/s | 355 MB 00:09 2024-03-29T07:39:29.676 INFO:teuthology.orchestra.run.smithi037.stdout:-------------------------------------------------------------------------------- 2024-03-29T07:39:29.676 INFO:teuthology.orchestra.run.smithi037.stdout:Total 36 MB/s | 355 MB 00:09 2024-03-29T07:39:30.526 INFO:teuthology.orchestra.run.smithi148.stdout:linux-firmware-20240111-121.gitb3132c18.el8.noa 37 MB/s | 355 MB 00:09 2024-03-29T07:39:30.526 INFO:teuthology.orchestra.run.smithi148.stdout:-------------------------------------------------------------------------------- 2024-03-29T07:39:30.526 INFO:teuthology.orchestra.run.smithi148.stdout:Total 37 MB/s | 355 MB 00:09 2024-03-29T07:39:31.883 INFO:teuthology.orchestra.run.smithi037.stdout:Running transaction check 2024-03-29T07:39:31.960 INFO:teuthology.orchestra.run.smithi037.stdout:Transaction check succeeded. 2024-03-29T07:39:31.960 INFO:teuthology.orchestra.run.smithi037.stdout:Running transaction test 2024-03-29T07:39:32.626 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction check 2024-03-29T07:39:32.679 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction check succeeded. 2024-03-29T07:39:32.679 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction test 2024-03-29T07:39:34.271 INFO:teuthology.orchestra.run.smithi037.stdout:Transaction test succeeded. 2024-03-29T07:39:34.304 INFO:teuthology.orchestra.run.smithi037.stdout:Running transaction 2024-03-29T07:39:34.879 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction test succeeded. 2024-03-29T07:39:34.907 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction 2024-03-29T07:39:57.914 INFO:teuthology.orchestra.run.smithi148.stdout: Preparing : 1/1 2024-03-29T07:39:58.253 INFO:teuthology.orchestra.run.smithi148.stdout: Upgrading : linux-firmware-20240111-121.gitb3132c18.el8.noarch 1/2 2024-03-29T07:39:58.444 INFO:teuthology.orchestra.run.smithi037.stdout: Preparing : 1/1 2024-03-29T07:39:58.490 INFO:teuthology.orchestra.run.smithi148.stdout: Cleanup : linux-firmware-20230217-113.git83f1d778.el8.noarch 2/2 2024-03-29T07:39:58.719 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: linux-firmware-20230217-113.git83f1d778.el8.noarch 2/2 2024-03-29T07:39:58.719 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : linux-firmware-20240111-121.gitb3132c18.el8.noarch 1/2 2024-03-29T07:39:58.824 INFO:teuthology.orchestra.run.smithi037.stdout: Upgrading : linux-firmware-20240111-121.gitb3132c18.el8.noarch 1/2 2024-03-29T07:39:59.061 INFO:teuthology.orchestra.run.smithi037.stdout: Cleanup : linux-firmware-20230217-113.git83f1d778.el8.noarch 2/2 2024-03-29T07:39:59.214 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : linux-firmware-20230217-113.git83f1d778.el8.noarch 2/2 2024-03-29T07:39:59.214 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T07:39:59.214 INFO:teuthology.orchestra.run.smithi148.stdout:Upgraded: 2024-03-29T07:39:59.215 INFO:teuthology.orchestra.run.smithi148.stdout: linux-firmware-20240111-121.gitb3132c18.el8.noarch 2024-03-29T07:39:59.215 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T07:39:59.215 INFO:teuthology.orchestra.run.smithi148.stdout:Complete! 2024-03-29T07:39:59.311 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: linux-firmware-20230217-113.git83f1d778.el8.noarch 2/2 2024-03-29T07:39:59.311 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : linux-firmware-20240111-121.gitb3132c18.el8.noarch 1/2 2024-03-29T07:39:59.407 INFO:teuthology.task.kernel:Downloading kernel 1df0c7e8a2354c42d738e5454ba83ea7332deac2 on host.b... 2024-03-29T07:39:59.407 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F8%2Fx86_64&sha1=1df0c7e8a2354c42d738e5454ba83ea7332deac2 2024-03-29T07:39:59.638 INFO:teuthology.task.kernel:fetching, builder baseurl is https://1.chacra.ceph.com/r/kernel/testing/1df0c7e8a2354c42d738e5454ba83ea7332deac2/centos/8/flavors/default/x86_64/ 2024-03-29T07:39:59.638 DEBUG:teuthology.orchestra.run.smithi148:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.8.0_g1df0c7e8a235-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://1.chacra.ceph.com/r/kernel/testing/1df0c7e8a2354c42d738e5454ba83ea7332deac2/centos/8/flavors/default/x86_64/ --input-file=- 2024-03-29T07:39:59.641 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi148.front.sepia.ceph.com to complete... 2024-03-29T07:39:59.800 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : linux-firmware-20230217-113.git83f1d778.el8.noarch 2/2 2024-03-29T07:39:59.801 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T07:39:59.801 INFO:teuthology.orchestra.run.smithi037.stdout:Upgraded: 2024-03-29T07:39:59.801 INFO:teuthology.orchestra.run.smithi037.stdout: linux-firmware-20240111-121.gitb3132c18.el8.noarch 2024-03-29T07:39:59.801 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T07:39:59.801 INFO:teuthology.orchestra.run.smithi037.stdout:Complete! 2024-03-29T07:39:59.991 INFO:teuthology.task.kernel:Downloading kernel 1df0c7e8a2354c42d738e5454ba83ea7332deac2 on host.a... 2024-03-29T07:39:59.991 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F8%2Fx86_64&sha1=1df0c7e8a2354c42d738e5454ba83ea7332deac2 2024-03-29T07:40:00.136 INFO:teuthology.task.kernel:fetching, builder baseurl is https://1.chacra.ceph.com/r/kernel/testing/1df0c7e8a2354c42d738e5454ba83ea7332deac2/centos/8/flavors/default/x86_64/ 2024-03-29T07:40:00.136 DEBUG:teuthology.orchestra.run.smithi037:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.8.0_g1df0c7e8a235-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://1.chacra.ceph.com/r/kernel/testing/1df0c7e8a2354c42d738e5454ba83ea7332deac2/centos/8/flavors/default/x86_64/ --input-file=- 2024-03-29T07:40:00.140 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi037.front.sepia.ceph.com to complete... 2024-03-29T07:40:13.793 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29 07:40:13 URL:https://1.chacra.ceph.com/r/kernel/testing/1df0c7e8a2354c42d738e5454ba83ea7332deac2/centos/8/flavors/default/x86_64/kernel-6.8.0_g1df0c7e8a235-1.x86_64.rpm [401601944/401601944] -> "/tmp/kernel.x86_64.rpm" [1] 2024-03-29T07:40:13.793 INFO:teuthology.orchestra.run.smithi148.stderr:FINISHED --2024-03-29 07:40:13-- 2024-03-29T07:40:13.793 INFO:teuthology.orchestra.run.smithi148.stderr:Total wall clock time: 14s 2024-03-29T07:40:13.793 INFO:teuthology.orchestra.run.smithi148.stderr:Downloaded: 1 files, 383M in 14s (27.7 MB/s) 2024-03-29T07:40:13.796 INFO:teuthology.task.kernel:Installing kernel 1df0c7e8a2354c42d738e5454ba83ea7332deac2 on host.b... 2024-03-29T07:40:13.796 DEBUG:teuthology.orchestra.run.smithi148:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-03-29T07:40:15.543 INFO:teuthology.orchestra.run.smithi148.stdout:Verifying... ######################################## 2024-03-29T07:40:15.588 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing... ######################################## 2024-03-29T07:40:15.698 INFO:teuthology.orchestra.run.smithi148.stdout:Updating / installing... 2024-03-29T07:40:29.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29 07:40:29 URL:https://1.chacra.ceph.com/r/kernel/testing/1df0c7e8a2354c42d738e5454ba83ea7332deac2/centos/8/flavors/default/x86_64/kernel-6.8.0_g1df0c7e8a235-1.x86_64.rpm [401601944/401601944] -> "/tmp/kernel.x86_64.rpm" [1] 2024-03-29T07:40:29.656 INFO:teuthology.orchestra.run.smithi037.stderr:FINISHED --2024-03-29 07:40:29-- 2024-03-29T07:40:29.656 INFO:teuthology.orchestra.run.smithi037.stderr:Total wall clock time: 29s 2024-03-29T07:40:29.657 INFO:teuthology.orchestra.run.smithi037.stderr:Downloaded: 1 files, 383M in 29s (13.1 MB/s) 2024-03-29T07:40:29.659 INFO:teuthology.task.kernel:Installing kernel 1df0c7e8a2354c42d738e5454ba83ea7332deac2 on host.a... 2024-03-29T07:40:29.659 DEBUG:teuthology.orchestra.run.smithi037:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-03-29T07:40:31.502 INFO:teuthology.orchestra.run.smithi037.stdout:Verifying... ######################################## 2024-03-29T07:40:31.548 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing... ######################################## 2024-03-29T07:40:31.664 INFO:teuthology.orchestra.run.smithi037.stdout:Updating / installing... 2024-03-29T07:40:56.122 INFO:teuthology.orchestra.run.smithi148.stdout:kernel-6.8.0_g1df0c7e8a235-1 ######################################## 2024-03-29T07:40:57.451 INFO:teuthology.orchestra.run.smithi148.stderr:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-03-29T07:41:14.643 INFO:teuthology.orchestra.run.smithi037.stdout:kernel-6.8.0_g1df0c7e8a235-1 ######################################## 2024-03-29T07:41:18.977 INFO:teuthology.orchestra.run.smithi037.stderr:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-03-29T07:41:19.506 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi148.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-03-29T07:41:19.506 DEBUG:teuthology.orchestra.run.smithi148:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-03-29T07:41:19.535 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235 2024-03-29T07:41:19.535 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/System.map 2024-03-29T07:41:19.535 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/config 2024-03-29T07:41:19.535 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel 2024-03-29T07:41:19.535 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch 2024-03-29T07:41:19.535 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86 2024-03-29T07:41:19.535 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto 2024-03-29T07:41:19.535 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-03-29T07:41:19.535 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-03-29T07:41:19.536 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-03-29T07:41:19.536 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-03-29T07:41:19.536 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-03-29T07:41:19.536 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-03-29T07:41:19.536 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-03-29T07:41:19.536 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/crc32c-intel.ko 2024-03-29T07:41:19.536 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-03-29T07:41:19.537 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-03-29T07:41:19.537 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-03-29T07:41:19.537 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-03-29T07:41:19.537 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/serpent-avx2.ko 2024-03-29T07:41:19.537 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-03-29T07:41:19.537 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-03-29T07:41:19.537 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-03-29T07:41:19.537 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-03-29T07:41:19.537 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-03-29T07:41:19.537 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-03-29T07:41:19.537 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-03-29T07:41:19.537 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kernel 2024-03-29T07:41:19.537 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kernel/cpu 2024-03-29T07:41:19.537 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kernel/cpu/mce 2024-03-29T07:41:19.538 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-03-29T07:41:19.538 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kvm 2024-03-29T07:41:19.538 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kvm/kvm-amd.ko 2024-03-29T07:41:19.538 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kvm/kvm-intel.ko 2024-03-29T07:41:19.538 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kvm/kvm.ko 2024-03-29T07:41:19.538 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto 2024-03-29T07:41:19.538 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/ansi_cprng.ko 2024-03-29T07:41:19.538 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/anubis.ko 2024-03-29T07:41:19.538 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/arc4.ko 2024-03-29T07:41:19.539 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx 2024-03-29T07:41:19.539 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/async_memcpy.ko 2024-03-29T07:41:19.539 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/async_pq.ko 2024-03-29T07:41:19.539 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/async_raid6_recov.ko 2024-03-29T07:41:19.539 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/async_tx.ko 2024-03-29T07:41:19.539 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/async_xor.ko 2024-03-29T07:41:19.539 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/raid6test.ko 2024-03-29T07:41:19.539 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/authenc.ko 2024-03-29T07:41:19.539 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/authencesn.ko 2024-03-29T07:41:19.539 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/blake2b_generic.ko 2024-03-29T07:41:19.539 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/blowfish_common.ko 2024-03-29T07:41:19.539 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/blowfish_generic.ko 2024-03-29T07:41:19.539 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/camellia_generic.ko 2024-03-29T07:41:19.540 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/cast5_generic.ko 2024-03-29T07:41:19.540 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/cast6_generic.ko 2024-03-29T07:41:19.540 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/cast_common.ko 2024-03-29T07:41:19.540 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/ccm.ko 2024-03-29T07:41:19.540 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/cmac.ko 2024-03-29T07:41:19.540 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/crc32_generic.ko 2024-03-29T07:41:19.540 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/crypto_engine.ko 2024-03-29T07:41:19.540 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/crypto_user.ko 2024-03-29T07:41:19.540 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/cts.ko 2024-03-29T07:41:19.540 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/des_generic.ko 2024-03-29T07:41:19.541 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/dh_generic.ko 2024-03-29T07:41:19.541 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/echainiv.ko 2024-03-29T07:41:19.541 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/essiv.ko 2024-03-29T07:41:19.541 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/fcrypt.ko 2024-03-29T07:41:19.541 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/khazad.ko 2024-03-29T07:41:19.541 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/lz4.ko 2024-03-29T07:41:19.541 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/lz4hc.ko 2024-03-29T07:41:19.541 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/md4.ko 2024-03-29T07:41:19.541 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/michael_mic.ko 2024-03-29T07:41:19.541 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/pcbc.ko 2024-03-29T07:41:19.541 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/pcrypt.ko 2024-03-29T07:41:19.541 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/rmd160.ko 2024-03-29T07:41:19.541 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/seed.ko 2024-03-29T07:41:19.541 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/serpent_generic.ko 2024-03-29T07:41:19.541 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/tcrypt.ko 2024-03-29T07:41:19.542 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/tea.ko 2024-03-29T07:41:19.542 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/twofish_common.ko 2024-03-29T07:41:19.542 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/twofish_generic.ko 2024-03-29T07:41:19.542 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/vmac.ko 2024-03-29T07:41:19.542 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/wp512.ko 2024-03-29T07:41:19.542 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/xcbc.ko 2024-03-29T07:41:19.542 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/xor.ko 2024-03-29T07:41:19.542 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/xxhash_generic.ko 2024-03-29T07:41:19.543 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers 2024-03-29T07:41:19.543 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi 2024-03-29T07:41:19.543 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/acpi_ipmi.ko 2024-03-29T07:41:19.543 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/acpi_pad.ko 2024-03-29T07:41:19.543 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/custom_method.ko 2024-03-29T07:41:19.543 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/ec_sys.ko 2024-03-29T07:41:19.544 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/platform_profile.ko 2024-03-29T07:41:19.544 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/sbs.ko 2024-03-29T07:41:19.544 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/sbshc.ko 2024-03-29T07:41:19.544 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/video.ko 2024-03-29T07:41:19.544 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata 2024-03-29T07:41:19.544 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/acard-ahci.ko 2024-03-29T07:41:19.544 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/ahci_platform.ko 2024-03-29T07:41:19.545 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/ata_generic.ko 2024-03-29T07:41:19.545 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/libahci_platform.ko 2024-03-29T07:41:19.545 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_acpi.ko 2024-03-29T07:41:19.545 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_ali.ko 2024-03-29T07:41:19.546 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_amd.ko 2024-03-29T07:41:19.546 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_artop.ko 2024-03-29T07:41:19.546 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_atiixp.ko 2024-03-29T07:41:19.546 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_atp867x.ko 2024-03-29T07:41:19.546 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_cmd640.ko 2024-03-29T07:41:19.547 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_cmd64x.ko 2024-03-29T07:41:19.547 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_cypress.ko 2024-03-29T07:41:19.547 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_efar.ko 2024-03-29T07:41:19.548 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_hpt366.ko 2024-03-29T07:41:19.548 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_hpt37x.ko 2024-03-29T07:41:19.548 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_hpt3x2n.ko 2024-03-29T07:41:19.548 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_hpt3x3.ko 2024-03-29T07:41:19.548 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_it8213.ko 2024-03-29T07:41:19.549 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_it821x.ko 2024-03-29T07:41:19.549 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_jmicron.ko 2024-03-29T07:41:19.549 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_marvell.ko 2024-03-29T07:41:19.549 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_mpiix.ko 2024-03-29T07:41:19.549 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_netcell.ko 2024-03-29T07:41:19.549 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_ninja32.ko 2024-03-29T07:41:19.550 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_ns87410.ko 2024-03-29T07:41:19.550 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_ns87415.ko 2024-03-29T07:41:19.550 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_oldpiix.ko 2024-03-29T07:41:19.550 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_opti.ko 2024-03-29T07:41:19.550 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_optidma.ko 2024-03-29T07:41:19.550 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_pcmcia.ko 2024-03-29T07:41:19.551 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_pdc2027x.ko 2024-03-29T07:41:19.551 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-03-29T07:41:19.551 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_piccolo.ko 2024-03-29T07:41:19.551 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_rdc.ko 2024-03-29T07:41:19.551 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_sch.ko 2024-03-29T07:41:19.551 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_serverworks.ko 2024-03-29T07:41:19.552 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_sil680.ko 2024-03-29T07:41:19.552 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_sis.ko 2024-03-29T07:41:19.552 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_sl82c105.ko 2024-03-29T07:41:19.552 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_triflex.ko 2024-03-29T07:41:19.552 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_via.ko 2024-03-29T07:41:19.552 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pdc_adma.ko 2024-03-29T07:41:19.583 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_inic162x.ko 2024-03-29T07:41:19.583 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_mv.ko 2024-03-29T07:41:19.583 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_nv.ko 2024-03-29T07:41:19.584 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_promise.ko 2024-03-29T07:41:19.584 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_qstor.ko 2024-03-29T07:41:19.584 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_sil.ko 2024-03-29T07:41:19.584 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_sil24.ko 2024-03-29T07:41:19.584 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_sis.ko 2024-03-29T07:41:19.584 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_svw.ko 2024-03-29T07:41:19.584 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_sx4.ko 2024-03-29T07:41:19.584 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_uli.ko 2024-03-29T07:41:19.584 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_via.ko 2024-03-29T07:41:19.585 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_vsc.ko 2024-03-29T07:41:19.585 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/auxdisplay 2024-03-29T07:41:19.585 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/auxdisplay/ks0108.ko 2024-03-29T07:41:19.585 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/base 2024-03-29T07:41:19.585 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/base/regmap 2024-03-29T07:41:19.585 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/base/regmap/regmap-i2c.ko 2024-03-29T07:41:19.585 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/base/regmap/regmap-mmio.ko 2024-03-29T07:41:19.586 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/bcma 2024-03-29T07:41:19.586 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/bcma/bcma.ko 2024-03-29T07:41:19.586 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block 2024-03-29T07:41:19.586 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/aoe 2024-03-29T07:41:19.586 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/aoe/aoe.ko 2024-03-29T07:41:19.586 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/brd.ko 2024-03-29T07:41:19.586 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/drbd 2024-03-29T07:41:19.586 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/drbd/drbd.ko 2024-03-29T07:41:19.586 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/floppy.ko 2024-03-29T07:41:19.587 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/loop.ko 2024-03-29T07:41:19.587 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/mtip32xx 2024-03-29T07:41:19.587 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-03-29T07:41:19.587 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/nbd.ko 2024-03-29T07:41:19.587 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/null_blk 2024-03-29T07:41:19.587 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/null_blk/null_blk.ko 2024-03-29T07:41:19.587 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/pktcdvd.ko 2024-03-29T07:41:19.587 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/rbd.ko 2024-03-29T07:41:19.587 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/virtio_blk.ko 2024-03-29T07:41:19.587 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/xen-blkback 2024-03-29T07:41:19.589 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-03-29T07:41:19.589 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/xen-blkfront.ko 2024-03-29T07:41:19.589 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/zram 2024-03-29T07:41:19.589 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/zram/zram.ko 2024-03-29T07:41:19.589 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char 2024-03-29T07:41:19.589 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hangcheck-timer.ko 2024-03-29T07:41:19.590 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random 2024-03-29T07:41:19.590 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random/amd-rng.ko 2024-03-29T07:41:19.590 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random/intel-rng.ko 2024-03-29T07:41:19.590 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-03-29T07:41:19.590 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random/via-rng.ko 2024-03-29T07:41:19.590 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random/virtio-rng.ko 2024-03-29T07:41:19.590 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi 2024-03-29T07:41:19.590 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-03-29T07:41:19.590 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-03-29T07:41:19.590 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-03-29T07:41:19.590 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_si.ko 2024-03-29T07:41:19.590 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-03-29T07:41:19.590 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-03-29T07:41:19.591 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/lp.ko 2024-03-29T07:41:19.591 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/mwave 2024-03-29T07:41:19.591 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/mwave/mwave.ko 2024-03-29T07:41:19.591 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ppdev.ko 2024-03-29T07:41:19.591 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tlclk.ko 2024-03-29T07:41:19.591 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm 2024-03-29T07:41:19.591 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm.ko 2024-03-29T07:41:19.591 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm_atmel.ko 2024-03-29T07:41:19.591 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm_infineon.ko 2024-03-29T07:41:19.592 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm_nsc.ko 2024-03-29T07:41:19.592 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm_tis.ko 2024-03-29T07:41:19.592 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-03-29T07:41:19.592 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/uv_mmtimer.ko 2024-03-29T07:41:19.592 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/virtio_console.ko 2024-03-29T07:41:19.592 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq 2024-03-29T07:41:19.592 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-03-29T07:41:19.592 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-03-29T07:41:19.592 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/p4-clockmod.ko 2024-03-29T07:41:19.592 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-03-29T07:41:19.593 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/powernow-k8.ko 2024-03-29T07:41:19.593 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/speedstep-lib.ko 2024-03-29T07:41:19.593 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto 2024-03-29T07:41:19.593 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/ccp 2024-03-29T07:41:19.593 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-03-29T07:41:19.593 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/ccp/ccp.ko 2024-03-29T07:41:19.593 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel 2024-03-29T07:41:19.593 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel/qat 2024-03-29T07:41:19.593 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel/qat/qat_common 2024-03-29T07:41:19.593 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-03-29T07:41:19.594 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-03-29T07:41:19.594 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-03-29T07:41:19.594 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/padlock-aes.ko 2024-03-29T07:41:19.594 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/padlock-sha.ko 2024-03-29T07:41:19.594 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/virtio 2024-03-29T07:41:19.594 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-03-29T07:41:19.595 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dca 2024-03-29T07:41:19.595 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dca/dca.ko 2024-03-29T07:41:19.595 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/devfreq 2024-03-29T07:41:19.595 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-03-29T07:41:19.595 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dma 2024-03-29T07:41:19.595 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dma/dw 2024-03-29T07:41:19.595 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dma/dw/dw_dmac.ko 2024-03-29T07:41:19.595 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dma/ioat 2024-03-29T07:41:19.595 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dma/ioat/ioatdma.ko 2024-03-29T07:41:19.595 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac 2024-03-29T07:41:19.595 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/amd64_edac.ko 2024-03-29T07:41:19.596 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/e752x_edac.ko 2024-03-29T07:41:19.596 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/edac_mce_amd.ko 2024-03-29T07:41:19.596 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i3000_edac.ko 2024-03-29T07:41:19.596 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i3200_edac.ko 2024-03-29T07:41:19.596 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i5100_edac.ko 2024-03-29T07:41:19.596 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i5400_edac.ko 2024-03-29T07:41:19.596 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i7300_edac.ko 2024-03-29T07:41:19.596 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i7core_edac.ko 2024-03-29T07:41:19.596 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i82975x_edac.ko 2024-03-29T07:41:19.596 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/ie31200_edac.ko 2024-03-29T07:41:19.596 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/sb_edac.ko 2024-03-29T07:41:19.597 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/x38_edac.ko 2024-03-29T07:41:19.597 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/firmware 2024-03-29T07:41:19.597 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/firmware/edd.ko 2024-03-29T07:41:19.597 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/firmware/iscsi_ibft.ko 2024-03-29T07:41:19.597 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio 2024-03-29T07:41:19.597 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio/gpio-ich.ko 2024-03-29T07:41:19.597 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio/gpio-viperboard.ko 2024-03-29T07:41:19.597 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid 2024-03-29T07:41:19.598 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-appleir.ko 2024-03-29T07:41:19.598 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-aureal.ko 2024-03-29T07:41:19.598 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-axff.ko 2024-03-29T07:41:19.598 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-dr.ko 2024-03-29T07:41:19.598 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-elecom.ko 2024-03-29T07:41:19.598 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-elo.ko 2024-03-29T07:41:19.598 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-emsff.ko 2024-03-29T07:41:19.598 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-gaff.ko 2024-03-29T07:41:19.598 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-gt683r.ko 2024-03-29T07:41:19.598 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-gyration.ko 2024-03-29T07:41:19.599 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-holtek-kbd.ko 2024-03-29T07:41:19.599 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-holtek-mouse.ko 2024-03-29T07:41:19.599 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-holtekff.ko 2024-03-29T07:41:19.599 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-hyperv.ko 2024-03-29T07:41:19.599 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-icade.ko 2024-03-29T07:41:19.599 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-keytouch.ko 2024-03-29T07:41:19.599 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-kye.ko 2024-03-29T07:41:19.599 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-lcpower.ko 2024-03-29T07:41:19.599 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-led.ko 2024-03-29T07:41:19.599 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-lenovo.ko 2024-03-29T07:41:19.599 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-logitech-dj.ko 2024-03-29T07:41:19.599 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-03-29T07:41:19.599 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-multitouch.ko 2024-03-29T07:41:19.600 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-ortek.ko 2024-03-29T07:41:19.600 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-penmount.ko 2024-03-29T07:41:19.600 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-petalynx.ko 2024-03-29T07:41:19.600 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-picolcd.ko 2024-03-29T07:41:19.600 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-pl.ko 2024-03-29T07:41:19.600 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-plantronics.ko 2024-03-29T07:41:19.600 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-primax.ko 2024-03-29T07:41:19.600 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-rmi.ko 2024-03-29T07:41:19.600 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-arvo.ko 2024-03-29T07:41:19.600 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-common.ko 2024-03-29T07:41:19.600 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-isku.ko 2024-03-29T07:41:19.600 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-kone.ko 2024-03-29T07:41:19.600 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-03-29T07:41:19.600 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-konepure.ko 2024-03-29T07:41:19.601 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-03-29T07:41:19.601 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-lua.ko 2024-03-29T07:41:19.601 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-pyra.ko 2024-03-29T07:41:19.601 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-ryos.ko 2024-03-29T07:41:19.601 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-savu.ko 2024-03-29T07:41:19.601 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat.ko 2024-03-29T07:41:19.601 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-saitek.ko 2024-03-29T07:41:19.601 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-samsung.ko 2024-03-29T07:41:19.601 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-sensor-hub.ko 2024-03-29T07:41:19.601 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-sjoy.ko 2024-03-29T07:41:19.601 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-sony.ko 2024-03-29T07:41:19.601 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-speedlink.ko 2024-03-29T07:41:19.601 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-steelseries.ko 2024-03-29T07:41:19.601 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-sunplus.ko 2024-03-29T07:41:19.602 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-thrustmaster.ko 2024-03-29T07:41:19.602 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-tivo.ko 2024-03-29T07:41:19.602 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-tmff.ko 2024-03-29T07:41:19.602 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-topseed.ko 2024-03-29T07:41:19.602 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-twinhan.ko 2024-03-29T07:41:19.602 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-uclogic.ko 2024-03-29T07:41:19.602 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-waltop.ko 2024-03-29T07:41:19.602 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-wiimote.ko 2024-03-29T07:41:19.603 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-xinmo.ko 2024-03-29T07:41:19.603 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-zpff.ko 2024-03-29T07:41:19.603 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-zydacron.ko 2024-03-29T07:41:19.603 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/uhid.ko 2024-03-29T07:41:19.603 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/wacom.ko 2024-03-29T07:41:19.603 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hv 2024-03-29T07:41:19.603 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hv/hv_balloon.ko 2024-03-29T07:41:19.603 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hv/hv_utils.ko 2024-03-29T07:41:19.603 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hv/hv_vmbus.ko 2024-03-29T07:41:19.603 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon 2024-03-29T07:41:19.603 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/abituguru.ko 2024-03-29T07:41:19.603 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/abituguru3.ko 2024-03-29T07:41:19.603 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/acpi_power_meter.ko 2024-03-29T07:41:19.604 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ad7414.ko 2024-03-29T07:41:19.604 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ad7418.ko 2024-03-29T07:41:19.604 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adc128d818.ko 2024-03-29T07:41:19.604 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adm1025.ko 2024-03-29T07:41:19.604 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adm1026.ko 2024-03-29T07:41:19.604 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adm1029.ko 2024-03-29T07:41:19.604 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adm1031.ko 2024-03-29T07:41:19.604 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adm9240.ko 2024-03-29T07:41:19.604 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ads7828.ko 2024-03-29T07:41:19.604 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7410.ko 2024-03-29T07:41:19.604 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7411.ko 2024-03-29T07:41:19.604 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7462.ko 2024-03-29T07:41:19.604 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7470.ko 2024-03-29T07:41:19.604 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7475.ko 2024-03-29T07:41:19.605 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7x10.ko 2024-03-29T07:41:19.605 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/amc6821.ko 2024-03-29T07:41:19.605 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/applesmc.ko 2024-03-29T07:41:19.605 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/asb100.ko 2024-03-29T07:41:19.605 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/asc7621.ko 2024-03-29T07:41:19.605 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/asus_atk0110.ko 2024-03-29T07:41:19.605 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/atxp1.ko 2024-03-29T07:41:19.605 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/coretemp.ko 2024-03-29T07:41:19.605 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-03-29T07:41:19.605 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/dme1737.ko 2024-03-29T07:41:19.605 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ds1621.ko 2024-03-29T07:41:19.605 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ds620.ko 2024-03-29T07:41:19.606 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/emc1403.ko 2024-03-29T07:41:19.606 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/emc6w201.ko 2024-03-29T07:41:19.606 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/f71805f.ko 2024-03-29T07:41:19.606 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/f71882fg.ko 2024-03-29T07:41:19.606 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/f75375s.ko 2024-03-29T07:41:19.606 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/fam15h_power.ko 2024-03-29T07:41:19.606 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/fschmd.ko 2024-03-29T07:41:19.606 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/g760a.ko 2024-03-29T07:41:19.606 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/g762.ko 2024-03-29T07:41:19.606 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/gl518sm.ko 2024-03-29T07:41:19.606 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/gl520sm.ko 2024-03-29T07:41:19.606 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/hwmon-vid.ko 2024-03-29T07:41:19.606 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/i5500_temp.ko 2024-03-29T07:41:19.606 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/i5k_amb.ko 2024-03-29T07:41:19.607 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ibmaem.ko 2024-03-29T07:41:19.607 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ibmpex.ko 2024-03-29T07:41:19.607 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ina209.ko 2024-03-29T07:41:19.607 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ina2xx.ko 2024-03-29T07:41:19.607 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/it87.ko 2024-03-29T07:41:19.607 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/k10temp.ko 2024-03-29T07:41:19.607 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/k8temp.ko 2024-03-29T07:41:19.607 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lineage-pem.ko 2024-03-29T07:41:19.607 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm63.ko 2024-03-29T07:41:19.607 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm73.ko 2024-03-29T07:41:19.607 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm75.ko 2024-03-29T07:41:19.607 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm77.ko 2024-03-29T07:41:19.607 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm78.ko 2024-03-29T07:41:19.607 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm80.ko 2024-03-29T07:41:19.608 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm83.ko 2024-03-29T07:41:19.608 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm85.ko 2024-03-29T07:41:19.608 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm87.ko 2024-03-29T07:41:19.608 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm90.ko 2024-03-29T07:41:19.608 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm92.ko 2024-03-29T07:41:19.608 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm93.ko 2024-03-29T07:41:19.608 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm95234.ko 2024-03-29T07:41:19.608 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm95241.ko 2024-03-29T07:41:19.608 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm95245.ko 2024-03-29T07:41:19.608 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc2945.ko 2024-03-29T07:41:19.608 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4151.ko 2024-03-29T07:41:19.608 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4215.ko 2024-03-29T07:41:19.608 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4222.ko 2024-03-29T07:41:19.608 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4245.ko 2024-03-29T07:41:19.609 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4260.ko 2024-03-29T07:41:19.609 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4261.ko 2024-03-29T07:41:19.609 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max16065.ko 2024-03-29T07:41:19.609 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max1619.ko 2024-03-29T07:41:19.609 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max1668.ko 2024-03-29T07:41:19.609 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max197.ko 2024-03-29T07:41:19.609 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max6639.ko 2024-03-29T07:41:19.609 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max6650.ko 2024-03-29T07:41:19.609 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max6697.ko 2024-03-29T07:41:19.609 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/mcp3021.ko 2024-03-29T07:41:19.609 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/nct6683.ko 2024-03-29T07:41:19.609 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/nct6775-core.ko 2024-03-29T07:41:19.609 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/nct6775.ko 2024-03-29T07:41:19.609 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/nct7802.ko 2024-03-29T07:41:19.609 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pc87360.ko 2024-03-29T07:41:19.610 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pc87427.ko 2024-03-29T07:41:19.610 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pcf8591.ko 2024-03-29T07:41:19.610 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus 2024-03-29T07:41:19.610 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-03-29T07:41:19.610 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-03-29T07:41:19.610 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-03-29T07:41:19.610 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/max16064.ko 2024-03-29T07:41:19.610 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/max34440.ko 2024-03-29T07:41:19.610 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/max8688.ko 2024-03-29T07:41:19.610 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-03-29T07:41:19.610 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-03-29T07:41:19.610 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-03-29T07:41:19.610 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-03-29T07:41:19.611 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-03-29T07:41:19.611 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-03-29T07:41:19.611 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/powr1220.ko 2024-03-29T07:41:19.611 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sch5627.ko 2024-03-29T07:41:19.611 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sch5636.ko 2024-03-29T07:41:19.611 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sch56xx-common.ko 2024-03-29T07:41:19.611 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sht15.ko 2024-03-29T07:41:19.611 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sht21.ko 2024-03-29T07:41:19.611 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/shtc1.ko 2024-03-29T07:41:19.611 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sis5595.ko 2024-03-29T07:41:19.611 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/smsc47b397.ko 2024-03-29T07:41:19.611 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/smsc47m1.ko 2024-03-29T07:41:19.611 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/smsc47m192.ko 2024-03-29T07:41:19.611 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/thmc50.ko 2024-03-29T07:41:19.611 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/tmp102.ko 2024-03-29T07:41:19.612 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/tmp103.ko 2024-03-29T07:41:19.612 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/tmp401.ko 2024-03-29T07:41:19.612 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/tmp421.ko 2024-03-29T07:41:19.612 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/via-cputemp.ko 2024-03-29T07:41:19.612 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/via686a.ko 2024-03-29T07:41:19.612 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/vt1211.ko 2024-03-29T07:41:19.612 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/vt8231.ko 2024-03-29T07:41:19.612 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83627ehf.ko 2024-03-29T07:41:19.612 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83627hf.ko 2024-03-29T07:41:19.612 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83781d.ko 2024-03-29T07:41:19.612 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83791d.ko 2024-03-29T07:41:19.612 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83792d.ko 2024-03-29T07:41:19.612 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83793.ko 2024-03-29T07:41:19.612 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83795.ko 2024-03-29T07:41:19.613 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83l785ts.ko 2024-03-29T07:41:19.613 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83l786ng.ko 2024-03-29T07:41:19.613 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c 2024-03-29T07:41:19.613 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/algos 2024-03-29T07:41:19.613 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-03-29T07:41:19.613 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-03-29T07:41:19.613 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses 2024-03-29T07:41:19.613 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-03-29T07:41:19.613 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-03-29T07:41:19.613 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-03-29T07:41:19.613 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-03-29T07:41:19.613 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-03-29T07:41:19.613 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-03-29T07:41:19.613 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-03-29T07:41:19.614 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-03-29T07:41:19.614 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-i801.ko 2024-03-29T07:41:19.614 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-isch.ko 2024-03-29T07:41:19.614 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-03-29T07:41:19.614 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-03-29T07:41:19.614 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-03-29T07:41:19.614 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-parport.ko 2024-03-29T07:41:19.614 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-03-29T07:41:19.614 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-03-29T07:41:19.614 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-03-29T07:41:19.614 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-03-29T07:41:19.614 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-03-29T07:41:19.614 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-03-29T07:41:19.614 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-via.ko 2024-03-29T07:41:19.615 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-03-29T07:41:19.615 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-03-29T07:41:19.615 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/i2c-dev.ko 2024-03-29T07:41:19.615 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/i2c-mux.ko 2024-03-29T07:41:19.615 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-03-29T07:41:19.615 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/i2c-smbus.ko 2024-03-29T07:41:19.615 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/i2c-stub.ko 2024-03-29T07:41:19.615 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband 2024-03-29T07:41:19.615 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core 2024-03-29T07:41:19.615 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/ib_cm.ko 2024-03-29T07:41:19.615 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/ib_core.ko 2024-03-29T07:41:19.615 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/ib_umad.ko 2024-03-29T07:41:19.615 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-03-29T07:41:19.615 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/iw_cm.ko 2024-03-29T07:41:19.615 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/rdma_cm.ko 2024-03-29T07:41:19.616 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-03-29T07:41:19.616 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw 2024-03-29T07:41:19.616 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/cxgb4 2024-03-29T07:41:19.616 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-03-29T07:41:19.616 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mlx4 2024-03-29T07:41:19.616 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-03-29T07:41:19.616 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mlx5 2024-03-29T07:41:19.616 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-03-29T07:41:19.616 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mthca 2024-03-29T07:41:19.616 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-03-29T07:41:19.616 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp 2024-03-29T07:41:19.617 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/ipoib 2024-03-29T07:41:19.617 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-03-29T07:41:19.617 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/iser 2024-03-29T07:41:19.617 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-03-29T07:41:19.617 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/isert 2024-03-29T07:41:19.617 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-03-29T07:41:19.617 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/srp 2024-03-29T07:41:19.617 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-03-29T07:41:19.617 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/srpt 2024-03-29T07:41:19.617 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-03-29T07:41:19.617 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input 2024-03-29T07:41:19.617 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/gameport 2024-03-29T07:41:19.617 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-03-29T07:41:19.617 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/gameport/fm801-gp.ko 2024-03-29T07:41:19.617 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/gameport/gameport.ko 2024-03-29T07:41:19.618 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/joydev.ko 2024-03-29T07:41:19.618 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/keyboard 2024-03-29T07:41:19.618 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/keyboard/gpio_keys.ko 2024-03-29T07:41:19.618 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse 2024-03-29T07:41:19.618 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/appletouch.ko 2024-03-29T07:41:19.618 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/bcm5974.ko 2024-03-29T07:41:19.618 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/cyapatp.ko 2024-03-29T07:41:19.618 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/elan_i2c.ko 2024-03-29T07:41:19.618 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/sermouse.ko 2024-03-29T07:41:19.618 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-03-29T07:41:19.618 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/synaptics_usb.ko 2024-03-29T07:41:19.618 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/vsxxxaa.ko 2024-03-29T07:41:19.619 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/rmi4 2024-03-29T07:41:19.619 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/rmi4/rmi_core.ko 2024-03-29T07:41:19.619 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio 2024-03-29T07:41:19.619 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio/altera_ps2.ko 2024-03-29T07:41:19.619 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio/arc_ps2.ko 2024-03-29T07:41:19.619 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-03-29T07:41:19.619 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio/serio_raw.ko 2024-03-29T07:41:19.619 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/sparse-keymap.ko 2024-03-29T07:41:19.619 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds 2024-03-29T07:41:19.619 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-blinkm.ko 2024-03-29T07:41:19.619 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-lm3530.ko 2024-03-29T07:41:19.619 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-lp3944.ko 2024-03-29T07:41:19.619 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-ss4200.ko 2024-03-29T07:41:19.619 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger 2024-03-29T07:41:19.620 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-03-29T07:41:19.620 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-03-29T07:41:19.620 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-03-29T07:41:19.620 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-03-29T07:41:19.620 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-03-29T07:41:19.620 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-03-29T07:41:19.620 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-03-29T07:41:19.620 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-03-29T07:41:19.620 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-03-29T07:41:19.620 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md 2024-03-29T07:41:19.620 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/bcache 2024-03-29T07:41:19.620 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/bcache/bcache.ko 2024-03-29T07:41:19.621 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-bio-prison.ko 2024-03-29T07:41:19.621 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-cache-smq.ko 2024-03-29T07:41:19.621 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-cache.ko 2024-03-29T07:41:19.621 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-crypt.ko 2024-03-29T07:41:19.621 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-delay.ko 2024-03-29T07:41:19.621 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-flakey.ko 2024-03-29T07:41:19.621 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-log-userspace.ko 2024-03-29T07:41:19.621 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-multipath.ko 2024-03-29T07:41:19.622 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-queue-length.ko 2024-03-29T07:41:19.622 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-raid.ko 2024-03-29T07:41:19.622 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-round-robin.ko 2024-03-29T07:41:19.622 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-service-time.ko 2024-03-29T07:41:19.622 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-switch.ko 2024-03-29T07:41:19.622 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-thin-pool.ko 2024-03-29T07:41:19.622 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-verity.ko 2024-03-29T07:41:19.623 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/persistent-data 2024-03-29T07:41:19.623 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-03-29T07:41:19.623 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/raid0.ko 2024-03-29T07:41:19.623 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/raid1.ko 2024-03-29T07:41:19.623 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/raid10.ko 2024-03-29T07:41:19.623 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/raid456.ko 2024-03-29T07:41:19.623 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message 2024-03-29T07:41:19.623 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion 2024-03-29T07:41:19.624 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptbase.ko 2024-03-29T07:41:19.624 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptctl.ko 2024-03-29T07:41:19.624 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptfc.ko 2024-03-29T07:41:19.624 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptlan.ko 2024-03-29T07:41:19.624 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptsas.ko 2024-03-29T07:41:19.624 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptscsih.ko 2024-03-29T07:41:19.624 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptspi.ko 2024-03-29T07:41:19.625 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd 2024-03-29T07:41:19.625 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/lpc_ich.ko 2024-03-29T07:41:19.625 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/lpc_sch.ko 2024-03-29T07:41:19.625 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/mfd-core.ko 2024-03-29T07:41:19.625 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/sm501.ko 2024-03-29T07:41:19.625 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/viperboard.ko 2024-03-29T07:41:19.625 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/vx855.ko 2024-03-29T07:41:19.625 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/wl1273-core.ko 2024-03-29T07:41:19.625 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc 2024-03-29T07:41:19.625 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/altera-stapl 2024-03-29T07:41:19.625 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-03-29T07:41:19.626 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/apds9802als.ko 2024-03-29T07:41:19.626 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/apds990x.ko 2024-03-29T07:41:19.626 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/bh1770glc.ko 2024-03-29T07:41:19.626 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/cb710 2024-03-29T07:41:19.626 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/cb710/cb710.ko 2024-03-29T07:41:19.626 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/eeprom 2024-03-29T07:41:19.626 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/eeprom/at24.ko 2024-03-29T07:41:19.626 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-03-29T07:41:19.626 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/eeprom/max6875.ko 2024-03-29T07:41:19.626 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/enclosure.ko 2024-03-29T07:41:19.626 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/hpilo.ko 2024-03-29T07:41:19.626 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/isl29003.ko 2024-03-29T07:41:19.626 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/isl29020.ko 2024-03-29T07:41:19.627 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/lis3lv02d 2024-03-29T07:41:19.627 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-03-29T07:41:19.627 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-03-29T07:41:19.627 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/mei 2024-03-29T07:41:19.627 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/mei/mei-me.ko 2024-03-29T07:41:19.627 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/mei/mei-txe.ko 2024-03-29T07:41:19.627 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/mei/mei.ko 2024-03-29T07:41:19.627 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-gru 2024-03-29T07:41:19.627 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-gru/gru.ko 2024-03-29T07:41:19.627 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-xp 2024-03-29T07:41:19.627 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-xp/xp.ko 2024-03-29T07:41:19.627 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-xp/xpc.ko 2024-03-29T07:41:19.628 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-03-29T07:41:19.628 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/tifm_7xx1.ko 2024-03-29T07:41:19.628 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/tifm_core.ko 2024-03-29T07:41:19.628 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/tsl2550.ko 2024-03-29T07:41:19.628 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/vmw_balloon.ko 2024-03-29T07:41:19.628 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/vmw_vmci 2024-03-29T07:41:19.628 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-03-29T07:41:19.628 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net 2024-03-29T07:41:19.628 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/bonding 2024-03-29T07:41:19.628 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/bonding/bonding.ko 2024-03-29T07:41:19.628 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa 2024-03-29T07:41:19.629 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/b53 2024-03-29T07:41:19.629 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/b53/b53_common.ko 2024-03-29T07:41:19.629 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/bcm-sf2.ko 2024-03-29T07:41:19.629 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/mv88e6060.ko 2024-03-29T07:41:19.629 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/mv88e6xxx 2024-03-29T07:41:19.629 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-03-29T07:41:19.629 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dummy.ko 2024-03-29T07:41:19.629 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/eql.ko 2024-03-29T07:41:19.629 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet 2024-03-29T07:41:19.629 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/3com 2024-03-29T07:41:19.629 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-03-29T07:41:19.629 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-03-29T07:41:19.629 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-03-29T07:41:19.629 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-03-29T07:41:19.629 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/8390 2024-03-29T07:41:19.630 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/8390/8390.ko 2024-03-29T07:41:19.630 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-03-29T07:41:19.630 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-03-29T07:41:19.630 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-03-29T07:41:19.630 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/adaptec 2024-03-29T07:41:19.630 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-03-29T07:41:19.630 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/agere 2024-03-29T07:41:19.630 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/agere/et131x.ko 2024-03-29T07:41:19.630 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/alteon 2024-03-29T07:41:19.630 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-03-29T07:41:19.630 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/altera 2024-03-29T07:41:19.630 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-03-29T07:41:19.630 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/amd 2024-03-29T07:41:19.630 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-03-29T07:41:19.631 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-03-29T07:41:19.631 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-03-29T07:41:19.631 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros 2024-03-29T07:41:19.631 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/alx 2024-03-29T07:41:19.631 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-03-29T07:41:19.631 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atl1c 2024-03-29T07:41:19.631 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-03-29T07:41:19.631 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atl1e 2024-03-29T07:41:19.631 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-03-29T07:41:19.631 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atlx 2024-03-29T07:41:19.631 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-03-29T07:41:19.631 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-03-29T07:41:19.632 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom 2024-03-29T07:41:19.632 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-03-29T07:41:19.632 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-03-29T07:41:19.632 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-03-29T07:41:19.632 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-03-29T07:41:19.632 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-03-29T07:41:19.632 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/genet 2024-03-29T07:41:19.632 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-03-29T07:41:19.632 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-03-29T07:41:19.632 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/brocade 2024-03-29T07:41:19.632 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/brocade/bna 2024-03-29T07:41:19.632 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-03-29T07:41:19.632 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio 2024-03-29T07:41:19.632 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb 2024-03-29T07:41:19.632 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-03-29T07:41:19.633 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-03-29T07:41:19.633 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-03-29T07:41:19.633 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-03-29T07:41:19.633 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-03-29T07:41:19.633 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-03-29T07:41:19.633 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-03-29T07:41:19.633 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-03-29T07:41:19.633 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-03-29T07:41:19.633 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/cisco 2024-03-29T07:41:19.633 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/cisco/enic 2024-03-29T07:41:19.633 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-03-29T07:41:19.633 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec 2024-03-29T07:41:19.633 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip 2024-03-29T07:41:19.633 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-03-29T07:41:19.633 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-03-29T07:41:19.634 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-03-29T07:41:19.634 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-03-29T07:41:19.634 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-03-29T07:41:19.634 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-03-29T07:41:19.634 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dlink 2024-03-29T07:41:19.634 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-03-29T07:41:19.634 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-03-29T07:41:19.634 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dnet.ko 2024-03-29T07:41:19.634 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/emulex 2024-03-29T07:41:19.634 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/emulex/benet 2024-03-29T07:41:19.634 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-03-29T07:41:19.634 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/ethoc.ko 2024-03-29T07:41:19.634 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/fealnx.ko 2024-03-29T07:41:19.634 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel 2024-03-29T07:41:19.634 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/e100.ko 2024-03-29T07:41:19.635 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/e1000 2024-03-29T07:41:19.635 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-03-29T07:41:19.635 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/e1000e 2024-03-29T07:41:19.635 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-03-29T07:41:19.635 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/fm10k 2024-03-29T07:41:19.635 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-03-29T07:41:19.635 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/i40e 2024-03-29T07:41:19.635 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-03-29T07:41:19.635 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/iavf 2024-03-29T07:41:19.635 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-03-29T07:41:19.635 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/igb 2024-03-29T07:41:19.635 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-03-29T07:41:19.635 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/igbvf 2024-03-29T07:41:19.635 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-03-29T07:41:19.636 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/ixgbe 2024-03-29T07:41:19.636 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-03-29T07:41:19.636 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/ixgbevf 2024-03-29T07:41:19.636 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-03-29T07:41:19.636 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/jme.ko 2024-03-29T07:41:19.636 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/marvell 2024-03-29T07:41:19.636 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-03-29T07:41:19.636 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/marvell/skge.ko 2024-03-29T07:41:19.636 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-03-29T07:41:19.636 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox 2024-03-29T07:41:19.636 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx4 2024-03-29T07:41:19.636 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-03-29T07:41:19.636 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-03-29T07:41:19.636 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx5 2024-03-29T07:41:19.636 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-03-29T07:41:19.637 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-03-29T07:41:19.637 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/micrel 2024-03-29T07:41:19.637 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-03-29T07:41:19.637 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/myricom 2024-03-29T07:41:19.637 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/myricom/myri10ge 2024-03-29T07:41:19.637 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-03-29T07:41:19.637 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/natsemi 2024-03-29T07:41:19.637 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-03-29T07:41:19.637 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-03-29T07:41:19.637 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/neterion 2024-03-29T07:41:19.637 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-03-29T07:41:19.637 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/nvidia 2024-03-29T07:41:19.637 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-03-29T07:41:19.637 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/packetengines 2024-03-29T07:41:19.637 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-03-29T07:41:19.638 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-03-29T07:41:19.638 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic 2024-03-29T07:41:19.638 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic/netxen 2024-03-29T07:41:19.638 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-03-29T07:41:19.638 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-03-29T07:41:19.638 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-03-29T07:41:19.638 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-03-29T07:41:19.638 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/rdc 2024-03-29T07:41:19.638 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-03-29T07:41:19.638 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/realtek 2024-03-29T07:41:19.638 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-03-29T07:41:19.638 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-03-29T07:41:19.638 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/realtek/atp.ko 2024-03-29T07:41:19.639 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-03-29T07:41:19.639 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/rocker 2024-03-29T07:41:19.639 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-03-29T07:41:19.639 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sfc 2024-03-29T07:41:19.639 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-03-29T07:41:19.639 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/silan 2024-03-29T07:41:19.639 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-03-29T07:41:19.639 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sis 2024-03-29T07:41:19.639 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sis/sis190.ko 2024-03-29T07:41:19.639 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sis/sis900.ko 2024-03-29T07:41:19.639 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/smsc 2024-03-29T07:41:19.639 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-03-29T07:41:19.639 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-03-29T07:41:19.639 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-03-29T07:41:19.639 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-03-29T07:41:19.640 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/stmicro 2024-03-29T07:41:19.640 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/stmicro/stmmac 2024-03-29T07:41:19.640 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-03-29T07:41:19.640 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-03-29T07:41:19.640 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sun 2024-03-29T07:41:19.640 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sun/cassini.ko 2024-03-29T07:41:19.640 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sun/niu.ko 2024-03-29T07:41:19.640 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sun/sungem.ko 2024-03-29T07:41:19.640 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-03-29T07:41:19.640 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/tehuti 2024-03-29T07:41:19.640 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-03-29T07:41:19.640 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/ti 2024-03-29T07:41:19.640 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/ti/tlan.ko 2024-03-29T07:41:19.641 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/via 2024-03-29T07:41:19.641 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-03-29T07:41:19.641 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-03-29T07:41:19.641 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/wiznet 2024-03-29T07:41:19.641 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-03-29T07:41:19.641 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-03-29T07:41:19.641 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/xircom 2024-03-29T07:41:19.641 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-03-29T07:41:19.641 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/geneve.ko 2024-03-29T07:41:19.642 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/hyperv 2024-03-29T07:41:19.642 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-03-29T07:41:19.642 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ieee802154 2024-03-29T07:41:19.642 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ieee802154/fakelb.ko 2024-03-29T07:41:19.642 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ifb.ko 2024-03-29T07:41:19.642 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ipvlan 2024-03-29T07:41:19.642 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ipvlan/ipvlan.ko 2024-03-29T07:41:19.642 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/macvlan.ko 2024-03-29T07:41:19.642 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/macvtap.ko 2024-03-29T07:41:19.642 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mdio 2024-03-29T07:41:19.642 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mdio.ko 2024-03-29T07:41:19.642 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-03-29T07:41:19.642 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-03-29T07:41:19.642 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mdio/mdio-regmap.ko 2024-03-29T07:41:19.643 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mii.ko 2024-03-29T07:41:19.643 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/net_failover.ko 2024-03-29T07:41:19.643 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/netconsole.ko 2024-03-29T07:41:19.643 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/nlmon.ko 2024-03-29T07:41:19.643 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/pcs 2024-03-29T07:41:19.643 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/pcs/pcs-lynx.ko 2024-03-29T07:41:19.643 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-03-29T07:41:19.643 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy 2024-03-29T07:41:19.643 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/amd.ko 2024-03-29T07:41:19.643 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-03-29T07:41:19.643 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-03-29T07:41:19.643 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/bcm7xxx.ko 2024-03-29T07:41:19.643 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/bcm87xx.ko 2024-03-29T07:41:19.644 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/broadcom.ko 2024-03-29T07:41:19.644 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/cicada.ko 2024-03-29T07:41:19.644 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/davicom.ko 2024-03-29T07:41:19.644 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/dp83640.ko 2024-03-29T07:41:19.644 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/et1011c.ko 2024-03-29T07:41:19.644 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/icplus.ko 2024-03-29T07:41:19.644 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/lxt.ko 2024-03-29T07:41:19.644 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/marvell.ko 2024-03-29T07:41:19.644 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/micrel.ko 2024-03-29T07:41:19.644 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/national.ko 2024-03-29T07:41:19.645 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/phylink.ko 2024-03-29T07:41:19.645 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/qsemi.ko 2024-03-29T07:41:19.645 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/realtek.ko 2024-03-29T07:41:19.645 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/smsc.ko 2024-03-29T07:41:19.645 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/ste10Xp.ko 2024-03-29T07:41:19.645 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/vitesse.ko 2024-03-29T07:41:19.645 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp 2024-03-29T07:41:19.646 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/bsd_comp.ko 2024-03-29T07:41:19.646 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/ppp_async.ko 2024-03-29T07:41:19.646 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/ppp_deflate.ko 2024-03-29T07:41:19.646 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/ppp_generic.ko 2024-03-29T07:41:19.646 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/ppp_mppe.ko 2024-03-29T07:41:19.646 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/ppp_synctty.ko 2024-03-29T07:41:19.646 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/pppoe.ko 2024-03-29T07:41:19.646 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/pppox.ko 2024-03-29T07:41:19.646 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/pptp.ko 2024-03-29T07:41:19.646 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/slip 2024-03-29T07:41:19.646 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/slip/slhc.ko 2024-03-29T07:41:19.646 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/slip/slip.ko 2024-03-29T07:41:19.647 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/sungem_phy.ko 2024-03-29T07:41:19.647 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/tap.ko 2024-03-29T07:41:19.647 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team 2024-03-29T07:41:19.647 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team.ko 2024-03-29T07:41:19.647 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team_mode_activebackup.ko 2024-03-29T07:41:19.647 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team_mode_broadcast.ko 2024-03-29T07:41:19.647 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-03-29T07:41:19.647 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team_mode_random.ko 2024-03-29T07:41:19.647 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-03-29T07:41:19.647 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/tun.ko 2024-03-29T07:41:19.647 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/veth.ko 2024-03-29T07:41:19.647 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/virtio_net.ko 2024-03-29T07:41:19.647 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/vmxnet3 2024-03-29T07:41:19.647 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-03-29T07:41:19.647 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/vxlan 2024-03-29T07:41:19.648 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/vxlan/vxlan.ko 2024-03-29T07:41:19.648 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/xen-netback 2024-03-29T07:41:19.648 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/xen-netback/xen-netback.ko 2024-03-29T07:41:19.648 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/xen-netfront.ko 2024-03-29T07:41:19.648 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ntb 2024-03-29T07:41:19.648 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ntb/ntb.ko 2024-03-29T07:41:19.648 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/nvme 2024-03-29T07:41:19.648 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/nvme/host 2024-03-29T07:41:19.648 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/nvme/host/nvme-core.ko 2024-03-29T07:41:19.649 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/nvme/host/nvme.ko 2024-03-29T07:41:19.649 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/parport 2024-03-29T07:41:19.649 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/parport/parport.ko 2024-03-29T07:41:19.649 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/parport/parport_cs.ko 2024-03-29T07:41:19.649 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/parport/parport_pc.ko 2024-03-29T07:41:19.649 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/parport/parport_serial.ko 2024-03-29T07:41:19.649 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci 2024-03-29T07:41:19.649 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci/hotplug 2024-03-29T07:41:19.649 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-03-29T07:41:19.649 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci/pcie 2024-03-29T07:41:19.649 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci/pcie/aer_inject.ko 2024-03-29T07:41:19.649 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci/xen-pcifront.ko 2024-03-29T07:41:19.649 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pcmcia 2024-03-29T07:41:19.650 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pcmcia/i82092.ko 2024-03-29T07:41:19.650 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pcmcia/pd6729.ko 2024-03-29T07:41:19.650 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pcmcia/yenta_socket.ko 2024-03-29T07:41:19.650 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pinctrl 2024-03-29T07:41:19.650 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pinctrl/intel 2024-03-29T07:41:19.650 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-03-29T07:41:19.650 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform 2024-03-29T07:41:19.650 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86 2024-03-29T07:41:19.650 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/acer-wmi.ko 2024-03-29T07:41:19.650 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/acerhdf.ko 2024-03-29T07:41:19.650 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/apple-gmux.ko 2024-03-29T07:41:19.650 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/asus-laptop.ko 2024-03-29T07:41:19.650 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-03-29T07:41:19.650 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/asus-wmi.ko 2024-03-29T07:41:19.651 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/classmate-laptop.ko 2024-03-29T07:41:19.651 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-03-29T07:41:19.651 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-03-29T07:41:19.651 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-03-29T07:41:19.651 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-03-29T07:41:19.651 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/hdaps.ko 2024-03-29T07:41:19.651 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/intel 2024-03-29T07:41:19.651 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-03-29T07:41:19.651 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/intel_ips.ko 2024-03-29T07:41:19.651 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/msi-wmi.ko 2024-03-29T07:41:19.651 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/mxm-wmi.ko 2024-03-29T07:41:19.651 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-03-29T07:41:19.651 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/samsung-laptop.ko 2024-03-29T07:41:19.651 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/samsung-q10.ko 2024-03-29T07:41:19.651 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/topstar-laptop.ko 2024-03-29T07:41:19.652 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-03-29T07:41:19.652 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/toshiba_haps.ko 2024-03-29T07:41:19.652 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/wmi-bmof.ko 2024-03-29T07:41:19.652 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/wmi.ko 2024-03-29T07:41:19.652 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/powercap 2024-03-29T07:41:19.652 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/powercap/intel_rapl_common.ko 2024-03-29T07:41:19.652 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/powercap/intel_rapl_msr.ko 2024-03-29T07:41:19.652 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps 2024-03-29T07:41:19.652 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps/clients 2024-03-29T07:41:19.652 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps/clients/pps-gpio.ko 2024-03-29T07:41:19.653 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps/clients/pps-ldisc.ko 2024-03-29T07:41:19.653 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps/clients/pps_parport.ko 2024-03-29T07:41:19.653 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps/pps_core.ko 2024-03-29T07:41:19.653 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ptp 2024-03-29T07:41:19.653 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ptp/ptp.ko 2024-03-29T07:41:19.653 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ptp/ptp_kvm.ko 2024-03-29T07:41:19.653 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc 2024-03-29T07:41:19.653 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-bq32k.ko 2024-03-29T07:41:19.653 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1286.ko 2024-03-29T07:41:19.653 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1307.ko 2024-03-29T07:41:19.653 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1374.ko 2024-03-29T07:41:19.653 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1511.ko 2024-03-29T07:41:19.653 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1553.ko 2024-03-29T07:41:19.653 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1672.ko 2024-03-29T07:41:19.653 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1742.ko 2024-03-29T07:41:19.654 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds2404.ko 2024-03-29T07:41:19.654 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds3232.ko 2024-03-29T07:41:19.654 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-em3027.ko 2024-03-29T07:41:19.654 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-fm3130.ko 2024-03-29T07:41:19.654 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-isl12022.ko 2024-03-29T07:41:19.654 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-isl1208.ko 2024-03-29T07:41:19.654 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-m41t80.ko 2024-03-29T07:41:19.654 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-m48t35.ko 2024-03-29T07:41:19.654 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-m48t59.ko 2024-03-29T07:41:19.654 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-max6900.ko 2024-03-29T07:41:19.654 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-msm6242.ko 2024-03-29T07:41:19.655 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-pcf2127.ko 2024-03-29T07:41:19.655 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-pcf85063.ko 2024-03-29T07:41:19.655 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-pcf8523.ko 2024-03-29T07:41:19.655 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-pcf8563.ko 2024-03-29T07:41:19.655 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-pcf8583.ko 2024-03-29T07:41:19.655 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-rp5c01.ko 2024-03-29T07:41:19.655 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-rs5c372.ko 2024-03-29T07:41:19.655 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-03-29T07:41:19.655 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-rx8025.ko 2024-03-29T07:41:19.655 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-rx8581.ko 2024-03-29T07:41:19.656 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-03-29T07:41:19.656 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-x1205.ko 2024-03-29T07:41:19.656 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi 2024-03-29T07:41:19.656 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/3w-9xxx.ko 2024-03-29T07:41:19.656 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/3w-sas.ko 2024-03-29T07:41:19.656 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/3w-xxxx.ko 2024-03-29T07:41:19.656 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/BusLogic.ko 2024-03-29T07:41:19.656 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/a100u2w.ko 2024-03-29T07:41:19.657 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/aacraid 2024-03-29T07:41:19.657 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/aacraid/aacraid.ko 2024-03-29T07:41:19.657 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/advansys.ko 2024-03-29T07:41:19.657 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/aic7xxx 2024-03-29T07:41:19.657 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-03-29T07:41:19.657 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-03-29T07:41:19.657 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/am53c974.ko 2024-03-29T07:41:19.657 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/arcmsr 2024-03-29T07:41:19.657 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-03-29T07:41:19.657 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/atp870u.ko 2024-03-29T07:41:19.658 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/be2iscsi 2024-03-29T07:41:19.658 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-03-29T07:41:19.658 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bfa 2024-03-29T07:41:19.658 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bfa/bfa.ko 2024-03-29T07:41:19.658 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bnx2fc 2024-03-29T07:41:19.658 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-03-29T07:41:19.658 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bnx2i 2024-03-29T07:41:19.658 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-03-29T07:41:19.658 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/ch.ko 2024-03-29T07:41:19.658 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/csiostor 2024-03-29T07:41:19.658 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/csiostor/csiostor.ko 2024-03-29T07:41:19.658 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi 2024-03-29T07:41:19.658 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi/cxgb3i 2024-03-29T07:41:19.658 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-03-29T07:41:19.659 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi/cxgb4i 2024-03-29T07:41:19.659 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-03-29T07:41:19.659 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-03-29T07:41:19.659 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/dc395x.ko 2024-03-29T07:41:19.659 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/device_handler 2024-03-29T07:41:19.659 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-03-29T07:41:19.659 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-03-29T07:41:19.659 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-03-29T07:41:19.659 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-03-29T07:41:19.659 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/dmx3191d.ko 2024-03-29T07:41:19.659 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/esas2r 2024-03-29T07:41:19.659 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/esas2r/esas2r.ko 2024-03-29T07:41:19.660 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/esp_scsi.ko 2024-03-29T07:41:19.660 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/fcoe 2024-03-29T07:41:19.660 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/fcoe/fcoe.ko 2024-03-29T07:41:19.660 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-03-29T07:41:19.660 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/fnic 2024-03-29T07:41:19.660 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/fnic/fnic.ko 2024-03-29T07:41:19.660 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/hpsa.ko 2024-03-29T07:41:19.660 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/hptiop.ko 2024-03-29T07:41:19.660 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/hv_storvsc.ko 2024-03-29T07:41:19.660 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/initio.ko 2024-03-29T07:41:19.660 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/ipr.ko 2024-03-29T07:41:19.660 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/ips.ko 2024-03-29T07:41:19.660 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/isci 2024-03-29T07:41:19.660 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/isci/isci.ko 2024-03-29T07:41:19.661 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-03-29T07:41:19.661 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/iscsi_tcp.ko 2024-03-29T07:41:19.661 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libfc 2024-03-29T07:41:19.661 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libfc/libfc.ko 2024-03-29T07:41:19.661 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libiscsi.ko 2024-03-29T07:41:19.661 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libiscsi_tcp.ko 2024-03-29T07:41:19.661 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libsas 2024-03-29T07:41:19.661 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libsas/libsas.ko 2024-03-29T07:41:19.661 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/lpfc 2024-03-29T07:41:19.661 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/lpfc/lpfc.ko 2024-03-29T07:41:19.661 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/megaraid 2024-03-29T07:41:19.661 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/megaraid.ko 2024-03-29T07:41:19.661 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-03-29T07:41:19.661 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-03-29T07:41:19.661 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-03-29T07:41:19.662 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/mpt3sas 2024-03-29T07:41:19.662 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-03-29T07:41:19.662 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/mvsas 2024-03-29T07:41:19.662 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/mvsas/mvsas.ko 2024-03-29T07:41:19.662 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/mvumi.ko 2024-03-29T07:41:19.662 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/pm8001 2024-03-29T07:41:19.662 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-03-29T07:41:19.662 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/pmcraid.ko 2024-03-29T07:41:19.662 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla1280.ko 2024-03-29T07:41:19.662 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla2xxx 2024-03-29T07:41:19.663 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-03-29T07:41:19.663 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-03-29T07:41:19.663 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla4xxx 2024-03-29T07:41:19.663 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-03-29T07:41:19.663 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/raid_class.ko 2024-03-29T07:41:19.663 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_debug.ko 2024-03-29T07:41:19.663 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_transport_fc.ko 2024-03-29T07:41:19.663 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-03-29T07:41:19.663 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_transport_sas.ko 2024-03-29T07:41:19.664 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_transport_spi.ko 2024-03-29T07:41:19.664 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_transport_srp.ko 2024-03-29T07:41:19.664 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/ses.ko 2024-03-29T07:41:19.664 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/st.ko 2024-03-29T07:41:19.664 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/stex.ko 2024-03-29T07:41:19.664 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/sym53c8xx_2 2024-03-29T07:41:19.664 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-03-29T07:41:19.664 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/virtio_scsi.ko 2024-03-29T07:41:19.664 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/vmw_pvscsi.ko 2024-03-29T07:41:19.664 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/wd719x.ko 2024-03-29T07:41:19.664 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ssb 2024-03-29T07:41:19.664 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ssb/ssb.ko 2024-03-29T07:41:19.664 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target 2024-03-29T07:41:19.664 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/iscsi 2024-03-29T07:41:19.665 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-03-29T07:41:19.665 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/loopback 2024-03-29T07:41:19.665 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/loopback/tcm_loop.ko 2024-03-29T07:41:19.665 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/target_core_file.ko 2024-03-29T07:41:19.665 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/target_core_iblock.ko 2024-03-29T07:41:19.665 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/target_core_mod.ko 2024-03-29T07:41:19.665 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/target_core_pscsi.ko 2024-03-29T07:41:19.665 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/target_core_user.ko 2024-03-29T07:41:19.665 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/tcm_fc 2024-03-29T07:41:19.665 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-03-29T07:41:19.665 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal 2024-03-29T07:41:19.665 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel 2024-03-29T07:41:19.665 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal 2024-03-29T07:41:19.665 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-03-29T07:41:19.665 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-03-29T07:41:19.666 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-03-29T07:41:19.666 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-03-29T07:41:19.666 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-03-29T07:41:19.666 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-03-29T07:41:19.666 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-03-29T07:41:19.666 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-03-29T07:41:19.666 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-03-29T07:41:19.666 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-03-29T07:41:19.666 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-03-29T07:41:19.666 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-03-29T07:41:19.666 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-03-29T07:41:19.666 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-03-29T07:41:19.666 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-03-29T07:41:19.667 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-03-29T07:41:19.667 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-03-29T07:41:19.667 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-03-29T07:41:19.667 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty 2024-03-29T07:41:19.667 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/ipwireless 2024-03-29T07:41:19.667 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-03-29T07:41:19.667 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/n_gsm.ko 2024-03-29T07:41:19.667 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/n_hdlc.ko 2024-03-29T07:41:19.667 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/nozomi.ko 2024-03-29T07:41:19.667 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial 2024-03-29T07:41:19.667 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/8250 2024-03-29T07:41:19.667 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-03-29T07:41:19.667 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/arc_uart.ko 2024-03-29T07:41:19.667 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/jsm 2024-03-29T07:41:19.668 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/jsm/jsm.ko 2024-03-29T07:41:19.668 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ufs 2024-03-29T07:41:19.668 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ufs/core 2024-03-29T07:41:19.668 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ufs/core/ufshcd-core.ko 2024-03-29T07:41:19.668 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ufs/host 2024-03-29T07:41:19.668 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-03-29T07:41:19.668 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio 2024-03-29T07:41:19.668 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio.ko 2024-03-29T07:41:19.668 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio_aec.ko 2024-03-29T07:41:19.668 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio_cif.ko 2024-03-29T07:41:19.668 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio_pci_generic.ko 2024-03-29T07:41:19.668 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio_sercos3.ko 2024-03-29T07:41:19.668 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb 2024-03-29T07:41:19.668 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/class 2024-03-29T07:41:19.668 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/class/cdc-acm.ko 2024-03-29T07:41:19.669 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/class/cdc-wdm.ko 2024-03-29T07:41:19.669 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/class/usblp.ko 2024-03-29T07:41:19.669 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/class/usbtmc.ko 2024-03-29T07:41:19.669 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/host 2024-03-29T07:41:19.669 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/host/sl811-hcd.ko 2024-03-29T07:41:19.669 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/image 2024-03-29T07:41:19.669 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/image/mdc800.ko 2024-03-29T07:41:19.669 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/image/microtek.ko 2024-03-29T07:41:19.669 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc 2024-03-29T07:41:19.669 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/adutux.ko 2024-03-29T07:41:19.669 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/appledisplay.ko 2024-03-29T07:41:19.669 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/emi26.ko 2024-03-29T07:41:19.669 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/emi62.ko 2024-03-29T07:41:19.670 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/ezusb.ko 2024-03-29T07:41:19.670 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/idmouse.ko 2024-03-29T07:41:19.670 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/iowarrior.ko 2024-03-29T07:41:19.670 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/isight_firmware.ko 2024-03-29T07:41:19.670 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/ldusb.ko 2024-03-29T07:41:19.670 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/legousbtower.ko 2024-03-29T07:41:19.671 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/sisusbvga 2024-03-29T07:41:19.671 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-03-29T07:41:19.671 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/trancevibrator.ko 2024-03-29T07:41:19.671 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/usb3503.ko 2024-03-29T07:41:19.671 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/usblcd.ko 2024-03-29T07:41:19.671 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/usbsevseg.ko 2024-03-29T07:41:19.671 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/uss720.ko 2024-03-29T07:41:19.672 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/yurex.ko 2024-03-29T07:41:19.672 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/phy 2024-03-29T07:41:19.672 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/phy/phy-generic.ko 2024-03-29T07:41:19.672 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial 2024-03-29T07:41:19.672 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/aircable.ko 2024-03-29T07:41:19.672 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ark3116.ko 2024-03-29T07:41:19.672 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/belkin_sa.ko 2024-03-29T07:41:19.673 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ch341.ko 2024-03-29T07:41:19.673 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/cp210x.ko 2024-03-29T07:41:19.673 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/cyberjack.ko 2024-03-29T07:41:19.673 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/cypress_m8.ko 2024-03-29T07:41:19.673 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/digi_acceleport.ko 2024-03-29T07:41:19.673 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/empeg.ko 2024-03-29T07:41:19.674 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ftdi_sio.ko 2024-03-29T07:41:19.674 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/garmin_gps.ko 2024-03-29T07:41:19.674 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/io_edgeport.ko 2024-03-29T07:41:19.674 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/io_ti.ko 2024-03-29T07:41:19.674 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ipaq.ko 2024-03-29T07:41:19.674 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ipw.ko 2024-03-29T07:41:19.674 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ir-usb.ko 2024-03-29T07:41:19.675 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-03-29T07:41:19.675 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/keyspan.ko 2024-03-29T07:41:19.675 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/keyspan_pda.ko 2024-03-29T07:41:19.675 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/kl5kusb105.ko 2024-03-29T07:41:19.675 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/kobil_sct.ko 2024-03-29T07:41:19.675 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/mct_u232.ko 2024-03-29T07:41:19.676 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/mos7720.ko 2024-03-29T07:41:19.676 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/mos7840.ko 2024-03-29T07:41:19.676 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/navman.ko 2024-03-29T07:41:19.676 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/omninet.ko 2024-03-29T07:41:19.676 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/opticon.ko 2024-03-29T07:41:19.676 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/option.ko 2024-03-29T07:41:19.677 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/oti6858.ko 2024-03-29T07:41:19.677 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/pl2303.ko 2024-03-29T07:41:19.677 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/qcaux.ko 2024-03-29T07:41:19.677 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/qcserial.ko 2024-03-29T07:41:19.677 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/quatech2.ko 2024-03-29T07:41:19.677 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/safe_serial.ko 2024-03-29T07:41:19.677 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/sierra.ko 2024-03-29T07:41:19.678 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/spcp8x5.ko 2024-03-29T07:41:19.678 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ssu100.ko 2024-03-29T07:41:19.678 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/symbolserial.ko 2024-03-29T07:41:19.678 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-03-29T07:41:19.678 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-03-29T07:41:19.678 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/usb_debug.ko 2024-03-29T07:41:19.679 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/usb_wwan.ko 2024-03-29T07:41:19.679 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/visor.ko 2024-03-29T07:41:19.679 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/whiteheat.ko 2024-03-29T07:41:19.679 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/xsens_mt.ko 2024-03-29T07:41:19.679 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage 2024-03-29T07:41:19.679 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/uas.ko 2024-03-29T07:41:19.679 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-alauda.ko 2024-03-29T07:41:19.680 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-cypress.ko 2024-03-29T07:41:19.680 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-datafab.ko 2024-03-29T07:41:19.680 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-03-29T07:41:19.680 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-freecom.ko 2024-03-29T07:41:19.680 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-isd200.ko 2024-03-29T07:41:19.680 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-03-29T07:41:19.681 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-karma.ko 2024-03-29T07:41:19.681 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-onetouch.ko 2024-03-29T07:41:19.681 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-realtek.ko 2024-03-29T07:41:19.681 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-sddr09.ko 2024-03-29T07:41:19.681 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-sddr55.ko 2024-03-29T07:41:19.681 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-usbat.ko 2024-03-29T07:41:19.682 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/usb-storage.ko 2024-03-29T07:41:19.682 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/usbip 2024-03-29T07:41:19.682 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/usbip/usbip-core.ko 2024-03-29T07:41:19.682 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/usbip/usbip-host.ko 2024-03-29T07:41:19.682 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-03-29T07:41:19.682 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio 2024-03-29T07:41:19.682 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio/pci 2024-03-29T07:41:19.683 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-03-29T07:41:19.683 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio/pci/vfio-pci.ko 2024-03-29T07:41:19.683 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio/vfio.ko 2024-03-29T07:41:19.683 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-03-29T07:41:19.683 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vhost 2024-03-29T07:41:19.683 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vhost/vhost.ko 2024-03-29T07:41:19.684 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vhost/vhost_iotlb.ko 2024-03-29T07:41:19.684 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vhost/vhost_net.ko 2024-03-29T07:41:19.684 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vhost/vhost_scsi.ko 2024-03-29T07:41:19.684 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video 2024-03-29T07:41:19.684 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight 2024-03-29T07:41:19.684 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight/apple_bl.ko 2024-03-29T07:41:19.685 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight/lcd.ko 2024-03-29T07:41:19.685 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight/platform_lcd.ko 2024-03-29T07:41:19.685 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio 2024-03-29T07:41:19.685 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio.ko 2024-03-29T07:41:19.685 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_balloon.ko 2024-03-29T07:41:19.685 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_mmio.ko 2024-03-29T07:41:19.686 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_pci.ko 2024-03-29T07:41:19.686 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-03-29T07:41:19.686 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-03-29T07:41:19.686 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_ring.ko 2024-03-29T07:41:19.686 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1 2024-03-29T07:41:19.686 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/masters 2024-03-29T07:41:19.687 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/masters/ds2482.ko 2024-03-29T07:41:19.687 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/masters/ds2490.ko 2024-03-29T07:41:19.687 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves 2024-03-29T07:41:19.687 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-03-29T07:41:19.687 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-03-29T07:41:19.687 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-03-29T07:41:19.688 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-03-29T07:41:19.688 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-03-29T07:41:19.688 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-03-29T07:41:19.688 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-03-29T07:41:19.688 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-03-29T07:41:19.688 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-03-29T07:41:19.688 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_smem.ko 2024-03-29T07:41:19.689 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_therm.ko 2024-03-29T07:41:19.689 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/wire.ko 2024-03-29T07:41:19.689 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog 2024-03-29T07:41:19.689 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/alim1535_wdt.ko 2024-03-29T07:41:19.689 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/alim7101_wdt.ko 2024-03-29T07:41:19.689 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/f71808e_wdt.ko 2024-03-29T07:41:19.689 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/hpwdt.ko 2024-03-29T07:41:19.690 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/i6300esb.ko 2024-03-29T07:41:19.690 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-03-29T07:41:19.690 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/iTCO_wdt.ko 2024-03-29T07:41:19.690 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/ib700wdt.ko 2024-03-29T07:41:19.690 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/ibmasr.ko 2024-03-29T07:41:19.691 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-03-29T07:41:19.691 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/it8712f_wdt.ko 2024-03-29T07:41:19.691 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/it87_wdt.ko 2024-03-29T07:41:19.691 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/machzwd.ko 2024-03-29T07:41:19.691 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/nv_tco.ko 2024-03-29T07:41:19.691 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/pcwd_pci.ko 2024-03-29T07:41:19.691 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/pcwd_usb.ko 2024-03-29T07:41:19.692 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-03-29T07:41:19.692 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/sch311x_wdt.ko 2024-03-29T07:41:19.692 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/softdog.ko 2024-03-29T07:41:19.692 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/sp5100_tco.ko 2024-03-29T07:41:19.692 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/via_wdt.ko 2024-03-29T07:41:19.692 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-03-29T07:41:19.693 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/w83877f_wdt.ko 2024-03-29T07:41:19.693 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/w83977f_wdt.ko 2024-03-29T07:41:19.693 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/wdt_pci.ko 2024-03-29T07:41:19.693 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/xen_wdt.ko 2024-03-29T07:41:19.693 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen 2024-03-29T07:41:19.693 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-acpi-processor.ko 2024-03-29T07:41:19.694 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-evtchn.ko 2024-03-29T07:41:19.694 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-gntalloc.ko 2024-03-29T07:41:19.694 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-gntdev.ko 2024-03-29T07:41:19.694 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-pciback 2024-03-29T07:41:19.694 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-03-29T07:41:19.694 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-privcmd.ko 2024-03-29T07:41:19.694 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xenfs 2024-03-29T07:41:19.694 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xenfs/xenfs.ko 2024-03-29T07:41:19.694 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs 2024-03-29T07:41:19.695 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/9p 2024-03-29T07:41:19.695 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/9p/9p.ko 2024-03-29T07:41:19.695 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/affs 2024-03-29T07:41:19.695 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/affs/affs.ko 2024-03-29T07:41:19.695 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/befs 2024-03-29T07:41:19.695 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/befs/befs.ko 2024-03-29T07:41:19.695 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/binfmt_misc.ko 2024-03-29T07:41:19.695 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/btrfs 2024-03-29T07:41:19.696 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/btrfs/btrfs.ko 2024-03-29T07:41:19.696 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/cachefiles 2024-03-29T07:41:19.696 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/cachefiles/cachefiles.ko 2024-03-29T07:41:19.696 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ceph 2024-03-29T07:41:19.696 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ceph/ceph.ko 2024-03-29T07:41:19.696 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/coda 2024-03-29T07:41:19.696 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/coda/coda.ko 2024-03-29T07:41:19.696 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/cramfs 2024-03-29T07:41:19.696 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/cramfs/cramfs.ko 2024-03-29T07:41:19.697 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/dlm 2024-03-29T07:41:19.697 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/dlm/dlm.ko 2024-03-29T07:41:19.697 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ecryptfs 2024-03-29T07:41:19.697 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ecryptfs/ecryptfs.ko 2024-03-29T07:41:19.697 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/f2fs 2024-03-29T07:41:19.697 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/f2fs/f2fs.ko 2024-03-29T07:41:19.697 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fat 2024-03-29T07:41:19.697 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fat/fat.ko 2024-03-29T07:41:19.697 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fat/msdos.ko 2024-03-29T07:41:19.698 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fat/vfat.ko 2024-03-29T07:41:19.698 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fuse 2024-03-29T07:41:19.698 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fuse/cuse.ko 2024-03-29T07:41:19.698 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fuse/fuse.ko 2024-03-29T07:41:19.698 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/gfs2 2024-03-29T07:41:19.698 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/gfs2/gfs2.ko 2024-03-29T07:41:19.698 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/hfs 2024-03-29T07:41:19.698 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/hfs/hfs.ko 2024-03-29T07:41:19.699 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/hfsplus 2024-03-29T07:41:19.699 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/hfsplus/hfsplus.ko 2024-03-29T07:41:19.699 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/isofs 2024-03-29T07:41:19.699 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/isofs/isofs.ko 2024-03-29T07:41:19.699 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/jfs 2024-03-29T07:41:19.699 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/jfs/jfs.ko 2024-03-29T07:41:19.699 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/lockd 2024-03-29T07:41:19.699 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/lockd/lockd.ko 2024-03-29T07:41:19.700 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/minix 2024-03-29T07:41:19.700 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/minix/minix.ko 2024-03-29T07:41:19.700 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/netfs 2024-03-29T07:41:19.700 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/netfs/netfs.ko 2024-03-29T07:41:19.700 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs 2024-03-29T07:41:19.700 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/blocklayout 2024-03-29T07:41:19.700 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-03-29T07:41:19.700 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/filelayout 2024-03-29T07:41:19.700 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-03-29T07:41:19.700 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/flexfilelayout 2024-03-29T07:41:19.700 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-03-29T07:41:19.701 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/nfs.ko 2024-03-29T07:41:19.701 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/nfsv3.ko 2024-03-29T07:41:19.701 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/nfsv4.ko 2024-03-29T07:41:19.701 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs_common 2024-03-29T07:41:19.701 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs_common/grace.ko 2024-03-29T07:41:19.701 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs_common/nfs_acl.ko 2024-03-29T07:41:19.701 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfsd 2024-03-29T07:41:19.701 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfsd/nfsd.ko 2024-03-29T07:41:19.701 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nilfs2 2024-03-29T07:41:19.701 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nilfs2/nilfs2.ko 2024-03-29T07:41:19.701 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls 2024-03-29T07:41:19.701 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-celtic.ko 2024-03-29T07:41:19.701 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-centeuro.ko 2024-03-29T07:41:19.701 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-croatian.ko 2024-03-29T07:41:19.702 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-cyrillic.ko 2024-03-29T07:41:19.702 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-gaelic.ko 2024-03-29T07:41:19.702 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-greek.ko 2024-03-29T07:41:19.702 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-iceland.ko 2024-03-29T07:41:19.702 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-inuit.ko 2024-03-29T07:41:19.702 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-roman.ko 2024-03-29T07:41:19.702 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-romanian.ko 2024-03-29T07:41:19.702 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-turkish.ko 2024-03-29T07:41:19.702 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp1250.ko 2024-03-29T07:41:19.702 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp1251.ko 2024-03-29T07:41:19.702 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp1255.ko 2024-03-29T07:41:19.702 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp737.ko 2024-03-29T07:41:19.702 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp775.ko 2024-03-29T07:41:19.702 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp850.ko 2024-03-29T07:41:19.703 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp852.ko 2024-03-29T07:41:19.703 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp855.ko 2024-03-29T07:41:19.703 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp857.ko 2024-03-29T07:41:19.703 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp860.ko 2024-03-29T07:41:19.703 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp861.ko 2024-03-29T07:41:19.703 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp862.ko 2024-03-29T07:41:19.703 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp863.ko 2024-03-29T07:41:19.703 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp864.ko 2024-03-29T07:41:19.703 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp865.ko 2024-03-29T07:41:19.703 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp866.ko 2024-03-29T07:41:19.703 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp869.ko 2024-03-29T07:41:19.703 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp874.ko 2024-03-29T07:41:19.703 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp932.ko 2024-03-29T07:41:19.703 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp936.ko 2024-03-29T07:41:19.703 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp949.ko 2024-03-29T07:41:19.704 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp950.ko 2024-03-29T07:41:19.704 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_euc-jp.ko 2024-03-29T07:41:19.704 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-1.ko 2024-03-29T07:41:19.704 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-13.ko 2024-03-29T07:41:19.704 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-14.ko 2024-03-29T07:41:19.704 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-15.ko 2024-03-29T07:41:19.704 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-2.ko 2024-03-29T07:41:19.704 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-3.ko 2024-03-29T07:41:19.704 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-4.ko 2024-03-29T07:41:19.704 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-5.ko 2024-03-29T07:41:19.704 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-6.ko 2024-03-29T07:41:19.704 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-7.ko 2024-03-29T07:41:19.704 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-9.ko 2024-03-29T07:41:19.704 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_koi8-r.ko 2024-03-29T07:41:19.704 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_koi8-ru.ko 2024-03-29T07:41:19.704 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_koi8-u.ko 2024-03-29T07:41:19.705 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_ucs2_utils.ko 2024-03-29T07:41:19.705 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_utf8.ko 2024-03-29T07:41:19.705 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2 2024-03-29T07:41:19.705 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/cluster 2024-03-29T07:41:19.705 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-03-29T07:41:19.705 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/dlm 2024-03-29T07:41:19.705 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-03-29T07:41:19.705 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/dlmfs 2024-03-29T07:41:19.705 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-03-29T07:41:19.705 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/ocfs2.ko 2024-03-29T07:41:19.705 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-03-29T07:41:19.705 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-03-29T07:41:19.705 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-03-29T07:41:19.705 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/overlayfs 2024-03-29T07:41:19.706 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/overlayfs/overlay.ko 2024-03-29T07:41:19.706 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/pstore 2024-03-29T07:41:19.706 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/pstore/ramoops.ko 2024-03-29T07:41:19.706 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/reiserfs 2024-03-29T07:41:19.706 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/reiserfs/reiserfs.ko 2024-03-29T07:41:19.706 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/romfs 2024-03-29T07:41:19.706 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/romfs/romfs.ko 2024-03-29T07:41:19.706 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb 2024-03-29T07:41:19.706 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb/client 2024-03-29T07:41:19.706 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb/client/cifs.ko 2024-03-29T07:41:19.706 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb/common 2024-03-29T07:41:19.706 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb/common/cifs_arc4.ko 2024-03-29T07:41:19.706 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb/common/cifs_md4.ko 2024-03-29T07:41:19.706 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/squashfs 2024-03-29T07:41:19.707 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/squashfs/squashfs.ko 2024-03-29T07:41:19.707 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/sysv 2024-03-29T07:41:19.707 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/sysv/sysv.ko 2024-03-29T07:41:19.707 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/udf 2024-03-29T07:41:19.707 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/udf/udf.ko 2024-03-29T07:41:19.707 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ufs 2024-03-29T07:41:19.707 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ufs/ufs.ko 2024-03-29T07:41:19.707 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/xfs 2024-03-29T07:41:19.707 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/xfs/xfs.ko 2024-03-29T07:41:19.707 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel 2024-03-29T07:41:19.707 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel/rcu 2024-03-29T07:41:19.707 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel/rcu/rcutorture.ko 2024-03-29T07:41:19.708 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel/torture.ko 2024-03-29T07:41:19.708 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel/trace 2024-03-29T07:41:19.708 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-03-29T07:41:19.708 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib 2024-03-29T07:41:19.708 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/asn1_encoder.ko 2024-03-29T07:41:19.708 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/crc-itu-t.ko 2024-03-29T07:41:19.708 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/crc8.ko 2024-03-29T07:41:19.708 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/crypto 2024-03-29T07:41:19.708 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/crypto/libarc4.ko 2024-03-29T07:41:19.708 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/crypto/libdes.ko 2024-03-29T07:41:19.708 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/libcrc32c.ko 2024-03-29T07:41:19.708 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/lru_cache.ko 2024-03-29T07:41:19.708 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/lz4 2024-03-29T07:41:19.708 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/lz4/lz4_compress.ko 2024-03-29T07:41:19.709 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/lz4/lz4hc_compress.ko 2024-03-29T07:41:19.709 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/math 2024-03-29T07:41:19.709 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/math/cordic.ko 2024-03-29T07:41:19.709 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/raid6 2024-03-29T07:41:19.709 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/raid6/raid6_pq.ko 2024-03-29T07:41:19.709 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/reed_solomon 2024-03-29T07:41:19.709 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/reed_solomon/reed_solomon.ko 2024-03-29T07:41:19.709 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/ts_bm.ko 2024-03-29T07:41:19.709 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/ts_fsm.ko 2024-03-29T07:41:19.709 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/ts_kmp.ko 2024-03-29T07:41:19.709 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/zstd 2024-03-29T07:41:19.709 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/zstd/zstd_compress.ko 2024-03-29T07:41:19.709 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/mm 2024-03-29T07:41:19.709 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/mm/hwpoison-inject.ko 2024-03-29T07:41:19.709 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net 2024-03-29T07:41:19.710 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan 2024-03-29T07:41:19.710 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/6lowpan.ko 2024-03-29T07:41:19.710 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_dest.ko 2024-03-29T07:41:19.710 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_fragment.ko 2024-03-29T07:41:19.710 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_hop.ko 2024-03-29T07:41:19.710 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_ipv6.ko 2024-03-29T07:41:19.710 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_mobility.ko 2024-03-29T07:41:19.710 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_routing.ko 2024-03-29T07:41:19.710 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_udp.ko 2024-03-29T07:41:19.710 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802 2024-03-29T07:41:19.710 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802/garp.ko 2024-03-29T07:41:19.710 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802/mrp.ko 2024-03-29T07:41:19.710 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802/p8022.ko 2024-03-29T07:41:19.711 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802/psnap.ko 2024-03-29T07:41:19.711 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802/stp.ko 2024-03-29T07:41:19.711 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/8021q 2024-03-29T07:41:19.711 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/8021q/8021q.ko 2024-03-29T07:41:19.711 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/9p 2024-03-29T07:41:19.711 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/9p/9pnet.ko 2024-03-29T07:41:19.711 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/9p/9pnet_fd.ko 2024-03-29T07:41:19.711 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/9p/9pnet_rdma.ko 2024-03-29T07:41:19.711 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/9p/9pnet_virtio.ko 2024-03-29T07:41:19.711 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/batman-adv 2024-03-29T07:41:19.711 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/batman-adv/batman-adv.ko 2024-03-29T07:41:19.711 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge 2024-03-29T07:41:19.711 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/br_netfilter.ko 2024-03-29T07:41:19.711 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/bridge.ko 2024-03-29T07:41:19.711 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter 2024-03-29T07:41:19.712 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-03-29T07:41:19.712 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_among.ko 2024-03-29T07:41:19.712 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_arp.ko 2024-03-29T07:41:19.712 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-03-29T07:41:19.712 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-03-29T07:41:19.712 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_ip.ko 2024-03-29T07:41:19.712 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-03-29T07:41:19.712 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_limit.ko 2024-03-29T07:41:19.712 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_log.ko 2024-03-29T07:41:19.712 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_mark.ko 2024-03-29T07:41:19.712 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-03-29T07:41:19.712 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-03-29T07:41:19.712 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-03-29T07:41:19.712 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-03-29T07:41:19.713 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_snat.ko 2024-03-29T07:41:19.713 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_stp.ko 2024-03-29T07:41:19.713 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-03-29T07:41:19.713 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-03-29T07:41:19.713 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-03-29T07:41:19.713 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-03-29T07:41:19.713 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebtables.ko 2024-03-29T07:41:19.713 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-03-29T07:41:19.713 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-03-29T07:41:19.713 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-03-29T07:41:19.713 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ceph 2024-03-29T07:41:19.713 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ceph/libceph.ko 2024-03-29T07:41:19.714 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/core 2024-03-29T07:41:19.714 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/core/failover.ko 2024-03-29T07:41:19.714 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/core/pktgen.ko 2024-03-29T07:41:19.714 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dccp 2024-03-29T07:41:19.714 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dccp/dccp.ko 2024-03-29T07:41:19.714 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dccp/dccp_diag.ko 2024-03-29T07:41:19.714 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dccp/dccp_ipv4.ko 2024-03-29T07:41:19.714 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dccp/dccp_ipv6.ko 2024-03-29T07:41:19.714 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dns_resolver 2024-03-29T07:41:19.714 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dns_resolver/dns_resolver.ko 2024-03-29T07:41:19.714 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa 2024-03-29T07:41:19.714 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa/dsa_core.ko 2024-03-29T07:41:19.715 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa/tag_brcm.ko 2024-03-29T07:41:19.715 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa/tag_dsa.ko 2024-03-29T07:41:19.715 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa/tag_none.ko 2024-03-29T07:41:19.715 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa/tag_trailer.ko 2024-03-29T07:41:19.715 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ieee802154 2024-03-29T07:41:19.715 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ieee802154/6lowpan 2024-03-29T07:41:19.715 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-03-29T07:41:19.715 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ieee802154/ieee802154.ko 2024-03-29T07:41:19.715 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ieee802154/ieee802154_socket.ko 2024-03-29T07:41:19.716 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4 2024-03-29T07:41:19.716 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ah4.ko 2024-03-29T07:41:19.716 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/esp4.ko 2024-03-29T07:41:19.716 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/fou.ko 2024-03-29T07:41:19.716 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/gre.ko 2024-03-29T07:41:19.716 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/inet_diag.ko 2024-03-29T07:41:19.716 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ip_gre.ko 2024-03-29T07:41:19.716 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ip_tunnel.ko 2024-03-29T07:41:19.716 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ip_vti.ko 2024-03-29T07:41:19.717 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ipcomp.ko 2024-03-29T07:41:19.717 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ipip.ko 2024-03-29T07:41:19.717 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter 2024-03-29T07:41:19.717 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/arp_tables.ko 2024-03-29T07:41:19.717 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-03-29T07:41:19.717 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-03-29T07:41:19.717 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-03-29T07:41:19.717 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-03-29T07:41:19.717 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-03-29T07:41:19.718 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-03-29T07:41:19.718 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-03-29T07:41:19.718 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-03-29T07:41:19.718 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-03-29T07:41:19.718 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/iptable_security.ko 2024-03-29T07:41:19.718 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-03-29T07:41:19.718 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-03-29T07:41:19.718 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-03-29T07:41:19.719 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-03-29T07:41:19.719 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-03-29T07:41:19.719 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-03-29T07:41:19.719 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-03-29T07:41:19.719 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-03-29T07:41:19.719 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-03-29T07:41:19.719 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-03-29T07:41:19.719 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_bic.ko 2024-03-29T07:41:19.720 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_dctcp.ko 2024-03-29T07:41:19.720 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_diag.ko 2024-03-29T07:41:19.720 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_highspeed.ko 2024-03-29T07:41:19.720 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_htcp.ko 2024-03-29T07:41:19.720 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_hybla.ko 2024-03-29T07:41:19.720 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_illinois.ko 2024-03-29T07:41:19.720 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_lp.ko 2024-03-29T07:41:19.720 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_scalable.ko 2024-03-29T07:41:19.720 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_vegas.ko 2024-03-29T07:41:19.720 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_veno.ko 2024-03-29T07:41:19.720 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_westwood.ko 2024-03-29T07:41:19.720 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_yeah.ko 2024-03-29T07:41:19.720 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tunnel4.ko 2024-03-29T07:41:19.721 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/udp_diag.ko 2024-03-29T07:41:19.721 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/udp_tunnel.ko 2024-03-29T07:41:19.721 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/xfrm4_tunnel.ko 2024-03-29T07:41:19.721 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6 2024-03-29T07:41:19.721 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/ah6.ko 2024-03-29T07:41:19.721 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/esp6.ko 2024-03-29T07:41:19.721 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/fou6.ko 2024-03-29T07:41:19.721 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/ip6_tunnel.ko 2024-03-29T07:41:19.721 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-03-29T07:41:19.721 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/ip6_vti.ko 2024-03-29T07:41:19.721 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/ipcomp6.ko 2024-03-29T07:41:19.721 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter 2024-03-29T07:41:19.721 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-03-29T07:41:19.721 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-03-29T07:41:19.721 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-03-29T07:41:19.722 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-03-29T07:41:19.722 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-03-29T07:41:19.722 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-03-29T07:41:19.722 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-03-29T07:41:19.722 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-03-29T07:41:19.722 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-03-29T07:41:19.723 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-03-29T07:41:19.723 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-03-29T07:41:19.723 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-03-29T07:41:19.723 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-03-29T07:41:19.723 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-03-29T07:41:19.723 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-03-29T07:41:19.723 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-03-29T07:41:19.723 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-03-29T07:41:19.723 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-03-29T07:41:19.723 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-03-29T07:41:19.723 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-03-29T07:41:19.723 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-03-29T07:41:19.723 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-03-29T07:41:19.723 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-03-29T07:41:19.723 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-03-29T07:41:19.724 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/sit.ko 2024-03-29T07:41:19.724 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/tunnel6.ko 2024-03-29T07:41:19.724 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/xfrm6_tunnel.ko 2024-03-29T07:41:19.724 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/key 2024-03-29T07:41:19.724 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/key/af_key.ko 2024-03-29T07:41:19.724 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp 2024-03-29T07:41:19.724 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_core.ko 2024-03-29T07:41:19.724 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_debugfs.ko 2024-03-29T07:41:19.724 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_eth.ko 2024-03-29T07:41:19.724 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_ip.ko 2024-03-29T07:41:19.724 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_ip6.ko 2024-03-29T07:41:19.724 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_netlink.ko 2024-03-29T07:41:19.724 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_ppp.ko 2024-03-29T07:41:19.724 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/llc 2024-03-29T07:41:19.725 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/llc/llc.ko 2024-03-29T07:41:19.725 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/mac802154 2024-03-29T07:41:19.725 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/mac802154/mac802154.ko 2024-03-29T07:41:19.725 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/mpls 2024-03-29T07:41:19.725 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/mpls/mpls_gso.ko 2024-03-29T07:41:19.725 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter 2024-03-29T07:41:19.725 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset 2024-03-29T07:41:19.725 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set.ko 2024-03-29T07:41:19.725 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-03-29T07:41:19.725 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-03-29T07:41:19.725 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-03-29T07:41:19.725 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-03-29T07:41:19.725 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-03-29T07:41:19.725 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-03-29T07:41:19.725 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-03-29T07:41:19.725 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-03-29T07:41:19.726 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-03-29T07:41:19.726 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-03-29T07:41:19.726 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-03-29T07:41:19.726 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-03-29T07:41:19.726 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-03-29T07:41:19.726 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-03-29T07:41:19.726 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-03-29T07:41:19.726 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs 2024-03-29T07:41:19.726 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs.ko 2024-03-29T07:41:19.726 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-03-29T07:41:19.726 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-03-29T07:41:19.726 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-03-29T07:41:19.726 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-03-29T07:41:19.726 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-03-29T07:41:19.727 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-03-29T07:41:19.727 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-03-29T07:41:19.727 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-03-29T07:41:19.727 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-03-29T07:41:19.727 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-03-29T07:41:19.727 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-03-29T07:41:19.727 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-03-29T07:41:19.727 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-03-29T07:41:19.727 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conncount.ko 2024-03-29T07:41:19.727 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack.ko 2024-03-29T07:41:19.727 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-03-29T07:41:19.727 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-03-29T07:41:19.727 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-03-29T07:41:19.727 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_h323.ko 2024-03-29T07:41:19.727 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_irc.ko 2024-03-29T07:41:19.728 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-03-29T07:41:19.728 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-03-29T07:41:19.728 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-03-29T07:41:19.728 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_sane.ko 2024-03-29T07:41:19.728 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_sip.ko 2024-03-29T07:41:19.728 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-03-29T07:41:19.728 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-03-29T07:41:19.728 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_dup_netdev.ko 2024-03-29T07:41:19.728 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_flow_table.ko 2024-03-29T07:41:19.728 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_flow_table_inet.ko 2024-03-29T07:41:19.728 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_log_syslog.ko 2024-03-29T07:41:19.728 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat.ko 2024-03-29T07:41:19.728 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat_amanda.ko 2024-03-29T07:41:19.728 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat_ftp.ko 2024-03-29T07:41:19.728 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat_irc.ko 2024-03-29T07:41:19.729 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat_sip.ko 2024-03-29T07:41:19.729 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat_tftp.ko 2024-03-29T07:41:19.729 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_synproxy_core.ko 2024-03-29T07:41:19.729 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_tables.ko 2024-03-29T07:41:19.729 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nfnetlink.ko 2024-03-29T07:41:19.729 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nfnetlink_acct.ko 2024-03-29T07:41:19.729 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nfnetlink_log.ko 2024-03-29T07:41:19.729 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nfnetlink_osf.ko 2024-03-29T07:41:19.729 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nfnetlink_queue.ko 2024-03-29T07:41:19.729 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_chain_nat.ko 2024-03-29T07:41:19.729 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_compat.ko 2024-03-29T07:41:19.729 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_ct.ko 2024-03-29T07:41:19.729 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_dup_netdev.ko 2024-03-29T07:41:19.729 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_fib.ko 2024-03-29T07:41:19.729 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_fwd_netdev.ko 2024-03-29T07:41:19.730 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_hash.ko 2024-03-29T07:41:19.730 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_limit.ko 2024-03-29T07:41:19.730 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_log.ko 2024-03-29T07:41:19.730 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_masq.ko 2024-03-29T07:41:19.730 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_nat.ko 2024-03-29T07:41:19.730 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_numgen.ko 2024-03-29T07:41:19.730 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_queue.ko 2024-03-29T07:41:19.730 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_quota.ko 2024-03-29T07:41:19.730 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_redir.ko 2024-03-29T07:41:19.730 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_reject.ko 2024-03-29T07:41:19.730 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_reject_inet.ko 2024-03-29T07:41:19.730 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_socket.ko 2024-03-29T07:41:19.730 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_tunnel.ko 2024-03-29T07:41:19.730 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_AUDIT.ko 2024-03-29T07:41:19.730 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_CHECKSUM.ko 2024-03-29T07:41:19.731 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_CLASSIFY.ko 2024-03-29T07:41:19.731 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-03-29T07:41:19.731 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_CT.ko 2024-03-29T07:41:19.731 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_DSCP.ko 2024-03-29T07:41:19.731 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_HL.ko 2024-03-29T07:41:19.731 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_HMARK.ko 2024-03-29T07:41:19.731 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_IDLETIMER.ko 2024-03-29T07:41:19.731 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_LED.ko 2024-03-29T07:41:19.731 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_LOG.ko 2024-03-29T07:41:19.731 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_MASQUERADE.ko 2024-03-29T07:41:19.731 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_NETMAP.ko 2024-03-29T07:41:19.731 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_NFLOG.ko 2024-03-29T07:41:19.731 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_NFQUEUE.ko 2024-03-29T07:41:19.731 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_RATEEST.ko 2024-03-29T07:41:19.731 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_REDIRECT.ko 2024-03-29T07:41:19.732 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_SECMARK.ko 2024-03-29T07:41:19.732 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_TCPMSS.ko 2024-03-29T07:41:19.732 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-03-29T07:41:19.732 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_TEE.ko 2024-03-29T07:41:19.732 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_TPROXY.ko 2024-03-29T07:41:19.732 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_TRACE.ko 2024-03-29T07:41:19.732 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_addrtype.ko 2024-03-29T07:41:19.732 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_bpf.ko 2024-03-29T07:41:19.732 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_cgroup.ko 2024-03-29T07:41:19.732 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_cluster.ko 2024-03-29T07:41:19.732 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_comment.ko 2024-03-29T07:41:19.732 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_connbytes.ko 2024-03-29T07:41:19.732 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_connlabel.ko 2024-03-29T07:41:19.732 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_connlimit.ko 2024-03-29T07:41:19.732 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_connmark.ko 2024-03-29T07:41:19.733 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_conntrack.ko 2024-03-29T07:41:19.733 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_cpu.ko 2024-03-29T07:41:19.733 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_dccp.ko 2024-03-29T07:41:19.733 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_devgroup.ko 2024-03-29T07:41:19.733 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_dscp.ko 2024-03-29T07:41:19.733 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_ecn.ko 2024-03-29T07:41:19.733 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_esp.ko 2024-03-29T07:41:19.733 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_hashlimit.ko 2024-03-29T07:41:19.733 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_helper.ko 2024-03-29T07:41:19.733 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_hl.ko 2024-03-29T07:41:19.733 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_ipcomp.ko 2024-03-29T07:41:19.733 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_iprange.ko 2024-03-29T07:41:19.733 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_ipvs.ko 2024-03-29T07:41:19.733 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_l2tp.ko 2024-03-29T07:41:19.733 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_length.ko 2024-03-29T07:41:19.733 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_limit.ko 2024-03-29T07:41:19.734 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_mac.ko 2024-03-29T07:41:19.734 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_mark.ko 2024-03-29T07:41:19.734 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_multiport.ko 2024-03-29T07:41:19.734 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_nat.ko 2024-03-29T07:41:19.734 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_nfacct.ko 2024-03-29T07:41:19.734 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_osf.ko 2024-03-29T07:41:19.734 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_owner.ko 2024-03-29T07:41:19.734 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_physdev.ko 2024-03-29T07:41:19.734 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_pkttype.ko 2024-03-29T07:41:19.734 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_policy.ko 2024-03-29T07:41:19.734 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_quota.ko 2024-03-29T07:41:19.734 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_rateest.ko 2024-03-29T07:41:19.734 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_realm.ko 2024-03-29T07:41:19.734 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_recent.ko 2024-03-29T07:41:19.734 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_sctp.ko 2024-03-29T07:41:19.735 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_set.ko 2024-03-29T07:41:19.735 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_socket.ko 2024-03-29T07:41:19.735 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_state.ko 2024-03-29T07:41:19.735 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_statistic.ko 2024-03-29T07:41:19.735 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_string.ko 2024-03-29T07:41:19.735 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_tcpmss.ko 2024-03-29T07:41:19.735 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_time.ko 2024-03-29T07:41:19.735 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_u32.ko 2024-03-29T07:41:19.735 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netlink 2024-03-29T07:41:19.735 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netlink/netlink_diag.ko 2024-03-29T07:41:19.735 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/nsh 2024-03-29T07:41:19.735 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/nsh/nsh.ko 2024-03-29T07:41:19.735 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/openvswitch 2024-03-29T07:41:19.735 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/openvswitch/openvswitch.ko 2024-03-29T07:41:19.735 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/openvswitch/vport-geneve.ko 2024-03-29T07:41:19.736 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/openvswitch/vport-gre.ko 2024-03-29T07:41:19.736 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/openvswitch/vport-vxlan.ko 2024-03-29T07:41:19.736 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/packet 2024-03-29T07:41:19.736 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/packet/af_packet_diag.ko 2024-03-29T07:41:19.736 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/rds 2024-03-29T07:41:19.736 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/rds/rds.ko 2024-03-29T07:41:19.736 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/rds/rds_rdma.ko 2024-03-29T07:41:19.736 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/rds/rds_tcp.ko 2024-03-29T07:41:19.736 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched 2024-03-29T07:41:19.736 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_csum.ko 2024-03-29T07:41:19.736 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_gact.ko 2024-03-29T07:41:19.736 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_mirred.ko 2024-03-29T07:41:19.736 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_nat.ko 2024-03-29T07:41:19.736 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_pedit.ko 2024-03-29T07:41:19.736 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_police.ko 2024-03-29T07:41:19.737 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_simple.ko 2024-03-29T07:41:19.737 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_skbedit.ko 2024-03-29T07:41:19.737 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_vlan.ko 2024-03-29T07:41:19.737 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_basic.ko 2024-03-29T07:41:19.737 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_bpf.ko 2024-03-29T07:41:19.737 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_flow.ko 2024-03-29T07:41:19.737 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_fw.ko 2024-03-29T07:41:19.737 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_route.ko 2024-03-29T07:41:19.737 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_u32.ko 2024-03-29T07:41:19.737 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_cmp.ko 2024-03-29T07:41:19.737 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_ipset.ko 2024-03-29T07:41:19.737 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_meta.ko 2024-03-29T07:41:19.737 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_nbyte.ko 2024-03-29T07:41:19.737 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_text.ko 2024-03-29T07:41:19.737 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_u32.ko 2024-03-29T07:41:19.738 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_choke.ko 2024-03-29T07:41:19.738 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_codel.ko 2024-03-29T07:41:19.738 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_drr.ko 2024-03-29T07:41:19.738 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_fq.ko 2024-03-29T07:41:19.738 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_gred.ko 2024-03-29T07:41:19.738 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_hfsc.ko 2024-03-29T07:41:19.738 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_hhf.ko 2024-03-29T07:41:19.738 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_htb.ko 2024-03-29T07:41:19.738 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_ingress.ko 2024-03-29T07:41:19.738 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_mqprio.ko 2024-03-29T07:41:19.738 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_mqprio_lib.ko 2024-03-29T07:41:19.738 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_multiq.ko 2024-03-29T07:41:19.738 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_netem.ko 2024-03-29T07:41:19.738 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_pie.ko 2024-03-29T07:41:19.738 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_plug.ko 2024-03-29T07:41:19.739 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_prio.ko 2024-03-29T07:41:19.739 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_qfq.ko 2024-03-29T07:41:19.739 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_red.ko 2024-03-29T07:41:19.739 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_sfb.ko 2024-03-29T07:41:19.739 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_sfq.ko 2024-03-29T07:41:19.739 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_tbf.ko 2024-03-29T07:41:19.739 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_teql.ko 2024-03-29T07:41:19.739 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sctp 2024-03-29T07:41:19.739 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sctp/sctp.ko 2024-03-29T07:41:19.739 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sctp/sctp_diag.ko 2024-03-29T07:41:19.739 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc 2024-03-29T07:41:19.739 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/auth_gss 2024-03-29T07:41:19.739 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-03-29T07:41:19.739 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-03-29T07:41:19.739 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/sunrpc.ko 2024-03-29T07:41:19.740 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/xprtrdma 2024-03-29T07:41:19.740 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-03-29T07:41:19.740 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/tipc 2024-03-29T07:41:19.740 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/tipc/diag.ko 2024-03-29T07:41:19.740 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/tipc/tipc.ko 2024-03-29T07:41:19.740 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/unix 2024-03-29T07:41:19.740 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/unix/unix_diag.ko 2024-03-29T07:41:19.740 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock 2024-03-29T07:41:19.740 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-03-29T07:41:19.740 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-03-29T07:41:19.740 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock/vsock.ko 2024-03-29T07:41:19.740 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock/vsock_diag.ko 2024-03-29T07:41:19.740 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock/vsock_loopback.ko 2024-03-29T07:41:19.740 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/xfrm 2024-03-29T07:41:19.740 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/xfrm/xfrm_ipcomp.ko 2024-03-29T07:41:19.741 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security 2024-03-29T07:41:19.741 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security/keys 2024-03-29T07:41:19.741 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security/keys/encrypted-keys 2024-03-29T07:41:19.741 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-03-29T07:41:19.741 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security/keys/trusted-keys 2024-03-29T07:41:19.741 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security/keys/trusted-keys/trusted.ko 2024-03-29T07:41:19.741 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/virt 2024-03-29T07:41:19.741 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/virt/lib 2024-03-29T07:41:19.741 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/virt/lib/irqbypass.ko 2024-03-29T07:41:19.741 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.alias 2024-03-29T07:41:19.741 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.alias.bin 2024-03-29T07:41:19.741 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.builtin 2024-03-29T07:41:19.741 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.builtin.bin 2024-03-29T07:41:19.741 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.builtin.modinfo 2024-03-29T07:41:19.741 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.dep 2024-03-29T07:41:19.742 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.dep.bin 2024-03-29T07:41:19.742 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.devname 2024-03-29T07:41:19.742 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.order 2024-03-29T07:41:19.742 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.softdep 2024-03-29T07:41:19.742 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.symbols 2024-03-29T07:41:19.742 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.symbols.bin 2024-03-29T07:41:19.742 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.8.0-g1df0c7e8a235/vmlinuz 2024-03-29T07:41:19.746 DEBUG:teuthology.task.kernel:get_image_version: 6.8.0-g1df0c7e8a235 2024-03-29T07:41:19.746 DEBUG:teuthology.orchestra.run.smithi148:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-03-29T07:41:19.772 INFO:teuthology.orchestra.run.smithi148.stdout:postinstall scriptlet (using /bin/sh): 2024-03-29T07:41:19.772 INFO:teuthology.orchestra.run.smithi148.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-03-29T07:41:19.772 INFO:teuthology.orchestra.run.smithi148.stdout: /usr/bin/kernel-install add 6.8.0-g1df0c7e8a235 /lib/modules/6.8.0-g1df0c7e8a235/vmlinuz 2024-03-29T07:41:19.772 INFO:teuthology.orchestra.run.smithi148.stdout:fi 2024-03-29T07:41:19.772 INFO:teuthology.orchestra.run.smithi148.stdout:for file in vmlinuz System.map config; do 2024-03-29T07:41:19.772 INFO:teuthology.orchestra.run.smithi148.stdout: if ! cmp --silent "/lib/modules/6.8.0-g1df0c7e8a235/${file}" "/boot/${file}-6.8.0-g1df0c7e8a235"; then 2024-03-29T07:41:19.772 INFO:teuthology.orchestra.run.smithi148.stdout: cp "/lib/modules/6.8.0-g1df0c7e8a235/${file}" "/boot/${file}-6.8.0-g1df0c7e8a235" 2024-03-29T07:41:19.772 INFO:teuthology.orchestra.run.smithi148.stdout: fi 2024-03-29T07:41:19.772 INFO:teuthology.orchestra.run.smithi148.stdout:done 2024-03-29T07:41:19.772 INFO:teuthology.orchestra.run.smithi148.stdout:preuninstall scriptlet (using /bin/sh): 2024-03-29T07:41:19.772 INFO:teuthology.orchestra.run.smithi148.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2024-03-29T07:41:19.773 INFO:teuthology.orchestra.run.smithi148.stdout:new-kernel-pkg --remove 6.8.0-g1df0c7e8a235 --rminitrd --initrdfile=/boot/initramfs-6.8.0-g1df0c7e8a235.img 2024-03-29T07:41:19.773 INFO:teuthology.orchestra.run.smithi148.stdout:elif [ -x /usr/bin/kernel-install ]; then 2024-03-29T07:41:19.773 INFO:teuthology.orchestra.run.smithi148.stdout:kernel-install remove 6.8.0-g1df0c7e8a235 2024-03-29T07:41:19.773 INFO:teuthology.orchestra.run.smithi148.stdout:fi 2024-03-29T07:41:19.773 INFO:teuthology.orchestra.run.smithi148.stdout:postuninstall scriptlet (using /bin/sh): 2024-03-29T07:41:19.773 INFO:teuthology.orchestra.run.smithi148.stdout:if [ -x /sbin/update-bootloader ]; then 2024-03-29T07:41:19.773 INFO:teuthology.orchestra.run.smithi148.stdout:/sbin/update-bootloader --remove 6.8.0-g1df0c7e8a235 2024-03-29T07:41:19.773 INFO:teuthology.orchestra.run.smithi148.stdout:fi 2024-03-29T07:41:19.774 DEBUG:teuthology.orchestra.run.smithi148:> sudo rpm -qi grub2-tools 2024-03-29T07:41:19.851 INFO:teuthology.orchestra.run.smithi148.stdout:Name : grub2-tools 2024-03-29T07:41:19.851 INFO:teuthology.orchestra.run.smithi148.stdout:Epoch : 1 2024-03-29T07:41:19.851 INFO:teuthology.orchestra.run.smithi148.stdout:Version : 2.02 2024-03-29T07:41:19.851 INFO:teuthology.orchestra.run.smithi148.stdout:Release : 129.el8 2024-03-29T07:41:19.851 INFO:teuthology.orchestra.run.smithi148.stdout:Architecture: x86_64 2024-03-29T07:41:19.851 INFO:teuthology.orchestra.run.smithi148.stdout:Install Date: Thu 04 May 2023 10:17:48 PM UTC 2024-03-29T07:41:19.852 INFO:teuthology.orchestra.run.smithi148.stdout:Group : System Environment/Base 2024-03-29T07:41:19.852 INFO:teuthology.orchestra.run.smithi148.stdout:Size : 9265586 2024-03-29T07:41:19.852 INFO:teuthology.orchestra.run.smithi148.stdout:License : GPLv3+ 2024-03-29T07:41:19.852 INFO:teuthology.orchestra.run.smithi148.stdout:Signature : RSA/SHA256, Tue 19 Jul 2022 02:52:06 PM UTC, Key ID 05b555b38483c65d 2024-03-29T07:41:19.852 INFO:teuthology.orchestra.run.smithi148.stdout:Source RPM : grub2-2.02-129.el8.src.rpm 2024-03-29T07:41:19.852 INFO:teuthology.orchestra.run.smithi148.stdout:Build Date : Tue 19 Jul 2022 02:30:39 PM UTC 2024-03-29T07:41:19.852 INFO:teuthology.orchestra.run.smithi148.stdout:Build Host : kbuilder.bsys.centos.org 2024-03-29T07:41:19.852 INFO:teuthology.orchestra.run.smithi148.stdout:Relocations : (not relocatable) 2024-03-29T07:41:19.853 INFO:teuthology.orchestra.run.smithi148.stdout:Packager : CentOS BuildSystem 2024-03-29T07:41:19.853 INFO:teuthology.orchestra.run.smithi148.stdout:Vendor : CentOS 2024-03-29T07:41:19.853 INFO:teuthology.orchestra.run.smithi148.stdout:URL : http://www.gnu.org/software/grub/ 2024-03-29T07:41:19.853 INFO:teuthology.orchestra.run.smithi148.stdout:Summary : Support tools for GRUB. 2024-03-29T07:41:19.853 INFO:teuthology.orchestra.run.smithi148.stdout:Description : 2024-03-29T07:41:19.853 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T07:41:19.853 INFO:teuthology.orchestra.run.smithi148.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-03-29T07:41:19.854 INFO:teuthology.orchestra.run.smithi148.stdout:customizable bootloader with modular architecture. It supports a rich 2024-03-29T07:41:19.854 INFO:teuthology.orchestra.run.smithi148.stdout:variety of kernel formats, file systems, computer architectures and 2024-03-29T07:41:19.854 INFO:teuthology.orchestra.run.smithi148.stdout:hardware devices. 2024-03-29T07:41:19.854 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T07:41:19.854 INFO:teuthology.orchestra.run.smithi148.stdout:This subpackage provides tools for support of all platforms. 2024-03-29T07:41:19.856 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-03-29T07:41:19.856 INFO:teuthology.task.kernel:Updating grub on smithi148 to boot 6.8.0-g1df0c7e8a235 2024-03-29T07:41:19.856 DEBUG:teuthology.orchestra.run.smithi148:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-03-29T07:41:23.956 INFO:teuthology.orchestra.run.smithi148.stderr:Generating grub configuration file ... 2024-03-29T07:41:26.093 INFO:teuthology.orchestra.run.smithi148.stderr:done 2024-03-29T07:41:26.095 DEBUG:teuthology.orchestra.run.smithi148:> mktemp 2024-03-29T07:41:26.121 INFO:teuthology.orchestra.run.smithi148.stdout:/tmp/tmp.I6fvEOnP6X 2024-03-29T07:41:26.121 DEBUG:teuthology.orchestra.run.smithi148:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.I6fvEOnP6X 2024-03-29T07:41:26.198 DEBUG:teuthology.orchestra.run.smithi148:> sudo chmod 0666 /tmp/tmp.I6fvEOnP6X 2024-03-29T07:41:26.411 DEBUG:teuthology.orchestra.remote:smithi148:/tmp/tmp.I6fvEOnP6X is 6KB 2024-03-29T07:41:26.459 DEBUG:teuthology.orchestra.run.smithi148:> rm -fr /tmp/tmp.I6fvEOnP6X 2024-03-29T07:41:26.477 DEBUG:teuthology.orchestra.run.smithi148:> sudo /bin/ls /boot/loader/entries || true 2024-03-29T07:41:26.546 INFO:teuthology.orchestra.run.smithi148.stdout:a1b29d6c289747bdb22930598d39874a-0-rescue.conf 2024-03-29T07:41:26.546 INFO:teuthology.orchestra.run.smithi148.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-348.el8.x86_64.conf 2024-03-29T07:41:26.546 INFO:teuthology.orchestra.run.smithi148.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-489.el8.x86_64.conf 2024-03-29T07:41:26.546 INFO:teuthology.orchestra.run.smithi148.stdout:a1b29d6c289747bdb22930598d39874a-6.8.0-g1df0c7e8a235.conf 2024-03-29T07:41:26.547 DEBUG:teuthology.orchestra.run.smithi148:> sudo grub2-set-default a1b29d6c289747bdb22930598d39874a-6.8.0-g1df0c7e8a235 2024-03-29T07:41:26.709 DEBUG:teuthology.orchestra.run.smithi148:> sudo shutdown -r now 2024-03-29T07:41:39.516 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi037.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-03-29T07:41:39.516 DEBUG:teuthology.orchestra.run.smithi037:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-03-29T07:41:39.575 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235 2024-03-29T07:41:39.576 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/System.map 2024-03-29T07:41:39.576 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/config 2024-03-29T07:41:39.576 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel 2024-03-29T07:41:39.576 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch 2024-03-29T07:41:39.576 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86 2024-03-29T07:41:39.576 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto 2024-03-29T07:41:39.576 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-03-29T07:41:39.576 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-03-29T07:41:39.576 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-03-29T07:41:39.576 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-03-29T07:41:39.576 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-03-29T07:41:39.576 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-03-29T07:41:39.576 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-03-29T07:41:39.576 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/crc32c-intel.ko 2024-03-29T07:41:39.577 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-03-29T07:41:39.577 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-03-29T07:41:39.577 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-03-29T07:41:39.577 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-03-29T07:41:39.577 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/serpent-avx2.ko 2024-03-29T07:41:39.577 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-03-29T07:41:39.577 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-03-29T07:41:39.577 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-03-29T07:41:39.577 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-03-29T07:41:39.577 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-03-29T07:41:39.577 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-03-29T07:41:39.577 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-03-29T07:41:39.577 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kernel 2024-03-29T07:41:39.577 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kernel/cpu 2024-03-29T07:41:39.578 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kernel/cpu/mce 2024-03-29T07:41:39.578 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-03-29T07:41:39.578 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kvm 2024-03-29T07:41:39.578 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kvm/kvm-amd.ko 2024-03-29T07:41:39.578 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kvm/kvm-intel.ko 2024-03-29T07:41:39.578 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kvm/kvm.ko 2024-03-29T07:41:39.578 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto 2024-03-29T07:41:39.578 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/ansi_cprng.ko 2024-03-29T07:41:39.578 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/anubis.ko 2024-03-29T07:41:39.578 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/arc4.ko 2024-03-29T07:41:39.578 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx 2024-03-29T07:41:39.578 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/async_memcpy.ko 2024-03-29T07:41:39.578 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/async_pq.ko 2024-03-29T07:41:39.578 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/async_raid6_recov.ko 2024-03-29T07:41:39.578 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/async_tx.ko 2024-03-29T07:41:39.579 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/async_xor.ko 2024-03-29T07:41:39.579 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/raid6test.ko 2024-03-29T07:41:39.579 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/authenc.ko 2024-03-29T07:41:39.579 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/authencesn.ko 2024-03-29T07:41:39.579 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/blake2b_generic.ko 2024-03-29T07:41:39.579 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/blowfish_common.ko 2024-03-29T07:41:39.579 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/blowfish_generic.ko 2024-03-29T07:41:39.579 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/camellia_generic.ko 2024-03-29T07:41:39.579 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/cast5_generic.ko 2024-03-29T07:41:39.579 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/cast6_generic.ko 2024-03-29T07:41:39.579 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/cast_common.ko 2024-03-29T07:41:39.579 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/ccm.ko 2024-03-29T07:41:39.579 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/cmac.ko 2024-03-29T07:41:39.579 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/crc32_generic.ko 2024-03-29T07:41:39.580 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/crypto_engine.ko 2024-03-29T07:41:39.580 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/crypto_user.ko 2024-03-29T07:41:39.580 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/cts.ko 2024-03-29T07:41:39.580 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/des_generic.ko 2024-03-29T07:41:39.580 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/dh_generic.ko 2024-03-29T07:41:39.580 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/echainiv.ko 2024-03-29T07:41:39.580 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/essiv.ko 2024-03-29T07:41:39.580 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/fcrypt.ko 2024-03-29T07:41:39.580 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/khazad.ko 2024-03-29T07:41:39.580 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/lz4.ko 2024-03-29T07:41:39.580 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/lz4hc.ko 2024-03-29T07:41:39.580 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/md4.ko 2024-03-29T07:41:39.580 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/michael_mic.ko 2024-03-29T07:41:39.580 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/pcbc.ko 2024-03-29T07:41:39.581 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/pcrypt.ko 2024-03-29T07:41:39.581 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/rmd160.ko 2024-03-29T07:41:39.581 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/seed.ko 2024-03-29T07:41:39.581 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/serpent_generic.ko 2024-03-29T07:41:39.581 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/tcrypt.ko 2024-03-29T07:41:39.581 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/tea.ko 2024-03-29T07:41:39.581 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/twofish_common.ko 2024-03-29T07:41:39.581 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/twofish_generic.ko 2024-03-29T07:41:39.581 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/vmac.ko 2024-03-29T07:41:39.581 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/wp512.ko 2024-03-29T07:41:39.581 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/xcbc.ko 2024-03-29T07:41:39.581 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/xor.ko 2024-03-29T07:41:39.581 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/xxhash_generic.ko 2024-03-29T07:41:39.581 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers 2024-03-29T07:41:39.581 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi 2024-03-29T07:41:39.581 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/acpi_ipmi.ko 2024-03-29T07:41:39.582 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/acpi_pad.ko 2024-03-29T07:41:39.582 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/custom_method.ko 2024-03-29T07:41:39.582 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/ec_sys.ko 2024-03-29T07:41:39.582 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/platform_profile.ko 2024-03-29T07:41:39.582 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/sbs.ko 2024-03-29T07:41:39.582 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/sbshc.ko 2024-03-29T07:41:39.582 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/video.ko 2024-03-29T07:41:39.582 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata 2024-03-29T07:41:39.582 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/acard-ahci.ko 2024-03-29T07:41:39.582 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/ahci_platform.ko 2024-03-29T07:41:39.582 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/ata_generic.ko 2024-03-29T07:41:39.582 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/libahci_platform.ko 2024-03-29T07:41:39.582 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_acpi.ko 2024-03-29T07:41:39.582 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_ali.ko 2024-03-29T07:41:39.583 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_amd.ko 2024-03-29T07:41:39.583 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_artop.ko 2024-03-29T07:41:39.583 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_atiixp.ko 2024-03-29T07:41:39.583 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_atp867x.ko 2024-03-29T07:41:39.583 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_cmd640.ko 2024-03-29T07:41:39.583 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_cmd64x.ko 2024-03-29T07:41:39.583 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_cypress.ko 2024-03-29T07:41:39.583 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_efar.ko 2024-03-29T07:41:39.583 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_hpt366.ko 2024-03-29T07:41:39.583 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_hpt37x.ko 2024-03-29T07:41:39.583 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_hpt3x2n.ko 2024-03-29T07:41:39.583 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_hpt3x3.ko 2024-03-29T07:41:39.583 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_it8213.ko 2024-03-29T07:41:39.583 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_it821x.ko 2024-03-29T07:41:39.583 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_jmicron.ko 2024-03-29T07:41:39.583 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_marvell.ko 2024-03-29T07:41:39.584 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_mpiix.ko 2024-03-29T07:41:39.584 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_netcell.ko 2024-03-29T07:41:39.584 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_ninja32.ko 2024-03-29T07:41:39.584 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_ns87410.ko 2024-03-29T07:41:39.584 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_ns87415.ko 2024-03-29T07:41:39.584 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_oldpiix.ko 2024-03-29T07:41:39.584 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_opti.ko 2024-03-29T07:41:39.584 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_optidma.ko 2024-03-29T07:41:39.584 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_pcmcia.ko 2024-03-29T07:41:39.584 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_pdc2027x.ko 2024-03-29T07:41:39.584 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-03-29T07:41:39.584 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_piccolo.ko 2024-03-29T07:41:39.584 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_rdc.ko 2024-03-29T07:41:39.584 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_sch.ko 2024-03-29T07:41:39.584 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_serverworks.ko 2024-03-29T07:41:39.585 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_sil680.ko 2024-03-29T07:41:39.585 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_sis.ko 2024-03-29T07:41:39.585 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_sl82c105.ko 2024-03-29T07:41:39.585 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_triflex.ko 2024-03-29T07:41:39.585 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_via.ko 2024-03-29T07:41:39.585 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pdc_adma.ko 2024-03-29T07:41:39.585 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_inic162x.ko 2024-03-29T07:41:39.585 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_mv.ko 2024-03-29T07:41:39.585 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_nv.ko 2024-03-29T07:41:39.585 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_promise.ko 2024-03-29T07:41:39.585 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_qstor.ko 2024-03-29T07:41:39.585 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_sil.ko 2024-03-29T07:41:39.585 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_sil24.ko 2024-03-29T07:41:39.585 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_sis.ko 2024-03-29T07:41:39.586 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_svw.ko 2024-03-29T07:41:39.586 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_sx4.ko 2024-03-29T07:41:39.586 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_uli.ko 2024-03-29T07:41:39.586 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_via.ko 2024-03-29T07:41:39.586 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_vsc.ko 2024-03-29T07:41:39.586 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/auxdisplay 2024-03-29T07:41:39.586 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/auxdisplay/ks0108.ko 2024-03-29T07:41:39.586 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/base 2024-03-29T07:41:39.586 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/base/regmap 2024-03-29T07:41:39.586 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/base/regmap/regmap-i2c.ko 2024-03-29T07:41:39.586 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/base/regmap/regmap-mmio.ko 2024-03-29T07:41:39.586 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/bcma 2024-03-29T07:41:39.586 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/bcma/bcma.ko 2024-03-29T07:41:39.586 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block 2024-03-29T07:41:39.587 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/aoe 2024-03-29T07:41:39.587 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/aoe/aoe.ko 2024-03-29T07:41:39.587 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/brd.ko 2024-03-29T07:41:39.587 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/drbd 2024-03-29T07:41:39.587 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/drbd/drbd.ko 2024-03-29T07:41:39.587 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/floppy.ko 2024-03-29T07:41:39.587 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/loop.ko 2024-03-29T07:41:39.587 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/mtip32xx 2024-03-29T07:41:39.587 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-03-29T07:41:39.587 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/nbd.ko 2024-03-29T07:41:39.587 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/null_blk 2024-03-29T07:41:39.587 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/null_blk/null_blk.ko 2024-03-29T07:41:39.587 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/pktcdvd.ko 2024-03-29T07:41:39.587 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/rbd.ko 2024-03-29T07:41:39.587 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/virtio_blk.ko 2024-03-29T07:41:39.588 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/xen-blkback 2024-03-29T07:41:39.588 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-03-29T07:41:39.588 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/xen-blkfront.ko 2024-03-29T07:41:39.588 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/zram 2024-03-29T07:41:39.588 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/zram/zram.ko 2024-03-29T07:41:39.588 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char 2024-03-29T07:41:39.588 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hangcheck-timer.ko 2024-03-29T07:41:39.588 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random 2024-03-29T07:41:39.588 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random/amd-rng.ko 2024-03-29T07:41:39.588 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random/intel-rng.ko 2024-03-29T07:41:39.588 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-03-29T07:41:39.588 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random/via-rng.ko 2024-03-29T07:41:39.588 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random/virtio-rng.ko 2024-03-29T07:41:39.588 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi 2024-03-29T07:41:39.588 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-03-29T07:41:39.589 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-03-29T07:41:39.589 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-03-29T07:41:39.589 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_si.ko 2024-03-29T07:41:39.589 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-03-29T07:41:39.589 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-03-29T07:41:39.589 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/lp.ko 2024-03-29T07:41:39.589 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/mwave 2024-03-29T07:41:39.589 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/mwave/mwave.ko 2024-03-29T07:41:39.589 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ppdev.ko 2024-03-29T07:41:39.589 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tlclk.ko 2024-03-29T07:41:39.589 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm 2024-03-29T07:41:39.589 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm.ko 2024-03-29T07:41:39.589 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm_atmel.ko 2024-03-29T07:41:39.589 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm_infineon.ko 2024-03-29T07:41:39.589 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm_nsc.ko 2024-03-29T07:41:39.590 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm_tis.ko 2024-03-29T07:41:39.590 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-03-29T07:41:39.590 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/uv_mmtimer.ko 2024-03-29T07:41:39.590 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/virtio_console.ko 2024-03-29T07:41:39.590 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq 2024-03-29T07:41:39.590 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-03-29T07:41:39.590 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-03-29T07:41:39.590 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/p4-clockmod.ko 2024-03-29T07:41:39.590 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-03-29T07:41:39.590 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/powernow-k8.ko 2024-03-29T07:41:39.590 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/speedstep-lib.ko 2024-03-29T07:41:39.590 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto 2024-03-29T07:41:39.590 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/ccp 2024-03-29T07:41:39.590 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-03-29T07:41:39.590 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/ccp/ccp.ko 2024-03-29T07:41:39.591 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel 2024-03-29T07:41:39.591 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel/qat 2024-03-29T07:41:39.591 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel/qat/qat_common 2024-03-29T07:41:39.591 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-03-29T07:41:39.591 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-03-29T07:41:39.591 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-03-29T07:41:39.591 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/padlock-aes.ko 2024-03-29T07:41:39.591 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/padlock-sha.ko 2024-03-29T07:41:39.591 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/virtio 2024-03-29T07:41:39.591 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-03-29T07:41:39.591 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dca 2024-03-29T07:41:39.591 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dca/dca.ko 2024-03-29T07:41:39.591 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/devfreq 2024-03-29T07:41:39.591 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-03-29T07:41:39.591 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dma 2024-03-29T07:41:39.592 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dma/dw 2024-03-29T07:41:39.592 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dma/dw/dw_dmac.ko 2024-03-29T07:41:39.592 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dma/ioat 2024-03-29T07:41:39.592 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dma/ioat/ioatdma.ko 2024-03-29T07:41:39.592 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac 2024-03-29T07:41:39.592 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/amd64_edac.ko 2024-03-29T07:41:39.592 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/e752x_edac.ko 2024-03-29T07:41:39.592 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/edac_mce_amd.ko 2024-03-29T07:41:39.592 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i3000_edac.ko 2024-03-29T07:41:39.592 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i3200_edac.ko 2024-03-29T07:41:39.592 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i5100_edac.ko 2024-03-29T07:41:39.592 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i5400_edac.ko 2024-03-29T07:41:39.592 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i7300_edac.ko 2024-03-29T07:41:39.592 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i7core_edac.ko 2024-03-29T07:41:39.592 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i82975x_edac.ko 2024-03-29T07:41:39.593 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/ie31200_edac.ko 2024-03-29T07:41:39.593 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/sb_edac.ko 2024-03-29T07:41:39.593 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/x38_edac.ko 2024-03-29T07:41:39.593 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/firmware 2024-03-29T07:41:39.593 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/firmware/edd.ko 2024-03-29T07:41:39.593 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/firmware/iscsi_ibft.ko 2024-03-29T07:41:39.593 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio 2024-03-29T07:41:39.593 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio/gpio-ich.ko 2024-03-29T07:41:39.593 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio/gpio-viperboard.ko 2024-03-29T07:41:39.593 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid 2024-03-29T07:41:39.593 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-appleir.ko 2024-03-29T07:41:39.593 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-aureal.ko 2024-03-29T07:41:39.593 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-axff.ko 2024-03-29T07:41:39.593 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-dr.ko 2024-03-29T07:41:39.594 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-elecom.ko 2024-03-29T07:41:39.594 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-elo.ko 2024-03-29T07:41:39.594 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-emsff.ko 2024-03-29T07:41:39.594 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-gaff.ko 2024-03-29T07:41:39.594 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-gt683r.ko 2024-03-29T07:41:39.594 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-gyration.ko 2024-03-29T07:41:39.594 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-holtek-kbd.ko 2024-03-29T07:41:39.594 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-holtek-mouse.ko 2024-03-29T07:41:39.594 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-holtekff.ko 2024-03-29T07:41:39.594 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-hyperv.ko 2024-03-29T07:41:39.594 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-icade.ko 2024-03-29T07:41:39.594 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-keytouch.ko 2024-03-29T07:41:39.594 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-kye.ko 2024-03-29T07:41:39.594 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-lcpower.ko 2024-03-29T07:41:39.594 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-led.ko 2024-03-29T07:41:39.595 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-lenovo.ko 2024-03-29T07:41:39.595 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-logitech-dj.ko 2024-03-29T07:41:39.595 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-03-29T07:41:39.595 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-multitouch.ko 2024-03-29T07:41:39.595 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-ortek.ko 2024-03-29T07:41:39.595 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-penmount.ko 2024-03-29T07:41:39.595 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-petalynx.ko 2024-03-29T07:41:39.595 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-picolcd.ko 2024-03-29T07:41:39.595 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-pl.ko 2024-03-29T07:41:39.595 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-plantronics.ko 2024-03-29T07:41:39.595 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-primax.ko 2024-03-29T07:41:39.595 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-rmi.ko 2024-03-29T07:41:39.595 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-arvo.ko 2024-03-29T07:41:39.595 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-common.ko 2024-03-29T07:41:39.596 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-isku.ko 2024-03-29T07:41:39.596 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-kone.ko 2024-03-29T07:41:39.596 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-03-29T07:41:39.596 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-konepure.ko 2024-03-29T07:41:39.596 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-03-29T07:41:39.596 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-lua.ko 2024-03-29T07:41:39.596 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-pyra.ko 2024-03-29T07:41:39.596 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-ryos.ko 2024-03-29T07:41:39.596 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-savu.ko 2024-03-29T07:41:39.596 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat.ko 2024-03-29T07:41:39.596 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-saitek.ko 2024-03-29T07:41:39.596 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-samsung.ko 2024-03-29T07:41:39.596 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-sensor-hub.ko 2024-03-29T07:41:39.596 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-sjoy.ko 2024-03-29T07:41:39.597 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-sony.ko 2024-03-29T07:41:39.597 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-speedlink.ko 2024-03-29T07:41:39.597 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-steelseries.ko 2024-03-29T07:41:39.597 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-sunplus.ko 2024-03-29T07:41:39.597 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-thrustmaster.ko 2024-03-29T07:41:39.597 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-tivo.ko 2024-03-29T07:41:39.597 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-tmff.ko 2024-03-29T07:41:39.597 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-topseed.ko 2024-03-29T07:41:39.597 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-twinhan.ko 2024-03-29T07:41:39.597 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-uclogic.ko 2024-03-29T07:41:39.597 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-waltop.ko 2024-03-29T07:41:39.597 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-wiimote.ko 2024-03-29T07:41:39.597 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-xinmo.ko 2024-03-29T07:41:39.597 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-zpff.ko 2024-03-29T07:41:39.597 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-zydacron.ko 2024-03-29T07:41:39.598 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/uhid.ko 2024-03-29T07:41:39.598 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/wacom.ko 2024-03-29T07:41:39.598 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hv 2024-03-29T07:41:39.598 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hv/hv_balloon.ko 2024-03-29T07:41:39.598 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hv/hv_utils.ko 2024-03-29T07:41:39.598 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hv/hv_vmbus.ko 2024-03-29T07:41:39.598 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon 2024-03-29T07:41:39.598 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/abituguru.ko 2024-03-29T07:41:39.598 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/abituguru3.ko 2024-03-29T07:41:39.598 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/acpi_power_meter.ko 2024-03-29T07:41:39.598 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ad7414.ko 2024-03-29T07:41:39.598 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ad7418.ko 2024-03-29T07:41:39.598 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adc128d818.ko 2024-03-29T07:41:39.598 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adm1025.ko 2024-03-29T07:41:39.598 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adm1026.ko 2024-03-29T07:41:39.599 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adm1029.ko 2024-03-29T07:41:39.599 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adm1031.ko 2024-03-29T07:41:39.599 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adm9240.ko 2024-03-29T07:41:39.599 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ads7828.ko 2024-03-29T07:41:39.599 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7410.ko 2024-03-29T07:41:39.599 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7411.ko 2024-03-29T07:41:39.599 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7462.ko 2024-03-29T07:41:39.599 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7470.ko 2024-03-29T07:41:39.599 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7475.ko 2024-03-29T07:41:39.599 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7x10.ko 2024-03-29T07:41:39.599 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/amc6821.ko 2024-03-29T07:41:39.599 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/applesmc.ko 2024-03-29T07:41:39.599 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/asb100.ko 2024-03-29T07:41:39.599 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/asc7621.ko 2024-03-29T07:41:39.599 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/asus_atk0110.ko 2024-03-29T07:41:39.600 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/atxp1.ko 2024-03-29T07:41:39.600 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/coretemp.ko 2024-03-29T07:41:39.600 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-03-29T07:41:39.600 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/dme1737.ko 2024-03-29T07:41:39.600 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ds1621.ko 2024-03-29T07:41:39.600 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ds620.ko 2024-03-29T07:41:39.600 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/emc1403.ko 2024-03-29T07:41:39.600 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/emc6w201.ko 2024-03-29T07:41:39.600 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/f71805f.ko 2024-03-29T07:41:39.600 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/f71882fg.ko 2024-03-29T07:41:39.600 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/f75375s.ko 2024-03-29T07:41:39.600 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/fam15h_power.ko 2024-03-29T07:41:39.600 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/fschmd.ko 2024-03-29T07:41:39.600 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/g760a.ko 2024-03-29T07:41:39.600 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/g762.ko 2024-03-29T07:41:39.601 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/gl518sm.ko 2024-03-29T07:41:39.601 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/gl520sm.ko 2024-03-29T07:41:39.601 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/hwmon-vid.ko 2024-03-29T07:41:39.601 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/i5500_temp.ko 2024-03-29T07:41:39.601 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/i5k_amb.ko 2024-03-29T07:41:39.601 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ibmaem.ko 2024-03-29T07:41:39.601 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ibmpex.ko 2024-03-29T07:41:39.601 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ina209.ko 2024-03-29T07:41:39.601 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ina2xx.ko 2024-03-29T07:41:39.601 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/it87.ko 2024-03-29T07:41:39.601 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/k10temp.ko 2024-03-29T07:41:39.601 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/k8temp.ko 2024-03-29T07:41:39.601 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lineage-pem.ko 2024-03-29T07:41:39.602 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm63.ko 2024-03-29T07:41:39.602 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm73.ko 2024-03-29T07:41:39.602 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm75.ko 2024-03-29T07:41:39.602 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm77.ko 2024-03-29T07:41:39.602 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm78.ko 2024-03-29T07:41:39.602 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm80.ko 2024-03-29T07:41:39.602 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm83.ko 2024-03-29T07:41:39.602 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm85.ko 2024-03-29T07:41:39.602 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm87.ko 2024-03-29T07:41:39.602 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm90.ko 2024-03-29T07:41:39.602 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm92.ko 2024-03-29T07:41:39.602 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm93.ko 2024-03-29T07:41:39.602 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm95234.ko 2024-03-29T07:41:39.602 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm95241.ko 2024-03-29T07:41:39.603 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm95245.ko 2024-03-29T07:41:39.603 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc2945.ko 2024-03-29T07:41:39.603 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4151.ko 2024-03-29T07:41:39.603 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4215.ko 2024-03-29T07:41:39.603 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4222.ko 2024-03-29T07:41:39.603 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4245.ko 2024-03-29T07:41:39.603 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4260.ko 2024-03-29T07:41:39.603 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4261.ko 2024-03-29T07:41:39.603 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max16065.ko 2024-03-29T07:41:39.603 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max1619.ko 2024-03-29T07:41:39.603 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max1668.ko 2024-03-29T07:41:39.603 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max197.ko 2024-03-29T07:41:39.603 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max6639.ko 2024-03-29T07:41:39.603 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max6650.ko 2024-03-29T07:41:39.603 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max6697.ko 2024-03-29T07:41:39.604 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/mcp3021.ko 2024-03-29T07:41:39.604 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/nct6683.ko 2024-03-29T07:41:39.604 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/nct6775-core.ko 2024-03-29T07:41:39.604 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/nct6775.ko 2024-03-29T07:41:39.604 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/nct7802.ko 2024-03-29T07:41:39.604 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pc87360.ko 2024-03-29T07:41:39.604 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pc87427.ko 2024-03-29T07:41:39.604 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pcf8591.ko 2024-03-29T07:41:39.604 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus 2024-03-29T07:41:39.604 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-03-29T07:41:39.604 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-03-29T07:41:39.604 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-03-29T07:41:39.604 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/max16064.ko 2024-03-29T07:41:39.604 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/max34440.ko 2024-03-29T07:41:39.604 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/max8688.ko 2024-03-29T07:41:39.605 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-03-29T07:41:39.605 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-03-29T07:41:39.605 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-03-29T07:41:39.605 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-03-29T07:41:39.605 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-03-29T07:41:39.605 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-03-29T07:41:39.605 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/powr1220.ko 2024-03-29T07:41:39.605 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sch5627.ko 2024-03-29T07:41:39.605 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sch5636.ko 2024-03-29T07:41:39.605 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sch56xx-common.ko 2024-03-29T07:41:39.605 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sht15.ko 2024-03-29T07:41:39.605 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sht21.ko 2024-03-29T07:41:39.605 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/shtc1.ko 2024-03-29T07:41:39.605 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sis5595.ko 2024-03-29T07:41:39.606 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/smsc47b397.ko 2024-03-29T07:41:39.606 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/smsc47m1.ko 2024-03-29T07:41:39.606 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/smsc47m192.ko 2024-03-29T07:41:39.606 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/thmc50.ko 2024-03-29T07:41:39.606 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/tmp102.ko 2024-03-29T07:41:39.606 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/tmp103.ko 2024-03-29T07:41:39.606 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/tmp401.ko 2024-03-29T07:41:39.606 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/tmp421.ko 2024-03-29T07:41:39.606 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/via-cputemp.ko 2024-03-29T07:41:39.606 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/via686a.ko 2024-03-29T07:41:39.606 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/vt1211.ko 2024-03-29T07:41:39.606 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/vt8231.ko 2024-03-29T07:41:39.606 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83627ehf.ko 2024-03-29T07:41:39.606 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83627hf.ko 2024-03-29T07:41:39.607 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83781d.ko 2024-03-29T07:41:39.607 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83791d.ko 2024-03-29T07:41:39.607 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83792d.ko 2024-03-29T07:41:39.607 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83793.ko 2024-03-29T07:41:39.607 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83795.ko 2024-03-29T07:41:39.607 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83l785ts.ko 2024-03-29T07:41:39.607 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83l786ng.ko 2024-03-29T07:41:39.607 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c 2024-03-29T07:41:39.607 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/algos 2024-03-29T07:41:39.607 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-03-29T07:41:39.607 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-03-29T07:41:39.607 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses 2024-03-29T07:41:39.607 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-03-29T07:41:39.607 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-03-29T07:41:39.607 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-03-29T07:41:39.608 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-03-29T07:41:39.608 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-03-29T07:41:39.608 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-03-29T07:41:39.608 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-03-29T07:41:39.608 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-03-29T07:41:39.608 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-i801.ko 2024-03-29T07:41:39.608 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-isch.ko 2024-03-29T07:41:39.608 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-03-29T07:41:39.608 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-03-29T07:41:39.608 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-03-29T07:41:39.608 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-parport.ko 2024-03-29T07:41:39.608 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-03-29T07:41:39.608 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-03-29T07:41:39.608 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-03-29T07:41:39.608 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-03-29T07:41:39.609 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-03-29T07:41:39.609 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-03-29T07:41:39.609 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-via.ko 2024-03-29T07:41:39.609 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-03-29T07:41:39.609 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-03-29T07:41:39.609 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/i2c-dev.ko 2024-03-29T07:41:39.609 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/i2c-mux.ko 2024-03-29T07:41:39.609 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-03-29T07:41:39.609 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/i2c-smbus.ko 2024-03-29T07:41:39.609 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/i2c-stub.ko 2024-03-29T07:41:39.609 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband 2024-03-29T07:41:39.609 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core 2024-03-29T07:41:39.610 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/ib_cm.ko 2024-03-29T07:41:39.610 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/ib_core.ko 2024-03-29T07:41:39.610 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/ib_umad.ko 2024-03-29T07:41:39.610 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-03-29T07:41:39.610 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/iw_cm.ko 2024-03-29T07:41:39.610 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/rdma_cm.ko 2024-03-29T07:41:39.610 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-03-29T07:41:39.610 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw 2024-03-29T07:41:39.610 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/cxgb4 2024-03-29T07:41:39.610 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-03-29T07:41:39.610 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mlx4 2024-03-29T07:41:39.610 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-03-29T07:41:39.611 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mlx5 2024-03-29T07:41:39.611 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-03-29T07:41:39.611 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mthca 2024-03-29T07:41:39.611 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-03-29T07:41:39.611 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp 2024-03-29T07:41:39.611 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/ipoib 2024-03-29T07:41:39.611 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-03-29T07:41:39.611 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/iser 2024-03-29T07:41:39.611 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-03-29T07:41:39.611 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/isert 2024-03-29T07:41:39.611 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-03-29T07:41:39.611 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/srp 2024-03-29T07:41:39.611 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-03-29T07:41:39.611 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/srpt 2024-03-29T07:41:39.611 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-03-29T07:41:39.612 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input 2024-03-29T07:41:39.612 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/gameport 2024-03-29T07:41:39.612 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-03-29T07:41:39.612 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/gameport/fm801-gp.ko 2024-03-29T07:41:39.612 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/gameport/gameport.ko 2024-03-29T07:41:39.612 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/joydev.ko 2024-03-29T07:41:39.612 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/keyboard 2024-03-29T07:41:39.612 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/keyboard/gpio_keys.ko 2024-03-29T07:41:39.612 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse 2024-03-29T07:41:39.612 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/appletouch.ko 2024-03-29T07:41:39.612 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/bcm5974.ko 2024-03-29T07:41:39.612 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/cyapatp.ko 2024-03-29T07:41:39.612 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/elan_i2c.ko 2024-03-29T07:41:39.612 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/sermouse.ko 2024-03-29T07:41:39.612 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-03-29T07:41:39.613 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/synaptics_usb.ko 2024-03-29T07:41:39.613 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/vsxxxaa.ko 2024-03-29T07:41:39.613 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/rmi4 2024-03-29T07:41:39.613 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/rmi4/rmi_core.ko 2024-03-29T07:41:39.613 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio 2024-03-29T07:41:39.613 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio/altera_ps2.ko 2024-03-29T07:41:39.613 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio/arc_ps2.ko 2024-03-29T07:41:39.613 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-03-29T07:41:39.613 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio/serio_raw.ko 2024-03-29T07:41:39.613 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/sparse-keymap.ko 2024-03-29T07:41:39.613 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds 2024-03-29T07:41:39.613 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-blinkm.ko 2024-03-29T07:41:39.613 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-lm3530.ko 2024-03-29T07:41:39.613 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-lp3944.ko 2024-03-29T07:41:39.614 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-ss4200.ko 2024-03-29T07:41:39.614 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger 2024-03-29T07:41:39.614 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-03-29T07:41:39.614 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-03-29T07:41:39.614 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-03-29T07:41:39.614 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-03-29T07:41:39.614 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-03-29T07:41:39.614 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-03-29T07:41:39.614 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-03-29T07:41:39.614 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-03-29T07:41:39.614 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-03-29T07:41:39.614 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md 2024-03-29T07:41:39.614 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/bcache 2024-03-29T07:41:39.614 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/bcache/bcache.ko 2024-03-29T07:41:39.615 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-bio-prison.ko 2024-03-29T07:41:39.615 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-cache-smq.ko 2024-03-29T07:41:39.615 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-cache.ko 2024-03-29T07:41:39.615 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-crypt.ko 2024-03-29T07:41:39.615 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-delay.ko 2024-03-29T07:41:39.615 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-flakey.ko 2024-03-29T07:41:39.615 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-log-userspace.ko 2024-03-29T07:41:39.615 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-multipath.ko 2024-03-29T07:41:39.615 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-queue-length.ko 2024-03-29T07:41:39.615 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-raid.ko 2024-03-29T07:41:39.615 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-round-robin.ko 2024-03-29T07:41:39.615 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-service-time.ko 2024-03-29T07:41:39.615 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-switch.ko 2024-03-29T07:41:39.615 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-thin-pool.ko 2024-03-29T07:41:39.615 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-verity.ko 2024-03-29T07:41:39.615 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/persistent-data 2024-03-29T07:41:39.616 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-03-29T07:41:39.616 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/raid0.ko 2024-03-29T07:41:39.616 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/raid1.ko 2024-03-29T07:41:39.616 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/raid10.ko 2024-03-29T07:41:39.616 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/raid456.ko 2024-03-29T07:41:39.616 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message 2024-03-29T07:41:39.616 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion 2024-03-29T07:41:39.616 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptbase.ko 2024-03-29T07:41:39.616 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptctl.ko 2024-03-29T07:41:39.616 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptfc.ko 2024-03-29T07:41:39.616 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptlan.ko 2024-03-29T07:41:39.616 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptsas.ko 2024-03-29T07:41:39.616 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptscsih.ko 2024-03-29T07:41:39.616 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptspi.ko 2024-03-29T07:41:39.616 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd 2024-03-29T07:41:39.617 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/lpc_ich.ko 2024-03-29T07:41:39.617 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/lpc_sch.ko 2024-03-29T07:41:39.617 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/mfd-core.ko 2024-03-29T07:41:39.617 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/sm501.ko 2024-03-29T07:41:39.617 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/viperboard.ko 2024-03-29T07:41:39.617 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/vx855.ko 2024-03-29T07:41:39.617 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/wl1273-core.ko 2024-03-29T07:41:39.617 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc 2024-03-29T07:41:39.617 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/altera-stapl 2024-03-29T07:41:39.617 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-03-29T07:41:39.617 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/apds9802als.ko 2024-03-29T07:41:39.617 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/apds990x.ko 2024-03-29T07:41:39.617 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/bh1770glc.ko 2024-03-29T07:41:39.617 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/cb710 2024-03-29T07:41:39.617 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/cb710/cb710.ko 2024-03-29T07:41:39.618 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/eeprom 2024-03-29T07:41:39.618 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/eeprom/at24.ko 2024-03-29T07:41:39.618 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-03-29T07:41:39.618 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/eeprom/max6875.ko 2024-03-29T07:41:39.618 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/enclosure.ko 2024-03-29T07:41:39.618 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/hpilo.ko 2024-03-29T07:41:39.618 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/isl29003.ko 2024-03-29T07:41:39.618 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/isl29020.ko 2024-03-29T07:41:39.618 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/lis3lv02d 2024-03-29T07:41:39.618 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-03-29T07:41:39.618 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-03-29T07:41:39.618 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/mei 2024-03-29T07:41:39.618 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/mei/mei-me.ko 2024-03-29T07:41:39.618 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/mei/mei-txe.ko 2024-03-29T07:41:39.619 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/mei/mei.ko 2024-03-29T07:41:39.619 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-gru 2024-03-29T07:41:39.619 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-gru/gru.ko 2024-03-29T07:41:39.619 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-xp 2024-03-29T07:41:39.619 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-xp/xp.ko 2024-03-29T07:41:39.619 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-xp/xpc.ko 2024-03-29T07:41:39.619 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-03-29T07:41:39.619 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/tifm_7xx1.ko 2024-03-29T07:41:39.619 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/tifm_core.ko 2024-03-29T07:41:39.619 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/tsl2550.ko 2024-03-29T07:41:39.619 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/vmw_balloon.ko 2024-03-29T07:41:39.619 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/vmw_vmci 2024-03-29T07:41:39.619 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-03-29T07:41:39.619 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net 2024-03-29T07:41:39.619 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/bonding 2024-03-29T07:41:39.620 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/bonding/bonding.ko 2024-03-29T07:41:39.620 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa 2024-03-29T07:41:39.620 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/b53 2024-03-29T07:41:39.620 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/b53/b53_common.ko 2024-03-29T07:41:39.620 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/bcm-sf2.ko 2024-03-29T07:41:39.620 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/mv88e6060.ko 2024-03-29T07:41:39.620 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/mv88e6xxx 2024-03-29T07:41:39.620 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-03-29T07:41:39.620 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dummy.ko 2024-03-29T07:41:39.620 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/eql.ko 2024-03-29T07:41:39.620 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet 2024-03-29T07:41:39.620 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/3com 2024-03-29T07:41:39.620 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-03-29T07:41:39.620 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-03-29T07:41:39.621 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-03-29T07:41:39.621 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-03-29T07:41:39.621 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/8390 2024-03-29T07:41:39.621 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/8390/8390.ko 2024-03-29T07:41:39.621 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-03-29T07:41:39.621 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-03-29T07:41:39.621 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-03-29T07:41:39.621 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/adaptec 2024-03-29T07:41:39.621 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-03-29T07:41:39.621 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/agere 2024-03-29T07:41:39.621 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/agere/et131x.ko 2024-03-29T07:41:39.621 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/alteon 2024-03-29T07:41:39.621 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-03-29T07:41:39.621 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/altera 2024-03-29T07:41:39.621 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-03-29T07:41:39.622 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/amd 2024-03-29T07:41:39.622 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-03-29T07:41:39.622 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-03-29T07:41:39.622 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-03-29T07:41:39.622 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros 2024-03-29T07:41:39.622 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/alx 2024-03-29T07:41:39.622 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-03-29T07:41:39.622 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atl1c 2024-03-29T07:41:39.622 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-03-29T07:41:39.622 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atl1e 2024-03-29T07:41:39.622 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-03-29T07:41:39.622 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atlx 2024-03-29T07:41:39.622 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-03-29T07:41:39.622 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-03-29T07:41:39.623 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom 2024-03-29T07:41:39.623 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-03-29T07:41:39.623 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-03-29T07:41:39.623 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-03-29T07:41:39.623 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-03-29T07:41:39.623 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-03-29T07:41:39.623 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/genet 2024-03-29T07:41:39.623 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-03-29T07:41:39.623 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-03-29T07:41:39.623 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/brocade 2024-03-29T07:41:39.623 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/brocade/bna 2024-03-29T07:41:39.623 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-03-29T07:41:39.623 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio 2024-03-29T07:41:39.623 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb 2024-03-29T07:41:39.623 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-03-29T07:41:39.624 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-03-29T07:41:39.624 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-03-29T07:41:39.624 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-03-29T07:41:39.624 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-03-29T07:41:39.624 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-03-29T07:41:39.624 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-03-29T07:41:39.624 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-03-29T07:41:39.624 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-03-29T07:41:39.624 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/cisco 2024-03-29T07:41:39.624 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/cisco/enic 2024-03-29T07:41:39.624 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-03-29T07:41:39.624 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec 2024-03-29T07:41:39.624 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip 2024-03-29T07:41:39.624 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-03-29T07:41:39.624 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-03-29T07:41:39.625 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-03-29T07:41:39.625 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-03-29T07:41:39.625 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-03-29T07:41:39.625 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-03-29T07:41:39.625 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dlink 2024-03-29T07:41:39.625 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-03-29T07:41:39.625 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-03-29T07:41:39.625 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dnet.ko 2024-03-29T07:41:39.625 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/emulex 2024-03-29T07:41:39.625 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/emulex/benet 2024-03-29T07:41:39.625 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-03-29T07:41:39.625 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/ethoc.ko 2024-03-29T07:41:39.625 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/fealnx.ko 2024-03-29T07:41:39.625 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel 2024-03-29T07:41:39.626 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/e100.ko 2024-03-29T07:41:39.626 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/e1000 2024-03-29T07:41:39.626 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-03-29T07:41:39.626 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/e1000e 2024-03-29T07:41:39.626 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-03-29T07:41:39.626 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/fm10k 2024-03-29T07:41:39.626 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-03-29T07:41:39.626 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/i40e 2024-03-29T07:41:39.626 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-03-29T07:41:39.626 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/iavf 2024-03-29T07:41:39.626 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-03-29T07:41:39.626 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/igb 2024-03-29T07:41:39.626 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-03-29T07:41:39.626 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/igbvf 2024-03-29T07:41:39.627 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-03-29T07:41:39.627 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/ixgbe 2024-03-29T07:41:39.627 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-03-29T07:41:39.627 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/ixgbevf 2024-03-29T07:41:39.627 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-03-29T07:41:39.627 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/jme.ko 2024-03-29T07:41:39.627 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/marvell 2024-03-29T07:41:39.627 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-03-29T07:41:39.627 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/marvell/skge.ko 2024-03-29T07:41:39.627 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-03-29T07:41:39.627 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox 2024-03-29T07:41:39.627 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx4 2024-03-29T07:41:39.627 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-03-29T07:41:39.627 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-03-29T07:41:39.628 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx5 2024-03-29T07:41:39.628 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-03-29T07:41:39.628 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-03-29T07:41:39.628 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/micrel 2024-03-29T07:41:39.628 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-03-29T07:41:39.628 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/myricom 2024-03-29T07:41:39.628 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/myricom/myri10ge 2024-03-29T07:41:39.628 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-03-29T07:41:39.628 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/natsemi 2024-03-29T07:41:39.628 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-03-29T07:41:39.628 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-03-29T07:41:39.628 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/neterion 2024-03-29T07:41:39.628 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-03-29T07:41:39.628 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/nvidia 2024-03-29T07:41:39.628 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-03-29T07:41:39.629 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/packetengines 2024-03-29T07:41:39.629 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-03-29T07:41:39.629 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-03-29T07:41:39.629 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic 2024-03-29T07:41:39.629 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic/netxen 2024-03-29T07:41:39.629 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-03-29T07:41:39.629 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-03-29T07:41:39.629 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-03-29T07:41:39.629 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-03-29T07:41:39.629 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/rdc 2024-03-29T07:41:39.629 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-03-29T07:41:39.629 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/realtek 2024-03-29T07:41:39.629 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-03-29T07:41:39.629 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-03-29T07:41:39.629 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/realtek/atp.ko 2024-03-29T07:41:39.630 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-03-29T07:41:39.630 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/rocker 2024-03-29T07:41:39.630 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-03-29T07:41:39.630 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sfc 2024-03-29T07:41:39.630 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-03-29T07:41:39.630 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/silan 2024-03-29T07:41:39.630 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-03-29T07:41:39.630 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sis 2024-03-29T07:41:39.630 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sis/sis190.ko 2024-03-29T07:41:39.630 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sis/sis900.ko 2024-03-29T07:41:39.630 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/smsc 2024-03-29T07:41:39.630 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-03-29T07:41:39.630 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-03-29T07:41:39.630 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-03-29T07:41:39.631 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-03-29T07:41:39.631 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/stmicro 2024-03-29T07:41:39.631 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/stmicro/stmmac 2024-03-29T07:41:39.631 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-03-29T07:41:39.631 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-03-29T07:41:39.631 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sun 2024-03-29T07:41:39.631 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sun/cassini.ko 2024-03-29T07:41:39.631 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sun/niu.ko 2024-03-29T07:41:39.631 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sun/sungem.ko 2024-03-29T07:41:39.631 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-03-29T07:41:39.631 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/tehuti 2024-03-29T07:41:39.631 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-03-29T07:41:39.631 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/ti 2024-03-29T07:41:39.631 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/ti/tlan.ko 2024-03-29T07:41:39.631 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/via 2024-03-29T07:41:39.632 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-03-29T07:41:39.632 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-03-29T07:41:39.632 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/wiznet 2024-03-29T07:41:39.632 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-03-29T07:41:39.632 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-03-29T07:41:39.632 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/xircom 2024-03-29T07:41:39.632 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-03-29T07:41:39.632 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/geneve.ko 2024-03-29T07:41:39.632 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/hyperv 2024-03-29T07:41:39.632 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-03-29T07:41:39.632 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ieee802154 2024-03-29T07:41:39.632 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ieee802154/fakelb.ko 2024-03-29T07:41:39.632 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ifb.ko 2024-03-29T07:41:39.632 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ipvlan 2024-03-29T07:41:39.632 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ipvlan/ipvlan.ko 2024-03-29T07:41:39.633 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/macvlan.ko 2024-03-29T07:41:39.633 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/macvtap.ko 2024-03-29T07:41:39.633 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mdio 2024-03-29T07:41:39.633 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mdio.ko 2024-03-29T07:41:39.633 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-03-29T07:41:39.633 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-03-29T07:41:39.633 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mdio/mdio-regmap.ko 2024-03-29T07:41:39.633 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mii.ko 2024-03-29T07:41:39.633 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/net_failover.ko 2024-03-29T07:41:39.633 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/netconsole.ko 2024-03-29T07:41:39.633 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/nlmon.ko 2024-03-29T07:41:39.633 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/pcs 2024-03-29T07:41:39.633 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/pcs/pcs-lynx.ko 2024-03-29T07:41:39.633 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-03-29T07:41:39.633 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy 2024-03-29T07:41:39.634 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/amd.ko 2024-03-29T07:41:39.634 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-03-29T07:41:39.634 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-03-29T07:41:39.634 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/bcm7xxx.ko 2024-03-29T07:41:39.634 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/bcm87xx.ko 2024-03-29T07:41:39.634 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/broadcom.ko 2024-03-29T07:41:39.634 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/cicada.ko 2024-03-29T07:41:39.634 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/davicom.ko 2024-03-29T07:41:39.634 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/dp83640.ko 2024-03-29T07:41:39.634 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/et1011c.ko 2024-03-29T07:41:39.634 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/icplus.ko 2024-03-29T07:41:39.634 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/lxt.ko 2024-03-29T07:41:39.634 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/marvell.ko 2024-03-29T07:41:39.634 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/micrel.ko 2024-03-29T07:41:39.635 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/national.ko 2024-03-29T07:41:39.635 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/phylink.ko 2024-03-29T07:41:39.635 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/qsemi.ko 2024-03-29T07:41:39.635 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/realtek.ko 2024-03-29T07:41:39.635 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/smsc.ko 2024-03-29T07:41:39.635 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/ste10Xp.ko 2024-03-29T07:41:39.635 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/vitesse.ko 2024-03-29T07:41:39.635 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp 2024-03-29T07:41:39.635 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/bsd_comp.ko 2024-03-29T07:41:39.635 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/ppp_async.ko 2024-03-29T07:41:39.635 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/ppp_deflate.ko 2024-03-29T07:41:39.635 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/ppp_generic.ko 2024-03-29T07:41:39.635 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/ppp_mppe.ko 2024-03-29T07:41:39.635 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/ppp_synctty.ko 2024-03-29T07:41:39.635 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/pppoe.ko 2024-03-29T07:41:39.636 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/pppox.ko 2024-03-29T07:41:39.636 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/pptp.ko 2024-03-29T07:41:39.636 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/slip 2024-03-29T07:41:39.636 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/slip/slhc.ko 2024-03-29T07:41:39.636 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/slip/slip.ko 2024-03-29T07:41:39.636 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/sungem_phy.ko 2024-03-29T07:41:39.636 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/tap.ko 2024-03-29T07:41:39.636 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team 2024-03-29T07:41:39.636 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team.ko 2024-03-29T07:41:39.636 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team_mode_activebackup.ko 2024-03-29T07:41:39.636 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team_mode_broadcast.ko 2024-03-29T07:41:39.636 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-03-29T07:41:39.636 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team_mode_random.ko 2024-03-29T07:41:39.636 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-03-29T07:41:39.636 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/tun.ko 2024-03-29T07:41:39.637 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/veth.ko 2024-03-29T07:41:39.637 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/virtio_net.ko 2024-03-29T07:41:39.637 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/vmxnet3 2024-03-29T07:41:39.637 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-03-29T07:41:39.637 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/vxlan 2024-03-29T07:41:39.637 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/vxlan/vxlan.ko 2024-03-29T07:41:39.637 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/xen-netback 2024-03-29T07:41:39.637 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/xen-netback/xen-netback.ko 2024-03-29T07:41:39.637 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/xen-netfront.ko 2024-03-29T07:41:39.637 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ntb 2024-03-29T07:41:39.637 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ntb/ntb.ko 2024-03-29T07:41:39.637 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/nvme 2024-03-29T07:41:39.637 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/nvme/host 2024-03-29T07:41:39.637 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/nvme/host/nvme-core.ko 2024-03-29T07:41:39.637 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/nvme/host/nvme.ko 2024-03-29T07:41:39.637 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/parport 2024-03-29T07:41:39.638 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/parport/parport.ko 2024-03-29T07:41:39.638 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/parport/parport_cs.ko 2024-03-29T07:41:39.638 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/parport/parport_pc.ko 2024-03-29T07:41:39.638 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/parport/parport_serial.ko 2024-03-29T07:41:39.638 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci 2024-03-29T07:41:39.638 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci/hotplug 2024-03-29T07:41:39.638 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-03-29T07:41:39.638 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci/pcie 2024-03-29T07:41:39.638 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci/pcie/aer_inject.ko 2024-03-29T07:41:39.638 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci/xen-pcifront.ko 2024-03-29T07:41:39.638 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pcmcia 2024-03-29T07:41:39.638 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pcmcia/i82092.ko 2024-03-29T07:41:39.638 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pcmcia/pd6729.ko 2024-03-29T07:41:39.638 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pcmcia/yenta_socket.ko 2024-03-29T07:41:39.639 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pinctrl 2024-03-29T07:41:39.639 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pinctrl/intel 2024-03-29T07:41:39.639 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-03-29T07:41:39.639 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform 2024-03-29T07:41:39.639 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86 2024-03-29T07:41:39.639 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/acer-wmi.ko 2024-03-29T07:41:39.639 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/acerhdf.ko 2024-03-29T07:41:39.639 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/apple-gmux.ko 2024-03-29T07:41:39.639 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/asus-laptop.ko 2024-03-29T07:41:39.639 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-03-29T07:41:39.639 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/asus-wmi.ko 2024-03-29T07:41:39.639 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/classmate-laptop.ko 2024-03-29T07:41:39.639 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-03-29T07:41:39.639 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-03-29T07:41:39.639 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-03-29T07:41:39.639 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-03-29T07:41:39.640 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/hdaps.ko 2024-03-29T07:41:39.640 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/intel 2024-03-29T07:41:39.640 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-03-29T07:41:39.640 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/intel_ips.ko 2024-03-29T07:41:39.640 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/msi-wmi.ko 2024-03-29T07:41:39.640 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/mxm-wmi.ko 2024-03-29T07:41:39.640 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-03-29T07:41:39.640 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/samsung-laptop.ko 2024-03-29T07:41:39.640 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/samsung-q10.ko 2024-03-29T07:41:39.640 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/topstar-laptop.ko 2024-03-29T07:41:39.640 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-03-29T07:41:39.640 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/toshiba_haps.ko 2024-03-29T07:41:39.640 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/wmi-bmof.ko 2024-03-29T07:41:39.640 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/wmi.ko 2024-03-29T07:41:39.640 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/powercap 2024-03-29T07:41:39.641 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/powercap/intel_rapl_common.ko 2024-03-29T07:41:39.641 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/powercap/intel_rapl_msr.ko 2024-03-29T07:41:39.641 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps 2024-03-29T07:41:39.641 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps/clients 2024-03-29T07:41:39.641 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps/clients/pps-gpio.ko 2024-03-29T07:41:39.641 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps/clients/pps-ldisc.ko 2024-03-29T07:41:39.641 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps/clients/pps_parport.ko 2024-03-29T07:41:39.641 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps/pps_core.ko 2024-03-29T07:41:39.641 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ptp 2024-03-29T07:41:39.641 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ptp/ptp.ko 2024-03-29T07:41:39.641 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ptp/ptp_kvm.ko 2024-03-29T07:41:39.641 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc 2024-03-29T07:41:39.641 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-bq32k.ko 2024-03-29T07:41:39.641 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1286.ko 2024-03-29T07:41:39.642 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1307.ko 2024-03-29T07:41:39.642 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1374.ko 2024-03-29T07:41:39.642 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1511.ko 2024-03-29T07:41:39.642 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1553.ko 2024-03-29T07:41:39.642 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1672.ko 2024-03-29T07:41:39.642 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1742.ko 2024-03-29T07:41:39.642 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds2404.ko 2024-03-29T07:41:39.642 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds3232.ko 2024-03-29T07:41:39.642 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-em3027.ko 2024-03-29T07:41:39.642 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-fm3130.ko 2024-03-29T07:41:39.642 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-isl12022.ko 2024-03-29T07:41:39.642 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-isl1208.ko 2024-03-29T07:41:39.642 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-m41t80.ko 2024-03-29T07:41:39.642 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-m48t35.ko 2024-03-29T07:41:39.642 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-m48t59.ko 2024-03-29T07:41:39.643 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-max6900.ko 2024-03-29T07:41:39.643 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-msm6242.ko 2024-03-29T07:41:39.643 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-pcf2127.ko 2024-03-29T07:41:39.643 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-pcf85063.ko 2024-03-29T07:41:39.643 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-pcf8523.ko 2024-03-29T07:41:39.643 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-pcf8563.ko 2024-03-29T07:41:39.643 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-pcf8583.ko 2024-03-29T07:41:39.643 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-rp5c01.ko 2024-03-29T07:41:39.643 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-rs5c372.ko 2024-03-29T07:41:39.643 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-03-29T07:41:39.643 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-rx8025.ko 2024-03-29T07:41:39.643 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-rx8581.ko 2024-03-29T07:41:39.643 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-03-29T07:41:39.643 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-x1205.ko 2024-03-29T07:41:39.643 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi 2024-03-29T07:41:39.644 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/3w-9xxx.ko 2024-03-29T07:41:39.644 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/3w-sas.ko 2024-03-29T07:41:39.644 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/3w-xxxx.ko 2024-03-29T07:41:39.644 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/BusLogic.ko 2024-03-29T07:41:39.644 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/a100u2w.ko 2024-03-29T07:41:39.644 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/aacraid 2024-03-29T07:41:39.644 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/aacraid/aacraid.ko 2024-03-29T07:41:39.644 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/advansys.ko 2024-03-29T07:41:39.644 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/aic7xxx 2024-03-29T07:41:39.644 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-03-29T07:41:39.644 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-03-29T07:41:39.644 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/am53c974.ko 2024-03-29T07:41:39.644 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/arcmsr 2024-03-29T07:41:39.644 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-03-29T07:41:39.644 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/atp870u.ko 2024-03-29T07:41:39.645 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/be2iscsi 2024-03-29T07:41:39.645 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-03-29T07:41:39.645 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bfa 2024-03-29T07:41:39.645 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bfa/bfa.ko 2024-03-29T07:41:39.645 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bnx2fc 2024-03-29T07:41:39.645 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-03-29T07:41:39.645 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bnx2i 2024-03-29T07:41:39.645 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-03-29T07:41:39.645 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/ch.ko 2024-03-29T07:41:39.645 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/csiostor 2024-03-29T07:41:39.645 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/csiostor/csiostor.ko 2024-03-29T07:41:39.645 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi 2024-03-29T07:41:39.645 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi/cxgb3i 2024-03-29T07:41:39.645 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-03-29T07:41:39.645 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi/cxgb4i 2024-03-29T07:41:39.646 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-03-29T07:41:39.646 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-03-29T07:41:39.646 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/dc395x.ko 2024-03-29T07:41:39.646 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/device_handler 2024-03-29T07:41:39.646 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-03-29T07:41:39.646 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-03-29T07:41:39.646 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-03-29T07:41:39.646 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-03-29T07:41:39.646 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/dmx3191d.ko 2024-03-29T07:41:39.646 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/esas2r 2024-03-29T07:41:39.646 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/esas2r/esas2r.ko 2024-03-29T07:41:39.646 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/esp_scsi.ko 2024-03-29T07:41:39.646 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/fcoe 2024-03-29T07:41:39.647 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/fcoe/fcoe.ko 2024-03-29T07:41:39.647 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-03-29T07:41:39.647 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/fnic 2024-03-29T07:41:39.647 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/fnic/fnic.ko 2024-03-29T07:41:39.647 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/hpsa.ko 2024-03-29T07:41:39.647 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/hptiop.ko 2024-03-29T07:41:39.647 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/hv_storvsc.ko 2024-03-29T07:41:39.647 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/initio.ko 2024-03-29T07:41:39.647 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/ipr.ko 2024-03-29T07:41:39.647 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/ips.ko 2024-03-29T07:41:39.647 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/isci 2024-03-29T07:41:39.647 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/isci/isci.ko 2024-03-29T07:41:39.647 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-03-29T07:41:39.647 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/iscsi_tcp.ko 2024-03-29T07:41:39.647 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libfc 2024-03-29T07:41:39.648 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libfc/libfc.ko 2024-03-29T07:41:39.648 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libiscsi.ko 2024-03-29T07:41:39.648 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libiscsi_tcp.ko 2024-03-29T07:41:39.648 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libsas 2024-03-29T07:41:39.648 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libsas/libsas.ko 2024-03-29T07:41:39.648 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/lpfc 2024-03-29T07:41:39.648 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/lpfc/lpfc.ko 2024-03-29T07:41:39.648 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/megaraid 2024-03-29T07:41:39.648 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/megaraid.ko 2024-03-29T07:41:39.648 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-03-29T07:41:39.648 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-03-29T07:41:39.648 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-03-29T07:41:39.648 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/mpt3sas 2024-03-29T07:41:39.648 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-03-29T07:41:39.648 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/mvsas 2024-03-29T07:41:39.648 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/mvsas/mvsas.ko 2024-03-29T07:41:39.649 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/mvumi.ko 2024-03-29T07:41:39.649 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/pm8001 2024-03-29T07:41:39.649 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-03-29T07:41:39.649 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/pmcraid.ko 2024-03-29T07:41:39.649 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla1280.ko 2024-03-29T07:41:39.649 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla2xxx 2024-03-29T07:41:39.649 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-03-29T07:41:39.649 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-03-29T07:41:39.649 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla4xxx 2024-03-29T07:41:39.649 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-03-29T07:41:39.649 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/raid_class.ko 2024-03-29T07:41:39.649 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_debug.ko 2024-03-29T07:41:39.649 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_transport_fc.ko 2024-03-29T07:41:39.649 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-03-29T07:41:39.649 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_transport_sas.ko 2024-03-29T07:41:39.650 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_transport_spi.ko 2024-03-29T07:41:39.650 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_transport_srp.ko 2024-03-29T07:41:39.650 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/ses.ko 2024-03-29T07:41:39.650 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/st.ko 2024-03-29T07:41:39.650 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/stex.ko 2024-03-29T07:41:39.650 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/sym53c8xx_2 2024-03-29T07:41:39.650 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-03-29T07:41:39.650 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/virtio_scsi.ko 2024-03-29T07:41:39.650 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/vmw_pvscsi.ko 2024-03-29T07:41:39.650 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/wd719x.ko 2024-03-29T07:41:39.650 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ssb 2024-03-29T07:41:39.650 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ssb/ssb.ko 2024-03-29T07:41:39.650 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target 2024-03-29T07:41:39.650 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/iscsi 2024-03-29T07:41:39.651 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-03-29T07:41:39.651 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/loopback 2024-03-29T07:41:39.651 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/loopback/tcm_loop.ko 2024-03-29T07:41:39.651 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/target_core_file.ko 2024-03-29T07:41:39.651 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/target_core_iblock.ko 2024-03-29T07:41:39.651 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/target_core_mod.ko 2024-03-29T07:41:39.651 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/target_core_pscsi.ko 2024-03-29T07:41:39.651 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/target_core_user.ko 2024-03-29T07:41:39.651 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/tcm_fc 2024-03-29T07:41:39.651 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-03-29T07:41:39.651 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal 2024-03-29T07:41:39.651 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel 2024-03-29T07:41:39.651 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal 2024-03-29T07:41:39.651 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-03-29T07:41:39.652 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-03-29T07:41:39.652 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-03-29T07:41:39.652 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-03-29T07:41:39.652 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-03-29T07:41:39.652 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-03-29T07:41:39.652 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-03-29T07:41:39.652 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-03-29T07:41:39.652 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-03-29T07:41:39.652 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-03-29T07:41:39.652 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-03-29T07:41:39.652 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-03-29T07:41:39.652 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-03-29T07:41:39.652 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-03-29T07:41:39.652 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-03-29T07:41:39.652 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-03-29T07:41:39.653 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-03-29T07:41:39.653 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-03-29T07:41:39.653 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty 2024-03-29T07:41:39.653 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/ipwireless 2024-03-29T07:41:39.653 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-03-29T07:41:39.653 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/n_gsm.ko 2024-03-29T07:41:39.653 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/n_hdlc.ko 2024-03-29T07:41:39.653 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/nozomi.ko 2024-03-29T07:41:39.653 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial 2024-03-29T07:41:39.653 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/8250 2024-03-29T07:41:39.653 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-03-29T07:41:39.653 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/arc_uart.ko 2024-03-29T07:41:39.653 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/jsm 2024-03-29T07:41:39.653 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/jsm/jsm.ko 2024-03-29T07:41:39.653 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ufs 2024-03-29T07:41:39.654 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ufs/core 2024-03-29T07:41:39.654 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ufs/core/ufshcd-core.ko 2024-03-29T07:41:39.654 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ufs/host 2024-03-29T07:41:39.654 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-03-29T07:41:39.654 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio 2024-03-29T07:41:39.654 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio.ko 2024-03-29T07:41:39.654 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio_aec.ko 2024-03-29T07:41:39.654 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio_cif.ko 2024-03-29T07:41:39.654 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio_pci_generic.ko 2024-03-29T07:41:39.654 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio_sercos3.ko 2024-03-29T07:41:39.654 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb 2024-03-29T07:41:39.654 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/class 2024-03-29T07:41:39.654 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/class/cdc-acm.ko 2024-03-29T07:41:39.654 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/class/cdc-wdm.ko 2024-03-29T07:41:39.654 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/class/usblp.ko 2024-03-29T07:41:39.655 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/class/usbtmc.ko 2024-03-29T07:41:39.655 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/host 2024-03-29T07:41:39.655 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/host/sl811-hcd.ko 2024-03-29T07:41:39.655 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/image 2024-03-29T07:41:39.655 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/image/mdc800.ko 2024-03-29T07:41:39.655 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/image/microtek.ko 2024-03-29T07:41:39.655 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc 2024-03-29T07:41:39.655 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/adutux.ko 2024-03-29T07:41:39.655 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/appledisplay.ko 2024-03-29T07:41:39.655 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/emi26.ko 2024-03-29T07:41:39.655 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/emi62.ko 2024-03-29T07:41:39.655 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/ezusb.ko 2024-03-29T07:41:39.655 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/idmouse.ko 2024-03-29T07:41:39.655 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/iowarrior.ko 2024-03-29T07:41:39.655 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/isight_firmware.ko 2024-03-29T07:41:39.656 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/ldusb.ko 2024-03-29T07:41:39.656 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/legousbtower.ko 2024-03-29T07:41:39.656 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/sisusbvga 2024-03-29T07:41:39.656 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-03-29T07:41:39.656 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/trancevibrator.ko 2024-03-29T07:41:39.656 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/usb3503.ko 2024-03-29T07:41:39.656 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/usblcd.ko 2024-03-29T07:41:39.656 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/usbsevseg.ko 2024-03-29T07:41:39.656 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/uss720.ko 2024-03-29T07:41:39.656 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/yurex.ko 2024-03-29T07:41:39.656 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/phy 2024-03-29T07:41:39.656 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/phy/phy-generic.ko 2024-03-29T07:41:39.656 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial 2024-03-29T07:41:39.656 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/aircable.ko 2024-03-29T07:41:39.656 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ark3116.ko 2024-03-29T07:41:39.657 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/belkin_sa.ko 2024-03-29T07:41:39.657 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ch341.ko 2024-03-29T07:41:39.657 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/cp210x.ko 2024-03-29T07:41:39.657 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/cyberjack.ko 2024-03-29T07:41:39.657 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/cypress_m8.ko 2024-03-29T07:41:39.657 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/digi_acceleport.ko 2024-03-29T07:41:39.657 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/empeg.ko 2024-03-29T07:41:39.657 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ftdi_sio.ko 2024-03-29T07:41:39.657 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/garmin_gps.ko 2024-03-29T07:41:39.657 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/io_edgeport.ko 2024-03-29T07:41:39.657 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/io_ti.ko 2024-03-29T07:41:39.657 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ipaq.ko 2024-03-29T07:41:39.657 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ipw.ko 2024-03-29T07:41:39.657 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ir-usb.ko 2024-03-29T07:41:39.657 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-03-29T07:41:39.658 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/keyspan.ko 2024-03-29T07:41:39.658 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/keyspan_pda.ko 2024-03-29T07:41:39.658 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/kl5kusb105.ko 2024-03-29T07:41:39.658 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/kobil_sct.ko 2024-03-29T07:41:39.658 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/mct_u232.ko 2024-03-29T07:41:39.658 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/mos7720.ko 2024-03-29T07:41:39.658 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/mos7840.ko 2024-03-29T07:41:39.658 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/navman.ko 2024-03-29T07:41:39.658 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/omninet.ko 2024-03-29T07:41:39.658 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/opticon.ko 2024-03-29T07:41:39.658 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/option.ko 2024-03-29T07:41:39.658 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/oti6858.ko 2024-03-29T07:41:39.658 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/pl2303.ko 2024-03-29T07:41:39.658 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/qcaux.ko 2024-03-29T07:41:39.658 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/qcserial.ko 2024-03-29T07:41:39.659 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/quatech2.ko 2024-03-29T07:41:39.659 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/safe_serial.ko 2024-03-29T07:41:39.659 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/sierra.ko 2024-03-29T07:41:39.659 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/spcp8x5.ko 2024-03-29T07:41:39.659 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ssu100.ko 2024-03-29T07:41:39.659 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/symbolserial.ko 2024-03-29T07:41:39.659 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-03-29T07:41:39.659 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-03-29T07:41:39.659 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/usb_debug.ko 2024-03-29T07:41:39.659 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/usb_wwan.ko 2024-03-29T07:41:39.659 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/visor.ko 2024-03-29T07:41:39.659 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/whiteheat.ko 2024-03-29T07:41:39.659 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/xsens_mt.ko 2024-03-29T07:41:39.659 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage 2024-03-29T07:41:39.659 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/uas.ko 2024-03-29T07:41:39.660 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-alauda.ko 2024-03-29T07:41:39.660 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-cypress.ko 2024-03-29T07:41:39.660 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-datafab.ko 2024-03-29T07:41:39.660 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-03-29T07:41:39.660 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-freecom.ko 2024-03-29T07:41:39.660 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-isd200.ko 2024-03-29T07:41:39.660 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-03-29T07:41:39.660 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-karma.ko 2024-03-29T07:41:39.660 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-onetouch.ko 2024-03-29T07:41:39.660 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-realtek.ko 2024-03-29T07:41:39.660 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-sddr09.ko 2024-03-29T07:41:39.660 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-sddr55.ko 2024-03-29T07:41:39.660 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-usbat.ko 2024-03-29T07:41:39.660 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/usb-storage.ko 2024-03-29T07:41:39.660 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/usbip 2024-03-29T07:41:39.661 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/usbip/usbip-core.ko 2024-03-29T07:41:39.661 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/usbip/usbip-host.ko 2024-03-29T07:41:39.661 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-03-29T07:41:39.661 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio 2024-03-29T07:41:39.661 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio/pci 2024-03-29T07:41:39.661 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-03-29T07:41:39.661 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio/pci/vfio-pci.ko 2024-03-29T07:41:39.661 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio/vfio.ko 2024-03-29T07:41:39.661 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-03-29T07:41:39.661 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vhost 2024-03-29T07:41:39.661 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vhost/vhost.ko 2024-03-29T07:41:39.661 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vhost/vhost_iotlb.ko 2024-03-29T07:41:39.661 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vhost/vhost_net.ko 2024-03-29T07:41:39.661 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vhost/vhost_scsi.ko 2024-03-29T07:41:39.661 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video 2024-03-29T07:41:39.662 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight 2024-03-29T07:41:39.662 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight/apple_bl.ko 2024-03-29T07:41:39.662 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight/lcd.ko 2024-03-29T07:41:39.662 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight/platform_lcd.ko 2024-03-29T07:41:39.662 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio 2024-03-29T07:41:39.662 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio.ko 2024-03-29T07:41:39.662 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_balloon.ko 2024-03-29T07:41:39.662 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_mmio.ko 2024-03-29T07:41:39.662 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_pci.ko 2024-03-29T07:41:39.662 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-03-29T07:41:39.662 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-03-29T07:41:39.662 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_ring.ko 2024-03-29T07:41:39.662 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1 2024-03-29T07:41:39.662 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/masters 2024-03-29T07:41:39.662 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/masters/ds2482.ko 2024-03-29T07:41:39.663 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/masters/ds2490.ko 2024-03-29T07:41:39.663 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves 2024-03-29T07:41:39.663 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-03-29T07:41:39.663 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-03-29T07:41:39.663 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-03-29T07:41:39.663 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-03-29T07:41:39.663 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-03-29T07:41:39.663 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-03-29T07:41:39.663 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-03-29T07:41:39.663 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-03-29T07:41:39.663 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-03-29T07:41:39.663 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_smem.ko 2024-03-29T07:41:39.663 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_therm.ko 2024-03-29T07:41:39.663 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/wire.ko 2024-03-29T07:41:39.663 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog 2024-03-29T07:41:39.664 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/alim1535_wdt.ko 2024-03-29T07:41:39.664 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/alim7101_wdt.ko 2024-03-29T07:41:39.664 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/f71808e_wdt.ko 2024-03-29T07:41:39.664 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/hpwdt.ko 2024-03-29T07:41:39.664 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/i6300esb.ko 2024-03-29T07:41:39.664 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-03-29T07:41:39.664 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/iTCO_wdt.ko 2024-03-29T07:41:39.664 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/ib700wdt.ko 2024-03-29T07:41:39.664 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/ibmasr.ko 2024-03-29T07:41:39.664 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-03-29T07:41:39.664 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/it8712f_wdt.ko 2024-03-29T07:41:39.664 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/it87_wdt.ko 2024-03-29T07:41:39.664 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/machzwd.ko 2024-03-29T07:41:39.664 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/nv_tco.ko 2024-03-29T07:41:39.664 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/pcwd_pci.ko 2024-03-29T07:41:39.664 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/pcwd_usb.ko 2024-03-29T07:41:39.665 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-03-29T07:41:39.665 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/sch311x_wdt.ko 2024-03-29T07:41:39.665 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/softdog.ko 2024-03-29T07:41:39.665 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/sp5100_tco.ko 2024-03-29T07:41:39.665 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/via_wdt.ko 2024-03-29T07:41:39.665 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-03-29T07:41:39.665 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/w83877f_wdt.ko 2024-03-29T07:41:39.665 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/w83977f_wdt.ko 2024-03-29T07:41:39.665 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/wdt_pci.ko 2024-03-29T07:41:39.665 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/xen_wdt.ko 2024-03-29T07:41:39.665 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen 2024-03-29T07:41:39.665 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-acpi-processor.ko 2024-03-29T07:41:39.665 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-evtchn.ko 2024-03-29T07:41:39.665 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-gntalloc.ko 2024-03-29T07:41:39.666 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-gntdev.ko 2024-03-29T07:41:39.666 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-pciback 2024-03-29T07:41:39.666 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-03-29T07:41:39.666 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-privcmd.ko 2024-03-29T07:41:39.666 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xenfs 2024-03-29T07:41:39.666 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xenfs/xenfs.ko 2024-03-29T07:41:39.666 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs 2024-03-29T07:41:39.666 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/9p 2024-03-29T07:41:39.666 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/9p/9p.ko 2024-03-29T07:41:39.666 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/affs 2024-03-29T07:41:39.666 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/affs/affs.ko 2024-03-29T07:41:39.666 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/befs 2024-03-29T07:41:39.666 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/befs/befs.ko 2024-03-29T07:41:39.666 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/binfmt_misc.ko 2024-03-29T07:41:39.667 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/btrfs 2024-03-29T07:41:39.667 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/btrfs/btrfs.ko 2024-03-29T07:41:39.667 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/cachefiles 2024-03-29T07:41:39.667 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/cachefiles/cachefiles.ko 2024-03-29T07:41:39.667 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ceph 2024-03-29T07:41:39.667 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ceph/ceph.ko 2024-03-29T07:41:39.667 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/coda 2024-03-29T07:41:39.667 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/coda/coda.ko 2024-03-29T07:41:39.667 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/cramfs 2024-03-29T07:41:39.667 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/cramfs/cramfs.ko 2024-03-29T07:41:39.667 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/dlm 2024-03-29T07:41:39.667 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/dlm/dlm.ko 2024-03-29T07:41:39.667 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ecryptfs 2024-03-29T07:41:39.667 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ecryptfs/ecryptfs.ko 2024-03-29T07:41:39.667 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/f2fs 2024-03-29T07:41:39.668 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/f2fs/f2fs.ko 2024-03-29T07:41:39.668 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fat 2024-03-29T07:41:39.668 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fat/fat.ko 2024-03-29T07:41:39.668 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fat/msdos.ko 2024-03-29T07:41:39.668 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fat/vfat.ko 2024-03-29T07:41:39.668 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fuse 2024-03-29T07:41:39.668 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fuse/cuse.ko 2024-03-29T07:41:39.668 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fuse/fuse.ko 2024-03-29T07:41:39.668 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/gfs2 2024-03-29T07:41:39.668 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/gfs2/gfs2.ko 2024-03-29T07:41:39.668 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/hfs 2024-03-29T07:41:39.668 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/hfs/hfs.ko 2024-03-29T07:41:39.668 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/hfsplus 2024-03-29T07:41:39.668 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/hfsplus/hfsplus.ko 2024-03-29T07:41:39.668 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/isofs 2024-03-29T07:41:39.669 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/isofs/isofs.ko 2024-03-29T07:41:39.669 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/jfs 2024-03-29T07:41:39.669 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/jfs/jfs.ko 2024-03-29T07:41:39.669 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/lockd 2024-03-29T07:41:39.669 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/lockd/lockd.ko 2024-03-29T07:41:39.669 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/minix 2024-03-29T07:41:39.669 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/minix/minix.ko 2024-03-29T07:41:39.669 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/netfs 2024-03-29T07:41:39.669 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/netfs/netfs.ko 2024-03-29T07:41:39.669 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs 2024-03-29T07:41:39.669 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/blocklayout 2024-03-29T07:41:39.669 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-03-29T07:41:39.669 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/filelayout 2024-03-29T07:41:39.669 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-03-29T07:41:39.669 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/flexfilelayout 2024-03-29T07:41:39.670 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-03-29T07:41:39.670 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/nfs.ko 2024-03-29T07:41:39.670 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/nfsv3.ko 2024-03-29T07:41:39.670 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/nfsv4.ko 2024-03-29T07:41:39.670 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs_common 2024-03-29T07:41:39.670 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs_common/grace.ko 2024-03-29T07:41:39.670 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs_common/nfs_acl.ko 2024-03-29T07:41:39.670 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfsd 2024-03-29T07:41:39.670 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfsd/nfsd.ko 2024-03-29T07:41:39.670 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nilfs2 2024-03-29T07:41:39.670 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nilfs2/nilfs2.ko 2024-03-29T07:41:39.670 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls 2024-03-29T07:41:39.670 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-celtic.ko 2024-03-29T07:41:39.670 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-centeuro.ko 2024-03-29T07:41:39.670 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-croatian.ko 2024-03-29T07:41:39.671 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-cyrillic.ko 2024-03-29T07:41:39.671 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-gaelic.ko 2024-03-29T07:41:39.671 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-greek.ko 2024-03-29T07:41:39.671 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-iceland.ko 2024-03-29T07:41:39.671 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-inuit.ko 2024-03-29T07:41:39.671 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-roman.ko 2024-03-29T07:41:39.671 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-romanian.ko 2024-03-29T07:41:39.671 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-turkish.ko 2024-03-29T07:41:39.671 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp1250.ko 2024-03-29T07:41:39.671 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp1251.ko 2024-03-29T07:41:39.671 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp1255.ko 2024-03-29T07:41:39.671 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp737.ko 2024-03-29T07:41:39.671 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp775.ko 2024-03-29T07:41:39.671 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp850.ko 2024-03-29T07:41:39.671 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp852.ko 2024-03-29T07:41:39.672 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp855.ko 2024-03-29T07:41:39.672 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp857.ko 2024-03-29T07:41:39.672 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp860.ko 2024-03-29T07:41:39.672 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp861.ko 2024-03-29T07:41:39.672 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp862.ko 2024-03-29T07:41:39.672 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp863.ko 2024-03-29T07:41:39.672 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp864.ko 2024-03-29T07:41:39.672 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp865.ko 2024-03-29T07:41:39.672 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp866.ko 2024-03-29T07:41:39.672 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp869.ko 2024-03-29T07:41:39.672 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp874.ko 2024-03-29T07:41:39.672 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp932.ko 2024-03-29T07:41:39.672 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp936.ko 2024-03-29T07:41:39.672 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp949.ko 2024-03-29T07:41:39.672 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp950.ko 2024-03-29T07:41:39.673 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_euc-jp.ko 2024-03-29T07:41:39.673 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-1.ko 2024-03-29T07:41:39.673 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-13.ko 2024-03-29T07:41:39.673 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-14.ko 2024-03-29T07:41:39.673 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-15.ko 2024-03-29T07:41:39.673 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-2.ko 2024-03-29T07:41:39.673 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-3.ko 2024-03-29T07:41:39.673 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-4.ko 2024-03-29T07:41:39.673 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-5.ko 2024-03-29T07:41:39.673 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-6.ko 2024-03-29T07:41:39.673 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-7.ko 2024-03-29T07:41:39.673 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-9.ko 2024-03-29T07:41:39.673 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_koi8-r.ko 2024-03-29T07:41:39.673 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_koi8-ru.ko 2024-03-29T07:41:39.673 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_koi8-u.ko 2024-03-29T07:41:39.674 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_ucs2_utils.ko 2024-03-29T07:41:39.674 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_utf8.ko 2024-03-29T07:41:39.674 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2 2024-03-29T07:41:39.674 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/cluster 2024-03-29T07:41:39.674 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-03-29T07:41:39.674 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/dlm 2024-03-29T07:41:39.674 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-03-29T07:41:39.674 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/dlmfs 2024-03-29T07:41:39.674 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-03-29T07:41:39.674 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/ocfs2.ko 2024-03-29T07:41:39.674 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-03-29T07:41:39.674 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-03-29T07:41:39.674 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-03-29T07:41:39.674 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/overlayfs 2024-03-29T07:41:39.675 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/overlayfs/overlay.ko 2024-03-29T07:41:39.677 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/pstore 2024-03-29T07:41:39.678 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/pstore/ramoops.ko 2024-03-29T07:41:39.678 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/reiserfs 2024-03-29T07:41:39.678 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/reiserfs/reiserfs.ko 2024-03-29T07:41:39.678 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/romfs 2024-03-29T07:41:39.678 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/romfs/romfs.ko 2024-03-29T07:41:39.678 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb 2024-03-29T07:41:39.678 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb/client 2024-03-29T07:41:39.678 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb/client/cifs.ko 2024-03-29T07:41:39.678 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb/common 2024-03-29T07:41:39.678 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb/common/cifs_arc4.ko 2024-03-29T07:41:39.678 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb/common/cifs_md4.ko 2024-03-29T07:41:39.678 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/squashfs 2024-03-29T07:41:39.678 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/squashfs/squashfs.ko 2024-03-29T07:41:39.678 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/sysv 2024-03-29T07:41:39.679 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/sysv/sysv.ko 2024-03-29T07:41:39.679 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/udf 2024-03-29T07:41:39.679 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/udf/udf.ko 2024-03-29T07:41:39.679 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ufs 2024-03-29T07:41:39.679 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ufs/ufs.ko 2024-03-29T07:41:39.679 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/xfs 2024-03-29T07:41:39.679 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/xfs/xfs.ko 2024-03-29T07:41:39.679 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel 2024-03-29T07:41:39.679 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel/rcu 2024-03-29T07:41:39.679 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel/rcu/rcutorture.ko 2024-03-29T07:41:39.679 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel/torture.ko 2024-03-29T07:41:39.679 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel/trace 2024-03-29T07:41:39.679 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-03-29T07:41:39.679 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib 2024-03-29T07:41:39.679 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/asn1_encoder.ko 2024-03-29T07:41:39.680 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/crc-itu-t.ko 2024-03-29T07:41:39.680 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/crc8.ko 2024-03-29T07:41:39.680 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/crypto 2024-03-29T07:41:39.680 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/crypto/libarc4.ko 2024-03-29T07:41:39.680 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/crypto/libdes.ko 2024-03-29T07:41:39.680 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/libcrc32c.ko 2024-03-29T07:41:39.680 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/lru_cache.ko 2024-03-29T07:41:39.680 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/lz4 2024-03-29T07:41:39.680 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/lz4/lz4_compress.ko 2024-03-29T07:41:39.680 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/lz4/lz4hc_compress.ko 2024-03-29T07:41:39.680 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/math 2024-03-29T07:41:39.680 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/math/cordic.ko 2024-03-29T07:41:39.680 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/raid6 2024-03-29T07:41:39.680 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/raid6/raid6_pq.ko 2024-03-29T07:41:39.680 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/reed_solomon 2024-03-29T07:41:39.681 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/reed_solomon/reed_solomon.ko 2024-03-29T07:41:39.681 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/ts_bm.ko 2024-03-29T07:41:39.681 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/ts_fsm.ko 2024-03-29T07:41:39.681 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/ts_kmp.ko 2024-03-29T07:41:39.681 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/zstd 2024-03-29T07:41:39.681 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/zstd/zstd_compress.ko 2024-03-29T07:41:39.681 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/mm 2024-03-29T07:41:39.681 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/mm/hwpoison-inject.ko 2024-03-29T07:41:39.681 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net 2024-03-29T07:41:39.681 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan 2024-03-29T07:41:39.681 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/6lowpan.ko 2024-03-29T07:41:39.681 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_dest.ko 2024-03-29T07:41:39.681 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_fragment.ko 2024-03-29T07:41:39.681 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_hop.ko 2024-03-29T07:41:39.681 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_ipv6.ko 2024-03-29T07:41:39.682 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_mobility.ko 2024-03-29T07:41:39.682 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_routing.ko 2024-03-29T07:41:39.682 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_udp.ko 2024-03-29T07:41:39.682 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802 2024-03-29T07:41:39.682 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802/garp.ko 2024-03-29T07:41:39.682 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802/mrp.ko 2024-03-29T07:41:39.682 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802/p8022.ko 2024-03-29T07:41:39.682 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802/psnap.ko 2024-03-29T07:41:39.682 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802/stp.ko 2024-03-29T07:41:39.682 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/8021q 2024-03-29T07:41:39.682 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/8021q/8021q.ko 2024-03-29T07:41:39.682 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/9p 2024-03-29T07:41:39.682 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/9p/9pnet.ko 2024-03-29T07:41:39.682 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/9p/9pnet_fd.ko 2024-03-29T07:41:39.682 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/9p/9pnet_rdma.ko 2024-03-29T07:41:39.683 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/9p/9pnet_virtio.ko 2024-03-29T07:41:39.683 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/batman-adv 2024-03-29T07:41:39.683 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/batman-adv/batman-adv.ko 2024-03-29T07:41:39.683 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge 2024-03-29T07:41:39.683 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/br_netfilter.ko 2024-03-29T07:41:39.683 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/bridge.ko 2024-03-29T07:41:39.683 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter 2024-03-29T07:41:39.683 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-03-29T07:41:39.683 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_among.ko 2024-03-29T07:41:39.683 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_arp.ko 2024-03-29T07:41:39.683 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-03-29T07:41:39.683 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-03-29T07:41:39.683 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_ip.ko 2024-03-29T07:41:39.683 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-03-29T07:41:39.683 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_limit.ko 2024-03-29T07:41:39.683 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_log.ko 2024-03-29T07:41:39.684 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_mark.ko 2024-03-29T07:41:39.684 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-03-29T07:41:39.684 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-03-29T07:41:39.684 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-03-29T07:41:39.684 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-03-29T07:41:39.684 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_snat.ko 2024-03-29T07:41:39.684 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_stp.ko 2024-03-29T07:41:39.684 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-03-29T07:41:39.684 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-03-29T07:41:39.684 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-03-29T07:41:39.684 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-03-29T07:41:39.684 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebtables.ko 2024-03-29T07:41:39.684 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-03-29T07:41:39.684 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-03-29T07:41:39.684 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-03-29T07:41:39.685 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ceph 2024-03-29T07:41:39.685 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ceph/libceph.ko 2024-03-29T07:41:39.685 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/core 2024-03-29T07:41:39.685 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/core/failover.ko 2024-03-29T07:41:39.685 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/core/pktgen.ko 2024-03-29T07:41:39.685 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dccp 2024-03-29T07:41:39.685 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dccp/dccp.ko 2024-03-29T07:41:39.685 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dccp/dccp_diag.ko 2024-03-29T07:41:39.685 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dccp/dccp_ipv4.ko 2024-03-29T07:41:39.685 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dccp/dccp_ipv6.ko 2024-03-29T07:41:39.685 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dns_resolver 2024-03-29T07:41:39.685 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dns_resolver/dns_resolver.ko 2024-03-29T07:41:39.685 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa 2024-03-29T07:41:39.685 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa/dsa_core.ko 2024-03-29T07:41:39.685 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa/tag_brcm.ko 2024-03-29T07:41:39.686 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa/tag_dsa.ko 2024-03-29T07:41:39.686 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa/tag_none.ko 2024-03-29T07:41:39.686 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa/tag_trailer.ko 2024-03-29T07:41:39.686 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ieee802154 2024-03-29T07:41:39.686 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ieee802154/6lowpan 2024-03-29T07:41:39.686 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-03-29T07:41:39.686 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ieee802154/ieee802154.ko 2024-03-29T07:41:39.686 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ieee802154/ieee802154_socket.ko 2024-03-29T07:41:39.686 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4 2024-03-29T07:41:39.686 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ah4.ko 2024-03-29T07:41:39.686 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/esp4.ko 2024-03-29T07:41:39.686 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/fou.ko 2024-03-29T07:41:39.686 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/gre.ko 2024-03-29T07:41:39.686 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/inet_diag.ko 2024-03-29T07:41:39.687 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ip_gre.ko 2024-03-29T07:41:39.687 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ip_tunnel.ko 2024-03-29T07:41:39.687 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ip_vti.ko 2024-03-29T07:41:39.687 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ipcomp.ko 2024-03-29T07:41:39.687 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ipip.ko 2024-03-29T07:41:39.687 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter 2024-03-29T07:41:39.687 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/arp_tables.ko 2024-03-29T07:41:39.687 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-03-29T07:41:39.687 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-03-29T07:41:39.687 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-03-29T07:41:39.687 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-03-29T07:41:39.687 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-03-29T07:41:39.687 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-03-29T07:41:39.687 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-03-29T07:41:39.688 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-03-29T07:41:39.688 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-03-29T07:41:39.688 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/iptable_security.ko 2024-03-29T07:41:39.688 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-03-29T07:41:39.688 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-03-29T07:41:39.688 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-03-29T07:41:39.688 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-03-29T07:41:39.688 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-03-29T07:41:39.688 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-03-29T07:41:39.688 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-03-29T07:41:39.688 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-03-29T07:41:39.688 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-03-29T07:41:39.688 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-03-29T07:41:39.688 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_bic.ko 2024-03-29T07:41:39.688 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_dctcp.ko 2024-03-29T07:41:39.689 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_diag.ko 2024-03-29T07:41:39.689 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_highspeed.ko 2024-03-29T07:41:39.689 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_htcp.ko 2024-03-29T07:41:39.689 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_hybla.ko 2024-03-29T07:41:39.689 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_illinois.ko 2024-03-29T07:41:39.689 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_lp.ko 2024-03-29T07:41:39.689 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_scalable.ko 2024-03-29T07:41:39.689 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_vegas.ko 2024-03-29T07:41:39.689 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_veno.ko 2024-03-29T07:41:39.689 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_westwood.ko 2024-03-29T07:41:39.689 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_yeah.ko 2024-03-29T07:41:39.689 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tunnel4.ko 2024-03-29T07:41:39.689 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/udp_diag.ko 2024-03-29T07:41:39.689 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/udp_tunnel.ko 2024-03-29T07:41:39.689 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/xfrm4_tunnel.ko 2024-03-29T07:41:39.689 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6 2024-03-29T07:41:39.690 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/ah6.ko 2024-03-29T07:41:39.690 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/esp6.ko 2024-03-29T07:41:39.690 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/fou6.ko 2024-03-29T07:41:39.690 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/ip6_tunnel.ko 2024-03-29T07:41:39.690 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-03-29T07:41:39.690 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/ip6_vti.ko 2024-03-29T07:41:39.690 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/ipcomp6.ko 2024-03-29T07:41:39.690 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter 2024-03-29T07:41:39.690 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-03-29T07:41:39.690 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-03-29T07:41:39.690 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-03-29T07:41:39.690 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-03-29T07:41:39.690 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-03-29T07:41:39.690 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-03-29T07:41:39.690 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-03-29T07:41:39.691 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-03-29T07:41:39.691 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-03-29T07:41:39.691 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-03-29T07:41:39.691 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-03-29T07:41:39.691 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-03-29T07:41:39.691 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-03-29T07:41:39.691 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-03-29T07:41:39.691 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-03-29T07:41:39.691 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-03-29T07:41:39.691 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-03-29T07:41:39.691 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-03-29T07:41:39.691 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-03-29T07:41:39.691 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-03-29T07:41:39.691 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-03-29T07:41:39.691 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-03-29T07:41:39.692 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-03-29T07:41:39.692 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-03-29T07:41:39.692 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/sit.ko 2024-03-29T07:41:39.692 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/tunnel6.ko 2024-03-29T07:41:39.692 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/xfrm6_tunnel.ko 2024-03-29T07:41:39.692 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/key 2024-03-29T07:41:39.692 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/key/af_key.ko 2024-03-29T07:41:39.692 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp 2024-03-29T07:41:39.692 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_core.ko 2024-03-29T07:41:39.692 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_debugfs.ko 2024-03-29T07:41:39.692 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_eth.ko 2024-03-29T07:41:39.692 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_ip.ko 2024-03-29T07:41:39.692 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_ip6.ko 2024-03-29T07:41:39.692 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_netlink.ko 2024-03-29T07:41:39.692 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_ppp.ko 2024-03-29T07:41:39.693 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/llc 2024-03-29T07:41:39.693 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/llc/llc.ko 2024-03-29T07:41:39.693 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/mac802154 2024-03-29T07:41:39.693 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/mac802154/mac802154.ko 2024-03-29T07:41:39.693 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/mpls 2024-03-29T07:41:39.693 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/mpls/mpls_gso.ko 2024-03-29T07:41:39.693 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter 2024-03-29T07:41:39.693 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset 2024-03-29T07:41:39.693 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set.ko 2024-03-29T07:41:39.693 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-03-29T07:41:39.693 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-03-29T07:41:39.693 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-03-29T07:41:39.693 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-03-29T07:41:39.693 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-03-29T07:41:39.693 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-03-29T07:41:39.694 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-03-29T07:41:39.694 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-03-29T07:41:39.694 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-03-29T07:41:39.694 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-03-29T07:41:39.694 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-03-29T07:41:39.694 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-03-29T07:41:39.694 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-03-29T07:41:39.694 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-03-29T07:41:39.694 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-03-29T07:41:39.694 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs 2024-03-29T07:41:39.694 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs.ko 2024-03-29T07:41:39.694 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-03-29T07:41:39.694 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-03-29T07:41:39.694 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-03-29T07:41:39.694 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-03-29T07:41:39.695 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-03-29T07:41:39.695 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-03-29T07:41:39.695 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-03-29T07:41:39.695 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-03-29T07:41:39.695 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-03-29T07:41:39.695 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-03-29T07:41:39.695 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-03-29T07:41:39.695 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-03-29T07:41:39.695 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-03-29T07:41:39.695 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conncount.ko 2024-03-29T07:41:39.695 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack.ko 2024-03-29T07:41:39.695 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-03-29T07:41:39.695 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-03-29T07:41:39.695 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-03-29T07:41:39.695 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_h323.ko 2024-03-29T07:41:39.696 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_irc.ko 2024-03-29T07:41:39.696 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-03-29T07:41:39.696 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-03-29T07:41:39.696 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-03-29T07:41:39.696 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_sane.ko 2024-03-29T07:41:39.696 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_sip.ko 2024-03-29T07:41:39.696 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-03-29T07:41:39.696 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-03-29T07:41:39.696 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_dup_netdev.ko 2024-03-29T07:41:39.696 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_flow_table.ko 2024-03-29T07:41:39.696 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_flow_table_inet.ko 2024-03-29T07:41:39.696 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_log_syslog.ko 2024-03-29T07:41:39.696 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat.ko 2024-03-29T07:41:39.696 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat_amanda.ko 2024-03-29T07:41:39.696 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat_ftp.ko 2024-03-29T07:41:39.697 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat_irc.ko 2024-03-29T07:41:39.697 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat_sip.ko 2024-03-29T07:41:39.697 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat_tftp.ko 2024-03-29T07:41:39.697 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_synproxy_core.ko 2024-03-29T07:41:39.697 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_tables.ko 2024-03-29T07:41:39.697 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nfnetlink.ko 2024-03-29T07:41:39.697 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nfnetlink_acct.ko 2024-03-29T07:41:39.697 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nfnetlink_log.ko 2024-03-29T07:41:39.697 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nfnetlink_osf.ko 2024-03-29T07:41:39.697 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nfnetlink_queue.ko 2024-03-29T07:41:39.697 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_chain_nat.ko 2024-03-29T07:41:39.697 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_compat.ko 2024-03-29T07:41:39.697 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_ct.ko 2024-03-29T07:41:39.697 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_dup_netdev.ko 2024-03-29T07:41:39.697 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_fib.ko 2024-03-29T07:41:39.698 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_fwd_netdev.ko 2024-03-29T07:41:39.698 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_hash.ko 2024-03-29T07:41:39.698 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_limit.ko 2024-03-29T07:41:39.698 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_log.ko 2024-03-29T07:41:39.698 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_masq.ko 2024-03-29T07:41:39.698 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_nat.ko 2024-03-29T07:41:39.698 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_numgen.ko 2024-03-29T07:41:39.698 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_queue.ko 2024-03-29T07:41:39.698 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_quota.ko 2024-03-29T07:41:39.698 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_redir.ko 2024-03-29T07:41:39.698 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_reject.ko 2024-03-29T07:41:39.698 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_reject_inet.ko 2024-03-29T07:41:39.698 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_socket.ko 2024-03-29T07:41:39.698 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_tunnel.ko 2024-03-29T07:41:39.699 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_AUDIT.ko 2024-03-29T07:41:39.699 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_CHECKSUM.ko 2024-03-29T07:41:39.699 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_CLASSIFY.ko 2024-03-29T07:41:39.699 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-03-29T07:41:39.699 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_CT.ko 2024-03-29T07:41:39.699 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_DSCP.ko 2024-03-29T07:41:39.699 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_HL.ko 2024-03-29T07:41:39.699 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_HMARK.ko 2024-03-29T07:41:39.699 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_IDLETIMER.ko 2024-03-29T07:41:39.699 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_LED.ko 2024-03-29T07:41:39.699 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_LOG.ko 2024-03-29T07:41:39.699 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_MASQUERADE.ko 2024-03-29T07:41:39.699 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_NETMAP.ko 2024-03-29T07:41:39.699 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_NFLOG.ko 2024-03-29T07:41:39.699 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_NFQUEUE.ko 2024-03-29T07:41:39.699 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_RATEEST.ko 2024-03-29T07:41:39.700 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_REDIRECT.ko 2024-03-29T07:41:39.700 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_SECMARK.ko 2024-03-29T07:41:39.700 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_TCPMSS.ko 2024-03-29T07:41:39.700 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-03-29T07:41:39.700 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_TEE.ko 2024-03-29T07:41:39.700 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_TPROXY.ko 2024-03-29T07:41:39.700 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_TRACE.ko 2024-03-29T07:41:39.700 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_addrtype.ko 2024-03-29T07:41:39.700 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_bpf.ko 2024-03-29T07:41:39.700 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_cgroup.ko 2024-03-29T07:41:39.700 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_cluster.ko 2024-03-29T07:41:39.700 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_comment.ko 2024-03-29T07:41:39.700 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_connbytes.ko 2024-03-29T07:41:39.700 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_connlabel.ko 2024-03-29T07:41:39.700 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_connlimit.ko 2024-03-29T07:41:39.701 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_connmark.ko 2024-03-29T07:41:39.701 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_conntrack.ko 2024-03-29T07:41:39.701 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_cpu.ko 2024-03-29T07:41:39.701 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_dccp.ko 2024-03-29T07:41:39.701 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_devgroup.ko 2024-03-29T07:41:39.701 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_dscp.ko 2024-03-29T07:41:39.701 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_ecn.ko 2024-03-29T07:41:39.701 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_esp.ko 2024-03-29T07:41:39.701 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_hashlimit.ko 2024-03-29T07:41:39.701 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_helper.ko 2024-03-29T07:41:39.701 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_hl.ko 2024-03-29T07:41:39.701 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_ipcomp.ko 2024-03-29T07:41:39.701 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_iprange.ko 2024-03-29T07:41:39.701 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_ipvs.ko 2024-03-29T07:41:39.701 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_l2tp.ko 2024-03-29T07:41:39.701 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_length.ko 2024-03-29T07:41:39.702 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_limit.ko 2024-03-29T07:41:39.702 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_mac.ko 2024-03-29T07:41:39.702 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_mark.ko 2024-03-29T07:41:39.702 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_multiport.ko 2024-03-29T07:41:39.702 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_nat.ko 2024-03-29T07:41:39.702 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_nfacct.ko 2024-03-29T07:41:39.702 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_osf.ko 2024-03-29T07:41:39.702 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_owner.ko 2024-03-29T07:41:39.702 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_physdev.ko 2024-03-29T07:41:39.702 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_pkttype.ko 2024-03-29T07:41:39.702 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_policy.ko 2024-03-29T07:41:39.702 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_quota.ko 2024-03-29T07:41:39.702 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_rateest.ko 2024-03-29T07:41:39.702 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_realm.ko 2024-03-29T07:41:39.703 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_recent.ko 2024-03-29T07:41:39.703 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_sctp.ko 2024-03-29T07:41:39.703 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_set.ko 2024-03-29T07:41:39.703 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_socket.ko 2024-03-29T07:41:39.703 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_state.ko 2024-03-29T07:41:39.703 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_statistic.ko 2024-03-29T07:41:39.703 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_string.ko 2024-03-29T07:41:39.703 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_tcpmss.ko 2024-03-29T07:41:39.703 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_time.ko 2024-03-29T07:41:39.703 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_u32.ko 2024-03-29T07:41:39.703 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netlink 2024-03-29T07:41:39.703 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netlink/netlink_diag.ko 2024-03-29T07:41:39.703 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/nsh 2024-03-29T07:41:39.703 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/nsh/nsh.ko 2024-03-29T07:41:39.703 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/openvswitch 2024-03-29T07:41:39.704 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/openvswitch/openvswitch.ko 2024-03-29T07:41:39.704 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/openvswitch/vport-geneve.ko 2024-03-29T07:41:39.704 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/openvswitch/vport-gre.ko 2024-03-29T07:41:39.704 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/openvswitch/vport-vxlan.ko 2024-03-29T07:41:39.704 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/packet 2024-03-29T07:41:39.704 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/packet/af_packet_diag.ko 2024-03-29T07:41:39.704 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/rds 2024-03-29T07:41:39.704 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/rds/rds.ko 2024-03-29T07:41:39.704 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/rds/rds_rdma.ko 2024-03-29T07:41:39.704 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/rds/rds_tcp.ko 2024-03-29T07:41:39.704 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched 2024-03-29T07:41:39.704 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_csum.ko 2024-03-29T07:41:39.704 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_gact.ko 2024-03-29T07:41:39.704 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_mirred.ko 2024-03-29T07:41:39.704 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_nat.ko 2024-03-29T07:41:39.705 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_pedit.ko 2024-03-29T07:41:39.705 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_police.ko 2024-03-29T07:41:39.705 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_simple.ko 2024-03-29T07:41:39.705 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_skbedit.ko 2024-03-29T07:41:39.705 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_vlan.ko 2024-03-29T07:41:39.705 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_basic.ko 2024-03-29T07:41:39.705 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_bpf.ko 2024-03-29T07:41:39.705 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_flow.ko 2024-03-29T07:41:39.705 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_fw.ko 2024-03-29T07:41:39.705 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_route.ko 2024-03-29T07:41:39.705 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_u32.ko 2024-03-29T07:41:39.705 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_cmp.ko 2024-03-29T07:41:39.705 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_ipset.ko 2024-03-29T07:41:39.705 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_meta.ko 2024-03-29T07:41:39.705 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_nbyte.ko 2024-03-29T07:41:39.706 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_text.ko 2024-03-29T07:41:39.706 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_u32.ko 2024-03-29T07:41:39.706 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_choke.ko 2024-03-29T07:41:39.706 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_codel.ko 2024-03-29T07:41:39.706 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_drr.ko 2024-03-29T07:41:39.706 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_fq.ko 2024-03-29T07:41:39.706 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_gred.ko 2024-03-29T07:41:39.706 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_hfsc.ko 2024-03-29T07:41:39.706 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_hhf.ko 2024-03-29T07:41:39.706 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_htb.ko 2024-03-29T07:41:39.706 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_ingress.ko 2024-03-29T07:41:39.706 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_mqprio.ko 2024-03-29T07:41:39.707 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_mqprio_lib.ko 2024-03-29T07:41:39.707 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_multiq.ko 2024-03-29T07:41:39.707 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_netem.ko 2024-03-29T07:41:39.707 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_pie.ko 2024-03-29T07:41:39.707 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_plug.ko 2024-03-29T07:41:39.707 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_prio.ko 2024-03-29T07:41:39.707 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_qfq.ko 2024-03-29T07:41:39.707 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_red.ko 2024-03-29T07:41:39.707 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_sfb.ko 2024-03-29T07:41:39.707 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_sfq.ko 2024-03-29T07:41:39.707 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_tbf.ko 2024-03-29T07:41:39.707 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_teql.ko 2024-03-29T07:41:39.707 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sctp 2024-03-29T07:41:39.707 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sctp/sctp.ko 2024-03-29T07:41:39.707 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sctp/sctp_diag.ko 2024-03-29T07:41:39.708 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc 2024-03-29T07:41:39.708 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/auth_gss 2024-03-29T07:41:39.708 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-03-29T07:41:39.708 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-03-29T07:41:39.708 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/sunrpc.ko 2024-03-29T07:41:39.708 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/xprtrdma 2024-03-29T07:41:39.708 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-03-29T07:41:39.708 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/tipc 2024-03-29T07:41:39.708 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/tipc/diag.ko 2024-03-29T07:41:39.708 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/tipc/tipc.ko 2024-03-29T07:41:39.708 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/unix 2024-03-29T07:41:39.708 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/unix/unix_diag.ko 2024-03-29T07:41:39.708 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock 2024-03-29T07:41:39.708 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-03-29T07:41:39.708 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-03-29T07:41:39.708 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock/vsock.ko 2024-03-29T07:41:39.709 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock/vsock_diag.ko 2024-03-29T07:41:39.709 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock/vsock_loopback.ko 2024-03-29T07:41:39.709 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/xfrm 2024-03-29T07:41:39.709 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/xfrm/xfrm_ipcomp.ko 2024-03-29T07:41:39.709 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security 2024-03-29T07:41:39.709 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security/keys 2024-03-29T07:41:39.709 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security/keys/encrypted-keys 2024-03-29T07:41:39.709 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-03-29T07:41:39.709 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security/keys/trusted-keys 2024-03-29T07:41:39.709 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security/keys/trusted-keys/trusted.ko 2024-03-29T07:41:39.709 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/virt 2024-03-29T07:41:39.709 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/virt/lib 2024-03-29T07:41:39.709 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/virt/lib/irqbypass.ko 2024-03-29T07:41:39.709 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.alias 2024-03-29T07:41:39.709 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.alias.bin 2024-03-29T07:41:39.710 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.builtin 2024-03-29T07:41:39.710 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.builtin.bin 2024-03-29T07:41:39.710 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.builtin.modinfo 2024-03-29T07:41:39.710 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.dep 2024-03-29T07:41:39.710 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.dep.bin 2024-03-29T07:41:39.710 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.devname 2024-03-29T07:41:39.710 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.order 2024-03-29T07:41:39.710 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.softdep 2024-03-29T07:41:39.710 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.symbols 2024-03-29T07:41:39.710 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.symbols.bin 2024-03-29T07:41:39.710 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.8.0-g1df0c7e8a235/vmlinuz 2024-03-29T07:41:39.714 DEBUG:teuthology.task.kernel:get_image_version: 6.8.0-g1df0c7e8a235 2024-03-29T07:41:39.714 DEBUG:teuthology.orchestra.run.smithi037:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-03-29T07:41:39.744 INFO:teuthology.orchestra.run.smithi037.stdout:postinstall scriptlet (using /bin/sh): 2024-03-29T07:41:39.744 INFO:teuthology.orchestra.run.smithi037.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-03-29T07:41:39.744 INFO:teuthology.orchestra.run.smithi037.stdout: /usr/bin/kernel-install add 6.8.0-g1df0c7e8a235 /lib/modules/6.8.0-g1df0c7e8a235/vmlinuz 2024-03-29T07:41:39.744 INFO:teuthology.orchestra.run.smithi037.stdout:fi 2024-03-29T07:41:39.744 INFO:teuthology.orchestra.run.smithi037.stdout:for file in vmlinuz System.map config; do 2024-03-29T07:41:39.744 INFO:teuthology.orchestra.run.smithi037.stdout: if ! cmp --silent "/lib/modules/6.8.0-g1df0c7e8a235/${file}" "/boot/${file}-6.8.0-g1df0c7e8a235"; then 2024-03-29T07:41:39.744 INFO:teuthology.orchestra.run.smithi037.stdout: cp "/lib/modules/6.8.0-g1df0c7e8a235/${file}" "/boot/${file}-6.8.0-g1df0c7e8a235" 2024-03-29T07:41:39.744 INFO:teuthology.orchestra.run.smithi037.stdout: fi 2024-03-29T07:41:39.744 INFO:teuthology.orchestra.run.smithi037.stdout:done 2024-03-29T07:41:39.745 INFO:teuthology.orchestra.run.smithi037.stdout:preuninstall scriptlet (using /bin/sh): 2024-03-29T07:41:39.745 INFO:teuthology.orchestra.run.smithi037.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2024-03-29T07:41:39.745 INFO:teuthology.orchestra.run.smithi037.stdout:new-kernel-pkg --remove 6.8.0-g1df0c7e8a235 --rminitrd --initrdfile=/boot/initramfs-6.8.0-g1df0c7e8a235.img 2024-03-29T07:41:39.745 INFO:teuthology.orchestra.run.smithi037.stdout:elif [ -x /usr/bin/kernel-install ]; then 2024-03-29T07:41:39.745 INFO:teuthology.orchestra.run.smithi037.stdout:kernel-install remove 6.8.0-g1df0c7e8a235 2024-03-29T07:41:39.745 INFO:teuthology.orchestra.run.smithi037.stdout:fi 2024-03-29T07:41:39.745 INFO:teuthology.orchestra.run.smithi037.stdout:postuninstall scriptlet (using /bin/sh): 2024-03-29T07:41:39.745 INFO:teuthology.orchestra.run.smithi037.stdout:if [ -x /sbin/update-bootloader ]; then 2024-03-29T07:41:39.745 INFO:teuthology.orchestra.run.smithi037.stdout:/sbin/update-bootloader --remove 6.8.0-g1df0c7e8a235 2024-03-29T07:41:39.745 INFO:teuthology.orchestra.run.smithi037.stdout:fi 2024-03-29T07:41:39.746 DEBUG:teuthology.orchestra.run.smithi037:> sudo rpm -qi grub2-tools 2024-03-29T07:41:39.926 INFO:teuthology.orchestra.run.smithi037.stdout:Name : grub2-tools 2024-03-29T07:41:39.927 INFO:teuthology.orchestra.run.smithi037.stdout:Epoch : 1 2024-03-29T07:41:39.927 INFO:teuthology.orchestra.run.smithi037.stdout:Version : 2.02 2024-03-29T07:41:39.927 INFO:teuthology.orchestra.run.smithi037.stdout:Release : 129.el8 2024-03-29T07:41:39.927 INFO:teuthology.orchestra.run.smithi037.stdout:Architecture: x86_64 2024-03-29T07:41:39.927 INFO:teuthology.orchestra.run.smithi037.stdout:Install Date: Thu 04 May 2023 10:17:48 PM UTC 2024-03-29T07:41:39.927 INFO:teuthology.orchestra.run.smithi037.stdout:Group : System Environment/Base 2024-03-29T07:41:39.927 INFO:teuthology.orchestra.run.smithi037.stdout:Size : 9265586 2024-03-29T07:41:39.927 INFO:teuthology.orchestra.run.smithi037.stdout:License : GPLv3+ 2024-03-29T07:41:39.927 INFO:teuthology.orchestra.run.smithi037.stdout:Signature : RSA/SHA256, Tue 19 Jul 2022 02:52:06 PM UTC, Key ID 05b555b38483c65d 2024-03-29T07:41:39.927 INFO:teuthology.orchestra.run.smithi037.stdout:Source RPM : grub2-2.02-129.el8.src.rpm 2024-03-29T07:41:39.927 INFO:teuthology.orchestra.run.smithi037.stdout:Build Date : Tue 19 Jul 2022 02:30:39 PM UTC 2024-03-29T07:41:39.927 INFO:teuthology.orchestra.run.smithi037.stdout:Build Host : kbuilder.bsys.centos.org 2024-03-29T07:41:39.927 INFO:teuthology.orchestra.run.smithi037.stdout:Relocations : (not relocatable) 2024-03-29T07:41:39.928 INFO:teuthology.orchestra.run.smithi037.stdout:Packager : CentOS BuildSystem 2024-03-29T07:41:39.928 INFO:teuthology.orchestra.run.smithi037.stdout:Vendor : CentOS 2024-03-29T07:41:39.928 INFO:teuthology.orchestra.run.smithi037.stdout:URL : http://www.gnu.org/software/grub/ 2024-03-29T07:41:39.928 INFO:teuthology.orchestra.run.smithi037.stdout:Summary : Support tools for GRUB. 2024-03-29T07:41:39.928 INFO:teuthology.orchestra.run.smithi037.stdout:Description : 2024-03-29T07:41:39.928 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T07:41:39.928 INFO:teuthology.orchestra.run.smithi037.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-03-29T07:41:39.928 INFO:teuthology.orchestra.run.smithi037.stdout:customizable bootloader with modular architecture. It supports a rich 2024-03-29T07:41:39.928 INFO:teuthology.orchestra.run.smithi037.stdout:variety of kernel formats, file systems, computer architectures and 2024-03-29T07:41:39.928 INFO:teuthology.orchestra.run.smithi037.stdout:hardware devices. 2024-03-29T07:41:39.928 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T07:41:39.928 INFO:teuthology.orchestra.run.smithi037.stdout:This subpackage provides tools for support of all platforms. 2024-03-29T07:41:39.929 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-03-29T07:41:39.929 INFO:teuthology.task.kernel:Updating grub on smithi037 to boot 6.8.0-g1df0c7e8a235 2024-03-29T07:41:39.929 DEBUG:teuthology.orchestra.run.smithi037:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-03-29T07:41:44.474 INFO:teuthology.orchestra.run.smithi037.stderr:Generating grub configuration file ... 2024-03-29T07:41:46.546 INFO:teuthology.orchestra.run.smithi037.stderr:done 2024-03-29T07:41:46.547 DEBUG:teuthology.orchestra.run.smithi037:> mktemp 2024-03-29T07:41:46.568 INFO:teuthology.orchestra.run.smithi037.stdout:/tmp/tmp.0H6Fa7XR1a 2024-03-29T07:41:46.569 DEBUG:teuthology.orchestra.run.smithi037:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.0H6Fa7XR1a 2024-03-29T07:41:46.642 DEBUG:teuthology.orchestra.run.smithi037:> sudo chmod 0666 /tmp/tmp.0H6Fa7XR1a 2024-03-29T07:41:46.933 DEBUG:teuthology.orchestra.remote:smithi037:/tmp/tmp.0H6Fa7XR1a is 6KB 2024-03-29T07:41:46.982 DEBUG:teuthology.orchestra.run.smithi037:> rm -fr /tmp/tmp.0H6Fa7XR1a 2024-03-29T07:41:47.000 DEBUG:teuthology.orchestra.run.smithi037:> sudo /bin/ls /boot/loader/entries || true 2024-03-29T07:41:47.066 INFO:teuthology.orchestra.run.smithi037.stdout:a1b29d6c289747bdb22930598d39874a-0-rescue.conf 2024-03-29T07:41:47.066 INFO:teuthology.orchestra.run.smithi037.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-348.el8.x86_64.conf 2024-03-29T07:41:47.066 INFO:teuthology.orchestra.run.smithi037.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-489.el8.x86_64.conf 2024-03-29T07:41:47.066 INFO:teuthology.orchestra.run.smithi037.stdout:a1b29d6c289747bdb22930598d39874a-6.8.0-g1df0c7e8a235.conf 2024-03-29T07:41:47.067 DEBUG:teuthology.orchestra.run.smithi037:> sudo grub2-set-default a1b29d6c289747bdb22930598d39874a-6.8.0-g1df0c7e8a235 2024-03-29T07:41:47.236 DEBUG:teuthology.orchestra.run.smithi037:> sudo shutdown -r now 2024-03-29T07:41:56.719 INFO:teuthology.task.kernel:Checking client host.b for new kernel version... 2024-03-29T07:41:56.719 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi148.front.sepia.ceph.com' 2024-03-29T07:41:56.720 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi148.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-29T07:42:17.259 INFO:teuthology.task.kernel:Checking client host.a for new kernel version... 2024-03-29T07:42:17.260 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi037.front.sepia.ceph.com' 2024-03-29T07:42:17.261 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi037.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-29T07:42:56.759 DEBUG:teuthology.orchestra.remote:timed out 2024-03-29T07:43:05.768 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi148.front.sepia.ceph.com' 2024-03-29T07:43:05.769 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi148.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-29T07:43:17.274 DEBUG:teuthology.orchestra.remote:timed out 2024-03-29T07:43:21.430 DEBUG:teuthology.orchestra.run.smithi148:> true 2024-03-29T07:43:21.848 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi148.front.sepia.ceph.com' 2024-03-29T07:43:21.848 INFO:teuthology.task.kernel:Checking kernel version of host.b, want "6.8.0-g1df0c7e8a235"... 2024-03-29T07:43:21.849 DEBUG:teuthology.orchestra.run.smithi148:> uname -r 2024-03-29T07:43:21.892 INFO:teuthology.orchestra.run.smithi148.stdout:6.8.0-g1df0c7e8a235 2024-03-29T07:43:21.893 DEBUG:teuthology.task.kernel:current kernel version is 6.8.0-g1df0c7e8a235 vs 6.8.0-g1df0c7e8a235 2024-03-29T07:43:21.893 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-03-29T07:43:21.893 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-03-29T07:43:22.893 INFO:teuthology.task.kernel:Enabling kdb on host.b... 2024-03-29T07:43:22.894 DEBUG:teuthology.orchestra.run.smithi148:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-03-29T07:43:23.048 INFO:teuthology.orchestra.run.smithi148.stdout:ttyS1 2024-03-29T07:43:23.074 DEBUG:teuthology.parallel:result is None 2024-03-29T07:43:26.275 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi037.front.sepia.ceph.com' 2024-03-29T07:43:26.276 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi037.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-29T07:43:26.638 DEBUG:teuthology.orchestra.run.smithi037:> true 2024-03-29T07:43:27.047 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi037.front.sepia.ceph.com' 2024-03-29T07:43:27.047 INFO:teuthology.task.kernel:Checking kernel version of host.a, want "6.8.0-g1df0c7e8a235"... 2024-03-29T07:43:27.047 DEBUG:teuthology.orchestra.run.smithi037:> uname -r 2024-03-29T07:43:27.090 INFO:teuthology.orchestra.run.smithi037.stdout:6.8.0-g1df0c7e8a235 2024-03-29T07:43:27.091 DEBUG:teuthology.task.kernel:current kernel version is 6.8.0-g1df0c7e8a235 vs 6.8.0-g1df0c7e8a235 2024-03-29T07:43:27.091 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-03-29T07:43:27.091 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-03-29T07:43:28.091 INFO:teuthology.task.kernel:Enabling kdb on host.a... 2024-03-29T07:43:28.092 DEBUG:teuthology.orchestra.run.smithi037:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-03-29T07:43:28.240 INFO:teuthology.orchestra.run.smithi037.stdout:ttyS1 2024-03-29T07:43:28.266 DEBUG:teuthology.parallel:result is None 2024-03-29T07:43:28.267 INFO:teuthology.run_tasks:Running task internal.base... 2024-03-29T07:43:28.274 INFO:teuthology.task.internal:Creating test directory... 2024-03-29T07:43:28.274 DEBUG:teuthology.orchestra.run.smithi037:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-29T07:43:28.278 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-29T07:43:28.327 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-03-29T07:43:28.334 INFO:teuthology.run_tasks:Running task internal.archive... 2024-03-29T07:43:28.361 INFO:teuthology.task.internal:Creating archive directory... 2024-03-29T07:43:28.361 DEBUG:teuthology.orchestra.run.smithi037:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-29T07:43:28.365 DEBUG:teuthology.orchestra.run.smithi148:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-29T07:43:28.429 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-03-29T07:43:28.439 INFO:teuthology.task.internal:Enabling coredump saving... 2024-03-29T07:43:28.440 DEBUG:teuthology.orchestra.run.smithi037:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-29T07:43:28.445 DEBUG:teuthology.orchestra.run.smithi148:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-29T07:43:28.531 INFO:teuthology.orchestra.run.smithi037.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-29T07:43:28.533 INFO:teuthology.orchestra.run.smithi148.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-29T07:43:28.558 INFO:teuthology.orchestra.run.smithi037.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-29T07:43:28.558 INFO:teuthology.orchestra.run.smithi148.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-29T07:43:28.562 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-03-29T07:43:28.573 INFO:teuthology.task.internal:Configuring sudo... 2024-03-29T07:43:28.573 DEBUG:teuthology.orchestra.run.smithi037:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-29T07:43:28.578 DEBUG:teuthology.orchestra.run.smithi148:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-29T07:43:28.645 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-03-29T07:43:28.654 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-03-29T07:43:28.655 DEBUG:teuthology.orchestra.run.smithi037:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-29T07:43:28.659 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-29T07:43:28.703 DEBUG:teuthology.orchestra.run.smithi037:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-29T07:43:28.811 DEBUG:teuthology.orchestra.run.smithi037:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-29T07:43:28.877 DEBUG:teuthology.orchestra.run.smithi037:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-29T07:43:28.951 DEBUG:teuthology.orchestra.run.smithi037:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-29T07:43:29.016 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-03-29T07:43:29.017 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-29T07:43:29.082 DEBUG:teuthology.orchestra.run.smithi148:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-29T07:43:29.154 DEBUG:teuthology.orchestra.run.smithi148:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-29T07:43:29.221 DEBUG:teuthology.orchestra.run.smithi148:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-29T07:43:29.291 DEBUG:teuthology.orchestra.run.smithi148:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-29T07:43:29.358 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-03-29T07:43:29.358 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-29T07:43:29.425 DEBUG:teuthology.orchestra.run.smithi037:> sudo service rsyslog restart 2024-03-29T07:43:29.429 DEBUG:teuthology.orchestra.run.smithi148:> sudo service rsyslog restart 2024-03-29T07:43:29.546 INFO:teuthology.orchestra.run.smithi037.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-29T07:43:29.586 INFO:teuthology.orchestra.run.smithi148.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-29T07:43:29.958 INFO:teuthology.run_tasks:Running task internal.timer... 2024-03-29T07:43:29.966 INFO:teuthology.task.internal:Starting timer... 2024-03-29T07:43:29.966 INFO:teuthology.run_tasks:Running task pcp... 2024-03-29T07:43:30.007 INFO:teuthology.run_tasks:Running task selinux... 2024-03-29T07:43:30.016 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2024-03-29T07:43:30.017 DEBUG:teuthology.orchestra.run.smithi037:> sudo service auditd rotate 2024-03-29T07:43:30.161 INFO:teuthology.orchestra.run.smithi037.stdout:Rotating logs: 2024-03-29T07:43:30.165 DEBUG:teuthology.orchestra.run.smithi148:> sudo service auditd rotate 2024-03-29T07:43:30.295 INFO:teuthology.orchestra.run.smithi148.stdout:Rotating logs: 2024-03-29T07:43:30.298 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-03-29T07:43:30.298 DEBUG:teuthology.orchestra.run.smithi037:> /usr/sbin/getenforce 2024-03-29T07:43:30.350 INFO:teuthology.orchestra.run.smithi037.stdout:Permissive 2024-03-29T07:43:30.351 DEBUG:teuthology.orchestra.run.smithi148:> /usr/sbin/getenforce 2024-03-29T07:43:30.405 INFO:teuthology.orchestra.run.smithi148.stdout:Permissive 2024-03-29T07:43:30.405 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi037.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi148.front.sepia.ceph.com': 'permissive'} 2024-03-29T07:43:30.406 DEBUG:teuthology.orchestra.run.smithi037:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-03-29T07:43:30.472 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-29T07:43:30.473 DEBUG:teuthology.orchestra.run.smithi148:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-03-29T07:43:30.540 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-29T07:43:30.540 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-03-29T07:43:30.541 DEBUG:teuthology.orchestra.run.smithi037:> sudo /usr/sbin/setenforce permissive 2024-03-29T07:43:30.607 DEBUG:teuthology.orchestra.run.smithi148:> sudo /usr/sbin/setenforce permissive 2024-03-29T07:43:30.673 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-03-29T07:43:30.683 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-03-29T07:43:30.689 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-03-29T07:43:30.822 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-03-29T07:43:30.840 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-03-29T07:43:30.841 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi037.front.sepia.ceph.com,smithi148.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-03-29T07:50:45.317 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi037.front.sepia.ceph.com'), Remote(name='ubuntu@smithi148.front.sepia.ceph.com')] 2024-03-29T07:50:45.318 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi037.front.sepia.ceph.com' 2024-03-29T07:50:45.319 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi037.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-29T07:50:45.412 DEBUG:teuthology.orchestra.run.smithi037:> true 2024-03-29T07:50:45.531 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi037.front.sepia.ceph.com' 2024-03-29T07:50:45.531 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi148.front.sepia.ceph.com' 2024-03-29T07:50:45.532 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi148.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-29T07:50:45.620 DEBUG:teuthology.orchestra.run.smithi148:> true 2024-03-29T07:50:45.739 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi148.front.sepia.ceph.com' 2024-03-29T07:50:45.740 INFO:teuthology.run_tasks:Running task clock... 2024-03-29T07:50:45.751 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-03-29T07:50:45.752 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-29T07:50:45.752 DEBUG:teuthology.orchestra.run.smithi037:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-29T07:50:45.756 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-29T07:50:45.756 DEBUG:teuthology.orchestra.run.smithi148:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-29T07:50:45.826 INFO:teuthology.orchestra.run.smithi037.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-29T07:50:45.852 INFO:teuthology.orchestra.run.smithi148.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-29T07:50:45.858 INFO:teuthology.orchestra.run.smithi037.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-29T07:50:45.883 INFO:teuthology.orchestra.run.smithi148.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-29T07:50:45.945 INFO:teuthology.orchestra.run.smithi037.stderr:sudo: ntpd: command not found 2024-03-29T07:50:45.966 INFO:teuthology.orchestra.run.smithi148.stderr:sudo: ntpd: command not found 2024-03-29T07:50:45.972 INFO:teuthology.orchestra.run.smithi037.stdout:506 Cannot talk to daemon 2024-03-29T07:50:45.993 INFO:teuthology.orchestra.run.smithi148.stdout:506 Cannot talk to daemon 2024-03-29T07:50:46.005 INFO:teuthology.orchestra.run.smithi037.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-29T07:50:46.024 INFO:teuthology.orchestra.run.smithi148.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-29T07:50:46.036 INFO:teuthology.orchestra.run.smithi037.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-29T07:50:46.055 INFO:teuthology.orchestra.run.smithi148.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-29T07:50:46.133 INFO:teuthology.orchestra.run.smithi037.stderr:bash: ntpq: command not found 2024-03-29T07:50:46.140 INFO:teuthology.orchestra.run.smithi037.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-29T07:50:46.140 INFO:teuthology.orchestra.run.smithi037.stdout:=============================================================================== 2024-03-29T07:50:46.140 INFO:teuthology.orchestra.run.smithi037.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-29T07:50:46.140 INFO:teuthology.orchestra.run.smithi037.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-29T07:50:46.140 INFO:teuthology.orchestra.run.smithi037.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-29T07:50:46.140 INFO:teuthology.orchestra.run.smithi037.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-29T07:50:46.148 INFO:teuthology.orchestra.run.smithi148.stderr:bash: ntpq: command not found 2024-03-29T07:50:46.156 INFO:teuthology.orchestra.run.smithi148.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-29T07:50:46.156 INFO:teuthology.orchestra.run.smithi148.stdout:=============================================================================== 2024-03-29T07:50:46.156 INFO:teuthology.orchestra.run.smithi148.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-29T07:50:46.156 INFO:teuthology.orchestra.run.smithi148.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-29T07:50:46.156 INFO:teuthology.orchestra.run.smithi148.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-29T07:50:46.156 INFO:teuthology.orchestra.run.smithi148.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-29T07:50:46.158 INFO:teuthology.run_tasks:Running task pexec... 2024-03-29T07:50:46.168 INFO:teuthology.task.pexec:Executing custom commands... 2024-03-29T07:50:46.168 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi037.front.sepia.ceph.com 2024-03-29T07:50:46.168 DEBUG:teuthology.orchestra.run.smithi037:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-03-29T07:50:46.168 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi148.front.sepia.ceph.com 2024-03-29T07:50:46.169 DEBUG:teuthology.orchestra.run.smithi148:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-03-29T07:50:47.103 INFO:teuthology.orchestra.run.smithi148.stdout:Last metadata expiration check: 0:01:40 ago on Fri 29 Mar 2024 07:49:06 AM UTC. 2024-03-29T07:50:47.137 INFO:teuthology.orchestra.run.smithi037.stdout:Last metadata expiration check: 0:01:34 ago on Fri 29 Mar 2024 07:49:12 AM UTC. 2024-03-29T07:50:49.473 INFO:teuthology.orchestra.run.smithi148.stdout:Dependencies resolved. 2024-03-29T07:50:49.474 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-03-29T07:50:49.474 INFO:teuthology.orchestra.run.smithi148.stdout: Package Architecture Version Repository Size 2024-03-29T07:50:49.474 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-03-29T07:50:49.474 INFO:teuthology.orchestra.run.smithi148.stdout:Resetting modules: 2024-03-29T07:50:49.474 INFO:teuthology.orchestra.run.smithi148.stdout: container-tools 2024-03-29T07:50:49.474 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T07:50:49.474 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction Summary 2024-03-29T07:50:49.474 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-03-29T07:50:49.475 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T07:50:49.545 INFO:teuthology.orchestra.run.smithi037.stdout:Dependencies resolved. 2024-03-29T07:50:49.546 INFO:teuthology.orchestra.run.smithi037.stdout:================================================================================ 2024-03-29T07:50:49.546 INFO:teuthology.orchestra.run.smithi037.stdout: Package Architecture Version Repository Size 2024-03-29T07:50:49.547 INFO:teuthology.orchestra.run.smithi037.stdout:================================================================================ 2024-03-29T07:50:49.547 INFO:teuthology.orchestra.run.smithi037.stdout:Resetting modules: 2024-03-29T07:50:49.547 INFO:teuthology.orchestra.run.smithi037.stdout: container-tools 2024-03-29T07:50:49.547 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T07:50:49.547 INFO:teuthology.orchestra.run.smithi037.stdout:Transaction Summary 2024-03-29T07:50:49.547 INFO:teuthology.orchestra.run.smithi037.stdout:================================================================================ 2024-03-29T07:50:49.547 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T07:50:49.750 INFO:teuthology.orchestra.run.smithi148.stdout:Complete! 2024-03-29T07:50:49.833 INFO:teuthology.orchestra.run.smithi037.stdout:Complete! 2024-03-29T07:50:50.744 INFO:teuthology.orchestra.run.smithi148.stdout:Last metadata expiration check: 0:01:44 ago on Fri 29 Mar 2024 07:49:06 AM UTC. 2024-03-29T07:50:50.878 INFO:teuthology.orchestra.run.smithi037.stdout:Last metadata expiration check: 0:01:38 ago on Fri 29 Mar 2024 07:49:12 AM UTC. 2024-03-29T07:50:52.946 INFO:teuthology.orchestra.run.smithi148.stdout:Dependencies resolved. 2024-03-29T07:50:52.986 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-03-29T07:50:52.986 INFO:teuthology.orchestra.run.smithi148.stdout: Package Arch Version Repository Size 2024-03-29T07:50:52.986 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-03-29T07:50:52.986 INFO:teuthology.orchestra.run.smithi148.stdout:Installing group/module packages: 2024-03-29T07:50:52.986 INFO:teuthology.orchestra.run.smithi148.stdout: aardvark-dns x86_64 2:1.10.0-1.module_el8+871+552be958 appstream 1.0 M 2024-03-29T07:50:52.986 INFO:teuthology.orchestra.run.smithi148.stdout: buildah x86_64 2:1.33.6-2.module_el8+976+11360731 appstream 9.6 M 2024-03-29T07:50:52.986 INFO:teuthology.orchestra.run.smithi148.stdout: cockpit-podman noarch 84.1-1.module_el8+977+8ab54275 appstream 689 k 2024-03-29T07:50:52.987 INFO:teuthology.orchestra.run.smithi148.stdout: crun x86_64 1.14.3-2.module_el8+968+fbb249c7 appstream 258 k 2024-03-29T07:50:52.987 INFO:teuthology.orchestra.run.smithi148.stdout: netavark x86_64 2:1.10.3-1.module_el8+951+32019cde appstream 4.1 M 2024-03-29T07:50:52.987 INFO:teuthology.orchestra.run.smithi148.stdout: python3-podman noarch 4.9.0-1.module_el8+847+7863d4e6 appstream 180 k 2024-03-29T07:50:52.987 INFO:teuthology.orchestra.run.smithi148.stdout: skopeo x86_64 2:1.14.3-0.1.module_el8+885+7da147f3 appstream 8.8 M 2024-03-29T07:50:52.987 INFO:teuthology.orchestra.run.smithi148.stdout: toolbox x86_64 0.0.99.5-2.module_el8+971+3d3df00d appstream 2.5 M 2024-03-29T07:50:52.987 INFO:teuthology.orchestra.run.smithi148.stdout: udica noarch 0.2.6-20.module_el8+804+f131391c appstream 53 k 2024-03-29T07:50:52.987 INFO:teuthology.orchestra.run.smithi148.stdout:Installing dependencies: 2024-03-29T07:50:52.987 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 appstream 25 k 2024-03-29T07:50:52.987 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pyxdg noarch 0.25-16.el8 appstream 94 k 2024-03-29T07:50:52.987 INFO:teuthology.orchestra.run.smithi148.stdout: yajl x86_64 2.1.0-12.el8 appstream 41 k 2024-03-29T07:50:52.987 INFO:teuthology.orchestra.run.smithi148.stdout:Installing weak dependencies: 2024-03-29T07:50:52.987 INFO:teuthology.orchestra.run.smithi148.stdout: criu-libs x86_64 3.15-3.module_el8.7.0+1216+b022c01d appstream 38 k 2024-03-29T07:50:52.987 INFO:teuthology.orchestra.run.smithi148.stdout:Installing module profiles: 2024-03-29T07:50:52.987 INFO:teuthology.orchestra.run.smithi148.stdout: container-tools/common 2024-03-29T07:50:52.987 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T07:50:52.988 INFO:teuthology.orchestra.run.smithi148.stdout:Enabling module streams: 2024-03-29T07:50:52.988 INFO:teuthology.orchestra.run.smithi148.stdout: container-tools rhel8 2024-03-29T07:50:52.988 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T07:50:52.988 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction Summary 2024-03-29T07:50:52.988 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-03-29T07:50:52.988 INFO:teuthology.orchestra.run.smithi148.stdout:Install 13 Packages 2024-03-29T07:50:52.988 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T07:50:52.988 INFO:teuthology.orchestra.run.smithi148.stdout:Total download size: 27 M 2024-03-29T07:50:52.988 INFO:teuthology.orchestra.run.smithi148.stdout:Installed size: 91 M 2024-03-29T07:50:52.988 INFO:teuthology.orchestra.run.smithi148.stdout:Downloading Packages: 2024-03-29T07:50:53.237 INFO:teuthology.orchestra.run.smithi037.stdout:Dependencies resolved. 2024-03-29T07:50:53.280 INFO:teuthology.orchestra.run.smithi037.stdout:================================================================================ 2024-03-29T07:50:53.280 INFO:teuthology.orchestra.run.smithi037.stdout: Package Arch Version Repository Size 2024-03-29T07:50:53.280 INFO:teuthology.orchestra.run.smithi037.stdout:================================================================================ 2024-03-29T07:50:53.280 INFO:teuthology.orchestra.run.smithi037.stdout:Installing group/module packages: 2024-03-29T07:50:53.280 INFO:teuthology.orchestra.run.smithi037.stdout: aardvark-dns x86_64 2:1.10.0-1.module_el8+871+552be958 appstream 1.0 M 2024-03-29T07:50:53.280 INFO:teuthology.orchestra.run.smithi037.stdout: buildah x86_64 2:1.33.6-2.module_el8+976+11360731 appstream 9.6 M 2024-03-29T07:50:53.280 INFO:teuthology.orchestra.run.smithi037.stdout: cockpit-podman noarch 84.1-1.module_el8+977+8ab54275 appstream 689 k 2024-03-29T07:50:53.280 INFO:teuthology.orchestra.run.smithi037.stdout: crun x86_64 1.14.3-2.module_el8+968+fbb249c7 appstream 258 k 2024-03-29T07:50:53.280 INFO:teuthology.orchestra.run.smithi037.stdout: netavark x86_64 2:1.10.3-1.module_el8+951+32019cde appstream 4.1 M 2024-03-29T07:50:53.280 INFO:teuthology.orchestra.run.smithi037.stdout: python3-podman noarch 4.9.0-1.module_el8+847+7863d4e6 appstream 180 k 2024-03-29T07:50:53.280 INFO:teuthology.orchestra.run.smithi037.stdout: skopeo x86_64 2:1.14.3-0.1.module_el8+885+7da147f3 appstream 8.8 M 2024-03-29T07:50:53.280 INFO:teuthology.orchestra.run.smithi037.stdout: toolbox x86_64 0.0.99.5-2.module_el8+971+3d3df00d appstream 2.5 M 2024-03-29T07:50:53.281 INFO:teuthology.orchestra.run.smithi037.stdout: udica noarch 0.2.6-20.module_el8+804+f131391c appstream 53 k 2024-03-29T07:50:53.281 INFO:teuthology.orchestra.run.smithi037.stdout:Installing dependencies: 2024-03-29T07:50:53.281 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 appstream 25 k 2024-03-29T07:50:53.281 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pyxdg noarch 0.25-16.el8 appstream 94 k 2024-03-29T07:50:53.281 INFO:teuthology.orchestra.run.smithi037.stdout: yajl x86_64 2.1.0-12.el8 appstream 41 k 2024-03-29T07:50:53.281 INFO:teuthology.orchestra.run.smithi037.stdout:Installing weak dependencies: 2024-03-29T07:50:53.281 INFO:teuthology.orchestra.run.smithi037.stdout: criu-libs x86_64 3.15-3.module_el8.7.0+1216+b022c01d appstream 38 k 2024-03-29T07:50:53.281 INFO:teuthology.orchestra.run.smithi037.stdout:Installing module profiles: 2024-03-29T07:50:53.281 INFO:teuthology.orchestra.run.smithi037.stdout: container-tools/common 2024-03-29T07:50:53.281 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T07:50:53.281 INFO:teuthology.orchestra.run.smithi037.stdout:Enabling module streams: 2024-03-29T07:50:53.281 INFO:teuthology.orchestra.run.smithi037.stdout: container-tools rhel8 2024-03-29T07:50:53.281 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T07:50:53.281 INFO:teuthology.orchestra.run.smithi037.stdout:Transaction Summary 2024-03-29T07:50:53.281 INFO:teuthology.orchestra.run.smithi037.stdout:================================================================================ 2024-03-29T07:50:53.282 INFO:teuthology.orchestra.run.smithi037.stdout:Install 13 Packages 2024-03-29T07:50:53.282 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T07:50:53.282 INFO:teuthology.orchestra.run.smithi037.stdout:Total download size: 27 M 2024-03-29T07:50:53.282 INFO:teuthology.orchestra.run.smithi037.stdout:Installed size: 91 M 2024-03-29T07:50:53.282 INFO:teuthology.orchestra.run.smithi037.stdout:Downloading Packages: 2024-03-29T07:50:53.414 INFO:teuthology.orchestra.run.smithi148.stdout:(1/13): aardvark-dns-1.10.0-1.module_el8+871+55 4.5 MB/s | 1.0 MB 00:00 2024-03-29T07:50:53.455 INFO:teuthology.orchestra.run.smithi148.stdout:(2/13): cockpit-podman-84.1-1.module_el8+977+8a 2.5 MB/s | 689 kB 00:00 2024-03-29T07:50:53.539 INFO:teuthology.orchestra.run.smithi148.stdout:(3/13): criu-libs-3.15-3.module_el8.7.0+1216+b0 303 kB/s | 38 kB 00:00 2024-03-29T07:50:53.590 INFO:teuthology.orchestra.run.smithi148.stdout:(4/13): crun-1.14.3-2.module_el8+968+fbb249c7.x 1.9 MB/s | 258 kB 00:00 2024-03-29T07:50:53.723 INFO:teuthology.orchestra.run.smithi148.stdout:(5/13): python3-podman-4.9.0-1.module_el8+847+7 1.3 MB/s | 180 kB 00:00 2024-03-29T07:50:53.849 INFO:teuthology.orchestra.run.smithi148.stdout:(6/13): python3-pytoml-0.1.14-5.git7dea353.el8. 198 kB/s | 25 kB 00:00 2024-03-29T07:50:53.874 INFO:teuthology.orchestra.run.smithi037.stdout:(1/13): cockpit-podman-84.1-1.module_el8+977+8a 1.7 MB/s | 689 kB 00:00 2024-03-29T07:50:53.932 INFO:teuthology.orchestra.run.smithi148.stdout:(7/13): netavark-1.10.3-1.module_el8+951+32019c 10 MB/s | 4.1 MB 00:00 2024-03-29T07:50:53.941 INFO:teuthology.orchestra.run.smithi037.stdout:(2/13): aardvark-dns-1.10.0-1.module_el8+871+55 2.2 MB/s | 1.0 MB 00:00 2024-03-29T07:50:53.966 INFO:teuthology.orchestra.run.smithi148.stdout:(8/13): python3-pyxdg-0.25-16.el8.noarch.rpm 802 kB/s | 94 kB 00:00 2024-03-29T07:50:53.966 INFO:teuthology.orchestra.run.smithi037.stdout:(3/13): criu-libs-3.15-3.module_el8.7.0+1216+b0 413 kB/s | 38 kB 00:00 2024-03-29T07:50:54.075 INFO:teuthology.orchestra.run.smithi037.stdout:(4/13): crun-1.14.3-2.module_el8+968+fbb249c7.x 1.9 MB/s | 258 kB 00:00 2024-03-29T07:50:54.108 INFO:teuthology.orchestra.run.smithi148.stdout:(9/13): buildah-1.33.6-2.module_el8+976+1136073 11 MB/s | 9.6 MB 00:00 2024-03-29T07:50:54.233 INFO:teuthology.orchestra.run.smithi037.stdout:(5/13): python3-podman-4.9.0-1.module_el8+847+7 1.1 MB/s | 180 kB 00:00 2024-03-29T07:50:54.242 INFO:teuthology.orchestra.run.smithi148.stdout:(10/13): udica-0.2.6-20.module_el8+804+f131391c 395 kB/s | 53 kB 00:00 2024-03-29T07:50:54.300 INFO:teuthology.orchestra.run.smithi148.stdout:(11/13): toolbox-0.0.99.5-2.module_el8+971+3d3d 7.6 MB/s | 2.5 MB 00:00 2024-03-29T07:50:54.350 INFO:teuthology.orchestra.run.smithi037.stdout:(6/13): python3-pytoml-0.1.14-5.git7dea353.el8. 212 kB/s | 25 kB 00:00 2024-03-29T07:50:54.351 INFO:teuthology.orchestra.run.smithi148.stdout:(12/13): yajl-2.1.0-12.el8.x86_64.rpm 378 kB/s | 41 kB 00:00 2024-03-29T07:50:54.467 INFO:teuthology.orchestra.run.smithi037.stdout:(7/13): python3-pyxdg-0.25-16.el8.noarch.rpm 807 kB/s | 94 kB 00:00 2024-03-29T07:50:54.576 INFO:teuthology.orchestra.run.smithi148.stdout:(13/13): skopeo-1.14.3-0.1.module_el8+885+7da14 14 MB/s | 8.8 MB 00:00 2024-03-29T07:50:54.576 INFO:teuthology.orchestra.run.smithi148.stdout:-------------------------------------------------------------------------------- 2024-03-29T07:50:54.576 INFO:teuthology.orchestra.run.smithi148.stdout:Total 17 MB/s | 27 MB 00:01 2024-03-29T07:50:54.734 INFO:teuthology.orchestra.run.smithi037.stdout:(8/13): buildah-1.33.6-2.module_el8+976+1136073 7.7 MB/s | 9.6 MB 00:01 2024-03-29T07:50:54.893 INFO:teuthology.orchestra.run.smithi037.stdout:(9/13): netavark-1.10.3-1.module_el8+951+32019c 4.4 MB/s | 4.1 MB 00:00 2024-03-29T07:50:54.985 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction check 2024-03-29T07:50:54.993 INFO:teuthology.orchestra.run.smithi037.stdout:(10/13): toolbox-0.0.99.5-2.module_el8+971+3d3d 9.8 MB/s | 2.5 MB 00:00 2024-03-29T07:50:55.010 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction check succeeded. 2024-03-29T07:50:55.011 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction test 2024-03-29T07:50:55.018 INFO:teuthology.orchestra.run.smithi037.stdout:(11/13): udica-0.2.6-20.module_el8+804+f131391c 420 kB/s | 53 kB 00:00 2024-03-29T07:50:55.077 INFO:teuthology.orchestra.run.smithi037.stdout:(12/13): yajl-2.1.0-12.el8.x86_64.rpm 490 kB/s | 41 kB 00:00 2024-03-29T07:50:55.294 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction test succeeded. 2024-03-29T07:50:55.298 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction 2024-03-29T07:50:55.394 INFO:teuthology.orchestra.run.smithi037.stdout:(13/13): skopeo-1.14.3-0.1.module_el8+885+7da14 9.5 MB/s | 8.8 MB 00:00 2024-03-29T07:50:55.395 INFO:teuthology.orchestra.run.smithi037.stdout:-------------------------------------------------------------------------------- 2024-03-29T07:50:55.395 INFO:teuthology.orchestra.run.smithi037.stdout:Total 13 MB/s | 27 MB 00:02 2024-03-29T07:50:55.818 INFO:teuthology.orchestra.run.smithi037.stdout:Running transaction check 2024-03-29T07:50:55.845 INFO:teuthology.orchestra.run.smithi037.stdout:Transaction check succeeded. 2024-03-29T07:50:55.845 INFO:teuthology.orchestra.run.smithi037.stdout:Running transaction test 2024-03-29T07:50:56.003 INFO:teuthology.orchestra.run.smithi148.stdout: Preparing : 1/1 2024-03-29T07:50:56.134 INFO:teuthology.orchestra.run.smithi037.stdout:Transaction test succeeded. 2024-03-29T07:50:56.138 INFO:teuthology.orchestra.run.smithi037.stdout:Running transaction 2024-03-29T07:50:56.832 INFO:teuthology.orchestra.run.smithi037.stdout: Preparing : 1/1 2024-03-29T07:50:56.971 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : yajl-2.1.0-12.el8.x86_64 1/13 2024-03-29T07:50:57.434 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : skopeo-2:1.14.3-0.1.module_el8+885+7da147f3.x86_64 2/13 2024-03-29T07:50:57.602 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 3/13 2024-03-29T07:50:57.799 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 4/13 2024-03-29T07:50:57.897 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : yajl-2.1.0-12.el8.x86_64 1/13 2024-03-29T07:50:58.023 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : criu-libs-3.15-3.module_el8.7.0+1216+b022c01d.x86_ 5/13 2024-03-29T07:50:58.169 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : skopeo-2:1.14.3-0.1.module_el8+885+7da147f3.x86_64 2/13 2024-03-29T07:50:58.335 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 3/13 2024-03-29T07:50:58.554 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 4/13 2024-03-29T07:50:58.615 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : aardvark-dns-2:1.10.0-1.module_el8+871+552be958.x8 6/13 2024-03-29T07:50:58.796 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : criu-libs-3.15-3.module_el8.7.0+1216+b022c01d.x86_ 5/13 2024-03-29T07:50:59.398 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : aardvark-dns-2:1.10.0-1.module_el8+871+552be958.x8 6/13 2024-03-29T07:50:59.692 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : netavark-2:1.10.3-1.module_el8+951+32019cde.x86_64 7/13 2024-03-29T07:50:59.963 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : buildah-2:1.33.6-2.module_el8+976+11360731.x86_64 8/13 2024-03-29T07:51:00.236 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : crun-1.14.3-2.module_el8+968+fbb249c7.x86_64 9/13 2024-03-29T07:51:00.484 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : netavark-2:1.10.3-1.module_el8+951+32019cde.x86_64 7/13 2024-03-29T07:51:00.661 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-podman-4.9.0-1.module_el8+847+7863d4e6.noa 10/13 2024-03-29T07:51:00.745 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : buildah-2:1.33.6-2.module_el8+976+11360731.x86_64 8/13 2024-03-29T07:51:00.906 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : toolbox-0.0.99.5-2.module_el8+971+3d3df00d.x86_64 11/13 2024-03-29T07:51:01.013 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : crun-1.14.3-2.module_el8+968+fbb249c7.x86_64 9/13 2024-03-29T07:51:01.124 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : udica-0.2.6-20.module_el8+804+f131391c.noarch 12/13 2024-03-29T07:51:01.357 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : cockpit-podman-84.1-1.module_el8+977+8ab54275.noar 13/13 2024-03-29T07:51:01.501 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-podman-4.9.0-1.module_el8+847+7863d4e6.noa 10/13 2024-03-29T07:51:01.737 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : toolbox-0.0.99.5-2.module_el8+971+3d3df00d.x86_64 11/13 2024-03-29T07:51:01.945 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : udica-0.2.6-20.module_el8+804+f131391c.noarch 12/13 2024-03-29T07:51:02.012 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: cockpit-podman-84.1-1.module_el8+977+8ab54275.noar 13/13 2024-03-29T07:51:02.012 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : aardvark-dns-2:1.10.0-1.module_el8+871+552be958.x8 1/13 2024-03-29T07:51:02.012 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : buildah-2:1.33.6-2.module_el8+976+11360731.x86_64 2/13 2024-03-29T07:51:02.012 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : cockpit-podman-84.1-1.module_el8+977+8ab54275.noar 3/13 2024-03-29T07:51:02.012 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : criu-libs-3.15-3.module_el8.7.0+1216+b022c01d.x86_ 4/13 2024-03-29T07:51:02.012 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : crun-1.14.3-2.module_el8+968+fbb249c7.x86_64 5/13 2024-03-29T07:51:02.013 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : netavark-2:1.10.3-1.module_el8+951+32019cde.x86_64 6/13 2024-03-29T07:51:02.013 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-podman-4.9.0-1.module_el8+847+7863d4e6.noa 7/13 2024-03-29T07:51:02.013 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 8/13 2024-03-29T07:51:02.013 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 9/13 2024-03-29T07:51:02.013 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : skopeo-2:1.14.3-0.1.module_el8+885+7da147f3.x86_64 10/13 2024-03-29T07:51:02.013 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : toolbox-0.0.99.5-2.module_el8+971+3d3df00d.x86_64 11/13 2024-03-29T07:51:02.013 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : udica-0.2.6-20.module_el8+804+f131391c.noarch 12/13 2024-03-29T07:51:02.140 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : cockpit-podman-84.1-1.module_el8+977+8ab54275.noar 13/13 2024-03-29T07:51:02.607 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : yajl-2.1.0-12.el8.x86_64 13/13 2024-03-29T07:51:02.607 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T07:51:02.607 INFO:teuthology.orchestra.run.smithi148.stdout:Installed: 2024-03-29T07:51:02.607 INFO:teuthology.orchestra.run.smithi148.stdout: aardvark-dns-2:1.10.0-1.module_el8+871+552be958.x86_64 2024-03-29T07:51:02.607 INFO:teuthology.orchestra.run.smithi148.stdout: buildah-2:1.33.6-2.module_el8+976+11360731.x86_64 2024-03-29T07:51:02.607 INFO:teuthology.orchestra.run.smithi148.stdout: cockpit-podman-84.1-1.module_el8+977+8ab54275.noarch 2024-03-29T07:51:02.607 INFO:teuthology.orchestra.run.smithi148.stdout: criu-libs-3.15-3.module_el8.7.0+1216+b022c01d.x86_64 2024-03-29T07:51:02.607 INFO:teuthology.orchestra.run.smithi148.stdout: crun-1.14.3-2.module_el8+968+fbb249c7.x86_64 2024-03-29T07:51:02.607 INFO:teuthology.orchestra.run.smithi148.stdout: netavark-2:1.10.3-1.module_el8+951+32019cde.x86_64 2024-03-29T07:51:02.607 INFO:teuthology.orchestra.run.smithi148.stdout: python3-podman-4.9.0-1.module_el8+847+7863d4e6.noarch 2024-03-29T07:51:02.608 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2024-03-29T07:51:02.608 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pyxdg-0.25-16.el8.noarch 2024-03-29T07:51:02.608 INFO:teuthology.orchestra.run.smithi148.stdout: skopeo-2:1.14.3-0.1.module_el8+885+7da147f3.x86_64 2024-03-29T07:51:02.608 INFO:teuthology.orchestra.run.smithi148.stdout: toolbox-0.0.99.5-2.module_el8+971+3d3df00d.x86_64 2024-03-29T07:51:02.608 INFO:teuthology.orchestra.run.smithi148.stdout: udica-0.2.6-20.module_el8+804+f131391c.noarch 2024-03-29T07:51:02.608 INFO:teuthology.orchestra.run.smithi148.stdout: yajl-2.1.0-12.el8.x86_64 2024-03-29T07:51:02.608 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T07:51:02.608 INFO:teuthology.orchestra.run.smithi148.stdout:Complete! 2024-03-29T07:51:02.809 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: cockpit-podman-84.1-1.module_el8+977+8ab54275.noar 13/13 2024-03-29T07:51:02.810 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : aardvark-dns-2:1.10.0-1.module_el8+871+552be958.x8 1/13 2024-03-29T07:51:02.810 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : buildah-2:1.33.6-2.module_el8+976+11360731.x86_64 2/13 2024-03-29T07:51:02.810 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : cockpit-podman-84.1-1.module_el8+977+8ab54275.noar 3/13 2024-03-29T07:51:02.810 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : criu-libs-3.15-3.module_el8.7.0+1216+b022c01d.x86_ 4/13 2024-03-29T07:51:02.810 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : crun-1.14.3-2.module_el8+968+fbb249c7.x86_64 5/13 2024-03-29T07:51:02.810 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : netavark-2:1.10.3-1.module_el8+951+32019cde.x86_64 6/13 2024-03-29T07:51:02.810 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-podman-4.9.0-1.module_el8+847+7863d4e6.noa 7/13 2024-03-29T07:51:02.810 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 8/13 2024-03-29T07:51:02.810 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 9/13 2024-03-29T07:51:02.810 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : skopeo-2:1.14.3-0.1.module_el8+885+7da147f3.x86_64 10/13 2024-03-29T07:51:02.810 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : toolbox-0.0.99.5-2.module_el8+971+3d3df00d.x86_64 11/13 2024-03-29T07:51:02.810 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : udica-0.2.6-20.module_el8+804+f131391c.noarch 12/13 2024-03-29T07:51:02.822 DEBUG:teuthology.parallel:result is None 2024-03-29T07:51:03.391 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : yajl-2.1.0-12.el8.x86_64 13/13 2024-03-29T07:51:03.391 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T07:51:03.391 INFO:teuthology.orchestra.run.smithi037.stdout:Installed: 2024-03-29T07:51:03.391 INFO:teuthology.orchestra.run.smithi037.stdout: aardvark-dns-2:1.10.0-1.module_el8+871+552be958.x86_64 2024-03-29T07:51:03.391 INFO:teuthology.orchestra.run.smithi037.stdout: buildah-2:1.33.6-2.module_el8+976+11360731.x86_64 2024-03-29T07:51:03.391 INFO:teuthology.orchestra.run.smithi037.stdout: cockpit-podman-84.1-1.module_el8+977+8ab54275.noarch 2024-03-29T07:51:03.392 INFO:teuthology.orchestra.run.smithi037.stdout: criu-libs-3.15-3.module_el8.7.0+1216+b022c01d.x86_64 2024-03-29T07:51:03.392 INFO:teuthology.orchestra.run.smithi037.stdout: crun-1.14.3-2.module_el8+968+fbb249c7.x86_64 2024-03-29T07:51:03.392 INFO:teuthology.orchestra.run.smithi037.stdout: netavark-2:1.10.3-1.module_el8+951+32019cde.x86_64 2024-03-29T07:51:03.392 INFO:teuthology.orchestra.run.smithi037.stdout: python3-podman-4.9.0-1.module_el8+847+7863d4e6.noarch 2024-03-29T07:51:03.392 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2024-03-29T07:51:03.392 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pyxdg-0.25-16.el8.noarch 2024-03-29T07:51:03.392 INFO:teuthology.orchestra.run.smithi037.stdout: skopeo-2:1.14.3-0.1.module_el8+885+7da147f3.x86_64 2024-03-29T07:51:03.392 INFO:teuthology.orchestra.run.smithi037.stdout: toolbox-0.0.99.5-2.module_el8+971+3d3df00d.x86_64 2024-03-29T07:51:03.392 INFO:teuthology.orchestra.run.smithi037.stdout: udica-0.2.6-20.module_el8+804+f131391c.noarch 2024-03-29T07:51:03.392 INFO:teuthology.orchestra.run.smithi037.stdout: yajl-2.1.0-12.el8.x86_64 2024-03-29T07:51:03.392 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T07:51:03.392 INFO:teuthology.orchestra.run.smithi037.stdout:Complete! 2024-03-29T07:51:03.604 DEBUG:teuthology.parallel:result is None 2024-03-29T07:51:03.604 INFO:teuthology.run_tasks:Running task install... 2024-03-29T07:51:03.615 DEBUG:teuthology.task.install:project ceph 2024-03-29T07:51:03.615 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'd0e392d9aee635f2b65768c460c7a151ead7a5e2'}} 2024-03-29T07:51:03.615 DEBUG:teuthology.task.install:config {'branch': 'quincy', 'exclude_packages': ['ceph-volume'], 'flavor': 'default', 'sha1': 'd0e392d9aee635f2b65768c460c7a151ead7a5e2'} 2024-03-29T07:51:03.615 INFO:teuthology.task.install:Using flavor: default 2024-03-29T07:51:03.624 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-03-29T07:51:03.624 INFO:teuthology.task.install:extra packages: [] 2024-03-29T07:51:03.624 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': 'quincy', 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': ['ceph-volume'], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd0e392d9aee635f2b65768c460c7a151ead7a5e2', 'tag': None, 'wait_for_package': False} 2024-03-29T07:51:03.625 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-03-29T07:51:03.625 INFO:teuthology.packaging:ref: None 2024-03-29T07:51:03.625 INFO:teuthology.packaging:tag: None 2024-03-29T07:51:03.626 INFO:teuthology.packaging:branch: quincy 2024-03-29T07:51:03.626 INFO:teuthology.packaging:sha1: d0e392d9aee635f2b65768c460c7a151ead7a5e2 2024-03-29T07:51:03.630 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=quincy 2024-03-29T07:51:03.634 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': 'quincy', 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': ['ceph-volume'], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd0e392d9aee635f2b65768c460c7a151ead7a5e2', 'tag': None, 'wait_for_package': False} 2024-03-29T07:51:03.634 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-03-29T07:51:03.634 INFO:teuthology.packaging:ref: None 2024-03-29T07:51:03.634 INFO:teuthology.packaging:tag: None 2024-03-29T07:51:03.634 INFO:teuthology.packaging:branch: quincy 2024-03-29T07:51:03.635 INFO:teuthology.packaging:sha1: d0e392d9aee635f2b65768c460c7a151ead7a5e2 2024-03-29T07:51:03.635 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=quincy 2024-03-29T07:51:03.890 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/quincy/4213e17c4de67016ed14f55d22503f4923b3a901/centos/8/flavors/default/ 2024-03-29T07:51:03.890 INFO:teuthology.task.install.rpm:Package version is 17.2.7-789.g4213e17c 2024-03-29T07:51:03.954 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/quincy/4213e17c4de67016ed14f55d22503f4923b3a901/centos/8/flavors/default/ 2024-03-29T07:51:03.954 INFO:teuthology.task.install.rpm:Package version is 17.2.7-789.g4213e17c 2024-03-29T07:51:04.061 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/quincy/4213e17c4de67016ed14f55d22503f4923b3a901/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/quincy/4213e17c4de67016ed14f55d22503f4923b3a901/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/quincy/4213e17c4de67016ed14f55d22503f4923b3a901/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-29T07:51:04.062 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-03-29T07:51:04.062 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-29T07:51:04.131 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/quincy/4213e17c4de67016ed14f55d22503f4923b3a901/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/quincy/4213e17c4de67016ed14f55d22503f4923b3a901/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/quincy/4213e17c4de67016ed14f55d22503f4923b3a901/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-29T07:51:04.131 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-03-29T07:51:04.131 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-29T07:51:04.133 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-03-29T07:51:04.133 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-03-29T07:51:04.133 INFO:teuthology.packaging:ref: None 2024-03-29T07:51:04.134 INFO:teuthology.packaging:tag: None 2024-03-29T07:51:04.134 INFO:teuthology.packaging:branch: quincy 2024-03-29T07:51:04.134 INFO:teuthology.packaging:sha1: d0e392d9aee635f2b65768c460c7a151ead7a5e2 2024-03-29T07:51:04.134 DEBUG:teuthology.orchestra.run.smithi037:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;ref/quincy/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-29T07:51:04.206 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-03-29T07:51:04.206 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-03-29T07:51:04.207 INFO:teuthology.packaging:ref: None 2024-03-29T07:51:04.207 INFO:teuthology.packaging:tag: None 2024-03-29T07:51:04.207 INFO:teuthology.packaging:branch: quincy 2024-03-29T07:51:04.207 INFO:teuthology.packaging:sha1: d0e392d9aee635f2b65768c460c7a151ead7a5e2 2024-03-29T07:51:04.207 DEBUG:teuthology.orchestra.run.smithi148:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;ref/quincy/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-29T07:51:04.242 DEBUG:teuthology.orchestra.run.smithi037:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-29T07:51:04.315 DEBUG:teuthology.orchestra.run.smithi148:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-29T07:51:04.334 DEBUG:teuthology.orchestra.run.smithi037:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-29T07:51:04.401 INFO:teuthology.orchestra.run.smithi037.stdout:check_obsoletes = 1 2024-03-29T07:51:04.404 DEBUG:teuthology.orchestra.run.smithi037:> sudo yum clean all 2024-03-29T07:51:04.406 DEBUG:teuthology.orchestra.run.smithi148:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-29T07:51:04.473 INFO:teuthology.orchestra.run.smithi148.stdout:check_obsoletes = 1 2024-03-29T07:51:04.476 DEBUG:teuthology.orchestra.run.smithi148:> sudo yum clean all 2024-03-29T07:51:04.770 INFO:teuthology.orchestra.run.smithi037.stdout:76 files removed 2024-03-29T07:51:04.807 DEBUG:teuthology.orchestra.run.smithi037:> sudo yum -y install ceph-radosgw 2024-03-29T07:51:04.817 INFO:teuthology.orchestra.run.smithi148.stdout:76 files removed 2024-03-29T07:51:04.853 DEBUG:teuthology.orchestra.run.smithi148:> sudo yum -y install ceph-radosgw 2024-03-29T07:51:05.731 INFO:teuthology.orchestra.run.smithi037.stdout:ceph packages for x86_64 223 kB/s | 79 kB 00:00 2024-03-29T07:51:05.768 INFO:teuthology.orchestra.run.smithi148.stdout:ceph packages for x86_64 211 kB/s | 79 kB 00:00 2024-03-29T07:51:06.098 INFO:teuthology.orchestra.run.smithi037.stdout:ceph noarch packages 28 kB/s | 9.1 kB 00:00 2024-03-29T07:51:06.160 INFO:teuthology.orchestra.run.smithi148.stdout:ceph noarch packages 26 kB/s | 9.1 kB 00:00 2024-03-29T07:51:06.391 INFO:teuthology.orchestra.run.smithi037.stdout:ceph source packages 5.5 kB/s | 1.5 kB 00:00 2024-03-29T07:51:06.553 INFO:teuthology.orchestra.run.smithi148.stdout:ceph source packages 4.0 kB/s | 1.5 kB 00:00 2024-03-29T07:51:08.683 INFO:teuthology.orchestra.run.smithi037.stdout:CentOS Stream 8 - AppStream 12 MB/s | 28 MB 00:02 2024-03-29T07:51:08.753 INFO:teuthology.orchestra.run.smithi148.stdout:CentOS Stream 8 - AppStream 13 MB/s | 28 MB 00:02 2024-03-29T07:51:15.003 INFO:teuthology.orchestra.run.smithi148.stdout:CentOS Stream 8 - BaseOS 12 MB/s | 10 MB 00:00 2024-03-29T07:51:15.292 INFO:teuthology.orchestra.run.smithi037.stdout:CentOS Stream 8 - BaseOS 10 MB/s | 10 MB 00:00 2024-03-29T07:51:17.104 INFO:teuthology.orchestra.run.smithi148.stdout:CentOS Stream 8 - Extras 44 kB/s | 18 kB 00:00 2024-03-29T07:51:17.413 INFO:teuthology.orchestra.run.smithi148.stdout:CentOS Stream 8 - Extras common packages 38 kB/s | 7.5 kB 00:00 2024-03-29T07:51:17.625 INFO:teuthology.orchestra.run.smithi037.stdout:CentOS Stream 8 - Extras 32 kB/s | 18 kB 00:00 2024-03-29T07:51:18.201 INFO:teuthology.orchestra.run.smithi037.stdout:CentOS Stream 8 - Extras common packages 17 kB/s | 7.5 kB 00:00 2024-03-29T07:51:18.506 INFO:teuthology.orchestra.run.smithi148.stdout:CentOS Stream 8 - PowerTools 4.9 MB/s | 4.9 MB 00:01 2024-03-29T07:51:19.294 INFO:teuthology.orchestra.run.smithi037.stdout:CentOS Stream 8 - PowerTools 5.0 MB/s | 4.9 MB 00:00 2024-03-29T07:51:19.616 INFO:teuthology.orchestra.run.smithi148.stdout:Copr repo for python3-asyncssh owned by ceph 26 kB/s | 3.5 kB 00:00 2024-03-29T07:51:20.462 INFO:teuthology.orchestra.run.smithi037.stdout:Copr repo for python3-asyncssh owned by ceph 27 kB/s | 3.5 kB 00:00 2024-03-29T07:51:21.680 INFO:teuthology.orchestra.run.smithi037.stdout:Extra Packages for Enterprise Linux 15 MB/s | 16 MB 00:01 2024-03-29T07:51:21.792 INFO:teuthology.orchestra.run.smithi148.stdout:Extra Packages for Enterprise Linux 7.8 MB/s | 16 MB 00:02 2024-03-29T07:51:26.300 INFO:teuthology.orchestra.run.smithi148.stdout:lab-extras 400 kB/s | 24 kB 00:00 2024-03-29T07:51:26.413 INFO:teuthology.orchestra.run.smithi037.stdout:lab-extras 454 kB/s | 24 kB 00:00 2024-03-29T07:51:30.032 INFO:teuthology.orchestra.run.smithi148.stdout:Dependencies resolved. 2024-03-29T07:51:30.034 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-03-29T07:51:30.035 INFO:teuthology.orchestra.run.smithi148.stdout: Package Arch Version Repository Size 2024-03-29T07:51:30.035 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-03-29T07:51:30.035 INFO:teuthology.orchestra.run.smithi148.stdout:Installing: 2024-03-29T07:51:30.035 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-radosgw x86_64 2:17.2.7-789.g4213e17c.el8 ceph 17 M 2024-03-29T07:51:30.035 INFO:teuthology.orchestra.run.smithi148.stdout:Upgrading: 2024-03-29T07:51:30.035 INFO:teuthology.orchestra.run.smithi148.stdout: librados2 x86_64 2:17.2.7-789.g4213e17c.el8 ceph 3.5 M 2024-03-29T07:51:30.035 INFO:teuthology.orchestra.run.smithi148.stdout: librbd1 x86_64 2:17.2.7-789.g4213e17c.el8 ceph 3.6 M 2024-03-29T07:51:30.035 INFO:teuthology.orchestra.run.smithi148.stdout:Installing dependencies: 2024-03-29T07:51:30.035 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-base x86_64 2:17.2.7-789.g4213e17c.el8 ceph 5.3 M 2024-03-29T07:51:30.035 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-common x86_64 2:17.2.7-789.g4213e17c.el8 ceph 24 M 2024-03-29T07:51:30.035 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-selinux x86_64 2:17.2.7-789.g4213e17c.el8 ceph 24 k 2024-03-29T07:51:30.036 INFO:teuthology.orchestra.run.smithi148.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2024-03-29T07:51:30.036 INFO:teuthology.orchestra.run.smithi148.stdout: libcephfs2 x86_64 2:17.2.7-789.g4213e17c.el8 ceph 798 k 2024-03-29T07:51:30.036 INFO:teuthology.orchestra.run.smithi148.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2024-03-29T07:51:30.036 INFO:teuthology.orchestra.run.smithi148.stdout: librabbitmq x86_64 0.9.0-5.el8 baseos 47 k 2024-03-29T07:51:30.036 INFO:teuthology.orchestra.run.smithi148.stdout: libradosstriper1 x86_64 2:17.2.7-789.g4213e17c.el8 ceph 505 k 2024-03-29T07:51:30.036 INFO:teuthology.orchestra.run.smithi148.stdout: librdkafka x86_64 1.6.1-1.el8 appstream 677 k 2024-03-29T07:51:30.036 INFO:teuthology.orchestra.run.smithi148.stdout: librgw2 x86_64 2:17.2.7-789.g4213e17c.el8 ceph 5.6 M 2024-03-29T07:51:30.036 INFO:teuthology.orchestra.run.smithi148.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2024-03-29T07:51:30.036 INFO:teuthology.orchestra.run.smithi148.stdout: lttng-ust x86_64 2.8.1-11.el8 appstream 259 k 2024-03-29T07:51:30.036 INFO:teuthology.orchestra.run.smithi148.stdout: python3-ceph-argparse x86_64 2:17.2.7-789.g4213e17c.el8 ceph 45 k 2024-03-29T07:51:30.036 INFO:teuthology.orchestra.run.smithi148.stdout: python3-ceph-common x86_64 2:17.2.7-789.g4213e17c.el8 ceph 117 k 2024-03-29T07:51:30.036 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cephfs x86_64 2:17.2.7-789.g4213e17c.el8 ceph 190 k 2024-03-29T07:51:30.036 INFO:teuthology.orchestra.run.smithi148.stdout: python3-prettytable noarch 0.7.2-14.el8 appstream 44 k 2024-03-29T07:51:30.036 INFO:teuthology.orchestra.run.smithi148.stdout: python3-rados x86_64 2:17.2.7-789.g4213e17c.el8 ceph 370 k 2024-03-29T07:51:30.037 INFO:teuthology.orchestra.run.smithi148.stdout: python3-rbd x86_64 2:17.2.7-789.g4213e17c.el8 ceph 358 k 2024-03-29T07:51:30.037 INFO:teuthology.orchestra.run.smithi148.stdout: python3-rgw x86_64 2:17.2.7-789.g4213e17c.el8 ceph 106 k 2024-03-29T07:51:30.037 INFO:teuthology.orchestra.run.smithi148.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2024-03-29T07:51:30.037 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T07:51:30.037 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction Summary 2024-03-29T07:51:30.037 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-03-29T07:51:30.037 INFO:teuthology.orchestra.run.smithi148.stdout:Install 21 Packages 2024-03-29T07:51:30.037 INFO:teuthology.orchestra.run.smithi148.stdout:Upgrade 2 Packages 2024-03-29T07:51:30.037 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T07:51:30.038 INFO:teuthology.orchestra.run.smithi148.stdout:Total download size: 64 M 2024-03-29T07:51:30.038 INFO:teuthology.orchestra.run.smithi148.stdout:Downloading Packages: 2024-03-29T07:51:30.562 INFO:teuthology.orchestra.run.smithi037.stdout:Dependencies resolved. 2024-03-29T07:51:30.565 INFO:teuthology.orchestra.run.smithi037.stdout:================================================================================ 2024-03-29T07:51:30.565 INFO:teuthology.orchestra.run.smithi037.stdout: Package Arch Version Repository Size 2024-03-29T07:51:30.565 INFO:teuthology.orchestra.run.smithi037.stdout:================================================================================ 2024-03-29T07:51:30.566 INFO:teuthology.orchestra.run.smithi037.stdout:Installing: 2024-03-29T07:51:30.566 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-radosgw x86_64 2:17.2.7-789.g4213e17c.el8 ceph 17 M 2024-03-29T07:51:30.566 INFO:teuthology.orchestra.run.smithi037.stdout:Upgrading: 2024-03-29T07:51:30.566 INFO:teuthology.orchestra.run.smithi037.stdout: librados2 x86_64 2:17.2.7-789.g4213e17c.el8 ceph 3.5 M 2024-03-29T07:51:30.566 INFO:teuthology.orchestra.run.smithi037.stdout: librbd1 x86_64 2:17.2.7-789.g4213e17c.el8 ceph 3.6 M 2024-03-29T07:51:30.566 INFO:teuthology.orchestra.run.smithi037.stdout:Installing dependencies: 2024-03-29T07:51:30.566 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-base x86_64 2:17.2.7-789.g4213e17c.el8 ceph 5.3 M 2024-03-29T07:51:30.566 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-common x86_64 2:17.2.7-789.g4213e17c.el8 ceph 24 M 2024-03-29T07:51:30.566 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-selinux x86_64 2:17.2.7-789.g4213e17c.el8 ceph 24 k 2024-03-29T07:51:30.566 INFO:teuthology.orchestra.run.smithi037.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2024-03-29T07:51:30.566 INFO:teuthology.orchestra.run.smithi037.stdout: libcephfs2 x86_64 2:17.2.7-789.g4213e17c.el8 ceph 798 k 2024-03-29T07:51:30.566 INFO:teuthology.orchestra.run.smithi037.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2024-03-29T07:51:30.566 INFO:teuthology.orchestra.run.smithi037.stdout: librabbitmq x86_64 0.9.0-5.el8 baseos 47 k 2024-03-29T07:51:30.567 INFO:teuthology.orchestra.run.smithi037.stdout: libradosstriper1 x86_64 2:17.2.7-789.g4213e17c.el8 ceph 505 k 2024-03-29T07:51:30.567 INFO:teuthology.orchestra.run.smithi037.stdout: librdkafka x86_64 1.6.1-1.el8 appstream 677 k 2024-03-29T07:51:30.567 INFO:teuthology.orchestra.run.smithi037.stdout: librgw2 x86_64 2:17.2.7-789.g4213e17c.el8 ceph 5.6 M 2024-03-29T07:51:30.567 INFO:teuthology.orchestra.run.smithi037.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2024-03-29T07:51:30.567 INFO:teuthology.orchestra.run.smithi037.stdout: lttng-ust x86_64 2.8.1-11.el8 appstream 259 k 2024-03-29T07:51:30.567 INFO:teuthology.orchestra.run.smithi037.stdout: python3-ceph-argparse x86_64 2:17.2.7-789.g4213e17c.el8 ceph 45 k 2024-03-29T07:51:30.567 INFO:teuthology.orchestra.run.smithi037.stdout: python3-ceph-common x86_64 2:17.2.7-789.g4213e17c.el8 ceph 117 k 2024-03-29T07:51:30.567 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cephfs x86_64 2:17.2.7-789.g4213e17c.el8 ceph 190 k 2024-03-29T07:51:30.567 INFO:teuthology.orchestra.run.smithi037.stdout: python3-prettytable noarch 0.7.2-14.el8 appstream 44 k 2024-03-29T07:51:30.567 INFO:teuthology.orchestra.run.smithi037.stdout: python3-rados x86_64 2:17.2.7-789.g4213e17c.el8 ceph 370 k 2024-03-29T07:51:30.567 INFO:teuthology.orchestra.run.smithi037.stdout: python3-rbd x86_64 2:17.2.7-789.g4213e17c.el8 ceph 358 k 2024-03-29T07:51:30.567 INFO:teuthology.orchestra.run.smithi037.stdout: python3-rgw x86_64 2:17.2.7-789.g4213e17c.el8 ceph 106 k 2024-03-29T07:51:30.567 INFO:teuthology.orchestra.run.smithi037.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2024-03-29T07:51:30.567 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T07:51:30.568 INFO:teuthology.orchestra.run.smithi037.stdout:Transaction Summary 2024-03-29T07:51:30.568 INFO:teuthology.orchestra.run.smithi037.stdout:================================================================================ 2024-03-29T07:51:30.568 INFO:teuthology.orchestra.run.smithi037.stdout:Install 21 Packages 2024-03-29T07:51:30.568 INFO:teuthology.orchestra.run.smithi037.stdout:Upgrade 2 Packages 2024-03-29T07:51:30.568 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T07:51:30.568 INFO:teuthology.orchestra.run.smithi037.stdout:Total download size: 64 M 2024-03-29T07:51:30.568 INFO:teuthology.orchestra.run.smithi037.stdout:Downloading Packages: 2024-03-29T07:51:31.393 INFO:teuthology.orchestra.run.smithi148.stdout:(1/23): ceph-radosgw-17.2.7-789.g4213e17c.el8.x 17 MB/s | 17 MB 00:01 2024-03-29T07:51:31.452 INFO:teuthology.orchestra.run.smithi148.stdout:(2/23): ceph-selinux-17.2.7-789.g4213e17c.el8.x 416 kB/s | 24 kB 00:00 2024-03-29T07:51:31.552 INFO:teuthology.orchestra.run.smithi148.stdout:(3/23): libcephfs2-17.2.7-789.g4213e17c.el8.x86 7.8 MB/s | 798 kB 00:00 2024-03-29T07:51:31.652 INFO:teuthology.orchestra.run.smithi148.stdout:(4/23): libradosstriper1-17.2.7-789.g4213e17c.e 4.9 MB/s | 505 kB 00:00 2024-03-29T07:51:32.261 INFO:teuthology.orchestra.run.smithi148.stdout:(5/23): librgw2-17.2.7-789.g4213e17c.el8.x86_64 9.2 MB/s | 5.6 MB 00:00 2024-03-29T07:51:32.336 INFO:teuthology.orchestra.run.smithi148.stdout:(6/23): python3-ceph-argparse-17.2.7-789.g4213e 599 kB/s | 45 kB 00:00 2024-03-29T07:51:32.416 INFO:teuthology.orchestra.run.smithi148.stdout:(7/23): python3-ceph-common-17.2.7-789.g4213e17 1.4 MB/s | 117 kB 00:00 2024-03-29T07:51:32.470 INFO:teuthology.orchestra.run.smithi148.stdout:(8/23): python3-cephfs-17.2.7-789.g4213e17c.el8 3.5 MB/s | 190 kB 00:00 2024-03-29T07:51:32.506 INFO:teuthology.orchestra.run.smithi037.stdout:(1/23): ceph-radosgw-17.2.7-789.g4213e17c.el8.x 12 MB/s | 17 MB 00:01 2024-03-29T07:51:32.528 INFO:teuthology.orchestra.run.smithi148.stdout:(9/23): python3-rados-17.2.7-789.g4213e17c.el8. 6.2 MB/s | 370 kB 00:00 2024-03-29T07:51:32.556 INFO:teuthology.orchestra.run.smithi037.stdout:(2/23): ceph-selinux-17.2.7-789.g4213e17c.el8.x 490 kB/s | 24 kB 00:00 2024-03-29T07:51:32.587 INFO:teuthology.orchestra.run.smithi148.stdout:(10/23): python3-rbd-17.2.7-789.g4213e17c.el8.x 6.0 MB/s | 358 kB 00:00 2024-03-29T07:51:32.632 INFO:teuthology.orchestra.run.smithi037.stdout:(3/23): libcephfs2-17.2.7-789.g4213e17c.el8.x86 10 MB/s | 798 kB 00:00 2024-03-29T07:51:32.637 INFO:teuthology.orchestra.run.smithi148.stdout:(11/23): python3-rgw-17.2.7-789.g4213e17c.el8.x 2.1 MB/s | 106 kB 00:00 2024-03-29T07:51:32.699 INFO:teuthology.orchestra.run.smithi037.stdout:(4/23): libradosstriper1-17.2.7-789.g4213e17c.e 7.4 MB/s | 505 kB 00:00 2024-03-29T07:51:32.846 INFO:teuthology.orchestra.run.smithi148.stdout:(12/23): librdkafka-1.6.1-1.el8.x86_64.rpm 3.2 MB/s | 677 kB 00:00 2024-03-29T07:51:33.066 INFO:teuthology.orchestra.run.smithi037.stdout:(5/23): librgw2-17.2.7-789.g4213e17c.el8.x86_64 15 MB/s | 5.6 MB 00:00 2024-03-29T07:51:33.116 INFO:teuthology.orchestra.run.smithi037.stdout:(6/23): python3-ceph-argparse-17.2.7-789.g4213e 897 kB/s | 45 kB 00:00 2024-03-29T07:51:33.166 INFO:teuthology.orchestra.run.smithi037.stdout:(7/23): python3-ceph-common-17.2.7-789.g4213e17 2.3 MB/s | 117 kB 00:00 2024-03-29T07:51:33.225 INFO:teuthology.orchestra.run.smithi037.stdout:(8/23): python3-cephfs-17.2.7-789.g4213e17c.el8 3.2 MB/s | 190 kB 00:00 2024-03-29T07:51:33.292 INFO:teuthology.orchestra.run.smithi037.stdout:(9/23): python3-rados-17.2.7-789.g4213e17c.el8. 5.4 MB/s | 370 kB 00:00 2024-03-29T07:51:33.350 INFO:teuthology.orchestra.run.smithi037.stdout:(10/23): python3-rbd-17.2.7-789.g4213e17c.el8.x 6.0 MB/s | 358 kB 00:00 2024-03-29T07:51:33.363 INFO:teuthology.orchestra.run.smithi148.stdout:(13/23): lttng-ust-2.8.1-11.el8.x86_64.rpm 502 kB/s | 259 kB 00:00 2024-03-29T07:51:33.392 INFO:teuthology.orchestra.run.smithi037.stdout:(11/23): python3-rgw-17.2.7-789.g4213e17c.el8.x 2.5 MB/s | 106 kB 00:00 2024-03-29T07:51:33.464 INFO:teuthology.orchestra.run.smithi148.stdout:(14/23): python3-prettytable-0.7.2-14.el8.noarc 440 kB/s | 44 kB 00:00 2024-03-29T07:51:33.543 INFO:teuthology.orchestra.run.smithi037.stdout:(12/23): ceph-base-17.2.7-789.g4213e17c.el8.x86 2.2 MB/s | 5.3 MB 00:02 2024-03-29T07:51:33.660 INFO:teuthology.orchestra.run.smithi037.stdout:(13/23): librdkafka-1.6.1-1.el8.x86_64.rpm 2.5 MB/s | 677 kB 00:00 2024-03-29T07:51:33.723 INFO:teuthology.orchestra.run.smithi148.stdout:(15/23): librabbitmq-0.9.0-5.el8.x86_64.rpm 182 kB/s | 47 kB 00:00 2024-03-29T07:51:33.726 INFO:teuthology.orchestra.run.smithi037.stdout:(14/23): lttng-ust-2.8.1-11.el8.x86_64.rpm 1.4 MB/s | 259 kB 00:00 2024-03-29T07:51:33.752 INFO:teuthology.orchestra.run.smithi037.stdout:(15/23): python3-prettytable-0.7.2-14.el8.noarc 478 kB/s | 44 kB 00:00 2024-03-29T07:51:33.823 INFO:teuthology.orchestra.run.smithi148.stdout:(16/23): gperftools-libs-2.7-9.el8.x86_64.rpm 3.0 MB/s | 306 kB 00:00 2024-03-29T07:51:33.856 INFO:teuthology.orchestra.run.smithi148.stdout:(17/23): liboath-2.6.2-3.el8.x86_64.rpm 1.8 MB/s | 59 kB 00:00 2024-03-29T07:51:33.898 INFO:teuthology.orchestra.run.smithi148.stdout:(18/23): libunwind-1.3.1-3.el8.x86_64.rpm 1.8 MB/s | 75 kB 00:00 2024-03-29T07:51:33.982 INFO:teuthology.orchestra.run.smithi148.stdout:(19/23): thrift-0.13.0-2.el8.x86_64.rpm 21 MB/s | 1.7 MB 00:00 2024-03-29T07:51:34.220 INFO:teuthology.orchestra.run.smithi037.stdout:(16/23): librabbitmq-0.9.0-5.el8.x86_64.rpm 95 kB/s | 47 kB 00:00 2024-03-29T07:51:34.415 INFO:teuthology.orchestra.run.smithi148.stdout:(20/23): librados2-17.2.7-789.g4213e17c.el8.x86 8.0 MB/s | 3.5 MB 00:00 2024-03-29T07:51:34.545 INFO:teuthology.orchestra.run.smithi037.stdout:(17/23): gperftools-libs-2.7-9.el8.x86_64.rpm 386 kB/s | 306 kB 00:00 2024-03-29T07:51:34.570 INFO:teuthology.orchestra.run.smithi037.stdout:(18/23): liboath-2.6.2-3.el8.x86_64.rpm 168 kB/s | 59 kB 00:00 2024-03-29T07:51:34.646 INFO:teuthology.orchestra.run.smithi037.stdout:(19/23): libunwind-1.3.1-3.el8.x86_64.rpm 751 kB/s | 75 kB 00:00 2024-03-29T07:51:34.791 INFO:teuthology.orchestra.run.smithi148.stdout:(21/23): ceph-common-17.2.7-789.g4213e17c.el8.x 5.4 MB/s | 24 MB 00:04 2024-03-29T07:51:34.858 INFO:teuthology.orchestra.run.smithi148.stdout:(22/23): librbd1-17.2.7-789.g4213e17c.el8.x86_6 8.1 MB/s | 3.6 MB 00:00 2024-03-29T07:51:34.999 INFO:teuthology.orchestra.run.smithi148.stdout:(23/23): ceph-base-17.2.7-789.g4213e17c.el8.x86 1.1 MB/s | 5.3 MB 00:04 2024-03-29T07:51:35.002 INFO:teuthology.orchestra.run.smithi148.stdout:-------------------------------------------------------------------------------- 2024-03-29T07:51:35.002 INFO:teuthology.orchestra.run.smithi148.stdout:Total 13 MB/s | 64 MB 00:04 2024-03-29T07:51:35.029 INFO:teuthology.orchestra.run.smithi037.stdout:(20/23): thrift-0.13.0-2.el8.x86_64.rpm 3.8 MB/s | 1.7 MB 00:00 2024-03-29T07:51:35.087 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction check 2024-03-29T07:51:35.137 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction check succeeded. 2024-03-29T07:51:35.137 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction test 2024-03-29T07:51:35.312 INFO:teuthology.orchestra.run.smithi037.stdout:(21/23): librados2-17.2.7-789.g4213e17c.el8.x86 5.2 MB/s | 3.5 MB 00:00 2024-03-29T07:51:35.371 INFO:teuthology.orchestra.run.smithi037.stdout:(22/23): librbd1-17.2.7-789.g4213e17c.el8.x86_6 11 MB/s | 3.6 MB 00:00 2024-03-29T07:51:35.548 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction test succeeded. 2024-03-29T07:51:35.551 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction 2024-03-29T07:51:36.649 INFO:teuthology.orchestra.run.smithi148.stdout: Preparing : 1/1 2024-03-29T07:51:36.903 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : liboath-2.6.2-3.el8.x86_64 1/25 2024-03-29T07:51:37.136 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 2/25 2024-03-29T07:51:37.593 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 2/25 2024-03-29T07:51:37.785 INFO:teuthology.orchestra.run.smithi148.stdout: Upgrading : librados2-2:17.2.7-789.g4213e17c.el8.x86_64 3/25 2024-03-29T07:51:37.885 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: librados2-2:17.2.7-789.g4213e17c.el8.x86_64 3/25 2024-03-29T07:51:38.155 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-rados-2:17.2.7-789.g4213e17c.el8.x86_64 4/25 2024-03-29T07:51:38.344 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libcephfs2-2:17.2.7-789.g4213e17c.el8.x86_64 5/25 2024-03-29T07:51:38.770 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: libcephfs2-2:17.2.7-789.g4213e17c.el8.x86_64 5/25 2024-03-29T07:51:38.977 INFO:teuthology.orchestra.run.smithi148.stdout: Upgrading : librbd1-2:17.2.7-789.g4213e17c.el8.x86_64 6/25 2024-03-29T07:51:39.222 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: librbd1-2:17.2.7-789.g4213e17c.el8.x86_64 6/25 2024-03-29T07:51:39.455 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : thrift-0.13.0-2.el8.x86_64 7/25 2024-03-29T07:51:39.666 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : librabbitmq-0.9.0-5.el8.x86_64 8/25 2024-03-29T07:51:40.030 INFO:teuthology.orchestra.run.smithi037.stdout:(23/23): ceph-common-17.2.7-789.g4213e17c.el8.x 2.7 MB/s | 24 MB 00:08 2024-03-29T07:51:40.033 INFO:teuthology.orchestra.run.smithi037.stdout:-------------------------------------------------------------------------------- 2024-03-29T07:51:40.033 INFO:teuthology.orchestra.run.smithi037.stdout:Total 6.8 MB/s | 64 MB 00:09 2024-03-29T07:51:40.128 INFO:teuthology.orchestra.run.smithi037.stdout:Running transaction check 2024-03-29T07:51:40.183 INFO:teuthology.orchestra.run.smithi037.stdout:Transaction check succeeded. 2024-03-29T07:51:40.183 INFO:teuthology.orchestra.run.smithi037.stdout:Running transaction test 2024-03-29T07:51:40.360 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : librdkafka-1.6.1-1.el8.x86_64 9/25 2024-03-29T07:51:40.518 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : librgw2-2:17.2.7-789.g4213e17c.el8.x86_64 10/25 2024-03-29T07:51:40.603 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: librgw2-2:17.2.7-789.g4213e17c.el8.x86_64 10/25 2024-03-29T07:51:40.603 INFO:teuthology.orchestra.run.smithi037.stdout:Transaction test succeeded. 2024-03-29T07:51:40.607 INFO:teuthology.orchestra.run.smithi037.stdout:Running transaction 2024-03-29T07:51:40.765 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-ceph-argparse-2:17.2.7-789.g4213e17c.el8.x 11/25 2024-03-29T07:51:40.947 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-cephfs-2:17.2.7-789.g4213e17c.el8.x86_64 12/25 2024-03-29T07:51:41.155 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-rgw-2:17.2.7-789.g4213e17c.el8.x86_64 13/25 2024-03-29T07:51:41.358 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-rbd-2:17.2.7-789.g4213e17c.el8.x86_64 14/25 2024-03-29T07:51:41.527 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libradosstriper1-2:17.2.7-789.g4213e17c.el8.x86_64 15/25 2024-03-29T07:51:41.593 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: libradosstriper1-2:17.2.7-789.g4213e17c.el8.x86_64 15/25 2024-03-29T07:51:41.679 INFO:teuthology.orchestra.run.smithi037.stdout: Preparing : 1/1 2024-03-29T07:51:41.832 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 16/25 2024-03-29T07:51:41.886 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : liboath-2.6.2-3.el8.x86_64 1/25 2024-03-29T07:51:41.994 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 17/25 2024-03-29T07:51:42.094 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 2/25 2024-03-29T07:51:42.217 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 18/25 2024-03-29T07:51:42.468 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-ceph-common-2:17.2.7-789.g4213e17c.el8.x86 19/25 2024-03-29T07:51:42.545 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 2/25 2024-03-29T07:51:42.727 INFO:teuthology.orchestra.run.smithi037.stdout: Upgrading : librados2-2:17.2.7-789.g4213e17c.el8.x86_64 3/25 2024-03-29T07:51:42.851 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: librados2-2:17.2.7-789.g4213e17c.el8.x86_64 3/25 2024-03-29T07:51:43.111 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-rados-2:17.2.7-789.g4213e17c.el8.x86_64 4/25 2024-03-29T07:51:43.294 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libcephfs2-2:17.2.7-789.g4213e17c.el8.x86_64 5/25 2024-03-29T07:51:43.760 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: libcephfs2-2:17.2.7-789.g4213e17c.el8.x86_64 5/25 2024-03-29T07:51:43.952 INFO:teuthology.orchestra.run.smithi037.stdout: Upgrading : librbd1-2:17.2.7-789.g4213e17c.el8.x86_64 6/25 2024-03-29T07:51:44.234 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: librbd1-2:17.2.7-789.g4213e17c.el8.x86_64 6/25 2024-03-29T07:51:44.477 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : thrift-0.13.0-2.el8.x86_64 7/25 2024-03-29T07:51:44.687 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : librabbitmq-0.9.0-5.el8.x86_64 8/25 2024-03-29T07:51:45.416 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : librdkafka-1.6.1-1.el8.x86_64 9/25 2024-03-29T07:51:45.432 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-common-2:17.2.7-789.g4213e17c.el8.x86_64 20/25 2024-03-29T07:51:45.569 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : librgw2-2:17.2.7-789.g4213e17c.el8.x86_64 10/25 2024-03-29T07:51:45.651 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: librgw2-2:17.2.7-789.g4213e17c.el8.x86_64 10/25 2024-03-29T07:51:45.685 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-common-2:17.2.7-789.g4213e17c.el8.x86_64 20/25 2024-03-29T07:51:45.791 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-ceph-argparse-2:17.2.7-789.g4213e17c.el8.x 11/25 2024-03-29T07:51:45.973 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-cephfs-2:17.2.7-789.g4213e17c.el8.x86_64 12/25 2024-03-29T07:51:46.184 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-rgw-2:17.2.7-789.g4213e17c.el8.x86_64 13/25 2024-03-29T07:51:46.355 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-common-2:17.2.7-789.g4213e17c.el8.x86_64 20/25 2024-03-29T07:51:46.461 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-rbd-2:17.2.7-789.g4213e17c.el8.x86_64 14/25 2024-03-29T07:51:46.627 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libradosstriper1-2:17.2.7-789.g4213e17c.el8.x86_64 15/25 2024-03-29T07:51:46.667 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-base-2:17.2.7-789.g4213e17c.el8.x86_64 21/25 2024-03-29T07:51:46.715 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: libradosstriper1-2:17.2.7-789.g4213e17c.el8.x86_64 15/25 2024-03-29T07:51:46.849 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-base-2:17.2.7-789.g4213e17c.el8.x86_64 21/25 2024-03-29T07:51:46.927 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 16/25 2024-03-29T07:51:47.084 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-selinux-2:17.2.7-789.g4213e17c.el8.x86_64 22/25 2024-03-29T07:51:47.096 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 17/25 2024-03-29T07:51:47.276 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 18/25 2024-03-29T07:51:47.519 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-ceph-common-2:17.2.7-789.g4213e17c.el8.x86 19/25 2024-03-29T07:51:50.670 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-common-2:17.2.7-789.g4213e17c.el8.x86_64 20/25 2024-03-29T07:51:51.044 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-common-2:17.2.7-789.g4213e17c.el8.x86_64 20/25 2024-03-29T07:51:51.738 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-common-2:17.2.7-789.g4213e17c.el8.x86_64 20/25 2024-03-29T07:51:52.032 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-base-2:17.2.7-789.g4213e17c.el8.x86_64 21/25 2024-03-29T07:51:52.224 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-base-2:17.2.7-789.g4213e17c.el8.x86_64 21/25 2024-03-29T07:51:52.434 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-selinux-2:17.2.7-789.g4213e17c.el8.x86_64 22/25 2024-03-29T07:52:07.909 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-selinux-2:17.2.7-789.g4213e17c.el8.x86_64 22/25 2024-03-29T07:52:07.909 INFO:teuthology.orchestra.run.smithi148.stdout:skipping the directory /sys 2024-03-29T07:52:07.909 INFO:teuthology.orchestra.run.smithi148.stdout:skipping the directory /proc 2024-03-29T07:52:07.909 INFO:teuthology.orchestra.run.smithi148.stdout:skipping the directory /dev 2024-03-29T07:52:07.909 INFO:teuthology.orchestra.run.smithi148.stdout:skipping the directory /run 2024-03-29T07:52:07.909 INFO:teuthology.orchestra.run.smithi148.stdout:skipping the directory /mnt 2024-03-29T07:52:07.909 INFO:teuthology.orchestra.run.smithi148.stdout:skipping the directory /var/tmp 2024-03-29T07:52:07.909 INFO:teuthology.orchestra.run.smithi148.stdout:skipping the directory /home 2024-03-29T07:52:07.909 INFO:teuthology.orchestra.run.smithi148.stdout:skipping the directory /tmp 2024-03-29T07:52:07.909 INFO:teuthology.orchestra.run.smithi148.stdout:skipping the directory /dev 2024-03-29T07:52:07.909 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T07:52:09.739 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-radosgw-2:17.2.7-789.g4213e17c.el8.x86_64 23/25 2024-03-29T07:52:09.830 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-radosgw-2:17.2.7-789.g4213e17c.el8.x86_64 23/25 2024-03-29T07:52:09.831 INFO:teuthology.orchestra.run.smithi148.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 24/25 2024-03-29T07:52:10.102 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 24/25 2024-03-29T07:52:10.103 INFO:teuthology.orchestra.run.smithi148.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 25/25 2024-03-29T07:52:11.093 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 25/25 2024-03-29T07:52:11.094 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-base-2:17.2.7-789.g4213e17c.el8.x86_64 1/25 2024-03-29T07:52:11.094 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-common-2:17.2.7-789.g4213e17c.el8.x86_64 2/25 2024-03-29T07:52:11.094 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-radosgw-2:17.2.7-789.g4213e17c.el8.x86_64 3/25 2024-03-29T07:52:11.094 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-selinux-2:17.2.7-789.g4213e17c.el8.x86_64 4/25 2024-03-29T07:52:11.094 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libcephfs2-2:17.2.7-789.g4213e17c.el8.x86_64 5/25 2024-03-29T07:52:11.094 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libradosstriper1-2:17.2.7-789.g4213e17c.el8.x86_64 6/25 2024-03-29T07:52:11.094 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : librgw2-2:17.2.7-789.g4213e17c.el8.x86_64 7/25 2024-03-29T07:52:11.094 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-ceph-argparse-2:17.2.7-789.g4213e17c.el8.x 8/25 2024-03-29T07:52:11.094 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-ceph-common-2:17.2.7-789.g4213e17c.el8.x86 9/25 2024-03-29T07:52:11.094 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-cephfs-2:17.2.7-789.g4213e17c.el8.x86_64 10/25 2024-03-29T07:52:11.095 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-rados-2:17.2.7-789.g4213e17c.el8.x86_64 11/25 2024-03-29T07:52:11.095 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-rbd-2:17.2.7-789.g4213e17c.el8.x86_64 12/25 2024-03-29T07:52:11.095 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-rgw-2:17.2.7-789.g4213e17c.el8.x86_64 13/25 2024-03-29T07:52:11.095 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : librdkafka-1.6.1-1.el8.x86_64 14/25 2024-03-29T07:52:11.095 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 15/25 2024-03-29T07:52:11.095 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 16/25 2024-03-29T07:52:11.095 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : librabbitmq-0.9.0-5.el8.x86_64 17/25 2024-03-29T07:52:11.095 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 18/25 2024-03-29T07:52:11.095 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 19/25 2024-03-29T07:52:11.095 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 20/25 2024-03-29T07:52:11.095 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 21/25 2024-03-29T07:52:11.095 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : librados2-2:17.2.7-789.g4213e17c.el8.x86_64 22/25 2024-03-29T07:52:11.095 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 23/25 2024-03-29T07:52:11.096 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : librbd1-2:17.2.7-789.g4213e17c.el8.x86_64 24/25 2024-03-29T07:52:11.810 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 25/25 2024-03-29T07:52:11.811 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T07:52:11.811 INFO:teuthology.orchestra.run.smithi148.stdout:Upgraded: 2024-03-29T07:52:11.811 INFO:teuthology.orchestra.run.smithi148.stdout: librados2-2:17.2.7-789.g4213e17c.el8.x86_64 2024-03-29T07:52:11.811 INFO:teuthology.orchestra.run.smithi148.stdout: librbd1-2:17.2.7-789.g4213e17c.el8.x86_64 2024-03-29T07:52:11.811 INFO:teuthology.orchestra.run.smithi148.stdout:Installed: 2024-03-29T07:52:11.811 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-base-2:17.2.7-789.g4213e17c.el8.x86_64 2024-03-29T07:52:11.811 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-common-2:17.2.7-789.g4213e17c.el8.x86_64 2024-03-29T07:52:11.811 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-radosgw-2:17.2.7-789.g4213e17c.el8.x86_64 2024-03-29T07:52:11.811 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-selinux-2:17.2.7-789.g4213e17c.el8.x86_64 2024-03-29T07:52:11.811 INFO:teuthology.orchestra.run.smithi148.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2024-03-29T07:52:11.811 INFO:teuthology.orchestra.run.smithi148.stdout: libcephfs2-2:17.2.7-789.g4213e17c.el8.x86_64 2024-03-29T07:52:11.811 INFO:teuthology.orchestra.run.smithi148.stdout: liboath-2.6.2-3.el8.x86_64 2024-03-29T07:52:11.811 INFO:teuthology.orchestra.run.smithi148.stdout: librabbitmq-0.9.0-5.el8.x86_64 2024-03-29T07:52:11.812 INFO:teuthology.orchestra.run.smithi148.stdout: libradosstriper1-2:17.2.7-789.g4213e17c.el8.x86_64 2024-03-29T07:52:11.812 INFO:teuthology.orchestra.run.smithi148.stdout: librdkafka-1.6.1-1.el8.x86_64 2024-03-29T07:52:11.812 INFO:teuthology.orchestra.run.smithi148.stdout: librgw2-2:17.2.7-789.g4213e17c.el8.x86_64 2024-03-29T07:52:11.812 INFO:teuthology.orchestra.run.smithi148.stdout: libunwind-1.3.1-3.el8.x86_64 2024-03-29T07:52:11.812 INFO:teuthology.orchestra.run.smithi148.stdout: lttng-ust-2.8.1-11.el8.x86_64 2024-03-29T07:52:11.812 INFO:teuthology.orchestra.run.smithi148.stdout: python3-ceph-argparse-2:17.2.7-789.g4213e17c.el8.x86_64 2024-03-29T07:52:11.812 INFO:teuthology.orchestra.run.smithi148.stdout: python3-ceph-common-2:17.2.7-789.g4213e17c.el8.x86_64 2024-03-29T07:52:11.812 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cephfs-2:17.2.7-789.g4213e17c.el8.x86_64 2024-03-29T07:52:11.812 INFO:teuthology.orchestra.run.smithi148.stdout: python3-prettytable-0.7.2-14.el8.noarch 2024-03-29T07:52:11.812 INFO:teuthology.orchestra.run.smithi148.stdout: python3-rados-2:17.2.7-789.g4213e17c.el8.x86_64 2024-03-29T07:52:11.812 INFO:teuthology.orchestra.run.smithi148.stdout: python3-rbd-2:17.2.7-789.g4213e17c.el8.x86_64 2024-03-29T07:52:11.812 INFO:teuthology.orchestra.run.smithi148.stdout: python3-rgw-2:17.2.7-789.g4213e17c.el8.x86_64 2024-03-29T07:52:11.812 INFO:teuthology.orchestra.run.smithi148.stdout: thrift-0.13.0-2.el8.x86_64 2024-03-29T07:52:11.813 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T07:52:11.813 INFO:teuthology.orchestra.run.smithi148.stdout:Complete! 2024-03-29T07:52:12.138 DEBUG:teuthology.orchestra.run.smithi148:> sudo yum -y install ceph-test 2024-03-29T07:52:13.074 INFO:teuthology.orchestra.run.smithi148.stdout:Last metadata expiration check: 0:00:47 ago on Fri 29 Mar 2024 07:51:26 AM UTC. 2024-03-29T07:52:14.264 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-selinux-2:17.2.7-789.g4213e17c.el8.x86_64 22/25 2024-03-29T07:52:14.264 INFO:teuthology.orchestra.run.smithi037.stdout:skipping the directory /sys 2024-03-29T07:52:14.264 INFO:teuthology.orchestra.run.smithi037.stdout:skipping the directory /proc 2024-03-29T07:52:14.264 INFO:teuthology.orchestra.run.smithi037.stdout:skipping the directory /dev 2024-03-29T07:52:14.264 INFO:teuthology.orchestra.run.smithi037.stdout:skipping the directory /run 2024-03-29T07:52:14.264 INFO:teuthology.orchestra.run.smithi037.stdout:skipping the directory /mnt 2024-03-29T07:52:14.265 INFO:teuthology.orchestra.run.smithi037.stdout:skipping the directory /var/tmp 2024-03-29T07:52:14.265 INFO:teuthology.orchestra.run.smithi037.stdout:skipping the directory /home 2024-03-29T07:52:14.265 INFO:teuthology.orchestra.run.smithi037.stdout:skipping the directory /tmp 2024-03-29T07:52:14.265 INFO:teuthology.orchestra.run.smithi037.stdout:skipping the directory /dev 2024-03-29T07:52:14.265 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T07:52:14.826 INFO:teuthology.orchestra.run.smithi148.stdout:Dependencies resolved. 2024-03-29T07:52:14.828 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-03-29T07:52:14.828 INFO:teuthology.orchestra.run.smithi148.stdout: Package Arch Version Repository Size 2024-03-29T07:52:14.828 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-03-29T07:52:14.828 INFO:teuthology.orchestra.run.smithi148.stdout:Installing: 2024-03-29T07:52:14.828 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-test x86_64 2:17.2.7-789.g4213e17c.el8 ceph 53 M 2024-03-29T07:52:14.828 INFO:teuthology.orchestra.run.smithi148.stdout:Installing dependencies: 2024-03-29T07:52:14.828 INFO:teuthology.orchestra.run.smithi148.stdout: jq x86_64 1.6-8.el8 appstream 203 k 2024-03-29T07:52:14.828 INFO:teuthology.orchestra.run.smithi148.stdout: libcephsqlite x86_64 2:17.2.7-789.g4213e17c.el8 ceph 179 k 2024-03-29T07:52:14.828 INFO:teuthology.orchestra.run.smithi148.stdout: oniguruma x86_64 6.8.2-3.el8 appstream 188 k 2024-03-29T07:52:14.828 INFO:teuthology.orchestra.run.smithi148.stdout: socat x86_64 1.7.4.1-1.el8 appstream 323 k 2024-03-29T07:52:14.829 INFO:teuthology.orchestra.run.smithi148.stdout: xmlstarlet x86_64 1.6.1-20.el8 appstream 71 k 2024-03-29T07:52:14.829 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T07:52:14.829 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction Summary 2024-03-29T07:52:14.829 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-03-29T07:52:14.829 INFO:teuthology.orchestra.run.smithi148.stdout:Install 6 Packages 2024-03-29T07:52:14.829 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T07:52:14.830 INFO:teuthology.orchestra.run.smithi148.stdout:Total download size: 54 M 2024-03-29T07:52:14.830 INFO:teuthology.orchestra.run.smithi148.stdout:Installed size: 236 M 2024-03-29T07:52:14.830 INFO:teuthology.orchestra.run.smithi148.stdout:Downloading Packages: 2024-03-29T07:52:15.201 INFO:teuthology.orchestra.run.smithi148.stdout:(1/6): jq-1.6-8.el8.x86_64.rpm 1.2 MB/s | 203 kB 00:00 2024-03-29T07:52:15.234 INFO:teuthology.orchestra.run.smithi148.stdout:(2/6): libcephsqlite-17.2.7-789.g4213e17c.el8.x 876 kB/s | 179 kB 00:00 2024-03-29T07:52:15.334 INFO:teuthology.orchestra.run.smithi148.stdout:(3/6): oniguruma-6.8.2-3.el8.x86_64.rpm 1.4 MB/s | 188 kB 00:00 2024-03-29T07:52:15.434 INFO:teuthology.orchestra.run.smithi148.stdout:(4/6): socat-1.7.4.1-1.el8.x86_64.rpm 1.6 MB/s | 323 kB 00:00 2024-03-29T07:52:15.460 INFO:teuthology.orchestra.run.smithi148.stdout:(5/6): xmlstarlet-1.6.1-20.el8.x86_64.rpm 564 kB/s | 71 kB 00:00 2024-03-29T07:52:16.321 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-radosgw-2:17.2.7-789.g4213e17c.el8.x86_64 23/25 2024-03-29T07:52:16.436 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-radosgw-2:17.2.7-789.g4213e17c.el8.x86_64 23/25 2024-03-29T07:52:16.436 INFO:teuthology.orchestra.run.smithi037.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 24/25 2024-03-29T07:52:16.678 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 24/25 2024-03-29T07:52:16.679 INFO:teuthology.orchestra.run.smithi037.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 25/25 2024-03-29T07:52:17.602 INFO:teuthology.orchestra.run.smithi148.stdout:(6/6): ceph-test-17.2.7-789.g4213e17c.el8.x86_6 20 MB/s | 53 MB 00:02 2024-03-29T07:52:17.604 INFO:teuthology.orchestra.run.smithi148.stdout:-------------------------------------------------------------------------------- 2024-03-29T07:52:17.604 INFO:teuthology.orchestra.run.smithi148.stdout:Total 19 MB/s | 54 MB 00:02 2024-03-29T07:52:17.651 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 25/25 2024-03-29T07:52:17.651 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-base-2:17.2.7-789.g4213e17c.el8.x86_64 1/25 2024-03-29T07:52:17.651 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-common-2:17.2.7-789.g4213e17c.el8.x86_64 2/25 2024-03-29T07:52:17.651 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-radosgw-2:17.2.7-789.g4213e17c.el8.x86_64 3/25 2024-03-29T07:52:17.652 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-selinux-2:17.2.7-789.g4213e17c.el8.x86_64 4/25 2024-03-29T07:52:17.652 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libcephfs2-2:17.2.7-789.g4213e17c.el8.x86_64 5/25 2024-03-29T07:52:17.652 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libradosstriper1-2:17.2.7-789.g4213e17c.el8.x86_64 6/25 2024-03-29T07:52:17.652 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : librgw2-2:17.2.7-789.g4213e17c.el8.x86_64 7/25 2024-03-29T07:52:17.652 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-ceph-argparse-2:17.2.7-789.g4213e17c.el8.x 8/25 2024-03-29T07:52:17.652 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-ceph-common-2:17.2.7-789.g4213e17c.el8.x86 9/25 2024-03-29T07:52:17.652 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-cephfs-2:17.2.7-789.g4213e17c.el8.x86_64 10/25 2024-03-29T07:52:17.652 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-rados-2:17.2.7-789.g4213e17c.el8.x86_64 11/25 2024-03-29T07:52:17.652 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-rbd-2:17.2.7-789.g4213e17c.el8.x86_64 12/25 2024-03-29T07:52:17.652 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-rgw-2:17.2.7-789.g4213e17c.el8.x86_64 13/25 2024-03-29T07:52:17.652 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : librdkafka-1.6.1-1.el8.x86_64 14/25 2024-03-29T07:52:17.652 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 15/25 2024-03-29T07:52:17.652 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 16/25 2024-03-29T07:52:17.652 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : librabbitmq-0.9.0-5.el8.x86_64 17/25 2024-03-29T07:52:17.653 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 18/25 2024-03-29T07:52:17.653 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 19/25 2024-03-29T07:52:17.653 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 20/25 2024-03-29T07:52:17.653 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 21/25 2024-03-29T07:52:17.653 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : librados2-2:17.2.7-789.g4213e17c.el8.x86_64 22/25 2024-03-29T07:52:17.653 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 23/25 2024-03-29T07:52:17.653 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : librbd1-2:17.2.7-789.g4213e17c.el8.x86_64 24/25 2024-03-29T07:52:17.695 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction check 2024-03-29T07:52:17.719 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction check succeeded. 2024-03-29T07:52:17.719 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction test 2024-03-29T07:52:18.040 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction test succeeded. 2024-03-29T07:52:18.044 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction 2024-03-29T07:52:18.583 INFO:teuthology.orchestra.run.smithi148.stdout: Preparing : 1/1 2024-03-29T07:52:18.596 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 25/25 2024-03-29T07:52:18.596 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T07:52:18.596 INFO:teuthology.orchestra.run.smithi037.stdout:Upgraded: 2024-03-29T07:52:18.596 INFO:teuthology.orchestra.run.smithi037.stdout: librados2-2:17.2.7-789.g4213e17c.el8.x86_64 2024-03-29T07:52:18.596 INFO:teuthology.orchestra.run.smithi037.stdout: librbd1-2:17.2.7-789.g4213e17c.el8.x86_64 2024-03-29T07:52:18.596 INFO:teuthology.orchestra.run.smithi037.stdout:Installed: 2024-03-29T07:52:18.596 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-base-2:17.2.7-789.g4213e17c.el8.x86_64 2024-03-29T07:52:18.596 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-common-2:17.2.7-789.g4213e17c.el8.x86_64 2024-03-29T07:52:18.597 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-radosgw-2:17.2.7-789.g4213e17c.el8.x86_64 2024-03-29T07:52:18.597 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-selinux-2:17.2.7-789.g4213e17c.el8.x86_64 2024-03-29T07:52:18.597 INFO:teuthology.orchestra.run.smithi037.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2024-03-29T07:52:18.597 INFO:teuthology.orchestra.run.smithi037.stdout: libcephfs2-2:17.2.7-789.g4213e17c.el8.x86_64 2024-03-29T07:52:18.597 INFO:teuthology.orchestra.run.smithi037.stdout: liboath-2.6.2-3.el8.x86_64 2024-03-29T07:52:18.597 INFO:teuthology.orchestra.run.smithi037.stdout: librabbitmq-0.9.0-5.el8.x86_64 2024-03-29T07:52:18.597 INFO:teuthology.orchestra.run.smithi037.stdout: libradosstriper1-2:17.2.7-789.g4213e17c.el8.x86_64 2024-03-29T07:52:18.597 INFO:teuthology.orchestra.run.smithi037.stdout: librdkafka-1.6.1-1.el8.x86_64 2024-03-29T07:52:18.597 INFO:teuthology.orchestra.run.smithi037.stdout: librgw2-2:17.2.7-789.g4213e17c.el8.x86_64 2024-03-29T07:52:18.597 INFO:teuthology.orchestra.run.smithi037.stdout: libunwind-1.3.1-3.el8.x86_64 2024-03-29T07:52:18.597 INFO:teuthology.orchestra.run.smithi037.stdout: lttng-ust-2.8.1-11.el8.x86_64 2024-03-29T07:52:18.597 INFO:teuthology.orchestra.run.smithi037.stdout: python3-ceph-argparse-2:17.2.7-789.g4213e17c.el8.x86_64 2024-03-29T07:52:18.597 INFO:teuthology.orchestra.run.smithi037.stdout: python3-ceph-common-2:17.2.7-789.g4213e17c.el8.x86_64 2024-03-29T07:52:18.597 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cephfs-2:17.2.7-789.g4213e17c.el8.x86_64 2024-03-29T07:52:18.597 INFO:teuthology.orchestra.run.smithi037.stdout: python3-prettytable-0.7.2-14.el8.noarch 2024-03-29T07:52:18.597 INFO:teuthology.orchestra.run.smithi037.stdout: python3-rados-2:17.2.7-789.g4213e17c.el8.x86_64 2024-03-29T07:52:18.598 INFO:teuthology.orchestra.run.smithi037.stdout: python3-rbd-2:17.2.7-789.g4213e17c.el8.x86_64 2024-03-29T07:52:18.598 INFO:teuthology.orchestra.run.smithi037.stdout: python3-rgw-2:17.2.7-789.g4213e17c.el8.x86_64 2024-03-29T07:52:18.598 INFO:teuthology.orchestra.run.smithi037.stdout: thrift-0.13.0-2.el8.x86_64 2024-03-29T07:52:18.598 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T07:52:18.598 INFO:teuthology.orchestra.run.smithi037.stdout:Complete! 2024-03-29T07:52:18.801 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 1/6 2024-03-29T07:52:18.957 DEBUG:teuthology.orchestra.run.smithi037:> sudo yum -y install ceph-test 2024-03-29T07:52:19.001 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 2/6 2024-03-29T07:52:19.179 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : oniguruma-6.8.2-3.el8.x86_64 3/6 2024-03-29T07:52:19.275 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: oniguruma-6.8.2-3.el8.x86_64 3/6 2024-03-29T07:52:19.496 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : jq-1.6-8.el8.x86_64 4/6 2024-03-29T07:52:19.662 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libcephsqlite-2:17.2.7-789.g4213e17c.el8.x86_64 5/6 2024-03-29T07:52:19.930 INFO:teuthology.orchestra.run.smithi037.stdout:Last metadata expiration check: 0:00:53 ago on Fri 29 Mar 2024 07:51:26 AM UTC. 2024-03-29T07:52:21.746 INFO:teuthology.orchestra.run.smithi037.stdout:Dependencies resolved. 2024-03-29T07:52:21.747 INFO:teuthology.orchestra.run.smithi037.stdout:================================================================================ 2024-03-29T07:52:21.748 INFO:teuthology.orchestra.run.smithi037.stdout: Package Arch Version Repository Size 2024-03-29T07:52:21.748 INFO:teuthology.orchestra.run.smithi037.stdout:================================================================================ 2024-03-29T07:52:21.748 INFO:teuthology.orchestra.run.smithi037.stdout:Installing: 2024-03-29T07:52:21.748 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-test x86_64 2:17.2.7-789.g4213e17c.el8 ceph 53 M 2024-03-29T07:52:21.748 INFO:teuthology.orchestra.run.smithi037.stdout:Installing dependencies: 2024-03-29T07:52:21.748 INFO:teuthology.orchestra.run.smithi037.stdout: jq x86_64 1.6-8.el8 appstream 203 k 2024-03-29T07:52:21.748 INFO:teuthology.orchestra.run.smithi037.stdout: libcephsqlite x86_64 2:17.2.7-789.g4213e17c.el8 ceph 179 k 2024-03-29T07:52:21.748 INFO:teuthology.orchestra.run.smithi037.stdout: oniguruma x86_64 6.8.2-3.el8 appstream 188 k 2024-03-29T07:52:21.748 INFO:teuthology.orchestra.run.smithi037.stdout: socat x86_64 1.7.4.1-1.el8 appstream 323 k 2024-03-29T07:52:21.748 INFO:teuthology.orchestra.run.smithi037.stdout: xmlstarlet x86_64 1.6.1-20.el8 appstream 71 k 2024-03-29T07:52:21.748 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T07:52:21.748 INFO:teuthology.orchestra.run.smithi037.stdout:Transaction Summary 2024-03-29T07:52:21.748 INFO:teuthology.orchestra.run.smithi037.stdout:================================================================================ 2024-03-29T07:52:21.749 INFO:teuthology.orchestra.run.smithi037.stdout:Install 6 Packages 2024-03-29T07:52:21.749 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T07:52:21.749 INFO:teuthology.orchestra.run.smithi037.stdout:Total download size: 54 M 2024-03-29T07:52:21.749 INFO:teuthology.orchestra.run.smithi037.stdout:Installed size: 236 M 2024-03-29T07:52:21.749 INFO:teuthology.orchestra.run.smithi037.stdout:Downloading Packages: 2024-03-29T07:52:21.977 INFO:teuthology.orchestra.run.smithi037.stdout:(1/6): jq-1.6-8.el8.x86_64.rpm 1.2 MB/s | 203 kB 00:00 2024-03-29T07:52:22.102 INFO:teuthology.orchestra.run.smithi037.stdout:(2/6): oniguruma-6.8.2-3.el8.x86_64.rpm 1.5 MB/s | 188 kB 00:00 2024-03-29T07:52:22.127 INFO:teuthology.orchestra.run.smithi037.stdout:(3/6): libcephsqlite-17.2.7-789.g4213e17c.el8.x 563 kB/s | 179 kB 00:00 2024-03-29T07:52:22.244 INFO:teuthology.orchestra.run.smithi037.stdout:(4/6): socat-1.7.4.1-1.el8.x86_64.rpm 2.2 MB/s | 323 kB 00:00 2024-03-29T07:52:22.269 INFO:teuthology.orchestra.run.smithi037.stdout:(5/6): xmlstarlet-1.6.1-20.el8.x86_64.rpm 498 kB/s | 71 kB 00:00 2024-03-29T07:52:24.520 INFO:teuthology.orchestra.run.smithi037.stdout:(6/6): ceph-test-17.2.7-789.g4213e17c.el8.x86_6 19 MB/s | 53 MB 00:02 2024-03-29T07:52:24.522 INFO:teuthology.orchestra.run.smithi037.stdout:-------------------------------------------------------------------------------- 2024-03-29T07:52:24.522 INFO:teuthology.orchestra.run.smithi037.stdout:Total 19 MB/s | 54 MB 00:02 2024-03-29T07:52:24.614 INFO:teuthology.orchestra.run.smithi037.stdout:Running transaction check 2024-03-29T07:52:24.640 INFO:teuthology.orchestra.run.smithi037.stdout:Transaction check succeeded. 2024-03-29T07:52:24.641 INFO:teuthology.orchestra.run.smithi037.stdout:Running transaction test 2024-03-29T07:52:24.979 INFO:teuthology.orchestra.run.smithi037.stdout:Transaction test succeeded. 2024-03-29T07:52:24.983 INFO:teuthology.orchestra.run.smithi037.stdout:Running transaction 2024-03-29T07:52:25.064 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: libcephsqlite-2:17.2.7-789.g4213e17c.el8.x86_64 5/6 2024-03-29T07:52:25.302 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-test-2:17.2.7-789.g4213e17c.el8.x86_64 6/6 2024-03-29T07:52:25.505 INFO:teuthology.orchestra.run.smithi037.stdout: Preparing : 1/1 2024-03-29T07:52:25.699 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-test-2:17.2.7-789.g4213e17c.el8.x86_64 6/6 2024-03-29T07:52:25.699 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-test-2:17.2.7-789.g4213e17c.el8.x86_64 1/6 2024-03-29T07:52:25.699 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libcephsqlite-2:17.2.7-789.g4213e17c.el8.x86_64 2/6 2024-03-29T07:52:25.699 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : jq-1.6-8.el8.x86_64 3/6 2024-03-29T07:52:25.699 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : oniguruma-6.8.2-3.el8.x86_64 4/6 2024-03-29T07:52:25.700 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 5/6 2024-03-29T07:52:25.717 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 1/6 2024-03-29T07:52:25.886 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 2/6 2024-03-29T07:52:26.022 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : oniguruma-6.8.2-3.el8.x86_64 3/6 2024-03-29T07:52:26.128 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: oniguruma-6.8.2-3.el8.x86_64 3/6 2024-03-29T07:52:26.173 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 6/6 2024-03-29T07:52:26.173 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T07:52:26.173 INFO:teuthology.orchestra.run.smithi148.stdout:Installed: 2024-03-29T07:52:26.173 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-test-2:17.2.7-789.g4213e17c.el8.x86_64 2024-03-29T07:52:26.174 INFO:teuthology.orchestra.run.smithi148.stdout: jq-1.6-8.el8.x86_64 2024-03-29T07:52:26.174 INFO:teuthology.orchestra.run.smithi148.stdout: libcephsqlite-2:17.2.7-789.g4213e17c.el8.x86_64 2024-03-29T07:52:26.174 INFO:teuthology.orchestra.run.smithi148.stdout: oniguruma-6.8.2-3.el8.x86_64 2024-03-29T07:52:26.174 INFO:teuthology.orchestra.run.smithi148.stdout: socat-1.7.4.1-1.el8.x86_64 2024-03-29T07:52:26.174 INFO:teuthology.orchestra.run.smithi148.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2024-03-29T07:52:26.174 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T07:52:26.174 INFO:teuthology.orchestra.run.smithi148.stdout:Complete! 2024-03-29T07:52:26.340 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : jq-1.6-8.el8.x86_64 4/6 2024-03-29T07:52:26.370 DEBUG:teuthology.orchestra.run.smithi148:> sudo yum -y install ceph 2024-03-29T07:52:26.513 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libcephsqlite-2:17.2.7-789.g4213e17c.el8.x86_64 5/6 2024-03-29T07:52:27.324 INFO:teuthology.orchestra.run.smithi148.stdout:Last metadata expiration check: 0:01:01 ago on Fri 29 Mar 2024 07:51:26 AM UTC. 2024-03-29T07:52:29.037 INFO:teuthology.orchestra.run.smithi148.stdout:Dependencies resolved. 2024-03-29T07:52:29.041 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-03-29T07:52:29.041 INFO:teuthology.orchestra.run.smithi148.stdout: Package Arch Version Repository Size 2024-03-29T07:52:29.041 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-03-29T07:52:29.041 INFO:teuthology.orchestra.run.smithi148.stdout:Installing: 2024-03-29T07:52:29.041 INFO:teuthology.orchestra.run.smithi148.stdout: ceph x86_64 2:17.2.7-789.g4213e17c.el8 ceph 6.4 k 2024-03-29T07:52:29.041 INFO:teuthology.orchestra.run.smithi148.stdout:Installing dependencies: 2024-03-29T07:52:29.041 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mds x86_64 2:17.2.7-789.g4213e17c.el8 ceph 2.3 M 2024-03-29T07:52:29.041 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr x86_64 2:17.2.7-789.g4213e17c.el8 ceph 1.6 M 2024-03-29T07:52:29.041 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-modules-core noarch 2:17.2.7-789.g4213e17c.el8 ceph-noarch 240 k 2024-03-29T07:52:29.041 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mon x86_64 2:17.2.7-789.g4213e17c.el8 ceph 4.0 M 2024-03-29T07:52:29.041 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-osd x86_64 2:17.2.7-789.g4213e17c.el8 ceph 17 M 2024-03-29T07:52:29.042 INFO:teuthology.orchestra.run.smithi148.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2024-03-29T07:52:29.042 INFO:teuthology.orchestra.run.smithi148.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2024-03-29T07:52:29.042 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cffi x86_64 1.11.5-6.el8 baseos 249 k 2024-03-29T07:52:29.042 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2024-03-29T07:52:29.042 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2024-03-29T07:52:29.042 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cryptography x86_64 3.2.1-7.el8 baseos 641 k 2024-03-29T07:52:29.042 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2024-03-29T07:52:29.042 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2024-03-29T07:52:29.042 INFO:teuthology.orchestra.run.smithi148.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2024-03-29T07:52:29.042 INFO:teuthology.orchestra.run.smithi148.stdout: python3-mako noarch 1.0.6-14.el8 appstream 157 k 2024-03-29T07:52:29.042 INFO:teuthology.orchestra.run.smithi148.stdout: python3-markupsafe x86_64 0.23-19.el8 appstream 39 k 2024-03-29T07:52:29.042 INFO:teuthology.orchestra.run.smithi148.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2024-03-29T07:52:29.042 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2024-03-29T07:52:29.043 INFO:teuthology.orchestra.run.smithi148.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2024-03-29T07:52:29.043 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 appstream 103 k 2024-03-29T07:52:29.043 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pycparser noarch 2.14-14.el8 baseos 109 k 2024-03-29T07:52:29.043 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pytz noarch 2017.2-11.el8 appstream 57 k 2024-03-29T07:52:29.043 INFO:teuthology.orchestra.run.smithi148.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2024-03-29T07:52:29.043 INFO:teuthology.orchestra.run.smithi148.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2024-03-29T07:52:29.043 INFO:teuthology.orchestra.run.smithi148.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2024-03-29T07:52:29.043 INFO:teuthology.orchestra.run.smithi148.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2024-03-29T07:52:29.043 INFO:teuthology.orchestra.run.smithi148.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2024-03-29T07:52:29.043 INFO:teuthology.orchestra.run.smithi148.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2024-03-29T07:52:29.043 INFO:teuthology.orchestra.run.smithi148.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2024-03-29T07:52:29.043 INFO:teuthology.orchestra.run.smithi148.stdout: python3-werkzeug noarch 0.12.2-4.el8 appstream 457 k 2024-03-29T07:52:29.043 INFO:teuthology.orchestra.run.smithi148.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2024-03-29T07:52:29.043 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T07:52:29.043 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction Summary 2024-03-29T07:52:29.044 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-03-29T07:52:29.044 INFO:teuthology.orchestra.run.smithi148.stdout:Install 32 Packages 2024-03-29T07:52:29.044 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T07:52:29.044 INFO:teuthology.orchestra.run.smithi148.stdout:Total download size: 29 M 2024-03-29T07:52:29.044 INFO:teuthology.orchestra.run.smithi148.stdout:Installed size: 113 M 2024-03-29T07:52:29.044 INFO:teuthology.orchestra.run.smithi148.stdout:Downloading Packages: 2024-03-29T07:52:29.545 INFO:teuthology.orchestra.run.smithi148.stdout:(1/32): ceph-17.2.7-789.g4213e17c.el8.x86_64.rp 34 kB/s | 6.4 kB 00:00 2024-03-29T07:52:29.653 INFO:teuthology.orchestra.run.smithi148.stdout:(2/32): ceph-mgr-17.2.7-789.g4213e17c.el8.x86_6 5.3 MB/s | 1.6 MB 00:00 2024-03-29T07:52:30.220 INFO:teuthology.orchestra.run.smithi148.stdout:(3/32): ceph-mds-17.2.7-789.g4213e17c.el8.x86_6 2.7 MB/s | 2.3 MB 00:00 2024-03-29T07:52:30.320 INFO:teuthology.orchestra.run.smithi148.stdout:(4/32): ceph-mgr-modules-core-17.2.7-789.g4213e 2.3 MB/s | 240 kB 00:00 2024-03-29T07:52:30.604 INFO:teuthology.orchestra.run.smithi148.stdout:(5/32): ceph-osd-17.2.7-789.g4213e17c.el8.x86_6 18 MB/s | 17 MB 00:00 2024-03-29T07:52:30.679 INFO:teuthology.orchestra.run.smithi148.stdout:(6/32): python3-mako-1.0.6-14.el8.noarch.rpm 438 kB/s | 157 kB 00:00 2024-03-29T07:52:30.721 INFO:teuthology.orchestra.run.smithi148.stdout:(7/32): python3-markupsafe-0.23-19.el8.x86_64.r 333 kB/s | 39 kB 00:00 2024-03-29T07:52:30.813 INFO:teuthology.orchestra.run.smithi148.stdout:(8/32): ceph-mon-17.2.7-789.g4213e17c.el8.x86_6 3.2 MB/s | 4.0 MB 00:01 2024-03-29T07:52:30.838 INFO:teuthology.orchestra.run.smithi148.stdout:(9/32): python3-pyOpenSSL-19.0.0-1.el8.noarch.r 653 kB/s | 103 kB 00:00 2024-03-29T07:52:30.864 INFO:teuthology.orchestra.run.smithi148.stdout:(10/32): python3-pytz-2017.2-11.el8.noarch.rpm 400 kB/s | 57 kB 00:00 2024-03-29T07:52:30.956 INFO:teuthology.orchestra.run.smithi148.stdout:(11/32): python3-werkzeug-0.12.2-4.el8.noarch.r 3.1 MB/s | 457 kB 00:00 2024-03-29T07:52:31.223 INFO:teuthology.orchestra.run.smithi148.stdout:(12/32): python3-cffi-1.11.5-6.el8.x86_64.rpm 647 kB/s | 249 kB 00:00 2024-03-29T07:52:31.273 INFO:teuthology.orchestra.run.smithi148.stdout:(13/32): python3-pycparser-2.14-14.el8.noarch.r 345 kB/s | 109 kB 00:00 2024-03-29T07:52:31.315 INFO:teuthology.orchestra.run.smithi148.stdout:(14/32): python3-cryptography-3.2.1-7.el8.x86_6 1.4 MB/s | 641 kB 00:00 2024-03-29T07:52:31.349 INFO:teuthology.orchestra.run.smithi148.stdout:(15/32): python3-bcrypt-3.1.6-2.el8.1.x86_64.rp 348 kB/s | 44 kB 00:00 2024-03-29T07:52:31.424 INFO:teuthology.orchestra.run.smithi148.stdout:(16/32): python3-beautifulsoup4-4.6.3-2.el8.1.n 1.2 MB/s | 185 kB 00:00 2024-03-29T07:52:31.458 INFO:teuthology.orchestra.run.smithi148.stdout:(17/32): python3-cherrypy-18.4.0-1.el8.noarch.r 3.5 MB/s | 384 kB 00:00 2024-03-29T07:52:31.483 INFO:teuthology.orchestra.run.smithi148.stdout:(18/32): python3-jaraco-6.2-6.el8.noarch.rpm 189 kB/s | 11 kB 00:00 2024-03-29T07:52:31.517 INFO:teuthology.orchestra.run.smithi148.stdout:(19/32): python3-cheroot-8.5.2-1.el8.noarch.rpm 859 kB/s | 173 kB 00:00 2024-03-29T07:52:31.542 INFO:teuthology.orchestra.run.smithi148.stdout:(20/32): python3-jaraco-functools-2.0-4.el8.noa 210 kB/s | 18 kB 00:00 2024-03-29T07:52:31.567 INFO:teuthology.orchestra.run.smithi148.stdout:(21/32): python3-logutils-0.3.5-11.el8.noarch.r 583 kB/s | 49 kB 00:00 2024-03-29T07:52:31.626 INFO:teuthology.orchestra.run.smithi148.stdout:(22/32): python3-more-itertools-7.2.0-3.el8.noa 704 kB/s | 59 kB 00:00 2024-03-29T07:52:31.668 INFO:teuthology.orchestra.run.smithi148.stdout:(23/32): python3-portend-2.6-1.el8.noarch.rpm 155 kB/s | 16 kB 00:00 2024-03-29T07:52:31.693 INFO:teuthology.orchestra.run.smithi148.stdout:(24/32): python3-simplegeneric-0.8.1-17.el8.noa 280 kB/s | 19 kB 00:00 2024-03-29T07:52:31.718 INFO:teuthology.orchestra.run.smithi148.stdout:(25/32): python3-singledispatch-3.4.0.3-18.el8. 477 kB/s | 24 kB 00:00 2024-03-29T07:52:31.744 INFO:teuthology.orchestra.run.smithi148.stdout:(26/32): python3-tempora-1.14.1-5.el8.noarch.rp 586 kB/s | 29 kB 00:00 2024-03-29T07:52:31.769 INFO:teuthology.orchestra.run.smithi148.stdout:(27/32): python3-trustme-0.6.0-4.el8.noarch.rpm 550 kB/s | 27 kB 00:00 2024-03-29T07:52:31.794 INFO:teuthology.orchestra.run.smithi148.stdout:(28/32): python3-pecan-1.3.2-9.el8.noarch.rpm 1.1 MB/s | 283 kB 00:00 2024-03-29T07:52:31.819 INFO:teuthology.orchestra.run.smithi148.stdout:(29/32): python3-waitress-1.4.3-1.el8.noarch.rp 3.3 MB/s | 255 kB 00:00 2024-03-29T07:52:31.845 INFO:teuthology.orchestra.run.smithi148.stdout:(30/32): python3-webob-1.8.5-1.el8.1.noarch.rpm 3.3 MB/s | 251 kB 00:00 2024-03-29T07:52:31.870 INFO:teuthology.orchestra.run.smithi148.stdout:(31/32): python3-webtest-2.0.33-1.el8.noarch.rp 1.1 MB/s | 85 kB 00:00 2024-03-29T07:52:31.887 INFO:teuthology.orchestra.run.smithi148.stdout:(32/32): python3-zc-lockfile-2.0-2.el8.noarch.r 342 kB/s | 23 kB 00:00 2024-03-29T07:52:31.888 INFO:teuthology.orchestra.run.smithi148.stdout:-------------------------------------------------------------------------------- 2024-03-29T07:52:31.889 INFO:teuthology.orchestra.run.smithi148.stdout:Total 10 MB/s | 29 MB 00:02 2024-03-29T07:52:32.062 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction check 2024-03-29T07:52:32.103 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction check succeeded. 2024-03-29T07:52:32.104 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction test 2024-03-29T07:52:32.262 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: libcephsqlite-2:17.2.7-789.g4213e17c.el8.x86_64 5/6 2024-03-29T07:52:32.437 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction test succeeded. 2024-03-29T07:52:32.441 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction 2024-03-29T07:52:32.529 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-test-2:17.2.7-789.g4213e17c.el8.x86_64 6/6 2024-03-29T07:52:32.934 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-test-2:17.2.7-789.g4213e17c.el8.x86_64 6/6 2024-03-29T07:52:32.934 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-test-2:17.2.7-789.g4213e17c.el8.x86_64 1/6 2024-03-29T07:52:32.935 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libcephsqlite-2:17.2.7-789.g4213e17c.el8.x86_64 2/6 2024-03-29T07:52:32.935 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : jq-1.6-8.el8.x86_64 3/6 2024-03-29T07:52:32.935 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : oniguruma-6.8.2-3.el8.x86_64 4/6 2024-03-29T07:52:32.935 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 5/6 2024-03-29T07:52:33.439 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 6/6 2024-03-29T07:52:33.439 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T07:52:33.440 INFO:teuthology.orchestra.run.smithi037.stdout:Installed: 2024-03-29T07:52:33.440 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-test-2:17.2.7-789.g4213e17c.el8.x86_64 2024-03-29T07:52:33.440 INFO:teuthology.orchestra.run.smithi037.stdout: jq-1.6-8.el8.x86_64 2024-03-29T07:52:33.440 INFO:teuthology.orchestra.run.smithi037.stdout: libcephsqlite-2:17.2.7-789.g4213e17c.el8.x86_64 2024-03-29T07:52:33.440 INFO:teuthology.orchestra.run.smithi037.stdout: oniguruma-6.8.2-3.el8.x86_64 2024-03-29T07:52:33.440 INFO:teuthology.orchestra.run.smithi037.stdout: socat-1.7.4.1-1.el8.x86_64 2024-03-29T07:52:33.440 INFO:teuthology.orchestra.run.smithi037.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2024-03-29T07:52:33.440 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T07:52:33.440 INFO:teuthology.orchestra.run.smithi037.stdout:Complete! 2024-03-29T07:52:33.625 DEBUG:teuthology.orchestra.run.smithi037:> sudo yum -y install ceph 2024-03-29T07:52:33.647 INFO:teuthology.orchestra.run.smithi148.stdout: Preparing : 1/1 2024-03-29T07:52:33.911 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 1/32 2024-03-29T07:52:34.215 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 2/32 2024-03-29T07:52:34.500 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 3/32 2024-03-29T07:52:34.606 INFO:teuthology.orchestra.run.smithi037.stdout:Last metadata expiration check: 0:01:08 ago on Fri 29 Mar 2024 07:51:26 AM UTC. 2024-03-29T07:52:34.776 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 4/32 2024-03-29T07:52:34.957 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 5/32 2024-03-29T07:52:35.208 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 6/32 2024-03-29T07:52:35.414 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 7/32 2024-03-29T07:52:35.633 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 8/32 2024-03-29T07:52:35.900 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 9/32 2024-03-29T07:52:36.177 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 10/32 2024-03-29T07:52:36.432 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 11/32 2024-03-29T07:52:36.625 INFO:teuthology.orchestra.run.smithi037.stdout:Dependencies resolved. 2024-03-29T07:52:36.629 INFO:teuthology.orchestra.run.smithi037.stdout:================================================================================ 2024-03-29T07:52:36.629 INFO:teuthology.orchestra.run.smithi037.stdout: Package Arch Version Repository Size 2024-03-29T07:52:36.629 INFO:teuthology.orchestra.run.smithi037.stdout:================================================================================ 2024-03-29T07:52:36.629 INFO:teuthology.orchestra.run.smithi037.stdout:Installing: 2024-03-29T07:52:36.629 INFO:teuthology.orchestra.run.smithi037.stdout: ceph x86_64 2:17.2.7-789.g4213e17c.el8 ceph 6.4 k 2024-03-29T07:52:36.629 INFO:teuthology.orchestra.run.smithi037.stdout:Installing dependencies: 2024-03-29T07:52:36.630 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mds x86_64 2:17.2.7-789.g4213e17c.el8 ceph 2.3 M 2024-03-29T07:52:36.630 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mgr x86_64 2:17.2.7-789.g4213e17c.el8 ceph 1.6 M 2024-03-29T07:52:36.630 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mgr-modules-core noarch 2:17.2.7-789.g4213e17c.el8 ceph-noarch 240 k 2024-03-29T07:52:36.630 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mon x86_64 2:17.2.7-789.g4213e17c.el8 ceph 4.0 M 2024-03-29T07:52:36.630 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-osd x86_64 2:17.2.7-789.g4213e17c.el8 ceph 17 M 2024-03-29T07:52:36.630 INFO:teuthology.orchestra.run.smithi037.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2024-03-29T07:52:36.630 INFO:teuthology.orchestra.run.smithi037.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2024-03-29T07:52:36.630 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cffi x86_64 1.11.5-6.el8 baseos 249 k 2024-03-29T07:52:36.630 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2024-03-29T07:52:36.630 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2024-03-29T07:52:36.630 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cryptography x86_64 3.2.1-7.el8 baseos 641 k 2024-03-29T07:52:36.630 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2024-03-29T07:52:36.630 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2024-03-29T07:52:36.631 INFO:teuthology.orchestra.run.smithi037.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2024-03-29T07:52:36.631 INFO:teuthology.orchestra.run.smithi037.stdout: python3-mako noarch 1.0.6-14.el8 appstream 157 k 2024-03-29T07:52:36.631 INFO:teuthology.orchestra.run.smithi037.stdout: python3-markupsafe x86_64 0.23-19.el8 appstream 39 k 2024-03-29T07:52:36.631 INFO:teuthology.orchestra.run.smithi037.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2024-03-29T07:52:36.631 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2024-03-29T07:52:36.631 INFO:teuthology.orchestra.run.smithi037.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2024-03-29T07:52:36.631 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 appstream 103 k 2024-03-29T07:52:36.631 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pycparser noarch 2.14-14.el8 baseos 109 k 2024-03-29T07:52:36.631 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pytz noarch 2017.2-11.el8 appstream 57 k 2024-03-29T07:52:36.631 INFO:teuthology.orchestra.run.smithi037.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2024-03-29T07:52:36.631 INFO:teuthology.orchestra.run.smithi037.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2024-03-29T07:52:36.631 INFO:teuthology.orchestra.run.smithi037.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2024-03-29T07:52:36.631 INFO:teuthology.orchestra.run.smithi037.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2024-03-29T07:52:36.631 INFO:teuthology.orchestra.run.smithi037.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2024-03-29T07:52:36.633 INFO:teuthology.orchestra.run.smithi037.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2024-03-29T07:52:36.633 INFO:teuthology.orchestra.run.smithi037.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2024-03-29T07:52:36.634 INFO:teuthology.orchestra.run.smithi037.stdout: python3-werkzeug noarch 0.12.2-4.el8 appstream 457 k 2024-03-29T07:52:36.634 INFO:teuthology.orchestra.run.smithi037.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2024-03-29T07:52:36.634 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T07:52:36.634 INFO:teuthology.orchestra.run.smithi037.stdout:Transaction Summary 2024-03-29T07:52:36.634 INFO:teuthology.orchestra.run.smithi037.stdout:================================================================================ 2024-03-29T07:52:36.634 INFO:teuthology.orchestra.run.smithi037.stdout:Install 32 Packages 2024-03-29T07:52:36.634 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T07:52:36.635 INFO:teuthology.orchestra.run.smithi037.stdout:Total download size: 29 M 2024-03-29T07:52:36.635 INFO:teuthology.orchestra.run.smithi037.stdout:Installed size: 113 M 2024-03-29T07:52:36.635 INFO:teuthology.orchestra.run.smithi037.stdout:Downloading Packages: 2024-03-29T07:52:36.700 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 12/32 2024-03-29T07:52:37.175 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-cffi-1.11.5-6.el8.x86_64 13/32 2024-03-29T07:52:37.204 INFO:teuthology.orchestra.run.smithi037.stdout:(1/32): ceph-17.2.7-789.g4213e17c.el8.x86_64.rp 34 kB/s | 6.4 kB 00:00 2024-03-29T07:52:37.441 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-cryptography-3.2.1-7.el8.x86_64 14/32 2024-03-29T07:52:37.554 INFO:teuthology.orchestra.run.smithi037.stdout:(2/32): ceph-mgr-17.2.7-789.g4213e17c.el8.x86_6 2.9 MB/s | 1.6 MB 00:00 2024-03-29T07:52:37.613 INFO:teuthology.orchestra.run.smithi037.stdout:(3/32): ceph-mds-17.2.7-789.g4213e17c.el8.x86_6 3.9 MB/s | 2.3 MB 00:00 2024-03-29T07:52:37.636 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 15/32 2024-03-29T07:52:37.697 INFO:teuthology.orchestra.run.smithi037.stdout:(4/32): ceph-mgr-modules-core-17.2.7-789.g4213e 2.8 MB/s | 240 kB 00:00 2024-03-29T07:52:37.872 INFO:teuthology.orchestra.run.smithi037.stdout:(5/32): ceph-mon-17.2.7-789.g4213e17c.el8.x86_6 6.0 MB/s | 4.0 MB 00:00 2024-03-29T07:52:37.888 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 16/32 2024-03-29T07:52:37.897 INFO:teuthology.orchestra.run.smithi037.stdout:(6/32): python3-mako-1.0.6-14.el8.noarch.rpm 784 kB/s | 157 kB 00:00 2024-03-29T07:52:37.981 INFO:teuthology.orchestra.run.smithi037.stdout:(7/32): python3-markupsafe-0.23-19.el8.x86_64.r 358 kB/s | 39 kB 00:00 2024-03-29T07:52:38.026 INFO:teuthology.orchestra.run.smithi037.stdout:(8/32): python3-pyOpenSSL-19.0.0-1.el8.noarch.r 808 kB/s | 103 kB 00:00 2024-03-29T07:52:38.073 INFO:teuthology.orchestra.run.smithi037.stdout:(9/32): python3-pytz-2017.2-11.el8.noarch.rpm 621 kB/s | 57 kB 00:00 2024-03-29T07:52:38.129 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 17/32 2024-03-29T07:52:38.165 INFO:teuthology.orchestra.run.smithi037.stdout:(10/32): python3-werkzeug-0.12.2-4.el8.noarch.r 3.2 MB/s | 457 kB 00:00 2024-03-29T07:52:38.432 INFO:teuthology.orchestra.run.smithi037.stdout:(11/32): python3-cffi-1.11.5-6.el8.x86_64.rpm 694 kB/s | 249 kB 00:00 2024-03-29T07:52:38.487 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 18/32 2024-03-29T07:52:38.524 INFO:teuthology.orchestra.run.smithi037.stdout:(12/32): python3-pycparser-2.14-14.el8.noarch.r 1.2 MB/s | 109 kB 00:00 2024-03-29T07:52:38.599 INFO:teuthology.orchestra.run.smithi037.stdout:(13/32): python3-cryptography-3.2.1-7.el8.x86_6 1.4 MB/s | 641 kB 00:00 2024-03-29T07:52:38.616 INFO:teuthology.orchestra.run.smithi037.stdout:(14/32): python3-bcrypt-3.1.6-2.el8.1.x86_64.rp 474 kB/s | 44 kB 00:00 2024-03-29T07:52:38.683 INFO:teuthology.orchestra.run.smithi037.stdout:(15/32): python3-beautifulsoup4-4.6.3-2.el8.1.n 2.2 MB/s | 185 kB 00:00 2024-03-29T07:52:38.718 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 19/32 2024-03-29T07:52:38.884 INFO:teuthology.orchestra.run.smithi037.stdout:(16/32): ceph-osd-17.2.7-789.g4213e17c.el8.x86_ 13 MB/s | 17 MB 00:01 2024-03-29T07:52:38.909 INFO:teuthology.orchestra.run.smithi037.stdout:(17/32): python3-cheroot-8.5.2-1.el8.noarch.rpm 591 kB/s | 173 kB 00:00 2024-03-29T07:52:38.923 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-pytz-2017.2-11.el8.noarch 20/32 2024-03-29T07:52:38.934 INFO:teuthology.orchestra.run.smithi037.stdout:(18/32): python3-cherrypy-18.4.0-1.el8.noarch.r 1.5 MB/s | 384 kB 00:00 2024-03-29T07:52:38.959 INFO:teuthology.orchestra.run.smithi037.stdout:(19/32): python3-jaraco-6.2-6.el8.noarch.rpm 148 kB/s | 11 kB 00:00 2024-03-29T07:52:38.985 INFO:teuthology.orchestra.run.smithi037.stdout:(20/32): python3-jaraco-functools-2.0-4.el8.noa 233 kB/s | 18 kB 00:00 2024-03-29T07:52:39.010 INFO:teuthology.orchestra.run.smithi037.stdout:(21/32): python3-logutils-0.3.5-11.el8.noarch.r 646 kB/s | 49 kB 00:00 2024-03-29T07:52:39.035 INFO:teuthology.orchestra.run.smithi037.stdout:(22/32): python3-more-itertools-7.2.0-3.el8.noa 781 kB/s | 59 kB 00:00 2024-03-29T07:52:39.061 INFO:teuthology.orchestra.run.smithi037.stdout:(23/32): python3-portend-2.6-1.el8.noarch.rpm 310 kB/s | 16 kB 00:00 2024-03-29T07:52:39.086 INFO:teuthology.orchestra.run.smithi037.stdout:(24/32): python3-simplegeneric-0.8.1-17.el8.noa 373 kB/s | 19 kB 00:00 2024-03-29T07:52:39.128 INFO:teuthology.orchestra.run.smithi037.stdout:(25/32): python3-singledispatch-3.4.0.3-18.el8. 358 kB/s | 24 kB 00:00 2024-03-29T07:52:39.140 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 21/32 2024-03-29T07:52:39.153 INFO:teuthology.orchestra.run.smithi037.stdout:(26/32): python3-tempora-1.14.1-5.el8.noarch.rp 437 kB/s | 29 kB 00:00 2024-03-29T07:52:39.178 INFO:teuthology.orchestra.run.smithi037.stdout:(27/32): python3-trustme-0.6.0-4.el8.noarch.rpm 548 kB/s | 27 kB 00:00 2024-03-29T07:52:39.204 INFO:teuthology.orchestra.run.smithi037.stdout:(28/32): python3-pecan-1.3.2-9.el8.noarch.rpm 1.3 MB/s | 283 kB 00:00 2024-03-29T07:52:39.229 INFO:teuthology.orchestra.run.smithi037.stdout:(29/32): python3-waitress-1.4.3-1.el8.noarch.rp 3.3 MB/s | 255 kB 00:00 2024-03-29T07:52:39.254 INFO:teuthology.orchestra.run.smithi037.stdout:(30/32): python3-webtest-2.0.33-1.el8.noarch.rp 1.7 MB/s | 85 kB 00:00 2024-03-29T07:52:39.280 INFO:teuthology.orchestra.run.smithi037.stdout:(31/32): python3-zc-lockfile-2.0-2.el8.noarch.r 460 kB/s | 23 kB 00:00 2024-03-29T07:52:39.305 INFO:teuthology.orchestra.run.smithi037.stdout:(32/32): python3-webob-1.8.5-1.el8.1.noarch.rpm 2.0 MB/s | 251 kB 00:00 2024-03-29T07:52:39.306 INFO:teuthology.orchestra.run.smithi037.stdout:-------------------------------------------------------------------------------- 2024-03-29T07:52:39.307 INFO:teuthology.orchestra.run.smithi037.stdout:Total 11 MB/s | 29 MB 00:02 2024-03-29T07:52:39.407 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-portend-2.6-1.el8.noarch 22/32 2024-03-29T07:52:39.486 INFO:teuthology.orchestra.run.smithi037.stdout:Running transaction check 2024-03-29T07:52:39.532 INFO:teuthology.orchestra.run.smithi037.stdout:Transaction check succeeded. 2024-03-29T07:52:39.532 INFO:teuthology.orchestra.run.smithi037.stdout:Running transaction test 2024-03-29T07:52:39.666 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 23/32 2024-03-29T07:52:39.891 INFO:teuthology.orchestra.run.smithi037.stdout:Transaction test succeeded. 2024-03-29T07:52:39.894 INFO:teuthology.orchestra.run.smithi037.stdout:Running transaction 2024-03-29T07:52:39.941 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 24/32 2024-03-29T07:52:40.312 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-mako-1.0.6-14.el8.noarch 25/32 2024-03-29T07:52:40.677 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 26/32 2024-03-29T07:52:41.027 INFO:teuthology.orchestra.run.smithi037.stdout: Preparing : 1/1 2024-03-29T07:52:41.058 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-mgr-modules-core-2:17.2.7-789.g4213e17c.el8.n 27/32 2024-03-29T07:52:41.275 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 1/32 2024-03-29T07:52:41.300 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-mgr-2:17.2.7-789.g4213e17c.el8.x86_64 28/32 2024-03-29T07:52:41.576 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 2/32 2024-03-29T07:52:41.893 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 3/32 2024-03-29T07:52:42.153 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 4/32 2024-03-29T07:52:42.342 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 5/32 2024-03-29T07:52:42.586 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 6/32 2024-03-29T07:52:42.815 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 7/32 2024-03-29T07:52:42.980 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-mgr-2:17.2.7-789.g4213e17c.el8.x86_64 28/32 2024-03-29T07:52:42.995 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 8/32 2024-03-29T07:52:43.217 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-osd-2:17.2.7-789.g4213e17c.el8.x86_64 29/32 2024-03-29T07:52:43.238 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 9/32 2024-03-29T07:52:43.539 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 10/32 2024-03-29T07:52:43.697 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-osd-2:17.2.7-789.g4213e17c.el8.x86_64 29/32 2024-03-29T07:52:43.803 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 11/32 2024-03-29T07:52:43.926 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-mon-2:17.2.7-789.g4213e17c.el8.x86_64 30/32 2024-03-29T07:52:44.080 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 12/32 2024-03-29T07:52:44.182 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-mon-2:17.2.7-789.g4213e17c.el8.x86_64 30/32 2024-03-29T07:52:44.409 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-mds-2:17.2.7-789.g4213e17c.el8.x86_64 31/32 2024-03-29T07:52:44.451 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-mds-2:17.2.7-789.g4213e17c.el8.x86_64 31/32 2024-03-29T07:52:44.567 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-cffi-1.11.5-6.el8.x86_64 13/32 2024-03-29T07:52:44.609 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-2:17.2.7-789.g4213e17c.el8.x86_64 32/32 2024-03-29T07:52:44.694 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-2:17.2.7-789.g4213e17c.el8.x86_64 32/32 2024-03-29T07:52:44.695 INFO:teuthology.orchestra.run.smithi148.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-03-29T07:52:44.695 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T07:52:44.837 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-cryptography-3.2.1-7.el8.x86_64 14/32 2024-03-29T07:52:45.046 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 15/32 2024-03-29T07:52:45.235 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-2:17.2.7-789.g4213e17c.el8.x86_64 1/32 2024-03-29T07:52:45.236 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-mds-2:17.2.7-789.g4213e17c.el8.x86_64 2/32 2024-03-29T07:52:45.236 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-mgr-2:17.2.7-789.g4213e17c.el8.x86_64 3/32 2024-03-29T07:52:45.236 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-mon-2:17.2.7-789.g4213e17c.el8.x86_64 4/32 2024-03-29T07:52:45.236 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-osd-2:17.2.7-789.g4213e17c.el8.x86_64 5/32 2024-03-29T07:52:45.236 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-mgr-modules-core-2:17.2.7-789.g4213e17c.el8.n 6/32 2024-03-29T07:52:45.236 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-mako-1.0.6-14.el8.noarch 7/32 2024-03-29T07:52:45.236 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 8/32 2024-03-29T07:52:45.236 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 9/32 2024-03-29T07:52:45.236 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-pytz-2017.2-11.el8.noarch 10/32 2024-03-29T07:52:45.236 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 11/32 2024-03-29T07:52:45.236 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-cffi-1.11.5-6.el8.x86_64 12/32 2024-03-29T07:52:45.236 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-cryptography-3.2.1-7.el8.x86_64 13/32 2024-03-29T07:52:45.236 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 14/32 2024-03-29T07:52:45.237 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 15/32 2024-03-29T07:52:45.237 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 16/32 2024-03-29T07:52:45.237 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 17/32 2024-03-29T07:52:45.237 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 18/32 2024-03-29T07:52:45.237 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 19/32 2024-03-29T07:52:45.237 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 20/32 2024-03-29T07:52:45.237 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 21/32 2024-03-29T07:52:45.237 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 22/32 2024-03-29T07:52:45.237 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 23/32 2024-03-29T07:52:45.237 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-portend-2.6-1.el8.noarch 24/32 2024-03-29T07:52:45.237 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 25/32 2024-03-29T07:52:45.238 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 26/32 2024-03-29T07:52:45.238 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 27/32 2024-03-29T07:52:45.238 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 28/32 2024-03-29T07:52:45.238 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 29/32 2024-03-29T07:52:45.239 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 30/32 2024-03-29T07:52:45.239 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 31/32 2024-03-29T07:52:45.329 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 16/32 2024-03-29T07:52:45.582 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 17/32 2024-03-29T07:52:45.966 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 18/32 2024-03-29T07:52:46.174 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 32/32 2024-03-29T07:52:46.174 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T07:52:46.174 INFO:teuthology.orchestra.run.smithi148.stdout:Installed: 2024-03-29T07:52:46.174 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-2:17.2.7-789.g4213e17c.el8.x86_64 2024-03-29T07:52:46.174 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mds-2:17.2.7-789.g4213e17c.el8.x86_64 2024-03-29T07:52:46.174 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-2:17.2.7-789.g4213e17c.el8.x86_64 2024-03-29T07:52:46.174 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-modules-core-2:17.2.7-789.g4213e17c.el8.noarch 2024-03-29T07:52:46.175 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mon-2:17.2.7-789.g4213e17c.el8.x86_64 2024-03-29T07:52:46.175 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-osd-2:17.2.7-789.g4213e17c.el8.x86_64 2024-03-29T07:52:46.175 INFO:teuthology.orchestra.run.smithi148.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2024-03-29T07:52:46.175 INFO:teuthology.orchestra.run.smithi148.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2024-03-29T07:52:46.175 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cffi-1.11.5-6.el8.x86_64 2024-03-29T07:52:46.175 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cheroot-8.5.2-1.el8.noarch 2024-03-29T07:52:46.175 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2024-03-29T07:52:46.175 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cryptography-3.2.1-7.el8.x86_64 2024-03-29T07:52:46.175 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jaraco-6.2-6.el8.noarch 2024-03-29T07:52:46.175 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2024-03-29T07:52:46.175 INFO:teuthology.orchestra.run.smithi148.stdout: python3-logutils-0.3.5-11.el8.noarch 2024-03-29T07:52:46.175 INFO:teuthology.orchestra.run.smithi148.stdout: python3-mako-1.0.6-14.el8.noarch 2024-03-29T07:52:46.175 INFO:teuthology.orchestra.run.smithi148.stdout: python3-markupsafe-0.23-19.el8.x86_64 2024-03-29T07:52:46.175 INFO:teuthology.orchestra.run.smithi148.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2024-03-29T07:52:46.175 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pecan-1.3.2-9.el8.noarch 2024-03-29T07:52:46.175 INFO:teuthology.orchestra.run.smithi148.stdout: python3-portend-2.6-1.el8.noarch 2024-03-29T07:52:46.176 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2024-03-29T07:52:46.176 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pycparser-2.14-14.el8.noarch 2024-03-29T07:52:46.176 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pytz-2017.2-11.el8.noarch 2024-03-29T07:52:46.176 INFO:teuthology.orchestra.run.smithi148.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2024-03-29T07:52:46.176 INFO:teuthology.orchestra.run.smithi148.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2024-03-29T07:52:46.176 INFO:teuthology.orchestra.run.smithi148.stdout: python3-tempora-1.14.1-5.el8.noarch 2024-03-29T07:52:46.176 INFO:teuthology.orchestra.run.smithi148.stdout: python3-trustme-0.6.0-4.el8.noarch 2024-03-29T07:52:46.176 INFO:teuthology.orchestra.run.smithi148.stdout: python3-waitress-1.4.3-1.el8.noarch 2024-03-29T07:52:46.176 INFO:teuthology.orchestra.run.smithi148.stdout: python3-webob-1.8.5-1.el8.1.noarch 2024-03-29T07:52:46.176 INFO:teuthology.orchestra.run.smithi148.stdout: python3-webtest-2.0.33-1.el8.noarch 2024-03-29T07:52:46.176 INFO:teuthology.orchestra.run.smithi148.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2024-03-29T07:52:46.176 INFO:teuthology.orchestra.run.smithi148.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2024-03-29T07:52:46.176 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T07:52:46.176 INFO:teuthology.orchestra.run.smithi148.stdout:Complete! 2024-03-29T07:52:46.180 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 19/32 2024-03-29T07:52:46.392 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-pytz-2017.2-11.el8.noarch 20/32 2024-03-29T07:52:46.499 DEBUG:teuthology.orchestra.run.smithi148:> sudo yum -y install ceph-base 2024-03-29T07:52:46.602 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 21/32 2024-03-29T07:52:46.959 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-portend-2.6-1.el8.noarch 22/32 2024-03-29T07:52:47.437 INFO:teuthology.orchestra.run.smithi148.stdout:Last metadata expiration check: 0:01:21 ago on Fri 29 Mar 2024 07:51:26 AM UTC. 2024-03-29T07:52:47.680 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 23/32 2024-03-29T07:52:47.982 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 24/32 2024-03-29T07:52:48.383 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-mako-1.0.6-14.el8.noarch 25/32 2024-03-29T07:52:48.748 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 26/32 2024-03-29T07:52:49.064 INFO:teuthology.orchestra.run.smithi148.stdout:Package ceph-base-2:17.2.7-789.g4213e17c.el8.x86_64 is already installed. 2024-03-29T07:52:49.117 INFO:teuthology.orchestra.run.smithi148.stdout:Dependencies resolved. 2024-03-29T07:52:49.118 INFO:teuthology.orchestra.run.smithi148.stdout:Nothing to do. 2024-03-29T07:52:49.118 INFO:teuthology.orchestra.run.smithi148.stdout:Complete! 2024-03-29T07:52:49.198 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-mgr-modules-core-2:17.2.7-789.g4213e17c.el8.n 27/32 2024-03-29T07:52:49.234 DEBUG:teuthology.orchestra.run.smithi148:> sudo yum -y install cephadm 2024-03-29T07:52:49.433 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-mgr-2:17.2.7-789.g4213e17c.el8.x86_64 28/32 2024-03-29T07:52:50.157 INFO:teuthology.orchestra.run.smithi148.stdout:Last metadata expiration check: 0:01:24 ago on Fri 29 Mar 2024 07:51:26 AM UTC. 2024-03-29T07:52:51.200 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-mgr-2:17.2.7-789.g4213e17c.el8.x86_64 28/32 2024-03-29T07:52:51.408 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-osd-2:17.2.7-789.g4213e17c.el8.x86_64 29/32 2024-03-29T07:52:51.859 INFO:teuthology.orchestra.run.smithi148.stdout:Dependencies resolved. 2024-03-29T07:52:51.860 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-03-29T07:52:51.860 INFO:teuthology.orchestra.run.smithi148.stdout: Package Arch Version Repository Size 2024-03-29T07:52:51.860 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-03-29T07:52:51.861 INFO:teuthology.orchestra.run.smithi148.stdout:Installing: 2024-03-29T07:52:51.861 INFO:teuthology.orchestra.run.smithi148.stdout: cephadm noarch 2:17.2.7-789.g4213e17c.el8 ceph-noarch 82 k 2024-03-29T07:52:51.861 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T07:52:51.861 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction Summary 2024-03-29T07:52:51.861 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-03-29T07:52:51.861 INFO:teuthology.orchestra.run.smithi148.stdout:Install 1 Package 2024-03-29T07:52:51.861 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T07:52:51.861 INFO:teuthology.orchestra.run.smithi148.stdout:Total download size: 82 k 2024-03-29T07:52:51.861 INFO:teuthology.orchestra.run.smithi148.stdout:Installed size: 372 k 2024-03-29T07:52:51.861 INFO:teuthology.orchestra.run.smithi148.stdout:Downloading Packages: 2024-03-29T07:52:51.909 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-osd-2:17.2.7-789.g4213e17c.el8.x86_64 29/32 2024-03-29T07:52:52.135 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-mon-2:17.2.7-789.g4213e17c.el8.x86_64 30/32 2024-03-29T07:52:52.145 INFO:teuthology.orchestra.run.smithi148.stdout:cephadm-17.2.7-789.g4213e17c.el8.noarch.rpm 289 kB/s | 82 kB 00:00 2024-03-29T07:52:52.146 INFO:teuthology.orchestra.run.smithi148.stdout:-------------------------------------------------------------------------------- 2024-03-29T07:52:52.146 INFO:teuthology.orchestra.run.smithi148.stdout:Total 287 kB/s | 82 kB 00:00 2024-03-29T07:52:52.147 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction check 2024-03-29T07:52:52.157 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction check succeeded. 2024-03-29T07:52:52.157 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction test 2024-03-29T07:52:52.160 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction test succeeded. 2024-03-29T07:52:52.164 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction 2024-03-29T07:52:52.239 INFO:teuthology.orchestra.run.smithi148.stdout: Preparing : 1/1 2024-03-29T07:52:52.405 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-mon-2:17.2.7-789.g4213e17c.el8.x86_64 30/32 2024-03-29T07:52:52.602 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-mds-2:17.2.7-789.g4213e17c.el8.x86_64 31/32 2024-03-29T07:52:52.645 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-mds-2:17.2.7-789.g4213e17c.el8.x86_64 31/32 2024-03-29T07:52:52.811 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-2:17.2.7-789.g4213e17c.el8.x86_64 32/32 2024-03-29T07:52:52.913 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-2:17.2.7-789.g4213e17c.el8.x86_64 32/32 2024-03-29T07:52:52.913 INFO:teuthology.orchestra.run.smithi037.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-03-29T07:52:52.913 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T07:52:53.454 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-2:17.2.7-789.g4213e17c.el8.x86_64 1/32 2024-03-29T07:52:53.454 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-mds-2:17.2.7-789.g4213e17c.el8.x86_64 2/32 2024-03-29T07:52:53.454 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-mgr-2:17.2.7-789.g4213e17c.el8.x86_64 3/32 2024-03-29T07:52:53.454 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-mon-2:17.2.7-789.g4213e17c.el8.x86_64 4/32 2024-03-29T07:52:53.454 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-osd-2:17.2.7-789.g4213e17c.el8.x86_64 5/32 2024-03-29T07:52:53.454 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-mgr-modules-core-2:17.2.7-789.g4213e17c.el8.n 6/32 2024-03-29T07:52:53.454 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-mako-1.0.6-14.el8.noarch 7/32 2024-03-29T07:52:53.454 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 8/32 2024-03-29T07:52:53.455 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 9/32 2024-03-29T07:52:53.455 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-pytz-2017.2-11.el8.noarch 10/32 2024-03-29T07:52:53.455 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 11/32 2024-03-29T07:52:53.455 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-cffi-1.11.5-6.el8.x86_64 12/32 2024-03-29T07:52:53.455 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-cryptography-3.2.1-7.el8.x86_64 13/32 2024-03-29T07:52:53.455 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 14/32 2024-03-29T07:52:53.455 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 15/32 2024-03-29T07:52:53.455 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 16/32 2024-03-29T07:52:53.455 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 17/32 2024-03-29T07:52:53.455 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 18/32 2024-03-29T07:52:53.455 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 19/32 2024-03-29T07:52:53.455 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 20/32 2024-03-29T07:52:53.455 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 21/32 2024-03-29T07:52:53.456 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 22/32 2024-03-29T07:52:53.456 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 23/32 2024-03-29T07:52:53.456 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-portend-2.6-1.el8.noarch 24/32 2024-03-29T07:52:53.456 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 25/32 2024-03-29T07:52:53.456 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 26/32 2024-03-29T07:52:53.456 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 27/32 2024-03-29T07:52:53.456 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 28/32 2024-03-29T07:52:53.456 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 29/32 2024-03-29T07:52:53.456 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 30/32 2024-03-29T07:52:53.456 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 31/32 2024-03-29T07:52:54.041 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: cephadm-2:17.2.7-789.g4213e17c.el8.noarch 1/1 2024-03-29T07:52:54.246 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 32/32 2024-03-29T07:52:54.246 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T07:52:54.246 INFO:teuthology.orchestra.run.smithi037.stdout:Installed: 2024-03-29T07:52:54.246 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-2:17.2.7-789.g4213e17c.el8.x86_64 2024-03-29T07:52:54.246 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mds-2:17.2.7-789.g4213e17c.el8.x86_64 2024-03-29T07:52:54.246 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mgr-2:17.2.7-789.g4213e17c.el8.x86_64 2024-03-29T07:52:54.246 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mgr-modules-core-2:17.2.7-789.g4213e17c.el8.noarch 2024-03-29T07:52:54.247 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mon-2:17.2.7-789.g4213e17c.el8.x86_64 2024-03-29T07:52:54.247 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-osd-2:17.2.7-789.g4213e17c.el8.x86_64 2024-03-29T07:52:54.247 INFO:teuthology.orchestra.run.smithi037.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2024-03-29T07:52:54.247 INFO:teuthology.orchestra.run.smithi037.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2024-03-29T07:52:54.247 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cffi-1.11.5-6.el8.x86_64 2024-03-29T07:52:54.247 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cheroot-8.5.2-1.el8.noarch 2024-03-29T07:52:54.247 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2024-03-29T07:52:54.247 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cryptography-3.2.1-7.el8.x86_64 2024-03-29T07:52:54.247 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jaraco-6.2-6.el8.noarch 2024-03-29T07:52:54.247 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2024-03-29T07:52:54.247 INFO:teuthology.orchestra.run.smithi037.stdout: python3-logutils-0.3.5-11.el8.noarch 2024-03-29T07:52:54.248 INFO:teuthology.orchestra.run.smithi037.stdout: python3-mako-1.0.6-14.el8.noarch 2024-03-29T07:52:54.248 INFO:teuthology.orchestra.run.smithi037.stdout: python3-markupsafe-0.23-19.el8.x86_64 2024-03-29T07:52:54.248 INFO:teuthology.orchestra.run.smithi037.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2024-03-29T07:52:54.248 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pecan-1.3.2-9.el8.noarch 2024-03-29T07:52:54.248 INFO:teuthology.orchestra.run.smithi037.stdout: python3-portend-2.6-1.el8.noarch 2024-03-29T07:52:54.248 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2024-03-29T07:52:54.248 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pycparser-2.14-14.el8.noarch 2024-03-29T07:52:54.248 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pytz-2017.2-11.el8.noarch 2024-03-29T07:52:54.248 INFO:teuthology.orchestra.run.smithi037.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2024-03-29T07:52:54.248 INFO:teuthology.orchestra.run.smithi037.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2024-03-29T07:52:54.248 INFO:teuthology.orchestra.run.smithi037.stdout: python3-tempora-1.14.1-5.el8.noarch 2024-03-29T07:52:54.248 INFO:teuthology.orchestra.run.smithi037.stdout: python3-trustme-0.6.0-4.el8.noarch 2024-03-29T07:52:54.248 INFO:teuthology.orchestra.run.smithi037.stdout: python3-waitress-1.4.3-1.el8.noarch 2024-03-29T07:52:54.248 INFO:teuthology.orchestra.run.smithi037.stdout: python3-webob-1.8.5-1.el8.1.noarch 2024-03-29T07:52:54.248 INFO:teuthology.orchestra.run.smithi037.stdout: python3-webtest-2.0.33-1.el8.noarch 2024-03-29T07:52:54.249 INFO:teuthology.orchestra.run.smithi037.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2024-03-29T07:52:54.249 INFO:teuthology.orchestra.run.smithi037.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2024-03-29T07:52:54.249 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T07:52:54.249 INFO:teuthology.orchestra.run.smithi037.stdout:Complete! 2024-03-29T07:52:54.261 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : cephadm-2:17.2.7-789.g4213e17c.el8.noarch 1/1 2024-03-29T07:52:54.566 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: cephadm-2:17.2.7-789.g4213e17c.el8.noarch 1/1 2024-03-29T07:52:54.574 DEBUG:teuthology.orchestra.run.smithi037:> sudo yum -y install ceph-base 2024-03-29T07:52:55.015 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : cephadm-2:17.2.7-789.g4213e17c.el8.noarch 1/1 2024-03-29T07:52:55.015 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T07:52:55.015 INFO:teuthology.orchestra.run.smithi148.stdout:Installed: 2024-03-29T07:52:55.016 INFO:teuthology.orchestra.run.smithi148.stdout: cephadm-2:17.2.7-789.g4213e17c.el8.noarch 2024-03-29T07:52:55.016 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T07:52:55.016 INFO:teuthology.orchestra.run.smithi148.stdout:Complete! 2024-03-29T07:52:55.223 DEBUG:teuthology.orchestra.run.smithi148:> sudo yum -y install ceph-immutable-object-cache 2024-03-29T07:52:55.543 INFO:teuthology.orchestra.run.smithi037.stdout:Last metadata expiration check: 0:01:29 ago on Fri 29 Mar 2024 07:51:26 AM UTC. 2024-03-29T07:52:56.155 INFO:teuthology.orchestra.run.smithi148.stdout:Last metadata expiration check: 0:01:30 ago on Fri 29 Mar 2024 07:51:26 AM UTC. 2024-03-29T07:52:57.310 INFO:teuthology.orchestra.run.smithi037.stdout:Package ceph-base-2:17.2.7-789.g4213e17c.el8.x86_64 is already installed. 2024-03-29T07:52:57.366 INFO:teuthology.orchestra.run.smithi037.stdout:Dependencies resolved. 2024-03-29T07:52:57.367 INFO:teuthology.orchestra.run.smithi037.stdout:Nothing to do. 2024-03-29T07:52:57.368 INFO:teuthology.orchestra.run.smithi037.stdout:Complete! 2024-03-29T07:52:57.491 DEBUG:teuthology.orchestra.run.smithi037:> sudo yum -y install cephadm 2024-03-29T07:52:57.921 INFO:teuthology.orchestra.run.smithi148.stdout:Dependencies resolved. 2024-03-29T07:52:57.922 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-03-29T07:52:57.922 INFO:teuthology.orchestra.run.smithi148.stdout: Package Arch Version Repo Size 2024-03-29T07:52:57.922 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-03-29T07:52:57.922 INFO:teuthology.orchestra.run.smithi148.stdout:Installing: 2024-03-29T07:52:57.922 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-immutable-object-cache x86_64 2:17.2.7-789.g4213e17c.el8 ceph 162 k 2024-03-29T07:52:57.922 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T07:52:57.922 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction Summary 2024-03-29T07:52:57.922 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-03-29T07:52:57.922 INFO:teuthology.orchestra.run.smithi148.stdout:Install 1 Package 2024-03-29T07:52:57.923 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T07:52:57.923 INFO:teuthology.orchestra.run.smithi148.stdout:Total download size: 162 k 2024-03-29T07:52:57.923 INFO:teuthology.orchestra.run.smithi148.stdout:Installed size: 494 k 2024-03-29T07:52:57.923 INFO:teuthology.orchestra.run.smithi148.stdout:Downloading Packages: 2024-03-29T07:52:58.409 INFO:teuthology.orchestra.run.smithi148.stdout:ceph-immutable-object-cache-17.2.7-789.g4213e17 333 kB/s | 162 kB 00:00 2024-03-29T07:52:58.410 INFO:teuthology.orchestra.run.smithi148.stdout:-------------------------------------------------------------------------------- 2024-03-29T07:52:58.410 INFO:teuthology.orchestra.run.smithi148.stdout:Total 332 kB/s | 162 kB 00:00 2024-03-29T07:52:58.411 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction check 2024-03-29T07:52:58.433 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction check succeeded. 2024-03-29T07:52:58.433 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction test 2024-03-29T07:52:58.477 INFO:teuthology.orchestra.run.smithi037.stdout:Last metadata expiration check: 0:01:32 ago on Fri 29 Mar 2024 07:51:26 AM UTC. 2024-03-29T07:52:58.491 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction test succeeded. 2024-03-29T07:52:58.495 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction 2024-03-29T07:52:59.229 INFO:teuthology.orchestra.run.smithi148.stdout: Preparing : 1/1 2024-03-29T07:52:59.424 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-immutable-object-cache-2:17.2.7-789.g4213e17c.e 1/1 2024-03-29T07:53:00.067 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-immutable-object-cache-2:17.2.7-789.g4213e17c.e 1/1 2024-03-29T07:53:00.288 INFO:teuthology.orchestra.run.smithi037.stdout:Dependencies resolved. 2024-03-29T07:53:00.289 INFO:teuthology.orchestra.run.smithi037.stdout:================================================================================ 2024-03-29T07:53:00.289 INFO:teuthology.orchestra.run.smithi037.stdout: Package Arch Version Repository Size 2024-03-29T07:53:00.289 INFO:teuthology.orchestra.run.smithi037.stdout:================================================================================ 2024-03-29T07:53:00.289 INFO:teuthology.orchestra.run.smithi037.stdout:Installing: 2024-03-29T07:53:00.289 INFO:teuthology.orchestra.run.smithi037.stdout: cephadm noarch 2:17.2.7-789.g4213e17c.el8 ceph-noarch 82 k 2024-03-29T07:53:00.289 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T07:53:00.289 INFO:teuthology.orchestra.run.smithi037.stdout:Transaction Summary 2024-03-29T07:53:00.289 INFO:teuthology.orchestra.run.smithi037.stdout:================================================================================ 2024-03-29T07:53:00.289 INFO:teuthology.orchestra.run.smithi037.stdout:Install 1 Package 2024-03-29T07:53:00.289 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T07:53:00.290 INFO:teuthology.orchestra.run.smithi037.stdout:Total download size: 82 k 2024-03-29T07:53:00.290 INFO:teuthology.orchestra.run.smithi037.stdout:Installed size: 372 k 2024-03-29T07:53:00.290 INFO:teuthology.orchestra.run.smithi037.stdout:Downloading Packages: 2024-03-29T07:53:00.521 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-immutable-object-cache-2:17.2.7-789.g4213e17c.e 1/1 2024-03-29T07:53:00.522 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T07:53:00.522 INFO:teuthology.orchestra.run.smithi148.stdout:Installed: 2024-03-29T07:53:00.522 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-immutable-object-cache-2:17.2.7-789.g4213e17c.el8.x86_64 2024-03-29T07:53:00.522 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T07:53:00.522 INFO:teuthology.orchestra.run.smithi148.stdout:Complete! 2024-03-29T07:53:00.581 INFO:teuthology.orchestra.run.smithi037.stdout:cephadm-17.2.7-789.g4213e17c.el8.noarch.rpm 282 kB/s | 82 kB 00:00 2024-03-29T07:53:00.581 INFO:teuthology.orchestra.run.smithi037.stdout:-------------------------------------------------------------------------------- 2024-03-29T07:53:00.582 INFO:teuthology.orchestra.run.smithi037.stdout:Total 281 kB/s | 82 kB 00:00 2024-03-29T07:53:00.582 INFO:teuthology.orchestra.run.smithi037.stdout:Running transaction check 2024-03-29T07:53:00.593 INFO:teuthology.orchestra.run.smithi037.stdout:Transaction check succeeded. 2024-03-29T07:53:00.593 INFO:teuthology.orchestra.run.smithi037.stdout:Running transaction test 2024-03-29T07:53:00.596 INFO:teuthology.orchestra.run.smithi037.stdout:Transaction test succeeded. 2024-03-29T07:53:00.600 INFO:teuthology.orchestra.run.smithi037.stdout:Running transaction 2024-03-29T07:53:00.661 INFO:teuthology.orchestra.run.smithi037.stdout: Preparing : 1/1 2024-03-29T07:53:00.758 DEBUG:teuthology.orchestra.run.smithi148:> sudo yum -y install ceph-mgr 2024-03-29T07:53:01.276 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: cephadm-2:17.2.7-789.g4213e17c.el8.noarch 1/1 2024-03-29T07:53:01.470 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : cephadm-2:17.2.7-789.g4213e17c.el8.noarch 1/1 2024-03-29T07:53:01.701 INFO:teuthology.orchestra.run.smithi148.stdout:Last metadata expiration check: 0:01:35 ago on Fri 29 Mar 2024 07:51:26 AM UTC. 2024-03-29T07:53:01.776 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: cephadm-2:17.2.7-789.g4213e17c.el8.noarch 1/1 2024-03-29T07:53:02.206 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : cephadm-2:17.2.7-789.g4213e17c.el8.noarch 1/1 2024-03-29T07:53:02.206 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T07:53:02.206 INFO:teuthology.orchestra.run.smithi037.stdout:Installed: 2024-03-29T07:53:02.206 INFO:teuthology.orchestra.run.smithi037.stdout: cephadm-2:17.2.7-789.g4213e17c.el8.noarch 2024-03-29T07:53:02.206 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T07:53:02.206 INFO:teuthology.orchestra.run.smithi037.stdout:Complete! 2024-03-29T07:53:02.430 DEBUG:teuthology.orchestra.run.smithi037:> sudo yum -y install ceph-immutable-object-cache 2024-03-29T07:53:03.340 INFO:teuthology.orchestra.run.smithi148.stdout:Package ceph-mgr-2:17.2.7-789.g4213e17c.el8.x86_64 is already installed. 2024-03-29T07:53:03.391 INFO:teuthology.orchestra.run.smithi037.stdout:Last metadata expiration check: 0:01:37 ago on Fri 29 Mar 2024 07:51:26 AM UTC. 2024-03-29T07:53:03.392 INFO:teuthology.orchestra.run.smithi148.stdout:Dependencies resolved. 2024-03-29T07:53:03.393 INFO:teuthology.orchestra.run.smithi148.stdout:Nothing to do. 2024-03-29T07:53:03.394 INFO:teuthology.orchestra.run.smithi148.stdout:Complete! 2024-03-29T07:53:03.508 DEBUG:teuthology.orchestra.run.smithi148:> sudo yum -y install ceph-mgr-dashboard 2024-03-29T07:53:04.433 INFO:teuthology.orchestra.run.smithi148.stdout:Last metadata expiration check: 0:01:38 ago on Fri 29 Mar 2024 07:51:26 AM UTC. 2024-03-29T07:53:05.211 INFO:teuthology.orchestra.run.smithi037.stdout:Dependencies resolved. 2024-03-29T07:53:05.212 INFO:teuthology.orchestra.run.smithi037.stdout:================================================================================ 2024-03-29T07:53:05.212 INFO:teuthology.orchestra.run.smithi037.stdout: Package Arch Version Repo Size 2024-03-29T07:53:05.212 INFO:teuthology.orchestra.run.smithi037.stdout:================================================================================ 2024-03-29T07:53:05.212 INFO:teuthology.orchestra.run.smithi037.stdout:Installing: 2024-03-29T07:53:05.212 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-immutable-object-cache x86_64 2:17.2.7-789.g4213e17c.el8 ceph 162 k 2024-03-29T07:53:05.212 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T07:53:05.212 INFO:teuthology.orchestra.run.smithi037.stdout:Transaction Summary 2024-03-29T07:53:05.213 INFO:teuthology.orchestra.run.smithi037.stdout:================================================================================ 2024-03-29T07:53:05.213 INFO:teuthology.orchestra.run.smithi037.stdout:Install 1 Package 2024-03-29T07:53:05.213 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T07:53:05.213 INFO:teuthology.orchestra.run.smithi037.stdout:Total download size: 162 k 2024-03-29T07:53:05.213 INFO:teuthology.orchestra.run.smithi037.stdout:Installed size: 494 k 2024-03-29T07:53:05.213 INFO:teuthology.orchestra.run.smithi037.stdout:Downloading Packages: 2024-03-29T07:53:05.419 INFO:teuthology.orchestra.run.smithi037.stdout:ceph-immutable-object-cache-17.2.7-789.g4213e17 788 kB/s | 162 kB 00:00 2024-03-29T07:53:05.419 INFO:teuthology.orchestra.run.smithi037.stdout:-------------------------------------------------------------------------------- 2024-03-29T07:53:05.420 INFO:teuthology.orchestra.run.smithi037.stdout:Total 783 kB/s | 162 kB 00:00 2024-03-29T07:53:05.420 INFO:teuthology.orchestra.run.smithi037.stdout:Running transaction check 2024-03-29T07:53:05.434 INFO:teuthology.orchestra.run.smithi037.stdout:Transaction check succeeded. 2024-03-29T07:53:05.434 INFO:teuthology.orchestra.run.smithi037.stdout:Running transaction test 2024-03-29T07:53:05.491 INFO:teuthology.orchestra.run.smithi037.stdout:Transaction test succeeded. 2024-03-29T07:53:05.494 INFO:teuthology.orchestra.run.smithi037.stdout:Running transaction 2024-03-29T07:53:05.628 INFO:teuthology.orchestra.run.smithi037.stdout: Preparing : 1/1 2024-03-29T07:53:05.800 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-immutable-object-cache-2:17.2.7-789.g4213e17c.e 1/1 2024-03-29T07:53:06.286 INFO:teuthology.orchestra.run.smithi148.stdout:Dependencies resolved. 2024-03-29T07:53:06.287 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-03-29T07:53:06.288 INFO:teuthology.orchestra.run.smithi148.stdout: Package Arch Version Repository Size 2024-03-29T07:53:06.288 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-03-29T07:53:06.288 INFO:teuthology.orchestra.run.smithi148.stdout:Installing: 2024-03-29T07:53:06.288 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-dashboard noarch 2:17.2.7-789.g4213e17c.el8 ceph-noarch 1.7 M 2024-03-29T07:53:06.288 INFO:teuthology.orchestra.run.smithi148.stdout:Installing dependencies: 2024-03-29T07:53:06.288 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-grafana-dashboards noarch 2:17.2.7-789.g4213e17c.el8 ceph-noarch 23 k 2024-03-29T07:53:06.288 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-prometheus-alerts noarch 2:17.2.7-789.g4213e17c.el8 ceph-noarch 14 k 2024-03-29T07:53:06.288 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jwt noarch 1.6.1-2.el8 baseos 43 k 2024-03-29T07:53:06.288 INFO:teuthology.orchestra.run.smithi148.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2024-03-29T07:53:06.288 INFO:teuthology.orchestra.run.smithi148.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2024-03-29T07:53:06.288 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T07:53:06.288 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction Summary 2024-03-29T07:53:06.288 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-03-29T07:53:06.289 INFO:teuthology.orchestra.run.smithi148.stdout:Install 6 Packages 2024-03-29T07:53:06.289 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T07:53:06.289 INFO:teuthology.orchestra.run.smithi148.stdout:Total download size: 2.0 M 2024-03-29T07:53:06.289 INFO:teuthology.orchestra.run.smithi148.stdout:Installed size: 8.0 M 2024-03-29T07:53:06.289 INFO:teuthology.orchestra.run.smithi148.stdout:Downloading Packages: 2024-03-29T07:53:06.482 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-immutable-object-cache-2:17.2.7-789.g4213e17c.e 1/1 2024-03-29T07:53:06.845 INFO:teuthology.orchestra.run.smithi148.stdout:(1/6): ceph-grafana-dashboards-17.2.7-789.g4213 68 kB/s | 23 kB 00:00 2024-03-29T07:53:06.862 INFO:teuthology.orchestra.run.smithi148.stdout:(2/6): ceph-prometheus-alerts-17.2.7-789.g4213e 40 kB/s | 14 kB 00:00 2024-03-29T07:53:06.890 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-immutable-object-cache-2:17.2.7-789.g4213e17c.e 1/1 2024-03-29T07:53:06.890 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T07:53:06.890 INFO:teuthology.orchestra.run.smithi037.stdout:Installed: 2024-03-29T07:53:06.890 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-immutable-object-cache-2:17.2.7-789.g4213e17c.el8.x86_64 2024-03-29T07:53:06.890 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T07:53:06.890 INFO:teuthology.orchestra.run.smithi037.stdout:Complete! 2024-03-29T07:53:07.079 INFO:teuthology.orchestra.run.smithi148.stdout:(3/6): python3-jwt-1.6.1-2.el8.noarch.rpm 182 kB/s | 43 kB 00:00 2024-03-29T07:53:07.130 INFO:teuthology.orchestra.run.smithi148.stdout:(4/6): python3-repoze-lru-0.7-6.el8.noarch.rpm 126 kB/s | 34 kB 00:00 2024-03-29T07:53:07.133 DEBUG:teuthology.orchestra.run.smithi037:> sudo yum -y install ceph-mgr 2024-03-29T07:53:07.255 INFO:teuthology.orchestra.run.smithi148.stdout:(5/6): ceph-mgr-dashboard-17.2.7-789.g4213e17c. 2.3 MB/s | 1.7 MB 00:00 2024-03-29T07:53:07.530 INFO:teuthology.orchestra.run.smithi148.stdout:(6/6): python3-routes-2.4.1-12.el8.noarch.rpm 436 kB/s | 196 kB 00:00 2024-03-29T07:53:07.532 INFO:teuthology.orchestra.run.smithi148.stdout:-------------------------------------------------------------------------------- 2024-03-29T07:53:07.533 INFO:teuthology.orchestra.run.smithi148.stdout:Total 1.6 MB/s | 2.0 MB 00:01 2024-03-29T07:53:07.554 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction check 2024-03-29T07:53:07.567 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction check succeeded. 2024-03-29T07:53:07.567 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction test 2024-03-29T07:53:07.667 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction test succeeded. 2024-03-29T07:53:07.670 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction 2024-03-29T07:53:07.985 INFO:teuthology.orchestra.run.smithi148.stdout: Preparing : 1/1 2024-03-29T07:53:08.103 INFO:teuthology.orchestra.run.smithi037.stdout:Last metadata expiration check: 0:01:42 ago on Fri 29 Mar 2024 07:51:26 AM UTC. 2024-03-29T07:53:08.210 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 1/6 2024-03-29T07:53:08.470 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 2/6 2024-03-29T07:53:08.708 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 3/6 2024-03-29T07:53:08.830 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-prometheus-alerts-2:17.2.7-789.g4213e17c.el8.no 4/6 2024-03-29T07:53:09.255 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-grafana-dashboards-2:17.2.7-789.g4213e17c.el8.n 5/6 2024-03-29T07:53:09.804 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-mgr-dashboard-2:17.2.7-789.g4213e17c.el8.noarch 6/6 2024-03-29T07:53:09.865 INFO:teuthology.orchestra.run.smithi037.stdout:Package ceph-mgr-2:17.2.7-789.g4213e17c.el8.x86_64 is already installed. 2024-03-29T07:53:09.921 INFO:teuthology.orchestra.run.smithi037.stdout:Dependencies resolved. 2024-03-29T07:53:09.923 INFO:teuthology.orchestra.run.smithi037.stdout:Nothing to do. 2024-03-29T07:53:09.923 INFO:teuthology.orchestra.run.smithi037.stdout:Complete! 2024-03-29T07:53:10.038 DEBUG:teuthology.orchestra.run.smithi037:> sudo yum -y install ceph-mgr-dashboard 2024-03-29T07:53:10.152 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-mgr-dashboard-2:17.2.7-789.g4213e17c.el8.noarch 6/6 2024-03-29T07:53:10.153 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-grafana-dashboards-2:17.2.7-789.g4213e17c.el8.n 1/6 2024-03-29T07:53:10.153 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-mgr-dashboard-2:17.2.7-789.g4213e17c.el8.noarch 2/6 2024-03-29T07:53:10.153 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-prometheus-alerts-2:17.2.7-789.g4213e17c.el8.no 3/6 2024-03-29T07:53:10.153 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 4/6 2024-03-29T07:53:10.153 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 5/6 2024-03-29T07:53:10.586 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 6/6 2024-03-29T07:53:10.586 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T07:53:10.586 INFO:teuthology.orchestra.run.smithi148.stdout:Installed: 2024-03-29T07:53:10.586 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-grafana-dashboards-2:17.2.7-789.g4213e17c.el8.noarch 2024-03-29T07:53:10.586 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-dashboard-2:17.2.7-789.g4213e17c.el8.noarch 2024-03-29T07:53:10.587 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-prometheus-alerts-2:17.2.7-789.g4213e17c.el8.noarch 2024-03-29T07:53:10.587 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jwt-1.6.1-2.el8.noarch 2024-03-29T07:53:10.587 INFO:teuthology.orchestra.run.smithi148.stdout: python3-repoze-lru-0.7-6.el8.noarch 2024-03-29T07:53:10.587 INFO:teuthology.orchestra.run.smithi148.stdout: python3-routes-2.4.1-12.el8.noarch 2024-03-29T07:53:10.587 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T07:53:10.587 INFO:teuthology.orchestra.run.smithi148.stdout:Complete! 2024-03-29T07:53:10.749 DEBUG:teuthology.orchestra.run.smithi148:> sudo yum -y install ceph-mgr-diskprediction-local 2024-03-29T07:53:11.010 INFO:teuthology.orchestra.run.smithi037.stdout:Last metadata expiration check: 0:01:45 ago on Fri 29 Mar 2024 07:51:26 AM UTC. 2024-03-29T07:53:11.670 INFO:teuthology.orchestra.run.smithi148.stdout:Last metadata expiration check: 0:01:45 ago on Fri 29 Mar 2024 07:51:26 AM UTC. 2024-03-29T07:53:12.863 INFO:teuthology.orchestra.run.smithi037.stdout:Dependencies resolved. 2024-03-29T07:53:12.864 INFO:teuthology.orchestra.run.smithi037.stdout:================================================================================ 2024-03-29T07:53:12.864 INFO:teuthology.orchestra.run.smithi037.stdout: Package Arch Version Repository Size 2024-03-29T07:53:12.865 INFO:teuthology.orchestra.run.smithi037.stdout:================================================================================ 2024-03-29T07:53:12.865 INFO:teuthology.orchestra.run.smithi037.stdout:Installing: 2024-03-29T07:53:12.865 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mgr-dashboard noarch 2:17.2.7-789.g4213e17c.el8 ceph-noarch 1.7 M 2024-03-29T07:53:12.865 INFO:teuthology.orchestra.run.smithi037.stdout:Installing dependencies: 2024-03-29T07:53:12.865 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-grafana-dashboards noarch 2:17.2.7-789.g4213e17c.el8 ceph-noarch 23 k 2024-03-29T07:53:12.865 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-prometheus-alerts noarch 2:17.2.7-789.g4213e17c.el8 ceph-noarch 14 k 2024-03-29T07:53:12.865 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jwt noarch 1.6.1-2.el8 baseos 43 k 2024-03-29T07:53:12.865 INFO:teuthology.orchestra.run.smithi037.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2024-03-29T07:53:12.865 INFO:teuthology.orchestra.run.smithi037.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2024-03-29T07:53:12.865 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T07:53:12.865 INFO:teuthology.orchestra.run.smithi037.stdout:Transaction Summary 2024-03-29T07:53:12.865 INFO:teuthology.orchestra.run.smithi037.stdout:================================================================================ 2024-03-29T07:53:12.865 INFO:teuthology.orchestra.run.smithi037.stdout:Install 6 Packages 2024-03-29T07:53:12.865 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T07:53:12.866 INFO:teuthology.orchestra.run.smithi037.stdout:Total download size: 2.0 M 2024-03-29T07:53:12.866 INFO:teuthology.orchestra.run.smithi037.stdout:Installed size: 8.0 M 2024-03-29T07:53:12.866 INFO:teuthology.orchestra.run.smithi037.stdout:Downloading Packages: 2024-03-29T07:53:13.338 INFO:teuthology.orchestra.run.smithi037.stdout:(1/6): ceph-grafana-dashboards-17.2.7-789.g4213 160 kB/s | 23 kB 00:00 2024-03-29T07:53:13.363 INFO:teuthology.orchestra.run.smithi037.stdout:(2/6): ceph-prometheus-alerts-17.2.7-789.g4213e 85 kB/s | 14 kB 00:00 2024-03-29T07:53:13.364 INFO:teuthology.orchestra.run.smithi148.stdout:Dependencies resolved. 2024-03-29T07:53:13.365 INFO:teuthology.orchestra.run.smithi148.stdout:============================================================================================ 2024-03-29T07:53:13.365 INFO:teuthology.orchestra.run.smithi148.stdout: Package Arch Version Repository Size 2024-03-29T07:53:13.365 INFO:teuthology.orchestra.run.smithi148.stdout:============================================================================================ 2024-03-29T07:53:13.365 INFO:teuthology.orchestra.run.smithi148.stdout:Installing: 2024-03-29T07:53:13.366 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-diskprediction-local noarch 2:17.2.7-789.g4213e17c.el8 ceph-noarch 7.4 M 2024-03-29T07:53:13.366 INFO:teuthology.orchestra.run.smithi148.stdout:Installing dependencies: 2024-03-29T07:53:13.366 INFO:teuthology.orchestra.run.smithi148.stdout: libgfortran x86_64 8.5.0-21.el8 baseos 645 k 2024-03-29T07:53:13.366 INFO:teuthology.orchestra.run.smithi148.stdout: libquadmath x86_64 8.5.0-21.el8 baseos 172 k 2024-03-29T07:53:13.366 INFO:teuthology.orchestra.run.smithi148.stdout: openblas x86_64 0.3.15-4.el8 appstream 4.7 M 2024-03-29T07:53:13.366 INFO:teuthology.orchestra.run.smithi148.stdout: openblas-threads x86_64 0.3.15-4.el8 appstream 4.9 M 2024-03-29T07:53:13.366 INFO:teuthology.orchestra.run.smithi148.stdout: python3-numpy x86_64 1:1.14.3-10.el8 appstream 3.7 M 2024-03-29T07:53:13.366 INFO:teuthology.orchestra.run.smithi148.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 appstream 225 k 2024-03-29T07:53:13.366 INFO:teuthology.orchestra.run.smithi148.stdout: python3-scipy x86_64 1.0.0-21.module_el8+762+77bd8591 appstream 14 M 2024-03-29T07:53:13.366 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T07:53:13.366 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction Summary 2024-03-29T07:53:13.366 INFO:teuthology.orchestra.run.smithi148.stdout:============================================================================================ 2024-03-29T07:53:13.366 INFO:teuthology.orchestra.run.smithi148.stdout:Install 8 Packages 2024-03-29T07:53:13.367 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T07:53:13.367 INFO:teuthology.orchestra.run.smithi148.stdout:Total download size: 36 M 2024-03-29T07:53:13.367 INFO:teuthology.orchestra.run.smithi148.stdout:Installed size: 210 M 2024-03-29T07:53:13.367 INFO:teuthology.orchestra.run.smithi148.stdout:Downloading Packages: 2024-03-29T07:53:13.447 INFO:teuthology.orchestra.run.smithi037.stdout:(3/6): python3-repoze-lru-0.7-6.el8.noarch.rpm 405 kB/s | 34 kB 00:00 2024-03-29T07:53:13.555 INFO:teuthology.orchestra.run.smithi037.stdout:(4/6): python3-routes-2.4.1-12.el8.noarch.rpm 1.8 MB/s | 196 kB 00:00 2024-03-29T07:53:13.589 INFO:teuthology.orchestra.run.smithi037.stdout:(5/6): python3-jwt-1.6.1-2.el8.noarch.rpm 170 kB/s | 43 kB 00:00 2024-03-29T07:53:14.122 INFO:teuthology.orchestra.run.smithi037.stdout:(6/6): ceph-mgr-dashboard-17.2.7-789.g4213e17c. 1.8 MB/s | 1.7 MB 00:00 2024-03-29T07:53:14.124 INFO:teuthology.orchestra.run.smithi037.stdout:-------------------------------------------------------------------------------- 2024-03-29T07:53:14.125 INFO:teuthology.orchestra.run.smithi037.stdout:Total 1.6 MB/s | 2.0 MB 00:01 2024-03-29T07:53:14.150 INFO:teuthology.orchestra.run.smithi037.stdout:Running transaction check 2024-03-29T07:53:14.164 INFO:teuthology.orchestra.run.smithi037.stdout:Transaction check succeeded. 2024-03-29T07:53:14.164 INFO:teuthology.orchestra.run.smithi037.stdout:Running transaction test 2024-03-29T07:53:14.282 INFO:teuthology.orchestra.run.smithi037.stdout:Transaction test succeeded. 2024-03-29T07:53:14.286 INFO:teuthology.orchestra.run.smithi037.stdout:Running transaction 2024-03-29T07:53:14.312 INFO:teuthology.orchestra.run.smithi148.stdout:(1/8): openblas-0.3.15-4.el8.x86_64.rpm 9.6 MB/s | 4.7 MB 00:00 2024-03-29T07:53:14.388 INFO:teuthology.orchestra.run.smithi148.stdout:(2/8): openblas-threads-0.3.15-4.el8.x86_64.rpm 8.6 MB/s | 4.9 MB 00:00 2024-03-29T07:53:14.554 INFO:teuthology.orchestra.run.smithi148.stdout:(3/8): python3-numpy-f2py-1.14.3-10.el8.x86_64. 1.3 MB/s | 225 kB 00:00 2024-03-29T07:53:14.583 INFO:teuthology.orchestra.run.smithi037.stdout: Preparing : 1/1 2024-03-29T07:53:14.630 INFO:teuthology.orchestra.run.smithi148.stdout:(4/8): python3-numpy-1.14.3-10.el8.x86_64.rpm 12 MB/s | 3.7 MB 00:00 2024-03-29T07:53:14.930 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 1/6 2024-03-29T07:53:15.189 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 2/6 2024-03-29T07:53:15.365 INFO:teuthology.orchestra.run.smithi148.stdout:(5/8): ceph-mgr-diskprediction-local-17.2.7-789 4.8 MB/s | 7.4 MB 00:01 2024-03-29T07:53:15.377 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 3/6 2024-03-29T07:53:15.499 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-prometheus-alerts-2:17.2.7-789.g4213e17c.el8.no 4/6 2024-03-29T07:53:15.791 INFO:teuthology.orchestra.run.smithi148.stdout:(6/8): python3-scipy-1.0.0-21.module_el8+762+77 11 MB/s | 14 MB 00:01 2024-03-29T07:53:15.816 INFO:teuthology.orchestra.run.smithi148.stdout:(7/8): libquadmath-8.5.0-21.el8.x86_64.rpm 383 kB/s | 172 kB 00:00 2024-03-29T07:53:15.900 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-grafana-dashboards-2:17.2.7-789.g4213e17c.el8.n 5/6 2024-03-29T07:53:16.058 INFO:teuthology.orchestra.run.smithi148.stdout:(8/8): libgfortran-8.5.0-21.el8.x86_64.rpm 452 kB/s | 645 kB 00:01 2024-03-29T07:53:16.059 INFO:teuthology.orchestra.run.smithi148.stdout:-------------------------------------------------------------------------------- 2024-03-29T07:53:16.059 INFO:teuthology.orchestra.run.smithi148.stdout:Total 13 MB/s | 36 MB 00:02 2024-03-29T07:53:16.121 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-mgr-dashboard-2:17.2.7-789.g4213e17c.el8.noarch 6/6 2024-03-29T07:53:16.371 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction check 2024-03-29T07:53:16.393 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction check succeeded. 2024-03-29T07:53:16.393 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction test 2024-03-29T07:53:16.473 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-mgr-dashboard-2:17.2.7-789.g4213e17c.el8.noarch 6/6 2024-03-29T07:53:16.473 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-grafana-dashboards-2:17.2.7-789.g4213e17c.el8.n 1/6 2024-03-29T07:53:16.473 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-mgr-dashboard-2:17.2.7-789.g4213e17c.el8.noarch 2/6 2024-03-29T07:53:16.473 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-prometheus-alerts-2:17.2.7-789.g4213e17c.el8.no 3/6 2024-03-29T07:53:16.473 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 4/6 2024-03-29T07:53:16.473 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 5/6 2024-03-29T07:53:16.842 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction test succeeded. 2024-03-29T07:53:16.846 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction 2024-03-29T07:53:16.922 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 6/6 2024-03-29T07:53:16.922 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T07:53:16.922 INFO:teuthology.orchestra.run.smithi037.stdout:Installed: 2024-03-29T07:53:16.922 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-grafana-dashboards-2:17.2.7-789.g4213e17c.el8.noarch 2024-03-29T07:53:16.922 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mgr-dashboard-2:17.2.7-789.g4213e17c.el8.noarch 2024-03-29T07:53:16.922 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-prometheus-alerts-2:17.2.7-789.g4213e17c.el8.noarch 2024-03-29T07:53:16.922 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jwt-1.6.1-2.el8.noarch 2024-03-29T07:53:16.922 INFO:teuthology.orchestra.run.smithi037.stdout: python3-repoze-lru-0.7-6.el8.noarch 2024-03-29T07:53:16.922 INFO:teuthology.orchestra.run.smithi037.stdout: python3-routes-2.4.1-12.el8.noarch 2024-03-29T07:53:16.922 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T07:53:16.923 INFO:teuthology.orchestra.run.smithi037.stdout:Complete! 2024-03-29T07:53:17.073 DEBUG:teuthology.orchestra.run.smithi037:> sudo yum -y install ceph-mgr-diskprediction-local 2024-03-29T07:53:17.474 INFO:teuthology.orchestra.run.smithi148.stdout: Preparing : 1/1 2024-03-29T07:53:17.645 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libquadmath-8.5.0-21.el8.x86_64 1/8 2024-03-29T07:53:17.818 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: libquadmath-8.5.0-21.el8.x86_64 1/8 2024-03-29T07:53:17.978 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libgfortran-8.5.0-21.el8.x86_64 2/8 2024-03-29T07:53:18.032 INFO:teuthology.orchestra.run.smithi037.stdout:Last metadata expiration check: 0:01:52 ago on Fri 29 Mar 2024 07:51:26 AM UTC. 2024-03-29T07:53:18.623 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: libgfortran-8.5.0-21.el8.x86_64 2/8 2024-03-29T07:53:18.820 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : openblas-0.3.15-4.el8.x86_64 3/8 2024-03-29T07:53:19.501 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: openblas-0.3.15-4.el8.x86_64 3/8 2024-03-29T07:53:19.688 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : openblas-threads-0.3.15-4.el8.x86_64 4/8 2024-03-29T07:53:19.852 INFO:teuthology.orchestra.run.smithi037.stdout:Dependencies resolved. 2024-03-29T07:53:19.854 INFO:teuthology.orchestra.run.smithi037.stdout:============================================================================================ 2024-03-29T07:53:19.854 INFO:teuthology.orchestra.run.smithi037.stdout: Package Arch Version Repository Size 2024-03-29T07:53:19.854 INFO:teuthology.orchestra.run.smithi037.stdout:============================================================================================ 2024-03-29T07:53:19.854 INFO:teuthology.orchestra.run.smithi037.stdout:Installing: 2024-03-29T07:53:19.854 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mgr-diskprediction-local noarch 2:17.2.7-789.g4213e17c.el8 ceph-noarch 7.4 M 2024-03-29T07:53:19.854 INFO:teuthology.orchestra.run.smithi037.stdout:Installing dependencies: 2024-03-29T07:53:19.854 INFO:teuthology.orchestra.run.smithi037.stdout: libgfortran x86_64 8.5.0-21.el8 baseos 645 k 2024-03-29T07:53:19.855 INFO:teuthology.orchestra.run.smithi037.stdout: libquadmath x86_64 8.5.0-21.el8 baseos 172 k 2024-03-29T07:53:19.855 INFO:teuthology.orchestra.run.smithi037.stdout: openblas x86_64 0.3.15-4.el8 appstream 4.7 M 2024-03-29T07:53:19.855 INFO:teuthology.orchestra.run.smithi037.stdout: openblas-threads x86_64 0.3.15-4.el8 appstream 4.9 M 2024-03-29T07:53:19.855 INFO:teuthology.orchestra.run.smithi037.stdout: python3-numpy x86_64 1:1.14.3-10.el8 appstream 3.7 M 2024-03-29T07:53:19.855 INFO:teuthology.orchestra.run.smithi037.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 appstream 225 k 2024-03-29T07:53:19.855 INFO:teuthology.orchestra.run.smithi037.stdout: python3-scipy x86_64 1.0.0-21.module_el8+762+77bd8591 appstream 14 M 2024-03-29T07:53:19.855 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T07:53:19.855 INFO:teuthology.orchestra.run.smithi037.stdout:Transaction Summary 2024-03-29T07:53:19.855 INFO:teuthology.orchestra.run.smithi037.stdout:============================================================================================ 2024-03-29T07:53:19.855 INFO:teuthology.orchestra.run.smithi037.stdout:Install 8 Packages 2024-03-29T07:53:19.855 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T07:53:19.856 INFO:teuthology.orchestra.run.smithi037.stdout:Total download size: 36 M 2024-03-29T07:53:19.856 INFO:teuthology.orchestra.run.smithi037.stdout:Installed size: 210 M 2024-03-29T07:53:19.856 INFO:teuthology.orchestra.run.smithi037.stdout:Downloading Packages: 2024-03-29T07:53:20.497 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: openblas-threads-0.3.15-4.el8.x86_64 4/8 2024-03-29T07:53:20.523 INFO:teuthology.orchestra.run.smithi037.stdout:(1/8): openblas-0.3.15-4.el8.x86_64.rpm 12 MB/s | 4.7 MB 00:00 2024-03-29T07:53:20.598 INFO:teuthology.orchestra.run.smithi037.stdout:(2/8): openblas-threads-0.3.15-4.el8.x86_64.rpm 10 MB/s | 4.9 MB 00:00 2024-03-29T07:53:20.832 INFO:teuthology.orchestra.run.smithi037.stdout:(3/8): python3-numpy-f2py-1.14.3-10.el8.x86_64. 966 kB/s | 225 kB 00:00 2024-03-29T07:53:20.861 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 5/8 2024-03-29T07:53:20.957 INFO:teuthology.orchestra.run.smithi037.stdout:(4/8): ceph-mgr-diskprediction-local-17.2.7-789 9.0 MB/s | 7.4 MB 00:00 2024-03-29T07:53:21.042 INFO:teuthology.orchestra.run.smithi037.stdout:(5/8): python3-numpy-1.14.3-10.el8.x86_64.rpm 7.2 MB/s | 3.7 MB 00:00 2024-03-29T07:53:21.095 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 6/8 2024-03-29T07:53:21.499 INFO:teuthology.orchestra.run.smithi037.stdout:(6/8): python3-scipy-1.0.0-21.module_el8+762+77 21 MB/s | 14 MB 00:00 2024-03-29T07:53:21.516 INFO:teuthology.orchestra.run.smithi037.stdout:(7/8): libquadmath-8.5.0-21.el8.x86_64.rpm 363 kB/s | 172 kB 00:00 2024-03-29T07:53:21.575 INFO:teuthology.orchestra.run.smithi037.stdout:(8/8): libgfortran-8.5.0-21.el8.x86_64.rpm 1.0 MB/s | 645 kB 00:00 2024-03-29T07:53:21.575 INFO:teuthology.orchestra.run.smithi037.stdout:-------------------------------------------------------------------------------- 2024-03-29T07:53:21.575 INFO:teuthology.orchestra.run.smithi037.stdout:Total 21 MB/s | 36 MB 00:01 2024-03-29T07:53:21.898 INFO:teuthology.orchestra.run.smithi037.stdout:Running transaction check 2024-03-29T07:53:21.921 INFO:teuthology.orchestra.run.smithi037.stdout:Transaction check succeeded. 2024-03-29T07:53:21.922 INFO:teuthology.orchestra.run.smithi037.stdout:Running transaction test 2024-03-29T07:53:22.383 INFO:teuthology.orchestra.run.smithi037.stdout:Transaction test succeeded. 2024-03-29T07:53:22.387 INFO:teuthology.orchestra.run.smithi037.stdout:Running transaction 2024-03-29T07:53:22.996 INFO:teuthology.orchestra.run.smithi037.stdout: Preparing : 1/1 2024-03-29T07:53:23.148 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libquadmath-8.5.0-21.el8.x86_64 1/8 2024-03-29T07:53:23.168 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 6/8 2024-03-29T07:53:23.309 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: libquadmath-8.5.0-21.el8.x86_64 1/8 2024-03-29T07:53:23.448 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libgfortran-8.5.0-21.el8.x86_64 2/8 2024-03-29T07:53:24.145 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: libgfortran-8.5.0-21.el8.x86_64 2/8 2024-03-29T07:53:24.323 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : openblas-0.3.15-4.el8.x86_64 3/8 2024-03-29T07:53:24.558 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-scipy-1.0.0-21.module_el8+762+77bd8591.x86_6 7/8 2024-03-29T07:53:24.802 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-mgr-diskprediction-local-2:17.2.7-789.g4213e17c 8/8 2024-03-29T07:53:25.041 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: openblas-0.3.15-4.el8.x86_64 3/8 2024-03-29T07:53:25.217 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:17.2.7-789.g4213e17c 8/8 2024-03-29T07:53:25.217 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-mgr-diskprediction-local-2:17.2.7-789.g4213e17c 1/8 2024-03-29T07:53:25.217 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : openblas-0.3.15-4.el8.x86_64 2/8 2024-03-29T07:53:25.217 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : openblas-threads-0.3.15-4.el8.x86_64 3/8 2024-03-29T07:53:25.217 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 4/8 2024-03-29T07:53:25.217 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 5/8 2024-03-29T07:53:25.217 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-scipy-1.0.0-21.module_el8+762+77bd8591.x86_6 6/8 2024-03-29T07:53:25.217 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libgfortran-8.5.0-21.el8.x86_64 7/8 2024-03-29T07:53:25.512 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : openblas-threads-0.3.15-4.el8.x86_64 4/8 2024-03-29T07:53:25.685 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libquadmath-8.5.0-21.el8.x86_64 8/8 2024-03-29T07:53:25.685 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T07:53:25.685 INFO:teuthology.orchestra.run.smithi148.stdout:Installed: 2024-03-29T07:53:25.685 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-diskprediction-local-2:17.2.7-789.g4213e17c.el8.noarch 2024-03-29T07:53:25.686 INFO:teuthology.orchestra.run.smithi148.stdout: libgfortran-8.5.0-21.el8.x86_64 2024-03-29T07:53:25.686 INFO:teuthology.orchestra.run.smithi148.stdout: libquadmath-8.5.0-21.el8.x86_64 2024-03-29T07:53:25.686 INFO:teuthology.orchestra.run.smithi148.stdout: openblas-0.3.15-4.el8.x86_64 2024-03-29T07:53:25.686 INFO:teuthology.orchestra.run.smithi148.stdout: openblas-threads-0.3.15-4.el8.x86_64 2024-03-29T07:53:25.686 INFO:teuthology.orchestra.run.smithi148.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2024-03-29T07:53:25.686 INFO:teuthology.orchestra.run.smithi148.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2024-03-29T07:53:25.686 INFO:teuthology.orchestra.run.smithi148.stdout: python3-scipy-1.0.0-21.module_el8+762+77bd8591.x86_64 2024-03-29T07:53:25.686 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T07:53:25.687 INFO:teuthology.orchestra.run.smithi148.stdout:Complete! 2024-03-29T07:53:25.877 DEBUG:teuthology.orchestra.run.smithi148:> sudo yum -y install ceph-mgr-rook 2024-03-29T07:53:26.379 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: openblas-threads-0.3.15-4.el8.x86_64 4/8 2024-03-29T07:53:26.720 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 5/8 2024-03-29T07:53:26.797 INFO:teuthology.orchestra.run.smithi148.stdout:Last metadata expiration check: 0:02:00 ago on Fri 29 Mar 2024 07:51:26 AM UTC. 2024-03-29T07:53:26.989 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 6/8 2024-03-29T07:53:28.533 INFO:teuthology.orchestra.run.smithi148.stdout:Dependencies resolved. 2024-03-29T07:53:28.535 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-03-29T07:53:28.535 INFO:teuthology.orchestra.run.smithi148.stdout: Package Arch Version Repository Size 2024-03-29T07:53:28.535 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-03-29T07:53:28.535 INFO:teuthology.orchestra.run.smithi148.stdout:Installing: 2024-03-29T07:53:28.535 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-rook noarch 2:17.2.7-789.g4213e17c.el8 ceph-noarch 47 k 2024-03-29T07:53:28.535 INFO:teuthology.orchestra.run.smithi148.stdout:Installing dependencies: 2024-03-29T07:53:28.535 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2024-03-29T07:53:28.535 INFO:teuthology.orchestra.run.smithi148.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2024-03-29T07:53:28.535 INFO:teuthology.orchestra.run.smithi148.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2024-03-29T07:53:28.535 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jsonpatch noarch 1.21-2.el8 appstream 27 k 2024-03-29T07:53:28.535 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jsonpointer noarch 1.10-11.el8 appstream 20 k 2024-03-29T07:53:28.536 INFO:teuthology.orchestra.run.smithi148.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2024-03-29T07:53:28.536 INFO:teuthology.orchestra.run.smithi148.stdout: python3-oauthlib noarch 2.1.0-1.el8 baseos 155 k 2024-03-29T07:53:28.536 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pyasn1 noarch 0.3.7-6.el8 appstream 126 k 2024-03-29T07:53:28.536 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 appstream 110 k 2024-03-29T07:53:28.536 INFO:teuthology.orchestra.run.smithi148.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 baseos 43 k 2024-03-29T07:53:28.536 INFO:teuthology.orchestra.run.smithi148.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2024-03-29T07:53:28.536 INFO:teuthology.orchestra.run.smithi148.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2024-03-29T07:53:28.536 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T07:53:28.536 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction Summary 2024-03-29T07:53:28.536 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-03-29T07:53:28.536 INFO:teuthology.orchestra.run.smithi148.stdout:Install 13 Packages 2024-03-29T07:53:28.536 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T07:53:28.537 INFO:teuthology.orchestra.run.smithi148.stdout:Total download size: 1.9 M 2024-03-29T07:53:28.537 INFO:teuthology.orchestra.run.smithi148.stdout:Installed size: 25 M 2024-03-29T07:53:28.537 INFO:teuthology.orchestra.run.smithi148.stdout:Downloading Packages: 2024-03-29T07:53:29.141 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 6/8 2024-03-29T07:53:29.190 INFO:teuthology.orchestra.run.smithi148.stdout:(1/13): python3-jsonpointer-1.10-11.el8.noarch. 134 kB/s | 20 kB 00:00 2024-03-29T07:53:29.214 INFO:teuthology.orchestra.run.smithi148.stdout:(2/13): python3-jsonpatch-1.21-2.el8.noarch.rpm 153 kB/s | 27 kB 00:00 2024-03-29T07:53:29.231 INFO:teuthology.orchestra.run.smithi148.stdout:(3/13): ceph-mgr-rook-17.2.7-789.g4213e17c.el8. 249 kB/s | 47 kB 00:00 2024-03-29T07:53:29.315 INFO:teuthology.orchestra.run.smithi148.stdout:(4/13): python3-pyasn1-0.3.7-6.el8.noarch.rpm 1.0 MB/s | 126 kB 00:00 2024-03-29T07:53:29.340 INFO:teuthology.orchestra.run.smithi148.stdout:(5/13): python3-pyasn1-modules-0.3.7-6.el8.noar 878 kB/s | 110 kB 00:00 2024-03-29T07:53:29.549 INFO:teuthology.orchestra.run.smithi148.stdout:(6/13): python3-requests-oauthlib-1.0.0-1.el8.n 183 kB/s | 43 kB 00:00 2024-03-29T07:53:29.574 INFO:teuthology.orchestra.run.smithi148.stdout:(7/13): python3-oauthlib-2.1.0-1.el8.noarch.rpm 452 kB/s | 155 kB 00:00 2024-03-29T07:53:29.608 INFO:teuthology.orchestra.run.smithi148.stdout:(8/13): python3-cachetools-3.1.1-4.el8.noarch.r 123 kB/s | 33 kB 00:00 2024-03-29T07:53:29.817 INFO:teuthology.orchestra.run.smithi148.stdout:(9/13): python3-certifi-2018.10.15-7.el8.noarch 61 kB/s | 16 kB 00:00 2024-03-29T07:53:30.017 INFO:teuthology.orchestra.run.smithi148.stdout:(10/13): python3-google-auth-1.1.1-10.el8.noarc 184 kB/s | 81 kB 00:00 2024-03-29T07:53:30.176 INFO:teuthology.orchestra.run.smithi148.stdout:(11/13): python3-rsa-4.9-2.el8.noarch.rpm 179 kB/s | 64 kB 00:00 2024-03-29T07:53:30.368 INFO:teuthology.orchestra.run.smithi148.stdout:(12/13): python3-kubernetes-11.0.0-6.el8.noarch 1.5 MB/s | 1.2 MB 00:00 2024-03-29T07:53:30.393 INFO:teuthology.orchestra.run.smithi148.stdout:(13/13): python3-websocket-client-0.56.0-5.el8. 163 kB/s | 61 kB 00:00 2024-03-29T07:53:30.394 INFO:teuthology.orchestra.run.smithi148.stdout:-------------------------------------------------------------------------------- 2024-03-29T07:53:30.394 INFO:teuthology.orchestra.run.smithi148.stdout:Total 1.0 MB/s | 1.9 MB 00:01 2024-03-29T07:53:30.513 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction check 2024-03-29T07:53:30.532 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction check succeeded. 2024-03-29T07:53:30.532 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction test 2024-03-29T07:53:30.550 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-scipy-1.0.0-21.module_el8+762+77bd8591.x86_6 7/8 2024-03-29T07:53:30.643 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction test succeeded. 2024-03-29T07:53:30.647 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction 2024-03-29T07:53:30.730 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-mgr-diskprediction-local-2:17.2.7-789.g4213e17c 8/8 2024-03-29T07:53:31.149 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:17.2.7-789.g4213e17c 8/8 2024-03-29T07:53:31.149 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-mgr-diskprediction-local-2:17.2.7-789.g4213e17c 1/8 2024-03-29T07:53:31.150 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : openblas-0.3.15-4.el8.x86_64 2/8 2024-03-29T07:53:31.150 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : openblas-threads-0.3.15-4.el8.x86_64 3/8 2024-03-29T07:53:31.150 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 4/8 2024-03-29T07:53:31.150 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 5/8 2024-03-29T07:53:31.150 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-scipy-1.0.0-21.module_el8+762+77bd8591.x86_6 6/8 2024-03-29T07:53:31.150 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libgfortran-8.5.0-21.el8.x86_64 7/8 2024-03-29T07:53:31.255 INFO:teuthology.orchestra.run.smithi148.stdout: Preparing : 1/1 2024-03-29T07:53:31.478 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 1/13 2024-03-29T07:53:31.564 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libquadmath-8.5.0-21.el8.x86_64 8/8 2024-03-29T07:53:31.564 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T07:53:31.564 INFO:teuthology.orchestra.run.smithi037.stdout:Installed: 2024-03-29T07:53:31.564 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mgr-diskprediction-local-2:17.2.7-789.g4213e17c.el8.noarch 2024-03-29T07:53:31.564 INFO:teuthology.orchestra.run.smithi037.stdout: libgfortran-8.5.0-21.el8.x86_64 2024-03-29T07:53:31.565 INFO:teuthology.orchestra.run.smithi037.stdout: libquadmath-8.5.0-21.el8.x86_64 2024-03-29T07:53:31.565 INFO:teuthology.orchestra.run.smithi037.stdout: openblas-0.3.15-4.el8.x86_64 2024-03-29T07:53:31.565 INFO:teuthology.orchestra.run.smithi037.stdout: openblas-threads-0.3.15-4.el8.x86_64 2024-03-29T07:53:31.565 INFO:teuthology.orchestra.run.smithi037.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2024-03-29T07:53:31.565 INFO:teuthology.orchestra.run.smithi037.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2024-03-29T07:53:31.565 INFO:teuthology.orchestra.run.smithi037.stdout: python3-scipy-1.0.0-21.module_el8+762+77bd8591.x86_64 2024-03-29T07:53:31.565 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T07:53:31.566 INFO:teuthology.orchestra.run.smithi037.stdout:Complete! 2024-03-29T07:53:31.698 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 2/13 2024-03-29T07:53:31.729 DEBUG:teuthology.orchestra.run.smithi037:> sudo yum -y install ceph-mgr-rook 2024-03-29T07:53:31.982 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-rsa-4.9-2.el8.noarch 3/13 2024-03-29T07:53:32.226 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 4/13 2024-03-29T07:53:32.451 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 5/13 2024-03-29T07:53:32.675 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 6/13 2024-03-29T07:53:32.695 INFO:teuthology.orchestra.run.smithi037.stdout:Last metadata expiration check: 0:02:06 ago on Fri 29 Mar 2024 07:51:26 AM UTC. 2024-03-29T07:53:32.987 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 7/13 2024-03-29T07:53:33.244 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 8/13 2024-03-29T07:53:34.503 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 9/13 2024-03-29T07:53:34.708 INFO:teuthology.orchestra.run.smithi037.stdout:Dependencies resolved. 2024-03-29T07:53:34.710 INFO:teuthology.orchestra.run.smithi037.stdout:================================================================================ 2024-03-29T07:53:34.711 INFO:teuthology.orchestra.run.smithi037.stdout: Package Arch Version Repository Size 2024-03-29T07:53:34.711 INFO:teuthology.orchestra.run.smithi037.stdout:================================================================================ 2024-03-29T07:53:34.711 INFO:teuthology.orchestra.run.smithi037.stdout:Installing: 2024-03-29T07:53:34.711 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mgr-rook noarch 2:17.2.7-789.g4213e17c.el8 ceph-noarch 47 k 2024-03-29T07:53:34.711 INFO:teuthology.orchestra.run.smithi037.stdout:Installing dependencies: 2024-03-29T07:53:34.712 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2024-03-29T07:53:34.712 INFO:teuthology.orchestra.run.smithi037.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2024-03-29T07:53:34.712 INFO:teuthology.orchestra.run.smithi037.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2024-03-29T07:53:34.712 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jsonpatch noarch 1.21-2.el8 appstream 27 k 2024-03-29T07:53:34.712 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jsonpointer noarch 1.10-11.el8 appstream 20 k 2024-03-29T07:53:34.712 INFO:teuthology.orchestra.run.smithi037.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2024-03-29T07:53:34.713 INFO:teuthology.orchestra.run.smithi037.stdout: python3-oauthlib noarch 2.1.0-1.el8 baseos 155 k 2024-03-29T07:53:34.713 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pyasn1 noarch 0.3.7-6.el8 appstream 126 k 2024-03-29T07:53:34.713 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 appstream 110 k 2024-03-29T07:53:34.713 INFO:teuthology.orchestra.run.smithi037.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 baseos 43 k 2024-03-29T07:53:34.713 INFO:teuthology.orchestra.run.smithi037.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2024-03-29T07:53:34.713 INFO:teuthology.orchestra.run.smithi037.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2024-03-29T07:53:34.713 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T07:53:34.714 INFO:teuthology.orchestra.run.smithi037.stdout:Transaction Summary 2024-03-29T07:53:34.714 INFO:teuthology.orchestra.run.smithi037.stdout:================================================================================ 2024-03-29T07:53:34.714 INFO:teuthology.orchestra.run.smithi037.stdout:Install 13 Packages 2024-03-29T07:53:34.714 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T07:53:34.715 INFO:teuthology.orchestra.run.smithi037.stdout:Total download size: 1.9 M 2024-03-29T07:53:34.715 INFO:teuthology.orchestra.run.smithi037.stdout:Installed size: 25 M 2024-03-29T07:53:34.715 INFO:teuthology.orchestra.run.smithi037.stdout:Downloading Packages: 2024-03-29T07:53:34.817 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 10/13 2024-03-29T07:53:34.989 INFO:teuthology.orchestra.run.smithi037.stdout:(1/13): python3-jsonpatch-1.21-2.el8.noarch.rpm 213 kB/s | 27 kB 00:00 2024-03-29T07:53:35.001 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 11/13 2024-03-29T07:53:35.014 INFO:teuthology.orchestra.run.smithi037.stdout:(2/13): python3-jsonpointer-1.10-11.el8.noarch. 133 kB/s | 20 kB 00:00 2024-03-29T07:53:35.122 INFO:teuthology.orchestra.run.smithi037.stdout:(3/13): python3-pyasn1-modules-0.3.7-6.el8.noar 1.0 MB/s | 110 kB 00:00 2024-03-29T07:53:35.147 INFO:teuthology.orchestra.run.smithi037.stdout:(4/13): python3-pyasn1-0.3.7-6.el8.noarch.rpm 951 kB/s | 126 kB 00:00 2024-03-29T07:53:35.231 INFO:teuthology.orchestra.run.smithi037.stdout:(5/13): ceph-mgr-rook-17.2.7-789.g4213e17c.el8. 129 kB/s | 47 kB 00:00 2024-03-29T07:53:35.255 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 12/13 2024-03-29T07:53:35.373 INFO:teuthology.orchestra.run.smithi037.stdout:(6/13): python3-requests-oauthlib-1.0.0-1.el8.n 189 kB/s | 43 kB 00:00 2024-03-29T07:53:35.473 INFO:teuthology.orchestra.run.smithi037.stdout:(7/13): python3-oauthlib-2.1.0-1.el8.noarch.rpm 442 kB/s | 155 kB 00:00 2024-03-29T07:53:35.492 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-mgr-rook-2:17.2.7-789.g4213e17c.el8.noarch 13/13 2024-03-29T07:53:35.499 INFO:teuthology.orchestra.run.smithi037.stdout:(8/13): python3-cachetools-3.1.1-4.el8.noarch.r 123 kB/s | 33 kB 00:00 2024-03-29T07:53:35.658 INFO:teuthology.orchestra.run.smithi037.stdout:(9/13): python3-certifi-2018.10.15-7.el8.noarch 57 kB/s | 16 kB 00:00 2024-03-29T07:53:35.816 INFO:teuthology.orchestra.run.smithi037.stdout:(10/13): python3-google-auth-1.1.1-10.el8.noarc 237 kB/s | 81 kB 00:00 2024-03-29T07:53:35.849 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-mgr-rook-2:17.2.7-789.g4213e17c.el8.noarch 13/13 2024-03-29T07:53:35.849 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-mgr-rook-2:17.2.7-789.g4213e17c.el8.noarch 1/13 2024-03-29T07:53:35.849 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 2/13 2024-03-29T07:53:35.849 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 3/13 2024-03-29T07:53:35.849 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 4/13 2024-03-29T07:53:35.850 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 5/13 2024-03-29T07:53:35.850 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 6/13 2024-03-29T07:53:35.850 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 7/13 2024-03-29T07:53:35.850 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 8/13 2024-03-29T07:53:35.850 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 9/13 2024-03-29T07:53:35.850 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 10/13 2024-03-29T07:53:35.850 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 11/13 2024-03-29T07:53:35.850 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 12/13 2024-03-29T07:53:36.000 INFO:teuthology.orchestra.run.smithi037.stdout:(11/13): python3-rsa-4.9-2.el8.noarch.rpm 188 kB/s | 64 kB 00:00 2024-03-29T07:53:36.158 INFO:teuthology.orchestra.run.smithi037.stdout:(12/13): python3-websocket-client-0.56.0-5.el8. 179 kB/s | 61 kB 00:00 2024-03-29T07:53:36.192 INFO:teuthology.orchestra.run.smithi037.stdout:(13/13): python3-kubernetes-11.0.0-6.el8.noarch 1.7 MB/s | 1.2 MB 00:00 2024-03-29T07:53:36.193 INFO:teuthology.orchestra.run.smithi037.stdout:-------------------------------------------------------------------------------- 2024-03-29T07:53:36.193 INFO:teuthology.orchestra.run.smithi037.stdout:Total 1.3 MB/s | 1.9 MB 00:01 2024-03-29T07:53:36.323 INFO:teuthology.orchestra.run.smithi037.stdout:Running transaction check 2024-03-29T07:53:36.335 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 13/13 2024-03-29T07:53:36.335 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T07:53:36.335 INFO:teuthology.orchestra.run.smithi148.stdout:Installed: 2024-03-29T07:53:36.335 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-rook-2:17.2.7-789.g4213e17c.el8.noarch 2024-03-29T07:53:36.335 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cachetools-3.1.1-4.el8.noarch 2024-03-29T07:53:36.335 INFO:teuthology.orchestra.run.smithi148.stdout: python3-certifi-2018.10.15-7.el8.noarch 2024-03-29T07:53:36.335 INFO:teuthology.orchestra.run.smithi148.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2024-03-29T07:53:36.335 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jsonpatch-1.21-2.el8.noarch 2024-03-29T07:53:36.335 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jsonpointer-1.10-11.el8.noarch 2024-03-29T07:53:36.335 INFO:teuthology.orchestra.run.smithi148.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2024-03-29T07:53:36.335 INFO:teuthology.orchestra.run.smithi148.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2024-03-29T07:53:36.335 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2024-03-29T07:53:36.336 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2024-03-29T07:53:36.336 INFO:teuthology.orchestra.run.smithi148.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2024-03-29T07:53:36.336 INFO:teuthology.orchestra.run.smithi148.stdout: python3-rsa-4.9-2.el8.noarch 2024-03-29T07:53:36.336 INFO:teuthology.orchestra.run.smithi148.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2024-03-29T07:53:36.336 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T07:53:36.336 INFO:teuthology.orchestra.run.smithi148.stdout:Complete! 2024-03-29T07:53:36.343 INFO:teuthology.orchestra.run.smithi037.stdout:Transaction check succeeded. 2024-03-29T07:53:36.343 INFO:teuthology.orchestra.run.smithi037.stdout:Running transaction test 2024-03-29T07:53:36.460 INFO:teuthology.orchestra.run.smithi037.stdout:Transaction test succeeded. 2024-03-29T07:53:36.463 INFO:teuthology.orchestra.run.smithi037.stdout:Running transaction 2024-03-29T07:53:36.488 DEBUG:teuthology.orchestra.run.smithi148:> sudo yum -y install ceph-mgr-cephadm 2024-03-29T07:53:37.062 INFO:teuthology.orchestra.run.smithi037.stdout: Preparing : 1/1 2024-03-29T07:53:37.302 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 1/13 2024-03-29T07:53:37.406 INFO:teuthology.orchestra.run.smithi148.stdout:Last metadata expiration check: 0:02:11 ago on Fri 29 Mar 2024 07:51:26 AM UTC. 2024-03-29T07:53:37.505 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 2/13 2024-03-29T07:53:37.722 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-rsa-4.9-2.el8.noarch 3/13 2024-03-29T07:53:37.980 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 4/13 2024-03-29T07:53:38.191 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 5/13 2024-03-29T07:53:38.433 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 6/13 2024-03-29T07:53:38.726 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 7/13 2024-03-29T07:53:38.992 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 8/13 2024-03-29T07:53:39.292 INFO:teuthology.orchestra.run.smithi148.stdout:Dependencies resolved. 2024-03-29T07:53:39.293 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-03-29T07:53:39.293 INFO:teuthology.orchestra.run.smithi148.stdout: Package Arch Version Repository Size 2024-03-29T07:53:39.293 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-03-29T07:53:39.293 INFO:teuthology.orchestra.run.smithi148.stdout:Installing: 2024-03-29T07:53:39.293 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-cephadm noarch 2:17.2.7-789.g4213e17c.el8 ceph-noarch 123 k 2024-03-29T07:53:39.293 INFO:teuthology.orchestra.run.smithi148.stdout:Installing dependencies: 2024-03-29T07:53:39.293 INFO:teuthology.orchestra.run.smithi148.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2024-03-29T07:53:39.294 INFO:teuthology.orchestra.run.smithi148.stdout: python3-babel noarch 2.5.1-7.el8 appstream 4.8 M 2024-03-29T07:53:39.294 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jinja2 noarch 2.10.1-4.el8 appstream 578 k 2024-03-29T07:53:39.294 INFO:teuthology.orchestra.run.smithi148.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2024-03-29T07:53:39.294 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T07:53:39.294 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction Summary 2024-03-29T07:53:39.294 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-03-29T07:53:39.294 INFO:teuthology.orchestra.run.smithi148.stdout:Install 5 Packages 2024-03-29T07:53:39.294 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T07:53:39.295 INFO:teuthology.orchestra.run.smithi148.stdout:Total download size: 6.0 M 2024-03-29T07:53:39.295 INFO:teuthology.orchestra.run.smithi148.stdout:Installed size: 26 M 2024-03-29T07:53:39.295 INFO:teuthology.orchestra.run.smithi148.stdout:Downloading Packages: 2024-03-29T07:53:40.010 INFO:teuthology.orchestra.run.smithi148.stdout:(1/5): python3-jinja2-2.10.1-4.el8.noarch.rpm 2.2 MB/s | 578 kB 00:00 2024-03-29T07:53:40.093 INFO:teuthology.orchestra.run.smithi148.stdout:(2/5): ceph-mgr-cephadm-17.2.7-789.g4213e17c.el 368 kB/s | 123 kB 00:00 2024-03-29T07:53:40.168 INFO:teuthology.orchestra.run.smithi148.stdout:(3/5): python3-babel-2.5.1-7.el8.noarch.rpm 12 MB/s | 4.8 MB 00:00 2024-03-29T07:53:40.310 INFO:teuthology.orchestra.run.smithi148.stdout:(4/5): python3-natsort-7.1.1-2.el8.noarch.rpm 278 kB/s | 60 kB 00:00 2024-03-29T07:53:40.364 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 9/13 2024-03-29T07:53:40.368 INFO:teuthology.orchestra.run.smithi148.stdout:(5/5): python3-asyncssh-2.7.0-2.el8.noarch.rpm 1.3 MB/s | 476 kB 00:00 2024-03-29T07:53:40.369 INFO:teuthology.orchestra.run.smithi148.stdout:-------------------------------------------------------------------------------- 2024-03-29T07:53:40.370 INFO:teuthology.orchestra.run.smithi148.stdout:Total 5.6 MB/s | 6.0 MB 00:01 2024-03-29T07:53:40.444 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction check 2024-03-29T07:53:40.458 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction check succeeded. 2024-03-29T07:53:40.458 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction test 2024-03-29T07:53:40.584 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction test succeeded. 2024-03-29T07:53:40.587 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction 2024-03-29T07:53:40.638 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 10/13 2024-03-29T07:53:40.830 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 11/13 2024-03-29T07:53:40.906 INFO:teuthology.orchestra.run.smithi148.stdout: Preparing : 1/1 2024-03-29T07:53:41.052 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 12/13 2024-03-29T07:53:41.270 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-mgr-rook-2:17.2.7-789.g4213e17c.el8.noarch 13/13 2024-03-29T07:53:41.642 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-mgr-rook-2:17.2.7-789.g4213e17c.el8.noarch 13/13 2024-03-29T07:53:41.642 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-mgr-rook-2:17.2.7-789.g4213e17c.el8.noarch 1/13 2024-03-29T07:53:41.642 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 2/13 2024-03-29T07:53:41.642 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 3/13 2024-03-29T07:53:41.642 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 4/13 2024-03-29T07:53:41.642 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 5/13 2024-03-29T07:53:41.642 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 6/13 2024-03-29T07:53:41.642 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 7/13 2024-03-29T07:53:41.643 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 8/13 2024-03-29T07:53:41.643 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 9/13 2024-03-29T07:53:41.643 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 10/13 2024-03-29T07:53:41.643 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 11/13 2024-03-29T07:53:41.643 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 12/13 2024-03-29T07:53:41.697 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 1/5 2024-03-29T07:53:42.155 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 13/13 2024-03-29T07:53:42.155 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T07:53:42.155 INFO:teuthology.orchestra.run.smithi037.stdout:Installed: 2024-03-29T07:53:42.155 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mgr-rook-2:17.2.7-789.g4213e17c.el8.noarch 2024-03-29T07:53:42.155 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cachetools-3.1.1-4.el8.noarch 2024-03-29T07:53:42.156 INFO:teuthology.orchestra.run.smithi037.stdout: python3-certifi-2018.10.15-7.el8.noarch 2024-03-29T07:53:42.156 INFO:teuthology.orchestra.run.smithi037.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2024-03-29T07:53:42.156 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jsonpatch-1.21-2.el8.noarch 2024-03-29T07:53:42.156 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jsonpointer-1.10-11.el8.noarch 2024-03-29T07:53:42.156 INFO:teuthology.orchestra.run.smithi037.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2024-03-29T07:53:42.156 INFO:teuthology.orchestra.run.smithi037.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2024-03-29T07:53:42.156 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2024-03-29T07:53:42.156 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2024-03-29T07:53:42.156 INFO:teuthology.orchestra.run.smithi037.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2024-03-29T07:53:42.156 INFO:teuthology.orchestra.run.smithi037.stdout: python3-rsa-4.9-2.el8.noarch 2024-03-29T07:53:42.156 INFO:teuthology.orchestra.run.smithi037.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2024-03-29T07:53:42.156 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T07:53:42.156 INFO:teuthology.orchestra.run.smithi037.stdout:Complete! 2024-03-29T07:53:42.354 DEBUG:teuthology.orchestra.run.smithi037:> sudo yum -y install ceph-mgr-cephadm 2024-03-29T07:53:42.775 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 2/5 2024-03-29T07:53:43.206 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 3/5 2024-03-29T07:53:43.317 INFO:teuthology.orchestra.run.smithi037.stdout:Last metadata expiration check: 0:02:17 ago on Fri 29 Mar 2024 07:51:26 AM UTC. 2024-03-29T07:53:43.516 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-jinja2-2.10.1-4.el8.noarch 4/5 2024-03-29T07:53:43.758 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-mgr-cephadm-2:17.2.7-789.g4213e17c.el8.noarch 5/5 2024-03-29T07:53:44.107 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-mgr-cephadm-2:17.2.7-789.g4213e17c.el8.noarch 5/5 2024-03-29T07:53:44.107 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-mgr-cephadm-2:17.2.7-789.g4213e17c.el8.noarch 1/5 2024-03-29T07:53:44.107 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 2/5 2024-03-29T07:53:44.107 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-jinja2-2.10.1-4.el8.noarch 3/5 2024-03-29T07:53:44.107 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 4/5 2024-03-29T07:53:44.539 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 5/5 2024-03-29T07:53:44.539 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T07:53:44.540 INFO:teuthology.orchestra.run.smithi148.stdout:Installed: 2024-03-29T07:53:44.540 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-cephadm-2:17.2.7-789.g4213e17c.el8.noarch 2024-03-29T07:53:44.540 INFO:teuthology.orchestra.run.smithi148.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2024-03-29T07:53:44.540 INFO:teuthology.orchestra.run.smithi148.stdout: python3-babel-2.5.1-7.el8.noarch 2024-03-29T07:53:44.540 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jinja2-2.10.1-4.el8.noarch 2024-03-29T07:53:44.540 INFO:teuthology.orchestra.run.smithi148.stdout: python3-natsort-7.1.1-2.el8.noarch 2024-03-29T07:53:44.540 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T07:53:44.540 INFO:teuthology.orchestra.run.smithi148.stdout:Complete! 2024-03-29T07:53:44.702 DEBUG:teuthology.orchestra.run.smithi148:> sudo yum -y install ceph-fuse 2024-03-29T07:53:45.142 INFO:teuthology.orchestra.run.smithi037.stdout:Dependencies resolved. 2024-03-29T07:53:45.142 INFO:teuthology.orchestra.run.smithi037.stdout:================================================================================ 2024-03-29T07:53:45.142 INFO:teuthology.orchestra.run.smithi037.stdout: Package Arch Version Repository Size 2024-03-29T07:53:45.142 INFO:teuthology.orchestra.run.smithi037.stdout:================================================================================ 2024-03-29T07:53:45.142 INFO:teuthology.orchestra.run.smithi037.stdout:Installing: 2024-03-29T07:53:45.142 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mgr-cephadm noarch 2:17.2.7-789.g4213e17c.el8 ceph-noarch 123 k 2024-03-29T07:53:45.142 INFO:teuthology.orchestra.run.smithi037.stdout:Installing dependencies: 2024-03-29T07:53:45.142 INFO:teuthology.orchestra.run.smithi037.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2024-03-29T07:53:45.143 INFO:teuthology.orchestra.run.smithi037.stdout: python3-babel noarch 2.5.1-7.el8 appstream 4.8 M 2024-03-29T07:53:45.143 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jinja2 noarch 2.10.1-4.el8 appstream 578 k 2024-03-29T07:53:45.143 INFO:teuthology.orchestra.run.smithi037.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2024-03-29T07:53:45.143 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T07:53:45.143 INFO:teuthology.orchestra.run.smithi037.stdout:Transaction Summary 2024-03-29T07:53:45.143 INFO:teuthology.orchestra.run.smithi037.stdout:================================================================================ 2024-03-29T07:53:45.143 INFO:teuthology.orchestra.run.smithi037.stdout:Install 5 Packages 2024-03-29T07:53:45.143 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T07:53:45.144 INFO:teuthology.orchestra.run.smithi037.stdout:Total download size: 6.0 M 2024-03-29T07:53:45.144 INFO:teuthology.orchestra.run.smithi037.stdout:Installed size: 26 M 2024-03-29T07:53:45.144 INFO:teuthology.orchestra.run.smithi037.stdout:Downloading Packages: 2024-03-29T07:53:45.623 INFO:teuthology.orchestra.run.smithi148.stdout:Last metadata expiration check: 0:02:19 ago on Fri 29 Mar 2024 07:51:26 AM UTC. 2024-03-29T07:53:45.714 INFO:teuthology.orchestra.run.smithi037.stdout:(1/5): python3-jinja2-2.10.1-4.el8.noarch.rpm 1.2 MB/s | 578 kB 00:00 2024-03-29T07:53:45.743 INFO:teuthology.orchestra.run.smithi037.stdout:(2/5): ceph-mgr-cephadm-17.2.7-789.g4213e17c.el 252 kB/s | 123 kB 00:00 2024-03-29T07:53:45.876 INFO:teuthology.orchestra.run.smithi037.stdout:(3/5): python3-babel-2.5.1-7.el8.noarch.rpm 7.7 MB/s | 4.8 MB 00:00 2024-03-29T07:53:45.902 INFO:teuthology.orchestra.run.smithi037.stdout:(4/5): python3-natsort-7.1.1-2.el8.noarch.rpm 380 kB/s | 60 kB 00:00 2024-03-29T07:53:46.027 INFO:teuthology.orchestra.run.smithi037.stdout:(5/5): python3-asyncssh-2.7.0-2.el8.noarch.rpm 1.5 MB/s | 476 kB 00:00 2024-03-29T07:53:46.027 INFO:teuthology.orchestra.run.smithi037.stdout:-------------------------------------------------------------------------------- 2024-03-29T07:53:46.027 INFO:teuthology.orchestra.run.smithi037.stdout:Total 6.8 MB/s | 6.0 MB 00:00 2024-03-29T07:53:46.104 INFO:teuthology.orchestra.run.smithi037.stdout:Running transaction check 2024-03-29T07:53:46.119 INFO:teuthology.orchestra.run.smithi037.stdout:Transaction check succeeded. 2024-03-29T07:53:46.120 INFO:teuthology.orchestra.run.smithi037.stdout:Running transaction test 2024-03-29T07:53:46.246 INFO:teuthology.orchestra.run.smithi037.stdout:Transaction test succeeded. 2024-03-29T07:53:46.250 INFO:teuthology.orchestra.run.smithi037.stdout:Running transaction 2024-03-29T07:53:46.552 INFO:teuthology.orchestra.run.smithi037.stdout: Preparing : 1/1 2024-03-29T07:53:46.940 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 1/5 2024-03-29T07:53:47.307 INFO:teuthology.orchestra.run.smithi148.stdout:Dependencies resolved. 2024-03-29T07:53:47.308 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-03-29T07:53:47.308 INFO:teuthology.orchestra.run.smithi148.stdout: Package Arch Version Repository Size 2024-03-29T07:53:47.308 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-03-29T07:53:47.308 INFO:teuthology.orchestra.run.smithi148.stdout:Installing: 2024-03-29T07:53:47.309 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-fuse x86_64 2:17.2.7-789.g4213e17c.el8 ceph 876 k 2024-03-29T07:53:47.309 INFO:teuthology.orchestra.run.smithi148.stdout:Installing dependencies: 2024-03-29T07:53:47.309 INFO:teuthology.orchestra.run.smithi148.stdout: fuse x86_64 2.9.7-19.el8 baseos 83 k 2024-03-29T07:53:47.309 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T07:53:47.309 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction Summary 2024-03-29T07:53:47.309 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-03-29T07:53:47.309 INFO:teuthology.orchestra.run.smithi148.stdout:Install 2 Packages 2024-03-29T07:53:47.309 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T07:53:47.310 INFO:teuthology.orchestra.run.smithi148.stdout:Total download size: 959 k 2024-03-29T07:53:47.310 INFO:teuthology.orchestra.run.smithi148.stdout:Installed size: 3.1 M 2024-03-29T07:53:47.310 INFO:teuthology.orchestra.run.smithi148.stdout:Downloading Packages: 2024-03-29T07:53:47.661 INFO:teuthology.orchestra.run.smithi148.stdout:(1/2): fuse-2.9.7-19.el8.x86_64.rpm 286 kB/s | 83 kB 00:00 2024-03-29T07:53:47.811 INFO:teuthology.orchestra.run.smithi148.stdout:(2/2): ceph-fuse-17.2.7-789.g4213e17c.el8.x86_6 1.9 MB/s | 876 kB 00:00 2024-03-29T07:53:47.811 INFO:teuthology.orchestra.run.smithi148.stdout:-------------------------------------------------------------------------------- 2024-03-29T07:53:47.811 INFO:teuthology.orchestra.run.smithi148.stdout:Total 1.9 MB/s | 959 kB 00:00 2024-03-29T07:53:47.832 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction check 2024-03-29T07:53:47.857 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction check succeeded. 2024-03-29T07:53:47.858 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction test 2024-03-29T07:53:47.968 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction test succeeded. 2024-03-29T07:53:47.971 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction 2024-03-29T07:53:48.055 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 2/5 2024-03-29T07:53:48.170 INFO:teuthology.orchestra.run.smithi148.stdout: Preparing : 1/1 2024-03-29T07:53:48.445 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 3/5 2024-03-29T07:53:48.466 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : fuse-2.9.7-19.el8.x86_64 1/2 2024-03-29T07:53:48.665 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-fuse-2:17.2.7-789.g4213e17c.el8.x86_64 2/2 2024-03-29T07:53:48.740 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-jinja2-2.10.1-4.el8.noarch 4/5 2024-03-29T07:53:48.970 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-mgr-cephadm-2:17.2.7-789.g4213e17c.el8.noarch 5/5 2024-03-29T07:53:49.262 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-fuse-2:17.2.7-789.g4213e17c.el8.x86_64 2/2 2024-03-29T07:53:49.263 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-fuse-2:17.2.7-789.g4213e17c.el8.x86_64 1/2 2024-03-29T07:53:49.330 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-mgr-cephadm-2:17.2.7-789.g4213e17c.el8.noarch 5/5 2024-03-29T07:53:49.330 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-mgr-cephadm-2:17.2.7-789.g4213e17c.el8.noarch 1/5 2024-03-29T07:53:49.330 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 2/5 2024-03-29T07:53:49.330 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-jinja2-2.10.1-4.el8.noarch 3/5 2024-03-29T07:53:49.330 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 4/5 2024-03-29T07:53:49.694 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : fuse-2.9.7-19.el8.x86_64 2/2 2024-03-29T07:53:49.694 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T07:53:49.694 INFO:teuthology.orchestra.run.smithi148.stdout:Installed: 2024-03-29T07:53:49.695 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-fuse-2:17.2.7-789.g4213e17c.el8.x86_64 fuse-2.9.7-19.el8.x86_64 2024-03-29T07:53:49.695 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T07:53:49.695 INFO:teuthology.orchestra.run.smithi148.stdout:Complete! 2024-03-29T07:53:49.771 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 5/5 2024-03-29T07:53:49.771 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T07:53:49.771 INFO:teuthology.orchestra.run.smithi037.stdout:Installed: 2024-03-29T07:53:49.771 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mgr-cephadm-2:17.2.7-789.g4213e17c.el8.noarch 2024-03-29T07:53:49.772 INFO:teuthology.orchestra.run.smithi037.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2024-03-29T07:53:49.772 INFO:teuthology.orchestra.run.smithi037.stdout: python3-babel-2.5.1-7.el8.noarch 2024-03-29T07:53:49.772 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jinja2-2.10.1-4.el8.noarch 2024-03-29T07:53:49.772 INFO:teuthology.orchestra.run.smithi037.stdout: python3-natsort-7.1.1-2.el8.noarch 2024-03-29T07:53:49.772 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T07:53:49.772 INFO:teuthology.orchestra.run.smithi037.stdout:Complete! 2024-03-29T07:53:49.938 DEBUG:teuthology.orchestra.run.smithi037:> sudo yum -y install ceph-fuse 2024-03-29T07:53:49.961 DEBUG:teuthology.orchestra.run.smithi148:> sudo yum -y install librados-devel 2024-03-29T07:53:50.902 INFO:teuthology.orchestra.run.smithi148.stdout:Last metadata expiration check: 0:02:24 ago on Fri 29 Mar 2024 07:51:26 AM UTC. 2024-03-29T07:53:50.905 INFO:teuthology.orchestra.run.smithi037.stdout:Last metadata expiration check: 0:02:24 ago on Fri 29 Mar 2024 07:51:26 AM UTC. 2024-03-29T07:53:52.617 INFO:teuthology.orchestra.run.smithi148.stdout:Dependencies resolved. 2024-03-29T07:53:52.617 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-03-29T07:53:52.618 INFO:teuthology.orchestra.run.smithi148.stdout: Package Arch Version Repo Size 2024-03-29T07:53:52.618 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-03-29T07:53:52.618 INFO:teuthology.orchestra.run.smithi148.stdout:Installing: 2024-03-29T07:53:52.618 INFO:teuthology.orchestra.run.smithi148.stdout: librados-devel x86_64 2:17.2.7-789.g4213e17c.el8 ceph 130 k 2024-03-29T07:53:52.618 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T07:53:52.618 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction Summary 2024-03-29T07:53:52.618 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-03-29T07:53:52.618 INFO:teuthology.orchestra.run.smithi148.stdout:Install 1 Package 2024-03-29T07:53:52.618 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T07:53:52.619 INFO:teuthology.orchestra.run.smithi148.stdout:Total download size: 130 k 2024-03-29T07:53:52.619 INFO:teuthology.orchestra.run.smithi148.stdout:Installed size: 464 k 2024-03-29T07:53:52.619 INFO:teuthology.orchestra.run.smithi148.stdout:Downloading Packages: 2024-03-29T07:53:52.787 INFO:teuthology.orchestra.run.smithi037.stdout:Dependencies resolved. 2024-03-29T07:53:52.788 INFO:teuthology.orchestra.run.smithi037.stdout:================================================================================ 2024-03-29T07:53:52.788 INFO:teuthology.orchestra.run.smithi037.stdout: Package Arch Version Repository Size 2024-03-29T07:53:52.788 INFO:teuthology.orchestra.run.smithi037.stdout:================================================================================ 2024-03-29T07:53:52.788 INFO:teuthology.orchestra.run.smithi037.stdout:Installing: 2024-03-29T07:53:52.788 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-fuse x86_64 2:17.2.7-789.g4213e17c.el8 ceph 876 k 2024-03-29T07:53:52.788 INFO:teuthology.orchestra.run.smithi037.stdout:Installing dependencies: 2024-03-29T07:53:52.788 INFO:teuthology.orchestra.run.smithi037.stdout: fuse x86_64 2.9.7-19.el8 baseos 83 k 2024-03-29T07:53:52.788 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T07:53:52.788 INFO:teuthology.orchestra.run.smithi037.stdout:Transaction Summary 2024-03-29T07:53:52.788 INFO:teuthology.orchestra.run.smithi037.stdout:================================================================================ 2024-03-29T07:53:52.788 INFO:teuthology.orchestra.run.smithi037.stdout:Install 2 Packages 2024-03-29T07:53:52.789 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T07:53:52.789 INFO:teuthology.orchestra.run.smithi037.stdout:Total download size: 959 k 2024-03-29T07:53:52.789 INFO:teuthology.orchestra.run.smithi037.stdout:Installed size: 3.1 M 2024-03-29T07:53:52.789 INFO:teuthology.orchestra.run.smithi037.stdout:Downloading Packages: 2024-03-29T07:53:52.959 INFO:teuthology.orchestra.run.smithi148.stdout:librados-devel-17.2.7-789.g4213e17c.el8.x86_64. 382 kB/s | 130 kB 00:00 2024-03-29T07:53:52.959 INFO:teuthology.orchestra.run.smithi148.stdout:-------------------------------------------------------------------------------- 2024-03-29T07:53:52.960 INFO:teuthology.orchestra.run.smithi148.stdout:Total 380 kB/s | 130 kB 00:00 2024-03-29T07:53:52.960 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction check 2024-03-29T07:53:52.967 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction check succeeded. 2024-03-29T07:53:52.968 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction test 2024-03-29T07:53:53.025 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction test succeeded. 2024-03-29T07:53:53.031 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction 2024-03-29T07:53:53.135 INFO:teuthology.orchestra.run.smithi037.stdout:(1/2): fuse-2.9.7-19.el8.x86_64.rpm 291 kB/s | 83 kB 00:00 2024-03-29T07:53:53.197 INFO:teuthology.orchestra.run.smithi148.stdout: Preparing : 1/1 2024-03-29T07:53:53.252 INFO:teuthology.orchestra.run.smithi037.stdout:(2/2): ceph-fuse-17.2.7-789.g4213e17c.el8.x86_6 2.1 MB/s | 876 kB 00:00 2024-03-29T07:53:53.252 INFO:teuthology.orchestra.run.smithi037.stdout:-------------------------------------------------------------------------------- 2024-03-29T07:53:53.253 INFO:teuthology.orchestra.run.smithi037.stdout:Total 2.0 MB/s | 959 kB 00:00 2024-03-29T07:53:53.275 INFO:teuthology.orchestra.run.smithi037.stdout:Running transaction check 2024-03-29T07:53:53.301 INFO:teuthology.orchestra.run.smithi037.stdout:Transaction check succeeded. 2024-03-29T07:53:53.301 INFO:teuthology.orchestra.run.smithi037.stdout:Running transaction test 2024-03-29T07:53:53.417 INFO:teuthology.orchestra.run.smithi037.stdout:Transaction test succeeded. 2024-03-29T07:53:53.421 INFO:teuthology.orchestra.run.smithi037.stdout:Running transaction 2024-03-29T07:53:53.489 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : librados-devel-2:17.2.7-789.g4213e17c.el8.x86_64 1/1 2024-03-29T07:53:53.624 INFO:teuthology.orchestra.run.smithi037.stdout: Preparing : 1/1 2024-03-29T07:53:53.859 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: librados-devel-2:17.2.7-789.g4213e17c.el8.x86_64 1/1 2024-03-29T07:53:53.900 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : fuse-2.9.7-19.el8.x86_64 1/2 2024-03-29T07:53:54.094 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-fuse-2:17.2.7-789.g4213e17c.el8.x86_64 2/2 2024-03-29T07:53:54.306 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : librados-devel-2:17.2.7-789.g4213e17c.el8.x86_64 1/1 2024-03-29T07:53:54.306 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T07:53:54.307 INFO:teuthology.orchestra.run.smithi148.stdout:Installed: 2024-03-29T07:53:54.307 INFO:teuthology.orchestra.run.smithi148.stdout: librados-devel-2:17.2.7-789.g4213e17c.el8.x86_64 2024-03-29T07:53:54.307 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T07:53:54.307 INFO:teuthology.orchestra.run.smithi148.stdout:Complete! 2024-03-29T07:53:54.530 DEBUG:teuthology.orchestra.run.smithi148:> sudo yum -y install libcephfs2 2024-03-29T07:53:54.721 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-fuse-2:17.2.7-789.g4213e17c.el8.x86_64 2/2 2024-03-29T07:53:54.721 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-fuse-2:17.2.7-789.g4213e17c.el8.x86_64 1/2 2024-03-29T07:53:55.172 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : fuse-2.9.7-19.el8.x86_64 2/2 2024-03-29T07:53:55.172 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T07:53:55.172 INFO:teuthology.orchestra.run.smithi037.stdout:Installed: 2024-03-29T07:53:55.172 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-fuse-2:17.2.7-789.g4213e17c.el8.x86_64 fuse-2.9.7-19.el8.x86_64 2024-03-29T07:53:55.172 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T07:53:55.172 INFO:teuthology.orchestra.run.smithi037.stdout:Complete! 2024-03-29T07:53:55.398 DEBUG:teuthology.orchestra.run.smithi037:> sudo yum -y install librados-devel 2024-03-29T07:53:55.463 INFO:teuthology.orchestra.run.smithi148.stdout:Last metadata expiration check: 0:02:29 ago on Fri 29 Mar 2024 07:51:26 AM UTC. 2024-03-29T07:53:56.473 INFO:teuthology.orchestra.run.smithi037.stdout:Last metadata expiration check: 0:02:30 ago on Fri 29 Mar 2024 07:51:26 AM UTC. 2024-03-29T07:53:57.100 INFO:teuthology.orchestra.run.smithi148.stdout:Package libcephfs2-2:17.2.7-789.g4213e17c.el8.x86_64 is already installed. 2024-03-29T07:53:57.152 INFO:teuthology.orchestra.run.smithi148.stdout:Dependencies resolved. 2024-03-29T07:53:57.153 INFO:teuthology.orchestra.run.smithi148.stdout:Nothing to do. 2024-03-29T07:53:57.153 INFO:teuthology.orchestra.run.smithi148.stdout:Complete! 2024-03-29T07:53:57.273 DEBUG:teuthology.orchestra.run.smithi148:> sudo yum -y install libcephfs-devel 2024-03-29T07:53:58.200 INFO:teuthology.orchestra.run.smithi148.stdout:Last metadata expiration check: 0:02:32 ago on Fri 29 Mar 2024 07:51:26 AM UTC. 2024-03-29T07:53:58.341 INFO:teuthology.orchestra.run.smithi037.stdout:Dependencies resolved. 2024-03-29T07:53:58.343 INFO:teuthology.orchestra.run.smithi037.stdout:================================================================================ 2024-03-29T07:53:58.345 INFO:teuthology.orchestra.run.smithi037.stdout: Package Arch Version Repo Size 2024-03-29T07:53:58.345 INFO:teuthology.orchestra.run.smithi037.stdout:================================================================================ 2024-03-29T07:53:58.346 INFO:teuthology.orchestra.run.smithi037.stdout:Installing: 2024-03-29T07:53:58.346 INFO:teuthology.orchestra.run.smithi037.stdout: librados-devel x86_64 2:17.2.7-789.g4213e17c.el8 ceph 130 k 2024-03-29T07:53:58.346 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T07:53:58.346 INFO:teuthology.orchestra.run.smithi037.stdout:Transaction Summary 2024-03-29T07:53:58.346 INFO:teuthology.orchestra.run.smithi037.stdout:================================================================================ 2024-03-29T07:53:58.347 INFO:teuthology.orchestra.run.smithi037.stdout:Install 1 Package 2024-03-29T07:53:58.347 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T07:53:58.347 INFO:teuthology.orchestra.run.smithi037.stdout:Total download size: 130 k 2024-03-29T07:53:58.347 INFO:teuthology.orchestra.run.smithi037.stdout:Installed size: 464 k 2024-03-29T07:53:58.347 INFO:teuthology.orchestra.run.smithi037.stdout:Downloading Packages: 2024-03-29T07:53:58.674 INFO:teuthology.orchestra.run.smithi037.stdout:librados-devel-17.2.7-789.g4213e17c.el8.x86_64. 393 kB/s | 130 kB 00:00 2024-03-29T07:53:58.674 INFO:teuthology.orchestra.run.smithi037.stdout:-------------------------------------------------------------------------------- 2024-03-29T07:53:58.675 INFO:teuthology.orchestra.run.smithi037.stdout:Total 391 kB/s | 130 kB 00:00 2024-03-29T07:53:58.675 INFO:teuthology.orchestra.run.smithi037.stdout:Running transaction check 2024-03-29T07:53:58.683 INFO:teuthology.orchestra.run.smithi037.stdout:Transaction check succeeded. 2024-03-29T07:53:58.683 INFO:teuthology.orchestra.run.smithi037.stdout:Running transaction test 2024-03-29T07:53:58.745 INFO:teuthology.orchestra.run.smithi037.stdout:Transaction test succeeded. 2024-03-29T07:53:58.749 INFO:teuthology.orchestra.run.smithi037.stdout:Running transaction 2024-03-29T07:53:58.873 INFO:teuthology.orchestra.run.smithi037.stdout: Preparing : 1/1 2024-03-29T07:53:59.126 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : librados-devel-2:17.2.7-789.g4213e17c.el8.x86_64 1/1 2024-03-29T07:53:59.515 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: librados-devel-2:17.2.7-789.g4213e17c.el8.x86_64 1/1 2024-03-29T07:53:59.894 INFO:teuthology.orchestra.run.smithi148.stdout:Dependencies resolved. 2024-03-29T07:53:59.894 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-03-29T07:53:59.895 INFO:teuthology.orchestra.run.smithi148.stdout: Package Arch Version Repo Size 2024-03-29T07:53:59.895 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-03-29T07:53:59.895 INFO:teuthology.orchestra.run.smithi148.stdout:Installing: 2024-03-29T07:53:59.895 INFO:teuthology.orchestra.run.smithi148.stdout: libcephfs-devel x86_64 2:17.2.7-789.g4213e17c.el8 ceph 25 k 2024-03-29T07:53:59.895 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T07:53:59.895 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction Summary 2024-03-29T07:53:59.895 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-03-29T07:53:59.895 INFO:teuthology.orchestra.run.smithi148.stdout:Install 1 Package 2024-03-29T07:53:59.895 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T07:53:59.896 INFO:teuthology.orchestra.run.smithi148.stdout:Total download size: 25 k 2024-03-29T07:53:59.896 INFO:teuthology.orchestra.run.smithi148.stdout:Installed size: 109 k 2024-03-29T07:53:59.896 INFO:teuthology.orchestra.run.smithi148.stdout:Downloading Packages: 2024-03-29T07:53:59.984 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : librados-devel-2:17.2.7-789.g4213e17c.el8.x86_64 1/1 2024-03-29T07:53:59.984 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T07:53:59.984 INFO:teuthology.orchestra.run.smithi037.stdout:Installed: 2024-03-29T07:53:59.985 INFO:teuthology.orchestra.run.smithi037.stdout: librados-devel-2:17.2.7-789.g4213e17c.el8.x86_64 2024-03-29T07:53:59.985 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T07:53:59.985 INFO:teuthology.orchestra.run.smithi037.stdout:Complete! 2024-03-29T07:54:00.038 INFO:teuthology.orchestra.run.smithi148.stdout:libcephfs-devel-17.2.7-789.g4213e17c.el8.x86_64 175 kB/s | 25 kB 00:00 2024-03-29T07:54:00.038 INFO:teuthology.orchestra.run.smithi148.stdout:-------------------------------------------------------------------------------- 2024-03-29T07:54:00.039 INFO:teuthology.orchestra.run.smithi148.stdout:Total 173 kB/s | 25 kB 00:00 2024-03-29T07:54:00.039 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction check 2024-03-29T07:54:00.044 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction check succeeded. 2024-03-29T07:54:00.044 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction test 2024-03-29T07:54:00.048 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction test succeeded. 2024-03-29T07:54:00.051 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction 2024-03-29T07:54:00.107 INFO:teuthology.orchestra.run.smithi148.stdout: Preparing : 1/1 2024-03-29T07:54:00.244 DEBUG:teuthology.orchestra.run.smithi037:> sudo yum -y install libcephfs2 2024-03-29T07:54:00.337 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libcephfs-devel-2:17.2.7-789.g4213e17c.el8.x86_64 1/1 2024-03-29T07:54:00.659 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: libcephfs-devel-2:17.2.7-789.g4213e17c.el8.x86_64 1/1 2024-03-29T07:54:01.044 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libcephfs-devel-2:17.2.7-789.g4213e17c.el8.x86_64 1/1 2024-03-29T07:54:01.045 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T07:54:01.045 INFO:teuthology.orchestra.run.smithi148.stdout:Installed: 2024-03-29T07:54:01.045 INFO:teuthology.orchestra.run.smithi148.stdout: libcephfs-devel-2:17.2.7-789.g4213e17c.el8.x86_64 2024-03-29T07:54:01.045 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T07:54:01.045 INFO:teuthology.orchestra.run.smithi148.stdout:Complete! 2024-03-29T07:54:01.191 DEBUG:teuthology.orchestra.run.smithi148:> sudo yum -y install librados2 2024-03-29T07:54:01.212 INFO:teuthology.orchestra.run.smithi037.stdout:Last metadata expiration check: 0:02:35 ago on Fri 29 Mar 2024 07:51:26 AM UTC. 2024-03-29T07:54:02.108 INFO:teuthology.orchestra.run.smithi148.stdout:Last metadata expiration check: 0:02:36 ago on Fri 29 Mar 2024 07:51:26 AM UTC. 2024-03-29T07:54:02.966 INFO:teuthology.orchestra.run.smithi037.stdout:Package libcephfs2-2:17.2.7-789.g4213e17c.el8.x86_64 is already installed. 2024-03-29T07:54:03.023 INFO:teuthology.orchestra.run.smithi037.stdout:Dependencies resolved. 2024-03-29T07:54:03.024 INFO:teuthology.orchestra.run.smithi037.stdout:Nothing to do. 2024-03-29T07:54:03.024 INFO:teuthology.orchestra.run.smithi037.stdout:Complete! 2024-03-29T07:54:03.140 DEBUG:teuthology.orchestra.run.smithi037:> sudo yum -y install libcephfs-devel 2024-03-29T07:54:03.835 INFO:teuthology.orchestra.run.smithi148.stdout:Package librados2-2:17.2.7-789.g4213e17c.el8.x86_64 is already installed. 2024-03-29T07:54:03.886 INFO:teuthology.orchestra.run.smithi148.stdout:Dependencies resolved. 2024-03-29T07:54:03.888 INFO:teuthology.orchestra.run.smithi148.stdout:Nothing to do. 2024-03-29T07:54:03.888 INFO:teuthology.orchestra.run.smithi148.stdout:Complete! 2024-03-29T07:54:04.009 DEBUG:teuthology.orchestra.run.smithi148:> sudo yum -y install librbd1 2024-03-29T07:54:04.099 INFO:teuthology.orchestra.run.smithi037.stdout:Last metadata expiration check: 0:02:38 ago on Fri 29 Mar 2024 07:51:26 AM UTC. 2024-03-29T07:54:04.922 INFO:teuthology.orchestra.run.smithi148.stdout:Last metadata expiration check: 0:02:38 ago on Fri 29 Mar 2024 07:51:26 AM UTC. 2024-03-29T07:54:05.910 INFO:teuthology.orchestra.run.smithi037.stdout:Dependencies resolved. 2024-03-29T07:54:05.910 INFO:teuthology.orchestra.run.smithi037.stdout:================================================================================ 2024-03-29T07:54:05.911 INFO:teuthology.orchestra.run.smithi037.stdout: Package Arch Version Repo Size 2024-03-29T07:54:05.911 INFO:teuthology.orchestra.run.smithi037.stdout:================================================================================ 2024-03-29T07:54:05.911 INFO:teuthology.orchestra.run.smithi037.stdout:Installing: 2024-03-29T07:54:05.911 INFO:teuthology.orchestra.run.smithi037.stdout: libcephfs-devel x86_64 2:17.2.7-789.g4213e17c.el8 ceph 25 k 2024-03-29T07:54:05.911 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T07:54:05.911 INFO:teuthology.orchestra.run.smithi037.stdout:Transaction Summary 2024-03-29T07:54:05.911 INFO:teuthology.orchestra.run.smithi037.stdout:================================================================================ 2024-03-29T07:54:05.911 INFO:teuthology.orchestra.run.smithi037.stdout:Install 1 Package 2024-03-29T07:54:05.911 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T07:54:05.911 INFO:teuthology.orchestra.run.smithi037.stdout:Total download size: 25 k 2024-03-29T07:54:05.911 INFO:teuthology.orchestra.run.smithi037.stdout:Installed size: 109 k 2024-03-29T07:54:05.911 INFO:teuthology.orchestra.run.smithi037.stdout:Downloading Packages: 2024-03-29T07:54:06.130 INFO:teuthology.orchestra.run.smithi037.stdout:libcephfs-devel-17.2.7-789.g4213e17c.el8.x86_64 114 kB/s | 25 kB 00:00 2024-03-29T07:54:06.130 INFO:teuthology.orchestra.run.smithi037.stdout:-------------------------------------------------------------------------------- 2024-03-29T07:54:06.131 INFO:teuthology.orchestra.run.smithi037.stdout:Total 113 kB/s | 25 kB 00:00 2024-03-29T07:54:06.131 INFO:teuthology.orchestra.run.smithi037.stdout:Running transaction check 2024-03-29T07:54:06.137 INFO:teuthology.orchestra.run.smithi037.stdout:Transaction check succeeded. 2024-03-29T07:54:06.137 INFO:teuthology.orchestra.run.smithi037.stdout:Running transaction test 2024-03-29T07:54:06.141 INFO:teuthology.orchestra.run.smithi037.stdout:Transaction test succeeded. 2024-03-29T07:54:06.144 INFO:teuthology.orchestra.run.smithi037.stdout:Running transaction 2024-03-29T07:54:06.197 INFO:teuthology.orchestra.run.smithi037.stdout: Preparing : 1/1 2024-03-29T07:54:06.417 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libcephfs-devel-2:17.2.7-789.g4213e17c.el8.x86_64 1/1 2024-03-29T07:54:06.573 INFO:teuthology.orchestra.run.smithi148.stdout:Package librbd1-2:17.2.7-789.g4213e17c.el8.x86_64 is already installed. 2024-03-29T07:54:06.626 INFO:teuthology.orchestra.run.smithi148.stdout:Dependencies resolved. 2024-03-29T07:54:06.627 INFO:teuthology.orchestra.run.smithi148.stdout:Nothing to do. 2024-03-29T07:54:06.627 INFO:teuthology.orchestra.run.smithi148.stdout:Complete! 2024-03-29T07:54:06.740 DEBUG:teuthology.orchestra.run.smithi148:> sudo yum -y install python3-rados 2024-03-29T07:54:06.740 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: libcephfs-devel-2:17.2.7-789.g4213e17c.el8.x86_64 1/1 2024-03-29T07:54:07.149 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libcephfs-devel-2:17.2.7-789.g4213e17c.el8.x86_64 1/1 2024-03-29T07:54:07.149 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T07:54:07.149 INFO:teuthology.orchestra.run.smithi037.stdout:Installed: 2024-03-29T07:54:07.149 INFO:teuthology.orchestra.run.smithi037.stdout: libcephfs-devel-2:17.2.7-789.g4213e17c.el8.x86_64 2024-03-29T07:54:07.149 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T07:54:07.149 INFO:teuthology.orchestra.run.smithi037.stdout:Complete! 2024-03-29T07:54:07.299 DEBUG:teuthology.orchestra.run.smithi037:> sudo yum -y install librados2 2024-03-29T07:54:07.659 INFO:teuthology.orchestra.run.smithi148.stdout:Last metadata expiration check: 0:02:41 ago on Fri 29 Mar 2024 07:51:26 AM UTC. 2024-03-29T07:54:08.257 INFO:teuthology.orchestra.run.smithi037.stdout:Last metadata expiration check: 0:02:42 ago on Fri 29 Mar 2024 07:51:26 AM UTC. 2024-03-29T07:54:09.371 INFO:teuthology.orchestra.run.smithi148.stdout:Package python3-rados-2:17.2.7-789.g4213e17c.el8.x86_64 is already installed. 2024-03-29T07:54:09.423 INFO:teuthology.orchestra.run.smithi148.stdout:Dependencies resolved. 2024-03-29T07:54:09.424 INFO:teuthology.orchestra.run.smithi148.stdout:Nothing to do. 2024-03-29T07:54:09.425 INFO:teuthology.orchestra.run.smithi148.stdout:Complete! 2024-03-29T07:54:09.542 DEBUG:teuthology.orchestra.run.smithi148:> sudo yum -y install python3-rgw 2024-03-29T07:54:10.078 INFO:teuthology.orchestra.run.smithi037.stdout:Package librados2-2:17.2.7-789.g4213e17c.el8.x86_64 is already installed. 2024-03-29T07:54:10.133 INFO:teuthology.orchestra.run.smithi037.stdout:Dependencies resolved. 2024-03-29T07:54:10.135 INFO:teuthology.orchestra.run.smithi037.stdout:Nothing to do. 2024-03-29T07:54:10.135 INFO:teuthology.orchestra.run.smithi037.stdout:Complete! 2024-03-29T07:54:10.253 DEBUG:teuthology.orchestra.run.smithi037:> sudo yum -y install librbd1 2024-03-29T07:54:10.462 INFO:teuthology.orchestra.run.smithi148.stdout:Last metadata expiration check: 0:02:44 ago on Fri 29 Mar 2024 07:51:26 AM UTC. 2024-03-29T07:54:11.212 INFO:teuthology.orchestra.run.smithi037.stdout:Last metadata expiration check: 0:02:45 ago on Fri 29 Mar 2024 07:51:26 AM UTC. 2024-03-29T07:54:12.114 INFO:teuthology.orchestra.run.smithi148.stdout:Package python3-rgw-2:17.2.7-789.g4213e17c.el8.x86_64 is already installed. 2024-03-29T07:54:12.166 INFO:teuthology.orchestra.run.smithi148.stdout:Dependencies resolved. 2024-03-29T07:54:12.167 INFO:teuthology.orchestra.run.smithi148.stdout:Nothing to do. 2024-03-29T07:54:12.167 INFO:teuthology.orchestra.run.smithi148.stdout:Complete! 2024-03-29T07:54:12.290 DEBUG:teuthology.orchestra.run.smithi148:> sudo yum -y install python3-cephfs 2024-03-29T07:54:13.045 INFO:teuthology.orchestra.run.smithi037.stdout:Package librbd1-2:17.2.7-789.g4213e17c.el8.x86_64 is already installed. 2024-03-29T07:54:13.102 INFO:teuthology.orchestra.run.smithi037.stdout:Dependencies resolved. 2024-03-29T07:54:13.104 INFO:teuthology.orchestra.run.smithi037.stdout:Nothing to do. 2024-03-29T07:54:13.104 INFO:teuthology.orchestra.run.smithi037.stdout:Complete! 2024-03-29T07:54:13.214 INFO:teuthology.orchestra.run.smithi148.stdout:Last metadata expiration check: 0:02:47 ago on Fri 29 Mar 2024 07:51:26 AM UTC. 2024-03-29T07:54:13.220 DEBUG:teuthology.orchestra.run.smithi037:> sudo yum -y install python3-rados 2024-03-29T07:54:14.182 INFO:teuthology.orchestra.run.smithi037.stdout:Last metadata expiration check: 0:02:48 ago on Fri 29 Mar 2024 07:51:26 AM UTC. 2024-03-29T07:54:14.927 INFO:teuthology.orchestra.run.smithi148.stdout:Package python3-cephfs-2:17.2.7-789.g4213e17c.el8.x86_64 is already installed. 2024-03-29T07:54:14.980 INFO:teuthology.orchestra.run.smithi148.stdout:Dependencies resolved. 2024-03-29T07:54:14.981 INFO:teuthology.orchestra.run.smithi148.stdout:Nothing to do. 2024-03-29T07:54:14.981 INFO:teuthology.orchestra.run.smithi148.stdout:Complete! 2024-03-29T07:54:15.088 DEBUG:teuthology.orchestra.run.smithi148:> sudo yum -y install python3-rbd 2024-03-29T07:54:15.996 INFO:teuthology.orchestra.run.smithi037.stdout:Package python3-rados-2:17.2.7-789.g4213e17c.el8.x86_64 is already installed. 2024-03-29T07:54:16.004 INFO:teuthology.orchestra.run.smithi148.stdout:Last metadata expiration check: 0:02:50 ago on Fri 29 Mar 2024 07:51:26 AM UTC. 2024-03-29T07:54:16.054 INFO:teuthology.orchestra.run.smithi037.stdout:Dependencies resolved. 2024-03-29T07:54:16.056 INFO:teuthology.orchestra.run.smithi037.stdout:Nothing to do. 2024-03-29T07:54:16.056 INFO:teuthology.orchestra.run.smithi037.stdout:Complete! 2024-03-29T07:54:16.180 DEBUG:teuthology.orchestra.run.smithi037:> sudo yum -y install python3-rgw 2024-03-29T07:54:17.141 INFO:teuthology.orchestra.run.smithi037.stdout:Last metadata expiration check: 0:02:51 ago on Fri 29 Mar 2024 07:51:26 AM UTC. 2024-03-29T07:54:17.643 INFO:teuthology.orchestra.run.smithi148.stdout:Package python3-rbd-2:17.2.7-789.g4213e17c.el8.x86_64 is already installed. 2024-03-29T07:54:17.694 INFO:teuthology.orchestra.run.smithi148.stdout:Dependencies resolved. 2024-03-29T07:54:17.695 INFO:teuthology.orchestra.run.smithi148.stdout:Nothing to do. 2024-03-29T07:54:17.695 INFO:teuthology.orchestra.run.smithi148.stdout:Complete! 2024-03-29T07:54:17.814 DEBUG:teuthology.orchestra.run.smithi148:> sudo yum -y install rbd-fuse 2024-03-29T07:54:18.736 INFO:teuthology.orchestra.run.smithi148.stdout:Last metadata expiration check: 0:02:52 ago on Fri 29 Mar 2024 07:51:26 AM UTC. 2024-03-29T07:54:18.901 INFO:teuthology.orchestra.run.smithi037.stdout:Package python3-rgw-2:17.2.7-789.g4213e17c.el8.x86_64 is already installed. 2024-03-29T07:54:18.957 INFO:teuthology.orchestra.run.smithi037.stdout:Dependencies resolved. 2024-03-29T07:54:18.958 INFO:teuthology.orchestra.run.smithi037.stdout:Nothing to do. 2024-03-29T07:54:18.958 INFO:teuthology.orchestra.run.smithi037.stdout:Complete! 2024-03-29T07:54:19.076 DEBUG:teuthology.orchestra.run.smithi037:> sudo yum -y install python3-cephfs 2024-03-29T07:54:20.037 INFO:teuthology.orchestra.run.smithi037.stdout:Last metadata expiration check: 0:02:54 ago on Fri 29 Mar 2024 07:51:26 AM UTC. 2024-03-29T07:54:20.650 INFO:teuthology.orchestra.run.smithi148.stdout:Dependencies resolved. 2024-03-29T07:54:20.654 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-03-29T07:54:20.654 INFO:teuthology.orchestra.run.smithi148.stdout: Package Architecture Version Repository Size 2024-03-29T07:54:20.655 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-03-29T07:54:20.655 INFO:teuthology.orchestra.run.smithi148.stdout:Installing: 2024-03-29T07:54:20.655 INFO:teuthology.orchestra.run.smithi148.stdout: rbd-fuse x86_64 2:17.2.7-789.g4213e17c.el8 ceph 90 k 2024-03-29T07:54:20.655 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T07:54:20.655 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction Summary 2024-03-29T07:54:20.655 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-03-29T07:54:20.655 INFO:teuthology.orchestra.run.smithi148.stdout:Install 1 Package 2024-03-29T07:54:20.656 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T07:54:20.656 INFO:teuthology.orchestra.run.smithi148.stdout:Total download size: 90 k 2024-03-29T07:54:20.656 INFO:teuthology.orchestra.run.smithi148.stdout:Installed size: 268 k 2024-03-29T07:54:20.656 INFO:teuthology.orchestra.run.smithi148.stdout:Downloading Packages: 2024-03-29T07:54:20.836 INFO:teuthology.orchestra.run.smithi148.stdout:rbd-fuse-17.2.7-789.g4213e17c.el8.x86_64.rpm 493 kB/s | 90 kB 00:00 2024-03-29T07:54:20.836 INFO:teuthology.orchestra.run.smithi148.stdout:-------------------------------------------------------------------------------- 2024-03-29T07:54:20.836 INFO:teuthology.orchestra.run.smithi148.stdout:Total 489 kB/s | 90 kB 00:00 2024-03-29T07:54:20.837 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction check 2024-03-29T07:54:20.849 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction check succeeded. 2024-03-29T07:54:20.849 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction test 2024-03-29T07:54:20.907 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction test succeeded. 2024-03-29T07:54:20.910 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction 2024-03-29T07:54:21.053 INFO:teuthology.orchestra.run.smithi148.stdout: Preparing : 1/1 2024-03-29T07:54:21.301 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : rbd-fuse-2:17.2.7-789.g4213e17c.el8.x86_64 1/1 2024-03-29T07:54:21.669 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: rbd-fuse-2:17.2.7-789.g4213e17c.el8.x86_64 1/1 2024-03-29T07:54:22.041 INFO:teuthology.orchestra.run.smithi037.stdout:Package python3-cephfs-2:17.2.7-789.g4213e17c.el8.x86_64 is already installed. 2024-03-29T07:54:22.096 INFO:teuthology.orchestra.run.smithi037.stdout:Dependencies resolved. 2024-03-29T07:54:22.097 INFO:teuthology.orchestra.run.smithi037.stdout:Nothing to do. 2024-03-29T07:54:22.097 INFO:teuthology.orchestra.run.smithi037.stdout:Complete! 2024-03-29T07:54:22.143 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : rbd-fuse-2:17.2.7-789.g4213e17c.el8.x86_64 1/1 2024-03-29T07:54:22.143 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T07:54:22.143 INFO:teuthology.orchestra.run.smithi148.stdout:Installed: 2024-03-29T07:54:22.143 INFO:teuthology.orchestra.run.smithi148.stdout: rbd-fuse-2:17.2.7-789.g4213e17c.el8.x86_64 2024-03-29T07:54:22.143 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T07:54:22.143 INFO:teuthology.orchestra.run.smithi148.stdout:Complete! 2024-03-29T07:54:22.213 DEBUG:teuthology.orchestra.run.smithi037:> sudo yum -y install python3-rbd 2024-03-29T07:54:22.436 DEBUG:teuthology.orchestra.run.smithi148:> sudo yum -y install rbd-mirror 2024-03-29T07:54:23.174 INFO:teuthology.orchestra.run.smithi037.stdout:Last metadata expiration check: 0:02:57 ago on Fri 29 Mar 2024 07:51:26 AM UTC. 2024-03-29T07:54:23.373 INFO:teuthology.orchestra.run.smithi148.stdout:Last metadata expiration check: 0:02:57 ago on Fri 29 Mar 2024 07:51:26 AM UTC. 2024-03-29T07:54:24.922 INFO:teuthology.orchestra.run.smithi037.stdout:Package python3-rbd-2:17.2.7-789.g4213e17c.el8.x86_64 is already installed. 2024-03-29T07:54:24.978 INFO:teuthology.orchestra.run.smithi037.stdout:Dependencies resolved. 2024-03-29T07:54:24.980 INFO:teuthology.orchestra.run.smithi037.stdout:Nothing to do. 2024-03-29T07:54:24.980 INFO:teuthology.orchestra.run.smithi037.stdout:Complete! 2024-03-29T07:54:25.064 INFO:teuthology.orchestra.run.smithi148.stdout:Dependencies resolved. 2024-03-29T07:54:25.065 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-03-29T07:54:25.065 INFO:teuthology.orchestra.run.smithi148.stdout: Package Arch Version Repo Size 2024-03-29T07:54:25.065 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-03-29T07:54:25.065 INFO:teuthology.orchestra.run.smithi148.stdout:Installing: 2024-03-29T07:54:25.065 INFO:teuthology.orchestra.run.smithi148.stdout: rbd-mirror x86_64 2:17.2.7-789.g4213e17c.el8 ceph 3.5 M 2024-03-29T07:54:25.065 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T07:54:25.065 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction Summary 2024-03-29T07:54:25.065 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-03-29T07:54:25.065 INFO:teuthology.orchestra.run.smithi148.stdout:Install 1 Package 2024-03-29T07:54:25.065 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T07:54:25.066 INFO:teuthology.orchestra.run.smithi148.stdout:Total download size: 3.5 M 2024-03-29T07:54:25.066 INFO:teuthology.orchestra.run.smithi148.stdout:Installed size: 16 M 2024-03-29T07:54:25.066 INFO:teuthology.orchestra.run.smithi148.stdout:Downloading Packages: 2024-03-29T07:54:25.097 DEBUG:teuthology.orchestra.run.smithi037:> sudo yum -y install rbd-fuse 2024-03-29T07:54:25.498 INFO:teuthology.orchestra.run.smithi148.stdout:rbd-mirror-17.2.7-789.g4213e17c.el8.x86_64.rpm 8.0 MB/s | 3.5 MB 00:00 2024-03-29T07:54:25.498 INFO:teuthology.orchestra.run.smithi148.stdout:-------------------------------------------------------------------------------- 2024-03-29T07:54:25.499 INFO:teuthology.orchestra.run.smithi148.stdout:Total 8.0 MB/s | 3.5 MB 00:00 2024-03-29T07:54:25.499 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction check 2024-03-29T07:54:25.513 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction check succeeded. 2024-03-29T07:54:25.513 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction test 2024-03-29T07:54:25.583 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction test succeeded. 2024-03-29T07:54:25.586 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction 2024-03-29T07:54:26.054 INFO:teuthology.orchestra.run.smithi037.stdout:Last metadata expiration check: 0:03:00 ago on Fri 29 Mar 2024 07:51:26 AM UTC. 2024-03-29T07:54:26.065 INFO:teuthology.orchestra.run.smithi148.stdout: Preparing : 1/1 2024-03-29T07:54:26.285 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : rbd-mirror-2:17.2.7-789.g4213e17c.el8.x86_64 1/1 2024-03-29T07:54:26.953 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: rbd-mirror-2:17.2.7-789.g4213e17c.el8.x86_64 1/1 2024-03-29T07:54:27.377 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : rbd-mirror-2:17.2.7-789.g4213e17c.el8.x86_64 1/1 2024-03-29T07:54:27.377 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T07:54:27.377 INFO:teuthology.orchestra.run.smithi148.stdout:Installed: 2024-03-29T07:54:27.377 INFO:teuthology.orchestra.run.smithi148.stdout: rbd-mirror-2:17.2.7-789.g4213e17c.el8.x86_64 2024-03-29T07:54:27.377 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T07:54:27.377 INFO:teuthology.orchestra.run.smithi148.stdout:Complete! 2024-03-29T07:54:27.672 DEBUG:teuthology.orchestra.run.smithi148:> sudo yum -y install rbd-nbd 2024-03-29T07:54:27.874 INFO:teuthology.orchestra.run.smithi037.stdout:Dependencies resolved. 2024-03-29T07:54:27.875 INFO:teuthology.orchestra.run.smithi037.stdout:================================================================================ 2024-03-29T07:54:27.875 INFO:teuthology.orchestra.run.smithi037.stdout: Package Architecture Version Repository Size 2024-03-29T07:54:27.875 INFO:teuthology.orchestra.run.smithi037.stdout:================================================================================ 2024-03-29T07:54:27.875 INFO:teuthology.orchestra.run.smithi037.stdout:Installing: 2024-03-29T07:54:27.875 INFO:teuthology.orchestra.run.smithi037.stdout: rbd-fuse x86_64 2:17.2.7-789.g4213e17c.el8 ceph 90 k 2024-03-29T07:54:27.875 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T07:54:27.875 INFO:teuthology.orchestra.run.smithi037.stdout:Transaction Summary 2024-03-29T07:54:27.875 INFO:teuthology.orchestra.run.smithi037.stdout:================================================================================ 2024-03-29T07:54:27.875 INFO:teuthology.orchestra.run.smithi037.stdout:Install 1 Package 2024-03-29T07:54:27.875 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T07:54:27.876 INFO:teuthology.orchestra.run.smithi037.stdout:Total download size: 90 k 2024-03-29T07:54:27.876 INFO:teuthology.orchestra.run.smithi037.stdout:Installed size: 268 k 2024-03-29T07:54:27.876 INFO:teuthology.orchestra.run.smithi037.stdout:Downloading Packages: 2024-03-29T07:54:28.062 INFO:teuthology.orchestra.run.smithi037.stdout:rbd-fuse-17.2.7-789.g4213e17c.el8.x86_64.rpm 488 kB/s | 90 kB 00:00 2024-03-29T07:54:28.063 INFO:teuthology.orchestra.run.smithi037.stdout:-------------------------------------------------------------------------------- 2024-03-29T07:54:28.063 INFO:teuthology.orchestra.run.smithi037.stdout:Total 484 kB/s | 90 kB 00:00 2024-03-29T07:54:28.063 INFO:teuthology.orchestra.run.smithi037.stdout:Running transaction check 2024-03-29T07:54:28.077 INFO:teuthology.orchestra.run.smithi037.stdout:Transaction check succeeded. 2024-03-29T07:54:28.077 INFO:teuthology.orchestra.run.smithi037.stdout:Running transaction test 2024-03-29T07:54:28.137 INFO:teuthology.orchestra.run.smithi037.stdout:Transaction test succeeded. 2024-03-29T07:54:28.140 INFO:teuthology.orchestra.run.smithi037.stdout:Running transaction 2024-03-29T07:54:28.263 INFO:teuthology.orchestra.run.smithi037.stdout: Preparing : 1/1 2024-03-29T07:54:28.481 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : rbd-fuse-2:17.2.7-789.g4213e17c.el8.x86_64 1/1 2024-03-29T07:54:28.597 INFO:teuthology.orchestra.run.smithi148.stdout:Last metadata expiration check: 0:03:02 ago on Fri 29 Mar 2024 07:51:26 AM UTC. 2024-03-29T07:54:28.865 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: rbd-fuse-2:17.2.7-789.g4213e17c.el8.x86_64 1/1 2024-03-29T07:54:29.336 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : rbd-fuse-2:17.2.7-789.g4213e17c.el8.x86_64 1/1 2024-03-29T07:54:29.336 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T07:54:29.336 INFO:teuthology.orchestra.run.smithi037.stdout:Installed: 2024-03-29T07:54:29.336 INFO:teuthology.orchestra.run.smithi037.stdout: rbd-fuse-2:17.2.7-789.g4213e17c.el8.x86_64 2024-03-29T07:54:29.336 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T07:54:29.336 INFO:teuthology.orchestra.run.smithi037.stdout:Complete! 2024-03-29T07:54:29.570 DEBUG:teuthology.orchestra.run.smithi037:> sudo yum -y install rbd-mirror 2024-03-29T07:54:30.351 INFO:teuthology.orchestra.run.smithi148.stdout:Dependencies resolved. 2024-03-29T07:54:30.352 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-03-29T07:54:30.352 INFO:teuthology.orchestra.run.smithi148.stdout: Package Architecture Version Repository Size 2024-03-29T07:54:30.352 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-03-29T07:54:30.352 INFO:teuthology.orchestra.run.smithi148.stdout:Installing: 2024-03-29T07:54:30.352 INFO:teuthology.orchestra.run.smithi148.stdout: rbd-nbd x86_64 2:17.2.7-789.g4213e17c.el8 ceph 175 k 2024-03-29T07:54:30.352 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T07:54:30.352 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction Summary 2024-03-29T07:54:30.352 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-03-29T07:54:30.352 INFO:teuthology.orchestra.run.smithi148.stdout:Install 1 Package 2024-03-29T07:54:30.352 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T07:54:30.353 INFO:teuthology.orchestra.run.smithi148.stdout:Total download size: 175 k 2024-03-29T07:54:30.353 INFO:teuthology.orchestra.run.smithi148.stdout:Installed size: 556 k 2024-03-29T07:54:30.353 INFO:teuthology.orchestra.run.smithi148.stdout:Downloading Packages: 2024-03-29T07:54:30.534 INFO:teuthology.orchestra.run.smithi037.stdout:Last metadata expiration check: 0:03:04 ago on Fri 29 Mar 2024 07:51:26 AM UTC. 2024-03-29T07:54:30.667 INFO:teuthology.orchestra.run.smithi148.stdout:rbd-nbd-17.2.7-789.g4213e17c.el8.x86_64.rpm 557 kB/s | 175 kB 00:00 2024-03-29T07:54:30.668 INFO:teuthology.orchestra.run.smithi148.stdout:-------------------------------------------------------------------------------- 2024-03-29T07:54:30.669 INFO:teuthology.orchestra.run.smithi148.stdout:Total 554 kB/s | 175 kB 00:00 2024-03-29T07:54:30.669 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction check 2024-03-29T07:54:30.682 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction check succeeded. 2024-03-29T07:54:30.682 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction test 2024-03-29T07:54:30.741 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction test succeeded. 2024-03-29T07:54:30.744 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction 2024-03-29T07:54:30.871 INFO:teuthology.orchestra.run.smithi148.stdout: Preparing : 1/1 2024-03-29T07:54:31.083 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : rbd-nbd-2:17.2.7-789.g4213e17c.el8.x86_64 1/1 2024-03-29T07:54:31.471 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: rbd-nbd-2:17.2.7-789.g4213e17c.el8.x86_64 1/1 2024-03-29T07:54:31.912 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : rbd-nbd-2:17.2.7-789.g4213e17c.el8.x86_64 1/1 2024-03-29T07:54:31.913 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T07:54:31.913 INFO:teuthology.orchestra.run.smithi148.stdout:Installed: 2024-03-29T07:54:31.913 INFO:teuthology.orchestra.run.smithi148.stdout: rbd-nbd-2:17.2.7-789.g4213e17c.el8.x86_64 2024-03-29T07:54:31.913 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T07:54:31.913 INFO:teuthology.orchestra.run.smithi148.stdout:Complete! 2024-03-29T07:54:32.167 DEBUG:teuthology.parallel:result is None 2024-03-29T07:54:32.337 INFO:teuthology.orchestra.run.smithi037.stdout:Dependencies resolved. 2024-03-29T07:54:32.338 INFO:teuthology.orchestra.run.smithi037.stdout:================================================================================ 2024-03-29T07:54:32.338 INFO:teuthology.orchestra.run.smithi037.stdout: Package Arch Version Repo Size 2024-03-29T07:54:32.338 INFO:teuthology.orchestra.run.smithi037.stdout:================================================================================ 2024-03-29T07:54:32.338 INFO:teuthology.orchestra.run.smithi037.stdout:Installing: 2024-03-29T07:54:32.338 INFO:teuthology.orchestra.run.smithi037.stdout: rbd-mirror x86_64 2:17.2.7-789.g4213e17c.el8 ceph 3.5 M 2024-03-29T07:54:32.338 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T07:54:32.338 INFO:teuthology.orchestra.run.smithi037.stdout:Transaction Summary 2024-03-29T07:54:32.338 INFO:teuthology.orchestra.run.smithi037.stdout:================================================================================ 2024-03-29T07:54:32.338 INFO:teuthology.orchestra.run.smithi037.stdout:Install 1 Package 2024-03-29T07:54:32.338 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T07:54:32.339 INFO:teuthology.orchestra.run.smithi037.stdout:Total download size: 3.5 M 2024-03-29T07:54:32.339 INFO:teuthology.orchestra.run.smithi037.stdout:Installed size: 16 M 2024-03-29T07:54:32.339 INFO:teuthology.orchestra.run.smithi037.stdout:Downloading Packages: 2024-03-29T07:54:32.941 INFO:teuthology.orchestra.run.smithi037.stdout:rbd-mirror-17.2.7-789.g4213e17c.el8.x86_64.rpm 5.7 MB/s | 3.5 MB 00:00 2024-03-29T07:54:32.941 INFO:teuthology.orchestra.run.smithi037.stdout:-------------------------------------------------------------------------------- 2024-03-29T07:54:32.942 INFO:teuthology.orchestra.run.smithi037.stdout:Total 5.7 MB/s | 3.5 MB 00:00 2024-03-29T07:54:32.942 INFO:teuthology.orchestra.run.smithi037.stdout:Running transaction check 2024-03-29T07:54:32.958 INFO:teuthology.orchestra.run.smithi037.stdout:Transaction check succeeded. 2024-03-29T07:54:32.958 INFO:teuthology.orchestra.run.smithi037.stdout:Running transaction test 2024-03-29T07:54:33.031 INFO:teuthology.orchestra.run.smithi037.stdout:Transaction test succeeded. 2024-03-29T07:54:33.034 INFO:teuthology.orchestra.run.smithi037.stdout:Running transaction 2024-03-29T07:54:33.550 INFO:teuthology.orchestra.run.smithi037.stdout: Preparing : 1/1 2024-03-29T07:54:33.748 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : rbd-mirror-2:17.2.7-789.g4213e17c.el8.x86_64 1/1 2024-03-29T07:54:34.458 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: rbd-mirror-2:17.2.7-789.g4213e17c.el8.x86_64 1/1 2024-03-29T07:54:34.890 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : rbd-mirror-2:17.2.7-789.g4213e17c.el8.x86_64 1/1 2024-03-29T07:54:34.890 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T07:54:34.891 INFO:teuthology.orchestra.run.smithi037.stdout:Installed: 2024-03-29T07:54:34.891 INFO:teuthology.orchestra.run.smithi037.stdout: rbd-mirror-2:17.2.7-789.g4213e17c.el8.x86_64 2024-03-29T07:54:34.891 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T07:54:34.891 INFO:teuthology.orchestra.run.smithi037.stdout:Complete! 2024-03-29T07:54:35.131 DEBUG:teuthology.orchestra.run.smithi037:> sudo yum -y install rbd-nbd 2024-03-29T07:54:36.104 INFO:teuthology.orchestra.run.smithi037.stdout:Last metadata expiration check: 0:03:10 ago on Fri 29 Mar 2024 07:51:26 AM UTC. 2024-03-29T07:54:38.102 INFO:teuthology.orchestra.run.smithi037.stdout:Dependencies resolved. 2024-03-29T07:54:38.103 INFO:teuthology.orchestra.run.smithi037.stdout:================================================================================ 2024-03-29T07:54:38.103 INFO:teuthology.orchestra.run.smithi037.stdout: Package Architecture Version Repository Size 2024-03-29T07:54:38.103 INFO:teuthology.orchestra.run.smithi037.stdout:================================================================================ 2024-03-29T07:54:38.103 INFO:teuthology.orchestra.run.smithi037.stdout:Installing: 2024-03-29T07:54:38.103 INFO:teuthology.orchestra.run.smithi037.stdout: rbd-nbd x86_64 2:17.2.7-789.g4213e17c.el8 ceph 175 k 2024-03-29T07:54:38.103 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T07:54:38.103 INFO:teuthology.orchestra.run.smithi037.stdout:Transaction Summary 2024-03-29T07:54:38.103 INFO:teuthology.orchestra.run.smithi037.stdout:================================================================================ 2024-03-29T07:54:38.104 INFO:teuthology.orchestra.run.smithi037.stdout:Install 1 Package 2024-03-29T07:54:38.104 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T07:54:38.104 INFO:teuthology.orchestra.run.smithi037.stdout:Total download size: 175 k 2024-03-29T07:54:38.104 INFO:teuthology.orchestra.run.smithi037.stdout:Installed size: 556 k 2024-03-29T07:54:38.105 INFO:teuthology.orchestra.run.smithi037.stdout:Downloading Packages: 2024-03-29T07:54:38.569 INFO:teuthology.orchestra.run.smithi037.stdout:rbd-nbd-17.2.7-789.g4213e17c.el8.x86_64.rpm 377 kB/s | 175 kB 00:00 2024-03-29T07:54:38.569 INFO:teuthology.orchestra.run.smithi037.stdout:-------------------------------------------------------------------------------- 2024-03-29T07:54:38.570 INFO:teuthology.orchestra.run.smithi037.stdout:Total 375 kB/s | 175 kB 00:00 2024-03-29T07:54:38.570 INFO:teuthology.orchestra.run.smithi037.stdout:Running transaction check 2024-03-29T07:54:38.584 INFO:teuthology.orchestra.run.smithi037.stdout:Transaction check succeeded. 2024-03-29T07:54:38.585 INFO:teuthology.orchestra.run.smithi037.stdout:Running transaction test 2024-03-29T07:54:38.646 INFO:teuthology.orchestra.run.smithi037.stdout:Transaction test succeeded. 2024-03-29T07:54:38.650 INFO:teuthology.orchestra.run.smithi037.stdout:Running transaction 2024-03-29T07:54:38.774 INFO:teuthology.orchestra.run.smithi037.stdout: Preparing : 1/1 2024-03-29T07:54:38.991 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : rbd-nbd-2:17.2.7-789.g4213e17c.el8.x86_64 1/1 2024-03-29T07:54:39.380 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: rbd-nbd-2:17.2.7-789.g4213e17c.el8.x86_64 1/1 2024-03-29T07:54:39.853 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : rbd-nbd-2:17.2.7-789.g4213e17c.el8.x86_64 1/1 2024-03-29T07:54:39.853 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T07:54:39.853 INFO:teuthology.orchestra.run.smithi037.stdout:Installed: 2024-03-29T07:54:39.853 INFO:teuthology.orchestra.run.smithi037.stdout: rbd-nbd-2:17.2.7-789.g4213e17c.el8.x86_64 2024-03-29T07:54:39.853 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T07:54:39.853 INFO:teuthology.orchestra.run.smithi037.stdout:Complete! 2024-03-29T07:54:40.090 DEBUG:teuthology.parallel:result is None 2024-03-29T07:54:40.090 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-03-29T07:54:40.090 INFO:teuthology.packaging:ref: None 2024-03-29T07:54:40.090 INFO:teuthology.packaging:tag: None 2024-03-29T07:54:40.091 INFO:teuthology.packaging:branch: quincy 2024-03-29T07:54:40.091 INFO:teuthology.packaging:sha1: d0e392d9aee635f2b65768c460c7a151ead7a5e2 2024-03-29T07:54:40.091 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=quincy 2024-03-29T07:54:40.315 DEBUG:teuthology.orchestra.run.smithi037:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-29T07:54:40.368 INFO:teuthology.orchestra.run.smithi037.stdout:17.2.7-789.g4213e17c.el8 2024-03-29T07:54:40.368 INFO:teuthology.packaging:The installed version of ceph is 17.2.7-789.g4213e17c.el8 2024-03-29T07:54:40.369 INFO:teuthology.task.install:The correct ceph version 17.2.7-789.g4213e17c is installed. 2024-03-29T07:54:40.370 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-03-29T07:54:40.371 INFO:teuthology.packaging:ref: None 2024-03-29T07:54:40.371 INFO:teuthology.packaging:tag: None 2024-03-29T07:54:40.371 INFO:teuthology.packaging:branch: quincy 2024-03-29T07:54:40.371 INFO:teuthology.packaging:sha1: d0e392d9aee635f2b65768c460c7a151ead7a5e2 2024-03-29T07:54:40.371 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=quincy 2024-03-29T07:54:40.595 DEBUG:teuthology.orchestra.run.smithi148:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-29T07:54:40.654 INFO:teuthology.orchestra.run.smithi148.stdout:17.2.7-789.g4213e17c.el8 2024-03-29T07:54:40.654 INFO:teuthology.packaging:The installed version of ceph is 17.2.7-789.g4213e17c.el8 2024-03-29T07:54:40.654 INFO:teuthology.task.install:The correct ceph version 17.2.7-789.g4213e17c is installed. 2024-03-29T07:54:40.657 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-03-29T07:54:40.657 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-03-29T07:54:40.657 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-29T07:54:40.728 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-03-29T07:54:40.729 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-29T07:54:40.799 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-03-29T07:54:40.799 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-03-29T07:54:40.800 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd of=/usr/bin/daemon-helper 2024-03-29T07:54:40.867 DEBUG:teuthology.orchestra.run.smithi037:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-29T07:54:40.973 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-03-29T07:54:40.973 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd of=/usr/bin/daemon-helper 2024-03-29T07:54:41.042 DEBUG:teuthology.orchestra.run.smithi148:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-29T07:54:41.147 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-03-29T07:54:41.148 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-03-29T07:54:41.148 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-29T07:54:41.215 DEBUG:teuthology.orchestra.run.smithi037:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-29T07:54:41.320 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-03-29T07:54:41.321 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-29T07:54:41.387 DEBUG:teuthology.orchestra.run.smithi148:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-29T07:54:41.493 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-03-29T07:54:41.493 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-03-29T07:54:41.493 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd of=/usr/bin/stdin-killer 2024-03-29T07:54:41.559 DEBUG:teuthology.orchestra.run.smithi037:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-29T07:54:41.664 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-03-29T07:54:41.664 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd of=/usr/bin/stdin-killer 2024-03-29T07:54:41.730 DEBUG:teuthology.orchestra.run.smithi148:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-29T07:54:41.835 INFO:teuthology.run_tasks:Running task print... 2024-03-29T07:54:41.845 INFO:teuthology.task.print:**** done install task... 2024-03-29T07:54:41.845 INFO:teuthology.run_tasks:Running task cephadm... 2024-03-29T07:54:41.932 INFO:tasks.cephadm:Config: {'cephadm_branch': 'quincy', 'cephadm_git_url': 'https://github.com/ceph/ceph', 'conf': {'osd': {'osd_class_default_list': '*', 'osd_class_load_list': '*', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'mon pg warn min per osd': 0}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}}, 'image': 'quay.ceph.io/ceph-ci/ceph:quincy', 'roleless': True, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'OSD_DOWN', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding'], 'sha1': 'd0e392d9aee635f2b65768c460c7a151ead7a5e2'} 2024-03-29T07:54:41.932 INFO:tasks.cephadm:Cluster image is quay.ceph.io/ceph-ci/ceph:quincy 2024-03-29T07:54:41.933 INFO:tasks.cephadm:Cluster fsid is 99b492c2-eda1-11ee-b647-cb9ed24678a4 2024-03-29T07:54:41.933 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-03-29T07:54:41.933 INFO:tasks.cephadm:No mon roles; fabricating mons 2024-03-29T07:54:41.933 INFO:tasks.cephadm:Monitor IPs: {'mon.smithi037': '172.21.15.37', 'mon.smithi148': '172.21.15.148'} 2024-03-29T07:54:41.933 INFO:tasks.cephadm:Normalizing hostnames... 2024-03-29T07:54:41.933 DEBUG:teuthology.orchestra.run.smithi037:> sudo hostname $(hostname -s) 2024-03-29T07:54:42.002 DEBUG:teuthology.orchestra.run.smithi148:> sudo hostname $(hostname -s) 2024-03-29T07:54:42.070 INFO:tasks.cephadm:Downloading cephadm (repo https://github.com/ceph/ceph ref quincy)... 2024-03-29T07:54:42.071 DEBUG:teuthology.orchestra.run.smithi037:> curl --silent https://raw.githubusercontent.com/ceph/ceph/quincy/src/cephadm/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-03-29T07:54:42.325 INFO:teuthology.orchestra.run.smithi037.stdout:-rw-rw-r--. 1 ubuntu ubuntu 375153 Mar 29 07:54 /home/ubuntu/cephtest/cephadm 2024-03-29T07:54:42.325 DEBUG:teuthology.orchestra.run.smithi148:> curl --silent https://raw.githubusercontent.com/ceph/ceph/quincy/src/cephadm/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-03-29T07:54:42.498 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-rw-r--. 1 ubuntu ubuntu 375153 Mar 29 07:54 /home/ubuntu/cephtest/cephadm 2024-03-29T07:54:42.499 DEBUG:teuthology.orchestra.run.smithi037:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-03-29T07:54:42.547 DEBUG:teuthology.orchestra.run.smithi148:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-03-29T07:54:42.604 INFO:tasks.cephadm:Pulling image quay.ceph.io/ceph-ci/ceph:quincy on all hosts... 2024-03-29T07:54:42.604 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy pull 2024-03-29T07:54:42.608 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy pull 2024-03-29T07:54:43.975 INFO:teuthology.orchestra.run.smithi037.stderr:Pulling container image quay.ceph.io/ceph-ci/ceph:quincy... 2024-03-29T07:54:44.083 INFO:teuthology.orchestra.run.smithi148.stderr:Pulling container image quay.ceph.io/ceph-ci/ceph:quincy... 2024-03-29T07:55:11.725 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-29T07:55:11.726 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph_version": "ceph version 17.2.7-789-g4213e17c (4213e17c4de67016ed14f55d22503f4923b3a901) quincy (stable)", 2024-03-29T07:55:11.726 INFO:teuthology.orchestra.run.smithi037.stdout: "image_id": "cf22eb8683be731b0447946652ee17d169ccd1820a07c94b7041aa2020906e1a", 2024-03-29T07:55:11.726 INFO:teuthology.orchestra.run.smithi037.stdout: "repo_digests": [ 2024-03-29T07:55:11.726 INFO:teuthology.orchestra.run.smithi037.stdout: "quay.ceph.io/ceph-ci/ceph@sha256:644b551321c45a6aa0b4a11a5b6aa05b865864a8f1b5d84ccb67d6a3c899481b" 2024-03-29T07:55:11.726 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-03-29T07:55:11.726 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-29T07:55:11.902 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-03-29T07:55:11.902 INFO:teuthology.orchestra.run.smithi148.stdout: "ceph_version": "ceph version 17.2.7-789-g4213e17c (4213e17c4de67016ed14f55d22503f4923b3a901) quincy (stable)", 2024-03-29T07:55:11.902 INFO:teuthology.orchestra.run.smithi148.stdout: "image_id": "cf22eb8683be731b0447946652ee17d169ccd1820a07c94b7041aa2020906e1a", 2024-03-29T07:55:11.903 INFO:teuthology.orchestra.run.smithi148.stdout: "repo_digests": [ 2024-03-29T07:55:11.903 INFO:teuthology.orchestra.run.smithi148.stdout: "quay.ceph.io/ceph-ci/ceph@sha256:644b551321c45a6aa0b4a11a5b6aa05b865864a8f1b5d84ccb67d6a3c899481b" 2024-03-29T07:55:11.903 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-03-29T07:55:11.903 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-03-29T07:55:11.930 DEBUG:teuthology.orchestra.run.smithi037:> sudo mkdir -p /etc/ceph 2024-03-29T07:55:11.999 DEBUG:teuthology.orchestra.run.smithi148:> sudo mkdir -p /etc/ceph 2024-03-29T07:55:12.069 DEBUG:teuthology.orchestra.run.smithi037:> sudo chmod 777 /etc/ceph 2024-03-29T07:55:12.136 DEBUG:teuthology.orchestra.run.smithi148:> sudo chmod 777 /etc/ceph 2024-03-29T07:55:12.201 INFO:tasks.cephadm:Writing seed config... 2024-03-29T07:55:12.202 INFO:tasks.cephadm: override: [osd] osd_class_default_list = * 2024-03-29T07:55:12.202 INFO:tasks.cephadm: override: [osd] osd_class_load_list = * 2024-03-29T07:55:12.202 INFO:tasks.cephadm: override: [osd] bdev async discard = True 2024-03-29T07:55:12.202 INFO:tasks.cephadm: override: [osd] bdev enable discard = True 2024-03-29T07:55:12.202 INFO:tasks.cephadm: override: [osd] bluestore allocator = bitmap 2024-03-29T07:55:12.202 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-03-29T07:55:12.202 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-03-29T07:55:12.203 INFO:tasks.cephadm: override: [osd] debug bluefs = 1/20 2024-03-29T07:55:12.203 INFO:tasks.cephadm: override: [osd] debug bluestore = 1/20 2024-03-29T07:55:12.203 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-03-29T07:55:12.203 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-03-29T07:55:12.203 INFO:tasks.cephadm: override: [osd] debug rocksdb = 4/10 2024-03-29T07:55:12.203 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-03-29T07:55:12.203 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-03-29T07:55:12.203 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-03-29T07:55:12.203 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-03-29T07:55:12.203 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-03-29T07:55:12.203 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-03-29T07:55:12.203 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-03-29T07:55:12.203 INFO:tasks.cephadm: override: [client] debug client = 20 2024-03-29T07:55:12.203 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-03-29T07:55:12.203 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-03-29T07:55:12.203 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-03-29T07:55:12.204 INFO:tasks.cephadm: override: [global] mon pg warn min per osd = 0 2024-03-29T07:55:12.204 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-03-29T07:55:12.204 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-03-29T07:55:12.204 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-03-29T07:55:12.204 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-03-29T07:55:12.204 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-03-29T07:55:12.204 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-03-29T07:55:12.204 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-03-29T07:55:12.204 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-03-29T07:55:12.204 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-03-29T07:55:12.204 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-03-29T07:55:12.204 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-03-29T07:55:12.204 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-03-29T07:55:12.204 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-03-29T07:55:12.204 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-03-29T07:55:12.204 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-03-29T07:55:12.204 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-03-29T07:55:12.205 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-03-29T07:55:12.205 DEBUG:teuthology.orchestra.run.smithi037:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-03-29T07:55:12.250 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 99b492c2-eda1-11ee-b647-cb9ed24678a4 mon pg warn min per osd = 0 [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = true bdev debug aio = true osd sloppy crc = true osd_class_default_list = * osd_class_load_list = * bdev async discard = True bdev enable discard = True bluestore allocator = bitmap bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 1/20 debug bluestore = 1/20 debug ms = 1 debug osd = 20 debug rocksdb = 4/10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-03-29T07:55:12.250 DEBUG:teuthology.orchestra.run.smithi037:mon.smithi037> sudo journalctl -f -n 0 -u ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4@mon.smithi037.service 2024-03-29T07:55:12.293 INFO:tasks.cephadm:Bootstrapping... 2024-03-29T07:55:12.293 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy -v bootstrap --fsid 99b492c2-eda1-11ee-b647-cb9ed24678a4 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-ip 172.21.15.37 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-03-29T07:55:12.360 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:-- Logs begin at Fri 2024-03-29 07:43:01 UTC. -- 2024-03-29T07:55:12.596 INFO:teuthology.orchestra.run.smithi037.stdout:-------------------------------------------------------------------------------- 2024-03-29T07:55:12.596 INFO:teuthology.orchestra.run.smithi037.stdout:cephadm ['--image', 'quay.ceph.io/ceph-ci/ceph:quincy', '-v', 'bootstrap', '--fsid', '99b492c2-eda1-11ee-b647-cb9ed24678a4', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-ip', '172.21.15.37', '--skip-admin-label'] 2024-03-29T07:55:12.642 INFO:teuthology.orchestra.run.smithi037.stdout:/bin/podman: stdout 4.9.4-dev 2024-03-29T07:55:12.643 INFO:teuthology.orchestra.run.smithi037.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-03-29T07:55:12.643 INFO:teuthology.orchestra.run.smithi037.stdout:Verifying podman|docker is present... 2024-03-29T07:55:12.688 INFO:teuthology.orchestra.run.smithi037.stdout:/bin/podman: stdout 4.9.4-dev 2024-03-29T07:55:12.688 INFO:teuthology.orchestra.run.smithi037.stdout:Verifying lvm2 is present... 2024-03-29T07:55:12.688 INFO:teuthology.orchestra.run.smithi037.stdout:Verifying time synchronization is in place... 2024-03-29T07:55:12.702 INFO:teuthology.orchestra.run.smithi037.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-03-29T07:55:12.703 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-03-29T07:55:12.715 INFO:teuthology.orchestra.run.smithi037.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-03-29T07:55:12.715 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stdout inactive 2024-03-29T07:55:12.728 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stdout enabled 2024-03-29T07:55:12.740 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stdout active 2024-03-29T07:55:12.741 INFO:teuthology.orchestra.run.smithi037.stdout:Unit chronyd.service is enabled and running 2024-03-29T07:55:12.741 INFO:teuthology.orchestra.run.smithi037.stdout:Repeating the final host check... 2024-03-29T07:55:12.784 INFO:teuthology.orchestra.run.smithi037.stdout:/bin/podman: stdout 4.9.4-dev 2024-03-29T07:55:12.784 INFO:teuthology.orchestra.run.smithi037.stdout:podman (/bin/podman) version 4.9.4 is present 2024-03-29T07:55:12.784 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl is present 2024-03-29T07:55:12.784 INFO:teuthology.orchestra.run.smithi037.stdout:lvcreate is present 2024-03-29T07:55:12.797 INFO:teuthology.orchestra.run.smithi037.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-03-29T07:55:12.798 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-03-29T07:55:12.810 INFO:teuthology.orchestra.run.smithi037.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-03-29T07:55:12.810 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stdout inactive 2024-03-29T07:55:12.823 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stdout enabled 2024-03-29T07:55:12.835 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stdout active 2024-03-29T07:55:12.835 INFO:teuthology.orchestra.run.smithi037.stdout:Unit chronyd.service is enabled and running 2024-03-29T07:55:12.835 INFO:teuthology.orchestra.run.smithi037.stdout:Host looks OK 2024-03-29T07:55:12.835 INFO:teuthology.orchestra.run.smithi037.stdout:Cluster fsid: 99b492c2-eda1-11ee-b647-cb9ed24678a4 2024-03-29T07:55:12.836 INFO:teuthology.orchestra.run.smithi037.stdout:Acquiring lock 139741258109840 on /run/cephadm/99b492c2-eda1-11ee-b647-cb9ed24678a4.lock 2024-03-29T07:55:12.836 INFO:teuthology.orchestra.run.smithi037.stdout:Lock 139741258109840 acquired on /run/cephadm/99b492c2-eda1-11ee-b647-cb9ed24678a4.lock 2024-03-29T07:55:12.836 INFO:teuthology.orchestra.run.smithi037.stdout:Verifying IP 172.21.15.37 port 3300 ... 2024-03-29T07:55:12.837 INFO:teuthology.orchestra.run.smithi037.stdout:Verifying IP 172.21.15.37 port 6789 ... 2024-03-29T07:55:12.837 INFO:teuthology.orchestra.run.smithi037.stdout:Base mon IP(s) is [172.21.15.37:3300, 172.21.15.37:6789], mon addrv is [v2:172.21.15.37:3300,v1:172.21.15.37:6789] 2024-03-29T07:55:12.845 INFO:teuthology.orchestra.run.smithi037.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.37 metric 100 2024-03-29T07:55:12.845 INFO:teuthology.orchestra.run.smithi037.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.37 metric 100 2024-03-29T07:55:12.852 INFO:teuthology.orchestra.run.smithi037.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-03-29T07:55:12.852 INFO:teuthology.orchestra.run.smithi037.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 256 pref medium 2024-03-29T07:55:12.852 INFO:teuthology.orchestra.run.smithi037.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 1024 expires 1749sec hoplimit 64 pref medium 2024-03-29T07:55:12.858 INFO:teuthology.orchestra.run.smithi037.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-03-29T07:55:12.858 INFO:teuthology.orchestra.run.smithi037.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-03-29T07:55:12.858 INFO:teuthology.orchestra.run.smithi037.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-03-29T07:55:12.858 INFO:teuthology.orchestra.run.smithi037.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-03-29T07:55:12.858 INFO:teuthology.orchestra.run.smithi037.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:fe88:7794/64 scope link 2024-03-29T07:55:12.859 INFO:teuthology.orchestra.run.smithi037.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-03-29T07:55:12.859 INFO:teuthology.orchestra.run.smithi037.stdout:Mon IP `172.21.15.37` is in CIDR network `172.21.0.0/20` 2024-03-29T07:55:12.859 INFO:teuthology.orchestra.run.smithi037.stdout:Mon IP `172.21.15.37` is in CIDR network `172.21.0.0/20` 2024-03-29T07:55:12.860 INFO:teuthology.orchestra.run.smithi037.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-03-29T07:55:12.860 INFO:teuthology.orchestra.run.smithi037.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-03-29T07:55:12.861 INFO:teuthology.orchestra.run.smithi037.stdout:Pulling container image quay.ceph.io/ceph-ci/ceph:quincy... 2024-03-29T07:55:13.345 INFO:teuthology.orchestra.run.smithi037.stdout:/bin/podman: stdout cf22eb8683be731b0447946652ee17d169ccd1820a07c94b7041aa2020906e1a 2024-03-29T07:55:13.345 INFO:teuthology.orchestra.run.smithi037.stdout:/bin/podman: stderr Trying to pull quay.ceph.io/ceph-ci/ceph:quincy... 2024-03-29T07:55:13.345 INFO:teuthology.orchestra.run.smithi037.stdout:/bin/podman: stderr Getting image source signatures 2024-03-29T07:55:13.345 INFO:teuthology.orchestra.run.smithi037.stdout:/bin/podman: stderr Copying blob sha256:7ae200e79e73f55e16b47791a08422a21d8b9f8d5bb767e6ae856e160a74c37a 2024-03-29T07:55:13.345 INFO:teuthology.orchestra.run.smithi037.stdout:/bin/podman: stderr Copying blob sha256:11da15500eca1f826c328eb3265caa30b0a3730510e2a32023084d6fe9cee20c 2024-03-29T07:55:13.345 INFO:teuthology.orchestra.run.smithi037.stdout:/bin/podman: stderr Copying config sha256:cf22eb8683be731b0447946652ee17d169ccd1820a07c94b7041aa2020906e1a 2024-03-29T07:55:13.345 INFO:teuthology.orchestra.run.smithi037.stdout:/bin/podman: stderr Writing manifest to image destination 2024-03-29T07:55:14.416 INFO:teuthology.orchestra.run.smithi037.stdout:ceph: stdout ceph version 17.2.7-789-g4213e17c (4213e17c4de67016ed14f55d22503f4923b3a901) quincy (stable) 2024-03-29T07:55:14.416 INFO:teuthology.orchestra.run.smithi037.stdout:Ceph version: ceph version 17.2.7-789-g4213e17c (4213e17c4de67016ed14f55d22503f4923b3a901) quincy (stable) 2024-03-29T07:55:14.416 INFO:teuthology.orchestra.run.smithi037.stdout:Extracting ceph user uid/gid from container image... 2024-03-29T07:55:15.427 INFO:teuthology.orchestra.run.smithi037.stdout:stat: stdout 167 167 2024-03-29T07:55:15.427 INFO:teuthology.orchestra.run.smithi037.stdout:Creating initial keys... 2024-03-29T07:55:16.576 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph-authtool: stdout AQDjcwZm4hqPMhAAOWoq+GzopY7OEWdCKuUvYg== 2024-03-29T07:55:17.706 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph-authtool: stdout AQDkcwZmUdLJOhAAJK7h9IuBkK/Ekxwl6PY9Xw== 2024-03-29T07:55:18.813 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph-authtool: stdout AQDmcwZmUFL/CBAADd4usK9/0RJ15UrKKbzboQ== 2024-03-29T07:55:18.815 INFO:teuthology.orchestra.run.smithi037.stdout:Creating initial monmap... 2024-03-29T07:55:19.952 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-03-29T07:55:19.952 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = octopus 2024-03-29T07:55:19.952 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 99b492c2-eda1-11ee-b647-cb9ed24678a4 2024-03-29T07:55:19.952 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-03-29T07:55:19.952 INFO:teuthology.orchestra.run.smithi037.stdout:monmaptool for smithi037 [v2:172.21.15.37:3300,v1:172.21.15.37:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-03-29T07:55:19.952 INFO:teuthology.orchestra.run.smithi037.stdout:setting min_mon_release = octopus 2024-03-29T07:55:19.952 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/monmaptool: set fsid to 99b492c2-eda1-11ee-b647-cb9ed24678a4 2024-03-29T07:55:19.953 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-03-29T07:55:19.953 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T07:55:19.953 INFO:teuthology.orchestra.run.smithi037.stdout:Creating mon... 2024-03-29T07:55:21.930 INFO:teuthology.orchestra.run.smithi037.stdout:create mon.smithi037 on 2024-03-29T07:55:22.141 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stderr Removed /etc/systemd/system/multi-user.target.wants/ceph.target. 2024-03-29T07:55:22.331 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-03-29T07:55:22.534 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4.target → /etc/systemd/system/ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4.target. 2024-03-29T07:55:22.534 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4.target → /etc/systemd/system/ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4.target. 2024-03-29T07:55:22.810 INFO:teuthology.orchestra.run.smithi037.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4@mon.smithi037 2024-03-29T07:55:22.810 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stderr Failed to reset failed state of unit ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4@mon.smithi037.service: Unit ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4@mon.smithi037.service not loaded. 2024-03-29T07:55:23.010 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4.target.wants/ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4@mon.smithi037.service → /etc/systemd/system/ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4@.service. 2024-03-29T07:55:23.103 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:23 smithi037 systemd[1]: Starting Ceph mon.smithi037 for 99b492c2-eda1-11ee-b647-cb9ed24678a4... 2024-03-29T07:55:23.720 INFO:teuthology.orchestra.run.smithi037.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-29T07:55:23.720 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stdout disabled 2024-03-29T07:55:23.737 INFO:teuthology.orchestra.run.smithi037.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-29T07:55:23.737 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stdout inactive 2024-03-29T07:55:23.737 INFO:teuthology.orchestra.run.smithi037.stdout:firewalld.service is not enabled 2024-03-29T07:55:23.737 INFO:teuthology.orchestra.run.smithi037.stdout:Not possible to enable service . firewalld.service is not available 2024-03-29T07:55:23.738 INFO:teuthology.orchestra.run.smithi037.stdout:Waiting for mon to start... 2024-03-29T07:55:23.738 INFO:teuthology.orchestra.run.smithi037.stdout:Waiting for mon... 2024-03-29T07:55:23.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:23 smithi037 ceph-mon[34677]: mkfs 99b492c2-eda1-11ee-b647-cb9ed24678a4 2024-03-29T07:55:23.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:23 smithi037 ceph-mon[34677]: mon.smithi037 is new leader, mons smithi037 in quorum (ranks 0) 2024-03-29T07:55:23.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:23 smithi037 bash[34639]: 40554e7a8c85fa45510fdd2971ff5adb74494391bf8aaf585c791774efca19a7 2024-03-29T07:55:23.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:23 smithi037 systemd[1]: Started Ceph mon.smithi037 for 99b492c2-eda1-11ee-b647-cb9ed24678a4. 2024-03-29T07:55:24.886 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout cluster: 2024-03-29T07:55:24.886 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout id: 99b492c2-eda1-11ee-b647-cb9ed24678a4 2024-03-29T07:55:24.886 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-03-29T07:55:24.886 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout 2024-03-29T07:55:24.886 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout services: 2024-03-29T07:55:24.886 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum smithi037 (age 0.690684s) 2024-03-29T07:55:24.886 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-03-29T07:55:24.886 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-03-29T07:55:24.887 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout 2024-03-29T07:55:24.887 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout data: 2024-03-29T07:55:24.887 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-03-29T07:55:24.887 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-03-29T07:55:24.887 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-03-29T07:55:24.887 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout pgs: 2024-03-29T07:55:24.887 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout 2024-03-29T07:55:24.887 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:24.276+0000 7f1758600700 1 Processor -- start 2024-03-29T07:55:24.887 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:24.276+0000 7f1758600700 1 -- start start 2024-03-29T07:55:24.887 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:24.277+0000 7f1758600700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1750078d30 0x7f1750077240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T07:55:24.887 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:24.277+0000 7f1758600700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1750079140 con 0x7f1750078d30 2024-03-29T07:55:24.887 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:24.277+0000 7f1755c00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1750078d30 0x7f1750077240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T07:55:24.887 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:24.278+0000 7f1755c00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1750078d30 0x7f1750077240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59034/0 (socket says 172.21.15.37:59034) 2024-03-29T07:55:24.887 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:24.278+0000 7f1755c00700 1 -- 172.21.15.37:0/1936145838 learned_addr learned my addr 172.21.15.37:0/1936145838 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T07:55:24.887 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:24.278+0000 7f1755c00700 1 -- 172.21.15.37:0/1936145838 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1750077780 con 0x7f1750078d30 2024-03-29T07:55:24.888 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:24.278+0000 7f1755c00700 1 --2- 172.21.15.37:0/1936145838 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1750078d30 0x7f1750077240 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f17440098e0 tx=0x7f1744009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b015078aa8f2cf57 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T07:55:24.888 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:24.278+0000 7f174fe00700 1 -- 172.21.15.37:0/1936145838 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f174400b860 con 0x7f1750078d30 2024-03-29T07:55:24.888 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:24.279+0000 7f174fe00700 1 -- 172.21.15.37:0/1936145838 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f174400b9c0 con 0x7f1750078d30 2024-03-29T07:55:24.888 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:24.279+0000 7f174fe00700 1 -- 172.21.15.37:0/1936145838 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f174400bc90 con 0x7f1750078d30 2024-03-29T07:55:24.888 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:24.279+0000 7f1758600700 1 -- 172.21.15.37:0/1936145838 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1750078d30 msgr2=0x7f1750077240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T07:55:24.888 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:24.279+0000 7f1758600700 1 --2- 172.21.15.37:0/1936145838 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1750078d30 0x7f1750077240 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f17440098e0 tx=0x7f1744009250 comp rx=0 tx=0).stop 2024-03-29T07:55:24.888 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:24.279+0000 7f1758600700 1 -- 172.21.15.37:0/1936145838 shutdown_connections 2024-03-29T07:55:24.888 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:24.279+0000 7f1758600700 1 --2- 172.21.15.37:0/1936145838 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1750078d30 0x7f1750077240 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T07:55:24.888 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:24.279+0000 7f1758600700 1 -- 172.21.15.37:0/1936145838 >> 172.21.15.37:0/1936145838 conn(0x7f17500fdd00 msgr2=0x7f1750100110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T07:55:24.888 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:24.279+0000 7f1758600700 1 -- 172.21.15.37:0/1936145838 shutdown_connections 2024-03-29T07:55:24.888 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:24.279+0000 7f1758600700 1 -- 172.21.15.37:0/1936145838 wait complete. 2024-03-29T07:55:24.888 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:24.280+0000 7f1758600700 1 Processor -- start 2024-03-29T07:55:24.888 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:24.280+0000 7f1758600700 1 -- start start 2024-03-29T07:55:24.888 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:24.281+0000 7f1758600700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1750078d30 0x7f1750112e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T07:55:24.888 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:24.281+0000 7f1758600700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1750113380 con 0x7f1750078d30 2024-03-29T07:55:24.889 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:24.281+0000 7f1755c00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1750078d30 0x7f1750112e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T07:55:24.889 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:24.281+0000 7f1755c00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1750078d30 0x7f1750112e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59038/0 (socket says 172.21.15.37:59038) 2024-03-29T07:55:24.889 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:24.281+0000 7f1755c00700 1 -- 172.21.15.37:0/1729931417 learned_addr learned my addr 172.21.15.37:0/1729931417 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T07:55:24.889 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:24.281+0000 7f1755c00700 1 -- 172.21.15.37:0/1729931417 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17440094f0 con 0x7f1750078d30 2024-03-29T07:55:24.889 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:24.282+0000 7f1755c00700 1 --2- 172.21.15.37:0/1729931417 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1750078d30 0x7f1750112e40 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f1744009290 tx=0x7f1744016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T07:55:24.889 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:24.282+0000 7f174d600700 1 -- 172.21.15.37:0/1729931417 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f174401c5c0 con 0x7f1750078d30 2024-03-29T07:55:24.889 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:24.282+0000 7f174d600700 1 -- 172.21.15.37:0/1729931417 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f174401c740 con 0x7f1750078d30 2024-03-29T07:55:24.889 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:24.282+0000 7f1758600700 1 -- 172.21.15.37:0/1729931417 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1750113580 con 0x7f1750078d30 2024-03-29T07:55:24.889 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:24.282+0000 7f174d600700 1 -- 172.21.15.37:0/1729931417 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f174401ca50 con 0x7f1750078d30 2024-03-29T07:55:24.889 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:24.282+0000 7f1758600700 1 -- 172.21.15.37:0/1729931417 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1750113fb0 con 0x7f1750078d30 2024-03-29T07:55:24.889 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:24.282+0000 7f174d600700 1 -- 172.21.15.37:0/1729931417 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f174401a450 con 0x7f1750078d30 2024-03-29T07:55:24.889 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:24.282+0000 7f174d600700 1 -- 172.21.15.37:0/1729931417 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f174402e790 con 0x7f1750078d30 2024-03-29T07:55:24.889 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:24.283+0000 7f1758600700 1 -- 172.21.15.37:0/1729931417 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f175005f5f0 con 0x7f1750078d30 2024-03-29T07:55:24.889 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:24.286+0000 7f174d600700 1 -- 172.21.15.37:0/1729931417 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f174401ce30 con 0x7f1750078d30 2024-03-29T07:55:24.889 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:24.361+0000 7f1758600700 1 -- 172.21.15.37:0/1729931417 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7f1750114b20 con 0x7f1750078d30 2024-03-29T07:55:24.890 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:24.361+0000 7f174d600700 1 -- 172.21.15.37:0/1729931417 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+325 (secure 0 0 0) 0x7f174401a890 con 0x7f1750078d30 2024-03-29T07:55:24.890 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:24.362+0000 7f1758600700 1 -- 172.21.15.37:0/1729931417 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1750078d30 msgr2=0x7f1750112e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T07:55:24.890 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:24.362+0000 7f1758600700 1 --2- 172.21.15.37:0/1729931417 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1750078d30 0x7f1750112e40 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f1744009290 tx=0x7f1744016000 comp rx=0 tx=0).stop 2024-03-29T07:55:24.890 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:24.362+0000 7f1758600700 1 -- 172.21.15.37:0/1729931417 shutdown_connections 2024-03-29T07:55:24.890 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:24.362+0000 7f1758600700 1 --2- 172.21.15.37:0/1729931417 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1750078d30 0x7f1750112e40 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T07:55:24.890 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:24.362+0000 7f1758600700 1 -- 172.21.15.37:0/1729931417 >> 172.21.15.37:0/1729931417 conn(0x7f17500fdd00 msgr2=0x7f1750109bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T07:55:24.890 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:24.362+0000 7f1758600700 1 -- 172.21.15.37:0/1729931417 shutdown_connections 2024-03-29T07:55:24.890 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:24.362+0000 7f1758600700 1 -- 172.21.15.37:0/1729931417 wait complete. 2024-03-29T07:55:24.890 INFO:teuthology.orchestra.run.smithi037.stdout:mon is available 2024-03-29T07:55:24.890 INFO:teuthology.orchestra.run.smithi037.stdout:Assimilating anything we can from ceph.conf... 2024-03-29T07:55:24.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:24 smithi037 ceph-mon[34677]: mon.smithi037 is new leader, mons smithi037 in quorum (ranks 0) 2024-03-29T07:55:24.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:24 smithi037 ceph-mon[34677]: monmap e1: 1 mons at {smithi037=[v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-29T07:55:24.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:24 smithi037 ceph-mon[34677]: fsmap 2024-03-29T07:55:24.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:24 smithi037 ceph-mon[34677]: osdmap e1: 0 total, 0 up, 0 in 2024-03-29T07:55:24.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:24 smithi037 ceph-mon[34677]: mgrmap e1: no daemons active 2024-03-29T07:55:24.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:24 smithi037 ceph-mon[34677]: from='client.? 172.21.15.37:0/1729931417' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-03-29T07:55:25.905 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:25 smithi037 ceph-mon[34677]: from='client.? 172.21.15.37:0/4035579610' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-03-29T07:55:25.905 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:25 smithi037 ceph-mon[34677]: from='client.? 172.21.15.37:0/4035579610' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-03-29T07:55:26.192 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout 2024-03-29T07:55:26.193 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout [global] 2024-03-29T07:55:26.193 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout fsid = 99b492c2-eda1-11ee-b647-cb9ed24678a4 2024-03-29T07:55:26.193 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.37:3300,v1:172.21.15.37:6789] 2024-03-29T07:55:26.193 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-03-29T07:55:26.193 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-03-29T07:55:26.193 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-03-29T07:55:26.193 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-03-29T07:55:26.193 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout 2024-03-29T07:55:26.193 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout [mgr] 2024-03-29T07:55:26.193 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-03-29T07:55:26.193 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout 2024-03-29T07:55:26.193 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout [osd] 2024-03-29T07:55:26.193 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-03-29T07:55:26.194 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-03-29T07:55:26.194 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-03-29T07:55:26.194 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-03-29T07:55:26.194 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-03-29T07:55:26.194 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:25.457+0000 7ff692800700 1 Processor -- start 2024-03-29T07:55:26.194 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:25.457+0000 7ff692800700 1 -- start start 2024-03-29T07:55:26.194 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:25.458+0000 7ff692800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff68c078d50 0x7ff68c077260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T07:55:26.194 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:25.458+0000 7ff692800700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff68c079160 con 0x7ff68c078d50 2024-03-29T07:55:26.194 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:25.458+0000 7ff68b400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff68c078d50 0x7ff68c077260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T07:55:26.194 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:25.458+0000 7ff68b400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff68c078d50 0x7ff68c077260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59040/0 (socket says 172.21.15.37:59040) 2024-03-29T07:55:26.194 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:25.458+0000 7ff68b400700 1 -- 172.21.15.37:0/3901231298 learned_addr learned my addr 172.21.15.37:0/3901231298 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T07:55:26.194 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:25.459+0000 7ff68b400700 1 -- 172.21.15.37:0/3901231298 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff68c0777a0 con 0x7ff68c078d50 2024-03-29T07:55:26.195 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:25.459+0000 7ff68b400700 1 --2- 172.21.15.37:0/3901231298 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff68c078d50 0x7ff68c077260 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7ff67c0098e0 tx=0x7ff67c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14fccc83edbb0156 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T07:55:26.195 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:25.459+0000 7ff68a000700 1 -- 172.21.15.37:0/3901231298 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff67c00b860 con 0x7ff68c078d50 2024-03-29T07:55:26.195 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:25.459+0000 7ff68a000700 1 -- 172.21.15.37:0/3901231298 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff67c00b9c0 con 0x7ff68c078d50 2024-03-29T07:55:26.195 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:25.459+0000 7ff68a000700 1 -- 172.21.15.37:0/3901231298 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff67c00bc90 con 0x7ff68c078d50 2024-03-29T07:55:26.195 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:25.460+0000 7ff692800700 1 -- 172.21.15.37:0/3901231298 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff68c078d50 msgr2=0x7ff68c077260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T07:55:26.195 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:25.460+0000 7ff692800700 1 --2- 172.21.15.37:0/3901231298 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff68c078d50 0x7ff68c077260 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7ff67c0098e0 tx=0x7ff67c009250 comp rx=0 tx=0).stop 2024-03-29T07:55:26.195 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:25.460+0000 7ff692800700 1 -- 172.21.15.37:0/3901231298 shutdown_connections 2024-03-29T07:55:26.195 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:25.460+0000 7ff692800700 1 --2- 172.21.15.37:0/3901231298 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff68c078d50 0x7ff68c077260 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T07:55:26.195 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:25.460+0000 7ff692800700 1 -- 172.21.15.37:0/3901231298 >> 172.21.15.37:0/3901231298 conn(0x7ff68c0fdd00 msgr2=0x7ff68c100110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T07:55:26.195 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:25.460+0000 7ff692800700 1 -- 172.21.15.37:0/3901231298 shutdown_connections 2024-03-29T07:55:26.195 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:25.460+0000 7ff692800700 1 -- 172.21.15.37:0/3901231298 wait complete. 2024-03-29T07:55:26.195 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:25.461+0000 7ff692800700 1 Processor -- start 2024-03-29T07:55:26.195 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:25.461+0000 7ff692800700 1 -- start start 2024-03-29T07:55:26.195 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:25.461+0000 7ff692800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff68c078d50 0x7ff68c112e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T07:55:26.196 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:25.461+0000 7ff692800700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff68c113360 con 0x7ff68c078d50 2024-03-29T07:55:26.196 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:25.462+0000 7ff68b400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff68c078d50 0x7ff68c112e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T07:55:26.196 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:25.462+0000 7ff68b400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff68c078d50 0x7ff68c112e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59050/0 (socket says 172.21.15.37:59050) 2024-03-29T07:55:26.196 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:25.462+0000 7ff68b400700 1 -- 172.21.15.37:0/4035579610 learned_addr learned my addr 172.21.15.37:0/4035579610 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T07:55:26.196 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:25.462+0000 7ff68b400700 1 -- 172.21.15.37:0/4035579610 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff67c0094f0 con 0x7ff68c078d50 2024-03-29T07:55:26.196 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:25.463+0000 7ff68b400700 1 --2- 172.21.15.37:0/4035579610 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff68c078d50 0x7ff68c112e20 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7ff67c009290 tx=0x7ff67c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T07:55:26.196 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:25.463+0000 7ff683400700 1 -- 172.21.15.37:0/4035579610 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff67c01c5c0 con 0x7ff68c078d50 2024-03-29T07:55:26.196 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:25.463+0000 7ff683400700 1 -- 172.21.15.37:0/4035579610 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff67c01c740 con 0x7ff68c078d50 2024-03-29T07:55:26.196 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:25.463+0000 7ff692800700 1 -- 172.21.15.37:0/4035579610 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff68c113560 con 0x7ff68c078d50 2024-03-29T07:55:26.196 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:25.463+0000 7ff683400700 1 -- 172.21.15.37:0/4035579610 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff67c01ca50 con 0x7ff68c078d50 2024-03-29T07:55:26.197 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:25.463+0000 7ff692800700 1 -- 172.21.15.37:0/4035579610 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff68c113f90 con 0x7ff68c078d50 2024-03-29T07:55:26.197 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:25.463+0000 7ff683400700 1 -- 172.21.15.37:0/4035579610 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7ff67c01a450 con 0x7ff68c078d50 2024-03-29T07:55:26.197 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:25.463+0000 7ff683400700 1 -- 172.21.15.37:0/4035579610 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7ff67c02e790 con 0x7ff68c078d50 2024-03-29T07:55:26.197 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:25.464+0000 7ff692800700 1 -- 172.21.15.37:0/4035579610 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff68c05f5f0 con 0x7ff68c078d50 2024-03-29T07:55:26.197 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:25.467+0000 7ff683400700 1 -- 172.21.15.37:0/4035579610 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7ff67c01ce30 con 0x7ff68c078d50 2024-03-29T07:55:26.197 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:25.539+0000 7ff692800700 1 -- 172.21.15.37:0/4035579610 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7ff68c10ca10 con 0x7ff68c078d50 2024-03-29T07:55:26.197 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:25.544+0000 7ff683400700 1 -- 172.21.15.37:0/4035579610 <== mon.0 v2:172.21.15.37:3300/0 7 ==== config(26 keys) v1 ==== 1009+0+0 (secure 0 0 0) 0x7ff67c02e950 con 0x7ff68c078d50 2024-03-29T07:55:26.197 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:25.544+0000 7ff683400700 1 -- 172.21.15.37:0/4035579610 <== mon.0 v2:172.21.15.37:3300/0 8 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) v1 ==== 70+0+429 (secure 0 0 0) 0x7ff67c01b630 con 0x7ff68c078d50 2024-03-29T07:55:26.197 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:25.545+0000 7ff692800700 1 -- 172.21.15.37:0/4035579610 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff68c078d50 msgr2=0x7ff68c112e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T07:55:26.197 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:25.545+0000 7ff692800700 1 --2- 172.21.15.37:0/4035579610 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff68c078d50 0x7ff68c112e20 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7ff67c009290 tx=0x7ff67c016000 comp rx=0 tx=0).stop 2024-03-29T07:55:26.197 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:25.545+0000 7ff692800700 1 -- 172.21.15.37:0/4035579610 shutdown_connections 2024-03-29T07:55:26.197 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:25.545+0000 7ff692800700 1 --2- 172.21.15.37:0/4035579610 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff68c078d50 0x7ff68c112e20 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T07:55:26.197 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:25.545+0000 7ff692800700 1 -- 172.21.15.37:0/4035579610 >> 172.21.15.37:0/4035579610 conn(0x7ff68c0fdd00 msgr2=0x7ff68c109b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T07:55:26.198 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:25.545+0000 7ff692800700 1 -- 172.21.15.37:0/4035579610 shutdown_connections 2024-03-29T07:55:26.198 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:25.545+0000 7ff692800700 1 -- 172.21.15.37:0/4035579610 wait complete. 2024-03-29T07:55:26.198 INFO:teuthology.orchestra.run.smithi037.stdout:Generating new minimal ceph.conf... 2024-03-29T07:55:27.878 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:27 smithi037 ceph-mon[34677]: from='client.? 172.21.15.37:0/1614146120' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T07:55:28.108 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:27.464+0000 7f806d400700 1 Processor -- start 2024-03-29T07:55:28.113 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:27.464+0000 7f806d400700 1 -- start start 2024-03-29T07:55:28.113 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:27.464+0000 7f806d400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8068078d50 0x7f8068077260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T07:55:28.113 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:27.464+0000 7f806d400700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8068079160 con 0x7f8068078d50 2024-03-29T07:55:28.113 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:27.465+0000 7f8066800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8068078d50 0x7f8068077260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T07:55:28.113 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:27.465+0000 7f8066800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8068078d50 0x7f8068077260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59054/0 (socket says 172.21.15.37:59054) 2024-03-29T07:55:28.113 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:27.465+0000 7f8066800700 1 -- 172.21.15.37:0/1899993817 learned_addr learned my addr 172.21.15.37:0/1899993817 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T07:55:28.113 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:27.465+0000 7f8066800700 1 -- 172.21.15.37:0/1899993817 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80680777a0 con 0x7f8068078d50 2024-03-29T07:55:28.113 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:27.465+0000 7f8066800700 1 --2- 172.21.15.37:0/1899993817 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8068078d50 0x7f8068077260 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f80580098e0 tx=0x7f8058009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcd0b2dedfdfa6ec server_cookie=0 in_seq=0 out_seq=0 2024-03-29T07:55:28.114 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:27.465+0000 7f8065400700 1 -- 172.21.15.37:0/1899993817 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f805800b860 con 0x7f8068078d50 2024-03-29T07:55:28.114 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:27.466+0000 7f8065400700 1 -- 172.21.15.37:0/1899993817 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1009+0+0 (secure 0 0 0) 0x7f805800be30 con 0x7f8068078d50 2024-03-29T07:55:28.114 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:27.466+0000 7f8065400700 1 -- 172.21.15.37:0/1899993817 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8058010380 con 0x7f8068078d50 2024-03-29T07:55:28.114 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:27.466+0000 7f806d400700 1 -- 172.21.15.37:0/1899993817 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8068078d50 msgr2=0x7f8068077260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T07:55:28.114 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:27.466+0000 7f806d400700 1 --2- 172.21.15.37:0/1899993817 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8068078d50 0x7f8068077260 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f80580098e0 tx=0x7f8058009250 comp rx=0 tx=0).stop 2024-03-29T07:55:28.114 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:27.466+0000 7f806d400700 1 -- 172.21.15.37:0/1899993817 shutdown_connections 2024-03-29T07:55:28.114 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:27.466+0000 7f806d400700 1 --2- 172.21.15.37:0/1899993817 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8068078d50 0x7f8068077260 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T07:55:28.114 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:27.466+0000 7f806d400700 1 -- 172.21.15.37:0/1899993817 >> 172.21.15.37:0/1899993817 conn(0x7f80680fdd00 msgr2=0x7f8068100110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T07:55:28.114 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:27.466+0000 7f806d400700 1 -- 172.21.15.37:0/1899993817 shutdown_connections 2024-03-29T07:55:28.114 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:27.466+0000 7f806d400700 1 -- 172.21.15.37:0/1899993817 wait complete. 2024-03-29T07:55:28.114 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:27.467+0000 7f806d400700 1 Processor -- start 2024-03-29T07:55:28.114 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:27.467+0000 7f806d400700 1 -- start start 2024-03-29T07:55:28.114 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:27.468+0000 7f806d400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8068078d50 0x7f80681135a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T07:55:28.115 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:27.468+0000 7f806d400700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8068113ae0 con 0x7f8068078d50 2024-03-29T07:55:28.115 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:27.468+0000 7f8066800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8068078d50 0x7f80681135a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T07:55:28.115 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:27.468+0000 7f8066800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8068078d50 0x7f80681135a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59068/0 (socket says 172.21.15.37:59068) 2024-03-29T07:55:28.115 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:27.468+0000 7f8066800700 1 -- 172.21.15.37:0/1614146120 learned_addr learned my addr 172.21.15.37:0/1614146120 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T07:55:28.115 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:27.468+0000 7f8066800700 1 -- 172.21.15.37:0/1614146120 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80580094f0 con 0x7f8068078d50 2024-03-29T07:55:28.115 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:27.468+0000 7f8066800700 1 --2- 172.21.15.37:0/1614146120 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8068078d50 0x7f80681135a0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f8058006fd0 tx=0x7f8058009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T07:55:28.115 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:27.469+0000 7f805ea00700 1 -- 172.21.15.37:0/1614146120 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8058010750 con 0x7f8068078d50 2024-03-29T07:55:28.115 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:27.469+0000 7f805ea00700 1 -- 172.21.15.37:0/1614146120 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1009+0+0 (secure 0 0 0) 0x7f8058010d20 con 0x7f8068078d50 2024-03-29T07:55:28.115 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:27.469+0000 7f805ea00700 1 -- 172.21.15.37:0/1614146120 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f805801ada0 con 0x7f8068078d50 2024-03-29T07:55:28.115 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:27.469+0000 7f806d400700 1 -- 172.21.15.37:0/1614146120 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8068113ce0 con 0x7f8068078d50 2024-03-29T07:55:28.115 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:27.469+0000 7f806d400700 1 -- 172.21.15.37:0/1614146120 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8068114850 con 0x7f8068078d50 2024-03-29T07:55:28.115 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:27.469+0000 7f805ea00700 1 -- 172.21.15.37:0/1614146120 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f8058024680 con 0x7f8068078d50 2024-03-29T07:55:28.115 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:27.469+0000 7f805ea00700 1 -- 172.21.15.37:0/1614146120 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f8058019850 con 0x7f8068078d50 2024-03-29T07:55:28.116 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:27.470+0000 7f806d400700 1 -- 172.21.15.37:0/1614146120 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f806805f5f0 con 0x7f8068078d50 2024-03-29T07:55:28.116 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:27.473+0000 7f805ea00700 1 -- 172.21.15.37:0/1614146120 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f8058029060 con 0x7f8068078d50 2024-03-29T07:55:28.116 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:27.545+0000 7f806d400700 1 -- 172.21.15.37:0/1614146120 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f80681087f0 con 0x7f8068078d50 2024-03-29T07:55:28.116 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:27.546+0000 7f805ea00700 1 -- 172.21.15.37:0/1614146120 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) v1 ==== 76+0+175 (secure 0 0 0) 0x7f8058020070 con 0x7f8068078d50 2024-03-29T07:55:28.116 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:27.547+0000 7f806d400700 1 -- 172.21.15.37:0/1614146120 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8068078d50 msgr2=0x7f80681135a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T07:55:28.116 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:27.547+0000 7f806d400700 1 --2- 172.21.15.37:0/1614146120 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8068078d50 0x7f80681135a0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f8058006fd0 tx=0x7f8058009250 comp rx=0 tx=0).stop 2024-03-29T07:55:28.116 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:27.547+0000 7f806d400700 1 -- 172.21.15.37:0/1614146120 shutdown_connections 2024-03-29T07:55:28.116 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:27.547+0000 7f806d400700 1 --2- 172.21.15.37:0/1614146120 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8068078d50 0x7f80681135a0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T07:55:28.116 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:27.547+0000 7f806d400700 1 -- 172.21.15.37:0/1614146120 >> 172.21.15.37:0/1614146120 conn(0x7f80680fdd00 msgr2=0x7f80680fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T07:55:28.116 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:27.547+0000 7f806d400700 1 -- 172.21.15.37:0/1614146120 shutdown_connections 2024-03-29T07:55:28.116 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:27.547+0000 7f806d400700 1 -- 172.21.15.37:0/1614146120 wait complete. 2024-03-29T07:55:28.116 INFO:teuthology.orchestra.run.smithi037.stdout:Restarting the monitor... 2024-03-29T07:55:28.426 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:28 smithi037 systemd[1]: Stopping Ceph mon.smithi037 for 99b492c2-eda1-11ee-b647-cb9ed24678a4... 2024-03-29T07:55:28.782 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:28 smithi037 ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4-mon-smithi037[34654]: 2024-03-29T07:55:28.438+0000 7f6ec0e00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi037 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-29T07:55:28.782 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:28 smithi037 ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4-mon-smithi037[34654]: 2024-03-29T07:55:28.438+0000 7f6ec0e00700 -1 mon.smithi037@0(leader) e1 *** Got Signal Terminated *** 2024-03-29T07:55:29.052 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:28 smithi037 bash[35005]: ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4-mon-smithi037 2024-03-29T07:55:29.426 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 systemd[1]: ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4@mon.smithi037.service: Succeeded. 2024-03-29T07:55:29.426 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 systemd[1]: Stopped Ceph mon.smithi037 for 99b492c2-eda1-11ee-b647-cb9ed24678a4. 2024-03-29T07:55:29.426 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 systemd[1]: Starting Ceph mon.smithi037 for 99b492c2-eda1-11ee-b647-cb9ed24678a4... 2024-03-29T07:55:29.854 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: set uid:gid to 167:167 (ceph:ceph) 2024-03-29T07:55:29.901 INFO:teuthology.orchestra.run.smithi037.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-03-29T07:55:30.129 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: ceph version 17.2.7-789-g4213e17c (4213e17c4de67016ed14f55d22503f4923b3a901) quincy (stable), process ceph-mon, pid 7 2024-03-29T07:55:30.129 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: pidfile_write: ignore empty --pid-file 2024-03-29T07:55:30.129 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: load: jerasure load: lrc 2024-03-29T07:55:30.129 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: rocksdb: RocksDB version: 6.15.5 2024-03-29T07:55:30.130 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: rocksdb: Git sha rocksdb_build_git_sha:@0@ 2024-03-29T07:55:30.130 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: rocksdb: Compile date Mar 28 2024 2024-03-29T07:55:30.130 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: rocksdb: DB SUMMARY 2024-03-29T07:55:30.130 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: rocksdb: DB Session ID: Y7VLBQ0H6A4595UKCSLW 2024-03-29T07:55:30.130 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: rocksdb: CURRENT file: CURRENT 2024-03-29T07:55:30.130 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: rocksdb: IDENTITY file: IDENTITY 2024-03-29T07:55:30.130 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: rocksdb: MANIFEST file: MANIFEST-000009 size: 131 Bytes 2024-03-29T07:55:30.130 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi037/store.db dir, Total Num: 1, files: 000008.sst 2024-03-29T07:55:30.130 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi037/store.db: 000010.log size: 96760 ; 2024-03-29T07:55:30.130 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: rocksdb: Options.error_if_exists: 0 2024-03-29T07:55:30.131 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: rocksdb: Options.create_if_missing: 0 2024-03-29T07:55:30.131 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: rocksdb: Options.paranoid_checks: 1 2024-03-29T07:55:30.131 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-03-29T07:55:30.131 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: rocksdb: Options.env: 0x55f6f40c9a80 2024-03-29T07:55:30.131 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: rocksdb: Options.fs: Posix File System 2024-03-29T07:55:30.131 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: rocksdb: Options.info_log: 0x55f6f51bbc00 2024-03-29T07:55:30.131 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: rocksdb: Options.max_file_opening_threads: 16 2024-03-29T07:55:30.131 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: rocksdb: Options.statistics: (nil) 2024-03-29T07:55:30.131 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: rocksdb: Options.use_fsync: 0 2024-03-29T07:55:30.131 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: rocksdb: Options.max_log_file_size: 0 2024-03-29T07:55:30.132 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-03-29T07:55:30.132 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: rocksdb: Options.log_file_time_to_roll: 0 2024-03-29T07:55:30.132 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: rocksdb: Options.keep_log_file_num: 1000 2024-03-29T07:55:30.132 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: rocksdb: Options.recycle_log_file_num: 0 2024-03-29T07:55:30.132 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: rocksdb: Options.allow_fallocate: 1 2024-03-29T07:55:30.132 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: rocksdb: Options.allow_mmap_reads: 0 2024-03-29T07:55:30.132 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: rocksdb: Options.allow_mmap_writes: 0 2024-03-29T07:55:30.132 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: rocksdb: Options.use_direct_reads: 0 2024-03-29T07:55:30.132 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-03-29T07:55:30.132 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: rocksdb: Options.create_missing_column_families: 0 2024-03-29T07:55:30.132 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: rocksdb: Options.db_log_dir: 2024-03-29T07:55:30.132 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: rocksdb: Options.wal_dir: /var/lib/ceph/mon/ceph-smithi037/store.db 2024-03-29T07:55:30.133 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: rocksdb: Options.table_cache_numshardbits: 6 2024-03-29T07:55:30.133 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: rocksdb: Options.WAL_ttl_seconds: 0 2024-03-29T07:55:30.133 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: rocksdb: Options.WAL_size_limit_MB: 0 2024-03-29T07:55:30.133 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-03-29T07:55:30.133 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-03-29T07:55:30.133 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: rocksdb: Options.is_fd_close_on_exec: 1 2024-03-29T07:55:30.133 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: rocksdb: Options.advise_random_on_open: 1 2024-03-29T07:55:30.133 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: rocksdb: Options.db_write_buffer_size: 0 2024-03-29T07:55:30.133 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: rocksdb: Options.write_buffer_manager: 0x55f6f5f309f0 2024-03-29T07:55:30.133 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-03-29T07:55:30.133 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: rocksdb: Options.new_table_reader_for_compaction_inputs: 0 2024-03-29T07:55:30.134 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-03-29T07:55:30.134 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: rocksdb: Options.use_adaptive_mutex: 0 2024-03-29T07:55:30.134 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: rocksdb: Options.rate_limiter: (nil) 2024-03-29T07:55:30.134 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-03-29T07:55:30.134 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: rocksdb: Options.wal_recovery_mode: 2 2024-03-29T07:55:30.134 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: rocksdb: Options.enable_thread_tracking: 0 2024-03-29T07:55:30.134 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: rocksdb: Options.enable_pipelined_write: 0 2024-03-29T07:55:30.134 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: rocksdb: Options.unordered_write: 0 2024-03-29T07:55:30.134 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-03-29T07:55:30.134 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-03-29T07:55:30.134 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-03-29T07:55:30.134 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-03-29T07:55:30.135 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: rocksdb: Options.row_cache: None 2024-03-29T07:55:30.135 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: rocksdb: Options.wal_filter: None 2024-03-29T07:55:30.135 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-03-29T07:55:30.135 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: rocksdb: Options.allow_ingest_behind: 0 2024-03-29T07:55:30.135 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: rocksdb: Options.preserve_deletes: 0 2024-03-29T07:55:30.135 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: rocksdb: Options.two_write_queues: 0 2024-03-29T07:55:30.135 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: rocksdb: Options.manual_wal_flush: 0 2024-03-29T07:55:30.135 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: rocksdb: Options.atomic_flush: 0 2024-03-29T07:55:30.135 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-03-29T07:55:30.135 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: rocksdb: Options.persist_stats_to_disk: 0 2024-03-29T07:55:30.135 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: rocksdb: Options.write_dbid_to_manifest: 0 2024-03-29T07:55:30.136 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: rocksdb: Options.log_readahead_size: 0 2024-03-29T07:55:30.136 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-03-29T07:55:30.136 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: rocksdb: Options.best_efforts_recovery: 0 2024-03-29T07:55:30.136 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-03-29T07:55:30.136 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-03-29T07:55:30.136 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: rocksdb: Options.allow_data_in_errors: 0 2024-03-29T07:55:30.136 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: rocksdb: Options.db_host_id: __hostname__ 2024-03-29T07:55:30.136 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: rocksdb: Options.max_background_jobs: 2 2024-03-29T07:55:30.136 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: rocksdb: Options.max_background_compactions: -1 2024-03-29T07:55:30.137 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: rocksdb: Options.max_subcompactions: 1 2024-03-29T07:55:30.138 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-03-29T07:55:30.138 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-03-29T07:55:30.138 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: rocksdb: Options.delayed_write_rate : 16777216 2024-03-29T07:55:30.138 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: rocksdb: Options.max_total_wal_size: 0 2024-03-29T07:55:30.138 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-03-29T07:55:30.138 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: rocksdb: Options.stats_dump_period_sec: 600 2024-03-29T07:55:30.138 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: rocksdb: Options.stats_persist_period_sec: 600 2024-03-29T07:55:30.138 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-03-29T07:55:30.138 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: rocksdb: Options.max_open_files: -1 2024-03-29T07:55:30.139 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: rocksdb: Options.bytes_per_sync: 0 2024-03-29T07:55:30.139 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: rocksdb: Options.wal_bytes_per_sync: 0 2024-03-29T07:55:30.139 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: rocksdb: Options.strict_bytes_per_sync: 0 2024-03-29T07:55:30.139 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: rocksdb: Options.compaction_readahead_size: 0 2024-03-29T07:55:30.139 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: rocksdb: Options.max_background_flushes: -1 2024-03-29T07:55:30.139 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: rocksdb: Compression algorithms supported: 2024-03-29T07:55:30.139 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-03-29T07:55:30.139 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: rocksdb: kZSTD supported: 0 2024-03-29T07:55:30.139 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: rocksdb: kXpressCompression supported: 0 2024-03-29T07:55:30.139 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: rocksdb: kLZ4HCCompression supported: 1 2024-03-29T07:55:30.139 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: rocksdb: kZlibCompression supported: 1 2024-03-29T07:55:30.139 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: rocksdb: kSnappyCompression supported: 1 2024-03-29T07:55:30.140 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: rocksdb: kLZ4Compression supported: 1 2024-03-29T07:55:30.140 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: rocksdb: kBZip2Compression supported: 0 2024-03-29T07:55:30.140 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: rocksdb: Fast CRC32 supported: Supported on x86 2024-03-29T07:55:30.140 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: rocksdb: [db/version_set.cc:4724] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi037/store.db/MANIFEST-000009 2024-03-29T07:55:30.140 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: rocksdb: [db/column_family.cc:595] --------------- Options for column family [default]: 2024-03-29T07:55:30.140 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-03-29T07:55:30.140 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: rocksdb: Options.merge_operator: 2024-03-29T07:55:30.140 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: rocksdb: Options.compaction_filter: None 2024-03-29T07:55:30.140 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: rocksdb: Options.compaction_filter_factory: None 2024-03-29T07:55:30.140 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: rocksdb: Options.sst_partitioner_factory: None 2024-03-29T07:55:30.140 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: rocksdb: Options.memtable_factory: SkipListFactory 2024-03-29T07:55:30.140 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: rocksdb: Options.table_factory: BlockBasedTable 2024-03-29T07:55:30.140 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55f6f5210f58) 2024-03-29T07:55:30.141 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: cache_index_and_filter_blocks: 1 2024-03-29T07:55:30.141 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-03-29T07:55:30.141 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-03-29T07:55:30.141 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: pin_top_level_index_and_filter: 1 2024-03-29T07:55:30.141 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: index_type: 0 2024-03-29T07:55:30.141 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: data_block_index_type: 0 2024-03-29T07:55:30.141 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: index_shortening: 1 2024-03-29T07:55:30.141 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: data_block_hash_table_util_ratio: 0.750000 2024-03-29T07:55:30.141 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: hash_index_allow_collision: 1 2024-03-29T07:55:30.141 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: checksum: 1 2024-03-29T07:55:30.141 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: no_block_cache: 0 2024-03-29T07:55:30.141 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: block_cache: 0x55f6f5243090 2024-03-29T07:55:30.142 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: block_cache_name: BinnedLRUCache 2024-03-29T07:55:30.142 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: block_cache_options: 2024-03-29T07:55:30.142 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: capacity : 536870912 2024-03-29T07:55:30.142 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: num_shard_bits : 4 2024-03-29T07:55:30.142 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: strict_capacity_limit : 0 2024-03-29T07:55:30.142 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: high_pri_pool_ratio: 0.000 2024-03-29T07:55:30.142 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: block_cache_compressed: (nil) 2024-03-29T07:55:30.142 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: persistent_cache: (nil) 2024-03-29T07:55:30.142 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: block_size: 4096 2024-03-29T07:55:30.142 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: block_size_deviation: 10 2024-03-29T07:55:30.142 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: block_restart_interval: 16 2024-03-29T07:55:30.142 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: index_block_restart_interval: 1 2024-03-29T07:55:30.143 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: metadata_block_size: 4096 2024-03-29T07:55:30.143 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: partition_filters: 0 2024-03-29T07:55:30.143 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: use_delta_encoding: 1 2024-03-29T07:55:30.143 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: filter_policy: rocksdb.BuiltinBloomFilter 2024-03-29T07:55:30.143 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: whole_key_filtering: 1 2024-03-29T07:55:30.143 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: verify_compression: 0 2024-03-29T07:55:30.143 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: read_amp_bytes_per_bit: 0 2024-03-29T07:55:30.143 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: format_version: 4 2024-03-29T07:55:30.143 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: enable_index_compression: 1 2024-03-29T07:55:30.143 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: block_align: 0 2024-03-29T07:55:30.143 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: rocksdb: Options.write_buffer_size: 33554432 2024-03-29T07:55:30.143 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: rocksdb: Options.max_write_buffer_number: 2 2024-03-29T07:55:30.144 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: rocksdb: Options.compression: NoCompression 2024-03-29T07:55:30.144 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: rocksdb: Options.bottommost_compression: Disabled 2024-03-29T07:55:30.144 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: rocksdb: Options.prefix_extractor: nullptr 2024-03-29T07:55:30.144 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-03-29T07:55:30.144 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: rocksdb: Options.num_levels: 7 2024-03-29T07:55:30.144 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-03-29T07:55:30.144 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-03-29T07:55:30.144 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-03-29T07:55:30.144 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-03-29T07:55:30.144 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-03-29T07:55:30.144 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-03-29T07:55:30.144 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-03-29T07:55:30.145 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-03-29T07:55:30.145 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-03-29T07:55:30.145 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-03-29T07:55:30.145 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: rocksdb: Options.compression_opts.window_bits: -14 2024-03-29T07:55:30.145 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: rocksdb: Options.compression_opts.level: 32767 2024-03-29T07:55:30.145 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: rocksdb: Options.compression_opts.strategy: 0 2024-03-29T07:55:30.145 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-03-29T07:55:30.145 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-03-29T07:55:30.145 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-03-29T07:55:30.145 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: rocksdb: Options.compression_opts.enabled: false 2024-03-29T07:55:30.145 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-03-29T07:55:30.145 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-03-29T07:55:30.146 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-03-29T07:55:30.146 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: rocksdb: Options.target_file_size_base: 67108864 2024-03-29T07:55:30.146 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: rocksdb: Options.target_file_size_multiplier: 1 2024-03-29T07:55:30.146 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-03-29T07:55:30.146 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-03-29T07:55:30.146 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-03-29T07:55:30.146 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-03-29T07:55:30.146 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-03-29T07:55:30.146 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-03-29T07:55:30.146 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-03-29T07:55:30.146 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-03-29T07:55:30.147 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-03-29T07:55:30.147 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-03-29T07:55:30.147 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-03-29T07:55:30.147 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-03-29T07:55:30.147 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: rocksdb: Options.arena_block_size: 4194304 2024-03-29T07:55:30.147 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-03-29T07:55:30.147 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-03-29T07:55:30.149 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: rocksdb: Options.rate_limit_delay_max_milliseconds: 100 2024-03-29T07:55:30.149 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: rocksdb: Options.disable_auto_compactions: 0 2024-03-29T07:55:30.149 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-03-29T07:55:30.150 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-03-29T07:55:30.150 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-03-29T07:55:30.150 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-03-29T07:55:30.150 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-03-29T07:55:30.150 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-03-29T07:55:30.150 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-03-29T07:55:30.150 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-03-29T07:55:30.150 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-03-29T07:55:30.150 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-03-29T07:55:30.150 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: rocksdb: Options.table_properties_collectors: 2024-03-29T07:55:30.150 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: rocksdb: Options.inplace_update_support: 0 2024-03-29T07:55:30.150 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: rocksdb: Options.inplace_update_num_locks: 10000 2024-03-29T07:55:30.151 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-03-29T07:55:30.151 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-03-29T07:55:30.151 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: rocksdb: Options.memtable_huge_page_size: 0 2024-03-29T07:55:30.151 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: rocksdb: Options.bloom_locality: 0 2024-03-29T07:55:30.151 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: rocksdb: Options.max_successive_merges: 0 2024-03-29T07:55:30.151 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: rocksdb: Options.optimize_filters_for_hits: 0 2024-03-29T07:55:30.151 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: rocksdb: Options.paranoid_file_checks: 0 2024-03-29T07:55:30.151 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: rocksdb: Options.force_consistency_checks: 1 2024-03-29T07:55:30.151 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: rocksdb: Options.report_bg_io_stats: 0 2024-03-29T07:55:30.151 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: rocksdb: Options.ttl: 2592000 2024-03-29T07:55:30.151 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: rocksdb: Options.periodic_compaction_seconds: 0 2024-03-29T07:55:30.151 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: rocksdb: Options.enable_blob_files: false 2024-03-29T07:55:30.152 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: rocksdb: Options.min_blob_size: 0 2024-03-29T07:55:30.152 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: rocksdb: Options.blob_file_size: 268435456 2024-03-29T07:55:30.152 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: rocksdb: Options.blob_compression_type: NoCompression 2024-03-29T07:55:30.152 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: rocksdb: Options.enable_blob_garbage_collection: false 2024-03-29T07:55:30.152 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-03-29T07:55:30.152 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: rocksdb: [db/version_set.cc:4764] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi037/store.db/MANIFEST-000009 succeeded,manifest_file_number is 9, next_file_number is 11, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-03-29T07:55:30.152 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: rocksdb: [db/version_set.cc:4779] Column family [default] (ID 0), log number is 5 2024-03-29T07:55:30.152 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: rocksdb: [db/version_set.cc:4082] Creating manifest 13 2024-03-29T07:55:30.152 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711698929869994, "job": 1, "event": "recovery_started", "wal_files": [10]} 2024-03-29T07:55:30.152 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: rocksdb: [db/db_impl/db_impl_open.cc:845] Recovering log #10 mode 2 2024-03-29T07:55:30.153 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: rocksdb: [table/block_based/filter_policy.cc:991] Using legacy Bloom filter with high (20) bits/key. Dramatic filter space and/or accuracy improvement is available with format_version>=5. 2024-03-29T07:55:30.153 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711698929872067, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 14, "file_size": 92129, "file_checksum": "", "file_checksum_func_name": "Unknown", "table_properties": {"data_size": 90224, "index_size": 265, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 709, "raw_key_size": 13225, "raw_average_key_size": 49, "raw_value_size": 82963, "raw_average_value_size": 310, "num_data_blocks": 11, "num_entries": 267, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "rocksdb.BuiltinBloomFilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; ", "creation_time": 1711698929, "oldest_key_time": 0, "file_creation_time": 0, "db_id": "165e6a94-8278-4f08-bdc9-418bb7516d4f", "db_session_id": "Y7VLBQ0H6A4595UKCSLW"}} 2024-03-29T07:55:30.153 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: rocksdb: [db/version_set.cc:4082] Creating manifest 15 2024-03-29T07:55:30.153 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711698929872969, "job": 1, "event": "recovery_finished"} 2024-03-29T07:55:30.153 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: rocksdb: [file/delete_scheduler.cc:69] Deleted file /var/lib/ceph/mon/ceph-smithi037/store.db/000010.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-03-29T07:55:30.153 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: rocksdb: [db/db_impl/db_impl_open.cc:1700] SstFileManager instance 0x55f6f51d7180 2024-03-29T07:55:30.153 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: rocksdb: DB pointer 0x55f6f52ac000 2024-03-29T07:55:30.153 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: rocksdb: [db/db_impl/db_impl.cc:901] ------- DUMPING STATS ------- 2024-03-29T07:55:30.153 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: rocksdb: [db/db_impl/db_impl.cc:903] 2024-03-29T07:55:30.153 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: ** DB Stats ** 2024-03-29T07:55:30.153 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-29T07:55:30.154 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-03-29T07:55:30.154 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-29T07:55:30.154 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-29T07:55:30.154 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-03-29T07:55:30.154 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 MB, 0.00 MB/s 2024-03-29T07:55:30.154 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-29T07:55:30.154 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: 2024-03-29T07:55:30.154 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: ** Compaction Stats [default] ** 2024-03-29T07:55:30.154 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-03-29T07:55:30.154 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-29T07:55:30.154 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: L0 2/0 91.77 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 72.1 0.00 0.00 1 0.001 0 0 2024-03-29T07:55:30.154 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: Sum 2/0 91.77 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 72.1 0.00 0.00 1 0.001 0 0 2024-03-29T07:55:30.155 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 72.1 0.00 0.00 1 0.001 0 0 2024-03-29T07:55:30.155 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: 2024-03-29T07:55:30.155 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: ** Compaction Stats [default] ** 2024-03-29T07:55:30.155 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-03-29T07:55:30.155 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-29T07:55:30.155 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 72.1 0.00 0.00 1 0.001 0 0 2024-03-29T07:55:30.155 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-29T07:55:30.155 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-03-29T07:55:30.155 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-03-29T07:55:30.155 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-03-29T07:55:30.155 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-03-29T07:55:30.155 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: AddFile(Keys): cumulative 0, interval 0 2024-03-29T07:55:30.155 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: Cumulative compaction: 0.00 GB write, 9.70 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-29T07:55:30.156 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: Interval compaction: 0.00 GB write, 9.70 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-29T07:55:30.156 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-03-29T07:55:30.156 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: 2024-03-29T07:55:30.156 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: ** File Read Latency Histogram By Level [default] ** 2024-03-29T07:55:30.156 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: 2024-03-29T07:55:30.156 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: ** Compaction Stats [default] ** 2024-03-29T07:55:30.156 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-03-29T07:55:30.156 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-29T07:55:30.156 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: L0 2/0 91.77 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 72.1 0.00 0.00 1 0.001 0 0 2024-03-29T07:55:30.156 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: Sum 2/0 91.77 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 72.1 0.00 0.00 1 0.001 0 0 2024-03-29T07:55:30.156 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2024-03-29T07:55:30.156 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: 2024-03-29T07:55:30.157 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: ** Compaction Stats [default] ** 2024-03-29T07:55:30.157 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-03-29T07:55:30.157 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-29T07:55:30.157 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 72.1 0.00 0.00 1 0.001 0 0 2024-03-29T07:55:30.157 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-29T07:55:30.157 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-03-29T07:55:30.157 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-03-29T07:55:30.157 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-03-29T07:55:30.157 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-03-29T07:55:30.157 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: AddFile(Keys): cumulative 0, interval 0 2024-03-29T07:55:30.157 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: Cumulative compaction: 0.00 GB write, 9.66 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-29T07:55:30.157 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-29T07:55:30.157 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-03-29T07:55:30.158 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: 2024-03-29T07:55:30.158 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: ** File Read Latency Histogram By Level [default] ** 2024-03-29T07:55:30.158 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: starting mon.smithi037 rank 0 at public addrs [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] at bind addrs [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi037 fsid 99b492c2-eda1-11ee-b647-cb9ed24678a4 2024-03-29T07:55:30.158 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: mon.smithi037@-1(???) e1 preinit fsid 99b492c2-eda1-11ee-b647-cb9ed24678a4 2024-03-29T07:55:30.158 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: mon.smithi037@-1(???).mds e1 new map 2024-03-29T07:55:30.158 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: mon.smithi037@-1(???).mds e1 print_map 2024-03-29T07:55:30.158 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: e1 2024-03-29T07:55:30.158 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-03-29T07:55:30.158 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T07:55:30.158 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: legacy client fscid: -1 2024-03-29T07:55:30.158 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: 2024-03-29T07:55:30.158 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: No filesystems configured 2024-03-29T07:55:30.159 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: mon.smithi037@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-03-29T07:55:30.159 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: mon.smithi037@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-29T07:55:30.159 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: mon.smithi037@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-29T07:55:30.159 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: mon.smithi037@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-29T07:55:30.159 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: mon.smithi037@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-03-29T07:55:30.159 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 bash[35124]: bb21015057300b16907cba1da34dd8c99143eb4957d8adde92e2fbf0e6aeb88a 2024-03-29T07:55:30.159 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 systemd[1]: Started Ceph mon.smithi037 for 99b492c2-eda1-11ee-b647-cb9ed24678a4. 2024-03-29T07:55:30.159 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: mon.smithi037 is new leader, mons smithi037 in quorum (ranks 0) 2024-03-29T07:55:30.159 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: monmap e1: 1 mons at {smithi037=[v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-29T07:55:30.159 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: fsmap 2024-03-29T07:55:30.159 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: osdmap e1: 0 total, 0 up, 0 in 2024-03-29T07:55:30.159 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:29 smithi037 ceph-mon[35162]: mgrmap e1: no daemons active 2024-03-29T07:55:31.151 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:30.446+0000 7f256ee00700 1 Processor -- start 2024-03-29T07:55:31.151 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:30.446+0000 7f256ee00700 1 -- start start 2024-03-29T07:55:31.151 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:30.447+0000 7f256ee00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2568102530 0x7f2568102940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T07:55:31.151 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:30.447+0000 7f256ee00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2568102e80 con 0x7f2568102530 2024-03-29T07:55:31.151 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:30.448+0000 7f2567e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2568102530 0x7f2568102940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T07:55:31.151 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:30.448+0000 7f2567e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2568102530 0x7f2568102940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35924/0 (socket says 172.21.15.37:35924) 2024-03-29T07:55:31.151 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:30.448+0000 7f2567e00700 1 -- 172.21.15.37:0/2803331920 learned_addr learned my addr 172.21.15.37:0/2803331920 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T07:55:31.152 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:30.448+0000 7f2567e00700 1 -- 172.21.15.37:0/2803331920 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2568102fc0 con 0x7f2568102530 2024-03-29T07:55:31.152 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:30.448+0000 7f2567e00700 1 --2- 172.21.15.37:0/2803331920 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2568102530 0x7f2568102940 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f25580098e0 tx=0x7f2558009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9875657224db5287 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T07:55:31.152 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:30.449+0000 7f2566a00700 1 -- 172.21.15.37:0/2803331920 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f255800b860 con 0x7f2568102530 2024-03-29T07:55:31.152 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:30.449+0000 7f2566a00700 1 -- 172.21.15.37:0/2803331920 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1009+0+0 (secure 0 0 0) 0x7f255800be30 con 0x7f2568102530 2024-03-29T07:55:31.152 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:30.449+0000 7f2566a00700 1 -- 172.21.15.37:0/2803331920 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2558010380 con 0x7f2568102530 2024-03-29T07:55:31.152 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:30.449+0000 7f256ee00700 1 -- 172.21.15.37:0/2803331920 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2568102530 msgr2=0x7f2568102940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T07:55:31.152 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:30.449+0000 7f256ee00700 1 --2- 172.21.15.37:0/2803331920 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2568102530 0x7f2568102940 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f25580098e0 tx=0x7f2558009250 comp rx=0 tx=0).stop 2024-03-29T07:55:31.152 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:30.449+0000 7f256ee00700 1 -- 172.21.15.37:0/2803331920 shutdown_connections 2024-03-29T07:55:31.152 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:30.449+0000 7f256ee00700 1 --2- 172.21.15.37:0/2803331920 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2568102530 0x7f2568102940 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T07:55:31.152 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:30.449+0000 7f256ee00700 1 -- 172.21.15.37:0/2803331920 >> 172.21.15.37:0/2803331920 conn(0x7f25680fdcd0 msgr2=0x7f2568100100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T07:55:31.152 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:30.449+0000 7f256ee00700 1 -- 172.21.15.37:0/2803331920 shutdown_connections 2024-03-29T07:55:31.152 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:30.449+0000 7f256ee00700 1 -- 172.21.15.37:0/2803331920 wait complete. 2024-03-29T07:55:31.152 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:30.450+0000 7f256ee00700 1 Processor -- start 2024-03-29T07:55:31.152 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:30.450+0000 7f256ee00700 1 -- start start 2024-03-29T07:55:31.153 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:30.451+0000 7f256ee00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2568102530 0x7f2568113630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T07:55:31.153 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:30.451+0000 7f256ee00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2568113b70 con 0x7f2568102530 2024-03-29T07:55:31.153 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:30.451+0000 7f2567e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2568102530 0x7f2568113630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T07:55:31.153 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:30.451+0000 7f2567e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2568102530 0x7f2568113630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35930/0 (socket says 172.21.15.37:35930) 2024-03-29T07:55:31.153 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:30.451+0000 7f2567e00700 1 -- 172.21.15.37:0/2308905471 learned_addr learned my addr 172.21.15.37:0/2308905471 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T07:55:31.153 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:30.451+0000 7f2567e00700 1 -- 172.21.15.37:0/2308905471 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25580094f0 con 0x7f2568102530 2024-03-29T07:55:31.153 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:30.452+0000 7f2567e00700 1 --2- 172.21.15.37:0/2308905471 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2568102530 0x7f2568113630 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f2558006b00 tx=0x7f2558009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T07:55:31.153 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:30.452+0000 7f255fe00700 1 -- 172.21.15.37:0/2308905471 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2558010750 con 0x7f2568102530 2024-03-29T07:55:31.153 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:30.452+0000 7f255fe00700 1 -- 172.21.15.37:0/2308905471 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1009+0+0 (secure 0 0 0) 0x7f2558010d20 con 0x7f2568102530 2024-03-29T07:55:31.153 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:30.452+0000 7f256ee00700 1 -- 172.21.15.37:0/2308905471 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2568113d70 con 0x7f2568102530 2024-03-29T07:55:31.153 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:30.452+0000 7f255fe00700 1 -- 172.21.15.37:0/2308905471 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f255801acf0 con 0x7f2568102530 2024-03-29T07:55:31.153 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:30.452+0000 7f256ee00700 1 -- 172.21.15.37:0/2308905471 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25681148e0 con 0x7f2568102530 2024-03-29T07:55:31.153 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:30.453+0000 7f255fe00700 1 -- 172.21.15.37:0/2308905471 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f255802b440 con 0x7f2568102530 2024-03-29T07:55:31.153 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:30.453+0000 7f255fe00700 1 -- 172.21.15.37:0/2308905471 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f255801e070 con 0x7f2568102530 2024-03-29T07:55:31.154 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:30.453+0000 7f256ee00700 1 -- 172.21.15.37:0/2308905471 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f256805f5f0 con 0x7f2568102530 2024-03-29T07:55:31.154 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:30.456+0000 7f255fe00700 1 -- 172.21.15.37:0/2308905471 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f2558006d70 con 0x7f2568102530 2024-03-29T07:55:31.154 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:30.527+0000 7f256ee00700 1 -- 172.21.15.37:0/2308905471 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) v1 -- 0x7f25681043e0 con 0x7f2568102530 2024-03-29T07:55:31.154 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:30.530+0000 7f255fe00700 1 -- 172.21.15.37:0/2308905471 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) v1 ==== 124+0+0 (secure 0 0 0) 0x7f25580330c0 con 0x7f2568102530 2024-03-29T07:55:31.154 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:30.531+0000 7f256ee00700 1 -- 172.21.15.37:0/2308905471 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2568102530 msgr2=0x7f2568113630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T07:55:31.154 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:30.531+0000 7f256ee00700 1 --2- 172.21.15.37:0/2308905471 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2568102530 0x7f2568113630 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f2558006b00 tx=0x7f2558009250 comp rx=0 tx=0).stop 2024-03-29T07:55:31.154 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:30.531+0000 7f256ee00700 1 -- 172.21.15.37:0/2308905471 shutdown_connections 2024-03-29T07:55:31.154 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:30.531+0000 7f256ee00700 1 --2- 172.21.15.37:0/2308905471 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2568102530 0x7f2568113630 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T07:55:31.154 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:30.531+0000 7f256ee00700 1 -- 172.21.15.37:0/2308905471 >> 172.21.15.37:0/2308905471 conn(0x7f25680fdcd0 msgr2=0x7f25680fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T07:55:31.154 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:30.531+0000 7f256ee00700 1 -- 172.21.15.37:0/2308905471 shutdown_connections 2024-03-29T07:55:31.154 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:30.531+0000 7f256ee00700 1 -- 172.21.15.37:0/2308905471 wait complete. 2024-03-29T07:55:31.154 INFO:teuthology.orchestra.run.smithi037.stdout:Wrote config to /etc/ceph/ceph.conf 2024-03-29T07:55:31.154 INFO:teuthology.orchestra.run.smithi037.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-03-29T07:55:31.154 INFO:teuthology.orchestra.run.smithi037.stdout:Creating mgr... 2024-03-29T07:55:31.155 INFO:teuthology.orchestra.run.smithi037.stdout:Verifying port 9283 ... 2024-03-29T07:55:31.422 INFO:teuthology.orchestra.run.smithi037.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4@mgr.smithi037.wjikii 2024-03-29T07:55:31.423 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stderr Failed to reset failed state of unit ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4@mgr.smithi037.wjikii.service: Unit ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4@mgr.smithi037.wjikii.service not loaded. 2024-03-29T07:55:31.614 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4.target.wants/ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4@mgr.smithi037.wjikii.service → /etc/systemd/system/ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4@.service. 2024-03-29T07:55:31.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:31 smithi037 ceph-mon[35162]: from='client.? 172.21.15.37:0/2308905471' entity='client.admin' 2024-03-29T07:55:34.125 INFO:teuthology.orchestra.run.smithi037.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-29T07:55:34.125 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stdout disabled 2024-03-29T07:55:34.142 INFO:teuthology.orchestra.run.smithi037.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-29T07:55:34.142 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stdout inactive 2024-03-29T07:55:34.143 INFO:teuthology.orchestra.run.smithi037.stdout:firewalld.service is not enabled 2024-03-29T07:55:34.143 INFO:teuthology.orchestra.run.smithi037.stdout:Not possible to enable service . firewalld.service is not available 2024-03-29T07:55:34.160 INFO:teuthology.orchestra.run.smithi037.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-29T07:55:34.160 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stdout disabled 2024-03-29T07:55:34.177 INFO:teuthology.orchestra.run.smithi037.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-29T07:55:34.177 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stdout inactive 2024-03-29T07:55:34.177 INFO:teuthology.orchestra.run.smithi037.stdout:firewalld.service is not enabled 2024-03-29T07:55:34.177 INFO:teuthology.orchestra.run.smithi037.stdout:Not possible to open ports <[9283]>. firewalld.service is not available 2024-03-29T07:55:34.177 INFO:teuthology.orchestra.run.smithi037.stdout:Waiting for mgr to start... 2024-03-29T07:55:34.177 INFO:teuthology.orchestra.run.smithi037.stdout:Waiting for mgr... 2024-03-29T07:55:35.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:34 smithi037 ceph-mon[35162]: from='client.? 172.21.15.37:0/3380973480' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-29T07:55:35.545 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout 2024-03-29T07:55:35.545 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout { 2024-03-29T07:55:35.545 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "fsid": "99b492c2-eda1-11ee-b647-cb9ed24678a4", 2024-03-29T07:55:35.546 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "health": { 2024-03-29T07:55:35.546 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-29T07:55:35.546 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-29T07:55:35.546 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-29T07:55:35.546 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-03-29T07:55:35.546 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-29T07:55:35.546 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-29T07:55:35.546 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout 0 2024-03-29T07:55:35.546 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout ], 2024-03-29T07:55:35.546 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-29T07:55:35.546 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "smithi037" 2024-03-29T07:55:35.546 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout ], 2024-03-29T07:55:35.546 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "quorum_age": 4, 2024-03-29T07:55:35.546 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-29T07:55:35.546 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-29T07:55:35.547 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "quincy", 2024-03-29T07:55:35.547 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-29T07:55:35.547 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-03-29T07:55:35.547 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-29T07:55:35.547 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-29T07:55:35.547 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-29T07:55:35.547 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-29T07:55:35.547 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-29T07:55:35.547 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-29T07:55:35.547 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-29T07:55:35.547 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-29T07:55:35.547 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-03-29T07:55:35.547 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-29T07:55:35.547 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-29T07:55:35.547 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-29T07:55:35.548 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-29T07:55:35.548 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-29T07:55:35.548 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-29T07:55:35.548 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-29T07:55:35.548 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-29T07:55:35.548 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-29T07:55:35.548 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-03-29T07:55:35.548 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-29T07:55:35.548 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-29T07:55:35.548 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-29T07:55:35.548 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-29T07:55:35.548 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-03-29T07:55:35.548 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-29T07:55:35.548 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "available": false, 2024-03-29T07:55:35.548 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-29T07:55:35.548 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-29T07:55:35.549 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-29T07:55:35.549 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-29T07:55:35.549 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "restful" 2024-03-29T07:55:35.549 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout ], 2024-03-29T07:55:35.549 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-29T07:55:35.549 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-03-29T07:55:35.549 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-29T07:55:35.549 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-29T07:55:35.549 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "modified": "2024-03-29T07:55:23.674437+0000", 2024-03-29T07:55:35.549 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-29T07:55:35.549 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-03-29T07:55:35.549 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-29T07:55:35.549 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout } 2024-03-29T07:55:35.549 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:34.769+0000 7fd964e00700 1 Processor -- start 2024-03-29T07:55:35.549 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:34.769+0000 7fd964e00700 1 -- start start 2024-03-29T07:55:35.550 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:34.769+0000 7fd964e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd960078d30 0x7fd960077240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T07:55:35.550 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:34.769+0000 7fd964e00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd960079140 con 0x7fd960078d30 2024-03-29T07:55:35.550 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:34.770+0000 7fd95f400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd960078d30 0x7fd960077240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T07:55:35.550 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:34.770+0000 7fd95f400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd960078d30 0x7fd960077240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35956/0 (socket says 172.21.15.37:35956) 2024-03-29T07:55:35.550 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:34.770+0000 7fd95f400700 1 -- 172.21.15.37:0/430394907 learned_addr learned my addr 172.21.15.37:0/430394907 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T07:55:35.550 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:34.770+0000 7fd95f400700 1 -- 172.21.15.37:0/430394907 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd960077780 con 0x7fd960078d30 2024-03-29T07:55:35.550 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:34.770+0000 7fd95f400700 1 --2- 172.21.15.37:0/430394907 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd960078d30 0x7fd960077240 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fd9500098e0 tx=0x7fd950009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb60525ef4024072 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T07:55:35.550 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:34.771+0000 7fd95e000700 1 -- 172.21.15.37:0/430394907 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd95000b860 con 0x7fd960078d30 2024-03-29T07:55:35.550 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:34.771+0000 7fd95e000700 1 -- 172.21.15.37:0/430394907 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1009+0+0 (secure 0 0 0) 0x7fd95000be30 con 0x7fd960078d30 2024-03-29T07:55:35.550 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:34.771+0000 7fd95e000700 1 -- 172.21.15.37:0/430394907 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd950010380 con 0x7fd960078d30 2024-03-29T07:55:35.550 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:34.771+0000 7fd964e00700 1 -- 172.21.15.37:0/430394907 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd960078d30 msgr2=0x7fd960077240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T07:55:35.550 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:34.771+0000 7fd964e00700 1 --2- 172.21.15.37:0/430394907 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd960078d30 0x7fd960077240 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fd9500098e0 tx=0x7fd950009250 comp rx=0 tx=0).stop 2024-03-29T07:55:35.550 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:34.771+0000 7fd964e00700 1 -- 172.21.15.37:0/430394907 shutdown_connections 2024-03-29T07:55:35.551 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:34.771+0000 7fd964e00700 1 --2- 172.21.15.37:0/430394907 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd960078d30 0x7fd960077240 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T07:55:35.551 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:34.771+0000 7fd964e00700 1 -- 172.21.15.37:0/430394907 >> 172.21.15.37:0/430394907 conn(0x7fd9600fdb90 msgr2=0x7fd9600fffa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T07:55:35.551 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:34.771+0000 7fd964e00700 1 -- 172.21.15.37:0/430394907 shutdown_connections 2024-03-29T07:55:35.551 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:34.771+0000 7fd964e00700 1 -- 172.21.15.37:0/430394907 wait complete. 2024-03-29T07:55:35.551 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:34.772+0000 7fd964e00700 1 Processor -- start 2024-03-29T07:55:35.551 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:34.772+0000 7fd964e00700 1 -- start start 2024-03-29T07:55:35.551 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:34.773+0000 7fd964e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd960078d30 0x7fd960113460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T07:55:35.551 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:34.773+0000 7fd964e00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9601139a0 con 0x7fd960078d30 2024-03-29T07:55:35.551 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:34.773+0000 7fd95f400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd960078d30 0x7fd960113460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T07:55:35.551 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:34.773+0000 7fd95f400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd960078d30 0x7fd960113460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35962/0 (socket says 172.21.15.37:35962) 2024-03-29T07:55:35.551 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:34.773+0000 7fd95f400700 1 -- 172.21.15.37:0/3380973480 learned_addr learned my addr 172.21.15.37:0/3380973480 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T07:55:35.551 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:34.773+0000 7fd95f400700 1 -- 172.21.15.37:0/3380973480 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9500094f0 con 0x7fd960078d30 2024-03-29T07:55:35.551 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:34.774+0000 7fd95f400700 1 --2- 172.21.15.37:0/3380973480 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd960078d30 0x7fd960113460 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fd950006fd0 tx=0x7fd950009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T07:55:35.551 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:34.774+0000 7fd957400700 1 -- 172.21.15.37:0/3380973480 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd9500107a0 con 0x7fd960078d30 2024-03-29T07:55:35.552 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:34.774+0000 7fd957400700 1 -- 172.21.15.37:0/3380973480 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1009+0+0 (secure 0 0 0) 0x7fd950010d70 con 0x7fd960078d30 2024-03-29T07:55:35.552 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:34.774+0000 7fd957400700 1 -- 172.21.15.37:0/3380973480 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd95001ad60 con 0x7fd960078d30 2024-03-29T07:55:35.552 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:34.774+0000 7fd964e00700 1 -- 172.21.15.37:0/3380973480 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd960113ba0 con 0x7fd960078d30 2024-03-29T07:55:35.552 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:34.774+0000 7fd964e00700 1 -- 172.21.15.37:0/3380973480 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd960114710 con 0x7fd960078d30 2024-03-29T07:55:35.552 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:34.774+0000 7fd957400700 1 -- 172.21.15.37:0/3380973480 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7fd950024670 con 0x7fd960078d30 2024-03-29T07:55:35.552 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:34.774+0000 7fd957400700 1 -- 172.21.15.37:0/3380973480 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7fd9500197a0 con 0x7fd960078d30 2024-03-29T07:55:35.552 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:34.775+0000 7fd964e00700 1 -- 172.21.15.37:0/3380973480 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd96005f5f0 con 0x7fd960078d30 2024-03-29T07:55:35.552 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:34.778+0000 7fd957400700 1 -- 172.21.15.37:0/3380973480 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7fd950029060 con 0x7fd960078d30 2024-03-29T07:55:35.552 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:34.854+0000 7fd964e00700 1 -- 172.21.15.37:0/3380973480 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fd960108650 con 0x7fd960078d30 2024-03-29T07:55:35.552 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:34.855+0000 7fd957400700 1 -- 172.21.15.37:0/3380973480 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1248 (secure 0 0 0) 0x7fd950010900 con 0x7fd960078d30 2024-03-29T07:55:35.552 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:34.856+0000 7fd964e00700 1 -- 172.21.15.37:0/3380973480 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd960078d30 msgr2=0x7fd960113460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T07:55:35.552 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:34.856+0000 7fd964e00700 1 --2- 172.21.15.37:0/3380973480 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd960078d30 0x7fd960113460 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fd950006fd0 tx=0x7fd950009250 comp rx=0 tx=0).stop 2024-03-29T07:55:35.552 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:34.856+0000 7fd964e00700 1 -- 172.21.15.37:0/3380973480 shutdown_connections 2024-03-29T07:55:35.552 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:34.856+0000 7fd964e00700 1 --2- 172.21.15.37:0/3380973480 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd960078d30 0x7fd960113460 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T07:55:35.552 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:34.856+0000 7fd964e00700 1 -- 172.21.15.37:0/3380973480 >> 172.21.15.37:0/3380973480 conn(0x7fd9600fdb90 msgr2=0x7fd9600fe720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T07:55:35.553 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:34.856+0000 7fd964e00700 1 -- 172.21.15.37:0/3380973480 shutdown_connections 2024-03-29T07:55:35.553 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:34.856+0000 7fd964e00700 1 -- 172.21.15.37:0/3380973480 wait complete. 2024-03-29T07:55:35.553 INFO:teuthology.orchestra.run.smithi037.stdout:mgr not available, waiting (1/15)... 2024-03-29T07:55:38.481 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:38 smithi037 ceph-mon[35162]: from='client.? 172.21.15.37:0/1880014190' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-29T07:55:42.676 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:42 smithi037 ceph-mon[35162]: Activating manager daemon smithi037.wjikii 2024-03-29T07:55:42.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:42 smithi037 ceph-mon[35162]: mgrmap e2: smithi037.wjikii(active, starting, since 0.00411021s) 2024-03-29T07:55:42.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:42 smithi037 ceph-mon[35162]: from='mgr.14100 172.21.15.37:0/1592880007' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-29T07:55:42.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:42 smithi037 ceph-mon[35162]: from='mgr.14100 172.21.15.37:0/1592880007' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-29T07:55:42.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:42 smithi037 ceph-mon[35162]: from='mgr.14100 172.21.15.37:0/1592880007' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-29T07:55:42.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:42 smithi037 ceph-mon[35162]: from='mgr.14100 172.21.15.37:0/1592880007' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "mon metadata", "id": "smithi037"}]: dispatch 2024-03-29T07:55:42.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:42 smithi037 ceph-mon[35162]: from='mgr.14100 172.21.15.37:0/1592880007' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "mgr metadata", "who": "smithi037.wjikii", "id": "smithi037.wjikii"}]: dispatch 2024-03-29T07:55:42.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:42 smithi037 ceph-mon[35162]: Manager daemon smithi037.wjikii is now available 2024-03-29T07:55:42.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:42 smithi037 ceph-mon[35162]: from='mgr.14100 172.21.15.37:0/1592880007' entity='mgr.smithi037.wjikii' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi037.wjikii/mirror_snapshot_schedule"}]: dispatch 2024-03-29T07:55:42.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:42 smithi037 ceph-mon[35162]: from='mgr.14100 172.21.15.37:0/1592880007' entity='mgr.smithi037.wjikii' 2024-03-29T07:55:42.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:42 smithi037 ceph-mon[35162]: from='mgr.14100 172.21.15.37:0/1592880007' entity='mgr.smithi037.wjikii' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi037.wjikii/trash_purge_schedule"}]: dispatch 2024-03-29T07:55:42.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:42 smithi037 ceph-mon[35162]: from='mgr.14100 172.21.15.37:0/1592880007' entity='mgr.smithi037.wjikii' 2024-03-29T07:55:42.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:42 smithi037 ceph-mon[35162]: from='mgr.14100 172.21.15.37:0/1592880007' entity='mgr.smithi037.wjikii' 2024-03-29T07:55:44.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:44 smithi037 ceph-mon[35162]: mgrmap e3: smithi037.wjikii(active, since 1.01006s) 2024-03-29T07:55:45.653 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout 2024-03-29T07:55:45.654 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout { 2024-03-29T07:55:45.654 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "fsid": "99b492c2-eda1-11ee-b647-cb9ed24678a4", 2024-03-29T07:55:45.654 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "health": { 2024-03-29T07:55:45.654 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-29T07:55:45.657 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-29T07:55:45.657 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-29T07:55:45.657 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-03-29T07:55:45.657 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-29T07:55:45.657 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-29T07:55:45.657 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout 0 2024-03-29T07:55:45.657 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout ], 2024-03-29T07:55:45.658 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-29T07:55:45.658 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "smithi037" 2024-03-29T07:55:45.658 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout ], 2024-03-29T07:55:45.658 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "quorum_age": 8, 2024-03-29T07:55:45.658 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-29T07:55:45.658 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-29T07:55:45.658 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "quincy", 2024-03-29T07:55:45.658 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-29T07:55:45.658 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-03-29T07:55:45.658 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-29T07:55:45.658 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-29T07:55:45.658 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-29T07:55:45.658 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-29T07:55:45.658 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-29T07:55:45.658 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-29T07:55:45.659 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-29T07:55:45.659 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-29T07:55:45.659 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-03-29T07:55:45.660 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-29T07:55:45.660 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-29T07:55:45.660 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-29T07:55:45.660 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-29T07:55:45.660 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-29T07:55:45.660 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-29T07:55:45.660 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-29T07:55:45.660 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-29T07:55:45.660 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-29T07:55:45.661 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-03-29T07:55:45.668 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-29T07:55:45.668 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-29T07:55:45.668 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-29T07:55:45.668 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-29T07:55:45.668 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-03-29T07:55:45.668 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-29T07:55:45.668 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "available": false, 2024-03-29T07:55:45.668 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-29T07:55:45.669 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-29T07:55:45.669 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-29T07:55:45.669 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-29T07:55:45.669 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "restful" 2024-03-29T07:55:45.669 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout ], 2024-03-29T07:55:45.669 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-29T07:55:45.669 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-03-29T07:55:45.669 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-29T07:55:45.669 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-29T07:55:45.669 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "modified": "2024-03-29T07:55:23.674437+0000", 2024-03-29T07:55:45.669 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-29T07:55:45.669 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-03-29T07:55:45.669 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-29T07:55:45.669 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout } 2024-03-29T07:55:45.669 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:38.111+0000 7f40e1a00700 1 Processor -- start 2024-03-29T07:55:45.670 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:38.111+0000 7f40e1a00700 1 -- start start 2024-03-29T07:55:45.670 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:38.112+0000 7f40e1a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f40dc078d50 0x7f40dc077260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T07:55:45.670 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:38.112+0000 7f40e1a00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40dc079160 con 0x7f40dc078d50 2024-03-29T07:55:45.670 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:38.112+0000 7f40daa00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f40dc078d50 0x7f40dc077260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T07:55:45.670 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:38.112+0000 7f40daa00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f40dc078d50 0x7f40dc077260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35968/0 (socket says 172.21.15.37:35968) 2024-03-29T07:55:45.670 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:38.112+0000 7f40daa00700 1 -- 172.21.15.37:0/4217956938 learned_addr learned my addr 172.21.15.37:0/4217956938 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T07:55:45.670 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:38.113+0000 7f40daa00700 1 -- 172.21.15.37:0/4217956938 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40dc0777a0 con 0x7f40dc078d50 2024-03-29T07:55:45.670 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:38.113+0000 7f40daa00700 1 --2- 172.21.15.37:0/4217956938 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f40dc078d50 0x7f40dc077260 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f40cc0098e0 tx=0x7f40cc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce17961da4a5e0ea server_cookie=0 in_seq=0 out_seq=0 2024-03-29T07:55:45.670 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:38.113+0000 7f40d9600700 1 -- 172.21.15.37:0/4217956938 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f40cc00b860 con 0x7f40dc078d50 2024-03-29T07:55:45.670 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:38.113+0000 7f40d9600700 1 -- 172.21.15.37:0/4217956938 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1009+0+0 (secure 0 0 0) 0x7f40cc00be30 con 0x7f40dc078d50 2024-03-29T07:55:45.670 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:38.113+0000 7f40d9600700 1 -- 172.21.15.37:0/4217956938 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f40cc010380 con 0x7f40dc078d50 2024-03-29T07:55:45.670 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:38.114+0000 7f40e1a00700 1 -- 172.21.15.37:0/4217956938 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f40dc078d50 msgr2=0x7f40dc077260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T07:55:45.670 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:38.114+0000 7f40e1a00700 1 --2- 172.21.15.37:0/4217956938 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f40dc078d50 0x7f40dc077260 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f40cc0098e0 tx=0x7f40cc009250 comp rx=0 tx=0).stop 2024-03-29T07:55:45.671 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:38.114+0000 7f40e1a00700 1 -- 172.21.15.37:0/4217956938 shutdown_connections 2024-03-29T07:55:45.671 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:38.114+0000 7f40e1a00700 1 --2- 172.21.15.37:0/4217956938 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f40dc078d50 0x7f40dc077260 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T07:55:45.671 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:38.114+0000 7f40e1a00700 1 -- 172.21.15.37:0/4217956938 >> 172.21.15.37:0/4217956938 conn(0x7f40dc0fdd00 msgr2=0x7f40dc100110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T07:55:45.671 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:38.114+0000 7f40e1a00700 1 -- 172.21.15.37:0/4217956938 shutdown_connections 2024-03-29T07:55:45.671 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:38.114+0000 7f40e1a00700 1 -- 172.21.15.37:0/4217956938 wait complete. 2024-03-29T07:55:45.671 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:38.115+0000 7f40e1a00700 1 Processor -- start 2024-03-29T07:55:45.671 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:38.115+0000 7f40e1a00700 1 -- start start 2024-03-29T07:55:45.671 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:38.116+0000 7f40e1a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f40dc078d50 0x7f40dc1135a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T07:55:45.671 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:38.116+0000 7f40e1a00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40dc113ae0 con 0x7f40dc078d50 2024-03-29T07:55:45.671 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:38.116+0000 7f40daa00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f40dc078d50 0x7f40dc1135a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T07:55:45.671 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:38.116+0000 7f40daa00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f40dc078d50 0x7f40dc1135a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35976/0 (socket says 172.21.15.37:35976) 2024-03-29T07:55:45.671 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:38.116+0000 7f40daa00700 1 -- 172.21.15.37:0/1880014190 learned_addr learned my addr 172.21.15.37:0/1880014190 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T07:55:45.671 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:38.116+0000 7f40daa00700 1 -- 172.21.15.37:0/1880014190 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40cc0094f0 con 0x7f40dc078d50 2024-03-29T07:55:45.671 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:38.117+0000 7f40daa00700 1 --2- 172.21.15.37:0/1880014190 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f40dc078d50 0x7f40dc1135a0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f40cc006fd0 tx=0x7f40cc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T07:55:45.672 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:38.117+0000 7f40d2a00700 1 -- 172.21.15.37:0/1880014190 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f40cc010750 con 0x7f40dc078d50 2024-03-29T07:55:45.672 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:38.117+0000 7f40d2a00700 1 -- 172.21.15.37:0/1880014190 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1009+0+0 (secure 0 0 0) 0x7f40cc010d20 con 0x7f40dc078d50 2024-03-29T07:55:45.672 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:38.117+0000 7f40d2a00700 1 -- 172.21.15.37:0/1880014190 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f40cc01ada0 con 0x7f40dc078d50 2024-03-29T07:55:45.672 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:38.117+0000 7f40e1a00700 1 -- 172.21.15.37:0/1880014190 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40dc113ce0 con 0x7f40dc078d50 2024-03-29T07:55:45.672 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:38.117+0000 7f40e1a00700 1 -- 172.21.15.37:0/1880014190 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40dc114850 con 0x7f40dc078d50 2024-03-29T07:55:45.672 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:38.117+0000 7f40d2a00700 1 -- 172.21.15.37:0/1880014190 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f40cc024680 con 0x7f40dc078d50 2024-03-29T07:55:45.672 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:38.117+0000 7f40d2a00700 1 -- 172.21.15.37:0/1880014190 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f40cc0197f0 con 0x7f40dc078d50 2024-03-29T07:55:45.672 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:38.118+0000 7f40e1a00700 1 -- 172.21.15.37:0/1880014190 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40dc05f5f0 con 0x7f40dc078d50 2024-03-29T07:55:45.672 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:38.121+0000 7f40d2a00700 1 -- 172.21.15.37:0/1880014190 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f40cc029060 con 0x7f40dc078d50 2024-03-29T07:55:45.672 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:38.198+0000 7f40e1a00700 1 -- 172.21.15.37:0/1880014190 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f40dc1087f0 con 0x7f40dc078d50 2024-03-29T07:55:45.672 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:38.198+0000 7f40d2a00700 1 -- 172.21.15.37:0/1880014190 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1248 (secure 0 0 0) 0x7f40cc033090 con 0x7f40dc078d50 2024-03-29T07:55:45.672 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:38.199+0000 7f40e1a00700 1 -- 172.21.15.37:0/1880014190 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f40dc078d50 msgr2=0x7f40dc1135a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T07:55:45.672 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:38.199+0000 7f40e1a00700 1 --2- 172.21.15.37:0/1880014190 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f40dc078d50 0x7f40dc1135a0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f40cc006fd0 tx=0x7f40cc009250 comp rx=0 tx=0).stop 2024-03-29T07:55:45.672 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:38.199+0000 7f40e1a00700 1 -- 172.21.15.37:0/1880014190 shutdown_connections 2024-03-29T07:55:45.673 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:38.199+0000 7f40e1a00700 1 --2- 172.21.15.37:0/1880014190 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f40dc078d50 0x7f40dc1135a0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T07:55:45.673 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:38.199+0000 7f40e1a00700 1 -- 172.21.15.37:0/1880014190 >> 172.21.15.37:0/1880014190 conn(0x7f40dc0fdd00 msgr2=0x7f40dc0fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T07:55:45.673 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:38.199+0000 7f40e1a00700 1 -- 172.21.15.37:0/1880014190 shutdown_connections 2024-03-29T07:55:45.673 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:38.200+0000 7f40e1a00700 1 -- 172.21.15.37:0/1880014190 wait complete. 2024-03-29T07:55:45.673 INFO:teuthology.orchestra.run.smithi037.stdout:mgr not available, waiting (2/15)... 2024-03-29T07:55:45.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:45 smithi037 ceph-mon[35162]: mgrmap e4: smithi037.wjikii(active, since 2s) 2024-03-29T07:55:48.895 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:48 smithi037 ceph-mon[35162]: from='client.? 172.21.15.37:0/1930070715' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-29T07:55:49.176 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout 2024-03-29T07:55:49.177 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout { 2024-03-29T07:55:49.177 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "fsid": "99b492c2-eda1-11ee-b647-cb9ed24678a4", 2024-03-29T07:55:49.177 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "health": { 2024-03-29T07:55:49.177 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-29T07:55:49.177 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-29T07:55:49.177 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-29T07:55:49.177 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-03-29T07:55:49.177 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-29T07:55:49.177 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-29T07:55:49.177 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout 0 2024-03-29T07:55:49.177 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout ], 2024-03-29T07:55:49.177 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-29T07:55:49.177 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "smithi037" 2024-03-29T07:55:49.177 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout ], 2024-03-29T07:55:49.178 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "quorum_age": 18, 2024-03-29T07:55:49.178 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-29T07:55:49.178 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-29T07:55:49.178 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "quincy", 2024-03-29T07:55:49.178 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-29T07:55:49.178 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-03-29T07:55:49.178 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-29T07:55:49.178 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-29T07:55:49.178 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-29T07:55:49.178 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-29T07:55:49.178 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-29T07:55:49.178 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-29T07:55:49.178 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-29T07:55:49.178 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-29T07:55:49.179 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-03-29T07:55:49.179 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-29T07:55:49.179 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-29T07:55:49.179 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-29T07:55:49.179 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-29T07:55:49.179 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-29T07:55:49.179 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-29T07:55:49.179 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-29T07:55:49.179 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-29T07:55:49.179 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-29T07:55:49.179 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-03-29T07:55:49.179 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-29T07:55:49.179 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-29T07:55:49.179 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-29T07:55:49.179 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-29T07:55:49.180 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-03-29T07:55:49.180 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-29T07:55:49.180 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "available": true, 2024-03-29T07:55:49.180 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-29T07:55:49.180 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-29T07:55:49.180 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-29T07:55:49.180 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-29T07:55:49.180 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "restful" 2024-03-29T07:55:49.180 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout ], 2024-03-29T07:55:49.180 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-29T07:55:49.180 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-03-29T07:55:49.180 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-29T07:55:49.180 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-29T07:55:49.180 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "modified": "2024-03-29T07:55:23.674437+0000", 2024-03-29T07:55:49.180 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-29T07:55:49.181 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-03-29T07:55:49.181 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-29T07:55:49.181 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout } 2024-03-29T07:55:49.181 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:48.212+0000 7fa1f2a00700 1 Processor -- start 2024-03-29T07:55:49.181 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:48.212+0000 7fa1f2a00700 1 -- start start 2024-03-29T07:55:49.181 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:48.212+0000 7fa1f2a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa1ec078d10 0x7fa1ec077220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T07:55:49.181 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:48.212+0000 7fa1f2a00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1ec079120 con 0x7fa1ec078d10 2024-03-29T07:55:49.181 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:48.212+0000 7fa1f1600700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa1ec078d10 0x7fa1ec077220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T07:55:49.181 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:48.212+0000 7fa1f1600700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa1ec078d10 0x7fa1ec077220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52462/0 (socket says 172.21.15.37:52462) 2024-03-29T07:55:49.181 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:48.212+0000 7fa1f1600700 1 -- 172.21.15.37:0/626559599 learned_addr learned my addr 172.21.15.37:0/626559599 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T07:55:49.181 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:48.213+0000 7fa1f1600700 1 -- 172.21.15.37:0/626559599 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1ec077760 con 0x7fa1ec078d10 2024-03-29T07:55:49.181 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:48.213+0000 7fa1f1600700 1 --2- 172.21.15.37:0/626559599 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa1ec078d10 0x7fa1ec077220 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fa1e00098e0 tx=0x7fa1e0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdfd97aa4fed7a92 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T07:55:49.181 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:48.213+0000 7fa1ebe00700 1 -- 172.21.15.37:0/626559599 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa1e000b860 con 0x7fa1ec078d10 2024-03-29T07:55:49.181 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:48.213+0000 7fa1ebe00700 1 -- 172.21.15.37:0/626559599 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1009+0+0 (secure 0 0 0) 0x7fa1e000be30 con 0x7fa1ec078d10 2024-03-29T07:55:49.181 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:48.213+0000 7fa1ebe00700 1 -- 172.21.15.37:0/626559599 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa1e0010380 con 0x7fa1ec078d10 2024-03-29T07:55:49.181 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:48.214+0000 7fa1f2a00700 1 -- 172.21.15.37:0/626559599 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa1ec078d10 msgr2=0x7fa1ec077220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T07:55:49.182 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:48.214+0000 7fa1f2a00700 1 --2- 172.21.15.37:0/626559599 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa1ec078d10 0x7fa1ec077220 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fa1e00098e0 tx=0x7fa1e0009250 comp rx=0 tx=0).stop 2024-03-29T07:55:49.182 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:48.214+0000 7fa1f2a00700 1 -- 172.21.15.37:0/626559599 shutdown_connections 2024-03-29T07:55:49.182 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:48.214+0000 7fa1f2a00700 1 --2- 172.21.15.37:0/626559599 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa1ec078d10 0x7fa1ec077220 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T07:55:49.182 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:48.214+0000 7fa1f2a00700 1 -- 172.21.15.37:0/626559599 >> 172.21.15.37:0/626559599 conn(0x7fa1ec0fdb90 msgr2=0x7fa1ec0fffa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T07:55:49.182 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:48.214+0000 7fa1f2a00700 1 -- 172.21.15.37:0/626559599 shutdown_connections 2024-03-29T07:55:49.182 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:48.214+0000 7fa1f2a00700 1 -- 172.21.15.37:0/626559599 wait complete. 2024-03-29T07:55:49.182 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:48.215+0000 7fa1f2a00700 1 Processor -- start 2024-03-29T07:55:49.182 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:48.215+0000 7fa1f2a00700 1 -- start start 2024-03-29T07:55:49.182 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:48.215+0000 7fa1f2a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa1ec078d10 0x7fa1ec113410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T07:55:49.182 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:48.215+0000 7fa1f2a00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1ec113950 con 0x7fa1ec078d10 2024-03-29T07:55:49.182 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:48.216+0000 7fa1f1600700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa1ec078d10 0x7fa1ec113410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T07:55:49.182 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:48.216+0000 7fa1f1600700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa1ec078d10 0x7fa1ec113410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52474/0 (socket says 172.21.15.37:52474) 2024-03-29T07:55:49.182 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:48.216+0000 7fa1f1600700 1 -- 172.21.15.37:0/1930070715 learned_addr learned my addr 172.21.15.37:0/1930070715 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T07:55:49.182 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:48.216+0000 7fa1f1600700 1 -- 172.21.15.37:0/1930070715 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1e00094f0 con 0x7fa1ec078d10 2024-03-29T07:55:49.183 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:48.216+0000 7fa1f1600700 1 --2- 172.21.15.37:0/1930070715 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa1ec078d10 0x7fa1ec113410 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fa1e000b440 tx=0x7fa1e0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T07:55:49.183 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:48.216+0000 7fa1e9600700 1 -- 172.21.15.37:0/1930070715 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa1e0010960 con 0x7fa1ec078d10 2024-03-29T07:55:49.183 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:48.216+0000 7fa1e9600700 1 -- 172.21.15.37:0/1930070715 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1009+0+0 (secure 0 0 0) 0x7fa1e001a430 con 0x7fa1ec078d10 2024-03-29T07:55:49.183 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:48.216+0000 7fa1f2a00700 1 -- 172.21.15.37:0/1930070715 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1ec113b50 con 0x7fa1ec078d10 2024-03-29T07:55:49.183 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:48.216+0000 7fa1e9600700 1 -- 172.21.15.37:0/1930070715 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa1e00245b0 con 0x7fa1ec078d10 2024-03-29T07:55:49.183 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:48.217+0000 7fa1f2a00700 1 -- 172.21.15.37:0/1930070715 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1ec1146c0 con 0x7fa1ec078d10 2024-03-29T07:55:49.183 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:48.218+0000 7fa1e9600700 1 -- 172.21.15.37:0/1930070715 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 4) v1 ==== 43357+0+0 (secure 0 0 0) 0x7fa1e001a9a0 con 0x7fa1ec078d10 2024-03-29T07:55:49.183 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:48.218+0000 7fa1f2a00700 1 -- 172.21.15.37:0/1930070715 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1ec05f5f0 con 0x7fa1ec078d10 2024-03-29T07:55:49.183 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:48.218+0000 7fa1e9600700 1 --2- 172.21.15.37:0/1930070715 >> [v2:172.21.15.37:6800/2762339582,v1:172.21.15.37:6801/2762339582] conn(0x7fa1d4036790 0x7fa1d4038c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T07:55:49.183 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:48.218+0000 7fa1e9600700 1 -- 172.21.15.37:0/1930070715 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7fa1e004a640 con 0x7fa1ec078d10 2024-03-29T07:55:49.183 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:48.219+0000 7fa1f0c00700 1 --2- 172.21.15.37:0/1930070715 >> [v2:172.21.15.37:6800/2762339582,v1:172.21.15.37:6801/2762339582] conn(0x7fa1d4036790 0x7fa1d4038c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T07:55:49.183 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:48.219+0000 7fa1f0c00700 1 --2- 172.21.15.37:0/1930070715 >> [v2:172.21.15.37:6800/2762339582,v1:172.21.15.37:6801/2762339582] conn(0x7fa1d4036790 0x7fa1d4038c40 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fa1d8009990 tx=0x7fa1d8006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T07:55:49.183 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:48.224+0000 7fa1e9600700 1 -- 172.21.15.37:0/1930070715 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fa1e002c340 con 0x7fa1ec078d10 2024-03-29T07:55:49.183 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:48.507+0000 7fa1f2a00700 1 -- 172.21.15.37:0/1930070715 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fa1ec04bed0 con 0x7fa1ec078d10 2024-03-29T07:55:49.183 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:48.507+0000 7fa1e9600700 1 -- 172.21.15.37:0/1930070715 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1248 (secure 0 0 0) 0x7fa1e002ae60 con 0x7fa1ec078d10 2024-03-29T07:55:49.183 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:48.509+0000 7fa1f2a00700 1 -- 172.21.15.37:0/1930070715 >> [v2:172.21.15.37:6800/2762339582,v1:172.21.15.37:6801/2762339582] conn(0x7fa1d4036790 msgr2=0x7fa1d4038c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T07:55:49.184 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:48.509+0000 7fa1f2a00700 1 --2- 172.21.15.37:0/1930070715 >> [v2:172.21.15.37:6800/2762339582,v1:172.21.15.37:6801/2762339582] conn(0x7fa1d4036790 0x7fa1d4038c40 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fa1d8009990 tx=0x7fa1d8006e40 comp rx=0 tx=0).stop 2024-03-29T07:55:49.184 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:48.509+0000 7fa1f2a00700 1 -- 172.21.15.37:0/1930070715 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa1ec078d10 msgr2=0x7fa1ec113410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T07:55:49.184 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:48.509+0000 7fa1f2a00700 1 --2- 172.21.15.37:0/1930070715 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa1ec078d10 0x7fa1ec113410 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fa1e000b440 tx=0x7fa1e0009250 comp rx=0 tx=0).stop 2024-03-29T07:55:49.184 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:48.509+0000 7fa1f2a00700 1 -- 172.21.15.37:0/1930070715 shutdown_connections 2024-03-29T07:55:49.184 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:48.509+0000 7fa1f2a00700 1 --2- 172.21.15.37:0/1930070715 >> [v2:172.21.15.37:6800/2762339582,v1:172.21.15.37:6801/2762339582] conn(0x7fa1d4036790 0x7fa1d4038c40 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T07:55:49.184 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:48.509+0000 7fa1f2a00700 1 --2- 172.21.15.37:0/1930070715 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa1ec078d10 0x7fa1ec113410 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T07:55:49.184 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:48.510+0000 7fa1f2a00700 1 -- 172.21.15.37:0/1930070715 >> 172.21.15.37:0/1930070715 conn(0x7fa1ec0fdb90 msgr2=0x7fa1ec0fe710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T07:55:49.184 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:48.510+0000 7fa1f2a00700 1 -- 172.21.15.37:0/1930070715 shutdown_connections 2024-03-29T07:55:49.184 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:48.510+0000 7fa1f2a00700 1 -- 172.21.15.37:0/1930070715 wait complete. 2024-03-29T07:55:49.184 INFO:teuthology.orchestra.run.smithi037.stdout:mgr is available 2024-03-29T07:55:50.241 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:49 smithi037 ceph-mon[35162]: from='client.? 172.21.15.37:0/4142364609' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-03-29T07:55:50.495 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout 2024-03-29T07:55:50.496 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout [global] 2024-03-29T07:55:50.496 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout fsid = 99b492c2-eda1-11ee-b647-cb9ed24678a4 2024-03-29T07:55:50.496 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-03-29T07:55:50.496 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-03-29T07:55:50.496 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-03-29T07:55:50.496 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-03-29T07:55:50.496 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout 2024-03-29T07:55:50.496 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout [mgr] 2024-03-29T07:55:50.497 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-03-29T07:55:50.497 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout 2024-03-29T07:55:50.497 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout [osd] 2024-03-29T07:55:50.497 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-03-29T07:55:50.497 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-03-29T07:55:50.497 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-03-29T07:55:50.497 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-03-29T07:55:50.497 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-03-29T07:55:50.497 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:49.709+0000 7fbabca00700 1 Processor -- start 2024-03-29T07:55:50.497 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:49.709+0000 7fbabca00700 1 -- start start 2024-03-29T07:55:50.497 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:49.709+0000 7fbabca00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbab8078d50 0x7fbab8077260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T07:55:50.497 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:49.709+0000 7fbabca00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbab8079160 con 0x7fbab8078d50 2024-03-29T07:55:50.497 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:49.710+0000 7fbab5e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbab8078d50 0x7fbab8077260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T07:55:50.497 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:49.710+0000 7fbab5e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbab8078d50 0x7fbab8077260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52486/0 (socket says 172.21.15.37:52486) 2024-03-29T07:55:50.497 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:49.710+0000 7fbab5e00700 1 -- 172.21.15.37:0/2523427058 learned_addr learned my addr 172.21.15.37:0/2523427058 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T07:55:50.498 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:49.710+0000 7fbab5e00700 1 -- 172.21.15.37:0/2523427058 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbab80777a0 con 0x7fbab8078d50 2024-03-29T07:55:50.498 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:49.710+0000 7fbab5e00700 1 --2- 172.21.15.37:0/2523427058 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbab8078d50 0x7fbab8077260 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fbaa80098e0 tx=0x7fbaa8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f2224d078b5b75b server_cookie=0 in_seq=0 out_seq=0 2024-03-29T07:55:50.498 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:49.711+0000 7fbab4a00700 1 -- 172.21.15.37:0/2523427058 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbaa800b860 con 0x7fbab8078d50 2024-03-29T07:55:50.498 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:49.711+0000 7fbab4a00700 1 -- 172.21.15.37:0/2523427058 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1009+0+0 (secure 0 0 0) 0x7fbaa800be30 con 0x7fbab8078d50 2024-03-29T07:55:50.498 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:49.711+0000 7fbab4a00700 1 -- 172.21.15.37:0/2523427058 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbaa8010380 con 0x7fbab8078d50 2024-03-29T07:55:50.498 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:49.711+0000 7fbabca00700 1 -- 172.21.15.37:0/2523427058 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbab8078d50 msgr2=0x7fbab8077260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T07:55:50.498 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:49.711+0000 7fbabca00700 1 --2- 172.21.15.37:0/2523427058 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbab8078d50 0x7fbab8077260 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fbaa80098e0 tx=0x7fbaa8009250 comp rx=0 tx=0).stop 2024-03-29T07:55:50.498 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:49.711+0000 7fbabca00700 1 -- 172.21.15.37:0/2523427058 shutdown_connections 2024-03-29T07:55:50.498 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:49.711+0000 7fbabca00700 1 --2- 172.21.15.37:0/2523427058 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbab8078d50 0x7fbab8077260 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T07:55:50.498 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:49.711+0000 7fbabca00700 1 -- 172.21.15.37:0/2523427058 >> 172.21.15.37:0/2523427058 conn(0x7fbab80fdd00 msgr2=0x7fbab8100110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T07:55:50.498 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:49.711+0000 7fbabca00700 1 -- 172.21.15.37:0/2523427058 shutdown_connections 2024-03-29T07:55:50.498 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:49.711+0000 7fbabca00700 1 -- 172.21.15.37:0/2523427058 wait complete. 2024-03-29T07:55:50.498 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:49.712+0000 7fbabca00700 1 Processor -- start 2024-03-29T07:55:50.498 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:49.712+0000 7fbabca00700 1 -- start start 2024-03-29T07:55:50.499 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:49.713+0000 7fbabca00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbab8078d50 0x7fbab81135a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T07:55:50.499 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:49.713+0000 7fbabca00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbab8113ae0 con 0x7fbab8078d50 2024-03-29T07:55:50.499 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:49.713+0000 7fbab5e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbab8078d50 0x7fbab81135a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T07:55:50.499 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:49.713+0000 7fbab5e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbab8078d50 0x7fbab81135a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52494/0 (socket says 172.21.15.37:52494) 2024-03-29T07:55:50.499 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:49.713+0000 7fbab5e00700 1 -- 172.21.15.37:0/4142364609 learned_addr learned my addr 172.21.15.37:0/4142364609 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T07:55:50.499 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:49.713+0000 7fbab5e00700 1 -- 172.21.15.37:0/4142364609 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbaa80094f0 con 0x7fbab8078d50 2024-03-29T07:55:50.499 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:49.713+0000 7fbab5e00700 1 --2- 172.21.15.37:0/4142364609 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbab8078d50 0x7fbab81135a0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fbaa8006fd0 tx=0x7fbaa8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T07:55:50.499 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:49.714+0000 7fbaae000700 1 -- 172.21.15.37:0/4142364609 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbaa8010750 con 0x7fbab8078d50 2024-03-29T07:55:50.499 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:49.714+0000 7fbaae000700 1 -- 172.21.15.37:0/4142364609 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1009+0+0 (secure 0 0 0) 0x7fbaa8010d20 con 0x7fbab8078d50 2024-03-29T07:55:50.499 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:49.714+0000 7fbabca00700 1 -- 172.21.15.37:0/4142364609 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbab8113ce0 con 0x7fbab8078d50 2024-03-29T07:55:50.499 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:49.714+0000 7fbaae000700 1 -- 172.21.15.37:0/4142364609 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbaa801ada0 con 0x7fbab8078d50 2024-03-29T07:55:50.499 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:49.714+0000 7fbabca00700 1 -- 172.21.15.37:0/4142364609 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbab8114850 con 0x7fbab8078d50 2024-03-29T07:55:50.499 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:49.715+0000 7fbaae000700 1 -- 172.21.15.37:0/4142364609 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 4) v1 ==== 43357+0+0 (secure 0 0 0) 0x7fbaa80108b0 con 0x7fbab8078d50 2024-03-29T07:55:50.499 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:49.715+0000 7fbabca00700 1 -- 172.21.15.37:0/4142364609 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbab805f5f0 con 0x7fbab8078d50 2024-03-29T07:55:50.500 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:49.715+0000 7fbaae000700 1 --2- 172.21.15.37:0/4142364609 >> [v2:172.21.15.37:6800/2762339582,v1:172.21.15.37:6801/2762339582] conn(0x7fba9c036790 0x7fba9c038c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T07:55:50.500 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:49.715+0000 7fbaae000700 1 -- 172.21.15.37:0/4142364609 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7fbaa8049ca0 con 0x7fbab8078d50 2024-03-29T07:55:50.500 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:49.716+0000 7fbab5400700 1 --2- 172.21.15.37:0/4142364609 >> [v2:172.21.15.37:6800/2762339582,v1:172.21.15.37:6801/2762339582] conn(0x7fba9c036790 0x7fba9c038c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T07:55:50.500 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:49.716+0000 7fbab5400700 1 --2- 172.21.15.37:0/4142364609 >> [v2:172.21.15.37:6800/2762339582,v1:172.21.15.37:6801/2762339582] conn(0x7fba9c036790 0x7fba9c038c40 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fbaa4009990 tx=0x7fbaa4006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T07:55:50.500 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:49.720+0000 7fbaae000700 1 -- 172.21.15.37:0/4142364609 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fbaa801a5d0 con 0x7fbab8078d50 2024-03-29T07:55:50.500 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:49.925+0000 7fbabca00700 1 -- 172.21.15.37:0/4142364609 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7fbab804bed0 con 0x7fbab8078d50 2024-03-29T07:55:50.500 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:49.926+0000 7fbaae000700 1 -- 172.21.15.37:0/4142364609 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) v1 ==== 70+0+373 (secure 0 0 0) 0x7fbaa8020020 con 0x7fbab8078d50 2024-03-29T07:55:50.500 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:49.929+0000 7fbabca00700 1 -- 172.21.15.37:0/4142364609 >> [v2:172.21.15.37:6800/2762339582,v1:172.21.15.37:6801/2762339582] conn(0x7fba9c036790 msgr2=0x7fba9c038c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T07:55:50.500 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:49.929+0000 7fbabca00700 1 --2- 172.21.15.37:0/4142364609 >> [v2:172.21.15.37:6800/2762339582,v1:172.21.15.37:6801/2762339582] conn(0x7fba9c036790 0x7fba9c038c40 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fbaa4009990 tx=0x7fbaa4006e40 comp rx=0 tx=0).stop 2024-03-29T07:55:50.500 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:49.929+0000 7fbabca00700 1 -- 172.21.15.37:0/4142364609 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbab8078d50 msgr2=0x7fbab81135a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T07:55:50.500 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:49.929+0000 7fbabca00700 1 --2- 172.21.15.37:0/4142364609 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbab8078d50 0x7fbab81135a0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fbaa8006fd0 tx=0x7fbaa8009250 comp rx=0 tx=0).stop 2024-03-29T07:55:50.500 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:49.929+0000 7fbabca00700 1 -- 172.21.15.37:0/4142364609 shutdown_connections 2024-03-29T07:55:50.500 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:49.929+0000 7fbabca00700 1 --2- 172.21.15.37:0/4142364609 >> [v2:172.21.15.37:6800/2762339582,v1:172.21.15.37:6801/2762339582] conn(0x7fba9c036790 0x7fba9c038c40 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T07:55:50.500 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:49.929+0000 7fbabca00700 1 --2- 172.21.15.37:0/4142364609 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbab8078d50 0x7fbab81135a0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T07:55:50.500 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:49.929+0000 7fbabca00700 1 -- 172.21.15.37:0/4142364609 >> 172.21.15.37:0/4142364609 conn(0x7fbab80fdd00 msgr2=0x7fbab80fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T07:55:50.501 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:49.929+0000 7fbabca00700 1 -- 172.21.15.37:0/4142364609 shutdown_connections 2024-03-29T07:55:50.501 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:49.929+0000 7fbabca00700 1 -- 172.21.15.37:0/4142364609 wait complete. 2024-03-29T07:55:50.501 INFO:teuthology.orchestra.run.smithi037.stdout:Enabling cephadm module... 2024-03-29T07:55:51.649 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:51 smithi037 ceph-mon[35162]: from='client.? 172.21.15.37:0/460923479' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-03-29T07:55:52.075 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:51.068+0000 7f9786a00700 1 Processor -- start 2024-03-29T07:55:52.075 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:51.068+0000 7f9786a00700 1 -- start start 2024-03-29T07:55:52.075 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:51.068+0000 7f9786a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9780078d60 0x7f9780077260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T07:55:52.075 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:51.068+0000 7f9786a00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9780079170 con 0x7f9780078d60 2024-03-29T07:55:52.075 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:51.069+0000 7f977fe00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9780078d60 0x7f9780077260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T07:55:52.076 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:51.069+0000 7f977fe00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9780078d60 0x7f9780077260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55880/0 (socket says 172.21.15.37:55880) 2024-03-29T07:55:52.077 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:51.069+0000 7f977fe00700 1 -- 172.21.15.37:0/437451342 learned_addr learned my addr 172.21.15.37:0/437451342 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T07:55:52.077 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:51.069+0000 7f977fe00700 1 -- 172.21.15.37:0/437451342 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97800777a0 con 0x7f9780078d60 2024-03-29T07:55:52.077 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:51.069+0000 7f977fe00700 1 --2- 172.21.15.37:0/437451342 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9780078d60 0x7f9780077260 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f97700098e0 tx=0x7f9770009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50df10c706120813 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T07:55:52.077 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:51.069+0000 7f977ea00700 1 -- 172.21.15.37:0/437451342 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f977000b860 con 0x7f9780078d60 2024-03-29T07:55:52.077 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:51.070+0000 7f977ea00700 1 -- 172.21.15.37:0/437451342 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1009+0+0 (secure 0 0 0) 0x7f977000be30 con 0x7f9780078d60 2024-03-29T07:55:52.077 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:51.070+0000 7f977ea00700 1 -- 172.21.15.37:0/437451342 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9770010380 con 0x7f9780078d60 2024-03-29T07:55:52.077 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:51.070+0000 7f9786a00700 1 -- 172.21.15.37:0/437451342 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9780078d60 msgr2=0x7f9780077260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T07:55:52.077 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:51.070+0000 7f9786a00700 1 --2- 172.21.15.37:0/437451342 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9780078d60 0x7f9780077260 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f97700098e0 tx=0x7f9770009250 comp rx=0 tx=0).stop 2024-03-29T07:55:52.077 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:51.070+0000 7f9786a00700 1 -- 172.21.15.37:0/437451342 shutdown_connections 2024-03-29T07:55:52.078 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:51.070+0000 7f9786a00700 1 --2- 172.21.15.37:0/437451342 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9780078d60 0x7f9780077260 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T07:55:52.078 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:51.070+0000 7f9786a00700 1 -- 172.21.15.37:0/437451342 >> 172.21.15.37:0/437451342 conn(0x7f97800fdcd0 msgr2=0x7f9780100100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T07:55:52.078 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:51.070+0000 7f9786a00700 1 -- 172.21.15.37:0/437451342 shutdown_connections 2024-03-29T07:55:52.078 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:51.070+0000 7f9786a00700 1 -- 172.21.15.37:0/437451342 wait complete. 2024-03-29T07:55:52.078 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:51.071+0000 7f9786a00700 1 Processor -- start 2024-03-29T07:55:52.078 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:51.071+0000 7f9786a00700 1 -- start start 2024-03-29T07:55:52.078 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:51.072+0000 7f9786a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9780078d60 0x7f9780113630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T07:55:52.078 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:51.072+0000 7f9786a00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9780113b70 con 0x7f9780078d60 2024-03-29T07:55:52.078 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:51.072+0000 7f977fe00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9780078d60 0x7f9780113630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T07:55:52.078 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:51.072+0000 7f977fe00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9780078d60 0x7f9780113630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55886/0 (socket says 172.21.15.37:55886) 2024-03-29T07:55:52.078 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:51.072+0000 7f977fe00700 1 -- 172.21.15.37:0/460923479 learned_addr learned my addr 172.21.15.37:0/460923479 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T07:55:52.078 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:51.072+0000 7f977fe00700 1 -- 172.21.15.37:0/460923479 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97700094f0 con 0x7f9780078d60 2024-03-29T07:55:52.078 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:51.073+0000 7f977fe00700 1 --2- 172.21.15.37:0/460923479 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9780078d60 0x7f9780113630 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f9770006b00 tx=0x7f9770009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T07:55:52.079 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:51.073+0000 7f9777e00700 1 -- 172.21.15.37:0/460923479 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9770010750 con 0x7f9780078d60 2024-03-29T07:55:52.079 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:51.073+0000 7f9777e00700 1 -- 172.21.15.37:0/460923479 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1009+0+0 (secure 0 0 0) 0x7f9770010d20 con 0x7f9780078d60 2024-03-29T07:55:52.079 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:51.073+0000 7f9777e00700 1 -- 172.21.15.37:0/460923479 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f977001acb0 con 0x7f9780078d60 2024-03-29T07:55:52.079 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:51.073+0000 7f9786a00700 1 -- 172.21.15.37:0/460923479 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9780113d70 con 0x7f9780078d60 2024-03-29T07:55:52.079 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:51.073+0000 7f9786a00700 1 -- 172.21.15.37:0/460923479 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97801148e0 con 0x7f9780078d60 2024-03-29T07:55:52.079 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:51.074+0000 7f9777e00700 1 -- 172.21.15.37:0/460923479 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 4) v1 ==== 43357+0+0 (secure 0 0 0) 0x7f977001a430 con 0x7f9780078d60 2024-03-29T07:55:52.079 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:51.074+0000 7f9786a00700 1 -- 172.21.15.37:0/460923479 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f978005f5f0 con 0x7f9780078d60 2024-03-29T07:55:52.079 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:51.074+0000 7f9777e00700 1 --2- 172.21.15.37:0/460923479 >> [v2:172.21.15.37:6800/2762339582,v1:172.21.15.37:6801/2762339582] conn(0x7f9764036790 0x7f9764038c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T07:55:52.079 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:51.074+0000 7f9777e00700 1 -- 172.21.15.37:0/460923479 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f977001e070 con 0x7f9780078d60 2024-03-29T07:55:52.079 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:51.075+0000 7f977f400700 1 --2- 172.21.15.37:0/460923479 >> [v2:172.21.15.37:6800/2762339582,v1:172.21.15.37:6801/2762339582] conn(0x7f9764036790 0x7f9764038c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T07:55:52.079 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:51.075+0000 7f977f400700 1 --2- 172.21.15.37:0/460923479 >> [v2:172.21.15.37:6800/2762339582,v1:172.21.15.37:6801/2762339582] conn(0x7f9764036790 0x7f9764038c40 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f976c009990 tx=0x7f976c006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T07:55:52.079 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:51.080+0000 7f9777e00700 1 -- 172.21.15.37:0/460923479 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f97700187c0 con 0x7f9780078d60 2024-03-29T07:55:52.079 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:51.338+0000 7f9786a00700 1 -- 172.21.15.37:0/460923479 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) v1 -- 0x7f978004bed0 con 0x7f9780078d60 2024-03-29T07:55:52.079 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:51.391+0000 7f9777e00700 1 -- 172.21.15.37:0/460923479 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mgrmap(e 5) v1 ==== 43368+0+0 (secure 0 0 0) 0x7f977002b4e0 con 0x7f9780078d60 2024-03-29T07:55:52.079 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:51.393+0000 7f9777e00700 1 -- 172.21.15.37:0/460923479 <== mon.0 v2:172.21.15.37:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) v1 ==== 86+0+0 (secure 0 0 0) 0x7f9770019ab0 con 0x7f9780078d60 2024-03-29T07:55:52.080 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:51.396+0000 7f9786a00700 1 -- 172.21.15.37:0/460923479 >> [v2:172.21.15.37:6800/2762339582,v1:172.21.15.37:6801/2762339582] conn(0x7f9764036790 msgr2=0x7f9764038c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T07:55:52.080 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:51.396+0000 7f9786a00700 1 --2- 172.21.15.37:0/460923479 >> [v2:172.21.15.37:6800/2762339582,v1:172.21.15.37:6801/2762339582] conn(0x7f9764036790 0x7f9764038c40 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f976c009990 tx=0x7f976c006e40 comp rx=0 tx=0).stop 2024-03-29T07:55:52.080 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:51.396+0000 7f9786a00700 1 -- 172.21.15.37:0/460923479 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9780078d60 msgr2=0x7f9780113630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T07:55:52.080 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:51.396+0000 7f9786a00700 1 --2- 172.21.15.37:0/460923479 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9780078d60 0x7f9780113630 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f9770006b00 tx=0x7f9770009250 comp rx=0 tx=0).stop 2024-03-29T07:55:52.080 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:51.396+0000 7f9786a00700 1 -- 172.21.15.37:0/460923479 shutdown_connections 2024-03-29T07:55:52.080 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:51.396+0000 7f9786a00700 1 --2- 172.21.15.37:0/460923479 >> [v2:172.21.15.37:6800/2762339582,v1:172.21.15.37:6801/2762339582] conn(0x7f9764036790 0x7f9764038c40 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T07:55:52.080 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:51.396+0000 7f9786a00700 1 --2- 172.21.15.37:0/460923479 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9780078d60 0x7f9780113630 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T07:55:52.080 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:51.396+0000 7f9786a00700 1 -- 172.21.15.37:0/460923479 >> 172.21.15.37:0/460923479 conn(0x7f97800fdcd0 msgr2=0x7f97800fe8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T07:55:52.080 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:51.397+0000 7f9786a00700 1 -- 172.21.15.37:0/460923479 shutdown_connections 2024-03-29T07:55:52.080 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:51.397+0000 7f9786a00700 1 -- 172.21.15.37:0/460923479 wait complete. 2024-03-29T07:55:52.676 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:52 smithi037 ceph-mon[35162]: from='client.? 172.21.15.37:0/460923479' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-03-29T07:55:52.676 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:52 smithi037 ceph-mon[35162]: mgrmap e5: smithi037.wjikii(active, since 8s) 2024-03-29T07:55:53.540 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout { 2024-03-29T07:55:53.540 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-03-29T07:55:53.540 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "available": true, 2024-03-29T07:55:53.541 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "active_name": "smithi037.wjikii", 2024-03-29T07:55:53.541 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-03-29T07:55:53.541 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout } 2024-03-29T07:55:53.541 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:52.647+0000 7f9735e00700 1 Processor -- start 2024-03-29T07:55:53.541 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:52.647+0000 7f9735e00700 1 -- start start 2024-03-29T07:55:53.541 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:52.648+0000 7f9735e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9730078d50 0x7f9730077260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T07:55:53.541 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:52.648+0000 7f9735e00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9730079160 con 0x7f9730078d50 2024-03-29T07:55:53.541 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:52.648+0000 7f972ea00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9730078d50 0x7f9730077260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T07:55:53.541 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:52.648+0000 7f972ea00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9730078d50 0x7f9730077260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55916/0 (socket says 172.21.15.37:55916) 2024-03-29T07:55:53.541 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:52.648+0000 7f972ea00700 1 -- 172.21.15.37:0/2396843185 learned_addr learned my addr 172.21.15.37:0/2396843185 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T07:55:53.541 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:52.649+0000 7f972ea00700 1 -- 172.21.15.37:0/2396843185 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97300777a0 con 0x7f9730078d50 2024-03-29T07:55:53.541 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:52.649+0000 7f972ea00700 1 --2- 172.21.15.37:0/2396843185 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9730078d50 0x7f9730077260 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f97200098e0 tx=0x7f9720009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de649d34458ac349 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T07:55:53.541 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:52.649+0000 7f972d600700 1 -- 172.21.15.37:0/2396843185 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f972000b860 con 0x7f9730078d50 2024-03-29T07:55:53.542 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:52.649+0000 7f972d600700 1 -- 172.21.15.37:0/2396843185 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1009+0+0 (secure 0 0 0) 0x7f972000be30 con 0x7f9730078d50 2024-03-29T07:55:53.542 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:52.649+0000 7f972d600700 1 -- 172.21.15.37:0/2396843185 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9720010380 con 0x7f9730078d50 2024-03-29T07:55:53.542 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:52.650+0000 7f9735e00700 1 -- 172.21.15.37:0/2396843185 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9730078d50 msgr2=0x7f9730077260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T07:55:53.542 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:52.650+0000 7f9735e00700 1 --2- 172.21.15.37:0/2396843185 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9730078d50 0x7f9730077260 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f97200098e0 tx=0x7f9720009250 comp rx=0 tx=0).stop 2024-03-29T07:55:53.542 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:52.650+0000 7f9735e00700 1 -- 172.21.15.37:0/2396843185 shutdown_connections 2024-03-29T07:55:53.542 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:52.650+0000 7f9735e00700 1 --2- 172.21.15.37:0/2396843185 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9730078d50 0x7f9730077260 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T07:55:53.542 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:52.650+0000 7f9735e00700 1 -- 172.21.15.37:0/2396843185 >> 172.21.15.37:0/2396843185 conn(0x7f97300fdd00 msgr2=0x7f9730100110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T07:55:53.542 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:52.650+0000 7f9735e00700 1 -- 172.21.15.37:0/2396843185 shutdown_connections 2024-03-29T07:55:53.542 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:52.650+0000 7f9735e00700 1 -- 172.21.15.37:0/2396843185 wait complete. 2024-03-29T07:55:53.542 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:52.651+0000 7f9735e00700 1 Processor -- start 2024-03-29T07:55:53.542 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:52.652+0000 7f9735e00700 1 -- start start 2024-03-29T07:55:53.542 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:52.652+0000 7f9735e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9730078d50 0x7f97301135a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T07:55:53.543 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:52.652+0000 7f9735e00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9730113ae0 con 0x7f9730078d50 2024-03-29T07:55:53.543 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:52.652+0000 7f972ea00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9730078d50 0x7f97301135a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T07:55:53.543 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:52.652+0000 7f972ea00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9730078d50 0x7f97301135a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55930/0 (socket says 172.21.15.37:55930) 2024-03-29T07:55:53.543 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:52.652+0000 7f972ea00700 1 -- 172.21.15.37:0/3787425386 learned_addr learned my addr 172.21.15.37:0/3787425386 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T07:55:53.543 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:52.653+0000 7f972ea00700 1 -- 172.21.15.37:0/3787425386 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97200094f0 con 0x7f9730078d50 2024-03-29T07:55:53.543 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:52.653+0000 7f972ea00700 1 --2- 172.21.15.37:0/3787425386 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9730078d50 0x7f97301135a0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f9720006fd0 tx=0x7f9720009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T07:55:53.543 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:52.653+0000 7f9726a00700 1 -- 172.21.15.37:0/3787425386 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9720010750 con 0x7f9730078d50 2024-03-29T07:55:53.543 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:52.653+0000 7f9726a00700 1 -- 172.21.15.37:0/3787425386 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1009+0+0 (secure 0 0 0) 0x7f9720010d20 con 0x7f9730078d50 2024-03-29T07:55:53.543 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:52.653+0000 7f9726a00700 1 -- 172.21.15.37:0/3787425386 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f972001ada0 con 0x7f9730078d50 2024-03-29T07:55:53.543 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:52.653+0000 7f9735e00700 1 -- 172.21.15.37:0/3787425386 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9730113ce0 con 0x7f9730078d50 2024-03-29T07:55:53.543 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:52.653+0000 7f9735e00700 1 -- 172.21.15.37:0/3787425386 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9730114850 con 0x7f9730078d50 2024-03-29T07:55:53.543 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:52.654+0000 7f9735e00700 1 -- 172.21.15.37:0/3787425386 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f973005f5f0 con 0x7f9730078d50 2024-03-29T07:55:53.543 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:52.655+0000 7f9726a00700 1 -- 172.21.15.37:0/3787425386 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 43368+0+0 (secure 0 0 0) 0x7f97200108b0 con 0x7f9730078d50 2024-03-29T07:55:53.543 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:52.655+0000 7f9726a00700 1 --2- 172.21.15.37:0/3787425386 >> [v2:172.21.15.37:6800/2762339582,v1:172.21.15.37:6801/2762339582] conn(0x7f97140367e0 0x7f9714038c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T07:55:53.544 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:52.655+0000 7f9726a00700 1 -- 172.21.15.37:0/3787425386 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f97200499d0 con 0x7f9730078d50 2024-03-29T07:55:53.544 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:52.655+0000 7f972e000700 1 -- 172.21.15.37:0/3787425386 >> [v2:172.21.15.37:6800/2762339582,v1:172.21.15.37:6801/2762339582] conn(0x7f97140367e0 msgr2=0x7f9714038c90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/2762339582 2024-03-29T07:55:53.544 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:52.655+0000 7f972e000700 1 --2- 172.21.15.37:0/3787425386 >> [v2:172.21.15.37:6800/2762339582,v1:172.21.15.37:6801/2762339582] conn(0x7f97140367e0 0x7f9714038c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-29T07:55:53.544 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:52.664+0000 7f9726a00700 1 -- 172.21.15.37:0/3787425386 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f972001a5d0 con 0x7f9730078d50 2024-03-29T07:55:53.544 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:52.856+0000 7f972e000700 1 -- 172.21.15.37:0/3787425386 >> [v2:172.21.15.37:6800/2762339582,v1:172.21.15.37:6801/2762339582] conn(0x7f97140367e0 msgr2=0x7f9714038c90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/2762339582 2024-03-29T07:55:53.544 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:52.856+0000 7f972e000700 1 --2- 172.21.15.37:0/3787425386 >> [v2:172.21.15.37:6800/2762339582,v1:172.21.15.37:6801/2762339582] conn(0x7f97140367e0 0x7f9714038c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-29T07:55:53.544 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:52.924+0000 7f9735e00700 1 -- 172.21.15.37:0/3787425386 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f9730114b90 con 0x7f9730078d50 2024-03-29T07:55:53.544 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:52.924+0000 7f9726a00700 1 -- 172.21.15.37:0/3787425386 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) v1 ==== 56+0+103 (secure 0 0 0) 0x7f9720029070 con 0x7f9730078d50 2024-03-29T07:55:53.544 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:52.927+0000 7f9735e00700 1 -- 172.21.15.37:0/3787425386 >> [v2:172.21.15.37:6800/2762339582,v1:172.21.15.37:6801/2762339582] conn(0x7f97140367e0 msgr2=0x7f9714038c90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-29T07:55:53.544 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:52.927+0000 7f9735e00700 1 --2- 172.21.15.37:0/3787425386 >> [v2:172.21.15.37:6800/2762339582,v1:172.21.15.37:6801/2762339582] conn(0x7f97140367e0 0x7f9714038c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T07:55:53.544 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:52.927+0000 7f9735e00700 1 -- 172.21.15.37:0/3787425386 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9730078d50 msgr2=0x7f97301135a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T07:55:53.544 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:52.927+0000 7f9735e00700 1 --2- 172.21.15.37:0/3787425386 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9730078d50 0x7f97301135a0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f9720006fd0 tx=0x7f9720009250 comp rx=0 tx=0).stop 2024-03-29T07:55:53.544 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:52.927+0000 7f9735e00700 1 -- 172.21.15.37:0/3787425386 shutdown_connections 2024-03-29T07:55:53.544 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:52.927+0000 7f9735e00700 1 --2- 172.21.15.37:0/3787425386 >> [v2:172.21.15.37:6800/2762339582,v1:172.21.15.37:6801/2762339582] conn(0x7f97140367e0 0x7f9714038c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T07:55:53.545 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:52.927+0000 7f9735e00700 1 --2- 172.21.15.37:0/3787425386 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9730078d50 0x7f97301135a0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T07:55:53.545 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:52.927+0000 7f9735e00700 1 -- 172.21.15.37:0/3787425386 >> 172.21.15.37:0/3787425386 conn(0x7f97300fdd00 msgr2=0x7f97300fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T07:55:53.545 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:52.927+0000 7f9735e00700 1 -- 172.21.15.37:0/3787425386 shutdown_connections 2024-03-29T07:55:53.545 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:52.927+0000 7f9735e00700 1 -- 172.21.15.37:0/3787425386 wait complete. 2024-03-29T07:55:53.545 INFO:teuthology.orchestra.run.smithi037.stdout:Waiting for the mgr to restart... 2024-03-29T07:55:53.545 INFO:teuthology.orchestra.run.smithi037.stdout:Waiting for mgr epoch 5... 2024-03-29T07:55:53.545 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:55:53 smithi037 ceph-mon[35162]: from='client.? 172.21.15.37:0/3787425386' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-03-29T07:56:00.176 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:56:00 smithi037 ceph-mon[35162]: Active manager daemon smithi037.wjikii restarted 2024-03-29T07:56:00.176 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:56:00 smithi037 ceph-mon[35162]: Activating manager daemon smithi037.wjikii 2024-03-29T07:56:00.176 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:56:00 smithi037 ceph-mon[35162]: osdmap e2: 0 total, 0 up, 0 in 2024-03-29T07:56:00.176 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:56:00 smithi037 ceph-mon[35162]: mgrmap e6: smithi037.wjikii(active, starting, since 0.00367972s) 2024-03-29T07:56:00.176 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:56:00 smithi037 ceph-mon[35162]: from='mgr.14118 172.21.15.37:0/2997715107' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "mon metadata", "id": "smithi037"}]: dispatch 2024-03-29T07:56:00.176 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:56:00 smithi037 ceph-mon[35162]: from='mgr.14118 172.21.15.37:0/2997715107' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "mgr metadata", "who": "smithi037.wjikii", "id": "smithi037.wjikii"}]: dispatch 2024-03-29T07:56:00.176 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:56:00 smithi037 ceph-mon[35162]: from='mgr.14118 172.21.15.37:0/2997715107' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-29T07:56:00.176 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:56:00 smithi037 ceph-mon[35162]: from='mgr.14118 172.21.15.37:0/2997715107' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-29T07:56:00.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:56:00 smithi037 ceph-mon[35162]: from='mgr.14118 172.21.15.37:0/2997715107' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-29T07:56:00.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:56:00 smithi037 ceph-mon[35162]: Manager daemon smithi037.wjikii is now available 2024-03-29T07:56:00.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:56:00 smithi037 ceph-mon[35162]: from='mgr.14118 172.21.15.37:0/2997715107' entity='mgr.smithi037.wjikii' 2024-03-29T07:56:00.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:56:00 smithi037 ceph-mon[35162]: from='mgr.14118 172.21.15.37:0/2997715107' entity='mgr.smithi037.wjikii' 2024-03-29T07:56:00.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:56:00 smithi037 ceph-mon[35162]: from='mgr.14118 172.21.15.37:0/2997715107' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T07:56:01.360 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:56:01 smithi037 ceph-mon[35162]: Found migration_current of "None". Setting to last migration. 2024-03-29T07:56:01.360 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:56:01 smithi037 ceph-mon[35162]: from='mgr.14118 172.21.15.37:0/2997715107' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T07:56:01.361 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:56:01 smithi037 ceph-mon[35162]: from='mgr.14118 172.21.15.37:0/2997715107' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T07:56:01.361 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:56:01 smithi037 ceph-mon[35162]: from='mgr.14118 172.21.15.37:0/2997715107' entity='mgr.smithi037.wjikii' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi037.wjikii/mirror_snapshot_schedule"}]: dispatch 2024-03-29T07:56:01.361 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:56:01 smithi037 ceph-mon[35162]: from='mgr.14118 172.21.15.37:0/2997715107' entity='mgr.smithi037.wjikii' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi037.wjikii/trash_purge_schedule"}]: dispatch 2024-03-29T07:56:01.361 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:56:01 smithi037 ceph-mon[35162]: mgrmap e7: smithi037.wjikii(active, since 1.00614s) 2024-03-29T07:56:01.648 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout { 2024-03-29T07:56:01.649 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-03-29T07:56:01.649 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "initialized": true 2024-03-29T07:56:01.649 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout } 2024-03-29T07:56:01.649 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:54.072+0000 7fa066200700 1 Processor -- start 2024-03-29T07:56:01.649 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:54.072+0000 7fa066200700 1 -- start start 2024-03-29T07:56:01.649 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:54.072+0000 7fa066200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa060078d90 0x7fa0600772a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T07:56:01.649 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:54.072+0000 7fa066200700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0600791a0 con 0x7fa060078d90 2024-03-29T07:56:01.649 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:54.073+0000 7fa05f400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa060078d90 0x7fa0600772a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T07:56:01.649 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:54.073+0000 7fa05f400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa060078d90 0x7fa0600772a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55934/0 (socket says 172.21.15.37:55934) 2024-03-29T07:56:01.649 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:54.073+0000 7fa05f400700 1 -- 172.21.15.37:0/2994877435 learned_addr learned my addr 172.21.15.37:0/2994877435 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T07:56:01.649 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:54.073+0000 7fa05f400700 1 -- 172.21.15.37:0/2994877435 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0600777e0 con 0x7fa060078d90 2024-03-29T07:56:01.649 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:54.073+0000 7fa05f400700 1 --2- 172.21.15.37:0/2994877435 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa060078d90 0x7fa0600772a0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fa0500098e0 tx=0x7fa050009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=993ba9db532cd4ac server_cookie=0 in_seq=0 out_seq=0 2024-03-29T07:56:01.650 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:54.074+0000 7fa05e000700 1 -- 172.21.15.37:0/2994877435 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa05000b860 con 0x7fa060078d90 2024-03-29T07:56:01.650 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:54.074+0000 7fa05e000700 1 -- 172.21.15.37:0/2994877435 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1009+0+0 (secure 0 0 0) 0x7fa05000be30 con 0x7fa060078d90 2024-03-29T07:56:01.650 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:54.074+0000 7fa05e000700 1 -- 172.21.15.37:0/2994877435 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa050010380 con 0x7fa060078d90 2024-03-29T07:56:01.650 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:54.074+0000 7fa066200700 1 -- 172.21.15.37:0/2994877435 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa060078d90 msgr2=0x7fa0600772a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T07:56:01.650 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:54.074+0000 7fa066200700 1 --2- 172.21.15.37:0/2994877435 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa060078d90 0x7fa0600772a0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fa0500098e0 tx=0x7fa050009250 comp rx=0 tx=0).stop 2024-03-29T07:56:01.650 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:54.075+0000 7fa066200700 1 -- 172.21.15.37:0/2994877435 shutdown_connections 2024-03-29T07:56:01.650 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:54.075+0000 7fa066200700 1 --2- 172.21.15.37:0/2994877435 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa060078d90 0x7fa0600772a0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T07:56:01.650 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:54.075+0000 7fa066200700 1 -- 172.21.15.37:0/2994877435 >> 172.21.15.37:0/2994877435 conn(0x7fa0600fdd00 msgr2=0x7fa060100110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T07:56:01.650 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:54.075+0000 7fa066200700 1 -- 172.21.15.37:0/2994877435 shutdown_connections 2024-03-29T07:56:01.650 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:54.075+0000 7fa066200700 1 -- 172.21.15.37:0/2994877435 wait complete. 2024-03-29T07:56:01.650 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:54.076+0000 7fa066200700 1 Processor -- start 2024-03-29T07:56:01.651 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:54.076+0000 7fa066200700 1 -- start start 2024-03-29T07:56:01.651 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:54.077+0000 7fa066200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa060078d90 0x7fa0601179c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T07:56:01.651 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:54.077+0000 7fa066200700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa060117f00 con 0x7fa060078d90 2024-03-29T07:56:01.651 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:54.077+0000 7fa05f400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa060078d90 0x7fa0601179c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T07:56:01.651 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:54.077+0000 7fa05f400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa060078d90 0x7fa0601179c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55942/0 (socket says 172.21.15.37:55942) 2024-03-29T07:56:01.651 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:54.077+0000 7fa05f400700 1 -- 172.21.15.37:0/2373658828 learned_addr learned my addr 172.21.15.37:0/2373658828 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T07:56:01.651 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:54.077+0000 7fa05f400700 1 -- 172.21.15.37:0/2373658828 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0500094f0 con 0x7fa060078d90 2024-03-29T07:56:01.651 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:54.077+0000 7fa05f400700 1 --2- 172.21.15.37:0/2373658828 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa060078d90 0x7fa0601179c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fa050006fd0 tx=0x7fa050009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T07:56:01.651 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:54.078+0000 7fa057400700 1 -- 172.21.15.37:0/2373658828 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa050010750 con 0x7fa060078d90 2024-03-29T07:56:01.651 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:54.078+0000 7fa057400700 1 -- 172.21.15.37:0/2373658828 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1009+0+0 (secure 0 0 0) 0x7fa050010d20 con 0x7fa060078d90 2024-03-29T07:56:01.651 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:54.078+0000 7fa057400700 1 -- 172.21.15.37:0/2373658828 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa05001ada0 con 0x7fa060078d90 2024-03-29T07:56:01.651 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:54.078+0000 7fa066200700 1 -- 172.21.15.37:0/2373658828 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa060118100 con 0x7fa060078d90 2024-03-29T07:56:01.651 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:54.078+0000 7fa066200700 1 -- 172.21.15.37:0/2373658828 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa060118c70 con 0x7fa060078d90 2024-03-29T07:56:01.651 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:54.079+0000 7fa057400700 1 -- 172.21.15.37:0/2373658828 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 43368+0+0 (secure 0 0 0) 0x7fa0500193c0 con 0x7fa060078d90 2024-03-29T07:56:01.651 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:54.079+0000 7fa057400700 1 --2- 172.21.15.37:0/2373658828 >> [v2:172.21.15.37:6800/2762339582,v1:172.21.15.37:6801/2762339582] conn(0x7fa0440367e0 0x7fa044038c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T07:56:01.652 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:54.079+0000 7fa057400700 1 -- 172.21.15.37:0/2373658828 --> [v2:172.21.15.37:6800/2762339582,v1:172.21.15.37:6801/2762339582] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa0440393a0 con 0x7fa0440367e0 2024-03-29T07:56:01.652 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:54.079+0000 7fa057400700 1 -- 172.21.15.37:0/2373658828 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7fa050049e50 con 0x7fa060078d90 2024-03-29T07:56:01.652 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:54.079+0000 7fa05ea00700 1 -- 172.21.15.37:0/2373658828 >> [v2:172.21.15.37:6800/2762339582,v1:172.21.15.37:6801/2762339582] conn(0x7fa0440367e0 msgr2=0x7fa044038c90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/2762339582 2024-03-29T07:56:01.652 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:54.079+0000 7fa05ea00700 1 --2- 172.21.15.37:0/2373658828 >> [v2:172.21.15.37:6800/2762339582,v1:172.21.15.37:6801/2762339582] conn(0x7fa0440367e0 0x7fa044038c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-29T07:56:01.652 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:54.280+0000 7fa05ea00700 1 -- 172.21.15.37:0/2373658828 >> [v2:172.21.15.37:6800/2762339582,v1:172.21.15.37:6801/2762339582] conn(0x7fa0440367e0 msgr2=0x7fa044038c90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/2762339582 2024-03-29T07:56:01.652 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:54.280+0000 7fa05ea00700 1 --2- 172.21.15.37:0/2373658828 >> [v2:172.21.15.37:6800/2762339582,v1:172.21.15.37:6801/2762339582] conn(0x7fa0440367e0 0x7fa044038c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-29T07:56:01.652 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:54.681+0000 7fa05ea00700 1 -- 172.21.15.37:0/2373658828 >> [v2:172.21.15.37:6800/2762339582,v1:172.21.15.37:6801/2762339582] conn(0x7fa0440367e0 msgr2=0x7fa044038c90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/2762339582 2024-03-29T07:56:01.652 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:54.681+0000 7fa05ea00700 1 --2- 172.21.15.37:0/2373658828 >> [v2:172.21.15.37:6800/2762339582,v1:172.21.15.37:6801/2762339582] conn(0x7fa0440367e0 0x7fa044038c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-29T07:56:01.652 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:55.482+0000 7fa05ea00700 1 -- 172.21.15.37:0/2373658828 >> [v2:172.21.15.37:6800/2762339582,v1:172.21.15.37:6801/2762339582] conn(0x7fa0440367e0 msgr2=0x7fa044038c90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/2762339582 2024-03-29T07:56:01.652 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:55.482+0000 7fa05ea00700 1 --2- 172.21.15.37:0/2373658828 >> [v2:172.21.15.37:6800/2762339582,v1:172.21.15.37:6801/2762339582] conn(0x7fa0440367e0 0x7fa044038c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-29T07:56:01.652 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:57.083+0000 7fa05ea00700 1 -- 172.21.15.37:0/2373658828 >> [v2:172.21.15.37:6800/2762339582,v1:172.21.15.37:6801/2762339582] conn(0x7fa0440367e0 msgr2=0x7fa044038c90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/2762339582 2024-03-29T07:56:01.652 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:57.083+0000 7fa05ea00700 1 --2- 172.21.15.37:0/2373658828 >> [v2:172.21.15.37:6800/2762339582,v1:172.21.15.37:6801/2762339582] conn(0x7fa0440367e0 0x7fa044038c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-03-29T07:56:01.652 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:59.991+0000 7fa057400700 1 -- 172.21.15.37:0/2373658828 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mgrmap(e 6) v1 ==== 43135+0+0 (secure 0 0 0) 0x7fa050019d20 con 0x7fa060078d90 2024-03-29T07:56:01.653 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:59.991+0000 7fa057400700 1 -- 172.21.15.37:0/2373658828 >> [v2:172.21.15.37:6800/2762339582,v1:172.21.15.37:6801/2762339582] conn(0x7fa0440367e0 msgr2=0x7fa044038c90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-29T07:56:01.653 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:55:59.991+0000 7fa057400700 1 --2- 172.21.15.37:0/2373658828 >> [v2:172.21.15.37:6800/2762339582,v1:172.21.15.37:6801/2762339582] conn(0x7fa0440367e0 0x7fa044038c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T07:56:01.653 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:00.992+0000 7fa057400700 1 -- 172.21.15.37:0/2373658828 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mgrmap(e 7) v1 ==== 43262+0+0 (secure 0 0 0) 0x7fa050019800 con 0x7fa060078d90 2024-03-29T07:56:01.653 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:00.992+0000 7fa057400700 1 --2- 172.21.15.37:0/2373658828 >> [v2:172.21.15.37:6800/2746946658,v1:172.21.15.37:6801/2746946658] conn(0x7fa044039fc0 0x7fa04403c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T07:56:01.653 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:00.992+0000 7fa057400700 1 -- 172.21.15.37:0/2373658828 --> [v2:172.21.15.37:6800/2746946658,v1:172.21.15.37:6801/2746946658] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa0440393a0 con 0x7fa044039fc0 2024-03-29T07:56:01.653 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:00.994+0000 7fa05ea00700 1 --2- 172.21.15.37:0/2373658828 >> [v2:172.21.15.37:6800/2746946658,v1:172.21.15.37:6801/2746946658] conn(0x7fa044039fc0 0x7fa04403c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T07:56:01.653 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:00.995+0000 7fa05ea00700 1 --2- 172.21.15.37:0/2373658828 >> [v2:172.21.15.37:6800/2746946658,v1:172.21.15.37:6801/2746946658] conn(0x7fa044039fc0 0x7fa04403c3a0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fa04c0039d0 tx=0x7fa04c009350 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T07:56:01.653 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:00.996+0000 7fa057400700 1 -- 172.21.15.37:0/2373658828 <== mgr.14118 v2:172.21.15.37:6800/2746946658 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6549 (secure 0 0 0) 0x7fa0440393a0 con 0x7fa044039fc0 2024-03-29T07:56:01.653 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:01.002+0000 7fa066200700 1 -- 172.21.15.37:0/2373658828 --> [v2:172.21.15.37:6800/2746946658,v1:172.21.15.37:6801/2746946658] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7fa060118590 con 0x7fa044039fc0 2024-03-29T07:56:01.653 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:01.002+0000 7fa057400700 1 -- 172.21.15.37:0/2373658828 <== mgr.14118 v2:172.21.15.37:6800/2746946658 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7fa060118590 con 0x7fa044039fc0 2024-03-29T07:56:01.653 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:01.003+0000 7fa066200700 1 -- 172.21.15.37:0/2373658828 >> [v2:172.21.15.37:6800/2746946658,v1:172.21.15.37:6801/2746946658] conn(0x7fa044039fc0 msgr2=0x7fa04403c3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T07:56:01.653 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:01.003+0000 7fa066200700 1 --2- 172.21.15.37:0/2373658828 >> [v2:172.21.15.37:6800/2746946658,v1:172.21.15.37:6801/2746946658] conn(0x7fa044039fc0 0x7fa04403c3a0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fa04c0039d0 tx=0x7fa04c009350 comp rx=0 tx=0).stop 2024-03-29T07:56:01.653 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:01.003+0000 7fa066200700 1 -- 172.21.15.37:0/2373658828 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa060078d90 msgr2=0x7fa0601179c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T07:56:01.653 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:01.003+0000 7fa066200700 1 --2- 172.21.15.37:0/2373658828 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa060078d90 0x7fa0601179c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fa050006fd0 tx=0x7fa050009250 comp rx=0 tx=0).stop 2024-03-29T07:56:01.654 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:01.003+0000 7fa066200700 1 -- 172.21.15.37:0/2373658828 shutdown_connections 2024-03-29T07:56:01.654 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:01.003+0000 7fa066200700 1 --2- 172.21.15.37:0/2373658828 >> [v2:172.21.15.37:6800/2746946658,v1:172.21.15.37:6801/2746946658] conn(0x7fa044039fc0 0x7fa04403c3a0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T07:56:01.654 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:01.003+0000 7fa066200700 1 --2- 172.21.15.37:0/2373658828 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa060078d90 0x7fa0601179c0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T07:56:01.654 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:01.003+0000 7fa066200700 1 --2- 172.21.15.37:0/2373658828 >> [v2:172.21.15.37:6800/2762339582,v1:172.21.15.37:6801/2762339582] conn(0x7fa0440367e0 0x7fa044038c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T07:56:01.654 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:01.003+0000 7fa066200700 1 -- 172.21.15.37:0/2373658828 >> 172.21.15.37:0/2373658828 conn(0x7fa0600fdd00 msgr2=0x7fa06010ecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T07:56:01.654 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:01.003+0000 7fa066200700 1 -- 172.21.15.37:0/2373658828 shutdown_connections 2024-03-29T07:56:01.654 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:01.003+0000 7fa066200700 1 -- 172.21.15.37:0/2373658828 wait complete. 2024-03-29T07:56:01.654 INFO:teuthology.orchestra.run.smithi037.stdout:mgr epoch 5 is available 2024-03-29T07:56:01.654 INFO:teuthology.orchestra.run.smithi037.stdout:Setting orchestrator backend to cephadm... 2024-03-29T07:56:02.176 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:56:02 smithi037 ceph-mon[35162]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-03-29T07:56:02.176 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:56:02 smithi037 ceph-mon[35162]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-03-29T07:56:02.176 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:56:02 smithi037 ceph-mon[35162]: from='mgr.14118 172.21.15.37:0/2997715107' entity='mgr.smithi037.wjikii' 2024-03-29T07:56:02.176 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:56:02 smithi037 ceph-mon[35162]: from='mgr.14118 172.21.15.37:0/2997715107' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T07:56:03.018 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:02.202+0000 7f7553e00700 1 Processor -- start 2024-03-29T07:56:03.018 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:02.202+0000 7f7553e00700 1 -- start start 2024-03-29T07:56:03.018 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:02.202+0000 7f7553e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f754c078d60 0x7f754c077260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T07:56:03.019 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:02.202+0000 7f7553e00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f754c079170 con 0x7f754c078d60 2024-03-29T07:56:03.019 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:02.202+0000 7f7551400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f754c078d60 0x7f754c077260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T07:56:03.019 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:02.202+0000 7f7551400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f754c078d60 0x7f754c077260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59112/0 (socket says 172.21.15.37:59112) 2024-03-29T07:56:03.019 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:02.202+0000 7f7551400700 1 -- 172.21.15.37:0/1102714101 learned_addr learned my addr 172.21.15.37:0/1102714101 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T07:56:03.019 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:02.203+0000 7f7551400700 1 -- 172.21.15.37:0/1102714101 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f754c0777a0 con 0x7f754c078d60 2024-03-29T07:56:03.019 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:02.203+0000 7f7551400700 1 --2- 172.21.15.37:0/1102714101 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f754c078d60 0x7f754c077260 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f75400098e0 tx=0x7f7540009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9461e2dafb9b2a8 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T07:56:03.019 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:02.203+0000 7f754be00700 1 -- 172.21.15.37:0/1102714101 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f754000b860 con 0x7f754c078d60 2024-03-29T07:56:03.019 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:02.203+0000 7f754be00700 1 -- 172.21.15.37:0/1102714101 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1009+0+0 (secure 0 0 0) 0x7f754000be30 con 0x7f754c078d60 2024-03-29T07:56:03.019 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:02.203+0000 7f754be00700 1 -- 172.21.15.37:0/1102714101 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7540010380 con 0x7f754c078d60 2024-03-29T07:56:03.020 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:02.204+0000 7f7553e00700 1 -- 172.21.15.37:0/1102714101 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f754c078d60 msgr2=0x7f754c077260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T07:56:03.020 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:02.204+0000 7f7553e00700 1 --2- 172.21.15.37:0/1102714101 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f754c078d60 0x7f754c077260 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f75400098e0 tx=0x7f7540009250 comp rx=0 tx=0).stop 2024-03-29T07:56:03.020 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:02.204+0000 7f7553e00700 1 -- 172.21.15.37:0/1102714101 shutdown_connections 2024-03-29T07:56:03.020 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:02.204+0000 7f7553e00700 1 --2- 172.21.15.37:0/1102714101 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f754c078d60 0x7f754c077260 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T07:56:03.020 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:02.204+0000 7f7553e00700 1 -- 172.21.15.37:0/1102714101 >> 172.21.15.37:0/1102714101 conn(0x7f754c0fdcd0 msgr2=0x7f754c100100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T07:56:03.020 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:02.204+0000 7f7553e00700 1 -- 172.21.15.37:0/1102714101 shutdown_connections 2024-03-29T07:56:03.020 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:02.204+0000 7f7553e00700 1 -- 172.21.15.37:0/1102714101 wait complete. 2024-03-29T07:56:03.020 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:02.205+0000 7f7553e00700 1 Processor -- start 2024-03-29T07:56:03.020 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:02.205+0000 7f7553e00700 1 -- start start 2024-03-29T07:56:03.020 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:02.205+0000 7f7553e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f754c078d60 0x7f754c113630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T07:56:03.020 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:02.205+0000 7f7553e00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f754c113b70 con 0x7f754c078d60 2024-03-29T07:56:03.021 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:02.205+0000 7f7551400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f754c078d60 0x7f754c113630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T07:56:03.021 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:02.206+0000 7f7551400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f754c078d60 0x7f754c113630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59128/0 (socket says 172.21.15.37:59128) 2024-03-29T07:56:03.021 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:02.206+0000 7f7551400700 1 -- 172.21.15.37:0/3907359585 learned_addr learned my addr 172.21.15.37:0/3907359585 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T07:56:03.021 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:02.206+0000 7f7551400700 1 -- 172.21.15.37:0/3907359585 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75400094f0 con 0x7f754c078d60 2024-03-29T07:56:03.021 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:02.206+0000 7f7551400700 1 --2- 172.21.15.37:0/3907359585 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f754c078d60 0x7f754c113630 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f7540006b00 tx=0x7f7540009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T07:56:03.021 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:02.206+0000 7f7549600700 1 -- 172.21.15.37:0/3907359585 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7540010750 con 0x7f754c078d60 2024-03-29T07:56:03.021 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:02.206+0000 7f7549600700 1 -- 172.21.15.37:0/3907359585 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1009+0+0 (secure 0 0 0) 0x7f7540010d20 con 0x7f754c078d60 2024-03-29T07:56:03.021 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:02.206+0000 7f7553e00700 1 -- 172.21.15.37:0/3907359585 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f754c113d70 con 0x7f754c078d60 2024-03-29T07:56:03.021 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:02.206+0000 7f7549600700 1 -- 172.21.15.37:0/3907359585 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f754001acf0 con 0x7f754c078d60 2024-03-29T07:56:03.021 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:02.207+0000 7f7553e00700 1 -- 172.21.15.37:0/3907359585 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f754c1148e0 con 0x7f754c078d60 2024-03-29T07:56:03.021 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:02.207+0000 7f7549600700 1 -- 172.21.15.37:0/3907359585 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 7) v1 ==== 43262+0+0 (secure 0 0 0) 0x7f75400108b0 con 0x7f754c078d60 2024-03-29T07:56:03.021 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:02.207+0000 7f7553e00700 1 -- 172.21.15.37:0/3907359585 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f754c05f5f0 con 0x7f754c078d60 2024-03-29T07:56:03.021 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:02.208+0000 7f7549600700 1 --2- 172.21.15.37:0/3907359585 >> [v2:172.21.15.37:6800/2746946658,v1:172.21.15.37:6801/2746946658] conn(0x7f75340366b0 0x7f7534038b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T07:56:03.021 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:02.208+0000 7f7549600700 1 -- 172.21.15.37:0/3907359585 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f754001e070 con 0x7f754c078d60 2024-03-29T07:56:03.022 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:02.208+0000 7f7550a00700 1 --2- 172.21.15.37:0/3907359585 >> [v2:172.21.15.37:6800/2746946658,v1:172.21.15.37:6801/2746946658] conn(0x7f75340366b0 0x7f7534038b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T07:56:03.022 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:02.209+0000 7f7550a00700 1 --2- 172.21.15.37:0/3907359585 >> [v2:172.21.15.37:6800/2746946658,v1:172.21.15.37:6801/2746946658] conn(0x7f75340366b0 0x7f7534038b60 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f7538009990 tx=0x7f7538006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T07:56:03.022 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:02.214+0000 7f7549600700 1 -- 172.21.15.37:0/3907359585 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f754007c190 con 0x7f754c078d60 2024-03-29T07:56:03.022 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:02.435+0000 7f7553e00700 1 -- 172.21.15.37:0/3907359585 --> [v2:172.21.15.37:6800/2746946658,v1:172.21.15.37:6801/2746946658] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) v1 -- 0x7f754c0ff090 con 0x7f75340366b0 2024-03-29T07:56:03.022 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:02.444+0000 7f7549600700 1 -- 172.21.15.37:0/3907359585 <== mgr.14118 v2:172.21.15.37:6800/2746946658 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f754c0ff090 con 0x7f75340366b0 2024-03-29T07:56:03.022 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:02.447+0000 7f7553e00700 1 -- 172.21.15.37:0/3907359585 >> [v2:172.21.15.37:6800/2746946658,v1:172.21.15.37:6801/2746946658] conn(0x7f75340366b0 msgr2=0x7f7534038b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T07:56:03.022 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:02.447+0000 7f7553e00700 1 --2- 172.21.15.37:0/3907359585 >> [v2:172.21.15.37:6800/2746946658,v1:172.21.15.37:6801/2746946658] conn(0x7f75340366b0 0x7f7534038b60 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f7538009990 tx=0x7f7538006e40 comp rx=0 tx=0).stop 2024-03-29T07:56:03.022 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:02.447+0000 7f7553e00700 1 -- 172.21.15.37:0/3907359585 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f754c078d60 msgr2=0x7f754c113630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T07:56:03.022 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:02.447+0000 7f7553e00700 1 --2- 172.21.15.37:0/3907359585 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f754c078d60 0x7f754c113630 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f7540006b00 tx=0x7f7540009250 comp rx=0 tx=0).stop 2024-03-29T07:56:03.022 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:02.447+0000 7f7553e00700 1 -- 172.21.15.37:0/3907359585 shutdown_connections 2024-03-29T07:56:03.022 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:02.447+0000 7f7553e00700 1 --2- 172.21.15.37:0/3907359585 >> [v2:172.21.15.37:6800/2746946658,v1:172.21.15.37:6801/2746946658] conn(0x7f75340366b0 0x7f7534038b60 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T07:56:03.022 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:02.447+0000 7f7553e00700 1 --2- 172.21.15.37:0/3907359585 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f754c078d60 0x7f754c113630 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T07:56:03.022 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:02.447+0000 7f7553e00700 1 -- 172.21.15.37:0/3907359585 >> 172.21.15.37:0/3907359585 conn(0x7f754c0fdcd0 msgr2=0x7f754c0fe8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T07:56:03.022 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:02.447+0000 7f7553e00700 1 -- 172.21.15.37:0/3907359585 shutdown_connections 2024-03-29T07:56:03.023 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:02.447+0000 7f7553e00700 1 -- 172.21.15.37:0/3907359585 wait complete. 2024-03-29T07:56:03.676 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:56:03 smithi037 ceph-mon[35162]: [29/Mar/2024:07:56:01] ENGINE Bus STARTING 2024-03-29T07:56:03.676 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:56:03 smithi037 ceph-mon[35162]: [29/Mar/2024:07:56:01] ENGINE Serving on https://172.21.15.37:7150 2024-03-29T07:56:03.676 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:56:03 smithi037 ceph-mon[35162]: [29/Mar/2024:07:56:01] ENGINE Bus STARTED 2024-03-29T07:56:03.676 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:56:03 smithi037 ceph-mon[35162]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T07:56:03.676 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:56:03 smithi037 ceph-mon[35162]: from='mgr.14118 172.21.15.37:0/2997715107' entity='mgr.smithi037.wjikii' 2024-03-29T07:56:03.676 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:56:03 smithi037 ceph-mon[35162]: from='mgr.14118 172.21.15.37:0/2997715107' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T07:56:03.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:56:03 smithi037 ceph-mon[35162]: mgrmap e8: smithi037.wjikii(active, since 2s) 2024-03-29T07:56:04.398 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout value unchanged 2024-03-29T07:56:04.399 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:03.572+0000 7fd1f1c00700 1 Processor -- start 2024-03-29T07:56:04.399 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:03.573+0000 7fd1f1c00700 1 -- start start 2024-03-29T07:56:04.399 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:03.573+0000 7fd1f1c00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd1ec102580 0x7fd1ec102990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T07:56:04.399 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:03.573+0000 7fd1f1c00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1ec102ed0 con 0x7fd1ec102580 2024-03-29T07:56:04.399 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:03.573+0000 7fd1eaa00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd1ec102580 0x7fd1ec102990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T07:56:04.399 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:03.573+0000 7fd1eaa00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd1ec102580 0x7fd1ec102990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59132/0 (socket says 172.21.15.37:59132) 2024-03-29T07:56:04.399 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:03.573+0000 7fd1eaa00700 1 -- 172.21.15.37:0/837166034 learned_addr learned my addr 172.21.15.37:0/837166034 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T07:56:04.399 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:03.574+0000 7fd1eaa00700 1 -- 172.21.15.37:0/837166034 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1ec103010 con 0x7fd1ec102580 2024-03-29T07:56:04.399 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:03.574+0000 7fd1eaa00700 1 --2- 172.21.15.37:0/837166034 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd1ec102580 0x7fd1ec102990 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fd1dc0098e0 tx=0x7fd1dc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e54efdb6b4ad779a server_cookie=0 in_seq=0 out_seq=0 2024-03-29T07:56:04.399 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:03.574+0000 7fd1e9600700 1 -- 172.21.15.37:0/837166034 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd1dc00b860 con 0x7fd1ec102580 2024-03-29T07:56:04.399 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:03.574+0000 7fd1e9600700 1 -- 172.21.15.37:0/837166034 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1009+0+0 (secure 0 0 0) 0x7fd1dc00be30 con 0x7fd1ec102580 2024-03-29T07:56:04.400 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:03.574+0000 7fd1e9600700 1 -- 172.21.15.37:0/837166034 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd1dc010380 con 0x7fd1ec102580 2024-03-29T07:56:04.400 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:03.575+0000 7fd1f1c00700 1 -- 172.21.15.37:0/837166034 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd1ec102580 msgr2=0x7fd1ec102990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T07:56:04.400 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:03.575+0000 7fd1f1c00700 1 --2- 172.21.15.37:0/837166034 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd1ec102580 0x7fd1ec102990 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fd1dc0098e0 tx=0x7fd1dc009250 comp rx=0 tx=0).stop 2024-03-29T07:56:04.400 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:03.575+0000 7fd1f1c00700 1 -- 172.21.15.37:0/837166034 shutdown_connections 2024-03-29T07:56:04.400 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:03.575+0000 7fd1f1c00700 1 --2- 172.21.15.37:0/837166034 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd1ec102580 0x7fd1ec102990 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T07:56:04.400 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:03.575+0000 7fd1f1c00700 1 -- 172.21.15.37:0/837166034 >> 172.21.15.37:0/837166034 conn(0x7fd1ec0fdd00 msgr2=0x7fd1ec100110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T07:56:04.400 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:03.575+0000 7fd1f1c00700 1 -- 172.21.15.37:0/837166034 shutdown_connections 2024-03-29T07:56:04.400 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:03.575+0000 7fd1f1c00700 1 -- 172.21.15.37:0/837166034 wait complete. 2024-03-29T07:56:04.400 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:03.576+0000 7fd1f1c00700 1 Processor -- start 2024-03-29T07:56:04.400 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:03.576+0000 7fd1f1c00700 1 -- start start 2024-03-29T07:56:04.400 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:03.576+0000 7fd1f1c00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd1ec102580 0x7fd1ec111340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T07:56:04.400 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:03.576+0000 7fd1f1c00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1ec111880 con 0x7fd1ec102580 2024-03-29T07:56:04.400 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:03.577+0000 7fd1eaa00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd1ec102580 0x7fd1ec111340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T07:56:04.400 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:03.577+0000 7fd1eaa00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd1ec102580 0x7fd1ec111340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59142/0 (socket says 172.21.15.37:59142) 2024-03-29T07:56:04.401 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:03.577+0000 7fd1eaa00700 1 -- 172.21.15.37:0/634055125 learned_addr learned my addr 172.21.15.37:0/634055125 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T07:56:04.401 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:03.577+0000 7fd1eaa00700 1 -- 172.21.15.37:0/634055125 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1dc0094f0 con 0x7fd1ec102580 2024-03-29T07:56:04.401 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:03.577+0000 7fd1eaa00700 1 --2- 172.21.15.37:0/634055125 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd1ec102580 0x7fd1ec111340 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fd1dc000c00 tx=0x7fd1dc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T07:56:04.401 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:03.578+0000 7fd1e2a00700 1 -- 172.21.15.37:0/634055125 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd1dc010a80 con 0x7fd1ec102580 2024-03-29T07:56:04.401 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:03.578+0000 7fd1e2a00700 1 -- 172.21.15.37:0/634055125 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1009+0+0 (secure 0 0 0) 0x7fd1dc01a430 con 0x7fd1ec102580 2024-03-29T07:56:04.401 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:03.578+0000 7fd1e2a00700 1 -- 172.21.15.37:0/634055125 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd1dc0245b0 con 0x7fd1ec102580 2024-03-29T07:56:04.401 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:03.578+0000 7fd1f1c00700 1 -- 172.21.15.37:0/634055125 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1ec111a80 con 0x7fd1ec102580 2024-03-29T07:56:04.401 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:03.578+0000 7fd1f1c00700 1 -- 172.21.15.37:0/634055125 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1ec1125f0 con 0x7fd1ec102580 2024-03-29T07:56:04.401 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:03.579+0000 7fd1e2a00700 1 -- 172.21.15.37:0/634055125 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7fd1dc010be0 con 0x7fd1ec102580 2024-03-29T07:56:04.401 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:03.579+0000 7fd1f1c00700 1 -- 172.21.15.37:0/634055125 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1ec05f5f0 con 0x7fd1ec102580 2024-03-29T07:56:04.401 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:03.579+0000 7fd1e2a00700 1 --2- 172.21.15.37:0/634055125 >> [v2:172.21.15.37:6800/2746946658,v1:172.21.15.37:6801/2746946658] conn(0x7fd1d00367e0 0x7fd1d0038c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T07:56:04.401 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:03.579+0000 7fd1e2a00700 1 -- 172.21.15.37:0/634055125 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7fd1dc04ade0 con 0x7fd1ec102580 2024-03-29T07:56:04.401 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:03.579+0000 7fd1ea000700 1 --2- 172.21.15.37:0/634055125 >> [v2:172.21.15.37:6800/2746946658,v1:172.21.15.37:6801/2746946658] conn(0x7fd1d00367e0 0x7fd1d0038c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T07:56:04.401 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:03.580+0000 7fd1ea000700 1 --2- 172.21.15.37:0/634055125 >> [v2:172.21.15.37:6800/2746946658,v1:172.21.15.37:6801/2746946658] conn(0x7fd1d00367e0 0x7fd1d0038c90 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd1d8009990 tx=0x7fd1d8006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T07:56:04.401 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:03.584+0000 7fd1e2a00700 1 -- 172.21.15.37:0/634055125 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fd1dc0239e0 con 0x7fd1ec102580 2024-03-29T07:56:04.402 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:03.794+0000 7fd1f1c00700 1 -- 172.21.15.37:0/634055125 --> [v2:172.21.15.37:6800/2746946658,v1:172.21.15.37:6801/2746946658] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) v1 -- 0x7fd1ec0fef40 con 0x7fd1d00367e0 2024-03-29T07:56:04.402 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:03.795+0000 7fd1e2a00700 1 -- 172.21.15.37:0/634055125 <== mgr.14118 v2:172.21.15.37:6800/2746946658 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+16 (secure 0 0 0) 0x7fd1ec0fef40 con 0x7fd1d00367e0 2024-03-29T07:56:04.402 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:03.797+0000 7fd1f1c00700 1 -- 172.21.15.37:0/634055125 >> [v2:172.21.15.37:6800/2746946658,v1:172.21.15.37:6801/2746946658] conn(0x7fd1d00367e0 msgr2=0x7fd1d0038c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T07:56:04.402 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:03.797+0000 7fd1f1c00700 1 --2- 172.21.15.37:0/634055125 >> [v2:172.21.15.37:6800/2746946658,v1:172.21.15.37:6801/2746946658] conn(0x7fd1d00367e0 0x7fd1d0038c90 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd1d8009990 tx=0x7fd1d8006e40 comp rx=0 tx=0).stop 2024-03-29T07:56:04.402 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:03.797+0000 7fd1f1c00700 1 -- 172.21.15.37:0/634055125 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd1ec102580 msgr2=0x7fd1ec111340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T07:56:04.402 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:03.797+0000 7fd1f1c00700 1 --2- 172.21.15.37:0/634055125 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd1ec102580 0x7fd1ec111340 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fd1dc000c00 tx=0x7fd1dc009250 comp rx=0 tx=0).stop 2024-03-29T07:56:04.402 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:03.797+0000 7fd1f1c00700 1 -- 172.21.15.37:0/634055125 shutdown_connections 2024-03-29T07:56:04.402 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:03.797+0000 7fd1f1c00700 1 --2- 172.21.15.37:0/634055125 >> [v2:172.21.15.37:6800/2746946658,v1:172.21.15.37:6801/2746946658] conn(0x7fd1d00367e0 0x7fd1d0038c90 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T07:56:04.402 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:03.797+0000 7fd1f1c00700 1 --2- 172.21.15.37:0/634055125 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd1ec102580 0x7fd1ec111340 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T07:56:04.403 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:03.797+0000 7fd1f1c00700 1 -- 172.21.15.37:0/634055125 >> 172.21.15.37:0/634055125 conn(0x7fd1ec0fdd00 msgr2=0x7fd1ec0fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T07:56:04.403 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:03.798+0000 7fd1f1c00700 1 -- 172.21.15.37:0/634055125 shutdown_connections 2024-03-29T07:56:04.403 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:03.798+0000 7fd1f1c00700 1 -- 172.21.15.37:0/634055125 wait complete. 2024-03-29T07:56:04.403 INFO:teuthology.orchestra.run.smithi037.stdout:Generating ssh key... 2024-03-29T07:56:04.662 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:56:04 smithi037 ceph-mon[35162]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T07:56:05.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:56:05 smithi037 ceph-mon[35162]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T07:56:05.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:56:05 smithi037 ceph-mon[35162]: Generating ssh key... 2024-03-29T07:56:06.305 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:04.962+0000 7f47ffc00700 1 Processor -- start 2024-03-29T07:56:06.305 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:04.962+0000 7f47ffc00700 1 -- start start 2024-03-29T07:56:06.306 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:04.963+0000 7f47ffc00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f47f8102580 0x7f47f8102990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T07:56:06.306 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:04.963+0000 7f47ffc00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47f8102ed0 con 0x7f47f8102580 2024-03-29T07:56:06.306 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:04.963+0000 7f47fd200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f47f8102580 0x7f47f8102990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T07:56:06.306 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:04.963+0000 7f47fd200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f47f8102580 0x7f47f8102990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59152/0 (socket says 172.21.15.37:59152) 2024-03-29T07:56:06.306 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:04.963+0000 7f47fd200700 1 -- 172.21.15.37:0/1638258918 learned_addr learned my addr 172.21.15.37:0/1638258918 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T07:56:06.306 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:04.963+0000 7f47fd200700 1 -- 172.21.15.37:0/1638258918 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47f8103010 con 0x7f47f8102580 2024-03-29T07:56:06.306 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:04.963+0000 7f47fd200700 1 --2- 172.21.15.37:0/1638258918 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f47f8102580 0x7f47f8102990 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f47ec0098e0 tx=0x7f47ec009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c26928a4823544f5 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T07:56:06.306 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:04.964+0000 7f47f7400700 1 -- 172.21.15.37:0/1638258918 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f47ec00b860 con 0x7f47f8102580 2024-03-29T07:56:06.306 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:04.964+0000 7f47f7400700 1 -- 172.21.15.37:0/1638258918 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1009+0+0 (secure 0 0 0) 0x7f47ec00be30 con 0x7f47f8102580 2024-03-29T07:56:06.307 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:04.964+0000 7f47f7400700 1 -- 172.21.15.37:0/1638258918 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f47ec010380 con 0x7f47f8102580 2024-03-29T07:56:06.307 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:04.964+0000 7f47ffc00700 1 -- 172.21.15.37:0/1638258918 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f47f8102580 msgr2=0x7f47f8102990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T07:56:06.307 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:04.964+0000 7f47ffc00700 1 --2- 172.21.15.37:0/1638258918 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f47f8102580 0x7f47f8102990 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f47ec0098e0 tx=0x7f47ec009250 comp rx=0 tx=0).stop 2024-03-29T07:56:06.307 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:04.964+0000 7f47ffc00700 1 -- 172.21.15.37:0/1638258918 shutdown_connections 2024-03-29T07:56:06.307 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:04.964+0000 7f47ffc00700 1 --2- 172.21.15.37:0/1638258918 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f47f8102580 0x7f47f8102990 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T07:56:06.307 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:04.964+0000 7f47ffc00700 1 -- 172.21.15.37:0/1638258918 >> 172.21.15.37:0/1638258918 conn(0x7f47f80fdd00 msgr2=0x7f47f8100110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T07:56:06.307 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:04.964+0000 7f47ffc00700 1 -- 172.21.15.37:0/1638258918 shutdown_connections 2024-03-29T07:56:06.307 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:04.964+0000 7f47ffc00700 1 -- 172.21.15.37:0/1638258918 wait complete. 2024-03-29T07:56:06.307 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:04.965+0000 7f47ffc00700 1 Processor -- start 2024-03-29T07:56:06.307 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:04.965+0000 7f47ffc00700 1 -- start start 2024-03-29T07:56:06.307 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:04.966+0000 7f47ffc00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f47f8102580 0x7f47f8078c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T07:56:06.308 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:04.966+0000 7f47ffc00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47f8079190 con 0x7f47f8102580 2024-03-29T07:56:06.308 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:04.966+0000 7f47fd200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f47f8102580 0x7f47f8078c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T07:56:06.308 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:04.966+0000 7f47fd200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f47f8102580 0x7f47f8078c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59162/0 (socket says 172.21.15.37:59162) 2024-03-29T07:56:06.308 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:04.966+0000 7f47fd200700 1 -- 172.21.15.37:0/135432945 learned_addr learned my addr 172.21.15.37:0/135432945 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T07:56:06.308 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:04.966+0000 7f47fd200700 1 -- 172.21.15.37:0/135432945 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47ec0094f0 con 0x7f47f8102580 2024-03-29T07:56:06.308 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:04.966+0000 7f47fd200700 1 --2- 172.21.15.37:0/135432945 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f47f8102580 0x7f47f8078c50 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f47ec000c00 tx=0x7f47ec009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T07:56:06.308 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:04.967+0000 7f47f4c00700 1 -- 172.21.15.37:0/135432945 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f47ec010a80 con 0x7f47f8102580 2024-03-29T07:56:06.308 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:04.967+0000 7f47ffc00700 1 -- 172.21.15.37:0/135432945 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47f8077200 con 0x7f47f8102580 2024-03-29T07:56:06.308 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:04.967+0000 7f47f4c00700 1 -- 172.21.15.37:0/135432945 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1009+0+0 (secure 0 0 0) 0x7f47ec01a430 con 0x7f47f8102580 2024-03-29T07:56:06.308 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:04.967+0000 7f47f4c00700 1 -- 172.21.15.37:0/135432945 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f47ec0245b0 con 0x7f47f8102580 2024-03-29T07:56:06.308 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:04.967+0000 7f47ffc00700 1 -- 172.21.15.37:0/135432945 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47f80776f0 con 0x7f47f8102580 2024-03-29T07:56:06.308 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:04.968+0000 7f47ffc00700 1 -- 172.21.15.37:0/135432945 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47f804bed0 con 0x7f47f8102580 2024-03-29T07:56:06.308 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:04.968+0000 7f47f4c00700 1 -- 172.21.15.37:0/135432945 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7f47ec01a5a0 con 0x7f47f8102580 2024-03-29T07:56:06.309 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:04.969+0000 7f47f4c00700 1 --2- 172.21.15.37:0/135432945 >> [v2:172.21.15.37:6800/2746946658,v1:172.21.15.37:6801/2746946658] conn(0x7f47e00367e0 0x7f47e0038c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T07:56:06.309 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:04.969+0000 7f47f4c00700 1 -- 172.21.15.37:0/135432945 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f47ec04acf0 con 0x7f47f8102580 2024-03-29T07:56:06.309 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:04.969+0000 7f47f7e00700 1 --2- 172.21.15.37:0/135432945 >> [v2:172.21.15.37:6800/2746946658,v1:172.21.15.37:6801/2746946658] conn(0x7f47e00367e0 0x7f47e0038c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T07:56:06.309 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:04.970+0000 7f47f7e00700 1 --2- 172.21.15.37:0/135432945 >> [v2:172.21.15.37:6800/2746946658,v1:172.21.15.37:6801/2746946658] conn(0x7f47e00367e0 0x7f47e0038c90 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f47e4009990 tx=0x7f47e4006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T07:56:06.309 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:04.974+0000 7f47f4c00700 1 -- 172.21.15.37:0/135432945 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f47ec023df0 con 0x7f47f8102580 2024-03-29T07:56:06.309 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:05.182+0000 7f47ffc00700 1 -- 172.21.15.37:0/135432945 --> [v2:172.21.15.37:6800/2746946658,v1:172.21.15.37:6801/2746946658] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f47f8077eb0 con 0x7f47e00367e0 2024-03-29T07:56:06.309 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:05.662+0000 7f47f4c00700 1 -- 172.21.15.37:0/135432945 <== mgr.14118 v2:172.21.15.37:6800/2746946658 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f47f8077eb0 con 0x7f47e00367e0 2024-03-29T07:56:06.309 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:05.664+0000 7f47ffc00700 1 -- 172.21.15.37:0/135432945 >> [v2:172.21.15.37:6800/2746946658,v1:172.21.15.37:6801/2746946658] conn(0x7f47e00367e0 msgr2=0x7f47e0038c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T07:56:06.309 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:05.664+0000 7f47ffc00700 1 --2- 172.21.15.37:0/135432945 >> [v2:172.21.15.37:6800/2746946658,v1:172.21.15.37:6801/2746946658] conn(0x7f47e00367e0 0x7f47e0038c90 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f47e4009990 tx=0x7f47e4006e40 comp rx=0 tx=0).stop 2024-03-29T07:56:06.309 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:05.664+0000 7f47ffc00700 1 -- 172.21.15.37:0/135432945 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f47f8102580 msgr2=0x7f47f8078c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T07:56:06.309 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:05.664+0000 7f47ffc00700 1 --2- 172.21.15.37:0/135432945 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f47f8102580 0x7f47f8078c50 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f47ec000c00 tx=0x7f47ec009250 comp rx=0 tx=0).stop 2024-03-29T07:56:06.309 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:05.664+0000 7f47ffc00700 1 -- 172.21.15.37:0/135432945 shutdown_connections 2024-03-29T07:56:06.310 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:05.664+0000 7f47ffc00700 1 --2- 172.21.15.37:0/135432945 >> [v2:172.21.15.37:6800/2746946658,v1:172.21.15.37:6801/2746946658] conn(0x7f47e00367e0 0x7f47e0038c90 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T07:56:06.310 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:05.664+0000 7f47ffc00700 1 --2- 172.21.15.37:0/135432945 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f47f8102580 0x7f47f8078c50 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T07:56:06.310 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:05.664+0000 7f47ffc00700 1 -- 172.21.15.37:0/135432945 >> 172.21.15.37:0/135432945 conn(0x7f47f80fdd00 msgr2=0x7f47f80fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T07:56:06.310 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:05.664+0000 7f47ffc00700 1 -- 172.21.15.37:0/135432945 shutdown_connections 2024-03-29T07:56:06.310 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:05.664+0000 7f47ffc00700 1 -- 172.21.15.37:0/135432945 wait complete. 2024-03-29T07:56:06.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:56:06 smithi037 ceph-mon[35162]: from='mgr.14118 172.21.15.37:0/2997715107' entity='mgr.smithi037.wjikii' 2024-03-29T07:56:06.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:56:06 smithi037 ceph-mon[35162]: from='mgr.14118 172.21.15.37:0/2997715107' entity='mgr.smithi037.wjikii' 2024-03-29T07:56:07.837 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC2IjzXulokTgrTwcqS4Akc/Oice5q1DQRyhlC1Y6Eh93wYQdIdT65GOg73DPvyNhwNDOWWV6sV1PZVslqN3V5Gg4kKN3+oDCwZh5p478jH4YVYXjp94jwF3DLXXwarkwGxjWNbMrwdHDj5qAkHCKS6IXPXrprmhcvAaiMBJpdJkA7ucLRPDFGz3eidl98dHBCy3gYINCaei0Xpdob5F+QekicTiM9yllvoJ12WxUTmdV87GAU//Qi7rDe+ni4/UBfU+8W6twmqWHjpT4Ci3v2yfW/+23hMTFYhfRJXWdDrKsRxVxNlRc8unu9Eo5h/LWXf8H56qDug6eiJ2TyeWxVCBcY6LjkE99J4gsx8v/Qgrz0vGBJk2rUFAgA32xIz9LsDf+RaEP267Lr7aYc0Ic+jSicSqqWiB7ryrf9wQBtXIBYK4hqWF/fZ7gptMsoX21HjmJsOWtfQZ1N1wOT9YfxRxhvLqijb5gI98ORtc6PdEseM7+AFw4DQROewLHl0EM8= ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4 2024-03-29T07:56:07.837 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:06.843+0000 7fb709200700 1 Processor -- start 2024-03-29T07:56:07.837 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:06.843+0000 7fb709200700 1 -- start start 2024-03-29T07:56:07.837 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:06.844+0000 7fb709200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb704078d50 0x7fb704077260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T07:56:07.837 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:06.844+0000 7fb709200700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb704079160 con 0x7fb704078d50 2024-03-29T07:56:07.837 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:06.844+0000 7fb701e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb704078d50 0x7fb704077260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T07:56:07.837 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:06.844+0000 7fb701e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb704078d50 0x7fb704077260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59166/0 (socket says 172.21.15.37:59166) 2024-03-29T07:56:07.837 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:06.844+0000 7fb701e00700 1 -- 172.21.15.37:0/3924559744 learned_addr learned my addr 172.21.15.37:0/3924559744 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T07:56:07.837 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:06.844+0000 7fb701e00700 1 -- 172.21.15.37:0/3924559744 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7040777a0 con 0x7fb704078d50 2024-03-29T07:56:07.838 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:06.845+0000 7fb701e00700 1 --2- 172.21.15.37:0/3924559744 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb704078d50 0x7fb704077260 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fb6f40098e0 tx=0x7fb6f4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7318b9b2ec2e050c server_cookie=0 in_seq=0 out_seq=0 2024-03-29T07:56:07.838 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:06.845+0000 7fb700a00700 1 -- 172.21.15.37:0/3924559744 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb6f400b860 con 0x7fb704078d50 2024-03-29T07:56:07.838 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:06.845+0000 7fb700a00700 1 -- 172.21.15.37:0/3924559744 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1009+0+0 (secure 0 0 0) 0x7fb6f400be30 con 0x7fb704078d50 2024-03-29T07:56:07.838 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:06.845+0000 7fb700a00700 1 -- 172.21.15.37:0/3924559744 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb6f4010380 con 0x7fb704078d50 2024-03-29T07:56:07.838 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:06.845+0000 7fb709200700 1 -- 172.21.15.37:0/3924559744 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb704078d50 msgr2=0x7fb704077260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T07:56:07.838 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:06.845+0000 7fb709200700 1 --2- 172.21.15.37:0/3924559744 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb704078d50 0x7fb704077260 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fb6f40098e0 tx=0x7fb6f4009250 comp rx=0 tx=0).stop 2024-03-29T07:56:07.838 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:06.846+0000 7fb709200700 1 -- 172.21.15.37:0/3924559744 shutdown_connections 2024-03-29T07:56:07.838 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:06.846+0000 7fb709200700 1 --2- 172.21.15.37:0/3924559744 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb704078d50 0x7fb704077260 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T07:56:07.838 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:06.846+0000 7fb709200700 1 -- 172.21.15.37:0/3924559744 >> 172.21.15.37:0/3924559744 conn(0x7fb7040fdd00 msgr2=0x7fb704100110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T07:56:07.838 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:06.846+0000 7fb709200700 1 -- 172.21.15.37:0/3924559744 shutdown_connections 2024-03-29T07:56:07.838 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:06.846+0000 7fb709200700 1 -- 172.21.15.37:0/3924559744 wait complete. 2024-03-29T07:56:07.838 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:06.846+0000 7fb709200700 1 Processor -- start 2024-03-29T07:56:07.838 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:06.847+0000 7fb709200700 1 -- start start 2024-03-29T07:56:07.838 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:06.847+0000 7fb709200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb704078d50 0x7fb704102480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T07:56:07.839 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:06.847+0000 7fb709200700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7041029c0 con 0x7fb704078d50 2024-03-29T07:56:07.839 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:06.847+0000 7fb701e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb704078d50 0x7fb704102480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T07:56:07.839 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:06.847+0000 7fb701e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb704078d50 0x7fb704102480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59176/0 (socket says 172.21.15.37:59176) 2024-03-29T07:56:07.839 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:06.847+0000 7fb701e00700 1 -- 172.21.15.37:0/1083829178 learned_addr learned my addr 172.21.15.37:0/1083829178 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T07:56:07.839 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:06.848+0000 7fb701e00700 1 -- 172.21.15.37:0/1083829178 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6f40094f0 con 0x7fb704078d50 2024-03-29T07:56:07.839 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:06.848+0000 7fb701e00700 1 --2- 172.21.15.37:0/1083829178 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb704078d50 0x7fb704102480 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fb6f4006fd0 tx=0x7fb6f4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T07:56:07.839 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:06.848+0000 7fb6fa000700 1 -- 172.21.15.37:0/1083829178 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb6f4010750 con 0x7fb704078d50 2024-03-29T07:56:07.839 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:06.848+0000 7fb6fa000700 1 -- 172.21.15.37:0/1083829178 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1009+0+0 (secure 0 0 0) 0x7fb6f4010d20 con 0x7fb704078d50 2024-03-29T07:56:07.839 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:06.848+0000 7fb709200700 1 -- 172.21.15.37:0/1083829178 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb704102bc0 con 0x7fb704078d50 2024-03-29T07:56:07.839 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:06.848+0000 7fb6fa000700 1 -- 172.21.15.37:0/1083829178 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb6f401ada0 con 0x7fb704078d50 2024-03-29T07:56:07.839 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:06.848+0000 7fb709200700 1 -- 172.21.15.37:0/1083829178 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb704103050 con 0x7fb704078d50 2024-03-29T07:56:07.839 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:06.849+0000 7fb6fa000700 1 -- 172.21.15.37:0/1083829178 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7fb6f40108b0 con 0x7fb704078d50 2024-03-29T07:56:07.839 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:06.849+0000 7fb709200700 1 -- 172.21.15.37:0/1083829178 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb70404bed0 con 0x7fb704078d50 2024-03-29T07:56:07.839 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:06.850+0000 7fb6fa000700 1 --2- 172.21.15.37:0/1083829178 >> [v2:172.21.15.37:6800/2746946658,v1:172.21.15.37:6801/2746946658] conn(0x7fb6e8036830 0x7fb6e8038ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T07:56:07.840 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:06.850+0000 7fb6fa000700 1 -- 172.21.15.37:0/1083829178 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7fb6f4049e10 con 0x7fb704078d50 2024-03-29T07:56:07.840 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:06.850+0000 7fb701400700 1 --2- 172.21.15.37:0/1083829178 >> [v2:172.21.15.37:6800/2746946658,v1:172.21.15.37:6801/2746946658] conn(0x7fb6e8036830 0x7fb6e8038ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T07:56:07.840 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:06.851+0000 7fb701400700 1 --2- 172.21.15.37:0/1083829178 >> [v2:172.21.15.37:6800/2746946658,v1:172.21.15.37:6801/2746946658] conn(0x7fb6e8036830 0x7fb6e8038ce0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fb6f0009990 tx=0x7fb6f0006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T07:56:07.840 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:06.855+0000 7fb6fa000700 1 -- 172.21.15.37:0/1083829178 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fb6f4010e90 con 0x7fb704078d50 2024-03-29T07:56:07.840 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:07.061+0000 7fb709200700 1 -- 172.21.15.37:0/1083829178 --> [v2:172.21.15.37:6800/2746946658,v1:172.21.15.37:6801/2746946658] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) v1 -- 0x7fb7040fef50 con 0x7fb6e8036830 2024-03-29T07:56:07.840 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:07.062+0000 7fb6fa000700 1 -- 172.21.15.37:0/1083829178 <== mgr.14118 v2:172.21.15.37:6800/2746946658 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+595 (secure 0 0 0) 0x7fb7040fef50 con 0x7fb6e8036830 2024-03-29T07:56:07.840 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:07.064+0000 7fb709200700 1 -- 172.21.15.37:0/1083829178 >> [v2:172.21.15.37:6800/2746946658,v1:172.21.15.37:6801/2746946658] conn(0x7fb6e8036830 msgr2=0x7fb6e8038ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T07:56:07.840 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:07.064+0000 7fb709200700 1 --2- 172.21.15.37:0/1083829178 >> [v2:172.21.15.37:6800/2746946658,v1:172.21.15.37:6801/2746946658] conn(0x7fb6e8036830 0x7fb6e8038ce0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fb6f0009990 tx=0x7fb6f0006e40 comp rx=0 tx=0).stop 2024-03-29T07:56:07.840 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:07.064+0000 7fb709200700 1 -- 172.21.15.37:0/1083829178 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb704078d50 msgr2=0x7fb704102480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T07:56:07.840 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:07.064+0000 7fb709200700 1 --2- 172.21.15.37:0/1083829178 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb704078d50 0x7fb704102480 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fb6f4006fd0 tx=0x7fb6f4009250 comp rx=0 tx=0).stop 2024-03-29T07:56:07.840 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:07.064+0000 7fb709200700 1 -- 172.21.15.37:0/1083829178 shutdown_connections 2024-03-29T07:56:07.840 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:07.064+0000 7fb709200700 1 --2- 172.21.15.37:0/1083829178 >> [v2:172.21.15.37:6800/2746946658,v1:172.21.15.37:6801/2746946658] conn(0x7fb6e8036830 0x7fb6e8038ce0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T07:56:07.840 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:07.064+0000 7fb709200700 1 --2- 172.21.15.37:0/1083829178 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb704078d50 0x7fb704102480 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T07:56:07.840 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:07.064+0000 7fb709200700 1 -- 172.21.15.37:0/1083829178 >> 172.21.15.37:0/1083829178 conn(0x7fb7040fdd00 msgr2=0x7fb7040fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T07:56:07.841 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:07.064+0000 7fb709200700 1 -- 172.21.15.37:0/1083829178 shutdown_connections 2024-03-29T07:56:07.841 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:07.064+0000 7fb709200700 1 -- 172.21.15.37:0/1083829178 wait complete. 2024-03-29T07:56:07.841 INFO:teuthology.orchestra.run.smithi037.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-03-29T07:56:07.841 INFO:teuthology.orchestra.run.smithi037.stdout:Adding key to root@localhost authorized_keys... 2024-03-29T07:56:07.841 INFO:teuthology.orchestra.run.smithi037.stdout:Adding host smithi037... 2024-03-29T07:56:07.841 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:56:07 smithi037 ceph-mon[35162]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T07:56:09.676 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:56:09 smithi037 ceph-mon[35162]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi037", "addr": "172.21.15.37", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T07:56:09.676 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:56:09 smithi037 ceph-mon[35162]: Deploying cephadm binary to smithi037 2024-03-29T07:56:11.901 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout Added host 'smithi037' with addr '172.21.15.37' 2024-03-29T07:56:11.901 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:08.419+0000 7f3795e00700 1 Processor -- start 2024-03-29T07:56:11.902 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:08.419+0000 7f3795e00700 1 -- start start 2024-03-29T07:56:11.902 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:08.420+0000 7f3795e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3790078d60 0x7f3790077260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T07:56:11.902 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:08.420+0000 7f3795e00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3790079170 con 0x7f3790078d60 2024-03-29T07:56:11.902 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:08.420+0000 7f378ea00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3790078d60 0x7f3790077260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T07:56:11.902 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:08.420+0000 7f378ea00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3790078d60 0x7f3790077260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59178/0 (socket says 172.21.15.37:59178) 2024-03-29T07:56:11.902 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:08.420+0000 7f378ea00700 1 -- 172.21.15.37:0/2010168297 learned_addr learned my addr 172.21.15.37:0/2010168297 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T07:56:11.902 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:08.420+0000 7f378ea00700 1 -- 172.21.15.37:0/2010168297 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37900777a0 con 0x7f3790078d60 2024-03-29T07:56:11.902 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:08.421+0000 7f378ea00700 1 --2- 172.21.15.37:0/2010168297 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3790078d60 0x7f3790077260 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f37800098e0 tx=0x7f3780009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe9d7ad2af63514d server_cookie=0 in_seq=0 out_seq=0 2024-03-29T07:56:11.902 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:08.421+0000 7f378d600700 1 -- 172.21.15.37:0/2010168297 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f378000b860 con 0x7f3790078d60 2024-03-29T07:56:11.902 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:08.421+0000 7f378d600700 1 -- 172.21.15.37:0/2010168297 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1009+0+0 (secure 0 0 0) 0x7f378000be30 con 0x7f3790078d60 2024-03-29T07:56:11.902 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:08.421+0000 7f378d600700 1 -- 172.21.15.37:0/2010168297 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3780010380 con 0x7f3790078d60 2024-03-29T07:56:11.902 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:08.422+0000 7f3795e00700 1 -- 172.21.15.37:0/2010168297 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3790078d60 msgr2=0x7f3790077260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T07:56:11.903 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:08.422+0000 7f3795e00700 1 --2- 172.21.15.37:0/2010168297 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3790078d60 0x7f3790077260 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f37800098e0 tx=0x7f3780009250 comp rx=0 tx=0).stop 2024-03-29T07:56:11.903 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:08.422+0000 7f3795e00700 1 -- 172.21.15.37:0/2010168297 shutdown_connections 2024-03-29T07:56:11.903 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:08.422+0000 7f3795e00700 1 --2- 172.21.15.37:0/2010168297 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3790078d60 0x7f3790077260 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T07:56:11.903 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:08.422+0000 7f3795e00700 1 -- 172.21.15.37:0/2010168297 >> 172.21.15.37:0/2010168297 conn(0x7f37900fdcd0 msgr2=0x7f3790100100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T07:56:11.903 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:08.422+0000 7f3795e00700 1 -- 172.21.15.37:0/2010168297 shutdown_connections 2024-03-29T07:56:11.903 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:08.422+0000 7f3795e00700 1 -- 172.21.15.37:0/2010168297 wait complete. 2024-03-29T07:56:11.903 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:08.423+0000 7f3795e00700 1 Processor -- start 2024-03-29T07:56:11.903 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:08.423+0000 7f3795e00700 1 -- start start 2024-03-29T07:56:11.903 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:08.423+0000 7f3795e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3790078d60 0x7f3790113630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T07:56:11.903 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:08.423+0000 7f3795e00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3790113b70 con 0x7f3790078d60 2024-03-29T07:56:11.903 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:08.423+0000 7f378ea00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3790078d60 0x7f3790113630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T07:56:11.903 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:08.423+0000 7f378ea00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3790078d60 0x7f3790113630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59190/0 (socket says 172.21.15.37:59190) 2024-03-29T07:56:11.903 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:08.423+0000 7f378ea00700 1 -- 172.21.15.37:0/2978229991 learned_addr learned my addr 172.21.15.37:0/2978229991 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T07:56:11.903 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:08.424+0000 7f378ea00700 1 -- 172.21.15.37:0/2978229991 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37800094f0 con 0x7f3790078d60 2024-03-29T07:56:11.903 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:08.424+0000 7f378ea00700 1 --2- 172.21.15.37:0/2978229991 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3790078d60 0x7f3790113630 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f3780006b00 tx=0x7f3780009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T07:56:11.903 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:08.424+0000 7f3786a00700 1 -- 172.21.15.37:0/2978229991 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3780010750 con 0x7f3790078d60 2024-03-29T07:56:11.904 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:08.424+0000 7f3786a00700 1 -- 172.21.15.37:0/2978229991 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1009+0+0 (secure 0 0 0) 0x7f3780010d20 con 0x7f3790078d60 2024-03-29T07:56:11.904 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:08.424+0000 7f3786a00700 1 -- 172.21.15.37:0/2978229991 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f378001acf0 con 0x7f3790078d60 2024-03-29T07:56:11.904 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:08.424+0000 7f3795e00700 1 -- 172.21.15.37:0/2978229991 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3790113d70 con 0x7f3790078d60 2024-03-29T07:56:11.904 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:08.424+0000 7f3795e00700 1 -- 172.21.15.37:0/2978229991 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37901148e0 con 0x7f3790078d60 2024-03-29T07:56:11.904 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:08.425+0000 7f3786a00700 1 -- 172.21.15.37:0/2978229991 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7f378001a5a0 con 0x7f3790078d60 2024-03-29T07:56:11.904 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:08.425+0000 7f3795e00700 1 -- 172.21.15.37:0/2978229991 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f379005f5f0 con 0x7f3790078d60 2024-03-29T07:56:11.904 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:08.426+0000 7f3786a00700 1 --2- 172.21.15.37:0/2978229991 >> [v2:172.21.15.37:6800/2746946658,v1:172.21.15.37:6801/2746946658] conn(0x7f37740367e0 0x7f3774038c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T07:56:11.904 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:08.426+0000 7f3786a00700 1 -- 172.21.15.37:0/2978229991 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f378001e070 con 0x7f3790078d60 2024-03-29T07:56:11.904 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:08.426+0000 7f378e000700 1 --2- 172.21.15.37:0/2978229991 >> [v2:172.21.15.37:6800/2746946658,v1:172.21.15.37:6801/2746946658] conn(0x7f37740367e0 0x7f3774038c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T07:56:11.904 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:08.427+0000 7f378e000700 1 --2- 172.21.15.37:0/2978229991 >> [v2:172.21.15.37:6800/2746946658,v1:172.21.15.37:6801/2746946658] conn(0x7f37740367e0 0x7f3774038c90 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f377c009990 tx=0x7f377c006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T07:56:11.904 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:08.431+0000 7f3786a00700 1 -- 172.21.15.37:0/2978229991 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f3780010e90 con 0x7f3790078d60 2024-03-29T07:56:11.904 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:08.641+0000 7f3795e00700 1 -- 172.21.15.37:0/2978229991 --> [v2:172.21.15.37:6800/2746946658,v1:172.21.15.37:6801/2746946658] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi037", "addr": "172.21.15.37", "target": ["mon-mgr", ""]}) v1 -- 0x7f37900ff090 con 0x7f37740367e0 2024-03-29T07:56:11.904 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:11.228+0000 7f3786a00700 1 -- 172.21.15.37:0/2978229991 <== mgr.14118 v2:172.21.15.37:6800/2746946658 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+48 (secure 0 0 0) 0x7f37900ff090 con 0x7f37740367e0 2024-03-29T07:56:11.904 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:11.231+0000 7f3795e00700 1 -- 172.21.15.37:0/2978229991 >> [v2:172.21.15.37:6800/2746946658,v1:172.21.15.37:6801/2746946658] conn(0x7f37740367e0 msgr2=0x7f3774038c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T07:56:11.905 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:11.231+0000 7f3795e00700 1 --2- 172.21.15.37:0/2978229991 >> [v2:172.21.15.37:6800/2746946658,v1:172.21.15.37:6801/2746946658] conn(0x7f37740367e0 0x7f3774038c90 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f377c009990 tx=0x7f377c006e40 comp rx=0 tx=0).stop 2024-03-29T07:56:11.905 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:11.231+0000 7f3795e00700 1 -- 172.21.15.37:0/2978229991 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3790078d60 msgr2=0x7f3790113630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T07:56:11.905 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:11.231+0000 7f3795e00700 1 --2- 172.21.15.37:0/2978229991 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3790078d60 0x7f3790113630 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f3780006b00 tx=0x7f3780009250 comp rx=0 tx=0).stop 2024-03-29T07:56:11.905 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:11.232+0000 7f3795e00700 1 -- 172.21.15.37:0/2978229991 shutdown_connections 2024-03-29T07:56:11.905 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:11.232+0000 7f3795e00700 1 --2- 172.21.15.37:0/2978229991 >> [v2:172.21.15.37:6800/2746946658,v1:172.21.15.37:6801/2746946658] conn(0x7f37740367e0 0x7f3774038c90 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T07:56:11.905 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:11.232+0000 7f3795e00700 1 --2- 172.21.15.37:0/2978229991 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3790078d60 0x7f3790113630 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T07:56:11.905 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:11.232+0000 7f3795e00700 1 -- 172.21.15.37:0/2978229991 >> 172.21.15.37:0/2978229991 conn(0x7f37900fdcd0 msgr2=0x7f37900fe8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T07:56:11.905 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:11.232+0000 7f3795e00700 1 -- 172.21.15.37:0/2978229991 shutdown_connections 2024-03-29T07:56:11.905 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:11.232+0000 7f3795e00700 1 -- 172.21.15.37:0/2978229991 wait complete. 2024-03-29T07:56:11.905 INFO:teuthology.orchestra.run.smithi037.stdout:Deploying mon service with default placement... 2024-03-29T07:56:12.676 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:56:12 smithi037 ceph-mon[35162]: from='mgr.14118 172.21.15.37:0/2997715107' entity='mgr.smithi037.wjikii' 2024-03-29T07:56:12.676 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:56:12 smithi037 ceph-mon[35162]: Added host smithi037 2024-03-29T07:56:12.676 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:56:12 smithi037 ceph-mon[35162]: from='mgr.14118 172.21.15.37:0/2997715107' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T07:56:13.992 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:56:13 smithi037 ceph-mon[35162]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T07:56:13.992 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:56:13 smithi037 ceph-mon[35162]: Saving service mon spec with placement count:5 2024-03-29T07:56:13.992 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:56:13 smithi037 ceph-mon[35162]: from='mgr.14118 172.21.15.37:0/2997715107' entity='mgr.smithi037.wjikii' 2024-03-29T07:56:14.362 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-03-29T07:56:14.362 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:12.593+0000 7f204e200700 1 Processor -- start 2024-03-29T07:56:14.362 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:12.594+0000 7f204e200700 1 -- start start 2024-03-29T07:56:14.362 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:12.594+0000 7f204e200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2048076c80 0x7f2048102fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T07:56:14.362 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:12.594+0000 7f204e200700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20481034e0 con 0x7f2048076c80 2024-03-29T07:56:14.362 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:12.594+0000 7f2047400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2048076c80 0x7f2048102fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T07:56:14.362 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:12.595+0000 7f2047400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2048076c80 0x7f2048102fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:33346/0 (socket says 172.21.15.37:33346) 2024-03-29T07:56:14.362 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:12.595+0000 7f2047400700 1 -- 172.21.15.37:0/1866774081 learned_addr learned my addr 172.21.15.37:0/1866774081 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T07:56:14.362 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:12.595+0000 7f2047400700 1 -- 172.21.15.37:0/1866774081 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2048103620 con 0x7f2048076c80 2024-03-29T07:56:14.363 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:12.595+0000 7f2047400700 1 --2- 172.21.15.37:0/1866774081 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2048076c80 0x7f2048102fa0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f20380098e0 tx=0x7f2038009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edf7ba690e036395 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T07:56:14.363 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:12.595+0000 7f2046000700 1 -- 172.21.15.37:0/1866774081 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f203800b860 con 0x7f2048076c80 2024-03-29T07:56:14.363 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:12.595+0000 7f2046000700 1 -- 172.21.15.37:0/1866774081 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1009+0+0 (secure 0 0 0) 0x7f203800be30 con 0x7f2048076c80 2024-03-29T07:56:14.363 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:12.595+0000 7f2046000700 1 -- 172.21.15.37:0/1866774081 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2038010380 con 0x7f2048076c80 2024-03-29T07:56:14.363 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:12.596+0000 7f204e200700 1 -- 172.21.15.37:0/1866774081 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2048076c80 msgr2=0x7f2048102fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T07:56:14.363 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:12.596+0000 7f204e200700 1 --2- 172.21.15.37:0/1866774081 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2048076c80 0x7f2048102fa0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f20380098e0 tx=0x7f2038009250 comp rx=0 tx=0).stop 2024-03-29T07:56:14.363 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:12.596+0000 7f204e200700 1 -- 172.21.15.37:0/1866774081 shutdown_connections 2024-03-29T07:56:14.363 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:12.596+0000 7f204e200700 1 --2- 172.21.15.37:0/1866774081 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2048076c80 0x7f2048102fa0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T07:56:14.363 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:12.596+0000 7f204e200700 1 -- 172.21.15.37:0/1866774081 >> 172.21.15.37:0/1866774081 conn(0x7f20480fcc60 msgr2=0x7f20480ff070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T07:56:14.363 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:12.596+0000 7f204e200700 1 -- 172.21.15.37:0/1866774081 shutdown_connections 2024-03-29T07:56:14.363 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:12.596+0000 7f204e200700 1 -- 172.21.15.37:0/1866774081 wait complete. 2024-03-29T07:56:14.363 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:12.597+0000 7f204e200700 1 Processor -- start 2024-03-29T07:56:14.363 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:12.598+0000 7f204e200700 1 -- start start 2024-03-29T07:56:14.364 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:12.598+0000 7f204e200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2048076c80 0x7f2048117970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T07:56:14.364 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:12.598+0000 7f204e200700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2048117eb0 con 0x7f2048076c80 2024-03-29T07:56:14.364 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:12.598+0000 7f2047400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2048076c80 0x7f2048117970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T07:56:14.364 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:12.598+0000 7f2047400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2048076c80 0x7f2048117970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:33358/0 (socket says 172.21.15.37:33358) 2024-03-29T07:56:14.364 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:12.598+0000 7f2047400700 1 -- 172.21.15.37:0/3591722435 learned_addr learned my addr 172.21.15.37:0/3591722435 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T07:56:14.364 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:12.599+0000 7f2047400700 1 -- 172.21.15.37:0/3591722435 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20380094f0 con 0x7f2048076c80 2024-03-29T07:56:14.364 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:12.599+0000 7f2047400700 1 --2- 172.21.15.37:0/3591722435 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2048076c80 0x7f2048117970 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f2038006b00 tx=0x7f2038009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T07:56:14.364 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:12.599+0000 7f203f400700 1 -- 172.21.15.37:0/3591722435 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2038010a80 con 0x7f2048076c80 2024-03-29T07:56:14.364 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:12.599+0000 7f203f400700 1 -- 172.21.15.37:0/3591722435 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1009+0+0 (secure 0 0 0) 0x7f203801a430 con 0x7f2048076c80 2024-03-29T07:56:14.364 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:12.599+0000 7f203f400700 1 -- 172.21.15.37:0/3591722435 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f20380245b0 con 0x7f2048076c80 2024-03-29T07:56:14.364 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:12.599+0000 7f204e200700 1 -- 172.21.15.37:0/3591722435 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20481180b0 con 0x7f2048076c80 2024-03-29T07:56:14.364 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:12.599+0000 7f204e200700 1 -- 172.21.15.37:0/3591722435 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2048118c20 con 0x7f2048076c80 2024-03-29T07:56:14.364 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:12.600+0000 7f203f400700 1 -- 172.21.15.37:0/3591722435 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7f20380247e0 con 0x7f2048076c80 2024-03-29T07:56:14.364 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:12.600+0000 7f204e200700 1 -- 172.21.15.37:0/3591722435 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f204805f5f0 con 0x7f2048076c80 2024-03-29T07:56:14.365 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:12.601+0000 7f203f400700 1 --2- 172.21.15.37:0/3591722435 >> [v2:172.21.15.37:6800/2746946658,v1:172.21.15.37:6801/2746946658] conn(0x7f202c036830 0x7f202c038ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T07:56:14.365 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:12.601+0000 7f203f400700 1 -- 172.21.15.37:0/3591722435 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f203804adc0 con 0x7f2048076c80 2024-03-29T07:56:14.365 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:12.601+0000 7f2046a00700 1 --2- 172.21.15.37:0/3591722435 >> [v2:172.21.15.37:6800/2746946658,v1:172.21.15.37:6801/2746946658] conn(0x7f202c036830 0x7f202c038ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T07:56:14.365 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:12.602+0000 7f2046a00700 1 --2- 172.21.15.37:0/3591722435 >> [v2:172.21.15.37:6800/2746946658,v1:172.21.15.37:6801/2746946658] conn(0x7f202c036830 0x7f202c038ce0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f2034009990 tx=0x7f2034006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T07:56:14.365 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:12.606+0000 7f203f400700 1 -- 172.21.15.37:0/3591722435 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f2038018350 con 0x7f2048076c80 2024-03-29T07:56:14.365 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:12.833+0000 7f204e200700 1 -- 172.21.15.37:0/3591722435 --> [v2:172.21.15.37:6800/2746946658,v1:172.21.15.37:6801/2746946658] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "target": ["mon-mgr", ""]}) v1 -- 0x7f20481197a0 con 0x7f202c036830 2024-03-29T07:56:14.365 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:12.838+0000 7f203f400700 1 -- 172.21.15.37:0/3591722435 <== mgr.14118 v2:172.21.15.37:6800/2746946658 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f20481197a0 con 0x7f202c036830 2024-03-29T07:56:14.365 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:12.841+0000 7f204e200700 1 -- 172.21.15.37:0/3591722435 >> [v2:172.21.15.37:6800/2746946658,v1:172.21.15.37:6801/2746946658] conn(0x7f202c036830 msgr2=0x7f202c038ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T07:56:14.365 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:12.841+0000 7f204e200700 1 --2- 172.21.15.37:0/3591722435 >> [v2:172.21.15.37:6800/2746946658,v1:172.21.15.37:6801/2746946658] conn(0x7f202c036830 0x7f202c038ce0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f2034009990 tx=0x7f2034006e40 comp rx=0 tx=0).stop 2024-03-29T07:56:14.365 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:12.841+0000 7f204e200700 1 -- 172.21.15.37:0/3591722435 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2048076c80 msgr2=0x7f2048117970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T07:56:14.365 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:12.841+0000 7f204e200700 1 --2- 172.21.15.37:0/3591722435 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2048076c80 0x7f2048117970 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f2038006b00 tx=0x7f2038009250 comp rx=0 tx=0).stop 2024-03-29T07:56:14.365 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:12.841+0000 7f204e200700 1 -- 172.21.15.37:0/3591722435 shutdown_connections 2024-03-29T07:56:14.366 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:12.841+0000 7f204e200700 1 --2- 172.21.15.37:0/3591722435 >> [v2:172.21.15.37:6800/2746946658,v1:172.21.15.37:6801/2746946658] conn(0x7f202c036830 0x7f202c038ce0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T07:56:14.366 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:12.841+0000 7f204e200700 1 --2- 172.21.15.37:0/3591722435 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2048076c80 0x7f2048117970 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T07:56:14.366 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:12.841+0000 7f204e200700 1 -- 172.21.15.37:0/3591722435 >> 172.21.15.37:0/3591722435 conn(0x7f20480fcc60 msgr2=0x7f2048106a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T07:56:14.366 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:12.841+0000 7f204e200700 1 -- 172.21.15.37:0/3591722435 shutdown_connections 2024-03-29T07:56:14.366 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:12.841+0000 7f204e200700 1 -- 172.21.15.37:0/3591722435 wait complete. 2024-03-29T07:56:14.366 INFO:teuthology.orchestra.run.smithi037.stdout:Deploying mgr service with default placement... 2024-03-29T07:56:15.426 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:56:15 smithi037 ceph-mon[35162]: from='mgr.14118 172.21.15.37:0/2997715107' entity='mgr.smithi037.wjikii' 2024-03-29T07:56:15.426 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:56:15 smithi037 ceph-mon[35162]: from='mgr.14118 172.21.15.37:0/2997715107' entity='mgr.smithi037.wjikii' 2024-03-29T07:56:15.831 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-03-29T07:56:15.831 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:14.967+0000 7f046da00700 1 Processor -- start 2024-03-29T07:56:15.831 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:14.967+0000 7f046da00700 1 -- start start 2024-03-29T07:56:15.831 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:14.968+0000 7f046da00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0468078d50 0x7f0468077260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T07:56:15.831 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:14.968+0000 7f046da00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0468079160 con 0x7f0468078d50 2024-03-29T07:56:15.831 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:14.968+0000 7f0466a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0468078d50 0x7f0468077260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T07:56:15.832 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:14.968+0000 7f0466a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0468078d50 0x7f0468077260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:33366/0 (socket says 172.21.15.37:33366) 2024-03-29T07:56:15.832 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:14.968+0000 7f0466a00700 1 -- 172.21.15.37:0/884347803 learned_addr learned my addr 172.21.15.37:0/884347803 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T07:56:15.832 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:14.968+0000 7f0466a00700 1 -- 172.21.15.37:0/884347803 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04680777a0 con 0x7f0468078d50 2024-03-29T07:56:15.832 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:14.969+0000 7f0466a00700 1 --2- 172.21.15.37:0/884347803 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0468078d50 0x7f0468077260 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f04580098e0 tx=0x7f0458009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1c167bbda00359 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T07:56:15.832 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:14.969+0000 7f0465600700 1 -- 172.21.15.37:0/884347803 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f045800b860 con 0x7f0468078d50 2024-03-29T07:56:15.832 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:14.969+0000 7f0465600700 1 -- 172.21.15.37:0/884347803 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f045800be70 con 0x7f0468078d50 2024-03-29T07:56:15.832 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:14.969+0000 7f0465600700 1 -- 172.21.15.37:0/884347803 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0458010380 con 0x7f0468078d50 2024-03-29T07:56:15.832 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:14.970+0000 7f046da00700 1 -- 172.21.15.37:0/884347803 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0468078d50 msgr2=0x7f0468077260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T07:56:15.832 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:14.970+0000 7f046da00700 1 --2- 172.21.15.37:0/884347803 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0468078d50 0x7f0468077260 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f04580098e0 tx=0x7f0458009250 comp rx=0 tx=0).stop 2024-03-29T07:56:15.832 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:14.970+0000 7f046da00700 1 -- 172.21.15.37:0/884347803 shutdown_connections 2024-03-29T07:56:15.832 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:14.970+0000 7f046da00700 1 --2- 172.21.15.37:0/884347803 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0468078d50 0x7f0468077260 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T07:56:15.832 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:14.970+0000 7f046da00700 1 -- 172.21.15.37:0/884347803 >> 172.21.15.37:0/884347803 conn(0x7f04680fdd00 msgr2=0x7f0468100110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T07:56:15.832 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:14.970+0000 7f046da00700 1 -- 172.21.15.37:0/884347803 shutdown_connections 2024-03-29T07:56:15.832 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:14.970+0000 7f046da00700 1 -- 172.21.15.37:0/884347803 wait complete. 2024-03-29T07:56:15.832 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:14.971+0000 7f046da00700 1 Processor -- start 2024-03-29T07:56:15.833 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:14.971+0000 7f046da00700 1 -- start start 2024-03-29T07:56:15.833 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:14.972+0000 7f046da00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0468078d50 0x7f04681135a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T07:56:15.833 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:14.972+0000 7f046da00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0468113ae0 con 0x7f0468078d50 2024-03-29T07:56:15.833 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:14.972+0000 7f0466a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0468078d50 0x7f04681135a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T07:56:15.833 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:14.972+0000 7f0466a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0468078d50 0x7f04681135a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:33380/0 (socket says 172.21.15.37:33380) 2024-03-29T07:56:15.833 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:14.972+0000 7f0466a00700 1 -- 172.21.15.37:0/2547904729 learned_addr learned my addr 172.21.15.37:0/2547904729 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T07:56:15.833 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:14.972+0000 7f0466a00700 1 -- 172.21.15.37:0/2547904729 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04580094f0 con 0x7f0468078d50 2024-03-29T07:56:15.833 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:14.973+0000 7f0466a00700 1 --2- 172.21.15.37:0/2547904729 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0468078d50 0x7f04681135a0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f04580094c0 tx=0x7f0458009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T07:56:15.833 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:14.973+0000 7f045ea00700 1 -- 172.21.15.37:0/2547904729 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0458010850 con 0x7f0468078d50 2024-03-29T07:56:15.833 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:14.973+0000 7f045ea00700 1 -- 172.21.15.37:0/2547904729 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f0458010e60 con 0x7f0468078d50 2024-03-29T07:56:15.833 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:14.973+0000 7f046da00700 1 -- 172.21.15.37:0/2547904729 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0468113ce0 con 0x7f0468078d50 2024-03-29T07:56:15.833 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:14.973+0000 7f045ea00700 1 -- 172.21.15.37:0/2547904729 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f045801a430 con 0x7f0468078d50 2024-03-29T07:56:15.833 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:14.973+0000 7f046da00700 1 -- 172.21.15.37:0/2547904729 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0468114850 con 0x7f0468078d50 2024-03-29T07:56:15.833 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:14.974+0000 7f045ea00700 1 -- 172.21.15.37:0/2547904729 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7f045801a6b0 con 0x7f0468078d50 2024-03-29T07:56:15.833 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:14.974+0000 7f046da00700 1 -- 172.21.15.37:0/2547904729 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f046805f5f0 con 0x7f0468078d50 2024-03-29T07:56:15.834 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:14.975+0000 7f045ea00700 1 --2- 172.21.15.37:0/2547904729 >> [v2:172.21.15.37:6800/2746946658,v1:172.21.15.37:6801/2746946658] conn(0x7f044c036790 0x7f044c038c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T07:56:15.834 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:14.975+0000 7f045ea00700 1 -- 172.21.15.37:0/2547904729 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f045804a6f0 con 0x7f0468078d50 2024-03-29T07:56:15.834 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:14.975+0000 7f0466000700 1 --2- 172.21.15.37:0/2547904729 >> [v2:172.21.15.37:6800/2746946658,v1:172.21.15.37:6801/2746946658] conn(0x7f044c036790 0x7f044c038c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T07:56:15.834 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:14.976+0000 7f0466000700 1 --2- 172.21.15.37:0/2547904729 >> [v2:172.21.15.37:6800/2746946658,v1:172.21.15.37:6801/2746946658] conn(0x7f044c036790 0x7f044c038c40 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f0454009990 tx=0x7f0454006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T07:56:15.834 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:14.980+0000 7f045ea00700 1 -- 172.21.15.37:0/2547904729 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f0458018530 con 0x7f0468078d50 2024-03-29T07:56:15.834 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:15.218+0000 7f046da00700 1 -- 172.21.15.37:0/2547904729 --> [v2:172.21.15.37:6800/2746946658,v1:172.21.15.37:6801/2746946658] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "target": ["mon-mgr", ""]}) v1 -- 0x7f04680fef50 con 0x7f044c036790 2024-03-29T07:56:15.834 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:15.223+0000 7f045ea00700 1 -- 172.21.15.37:0/2547904729 <== mgr.14118 v2:172.21.15.37:6800/2746946658 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f04680fef50 con 0x7f044c036790 2024-03-29T07:56:15.834 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:15.225+0000 7f046da00700 1 -- 172.21.15.37:0/2547904729 >> [v2:172.21.15.37:6800/2746946658,v1:172.21.15.37:6801/2746946658] conn(0x7f044c036790 msgr2=0x7f044c038c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T07:56:15.834 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:15.225+0000 7f046da00700 1 --2- 172.21.15.37:0/2547904729 >> [v2:172.21.15.37:6800/2746946658,v1:172.21.15.37:6801/2746946658] conn(0x7f044c036790 0x7f044c038c40 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f0454009990 tx=0x7f0454006e40 comp rx=0 tx=0).stop 2024-03-29T07:56:15.834 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:15.225+0000 7f046da00700 1 -- 172.21.15.37:0/2547904729 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0468078d50 msgr2=0x7f04681135a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T07:56:15.834 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:15.225+0000 7f046da00700 1 --2- 172.21.15.37:0/2547904729 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0468078d50 0x7f04681135a0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f04580094c0 tx=0x7f0458009250 comp rx=0 tx=0).stop 2024-03-29T07:56:15.834 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:15.225+0000 7f046da00700 1 -- 172.21.15.37:0/2547904729 shutdown_connections 2024-03-29T07:56:15.834 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:15.225+0000 7f046da00700 1 --2- 172.21.15.37:0/2547904729 >> [v2:172.21.15.37:6800/2746946658,v1:172.21.15.37:6801/2746946658] conn(0x7f044c036790 0x7f044c038c40 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T07:56:15.835 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:15.225+0000 7f046da00700 1 --2- 172.21.15.37:0/2547904729 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0468078d50 0x7f04681135a0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T07:56:15.835 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:15.225+0000 7f046da00700 1 -- 172.21.15.37:0/2547904729 >> 172.21.15.37:0/2547904729 conn(0x7f04680fdd00 msgr2=0x7f04680fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T07:56:15.835 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:15.226+0000 7f046da00700 1 -- 172.21.15.37:0/2547904729 shutdown_connections 2024-03-29T07:56:15.835 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:15.226+0000 7f046da00700 1 -- 172.21.15.37:0/2547904729 wait complete. 2024-03-29T07:56:15.835 INFO:teuthology.orchestra.run.smithi037.stdout:Deploying crash service with default placement... 2024-03-29T07:56:16.676 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:56:16 smithi037 ceph-mon[35162]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T07:56:16.676 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:56:16 smithi037 ceph-mon[35162]: Saving service mgr spec with placement count:2 2024-03-29T07:56:16.676 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:56:16 smithi037 ceph-mon[35162]: from='mgr.14118 172.21.15.37:0/2997715107' entity='mgr.smithi037.wjikii' 2024-03-29T07:56:17.469 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout Scheduled crash update... 2024-03-29T07:56:17.469 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:16.599+0000 7f209dc00700 1 Processor -- start 2024-03-29T07:56:17.469 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:16.599+0000 7f209dc00700 1 -- start start 2024-03-29T07:56:17.469 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:16.599+0000 7f209dc00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2098078d70 0x7f2098077280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T07:56:17.469 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:16.599+0000 7f209dc00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2098079180 con 0x7f2098078d70 2024-03-29T07:56:17.469 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:16.600+0000 7f2096a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2098078d70 0x7f2098077280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T07:56:17.469 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:16.600+0000 7f2096a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2098078d70 0x7f2098077280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:33390/0 (socket says 172.21.15.37:33390) 2024-03-29T07:56:17.469 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:16.600+0000 7f2096a00700 1 -- 172.21.15.37:0/3334971145 learned_addr learned my addr 172.21.15.37:0/3334971145 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T07:56:17.469 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:16.600+0000 7f2096a00700 1 -- 172.21.15.37:0/3334971145 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20980777c0 con 0x7f2098078d70 2024-03-29T07:56:17.469 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:16.600+0000 7f2096a00700 1 --2- 172.21.15.37:0/3334971145 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2098078d70 0x7f2098077280 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f20880098e0 tx=0x7f2088009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=569877868d3dbfcd server_cookie=0 in_seq=0 out_seq=0 2024-03-29T07:56:17.470 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:16.600+0000 7f2095600700 1 -- 172.21.15.37:0/3334971145 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f208800b860 con 0x7f2098078d70 2024-03-29T07:56:17.470 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:16.601+0000 7f2095600700 1 -- 172.21.15.37:0/3334971145 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f208800be70 con 0x7f2098078d70 2024-03-29T07:56:17.470 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:16.601+0000 7f2095600700 1 -- 172.21.15.37:0/3334971145 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2088010380 con 0x7f2098078d70 2024-03-29T07:56:17.470 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:16.601+0000 7f209dc00700 1 -- 172.21.15.37:0/3334971145 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2098078d70 msgr2=0x7f2098077280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T07:56:17.470 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:16.601+0000 7f209dc00700 1 --2- 172.21.15.37:0/3334971145 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2098078d70 0x7f2098077280 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f20880098e0 tx=0x7f2088009250 comp rx=0 tx=0).stop 2024-03-29T07:56:17.470 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:16.601+0000 7f209dc00700 1 -- 172.21.15.37:0/3334971145 shutdown_connections 2024-03-29T07:56:17.470 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:16.601+0000 7f209dc00700 1 --2- 172.21.15.37:0/3334971145 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2098078d70 0x7f2098077280 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T07:56:17.470 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:16.601+0000 7f209dc00700 1 -- 172.21.15.37:0/3334971145 >> 172.21.15.37:0/3334971145 conn(0x7f20980fdd00 msgr2=0x7f2098100110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T07:56:17.470 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:16.601+0000 7f209dc00700 1 -- 172.21.15.37:0/3334971145 shutdown_connections 2024-03-29T07:56:17.470 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:16.601+0000 7f209dc00700 1 -- 172.21.15.37:0/3334971145 wait complete. 2024-03-29T07:56:17.470 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:16.602+0000 7f209dc00700 1 Processor -- start 2024-03-29T07:56:17.470 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:16.602+0000 7f209dc00700 1 -- start start 2024-03-29T07:56:17.470 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:16.603+0000 7f209dc00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2098078d70 0x7f20981135c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T07:56:17.470 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:16.603+0000 7f209dc00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2098113b00 con 0x7f2098078d70 2024-03-29T07:56:17.471 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:16.603+0000 7f2096a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2098078d70 0x7f20981135c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T07:56:17.471 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:16.603+0000 7f2096a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2098078d70 0x7f20981135c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:33398/0 (socket says 172.21.15.37:33398) 2024-03-29T07:56:17.471 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:16.603+0000 7f2096a00700 1 -- 172.21.15.37:0/514594261 learned_addr learned my addr 172.21.15.37:0/514594261 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T07:56:17.471 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:16.603+0000 7f2096a00700 1 -- 172.21.15.37:0/514594261 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20880094f0 con 0x7f2098078d70 2024-03-29T07:56:17.471 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:16.603+0000 7f2096a00700 1 --2- 172.21.15.37:0/514594261 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2098078d70 0x7f20981135c0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f2088006d40 tx=0x7f2088009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T07:56:17.471 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:16.604+0000 7f208ea00700 1 -- 172.21.15.37:0/514594261 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2088010960 con 0x7f2098078d70 2024-03-29T07:56:17.471 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:16.604+0000 7f208ea00700 1 -- 172.21.15.37:0/514594261 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f208801a430 con 0x7f2098078d70 2024-03-29T07:56:17.471 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:16.604+0000 7f209dc00700 1 -- 172.21.15.37:0/514594261 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2098113d00 con 0x7f2098078d70 2024-03-29T07:56:17.471 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:16.604+0000 7f208ea00700 1 -- 172.21.15.37:0/514594261 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f20880245b0 con 0x7f2098078d70 2024-03-29T07:56:17.471 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:16.604+0000 7f209dc00700 1 -- 172.21.15.37:0/514594261 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2098114870 con 0x7f2098078d70 2024-03-29T07:56:17.472 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:16.605+0000 7f208ea00700 1 -- 172.21.15.37:0/514594261 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7f208802d3f0 con 0x7f2098078d70 2024-03-29T07:56:17.472 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:16.605+0000 7f209dc00700 1 -- 172.21.15.37:0/514594261 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f209805f5f0 con 0x7f2098078d70 2024-03-29T07:56:17.472 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:16.605+0000 7f208ea00700 1 --2- 172.21.15.37:0/514594261 >> [v2:172.21.15.37:6800/2746946658,v1:172.21.15.37:6801/2746946658] conn(0x7f207c036830 0x7f207c038ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T07:56:17.472 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:16.605+0000 7f208ea00700 1 -- 172.21.15.37:0/514594261 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f208804a4f0 con 0x7f2098078d70 2024-03-29T07:56:17.472 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:16.606+0000 7f2096000700 1 --2- 172.21.15.37:0/514594261 >> [v2:172.21.15.37:6800/2746946658,v1:172.21.15.37:6801/2746946658] conn(0x7f207c036830 0x7f207c038ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T07:56:17.472 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:16.606+0000 7f2096000700 1 --2- 172.21.15.37:0/514594261 >> [v2:172.21.15.37:6800/2746946658,v1:172.21.15.37:6801/2746946658] conn(0x7f207c036830 0x7f207c038ce0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f2084009990 tx=0x7f2084006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T07:56:17.472 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:16.610+0000 7f208ea00700 1 -- 172.21.15.37:0/514594261 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f2088018b00 con 0x7f2098078d70 2024-03-29T07:56:17.472 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:16.825+0000 7f209dc00700 1 -- 172.21.15.37:0/514594261 --> [v2:172.21.15.37:6800/2746946658,v1:172.21.15.37:6801/2746946658] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "crash", "target": ["mon-mgr", ""]}) v1 -- 0x7f20980fef90 con 0x7f207c036830 2024-03-29T07:56:17.472 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:16.830+0000 7f208ea00700 1 -- 172.21.15.37:0/514594261 <== mgr.14118 v2:172.21.15.37:6800/2746946658 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+26 (secure 0 0 0) 0x7f20980fef90 con 0x7f207c036830 2024-03-29T07:56:17.472 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:16.832+0000 7f209dc00700 1 -- 172.21.15.37:0/514594261 >> [v2:172.21.15.37:6800/2746946658,v1:172.21.15.37:6801/2746946658] conn(0x7f207c036830 msgr2=0x7f207c038ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T07:56:17.472 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:16.832+0000 7f209dc00700 1 --2- 172.21.15.37:0/514594261 >> [v2:172.21.15.37:6800/2746946658,v1:172.21.15.37:6801/2746946658] conn(0x7f207c036830 0x7f207c038ce0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f2084009990 tx=0x7f2084006e40 comp rx=0 tx=0).stop 2024-03-29T07:56:17.472 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:16.832+0000 7f209dc00700 1 -- 172.21.15.37:0/514594261 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2098078d70 msgr2=0x7f20981135c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T07:56:17.472 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:16.832+0000 7f209dc00700 1 --2- 172.21.15.37:0/514594261 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2098078d70 0x7f20981135c0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f2088006d40 tx=0x7f2088009250 comp rx=0 tx=0).stop 2024-03-29T07:56:17.472 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:16.832+0000 7f209dc00700 1 -- 172.21.15.37:0/514594261 shutdown_connections 2024-03-29T07:56:17.473 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:16.832+0000 7f209dc00700 1 --2- 172.21.15.37:0/514594261 >> [v2:172.21.15.37:6800/2746946658,v1:172.21.15.37:6801/2746946658] conn(0x7f207c036830 0x7f207c038ce0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T07:56:17.473 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:16.832+0000 7f209dc00700 1 --2- 172.21.15.37:0/514594261 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2098078d70 0x7f20981135c0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T07:56:17.473 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:16.832+0000 7f209dc00700 1 -- 172.21.15.37:0/514594261 >> 172.21.15.37:0/514594261 conn(0x7f20980fdd00 msgr2=0x7f20980fe880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T07:56:17.473 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:16.832+0000 7f209dc00700 1 -- 172.21.15.37:0/514594261 shutdown_connections 2024-03-29T07:56:17.473 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:16.832+0000 7f209dc00700 1 -- 172.21.15.37:0/514594261 wait complete. 2024-03-29T07:56:17.473 INFO:teuthology.orchestra.run.smithi037.stdout:Deploying prometheus service with default placement... 2024-03-29T07:56:18.176 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:56:17 smithi037 ceph-mon[35162]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "crash", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T07:56:18.176 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:56:17 smithi037 ceph-mon[35162]: Saving service crash spec with placement * 2024-03-29T07:56:18.176 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:56:17 smithi037 ceph-mon[35162]: from='mgr.14118 172.21.15.37:0/2997715107' entity='mgr.smithi037.wjikii' 2024-03-29T07:56:18.176 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:56:17 smithi037 ceph-mon[35162]: from='mgr.14118 172.21.15.37:0/2997715107' entity='mgr.smithi037.wjikii' 2024-03-29T07:56:18.922 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout Scheduled prometheus update... 2024-03-29T07:56:18.922 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:18.097+0000 7fae34c00700 1 Processor -- start 2024-03-29T07:56:18.923 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:18.097+0000 7fae34c00700 1 -- start start 2024-03-29T07:56:18.923 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:18.097+0000 7fae34c00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fae30102580 0x7fae30102990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T07:56:18.923 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:18.097+0000 7fae34c00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae30102ed0 con 0x7fae30102580 2024-03-29T07:56:18.923 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:18.098+0000 7fae2de00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fae30102580 0x7fae30102990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T07:56:18.923 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:18.098+0000 7fae2de00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fae30102580 0x7fae30102990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:33408/0 (socket says 172.21.15.37:33408) 2024-03-29T07:56:18.923 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:18.098+0000 7fae2de00700 1 -- 172.21.15.37:0/2334331326 learned_addr learned my addr 172.21.15.37:0/2334331326 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T07:56:18.923 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:18.098+0000 7fae2de00700 1 -- 172.21.15.37:0/2334331326 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae30103010 con 0x7fae30102580 2024-03-29T07:56:18.923 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:18.099+0000 7fae2de00700 1 --2- 172.21.15.37:0/2334331326 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fae30102580 0x7fae30102990 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fae200098e0 tx=0x7fae20009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db442c1b444a43da server_cookie=0 in_seq=0 out_seq=0 2024-03-29T07:56:18.923 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:18.099+0000 7fae2ca00700 1 -- 172.21.15.37:0/2334331326 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fae2000b860 con 0x7fae30102580 2024-03-29T07:56:18.923 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:18.099+0000 7fae2ca00700 1 -- 172.21.15.37:0/2334331326 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fae2000be70 con 0x7fae30102580 2024-03-29T07:56:18.923 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:18.099+0000 7fae2ca00700 1 -- 172.21.15.37:0/2334331326 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fae20010380 con 0x7fae30102580 2024-03-29T07:56:18.923 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:18.099+0000 7fae34c00700 1 -- 172.21.15.37:0/2334331326 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fae30102580 msgr2=0x7fae30102990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T07:56:18.923 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:18.099+0000 7fae34c00700 1 --2- 172.21.15.37:0/2334331326 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fae30102580 0x7fae30102990 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fae200098e0 tx=0x7fae20009250 comp rx=0 tx=0).stop 2024-03-29T07:56:18.924 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:18.100+0000 7fae34c00700 1 -- 172.21.15.37:0/2334331326 shutdown_connections 2024-03-29T07:56:18.924 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:18.100+0000 7fae34c00700 1 --2- 172.21.15.37:0/2334331326 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fae30102580 0x7fae30102990 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T07:56:18.924 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:18.100+0000 7fae34c00700 1 -- 172.21.15.37:0/2334331326 >> 172.21.15.37:0/2334331326 conn(0x7fae300fdd00 msgr2=0x7fae30100110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T07:56:18.924 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:18.100+0000 7fae34c00700 1 -- 172.21.15.37:0/2334331326 shutdown_connections 2024-03-29T07:56:18.924 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:18.100+0000 7fae34c00700 1 -- 172.21.15.37:0/2334331326 wait complete. 2024-03-29T07:56:18.924 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:18.101+0000 7fae34c00700 1 Processor -- start 2024-03-29T07:56:18.924 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:18.101+0000 7fae34c00700 1 -- start start 2024-03-29T07:56:18.924 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:18.101+0000 7fae34c00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fae30102580 0x7fae30113580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T07:56:18.924 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:18.101+0000 7fae34c00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae30113ac0 con 0x7fae30102580 2024-03-29T07:56:18.924 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:18.101+0000 7fae2de00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fae30102580 0x7fae30113580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T07:56:18.924 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:18.101+0000 7fae2de00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fae30102580 0x7fae30113580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:33414/0 (socket says 172.21.15.37:33414) 2024-03-29T07:56:18.924 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:18.101+0000 7fae2de00700 1 -- 172.21.15.37:0/913481951 learned_addr learned my addr 172.21.15.37:0/913481951 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T07:56:18.924 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:18.102+0000 7fae2de00700 1 -- 172.21.15.37:0/913481951 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae200094f0 con 0x7fae30102580 2024-03-29T07:56:18.924 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:18.102+0000 7fae2de00700 1 --2- 172.21.15.37:0/913481951 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fae30102580 0x7fae30113580 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fae2000c010 tx=0x7fae20009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T07:56:18.924 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:18.102+0000 7fae26000700 1 -- 172.21.15.37:0/913481951 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fae20010960 con 0x7fae30102580 2024-03-29T07:56:18.925 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:18.102+0000 7fae26000700 1 -- 172.21.15.37:0/913481951 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fae2001a430 con 0x7fae30102580 2024-03-29T07:56:18.925 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:18.102+0000 7fae34c00700 1 -- 172.21.15.37:0/913481951 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae30113cc0 con 0x7fae30102580 2024-03-29T07:56:18.925 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:18.102+0000 7fae26000700 1 -- 172.21.15.37:0/913481951 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fae20019580 con 0x7fae30102580 2024-03-29T07:56:18.925 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:18.102+0000 7fae34c00700 1 -- 172.21.15.37:0/913481951 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae30114830 con 0x7fae30102580 2024-03-29T07:56:18.925 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:18.103+0000 7fae26000700 1 -- 172.21.15.37:0/913481951 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7fae200197b0 con 0x7fae30102580 2024-03-29T07:56:18.925 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:18.104+0000 7fae34c00700 1 -- 172.21.15.37:0/913481951 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae3005f5f0 con 0x7fae30102580 2024-03-29T07:56:18.925 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:18.104+0000 7fae26000700 1 --2- 172.21.15.37:0/913481951 >> [v2:172.21.15.37:6800/2746946658,v1:172.21.15.37:6801/2746946658] conn(0x7fae140367e0 0x7fae14038c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T07:56:18.925 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:18.104+0000 7fae26000700 1 -- 172.21.15.37:0/913481951 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7fae2004a620 con 0x7fae30102580 2024-03-29T07:56:18.925 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:18.104+0000 7fae2d400700 1 --2- 172.21.15.37:0/913481951 >> [v2:172.21.15.37:6800/2746946658,v1:172.21.15.37:6801/2746946658] conn(0x7fae140367e0 0x7fae14038c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T07:56:18.925 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:18.105+0000 7fae2d400700 1 --2- 172.21.15.37:0/913481951 >> [v2:172.21.15.37:6800/2746946658,v1:172.21.15.37:6801/2746946658] conn(0x7fae140367e0 0x7fae14038c90 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fae1c009990 tx=0x7fae1c006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T07:56:18.925 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:18.109+0000 7fae26000700 1 -- 172.21.15.37:0/913481951 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fae2002b950 con 0x7fae30102580 2024-03-29T07:56:18.925 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:18.331+0000 7fae34c00700 1 -- 172.21.15.37:0/913481951 --> [v2:172.21.15.37:6800/2746946658,v1:172.21.15.37:6801/2746946658] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "prometheus", "target": ["mon-mgr", ""]}) v1 -- 0x7fae300fef40 con 0x7fae140367e0 2024-03-29T07:56:18.925 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:18.337+0000 7fae26000700 1 -- 172.21.15.37:0/913481951 <== mgr.14118 v2:172.21.15.37:6800/2746946658 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+31 (secure 0 0 0) 0x7fae300fef40 con 0x7fae140367e0 2024-03-29T07:56:18.925 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:18.339+0000 7fae34c00700 1 -- 172.21.15.37:0/913481951 >> [v2:172.21.15.37:6800/2746946658,v1:172.21.15.37:6801/2746946658] conn(0x7fae140367e0 msgr2=0x7fae14038c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T07:56:18.925 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:18.339+0000 7fae34c00700 1 --2- 172.21.15.37:0/913481951 >> [v2:172.21.15.37:6800/2746946658,v1:172.21.15.37:6801/2746946658] conn(0x7fae140367e0 0x7fae14038c90 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fae1c009990 tx=0x7fae1c006e40 comp rx=0 tx=0).stop 2024-03-29T07:56:18.926 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:18.339+0000 7fae34c00700 1 -- 172.21.15.37:0/913481951 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fae30102580 msgr2=0x7fae30113580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T07:56:18.926 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:18.339+0000 7fae34c00700 1 --2- 172.21.15.37:0/913481951 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fae30102580 0x7fae30113580 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fae2000c010 tx=0x7fae20009250 comp rx=0 tx=0).stop 2024-03-29T07:56:18.926 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:18.339+0000 7fae34c00700 1 -- 172.21.15.37:0/913481951 shutdown_connections 2024-03-29T07:56:18.926 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:18.339+0000 7fae34c00700 1 --2- 172.21.15.37:0/913481951 >> [v2:172.21.15.37:6800/2746946658,v1:172.21.15.37:6801/2746946658] conn(0x7fae140367e0 0x7fae14038c90 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T07:56:18.926 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:18.339+0000 7fae34c00700 1 --2- 172.21.15.37:0/913481951 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fae30102580 0x7fae30113580 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T07:56:18.926 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:18.339+0000 7fae34c00700 1 -- 172.21.15.37:0/913481951 >> 172.21.15.37:0/913481951 conn(0x7fae300fdd00 msgr2=0x7fae300fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T07:56:18.926 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:18.340+0000 7fae34c00700 1 -- 172.21.15.37:0/913481951 shutdown_connections 2024-03-29T07:56:18.926 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:18.340+0000 7fae34c00700 1 -- 172.21.15.37:0/913481951 wait complete. 2024-03-29T07:56:18.926 INFO:teuthology.orchestra.run.smithi037.stdout:Deploying grafana service with default placement... 2024-03-29T07:56:19.676 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:56:19 smithi037 ceph-mon[35162]: from='client.14146 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T07:56:19.676 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:56:19 smithi037 ceph-mon[35162]: Saving service prometheus spec with placement count:1 2024-03-29T07:56:19.676 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:56:19 smithi037 ceph-mon[35162]: from='mgr.14118 172.21.15.37:0/2997715107' entity='mgr.smithi037.wjikii' 2024-03-29T07:56:19.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:56:19 smithi037 ceph-mon[35162]: from='mgr.14118 172.21.15.37:0/2997715107' entity='mgr.smithi037.wjikii' 2024-03-29T07:56:20.908 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout Scheduled grafana update... 2024-03-29T07:56:20.908 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:19.558+0000 7f5105a00700 1 Processor -- start 2024-03-29T07:56:20.908 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:19.558+0000 7f5105a00700 1 -- start start 2024-03-29T07:56:20.908 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:19.558+0000 7f5105a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5100078d50 0x7f5100077260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T07:56:20.908 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:19.558+0000 7f5105a00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5100079160 con 0x7f5100078d50 2024-03-29T07:56:20.908 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:19.559+0000 7f50fea00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5100078d50 0x7f5100077260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T07:56:20.908 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:19.559+0000 7f50fea00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5100078d50 0x7f5100077260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:33418/0 (socket says 172.21.15.37:33418) 2024-03-29T07:56:20.908 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:19.559+0000 7f50fea00700 1 -- 172.21.15.37:0/2533726575 learned_addr learned my addr 172.21.15.37:0/2533726575 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T07:56:20.909 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:19.559+0000 7f50fea00700 1 -- 172.21.15.37:0/2533726575 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51000777a0 con 0x7f5100078d50 2024-03-29T07:56:20.909 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:19.559+0000 7f50fea00700 1 --2- 172.21.15.37:0/2533726575 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5100078d50 0x7f5100077260 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f50f00098e0 tx=0x7f50f0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2b6f5ca08acd305 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T07:56:20.909 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:19.560+0000 7f50fd600700 1 -- 172.21.15.37:0/2533726575 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f50f000b860 con 0x7f5100078d50 2024-03-29T07:56:20.909 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:19.560+0000 7f50fd600700 1 -- 172.21.15.37:0/2533726575 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f50f000be70 con 0x7f5100078d50 2024-03-29T07:56:20.909 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:19.560+0000 7f50fd600700 1 -- 172.21.15.37:0/2533726575 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f50f0010380 con 0x7f5100078d50 2024-03-29T07:56:20.909 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:19.560+0000 7f5105a00700 1 -- 172.21.15.37:0/2533726575 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5100078d50 msgr2=0x7f5100077260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T07:56:20.909 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:19.560+0000 7f5105a00700 1 --2- 172.21.15.37:0/2533726575 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5100078d50 0x7f5100077260 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f50f00098e0 tx=0x7f50f0009250 comp rx=0 tx=0).stop 2024-03-29T07:56:20.909 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:19.561+0000 7f5105a00700 1 -- 172.21.15.37:0/2533726575 shutdown_connections 2024-03-29T07:56:20.909 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:19.561+0000 7f5105a00700 1 --2- 172.21.15.37:0/2533726575 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5100078d50 0x7f5100077260 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T07:56:20.909 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:19.561+0000 7f5105a00700 1 -- 172.21.15.37:0/2533726575 >> 172.21.15.37:0/2533726575 conn(0x7f51000fdd00 msgr2=0x7f5100100110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T07:56:20.909 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:19.561+0000 7f5105a00700 1 -- 172.21.15.37:0/2533726575 shutdown_connections 2024-03-29T07:56:20.909 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:19.561+0000 7f5105a00700 1 -- 172.21.15.37:0/2533726575 wait complete. 2024-03-29T07:56:20.909 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:19.562+0000 7f5105a00700 1 Processor -- start 2024-03-29T07:56:20.910 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:19.562+0000 7f5105a00700 1 -- start start 2024-03-29T07:56:20.910 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:19.562+0000 7f5105a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5100078d50 0x7f51001135a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T07:56:20.910 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:19.562+0000 7f5105a00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5100113ae0 con 0x7f5100078d50 2024-03-29T07:56:20.910 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:19.562+0000 7f50fea00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5100078d50 0x7f51001135a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T07:56:20.910 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:19.563+0000 7f50fea00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5100078d50 0x7f51001135a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:33422/0 (socket says 172.21.15.37:33422) 2024-03-29T07:56:20.910 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:19.563+0000 7f50fea00700 1 -- 172.21.15.37:0/3844375952 learned_addr learned my addr 172.21.15.37:0/3844375952 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T07:56:20.910 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:19.563+0000 7f50fea00700 1 -- 172.21.15.37:0/3844375952 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50f00094f0 con 0x7f5100078d50 2024-03-29T07:56:20.910 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:19.563+0000 7f50fea00700 1 --2- 172.21.15.37:0/3844375952 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5100078d50 0x7f51001135a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f50f0006d40 tx=0x7f50f0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T07:56:20.910 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:19.563+0000 7f50f6a00700 1 -- 172.21.15.37:0/3844375952 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f50f0010960 con 0x7f5100078d50 2024-03-29T07:56:20.910 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:19.563+0000 7f50f6a00700 1 -- 172.21.15.37:0/3844375952 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f50f001a430 con 0x7f5100078d50 2024-03-29T07:56:20.910 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:19.563+0000 7f5105a00700 1 -- 172.21.15.37:0/3844375952 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5100113ce0 con 0x7f5100078d50 2024-03-29T07:56:20.910 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:19.563+0000 7f50f6a00700 1 -- 172.21.15.37:0/3844375952 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f50f00245b0 con 0x7f5100078d50 2024-03-29T07:56:20.911 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:19.563+0000 7f5105a00700 1 -- 172.21.15.37:0/3844375952 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5100114850 con 0x7f5100078d50 2024-03-29T07:56:20.911 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:19.565+0000 7f5105a00700 1 -- 172.21.15.37:0/3844375952 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f510005f5f0 con 0x7f5100078d50 2024-03-29T07:56:20.911 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:19.565+0000 7f50f6a00700 1 -- 172.21.15.37:0/3844375952 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7f50f002d3f0 con 0x7f5100078d50 2024-03-29T07:56:20.911 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:19.565+0000 7f50f6a00700 1 --2- 172.21.15.37:0/3844375952 >> [v2:172.21.15.37:6800/2746946658,v1:172.21.15.37:6801/2746946658] conn(0x7f50e40367e0 0x7f50e4038c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T07:56:20.911 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:19.565+0000 7f50f6a00700 1 -- 172.21.15.37:0/3844375952 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f50f004a290 con 0x7f5100078d50 2024-03-29T07:56:20.911 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:19.565+0000 7f50fe000700 1 --2- 172.21.15.37:0/3844375952 >> [v2:172.21.15.37:6800/2746946658,v1:172.21.15.37:6801/2746946658] conn(0x7f50e40367e0 0x7f50e4038c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T07:56:20.911 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:19.566+0000 7f50fe000700 1 --2- 172.21.15.37:0/3844375952 >> [v2:172.21.15.37:6800/2746946658,v1:172.21.15.37:6801/2746946658] conn(0x7f50e40367e0 0x7f50e4038c90 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f50ec009990 tx=0x7f50ec006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T07:56:20.911 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:19.570+0000 7f50f6a00700 1 -- 172.21.15.37:0/3844375952 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f50f002b2f0 con 0x7f5100078d50 2024-03-29T07:56:20.911 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:19.814+0000 7f5105a00700 1 -- 172.21.15.37:0/3844375952 --> [v2:172.21.15.37:6800/2746946658,v1:172.21.15.37:6801/2746946658] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "grafana", "target": ["mon-mgr", ""]}) v1 -- 0x7f51000fef50 con 0x7f50e40367e0 2024-03-29T07:56:20.911 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:19.819+0000 7f50f6a00700 1 -- 172.21.15.37:0/3844375952 <== mgr.14118 v2:172.21.15.37:6800/2746946658 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+28 (secure 0 0 0) 0x7f51000fef50 con 0x7f50e40367e0 2024-03-29T07:56:20.911 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:19.821+0000 7f5105a00700 1 -- 172.21.15.37:0/3844375952 >> [v2:172.21.15.37:6800/2746946658,v1:172.21.15.37:6801/2746946658] conn(0x7f50e40367e0 msgr2=0x7f50e4038c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T07:56:20.911 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:19.821+0000 7f5105a00700 1 --2- 172.21.15.37:0/3844375952 >> [v2:172.21.15.37:6800/2746946658,v1:172.21.15.37:6801/2746946658] conn(0x7f50e40367e0 0x7f50e4038c90 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f50ec009990 tx=0x7f50ec006e40 comp rx=0 tx=0).stop 2024-03-29T07:56:20.911 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:19.821+0000 7f5105a00700 1 -- 172.21.15.37:0/3844375952 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5100078d50 msgr2=0x7f51001135a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T07:56:20.911 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:19.821+0000 7f5105a00700 1 --2- 172.21.15.37:0/3844375952 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5100078d50 0x7f51001135a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f50f0006d40 tx=0x7f50f0009250 comp rx=0 tx=0).stop 2024-03-29T07:56:20.912 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:19.821+0000 7f5105a00700 1 -- 172.21.15.37:0/3844375952 shutdown_connections 2024-03-29T07:56:20.912 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:19.821+0000 7f5105a00700 1 --2- 172.21.15.37:0/3844375952 >> [v2:172.21.15.37:6800/2746946658,v1:172.21.15.37:6801/2746946658] conn(0x7f50e40367e0 0x7f50e4038c90 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T07:56:20.912 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:19.821+0000 7f5105a00700 1 --2- 172.21.15.37:0/3844375952 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5100078d50 0x7f51001135a0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T07:56:20.912 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:19.821+0000 7f5105a00700 1 -- 172.21.15.37:0/3844375952 >> 172.21.15.37:0/3844375952 conn(0x7f51000fdd00 msgr2=0x7f51000fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T07:56:20.912 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:19.821+0000 7f5105a00700 1 -- 172.21.15.37:0/3844375952 shutdown_connections 2024-03-29T07:56:20.912 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:19.821+0000 7f5105a00700 1 -- 172.21.15.37:0/3844375952 wait complete. 2024-03-29T07:56:20.912 INFO:teuthology.orchestra.run.smithi037.stdout:Deploying node-exporter service with default placement... 2024-03-29T07:56:21.176 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:56:20 smithi037 ceph-mon[35162]: from='client.14148 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T07:56:21.176 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:56:20 smithi037 ceph-mon[35162]: Saving service grafana spec with placement count:1 2024-03-29T07:56:21.176 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:56:20 smithi037 ceph-mon[35162]: from='mgr.14118 172.21.15.37:0/2997715107' entity='mgr.smithi037.wjikii' 2024-03-29T07:56:21.176 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:56:20 smithi037 ceph-mon[35162]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-29T07:56:22.578 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout Scheduled node-exporter update... 2024-03-29T07:56:22.578 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:21.693+0000 7fbfd7600700 1 Processor -- start 2024-03-29T07:56:22.578 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:21.693+0000 7fbfd7600700 1 -- start start 2024-03-29T07:56:22.578 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:21.694+0000 7fbfd7600700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbfd0078d50 0x7fbfd0077260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T07:56:22.578 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:21.694+0000 7fbfd7600700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfd0079160 con 0x7fbfd0078d50 2024-03-29T07:56:22.578 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:21.694+0000 7fbfd4c00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbfd0078d50 0x7fbfd0077260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T07:56:22.578 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:21.694+0000 7fbfd4c00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbfd0078d50 0x7fbfd0077260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39808/0 (socket says 172.21.15.37:39808) 2024-03-29T07:56:22.579 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:21.694+0000 7fbfd4c00700 1 -- 172.21.15.37:0/3591545297 learned_addr learned my addr 172.21.15.37:0/3591545297 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T07:56:22.579 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:21.694+0000 7fbfd4c00700 1 -- 172.21.15.37:0/3591545297 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfd00777a0 con 0x7fbfd0078d50 2024-03-29T07:56:22.579 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:21.695+0000 7fbfd4c00700 1 --2- 172.21.15.37:0/3591545297 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbfd0078d50 0x7fbfd0077260 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fbfc40098e0 tx=0x7fbfc4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc969e0316684626 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T07:56:22.579 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:21.695+0000 7fbfcf400700 1 -- 172.21.15.37:0/3591545297 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbfc400b860 con 0x7fbfd0078d50 2024-03-29T07:56:22.579 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:21.695+0000 7fbfcf400700 1 -- 172.21.15.37:0/3591545297 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fbfc400be70 con 0x7fbfd0078d50 2024-03-29T07:56:22.579 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:21.695+0000 7fbfcf400700 1 -- 172.21.15.37:0/3591545297 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbfc4010380 con 0x7fbfd0078d50 2024-03-29T07:56:22.579 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:21.695+0000 7fbfd7600700 1 -- 172.21.15.37:0/3591545297 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbfd0078d50 msgr2=0x7fbfd0077260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T07:56:22.579 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:21.695+0000 7fbfd7600700 1 --2- 172.21.15.37:0/3591545297 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbfd0078d50 0x7fbfd0077260 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fbfc40098e0 tx=0x7fbfc4009250 comp rx=0 tx=0).stop 2024-03-29T07:56:22.580 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:21.696+0000 7fbfd7600700 1 -- 172.21.15.37:0/3591545297 shutdown_connections 2024-03-29T07:56:22.580 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:21.696+0000 7fbfd7600700 1 --2- 172.21.15.37:0/3591545297 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbfd0078d50 0x7fbfd0077260 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T07:56:22.580 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:21.696+0000 7fbfd7600700 1 -- 172.21.15.37:0/3591545297 >> 172.21.15.37:0/3591545297 conn(0x7fbfd00fdd00 msgr2=0x7fbfd0100110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T07:56:22.580 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:21.696+0000 7fbfd7600700 1 -- 172.21.15.37:0/3591545297 shutdown_connections 2024-03-29T07:56:22.580 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:21.696+0000 7fbfd7600700 1 -- 172.21.15.37:0/3591545297 wait complete. 2024-03-29T07:56:22.580 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:21.697+0000 7fbfd7600700 1 Processor -- start 2024-03-29T07:56:22.580 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:21.697+0000 7fbfd7600700 1 -- start start 2024-03-29T07:56:22.580 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:21.697+0000 7fbfd7600700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbfd0078d50 0x7fbfd01135a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T07:56:22.580 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:21.697+0000 7fbfd7600700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfd0113ae0 con 0x7fbfd0078d50 2024-03-29T07:56:22.580 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:21.697+0000 7fbfd4c00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbfd0078d50 0x7fbfd01135a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T07:56:22.580 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:21.697+0000 7fbfd4c00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbfd0078d50 0x7fbfd01135a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39814/0 (socket says 172.21.15.37:39814) 2024-03-29T07:56:22.580 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:21.697+0000 7fbfd4c00700 1 -- 172.21.15.37:0/4259880578 learned_addr learned my addr 172.21.15.37:0/4259880578 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T07:56:22.580 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:21.698+0000 7fbfd4c00700 1 -- 172.21.15.37:0/4259880578 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfc40094f0 con 0x7fbfd0078d50 2024-03-29T07:56:22.580 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:21.698+0000 7fbfd4c00700 1 --2- 172.21.15.37:0/4259880578 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbfd0078d50 0x7fbfd01135a0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fbfc4006d40 tx=0x7fbfc4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T07:56:22.580 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:21.698+0000 7fbfccc00700 1 -- 172.21.15.37:0/4259880578 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbfc4010960 con 0x7fbfd0078d50 2024-03-29T07:56:22.581 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:21.698+0000 7fbfd7600700 1 -- 172.21.15.37:0/4259880578 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfd0113ce0 con 0x7fbfd0078d50 2024-03-29T07:56:22.581 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:21.698+0000 7fbfccc00700 1 -- 172.21.15.37:0/4259880578 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fbfc401a430 con 0x7fbfd0078d50 2024-03-29T07:56:22.581 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:21.698+0000 7fbfccc00700 1 -- 172.21.15.37:0/4259880578 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbfc40245b0 con 0x7fbfd0078d50 2024-03-29T07:56:22.581 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:21.698+0000 7fbfd7600700 1 -- 172.21.15.37:0/4259880578 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfd0114850 con 0x7fbfd0078d50 2024-03-29T07:56:22.581 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:21.699+0000 7fbfccc00700 1 -- 172.21.15.37:0/4259880578 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7fbfc401a9f0 con 0x7fbfd0078d50 2024-03-29T07:56:22.581 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:21.699+0000 7fbfd7600700 1 -- 172.21.15.37:0/4259880578 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfd005f5f0 con 0x7fbfd0078d50 2024-03-29T07:56:22.581 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:21.700+0000 7fbfccc00700 1 --2- 172.21.15.37:0/4259880578 >> [v2:172.21.15.37:6800/2746946658,v1:172.21.15.37:6801/2746946658] conn(0x7fbfb80367e0 0x7fbfb8038c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T07:56:22.581 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:21.700+0000 7fbfccc00700 1 -- 172.21.15.37:0/4259880578 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7fbfc404a310 con 0x7fbfd0078d50 2024-03-29T07:56:22.582 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:21.700+0000 7fbfcfe00700 1 --2- 172.21.15.37:0/4259880578 >> [v2:172.21.15.37:6800/2746946658,v1:172.21.15.37:6801/2746946658] conn(0x7fbfb80367e0 0x7fbfb8038c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T07:56:22.582 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:21.700+0000 7fbfcfe00700 1 --2- 172.21.15.37:0/4259880578 >> [v2:172.21.15.37:6800/2746946658,v1:172.21.15.37:6801/2746946658] conn(0x7fbfb80367e0 0x7fbfb8038c90 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fbfbc009990 tx=0x7fbfbc006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T07:56:22.582 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:21.705+0000 7fbfccc00700 1 -- 172.21.15.37:0/4259880578 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fbfc402b8c0 con 0x7fbfd0078d50 2024-03-29T07:56:22.582 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:21.923+0000 7fbfd7600700 1 -- 172.21.15.37:0/4259880578 --> [v2:172.21.15.37:6800/2746946658,v1:172.21.15.37:6801/2746946658] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "node-exporter", "target": ["mon-mgr", ""]}) v1 -- 0x7fbfd00fef50 con 0x7fbfb80367e0 2024-03-29T07:56:22.582 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:21.927+0000 7fbfccc00700 1 -- 172.21.15.37:0/4259880578 <== mgr.14118 v2:172.21.15.37:6800/2746946658 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+34 (secure 0 0 0) 0x7fbfd00fef50 con 0x7fbfb80367e0 2024-03-29T07:56:22.583 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:21.930+0000 7fbfd7600700 1 -- 172.21.15.37:0/4259880578 >> [v2:172.21.15.37:6800/2746946658,v1:172.21.15.37:6801/2746946658] conn(0x7fbfb80367e0 msgr2=0x7fbfb8038c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T07:56:22.583 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:21.930+0000 7fbfd7600700 1 --2- 172.21.15.37:0/4259880578 >> [v2:172.21.15.37:6800/2746946658,v1:172.21.15.37:6801/2746946658] conn(0x7fbfb80367e0 0x7fbfb8038c90 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fbfbc009990 tx=0x7fbfbc006e40 comp rx=0 tx=0).stop 2024-03-29T07:56:22.583 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:21.930+0000 7fbfd7600700 1 -- 172.21.15.37:0/4259880578 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbfd0078d50 msgr2=0x7fbfd01135a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T07:56:22.583 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:21.930+0000 7fbfd7600700 1 --2- 172.21.15.37:0/4259880578 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbfd0078d50 0x7fbfd01135a0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fbfc4006d40 tx=0x7fbfc4009250 comp rx=0 tx=0).stop 2024-03-29T07:56:22.583 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:21.930+0000 7fbfd7600700 1 -- 172.21.15.37:0/4259880578 shutdown_connections 2024-03-29T07:56:22.583 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:21.930+0000 7fbfd7600700 1 --2- 172.21.15.37:0/4259880578 >> [v2:172.21.15.37:6800/2746946658,v1:172.21.15.37:6801/2746946658] conn(0x7fbfb80367e0 0x7fbfb8038c90 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T07:56:22.583 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:21.930+0000 7fbfd7600700 1 --2- 172.21.15.37:0/4259880578 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbfd0078d50 0x7fbfd01135a0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T07:56:22.583 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:21.930+0000 7fbfd7600700 1 -- 172.21.15.37:0/4259880578 >> 172.21.15.37:0/4259880578 conn(0x7fbfd00fdd00 msgr2=0x7fbfd00fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T07:56:22.583 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:21.930+0000 7fbfd7600700 1 -- 172.21.15.37:0/4259880578 shutdown_connections 2024-03-29T07:56:22.583 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:21.930+0000 7fbfd7600700 1 -- 172.21.15.37:0/4259880578 wait complete. 2024-03-29T07:56:22.583 INFO:teuthology.orchestra.run.smithi037.stdout:Deploying alertmanager service with default placement... 2024-03-29T07:56:23.176 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:56:22 smithi037 ceph-mon[35162]: from='client.14150 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T07:56:23.176 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:56:22 smithi037 ceph-mon[35162]: Saving service node-exporter spec with placement * 2024-03-29T07:56:23.176 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:56:22 smithi037 ceph-mon[35162]: from='mgr.14118 172.21.15.37:0/2997715107' entity='mgr.smithi037.wjikii' 2024-03-29T07:56:23.176 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:56:22 smithi037 ceph-mon[35162]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-29T07:56:23.882 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout Scheduled alertmanager update... 2024-03-29T07:56:23.882 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:23.166+0000 7fa61f400700 1 Processor -- start 2024-03-29T07:56:23.882 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:23.166+0000 7fa61f400700 1 -- start start 2024-03-29T07:56:23.882 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:23.166+0000 7fa61f400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa618078d30 0x7fa618077240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T07:56:23.882 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:23.167+0000 7fa61f400700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa618079140 con 0x7fa618078d30 2024-03-29T07:56:23.882 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:23.167+0000 7fa61ca00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa618078d30 0x7fa618077240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T07:56:23.882 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:23.167+0000 7fa61ca00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa618078d30 0x7fa618077240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39830/0 (socket says 172.21.15.37:39830) 2024-03-29T07:56:23.883 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:23.167+0000 7fa61ca00700 1 -- 172.21.15.37:0/1691340047 learned_addr learned my addr 172.21.15.37:0/1691340047 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T07:56:23.883 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:23.167+0000 7fa61ca00700 1 -- 172.21.15.37:0/1691340047 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa618077780 con 0x7fa618078d30 2024-03-29T07:56:23.883 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:23.167+0000 7fa61ca00700 1 --2- 172.21.15.37:0/1691340047 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa618078d30 0x7fa618077240 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fa60c0098e0 tx=0x7fa60c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86b6c32076b05e9f server_cookie=0 in_seq=0 out_seq=0 2024-03-29T07:56:23.883 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:23.168+0000 7fa617400700 1 -- 172.21.15.37:0/1691340047 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa60c00b860 con 0x7fa618078d30 2024-03-29T07:56:23.883 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:23.168+0000 7fa617400700 1 -- 172.21.15.37:0/1691340047 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fa60c00be70 con 0x7fa618078d30 2024-03-29T07:56:23.883 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:23.168+0000 7fa617400700 1 -- 172.21.15.37:0/1691340047 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa60c010380 con 0x7fa618078d30 2024-03-29T07:56:23.883 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:23.168+0000 7fa61f400700 1 -- 172.21.15.37:0/1691340047 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa618078d30 msgr2=0x7fa618077240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T07:56:23.883 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:23.168+0000 7fa61f400700 1 --2- 172.21.15.37:0/1691340047 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa618078d30 0x7fa618077240 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fa60c0098e0 tx=0x7fa60c009250 comp rx=0 tx=0).stop 2024-03-29T07:56:23.883 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:23.168+0000 7fa61f400700 1 -- 172.21.15.37:0/1691340047 shutdown_connections 2024-03-29T07:56:23.883 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:23.168+0000 7fa61f400700 1 --2- 172.21.15.37:0/1691340047 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa618078d30 0x7fa618077240 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T07:56:23.883 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:23.168+0000 7fa61f400700 1 -- 172.21.15.37:0/1691340047 >> 172.21.15.37:0/1691340047 conn(0x7fa6180fdd00 msgr2=0x7fa618100110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T07:56:23.883 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:23.168+0000 7fa61f400700 1 -- 172.21.15.37:0/1691340047 shutdown_connections 2024-03-29T07:56:23.883 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:23.169+0000 7fa61f400700 1 -- 172.21.15.37:0/1691340047 wait complete. 2024-03-29T07:56:23.883 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:23.169+0000 7fa61f400700 1 Processor -- start 2024-03-29T07:56:23.884 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:23.169+0000 7fa61f400700 1 -- start start 2024-03-29T07:56:23.884 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:23.170+0000 7fa61f400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa618078d30 0x7fa6181135b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T07:56:23.884 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:23.170+0000 7fa61f400700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa618113af0 con 0x7fa618078d30 2024-03-29T07:56:23.884 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:23.170+0000 7fa61ca00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa618078d30 0x7fa6181135b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T07:56:23.884 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:23.170+0000 7fa61ca00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa618078d30 0x7fa6181135b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39838/0 (socket says 172.21.15.37:39838) 2024-03-29T07:56:23.884 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:23.170+0000 7fa61ca00700 1 -- 172.21.15.37:0/1762478084 learned_addr learned my addr 172.21.15.37:0/1762478084 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T07:56:23.884 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:23.170+0000 7fa61ca00700 1 -- 172.21.15.37:0/1762478084 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa60c0094f0 con 0x7fa618078d30 2024-03-29T07:56:23.884 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:23.171+0000 7fa61ca00700 1 --2- 172.21.15.37:0/1762478084 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa618078d30 0x7fa6181135b0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fa60c006d40 tx=0x7fa60c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T07:56:23.884 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:23.171+0000 7fa614c00700 1 -- 172.21.15.37:0/1762478084 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa60c010960 con 0x7fa618078d30 2024-03-29T07:56:23.884 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:23.171+0000 7fa614c00700 1 -- 172.21.15.37:0/1762478084 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fa60c01a430 con 0x7fa618078d30 2024-03-29T07:56:23.884 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:23.171+0000 7fa61f400700 1 -- 172.21.15.37:0/1762478084 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa618113cf0 con 0x7fa618078d30 2024-03-29T07:56:23.884 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:23.171+0000 7fa614c00700 1 -- 172.21.15.37:0/1762478084 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa60c0245b0 con 0x7fa618078d30 2024-03-29T07:56:23.884 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:23.171+0000 7fa61f400700 1 -- 172.21.15.37:0/1762478084 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa618114860 con 0x7fa618078d30 2024-03-29T07:56:23.884 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:23.172+0000 7fa614c00700 1 -- 172.21.15.37:0/1762478084 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7fa60c01a9f0 con 0x7fa618078d30 2024-03-29T07:56:23.885 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:23.172+0000 7fa61f400700 1 -- 172.21.15.37:0/1762478084 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa61805f5f0 con 0x7fa618078d30 2024-03-29T07:56:23.885 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:23.172+0000 7fa614c00700 1 --2- 172.21.15.37:0/1762478084 >> [v2:172.21.15.37:6800/2746946658,v1:172.21.15.37:6801/2746946658] conn(0x7fa6000367e0 0x7fa600038c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T07:56:23.885 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:23.173+0000 7fa614c00700 1 -- 172.21.15.37:0/1762478084 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7fa60c04a250 con 0x7fa618078d30 2024-03-29T07:56:23.885 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:23.173+0000 7fa617e00700 1 --2- 172.21.15.37:0/1762478084 >> [v2:172.21.15.37:6800/2746946658,v1:172.21.15.37:6801/2746946658] conn(0x7fa6000367e0 0x7fa600038c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T07:56:23.885 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:23.173+0000 7fa617e00700 1 --2- 172.21.15.37:0/1762478084 >> [v2:172.21.15.37:6800/2746946658,v1:172.21.15.37:6801/2746946658] conn(0x7fa6000367e0 0x7fa600038c90 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fa604009990 tx=0x7fa604006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T07:56:23.885 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:23.177+0000 7fa614c00700 1 -- 172.21.15.37:0/1762478084 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fa60c02b8c0 con 0x7fa618078d30 2024-03-29T07:56:23.885 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:23.394+0000 7fa61f400700 1 -- 172.21.15.37:0/1762478084 --> [v2:172.21.15.37:6800/2746946658,v1:172.21.15.37:6801/2746946658] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "alertmanager", "target": ["mon-mgr", ""]}) v1 -- 0x7fa6180fef50 con 0x7fa6000367e0 2024-03-29T07:56:23.885 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:23.401+0000 7fa614c00700 1 -- 172.21.15.37:0/1762478084 <== mgr.14118 v2:172.21.15.37:6800/2746946658 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+33 (secure 0 0 0) 0x7fa6180fef50 con 0x7fa6000367e0 2024-03-29T07:56:23.885 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:23.403+0000 7fa61f400700 1 -- 172.21.15.37:0/1762478084 >> [v2:172.21.15.37:6800/2746946658,v1:172.21.15.37:6801/2746946658] conn(0x7fa6000367e0 msgr2=0x7fa600038c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T07:56:23.885 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:23.403+0000 7fa61f400700 1 --2- 172.21.15.37:0/1762478084 >> [v2:172.21.15.37:6800/2746946658,v1:172.21.15.37:6801/2746946658] conn(0x7fa6000367e0 0x7fa600038c90 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fa604009990 tx=0x7fa604006e40 comp rx=0 tx=0).stop 2024-03-29T07:56:23.885 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:23.403+0000 7fa61f400700 1 -- 172.21.15.37:0/1762478084 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa618078d30 msgr2=0x7fa6181135b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T07:56:23.885 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:23.403+0000 7fa61f400700 1 --2- 172.21.15.37:0/1762478084 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa618078d30 0x7fa6181135b0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fa60c006d40 tx=0x7fa60c009250 comp rx=0 tx=0).stop 2024-03-29T07:56:23.885 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:23.403+0000 7fa61f400700 1 -- 172.21.15.37:0/1762478084 shutdown_connections 2024-03-29T07:56:23.885 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:23.403+0000 7fa61f400700 1 --2- 172.21.15.37:0/1762478084 >> [v2:172.21.15.37:6800/2746946658,v1:172.21.15.37:6801/2746946658] conn(0x7fa6000367e0 0x7fa600038c90 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T07:56:23.886 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:23.403+0000 7fa61f400700 1 --2- 172.21.15.37:0/1762478084 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa618078d30 0x7fa6181135b0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T07:56:23.886 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:23.403+0000 7fa61f400700 1 -- 172.21.15.37:0/1762478084 >> 172.21.15.37:0/1762478084 conn(0x7fa6180fdd00 msgr2=0x7fa6180fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T07:56:23.886 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:23.404+0000 7fa61f400700 1 -- 172.21.15.37:0/1762478084 shutdown_connections 2024-03-29T07:56:23.886 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:23.404+0000 7fa61f400700 1 -- 172.21.15.37:0/1762478084 wait complete. 2024-03-29T07:56:24.676 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:56:24 smithi037 ceph-mon[35162]: from='client.14152 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T07:56:24.676 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:56:24 smithi037 ceph-mon[35162]: Saving service alertmanager spec with placement count:1 2024-03-29T07:56:24.676 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:56:24 smithi037 ceph-mon[35162]: from='mgr.14118 172.21.15.37:0/2997715107' entity='mgr.smithi037.wjikii' 2024-03-29T07:56:25.241 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:24.433+0000 7f1925a00700 1 Processor -- start 2024-03-29T07:56:25.241 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:24.434+0000 7f1925a00700 1 -- start start 2024-03-29T07:56:25.241 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:24.434+0000 7f1925a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1920102580 0x7f1920102990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T07:56:25.241 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:24.434+0000 7f1925a00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1920102ed0 con 0x7f1920102580 2024-03-29T07:56:25.241 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:24.434+0000 7f191ea00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1920102580 0x7f1920102990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T07:56:25.242 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:24.434+0000 7f191ea00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1920102580 0x7f1920102990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39852/0 (socket says 172.21.15.37:39852) 2024-03-29T07:56:25.242 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:24.434+0000 7f191ea00700 1 -- 172.21.15.37:0/2643613860 learned_addr learned my addr 172.21.15.37:0/2643613860 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T07:56:25.242 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:24.435+0000 7f191ea00700 1 -- 172.21.15.37:0/2643613860 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1920103010 con 0x7f1920102580 2024-03-29T07:56:25.242 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:24.435+0000 7f191ea00700 1 --2- 172.21.15.37:0/2643613860 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1920102580 0x7f1920102990 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f19100098e0 tx=0x7f1910009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=298f1f91c087427c server_cookie=0 in_seq=0 out_seq=0 2024-03-29T07:56:25.242 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:24.435+0000 7f191d600700 1 -- 172.21.15.37:0/2643613860 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f191000b860 con 0x7f1920102580 2024-03-29T07:56:25.242 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:24.435+0000 7f191d600700 1 -- 172.21.15.37:0/2643613860 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f191000be70 con 0x7f1920102580 2024-03-29T07:56:25.242 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:24.435+0000 7f191d600700 1 -- 172.21.15.37:0/2643613860 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1910010380 con 0x7f1920102580 2024-03-29T07:56:25.242 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:24.436+0000 7f1925a00700 1 -- 172.21.15.37:0/2643613860 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1920102580 msgr2=0x7f1920102990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T07:56:25.242 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:24.436+0000 7f1925a00700 1 --2- 172.21.15.37:0/2643613860 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1920102580 0x7f1920102990 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f19100098e0 tx=0x7f1910009250 comp rx=0 tx=0).stop 2024-03-29T07:56:25.242 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:24.436+0000 7f1925a00700 1 -- 172.21.15.37:0/2643613860 shutdown_connections 2024-03-29T07:56:25.242 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:24.436+0000 7f1925a00700 1 --2- 172.21.15.37:0/2643613860 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1920102580 0x7f1920102990 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T07:56:25.242 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:24.436+0000 7f1925a00700 1 -- 172.21.15.37:0/2643613860 >> 172.21.15.37:0/2643613860 conn(0x7f19200fdd00 msgr2=0x7f1920100110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T07:56:25.243 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:24.436+0000 7f1925a00700 1 -- 172.21.15.37:0/2643613860 shutdown_connections 2024-03-29T07:56:25.243 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:24.436+0000 7f1925a00700 1 -- 172.21.15.37:0/2643613860 wait complete. 2024-03-29T07:56:25.243 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:24.437+0000 7f1925a00700 1 Processor -- start 2024-03-29T07:56:25.243 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:24.437+0000 7f1925a00700 1 -- start start 2024-03-29T07:56:25.243 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:24.438+0000 7f1925a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1920102580 0x7f1920078c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T07:56:25.243 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:24.438+0000 7f1925a00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1920079190 con 0x7f1920102580 2024-03-29T07:56:25.243 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:24.438+0000 7f191ea00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1920102580 0x7f1920078c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T07:56:25.243 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:24.438+0000 7f191ea00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1920102580 0x7f1920078c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39854/0 (socket says 172.21.15.37:39854) 2024-03-29T07:56:25.243 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:24.438+0000 7f191ea00700 1 -- 172.21.15.37:0/3259836715 learned_addr learned my addr 172.21.15.37:0/3259836715 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T07:56:25.243 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:24.438+0000 7f191ea00700 1 -- 172.21.15.37:0/3259836715 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19100094f0 con 0x7f1920102580 2024-03-29T07:56:25.243 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:24.438+0000 7f191ea00700 1 --2- 172.21.15.37:0/3259836715 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1920102580 0x7f1920078c50 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f191000c010 tx=0x7f1910009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T07:56:25.243 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:24.439+0000 7f1916a00700 1 -- 172.21.15.37:0/3259836715 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1910010960 con 0x7f1920102580 2024-03-29T07:56:25.243 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:24.439+0000 7f1916a00700 1 -- 172.21.15.37:0/3259836715 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f191001a430 con 0x7f1920102580 2024-03-29T07:56:25.243 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:24.439+0000 7f1916a00700 1 -- 172.21.15.37:0/3259836715 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1910019580 con 0x7f1920102580 2024-03-29T07:56:25.243 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:24.439+0000 7f1925a00700 1 -- 172.21.15.37:0/3259836715 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1920077200 con 0x7f1920102580 2024-03-29T07:56:25.244 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:24.439+0000 7f1925a00700 1 -- 172.21.15.37:0/3259836715 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19200776f0 con 0x7f1920102580 2024-03-29T07:56:25.244 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:24.440+0000 7f1925a00700 1 -- 172.21.15.37:0/3259836715 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f192004bed0 con 0x7f1920102580 2024-03-29T07:56:25.244 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:24.440+0000 7f1916a00700 1 -- 172.21.15.37:0/3259836715 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7f191001a980 con 0x7f1920102580 2024-03-29T07:56:25.244 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:24.440+0000 7f1916a00700 1 --2- 172.21.15.37:0/3259836715 >> [v2:172.21.15.37:6800/2746946658,v1:172.21.15.37:6801/2746946658] conn(0x7f19040367e0 0x7f1904038c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T07:56:25.244 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:24.440+0000 7f1916a00700 1 -- 172.21.15.37:0/3259836715 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f191004a620 con 0x7f1920102580 2024-03-29T07:56:25.244 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:24.441+0000 7f191e000700 1 --2- 172.21.15.37:0/3259836715 >> [v2:172.21.15.37:6800/2746946658,v1:172.21.15.37:6801/2746946658] conn(0x7f19040367e0 0x7f1904038c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T07:56:25.244 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:24.441+0000 7f191e000700 1 --2- 172.21.15.37:0/3259836715 >> [v2:172.21.15.37:6800/2746946658,v1:172.21.15.37:6801/2746946658] conn(0x7f19040367e0 0x7f1904038c90 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f190c009990 tx=0x7f190c006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T07:56:25.244 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:24.445+0000 7f1916a00700 1 -- 172.21.15.37:0/3259836715 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f191002bb30 con 0x7f1920102580 2024-03-29T07:56:25.244 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:24.650+0000 7f1925a00700 1 -- 172.21.15.37:0/3259836715 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) v1 -- 0x7f1920104360 con 0x7f1920102580 2024-03-29T07:56:25.244 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:24.656+0000 7f1916a00700 1 -- 172.21.15.37:0/3259836715 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) v1 ==== 142+0+0 (secure 0 0 0) 0x7f191002b300 con 0x7f1920102580 2024-03-29T07:56:25.244 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:24.658+0000 7f1925a00700 1 -- 172.21.15.37:0/3259836715 >> [v2:172.21.15.37:6800/2746946658,v1:172.21.15.37:6801/2746946658] conn(0x7f19040367e0 msgr2=0x7f1904038c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T07:56:25.244 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:24.658+0000 7f1925a00700 1 --2- 172.21.15.37:0/3259836715 >> [v2:172.21.15.37:6800/2746946658,v1:172.21.15.37:6801/2746946658] conn(0x7f19040367e0 0x7f1904038c90 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f190c009990 tx=0x7f190c006e40 comp rx=0 tx=0).stop 2024-03-29T07:56:25.244 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:24.658+0000 7f1925a00700 1 -- 172.21.15.37:0/3259836715 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1920102580 msgr2=0x7f1920078c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T07:56:25.244 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:24.658+0000 7f1925a00700 1 --2- 172.21.15.37:0/3259836715 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1920102580 0x7f1920078c50 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f191000c010 tx=0x7f1910009250 comp rx=0 tx=0).stop 2024-03-29T07:56:25.245 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:24.658+0000 7f1925a00700 1 -- 172.21.15.37:0/3259836715 shutdown_connections 2024-03-29T07:56:25.245 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:24.658+0000 7f1925a00700 1 --2- 172.21.15.37:0/3259836715 >> [v2:172.21.15.37:6800/2746946658,v1:172.21.15.37:6801/2746946658] conn(0x7f19040367e0 0x7f1904038c90 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T07:56:25.245 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:24.658+0000 7f1925a00700 1 --2- 172.21.15.37:0/3259836715 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1920102580 0x7f1920078c50 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T07:56:25.245 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:24.658+0000 7f1925a00700 1 -- 172.21.15.37:0/3259836715 >> 172.21.15.37:0/3259836715 conn(0x7f19200fdd00 msgr2=0x7f19200fe760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T07:56:25.245 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:24.659+0000 7f1925a00700 1 -- 172.21.15.37:0/3259836715 shutdown_connections 2024-03-29T07:56:25.245 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:24.659+0000 7f1925a00700 1 -- 172.21.15.37:0/3259836715 wait complete. 2024-03-29T07:56:25.504 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:56:25 smithi037 ceph-mon[35162]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-29T07:56:25.505 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:56:25 smithi037 ceph-mon[35162]: from='client.? 172.21.15.37:0/3259836715' entity='client.admin' 2024-03-29T07:56:26.730 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:25.772+0000 7f56a8800700 1 Processor -- start 2024-03-29T07:56:26.730 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:25.772+0000 7f56a8800700 1 -- start start 2024-03-29T07:56:26.730 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:25.772+0000 7f56a8800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f56a01030c0 0x7f56a01054a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T07:56:26.730 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:25.772+0000 7f56a8800700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56a0076d20 con 0x7f56a01030c0 2024-03-29T07:56:26.730 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:25.773+0000 7f56a5e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f56a01030c0 0x7f56a01054a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T07:56:26.731 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:25.773+0000 7f56a5e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f56a01030c0 0x7f56a01054a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39866/0 (socket says 172.21.15.37:39866) 2024-03-29T07:56:26.731 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:25.773+0000 7f56a5e00700 1 -- 172.21.15.37:0/1719499337 learned_addr learned my addr 172.21.15.37:0/1719499337 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T07:56:26.731 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:25.773+0000 7f56a5e00700 1 -- 172.21.15.37:0/1719499337 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56a0076e60 con 0x7f56a01030c0 2024-03-29T07:56:26.731 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:25.773+0000 7f56a5e00700 1 --2- 172.21.15.37:0/1719499337 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f56a01030c0 0x7f56a01054a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f56940098e0 tx=0x7f5694009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cec44e903933da server_cookie=0 in_seq=0 out_seq=0 2024-03-29T07:56:26.731 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:25.773+0000 7f56a4a00700 1 -- 172.21.15.37:0/1719499337 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f569400b860 con 0x7f56a01030c0 2024-03-29T07:56:26.731 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:25.774+0000 7f56a4a00700 1 -- 172.21.15.37:0/1719499337 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f569400be70 con 0x7f56a01030c0 2024-03-29T07:56:26.731 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:25.774+0000 7f56a4a00700 1 -- 172.21.15.37:0/1719499337 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5694010380 con 0x7f56a01030c0 2024-03-29T07:56:26.731 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:25.774+0000 7f56a8800700 1 -- 172.21.15.37:0/1719499337 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f56a01030c0 msgr2=0x7f56a01054a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T07:56:26.731 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:25.774+0000 7f56a8800700 1 --2- 172.21.15.37:0/1719499337 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f56a01030c0 0x7f56a01054a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f56940098e0 tx=0x7f5694009250 comp rx=0 tx=0).stop 2024-03-29T07:56:26.731 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:25.774+0000 7f56a8800700 1 -- 172.21.15.37:0/1719499337 shutdown_connections 2024-03-29T07:56:26.731 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:25.774+0000 7f56a8800700 1 --2- 172.21.15.37:0/1719499337 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f56a01030c0 0x7f56a01054a0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T07:56:26.731 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:25.774+0000 7f56a8800700 1 -- 172.21.15.37:0/1719499337 >> 172.21.15.37:0/1719499337 conn(0x7f56a00fcb20 msgr2=0x7f56a00fef70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T07:56:26.731 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:25.774+0000 7f56a8800700 1 -- 172.21.15.37:0/1719499337 shutdown_connections 2024-03-29T07:56:26.731 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:25.774+0000 7f56a8800700 1 -- 172.21.15.37:0/1719499337 wait complete. 2024-03-29T07:56:26.732 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:25.775+0000 7f56a8800700 1 Processor -- start 2024-03-29T07:56:26.732 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:25.775+0000 7f56a8800700 1 -- start start 2024-03-29T07:56:26.732 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:25.776+0000 7f56a8800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f56a01030c0 0x7f56a0102a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T07:56:26.732 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:25.776+0000 7f56a8800700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56a0102f60 con 0x7f56a01030c0 2024-03-29T07:56:26.732 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:25.776+0000 7f56a5e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f56a01030c0 0x7f56a0102a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T07:56:26.732 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:25.776+0000 7f56a5e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f56a01030c0 0x7f56a0102a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39874/0 (socket says 172.21.15.37:39874) 2024-03-29T07:56:26.732 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:25.776+0000 7f56a5e00700 1 -- 172.21.15.37:0/2082106760 learned_addr learned my addr 172.21.15.37:0/2082106760 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T07:56:26.732 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:25.776+0000 7f56a5e00700 1 -- 172.21.15.37:0/2082106760 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56940094f0 con 0x7f56a01030c0 2024-03-29T07:56:26.732 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:25.776+0000 7f56a5e00700 1 --2- 172.21.15.37:0/2082106760 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f56a01030c0 0x7f56a0102a20 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f5694006d40 tx=0x7f5694009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T07:56:26.732 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:25.777+0000 7f569a000700 1 -- 172.21.15.37:0/2082106760 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5694010960 con 0x7f56a01030c0 2024-03-29T07:56:26.732 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:25.777+0000 7f569a000700 1 -- 172.21.15.37:0/2082106760 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f569401a430 con 0x7f56a01030c0 2024-03-29T07:56:26.732 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:25.777+0000 7f56a8800700 1 -- 172.21.15.37:0/2082106760 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56a0100fd0 con 0x7f56a01030c0 2024-03-29T07:56:26.732 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:25.777+0000 7f569a000700 1 -- 172.21.15.37:0/2082106760 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f56940245b0 con 0x7f56a01030c0 2024-03-29T07:56:26.732 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:25.777+0000 7f56a8800700 1 -- 172.21.15.37:0/2082106760 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56a01014c0 con 0x7f56a01030c0 2024-03-29T07:56:26.732 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:25.778+0000 7f569a000700 1 -- 172.21.15.37:0/2082106760 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7f569401a9f0 con 0x7f56a01030c0 2024-03-29T07:56:26.732 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:25.778+0000 7f56a8800700 1 -- 172.21.15.37:0/2082106760 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56a004bed0 con 0x7f56a01030c0 2024-03-29T07:56:26.733 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:25.778+0000 7f569a000700 1 --2- 172.21.15.37:0/2082106760 >> [v2:172.21.15.37:6800/2746946658,v1:172.21.15.37:6801/2746946658] conn(0x7f56880367e0 0x7f5688038c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T07:56:26.733 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:25.778+0000 7f569a000700 1 -- 172.21.15.37:0/2082106760 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f569404a250 con 0x7f56a01030c0 2024-03-29T07:56:26.733 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:25.778+0000 7f56a5400700 1 --2- 172.21.15.37:0/2082106760 >> [v2:172.21.15.37:6800/2746946658,v1:172.21.15.37:6801/2746946658] conn(0x7f56880367e0 0x7f5688038c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T07:56:26.733 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:25.779+0000 7f56a5400700 1 --2- 172.21.15.37:0/2082106760 >> [v2:172.21.15.37:6800/2746946658,v1:172.21.15.37:6801/2746946658] conn(0x7f56880367e0 0x7f5688038c90 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f5690009990 tx=0x7f5690006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T07:56:26.733 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:25.784+0000 7f569a000700 1 -- 172.21.15.37:0/2082106760 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f569402b8c0 con 0x7f56a01030c0 2024-03-29T07:56:26.733 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:25.989+0000 7f56a8800700 1 -- 172.21.15.37:0/2082106760 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) v1 -- 0x7f56a0101940 con 0x7f56a01030c0 2024-03-29T07:56:26.733 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:25.992+0000 7f569a000700 1 -- 172.21.15.37:0/2082106760 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) v1 ==== 130+0+0 (secure 0 0 0) 0x7f569402b2f0 con 0x7f56a01030c0 2024-03-29T07:56:26.733 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:25.994+0000 7f56a8800700 1 -- 172.21.15.37:0/2082106760 >> [v2:172.21.15.37:6800/2746946658,v1:172.21.15.37:6801/2746946658] conn(0x7f56880367e0 msgr2=0x7f5688038c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T07:56:26.733 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:25.994+0000 7f56a8800700 1 --2- 172.21.15.37:0/2082106760 >> [v2:172.21.15.37:6800/2746946658,v1:172.21.15.37:6801/2746946658] conn(0x7f56880367e0 0x7f5688038c90 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f5690009990 tx=0x7f5690006e40 comp rx=0 tx=0).stop 2024-03-29T07:56:26.733 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:25.994+0000 7f56a8800700 1 -- 172.21.15.37:0/2082106760 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f56a01030c0 msgr2=0x7f56a0102a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T07:56:26.733 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:25.994+0000 7f56a8800700 1 --2- 172.21.15.37:0/2082106760 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f56a01030c0 0x7f56a0102a20 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f5694006d40 tx=0x7f5694009250 comp rx=0 tx=0).stop 2024-03-29T07:56:26.733 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:25.994+0000 7f56a8800700 1 -- 172.21.15.37:0/2082106760 shutdown_connections 2024-03-29T07:56:26.733 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:25.994+0000 7f56a8800700 1 --2- 172.21.15.37:0/2082106760 >> [v2:172.21.15.37:6800/2746946658,v1:172.21.15.37:6801/2746946658] conn(0x7f56880367e0 0x7f5688038c90 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T07:56:26.733 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:25.994+0000 7f56a8800700 1 --2- 172.21.15.37:0/2082106760 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f56a01030c0 0x7f56a0102a20 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T07:56:26.734 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:25.994+0000 7f56a8800700 1 -- 172.21.15.37:0/2082106760 >> 172.21.15.37:0/2082106760 conn(0x7f56a00fcb20 msgr2=0x7f56a00fd6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T07:56:26.734 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:25.995+0000 7f56a8800700 1 -- 172.21.15.37:0/2082106760 shutdown_connections 2024-03-29T07:56:26.734 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:25.995+0000 7f56a8800700 1 -- 172.21.15.37:0/2082106760 wait complete. 2024-03-29T07:56:26.734 INFO:teuthology.orchestra.run.smithi037.stdout:Enabling the dashboard module... 2024-03-29T07:56:27.426 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:56:26 smithi037 ceph-mon[35162]: from='client.? 172.21.15.37:0/2082106760' entity='client.admin' 2024-03-29T07:56:27.426 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:56:26 smithi037 ceph-mon[35162]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-29T07:56:28.295 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:56:27 smithi037 ceph-mon[35162]: from='client.? 172.21.15.37:0/4049992534' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-03-29T07:56:28.737 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:27.268+0000 7f801f000700 1 Processor -- start 2024-03-29T07:56:28.738 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:27.268+0000 7f801f000700 1 -- start start 2024-03-29T07:56:28.738 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:27.268+0000 7f801f000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8018102530 0x7f8018102940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T07:56:28.738 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:27.268+0000 7f801f000700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8018102e80 con 0x7f8018102530 2024-03-29T07:56:28.738 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:27.269+0000 7f8017e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8018102530 0x7f8018102940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T07:56:28.738 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:27.269+0000 7f8017e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8018102530 0x7f8018102940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39886/0 (socket says 172.21.15.37:39886) 2024-03-29T07:56:28.738 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:27.269+0000 7f8017e00700 1 -- 172.21.15.37:0/2916118530 learned_addr learned my addr 172.21.15.37:0/2916118530 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T07:56:28.738 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:27.269+0000 7f8017e00700 1 -- 172.21.15.37:0/2916118530 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8018102fc0 con 0x7f8018102530 2024-03-29T07:56:28.738 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:27.269+0000 7f8017e00700 1 --2- 172.21.15.37:0/2916118530 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8018102530 0x7f8018102940 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f80080098e0 tx=0x7f8008009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89994f06b9c38410 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T07:56:28.738 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:27.269+0000 7f8016a00700 1 -- 172.21.15.37:0/2916118530 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f800800b860 con 0x7f8018102530 2024-03-29T07:56:28.738 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:27.269+0000 7f8016a00700 1 -- 172.21.15.37:0/2916118530 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f800800be70 con 0x7f8018102530 2024-03-29T07:56:28.738 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:27.270+0000 7f8016a00700 1 -- 172.21.15.37:0/2916118530 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8008010380 con 0x7f8018102530 2024-03-29T07:56:28.738 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:27.270+0000 7f801f000700 1 -- 172.21.15.37:0/2916118530 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8018102530 msgr2=0x7f8018102940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T07:56:28.738 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:27.270+0000 7f801f000700 1 --2- 172.21.15.37:0/2916118530 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8018102530 0x7f8018102940 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f80080098e0 tx=0x7f8008009250 comp rx=0 tx=0).stop 2024-03-29T07:56:28.739 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:27.270+0000 7f801f000700 1 -- 172.21.15.37:0/2916118530 shutdown_connections 2024-03-29T07:56:28.739 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:27.270+0000 7f801f000700 1 --2- 172.21.15.37:0/2916118530 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8018102530 0x7f8018102940 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T07:56:28.739 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:27.270+0000 7f801f000700 1 -- 172.21.15.37:0/2916118530 >> 172.21.15.37:0/2916118530 conn(0x7f80180fdcd0 msgr2=0x7f8018100100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T07:56:28.739 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:27.270+0000 7f801f000700 1 -- 172.21.15.37:0/2916118530 shutdown_connections 2024-03-29T07:56:28.739 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:27.270+0000 7f801f000700 1 -- 172.21.15.37:0/2916118530 wait complete. 2024-03-29T07:56:28.739 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:27.271+0000 7f801f000700 1 Processor -- start 2024-03-29T07:56:28.739 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:27.271+0000 7f801f000700 1 -- start start 2024-03-29T07:56:28.739 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:27.271+0000 7f801f000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8018102530 0x7f8018078c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T07:56:28.739 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:27.271+0000 7f801f000700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8018079190 con 0x7f8018102530 2024-03-29T07:56:28.739 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:27.272+0000 7f8017e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8018102530 0x7f8018078c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T07:56:28.739 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:27.272+0000 7f8017e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8018102530 0x7f8018078c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39900/0 (socket says 172.21.15.37:39900) 2024-03-29T07:56:28.739 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:27.272+0000 7f8017e00700 1 -- 172.21.15.37:0/4049992534 learned_addr learned my addr 172.21.15.37:0/4049992534 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T07:56:28.739 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:27.272+0000 7f8017e00700 1 -- 172.21.15.37:0/4049992534 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80080094f0 con 0x7f8018102530 2024-03-29T07:56:28.739 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:27.272+0000 7f8017e00700 1 --2- 172.21.15.37:0/4049992534 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8018102530 0x7f8018078c50 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f8008000c00 tx=0x7f8008009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T07:56:28.739 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:27.272+0000 7f800fe00700 1 -- 172.21.15.37:0/4049992534 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f80080105f0 con 0x7f8018102530 2024-03-29T07:56:28.740 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:27.273+0000 7f801f000700 1 -- 172.21.15.37:0/4049992534 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8018077200 con 0x7f8018102530 2024-03-29T07:56:28.740 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:27.273+0000 7f800fe00700 1 -- 172.21.15.37:0/4049992534 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f8008010c00 con 0x7f8018102530 2024-03-29T07:56:28.740 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:27.273+0000 7f800fe00700 1 -- 172.21.15.37:0/4049992534 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f800801ac70 con 0x7f8018102530 2024-03-29T07:56:28.740 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:27.273+0000 7f801f000700 1 -- 172.21.15.37:0/4049992534 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80180776f0 con 0x7f8018102530 2024-03-29T07:56:28.740 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:27.274+0000 7f800fe00700 1 -- 172.21.15.37:0/4049992534 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7f8008010d70 con 0x7f8018102530 2024-03-29T07:56:28.740 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:27.274+0000 7f801f000700 1 -- 172.21.15.37:0/4049992534 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f801804bed0 con 0x7f8018102530 2024-03-29T07:56:28.740 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:27.274+0000 7f800fe00700 1 --2- 172.21.15.37:0/4049992534 >> [v2:172.21.15.37:6800/2746946658,v1:172.21.15.37:6801/2746946658] conn(0x7f7ffc036790 0x7f7ffc038c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T07:56:28.740 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:27.274+0000 7f800fe00700 1 -- 172.21.15.37:0/4049992534 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f800804a360 con 0x7f8018102530 2024-03-29T07:56:28.740 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:27.274+0000 7f8017400700 1 --2- 172.21.15.37:0/4049992534 >> [v2:172.21.15.37:6800/2746946658,v1:172.21.15.37:6801/2746946658] conn(0x7f7ffc036790 0x7f7ffc038c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T07:56:28.740 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:27.275+0000 7f8017400700 1 --2- 172.21.15.37:0/4049992534 >> [v2:172.21.15.37:6800/2746946658,v1:172.21.15.37:6801/2746946658] conn(0x7f7ffc036790 0x7f7ffc038c40 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f8004009990 tx=0x7f8004006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T07:56:28.740 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:27.279+0000 7f800fe00700 1 -- 172.21.15.37:0/4049992534 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f8008018350 con 0x7f8018102530 2024-03-29T07:56:28.740 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:27.539+0000 7f801f000700 1 -- 172.21.15.37:0/4049992534 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) v1 -- 0x7f8018077e40 con 0x7f8018102530 2024-03-29T07:56:28.740 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:28.006+0000 7f800fe00700 1 -- 172.21.15.37:0/4049992534 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mgrmap(e 9) v1 ==== 43381+0+0 (secure 0 0 0) 0x7f8008025450 con 0x7f8018102530 2024-03-29T07:56:28.741 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:28.008+0000 7f800fe00700 1 -- 172.21.15.37:0/4049992534 <== mon.0 v2:172.21.15.37:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) v1 ==== 88+0+0 (secure 0 0 0) 0x7f80080193c0 con 0x7f8018102530 2024-03-29T07:56:28.741 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:28.011+0000 7f801f000700 1 -- 172.21.15.37:0/4049992534 >> [v2:172.21.15.37:6800/2746946658,v1:172.21.15.37:6801/2746946658] conn(0x7f7ffc036790 msgr2=0x7f7ffc038c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T07:56:28.741 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:28.011+0000 7f801f000700 1 --2- 172.21.15.37:0/4049992534 >> [v2:172.21.15.37:6800/2746946658,v1:172.21.15.37:6801/2746946658] conn(0x7f7ffc036790 0x7f7ffc038c40 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f8004009990 tx=0x7f8004006e40 comp rx=0 tx=0).stop 2024-03-29T07:56:28.741 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:28.011+0000 7f801f000700 1 -- 172.21.15.37:0/4049992534 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8018102530 msgr2=0x7f8018078c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T07:56:28.741 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:28.011+0000 7f801f000700 1 --2- 172.21.15.37:0/4049992534 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8018102530 0x7f8018078c50 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f8008000c00 tx=0x7f8008009250 comp rx=0 tx=0).stop 2024-03-29T07:56:28.741 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:28.011+0000 7f801f000700 1 -- 172.21.15.37:0/4049992534 shutdown_connections 2024-03-29T07:56:28.741 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:28.011+0000 7f801f000700 1 --2- 172.21.15.37:0/4049992534 >> [v2:172.21.15.37:6800/2746946658,v1:172.21.15.37:6801/2746946658] conn(0x7f7ffc036790 0x7f7ffc038c40 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T07:56:28.741 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:28.011+0000 7f801f000700 1 --2- 172.21.15.37:0/4049992534 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8018102530 0x7f8018078c50 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T07:56:28.741 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:28.011+0000 7f801f000700 1 -- 172.21.15.37:0/4049992534 >> 172.21.15.37:0/4049992534 conn(0x7f80180fdcd0 msgr2=0x7f80180fe910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T07:56:28.741 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:28.011+0000 7f801f000700 1 -- 172.21.15.37:0/4049992534 shutdown_connections 2024-03-29T07:56:28.742 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:28.011+0000 7f801f000700 1 -- 172.21.15.37:0/4049992534 wait complete. 2024-03-29T07:56:29.426 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:56:29 smithi037 ceph-mon[35162]: from='client.? 172.21.15.37:0/4049992534' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-03-29T07:56:29.426 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:56:29 smithi037 ceph-mon[35162]: mgrmap e9: smithi037.wjikii(active, since 28s) 2024-03-29T07:56:30.176 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:56:30 smithi037 ceph-mon[35162]: from='client.? 172.21.15.37:0/4261647212' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-03-29T07:56:30.267 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout { 2024-03-29T07:56:30.267 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-03-29T07:56:30.267 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "available": true, 2024-03-29T07:56:30.267 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "active_name": "smithi037.wjikii", 2024-03-29T07:56:30.268 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-03-29T07:56:30.268 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout } 2024-03-29T07:56:30.268 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:29.345+0000 7fdb34c00700 1 Processor -- start 2024-03-29T07:56:30.268 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:29.345+0000 7fdb34c00700 1 -- start start 2024-03-29T07:56:30.268 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:29.345+0000 7fdb34c00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdb30078d50 0x7fdb30077260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T07:56:30.268 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:29.345+0000 7fdb34c00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb30079160 con 0x7fdb30078d50 2024-03-29T07:56:30.268 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:29.346+0000 7fdb2de00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdb30078d50 0x7fdb30077260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T07:56:30.268 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:29.346+0000 7fdb2de00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdb30078d50 0x7fdb30077260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39946/0 (socket says 172.21.15.37:39946) 2024-03-29T07:56:30.268 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:29.346+0000 7fdb2de00700 1 -- 172.21.15.37:0/1756963378 learned_addr learned my addr 172.21.15.37:0/1756963378 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T07:56:30.268 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:29.346+0000 7fdb2de00700 1 -- 172.21.15.37:0/1756963378 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb300777a0 con 0x7fdb30078d50 2024-03-29T07:56:30.268 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:29.346+0000 7fdb2de00700 1 --2- 172.21.15.37:0/1756963378 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdb30078d50 0x7fdb30077260 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fdb200098e0 tx=0x7fdb20009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9df507a80bc1ddc7 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T07:56:30.268 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:29.347+0000 7fdb2ca00700 1 -- 172.21.15.37:0/1756963378 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdb2000b860 con 0x7fdb30078d50 2024-03-29T07:56:30.268 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:29.347+0000 7fdb2ca00700 1 -- 172.21.15.37:0/1756963378 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fdb2000be70 con 0x7fdb30078d50 2024-03-29T07:56:30.268 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:29.347+0000 7fdb2ca00700 1 -- 172.21.15.37:0/1756963378 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdb20010380 con 0x7fdb30078d50 2024-03-29T07:56:30.268 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:29.347+0000 7fdb34c00700 1 -- 172.21.15.37:0/1756963378 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdb30078d50 msgr2=0x7fdb30077260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T07:56:30.269 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:29.347+0000 7fdb34c00700 1 --2- 172.21.15.37:0/1756963378 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdb30078d50 0x7fdb30077260 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fdb200098e0 tx=0x7fdb20009250 comp rx=0 tx=0).stop 2024-03-29T07:56:30.269 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:29.347+0000 7fdb34c00700 1 -- 172.21.15.37:0/1756963378 shutdown_connections 2024-03-29T07:56:30.269 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:29.347+0000 7fdb34c00700 1 --2- 172.21.15.37:0/1756963378 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdb30078d50 0x7fdb30077260 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T07:56:30.269 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:29.347+0000 7fdb34c00700 1 -- 172.21.15.37:0/1756963378 >> 172.21.15.37:0/1756963378 conn(0x7fdb300fdd00 msgr2=0x7fdb30100110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T07:56:30.269 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:29.348+0000 7fdb34c00700 1 -- 172.21.15.37:0/1756963378 shutdown_connections 2024-03-29T07:56:30.269 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:29.348+0000 7fdb34c00700 1 -- 172.21.15.37:0/1756963378 wait complete. 2024-03-29T07:56:30.269 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:29.348+0000 7fdb34c00700 1 Processor -- start 2024-03-29T07:56:30.269 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:29.349+0000 7fdb34c00700 1 -- start start 2024-03-29T07:56:30.269 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:29.349+0000 7fdb34c00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdb30078d50 0x7fdb301135a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T07:56:30.269 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:29.349+0000 7fdb34c00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb30113ae0 con 0x7fdb30078d50 2024-03-29T07:56:30.269 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:29.349+0000 7fdb2de00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdb30078d50 0x7fdb301135a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T07:56:30.269 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:29.349+0000 7fdb2de00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdb30078d50 0x7fdb301135a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39954/0 (socket says 172.21.15.37:39954) 2024-03-29T07:56:30.269 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:29.349+0000 7fdb2de00700 1 -- 172.21.15.37:0/4261647212 learned_addr learned my addr 172.21.15.37:0/4261647212 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T07:56:30.269 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:29.350+0000 7fdb2de00700 1 -- 172.21.15.37:0/4261647212 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb200094f0 con 0x7fdb30078d50 2024-03-29T07:56:30.269 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:29.350+0000 7fdb2de00700 1 --2- 172.21.15.37:0/4261647212 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdb30078d50 0x7fdb301135a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fdb20006d40 tx=0x7fdb20009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T07:56:30.270 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:29.350+0000 7fdb26000700 1 -- 172.21.15.37:0/4261647212 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdb20010960 con 0x7fdb30078d50 2024-03-29T07:56:30.270 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:29.350+0000 7fdb26000700 1 -- 172.21.15.37:0/4261647212 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fdb2001a430 con 0x7fdb30078d50 2024-03-29T07:56:30.270 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:29.350+0000 7fdb26000700 1 -- 172.21.15.37:0/4261647212 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdb200245b0 con 0x7fdb30078d50 2024-03-29T07:56:30.270 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:29.350+0000 7fdb34c00700 1 -- 172.21.15.37:0/4261647212 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb30113ce0 con 0x7fdb30078d50 2024-03-29T07:56:30.270 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:29.350+0000 7fdb34c00700 1 -- 172.21.15.37:0/4261647212 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb30114850 con 0x7fdb30078d50 2024-03-29T07:56:30.270 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:29.351+0000 7fdb34c00700 1 -- 172.21.15.37:0/4261647212 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb3005f5f0 con 0x7fdb30078d50 2024-03-29T07:56:30.270 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:29.352+0000 7fdb26000700 1 -- 172.21.15.37:0/4261647212 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 9) v1 ==== 43381+0+0 (secure 0 0 0) 0x7fdb2001a9f0 con 0x7fdb30078d50 2024-03-29T07:56:30.270 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:29.352+0000 7fdb26000700 1 --2- 172.21.15.37:0/4261647212 >> [v2:172.21.15.37:6800/2746946658,v1:172.21.15.37:6801/2746946658] conn(0x7fdb14036830 0x7fdb14038ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T07:56:30.270 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:29.352+0000 7fdb26000700 1 -- 172.21.15.37:0/4261647212 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7fdb2004a6b0 con 0x7fdb30078d50 2024-03-29T07:56:30.270 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:29.353+0000 7fdb2d400700 1 -- 172.21.15.37:0/4261647212 >> [v2:172.21.15.37:6800/2746946658,v1:172.21.15.37:6801/2746946658] conn(0x7fdb14036830 msgr2=0x7fdb14038ce0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/2746946658 2024-03-29T07:56:30.270 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:29.353+0000 7fdb2d400700 1 --2- 172.21.15.37:0/4261647212 >> [v2:172.21.15.37:6800/2746946658,v1:172.21.15.37:6801/2746946658] conn(0x7fdb14036830 0x7fdb14038ce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-29T07:56:30.270 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:29.357+0000 7fdb26000700 1 -- 172.21.15.37:0/4261647212 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fdb20018db0 con 0x7fdb30078d50 2024-03-29T07:56:30.270 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:29.554+0000 7fdb2d400700 1 -- 172.21.15.37:0/4261647212 >> [v2:172.21.15.37:6800/2746946658,v1:172.21.15.37:6801/2746946658] conn(0x7fdb14036830 msgr2=0x7fdb14038ce0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/2746946658 2024-03-29T07:56:30.270 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:29.554+0000 7fdb2d400700 1 --2- 172.21.15.37:0/4261647212 >> [v2:172.21.15.37:6800/2746946658,v1:172.21.15.37:6801/2746946658] conn(0x7fdb14036830 0x7fdb14038ce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-29T07:56:30.271 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:29.619+0000 7fdb34c00700 1 -- 172.21.15.37:0/4261647212 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7fdb30114b90 con 0x7fdb30078d50 2024-03-29T07:56:30.271 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:29.619+0000 7fdb26000700 1 -- 172.21.15.37:0/4261647212 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) v1 ==== 56+0+103 (secure 0 0 0) 0x7fdb20018350 con 0x7fdb30078d50 2024-03-29T07:56:30.271 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:29.622+0000 7fdb34c00700 1 -- 172.21.15.37:0/4261647212 >> [v2:172.21.15.37:6800/2746946658,v1:172.21.15.37:6801/2746946658] conn(0x7fdb14036830 msgr2=0x7fdb14038ce0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-29T07:56:30.271 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:29.622+0000 7fdb34c00700 1 --2- 172.21.15.37:0/4261647212 >> [v2:172.21.15.37:6800/2746946658,v1:172.21.15.37:6801/2746946658] conn(0x7fdb14036830 0x7fdb14038ce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T07:56:30.271 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:29.622+0000 7fdb34c00700 1 -- 172.21.15.37:0/4261647212 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdb30078d50 msgr2=0x7fdb301135a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T07:56:30.271 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:29.622+0000 7fdb34c00700 1 --2- 172.21.15.37:0/4261647212 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdb30078d50 0x7fdb301135a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fdb20006d40 tx=0x7fdb20009250 comp rx=0 tx=0).stop 2024-03-29T07:56:30.271 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:29.622+0000 7fdb34c00700 1 -- 172.21.15.37:0/4261647212 shutdown_connections 2024-03-29T07:56:30.271 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:29.622+0000 7fdb34c00700 1 --2- 172.21.15.37:0/4261647212 >> [v2:172.21.15.37:6800/2746946658,v1:172.21.15.37:6801/2746946658] conn(0x7fdb14036830 0x7fdb14038ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T07:56:30.271 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:29.622+0000 7fdb34c00700 1 --2- 172.21.15.37:0/4261647212 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdb30078d50 0x7fdb301135a0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T07:56:30.271 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:29.622+0000 7fdb34c00700 1 -- 172.21.15.37:0/4261647212 >> 172.21.15.37:0/4261647212 conn(0x7fdb300fdd00 msgr2=0x7fdb300fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T07:56:30.271 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:29.622+0000 7fdb34c00700 1 -- 172.21.15.37:0/4261647212 shutdown_connections 2024-03-29T07:56:30.271 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:29.622+0000 7fdb34c00700 1 -- 172.21.15.37:0/4261647212 wait complete. 2024-03-29T07:56:30.271 INFO:teuthology.orchestra.run.smithi037.stdout:Waiting for the mgr to restart... 2024-03-29T07:56:30.271 INFO:teuthology.orchestra.run.smithi037.stdout:Waiting for mgr epoch 9... 2024-03-29T07:56:36.761 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:56:36 smithi037 ceph-mon[35162]: Active manager daemon smithi037.wjikii restarted 2024-03-29T07:56:36.761 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:56:36 smithi037 ceph-mon[35162]: Activating manager daemon smithi037.wjikii 2024-03-29T07:56:36.761 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:56:36 smithi037 ceph-mon[35162]: osdmap e3: 0 total, 0 up, 0 in 2024-03-29T07:56:36.761 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:56:36 smithi037 ceph-mon[35162]: mgrmap e10: smithi037.wjikii(active, starting, since 0.00400741s) 2024-03-29T07:56:36.761 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:56:36 smithi037 ceph-mon[35162]: from='mgr.14160 172.21.15.37:0/1836651950' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "mon metadata", "id": "smithi037"}]: dispatch 2024-03-29T07:56:36.761 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:56:36 smithi037 ceph-mon[35162]: from='mgr.14160 172.21.15.37:0/1836651950' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "mgr metadata", "who": "smithi037.wjikii", "id": "smithi037.wjikii"}]: dispatch 2024-03-29T07:56:36.761 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:56:36 smithi037 ceph-mon[35162]: from='mgr.14160 172.21.15.37:0/1836651950' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-29T07:56:36.761 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:56:36 smithi037 ceph-mon[35162]: from='mgr.14160 172.21.15.37:0/1836651950' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-29T07:56:36.762 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:56:36 smithi037 ceph-mon[35162]: from='mgr.14160 172.21.15.37:0/1836651950' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-29T07:56:36.762 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:56:36 smithi037 ceph-mon[35162]: Manager daemon smithi037.wjikii is now available 2024-03-29T07:56:36.762 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:56:36 smithi037 ceph-mon[35162]: from='mgr.14160 172.21.15.37:0/1836651950' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T07:56:36.762 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:56:36 smithi037 ceph-mon[35162]: from='mgr.14160 172.21.15.37:0/1836651950' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T07:56:37.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:56:37 smithi037 ceph-mon[35162]: from='mgr.14160 172.21.15.37:0/1836651950' entity='mgr.smithi037.wjikii' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi037.wjikii/mirror_snapshot_schedule"}]: dispatch 2024-03-29T07:56:37.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:56:37 smithi037 ceph-mon[35162]: from='mgr.14160 172.21.15.37:0/1836651950' entity='mgr.smithi037.wjikii' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi037.wjikii/trash_purge_schedule"}]: dispatch 2024-03-29T07:56:37.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:56:37 smithi037 ceph-mon[35162]: from='mgr.14160 172.21.15.37:0/1836651950' entity='mgr.smithi037.wjikii' 2024-03-29T07:56:37.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:56:37 smithi037 ceph-mon[35162]: mgrmap e11: smithi037.wjikii(active, since 1.00938s) 2024-03-29T07:56:38.258 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout { 2024-03-29T07:56:38.259 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-03-29T07:56:38.259 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "initialized": true 2024-03-29T07:56:38.259 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout } 2024-03-29T07:56:38.259 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:30.832+0000 7f8c51200700 1 Processor -- start 2024-03-29T07:56:38.259 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:30.832+0000 7f8c51200700 1 -- start start 2024-03-29T07:56:38.259 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:30.833+0000 7f8c51200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8c4c078d50 0x7f8c4c077260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T07:56:38.259 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:30.833+0000 7f8c51200700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c4c079160 con 0x7f8c4c078d50 2024-03-29T07:56:38.259 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:30.834+0000 7f8c49e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8c4c078d50 0x7f8c4c077260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T07:56:38.259 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:30.834+0000 7f8c49e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8c4c078d50 0x7f8c4c077260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58770/0 (socket says 172.21.15.37:58770) 2024-03-29T07:56:38.259 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:30.834+0000 7f8c49e00700 1 -- 172.21.15.37:0/3461128753 learned_addr learned my addr 172.21.15.37:0/3461128753 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T07:56:38.259 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:30.834+0000 7f8c49e00700 1 -- 172.21.15.37:0/3461128753 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c4c0777a0 con 0x7f8c4c078d50 2024-03-29T07:56:38.259 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:30.834+0000 7f8c49e00700 1 --2- 172.21.15.37:0/3461128753 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8c4c078d50 0x7f8c4c077260 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f8c3c0098e0 tx=0x7f8c3c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e4b047564f82da9 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T07:56:38.260 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:30.834+0000 7f8c48a00700 1 -- 172.21.15.37:0/3461128753 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8c3c00b860 con 0x7f8c4c078d50 2024-03-29T07:56:38.260 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:30.834+0000 7f8c48a00700 1 -- 172.21.15.37:0/3461128753 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f8c3c00be70 con 0x7f8c4c078d50 2024-03-29T07:56:38.260 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:30.835+0000 7f8c48a00700 1 -- 172.21.15.37:0/3461128753 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8c3c010380 con 0x7f8c4c078d50 2024-03-29T07:56:38.260 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:30.835+0000 7f8c51200700 1 -- 172.21.15.37:0/3461128753 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8c4c078d50 msgr2=0x7f8c4c077260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T07:56:38.260 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:30.835+0000 7f8c51200700 1 --2- 172.21.15.37:0/3461128753 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8c4c078d50 0x7f8c4c077260 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f8c3c0098e0 tx=0x7f8c3c009250 comp rx=0 tx=0).stop 2024-03-29T07:56:38.260 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:30.835+0000 7f8c51200700 1 -- 172.21.15.37:0/3461128753 shutdown_connections 2024-03-29T07:56:38.260 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:30.835+0000 7f8c51200700 1 --2- 172.21.15.37:0/3461128753 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8c4c078d50 0x7f8c4c077260 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T07:56:38.260 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:30.835+0000 7f8c51200700 1 -- 172.21.15.37:0/3461128753 >> 172.21.15.37:0/3461128753 conn(0x7f8c4c0fdd00 msgr2=0x7f8c4c100110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T07:56:38.260 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:30.835+0000 7f8c51200700 1 -- 172.21.15.37:0/3461128753 shutdown_connections 2024-03-29T07:56:38.260 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:30.835+0000 7f8c51200700 1 -- 172.21.15.37:0/3461128753 wait complete. 2024-03-29T07:56:38.260 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:30.836+0000 7f8c51200700 1 Processor -- start 2024-03-29T07:56:38.260 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:30.836+0000 7f8c51200700 1 -- start start 2024-03-29T07:56:38.260 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:30.837+0000 7f8c51200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8c4c078d50 0x7f8c4c1135a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T07:56:38.260 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:30.837+0000 7f8c51200700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c4c113ae0 con 0x7f8c4c078d50 2024-03-29T07:56:38.261 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:30.837+0000 7f8c49e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8c4c078d50 0x7f8c4c1135a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T07:56:38.261 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:30.837+0000 7f8c49e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8c4c078d50 0x7f8c4c1135a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58784/0 (socket says 172.21.15.37:58784) 2024-03-29T07:56:38.261 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:30.837+0000 7f8c49e00700 1 -- 172.21.15.37:0/3225079583 learned_addr learned my addr 172.21.15.37:0/3225079583 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T07:56:38.261 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:30.837+0000 7f8c49e00700 1 -- 172.21.15.37:0/3225079583 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c3c0094f0 con 0x7f8c4c078d50 2024-03-29T07:56:38.261 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:30.838+0000 7f8c49e00700 1 --2- 172.21.15.37:0/3225079583 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8c4c078d50 0x7f8c4c1135a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f8c3c006d40 tx=0x7f8c3c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T07:56:38.261 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:30.838+0000 7f8c42000700 1 -- 172.21.15.37:0/3225079583 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8c3c010960 con 0x7f8c4c078d50 2024-03-29T07:56:38.261 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:30.838+0000 7f8c42000700 1 -- 172.21.15.37:0/3225079583 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f8c3c01a430 con 0x7f8c4c078d50 2024-03-29T07:56:38.261 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:30.838+0000 7f8c51200700 1 -- 172.21.15.37:0/3225079583 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c4c113ce0 con 0x7f8c4c078d50 2024-03-29T07:56:38.261 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:30.838+0000 7f8c42000700 1 -- 172.21.15.37:0/3225079583 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8c3c0245b0 con 0x7f8c4c078d50 2024-03-29T07:56:38.261 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:30.838+0000 7f8c51200700 1 -- 172.21.15.37:0/3225079583 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c4c114850 con 0x7f8c4c078d50 2024-03-29T07:56:38.261 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:30.839+0000 7f8c42000700 1 -- 172.21.15.37:0/3225079583 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 9) v1 ==== 43381+0+0 (secure 0 0 0) 0x7f8c3c010ac0 con 0x7f8c4c078d50 2024-03-29T07:56:38.261 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:30.840+0000 7f8c42000700 1 --2- 172.21.15.37:0/3225079583 >> [v2:172.21.15.37:6800/2746946658,v1:172.21.15.37:6801/2746946658] conn(0x7f8c300367e0 0x7f8c30038c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T07:56:38.261 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:30.840+0000 7f8c42000700 1 -- 172.21.15.37:0/3225079583 --> [v2:172.21.15.37:6800/2746946658,v1:172.21.15.37:6801/2746946658] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8c300393a0 con 0x7f8c300367e0 2024-03-29T07:56:38.262 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:30.840+0000 7f8c42000700 1 -- 172.21.15.37:0/3225079583 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f8c3c04a410 con 0x7f8c4c078d50 2024-03-29T07:56:38.262 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:30.840+0000 7f8c49400700 1 -- 172.21.15.37:0/3225079583 >> [v2:172.21.15.37:6800/2746946658,v1:172.21.15.37:6801/2746946658] conn(0x7f8c300367e0 msgr2=0x7f8c30038c90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/2746946658 2024-03-29T07:56:38.262 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:30.840+0000 7f8c49400700 1 --2- 172.21.15.37:0/3225079583 >> [v2:172.21.15.37:6800/2746946658,v1:172.21.15.37:6801/2746946658] conn(0x7f8c300367e0 0x7f8c30038c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-29T07:56:38.262 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:31.041+0000 7f8c49400700 1 -- 172.21.15.37:0/3225079583 >> [v2:172.21.15.37:6800/2746946658,v1:172.21.15.37:6801/2746946658] conn(0x7f8c300367e0 msgr2=0x7f8c30038c90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/2746946658 2024-03-29T07:56:38.262 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:31.041+0000 7f8c49400700 1 --2- 172.21.15.37:0/3225079583 >> [v2:172.21.15.37:6800/2746946658,v1:172.21.15.37:6801/2746946658] conn(0x7f8c300367e0 0x7f8c30038c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-29T07:56:38.262 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:31.442+0000 7f8c49400700 1 -- 172.21.15.37:0/3225079583 >> [v2:172.21.15.37:6800/2746946658,v1:172.21.15.37:6801/2746946658] conn(0x7f8c300367e0 msgr2=0x7f8c30038c90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/2746946658 2024-03-29T07:56:38.262 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:31.442+0000 7f8c49400700 1 --2- 172.21.15.37:0/3225079583 >> [v2:172.21.15.37:6800/2746946658,v1:172.21.15.37:6801/2746946658] conn(0x7f8c300367e0 0x7f8c30038c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-29T07:56:38.262 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:32.243+0000 7f8c49400700 1 -- 172.21.15.37:0/3225079583 >> [v2:172.21.15.37:6800/2746946658,v1:172.21.15.37:6801/2746946658] conn(0x7f8c300367e0 msgr2=0x7f8c30038c90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/2746946658 2024-03-29T07:56:38.262 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:32.243+0000 7f8c49400700 1 --2- 172.21.15.37:0/3225079583 >> [v2:172.21.15.37:6800/2746946658,v1:172.21.15.37:6801/2746946658] conn(0x7f8c300367e0 0x7f8c30038c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-29T07:56:38.262 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:33.844+0000 7f8c49400700 1 -- 172.21.15.37:0/3225079583 >> [v2:172.21.15.37:6800/2746946658,v1:172.21.15.37:6801/2746946658] conn(0x7f8c300367e0 msgr2=0x7f8c30038c90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/2746946658 2024-03-29T07:56:38.262 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:33.844+0000 7f8c49400700 1 --2- 172.21.15.37:0/3225079583 >> [v2:172.21.15.37:6800/2746946658,v1:172.21.15.37:6801/2746946658] conn(0x7f8c300367e0 0x7f8c30038c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-03-29T07:56:38.262 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:36.634+0000 7f8c42000700 1 -- 172.21.15.37:0/3225079583 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mgrmap(e 10) v1 ==== 43148+0+0 (secure 0 0 0) 0x7f8c3c02d3f0 con 0x7f8c4c078d50 2024-03-29T07:56:38.262 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:36.634+0000 7f8c42000700 1 -- 172.21.15.37:0/3225079583 >> [v2:172.21.15.37:6800/2746946658,v1:172.21.15.37:6801/2746946658] conn(0x7f8c300367e0 msgr2=0x7f8c30038c90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-29T07:56:38.263 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:36.634+0000 7f8c42000700 1 --2- 172.21.15.37:0/3225079583 >> [v2:172.21.15.37:6800/2746946658,v1:172.21.15.37:6801/2746946658] conn(0x7f8c300367e0 0x7f8c30038c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T07:56:38.263 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:37.639+0000 7f8c42000700 1 -- 172.21.15.37:0/3225079583 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mgrmap(e 11) v1 ==== 43275+0+0 (secure 0 0 0) 0x7f8c3c010d70 con 0x7f8c4c078d50 2024-03-29T07:56:38.263 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:37.639+0000 7f8c42000700 1 --2- 172.21.15.37:0/3225079583 >> [v2:172.21.15.37:6800/4245458578,v1:172.21.15.37:6801/4245458578] conn(0x7f8c30039fd0 0x7f8c3003c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T07:56:38.263 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:37.639+0000 7f8c42000700 1 -- 172.21.15.37:0/3225079583 --> [v2:172.21.15.37:6800/4245458578,v1:172.21.15.37:6801/4245458578] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8c300393a0 con 0x7f8c30039fd0 2024-03-29T07:56:38.263 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:37.640+0000 7f8c49400700 1 --2- 172.21.15.37:0/3225079583 >> [v2:172.21.15.37:6800/4245458578,v1:172.21.15.37:6801/4245458578] conn(0x7f8c30039fd0 0x7f8c3003c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T07:56:38.263 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:37.641+0000 7f8c49400700 1 --2- 172.21.15.37:0/3225079583 >> [v2:172.21.15.37:6800/4245458578,v1:172.21.15.37:6801/4245458578] conn(0x7f8c30039fd0 0x7f8c3003c3b0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f8c380039d0 tx=0x7f8c380092f0 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T07:56:38.263 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:37.642+0000 7f8c42000700 1 -- 172.21.15.37:0/3225079583 <== mgr.14160 v2:172.21.15.37:6800/4245458578 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6549 (secure 0 0 0) 0x7f8c300393a0 con 0x7f8c30039fd0 2024-03-29T07:56:38.263 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:37.649+0000 7f8c51200700 1 -- 172.21.15.37:0/3225079583 --> [v2:172.21.15.37:6800/4245458578,v1:172.21.15.37:6801/4245458578] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f8c4c114170 con 0x7f8c30039fd0 2024-03-29T07:56:38.263 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:37.649+0000 7f8c42000700 1 -- 172.21.15.37:0/3225079583 <== mgr.14160 v2:172.21.15.37:6800/4245458578 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+52 (secure 0 0 0) 0x7f8c4c114170 con 0x7f8c30039fd0 2024-03-29T07:56:38.263 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:37.649+0000 7f8c51200700 1 -- 172.21.15.37:0/3225079583 >> [v2:172.21.15.37:6800/4245458578,v1:172.21.15.37:6801/4245458578] conn(0x7f8c30039fd0 msgr2=0x7f8c3003c3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T07:56:38.263 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:37.649+0000 7f8c51200700 1 --2- 172.21.15.37:0/3225079583 >> [v2:172.21.15.37:6800/4245458578,v1:172.21.15.37:6801/4245458578] conn(0x7f8c30039fd0 0x7f8c3003c3b0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f8c380039d0 tx=0x7f8c380092f0 comp rx=0 tx=0).stop 2024-03-29T07:56:38.263 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:37.650+0000 7f8c51200700 1 -- 172.21.15.37:0/3225079583 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8c4c078d50 msgr2=0x7f8c4c1135a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T07:56:38.263 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:37.650+0000 7f8c51200700 1 --2- 172.21.15.37:0/3225079583 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8c4c078d50 0x7f8c4c1135a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f8c3c006d40 tx=0x7f8c3c009250 comp rx=0 tx=0).stop 2024-03-29T07:56:38.263 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:37.650+0000 7f8c51200700 1 -- 172.21.15.37:0/3225079583 shutdown_connections 2024-03-29T07:56:38.264 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:37.650+0000 7f8c51200700 1 --2- 172.21.15.37:0/3225079583 >> [v2:172.21.15.37:6800/4245458578,v1:172.21.15.37:6801/4245458578] conn(0x7f8c30039fd0 0x7f8c3003c3b0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T07:56:38.264 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:37.650+0000 7f8c51200700 1 --2- 172.21.15.37:0/3225079583 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8c4c078d50 0x7f8c4c1135a0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T07:56:38.264 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:37.650+0000 7f8c51200700 1 --2- 172.21.15.37:0/3225079583 >> [v2:172.21.15.37:6800/2746946658,v1:172.21.15.37:6801/2746946658] conn(0x7f8c300367e0 0x7f8c30038c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T07:56:38.264 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:37.650+0000 7f8c51200700 1 -- 172.21.15.37:0/3225079583 >> 172.21.15.37:0/3225079583 conn(0x7f8c4c0fdd00 msgr2=0x7f8c4c0fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T07:56:38.264 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:37.650+0000 7f8c51200700 1 -- 172.21.15.37:0/3225079583 shutdown_connections 2024-03-29T07:56:38.264 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:37.650+0000 7f8c51200700 1 -- 172.21.15.37:0/3225079583 wait complete. 2024-03-29T07:56:38.264 INFO:teuthology.orchestra.run.smithi037.stdout:mgr epoch 9 is available 2024-03-29T07:56:38.264 INFO:teuthology.orchestra.run.smithi037.stdout:Generating a dashboard self-signed certificate... 2024-03-29T07:56:38.952 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:56:38 smithi037 ceph-mon[35162]: [29/Mar/2024:07:56:37] ENGINE Bus STARTING 2024-03-29T07:56:38.952 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:56:38 smithi037 ceph-mon[35162]: [29/Mar/2024:07:56:37] ENGINE Serving on https://172.21.15.37:7150 2024-03-29T07:56:38.952 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:56:38 smithi037 ceph-mon[35162]: [29/Mar/2024:07:56:37] ENGINE Bus STARTED 2024-03-29T07:56:38.952 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:56:38 smithi037 ceph-mon[35162]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-03-29T07:56:38.952 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:56:38 smithi037 ceph-mon[35162]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-03-29T07:56:39.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:56:39 smithi037 ceph-mon[35162]: mgrmap e12: smithi037.wjikii(active, since 2s) 2024-03-29T07:56:39.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:56:39 smithi037 ceph-mon[35162]: from='mgr.14160 172.21.15.37:0/1836651950' entity='mgr.smithi037.wjikii' 2024-03-29T07:56:39.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:56:39 smithi037 ceph-mon[35162]: from='mgr.14160 172.21.15.37:0/1836651950' entity='mgr.smithi037.wjikii' 2024-03-29T07:56:39.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:56:39 smithi037 ceph-mon[35162]: from='mgr.14160 172.21.15.37:0/1836651950' entity='mgr.smithi037.wjikii' 2024-03-29T07:56:40.301 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-03-29T07:56:40.301 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:39.264+0000 7fbc7dc00700 1 Processor -- start 2024-03-29T07:56:40.301 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:39.264+0000 7fbc7dc00700 1 -- start start 2024-03-29T07:56:40.301 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:39.264+0000 7fbc7dc00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbc78076c80 0x7fbc781051b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T07:56:40.302 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:39.264+0000 7fbc7dc00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc781056f0 con 0x7fbc78076c80 2024-03-29T07:56:40.302 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:39.265+0000 7fbc76a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbc78076c80 0x7fbc781051b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T07:56:40.302 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:39.265+0000 7fbc76a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbc78076c80 0x7fbc781051b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58862/0 (socket says 172.21.15.37:58862) 2024-03-29T07:56:40.302 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:39.265+0000 7fbc76a00700 1 -- 172.21.15.37:0/2693045698 learned_addr learned my addr 172.21.15.37:0/2693045698 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T07:56:40.302 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:39.265+0000 7fbc76a00700 1 -- 172.21.15.37:0/2693045698 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc78105830 con 0x7fbc78076c80 2024-03-29T07:56:40.302 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:39.266+0000 7fbc76a00700 1 --2- 172.21.15.37:0/2693045698 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbc78076c80 0x7fbc781051b0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fbc680098e0 tx=0x7fbc68009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54eb472e7170bca5 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T07:56:40.302 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:39.266+0000 7fbc75600700 1 -- 172.21.15.37:0/2693045698 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbc6800b860 con 0x7fbc78076c80 2024-03-29T07:56:40.302 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:39.266+0000 7fbc75600700 1 -- 172.21.15.37:0/2693045698 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fbc6800be70 con 0x7fbc78076c80 2024-03-29T07:56:40.302 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:39.266+0000 7fbc75600700 1 -- 172.21.15.37:0/2693045698 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbc68010380 con 0x7fbc78076c80 2024-03-29T07:56:40.302 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:39.267+0000 7fbc7dc00700 1 -- 172.21.15.37:0/2693045698 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbc78076c80 msgr2=0x7fbc781051b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T07:56:40.302 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:39.267+0000 7fbc7dc00700 1 --2- 172.21.15.37:0/2693045698 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbc78076c80 0x7fbc781051b0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fbc680098e0 tx=0x7fbc68009250 comp rx=0 tx=0).stop 2024-03-29T07:56:40.302 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:39.267+0000 7fbc7dc00700 1 -- 172.21.15.37:0/2693045698 shutdown_connections 2024-03-29T07:56:40.302 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:39.267+0000 7fbc7dc00700 1 --2- 172.21.15.37:0/2693045698 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbc78076c80 0x7fbc781051b0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T07:56:40.303 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:39.267+0000 7fbc7dc00700 1 -- 172.21.15.37:0/2693045698 >> 172.21.15.37:0/2693045698 conn(0x7fbc780fcc60 msgr2=0x7fbc780ff070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T07:56:40.303 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:39.267+0000 7fbc7dc00700 1 -- 172.21.15.37:0/2693045698 shutdown_connections 2024-03-29T07:56:40.303 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:39.267+0000 7fbc7dc00700 1 -- 172.21.15.37:0/2693045698 wait complete. 2024-03-29T07:56:40.303 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:39.268+0000 7fbc7dc00700 1 Processor -- start 2024-03-29T07:56:40.303 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:39.268+0000 7fbc7dc00700 1 -- start start 2024-03-29T07:56:40.303 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:39.268+0000 7fbc7dc00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbc78076c80 0x7fbc781135b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T07:56:40.303 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:39.268+0000 7fbc7dc00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc78113af0 con 0x7fbc78076c80 2024-03-29T07:56:40.303 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:39.269+0000 7fbc76a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbc78076c80 0x7fbc781135b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T07:56:40.303 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:39.269+0000 7fbc76a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbc78076c80 0x7fbc781135b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58868/0 (socket says 172.21.15.37:58868) 2024-03-29T07:56:40.303 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:39.269+0000 7fbc76a00700 1 -- 172.21.15.37:0/24867022 learned_addr learned my addr 172.21.15.37:0/24867022 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T07:56:40.303 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:39.269+0000 7fbc76a00700 1 -- 172.21.15.37:0/24867022 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc680094f0 con 0x7fbc78076c80 2024-03-29T07:56:40.303 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:39.269+0000 7fbc76a00700 1 --2- 172.21.15.37:0/24867022 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbc78076c80 0x7fbc781135b0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fbc68006d40 tx=0x7fbc68009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T07:56:40.303 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:39.269+0000 7fbc6ea00700 1 -- 172.21.15.37:0/24867022 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbc68010960 con 0x7fbc78076c80 2024-03-29T07:56:40.303 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:39.269+0000 7fbc6ea00700 1 -- 172.21.15.37:0/24867022 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fbc6801a430 con 0x7fbc78076c80 2024-03-29T07:56:40.303 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:39.269+0000 7fbc6ea00700 1 -- 172.21.15.37:0/24867022 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbc680245b0 con 0x7fbc78076c80 2024-03-29T07:56:40.303 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:39.269+0000 7fbc7dc00700 1 -- 172.21.15.37:0/24867022 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc78113cf0 con 0x7fbc78076c80 2024-03-29T07:56:40.304 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:39.270+0000 7fbc7dc00700 1 -- 172.21.15.37:0/24867022 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc78114860 con 0x7fbc78076c80 2024-03-29T07:56:40.304 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:39.271+0000 7fbc7dc00700 1 -- 172.21.15.37:0/24867022 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc7805f5f0 con 0x7fbc78076c80 2024-03-29T07:56:40.304 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:39.271+0000 7fbc6ea00700 1 -- 172.21.15.37:0/24867022 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 12) v1 ==== 43381+0+0 (secure 0 0 0) 0x7fbc6801a9f0 con 0x7fbc78076c80 2024-03-29T07:56:40.304 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:39.272+0000 7fbc6ea00700 1 --2- 172.21.15.37:0/24867022 >> [v2:172.21.15.37:6800/4245458578,v1:172.21.15.37:6801/4245458578] conn(0x7fbc5c0367e0 0x7fbc5c038c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T07:56:40.304 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:39.272+0000 7fbc6ea00700 1 -- 172.21.15.37:0/24867022 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7fbc6804a420 con 0x7fbc78076c80 2024-03-29T07:56:40.304 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:39.272+0000 7fbc76000700 1 --2- 172.21.15.37:0/24867022 >> [v2:172.21.15.37:6800/4245458578,v1:172.21.15.37:6801/4245458578] conn(0x7fbc5c0367e0 0x7fbc5c038c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T07:56:40.304 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:39.273+0000 7fbc76000700 1 --2- 172.21.15.37:0/24867022 >> [v2:172.21.15.37:6800/4245458578,v1:172.21.15.37:6801/4245458578] conn(0x7fbc5c0367e0 0x7fbc5c038c90 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fbc64009990 tx=0x7fbc64006e40 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T07:56:40.304 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:39.277+0000 7fbc6ea00700 1 -- 172.21.15.37:0/24867022 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fbc6802b8e0 con 0x7fbc78076c80 2024-03-29T07:56:40.304 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:39.491+0000 7fbc7dc00700 1 -- 172.21.15.37:0/24867022 --> [v2:172.21.15.37:6800/4245458578,v1:172.21.15.37:6801/4245458578] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) v1 -- 0x7fbc780fde50 con 0x7fbc5c0367e0 2024-03-29T07:56:40.304 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:39.634+0000 7fbc6ea00700 1 -- 172.21.15.37:0/24867022 <== mgr.14160 v2:172.21.15.37:6800/4245458578 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (secure 0 0 0) 0x7fbc780fde50 con 0x7fbc5c0367e0 2024-03-29T07:56:40.305 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:39.636+0000 7fbc7dc00700 1 -- 172.21.15.37:0/24867022 >> [v2:172.21.15.37:6800/4245458578,v1:172.21.15.37:6801/4245458578] conn(0x7fbc5c0367e0 msgr2=0x7fbc5c038c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T07:56:40.305 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:39.636+0000 7fbc7dc00700 1 --2- 172.21.15.37:0/24867022 >> [v2:172.21.15.37:6800/4245458578,v1:172.21.15.37:6801/4245458578] conn(0x7fbc5c0367e0 0x7fbc5c038c90 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fbc64009990 tx=0x7fbc64006e40 comp rx=0 tx=0).stop 2024-03-29T07:56:40.305 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:39.636+0000 7fbc7dc00700 1 -- 172.21.15.37:0/24867022 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbc78076c80 msgr2=0x7fbc781135b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T07:56:40.305 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:39.636+0000 7fbc7dc00700 1 --2- 172.21.15.37:0/24867022 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbc78076c80 0x7fbc781135b0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fbc68006d40 tx=0x7fbc68009250 comp rx=0 tx=0).stop 2024-03-29T07:56:40.305 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:39.636+0000 7fbc7dc00700 1 -- 172.21.15.37:0/24867022 shutdown_connections 2024-03-29T07:56:40.305 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:39.636+0000 7fbc7dc00700 1 --2- 172.21.15.37:0/24867022 >> [v2:172.21.15.37:6800/4245458578,v1:172.21.15.37:6801/4245458578] conn(0x7fbc5c0367e0 0x7fbc5c038c90 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T07:56:40.305 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:39.636+0000 7fbc7dc00700 1 --2- 172.21.15.37:0/24867022 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbc78076c80 0x7fbc781135b0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T07:56:40.305 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:39.636+0000 7fbc7dc00700 1 -- 172.21.15.37:0/24867022 >> 172.21.15.37:0/24867022 conn(0x7fbc780fcc60 msgr2=0x7fbc780fd740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T07:56:40.305 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:39.636+0000 7fbc7dc00700 1 -- 172.21.15.37:0/24867022 shutdown_connections 2024-03-29T07:56:40.305 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:39.636+0000 7fbc7dc00700 1 -- 172.21.15.37:0/24867022 wait complete. 2024-03-29T07:56:40.305 INFO:teuthology.orchestra.run.smithi037.stdout:Creating initial admin user... 2024-03-29T07:56:40.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:56:40 smithi037 ceph-mon[35162]: from='client.14172 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T07:56:40.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:56:40 smithi037 ceph-mon[35162]: from='mgr.14160 172.21.15.37:0/1836651950' entity='mgr.smithi037.wjikii' 2024-03-29T07:56:42.437 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$1N5f3OQp6QdCAIoUbvZbFOTlm3tSvLzsKT.JrMioy5GgR16tmtYOu", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1711699001, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-03-29T07:56:42.437 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:40.950+0000 7fa54ce00700 1 Processor -- start 2024-03-29T07:56:42.437 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:40.950+0000 7fa54ce00700 1 -- start start 2024-03-29T07:56:42.437 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:40.950+0000 7fa54ce00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa548078d50 0x7fa548077260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T07:56:42.437 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:40.950+0000 7fa54ce00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa548079160 con 0x7fa548078d50 2024-03-29T07:56:42.437 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:40.951+0000 7fa545e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa548078d50 0x7fa548077260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T07:56:42.437 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:40.951+0000 7fa545e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa548078d50 0x7fa548077260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46352/0 (socket says 172.21.15.37:46352) 2024-03-29T07:56:42.437 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:40.951+0000 7fa545e00700 1 -- 172.21.15.37:0/1608330769 learned_addr learned my addr 172.21.15.37:0/1608330769 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T07:56:42.437 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:40.951+0000 7fa545e00700 1 -- 172.21.15.37:0/1608330769 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5480777a0 con 0x7fa548078d50 2024-03-29T07:56:42.437 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:40.951+0000 7fa545e00700 1 --2- 172.21.15.37:0/1608330769 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa548078d50 0x7fa548077260 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fa5380098e0 tx=0x7fa538009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4aa9558f3bbc3b84 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T07:56:42.437 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:40.952+0000 7fa544a00700 1 -- 172.21.15.37:0/1608330769 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa53800b860 con 0x7fa548078d50 2024-03-29T07:56:42.438 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:40.952+0000 7fa544a00700 1 -- 172.21.15.37:0/1608330769 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fa53800be70 con 0x7fa548078d50 2024-03-29T07:56:42.438 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:40.952+0000 7fa544a00700 1 -- 172.21.15.37:0/1608330769 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa538010380 con 0x7fa548078d50 2024-03-29T07:56:42.438 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:40.952+0000 7fa54ce00700 1 -- 172.21.15.37:0/1608330769 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa548078d50 msgr2=0x7fa548077260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T07:56:42.438 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:40.952+0000 7fa54ce00700 1 --2- 172.21.15.37:0/1608330769 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa548078d50 0x7fa548077260 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fa5380098e0 tx=0x7fa538009250 comp rx=0 tx=0).stop 2024-03-29T07:56:42.438 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:40.952+0000 7fa54ce00700 1 -- 172.21.15.37:0/1608330769 shutdown_connections 2024-03-29T07:56:42.438 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:40.952+0000 7fa54ce00700 1 --2- 172.21.15.37:0/1608330769 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa548078d50 0x7fa548077260 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T07:56:42.438 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:40.952+0000 7fa54ce00700 1 -- 172.21.15.37:0/1608330769 >> 172.21.15.37:0/1608330769 conn(0x7fa5480fdd00 msgr2=0x7fa548100110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T07:56:42.438 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:40.952+0000 7fa54ce00700 1 -- 172.21.15.37:0/1608330769 shutdown_connections 2024-03-29T07:56:42.438 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:40.953+0000 7fa54ce00700 1 -- 172.21.15.37:0/1608330769 wait complete. 2024-03-29T07:56:42.438 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:40.953+0000 7fa54ce00700 1 Processor -- start 2024-03-29T07:56:42.438 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:40.954+0000 7fa54ce00700 1 -- start start 2024-03-29T07:56:42.438 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:40.954+0000 7fa54ce00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa548078d50 0x7fa5481135b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T07:56:42.438 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:40.954+0000 7fa54ce00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa548113af0 con 0x7fa548078d50 2024-03-29T07:56:42.438 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:40.954+0000 7fa545e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa548078d50 0x7fa5481135b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T07:56:42.439 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:40.954+0000 7fa545e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa548078d50 0x7fa5481135b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46364/0 (socket says 172.21.15.37:46364) 2024-03-29T07:56:42.439 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:40.954+0000 7fa545e00700 1 -- 172.21.15.37:0/241275074 learned_addr learned my addr 172.21.15.37:0/241275074 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T07:56:42.439 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:40.955+0000 7fa545e00700 1 -- 172.21.15.37:0/241275074 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5380094f0 con 0x7fa548078d50 2024-03-29T07:56:42.439 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:40.955+0000 7fa545e00700 1 --2- 172.21.15.37:0/241275074 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa548078d50 0x7fa5481135b0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa538006d40 tx=0x7fa538009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T07:56:42.439 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:40.955+0000 7fa53e000700 1 -- 172.21.15.37:0/241275074 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa538010960 con 0x7fa548078d50 2024-03-29T07:56:42.439 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:40.955+0000 7fa54ce00700 1 -- 172.21.15.37:0/241275074 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa548113cf0 con 0x7fa548078d50 2024-03-29T07:56:42.439 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:40.956+0000 7fa53e000700 1 -- 172.21.15.37:0/241275074 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fa53801a430 con 0x7fa548078d50 2024-03-29T07:56:42.439 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:40.956+0000 7fa53e000700 1 -- 172.21.15.37:0/241275074 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa5380245b0 con 0x7fa548078d50 2024-03-29T07:56:42.439 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:40.956+0000 7fa54ce00700 1 -- 172.21.15.37:0/241275074 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa548114860 con 0x7fa548078d50 2024-03-29T07:56:42.439 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:40.957+0000 7fa54ce00700 1 -- 172.21.15.37:0/241275074 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa54805f5f0 con 0x7fa548078d50 2024-03-29T07:56:42.439 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:40.957+0000 7fa53e000700 1 -- 172.21.15.37:0/241275074 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 12) v1 ==== 43381+0+0 (secure 0 0 0) 0x7fa53801a9f0 con 0x7fa548078d50 2024-03-29T07:56:42.439 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:40.958+0000 7fa53e000700 1 --2- 172.21.15.37:0/241275074 >> [v2:172.21.15.37:6800/4245458578,v1:172.21.15.37:6801/4245458578] conn(0x7fa52c036830 0x7fa52c038ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T07:56:42.439 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:40.958+0000 7fa53e000700 1 -- 172.21.15.37:0/241275074 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7fa53804a3f0 con 0x7fa548078d50 2024-03-29T07:56:42.439 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:40.958+0000 7fa545400700 1 --2- 172.21.15.37:0/241275074 >> [v2:172.21.15.37:6800/4245458578,v1:172.21.15.37:6801/4245458578] conn(0x7fa52c036830 0x7fa52c038ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T07:56:42.439 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:40.959+0000 7fa545400700 1 --2- 172.21.15.37:0/241275074 >> [v2:172.21.15.37:6800/4245458578,v1:172.21.15.37:6801/4245458578] conn(0x7fa52c036830 0x7fa52c038ce0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa534009990 tx=0x7fa534006e40 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T07:56:42.440 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:40.963+0000 7fa53e000700 1 -- 172.21.15.37:0/241275074 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fa53802b8c0 con 0x7fa548078d50 2024-03-29T07:56:42.440 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:41.205+0000 7fa54ce00700 1 -- 172.21.15.37:0/241275074 --> [v2:172.21.15.37:6800/4245458578,v1:172.21.15.37:6801/4245458578] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) v1 -- 0x7fa54805f410 con 0x7fa52c036830 2024-03-29T07:56:42.440 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:41.457+0000 7fa53e000700 1 -- 172.21.15.37:0/241275074 <== mgr.14160 v2:172.21.15.37:6800/4245458578 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+252 (secure 0 0 0) 0x7fa54805f410 con 0x7fa52c036830 2024-03-29T07:56:42.440 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:41.460+0000 7fa54ce00700 1 -- 172.21.15.37:0/241275074 >> [v2:172.21.15.37:6800/4245458578,v1:172.21.15.37:6801/4245458578] conn(0x7fa52c036830 msgr2=0x7fa52c038ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T07:56:42.440 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:41.460+0000 7fa54ce00700 1 --2- 172.21.15.37:0/241275074 >> [v2:172.21.15.37:6800/4245458578,v1:172.21.15.37:6801/4245458578] conn(0x7fa52c036830 0x7fa52c038ce0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa534009990 tx=0x7fa534006e40 comp rx=0 tx=0).stop 2024-03-29T07:56:42.440 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:41.460+0000 7fa54ce00700 1 -- 172.21.15.37:0/241275074 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa548078d50 msgr2=0x7fa5481135b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T07:56:42.440 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:41.460+0000 7fa54ce00700 1 --2- 172.21.15.37:0/241275074 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa548078d50 0x7fa5481135b0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa538006d40 tx=0x7fa538009250 comp rx=0 tx=0).stop 2024-03-29T07:56:42.440 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:41.460+0000 7fa54ce00700 1 -- 172.21.15.37:0/241275074 shutdown_connections 2024-03-29T07:56:42.440 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:41.460+0000 7fa54ce00700 1 --2- 172.21.15.37:0/241275074 >> [v2:172.21.15.37:6800/4245458578,v1:172.21.15.37:6801/4245458578] conn(0x7fa52c036830 0x7fa52c038ce0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T07:56:42.440 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:41.460+0000 7fa54ce00700 1 --2- 172.21.15.37:0/241275074 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa548078d50 0x7fa5481135b0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T07:56:42.440 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:41.460+0000 7fa54ce00700 1 -- 172.21.15.37:0/241275074 >> 172.21.15.37:0/241275074 conn(0x7fa5480fdd00 msgr2=0x7fa5480fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T07:56:42.440 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:41.460+0000 7fa54ce00700 1 -- 172.21.15.37:0/241275074 shutdown_connections 2024-03-29T07:56:42.440 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:41.460+0000 7fa54ce00700 1 -- 172.21.15.37:0/241275074 wait complete. 2024-03-29T07:56:42.440 INFO:teuthology.orchestra.run.smithi037.stdout:Fetching dashboard port number... 2024-03-29T07:56:42.698 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:56:42 smithi037 ceph-mon[35162]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T07:56:42.698 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:56:42 smithi037 ceph-mon[35162]: from='mgr.14160 172.21.15.37:0/1836651950' entity='mgr.smithi037.wjikii' 2024-03-29T07:56:43.676 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:56:43 smithi037 ceph-mon[35162]: from='client.? 172.21.15.37:0/1272122810' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-03-29T07:56:43.968 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout 8443 2024-03-29T07:56:43.968 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:43.160+0000 7f6e3d400700 1 Processor -- start 2024-03-29T07:56:43.968 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:43.160+0000 7f6e3d400700 1 -- start start 2024-03-29T07:56:43.968 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:43.160+0000 7f6e3d400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6e38102530 0x7f6e38102940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T07:56:43.969 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:43.160+0000 7f6e3d400700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e38102e80 con 0x7f6e38102530 2024-03-29T07:56:43.969 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:43.160+0000 7f6e36800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6e38102530 0x7f6e38102940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T07:56:43.969 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:43.160+0000 7f6e36800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6e38102530 0x7f6e38102940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46376/0 (socket says 172.21.15.37:46376) 2024-03-29T07:56:43.969 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:43.160+0000 7f6e36800700 1 -- 172.21.15.37:0/4198537536 learned_addr learned my addr 172.21.15.37:0/4198537536 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T07:56:43.969 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:43.161+0000 7f6e36800700 1 -- 172.21.15.37:0/4198537536 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e38102fc0 con 0x7f6e38102530 2024-03-29T07:56:43.969 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:43.161+0000 7f6e36800700 1 --2- 172.21.15.37:0/4198537536 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6e38102530 0x7f6e38102940 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f6e280098e0 tx=0x7f6e28009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b675b6a5fa808655 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T07:56:43.969 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:43.161+0000 7f6e35400700 1 -- 172.21.15.37:0/4198537536 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6e2800b860 con 0x7f6e38102530 2024-03-29T07:56:43.969 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:43.161+0000 7f6e35400700 1 -- 172.21.15.37:0/4198537536 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f6e2800be70 con 0x7f6e38102530 2024-03-29T07:56:43.969 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:43.161+0000 7f6e35400700 1 -- 172.21.15.37:0/4198537536 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6e28010380 con 0x7f6e38102530 2024-03-29T07:56:43.969 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:43.162+0000 7f6e3d400700 1 -- 172.21.15.37:0/4198537536 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6e38102530 msgr2=0x7f6e38102940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T07:56:43.969 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:43.162+0000 7f6e3d400700 1 --2- 172.21.15.37:0/4198537536 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6e38102530 0x7f6e38102940 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f6e280098e0 tx=0x7f6e28009250 comp rx=0 tx=0).stop 2024-03-29T07:56:43.969 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:43.162+0000 7f6e3d400700 1 -- 172.21.15.37:0/4198537536 shutdown_connections 2024-03-29T07:56:43.969 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:43.162+0000 7f6e3d400700 1 --2- 172.21.15.37:0/4198537536 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6e38102530 0x7f6e38102940 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T07:56:43.969 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:43.162+0000 7f6e3d400700 1 -- 172.21.15.37:0/4198537536 >> 172.21.15.37:0/4198537536 conn(0x7f6e380fdcd0 msgr2=0x7f6e38100100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T07:56:43.970 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:43.162+0000 7f6e3d400700 1 -- 172.21.15.37:0/4198537536 shutdown_connections 2024-03-29T07:56:43.970 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:43.162+0000 7f6e3d400700 1 -- 172.21.15.37:0/4198537536 wait complete. 2024-03-29T07:56:43.970 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:43.163+0000 7f6e3d400700 1 Processor -- start 2024-03-29T07:56:43.970 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:43.163+0000 7f6e3d400700 1 -- start start 2024-03-29T07:56:43.970 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:43.163+0000 7f6e3d400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6e38102530 0x7f6e38113670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T07:56:43.970 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:43.163+0000 7f6e3d400700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e38113bb0 con 0x7f6e38102530 2024-03-29T07:56:43.971 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:43.164+0000 7f6e36800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6e38102530 0x7f6e38113670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T07:56:43.971 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:43.164+0000 7f6e36800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6e38102530 0x7f6e38113670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46392/0 (socket says 172.21.15.37:46392) 2024-03-29T07:56:43.971 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:43.164+0000 7f6e36800700 1 -- 172.21.15.37:0/1272122810 learned_addr learned my addr 172.21.15.37:0/1272122810 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T07:56:43.971 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:43.164+0000 7f6e36800700 1 -- 172.21.15.37:0/1272122810 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e280094f0 con 0x7f6e38102530 2024-03-29T07:56:43.971 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:43.164+0000 7f6e36800700 1 --2- 172.21.15.37:0/1272122810 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6e38102530 0x7f6e38113670 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f6e28000c00 tx=0x7f6e28009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T07:56:43.971 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:43.164+0000 7f6e2ea00700 1 -- 172.21.15.37:0/1272122810 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6e280105f0 con 0x7f6e38102530 2024-03-29T07:56:43.971 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:43.164+0000 7f6e2ea00700 1 -- 172.21.15.37:0/1272122810 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f6e28010c00 con 0x7f6e38102530 2024-03-29T07:56:43.971 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:43.164+0000 7f6e3d400700 1 -- 172.21.15.37:0/1272122810 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e38113db0 con 0x7f6e38102530 2024-03-29T07:56:43.971 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:43.164+0000 7f6e2ea00700 1 -- 172.21.15.37:0/1272122810 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6e2801ac70 con 0x7f6e38102530 2024-03-29T07:56:43.971 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:43.165+0000 7f6e3d400700 1 -- 172.21.15.37:0/1272122810 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e38114920 con 0x7f6e38102530 2024-03-29T07:56:43.971 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:43.165+0000 7f6e2ea00700 1 -- 172.21.15.37:0/1272122810 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 12) v1 ==== 43381+0+0 (secure 0 0 0) 0x7f6e28010d70 con 0x7f6e38102530 2024-03-29T07:56:43.971 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:43.166+0000 7f6e3d400700 1 -- 172.21.15.37:0/1272122810 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e3805f5f0 con 0x7f6e38102530 2024-03-29T07:56:43.971 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:43.166+0000 7f6e2ea00700 1 --2- 172.21.15.37:0/1272122810 >> [v2:172.21.15.37:6800/4245458578,v1:172.21.15.37:6801/4245458578] conn(0x7f6e1c0367e0 0x7f6e1c038c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T07:56:43.971 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:43.166+0000 7f6e2ea00700 1 -- 172.21.15.37:0/1272122810 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f6e2804a4b0 con 0x7f6e38102530 2024-03-29T07:56:43.971 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:43.166+0000 7f6e35e00700 1 --2- 172.21.15.37:0/1272122810 >> [v2:172.21.15.37:6800/4245458578,v1:172.21.15.37:6801/4245458578] conn(0x7f6e1c0367e0 0x7f6e1c038c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T07:56:43.972 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:43.167+0000 7f6e35e00700 1 --2- 172.21.15.37:0/1272122810 >> [v2:172.21.15.37:6800/4245458578,v1:172.21.15.37:6801/4245458578] conn(0x7f6e1c0367e0 0x7f6e1c038c90 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f6e24009990 tx=0x7f6e24006e40 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T07:56:43.972 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:43.171+0000 7f6e2ea00700 1 -- 172.21.15.37:0/1272122810 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f6e28018350 con 0x7f6e38102530 2024-03-29T07:56:43.972 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:43.377+0000 7f6e3d400700 1 -- 172.21.15.37:0/1272122810 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) v1 -- 0x7f6e3804bed0 con 0x7f6e38102530 2024-03-29T07:56:43.972 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:43.378+0000 7f6e2ea00700 1 -- 172.21.15.37:0/1272122810 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) v1 ==== 112+0+5 (secure 0 0 0) 0x7f6e28020070 con 0x7f6e38102530 2024-03-29T07:56:43.972 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:43.380+0000 7f6e3d400700 1 -- 172.21.15.37:0/1272122810 >> [v2:172.21.15.37:6800/4245458578,v1:172.21.15.37:6801/4245458578] conn(0x7f6e1c0367e0 msgr2=0x7f6e1c038c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T07:56:43.972 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:43.380+0000 7f6e3d400700 1 --2- 172.21.15.37:0/1272122810 >> [v2:172.21.15.37:6800/4245458578,v1:172.21.15.37:6801/4245458578] conn(0x7f6e1c0367e0 0x7f6e1c038c90 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f6e24009990 tx=0x7f6e24006e40 comp rx=0 tx=0).stop 2024-03-29T07:56:43.972 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:43.380+0000 7f6e3d400700 1 -- 172.21.15.37:0/1272122810 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6e38102530 msgr2=0x7f6e38113670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T07:56:43.972 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:43.380+0000 7f6e3d400700 1 --2- 172.21.15.37:0/1272122810 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6e38102530 0x7f6e38113670 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f6e28000c00 tx=0x7f6e28009250 comp rx=0 tx=0).stop 2024-03-29T07:56:43.972 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:43.380+0000 7f6e3d400700 1 -- 172.21.15.37:0/1272122810 shutdown_connections 2024-03-29T07:56:43.972 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:43.380+0000 7f6e3d400700 1 --2- 172.21.15.37:0/1272122810 >> [v2:172.21.15.37:6800/4245458578,v1:172.21.15.37:6801/4245458578] conn(0x7f6e1c0367e0 0x7f6e1c038c90 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T07:56:43.972 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:43.380+0000 7f6e3d400700 1 --2- 172.21.15.37:0/1272122810 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6e38102530 0x7f6e38113670 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T07:56:43.972 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:43.380+0000 7f6e3d400700 1 -- 172.21.15.37:0/1272122810 >> 172.21.15.37:0/1272122810 conn(0x7f6e380fdcd0 msgr2=0x7f6e380fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T07:56:43.972 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:43.381+0000 7f6e3d400700 1 -- 172.21.15.37:0/1272122810 shutdown_connections 2024-03-29T07:56:43.972 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:43.381+0000 7f6e3d400700 1 -- 172.21.15.37:0/1272122810 wait complete. 2024-03-29T07:56:43.986 INFO:teuthology.orchestra.run.smithi037.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-29T07:56:43.986 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stdout disabled 2024-03-29T07:56:44.003 INFO:teuthology.orchestra.run.smithi037.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-29T07:56:44.003 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stdout inactive 2024-03-29T07:56:44.003 INFO:teuthology.orchestra.run.smithi037.stdout:firewalld.service is not enabled 2024-03-29T07:56:44.003 INFO:teuthology.orchestra.run.smithi037.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-03-29T07:56:44.006 INFO:teuthology.orchestra.run.smithi037.stdout:Ceph Dashboard is now available at: 2024-03-29T07:56:44.006 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T07:56:44.006 INFO:teuthology.orchestra.run.smithi037.stdout: URL: https://smithi037.front.sepia.ceph.com:8443/ 2024-03-29T07:56:44.006 INFO:teuthology.orchestra.run.smithi037.stdout: User: admin 2024-03-29T07:56:44.007 INFO:teuthology.orchestra.run.smithi037.stdout: Password: sgiw9965f1 2024-03-29T07:56:44.007 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T07:56:44.007 INFO:teuthology.orchestra.run.smithi037.stdout:Saving cluster configuration to /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/config directory 2024-03-29T07:56:44.007 INFO:teuthology.orchestra.run.smithi037.stdout:Enabling autotune for osd_memory_target 2024-03-29T07:56:44.676 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:56:44 smithi037 ceph-mon[35162]: mgrmap e13: smithi037.wjikii(active, since 6s) 2024-03-29T07:56:45.383 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:44.534+0000 7f8e9b200700 1 Processor -- start 2024-03-29T07:56:45.383 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:44.534+0000 7f8e9b200700 1 -- start start 2024-03-29T07:56:45.383 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:44.535+0000 7f8e9b200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8e94078d60 0x7f8e94077260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T07:56:45.383 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:44.535+0000 7f8e9b200700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e94079170 con 0x7f8e94078d60 2024-03-29T07:56:45.383 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:44.535+0000 7f8e93e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8e94078d60 0x7f8e94077260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T07:56:45.384 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:44.535+0000 7f8e93e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8e94078d60 0x7f8e94077260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46408/0 (socket says 172.21.15.37:46408) 2024-03-29T07:56:45.384 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:44.535+0000 7f8e93e00700 1 -- 172.21.15.37:0/2759943072 learned_addr learned my addr 172.21.15.37:0/2759943072 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T07:56:45.384 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:44.536+0000 7f8e93e00700 1 -- 172.21.15.37:0/2759943072 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e940777a0 con 0x7f8e94078d60 2024-03-29T07:56:45.384 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:44.536+0000 7f8e93e00700 1 --2- 172.21.15.37:0/2759943072 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8e94078d60 0x7f8e94077260 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f8e840098e0 tx=0x7f8e84009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e99a273ce8a75063 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T07:56:45.384 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:44.536+0000 7f8e92a00700 1 -- 172.21.15.37:0/2759943072 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8e8400b860 con 0x7f8e94078d60 2024-03-29T07:56:45.384 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:44.536+0000 7f8e92a00700 1 -- 172.21.15.37:0/2759943072 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f8e8400be70 con 0x7f8e94078d60 2024-03-29T07:56:45.384 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:44.536+0000 7f8e92a00700 1 -- 172.21.15.37:0/2759943072 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8e84010380 con 0x7f8e94078d60 2024-03-29T07:56:45.384 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:44.537+0000 7f8e9b200700 1 -- 172.21.15.37:0/2759943072 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8e94078d60 msgr2=0x7f8e94077260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T07:56:45.384 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:44.537+0000 7f8e9b200700 1 --2- 172.21.15.37:0/2759943072 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8e94078d60 0x7f8e94077260 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f8e840098e0 tx=0x7f8e84009250 comp rx=0 tx=0).stop 2024-03-29T07:56:45.384 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:44.537+0000 7f8e9b200700 1 -- 172.21.15.37:0/2759943072 shutdown_connections 2024-03-29T07:56:45.385 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:44.537+0000 7f8e9b200700 1 --2- 172.21.15.37:0/2759943072 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8e94078d60 0x7f8e94077260 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T07:56:45.385 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:44.537+0000 7f8e9b200700 1 -- 172.21.15.37:0/2759943072 >> 172.21.15.37:0/2759943072 conn(0x7f8e940fdcd0 msgr2=0x7f8e94100100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T07:56:45.385 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:44.537+0000 7f8e9b200700 1 -- 172.21.15.37:0/2759943072 shutdown_connections 2024-03-29T07:56:45.385 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:44.537+0000 7f8e9b200700 1 -- 172.21.15.37:0/2759943072 wait complete. 2024-03-29T07:56:45.385 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:44.538+0000 7f8e9b200700 1 Processor -- start 2024-03-29T07:56:45.385 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:44.538+0000 7f8e9b200700 1 -- start start 2024-03-29T07:56:45.385 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:44.538+0000 7f8e9b200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8e94078d60 0x7f8e94113630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T07:56:45.385 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:44.538+0000 7f8e9b200700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e94113b70 con 0x7f8e94078d60 2024-03-29T07:56:45.386 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:44.538+0000 7f8e93e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8e94078d60 0x7f8e94113630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T07:56:45.386 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:44.538+0000 7f8e93e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8e94078d60 0x7f8e94113630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46422/0 (socket says 172.21.15.37:46422) 2024-03-29T07:56:45.386 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:44.538+0000 7f8e93e00700 1 -- 172.21.15.37:0/1917245681 learned_addr learned my addr 172.21.15.37:0/1917245681 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T07:56:45.386 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:44.539+0000 7f8e93e00700 1 -- 172.21.15.37:0/1917245681 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e840094f0 con 0x7f8e94078d60 2024-03-29T07:56:45.386 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:44.539+0000 7f8e93e00700 1 --2- 172.21.15.37:0/1917245681 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8e94078d60 0x7f8e94113630 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f8e84011b60 tx=0x7f8e84009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T07:56:45.386 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:44.539+0000 7f8e8be00700 1 -- 172.21.15.37:0/1917245681 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8e84010800 con 0x7f8e94078d60 2024-03-29T07:56:45.386 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:44.539+0000 7f8e8be00700 1 -- 172.21.15.37:0/1917245681 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f8e84010e10 con 0x7f8e94078d60 2024-03-29T07:56:45.386 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:44.539+0000 7f8e8be00700 1 -- 172.21.15.37:0/1917245681 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8e8401ae60 con 0x7f8e94078d60 2024-03-29T07:56:45.386 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:44.539+0000 7f8e9b200700 1 -- 172.21.15.37:0/1917245681 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e94113d70 con 0x7f8e94078d60 2024-03-29T07:56:45.387 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:44.540+0000 7f8e9b200700 1 -- 172.21.15.37:0/1917245681 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e941148e0 con 0x7f8e94078d60 2024-03-29T07:56:45.387 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:44.540+0000 7f8e8be00700 1 -- 172.21.15.37:0/1917245681 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 13) v1 ==== 43424+0+0 (secure 0 0 0) 0x7f8e8401a430 con 0x7f8e94078d60 2024-03-29T07:56:45.387 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:44.541+0000 7f8e9b200700 1 -- 172.21.15.37:0/1917245681 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e9405f5f0 con 0x7f8e94078d60 2024-03-29T07:56:45.387 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:44.541+0000 7f8e8be00700 1 --2- 172.21.15.37:0/1917245681 >> [v2:172.21.15.37:6800/4245458578,v1:172.21.15.37:6801/4245458578] conn(0x7f8e78036880 0x7f8e78038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T07:56:45.387 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:44.541+0000 7f8e8be00700 1 -- 172.21.15.37:0/1917245681 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f8e8404a8d0 con 0x7f8e94078d60 2024-03-29T07:56:45.387 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:44.541+0000 7f8e93400700 1 --2- 172.21.15.37:0/1917245681 >> [v2:172.21.15.37:6800/4245458578,v1:172.21.15.37:6801/4245458578] conn(0x7f8e78036880 0x7f8e78038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T07:56:45.387 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:44.542+0000 7f8e93400700 1 --2- 172.21.15.37:0/1917245681 >> [v2:172.21.15.37:6800/4245458578,v1:172.21.15.37:6801/4245458578] conn(0x7f8e78036880 0x7f8e78038d30 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f8e80009990 tx=0x7f8e80006e40 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T07:56:45.387 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:44.546+0000 7f8e8be00700 1 -- 172.21.15.37:0/1917245681 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f8e8402b950 con 0x7f8e94078d60 2024-03-29T07:56:45.388 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:44.751+0000 7f8e9b200700 1 -- 172.21.15.37:0/1917245681 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) v1 -- 0x7f8e9404bed0 con 0x7f8e94078d60 2024-03-29T07:56:45.388 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:44.752+0000 7f8e8be00700 1 -- 172.21.15.37:0/1917245681 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) v1 ==== 127+0+0 (secure 0 0 0) 0x7f8e8402b300 con 0x7f8e94078d60 2024-03-29T07:56:45.388 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:44.754+0000 7f8e9b200700 1 -- 172.21.15.37:0/1917245681 >> [v2:172.21.15.37:6800/4245458578,v1:172.21.15.37:6801/4245458578] conn(0x7f8e78036880 msgr2=0x7f8e78038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T07:56:45.388 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:44.754+0000 7f8e9b200700 1 --2- 172.21.15.37:0/1917245681 >> [v2:172.21.15.37:6800/4245458578,v1:172.21.15.37:6801/4245458578] conn(0x7f8e78036880 0x7f8e78038d30 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f8e80009990 tx=0x7f8e80006e40 comp rx=0 tx=0).stop 2024-03-29T07:56:45.388 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:44.754+0000 7f8e9b200700 1 -- 172.21.15.37:0/1917245681 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8e94078d60 msgr2=0x7f8e94113630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T07:56:45.388 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:44.754+0000 7f8e9b200700 1 --2- 172.21.15.37:0/1917245681 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8e94078d60 0x7f8e94113630 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f8e84011b60 tx=0x7f8e84009250 comp rx=0 tx=0).stop 2024-03-29T07:56:45.388 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:44.754+0000 7f8e9b200700 1 -- 172.21.15.37:0/1917245681 shutdown_connections 2024-03-29T07:56:45.388 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:44.754+0000 7f8e9b200700 1 --2- 172.21.15.37:0/1917245681 >> [v2:172.21.15.37:6800/4245458578,v1:172.21.15.37:6801/4245458578] conn(0x7f8e78036880 0x7f8e78038d30 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T07:56:45.388 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:44.754+0000 7f8e9b200700 1 --2- 172.21.15.37:0/1917245681 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8e94078d60 0x7f8e94113630 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T07:56:45.388 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:44.754+0000 7f8e9b200700 1 -- 172.21.15.37:0/1917245681 >> 172.21.15.37:0/1917245681 conn(0x7f8e940fdcd0 msgr2=0x7f8e940fe8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T07:56:45.389 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:44.754+0000 7f8e9b200700 1 -- 172.21.15.37:0/1917245681 shutdown_connections 2024-03-29T07:56:45.389 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:44.755+0000 7f8e9b200700 1 -- 172.21.15.37:0/1917245681 wait complete. 2024-03-29T07:56:46.805 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:45.905+0000 7f600f400700 1 Processor -- start 2024-03-29T07:56:46.805 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:45.905+0000 7f600f400700 1 -- start start 2024-03-29T07:56:46.805 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:45.906+0000 7f600f400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6008078d60 0x7f6008077260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T07:56:46.805 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:45.906+0000 7f600f400700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6008079170 con 0x7f6008078d60 2024-03-29T07:56:46.805 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:45.906+0000 7f600ca00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6008078d60 0x7f6008077260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T07:56:46.806 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:45.906+0000 7f600ca00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6008078d60 0x7f6008077260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46438/0 (socket says 172.21.15.37:46438) 2024-03-29T07:56:46.806 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:45.906+0000 7f600ca00700 1 -- 172.21.15.37:0/2611540914 learned_addr learned my addr 172.21.15.37:0/2611540914 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T07:56:46.806 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:45.907+0000 7f600ca00700 1 -- 172.21.15.37:0/2611540914 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60080777a0 con 0x7f6008078d60 2024-03-29T07:56:46.806 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:45.907+0000 7f600ca00700 1 --2- 172.21.15.37:0/2611540914 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6008078d60 0x7f6008077260 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f5ffc0098e0 tx=0x7f5ffc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b9f283ded9bc718 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T07:56:46.806 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:45.907+0000 7f6007400700 1 -- 172.21.15.37:0/2611540914 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5ffc00b860 con 0x7f6008078d60 2024-03-29T07:56:46.806 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:45.907+0000 7f6007400700 1 -- 172.21.15.37:0/2611540914 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f5ffc00be70 con 0x7f6008078d60 2024-03-29T07:56:46.806 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:45.907+0000 7f6007400700 1 -- 172.21.15.37:0/2611540914 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5ffc010380 con 0x7f6008078d60 2024-03-29T07:56:46.806 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:45.908+0000 7f600f400700 1 -- 172.21.15.37:0/2611540914 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6008078d60 msgr2=0x7f6008077260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T07:56:46.806 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:45.908+0000 7f600f400700 1 --2- 172.21.15.37:0/2611540914 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6008078d60 0x7f6008077260 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f5ffc0098e0 tx=0x7f5ffc009250 comp rx=0 tx=0).stop 2024-03-29T07:56:46.806 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:45.908+0000 7f600f400700 1 -- 172.21.15.37:0/2611540914 shutdown_connections 2024-03-29T07:56:46.806 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:45.908+0000 7f600f400700 1 --2- 172.21.15.37:0/2611540914 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6008078d60 0x7f6008077260 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T07:56:46.806 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:45.908+0000 7f600f400700 1 -- 172.21.15.37:0/2611540914 >> 172.21.15.37:0/2611540914 conn(0x7f60080fdcd0 msgr2=0x7f6008100100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T07:56:46.806 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:45.908+0000 7f600f400700 1 -- 172.21.15.37:0/2611540914 shutdown_connections 2024-03-29T07:56:46.807 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:45.908+0000 7f600f400700 1 -- 172.21.15.37:0/2611540914 wait complete. 2024-03-29T07:56:46.807 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:45.909+0000 7f600f400700 1 Processor -- start 2024-03-29T07:56:46.807 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:45.909+0000 7f600f400700 1 -- start start 2024-03-29T07:56:46.807 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:45.909+0000 7f600f400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6008078d60 0x7f6008113630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T07:56:46.807 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:45.909+0000 7f600f400700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6008113b70 con 0x7f6008078d60 2024-03-29T07:56:46.807 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:45.909+0000 7f600ca00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6008078d60 0x7f6008113630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T07:56:46.807 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:45.910+0000 7f600ca00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6008078d60 0x7f6008113630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46450/0 (socket says 172.21.15.37:46450) 2024-03-29T07:56:46.807 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:45.910+0000 7f600ca00700 1 -- 172.21.15.37:0/365087369 learned_addr learned my addr 172.21.15.37:0/365087369 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T07:56:46.807 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:45.910+0000 7f600ca00700 1 -- 172.21.15.37:0/365087369 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ffc0094f0 con 0x7f6008078d60 2024-03-29T07:56:46.807 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:45.910+0000 7f600ca00700 1 --2- 172.21.15.37:0/365087369 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6008078d60 0x7f6008113630 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f5ffc011b60 tx=0x7f5ffc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T07:56:46.807 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:45.910+0000 7f6004c00700 1 -- 172.21.15.37:0/365087369 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5ffc010800 con 0x7f6008078d60 2024-03-29T07:56:46.807 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:45.910+0000 7f6004c00700 1 -- 172.21.15.37:0/365087369 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f5ffc010e10 con 0x7f6008078d60 2024-03-29T07:56:46.807 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:45.910+0000 7f600f400700 1 -- 172.21.15.37:0/365087369 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6008113d70 con 0x7f6008078d60 2024-03-29T07:56:46.807 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:45.910+0000 7f6004c00700 1 -- 172.21.15.37:0/365087369 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5ffc01ae60 con 0x7f6008078d60 2024-03-29T07:56:46.808 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:45.911+0000 7f600f400700 1 -- 172.21.15.37:0/365087369 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60081148e0 con 0x7f6008078d60 2024-03-29T07:56:46.808 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:45.912+0000 7f6004c00700 1 -- 172.21.15.37:0/365087369 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 13) v1 ==== 43424+0+0 (secure 0 0 0) 0x7f5ffc01a430 con 0x7f6008078d60 2024-03-29T07:56:46.808 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:45.912+0000 7f600f400700 1 -- 172.21.15.37:0/365087369 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f600805f5f0 con 0x7f6008078d60 2024-03-29T07:56:46.808 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:45.912+0000 7f6004c00700 1 --2- 172.21.15.37:0/365087369 >> [v2:172.21.15.37:6800/4245458578,v1:172.21.15.37:6801/4245458578] conn(0x7f5ff00368d0 0x7f5ff0038d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T07:56:46.808 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:45.912+0000 7f6004c00700 1 -- 172.21.15.37:0/365087369 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f5ffc04a770 con 0x7f6008078d60 2024-03-29T07:56:46.808 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:45.912+0000 7f6007e00700 1 --2- 172.21.15.37:0/365087369 >> [v2:172.21.15.37:6800/4245458578,v1:172.21.15.37:6801/4245458578] conn(0x7f5ff00368d0 0x7f5ff0038d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T07:56:46.808 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:45.913+0000 7f6007e00700 1 --2- 172.21.15.37:0/365087369 >> [v2:172.21.15.37:6800/4245458578,v1:172.21.15.37:6801/4245458578] conn(0x7f5ff00368d0 0x7f5ff0038d80 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f5ff4009990 tx=0x7f5ff4006e40 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T07:56:46.808 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:45.917+0000 7f6004c00700 1 -- 172.21.15.37:0/365087369 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f5ffc02b950 con 0x7f6008078d60 2024-03-29T07:56:46.808 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:46.232+0000 7f600f400700 1 -- 172.21.15.37:0/365087369 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) v1 -- 0x7f600804bed0 con 0x7f6008078d60 2024-03-29T07:56:46.808 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:46.234+0000 7f6004c00700 1 -- 172.21.15.37:0/365087369 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v33)=0 set mgr/dashboard/cluster/status v33) v1 ==== 153+0+0 (secure 0 0 0) 0x7f5ffc02b300 con 0x7f6008078d60 2024-03-29T07:56:46.808 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:46.237+0000 7f600f400700 1 -- 172.21.15.37:0/365087369 >> [v2:172.21.15.37:6800/4245458578,v1:172.21.15.37:6801/4245458578] conn(0x7f5ff00368d0 msgr2=0x7f5ff0038d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T07:56:46.808 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:46.237+0000 7f600f400700 1 --2- 172.21.15.37:0/365087369 >> [v2:172.21.15.37:6800/4245458578,v1:172.21.15.37:6801/4245458578] conn(0x7f5ff00368d0 0x7f5ff0038d80 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f5ff4009990 tx=0x7f5ff4006e40 comp rx=0 tx=0).stop 2024-03-29T07:56:46.808 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:46.237+0000 7f600f400700 1 -- 172.21.15.37:0/365087369 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6008078d60 msgr2=0x7f6008113630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T07:56:46.809 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:46.237+0000 7f600f400700 1 --2- 172.21.15.37:0/365087369 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6008078d60 0x7f6008113630 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f5ffc011b60 tx=0x7f5ffc009250 comp rx=0 tx=0).stop 2024-03-29T07:56:46.809 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:46.237+0000 7f600f400700 1 -- 172.21.15.37:0/365087369 shutdown_connections 2024-03-29T07:56:46.809 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:46.237+0000 7f600f400700 1 --2- 172.21.15.37:0/365087369 >> [v2:172.21.15.37:6800/4245458578,v1:172.21.15.37:6801/4245458578] conn(0x7f5ff00368d0 0x7f5ff0038d80 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T07:56:46.809 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:46.237+0000 7f600f400700 1 --2- 172.21.15.37:0/365087369 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6008078d60 0x7f6008113630 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T07:56:46.809 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:46.237+0000 7f600f400700 1 -- 172.21.15.37:0/365087369 >> 172.21.15.37:0/365087369 conn(0x7f60080fdcd0 msgr2=0x7f60080fe8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T07:56:46.809 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:46.237+0000 7f600f400700 1 -- 172.21.15.37:0/365087369 shutdown_connections 2024-03-29T07:56:46.809 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-03-29T07:56:46.237+0000 7f600f400700 1 -- 172.21.15.37:0/365087369 wait complete. 2024-03-29T07:56:46.809 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-03-29T07:56:46.809 INFO:teuthology.orchestra.run.smithi037.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-03-29T07:56:46.809 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T07:56:46.809 INFO:teuthology.orchestra.run.smithi037.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 99b492c2-eda1-11ee-b647-cb9ed24678a4 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-03-29T07:56:46.809 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T07:56:46.809 INFO:teuthology.orchestra.run.smithi037.stdout:Or, if you are only running a single cluster on this host: 2024-03-29T07:56:46.809 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T07:56:46.809 INFO:teuthology.orchestra.run.smithi037.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-03-29T07:56:46.810 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T07:56:46.810 INFO:teuthology.orchestra.run.smithi037.stdout:Please consider enabling telemetry to help improve Ceph: 2024-03-29T07:56:46.810 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T07:56:46.810 INFO:teuthology.orchestra.run.smithi037.stdout: ceph telemetry on 2024-03-29T07:56:46.810 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T07:56:46.810 INFO:teuthology.orchestra.run.smithi037.stdout:For more information see: 2024-03-29T07:56:46.810 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T07:56:46.810 INFO:teuthology.orchestra.run.smithi037.stdout: https://docs.ceph.com/docs/master/mgr/telemetry/ 2024-03-29T07:56:46.810 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T07:56:46.810 INFO:teuthology.orchestra.run.smithi037.stdout:Bootstrap complete. 2024-03-29T07:56:46.862 INFO:tasks.cephadm:Fetching config... 2024-03-29T07:56:46.862 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-03-29T07:56:46.862 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-03-29T07:56:46.908 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-03-29T07:56:46.908 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-03-29T07:56:46.908 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-03-29T07:56:46.994 INFO:tasks.cephadm:Fetching mon keyring... 2024-03-29T07:56:46.994 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-03-29T07:56:46.994 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd if=/var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/mon.smithi037/keyring of=/dev/stdout 2024-03-29T07:56:47.102 INFO:tasks.cephadm:Fetching pub ssh key... 2024-03-29T07:56:47.103 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-03-29T07:56:47.103 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-03-29T07:56:47.190 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-03-29T07:56:47.191 DEBUG:teuthology.orchestra.run.smithi037:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC2IjzXulokTgrTwcqS4Akc/Oice5q1DQRyhlC1Y6Eh93wYQdIdT65GOg73DPvyNhwNDOWWV6sV1PZVslqN3V5Gg4kKN3+oDCwZh5p478jH4YVYXjp94jwF3DLXXwarkwGxjWNbMrwdHDj5qAkHCKS6IXPXrprmhcvAaiMBJpdJkA7ucLRPDFGz3eidl98dHBCy3gYINCaei0Xpdob5F+QekicTiM9yllvoJ12WxUTmdV87GAU//Qi7rDe+ni4/UBfU+8W6twmqWHjpT4Ci3v2yfW/+23hMTFYhfRJXWdDrKsRxVxNlRc8unu9Eo5h/LWXf8H56qDug6eiJ2TyeWxVCBcY6LjkE99J4gsx8v/Qgrz0vGBJk2rUFAgA32xIz9LsDf+RaEP267Lr7aYc0Ic+jSicSqqWiB7ryrf9wQBtXIBYK4hqWF/fZ7gptMsoX21HjmJsOWtfQZ1N1wOT9YfxRxhvLqijb5gI98ORtc6PdEseM7+AFw4DQROewLHl0EM8= ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-03-29T07:56:47.325 INFO:teuthology.orchestra.run.smithi037.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC2IjzXulokTgrTwcqS4Akc/Oice5q1DQRyhlC1Y6Eh93wYQdIdT65GOg73DPvyNhwNDOWWV6sV1PZVslqN3V5Gg4kKN3+oDCwZh5p478jH4YVYXjp94jwF3DLXXwarkwGxjWNbMrwdHDj5qAkHCKS6IXPXrprmhcvAaiMBJpdJkA7ucLRPDFGz3eidl98dHBCy3gYINCaei0Xpdob5F+QekicTiM9yllvoJ12WxUTmdV87GAU//Qi7rDe+ni4/UBfU+8W6twmqWHjpT4Ci3v2yfW/+23hMTFYhfRJXWdDrKsRxVxNlRc8unu9Eo5h/LWXf8H56qDug6eiJ2TyeWxVCBcY6LjkE99J4gsx8v/Qgrz0vGBJk2rUFAgA32xIz9LsDf+RaEP267Lr7aYc0Ic+jSicSqqWiB7ryrf9wQBtXIBYK4hqWF/fZ7gptMsoX21HjmJsOWtfQZ1N1wOT9YfxRxhvLqijb5gI98ORtc6PdEseM7+AFw4DQROewLHl0EM8= ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4 2024-03-29T07:56:47.345 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:56:47 smithi037 ceph-mon[35162]: from='client.? 172.21.15.37:0/365087369' entity='client.admin' 2024-03-29T07:56:47.353 DEBUG:teuthology.orchestra.run.smithi148:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC2IjzXulokTgrTwcqS4Akc/Oice5q1DQRyhlC1Y6Eh93wYQdIdT65GOg73DPvyNhwNDOWWV6sV1PZVslqN3V5Gg4kKN3+oDCwZh5p478jH4YVYXjp94jwF3DLXXwarkwGxjWNbMrwdHDj5qAkHCKS6IXPXrprmhcvAaiMBJpdJkA7ucLRPDFGz3eidl98dHBCy3gYINCaei0Xpdob5F+QekicTiM9yllvoJ12WxUTmdV87GAU//Qi7rDe+ni4/UBfU+8W6twmqWHjpT4Ci3v2yfW/+23hMTFYhfRJXWdDrKsRxVxNlRc8unu9Eo5h/LWXf8H56qDug6eiJ2TyeWxVCBcY6LjkE99J4gsx8v/Qgrz0vGBJk2rUFAgA32xIz9LsDf+RaEP267Lr7aYc0Ic+jSicSqqWiB7ryrf9wQBtXIBYK4hqWF/fZ7gptMsoX21HjmJsOWtfQZ1N1wOT9YfxRxhvLqijb5gI98ORtc6PdEseM7+AFw4DQROewLHl0EM8= ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-03-29T07:56:47.451 INFO:teuthology.orchestra.run.smithi148.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC2IjzXulokTgrTwcqS4Akc/Oice5q1DQRyhlC1Y6Eh93wYQdIdT65GOg73DPvyNhwNDOWWV6sV1PZVslqN3V5Gg4kKN3+oDCwZh5p478jH4YVYXjp94jwF3DLXXwarkwGxjWNbMrwdHDj5qAkHCKS6IXPXrprmhcvAaiMBJpdJkA7ucLRPDFGz3eidl98dHBCy3gYINCaei0Xpdob5F+QekicTiM9yllvoJ12WxUTmdV87GAU//Qi7rDe+ni4/UBfU+8W6twmqWHjpT4Ci3v2yfW/+23hMTFYhfRJXWdDrKsRxVxNlRc8unu9Eo5h/LWXf8H56qDug6eiJ2TyeWxVCBcY6LjkE99J4gsx8v/Qgrz0vGBJk2rUFAgA32xIz9LsDf+RaEP267Lr7aYc0Ic+jSicSqqWiB7ryrf9wQBtXIBYK4hqWF/fZ7gptMsoX21HjmJsOWtfQZ1N1wOT9YfxRxhvLqijb5gI98ORtc6PdEseM7+AFw4DQROewLHl0EM8= ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4 2024-03-29T07:56:47.478 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 99b492c2-eda1-11ee-b647-cb9ed24678a4 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-03-29T07:56:47.906 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/mon.smithi037/config 2024-03-29T07:56:50.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:56:50.350+0000 7fefe1400700 1 -- 172.21.15.37:0/2703007503 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fefdc071ea0 msgr2=0x7fefdc0703b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T07:56:50.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:56:50.350+0000 7fefe1400700 1 --2- 172.21.15.37:0/2703007503 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fefdc071ea0 0x7fefdc0703b0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fefcc0048c0 tx=0x7fefcc009e30 comp rx=0 tx=0).stop 2024-03-29T07:56:50.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:56:50.350+0000 7fefe1400700 1 -- 172.21.15.37:0/2703007503 shutdown_connections 2024-03-29T07:56:50.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:56:50.350+0000 7fefe1400700 1 --2- 172.21.15.37:0/2703007503 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fefdc071ea0 0x7fefdc0703b0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T07:56:50.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:56:50.350+0000 7fefe1400700 1 -- 172.21.15.37:0/2703007503 >> 172.21.15.37:0/2703007503 conn(0x7fefdc0f81c0 msgr2=0x7fefdc0fa5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T07:56:50.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:56:50.350+0000 7fefe1400700 1 -- 172.21.15.37:0/2703007503 shutdown_connections 2024-03-29T07:56:50.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:56:50.350+0000 7fefe1400700 1 -- 172.21.15.37:0/2703007503 wait complete. 2024-03-29T07:56:50.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:56:50.351+0000 7fefe1400700 1 Processor -- start 2024-03-29T07:56:50.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:56:50.351+0000 7fefe1400700 1 -- start start 2024-03-29T07:56:50.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:56:50.352+0000 7fefe1400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fefdc071ea0 0x7fefdc113600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T07:56:50.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:56:50.352+0000 7fefe1400700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefdc113b40 con 0x7fefdc071ea0 2024-03-29T07:56:50.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:56:50.352+0000 7fefda800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fefdc071ea0 0x7fefdc113600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T07:56:50.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:56:50.352+0000 7fefda800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fefdc071ea0 0x7fefdc113600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54500/0 (socket says 172.21.15.37:54500) 2024-03-29T07:56:50.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:56:50.352+0000 7fefda800700 1 -- 172.21.15.37:0/4054983208 learned_addr learned my addr 172.21.15.37:0/4054983208 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T07:56:50.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:56:50.352+0000 7fefda800700 1 -- 172.21.15.37:0/4054983208 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefcc005040 con 0x7fefdc071ea0 2024-03-29T07:56:50.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:56:50.352+0000 7fefda800700 1 --2- 172.21.15.37:0/4054983208 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fefdc071ea0 0x7fefdc113600 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fefcc00b860 tx=0x7fefcc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T07:56:50.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:56:50.353+0000 7fefd2a00700 1 -- 172.21.15.37:0/4054983208 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fefcc009580 con 0x7fefdc071ea0 2024-03-29T07:56:50.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:56:50.353+0000 7fefd2a00700 1 -- 172.21.15.37:0/4054983208 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fefcc0096e0 con 0x7fefdc071ea0 2024-03-29T07:56:50.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:56:50.353+0000 7fefd2a00700 1 -- 172.21.15.37:0/4054983208 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fefcc01c5b0 con 0x7fefdc071ea0 2024-03-29T07:56:50.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:56:50.353+0000 7fefe1400700 1 -- 172.21.15.37:0/4054983208 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefdc113d40 con 0x7fefdc071ea0 2024-03-29T07:56:50.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:56:50.353+0000 7fefe1400700 1 -- 172.21.15.37:0/4054983208 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefdc1148b0 con 0x7fefdc071ea0 2024-03-29T07:56:50.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:56:50.354+0000 7fefd2a00700 1 -- 172.21.15.37:0/4054983208 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 13) v1 ==== 43424+0+0 (secure 0 0 0) 0x7fefcc013420 con 0x7fefdc071ea0 2024-03-29T07:56:50.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:56:50.354+0000 7fefd2a00700 1 --2- 172.21.15.37:0/4054983208 >> [v2:172.21.15.37:6800/4245458578,v1:172.21.15.37:6801/4245458578] conn(0x7fefc00368d0 0x7fefc0038d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T07:56:50.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:56:50.354+0000 7fefd2a00700 1 -- 172.21.15.37:0/4054983208 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7fefcc049ba0 con 0x7fefdc071ea0 2024-03-29T07:56:50.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:56:50.354+0000 7fefe1400700 1 -- 172.21.15.37:0/4054983208 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fefdc1094b0 con 0x7fefdc071ea0 2024-03-29T07:56:50.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:56:50.355+0000 7fefd9e00700 1 --2- 172.21.15.37:0/4054983208 >> [v2:172.21.15.37:6800/4245458578,v1:172.21.15.37:6801/4245458578] conn(0x7fefc00368d0 0x7fefc0038d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T07:56:50.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:56:50.355+0000 7fefd9e00700 1 --2- 172.21.15.37:0/4054983208 >> [v2:172.21.15.37:6800/4245458578,v1:172.21.15.37:6801/4245458578] conn(0x7fefc00368d0 0x7fefc0038d80 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fefc8009990 tx=0x7fefc8006e40 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T07:56:50.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:56:50.364+0000 7fefd2a00700 1 -- 172.21.15.37:0/4054983208 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fefcc013770 con 0x7fefdc071ea0 2024-03-29T07:56:50.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:56:50.578+0000 7fefe1400700 1 -- 172.21.15.37:0/4054983208 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) v1 -- 0x7fefdc05f5f0 con 0x7fefdc071ea0 2024-03-29T07:56:50.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:56:50.582+0000 7fefd2a00700 1 -- 172.21.15.37:0/4054983208 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) v1 ==== 125+0+0 (secure 0 0 0) 0x7fefdc05f5f0 con 0x7fefdc071ea0 2024-03-29T07:56:50.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:56:50.584+0000 7fefe1400700 1 -- 172.21.15.37:0/4054983208 >> [v2:172.21.15.37:6800/4245458578,v1:172.21.15.37:6801/4245458578] conn(0x7fefc00368d0 msgr2=0x7fefc0038d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T07:56:50.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:56:50.584+0000 7fefe1400700 1 --2- 172.21.15.37:0/4054983208 >> [v2:172.21.15.37:6800/4245458578,v1:172.21.15.37:6801/4245458578] conn(0x7fefc00368d0 0x7fefc0038d80 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fefc8009990 tx=0x7fefc8006e40 comp rx=0 tx=0).stop 2024-03-29T07:56:50.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:56:50.584+0000 7fefe1400700 1 -- 172.21.15.37:0/4054983208 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fefdc071ea0 msgr2=0x7fefdc113600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T07:56:50.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:56:50.584+0000 7fefe1400700 1 --2- 172.21.15.37:0/4054983208 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fefdc071ea0 0x7fefdc113600 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fefcc00b860 tx=0x7fefcc009e30 comp rx=0 tx=0).stop 2024-03-29T07:56:50.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:56:50.584+0000 7fefe1400700 1 -- 172.21.15.37:0/4054983208 shutdown_connections 2024-03-29T07:56:50.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:56:50.584+0000 7fefe1400700 1 --2- 172.21.15.37:0/4054983208 >> [v2:172.21.15.37:6800/4245458578,v1:172.21.15.37:6801/4245458578] conn(0x7fefc00368d0 0x7fefc0038d80 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T07:56:50.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:56:50.584+0000 7fefe1400700 1 --2- 172.21.15.37:0/4054983208 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fefdc071ea0 0x7fefdc113600 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T07:56:50.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:56:50.584+0000 7fefe1400700 1 -- 172.21.15.37:0/4054983208 >> 172.21.15.37:0/4054983208 conn(0x7fefdc0f81c0 msgr2=0x7fefdc0f8d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T07:56:50.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:56:50.585+0000 7fefe1400700 1 -- 172.21.15.37:0/4054983208 shutdown_connections 2024-03-29T07:56:50.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:56:50.585+0000 7fefe1400700 1 -- 172.21.15.37:0/4054983208 wait complete. 2024-03-29T07:56:50.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:56:50 smithi037 ceph-mon[35162]: from='mgr.14160 172.21.15.37:0/1836651950' entity='mgr.smithi037.wjikii' 2024-03-29T07:56:50.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:56:50 smithi037 ceph-mon[35162]: from='mgr.14160 172.21.15.37:0/1836651950' entity='mgr.smithi037.wjikii' 2024-03-29T07:56:50.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:56:50 smithi037 ceph-mon[35162]: from='mgr.14160 172.21.15.37:0/1836651950' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config rm", "who": "osd/host:smithi037", "name": "osd_memory_target"}]: dispatch 2024-03-29T07:56:50.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:56:50 smithi037 ceph-mon[35162]: from='mgr.14160 172.21.15.37:0/1836651950' entity='mgr.smithi037.wjikii' 2024-03-29T07:56:50.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:56:50 smithi037 ceph-mon[35162]: from='mgr.14160 172.21.15.37:0/1836651950' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi037", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-29T07:56:50.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:56:50 smithi037 ceph-mon[35162]: from='mgr.14160 172.21.15.37:0/1836651950' entity='mgr.smithi037.wjikii' cmd='[{"prefix": "auth get-or-create", "entity": "client.crash.smithi037", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]': finished 2024-03-29T07:56:50.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:56:50 smithi037 ceph-mon[35162]: from='mgr.14160 172.21.15.37:0/1836651950' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T07:56:50.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:56:50 smithi037 ceph-mon[35162]: Deploying daemon crash.smithi037 on smithi037 2024-03-29T07:56:50.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:56:50 smithi037 ceph-mon[35162]: from='client.? 172.21.15.37:0/4054983208' entity='client.admin' 2024-03-29T07:56:51.349 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-03-29T07:56:51.349 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 99b492c2-eda1-11ee-b647-cb9ed24678a4 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-03-29T07:56:51.945 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/mon.smithi037/config 2024-03-29T07:56:55.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:56:55.155+0000 7f19e4200700 1 -- 172.21.15.37:0/1271817192 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f19dc071e40 msgr2=0x7f19dc070350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T07:56:55.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:56:55.155+0000 7f19e4200700 1 --2- 172.21.15.37:0/1271817192 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f19dc071e40 0x7f19dc070350 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f19d00048c0 tx=0x7f19d0009e30 comp rx=0 tx=0).stop 2024-03-29T07:56:55.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:56:55.155+0000 7f19e4200700 1 -- 172.21.15.37:0/1271817192 shutdown_connections 2024-03-29T07:56:55.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:56:55.155+0000 7f19e4200700 1 --2- 172.21.15.37:0/1271817192 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f19dc071e40 0x7f19dc070350 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T07:56:55.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:56:55.155+0000 7f19e4200700 1 -- 172.21.15.37:0/1271817192 >> 172.21.15.37:0/1271817192 conn(0x7f19dc0f81c0 msgr2=0x7f19dc0fa5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T07:56:55.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:56:55.156+0000 7f19e4200700 1 -- 172.21.15.37:0/1271817192 shutdown_connections 2024-03-29T07:56:55.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:56:55.156+0000 7f19e4200700 1 -- 172.21.15.37:0/1271817192 wait complete. 2024-03-29T07:56:55.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:56:55.156+0000 7f19e4200700 1 Processor -- start 2024-03-29T07:56:55.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:56:55.157+0000 7f19e4200700 1 -- start start 2024-03-29T07:56:55.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:56:55.157+0000 7f19e4200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f19dc071e40 0x7f19dc10f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T07:56:55.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:56:55.157+0000 7f19e4200700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19dc10f720 con 0x7f19dc071e40 2024-03-29T07:56:55.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:56:55.157+0000 7f19e1800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f19dc071e40 0x7f19dc10f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T07:56:55.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:56:55.157+0000 7f19e1800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f19dc071e40 0x7f19dc10f1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54516/0 (socket says 172.21.15.37:54516) 2024-03-29T07:56:55.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:56:55.157+0000 7f19e1800700 1 -- 172.21.15.37:0/1338165340 learned_addr learned my addr 172.21.15.37:0/1338165340 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T07:56:55.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:56:55.158+0000 7f19e1800700 1 -- 172.21.15.37:0/1338165340 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19d0005040 con 0x7f19dc071e40 2024-03-29T07:56:55.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:56:55.158+0000 7f19e1800700 1 --2- 172.21.15.37:0/1338165340 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f19dc071e40 0x7f19dc10f1e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f19d000b860 tx=0x7f19d0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T07:56:55.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:56:55.158+0000 7f19d9600700 1 -- 172.21.15.37:0/1338165340 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f19d0009580 con 0x7f19dc071e40 2024-03-29T07:56:55.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:56:55.158+0000 7f19d9600700 1 -- 172.21.15.37:0/1338165340 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f19d00096e0 con 0x7f19dc071e40 2024-03-29T07:56:55.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:56:55.158+0000 7f19d9600700 1 -- 172.21.15.37:0/1338165340 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f19d001c5b0 con 0x7f19dc071e40 2024-03-29T07:56:55.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:56:55.158+0000 7f19e4200700 1 -- 172.21.15.37:0/1338165340 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19dc10f920 con 0x7f19dc071e40 2024-03-29T07:56:55.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:56:55.158+0000 7f19e4200700 1 -- 172.21.15.37:0/1338165340 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19dc110490 con 0x7f19dc071e40 2024-03-29T07:56:55.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:56:55.159+0000 7f19d9600700 1 -- 172.21.15.37:0/1338165340 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 13) v1 ==== 43424+0+0 (secure 0 0 0) 0x7f19d001c820 con 0x7f19dc071e40 2024-03-29T07:56:55.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:56:55.159+0000 7f19e4200700 1 -- 172.21.15.37:0/1338165340 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19dc105020 con 0x7f19dc071e40 2024-03-29T07:56:55.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:56:55.160+0000 7f19d9600700 1 --2- 172.21.15.37:0/1338165340 >> [v2:172.21.15.37:6800/4245458578,v1:172.21.15.37:6801/4245458578] conn(0x7f19c4036880 0x7f19c4038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T07:56:55.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:56:55.160+0000 7f19d9600700 1 -- 172.21.15.37:0/1338165340 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f19d0049ae0 con 0x7f19dc071e40 2024-03-29T07:56:55.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:56:55.160+0000 7f19e0e00700 1 --2- 172.21.15.37:0/1338165340 >> [v2:172.21.15.37:6800/4245458578,v1:172.21.15.37:6801/4245458578] conn(0x7f19c4036880 0x7f19c4038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T07:56:55.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:56:55.161+0000 7f19e0e00700 1 --2- 172.21.15.37:0/1338165340 >> [v2:172.21.15.37:6800/4245458578,v1:172.21.15.37:6801/4245458578] conn(0x7f19c4036880 0x7f19c4038d30 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f19c8009990 tx=0x7f19c8006e40 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T07:56:55.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:56:55.165+0000 7f19d9600700 1 -- 172.21.15.37:0/1338165340 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f19d0023300 con 0x7f19dc071e40 2024-03-29T07:56:55.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:56:55.392+0000 7f19e4200700 1 -- 172.21.15.37:0/1338165340 --> [v2:172.21.15.37:6800/4245458578,v1:172.21.15.37:6801/4245458578] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) v1 -- 0x7f19dc1106d0 con 0x7f19c4036880 2024-03-29T07:56:55.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:56:55.396+0000 7f19d9600700 1 -- 172.21.15.37:0/1338165340 <== mgr.14160 v2:172.21.15.37:6800/4245458578 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f19dc1106d0 con 0x7f19c4036880 2024-03-29T07:56:55.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:56:55.399+0000 7f19e4200700 1 -- 172.21.15.37:0/1338165340 >> [v2:172.21.15.37:6800/4245458578,v1:172.21.15.37:6801/4245458578] conn(0x7f19c4036880 msgr2=0x7f19c4038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T07:56:55.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:56:55.399+0000 7f19e4200700 1 --2- 172.21.15.37:0/1338165340 >> [v2:172.21.15.37:6800/4245458578,v1:172.21.15.37:6801/4245458578] conn(0x7f19c4036880 0x7f19c4038d30 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f19c8009990 tx=0x7f19c8006e40 comp rx=0 tx=0).stop 2024-03-29T07:56:55.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:56:55.399+0000 7f19e4200700 1 -- 172.21.15.37:0/1338165340 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f19dc071e40 msgr2=0x7f19dc10f1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T07:56:55.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:56:55.399+0000 7f19e4200700 1 --2- 172.21.15.37:0/1338165340 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f19dc071e40 0x7f19dc10f1e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f19d000b860 tx=0x7f19d0009e30 comp rx=0 tx=0).stop 2024-03-29T07:56:55.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:56:55.399+0000 7f19e4200700 1 -- 172.21.15.37:0/1338165340 shutdown_connections 2024-03-29T07:56:55.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:56:55.399+0000 7f19e4200700 1 --2- 172.21.15.37:0/1338165340 >> [v2:172.21.15.37:6800/4245458578,v1:172.21.15.37:6801/4245458578] conn(0x7f19c4036880 0x7f19c4038d30 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T07:56:55.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:56:55.399+0000 7f19e4200700 1 --2- 172.21.15.37:0/1338165340 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f19dc071e40 0x7f19dc10f1e0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T07:56:55.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:56:55.399+0000 7f19e4200700 1 -- 172.21.15.37:0/1338165340 >> 172.21.15.37:0/1338165340 conn(0x7f19dc0f81c0 msgr2=0x7f19dc0f89f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T07:56:55.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:56:55.399+0000 7f19e4200700 1 -- 172.21.15.37:0/1338165340 shutdown_connections 2024-03-29T07:56:55.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:56:55.399+0000 7f19e4200700 1 -- 172.21.15.37:0/1338165340 wait complete. 2024-03-29T07:56:56.477 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:56:56 smithi037 ceph-mon[35162]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T07:56:56.478 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:56:56 smithi037 ceph-mon[35162]: from='mgr.14160 172.21.15.37:0/1836651950' entity='mgr.smithi037.wjikii' 2024-03-29T07:56:56.478 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:56:56 smithi037 ceph-mon[35162]: from='mgr.14160 172.21.15.37:0/1836651950' entity='mgr.smithi037.wjikii' 2024-03-29T07:56:56.478 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:56:56 smithi037 ceph-mon[35162]: from='mgr.14160 172.21.15.37:0/1836651950' entity='mgr.smithi037.wjikii' 2024-03-29T07:56:56.478 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:56:56 smithi037 ceph-mon[35162]: from='mgr.14160 172.21.15.37:0/1836651950' entity='mgr.smithi037.wjikii' 2024-03-29T07:56:56.478 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:56:56 smithi037 ceph-mon[35162]: from='mgr.14160 172.21.15.37:0/1836651950' entity='mgr.smithi037.wjikii' 2024-03-29T07:56:56.480 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi148 2024-03-29T07:56:56.480 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-03-29T07:56:56.480 DEBUG:teuthology.orchestra.run.smithi148:> dd of=/etc/ceph/ceph.conf 2024-03-29T07:56:56.529 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-03-29T07:56:56.529 DEBUG:teuthology.orchestra.run.smithi148:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-29T07:56:56.616 INFO:tasks.cephadm:Adding host smithi148 to orchestrator... 2024-03-29T07:56:56.616 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 99b492c2-eda1-11ee-b647-cb9ed24678a4 -- ceph orch host add smithi148 2024-03-29T07:56:57.071 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/mon.smithi037/config 2024-03-29T07:56:57.910 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:56:57 smithi037 ceph-mon[35162]: Deploying daemon node-exporter.smithi037 on smithi037 2024-03-29T07:56:57.910 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:56:57 smithi037 ceph-mon[35162]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-29T07:56:57.910 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:56:57 smithi037 ceph-mon[35162]: from='mgr.14160 172.21.15.37:0/1836651950' entity='mgr.smithi037.wjikii' 2024-03-29T07:56:58.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:56:58.876+0000 7f3c58a00700 1 -- 172.21.15.37:0/2464079895 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3c54071ea0 msgr2=0x7f3c540703b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T07:56:58.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:56:58.876+0000 7f3c58a00700 1 --2- 172.21.15.37:0/2464079895 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3c54071ea0 0x7f3c540703b0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f3c440048c0 tx=0x7f3c44009e30 comp rx=0 tx=0).stop 2024-03-29T07:56:58.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:56:58.876+0000 7f3c58a00700 1 -- 172.21.15.37:0/2464079895 shutdown_connections 2024-03-29T07:56:58.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:56:58.876+0000 7f3c58a00700 1 --2- 172.21.15.37:0/2464079895 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3c54071ea0 0x7f3c540703b0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T07:56:58.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:56:58.876+0000 7f3c58a00700 1 -- 172.21.15.37:0/2464079895 >> 172.21.15.37:0/2464079895 conn(0x7f3c540f81c0 msgr2=0x7f3c540fa5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T07:56:58.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:56:58.876+0000 7f3c58a00700 1 -- 172.21.15.37:0/2464079895 shutdown_connections 2024-03-29T07:56:58.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:56:58.876+0000 7f3c58a00700 1 -- 172.21.15.37:0/2464079895 wait complete. 2024-03-29T07:56:58.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:56:58.877+0000 7f3c58a00700 1 Processor -- start 2024-03-29T07:56:58.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:56:58.877+0000 7f3c58a00700 1 -- start start 2024-03-29T07:56:58.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:56:58.877+0000 7f3c58a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3c54071ea0 0x7f3c5410f200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T07:56:58.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:56:58.877+0000 7f3c58a00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c5410f740 con 0x7f3c54071ea0 2024-03-29T07:56:58.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:56:58.877+0000 7f3c51e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3c54071ea0 0x7f3c5410f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T07:56:58.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:56:58.877+0000 7f3c51e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3c54071ea0 0x7f3c5410f200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54538/0 (socket says 172.21.15.37:54538) 2024-03-29T07:56:58.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:56:58.877+0000 7f3c51e00700 1 -- 172.21.15.37:0/1522485767 learned_addr learned my addr 172.21.15.37:0/1522485767 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T07:56:58.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:56:58.878+0000 7f3c51e00700 1 -- 172.21.15.37:0/1522485767 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c44005040 con 0x7f3c54071ea0 2024-03-29T07:56:58.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:56:58.878+0000 7f3c51e00700 1 --2- 172.21.15.37:0/1522485767 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3c54071ea0 0x7f3c5410f200 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f3c4400b860 tx=0x7f3c44009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T07:56:58.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:56:58.878+0000 7f3c4a000700 1 -- 172.21.15.37:0/1522485767 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3c44009580 con 0x7f3c54071ea0 2024-03-29T07:56:58.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:56:58.878+0000 7f3c4a000700 1 -- 172.21.15.37:0/1522485767 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f3c440096e0 con 0x7f3c54071ea0 2024-03-29T07:56:58.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:56:58.878+0000 7f3c4a000700 1 -- 172.21.15.37:0/1522485767 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3c4401c5b0 con 0x7f3c54071ea0 2024-03-29T07:56:58.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:56:58.878+0000 7f3c58a00700 1 -- 172.21.15.37:0/1522485767 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c5410f940 con 0x7f3c54071ea0 2024-03-29T07:56:58.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:56:58.879+0000 7f3c58a00700 1 -- 172.21.15.37:0/1522485767 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c541104b0 con 0x7f3c54071ea0 2024-03-29T07:56:58.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:56:58.880+0000 7f3c4a000700 1 -- 172.21.15.37:0/1522485767 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 13) v1 ==== 43424+0+0 (secure 0 0 0) 0x7f3c44013420 con 0x7f3c54071ea0 2024-03-29T07:56:58.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:56:58.880+0000 7f3c58a00700 1 -- 172.21.15.37:0/1522485767 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c54105040 con 0x7f3c54071ea0 2024-03-29T07:56:58.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:56:58.880+0000 7f3c4a000700 1 --2- 172.21.15.37:0/1522485767 >> [v2:172.21.15.37:6800/4245458578,v1:172.21.15.37:6801/4245458578] conn(0x7f3c380368d0 0x7f3c38038d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T07:56:58.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:56:58.880+0000 7f3c4a000700 1 -- 172.21.15.37:0/1522485767 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f3c440499f0 con 0x7f3c54071ea0 2024-03-29T07:56:58.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:56:58.880+0000 7f3c51400700 1 --2- 172.21.15.37:0/1522485767 >> [v2:172.21.15.37:6800/4245458578,v1:172.21.15.37:6801/4245458578] conn(0x7f3c380368d0 0x7f3c38038d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T07:56:58.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:56:58.881+0000 7f3c51400700 1 --2- 172.21.15.37:0/1522485767 >> [v2:172.21.15.37:6800/4245458578,v1:172.21.15.37:6801/4245458578] conn(0x7f3c380368d0 0x7f3c38038d80 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f3c40009990 tx=0x7f3c40006e40 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T07:56:58.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:56:58.885+0000 7f3c4a000700 1 -- 172.21.15.37:0/1522485767 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f3c44013960 con 0x7f3c54071ea0 2024-03-29T07:56:59.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:56:59.121+0000 7f3c58a00700 1 -- 172.21.15.37:0/1522485767 --> [v2:172.21.15.37:6800/4245458578,v1:172.21.15.37:6801/4245458578] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi148", "target": ["mon-mgr", ""]}) v1 -- 0x7f3c541106f0 con 0x7f3c380368d0 2024-03-29T07:57:00.176 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:56:59 smithi037 ceph-mon[35162]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-29T07:57:00.176 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:56:59 smithi037 ceph-mon[35162]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi148", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T07:57:01.048 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:57:00 smithi037 ceph-mon[35162]: Deploying cephadm binary to smithi148 2024-03-29T07:57:01.858 INFO:teuthology.orchestra.run.smithi037.stdout:Added host 'smithi148' with addr '172.21.15.148' 2024-03-29T07:57:01.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:57:01.859+0000 7f3c4a000700 1 -- 172.21.15.37:0/1522485767 <== mgr.14160 v2:172.21.15.37:6800/4245458578 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f3c541106f0 con 0x7f3c380368d0 2024-03-29T07:57:01.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:57:01.861+0000 7f3c58a00700 1 -- 172.21.15.37:0/1522485767 >> [v2:172.21.15.37:6800/4245458578,v1:172.21.15.37:6801/4245458578] conn(0x7f3c380368d0 msgr2=0x7f3c38038d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T07:57:01.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:57:01.861+0000 7f3c58a00700 1 --2- 172.21.15.37:0/1522485767 >> [v2:172.21.15.37:6800/4245458578,v1:172.21.15.37:6801/4245458578] conn(0x7f3c380368d0 0x7f3c38038d80 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f3c40009990 tx=0x7f3c40006e40 comp rx=0 tx=0).stop 2024-03-29T07:57:01.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:57:01.861+0000 7f3c58a00700 1 -- 172.21.15.37:0/1522485767 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3c54071ea0 msgr2=0x7f3c5410f200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T07:57:01.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:57:01.861+0000 7f3c58a00700 1 --2- 172.21.15.37:0/1522485767 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3c54071ea0 0x7f3c5410f200 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f3c4400b860 tx=0x7f3c44009e30 comp rx=0 tx=0).stop 2024-03-29T07:57:01.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:57:01.861+0000 7f3c58a00700 1 -- 172.21.15.37:0/1522485767 shutdown_connections 2024-03-29T07:57:01.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:57:01.861+0000 7f3c58a00700 1 --2- 172.21.15.37:0/1522485767 >> [v2:172.21.15.37:6800/4245458578,v1:172.21.15.37:6801/4245458578] conn(0x7f3c380368d0 0x7f3c38038d80 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T07:57:01.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:57:01.861+0000 7f3c58a00700 1 --2- 172.21.15.37:0/1522485767 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3c54071ea0 0x7f3c5410f200 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T07:57:01.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:57:01.861+0000 7f3c58a00700 1 -- 172.21.15.37:0/1522485767 >> 172.21.15.37:0/1522485767 conn(0x7f3c540f81c0 msgr2=0x7f3c540f8aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T07:57:01.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:57:01.862+0000 7f3c58a00700 1 -- 172.21.15.37:0/1522485767 shutdown_connections 2024-03-29T07:57:01.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:57:01.862+0000 7f3c58a00700 1 -- 172.21.15.37:0/1522485767 wait complete. 2024-03-29T07:57:02.176 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:57:01 smithi037 ceph-mon[35162]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-29T07:57:02.176 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:57:01 smithi037 ceph-mon[35162]: from='mgr.14160 172.21.15.37:0/1836651950' entity='mgr.smithi037.wjikii' 2024-03-29T07:57:02.176 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:57:01 smithi037 ceph-mon[35162]: from='mgr.14160 172.21.15.37:0/1836651950' entity='mgr.smithi037.wjikii' 2024-03-29T07:57:02.176 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:57:01 smithi037 ceph-mon[35162]: from='mgr.14160 172.21.15.37:0/1836651950' entity='mgr.smithi037.wjikii' 2024-03-29T07:57:02.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:57:01 smithi037 ceph-mon[35162]: from='mgr.14160 172.21.15.37:0/1836651950' entity='mgr.smithi037.wjikii' 2024-03-29T07:57:02.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:57:01 smithi037 ceph-mon[35162]: from='mgr.14160 172.21.15.37:0/1836651950' entity='mgr.smithi037.wjikii' 2024-03-29T07:57:02.531 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 99b492c2-eda1-11ee-b647-cb9ed24678a4 -- ceph orch host ls --format=json 2024-03-29T07:57:02.946 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/mon.smithi037/config 2024-03-29T07:57:03.158 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:57:02 smithi037 ceph-mon[35162]: Deploying daemon alertmanager.smithi037 on smithi037 2024-03-29T07:57:03.158 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:57:02 smithi037 ceph-mon[35162]: from='mgr.14160 172.21.15.37:0/1836651950' entity='mgr.smithi037.wjikii' 2024-03-29T07:57:03.158 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:57:02 smithi037 ceph-mon[35162]: Added host smithi148 2024-03-29T07:57:04.176 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:57:03 smithi037 ceph-mon[35162]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-29T07:57:05.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:57:05.095+0000 7f9343e00700 1 -- 172.21.15.37:0/4081184001 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f933c0fca70 msgr2=0x7f933c0fce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T07:57:05.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:57:05.095+0000 7f9343e00700 1 --2- 172.21.15.37:0/4081184001 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f933c0fca70 0x7f933c0fce80 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f93300048c0 tx=0x7f9330009e30 comp rx=0 tx=0).stop 2024-03-29T07:57:05.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:57:05.095+0000 7f9343e00700 1 -- 172.21.15.37:0/4081184001 shutdown_connections 2024-03-29T07:57:05.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:57:05.095+0000 7f9343e00700 1 --2- 172.21.15.37:0/4081184001 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f933c0fca70 0x7f933c0fce80 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T07:57:05.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:57:05.095+0000 7f9343e00700 1 -- 172.21.15.37:0/4081184001 >> 172.21.15.37:0/4081184001 conn(0x7f933c0f81d0 msgr2=0x7f933c0fa600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T07:57:05.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:57:05.095+0000 7f9343e00700 1 -- 172.21.15.37:0/4081184001 shutdown_connections 2024-03-29T07:57:05.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:57:05.095+0000 7f9343e00700 1 -- 172.21.15.37:0/4081184001 wait complete. 2024-03-29T07:57:05.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:57:05.096+0000 7f9343e00700 1 Processor -- start 2024-03-29T07:57:05.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:57:05.096+0000 7f9343e00700 1 -- start start 2024-03-29T07:57:05.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:57:05.097+0000 7f9343e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f933c0fca70 0x7f933c071d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T07:57:05.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:57:05.097+0000 7f9343e00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f933c0722a0 con 0x7f933c0fca70 2024-03-29T07:57:05.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:57:05.097+0000 7f9341400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f933c0fca70 0x7f933c071d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T07:57:05.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:57:05.097+0000 7f9341400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f933c0fca70 0x7f933c071d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51216/0 (socket says 172.21.15.37:51216) 2024-03-29T07:57:05.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:57:05.097+0000 7f9341400700 1 -- 172.21.15.37:0/1602031057 learned_addr learned my addr 172.21.15.37:0/1602031057 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T07:57:05.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:57:05.097+0000 7f9341400700 1 -- 172.21.15.37:0/1602031057 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9330005040 con 0x7f933c0fca70 2024-03-29T07:57:05.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:57:05.098+0000 7f9341400700 1 --2- 172.21.15.37:0/1602031057 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f933c0fca70 0x7f933c071d60 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f9330000c00 tx=0x7f9330009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T07:57:05.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:57:05.098+0000 7f9339600700 1 -- 172.21.15.37:0/1602031057 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9330009620 con 0x7f933c0fca70 2024-03-29T07:57:05.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:57:05.098+0000 7f9339600700 1 -- 172.21.15.37:0/1602031057 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f9330009780 con 0x7f933c0fca70 2024-03-29T07:57:05.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:57:05.098+0000 7f9343e00700 1 -- 172.21.15.37:0/1602031057 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f933c070370 con 0x7f933c0fca70 2024-03-29T07:57:05.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:57:05.098+0000 7f9339600700 1 -- 172.21.15.37:0/1602031057 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f933001c5b0 con 0x7f933c0fca70 2024-03-29T07:57:05.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:57:05.098+0000 7f9343e00700 1 -- 172.21.15.37:0/1602031057 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f933c070860 con 0x7f933c0fca70 2024-03-29T07:57:05.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:57:05.099+0000 7f9339600700 1 -- 172.21.15.37:0/1602031057 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 13) v1 ==== 43424+0+0 (secure 0 0 0) 0x7f9330013420 con 0x7f933c0fca70 2024-03-29T07:57:05.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:57:05.099+0000 7f9343e00700 1 -- 172.21.15.37:0/1602031057 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f933c05f5f0 con 0x7f933c0fca70 2024-03-29T07:57:05.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:57:05.099+0000 7f9339600700 1 --2- 172.21.15.37:0/1602031057 >> [v2:172.21.15.37:6800/4245458578,v1:172.21.15.37:6801/4245458578] conn(0x7f9324036920 0x7f9324038dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T07:57:05.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:57:05.099+0000 7f9339600700 1 -- 172.21.15.37:0/1602031057 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f9330049ad0 con 0x7f933c0fca70 2024-03-29T07:57:05.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:57:05.100+0000 7f9340a00700 1 --2- 172.21.15.37:0/1602031057 >> [v2:172.21.15.37:6800/4245458578,v1:172.21.15.37:6801/4245458578] conn(0x7f9324036920 0x7f9324038dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T07:57:05.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:57:05.100+0000 7f9340a00700 1 --2- 172.21.15.37:0/1602031057 >> [v2:172.21.15.37:6800/4245458578,v1:172.21.15.37:6801/4245458578] conn(0x7f9324036920 0x7f9324038dd0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f9328009990 tx=0x7f9328006e40 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T07:57:05.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:57:05.105+0000 7f9339600700 1 -- 172.21.15.37:0/1602031057 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f9330011920 con 0x7f933c0fca70 2024-03-29T07:57:05.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:57:05.361+0000 7f9343e00700 1 -- 172.21.15.37:0/1602031057 --> [v2:172.21.15.37:6800/4245458578,v1:172.21.15.37:6801/4245458578] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f933c106c00 con 0x7f9324036920 2024-03-29T07:57:05.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:57:05.362+0000 7f9339600700 1 -- 172.21.15.37:0/1602031057 <== mgr.14160 v2:172.21.15.37:6800/4245458578 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+160 (secure 0 0 0) 0x7f933c106c00 con 0x7f9324036920 2024-03-29T07:57:05.361 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T07:57:05.361 INFO:teuthology.orchestra.run.smithi037.stdout:[{"addr": "172.21.15.37", "hostname": "smithi037", "labels": [], "status": ""}, {"addr": "172.21.15.148", "hostname": "smithi148", "labels": [], "status": ""}] 2024-03-29T07:57:05.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:57:05.364+0000 7f9343e00700 1 -- 172.21.15.37:0/1602031057 >> [v2:172.21.15.37:6800/4245458578,v1:172.21.15.37:6801/4245458578] conn(0x7f9324036920 msgr2=0x7f9324038dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T07:57:05.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:57:05.364+0000 7f9343e00700 1 --2- 172.21.15.37:0/1602031057 >> [v2:172.21.15.37:6800/4245458578,v1:172.21.15.37:6801/4245458578] conn(0x7f9324036920 0x7f9324038dd0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f9328009990 tx=0x7f9328006e40 comp rx=0 tx=0).stop 2024-03-29T07:57:05.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:57:05.364+0000 7f9343e00700 1 -- 172.21.15.37:0/1602031057 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f933c0fca70 msgr2=0x7f933c071d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T07:57:05.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:57:05.364+0000 7f9343e00700 1 --2- 172.21.15.37:0/1602031057 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f933c0fca70 0x7f933c071d60 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f9330000c00 tx=0x7f9330009e30 comp rx=0 tx=0).stop 2024-03-29T07:57:05.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:57:05.365+0000 7f9343e00700 1 -- 172.21.15.37:0/1602031057 shutdown_connections 2024-03-29T07:57:05.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:57:05.365+0000 7f9343e00700 1 --2- 172.21.15.37:0/1602031057 >> [v2:172.21.15.37:6800/4245458578,v1:172.21.15.37:6801/4245458578] conn(0x7f9324036920 0x7f9324038dd0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T07:57:05.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:57:05.365+0000 7f9343e00700 1 --2- 172.21.15.37:0/1602031057 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f933c0fca70 0x7f933c071d60 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T07:57:05.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:57:05.365+0000 7f9343e00700 1 -- 172.21.15.37:0/1602031057 >> 172.21.15.37:0/1602031057 conn(0x7f933c0f81d0 msgr2=0x7f933c0f8a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T07:57:05.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:57:05.365+0000 7f9343e00700 1 -- 172.21.15.37:0/1602031057 shutdown_connections 2024-03-29T07:57:05.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:57:05.365+0000 7f9343e00700 1 -- 172.21.15.37:0/1602031057 wait complete. 2024-03-29T07:57:06.176 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:57:05 smithi037 ceph-mon[35162]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-29T07:57:06.265 INFO:tasks.cephadm:Setting crush tunables to default 2024-03-29T07:57:06.266 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 99b492c2-eda1-11ee-b647-cb9ed24678a4 -- ceph osd crush tunables default 2024-03-29T07:57:06.731 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/mon.smithi037/config 2024-03-29T07:57:07.176 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:57:06 smithi037 ceph-mon[35162]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-29T07:57:08.138 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:57:07 smithi037 ceph-mon[35162]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-29T07:57:09.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:57:09.777+0000 7ff336400700 1 -- 172.21.15.37:0/3652720384 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff330071ea0 msgr2=0x7ff3300703b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T07:57:09.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:57:09.777+0000 7ff336400700 1 --2- 172.21.15.37:0/3652720384 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff330071ea0 0x7ff3300703b0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7ff3200048c0 tx=0x7ff320009e30 comp rx=0 tx=0).stop 2024-03-29T07:57:09.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:57:09.777+0000 7ff336400700 1 -- 172.21.15.37:0/3652720384 shutdown_connections 2024-03-29T07:57:09.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:57:09.777+0000 7ff336400700 1 --2- 172.21.15.37:0/3652720384 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff330071ea0 0x7ff3300703b0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T07:57:09.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:57:09.777+0000 7ff336400700 1 -- 172.21.15.37:0/3652720384 >> 172.21.15.37:0/3652720384 conn(0x7ff3300f81d0 msgr2=0x7ff3300fa600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T07:57:09.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:57:09.777+0000 7ff336400700 1 -- 172.21.15.37:0/3652720384 shutdown_connections 2024-03-29T07:57:09.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:57:09.777+0000 7ff336400700 1 -- 172.21.15.37:0/3652720384 wait complete. 2024-03-29T07:57:09.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:57:09.778+0000 7ff336400700 1 Processor -- start 2024-03-29T07:57:09.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:57:09.778+0000 7ff336400700 1 -- start start 2024-03-29T07:57:09.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:57:09.778+0000 7ff336400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff330071ea0 0x7ff33010f240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T07:57:09.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:57:09.778+0000 7ff336400700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff33010f780 con 0x7ff330071ea0 2024-03-29T07:57:09.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:57:09.778+0000 7ff32f400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff330071ea0 0x7ff33010f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T07:57:09.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:57:09.778+0000 7ff32f400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff330071ea0 0x7ff33010f240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51242/0 (socket says 172.21.15.37:51242) 2024-03-29T07:57:09.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:57:09.779+0000 7ff32f400700 1 -- 172.21.15.37:0/2950362354 learned_addr learned my addr 172.21.15.37:0/2950362354 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T07:57:09.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:57:09.779+0000 7ff32f400700 1 -- 172.21.15.37:0/2950362354 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff320005040 con 0x7ff330071ea0 2024-03-29T07:57:09.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:57:09.779+0000 7ff32f400700 1 --2- 172.21.15.37:0/2950362354 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff330071ea0 0x7ff33010f240 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ff32000b860 tx=0x7ff320009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T07:57:09.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:57:09.779+0000 7ff327400700 1 -- 172.21.15.37:0/2950362354 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff320009580 con 0x7ff330071ea0 2024-03-29T07:57:09.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:57:09.779+0000 7ff327400700 1 -- 172.21.15.37:0/2950362354 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ff3200096e0 con 0x7ff330071ea0 2024-03-29T07:57:09.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:57:09.779+0000 7ff327400700 1 -- 172.21.15.37:0/2950362354 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff32001c5b0 con 0x7ff330071ea0 2024-03-29T07:57:09.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:57:09.779+0000 7ff336400700 1 -- 172.21.15.37:0/2950362354 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff33010f980 con 0x7ff330071ea0 2024-03-29T07:57:09.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:57:09.780+0000 7ff336400700 1 -- 172.21.15.37:0/2950362354 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3301104f0 con 0x7ff330071ea0 2024-03-29T07:57:09.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:57:09.780+0000 7ff336400700 1 -- 172.21.15.37:0/2950362354 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff330105050 con 0x7ff330071ea0 2024-03-29T07:57:09.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:57:09.781+0000 7ff327400700 1 -- 172.21.15.37:0/2950362354 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 13) v1 ==== 43424+0+0 (secure 0 0 0) 0x7ff320013420 con 0x7ff330071ea0 2024-03-29T07:57:09.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:57:09.781+0000 7ff327400700 1 --2- 172.21.15.37:0/2950362354 >> [v2:172.21.15.37:6800/4245458578,v1:172.21.15.37:6801/4245458578] conn(0x7ff3140368d0 0x7ff314038d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T07:57:09.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:57:09.781+0000 7ff327400700 1 -- 172.21.15.37:0/2950362354 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7ff320049a80 con 0x7ff330071ea0 2024-03-29T07:57:09.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:57:09.782+0000 7ff32ea00700 1 --2- 172.21.15.37:0/2950362354 >> [v2:172.21.15.37:6800/4245458578,v1:172.21.15.37:6801/4245458578] conn(0x7ff3140368d0 0x7ff314038d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T07:57:09.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:57:09.782+0000 7ff32ea00700 1 --2- 172.21.15.37:0/2950362354 >> [v2:172.21.15.37:6800/4245458578,v1:172.21.15.37:6801/4245458578] conn(0x7ff3140368d0 0x7ff314038d80 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7ff31c009990 tx=0x7ff31c006e40 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T07:57:09.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:57:09.786+0000 7ff327400700 1 -- 172.21.15.37:0/2950362354 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7ff320013960 con 0x7ff330071ea0 2024-03-29T07:57:10.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:57:10.001+0000 7ff336400700 1 -- 172.21.15.37:0/2950362354 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7ff33005f5f0 con 0x7ff330071ea0 2024-03-29T07:57:10.127 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:57:09 smithi037 ceph-mon[35162]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-29T07:57:10.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:57:10.874+0000 7ff327400700 1 -- 172.21.15.37:0/2950362354 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) v1 ==== 124+0+0 (secure 0 0 0) 0x7ff320011370 con 0x7ff330071ea0 2024-03-29T07:57:10.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:57:10.876+0000 7ff336400700 1 -- 172.21.15.37:0/2950362354 >> [v2:172.21.15.37:6800/4245458578,v1:172.21.15.37:6801/4245458578] conn(0x7ff3140368d0 msgr2=0x7ff314038d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T07:57:10.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:57:10.876+0000 7ff336400700 1 --2- 172.21.15.37:0/2950362354 >> [v2:172.21.15.37:6800/4245458578,v1:172.21.15.37:6801/4245458578] conn(0x7ff3140368d0 0x7ff314038d80 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7ff31c009990 tx=0x7ff31c006e40 comp rx=0 tx=0).stop 2024-03-29T07:57:10.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:57:10.876+0000 7ff336400700 1 -- 172.21.15.37:0/2950362354 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff330071ea0 msgr2=0x7ff33010f240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T07:57:10.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:57:10.876+0000 7ff336400700 1 --2- 172.21.15.37:0/2950362354 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff330071ea0 0x7ff33010f240 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ff32000b860 tx=0x7ff320009e30 comp rx=0 tx=0).stop 2024-03-29T07:57:10.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:57:10.876+0000 7ff336400700 1 -- 172.21.15.37:0/2950362354 shutdown_connections 2024-03-29T07:57:10.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:57:10.876+0000 7ff336400700 1 --2- 172.21.15.37:0/2950362354 >> [v2:172.21.15.37:6800/4245458578,v1:172.21.15.37:6801/4245458578] conn(0x7ff3140368d0 0x7ff314038d80 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T07:57:10.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:57:10.876+0000 7ff336400700 1 --2- 172.21.15.37:0/2950362354 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff330071ea0 0x7ff33010f240 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T07:57:10.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:57:10.876+0000 7ff336400700 1 -- 172.21.15.37:0/2950362354 >> 172.21.15.37:0/2950362354 conn(0x7ff3300f81d0 msgr2=0x7ff3300f8e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T07:57:10.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:57:10.876+0000 7ff336400700 1 -- 172.21.15.37:0/2950362354 shutdown_connections 2024-03-29T07:57:10.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:57:10.877+0000 7ff336400700 1 -- 172.21.15.37:0/2950362354 wait complete. 2024-03-29T07:57:10.876 INFO:teuthology.orchestra.run.smithi037.stderr:adjusted tunables profile to default 2024-03-29T07:57:11.176 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:57:10 smithi037 ceph-mon[35162]: from='client.? 172.21.15.37:0/2950362354' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-03-29T07:57:11.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:57:11 smithi037 ceph-mon[35162]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-29T07:57:11.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:57:11 smithi037 ceph-mon[35162]: from='client.? 172.21.15.37:0/2950362354' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-03-29T07:57:11.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:57:11 smithi037 ceph-mon[35162]: osdmap e4: 0 total, 0 up, 0 in 2024-03-29T07:57:12.685 INFO:tasks.cephadm:Adding mon.smithi037 on smithi037 2024-03-29T07:57:12.685 INFO:tasks.cephadm:Adding mon.smithi148 on smithi148 2024-03-29T07:57:12.685 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 99b492c2-eda1-11ee-b647-cb9ed24678a4 -- ceph orch apply mon '2;smithi037:172.21.15.37=smithi037;smithi148:172.21.15.148=smithi148' 2024-03-29T07:57:13.090 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-29T07:57:13.178 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-29T07:57:14.176 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:57:13 smithi037 ceph-mon[35162]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-29T07:57:14.176 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:57:13 smithi037 ceph-mon[35162]: from='mgr.14160 172.21.15.37:0/1836651950' entity='mgr.smithi037.wjikii' 2024-03-29T07:57:14.176 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:57:13 smithi037 ceph-mon[35162]: from='mgr.14160 172.21.15.37:0/1836651950' entity='mgr.smithi037.wjikii' 2024-03-29T07:57:14.176 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:57:13 smithi037 ceph-mon[35162]: from='mgr.14160 172.21.15.37:0/1836651950' entity='mgr.smithi037.wjikii' 2024-03-29T07:57:14.176 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:57:13 smithi037 ceph-mon[35162]: from='mgr.14160 172.21.15.37:0/1836651950' entity='mgr.smithi037.wjikii' 2024-03-29T07:57:14.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:57:13 smithi037 ceph-mon[35162]: from='mgr.14160 172.21.15.37:0/1836651950' entity='mgr.smithi037.wjikii' 2024-03-29T07:57:14.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:57:13 smithi037 ceph-mon[35162]: from='mgr.14160 172.21.15.37:0/1836651950' entity='mgr.smithi037.wjikii' 2024-03-29T07:57:14.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:57:13 smithi037 ceph-mon[35162]: from='mgr.14160 172.21.15.37:0/1836651950' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-03-29T07:57:14.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:57:13 smithi037 ceph-mon[35162]: from='mgr.14160 172.21.15.37:0/1836651950' entity='mgr.smithi037.wjikii' 2024-03-29T07:57:15.088 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:15.089+0000 7fe753200700 1 -- 172.21.15.148:0/1448161711 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe7540fc890 msgr2=0x7fe7540fcca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T07:57:15.089 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:15.089+0000 7fe753200700 1 --2- 172.21.15.148:0/1448161711 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe7540fc890 0x7fe7540fcca0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fe7440048c0 tx=0x7fe744009e30 comp rx=0 tx=0).stop 2024-03-29T07:57:15.089 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:15.089+0000 7fe753200700 1 -- 172.21.15.148:0/1448161711 shutdown_connections 2024-03-29T07:57:15.089 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:15.089+0000 7fe753200700 1 --2- 172.21.15.148:0/1448161711 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe7540fc890 0x7fe7540fcca0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T07:57:15.089 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:15.089+0000 7fe753200700 1 -- 172.21.15.148:0/1448161711 >> 172.21.15.148:0/1448161711 conn(0x7fe7540f7fd0 msgr2=0x7fe7540fa420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T07:57:15.089 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:15.089+0000 7fe753200700 1 -- 172.21.15.148:0/1448161711 shutdown_connections 2024-03-29T07:57:15.089 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:15.089+0000 7fe753200700 1 -- 172.21.15.148:0/1448161711 wait complete. 2024-03-29T07:57:15.090 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:15.090+0000 7fe753200700 1 Processor -- start 2024-03-29T07:57:15.090 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:15.090+0000 7fe753200700 1 -- start start 2024-03-29T07:57:15.090 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:15.090+0000 7fe753200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe7540fc890 0x7fe75410ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T07:57:15.090 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:15.090+0000 7fe753200700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe75410d3b0 con 0x7fe7540fc890 2024-03-29T07:57:15.090 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:15.091+0000 7fe751e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe7540fc890 0x7fe75410ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T07:57:15.090 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:15.091+0000 7fe751e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe7540fc890 0x7fe75410ce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.148:59324/0 (socket says 172.21.15.148:59324) 2024-03-29T07:57:15.090 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:15.091+0000 7fe751e00700 1 -- 172.21.15.148:0/2497636412 learned_addr learned my addr 172.21.15.148:0/2497636412 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T07:57:15.091 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:15.091+0000 7fe751e00700 1 -- 172.21.15.148:0/2497636412 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe744005040 con 0x7fe7540fc890 2024-03-29T07:57:15.091 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:15.091+0000 7fe751e00700 1 --2- 172.21.15.148:0/2497636412 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe7540fc890 0x7fe75410ce70 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fe744009300 tx=0x7fe744009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T07:57:15.091 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:15.092+0000 7fe74a000700 1 -- 172.21.15.148:0/2497636412 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe744009600 con 0x7fe7540fc890 2024-03-29T07:57:15.091 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:15.092+0000 7fe74a000700 1 -- 172.21.15.148:0/2497636412 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fe744009760 con 0x7fe7540fc890 2024-03-29T07:57:15.091 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:15.092+0000 7fe753200700 1 -- 172.21.15.148:0/2497636412 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe75410d5b0 con 0x7fe7540fc890 2024-03-29T07:57:15.091 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:15.092+0000 7fe74a000700 1 -- 172.21.15.148:0/2497636412 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe74401c5b0 con 0x7fe7540fc890 2024-03-29T07:57:15.091 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:15.092+0000 7fe753200700 1 -- 172.21.15.148:0/2497636412 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe75410e070 con 0x7fe7540fc890 2024-03-29T07:57:15.092 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:15.093+0000 7fe753200700 1 -- 172.21.15.148:0/2497636412 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe754102c60 con 0x7fe7540fc890 2024-03-29T07:57:15.092 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:15.093+0000 7fe74a000700 1 -- 172.21.15.148:0/2497636412 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 13) v1 ==== 43424+0+0 (secure 0 0 0) 0x7fe744013960 con 0x7fe7540fc890 2024-03-29T07:57:15.093 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:15.093+0000 7fe74a000700 1 --2- 172.21.15.148:0/2497636412 >> [v2:172.21.15.37:6800/4245458578,v1:172.21.15.37:6801/4245458578] conn(0x7fe738036880 0x7fe738038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T07:57:15.093 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:15.093+0000 7fe74a000700 1 -- 172.21.15.148:0/2497636412 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7fe744049b50 con 0x7fe7540fc890 2024-03-29T07:57:15.093 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:15.094+0000 7fe751400700 1 --2- 172.21.15.148:0/2497636412 >> [v2:172.21.15.37:6800/4245458578,v1:172.21.15.37:6801/4245458578] conn(0x7fe738036880 0x7fe738038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T07:57:15.094 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:15.095+0000 7fe751400700 1 --2- 172.21.15.148:0/2497636412 >> [v2:172.21.15.37:6800/4245458578,v1:172.21.15.37:6801/4245458578] conn(0x7fe738036880 0x7fe738038d30 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fe740009990 tx=0x7fe740006e40 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T07:57:15.098 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:15.099+0000 7fe74a000700 1 -- 172.21.15.148:0/2497636412 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fe7440098d0 con 0x7fe7540fc890 2024-03-29T07:57:15.176 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:57:14 smithi037 ceph-mon[35162]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-03-29T07:57:15.176 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:57:14 smithi037 ceph-mon[35162]: Deploying daemon grafana.smithi037 on smithi037 2024-03-29T07:57:15.302 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:15.303+0000 7fe753200700 1 -- 172.21.15.148:0/2497636412 --> [v2:172.21.15.37:6800/4245458578,v1:172.21.15.37:6801/4245458578] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "2;smithi037:172.21.15.37=smithi037;smithi148:172.21.15.148=smithi148", "target": ["mon-mgr", ""]}) v1 -- 0x7fe75410e360 con 0x7fe738036880 2024-03-29T07:57:15.312 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:15.314+0000 7fe74a000700 1 -- 172.21.15.148:0/2497636412 <== mgr.14160 v2:172.21.15.37:6800/4245458578 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fe75410e360 con 0x7fe738036880 2024-03-29T07:57:15.312 INFO:teuthology.orchestra.run.smithi148.stdout:Scheduled mon update... 2024-03-29T07:57:15.315 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:15.315+0000 7fe753200700 1 -- 172.21.15.148:0/2497636412 >> [v2:172.21.15.37:6800/4245458578,v1:172.21.15.37:6801/4245458578] conn(0x7fe738036880 msgr2=0x7fe738038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T07:57:15.315 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:15.315+0000 7fe753200700 1 --2- 172.21.15.148:0/2497636412 >> [v2:172.21.15.37:6800/4245458578,v1:172.21.15.37:6801/4245458578] conn(0x7fe738036880 0x7fe738038d30 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fe740009990 tx=0x7fe740006e40 comp rx=0 tx=0).stop 2024-03-29T07:57:15.315 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:15.315+0000 7fe753200700 1 -- 172.21.15.148:0/2497636412 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe7540fc890 msgr2=0x7fe75410ce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T07:57:15.315 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:15.315+0000 7fe753200700 1 --2- 172.21.15.148:0/2497636412 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe7540fc890 0x7fe75410ce70 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fe744009300 tx=0x7fe744009e30 comp rx=0 tx=0).stop 2024-03-29T07:57:15.315 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:15.315+0000 7fe753200700 1 -- 172.21.15.148:0/2497636412 shutdown_connections 2024-03-29T07:57:15.315 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:15.315+0000 7fe753200700 1 --2- 172.21.15.148:0/2497636412 >> [v2:172.21.15.37:6800/4245458578,v1:172.21.15.37:6801/4245458578] conn(0x7fe738036880 0x7fe738038d30 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T07:57:15.315 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:15.315+0000 7fe753200700 1 --2- 172.21.15.148:0/2497636412 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe7540fc890 0x7fe75410ce70 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T07:57:15.315 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:15.315+0000 7fe753200700 1 -- 172.21.15.148:0/2497636412 >> 172.21.15.148:0/2497636412 conn(0x7fe7540f7fd0 msgr2=0x7fe7540f88b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T07:57:15.315 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:15.315+0000 7fe753200700 1 -- 172.21.15.148:0/2497636412 shutdown_connections 2024-03-29T07:57:15.315 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:15.315+0000 7fe753200700 1 -- 172.21.15.148:0/2497636412 wait complete. 2024-03-29T07:57:15.914 DEBUG:teuthology.orchestra.run.smithi148:mon.smithi148> sudo journalctl -f -n 0 -u ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4@mon.smithi148.service 2024-03-29T07:57:15.918 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-29T07:57:15.919 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 99b492c2-eda1-11ee-b647-cb9ed24678a4 -- ceph mon dump -f json 2024-03-29T07:57:15.987 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:-- Logs begin at Fri 2024-03-29 07:42:56 UTC. -- 2024-03-29T07:57:16.176 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:57:15 smithi037 ceph-mon[35162]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-29T07:57:16.176 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:57:15 smithi037 ceph-mon[35162]: from='mgr.14160 172.21.15.37:0/1836651950' entity='mgr.smithi037.wjikii' 2024-03-29T07:57:16.352 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-29T07:57:16.439 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-29T07:57:17.176 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:57:16 smithi037 ceph-mon[35162]: from='client.14192 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "2;smithi037:172.21.15.37=smithi037;smithi148:172.21.15.148=smithi148", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T07:57:17.176 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:57:16 smithi037 ceph-mon[35162]: Saving service mon spec with placement smithi037:172.21.15.37=smithi037;smithi148:172.21.15.148=smithi148;count:2 2024-03-29T07:57:17.176 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:57:16 smithi037 ceph-mon[35162]: from='mgr.14160 172.21.15.37:0/1836651950' entity='mgr.smithi037.wjikii' 2024-03-29T07:57:18.176 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:57:17 smithi037 ceph-mon[35162]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-29T07:57:18.239 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:18.239+0000 7f83afe00700 1 -- 172.21.15.148:0/1526302443 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f83a80fca70 msgr2=0x7f83a80fce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T07:57:18.239 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:18.239+0000 7f83afe00700 1 --2- 172.21.15.148:0/1526302443 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f83a80fca70 0x7f83a80fce80 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f839c0048c0 tx=0x7f839c009e30 comp rx=0 tx=0).stop 2024-03-29T07:57:18.239 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:18.239+0000 7f83afe00700 1 -- 172.21.15.148:0/1526302443 shutdown_connections 2024-03-29T07:57:18.239 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:18.239+0000 7f83afe00700 1 --2- 172.21.15.148:0/1526302443 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f83a80fca70 0x7f83a80fce80 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T07:57:18.240 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:18.239+0000 7f83afe00700 1 -- 172.21.15.148:0/1526302443 >> 172.21.15.148:0/1526302443 conn(0x7f83a80f81d0 msgr2=0x7f83a80fa600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T07:57:18.240 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:18.240+0000 7f83afe00700 1 -- 172.21.15.148:0/1526302443 shutdown_connections 2024-03-29T07:57:18.240 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:18.240+0000 7f83afe00700 1 -- 172.21.15.148:0/1526302443 wait complete. 2024-03-29T07:57:18.240 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:18.240+0000 7f83afe00700 1 Processor -- start 2024-03-29T07:57:18.241 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:18.241+0000 7f83afe00700 1 -- start start 2024-03-29T07:57:18.241 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:18.241+0000 7f83afe00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f83a80fca70 0x7f83a8071d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T07:57:18.241 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:18.241+0000 7f83afe00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83a80722a0 con 0x7f83a80fca70 2024-03-29T07:57:18.241 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:18.241+0000 7f83ad400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f83a80fca70 0x7f83a8071d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T07:57:18.241 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:18.241+0000 7f83ad400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f83a80fca70 0x7f83a8071d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.148:33118/0 (socket says 172.21.15.148:33118) 2024-03-29T07:57:18.241 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:18.241+0000 7f83ad400700 1 -- 172.21.15.148:0/4124981261 learned_addr learned my addr 172.21.15.148:0/4124981261 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T07:57:18.241 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:18.242+0000 7f83ad400700 1 -- 172.21.15.148:0/4124981261 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f839c005040 con 0x7f83a80fca70 2024-03-29T07:57:18.242 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:18.242+0000 7f83ad400700 1 --2- 172.21.15.148:0/4124981261 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f83a80fca70 0x7f83a8071d60 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f839c009300 tx=0x7f839c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T07:57:18.242 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:18.242+0000 7f83a5600700 1 -- 172.21.15.148:0/4124981261 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f839c009600 con 0x7f83a80fca70 2024-03-29T07:57:18.242 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:18.242+0000 7f83a5600700 1 -- 172.21.15.148:0/4124981261 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f839c009760 con 0x7f83a80fca70 2024-03-29T07:57:18.242 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:18.242+0000 7f83a5600700 1 -- 172.21.15.148:0/4124981261 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f839c01c5b0 con 0x7f83a80fca70 2024-03-29T07:57:18.242 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:18.242+0000 7f83afe00700 1 -- 172.21.15.148:0/4124981261 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83a8070370 con 0x7f83a80fca70 2024-03-29T07:57:18.242 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:18.242+0000 7f83afe00700 1 -- 172.21.15.148:0/4124981261 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83a8070860 con 0x7f83a80fca70 2024-03-29T07:57:18.242 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:18.243+0000 7f83a5600700 1 -- 172.21.15.148:0/4124981261 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 13) v1 ==== 43424+0+0 (secure 0 0 0) 0x7f839c013960 con 0x7f83a80fca70 2024-03-29T07:57:18.242 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:18.243+0000 7f83afe00700 1 -- 172.21.15.148:0/4124981261 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83a805f5f0 con 0x7f83a80fca70 2024-03-29T07:57:18.243 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:18.244+0000 7f83a5600700 1 --2- 172.21.15.148:0/4124981261 >> [v2:172.21.15.37:6800/4245458578,v1:172.21.15.37:6801/4245458578] conn(0x7f83900368d0 0x7f8390038d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T07:57:18.243 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:18.244+0000 7f83a5600700 1 -- 172.21.15.148:0/4124981261 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f839c049b00 con 0x7f83a80fca70 2024-03-29T07:57:18.243 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:18.244+0000 7f83aca00700 1 --2- 172.21.15.148:0/4124981261 >> [v2:172.21.15.37:6800/4245458578,v1:172.21.15.37:6801/4245458578] conn(0x7f83900368d0 0x7f8390038d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T07:57:18.244 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:18.245+0000 7f83aca00700 1 --2- 172.21.15.148:0/4124981261 >> [v2:172.21.15.37:6800/4245458578,v1:172.21.15.37:6801/4245458578] conn(0x7f83900368d0 0x7f8390038d80 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f8394009990 tx=0x7f8394006e40 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T07:57:18.248 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:18.248+0000 7f83a5600700 1 -- 172.21.15.148:0/4124981261 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f839c0098d0 con 0x7f83a80fca70 2024-03-29T07:57:18.519 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:18.520+0000 7f83afe00700 1 -- 172.21.15.148:0/4124981261 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f83a8106b90 con 0x7f83a80fca70 2024-03-29T07:57:18.519 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:18.520+0000 7f83a5600700 1 -- 172.21.15.148:0/4124981261 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7f839c011370 con 0x7f83a80fca70 2024-03-29T07:57:18.520 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T07:57:18.520 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":1,"fsid":"99b492c2-eda1-11ee-b647-cb9ed24678a4","modified":"2024-03-29T07:55:19.261744Z","created":"2024-03-29T07:55:19.261744Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi037","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:3300","nonce":0},{"type":"v1","addr":"172.21.15.37:6789","nonce":0}]},"addr":"172.21.15.37:6789/0","public_addr":"172.21.15.37:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-29T07:57:18.522 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:18.522+0000 7f83afe00700 1 -- 172.21.15.148:0/4124981261 >> [v2:172.21.15.37:6800/4245458578,v1:172.21.15.37:6801/4245458578] conn(0x7f83900368d0 msgr2=0x7f8390038d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T07:57:18.522 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:18.522+0000 7f83afe00700 1 --2- 172.21.15.148:0/4124981261 >> [v2:172.21.15.37:6800/4245458578,v1:172.21.15.37:6801/4245458578] conn(0x7f83900368d0 0x7f8390038d80 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f8394009990 tx=0x7f8394006e40 comp rx=0 tx=0).stop 2024-03-29T07:57:18.522 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:18.522+0000 7f83afe00700 1 -- 172.21.15.148:0/4124981261 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f83a80fca70 msgr2=0x7f83a8071d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T07:57:18.522 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:18.522+0000 7f83afe00700 1 --2- 172.21.15.148:0/4124981261 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f83a80fca70 0x7f83a8071d60 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f839c009300 tx=0x7f839c009e30 comp rx=0 tx=0).stop 2024-03-29T07:57:18.522 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:18.523+0000 7f83afe00700 1 -- 172.21.15.148:0/4124981261 shutdown_connections 2024-03-29T07:57:18.522 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:18.523+0000 7f83afe00700 1 --2- 172.21.15.148:0/4124981261 >> [v2:172.21.15.37:6800/4245458578,v1:172.21.15.37:6801/4245458578] conn(0x7f83900368d0 0x7f8390038d80 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T07:57:18.523 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:18.523+0000 7f83afe00700 1 --2- 172.21.15.148:0/4124981261 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f83a80fca70 0x7f83a8071d60 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T07:57:18.523 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:18.523+0000 7f83afe00700 1 -- 172.21.15.148:0/4124981261 >> 172.21.15.148:0/4124981261 conn(0x7f83a80f81d0 msgr2=0x7f83a80f8a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T07:57:18.523 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:18.523+0000 7f83afe00700 1 -- 172.21.15.148:0/4124981261 shutdown_connections 2024-03-29T07:57:18.523 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:18.523+0000 7f83afe00700 1 -- 172.21.15.148:0/4124981261 wait complete. 2024-03-29T07:57:18.523 INFO:teuthology.orchestra.run.smithi148.stderr:dumped monmap epoch 1 2024-03-29T07:57:19.176 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:57:18 smithi037 ceph-mon[35162]: from='client.? 172.21.15.148:0/4124981261' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-29T07:57:20.176 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:57:19 smithi037 ceph-mon[35162]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-29T07:57:20.226 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-29T07:57:20.227 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 99b492c2-eda1-11ee-b647-cb9ed24678a4 -- ceph mon dump -f json 2024-03-29T07:57:20.633 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-29T07:57:20.720 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-29T07:57:22.176 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:57:21 smithi037 ceph-mon[35162]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-29T07:57:22.418 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:22.419+0000 7ff86c800700 1 -- 172.21.15.148:0/2621988207 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff8640fca70 msgr2=0x7ff8640fce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T07:57:22.418 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:22.419+0000 7ff86c800700 1 --2- 172.21.15.148:0/2621988207 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff8640fca70 0x7ff8640fce80 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7ff8580048c0 tx=0x7ff858009e30 comp rx=0 tx=0).stop 2024-03-29T07:57:22.419 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:22.419+0000 7ff86c800700 1 -- 172.21.15.148:0/2621988207 shutdown_connections 2024-03-29T07:57:22.419 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:22.419+0000 7ff86c800700 1 --2- 172.21.15.148:0/2621988207 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff8640fca70 0x7ff8640fce80 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T07:57:22.419 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:22.419+0000 7ff86c800700 1 -- 172.21.15.148:0/2621988207 >> 172.21.15.148:0/2621988207 conn(0x7ff8640f81d0 msgr2=0x7ff8640fa600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T07:57:22.419 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:22.419+0000 7ff86c800700 1 -- 172.21.15.148:0/2621988207 shutdown_connections 2024-03-29T07:57:22.419 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:22.419+0000 7ff86c800700 1 -- 172.21.15.148:0/2621988207 wait complete. 2024-03-29T07:57:22.419 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:22.420+0000 7ff86c800700 1 Processor -- start 2024-03-29T07:57:22.419 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:22.420+0000 7ff86c800700 1 -- start start 2024-03-29T07:57:22.419 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:22.420+0000 7ff86c800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff8640fca70 0x7ff86410f200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T07:57:22.419 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:22.420+0000 7ff86c800700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff86410f740 con 0x7ff8640fca70 2024-03-29T07:57:22.420 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:22.421+0000 7ff869e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff8640fca70 0x7ff86410f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T07:57:22.420 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:22.421+0000 7ff869e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff8640fca70 0x7ff86410f200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.148:33132/0 (socket says 172.21.15.148:33132) 2024-03-29T07:57:22.420 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:22.421+0000 7ff869e00700 1 -- 172.21.15.148:0/3308985994 learned_addr learned my addr 172.21.15.148:0/3308985994 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T07:57:22.420 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:22.421+0000 7ff869e00700 1 -- 172.21.15.148:0/3308985994 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff858005040 con 0x7ff8640fca70 2024-03-29T07:57:22.420 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:22.421+0000 7ff869e00700 1 --2- 172.21.15.148:0/3308985994 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff8640fca70 0x7ff86410f200 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ff858000c00 tx=0x7ff858009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T07:57:22.421 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:22.421+0000 7ff85e000700 1 -- 172.21.15.148:0/3308985994 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff858009620 con 0x7ff8640fca70 2024-03-29T07:57:22.421 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:22.421+0000 7ff85e000700 1 -- 172.21.15.148:0/3308985994 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ff858009780 con 0x7ff8640fca70 2024-03-29T07:57:22.421 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:22.422+0000 7ff86c800700 1 -- 172.21.15.148:0/3308985994 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff86410f940 con 0x7ff8640fca70 2024-03-29T07:57:22.421 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:22.422+0000 7ff85e000700 1 -- 172.21.15.148:0/3308985994 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff85801c5b0 con 0x7ff8640fca70 2024-03-29T07:57:22.421 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:22.422+0000 7ff86c800700 1 -- 172.21.15.148:0/3308985994 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8641104b0 con 0x7ff8640fca70 2024-03-29T07:57:22.421 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:22.423+0000 7ff86c800700 1 -- 172.21.15.148:0/3308985994 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff864102e40 con 0x7ff8640fca70 2024-03-29T07:57:22.422 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:22.423+0000 7ff85e000700 1 -- 172.21.15.148:0/3308985994 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 13) v1 ==== 43424+0+0 (secure 0 0 0) 0x7ff858013a60 con 0x7ff8640fca70 2024-03-29T07:57:22.422 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:22.423+0000 7ff85e000700 1 --2- 172.21.15.148:0/3308985994 >> [v2:172.21.15.37:6800/4245458578,v1:172.21.15.37:6801/4245458578] conn(0x7ff84c0368d0 0x7ff84c038d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T07:57:22.422 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:22.423+0000 7ff85e000700 1 -- 172.21.15.148:0/3308985994 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7ff858049aa0 con 0x7ff8640fca70 2024-03-29T07:57:22.422 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:22.424+0000 7ff869400700 1 --2- 172.21.15.148:0/3308985994 >> [v2:172.21.15.37:6800/4245458578,v1:172.21.15.37:6801/4245458578] conn(0x7ff84c0368d0 0x7ff84c038d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T07:57:22.422 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:22.424+0000 7ff869400700 1 --2- 172.21.15.148:0/3308985994 >> [v2:172.21.15.37:6800/4245458578,v1:172.21.15.37:6801/4245458578] conn(0x7ff84c0368d0 0x7ff84c038d80 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7ff854009990 tx=0x7ff854006e40 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T07:57:22.428 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:22.430+0000 7ff85e000700 1 -- 172.21.15.148:0/3308985994 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7ff858025400 con 0x7ff8640fca70 2024-03-29T07:57:22.697 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:22.699+0000 7ff86c800700 1 -- 172.21.15.148:0/3308985994 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7ff8641107b0 con 0x7ff8640fca70 2024-03-29T07:57:22.698 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:22.699+0000 7ff85e000700 1 -- 172.21.15.148:0/3308985994 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7ff858011b60 con 0x7ff8640fca70 2024-03-29T07:57:22.698 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T07:57:22.698 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":1,"fsid":"99b492c2-eda1-11ee-b647-cb9ed24678a4","modified":"2024-03-29T07:55:19.261744Z","created":"2024-03-29T07:55:19.261744Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi037","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:3300","nonce":0},{"type":"v1","addr":"172.21.15.37:6789","nonce":0}]},"addr":"172.21.15.37:6789/0","public_addr":"172.21.15.37:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-29T07:57:22.700 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:22.702+0000 7ff86c800700 1 -- 172.21.15.148:0/3308985994 >> [v2:172.21.15.37:6800/4245458578,v1:172.21.15.37:6801/4245458578] conn(0x7ff84c0368d0 msgr2=0x7ff84c038d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T07:57:22.700 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:22.702+0000 7ff86c800700 1 --2- 172.21.15.148:0/3308985994 >> [v2:172.21.15.37:6800/4245458578,v1:172.21.15.37:6801/4245458578] conn(0x7ff84c0368d0 0x7ff84c038d80 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7ff854009990 tx=0x7ff854006e40 comp rx=0 tx=0).stop 2024-03-29T07:57:22.701 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:22.702+0000 7ff86c800700 1 -- 172.21.15.148:0/3308985994 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff8640fca70 msgr2=0x7ff86410f200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T07:57:22.701 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:22.702+0000 7ff86c800700 1 --2- 172.21.15.148:0/3308985994 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff8640fca70 0x7ff86410f200 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ff858000c00 tx=0x7ff858009e30 comp rx=0 tx=0).stop 2024-03-29T07:57:22.701 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:22.702+0000 7ff86c800700 1 -- 172.21.15.148:0/3308985994 shutdown_connections 2024-03-29T07:57:22.701 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:22.702+0000 7ff86c800700 1 --2- 172.21.15.148:0/3308985994 >> [v2:172.21.15.37:6800/4245458578,v1:172.21.15.37:6801/4245458578] conn(0x7ff84c0368d0 0x7ff84c038d80 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T07:57:22.701 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:22.702+0000 7ff86c800700 1 --2- 172.21.15.148:0/3308985994 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff8640fca70 0x7ff86410f200 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T07:57:22.701 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:22.702+0000 7ff86c800700 1 -- 172.21.15.148:0/3308985994 >> 172.21.15.148:0/3308985994 conn(0x7ff8640f81d0 msgr2=0x7ff8640fa600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T07:57:22.701 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:22.702+0000 7ff86c800700 1 -- 172.21.15.148:0/3308985994 shutdown_connections 2024-03-29T07:57:22.701 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:22.702+0000 7ff86c800700 1 -- 172.21.15.148:0/3308985994 wait complete. 2024-03-29T07:57:22.701 INFO:teuthology.orchestra.run.smithi148.stderr:dumped monmap epoch 1 2024-03-29T07:57:23.176 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:57:22 smithi037 ceph-mon[35162]: from='client.? 172.21.15.148:0/3308985994' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-29T07:57:24.176 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:57:23 smithi037 ceph-mon[35162]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-29T07:57:24.387 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-29T07:57:24.388 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 99b492c2-eda1-11ee-b647-cb9ed24678a4 -- ceph mon dump -f json 2024-03-29T07:57:24.790 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-29T07:57:24.878 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-29T07:57:26.176 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:57:25 smithi037 ceph-mon[35162]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-29T07:57:26.699 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:26.700+0000 7f18e0a00700 1 -- 172.21.15.148:0/3120707866 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f18dc0fca70 msgr2=0x7f18dc0fce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T07:57:26.699 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:26.700+0000 7f18e0a00700 1 --2- 172.21.15.148:0/3120707866 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f18dc0fca70 0x7f18dc0fce80 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f18cc0048c0 tx=0x7f18cc009e30 comp rx=0 tx=0).stop 2024-03-29T07:57:26.699 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:26.700+0000 7f18e0a00700 1 -- 172.21.15.148:0/3120707866 shutdown_connections 2024-03-29T07:57:26.699 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:26.700+0000 7f18e0a00700 1 --2- 172.21.15.148:0/3120707866 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f18dc0fca70 0x7f18dc0fce80 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T07:57:26.699 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:26.700+0000 7f18e0a00700 1 -- 172.21.15.148:0/3120707866 >> 172.21.15.148:0/3120707866 conn(0x7f18dc0f81d0 msgr2=0x7f18dc0fa600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T07:57:26.700 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:26.700+0000 7f18e0a00700 1 -- 172.21.15.148:0/3120707866 shutdown_connections 2024-03-29T07:57:26.700 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:26.701+0000 7f18e0a00700 1 -- 172.21.15.148:0/3120707866 wait complete. 2024-03-29T07:57:26.700 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:26.701+0000 7f18e0a00700 1 Processor -- start 2024-03-29T07:57:26.700 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:26.702+0000 7f18e0a00700 1 -- start start 2024-03-29T07:57:26.701 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:26.702+0000 7f18e0a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f18dc0fca70 0x7f18dc10f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T07:57:26.701 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:26.702+0000 7f18e0a00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18dc10f720 con 0x7f18dc0fca70 2024-03-29T07:57:26.701 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:26.702+0000 7f18d9e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f18dc0fca70 0x7f18dc10f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T07:57:26.701 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:26.702+0000 7f18d9e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f18dc0fca70 0x7f18dc10f1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.148:50156/0 (socket says 172.21.15.148:50156) 2024-03-29T07:57:26.701 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:26.702+0000 7f18d9e00700 1 -- 172.21.15.148:0/68445998 learned_addr learned my addr 172.21.15.148:0/68445998 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T07:57:26.702 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:26.703+0000 7f18d9e00700 1 -- 172.21.15.148:0/68445998 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18cc005040 con 0x7f18dc0fca70 2024-03-29T07:57:26.702 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:26.703+0000 7f18d9e00700 1 --2- 172.21.15.148:0/68445998 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f18dc0fca70 0x7f18dc10f1e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f18cc009300 tx=0x7f18cc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T07:57:26.702 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:26.703+0000 7f18d2000700 1 -- 172.21.15.148:0/68445998 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f18cc009600 con 0x7f18dc0fca70 2024-03-29T07:57:26.702 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:26.703+0000 7f18e0a00700 1 -- 172.21.15.148:0/68445998 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18dc10f920 con 0x7f18dc0fca70 2024-03-29T07:57:26.702 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:26.703+0000 7f18d2000700 1 -- 172.21.15.148:0/68445998 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f18cc009760 con 0x7f18dc0fca70 2024-03-29T07:57:26.702 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:26.703+0000 7f18d2000700 1 -- 172.21.15.148:0/68445998 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f18cc01c5b0 con 0x7f18dc0fca70 2024-03-29T07:57:26.702 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:26.703+0000 7f18e0a00700 1 -- 172.21.15.148:0/68445998 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18dc110490 con 0x7f18dc0fca70 2024-03-29T07:57:26.702 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:26.704+0000 7f18d2000700 1 -- 172.21.15.148:0/68445998 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 13) v1 ==== 43424+0+0 (secure 0 0 0) 0x7f18cc01c870 con 0x7f18dc0fca70 2024-03-29T07:57:26.704 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:26.705+0000 7f18e0a00700 1 -- 172.21.15.148:0/68445998 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18dc102e40 con 0x7f18dc0fca70 2024-03-29T07:57:26.705 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:26.705+0000 7f18d2000700 1 --2- 172.21.15.148:0/68445998 >> [v2:172.21.15.37:6800/4245458578,v1:172.21.15.37:6801/4245458578] conn(0x7f18c00368d0 0x7f18c0038d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T07:57:26.705 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:26.705+0000 7f18d2000700 1 -- 172.21.15.148:0/68445998 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f18cc049b20 con 0x7f18dc0fca70 2024-03-29T07:57:26.705 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:26.705+0000 7f18d9400700 1 --2- 172.21.15.148:0/68445998 >> [v2:172.21.15.37:6800/4245458578,v1:172.21.15.37:6801/4245458578] conn(0x7f18c00368d0 0x7f18c0038d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T07:57:26.705 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:26.706+0000 7f18d9400700 1 --2- 172.21.15.148:0/68445998 >> [v2:172.21.15.37:6800/4245458578,v1:172.21.15.37:6801/4245458578] conn(0x7f18c00368d0 0x7f18c0038d80 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f18c8009990 tx=0x7f18c8006e40 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T07:57:26.710 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:26.712+0000 7f18d2000700 1 -- 172.21.15.148:0/68445998 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f18cc011db0 con 0x7f18dc0fca70 2024-03-29T07:57:26.978 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:26.978+0000 7f18e0a00700 1 -- 172.21.15.148:0/68445998 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f18dc1106d0 con 0x7f18dc0fca70 2024-03-29T07:57:26.978 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:26.979+0000 7f18d2000700 1 -- 172.21.15.148:0/68445998 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7f18cc023990 con 0x7f18dc0fca70 2024-03-29T07:57:26.978 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T07:57:26.979 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":1,"fsid":"99b492c2-eda1-11ee-b647-cb9ed24678a4","modified":"2024-03-29T07:55:19.261744Z","created":"2024-03-29T07:55:19.261744Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi037","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:3300","nonce":0},{"type":"v1","addr":"172.21.15.37:6789","nonce":0}]},"addr":"172.21.15.37:6789/0","public_addr":"172.21.15.37:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-29T07:57:26.980 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:26.981+0000 7f18e0a00700 1 -- 172.21.15.148:0/68445998 >> [v2:172.21.15.37:6800/4245458578,v1:172.21.15.37:6801/4245458578] conn(0x7f18c00368d0 msgr2=0x7f18c0038d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T07:57:26.981 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:26.981+0000 7f18e0a00700 1 --2- 172.21.15.148:0/68445998 >> [v2:172.21.15.37:6800/4245458578,v1:172.21.15.37:6801/4245458578] conn(0x7f18c00368d0 0x7f18c0038d80 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f18c8009990 tx=0x7f18c8006e40 comp rx=0 tx=0).stop 2024-03-29T07:57:26.981 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:26.981+0000 7f18e0a00700 1 -- 172.21.15.148:0/68445998 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f18dc0fca70 msgr2=0x7f18dc10f1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T07:57:26.981 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:26.981+0000 7f18e0a00700 1 --2- 172.21.15.148:0/68445998 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f18dc0fca70 0x7f18dc10f1e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f18cc009300 tx=0x7f18cc009e30 comp rx=0 tx=0).stop 2024-03-29T07:57:26.981 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:26.981+0000 7f18e0a00700 1 -- 172.21.15.148:0/68445998 shutdown_connections 2024-03-29T07:57:26.981 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:26.981+0000 7f18e0a00700 1 --2- 172.21.15.148:0/68445998 >> [v2:172.21.15.37:6800/4245458578,v1:172.21.15.37:6801/4245458578] conn(0x7f18c00368d0 0x7f18c0038d80 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T07:57:26.981 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:26.981+0000 7f18e0a00700 1 --2- 172.21.15.148:0/68445998 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f18dc0fca70 0x7f18dc10f1e0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T07:57:26.981 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:26.981+0000 7f18e0a00700 1 -- 172.21.15.148:0/68445998 >> 172.21.15.148:0/68445998 conn(0x7f18dc0f81d0 msgr2=0x7f18dc0fa600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T07:57:26.981 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:26.981+0000 7f18e0a00700 1 -- 172.21.15.148:0/68445998 shutdown_connections 2024-03-29T07:57:26.981 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:26.981+0000 7f18e0a00700 1 -- 172.21.15.148:0/68445998 wait complete. 2024-03-29T07:57:26.982 INFO:teuthology.orchestra.run.smithi148.stderr:dumped monmap epoch 1 2024-03-29T07:57:28.176 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:57:27 smithi037 ceph-mon[35162]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-29T07:57:28.176 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:57:27 smithi037 ceph-mon[35162]: from='client.? 172.21.15.148:0/68445998' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-29T07:57:28.668 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-29T07:57:28.668 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 99b492c2-eda1-11ee-b647-cb9ed24678a4 -- ceph mon dump -f json 2024-03-29T07:57:29.079 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-29T07:57:29.169 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-29T07:57:29.996 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:57:29 smithi037 ceph-mon[35162]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-29T07:57:30.923 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:30.922+0000 7fc7a5200700 1 -- 172.21.15.148:0/2550429929 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc7a00fca70 msgr2=0x7fc7a00fce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T07:57:30.923 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:30.922+0000 7fc7a5200700 1 --2- 172.21.15.148:0/2550429929 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc7a00fca70 0x7fc7a00fce80 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fc7900048c0 tx=0x7fc790009e30 comp rx=0 tx=0).stop 2024-03-29T07:57:30.923 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:30.922+0000 7fc7a5200700 1 -- 172.21.15.148:0/2550429929 shutdown_connections 2024-03-29T07:57:30.923 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:30.922+0000 7fc7a5200700 1 --2- 172.21.15.148:0/2550429929 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc7a00fca70 0x7fc7a00fce80 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T07:57:30.923 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:30.922+0000 7fc7a5200700 1 -- 172.21.15.148:0/2550429929 >> 172.21.15.148:0/2550429929 conn(0x7fc7a00f81d0 msgr2=0x7fc7a00fa600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T07:57:30.923 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:30.922+0000 7fc7a5200700 1 -- 172.21.15.148:0/2550429929 shutdown_connections 2024-03-29T07:57:30.923 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:30.922+0000 7fc7a5200700 1 -- 172.21.15.148:0/2550429929 wait complete. 2024-03-29T07:57:30.923 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:30.923+0000 7fc7a5200700 1 Processor -- start 2024-03-29T07:57:30.923 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:30.923+0000 7fc7a5200700 1 -- start start 2024-03-29T07:57:30.924 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:30.923+0000 7fc7a5200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc7a00fca70 0x7fc7a010f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T07:57:30.924 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:30.923+0000 7fc7a5200700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7a010f720 con 0x7fc7a00fca70 2024-03-29T07:57:30.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:30.924+0000 7fc79de00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc7a00fca70 0x7fc7a010f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T07:57:30.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:30.924+0000 7fc79de00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc7a00fca70 0x7fc7a010f1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.148:50188/0 (socket says 172.21.15.148:50188) 2024-03-29T07:57:30.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:30.924+0000 7fc79de00700 1 -- 172.21.15.148:0/1512405700 learned_addr learned my addr 172.21.15.148:0/1512405700 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T07:57:30.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:30.924+0000 7fc79de00700 1 -- 172.21.15.148:0/1512405700 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc790005040 con 0x7fc7a00fca70 2024-03-29T07:57:30.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:30.925+0000 7fc79de00700 1 --2- 172.21.15.148:0/1512405700 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc7a00fca70 0x7fc7a010f1e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fc790009300 tx=0x7fc790009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T07:57:30.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:30.925+0000 7fc796000700 1 -- 172.21.15.148:0/1512405700 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc790009600 con 0x7fc7a00fca70 2024-03-29T07:57:30.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:30.925+0000 7fc7a5200700 1 -- 172.21.15.148:0/1512405700 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7a010f920 con 0x7fc7a00fca70 2024-03-29T07:57:30.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:30.925+0000 7fc796000700 1 -- 172.21.15.148:0/1512405700 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fc790009760 con 0x7fc7a00fca70 2024-03-29T07:57:30.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:30.925+0000 7fc796000700 1 -- 172.21.15.148:0/1512405700 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc79001c5b0 con 0x7fc7a00fca70 2024-03-29T07:57:30.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:30.925+0000 7fc7a5200700 1 -- 172.21.15.148:0/1512405700 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7a0110490 con 0x7fc7a00fca70 2024-03-29T07:57:30.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:30.926+0000 7fc796000700 1 -- 172.21.15.148:0/1512405700 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 13) v1 ==== 43424+0+0 (secure 0 0 0) 0x7fc790013960 con 0x7fc7a00fca70 2024-03-29T07:57:30.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:30.926+0000 7fc7a5200700 1 -- 172.21.15.148:0/1512405700 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7a0102e40 con 0x7fc7a00fca70 2024-03-29T07:57:30.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:30.926+0000 7fc796000700 1 --2- 172.21.15.148:0/1512405700 >> [v2:172.21.15.37:6800/4245458578,v1:172.21.15.37:6801/4245458578] conn(0x7fc784036920 0x7fc784038dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T07:57:30.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:30.926+0000 7fc796000700 1 -- 172.21.15.148:0/1512405700 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7fc790049ac0 con 0x7fc7a00fca70 2024-03-29T07:57:30.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:30.927+0000 7fc79d400700 1 --2- 172.21.15.148:0/1512405700 >> [v2:172.21.15.37:6800/4245458578,v1:172.21.15.37:6801/4245458578] conn(0x7fc784036920 0x7fc784038dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T07:57:30.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:30.927+0000 7fc79d400700 1 --2- 172.21.15.148:0/1512405700 >> [v2:172.21.15.37:6800/4245458578,v1:172.21.15.37:6801/4245458578] conn(0x7fc784036920 0x7fc784038dd0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fc78c009990 tx=0x7fc78c006e40 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T07:57:30.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:30.932+0000 7fc796000700 1 -- 172.21.15.148:0/1512405700 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fc790011550 con 0x7fc7a00fca70 2024-03-29T07:57:31.201 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:31.201+0000 7fc7a5200700 1 -- 172.21.15.148:0/1512405700 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fc7a01106d0 con 0x7fc7a00fca70 2024-03-29T07:57:31.201 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:31.202+0000 7fc796000700 1 -- 172.21.15.148:0/1512405700 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7fc790011370 con 0x7fc7a00fca70 2024-03-29T07:57:31.202 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T07:57:31.202 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":1,"fsid":"99b492c2-eda1-11ee-b647-cb9ed24678a4","modified":"2024-03-29T07:55:19.261744Z","created":"2024-03-29T07:55:19.261744Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi037","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:3300","nonce":0},{"type":"v1","addr":"172.21.15.37:6789","nonce":0}]},"addr":"172.21.15.37:6789/0","public_addr":"172.21.15.37:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-29T07:57:31.204 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:31.204+0000 7fc7a5200700 1 -- 172.21.15.148:0/1512405700 >> [v2:172.21.15.37:6800/4245458578,v1:172.21.15.37:6801/4245458578] conn(0x7fc784036920 msgr2=0x7fc784038dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T07:57:31.204 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:31.204+0000 7fc7a5200700 1 --2- 172.21.15.148:0/1512405700 >> [v2:172.21.15.37:6800/4245458578,v1:172.21.15.37:6801/4245458578] conn(0x7fc784036920 0x7fc784038dd0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fc78c009990 tx=0x7fc78c006e40 comp rx=0 tx=0).stop 2024-03-29T07:57:31.204 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:31.204+0000 7fc7a5200700 1 -- 172.21.15.148:0/1512405700 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc7a00fca70 msgr2=0x7fc7a010f1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T07:57:31.204 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:31.204+0000 7fc7a5200700 1 --2- 172.21.15.148:0/1512405700 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc7a00fca70 0x7fc7a010f1e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fc790009300 tx=0x7fc790009e30 comp rx=0 tx=0).stop 2024-03-29T07:57:31.204 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:31.205+0000 7fc7a5200700 1 -- 172.21.15.148:0/1512405700 shutdown_connections 2024-03-29T07:57:31.204 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:31.205+0000 7fc7a5200700 1 --2- 172.21.15.148:0/1512405700 >> [v2:172.21.15.37:6800/4245458578,v1:172.21.15.37:6801/4245458578] conn(0x7fc784036920 0x7fc784038dd0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T07:57:31.205 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:31.205+0000 7fc7a5200700 1 --2- 172.21.15.148:0/1512405700 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc7a00fca70 0x7fc7a010f1e0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T07:57:31.205 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:31.205+0000 7fc7a5200700 1 -- 172.21.15.148:0/1512405700 >> 172.21.15.148:0/1512405700 conn(0x7fc7a00f81d0 msgr2=0x7fc7a00fa600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T07:57:31.205 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:31.205+0000 7fc7a5200700 1 -- 172.21.15.148:0/1512405700 shutdown_connections 2024-03-29T07:57:31.205 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:31.205+0000 7fc7a5200700 1 -- 172.21.15.148:0/1512405700 wait complete. 2024-03-29T07:57:31.205 INFO:teuthology.orchestra.run.smithi148.stderr:dumped monmap epoch 1 2024-03-29T07:57:32.176 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:57:31 smithi037 ceph-mon[35162]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-29T07:57:32.176 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:57:31 smithi037 ceph-mon[35162]: from='client.? 172.21.15.148:0/1512405700' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-29T07:57:32.819 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-29T07:57:32.819 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 99b492c2-eda1-11ee-b647-cb9ed24678a4 -- ceph mon dump -f json 2024-03-29T07:57:33.221 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-29T07:57:33.308 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-29T07:57:33.676 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:57:33 smithi037 ceph-mon[35162]: from='mgr.14160 172.21.15.37:0/1836651950' entity='mgr.smithi037.wjikii' 2024-03-29T07:57:33.676 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:57:33 smithi037 ceph-mon[35162]: from='mgr.14160 172.21.15.37:0/1836651950' entity='mgr.smithi037.wjikii' 2024-03-29T07:57:33.676 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:57:33 smithi037 ceph-mon[35162]: from='mgr.14160 172.21.15.37:0/1836651950' entity='mgr.smithi037.wjikii' 2024-03-29T07:57:33.676 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:57:33 smithi037 ceph-mon[35162]: from='mgr.14160 172.21.15.37:0/1836651950' entity='mgr.smithi037.wjikii' 2024-03-29T07:57:33.676 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:57:33 smithi037 ceph-mon[35162]: from='mgr.14160 172.21.15.37:0/1836651950' entity='mgr.smithi037.wjikii' 2024-03-29T07:57:33.676 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:57:33 smithi037 ceph-mon[35162]: from='mgr.14160 172.21.15.37:0/1836651950' entity='mgr.smithi037.wjikii' 2024-03-29T07:57:33.676 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:57:33 smithi037 ceph-mon[35162]: Deploying daemon prometheus.smithi037 on smithi037 2024-03-29T07:57:33.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:57:33 smithi037 ceph-mon[35162]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-29T07:57:35.161 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:35.161+0000 7f8ae0400700 1 -- 172.21.15.148:0/2863226194 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8ad8071ea0 msgr2=0x7f8ad80703b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T07:57:35.161 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:35.161+0000 7f8ae0400700 1 --2- 172.21.15.148:0/2863226194 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8ad8071ea0 0x7f8ad80703b0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f8acc0048c0 tx=0x7f8acc009e30 comp rx=0 tx=0).stop 2024-03-29T07:57:35.161 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:35.161+0000 7f8ae0400700 1 -- 172.21.15.148:0/2863226194 shutdown_connections 2024-03-29T07:57:35.161 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:35.161+0000 7f8ae0400700 1 --2- 172.21.15.148:0/2863226194 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8ad8071ea0 0x7f8ad80703b0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T07:57:35.161 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:35.161+0000 7f8ae0400700 1 -- 172.21.15.148:0/2863226194 >> 172.21.15.148:0/2863226194 conn(0x7f8ad80f81d0 msgr2=0x7f8ad80fa600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T07:57:35.161 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:35.161+0000 7f8ae0400700 1 -- 172.21.15.148:0/2863226194 shutdown_connections 2024-03-29T07:57:35.161 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:35.161+0000 7f8ae0400700 1 -- 172.21.15.148:0/2863226194 wait complete. 2024-03-29T07:57:35.161 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:35.162+0000 7f8ae0400700 1 Processor -- start 2024-03-29T07:57:35.161 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:35.162+0000 7f8ae0400700 1 -- start start 2024-03-29T07:57:35.163 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:35.162+0000 7f8ae0400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8ad8071ea0 0x7f8ad810f240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T07:57:35.163 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:35.162+0000 7f8ae0400700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ad810f780 con 0x7f8ad8071ea0 2024-03-29T07:57:35.163 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:35.162+0000 7f8adda00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8ad8071ea0 0x7f8ad810f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T07:57:35.163 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:35.162+0000 7f8adda00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8ad8071ea0 0x7f8ad810f240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.148:50204/0 (socket says 172.21.15.148:50204) 2024-03-29T07:57:35.163 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:35.162+0000 7f8adda00700 1 -- 172.21.15.148:0/3318702897 learned_addr learned my addr 172.21.15.148:0/3318702897 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T07:57:35.163 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:35.163+0000 7f8adda00700 1 -- 172.21.15.148:0/3318702897 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8acc005040 con 0x7f8ad8071ea0 2024-03-29T07:57:35.163 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:35.163+0000 7f8adda00700 1 --2- 172.21.15.148:0/3318702897 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8ad8071ea0 0x7f8ad810f240 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f8acc00b860 tx=0x7f8acc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T07:57:35.163 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:35.163+0000 7f8ad5600700 1 -- 172.21.15.148:0/3318702897 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8acc009580 con 0x7f8ad8071ea0 2024-03-29T07:57:35.163 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:35.163+0000 7f8ae0400700 1 -- 172.21.15.148:0/3318702897 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ad810f980 con 0x7f8ad8071ea0 2024-03-29T07:57:35.163 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:35.163+0000 7f8ad5600700 1 -- 172.21.15.148:0/3318702897 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f8acc0096e0 con 0x7f8ad8071ea0 2024-03-29T07:57:35.163 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:35.163+0000 7f8ad5600700 1 -- 172.21.15.148:0/3318702897 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8acc01c5b0 con 0x7f8ad8071ea0 2024-03-29T07:57:35.163 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:35.163+0000 7f8ae0400700 1 -- 172.21.15.148:0/3318702897 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ad81104f0 con 0x7f8ad8071ea0 2024-03-29T07:57:35.164 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:35.164+0000 7f8ad5600700 1 -- 172.21.15.148:0/3318702897 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 13) v1 ==== 43424+0+0 (secure 0 0 0) 0x7f8acc013420 con 0x7f8ad8071ea0 2024-03-29T07:57:35.164 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:35.164+0000 7f8ae0400700 1 -- 172.21.15.148:0/3318702897 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ad8105050 con 0x7f8ad8071ea0 2024-03-29T07:57:35.164 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:35.165+0000 7f8ad5600700 1 --2- 172.21.15.148:0/3318702897 >> [v2:172.21.15.37:6800/4245458578,v1:172.21.15.37:6801/4245458578] conn(0x7f8ac0036880 0x7f8ac0038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T07:57:35.164 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:35.165+0000 7f8ad5600700 1 -- 172.21.15.148:0/3318702897 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f8acc049ad0 con 0x7f8ad8071ea0 2024-03-29T07:57:35.164 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:35.165+0000 7f8add000700 1 --2- 172.21.15.148:0/3318702897 >> [v2:172.21.15.37:6800/4245458578,v1:172.21.15.37:6801/4245458578] conn(0x7f8ac0036880 0x7f8ac0038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T07:57:35.165 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:35.166+0000 7f8add000700 1 --2- 172.21.15.148:0/3318702897 >> [v2:172.21.15.37:6800/4245458578,v1:172.21.15.37:6801/4245458578] conn(0x7f8ac0036880 0x7f8ac0038d30 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f8ac4009990 tx=0x7f8ac4006e40 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T07:57:35.169 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:35.170+0000 7f8ad5600700 1 -- 172.21.15.148:0/3318702897 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f8acc013960 con 0x7f8ad8071ea0 2024-03-29T07:57:35.435 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:35.436+0000 7f8ae0400700 1 -- 172.21.15.148:0/3318702897 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f8ad805f5f0 con 0x7f8ad8071ea0 2024-03-29T07:57:35.436 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:35.437+0000 7f8ad5600700 1 -- 172.21.15.148:0/3318702897 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7f8acc011370 con 0x7f8ad8071ea0 2024-03-29T07:57:35.436 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T07:57:35.436 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":1,"fsid":"99b492c2-eda1-11ee-b647-cb9ed24678a4","modified":"2024-03-29T07:55:19.261744Z","created":"2024-03-29T07:55:19.261744Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi037","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:3300","nonce":0},{"type":"v1","addr":"172.21.15.37:6789","nonce":0}]},"addr":"172.21.15.37:6789/0","public_addr":"172.21.15.37:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-29T07:57:35.438 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:35.439+0000 7f8ae0400700 1 -- 172.21.15.148:0/3318702897 >> [v2:172.21.15.37:6800/4245458578,v1:172.21.15.37:6801/4245458578] conn(0x7f8ac0036880 msgr2=0x7f8ac0038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T07:57:35.438 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:35.439+0000 7f8ae0400700 1 --2- 172.21.15.148:0/3318702897 >> [v2:172.21.15.37:6800/4245458578,v1:172.21.15.37:6801/4245458578] conn(0x7f8ac0036880 0x7f8ac0038d30 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f8ac4009990 tx=0x7f8ac4006e40 comp rx=0 tx=0).stop 2024-03-29T07:57:35.438 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:35.439+0000 7f8ae0400700 1 -- 172.21.15.148:0/3318702897 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8ad8071ea0 msgr2=0x7f8ad810f240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T07:57:35.439 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:35.439+0000 7f8ae0400700 1 --2- 172.21.15.148:0/3318702897 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8ad8071ea0 0x7f8ad810f240 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f8acc00b860 tx=0x7f8acc009e30 comp rx=0 tx=0).stop 2024-03-29T07:57:35.439 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:35.439+0000 7f8ae0400700 1 -- 172.21.15.148:0/3318702897 shutdown_connections 2024-03-29T07:57:35.439 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:35.439+0000 7f8ae0400700 1 --2- 172.21.15.148:0/3318702897 >> [v2:172.21.15.37:6800/4245458578,v1:172.21.15.37:6801/4245458578] conn(0x7f8ac0036880 0x7f8ac0038d30 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T07:57:35.439 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:35.439+0000 7f8ae0400700 1 --2- 172.21.15.148:0/3318702897 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8ad8071ea0 0x7f8ad810f240 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T07:57:35.439 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:35.439+0000 7f8ae0400700 1 -- 172.21.15.148:0/3318702897 >> 172.21.15.148:0/3318702897 conn(0x7f8ad80f81d0 msgr2=0x7f8ad80f8e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T07:57:35.439 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:35.439+0000 7f8ae0400700 1 -- 172.21.15.148:0/3318702897 shutdown_connections 2024-03-29T07:57:35.439 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:35.439+0000 7f8ae0400700 1 -- 172.21.15.148:0/3318702897 wait complete. 2024-03-29T07:57:35.439 INFO:teuthology.orchestra.run.smithi148.stderr:dumped monmap epoch 1 2024-03-29T07:57:36.176 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:57:35 smithi037 ceph-mon[35162]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-29T07:57:36.176 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:57:35 smithi037 ceph-mon[35162]: from='client.? 172.21.15.148:0/3318702897' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-29T07:57:37.095 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-29T07:57:37.095 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 99b492c2-eda1-11ee-b647-cb9ed24678a4 -- ceph mon dump -f json 2024-03-29T07:57:37.501 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-29T07:57:37.589 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-29T07:57:38.176 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:57:37 smithi037 ceph-mon[35162]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-29T07:57:38.176 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:57:37 smithi037 ceph-mon[35162]: from='mgr.14160 172.21.15.37:0/1836651950' entity='mgr.smithi037.wjikii' 2024-03-29T07:57:39.372 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:39.371+0000 7fa580000700 1 -- 172.21.15.148:0/75165570 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa5780fca70 msgr2=0x7fa5780fce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T07:57:39.373 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:39.371+0000 7fa580000700 1 --2- 172.21.15.148:0/75165570 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa5780fca70 0x7fa5780fce80 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fa56c0048c0 tx=0x7fa56c009e30 comp rx=0 tx=0).stop 2024-03-29T07:57:39.373 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:39.371+0000 7fa580000700 1 -- 172.21.15.148:0/75165570 shutdown_connections 2024-03-29T07:57:39.373 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:39.371+0000 7fa580000700 1 --2- 172.21.15.148:0/75165570 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa5780fca70 0x7fa5780fce80 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T07:57:39.373 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:39.371+0000 7fa580000700 1 -- 172.21.15.148:0/75165570 >> 172.21.15.148:0/75165570 conn(0x7fa5780f81d0 msgr2=0x7fa5780fa600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T07:57:39.373 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:39.372+0000 7fa580000700 1 -- 172.21.15.148:0/75165570 shutdown_connections 2024-03-29T07:57:39.373 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:39.372+0000 7fa580000700 1 -- 172.21.15.148:0/75165570 wait complete. 2024-03-29T07:57:39.373 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:39.372+0000 7fa580000700 1 Processor -- start 2024-03-29T07:57:39.373 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:39.373+0000 7fa580000700 1 -- start start 2024-03-29T07:57:39.373 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:39.373+0000 7fa580000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa5780fca70 0x7fa578071d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T07:57:39.373 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:39.373+0000 7fa580000700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5780722a0 con 0x7fa5780fca70 2024-03-29T07:57:39.373 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:39.373+0000 7fa57d600700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa5780fca70 0x7fa578071d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T07:57:39.373 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:39.373+0000 7fa57d600700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa5780fca70 0x7fa578071d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.148:59414/0 (socket says 172.21.15.148:59414) 2024-03-29T07:57:39.373 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:39.373+0000 7fa57d600700 1 -- 172.21.15.148:0/3476066720 learned_addr learned my addr 172.21.15.148:0/3476066720 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T07:57:39.375 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:39.373+0000 7fa57d600700 1 -- 172.21.15.148:0/3476066720 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa56c005040 con 0x7fa5780fca70 2024-03-29T07:57:39.375 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:39.374+0000 7fa57d600700 1 --2- 172.21.15.148:0/3476066720 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa5780fca70 0x7fa578071d60 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fa56c000c00 tx=0x7fa56c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T07:57:39.375 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:39.374+0000 7fa575600700 1 -- 172.21.15.148:0/3476066720 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa56c009620 con 0x7fa5780fca70 2024-03-29T07:57:39.375 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:39.374+0000 7fa575600700 1 -- 172.21.15.148:0/3476066720 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fa56c009780 con 0x7fa5780fca70 2024-03-29T07:57:39.375 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:39.374+0000 7fa575600700 1 -- 172.21.15.148:0/3476066720 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa56c01c5b0 con 0x7fa5780fca70 2024-03-29T07:57:39.375 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:39.374+0000 7fa580000700 1 -- 172.21.15.148:0/3476066720 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa578070370 con 0x7fa5780fca70 2024-03-29T07:57:39.375 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:39.374+0000 7fa580000700 1 -- 172.21.15.148:0/3476066720 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa578070860 con 0x7fa5780fca70 2024-03-29T07:57:39.375 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:39.375+0000 7fa580000700 1 -- 172.21.15.148:0/3476066720 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa57805f5f0 con 0x7fa5780fca70 2024-03-29T07:57:39.375 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:39.375+0000 7fa575600700 1 -- 172.21.15.148:0/3476066720 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 13) v1 ==== 43424+0+0 (secure 0 0 0) 0x7fa56c013a60 con 0x7fa5780fca70 2024-03-29T07:57:39.376 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:39.376+0000 7fa575600700 1 --2- 172.21.15.148:0/3476066720 >> [v2:172.21.15.37:6800/4245458578,v1:172.21.15.37:6801/4245458578] conn(0x7fa5600368d0 0x7fa560038d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T07:57:39.376 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:39.376+0000 7fa575600700 1 -- 172.21.15.148:0/3476066720 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7fa56c049ae0 con 0x7fa5780fca70 2024-03-29T07:57:39.376 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:39.376+0000 7fa57cc00700 1 --2- 172.21.15.148:0/3476066720 >> [v2:172.21.15.37:6800/4245458578,v1:172.21.15.37:6801/4245458578] conn(0x7fa5600368d0 0x7fa560038d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T07:57:39.376 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:39.377+0000 7fa57cc00700 1 --2- 172.21.15.148:0/3476066720 >> [v2:172.21.15.37:6800/4245458578,v1:172.21.15.37:6801/4245458578] conn(0x7fa5600368d0 0x7fa560038d80 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fa564009990 tx=0x7fa564006e40 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T07:57:39.380 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:39.381+0000 7fa575600700 1 -- 172.21.15.148:0/3476066720 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fa56c025400 con 0x7fa5780fca70 2024-03-29T07:57:39.649 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:39.650+0000 7fa580000700 1 -- 172.21.15.148:0/3476066720 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fa578106b90 con 0x7fa5780fca70 2024-03-29T07:57:39.650 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:39.651+0000 7fa575600700 1 -- 172.21.15.148:0/3476066720 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7fa56c011b60 con 0x7fa5780fca70 2024-03-29T07:57:39.651 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T07:57:39.651 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":1,"fsid":"99b492c2-eda1-11ee-b647-cb9ed24678a4","modified":"2024-03-29T07:55:19.261744Z","created":"2024-03-29T07:55:19.261744Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi037","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:3300","nonce":0},{"type":"v1","addr":"172.21.15.37:6789","nonce":0}]},"addr":"172.21.15.37:6789/0","public_addr":"172.21.15.37:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-29T07:57:39.652 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:39.653+0000 7fa580000700 1 -- 172.21.15.148:0/3476066720 >> [v2:172.21.15.37:6800/4245458578,v1:172.21.15.37:6801/4245458578] conn(0x7fa5600368d0 msgr2=0x7fa560038d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T07:57:39.653 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:39.653+0000 7fa580000700 1 --2- 172.21.15.148:0/3476066720 >> [v2:172.21.15.37:6800/4245458578,v1:172.21.15.37:6801/4245458578] conn(0x7fa5600368d0 0x7fa560038d80 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fa564009990 tx=0x7fa564006e40 comp rx=0 tx=0).stop 2024-03-29T07:57:39.653 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:39.653+0000 7fa580000700 1 -- 172.21.15.148:0/3476066720 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa5780fca70 msgr2=0x7fa578071d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T07:57:39.653 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:39.653+0000 7fa580000700 1 --2- 172.21.15.148:0/3476066720 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa5780fca70 0x7fa578071d60 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fa56c000c00 tx=0x7fa56c009e30 comp rx=0 tx=0).stop 2024-03-29T07:57:39.653 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:39.653+0000 7fa580000700 1 -- 172.21.15.148:0/3476066720 shutdown_connections 2024-03-29T07:57:39.653 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:39.653+0000 7fa580000700 1 --2- 172.21.15.148:0/3476066720 >> [v2:172.21.15.37:6800/4245458578,v1:172.21.15.37:6801/4245458578] conn(0x7fa5600368d0 0x7fa560038d80 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T07:57:39.653 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:39.653+0000 7fa580000700 1 --2- 172.21.15.148:0/3476066720 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa5780fca70 0x7fa578071d60 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T07:57:39.653 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:39.653+0000 7fa580000700 1 -- 172.21.15.148:0/3476066720 >> 172.21.15.148:0/3476066720 conn(0x7fa5780f81d0 msgr2=0x7fa5780f8a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T07:57:39.653 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:39.653+0000 7fa580000700 1 -- 172.21.15.148:0/3476066720 shutdown_connections 2024-03-29T07:57:39.653 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:39.653+0000 7fa580000700 1 -- 172.21.15.148:0/3476066720 wait complete. 2024-03-29T07:57:39.654 INFO:teuthology.orchestra.run.smithi148.stderr:dumped monmap epoch 1 2024-03-29T07:57:40.176 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:57:39 smithi037 ceph-mon[35162]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-29T07:57:40.176 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:57:39 smithi037 ceph-mon[35162]: from='client.? 172.21.15.148:0/3476066720' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-29T07:57:41.308 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-29T07:57:41.308 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 99b492c2-eda1-11ee-b647-cb9ed24678a4 -- ceph mon dump -f json 2024-03-29T07:57:41.712 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-29T07:57:41.800 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-29T07:57:42.023 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:57:41 smithi037 ceph-mon[35162]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-29T07:57:43.658 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:43.657+0000 7fe5b5800700 1 -- 172.21.15.148:0/1695599367 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe5b00fca70 msgr2=0x7fe5b00fce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T07:57:43.658 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:43.657+0000 7fe5b5800700 1 --2- 172.21.15.148:0/1695599367 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe5b00fca70 0x7fe5b00fce80 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fe5a00048c0 tx=0x7fe5a0009e30 comp rx=0 tx=0).stop 2024-03-29T07:57:43.658 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:43.658+0000 7fe5b5800700 1 -- 172.21.15.148:0/1695599367 shutdown_connections 2024-03-29T07:57:43.658 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:43.658+0000 7fe5b5800700 1 --2- 172.21.15.148:0/1695599367 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe5b00fca70 0x7fe5b00fce80 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T07:57:43.658 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:43.658+0000 7fe5b5800700 1 -- 172.21.15.148:0/1695599367 >> 172.21.15.148:0/1695599367 conn(0x7fe5b00f81d0 msgr2=0x7fe5b00fa600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T07:57:43.658 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:43.658+0000 7fe5b5800700 1 -- 172.21.15.148:0/1695599367 shutdown_connections 2024-03-29T07:57:43.658 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:43.658+0000 7fe5b5800700 1 -- 172.21.15.148:0/1695599367 wait complete. 2024-03-29T07:57:43.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:43.659+0000 7fe5b5800700 1 Processor -- start 2024-03-29T07:57:43.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:43.659+0000 7fe5b5800700 1 -- start start 2024-03-29T07:57:43.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:43.659+0000 7fe5b5800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe5b00fca70 0x7fe5b010f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T07:57:43.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:43.659+0000 7fe5b5800700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5b010f720 con 0x7fe5b00fca70 2024-03-29T07:57:43.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:43.660+0000 7fe5aea00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe5b00fca70 0x7fe5b010f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T07:57:43.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:43.660+0000 7fe5aea00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe5b00fca70 0x7fe5b010f1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.148:59432/0 (socket says 172.21.15.148:59432) 2024-03-29T07:57:43.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:43.660+0000 7fe5aea00700 1 -- 172.21.15.148:0/1083445771 learned_addr learned my addr 172.21.15.148:0/1083445771 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T07:57:43.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:43.660+0000 7fe5aea00700 1 -- 172.21.15.148:0/1083445771 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5a0005040 con 0x7fe5b00fca70 2024-03-29T07:57:43.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:43.660+0000 7fe5aea00700 1 --2- 172.21.15.148:0/1083445771 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe5b00fca70 0x7fe5b010f1e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fe5a0000c00 tx=0x7fe5a0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T07:57:43.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:43.660+0000 7fe5a6a00700 1 -- 172.21.15.148:0/1083445771 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe5a0009620 con 0x7fe5b00fca70 2024-03-29T07:57:43.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:43.660+0000 7fe5a6a00700 1 -- 172.21.15.148:0/1083445771 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fe5a0009780 con 0x7fe5b00fca70 2024-03-29T07:57:43.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:43.661+0000 7fe5b5800700 1 -- 172.21.15.148:0/1083445771 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5b010f920 con 0x7fe5b00fca70 2024-03-29T07:57:43.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:43.661+0000 7fe5a6a00700 1 -- 172.21.15.148:0/1083445771 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe5a001c5b0 con 0x7fe5b00fca70 2024-03-29T07:57:43.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:43.661+0000 7fe5b5800700 1 -- 172.21.15.148:0/1083445771 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5b0110490 con 0x7fe5b00fca70 2024-03-29T07:57:43.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:43.662+0000 7fe5b5800700 1 -- 172.21.15.148:0/1083445771 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5b0102e40 con 0x7fe5b00fca70 2024-03-29T07:57:43.662 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:43.662+0000 7fe5a6a00700 1 -- 172.21.15.148:0/1083445771 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 13) v1 ==== 43424+0+0 (secure 0 0 0) 0x7fe5a0013a60 con 0x7fe5b00fca70 2024-03-29T07:57:43.662 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:43.662+0000 7fe5a6a00700 1 --2- 172.21.15.148:0/1083445771 >> [v2:172.21.15.37:6800/4245458578,v1:172.21.15.37:6801/4245458578] conn(0x7fe5940368d0 0x7fe594038d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T07:57:43.662 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:43.662+0000 7fe5a6a00700 1 -- 172.21.15.148:0/1083445771 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7fe5a0049ae0 con 0x7fe5b00fca70 2024-03-29T07:57:43.662 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:43.663+0000 7fe5ae000700 1 --2- 172.21.15.148:0/1083445771 >> [v2:172.21.15.37:6800/4245458578,v1:172.21.15.37:6801/4245458578] conn(0x7fe5940368d0 0x7fe594038d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T07:57:43.662 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:43.663+0000 7fe5ae000700 1 --2- 172.21.15.148:0/1083445771 >> [v2:172.21.15.37:6800/4245458578,v1:172.21.15.37:6801/4245458578] conn(0x7fe5940368d0 0x7fe594038d80 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fe59c009990 tx=0x7fe59c006e40 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T07:57:43.668 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:43.668+0000 7fe5a6a00700 1 -- 172.21.15.148:0/1083445771 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fe5a0025400 con 0x7fe5b00fca70 2024-03-29T07:57:43.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:57:43 smithi037 ceph-mon[35162]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-29T07:57:43.934 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:43.934+0000 7fe5b5800700 1 -- 172.21.15.148:0/1083445771 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fe5b01106d0 con 0x7fe5b00fca70 2024-03-29T07:57:43.934 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:43.935+0000 7fe5a6a00700 1 -- 172.21.15.148:0/1083445771 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7fe5a0011b60 con 0x7fe5b00fca70 2024-03-29T07:57:43.934 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T07:57:43.934 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":1,"fsid":"99b492c2-eda1-11ee-b647-cb9ed24678a4","modified":"2024-03-29T07:55:19.261744Z","created":"2024-03-29T07:55:19.261744Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi037","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:3300","nonce":0},{"type":"v1","addr":"172.21.15.37:6789","nonce":0}]},"addr":"172.21.15.37:6789/0","public_addr":"172.21.15.37:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-29T07:57:43.937 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:43.937+0000 7fe5b5800700 1 -- 172.21.15.148:0/1083445771 >> [v2:172.21.15.37:6800/4245458578,v1:172.21.15.37:6801/4245458578] conn(0x7fe5940368d0 msgr2=0x7fe594038d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T07:57:43.937 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:43.937+0000 7fe5b5800700 1 --2- 172.21.15.148:0/1083445771 >> [v2:172.21.15.37:6800/4245458578,v1:172.21.15.37:6801/4245458578] conn(0x7fe5940368d0 0x7fe594038d80 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fe59c009990 tx=0x7fe59c006e40 comp rx=0 tx=0).stop 2024-03-29T07:57:43.937 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:43.937+0000 7fe5b5800700 1 -- 172.21.15.148:0/1083445771 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe5b00fca70 msgr2=0x7fe5b010f1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T07:57:43.937 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:43.937+0000 7fe5b5800700 1 --2- 172.21.15.148:0/1083445771 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe5b00fca70 0x7fe5b010f1e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fe5a0000c00 tx=0x7fe5a0009e30 comp rx=0 tx=0).stop 2024-03-29T07:57:43.937 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:43.937+0000 7fe5b5800700 1 -- 172.21.15.148:0/1083445771 shutdown_connections 2024-03-29T07:57:43.937 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:43.937+0000 7fe5b5800700 1 --2- 172.21.15.148:0/1083445771 >> [v2:172.21.15.37:6800/4245458578,v1:172.21.15.37:6801/4245458578] conn(0x7fe5940368d0 0x7fe594038d80 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T07:57:43.937 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:43.937+0000 7fe5b5800700 1 --2- 172.21.15.148:0/1083445771 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe5b00fca70 0x7fe5b010f1e0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T07:57:43.937 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:43.937+0000 7fe5b5800700 1 -- 172.21.15.148:0/1083445771 >> 172.21.15.148:0/1083445771 conn(0x7fe5b00f81d0 msgr2=0x7fe5b00fa600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T07:57:43.937 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:43.938+0000 7fe5b5800700 1 -- 172.21.15.148:0/1083445771 shutdown_connections 2024-03-29T07:57:43.938 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:43.938+0000 7fe5b5800700 1 -- 172.21.15.148:0/1083445771 wait complete. 2024-03-29T07:57:43.938 INFO:teuthology.orchestra.run.smithi148.stderr:dumped monmap epoch 1 2024-03-29T07:57:44.964 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:57:44 smithi037 ceph-mon[35162]: from='client.? 172.21.15.148:0/1083445771' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-29T07:57:45.551 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-29T07:57:45.552 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 99b492c2-eda1-11ee-b647-cb9ed24678a4 -- ceph mon dump -f json 2024-03-29T07:57:45.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:57:45 smithi037 ceph-mon[35162]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-29T07:57:45.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:57:45 smithi037 ceph-mon[35162]: from='mgr.14160 172.21.15.37:0/1836651950' entity='mgr.smithi037.wjikii' 2024-03-29T07:57:45.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:57:45 smithi037 ceph-mon[35162]: from='mgr.14160 172.21.15.37:0/1836651950' entity='mgr.smithi037.wjikii' 2024-03-29T07:57:45.957 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-29T07:57:46.045 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-29T07:57:47.021 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:57:46 smithi037 ceph-mon[35162]: from='mgr.14160 172.21.15.37:0/1836651950' entity='mgr.smithi037.wjikii' 2024-03-29T07:57:47.021 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:57:46 smithi037 ceph-mon[35162]: from='mgr.14160 172.21.15.37:0/1836651950' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-03-29T07:57:47.021 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:57:46 smithi037 ceph-mon[35162]: from='mgr.14160 172.21.15.37:0/1836651950' entity='mgr.smithi037.wjikii' 2024-03-29T07:57:47.884 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:47.884+0000 7f52b0c00700 1 -- 172.21.15.148:0/437112999 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f52ac0fca70 msgr2=0x7f52ac0fce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T07:57:47.884 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:47.884+0000 7f52b0c00700 1 --2- 172.21.15.148:0/437112999 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f52ac0fca70 0x7f52ac0fce80 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f529c0048c0 tx=0x7f529c009e30 comp rx=0 tx=0).stop 2024-03-29T07:57:47.884 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:47.884+0000 7f52b0c00700 1 -- 172.21.15.148:0/437112999 shutdown_connections 2024-03-29T07:57:47.884 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:47.884+0000 7f52b0c00700 1 --2- 172.21.15.148:0/437112999 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f52ac0fca70 0x7f52ac0fce80 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T07:57:47.885 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:47.884+0000 7f52b0c00700 1 -- 172.21.15.148:0/437112999 >> 172.21.15.148:0/437112999 conn(0x7f52ac0f81d0 msgr2=0x7f52ac0fa600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T07:57:47.885 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:47.884+0000 7f52b0c00700 1 -- 172.21.15.148:0/437112999 shutdown_connections 2024-03-29T07:57:47.885 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:47.884+0000 7f52b0c00700 1 -- 172.21.15.148:0/437112999 wait complete. 2024-03-29T07:57:47.885 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:47.885+0000 7f52b0c00700 1 Processor -- start 2024-03-29T07:57:47.885 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:47.885+0000 7f52b0c00700 1 -- start start 2024-03-29T07:57:47.886 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:47.885+0000 7f52b0c00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f52ac0fca70 0x7f52ac10cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T07:57:47.886 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:47.885+0000 7f52b0c00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52ac10d510 con 0x7f52ac0fca70 2024-03-29T07:57:47.886 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:47.886+0000 7f52a9e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f52ac0fca70 0x7f52ac10cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T07:57:47.886 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:47.886+0000 7f52a9e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f52ac0fca70 0x7f52ac10cfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.148:39766/0 (socket says 172.21.15.148:39766) 2024-03-29T07:57:47.887 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:47.886+0000 7f52a9e00700 1 -- 172.21.15.148:0/3628016682 learned_addr learned my addr 172.21.15.148:0/3628016682 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T07:57:47.887 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:47.886+0000 7f52a9e00700 1 -- 172.21.15.148:0/3628016682 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f529c005040 con 0x7f52ac0fca70 2024-03-29T07:57:47.887 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:47.886+0000 7f52a9e00700 1 --2- 172.21.15.148:0/3628016682 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f52ac0fca70 0x7f52ac10cfd0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f529c000c00 tx=0x7f529c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T07:57:47.887 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:47.886+0000 7f52a2000700 1 -- 172.21.15.148:0/3628016682 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f529c009620 con 0x7f52ac0fca70 2024-03-29T07:57:47.887 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:47.886+0000 7f52a2000700 1 -- 172.21.15.148:0/3628016682 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f529c009780 con 0x7f52ac0fca70 2024-03-29T07:57:47.887 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:47.886+0000 7f52b0c00700 1 -- 172.21.15.148:0/3628016682 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52ac10d710 con 0x7f52ac0fca70 2024-03-29T07:57:47.887 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:47.886+0000 7f52a2000700 1 -- 172.21.15.148:0/3628016682 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f529c01c5b0 con 0x7f52ac0fca70 2024-03-29T07:57:47.887 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:47.887+0000 7f52b0c00700 1 -- 172.21.15.148:0/3628016682 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52ac10e280 con 0x7f52ac0fca70 2024-03-29T07:57:47.888 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:47.887+0000 7f52b0c00700 1 -- 172.21.15.148:0/3628016682 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52ac102e40 con 0x7f52ac0fca70 2024-03-29T07:57:47.888 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:47.888+0000 7f52a2000700 1 -- 172.21.15.148:0/3628016682 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 14) v1 ==== 43438+0+0 (secure 0 0 0) 0x7f529c013a60 con 0x7f52ac0fca70 2024-03-29T07:57:47.888 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:47.888+0000 7f52a2000700 1 --2- 172.21.15.148:0/3628016682 >> [v2:172.21.15.37:6800/4245458578,v1:172.21.15.37:6801/4245458578] conn(0x7f52900368d0 0x7f5290038d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T07:57:47.888 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:47.888+0000 7f52a2000700 1 -- 172.21.15.148:0/3628016682 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f529c049bc0 con 0x7f52ac0fca70 2024-03-29T07:57:47.888 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:47.888+0000 7f52a9400700 1 -- 172.21.15.148:0/3628016682 >> [v2:172.21.15.37:6800/4245458578,v1:172.21.15.37:6801/4245458578] conn(0x7f52900368d0 msgr2=0x7f5290038d80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/4245458578 2024-03-29T07:57:47.889 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:47.888+0000 7f52a9400700 1 --2- 172.21.15.148:0/3628016682 >> [v2:172.21.15.37:6800/4245458578,v1:172.21.15.37:6801/4245458578] conn(0x7f52900368d0 0x7f5290038d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-29T07:57:47.892 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:47.893+0000 7f52a2000700 1 -- 172.21.15.148:0/3628016682 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f529c025400 con 0x7f52ac0fca70 2024-03-29T07:57:47.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:57:47 smithi037 ceph-mon[35162]: from='mgr.14160 172.21.15.37:0/1836651950' entity='mgr.smithi037.wjikii' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-03-29T07:57:47.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:57:47 smithi037 ceph-mon[35162]: mgrmap e14: smithi037.wjikii(active, since 70s) 2024-03-29T07:57:48.088 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:48.089+0000 7f52a9400700 1 -- 172.21.15.148:0/3628016682 >> [v2:172.21.15.37:6800/4245458578,v1:172.21.15.37:6801/4245458578] conn(0x7f52900368d0 msgr2=0x7f5290038d80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/4245458578 2024-03-29T07:57:48.089 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:48.089+0000 7f52a9400700 1 --2- 172.21.15.148:0/3628016682 >> [v2:172.21.15.37:6800/4245458578,v1:172.21.15.37:6801/4245458578] conn(0x7f52900368d0 0x7f5290038d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-29T07:57:48.161 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:48.161+0000 7f52b0c00700 1 -- 172.21.15.148:0/3628016682 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f52ac10ec90 con 0x7f52ac0fca70 2024-03-29T07:57:48.161 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:48.162+0000 7f52a2000700 1 -- 172.21.15.148:0/3628016682 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7f529c011b60 con 0x7f52ac0fca70 2024-03-29T07:57:48.161 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T07:57:48.161 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":1,"fsid":"99b492c2-eda1-11ee-b647-cb9ed24678a4","modified":"2024-03-29T07:55:19.261744Z","created":"2024-03-29T07:55:19.261744Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi037","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:3300","nonce":0},{"type":"v1","addr":"172.21.15.37:6789","nonce":0}]},"addr":"172.21.15.37:6789/0","public_addr":"172.21.15.37:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-29T07:57:48.163 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:48.164+0000 7f52b0c00700 1 -- 172.21.15.148:0/3628016682 >> [v2:172.21.15.37:6800/4245458578,v1:172.21.15.37:6801/4245458578] conn(0x7f52900368d0 msgr2=0x7f5290038d80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-29T07:57:48.164 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:48.164+0000 7f52b0c00700 1 --2- 172.21.15.148:0/3628016682 >> [v2:172.21.15.37:6800/4245458578,v1:172.21.15.37:6801/4245458578] conn(0x7f52900368d0 0x7f5290038d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T07:57:48.164 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:48.164+0000 7f52b0c00700 1 -- 172.21.15.148:0/3628016682 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f52ac0fca70 msgr2=0x7f52ac10cfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T07:57:48.164 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:48.164+0000 7f52b0c00700 1 --2- 172.21.15.148:0/3628016682 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f52ac0fca70 0x7f52ac10cfd0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f529c000c00 tx=0x7f529c009e30 comp rx=0 tx=0).stop 2024-03-29T07:57:48.164 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:48.164+0000 7f52b0c00700 1 -- 172.21.15.148:0/3628016682 shutdown_connections 2024-03-29T07:57:48.164 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:48.164+0000 7f52b0c00700 1 --2- 172.21.15.148:0/3628016682 >> [v2:172.21.15.37:6800/4245458578,v1:172.21.15.37:6801/4245458578] conn(0x7f52900368d0 0x7f5290038d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T07:57:48.164 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:48.164+0000 7f52b0c00700 1 --2- 172.21.15.148:0/3628016682 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f52ac0fca70 0x7f52ac10cfd0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T07:57:48.164 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:48.164+0000 7f52b0c00700 1 -- 172.21.15.148:0/3628016682 >> 172.21.15.148:0/3628016682 conn(0x7f52ac0f81d0 msgr2=0x7f52ac0f8a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T07:57:48.164 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:48.164+0000 7f52b0c00700 1 -- 172.21.15.148:0/3628016682 shutdown_connections 2024-03-29T07:57:48.164 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:48.164+0000 7f52b0c00700 1 -- 172.21.15.148:0/3628016682 wait complete. 2024-03-29T07:57:48.165 INFO:teuthology.orchestra.run.smithi148.stderr:dumped monmap epoch 1 2024-03-29T07:57:48.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:57:48 smithi037 ceph-mon[35162]: from='client.? 172.21.15.148:0/3628016682' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-29T07:57:49.821 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-29T07:57:49.821 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 99b492c2-eda1-11ee-b647-cb9ed24678a4 -- ceph mon dump -f json 2024-03-29T07:57:50.222 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-29T07:57:50.310 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-29T07:57:52.053 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:52.053+0000 7f053fc00700 1 -- 172.21.15.148:0/2598673552 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f05380fca70 msgr2=0x7f05380fce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T07:57:52.053 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:52.053+0000 7f053fc00700 1 --2- 172.21.15.148:0/2598673552 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f05380fca70 0x7f05380fce80 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f052c0048c0 tx=0x7f052c009e30 comp rx=0 tx=0).stop 2024-03-29T07:57:52.053 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:52.053+0000 7f053fc00700 1 -- 172.21.15.148:0/2598673552 shutdown_connections 2024-03-29T07:57:52.053 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:52.053+0000 7f053fc00700 1 --2- 172.21.15.148:0/2598673552 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f05380fca70 0x7f05380fce80 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T07:57:52.053 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:52.053+0000 7f053fc00700 1 -- 172.21.15.148:0/2598673552 >> 172.21.15.148:0/2598673552 conn(0x7f05380f81d0 msgr2=0x7f05380fa600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T07:57:52.053 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:52.053+0000 7f053fc00700 1 -- 172.21.15.148:0/2598673552 shutdown_connections 2024-03-29T07:57:52.054 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:52.053+0000 7f053fc00700 1 -- 172.21.15.148:0/2598673552 wait complete. 2024-03-29T07:57:52.054 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:52.055+0000 7f053fc00700 1 Processor -- start 2024-03-29T07:57:52.054 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:52.055+0000 7f053fc00700 1 -- start start 2024-03-29T07:57:52.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:52.055+0000 7f053fc00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f05380fca70 0x7f053810cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T07:57:52.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:52.055+0000 7f053fc00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f053810d510 con 0x7f05380fca70 2024-03-29T07:57:52.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:52.056+0000 7f053d200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f05380fca70 0x7f053810cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T07:57:52.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:52.056+0000 7f053d200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f05380fca70 0x7f053810cfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.148:39780/0 (socket says 172.21.15.148:39780) 2024-03-29T07:57:52.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:52.056+0000 7f053d200700 1 -- 172.21.15.148:0/2206455253 learned_addr learned my addr 172.21.15.148:0/2206455253 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T07:57:52.056 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:52.056+0000 7f053d200700 1 -- 172.21.15.148:0/2206455253 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f052c005040 con 0x7f05380fca70 2024-03-29T07:57:52.056 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:52.056+0000 7f053d200700 1 --2- 172.21.15.148:0/2206455253 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f05380fca70 0x7f053810cfd0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f052c000c00 tx=0x7f052c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T07:57:52.056 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:52.056+0000 7f0534c00700 1 -- 172.21.15.148:0/2206455253 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f052c009620 con 0x7f05380fca70 2024-03-29T07:57:52.056 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:52.056+0000 7f053fc00700 1 -- 172.21.15.148:0/2206455253 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f053810d710 con 0x7f05380fca70 2024-03-29T07:57:52.056 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:52.056+0000 7f0534c00700 1 -- 172.21.15.148:0/2206455253 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f052c009780 con 0x7f05380fca70 2024-03-29T07:57:52.056 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:52.056+0000 7f0534c00700 1 -- 172.21.15.148:0/2206455253 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f052c01c5b0 con 0x7f05380fca70 2024-03-29T07:57:52.056 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:52.057+0000 7f053fc00700 1 -- 172.21.15.148:0/2206455253 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f053810e280 con 0x7f05380fca70 2024-03-29T07:57:52.057 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:52.057+0000 7f053fc00700 1 -- 172.21.15.148:0/2206455253 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0538102e40 con 0x7f05380fca70 2024-03-29T07:57:52.057 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:52.058+0000 7f0534c00700 1 -- 172.21.15.148:0/2206455253 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 14) v1 ==== 43438+0+0 (secure 0 0 0) 0x7f052c013a60 con 0x7f05380fca70 2024-03-29T07:57:52.058 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:52.058+0000 7f0534c00700 1 --2- 172.21.15.148:0/2206455253 >> [v2:172.21.15.37:6800/4245458578,v1:172.21.15.37:6801/4245458578] conn(0x7f0520036920 0x7f0520038dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T07:57:52.058 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:52.058+0000 7f0534c00700 1 -- 172.21.15.148:0/2206455253 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f052c049b40 con 0x7f05380fca70 2024-03-29T07:57:52.058 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:52.058+0000 7f0537e00700 1 -- 172.21.15.148:0/2206455253 >> [v2:172.21.15.37:6800/4245458578,v1:172.21.15.37:6801/4245458578] conn(0x7f0520036920 msgr2=0x7f0520038dd0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/4245458578 2024-03-29T07:57:52.058 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:52.058+0000 7f0537e00700 1 --2- 172.21.15.148:0/2206455253 >> [v2:172.21.15.37:6800/4245458578,v1:172.21.15.37:6801/4245458578] conn(0x7f0520036920 0x7f0520038dd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-29T07:57:52.063 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:52.064+0000 7f0534c00700 1 -- 172.21.15.148:0/2206455253 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f052c025400 con 0x7f05380fca70 2024-03-29T07:57:52.258 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:52.259+0000 7f0537e00700 1 -- 172.21.15.148:0/2206455253 >> [v2:172.21.15.37:6800/4245458578,v1:172.21.15.37:6801/4245458578] conn(0x7f0520036920 msgr2=0x7f0520038dd0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/4245458578 2024-03-29T07:57:52.258 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:52.259+0000 7f0537e00700 1 --2- 172.21.15.148:0/2206455253 >> [v2:172.21.15.37:6800/4245458578,v1:172.21.15.37:6801/4245458578] conn(0x7f0520036920 0x7f0520038dd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-29T07:57:52.330 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:52.331+0000 7f053fc00700 1 -- 172.21.15.148:0/2206455253 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f053810ec90 con 0x7f05380fca70 2024-03-29T07:57:52.331 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:52.331+0000 7f0534c00700 1 -- 172.21.15.148:0/2206455253 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7f052c011b60 con 0x7f05380fca70 2024-03-29T07:57:52.331 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T07:57:52.331 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":1,"fsid":"99b492c2-eda1-11ee-b647-cb9ed24678a4","modified":"2024-03-29T07:55:19.261744Z","created":"2024-03-29T07:55:19.261744Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi037","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:3300","nonce":0},{"type":"v1","addr":"172.21.15.37:6789","nonce":0}]},"addr":"172.21.15.37:6789/0","public_addr":"172.21.15.37:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-29T07:57:52.333 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:52.333+0000 7f053fc00700 1 -- 172.21.15.148:0/2206455253 >> [v2:172.21.15.37:6800/4245458578,v1:172.21.15.37:6801/4245458578] conn(0x7f0520036920 msgr2=0x7f0520038dd0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-29T07:57:52.333 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:52.333+0000 7f053fc00700 1 --2- 172.21.15.148:0/2206455253 >> [v2:172.21.15.37:6800/4245458578,v1:172.21.15.37:6801/4245458578] conn(0x7f0520036920 0x7f0520038dd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T07:57:52.333 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:52.333+0000 7f053fc00700 1 -- 172.21.15.148:0/2206455253 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f05380fca70 msgr2=0x7f053810cfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T07:57:52.333 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:52.333+0000 7f053fc00700 1 --2- 172.21.15.148:0/2206455253 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f05380fca70 0x7f053810cfd0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f052c000c00 tx=0x7f052c009e30 comp rx=0 tx=0).stop 2024-03-29T07:57:52.334 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:52.333+0000 7f053fc00700 1 -- 172.21.15.148:0/2206455253 shutdown_connections 2024-03-29T07:57:52.334 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:52.333+0000 7f053fc00700 1 --2- 172.21.15.148:0/2206455253 >> [v2:172.21.15.37:6800/4245458578,v1:172.21.15.37:6801/4245458578] conn(0x7f0520036920 0x7f0520038dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T07:57:52.334 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:52.333+0000 7f053fc00700 1 --2- 172.21.15.148:0/2206455253 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f05380fca70 0x7f053810cfd0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T07:57:52.334 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:52.333+0000 7f053fc00700 1 -- 172.21.15.148:0/2206455253 >> 172.21.15.148:0/2206455253 conn(0x7f05380f81d0 msgr2=0x7f05380f8a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T07:57:52.334 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:52.334+0000 7f053fc00700 1 -- 172.21.15.148:0/2206455253 shutdown_connections 2024-03-29T07:57:52.334 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:52.334+0000 7f053fc00700 1 -- 172.21.15.148:0/2206455253 wait complete. 2024-03-29T07:57:52.334 INFO:teuthology.orchestra.run.smithi148.stderr:dumped monmap epoch 1 2024-03-29T07:57:52.676 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:57:52 smithi037 ceph-mon[35162]: from='client.? 172.21.15.148:0/2206455253' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-29T07:57:54.092 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-29T07:57:54.093 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 99b492c2-eda1-11ee-b647-cb9ed24678a4 -- ceph mon dump -f json 2024-03-29T07:57:54.497 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-29T07:57:54.584 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-29T07:57:55.530 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:57:55 smithi037 ceph-mon[35162]: Active manager daemon smithi037.wjikii restarted 2024-03-29T07:57:55.530 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:57:55 smithi037 ceph-mon[35162]: Activating manager daemon smithi037.wjikii 2024-03-29T07:57:55.530 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:57:55 smithi037 ceph-mon[35162]: osdmap e5: 0 total, 0 up, 0 in 2024-03-29T07:57:55.530 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:57:55 smithi037 ceph-mon[35162]: mgrmap e15: smithi037.wjikii(active, starting, since 0.00376023s) 2024-03-29T07:57:55.530 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:57:55 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "mon metadata", "id": "smithi037"}]: dispatch 2024-03-29T07:57:55.531 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:57:55 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "mgr metadata", "who": "smithi037.wjikii", "id": "smithi037.wjikii"}]: dispatch 2024-03-29T07:57:55.531 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:57:55 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-29T07:57:55.531 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:57:55 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-29T07:57:55.531 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:57:55 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-29T07:57:55.531 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:57:55 smithi037 ceph-mon[35162]: Manager daemon smithi037.wjikii is now available 2024-03-29T07:57:55.531 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:57:55 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T07:57:55.531 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:57:55 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T07:57:56.583 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:56.583+0000 7f733ec00700 1 -- 172.21.15.148:0/1230429895 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f73380fca70 msgr2=0x7f73380fce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T07:57:56.583 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:56.583+0000 7f733ec00700 1 --2- 172.21.15.148:0/1230429895 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f73380fca70 0x7f73380fce80 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f73280048c0 tx=0x7f7328009e30 comp rx=0 tx=0).stop 2024-03-29T07:57:56.583 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:56.583+0000 7f733ec00700 1 -- 172.21.15.148:0/1230429895 shutdown_connections 2024-03-29T07:57:56.583 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:56.583+0000 7f733ec00700 1 --2- 172.21.15.148:0/1230429895 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f73380fca70 0x7f73380fce80 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T07:57:56.584 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:56.583+0000 7f733ec00700 1 -- 172.21.15.148:0/1230429895 >> 172.21.15.148:0/1230429895 conn(0x7f73380f81d0 msgr2=0x7f73380fa600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T07:57:56.584 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:56.583+0000 7f733ec00700 1 -- 172.21.15.148:0/1230429895 shutdown_connections 2024-03-29T07:57:56.584 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:56.583+0000 7f733ec00700 1 -- 172.21.15.148:0/1230429895 wait complete. 2024-03-29T07:57:56.585 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:56.584+0000 7f733ec00700 1 Processor -- start 2024-03-29T07:57:56.585 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:56.584+0000 7f733ec00700 1 -- start start 2024-03-29T07:57:56.585 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:56.585+0000 7f733ec00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f73380fca70 0x7f733810f200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T07:57:56.585 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:56.585+0000 7f733ec00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f733810f740 con 0x7f73380fca70 2024-03-29T07:57:56.585 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:56.585+0000 7f7337e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f73380fca70 0x7f733810f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T07:57:56.585 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:56.585+0000 7f7337e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f73380fca70 0x7f733810f200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.148:35396/0 (socket says 172.21.15.148:35396) 2024-03-29T07:57:56.585 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:56.585+0000 7f7337e00700 1 -- 172.21.15.148:0/274997815 learned_addr learned my addr 172.21.15.148:0/274997815 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T07:57:56.586 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:56.586+0000 7f7337e00700 1 -- 172.21.15.148:0/274997815 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7328005040 con 0x7f73380fca70 2024-03-29T07:57:56.586 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:56.586+0000 7f7337e00700 1 --2- 172.21.15.148:0/274997815 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f73380fca70 0x7f733810f200 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f7328000c00 tx=0x7f7328009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T07:57:56.586 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:56.586+0000 7f732fe00700 1 -- 172.21.15.148:0/274997815 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7328009620 con 0x7f73380fca70 2024-03-29T07:57:56.587 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:56.586+0000 7f732fe00700 1 -- 172.21.15.148:0/274997815 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f7328009780 con 0x7f73380fca70 2024-03-29T07:57:56.587 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:56.586+0000 7f732fe00700 1 -- 172.21.15.148:0/274997815 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f732801c5b0 con 0x7f73380fca70 2024-03-29T07:57:56.587 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:56.586+0000 7f733ec00700 1 -- 172.21.15.148:0/274997815 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f733810f940 con 0x7f73380fca70 2024-03-29T07:57:56.587 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:56.586+0000 7f733ec00700 1 -- 172.21.15.148:0/274997815 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73381104b0 con 0x7f73380fca70 2024-03-29T07:57:56.587 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:56.587+0000 7f732fe00700 1 -- 172.21.15.148:0/274997815 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) v1 ==== 43289+0+0 (secure 0 0 0) 0x7f7328013420 con 0x7f73380fca70 2024-03-29T07:57:56.587 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:56.587+0000 7f733ec00700 1 -- 172.21.15.148:0/274997815 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7338102e40 con 0x7f73380fca70 2024-03-29T07:57:56.587 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:56.588+0000 7f732fe00700 1 --2- 172.21.15.148:0/274997815 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f731c036750 0x7f731c038c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T07:57:56.588 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:56.588+0000 7f732fe00700 1 -- 172.21.15.148:0/274997815 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7f73280499c0 con 0x7f73380fca70 2024-03-29T07:57:56.588 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:56.588+0000 7f7337400700 1 --2- 172.21.15.148:0/274997815 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f731c036750 0x7f731c038c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T07:57:56.588 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:56.589+0000 7f7337400700 1 --2- 172.21.15.148:0/274997815 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f731c036750 0x7f731c038c00 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f7324009990 tx=0x7f7324006e40 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T07:57:56.592 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:56.593+0000 7f732fe00700 1 -- 172.21.15.148:0/274997815 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f73280119c0 con 0x7f73380fca70 2024-03-29T07:57:56.886 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:56.887+0000 7f733ec00700 1 -- 172.21.15.148:0/274997815 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f73381106f0 con 0x7f73380fca70 2024-03-29T07:57:56.887 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:56.888+0000 7f732fe00700 1 -- 172.21.15.148:0/274997815 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7f7328011370 con 0x7f73380fca70 2024-03-29T07:57:56.888 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T07:57:56.888 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":1,"fsid":"99b492c2-eda1-11ee-b647-cb9ed24678a4","modified":"2024-03-29T07:55:19.261744Z","created":"2024-03-29T07:55:19.261744Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi037","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:3300","nonce":0},{"type":"v1","addr":"172.21.15.37:6789","nonce":0}]},"addr":"172.21.15.37:6789/0","public_addr":"172.21.15.37:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-29T07:57:56.890 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:56.890+0000 7f733ec00700 1 -- 172.21.15.148:0/274997815 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f731c036750 msgr2=0x7f731c038c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T07:57:56.890 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:56.890+0000 7f733ec00700 1 --2- 172.21.15.148:0/274997815 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f731c036750 0x7f731c038c00 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f7324009990 tx=0x7f7324006e40 comp rx=0 tx=0).stop 2024-03-29T07:57:56.890 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:56.890+0000 7f733ec00700 1 -- 172.21.15.148:0/274997815 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f73380fca70 msgr2=0x7f733810f200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T07:57:56.890 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:56.890+0000 7f733ec00700 1 --2- 172.21.15.148:0/274997815 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f73380fca70 0x7f733810f200 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f7328000c00 tx=0x7f7328009e30 comp rx=0 tx=0).stop 2024-03-29T07:57:56.890 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:56.890+0000 7f733ec00700 1 -- 172.21.15.148:0/274997815 shutdown_connections 2024-03-29T07:57:56.890 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:56.890+0000 7f733ec00700 1 --2- 172.21.15.148:0/274997815 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f731c036750 0x7f731c038c00 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T07:57:56.890 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:56.890+0000 7f733ec00700 1 --2- 172.21.15.148:0/274997815 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f73380fca70 0x7f733810f200 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T07:57:56.890 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:56.890+0000 7f733ec00700 1 -- 172.21.15.148:0/274997815 >> 172.21.15.148:0/274997815 conn(0x7f73380f81d0 msgr2=0x7f73380fa600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T07:57:56.890 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:56.891+0000 7f733ec00700 1 -- 172.21.15.148:0/274997815 shutdown_connections 2024-03-29T07:57:56.891 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:57:56.891+0000 7f733ec00700 1 -- 172.21.15.148:0/274997815 wait complete. 2024-03-29T07:57:56.891 INFO:teuthology.orchestra.run.smithi148.stderr:dumped monmap epoch 1 2024-03-29T07:57:56.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:57:56 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T07:57:56.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:57:56 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T07:57:56.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:57:56 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi037.wjikii/mirror_snapshot_schedule"}]: dispatch 2024-03-29T07:57:56.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:57:56 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi037.wjikii/trash_purge_schedule"}]: dispatch 2024-03-29T07:57:56.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:57:56 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T07:57:56.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:57:56 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T07:57:56.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:57:56 smithi037 ceph-mon[35162]: mgrmap e16: smithi037.wjikii(active, since 1.00675s) 2024-03-29T07:57:57.676 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:57:57 smithi037 ceph-mon[35162]: [29/Mar/2024:07:57:56] ENGINE Bus STARTING 2024-03-29T07:57:57.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:57:57 smithi037 ceph-mon[35162]: [29/Mar/2024:07:57:56] ENGINE Serving on https://172.21.15.37:7150 2024-03-29T07:57:57.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:57:57 smithi037 ceph-mon[35162]: [29/Mar/2024:07:57:56] ENGINE Bus STARTED 2024-03-29T07:57:57.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:57:57 smithi037 ceph-mon[35162]: from='client.? 172.21.15.148:0/274997815' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-29T07:57:57.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:57:57 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T07:57:58.544 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-29T07:57:58.545 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 99b492c2-eda1-11ee-b647-cb9ed24678a4 -- ceph mon dump -f json 2024-03-29T07:57:59.176 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:57:58 smithi037 ceph-mon[35162]: mgrmap e17: smithi037.wjikii(active, since 2s) 2024-03-29T07:57:59.176 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:57:58 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T07:57:59.549 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-29T07:58:00.001 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-29T07:58:02.128 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:02.128+0000 7f292c600700 1 -- 172.21.15.148:0/520305123 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f29240fca70 msgr2=0x7f29240fce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T07:58:02.129 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:02.128+0000 7f292c600700 1 --2- 172.21.15.148:0/520305123 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f29240fca70 0x7f29240fce80 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f29180048c0 tx=0x7f2918009e30 comp rx=0 tx=0).stop 2024-03-29T07:58:02.129 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:02.128+0000 7f292c600700 1 -- 172.21.15.148:0/520305123 shutdown_connections 2024-03-29T07:58:02.129 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:02.128+0000 7f292c600700 1 --2- 172.21.15.148:0/520305123 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f29240fca70 0x7f29240fce80 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T07:58:02.129 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:02.128+0000 7f292c600700 1 -- 172.21.15.148:0/520305123 >> 172.21.15.148:0/520305123 conn(0x7f29240f81d0 msgr2=0x7f29240fa600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T07:58:02.129 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:02.128+0000 7f292c600700 1 -- 172.21.15.148:0/520305123 shutdown_connections 2024-03-29T07:58:02.129 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:02.128+0000 7f292c600700 1 -- 172.21.15.148:0/520305123 wait complete. 2024-03-29T07:58:02.129 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:02.129+0000 7f292c600700 1 Processor -- start 2024-03-29T07:58:02.129 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:02.129+0000 7f292c600700 1 -- start start 2024-03-29T07:58:02.130 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:02.130+0000 7f292c600700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f29240fca70 0x7f2924071d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T07:58:02.130 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:02.130+0000 7f292c600700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29240722a0 con 0x7f29240fca70 2024-03-29T07:58:02.131 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:02.130+0000 7f2929c00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f29240fca70 0x7f2924071d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T07:58:02.131 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:02.130+0000 7f2929c00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f29240fca70 0x7f2924071d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.148:35414/0 (socket says 172.21.15.148:35414) 2024-03-29T07:58:02.131 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:02.130+0000 7f2929c00700 1 -- 172.21.15.148:0/604027461 learned_addr learned my addr 172.21.15.148:0/604027461 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T07:58:02.131 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:02.130+0000 7f2929c00700 1 -- 172.21.15.148:0/604027461 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2918005040 con 0x7f29240fca70 2024-03-29T07:58:02.131 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:02.131+0000 7f2929c00700 1 --2- 172.21.15.148:0/604027461 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f29240fca70 0x7f2924071d60 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f2918000c00 tx=0x7f2918009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T07:58:02.131 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:02.131+0000 7f2921600700 1 -- 172.21.15.148:0/604027461 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2918009620 con 0x7f29240fca70 2024-03-29T07:58:02.131 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:02.131+0000 7f2921600700 1 -- 172.21.15.148:0/604027461 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f2918009780 con 0x7f29240fca70 2024-03-29T07:58:02.131 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:02.131+0000 7f292c600700 1 -- 172.21.15.148:0/604027461 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2924070370 con 0x7f29240fca70 2024-03-29T07:58:02.131 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:02.131+0000 7f2921600700 1 -- 172.21.15.148:0/604027461 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f291801c5b0 con 0x7f29240fca70 2024-03-29T07:58:02.131 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:02.131+0000 7f292c600700 1 -- 172.21.15.148:0/604027461 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2924070860 con 0x7f29240fca70 2024-03-29T07:58:02.131 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:02.132+0000 7f292c600700 1 -- 172.21.15.148:0/604027461 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f292405f5f0 con 0x7f29240fca70 2024-03-29T07:58:02.132 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:02.132+0000 7f2921600700 1 -- 172.21.15.148:0/604027461 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) v1 ==== 43481+0+0 (secure 0 0 0) 0x7f2918013420 con 0x7f29240fca70 2024-03-29T07:58:02.132 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:02.133+0000 7f2921600700 1 --2- 172.21.15.148:0/604027461 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f290c0369c0 0x7f290c038e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T07:58:02.132 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:02.133+0000 7f2921600700 1 -- 172.21.15.148:0/604027461 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7f2918049d10 con 0x7f29240fca70 2024-03-29T07:58:02.132 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:02.133+0000 7f2929200700 1 --2- 172.21.15.148:0/604027461 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f290c0369c0 0x7f290c038e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T07:58:02.133 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:02.134+0000 7f2929200700 1 --2- 172.21.15.148:0/604027461 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f290c0369c0 0x7f290c038e70 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f2910009990 tx=0x7f2910006e40 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T07:58:02.137 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:02.138+0000 7f2921600700 1 -- 172.21.15.148:0/604027461 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f29180119c0 con 0x7f29240fca70 2024-03-29T07:58:02.176 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:58:01 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T07:58:02.176 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:58:01 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T07:58:02.419 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:02.420+0000 7f292c600700 1 -- 172.21.15.148:0/604027461 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f2924106b90 con 0x7f29240fca70 2024-03-29T07:58:02.419 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:02.420+0000 7f2921600700 1 -- 172.21.15.148:0/604027461 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7f2918011370 con 0x7f29240fca70 2024-03-29T07:58:02.420 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T07:58:02.420 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":1,"fsid":"99b492c2-eda1-11ee-b647-cb9ed24678a4","modified":"2024-03-29T07:55:19.261744Z","created":"2024-03-29T07:55:19.261744Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi037","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:3300","nonce":0},{"type":"v1","addr":"172.21.15.37:6789","nonce":0}]},"addr":"172.21.15.37:6789/0","public_addr":"172.21.15.37:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-29T07:58:02.423 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:02.423+0000 7f292c600700 1 -- 172.21.15.148:0/604027461 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f290c0369c0 msgr2=0x7f290c038e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T07:58:02.423 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:02.423+0000 7f292c600700 1 --2- 172.21.15.148:0/604027461 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f290c0369c0 0x7f290c038e70 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f2910009990 tx=0x7f2910006e40 comp rx=0 tx=0).stop 2024-03-29T07:58:02.423 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:02.423+0000 7f292c600700 1 -- 172.21.15.148:0/604027461 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f29240fca70 msgr2=0x7f2924071d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T07:58:02.423 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:02.423+0000 7f292c600700 1 --2- 172.21.15.148:0/604027461 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f29240fca70 0x7f2924071d60 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f2918000c00 tx=0x7f2918009e30 comp rx=0 tx=0).stop 2024-03-29T07:58:02.423 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:02.424+0000 7f292c600700 1 -- 172.21.15.148:0/604027461 shutdown_connections 2024-03-29T07:58:02.423 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:02.424+0000 7f292c600700 1 --2- 172.21.15.148:0/604027461 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f290c0369c0 0x7f290c038e70 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T07:58:02.423 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:02.424+0000 7f292c600700 1 --2- 172.21.15.148:0/604027461 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f29240fca70 0x7f2924071d60 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T07:58:02.424 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:02.424+0000 7f292c600700 1 -- 172.21.15.148:0/604027461 >> 172.21.15.148:0/604027461 conn(0x7f29240f81d0 msgr2=0x7f29240f8a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T07:58:02.424 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:02.424+0000 7f292c600700 1 -- 172.21.15.148:0/604027461 shutdown_connections 2024-03-29T07:58:02.424 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:02.424+0000 7f292c600700 1 -- 172.21.15.148:0/604027461 wait complete. 2024-03-29T07:58:02.424 INFO:teuthology.orchestra.run.smithi148.stderr:dumped monmap epoch 1 2024-03-29T07:58:03.426 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:58:02 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T07:58:03.426 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:58:02 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T07:58:03.426 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:58:02 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config rm", "who": "osd/host:smithi037", "name": "osd_memory_target"}]: dispatch 2024-03-29T07:58:03.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:58:02 smithi037 ceph-mon[35162]: from='client.? 172.21.15.148:0/604027461' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-29T07:58:04.676 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:58:04 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T07:58:04.676 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:58:04 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T07:58:04.676 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:58:04 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T07:58:04.676 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:58:04 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T07:58:04.676 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:58:04 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config rm", "who": "osd/host:smithi148", "name": "osd_memory_target"}]: dispatch 2024-03-29T07:58:04.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:58:04 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T07:58:04.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:58:04 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T07:58:04.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:58:04 smithi037 ceph-mon[35162]: Updating smithi037:/etc/ceph/ceph.conf 2024-03-29T07:58:04.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:58:04 smithi037 ceph-mon[35162]: Updating smithi148:/etc/ceph/ceph.conf 2024-03-29T07:58:04.845 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-29T07:58:04.845 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 99b492c2-eda1-11ee-b647-cb9ed24678a4 -- ceph mon dump -f json 2024-03-29T07:58:05.261 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/config/ceph.conf 2024-03-29T07:58:05.676 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:58:05 smithi037 ceph-mon[35162]: Updating smithi148:/var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/config/ceph.conf 2024-03-29T07:58:05.676 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:58:05 smithi037 ceph-mon[35162]: Updating smithi037:/var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/config/ceph.conf 2024-03-29T07:58:05.676 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:58:05 smithi037 ceph-mon[35162]: Updating smithi037:/etc/ceph/ceph.client.admin.keyring 2024-03-29T07:58:05.676 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:58:05 smithi037 ceph-mon[35162]: Updating smithi148:/etc/ceph/ceph.client.admin.keyring 2024-03-29T07:58:06.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:58:06 smithi037 ceph-mon[35162]: Updating smithi037:/var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/config/ceph.client.admin.keyring 2024-03-29T07:58:06.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:58:06 smithi037 ceph-mon[35162]: Updating smithi148:/var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/config/ceph.client.admin.keyring 2024-03-29T07:58:06.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:58:06 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T07:58:06.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:58:06 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T07:58:06.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:58:06 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T07:58:06.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:58:06 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T07:58:06.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:58:06 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T07:58:06.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:58:06 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi148", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-29T07:58:06.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:58:06 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd='[{"prefix": "auth get-or-create", "entity": "client.crash.smithi148", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]': finished 2024-03-29T07:58:06.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:58:06 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T07:58:06.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:58:06 smithi037 ceph-mon[35162]: Deploying daemon crash.smithi148 on smithi148 2024-03-29T07:58:07.646 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:07.232+0000 7ff510c00700 1 -- 172.21.15.148:0/3920624678 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff50c102ae0 msgr2=0x7ff50c102eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T07:58:07.646 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:07.232+0000 7ff510c00700 1 --2- 172.21.15.148:0/3920624678 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff50c102ae0 0x7ff50c102eb0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7ff4fc0048c0 tx=0x7ff4fc009e30 comp rx=0 tx=0).stop 2024-03-29T07:58:07.646 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:07.232+0000 7ff510c00700 1 -- 172.21.15.148:0/3920624678 shutdown_connections 2024-03-29T07:58:07.646 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:07.232+0000 7ff510c00700 1 --2- 172.21.15.148:0/3920624678 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff50c102ae0 0x7ff50c102eb0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T07:58:07.646 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:07.232+0000 7ff510c00700 1 -- 172.21.15.148:0/3920624678 >> 172.21.15.148:0/3920624678 conn(0x7ff50c0f85e0 msgr2=0x7ff50c0fa9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T07:58:07.646 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:07.232+0000 7ff510c00700 1 -- 172.21.15.148:0/3920624678 shutdown_connections 2024-03-29T07:58:07.646 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:07.233+0000 7ff510c00700 1 -- 172.21.15.148:0/3920624678 wait complete. 2024-03-29T07:58:07.646 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:07.233+0000 7ff510c00700 1 Processor -- start 2024-03-29T07:58:07.646 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:07.233+0000 7ff510c00700 1 -- start start 2024-03-29T07:58:07.647 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:07.234+0000 7ff510c00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff50c102ae0 0x7ff50c1152e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T07:58:07.647 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:07.234+0000 7ff510c00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff50c115820 con 0x7ff50c102ae0 2024-03-29T07:58:07.647 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:07.234+0000 7ff509e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff50c102ae0 0x7ff50c1152e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T07:58:07.647 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:07.234+0000 7ff509e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff50c102ae0 0x7ff50c1152e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.148:49522/0 (socket says 172.21.15.148:49522) 2024-03-29T07:58:07.647 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:07.234+0000 7ff509e00700 1 -- 172.21.15.148:0/1582161036 learned_addr learned my addr 172.21.15.148:0/1582161036 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T07:58:07.647 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:07.235+0000 7ff509e00700 1 -- 172.21.15.148:0/1582161036 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4fc005040 con 0x7ff50c102ae0 2024-03-29T07:58:07.647 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:07.235+0000 7ff509e00700 1 --2- 172.21.15.148:0/1582161036 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff50c102ae0 0x7ff50c1152e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7ff4fc007e30 tx=0x7ff4fc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T07:58:07.647 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:07.235+0000 7ff502000700 1 -- 172.21.15.148:0/1582161036 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff4fc0093c0 con 0x7ff50c102ae0 2024-03-29T07:58:07.647 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:07.235+0000 7ff510c00700 1 -- 172.21.15.148:0/1582161036 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff50c115a20 con 0x7ff50c102ae0 2024-03-29T07:58:07.647 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:07.235+0000 7ff502000700 1 -- 172.21.15.148:0/1582161036 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ff4fc013420 con 0x7ff50c102ae0 2024-03-29T07:58:07.647 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:07.235+0000 7ff502000700 1 -- 172.21.15.148:0/1582161036 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff4fc01c5b0 con 0x7ff50c102ae0 2024-03-29T07:58:07.648 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T07:58:07.648 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":1,"fsid":"99b492c2-eda1-11ee-b647-cb9ed24678a4","modified":"2024-03-29T07:55:19.261744Z","created":"2024-03-29T07:55:19.261744Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi037","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:3300","nonce":0},{"type":"v1","addr":"172.21.15.37:6789","nonce":0}]},"addr":"172.21.15.37:6789/0","public_addr":"172.21.15.37:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-29T07:58:07.648 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:07.235+0000 7ff510c00700 1 -- 172.21.15.148:0/1582161036 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff50c115e90 con 0x7ff50c102ae0 2024-03-29T07:58:07.648 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:07.236+0000 7ff502000700 1 -- 172.21.15.148:0/1582161036 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) v1 ==== 43481+0+0 (secure 0 0 0) 0x7ff4fc009520 con 0x7ff50c102ae0 2024-03-29T07:58:07.648 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:07.237+0000 7ff502000700 1 --2- 172.21.15.148:0/1582161036 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7ff4f0036970 0x7ff4f0038e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T07:58:07.648 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:07.237+0000 7ff510c00700 1 -- 172.21.15.148:0/1582161036 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff50c0fe1e0 con 0x7ff50c102ae0 2024-03-29T07:58:07.648 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:07.237+0000 7ff502000700 1 -- 172.21.15.148:0/1582161036 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7ff4fc049a50 con 0x7ff50c102ae0 2024-03-29T07:58:07.649 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:07.237+0000 7ff509400700 1 --2- 172.21.15.148:0/1582161036 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7ff4f0036970 0x7ff4f0038e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T07:58:07.649 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:07.238+0000 7ff509400700 1 --2- 172.21.15.148:0/1582161036 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7ff4f0036970 0x7ff4f0038e20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff4f8009990 tx=0x7ff4f8006e40 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T07:58:07.649 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:07.243+0000 7ff502000700 1 -- 172.21.15.148:0/1582161036 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7ff4fc013970 con 0x7ff50c102ae0 2024-03-29T07:58:07.649 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:07.532+0000 7ff510c00700 1 -- 172.21.15.148:0/1582161036 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7ff50c05f5f0 con 0x7ff50c102ae0 2024-03-29T07:58:07.649 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:07.532+0000 7ff502000700 1 -- 172.21.15.148:0/1582161036 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7ff4fc011370 con 0x7ff50c102ae0 2024-03-29T07:58:07.649 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:07.535+0000 7ff510c00700 1 -- 172.21.15.148:0/1582161036 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7ff4f0036970 msgr2=0x7ff4f0038e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T07:58:07.649 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:07.535+0000 7ff510c00700 1 --2- 172.21.15.148:0/1582161036 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7ff4f0036970 0x7ff4f0038e20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff4f8009990 tx=0x7ff4f8006e40 comp rx=0 tx=0).stop 2024-03-29T07:58:07.649 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:07.535+0000 7ff510c00700 1 -- 172.21.15.148:0/1582161036 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff50c102ae0 msgr2=0x7ff50c1152e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T07:58:07.649 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:07.535+0000 7ff510c00700 1 --2- 172.21.15.148:0/1582161036 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff50c102ae0 0x7ff50c1152e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7ff4fc007e30 tx=0x7ff4fc009e30 comp rx=0 tx=0).stop 2024-03-29T07:58:07.649 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:07.535+0000 7ff510c00700 1 -- 172.21.15.148:0/1582161036 shutdown_connections 2024-03-29T07:58:07.649 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:07.535+0000 7ff510c00700 1 --2- 172.21.15.148:0/1582161036 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7ff4f0036970 0x7ff4f0038e20 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T07:58:07.649 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:07.535+0000 7ff510c00700 1 --2- 172.21.15.148:0/1582161036 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff50c102ae0 0x7ff50c1152e0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T07:58:07.649 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:07.535+0000 7ff510c00700 1 -- 172.21.15.148:0/1582161036 >> 172.21.15.148:0/1582161036 conn(0x7ff50c0f85e0 msgr2=0x7ff50c071430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T07:58:07.649 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:07.535+0000 7ff510c00700 1 -- 172.21.15.148:0/1582161036 shutdown_connections 2024-03-29T07:58:07.650 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:07.535+0000 7ff510c00700 1 -- 172.21.15.148:0/1582161036 wait complete. 2024-03-29T07:58:07.650 INFO:teuthology.orchestra.run.smithi148.stderr:dumped monmap epoch 1 2024-03-29T07:58:07.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:58:07 smithi037 ceph-mon[35162]: from='client.? 172.21.15.148:0/1582161036' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-29T07:58:09.664 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-29T07:58:09.664 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 99b492c2-eda1-11ee-b647-cb9ed24678a4 -- ceph mon dump -f json 2024-03-29T07:58:10.241 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/config/ceph.conf 2024-03-29T07:58:11.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:58:11 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T07:58:11.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:58:11 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T07:58:11.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:58:11 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T07:58:11.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:58:11 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T07:58:11.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:58:11 smithi037 ceph-mon[35162]: Deploying daemon node-exporter.smithi148 on smithi148 2024-03-29T07:58:11.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:58:11 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T07:58:11.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:58:11 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T07:58:12.404 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:12.404+0000 7f9d30a00700 1 -- 172.21.15.148:0/1890583354 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9d2c102ac0 msgr2=0x7f9d2c102e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T07:58:12.404 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:12.404+0000 7f9d30a00700 1 --2- 172.21.15.148:0/1890583354 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9d2c102ac0 0x7f9d2c102e90 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f9d1c0048c0 tx=0x7f9d1c009e30 comp rx=0 tx=0).stop 2024-03-29T07:58:12.404 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:12.404+0000 7f9d30a00700 1 -- 172.21.15.148:0/1890583354 shutdown_connections 2024-03-29T07:58:12.405 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:12.404+0000 7f9d30a00700 1 --2- 172.21.15.148:0/1890583354 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9d2c102ac0 0x7f9d2c102e90 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T07:58:12.405 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:12.404+0000 7f9d30a00700 1 -- 172.21.15.148:0/1890583354 >> 172.21.15.148:0/1890583354 conn(0x7f9d2c0f85e0 msgr2=0x7f9d2c0fa9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T07:58:12.405 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:12.404+0000 7f9d30a00700 1 -- 172.21.15.148:0/1890583354 shutdown_connections 2024-03-29T07:58:12.405 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:12.404+0000 7f9d30a00700 1 -- 172.21.15.148:0/1890583354 wait complete. 2024-03-29T07:58:12.406 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:12.405+0000 7f9d30a00700 1 Processor -- start 2024-03-29T07:58:12.406 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:12.405+0000 7f9d30a00700 1 -- start start 2024-03-29T07:58:12.406 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:12.406+0000 7f9d30a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9d2c102ac0 0x7f9d2c115240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T07:58:12.406 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:12.406+0000 7f9d30a00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d2c115780 con 0x7f9d2c102ac0 2024-03-29T07:58:12.406 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:12.406+0000 7f9d29e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9d2c102ac0 0x7f9d2c115240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T07:58:12.406 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:12.406+0000 7f9d29e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9d2c102ac0 0x7f9d2c115240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.148:49554/0 (socket says 172.21.15.148:49554) 2024-03-29T07:58:12.406 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:12.406+0000 7f9d29e00700 1 -- 172.21.15.148:0/1101374301 learned_addr learned my addr 172.21.15.148:0/1101374301 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T07:58:12.406 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:12.406+0000 7f9d29e00700 1 -- 172.21.15.148:0/1101374301 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d1c005040 con 0x7f9d2c102ac0 2024-03-29T07:58:12.407 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:12.407+0000 7f9d29e00700 1 --2- 172.21.15.148:0/1101374301 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9d2c102ac0 0x7f9d2c115240 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f9d1c007e30 tx=0x7f9d1c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T07:58:12.407 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:12.407+0000 7f9d22000700 1 -- 172.21.15.148:0/1101374301 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9d1c0093c0 con 0x7f9d2c102ac0 2024-03-29T07:58:12.407 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:12.407+0000 7f9d30a00700 1 -- 172.21.15.148:0/1101374301 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d2c115980 con 0x7f9d2c102ac0 2024-03-29T07:58:12.407 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:12.407+0000 7f9d30a00700 1 -- 172.21.15.148:0/1101374301 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d2c115e70 con 0x7f9d2c102ac0 2024-03-29T07:58:12.407 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:12.407+0000 7f9d22000700 1 -- 172.21.15.148:0/1101374301 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f9d1c013420 con 0x7f9d2c102ac0 2024-03-29T07:58:12.407 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:12.407+0000 7f9d22000700 1 -- 172.21.15.148:0/1101374301 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9d1c0255c0 con 0x7f9d2c102ac0 2024-03-29T07:58:12.407 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:12.408+0000 7f9d22000700 1 -- 172.21.15.148:0/1101374301 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) v1 ==== 43481+0+0 (secure 0 0 0) 0x7f9d1c01c3f0 con 0x7f9d2c102ac0 2024-03-29T07:58:12.408 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:12.408+0000 7f9d30a00700 1 -- 172.21.15.148:0/1101374301 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d2c0fe1c0 con 0x7f9d2c102ac0 2024-03-29T07:58:12.408 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:12.409+0000 7f9d22000700 1 --2- 172.21.15.148:0/1101374301 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f9d10036970 0x7f9d10038e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T07:58:12.408 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:12.409+0000 7f9d22000700 1 -- 172.21.15.148:0/1101374301 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7f9d1c049ab0 con 0x7f9d2c102ac0 2024-03-29T07:58:12.408 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:12.409+0000 7f9d29400700 1 --2- 172.21.15.148:0/1101374301 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f9d10036970 0x7f9d10038e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T07:58:12.409 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:12.410+0000 7f9d29400700 1 --2- 172.21.15.148:0/1101374301 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f9d10036970 0x7f9d10038e20 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f9d18009990 tx=0x7f9d18006e40 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T07:58:12.413 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:12.414+0000 7f9d22000700 1 -- 172.21.15.148:0/1101374301 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f9d1c01cc20 con 0x7f9d2c102ac0 2024-03-29T07:58:12.716 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:12.716+0000 7f9d30a00700 1 -- 172.21.15.148:0/1101374301 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f9d2c05f5f0 con 0x7f9d2c102ac0 2024-03-29T07:58:12.716 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:12.717+0000 7f9d22000700 1 -- 172.21.15.148:0/1101374301 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7f9d1c011370 con 0x7f9d2c102ac0 2024-03-29T07:58:12.717 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T07:58:12.717 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":1,"fsid":"99b492c2-eda1-11ee-b647-cb9ed24678a4","modified":"2024-03-29T07:55:19.261744Z","created":"2024-03-29T07:55:19.261744Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi037","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:3300","nonce":0},{"type":"v1","addr":"172.21.15.37:6789","nonce":0}]},"addr":"172.21.15.37:6789/0","public_addr":"172.21.15.37:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-29T07:58:12.719 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:12.719+0000 7f9d30a00700 1 -- 172.21.15.148:0/1101374301 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f9d10036970 msgr2=0x7f9d10038e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T07:58:12.719 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:12.719+0000 7f9d30a00700 1 --2- 172.21.15.148:0/1101374301 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f9d10036970 0x7f9d10038e20 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f9d18009990 tx=0x7f9d18006e40 comp rx=0 tx=0).stop 2024-03-29T07:58:12.719 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:12.719+0000 7f9d30a00700 1 -- 172.21.15.148:0/1101374301 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9d2c102ac0 msgr2=0x7f9d2c115240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T07:58:12.719 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:12.719+0000 7f9d30a00700 1 --2- 172.21.15.148:0/1101374301 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9d2c102ac0 0x7f9d2c115240 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f9d1c007e30 tx=0x7f9d1c009e30 comp rx=0 tx=0).stop 2024-03-29T07:58:12.719 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:12.719+0000 7f9d30a00700 1 -- 172.21.15.148:0/1101374301 shutdown_connections 2024-03-29T07:58:12.719 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:12.719+0000 7f9d30a00700 1 --2- 172.21.15.148:0/1101374301 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f9d10036970 0x7f9d10038e20 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T07:58:12.719 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:12.719+0000 7f9d30a00700 1 --2- 172.21.15.148:0/1101374301 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9d2c102ac0 0x7f9d2c115240 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T07:58:12.719 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:12.719+0000 7f9d30a00700 1 -- 172.21.15.148:0/1101374301 >> 172.21.15.148:0/1101374301 conn(0x7f9d2c0f85e0 msgr2=0x7f9d2c071430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T07:58:12.719 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:12.719+0000 7f9d30a00700 1 -- 172.21.15.148:0/1101374301 shutdown_connections 2024-03-29T07:58:12.719 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:12.719+0000 7f9d30a00700 1 -- 172.21.15.148:0/1101374301 wait complete. 2024-03-29T07:58:12.720 INFO:teuthology.orchestra.run.smithi148.stderr:dumped monmap epoch 1 2024-03-29T07:58:13.176 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:58:12 smithi037 ceph-mon[35162]: from='client.? 172.21.15.148:0/1101374301' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-29T07:58:14.682 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-29T07:58:14.682 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 99b492c2-eda1-11ee-b647-cb9ed24678a4 -- ceph mon dump -f json 2024-03-29T07:58:15.095 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/config/ceph.conf 2024-03-29T07:58:16.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:58:16 smithi037 ceph-mon[35162]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-29T07:58:17.780 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:17.774+0000 7f1f9fc00700 1 -- 172.21.15.148:0/2413373041 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1f98102ac0 msgr2=0x7f1f98102e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T07:58:17.780 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:17.774+0000 7f1f9fc00700 1 --2- 172.21.15.148:0/2413373041 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1f98102ac0 0x7f1f98102e90 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f1f8c0048c0 tx=0x7f1f8c009e30 comp rx=0 tx=0).stop 2024-03-29T07:58:17.780 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:17.774+0000 7f1f9fc00700 1 -- 172.21.15.148:0/2413373041 shutdown_connections 2024-03-29T07:58:17.780 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:17.774+0000 7f1f9fc00700 1 --2- 172.21.15.148:0/2413373041 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1f98102ac0 0x7f1f98102e90 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T07:58:17.780 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:17.774+0000 7f1f9fc00700 1 -- 172.21.15.148:0/2413373041 >> 172.21.15.148:0/2413373041 conn(0x7f1f980f85e0 msgr2=0x7f1f980fa9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T07:58:17.780 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:17.774+0000 7f1f9fc00700 1 -- 172.21.15.148:0/2413373041 shutdown_connections 2024-03-29T07:58:17.780 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:17.774+0000 7f1f9fc00700 1 -- 172.21.15.148:0/2413373041 wait complete. 2024-03-29T07:58:17.780 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:17.775+0000 7f1f9fc00700 1 Processor -- start 2024-03-29T07:58:17.780 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:17.775+0000 7f1f9fc00700 1 -- start start 2024-03-29T07:58:17.781 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:17.775+0000 7f1f9fc00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1f98102ac0 0x7f1f980702c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T07:58:17.781 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:17.775+0000 7f1f9fc00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f980722c0 con 0x7f1f98102ac0 2024-03-29T07:58:17.781 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:17.775+0000 7f1f9d200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1f98102ac0 0x7f1f980702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T07:58:17.781 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:17.775+0000 7f1f9d200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1f98102ac0 0x7f1f980702c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.148:54470/0 (socket says 172.21.15.148:54470) 2024-03-29T07:58:17.781 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:17.775+0000 7f1f9d200700 1 -- 172.21.15.148:0/4149743460 learned_addr learned my addr 172.21.15.148:0/4149743460 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T07:58:17.781 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:17.776+0000 7f1f9d200700 1 -- 172.21.15.148:0/4149743460 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f8c005040 con 0x7f1f98102ac0 2024-03-29T07:58:17.781 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:17.776+0000 7f1f9d200700 1 --2- 172.21.15.148:0/4149743460 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1f98102ac0 0x7f1f980702c0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f1f8c007e30 tx=0x7f1f8c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T07:58:17.781 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:17.776+0000 7f1f94c00700 1 -- 172.21.15.148:0/4149743460 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1f8c0093c0 con 0x7f1f98102ac0 2024-03-29T07:58:17.781 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:17.776+0000 7f1f9fc00700 1 -- 172.21.15.148:0/4149743460 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f98070860 con 0x7f1f98102ac0 2024-03-29T07:58:17.781 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:17.776+0000 7f1f94c00700 1 -- 172.21.15.148:0/4149743460 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f1f8c013420 con 0x7f1f98102ac0 2024-03-29T07:58:17.781 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:17.776+0000 7f1f94c00700 1 -- 172.21.15.148:0/4149743460 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1f8c01c5b0 con 0x7f1f98102ac0 2024-03-29T07:58:17.781 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:17.777+0000 7f1f9fc00700 1 -- 172.21.15.148:0/4149743460 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f98070d50 con 0x7f1f98102ac0 2024-03-29T07:58:17.781 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:17.777+0000 7f1f9fc00700 1 -- 172.21.15.148:0/4149743460 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f98101d30 con 0x7f1f98102ac0 2024-03-29T07:58:17.781 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:17.778+0000 7f1f94c00700 1 -- 172.21.15.148:0/4149743460 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) v1 ==== 43481+0+0 (secure 0 0 0) 0x7f1f8c013970 con 0x7f1f98102ac0 2024-03-29T07:58:17.782 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:17.778+0000 7f1f94c00700 1 --2- 172.21.15.148:0/4149743460 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f1f80036970 0x7f1f80038e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T07:58:17.782 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:17.778+0000 7f1f94c00700 1 -- 172.21.15.148:0/4149743460 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7f1f8c049b10 con 0x7f1f98102ac0 2024-03-29T07:58:17.782 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:17.779+0000 7f1f97e00700 1 --2- 172.21.15.148:0/4149743460 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f1f80036970 0x7f1f80038e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T07:58:17.782 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:17.780+0000 7f1f97e00700 1 --2- 172.21.15.148:0/4149743460 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f1f80036970 0x7f1f80038e20 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f1f84009990 tx=0x7f1f84006e40 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T07:58:17.783 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:17.784+0000 7f1f94c00700 1 -- 172.21.15.148:0/4149743460 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f1f8c011960 con 0x7f1f98102ac0 2024-03-29T07:58:18.069 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:18.069+0000 7f1f9fc00700 1 -- 172.21.15.148:0/4149743460 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f1f9804bed0 con 0x7f1f98102ac0 2024-03-29T07:58:18.069 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:18.070+0000 7f1f94c00700 1 -- 172.21.15.148:0/4149743460 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7f1f8c01b390 con 0x7f1f98102ac0 2024-03-29T07:58:18.069 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T07:58:18.070 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":1,"fsid":"99b492c2-eda1-11ee-b647-cb9ed24678a4","modified":"2024-03-29T07:55:19.261744Z","created":"2024-03-29T07:55:19.261744Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi037","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:3300","nonce":0},{"type":"v1","addr":"172.21.15.37:6789","nonce":0}]},"addr":"172.21.15.37:6789/0","public_addr":"172.21.15.37:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-29T07:58:18.072 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:18.072+0000 7f1f9fc00700 1 -- 172.21.15.148:0/4149743460 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f1f80036970 msgr2=0x7f1f80038e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T07:58:18.072 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:18.072+0000 7f1f9fc00700 1 --2- 172.21.15.148:0/4149743460 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f1f80036970 0x7f1f80038e20 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f1f84009990 tx=0x7f1f84006e40 comp rx=0 tx=0).stop 2024-03-29T07:58:18.072 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:18.072+0000 7f1f9fc00700 1 -- 172.21.15.148:0/4149743460 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1f98102ac0 msgr2=0x7f1f980702c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T07:58:18.073 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:18.072+0000 7f1f9fc00700 1 --2- 172.21.15.148:0/4149743460 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1f98102ac0 0x7f1f980702c0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f1f8c007e30 tx=0x7f1f8c009e30 comp rx=0 tx=0).stop 2024-03-29T07:58:18.073 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:18.073+0000 7f1f9fc00700 1 -- 172.21.15.148:0/4149743460 shutdown_connections 2024-03-29T07:58:18.073 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:18.073+0000 7f1f9fc00700 1 --2- 172.21.15.148:0/4149743460 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f1f80036970 0x7f1f80038e20 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T07:58:18.073 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:18.073+0000 7f1f9fc00700 1 --2- 172.21.15.148:0/4149743460 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1f98102ac0 0x7f1f980702c0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T07:58:18.073 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:18.073+0000 7f1f9fc00700 1 -- 172.21.15.148:0/4149743460 >> 172.21.15.148:0/4149743460 conn(0x7f1f980f85e0 msgr2=0x7f1f980f9d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T07:58:18.073 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:18.073+0000 7f1f9fc00700 1 -- 172.21.15.148:0/4149743460 shutdown_connections 2024-03-29T07:58:18.073 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:18.073+0000 7f1f9fc00700 1 -- 172.21.15.148:0/4149743460 wait complete. 2024-03-29T07:58:18.074 INFO:teuthology.orchestra.run.smithi148.stderr:dumped monmap epoch 1 2024-03-29T07:58:18.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:58:18 smithi037 ceph-mon[35162]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-29T07:58:18.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:58:18 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T07:58:18.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:58:18 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T07:58:18.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:58:18 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T07:58:18.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:58:18 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T07:58:18.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:58:18 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi148.onhgvm", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-29T07:58:18.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:58:18 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.smithi148.onhgvm", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-29T07:58:18.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:58:18 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-29T07:58:18.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:58:18 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T07:58:18.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:58:18 smithi037 ceph-mon[35162]: Deploying daemon mgr.smithi148.onhgvm on smithi148 2024-03-29T07:58:18.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:58:18 smithi037 ceph-mon[35162]: from='client.? 172.21.15.148:0/4149743460' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-29T07:58:19.861 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-29T07:58:19.861 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 99b492c2-eda1-11ee-b647-cb9ed24678a4 -- ceph mon dump -f json 2024-03-29T07:58:20.296 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/config/ceph.conf 2024-03-29T07:58:20.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:58:20 smithi037 ceph-mon[35162]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-29T07:58:20.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:58:20 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T07:58:22.509 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:22.445+0000 7ff5a1200700 1 -- 172.21.15.148:0/149464027 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff59c070360 msgr2=0x7ff59c070730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T07:58:22.509 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:22.445+0000 7ff5a1200700 1 --2- 172.21.15.148:0/149464027 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff59c070360 0x7ff59c070730 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7ff58c0048c0 tx=0x7ff58c009e30 comp rx=0 tx=0).stop 2024-03-29T07:58:22.509 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:22.445+0000 7ff5a1200700 1 -- 172.21.15.148:0/149464027 shutdown_connections 2024-03-29T07:58:22.509 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:22.445+0000 7ff5a1200700 1 --2- 172.21.15.148:0/149464027 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff59c070360 0x7ff59c070730 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T07:58:22.509 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:22.445+0000 7ff5a1200700 1 -- 172.21.15.148:0/149464027 >> 172.21.15.148:0/149464027 conn(0x7ff59c0f85e0 msgr2=0x7ff59c0fa9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T07:58:22.510 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:22.445+0000 7ff5a1200700 1 -- 172.21.15.148:0/149464027 shutdown_connections 2024-03-29T07:58:22.510 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:22.445+0000 7ff5a1200700 1 -- 172.21.15.148:0/149464027 wait complete. 2024-03-29T07:58:22.510 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:22.446+0000 7ff5a1200700 1 Processor -- start 2024-03-29T07:58:22.510 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:22.447+0000 7ff5a1200700 1 -- start start 2024-03-29T07:58:22.510 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:22.447+0000 7ff5a1200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff59c070360 0x7ff59c1152d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T07:58:22.510 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:22.447+0000 7ff5a1200700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff59c115810 con 0x7ff59c070360 2024-03-29T07:58:22.510 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:22.448+0000 7ff599e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff59c070360 0x7ff59c1152d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T07:58:22.510 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:22.448+0000 7ff599e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff59c070360 0x7ff59c1152d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.148:54514/0 (socket says 172.21.15.148:54514) 2024-03-29T07:58:22.510 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:22.448+0000 7ff599e00700 1 -- 172.21.15.148:0/1073474550 learned_addr learned my addr 172.21.15.148:0/1073474550 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T07:58:22.510 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:22.448+0000 7ff599e00700 1 -- 172.21.15.148:0/1073474550 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff58c005040 con 0x7ff59c070360 2024-03-29T07:58:22.510 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:22.449+0000 7ff599e00700 1 --2- 172.21.15.148:0/1073474550 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff59c070360 0x7ff59c1152d0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7ff58c007e30 tx=0x7ff58c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T07:58:22.510 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:22.449+0000 7ff592000700 1 -- 172.21.15.148:0/1073474550 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff58c0093c0 con 0x7ff59c070360 2024-03-29T07:58:22.511 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:22.449+0000 7ff5a1200700 1 -- 172.21.15.148:0/1073474550 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff59c115a10 con 0x7ff59c070360 2024-03-29T07:58:22.511 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:22.449+0000 7ff592000700 1 -- 172.21.15.148:0/1073474550 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ff58c013420 con 0x7ff59c070360 2024-03-29T07:58:22.511 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:22.449+0000 7ff592000700 1 -- 172.21.15.148:0/1073474550 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff58c01c5b0 con 0x7ff59c070360 2024-03-29T07:58:22.511 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:22.449+0000 7ff5a1200700 1 -- 172.21.15.148:0/1073474550 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff59c115e80 con 0x7ff59c070360 2024-03-29T07:58:22.511 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:22.450+0000 7ff5a1200700 1 -- 172.21.15.148:0/1073474550 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff59c071aa0 con 0x7ff59c070360 2024-03-29T07:58:22.511 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:22.451+0000 7ff592000700 1 -- 172.21.15.148:0/1073474550 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) v1 ==== 43481+0+0 (secure 0 0 0) 0x7ff58c009520 con 0x7ff59c070360 2024-03-29T07:58:22.511 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:22.451+0000 7ff592000700 1 --2- 172.21.15.148:0/1073474550 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7ff580036970 0x7ff580038e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T07:58:22.511 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:22.451+0000 7ff592000700 1 -- 172.21.15.148:0/1073474550 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7ff58c049a50 con 0x7ff59c070360 2024-03-29T07:58:22.511 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:22.453+0000 7ff599400700 1 --2- 172.21.15.148:0/1073474550 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7ff580036970 0x7ff580038e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T07:58:22.511 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:22.454+0000 7ff599400700 1 --2- 172.21.15.148:0/1073474550 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7ff580036970 0x7ff580038e20 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7ff588009990 tx=0x7ff588006e40 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T07:58:22.511 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:22.457+0000 7ff592000700 1 -- 172.21.15.148:0/1073474550 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7ff58c013970 con 0x7ff59c070360 2024-03-29T07:58:22.790 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:22.790+0000 7ff5a1200700 1 -- 172.21.15.148:0/1073474550 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7ff59c04bed0 con 0x7ff59c070360 2024-03-29T07:58:22.790 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:22.791+0000 7ff592000700 1 -- 172.21.15.148:0/1073474550 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7ff58c011370 con 0x7ff59c070360 2024-03-29T07:58:22.790 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T07:58:22.790 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":1,"fsid":"99b492c2-eda1-11ee-b647-cb9ed24678a4","modified":"2024-03-29T07:55:19.261744Z","created":"2024-03-29T07:55:19.261744Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi037","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:3300","nonce":0},{"type":"v1","addr":"172.21.15.37:6789","nonce":0}]},"addr":"172.21.15.37:6789/0","public_addr":"172.21.15.37:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-29T07:58:22.793 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:22.793+0000 7ff5a1200700 1 -- 172.21.15.148:0/1073474550 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7ff580036970 msgr2=0x7ff580038e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T07:58:22.793 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:22.793+0000 7ff5a1200700 1 --2- 172.21.15.148:0/1073474550 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7ff580036970 0x7ff580038e20 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7ff588009990 tx=0x7ff588006e40 comp rx=0 tx=0).stop 2024-03-29T07:58:22.793 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:22.793+0000 7ff5a1200700 1 -- 172.21.15.148:0/1073474550 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff59c070360 msgr2=0x7ff59c1152d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T07:58:22.793 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:22.793+0000 7ff5a1200700 1 --2- 172.21.15.148:0/1073474550 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff59c070360 0x7ff59c1152d0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7ff58c007e30 tx=0x7ff58c009e30 comp rx=0 tx=0).stop 2024-03-29T07:58:22.793 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:22.793+0000 7ff5a1200700 1 -- 172.21.15.148:0/1073474550 shutdown_connections 2024-03-29T07:58:22.793 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:22.793+0000 7ff5a1200700 1 --2- 172.21.15.148:0/1073474550 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7ff580036970 0x7ff580038e20 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T07:58:22.793 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:22.793+0000 7ff5a1200700 1 --2- 172.21.15.148:0/1073474550 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff59c070360 0x7ff59c1152d0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T07:58:22.793 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:22.793+0000 7ff5a1200700 1 -- 172.21.15.148:0/1073474550 >> 172.21.15.148:0/1073474550 conn(0x7ff59c0f85e0 msgr2=0x7ff59c105b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T07:58:22.793 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:22.794+0000 7ff5a1200700 1 -- 172.21.15.148:0/1073474550 shutdown_connections 2024-03-29T07:58:22.793 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:22.794+0000 7ff5a1200700 1 -- 172.21.15.148:0/1073474550 wait complete. 2024-03-29T07:58:22.794 INFO:teuthology.orchestra.run.smithi148.stderr:dumped monmap epoch 1 2024-03-29T07:58:22.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:58:22 smithi037 ceph-mon[35162]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-29T07:58:22.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:58:22 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T07:58:22.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:58:22 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T07:58:22.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:58:22 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T07:58:22.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:58:22 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T07:58:22.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:58:22 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-29T07:58:22.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:58:22 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T07:58:23.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:58:23 smithi037 ceph-mon[35162]: Deploying daemon mon.smithi148 on smithi148 2024-03-29T07:58:23.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:58:23 smithi037 ceph-mon[35162]: from='client.? 172.21.15.148:0/1073474550' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-29T07:58:24.516 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-29T07:58:24.517 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 99b492c2-eda1-11ee-b647-cb9ed24678a4 -- ceph mon dump -f json 2024-03-29T07:58:24.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:58:24 smithi037 ceph-mon[35162]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-29T07:58:25.612 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/config/ceph.conf 2024-03-29T07:58:25.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:58:25 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T07:58:25.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:58:25 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T07:58:26.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:58:26 smithi037 ceph-mon[35162]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-29T07:58:28.259 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:58:28 smithi148 systemd[1]: Started Ceph mon.smithi148 for 99b492c2-eda1-11ee-b647-cb9ed24678a4. 2024-03-29T07:58:28.260 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:58:28 smithi148 ceph-mon[44992]: mon.smithi148@-1(synchronizing).mds e1 new map 2024-03-29T07:58:28.260 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:58:28 smithi148 ceph-mon[44992]: mon.smithi148@-1(synchronizing).mds e1 print_map 2024-03-29T07:58:28.260 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: e1 2024-03-29T07:58:28.260 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-03-29T07:58:28.260 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T07:58:28.260 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: legacy client fscid: -1 2024-03-29T07:58:28.260 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: 2024-03-29T07:58:28.260 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: No filesystems configured 2024-03-29T07:58:28.260 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:58:28 smithi148 ceph-mon[44992]: mon.smithi148@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-03-29T07:58:28.260 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:58:28 smithi148 ceph-mon[44992]: mon.smithi148@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-03-29T07:58:28.260 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:58:28 smithi148 ceph-mon[44992]: mon.smithi148@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-03-29T07:58:28.260 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:58:28 smithi148 ceph-mon[44992]: mon.smithi148@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-03-29T07:58:28.260 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:58:28 smithi148 ceph-mon[44992]: mon.smithi148@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-03-29T07:58:28.260 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:58:28 smithi148 ceph-mon[44992]: mon.smithi148@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-03-29T07:58:28.261 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:58:28 smithi148 ceph-mon[44992]: mon.smithi148@-1(synchronizing).osd e5 e5: 0 total, 0 up, 0 in 2024-03-29T07:58:28.261 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:58:28 smithi148 ceph-mon[44992]: mon.smithi148@-1(synchronizing).osd e5 crush map has features 3314932999778484224, adjusting msgr requires 2024-03-29T07:58:28.261 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:58:28 smithi148 ceph-mon[44992]: mon.smithi148@-1(synchronizing).osd e5 crush map has features 288514050185494528, adjusting msgr requires 2024-03-29T07:58:28.261 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:58:28 smithi148 ceph-mon[44992]: mon.smithi148@-1(synchronizing).osd e5 crush map has features 288514050185494528, adjusting msgr requires 2024-03-29T07:58:28.261 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:58:28 smithi148 ceph-mon[44992]: mon.smithi148@-1(synchronizing).osd e5 crush map has features 288514050185494528, adjusting msgr requires 2024-03-29T07:58:28.261 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:58:28 smithi148 ceph-mon[44992]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-29T07:58:28.305 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:58:28 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T07:58:28.305 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:58:28 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T07:58:28.305 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:58:28 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T07:58:28.305 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:58:28 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T07:58:28.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:58:28 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi148.onhgvm", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-29T07:58:28.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:58:28 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.smithi148.onhgvm", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-29T07:58:28.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:58:28 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-29T07:58:28.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:58:28 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T07:58:28.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:58:28 smithi148 ceph-mon[44992]: Deploying daemon mgr.smithi148.onhgvm on smithi148 2024-03-29T07:58:28.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:58:28 smithi148 ceph-mon[44992]: from='client.? 172.21.15.148:0/4149743460' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-29T07:58:28.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:58:28 smithi148 ceph-mon[44992]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-29T07:58:28.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:58:28 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T07:58:28.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:58:28 smithi148 ceph-mon[44992]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-29T07:58:28.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:58:28 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T07:58:28.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:58:28 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T07:58:28.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:58:28 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T07:58:28.307 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:58:28 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T07:58:28.307 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:58:28 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-29T07:58:28.307 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:58:28 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T07:58:28.307 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:58:28 smithi148 ceph-mon[44992]: Deploying daemon mon.smithi148 on smithi148 2024-03-29T07:58:28.307 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:58:28 smithi148 ceph-mon[44992]: from='client.? 172.21.15.148:0/1073474550' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-29T07:58:28.307 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:58:28 smithi148 ceph-mon[44992]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-29T07:58:28.307 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:58:28 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T07:58:28.307 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:58:28 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T07:58:28.307 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:58:28 smithi148 ceph-mon[44992]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-29T07:58:28.307 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:58:28 smithi148 ceph-mon[44992]: mon.smithi148@-1(synchronizing).paxosservice(auth 1..6) refresh upgraded, format 0 -> 3 2024-03-29T07:58:28.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:28.155+0000 7f119a800700 1 -- 172.21.15.148:0/2888840483 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1194070360 msgr2=0x7f1194070730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T07:58:28.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:28.155+0000 7f119a800700 1 --2- 172.21.15.148:0/2888840483 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1194070360 0x7f1194070730 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f11840048c0 tx=0x7f1184009e30 comp rx=0 tx=0).stop 2024-03-29T07:58:28.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:28.155+0000 7f119a800700 1 -- 172.21.15.148:0/2888840483 shutdown_connections 2024-03-29T07:58:28.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:28.155+0000 7f119a800700 1 --2- 172.21.15.148:0/2888840483 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1194070360 0x7f1194070730 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T07:58:28.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:28.155+0000 7f119a800700 1 -- 172.21.15.148:0/2888840483 >> 172.21.15.148:0/2888840483 conn(0x7f11940f85e0 msgr2=0x7f11940fa9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T07:58:28.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:28.155+0000 7f119a800700 1 -- 172.21.15.148:0/2888840483 shutdown_connections 2024-03-29T07:58:28.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:28.155+0000 7f119a800700 1 -- 172.21.15.148:0/2888840483 wait complete. 2024-03-29T07:58:28.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:28.156+0000 7f119a800700 1 Processor -- start 2024-03-29T07:58:28.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:28.156+0000 7f119a800700 1 -- start start 2024-03-29T07:58:28.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:28.156+0000 7f119a800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1194070360 0x7f11941152d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T07:58:28.398 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:28.156+0000 7f119a800700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1194115810 con 0x7f1194070360 2024-03-29T07:58:28.398 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:28.157+0000 7f1193400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1194070360 0x7f11941152d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T07:58:28.398 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:28.157+0000 7f1193400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1194070360 0x7f11941152d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.148:47674/0 (socket says 172.21.15.148:47674) 2024-03-29T07:58:28.398 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:28.157+0000 7f1193400700 1 -- 172.21.15.148:0/3040965198 learned_addr learned my addr 172.21.15.148:0/3040965198 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T07:58:28.398 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:28.157+0000 7f1193400700 1 -- 172.21.15.148:0/3040965198 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1184005040 con 0x7f1194070360 2024-03-29T07:58:28.398 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:28.158+0000 7f1193400700 1 --2- 172.21.15.148:0/3040965198 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1194070360 0x7f11941152d0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f1184007e30 tx=0x7f1184009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T07:58:28.398 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:28.158+0000 7f118b400700 1 -- 172.21.15.148:0/3040965198 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f11840093c0 con 0x7f1194070360 2024-03-29T07:58:28.398 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:28.158+0000 7f118b400700 1 -- 172.21.15.148:0/3040965198 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f1184013420 con 0x7f1194070360 2024-03-29T07:58:28.398 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:28.158+0000 7f118b400700 1 -- 172.21.15.148:0/3040965198 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f118401c5b0 con 0x7f1194070360 2024-03-29T07:58:28.398 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:28.158+0000 7f119a800700 1 -- 172.21.15.148:0/3040965198 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1194115a10 con 0x7f1194070360 2024-03-29T07:58:28.398 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:28.158+0000 7f119a800700 1 -- 172.21.15.148:0/3040965198 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1194115e80 con 0x7f1194070360 2024-03-29T07:58:28.398 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:28.159+0000 7f118b400700 1 -- 172.21.15.148:0/3040965198 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) v1 ==== 43481+0+0 (secure 0 0 0) 0x7f1184013970 con 0x7f1194070360 2024-03-29T07:58:28.398 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:28.159+0000 7f119a800700 1 -- 172.21.15.148:0/3040965198 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1194071aa0 con 0x7f1194070360 2024-03-29T07:58:28.399 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:28.160+0000 7f118b400700 1 --2- 172.21.15.148:0/3040965198 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f11780369c0 0x7f1178038e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T07:58:28.399 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:28.160+0000 7f118b400700 1 -- 172.21.15.148:0/3040965198 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7f1184049b10 con 0x7f1194070360 2024-03-29T07:58:28.399 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:28.160+0000 7f1192a00700 1 --2- 172.21.15.148:0/3040965198 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f11780369c0 0x7f1178038e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T07:58:28.399 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:28.161+0000 7f1192a00700 1 --2- 172.21.15.148:0/3040965198 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f11780369c0 0x7f1178038e70 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f1180009990 tx=0x7f1180006e40 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T07:58:28.399 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:28.165+0000 7f118b400700 1 -- 172.21.15.148:0/3040965198 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f1184011960 con 0x7f1194070360 2024-03-29T07:58:28.399 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:28.271+0000 7f118b400700 1 -- 172.21.15.148:0/3040965198 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f11840492a0 con 0x7f1194070360 2024-03-29T07:58:28.517 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:28.518+0000 7f119a800700 1 -- 172.21.15.148:0/3040965198 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f119405f5f0 con 0x7f1194070360 2024-03-29T07:58:33.289 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:33.289+0000 7f118b400700 1 -- 172.21.15.148:0/3040965198 <== mon.0 v2:172.21.15.37:3300/0 8 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) v1 ==== 95+0+1017 (secure 0 0 0) 0x7f118401b390 con 0x7f1194070360 2024-03-29T07:58:33.290 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-03-29T07:58:33.290 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":2,"fsid":"99b492c2-eda1-11ee-b647-cb9ed24678a4","modified":"2024-03-29T07:58:28.271116Z","created":"2024-03-29T07:55:19.261744Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi037","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:3300","nonce":0},{"type":"v1","addr":"172.21.15.37:6789","nonce":0}]},"addr":"172.21.15.37:6789/0","public_addr":"172.21.15.37:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"smithi148","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:3300","nonce":0},{"type":"v1","addr":"172.21.15.148:6789","nonce":0}]},"addr":"172.21.15.148:6789/0","public_addr":"172.21.15.148:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-03-29T07:58:33.293 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:33.293+0000 7f119a800700 1 -- 172.21.15.148:0/3040965198 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f11780369c0 msgr2=0x7f1178038e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T07:58:33.293 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:33.293+0000 7f119a800700 1 --2- 172.21.15.148:0/3040965198 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f11780369c0 0x7f1178038e70 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f1180009990 tx=0x7f1180006e40 comp rx=0 tx=0).stop 2024-03-29T07:58:33.293 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:33.293+0000 7f119a800700 1 -- 172.21.15.148:0/3040965198 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1194070360 msgr2=0x7f11941152d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T07:58:33.293 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:33.293+0000 7f119a800700 1 --2- 172.21.15.148:0/3040965198 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1194070360 0x7f11941152d0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f1184007e30 tx=0x7f1184009e30 comp rx=0 tx=0).stop 2024-03-29T07:58:33.293 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:33.293+0000 7f119a800700 1 -- 172.21.15.148:0/3040965198 shutdown_connections 2024-03-29T07:58:33.293 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:33.293+0000 7f119a800700 1 --2- 172.21.15.148:0/3040965198 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f11780369c0 0x7f1178038e70 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T07:58:33.294 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:33.293+0000 7f119a800700 1 --2- 172.21.15.148:0/3040965198 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1194070360 0x7f11941152d0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T07:58:33.294 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:33.293+0000 7f119a800700 1 -- 172.21.15.148:0/3040965198 >> 172.21.15.148:0/3040965198 conn(0x7f11940f85e0 msgr2=0x7f1194105b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T07:58:33.294 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:33.294+0000 7f119a800700 1 -- 172.21.15.148:0/3040965198 shutdown_connections 2024-03-29T07:58:33.294 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T07:58:33.294+0000 7f119a800700 1 -- 172.21.15.148:0/3040965198 wait complete. 2024-03-29T07:58:33.294 INFO:teuthology.orchestra.run.smithi148.stderr:dumped monmap epoch 2 2024-03-29T07:58:33.676 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:58:33 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "mon metadata", "id": "smithi037"}]: dispatch 2024-03-29T07:58:33.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:58:33 smithi037 ceph-mon[35162]: mon.smithi037 calling monitor election 2024-03-29T07:58:33.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:58:33 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "mon metadata", "id": "smithi148"}]: dispatch 2024-03-29T07:58:33.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:58:33 smithi037 ceph-mon[35162]: from='client.? 172.21.15.148:0/3040965198' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-29T07:58:33.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:58:33 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "mon metadata", "id": "smithi148"}]: dispatch 2024-03-29T07:58:33.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:58:33 smithi037 ceph-mon[35162]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-29T07:58:33.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:58:33 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "mon metadata", "id": "smithi148"}]: dispatch 2024-03-29T07:58:33.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:58:33 smithi037 ceph-mon[35162]: mon.smithi148 calling monitor election 2024-03-29T07:58:33.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:58:33 smithi037 ceph-mon[35162]: from='mgr.? 172.21.15.148:0/3101923077' entity='mgr.smithi148.onhgvm' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi148.onhgvm/crt"}]: dispatch 2024-03-29T07:58:33.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:58:33 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "mon metadata", "id": "smithi148"}]: dispatch 2024-03-29T07:58:33.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:58:33 smithi037 ceph-mon[35162]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-29T07:58:33.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:58:33 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "mon metadata", "id": "smithi148"}]: dispatch 2024-03-29T07:58:33.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:58:33 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "mon metadata", "id": "smithi148"}]: dispatch 2024-03-29T07:58:33.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:58:33 smithi037 ceph-mon[35162]: mon.smithi037 is new leader, mons smithi037,smithi148 in quorum (ranks 0,1) 2024-03-29T07:58:33.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:58:33 smithi037 ceph-mon[35162]: monmap e2: 2 mons at {smithi037=[v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0],smithi148=[v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-29T07:58:33.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:58:33 smithi037 ceph-mon[35162]: fsmap 2024-03-29T07:58:33.678 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:58:33 smithi037 ceph-mon[35162]: osdmap e5: 0 total, 0 up, 0 in 2024-03-29T07:58:33.678 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:58:33 smithi037 ceph-mon[35162]: mgrmap e17: smithi037.wjikii(active, since 37s) 2024-03-29T07:58:33.678 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:58:33 smithi037 ceph-mon[35162]: overall HEALTH_OK 2024-03-29T07:58:33.678 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:58:33 smithi037 ceph-mon[35162]: Standby manager daemon smithi148.onhgvm started 2024-03-29T07:58:33.678 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:58:33 smithi037 ceph-mon[35162]: from='mgr.? 172.21.15.148:0/3101923077' entity='mgr.smithi148.onhgvm' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-29T07:58:33.678 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:58:33 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T07:58:33.678 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:58:33 smithi037 ceph-mon[35162]: from='mgr.? 172.21.15.148:0/3101923077' entity='mgr.smithi148.onhgvm' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi148.onhgvm/key"}]: dispatch 2024-03-29T07:58:33.678 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:58:33 smithi037 ceph-mon[35162]: from='mgr.? 172.21.15.148:0/3101923077' entity='mgr.smithi148.onhgvm' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-29T07:58:33.678 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:58:33 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T07:58:33.678 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:58:33 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T07:58:33.678 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:58:33 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T07:58:33.678 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:58:33 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T07:58:33.725 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:58:33 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "mon metadata", "id": "smithi037"}]: dispatch 2024-03-29T07:58:33.725 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:58:33 smithi148 ceph-mon[44992]: mon.smithi037 calling monitor election 2024-03-29T07:58:33.725 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:58:33 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "mon metadata", "id": "smithi148"}]: dispatch 2024-03-29T07:58:33.725 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:58:33 smithi148 ceph-mon[44992]: from='client.? 172.21.15.148:0/3040965198' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-29T07:58:33.725 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:58:33 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "mon metadata", "id": "smithi148"}]: dispatch 2024-03-29T07:58:33.725 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:58:33 smithi148 ceph-mon[44992]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-29T07:58:33.725 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:58:33 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "mon metadata", "id": "smithi148"}]: dispatch 2024-03-29T07:58:33.725 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:58:33 smithi148 ceph-mon[44992]: mon.smithi148 calling monitor election 2024-03-29T07:58:33.725 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:58:33 smithi148 ceph-mon[44992]: from='mgr.? 172.21.15.148:0/3101923077' entity='mgr.smithi148.onhgvm' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi148.onhgvm/crt"}]: dispatch 2024-03-29T07:58:33.725 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:58:33 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "mon metadata", "id": "smithi148"}]: dispatch 2024-03-29T07:58:33.725 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:58:33 smithi148 ceph-mon[44992]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-29T07:58:33.725 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:58:33 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "mon metadata", "id": "smithi148"}]: dispatch 2024-03-29T07:58:33.725 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:58:33 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "mon metadata", "id": "smithi148"}]: dispatch 2024-03-29T07:58:33.725 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:58:33 smithi148 ceph-mon[44992]: mon.smithi037 is new leader, mons smithi037,smithi148 in quorum (ranks 0,1) 2024-03-29T07:58:33.726 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:58:33 smithi148 ceph-mon[44992]: monmap e2: 2 mons at {smithi037=[v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0],smithi148=[v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-29T07:58:33.726 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:58:33 smithi148 ceph-mon[44992]: fsmap 2024-03-29T07:58:33.726 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:58:33 smithi148 ceph-mon[44992]: osdmap e5: 0 total, 0 up, 0 in 2024-03-29T07:58:33.726 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:58:33 smithi148 ceph-mon[44992]: mgrmap e17: smithi037.wjikii(active, since 37s) 2024-03-29T07:58:33.726 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:58:33 smithi148 ceph-mon[44992]: overall HEALTH_OK 2024-03-29T07:58:33.726 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:58:33 smithi148 ceph-mon[44992]: Standby manager daemon smithi148.onhgvm started 2024-03-29T07:58:33.726 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:58:33 smithi148 ceph-mon[44992]: from='mgr.? 172.21.15.148:0/3101923077' entity='mgr.smithi148.onhgvm' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-29T07:58:33.726 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:58:33 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T07:58:33.726 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:58:33 smithi148 ceph-mon[44992]: from='mgr.? 172.21.15.148:0/3101923077' entity='mgr.smithi148.onhgvm' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi148.onhgvm/key"}]: dispatch 2024-03-29T07:58:33.726 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:58:33 smithi148 ceph-mon[44992]: from='mgr.? 172.21.15.148:0/3101923077' entity='mgr.smithi148.onhgvm' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-29T07:58:33.726 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:58:33 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T07:58:33.726 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:58:33 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T07:58:33.726 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:58:33 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T07:58:33.726 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:58:33 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T07:58:34.067 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-03-29T07:58:34.067 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 99b492c2-eda1-11ee-b647-cb9ed24678a4 -- ceph config generate-minimal-conf 2024-03-29T07:58:34.426 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:58:34 smithi037 ceph-mon[35162]: mgrmap e18: smithi037.wjikii(active, since 37s), standbys: smithi148.onhgvm 2024-03-29T07:58:34.426 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:58:34 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "mgr metadata", "who": "smithi148.onhgvm", "id": "smithi148.onhgvm"}]: dispatch 2024-03-29T07:58:34.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:58:34 smithi037 ceph-mon[35162]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-29T07:58:34.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:58:34 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T07:58:34.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:58:34 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "mon metadata", "id": "smithi148"}]: dispatch 2024-03-29T07:58:34.496 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/mon.smithi037/config 2024-03-29T07:58:34.705 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:58:34 smithi148 ceph-mon[44992]: mgrmap e18: smithi037.wjikii(active, since 37s), standbys: smithi148.onhgvm 2024-03-29T07:58:34.705 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:58:34 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "mgr metadata", "who": "smithi148.onhgvm", "id": "smithi148.onhgvm"}]: dispatch 2024-03-29T07:58:34.705 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:58:34 smithi148 ceph-mon[44992]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-29T07:58:34.705 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:58:34 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T07:58:34.705 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:58:34 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "mon metadata", "id": "smithi148"}]: dispatch 2024-03-29T07:58:36.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:58:36.212+0000 7fc354a00700 1 -- 172.21.15.37:0/2405288668 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc350102ac0 msgr2=0x7fc350102e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T07:58:36.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:58:36.212+0000 7fc354a00700 1 --2- 172.21.15.37:0/2405288668 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc350102ac0 0x7fc350102e90 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fc3400048c0 tx=0x7fc340009e30 comp rx=0 tx=0).stop 2024-03-29T07:58:36.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:58:36.212+0000 7fc354a00700 1 -- 172.21.15.37:0/2405288668 shutdown_connections 2024-03-29T07:58:36.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:58:36.212+0000 7fc354a00700 1 --2- 172.21.15.37:0/2405288668 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc350102ac0 0x7fc350102e90 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T07:58:36.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:58:36.212+0000 7fc354a00700 1 -- 172.21.15.37:0/2405288668 >> 172.21.15.37:0/2405288668 conn(0x7fc3500f85e0 msgr2=0x7fc3500fa9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T07:58:36.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:58:36.212+0000 7fc354a00700 1 -- 172.21.15.37:0/2405288668 shutdown_connections 2024-03-29T07:58:36.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:58:36.212+0000 7fc354a00700 1 -- 172.21.15.37:0/2405288668 wait complete. 2024-03-29T07:58:36.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:58:36.213+0000 7fc354a00700 1 Processor -- start 2024-03-29T07:58:36.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:58:36.213+0000 7fc354a00700 1 -- start start 2024-03-29T07:58:36.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:58:36.213+0000 7fc354a00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc350102ac0 0x7fc350115380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T07:58:36.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:58:36.213+0000 7fc354a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc3501158c0 0x7fc35010f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T07:58:36.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:58:36.213+0000 7fc354a00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc350115ed0 con 0x7fc3501158c0 2024-03-29T07:58:36.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:58:36.213+0000 7fc354a00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc35010f820 con 0x7fc350102ac0 2024-03-29T07:58:36.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:58:36.214+0000 7fc34d400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc3501158c0 0x7fc35010f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T07:58:36.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:58:36.214+0000 7fc34d400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc3501158c0 0x7fc35010f2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49346/0 (socket says 172.21.15.37:49346) 2024-03-29T07:58:36.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:58:36.214+0000 7fc34d400700 1 -- 172.21.15.37:0/1432296545 learned_addr learned my addr 172.21.15.37:0/1432296545 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T07:58:36.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:58:36.214+0000 7fc34de00700 1 --2- 172.21.15.37:0/1432296545 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc350102ac0 0x7fc350115380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T07:58:36.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:58:36.214+0000 7fc34d400700 1 -- 172.21.15.37:0/1432296545 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc350102ac0 msgr2=0x7fc350115380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T07:58:36.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:58:36.214+0000 7fc34d400700 1 --2- 172.21.15.37:0/1432296545 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc350102ac0 0x7fc350115380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T07:58:36.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:58:36.214+0000 7fc34d400700 1 -- 172.21.15.37:0/1432296545 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc340005040 con 0x7fc3501158c0 2024-03-29T07:58:36.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:58:36.215+0000 7fc34d400700 1 --2- 172.21.15.37:0/1432296545 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc3501158c0 0x7fc35010f2e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fc33c00b790 tx=0x7fc33c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T07:58:36.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:58:36.215+0000 7fc346000700 1 -- 172.21.15.37:0/1432296545 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc33c010870 con 0x7fc3501158c0 2024-03-29T07:58:36.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:58:36.215+0000 7fc346000700 1 -- 172.21.15.37:0/1432296545 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fc33c010e80 con 0x7fc3501158c0 2024-03-29T07:58:36.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:58:36.215+0000 7fc354a00700 1 -- 172.21.15.37:0/1432296545 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc35010fb00 con 0x7fc3501158c0 2024-03-29T07:58:36.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:58:36.215+0000 7fc346000700 1 -- 172.21.15.37:0/1432296545 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc33c00d3b0 con 0x7fc3501158c0 2024-03-29T07:58:36.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:58:36.215+0000 7fc354a00700 1 -- 172.21.15.37:0/1432296545 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3501100c0 con 0x7fc3501158c0 2024-03-29T07:58:36.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:58:36.216+0000 7fc354a00700 1 -- 172.21.15.37:0/1432296545 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3500ffd20 con 0x7fc3501158c0 2024-03-29T07:58:36.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:58:36.218+0000 7fc346000700 1 -- 172.21.15.37:0/1432296545 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 18) v1 ==== 86565+0+0 (secure 0 0 0) 0x7fc33c0109d0 con 0x7fc3501158c0 2024-03-29T07:58:36.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:58:36.219+0000 7fc346000700 1 --2- 172.21.15.37:0/1432296545 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7fc3340693b0 0x7fc33406b860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T07:58:36.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:58:36.219+0000 7fc346000700 1 -- 172.21.15.37:0/1432296545 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7fc33c086780 con 0x7fc3501158c0 2024-03-29T07:58:36.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:58:36.219+0000 7fc34de00700 1 --2- 172.21.15.37:0/1432296545 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7fc3340693b0 0x7fc33406b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T07:58:36.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:58:36.220+0000 7fc34de00700 1 --2- 172.21.15.37:0/1432296545 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7fc3340693b0 0x7fc33406b860 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fc34000b3b0 tx=0x7fc340009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T07:58:36.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:58:36.224+0000 7fc346000700 1 -- 172.21.15.37:0/1432296545 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fc33c057500 con 0x7fc3501158c0 2024-03-29T07:58:36.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:58:36.434+0000 7fc354a00700 1 -- 172.21.15.37:0/1432296545 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7fc3500fe1c0 con 0x7fc3501158c0 2024-03-29T07:58:36.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:58:36.434+0000 7fc346000700 1 -- 172.21.15.37:0/1432296545 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v10) v1 ==== 76+0+225 (secure 0 0 0) 0x7fc33c014100 con 0x7fc3501158c0 2024-03-29T07:58:36.434 INFO:teuthology.orchestra.run.smithi037.stdout:# minimal ceph.conf for 99b492c2-eda1-11ee-b647-cb9ed24678a4 2024-03-29T07:58:36.435 INFO:teuthology.orchestra.run.smithi037.stdout:[global] 2024-03-29T07:58:36.435 INFO:teuthology.orchestra.run.smithi037.stdout: fsid = 99b492c2-eda1-11ee-b647-cb9ed24678a4 2024-03-29T07:58:36.435 INFO:teuthology.orchestra.run.smithi037.stdout: mon_host = [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] 2024-03-29T07:58:36.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:58:36.437+0000 7fc354a00700 1 -- 172.21.15.37:0/1432296545 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7fc3340693b0 msgr2=0x7fc33406b860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T07:58:36.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:58:36.437+0000 7fc354a00700 1 --2- 172.21.15.37:0/1432296545 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7fc3340693b0 0x7fc33406b860 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fc34000b3b0 tx=0x7fc340009e30 comp rx=0 tx=0).stop 2024-03-29T07:58:36.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:58:36.437+0000 7fc354a00700 1 -- 172.21.15.37:0/1432296545 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc3501158c0 msgr2=0x7fc35010f2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T07:58:36.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:58:36.437+0000 7fc354a00700 1 --2- 172.21.15.37:0/1432296545 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc3501158c0 0x7fc35010f2e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fc33c00b790 tx=0x7fc33c009250 comp rx=0 tx=0).stop 2024-03-29T07:58:36.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:58:36.437+0000 7fc354a00700 1 -- 172.21.15.37:0/1432296545 shutdown_connections 2024-03-29T07:58:36.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:58:36.437+0000 7fc354a00700 1 --2- 172.21.15.37:0/1432296545 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7fc3340693b0 0x7fc33406b860 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T07:58:36.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:58:36.437+0000 7fc354a00700 1 --2- 172.21.15.37:0/1432296545 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc350102ac0 0x7fc350115380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T07:58:36.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:58:36.437+0000 7fc354a00700 1 --2- 172.21.15.37:0/1432296545 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc3501158c0 0x7fc35010f2e0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T07:58:36.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:58:36.437+0000 7fc354a00700 1 -- 172.21.15.37:0/1432296545 >> 172.21.15.37:0/1432296545 conn(0x7fc3500f85e0 msgr2=0x7fc3500715f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T07:58:36.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:58:36.437+0000 7fc354a00700 1 -- 172.21.15.37:0/1432296545 shutdown_connections 2024-03-29T07:58:36.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:58:36.437+0000 7fc354a00700 1 -- 172.21.15.37:0/1432296545 wait complete. 2024-03-29T07:58:36.805 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:58:36 smithi148 ceph-mon[44992]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-29T07:58:36.805 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:58:36 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T07:58:36.805 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:58:36 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T07:58:36.805 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:58:36 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T07:58:36.805 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:58:36 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T07:58:36.805 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:58:36 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T07:58:36.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:58:36 smithi148 ceph-mon[44992]: from='client.? 172.21.15.37:0/1432296545' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T07:58:36.845 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:58:36 smithi037 ceph-mon[35162]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-29T07:58:36.845 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:58:36 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T07:58:36.845 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:58:36 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T07:58:36.845 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:58:36 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T07:58:36.845 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:58:36 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T07:58:36.845 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:58:36 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T07:58:36.845 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:58:36 smithi037 ceph-mon[35162]: from='client.? 172.21.15.37:0/1432296545' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T07:58:37.234 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-03-29T07:58:37.234 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-03-29T07:58:37.234 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd of=/etc/ceph/ceph.conf 2024-03-29T07:58:37.307 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-03-29T07:58:37.307 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-29T07:58:37.415 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-03-29T07:58:37.415 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd of=/etc/ceph/ceph.conf 2024-03-29T07:58:37.484 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-03-29T07:58:37.484 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-29T07:58:37.591 INFO:tasks.cephadm:Deploying OSDs... 2024-03-29T07:58:37.591 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-03-29T07:58:37.591 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/scratch_devs of=/dev/stdout 2024-03-29T07:58:37.635 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-29T07:58:37.635 DEBUG:teuthology.orchestra.run.smithi037:> stat /dev/vg_nvme/lv_1 2024-03-29T07:58:37.720 INFO:teuthology.orchestra.run.smithi037.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-29T07:58:37.721 INFO:teuthology.orchestra.run.smithi037.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-29T07:58:37.721 INFO:teuthology.orchestra.run.smithi037.stdout:Device: 5h/5d Inode: 447 Links: 1 2024-03-29T07:58:37.721 INFO:teuthology.orchestra.run.smithi037.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-29T07:58:37.721 INFO:teuthology.orchestra.run.smithi037.stdout:Context: system_u:object_r:device_t:s0 2024-03-29T07:58:37.721 INFO:teuthology.orchestra.run.smithi037.stdout:Access: 2024-03-29 07:57:44.836105838 +0000 2024-03-29T07:58:37.721 INFO:teuthology.orchestra.run.smithi037.stdout:Modify: 2024-03-29 07:50:08.716870411 +0000 2024-03-29T07:58:37.721 INFO:teuthology.orchestra.run.smithi037.stdout:Change: 2024-03-29 07:50:08.716870411 +0000 2024-03-29T07:58:37.721 INFO:teuthology.orchestra.run.smithi037.stdout: Birth: 2024-03-29 07:50:08.691870736 +0000 2024-03-29T07:58:37.721 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-29T07:58:37.805 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:58:37 smithi148 ceph-mon[44992]: Updating smithi037:/etc/ceph/ceph.conf 2024-03-29T07:58:37.805 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:58:37 smithi148 ceph-mon[44992]: Updating smithi148:/etc/ceph/ceph.conf 2024-03-29T07:58:37.805 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:58:37 smithi148 ceph-mon[44992]: Updating smithi148:/var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/config/ceph.conf 2024-03-29T07:58:37.805 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:58:37 smithi148 ceph-mon[44992]: Updating smithi037:/var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/config/ceph.conf 2024-03-29T07:58:37.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:58:37 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T07:58:37.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:58:37 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T07:58:37.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:58:37 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T07:58:37.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:58:37 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T07:58:37.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:58:37 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T07:58:37.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:58:37 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-29T07:58:37.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:58:37 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-29T07:58:37.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:58:37 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T07:58:37.821 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:58:37 smithi037 ceph-mon[35162]: Updating smithi037:/etc/ceph/ceph.conf 2024-03-29T07:58:37.821 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:58:37 smithi037 ceph-mon[35162]: Updating smithi148:/etc/ceph/ceph.conf 2024-03-29T07:58:37.821 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:58:37 smithi037 ceph-mon[35162]: Updating smithi148:/var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/config/ceph.conf 2024-03-29T07:58:37.821 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:58:37 smithi037 ceph-mon[35162]: Updating smithi037:/var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/config/ceph.conf 2024-03-29T07:58:37.822 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:58:37 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T07:58:37.822 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:58:37 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T07:58:37.822 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:58:37 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T07:58:37.822 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:58:37 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T07:58:37.822 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:58:37 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T07:58:37.822 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:58:37 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-29T07:58:37.822 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:58:37 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-29T07:58:37.822 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:58:37 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T07:58:37.828 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-03-29T07:58:37.828 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-03-29T07:58:37.828 INFO:teuthology.orchestra.run.smithi037.stderr:512 bytes copied, 0.000318714 s, 1.6 MB/s 2024-03-29T07:58:37.831 DEBUG:teuthology.orchestra.run.smithi037:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-29T07:58:37.880 DEBUG:teuthology.orchestra.run.smithi037:> stat /dev/vg_nvme/lv_2 2024-03-29T07:58:37.926 INFO:teuthology.orchestra.run.smithi037.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-29T07:58:37.926 INFO:teuthology.orchestra.run.smithi037.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-29T07:58:37.926 INFO:teuthology.orchestra.run.smithi037.stdout:Device: 5h/5d Inode: 453 Links: 1 2024-03-29T07:58:37.926 INFO:teuthology.orchestra.run.smithi037.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-29T07:58:37.926 INFO:teuthology.orchestra.run.smithi037.stdout:Context: system_u:object_r:device_t:s0 2024-03-29T07:58:37.926 INFO:teuthology.orchestra.run.smithi037.stdout:Access: 2024-03-29 07:57:44.836105838 +0000 2024-03-29T07:58:37.926 INFO:teuthology.orchestra.run.smithi037.stdout:Modify: 2024-03-29 07:50:09.196864194 +0000 2024-03-29T07:58:37.927 INFO:teuthology.orchestra.run.smithi037.stdout:Change: 2024-03-29 07:50:09.196864194 +0000 2024-03-29T07:58:37.927 INFO:teuthology.orchestra.run.smithi037.stdout: Birth: 2024-03-29 07:50:09.173864492 +0000 2024-03-29T07:58:37.927 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-29T07:58:37.993 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-03-29T07:58:37.993 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-03-29T07:58:37.993 INFO:teuthology.orchestra.run.smithi037.stderr:512 bytes copied, 0.000279942 s, 1.8 MB/s 2024-03-29T07:58:37.995 DEBUG:teuthology.orchestra.run.smithi037:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-29T07:58:38.044 DEBUG:teuthology.orchestra.run.smithi037:> stat /dev/vg_nvme/lv_3 2024-03-29T07:58:38.088 INFO:teuthology.orchestra.run.smithi037.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-29T07:58:38.088 INFO:teuthology.orchestra.run.smithi037.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-29T07:58:38.088 INFO:teuthology.orchestra.run.smithi037.stdout:Device: 5h/5d Inode: 459 Links: 1 2024-03-29T07:58:38.089 INFO:teuthology.orchestra.run.smithi037.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-29T07:58:38.089 INFO:teuthology.orchestra.run.smithi037.stdout:Context: system_u:object_r:device_t:s0 2024-03-29T07:58:38.089 INFO:teuthology.orchestra.run.smithi037.stdout:Access: 2024-03-29 07:57:44.836105838 +0000 2024-03-29T07:58:38.089 INFO:teuthology.orchestra.run.smithi037.stdout:Modify: 2024-03-29 07:50:09.671858042 +0000 2024-03-29T07:58:38.089 INFO:teuthology.orchestra.run.smithi037.stdout:Change: 2024-03-29 07:50:09.671858042 +0000 2024-03-29T07:58:38.089 INFO:teuthology.orchestra.run.smithi037.stdout: Birth: 2024-03-29 07:50:09.647858353 +0000 2024-03-29T07:58:38.089 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-29T07:58:38.200 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-03-29T07:58:38.200 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-03-29T07:58:38.200 INFO:teuthology.orchestra.run.smithi037.stderr:512 bytes copied, 0.000285646 s, 1.8 MB/s 2024-03-29T07:58:38.204 DEBUG:teuthology.orchestra.run.smithi037:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-29T07:58:38.252 DEBUG:teuthology.orchestra.run.smithi037:> stat /dev/vg_nvme/lv_4 2024-03-29T07:58:38.296 INFO:teuthology.orchestra.run.smithi037.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-29T07:58:38.297 INFO:teuthology.orchestra.run.smithi037.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-29T07:58:38.297 INFO:teuthology.orchestra.run.smithi037.stdout:Device: 5h/5d Inode: 465 Links: 1 2024-03-29T07:58:38.297 INFO:teuthology.orchestra.run.smithi037.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-29T07:58:38.297 INFO:teuthology.orchestra.run.smithi037.stdout:Context: system_u:object_r:device_t:s0 2024-03-29T07:58:38.297 INFO:teuthology.orchestra.run.smithi037.stdout:Access: 2024-03-29 07:57:44.836105838 +0000 2024-03-29T07:58:38.297 INFO:teuthology.orchestra.run.smithi037.stdout:Modify: 2024-03-29 07:50:10.151851825 +0000 2024-03-29T07:58:38.297 INFO:teuthology.orchestra.run.smithi037.stdout:Change: 2024-03-29 07:50:10.151851825 +0000 2024-03-29T07:58:38.297 INFO:teuthology.orchestra.run.smithi037.stdout: Birth: 2024-03-29 07:50:10.127852136 +0000 2024-03-29T07:58:38.297 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-29T07:58:38.404 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-03-29T07:58:38.404 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-03-29T07:58:38.404 INFO:teuthology.orchestra.run.smithi037.stderr:512 bytes copied, 0.000302399 s, 1.7 MB/s 2024-03-29T07:58:38.407 DEBUG:teuthology.orchestra.run.smithi037:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-29T07:58:38.455 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-03-29T07:58:38.455 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/scratch_devs of=/dev/stdout 2024-03-29T07:58:38.505 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-29T07:58:38.505 DEBUG:teuthology.orchestra.run.smithi148:> stat /dev/vg_nvme/lv_1 2024-03-29T07:58:38.552 INFO:teuthology.orchestra.run.smithi148.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-29T07:58:38.553 INFO:teuthology.orchestra.run.smithi148.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-29T07:58:38.553 INFO:teuthology.orchestra.run.smithi148.stdout:Device: 5h/5d Inode: 447 Links: 1 2024-03-29T07:58:38.553 INFO:teuthology.orchestra.run.smithi148.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-29T07:58:38.553 INFO:teuthology.orchestra.run.smithi148.stdout:Context: system_u:object_r:device_t:s0 2024-03-29T07:58:38.553 INFO:teuthology.orchestra.run.smithi148.stdout:Access: 2024-03-29 07:58:26.773599819 +0000 2024-03-29T07:58:38.553 INFO:teuthology.orchestra.run.smithi148.stdout:Modify: 2024-03-29 07:50:02.101156564 +0000 2024-03-29T07:58:38.553 INFO:teuthology.orchestra.run.smithi148.stdout:Change: 2024-03-29 07:50:02.101156564 +0000 2024-03-29T07:58:38.553 INFO:teuthology.orchestra.run.smithi148.stdout: Birth: 2024-03-29 07:50:02.073157214 +0000 2024-03-29T07:58:38.553 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-29T07:58:38.617 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-03-29T07:58:38.618 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-03-29T07:58:38.618 INFO:teuthology.orchestra.run.smithi148.stderr:512 bytes copied, 0.000298292 s, 1.7 MB/s 2024-03-29T07:58:38.620 DEBUG:teuthology.orchestra.run.smithi148:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-29T07:58:38.666 DEBUG:teuthology.orchestra.run.smithi148:> stat /dev/vg_nvme/lv_2 2024-03-29T07:58:38.676 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:58:38 smithi037 ceph-mon[35162]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-29T07:58:38.676 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:58:38 smithi037 ceph-mon[35162]: Reconfiguring mon.smithi037 (unknown last config time)... 2024-03-29T07:58:38.676 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:58:38 smithi037 ceph-mon[35162]: Reconfiguring daemon mon.smithi037 on smithi037 2024-03-29T07:58:38.751 INFO:teuthology.orchestra.run.smithi148.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-29T07:58:38.752 INFO:teuthology.orchestra.run.smithi148.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-29T07:58:38.752 INFO:teuthology.orchestra.run.smithi148.stdout:Device: 5h/5d Inode: 453 Links: 1 2024-03-29T07:58:38.752 INFO:teuthology.orchestra.run.smithi148.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-29T07:58:38.752 INFO:teuthology.orchestra.run.smithi148.stdout:Context: system_u:object_r:device_t:s0 2024-03-29T07:58:38.752 INFO:teuthology.orchestra.run.smithi148.stdout:Access: 2024-03-29 07:58:26.773599819 +0000 2024-03-29T07:58:38.752 INFO:teuthology.orchestra.run.smithi148.stdout:Modify: 2024-03-29 07:50:02.605144852 +0000 2024-03-29T07:58:38.752 INFO:teuthology.orchestra.run.smithi148.stdout:Change: 2024-03-29 07:50:02.605144852 +0000 2024-03-29T07:58:38.752 INFO:teuthology.orchestra.run.smithi148.stdout: Birth: 2024-03-29 07:50:02.581145410 +0000 2024-03-29T07:58:38.753 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-29T07:58:38.805 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:58:38 smithi148 ceph-mon[44992]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-29T07:58:38.805 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:58:38 smithi148 ceph-mon[44992]: Reconfiguring mon.smithi037 (unknown last config time)... 2024-03-29T07:58:38.805 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:58:38 smithi148 ceph-mon[44992]: Reconfiguring daemon mon.smithi037 on smithi037 2024-03-29T07:58:38.855 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-03-29T07:58:38.856 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-03-29T07:58:38.856 INFO:teuthology.orchestra.run.smithi148.stderr:512 bytes copied, 0.000293812 s, 1.7 MB/s 2024-03-29T07:58:38.858 DEBUG:teuthology.orchestra.run.smithi148:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-29T07:58:38.945 DEBUG:teuthology.orchestra.run.smithi148:> stat /dev/vg_nvme/lv_3 2024-03-29T07:58:38.990 INFO:teuthology.orchestra.run.smithi148.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-29T07:58:38.990 INFO:teuthology.orchestra.run.smithi148.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-29T07:58:38.990 INFO:teuthology.orchestra.run.smithi148.stdout:Device: 5h/5d Inode: 459 Links: 1 2024-03-29T07:58:38.990 INFO:teuthology.orchestra.run.smithi148.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-29T07:58:38.990 INFO:teuthology.orchestra.run.smithi148.stdout:Context: system_u:object_r:device_t:s0 2024-03-29T07:58:38.990 INFO:teuthology.orchestra.run.smithi148.stdout:Access: 2024-03-29 07:58:26.774599796 +0000 2024-03-29T07:58:38.990 INFO:teuthology.orchestra.run.smithi148.stdout:Modify: 2024-03-29 07:50:03.086133675 +0000 2024-03-29T07:58:38.990 INFO:teuthology.orchestra.run.smithi148.stdout:Change: 2024-03-29 07:50:03.086133675 +0000 2024-03-29T07:58:38.991 INFO:teuthology.orchestra.run.smithi148.stdout: Birth: 2024-03-29 07:50:03.062134233 +0000 2024-03-29T07:58:38.991 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-29T07:58:39.094 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-03-29T07:58:39.094 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-03-29T07:58:39.094 INFO:teuthology.orchestra.run.smithi148.stderr:512 bytes copied, 0.000290425 s, 1.8 MB/s 2024-03-29T07:58:39.097 DEBUG:teuthology.orchestra.run.smithi148:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-29T07:58:39.143 DEBUG:teuthology.orchestra.run.smithi148:> stat /dev/vg_nvme/lv_4 2024-03-29T07:58:39.228 INFO:teuthology.orchestra.run.smithi148.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-29T07:58:39.228 INFO:teuthology.orchestra.run.smithi148.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-29T07:58:39.228 INFO:teuthology.orchestra.run.smithi148.stdout:Device: 5h/5d Inode: 465 Links: 1 2024-03-29T07:58:39.228 INFO:teuthology.orchestra.run.smithi148.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-29T07:58:39.228 INFO:teuthology.orchestra.run.smithi148.stdout:Context: system_u:object_r:device_t:s0 2024-03-29T07:58:39.228 INFO:teuthology.orchestra.run.smithi148.stdout:Access: 2024-03-29 07:58:26.774599796 +0000 2024-03-29T07:58:39.228 INFO:teuthology.orchestra.run.smithi148.stdout:Modify: 2024-03-29 07:50:03.571122405 +0000 2024-03-29T07:58:39.229 INFO:teuthology.orchestra.run.smithi148.stdout:Change: 2024-03-29 07:50:03.571122405 +0000 2024-03-29T07:58:39.229 INFO:teuthology.orchestra.run.smithi148.stdout: Birth: 2024-03-29 07:50:03.545123009 +0000 2024-03-29T07:58:39.229 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-29T07:58:39.332 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-03-29T07:58:39.332 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-03-29T07:58:39.332 INFO:teuthology.orchestra.run.smithi148.stderr:512 bytes copied, 0.000458837 s, 1.1 MB/s 2024-03-29T07:58:39.334 DEBUG:teuthology.orchestra.run.smithi148:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-29T07:58:39.380 INFO:tasks.cephadm:Deploying osd.0 on smithi037 with /dev/vg_nvme/lv_4... 2024-03-29T07:58:39.380 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 99b492c2-eda1-11ee-b647-cb9ed24678a4 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-03-29T07:58:40.426 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:58:40 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T07:58:40.426 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:58:40 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T07:58:40.426 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:58:40 smithi037 ceph-mon[35162]: Reconfiguring mgr.smithi037.wjikii (unknown last config time)... 2024-03-29T07:58:40.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:58:40 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi037.wjikii", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-29T07:58:40.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:58:40 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-29T07:58:40.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:58:40 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T07:58:40.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:58:40 smithi037 ceph-mon[35162]: Reconfiguring daemon mgr.smithi037.wjikii on smithi037 2024-03-29T07:58:40.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:58:40 smithi037 ceph-mon[35162]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-29T07:58:40.555 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:58:40 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T07:58:40.555 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:58:40 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T07:58:40.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:58:40 smithi148 ceph-mon[44992]: Reconfiguring mgr.smithi037.wjikii (unknown last config time)... 2024-03-29T07:58:40.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:58:40 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi037.wjikii", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-29T07:58:40.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:58:40 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-29T07:58:40.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:58:40 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T07:58:40.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:58:40 smithi148 ceph-mon[44992]: Reconfiguring daemon mgr.smithi037.wjikii on smithi037 2024-03-29T07:58:40.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:58:40 smithi148 ceph-mon[44992]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-29T07:58:40.796 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/mon.smithi037/config 2024-03-29T07:58:41.426 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:58:41 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T07:58:41.426 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:58:41 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T07:58:41.426 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:58:41 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T07:58:41.426 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:58:41 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi037", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-29T07:58:41.426 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:58:41 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T07:58:41.558 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:58:41 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T07:58:41.559 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:58:41 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T07:58:41.559 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:58:41 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T07:58:41.559 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:58:41 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi037", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-29T07:58:41.559 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:58:41 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T07:58:42.426 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:58:42 smithi037 ceph-mon[35162]: Reconfiguring crash.smithi037 (monmap changed)... 2024-03-29T07:58:42.426 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:58:42 smithi037 ceph-mon[35162]: Reconfiguring daemon crash.smithi037 on smithi037 2024-03-29T07:58:42.426 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:58:42 smithi037 ceph-mon[35162]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-29T07:58:42.555 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:58:42 smithi148 ceph-mon[44992]: Reconfiguring crash.smithi037 (monmap changed)... 2024-03-29T07:58:42.555 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:58:42 smithi148 ceph-mon[44992]: Reconfiguring daemon crash.smithi037 on smithi037 2024-03-29T07:58:42.555 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:58:42 smithi148 ceph-mon[44992]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-29T07:58:43.872 INFO:teuthology.orchestra.run.smithi037.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-03-29T07:58:43.872 INFO:teuthology.orchestra.run.smithi037.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-03-29T07:58:43.872 INFO:teuthology.orchestra.run.smithi037.stderr: stderr: 10+0 records in 2024-03-29T07:58:43.872 INFO:teuthology.orchestra.run.smithi037.stderr:10+0 records out 2024-03-29T07:58:43.872 INFO:teuthology.orchestra.run.smithi037.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0422606 s, 248 MB/s 2024-03-29T07:58:43.872 INFO:teuthology.orchestra.run.smithi037.stderr:--> Zapping successful for: 2024-03-29T07:58:44.603 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:58:44 smithi037 ceph-mon[35162]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-29T07:58:44.603 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:58:44 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T07:58:44.603 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:58:44 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T07:58:44.603 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:58:44 smithi037 ceph-mon[35162]: Reconfiguring alertmanager.smithi037 (dependencies changed)... 2024-03-29T07:58:44.604 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:58:44 smithi037 ceph-mon[35162]: Reconfiguring daemon alertmanager.smithi037 on smithi037 2024-03-29T07:58:44.627 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 99b492c2-eda1-11ee-b647-cb9ed24678a4 -- ceph orch daemon add osd smithi037:vg_nvme/lv_4 2024-03-29T07:58:44.805 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:58:44 smithi148 ceph-mon[44992]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-29T07:58:44.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:58:44 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T07:58:44.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:58:44 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T07:58:44.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:58:44 smithi148 ceph-mon[44992]: Reconfiguring alertmanager.smithi037 (dependencies changed)... 2024-03-29T07:58:44.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:58:44 smithi148 ceph-mon[44992]: Reconfiguring daemon alertmanager.smithi037 on smithi037 2024-03-29T07:58:45.974 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/mon.smithi037/config 2024-03-29T07:58:46.631 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:58:46 smithi037 ceph-mon[35162]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-29T07:58:46.805 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:58:46 smithi148 ceph-mon[44992]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-29T07:58:48.676 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:58:48 smithi037 ceph-mon[35162]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-29T07:58:48.805 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:58:48 smithi148 ceph-mon[44992]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-29T07:58:49.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:58:49.843+0000 7f3913a00700 1 -- 172.21.15.37:0/2505168307 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f390c071f50 msgr2=0x7f390c070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T07:58:49.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:58:49.843+0000 7f3913a00700 1 --2- 172.21.15.37:0/2505168307 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f390c071f50 0x7f390c070460 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f3900004940 tx=0x7f3900009e30 comp rx=0 tx=0).stop 2024-03-29T07:58:49.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:58:49.844+0000 7f3913a00700 1 -- 172.21.15.37:0/2505168307 shutdown_connections 2024-03-29T07:58:49.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:58:49.844+0000 7f3913a00700 1 --2- 172.21.15.37:0/2505168307 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f390c0709a0 0x7f390c070e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T07:58:49.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:58:49.844+0000 7f3913a00700 1 --2- 172.21.15.37:0/2505168307 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f390c071f50 0x7f390c070460 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T07:58:49.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:58:49.844+0000 7f3913a00700 1 -- 172.21.15.37:0/2505168307 >> 172.21.15.37:0/2505168307 conn(0x7f390c0f8560 msgr2=0x7f390c0fa970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T07:58:49.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:58:49.844+0000 7f3913a00700 1 -- 172.21.15.37:0/2505168307 shutdown_connections 2024-03-29T07:58:49.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:58:49.844+0000 7f3913a00700 1 -- 172.21.15.37:0/2505168307 wait complete. 2024-03-29T07:58:49.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:58:49.845+0000 7f3913a00700 1 Processor -- start 2024-03-29T07:58:49.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:58:49.845+0000 7f3913a00700 1 -- start start 2024-03-29T07:58:49.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:58:49.845+0000 7f3913a00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f390c0709a0 0x7f390c10fad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T07:58:49.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:58:49.845+0000 7f3913a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f390c071f50 0x7f390c110010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T07:58:49.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:58:49.845+0000 7f3913a00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f390c110610 con 0x7f390c071f50 2024-03-29T07:58:49.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:58:49.845+0000 7f3913a00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f390c110e60 con 0x7f390c0709a0 2024-03-29T07:58:49.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:58:49.845+0000 7f390be00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f390c071f50 0x7f390c110010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T07:58:49.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:58:49.845+0000 7f390be00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f390c071f50 0x7f390c110010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43394/0 (socket says 172.21.15.37:43394) 2024-03-29T07:58:49.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:58:49.845+0000 7f390be00700 1 -- 172.21.15.37:0/1066665764 learned_addr learned my addr 172.21.15.37:0/1066665764 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T07:58:49.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:58:49.846+0000 7f3911000700 1 --2- 172.21.15.37:0/1066665764 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f390c0709a0 0x7f390c10fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T07:58:49.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:58:49.846+0000 7f390be00700 1 -- 172.21.15.37:0/1066665764 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f390c0709a0 msgr2=0x7f390c10fad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T07:58:49.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:58:49.846+0000 7f390be00700 1 --2- 172.21.15.37:0/1066665764 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f390c0709a0 0x7f390c10fad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T07:58:49.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:58:49.846+0000 7f390be00700 1 -- 172.21.15.37:0/1066665764 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3900005040 con 0x7f390c071f50 2024-03-29T07:58:49.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:58:49.846+0000 7f390be00700 1 --2- 172.21.15.37:0/1066665764 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f390c071f50 0x7f390c110010 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f38f800eb40 tx=0x7f38f800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T07:58:49.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:58:49.846+0000 7f3908c00700 1 -- 172.21.15.37:0/1066665764 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f38f8007cc0 con 0x7f390c071f50 2024-03-29T07:58:49.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:58:49.846+0000 7f3908c00700 1 -- 172.21.15.37:0/1066665764 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f38f800f040 con 0x7f390c071f50 2024-03-29T07:58:49.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:58:49.846+0000 7f3913a00700 1 -- 172.21.15.37:0/1066665764 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f390c1110e0 con 0x7f390c071f50 2024-03-29T07:58:49.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:58:49.846+0000 7f3908c00700 1 -- 172.21.15.37:0/1066665764 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f38f800c990 con 0x7f390c071f50 2024-03-29T07:58:49.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:58:49.847+0000 7f3913a00700 1 -- 172.21.15.37:0/1066665764 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f390c111620 con 0x7f390c071f50 2024-03-29T07:58:49.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:58:49.848+0000 7f3913a00700 1 -- 172.21.15.37:0/1066665764 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f390c05f5f0 con 0x7f390c071f50 2024-03-29T07:58:49.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:58:49.848+0000 7f3908c00700 1 -- 172.21.15.37:0/1066665764 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 18) v1 ==== 86565+0+0 (secure 0 0 0) 0x7f38f800cc00 con 0x7f390c071f50 2024-03-29T07:58:49.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:58:49.850+0000 7f3908c00700 1 --2- 172.21.15.37:0/1066665764 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f38f4069210 0x7f38f406b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T07:58:49.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:58:49.850+0000 7f3908c00700 1 -- 172.21.15.37:0/1066665764 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7f38f8085ce0 con 0x7f390c071f50 2024-03-29T07:58:49.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:58:49.850+0000 7f3911000700 1 --2- 172.21.15.37:0/1066665764 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f38f4069210 0x7f38f406b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T07:58:49.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:58:49.850+0000 7f3911000700 1 --2- 172.21.15.37:0/1066665764 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f38f4069210 0x7f38f406b6c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f390000b0d0 tx=0x7f3900009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T07:58:49.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:58:49.854+0000 7f3908c00700 1 -- 172.21.15.37:0/1066665764 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f38f804e190 con 0x7f390c071f50 2024-03-29T07:58:50.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:58:50.077+0000 7f3913a00700 1 -- 172.21.15.37:0/1066665764 --> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi037:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f390c111950 con 0x7f38f4069210 2024-03-29T07:58:50.725 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:58:50 smithi037 ceph-mon[35162]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-29T07:58:50.725 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:58:50 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-29T07:58:50.725 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:58:50 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-29T07:58:50.725 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:58:50 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T07:58:50.805 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:58:50 smithi148 ceph-mon[44992]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-29T07:58:50.805 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:58:50 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-29T07:58:50.805 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:58:50 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-29T07:58:50.805 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:58:50 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T07:58:52.055 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:58:51 smithi148 ceph-mon[44992]: from='client.14242 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi037:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T07:58:52.055 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:58:51 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T07:58:52.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:58:51 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T07:58:52.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:58:51 smithi148 ceph-mon[44992]: Reconfiguring grafana.smithi037 (dependencies changed)... 2024-03-29T07:58:52.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:58:51 smithi148 ceph-mon[44992]: Reconfiguring daemon grafana.smithi037 on smithi037 2024-03-29T07:58:52.069 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:58:51 smithi037 ceph-mon[35162]: from='client.14242 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi037:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T07:58:52.069 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:58:51 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T07:58:52.069 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:58:51 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T07:58:52.069 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:58:51 smithi037 ceph-mon[35162]: Reconfiguring grafana.smithi037 (dependencies changed)... 2024-03-29T07:58:52.069 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:58:51 smithi037 ceph-mon[35162]: Reconfiguring daemon grafana.smithi037 on smithi037 2024-03-29T07:58:52.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:58:52 smithi037 ceph-mon[35162]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-29T07:58:53.055 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:58:52 smithi148 ceph-mon[44992]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-29T07:58:54.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:58:54 smithi037 ceph-mon[35162]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-29T07:58:55.055 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:58:54 smithi148 ceph-mon[44992]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-29T07:58:55.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:58:55 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T07:58:56.055 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:58:55 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T07:58:57.037 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:58:56 smithi037 ceph-mon[35162]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-29T07:58:57.037 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:58:56 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T07:58:57.037 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:58:56 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T07:58:57.055 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:58:56 smithi148 ceph-mon[44992]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-29T07:58:57.055 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:58:56 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T07:58:57.055 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:58:56 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T07:58:57.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:58:57 smithi037 ceph-mon[35162]: Reconfiguring prometheus.smithi037 (dependencies changed)... 2024-03-29T07:58:57.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:58:57 smithi037 ceph-mon[35162]: Reconfiguring daemon prometheus.smithi037 on smithi037 2024-03-29T07:58:58.055 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:58:57 smithi148 ceph-mon[44992]: Reconfiguring prometheus.smithi037 (dependencies changed)... 2024-03-29T07:58:58.055 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:58:57 smithi148 ceph-mon[44992]: Reconfiguring daemon prometheus.smithi037 on smithi037 2024-03-29T07:58:59.055 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:58:58 smithi148 ceph-mon[44992]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-29T07:58:59.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:58:58 smithi148 ceph-mon[44992]: from='client.? 172.21.15.37:0/1267395123' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dba18fb6-b27d-4258-b769-dc89f4fef44d"}]: dispatch 2024-03-29T07:58:59.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:58:58 smithi148 ceph-mon[44992]: from='client.? 172.21.15.37:0/1267395123' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "dba18fb6-b27d-4258-b769-dc89f4fef44d"}]': finished 2024-03-29T07:58:59.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:58:58 smithi148 ceph-mon[44992]: osdmap e6: 1 total, 0 up, 1 in 2024-03-29T07:58:59.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:58:58 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-29T07:58:59.176 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:58:58 smithi037 ceph-mon[35162]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-29T07:58:59.176 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:58:58 smithi037 ceph-mon[35162]: from='client.? 172.21.15.37:0/1267395123' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dba18fb6-b27d-4258-b769-dc89f4fef44d"}]: dispatch 2024-03-29T07:58:59.176 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:58:58 smithi037 ceph-mon[35162]: from='client.? 172.21.15.37:0/1267395123' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "dba18fb6-b27d-4258-b769-dc89f4fef44d"}]': finished 2024-03-29T07:58:59.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:58:58 smithi037 ceph-mon[35162]: osdmap e6: 1 total, 0 up, 1 in 2024-03-29T07:58:59.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:58:58 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-29T07:59:00.055 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:58:59 smithi148 ceph-mon[44992]: from='client.? 172.21.15.37:0/4109226831' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-29T07:59:00.178 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:58:59 smithi037 ceph-mon[35162]: from='client.? 172.21.15.37:0/4109226831' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-29T07:59:00.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:59:00 smithi037 ceph-mon[35162]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-29T07:59:01.055 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:59:00 smithi148 ceph-mon[44992]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-29T07:59:02.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:59:02 smithi037 ceph-mon[35162]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-29T07:59:03.055 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:59:02 smithi148 ceph-mon[44992]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-29T07:59:04.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:59:04 smithi037 ceph-mon[35162]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-29T07:59:05.055 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:59:04 smithi148 ceph-mon[44992]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-29T07:59:06.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:59:06 smithi037 ceph-mon[35162]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-29T07:59:07.305 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:59:06 smithi148 ceph-mon[44992]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-29T07:59:09.176 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:59:08 smithi037 ceph-mon[35162]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-29T07:59:09.305 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:59:08 smithi148 ceph-mon[44992]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-29T07:59:10.055 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:59:10 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T07:59:10.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:59:10 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T07:59:10.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:59:10 smithi148 ceph-mon[44992]: Reconfiguring crash.smithi148 (monmap changed)... 2024-03-29T07:59:10.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:59:10 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi148", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-29T07:59:10.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:59:10 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T07:59:10.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:59:10 smithi148 ceph-mon[44992]: Reconfiguring daemon crash.smithi148 on smithi148 2024-03-29T07:59:10.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:59:10 smithi148 ceph-mon[44992]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-29T07:59:10.266 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:59:10 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T07:59:10.267 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:59:10 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T07:59:10.267 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:59:10 smithi037 ceph-mon[35162]: Reconfiguring crash.smithi148 (monmap changed)... 2024-03-29T07:59:10.267 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:59:10 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi148", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-29T07:59:10.267 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:59:10 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T07:59:10.267 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:59:10 smithi037 ceph-mon[35162]: Reconfiguring daemon crash.smithi148 on smithi148 2024-03-29T07:59:10.267 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:59:10 smithi037 ceph-mon[35162]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-29T07:59:11.305 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:59:11 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T07:59:11.305 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:59:11 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-29T07:59:11.305 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:59:11 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T07:59:11.305 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:59:11 smithi148 ceph-mon[44992]: Deploying daemon osd.0 on smithi037 2024-03-29T07:59:11.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:59:11 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T07:59:11.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:59:11 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T07:59:11.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:59:11 smithi148 ceph-mon[44992]: Reconfiguring mgr.smithi148.onhgvm (monmap changed)... 2024-03-29T07:59:11.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:59:11 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi148.onhgvm", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-29T07:59:11.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:59:11 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-29T07:59:11.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:59:11 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T07:59:11.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:59:11 smithi148 ceph-mon[44992]: Reconfiguring daemon mgr.smithi148.onhgvm on smithi148 2024-03-29T07:59:11.396 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:59:11 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T07:59:11.397 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:59:11 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-29T07:59:11.397 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:59:11 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T07:59:11.397 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:59:11 smithi037 ceph-mon[35162]: Deploying daemon osd.0 on smithi037 2024-03-29T07:59:11.397 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:59:11 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T07:59:11.397 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:59:11 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T07:59:11.397 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:59:11 smithi037 ceph-mon[35162]: Reconfiguring mgr.smithi148.onhgvm (monmap changed)... 2024-03-29T07:59:11.397 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:59:11 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi148.onhgvm", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-29T07:59:11.397 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:59:11 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-29T07:59:11.397 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:59:11 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T07:59:11.397 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:59:11 smithi037 ceph-mon[35162]: Reconfiguring daemon mgr.smithi148.onhgvm on smithi148 2024-03-29T07:59:12.348 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:59:12 smithi148 ceph-mon[44992]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-29T07:59:12.426 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:59:12 smithi037 ceph-mon[35162]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-29T07:59:13.676 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:59:13 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T07:59:13.676 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:59:13 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T07:59:13.676 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:59:13 smithi037 ceph-mon[35162]: Reconfiguring mon.smithi148 (monmap changed)... 2024-03-29T07:59:13.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:59:13 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-29T07:59:13.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:59:13 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-29T07:59:13.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:59:13 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T07:59:13.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:59:13 smithi037 ceph-mon[35162]: Reconfiguring daemon mon.smithi148 on smithi148 2024-03-29T07:59:13.798 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:59:13 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T07:59:13.798 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:59:13 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T07:59:13.798 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:59:13 smithi148 ceph-mon[44992]: Reconfiguring mon.smithi148 (monmap changed)... 2024-03-29T07:59:13.798 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:59:13 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-29T07:59:13.798 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:59:13 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-29T07:59:13.798 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:59:13 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T07:59:13.798 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:59:13 smithi148 ceph-mon[44992]: Reconfiguring daemon mon.smithi148 on smithi148 2024-03-29T07:59:14.500 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:59:14 smithi037 ceph-mon[35162]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-29T07:59:14.501 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:59:14 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T07:59:14.501 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:59:14 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T07:59:14.501 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:59:14 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-29T07:59:14.501 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:59:14 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi037.front.sepia.ceph.com:9093"}]: dispatch 2024-03-29T07:59:14.501 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:59:14 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T07:59:14.501 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:59:14 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-29T07:59:14.501 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:59:14 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi037.front.sepia.ceph.com:3000"}]: dispatch 2024-03-29T07:59:14.501 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:59:14 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T07:59:14.501 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:59:14 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-29T07:59:14.501 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:59:14 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi037.front.sepia.ceph.com:9095"}]: dispatch 2024-03-29T07:59:14.501 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:59:14 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T07:59:14.501 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:59:14 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T07:59:14.805 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:59:14 smithi148 ceph-mon[44992]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-29T07:59:14.805 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:59:14 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T07:59:14.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:59:14 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T07:59:14.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:59:14 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-29T07:59:14.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:59:14 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi037.front.sepia.ceph.com:9093"}]: dispatch 2024-03-29T07:59:14.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:59:14 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T07:59:14.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:59:14 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-29T07:59:14.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:59:14 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi037.front.sepia.ceph.com:3000"}]: dispatch 2024-03-29T07:59:14.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:59:14 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T07:59:14.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:59:14 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-29T07:59:14.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:59:14 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi037.front.sepia.ceph.com:9095"}]: dispatch 2024-03-29T07:59:14.807 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:59:14 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T07:59:14.807 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:59:14 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T07:59:15.805 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:59:15 smithi148 ceph-mon[44992]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-29T07:59:15.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:59:15 smithi148 ceph-mon[44992]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi037.front.sepia.ceph.com:9093"}]: dispatch 2024-03-29T07:59:15.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:59:15 smithi148 ceph-mon[44992]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-29T07:59:15.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:59:15 smithi148 ceph-mon[44992]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi037.front.sepia.ceph.com:3000"}]: dispatch 2024-03-29T07:59:15.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:59:15 smithi148 ceph-mon[44992]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-29T07:59:15.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:59:15 smithi148 ceph-mon[44992]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi037.front.sepia.ceph.com:9095"}]: dispatch 2024-03-29T07:59:15.816 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:59:15 smithi037 ceph-mon[35162]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-29T07:59:15.816 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:59:15 smithi037 ceph-mon[35162]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi037.front.sepia.ceph.com:9093"}]: dispatch 2024-03-29T07:59:15.816 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:59:15 smithi037 ceph-mon[35162]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-29T07:59:15.816 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:59:15 smithi037 ceph-mon[35162]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi037.front.sepia.ceph.com:3000"}]: dispatch 2024-03-29T07:59:15.817 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:59:15 smithi037 ceph-mon[35162]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-29T07:59:15.817 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:59:15 smithi037 ceph-mon[35162]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi037.front.sepia.ceph.com:9095"}]: dispatch 2024-03-29T07:59:16.805 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:59:16 smithi148 ceph-mon[44992]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-29T07:59:16.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:59:16 smithi037 ceph-mon[35162]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-29T07:59:17.998 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:59:17 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T07:59:17.998 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:59:17 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T07:59:18.028 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:59:17 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T07:59:18.029 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:59:17 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T07:59:19.055 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:59:18 smithi148 ceph-mon[44992]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-29T07:59:19.055 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:59:18 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T07:59:19.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:59:18 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T07:59:19.176 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:59:18 smithi037 ceph-mon[35162]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-29T07:59:19.176 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:59:18 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T07:59:19.176 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:59:18 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T07:59:21.055 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:59:20 smithi148 ceph-mon[44992]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-29T07:59:21.169 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:59:20 smithi037 ceph-mon[35162]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-29T07:59:22.827 INFO:teuthology.orchestra.run.smithi037.stdout:Created osd(s) 0 on host 'smithi037' 2024-03-29T07:59:22.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:59:22.826+0000 7f3908c00700 1 -- 172.21.15.37:0/1066665764 <== mgr.14208 v2:172.21.15.37:6800/1802836043 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f390c111950 con 0x7f38f4069210 2024-03-29T07:59:22.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:59:22.829+0000 7f3913a00700 1 -- 172.21.15.37:0/1066665764 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f38f4069210 msgr2=0x7f38f406b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T07:59:22.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:59:22.829+0000 7f3913a00700 1 --2- 172.21.15.37:0/1066665764 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f38f4069210 0x7f38f406b6c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f390000b0d0 tx=0x7f3900009e30 comp rx=0 tx=0).stop 2024-03-29T07:59:22.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:59:22.829+0000 7f3913a00700 1 -- 172.21.15.37:0/1066665764 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f390c071f50 msgr2=0x7f390c110010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T07:59:22.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:59:22.829+0000 7f3913a00700 1 --2- 172.21.15.37:0/1066665764 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f390c071f50 0x7f390c110010 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f38f800eb40 tx=0x7f38f800b040 comp rx=0 tx=0).stop 2024-03-29T07:59:22.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:59:22.829+0000 7f3913a00700 1 -- 172.21.15.37:0/1066665764 shutdown_connections 2024-03-29T07:59:22.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:59:22.829+0000 7f3913a00700 1 --2- 172.21.15.37:0/1066665764 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f38f4069210 0x7f38f406b6c0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T07:59:22.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:59:22.829+0000 7f3913a00700 1 --2- 172.21.15.37:0/1066665764 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f390c0709a0 0x7f390c10fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T07:59:22.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:59:22.829+0000 7f3913a00700 1 --2- 172.21.15.37:0/1066665764 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f390c071f50 0x7f390c110010 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T07:59:22.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:59:22.829+0000 7f3913a00700 1 -- 172.21.15.37:0/1066665764 >> 172.21.15.37:0/1066665764 conn(0x7f390c0f8560 msgr2=0x7f390c0f9cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T07:59:22.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:59:22.829+0000 7f3913a00700 1 -- 172.21.15.37:0/1066665764 shutdown_connections 2024-03-29T07:59:22.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:59:22.829+0000 7f3913a00700 1 -- 172.21.15.37:0/1066665764 wait complete. 2024-03-29T07:59:23.055 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:59:22 smithi148 ceph-mon[44992]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-29T07:59:23.055 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:59:22 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T07:59:23.055 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:59:22 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T07:59:23.055 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:59:22 smithi148 ceph-mon[44992]: from='osd.0 [v2:172.21.15.37:6802/3851364162,v1:172.21.15.37:6803/3851364162]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-29T07:59:23.176 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:59:22 smithi037 ceph-mon[35162]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-29T07:59:23.176 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:59:22 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T07:59:23.176 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:59:22 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T07:59:23.176 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:59:22 smithi037 ceph-mon[35162]: from='osd.0 [v2:172.21.15.37:6802/3851364162,v1:172.21.15.37:6803/3851364162]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-29T07:59:23.491 DEBUG:teuthology.orchestra.run.smithi037:osd.0> sudo journalctl -f -n 0 -u ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4@osd.0.service 2024-03-29T07:59:23.497 INFO:tasks.cephadm:Deploying osd.1 on smithi037 with /dev/vg_nvme/lv_3... 2024-03-29T07:59:23.497 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 99b492c2-eda1-11ee-b647-cb9ed24678a4 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-03-29T07:59:23.575 INFO:journalctl@ceph.osd.0.smithi037.stdout:-- Logs begin at Fri 2024-03-29 07:43:01 UTC. -- 2024-03-29T07:59:24.031 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:59:23 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T07:59:24.031 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:59:23 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T07:59:24.031 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:59:23 smithi037 ceph-mon[35162]: from='osd.0 [v2:172.21.15.37:6802/3851364162,v1:172.21.15.37:6803/3851364162]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-29T07:59:24.031 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:59:23 smithi037 ceph-mon[35162]: osdmap e7: 1 total, 0 up, 1 in 2024-03-29T07:59:24.031 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:59:23 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-29T07:59:24.031 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:59:23 smithi037 ceph-mon[35162]: from='osd.0 [v2:172.21.15.37:6802/3851364162,v1:172.21.15.37:6803/3851364162]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]: dispatch 2024-03-29T07:59:24.032 INFO:journalctl@ceph.osd.0.smithi037.stdout:Mar 29 07:59:24 smithi037 ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4-osd-0[58581]: 2024-03-29T07:59:24.000+0000 7fcc6a800700 -1 osd.0 0 waiting for initial osdmap 2024-03-29T07:59:24.032 INFO:journalctl@ceph.osd.0.smithi037.stdout:Mar 29 07:59:24 smithi037 ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4-osd-0[58581]: 2024-03-29T07:59:24.004+0000 7fcc64e00700 -1 osd.0 8 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-29T07:59:24.055 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:59:23 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T07:59:24.055 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:59:23 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T07:59:24.055 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:59:23 smithi148 ceph-mon[44992]: from='osd.0 [v2:172.21.15.37:6802/3851364162,v1:172.21.15.37:6803/3851364162]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-29T07:59:24.055 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:59:23 smithi148 ceph-mon[44992]: osdmap e7: 1 total, 0 up, 1 in 2024-03-29T07:59:24.055 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:59:23 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-29T07:59:24.055 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:59:23 smithi148 ceph-mon[44992]: from='osd.0 [v2:172.21.15.37:6802/3851364162,v1:172.21.15.37:6803/3851364162]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]: dispatch 2024-03-29T07:59:24.856 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/mon.smithi037/config 2024-03-29T07:59:25.055 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:59:24 smithi148 ceph-mon[44992]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-29T07:59:25.055 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:59:24 smithi148 ceph-mon[44992]: from='osd.0 [v2:172.21.15.37:6802/3851364162,v1:172.21.15.37:6803/3851364162]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]': finished 2024-03-29T07:59:25.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:59:24 smithi148 ceph-mon[44992]: osdmap e8: 1 total, 0 up, 1 in 2024-03-29T07:59:25.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:59:24 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-29T07:59:25.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:59:24 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-29T07:59:25.176 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:59:24 smithi037 ceph-mon[35162]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-29T07:59:25.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:59:24 smithi037 ceph-mon[35162]: from='osd.0 [v2:172.21.15.37:6802/3851364162,v1:172.21.15.37:6803/3851364162]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]': finished 2024-03-29T07:59:25.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:59:24 smithi037 ceph-mon[35162]: osdmap e8: 1 total, 0 up, 1 in 2024-03-29T07:59:25.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:59:24 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-29T07:59:25.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:59:24 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-29T07:59:26.305 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:59:26 smithi148 ceph-mon[44992]: purged_snaps scrub starts 2024-03-29T07:59:26.305 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:59:26 smithi148 ceph-mon[44992]: purged_snaps scrub ok 2024-03-29T07:59:26.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:59:26 smithi148 ceph-mon[44992]: osd.0 [v2:172.21.15.37:6802/3851364162,v1:172.21.15.37:6803/3851364162] boot 2024-03-29T07:59:26.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:59:26 smithi148 ceph-mon[44992]: osdmap e9: 1 total, 1 up, 1 in 2024-03-29T07:59:26.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:59:26 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-29T07:59:26.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:59:26 smithi148 ceph-mon[44992]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-29T07:59:26.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:59:26 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T07:59:26.405 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:59:26 smithi037 ceph-mon[35162]: purged_snaps scrub starts 2024-03-29T07:59:26.405 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:59:26 smithi037 ceph-mon[35162]: purged_snaps scrub ok 2024-03-29T07:59:26.405 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:59:26 smithi037 ceph-mon[35162]: osd.0 [v2:172.21.15.37:6802/3851364162,v1:172.21.15.37:6803/3851364162] boot 2024-03-29T07:59:26.405 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:59:26 smithi037 ceph-mon[35162]: osdmap e9: 1 total, 1 up, 1 in 2024-03-29T07:59:26.405 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:59:26 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-29T07:59:26.406 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:59:26 smithi037 ceph-mon[35162]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-29T07:59:26.406 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:59:26 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T07:59:27.176 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:59:27 smithi037 ceph-mon[35162]: osdmap e10: 1 total, 1 up, 1 in 2024-03-29T07:59:27.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:59:27 smithi148 ceph-mon[44992]: osdmap e10: 1 total, 1 up, 1 in 2024-03-29T07:59:27.747 INFO:teuthology.orchestra.run.smithi037.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-03-29T07:59:27.747 INFO:teuthology.orchestra.run.smithi037.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-03-29T07:59:27.747 INFO:teuthology.orchestra.run.smithi037.stderr: stderr: 10+0 records in 2024-03-29T07:59:27.747 INFO:teuthology.orchestra.run.smithi037.stderr:10+0 records out 2024-03-29T07:59:27.747 INFO:teuthology.orchestra.run.smithi037.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0347949 s, 301 MB/s 2024-03-29T07:59:27.747 INFO:teuthology.orchestra.run.smithi037.stderr:--> Zapping successful for: 2024-03-29T07:59:28.176 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:59:28 smithi037 ceph-mon[35162]: pgmap v44: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-29T07:59:28.305 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:59:28 smithi148 ceph-mon[44992]: pgmap v44: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-29T07:59:28.736 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 99b492c2-eda1-11ee-b647-cb9ed24678a4 -- ceph orch daemon add osd smithi037:vg_nvme/lv_3 2024-03-29T07:59:29.235 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/mon.smithi037/config 2024-03-29T07:59:29.592 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:59:29 smithi037 ceph-mon[35162]: Detected new or changed devices on smithi037 2024-03-29T07:59:29.592 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:59:29 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T07:59:29.592 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:59:29 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T07:59:29.592 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:59:29 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-29T07:59:29.592 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:59:29 smithi037 ceph-mon[35162]: Adjusting osd_memory_target on smithi037 to 13050M 2024-03-29T07:59:29.592 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:59:29 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T07:59:29.592 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:59:29 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T07:59:29.593 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:59:29 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T07:59:29.593 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:59:29 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T07:59:29.593 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:59:29 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T07:59:29.663 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:59:29 smithi148 ceph-mon[44992]: Detected new or changed devices on smithi037 2024-03-29T07:59:29.663 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:59:29 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T07:59:29.663 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:59:29 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T07:59:29.663 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:59:29 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-29T07:59:29.663 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:59:29 smithi148 ceph-mon[44992]: Adjusting osd_memory_target on smithi037 to 13050M 2024-03-29T07:59:29.663 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:59:29 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T07:59:29.663 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:59:29 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T07:59:29.663 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:59:29 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T07:59:29.663 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:59:29 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T07:59:29.663 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:59:29 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T07:59:30.583 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:59:30 smithi037 ceph-mon[35162]: pgmap v45: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-29T07:59:30.805 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:59:30 smithi148 ceph-mon[44992]: pgmap v45: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-29T07:59:31.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:59:31.569+0000 7fe623800700 1 -- 172.21.15.37:0/1398146333 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe61c071f50 msgr2=0x7fe61c070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T07:59:31.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:59:31.569+0000 7fe623800700 1 --2- 172.21.15.37:0/1398146333 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe61c071f50 0x7fe61c070460 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fe610004940 tx=0x7fe610009e30 comp rx=0 tx=0).stop 2024-03-29T07:59:31.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:59:31.570+0000 7fe623800700 1 -- 172.21.15.37:0/1398146333 shutdown_connections 2024-03-29T07:59:31.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:59:31.570+0000 7fe623800700 1 --2- 172.21.15.37:0/1398146333 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe61c0709a0 0x7fe61c070e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T07:59:31.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:59:31.570+0000 7fe623800700 1 --2- 172.21.15.37:0/1398146333 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe61c071f50 0x7fe61c070460 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T07:59:31.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:59:31.570+0000 7fe623800700 1 -- 172.21.15.37:0/1398146333 >> 172.21.15.37:0/1398146333 conn(0x7fe61c0f8560 msgr2=0x7fe61c0fa970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T07:59:31.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:59:31.570+0000 7fe623800700 1 -- 172.21.15.37:0/1398146333 shutdown_connections 2024-03-29T07:59:31.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:59:31.570+0000 7fe623800700 1 -- 172.21.15.37:0/1398146333 wait complete. 2024-03-29T07:59:31.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:59:31.571+0000 7fe623800700 1 Processor -- start 2024-03-29T07:59:31.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:59:31.571+0000 7fe623800700 1 -- start start 2024-03-29T07:59:31.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:59:31.571+0000 7fe623800700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe61c0709a0 0x7fe61c113f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T07:59:31.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:59:31.571+0000 7fe623800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe61c071f50 0x7fe61c1144a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T07:59:31.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:59:31.571+0000 7fe623800700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe61c114b30 con 0x7fe61c071f50 2024-03-29T07:59:31.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:59:31.571+0000 7fe623800700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe61c115320 con 0x7fe61c0709a0 2024-03-29T07:59:31.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:59:31.572+0000 7fe61be00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe61c071f50 0x7fe61c1144a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T07:59:31.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:59:31.572+0000 7fe61be00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe61c071f50 0x7fe61c1144a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49662/0 (socket says 172.21.15.37:49662) 2024-03-29T07:59:31.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:59:31.572+0000 7fe61be00700 1 -- 172.21.15.37:0/3530984112 learned_addr learned my addr 172.21.15.37:0/3530984112 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T07:59:31.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:59:31.572+0000 7fe620e00700 1 --2- 172.21.15.37:0/3530984112 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe61c0709a0 0x7fe61c113f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T07:59:31.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:59:31.572+0000 7fe61be00700 1 -- 172.21.15.37:0/3530984112 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe61c0709a0 msgr2=0x7fe61c113f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T07:59:31.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:59:31.572+0000 7fe61be00700 1 --2- 172.21.15.37:0/3530984112 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe61c0709a0 0x7fe61c113f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T07:59:31.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:59:31.572+0000 7fe61be00700 1 -- 172.21.15.37:0/3530984112 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe610005040 con 0x7fe61c071f50 2024-03-29T07:59:31.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:59:31.572+0000 7fe61be00700 1 --2- 172.21.15.37:0/3530984112 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe61c071f50 0x7fe61c1144a0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fe60800b790 tx=0x7fe608009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T07:59:31.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:59:31.573+0000 7fe618c00700 1 -- 172.21.15.37:0/3530984112 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe60800f770 con 0x7fe61c071f50 2024-03-29T07:59:31.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:59:31.573+0000 7fe623800700 1 -- 172.21.15.37:0/3530984112 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe61c115600 con 0x7fe61c071f50 2024-03-29T07:59:31.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:59:31.573+0000 7fe618c00700 1 -- 172.21.15.37:0/3530984112 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fe60800fd80 con 0x7fe61c071f50 2024-03-29T07:59:31.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:59:31.573+0000 7fe618c00700 1 -- 172.21.15.37:0/3530984112 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe60800d310 con 0x7fe61c071f50 2024-03-29T07:59:31.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:59:31.573+0000 7fe623800700 1 -- 172.21.15.37:0/3530984112 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe61c115bc0 con 0x7fe61c071f50 2024-03-29T07:59:31.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:59:31.574+0000 7fe623800700 1 -- 172.21.15.37:0/3530984112 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe61c05f5f0 con 0x7fe61c071f50 2024-03-29T07:59:31.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:59:31.575+0000 7fe618c00700 1 -- 172.21.15.37:0/3530984112 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 18) v1 ==== 86565+0+0 (secure 0 0 0) 0x7fe608010420 con 0x7fe61c071f50 2024-03-29T07:59:31.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:59:31.577+0000 7fe618c00700 1 --2- 172.21.15.37:0/3530984112 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7fe604069210 0x7fe60406b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T07:59:31.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:59:31.577+0000 7fe618c00700 1 -- 172.21.15.37:0/3530984112 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1978+0+0 (secure 0 0 0) 0x7fe608086240 con 0x7fe61c071f50 2024-03-29T07:59:31.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:59:31.577+0000 7fe620e00700 1 --2- 172.21.15.37:0/3530984112 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7fe604069210 0x7fe60406b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T07:59:31.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:59:31.577+0000 7fe620e00700 1 --2- 172.21.15.37:0/3530984112 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7fe604069210 0x7fe60406b6c0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fe610009fd0 tx=0x7fe610009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T07:59:31.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:59:31.581+0000 7fe618c00700 1 -- 172.21.15.37:0/3530984112 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fe608056e60 con 0x7fe61c071f50 2024-03-29T07:59:31.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:59:31.806+0000 7fe623800700 1 -- 172.21.15.37:0/3530984112 --> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi037:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7fe61c115f00 con 0x7fe604069210 2024-03-29T07:59:32.426 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:59:32 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T07:59:32.426 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:59:32 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T07:59:32.426 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:59:32 smithi037 ceph-mon[35162]: pgmap v46: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-29T07:59:32.426 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:59:32 smithi037 ceph-mon[35162]: from='client.14260 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi037:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T07:59:32.426 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:59:32 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-29T07:59:32.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:59:32 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-29T07:59:32.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:59:32 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T07:59:32.555 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:59:32 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T07:59:32.555 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:59:32 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T07:59:32.555 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:59:32 smithi148 ceph-mon[44992]: pgmap v46: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-29T07:59:32.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:59:32 smithi148 ceph-mon[44992]: from='client.14260 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi037:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T07:59:32.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:59:32 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-29T07:59:32.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:59:32 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-29T07:59:32.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:59:32 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T07:59:34.805 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:59:34 smithi148 ceph-mon[44992]: pgmap v47: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-29T07:59:34.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:59:34 smithi037 ceph-mon[35162]: pgmap v47: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-29T07:59:36.805 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:59:36 smithi148 ceph-mon[44992]: pgmap v48: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-29T07:59:36.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:59:36 smithi148 ceph-mon[44992]: from='client.? 172.21.15.37:0/3846486455' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2876dd3f-aefb-4298-8fe4-562245e1103e"}]: dispatch 2024-03-29T07:59:36.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:59:36 smithi148 ceph-mon[44992]: from='client.? 172.21.15.37:0/3846486455' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2876dd3f-aefb-4298-8fe4-562245e1103e"}]': finished 2024-03-29T07:59:36.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:59:36 smithi148 ceph-mon[44992]: osdmap e11: 2 total, 1 up, 2 in 2024-03-29T07:59:36.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:59:36 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-29T07:59:36.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:59:36 smithi037 ceph-mon[35162]: pgmap v48: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-29T07:59:36.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:59:36 smithi037 ceph-mon[35162]: from='client.? 172.21.15.37:0/3846486455' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2876dd3f-aefb-4298-8fe4-562245e1103e"}]: dispatch 2024-03-29T07:59:36.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:59:36 smithi037 ceph-mon[35162]: from='client.? 172.21.15.37:0/3846486455' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2876dd3f-aefb-4298-8fe4-562245e1103e"}]': finished 2024-03-29T07:59:36.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:59:36 smithi037 ceph-mon[35162]: osdmap e11: 2 total, 1 up, 2 in 2024-03-29T07:59:36.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:59:36 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-29T07:59:37.805 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:59:37 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T07:59:37.805 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:59:37 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T07:59:37.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:59:37 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T07:59:37.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:59:37 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T07:59:37.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:59:37 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T07:59:37.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:59:37 smithi148 ceph-mon[44992]: from='client.? 172.21.15.37:0/1069448876' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-29T07:59:37.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:59:37 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T07:59:37.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:59:37 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T07:59:37.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:59:37 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T07:59:37.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:59:37 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T07:59:37.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:59:37 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T07:59:37.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:59:37 smithi037 ceph-mon[35162]: from='client.? 172.21.15.37:0/1069448876' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-29T07:59:38.805 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:59:38 smithi148 ceph-mon[44992]: pgmap v50: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-29T07:59:38.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:59:38 smithi037 ceph-mon[35162]: pgmap v50: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-29T07:59:40.605 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:59:40 smithi037 ceph-mon[35162]: pgmap v51: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-29T07:59:40.606 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:59:40 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T07:59:40.805 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:59:40 smithi148 ceph-mon[44992]: pgmap v51: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-29T07:59:40.805 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:59:40 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T07:59:42.671 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:59:42 smithi037 ceph-mon[35162]: pgmap v52: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-29T07:59:42.805 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:59:42 smithi148 ceph-mon[44992]: pgmap v52: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-29T07:59:44.805 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:59:44 smithi148 ceph-mon[44992]: pgmap v53: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-29T07:59:44.805 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:59:44 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-29T07:59:44.805 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:59:44 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T07:59:44.888 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:59:44 smithi037 ceph-mon[35162]: pgmap v53: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-29T07:59:44.888 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:59:44 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-29T07:59:44.888 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:59:44 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T07:59:45.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:59:45 smithi037 ceph-mon[35162]: Deploying daemon osd.1 on smithi037 2024-03-29T07:59:46.055 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:59:45 smithi148 ceph-mon[44992]: Deploying daemon osd.1 on smithi037 2024-03-29T07:59:46.894 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:59:46 smithi037 ceph-mon[35162]: pgmap v54: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-29T07:59:47.055 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:59:46 smithi148 ceph-mon[44992]: pgmap v54: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-29T07:59:48.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:59:48 smithi037 ceph-mon[35162]: pgmap v55: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-29T07:59:49.058 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:59:48 smithi148 ceph-mon[44992]: pgmap v55: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-29T07:59:50.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:59:50 smithi037 ceph-mon[35162]: pgmap v56: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-29T07:59:51.055 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:59:50 smithi148 ceph-mon[44992]: pgmap v56: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-29T07:59:51.904 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:59:51 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T07:59:51.905 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:59:51 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T07:59:51.905 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:59:51 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T07:59:52.055 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:59:51 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T07:59:52.055 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:59:51 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T07:59:52.055 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:59:51 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T07:59:52.832 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:59:52 smithi037 ceph-mon[35162]: pgmap v57: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-29T07:59:53.055 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:59:52 smithi148 ceph-mon[44992]: pgmap v57: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-29T07:59:54.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:59:54 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T07:59:54.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:59:54 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T07:59:54.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:59:54 smithi037 ceph-mon[35162]: pgmap v58: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-29T07:59:54.555 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:59:54 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T07:59:54.555 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:59:54 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T07:59:54.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:59:54 smithi148 ceph-mon[44992]: pgmap v58: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-29T07:59:55.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:59:55.128+0000 7fe618c00700 1 -- 172.21.15.37:0/3530984112 <== mgr.14208 v2:172.21.15.37:6800/1802836043 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fe61c115f00 con 0x7fe604069210 2024-03-29T07:59:55.128 INFO:teuthology.orchestra.run.smithi037.stdout:Created osd(s) 1 on host 'smithi037' 2024-03-29T07:59:55.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:59:55.130+0000 7fe623800700 1 -- 172.21.15.37:0/3530984112 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7fe604069210 msgr2=0x7fe60406b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T07:59:55.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:59:55.130+0000 7fe623800700 1 --2- 172.21.15.37:0/3530984112 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7fe604069210 0x7fe60406b6c0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fe610009fd0 tx=0x7fe610009e30 comp rx=0 tx=0).stop 2024-03-29T07:59:55.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:59:55.130+0000 7fe623800700 1 -- 172.21.15.37:0/3530984112 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe61c071f50 msgr2=0x7fe61c1144a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T07:59:55.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:59:55.130+0000 7fe623800700 1 --2- 172.21.15.37:0/3530984112 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe61c071f50 0x7fe61c1144a0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fe60800b790 tx=0x7fe608009250 comp rx=0 tx=0).stop 2024-03-29T07:59:55.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:59:55.130+0000 7fe623800700 1 -- 172.21.15.37:0/3530984112 shutdown_connections 2024-03-29T07:59:55.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:59:55.130+0000 7fe623800700 1 --2- 172.21.15.37:0/3530984112 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7fe604069210 0x7fe60406b6c0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T07:59:55.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:59:55.130+0000 7fe623800700 1 --2- 172.21.15.37:0/3530984112 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe61c0709a0 0x7fe61c113f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T07:59:55.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:59:55.130+0000 7fe623800700 1 --2- 172.21.15.37:0/3530984112 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe61c071f50 0x7fe61c1144a0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T07:59:55.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:59:55.130+0000 7fe623800700 1 -- 172.21.15.37:0/3530984112 >> 172.21.15.37:0/3530984112 conn(0x7fe61c0f8560 msgr2=0x7fe61c0f9ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T07:59:55.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:59:55.131+0000 7fe623800700 1 -- 172.21.15.37:0/3530984112 shutdown_connections 2024-03-29T07:59:55.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T07:59:55.131+0000 7fe623800700 1 -- 172.21.15.37:0/3530984112 wait complete. 2024-03-29T07:59:55.426 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:59:55 smithi037 ceph-mon[35162]: from='osd.1 [v2:172.21.15.37:6810/3949957441,v1:172.21.15.37:6811/3949957441]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-29T07:59:55.426 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:59:55 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T07:59:55.426 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:59:55 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T07:59:55.555 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:59:55 smithi148 ceph-mon[44992]: from='osd.1 [v2:172.21.15.37:6810/3949957441,v1:172.21.15.37:6811/3949957441]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-29T07:59:55.555 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:59:55 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T07:59:55.555 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:59:55 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T07:59:55.962 DEBUG:teuthology.orchestra.run.smithi037:osd.1> sudo journalctl -f -n 0 -u ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4@osd.1.service 2024-03-29T07:59:55.967 INFO:tasks.cephadm:Deploying osd.2 on smithi037 with /dev/vg_nvme/lv_2... 2024-03-29T07:59:55.967 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 99b492c2-eda1-11ee-b647-cb9ed24678a4 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-03-29T07:59:56.040 INFO:journalctl@ceph.osd.1.smithi037.stdout:-- Logs begin at Fri 2024-03-29 07:43:01 UTC. -- 2024-03-29T07:59:56.555 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:59:56 smithi148 ceph-mon[44992]: from='osd.1 [v2:172.21.15.37:6810/3949957441,v1:172.21.15.37:6811/3949957441]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-29T07:59:56.555 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:59:56 smithi148 ceph-mon[44992]: osdmap e12: 2 total, 1 up, 2 in 2024-03-29T07:59:56.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:59:56 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-29T07:59:56.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:59:56 smithi148 ceph-mon[44992]: from='osd.1 [v2:172.21.15.37:6810/3949957441,v1:172.21.15.37:6811/3949957441]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]: dispatch 2024-03-29T07:59:56.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:59:56 smithi148 ceph-mon[44992]: pgmap v60: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-29T07:59:56.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:59:56 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T07:59:56.627 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/mon.smithi037/config 2024-03-29T07:59:56.676 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:59:56 smithi037 ceph-mon[35162]: from='osd.1 [v2:172.21.15.37:6810/3949957441,v1:172.21.15.37:6811/3949957441]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-29T07:59:56.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:59:56 smithi037 ceph-mon[35162]: osdmap e12: 2 total, 1 up, 2 in 2024-03-29T07:59:56.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:59:56 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-29T07:59:56.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:59:56 smithi037 ceph-mon[35162]: from='osd.1 [v2:172.21.15.37:6810/3949957441,v1:172.21.15.37:6811/3949957441]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]: dispatch 2024-03-29T07:59:56.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:59:56 smithi037 ceph-mon[35162]: pgmap v60: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-29T07:59:56.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:59:56 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T07:59:56.677 INFO:journalctl@ceph.osd.1.smithi037.stdout:Mar 29 07:59:56 smithi037 ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4-osd-1[67815]: 2024-03-29T07:59:56.268+0000 7fca3a400700 -1 osd.1 0 waiting for initial osdmap 2024-03-29T07:59:56.677 INFO:journalctl@ceph.osd.1.smithi037.stdout:Mar 29 07:59:56 smithi037 ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4-osd-1[67815]: 2024-03-29T07:59:56.273+0000 7fca32c00700 -1 osd.1 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-29T07:59:57.426 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:59:57 smithi037 ceph-mon[35162]: from='osd.1 [v2:172.21.15.37:6810/3949957441,v1:172.21.15.37:6811/3949957441]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]': finished 2024-03-29T07:59:57.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:59:57 smithi037 ceph-mon[35162]: osdmap e13: 2 total, 1 up, 2 in 2024-03-29T07:59:57.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:59:57 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-29T07:59:57.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:59:57 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-29T07:59:57.555 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:59:57 smithi148 ceph-mon[44992]: from='osd.1 [v2:172.21.15.37:6810/3949957441,v1:172.21.15.37:6811/3949957441]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]': finished 2024-03-29T07:59:57.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:59:57 smithi148 ceph-mon[44992]: osdmap e13: 2 total, 1 up, 2 in 2024-03-29T07:59:57.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:59:57 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-29T07:59:57.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:59:57 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-29T07:59:58.555 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:59:58 smithi148 ceph-mon[44992]: purged_snaps scrub starts 2024-03-29T07:59:58.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:59:58 smithi148 ceph-mon[44992]: purged_snaps scrub ok 2024-03-29T07:59:58.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:59:58 smithi148 ceph-mon[44992]: osd.1 [v2:172.21.15.37:6810/3949957441,v1:172.21.15.37:6811/3949957441] boot 2024-03-29T07:59:58.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:59:58 smithi148 ceph-mon[44992]: osdmap e14: 2 total, 2 up, 2 in 2024-03-29T07:59:58.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:59:58 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-29T07:59:58.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:59:58 smithi148 ceph-mon[44992]: pgmap v63: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-29T07:59:58.623 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:59:58 smithi037 ceph-mon[35162]: purged_snaps scrub starts 2024-03-29T07:59:58.623 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:59:58 smithi037 ceph-mon[35162]: purged_snaps scrub ok 2024-03-29T07:59:58.623 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:59:58 smithi037 ceph-mon[35162]: osd.1 [v2:172.21.15.37:6810/3949957441,v1:172.21.15.37:6811/3949957441] boot 2024-03-29T07:59:58.623 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:59:58 smithi037 ceph-mon[35162]: osdmap e14: 2 total, 2 up, 2 in 2024-03-29T07:59:58.623 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:59:58 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-29T07:59:58.623 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:59:58 smithi037 ceph-mon[35162]: pgmap v63: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-29T07:59:59.555 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 07:59:59 smithi148 ceph-mon[44992]: osdmap e15: 2 total, 2 up, 2 in 2024-03-29T07:59:59.676 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 07:59:59 smithi037 ceph-mon[35162]: osdmap e15: 2 total, 2 up, 2 in 2024-03-29T07:59:59.722 INFO:teuthology.orchestra.run.smithi037.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-03-29T07:59:59.722 INFO:teuthology.orchestra.run.smithi037.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-03-29T07:59:59.722 INFO:teuthology.orchestra.run.smithi037.stderr: stderr: 10+0 records in 2024-03-29T07:59:59.722 INFO:teuthology.orchestra.run.smithi037.stderr:10+0 records out 2024-03-29T07:59:59.723 INFO:teuthology.orchestra.run.smithi037.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0573609 s, 183 MB/s 2024-03-29T07:59:59.723 INFO:teuthology.orchestra.run.smithi037.stderr:--> Zapping successful for: 2024-03-29T08:00:00.513 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 99b492c2-eda1-11ee-b647-cb9ed24678a4 -- ceph orch daemon add osd smithi037:vg_nvme/lv_2 2024-03-29T08:00:00.807 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:00:00 smithi037 ceph-mon[35162]: pgmap v65: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-29T08:00:00.807 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:00:00 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:00:00.807 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:00:00 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:00:00.807 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:00:00 smithi037 ceph-mon[35162]: overall HEALTH_OK 2024-03-29T08:00:01.055 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:00:00 smithi148 ceph-mon[44992]: pgmap v65: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-29T08:00:01.055 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:00:00 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:00:01.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:00:00 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:00:01.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:00:00 smithi148 ceph-mon[44992]: overall HEALTH_OK 2024-03-29T08:00:01.737 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/mon.smithi037/config 2024-03-29T08:00:02.856 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:00:02 smithi037 ceph-mon[35162]: pgmap v66: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-29T08:00:03.055 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:00:02 smithi148 ceph-mon[44992]: pgmap v66: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-29T08:00:04.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:00:04.058+0000 7f5e35a00700 1 -- 172.21.15.37:0/1594236898 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5e30071f50 msgr2=0x7f5e30070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:00:04.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:00:04.058+0000 7f5e35a00700 1 --2- 172.21.15.37:0/1594236898 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5e30071f50 0x7f5e30070460 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f5e20004940 tx=0x7f5e20009e30 comp rx=0 tx=0).stop 2024-03-29T08:00:04.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:00:04.058+0000 7f5e35a00700 1 -- 172.21.15.37:0/1594236898 shutdown_connections 2024-03-29T08:00:04.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:00:04.058+0000 7f5e35a00700 1 --2- 172.21.15.37:0/1594236898 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5e300709a0 0x7f5e30070e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:00:04.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:00:04.058+0000 7f5e35a00700 1 --2- 172.21.15.37:0/1594236898 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5e30071f50 0x7f5e30070460 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:00:04.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:00:04.058+0000 7f5e35a00700 1 -- 172.21.15.37:0/1594236898 >> 172.21.15.37:0/1594236898 conn(0x7f5e300f8560 msgr2=0x7f5e300fa970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:00:04.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:00:04.059+0000 7f5e35a00700 1 -- 172.21.15.37:0/1594236898 shutdown_connections 2024-03-29T08:00:04.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:00:04.059+0000 7f5e35a00700 1 -- 172.21.15.37:0/1594236898 wait complete. 2024-03-29T08:00:04.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:00:04.060+0000 7f5e35a00700 1 Processor -- start 2024-03-29T08:00:04.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:00:04.060+0000 7f5e35a00700 1 -- start start 2024-03-29T08:00:04.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:00:04.060+0000 7f5e35a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5e300709a0 0x7f5e3010fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:00:04.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:00:04.060+0000 7f5e35a00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5e30071f50 0x7f5e30110090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:00:04.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:00:04.060+0000 7f5e35a00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e30110690 con 0x7f5e300709a0 2024-03-29T08:00:04.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:00:04.060+0000 7f5e35a00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e30110ee0 con 0x7f5e30071f50 2024-03-29T08:00:04.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:00:04.061+0000 7f5e2ea00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5e300709a0 0x7f5e3010fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:00:04.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:00:04.061+0000 7f5e2ea00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5e300709a0 0x7f5e3010fb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44662/0 (socket says 172.21.15.37:44662) 2024-03-29T08:00:04.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:00:04.061+0000 7f5e2ea00700 1 -- 172.21.15.37:0/3129505764 learned_addr learned my addr 172.21.15.37:0/3129505764 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:00:04.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:00:04.061+0000 7f5e2e000700 1 --2- 172.21.15.37:0/3129505764 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5e30071f50 0x7f5e30110090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:00:04.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:00:04.061+0000 7f5e2ea00700 1 -- 172.21.15.37:0/3129505764 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5e30071f50 msgr2=0x7f5e30110090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:00:04.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:00:04.061+0000 7f5e2ea00700 1 --2- 172.21.15.37:0/3129505764 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5e30071f50 0x7f5e30110090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:00:04.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:00:04.061+0000 7f5e2ea00700 1 -- 172.21.15.37:0/3129505764 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e20005040 con 0x7f5e300709a0 2024-03-29T08:00:04.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:00:04.061+0000 7f5e2ea00700 1 --2- 172.21.15.37:0/3129505764 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5e300709a0 0x7f5e3010fb50 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f5e2000b3b0 tx=0x7f5e20009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:00:04.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:00:04.062+0000 7f5e26a00700 1 -- 172.21.15.37:0/3129505764 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5e2000e040 con 0x7f5e300709a0 2024-03-29T08:00:04.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:00:04.062+0000 7f5e26a00700 1 -- 172.21.15.37:0/3129505764 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f5e20004290 con 0x7f5e300709a0 2024-03-29T08:00:04.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:00:04.062+0000 7f5e35a00700 1 -- 172.21.15.37:0/3129505764 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e30111160 con 0x7f5e300709a0 2024-03-29T08:00:04.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:00:04.062+0000 7f5e26a00700 1 -- 172.21.15.37:0/3129505764 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5e2001c3f0 con 0x7f5e300709a0 2024-03-29T08:00:04.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:00:04.062+0000 7f5e35a00700 1 -- 172.21.15.37:0/3129505764 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e301116a0 con 0x7f5e300709a0 2024-03-29T08:00:04.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:00:04.063+0000 7f5e35a00700 1 -- 172.21.15.37:0/3129505764 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e3005f5f0 con 0x7f5e300709a0 2024-03-29T08:00:04.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:00:04.064+0000 7f5e26a00700 1 -- 172.21.15.37:0/3129505764 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 18) v1 ==== 86565+0+0 (secure 0 0 0) 0x7f5e2000bc50 con 0x7f5e300709a0 2024-03-29T08:00:04.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:00:04.065+0000 7f5e26a00700 1 --2- 172.21.15.37:0/3129505764 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f5e14069210 0x7f5e1406b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:00:04.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:00:04.065+0000 7f5e26a00700 1 -- 172.21.15.37:0/3129505764 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 2410+0+0 (secure 0 0 0) 0x7f5e20087740 con 0x7f5e300709a0 2024-03-29T08:00:04.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:00:04.066+0000 7f5e2e000700 1 --2- 172.21.15.37:0/3129505764 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f5e14069210 0x7f5e1406b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:00:04.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:00:04.066+0000 7f5e2e000700 1 --2- 172.21.15.37:0/3129505764 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f5e14069210 0x7f5e1406b6c0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f5e1c004990 tx=0x7f5e1c009ee0 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:00:04.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:00:04.070+0000 7f5e26a00700 1 -- 172.21.15.37:0/3129505764 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f5e20058270 con 0x7f5e300709a0 2024-03-29T08:00:04.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:00:04.284+0000 7f5e35a00700 1 -- 172.21.15.37:0/3129505764 --> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi037:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f5e301119d0 con 0x7f5e14069210 2024-03-29T08:00:04.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:00:04 smithi037 ceph-mon[35162]: pgmap v67: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-29T08:00:04.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:00:04 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-29T08:00:04.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:00:04 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-29T08:00:04.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:00:04 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T08:00:05.055 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:00:04 smithi148 ceph-mon[44992]: pgmap v67: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-29T08:00:05.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:00:04 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-29T08:00:05.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:00:04 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-29T08:00:05.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:00:04 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T08:00:05.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:00:05 smithi037 ceph-mon[35162]: from='client.14278 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi037:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:00:05.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:00:05 smithi037 ceph-mon[35162]: Detected new or changed devices on smithi037 2024-03-29T08:00:05.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:00:05 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:00:05.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:00:05 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:00:05.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:00:05 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-29T08:00:05.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:00:05 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-29T08:00:05.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:00:05 smithi037 ceph-mon[35162]: Adjusting osd_memory_target on smithi037 to 6525M 2024-03-29T08:00:05.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:00:05 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:00:05.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:00:05 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T08:00:05.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:00:05 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T08:00:05.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:00:05 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:00:05.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:00:05 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T08:00:06.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:00:05 smithi148 ceph-mon[44992]: from='client.14278 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi037:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:00:06.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:00:05 smithi148 ceph-mon[44992]: Detected new or changed devices on smithi037 2024-03-29T08:00:06.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:00:05 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:00:06.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:00:05 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:00:06.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:00:05 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-29T08:00:06.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:00:05 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-29T08:00:06.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:00:05 smithi148 ceph-mon[44992]: Adjusting osd_memory_target on smithi037 to 6525M 2024-03-29T08:00:06.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:00:05 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:00:06.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:00:05 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T08:00:06.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:00:05 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T08:00:06.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:00:05 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:00:06.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:00:05 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T08:00:06.676 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:00:06 smithi037 ceph-mon[35162]: pgmap v68: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-29T08:00:07.055 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:00:06 smithi148 ceph-mon[44992]: pgmap v68: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-29T08:00:07.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:00:07 smithi037 ceph-mon[35162]: from='client.? 172.21.15.37:0/3737432348' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bba0f95c-f4e3-404e-adf7-b90353a96af4"}]: dispatch 2024-03-29T08:00:07.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:00:07 smithi037 ceph-mon[35162]: from='client.? 172.21.15.37:0/3737432348' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bba0f95c-f4e3-404e-adf7-b90353a96af4"}]': finished 2024-03-29T08:00:07.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:00:07 smithi037 ceph-mon[35162]: osdmap e16: 3 total, 2 up, 3 in 2024-03-29T08:00:07.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:00:07 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-29T08:00:07.996 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:00:07 smithi148 ceph-mon[44992]: from='client.? 172.21.15.37:0/3737432348' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bba0f95c-f4e3-404e-adf7-b90353a96af4"}]: dispatch 2024-03-29T08:00:07.997 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:00:07 smithi148 ceph-mon[44992]: from='client.? 172.21.15.37:0/3737432348' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bba0f95c-f4e3-404e-adf7-b90353a96af4"}]': finished 2024-03-29T08:00:07.997 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:00:07 smithi148 ceph-mon[44992]: osdmap e16: 3 total, 2 up, 3 in 2024-03-29T08:00:07.997 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:00:07 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-29T08:00:08.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:00:08 smithi037 ceph-mon[35162]: pgmap v69: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-29T08:00:08.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:00:08 smithi037 ceph-mon[35162]: from='client.? 172.21.15.37:0/2216086752' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-29T08:00:09.055 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:00:08 smithi148 ceph-mon[44992]: pgmap v69: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-29T08:00:09.055 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:00:08 smithi148 ceph-mon[44992]: from='client.? 172.21.15.37:0/2216086752' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-29T08:00:10.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:00:10 smithi037 ceph-mon[35162]: pgmap v71: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-29T08:00:10.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:00:10 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:00:11.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:00:10 smithi148 ceph-mon[44992]: pgmap v71: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-29T08:00:11.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:00:10 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:00:12.555 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:00:12 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:00:12.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:00:12 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:00:12.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:00:12 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T08:00:12.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:00:12 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T08:00:12.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:00:12 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:00:12.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:00:12 smithi148 ceph-mon[44992]: pgmap v72: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-29T08:00:12.590 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:00:12 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:00:12.590 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:00:12 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:00:12.590 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:00:12 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T08:00:12.590 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:00:12 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T08:00:12.590 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:00:12 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:00:12.590 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:00:12 smithi037 ceph-mon[35162]: pgmap v72: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-29T08:00:14.676 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:00:14 smithi037 ceph-mon[35162]: pgmap v73: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-29T08:00:14.805 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:00:14 smithi148 ceph-mon[44992]: pgmap v73: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-29T08:00:16.805 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:00:16 smithi148 ceph-mon[44992]: pgmap v74: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-29T08:00:16.805 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:00:16 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-29T08:00:16.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:00:16 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T08:00:16.907 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:00:16 smithi037 ceph-mon[35162]: pgmap v74: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-29T08:00:16.907 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:00:16 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-29T08:00:16.907 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:00:16 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T08:00:17.676 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:00:17 smithi037 ceph-mon[35162]: Deploying daemon osd.2 on smithi037 2024-03-29T08:00:17.805 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:00:17 smithi148 ceph-mon[44992]: Deploying daemon osd.2 on smithi037 2024-03-29T08:00:18.805 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:00:18 smithi148 ceph-mon[44992]: pgmap v75: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-29T08:00:18.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:00:18 smithi037 ceph-mon[35162]: pgmap v75: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-29T08:00:20.805 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:00:20 smithi148 ceph-mon[44992]: pgmap v76: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-29T08:00:20.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:00:20 smithi037 ceph-mon[35162]: pgmap v76: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-29T08:00:22.805 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:00:22 smithi148 ceph-mon[44992]: pgmap v77: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-29T08:00:22.881 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:00:22 smithi037 ceph-mon[35162]: pgmap v77: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-29T08:00:24.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:00:24 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:00:24.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:00:24 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T08:00:24.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:00:24 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:00:24.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:00:24 smithi037 ceph-mon[35162]: pgmap v78: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-29T08:00:24.555 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:00:24 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:00:24.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:00:24 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T08:00:24.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:00:24 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:00:24.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:00:24 smithi148 ceph-mon[44992]: pgmap v78: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-29T08:00:25.678 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:00:25 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:00:25.805 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:00:25 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:00:26.805 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:00:26 smithi148 ceph-mon[44992]: pgmap v79: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-29T08:00:26.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:00:26 smithi037 ceph-mon[35162]: pgmap v79: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-29T08:00:27.805 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:00:27 smithi148 ceph-mon[44992]: from='osd.2 [v2:172.21.15.37:6818/4002901863,v1:172.21.15.37:6819/4002901863]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-29T08:00:27.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:00:27 smithi037 ceph-mon[35162]: from='osd.2 [v2:172.21.15.37:6818/4002901863,v1:172.21.15.37:6819/4002901863]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-29T08:00:28.156 INFO:teuthology.orchestra.run.smithi037.stdout:Created osd(s) 2 on host 'smithi037' 2024-03-29T08:00:28.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:00:28.155+0000 7f5e26a00700 1 -- 172.21.15.37:0/3129505764 <== mgr.14208 v2:172.21.15.37:6800/1802836043 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f5e301119d0 con 0x7f5e14069210 2024-03-29T08:00:28.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:00:28.158+0000 7f5e35a00700 1 -- 172.21.15.37:0/3129505764 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f5e14069210 msgr2=0x7f5e1406b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:00:28.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:00:28.158+0000 7f5e35a00700 1 --2- 172.21.15.37:0/3129505764 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f5e14069210 0x7f5e1406b6c0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f5e1c004990 tx=0x7f5e1c009ee0 comp rx=0 tx=0).stop 2024-03-29T08:00:28.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:00:28.158+0000 7f5e35a00700 1 -- 172.21.15.37:0/3129505764 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5e300709a0 msgr2=0x7f5e3010fb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:00:28.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:00:28.158+0000 7f5e35a00700 1 --2- 172.21.15.37:0/3129505764 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5e300709a0 0x7f5e3010fb50 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f5e2000b3b0 tx=0x7f5e20009e30 comp rx=0 tx=0).stop 2024-03-29T08:00:28.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:00:28.158+0000 7f5e35a00700 1 -- 172.21.15.37:0/3129505764 shutdown_connections 2024-03-29T08:00:28.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:00:28.158+0000 7f5e35a00700 1 --2- 172.21.15.37:0/3129505764 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f5e14069210 0x7f5e1406b6c0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:00:28.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:00:28.158+0000 7f5e35a00700 1 --2- 172.21.15.37:0/3129505764 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5e300709a0 0x7f5e3010fb50 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:00:28.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:00:28.158+0000 7f5e35a00700 1 --2- 172.21.15.37:0/3129505764 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5e30071f50 0x7f5e30110090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:00:28.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:00:28.158+0000 7f5e35a00700 1 -- 172.21.15.37:0/3129505764 >> 172.21.15.37:0/3129505764 conn(0x7f5e300f8560 msgr2=0x7f5e300f9d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:00:28.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:00:28.158+0000 7f5e35a00700 1 -- 172.21.15.37:0/3129505764 shutdown_connections 2024-03-29T08:00:28.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:00:28.158+0000 7f5e35a00700 1 -- 172.21.15.37:0/3129505764 wait complete. 2024-03-29T08:00:28.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:00:28 smithi148 ceph-mon[44992]: pgmap v80: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-29T08:00:28.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:00:28 smithi148 ceph-mon[44992]: from='osd.2 [v2:172.21.15.37:6818/4002901863,v1:172.21.15.37:6819/4002901863]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-29T08:00:28.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:00:28 smithi148 ceph-mon[44992]: osdmap e17: 3 total, 2 up, 3 in 2024-03-29T08:00:28.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:00:28 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-29T08:00:28.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:00:28 smithi148 ceph-mon[44992]: from='osd.2 [v2:172.21.15.37:6818/4002901863,v1:172.21.15.37:6819/4002901863]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]: dispatch 2024-03-29T08:00:28.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:00:28 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:00:28.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:00:28 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:00:28.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:00:28 smithi037 ceph-mon[35162]: pgmap v80: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-29T08:00:28.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:00:28 smithi037 ceph-mon[35162]: from='osd.2 [v2:172.21.15.37:6818/4002901863,v1:172.21.15.37:6819/4002901863]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-29T08:00:28.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:00:28 smithi037 ceph-mon[35162]: osdmap e17: 3 total, 2 up, 3 in 2024-03-29T08:00:28.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:00:28 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-29T08:00:28.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:00:28 smithi037 ceph-mon[35162]: from='osd.2 [v2:172.21.15.37:6818/4002901863,v1:172.21.15.37:6819/4002901863]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]: dispatch 2024-03-29T08:00:28.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:00:28 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:00:28.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:00:28 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:00:29.155 DEBUG:teuthology.orchestra.run.smithi037:osd.2> sudo journalctl -f -n 0 -u ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4@osd.2.service 2024-03-29T08:00:29.159 INFO:tasks.cephadm:Deploying osd.3 on smithi148 with /dev/vg_nvme/lv_4... 2024-03-29T08:00:29.159 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 99b492c2-eda1-11ee-b647-cb9ed24678a4 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-03-29T08:00:29.231 INFO:journalctl@ceph.osd.2.smithi037.stdout:-- Logs begin at Fri 2024-03-29 07:43:01 UTC. -- 2024-03-29T08:00:29.576 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/mon.smithi148/config 2024-03-29T08:00:29.805 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:00:29 smithi148 ceph-mon[44992]: from='osd.2 [v2:172.21.15.37:6818/4002901863,v1:172.21.15.37:6819/4002901863]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]': finished 2024-03-29T08:00:29.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:00:29 smithi148 ceph-mon[44992]: osdmap e18: 3 total, 2 up, 3 in 2024-03-29T08:00:29.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:00:29 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-29T08:00:29.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:00:29 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-29T08:00:29.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:00:29 smithi037 ceph-mon[35162]: from='osd.2 [v2:172.21.15.37:6818/4002901863,v1:172.21.15.37:6819/4002901863]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]': finished 2024-03-29T08:00:29.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:00:29 smithi037 ceph-mon[35162]: osdmap e18: 3 total, 2 up, 3 in 2024-03-29T08:00:29.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:00:29 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-29T08:00:29.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:00:29 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-29T08:00:30.805 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:00:30 smithi148 ceph-mon[44992]: purged_snaps scrub starts 2024-03-29T08:00:30.805 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:00:30 smithi148 ceph-mon[44992]: purged_snaps scrub ok 2024-03-29T08:00:30.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:00:30 smithi148 ceph-mon[44992]: pgmap v83: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-29T08:00:30.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:00:30 smithi148 ceph-mon[44992]: osd.2 [v2:172.21.15.37:6818/4002901863,v1:172.21.15.37:6819/4002901863] boot 2024-03-29T08:00:30.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:00:30 smithi148 ceph-mon[44992]: osdmap e19: 3 total, 3 up, 3 in 2024-03-29T08:00:30.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:00:30 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-29T08:00:30.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:00:30 smithi037 ceph-mon[35162]: purged_snaps scrub starts 2024-03-29T08:00:30.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:00:30 smithi037 ceph-mon[35162]: purged_snaps scrub ok 2024-03-29T08:00:30.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:00:30 smithi037 ceph-mon[35162]: pgmap v83: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-29T08:00:30.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:00:30 smithi037 ceph-mon[35162]: osd.2 [v2:172.21.15.37:6818/4002901863,v1:172.21.15.37:6819/4002901863] boot 2024-03-29T08:00:30.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:00:30 smithi037 ceph-mon[35162]: osdmap e19: 3 total, 3 up, 3 in 2024-03-29T08:00:30.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:00:30 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-29T08:00:32.016 INFO:teuthology.orchestra.run.smithi148.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-03-29T08:00:32.017 INFO:teuthology.orchestra.run.smithi148.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-03-29T08:00:32.017 INFO:teuthology.orchestra.run.smithi148.stderr: stderr: 10+0 records in 2024-03-29T08:00:32.017 INFO:teuthology.orchestra.run.smithi148.stderr:10+0 records out 2024-03-29T08:00:32.017 INFO:teuthology.orchestra.run.smithi148.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0451026 s, 232 MB/s 2024-03-29T08:00:32.017 INFO:teuthology.orchestra.run.smithi148.stderr:--> Zapping successful for: 2024-03-29T08:00:32.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:00:32 smithi148 ceph-mon[44992]: osdmap e20: 3 total, 3 up, 3 in 2024-03-29T08:00:32.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:00:32 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:00:32.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:00:32 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:00:32.423 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:00:32 smithi037 ceph-mon[35162]: osdmap e20: 3 total, 3 up, 3 in 2024-03-29T08:00:32.423 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:00:32 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:00:32.423 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:00:32 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:00:32.776 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 99b492c2-eda1-11ee-b647-cb9ed24678a4 -- ceph orch daemon add osd smithi148:vg_nvme/lv_4 2024-03-29T08:00:33.190 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/mon.smithi148/config 2024-03-29T08:00:33.305 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:00:33 smithi148 ceph-mon[44992]: pgmap v86: 0 pgs: ; 0 B data, 872 MiB used, 267 GiB / 268 GiB avail 2024-03-29T08:00:33.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:00:33 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]: dispatch 2024-03-29T08:00:33.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:00:33 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]': finished 2024-03-29T08:00:33.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:00:33 smithi148 ceph-mon[44992]: osdmap e21: 3 total, 3 up, 3 in 2024-03-29T08:00:33.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:00:33 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-03-29T08:00:33.347 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:00:33 smithi037 ceph-mon[35162]: pgmap v86: 0 pgs: ; 0 B data, 872 MiB used, 267 GiB / 268 GiB avail 2024-03-29T08:00:33.347 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:00:33 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]: dispatch 2024-03-29T08:00:33.347 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:00:33 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]': finished 2024-03-29T08:00:33.347 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:00:33 smithi037 ceph-mon[35162]: osdmap e21: 3 total, 3 up, 3 in 2024-03-29T08:00:33.347 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:00:33 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-03-29T08:00:34.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:00:34 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-03-29T08:00:34.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:00:34 smithi037 ceph-mon[35162]: osdmap e22: 3 total, 3 up, 3 in 2024-03-29T08:00:34.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:00:34 smithi037 ceph-mon[35162]: pgmap v89: 1 pgs: 1 unknown; 0 B data, 872 MiB used, 267 GiB / 268 GiB avail 2024-03-29T08:00:34.458 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:00:34 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-03-29T08:00:34.458 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:00:34 smithi148 ceph-mon[44992]: osdmap e22: 3 total, 3 up, 3 in 2024-03-29T08:00:34.458 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:00:34 smithi148 ceph-mon[44992]: pgmap v89: 1 pgs: 1 unknown; 0 B data, 872 MiB used, 267 GiB / 268 GiB avail 2024-03-29T08:00:34.926 INFO:journalctl@ceph.osd.0.smithi037.stdout:Mar 29 08:00:34 smithi037 sudo[79723]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-29T08:00:34.926 INFO:journalctl@ceph.osd.0.smithi037.stdout:Mar 29 08:00:34 smithi037 sudo[79723]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-29T08:00:34.926 INFO:journalctl@ceph.osd.0.smithi037.stdout:Mar 29 08:00:34 smithi037 sudo[79723]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-29T08:00:34.927 INFO:journalctl@ceph.osd.0.smithi037.stdout:Mar 29 08:00:34 smithi037 sudo[79723]: pam_unix(sudo:session): session closed for user root 2024-03-29T08:00:35.118 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T08:00:35.118+0000 7fe4b9c00700 1 -- 172.21.15.148:0/4073995216 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe4b40fe0d0 msgr2=0x7fe4b40fe540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:00:35.119 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T08:00:35.118+0000 7fe4b9c00700 1 --2- 172.21.15.148:0/4073995216 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe4b40fe0d0 0x7fe4b40fe540 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fe49c004970 tx=0x7fe49c009e30 comp rx=0 tx=0).stop 2024-03-29T08:00:35.119 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T08:00:35.118+0000 7fe4b9c00700 1 -- 172.21.15.148:0/4073995216 shutdown_connections 2024-03-29T08:00:35.119 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T08:00:35.118+0000 7fe4b9c00700 1 --2- 172.21.15.148:0/4073995216 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe4b40fe0d0 0x7fe4b40fe540 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:00:35.119 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T08:00:35.118+0000 7fe4b9c00700 1 --2- 172.21.15.148:0/4073995216 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe4b40fcee0 0x7fe4b40fd310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:00:35.119 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T08:00:35.118+0000 7fe4b9c00700 1 -- 172.21.15.148:0/4073995216 >> 172.21.15.148:0/4073995216 conn(0x7fe4b40f8560 msgr2=0x7fe4b40fa970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:00:35.119 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T08:00:35.118+0000 7fe4b9c00700 1 -- 172.21.15.148:0/4073995216 shutdown_connections 2024-03-29T08:00:35.119 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T08:00:35.118+0000 7fe4b9c00700 1 -- 172.21.15.148:0/4073995216 wait complete. 2024-03-29T08:00:35.119 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T08:00:35.119+0000 7fe4b9c00700 1 Processor -- start 2024-03-29T08:00:35.119 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T08:00:35.119+0000 7fe4b9c00700 1 -- start start 2024-03-29T08:00:35.121 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T08:00:35.119+0000 7fe4b9c00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe4b40fcee0 0x7fe4b410faf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:00:35.121 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T08:00:35.119+0000 7fe4b9c00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe4b40fe0d0 0x7fe4b4110030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:00:35.121 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T08:00:35.119+0000 7fe4b9c00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4b4110630 con 0x7fe4b40fe0d0 2024-03-29T08:00:35.121 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T08:00:35.119+0000 7fe4b9c00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4b4110e80 con 0x7fe4b40fcee0 2024-03-29T08:00:35.121 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T08:00:35.120+0000 7fe4b2a00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe4b40fcee0 0x7fe4b410faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:00:35.121 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T08:00:35.120+0000 7fe4b2a00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe4b40fcee0 0x7fe4b410faf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:52188/0 (socket says 172.21.15.148:52188) 2024-03-29T08:00:35.121 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T08:00:35.120+0000 7fe4b2a00700 1 -- 172.21.15.148:0/3162436302 learned_addr learned my addr 172.21.15.148:0/3162436302 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T08:00:35.121 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T08:00:35.120+0000 7fe4abe00700 1 --2- 172.21.15.148:0/3162436302 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe4b40fe0d0 0x7fe4b4110030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:00:35.121 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T08:00:35.120+0000 7fe4b2a00700 1 -- 172.21.15.148:0/3162436302 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe4b40fe0d0 msgr2=0x7fe4b4110030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:00:35.121 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T08:00:35.120+0000 7fe4b2a00700 1 --2- 172.21.15.148:0/3162436302 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe4b40fe0d0 0x7fe4b4110030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:00:35.122 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T08:00:35.120+0000 7fe4b2a00700 1 -- 172.21.15.148:0/3162436302 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe49c005040 con 0x7fe4b40fcee0 2024-03-29T08:00:35.122 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T08:00:35.120+0000 7fe4b2a00700 1 --2- 172.21.15.148:0/3162436302 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe4b40fcee0 0x7fe4b410faf0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fe4a400eb70 tx=0x7fe4a400b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:00:35.122 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T08:00:35.121+0000 7fe4aaa00700 1 -- 172.21.15.148:0/3162436302 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe4a4007cc0 con 0x7fe4b40fcee0 2024-03-29T08:00:35.122 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T08:00:35.121+0000 7fe4aaa00700 1 -- 172.21.15.148:0/3162436302 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fe4a400f040 con 0x7fe4b40fcee0 2024-03-29T08:00:35.122 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T08:00:35.121+0000 7fe4b9c00700 1 -- 172.21.15.148:0/3162436302 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4b4111100 con 0x7fe4b40fcee0 2024-03-29T08:00:35.122 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T08:00:35.121+0000 7fe4aaa00700 1 -- 172.21.15.148:0/3162436302 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe4a400c9d0 con 0x7fe4b40fcee0 2024-03-29T08:00:35.122 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T08:00:35.121+0000 7fe4b9c00700 1 -- 172.21.15.148:0/3162436302 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4b4111640 con 0x7fe4b40fcee0 2024-03-29T08:00:35.122 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T08:00:35.122+0000 7fe4b9c00700 1 -- 172.21.15.148:0/3162436302 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4b4101a60 con 0x7fe4b40fcee0 2024-03-29T08:00:35.122 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T08:00:35.123+0000 7fe4aaa00700 1 -- 172.21.15.148:0/3162436302 <== mon.1 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 18) v1 ==== 86565+0+0 (secure 0 0 0) 0x7fe4a400cc40 con 0x7fe4b40fcee0 2024-03-29T08:00:35.123 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T08:00:35.123+0000 7fe4aaa00700 1 --2- 172.21.15.148:0/3162436302 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7fe498069210 0x7fe49806b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:00:35.123 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T08:00:35.124+0000 7fe4aaa00700 1 -- 172.21.15.148:0/3162436302 <== mon.1 v2:172.21.15.148:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 3228+0+0 (secure 0 0 0) 0x7fe4a40862e0 con 0x7fe4b40fcee0 2024-03-29T08:00:35.123 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T08:00:35.124+0000 7fe4abe00700 1 --2- 172.21.15.148:0/3162436302 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7fe498069210 0x7fe49806b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:00:35.124 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T08:00:35.124+0000 7fe4abe00700 1 --2- 172.21.15.148:0/3162436302 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7fe498069210 0x7fe49806b6c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fe49c009fd0 tx=0x7fe49c009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:00:35.128 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T08:00:35.129+0000 7fe4aaa00700 1 -- 172.21.15.148:0/3162436302 <== mon.1 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fe4a404e190 con 0x7fe4b40fcee0 2024-03-29T08:00:35.329 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T08:00:35.329+0000 7fe4b9c00700 1 -- 172.21.15.148:0/3162436302 --> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi148:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7fe4b4111860 con 0x7fe498069210 2024-03-29T08:00:35.426 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:00:35 smithi037 ceph-mon[35162]: osdmap e23: 3 total, 3 up, 3 in 2024-03-29T08:00:35.427 INFO:journalctl@ceph.osd.0.smithi037.stdout:Mar 29 08:00:35 smithi037 sudo[82777]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-29T08:00:35.427 INFO:journalctl@ceph.osd.0.smithi037.stdout:Mar 29 08:00:35 smithi037 sudo[82777]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-29T08:00:35.427 INFO:journalctl@ceph.osd.0.smithi037.stdout:Mar 29 08:00:35 smithi037 sudo[82777]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-29T08:00:35.427 INFO:journalctl@ceph.osd.0.smithi037.stdout:Mar 29 08:00:35 smithi037 sudo[82777]: pam_unix(sudo:session): session closed for user root 2024-03-29T08:00:35.555 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:00:35 smithi148 ceph-mon[44992]: osdmap e23: 3 total, 3 up, 3 in 2024-03-29T08:00:35.827 INFO:journalctl@ceph.osd.1.smithi037.stdout:Mar 29 08:00:35 smithi037 sudo[82788]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-29T08:00:35.827 INFO:journalctl@ceph.osd.1.smithi037.stdout:Mar 29 08:00:35 smithi037 sudo[82788]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-29T08:00:35.827 INFO:journalctl@ceph.osd.1.smithi037.stdout:Mar 29 08:00:35 smithi037 sudo[82788]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-29T08:00:35.827 INFO:journalctl@ceph.osd.1.smithi037.stdout:Mar 29 08:00:35 smithi037 sudo[82788]: pam_unix(sudo:session): session closed for user root 2024-03-29T08:00:36.283 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T08:00:36.283+0000 7fe4aaa00700 1 -- 172.21.15.148:0/3162436302 <== mon.1 v2:172.21.15.148:3300/0 7 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fe4a4053310 con 0x7fe4b40fcee0 2024-03-29T08:00:36.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:00:36 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:00:36.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:00:36 smithi148 ceph-mon[44992]: from='client.24151 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi148:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:00:36.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:00:36 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-29T08:00:36.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:00:36 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-29T08:00:36.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:00:36 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T08:00:36.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:00:36 smithi148 ceph-mon[44992]: pgmap v91: 1 pgs: 1 unknown; 0 B data, 872 MiB used, 267 GiB / 268 GiB avail 2024-03-29T08:00:36.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:00:36 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:00:36.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:00:36 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:00:36.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:00:36 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-29T08:00:36.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:00:36 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-29T08:00:36.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:00:36 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-29T08:00:36.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:00:36 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:00:36.557 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:00:36 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T08:00:36.557 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:00:36 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T08:00:36.557 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:00:36 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:00:36.557 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:00:36 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T08:00:36.676 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:00:36 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:00:36.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:00:36 smithi037 ceph-mon[35162]: from='client.24151 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi148:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:00:36.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:00:36 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-29T08:00:36.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:00:36 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-29T08:00:36.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:00:36 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T08:00:36.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:00:36 smithi037 ceph-mon[35162]: pgmap v91: 1 pgs: 1 unknown; 0 B data, 872 MiB used, 267 GiB / 268 GiB avail 2024-03-29T08:00:36.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:00:36 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:00:36.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:00:36 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:00:36.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:00:36 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-29T08:00:36.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:00:36 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-29T08:00:36.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:00:36 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-29T08:00:36.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:00:36 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:00:36.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:00:36 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T08:00:36.678 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:00:36 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T08:00:36.678 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:00:36 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:00:36.678 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:00:36 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T08:00:36.678 INFO:journalctl@ceph.osd.1.smithi037.stdout:Mar 29 08:00:36 smithi037 sudo[82872]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-29T08:00:36.678 INFO:journalctl@ceph.osd.1.smithi037.stdout:Mar 29 08:00:36 smithi037 sudo[82872]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-29T08:00:36.678 INFO:journalctl@ceph.osd.1.smithi037.stdout:Mar 29 08:00:36 smithi037 sudo[82872]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-29T08:00:36.678 INFO:journalctl@ceph.osd.1.smithi037.stdout:Mar 29 08:00:36 smithi037 sudo[82872]: pam_unix(sudo:session): session closed for user root 2024-03-29T08:00:37.279 INFO:journalctl@ceph.osd.2.smithi037.stdout:Mar 29 08:00:36 smithi037 sudo[82977]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-29T08:00:37.279 INFO:journalctl@ceph.osd.2.smithi037.stdout:Mar 29 08:00:36 smithi037 sudo[82977]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-29T08:00:37.279 INFO:journalctl@ceph.osd.2.smithi037.stdout:Mar 29 08:00:36 smithi037 sudo[82977]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-29T08:00:37.279 INFO:journalctl@ceph.osd.2.smithi037.stdout:Mar 29 08:00:36 smithi037 sudo[82977]: pam_unix(sudo:session): session closed for user root 2024-03-29T08:00:37.398 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:00:37 smithi148 ceph-mon[44992]: Detected new or changed devices on smithi037 2024-03-29T08:00:37.398 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:00:37 smithi148 ceph-mon[44992]: Adjusting osd_memory_target on smithi037 to 4350M 2024-03-29T08:00:37.399 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:00:37 smithi148 ceph-mon[44992]: mgrmap e19: smithi037.wjikii(active, since 2m), standbys: smithi148.onhgvm 2024-03-29T08:00:37.676 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:00:37 smithi037 ceph-mon[35162]: Detected new or changed devices on smithi037 2024-03-29T08:00:37.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:00:37 smithi037 ceph-mon[35162]: Adjusting osd_memory_target on smithi037 to 4350M 2024-03-29T08:00:37.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:00:37 smithi037 ceph-mon[35162]: mgrmap e19: smithi037.wjikii(active, since 2m), standbys: smithi148.onhgvm 2024-03-29T08:00:37.677 INFO:journalctl@ceph.osd.2.smithi037.stdout:Mar 29 08:00:37 smithi037 sudo[83025]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-29T08:00:37.677 INFO:journalctl@ceph.osd.2.smithi037.stdout:Mar 29 08:00:37 smithi037 sudo[83025]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-29T08:00:37.677 INFO:journalctl@ceph.osd.2.smithi037.stdout:Mar 29 08:00:37 smithi037 sudo[83025]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-29T08:00:37.677 INFO:journalctl@ceph.osd.2.smithi037.stdout:Mar 29 08:00:37 smithi037 sudo[83025]: pam_unix(sudo:session): session closed for user root 2024-03-29T08:00:38.426 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:00:38 smithi037 sudo[83111]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-29T08:00:38.426 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:00:38 smithi037 sudo[83111]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-29T08:00:38.426 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:00:38 smithi037 sudo[83111]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-29T08:00:38.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:00:38 smithi037 sudo[83111]: pam_unix(sudo:session): session closed for user root 2024-03-29T08:00:38.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:00:38 smithi037 sudo[83148]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-29T08:00:38.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:00:38 smithi037 sudo[83148]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-29T08:00:38.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:00:38 smithi037 sudo[83148]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-29T08:00:38.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:00:38 smithi037 sudo[83148]: pam_unix(sudo:session): session closed for user root 2024-03-29T08:00:39.555 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:00:39 smithi148 sudo[48934]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-29T08:00:39.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:00:39 smithi148 sudo[48934]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-29T08:00:39.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:00:39 smithi148 sudo[48934]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-29T08:00:39.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:00:39 smithi148 sudo[48934]: pam_unix(sudo:session): session closed for user root 2024-03-29T08:00:40.054 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:00:39 smithi037 ceph-mon[35162]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-03-29T08:00:40.054 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:00:39 smithi037 ceph-mon[35162]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-29T08:00:40.054 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:00:39 smithi037 ceph-mon[35162]: from='client.? 172.21.15.148:0/1583097539' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8da1c140-ada5-4879-b5d9-1ed81b0c9aef"}]: dispatch 2024-03-29T08:00:40.054 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:00:39 smithi037 ceph-mon[35162]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-29T08:00:40.054 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:00:39 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "mon metadata", "id": "smithi037"}]: dispatch 2024-03-29T08:00:40.055 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:00:39 smithi037 ceph-mon[35162]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8da1c140-ada5-4879-b5d9-1ed81b0c9aef"}]: dispatch 2024-03-29T08:00:40.055 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:00:39 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "mon metadata", "id": "smithi148"}]: dispatch 2024-03-29T08:00:40.055 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:00:39 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "mon metadata", "id": "smithi037"}]: dispatch 2024-03-29T08:00:40.055 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:00:39 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "mon metadata", "id": "smithi148"}]: dispatch 2024-03-29T08:00:40.055 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:00:39 smithi148 sudo[48937]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-29T08:00:40.055 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:00:39 smithi148 sudo[48937]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-29T08:00:40.055 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:00:39 smithi148 sudo[48937]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-29T08:00:40.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:00:39 smithi148 sudo[48937]: pam_unix(sudo:session): session closed for user root 2024-03-29T08:00:40.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:00:39 smithi148 ceph-mon[44992]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-03-29T08:00:40.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:00:39 smithi148 ceph-mon[44992]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-29T08:00:40.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:00:39 smithi148 ceph-mon[44992]: from='client.? 172.21.15.148:0/1583097539' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8da1c140-ada5-4879-b5d9-1ed81b0c9aef"}]: dispatch 2024-03-29T08:00:40.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:00:39 smithi148 ceph-mon[44992]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-29T08:00:40.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:00:39 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "mon metadata", "id": "smithi037"}]: dispatch 2024-03-29T08:00:40.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:00:39 smithi148 ceph-mon[44992]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8da1c140-ada5-4879-b5d9-1ed81b0c9aef"}]: dispatch 2024-03-29T08:00:40.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:00:39 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "mon metadata", "id": "smithi148"}]: dispatch 2024-03-29T08:00:40.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:00:39 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "mon metadata", "id": "smithi037"}]: dispatch 2024-03-29T08:00:40.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:00:39 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "mon metadata", "id": "smithi148"}]: dispatch 2024-03-29T08:00:41.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:00:40 smithi148 ceph-mon[44992]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-29T08:00:41.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:00:40 smithi148 ceph-mon[44992]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-03-29T08:00:41.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:00:40 smithi148 ceph-mon[44992]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-29T08:00:41.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:00:40 smithi148 ceph-mon[44992]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8da1c140-ada5-4879-b5d9-1ed81b0c9aef"}]': finished 2024-03-29T08:00:41.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:00:40 smithi148 ceph-mon[44992]: osdmap e24: 4 total, 3 up, 4 in 2024-03-29T08:00:41.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:00:40 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-29T08:00:41.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:00:40 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:00:41.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:00:40 smithi148 ceph-mon[44992]: from='client.? 172.21.15.148:0/1749085036' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-29T08:00:41.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:00:40 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:00:41.176 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:00:40 smithi037 ceph-mon[35162]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-29T08:00:41.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:00:40 smithi037 ceph-mon[35162]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-03-29T08:00:41.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:00:40 smithi037 ceph-mon[35162]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-29T08:00:41.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:00:40 smithi037 ceph-mon[35162]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8da1c140-ada5-4879-b5d9-1ed81b0c9aef"}]': finished 2024-03-29T08:00:41.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:00:40 smithi037 ceph-mon[35162]: osdmap e24: 4 total, 3 up, 4 in 2024-03-29T08:00:41.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:00:40 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-29T08:00:41.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:00:40 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:00:41.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:00:40 smithi037 ceph-mon[35162]: from='client.? 172.21.15.148:0/1749085036' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-29T08:00:41.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:00:40 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:00:42.805 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:00:42 smithi148 ceph-mon[44992]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-03-29T08:00:42.805 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:00:42 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:00:42.805 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:00:42 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:00:42.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:00:42 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T08:00:42.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:00:42 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T08:00:42.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:00:42 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:00:42.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:00:42 smithi037 ceph-mon[35162]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-03-29T08:00:42.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:00:42 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:00:42.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:00:42 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:00:42.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:00:42 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T08:00:42.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:00:42 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T08:00:42.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:00:42 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:00:44.805 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:00:44 smithi148 ceph-mon[44992]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 3.0 KiB/s rd, 131 KiB/s wr, 9 op/s 2024-03-29T08:00:44.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:00:44 smithi037 ceph-mon[35162]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 3.0 KiB/s rd, 131 KiB/s wr, 9 op/s 2024-03-29T08:00:46.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:00:46 smithi148 ceph-mon[44992]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 2.2 KiB/s rd, 98 KiB/s wr, 6 op/s 2024-03-29T08:00:46.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:00:46 smithi037 ceph-mon[35162]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 2.2 KiB/s rd, 98 KiB/s wr, 6 op/s 2024-03-29T08:00:48.758 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:00:48 smithi148 ceph-mon[44992]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 1.8 KiB/s rd, 78 KiB/s wr, 5 op/s 2024-03-29T08:00:48.758 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:00:48 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-29T08:00:48.758 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:00:48 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T08:00:48.758 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:00:48 smithi148 ceph-mon[44992]: Deploying daemon osd.3 on smithi148 2024-03-29T08:00:48.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:00:48 smithi037 ceph-mon[35162]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 1.8 KiB/s rd, 78 KiB/s wr, 5 op/s 2024-03-29T08:00:48.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:00:48 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-29T08:00:48.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:00:48 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T08:00:48.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:00:48 smithi037 ceph-mon[35162]: Deploying daemon osd.3 on smithi148 2024-03-29T08:00:50.787 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:00:50 smithi148 ceph-mon[44992]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 1.8 KiB/s rd, 78 KiB/s wr, 5 op/s 2024-03-29T08:00:50.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:00:50 smithi037 ceph-mon[35162]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 1.8 KiB/s rd, 78 KiB/s wr, 5 op/s 2024-03-29T08:00:52.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:00:52 smithi148 ceph-mon[44992]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 1.5 KiB/s rd, 67 KiB/s wr, 4 op/s 2024-03-29T08:00:52.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:00:52 smithi037 ceph-mon[35162]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 1.5 KiB/s rd, 67 KiB/s wr, 4 op/s 2024-03-29T08:00:54.805 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:00:54 smithi148 ceph-mon[44992]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-03-29T08:00:54.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:00:54 smithi037 ceph-mon[35162]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-03-29T08:00:55.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:00:55 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T08:00:55.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:00:55 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:00:55.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:00:55 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:00:55.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:00:55 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:00:55.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:00:55 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T08:00:55.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:00:55 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:00:55.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:00:55 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:00:55.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:00:55 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:00:56.788 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:00:56 smithi148 ceph-mon[44992]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-03-29T08:00:56.929 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:00:56 smithi037 ceph-mon[35162]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-03-29T08:00:58.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:00:58 smithi148 ceph-mon[44992]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-03-29T08:00:58.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:00:58 smithi148 ceph-mon[44992]: from='osd.3 [v2:172.21.15.148:6800/267019749,v1:172.21.15.148:6801/267019749]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-29T08:00:58.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:00:58 smithi148 ceph-mon[44992]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-29T08:00:58.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:00:58 smithi037 ceph-mon[35162]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-03-29T08:00:58.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:00:58 smithi037 ceph-mon[35162]: from='osd.3 [v2:172.21.15.148:6800/267019749,v1:172.21.15.148:6801/267019749]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-29T08:00:58.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:00:58 smithi037 ceph-mon[35162]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-29T08:00:59.108 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T08:00:59.109+0000 7fe4aaa00700 1 -- 172.21.15.148:0/3162436302 <== mgr.14208 v2:172.21.15.37:6800/1802836043 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fe4b4111860 con 0x7fe498069210 2024-03-29T08:00:59.108 INFO:teuthology.orchestra.run.smithi148.stdout:Created osd(s) 3 on host 'smithi148' 2024-03-29T08:00:59.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T08:00:59.112+0000 7fe4b9c00700 1 -- 172.21.15.148:0/3162436302 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7fe498069210 msgr2=0x7fe49806b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:00:59.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T08:00:59.112+0000 7fe4b9c00700 1 --2- 172.21.15.148:0/3162436302 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7fe498069210 0x7fe49806b6c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fe49c009fd0 tx=0x7fe49c009e30 comp rx=0 tx=0).stop 2024-03-29T08:00:59.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T08:00:59.112+0000 7fe4b9c00700 1 -- 172.21.15.148:0/3162436302 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe4b40fcee0 msgr2=0x7fe4b410faf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:00:59.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T08:00:59.112+0000 7fe4b9c00700 1 --2- 172.21.15.148:0/3162436302 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe4b40fcee0 0x7fe4b410faf0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fe4a400eb70 tx=0x7fe4a400b040 comp rx=0 tx=0).stop 2024-03-29T08:00:59.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T08:00:59.112+0000 7fe4b9c00700 1 -- 172.21.15.148:0/3162436302 shutdown_connections 2024-03-29T08:00:59.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T08:00:59.112+0000 7fe4b9c00700 1 --2- 172.21.15.148:0/3162436302 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7fe498069210 0x7fe49806b6c0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:00:59.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T08:00:59.112+0000 7fe4b9c00700 1 --2- 172.21.15.148:0/3162436302 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe4b40fcee0 0x7fe4b410faf0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:00:59.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T08:00:59.112+0000 7fe4b9c00700 1 --2- 172.21.15.148:0/3162436302 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe4b40fe0d0 0x7fe4b4110030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:00:59.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T08:00:59.112+0000 7fe4b9c00700 1 -- 172.21.15.148:0/3162436302 >> 172.21.15.148:0/3162436302 conn(0x7fe4b40f8560 msgr2=0x7fe4b40f9bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:00:59.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T08:00:59.113+0000 7fe4b9c00700 1 -- 172.21.15.148:0/3162436302 shutdown_connections 2024-03-29T08:00:59.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T08:00:59.113+0000 7fe4b9c00700 1 -- 172.21.15.148:0/3162436302 wait complete. 2024-03-29T08:00:59.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:00:59 smithi148 ceph-mon[44992]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-29T08:00:59.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:00:59 smithi148 ceph-mon[44992]: osdmap e25: 4 total, 3 up, 4 in 2024-03-29T08:00:59.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:00:59 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-29T08:00:59.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:00:59 smithi148 ceph-mon[44992]: from='osd.3 [v2:172.21.15.148:6800/267019749,v1:172.21.15.148:6801/267019749]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-03-29T08:00:59.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:00:59 smithi148 ceph-mon[44992]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-03-29T08:00:59.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:00:59 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:00:59.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:00:59 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:00:59.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:00:59 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:00:59.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:00:59 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:00:59.881 DEBUG:teuthology.orchestra.run.smithi148:osd.3> sudo journalctl -f -n 0 -u ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4@osd.3.service 2024-03-29T08:00:59.885 INFO:tasks.cephadm:Deploying osd.4 on smithi148 with /dev/vg_nvme/lv_3... 2024-03-29T08:00:59.885 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 99b492c2-eda1-11ee-b647-cb9ed24678a4 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-03-29T08:00:59.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:00:59 smithi037 ceph-mon[35162]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-29T08:00:59.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:00:59 smithi037 ceph-mon[35162]: osdmap e25: 4 total, 3 up, 4 in 2024-03-29T08:00:59.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:00:59 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-29T08:00:59.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:00:59 smithi037 ceph-mon[35162]: from='osd.3 [v2:172.21.15.148:6800/267019749,v1:172.21.15.148:6801/267019749]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-03-29T08:00:59.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:00:59 smithi037 ceph-mon[35162]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-03-29T08:00:59.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:00:59 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:00:59.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:00:59 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:00:59.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:00:59 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:00:59.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:00:59 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:00:59.959 INFO:journalctl@ceph.osd.3.smithi148.stdout:-- Logs begin at Fri 2024-03-29 07:42:56 UTC. -- 2024-03-29T08:01:00.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:01:00 smithi148 ceph-mon[44992]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-03-29T08:01:00.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:01:00 smithi148 ceph-mon[44992]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]': finished 2024-03-29T08:01:00.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:01:00 smithi148 ceph-mon[44992]: osdmap e26: 4 total, 3 up, 4 in 2024-03-29T08:01:00.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:01:00 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-29T08:01:00.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:01:00 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-29T08:01:00.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:01:00 smithi148 ceph-mon[44992]: osd.3 [v2:172.21.15.148:6800/267019749,v1:172.21.15.148:6801/267019749] boot 2024-03-29T08:01:00.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:01:00 smithi148 ceph-mon[44992]: osdmap e27: 4 total, 4 up, 4 in 2024-03-29T08:01:00.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:01:00 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-29T08:01:00.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:01:00 smithi037 ceph-mon[35162]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-03-29T08:01:00.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:01:00 smithi037 ceph-mon[35162]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]': finished 2024-03-29T08:01:00.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:01:00 smithi037 ceph-mon[35162]: osdmap e26: 4 total, 3 up, 4 in 2024-03-29T08:01:00.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:01:00 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-29T08:01:00.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:01:00 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-29T08:01:00.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:01:00 smithi037 ceph-mon[35162]: osd.3 [v2:172.21.15.148:6800/267019749,v1:172.21.15.148:6801/267019749] boot 2024-03-29T08:01:00.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:01:00 smithi037 ceph-mon[35162]: osdmap e27: 4 total, 4 up, 4 in 2024-03-29T08:01:00.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:01:00 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-29T08:01:01.237 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/mon.smithi148/config 2024-03-29T08:01:02.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:01:01 smithi148 ceph-mon[44992]: purged_snaps scrub starts 2024-03-29T08:01:02.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:01:01 smithi148 ceph-mon[44992]: purged_snaps scrub ok 2024-03-29T08:01:02.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:01:01 smithi148 ceph-mon[44992]: osdmap e28: 4 total, 4 up, 4 in 2024-03-29T08:01:02.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:01:01 smithi148 ceph-mon[44992]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-03-29T08:01:02.426 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:01:01 smithi037 ceph-mon[35162]: purged_snaps scrub starts 2024-03-29T08:01:02.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:01:01 smithi037 ceph-mon[35162]: purged_snaps scrub ok 2024-03-29T08:01:02.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:01:01 smithi037 ceph-mon[35162]: osdmap e28: 4 total, 4 up, 4 in 2024-03-29T08:01:02.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:01:01 smithi037 ceph-mon[35162]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-03-29T08:01:03.257 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:01:02 smithi148 ceph-mon[44992]: osdmap e29: 4 total, 4 up, 4 in 2024-03-29T08:01:03.426 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:01:02 smithi037 ceph-mon[35162]: osdmap e29: 4 total, 4 up, 4 in 2024-03-29T08:01:04.193 INFO:teuthology.orchestra.run.smithi148.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-03-29T08:01:04.193 INFO:teuthology.orchestra.run.smithi148.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-03-29T08:01:04.193 INFO:teuthology.orchestra.run.smithi148.stderr: stderr: 10+0 records in 2024-03-29T08:01:04.193 INFO:teuthology.orchestra.run.smithi148.stderr:10+0 records out 2024-03-29T08:01:04.193 INFO:teuthology.orchestra.run.smithi148.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0488529 s, 215 MB/s 2024-03-29T08:01:04.193 INFO:teuthology.orchestra.run.smithi148.stderr:--> Zapping successful for: 2024-03-29T08:01:04.229 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:01:03 smithi148 ceph-mon[44992]: osdmap e30: 4 total, 4 up, 4 in 2024-03-29T08:01:04.229 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:01:03 smithi148 ceph-mon[44992]: pgmap v112: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 147 KiB/s, 0 objects/s recovering 2024-03-29T08:01:04.230 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:01:03 smithi148 ceph-mon[44992]: osdmap e31: 4 total, 4 up, 4 in 2024-03-29T08:01:04.426 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:01:03 smithi037 ceph-mon[35162]: osdmap e30: 4 total, 4 up, 4 in 2024-03-29T08:01:04.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:01:03 smithi037 ceph-mon[35162]: pgmap v112: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 147 KiB/s, 0 objects/s recovering 2024-03-29T08:01:04.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:01:03 smithi037 ceph-mon[35162]: osdmap e31: 4 total, 4 up, 4 in 2024-03-29T08:01:05.239 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 99b492c2-eda1-11ee-b647-cb9ed24678a4 -- ceph orch daemon add osd smithi148:vg_nvme/lv_3 2024-03-29T08:01:05.691 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/mon.smithi148/config 2024-03-29T08:01:05.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:01:05 smithi037 ceph-mon[35162]: Detected new or changed devices on smithi148 2024-03-29T08:01:05.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:01:05 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:01:05.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:01:05 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:01:05.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:01:05 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-29T08:01:05.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:01:05 smithi037 ceph-mon[35162]: Adjusting osd_memory_target on smithi148 to 16118M 2024-03-29T08:01:05.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:01:05 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:01:05.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:01:05 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T08:01:05.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:01:05 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T08:01:05.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:01:05 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:01:05.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:01:05 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T08:01:05.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:01:05 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T08:01:05.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:01:05 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T08:01:05.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:01:05 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:01:05.934 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:01:05 smithi148 ceph-mon[44992]: Detected new or changed devices on smithi148 2024-03-29T08:01:05.934 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:01:05 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:01:05.934 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:01:05 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:01:05.934 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:01:05 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-29T08:01:05.934 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:01:05 smithi148 ceph-mon[44992]: Adjusting osd_memory_target on smithi148 to 16118M 2024-03-29T08:01:05.934 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:01:05 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:01:05.934 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:01:05 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T08:01:05.934 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:01:05 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T08:01:05.935 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:01:05 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:01:05.935 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:01:05 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T08:01:05.935 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:01:05 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T08:01:05.935 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:01:05 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T08:01:05.935 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:01:05 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:01:06.951 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:01:06 smithi148 ceph-mon[44992]: pgmap v114: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 128 KiB/s, 0 objects/s recovering 2024-03-29T08:01:07.176 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:01:06 smithi037 ceph-mon[35162]: pgmap v114: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 128 KiB/s, 0 objects/s recovering 2024-03-29T08:01:07.726 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T08:01:07.726+0000 7fd14d000700 1 -- 172.21.15.148:0/333433226 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd148071f50 msgr2=0x7fd148070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:01:07.727 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T08:01:07.726+0000 7fd14d000700 1 --2- 172.21.15.148:0/333433226 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd148071f50 0x7fd148070460 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fd138004940 tx=0x7fd138009e30 comp rx=0 tx=0).stop 2024-03-29T08:01:07.727 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T08:01:07.726+0000 7fd14d000700 1 -- 172.21.15.148:0/333433226 shutdown_connections 2024-03-29T08:01:07.727 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T08:01:07.726+0000 7fd14d000700 1 --2- 172.21.15.148:0/333433226 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd1480709a0 0x7fd148070e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:01:07.727 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T08:01:07.726+0000 7fd14d000700 1 --2- 172.21.15.148:0/333433226 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd148071f50 0x7fd148070460 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:01:07.727 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T08:01:07.726+0000 7fd14d000700 1 -- 172.21.15.148:0/333433226 >> 172.21.15.148:0/333433226 conn(0x7fd1480f8560 msgr2=0x7fd1480fa970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:01:07.727 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T08:01:07.726+0000 7fd14d000700 1 -- 172.21.15.148:0/333433226 shutdown_connections 2024-03-29T08:01:07.727 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T08:01:07.727+0000 7fd14d000700 1 -- 172.21.15.148:0/333433226 wait complete. 2024-03-29T08:01:07.729 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T08:01:07.728+0000 7fd14d000700 1 Processor -- start 2024-03-29T08:01:07.730 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T08:01:07.728+0000 7fd14d000700 1 -- start start 2024-03-29T08:01:07.730 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T08:01:07.728+0000 7fd14d000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd1480709a0 0x7fd14810fad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:01:07.730 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T08:01:07.728+0000 7fd14d000700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd148071f50 0x7fd148110010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:01:07.730 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T08:01:07.728+0000 7fd14d000700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd148110610 con 0x7fd1480709a0 2024-03-29T08:01:07.730 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T08:01:07.728+0000 7fd14d000700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd148110e60 con 0x7fd148071f50 2024-03-29T08:01:07.730 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T08:01:07.729+0000 7fd145400700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd148071f50 0x7fd148110010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:01:07.730 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T08:01:07.729+0000 7fd145400700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd148071f50 0x7fd148110010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:48480/0 (socket says 172.21.15.148:48480) 2024-03-29T08:01:07.731 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T08:01:07.729+0000 7fd145400700 1 -- 172.21.15.148:0/3497450713 learned_addr learned my addr 172.21.15.148:0/3497450713 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T08:01:07.731 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T08:01:07.729+0000 7fd145e00700 1 --2- 172.21.15.148:0/3497450713 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd1480709a0 0x7fd14810fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:01:07.731 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T08:01:07.729+0000 7fd145400700 1 -- 172.21.15.148:0/3497450713 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd1480709a0 msgr2=0x7fd14810fad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:01:07.731 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T08:01:07.729+0000 7fd145400700 1 --2- 172.21.15.148:0/3497450713 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd1480709a0 0x7fd14810fad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:01:07.731 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T08:01:07.729+0000 7fd145400700 1 -- 172.21.15.148:0/3497450713 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd138005040 con 0x7fd148071f50 2024-03-29T08:01:07.731 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T08:01:07.729+0000 7fd145400700 1 --2- 172.21.15.148:0/3497450713 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd148071f50 0x7fd148110010 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fd134002a80 tx=0x7fd1340046b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:01:07.731 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T08:01:07.730+0000 7fd13e000700 1 -- 172.21.15.148:0/3497450713 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd134007cc0 con 0x7fd148071f50 2024-03-29T08:01:07.731 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T08:01:07.730+0000 7fd13e000700 1 -- 172.21.15.148:0/3497450713 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd13400f040 con 0x7fd148071f50 2024-03-29T08:01:07.731 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T08:01:07.730+0000 7fd13e000700 1 -- 172.21.15.148:0/3497450713 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd13400da60 con 0x7fd148071f50 2024-03-29T08:01:07.731 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T08:01:07.730+0000 7fd14d000700 1 -- 172.21.15.148:0/3497450713 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd148111140 con 0x7fd148071f50 2024-03-29T08:01:07.731 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T08:01:07.730+0000 7fd14d000700 1 -- 172.21.15.148:0/3497450713 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd148111680 con 0x7fd148071f50 2024-03-29T08:01:07.732 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T08:01:07.731+0000 7fd14d000700 1 -- 172.21.15.148:0/3497450713 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd14805f5f0 con 0x7fd148071f50 2024-03-29T08:01:07.732 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T08:01:07.732+0000 7fd13e000700 1 -- 172.21.15.148:0/3497450713 <== mon.1 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fd13400dbc0 con 0x7fd148071f50 2024-03-29T08:01:07.732 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T08:01:07.733+0000 7fd13e000700 1 --2- 172.21.15.148:0/3497450713 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7fd12c0692b0 0x7fd12c06b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:01:07.732 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T08:01:07.733+0000 7fd13e000700 1 -- 172.21.15.148:0/3497450713 <== mon.1 v2:172.21.15.148:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 3770+0+0 (secure 0 0 0) 0x7fd134086740 con 0x7fd148071f50 2024-03-29T08:01:07.733 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T08:01:07.733+0000 7fd145e00700 1 --2- 172.21.15.148:0/3497450713 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7fd12c0692b0 0x7fd12c06b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:01:07.734 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T08:01:07.734+0000 7fd145e00700 1 --2- 172.21.15.148:0/3497450713 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7fd12c0692b0 0x7fd12c06b760 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fd13800b3b0 tx=0x7fd138009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:01:07.742 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T08:01:07.743+0000 7fd13e000700 1 -- 172.21.15.148:0/3497450713 <== mon.1 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fd134056b00 con 0x7fd148071f50 2024-03-29T08:01:07.944 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T08:01:07.945+0000 7fd14d000700 1 -- 172.21.15.148:0/3497450713 --> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi148:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7fd148111a00 con 0x7fd12c0692b0 2024-03-29T08:01:09.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:01:08 smithi148 ceph-mon[44992]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-29T08:01:09.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:01:08 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-29T08:01:09.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:01:08 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-29T08:01:09.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:01:08 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T08:01:09.176 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:01:08 smithi037 ceph-mon[35162]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-29T08:01:09.176 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:01:08 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-29T08:01:09.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:01:08 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-29T08:01:09.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:01:08 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T08:01:10.055 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:01:09 smithi148 ceph-mon[44992]: from='client.24171 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi148:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:01:10.176 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:01:09 smithi037 ceph-mon[35162]: from='client.24171 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi148:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:01:11.176 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:01:10 smithi037 ceph-mon[35162]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-03-29T08:01:11.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:01:10 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:01:11.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:01:10 smithi148 ceph-mon[44992]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-03-29T08:01:11.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:01:10 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:01:12.176 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:01:11 smithi037 ceph-mon[35162]: from='client.? 172.21.15.148:0/2078205315' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9af70e66-bbca-48c7-bad1-7ac78270cf16"}]: dispatch 2024-03-29T08:01:12.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:01:11 smithi037 ceph-mon[35162]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9af70e66-bbca-48c7-bad1-7ac78270cf16"}]: dispatch 2024-03-29T08:01:12.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:01:11 smithi037 ceph-mon[35162]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9af70e66-bbca-48c7-bad1-7ac78270cf16"}]': finished 2024-03-29T08:01:12.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:01:11 smithi037 ceph-mon[35162]: osdmap e32: 5 total, 4 up, 5 in 2024-03-29T08:01:12.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:01:11 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-29T08:01:12.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:01:11 smithi148 ceph-mon[44992]: from='client.? 172.21.15.148:0/2078205315' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9af70e66-bbca-48c7-bad1-7ac78270cf16"}]: dispatch 2024-03-29T08:01:12.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:01:11 smithi148 ceph-mon[44992]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9af70e66-bbca-48c7-bad1-7ac78270cf16"}]: dispatch 2024-03-29T08:01:12.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:01:11 smithi148 ceph-mon[44992]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9af70e66-bbca-48c7-bad1-7ac78270cf16"}]': finished 2024-03-29T08:01:12.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:01:11 smithi148 ceph-mon[44992]: osdmap e32: 5 total, 4 up, 5 in 2024-03-29T08:01:12.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:01:11 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-29T08:01:13.176 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:01:12 smithi037 ceph-mon[35162]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-03-29T08:01:13.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:01:12 smithi037 ceph-mon[35162]: from='client.? 172.21.15.148:0/2694490429' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-29T08:01:13.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:01:12 smithi148 ceph-mon[44992]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-03-29T08:01:13.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:01:12 smithi148 ceph-mon[44992]: from='client.? 172.21.15.148:0/2694490429' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-29T08:01:15.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:01:14 smithi037 ceph-mon[35162]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-03-29T08:01:15.305 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:01:14 smithi148 ceph-mon[44992]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-03-29T08:01:17.176 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:01:16 smithi037 ceph-mon[35162]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-03-29T08:01:17.305 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:01:16 smithi148 ceph-mon[44992]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-03-29T08:01:19.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:01:18 smithi148 ceph-mon[44992]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-03-29T08:01:19.176 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:01:18 smithi037 ceph-mon[35162]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-03-29T08:01:20.176 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:01:19 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-29T08:01:20.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:01:19 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T08:01:20.305 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:01:19 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-29T08:01:20.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:01:19 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T08:01:21.176 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:01:20 smithi037 ceph-mon[35162]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-03-29T08:01:21.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:01:20 smithi037 ceph-mon[35162]: Deploying daemon osd.4 on smithi148 2024-03-29T08:01:21.229 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:01:20 smithi148 ceph-mon[44992]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-03-29T08:01:21.229 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:01:20 smithi148 ceph-mon[44992]: Deploying daemon osd.4 on smithi148 2024-03-29T08:01:23.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:01:22 smithi148 ceph-mon[44992]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-03-29T08:01:23.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:01:22 smithi037 ceph-mon[35162]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-03-29T08:01:25.176 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:01:24 smithi037 ceph-mon[35162]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-03-29T08:01:25.196 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:01:24 smithi148 ceph-mon[44992]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-03-29T08:01:26.176 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:01:25 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:01:26.216 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:01:25 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:01:27.055 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:01:26 smithi148 ceph-mon[44992]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-03-29T08:01:27.055 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:01:26 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:01:27.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:01:26 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T08:01:27.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:01:26 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:01:27.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:01:26 smithi037 ceph-mon[35162]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-03-29T08:01:27.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:01:26 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:01:27.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:01:26 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T08:01:27.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:01:26 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:01:29.145 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:01:28 smithi148 ceph-mon[44992]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-03-29T08:01:29.176 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:01:28 smithi037 ceph-mon[35162]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-03-29T08:01:30.779 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T08:01:30.778+0000 7fd13e000700 1 -- 172.21.15.148:0/3497450713 <== mgr.14208 v2:172.21.15.37:6800/1802836043 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fd148111a00 con 0x7fd12c0692b0 2024-03-29T08:01:30.779 INFO:teuthology.orchestra.run.smithi148.stdout:Created osd(s) 4 on host 'smithi148' 2024-03-29T08:01:30.781 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T08:01:30.781+0000 7fd14d000700 1 -- 172.21.15.148:0/3497450713 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7fd12c0692b0 msgr2=0x7fd12c06b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:01:30.781 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T08:01:30.781+0000 7fd14d000700 1 --2- 172.21.15.148:0/3497450713 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7fd12c0692b0 0x7fd12c06b760 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fd13800b3b0 tx=0x7fd138009e30 comp rx=0 tx=0).stop 2024-03-29T08:01:30.782 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T08:01:30.781+0000 7fd14d000700 1 -- 172.21.15.148:0/3497450713 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd148071f50 msgr2=0x7fd148110010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:01:30.782 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T08:01:30.781+0000 7fd14d000700 1 --2- 172.21.15.148:0/3497450713 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd148071f50 0x7fd148110010 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fd134002a80 tx=0x7fd1340046b0 comp rx=0 tx=0).stop 2024-03-29T08:01:30.782 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T08:01:30.782+0000 7fd14d000700 1 -- 172.21.15.148:0/3497450713 shutdown_connections 2024-03-29T08:01:30.782 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T08:01:30.782+0000 7fd14d000700 1 --2- 172.21.15.148:0/3497450713 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7fd12c0692b0 0x7fd12c06b760 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:01:30.782 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T08:01:30.782+0000 7fd14d000700 1 --2- 172.21.15.148:0/3497450713 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd1480709a0 0x7fd14810fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:01:30.782 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T08:01:30.782+0000 7fd14d000700 1 --2- 172.21.15.148:0/3497450713 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd148071f50 0x7fd148110010 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:01:30.782 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T08:01:30.782+0000 7fd14d000700 1 -- 172.21.15.148:0/3497450713 >> 172.21.15.148:0/3497450713 conn(0x7fd1480f8560 msgr2=0x7fd1480f9cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:01:30.782 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T08:01:30.782+0000 7fd14d000700 1 -- 172.21.15.148:0/3497450713 shutdown_connections 2024-03-29T08:01:30.782 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T08:01:30.782+0000 7fd14d000700 1 -- 172.21.15.148:0/3497450713 wait complete. 2024-03-29T08:01:31.055 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:01:30 smithi148 ceph-mon[44992]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-03-29T08:01:31.055 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:01:30 smithi148 ceph-mon[44992]: from='osd.4 [v2:172.21.15.148:6808/4113691443,v1:172.21.15.148:6809/4113691443]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-29T08:01:31.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:01:30 smithi148 ceph-mon[44992]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-29T08:01:31.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:01:30 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:01:31.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:01:30 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:01:31.176 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:01:30 smithi037 ceph-mon[35162]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-03-29T08:01:31.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:01:30 smithi037 ceph-mon[35162]: from='osd.4 [v2:172.21.15.148:6808/4113691443,v1:172.21.15.148:6809/4113691443]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-29T08:01:31.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:01:30 smithi037 ceph-mon[35162]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-29T08:01:31.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:01:30 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:01:31.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:01:30 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:01:31.749 DEBUG:teuthology.orchestra.run.smithi148:osd.4> sudo journalctl -f -n 0 -u ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4@osd.4.service 2024-03-29T08:01:31.754 INFO:tasks.cephadm:Deploying osd.5 on smithi148 with /dev/vg_nvme/lv_2... 2024-03-29T08:01:31.754 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 99b492c2-eda1-11ee-b647-cb9ed24678a4 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-03-29T08:01:31.827 INFO:journalctl@ceph.osd.4.smithi148.stdout:-- Logs begin at Fri 2024-03-29 07:42:56 UTC. -- 2024-03-29T08:01:32.055 INFO:journalctl@ceph.osd.4.smithi148.stdout:Mar 29 08:01:31 smithi148 ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4-osd-4[59397]: 2024-03-29T08:01:31.911+0000 7f287c800700 -1 osd.4 0 waiting for initial osdmap 2024-03-29T08:01:32.056 INFO:journalctl@ceph.osd.4.smithi148.stdout:Mar 29 08:01:31 smithi148 ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4-osd-4[59397]: 2024-03-29T08:01:31.922+0000 7f2875a00700 -1 osd.4 34 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-29T08:01:32.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:01:31 smithi148 ceph-mon[44992]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-29T08:01:32.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:01:31 smithi148 ceph-mon[44992]: osdmap e33: 5 total, 4 up, 5 in 2024-03-29T08:01:32.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:01:31 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-29T08:01:32.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:01:31 smithi148 ceph-mon[44992]: from='osd.4 [v2:172.21.15.148:6808/4113691443,v1:172.21.15.148:6809/4113691443]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-03-29T08:01:32.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:01:31 smithi148 ceph-mon[44992]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-03-29T08:01:32.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:01:31 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:01:32.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:01:31 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:01:32.176 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:01:31 smithi037 ceph-mon[35162]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-29T08:01:32.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:01:31 smithi037 ceph-mon[35162]: osdmap e33: 5 total, 4 up, 5 in 2024-03-29T08:01:32.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:01:31 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-29T08:01:32.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:01:31 smithi037 ceph-mon[35162]: from='osd.4 [v2:172.21.15.148:6808/4113691443,v1:172.21.15.148:6809/4113691443]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-03-29T08:01:32.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:01:31 smithi037 ceph-mon[35162]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-03-29T08:01:32.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:01:31 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:01:32.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:01:31 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:01:32.237 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/mon.smithi148/config 2024-03-29T08:01:33.176 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:01:32 smithi037 ceph-mon[35162]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-03-29T08:01:33.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:01:32 smithi037 ceph-mon[35162]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]': finished 2024-03-29T08:01:33.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:01:32 smithi037 ceph-mon[35162]: osdmap e34: 5 total, 4 up, 5 in 2024-03-29T08:01:33.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:01:32 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-29T08:01:33.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:01:32 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-29T08:01:33.305 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:01:32 smithi148 ceph-mon[44992]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-03-29T08:01:33.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:01:32 smithi148 ceph-mon[44992]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]': finished 2024-03-29T08:01:33.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:01:32 smithi148 ceph-mon[44992]: osdmap e34: 5 total, 4 up, 5 in 2024-03-29T08:01:33.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:01:32 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-29T08:01:33.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:01:32 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-29T08:01:34.156 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:01:33 smithi148 ceph-mon[44992]: purged_snaps scrub starts 2024-03-29T08:01:34.157 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:01:33 smithi148 ceph-mon[44992]: purged_snaps scrub ok 2024-03-29T08:01:34.157 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:01:33 smithi148 ceph-mon[44992]: osd.4 [v2:172.21.15.148:6808/4113691443,v1:172.21.15.148:6809/4113691443] boot 2024-03-29T08:01:34.157 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:01:33 smithi148 ceph-mon[44992]: osdmap e35: 5 total, 5 up, 5 in 2024-03-29T08:01:34.157 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:01:33 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-29T08:01:34.157 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:01:33 smithi148 ceph-mon[44992]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-29T08:01:34.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:01:33 smithi037 ceph-mon[35162]: purged_snaps scrub starts 2024-03-29T08:01:34.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:01:33 smithi037 ceph-mon[35162]: purged_snaps scrub ok 2024-03-29T08:01:34.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:01:33 smithi037 ceph-mon[35162]: osd.4 [v2:172.21.15.148:6808/4113691443,v1:172.21.15.148:6809/4113691443] boot 2024-03-29T08:01:34.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:01:33 smithi037 ceph-mon[35162]: osdmap e35: 5 total, 5 up, 5 in 2024-03-29T08:01:34.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:01:33 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-29T08:01:34.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:01:33 smithi037 ceph-mon[35162]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-29T08:01:34.997 INFO:teuthology.orchestra.run.smithi148.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-03-29T08:01:34.997 INFO:teuthology.orchestra.run.smithi148.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-03-29T08:01:34.997 INFO:teuthology.orchestra.run.smithi148.stderr: stderr: 10+0 records in 2024-03-29T08:01:34.997 INFO:teuthology.orchestra.run.smithi148.stderr:10+0 records out 2024-03-29T08:01:34.997 INFO:teuthology.orchestra.run.smithi148.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0323034 s, 325 MB/s 2024-03-29T08:01:34.997 INFO:teuthology.orchestra.run.smithi148.stderr:--> Zapping successful for: 2024-03-29T08:01:35.176 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:01:34 smithi037 ceph-mon[35162]: osdmap e36: 5 total, 5 up, 5 in 2024-03-29T08:01:35.282 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:01:34 smithi148 ceph-mon[44992]: osdmap e36: 5 total, 5 up, 5 in 2024-03-29T08:01:36.176 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:01:35 smithi037 ceph-mon[35162]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-29T08:01:36.206 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:01:35 smithi148 ceph-mon[44992]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-29T08:01:36.228 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 99b492c2-eda1-11ee-b647-cb9ed24678a4 -- ceph orch daemon add osd smithi148:vg_nvme/lv_2 2024-03-29T08:01:36.687 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/mon.smithi148/config 2024-03-29T08:01:38.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:01:38 smithi148 ceph-mon[44992]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-29T08:01:38.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:01:38 smithi037 ceph-mon[35162]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-29T08:01:39.535 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T08:01:39.535+0000 7f3281200700 1 -- 172.21.15.148:0/633351736 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f327c071f50 msgr2=0x7f327c070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:01:39.535 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T08:01:39.535+0000 7f3281200700 1 --2- 172.21.15.148:0/633351736 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f327c071f50 0x7f327c070460 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f326c004880 tx=0x7f326c009e30 comp rx=0 tx=0).stop 2024-03-29T08:01:39.536 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T08:01:39.535+0000 7f3281200700 1 -- 172.21.15.148:0/633351736 shutdown_connections 2024-03-29T08:01:39.536 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T08:01:39.535+0000 7f3281200700 1 --2- 172.21.15.148:0/633351736 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f327c0709a0 0x7f327c070e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:01:39.536 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T08:01:39.535+0000 7f3281200700 1 --2- 172.21.15.148:0/633351736 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f327c071f50 0x7f327c070460 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:01:39.536 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T08:01:39.535+0000 7f3281200700 1 -- 172.21.15.148:0/633351736 >> 172.21.15.148:0/633351736 conn(0x7f327c0f8560 msgr2=0x7f327c0fa970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:01:39.536 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T08:01:39.535+0000 7f3281200700 1 -- 172.21.15.148:0/633351736 shutdown_connections 2024-03-29T08:01:39.536 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T08:01:39.535+0000 7f3281200700 1 -- 172.21.15.148:0/633351736 wait complete. 2024-03-29T08:01:39.536 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T08:01:39.536+0000 7f3281200700 1 Processor -- start 2024-03-29T08:01:39.536 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T08:01:39.536+0000 7f3281200700 1 -- start start 2024-03-29T08:01:39.536 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T08:01:39.536+0000 7f3281200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f327c0709a0 0x7f327c10fad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:01:39.536 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T08:01:39.536+0000 7f3281200700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f327c071f50 0x7f327c110010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:01:39.536 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T08:01:39.536+0000 7f3281200700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f327c110610 con 0x7f327c0709a0 2024-03-29T08:01:39.536 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T08:01:39.536+0000 7f3281200700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f327c110e60 con 0x7f327c071f50 2024-03-29T08:01:39.537 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T08:01:39.537+0000 7f3279e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f327c0709a0 0x7f327c10fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:01:39.537 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T08:01:39.537+0000 7f3279400700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f327c071f50 0x7f327c110010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:01:39.537 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T08:01:39.537+0000 7f3279e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f327c0709a0 0x7f327c10fad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.148:50252/0 (socket says 172.21.15.148:50252) 2024-03-29T08:01:39.537 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T08:01:39.537+0000 7f3279e00700 1 -- 172.21.15.148:0/826843860 learned_addr learned my addr 172.21.15.148:0/826843860 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T08:01:39.537 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T08:01:39.537+0000 7f3279e00700 1 -- 172.21.15.148:0/826843860 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f327c071f50 msgr2=0x7f327c110010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:01:39.537 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T08:01:39.537+0000 7f3279e00700 1 --2- 172.21.15.148:0/826843860 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f327c071f50 0x7f327c110010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:01:39.537 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T08:01:39.537+0000 7f3279e00700 1 -- 172.21.15.148:0/826843860 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f326c005040 con 0x7f327c0709a0 2024-03-29T08:01:39.537 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T08:01:39.537+0000 7f3279400700 1 --2- 172.21.15.148:0/826843860 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f327c071f50 0x7f327c110010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T08:01:39.538 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T08:01:39.537+0000 7f3279e00700 1 --2- 172.21.15.148:0/826843860 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f327c0709a0 0x7f327c10fad0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f326c0137a0 tx=0x7f326c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:01:39.538 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T08:01:39.538+0000 7f3272000700 1 -- 172.21.15.148:0/826843860 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f326c00e030 con 0x7f327c0709a0 2024-03-29T08:01:39.538 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T08:01:39.538+0000 7f3281200700 1 -- 172.21.15.148:0/826843860 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f327c1110e0 con 0x7f327c0709a0 2024-03-29T08:01:39.539 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T08:01:39.538+0000 7f3272000700 1 -- 172.21.15.148:0/826843860 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f326c01c400 con 0x7f327c0709a0 2024-03-29T08:01:39.539 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T08:01:39.538+0000 7f3272000700 1 -- 172.21.15.148:0/826843860 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f326c012600 con 0x7f327c0709a0 2024-03-29T08:01:39.539 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T08:01:39.538+0000 7f3281200700 1 -- 172.21.15.148:0/826843860 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f327c111620 con 0x7f327c0709a0 2024-03-29T08:01:39.539 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T08:01:39.539+0000 7f3281200700 1 -- 172.21.15.148:0/826843860 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f327c05f5f0 con 0x7f327c0709a0 2024-03-29T08:01:39.539 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T08:01:39.540+0000 7f3272000700 1 -- 172.21.15.148:0/826843860 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f326c01c570 con 0x7f327c0709a0 2024-03-29T08:01:39.539 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T08:01:39.540+0000 7f3272000700 1 --2- 172.21.15.148:0/826843860 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f32600692b0 0x7f326006b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:01:39.540 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T08:01:39.541+0000 7f3272000700 1 -- 172.21.15.148:0/826843860 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 4202+0+0 (secure 0 0 0) 0x7f326c088680 con 0x7f327c0709a0 2024-03-29T08:01:39.540 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T08:01:39.541+0000 7f3279400700 1 --2- 172.21.15.148:0/826843860 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f32600692b0 0x7f326006b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:01:39.540 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T08:01:39.541+0000 7f3279400700 1 --2- 172.21.15.148:0/826843860 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f32600692b0 0x7f326006b760 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f3268009b80 tx=0x7f326800b040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:01:39.546 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T08:01:39.546+0000 7f3272000700 1 -- 172.21.15.148:0/826843860 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f326c058950 con 0x7f327c0709a0 2024-03-29T08:01:39.743 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T08:01:39.745+0000 7f3281200700 1 -- 172.21.15.148:0/826843860 --> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi148:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f327c1119b0 con 0x7f32600692b0 2024-03-29T08:01:40.305 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:01:40 smithi148 ceph-mon[44992]: Detected new or changed devices on smithi148 2024-03-29T08:01:40.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:01:40 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:01:40.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:01:40 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:01:40.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:01:40 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-29T08:01:40.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:01:40 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-29T08:01:40.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:01:40 smithi148 ceph-mon[44992]: Adjusting osd_memory_target on smithi148 to 8059M 2024-03-29T08:01:40.307 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:01:40 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:01:40.307 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:01:40 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T08:01:40.307 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:01:40 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T08:01:40.307 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:01:40 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:01:40.307 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:01:40 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T08:01:40.307 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:01:40 smithi148 ceph-mon[44992]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-29T08:01:40.307 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:01:40 smithi148 ceph-mon[44992]: from='client.14334 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi148:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:01:40.307 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:01:40 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-29T08:01:40.307 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:01:40 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-29T08:01:40.307 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:01:40 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T08:01:40.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:01:40 smithi037 ceph-mon[35162]: Detected new or changed devices on smithi148 2024-03-29T08:01:40.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:01:40 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:01:40.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:01:40 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:01:40.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:01:40 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-29T08:01:40.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:01:40 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-29T08:01:40.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:01:40 smithi037 ceph-mon[35162]: Adjusting osd_memory_target on smithi148 to 8059M 2024-03-29T08:01:40.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:01:40 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:01:40.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:01:40 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T08:01:40.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:01:40 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T08:01:40.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:01:40 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:01:40.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:01:40 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T08:01:40.428 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:01:40 smithi037 ceph-mon[35162]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-29T08:01:40.428 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:01:40 smithi037 ceph-mon[35162]: from='client.14334 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi148:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:01:40.428 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:01:40 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-29T08:01:40.428 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:01:40 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-29T08:01:40.428 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:01:40 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T08:01:41.318 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:01:41 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T08:01:41.318 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:01:41 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T08:01:41.318 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:01:41 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:01:41.318 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:01:41 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:01:41.426 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:01:41 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T08:01:41.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:01:41 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T08:01:41.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:01:41 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:01:41.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:01:41 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:01:42.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:01:42 smithi148 ceph-mon[44992]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-29T08:01:42.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:01:42 smithi037 ceph-mon[35162]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-29T08:01:43.426 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:01:43 smithi037 ceph-mon[35162]: from='client.? 172.21.15.148:0/1539381365' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a66944e4-38bd-4403-b45f-7de01ddfb7a1"}]: dispatch 2024-03-29T08:01:43.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:01:43 smithi037 ceph-mon[35162]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a66944e4-38bd-4403-b45f-7de01ddfb7a1"}]: dispatch 2024-03-29T08:01:43.555 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:01:43 smithi148 ceph-mon[44992]: from='client.? 172.21.15.148:0/1539381365' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a66944e4-38bd-4403-b45f-7de01ddfb7a1"}]: dispatch 2024-03-29T08:01:43.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:01:43 smithi148 ceph-mon[44992]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a66944e4-38bd-4403-b45f-7de01ddfb7a1"}]: dispatch 2024-03-29T08:01:44.426 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:01:44 smithi037 ceph-mon[35162]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a66944e4-38bd-4403-b45f-7de01ddfb7a1"}]': finished 2024-03-29T08:01:44.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:01:44 smithi037 ceph-mon[35162]: osdmap e37: 6 total, 5 up, 6 in 2024-03-29T08:01:44.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:01:44 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-29T08:01:44.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:01:44 smithi037 ceph-mon[35162]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-29T08:01:44.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:01:44 smithi037 ceph-mon[35162]: from='client.? 172.21.15.148:0/3834562004' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-29T08:01:44.555 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:01:44 smithi148 ceph-mon[44992]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a66944e4-38bd-4403-b45f-7de01ddfb7a1"}]': finished 2024-03-29T08:01:44.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:01:44 smithi148 ceph-mon[44992]: osdmap e37: 6 total, 5 up, 6 in 2024-03-29T08:01:44.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:01:44 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-29T08:01:44.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:01:44 smithi148 ceph-mon[44992]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-29T08:01:44.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:01:44 smithi148 ceph-mon[44992]: from='client.? 172.21.15.148:0/3834562004' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-29T08:01:46.805 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:01:46 smithi148 ceph-mon[44992]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-29T08:01:46.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:01:46 smithi037 ceph-mon[35162]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-29T08:01:48.775 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:01:48 smithi148 ceph-mon[44992]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-29T08:01:48.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:01:48 smithi037 ceph-mon[35162]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-29T08:01:50.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:01:50 smithi148 ceph-mon[44992]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-29T08:01:50.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:01:50 smithi037 ceph-mon[35162]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-29T08:01:51.805 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:01:51 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-29T08:01:51.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:01:51 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T08:01:51.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:01:51 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-29T08:01:51.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:01:51 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T08:01:52.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:01:52 smithi148 ceph-mon[44992]: Deploying daemon osd.5 on smithi148 2024-03-29T08:01:52.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:01:52 smithi148 ceph-mon[44992]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-29T08:01:52.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:01:52 smithi037 ceph-mon[35162]: Deploying daemon osd.5 on smithi148 2024-03-29T08:01:52.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:01:52 smithi037 ceph-mon[35162]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-29T08:01:54.737 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:01:54 smithi148 ceph-mon[44992]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-29T08:01:54.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:01:54 smithi037 ceph-mon[35162]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-29T08:01:55.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:01:55 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:01:55.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:01:55 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:01:56.777 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:01:56 smithi148 ceph-mon[44992]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-29T08:01:56.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:01:56 smithi037 ceph-mon[35162]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-29T08:01:58.805 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:01:58 smithi148 ceph-mon[44992]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-29T08:01:58.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:01:58 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:01:58.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:01:58 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T08:01:58.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:01:58 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:01:58.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:01:58 smithi037 ceph-mon[35162]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-29T08:01:58.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:01:58 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:01:58.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:01:58 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T08:01:58.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:01:58 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:02:00.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:02:00 smithi148 ceph-mon[44992]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-29T08:02:00.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:02:00 smithi037 ceph-mon[35162]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-29T08:02:02.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:02:02 smithi148 ceph-mon[44992]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-29T08:02:02.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:02:02 smithi148 ceph-mon[44992]: from='osd.5 [v2:172.21.15.148:6816/267287812,v1:172.21.15.148:6817/267287812]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-29T08:02:02.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:02:02 smithi148 ceph-mon[44992]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-29T08:02:02.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:02:02 smithi037 ceph-mon[35162]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-29T08:02:02.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:02:02 smithi037 ceph-mon[35162]: from='osd.5 [v2:172.21.15.148:6816/267287812,v1:172.21.15.148:6817/267287812]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-29T08:02:02.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:02:02 smithi037 ceph-mon[35162]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-29T08:02:02.983 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T08:02:02.982+0000 7f3272000700 1 -- 172.21.15.148:0/826843860 <== mgr.14208 v2:172.21.15.37:6800/1802836043 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f327c1119b0 con 0x7f32600692b0 2024-03-29T08:02:02.983 INFO:teuthology.orchestra.run.smithi148.stdout:Created osd(s) 5 on host 'smithi148' 2024-03-29T08:02:02.986 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T08:02:02.986+0000 7f3281200700 1 -- 172.21.15.148:0/826843860 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f32600692b0 msgr2=0x7f326006b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:02:02.986 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T08:02:02.986+0000 7f3281200700 1 --2- 172.21.15.148:0/826843860 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f32600692b0 0x7f326006b760 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f3268009b80 tx=0x7f326800b040 comp rx=0 tx=0).stop 2024-03-29T08:02:02.986 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T08:02:02.986+0000 7f3281200700 1 -- 172.21.15.148:0/826843860 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f327c0709a0 msgr2=0x7f327c10fad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:02:02.986 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T08:02:02.986+0000 7f3281200700 1 --2- 172.21.15.148:0/826843860 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f327c0709a0 0x7f327c10fad0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f326c0137a0 tx=0x7f326c009e30 comp rx=0 tx=0).stop 2024-03-29T08:02:02.986 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T08:02:02.986+0000 7f3281200700 1 -- 172.21.15.148:0/826843860 shutdown_connections 2024-03-29T08:02:02.986 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T08:02:02.986+0000 7f3281200700 1 --2- 172.21.15.148:0/826843860 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f32600692b0 0x7f326006b760 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:02:02.986 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T08:02:02.986+0000 7f3281200700 1 --2- 172.21.15.148:0/826843860 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f327c0709a0 0x7f327c10fad0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:02:02.986 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T08:02:02.986+0000 7f3281200700 1 --2- 172.21.15.148:0/826843860 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f327c071f50 0x7f327c110010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:02:02.986 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T08:02:02.986+0000 7f3281200700 1 -- 172.21.15.148:0/826843860 >> 172.21.15.148:0/826843860 conn(0x7f327c0f8560 msgr2=0x7f327c0f9cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:02:02.986 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T08:02:02.986+0000 7f3281200700 1 -- 172.21.15.148:0/826843860 shutdown_connections 2024-03-29T08:02:02.986 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T08:02:02.986+0000 7f3281200700 1 -- 172.21.15.148:0/826843860 wait complete. 2024-03-29T08:02:03.687 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:02:03 smithi148 ceph-mon[44992]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-29T08:02:03.687 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:02:03 smithi148 ceph-mon[44992]: osdmap e38: 6 total, 5 up, 6 in 2024-03-29T08:02:03.687 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:02:03 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-29T08:02:03.687 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:02:03 smithi148 ceph-mon[44992]: from='osd.5 [v2:172.21.15.148:6816/267287812,v1:172.21.15.148:6817/267287812]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-03-29T08:02:03.687 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:02:03 smithi148 ceph-mon[44992]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-03-29T08:02:03.687 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:02:03 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:02:03.687 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:02:03 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:02:03.722 DEBUG:teuthology.orchestra.run.smithi148:osd.5> sudo journalctl -f -n 0 -u ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4@osd.5.service 2024-03-29T08:02:03.729 INFO:tasks.cephadm:Waiting for 6 OSDs to come up... 2024-03-29T08:02:03.729 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 99b492c2-eda1-11ee-b647-cb9ed24678a4 -- ceph osd stat -f json 2024-03-29T08:02:03.803 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:02:03 smithi037 ceph-mon[35162]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-29T08:02:03.803 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:02:03 smithi037 ceph-mon[35162]: osdmap e38: 6 total, 5 up, 6 in 2024-03-29T08:02:03.803 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:02:03 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-29T08:02:03.803 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:02:03 smithi037 ceph-mon[35162]: from='osd.5 [v2:172.21.15.148:6816/267287812,v1:172.21.15.148:6817/267287812]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-03-29T08:02:03.803 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:02:03 smithi037 ceph-mon[35162]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-03-29T08:02:03.803 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:02:03 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:02:03.803 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:02:03 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:02:03.803 INFO:journalctl@ceph.osd.5.smithi148.stdout:-- Logs begin at Fri 2024-03-29 07:42:56 UTC. -- 2024-03-29T08:02:04.175 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/mon.smithi037/config 2024-03-29T08:02:04.859 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:02:04 smithi148 ceph-mon[44992]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-29T08:02:04.859 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:02:04 smithi148 ceph-mon[44992]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]': finished 2024-03-29T08:02:04.859 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:02:04 smithi148 ceph-mon[44992]: osdmap e39: 6 total, 5 up, 6 in 2024-03-29T08:02:04.859 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:02:04 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-29T08:02:04.859 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:02:04 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-29T08:02:04.859 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:02:04 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:02:04.859 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:02:04 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:02:04.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:02:04 smithi037 ceph-mon[35162]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-29T08:02:04.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:02:04 smithi037 ceph-mon[35162]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]': finished 2024-03-29T08:02:04.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:02:04 smithi037 ceph-mon[35162]: osdmap e39: 6 total, 5 up, 6 in 2024-03-29T08:02:04.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:02:04 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-29T08:02:04.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:02:04 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-29T08:02:04.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:02:04 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:02:04.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:02:04 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:02:05.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:02:05 smithi037 ceph-mon[35162]: purged_snaps scrub starts 2024-03-29T08:02:05.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:02:05 smithi037 ceph-mon[35162]: purged_snaps scrub ok 2024-03-29T08:02:05.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:02:05 smithi037 ceph-mon[35162]: osd.5 [v2:172.21.15.148:6816/267287812,v1:172.21.15.148:6817/267287812] boot 2024-03-29T08:02:05.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:02:05 smithi037 ceph-mon[35162]: osdmap e40: 6 total, 6 up, 6 in 2024-03-29T08:02:05.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:02:05 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-29T08:02:05.948 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:02:05 smithi148 ceph-mon[44992]: purged_snaps scrub starts 2024-03-29T08:02:05.949 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:02:05 smithi148 ceph-mon[44992]: purged_snaps scrub ok 2024-03-29T08:02:05.949 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:02:05 smithi148 ceph-mon[44992]: osd.5 [v2:172.21.15.148:6816/267287812,v1:172.21.15.148:6817/267287812] boot 2024-03-29T08:02:05.949 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:02:05 smithi148 ceph-mon[44992]: osdmap e40: 6 total, 6 up, 6 in 2024-03-29T08:02:05.949 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:02:05 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-29T08:02:05.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:05.973+0000 7f9d0b000700 1 -- 172.21.15.37:0/235065571 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9d040fcee0 msgr2=0x7f9d040fd310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:02:05.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:05.973+0000 7f9d0b000700 1 --2- 172.21.15.37:0/235065571 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9d040fcee0 0x7f9d040fd310 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f9cf4004940 tx=0x7f9cf4009e30 comp rx=0 tx=0).stop 2024-03-29T08:02:05.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:05.973+0000 7f9d0b000700 1 -- 172.21.15.37:0/235065571 shutdown_connections 2024-03-29T08:02:05.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:05.973+0000 7f9d0b000700 1 --2- 172.21.15.37:0/235065571 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9d040fe0d0 0x7f9d040fe540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:02:05.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:05.973+0000 7f9d0b000700 1 --2- 172.21.15.37:0/235065571 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9d040fcee0 0x7f9d040fd310 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:02:05.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:05.973+0000 7f9d0b000700 1 -- 172.21.15.37:0/235065571 >> 172.21.15.37:0/235065571 conn(0x7f9d040f8560 msgr2=0x7f9d040fa970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:02:05.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:05.973+0000 7f9d0b000700 1 -- 172.21.15.37:0/235065571 shutdown_connections 2024-03-29T08:02:05.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:05.973+0000 7f9d0b000700 1 -- 172.21.15.37:0/235065571 wait complete. 2024-03-29T08:02:05.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:05.974+0000 7f9d0b000700 1 Processor -- start 2024-03-29T08:02:05.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:05.974+0000 7f9d0b000700 1 -- start start 2024-03-29T08:02:05.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:05.975+0000 7f9d0b000700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9d040fcee0 0x7f9d04071dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:02:05.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:05.975+0000 7f9d0b000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9d040fe0d0 0x7f9d04070310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:02:05.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:05.975+0000 7f9d0b000700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d040708a0 con 0x7f9d040fe0d0 2024-03-29T08:02:05.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:05.975+0000 7f9d0b000700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d040709e0 con 0x7f9d040fcee0 2024-03-29T08:02:05.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:05.975+0000 7f9d03400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9d040fe0d0 0x7f9d04070310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:02:05.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:05.975+0000 7f9d03e00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9d040fcee0 0x7f9d04071dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:02:05.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:05.975+0000 7f9d03400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9d040fe0d0 0x7f9d04070310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43352/0 (socket says 172.21.15.37:43352) 2024-03-29T08:02:05.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:05.975+0000 7f9d03400700 1 -- 172.21.15.37:0/1480787202 learned_addr learned my addr 172.21.15.37:0/1480787202 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:02:05.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:05.975+0000 7f9d03e00700 1 -- 172.21.15.37:0/1480787202 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9d040fe0d0 msgr2=0x7f9d04070310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:02:05.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:05.975+0000 7f9d03e00700 1 --2- 172.21.15.37:0/1480787202 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9d040fe0d0 0x7f9d04070310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:02:05.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:05.975+0000 7f9d03e00700 1 -- 172.21.15.37:0/1480787202 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cf4005040 con 0x7f9d040fcee0 2024-03-29T08:02:05.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:05.975+0000 7f9d03400700 1 --2- 172.21.15.37:0/1480787202 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9d040fe0d0 0x7f9d04070310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T08:02:05.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:05.976+0000 7f9d03e00700 1 --2- 172.21.15.37:0/1480787202 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9d040fcee0 0x7f9d04071dd0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f9cf4013740 tx=0x7f9cf4009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:02:05.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:05.976+0000 7f9cfbe00700 1 -- 172.21.15.37:0/1480787202 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9cf400e030 con 0x7f9d040fcee0 2024-03-29T08:02:05.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:05.976+0000 7f9d0b000700 1 -- 172.21.15.37:0/1480787202 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d04070c30 con 0x7f9d040fcee0 2024-03-29T08:02:05.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:05.976+0000 7f9cfbe00700 1 -- 172.21.15.37:0/1480787202 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f9cf40093b0 con 0x7f9d040fcee0 2024-03-29T08:02:05.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:05.976+0000 7f9cfbe00700 1 -- 172.21.15.37:0/1480787202 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9cf401ce10 con 0x7f9d040fcee0 2024-03-29T08:02:05.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:05.976+0000 7f9d0b000700 1 -- 172.21.15.37:0/1480787202 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d04115a00 con 0x7f9d040fcee0 2024-03-29T08:02:05.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:05.977+0000 7f9d0b000700 1 -- 172.21.15.37:0/1480787202 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d0405f5f0 con 0x7f9d040fcee0 2024-03-29T08:02:05.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:05.978+0000 7f9cfbe00700 1 -- 172.21.15.37:0/1480787202 <== mon.1 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f9cf401c600 con 0x7f9d040fcee0 2024-03-29T08:02:05.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:05.979+0000 7f9cfbe00700 1 --2- 172.21.15.37:0/1480787202 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f9ce80692a0 0x7f9ce806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:02:05.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:05.979+0000 7f9cfbe00700 1 -- 172.21.15.37:0/1480787202 <== mon.1 v2:172.21.15.148:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f9cf4088720 con 0x7f9d040fcee0 2024-03-29T08:02:05.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:05.979+0000 7f9d03400700 1 --2- 172.21.15.37:0/1480787202 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f9ce80692a0 0x7f9ce806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:02:05.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:05.980+0000 7f9d03400700 1 --2- 172.21.15.37:0/1480787202 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f9ce80692a0 0x7f9ce806b750 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f9cf00041d0 tx=0x7f9cf000b040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:02:05.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:05.984+0000 7f9cfbe00700 1 -- 172.21.15.37:0/1480787202 <== mon.1 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f9cf4054cf0 con 0x7f9d040fcee0 2024-03-29T08:02:06.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:06.190+0000 7f9d0b000700 1 -- 172.21.15.37:0/1480787202 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7f9d0404bed0 con 0x7f9d040fcee0 2024-03-29T08:02:06.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:06.190+0000 7f9cfbe00700 1 -- 172.21.15.37:0/1480787202 <== mon.1 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v41) v1 ==== 74+0+130 (secure 0 0 0) 0x7f9cf4004490 con 0x7f9d040fcee0 2024-03-29T08:02:06.190 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:02:06.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:06.193+0000 7f9d0b000700 1 -- 172.21.15.37:0/1480787202 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f9ce80692a0 msgr2=0x7f9ce806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:02:06.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:06.193+0000 7f9d0b000700 1 --2- 172.21.15.37:0/1480787202 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f9ce80692a0 0x7f9ce806b750 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f9cf00041d0 tx=0x7f9cf000b040 comp rx=0 tx=0).stop 2024-03-29T08:02:06.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:06.193+0000 7f9d0b000700 1 -- 172.21.15.37:0/1480787202 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9d040fcee0 msgr2=0x7f9d04071dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:02:06.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:06.193+0000 7f9d0b000700 1 --2- 172.21.15.37:0/1480787202 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9d040fcee0 0x7f9d04071dd0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f9cf4013740 tx=0x7f9cf4009e30 comp rx=0 tx=0).stop 2024-03-29T08:02:06.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:06.193+0000 7f9d0b000700 1 -- 172.21.15.37:0/1480787202 shutdown_connections 2024-03-29T08:02:06.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:06.193+0000 7f9d0b000700 1 --2- 172.21.15.37:0/1480787202 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f9ce80692a0 0x7f9ce806b750 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:02:06.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:06.193+0000 7f9d0b000700 1 --2- 172.21.15.37:0/1480787202 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9d040fcee0 0x7f9d04071dd0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:02:06.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:06.193+0000 7f9d0b000700 1 --2- 172.21.15.37:0/1480787202 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9d040fe0d0 0x7f9d04070310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:02:06.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:06.193+0000 7f9d0b000700 1 -- 172.21.15.37:0/1480787202 >> 172.21.15.37:0/1480787202 conn(0x7f9d040f8560 msgr2=0x7f9d040f9da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:02:06.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:06.193+0000 7f9d0b000700 1 -- 172.21.15.37:0/1480787202 shutdown_connections 2024-03-29T08:02:06.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:06.193+0000 7f9d0b000700 1 -- 172.21.15.37:0/1480787202 wait complete. 2024-03-29T08:02:06.827 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:02:06 smithi037 ceph-mon[35162]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-29T08:02:06.827 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:02:06 smithi037 ceph-mon[35162]: osdmap e41: 6 total, 6 up, 6 in 2024-03-29T08:02:06.827 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:02:06 smithi037 ceph-mon[35162]: from='client.? 172.21.15.37:0/1480787202' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-29T08:02:06.854 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":41,"num_osds":6,"num_up_osds":6,"osd_up_since":1711699324,"num_in_osds":6,"osd_in_since":1711699303,"num_remapped_pgs":0} 2024-03-29T08:02:06.854 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 99b492c2-eda1-11ee-b647-cb9ed24678a4 -- ceph osd dump --format=json 2024-03-29T08:02:07.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:02:06 smithi148 ceph-mon[44992]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-29T08:02:07.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:02:06 smithi148 ceph-mon[44992]: osdmap e41: 6 total, 6 up, 6 in 2024-03-29T08:02:07.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:02:06 smithi148 ceph-mon[44992]: from='client.? 172.21.15.37:0/1480787202' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-29T08:02:07.285 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/mon.smithi037/config 2024-03-29T08:02:08.828 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:02:08 smithi037 ceph-mon[35162]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:02:08.828 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:02:08 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:02:08.828 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:02:08 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:02:08.828 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:02:08 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-29T08:02:08.829 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:02:08 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-29T08:02:08.829 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:02:08 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-29T08:02:08.829 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:02:08 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:02:08.829 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:02:08 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T08:02:08.829 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:02:08 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T08:02:08.829 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:02:08 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:02:08.829 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:02:08 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T08:02:08.829 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:02:08 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T08:02:08.829 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:02:08 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T08:02:08.829 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:02:08 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:02:09.055 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:02:08 smithi148 ceph-mon[44992]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:02:09.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:02:08 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:02:09.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:02:08 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:02:09.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:02:08 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-29T08:02:09.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:02:08 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-29T08:02:09.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:02:08 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-29T08:02:09.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:02:08 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:02:09.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:02:08 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T08:02:09.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:02:08 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T08:02:09.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:02:08 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:02:09.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:02:08 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T08:02:09.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:02:08 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T08:02:09.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:02:08 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T08:02:09.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:02:08 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:02:09.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:09.126+0000 7fa0b6600700 1 -- 172.21.15.37:0/254785980 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa0b01037a0 msgr2=0x7fa0b00fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:02:09.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:09.126+0000 7fa0b6600700 1 --2- 172.21.15.37:0/254785980 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa0b01037a0 0x7fa0b00fce10 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fa09c004940 tx=0x7fa09c009e30 comp rx=0 tx=0).stop 2024-03-29T08:02:09.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:09.126+0000 7fa0b6600700 1 -- 172.21.15.37:0/254785980 shutdown_connections 2024-03-29T08:02:09.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:09.126+0000 7fa0b6600700 1 --2- 172.21.15.37:0/254785980 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa0b01037a0 0x7fa0b00fce10 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:02:09.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:09.126+0000 7fa0b6600700 1 --2- 172.21.15.37:0/254785980 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa0b0102e00 0x7fa0b01031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:02:09.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:09.126+0000 7fa0b6600700 1 -- 172.21.15.37:0/254785980 >> 172.21.15.37:0/254785980 conn(0x7fa0b00f89b0 msgr2=0x7fa0b00fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:02:09.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:09.126+0000 7fa0b6600700 1 -- 172.21.15.37:0/254785980 shutdown_connections 2024-03-29T08:02:09.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:09.126+0000 7fa0b6600700 1 -- 172.21.15.37:0/254785980 wait complete. 2024-03-29T08:02:09.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:09.127+0000 7fa0b6600700 1 Processor -- start 2024-03-29T08:02:09.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:09.127+0000 7fa0b6600700 1 -- start start 2024-03-29T08:02:09.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:09.127+0000 7fa0b6600700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa0b0102e00 0x7fa0b0115cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:02:09.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:09.127+0000 7fa0b6600700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa0b01037a0 0x7fa0b01161f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:02:09.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:09.127+0000 7fa0b6600700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0b01168d0 con 0x7fa0b0102e00 2024-03-29T08:02:09.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:09.127+0000 7fa0b6600700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0b010fc10 con 0x7fa0b01037a0 2024-03-29T08:02:09.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:09.127+0000 7fa0af400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa0b0102e00 0x7fa0b0115cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:02:09.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:09.127+0000 7fa0af400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa0b0102e00 0x7fa0b0115cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43370/0 (socket says 172.21.15.37:43370) 2024-03-29T08:02:09.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:09.127+0000 7fa0af400700 1 -- 172.21.15.37:0/4241377622 learned_addr learned my addr 172.21.15.37:0/4241377622 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:02:09.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:09.128+0000 7fa0aea00700 1 --2- 172.21.15.37:0/4241377622 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa0b01037a0 0x7fa0b01161f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:02:09.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:09.128+0000 7fa0af400700 1 -- 172.21.15.37:0/4241377622 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa0b01037a0 msgr2=0x7fa0b01161f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:02:09.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:09.128+0000 7fa0af400700 1 --2- 172.21.15.37:0/4241377622 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa0b01037a0 0x7fa0b01161f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:02:09.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:09.128+0000 7fa0af400700 1 -- 172.21.15.37:0/4241377622 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa09c005040 con 0x7fa0b0102e00 2024-03-29T08:02:09.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:09.128+0000 7fa0af400700 1 --2- 172.21.15.37:0/4241377622 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa0b0102e00 0x7fa0b0115cb0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fa0a000b790 tx=0x7fa0a0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:02:09.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:09.128+0000 7fa0a7400700 1 -- 172.21.15.37:0/4241377622 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa0a000f770 con 0x7fa0b0102e00 2024-03-29T08:02:09.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:09.128+0000 7fa0a7400700 1 -- 172.21.15.37:0/4241377622 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fa0a000fd80 con 0x7fa0b0102e00 2024-03-29T08:02:09.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:09.128+0000 7fa0b6600700 1 -- 172.21.15.37:0/4241377622 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0b010fe90 con 0x7fa0b0102e00 2024-03-29T08:02:09.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:09.128+0000 7fa0a7400700 1 -- 172.21.15.37:0/4241377622 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa0a000d310 con 0x7fa0b0102e00 2024-03-29T08:02:09.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:09.129+0000 7fa0b6600700 1 -- 172.21.15.37:0/4241377622 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0b0110450 con 0x7fa0b0102e00 2024-03-29T08:02:09.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:09.130+0000 7fa0b6600700 1 -- 172.21.15.37:0/4241377622 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0b00fe550 con 0x7fa0b0102e00 2024-03-29T08:02:09.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:09.131+0000 7fa0a7400700 1 -- 172.21.15.37:0/4241377622 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fa0a0010420 con 0x7fa0b0102e00 2024-03-29T08:02:09.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:09.131+0000 7fa0a7400700 1 --2- 172.21.15.37:0/4241377622 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7fa0940692b0 0x7fa09406b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:02:09.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:09.132+0000 7fa0a7400700 1 -- 172.21.15.37:0/4241377622 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fa0a0086e90 con 0x7fa0b0102e00 2024-03-29T08:02:09.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:09.132+0000 7fa0aea00700 1 --2- 172.21.15.37:0/4241377622 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7fa0940692b0 0x7fa09406b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:02:09.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:09.132+0000 7fa0aea00700 1 --2- 172.21.15.37:0/4241377622 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7fa0940692b0 0x7fa09406b760 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fa09c00b020 tx=0x7fa09c009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:02:09.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:09.136+0000 7fa0a7400700 1 -- 172.21.15.37:0/4241377622 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fa0a0056ef0 con 0x7fa0b0102e00 2024-03-29T08:02:09.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:09.345+0000 7fa0b6600700 1 -- 172.21.15.37:0/4241377622 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa0b005f5f0 con 0x7fa0b0102e00 2024-03-29T08:02:09.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:09.346+0000 7fa0a7400700 1 -- 172.21.15.37:0/4241377622 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) v1 ==== 74+0+11071 (secure 0 0 0) 0x7fa0a0056a80 con 0x7fa0b0102e00 2024-03-29T08:02:09.346 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:02:09.346 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":41,"fsid":"99b492c2-eda1-11ee-b647-cb9ed24678a4","created":"2024-03-29T07:55:23.673934+0000","modified":"2024-03-29T08:02:05.562456+0000","last_up_change":"2024-03-29T08:02:04.552402+0000","last_in_change":"2024-03-29T08:01:43.066485+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T08:00:32.022416+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"dba18fb6-b27d-4258-b769-dc89f4fef44d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":3851364162},{"type":"v1","addr":"172.21.15.37:6803","nonce":3851364162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":3851364162},{"type":"v1","addr":"172.21.15.37:6805","nonce":3851364162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":3851364162},{"type":"v1","addr":"172.21.15.37:6809","nonce":3851364162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":3851364162},{"type":"v1","addr":"172.21.15.37:6807","nonce":3851364162}]},"public_addr":"172.21.15.37:6803/3851364162","cluster_addr":"172.21.15.37:6805/3851364162","heartbeat_back_addr":"172.21.15.37:6809/3851364162","heartbeat_front_addr":"172.21.15.37:6807/3851364162","state":["exists","up"]},{"osd":1,"uuid":"2876dd3f-aefb-4298-8fe4-562245e1103e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":3949957441},{"type":"v1","addr":"172.21.15.37:6811","nonce":3949957441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":3949957441},{"type":"v1","addr":"172.21.15.37:6813","nonce":3949957441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":3949957441},{"type":"v1","addr":"172.21.15.37:6817","nonce":3949957441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":3949957441},{"type":"v1","addr":"172.21.15.37:6815","nonce":3949957441}]},"public_addr":"172.21.15.37:6811/3949957441","cluster_addr":"172.21.15.37:6813/3949957441","heartbeat_back_addr":"172.21.15.37:6817/3949957441","heartbeat_front_addr":"172.21.15.37:6815/3949957441","state":["exists","up"]},{"osd":2,"uuid":"bba0f95c-f4e3-404e-adf7-b90353a96af4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":4002901863},{"type":"v1","addr":"172.21.15.37:6819","nonce":4002901863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":4002901863},{"type":"v1","addr":"172.21.15.37:6821","nonce":4002901863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":4002901863},{"type":"v1","addr":"172.21.15.37:6825","nonce":4002901863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":4002901863},{"type":"v1","addr":"172.21.15.37:6823","nonce":4002901863}]},"public_addr":"172.21.15.37:6819/4002901863","cluster_addr":"172.21.15.37:6821/4002901863","heartbeat_back_addr":"172.21.15.37:6825/4002901863","heartbeat_front_addr":"172.21.15.37:6823/4002901863","state":["exists","up"]},{"osd":3,"uuid":"8da1c140-ada5-4879-b5d9-1ed81b0c9aef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":267019749},{"type":"v1","addr":"172.21.15.148:6801","nonce":267019749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":267019749},{"type":"v1","addr":"172.21.15.148:6803","nonce":267019749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":267019749},{"type":"v1","addr":"172.21.15.148:6807","nonce":267019749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":267019749},{"type":"v1","addr":"172.21.15.148:6805","nonce":267019749}]},"public_addr":"172.21.15.148:6801/267019749","cluster_addr":"172.21.15.148:6803/267019749","heartbeat_back_addr":"172.21.15.148:6807/267019749","heartbeat_front_addr":"172.21.15.148:6805/267019749","state":["exists","up"]},{"osd":4,"uuid":"9af70e66-bbca-48c7-bad1-7ac78270cf16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":4113691443},{"type":"v1","addr":"172.21.15.148:6809","nonce":4113691443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":4113691443},{"type":"v1","addr":"172.21.15.148:6811","nonce":4113691443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":4113691443},{"type":"v1","addr":"172.21.15.148:6815","nonce":4113691443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":4113691443},{"type":"v1","addr":"172.21.15.148:6813","nonce":4113691443}]},"public_addr":"172.21.15.148:6809/4113691443","cluster_addr":"172.21.15.148:6811/4113691443","heartbeat_back_addr":"172.21.15.148:6815/4113691443","heartbeat_front_addr":"172.21.15.148:6813/4113691443","state":["exists","up"]},{"osd":5,"uuid":"a66944e4-38bd-4403-b45f-7de01ddfb7a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":267287812},{"type":"v1","addr":"172.21.15.148:6817","nonce":267287812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":267287812},{"type":"v1","addr":"172.21.15.148:6819","nonce":267287812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":267287812},{"type":"v1","addr":"172.21.15.148:6823","nonce":267287812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":267287812},{"type":"v1","addr":"172.21.15.148:6821","nonce":267287812}]},"public_addr":"172.21.15.148:6817/267287812","cluster_addr":"172.21.15.148:6819/267287812","heartbeat_back_addr":"172.21.15.148:6823/267287812","heartbeat_front_addr":"172.21.15.148:6821/267287812","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T07:59:23.110900+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T07:59:55.540549+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:00:28.082290+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:00:59.496893+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:01:31.307445+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:02:03.191177+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:0/1421768676":"2024-03-30T07:57:55.402909+0000","172.21.15.37:6801/4245458578":"2024-03-30T07:57:55.402909+0000","172.21.15.37:6800/4245458578":"2024-03-30T07:57:55.402909+0000","172.21.15.37:0/368538117":"2024-03-30T07:57:55.402909+0000","172.21.15.37:0/2483635161":"2024-03-30T07:56:36.633439+0000","172.21.15.37:0/1131673968":"2024-03-30T07:56:36.633439+0000","172.21.15.37:0/3846762095":"2024-03-30T07:57:55.402909+0000","172.21.15.37:0/1909774742":"2024-03-30T07:55:59.989528+0000","172.21.15.37:0/959705723":"2024-03-30T07:56:36.633439+0000","172.21.15.37:0/3644857958":"2024-03-30T07:55:59.989528+0000","172.21.15.37:6800/2746946658":"2024-03-30T07:56:36.633439+0000","172.21.15.37:6801/2746946658":"2024-03-30T07:56:36.633439+0000","172.21.15.37:0/3116346321":"2024-03-30T07:55:59.989528+0000","172.21.15.37:6800/2762339582":"2024-03-30T07:55:59.989528+0000","172.21.15.37:6801/2762339582":"2024-03-30T07:55:59.989528+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T08:02:09.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:09.349+0000 7fa0b6600700 1 -- 172.21.15.37:0/4241377622 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7fa0940692b0 msgr2=0x7fa09406b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:02:09.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:09.349+0000 7fa0b6600700 1 --2- 172.21.15.37:0/4241377622 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7fa0940692b0 0x7fa09406b760 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fa09c00b020 tx=0x7fa09c009e30 comp rx=0 tx=0).stop 2024-03-29T08:02:09.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:09.349+0000 7fa0b6600700 1 -- 172.21.15.37:0/4241377622 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa0b0102e00 msgr2=0x7fa0b0115cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:02:09.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:09.349+0000 7fa0b6600700 1 --2- 172.21.15.37:0/4241377622 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa0b0102e00 0x7fa0b0115cb0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fa0a000b790 tx=0x7fa0a0009250 comp rx=0 tx=0).stop 2024-03-29T08:02:09.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:09.349+0000 7fa0b6600700 1 -- 172.21.15.37:0/4241377622 shutdown_connections 2024-03-29T08:02:09.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:09.349+0000 7fa0b6600700 1 --2- 172.21.15.37:0/4241377622 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7fa0940692b0 0x7fa09406b760 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:02:09.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:09.349+0000 7fa0b6600700 1 --2- 172.21.15.37:0/4241377622 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa0b0102e00 0x7fa0b0115cb0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:02:09.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:09.349+0000 7fa0b6600700 1 --2- 172.21.15.37:0/4241377622 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa0b01037a0 0x7fa0b01161f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:02:09.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:09.349+0000 7fa0b6600700 1 -- 172.21.15.37:0/4241377622 >> 172.21.15.37:0/4241377622 conn(0x7fa0b00f89b0 msgr2=0x7fa0b00f9910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:02:09.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:09.349+0000 7fa0b6600700 1 -- 172.21.15.37:0/4241377622 shutdown_connections 2024-03-29T08:02:09.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:09.349+0000 7fa0b6600700 1 -- 172.21.15.37:0/4241377622 wait complete. 2024-03-29T08:02:09.676 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:02:09 smithi037 ceph-mon[35162]: Detected new or changed devices on smithi148 2024-03-29T08:02:09.676 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:02:09 smithi037 ceph-mon[35162]: Adjusting osd_memory_target on smithi148 to 5372M 2024-03-29T08:02:09.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:02:09 smithi037 ceph-mon[35162]: from='client.? 172.21.15.37:0/4241377622' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-29T08:02:10.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:02:09 smithi148 ceph-mon[44992]: Detected new or changed devices on smithi148 2024-03-29T08:02:10.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:02:09 smithi148 ceph-mon[44992]: Adjusting osd_memory_target on smithi148 to 5372M 2024-03-29T08:02:10.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:02:09 smithi148 ceph-mon[44992]: from='client.? 172.21.15.37:0/4241377622' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-29T08:02:10.140 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-29T08:00:32.022416+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2024-03-29T08:02:10.140 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 99b492c2-eda1-11ee-b647-cb9ed24678a4 -- ceph osd pool get .mgr pg_num 2024-03-29T08:02:10.572 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/mon.smithi037/config 2024-03-29T08:02:10.841 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:02:10 smithi037 ceph-mon[35162]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:02:10.841 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:02:10 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:02:11.055 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:02:10 smithi148 ceph-mon[44992]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:02:11.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:02:10 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:02:12.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:12.475+0000 7f57eb400700 1 -- 172.21.15.37:0/793324936 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f57e4102e20 msgr2=0x7f57e41031f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:02:12.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:12.475+0000 7f57eb400700 1 --2- 172.21.15.37:0/793324936 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f57e4102e20 0x7f57e41031f0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f57d8004940 tx=0x7f57d8009e30 comp rx=0 tx=0).stop 2024-03-29T08:02:12.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:12.475+0000 7f57eb400700 1 -- 172.21.15.37:0/793324936 shutdown_connections 2024-03-29T08:02:12.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:12.475+0000 7f57eb400700 1 --2- 172.21.15.37:0/793324936 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f57e41037c0 0x7f57e40fce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:02:12.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:12.475+0000 7f57eb400700 1 --2- 172.21.15.37:0/793324936 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f57e4102e20 0x7f57e41031f0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:02:12.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:12.475+0000 7f57eb400700 1 -- 172.21.15.37:0/793324936 >> 172.21.15.37:0/793324936 conn(0x7f57e40f89d0 msgr2=0x7f57e40fade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:02:12.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:12.475+0000 7f57eb400700 1 -- 172.21.15.37:0/793324936 shutdown_connections 2024-03-29T08:02:12.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:12.475+0000 7f57eb400700 1 -- 172.21.15.37:0/793324936 wait complete. 2024-03-29T08:02:12.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:12.476+0000 7f57eb400700 1 Processor -- start 2024-03-29T08:02:12.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:12.476+0000 7f57eb400700 1 -- start start 2024-03-29T08:02:12.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:12.476+0000 7f57eb400700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f57e4102e20 0x7f57e40702c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:02:12.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:12.476+0000 7f57eb400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f57e41037c0 0x7f57e4070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:02:12.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:12.476+0000 7f57eb400700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57e4070d90 con 0x7f57e41037c0 2024-03-29T08:02:12.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:12.476+0000 7f57eb400700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57e4070ed0 con 0x7f57e4102e20 2024-03-29T08:02:12.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:12.477+0000 7f57e3e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f57e41037c0 0x7f57e4070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:02:12.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:12.477+0000 7f57e3e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f57e41037c0 0x7f57e4070800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42622/0 (socket says 172.21.15.37:42622) 2024-03-29T08:02:12.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:12.477+0000 7f57e3e00700 1 -- 172.21.15.37:0/137792497 learned_addr learned my addr 172.21.15.37:0/137792497 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:02:12.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:12.477+0000 7f57e8a00700 1 --2- 172.21.15.37:0/137792497 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f57e4102e20 0x7f57e40702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:02:12.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:12.477+0000 7f57e3e00700 1 -- 172.21.15.37:0/137792497 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f57e4102e20 msgr2=0x7f57e40702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:02:12.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:12.477+0000 7f57e3e00700 1 --2- 172.21.15.37:0/137792497 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f57e4102e20 0x7f57e40702c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:02:12.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:12.477+0000 7f57e3e00700 1 -- 172.21.15.37:0/137792497 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57d8005040 con 0x7f57e41037c0 2024-03-29T08:02:12.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:12.477+0000 7f57e3e00700 1 --2- 172.21.15.37:0/137792497 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f57e41037c0 0x7f57e4070800 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f57d0002a50 tx=0x7f57d0004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:02:12.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:12.478+0000 7f57e0c00700 1 -- 172.21.15.37:0/137792497 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f57d0007cc0 con 0x7f57e41037c0 2024-03-29T08:02:12.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:12.478+0000 7f57e0c00700 1 -- 172.21.15.37:0/137792497 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f57d000f040 con 0x7f57e41037c0 2024-03-29T08:02:12.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:12.478+0000 7f57eb400700 1 -- 172.21.15.37:0/137792497 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57e4119e00 con 0x7f57e41037c0 2024-03-29T08:02:12.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:12.478+0000 7f57e0c00700 1 -- 172.21.15.37:0/137792497 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f57d000d920 con 0x7f57e41037c0 2024-03-29T08:02:12.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:12.478+0000 7f57eb400700 1 -- 172.21.15.37:0/137792497 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57e411a280 con 0x7f57e41037c0 2024-03-29T08:02:12.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:12.479+0000 7f57eb400700 1 -- 172.21.15.37:0/137792497 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57e40fe570 con 0x7f57e41037c0 2024-03-29T08:02:12.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:12.480+0000 7f57e0c00700 1 -- 172.21.15.37:0/137792497 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f57d0009680 con 0x7f57e41037c0 2024-03-29T08:02:12.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:12.481+0000 7f57e0c00700 1 --2- 172.21.15.37:0/137792497 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f57cc0692b0 0x7f57cc06b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:02:12.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:12.481+0000 7f57e0c00700 1 -- 172.21.15.37:0/137792497 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f57d0086b40 con 0x7f57e41037c0 2024-03-29T08:02:12.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:12.481+0000 7f57e8a00700 1 --2- 172.21.15.37:0/137792497 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f57cc0692b0 0x7f57cc06b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:02:12.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:12.482+0000 7f57e8a00700 1 --2- 172.21.15.37:0/137792497 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f57cc0692b0 0x7f57cc06b760 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f57d800b840 tx=0x7f57d8009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:02:12.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:12.485+0000 7f57e0c00700 1 -- 172.21.15.37:0/137792497 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f57d0053130 con 0x7f57e41037c0 2024-03-29T08:02:12.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:12.692+0000 7f57eb400700 1 -- 172.21.15.37:0/137792497 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f57e404bed0 con 0x7f57e41037c0 2024-03-29T08:02:12.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:12.692+0000 7f57e0c00700 1 -- 172.21.15.37:0/137792497 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v41) v1 ==== 93+0+10 (secure 0 0 0) 0x7f57d0056750 con 0x7f57e41037c0 2024-03-29T08:02:12.692 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 1 2024-03-29T08:02:12.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:12.695+0000 7f57eb400700 1 -- 172.21.15.37:0/137792497 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f57cc0692b0 msgr2=0x7f57cc06b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:02:12.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:12.695+0000 7f57eb400700 1 --2- 172.21.15.37:0/137792497 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f57cc0692b0 0x7f57cc06b760 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f57d800b840 tx=0x7f57d8009e30 comp rx=0 tx=0).stop 2024-03-29T08:02:12.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:12.695+0000 7f57eb400700 1 -- 172.21.15.37:0/137792497 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f57e41037c0 msgr2=0x7f57e4070800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:02:12.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:12.695+0000 7f57eb400700 1 --2- 172.21.15.37:0/137792497 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f57e41037c0 0x7f57e4070800 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f57d0002a50 tx=0x7f57d0004600 comp rx=0 tx=0).stop 2024-03-29T08:02:12.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:12.695+0000 7f57eb400700 1 -- 172.21.15.37:0/137792497 shutdown_connections 2024-03-29T08:02:12.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:12.695+0000 7f57eb400700 1 --2- 172.21.15.37:0/137792497 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f57cc0692b0 0x7f57cc06b760 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:02:12.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:12.695+0000 7f57eb400700 1 --2- 172.21.15.37:0/137792497 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f57e4102e20 0x7f57e40702c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:02:12.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:12.695+0000 7f57eb400700 1 --2- 172.21.15.37:0/137792497 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f57e41037c0 0x7f57e4070800 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:02:12.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:12.695+0000 7f57eb400700 1 -- 172.21.15.37:0/137792497 >> 172.21.15.37:0/137792497 conn(0x7f57e40f89d0 msgr2=0x7f57e40f9ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:02:12.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:12.695+0000 7f57eb400700 1 -- 172.21.15.37:0/137792497 shutdown_connections 2024-03-29T08:02:12.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:12.695+0000 7f57eb400700 1 -- 172.21.15.37:0/137792497 wait complete. 2024-03-29T08:02:12.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:02:12 smithi037 ceph-mon[35162]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:02:13.055 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:02:12 smithi148 ceph-mon[44992]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:02:13.234 INFO:tasks.cephadm:Setting up client nodes... 2024-03-29T08:02:13.234 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 99b492c2-eda1-11ee-b647-cb9ed24678a4 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-03-29T08:02:13.664 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/mon.smithi037/config 2024-03-29T08:02:13.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:02:13 smithi037 ceph-mon[35162]: from='client.? 172.21.15.37:0/137792497' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-29T08:02:14.055 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:02:13 smithi148 ceph-mon[44992]: from='client.? 172.21.15.37:0/137792497' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-29T08:02:14.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:02:14 smithi037 ceph-mon[35162]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:02:15.055 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:02:14 smithi148 ceph-mon[44992]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:02:15.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:15.559+0000 7fb972800700 1 -- 172.21.15.37:0/3239764487 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb96c0fce70 msgr2=0x7fb96c0fd280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:02:15.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:15.559+0000 7fb972800700 1 --2- 172.21.15.37:0/3239764487 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb96c0fce70 0x7fb96c0fd280 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fb95c004940 tx=0x7fb95c009e30 comp rx=0 tx=0).stop 2024-03-29T08:02:15.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:15.559+0000 7fb972800700 1 -- 172.21.15.37:0/3239764487 shutdown_connections 2024-03-29T08:02:15.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:15.559+0000 7fb972800700 1 --2- 172.21.15.37:0/3239764487 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb96c0fe070 0x7fb96c0fe4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:02:15.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:15.559+0000 7fb972800700 1 --2- 172.21.15.37:0/3239764487 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb96c0fce70 0x7fb96c0fd280 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:02:15.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:15.559+0000 7fb972800700 1 -- 172.21.15.37:0/3239764487 >> 172.21.15.37:0/3239764487 conn(0x7fb96c0f8540 msgr2=0x7fb96c0fa9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:02:15.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:15.559+0000 7fb972800700 1 -- 172.21.15.37:0/3239764487 shutdown_connections 2024-03-29T08:02:15.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:15.559+0000 7fb972800700 1 -- 172.21.15.37:0/3239764487 wait complete. 2024-03-29T08:02:15.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:15.560+0000 7fb972800700 1 Processor -- start 2024-03-29T08:02:15.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:15.560+0000 7fb972800700 1 -- start start 2024-03-29T08:02:15.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:15.560+0000 7fb972800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb96c0fce70 0x7fb96c10d850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:02:15.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:15.560+0000 7fb972800700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb96c0fe070 0x7fb96c10dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:02:15.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:15.560+0000 7fb972800700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb96c10e390 con 0x7fb96c0fce70 2024-03-29T08:02:15.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:15.560+0000 7fb972800700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb96c10e500 con 0x7fb96c0fe070 2024-03-29T08:02:15.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:15.561+0000 7fb96b400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb96c0fce70 0x7fb96c10d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:02:15.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:15.561+0000 7fb96b400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb96c0fce70 0x7fb96c10d850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42642/0 (socket says 172.21.15.37:42642) 2024-03-29T08:02:15.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:15.561+0000 7fb96b400700 1 -- 172.21.15.37:0/3654118891 learned_addr learned my addr 172.21.15.37:0/3654118891 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:02:15.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:15.561+0000 7fb96b400700 1 -- 172.21.15.37:0/3654118891 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb96c0fe070 msgr2=0x7fb96c10dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:02:15.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:15.561+0000 7fb96b400700 1 --2- 172.21.15.37:0/3654118891 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb96c0fe070 0x7fb96c10dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:02:15.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:15.561+0000 7fb96b400700 1 -- 172.21.15.37:0/3654118891 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb95c005040 con 0x7fb96c0fce70 2024-03-29T08:02:15.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:15.561+0000 7fb96b400700 1 --2- 172.21.15.37:0/3654118891 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb96c0fce70 0x7fb96c10d850 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fb95c00b900 tx=0x7fb95c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:02:15.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:15.561+0000 7fb963400700 1 -- 172.21.15.37:0/3654118891 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb95c00e040 con 0x7fb96c0fce70 2024-03-29T08:02:15.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:15.562+0000 7fb963400700 1 -- 172.21.15.37:0/3654118891 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fb95c004290 con 0x7fb96c0fce70 2024-03-29T08:02:15.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:15.562+0000 7fb972800700 1 -- 172.21.15.37:0/3654118891 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb96c112f00 con 0x7fb96c0fce70 2024-03-29T08:02:15.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:15.562+0000 7fb963400700 1 -- 172.21.15.37:0/3654118891 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb95c01c3f0 con 0x7fb96c0fce70 2024-03-29T08:02:15.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:15.562+0000 7fb972800700 1 -- 172.21.15.37:0/3654118891 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb96c113460 con 0x7fb96c0fce70 2024-03-29T08:02:15.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:15.563+0000 7fb972800700 1 -- 172.21.15.37:0/3654118891 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb96c04c710 con 0x7fb96c0fce70 2024-03-29T08:02:15.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:15.564+0000 7fb963400700 1 -- 172.21.15.37:0/3654118891 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fb95c013420 con 0x7fb96c0fce70 2024-03-29T08:02:15.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:15.564+0000 7fb963400700 1 --2- 172.21.15.37:0/3654118891 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7fb9500692b0 0x7fb95006b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:02:15.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:15.564+0000 7fb963400700 1 -- 172.21.15.37:0/3654118891 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fb95c088130 con 0x7fb96c0fce70 2024-03-29T08:02:15.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:15.565+0000 7fb96aa00700 1 --2- 172.21.15.37:0/3654118891 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7fb9500692b0 0x7fb95006b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:02:15.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:15.565+0000 7fb96aa00700 1 --2- 172.21.15.37:0/3654118891 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7fb9500692b0 0x7fb95006b760 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fb958004d00 tx=0x7fb958009ce0 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:02:15.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:15.569+0000 7fb963400700 1 -- 172.21.15.37:0/3654118891 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fb95c058210 con 0x7fb96c0fce70 2024-03-29T08:02:15.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:15.862+0000 7fb972800700 1 -- 172.21.15.37:0/3654118891 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7fb96c1057c0 con 0x7fb96c0fce70 2024-03-29T08:02:15.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:15.868+0000 7fb963400700 1 -- 172.21.15.37:0/3654118891 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v14) v1 ==== 170+0+59 (secure 0 0 0) 0x7fb95c057da0 con 0x7fb96c0fce70 2024-03-29T08:02:15.868 INFO:teuthology.orchestra.run.smithi037.stdout:[client.0] 2024-03-29T08:02:15.868 INFO:teuthology.orchestra.run.smithi037.stdout: key = AQCHdQZmerd+MxAAKGAzn8USxOd1r2SsR6/FKA== 2024-03-29T08:02:15.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:15.870+0000 7fb972800700 1 -- 172.21.15.37:0/3654118891 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7fb9500692b0 msgr2=0x7fb95006b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:02:15.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:15.870+0000 7fb972800700 1 --2- 172.21.15.37:0/3654118891 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7fb9500692b0 0x7fb95006b760 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fb958004d00 tx=0x7fb958009ce0 comp rx=0 tx=0).stop 2024-03-29T08:02:15.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:15.870+0000 7fb972800700 1 -- 172.21.15.37:0/3654118891 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb96c0fce70 msgr2=0x7fb96c10d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:02:15.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:15.870+0000 7fb972800700 1 --2- 172.21.15.37:0/3654118891 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb96c0fce70 0x7fb96c10d850 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fb95c00b900 tx=0x7fb95c009e30 comp rx=0 tx=0).stop 2024-03-29T08:02:15.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:15.870+0000 7fb972800700 1 -- 172.21.15.37:0/3654118891 shutdown_connections 2024-03-29T08:02:15.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:15.870+0000 7fb972800700 1 --2- 172.21.15.37:0/3654118891 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7fb9500692b0 0x7fb95006b760 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:02:15.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:15.870+0000 7fb972800700 1 --2- 172.21.15.37:0/3654118891 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb96c0fce70 0x7fb96c10d850 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:02:15.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:15.870+0000 7fb972800700 1 --2- 172.21.15.37:0/3654118891 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb96c0fe070 0x7fb96c10dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:02:15.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:15.870+0000 7fb972800700 1 -- 172.21.15.37:0/3654118891 >> 172.21.15.37:0/3654118891 conn(0x7fb96c0f8540 msgr2=0x7fb96c0f9ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:02:15.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:15.871+0000 7fb972800700 1 -- 172.21.15.37:0/3654118891 shutdown_connections 2024-03-29T08:02:15.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:15.871+0000 7fb972800700 1 -- 172.21.15.37:0/3654118891 wait complete. 2024-03-29T08:02:16.579 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-03-29T08:02:16.579 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-03-29T08:02:16.579 DEBUG:teuthology.orchestra.run.smithi037:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-03-29T08:02:16.675 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 99b492c2-eda1-11ee-b647-cb9ed24678a4 -- ceph auth get-or-create client.1 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-03-29T08:02:16.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:02:16 smithi037 ceph-mon[35162]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:02:16.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:02:16 smithi037 ceph-mon[35162]: from='client.? 172.21.15.37:0/3654118891' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-29T08:02:16.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:02:16 smithi037 ceph-mon[35162]: from='client.? 172.21.15.37:0/3654118891' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-29T08:02:17.055 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:02:16 smithi148 ceph-mon[44992]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:02:17.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:02:16 smithi148 ceph-mon[44992]: from='client.? 172.21.15.37:0/3654118891' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-29T08:02:17.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:02:16 smithi148 ceph-mon[44992]: from='client.? 172.21.15.37:0/3654118891' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-29T08:02:17.101 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/mon.smithi148/config 2024-03-29T08:02:18.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:02:18 smithi148 ceph-mon[44992]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:02:18.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:02:18 smithi037 ceph-mon[35162]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:02:19.227 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T08:02:19.228+0000 7fcfcf400700 1 -- 172.21.15.148:0/1930473139 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcfc80fce70 msgr2=0x7fcfc80fd280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:02:19.227 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T08:02:19.228+0000 7fcfcf400700 1 --2- 172.21.15.148:0/1930473139 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcfc80fce70 0x7fcfc80fd280 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fcfbc004940 tx=0x7fcfbc009e30 comp rx=0 tx=0).stop 2024-03-29T08:02:19.228 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T08:02:19.228+0000 7fcfcf400700 1 -- 172.21.15.148:0/1930473139 shutdown_connections 2024-03-29T08:02:19.228 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T08:02:19.228+0000 7fcfcf400700 1 --2- 172.21.15.148:0/1930473139 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcfc80fe070 0x7fcfc80fe4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:02:19.228 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T08:02:19.228+0000 7fcfcf400700 1 --2- 172.21.15.148:0/1930473139 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcfc80fce70 0x7fcfc80fd280 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:02:19.228 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T08:02:19.228+0000 7fcfcf400700 1 -- 172.21.15.148:0/1930473139 >> 172.21.15.148:0/1930473139 conn(0x7fcfc80f8540 msgr2=0x7fcfc80fa9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:02:19.228 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T08:02:19.228+0000 7fcfcf400700 1 -- 172.21.15.148:0/1930473139 shutdown_connections 2024-03-29T08:02:19.228 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T08:02:19.228+0000 7fcfcf400700 1 -- 172.21.15.148:0/1930473139 wait complete. 2024-03-29T08:02:19.229 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T08:02:19.229+0000 7fcfcf400700 1 Processor -- start 2024-03-29T08:02:19.229 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T08:02:19.229+0000 7fcfcf400700 1 -- start start 2024-03-29T08:02:19.229 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T08:02:19.230+0000 7fcfcf400700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcfc80fce70 0x7fcfc810f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:02:19.229 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T08:02:19.230+0000 7fcfcf400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcfc80fe070 0x7fcfc810ff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:02:19.230 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T08:02:19.230+0000 7fcfcf400700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfc8110500 con 0x7fcfc80fe070 2024-03-29T08:02:19.230 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T08:02:19.230+0000 7fcfcf400700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfc8110670 con 0x7fcfc80fce70 2024-03-29T08:02:19.230 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T08:02:19.230+0000 7fcfcca00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcfc80fce70 0x7fcfc810f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:02:19.230 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T08:02:19.230+0000 7fcfcca00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcfc80fce70 0x7fcfc810f9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:59018/0 (socket says 172.21.15.148:59018) 2024-03-29T08:02:19.230 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T08:02:19.230+0000 7fcfcca00700 1 -- 172.21.15.148:0/3541888235 learned_addr learned my addr 172.21.15.148:0/3541888235 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-03-29T08:02:19.230 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T08:02:19.230+0000 7fcfcca00700 1 -- 172.21.15.148:0/3541888235 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcfc80fe070 msgr2=0x7fcfc810ff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:02:19.230 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T08:02:19.230+0000 7fcfcca00700 1 --2- 172.21.15.148:0/3541888235 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcfc80fe070 0x7fcfc810ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:02:19.230 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T08:02:19.230+0000 7fcfcca00700 1 -- 172.21.15.148:0/3541888235 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfbc005040 con 0x7fcfc80fce70 2024-03-29T08:02:19.230 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T08:02:19.230+0000 7fcfcca00700 1 --2- 172.21.15.148:0/3541888235 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcfc80fce70 0x7fcfc810f9c0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fcfbc00b0d0 tx=0x7fcfbc009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:02:19.230 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T08:02:19.231+0000 7fcfc4c00700 1 -- 172.21.15.148:0/3541888235 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcfbc00e070 con 0x7fcfc80fce70 2024-03-29T08:02:19.230 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T08:02:19.231+0000 7fcfc4c00700 1 -- 172.21.15.148:0/3541888235 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fcfbc01d460 con 0x7fcfc80fce70 2024-03-29T08:02:19.230 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T08:02:19.231+0000 7fcfc4c00700 1 -- 172.21.15.148:0/3541888235 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcfbc00b650 con 0x7fcfc80fce70 2024-03-29T08:02:19.230 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T08:02:19.231+0000 7fcfcf400700 1 -- 172.21.15.148:0/3541888235 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfc8115000 con 0x7fcfc80fce70 2024-03-29T08:02:19.230 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T08:02:19.231+0000 7fcfcf400700 1 -- 172.21.15.148:0/3541888235 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfc8115570 con 0x7fcfc80fce70 2024-03-29T08:02:19.231 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T08:02:19.232+0000 7fcfcf400700 1 -- 172.21.15.148:0/3541888235 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfc8101a00 con 0x7fcfc80fce70 2024-03-29T08:02:19.231 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T08:02:19.233+0000 7fcfc4c00700 1 -- 172.21.15.148:0/3541888235 <== mon.1 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fcfbc01da20 con 0x7fcfc80fce70 2024-03-29T08:02:19.232 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T08:02:19.234+0000 7fcfc4c00700 1 --2- 172.21.15.148:0/3541888235 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7fcfb00692b0 0x7fcfb006b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:02:19.232 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T08:02:19.234+0000 7fcfc4c00700 1 -- 172.21.15.148:0/3541888235 <== mon.1 v2:172.21.15.148:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fcfbc087e70 con 0x7fcfc80fce70 2024-03-29T08:02:19.233 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T08:02:19.234+0000 7fcfc7e00700 1 --2- 172.21.15.148:0/3541888235 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7fcfb00692b0 0x7fcfb006b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:02:19.233 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T08:02:19.235+0000 7fcfc7e00700 1 --2- 172.21.15.148:0/3541888235 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7fcfb00692b0 0x7fcfb006b760 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fcfb4009fd0 tx=0x7fcfb400b040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:02:19.237 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T08:02:19.238+0000 7fcfc4c00700 1 -- 172.21.15.148:0/3541888235 <== mon.1 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fcfbc058000 con 0x7fcfc80fce70 2024-03-29T08:02:19.503 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T08:02:19.505+0000 7fcfcf400700 1 -- 172.21.15.148:0/3541888235 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7fcfc804c710 con 0x7fcfc80fce70 2024-03-29T08:02:19.509 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T08:02:19.511+0000 7fcfc4c00700 1 -- 172.21.15.148:0/3541888235 <== mon.1 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v15) v1 ==== 170+0+59 (secure 0 0 0) 0x7fcfbc022020 con 0x7fcfc80fce70 2024-03-29T08:02:19.510 INFO:teuthology.orchestra.run.smithi148.stdout:[client.1] 2024-03-29T08:02:19.510 INFO:teuthology.orchestra.run.smithi148.stdout: key = AQCLdQZmIDctHhAATBWfapPomw8LNAL/qdpfjA== 2024-03-29T08:02:19.512 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T08:02:19.513+0000 7fcfcf400700 1 -- 172.21.15.148:0/3541888235 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7fcfb00692b0 msgr2=0x7fcfb006b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:02:19.512 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T08:02:19.513+0000 7fcfcf400700 1 --2- 172.21.15.148:0/3541888235 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7fcfb00692b0 0x7fcfb006b760 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fcfb4009fd0 tx=0x7fcfb400b040 comp rx=0 tx=0).stop 2024-03-29T08:02:19.512 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T08:02:19.513+0000 7fcfcf400700 1 -- 172.21.15.148:0/3541888235 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcfc80fce70 msgr2=0x7fcfc810f9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:02:19.512 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T08:02:19.513+0000 7fcfcf400700 1 --2- 172.21.15.148:0/3541888235 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcfc80fce70 0x7fcfc810f9c0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fcfbc00b0d0 tx=0x7fcfbc009e30 comp rx=0 tx=0).stop 2024-03-29T08:02:19.512 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T08:02:19.513+0000 7fcfcf400700 1 -- 172.21.15.148:0/3541888235 shutdown_connections 2024-03-29T08:02:19.512 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T08:02:19.513+0000 7fcfcf400700 1 --2- 172.21.15.148:0/3541888235 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7fcfb00692b0 0x7fcfb006b760 secure :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fcfb4009fd0 tx=0x7fcfb400b040 comp rx=0 tx=0).stop 2024-03-29T08:02:19.512 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T08:02:19.513+0000 7fcfcf400700 1 --2- 172.21.15.148:0/3541888235 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcfc80fce70 0x7fcfc810f9c0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:02:19.512 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T08:02:19.513+0000 7fcfcf400700 1 --2- 172.21.15.148:0/3541888235 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcfc80fe070 0x7fcfc810ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:02:19.512 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T08:02:19.513+0000 7fcfcf400700 1 -- 172.21.15.148:0/3541888235 >> 172.21.15.148:0/3541888235 conn(0x7fcfc80f8540 msgr2=0x7fcfc80f9bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:02:19.512 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T08:02:19.513+0000 7fcfcf400700 1 -- 172.21.15.148:0/3541888235 shutdown_connections 2024-03-29T08:02:19.513 INFO:teuthology.orchestra.run.smithi148.stderr:2024-03-29T08:02:19.513+0000 7fcfcf400700 1 -- 172.21.15.148:0/3541888235 wait complete. 2024-03-29T08:02:19.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:02:19 smithi148 ceph-mon[44992]: from='client.? 172.21.15.148:0/3541888235' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-29T08:02:19.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:02:19 smithi148 ceph-mon[44992]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-29T08:02:19.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:02:19 smithi148 ceph-mon[44992]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-29T08:02:19.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:02:19 smithi037 ceph-mon[35162]: from='client.? 172.21.15.148:0/3541888235' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-29T08:02:19.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:02:19 smithi037 ceph-mon[35162]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-29T08:02:19.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:02:19 smithi037 ceph-mon[35162]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-29T08:02:20.210 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-03-29T08:02:20.211 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd of=/etc/ceph/ceph.client.1.keyring 2024-03-29T08:02:20.211 DEBUG:teuthology.orchestra.run.smithi148:> sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-03-29T08:02:20.306 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-03-29T08:02:20.306 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-03-29T08:02:20.306 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 99b492c2-eda1-11ee-b647-cb9ed24678a4 -- ceph mgr dump --format=json 2024-03-29T08:02:20.676 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:02:20 smithi037 ceph-mon[35162]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:02:20.746 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/mon.smithi037/config 2024-03-29T08:02:21.055 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:02:20 smithi148 ceph-mon[44992]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:02:22.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:22.462+0000 7f07c3400700 1 -- 172.21.15.37:0/3010625375 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f07bc0fe830 msgr2=0x7f07bc106f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:02:22.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:22.462+0000 7f07c3400700 1 --2- 172.21.15.37:0/3010625375 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f07bc0fe830 0x7f07bc106f00 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f07a8004940 tx=0x7f07a8009e30 comp rx=0 tx=0).stop 2024-03-29T08:02:22.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:22.462+0000 7f07c3400700 1 -- 172.21.15.37:0/3010625375 shutdown_connections 2024-03-29T08:02:22.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:22.462+0000 7f07c3400700 1 --2- 172.21.15.37:0/3010625375 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f07bc0fe830 0x7f07bc106f00 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:02:22.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:22.462+0000 7f07c3400700 1 --2- 172.21.15.37:0/3010625375 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f07bc0fde90 0x7f07bc0fe260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:02:22.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:22.462+0000 7f07c3400700 1 -- 172.21.15.37:0/3010625375 >> 172.21.15.37:0/3010625375 conn(0x7f07bc072bc0 msgr2=0x7f07bc072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:02:22.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:22.462+0000 7f07c3400700 1 -- 172.21.15.37:0/3010625375 shutdown_connections 2024-03-29T08:02:22.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:22.462+0000 7f07c3400700 1 -- 172.21.15.37:0/3010625375 wait complete. 2024-03-29T08:02:22.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:22.463+0000 7f07c3400700 1 Processor -- start 2024-03-29T08:02:22.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:22.463+0000 7f07c3400700 1 -- start start 2024-03-29T08:02:22.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:22.463+0000 7f07c3400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f07bc0fde90 0x7f07bc115d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:02:22.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:22.463+0000 7f07c3400700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f07bc0fe830 0x7f07bc116270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:02:22.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:22.463+0000 7f07c3400700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07bc1168c0 con 0x7f07bc0fde90 2024-03-29T08:02:22.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:22.463+0000 7f07c3400700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07bc10fc90 con 0x7f07bc0fe830 2024-03-29T08:02:22.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:22.463+0000 7f07c0a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f07bc0fde90 0x7f07bc115d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:02:22.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:22.463+0000 7f07c0a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f07bc0fde90 0x7f07bc115d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58572/0 (socket says 172.21.15.37:58572) 2024-03-29T08:02:22.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:22.463+0000 7f07c0a00700 1 -- 172.21.15.37:0/469443727 learned_addr learned my addr 172.21.15.37:0/469443727 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:02:22.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:22.464+0000 7f07c0a00700 1 -- 172.21.15.37:0/469443727 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f07bc0fe830 msgr2=0x7f07bc116270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:02:22.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:22.464+0000 7f07bbe00700 1 --2- 172.21.15.37:0/469443727 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f07bc0fe830 0x7f07bc116270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:02:22.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:22.464+0000 7f07c0a00700 1 --2- 172.21.15.37:0/469443727 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f07bc0fe830 0x7f07bc116270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:02:22.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:22.464+0000 7f07c0a00700 1 -- 172.21.15.37:0/469443727 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07a8005040 con 0x7f07bc0fde90 2024-03-29T08:02:22.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:22.464+0000 7f07bbe00700 1 --2- 172.21.15.37:0/469443727 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f07bc0fe830 0x7f07bc116270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:02:22.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:22.464+0000 7f07c0a00700 1 --2- 172.21.15.37:0/469443727 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f07bc0fde90 0x7f07bc115d30 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f07b0002a50 tx=0x7f07b00046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:02:22.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:22.464+0000 7f07b8c00700 1 -- 172.21.15.37:0/469443727 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f07b0007cc0 con 0x7f07bc0fde90 2024-03-29T08:02:22.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:22.464+0000 7f07b8c00700 1 -- 172.21.15.37:0/469443727 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f07b000f040 con 0x7f07bc0fde90 2024-03-29T08:02:22.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:22.464+0000 7f07c3400700 1 -- 172.21.15.37:0/469443727 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07bc10ff70 con 0x7f07bc0fde90 2024-03-29T08:02:22.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:22.464+0000 7f07b8c00700 1 -- 172.21.15.37:0/469443727 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f07b0009540 con 0x7f07bc0fde90 2024-03-29T08:02:22.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:22.465+0000 7f07c3400700 1 -- 172.21.15.37:0/469443727 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07bc1104b0 con 0x7f07bc0fde90 2024-03-29T08:02:22.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:22.466+0000 7f07c3400700 1 -- 172.21.15.37:0/469443727 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07bc106a20 con 0x7f07bc0fde90 2024-03-29T08:02:22.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:22.466+0000 7f07b8c00700 1 -- 172.21.15.37:0/469443727 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f07b00096a0 con 0x7f07bc0fde90 2024-03-29T08:02:22.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:22.468+0000 7f07b8c00700 1 --2- 172.21.15.37:0/469443727 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f07a40692b0 0x7f07a406b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:02:22.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:22.468+0000 7f07b8c00700 1 -- 172.21.15.37:0/469443727 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f07b0086c30 con 0x7f07bc0fde90 2024-03-29T08:02:22.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:22.468+0000 7f07bbe00700 1 --2- 172.21.15.37:0/469443727 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f07a40692b0 0x7f07a406b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:02:22.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:22.469+0000 7f07bbe00700 1 --2- 172.21.15.37:0/469443727 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f07a40692b0 0x7f07a406b760 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f07a800b3b0 tx=0x7f07a8009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:02:22.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:22.472+0000 7f07b8c00700 1 -- 172.21.15.37:0/469443727 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f07b0056c90 con 0x7f07bc0fde90 2024-03-29T08:02:22.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:22.733+0000 7f07c3400700 1 -- 172.21.15.37:0/469443727 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f07bc04bed0 con 0x7f07bc0fde90 2024-03-29T08:02:22.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:22.737+0000 7f07b8c00700 1 -- 172.21.15.37:0/469443727 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v19) v1 ==== 74+0+167169 (secure 0 0 0) 0x7f07b0056820 con 0x7f07bc0fde90 2024-03-29T08:02:22.739 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:02:22.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:22.740+0000 7f07c3400700 1 -- 172.21.15.37:0/469443727 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f07a40692b0 msgr2=0x7f07a406b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:02:22.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:22.740+0000 7f07c3400700 1 --2- 172.21.15.37:0/469443727 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f07a40692b0 0x7f07a406b760 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f07a800b3b0 tx=0x7f07a8009e30 comp rx=0 tx=0).stop 2024-03-29T08:02:22.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:22.740+0000 7f07c3400700 1 -- 172.21.15.37:0/469443727 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f07bc0fde90 msgr2=0x7f07bc115d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:02:22.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:22.740+0000 7f07c3400700 1 --2- 172.21.15.37:0/469443727 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f07bc0fde90 0x7f07bc115d30 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f07b0002a50 tx=0x7f07b00046b0 comp rx=0 tx=0).stop 2024-03-29T08:02:22.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:22.740+0000 7f07c3400700 1 -- 172.21.15.37:0/469443727 shutdown_connections 2024-03-29T08:02:22.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:22.740+0000 7f07c3400700 1 --2- 172.21.15.37:0/469443727 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f07a40692b0 0x7f07a406b760 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:02:22.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:22.740+0000 7f07c3400700 1 --2- 172.21.15.37:0/469443727 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f07bc0fde90 0x7f07bc115d30 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:02:22.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:22.740+0000 7f07c3400700 1 --2- 172.21.15.37:0/469443727 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f07bc0fe830 0x7f07bc116270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:02:22.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:22.740+0000 7f07c3400700 1 -- 172.21.15.37:0/469443727 >> 172.21.15.37:0/469443727 conn(0x7f07bc072bc0 msgr2=0x7f07bc0fb840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:02:22.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:22.740+0000 7f07c3400700 1 -- 172.21.15.37:0/469443727 shutdown_connections 2024-03-29T08:02:22.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:22.740+0000 7f07c3400700 1 -- 172.21.15.37:0/469443727 wait complete. 2024-03-29T08:02:22.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:02:22 smithi037 ceph-mon[35162]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:02:23.055 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:02:22 smithi148 ceph-mon[44992]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:02:23.385 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":19,"active_gid":14208,"active_name":"smithi037.wjikii","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":1802836043},{"type":"v1","addr":"172.21.15.37:6801","nonce":1802836043}]},"active_addr":"172.21.15.37:6801/1802836043","active_change":"2024-03-29T07:57:55.403020+0000","active_mgr_features":4540138320759226367,"available":true,"standbys":[{"gid":14230,"name":"smithi148.onhgvm","mgr_features":4540138320759226367,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","prometheus","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.37:8443/","prometheus":"http://172.21.15.37:9283/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":5,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.37:0","nonce":2242434702}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.37:0","nonce":1395183736}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.37:0","nonce":2522921436}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.37:0","nonce":2701457924}]}]} 2024-03-29T08:02:23.390 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-03-29T08:02:23.390 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-03-29T08:02:23.391 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 99b492c2-eda1-11ee-b647-cb9ed24678a4 -- ceph osd dump --format=json 2024-03-29T08:02:23.827 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/mon.smithi037/config 2024-03-29T08:02:23.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:02:23 smithi037 ceph-mon[35162]: from='client.? 172.21.15.37:0/469443727' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-03-29T08:02:24.055 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:02:23 smithi148 ceph-mon[44992]: from='client.? 172.21.15.37:0/469443727' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-03-29T08:02:24.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:02:24 smithi037 ceph-mon[35162]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:02:25.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:02:24 smithi148 ceph-mon[44992]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:02:25.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:25.617+0000 7f7142400700 1 -- 172.21.15.37:0/3664210466 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f713c1037a0 msgr2=0x7f713c0fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:02:25.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:25.617+0000 7f7142400700 1 --2- 172.21.15.37:0/3664210466 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f713c1037a0 0x7f713c0fce10 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f7128004940 tx=0x7f7128009e30 comp rx=0 tx=0).stop 2024-03-29T08:02:25.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:25.617+0000 7f7142400700 1 -- 172.21.15.37:0/3664210466 shutdown_connections 2024-03-29T08:02:25.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:25.617+0000 7f7142400700 1 --2- 172.21.15.37:0/3664210466 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f713c1037a0 0x7f713c0fce10 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:02:25.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:25.617+0000 7f7142400700 1 --2- 172.21.15.37:0/3664210466 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f713c102e00 0x7f713c1031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:02:25.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:25.617+0000 7f7142400700 1 -- 172.21.15.37:0/3664210466 >> 172.21.15.37:0/3664210466 conn(0x7f713c0f89b0 msgr2=0x7f713c0fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:02:25.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:25.617+0000 7f7142400700 1 -- 172.21.15.37:0/3664210466 shutdown_connections 2024-03-29T08:02:25.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:25.618+0000 7f7142400700 1 -- 172.21.15.37:0/3664210466 wait complete. 2024-03-29T08:02:25.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:25.618+0000 7f7142400700 1 Processor -- start 2024-03-29T08:02:25.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:25.618+0000 7f7142400700 1 -- start start 2024-03-29T08:02:25.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:25.619+0000 7f7142400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f713c102e00 0x7f713c115d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:02:25.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:25.619+0000 7f7142400700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f713c1037a0 0x7f713c116260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:02:25.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:25.619+0000 7f7142400700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f713c1168b0 con 0x7f713c102e00 2024-03-29T08:02:25.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:25.619+0000 7f7142400700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f713c10fc80 con 0x7f713c1037a0 2024-03-29T08:02:25.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:25.619+0000 7f713b400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f713c102e00 0x7f713c115d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:02:25.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:25.619+0000 7f713b400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f713c102e00 0x7f713c115d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58592/0 (socket says 172.21.15.37:58592) 2024-03-29T08:02:25.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:25.619+0000 7f713b400700 1 -- 172.21.15.37:0/1877696062 learned_addr learned my addr 172.21.15.37:0/1877696062 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:02:25.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:25.619+0000 7f713aa00700 1 --2- 172.21.15.37:0/1877696062 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f713c1037a0 0x7f713c116260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:02:25.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:25.619+0000 7f713b400700 1 -- 172.21.15.37:0/1877696062 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f713c1037a0 msgr2=0x7f713c116260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:02:25.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:25.620+0000 7f713b400700 1 --2- 172.21.15.37:0/1877696062 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f713c1037a0 0x7f713c116260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:02:25.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:25.620+0000 7f713b400700 1 -- 172.21.15.37:0/1877696062 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7128005040 con 0x7f713c102e00 2024-03-29T08:02:25.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:25.620+0000 7f713b400700 1 --2- 172.21.15.37:0/1877696062 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f713c102e00 0x7f713c115d20 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f712c00c8c0 tx=0x7f712c0044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:02:25.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:25.624+0000 7f7133400700 1 -- 172.21.15.37:0/1877696062 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f712c00a980 con 0x7f713c102e00 2024-03-29T08:02:25.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:25.624+0000 7f7142400700 1 -- 172.21.15.37:0/1877696062 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f713c10ff60 con 0x7f713c102e00 2024-03-29T08:02:25.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:25.624+0000 7f7133400700 1 -- 172.21.15.37:0/1877696062 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f712c00ce60 con 0x7f713c102e00 2024-03-29T08:02:25.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:25.624+0000 7f7133400700 1 -- 172.21.15.37:0/1877696062 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f712c0113a0 con 0x7f713c102e00 2024-03-29T08:02:25.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:25.624+0000 7f7142400700 1 -- 172.21.15.37:0/1877696062 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f713c110520 con 0x7f713c102e00 2024-03-29T08:02:25.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:25.626+0000 7f7142400700 1 -- 172.21.15.37:0/1877696062 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f713c0fe550 con 0x7f713c102e00 2024-03-29T08:02:25.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:25.627+0000 7f7133400700 1 -- 172.21.15.37:0/1877696062 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f712c012400 con 0x7f713c102e00 2024-03-29T08:02:25.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:25.628+0000 7f7133400700 1 --2- 172.21.15.37:0/1877696062 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f71200692b0 0x7f712006b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:02:25.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:25.628+0000 7f7133400700 1 -- 172.21.15.37:0/1877696062 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f712c086fe0 con 0x7f713c102e00 2024-03-29T08:02:25.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:25.628+0000 7f713aa00700 1 --2- 172.21.15.37:0/1877696062 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f71200692b0 0x7f712006b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:02:25.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:25.628+0000 7f713aa00700 1 --2- 172.21.15.37:0/1877696062 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f71200692b0 0x7f712006b760 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f713c1115c0 tx=0x7f7128009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:02:25.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:25.632+0000 7f7133400700 1 -- 172.21.15.37:0/1877696062 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f712c057060 con 0x7f713c102e00 2024-03-29T08:02:25.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:25.837+0000 7f7142400700 1 -- 172.21.15.37:0/1877696062 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f713c05f5f0 con 0x7f713c102e00 2024-03-29T08:02:25.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:25.838+0000 7f7133400700 1 -- 172.21.15.37:0/1877696062 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) v1 ==== 74+0+11071 (secure 0 0 0) 0x7f712c056bf0 con 0x7f713c102e00 2024-03-29T08:02:25.837 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:02:25.837 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":41,"fsid":"99b492c2-eda1-11ee-b647-cb9ed24678a4","created":"2024-03-29T07:55:23.673934+0000","modified":"2024-03-29T08:02:05.562456+0000","last_up_change":"2024-03-29T08:02:04.552402+0000","last_in_change":"2024-03-29T08:01:43.066485+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T08:00:32.022416+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"dba18fb6-b27d-4258-b769-dc89f4fef44d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":3851364162},{"type":"v1","addr":"172.21.15.37:6803","nonce":3851364162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":3851364162},{"type":"v1","addr":"172.21.15.37:6805","nonce":3851364162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":3851364162},{"type":"v1","addr":"172.21.15.37:6809","nonce":3851364162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":3851364162},{"type":"v1","addr":"172.21.15.37:6807","nonce":3851364162}]},"public_addr":"172.21.15.37:6803/3851364162","cluster_addr":"172.21.15.37:6805/3851364162","heartbeat_back_addr":"172.21.15.37:6809/3851364162","heartbeat_front_addr":"172.21.15.37:6807/3851364162","state":["exists","up"]},{"osd":1,"uuid":"2876dd3f-aefb-4298-8fe4-562245e1103e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":3949957441},{"type":"v1","addr":"172.21.15.37:6811","nonce":3949957441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":3949957441},{"type":"v1","addr":"172.21.15.37:6813","nonce":3949957441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":3949957441},{"type":"v1","addr":"172.21.15.37:6817","nonce":3949957441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":3949957441},{"type":"v1","addr":"172.21.15.37:6815","nonce":3949957441}]},"public_addr":"172.21.15.37:6811/3949957441","cluster_addr":"172.21.15.37:6813/3949957441","heartbeat_back_addr":"172.21.15.37:6817/3949957441","heartbeat_front_addr":"172.21.15.37:6815/3949957441","state":["exists","up"]},{"osd":2,"uuid":"bba0f95c-f4e3-404e-adf7-b90353a96af4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":4002901863},{"type":"v1","addr":"172.21.15.37:6819","nonce":4002901863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":4002901863},{"type":"v1","addr":"172.21.15.37:6821","nonce":4002901863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":4002901863},{"type":"v1","addr":"172.21.15.37:6825","nonce":4002901863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":4002901863},{"type":"v1","addr":"172.21.15.37:6823","nonce":4002901863}]},"public_addr":"172.21.15.37:6819/4002901863","cluster_addr":"172.21.15.37:6821/4002901863","heartbeat_back_addr":"172.21.15.37:6825/4002901863","heartbeat_front_addr":"172.21.15.37:6823/4002901863","state":["exists","up"]},{"osd":3,"uuid":"8da1c140-ada5-4879-b5d9-1ed81b0c9aef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":267019749},{"type":"v1","addr":"172.21.15.148:6801","nonce":267019749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":267019749},{"type":"v1","addr":"172.21.15.148:6803","nonce":267019749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":267019749},{"type":"v1","addr":"172.21.15.148:6807","nonce":267019749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":267019749},{"type":"v1","addr":"172.21.15.148:6805","nonce":267019749}]},"public_addr":"172.21.15.148:6801/267019749","cluster_addr":"172.21.15.148:6803/267019749","heartbeat_back_addr":"172.21.15.148:6807/267019749","heartbeat_front_addr":"172.21.15.148:6805/267019749","state":["exists","up"]},{"osd":4,"uuid":"9af70e66-bbca-48c7-bad1-7ac78270cf16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":4113691443},{"type":"v1","addr":"172.21.15.148:6809","nonce":4113691443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":4113691443},{"type":"v1","addr":"172.21.15.148:6811","nonce":4113691443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":4113691443},{"type":"v1","addr":"172.21.15.148:6815","nonce":4113691443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":4113691443},{"type":"v1","addr":"172.21.15.148:6813","nonce":4113691443}]},"public_addr":"172.21.15.148:6809/4113691443","cluster_addr":"172.21.15.148:6811/4113691443","heartbeat_back_addr":"172.21.15.148:6815/4113691443","heartbeat_front_addr":"172.21.15.148:6813/4113691443","state":["exists","up"]},{"osd":5,"uuid":"a66944e4-38bd-4403-b45f-7de01ddfb7a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":267287812},{"type":"v1","addr":"172.21.15.148:6817","nonce":267287812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":267287812},{"type":"v1","addr":"172.21.15.148:6819","nonce":267287812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":267287812},{"type":"v1","addr":"172.21.15.148:6823","nonce":267287812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":267287812},{"type":"v1","addr":"172.21.15.148:6821","nonce":267287812}]},"public_addr":"172.21.15.148:6817/267287812","cluster_addr":"172.21.15.148:6819/267287812","heartbeat_back_addr":"172.21.15.148:6823/267287812","heartbeat_front_addr":"172.21.15.148:6821/267287812","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T07:59:23.110900+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T07:59:55.540549+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:00:28.082290+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:00:59.496893+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:01:31.307445+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:02:03.191177+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:0/1421768676":"2024-03-30T07:57:55.402909+0000","172.21.15.37:6801/4245458578":"2024-03-30T07:57:55.402909+0000","172.21.15.37:6800/4245458578":"2024-03-30T07:57:55.402909+0000","172.21.15.37:0/368538117":"2024-03-30T07:57:55.402909+0000","172.21.15.37:0/2483635161":"2024-03-30T07:56:36.633439+0000","172.21.15.37:0/1131673968":"2024-03-30T07:56:36.633439+0000","172.21.15.37:0/3846762095":"2024-03-30T07:57:55.402909+0000","172.21.15.37:0/1909774742":"2024-03-30T07:55:59.989528+0000","172.21.15.37:0/959705723":"2024-03-30T07:56:36.633439+0000","172.21.15.37:0/3644857958":"2024-03-30T07:55:59.989528+0000","172.21.15.37:6800/2746946658":"2024-03-30T07:56:36.633439+0000","172.21.15.37:6801/2746946658":"2024-03-30T07:56:36.633439+0000","172.21.15.37:0/3116346321":"2024-03-30T07:55:59.989528+0000","172.21.15.37:6800/2762339582":"2024-03-30T07:55:59.989528+0000","172.21.15.37:6801/2762339582":"2024-03-30T07:55:59.989528+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T08:02:25.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:25.840+0000 7f7142400700 1 -- 172.21.15.37:0/1877696062 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f71200692b0 msgr2=0x7f712006b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:02:25.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:25.840+0000 7f7142400700 1 --2- 172.21.15.37:0/1877696062 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f71200692b0 0x7f712006b760 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f713c1115c0 tx=0x7f7128009e30 comp rx=0 tx=0).stop 2024-03-29T08:02:25.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:25.840+0000 7f7142400700 1 -- 172.21.15.37:0/1877696062 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f713c102e00 msgr2=0x7f713c115d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:02:25.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:25.840+0000 7f7142400700 1 --2- 172.21.15.37:0/1877696062 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f713c102e00 0x7f713c115d20 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f712c00c8c0 tx=0x7f712c0044c0 comp rx=0 tx=0).stop 2024-03-29T08:02:25.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:25.840+0000 7f7142400700 1 -- 172.21.15.37:0/1877696062 shutdown_connections 2024-03-29T08:02:25.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:25.840+0000 7f7142400700 1 --2- 172.21.15.37:0/1877696062 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f71200692b0 0x7f712006b760 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:02:25.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:25.840+0000 7f7142400700 1 --2- 172.21.15.37:0/1877696062 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f713c102e00 0x7f713c115d20 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:02:25.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:25.840+0000 7f7142400700 1 --2- 172.21.15.37:0/1877696062 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f713c1037a0 0x7f713c116260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:02:25.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:25.840+0000 7f7142400700 1 -- 172.21.15.37:0/1877696062 >> 172.21.15.37:0/1877696062 conn(0x7f713c0f89b0 msgr2=0x7f713c0f99f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:02:25.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:25.840+0000 7f7142400700 1 -- 172.21.15.37:0/1877696062 shutdown_connections 2024-03-29T08:02:25.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:25.840+0000 7f7142400700 1 -- 172.21.15.37:0/1877696062 wait complete. 2024-03-29T08:02:25.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:02:25 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:02:26.055 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:02:25 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:02:26.554 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-03-29T08:02:26.555 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 99b492c2-eda1-11ee-b647-cb9ed24678a4 -- ceph osd dump --format=json 2024-03-29T08:02:26.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:02:26 smithi037 ceph-mon[35162]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:02:26.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:02:26 smithi037 ceph-mon[35162]: from='client.? 172.21.15.37:0/1877696062' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-29T08:02:26.987 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/mon.smithi037/config 2024-03-29T08:02:27.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:02:26 smithi148 ceph-mon[44992]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:02:27.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:02:26 smithi148 ceph-mon[44992]: from='client.? 172.21.15.37:0/1877696062' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-29T08:02:28.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:28.770+0000 7f91a6c00700 1 -- 172.21.15.37:0/380768414 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f91a01037a0 msgr2=0x7f91a00fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:02:28.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:28.770+0000 7f91a6c00700 1 --2- 172.21.15.37:0/380768414 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f91a01037a0 0x7f91a00fce10 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f918c004940 tx=0x7f918c009e30 comp rx=0 tx=0).stop 2024-03-29T08:02:28.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:28.770+0000 7f91a6c00700 1 -- 172.21.15.37:0/380768414 shutdown_connections 2024-03-29T08:02:28.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:28.770+0000 7f91a6c00700 1 --2- 172.21.15.37:0/380768414 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f91a01037a0 0x7f91a00fce10 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:02:28.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:28.770+0000 7f91a6c00700 1 --2- 172.21.15.37:0/380768414 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f91a0102e00 0x7f91a01031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:02:28.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:28.770+0000 7f91a6c00700 1 -- 172.21.15.37:0/380768414 >> 172.21.15.37:0/380768414 conn(0x7f91a00f89b0 msgr2=0x7f91a00fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:02:28.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:28.770+0000 7f91a6c00700 1 -- 172.21.15.37:0/380768414 shutdown_connections 2024-03-29T08:02:28.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:28.770+0000 7f91a6c00700 1 -- 172.21.15.37:0/380768414 wait complete. 2024-03-29T08:02:28.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:28.771+0000 7f91a6c00700 1 Processor -- start 2024-03-29T08:02:28.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:28.771+0000 7f91a6c00700 1 -- start start 2024-03-29T08:02:28.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:28.771+0000 7f91a6c00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f91a0102e00 0x7f91a0113b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:02:28.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:28.771+0000 7f91a6c00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f91a01037a0 0x7f91a0114040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:02:28.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:28.771+0000 7f91a6c00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91a0114690 con 0x7f91a0102e00 2024-03-29T08:02:28.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:28.771+0000 7f91a6c00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91a010da80 con 0x7f91a01037a0 2024-03-29T08:02:28.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:28.771+0000 7f919fe00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f91a0102e00 0x7f91a0113b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:02:28.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:28.772+0000 7f919fe00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f91a0102e00 0x7f91a0113b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58618/0 (socket says 172.21.15.37:58618) 2024-03-29T08:02:28.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:28.772+0000 7f919fe00700 1 -- 172.21.15.37:0/781974379 learned_addr learned my addr 172.21.15.37:0/781974379 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:02:28.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:28.772+0000 7f919f400700 1 --2- 172.21.15.37:0/781974379 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f91a01037a0 0x7f91a0114040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:02:28.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:28.772+0000 7f919fe00700 1 -- 172.21.15.37:0/781974379 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f91a01037a0 msgr2=0x7f91a0114040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:02:28.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:28.772+0000 7f919fe00700 1 --2- 172.21.15.37:0/781974379 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f91a01037a0 0x7f91a0114040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:02:28.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:28.772+0000 7f919fe00700 1 -- 172.21.15.37:0/781974379 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f918c005040 con 0x7f91a0102e00 2024-03-29T08:02:28.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:28.772+0000 7f919fe00700 1 --2- 172.21.15.37:0/781974379 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f91a0102e00 0x7f91a0113b00 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f9190002a50 tx=0x7f91900046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:02:28.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:28.772+0000 7f9197e00700 1 -- 172.21.15.37:0/781974379 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9190007cc0 con 0x7f91a0102e00 2024-03-29T08:02:28.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:28.772+0000 7f9197e00700 1 -- 172.21.15.37:0/781974379 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f919000f040 con 0x7f91a0102e00 2024-03-29T08:02:28.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:28.772+0000 7f9197e00700 1 -- 172.21.15.37:0/781974379 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9190009540 con 0x7f91a0102e00 2024-03-29T08:02:28.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:28.772+0000 7f91a6c00700 1 -- 172.21.15.37:0/781974379 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91a010dd60 con 0x7f91a0102e00 2024-03-29T08:02:28.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:28.773+0000 7f91a6c00700 1 -- 172.21.15.37:0/781974379 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91a010e320 con 0x7f91a0102e00 2024-03-29T08:02:28.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:28.774+0000 7f91a6c00700 1 -- 172.21.15.37:0/781974379 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91a00fe550 con 0x7f91a0102e00 2024-03-29T08:02:28.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:28.774+0000 7f9197e00700 1 -- 172.21.15.37:0/781974379 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f91900096a0 con 0x7f91a0102e00 2024-03-29T08:02:28.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:28.776+0000 7f9197e00700 1 --2- 172.21.15.37:0/781974379 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f91840692b0 0x7f918406b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:02:28.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:28.776+0000 7f9197e00700 1 -- 172.21.15.37:0/781974379 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f9190086c30 con 0x7f91a0102e00 2024-03-29T08:02:28.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:28.776+0000 7f919f400700 1 --2- 172.21.15.37:0/781974379 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f91840692b0 0x7f918406b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:02:28.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:28.777+0000 7f919f400700 1 --2- 172.21.15.37:0/781974379 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f91840692b0 0x7f918406b760 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f91a010ec40 tx=0x7f918c009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:02:28.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:28.780+0000 7f9197e00700 1 -- 172.21.15.37:0/781974379 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f9190056c90 con 0x7f91a0102e00 2024-03-29T08:02:28.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:02:28 smithi037 ceph-mon[35162]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:02:28.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:28.985+0000 7f91a6c00700 1 -- 172.21.15.37:0/781974379 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f91a0109da0 con 0x7f91a0102e00 2024-03-29T08:02:28.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:28.986+0000 7f9197e00700 1 -- 172.21.15.37:0/781974379 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) v1 ==== 74+0+11071 (secure 0 0 0) 0x7f9190056820 con 0x7f91a0102e00 2024-03-29T08:02:28.986 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:02:28.986 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":41,"fsid":"99b492c2-eda1-11ee-b647-cb9ed24678a4","created":"2024-03-29T07:55:23.673934+0000","modified":"2024-03-29T08:02:05.562456+0000","last_up_change":"2024-03-29T08:02:04.552402+0000","last_in_change":"2024-03-29T08:01:43.066485+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T08:00:32.022416+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"dba18fb6-b27d-4258-b769-dc89f4fef44d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":3851364162},{"type":"v1","addr":"172.21.15.37:6803","nonce":3851364162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":3851364162},{"type":"v1","addr":"172.21.15.37:6805","nonce":3851364162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":3851364162},{"type":"v1","addr":"172.21.15.37:6809","nonce":3851364162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":3851364162},{"type":"v1","addr":"172.21.15.37:6807","nonce":3851364162}]},"public_addr":"172.21.15.37:6803/3851364162","cluster_addr":"172.21.15.37:6805/3851364162","heartbeat_back_addr":"172.21.15.37:6809/3851364162","heartbeat_front_addr":"172.21.15.37:6807/3851364162","state":["exists","up"]},{"osd":1,"uuid":"2876dd3f-aefb-4298-8fe4-562245e1103e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":3949957441},{"type":"v1","addr":"172.21.15.37:6811","nonce":3949957441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":3949957441},{"type":"v1","addr":"172.21.15.37:6813","nonce":3949957441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":3949957441},{"type":"v1","addr":"172.21.15.37:6817","nonce":3949957441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":3949957441},{"type":"v1","addr":"172.21.15.37:6815","nonce":3949957441}]},"public_addr":"172.21.15.37:6811/3949957441","cluster_addr":"172.21.15.37:6813/3949957441","heartbeat_back_addr":"172.21.15.37:6817/3949957441","heartbeat_front_addr":"172.21.15.37:6815/3949957441","state":["exists","up"]},{"osd":2,"uuid":"bba0f95c-f4e3-404e-adf7-b90353a96af4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":4002901863},{"type":"v1","addr":"172.21.15.37:6819","nonce":4002901863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":4002901863},{"type":"v1","addr":"172.21.15.37:6821","nonce":4002901863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":4002901863},{"type":"v1","addr":"172.21.15.37:6825","nonce":4002901863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":4002901863},{"type":"v1","addr":"172.21.15.37:6823","nonce":4002901863}]},"public_addr":"172.21.15.37:6819/4002901863","cluster_addr":"172.21.15.37:6821/4002901863","heartbeat_back_addr":"172.21.15.37:6825/4002901863","heartbeat_front_addr":"172.21.15.37:6823/4002901863","state":["exists","up"]},{"osd":3,"uuid":"8da1c140-ada5-4879-b5d9-1ed81b0c9aef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":267019749},{"type":"v1","addr":"172.21.15.148:6801","nonce":267019749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":267019749},{"type":"v1","addr":"172.21.15.148:6803","nonce":267019749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":267019749},{"type":"v1","addr":"172.21.15.148:6807","nonce":267019749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":267019749},{"type":"v1","addr":"172.21.15.148:6805","nonce":267019749}]},"public_addr":"172.21.15.148:6801/267019749","cluster_addr":"172.21.15.148:6803/267019749","heartbeat_back_addr":"172.21.15.148:6807/267019749","heartbeat_front_addr":"172.21.15.148:6805/267019749","state":["exists","up"]},{"osd":4,"uuid":"9af70e66-bbca-48c7-bad1-7ac78270cf16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":4113691443},{"type":"v1","addr":"172.21.15.148:6809","nonce":4113691443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":4113691443},{"type":"v1","addr":"172.21.15.148:6811","nonce":4113691443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":4113691443},{"type":"v1","addr":"172.21.15.148:6815","nonce":4113691443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":4113691443},{"type":"v1","addr":"172.21.15.148:6813","nonce":4113691443}]},"public_addr":"172.21.15.148:6809/4113691443","cluster_addr":"172.21.15.148:6811/4113691443","heartbeat_back_addr":"172.21.15.148:6815/4113691443","heartbeat_front_addr":"172.21.15.148:6813/4113691443","state":["exists","up"]},{"osd":5,"uuid":"a66944e4-38bd-4403-b45f-7de01ddfb7a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":267287812},{"type":"v1","addr":"172.21.15.148:6817","nonce":267287812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":267287812},{"type":"v1","addr":"172.21.15.148:6819","nonce":267287812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":267287812},{"type":"v1","addr":"172.21.15.148:6823","nonce":267287812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":267287812},{"type":"v1","addr":"172.21.15.148:6821","nonce":267287812}]},"public_addr":"172.21.15.148:6817/267287812","cluster_addr":"172.21.15.148:6819/267287812","heartbeat_back_addr":"172.21.15.148:6823/267287812","heartbeat_front_addr":"172.21.15.148:6821/267287812","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T07:59:23.110900+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T07:59:55.540549+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:00:28.082290+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:00:59.496893+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:01:31.307445+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:02:03.191177+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:0/1421768676":"2024-03-30T07:57:55.402909+0000","172.21.15.37:6801/4245458578":"2024-03-30T07:57:55.402909+0000","172.21.15.37:6800/4245458578":"2024-03-30T07:57:55.402909+0000","172.21.15.37:0/368538117":"2024-03-30T07:57:55.402909+0000","172.21.15.37:0/2483635161":"2024-03-30T07:56:36.633439+0000","172.21.15.37:0/1131673968":"2024-03-30T07:56:36.633439+0000","172.21.15.37:0/3846762095":"2024-03-30T07:57:55.402909+0000","172.21.15.37:0/1909774742":"2024-03-30T07:55:59.989528+0000","172.21.15.37:0/959705723":"2024-03-30T07:56:36.633439+0000","172.21.15.37:0/3644857958":"2024-03-30T07:55:59.989528+0000","172.21.15.37:6800/2746946658":"2024-03-30T07:56:36.633439+0000","172.21.15.37:6801/2746946658":"2024-03-30T07:56:36.633439+0000","172.21.15.37:0/3116346321":"2024-03-30T07:55:59.989528+0000","172.21.15.37:6800/2762339582":"2024-03-30T07:55:59.989528+0000","172.21.15.37:6801/2762339582":"2024-03-30T07:55:59.989528+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T08:02:28.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:28.988+0000 7f91a6c00700 1 -- 172.21.15.37:0/781974379 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f91840692b0 msgr2=0x7f918406b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:02:28.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:28.988+0000 7f91a6c00700 1 --2- 172.21.15.37:0/781974379 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f91840692b0 0x7f918406b760 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f91a010ec40 tx=0x7f918c009e30 comp rx=0 tx=0).stop 2024-03-29T08:02:28.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:28.988+0000 7f91a6c00700 1 -- 172.21.15.37:0/781974379 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f91a0102e00 msgr2=0x7f91a0113b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:02:28.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:28.988+0000 7f91a6c00700 1 --2- 172.21.15.37:0/781974379 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f91a0102e00 0x7f91a0113b00 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f9190002a50 tx=0x7f91900046b0 comp rx=0 tx=0).stop 2024-03-29T08:02:28.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:28.988+0000 7f91a6c00700 1 -- 172.21.15.37:0/781974379 shutdown_connections 2024-03-29T08:02:28.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:28.988+0000 7f91a6c00700 1 --2- 172.21.15.37:0/781974379 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f91840692b0 0x7f918406b760 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:02:28.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:28.988+0000 7f91a6c00700 1 --2- 172.21.15.37:0/781974379 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f91a0102e00 0x7f91a0113b00 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:02:28.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:28.988+0000 7f91a6c00700 1 --2- 172.21.15.37:0/781974379 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f91a01037a0 0x7f91a0114040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:02:28.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:28.988+0000 7f91a6c00700 1 -- 172.21.15.37:0/781974379 >> 172.21.15.37:0/781974379 conn(0x7f91a00f89b0 msgr2=0x7f91a00f9ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:02:28.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:28.988+0000 7f91a6c00700 1 -- 172.21.15.37:0/781974379 shutdown_connections 2024-03-29T08:02:28.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:28.988+0000 7f91a6c00700 1 -- 172.21.15.37:0/781974379 wait complete. 2024-03-29T08:02:29.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:02:28 smithi148 ceph-mon[44992]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:02:29.787 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 99b492c2-eda1-11ee-b647-cb9ed24678a4 -- ceph tell osd.0 flush_pg_stats 2024-03-29T08:02:29.788 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 99b492c2-eda1-11ee-b647-cb9ed24678a4 -- ceph tell osd.1 flush_pg_stats 2024-03-29T08:02:29.788 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 99b492c2-eda1-11ee-b647-cb9ed24678a4 -- ceph tell osd.2 flush_pg_stats 2024-03-29T08:02:29.789 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 99b492c2-eda1-11ee-b647-cb9ed24678a4 -- ceph tell osd.3 flush_pg_stats 2024-03-29T08:02:29.789 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 99b492c2-eda1-11ee-b647-cb9ed24678a4 -- ceph tell osd.4 flush_pg_stats 2024-03-29T08:02:29.789 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 99b492c2-eda1-11ee-b647-cb9ed24678a4 -- ceph tell osd.5 flush_pg_stats 2024-03-29T08:02:29.886 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:02:29 smithi037 ceph-mon[35162]: from='client.? 172.21.15.37:0/781974379' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-29T08:02:30.055 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:02:29 smithi148 ceph-mon[44992]: from='client.? 172.21.15.37:0/781974379' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-29T08:02:30.361 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/mon.smithi037/config 2024-03-29T08:02:30.376 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/mon.smithi037/config 2024-03-29T08:02:30.917 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:02:30 smithi037 ceph-mon[35162]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:02:31.055 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:02:30 smithi148 ceph-mon[44992]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:02:32.344 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/mon.smithi037/config 2024-03-29T08:02:32.344 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/mon.smithi037/config 2024-03-29T08:02:32.344 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/mon.smithi037/config 2024-03-29T08:02:32.346 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/mon.smithi037/config 2024-03-29T08:02:32.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:32.887+0000 7fb5bb200700 1 -- 172.21.15.37:0/3405810372 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb5b4071f90 msgr2=0x7fb5b4070360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:02:32.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:32.887+0000 7fb5bb200700 1 --2- 172.21.15.37:0/3405810372 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb5b4071f90 0x7fb5b4070360 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fb5a4004970 tx=0x7fb5a4009e30 comp rx=0 tx=0).stop 2024-03-29T08:02:32.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:32.887+0000 7fb5bb200700 1 -- 172.21.15.37:0/3405810372 shutdown_connections 2024-03-29T08:02:32.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:32.887+0000 7fb5bb200700 1 --2- 172.21.15.37:0/3405810372 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb5b40708a0 0x7fb5b4070d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:02:32.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:32.887+0000 7fb5bb200700 1 --2- 172.21.15.37:0/3405810372 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb5b4071f90 0x7fb5b4070360 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:02:32.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:32.887+0000 7fb5bb200700 1 -- 172.21.15.37:0/3405810372 >> 172.21.15.37:0/3405810372 conn(0x7fb5b40f89b0 msgr2=0x7fb5b40fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:02:32.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:32.887+0000 7fb5bb200700 1 -- 172.21.15.37:0/3405810372 shutdown_connections 2024-03-29T08:02:32.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:32.888+0000 7fb5bb200700 1 -- 172.21.15.37:0/3405810372 wait complete. 2024-03-29T08:02:32.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:32.888+0000 7fb5bb200700 1 Processor -- start 2024-03-29T08:02:32.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:32.888+0000 7fb5bb200700 1 -- start start 2024-03-29T08:02:32.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:32.889+0000 7fb5bb200700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb5b40708a0 0x7fb5b411a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:02:32.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:32.889+0000 7fb5bb200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb5b4071f90 0x7fb5b411a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:02:32.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:32.889+0000 7fb5bb200700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5b411ac50 con 0x7fb5b4071f90 2024-03-29T08:02:32.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:32.889+0000 7fb5bb200700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5b4114020 con 0x7fb5b40708a0 2024-03-29T08:02:32.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:32.889+0000 7fb5b3400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb5b4071f90 0x7fb5b411a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:02:32.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:32.889+0000 7fb5b3400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb5b4071f90 0x7fb5b411a600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37646/0 (socket says 172.21.15.37:37646) 2024-03-29T08:02:32.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:32.889+0000 7fb5b3400700 1 -- 172.21.15.37:0/2011363347 learned_addr learned my addr 172.21.15.37:0/2011363347 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:02:32.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:32.889+0000 7fb5b3e00700 1 --2- 172.21.15.37:0/2011363347 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb5b40708a0 0x7fb5b411a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:02:32.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:32.890+0000 7fb5b3400700 1 -- 172.21.15.37:0/2011363347 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb5b40708a0 msgr2=0x7fb5b411a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:02:32.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:32.890+0000 7fb5b3400700 1 --2- 172.21.15.37:0/2011363347 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb5b40708a0 0x7fb5b411a0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:02:32.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:32.890+0000 7fb5b3400700 1 -- 172.21.15.37:0/2011363347 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5a4005040 con 0x7fb5b4071f90 2024-03-29T08:02:32.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:32.890+0000 7fb5b3400700 1 --2- 172.21.15.37:0/2011363347 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb5b4071f90 0x7fb5b411a600 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fb5a0002a50 tx=0x7fb5a00046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:02:32.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:32.890+0000 7fb5abe00700 1 -- 172.21.15.37:0/2011363347 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb5a0007cc0 con 0x7fb5b4071f90 2024-03-29T08:02:32.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:32.890+0000 7fb5abe00700 1 -- 172.21.15.37:0/2011363347 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fb5a000f040 con 0x7fb5b4071f90 2024-03-29T08:02:32.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:32.890+0000 7fb5bb200700 1 -- 172.21.15.37:0/2011363347 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5b4114300 con 0x7fb5b4071f90 2024-03-29T08:02:32.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:32.890+0000 7fb5abe00700 1 -- 172.21.15.37:0/2011363347 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb5a000d9c0 con 0x7fb5b4071f90 2024-03-29T08:02:32.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:32.890+0000 7fb5bb200700 1 -- 172.21.15.37:0/2011363347 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5b4114840 con 0x7fb5b4071f90 2024-03-29T08:02:32.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:32.891+0000 7fb5bb200700 1 -- 172.21.15.37:0/2011363347 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb5b4114d20 con 0x7fb5b4071f90 2024-03-29T08:02:32.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:32.893+0000 7fb5abe00700 1 -- 172.21.15.37:0/2011363347 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fb5a000db20 con 0x7fb5b4071f90 2024-03-29T08:02:32.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:32.893+0000 7fb5abe00700 1 --2- 172.21.15.37:0/2011363347 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7fb5980692b0 0x7fb59806b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:02:32.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:32.894+0000 7fb5abe00700 1 -- 172.21.15.37:0/2011363347 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fb5a0086c10 con 0x7fb5b4071f90 2024-03-29T08:02:32.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:32.894+0000 7fb5b3e00700 1 --2- 172.21.15.37:0/2011363347 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7fb5980692b0 0x7fb59806b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:02:32.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:32.894+0000 7fb5abe00700 1 --2- 172.21.15.37:0/2011363347 >> [v2:172.21.15.37:6818/4002901863,v1:172.21.15.37:6819/4002901863] conn(0x7fb59806ef80 0x7fb598071390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:02:32.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:32.894+0000 7fb5abe00700 1 -- 172.21.15.37:0/2011363347 --> [v2:172.21.15.37:6818/4002901863,v1:172.21.15.37:6819/4002901863] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb598071a40 con 0x7fb59806ef80 2024-03-29T08:02:32.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:32.894+0000 7fb5abe00700 1 -- 172.21.15.37:0/2011363347 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb5a008b220 con 0x7fb5b4071f90 2024-03-29T08:02:32.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:32.894+0000 7fb5b3e00700 1 --2- 172.21.15.37:0/2011363347 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7fb5980692b0 0x7fb59806b760 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb5a4009fd0 tx=0x7fb5a4009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:02:32.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:32.894+0000 7fb5b9200700 1 --2- 172.21.15.37:0/2011363347 >> [v2:172.21.15.37:6818/4002901863,v1:172.21.15.37:6819/4002901863] conn(0x7fb59806ef80 0x7fb598071390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:02:32.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:32.895+0000 7fb5b9200700 1 --2- 172.21.15.37:0/2011363347 >> [v2:172.21.15.37:6818/4002901863,v1:172.21.15.37:6819/4002901863] conn(0x7fb59806ef80 0x7fb598071390 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:02:32.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:32.896+0000 7fb5abe00700 1 -- 172.21.15.37:0/2011363347 <== osd.2 v2:172.21.15.37:6818/4002901863 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7fb598071a40 con 0x7fb59806ef80 2024-03-29T08:02:32.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:32.912+0000 7fb5bb200700 1 -- 172.21.15.37:0/2011363347 --> [v2:172.21.15.37:6818/4002901863,v1:172.21.15.37:6819/4002901863] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fb5b40fa570 con 0x7fb59806ef80 2024-03-29T08:02:32.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:32.912+0000 7fb5abe00700 1 -- 172.21.15.37:0/2011363347 <== osd.2 v2:172.21.15.37:6818/4002901863 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fb5b40fa570 con 0x7fb59806ef80 2024-03-29T08:02:32.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:32.912+0000 7fb5bb200700 1 -- 172.21.15.37:0/2011363347 >> [v2:172.21.15.37:6818/4002901863,v1:172.21.15.37:6819/4002901863] conn(0x7fb59806ef80 msgr2=0x7fb598071390 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:02:32.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:32.912+0000 7fb5bb200700 1 --2- 172.21.15.37:0/2011363347 >> [v2:172.21.15.37:6818/4002901863,v1:172.21.15.37:6819/4002901863] conn(0x7fb59806ef80 0x7fb598071390 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:02:32.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:32.912+0000 7fb5bb200700 1 -- 172.21.15.37:0/2011363347 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7fb5980692b0 msgr2=0x7fb59806b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:02:32.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:32.912+0000 7fb5bb200700 1 --2- 172.21.15.37:0/2011363347 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7fb5980692b0 0x7fb59806b760 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb5a4009fd0 tx=0x7fb5a4009e30 comp rx=0 tx=0).stop 2024-03-29T08:02:32.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:32.912+0000 7fb5bb200700 1 -- 172.21.15.37:0/2011363347 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb5b4071f90 msgr2=0x7fb5b411a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:02:32.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:32.912+0000 7fb5bb200700 1 --2- 172.21.15.37:0/2011363347 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb5b4071f90 0x7fb5b411a600 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fb5a0002a50 tx=0x7fb5a00046b0 comp rx=0 tx=0).stop 2024-03-29T08:02:32.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:32.912+0000 7fb5bb200700 1 -- 172.21.15.37:0/2011363347 shutdown_connections 2024-03-29T08:02:32.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:32.912+0000 7fb5bb200700 1 --2- 172.21.15.37:0/2011363347 >> [v2:172.21.15.37:6818/4002901863,v1:172.21.15.37:6819/4002901863] conn(0x7fb59806ef80 0x7fb598071390 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:02:32.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:32.912+0000 7fb5bb200700 1 --2- 172.21.15.37:0/2011363347 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7fb5980692b0 0x7fb59806b760 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:02:32.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:32.912+0000 7fb5bb200700 1 --2- 172.21.15.37:0/2011363347 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb5b40708a0 0x7fb5b411a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:02:32.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:32.912+0000 7fb5bb200700 1 --2- 172.21.15.37:0/2011363347 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb5b4071f90 0x7fb5b411a600 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:02:32.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:32.912+0000 7fb5bb200700 1 -- 172.21.15.37:0/2011363347 >> 172.21.15.37:0/2011363347 conn(0x7fb5b40f89b0 msgr2=0x7fb5b40f9b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:02:32.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:32.913+0000 7fb5bb200700 1 -- 172.21.15.37:0/2011363347 shutdown_connections 2024-03-29T08:02:32.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:32.913+0000 7fb5bb200700 1 -- 172.21.15.37:0/2011363347 wait complete. 2024-03-29T08:02:32.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:02:32 smithi037 ceph-mon[35162]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:02:33.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:02:32 smithi148 ceph-mon[44992]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:02:33.866 INFO:teuthology.orchestra.run.smithi037.stdout:81604378650 2024-03-29T08:02:33.866 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 99b492c2-eda1-11ee-b647-cb9ed24678a4 -- ceph osd last-stat-seq osd.2 2024-03-29T08:02:34.676 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:02:34 smithi037 ceph-mon[35162]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:02:35.055 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:02:34 smithi148 ceph-mon[44992]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:02:36.830 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:02:36 smithi037 ceph-mon[35162]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:02:37.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:02:36 smithi148 ceph-mon[44992]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:02:38.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:38.336+0000 7f3c72200700 1 -- 172.21.15.37:0/2118055726 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3c6c070300 msgr2=0x7f3c6c0706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:02:38.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:38.336+0000 7f3c72200700 1 --2- 172.21.15.37:0/2118055726 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3c6c070300 0x7f3c6c0706d0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f3c5c004970 tx=0x7f3c5c009e30 comp rx=0 tx=0).stop 2024-03-29T08:02:38.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:38.336+0000 7f3c72200700 1 -- 172.21.15.37:0/2118055726 shutdown_connections 2024-03-29T08:02:38.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:38.336+0000 7f3c72200700 1 --2- 172.21.15.37:0/2118055726 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3c6c070c10 0x7f3c6c109190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:02:38.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:38.336+0000 7f3c72200700 1 --2- 172.21.15.37:0/2118055726 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3c6c070300 0x7f3c6c0706d0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:02:38.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:38.336+0000 7f3c72200700 1 -- 172.21.15.37:0/2118055726 >> 172.21.15.37:0/2118055726 conn(0x7f3c6c0f8970 msgr2=0x7f3c6c0fad80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:02:38.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:38.336+0000 7f3c72200700 1 -- 172.21.15.37:0/2118055726 shutdown_connections 2024-03-29T08:02:38.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:38.336+0000 7f3c72200700 1 -- 172.21.15.37:0/2118055726 wait complete. 2024-03-29T08:02:38.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:38.337+0000 7f3c72200700 1 Processor -- start 2024-03-29T08:02:38.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:38.337+0000 7f3c72200700 1 -- start start 2024-03-29T08:02:38.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:38.338+0000 7f3c72200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3c6c070300 0x7f3c6c115c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:02:38.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:38.338+0000 7f3c72200700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3c6c070c10 0x7f3c6c1161d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:02:38.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:38.338+0000 7f3c72200700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c6c116820 con 0x7f3c6c070300 2024-03-29T08:02:38.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:38.338+0000 7f3c72200700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c6c10fc30 con 0x7f3c6c070c10 2024-03-29T08:02:38.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:38.338+0000 7f3c6b400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3c6c070300 0x7f3c6c115c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:02:38.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:38.338+0000 7f3c6b400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3c6c070300 0x7f3c6c115c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37656/0 (socket says 172.21.15.37:37656) 2024-03-29T08:02:38.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:38.338+0000 7f3c6b400700 1 -- 172.21.15.37:0/3052116444 learned_addr learned my addr 172.21.15.37:0/3052116444 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:02:38.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:38.338+0000 7f3c6aa00700 1 --2- 172.21.15.37:0/3052116444 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3c6c070c10 0x7f3c6c1161d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:02:38.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:38.338+0000 7f3c6b400700 1 -- 172.21.15.37:0/3052116444 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3c6c070c10 msgr2=0x7f3c6c1161d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:02:38.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:38.338+0000 7f3c6b400700 1 --2- 172.21.15.37:0/3052116444 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3c6c070c10 0x7f3c6c1161d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:02:38.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:38.338+0000 7f3c6b400700 1 -- 172.21.15.37:0/3052116444 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c5c005040 con 0x7f3c6c070300 2024-03-29T08:02:38.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:38.339+0000 7f3c6b400700 1 --2- 172.21.15.37:0/3052116444 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3c6c070300 0x7f3c6c115c90 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f3c5c004f50 tx=0x7f3c5c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:02:38.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:38.339+0000 7f3c63400700 1 -- 172.21.15.37:0/3052116444 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3c5c00e040 con 0x7f3c6c070300 2024-03-29T08:02:38.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:38.339+0000 7f3c63400700 1 -- 172.21.15.37:0/3052116444 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f3c5c013420 con 0x7f3c6c070300 2024-03-29T08:02:38.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:38.339+0000 7f3c72200700 1 -- 172.21.15.37:0/3052116444 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c6c10feb0 con 0x7f3c6c070300 2024-03-29T08:02:38.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:38.339+0000 7f3c63400700 1 -- 172.21.15.37:0/3052116444 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3c5c01c610 con 0x7f3c6c070300 2024-03-29T08:02:38.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:38.339+0000 7f3c72200700 1 -- 172.21.15.37:0/3052116444 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c6c1103f0 con 0x7f3c6c070300 2024-03-29T08:02:38.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:38.340+0000 7f3c72200700 1 -- 172.21.15.37:0/3052116444 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f3c6c110960 con 0x7f3c6c070300 2024-03-29T08:02:38.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:38.341+0000 7f3c63400700 1 -- 172.21.15.37:0/3052116444 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f3c5c013590 con 0x7f3c6c070300 2024-03-29T08:02:38.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:38.342+0000 7f3c63400700 1 --2- 172.21.15.37:0/3052116444 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f3c500692b0 0x7f3c5006b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:02:38.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:38.342+0000 7f3c63400700 1 -- 172.21.15.37:0/3052116444 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f3c5c088080 con 0x7f3c6c070300 2024-03-29T08:02:38.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:38.342+0000 7f3c6aa00700 1 --2- 172.21.15.37:0/3052116444 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f3c500692b0 0x7f3c5006b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:02:38.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:38.342+0000 7f3c63400700 1 --2- 172.21.15.37:0/3052116444 >> [v2:172.21.15.37:6802/3851364162,v1:172.21.15.37:6803/3851364162] conn(0x7f3c5006ef80 0x7f3c50071390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:02:38.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:38.342+0000 7f3c63400700 1 -- 172.21.15.37:0/3052116444 --> [v2:172.21.15.37:6802/3851364162,v1:172.21.15.37:6803/3851364162] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3c50071a40 con 0x7f3c5006ef80 2024-03-29T08:02:38.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:38.342+0000 7f3c63400700 1 -- 172.21.15.37:0/3052116444 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3c5c08a570 con 0x7f3c6c070300 2024-03-29T08:02:38.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:38.343+0000 7f3c6be00700 1 --2- 172.21.15.37:0/3052116444 >> [v2:172.21.15.37:6802/3851364162,v1:172.21.15.37:6803/3851364162] conn(0x7f3c5006ef80 0x7f3c50071390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:02:38.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:38.343+0000 7f3c6aa00700 1 --2- 172.21.15.37:0/3052116444 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f3c500692b0 0x7f3c5006b760 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f3c6c111570 tx=0x7f3c58009ee0 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:02:38.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:38.343+0000 7f3c6be00700 1 --2- 172.21.15.37:0/3052116444 >> [v2:172.21.15.37:6802/3851364162,v1:172.21.15.37:6803/3851364162] conn(0x7f3c5006ef80 0x7f3c50071390 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:02:38.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:38.344+0000 7f3c63400700 1 -- 172.21.15.37:0/3052116444 <== osd.0 v2:172.21.15.37:6802/3851364162 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f3c50071a40 con 0x7f3c5006ef80 2024-03-29T08:02:38.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:38.360+0000 7f3c72200700 1 -- 172.21.15.37:0/3052116444 --> [v2:172.21.15.37:6802/3851364162,v1:172.21.15.37:6803/3851364162] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f3c6c071a40 con 0x7f3c5006ef80 2024-03-29T08:02:38.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:38.360+0000 7f3c63400700 1 -- 172.21.15.37:0/3052116444 <== osd.0 v2:172.21.15.37:6802/3851364162 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f3c6c071a40 con 0x7f3c5006ef80 2024-03-29T08:02:38.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:38.361+0000 7f3c72200700 1 -- 172.21.15.37:0/3052116444 >> [v2:172.21.15.37:6802/3851364162,v1:172.21.15.37:6803/3851364162] conn(0x7f3c5006ef80 msgr2=0x7f3c50071390 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:02:38.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:38.361+0000 7f3c72200700 1 --2- 172.21.15.37:0/3052116444 >> [v2:172.21.15.37:6802/3851364162,v1:172.21.15.37:6803/3851364162] conn(0x7f3c5006ef80 0x7f3c50071390 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:02:38.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:38.361+0000 7f3c72200700 1 -- 172.21.15.37:0/3052116444 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f3c500692b0 msgr2=0x7f3c5006b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:02:38.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:38.361+0000 7f3c72200700 1 --2- 172.21.15.37:0/3052116444 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f3c500692b0 0x7f3c5006b760 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f3c6c111570 tx=0x7f3c58009ee0 comp rx=0 tx=0).stop 2024-03-29T08:02:38.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:38.361+0000 7f3c72200700 1 -- 172.21.15.37:0/3052116444 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3c6c070300 msgr2=0x7f3c6c115c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:02:38.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:38.361+0000 7f3c72200700 1 --2- 172.21.15.37:0/3052116444 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3c6c070300 0x7f3c6c115c90 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f3c5c004f50 tx=0x7f3c5c009e30 comp rx=0 tx=0).stop 2024-03-29T08:02:38.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:38.361+0000 7f3c72200700 1 -- 172.21.15.37:0/3052116444 shutdown_connections 2024-03-29T08:02:38.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:38.361+0000 7f3c72200700 1 --2- 172.21.15.37:0/3052116444 >> [v2:172.21.15.37:6802/3851364162,v1:172.21.15.37:6803/3851364162] conn(0x7f3c5006ef80 0x7f3c50071390 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:02:38.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:38.361+0000 7f3c72200700 1 --2- 172.21.15.37:0/3052116444 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f3c500692b0 0x7f3c5006b760 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:02:38.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:38.361+0000 7f3c72200700 1 --2- 172.21.15.37:0/3052116444 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3c6c070300 0x7f3c6c115c90 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:02:38.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:38.361+0000 7f3c72200700 1 --2- 172.21.15.37:0/3052116444 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3c6c070c10 0x7f3c6c1161d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:02:38.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:38.361+0000 7f3c72200700 1 -- 172.21.15.37:0/3052116444 >> 172.21.15.37:0/3052116444 conn(0x7f3c6c0f8970 msgr2=0x7f3c6c0f9ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:02:38.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:38.361+0000 7f3c72200700 1 -- 172.21.15.37:0/3052116444 shutdown_connections 2024-03-29T08:02:38.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:38.361+0000 7f3c72200700 1 -- 172.21.15.37:0/3052116444 wait complete. 2024-03-29T08:02:38.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:02:38 smithi037 ceph-mon[35162]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:02:38.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:38.967+0000 7f3736800700 1 -- 172.21.15.37:0/39315553 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3730102e00 msgr2=0x7f37301031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:02:38.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:38.967+0000 7f3736800700 1 --2- 172.21.15.37:0/39315553 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3730102e00 0x7f37301031d0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f3720004940 tx=0x7f3720009e30 comp rx=0 tx=0).stop 2024-03-29T08:02:38.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:38.967+0000 7f3736800700 1 -- 172.21.15.37:0/39315553 shutdown_connections 2024-03-29T08:02:38.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:38.967+0000 7f3736800700 1 --2- 172.21.15.37:0/39315553 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f37301037a0 0x7f37300fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:02:38.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:38.967+0000 7f3736800700 1 --2- 172.21.15.37:0/39315553 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3730102e00 0x7f37301031d0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:02:38.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:38.967+0000 7f3736800700 1 -- 172.21.15.37:0/39315553 >> 172.21.15.37:0/39315553 conn(0x7f37300f89b0 msgr2=0x7f37300fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:02:38.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:38.967+0000 7f3736800700 1 -- 172.21.15.37:0/39315553 shutdown_connections 2024-03-29T08:02:38.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:38.967+0000 7f3736800700 1 -- 172.21.15.37:0/39315553 wait complete. 2024-03-29T08:02:38.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:38.968+0000 7f3736800700 1 Processor -- start 2024-03-29T08:02:38.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:38.968+0000 7f3736800700 1 -- start start 2024-03-29T08:02:38.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:38.969+0000 7f3736800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3730102e00 0x7f3730115d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:02:38.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:38.969+0000 7f3736800700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f37301037a0 0x7f3730116250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:02:38.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:38.969+0000 7f3736800700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37301168a0 con 0x7f3730102e00 2024-03-29T08:02:38.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:38.969+0000 7f3736800700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f373010fc70 con 0x7f37301037a0 2024-03-29T08:02:38.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:38.969+0000 7f372f400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3730102e00 0x7f3730115d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:02:38.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:38.969+0000 7f372f400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3730102e00 0x7f3730115d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37674/0 (socket says 172.21.15.37:37674) 2024-03-29T08:02:38.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:38.969+0000 7f372f400700 1 -- 172.21.15.37:0/3394377649 learned_addr learned my addr 172.21.15.37:0/3394377649 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:02:38.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:38.969+0000 7f372f400700 1 -- 172.21.15.37:0/3394377649 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f37301037a0 msgr2=0x7f3730116250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:02:38.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:38.969+0000 7f372ea00700 1 --2- 172.21.15.37:0/3394377649 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f37301037a0 0x7f3730116250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:02:38.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:38.969+0000 7f372f400700 1 --2- 172.21.15.37:0/3394377649 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f37301037a0 0x7f3730116250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:02:38.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:38.969+0000 7f372f400700 1 -- 172.21.15.37:0/3394377649 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3720005040 con 0x7f3730102e00 2024-03-29T08:02:38.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:38.969+0000 7f372ea00700 1 --2- 172.21.15.37:0/3394377649 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f37301037a0 0x7f3730116250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:02:38.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:38.969+0000 7f372f400700 1 --2- 172.21.15.37:0/3394377649 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3730102e00 0x7f3730115d10 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f3720009180 tx=0x7f3720009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:02:38.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:38.970+0000 7f3727400700 1 -- 172.21.15.37:0/3394377649 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f372000e040 con 0x7f3730102e00 2024-03-29T08:02:38.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:38.970+0000 7f3727400700 1 -- 172.21.15.37:0/3394377649 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f3720004290 con 0x7f3730102e00 2024-03-29T08:02:38.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:38.970+0000 7f3736800700 1 -- 172.21.15.37:0/3394377649 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f373010fef0 con 0x7f3730102e00 2024-03-29T08:02:38.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:38.970+0000 7f3727400700 1 -- 172.21.15.37:0/3394377649 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f372001c3f0 con 0x7f3730102e00 2024-03-29T08:02:38.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:38.970+0000 7f3736800700 1 -- 172.21.15.37:0/3394377649 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37301104b0 con 0x7f3730102e00 2024-03-29T08:02:38.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:38.971+0000 7f3736800700 1 -- 172.21.15.37:0/3394377649 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f3730110aa0 con 0x7f3730102e00 2024-03-29T08:02:38.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:38.972+0000 7f3727400700 1 -- 172.21.15.37:0/3394377649 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f3720013790 con 0x7f3730102e00 2024-03-29T08:02:38.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:38.973+0000 7f3727400700 1 --2- 172.21.15.37:0/3394377649 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f37140692a0 0x7f371406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:02:38.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:38.973+0000 7f3727400700 1 -- 172.21.15.37:0/3394377649 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f37200880c0 con 0x7f3730102e00 2024-03-29T08:02:38.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:38.973+0000 7f372ea00700 1 --2- 172.21.15.37:0/3394377649 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f37140692a0 0x7f371406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:02:38.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:38.973+0000 7f3727400700 1 --2- 172.21.15.37:0/3394377649 >> [v2:172.21.15.37:6810/3949957441,v1:172.21.15.37:6811/3949957441] conn(0x7f371406ef70 0x7f3714071380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:02:38.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:38.973+0000 7f3727400700 1 -- 172.21.15.37:0/3394377649 --> [v2:172.21.15.37:6810/3949957441,v1:172.21.15.37:6811/3949957441] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3714071a30 con 0x7f371406ef70 2024-03-29T08:02:38.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:38.973+0000 7f3727400700 1 -- 172.21.15.37:0/3394377649 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f372008a610 con 0x7f3730102e00 2024-03-29T08:02:38.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:38.973+0000 7f372fe00700 1 --2- 172.21.15.37:0/3394377649 >> [v2:172.21.15.37:6810/3949957441,v1:172.21.15.37:6811/3949957441] conn(0x7f371406ef70 0x7f3714071380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:02:38.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:38.974+0000 7f372ea00700 1 --2- 172.21.15.37:0/3394377649 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f37140692a0 0x7f371406b750 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f371c007fd0 tx=0x7f371c009040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:02:38.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:38.974+0000 7f372fe00700 1 --2- 172.21.15.37:0/3394377649 >> [v2:172.21.15.37:6810/3949957441,v1:172.21.15.37:6811/3949957441] conn(0x7f371406ef70 0x7f3714071380 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:02:38.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:38.975+0000 7f3727400700 1 -- 172.21.15.37:0/3394377649 <== osd.1 v2:172.21.15.37:6810/3949957441 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f3714071a30 con 0x7f371406ef70 2024-03-29T08:02:38.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:38.991+0000 7f3736800700 1 -- 172.21.15.37:0/3394377649 --> [v2:172.21.15.37:6810/3949957441,v1:172.21.15.37:6811/3949957441] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f3730110080 con 0x7f371406ef70 2024-03-29T08:02:38.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:38.991+0000 7f3727400700 1 -- 172.21.15.37:0/3394377649 <== osd.1 v2:172.21.15.37:6810/3949957441 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f3730110080 con 0x7f371406ef70 2024-03-29T08:02:38.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:38.991+0000 7f3736800700 1 -- 172.21.15.37:0/3394377649 >> [v2:172.21.15.37:6810/3949957441,v1:172.21.15.37:6811/3949957441] conn(0x7f371406ef70 msgr2=0x7f3714071380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:02:38.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:38.991+0000 7f3736800700 1 --2- 172.21.15.37:0/3394377649 >> [v2:172.21.15.37:6810/3949957441,v1:172.21.15.37:6811/3949957441] conn(0x7f371406ef70 0x7f3714071380 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:02:38.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:38.992+0000 7f3736800700 1 -- 172.21.15.37:0/3394377649 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f37140692a0 msgr2=0x7f371406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:02:38.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:38.992+0000 7f3736800700 1 --2- 172.21.15.37:0/3394377649 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f37140692a0 0x7f371406b750 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f371c007fd0 tx=0x7f371c009040 comp rx=0 tx=0).stop 2024-03-29T08:02:38.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:38.992+0000 7f3736800700 1 -- 172.21.15.37:0/3394377649 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3730102e00 msgr2=0x7f3730115d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:02:38.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:38.992+0000 7f3736800700 1 --2- 172.21.15.37:0/3394377649 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3730102e00 0x7f3730115d10 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f3720009180 tx=0x7f3720009e30 comp rx=0 tx=0).stop 2024-03-29T08:02:38.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:38.992+0000 7f3736800700 1 -- 172.21.15.37:0/3394377649 shutdown_connections 2024-03-29T08:02:38.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:38.992+0000 7f3736800700 1 --2- 172.21.15.37:0/3394377649 >> [v2:172.21.15.37:6810/3949957441,v1:172.21.15.37:6811/3949957441] conn(0x7f371406ef70 0x7f3714071380 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:02:38.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:38.992+0000 7f3736800700 1 --2- 172.21.15.37:0/3394377649 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f37140692a0 0x7f371406b750 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:02:38.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:38.992+0000 7f3736800700 1 --2- 172.21.15.37:0/3394377649 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3730102e00 0x7f3730115d10 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:02:38.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:38.992+0000 7f3736800700 1 --2- 172.21.15.37:0/3394377649 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f37301037a0 0x7f3730116250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:02:38.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:38.992+0000 7f3736800700 1 -- 172.21.15.37:0/3394377649 >> 172.21.15.37:0/3394377649 conn(0x7f37300f89b0 msgr2=0x7f37300704f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:02:38.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:38.992+0000 7f3736800700 1 -- 172.21.15.37:0/3394377649 shutdown_connections 2024-03-29T08:02:38.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:38.992+0000 7f3736800700 1 -- 172.21.15.37:0/3394377649 wait complete. 2024-03-29T08:02:39.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:02:38 smithi148 ceph-mon[44992]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:02:39.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:39.587+0000 7f324ba00700 1 -- 172.21.15.37:0/270416991 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3244102e00 msgr2=0x7f32441031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:02:39.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:39.587+0000 7f324ba00700 1 --2- 172.21.15.37:0/270416991 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3244102e00 0x7f32441031d0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f3238004940 tx=0x7f3238009e30 comp rx=0 tx=0).stop 2024-03-29T08:02:39.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:39.587+0000 7f324ba00700 1 -- 172.21.15.37:0/270416991 shutdown_connections 2024-03-29T08:02:39.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:39.587+0000 7f324ba00700 1 --2- 172.21.15.37:0/270416991 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f32441037a0 0x7f32440fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:02:39.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:39.587+0000 7f324ba00700 1 --2- 172.21.15.37:0/270416991 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3244102e00 0x7f32441031d0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:02:39.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:39.587+0000 7f324ba00700 1 -- 172.21.15.37:0/270416991 >> 172.21.15.37:0/270416991 conn(0x7f32440f89b0 msgr2=0x7f32440fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:02:39.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:39.587+0000 7f324ba00700 1 -- 172.21.15.37:0/270416991 shutdown_connections 2024-03-29T08:02:39.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:39.587+0000 7f324ba00700 1 -- 172.21.15.37:0/270416991 wait complete. 2024-03-29T08:02:39.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:39.588+0000 7f324ba00700 1 Processor -- start 2024-03-29T08:02:39.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:39.588+0000 7f324ba00700 1 -- start start 2024-03-29T08:02:39.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:39.588+0000 7f324ba00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3244102e00 0x7f3244115cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:02:39.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:39.588+0000 7f324ba00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f32441037a0 0x7f3244116230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:02:39.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:39.588+0000 7f324ba00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3244116880 con 0x7f32441037a0 2024-03-29T08:02:39.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:39.588+0000 7f324ba00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f324410fc50 con 0x7f3244102e00 2024-03-29T08:02:39.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:39.588+0000 7f3243e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f32441037a0 0x7f3244116230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:02:39.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:39.588+0000 7f3243e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f32441037a0 0x7f3244116230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37696/0 (socket says 172.21.15.37:37696) 2024-03-29T08:02:39.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:39.588+0000 7f3243e00700 1 -- 172.21.15.37:0/3976129587 learned_addr learned my addr 172.21.15.37:0/3976129587 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:02:39.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:39.589+0000 7f3249000700 1 --2- 172.21.15.37:0/3976129587 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3244102e00 0x7f3244115cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:02:39.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:39.589+0000 7f3243e00700 1 -- 172.21.15.37:0/3976129587 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3244102e00 msgr2=0x7f3244115cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:02:39.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:39.589+0000 7f3243e00700 1 --2- 172.21.15.37:0/3976129587 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3244102e00 0x7f3244115cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:02:39.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:39.589+0000 7f3243e00700 1 -- 172.21.15.37:0/3976129587 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3238005040 con 0x7f32441037a0 2024-03-29T08:02:39.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:39.589+0000 7f3243e00700 1 --2- 172.21.15.37:0/3976129587 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f32441037a0 0x7f3244116230 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f3230002a50 tx=0x7f3230004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:02:39.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:39.589+0000 7f3240c00700 1 -- 172.21.15.37:0/3976129587 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3230007cc0 con 0x7f32441037a0 2024-03-29T08:02:39.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:39.589+0000 7f324ba00700 1 -- 172.21.15.37:0/3976129587 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f324410ff30 con 0x7f32441037a0 2024-03-29T08:02:39.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:39.589+0000 7f3240c00700 1 -- 172.21.15.37:0/3976129587 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f323000f040 con 0x7f32441037a0 2024-03-29T08:02:39.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:39.589+0000 7f3240c00700 1 -- 172.21.15.37:0/3976129587 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f323000d920 con 0x7f32441037a0 2024-03-29T08:02:39.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:39.590+0000 7f324ba00700 1 -- 172.21.15.37:0/3976129587 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32441104f0 con 0x7f32441037a0 2024-03-29T08:02:39.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:39.590+0000 7f324ba00700 1 -- 172.21.15.37:0/3976129587 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f3244110a20 con 0x7f32441037a0 2024-03-29T08:02:39.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:39.592+0000 7f3240c00700 1 -- 172.21.15.37:0/3976129587 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f3230009680 con 0x7f32441037a0 2024-03-29T08:02:39.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:39.593+0000 7f3240c00700 1 --2- 172.21.15.37:0/3976129587 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f322c0692b0 0x7f322c06b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:02:39.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:39.593+0000 7f3240c00700 1 -- 172.21.15.37:0/3976129587 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f3230086ba0 con 0x7f32441037a0 2024-03-29T08:02:39.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:39.593+0000 7f3249000700 1 --2- 172.21.15.37:0/3976129587 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f322c0692b0 0x7f322c06b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:02:39.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:39.593+0000 7f3240c00700 1 --2- 172.21.15.37:0/3976129587 >> [v2:172.21.15.148:6808/4113691443,v1:172.21.15.148:6809/4113691443] conn(0x7f322c06ef80 0x7f322c071390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:02:39.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:39.593+0000 7f3240c00700 1 -- 172.21.15.37:0/3976129587 --> [v2:172.21.15.148:6808/4113691443,v1:172.21.15.148:6809/4113691443] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f322c071a40 con 0x7f322c06ef80 2024-03-29T08:02:39.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:39.593+0000 7f3240c00700 1 -- 172.21.15.37:0/3976129587 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f323008b220 con 0x7f32441037a0 2024-03-29T08:02:39.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:39.594+0000 7f3249000700 1 --2- 172.21.15.37:0/3976129587 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f322c0692b0 0x7f322c06b760 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f323800b840 tx=0x7f3238009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:02:39.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:39.594+0000 7f3249a00700 1 --2- 172.21.15.37:0/3976129587 >> [v2:172.21.15.148:6808/4113691443,v1:172.21.15.148:6809/4113691443] conn(0x7f322c06ef80 0x7f322c071390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:02:39.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:39.595+0000 7f3249a00700 1 --2- 172.21.15.37:0/3976129587 >> [v2:172.21.15.148:6808/4113691443,v1:172.21.15.148:6809/4113691443] conn(0x7f322c06ef80 0x7f322c071390 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:02:39.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:39.596+0000 7f3240c00700 1 -- 172.21.15.37:0/3976129587 <== osd.4 v2:172.21.15.148:6808/4113691443 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f322c071a40 con 0x7f322c06ef80 2024-03-29T08:02:39.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:39.612+0000 7f324ba00700 1 -- 172.21.15.37:0/3976129587 --> [v2:172.21.15.148:6808/4113691443,v1:172.21.15.148:6809/4113691443] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f3244071020 con 0x7f322c06ef80 2024-03-29T08:02:39.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:39.613+0000 7f3240c00700 1 -- 172.21.15.37:0/3976129587 <== osd.4 v2:172.21.15.148:6808/4113691443 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f3244071020 con 0x7f322c06ef80 2024-03-29T08:02:39.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:39.614+0000 7f324ba00700 1 -- 172.21.15.37:0/3976129587 >> [v2:172.21.15.148:6808/4113691443,v1:172.21.15.148:6809/4113691443] conn(0x7f322c06ef80 msgr2=0x7f322c071390 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:02:39.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:39.614+0000 7f324ba00700 1 --2- 172.21.15.37:0/3976129587 >> [v2:172.21.15.148:6808/4113691443,v1:172.21.15.148:6809/4113691443] conn(0x7f322c06ef80 0x7f322c071390 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:02:39.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:39.614+0000 7f324ba00700 1 -- 172.21.15.37:0/3976129587 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f322c0692b0 msgr2=0x7f322c06b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:02:39.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:39.614+0000 7f324ba00700 1 --2- 172.21.15.37:0/3976129587 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f322c0692b0 0x7f322c06b760 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f323800b840 tx=0x7f3238009e30 comp rx=0 tx=0).stop 2024-03-29T08:02:39.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:39.614+0000 7f324ba00700 1 -- 172.21.15.37:0/3976129587 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f32441037a0 msgr2=0x7f3244116230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:02:39.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:39.614+0000 7f324ba00700 1 --2- 172.21.15.37:0/3976129587 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f32441037a0 0x7f3244116230 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f3230002a50 tx=0x7f3230004600 comp rx=0 tx=0).stop 2024-03-29T08:02:39.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:39.614+0000 7f324ba00700 1 -- 172.21.15.37:0/3976129587 shutdown_connections 2024-03-29T08:02:39.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:39.614+0000 7f324ba00700 1 --2- 172.21.15.37:0/3976129587 >> [v2:172.21.15.148:6808/4113691443,v1:172.21.15.148:6809/4113691443] conn(0x7f322c06ef80 0x7f322c071390 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:02:39.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:39.614+0000 7f324ba00700 1 --2- 172.21.15.37:0/3976129587 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f322c0692b0 0x7f322c06b760 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:02:39.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:39.614+0000 7f324ba00700 1 --2- 172.21.15.37:0/3976129587 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3244102e00 0x7f3244115cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:02:39.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:39.614+0000 7f324ba00700 1 --2- 172.21.15.37:0/3976129587 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f32441037a0 0x7f3244116230 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:02:39.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:39.614+0000 7f324ba00700 1 -- 172.21.15.37:0/3976129587 >> 172.21.15.37:0/3976129587 conn(0x7f32440f89b0 msgr2=0x7f32440706d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:02:39.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:39.614+0000 7f324ba00700 1 -- 172.21.15.37:0/3976129587 shutdown_connections 2024-03-29T08:02:39.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:39.614+0000 7f324ba00700 1 -- 172.21.15.37:0/3976129587 wait complete. 2024-03-29T08:02:40.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:40.264+0000 7f965b000700 1 -- 172.21.15.37:0/2031142062 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f96541037a0 msgr2=0x7f96540fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:02:40.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:40.264+0000 7f965b000700 1 --2- 172.21.15.37:0/2031142062 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f96541037a0 0x7f96540fce10 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f9640004940 tx=0x7f9640009e30 comp rx=0 tx=0).stop 2024-03-29T08:02:40.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:40.265+0000 7f965b000700 1 -- 172.21.15.37:0/2031142062 shutdown_connections 2024-03-29T08:02:40.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:40.265+0000 7f965b000700 1 --2- 172.21.15.37:0/2031142062 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f96541037a0 0x7f96540fce10 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:02:40.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:40.265+0000 7f965b000700 1 --2- 172.21.15.37:0/2031142062 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9654102e00 0x7f96541031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:02:40.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:40.265+0000 7f965b000700 1 -- 172.21.15.37:0/2031142062 >> 172.21.15.37:0/2031142062 conn(0x7f96540f89b0 msgr2=0x7f96540fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:02:40.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:40.265+0000 7f965b000700 1 -- 172.21.15.37:0/2031142062 shutdown_connections 2024-03-29T08:02:40.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:40.265+0000 7f965b000700 1 -- 172.21.15.37:0/2031142062 wait complete. 2024-03-29T08:02:40.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:40.265+0000 7f965b000700 1 Processor -- start 2024-03-29T08:02:40.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:40.266+0000 7f965b000700 1 -- start start 2024-03-29T08:02:40.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:40.266+0000 7f965b000700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9654102e00 0x7f9654115d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:02:40.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:40.266+0000 7f965b000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f96541037a0 0x7f9654116240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:02:40.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:40.266+0000 7f965b000700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9654116890 con 0x7f96541037a0 2024-03-29T08:02:40.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:40.266+0000 7f965b000700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f965410fc60 con 0x7f9654102e00 2024-03-29T08:02:40.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:40.266+0000 7f9653400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f96541037a0 0x7f9654116240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:02:40.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:40.266+0000 7f9653400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f96541037a0 0x7f9654116240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60944/0 (socket says 172.21.15.37:60944) 2024-03-29T08:02:40.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:40.266+0000 7f9653400700 1 -- 172.21.15.37:0/1964041718 learned_addr learned my addr 172.21.15.37:0/1964041718 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:02:40.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:40.267+0000 7f9653400700 1 -- 172.21.15.37:0/1964041718 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9654102e00 msgr2=0x7f9654115d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:02:40.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:40.267+0000 7f9653400700 1 --2- 172.21.15.37:0/1964041718 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9654102e00 0x7f9654115d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:02:40.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:40.267+0000 7f9653400700 1 -- 172.21.15.37:0/1964041718 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9640005040 con 0x7f96541037a0 2024-03-29T08:02:40.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:40.267+0000 7f9653400700 1 --2- 172.21.15.37:0/1964041718 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f96541037a0 0x7f9654116240 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f9640009390 tx=0x7f9640009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:02:40.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:40.267+0000 7f964be00700 1 -- 172.21.15.37:0/1964041718 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f964000e040 con 0x7f96541037a0 2024-03-29T08:02:40.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:40.267+0000 7f964be00700 1 -- 172.21.15.37:0/1964041718 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f9640013420 con 0x7f96541037a0 2024-03-29T08:02:40.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:40.267+0000 7f965b000700 1 -- 172.21.15.37:0/1964041718 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f965410fee0 con 0x7f96541037a0 2024-03-29T08:02:40.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:40.267+0000 7f964be00700 1 -- 172.21.15.37:0/1964041718 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f964001c630 con 0x7f96541037a0 2024-03-29T08:02:40.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:40.267+0000 7f965b000700 1 -- 172.21.15.37:0/1964041718 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96541104a0 con 0x7f96541037a0 2024-03-29T08:02:40.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:40.268+0000 7f965b000700 1 -- 172.21.15.37:0/1964041718 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f9654110a90 con 0x7f96541037a0 2024-03-29T08:02:40.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:40.269+0000 7f964be00700 1 -- 172.21.15.37:0/1964041718 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f96400139e0 con 0x7f96541037a0 2024-03-29T08:02:40.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:40.270+0000 7f964be00700 1 --2- 172.21.15.37:0/1964041718 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f96380692b0 0x7f963806b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:02:40.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:40.270+0000 7f964be00700 1 -- 172.21.15.37:0/1964041718 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f96400880e0 con 0x7f96541037a0 2024-03-29T08:02:40.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:40.270+0000 7f9653e00700 1 --2- 172.21.15.37:0/1964041718 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f96380692b0 0x7f963806b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:02:40.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:40.270+0000 7f964be00700 1 --2- 172.21.15.37:0/1964041718 >> [v2:172.21.15.148:6816/267287812,v1:172.21.15.148:6817/267287812] conn(0x7f963806ef80 0x7f9638071390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:02:40.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:40.270+0000 7f964be00700 1 -- 172.21.15.37:0/1964041718 --> [v2:172.21.15.148:6816/267287812,v1:172.21.15.148:6817/267287812] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9638071a40 con 0x7f963806ef80 2024-03-29T08:02:40.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:40.270+0000 7f964be00700 1 -- 172.21.15.37:0/1964041718 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f964008a630 con 0x7f96541037a0 2024-03-29T08:02:40.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:40.271+0000 7f9659000700 1 --2- 172.21.15.37:0/1964041718 >> [v2:172.21.15.148:6816/267287812,v1:172.21.15.148:6817/267287812] conn(0x7f963806ef80 0x7f9638071390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:02:40.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:40.271+0000 7f9653e00700 1 --2- 172.21.15.37:0/1964041718 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f96380692b0 0x7f963806b760 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f9644004d00 tx=0x7f9644009ce0 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:02:40.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:40.271+0000 7f9659000700 1 --2- 172.21.15.37:0/1964041718 >> [v2:172.21.15.148:6816/267287812,v1:172.21.15.148:6817/267287812] conn(0x7f963806ef80 0x7f9638071390 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:02:40.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:40.273+0000 7f964be00700 1 -- 172.21.15.37:0/1964041718 <== osd.5 v2:172.21.15.148:6816/267287812 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f9638071a40 con 0x7f963806ef80 2024-03-29T08:02:40.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:40.289+0000 7f965b000700 1 -- 172.21.15.37:0/1964041718 --> [v2:172.21.15.148:6816/267287812,v1:172.21.15.148:6817/267287812] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f96540fa400 con 0x7f963806ef80 2024-03-29T08:02:40.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:40.289+0000 7f964be00700 1 -- 172.21.15.37:0/1964041718 <== osd.5 v2:172.21.15.148:6816/267287812 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f96540fa400 con 0x7f963806ef80 2024-03-29T08:02:40.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:40.290+0000 7f965b000700 1 -- 172.21.15.37:0/1964041718 >> [v2:172.21.15.148:6816/267287812,v1:172.21.15.148:6817/267287812] conn(0x7f963806ef80 msgr2=0x7f9638071390 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:02:40.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:40.290+0000 7f965b000700 1 --2- 172.21.15.37:0/1964041718 >> [v2:172.21.15.148:6816/267287812,v1:172.21.15.148:6817/267287812] conn(0x7f963806ef80 0x7f9638071390 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:02:40.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:40.290+0000 7f965b000700 1 -- 172.21.15.37:0/1964041718 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f96380692b0 msgr2=0x7f963806b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:02:40.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:40.290+0000 7f965b000700 1 --2- 172.21.15.37:0/1964041718 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f96380692b0 0x7f963806b760 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f9644004d00 tx=0x7f9644009ce0 comp rx=0 tx=0).stop 2024-03-29T08:02:40.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:40.290+0000 7f965b000700 1 -- 172.21.15.37:0/1964041718 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f96541037a0 msgr2=0x7f9654116240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:02:40.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:40.290+0000 7f965b000700 1 --2- 172.21.15.37:0/1964041718 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f96541037a0 0x7f9654116240 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f9640009390 tx=0x7f9640009e30 comp rx=0 tx=0).stop 2024-03-29T08:02:40.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:40.290+0000 7f965b000700 1 -- 172.21.15.37:0/1964041718 shutdown_connections 2024-03-29T08:02:40.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:40.290+0000 7f965b000700 1 --2- 172.21.15.37:0/1964041718 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f96380692b0 0x7f963806b760 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:02:40.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:40.290+0000 7f965b000700 1 --2- 172.21.15.37:0/1964041718 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9654102e00 0x7f9654115d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:02:40.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:40.290+0000 7f965b000700 1 --2- 172.21.15.37:0/1964041718 >> [v2:172.21.15.148:6816/267287812,v1:172.21.15.148:6817/267287812] conn(0x7f963806ef80 0x7f9638071390 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:02:40.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:40.290+0000 7f965b000700 1 --2- 172.21.15.37:0/1964041718 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f96541037a0 0x7f9654116240 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:02:40.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:40.290+0000 7f965b000700 1 -- 172.21.15.37:0/1964041718 >> 172.21.15.37:0/1964041718 conn(0x7f96540f89b0 msgr2=0x7f96540f9ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:02:40.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:40.290+0000 7f965b000700 1 -- 172.21.15.37:0/1964041718 shutdown_connections 2024-03-29T08:02:40.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:40.290+0000 7f965b000700 1 -- 172.21.15.37:0/1964041718 wait complete. 2024-03-29T08:02:40.608 INFO:teuthology.orchestra.run.smithi037.stdout:38654705705 2024-03-29T08:02:40.609 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 99b492c2-eda1-11ee-b647-cb9ed24678a4 -- ceph osd last-stat-seq osd.0 2024-03-29T08:02:40.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:02:40 smithi037 ceph-mon[35162]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:02:40.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:02:40 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:02:41.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:02:40 smithi148 ceph-mon[44992]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:02:41.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:02:40 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:02:41.289 INFO:teuthology.orchestra.run.smithi037.stdout:150323855376 2024-03-29T08:02:41.290 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 99b492c2-eda1-11ee-b647-cb9ed24678a4 -- ceph osd last-stat-seq osd.4 2024-03-29T08:02:41.918 INFO:teuthology.orchestra.run.smithi037.stdout:60129542178 2024-03-29T08:02:41.918 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 99b492c2-eda1-11ee-b647-cb9ed24678a4 -- ceph osd last-stat-seq osd.1 2024-03-29T08:02:42.430 INFO:teuthology.orchestra.run.smithi037.stdout:171798691848 2024-03-29T08:02:42.430 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 99b492c2-eda1-11ee-b647-cb9ed24678a4 -- ceph osd last-stat-seq osd.5 2024-03-29T08:02:42.674 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:02:42 smithi037 ceph-mon[35162]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:02:42.926 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/mon.smithi037/config 2024-03-29T08:02:42.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:42.991+0000 7f0f91400700 1 -- 172.21.15.37:0/3595069287 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0f8c1059b0 msgr2=0x7f0f8c0ff020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:02:42.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:42.991+0000 7f0f91400700 1 --2- 172.21.15.37:0/3595069287 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0f8c1059b0 0x7f0f8c0ff020 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f0f7c004c20 tx=0x7f0f7c009ca0 comp rx=0 tx=0).stop 2024-03-29T08:02:42.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:42.993+0000 7f0f91400700 1 -- 172.21.15.37:0/3595069287 shutdown_connections 2024-03-29T08:02:42.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:42.993+0000 7f0f91400700 1 --2- 172.21.15.37:0/3595069287 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0f8c1059b0 0x7f0f8c0ff020 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:02:42.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:42.993+0000 7f0f91400700 1 --2- 172.21.15.37:0/3595069287 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0f8c105010 0x7f0f8c1053e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:02:42.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:42.993+0000 7f0f91400700 1 -- 172.21.15.37:0/3595069287 >> 172.21.15.37:0/3595069287 conn(0x7f0f8c0faba0 msgr2=0x7f0f8c0fcfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:02:42.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:42.993+0000 7f0f91400700 1 -- 172.21.15.37:0/3595069287 shutdown_connections 2024-03-29T08:02:42.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:42.993+0000 7f0f91400700 1 -- 172.21.15.37:0/3595069287 wait complete. 2024-03-29T08:02:42.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:42.994+0000 7f0f91400700 1 Processor -- start 2024-03-29T08:02:42.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:42.994+0000 7f0f91400700 1 -- start start 2024-03-29T08:02:42.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:42.995+0000 7f0f91400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0f8c105010 0x7f0f8c11a140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:02:42.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:42.995+0000 7f0f91400700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0f8c1059b0 0x7f0f8c11a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:02:42.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:42.995+0000 7f0f91400700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f8c11acd0 con 0x7f0f8c105010 2024-03-29T08:02:42.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:42.995+0000 7f0f91400700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f8c1140a0 con 0x7f0f8c1059b0 2024-03-29T08:02:42.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:42.996+0000 7f0f8a800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0f8c105010 0x7f0f8c11a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:02:42.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:42.996+0000 7f0f8a800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0f8c105010 0x7f0f8c11a140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60962/0 (socket says 172.21.15.37:60962) 2024-03-29T08:02:42.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:42.996+0000 7f0f8a800700 1 -- 172.21.15.37:0/3715721008 learned_addr learned my addr 172.21.15.37:0/3715721008 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:02:42.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:42.996+0000 7f0f83e00700 1 --2- 172.21.15.37:0/3715721008 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0f8c1059b0 0x7f0f8c11a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:02:42.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:42.996+0000 7f0f8a800700 1 -- 172.21.15.37:0/3715721008 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0f8c1059b0 msgr2=0x7f0f8c11a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:02:42.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:42.996+0000 7f0f8a800700 1 --2- 172.21.15.37:0/3715721008 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0f8c1059b0 0x7f0f8c11a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:02:42.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:42.996+0000 7f0f8a800700 1 -- 172.21.15.37:0/3715721008 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f7c0048f0 con 0x7f0f8c105010 2024-03-29T08:02:42.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:42.996+0000 7f0f83e00700 1 --2- 172.21.15.37:0/3715721008 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0f8c1059b0 0x7f0f8c11a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T08:02:42.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:42.997+0000 7f0f8a800700 1 --2- 172.21.15.37:0/3715721008 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0f8c105010 0x7f0f8c11a140 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f0f740041d0 tx=0x7f0f7400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:02:42.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:42.997+0000 7f0f82a00700 1 -- 172.21.15.37:0/3715721008 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0f74007cc0 con 0x7f0f8c105010 2024-03-29T08:02:42.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:42.997+0000 7f0f82a00700 1 -- 172.21.15.37:0/3715721008 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f0f7400f040 con 0x7f0f8c105010 2024-03-29T08:02:42.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:42.997+0000 7f0f82a00700 1 -- 172.21.15.37:0/3715721008 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0f7400c990 con 0x7f0f8c105010 2024-03-29T08:02:42.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:42.998+0000 7f0f91400700 1 -- 172.21.15.37:0/3715721008 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f8c114320 con 0x7f0f8c105010 2024-03-29T08:02:42.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:42.998+0000 7f0f91400700 1 -- 172.21.15.37:0/3715721008 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f8c1148b0 con 0x7f0f8c105010 2024-03-29T08:02:42.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:42.999+0000 7f0f91400700 1 -- 172.21.15.37:0/3715721008 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f0f8c114de0 con 0x7f0f8c105010 2024-03-29T08:02:42.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:43.000+0000 7f0f82a00700 1 -- 172.21.15.37:0/3715721008 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f0f7400caf0 con 0x7f0f8c105010 2024-03-29T08:02:43.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:43.001+0000 7f0f82a00700 1 --2- 172.21.15.37:0/3715721008 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f0f70069300 0x7f0f7006b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:02:43.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:43.001+0000 7f0f82a00700 1 -- 172.21.15.37:0/3715721008 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f0f74086880 con 0x7f0f8c105010 2024-03-29T08:02:43.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:43.002+0000 7f0f83e00700 1 --2- 172.21.15.37:0/3715721008 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f0f70069300 0x7f0f7006b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:02:43.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:43.002+0000 7f0f82a00700 1 --2- 172.21.15.37:0/3715721008 >> [v2:172.21.15.148:6800/267019749,v1:172.21.15.148:6801/267019749] conn(0x7f0f7006efd0 0x7f0f700713e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:02:43.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:43.002+0000 7f0f82a00700 1 -- 172.21.15.37:0/3715721008 --> [v2:172.21.15.148:6800/267019749,v1:172.21.15.148:6801/267019749] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0f70071a90 con 0x7f0f7006efd0 2024-03-29T08:02:43.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:43.002+0000 7f0f82a00700 1 -- 172.21.15.37:0/3715721008 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0f74088dd0 con 0x7f0f8c105010 2024-03-29T08:02:43.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:43.003+0000 7f0f8b200700 1 --2- 172.21.15.37:0/3715721008 >> [v2:172.21.15.148:6800/267019749,v1:172.21.15.148:6801/267019749] conn(0x7f0f7006efd0 0x7f0f700713e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:02:43.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:43.003+0000 7f0f83e00700 1 --2- 172.21.15.37:0/3715721008 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f0f70069300 0x7f0f7006b7b0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f0f7c000f80 tx=0x7f0f7c009ca0 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:02:43.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:43.003+0000 7f0f8b200700 1 --2- 172.21.15.37:0/3715721008 >> [v2:172.21.15.148:6800/267019749,v1:172.21.15.148:6801/267019749] conn(0x7f0f7006efd0 0x7f0f700713e0 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:02:43.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:43.005+0000 7f0f82a00700 1 -- 172.21.15.37:0/3715721008 <== osd.3 v2:172.21.15.148:6800/267019749 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f0f70071a90 con 0x7f0f7006efd0 2024-03-29T08:02:43.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:43.032+0000 7f0f91400700 1 -- 172.21.15.37:0/3715721008 --> [v2:172.21.15.148:6800/267019749,v1:172.21.15.148:6801/267019749] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f0f8c100760 con 0x7f0f7006efd0 2024-03-29T08:02:43.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:43.033+0000 7f0f82a00700 1 -- 172.21.15.37:0/3715721008 <== osd.3 v2:172.21.15.148:6800/267019749 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f0f8c100760 con 0x7f0f7006efd0 2024-03-29T08:02:43.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:43.033+0000 7f0f91400700 1 -- 172.21.15.37:0/3715721008 >> [v2:172.21.15.148:6800/267019749,v1:172.21.15.148:6801/267019749] conn(0x7f0f7006efd0 msgr2=0x7f0f700713e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:02:43.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:43.033+0000 7f0f91400700 1 --2- 172.21.15.37:0/3715721008 >> [v2:172.21.15.148:6800/267019749,v1:172.21.15.148:6801/267019749] conn(0x7f0f7006efd0 0x7f0f700713e0 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:02:43.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:43.034+0000 7f0f91400700 1 -- 172.21.15.37:0/3715721008 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f0f70069300 msgr2=0x7f0f7006b7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:02:43.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:43.034+0000 7f0f91400700 1 --2- 172.21.15.37:0/3715721008 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f0f70069300 0x7f0f7006b7b0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f0f7c000f80 tx=0x7f0f7c009ca0 comp rx=0 tx=0).stop 2024-03-29T08:02:43.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:43.034+0000 7f0f91400700 1 -- 172.21.15.37:0/3715721008 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0f8c105010 msgr2=0x7f0f8c11a140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:02:43.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:43.034+0000 7f0f91400700 1 --2- 172.21.15.37:0/3715721008 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0f8c105010 0x7f0f8c11a140 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f0f740041d0 tx=0x7f0f7400b040 comp rx=0 tx=0).stop 2024-03-29T08:02:43.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:43.034+0000 7f0f91400700 1 -- 172.21.15.37:0/3715721008 shutdown_connections 2024-03-29T08:02:43.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:43.034+0000 7f0f91400700 1 --2- 172.21.15.37:0/3715721008 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f0f70069300 0x7f0f7006b7b0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:02:43.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:43.034+0000 7f0f91400700 1 --2- 172.21.15.37:0/3715721008 >> [v2:172.21.15.148:6800/267019749,v1:172.21.15.148:6801/267019749] conn(0x7f0f7006efd0 0x7f0f700713e0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:02:43.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:43.034+0000 7f0f91400700 1 --2- 172.21.15.37:0/3715721008 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0f8c105010 0x7f0f8c11a140 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:02:43.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:43.034+0000 7f0f91400700 1 --2- 172.21.15.37:0/3715721008 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0f8c1059b0 0x7f0f8c11a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:02:43.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:43.034+0000 7f0f91400700 1 -- 172.21.15.37:0/3715721008 >> 172.21.15.37:0/3715721008 conn(0x7f0f8c0faba0 msgr2=0x7f0f8c0fc130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:02:43.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:43.034+0000 7f0f91400700 1 -- 172.21.15.37:0/3715721008 shutdown_connections 2024-03-29T08:02:43.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:43.034+0000 7f0f91400700 1 -- 172.21.15.37:0/3715721008 wait complete. 2024-03-29T08:02:43.055 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:02:42 smithi148 ceph-mon[44992]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:02:43.466 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/mon.smithi037/config 2024-03-29T08:02:43.467 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/mon.smithi037/config 2024-03-29T08:02:43.476 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/mon.smithi037/config 2024-03-29T08:02:43.494 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/mon.smithi037/config 2024-03-29T08:02:44.650 INFO:teuthology.orchestra.run.smithi037.stdout:115964117014 2024-03-29T08:02:44.651 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 99b492c2-eda1-11ee-b647-cb9ed24678a4 -- ceph osd last-stat-seq osd.3 2024-03-29T08:02:44.907 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:02:44 smithi037 ceph-mon[35162]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:02:45.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:02:44 smithi148 ceph-mon[44992]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:02:46.840 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:02:46 smithi037 ceph-mon[35162]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:02:47.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:02:46 smithi148 ceph-mon[44992]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:02:48.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:48.177+0000 7f2d3ce00700 1 -- 172.21.15.37:0/2062276492 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2d38102e00 msgr2=0x7f2d381031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:02:48.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:48.177+0000 7f2d3ce00700 1 --2- 172.21.15.37:0/2062276492 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2d38102e00 0x7f2d381031d0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f2d28004940 tx=0x7f2d28009e30 comp rx=0 tx=0).stop 2024-03-29T08:02:48.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:48.177+0000 7f2d3ce00700 1 -- 172.21.15.37:0/2062276492 shutdown_connections 2024-03-29T08:02:48.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:48.177+0000 7f2d3ce00700 1 --2- 172.21.15.37:0/2062276492 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2d381037a0 0x7f2d380fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:02:48.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:48.177+0000 7f2d3ce00700 1 --2- 172.21.15.37:0/2062276492 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2d38102e00 0x7f2d381031d0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:02:48.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:48.177+0000 7f2d3ce00700 1 -- 172.21.15.37:0/2062276492 >> 172.21.15.37:0/2062276492 conn(0x7f2d380f89b0 msgr2=0x7f2d380fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:02:48.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:48.177+0000 7f2d3ce00700 1 -- 172.21.15.37:0/2062276492 shutdown_connections 2024-03-29T08:02:48.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:48.177+0000 7f2d3ce00700 1 -- 172.21.15.37:0/2062276492 wait complete. 2024-03-29T08:02:48.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:48.178+0000 7f2d3ce00700 1 Processor -- start 2024-03-29T08:02:48.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:48.178+0000 7f2d3ce00700 1 -- start start 2024-03-29T08:02:48.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:48.178+0000 7f2d3ce00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2d38102e00 0x7f2d38115d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:02:48.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:48.178+0000 7f2d3ce00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2d381037a0 0x7f2d38116250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:02:48.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:48.178+0000 7f2d3ce00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d381168a0 con 0x7f2d38102e00 2024-03-29T08:02:48.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:48.178+0000 7f2d3ce00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d3810fc70 con 0x7f2d381037a0 2024-03-29T08:02:48.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:48.179+0000 7f2d35e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2d38102e00 0x7f2d38115d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:02:48.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:48.179+0000 7f2d35e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2d38102e00 0x7f2d38115d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60988/0 (socket says 172.21.15.37:60988) 2024-03-29T08:02:48.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:48.179+0000 7f2d35e00700 1 -- 172.21.15.37:0/3563257192 learned_addr learned my addr 172.21.15.37:0/3563257192 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:02:48.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:48.179+0000 7f2d35400700 1 --2- 172.21.15.37:0/3563257192 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2d381037a0 0x7f2d38116250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:02:48.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:48.179+0000 7f2d35e00700 1 -- 172.21.15.37:0/3563257192 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2d381037a0 msgr2=0x7f2d38116250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:02:48.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:48.179+0000 7f2d35e00700 1 --2- 172.21.15.37:0/3563257192 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2d381037a0 0x7f2d38116250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:02:48.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:48.179+0000 7f2d35e00700 1 -- 172.21.15.37:0/3563257192 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d28005040 con 0x7f2d38102e00 2024-03-29T08:02:48.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:48.179+0000 7f2d35400700 1 --2- 172.21.15.37:0/3563257192 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2d381037a0 0x7f2d38116250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:02:48.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:48.179+0000 7f2d35e00700 1 --2- 172.21.15.37:0/3563257192 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2d38102e00 0x7f2d38115d10 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f2d28009390 tx=0x7f2d28009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:02:48.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:48.180+0000 7f2d2e000700 1 -- 172.21.15.37:0/3563257192 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2d2800e040 con 0x7f2d38102e00 2024-03-29T08:02:48.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:48.180+0000 7f2d2e000700 1 -- 172.21.15.37:0/3563257192 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f2d28013420 con 0x7f2d38102e00 2024-03-29T08:02:48.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:48.180+0000 7f2d3ce00700 1 -- 172.21.15.37:0/3563257192 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d3810fef0 con 0x7f2d38102e00 2024-03-29T08:02:48.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:48.180+0000 7f2d2e000700 1 -- 172.21.15.37:0/3563257192 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2d2801c630 con 0x7f2d38102e00 2024-03-29T08:02:48.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:48.180+0000 7f2d3ce00700 1 -- 172.21.15.37:0/3563257192 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d381104b0 con 0x7f2d38102e00 2024-03-29T08:02:48.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:48.181+0000 7f2d3ce00700 1 -- 172.21.15.37:0/3563257192 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d380fe550 con 0x7f2d38102e00 2024-03-29T08:02:48.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:48.182+0000 7f2d2e000700 1 -- 172.21.15.37:0/3563257192 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f2d280139e0 con 0x7f2d38102e00 2024-03-29T08:02:48.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:48.183+0000 7f2d2e000700 1 --2- 172.21.15.37:0/3563257192 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f2d1c0692b0 0x7f2d1c06b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:02:48.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:48.183+0000 7f2d2e000700 1 -- 172.21.15.37:0/3563257192 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f2d28087ff0 con 0x7f2d38102e00 2024-03-29T08:02:48.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:48.183+0000 7f2d35400700 1 --2- 172.21.15.37:0/3563257192 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f2d1c0692b0 0x7f2d1c06b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:02:48.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:48.183+0000 7f2d35400700 1 --2- 172.21.15.37:0/3563257192 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f2d1c0692b0 0x7f2d1c06b760 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f2d24009fd0 tx=0x7f2d2400b040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:02:48.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:48.188+0000 7f2d2e000700 1 -- 172.21.15.37:0/3563257192 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f2d280ba190 con 0x7f2d38102e00 2024-03-29T08:02:48.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:48.421+0000 7f2d3ce00700 1 -- 172.21.15.37:0/3563257192 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f2d3804bed0 con 0x7f2d38102e00 2024-03-29T08:02:48.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:48.422+0000 7f2d2e000700 1 -- 172.21.15.37:0/3563257192 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f2d28058180 con 0x7f2d38102e00 2024-03-29T08:02:48.421 INFO:teuthology.orchestra.run.smithi037.stdout:38654705707 2024-03-29T08:02:48.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:48.424+0000 7f2d3ce00700 1 -- 172.21.15.37:0/3563257192 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f2d1c0692b0 msgr2=0x7f2d1c06b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:02:48.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:48.424+0000 7f2d3ce00700 1 --2- 172.21.15.37:0/3563257192 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f2d1c0692b0 0x7f2d1c06b760 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f2d24009fd0 tx=0x7f2d2400b040 comp rx=0 tx=0).stop 2024-03-29T08:02:48.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:48.424+0000 7f2d3ce00700 1 -- 172.21.15.37:0/3563257192 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2d38102e00 msgr2=0x7f2d38115d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:02:48.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:48.424+0000 7f2d3ce00700 1 --2- 172.21.15.37:0/3563257192 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2d38102e00 0x7f2d38115d10 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f2d28009390 tx=0x7f2d28009e30 comp rx=0 tx=0).stop 2024-03-29T08:02:48.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:48.424+0000 7f2d3ce00700 1 -- 172.21.15.37:0/3563257192 shutdown_connections 2024-03-29T08:02:48.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:48.424+0000 7f2d3ce00700 1 --2- 172.21.15.37:0/3563257192 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f2d1c0692b0 0x7f2d1c06b760 secure :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f2d24009fd0 tx=0x7f2d2400b040 comp rx=0 tx=0).stop 2024-03-29T08:02:48.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:48.424+0000 7f2d3ce00700 1 --2- 172.21.15.37:0/3563257192 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2d38102e00 0x7f2d38115d10 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:02:48.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:48.424+0000 7f2d3ce00700 1 --2- 172.21.15.37:0/3563257192 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2d381037a0 0x7f2d38116250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:02:48.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:48.424+0000 7f2d3ce00700 1 -- 172.21.15.37:0/3563257192 >> 172.21.15.37:0/3563257192 conn(0x7f2d380f89b0 msgr2=0x7f2d380706d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:02:48.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:48.425+0000 7f2d3ce00700 1 -- 172.21.15.37:0/3563257192 shutdown_connections 2024-03-29T08:02:48.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:48.425+0000 7f2d3ce00700 1 -- 172.21.15.37:0/3563257192 wait complete. 2024-03-29T08:02:48.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:02:48 smithi037 ceph-mon[35162]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:02:48.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:02:48 smithi037 ceph-mon[35162]: from='client.? 172.21.15.37:0/3563257192' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-03-29T08:02:49.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:02:48 smithi148 ceph-mon[44992]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:02:49.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:02:48 smithi148 ceph-mon[44992]: from='client.? 172.21.15.37:0/3563257192' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-03-29T08:02:49.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:49.029+0000 7fad9e800700 1 -- 172.21.15.37:0/1473531332 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fad981037c0 msgr2=0x7fad980fce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:02:49.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:49.029+0000 7fad9e800700 1 --2- 172.21.15.37:0/1473531332 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fad981037c0 0x7fad980fce30 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fad84004940 tx=0x7fad84009e30 comp rx=0 tx=0).stop 2024-03-29T08:02:49.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:49.029+0000 7fad9e800700 1 -- 172.21.15.37:0/1473531332 shutdown_connections 2024-03-29T08:02:49.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:49.029+0000 7fad9e800700 1 --2- 172.21.15.37:0/1473531332 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fad981037c0 0x7fad980fce30 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:02:49.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:49.029+0000 7fad9e800700 1 --2- 172.21.15.37:0/1473531332 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fad98102e20 0x7fad981031f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:02:49.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:49.029+0000 7fad9e800700 1 -- 172.21.15.37:0/1473531332 >> 172.21.15.37:0/1473531332 conn(0x7fad980f8970 msgr2=0x7fad980fad80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:02:49.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:49.029+0000 7fad9e800700 1 -- 172.21.15.37:0/1473531332 shutdown_connections 2024-03-29T08:02:49.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:49.029+0000 7fad9e800700 1 -- 172.21.15.37:0/1473531332 wait complete. 2024-03-29T08:02:49.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:49.030+0000 7fad9e800700 1 Processor -- start 2024-03-29T08:02:49.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:49.030+0000 7fad9e800700 1 -- start start 2024-03-29T08:02:49.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:49.031+0000 7fad9e800700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fad98102e20 0x7fad980702c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:02:49.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:49.031+0000 7fad9e800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fad981037c0 0x7fad98070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:02:49.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:49.031+0000 7fad9e800700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad98070e20 con 0x7fad981037c0 2024-03-29T08:02:49.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:49.031+0000 7fad9e800700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad98070f60 con 0x7fad98102e20 2024-03-29T08:02:49.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:49.031+0000 7fad96a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fad981037c0 0x7fad98070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:02:49.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:49.031+0000 7fad96a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fad981037c0 0x7fad98070800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:32784/0 (socket says 172.21.15.37:32784) 2024-03-29T08:02:49.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:49.031+0000 7fad96a00700 1 -- 172.21.15.37:0/3288168170 learned_addr learned my addr 172.21.15.37:0/3288168170 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:02:49.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:49.031+0000 7fad97400700 1 --2- 172.21.15.37:0/3288168170 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fad98102e20 0x7fad980702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:02:49.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:49.031+0000 7fad96a00700 1 -- 172.21.15.37:0/3288168170 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fad98102e20 msgr2=0x7fad980702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:02:49.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:49.031+0000 7fad96a00700 1 --2- 172.21.15.37:0/3288168170 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fad98102e20 0x7fad980702c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:02:49.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:49.031+0000 7fad96a00700 1 -- 172.21.15.37:0/3288168170 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad84005040 con 0x7fad981037c0 2024-03-29T08:02:49.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:49.032+0000 7fad96a00700 1 --2- 172.21.15.37:0/3288168170 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fad981037c0 0x7fad98070800 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fad84009390 tx=0x7fad84009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:02:49.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:49.032+0000 7fad8f400700 1 -- 172.21.15.37:0/3288168170 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fad8400e040 con 0x7fad981037c0 2024-03-29T08:02:49.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:49.032+0000 7fad8f400700 1 -- 172.21.15.37:0/3288168170 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fad84013420 con 0x7fad981037c0 2024-03-29T08:02:49.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:49.032+0000 7fad8f400700 1 -- 172.21.15.37:0/3288168170 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fad8401c630 con 0x7fad981037c0 2024-03-29T08:02:49.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:49.032+0000 7fad9e800700 1 -- 172.21.15.37:0/3288168170 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad98119db0 con 0x7fad981037c0 2024-03-29T08:02:49.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:49.032+0000 7fad9e800700 1 -- 172.21.15.37:0/3288168170 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad9811a290 con 0x7fad981037c0 2024-03-29T08:02:49.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:49.033+0000 7fad9e800700 1 -- 172.21.15.37:0/3288168170 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad980fe570 con 0x7fad981037c0 2024-03-29T08:02:49.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:49.034+0000 7fad8f400700 1 -- 172.21.15.37:0/3288168170 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fad840139e0 con 0x7fad981037c0 2024-03-29T08:02:49.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:49.035+0000 7fad8f400700 1 --2- 172.21.15.37:0/3288168170 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7fad7c0692b0 0x7fad7c06b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:02:49.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:49.036+0000 7fad8f400700 1 -- 172.21.15.37:0/3288168170 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fad84088180 con 0x7fad981037c0 2024-03-29T08:02:49.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:49.036+0000 7fad97400700 1 --2- 172.21.15.37:0/3288168170 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7fad7c0692b0 0x7fad7c06b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:02:49.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:49.036+0000 7fad97400700 1 --2- 172.21.15.37:0/3288168170 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7fad7c0692b0 0x7fad7c06b760 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fad88006fd0 tx=0x7fad88008040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:02:49.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:49.040+0000 7fad8f400700 1 -- 172.21.15.37:0/3288168170 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fad84054750 con 0x7fad981037c0 2024-03-29T08:02:49.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:49.251+0000 7fad9e800700 1 -- 172.21.15.37:0/3288168170 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fad9805f5f0 con 0x7fad981037c0 2024-03-29T08:02:49.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:49.251+0000 7fad8f400700 1 -- 172.21.15.37:0/3288168170 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fad84057d70 con 0x7fad981037c0 2024-03-29T08:02:49.251 INFO:teuthology.orchestra.run.smithi037.stdout:81604378653 2024-03-29T08:02:49.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:49.254+0000 7fad9e800700 1 -- 172.21.15.37:0/3288168170 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7fad7c0692b0 msgr2=0x7fad7c06b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:02:49.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:49.254+0000 7fad9e800700 1 --2- 172.21.15.37:0/3288168170 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7fad7c0692b0 0x7fad7c06b760 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fad88006fd0 tx=0x7fad88008040 comp rx=0 tx=0).stop 2024-03-29T08:02:49.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:49.254+0000 7fad9e800700 1 -- 172.21.15.37:0/3288168170 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fad981037c0 msgr2=0x7fad98070800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:02:49.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:49.254+0000 7fad9e800700 1 --2- 172.21.15.37:0/3288168170 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fad981037c0 0x7fad98070800 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fad84009390 tx=0x7fad84009e30 comp rx=0 tx=0).stop 2024-03-29T08:02:49.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:49.254+0000 7fad9e800700 1 -- 172.21.15.37:0/3288168170 shutdown_connections 2024-03-29T08:02:49.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:49.254+0000 7fad9e800700 1 --2- 172.21.15.37:0/3288168170 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7fad7c0692b0 0x7fad7c06b760 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:02:49.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:49.254+0000 7fad9e800700 1 --2- 172.21.15.37:0/3288168170 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fad98102e20 0x7fad980702c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:02:49.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:49.254+0000 7fad9e800700 1 --2- 172.21.15.37:0/3288168170 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fad981037c0 0x7fad98070800 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:02:49.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:49.254+0000 7fad9e800700 1 -- 172.21.15.37:0/3288168170 >> 172.21.15.37:0/3288168170 conn(0x7fad980f8970 msgr2=0x7fad980f9990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:02:49.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:49.254+0000 7fad9e800700 1 -- 172.21.15.37:0/3288168170 shutdown_connections 2024-03-29T08:02:49.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:49.254+0000 7fad9e800700 1 -- 172.21.15.37:0/3288168170 wait complete. 2024-03-29T08:02:49.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:02:49 smithi037 ceph-mon[35162]: from='client.? 172.21.15.37:0/3863649339' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-03-29T08:02:49.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:02:49 smithi037 ceph-mon[35162]: from='client.? 172.21.15.37:0/3288168170' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-03-29T08:02:50.055 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:02:49 smithi148 ceph-mon[44992]: from='client.? 172.21.15.37:0/3863649339' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-03-29T08:02:50.055 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:02:49 smithi148 ceph-mon[44992]: from='client.? 172.21.15.37:0/3288168170' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-03-29T08:02:50.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:48.929+0000 7fac07400700 1 -- 172.21.15.37:0/435480467 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fac00102710 msgr2=0x7fac000fbd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:02:50.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:48.929+0000 7fac07400700 1 --2- 172.21.15.37:0/435480467 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fac00102710 0x7fac000fbd80 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fabec004940 tx=0x7fabec009e30 comp rx=0 tx=0).stop 2024-03-29T08:02:50.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:48.929+0000 7fac07400700 1 -- 172.21.15.37:0/435480467 shutdown_connections 2024-03-29T08:02:50.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:48.929+0000 7fac07400700 1 --2- 172.21.15.37:0/435480467 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fac00102710 0x7fac000fbd80 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:02:50.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:48.929+0000 7fac07400700 1 --2- 172.21.15.37:0/435480467 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fac00101d70 0x7fac00102140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:02:50.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:48.929+0000 7fac07400700 1 -- 172.21.15.37:0/435480467 >> 172.21.15.37:0/435480467 conn(0x7fac00072bc0 msgr2=0x7fac00072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:02:50.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:48.929+0000 7fac07400700 1 -- 172.21.15.37:0/435480467 shutdown_connections 2024-03-29T08:02:50.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:48.929+0000 7fac07400700 1 -- 172.21.15.37:0/435480467 wait complete. 2024-03-29T08:02:50.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:48.930+0000 7fac07400700 1 Processor -- start 2024-03-29T08:02:50.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:48.930+0000 7fac07400700 1 -- start start 2024-03-29T08:02:50.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:48.930+0000 7fac07400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fac00101d70 0x7fac0010c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:02:50.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:48.930+0000 7fac07400700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fac00102710 0x7fac0010cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:02:50.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:48.930+0000 7fac07400700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac0010d270 con 0x7fac00101d70 2024-03-29T08:02:50.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:48.930+0000 7fac07400700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac0010d3e0 con 0x7fac00102710 2024-03-29T08:02:50.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:48.931+0000 7fac04a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fac00101d70 0x7fac0010c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:02:50.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:48.931+0000 7fac04a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fac00101d70 0x7fac0010c7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60998/0 (socket says 172.21.15.37:60998) 2024-03-29T08:02:50.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:48.931+0000 7fac04a00700 1 -- 172.21.15.37:0/3863649339 learned_addr learned my addr 172.21.15.37:0/3863649339 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:02:50.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:48.931+0000 7fabffe00700 1 --2- 172.21.15.37:0/3863649339 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fac00102710 0x7fac0010cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:02:50.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:48.931+0000 7fac04a00700 1 -- 172.21.15.37:0/3863649339 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fac00102710 msgr2=0x7fac0010cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:02:50.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:48.931+0000 7fac04a00700 1 --2- 172.21.15.37:0/3863649339 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fac00102710 0x7fac0010cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:02:50.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:48.931+0000 7fac04a00700 1 -- 172.21.15.37:0/3863649339 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabec005040 con 0x7fac00101d70 2024-03-29T08:02:50.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:48.931+0000 7fac04a00700 1 --2- 172.21.15.37:0/3863649339 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fac00101d70 0x7fac0010c7f0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fabf4002a50 tx=0x7fabf40046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:02:50.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:48.931+0000 7fabfcc00700 1 -- 172.21.15.37:0/3863649339 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fabf4007cc0 con 0x7fac00101d70 2024-03-29T08:02:50.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:48.932+0000 7fabfcc00700 1 -- 172.21.15.37:0/3863649339 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fabf400f040 con 0x7fac00101d70 2024-03-29T08:02:50.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:48.932+0000 7fabfcc00700 1 -- 172.21.15.37:0/3863649339 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fabf400d9c0 con 0x7fac00101d70 2024-03-29T08:02:50.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:48.932+0000 7fac07400700 1 -- 172.21.15.37:0/3863649339 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac00119e60 con 0x7fac00101d70 2024-03-29T08:02:50.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:48.932+0000 7fac07400700 1 -- 172.21.15.37:0/3863649339 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac0011a280 con 0x7fac00101d70 2024-03-29T08:02:50.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:48.933+0000 7fac07400700 1 -- 172.21.15.37:0/3863649339 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac000ff080 con 0x7fac00101d70 2024-03-29T08:02:50.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:48.934+0000 7fabfcc00700 1 -- 172.21.15.37:0/3863649339 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fabf400db20 con 0x7fac00101d70 2024-03-29T08:02:50.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:48.935+0000 7fabfcc00700 1 --2- 172.21.15.37:0/3863649339 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7fabe80692b0 0x7fabe806b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:02:50.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:48.935+0000 7fabfcc00700 1 -- 172.21.15.37:0/3863649339 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fabf4086a40 con 0x7fac00101d70 2024-03-29T08:02:50.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:48.935+0000 7fabffe00700 1 --2- 172.21.15.37:0/3863649339 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7fabe80692b0 0x7fabe806b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:02:50.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:48.936+0000 7fabffe00700 1 --2- 172.21.15.37:0/3863649339 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7fabe80692b0 0x7fabe806b760 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fac00066110 tx=0x7fabec009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:02:50.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:48.939+0000 7fabfcc00700 1 -- 172.21.15.37:0/3863649339 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fabf4056b40 con 0x7fac00101d70 2024-03-29T08:02:50.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:49.166+0000 7fac07400700 1 -- 172.21.15.37:0/3863649339 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7fac00108d70 con 0x7fac00101d70 2024-03-29T08:02:50.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:49.166+0000 7fabfcc00700 1 -- 172.21.15.37:0/3863649339 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fabf40566d0 con 0x7fac00101d70 2024-03-29T08:02:50.136 INFO:teuthology.orchestra.run.smithi037.stdout:171798691850 2024-03-29T08:02:50.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:49.169+0000 7fac07400700 1 -- 172.21.15.37:0/3863649339 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7fabe80692b0 msgr2=0x7fabe806b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:02:50.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:49.169+0000 7fac07400700 1 --2- 172.21.15.37:0/3863649339 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7fabe80692b0 0x7fabe806b760 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fac00066110 tx=0x7fabec009e30 comp rx=0 tx=0).stop 2024-03-29T08:02:50.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:49.169+0000 7fac07400700 1 -- 172.21.15.37:0/3863649339 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fac00101d70 msgr2=0x7fac0010c7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:02:50.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:49.169+0000 7fac07400700 1 --2- 172.21.15.37:0/3863649339 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fac00101d70 0x7fac0010c7f0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fabf4002a50 tx=0x7fabf40046b0 comp rx=0 tx=0).stop 2024-03-29T08:02:50.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:49.169+0000 7fac07400700 1 -- 172.21.15.37:0/3863649339 shutdown_connections 2024-03-29T08:02:50.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:49.169+0000 7fac07400700 1 --2- 172.21.15.37:0/3863649339 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7fabe80692b0 0x7fabe806b760 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:02:50.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:49.169+0000 7fac07400700 1 --2- 172.21.15.37:0/3863649339 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fac00101d70 0x7fac0010c7f0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:02:50.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:49.169+0000 7fac07400700 1 --2- 172.21.15.37:0/3863649339 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fac00102710 0x7fac0010cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:02:50.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:49.169+0000 7fac07400700 1 -- 172.21.15.37:0/3863649339 >> 172.21.15.37:0/3863649339 conn(0x7fac00072bc0 msgr2=0x7fac000fb150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:02:50.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:49.169+0000 7fac07400700 1 -- 172.21.15.37:0/3863649339 shutdown_connections 2024-03-29T08:02:50.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:49.169+0000 7fac07400700 1 -- 172.21.15.37:0/3863649339 wait complete. 2024-03-29T08:02:50.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:02:50 smithi037 ceph-mon[35162]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:02:51.055 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:02:50 smithi148 ceph-mon[44992]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:02:51.176 INFO:tasks.cephadm.ceph_manager.ceph:need seq 171798691848 got 171798691850 for osd.5 2024-03-29T08:02:51.176 DEBUG:teuthology.parallel:result is None 2024-03-29T08:02:51.685 INFO:tasks.cephadm.ceph_manager.ceph:need seq 81604378650 got 81604378653 for osd.2 2024-03-29T08:02:51.685 DEBUG:teuthology.parallel:result is None 2024-03-29T08:02:52.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:51.924+0000 7f9008c00700 1 -- 172.21.15.37:0/632366510 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f90041037a0 msgr2=0x7f90040fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:02:52.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:51.924+0000 7f9008c00700 1 --2- 172.21.15.37:0/632366510 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f90041037a0 0x7f90040fce10 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f8ff0004940 tx=0x7f8ff0009e30 comp rx=0 tx=0).stop 2024-03-29T08:02:52.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:51.924+0000 7f9008c00700 1 -- 172.21.15.37:0/632366510 shutdown_connections 2024-03-29T08:02:52.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:51.924+0000 7f9008c00700 1 --2- 172.21.15.37:0/632366510 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f90041037a0 0x7f90040fce10 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:02:52.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:51.924+0000 7f9008c00700 1 --2- 172.21.15.37:0/632366510 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9004102e00 0x7f90041031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:02:52.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:51.924+0000 7f9008c00700 1 -- 172.21.15.37:0/632366510 >> 172.21.15.37:0/632366510 conn(0x7f90040f89b0 msgr2=0x7f90040fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:02:52.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:51.924+0000 7f9008c00700 1 -- 172.21.15.37:0/632366510 shutdown_connections 2024-03-29T08:02:52.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:51.924+0000 7f9008c00700 1 -- 172.21.15.37:0/632366510 wait complete. 2024-03-29T08:02:52.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:51.925+0000 7f9008c00700 1 Processor -- start 2024-03-29T08:02:52.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:51.926+0000 7f9008c00700 1 -- start start 2024-03-29T08:02:52.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:51.926+0000 7f9008c00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9004102e00 0x7f9004113b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:02:52.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:51.926+0000 7f9008c00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f90041037a0 0x7f9004114040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:02:52.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:51.926+0000 7f9008c00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9004114690 con 0x7f90041037a0 2024-03-29T08:02:52.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:51.926+0000 7f9008c00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f900410da80 con 0x7f9004102e00 2024-03-29T08:02:52.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:51.926+0000 7f9001400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f90041037a0 0x7f9004114040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:02:52.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:51.927+0000 7f9001400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f90041037a0 0x7f9004114040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57490/0 (socket says 172.21.15.37:57490) 2024-03-29T08:02:52.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:51.927+0000 7f9001400700 1 -- 172.21.15.37:0/790427437 learned_addr learned my addr 172.21.15.37:0/790427437 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:02:52.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:51.927+0000 7f9001400700 1 -- 172.21.15.37:0/790427437 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9004102e00 msgr2=0x7f9004113b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:02:52.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:51.927+0000 7f9001400700 1 --2- 172.21.15.37:0/790427437 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9004102e00 0x7f9004113b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:02:52.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:51.927+0000 7f9001400700 1 -- 172.21.15.37:0/790427437 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ff0005040 con 0x7f90041037a0 2024-03-29T08:02:52.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:51.927+0000 7f9001400700 1 --2- 172.21.15.37:0/790427437 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f90041037a0 0x7f9004114040 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f8ff00093b0 tx=0x7f8ff0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:02:52.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:51.927+0000 7f8ffa000700 1 -- 172.21.15.37:0/790427437 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8ff000e040 con 0x7f90041037a0 2024-03-29T08:02:52.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:51.927+0000 7f9008c00700 1 -- 172.21.15.37:0/790427437 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f900410dd00 con 0x7f90041037a0 2024-03-29T08:02:52.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:51.927+0000 7f9008c00700 1 -- 172.21.15.37:0/790427437 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f900410e2c0 con 0x7f90041037a0 2024-03-29T08:02:52.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:51.928+0000 7f8ffa000700 1 -- 172.21.15.37:0/790427437 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f8ff0013420 con 0x7f90041037a0 2024-03-29T08:02:52.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:51.928+0000 7f8ffa000700 1 -- 172.21.15.37:0/790427437 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8ff0025620 con 0x7f90041037a0 2024-03-29T08:02:52.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:51.929+0000 7f9008c00700 1 -- 172.21.15.37:0/790427437 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fe40052b0 con 0x7f90041037a0 2024-03-29T08:02:52.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:51.929+0000 7f8ffa000700 1 -- 172.21.15.37:0/790427437 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f8ff000bb80 con 0x7f90041037a0 2024-03-29T08:02:52.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:51.930+0000 7f8ffa000700 1 --2- 172.21.15.37:0/790427437 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f8fe80692a0 0x7f8fe806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:02:52.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:51.930+0000 7f8ffa000700 1 -- 172.21.15.37:0/790427437 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f8ff0087ff0 con 0x7f90041037a0 2024-03-29T08:02:52.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:51.931+0000 7f9001e00700 1 --2- 172.21.15.37:0/790427437 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f8fe80692a0 0x7f8fe806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:02:52.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:51.931+0000 7f9001e00700 1 --2- 172.21.15.37:0/790427437 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f8fe80692a0 0x7f8fe806b750 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f8ff4004ad0 tx=0x7f8ff4007d40 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:02:52.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:51.936+0000 7f8ffa000700 1 -- 172.21.15.37:0/790427437 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f8ff00ba190 con 0x7f90041037a0 2024-03-29T08:02:52.133 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/mon.smithi037/config 2024-03-29T08:02:52.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:52.156+0000 7f9008c00700 1 -- 172.21.15.37:0/790427437 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f8fe4005560 con 0x7f90041037a0 2024-03-29T08:02:52.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:52.157+0000 7f8ffa000700 1 -- 172.21.15.37:0/790427437 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8ff00580d0 con 0x7f90041037a0 2024-03-29T08:02:52.156 INFO:teuthology.orchestra.run.smithi037.stdout:150323855378 2024-03-29T08:02:52.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:52.159+0000 7f9008c00700 1 -- 172.21.15.37:0/790427437 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f8fe80692a0 msgr2=0x7f8fe806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:02:52.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:52.159+0000 7f9008c00700 1 --2- 172.21.15.37:0/790427437 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f8fe80692a0 0x7f8fe806b750 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f8ff4004ad0 tx=0x7f8ff4007d40 comp rx=0 tx=0).stop 2024-03-29T08:02:52.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:52.159+0000 7f9008c00700 1 -- 172.21.15.37:0/790427437 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f90041037a0 msgr2=0x7f9004114040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:02:52.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:52.159+0000 7f9008c00700 1 --2- 172.21.15.37:0/790427437 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f90041037a0 0x7f9004114040 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f8ff00093b0 tx=0x7f8ff0009e30 comp rx=0 tx=0).stop 2024-03-29T08:02:52.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:52.159+0000 7f9008c00700 1 -- 172.21.15.37:0/790427437 shutdown_connections 2024-03-29T08:02:52.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:52.159+0000 7f9008c00700 1 --2- 172.21.15.37:0/790427437 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f8fe80692a0 0x7f8fe806b750 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:02:52.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:52.159+0000 7f9008c00700 1 --2- 172.21.15.37:0/790427437 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9004102e00 0x7f9004113b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:02:52.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:52.159+0000 7f9008c00700 1 --2- 172.21.15.37:0/790427437 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f90041037a0 0x7f9004114040 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:02:52.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:52.159+0000 7f9008c00700 1 -- 172.21.15.37:0/790427437 >> 172.21.15.37:0/790427437 conn(0x7f90040f89b0 msgr2=0x7f90040f9ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:02:52.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:52.159+0000 7f9008c00700 1 -- 172.21.15.37:0/790427437 shutdown_connections 2024-03-29T08:02:52.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:52.159+0000 7f9008c00700 1 -- 172.21.15.37:0/790427437 wait complete. 2024-03-29T08:02:52.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:52.383+0000 7f7498800700 1 -- 172.21.15.37:0/1661834887 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7490104ff0 msgr2=0x7f74901053c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:02:52.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:52.383+0000 7f7498800700 1 --2- 172.21.15.37:0/1661834887 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7490104ff0 0x7f74901053c0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f7484004940 tx=0x7f7484009e30 comp rx=0 tx=0).stop 2024-03-29T08:02:52.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:52.384+0000 7f7498800700 1 -- 172.21.15.37:0/1661834887 shutdown_connections 2024-03-29T08:02:52.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:52.384+0000 7f7498800700 1 --2- 172.21.15.37:0/1661834887 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7490105990 0x7f74900ff000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:02:52.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:52.384+0000 7f7498800700 1 --2- 172.21.15.37:0/1661834887 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7490104ff0 0x7f74901053c0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:02:52.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:52.384+0000 7f7498800700 1 -- 172.21.15.37:0/1661834887 >> 172.21.15.37:0/1661834887 conn(0x7f74900faba0 msgr2=0x7f74900fcfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:02:52.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:52.384+0000 7f7498800700 1 -- 172.21.15.37:0/1661834887 shutdown_connections 2024-03-29T08:02:52.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:52.384+0000 7f7498800700 1 -- 172.21.15.37:0/1661834887 wait complete. 2024-03-29T08:02:52.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:52.384+0000 7f7498800700 1 Processor -- start 2024-03-29T08:02:52.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:52.385+0000 7f7498800700 1 -- start start 2024-03-29T08:02:52.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:52.385+0000 7f7498800700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7490104ff0 0x7f7490115a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:02:52.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:52.385+0000 7f7498800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7490105990 0x7f7490115f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:02:52.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:52.385+0000 7f7498800700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74901165e0 con 0x7f7490105990 2024-03-29T08:02:52.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:52.385+0000 7f7498800700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f749010fc00 con 0x7f7490104ff0 2024-03-29T08:02:52.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:52.385+0000 7f7495400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7490105990 0x7f7490115f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:02:52.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:52.385+0000 7f7495400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7490105990 0x7f7490115f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57498/0 (socket says 172.21.15.37:57498) 2024-03-29T08:02:52.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:52.385+0000 7f7495400700 1 -- 172.21.15.37:0/1011413944 learned_addr learned my addr 172.21.15.37:0/1011413944 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:02:52.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:52.385+0000 7f7495e00700 1 --2- 172.21.15.37:0/1011413944 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7490104ff0 0x7f7490115a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:02:52.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:52.386+0000 7f7495400700 1 -- 172.21.15.37:0/1011413944 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7490104ff0 msgr2=0x7f7490115a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:02:52.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:52.386+0000 7f7495400700 1 --2- 172.21.15.37:0/1011413944 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7490104ff0 0x7f7490115a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:02:52.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:52.386+0000 7f7495400700 1 -- 172.21.15.37:0/1011413944 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7484005040 con 0x7f7490105990 2024-03-29T08:02:52.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:52.386+0000 7f7495400700 1 --2- 172.21.15.37:0/1011413944 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7490105990 0x7f7490115f90 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f7480002a50 tx=0x7f7480004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:02:52.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:52.386+0000 7f748a000700 1 -- 172.21.15.37:0/1011413944 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7480007cc0 con 0x7f7490105990 2024-03-29T08:02:52.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:52.386+0000 7f748a000700 1 -- 172.21.15.37:0/1011413944 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f748000f040 con 0x7f7490105990 2024-03-29T08:02:52.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:52.386+0000 7f7498800700 1 -- 172.21.15.37:0/1011413944 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f749010fee0 con 0x7f7490105990 2024-03-29T08:02:52.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:52.386+0000 7f748a000700 1 -- 172.21.15.37:0/1011413944 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f748000d920 con 0x7f7490105990 2024-03-29T08:02:52.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:52.386+0000 7f7498800700 1 -- 172.21.15.37:0/1011413944 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74901104a0 con 0x7f7490105990 2024-03-29T08:02:52.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:52.387+0000 7f7498800700 1 -- 172.21.15.37:0/1011413944 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7490102300 con 0x7f7490105990 2024-03-29T08:02:52.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:52.388+0000 7f748a000700 1 -- 172.21.15.37:0/1011413944 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f7480009680 con 0x7f7490105990 2024-03-29T08:02:52.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:52.389+0000 7f748a000700 1 --2- 172.21.15.37:0/1011413944 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f74780692a0 0x7f747806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:02:52.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:52.389+0000 7f748a000700 1 -- 172.21.15.37:0/1011413944 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f7480086b30 con 0x7f7490105990 2024-03-29T08:02:52.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:52.389+0000 7f7495e00700 1 --2- 172.21.15.37:0/1011413944 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f74780692a0 0x7f747806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:02:52.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:52.390+0000 7f7495e00700 1 --2- 172.21.15.37:0/1011413944 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f74780692a0 0x7f747806b750 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f748400b840 tx=0x7f7484009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:02:52.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:52.393+0000 7f748a000700 1 -- 172.21.15.37:0/1011413944 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f7480053100 con 0x7f7490105990 2024-03-29T08:02:52.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:52.605+0000 7f7498800700 1 -- 172.21.15.37:0/1011413944 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f74901110e0 con 0x7f7490105990 2024-03-29T08:02:52.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:52.606+0000 7f748a000700 1 -- 172.21.15.37:0/1011413944 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f7480056720 con 0x7f7490105990 2024-03-29T08:02:52.605 INFO:teuthology.orchestra.run.smithi037.stdout:60129542180 2024-03-29T08:02:52.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:52.608+0000 7f7498800700 1 -- 172.21.15.37:0/1011413944 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f74780692a0 msgr2=0x7f747806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:02:52.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:52.608+0000 7f7498800700 1 --2- 172.21.15.37:0/1011413944 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f74780692a0 0x7f747806b750 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f748400b840 tx=0x7f7484009e30 comp rx=0 tx=0).stop 2024-03-29T08:02:52.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:52.608+0000 7f7498800700 1 -- 172.21.15.37:0/1011413944 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7490105990 msgr2=0x7f7490115f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:02:52.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:52.608+0000 7f7498800700 1 --2- 172.21.15.37:0/1011413944 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7490105990 0x7f7490115f90 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f7480002a50 tx=0x7f7480004600 comp rx=0 tx=0).stop 2024-03-29T08:02:52.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:52.608+0000 7f7498800700 1 -- 172.21.15.37:0/1011413944 shutdown_connections 2024-03-29T08:02:52.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:52.608+0000 7f7498800700 1 --2- 172.21.15.37:0/1011413944 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f74780692a0 0x7f747806b750 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:02:52.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:52.608+0000 7f7498800700 1 --2- 172.21.15.37:0/1011413944 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7490104ff0 0x7f7490115a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:02:52.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:52.608+0000 7f7498800700 1 --2- 172.21.15.37:0/1011413944 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7490105990 0x7f7490115f90 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:02:52.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:52.608+0000 7f7498800700 1 -- 172.21.15.37:0/1011413944 >> 172.21.15.37:0/1011413944 conn(0x7f74900faba0 msgr2=0x7f74900fca70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:02:52.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:52.609+0000 7f7498800700 1 -- 172.21.15.37:0/1011413944 shutdown_connections 2024-03-29T08:02:52.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:52.609+0000 7f7498800700 1 -- 172.21.15.37:0/1011413944 wait complete. 2024-03-29T08:02:52.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:02:52 smithi037 ceph-mon[35162]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:02:52.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:02:52 smithi037 ceph-mon[35162]: from='client.? 172.21.15.37:0/790427437' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-03-29T08:02:52.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:02:52 smithi037 ceph-mon[35162]: from='client.? 172.21.15.37:0/1011413944' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-03-29T08:02:53.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:02:52 smithi148 ceph-mon[44992]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:02:53.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:02:52 smithi148 ceph-mon[44992]: from='client.? 172.21.15.37:0/790427437' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-03-29T08:02:53.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:02:52 smithi148 ceph-mon[44992]: from='client.? 172.21.15.37:0/1011413944' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-03-29T08:02:53.133 INFO:tasks.cephadm.ceph_manager.ceph:need seq 150323855376 got 150323855378 for osd.4 2024-03-29T08:02:53.133 DEBUG:teuthology.parallel:result is None 2024-03-29T08:02:53.798 INFO:tasks.cephadm.ceph_manager.ceph:need seq 60129542178 got 60129542180 for osd.1 2024-03-29T08:02:53.798 DEBUG:teuthology.parallel:result is None 2024-03-29T08:02:54.324 INFO:tasks.cephadm.ceph_manager.ceph:need seq 38654705705 got 38654705707 for osd.0 2024-03-29T08:02:54.324 DEBUG:teuthology.parallel:result is None 2024-03-29T08:02:54.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:02:54 smithi037 ceph-mon[35162]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:02:55.055 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:02:54 smithi148 ceph-mon[44992]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:02:55.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:55.789+0000 7f90f9800700 1 -- 172.21.15.37:0/2582112508 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f90f4102e00 msgr2=0x7f90f41031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:02:55.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:55.789+0000 7f90f9800700 1 --2- 172.21.15.37:0/2582112508 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f90f4102e00 0x7f90f41031d0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f90e4004940 tx=0x7f90e4009e30 comp rx=0 tx=0).stop 2024-03-29T08:02:55.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:55.789+0000 7f90f9800700 1 -- 172.21.15.37:0/2582112508 shutdown_connections 2024-03-29T08:02:55.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:55.789+0000 7f90f9800700 1 --2- 172.21.15.37:0/2582112508 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f90f41037a0 0x7f90f40fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:02:55.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:55.789+0000 7f90f9800700 1 --2- 172.21.15.37:0/2582112508 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f90f4102e00 0x7f90f41031d0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:02:55.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:55.789+0000 7f90f9800700 1 -- 172.21.15.37:0/2582112508 >> 172.21.15.37:0/2582112508 conn(0x7f90f40f89b0 msgr2=0x7f90f40fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:02:55.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:55.789+0000 7f90f9800700 1 -- 172.21.15.37:0/2582112508 shutdown_connections 2024-03-29T08:02:55.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:55.789+0000 7f90f9800700 1 -- 172.21.15.37:0/2582112508 wait complete. 2024-03-29T08:02:55.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:55.790+0000 7f90f9800700 1 Processor -- start 2024-03-29T08:02:55.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:55.790+0000 7f90f9800700 1 -- start start 2024-03-29T08:02:55.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:55.791+0000 7f90f9800700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f90f4102e00 0x7f90f4115cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:02:55.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:55.791+0000 7f90f9800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f90f41037a0 0x7f90f4116230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:02:55.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:55.791+0000 7f90f9800700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90f4116880 con 0x7f90f41037a0 2024-03-29T08:02:55.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:55.791+0000 7f90f9800700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90f410fc50 con 0x7f90f4102e00 2024-03-29T08:02:55.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:55.791+0000 7f90f2000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f90f41037a0 0x7f90f4116230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:02:55.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:55.791+0000 7f90f2000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f90f41037a0 0x7f90f4116230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57512/0 (socket says 172.21.15.37:57512) 2024-03-29T08:02:55.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:55.791+0000 7f90f2000700 1 -- 172.21.15.37:0/514336879 learned_addr learned my addr 172.21.15.37:0/514336879 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:02:55.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:55.791+0000 7f90f2a00700 1 --2- 172.21.15.37:0/514336879 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f90f4102e00 0x7f90f4115cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:02:55.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:55.791+0000 7f90f2000700 1 -- 172.21.15.37:0/514336879 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f90f4102e00 msgr2=0x7f90f4115cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:02:55.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:55.791+0000 7f90f2000700 1 --2- 172.21.15.37:0/514336879 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f90f4102e00 0x7f90f4115cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:02:55.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:55.791+0000 7f90f2000700 1 -- 172.21.15.37:0/514336879 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90e4005040 con 0x7f90f41037a0 2024-03-29T08:02:55.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:55.792+0000 7f90f2000700 1 --2- 172.21.15.37:0/514336879 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f90f41037a0 0x7f90f4116230 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f90e0002a50 tx=0x7f90e0004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:02:55.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:55.792+0000 7f90eaa00700 1 -- 172.21.15.37:0/514336879 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f90e0007cc0 con 0x7f90f41037a0 2024-03-29T08:02:55.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:55.792+0000 7f90eaa00700 1 -- 172.21.15.37:0/514336879 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f90e000f040 con 0x7f90f41037a0 2024-03-29T08:02:55.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:55.792+0000 7f90eaa00700 1 -- 172.21.15.37:0/514336879 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f90e000d920 con 0x7f90f41037a0 2024-03-29T08:02:55.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:55.792+0000 7f90f9800700 1 -- 172.21.15.37:0/514336879 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90f410ff30 con 0x7f90f41037a0 2024-03-29T08:02:55.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:55.792+0000 7f90f9800700 1 -- 172.21.15.37:0/514336879 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90f41104f0 con 0x7f90f41037a0 2024-03-29T08:02:55.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:55.793+0000 7f90f9800700 1 -- 172.21.15.37:0/514336879 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90f40fe550 con 0x7f90f41037a0 2024-03-29T08:02:55.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:55.794+0000 7f90eaa00700 1 -- 172.21.15.37:0/514336879 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f90e0009680 con 0x7f90f41037a0 2024-03-29T08:02:55.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:55.795+0000 7f90eaa00700 1 --2- 172.21.15.37:0/514336879 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f90d80692b0 0x7f90d806b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:02:55.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:55.795+0000 7f90eaa00700 1 -- 172.21.15.37:0/514336879 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f90e0086ba0 con 0x7f90f41037a0 2024-03-29T08:02:55.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:55.795+0000 7f90f2a00700 1 --2- 172.21.15.37:0/514336879 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f90d80692b0 0x7f90d806b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:02:55.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:55.796+0000 7f90f2a00700 1 --2- 172.21.15.37:0/514336879 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f90d80692b0 0x7f90d806b760 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f90e4009fd0 tx=0x7f90e4009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:02:55.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:55.799+0000 7f90eaa00700 1 -- 172.21.15.37:0/514336879 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f90e0056c00 con 0x7f90f41037a0 2024-03-29T08:02:55.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:02:55 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:02:56.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:56.006+0000 7f90f9800700 1 -- 172.21.15.37:0/514336879 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f90f404bed0 con 0x7f90f41037a0 2024-03-29T08:02:56.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:56.007+0000 7f90eaa00700 1 -- 172.21.15.37:0/514336879 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f90e0056790 con 0x7f90f41037a0 2024-03-29T08:02:56.007 INFO:teuthology.orchestra.run.smithi037.stdout:115964117016 2024-03-29T08:02:56.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:56.009+0000 7f90f9800700 1 -- 172.21.15.37:0/514336879 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f90d80692b0 msgr2=0x7f90d806b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:02:56.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:56.009+0000 7f90f9800700 1 --2- 172.21.15.37:0/514336879 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f90d80692b0 0x7f90d806b760 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f90e4009fd0 tx=0x7f90e4009e30 comp rx=0 tx=0).stop 2024-03-29T08:02:56.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:56.009+0000 7f90f9800700 1 -- 172.21.15.37:0/514336879 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f90f41037a0 msgr2=0x7f90f4116230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:02:56.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:56.009+0000 7f90f9800700 1 --2- 172.21.15.37:0/514336879 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f90f41037a0 0x7f90f4116230 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f90e0002a50 tx=0x7f90e0004600 comp rx=0 tx=0).stop 2024-03-29T08:02:56.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:56.009+0000 7f90f9800700 1 -- 172.21.15.37:0/514336879 shutdown_connections 2024-03-29T08:02:56.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:56.009+0000 7f90f9800700 1 --2- 172.21.15.37:0/514336879 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f90d80692b0 0x7f90d806b760 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:02:56.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:56.009+0000 7f90f9800700 1 --2- 172.21.15.37:0/514336879 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f90f4102e00 0x7f90f4115cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:02:56.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:56.009+0000 7f90f9800700 1 --2- 172.21.15.37:0/514336879 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f90f41037a0 0x7f90f4116230 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:02:56.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:56.009+0000 7f90f9800700 1 -- 172.21.15.37:0/514336879 >> 172.21.15.37:0/514336879 conn(0x7f90f40f89b0 msgr2=0x7f90f40706d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:02:56.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:56.009+0000 7f90f9800700 1 -- 172.21.15.37:0/514336879 shutdown_connections 2024-03-29T08:02:56.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:56.009+0000 7f90f9800700 1 -- 172.21.15.37:0/514336879 wait complete. 2024-03-29T08:02:56.055 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:02:55 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:02:56.629 INFO:tasks.cephadm.ceph_manager.ceph:need seq 115964117014 got 115964117016 for osd.3 2024-03-29T08:02:56.630 DEBUG:teuthology.parallel:result is None 2024-03-29T08:02:56.630 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-03-29T08:02:56.630 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 99b492c2-eda1-11ee-b647-cb9ed24678a4 -- ceph pg dump --format=json 2024-03-29T08:02:56.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:02:56 smithi037 ceph-mon[35162]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:02:56.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:02:56 smithi037 ceph-mon[35162]: from='client.? 172.21.15.37:0/514336879' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-03-29T08:02:57.055 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:02:56 smithi148 ceph-mon[44992]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:02:57.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:02:56 smithi148 ceph-mon[44992]: from='client.? 172.21.15.37:0/514336879' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-03-29T08:02:57.064 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/mon.smithi037/config 2024-03-29T08:02:58.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:58.787+0000 7fbe8e600700 1 -- 172.21.15.37:0/2729530307 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbe88102e00 msgr2=0x7fbe881031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:02:58.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:58.787+0000 7fbe8e600700 1 --2- 172.21.15.37:0/2729530307 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbe88102e00 0x7fbe881031d0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fbe78004940 tx=0x7fbe78009e30 comp rx=0 tx=0).stop 2024-03-29T08:02:58.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:58.787+0000 7fbe8e600700 1 -- 172.21.15.37:0/2729530307 shutdown_connections 2024-03-29T08:02:58.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:58.787+0000 7fbe8e600700 1 --2- 172.21.15.37:0/2729530307 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbe881037a0 0x7fbe880fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:02:58.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:58.787+0000 7fbe8e600700 1 --2- 172.21.15.37:0/2729530307 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbe88102e00 0x7fbe881031d0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:02:58.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:58.787+0000 7fbe8e600700 1 -- 172.21.15.37:0/2729530307 >> 172.21.15.37:0/2729530307 conn(0x7fbe880f89b0 msgr2=0x7fbe880fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:02:58.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:58.787+0000 7fbe8e600700 1 -- 172.21.15.37:0/2729530307 shutdown_connections 2024-03-29T08:02:58.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:58.787+0000 7fbe8e600700 1 -- 172.21.15.37:0/2729530307 wait complete. 2024-03-29T08:02:58.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:58.788+0000 7fbe8e600700 1 Processor -- start 2024-03-29T08:02:58.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:58.788+0000 7fbe8e600700 1 -- start start 2024-03-29T08:02:58.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:58.788+0000 7fbe8e600700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbe88102e00 0x7fbe88113a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:02:58.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:58.788+0000 7fbe8e600700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbe881037a0 0x7fbe88113f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:02:58.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:58.788+0000 7fbe8e600700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe881145d0 con 0x7fbe88102e00 2024-03-29T08:02:58.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:58.788+0000 7fbe8e600700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe8810d9c0 con 0x7fbe881037a0 2024-03-29T08:02:58.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:58.789+0000 7fbe87400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbe88102e00 0x7fbe88113a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:02:58.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:58.789+0000 7fbe87400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbe88102e00 0x7fbe88113a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57522/0 (socket says 172.21.15.37:57522) 2024-03-29T08:02:58.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:58.789+0000 7fbe87400700 1 -- 172.21.15.37:0/3509930633 learned_addr learned my addr 172.21.15.37:0/3509930633 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:02:58.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:58.789+0000 7fbe86a00700 1 --2- 172.21.15.37:0/3509930633 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbe881037a0 0x7fbe88113f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:02:58.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:58.789+0000 7fbe87400700 1 -- 172.21.15.37:0/3509930633 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbe881037a0 msgr2=0x7fbe88113f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:02:58.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:58.789+0000 7fbe87400700 1 --2- 172.21.15.37:0/3509930633 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbe881037a0 0x7fbe88113f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:02:58.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:58.789+0000 7fbe87400700 1 -- 172.21.15.37:0/3509930633 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe78005040 con 0x7fbe88102e00 2024-03-29T08:02:58.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:58.789+0000 7fbe86a00700 1 --2- 172.21.15.37:0/3509930633 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbe881037a0 0x7fbe88113f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:02:58.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:58.789+0000 7fbe87400700 1 --2- 172.21.15.37:0/3509930633 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbe88102e00 0x7fbe88113a40 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fbe78009390 tx=0x7fbe78009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:02:58.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:58.790+0000 7fbe7f400700 1 -- 172.21.15.37:0/3509930633 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbe7800e040 con 0x7fbe88102e00 2024-03-29T08:02:58.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:58.790+0000 7fbe7f400700 1 -- 172.21.15.37:0/3509930633 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fbe78013420 con 0x7fbe88102e00 2024-03-29T08:02:58.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:58.790+0000 7fbe7f400700 1 -- 172.21.15.37:0/3509930633 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbe7801c630 con 0x7fbe88102e00 2024-03-29T08:02:58.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:58.790+0000 7fbe8e600700 1 -- 172.21.15.37:0/3509930633 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe8810dc40 con 0x7fbe88102e00 2024-03-29T08:02:58.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:58.790+0000 7fbe8e600700 1 -- 172.21.15.37:0/3509930633 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe8810e200 con 0x7fbe88102e00 2024-03-29T08:02:58.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:58.791+0000 7fbe8e600700 1 -- 172.21.15.37:0/3509930633 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe880fe550 con 0x7fbe88102e00 2024-03-29T08:02:58.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:58.792+0000 7fbe7f400700 1 -- 172.21.15.37:0/3509930633 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fbe780139e0 con 0x7fbe88102e00 2024-03-29T08:02:58.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:58.793+0000 7fbe7f400700 1 --2- 172.21.15.37:0/3509930633 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7fbe6c0692b0 0x7fbe6c06b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:02:58.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:58.793+0000 7fbe7f400700 1 -- 172.21.15.37:0/3509930633 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fbe78087ff0 con 0x7fbe88102e00 2024-03-29T08:02:58.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:58.793+0000 7fbe86a00700 1 --2- 172.21.15.37:0/3509930633 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7fbe6c0692b0 0x7fbe6c06b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:02:58.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:58.793+0000 7fbe86a00700 1 --2- 172.21.15.37:0/3509930633 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7fbe6c0692b0 0x7fbe6c06b760 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fbe74007fd0 tx=0x7fbe74009f60 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:02:58.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:58.797+0000 7fbe7f400700 1 -- 172.21.15.37:0/3509930633 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fbe780ba190 con 0x7fbe88102e00 2024-03-29T08:02:58.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:02:58 smithi037 ceph-mon[35162]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:02:59.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:59.001+0000 7fbe8e600700 1 -- 172.21.15.37:0/3509930633 --> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fbe8804bed0 con 0x7fbe6c0692b0 2024-03-29T08:02:59.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:59.003+0000 7fbe7f400700 1 -- 172.21.15.37:0/3509930633 <== mgr.14208 v2:172.21.15.37:6800/1802836043 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+28146 (secure 0 0 0) 0x7fbe8804bed0 con 0x7fbe6c0692b0 2024-03-29T08:02:59.003 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:02:59.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:59.005+0000 7fbe8e600700 1 -- 172.21.15.37:0/3509930633 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7fbe6c0692b0 msgr2=0x7fbe6c06b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:02:59.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:59.005+0000 7fbe8e600700 1 --2- 172.21.15.37:0/3509930633 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7fbe6c0692b0 0x7fbe6c06b760 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fbe74007fd0 tx=0x7fbe74009f60 comp rx=0 tx=0).stop 2024-03-29T08:02:59.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:59.005+0000 7fbe8e600700 1 -- 172.21.15.37:0/3509930633 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbe88102e00 msgr2=0x7fbe88113a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:02:59.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:59.005+0000 7fbe8e600700 1 --2- 172.21.15.37:0/3509930633 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbe88102e00 0x7fbe88113a40 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fbe78009390 tx=0x7fbe78009e30 comp rx=0 tx=0).stop 2024-03-29T08:02:59.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:59.005+0000 7fbe8e600700 1 -- 172.21.15.37:0/3509930633 shutdown_connections 2024-03-29T08:02:59.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:59.005+0000 7fbe8e600700 1 --2- 172.21.15.37:0/3509930633 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7fbe6c0692b0 0x7fbe6c06b760 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:02:59.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:59.005+0000 7fbe8e600700 1 --2- 172.21.15.37:0/3509930633 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbe88102e00 0x7fbe88113a40 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:02:59.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:59.005+0000 7fbe8e600700 1 --2- 172.21.15.37:0/3509930633 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbe881037a0 0x7fbe88113f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:02:59.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:59.005+0000 7fbe8e600700 1 -- 172.21.15.37:0/3509930633 >> 172.21.15.37:0/3509930633 conn(0x7fbe880f89b0 msgr2=0x7fbe880f99f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:02:59.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:59.006+0000 7fbe8e600700 1 -- 172.21.15.37:0/3509930633 shutdown_connections 2024-03-29T08:02:59.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:02:59.006+0000 7fbe8e600700 1 -- 172.21.15.37:0/3509930633 wait complete. 2024-03-29T08:02:59.006 INFO:teuthology.orchestra.run.smithi037.stderr:dumped all 2024-03-29T08:02:59.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:02:58 smithi148 ceph-mon[44992]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:02:59.623 INFO:teuthology.orchestra.run.smithi037.stdout:{"pg_ready":true,"pg_map":{"version":180,"stamp":"2024-03-29T08:02:57.460234+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":3,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":3,"kb":562470912,"kb_used":1788076,"kb_used_data":3924,"kb_used_omap":9,"kb_used_meta":1784054,"kb_avail":560682836,"statfs":{"total":575970213888,"available":574139224064,"internally_reserved":0,"allocated":4018176,"data_stored":2624466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10224,"internal_metadata":1826871312},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002222"},"pg_stats":[{"pgid":"1.0","version":"24'117","reported_seq":188,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-29T08:02:04.556970+0000","last_change":"2024-03-29T08:01:32.912101+0000","last_active":"2024-03-29T08:02:04.556970+0000","last_peered":"2024-03-29T08:02:04.556970+0000","last_clean":"2024-03-29T08:02:04.556970+0000","last_became_active":"2024-03-29T08:01:32.911626+0000","last_became_peered":"2024-03-29T08:01:32.911626+0000","last_unstale":"2024-03-29T08:02:04.556970+0000","last_undegraded":"2024-03-29T08:02:04.556970+0000","last_fullsized":"2024-03-29T08:02:04.556970+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:00:32.072578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:00:32.072578+0000","last_clean_scrub_stamp":"2024-03-29T08:00:32.072578+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:41:34.333359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":4}],"osd_stats":[{"osd":5,"up_from":40,"seq":171798691852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":297720,"kb_used_data":364,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447432,"statfs":{"total":95995035648,"available":95690170368,"internally_reserved":0,"allocated":372736,"data_stored":142227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54200000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70299999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.628}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66500000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71999999999999997}]}]},{"osd":4,"up_from":35,"seq":150323855379,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":297720,"kb_used_data":364,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447432,"statfs":{"total":95995035648,"available":95690170368,"internally_reserved":0,"allocated":372736,"data_stored":142227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:02:34 2024","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":2.8450000000000002,"5min":2.8450000000000002,"15min":2.8450000000000002},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":2.9060000000000001,"5min":2.9060000000000001,"15min":2.9060000000000001},"last":0.54200000000000004}]},{"osd":1,"last update":"Fri Mar 29 08:02:34 2024","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":5.625,"5min":5.625,"15min":5.625},"last":0.58399999999999996}]},{"osd":2,"last update":"Fri Mar 29 08:02:34 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":2.8679999999999999,"5min":2.8679999999999999,"15min":2.8679999999999999},"last":0.59199999999999997}]},{"osd":3,"last update":"Fri Mar 29 08:02:34 2024","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"last":0.504},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.56999999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48799999999999999}]}]},{"osd":3,"up_from":27,"seq":115964117017,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298300,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446852,"statfs":{"total":95995035648,"available":95689576448,"internally_reserved":0,"allocated":966656,"data_stored":732595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:02:03 2024","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.34200000000000003}]},{"osd":1,"last update":"Fri Mar 29 08:02:03 2024","interfaces":[{"interface":"back","average":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":7.3230000000000004,"5min":7.3230000000000004,"15min":7.3230000000000004},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.46200000000000002}]},{"osd":2,"last update":"Fri Mar 29 08:02:03 2024","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.39700000000000002},{"interface":"front","average":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":7.3499999999999996,"5min":7.3499999999999996,"15min":7.3499999999999996},"last":0.54000000000000004}]},{"osd":4,"last update":"Fri Mar 29 08:02:35 2024","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":1.4830000000000001,"5min":1.4830000000000001,"15min":1.4830000000000001},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.8420000000000001,"5min":1.8420000000000001,"15min":1.8420000000000001},"last":0.58799999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.622}]}]},{"osd":2,"up_from":19,"seq":81604378655,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":297724,"kb_used_data":364,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447428,"statfs":{"total":95995035648,"available":95690166272,"internally_reserved":0,"allocated":372736,"data_stored":142227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:02:33 2024","interfaces":[{"interface":"back","average":{"1min":0.40000000000000002,"5min":0.42099999999999999,"15min":0.42499999999999999},"min":{"1min":0.29799999999999999,"5min":0.252,"15min":0.252},"max":{"1min":0.53500000000000003,"5min":0.66700000000000004,"15min":0.66700000000000004},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.42599999999999999,"5min":0.47999999999999998,"15min":0.48899999999999999},"min":{"1min":0.29799999999999999,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.59599999999999997,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.495}]},{"osd":1,"last update":"Fri Mar 29 08:02:33 2024","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.45800000000000002,"15min":0.46200000000000002},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.58799999999999997,"5min":0.60499999999999998,"15min":0.60499999999999998},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.51000000000000001,"15min":0.51600000000000001},"min":{"1min":0.38,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.63,"5min":0.748,"15min":0.748},"last":0.57999999999999996}]},{"osd":3,"last update":"Fri Mar 29 08:02:02 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.92000000000000004},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.252,"5min":1.252,"15min":1.252},"last":0.73599999999999999}]},{"osd":4,"last update":"Fri Mar 29 08:02:33 2024","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.1799999999999999,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.21,"5min":1.21,"15min":1.21},"last":0.77400000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81399999999999995}]}]},{"osd":0,"up_from":9,"seq":38654705709,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298304,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446848,"statfs":{"total":95995035648,"available":95689572352,"internally_reserved":0,"allocated":966656,"data_stored":732595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Mar 29 08:02:01 2024","interfaces":[{"interface":"back","average":{"1min":0.42099999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"min":{"1min":0.317,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.52900000000000003,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.33100000000000002},{"interface":"front","average":{"1min":0.44600000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"min":{"1min":0.33100000000000002,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.56000000000000005,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.373}]},{"osd":2,"last update":"Fri Mar 29 08:02:37 2024","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.47699999999999998,"15min":0.48099999999999998},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.58099999999999996,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.312},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.48799999999999999,"15min":0.48999999999999999},"min":{"1min":0.378,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.64800000000000002,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.39000000000000001}]},{"osd":3,"last update":"Fri Mar 29 08:02:01 2024","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.56200000000000006}]},{"osd":4,"last update":"Fri Mar 29 08:02:37 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.46999999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.504}]}]},{"osd":1,"up_from":14,"seq":60129542182,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298308,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446844,"statfs":{"total":95995035648,"available":95689568256,"internally_reserved":0,"allocated":966656,"data_stored":732595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:02:00 2024","interfaces":[{"interface":"back","average":{"1min":0.40999999999999998,"5min":0.40600000000000003,"15min":0.40600000000000003},"min":{"1min":0.32800000000000001,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.57599999999999996,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.441,"5min":0.44400000000000001,"15min":0.44400000000000001},"min":{"1min":0.35899999999999999,"5min":0.318,"15min":0.318},"max":{"1min":0.65700000000000003,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.40500000000000003}]},{"osd":2,"last update":"Fri Mar 29 08:02:36 2024","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.47899999999999998,"15min":0.47999999999999998},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.71399999999999997,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.498,"5min":0.502,"15min":0.503},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.64200000000000002,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.44600000000000001}]},{"osd":3,"last update":"Fri Mar 29 08:02:00 2024","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.6830000000000001,"5min":1.6830000000000001,"15min":1.6830000000000001},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.704,"5min":1.704,"15min":1.704},"last":0.75700000000000001}]},{"osd":4,"last update":"Fri Mar 29 08:02:36 2024","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.73999999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71699999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-29T08:02:59.624 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 99b492c2-eda1-11ee-b647-cb9ed24678a4 -- ceph pg dump --format=json 2024-03-29T08:03:00.061 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/mon.smithi037/config 2024-03-29T08:03:01.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:03:00 smithi148 ceph-mon[44992]: from='client.14426 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-29T08:03:01.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:03:00 smithi148 ceph-mon[44992]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:03:01.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:03:00 smithi037 ceph-mon[35162]: from='client.14426 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-29T08:03:01.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:03:00 smithi037 ceph-mon[35162]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:03:01.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:01.855+0000 7f8e53e00700 1 -- 172.21.15.37:0/1806339955 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8e4c102e00 msgr2=0x7f8e4c1031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:03:01.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:01.855+0000 7f8e53e00700 1 --2- 172.21.15.37:0/1806339955 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8e4c102e00 0x7f8e4c1031d0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f8e40004940 tx=0x7f8e40009e30 comp rx=0 tx=0).stop 2024-03-29T08:03:01.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:01.855+0000 7f8e53e00700 1 -- 172.21.15.37:0/1806339955 shutdown_connections 2024-03-29T08:03:01.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:01.855+0000 7f8e53e00700 1 --2- 172.21.15.37:0/1806339955 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8e4c1037a0 0x7f8e4c0fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:03:01.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:01.855+0000 7f8e53e00700 1 --2- 172.21.15.37:0/1806339955 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8e4c102e00 0x7f8e4c1031d0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:03:01.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:01.855+0000 7f8e53e00700 1 -- 172.21.15.37:0/1806339955 >> 172.21.15.37:0/1806339955 conn(0x7f8e4c0f89b0 msgr2=0x7f8e4c0fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:03:01.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:01.855+0000 7f8e53e00700 1 -- 172.21.15.37:0/1806339955 shutdown_connections 2024-03-29T08:03:01.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:01.855+0000 7f8e53e00700 1 -- 172.21.15.37:0/1806339955 wait complete. 2024-03-29T08:03:01.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:01.856+0000 7f8e53e00700 1 Processor -- start 2024-03-29T08:03:01.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:01.856+0000 7f8e53e00700 1 -- start start 2024-03-29T08:03:01.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:01.857+0000 7f8e53e00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8e4c102e00 0x7f8e4c0702c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:03:01.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:01.857+0000 7f8e53e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8e4c1037a0 0x7f8e4c070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:03:01.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:01.857+0000 7f8e53e00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e4c070d90 con 0x7f8e4c1037a0 2024-03-29T08:03:01.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:01.857+0000 7f8e53e00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e4c070ed0 con 0x7f8e4c102e00 2024-03-29T08:03:01.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:01.857+0000 7f8e50a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8e4c1037a0 0x7f8e4c070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:03:01.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:01.857+0000 7f8e50a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8e4c1037a0 0x7f8e4c070800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:48058/0 (socket says 172.21.15.37:48058) 2024-03-29T08:03:01.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:01.857+0000 7f8e50a00700 1 -- 172.21.15.37:0/2563732005 learned_addr learned my addr 172.21.15.37:0/2563732005 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:03:01.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:01.857+0000 7f8e51400700 1 --2- 172.21.15.37:0/2563732005 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8e4c102e00 0x7f8e4c0702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:03:01.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:01.857+0000 7f8e50a00700 1 -- 172.21.15.37:0/2563732005 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8e4c102e00 msgr2=0x7f8e4c0702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:03:01.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:01.857+0000 7f8e50a00700 1 --2- 172.21.15.37:0/2563732005 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8e4c102e00 0x7f8e4c0702c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:03:01.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:01.857+0000 7f8e50a00700 1 -- 172.21.15.37:0/2563732005 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e40005040 con 0x7f8e4c1037a0 2024-03-29T08:03:01.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:01.857+0000 7f8e51400700 1 --2- 172.21.15.37:0/2563732005 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8e4c102e00 0x7f8e4c0702c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:03:01.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:01.858+0000 7f8e50a00700 1 --2- 172.21.15.37:0/2563732005 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8e4c1037a0 0x7f8e4c070800 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f8e38002a80 tx=0x7f8e38004630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:03:01.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:01.858+0000 7f8e49600700 1 -- 172.21.15.37:0/2563732005 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8e38007cc0 con 0x7f8e4c1037a0 2024-03-29T08:03:01.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:01.858+0000 7f8e49600700 1 -- 172.21.15.37:0/2563732005 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f8e3800f040 con 0x7f8e4c1037a0 2024-03-29T08:03:01.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:01.858+0000 7f8e53e00700 1 -- 172.21.15.37:0/2563732005 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e4c119e40 con 0x7f8e4c1037a0 2024-03-29T08:03:01.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:01.858+0000 7f8e49600700 1 -- 172.21.15.37:0/2563732005 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8e3800da40 con 0x7f8e4c1037a0 2024-03-29T08:03:01.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:01.858+0000 7f8e53e00700 1 -- 172.21.15.37:0/2563732005 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e4c11a2c0 con 0x7f8e4c1037a0 2024-03-29T08:03:01.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:01.859+0000 7f8e53e00700 1 -- 172.21.15.37:0/2563732005 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e4c0fe550 con 0x7f8e4c1037a0 2024-03-29T08:03:01.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:01.860+0000 7f8e49600700 1 -- 172.21.15.37:0/2563732005 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f8e3800dbe0 con 0x7f8e4c1037a0 2024-03-29T08:03:01.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:01.861+0000 7f8e49600700 1 --2- 172.21.15.37:0/2563732005 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f8e340692b0 0x7f8e3406b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:03:01.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:01.861+0000 7f8e49600700 1 -- 172.21.15.37:0/2563732005 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f8e38086540 con 0x7f8e4c1037a0 2024-03-29T08:03:01.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:01.861+0000 7f8e51400700 1 --2- 172.21.15.37:0/2563732005 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f8e340692b0 0x7f8e3406b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:03:01.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:01.862+0000 7f8e51400700 1 --2- 172.21.15.37:0/2563732005 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f8e340692b0 0x7f8e3406b760 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f8e40000c00 tx=0x7f8e40009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:03:01.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:01.865+0000 7f8e49600700 1 -- 172.21.15.37:0/2563732005 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f8e38052b10 con 0x7f8e4c1037a0 2024-03-29T08:03:02.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:02.065+0000 7f8e53e00700 1 -- 172.21.15.37:0/2563732005 --> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8e4c04bed0 con 0x7f8e340692b0 2024-03-29T08:03:02.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:02.067+0000 7f8e49600700 1 -- 172.21.15.37:0/2563732005 <== mgr.14208 v2:172.21.15.37:6800/1802836043 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+28159 (secure 0 0 0) 0x7f8e4c04bed0 con 0x7f8e340692b0 2024-03-29T08:03:02.067 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:03:02.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:02.069+0000 7f8e53e00700 1 -- 172.21.15.37:0/2563732005 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f8e340692b0 msgr2=0x7f8e3406b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:03:02.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:02.069+0000 7f8e53e00700 1 --2- 172.21.15.37:0/2563732005 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f8e340692b0 0x7f8e3406b760 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f8e40000c00 tx=0x7f8e40009e30 comp rx=0 tx=0).stop 2024-03-29T08:03:02.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:02.069+0000 7f8e53e00700 1 -- 172.21.15.37:0/2563732005 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8e4c1037a0 msgr2=0x7f8e4c070800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:03:02.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:02.069+0000 7f8e53e00700 1 --2- 172.21.15.37:0/2563732005 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8e4c1037a0 0x7f8e4c070800 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f8e38002a80 tx=0x7f8e38004630 comp rx=0 tx=0).stop 2024-03-29T08:03:02.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:02.069+0000 7f8e53e00700 1 -- 172.21.15.37:0/2563732005 shutdown_connections 2024-03-29T08:03:02.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:02.069+0000 7f8e53e00700 1 --2- 172.21.15.37:0/2563732005 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f8e340692b0 0x7f8e3406b760 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:03:02.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:02.069+0000 7f8e53e00700 1 --2- 172.21.15.37:0/2563732005 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8e4c102e00 0x7f8e4c0702c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:03:02.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:02.069+0000 7f8e53e00700 1 --2- 172.21.15.37:0/2563732005 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8e4c1037a0 0x7f8e4c070800 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:03:02.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:02.069+0000 7f8e53e00700 1 -- 172.21.15.37:0/2563732005 >> 172.21.15.37:0/2563732005 conn(0x7f8e4c0f89b0 msgr2=0x7f8e4c0f9a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:03:02.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:02.069+0000 7f8e53e00700 1 -- 172.21.15.37:0/2563732005 shutdown_connections 2024-03-29T08:03:02.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:02.069+0000 7f8e53e00700 1 -- 172.21.15.37:0/2563732005 wait complete. 2024-03-29T08:03:02.070 INFO:teuthology.orchestra.run.smithi037.stderr:dumped all 2024-03-29T08:03:02.742 INFO:teuthology.orchestra.run.smithi037.stdout:{"pg_ready":true,"pg_map":{"version":182,"stamp":"2024-03-29T08:03:01.461109+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":3,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":3,"kb":562470912,"kb_used":1788076,"kb_used_data":3924,"kb_used_omap":9,"kb_used_meta":1784054,"kb_avail":560682836,"statfs":{"total":575970213888,"available":574139224064,"internally_reserved":0,"allocated":4018176,"data_stored":2624466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10224,"internal_metadata":1826871312},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002303"},"pg_stats":[{"pgid":"1.0","version":"24'117","reported_seq":188,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-29T08:02:04.556970+0000","last_change":"2024-03-29T08:01:32.912101+0000","last_active":"2024-03-29T08:02:04.556970+0000","last_peered":"2024-03-29T08:02:04.556970+0000","last_clean":"2024-03-29T08:02:04.556970+0000","last_became_active":"2024-03-29T08:01:32.911626+0000","last_became_peered":"2024-03-29T08:01:32.911626+0000","last_unstale":"2024-03-29T08:02:04.556970+0000","last_undegraded":"2024-03-29T08:02:04.556970+0000","last_fullsized":"2024-03-29T08:02:04.556970+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:00:32.072578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:00:32.072578+0000","last_clean_scrub_stamp":"2024-03-29T08:00:32.072578+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:41:34.333359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":4}],"osd_stats":[{"osd":5,"up_from":40,"seq":171798691853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":297720,"kb_used_data":364,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447432,"statfs":{"total":95995035648,"available":95690170368,"internally_reserved":0,"allocated":372736,"data_stored":142227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49399999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55900000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53900000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39800000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.376}]}]},{"osd":4,"up_from":35,"seq":150323855380,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":297720,"kb_used_data":364,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447432,"statfs":{"total":95995035648,"available":95690170368,"internally_reserved":0,"allocated":372736,"data_stored":142227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:02:34 2024","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":2.8450000000000002,"5min":2.8450000000000002,"15min":2.8450000000000002},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":2.9060000000000001,"5min":2.9060000000000001,"15min":2.9060000000000001},"last":0.54600000000000004}]},{"osd":1,"last update":"Fri Mar 29 08:02:34 2024","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":5.625,"5min":5.625,"15min":5.625},"last":0.56599999999999995}]},{"osd":2,"last update":"Fri Mar 29 08:02:34 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":2.8679999999999999,"5min":2.8679999999999999,"15min":2.8679999999999999},"last":0.50900000000000001}]},{"osd":3,"last update":"Fri Mar 29 08:02:34 2024","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.42399999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58499999999999996}]}]},{"osd":3,"up_from":27,"seq":115964117018,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298300,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446852,"statfs":{"total":95995035648,"available":95689576448,"internally_reserved":0,"allocated":966656,"data_stored":732595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:02:03 2024","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.66800000000000004}]},{"osd":1,"last update":"Fri Mar 29 08:02:03 2024","interfaces":[{"interface":"back","average":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":7.3230000000000004,"5min":7.3230000000000004,"15min":7.3230000000000004},"last":0.625},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.79600000000000004}]},{"osd":2,"last update":"Fri Mar 29 08:02:03 2024","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.72699999999999998},{"interface":"front","average":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":7.3499999999999996,"5min":7.3499999999999996,"15min":7.3499999999999996},"last":0.71499999999999997}]},{"osd":4,"last update":"Fri Mar 29 08:02:35 2024","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":1.4830000000000001,"5min":1.4830000000000001,"15min":1.4830000000000001},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.8420000000000001,"5min":1.8420000000000001,"15min":1.8420000000000001},"last":0.45900000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.502}]}]},{"osd":2,"up_from":19,"seq":81604378656,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":297724,"kb_used_data":364,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447428,"statfs":{"total":95995035648,"available":95690166272,"internally_reserved":0,"allocated":372736,"data_stored":142227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:02:33 2024","interfaces":[{"interface":"back","average":{"1min":0.40000000000000002,"5min":0.42099999999999999,"15min":0.42499999999999999},"min":{"1min":0.29799999999999999,"5min":0.252,"15min":0.252},"max":{"1min":0.53500000000000003,"5min":0.66700000000000004,"15min":0.66700000000000004},"last":0.24099999999999999},{"interface":"front","average":{"1min":0.42599999999999999,"5min":0.47999999999999998,"15min":0.48899999999999999},"min":{"1min":0.29799999999999999,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.59599999999999997,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.28000000000000003}]},{"osd":1,"last update":"Fri Mar 29 08:02:33 2024","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.45800000000000002,"15min":0.46200000000000002},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.58799999999999997,"5min":0.60499999999999998,"15min":0.60499999999999998},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.51000000000000001,"15min":0.51600000000000001},"min":{"1min":0.38,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.63,"5min":0.748,"15min":0.748},"last":0.312}]},{"osd":3,"last update":"Fri Mar 29 08:02:02 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.747},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.252,"5min":1.252,"15min":1.252},"last":0.77200000000000002}]},{"osd":4,"last update":"Fri Mar 29 08:02:33 2024","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.1799999999999999,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.21,"5min":1.21,"15min":1.21},"last":0.81499999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78500000000000003}]}]},{"osd":0,"up_from":9,"seq":38654705710,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298304,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446848,"statfs":{"total":95995035648,"available":95689572352,"internally_reserved":0,"allocated":966656,"data_stored":732595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Mar 29 08:02:01 2024","interfaces":[{"interface":"back","average":{"1min":0.42099999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"min":{"1min":0.317,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.52900000000000003,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.44600000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"min":{"1min":0.33100000000000002,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.56000000000000005,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.38200000000000001}]},{"osd":2,"last update":"Fri Mar 29 08:02:37 2024","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.47699999999999998,"15min":0.48099999999999998},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.58099999999999996,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.48799999999999999,"15min":0.48999999999999999},"min":{"1min":0.378,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.64800000000000002,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.41099999999999998}]},{"osd":3,"last update":"Fri Mar 29 08:02:01 2024","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.90600000000000003},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.93600000000000005}]},{"osd":4,"last update":"Fri Mar 29 08:02:37 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.87},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.84899999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88800000000000001}]}]},{"osd":1,"up_from":14,"seq":60129542182,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298308,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446844,"statfs":{"total":95995035648,"available":95689568256,"internally_reserved":0,"allocated":966656,"data_stored":732595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:02:00 2024","interfaces":[{"interface":"back","average":{"1min":0.40999999999999998,"5min":0.40600000000000003,"15min":0.40600000000000003},"min":{"1min":0.32800000000000001,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.57599999999999996,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.441,"5min":0.44400000000000001,"15min":0.44400000000000001},"min":{"1min":0.35899999999999999,"5min":0.318,"15min":0.318},"max":{"1min":0.65700000000000003,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.40500000000000003}]},{"osd":2,"last update":"Fri Mar 29 08:02:36 2024","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.47899999999999998,"15min":0.47999999999999998},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.71399999999999997,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.498,"5min":0.502,"15min":0.503},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.64200000000000002,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.44600000000000001}]},{"osd":3,"last update":"Fri Mar 29 08:02:00 2024","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.6830000000000001,"5min":1.6830000000000001,"15min":1.6830000000000001},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.704,"5min":1.704,"15min":1.704},"last":0.75700000000000001}]},{"osd":4,"last update":"Fri Mar 29 08:02:36 2024","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.73999999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71699999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-29T08:03:02.743 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-03-29T08:03:02.743 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-03-29T08:03:02.743 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-03-29T08:03:02.743 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 99b492c2-eda1-11ee-b647-cb9ed24678a4 -- ceph health --format=json 2024-03-29T08:03:03.055 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:03:02 smithi148 ceph-mon[44992]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:03:03.174 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/mon.smithi037/config 2024-03-29T08:03:03.176 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:03:02 smithi037 ceph-mon[35162]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:03:03.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:03:03 smithi037 ceph-mon[35162]: from='client.14430 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-29T08:03:04.055 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:03:03 smithi148 ceph-mon[44992]: from='client.14430 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-29T08:03:05.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:05.008+0000 7f1d8fc00700 1 -- 172.21.15.37:0/300365892 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1d88102e00 msgr2=0x7f1d881031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:03:05.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:05.008+0000 7f1d8fc00700 1 --2- 172.21.15.37:0/300365892 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1d88102e00 0x7f1d881031d0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f1d7c004940 tx=0x7f1d7c009e30 comp rx=0 tx=0).stop 2024-03-29T08:03:05.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:05.008+0000 7f1d8fc00700 1 -- 172.21.15.37:0/300365892 shutdown_connections 2024-03-29T08:03:05.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:05.008+0000 7f1d8fc00700 1 --2- 172.21.15.37:0/300365892 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1d881037a0 0x7f1d880fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:03:05.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:05.008+0000 7f1d8fc00700 1 --2- 172.21.15.37:0/300365892 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1d88102e00 0x7f1d881031d0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:03:05.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:05.008+0000 7f1d8fc00700 1 -- 172.21.15.37:0/300365892 >> 172.21.15.37:0/300365892 conn(0x7f1d880f89b0 msgr2=0x7f1d880fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:03:05.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:05.008+0000 7f1d8fc00700 1 -- 172.21.15.37:0/300365892 shutdown_connections 2024-03-29T08:03:05.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:05.008+0000 7f1d8fc00700 1 -- 172.21.15.37:0/300365892 wait complete. 2024-03-29T08:03:05.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:05.009+0000 7f1d8fc00700 1 Processor -- start 2024-03-29T08:03:05.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:05.009+0000 7f1d8fc00700 1 -- start start 2024-03-29T08:03:05.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:05.010+0000 7f1d8fc00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1d88102e00 0x7f1d88115c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:03:05.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:05.010+0000 7f1d8fc00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1d881037a0 0x7f1d881161c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:03:05.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:05.010+0000 7f1d8fc00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d88116810 con 0x7f1d881037a0 2024-03-29T08:03:05.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:05.010+0000 7f1d8fc00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d8810fbe0 con 0x7f1d88102e00 2024-03-29T08:03:05.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:05.010+0000 7f1d87e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1d881037a0 0x7f1d881161c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:03:05.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:05.010+0000 7f1d87e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1d881037a0 0x7f1d881161c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:48078/0 (socket says 172.21.15.37:48078) 2024-03-29T08:03:05.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:05.010+0000 7f1d87e00700 1 -- 172.21.15.37:0/144797566 learned_addr learned my addr 172.21.15.37:0/144797566 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:03:05.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:05.010+0000 7f1d8d200700 1 --2- 172.21.15.37:0/144797566 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1d88102e00 0x7f1d88115c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:03:05.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:05.011+0000 7f1d87e00700 1 -- 172.21.15.37:0/144797566 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1d88102e00 msgr2=0x7f1d88115c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:03:05.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:05.011+0000 7f1d87e00700 1 --2- 172.21.15.37:0/144797566 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1d88102e00 0x7f1d88115c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:03:05.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:05.011+0000 7f1d87e00700 1 -- 172.21.15.37:0/144797566 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d7c005040 con 0x7f1d881037a0 2024-03-29T08:03:05.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:05.011+0000 7f1d87e00700 1 --2- 172.21.15.37:0/144797566 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1d881037a0 0x7f1d881161c0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f1d7400eb40 tx=0x7f1d7400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:03:05.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:05.011+0000 7f1d84c00700 1 -- 172.21.15.37:0/144797566 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1d74007cc0 con 0x7f1d881037a0 2024-03-29T08:03:05.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:05.011+0000 7f1d84c00700 1 -- 172.21.15.37:0/144797566 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f1d7400f040 con 0x7f1d881037a0 2024-03-29T08:03:05.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:05.011+0000 7f1d8fc00700 1 -- 172.21.15.37:0/144797566 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d8810fe60 con 0x7f1d881037a0 2024-03-29T08:03:05.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:05.011+0000 7f1d84c00700 1 -- 172.21.15.37:0/144797566 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1d7400c8f0 con 0x7f1d881037a0 2024-03-29T08:03:05.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:05.011+0000 7f1d8fc00700 1 -- 172.21.15.37:0/144797566 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d88110420 con 0x7f1d881037a0 2024-03-29T08:03:05.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:05.012+0000 7f1d8fc00700 1 -- 172.21.15.37:0/144797566 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d880fe550 con 0x7f1d881037a0 2024-03-29T08:03:05.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:05.013+0000 7f1d84c00700 1 -- 172.21.15.37:0/144797566 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f1d740090d0 con 0x7f1d881037a0 2024-03-29T08:03:05.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:05.014+0000 7f1d84c00700 1 --2- 172.21.15.37:0/144797566 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f1d700692b0 0x7f1d7006b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:03:05.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:05.015+0000 7f1d84c00700 1 -- 172.21.15.37:0/144797566 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f1d74086840 con 0x7f1d881037a0 2024-03-29T08:03:05.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:05.015+0000 7f1d8d200700 1 --2- 172.21.15.37:0/144797566 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f1d700692b0 0x7f1d7006b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:03:05.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:05.015+0000 7f1d8d200700 1 --2- 172.21.15.37:0/144797566 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f1d700692b0 0x7f1d7006b760 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f1d7c0045a0 tx=0x7f1d7c009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:03:05.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:05.019+0000 7f1d84c00700 1 -- 172.21.15.37:0/144797566 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f1d7404e190 con 0x7f1d881037a0 2024-03-29T08:03:05.055 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:03:04 smithi148 ceph-mon[44992]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:03:05.176 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:03:04 smithi037 ceph-mon[35162]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:03:05.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:05.268+0000 7f1d8fc00700 1 -- 172.21.15.37:0/144797566 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f1d8805f5f0 con 0x7f1d881037a0 2024-03-29T08:03:05.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:05.268+0000 7f1d84c00700 1 -- 172.21.15.37:0/144797566 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f1d74056950 con 0x7f1d881037a0 2024-03-29T08:03:05.269 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:03:05.269 INFO:teuthology.orchestra.run.smithi037.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-03-29T08:03:05.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:05.270+0000 7f1d8fc00700 1 -- 172.21.15.37:0/144797566 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f1d700692b0 msgr2=0x7f1d7006b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:03:05.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:05.270+0000 7f1d8fc00700 1 --2- 172.21.15.37:0/144797566 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f1d700692b0 0x7f1d7006b760 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f1d7c0045a0 tx=0x7f1d7c009e30 comp rx=0 tx=0).stop 2024-03-29T08:03:05.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:05.270+0000 7f1d8fc00700 1 -- 172.21.15.37:0/144797566 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1d881037a0 msgr2=0x7f1d881161c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:03:05.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:05.270+0000 7f1d8fc00700 1 --2- 172.21.15.37:0/144797566 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1d881037a0 0x7f1d881161c0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f1d7400eb40 tx=0x7f1d7400b040 comp rx=0 tx=0).stop 2024-03-29T08:03:05.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:05.270+0000 7f1d8fc00700 1 -- 172.21.15.37:0/144797566 shutdown_connections 2024-03-29T08:03:05.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:05.270+0000 7f1d8fc00700 1 --2- 172.21.15.37:0/144797566 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f1d700692b0 0x7f1d7006b760 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:03:05.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:05.270+0000 7f1d8fc00700 1 --2- 172.21.15.37:0/144797566 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1d88102e00 0x7f1d88115c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:03:05.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:05.270+0000 7f1d8fc00700 1 --2- 172.21.15.37:0/144797566 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1d881037a0 0x7f1d881161c0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:03:05.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:05.270+0000 7f1d8fc00700 1 -- 172.21.15.37:0/144797566 >> 172.21.15.37:0/144797566 conn(0x7f1d880f89b0 msgr2=0x7f1d880f9940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:03:05.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:05.271+0000 7f1d8fc00700 1 -- 172.21.15.37:0/144797566 shutdown_connections 2024-03-29T08:03:05.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:05.271+0000 7f1d8fc00700 1 -- 172.21.15.37:0/144797566 wait complete. 2024-03-29T08:03:06.032 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:03:05 smithi037 ceph-mon[35162]: from='client.? 172.21.15.37:0/144797566' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-03-29T08:03:06.055 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-03-29T08:03:06.055 INFO:tasks.cephadm:Setup complete, yielding 2024-03-29T08:03:06.056 INFO:teuthology.run_tasks:Running task print... 2024-03-29T08:03:06.066 INFO:teuthology.task.print:**** done end installing quincy cephadm ... 2024-03-29T08:03:06.066 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-29T08:03:06.088 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi037.front.sepia.ceph.com 2024-03-29T08:03:06.089 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 99b492c2-eda1-11ee-b647-cb9ed24678a4 -- bash -c 'ceph config set mgr mgr/cephadm/use_repo_digest true --force' 2024-03-29T08:03:06.090 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:03:05 smithi148 ceph-mon[44992]: from='client.? 172.21.15.37:0/144797566' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-03-29T08:03:06.521 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/mon.smithi037/config 2024-03-29T08:03:06.987 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:03:06 smithi037 ceph-mon[35162]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:03:07.055 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:03:06 smithi148 ceph-mon[44992]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:03:08.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:08.226+0000 7fd166e00700 1 -- 172.21.15.37:0/54099062 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd1600fcee0 msgr2=0x7fd1600fd310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:03:08.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:08.226+0000 7fd166e00700 1 --2- 172.21.15.37:0/54099062 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd1600fcee0 0x7fd1600fd310 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fd150004940 tx=0x7fd150009e30 comp rx=0 tx=0).stop 2024-03-29T08:03:08.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:08.226+0000 7fd166e00700 1 -- 172.21.15.37:0/54099062 shutdown_connections 2024-03-29T08:03:08.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:08.226+0000 7fd166e00700 1 --2- 172.21.15.37:0/54099062 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd1600fe0d0 0x7fd1600fe540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:03:08.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:08.226+0000 7fd166e00700 1 --2- 172.21.15.37:0/54099062 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd1600fcee0 0x7fd1600fd310 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:03:08.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:08.226+0000 7fd166e00700 1 -- 172.21.15.37:0/54099062 >> 172.21.15.37:0/54099062 conn(0x7fd1600f8560 msgr2=0x7fd1600fa970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:03:08.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:08.226+0000 7fd166e00700 1 -- 172.21.15.37:0/54099062 shutdown_connections 2024-03-29T08:03:08.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:08.226+0000 7fd166e00700 1 -- 172.21.15.37:0/54099062 wait complete. 2024-03-29T08:03:08.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:08.227+0000 7fd166e00700 1 Processor -- start 2024-03-29T08:03:08.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:08.227+0000 7fd166e00700 1 -- start start 2024-03-29T08:03:08.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:08.227+0000 7fd166e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd1600fcee0 0x7fd16010fb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:03:08.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:08.227+0000 7fd166e00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd1600fe0d0 0x7fd1601100d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:03:08.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:08.227+0000 7fd166e00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1601106d0 con 0x7fd1600fcee0 2024-03-29T08:03:08.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:08.227+0000 7fd166e00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd160110f20 con 0x7fd1600fe0d0 2024-03-29T08:03:08.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:08.228+0000 7fd15fe00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd1600fcee0 0x7fd16010fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:03:08.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:08.228+0000 7fd15fe00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd1600fcee0 0x7fd16010fb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:48104/0 (socket says 172.21.15.37:48104) 2024-03-29T08:03:08.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:08.228+0000 7fd15fe00700 1 -- 172.21.15.37:0/3472278902 learned_addr learned my addr 172.21.15.37:0/3472278902 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:03:08.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:08.228+0000 7fd15f400700 1 --2- 172.21.15.37:0/3472278902 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd1600fe0d0 0x7fd1601100d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:03:08.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:08.228+0000 7fd15fe00700 1 -- 172.21.15.37:0/3472278902 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd1600fe0d0 msgr2=0x7fd1601100d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:03:08.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:08.228+0000 7fd15fe00700 1 --2- 172.21.15.37:0/3472278902 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd1600fe0d0 0x7fd1601100d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:03:08.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:08.228+0000 7fd15fe00700 1 -- 172.21.15.37:0/3472278902 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd150005040 con 0x7fd1600fcee0 2024-03-29T08:03:08.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:08.228+0000 7fd15fe00700 1 --2- 172.21.15.37:0/3472278902 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd1600fcee0 0x7fd16010fb90 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fd150009380 tx=0x7fd150009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:03:08.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:08.229+0000 7fd157e00700 1 -- 172.21.15.37:0/3472278902 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd15000e040 con 0x7fd1600fcee0 2024-03-29T08:03:08.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:08.229+0000 7fd166e00700 1 -- 172.21.15.37:0/3472278902 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1601111a0 con 0x7fd1600fcee0 2024-03-29T08:03:08.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:08.229+0000 7fd157e00700 1 -- 172.21.15.37:0/3472278902 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd150004290 con 0x7fd1600fcee0 2024-03-29T08:03:08.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:08.229+0000 7fd157e00700 1 -- 172.21.15.37:0/3472278902 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd15001c3f0 con 0x7fd1600fcee0 2024-03-29T08:03:08.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:08.229+0000 7fd166e00700 1 -- 172.21.15.37:0/3472278902 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1601116e0 con 0x7fd1600fcee0 2024-03-29T08:03:08.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:08.230+0000 7fd166e00700 1 -- 172.21.15.37:0/3472278902 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd16005f5f0 con 0x7fd1600fcee0 2024-03-29T08:03:08.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:08.231+0000 7fd157e00700 1 -- 172.21.15.37:0/3472278902 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fd150013420 con 0x7fd1600fcee0 2024-03-29T08:03:08.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:08.232+0000 7fd157e00700 1 --2- 172.21.15.37:0/3472278902 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7fd1440692b0 0x7fd14406b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:03:08.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:08.232+0000 7fd157e00700 1 -- 172.21.15.37:0/3472278902 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fd1500882b0 con 0x7fd1600fcee0 2024-03-29T08:03:08.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:08.232+0000 7fd15f400700 1 --2- 172.21.15.37:0/3472278902 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7fd1440692b0 0x7fd14406b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:03:08.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:08.232+0000 7fd15f400700 1 --2- 172.21.15.37:0/3472278902 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7fd1440692b0 0x7fd14406b760 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fd14c007fd0 tx=0x7fd14c009040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:03:08.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:08.236+0000 7fd157e00700 1 -- 172.21.15.37:0/3472278902 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fd150058310 con 0x7fd1600fcee0 2024-03-29T08:03:08.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:08.445+0000 7fd166e00700 1 -- 172.21.15.37:0/3472278902 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/use_repo_digest}] v 0) v1 -- 0x7fd160111ad0 con 0x7fd1600fcee0 2024-03-29T08:03:08.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:08.450+0000 7fd157e00700 1 -- 172.21.15.37:0/3472278902 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/use_repo_digest}]=0 v20)=0 v20) v1 ==== 143+0+0 (secure 0 0 0) 0x7fd150057ea0 con 0x7fd1600fcee0 2024-03-29T08:03:08.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:08.453+0000 7fd166e00700 1 -- 172.21.15.37:0/3472278902 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7fd1440692b0 msgr2=0x7fd14406b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:03:08.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:08.453+0000 7fd166e00700 1 --2- 172.21.15.37:0/3472278902 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7fd1440692b0 0x7fd14406b760 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fd14c007fd0 tx=0x7fd14c009040 comp rx=0 tx=0).stop 2024-03-29T08:03:08.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:08.453+0000 7fd166e00700 1 -- 172.21.15.37:0/3472278902 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd1600fcee0 msgr2=0x7fd16010fb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:03:08.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:08.453+0000 7fd166e00700 1 --2- 172.21.15.37:0/3472278902 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd1600fcee0 0x7fd16010fb90 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fd150009380 tx=0x7fd150009e30 comp rx=0 tx=0).stop 2024-03-29T08:03:08.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:08.454+0000 7fd166e00700 1 -- 172.21.15.37:0/3472278902 shutdown_connections 2024-03-29T08:03:08.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:08.454+0000 7fd166e00700 1 --2- 172.21.15.37:0/3472278902 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7fd1440692b0 0x7fd14406b760 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:03:08.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:08.454+0000 7fd166e00700 1 --2- 172.21.15.37:0/3472278902 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd1600fcee0 0x7fd16010fb90 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:03:08.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:08.454+0000 7fd166e00700 1 --2- 172.21.15.37:0/3472278902 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd1600fe0d0 0x7fd1601100d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:03:08.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:08.454+0000 7fd166e00700 1 -- 172.21.15.37:0/3472278902 >> 172.21.15.37:0/3472278902 conn(0x7fd1600f8560 msgr2=0x7fd1600f9db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:03:08.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:08.454+0000 7fd166e00700 1 -- 172.21.15.37:0/3472278902 shutdown_connections 2024-03-29T08:03:08.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:08.454+0000 7fd166e00700 1 -- 172.21.15.37:0/3472278902 wait complete. 2024-03-29T08:03:08.847 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:03:08 smithi037 ceph-mon[35162]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:03:08.848 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:03:08 smithi037 ceph-mon[35162]: from='client.? 172.21.15.37:0/3472278902' entity='client.admin' 2024-03-29T08:03:08.848 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:03:08 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T08:03:09.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:03:08 smithi148 ceph-mon[44992]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:03:09.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:03:08 smithi148 ceph-mon[44992]: from='client.? 172.21.15.37:0/3472278902' entity='client.admin' 2024-03-29T08:03:09.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:03:08 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T08:03:09.145 INFO:teuthology.run_tasks:Running task print... 2024-03-29T08:03:09.154 INFO:teuthology.task.print:**** done cephadm.shell ceph config set mgr... 2024-03-29T08:03:09.155 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-29T08:03:09.163 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi037.front.sepia.ceph.com 2024-03-29T08:03:09.164 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 99b492c2-eda1-11ee-b647-cb9ed24678a4 -- bash -c 'ceph orch status' 2024-03-29T08:03:09.625 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/mon.smithi037/config 2024-03-29T08:03:09.989 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:03:09 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T08:03:09.989 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:03:09 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T08:03:09.989 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:03:09 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:03:10.055 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:03:09 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T08:03:10.055 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:03:09 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T08:03:10.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:03:09 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:03:10.865 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:03:10 smithi037 ceph-mon[35162]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:03:10.865 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:03:10 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:03:11.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:03:10 smithi148 ceph-mon[44992]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:03:11.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:03:10 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:03:11.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:11.509+0000 7fd0b8600700 1 -- 172.21.15.37:0/3500494951 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd0b00fe0d0 msgr2=0x7fd0b00fe540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:03:11.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:11.509+0000 7fd0b8600700 1 --2- 172.21.15.37:0/3500494951 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd0b00fe0d0 0x7fd0b00fe540 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fd09c004940 tx=0x7fd09c009e30 comp rx=0 tx=0).stop 2024-03-29T08:03:11.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:11.509+0000 7fd0b8600700 1 -- 172.21.15.37:0/3500494951 shutdown_connections 2024-03-29T08:03:11.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:11.509+0000 7fd0b8600700 1 --2- 172.21.15.37:0/3500494951 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd0b00fe0d0 0x7fd0b00fe540 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:03:11.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:11.509+0000 7fd0b8600700 1 --2- 172.21.15.37:0/3500494951 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd0b00fcee0 0x7fd0b00fd310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:03:11.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:11.509+0000 7fd0b8600700 1 -- 172.21.15.37:0/3500494951 >> 172.21.15.37:0/3500494951 conn(0x7fd0b00f8560 msgr2=0x7fd0b00fa970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:03:11.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:11.510+0000 7fd0b8600700 1 -- 172.21.15.37:0/3500494951 shutdown_connections 2024-03-29T08:03:11.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:11.510+0000 7fd0b8600700 1 -- 172.21.15.37:0/3500494951 wait complete. 2024-03-29T08:03:11.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:11.510+0000 7fd0b8600700 1 Processor -- start 2024-03-29T08:03:11.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:11.511+0000 7fd0b8600700 1 -- start start 2024-03-29T08:03:11.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:11.511+0000 7fd0b8600700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd0b00fcee0 0x7fd0b010d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:03:11.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:11.511+0000 7fd0b8600700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd0b00fe0d0 0x7fd0b010de40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:03:11.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:11.511+0000 7fd0b8600700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0b010e440 con 0x7fd0b00fe0d0 2024-03-29T08:03:11.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:11.511+0000 7fd0b8600700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0b010ec90 con 0x7fd0b00fcee0 2024-03-29T08:03:11.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:11.511+0000 7fd0b5200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd0b00fe0d0 0x7fd0b010de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:03:11.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:11.511+0000 7fd0b5200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd0b00fe0d0 0x7fd0b010de40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36578/0 (socket says 172.21.15.37:36578) 2024-03-29T08:03:11.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:11.511+0000 7fd0b5200700 1 -- 172.21.15.37:0/483566480 learned_addr learned my addr 172.21.15.37:0/483566480 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:03:11.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:11.511+0000 7fd0b5c00700 1 --2- 172.21.15.37:0/483566480 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd0b00fcee0 0x7fd0b010d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:03:11.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:11.512+0000 7fd0b5200700 1 -- 172.21.15.37:0/483566480 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd0b00fcee0 msgr2=0x7fd0b010d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:03:11.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:11.512+0000 7fd0b5200700 1 --2- 172.21.15.37:0/483566480 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd0b00fcee0 0x7fd0b010d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:03:11.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:11.512+0000 7fd0b5200700 1 -- 172.21.15.37:0/483566480 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd09c005040 con 0x7fd0b00fe0d0 2024-03-29T08:03:11.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:11.512+0000 7fd0b5200700 1 --2- 172.21.15.37:0/483566480 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd0b00fe0d0 0x7fd0b010de40 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fd09c0137b0 tx=0x7fd09c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:03:11.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:11.512+0000 7fd0ad600700 1 -- 172.21.15.37:0/483566480 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd09c00e070 con 0x7fd0b00fe0d0 2024-03-29T08:03:11.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:11.512+0000 7fd0ad600700 1 -- 172.21.15.37:0/483566480 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd09c01c400 con 0x7fd0b00fe0d0 2024-03-29T08:03:11.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:11.512+0000 7fd0b8600700 1 -- 172.21.15.37:0/483566480 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0b010ef10 con 0x7fd0b00fe0d0 2024-03-29T08:03:11.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:11.512+0000 7fd0ad600700 1 -- 172.21.15.37:0/483566480 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd09c00b650 con 0x7fd0b00fe0d0 2024-03-29T08:03:11.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:11.512+0000 7fd0b8600700 1 -- 172.21.15.37:0/483566480 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0b010f3f0 con 0x7fd0b00fe0d0 2024-03-29T08:03:11.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:11.513+0000 7fd0b8600700 1 -- 172.21.15.37:0/483566480 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0b005f5f0 con 0x7fd0b00fe0d0 2024-03-29T08:03:11.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:11.514+0000 7fd0ad600700 1 -- 172.21.15.37:0/483566480 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fd09c01c950 con 0x7fd0b00fe0d0 2024-03-29T08:03:11.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:11.515+0000 7fd0ad600700 1 --2- 172.21.15.37:0/483566480 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7fd0980692a0 0x7fd09806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:03:11.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:11.515+0000 7fd0ad600700 1 -- 172.21.15.37:0/483566480 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fd09c088420 con 0x7fd0b00fe0d0 2024-03-29T08:03:11.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:11.515+0000 7fd0b5c00700 1 --2- 172.21.15.37:0/483566480 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7fd0980692a0 0x7fd09806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:03:11.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:11.516+0000 7fd0b5c00700 1 --2- 172.21.15.37:0/483566480 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7fd0980692a0 0x7fd09806b750 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd0a4009b80 tx=0x7fd0a400b040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:03:11.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:11.520+0000 7fd0ad600700 1 -- 172.21.15.37:0/483566480 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fd09c058500 con 0x7fd0b00fe0d0 2024-03-29T08:03:11.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:11.750+0000 7fd0b8600700 1 -- 172.21.15.37:0/483566480 --> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd0b010f7c0 con 0x7fd0980692a0 2024-03-29T08:03:11.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:11.752+0000 7fd0ad600700 1 -- 172.21.15.37:0/483566480 <== mgr.14208 v2:172.21.15.37:6800/1802836043 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7fd0b010f7c0 con 0x7fd0980692a0 2024-03-29T08:03:11.752 INFO:teuthology.orchestra.run.smithi037.stdout:Backend: cephadm 2024-03-29T08:03:11.752 INFO:teuthology.orchestra.run.smithi037.stdout:Available: Yes 2024-03-29T08:03:11.752 INFO:teuthology.orchestra.run.smithi037.stdout:Paused: No 2024-03-29T08:03:11.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:11.754+0000 7fd0b8600700 1 -- 172.21.15.37:0/483566480 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7fd0980692a0 msgr2=0x7fd09806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:03:11.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:11.754+0000 7fd0b8600700 1 --2- 172.21.15.37:0/483566480 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7fd0980692a0 0x7fd09806b750 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd0a4009b80 tx=0x7fd0a400b040 comp rx=0 tx=0).stop 2024-03-29T08:03:11.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:11.754+0000 7fd0b8600700 1 -- 172.21.15.37:0/483566480 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd0b00fe0d0 msgr2=0x7fd0b010de40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:03:11.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:11.754+0000 7fd0b8600700 1 --2- 172.21.15.37:0/483566480 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd0b00fe0d0 0x7fd0b010de40 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fd09c0137b0 tx=0x7fd09c009e30 comp rx=0 tx=0).stop 2024-03-29T08:03:11.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:11.754+0000 7fd0b8600700 1 -- 172.21.15.37:0/483566480 shutdown_connections 2024-03-29T08:03:11.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:11.754+0000 7fd0b8600700 1 --2- 172.21.15.37:0/483566480 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7fd0980692a0 0x7fd09806b750 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:03:11.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:11.754+0000 7fd0b8600700 1 --2- 172.21.15.37:0/483566480 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd0b00fcee0 0x7fd0b010d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:03:11.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:11.754+0000 7fd0b8600700 1 --2- 172.21.15.37:0/483566480 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd0b00fe0d0 0x7fd0b010de40 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:03:11.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:11.754+0000 7fd0b8600700 1 -- 172.21.15.37:0/483566480 >> 172.21.15.37:0/483566480 conn(0x7fd0b00f8560 msgr2=0x7fd0b00f9cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:03:11.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:11.754+0000 7fd0b8600700 1 -- 172.21.15.37:0/483566480 shutdown_connections 2024-03-29T08:03:11.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:11.754+0000 7fd0b8600700 1 -- 172.21.15.37:0/483566480 wait complete. 2024-03-29T08:03:12.430 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 99b492c2-eda1-11ee-b647-cb9ed24678a4 -- bash -c 'ceph orch ps' 2024-03-29T08:03:12.861 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/mon.smithi037/config 2024-03-29T08:03:13.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:03:12 smithi148 ceph-mon[44992]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:03:13.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:03:12 smithi148 ceph-mon[44992]: from='client.14442 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:03:13.176 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:03:12 smithi037 ceph-mon[35162]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:03:13.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:03:12 smithi037 ceph-mon[35162]: from='client.14442 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:03:14.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:14.694+0000 7fa719600700 1 -- 172.21.15.37:0/1899355634 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa7140fe0d0 msgr2=0x7fa7140fe540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:03:14.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:14.694+0000 7fa719600700 1 --2- 172.21.15.37:0/1899355634 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa7140fe0d0 0x7fa7140fe540 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fa700004940 tx=0x7fa700009e30 comp rx=0 tx=0).stop 2024-03-29T08:03:14.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:14.695+0000 7fa719600700 1 -- 172.21.15.37:0/1899355634 shutdown_connections 2024-03-29T08:03:14.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:14.695+0000 7fa719600700 1 --2- 172.21.15.37:0/1899355634 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa7140fe0d0 0x7fa7140fe540 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:03:14.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:14.695+0000 7fa719600700 1 --2- 172.21.15.37:0/1899355634 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa7140fcee0 0x7fa7140fd310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:03:14.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:14.695+0000 7fa719600700 1 -- 172.21.15.37:0/1899355634 >> 172.21.15.37:0/1899355634 conn(0x7fa7140f8560 msgr2=0x7fa7140fa970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:03:14.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:14.695+0000 7fa719600700 1 -- 172.21.15.37:0/1899355634 shutdown_connections 2024-03-29T08:03:14.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:14.695+0000 7fa719600700 1 -- 172.21.15.37:0/1899355634 wait complete. 2024-03-29T08:03:14.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:14.696+0000 7fa719600700 1 Processor -- start 2024-03-29T08:03:14.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:14.696+0000 7fa719600700 1 -- start start 2024-03-29T08:03:14.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:14.696+0000 7fa719600700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa7140fcee0 0x7fa714071e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:03:14.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:14.696+0000 7fa719600700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa7140fe0d0 0x7fa714070310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:03:14.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:14.696+0000 7fa719600700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7140708c0 con 0x7fa7140fcee0 2024-03-29T08:03:14.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:14.696+0000 7fa719600700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa714070a30 con 0x7fa7140fe0d0 2024-03-29T08:03:14.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:14.696+0000 7fa712a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa7140fcee0 0x7fa714071e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:03:14.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:14.696+0000 7fa712a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa7140fcee0 0x7fa714071e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36608/0 (socket says 172.21.15.37:36608) 2024-03-29T08:03:14.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:14.696+0000 7fa712a00700 1 -- 172.21.15.37:0/3226131088 learned_addr learned my addr 172.21.15.37:0/3226131088 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:03:14.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:14.697+0000 7fa712000700 1 --2- 172.21.15.37:0/3226131088 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa7140fe0d0 0x7fa714070310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:03:14.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:14.697+0000 7fa712a00700 1 -- 172.21.15.37:0/3226131088 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa7140fe0d0 msgr2=0x7fa714070310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:03:14.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:14.697+0000 7fa712a00700 1 --2- 172.21.15.37:0/3226131088 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa7140fe0d0 0x7fa714070310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:03:14.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:14.697+0000 7fa712a00700 1 -- 172.21.15.37:0/3226131088 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa700005040 con 0x7fa7140fcee0 2024-03-29T08:03:14.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:14.697+0000 7fa712000700 1 --2- 172.21.15.37:0/3226131088 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa7140fe0d0 0x7fa714070310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:03:14.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:14.697+0000 7fa712a00700 1 --2- 172.21.15.37:0/3226131088 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa7140fcee0 0x7fa714071e40 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fa704002a80 tx=0x7fa704004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:03:14.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:14.697+0000 7fa70aa00700 1 -- 172.21.15.37:0/3226131088 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa704007cc0 con 0x7fa7140fcee0 2024-03-29T08:03:14.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:14.697+0000 7fa70aa00700 1 -- 172.21.15.37:0/3226131088 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fa70400f040 con 0x7fa7140fcee0 2024-03-29T08:03:14.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:14.697+0000 7fa70aa00700 1 -- 172.21.15.37:0/3226131088 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa70400da20 con 0x7fa7140fcee0 2024-03-29T08:03:14.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:14.697+0000 7fa719600700 1 -- 172.21.15.37:0/3226131088 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa714070d10 con 0x7fa7140fcee0 2024-03-29T08:03:14.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:14.697+0000 7fa719600700 1 -- 172.21.15.37:0/3226131088 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7141159a0 con 0x7fa7140fcee0 2024-03-29T08:03:14.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:14.698+0000 7fa719600700 1 -- 172.21.15.37:0/3226131088 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa71405f5f0 con 0x7fa7140fcee0 2024-03-29T08:03:14.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:14.699+0000 7fa70aa00700 1 -- 172.21.15.37:0/3226131088 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fa70400db80 con 0x7fa7140fcee0 2024-03-29T08:03:14.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:14.700+0000 7fa70aa00700 1 --2- 172.21.15.37:0/3226131088 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7fa6f80692b0 0x7fa6f806b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:03:14.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:14.700+0000 7fa70aa00700 1 -- 172.21.15.37:0/3226131088 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fa704086c10 con 0x7fa7140fcee0 2024-03-29T08:03:14.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:14.700+0000 7fa712000700 1 --2- 172.21.15.37:0/3226131088 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7fa6f80692b0 0x7fa6f806b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:03:14.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:14.701+0000 7fa712000700 1 --2- 172.21.15.37:0/3226131088 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7fa6f80692b0 0x7fa6f806b760 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fa70000b900 tx=0x7fa700009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:03:14.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:14.704+0000 7fa70aa00700 1 -- 172.21.15.37:0/3226131088 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fa7040531e0 con 0x7fa7140fcee0 2024-03-29T08:03:14.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:14.917+0000 7fa719600700 1 -- 172.21.15.37:0/3226131088 --> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fa714102570 con 0x7fa6f80692b0 2024-03-29T08:03:14.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:14.926+0000 7fa70aa00700 1 -- 172.21.15.37:0/3226131088 <== mgr.14208 v2:172.21.15.37:6800/1802836043 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+2718 (secure 0 0 0) 0x7fa714102570 con 0x7fa6f80692b0 2024-03-29T08:03:14.927 INFO:teuthology.orchestra.run.smithi037.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-29T08:03:14.927 INFO:teuthology.orchestra.run.smithi037.stdout:alertmanager.smithi037 smithi037 *:9093,9094 running (4m) 2m ago 6m 13.6M - 0.25.0 c8568f914cd2 a6824c801465 2024-03-29T08:03:14.927 INFO:teuthology.orchestra.run.smithi037.stdout:crash.smithi037 smithi037 running (6m) 2m ago 6m 6651k - 17.2.7-789-g4213e17c cf22eb8683be e0343935b25e 2024-03-29T08:03:14.927 INFO:teuthology.orchestra.run.smithi037.stdout:crash.smithi148 smithi148 running (5m) 71s ago 5m 6656k - 17.2.7-789-g4213e17c cf22eb8683be ff5164c552c1 2024-03-29T08:03:14.927 INFO:teuthology.orchestra.run.smithi037.stdout:grafana.smithi037 smithi037 *:3000 running (4m) 2m ago 5m 74.5M - 9.4.7 954c08fa6188 92b77c060e7a 2024-03-29T08:03:14.928 INFO:teuthology.orchestra.run.smithi037.stdout:mgr.smithi037.wjikii smithi037 *:9283 running (7m) 2m ago 7m 472M - 17.2.7-789-g4213e17c cf22eb8683be 60172b53efa2 2024-03-29T08:03:14.928 INFO:teuthology.orchestra.run.smithi037.stdout:mgr.smithi148.onhgvm smithi148 *:8443,9283 running (4m) 71s ago 4m 430M - 17.2.7-789-g4213e17c cf22eb8683be a0fe1c4dd2b0 2024-03-29T08:03:14.928 INFO:teuthology.orchestra.run.smithi037.stdout:mon.smithi037 smithi037 running (7m) 2m ago 7m 41.9M 2048M 17.2.7-789-g4213e17c cf22eb8683be bb2101505730 2024-03-29T08:03:14.928 INFO:teuthology.orchestra.run.smithi037.stdout:mon.smithi148 smithi148 running (4m) 71s ago 4m 37.5M 2048M 17.2.7-789-g4213e17c cf22eb8683be 18839ed2952a 2024-03-29T08:03:14.928 INFO:teuthology.orchestra.run.smithi037.stdout:node-exporter.smithi037 smithi037 *:9100 running (6m) 2m ago 6m 7402k - 1.5.0 0da6a335fe13 eb3242f9dc57 2024-03-29T08:03:14.928 INFO:teuthology.orchestra.run.smithi037.stdout:node-exporter.smithi148 smithi148 *:9100 running (4m) 71s ago 4m 7922k - 1.5.0 0da6a335fe13 bb455992ed53 2024-03-29T08:03:14.928 INFO:teuthology.orchestra.run.smithi037.stdout:osd.0 smithi037 running (3m) 2m ago 3m 38.8M 4350M 17.2.7-789-g4213e17c cf22eb8683be 559fccd0ebe1 2024-03-29T08:03:14.928 INFO:teuthology.orchestra.run.smithi037.stdout:osd.1 smithi037 running (3m) 2m ago 3m 38.8M 4350M 17.2.7-789-g4213e17c cf22eb8683be 5582cff11b7a 2024-03-29T08:03:14.928 INFO:teuthology.orchestra.run.smithi037.stdout:osd.2 smithi037 running (2m) 2m ago 2m 38.0M 4350M 17.2.7-789-g4213e17c cf22eb8683be f809bfab13e5 2024-03-29T08:03:14.928 INFO:teuthology.orchestra.run.smithi037.stdout:osd.3 smithi148 running (2m) 71s ago 2m 38.4M 5372M 17.2.7-789-g4213e17c cf22eb8683be a0a8224bb4b0 2024-03-29T08:03:14.928 INFO:teuthology.orchestra.run.smithi037.stdout:osd.4 smithi148 running (109s) 71s ago 108s 36.6M 5372M 17.2.7-789-g4213e17c cf22eb8683be a00b69047ed7 2024-03-29T08:03:14.928 INFO:teuthology.orchestra.run.smithi037.stdout:osd.5 smithi148 running (77s) 71s ago 76s 10.5M 5372M 17.2.7-789-g4213e17c cf22eb8683be 044a3f7db7f8 2024-03-29T08:03:14.928 INFO:teuthology.orchestra.run.smithi037.stdout:prometheus.smithi037 smithi037 *:9095 running (4m) 2m ago 5m 35.2M - 2.43.0 a07b618ecd1d e8c23666a746 2024-03-29T08:03:14.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:14.929+0000 7fa719600700 1 -- 172.21.15.37:0/3226131088 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7fa6f80692b0 msgr2=0x7fa6f806b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:03:14.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:14.929+0000 7fa719600700 1 --2- 172.21.15.37:0/3226131088 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7fa6f80692b0 0x7fa6f806b760 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fa70000b900 tx=0x7fa700009e30 comp rx=0 tx=0).stop 2024-03-29T08:03:14.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:14.929+0000 7fa719600700 1 -- 172.21.15.37:0/3226131088 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa7140fcee0 msgr2=0x7fa714071e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:03:14.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:14.929+0000 7fa719600700 1 --2- 172.21.15.37:0/3226131088 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa7140fcee0 0x7fa714071e40 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fa704002a80 tx=0x7fa704004600 comp rx=0 tx=0).stop 2024-03-29T08:03:14.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:14.929+0000 7fa719600700 1 -- 172.21.15.37:0/3226131088 shutdown_connections 2024-03-29T08:03:14.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:14.929+0000 7fa719600700 1 --2- 172.21.15.37:0/3226131088 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7fa6f80692b0 0x7fa6f806b760 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:03:14.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:14.929+0000 7fa719600700 1 --2- 172.21.15.37:0/3226131088 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa7140fcee0 0x7fa714071e40 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:03:14.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:14.929+0000 7fa719600700 1 --2- 172.21.15.37:0/3226131088 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa7140fe0d0 0x7fa714070310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:03:14.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:14.929+0000 7fa719600700 1 -- 172.21.15.37:0/3226131088 >> 172.21.15.37:0/3226131088 conn(0x7fa7140f8560 msgr2=0x7fa7140f9d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:03:14.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:14.929+0000 7fa719600700 1 -- 172.21.15.37:0/3226131088 shutdown_connections 2024-03-29T08:03:14.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:14.929+0000 7fa719600700 1 -- 172.21.15.37:0/3226131088 wait complete. 2024-03-29T08:03:15.055 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:03:14 smithi148 ceph-mon[44992]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:03:15.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:03:14 smithi037 ceph-mon[35162]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:03:15.530 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 99b492c2-eda1-11ee-b647-cb9ed24678a4 -- bash -c 'ceph orch ls' 2024-03-29T08:03:15.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:03:15 smithi037 ceph-mon[35162]: from='client.14446 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:03:15.968 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/mon.smithi037/config 2024-03-29T08:03:16.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:03:15 smithi148 ceph-mon[44992]: from='client.14446 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:03:17.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:03:16 smithi148 ceph-mon[44992]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:03:17.144 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:03:16 smithi037 ceph-mon[35162]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:03:17.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:17.793+0000 7ff974000700 1 -- 172.21.15.37:0/44858839 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff96c0fe0d0 msgr2=0x7ff96c0fe540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:03:17.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:17.793+0000 7ff974000700 1 --2- 172.21.15.37:0/44858839 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff96c0fe0d0 0x7ff96c0fe540 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7ff958004940 tx=0x7ff958009e30 comp rx=0 tx=0).stop 2024-03-29T08:03:17.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:17.793+0000 7ff974000700 1 -- 172.21.15.37:0/44858839 shutdown_connections 2024-03-29T08:03:17.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:17.793+0000 7ff974000700 1 --2- 172.21.15.37:0/44858839 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff96c0fe0d0 0x7ff96c0fe540 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:03:17.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:17.793+0000 7ff974000700 1 --2- 172.21.15.37:0/44858839 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff96c0fcee0 0x7ff96c0fd310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:03:17.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:17.793+0000 7ff974000700 1 -- 172.21.15.37:0/44858839 >> 172.21.15.37:0/44858839 conn(0x7ff96c0f8560 msgr2=0x7ff96c0fa970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:03:17.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:17.793+0000 7ff974000700 1 -- 172.21.15.37:0/44858839 shutdown_connections 2024-03-29T08:03:17.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:17.793+0000 7ff974000700 1 -- 172.21.15.37:0/44858839 wait complete. 2024-03-29T08:03:17.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:17.794+0000 7ff974000700 1 Processor -- start 2024-03-29T08:03:17.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:17.794+0000 7ff974000700 1 -- start start 2024-03-29T08:03:17.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:17.794+0000 7ff974000700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff96c0fcee0 0x7ff96c10faf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:03:17.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:17.794+0000 7ff974000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff96c0fe0d0 0x7ff96c110030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:03:17.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:17.794+0000 7ff974000700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff96c110630 con 0x7ff96c0fe0d0 2024-03-29T08:03:17.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:17.794+0000 7ff974000700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff96c110e80 con 0x7ff96c0fcee0 2024-03-29T08:03:17.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:17.795+0000 7ff970c00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff96c0fe0d0 0x7ff96c110030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:03:17.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:17.795+0000 7ff970c00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff96c0fe0d0 0x7ff96c110030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36614/0 (socket says 172.21.15.37:36614) 2024-03-29T08:03:17.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:17.795+0000 7ff970c00700 1 -- 172.21.15.37:0/1589749517 learned_addr learned my addr 172.21.15.37:0/1589749517 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:03:17.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:17.795+0000 7ff971600700 1 --2- 172.21.15.37:0/1589749517 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff96c0fcee0 0x7ff96c10faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:03:17.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:17.795+0000 7ff970c00700 1 -- 172.21.15.37:0/1589749517 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff96c0fcee0 msgr2=0x7ff96c10faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:03:17.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:17.795+0000 7ff970c00700 1 --2- 172.21.15.37:0/1589749517 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff96c0fcee0 0x7ff96c10faf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:03:17.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:17.795+0000 7ff970c00700 1 -- 172.21.15.37:0/1589749517 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff958005040 con 0x7ff96c0fe0d0 2024-03-29T08:03:17.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:17.795+0000 7ff970c00700 1 --2- 172.21.15.37:0/1589749517 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff96c0fe0d0 0x7ff96c110030 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7ff958009380 tx=0x7ff958009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:03:17.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:17.796+0000 7ff969600700 1 -- 172.21.15.37:0/1589749517 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff95800e040 con 0x7ff96c0fe0d0 2024-03-29T08:03:17.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:17.796+0000 7ff969600700 1 -- 172.21.15.37:0/1589749517 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ff958004290 con 0x7ff96c0fe0d0 2024-03-29T08:03:17.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:17.796+0000 7ff974000700 1 -- 172.21.15.37:0/1589749517 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff96c111100 con 0x7ff96c0fe0d0 2024-03-29T08:03:17.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:17.796+0000 7ff969600700 1 -- 172.21.15.37:0/1589749517 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff95801c3f0 con 0x7ff96c0fe0d0 2024-03-29T08:03:17.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:17.796+0000 7ff974000700 1 -- 172.21.15.37:0/1589749517 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff96c111640 con 0x7ff96c0fe0d0 2024-03-29T08:03:17.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:17.797+0000 7ff974000700 1 -- 172.21.15.37:0/1589749517 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff96c101a60 con 0x7ff96c0fe0d0 2024-03-29T08:03:17.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:17.798+0000 7ff969600700 1 -- 172.21.15.37:0/1589749517 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7ff958013420 con 0x7ff96c0fe0d0 2024-03-29T08:03:17.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:17.799+0000 7ff969600700 1 --2- 172.21.15.37:0/1589749517 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7ff9540692b0 0x7ff95406b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:03:17.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:17.799+0000 7ff969600700 1 -- 172.21.15.37:0/1589749517 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7ff958088220 con 0x7ff96c0fe0d0 2024-03-29T08:03:17.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:17.799+0000 7ff971600700 1 --2- 172.21.15.37:0/1589749517 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7ff9540692b0 0x7ff95406b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:03:17.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:17.800+0000 7ff971600700 1 --2- 172.21.15.37:0/1589749517 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7ff9540692b0 0x7ff95406b760 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ff96c0fcc20 tx=0x7ff960007ea0 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:03:17.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:17.803+0000 7ff969600700 1 -- 172.21.15.37:0/1589749517 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7ff958058300 con 0x7ff96c0fe0d0 2024-03-29T08:03:18.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:18.015+0000 7ff974000700 1 -- 172.21.15.37:0/1589749517 --> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) v1 -- 0x7ff96c1118b0 con 0x7ff9540692b0 2024-03-29T08:03:18.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:18.019+0000 7ff969600700 1 -- 172.21.15.37:0/1589749517 <== mgr.14208 v2:172.21.15.37:6800/1802836043 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+1170 (secure 0 0 0) 0x7ff96c1118b0 con 0x7ff9540692b0 2024-03-29T08:03:18.020 INFO:teuthology.orchestra.run.smithi037.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-03-29T08:03:18.020 INFO:teuthology.orchestra.run.smithi037.stdout:alertmanager ?:9093,9094 1/1 2m ago 6m count:1 2024-03-29T08:03:18.020 INFO:teuthology.orchestra.run.smithi037.stdout:crash 2/2 2m ago 7m * 2024-03-29T08:03:18.020 INFO:teuthology.orchestra.run.smithi037.stdout:grafana ?:3000 1/1 2m ago 6m count:1 2024-03-29T08:03:18.020 INFO:teuthology.orchestra.run.smithi037.stdout:mgr 2/2 2m ago 7m count:2 2024-03-29T08:03:18.020 INFO:teuthology.orchestra.run.smithi037.stdout:mon 2/2 2m ago 6m smithi037:172.21.15.37=smithi037;smithi148:172.21.15.148=smithi148;count:2 2024-03-29T08:03:18.020 INFO:teuthology.orchestra.run.smithi037.stdout:node-exporter ?:9100 2/2 2m ago 6m * 2024-03-29T08:03:18.020 INFO:teuthology.orchestra.run.smithi037.stdout:osd 6 2m ago - 2024-03-29T08:03:18.020 INFO:teuthology.orchestra.run.smithi037.stdout:prometheus ?:9095 1/1 2m ago 6m count:1 2024-03-29T08:03:18.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:18.022+0000 7ff974000700 1 -- 172.21.15.37:0/1589749517 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7ff9540692b0 msgr2=0x7ff95406b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:03:18.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:18.022+0000 7ff974000700 1 --2- 172.21.15.37:0/1589749517 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7ff9540692b0 0x7ff95406b760 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ff96c0fcc20 tx=0x7ff960007ea0 comp rx=0 tx=0).stop 2024-03-29T08:03:18.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:18.022+0000 7ff974000700 1 -- 172.21.15.37:0/1589749517 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff96c0fe0d0 msgr2=0x7ff96c110030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:03:18.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:18.022+0000 7ff974000700 1 --2- 172.21.15.37:0/1589749517 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff96c0fe0d0 0x7ff96c110030 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7ff958009380 tx=0x7ff958009e30 comp rx=0 tx=0).stop 2024-03-29T08:03:18.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:18.022+0000 7ff974000700 1 -- 172.21.15.37:0/1589749517 shutdown_connections 2024-03-29T08:03:18.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:18.022+0000 7ff974000700 1 --2- 172.21.15.37:0/1589749517 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7ff9540692b0 0x7ff95406b760 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:03:18.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:18.022+0000 7ff974000700 1 --2- 172.21.15.37:0/1589749517 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff96c0fcee0 0x7ff96c10faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:03:18.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:18.022+0000 7ff974000700 1 --2- 172.21.15.37:0/1589749517 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff96c0fe0d0 0x7ff96c110030 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:03:18.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:18.022+0000 7ff974000700 1 -- 172.21.15.37:0/1589749517 >> 172.21.15.37:0/1589749517 conn(0x7ff96c0f8560 msgr2=0x7ff96c0f9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:03:18.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:18.022+0000 7ff974000700 1 -- 172.21.15.37:0/1589749517 shutdown_connections 2024-03-29T08:03:18.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:18.022+0000 7ff974000700 1 -- 172.21.15.37:0/1589749517 wait complete. 2024-03-29T08:03:18.698 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 99b492c2-eda1-11ee-b647-cb9ed24678a4 -- bash -c 'ceph orch host ls' 2024-03-29T08:03:18.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:03:18 smithi037 ceph-mon[35162]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:03:19.055 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:03:18 smithi148 ceph-mon[44992]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:03:19.129 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/mon.smithi037/config 2024-03-29T08:03:20.043 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:03:19 smithi037 ceph-mon[35162]: from='client.14450 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:03:20.055 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:03:19 smithi148 ceph-mon[44992]: from='client.14450 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:03:20.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:20.998+0000 7f0395e00700 1 -- 172.21.15.37:0/3134259431 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f03900fe0d0 msgr2=0x7f03900fe540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:03:20.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:20.998+0000 7f0395e00700 1 --2- 172.21.15.37:0/3134259431 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f03900fe0d0 0x7f03900fe540 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f037c004940 tx=0x7f037c009e30 comp rx=0 tx=0).stop 2024-03-29T08:03:20.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:20.998+0000 7f0395e00700 1 -- 172.21.15.37:0/3134259431 shutdown_connections 2024-03-29T08:03:20.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:20.998+0000 7f0395e00700 1 --2- 172.21.15.37:0/3134259431 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f03900fe0d0 0x7f03900fe540 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:03:20.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:20.998+0000 7f0395e00700 1 --2- 172.21.15.37:0/3134259431 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f03900fcee0 0x7f03900fd310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:03:20.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:20.998+0000 7f0395e00700 1 -- 172.21.15.37:0/3134259431 >> 172.21.15.37:0/3134259431 conn(0x7f03900f8560 msgr2=0x7f03900fa970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:03:20.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:20.998+0000 7f0395e00700 1 -- 172.21.15.37:0/3134259431 shutdown_connections 2024-03-29T08:03:20.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:20.998+0000 7f0395e00700 1 -- 172.21.15.37:0/3134259431 wait complete. 2024-03-29T08:03:20.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:20.999+0000 7f0395e00700 1 Processor -- start 2024-03-29T08:03:20.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:20.999+0000 7f0395e00700 1 -- start start 2024-03-29T08:03:21.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:21.000+0000 7f0395e00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f03900fcee0 0x7f039010fb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:03:21.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:21.000+0000 7f0395e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f03900fe0d0 0x7f03901100d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:03:21.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:21.000+0000 7f0395e00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0390110760 con 0x7f03900fe0d0 2024-03-29T08:03:21.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:21.000+0000 7f0395e00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0390110f50 con 0x7f03900fcee0 2024-03-29T08:03:21.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:21.000+0000 7f038e000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f03900fe0d0 0x7f03901100d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:03:21.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:21.000+0000 7f038e000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f03900fe0d0 0x7f03901100d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55078/0 (socket says 172.21.15.37:55078) 2024-03-29T08:03:21.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:21.000+0000 7f038e000700 1 -- 172.21.15.37:0/4014713553 learned_addr learned my addr 172.21.15.37:0/4014713553 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:03:21.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:21.000+0000 7f038ea00700 1 --2- 172.21.15.37:0/4014713553 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f03900fcee0 0x7f039010fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:03:21.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:21.000+0000 7f038e000700 1 -- 172.21.15.37:0/4014713553 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f03900fcee0 msgr2=0x7f039010fb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:03:21.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:21.000+0000 7f038e000700 1 --2- 172.21.15.37:0/4014713553 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f03900fcee0 0x7f039010fb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:03:21.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:21.000+0000 7f038e000700 1 -- 172.21.15.37:0/4014713553 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f037c005040 con 0x7f03900fe0d0 2024-03-29T08:03:21.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:21.000+0000 7f038ea00700 1 --2- 172.21.15.37:0/4014713553 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f03900fcee0 0x7f039010fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:03:21.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:21.001+0000 7f038e000700 1 --2- 172.21.15.37:0/4014713553 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f03900fe0d0 0x7f03901100d0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f037c009380 tx=0x7f037c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:03:21.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:21.001+0000 7f0386a00700 1 -- 172.21.15.37:0/4014713553 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f037c00e040 con 0x7f03900fe0d0 2024-03-29T08:03:21.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:21.001+0000 7f0386a00700 1 -- 172.21.15.37:0/4014713553 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f037c004290 con 0x7f03900fe0d0 2024-03-29T08:03:21.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:21.001+0000 7f0395e00700 1 -- 172.21.15.37:0/4014713553 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03901111d0 con 0x7f03900fe0d0 2024-03-29T08:03:21.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:21.001+0000 7f0386a00700 1 -- 172.21.15.37:0/4014713553 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f037c01c3f0 con 0x7f03900fe0d0 2024-03-29T08:03:21.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:21.001+0000 7f0395e00700 1 -- 172.21.15.37:0/4014713553 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0390111790 con 0x7f03900fe0d0 2024-03-29T08:03:21.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:21.002+0000 7f0395e00700 1 -- 172.21.15.37:0/4014713553 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0390101a60 con 0x7f03900fe0d0 2024-03-29T08:03:21.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:21.003+0000 7f0386a00700 1 -- 172.21.15.37:0/4014713553 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f037c013420 con 0x7f03900fe0d0 2024-03-29T08:03:21.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:21.004+0000 7f0386a00700 1 --2- 172.21.15.37:0/4014713553 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f03740692b0 0x7f037406b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:03:21.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:21.004+0000 7f0386a00700 1 -- 172.21.15.37:0/4014713553 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f037c088230 con 0x7f03900fe0d0 2024-03-29T08:03:21.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:21.004+0000 7f038ea00700 1 --2- 172.21.15.37:0/4014713553 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f03740692b0 0x7f037406b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:03:21.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:21.004+0000 7f038ea00700 1 --2- 172.21.15.37:0/4014713553 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f03740692b0 0x7f037406b760 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f03900fcc20 tx=0x7f0380008040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:03:21.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:21.008+0000 7f0386a00700 1 -- 172.21.15.37:0/4014713553 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f037c0583c0 con 0x7f03900fe0d0 2024-03-29T08:03:21.055 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:03:20 smithi148 ceph-mon[44992]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:03:21.176 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:03:20 smithi037 ceph-mon[35162]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:03:21.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:21.219+0000 7f0395e00700 1 -- 172.21.15.37:0/4014713553 --> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f03900008d0 con 0x7f03740692b0 2024-03-29T08:03:21.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:21.220+0000 7f0386a00700 1 -- 172.21.15.37:0/4014713553 <== mgr.14208 v2:172.21.15.37:6800/1802836043 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+148 (secure 0 0 0) 0x7f03900008d0 con 0x7f03740692b0 2024-03-29T08:03:21.220 INFO:teuthology.orchestra.run.smithi037.stdout:HOST ADDR LABELS STATUS 2024-03-29T08:03:21.220 INFO:teuthology.orchestra.run.smithi037.stdout:smithi037 172.21.15.37 2024-03-29T08:03:21.220 INFO:teuthology.orchestra.run.smithi037.stdout:smithi148 172.21.15.148 2024-03-29T08:03:21.220 INFO:teuthology.orchestra.run.smithi037.stdout:2 hosts in cluster 2024-03-29T08:03:21.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:21.222+0000 7f0395e00700 1 -- 172.21.15.37:0/4014713553 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f03740692b0 msgr2=0x7f037406b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:03:21.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:21.222+0000 7f0395e00700 1 --2- 172.21.15.37:0/4014713553 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f03740692b0 0x7f037406b760 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f03900fcc20 tx=0x7f0380008040 comp rx=0 tx=0).stop 2024-03-29T08:03:21.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:21.222+0000 7f0395e00700 1 -- 172.21.15.37:0/4014713553 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f03900fe0d0 msgr2=0x7f03901100d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:03:21.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:21.222+0000 7f0395e00700 1 --2- 172.21.15.37:0/4014713553 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f03900fe0d0 0x7f03901100d0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f037c009380 tx=0x7f037c009e30 comp rx=0 tx=0).stop 2024-03-29T08:03:21.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:21.223+0000 7f0395e00700 1 -- 172.21.15.37:0/4014713553 shutdown_connections 2024-03-29T08:03:21.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:21.223+0000 7f0395e00700 1 --2- 172.21.15.37:0/4014713553 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f03740692b0 0x7f037406b760 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:03:21.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:21.223+0000 7f0395e00700 1 --2- 172.21.15.37:0/4014713553 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f03900fcee0 0x7f039010fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:03:21.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:21.223+0000 7f0395e00700 1 --2- 172.21.15.37:0/4014713553 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f03900fe0d0 0x7f03901100d0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:03:21.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:21.223+0000 7f0395e00700 1 -- 172.21.15.37:0/4014713553 >> 172.21.15.37:0/4014713553 conn(0x7f03900f8560 msgr2=0x7f03900f9d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:03:21.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:21.223+0000 7f0395e00700 1 -- 172.21.15.37:0/4014713553 shutdown_connections 2024-03-29T08:03:21.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:21.223+0000 7f0395e00700 1 -- 172.21.15.37:0/4014713553 wait complete. 2024-03-29T08:03:21.913 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 99b492c2-eda1-11ee-b647-cb9ed24678a4 -- bash -c 'ceph orch device ls' 2024-03-29T08:03:22.345 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/mon.smithi037/config 2024-03-29T08:03:22.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:03:22 smithi037 ceph-mon[35162]: from='client.14454 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:03:22.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:03:22 smithi037 ceph-mon[35162]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:03:23.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:03:22 smithi148 ceph-mon[44992]: from='client.14454 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:03:23.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:03:22 smithi148 ceph-mon[44992]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:03:24.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:24.254+0000 7f916b000700 1 -- 172.21.15.37:0/209348487 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f91640709a0 msgr2=0x7f9164070e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:03:24.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:24.254+0000 7f916b000700 1 --2- 172.21.15.37:0/209348487 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f91640709a0 0x7f9164070e10 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f9150004940 tx=0x7f9150009e30 comp rx=0 tx=0).stop 2024-03-29T08:03:24.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:24.254+0000 7f916b000700 1 -- 172.21.15.37:0/209348487 shutdown_connections 2024-03-29T08:03:24.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:24.254+0000 7f916b000700 1 --2- 172.21.15.37:0/209348487 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f91640709a0 0x7f9164070e10 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:03:24.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:24.254+0000 7f916b000700 1 --2- 172.21.15.37:0/209348487 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9164071f50 0x7f9164070460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:03:24.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:24.254+0000 7f916b000700 1 -- 172.21.15.37:0/209348487 >> 172.21.15.37:0/209348487 conn(0x7f91640f8560 msgr2=0x7f91640fa970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:03:24.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:24.254+0000 7f916b000700 1 -- 172.21.15.37:0/209348487 shutdown_connections 2024-03-29T08:03:24.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:24.254+0000 7f916b000700 1 -- 172.21.15.37:0/209348487 wait complete. 2024-03-29T08:03:24.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:24.255+0000 7f916b000700 1 Processor -- start 2024-03-29T08:03:24.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:24.255+0000 7f916b000700 1 -- start start 2024-03-29T08:03:24.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:24.255+0000 7f916b000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f91640709a0 0x7f916410fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:03:24.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:24.255+0000 7f916b000700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9164071f50 0x7f91641100c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:03:24.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:24.255+0000 7f916b000700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91641106c0 con 0x7f91640709a0 2024-03-29T08:03:24.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:24.255+0000 7f916b000700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9164110f10 con 0x7f9164071f50 2024-03-29T08:03:24.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:24.256+0000 7f9163e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f91640709a0 0x7f916410fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:03:24.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:24.256+0000 7f9163e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f91640709a0 0x7f916410fb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55084/0 (socket says 172.21.15.37:55084) 2024-03-29T08:03:24.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:24.256+0000 7f9163e00700 1 -- 172.21.15.37:0/1367083577 learned_addr learned my addr 172.21.15.37:0/1367083577 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:03:24.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:24.256+0000 7f9163400700 1 --2- 172.21.15.37:0/1367083577 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9164071f50 0x7f91641100c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:03:24.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:24.256+0000 7f9163e00700 1 -- 172.21.15.37:0/1367083577 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9164071f50 msgr2=0x7f91641100c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:03:24.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:24.256+0000 7f9163e00700 1 --2- 172.21.15.37:0/1367083577 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9164071f50 0x7f91641100c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:03:24.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:24.256+0000 7f9163e00700 1 -- 172.21.15.37:0/1367083577 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9150005040 con 0x7f91640709a0 2024-03-29T08:03:24.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:24.256+0000 7f9163e00700 1 --2- 172.21.15.37:0/1367083577 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f91640709a0 0x7f916410fb80 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f9154002a50 tx=0x7f9154004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:03:24.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:24.256+0000 7f915be00700 1 -- 172.21.15.37:0/1367083577 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9154007cc0 con 0x7f91640709a0 2024-03-29T08:03:24.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:24.256+0000 7f915be00700 1 -- 172.21.15.37:0/1367083577 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f915400f040 con 0x7f91640709a0 2024-03-29T08:03:24.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:24.256+0000 7f915be00700 1 -- 172.21.15.37:0/1367083577 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f915400d990 con 0x7f91640709a0 2024-03-29T08:03:24.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:24.257+0000 7f916b000700 1 -- 172.21.15.37:0/1367083577 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91641111f0 con 0x7f91640709a0 2024-03-29T08:03:24.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:24.257+0000 7f916b000700 1 -- 172.21.15.37:0/1367083577 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91641116e0 con 0x7f91640709a0 2024-03-29T08:03:24.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:24.258+0000 7f916b000700 1 -- 172.21.15.37:0/1367083577 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f916405f5f0 con 0x7f91640709a0 2024-03-29T08:03:24.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:24.259+0000 7f915be00700 1 -- 172.21.15.37:0/1367083577 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f9154009710 con 0x7f91640709a0 2024-03-29T08:03:24.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:24.259+0000 7f915be00700 1 --2- 172.21.15.37:0/1367083577 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f91480692b0 0x7f914806b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:03:24.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:24.259+0000 7f915be00700 1 -- 172.21.15.37:0/1367083577 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f9154086c40 con 0x7f91640709a0 2024-03-29T08:03:24.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:24.260+0000 7f9163400700 1 --2- 172.21.15.37:0/1367083577 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f91480692b0 0x7f914806b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:03:24.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:24.260+0000 7f9163400700 1 --2- 172.21.15.37:0/1367083577 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f91480692b0 0x7f914806b760 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f915000b3b0 tx=0x7f9150009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:03:24.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:24.263+0000 7f915be00700 1 -- 172.21.15.37:0/1367083577 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f9154056990 con 0x7f91640709a0 2024-03-29T08:03:24.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:24.476+0000 7f916b000700 1 -- 172.21.15.37:0/1367083577 --> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f9164111b10 con 0x7f91480692b0 2024-03-29T08:03:24.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:24.478+0000 7f915be00700 1 -- 172.21.15.37:0/1367083577 <== mgr.14208 v2:172.21.15.37:6800/1802836043 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+399 (secure 0 0 0) 0x7f9164111b10 con 0x7f91480692b0 2024-03-29T08:03:24.478 INFO:teuthology.orchestra.run.smithi037.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-03-29T08:03:24.478 INFO:teuthology.orchestra.run.smithi037.stdout:smithi037 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT5331005T400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-03-29T08:03:24.478 INFO:teuthology.orchestra.run.smithi037.stdout:smithi148 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT6233005G400BGN 372G No 76s ago Has a FileSystem, LVM detected 2024-03-29T08:03:24.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:24.480+0000 7f916b000700 1 -- 172.21.15.37:0/1367083577 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f91480692b0 msgr2=0x7f914806b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:03:24.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:24.480+0000 7f916b000700 1 --2- 172.21.15.37:0/1367083577 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f91480692b0 0x7f914806b760 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f915000b3b0 tx=0x7f9150009e30 comp rx=0 tx=0).stop 2024-03-29T08:03:24.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:24.480+0000 7f916b000700 1 -- 172.21.15.37:0/1367083577 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f91640709a0 msgr2=0x7f916410fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:03:24.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:24.480+0000 7f916b000700 1 --2- 172.21.15.37:0/1367083577 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f91640709a0 0x7f916410fb80 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f9154002a50 tx=0x7f9154004600 comp rx=0 tx=0).stop 2024-03-29T08:03:24.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:24.480+0000 7f916b000700 1 -- 172.21.15.37:0/1367083577 shutdown_connections 2024-03-29T08:03:24.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:24.480+0000 7f916b000700 1 --2- 172.21.15.37:0/1367083577 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f91480692b0 0x7f914806b760 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:03:24.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:24.480+0000 7f916b000700 1 --2- 172.21.15.37:0/1367083577 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f91640709a0 0x7f916410fb80 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:03:24.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:24.480+0000 7f916b000700 1 --2- 172.21.15.37:0/1367083577 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9164071f50 0x7f91641100c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:03:24.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:24.480+0000 7f916b000700 1 -- 172.21.15.37:0/1367083577 >> 172.21.15.37:0/1367083577 conn(0x7f91640f8560 msgr2=0x7f91640f9d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:03:24.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:24.480+0000 7f916b000700 1 -- 172.21.15.37:0/1367083577 shutdown_connections 2024-03-29T08:03:24.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:24.481+0000 7f916b000700 1 -- 172.21.15.37:0/1367083577 wait complete. 2024-03-29T08:03:25.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:03:24 smithi148 ceph-mon[44992]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:03:25.163 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:03:24 smithi037 ceph-mon[35162]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:03:25.186 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-29T08:03:25.197 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi037.front.sepia.ceph.com 2024-03-29T08:03:25.197 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 99b492c2-eda1-11ee-b647-cb9ed24678a4 -- bash -c 'ceph fs volume create cephfs --placement=4' 2024-03-29T08:03:25.630 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/mon.smithi037/config 2024-03-29T08:03:26.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:03:25 smithi148 ceph-mon[44992]: from='client.14458 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:03:26.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:03:25 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:03:26.070 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:03:25 smithi037 ceph-mon[35162]: from='client.14458 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:03:26.070 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:03:25 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:03:27.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:03:26 smithi148 ceph-mon[44992]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:03:27.089 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:03:26 smithi037 ceph-mon[35162]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:03:27.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:27.410+0000 7f6391600700 1 -- 172.21.15.37:0/3803394678 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f638c0662e0 msgr2=0x7f638c0ff9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:03:27.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:27.410+0000 7f6391600700 1 --2- 172.21.15.37:0/3803394678 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f638c0662e0 0x7f638c0ff9c0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f637c004940 tx=0x7f637c009e30 comp rx=0 tx=0).stop 2024-03-29T08:03:27.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:27.410+0000 7f6391600700 1 -- 172.21.15.37:0/3803394678 shutdown_connections 2024-03-29T08:03:27.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:27.410+0000 7f6391600700 1 --2- 172.21.15.37:0/3803394678 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f638c0fff00 0x7f638c102320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:03:27.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:27.410+0000 7f6391600700 1 --2- 172.21.15.37:0/3803394678 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f638c0662e0 0x7f638c0ff9c0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:03:27.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:27.410+0000 7f6391600700 1 -- 172.21.15.37:0/3803394678 >> 172.21.15.37:0/3803394678 conn(0x7f638c0f7430 msgr2=0x7f638c0f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:03:27.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:27.410+0000 7f6391600700 1 -- 172.21.15.37:0/3803394678 shutdown_connections 2024-03-29T08:03:27.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:27.410+0000 7f6391600700 1 -- 172.21.15.37:0/3803394678 wait complete. 2024-03-29T08:03:27.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:27.411+0000 7f6391600700 1 Processor -- start 2024-03-29T08:03:27.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:27.411+0000 7f6391600700 1 -- start start 2024-03-29T08:03:27.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:27.411+0000 7f6391600700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f638c0662e0 0x7f638c10d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:03:27.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:27.411+0000 7f6391600700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f638c0fff00 0x7f638c10de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:03:27.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:27.411+0000 7f6391600700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f638c10e490 con 0x7f638c0662e0 2024-03-29T08:03:27.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:27.411+0000 7f6391600700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f638c10ece0 con 0x7f638c0fff00 2024-03-29T08:03:27.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:27.411+0000 7f638aa00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f638c0662e0 0x7f638c10d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:03:27.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:27.411+0000 7f638aa00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f638c0662e0 0x7f638c10d950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55104/0 (socket says 172.21.15.37:55104) 2024-03-29T08:03:27.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:27.411+0000 7f638aa00700 1 -- 172.21.15.37:0/3851310297 learned_addr learned my addr 172.21.15.37:0/3851310297 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:03:27.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:27.412+0000 7f638aa00700 1 -- 172.21.15.37:0/3851310297 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f638c0fff00 msgr2=0x7f638c10de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:03:27.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:27.412+0000 7f638aa00700 1 --2- 172.21.15.37:0/3851310297 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f638c0fff00 0x7f638c10de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:03:27.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:27.412+0000 7f638aa00700 1 -- 172.21.15.37:0/3851310297 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f637c005040 con 0x7f638c0662e0 2024-03-29T08:03:27.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:27.412+0000 7f638aa00700 1 --2- 172.21.15.37:0/3851310297 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f638c0662e0 0x7f638c10d950 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f637c00b3b0 tx=0x7f637c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:03:27.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:27.412+0000 7f6382a00700 1 -- 172.21.15.37:0/3851310297 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f637c00e040 con 0x7f638c0662e0 2024-03-29T08:03:27.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:27.412+0000 7f6382a00700 1 -- 172.21.15.37:0/3851310297 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f637c004290 con 0x7f638c0662e0 2024-03-29T08:03:27.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:27.412+0000 7f6391600700 1 -- 172.21.15.37:0/3851310297 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f638c10ef60 con 0x7f638c0662e0 2024-03-29T08:03:27.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:27.412+0000 7f6382a00700 1 -- 172.21.15.37:0/3851310297 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f637c01c3f0 con 0x7f638c0662e0 2024-03-29T08:03:27.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:27.412+0000 7f6391600700 1 -- 172.21.15.37:0/3851310297 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f638c10f4a0 con 0x7f638c0662e0 2024-03-29T08:03:27.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:27.414+0000 7f6391600700 1 -- 172.21.15.37:0/3851310297 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f638c05f5f0 con 0x7f638c0662e0 2024-03-29T08:03:27.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:27.414+0000 7f6382a00700 1 -- 172.21.15.37:0/3851310297 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f637c00bc50 con 0x7f638c0662e0 2024-03-29T08:03:27.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:27.415+0000 7f6382a00700 1 --2- 172.21.15.37:0/3851310297 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f63700692b0 0x7f637006b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:03:27.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:27.415+0000 7f6382a00700 1 -- 172.21.15.37:0/3851310297 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f637c0881c0 con 0x7f638c0662e0 2024-03-29T08:03:27.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:27.415+0000 7f638a000700 1 --2- 172.21.15.37:0/3851310297 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f63700692b0 0x7f637006b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:03:27.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:27.416+0000 7f638a000700 1 --2- 172.21.15.37:0/3851310297 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f63700692b0 0x7f637006b760 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f6378004d00 tx=0x7f6378009ce0 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:03:27.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:27.420+0000 7f6382a00700 1 -- 172.21.15.37:0/3851310297 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f637c0582a0 con 0x7f638c0662e0 2024-03-29T08:03:27.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:27.657+0000 7f6391600700 1 -- 172.21.15.37:0/3851310297 --> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] -- mgr_command(tid 0: {"prefix": "fs volume create", "name": "cephfs", "placement": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7f638c10f8e0 con 0x7f63700692b0 2024-03-29T08:03:29.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:03:28 smithi148 ceph-mon[44992]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:03:29.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:03:28 smithi148 ceph-mon[44992]: from='client.14462 -' entity='client.admin' cmd=[{"prefix": "fs volume create", "name": "cephfs", "placement": "4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:03:29.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:03:28 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]: dispatch 2024-03-29T08:03:29.176 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:03:28 smithi037 ceph-mon[35162]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:03:29.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:03:28 smithi037 ceph-mon[35162]: from='client.14462 -' entity='client.admin' cmd=[{"prefix": "fs volume create", "name": "cephfs", "placement": "4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:03:29.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:03:28 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]: dispatch 2024-03-29T08:03:29.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:29.269+0000 7f6382a00700 1 -- 172.21.15.37:0/3851310297 <== mgr.14208 v2:172.21.15.37:6800/1802836043 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f638c10f8e0 con 0x7f63700692b0 2024-03-29T08:03:29.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:29.272+0000 7f6391600700 1 -- 172.21.15.37:0/3851310297 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f63700692b0 msgr2=0x7f637006b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:03:29.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:29.272+0000 7f6391600700 1 --2- 172.21.15.37:0/3851310297 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f63700692b0 0x7f637006b760 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f6378004d00 tx=0x7f6378009ce0 comp rx=0 tx=0).stop 2024-03-29T08:03:29.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:29.272+0000 7f6391600700 1 -- 172.21.15.37:0/3851310297 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f638c0662e0 msgr2=0x7f638c10d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:03:29.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:29.272+0000 7f6391600700 1 --2- 172.21.15.37:0/3851310297 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f638c0662e0 0x7f638c10d950 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f637c00b3b0 tx=0x7f637c009e30 comp rx=0 tx=0).stop 2024-03-29T08:03:29.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:29.272+0000 7f6391600700 1 -- 172.21.15.37:0/3851310297 shutdown_connections 2024-03-29T08:03:29.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:29.272+0000 7f6391600700 1 --2- 172.21.15.37:0/3851310297 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f63700692b0 0x7f637006b760 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:03:29.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:29.272+0000 7f6391600700 1 --2- 172.21.15.37:0/3851310297 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f638c0662e0 0x7f638c10d950 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:03:29.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:29.272+0000 7f6391600700 1 --2- 172.21.15.37:0/3851310297 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f638c0fff00 0x7f638c10de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:03:29.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:29.272+0000 7f6391600700 1 -- 172.21.15.37:0/3851310297 >> 172.21.15.37:0/3851310297 conn(0x7f638c0f7430 msgr2=0x7f638c0f7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:03:29.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:29.272+0000 7f6391600700 1 -- 172.21.15.37:0/3851310297 shutdown_connections 2024-03-29T08:03:29.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:29.272+0000 7f6391600700 1 -- 172.21.15.37:0/3851310297 wait complete. 2024-03-29T08:03:29.676 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:03:29 smithi037 ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4-mon-smithi037[35139]: 2024-03-29T08:03:29.240+0000 7f3e8f600700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-29T08:03:29.985 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:03:29 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd='[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]': finished 2024-03-29T08:03:29.985 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:03:29 smithi037 ceph-mon[35162]: osdmap e42: 6 total, 6 up, 6 in 2024-03-29T08:03:29.985 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:03:29 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]: dispatch 2024-03-29T08:03:29.985 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:03:29 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd='[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]': finished 2024-03-29T08:03:29.985 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:03:29 smithi037 ceph-mon[35162]: osdmap e43: 6 total, 6 up, 6 in 2024-03-29T08:03:29.985 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:03:29 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]: dispatch 2024-03-29T08:03:29.985 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:03:29 smithi037 ceph-mon[35162]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-29T08:03:29.985 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:03:29 smithi037 ceph-mon[35162]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-03-29T08:03:29.985 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:03:29 smithi037 ceph-mon[35162]: osdmap e44: 6 total, 6 up, 6 in 2024-03-29T08:03:29.985 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:03:29 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]': finished 2024-03-29T08:03:29.986 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:03:29 smithi037 ceph-mon[35162]: fsmap cephfs:0 2024-03-29T08:03:29.986 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:03:29 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:03:29.986 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:03:29 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T08:03:29.986 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:03:29 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T08:03:29.986 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:03:29 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T08:03:29.986 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:03:29 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:03:29.986 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:03:29 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi037.mmwgxe", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-29T08:03:29.986 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:03:29 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi037.mmwgxe", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-29T08:03:29.986 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:03:29 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T08:03:30.012 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 99b492c2-eda1-11ee-b647-cb9ed24678a4 -- bash -c 'ceph fs dump' 2024-03-29T08:03:30.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:03:29 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd='[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]': finished 2024-03-29T08:03:30.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:03:29 smithi148 ceph-mon[44992]: osdmap e42: 6 total, 6 up, 6 in 2024-03-29T08:03:30.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:03:29 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]: dispatch 2024-03-29T08:03:30.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:03:29 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd='[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]': finished 2024-03-29T08:03:30.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:03:29 smithi148 ceph-mon[44992]: osdmap e43: 6 total, 6 up, 6 in 2024-03-29T08:03:30.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:03:29 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]: dispatch 2024-03-29T08:03:30.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:03:29 smithi148 ceph-mon[44992]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-29T08:03:30.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:03:29 smithi148 ceph-mon[44992]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-03-29T08:03:30.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:03:29 smithi148 ceph-mon[44992]: osdmap e44: 6 total, 6 up, 6 in 2024-03-29T08:03:30.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:03:29 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]': finished 2024-03-29T08:03:30.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:03:29 smithi148 ceph-mon[44992]: fsmap cephfs:0 2024-03-29T08:03:30.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:03:29 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:03:30.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:03:29 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T08:03:30.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:03:29 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T08:03:30.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:03:29 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T08:03:30.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:03:29 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:03:30.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:03:29 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi037.mmwgxe", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-29T08:03:30.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:03:29 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi037.mmwgxe", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-29T08:03:30.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:03:29 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T08:03:31.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:03:30 smithi148 ceph-mon[44992]: Saving service mds.cephfs spec with placement count:4 2024-03-29T08:03:31.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:03:30 smithi148 ceph-mon[44992]: Deploying daemon mds.cephfs.smithi037.mmwgxe on smithi037 2024-03-29T08:03:31.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:03:30 smithi148 ceph-mon[44992]: pgmap v199: 65 pgs: 64 unknown, 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:03:31.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:03:30 smithi148 ceph-mon[44992]: osdmap e45: 6 total, 6 up, 6 in 2024-03-29T08:03:31.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:03:30 smithi037 ceph-mon[35162]: Saving service mds.cephfs spec with placement count:4 2024-03-29T08:03:31.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:03:30 smithi037 ceph-mon[35162]: Deploying daemon mds.cephfs.smithi037.mmwgxe on smithi037 2024-03-29T08:03:31.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:03:30 smithi037 ceph-mon[35162]: pgmap v199: 65 pgs: 64 unknown, 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:03:31.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:03:30 smithi037 ceph-mon[35162]: osdmap e45: 6 total, 6 up, 6 in 2024-03-29T08:03:31.363 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/mon.smithi037/config 2024-03-29T08:03:32.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:03:32 smithi148 ceph-mon[44992]: osdmap e46: 6 total, 6 up, 6 in 2024-03-29T08:03:32.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:03:32 smithi148 ceph-mon[44992]: pgmap v202: 65 pgs: 4 creating+peering, 43 active+clean, 18 unknown; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:03:32.651 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:03:32 smithi037 ceph-mon[35162]: osdmap e46: 6 total, 6 up, 6 in 2024-03-29T08:03:32.651 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:03:32 smithi037 ceph-mon[35162]: pgmap v202: 65 pgs: 4 creating+peering, 43 active+clean, 18 unknown; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:03:33.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:33.563+0000 7fefe7200700 1 -- 172.21.15.37:0/1161915034 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fefe00fcee0 msgr2=0x7fefe00fd310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:03:33.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:33.563+0000 7fefe7200700 1 --2- 172.21.15.37:0/1161915034 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fefe00fcee0 0x7fefe00fd310 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fefd0004940 tx=0x7fefd0009e30 comp rx=0 tx=0).stop 2024-03-29T08:03:33.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:33.564+0000 7fefe7200700 1 -- 172.21.15.37:0/1161915034 shutdown_connections 2024-03-29T08:03:33.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:33.564+0000 7fefe7200700 1 --2- 172.21.15.37:0/1161915034 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fefe00fe0d0 0x7fefe00fe540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:03:33.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:33.564+0000 7fefe7200700 1 --2- 172.21.15.37:0/1161915034 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fefe00fcee0 0x7fefe00fd310 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:03:33.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:33.564+0000 7fefe7200700 1 -- 172.21.15.37:0/1161915034 >> 172.21.15.37:0/1161915034 conn(0x7fefe00f8560 msgr2=0x7fefe00fa970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:03:33.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:33.564+0000 7fefe7200700 1 -- 172.21.15.37:0/1161915034 shutdown_connections 2024-03-29T08:03:33.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:33.564+0000 7fefe7200700 1 -- 172.21.15.37:0/1161915034 wait complete. 2024-03-29T08:03:33.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:33.564+0000 7fefe7200700 1 Processor -- start 2024-03-29T08:03:33.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:33.565+0000 7fefe7200700 1 -- start start 2024-03-29T08:03:33.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:33.565+0000 7fefe7200700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fefe00fcee0 0x7fefe010fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:03:33.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:33.565+0000 7fefe7200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fefe00fe0d0 0x7fefe0110090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:03:33.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:33.565+0000 7fefe7200700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefe0110690 con 0x7fefe00fe0d0 2024-03-29T08:03:33.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:33.565+0000 7fefe7200700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefe0110ee0 con 0x7fefe00fcee0 2024-03-29T08:03:33.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:33.565+0000 7fefdf400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fefe00fe0d0 0x7fefe0110090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:03:33.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:33.565+0000 7fefdf400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fefe00fe0d0 0x7fefe0110090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:33382/0 (socket says 172.21.15.37:33382) 2024-03-29T08:03:33.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:33.565+0000 7fefdf400700 1 -- 172.21.15.37:0/4022704313 learned_addr learned my addr 172.21.15.37:0/4022704313 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:03:33.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:33.566+0000 7fefdfe00700 1 --2- 172.21.15.37:0/4022704313 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fefe00fcee0 0x7fefe010fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:03:33.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:33.566+0000 7fefdf400700 1 -- 172.21.15.37:0/4022704313 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fefe00fcee0 msgr2=0x7fefe010fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:03:33.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:33.566+0000 7fefdf400700 1 --2- 172.21.15.37:0/4022704313 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fefe00fcee0 0x7fefe010fb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:03:33.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:33.566+0000 7fefdf400700 1 -- 172.21.15.37:0/4022704313 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefd0005040 con 0x7fefe00fe0d0 2024-03-29T08:03:33.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:33.566+0000 7fefdf400700 1 --2- 172.21.15.37:0/4022704313 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fefe00fe0d0 0x7fefe0110090 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fefcc002a50 tx=0x7fefcc0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:03:33.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:33.566+0000 7fefd7e00700 1 -- 172.21.15.37:0/4022704313 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fefcc007cc0 con 0x7fefe00fe0d0 2024-03-29T08:03:33.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:33.566+0000 7fefe7200700 1 -- 172.21.15.37:0/4022704313 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefe01111c0 con 0x7fefe00fe0d0 2024-03-29T08:03:33.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:33.566+0000 7fefd7e00700 1 -- 172.21.15.37:0/4022704313 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fefcc00f040 con 0x7fefe00fe0d0 2024-03-29T08:03:33.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:33.566+0000 7fefd7e00700 1 -- 172.21.15.37:0/4022704313 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fefcc00d9c0 con 0x7fefe00fe0d0 2024-03-29T08:03:33.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:33.566+0000 7fefe7200700 1 -- 172.21.15.37:0/4022704313 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefe0111700 con 0x7fefe00fe0d0 2024-03-29T08:03:33.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:33.568+0000 7fefe7200700 1 -- 172.21.15.37:0/4022704313 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fefe005f5f0 con 0x7fefe00fe0d0 2024-03-29T08:03:33.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:33.569+0000 7fefd7e00700 1 -- 172.21.15.37:0/4022704313 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fefcc00db20 con 0x7fefe00fe0d0 2024-03-29T08:03:33.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:33.570+0000 7fefd7e00700 1 --2- 172.21.15.37:0/4022704313 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7fefc40692a0 0x7fefc406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:03:33.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:33.570+0000 7fefd7e00700 1 -- 172.21.15.37:0/4022704313 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5464+0+0 (secure 0 0 0) 0x7fefcc086e80 con 0x7fefe00fe0d0 2024-03-29T08:03:33.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:33.570+0000 7fefdfe00700 1 --2- 172.21.15.37:0/4022704313 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7fefc40692a0 0x7fefc406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:03:33.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:33.570+0000 7fefdfe00700 1 --2- 172.21.15.37:0/4022704313 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7fefc40692a0 0x7fefc406b750 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fefd000b900 tx=0x7fefd0009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:03:33.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:33.574+0000 7fefd7e00700 1 -- 172.21.15.37:0/4022704313 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fefcc056ba0 con 0x7fefe00fe0d0 2024-03-29T08:03:33.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:33.838+0000 7fefe7200700 1 -- 172.21.15.37:0/4022704313 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fefe0111a10 con 0x7fefe00fe0d0 2024-03-29T08:03:33.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:33.838+0000 7fefd7e00700 1 -- 172.21.15.37:0/4022704313 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 2 v2) v1 ==== 75+0+1076 (secure 0 0 0) 0x7fefcc056730 con 0x7fefe00fe0d0 2024-03-29T08:03:33.839 INFO:teuthology.orchestra.run.smithi037.stdout:e2 2024-03-29T08:03:33.839 INFO:teuthology.orchestra.run.smithi037.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-29T08:03:33.839 INFO:teuthology.orchestra.run.smithi037.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T08:03:33.839 INFO:teuthology.orchestra.run.smithi037.stdout:legacy client fscid: 1 2024-03-29T08:03:33.839 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:03:33.839 INFO:teuthology.orchestra.run.smithi037.stdout:Filesystem 'cephfs' (1) 2024-03-29T08:03:33.839 INFO:teuthology.orchestra.run.smithi037.stdout:fs_name cephfs 2024-03-29T08:03:33.839 INFO:teuthology.orchestra.run.smithi037.stdout:epoch 2 2024-03-29T08:03:33.839 INFO:teuthology.orchestra.run.smithi037.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-03-29T08:03:33.839 INFO:teuthology.orchestra.run.smithi037.stdout:created 2024-03-29T08:03:29.241269+0000 2024-03-29T08:03:33.839 INFO:teuthology.orchestra.run.smithi037.stdout:modified 2024-03-29T08:03:29.241342+0000 2024-03-29T08:03:33.839 INFO:teuthology.orchestra.run.smithi037.stdout:tableserver 0 2024-03-29T08:03:33.839 INFO:teuthology.orchestra.run.smithi037.stdout:root 0 2024-03-29T08:03:33.840 INFO:teuthology.orchestra.run.smithi037.stdout:session_timeout 60 2024-03-29T08:03:33.840 INFO:teuthology.orchestra.run.smithi037.stdout:session_autoclose 300 2024-03-29T08:03:33.840 INFO:teuthology.orchestra.run.smithi037.stdout:max_file_size 1099511627776 2024-03-29T08:03:33.840 INFO:teuthology.orchestra.run.smithi037.stdout:required_client_features {} 2024-03-29T08:03:33.840 INFO:teuthology.orchestra.run.smithi037.stdout:last_failure 0 2024-03-29T08:03:33.840 INFO:teuthology.orchestra.run.smithi037.stdout:last_failure_osd_epoch 0 2024-03-29T08:03:33.840 INFO:teuthology.orchestra.run.smithi037.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T08:03:33.840 INFO:teuthology.orchestra.run.smithi037.stdout:max_mds 1 2024-03-29T08:03:33.840 INFO:teuthology.orchestra.run.smithi037.stdout:in 2024-03-29T08:03:33.840 INFO:teuthology.orchestra.run.smithi037.stdout:up {} 2024-03-29T08:03:33.840 INFO:teuthology.orchestra.run.smithi037.stdout:failed 2024-03-29T08:03:33.840 INFO:teuthology.orchestra.run.smithi037.stdout:damaged 2024-03-29T08:03:33.840 INFO:teuthology.orchestra.run.smithi037.stdout:stopped 2024-03-29T08:03:33.840 INFO:teuthology.orchestra.run.smithi037.stdout:data_pools [3] 2024-03-29T08:03:33.840 INFO:teuthology.orchestra.run.smithi037.stdout:metadata_pool 2 2024-03-29T08:03:33.840 INFO:teuthology.orchestra.run.smithi037.stdout:inline_data disabled 2024-03-29T08:03:33.841 INFO:teuthology.orchestra.run.smithi037.stdout:balancer 2024-03-29T08:03:33.841 INFO:teuthology.orchestra.run.smithi037.stdout:standby_count_wanted 0 2024-03-29T08:03:33.841 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:03:33.841 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:03:33.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:33.841+0000 7fefe7200700 1 -- 172.21.15.37:0/4022704313 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7fefc40692a0 msgr2=0x7fefc406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:03:33.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:33.841+0000 7fefe7200700 1 --2- 172.21.15.37:0/4022704313 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7fefc40692a0 0x7fefc406b750 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fefd000b900 tx=0x7fefd0009e30 comp rx=0 tx=0).stop 2024-03-29T08:03:33.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:33.841+0000 7fefe7200700 1 -- 172.21.15.37:0/4022704313 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fefe00fe0d0 msgr2=0x7fefe0110090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:03:33.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:33.841+0000 7fefe7200700 1 --2- 172.21.15.37:0/4022704313 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fefe00fe0d0 0x7fefe0110090 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fefcc002a50 tx=0x7fefcc0046b0 comp rx=0 tx=0).stop 2024-03-29T08:03:33.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:33.841+0000 7fefe7200700 1 -- 172.21.15.37:0/4022704313 shutdown_connections 2024-03-29T08:03:33.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:33.841+0000 7fefe7200700 1 --2- 172.21.15.37:0/4022704313 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7fefc40692a0 0x7fefc406b750 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:03:33.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:33.841+0000 7fefe7200700 1 --2- 172.21.15.37:0/4022704313 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fefe00fcee0 0x7fefe010fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:03:33.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:33.841+0000 7fefe7200700 1 --2- 172.21.15.37:0/4022704313 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fefe00fe0d0 0x7fefe0110090 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:03:33.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:33.841+0000 7fefe7200700 1 -- 172.21.15.37:0/4022704313 >> 172.21.15.37:0/4022704313 conn(0x7fefe00f8560 msgr2=0x7fefe00f9d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:03:33.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:33.841+0000 7fefe7200700 1 -- 172.21.15.37:0/4022704313 shutdown_connections 2024-03-29T08:03:33.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:33.841+0000 7fefe7200700 1 -- 172.21.15.37:0/4022704313 wait complete. 2024-03-29T08:03:33.842 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 2 2024-03-29T08:03:34.398 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-29T08:03:34.408 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi037.front.sepia.ceph.com 2024-03-29T08:03:34.408 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 99b492c2-eda1-11ee-b647-cb9ed24678a4 -- bash -c 'ceph fs set cephfs max_mds 2' 2024-03-29T08:03:34.851 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/mon.smithi037/config 2024-03-29T08:03:35.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:03:34 smithi148 ceph-mon[44992]: pgmap v203: 65 pgs: 4 creating+peering, 61 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:03:35.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:03:34 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:03:35.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:03:34 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:03:35.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:03:34 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:03:35.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:03:34 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi148.sqteih", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-29T08:03:35.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:03:34 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi148.sqteih", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-29T08:03:35.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:03:34 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T08:03:35.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:03:34 smithi148 ceph-mon[44992]: Deploying daemon mds.cephfs.smithi148.sqteih on smithi148 2024-03-29T08:03:35.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:03:34 smithi148 ceph-mon[44992]: from='client.? 172.21.15.37:0/4022704313' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T08:03:35.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:03:34 smithi148 ceph-mon[44992]: mds.? [v2:172.21.15.37:6826/1559618909,v1:172.21.15.37:6827/1559618909] up:boot 2024-03-29T08:03:35.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:03:34 smithi148 ceph-mon[44992]: daemon mds.cephfs.smithi037.mmwgxe assigned to filesystem cephfs as rank 0 (now has 1 ranks) 2024-03-29T08:03:35.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:03:34 smithi148 ceph-mon[44992]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-29T08:03:35.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:03:34 smithi148 ceph-mon[44992]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-03-29T08:03:35.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:03:34 smithi148 ceph-mon[44992]: Cluster is now healthy 2024-03-29T08:03:35.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:03:34 smithi148 ceph-mon[44992]: fsmap cephfs:0 1 up:standby 2024-03-29T08:03:35.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:03:34 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi037.mmwgxe"}]: dispatch 2024-03-29T08:03:35.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:03:34 smithi148 ceph-mon[44992]: fsmap cephfs:1 {0=cephfs.smithi037.mmwgxe=up:creating} 2024-03-29T08:03:35.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:03:34 smithi148 ceph-mon[44992]: daemon mds.cephfs.smithi037.mmwgxe is now active in filesystem cephfs as rank 0 2024-03-29T08:03:35.110 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:03:34 smithi037 ceph-mon[35162]: pgmap v203: 65 pgs: 4 creating+peering, 61 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:03:35.110 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:03:34 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:03:35.110 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:03:34 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:03:35.110 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:03:34 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:03:35.110 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:03:34 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi148.sqteih", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-29T08:03:35.111 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:03:34 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi148.sqteih", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-29T08:03:35.111 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:03:34 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T08:03:35.111 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:03:34 smithi037 ceph-mon[35162]: Deploying daemon mds.cephfs.smithi148.sqteih on smithi148 2024-03-29T08:03:35.111 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:03:34 smithi037 ceph-mon[35162]: from='client.? 172.21.15.37:0/4022704313' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T08:03:35.111 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:03:34 smithi037 ceph-mon[35162]: mds.? [v2:172.21.15.37:6826/1559618909,v1:172.21.15.37:6827/1559618909] up:boot 2024-03-29T08:03:35.111 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:03:34 smithi037 ceph-mon[35162]: daemon mds.cephfs.smithi037.mmwgxe assigned to filesystem cephfs as rank 0 (now has 1 ranks) 2024-03-29T08:03:35.111 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:03:34 smithi037 ceph-mon[35162]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-29T08:03:35.111 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:03:34 smithi037 ceph-mon[35162]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-03-29T08:03:35.111 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:03:34 smithi037 ceph-mon[35162]: Cluster is now healthy 2024-03-29T08:03:35.111 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:03:34 smithi037 ceph-mon[35162]: fsmap cephfs:0 1 up:standby 2024-03-29T08:03:35.111 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:03:34 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi037.mmwgxe"}]: dispatch 2024-03-29T08:03:35.111 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:03:34 smithi037 ceph-mon[35162]: fsmap cephfs:1 {0=cephfs.smithi037.mmwgxe=up:creating} 2024-03-29T08:03:35.111 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:03:34 smithi037 ceph-mon[35162]: daemon mds.cephfs.smithi037.mmwgxe is now active in filesystem cephfs as rank 0 2024-03-29T08:03:36.805 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:03:36 smithi148 ceph-mon[44992]: pgmap v204: 65 pgs: 65 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 0 B/s wr, 2 op/s 2024-03-29T08:03:36.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:03:36 smithi148 ceph-mon[44992]: mds.? [v2:172.21.15.37:6826/1559618909,v1:172.21.15.37:6827/1559618909] up:active 2024-03-29T08:03:36.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:03:36 smithi148 ceph-mon[44992]: fsmap cephfs:1 {0=cephfs.smithi037.mmwgxe=up:active} 2024-03-29T08:03:36.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:03:36 smithi037 ceph-mon[35162]: pgmap v204: 65 pgs: 65 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 0 B/s wr, 2 op/s 2024-03-29T08:03:36.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:03:36 smithi037 ceph-mon[35162]: mds.? [v2:172.21.15.37:6826/1559618909,v1:172.21.15.37:6827/1559618909] up:active 2024-03-29T08:03:36.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:03:36 smithi037 ceph-mon[35162]: fsmap cephfs:1 {0=cephfs.smithi037.mmwgxe=up:active} 2024-03-29T08:03:36.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:36.929+0000 7fb5fe200700 1 -- 172.21.15.37:0/39923864 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb5f80662e0 msgr2=0x7fb5f80ff9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:03:36.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:36.929+0000 7fb5fe200700 1 --2- 172.21.15.37:0/39923864 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb5f80662e0 0x7fb5f80ff9c0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fb5e8004940 tx=0x7fb5e8009e30 comp rx=0 tx=0).stop 2024-03-29T08:03:36.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:36.929+0000 7fb5fe200700 1 -- 172.21.15.37:0/39923864 shutdown_connections 2024-03-29T08:03:36.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:36.929+0000 7fb5fe200700 1 --2- 172.21.15.37:0/39923864 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb5f80fff00 0x7fb5f8102320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:03:36.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:36.929+0000 7fb5fe200700 1 --2- 172.21.15.37:0/39923864 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb5f80662e0 0x7fb5f80ff9c0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:03:36.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:36.929+0000 7fb5fe200700 1 -- 172.21.15.37:0/39923864 >> 172.21.15.37:0/39923864 conn(0x7fb5f80f7430 msgr2=0x7fb5f80f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:03:36.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:36.929+0000 7fb5fe200700 1 -- 172.21.15.37:0/39923864 shutdown_connections 2024-03-29T08:03:36.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:36.929+0000 7fb5fe200700 1 -- 172.21.15.37:0/39923864 wait complete. 2024-03-29T08:03:36.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:36.930+0000 7fb5fe200700 1 Processor -- start 2024-03-29T08:03:36.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:36.930+0000 7fb5fe200700 1 -- start start 2024-03-29T08:03:36.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:36.930+0000 7fb5fe200700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb5f80662e0 0x7fb5f80fd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:03:36.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:36.930+0000 7fb5fe200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb5f80fff00 0x7fb5f80fb8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:03:36.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:36.930+0000 7fb5fe200700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5f80fbe70 con 0x7fb5f80fff00 2024-03-29T08:03:36.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:36.930+0000 7fb5fe200700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5f80fbfe0 con 0x7fb5f80662e0 2024-03-29T08:03:36.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:36.931+0000 7fb5f6a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb5f80fff00 0x7fb5f80fb8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:03:36.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:36.931+0000 7fb5f6a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb5f80fff00 0x7fb5f80fb8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:33420/0 (socket says 172.21.15.37:33420) 2024-03-29T08:03:36.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:36.931+0000 7fb5f6a00700 1 -- 172.21.15.37:0/1165044633 learned_addr learned my addr 172.21.15.37:0/1165044633 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:03:36.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:36.931+0000 7fb5f7400700 1 --2- 172.21.15.37:0/1165044633 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb5f80662e0 0x7fb5f80fd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:03:36.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:36.931+0000 7fb5f6a00700 1 -- 172.21.15.37:0/1165044633 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb5f80662e0 msgr2=0x7fb5f80fd3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:03:36.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:36.931+0000 7fb5f6a00700 1 --2- 172.21.15.37:0/1165044633 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb5f80662e0 0x7fb5f80fd3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:03:36.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:36.931+0000 7fb5f6a00700 1 -- 172.21.15.37:0/1165044633 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5e8005040 con 0x7fb5f80fff00 2024-03-29T08:03:36.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:36.931+0000 7fb5f6a00700 1 --2- 172.21.15.37:0/1165044633 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb5f80fff00 0x7fb5f80fb8c0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fb5e400eb40 tx=0x7fb5e400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:03:36.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:36.931+0000 7fb5ef400700 1 -- 172.21.15.37:0/1165044633 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb5e4007cc0 con 0x7fb5f80fff00 2024-03-29T08:03:36.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:36.932+0000 7fb5fe200700 1 -- 172.21.15.37:0/1165044633 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5f80fc260 con 0x7fb5f80fff00 2024-03-29T08:03:36.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:36.932+0000 7fb5ef400700 1 -- 172.21.15.37:0/1165044633 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fb5e400f040 con 0x7fb5f80fff00 2024-03-29T08:03:36.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:36.932+0000 7fb5ef400700 1 -- 172.21.15.37:0/1165044633 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb5e400c990 con 0x7fb5f80fff00 2024-03-29T08:03:36.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:36.932+0000 7fb5fe200700 1 -- 172.21.15.37:0/1165044633 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5f8115900 con 0x7fb5f80fff00 2024-03-29T08:03:36.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:36.933+0000 7fb5fe200700 1 -- 172.21.15.37:0/1165044633 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5f805f5f0 con 0x7fb5f80fff00 2024-03-29T08:03:36.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:36.934+0000 7fb5ef400700 1 -- 172.21.15.37:0/1165044633 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fb5e400cc00 con 0x7fb5f80fff00 2024-03-29T08:03:36.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:36.934+0000 7fb5ef400700 1 --2- 172.21.15.37:0/1165044633 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7fb5dc0692b0 0x7fb5dc06b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:03:36.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:36.934+0000 7fb5ef400700 1 -- 172.21.15.37:0/1165044633 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5464+0+0 (secure 0 0 0) 0x7fb5e4086df0 con 0x7fb5f80fff00 2024-03-29T08:03:36.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:36.935+0000 7fb5f7400700 1 --2- 172.21.15.37:0/1165044633 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7fb5dc0692b0 0x7fb5dc06b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:03:36.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:36.935+0000 7fb5f7400700 1 --2- 172.21.15.37:0/1165044633 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7fb5dc0692b0 0x7fb5dc06b760 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fb5f80fd180 tx=0x7fb5e8009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:03:36.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:36.939+0000 7fb5ef400700 1 -- 172.21.15.37:0/1165044633 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fb5e404e190 con 0x7fb5f80fff00 2024-03-29T08:03:37.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:37.190+0000 7fb5fe200700 1 -- 172.21.15.37:0/1165044633 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"} v 0) v1 -- 0x7fb5f804bed0 con 0x7fb5f80fff00 2024-03-29T08:03:37.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:37.530+0000 7fb5ef400700 1 -- 172.21.15.37:0/1165044633 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]=0 v6) v1 ==== 105+0+0 (secure 0 0 0) 0x7fb5e40530b0 con 0x7fb5f80fff00 2024-03-29T08:03:37.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:37.533+0000 7fb5fe200700 1 -- 172.21.15.37:0/1165044633 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7fb5dc0692b0 msgr2=0x7fb5dc06b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:03:37.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:37.533+0000 7fb5fe200700 1 --2- 172.21.15.37:0/1165044633 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7fb5dc0692b0 0x7fb5dc06b760 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fb5f80fd180 tx=0x7fb5e8009e30 comp rx=0 tx=0).stop 2024-03-29T08:03:37.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:37.533+0000 7fb5fe200700 1 -- 172.21.15.37:0/1165044633 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb5f80fff00 msgr2=0x7fb5f80fb8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:03:37.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:37.533+0000 7fb5fe200700 1 --2- 172.21.15.37:0/1165044633 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb5f80fff00 0x7fb5f80fb8c0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fb5e400eb40 tx=0x7fb5e400b040 comp rx=0 tx=0).stop 2024-03-29T08:03:37.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:37.533+0000 7fb5fe200700 1 -- 172.21.15.37:0/1165044633 shutdown_connections 2024-03-29T08:03:37.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:37.533+0000 7fb5fe200700 1 --2- 172.21.15.37:0/1165044633 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7fb5dc0692b0 0x7fb5dc06b760 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:03:37.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:37.533+0000 7fb5fe200700 1 --2- 172.21.15.37:0/1165044633 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb5f80662e0 0x7fb5f80fd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:03:37.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:37.534+0000 7fb5fe200700 1 --2- 172.21.15.37:0/1165044633 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb5f80fff00 0x7fb5f80fb8c0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:03:37.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:37.534+0000 7fb5fe200700 1 -- 172.21.15.37:0/1165044633 >> 172.21.15.37:0/1165044633 conn(0x7fb5f80f7430 msgr2=0x7fb5f80f7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:03:37.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:37.534+0000 7fb5fe200700 1 -- 172.21.15.37:0/1165044633 shutdown_connections 2024-03-29T08:03:37.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:37.534+0000 7fb5fe200700 1 -- 172.21.15.37:0/1165044633 wait complete. 2024-03-29T08:03:38.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:03:38 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:03:38.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:03:38 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:03:38.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:03:38 smithi148 ceph-mon[44992]: from='client.? 172.21.15.37:0/1165044633' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]: dispatch 2024-03-29T08:03:38.307 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:03:38 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:03:38.307 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:03:38 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi037.iifsmh", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-29T08:03:38.307 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:03:38 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi037.iifsmh", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-29T08:03:38.307 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:03:38 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T08:03:38.307 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:03:38 smithi148 ceph-mon[44992]: Deploying daemon mds.cephfs.smithi037.iifsmh on smithi037 2024-03-29T08:03:38.307 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:03:38 smithi148 ceph-mon[44992]: pgmap v205: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.5 KiB/s wr, 4 op/s 2024-03-29T08:03:38.307 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:03:38 smithi148 ceph-mon[44992]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-03-29T08:03:38.307 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:03:38 smithi148 ceph-mon[44992]: mds.? [v2:172.21.15.148:6824/2050993679,v1:172.21.15.148:6825/2050993679] up:boot 2024-03-29T08:03:38.307 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:03:38 smithi148 ceph-mon[44992]: from='client.? 172.21.15.37:0/1165044633' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]': finished 2024-03-29T08:03:38.307 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:03:38 smithi148 ceph-mon[44992]: daemon mds.cephfs.smithi148.sqteih assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-03-29T08:03:38.307 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:03:38 smithi148 ceph-mon[44992]: Health check failed: insufficient standby MDS daemons available (MDS_INSUFFICIENT_STANDBY) 2024-03-29T08:03:38.307 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:03:38 smithi148 ceph-mon[44992]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-03-29T08:03:38.307 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:03:38 smithi148 ceph-mon[44992]: fsmap cephfs:1 {0=cephfs.smithi037.mmwgxe=up:active} 1 up:standby 2024-03-29T08:03:38.307 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:03:38 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi148.sqteih"}]: dispatch 2024-03-29T08:03:38.308 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:03:38 smithi148 ceph-mon[44992]: fsmap cephfs:2 {0=cephfs.smithi037.mmwgxe=up:active,1=cephfs.smithi148.sqteih=up:creating} 2024-03-29T08:03:38.308 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:03:38 smithi148 ceph-mon[44992]: daemon mds.cephfs.smithi148.sqteih is now active in filesystem cephfs as rank 1 2024-03-29T08:03:38.371 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:03:38 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:03:38.371 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:03:38 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:03:38.371 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:03:38 smithi037 ceph-mon[35162]: from='client.? 172.21.15.37:0/1165044633' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]: dispatch 2024-03-29T08:03:38.371 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:03:38 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:03:38.371 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:03:38 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi037.iifsmh", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-29T08:03:38.371 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:03:38 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi037.iifsmh", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-29T08:03:38.372 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:03:38 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T08:03:38.372 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:03:38 smithi037 ceph-mon[35162]: Deploying daemon mds.cephfs.smithi037.iifsmh on smithi037 2024-03-29T08:03:38.372 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:03:38 smithi037 ceph-mon[35162]: pgmap v205: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.5 KiB/s wr, 4 op/s 2024-03-29T08:03:38.372 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:03:38 smithi037 ceph-mon[35162]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-03-29T08:03:38.372 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:03:38 smithi037 ceph-mon[35162]: mds.? [v2:172.21.15.148:6824/2050993679,v1:172.21.15.148:6825/2050993679] up:boot 2024-03-29T08:03:38.372 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:03:38 smithi037 ceph-mon[35162]: from='client.? 172.21.15.37:0/1165044633' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]': finished 2024-03-29T08:03:38.372 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:03:38 smithi037 ceph-mon[35162]: daemon mds.cephfs.smithi148.sqteih assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-03-29T08:03:38.372 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:03:38 smithi037 ceph-mon[35162]: Health check failed: insufficient standby MDS daemons available (MDS_INSUFFICIENT_STANDBY) 2024-03-29T08:03:38.372 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:03:38 smithi037 ceph-mon[35162]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-03-29T08:03:38.372 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:03:38 smithi037 ceph-mon[35162]: fsmap cephfs:1 {0=cephfs.smithi037.mmwgxe=up:active} 1 up:standby 2024-03-29T08:03:38.372 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:03:38 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi148.sqteih"}]: dispatch 2024-03-29T08:03:38.372 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:03:38 smithi037 ceph-mon[35162]: fsmap cephfs:2 {0=cephfs.smithi037.mmwgxe=up:active,1=cephfs.smithi148.sqteih=up:creating} 2024-03-29T08:03:38.372 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:03:38 smithi037 ceph-mon[35162]: daemon mds.cephfs.smithi148.sqteih is now active in filesystem cephfs as rank 1 2024-03-29T08:03:38.395 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-29T08:03:38.405 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi037.front.sepia.ceph.com 2024-03-29T08:03:38.405 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 99b492c2-eda1-11ee-b647-cb9ed24678a4 -- bash -c 'ceph fs set cephfs allow_standby_replay false' 2024-03-29T08:03:39.603 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:03:39 smithi037 ceph-mon[35162]: mds.? [v2:172.21.15.148:6824/2050993679,v1:172.21.15.148:6825/2050993679] up:active 2024-03-29T08:03:39.604 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:03:39 smithi037 ceph-mon[35162]: fsmap cephfs:2 {0=cephfs.smithi037.mmwgxe=up:active,1=cephfs.smithi148.sqteih=up:active} 2024-03-29T08:03:39.689 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/mon.smithi037/config 2024-03-29T08:03:39.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:03:39 smithi148 ceph-mon[44992]: mds.? [v2:172.21.15.148:6824/2050993679,v1:172.21.15.148:6825/2050993679] up:active 2024-03-29T08:03:39.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:03:39 smithi148 ceph-mon[44992]: fsmap cephfs:2 {0=cephfs.smithi037.mmwgxe=up:active,1=cephfs.smithi148.sqteih=up:active} 2024-03-29T08:03:40.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:03:40 smithi148 ceph-mon[44992]: pgmap v206: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.4 KiB/s wr, 4 op/s 2024-03-29T08:03:40.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:03:40 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:03:40.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:03:40 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:03:40.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:03:40 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:03:40.840 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:03:40 smithi037 ceph-mon[35162]: pgmap v206: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.4 KiB/s wr, 4 op/s 2024-03-29T08:03:40.840 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:03:40 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:03:40.840 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:03:40 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:03:40.840 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:03:40 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:03:42.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:42.181+0000 7f6f56400700 1 -- 172.21.15.37:0/2296287003 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6f500fe0d0 msgr2=0x7f6f500fe540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:03:42.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:42.181+0000 7f6f56400700 1 --2- 172.21.15.37:0/2296287003 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6f500fe0d0 0x7f6f500fe540 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f6f3c004940 tx=0x7f6f3c009e30 comp rx=0 tx=0).stop 2024-03-29T08:03:42.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:42.181+0000 7f6f56400700 1 -- 172.21.15.37:0/2296287003 shutdown_connections 2024-03-29T08:03:42.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:42.181+0000 7f6f56400700 1 --2- 172.21.15.37:0/2296287003 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6f500fe0d0 0x7f6f500fe540 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:03:42.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:42.181+0000 7f6f56400700 1 --2- 172.21.15.37:0/2296287003 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6f500fcee0 0x7f6f500fd310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:03:42.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:42.181+0000 7f6f56400700 1 -- 172.21.15.37:0/2296287003 >> 172.21.15.37:0/2296287003 conn(0x7f6f500f8560 msgr2=0x7f6f500fa970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:03:42.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:42.181+0000 7f6f56400700 1 -- 172.21.15.37:0/2296287003 shutdown_connections 2024-03-29T08:03:42.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:42.181+0000 7f6f56400700 1 -- 172.21.15.37:0/2296287003 wait complete. 2024-03-29T08:03:42.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:42.182+0000 7f6f56400700 1 Processor -- start 2024-03-29T08:03:42.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:42.182+0000 7f6f56400700 1 -- start start 2024-03-29T08:03:42.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:42.183+0000 7f6f56400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6f500fcee0 0x7f6f5010d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:03:42.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:42.183+0000 7f6f56400700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6f500fe0d0 0x7f6f5010de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:03:42.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:42.183+0000 7f6f56400700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f5010e490 con 0x7f6f500fcee0 2024-03-29T08:03:42.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:42.183+0000 7f6f56400700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f5010ec70 con 0x7f6f500fe0d0 2024-03-29T08:03:42.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:42.183+0000 7f6f4f400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6f500fcee0 0x7f6f5010d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:03:42.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:42.183+0000 7f6f4f400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6f500fcee0 0x7f6f5010d950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56316/0 (socket says 172.21.15.37:56316) 2024-03-29T08:03:42.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:42.183+0000 7f6f4f400700 1 -- 172.21.15.37:0/1046396345 learned_addr learned my addr 172.21.15.37:0/1046396345 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:03:42.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:42.183+0000 7f6f4ea00700 1 --2- 172.21.15.37:0/1046396345 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6f500fe0d0 0x7f6f5010de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:03:42.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:42.184+0000 7f6f4f400700 1 -- 172.21.15.37:0/1046396345 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6f500fe0d0 msgr2=0x7f6f5010de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:03:42.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:42.184+0000 7f6f4f400700 1 --2- 172.21.15.37:0/1046396345 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6f500fe0d0 0x7f6f5010de90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:03:42.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:42.184+0000 7f6f4f400700 1 -- 172.21.15.37:0/1046396345 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f3c005040 con 0x7f6f500fcee0 2024-03-29T08:03:42.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:42.184+0000 7f6f4f400700 1 --2- 172.21.15.37:0/1046396345 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6f500fcee0 0x7f6f5010d950 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f6f4000eb70 tx=0x7f6f4000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:03:42.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:42.184+0000 7f6f47400700 1 -- 172.21.15.37:0/1046396345 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6f40007cc0 con 0x7f6f500fcee0 2024-03-29T08:03:42.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:42.184+0000 7f6f47400700 1 -- 172.21.15.37:0/1046396345 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f6f4000f040 con 0x7f6f500fcee0 2024-03-29T08:03:42.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:42.184+0000 7f6f56400700 1 -- 172.21.15.37:0/1046396345 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f5010eef0 con 0x7f6f500fcee0 2024-03-29T08:03:42.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:42.184+0000 7f6f47400700 1 -- 172.21.15.37:0/1046396345 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6f4000c9d0 con 0x7f6f500fcee0 2024-03-29T08:03:42.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:42.184+0000 7f6f56400700 1 -- 172.21.15.37:0/1046396345 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f5010f430 con 0x7f6f500fcee0 2024-03-29T08:03:42.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:42.186+0000 7f6f56400700 1 -- 172.21.15.37:0/1046396345 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f5005f5f0 con 0x7f6f500fcee0 2024-03-29T08:03:42.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:42.187+0000 7f6f47400700 1 -- 172.21.15.37:0/1046396345 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f6f4000cb30 con 0x7f6f500fcee0 2024-03-29T08:03:42.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:42.188+0000 7f6f47400700 1 --2- 172.21.15.37:0/1046396345 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f6f340692b0 0x7f6f3406b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:03:42.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:42.188+0000 7f6f47400700 1 -- 172.21.15.37:0/1046396345 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f6f40086c30 con 0x7f6f500fcee0 2024-03-29T08:03:42.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:42.188+0000 7f6f4ea00700 1 --2- 172.21.15.37:0/1046396345 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f6f340692b0 0x7f6f3406b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:03:42.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:42.189+0000 7f6f4ea00700 1 --2- 172.21.15.37:0/1046396345 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f6f340692b0 0x7f6f3406b760 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f6f3c00b0d0 tx=0x7f6f3c009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:03:42.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:42.193+0000 7f6f47400700 1 -- 172.21.15.37:0/1046396345 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f6f4004e190 con 0x7f6f500fcee0 2024-03-29T08:03:42.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:42.446+0000 7f6f56400700 1 -- 172.21.15.37:0/1046396345 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f6f5004bed0 con 0x7f6f500fcee0 2024-03-29T08:03:42.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:42.546+0000 7f6f47400700 1 -- 172.21.15.37:0/1046396345 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v9) v1 ==== 122+0+0 (secure 0 0 0) 0x7f6f40056a00 con 0x7f6f500fcee0 2024-03-29T08:03:42.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:42.548+0000 7f6f56400700 1 -- 172.21.15.37:0/1046396345 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f6f340692b0 msgr2=0x7f6f3406b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:03:42.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:42.548+0000 7f6f56400700 1 --2- 172.21.15.37:0/1046396345 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f6f340692b0 0x7f6f3406b760 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f6f3c00b0d0 tx=0x7f6f3c009e30 comp rx=0 tx=0).stop 2024-03-29T08:03:42.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:42.548+0000 7f6f56400700 1 -- 172.21.15.37:0/1046396345 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6f500fcee0 msgr2=0x7f6f5010d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:03:42.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:42.548+0000 7f6f56400700 1 --2- 172.21.15.37:0/1046396345 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6f500fcee0 0x7f6f5010d950 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f6f4000eb70 tx=0x7f6f4000b040 comp rx=0 tx=0).stop 2024-03-29T08:03:42.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:42.548+0000 7f6f56400700 1 -- 172.21.15.37:0/1046396345 shutdown_connections 2024-03-29T08:03:42.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:42.548+0000 7f6f56400700 1 --2- 172.21.15.37:0/1046396345 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f6f340692b0 0x7f6f3406b760 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:03:42.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:42.548+0000 7f6f56400700 1 --2- 172.21.15.37:0/1046396345 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6f500fcee0 0x7f6f5010d950 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:03:42.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:42.548+0000 7f6f56400700 1 --2- 172.21.15.37:0/1046396345 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6f500fe0d0 0x7f6f5010de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:03:42.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:42.548+0000 7f6f56400700 1 -- 172.21.15.37:0/1046396345 >> 172.21.15.37:0/1046396345 conn(0x7f6f500f8560 msgr2=0x7f6f500f9d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:03:42.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:42.548+0000 7f6f56400700 1 -- 172.21.15.37:0/1046396345 shutdown_connections 2024-03-29T08:03:42.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:42.548+0000 7f6f56400700 1 -- 172.21.15.37:0/1046396345 wait complete. 2024-03-29T08:03:42.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:03:42 smithi148 ceph-mon[44992]: pgmap v207: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s wr, 7 op/s 2024-03-29T08:03:42.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:03:42 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:03:42.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:03:42 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:03:42.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:03:42 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:03:42.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:03:42 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi148.qukpjh", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-29T08:03:42.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:03:42 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi148.qukpjh", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-29T08:03:42.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:03:42 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T08:03:42.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:03:42 smithi148 ceph-mon[44992]: from='client.? 172.21.15.37:0/1046396345' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]: dispatch 2024-03-29T08:03:42.874 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:03:42 smithi037 ceph-mon[35162]: pgmap v207: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s wr, 7 op/s 2024-03-29T08:03:42.875 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:03:42 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:03:42.875 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:03:42 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:03:42.875 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:03:42 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:03:42.875 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:03:42 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi148.qukpjh", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-29T08:03:42.875 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:03:42 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi148.qukpjh", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-29T08:03:42.875 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:03:42 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T08:03:42.875 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:03:42 smithi037 ceph-mon[35162]: from='client.? 172.21.15.37:0/1046396345' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]: dispatch 2024-03-29T08:03:43.243 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-29T08:03:43.253 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi037.front.sepia.ceph.com 2024-03-29T08:03:43.253 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 99b492c2-eda1-11ee-b647-cb9ed24678a4 -- bash -c 'ceph fs set cephfs inline_data false' 2024-03-29T08:03:43.692 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/mon.smithi037/config 2024-03-29T08:03:43.805 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:03:43 smithi148 ceph-mon[44992]: Deploying daemon mds.cephfs.smithi148.qukpjh on smithi148 2024-03-29T08:03:43.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:03:43 smithi148 ceph-mon[44992]: Health check cleared: MDS_INSUFFICIENT_STANDBY (was: insufficient standby MDS daemons available) 2024-03-29T08:03:43.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:03:43 smithi148 ceph-mon[44992]: Cluster is now healthy 2024-03-29T08:03:43.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:03:43 smithi148 ceph-mon[44992]: mds.? [v2:172.21.15.37:6828/2341530495,v1:172.21.15.37:6829/2341530495] up:boot 2024-03-29T08:03:43.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:03:43 smithi148 ceph-mon[44992]: from='client.? 172.21.15.37:0/1046396345' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]': finished 2024-03-29T08:03:43.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:03:43 smithi148 ceph-mon[44992]: fsmap cephfs:2 {0=cephfs.smithi037.mmwgxe=up:active,1=cephfs.smithi148.sqteih=up:active} 1 up:standby 2024-03-29T08:03:43.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:03:43 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi037.iifsmh"}]: dispatch 2024-03-29T08:03:43.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:03:43 smithi037 ceph-mon[35162]: Deploying daemon mds.cephfs.smithi148.qukpjh on smithi148 2024-03-29T08:03:43.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:03:43 smithi037 ceph-mon[35162]: Health check cleared: MDS_INSUFFICIENT_STANDBY (was: insufficient standby MDS daemons available) 2024-03-29T08:03:43.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:03:43 smithi037 ceph-mon[35162]: Cluster is now healthy 2024-03-29T08:03:43.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:03:43 smithi037 ceph-mon[35162]: mds.? [v2:172.21.15.37:6828/2341530495,v1:172.21.15.37:6829/2341530495] up:boot 2024-03-29T08:03:43.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:03:43 smithi037 ceph-mon[35162]: from='client.? 172.21.15.37:0/1046396345' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]': finished 2024-03-29T08:03:43.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:03:43 smithi037 ceph-mon[35162]: fsmap cephfs:2 {0=cephfs.smithi037.mmwgxe=up:active,1=cephfs.smithi148.sqteih=up:active} 1 up:standby 2024-03-29T08:03:43.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:03:43 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi037.iifsmh"}]: dispatch 2024-03-29T08:03:44.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:03:44 smithi148 ceph-mon[44992]: pgmap v208: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s wr, 7 op/s 2024-03-29T08:03:44.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:03:44 smithi037 ceph-mon[35162]: pgmap v208: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s wr, 7 op/s 2024-03-29T08:03:45.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:45.663+0000 7ff64a600700 1 -- 172.21.15.37:0/1054074210 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff6440709a0 msgr2=0x7ff644070e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:03:45.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:45.663+0000 7ff64a600700 1 --2- 172.21.15.37:0/1054074210 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff6440709a0 0x7ff644070e10 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7ff630004940 tx=0x7ff630009e30 comp rx=0 tx=0).stop 2024-03-29T08:03:45.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:45.664+0000 7ff64a600700 1 -- 172.21.15.37:0/1054074210 shutdown_connections 2024-03-29T08:03:45.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:45.664+0000 7ff64a600700 1 --2- 172.21.15.37:0/1054074210 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff6440709a0 0x7ff644070e10 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:03:45.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:45.664+0000 7ff64a600700 1 --2- 172.21.15.37:0/1054074210 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff644071f50 0x7ff644070460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:03:45.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:45.664+0000 7ff64a600700 1 -- 172.21.15.37:0/1054074210 >> 172.21.15.37:0/1054074210 conn(0x7ff6440f8560 msgr2=0x7ff6440fa970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:03:45.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:45.664+0000 7ff64a600700 1 -- 172.21.15.37:0/1054074210 shutdown_connections 2024-03-29T08:03:45.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:45.664+0000 7ff64a600700 1 -- 172.21.15.37:0/1054074210 wait complete. 2024-03-29T08:03:45.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:45.664+0000 7ff64a600700 1 Processor -- start 2024-03-29T08:03:45.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:45.665+0000 7ff64a600700 1 -- start start 2024-03-29T08:03:45.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:45.665+0000 7ff64a600700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff6440709a0 0x7ff644113f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:03:45.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:45.665+0000 7ff64a600700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff644071f50 0x7ff644114460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:03:45.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:45.665+0000 7ff64a600700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff644114a60 con 0x7ff644071f50 2024-03-29T08:03:45.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:45.665+0000 7ff64a600700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6441152b0 con 0x7ff6440709a0 2024-03-29T08:03:45.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:45.665+0000 7ff642a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff644071f50 0x7ff644114460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:03:45.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:45.665+0000 7ff643400700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff6440709a0 0x7ff644113f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:03:45.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:45.665+0000 7ff642a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff644071f50 0x7ff644114460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56336/0 (socket says 172.21.15.37:56336) 2024-03-29T08:03:45.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:45.665+0000 7ff643400700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff6440709a0 0x7ff644113f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.37:57070/0 (socket says 172.21.15.37:57070) 2024-03-29T08:03:45.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:45.665+0000 7ff642a00700 1 -- 172.21.15.37:0/3239418736 learned_addr learned my addr 172.21.15.37:0/3239418736 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:03:45.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:45.666+0000 7ff643400700 1 -- 172.21.15.37:0/3239418736 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff644071f50 msgr2=0x7ff644114460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:03:45.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:45.666+0000 7ff643400700 1 --2- 172.21.15.37:0/3239418736 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff644071f50 0x7ff644114460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:03:45.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:45.666+0000 7ff643400700 1 -- 172.21.15.37:0/3239418736 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff630005040 con 0x7ff6440709a0 2024-03-29T08:03:45.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:45.666+0000 7ff642a00700 1 --2- 172.21.15.37:0/3239418736 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff644071f50 0x7ff644114460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T08:03:45.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:45.666+0000 7ff643400700 1 --2- 172.21.15.37:0/3239418736 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff6440709a0 0x7ff644113f20 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7ff634002840 tx=0x7ff634004570 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:03:45.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:45.666+0000 7ff63b400700 1 -- 172.21.15.37:0/3239418736 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff634007cc0 con 0x7ff6440709a0 2024-03-29T08:03:45.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:45.666+0000 7ff63b400700 1 -- 172.21.15.37:0/3239418736 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ff63400f040 con 0x7ff6440709a0 2024-03-29T08:03:45.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:45.666+0000 7ff64a600700 1 -- 172.21.15.37:0/3239418736 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff644115590 con 0x7ff6440709a0 2024-03-29T08:03:45.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:45.666+0000 7ff63b400700 1 -- 172.21.15.37:0/3239418736 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff634013860 con 0x7ff6440709a0 2024-03-29T08:03:45.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:45.666+0000 7ff64a600700 1 -- 172.21.15.37:0/3239418736 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff644115ad0 con 0x7ff6440709a0 2024-03-29T08:03:45.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:45.668+0000 7ff64a600700 1 -- 172.21.15.37:0/3239418736 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff64405f5f0 con 0x7ff6440709a0 2024-03-29T08:03:45.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:45.668+0000 7ff63b400700 1 -- 172.21.15.37:0/3239418736 <== mon.1 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7ff634009070 con 0x7ff6440709a0 2024-03-29T08:03:45.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:45.669+0000 7ff63b400700 1 --2- 172.21.15.37:0/3239418736 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7ff6280692a0 0x7ff62806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:03:45.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:45.669+0000 7ff63b400700 1 -- 172.21.15.37:0/3239418736 <== mon.1 v2:172.21.15.148:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5464+0+0 (secure 0 0 0) 0x7ff634086ec0 con 0x7ff6440709a0 2024-03-29T08:03:45.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:45.669+0000 7ff642a00700 1 --2- 172.21.15.37:0/3239418736 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7ff6280692a0 0x7ff62806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:03:45.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:45.670+0000 7ff642a00700 1 --2- 172.21.15.37:0/3239418736 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7ff6280692a0 0x7ff62806b750 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7ff63000b3b0 tx=0x7ff630009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:03:45.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:45.674+0000 7ff63b400700 1 -- 172.21.15.37:0/3239418736 <== mon.1 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7ff634056b60 con 0x7ff6440709a0 2024-03-29T08:03:45.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:45.937+0000 7ff64a600700 1 -- 172.21.15.37:0/3239418736 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"} v 0) v1 -- 0x7ff64404bed0 con 0x7ff6440709a0 2024-03-29T08:03:46.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:46.637+0000 7ff63b400700 1 -- 172.21.15.37:0/3239418736 <== mon.1 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"}]=0 inline data disabled v11) v1 ==== 133+0+0 (secure 0 0 0) 0x7ff6340566f0 con 0x7ff6440709a0 2024-03-29T08:03:46.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:46.639+0000 7ff64a600700 1 -- 172.21.15.37:0/3239418736 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7ff6280692a0 msgr2=0x7ff62806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:03:46.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:46.639+0000 7ff64a600700 1 --2- 172.21.15.37:0/3239418736 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7ff6280692a0 0x7ff62806b750 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7ff63000b3b0 tx=0x7ff630009e30 comp rx=0 tx=0).stop 2024-03-29T08:03:46.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:46.639+0000 7ff64a600700 1 -- 172.21.15.37:0/3239418736 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff6440709a0 msgr2=0x7ff644113f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:03:46.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:46.639+0000 7ff64a600700 1 --2- 172.21.15.37:0/3239418736 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff6440709a0 0x7ff644113f20 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7ff634002840 tx=0x7ff634004570 comp rx=0 tx=0).stop 2024-03-29T08:03:46.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:46.639+0000 7ff64a600700 1 -- 172.21.15.37:0/3239418736 shutdown_connections 2024-03-29T08:03:46.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:46.639+0000 7ff64a600700 1 --2- 172.21.15.37:0/3239418736 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7ff6280692a0 0x7ff62806b750 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:03:46.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:46.639+0000 7ff64a600700 1 --2- 172.21.15.37:0/3239418736 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff6440709a0 0x7ff644113f20 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:03:46.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:46.639+0000 7ff64a600700 1 --2- 172.21.15.37:0/3239418736 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff644071f50 0x7ff644114460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:03:46.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:46.639+0000 7ff64a600700 1 -- 172.21.15.37:0/3239418736 >> 172.21.15.37:0/3239418736 conn(0x7ff6440f8560 msgr2=0x7ff6440f9ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:03:46.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:46.640+0000 7ff64a600700 1 -- 172.21.15.37:0/3239418736 shutdown_connections 2024-03-29T08:03:46.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:46.640+0000 7ff64a600700 1 -- 172.21.15.37:0/3239418736 wait complete. 2024-03-29T08:03:46.640 INFO:teuthology.orchestra.run.smithi037.stderr:inline data disabled 2024-03-29T08:03:46.930 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:03:46 smithi037 ceph-mon[35162]: pgmap v209: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s wr, 7 op/s 2024-03-29T08:03:46.930 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:03:46 smithi037 ceph-mon[35162]: mds.? [v2:172.21.15.148:6826/2641327912,v1:172.21.15.148:6827/2641327912] up:boot 2024-03-29T08:03:46.930 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:03:46 smithi037 ceph-mon[35162]: fsmap cephfs:2 {0=cephfs.smithi037.mmwgxe=up:active,1=cephfs.smithi148.sqteih=up:active} 2 up:standby 2024-03-29T08:03:46.930 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:03:46 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi148.qukpjh"}]: dispatch 2024-03-29T08:03:46.930 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:03:46 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:03:46.930 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:03:46 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:03:46.930 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:03:46 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:03:46.930 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:03:46 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:03:46.930 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:03:46 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:03:46.930 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:03:46 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T08:03:46.930 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:03:46 smithi037 ceph-mon[35162]: from='client.? 172.21.15.37:0/3239418736' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"}]: dispatch 2024-03-29T08:03:46.930 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:03:46 smithi037 ceph-mon[35162]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"}]: dispatch 2024-03-29T08:03:47.001 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:03:46 smithi148 ceph-mon[44992]: pgmap v209: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s wr, 7 op/s 2024-03-29T08:03:47.001 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:03:46 smithi148 ceph-mon[44992]: mds.? [v2:172.21.15.148:6826/2641327912,v1:172.21.15.148:6827/2641327912] up:boot 2024-03-29T08:03:47.001 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:03:46 smithi148 ceph-mon[44992]: fsmap cephfs:2 {0=cephfs.smithi037.mmwgxe=up:active,1=cephfs.smithi148.sqteih=up:active} 2 up:standby 2024-03-29T08:03:47.001 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:03:46 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi148.qukpjh"}]: dispatch 2024-03-29T08:03:47.001 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:03:46 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:03:47.001 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:03:46 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:03:47.001 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:03:46 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:03:47.002 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:03:46 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:03:47.002 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:03:46 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:03:47.002 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:03:46 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T08:03:47.002 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:03:46 smithi148 ceph-mon[44992]: from='client.? 172.21.15.37:0/3239418736' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"}]: dispatch 2024-03-29T08:03:47.002 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:03:46 smithi148 ceph-mon[44992]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"}]: dispatch 2024-03-29T08:03:47.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:03:47 smithi037 ceph-mon[35162]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"}]': finished 2024-03-29T08:03:47.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:03:47 smithi037 ceph-mon[35162]: mds.? [v2:172.21.15.37:6828/2341530495,v1:172.21.15.37:6829/2341530495] up:standby 2024-03-29T08:03:47.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:03:47 smithi037 ceph-mon[35162]: mds.? [v2:172.21.15.37:6826/1559618909,v1:172.21.15.37:6827/1559618909] up:active 2024-03-29T08:03:47.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:03:47 smithi037 ceph-mon[35162]: Dropping low affinity active daemon mds.cephfs.smithi148.sqteih in favor of higher affinity standby. 2024-03-29T08:03:47.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:03:47 smithi037 ceph-mon[35162]: Replacing daemon mds.cephfs.smithi148.sqteih as rank 1 with standby daemon mds.cephfs.smithi037.iifsmh 2024-03-29T08:03:47.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:03:47 smithi037 ceph-mon[35162]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-29T08:03:47.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:03:47 smithi037 ceph-mon[35162]: fsmap cephfs:2 {0=cephfs.smithi037.mmwgxe=up:active,1=cephfs.smithi148.sqteih=up:active} 2 up:standby 2024-03-29T08:03:47.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:03:47 smithi037 ceph-mon[35162]: osdmap e47: 6 total, 6 up, 6 in 2024-03-29T08:03:47.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:03:47 smithi037 ceph-mon[35162]: fsmap cephfs:2/2 {0=cephfs.smithi037.mmwgxe=up:active,1=cephfs.smithi037.iifsmh=up:replay} 1 up:standby 2024-03-29T08:03:47.987 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-29T08:03:47.997 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi037.front.sepia.ceph.com 2024-03-29T08:03:47.997 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 99b492c2-eda1-11ee-b647-cb9ed24678a4 -- bash -c 'ceph fs dump' 2024-03-29T08:03:47.997 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:03:47 smithi148 ceph-mon[44992]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"}]': finished 2024-03-29T08:03:47.998 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:03:47 smithi148 ceph-mon[44992]: mds.? [v2:172.21.15.37:6828/2341530495,v1:172.21.15.37:6829/2341530495] up:standby 2024-03-29T08:03:47.998 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:03:47 smithi148 ceph-mon[44992]: mds.? [v2:172.21.15.37:6826/1559618909,v1:172.21.15.37:6827/1559618909] up:active 2024-03-29T08:03:47.998 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:03:47 smithi148 ceph-mon[44992]: Dropping low affinity active daemon mds.cephfs.smithi148.sqteih in favor of higher affinity standby. 2024-03-29T08:03:47.998 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:03:47 smithi148 ceph-mon[44992]: Replacing daemon mds.cephfs.smithi148.sqteih as rank 1 with standby daemon mds.cephfs.smithi037.iifsmh 2024-03-29T08:03:47.998 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:03:47 smithi148 ceph-mon[44992]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-29T08:03:47.998 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:03:47 smithi148 ceph-mon[44992]: fsmap cephfs:2 {0=cephfs.smithi037.mmwgxe=up:active,1=cephfs.smithi148.sqteih=up:active} 2 up:standby 2024-03-29T08:03:47.998 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:03:47 smithi148 ceph-mon[44992]: osdmap e47: 6 total, 6 up, 6 in 2024-03-29T08:03:47.998 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:03:47 smithi148 ceph-mon[44992]: fsmap cephfs:2/2 {0=cephfs.smithi037.mmwgxe=up:active,1=cephfs.smithi037.iifsmh=up:replay} 1 up:standby 2024-03-29T08:03:48.881 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/mon.smithi037/config 2024-03-29T08:03:48.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:03:48 smithi037 ceph-mon[35162]: pgmap v211: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 614 B/s rd, 1.2 KiB/s wr, 5 op/s 2024-03-29T08:03:48.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:03:48 smithi037 ceph-mon[35162]: mds.? [v2:172.21.15.37:6828/2341530495,v1:172.21.15.37:6829/2341530495] up:resolve 2024-03-29T08:03:48.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:03:48 smithi037 ceph-mon[35162]: mds.? [v2:172.21.15.148:6824/3252351726,v1:172.21.15.148:6825/3252351726] up:boot 2024-03-29T08:03:48.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:03:48 smithi037 ceph-mon[35162]: fsmap cephfs:2/2 {0=cephfs.smithi037.mmwgxe=up:active,1=cephfs.smithi037.iifsmh=up:resolve} 2 up:standby 2024-03-29T08:03:48.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:03:48 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi148.sqteih"}]: dispatch 2024-03-29T08:03:49.055 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:03:48 smithi148 ceph-mon[44992]: pgmap v211: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 614 B/s rd, 1.2 KiB/s wr, 5 op/s 2024-03-29T08:03:49.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:03:48 smithi148 ceph-mon[44992]: mds.? [v2:172.21.15.37:6828/2341530495,v1:172.21.15.37:6829/2341530495] up:resolve 2024-03-29T08:03:49.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:03:48 smithi148 ceph-mon[44992]: mds.? [v2:172.21.15.148:6824/3252351726,v1:172.21.15.148:6825/3252351726] up:boot 2024-03-29T08:03:49.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:03:48 smithi148 ceph-mon[44992]: fsmap cephfs:2/2 {0=cephfs.smithi037.mmwgxe=up:active,1=cephfs.smithi037.iifsmh=up:resolve} 2 up:standby 2024-03-29T08:03:49.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:03:48 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi148.sqteih"}]: dispatch 2024-03-29T08:03:49.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:03:49 smithi037 ceph-mon[35162]: mds.? [v2:172.21.15.37:6828/2341530495,v1:172.21.15.37:6829/2341530495] up:reconnect 2024-03-29T08:03:49.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:03:49 smithi037 ceph-mon[35162]: fsmap cephfs:2/2 {0=cephfs.smithi037.mmwgxe=up:active,1=cephfs.smithi037.iifsmh=up:reconnect} 2 up:standby 2024-03-29T08:03:49.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:03:49 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:03:49.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:03:49 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:03:50.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:03:49 smithi148 ceph-mon[44992]: mds.? [v2:172.21.15.37:6828/2341530495,v1:172.21.15.37:6829/2341530495] up:reconnect 2024-03-29T08:03:50.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:03:49 smithi148 ceph-mon[44992]: fsmap cephfs:2/2 {0=cephfs.smithi037.mmwgxe=up:active,1=cephfs.smithi037.iifsmh=up:reconnect} 2 up:standby 2024-03-29T08:03:50.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:03:49 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:03:50.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:03:49 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:03:50.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:03:50 smithi037 ceph-mon[35162]: pgmap v212: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 614 B/s rd, 1.1 KiB/s wr, 4 op/s 2024-03-29T08:03:50.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:03:50 smithi037 ceph-mon[35162]: mds.? [v2:172.21.15.37:6828/2341530495,v1:172.21.15.37:6829/2341530495] up:rejoin 2024-03-29T08:03:50.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:03:50 smithi037 ceph-mon[35162]: mds.? [v2:172.21.15.148:6826/2641327912,v1:172.21.15.148:6827/2641327912] up:standby 2024-03-29T08:03:50.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:03:50 smithi037 ceph-mon[35162]: fsmap cephfs:2/2 {0=cephfs.smithi037.mmwgxe=up:active,1=cephfs.smithi037.iifsmh=up:rejoin} 2 up:standby 2024-03-29T08:03:50.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:03:50 smithi037 ceph-mon[35162]: daemon mds.cephfs.smithi037.iifsmh is now active in filesystem cephfs as rank 1 2024-03-29T08:03:50.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:03:50 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:03:51.055 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:03:50 smithi148 ceph-mon[44992]: pgmap v212: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 614 B/s rd, 1.1 KiB/s wr, 4 op/s 2024-03-29T08:03:51.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:03:50 smithi148 ceph-mon[44992]: mds.? [v2:172.21.15.37:6828/2341530495,v1:172.21.15.37:6829/2341530495] up:rejoin 2024-03-29T08:03:51.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:03:50 smithi148 ceph-mon[44992]: mds.? [v2:172.21.15.148:6826/2641327912,v1:172.21.15.148:6827/2641327912] up:standby 2024-03-29T08:03:51.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:03:50 smithi148 ceph-mon[44992]: fsmap cephfs:2/2 {0=cephfs.smithi037.mmwgxe=up:active,1=cephfs.smithi037.iifsmh=up:rejoin} 2 up:standby 2024-03-29T08:03:51.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:03:50 smithi148 ceph-mon[44992]: daemon mds.cephfs.smithi037.iifsmh is now active in filesystem cephfs as rank 1 2024-03-29T08:03:51.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:03:50 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:03:51.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:51.462+0000 7fdf4d600700 1 -- 172.21.15.37:0/421424906 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdf480fe0d0 msgr2=0x7fdf480fe540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:03:51.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:51.462+0000 7fdf4d600700 1 --2- 172.21.15.37:0/421424906 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdf480fe0d0 0x7fdf480fe540 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fdf30004940 tx=0x7fdf30009e30 comp rx=0 tx=0).stop 2024-03-29T08:03:51.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:51.462+0000 7fdf4d600700 1 -- 172.21.15.37:0/421424906 shutdown_connections 2024-03-29T08:03:51.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:51.462+0000 7fdf4d600700 1 --2- 172.21.15.37:0/421424906 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdf480fe0d0 0x7fdf480fe540 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:03:51.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:51.462+0000 7fdf4d600700 1 --2- 172.21.15.37:0/421424906 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdf480fcee0 0x7fdf480fd310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:03:51.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:51.462+0000 7fdf4d600700 1 -- 172.21.15.37:0/421424906 >> 172.21.15.37:0/421424906 conn(0x7fdf480f8560 msgr2=0x7fdf480fa970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:03:51.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:51.462+0000 7fdf4d600700 1 -- 172.21.15.37:0/421424906 shutdown_connections 2024-03-29T08:03:51.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:51.462+0000 7fdf4d600700 1 -- 172.21.15.37:0/421424906 wait complete. 2024-03-29T08:03:51.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:51.463+0000 7fdf4d600700 1 Processor -- start 2024-03-29T08:03:51.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:51.463+0000 7fdf4d600700 1 -- start start 2024-03-29T08:03:51.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:51.463+0000 7fdf4d600700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdf480fcee0 0x7fdf4810fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:03:51.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:51.463+0000 7fdf4d600700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdf480fe0d0 0x7fdf481100c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:03:51.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:51.463+0000 7fdf4d600700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf481106c0 con 0x7fdf480fe0d0 2024-03-29T08:03:51.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:51.463+0000 7fdf4d600700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf48110f10 con 0x7fdf480fcee0 2024-03-29T08:03:51.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:51.464+0000 7fdf3fe00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdf480fe0d0 0x7fdf481100c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:03:51.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:51.464+0000 7fdf3fe00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdf480fe0d0 0x7fdf481100c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53384/0 (socket says 172.21.15.37:53384) 2024-03-29T08:03:51.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:51.464+0000 7fdf3fe00700 1 -- 172.21.15.37:0/693826992 learned_addr learned my addr 172.21.15.37:0/693826992 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:03:51.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:51.464+0000 7fdf46a00700 1 --2- 172.21.15.37:0/693826992 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdf480fcee0 0x7fdf4810fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:03:51.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:51.464+0000 7fdf3fe00700 1 -- 172.21.15.37:0/693826992 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdf480fcee0 msgr2=0x7fdf4810fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:03:51.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:51.464+0000 7fdf3fe00700 1 --2- 172.21.15.37:0/693826992 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdf480fcee0 0x7fdf4810fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:03:51.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:51.464+0000 7fdf3fe00700 1 -- 172.21.15.37:0/693826992 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf30005040 con 0x7fdf480fe0d0 2024-03-29T08:03:51.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:51.464+0000 7fdf3fe00700 1 --2- 172.21.15.37:0/693826992 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdf480fe0d0 0x7fdf481100c0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fdf30009380 tx=0x7fdf30009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:03:51.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:51.464+0000 7fdf3ea00700 1 -- 172.21.15.37:0/693826992 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdf3000e040 con 0x7fdf480fe0d0 2024-03-29T08:03:51.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:51.465+0000 7fdf3ea00700 1 -- 172.21.15.37:0/693826992 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fdf30004290 con 0x7fdf480fe0d0 2024-03-29T08:03:51.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:51.465+0000 7fdf4d600700 1 -- 172.21.15.37:0/693826992 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf48111190 con 0x7fdf480fe0d0 2024-03-29T08:03:51.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:51.465+0000 7fdf3ea00700 1 -- 172.21.15.37:0/693826992 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdf3001c3f0 con 0x7fdf480fe0d0 2024-03-29T08:03:51.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:51.465+0000 7fdf4d600700 1 -- 172.21.15.37:0/693826992 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf481116d0 con 0x7fdf480fe0d0 2024-03-29T08:03:51.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:51.466+0000 7fdf4d600700 1 -- 172.21.15.37:0/693826992 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf4805f5f0 con 0x7fdf480fe0d0 2024-03-29T08:03:51.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:51.467+0000 7fdf3ea00700 1 -- 172.21.15.37:0/693826992 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fdf3000bb80 con 0x7fdf480fe0d0 2024-03-29T08:03:51.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:51.467+0000 7fdf3ea00700 1 --2- 172.21.15.37:0/693826992 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7fdf2c0692b0 0x7fdf2c06b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:03:51.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:51.468+0000 7fdf3ea00700 1 -- 172.21.15.37:0/693826992 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5550+0+0 (secure 0 0 0) 0x7fdf300884f0 con 0x7fdf480fe0d0 2024-03-29T08:03:51.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:51.468+0000 7fdf46a00700 1 --2- 172.21.15.37:0/693826992 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7fdf2c0692b0 0x7fdf2c06b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:03:51.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:51.468+0000 7fdf46a00700 1 --2- 172.21.15.37:0/693826992 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7fdf2c0692b0 0x7fdf2c06b760 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fdf480fcc20 tx=0x7fdf38009040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:03:51.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:51.472+0000 7fdf3ea00700 1 -- 172.21.15.37:0/693826992 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fdf30058310 con 0x7fdf480fe0d0 2024-03-29T08:03:51.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:51.728+0000 7fdf4d600700 1 -- 172.21.15.37:0/693826992 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fdf4804bed0 con 0x7fdf480fe0d0 2024-03-29T08:03:51.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:51.729+0000 7fdf3ea00700 1 -- 172.21.15.37:0/693826992 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 16 v16) v1 ==== 76+0+1820 (secure 0 0 0) 0x7fdf30057ea0 con 0x7fdf480fe0d0 2024-03-29T08:03:51.729 INFO:teuthology.orchestra.run.smithi037.stdout:e16 2024-03-29T08:03:51.729 INFO:teuthology.orchestra.run.smithi037.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-29T08:03:51.729 INFO:teuthology.orchestra.run.smithi037.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T08:03:51.729 INFO:teuthology.orchestra.run.smithi037.stdout:legacy client fscid: 1 2024-03-29T08:03:51.729 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:03:51.729 INFO:teuthology.orchestra.run.smithi037.stdout:Filesystem 'cephfs' (1) 2024-03-29T08:03:51.729 INFO:teuthology.orchestra.run.smithi037.stdout:fs_name cephfs 2024-03-29T08:03:51.729 INFO:teuthology.orchestra.run.smithi037.stdout:epoch 16 2024-03-29T08:03:51.729 INFO:teuthology.orchestra.run.smithi037.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-03-29T08:03:51.729 INFO:teuthology.orchestra.run.smithi037.stdout:created 2024-03-29T08:03:29.241269+0000 2024-03-29T08:03:51.729 INFO:teuthology.orchestra.run.smithi037.stdout:modified 2024-03-29T08:03:50.664746+0000 2024-03-29T08:03:51.729 INFO:teuthology.orchestra.run.smithi037.stdout:tableserver 0 2024-03-29T08:03:51.729 INFO:teuthology.orchestra.run.smithi037.stdout:root 0 2024-03-29T08:03:51.730 INFO:teuthology.orchestra.run.smithi037.stdout:session_timeout 60 2024-03-29T08:03:51.730 INFO:teuthology.orchestra.run.smithi037.stdout:session_autoclose 300 2024-03-29T08:03:51.730 INFO:teuthology.orchestra.run.smithi037.stdout:max_file_size 1099511627776 2024-03-29T08:03:51.730 INFO:teuthology.orchestra.run.smithi037.stdout:required_client_features {} 2024-03-29T08:03:51.730 INFO:teuthology.orchestra.run.smithi037.stdout:last_failure 0 2024-03-29T08:03:51.730 INFO:teuthology.orchestra.run.smithi037.stdout:last_failure_osd_epoch 47 2024-03-29T08:03:51.730 INFO:teuthology.orchestra.run.smithi037.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T08:03:51.730 INFO:teuthology.orchestra.run.smithi037.stdout:max_mds 2 2024-03-29T08:03:51.730 INFO:teuthology.orchestra.run.smithi037.stdout:in 0,1 2024-03-29T08:03:51.730 INFO:teuthology.orchestra.run.smithi037.stdout:up {0=14470,1=14482} 2024-03-29T08:03:51.730 INFO:teuthology.orchestra.run.smithi037.stdout:failed 2024-03-29T08:03:51.730 INFO:teuthology.orchestra.run.smithi037.stdout:damaged 2024-03-29T08:03:51.731 INFO:teuthology.orchestra.run.smithi037.stdout:stopped 2024-03-29T08:03:51.731 INFO:teuthology.orchestra.run.smithi037.stdout:data_pools [3] 2024-03-29T08:03:51.731 INFO:teuthology.orchestra.run.smithi037.stdout:metadata_pool 2 2024-03-29T08:03:51.731 INFO:teuthology.orchestra.run.smithi037.stdout:inline_data disabled 2024-03-29T08:03:51.731 INFO:teuthology.orchestra.run.smithi037.stdout:balancer 2024-03-29T08:03:51.731 INFO:teuthology.orchestra.run.smithi037.stdout:standby_count_wanted 1 2024-03-29T08:03:51.731 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi037.mmwgxe{0:14470} state up:active seq 5 join_fscid=1 addr [v2:172.21.15.37:6826/1559618909,v1:172.21.15.37:6827/1559618909] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:03:51.731 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi037.iifsmh{1:14482} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.37:6828/2341530495,v1:172.21.15.37:6829/2341530495] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:03:51.731 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:03:51.731 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:03:51.731 INFO:teuthology.orchestra.run.smithi037.stdout:Standby daemons: 2024-03-29T08:03:51.731 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:03:51.731 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi148.qukpjh{-1:24319} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.148:6826/2641327912,v1:172.21.15.148:6827/2641327912] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:03:51.731 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi148.sqteih{-1:24327} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.148:6824/3252351726,v1:172.21.15.148:6825/3252351726] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:03:51.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:51.731+0000 7fdf4d600700 1 -- 172.21.15.37:0/693826992 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7fdf2c0692b0 msgr2=0x7fdf2c06b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:03:51.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:51.731+0000 7fdf4d600700 1 --2- 172.21.15.37:0/693826992 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7fdf2c0692b0 0x7fdf2c06b760 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fdf480fcc20 tx=0x7fdf38009040 comp rx=0 tx=0).stop 2024-03-29T08:03:51.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:51.731+0000 7fdf4d600700 1 -- 172.21.15.37:0/693826992 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdf480fe0d0 msgr2=0x7fdf481100c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:03:51.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:51.731+0000 7fdf4d600700 1 --2- 172.21.15.37:0/693826992 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdf480fe0d0 0x7fdf481100c0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fdf30009380 tx=0x7fdf30009e30 comp rx=0 tx=0).stop 2024-03-29T08:03:51.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:51.731+0000 7fdf4d600700 1 -- 172.21.15.37:0/693826992 shutdown_connections 2024-03-29T08:03:51.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:51.731+0000 7fdf4d600700 1 --2- 172.21.15.37:0/693826992 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7fdf2c0692b0 0x7fdf2c06b760 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:03:51.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:51.731+0000 7fdf4d600700 1 --2- 172.21.15.37:0/693826992 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdf480fcee0 0x7fdf4810fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:03:51.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:51.731+0000 7fdf4d600700 1 --2- 172.21.15.37:0/693826992 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdf480fe0d0 0x7fdf481100c0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:03:51.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:51.731+0000 7fdf4d600700 1 -- 172.21.15.37:0/693826992 >> 172.21.15.37:0/693826992 conn(0x7fdf480f8560 msgr2=0x7fdf480f9ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:03:51.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:51.731+0000 7fdf4d600700 1 -- 172.21.15.37:0/693826992 shutdown_connections 2024-03-29T08:03:51.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:51.731+0000 7fdf4d600700 1 -- 172.21.15.37:0/693826992 wait complete. 2024-03-29T08:03:51.733 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 16 2024-03-29T08:03:51.754 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:03:51 smithi037 ceph-mon[35162]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-29T08:03:51.754 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:03:51 smithi037 ceph-mon[35162]: Cluster is now healthy 2024-03-29T08:03:51.754 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:03:51 smithi037 ceph-mon[35162]: mds.? [v2:172.21.15.37:6828/2341530495,v1:172.21.15.37:6829/2341530495] up:active 2024-03-29T08:03:51.754 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:03:51 smithi037 ceph-mon[35162]: fsmap cephfs:2 {0=cephfs.smithi037.mmwgxe=up:active,1=cephfs.smithi037.iifsmh=up:active} 2 up:standby 2024-03-29T08:03:52.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:03:51 smithi148 ceph-mon[44992]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-29T08:03:52.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:03:51 smithi148 ceph-mon[44992]: Cluster is now healthy 2024-03-29T08:03:52.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:03:51 smithi148 ceph-mon[44992]: mds.? [v2:172.21.15.37:6828/2341530495,v1:172.21.15.37:6829/2341530495] up:active 2024-03-29T08:03:52.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:03:51 smithi148 ceph-mon[44992]: fsmap cephfs:2 {0=cephfs.smithi037.mmwgxe=up:active,1=cephfs.smithi037.iifsmh=up:active} 2 up:standby 2024-03-29T08:03:52.544 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 99b492c2-eda1-11ee-b647-cb9ed24678a4 -- bash -c 'ceph --format=json fs dump | jq -e ".filesystems | length == 1"' 2024-03-29T08:03:52.852 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:03:52 smithi037 ceph-mon[35162]: pgmap v213: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.3 KiB/s rd, 409 B/s wr, 2 op/s 2024-03-29T08:03:52.853 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:03:52 smithi037 ceph-mon[35162]: from='client.? 172.21.15.37:0/693826992' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T08:03:53.055 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:03:52 smithi148 ceph-mon[44992]: pgmap v213: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.3 KiB/s rd, 409 B/s wr, 2 op/s 2024-03-29T08:03:53.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:03:52 smithi148 ceph-mon[44992]: from='client.? 172.21.15.37:0/693826992' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T08:03:53.281 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/mon.smithi037/config 2024-03-29T08:03:54.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:03:54 smithi037 ceph-mon[35162]: pgmap v214: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-29T08:03:54.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:03:54 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:03:54.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:03:54 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:03:54.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:03:54 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T08:03:54.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:03:54 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T08:03:54.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:03:54 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:03:54.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:03:54 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T08:03:55.055 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:03:54 smithi148 ceph-mon[44992]: pgmap v214: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-29T08:03:55.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:03:54 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:03:55.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:03:54 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:03:55.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:03:54 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T08:03:55.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:03:54 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T08:03:55.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:03:54 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:03:55.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:03:54 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T08:03:55.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:55.343+0000 7fe663000700 1 -- 172.21.15.37:0/452344059 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe65c0fcee0 msgr2=0x7fe65c0fd310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:03:55.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:55.343+0000 7fe663000700 1 --2- 172.21.15.37:0/452344059 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe65c0fcee0 0x7fe65c0fd310 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fe64c004940 tx=0x7fe64c009e30 comp rx=0 tx=0).stop 2024-03-29T08:03:55.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:55.343+0000 7fe663000700 1 -- 172.21.15.37:0/452344059 shutdown_connections 2024-03-29T08:03:55.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:55.343+0000 7fe663000700 1 --2- 172.21.15.37:0/452344059 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe65c0fe0d0 0x7fe65c0fe540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:03:55.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:55.343+0000 7fe663000700 1 --2- 172.21.15.37:0/452344059 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe65c0fcee0 0x7fe65c0fd310 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:03:55.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:55.343+0000 7fe663000700 1 -- 172.21.15.37:0/452344059 >> 172.21.15.37:0/452344059 conn(0x7fe65c0f8560 msgr2=0x7fe65c0fa970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:03:55.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:55.343+0000 7fe663000700 1 -- 172.21.15.37:0/452344059 shutdown_connections 2024-03-29T08:03:55.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:55.343+0000 7fe663000700 1 -- 172.21.15.37:0/452344059 wait complete. 2024-03-29T08:03:55.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:55.344+0000 7fe663000700 1 Processor -- start 2024-03-29T08:03:55.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:55.344+0000 7fe663000700 1 -- start start 2024-03-29T08:03:55.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:55.344+0000 7fe663000700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe65c0fcee0 0x7fe65c071e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:03:55.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:55.344+0000 7fe663000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe65c0fe0d0 0x7fe65c070310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:03:55.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:55.344+0000 7fe663000700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe65c0708c0 con 0x7fe65c0fe0d0 2024-03-29T08:03:55.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:55.344+0000 7fe663000700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe65c070a30 con 0x7fe65c0fcee0 2024-03-29T08:03:55.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:55.345+0000 7fe65b400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe65c0fe0d0 0x7fe65c070310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:03:55.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:55.345+0000 7fe65b400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe65c0fe0d0 0x7fe65c070310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53406/0 (socket says 172.21.15.37:53406) 2024-03-29T08:03:55.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:55.345+0000 7fe65b400700 1 -- 172.21.15.37:0/3528865862 learned_addr learned my addr 172.21.15.37:0/3528865862 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:03:55.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:55.345+0000 7fe65be00700 1 --2- 172.21.15.37:0/3528865862 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe65c0fcee0 0x7fe65c071e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:03:55.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:55.345+0000 7fe65b400700 1 -- 172.21.15.37:0/3528865862 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe65c0fcee0 msgr2=0x7fe65c071e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:03:55.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:55.345+0000 7fe65b400700 1 --2- 172.21.15.37:0/3528865862 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe65c0fcee0 0x7fe65c071e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:03:55.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:55.345+0000 7fe65b400700 1 -- 172.21.15.37:0/3528865862 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe64c005040 con 0x7fe65c0fe0d0 2024-03-29T08:03:55.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:55.345+0000 7fe65b400700 1 --2- 172.21.15.37:0/3528865862 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe65c0fe0d0 0x7fe65c070310 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fe64800eb40 tx=0x7fe64800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:03:55.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:55.346+0000 7fe653e00700 1 -- 172.21.15.37:0/3528865862 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe648007cc0 con 0x7fe65c0fe0d0 2024-03-29T08:03:55.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:55.346+0000 7fe653e00700 1 -- 172.21.15.37:0/3528865862 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fe64800f040 con 0x7fe65c0fe0d0 2024-03-29T08:03:55.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:55.346+0000 7fe653e00700 1 -- 172.21.15.37:0/3528865862 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe64800c990 con 0x7fe65c0fe0d0 2024-03-29T08:03:55.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:55.346+0000 7fe663000700 1 -- 172.21.15.37:0/3528865862 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe65c070cb0 con 0x7fe65c0fe0d0 2024-03-29T08:03:55.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:55.346+0000 7fe663000700 1 -- 172.21.15.37:0/3528865862 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe65c1158f0 con 0x7fe65c0fe0d0 2024-03-29T08:03:55.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:55.347+0000 7fe663000700 1 -- 172.21.15.37:0/3528865862 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe65c05f5f0 con 0x7fe65c0fe0d0 2024-03-29T08:03:55.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:55.350+0000 7fe653e00700 1 -- 172.21.15.37:0/3528865862 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fe64800caf0 con 0x7fe65c0fe0d0 2024-03-29T08:03:55.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:55.350+0000 7fe653e00700 1 --2- 172.21.15.37:0/3528865862 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7fe6400692b0 0x7fe64006b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:03:55.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:55.351+0000 7fe653e00700 1 -- 172.21.15.37:0/3528865862 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5550+0+0 (secure 0 0 0) 0x7fe648086d90 con 0x7fe65c0fe0d0 2024-03-29T08:03:55.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:55.351+0000 7fe65be00700 1 --2- 172.21.15.37:0/3528865862 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7fe6400692b0 0x7fe64006b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:03:55.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:55.352+0000 7fe65be00700 1 --2- 172.21.15.37:0/3528865862 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7fe6400692b0 0x7fe64006b760 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fe64c00b0d0 tx=0x7fe64c009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:03:55.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:55.356+0000 7fe653e00700 1 -- 172.21.15.37:0/3528865862 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fe64804e190 con 0x7fe65c0fe0d0 2024-03-29T08:03:55.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:55.621+0000 7fe663000700 1 -- 172.21.15.37:0/3528865862 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe65c105850 con 0x7fe65c0fe0d0 2024-03-29T08:03:55.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:55.621+0000 7fe653e00700 1 -- 172.21.15.37:0/3528865862 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4709 (secure 0 0 0) 0x7fe648053080 con 0x7fe65c0fe0d0 2024-03-29T08:03:55.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:55.623+0000 7fe663000700 1 -- 172.21.15.37:0/3528865862 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7fe6400692b0 msgr2=0x7fe64006b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:03:55.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:55.623+0000 7fe663000700 1 --2- 172.21.15.37:0/3528865862 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7fe6400692b0 0x7fe64006b760 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fe64c00b0d0 tx=0x7fe64c009e30 comp rx=0 tx=0).stop 2024-03-29T08:03:55.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:55.623+0000 7fe663000700 1 -- 172.21.15.37:0/3528865862 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe65c0fe0d0 msgr2=0x7fe65c070310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:03:55.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:55.623+0000 7fe663000700 1 --2- 172.21.15.37:0/3528865862 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe65c0fe0d0 0x7fe65c070310 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fe64800eb40 tx=0x7fe64800b040 comp rx=0 tx=0).stop 2024-03-29T08:03:55.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:55.624+0000 7fe663000700 1 -- 172.21.15.37:0/3528865862 shutdown_connections 2024-03-29T08:03:55.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:55.624+0000 7fe663000700 1 --2- 172.21.15.37:0/3528865862 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7fe6400692b0 0x7fe64006b760 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:03:55.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:55.624+0000 7fe663000700 1 --2- 172.21.15.37:0/3528865862 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe65c0fcee0 0x7fe65c071e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:03:55.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:55.624+0000 7fe663000700 1 --2- 172.21.15.37:0/3528865862 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe65c0fe0d0 0x7fe65c070310 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:03:55.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:55.624+0000 7fe663000700 1 -- 172.21.15.37:0/3528865862 >> 172.21.15.37:0/3528865862 conn(0x7fe65c0f8560 msgr2=0x7fe65c0f9cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:03:55.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:55.624+0000 7fe663000700 1 -- 172.21.15.37:0/3528865862 shutdown_connections 2024-03-29T08:03:55.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:03:55.624+0000 7fe663000700 1 -- 172.21.15.37:0/3528865862 wait complete. 2024-03-29T08:03:55.624 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 16 2024-03-29T08:03:55.637 INFO:teuthology.orchestra.run.smithi037.stdout:true 2024-03-29T08:03:56.805 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:03:56 smithi148 ceph-mon[44992]: pgmap v215: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.9 KiB/s rd, 0 B/s wr, 2 op/s 2024-03-29T08:03:56.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:03:56 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:03:56.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:03:56 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:03:56.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:03:56 smithi148 ceph-mon[44992]: from='client.? 172.21.15.37:0/3528865862' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-29T08:03:56.848 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:03:56 smithi037 ceph-mon[35162]: pgmap v215: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.9 KiB/s rd, 0 B/s wr, 2 op/s 2024-03-29T08:03:56.848 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:03:56 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:03:56.848 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:03:56 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:03:56.848 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:03:56 smithi037 ceph-mon[35162]: from='client.? 172.21.15.37:0/3528865862' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-29T08:03:56.873 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 99b492c2-eda1-11ee-b647-cb9ed24678a4 -- bash -c 'while ! ceph --format=json mds versions | jq -e ". | add == 4"; do sleep 1; done' 2024-03-29T08:03:57.760 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/mon.smithi037/config 2024-03-29T08:03:58.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:03:58 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:03:58.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:03:58 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:03:58.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:03:58 smithi037 ceph-mon[35162]: pgmap v216: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 4.2 KiB/s rd, 0 B/s wr, 4 op/s 2024-03-29T08:03:58.805 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:03:58 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:03:58.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:03:58 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:03:58.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:03:58 smithi148 ceph-mon[44992]: pgmap v216: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 4.2 KiB/s rd, 0 B/s wr, 4 op/s 2024-03-29T08:04:00.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:00.043+0000 7fdc93600700 1 -- 172.21.15.37:0/926202895 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdc8c0fff00 msgr2=0x7fdc8c102320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:04:00.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:00.043+0000 7fdc93600700 1 --2- 172.21.15.37:0/926202895 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdc8c0fff00 0x7fdc8c102320 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fdc78004940 tx=0x7fdc78009e30 comp rx=0 tx=0).stop 2024-03-29T08:04:00.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:00.043+0000 7fdc93600700 1 -- 172.21.15.37:0/926202895 shutdown_connections 2024-03-29T08:04:00.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:00.043+0000 7fdc93600700 1 --2- 172.21.15.37:0/926202895 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdc8c0fff00 0x7fdc8c102320 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:04:00.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:00.043+0000 7fdc93600700 1 --2- 172.21.15.37:0/926202895 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdc8c0662e0 0x7fdc8c0ff9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:04:00.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:00.043+0000 7fdc93600700 1 -- 172.21.15.37:0/926202895 >> 172.21.15.37:0/926202895 conn(0x7fdc8c0f7430 msgr2=0x7fdc8c0f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:04:00.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:00.043+0000 7fdc93600700 1 -- 172.21.15.37:0/926202895 shutdown_connections 2024-03-29T08:04:00.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:00.044+0000 7fdc93600700 1 -- 172.21.15.37:0/926202895 wait complete. 2024-03-29T08:04:00.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:00.044+0000 7fdc93600700 1 Processor -- start 2024-03-29T08:04:00.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:00.044+0000 7fdc93600700 1 -- start start 2024-03-29T08:04:00.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:00.045+0000 7fdc93600700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdc8c0662e0 0x7fdc8c0fd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:04:00.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:00.045+0000 7fdc93600700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdc8c0fff00 0x7fdc8c0fb8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:04:00.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:00.045+0000 7fdc93600700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc8c0fbe70 con 0x7fdc8c0fff00 2024-03-29T08:04:00.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:00.045+0000 7fdc93600700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc8c0fbfe0 con 0x7fdc8c0662e0 2024-03-29T08:04:00.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:00.045+0000 7fdc8be00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdc8c0fff00 0x7fdc8c0fb8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:04:00.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:00.045+0000 7fdc8be00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdc8c0fff00 0x7fdc8c0fb8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60700/0 (socket says 172.21.15.37:60700) 2024-03-29T08:04:00.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:00.045+0000 7fdc8be00700 1 -- 172.21.15.37:0/2445645019 learned_addr learned my addr 172.21.15.37:0/2445645019 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:04:00.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:00.045+0000 7fdc90c00700 1 --2- 172.21.15.37:0/2445645019 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdc8c0662e0 0x7fdc8c0fd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:04:00.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:00.045+0000 7fdc8be00700 1 -- 172.21.15.37:0/2445645019 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdc8c0662e0 msgr2=0x7fdc8c0fd3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:04:00.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:00.045+0000 7fdc8be00700 1 --2- 172.21.15.37:0/2445645019 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdc8c0662e0 0x7fdc8c0fd3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:04:00.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:00.045+0000 7fdc8be00700 1 -- 172.21.15.37:0/2445645019 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc78005040 con 0x7fdc8c0fff00 2024-03-29T08:04:00.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:00.046+0000 7fdc8be00700 1 --2- 172.21.15.37:0/2445645019 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdc8c0fff00 0x7fdc8c0fb8c0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fdc78009380 tx=0x7fdc78009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:04:00.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:00.046+0000 7fdc88c00700 1 -- 172.21.15.37:0/2445645019 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdc7800e040 con 0x7fdc8c0fff00 2024-03-29T08:04:00.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:00.046+0000 7fdc93600700 1 -- 172.21.15.37:0/2445645019 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc8c0fc260 con 0x7fdc8c0fff00 2024-03-29T08:04:00.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:00.046+0000 7fdc88c00700 1 -- 172.21.15.37:0/2445645019 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fdc7800bd20 con 0x7fdc8c0fff00 2024-03-29T08:04:00.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:00.046+0000 7fdc88c00700 1 -- 172.21.15.37:0/2445645019 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdc7801c630 con 0x7fdc8c0fff00 2024-03-29T08:04:00.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:00.046+0000 7fdc93600700 1 -- 172.21.15.37:0/2445645019 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc8c1159b0 con 0x7fdc8c0fff00 2024-03-29T08:04:00.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:00.047+0000 7fdc93600700 1 -- 172.21.15.37:0/2445645019 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc8c05f5f0 con 0x7fdc8c0fff00 2024-03-29T08:04:00.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:00.048+0000 7fdc88c00700 1 -- 172.21.15.37:0/2445645019 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fdc78013420 con 0x7fdc8c0fff00 2024-03-29T08:04:00.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:00.049+0000 7fdc88c00700 1 --2- 172.21.15.37:0/2445645019 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7fdc740692a0 0x7fdc7406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:04:00.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:00.049+0000 7fdc88c00700 1 -- 172.21.15.37:0/2445645019 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5550+0+0 (secure 0 0 0) 0x7fdc78087fb0 con 0x7fdc8c0fff00 2024-03-29T08:04:00.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:00.050+0000 7fdc90c00700 1 --2- 172.21.15.37:0/2445645019 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7fdc740692a0 0x7fdc7406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:04:00.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:00.050+0000 7fdc90c00700 1 --2- 172.21.15.37:0/2445645019 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7fdc740692a0 0x7fdc7406b750 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fdc8c0fd180 tx=0x7fdc8000b040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:04:00.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:00.054+0000 7fdc88c00700 1 -- 172.21.15.37:0/2445645019 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fdc780ba190 con 0x7fdc8c0fff00 2024-03-29T08:04:00.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:00.345+0000 7fdc93600700 1 -- 172.21.15.37:0/2445645019 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "mds versions", "format": "json"} v 0) v1 -- 0x7fdc8c04bed0 con 0x7fdc8c0fff00 2024-03-29T08:04:00.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:00.345+0000 7fdc88c00700 1 -- 172.21.15.37:0/2445645019 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "mds versions", "format": "json"}]=0 v16) v1 ==== 78+0+99 (secure 0 0 0) 0x7fdc780541f0 con 0x7fdc8c0fff00 2024-03-29T08:04:00.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:00.347+0000 7fdc93600700 1 -- 172.21.15.37:0/2445645019 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7fdc740692a0 msgr2=0x7fdc7406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:04:00.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:00.347+0000 7fdc93600700 1 --2- 172.21.15.37:0/2445645019 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7fdc740692a0 0x7fdc7406b750 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fdc8c0fd180 tx=0x7fdc8000b040 comp rx=0 tx=0).stop 2024-03-29T08:04:00.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:00.347+0000 7fdc93600700 1 -- 172.21.15.37:0/2445645019 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdc8c0fff00 msgr2=0x7fdc8c0fb8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:04:00.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:00.347+0000 7fdc93600700 1 --2- 172.21.15.37:0/2445645019 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdc8c0fff00 0x7fdc8c0fb8c0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fdc78009380 tx=0x7fdc78009e30 comp rx=0 tx=0).stop 2024-03-29T08:04:00.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:00.348+0000 7fdc93600700 1 -- 172.21.15.37:0/2445645019 shutdown_connections 2024-03-29T08:04:00.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:00.348+0000 7fdc93600700 1 --2- 172.21.15.37:0/2445645019 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7fdc740692a0 0x7fdc7406b750 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:04:00.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:00.348+0000 7fdc93600700 1 --2- 172.21.15.37:0/2445645019 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdc8c0662e0 0x7fdc8c0fd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:04:00.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:00.348+0000 7fdc93600700 1 --2- 172.21.15.37:0/2445645019 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdc8c0fff00 0x7fdc8c0fb8c0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:04:00.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:00.348+0000 7fdc93600700 1 -- 172.21.15.37:0/2445645019 >> 172.21.15.37:0/2445645019 conn(0x7fdc8c0f7430 msgr2=0x7fdc8c0f7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:04:00.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:00.348+0000 7fdc93600700 1 -- 172.21.15.37:0/2445645019 shutdown_connections 2024-03-29T08:04:00.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:00.348+0000 7fdc93600700 1 -- 172.21.15.37:0/2445645019 wait complete. 2024-03-29T08:04:00.363 INFO:teuthology.orchestra.run.smithi037.stdout:true 2024-03-29T08:04:00.676 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:04:00 smithi037 ceph-mon[35162]: pgmap v217: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 3.3 KiB/s rd, 0 B/s wr, 3 op/s 2024-03-29T08:04:00.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:04:00 smithi037 ceph-mon[35162]: from='client.? 172.21.15.37:0/2445645019' entity='client.admin' cmd=[{"prefix": "mds versions", "format": "json"}]: dispatch 2024-03-29T08:04:00.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:04:00 smithi148 ceph-mon[44992]: pgmap v217: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 3.3 KiB/s rd, 0 B/s wr, 3 op/s 2024-03-29T08:04:00.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:04:00 smithi148 ceph-mon[44992]: from='client.? 172.21.15.37:0/2445645019' entity='client.admin' cmd=[{"prefix": "mds versions", "format": "json"}]: dispatch 2024-03-29T08:04:01.321 INFO:teuthology.run_tasks:Running task fs.pre_upgrade_save... 2024-03-29T08:04:01.333 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T08:04:01.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:01.562+0000 7feab5600700 1 -- 172.21.15.37:0/1301836862 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feab0070890 msgr2=0x7feab0070d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:04:01.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:01.562+0000 7feab5600700 1 --2- 172.21.15.37:0/1301836862 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feab0070890 0x7feab0070d00 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fea9c004940 tx=0x7fea9c009e30 comp rx=0 tx=0).stop 2024-03-29T08:04:01.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:01.563+0000 7feab5600700 1 -- 172.21.15.37:0/1301836862 shutdown_connections 2024-03-29T08:04:01.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:01.563+0000 7feab5600700 1 --2- 172.21.15.37:0/1301836862 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feab0070890 0x7feab0070d00 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:04:01.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:01.563+0000 7feab5600700 1 --2- 172.21.15.37:0/1301836862 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feab0071e40 0x7feab0070350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:04:01.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:01.563+0000 7feab5600700 1 -- 172.21.15.37:0/1301836862 >> 172.21.15.37:0/1301836862 conn(0x7feab00f8410 msgr2=0x7feab00fa840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:04:01.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:01.563+0000 7feab5600700 1 -- 172.21.15.37:0/1301836862 shutdown_connections 2024-03-29T08:04:01.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:01.563+0000 7feab5600700 1 -- 172.21.15.37:0/1301836862 wait complete. 2024-03-29T08:04:01.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:01.564+0000 7feab5600700 1 Processor -- start 2024-03-29T08:04:01.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:01.564+0000 7feab5600700 1 -- start start 2024-03-29T08:04:01.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:01.564+0000 7feab5600700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feab0070890 0x7feab010f950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:04:01.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:01.564+0000 7feab5600700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feab0071e40 0x7feab010fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:04:01.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:01.564+0000 7feab5600700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feab0110490 con 0x7feab0070890 2024-03-29T08:04:01.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:01.564+0000 7feab5600700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feab0110600 con 0x7feab0071e40 2024-03-29T08:04:01.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:01.564+0000 7feaaea00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feab0070890 0x7feab010f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:04:01.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:01.564+0000 7feaaea00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feab0070890 0x7feab010f950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60720/0 (socket says 172.21.15.37:60720) 2024-03-29T08:04:01.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:01.564+0000 7feaaea00700 1 -- 172.21.15.37:0/2722244246 learned_addr learned my addr 172.21.15.37:0/2722244246 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:04:01.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:01.565+0000 7feaae000700 1 --2- 172.21.15.37:0/2722244246 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feab0071e40 0x7feab010fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:04:01.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:01.565+0000 7feaaea00700 1 -- 172.21.15.37:0/2722244246 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feab0071e40 msgr2=0x7feab010fe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:04:01.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:01.565+0000 7feaaea00700 1 --2- 172.21.15.37:0/2722244246 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feab0071e40 0x7feab010fe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:04:01.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:01.565+0000 7feaaea00700 1 -- 172.21.15.37:0/2722244246 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea9c005040 con 0x7feab0070890 2024-03-29T08:04:01.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:01.565+0000 7feaaea00700 1 --2- 172.21.15.37:0/2722244246 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feab0070890 0x7feab010f950 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7feaa000eb70 tx=0x7feaa000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:04:01.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:01.565+0000 7feaa6a00700 1 -- 172.21.15.37:0/2722244246 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feaa0007cc0 con 0x7feab0070890 2024-03-29T08:04:01.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:01.565+0000 7feaa6a00700 1 -- 172.21.15.37:0/2722244246 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7feaa000f040 con 0x7feab0070890 2024-03-29T08:04:01.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:01.565+0000 7feab5600700 1 -- 172.21.15.37:0/2722244246 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feab0115000 con 0x7feab0070890 2024-03-29T08:04:01.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:01.565+0000 7feaa6a00700 1 -- 172.21.15.37:0/2722244246 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feaa000cad0 con 0x7feab0070890 2024-03-29T08:04:01.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:01.566+0000 7feab5600700 1 -- 172.21.15.37:0/2722244246 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feab01155c0 con 0x7feab0070890 2024-03-29T08:04:01.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:01.567+0000 7feab5600700 1 -- 172.21.15.37:0/2722244246 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feab0064dc0 con 0x7feab0070890 2024-03-29T08:04:01.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:01.567+0000 7feaa6a00700 1 -- 172.21.15.37:0/2722244246 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7feaa000cc30 con 0x7feab0070890 2024-03-29T08:04:01.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:01.568+0000 7feaa6a00700 1 --2- 172.21.15.37:0/2722244246 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7fea940692a0 0x7fea9406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:04:01.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:01.569+0000 7feaa6a00700 1 -- 172.21.15.37:0/2722244246 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5550+0+0 (secure 0 0 0) 0x7feaa0086cd0 con 0x7feab0070890 2024-03-29T08:04:01.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:01.569+0000 7feaae000700 1 --2- 172.21.15.37:0/2722244246 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7fea940692a0 0x7fea9406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:04:01.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:01.569+0000 7feaae000700 1 --2- 172.21.15.37:0/2722244246 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7fea940692a0 0x7fea9406b750 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fea9c009fd0 tx=0x7fea9c009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:04:01.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:01.572+0000 7feaa6a00700 1 -- 172.21.15.37:0/2722244246 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7feaa004e190 con 0x7feab0070890 2024-03-29T08:04:01.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:01.788+0000 7feab5600700 1 -- 172.21.15.37:0/2722244246 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7feab0115a10 con 0x7feab0070890 2024-03-29T08:04:01.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:01.789+0000 7feaa6a00700 1 -- 172.21.15.37:0/2722244246 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v47) v1 ==== 74+0+14388 (secure 0 0 0) 0x7feaa0056a70 con 0x7feab0070890 2024-03-29T08:04:01.789 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:04:01.789 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":47,"fsid":"99b492c2-eda1-11ee-b647-cb9ed24678a4","created":"2024-03-29T07:55:23.673934+0000","modified":"2024-03-29T08:03:46.637292+0000","last_up_change":"2024-03-29T08:02:04.552402+0000","last_in_change":"2024-03-29T08:01:43.066485+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T08:00:32.022416+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs.cephfs.meta","create_time":"2024-03-29T08:03:28.176923+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs.cephfs.data","create_time":"2024-03-29T08:03:28.754148+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"dba18fb6-b27d-4258-b769-dc89f4fef44d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":3851364162},{"type":"v1","addr":"172.21.15.37:6803","nonce":3851364162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":3851364162},{"type":"v1","addr":"172.21.15.37:6805","nonce":3851364162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":3851364162},{"type":"v1","addr":"172.21.15.37:6809","nonce":3851364162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":3851364162},{"type":"v1","addr":"172.21.15.37:6807","nonce":3851364162}]},"public_addr":"172.21.15.37:6803/3851364162","cluster_addr":"172.21.15.37:6805/3851364162","heartbeat_back_addr":"172.21.15.37:6809/3851364162","heartbeat_front_addr":"172.21.15.37:6807/3851364162","state":["exists","up"]},{"osd":1,"uuid":"2876dd3f-aefb-4298-8fe4-562245e1103e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":3949957441},{"type":"v1","addr":"172.21.15.37:6811","nonce":3949957441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":3949957441},{"type":"v1","addr":"172.21.15.37:6813","nonce":3949957441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":3949957441},{"type":"v1","addr":"172.21.15.37:6817","nonce":3949957441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":3949957441},{"type":"v1","addr":"172.21.15.37:6815","nonce":3949957441}]},"public_addr":"172.21.15.37:6811/3949957441","cluster_addr":"172.21.15.37:6813/3949957441","heartbeat_back_addr":"172.21.15.37:6817/3949957441","heartbeat_front_addr":"172.21.15.37:6815/3949957441","state":["exists","up"]},{"osd":2,"uuid":"bba0f95c-f4e3-404e-adf7-b90353a96af4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":4002901863},{"type":"v1","addr":"172.21.15.37:6819","nonce":4002901863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":4002901863},{"type":"v1","addr":"172.21.15.37:6821","nonce":4002901863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":4002901863},{"type":"v1","addr":"172.21.15.37:6825","nonce":4002901863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":4002901863},{"type":"v1","addr":"172.21.15.37:6823","nonce":4002901863}]},"public_addr":"172.21.15.37:6819/4002901863","cluster_addr":"172.21.15.37:6821/4002901863","heartbeat_back_addr":"172.21.15.37:6825/4002901863","heartbeat_front_addr":"172.21.15.37:6823/4002901863","state":["exists","up"]},{"osd":3,"uuid":"8da1c140-ada5-4879-b5d9-1ed81b0c9aef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":267019749},{"type":"v1","addr":"172.21.15.148:6801","nonce":267019749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":267019749},{"type":"v1","addr":"172.21.15.148:6803","nonce":267019749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":267019749},{"type":"v1","addr":"172.21.15.148:6807","nonce":267019749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":267019749},{"type":"v1","addr":"172.21.15.148:6805","nonce":267019749}]},"public_addr":"172.21.15.148:6801/267019749","cluster_addr":"172.21.15.148:6803/267019749","heartbeat_back_addr":"172.21.15.148:6807/267019749","heartbeat_front_addr":"172.21.15.148:6805/267019749","state":["exists","up"]},{"osd":4,"uuid":"9af70e66-bbca-48c7-bad1-7ac78270cf16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":4113691443},{"type":"v1","addr":"172.21.15.148:6809","nonce":4113691443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":4113691443},{"type":"v1","addr":"172.21.15.148:6811","nonce":4113691443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":4113691443},{"type":"v1","addr":"172.21.15.148:6815","nonce":4113691443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":4113691443},{"type":"v1","addr":"172.21.15.148:6813","nonce":4113691443}]},"public_addr":"172.21.15.148:6809/4113691443","cluster_addr":"172.21.15.148:6811/4113691443","heartbeat_back_addr":"172.21.15.148:6815/4113691443","heartbeat_front_addr":"172.21.15.148:6813/4113691443","state":["exists","up"]},{"osd":5,"uuid":"a66944e4-38bd-4403-b45f-7de01ddfb7a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":267287812},{"type":"v1","addr":"172.21.15.148:6817","nonce":267287812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":267287812},{"type":"v1","addr":"172.21.15.148:6819","nonce":267287812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":267287812},{"type":"v1","addr":"172.21.15.148:6823","nonce":267287812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":267287812},{"type":"v1","addr":"172.21.15.148:6821","nonce":267287812}]},"public_addr":"172.21.15.148:6817/267287812","cluster_addr":"172.21.15.148:6819/267287812","heartbeat_back_addr":"172.21.15.148:6823/267287812","heartbeat_front_addr":"172.21.15.148:6821/267287812","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T07:59:23.110900+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T07:59:55.540549+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:00:28.082290+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:00:59.496893+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:01:31.307445+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:02:03.191177+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.148:6825/2050993679":"2024-03-30T08:03:46.637260+0000","172.21.15.37:6800/2762339582":"2024-03-30T07:55:59.989528+0000","172.21.15.37:0/3116346321":"2024-03-30T07:55:59.989528+0000","172.21.15.37:6801/2746946658":"2024-03-30T07:56:36.633439+0000","172.21.15.37:6801/2762339582":"2024-03-30T07:55:59.989528+0000","172.21.15.37:0/3644857958":"2024-03-30T07:55:59.989528+0000","172.21.15.37:0/959705723":"2024-03-30T07:56:36.633439+0000","172.21.15.37:0/3846762095":"2024-03-30T07:57:55.402909+0000","172.21.15.37:0/1131673968":"2024-03-30T07:56:36.633439+0000","172.21.15.37:0/2483635161":"2024-03-30T07:56:36.633439+0000","172.21.15.37:0/368538117":"2024-03-30T07:57:55.402909+0000","172.21.15.37:6800/4245458578":"2024-03-30T07:57:55.402909+0000","172.21.15.37:6801/4245458578":"2024-03-30T07:57:55.402909+0000","172.21.15.37:6800/2746946658":"2024-03-30T07:56:36.633439+0000","172.21.15.37:0/1909774742":"2024-03-30T07:55:59.989528+0000","172.21.15.37:0/1421768676":"2024-03-30T07:57:55.402909+0000","172.21.15.148:6824/2050993679":"2024-03-30T08:03:46.637260+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T08:04:01.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:01.792+0000 7feab5600700 1 -- 172.21.15.37:0/2722244246 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7fea940692a0 msgr2=0x7fea9406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:04:01.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:01.792+0000 7feab5600700 1 --2- 172.21.15.37:0/2722244246 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7fea940692a0 0x7fea9406b750 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fea9c009fd0 tx=0x7fea9c009e30 comp rx=0 tx=0).stop 2024-03-29T08:04:01.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:01.792+0000 7feab5600700 1 -- 172.21.15.37:0/2722244246 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feab0070890 msgr2=0x7feab010f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:04:01.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:01.792+0000 7feab5600700 1 --2- 172.21.15.37:0/2722244246 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feab0070890 0x7feab010f950 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7feaa000eb70 tx=0x7feaa000b040 comp rx=0 tx=0).stop 2024-03-29T08:04:01.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:01.792+0000 7feab5600700 1 -- 172.21.15.37:0/2722244246 shutdown_connections 2024-03-29T08:04:01.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:01.792+0000 7feab5600700 1 --2- 172.21.15.37:0/2722244246 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7fea940692a0 0x7fea9406b750 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:04:01.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:01.792+0000 7feab5600700 1 --2- 172.21.15.37:0/2722244246 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feab0070890 0x7feab010f950 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:04:01.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:01.792+0000 7feab5600700 1 --2- 172.21.15.37:0/2722244246 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feab0071e40 0x7feab010fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:04:01.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:01.792+0000 7feab5600700 1 -- 172.21.15.37:0/2722244246 >> 172.21.15.37:0/2722244246 conn(0x7feab00f8410 msgr2=0x7feab00f9f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:04:01.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:01.792+0000 7feab5600700 1 -- 172.21.15.37:0/2722244246 shutdown_connections 2024-03-29T08:04:01.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:01.792+0000 7feab5600700 1 -- 172.21.15.37:0/2722244246 wait complete. 2024-03-29T08:04:01.810 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-29T08:00:32.022416+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'cephfs.cephfs.meta', 'create_time': '2024-03-29T08:03:28.176923+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 3, 'pool_name': 'cephfs.cephfs.data', 'create_time': '2024-03-29T08:03:28.754148+0000', 'flags': 131073, 'flags_names': 'hashpspool,bulk', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '46', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-29T08:04:01.811 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-29T08:04:02.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:02.010+0000 7f0490a00700 1 -- 172.21.15.37:0/569553605 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f048c1001a0 msgr2=0x7f048c100610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:04:02.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:02.010+0000 7f0490a00700 1 --2- 172.21.15.37:0/569553605 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f048c1001a0 0x7f048c100610 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f0478004940 tx=0x7f0478009e30 comp rx=0 tx=0).stop 2024-03-29T08:04:02.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:02.010+0000 7f0490a00700 1 -- 172.21.15.37:0/569553605 shutdown_connections 2024-03-29T08:04:02.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:02.010+0000 7f0490a00700 1 --2- 172.21.15.37:0/569553605 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f048c1001a0 0x7f048c100610 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:04:02.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:02.010+0000 7f0490a00700 1 --2- 172.21.15.37:0/569553605 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f048c0feef0 0x7f048c0ff300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:04:02.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:02.010+0000 7f0490a00700 1 -- 172.21.15.37:0/569553605 >> 172.21.15.37:0/569553605 conn(0x7f048c0fa790 msgr2=0x7f048c0fcb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:04:02.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:02.010+0000 7f0490a00700 1 -- 172.21.15.37:0/569553605 shutdown_connections 2024-03-29T08:04:02.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:02.010+0000 7f0490a00700 1 -- 172.21.15.37:0/569553605 wait complete. 2024-03-29T08:04:02.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:02.011+0000 7f0490a00700 1 Processor -- start 2024-03-29T08:04:02.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:02.011+0000 7f0490a00700 1 -- start start 2024-03-29T08:04:02.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:02.011+0000 7f0490a00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f048c0feef0 0x7f048c10f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:04:02.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:02.011+0000 7f0490a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f048c1001a0 0x7f048c10fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:04:02.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:02.011+0000 7f0490a00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f048c110400 con 0x7f048c1001a0 2024-03-29T08:04:02.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:02.011+0000 7f0490a00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f048c110570 con 0x7f048c0feef0 2024-03-29T08:04:02.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:02.012+0000 7f0489400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f048c1001a0 0x7f048c10fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:04:02.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:02.012+0000 7f0489400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f048c1001a0 0x7f048c10fe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60734/0 (socket says 172.21.15.37:60734) 2024-03-29T08:04:02.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:02.012+0000 7f0489400700 1 -- 172.21.15.37:0/3130520844 learned_addr learned my addr 172.21.15.37:0/3130520844 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:04:02.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:02.012+0000 7f0489e00700 1 --2- 172.21.15.37:0/3130520844 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f048c0feef0 0x7f048c10f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:04:02.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:02.012+0000 7f0489400700 1 -- 172.21.15.37:0/3130520844 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f048c0feef0 msgr2=0x7f048c10f8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:04:02.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:02.012+0000 7f0489400700 1 --2- 172.21.15.37:0/3130520844 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f048c0feef0 0x7f048c10f8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:04:02.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:02.012+0000 7f0489400700 1 -- 172.21.15.37:0/3130520844 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0478005040 con 0x7f048c1001a0 2024-03-29T08:04:02.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:02.012+0000 7f0489400700 1 --2- 172.21.15.37:0/3130520844 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f048c1001a0 0x7f048c10fe00 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f04780137b0 tx=0x7f0478009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:04:02.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:02.012+0000 7f0482000700 1 -- 172.21.15.37:0/3130520844 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0478021070 con 0x7f048c1001a0 2024-03-29T08:04:02.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:02.013+0000 7f0482000700 1 -- 172.21.15.37:0/3130520844 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f047801c400 con 0x7f048c1001a0 2024-03-29T08:04:02.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:02.013+0000 7f0482000700 1 -- 172.21.15.37:0/3130520844 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f047800e070 con 0x7f048c1001a0 2024-03-29T08:04:02.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:02.013+0000 7f0490a00700 1 -- 172.21.15.37:0/3130520844 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f048c114f70 con 0x7f048c1001a0 2024-03-29T08:04:02.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:02.013+0000 7f0490a00700 1 -- 172.21.15.37:0/3130520844 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f048c115450 con 0x7f048c1001a0 2024-03-29T08:04:02.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:02.014+0000 7f0490a00700 1 -- 172.21.15.37:0/3130520844 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f048c064970 con 0x7f048c1001a0 2024-03-29T08:04:02.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:02.015+0000 7f0482000700 1 -- 172.21.15.37:0/3130520844 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f047801c570 con 0x7f048c1001a0 2024-03-29T08:04:02.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:02.015+0000 7f0482000700 1 --2- 172.21.15.37:0/3130520844 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f04700692a0 0x7f047006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:04:02.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:02.016+0000 7f0482000700 1 -- 172.21.15.37:0/3130520844 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f04780884a0 con 0x7f048c1001a0 2024-03-29T08:04:02.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:02.016+0000 7f0489e00700 1 --2- 172.21.15.37:0/3130520844 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f04700692a0 0x7f047006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:04:02.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:02.016+0000 7f0489e00700 1 --2- 172.21.15.37:0/3130520844 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f04700692a0 0x7f047006b750 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f047c004600 tx=0x7f047c00b040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:04:02.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:02.020+0000 7f0482000700 1 -- 172.21.15.37:0/3130520844 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f0478058170 con 0x7f048c1001a0 2024-03-29T08:04:02.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:02.260+0000 7f0490a00700 1 -- 172.21.15.37:0/3130520844 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f048c1156c0 con 0x7f048c1001a0 2024-03-29T08:04:02.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:02.261+0000 7f0482000700 1 -- 172.21.15.37:0/3130520844 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v47) v1 ==== 93+0+10 (secure 0 0 0) 0x7f047801c820 con 0x7f048c1001a0 2024-03-29T08:04:02.260 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 1 2024-03-29T08:04:02.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:02.264+0000 7f0490a00700 1 -- 172.21.15.37:0/3130520844 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f04700692a0 msgr2=0x7f047006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:04:02.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:02.264+0000 7f0490a00700 1 --2- 172.21.15.37:0/3130520844 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f04700692a0 0x7f047006b750 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f047c004600 tx=0x7f047c00b040 comp rx=0 tx=0).stop 2024-03-29T08:04:02.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:02.264+0000 7f0490a00700 1 -- 172.21.15.37:0/3130520844 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f048c1001a0 msgr2=0x7f048c10fe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:04:02.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:02.264+0000 7f0490a00700 1 --2- 172.21.15.37:0/3130520844 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f048c1001a0 0x7f048c10fe00 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f04780137b0 tx=0x7f0478009e30 comp rx=0 tx=0).stop 2024-03-29T08:04:02.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:02.264+0000 7f0490a00700 1 -- 172.21.15.37:0/3130520844 shutdown_connections 2024-03-29T08:04:02.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:02.264+0000 7f0490a00700 1 --2- 172.21.15.37:0/3130520844 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f04700692a0 0x7f047006b750 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:04:02.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:02.264+0000 7f0490a00700 1 --2- 172.21.15.37:0/3130520844 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f048c0feef0 0x7f048c10f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:04:02.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:02.264+0000 7f0490a00700 1 --2- 172.21.15.37:0/3130520844 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f048c1001a0 0x7f048c10fe00 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:04:02.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:02.264+0000 7f0490a00700 1 -- 172.21.15.37:0/3130520844 >> 172.21.15.37:0/3130520844 conn(0x7f048c0fa790 msgr2=0x7f048c0fce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:04:02.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:02.264+0000 7f0490a00700 1 -- 172.21.15.37:0/3130520844 shutdown_connections 2024-03-29T08:04:02.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:02.264+0000 7f0490a00700 1 -- 172.21.15.37:0/3130520844 wait complete. 2024-03-29T08:04:02.286 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.meta pg_num 2024-03-29T08:04:02.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:02.513+0000 7f1707e00700 1 -- 172.21.15.37:0/1584516461 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f17000fdf70 msgr2=0x7f17000fe3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:04:02.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:02.513+0000 7f1707e00700 1 --2- 172.21.15.37:0/1584516461 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f17000fdf70 0x7f17000fe3e0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f16ec004940 tx=0x7f16ec009e30 comp rx=0 tx=0).stop 2024-03-29T08:04:02.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:02.513+0000 7f1707e00700 1 -- 172.21.15.37:0/1584516461 shutdown_connections 2024-03-29T08:04:02.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:02.513+0000 7f1707e00700 1 --2- 172.21.15.37:0/1584516461 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f17000fdf70 0x7f17000fe3e0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:04:02.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:02.513+0000 7f1707e00700 1 --2- 172.21.15.37:0/1584516461 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f17000fccc0 0x7f17000fd0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:04:02.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:02.513+0000 7f1707e00700 1 -- 172.21.15.37:0/1584516461 >> 172.21.15.37:0/1584516461 conn(0x7f17000f8440 msgr2=0x7f17000fa850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:04:02.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:02.513+0000 7f1707e00700 1 -- 172.21.15.37:0/1584516461 shutdown_connections 2024-03-29T08:04:02.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:02.513+0000 7f1707e00700 1 -- 172.21.15.37:0/1584516461 wait complete. 2024-03-29T08:04:02.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:02.514+0000 7f1707e00700 1 Processor -- start 2024-03-29T08:04:02.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:02.515+0000 7f1707e00700 1 -- start start 2024-03-29T08:04:02.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:02.515+0000 7f1707e00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f17000fccc0 0x7f170010d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:04:02.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:02.515+0000 7f1707e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f17000fdf70 0x7f170010dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:04:02.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:02.515+0000 7f1707e00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f170010e200 con 0x7f17000fdf70 2024-03-29T08:04:02.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:02.515+0000 7f1707e00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f170010e370 con 0x7f17000fccc0 2024-03-29T08:04:02.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:02.515+0000 7f1704a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f17000fdf70 0x7f170010dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:04:02.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:02.515+0000 7f1704a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f17000fdf70 0x7f170010dc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60742/0 (socket says 172.21.15.37:60742) 2024-03-29T08:04:02.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:02.515+0000 7f1704a00700 1 -- 172.21.15.37:0/1558171743 learned_addr learned my addr 172.21.15.37:0/1558171743 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:04:02.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:02.516+0000 7f1705400700 1 --2- 172.21.15.37:0/1558171743 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f17000fccc0 0x7f170010d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:04:02.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:02.516+0000 7f1704a00700 1 -- 172.21.15.37:0/1558171743 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f17000fccc0 msgr2=0x7f170010d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:04:02.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:02.516+0000 7f1704a00700 1 --2- 172.21.15.37:0/1558171743 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f17000fccc0 0x7f170010d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:04:02.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:02.516+0000 7f1704a00700 1 -- 172.21.15.37:0/1558171743 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16ec005040 con 0x7f17000fdf70 2024-03-29T08:04:02.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:02.516+0000 7f1704a00700 1 --2- 172.21.15.37:0/1558171743 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f17000fdf70 0x7f170010dc00 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f16ec0137b0 tx=0x7f16ec009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:04:02.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:02.516+0000 7f16fd600700 1 -- 172.21.15.37:0/1558171743 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f16ec021070 con 0x7f17000fdf70 2024-03-29T08:04:02.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:02.516+0000 7f16fd600700 1 -- 172.21.15.37:0/1558171743 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f16ec01c400 con 0x7f17000fdf70 2024-03-29T08:04:02.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:02.516+0000 7f1707e00700 1 -- 172.21.15.37:0/1558171743 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1700112d20 con 0x7f17000fdf70 2024-03-29T08:04:02.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:02.516+0000 7f16fd600700 1 -- 172.21.15.37:0/1558171743 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f16ec00e070 con 0x7f17000fdf70 2024-03-29T08:04:02.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:02.516+0000 7f1707e00700 1 -- 172.21.15.37:0/1558171743 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1700113280 con 0x7f17000fdf70 2024-03-29T08:04:02.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:02.518+0000 7f1707e00700 1 -- 172.21.15.37:0/1558171743 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1700064970 con 0x7f17000fdf70 2024-03-29T08:04:02.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:02.518+0000 7f16fd600700 1 -- 172.21.15.37:0/1558171743 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f16ec01c570 con 0x7f17000fdf70 2024-03-29T08:04:02.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:02.519+0000 7f16fd600700 1 --2- 172.21.15.37:0/1558171743 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f16e80692a0 0x7f16e806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:04:02.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:02.519+0000 7f16fd600700 1 -- 172.21.15.37:0/1558171743 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f16ec088450 con 0x7f17000fdf70 2024-03-29T08:04:02.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:02.519+0000 7f1705400700 1 --2- 172.21.15.37:0/1558171743 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f16e80692a0 0x7f16e806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:04:02.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:02.520+0000 7f1705400700 1 --2- 172.21.15.37:0/1558171743 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f16e80692a0 0x7f16e806b750 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f16f4004600 tx=0x7f16f400b040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:04:02.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:02.523+0000 7f16fd600700 1 -- 172.21.15.37:0/1558171743 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f16ec0581a0 con 0x7f17000fdf70 2024-03-29T08:04:02.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:04:02 smithi037 ceph-mon[35162]: pgmap v218: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 3.3 KiB/s rd, 0 B/s wr, 3 op/s 2024-03-29T08:04:02.678 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:04:02 smithi037 ceph-mon[35162]: from='client.? 172.21.15.37:0/2722244246' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-29T08:04:02.678 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:04:02 smithi037 ceph-mon[35162]: from='client.? 172.21.15.37:0/3130520844' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-29T08:04:02.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:02.771+0000 7f1707e00700 1 -- 172.21.15.37:0/1558171743 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"} v 0) v1 -- 0x7f17001136e0 con 0x7f17000fdf70 2024-03-29T08:04:02.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:02.771+0000 7f16fd600700 1 -- 172.21.15.37:0/1558171743 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]=0 v47) v1 ==== 107+0+11 (secure 0 0 0) 0x7f16ec01c820 con 0x7f17000fdf70 2024-03-29T08:04:02.771 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 32 2024-03-29T08:04:02.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:02.773+0000 7f1707e00700 1 -- 172.21.15.37:0/1558171743 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f16e80692a0 msgr2=0x7f16e806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:04:02.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:02.773+0000 7f1707e00700 1 --2- 172.21.15.37:0/1558171743 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f16e80692a0 0x7f16e806b750 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f16f4004600 tx=0x7f16f400b040 comp rx=0 tx=0).stop 2024-03-29T08:04:02.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:02.773+0000 7f1707e00700 1 -- 172.21.15.37:0/1558171743 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f17000fdf70 msgr2=0x7f170010dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:04:02.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:02.773+0000 7f1707e00700 1 --2- 172.21.15.37:0/1558171743 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f17000fdf70 0x7f170010dc00 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f16ec0137b0 tx=0x7f16ec009e30 comp rx=0 tx=0).stop 2024-03-29T08:04:02.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:02.773+0000 7f1707e00700 1 -- 172.21.15.37:0/1558171743 shutdown_connections 2024-03-29T08:04:02.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:02.773+0000 7f1707e00700 1 --2- 172.21.15.37:0/1558171743 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f16e80692a0 0x7f16e806b750 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:04:02.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:02.773+0000 7f1707e00700 1 --2- 172.21.15.37:0/1558171743 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f17000fccc0 0x7f170010d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:04:02.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:02.773+0000 7f1707e00700 1 --2- 172.21.15.37:0/1558171743 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f17000fdf70 0x7f170010dc00 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:04:02.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:02.773+0000 7f1707e00700 1 -- 172.21.15.37:0/1558171743 >> 172.21.15.37:0/1558171743 conn(0x7f17000f8440 msgr2=0x7f17000f9b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:04:02.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:02.774+0000 7f1707e00700 1 -- 172.21.15.37:0/1558171743 shutdown_connections 2024-03-29T08:04:02.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:02.774+0000 7f1707e00700 1 -- 172.21.15.37:0/1558171743 wait complete. 2024-03-29T08:04:02.791 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.data pg_num 2024-03-29T08:04:02.805 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:04:02 smithi148 ceph-mon[44992]: pgmap v218: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 3.3 KiB/s rd, 0 B/s wr, 3 op/s 2024-03-29T08:04:02.805 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:04:02 smithi148 ceph-mon[44992]: from='client.? 172.21.15.37:0/2722244246' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-29T08:04:02.805 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:04:02 smithi148 ceph-mon[44992]: from='client.? 172.21.15.37:0/3130520844' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-29T08:04:02.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:02.987+0000 7f2eb7400700 1 -- 172.21.15.37:0/1690679074 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2eb00fccb0 msgr2=0x7f2eb00fd0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:04:02.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:02.987+0000 7f2eb7400700 1 --2- 172.21.15.37:0/1690679074 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2eb00fccb0 0x7f2eb00fd0c0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f2ea4004940 tx=0x7f2ea4009e30 comp rx=0 tx=0).stop 2024-03-29T08:04:02.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:02.987+0000 7f2eb7400700 1 -- 172.21.15.37:0/1690679074 shutdown_connections 2024-03-29T08:04:02.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:02.987+0000 7f2eb7400700 1 --2- 172.21.15.37:0/1690679074 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2eb00fdf60 0x7f2eb00fe3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:04:02.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:02.987+0000 7f2eb7400700 1 --2- 172.21.15.37:0/1690679074 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2eb00fccb0 0x7f2eb00fd0c0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:04:02.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:02.987+0000 7f2eb7400700 1 -- 172.21.15.37:0/1690679074 >> 172.21.15.37:0/1690679074 conn(0x7f2eb00f8430 msgr2=0x7f2eb00fa840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:04:02.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:02.987+0000 7f2eb7400700 1 -- 172.21.15.37:0/1690679074 shutdown_connections 2024-03-29T08:04:02.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:02.987+0000 7f2eb7400700 1 -- 172.21.15.37:0/1690679074 wait complete. 2024-03-29T08:04:02.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:02.988+0000 7f2eb7400700 1 Processor -- start 2024-03-29T08:04:02.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:02.988+0000 7f2eb7400700 1 -- start start 2024-03-29T08:04:02.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:02.988+0000 7f2eb7400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2eb00fccb0 0x7f2eb010f910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:04:02.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:02.989+0000 7f2eb7400700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2eb00fdf60 0x7f2eb010fe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:04:02.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:02.989+0000 7f2eb7400700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2eb0110450 con 0x7f2eb00fccb0 2024-03-29T08:04:02.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:02.989+0000 7f2eb7400700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2eb01105c0 con 0x7f2eb00fdf60 2024-03-29T08:04:02.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:02.989+0000 7f2eb4a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2eb00fccb0 0x7f2eb010f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:04:02.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:02.989+0000 7f2eb4a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2eb00fccb0 0x7f2eb010f910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60766/0 (socket says 172.21.15.37:60766) 2024-03-29T08:04:02.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:02.989+0000 7f2eb4a00700 1 -- 172.21.15.37:0/1369498702 learned_addr learned my addr 172.21.15.37:0/1369498702 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:04:02.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:02.989+0000 7f2eafe00700 1 --2- 172.21.15.37:0/1369498702 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2eb00fdf60 0x7f2eb010fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:04:02.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:02.989+0000 7f2eb4a00700 1 -- 172.21.15.37:0/1369498702 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2eb00fdf60 msgr2=0x7f2eb010fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:04:02.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:02.989+0000 7f2eb4a00700 1 --2- 172.21.15.37:0/1369498702 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2eb00fdf60 0x7f2eb010fe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:04:02.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:02.989+0000 7f2eb4a00700 1 -- 172.21.15.37:0/1369498702 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ea4005040 con 0x7f2eb00fccb0 2024-03-29T08:04:02.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:02.989+0000 7f2eafe00700 1 --2- 172.21.15.37:0/1369498702 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2eb00fdf60 0x7f2eb010fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:04:02.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:02.989+0000 7f2eb4a00700 1 --2- 172.21.15.37:0/1369498702 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2eb00fccb0 0x7f2eb010f910 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f2ea400b3b0 tx=0x7f2ea4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:04:02.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:02.990+0000 7f2eacc00700 1 -- 172.21.15.37:0/1369498702 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2ea400e040 con 0x7f2eb00fccb0 2024-03-29T08:04:02.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:02.990+0000 7f2eacc00700 1 -- 172.21.15.37:0/1369498702 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f2ea4004290 con 0x7f2eb00fccb0 2024-03-29T08:04:02.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:02.990+0000 7f2eb7400700 1 -- 172.21.15.37:0/1369498702 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2eb0114fc0 con 0x7f2eb00fccb0 2024-03-29T08:04:02.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:02.990+0000 7f2eacc00700 1 -- 172.21.15.37:0/1369498702 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2ea401c3f0 con 0x7f2eb00fccb0 2024-03-29T08:04:02.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:02.990+0000 7f2eb7400700 1 -- 172.21.15.37:0/1369498702 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2eb01154f0 con 0x7f2eb00fccb0 2024-03-29T08:04:02.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:02.991+0000 7f2eb7400700 1 -- 172.21.15.37:0/1369498702 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2eb01018f0 con 0x7f2eb00fccb0 2024-03-29T08:04:02.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:02.992+0000 7f2eacc00700 1 -- 172.21.15.37:0/1369498702 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f2ea4013420 con 0x7f2eb00fccb0 2024-03-29T08:04:02.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:02.993+0000 7f2eacc00700 1 --2- 172.21.15.37:0/1369498702 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f2e980692a0 0x7f2e9806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:04:02.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:02.993+0000 7f2eacc00700 1 -- 172.21.15.37:0/1369498702 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f2ea40884c0 con 0x7f2eb00fccb0 2024-03-29T08:04:02.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:02.993+0000 7f2eafe00700 1 --2- 172.21.15.37:0/1369498702 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f2e980692a0 0x7f2e9806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:04:02.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:02.993+0000 7f2eafe00700 1 --2- 172.21.15.37:0/1369498702 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f2e980692a0 0x7f2e9806b750 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f2e9c009fd0 tx=0x7f2e9c009ee0 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:04:02.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:02.997+0000 7f2eacc00700 1 -- 172.21.15.37:0/1369498702 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f2ea4058210 con 0x7f2eb00fccb0 2024-03-29T08:04:03.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:03.203+0000 7f2eb7400700 1 -- 172.21.15.37:0/1369498702 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"} v 0) v1 -- 0x7f2eb0064970 con 0x7f2eb00fccb0 2024-03-29T08:04:03.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:03.204+0000 7f2eacc00700 1 -- 172.21.15.37:0/1369498702 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]=0 v47) v1 ==== 107+0+11 (secure 0 0 0) 0x7f2ea4057da0 con 0x7f2eb00fccb0 2024-03-29T08:04:03.203 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 32 2024-03-29T08:04:03.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:03.206+0000 7f2eb7400700 1 -- 172.21.15.37:0/1369498702 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f2e980692a0 msgr2=0x7f2e9806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:04:03.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:03.206+0000 7f2eb7400700 1 --2- 172.21.15.37:0/1369498702 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f2e980692a0 0x7f2e9806b750 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f2e9c009fd0 tx=0x7f2e9c009ee0 comp rx=0 tx=0).stop 2024-03-29T08:04:03.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:03.206+0000 7f2eb7400700 1 -- 172.21.15.37:0/1369498702 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2eb00fccb0 msgr2=0x7f2eb010f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:04:03.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:03.206+0000 7f2eb7400700 1 --2- 172.21.15.37:0/1369498702 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2eb00fccb0 0x7f2eb010f910 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f2ea400b3b0 tx=0x7f2ea4009e30 comp rx=0 tx=0).stop 2024-03-29T08:04:03.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:03.206+0000 7f2eb7400700 1 -- 172.21.15.37:0/1369498702 shutdown_connections 2024-03-29T08:04:03.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:03.206+0000 7f2eb7400700 1 --2- 172.21.15.37:0/1369498702 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f2e980692a0 0x7f2e9806b750 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:04:03.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:03.206+0000 7f2eb7400700 1 --2- 172.21.15.37:0/1369498702 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2eb00fccb0 0x7f2eb010f910 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:04:03.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:03.206+0000 7f2eb7400700 1 --2- 172.21.15.37:0/1369498702 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2eb00fdf60 0x7f2eb010fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:04:03.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:03.206+0000 7f2eb7400700 1 -- 172.21.15.37:0/1369498702 >> 172.21.15.37:0/1369498702 conn(0x7f2eb00f8430 msgr2=0x7f2eb00f9b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:04:03.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:03.207+0000 7f2eb7400700 1 -- 172.21.15.37:0/1369498702 shutdown_connections 2024-03-29T08:04:03.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:03.207+0000 7f2eb7400700 1 -- 172.21.15.37:0/1369498702 wait complete. 2024-03-29T08:04:03.224 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-29T08:04:03.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:03.416+0000 7f3c1be00700 1 -- 172.21.15.37:0/2297039451 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3c140fdf80 msgr2=0x7f3c140fe3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:04:03.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:03.416+0000 7f3c1be00700 1 --2- 172.21.15.37:0/2297039451 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3c140fdf80 0x7f3c140fe3f0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f3c00004940 tx=0x7f3c00009e30 comp rx=0 tx=0).stop 2024-03-29T08:04:03.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:03.416+0000 7f3c1be00700 1 -- 172.21.15.37:0/2297039451 shutdown_connections 2024-03-29T08:04:03.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:03.416+0000 7f3c1be00700 1 --2- 172.21.15.37:0/2297039451 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3c140fdf80 0x7f3c140fe3f0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:04:03.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:03.416+0000 7f3c1be00700 1 --2- 172.21.15.37:0/2297039451 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3c140fccd0 0x7f3c140fd0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:04:03.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:03.416+0000 7f3c1be00700 1 -- 172.21.15.37:0/2297039451 >> 172.21.15.37:0/2297039451 conn(0x7f3c140f8430 msgr2=0x7f3c140fa860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:04:03.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:03.416+0000 7f3c1be00700 1 -- 172.21.15.37:0/2297039451 shutdown_connections 2024-03-29T08:04:03.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:03.416+0000 7f3c1be00700 1 -- 172.21.15.37:0/2297039451 wait complete. 2024-03-29T08:04:03.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:03.417+0000 7f3c1be00700 1 Processor -- start 2024-03-29T08:04:03.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:03.418+0000 7f3c1be00700 1 -- start start 2024-03-29T08:04:03.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:03.418+0000 7f3c1be00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3c140fccd0 0x7f3c14071c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:04:03.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:03.418+0000 7f3c1be00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3c140fdf80 0x7f3c14070310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:04:03.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:03.418+0000 7f3c1be00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c14072260 con 0x7f3c140fdf80 2024-03-29T08:04:03.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:03.418+0000 7f3c1be00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c14070850 con 0x7f3c140fccd0 2024-03-29T08:04:03.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:03.418+0000 7f3c18a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3c140fdf80 0x7f3c14070310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:04:03.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:03.418+0000 7f3c18a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3c140fdf80 0x7f3c14070310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60792/0 (socket says 172.21.15.37:60792) 2024-03-29T08:04:03.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:03.418+0000 7f3c18a00700 1 -- 172.21.15.37:0/334789905 learned_addr learned my addr 172.21.15.37:0/334789905 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:04:03.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:03.418+0000 7f3c19400700 1 --2- 172.21.15.37:0/334789905 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3c140fccd0 0x7f3c14071c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:04:03.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:03.419+0000 7f3c18a00700 1 -- 172.21.15.37:0/334789905 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3c140fccd0 msgr2=0x7f3c14071c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:04:03.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:03.419+0000 7f3c18a00700 1 --2- 172.21.15.37:0/334789905 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3c140fccd0 0x7f3c14071c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:04:03.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:03.419+0000 7f3c18a00700 1 -- 172.21.15.37:0/334789905 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c00005040 con 0x7f3c140fdf80 2024-03-29T08:04:03.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:03.419+0000 7f3c18a00700 1 --2- 172.21.15.37:0/334789905 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3c140fdf80 0x7f3c14070310 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f3c00009fd0 tx=0x7f3c00009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:04:03.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:03.419+0000 7f3c11600700 1 -- 172.21.15.37:0/334789905 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3c0000e040 con 0x7f3c140fdf80 2024-03-29T08:04:03.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:03.419+0000 7f3c11600700 1 -- 172.21.15.37:0/334789905 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f3c00004290 con 0x7f3c140fdf80 2024-03-29T08:04:03.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:03.419+0000 7f3c1be00700 1 -- 172.21.15.37:0/334789905 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c14070aa0 con 0x7f3c140fdf80 2024-03-29T08:04:03.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:03.419+0000 7f3c11600700 1 -- 172.21.15.37:0/334789905 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3c0001c3f0 con 0x7f3c140fdf80 2024-03-29T08:04:03.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:03.419+0000 7f3c1be00700 1 -- 172.21.15.37:0/334789905 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c14071060 con 0x7f3c140fdf80 2024-03-29T08:04:03.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:03.420+0000 7f3c1be00700 1 -- 172.21.15.37:0/334789905 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c14064970 con 0x7f3c140fdf80 2024-03-29T08:04:03.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:03.421+0000 7f3c11600700 1 -- 172.21.15.37:0/334789905 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f3c00013790 con 0x7f3c140fdf80 2024-03-29T08:04:03.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:03.422+0000 7f3c11600700 1 --2- 172.21.15.37:0/334789905 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f3bfc0692a0 0x7f3bfc06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:04:03.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:03.423+0000 7f3c11600700 1 -- 172.21.15.37:0/334789905 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f3c00088580 con 0x7f3c140fdf80 2024-03-29T08:04:03.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:03.423+0000 7f3c19400700 1 --2- 172.21.15.37:0/334789905 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f3bfc0692a0 0x7f3bfc06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:04:03.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:03.423+0000 7f3c19400700 1 --2- 172.21.15.37:0/334789905 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f3bfc0692a0 0x7f3bfc06b750 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f3c08004f40 tx=0x7f3c0800b040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:04:03.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:03.427+0000 7f3c11600700 1 -- 172.21.15.37:0/334789905 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f3c000582d0 con 0x7f3c140fdf80 2024-03-29T08:04:03.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:03.682+0000 7f3c1be00700 1 -- 172.21.15.37:0/334789905 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3c14105680 con 0x7f3c140fdf80 2024-03-29T08:04:03.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:03.682+0000 7f3c11600700 1 -- 172.21.15.37:0/334789905 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4709 (secure 0 0 0) 0x7f3c00013420 con 0x7f3c140fdf80 2024-03-29T08:04:03.682 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:04:03.682 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":16,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24319,"name":"cephfs.smithi148.qukpjh","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.148:6827/2641327912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2641327912},{"type":"v1","addr":"172.21.15.148:6827","nonce":2641327912}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15},{"gid":24327,"name":"cephfs.smithi148.sqteih","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6825/3252351726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":3252351726},{"type":"v1","addr":"172.21.15.148:6825","nonce":3252351726}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-29T08:03:29.241269+0000","modified":"2024-03-29T08:03:50.664746+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":47,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":14470,"mds_1":14482},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14470":{"gid":14470,"name":"cephfs.smithi037.mmwgxe","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.37:6827/1559618909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1559618909},{"type":"v1","addr":"172.21.15.37:6827","nonce":1559618909}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14482":{"gid":14482,"name":"cephfs.smithi037.iifsmh","rank":1,"incarnation":12,"state":"up:active","state_seq":6,"addr":"172.21.15.37:6829/2341530495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":2341530495},{"type":"v1","addr":"172.21.15.37:6829","nonce":2341530495}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":1},"id":1}]} 2024-03-29T08:04:03.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:03.684+0000 7f3c1be00700 1 -- 172.21.15.37:0/334789905 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f3bfc0692a0 msgr2=0x7f3bfc06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:04:03.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:03.684+0000 7f3c1be00700 1 --2- 172.21.15.37:0/334789905 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f3bfc0692a0 0x7f3bfc06b750 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f3c08004f40 tx=0x7f3c0800b040 comp rx=0 tx=0).stop 2024-03-29T08:04:03.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:03.684+0000 7f3c1be00700 1 -- 172.21.15.37:0/334789905 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3c140fdf80 msgr2=0x7f3c14070310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:04:03.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:03.684+0000 7f3c1be00700 1 --2- 172.21.15.37:0/334789905 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3c140fdf80 0x7f3c14070310 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f3c00009fd0 tx=0x7f3c00009e30 comp rx=0 tx=0).stop 2024-03-29T08:04:03.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:03.685+0000 7f3c1be00700 1 -- 172.21.15.37:0/334789905 shutdown_connections 2024-03-29T08:04:03.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:03.685+0000 7f3c1be00700 1 --2- 172.21.15.37:0/334789905 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f3bfc0692a0 0x7f3bfc06b750 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:04:03.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:03.685+0000 7f3c1be00700 1 --2- 172.21.15.37:0/334789905 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3c140fccd0 0x7f3c14071c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:04:03.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:03.685+0000 7f3c1be00700 1 --2- 172.21.15.37:0/334789905 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3c140fdf80 0x7f3c14070310 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:04:03.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:03.685+0000 7f3c1be00700 1 -- 172.21.15.37:0/334789905 >> 172.21.15.37:0/334789905 conn(0x7f3c140f8430 msgr2=0x7f3c140f9f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:04:03.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:03.685+0000 7f3c1be00700 1 -- 172.21.15.37:0/334789905 shutdown_connections 2024-03-29T08:04:03.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:03.685+0000 7f3c1be00700 1 -- 172.21.15.37:0/334789905 wait complete. 2024-03-29T08:04:03.685 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 16 2024-03-29T08:04:03.702 DEBUG:tasks.fs:fs fscid=1,name=cephfs state = {'epoch': 16, 'max_mds': 2, 'flags': 18} 2024-03-29T08:04:03.702 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-03-29T08:04:03.726 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-03-29T08:04:03.726 INFO:tasks.ceph_fuse:config is {'client.0': {}, 'client.1': {}} 2024-03-29T08:04:03.727 INFO:tasks.ceph_fuse:client.0 config is {} 2024-03-29T08:04:03.727 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-29T08:04:03.727 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-29T08:04:03.727 INFO:tasks.ceph_fuse:client.1 config is {} 2024-03-29T08:04:03.727 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-29T08:04:03.727 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-03-29T08:04:03.727 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:04:03.727 DEBUG:teuthology.orchestra.run.smithi037:> ip netns list 2024-03-29T08:04:03.773 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:04:03.773 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip link delete ceph-brx 2024-03-29T08:04:03.885 INFO:teuthology.orchestra.run.smithi037.stderr:Cannot find device "ceph-brx" 2024-03-29T08:04:03.885 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:04:03 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:04:03.885 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:04:03 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:04:03.885 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:04:03 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T08:04:03.886 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:04:03 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T08:04:03.886 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:04:03 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:04:03.886 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:04:03 smithi037 ceph-mon[35162]: from='client.? 172.21.15.37:0/1558171743' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-03-29T08:04:03.886 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:04:03 smithi037 ceph-mon[35162]: from='client.? 172.21.15.37:0/1369498702' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-03-29T08:04:03.887 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-29T08:04:03.888 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:04:03.888 DEBUG:teuthology.orchestra.run.smithi148:> ip netns list 2024-03-29T08:04:03.940 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:04:03.940 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip link delete ceph-brx 2024-03-29T08:04:04.004 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:04:03 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:04:04.004 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:04:03 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:04:04.005 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:04:03 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T08:04:04.005 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:04:03 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T08:04:04.005 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:04:03 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:04:04.005 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:04:03 smithi148 ceph-mon[44992]: from='client.? 172.21.15.37:0/1558171743' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-03-29T08:04:04.005 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:04:03 smithi148 ceph-mon[44992]: from='client.? 172.21.15.37:0/1369498702' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-03-29T08:04:04.018 INFO:teuthology.orchestra.run.smithi148.stderr:Cannot find device "ceph-brx" 2024-03-29T08:04:04.021 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-29T08:04:04.021 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-03-29T08:04:04.022 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 99b492c2-eda1-11ee-b647-cb9ed24678a4 -- ceph fs ls 2024-03-29T08:04:04.454 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/mon.smithi037/config 2024-03-29T08:04:04.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:04:04 smithi037 ceph-mon[35162]: pgmap v219: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-03-29T08:04:04.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:04:04 smithi037 ceph-mon[35162]: from='client.? 172.21.15.37:0/334789905' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-29T08:04:05.055 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:04:04 smithi148 ceph-mon[44992]: pgmap v219: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-03-29T08:04:05.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:04:04 smithi148 ceph-mon[44992]: from='client.? 172.21.15.37:0/334789905' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-29T08:04:06.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:06.339+0000 7f7cc8600700 1 -- 172.21.15.37:0/245848073 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7cc0102e00 msgr2=0x7f7cc01031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:04:06.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:06.339+0000 7f7cc8600700 1 --2- 172.21.15.37:0/245848073 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7cc0102e00 0x7f7cc01031d0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f7cb4004940 tx=0x7f7cb4009e30 comp rx=0 tx=0).stop 2024-03-29T08:04:06.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:06.339+0000 7f7cc8600700 1 -- 172.21.15.37:0/245848073 shutdown_connections 2024-03-29T08:04:06.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:06.339+0000 7f7cc8600700 1 --2- 172.21.15.37:0/245848073 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7cc01037a0 0x7f7cc00fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:04:06.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:06.339+0000 7f7cc8600700 1 --2- 172.21.15.37:0/245848073 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7cc0102e00 0x7f7cc01031d0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:04:06.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:06.339+0000 7f7cc8600700 1 -- 172.21.15.37:0/245848073 >> 172.21.15.37:0/245848073 conn(0x7f7cc00f89b0 msgr2=0x7f7cc00fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:04:06.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:06.340+0000 7f7cc8600700 1 -- 172.21.15.37:0/245848073 shutdown_connections 2024-03-29T08:04:06.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:06.340+0000 7f7cc8600700 1 -- 172.21.15.37:0/245848073 wait complete. 2024-03-29T08:04:06.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:06.340+0000 7f7cc8600700 1 Processor -- start 2024-03-29T08:04:06.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:06.341+0000 7f7cc8600700 1 -- start start 2024-03-29T08:04:06.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:06.341+0000 7f7cc8600700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7cc0102e00 0x7f7cc0113ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:04:06.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:06.341+0000 7f7cc8600700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7cc01037a0 0x7f7cc0114000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:04:06.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:06.341+0000 7f7cc8600700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cc0114650 con 0x7f7cc01037a0 2024-03-29T08:04:06.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:06.341+0000 7f7cc8600700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cc010da80 con 0x7f7cc0102e00 2024-03-29T08:04:06.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:06.341+0000 7f7cc5200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7cc01037a0 0x7f7cc0114000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:04:06.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:06.341+0000 7f7cc5200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7cc01037a0 0x7f7cc0114000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60802/0 (socket says 172.21.15.37:60802) 2024-03-29T08:04:06.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:06.341+0000 7f7cc5200700 1 -- 172.21.15.37:0/211601240 learned_addr learned my addr 172.21.15.37:0/211601240 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:04:06.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:06.341+0000 7f7cc5c00700 1 --2- 172.21.15.37:0/211601240 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7cc0102e00 0x7f7cc0113ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:04:06.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:06.342+0000 7f7cc5200700 1 -- 172.21.15.37:0/211601240 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7cc0102e00 msgr2=0x7f7cc0113ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:04:06.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:06.342+0000 7f7cc5200700 1 --2- 172.21.15.37:0/211601240 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7cc0102e00 0x7f7cc0113ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:04:06.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:06.342+0000 7f7cc5200700 1 -- 172.21.15.37:0/211601240 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cb4005040 con 0x7f7cc01037a0 2024-03-29T08:04:06.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:06.342+0000 7f7cc5200700 1 --2- 172.21.15.37:0/211601240 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7cc01037a0 0x7f7cc0114000 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f7cac002a50 tx=0x7f7cac004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:04:06.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:06.342+0000 7f7cbd600700 1 -- 172.21.15.37:0/211601240 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7cac007cc0 con 0x7f7cc01037a0 2024-03-29T08:04:06.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:06.342+0000 7f7cbd600700 1 -- 172.21.15.37:0/211601240 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f7cac00f040 con 0x7f7cc01037a0 2024-03-29T08:04:06.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:06.342+0000 7f7cc8600700 1 -- 172.21.15.37:0/211601240 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7cc010dd60 con 0x7f7cc01037a0 2024-03-29T08:04:06.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:06.342+0000 7f7cbd600700 1 -- 172.21.15.37:0/211601240 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7cac00d920 con 0x7f7cc01037a0 2024-03-29T08:04:06.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:06.342+0000 7f7cc8600700 1 -- 172.21.15.37:0/211601240 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cc010e320 con 0x7f7cc01037a0 2024-03-29T08:04:06.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:06.343+0000 7f7cc8600700 1 -- 172.21.15.37:0/211601240 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7cc00fe550 con 0x7f7cc01037a0 2024-03-29T08:04:06.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:06.344+0000 7f7cbd600700 1 -- 172.21.15.37:0/211601240 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f7cac009680 con 0x7f7cc01037a0 2024-03-29T08:04:06.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:06.345+0000 7f7cbd600700 1 --2- 172.21.15.37:0/211601240 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f7ca80692b0 0x7f7ca806b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:04:06.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:06.345+0000 7f7cbd600700 1 -- 172.21.15.37:0/211601240 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f7cac086ec0 con 0x7f7cc01037a0 2024-03-29T08:04:06.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:06.346+0000 7f7cc5c00700 1 --2- 172.21.15.37:0/211601240 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f7ca80692b0 0x7f7ca806b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:04:06.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:06.346+0000 7f7cc5c00700 1 --2- 172.21.15.37:0/211601240 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f7ca80692b0 0x7f7ca806b760 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f7cb4009fd0 tx=0x7f7cb4009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:04:06.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:06.350+0000 7f7cbd600700 1 -- 172.21.15.37:0/211601240 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f7cac056b90 con 0x7f7cc01037a0 2024-03-29T08:04:06.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:06.600+0000 7f7cc8600700 1 -- 172.21.15.37:0/211601240 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f7cc0109dc0 con 0x7f7cc01037a0 2024-03-29T08:04:06.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:06.600+0000 7f7cbd600700 1 -- 172.21.15.37:0/211601240 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v16) v1 ==== 53+0+83 (secure 0 0 0) 0x7f7cac056720 con 0x7f7cc01037a0 2024-03-29T08:04:06.600 INFO:teuthology.orchestra.run.smithi037.stdout:name: cephfs, metadata pool: cephfs.cephfs.meta, data pools: [cephfs.cephfs.data ] 2024-03-29T08:04:06.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:06.602+0000 7f7cc8600700 1 -- 172.21.15.37:0/211601240 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f7ca80692b0 msgr2=0x7f7ca806b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:04:06.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:06.602+0000 7f7cc8600700 1 --2- 172.21.15.37:0/211601240 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f7ca80692b0 0x7f7ca806b760 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f7cb4009fd0 tx=0x7f7cb4009e30 comp rx=0 tx=0).stop 2024-03-29T08:04:06.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:06.602+0000 7f7cc8600700 1 -- 172.21.15.37:0/211601240 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7cc01037a0 msgr2=0x7f7cc0114000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:04:06.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:06.602+0000 7f7cc8600700 1 --2- 172.21.15.37:0/211601240 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7cc01037a0 0x7f7cc0114000 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f7cac002a50 tx=0x7f7cac004600 comp rx=0 tx=0).stop 2024-03-29T08:04:06.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:06.602+0000 7f7cc8600700 1 -- 172.21.15.37:0/211601240 shutdown_connections 2024-03-29T08:04:06.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:06.602+0000 7f7cc8600700 1 --2- 172.21.15.37:0/211601240 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f7ca80692b0 0x7f7ca806b760 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:04:06.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:06.603+0000 7f7cc8600700 1 --2- 172.21.15.37:0/211601240 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7cc0102e00 0x7f7cc0113ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:04:06.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:06.603+0000 7f7cc8600700 1 --2- 172.21.15.37:0/211601240 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7cc01037a0 0x7f7cc0114000 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:04:06.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:06.603+0000 7f7cc8600700 1 -- 172.21.15.37:0/211601240 >> 172.21.15.37:0/211601240 conn(0x7f7cc00f89b0 msgr2=0x7f7cc00f9a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:04:06.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:06.603+0000 7f7cc8600700 1 -- 172.21.15.37:0/211601240 shutdown_connections 2024-03-29T08:04:06.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:06.603+0000 7f7cc8600700 1 -- 172.21.15.37:0/211601240 wait complete. 2024-03-29T08:04:06.895 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:04:06 smithi037 ceph-mon[35162]: pgmap v220: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 2 op/s 2024-03-29T08:04:06.895 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:04:06 smithi037 ceph-mon[35162]: from='client.? 172.21.15.37:0/211601240' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-29T08:04:07.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:04:06 smithi148 ceph-mon[44992]: pgmap v220: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 2 op/s 2024-03-29T08:04:07.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:04:06 smithi148 ceph-mon[44992]: from='client.? 172.21.15.37:0/211601240' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-29T08:04:07.475 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-29T08:04:07.475 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-29T08:04:07.475 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi037.front.sepia.ceph.com 2024-03-29T08:04:07.475 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-29T08:04:07.475 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-29T08:04:07.475 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-29T08:04:07.475 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-29T08:04:07.475 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-29T08:04:07.476 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-29T08:04:07.476 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:04:07.476 DEBUG:teuthology.orchestra.run.smithi037:> ip addr 2024-03-29T08:04:07.524 INFO:teuthology.orchestra.run.smithi037.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-29T08:04:07.524 INFO:teuthology.orchestra.run.smithi037.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-29T08:04:07.524 INFO:teuthology.orchestra.run.smithi037.stdout: inet 127.0.0.1/8 scope host lo 2024-03-29T08:04:07.524 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-03-29T08:04:07.525 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 ::1/128 scope host 2024-03-29T08:04:07.525 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-03-29T08:04:07.525 INFO:teuthology.orchestra.run.smithi037.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-29T08:04:07.525 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:6b:fd:18 brd ff:ff:ff:ff:ff:ff 2024-03-29T08:04:07.525 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp7s0f0 2024-03-29T08:04:07.525 INFO:teuthology.orchestra.run.smithi037.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-29T08:04:07.525 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:6b:fd:19 brd ff:ff:ff:ff:ff:ff 2024-03-29T08:04:07.525 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp7s0f1 2024-03-29T08:04:07.525 INFO:teuthology.orchestra.run.smithi037.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-29T08:04:07.525 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:88:77:94 brd ff:ff:ff:ff:ff:ff 2024-03-29T08:04:07.525 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp1s0f0 2024-03-29T08:04:07.525 INFO:teuthology.orchestra.run.smithi037.stdout: inet 172.21.15.37/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-03-29T08:04:07.525 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft 41951sec preferred_lft 41951sec 2024-03-29T08:04:07.525 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 fe80::ec4:7aff:fe88:7794/64 scope link 2024-03-29T08:04:07.526 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-03-29T08:04:07.526 INFO:teuthology.orchestra.run.smithi037.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-29T08:04:07.526 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:88:77:95 brd ff:ff:ff:ff:ff:ff 2024-03-29T08:04:07.526 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp1s0f1 2024-03-29T08:04:07.526 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-03-29T08:04:07.526 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:04:07.526 DEBUG:teuthology.orchestra.run.smithi037:> (cd / && exec bash -c ' 2024-03-29T08:04:07.527 DEBUG:teuthology.orchestra.run.smithi037:> set -e 2024-03-29T08:04:07.527 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip link add name ceph-brx type bridge 2024-03-29T08:04:07.527 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip addr flush dev ceph-brx 2024-03-29T08:04:07.527 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip link set ceph-brx up 2024-03-29T08:04:07.527 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-03-29T08:04:07.527 DEBUG:teuthology.orchestra.run.smithi037:> ') 2024-03-29T08:04:07.946 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:04:07.947 DEBUG:teuthology.orchestra.run.smithi037:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-29T08:04:08.015 INFO:teuthology.orchestra.run.smithi037.stdout:1 2024-03-29T08:04:08.020 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:04:08.020 DEBUG:teuthology.orchestra.run.smithi037:> route 2024-03-29T08:04:08.079 INFO:teuthology.orchestra.run.smithi037.stdout:Kernel IP routing table 2024-03-29T08:04:08.079 INFO:teuthology.orchestra.run.smithi037.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-29T08:04:08.079 INFO:teuthology.orchestra.run.smithi037.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-03-29T08:04:08.079 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-03-29T08:04:08.079 INFO:teuthology.orchestra.run.smithi037.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-29T08:04:08.080 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:04:08.080 DEBUG:teuthology.orchestra.run.smithi037:> (cd / && exec bash -c ' 2024-03-29T08:04:08.080 DEBUG:teuthology.orchestra.run.smithi037:> set -e 2024-03-29T08:04:08.080 DEBUG:teuthology.orchestra.run.smithi037:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-03-29T08:04:08.080 DEBUG:teuthology.orchestra.run.smithi037:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-03-29T08:04:08.080 DEBUG:teuthology.orchestra.run.smithi037:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-03-29T08:04:08.080 DEBUG:teuthology.orchestra.run.smithi037:> ') 2024-03-29T08:04:08.729 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:04:08 smithi037 ceph-mon[35162]: pgmap v221: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 1 op/s 2024-03-29T08:04:08.731 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:04:08.732 DEBUG:teuthology.orchestra.run.smithi037:> ip netns list 2024-03-29T08:04:08.778 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:04:08.778 DEBUG:teuthology.orchestra.run.smithi037:> ip netns list-id 2024-03-29T08:04:08.824 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:04:08.824 DEBUG:teuthology.orchestra.run.smithi037:> (cd / && exec bash -c ' 2024-03-29T08:04:08.824 DEBUG:teuthology.orchestra.run.smithi037:> set -e 2024-03-29T08:04:08.824 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-29T08:04:08.824 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-03-29T08:04:08.824 DEBUG:teuthology.orchestra.run.smithi037:> ') 2024-03-29T08:04:09.004 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-03-29T08:04:09.004 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:04:09.004 DEBUG:teuthology.orchestra.run.smithi037:> (cd / && exec bash -c ' 2024-03-29T08:04:09.005 DEBUG:teuthology.orchestra.run.smithi037:> set -e 2024-03-29T08:04:09.005 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-03-29T08:04:09.005 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-03-29T08:04:09.005 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-03-29T08:04:09.005 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-03-29T08:04:09.005 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-03-29T08:04:09.005 DEBUG:teuthology.orchestra.run.smithi037:> ') 2024-03-29T08:04:09.055 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:04:08 smithi148 ceph-mon[44992]: pgmap v221: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 1 op/s 2024-03-29T08:04:09.276 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:04:09.276 DEBUG:teuthology.orchestra.run.smithi037:> (cd / && exec bash -c ' 2024-03-29T08:04:09.276 DEBUG:teuthology.orchestra.run.smithi037:> set -e 2024-03-29T08:04:09.276 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip link set brx.0 up 2024-03-29T08:04:09.276 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip link set dev brx.0 master ceph-brx 2024-03-29T08:04:09.276 DEBUG:teuthology.orchestra.run.smithi037:> ') 2024-03-29T08:04:09.445 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-03-29T08:04:09.446 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-29T08:04:09.446 DEBUG:teuthology.orchestra.run.smithi037:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-29T08:04:09.491 INFO:teuthology.orchestra.run.smithi037.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-29T08:04:09.491 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-29T08:04:09.491 DEBUG:teuthology.orchestra.run.smithi037:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-29T08:04:09.576 DEBUG:teuthology.orchestra.run.smithi037:> sudo modprobe fuse 2024-03-29T08:04:09.686 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-03-29T08:04:09.686 DEBUG:teuthology.orchestra.run.smithi037:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-03-29T08:04:09.754 INFO:teuthology.orchestra.run.smithi037.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-03-29T08:04:09.757 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-03-29T08:04:09.757 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:04:09.757 DEBUG:teuthology.orchestra.run.smithi037:> ls /sys/fs/fuse/connections 2024-03-29T08:04:09.802 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-03-29T08:04:09.802 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-03-29T08:04:09.847 DEBUG:teuthology.orchestra.run.smithi037:> sudo modprobe fuse 2024-03-29T08:04:09.920 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-03-29T08:04:09.920 DEBUG:teuthology.orchestra.run.smithi037:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-03-29T08:04:09.988 INFO:teuthology.orchestra.run.smithi037.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-03-29T08:04:09.991 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-03-29T08:04:09.991 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:04:09.991 DEBUG:teuthology.orchestra.run.smithi037:> ls /sys/fs/fuse/connections 2024-03-29T08:04:10.027 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi037.stderr:2024-03-29T08:04:10.028+0000 7f234ae2c540 -1 init, newargv = 0x558884e3dfc0 newargc=15 2024-03-29T08:04:10.027 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi037.stderr:ceph-fuse[95187]: starting ceph client 2024-03-29T08:04:10.041 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi037.stderr:ceph-fuse[95187]: starting fuse 2024-03-29T08:04:10.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:04:10 smithi037 ceph-mon[35162]: pgmap v222: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:04:10.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:04:10 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:04:11.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:04:10 smithi148 ceph-mon[44992]: pgmap v222: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:04:11.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:04:10 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:04:11.142 DEBUG:teuthology.orchestra.run.smithi037:> sudo modprobe fuse 2024-03-29T08:04:11.219 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-03-29T08:04:11.220 DEBUG:teuthology.orchestra.run.smithi037:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-03-29T08:04:11.288 INFO:teuthology.orchestra.run.smithi037.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-03-29T08:04:11.290 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-03-29T08:04:11.291 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:04:11.291 DEBUG:teuthology.orchestra.run.smithi037:> ls /sys/fs/fuse/connections 2024-03-29T08:04:11.335 INFO:teuthology.orchestra.run.smithi037.stdout:159 2024-03-29T08:04:11.336 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [159] 2024-03-29T08:04:11.336 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-03-29T08:04:11.336 DEBUG:teuthology.orchestra.run.smithi037:> import glob 2024-03-29T08:04:11.336 DEBUG:teuthology.orchestra.run.smithi037:> import re 2024-03-29T08:04:11.336 DEBUG:teuthology.orchestra.run.smithi037:> import os 2024-03-29T08:04:11.336 DEBUG:teuthology.orchestra.run.smithi037:> import subprocess 2024-03-29T08:04:11.336 DEBUG:teuthology.orchestra.run.smithi037:> 2024-03-29T08:04:11.336 DEBUG:teuthology.orchestra.run.smithi037:> def _find_admin_socket(client_name): 2024-03-29T08:04:11.336 DEBUG:teuthology.orchestra.run.smithi037:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-03-29T08:04:11.336 DEBUG:teuthology.orchestra.run.smithi037:> files = glob.glob(asok_path) 2024-03-29T08:04:11.336 DEBUG:teuthology.orchestra.run.smithi037:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-03-29T08:04:11.337 DEBUG:teuthology.orchestra.run.smithi037:> 2024-03-29T08:04:11.337 DEBUG:teuthology.orchestra.run.smithi037:> # Given a non-glob path, it better be there 2024-03-29T08:04:11.337 DEBUG:teuthology.orchestra.run.smithi037:> if "*" not in asok_path: 2024-03-29T08:04:11.337 DEBUG:teuthology.orchestra.run.smithi037:> assert(len(files) == 1) 2024-03-29T08:04:11.337 DEBUG:teuthology.orchestra.run.smithi037:> return files[0] 2024-03-29T08:04:11.337 DEBUG:teuthology.orchestra.run.smithi037:> 2024-03-29T08:04:11.337 DEBUG:teuthology.orchestra.run.smithi037:> for f in files: 2024-03-29T08:04:11.337 DEBUG:teuthology.orchestra.run.smithi037:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-03-29T08:04:11.337 DEBUG:teuthology.orchestra.run.smithi037:> if os.path.exists("/proc/{0}".format(pid)): 2024-03-29T08:04:11.337 DEBUG:teuthology.orchestra.run.smithi037:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-03-29T08:04:11.337 DEBUG:teuthology.orchestra.run.smithi037:> contents = proc_f.read() 2024-03-29T08:04:11.337 DEBUG:teuthology.orchestra.run.smithi037:> if mountpoint in contents: 2024-03-29T08:04:11.337 DEBUG:teuthology.orchestra.run.smithi037:> return f 2024-03-29T08:04:11.337 DEBUG:teuthology.orchestra.run.smithi037:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-03-29T08:04:11.337 DEBUG:teuthology.orchestra.run.smithi037:> 2024-03-29T08:04:11.337 DEBUG:teuthology.orchestra.run.smithi037:> print(_find_admin_socket("client.0")) 2024-03-29T08:04:11.337 DEBUG:teuthology.orchestra.run.smithi037:> ' 2024-03-29T08:04:11.521 INFO:teuthology.orchestra.run.smithi037.stdout:/var/run/ceph/ceph-client.0.95187.asok 2024-03-29T08:04:11.698 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.95187.asok 2024-03-29T08:04:11.698 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:04:11.698 DEBUG:teuthology.orchestra.run.smithi037:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.95187.asok status 2024-03-29T08:04:11.920 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-29T08:04:11.920 INFO:teuthology.orchestra.run.smithi037.stdout: "metadata": { 2024-03-29T08:04:11.920 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph_sha1": "4213e17c4de67016ed14f55d22503f4923b3a901", 2024-03-29T08:04:11.920 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph_version": "ceph version 17.2.7-789-g4213e17c (4213e17c4de67016ed14f55d22503f4923b3a901) quincy (stable)", 2024-03-29T08:04:11.921 INFO:teuthology.orchestra.run.smithi037.stdout: "entity_id": "0", 2024-03-29T08:04:11.921 INFO:teuthology.orchestra.run.smithi037.stdout: "hostname": "smithi037", 2024-03-29T08:04:11.921 INFO:teuthology.orchestra.run.smithi037.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-03-29T08:04:11.921 INFO:teuthology.orchestra.run.smithi037.stdout: "pid": "95187", 2024-03-29T08:04:11.921 INFO:teuthology.orchestra.run.smithi037.stdout: "root": "/" 2024-03-29T08:04:11.921 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:04:11.921 INFO:teuthology.orchestra.run.smithi037.stdout: "dentry_count": 0, 2024-03-29T08:04:11.921 INFO:teuthology.orchestra.run.smithi037.stdout: "dentry_pinned_count": 0, 2024-03-29T08:04:11.921 INFO:teuthology.orchestra.run.smithi037.stdout: "id": 14536, 2024-03-29T08:04:11.921 INFO:teuthology.orchestra.run.smithi037.stdout: "inst": { 2024-03-29T08:04:11.921 INFO:teuthology.orchestra.run.smithi037.stdout: "name": { 2024-03-29T08:04:11.921 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "client", 2024-03-29T08:04:11.921 INFO:teuthology.orchestra.run.smithi037.stdout: "num": 14536 2024-03-29T08:04:11.921 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:04:11.922 INFO:teuthology.orchestra.run.smithi037.stdout: "addr": { 2024-03-29T08:04:11.922 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "v1", 2024-03-29T08:04:11.922 INFO:teuthology.orchestra.run.smithi037.stdout: "addr": "192.168.0.1:0", 2024-03-29T08:04:11.922 INFO:teuthology.orchestra.run.smithi037.stdout: "nonce": 2061834695 2024-03-29T08:04:11.922 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-29T08:04:11.922 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:04:11.922 INFO:teuthology.orchestra.run.smithi037.stdout: "addr": { 2024-03-29T08:04:11.922 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "v1", 2024-03-29T08:04:11.922 INFO:teuthology.orchestra.run.smithi037.stdout: "addr": "192.168.0.1:0", 2024-03-29T08:04:11.922 INFO:teuthology.orchestra.run.smithi037.stdout: "nonce": 2061834695 2024-03-29T08:04:11.922 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:04:11.922 INFO:teuthology.orchestra.run.smithi037.stdout: "inst_str": "client.14536 192.168.0.1:0/2061834695", 2024-03-29T08:04:11.922 INFO:teuthology.orchestra.run.smithi037.stdout: "addr_str": "192.168.0.1:0/2061834695", 2024-03-29T08:04:11.922 INFO:teuthology.orchestra.run.smithi037.stdout: "inode_count": 1, 2024-03-29T08:04:11.922 INFO:teuthology.orchestra.run.smithi037.stdout: "mds_epoch": 16, 2024-03-29T08:04:11.923 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_epoch": 47, 2024-03-29T08:04:11.923 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_epoch_barrier": 0, 2024-03-29T08:04:11.923 INFO:teuthology.orchestra.run.smithi037.stdout: "blocklisted": false, 2024-03-29T08:04:11.923 INFO:teuthology.orchestra.run.smithi037.stdout: "fs_name": "cephfs" 2024-03-29T08:04:11.923 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-29T08:04:11.934 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 99b492c2-eda1-11ee-b647-cb9ed24678a4 -- ceph fs ls 2024-03-29T08:04:12.371 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/mon.smithi037/config 2024-03-29T08:04:12.917 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:04:12 smithi037 ceph-mon[35162]: pgmap v223: 65 pgs: 65 active+clean; 581 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 255 B/s wr, 0 op/s 2024-03-29T08:04:13.055 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:04:12 smithi148 ceph-mon[44992]: pgmap v223: 65 pgs: 65 active+clean; 581 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 255 B/s wr, 0 op/s 2024-03-29T08:04:14.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:14.454+0000 7fbad1400700 1 -- 172.21.15.37:0/1018284737 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbacc1037a0 msgr2=0x7fbacc0fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:04:14.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:14.454+0000 7fbad1400700 1 --2- 172.21.15.37:0/1018284737 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbacc1037a0 0x7fbacc0fce10 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fbab4004940 tx=0x7fbab4009e30 comp rx=0 tx=0).stop 2024-03-29T08:04:14.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:14.454+0000 7fbad1400700 1 -- 172.21.15.37:0/1018284737 shutdown_connections 2024-03-29T08:04:14.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:14.454+0000 7fbad1400700 1 --2- 172.21.15.37:0/1018284737 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbacc1037a0 0x7fbacc0fce10 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:04:14.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:14.454+0000 7fbad1400700 1 --2- 172.21.15.37:0/1018284737 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbacc102e00 0x7fbacc1031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:04:14.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:14.454+0000 7fbad1400700 1 -- 172.21.15.37:0/1018284737 >> 172.21.15.37:0/1018284737 conn(0x7fbacc0f89b0 msgr2=0x7fbacc0fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:04:14.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:14.455+0000 7fbad1400700 1 -- 172.21.15.37:0/1018284737 shutdown_connections 2024-03-29T08:04:14.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:14.455+0000 7fbad1400700 1 -- 172.21.15.37:0/1018284737 wait complete. 2024-03-29T08:04:14.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:14.455+0000 7fbad1400700 1 Processor -- start 2024-03-29T08:04:14.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:14.456+0000 7fbad1400700 1 -- start start 2024-03-29T08:04:14.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:14.456+0000 7fbad1400700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbacc102e00 0x7fbacc115c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:04:14.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:14.456+0000 7fbad1400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbacc1037a0 0x7fbacc1161c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:04:14.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:14.456+0000 7fbad1400700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbacc116810 con 0x7fbacc1037a0 2024-03-29T08:04:14.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:14.456+0000 7fbad1400700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbacc10fbe0 con 0x7fbacc102e00 2024-03-29T08:04:14.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:14.456+0000 7fbac3e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbacc1037a0 0x7fbacc1161c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:04:14.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:14.456+0000 7fbac3e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbacc1037a0 0x7fbacc1161c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55454/0 (socket says 172.21.15.37:55454) 2024-03-29T08:04:14.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:14.456+0000 7fbac3e00700 1 -- 172.21.15.37:0/1021152387 learned_addr learned my addr 172.21.15.37:0/1021152387 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:04:14.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:14.456+0000 7fbaca800700 1 --2- 172.21.15.37:0/1021152387 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbacc102e00 0x7fbacc115c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:04:14.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:14.457+0000 7fbac3e00700 1 -- 172.21.15.37:0/1021152387 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbacc102e00 msgr2=0x7fbacc115c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:04:14.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:14.457+0000 7fbac3e00700 1 --2- 172.21.15.37:0/1021152387 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbacc102e00 0x7fbacc115c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:04:14.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:14.457+0000 7fbac3e00700 1 -- 172.21.15.37:0/1021152387 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbab4005040 con 0x7fbacc1037a0 2024-03-29T08:04:14.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:14.457+0000 7fbac3e00700 1 --2- 172.21.15.37:0/1021152387 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbacc1037a0 0x7fbacc1161c0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fbab4008d50 tx=0x7fbab4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:04:14.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:14.457+0000 7fbac2a00700 1 -- 172.21.15.37:0/1021152387 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbab400e030 con 0x7fbacc1037a0 2024-03-29T08:04:14.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:14.457+0000 7fbac2a00700 1 -- 172.21.15.37:0/1021152387 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fbab4013de0 con 0x7fbacc1037a0 2024-03-29T08:04:14.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:14.457+0000 7fbac2a00700 1 -- 172.21.15.37:0/1021152387 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbab401cd90 con 0x7fbacc1037a0 2024-03-29T08:04:14.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:14.457+0000 7fbad1400700 1 -- 172.21.15.37:0/1021152387 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbacc10fe60 con 0x7fbacc1037a0 2024-03-29T08:04:14.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:14.457+0000 7fbad1400700 1 -- 172.21.15.37:0/1021152387 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbacc110420 con 0x7fbacc1037a0 2024-03-29T08:04:14.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:14.458+0000 7fbad1400700 1 -- 172.21.15.37:0/1021152387 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbacc0fe550 con 0x7fbacc1037a0 2024-03-29T08:04:14.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:14.459+0000 7fbac2a00700 1 -- 172.21.15.37:0/1021152387 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fbab4013930 con 0x7fbacc1037a0 2024-03-29T08:04:14.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:14.460+0000 7fbac2a00700 1 --2- 172.21.15.37:0/1021152387 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7fbab00692b0 0x7fbab006b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:04:14.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:14.460+0000 7fbac2a00700 1 -- 172.21.15.37:0/1021152387 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5550+0+0 (secure 0 0 0) 0x7fbab4088b90 con 0x7fbacc1037a0 2024-03-29T08:04:14.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:14.461+0000 7fbaca800700 1 --2- 172.21.15.37:0/1021152387 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7fbab00692b0 0x7fbab006b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:04:14.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:14.461+0000 7fbaca800700 1 --2- 172.21.15.37:0/1021152387 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7fbab00692b0 0x7fbab006b760 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fbabc004600 tx=0x7fbabc00b040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:04:14.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:14.465+0000 7fbac2a00700 1 -- 172.21.15.37:0/1021152387 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fbab40588e0 con 0x7fbacc1037a0 2024-03-29T08:04:14.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:14.720+0000 7fbad1400700 1 -- 172.21.15.37:0/1021152387 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fbacc04bed0 con 0x7fbacc1037a0 2024-03-29T08:04:14.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:14.721+0000 7fbac2a00700 1 -- 172.21.15.37:0/1021152387 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v16) v1 ==== 53+0+83 (secure 0 0 0) 0x7fbab4058470 con 0x7fbacc1037a0 2024-03-29T08:04:14.721 INFO:teuthology.orchestra.run.smithi037.stdout:name: cephfs, metadata pool: cephfs.cephfs.meta, data pools: [cephfs.cephfs.data ] 2024-03-29T08:04:14.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:14.723+0000 7fbad1400700 1 -- 172.21.15.37:0/1021152387 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7fbab00692b0 msgr2=0x7fbab006b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:04:14.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:14.723+0000 7fbad1400700 1 --2- 172.21.15.37:0/1021152387 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7fbab00692b0 0x7fbab006b760 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fbabc004600 tx=0x7fbabc00b040 comp rx=0 tx=0).stop 2024-03-29T08:04:14.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:14.723+0000 7fbad1400700 1 -- 172.21.15.37:0/1021152387 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbacc1037a0 msgr2=0x7fbacc1161c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:04:14.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:14.723+0000 7fbad1400700 1 --2- 172.21.15.37:0/1021152387 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbacc1037a0 0x7fbacc1161c0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fbab4008d50 tx=0x7fbab4009e30 comp rx=0 tx=0).stop 2024-03-29T08:04:14.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:14.723+0000 7fbad1400700 1 -- 172.21.15.37:0/1021152387 shutdown_connections 2024-03-29T08:04:14.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:14.723+0000 7fbad1400700 1 --2- 172.21.15.37:0/1021152387 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7fbab00692b0 0x7fbab006b760 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:04:14.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:14.723+0000 7fbad1400700 1 --2- 172.21.15.37:0/1021152387 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbacc102e00 0x7fbacc115c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:04:14.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:14.723+0000 7fbad1400700 1 --2- 172.21.15.37:0/1021152387 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbacc1037a0 0x7fbacc1161c0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:04:14.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:14.723+0000 7fbad1400700 1 -- 172.21.15.37:0/1021152387 >> 172.21.15.37:0/1021152387 conn(0x7fbacc0f89b0 msgr2=0x7fbacc0f99a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:04:14.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:14.723+0000 7fbad1400700 1 -- 172.21.15.37:0/1021152387 shutdown_connections 2024-03-29T08:04:14.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:14.723+0000 7fbad1400700 1 -- 172.21.15.37:0/1021152387 wait complete. 2024-03-29T08:04:14.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:04:14 smithi037 ceph-mon[35162]: pgmap v224: 65 pgs: 65 active+clean; 581 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 341 B/s wr, 0 op/s 2024-03-29T08:04:15.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:04:14 smithi148 ceph-mon[44992]: pgmap v224: 65 pgs: 65 active+clean; 581 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 341 B/s wr, 0 op/s 2024-03-29T08:04:15.411 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-29T08:04:15.411 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-29T08:04:15.411 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi148.front.sepia.ceph.com 2024-03-29T08:04:15.411 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-03-29T08:04:15.411 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-03-29T08:04:15.411 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-29T08:04:15.411 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-29T08:04:15.411 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-29T08:04:15.411 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-03-29T08:04:15.411 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:04:15.412 DEBUG:teuthology.orchestra.run.smithi148:> ip addr 2024-03-29T08:04:15.462 INFO:teuthology.orchestra.run.smithi148.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-29T08:04:15.462 INFO:teuthology.orchestra.run.smithi148.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-29T08:04:15.462 INFO:teuthology.orchestra.run.smithi148.stdout: inet 127.0.0.1/8 scope host lo 2024-03-29T08:04:15.462 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-03-29T08:04:15.462 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 ::1/128 scope host 2024-03-29T08:04:15.463 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-03-29T08:04:15.463 INFO:teuthology.orchestra.run.smithi148.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-29T08:04:15.463 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 0c:c4:7a:da:6a:a4 brd ff:ff:ff:ff:ff:ff 2024-03-29T08:04:15.463 INFO:teuthology.orchestra.run.smithi148.stdout: altname enp7s0f0 2024-03-29T08:04:15.463 INFO:teuthology.orchestra.run.smithi148.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-29T08:04:15.463 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 0c:c4:7a:da:6a:a5 brd ff:ff:ff:ff:ff:ff 2024-03-29T08:04:15.463 INFO:teuthology.orchestra.run.smithi148.stdout: altname enp7s0f1 2024-03-29T08:04:15.463 INFO:teuthology.orchestra.run.smithi148.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-29T08:04:15.463 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 0c:c4:7a:88:72:56 brd ff:ff:ff:ff:ff:ff 2024-03-29T08:04:15.463 INFO:teuthology.orchestra.run.smithi148.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-29T08:04:15.463 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 0c:c4:7a:88:72:57 brd ff:ff:ff:ff:ff:ff 2024-03-29T08:04:15.463 INFO:teuthology.orchestra.run.smithi148.stdout: inet 172.21.15.148/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-03-29T08:04:15.463 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft 41938sec preferred_lft 41938sec 2024-03-29T08:04:15.463 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 fe80::ec4:7aff:fe88:7257/64 scope link 2024-03-29T08:04:15.464 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-03-29T08:04:15.464 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-03-29T08:04:15.465 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:04:15.465 DEBUG:teuthology.orchestra.run.smithi148:> (cd / && exec bash -c ' 2024-03-29T08:04:15.465 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-03-29T08:04:15.465 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip link add name ceph-brx type bridge 2024-03-29T08:04:15.465 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip addr flush dev ceph-brx 2024-03-29T08:04:15.465 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip link set ceph-brx up 2024-03-29T08:04:15.465 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-03-29T08:04:15.465 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-03-29T08:04:15.832 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:04:15 smithi148 ceph-mon[44992]: from='client.? 172.21.15.37:0/1021152387' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-29T08:04:15.868 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:04:15.868 DEBUG:teuthology.orchestra.run.smithi148:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-29T08:04:15.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:04:15 smithi037 ceph-mon[35162]: from='client.? 172.21.15.37:0/1021152387' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-29T08:04:15.936 INFO:teuthology.orchestra.run.smithi148.stdout:1 2024-03-29T08:04:15.939 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:04:15.940 DEBUG:teuthology.orchestra.run.smithi148:> route 2024-03-29T08:04:16.000 INFO:teuthology.orchestra.run.smithi148.stdout:Kernel IP routing table 2024-03-29T08:04:16.001 INFO:teuthology.orchestra.run.smithi148.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-29T08:04:16.001 INFO:teuthology.orchestra.run.smithi148.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-29T08:04:16.001 INFO:teuthology.orchestra.run.smithi148.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-29T08:04:16.001 INFO:teuthology.orchestra.run.smithi148.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-29T08:04:16.001 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:04:16.001 DEBUG:teuthology.orchestra.run.smithi148:> (cd / && exec bash -c ' 2024-03-29T08:04:16.001 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-03-29T08:04:16.001 DEBUG:teuthology.orchestra.run.smithi148:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-29T08:04:16.001 DEBUG:teuthology.orchestra.run.smithi148:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-29T08:04:16.001 DEBUG:teuthology.orchestra.run.smithi148:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-29T08:04:16.002 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-03-29T08:04:16.691 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:04:16.691 DEBUG:teuthology.orchestra.run.smithi148:> ip netns list 2024-03-29T08:04:16.738 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:04:16.738 DEBUG:teuthology.orchestra.run.smithi148:> ip netns list-id 2024-03-29T08:04:16.783 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:04:16.783 DEBUG:teuthology.orchestra.run.smithi148:> (cd / && exec bash -c ' 2024-03-29T08:04:16.784 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-03-29T08:04:16.784 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-29T08:04:16.784 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-03-29T08:04:16.784 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-03-29T08:04:16.933 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:04:16 smithi148 ceph-mon[44992]: pgmap v225: 65 pgs: 65 active+clean; 581 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 341 B/s wr, 0 op/s 2024-03-29T08:04:16.964 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-03-29T08:04:16.964 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:04:16.964 DEBUG:teuthology.orchestra.run.smithi148:> (cd / && exec bash -c ' 2024-03-29T08:04:16.964 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-03-29T08:04:16.964 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-03-29T08:04:16.964 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-03-29T08:04:16.964 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-03-29T08:04:16.964 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-03-29T08:04:16.964 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-03-29T08:04:16.964 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-03-29T08:04:17.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:04:16 smithi037 ceph-mon[35162]: pgmap v225: 65 pgs: 65 active+clean; 581 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 341 B/s wr, 0 op/s 2024-03-29T08:04:17.224 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:04:17.224 DEBUG:teuthology.orchestra.run.smithi148:> (cd / && exec bash -c ' 2024-03-29T08:04:17.224 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-03-29T08:04:17.225 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip link set brx.0 up 2024-03-29T08:04:17.225 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip link set dev brx.0 master ceph-brx 2024-03-29T08:04:17.225 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-03-29T08:04:17.395 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-03-29T08:04:17.395 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-29T08:04:17.395 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-03-29T08:04:17.440 INFO:teuthology.orchestra.run.smithi148.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-03-29T08:04:17.440 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-29T08:04:17.440 DEBUG:teuthology.orchestra.run.smithi148:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-03-29T08:04:17.525 DEBUG:teuthology.orchestra.run.smithi148:> sudo modprobe fuse 2024-03-29T08:04:17.632 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-03-29T08:04:17.633 DEBUG:teuthology.orchestra.run.smithi148:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-03-29T08:04:17.700 INFO:teuthology.orchestra.run.smithi148.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-03-29T08:04:17.703 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-03-29T08:04:17.703 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:04:17.703 DEBUG:teuthology.orchestra.run.smithi148:> ls /sys/fs/fuse/connections 2024-03-29T08:04:17.749 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-03-29T08:04:17.749 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-03-29T08:04:17.794 DEBUG:teuthology.orchestra.run.smithi148:> sudo modprobe fuse 2024-03-29T08:04:17.863 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-03-29T08:04:17.863 DEBUG:teuthology.orchestra.run.smithi148:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-03-29T08:04:17.928 INFO:teuthology.orchestra.run.smithi148.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-03-29T08:04:17.931 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-03-29T08:04:17.931 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:04:17.931 DEBUG:teuthology.orchestra.run.smithi148:> ls /sys/fs/fuse/connections 2024-03-29T08:04:18.009 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi148.stderr:2024-03-29T08:04:18.009+0000 7fb7c4ba9540 -1 init, newargv = 0x564499b9d010 newargc=15 2024-03-29T08:04:18.009 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi148.stderr:ceph-fuse[75700]: starting ceph client 2024-03-29T08:04:18.021 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi148.stderr:ceph-fuse[75700]: starting fuse 2024-03-29T08:04:19.055 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:04:18 smithi148 ceph-mon[44992]: pgmap v226: 65 pgs: 65 active+clean; 581 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 341 B/s wr, 0 op/s 2024-03-29T08:04:19.077 DEBUG:teuthology.orchestra.run.smithi148:> sudo modprobe fuse 2024-03-29T08:04:19.153 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-03-29T08:04:19.154 DEBUG:teuthology.orchestra.run.smithi148:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-03-29T08:04:19.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:04:18 smithi037 ceph-mon[35162]: pgmap v226: 65 pgs: 65 active+clean; 581 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 341 B/s wr, 0 op/s 2024-03-29T08:04:19.221 INFO:teuthology.orchestra.run.smithi148.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-03-29T08:04:19.223 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-03-29T08:04:19.224 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:04:19.224 DEBUG:teuthology.orchestra.run.smithi148:> ls /sys/fs/fuse/connections 2024-03-29T08:04:19.268 INFO:teuthology.orchestra.run.smithi148.stdout:123 2024-03-29T08:04:19.269 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [123] 2024-03-29T08:04:19.269 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-03-29T08:04:19.269 DEBUG:teuthology.orchestra.run.smithi148:> import glob 2024-03-29T08:04:19.269 DEBUG:teuthology.orchestra.run.smithi148:> import re 2024-03-29T08:04:19.269 DEBUG:teuthology.orchestra.run.smithi148:> import os 2024-03-29T08:04:19.269 DEBUG:teuthology.orchestra.run.smithi148:> import subprocess 2024-03-29T08:04:19.269 DEBUG:teuthology.orchestra.run.smithi148:> 2024-03-29T08:04:19.269 DEBUG:teuthology.orchestra.run.smithi148:> def _find_admin_socket(client_name): 2024-03-29T08:04:19.269 DEBUG:teuthology.orchestra.run.smithi148:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-03-29T08:04:19.269 DEBUG:teuthology.orchestra.run.smithi148:> files = glob.glob(asok_path) 2024-03-29T08:04:19.269 DEBUG:teuthology.orchestra.run.smithi148:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-03-29T08:04:19.269 DEBUG:teuthology.orchestra.run.smithi148:> 2024-03-29T08:04:19.269 DEBUG:teuthology.orchestra.run.smithi148:> # Given a non-glob path, it better be there 2024-03-29T08:04:19.270 DEBUG:teuthology.orchestra.run.smithi148:> if "*" not in asok_path: 2024-03-29T08:04:19.270 DEBUG:teuthology.orchestra.run.smithi148:> assert(len(files) == 1) 2024-03-29T08:04:19.270 DEBUG:teuthology.orchestra.run.smithi148:> return files[0] 2024-03-29T08:04:19.270 DEBUG:teuthology.orchestra.run.smithi148:> 2024-03-29T08:04:19.270 DEBUG:teuthology.orchestra.run.smithi148:> for f in files: 2024-03-29T08:04:19.270 DEBUG:teuthology.orchestra.run.smithi148:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-03-29T08:04:19.270 DEBUG:teuthology.orchestra.run.smithi148:> if os.path.exists("/proc/{0}".format(pid)): 2024-03-29T08:04:19.270 DEBUG:teuthology.orchestra.run.smithi148:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-03-29T08:04:19.270 DEBUG:teuthology.orchestra.run.smithi148:> contents = proc_f.read() 2024-03-29T08:04:19.270 DEBUG:teuthology.orchestra.run.smithi148:> if mountpoint in contents: 2024-03-29T08:04:19.270 DEBUG:teuthology.orchestra.run.smithi148:> return f 2024-03-29T08:04:19.270 DEBUG:teuthology.orchestra.run.smithi148:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-03-29T08:04:19.270 DEBUG:teuthology.orchestra.run.smithi148:> 2024-03-29T08:04:19.270 DEBUG:teuthology.orchestra.run.smithi148:> print(_find_admin_socket("client.1")) 2024-03-29T08:04:19.270 DEBUG:teuthology.orchestra.run.smithi148:> ' 2024-03-29T08:04:19.449 INFO:teuthology.orchestra.run.smithi148.stdout:/var/run/ceph/ceph-client.1.75700.asok 2024-03-29T08:04:19.628 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.75700.asok 2024-03-29T08:04:19.628 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:04:19.628 DEBUG:teuthology.orchestra.run.smithi148:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.75700.asok status 2024-03-29T08:04:19.843 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-03-29T08:04:19.843 INFO:teuthology.orchestra.run.smithi148.stdout: "metadata": { 2024-03-29T08:04:19.843 INFO:teuthology.orchestra.run.smithi148.stdout: "ceph_sha1": "4213e17c4de67016ed14f55d22503f4923b3a901", 2024-03-29T08:04:19.843 INFO:teuthology.orchestra.run.smithi148.stdout: "ceph_version": "ceph version 17.2.7-789-g4213e17c (4213e17c4de67016ed14f55d22503f4923b3a901) quincy (stable)", 2024-03-29T08:04:19.843 INFO:teuthology.orchestra.run.smithi148.stdout: "entity_id": "1", 2024-03-29T08:04:19.843 INFO:teuthology.orchestra.run.smithi148.stdout: "hostname": "smithi148", 2024-03-29T08:04:19.843 INFO:teuthology.orchestra.run.smithi148.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-03-29T08:04:19.843 INFO:teuthology.orchestra.run.smithi148.stdout: "pid": "75700", 2024-03-29T08:04:19.843 INFO:teuthology.orchestra.run.smithi148.stdout: "root": "/" 2024-03-29T08:04:19.843 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-03-29T08:04:19.843 INFO:teuthology.orchestra.run.smithi148.stdout: "dentry_count": 0, 2024-03-29T08:04:19.843 INFO:teuthology.orchestra.run.smithi148.stdout: "dentry_pinned_count": 0, 2024-03-29T08:04:19.843 INFO:teuthology.orchestra.run.smithi148.stdout: "id": 24373, 2024-03-29T08:04:19.844 INFO:teuthology.orchestra.run.smithi148.stdout: "inst": { 2024-03-29T08:04:19.844 INFO:teuthology.orchestra.run.smithi148.stdout: "name": { 2024-03-29T08:04:19.844 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "client", 2024-03-29T08:04:19.844 INFO:teuthology.orchestra.run.smithi148.stdout: "num": 24373 2024-03-29T08:04:19.844 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-03-29T08:04:19.844 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": { 2024-03-29T08:04:19.844 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2024-03-29T08:04:19.844 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "192.168.0.1:0", 2024-03-29T08:04:19.844 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 985505949 2024-03-29T08:04:19.844 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-03-29T08:04:19.844 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-03-29T08:04:19.844 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": { 2024-03-29T08:04:19.844 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2024-03-29T08:04:19.844 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "192.168.0.1:0", 2024-03-29T08:04:19.844 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 985505949 2024-03-29T08:04:19.845 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-03-29T08:04:19.845 INFO:teuthology.orchestra.run.smithi148.stdout: "inst_str": "client.24373 192.168.0.1:0/985505949", 2024-03-29T08:04:19.845 INFO:teuthology.orchestra.run.smithi148.stdout: "addr_str": "192.168.0.1:0/985505949", 2024-03-29T08:04:19.845 INFO:teuthology.orchestra.run.smithi148.stdout: "inode_count": 1, 2024-03-29T08:04:19.845 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_epoch": 16, 2024-03-29T08:04:19.845 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_epoch": 47, 2024-03-29T08:04:19.845 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_epoch_barrier": 0, 2024-03-29T08:04:19.845 INFO:teuthology.orchestra.run.smithi148.stdout: "blocklisted": false, 2024-03-29T08:04:19.845 INFO:teuthology.orchestra.run.smithi148.stdout: "fs_name": "cephfs" 2024-03-29T08:04:19.845 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-03-29T08:04:19.857 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:04:19.857 DEBUG:teuthology.orchestra.run.smithi037:> stat --file-system '--printf=%T 2024-03-29T08:04:19.857 DEBUG:teuthology.orchestra.run.smithi037:> ' -- /home/ubuntu/cephtest/mnt.0 2024-03-29T08:04:19.911 INFO:teuthology.orchestra.run.smithi037.stdout:fuseblk 2024-03-29T08:04:19.911 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-03-29T08:04:19.911 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:04:19.911 DEBUG:teuthology.orchestra.run.smithi037:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-29T08:04:19.988 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:04:19.988 DEBUG:teuthology.orchestra.run.smithi148:> stat --file-system '--printf=%T 2024-03-29T08:04:19.988 DEBUG:teuthology.orchestra.run.smithi148:> ' -- /home/ubuntu/cephtest/mnt.1 2024-03-29T08:04:20.034 INFO:teuthology.orchestra.run.smithi148.stdout:fuseblk 2024-03-29T08:04:20.034 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-03-29T08:04:20.034 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:04:20.034 DEBUG:teuthology.orchestra.run.smithi148:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-03-29T08:04:21.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:04:20 smithi148 ceph-mon[44992]: pgmap v227: 65 pgs: 65 active+clean; 581 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 341 B/s wr, 0 op/s 2024-03-29T08:04:21.176 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:04:20 smithi037 ceph-mon[35162]: pgmap v227: 65 pgs: 65 active+clean; 581 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 341 B/s wr, 0 op/s 2024-03-29T08:04:23.055 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:04:22 smithi148 ceph-mon[44992]: pgmap v228: 65 pgs: 65 active+clean; 581 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 426 B/s wr, 0 op/s 2024-03-29T08:04:23.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:04:22 smithi037 ceph-mon[35162]: pgmap v228: 65 pgs: 65 active+clean; 581 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 426 B/s wr, 0 op/s 2024-03-29T08:04:23.650 INFO:teuthology.run_tasks:Running task print... 2024-03-29T08:04:23.665 INFO:teuthology.task.print:**** done client 2024-03-29T08:04:23.665 INFO:teuthology.run_tasks:Running task parallel... 2024-03-29T08:04:23.690 INFO:teuthology.task.parallel:starting parallel... 2024-03-29T08:04:23.690 INFO:teuthology.task.parallel:In parallel, running task sequential... 2024-03-29T08:04:23.691 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-03-29T08:04:23.691 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi037.front.sepia.ceph.com 2024-03-29T08:04:23.691 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 99b492c2-eda1-11ee-b647-cb9ed24678a4 -e sha1=d0e392d9aee635f2b65768c460c7a151ead7a5e2 -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force' 2024-03-29T08:04:23.692 INFO:teuthology.task.parallel:In parallel, running task sequential... 2024-03-29T08:04:23.692 INFO:teuthology.task.sequential:In sequential, running task workunit... 2024-03-29T08:04:23.695 INFO:tasks.workunit:Pulling workunits from ref d0e392d9aee635f2b65768c460c7a151ead7a5e2 2024-03-29T08:04:23.696 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-03-29T08:04:23.696 INFO:tasks.workunit:timeout=3h 2024-03-29T08:04:23.696 INFO:tasks.workunit:cleanup=True 2024-03-29T08:04:23.696 DEBUG:teuthology.orchestra.run.smithi037:> stat -- /home/ubuntu/cephtest/mnt.0 2024-03-29T08:04:23.755 INFO:teuthology.orchestra.run.smithi037.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-03-29T08:04:23.755 INFO:teuthology.orchestra.run.smithi037.stdout: Size: 0 Blocks: 1 IO Block: 4194304 directory 2024-03-29T08:04:23.755 INFO:teuthology.orchestra.run.smithi037.stdout:Device: 9fh/159d Inode: 1 Links: 2 2024-03-29T08:04:23.755 INFO:teuthology.orchestra.run.smithi037.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-29T08:04:23.755 INFO:teuthology.orchestra.run.smithi037.stdout:Context: system_u:object_r:fusefs_t:s0 2024-03-29T08:04:23.755 INFO:teuthology.orchestra.run.smithi037.stdout:Access: 1970-01-01 00:00:00.000000000 +0000 2024-03-29T08:04:23.755 INFO:teuthology.orchestra.run.smithi037.stdout:Modify: 2024-03-29 08:03:34.529473782 +0000 2024-03-29T08:04:23.755 INFO:teuthology.orchestra.run.smithi037.stdout:Change: 2024-03-29 08:04:19.984721962 +0000 2024-03-29T08:04:23.755 INFO:teuthology.orchestra.run.smithi037.stdout: Birth: - 2024-03-29T08:04:23.755 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-03-29T08:04:23.756 DEBUG:teuthology.orchestra.run.smithi037:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-03-29T08:04:23.876 DEBUG:teuthology.orchestra.run.smithi148:> stat -- /home/ubuntu/cephtest/mnt.1 2024-03-29T08:04:23.930 INFO:teuthology.orchestra.run.smithi148.stdout: File: /home/ubuntu/cephtest/mnt.1 2024-03-29T08:04:23.930 INFO:teuthology.orchestra.run.smithi148.stdout: Size: 0 Blocks: 1 IO Block: 4194304 directory 2024-03-29T08:04:23.930 INFO:teuthology.orchestra.run.smithi148.stdout:Device: 7bh/123d Inode: 1 Links: 3 2024-03-29T08:04:23.930 INFO:teuthology.orchestra.run.smithi148.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-29T08:04:23.930 INFO:teuthology.orchestra.run.smithi148.stdout:Context: system_u:object_r:fusefs_t:s0 2024-03-29T08:04:23.930 INFO:teuthology.orchestra.run.smithi148.stdout:Access: 1970-01-01 00:00:00.000000000 +0000 2024-03-29T08:04:23.930 INFO:teuthology.orchestra.run.smithi148.stdout:Modify: 2024-03-29 08:04:23.868297902 +0000 2024-03-29T08:04:23.930 INFO:teuthology.orchestra.run.smithi148.stdout:Change: 2024-03-29 08:04:23.868297902 +0000 2024-03-29T08:04:23.930 INFO:teuthology.orchestra.run.smithi148.stdout: Birth: - 2024-03-29T08:04:23.931 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.1 2024-03-29T08:04:23.931 DEBUG:teuthology.orchestra.run.smithi148:> cd -- /home/ubuntu/cephtest/mnt.1 && sudo install -d -m 0755 --owner=ubuntu -- client.1 2024-03-29T08:04:24.011 DEBUG:teuthology.orchestra.run.smithi037:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout d0e392d9aee635f2b65768c460c7a151ead7a5e2 2024-03-29T08:04:24.011 DEBUG:teuthology.orchestra.run.smithi148:> rm -rf /home/ubuntu/cephtest/clone.client.1 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.1 && cd /home/ubuntu/cephtest/clone.client.1 && git checkout d0e392d9aee635f2b65768c460c7a151ead7a5e2 2024-03-29T08:04:24.059 INFO:tasks.workunit.client.1.smithi148.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.1'... 2024-03-29T08:04:24.061 INFO:tasks.workunit.client.0.smithi037.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-03-29T08:04:24.155 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/mon.smithi037/config 2024-03-29T08:04:24.705 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:04:24 smithi037 ceph-mon[35162]: pgmap v229: 65 pgs: 65 active+clean; 582 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 341 B/s wr, 0 op/s 2024-03-29T08:04:25.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:04:24 smithi148 ceph-mon[44992]: pgmap v229: 65 pgs: 65 active+clean; 582 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 341 B/s wr, 0 op/s 2024-03-29T08:04:25.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:04:25 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:04:26.055 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:04:25 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:04:26.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:26.301+0000 7fd613400700 1 -- 172.21.15.37:0/2003524102 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd60c066240 msgr2=0x7fd60c0ffa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:04:26.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:26.301+0000 7fd613400700 1 --2- 172.21.15.37:0/2003524102 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd60c066240 0x7fd60c0ffa60 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fd600004940 tx=0x7fd600009e30 comp rx=0 tx=0).stop 2024-03-29T08:04:26.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:26.302+0000 7fd613400700 1 -- 172.21.15.37:0/2003524102 shutdown_connections 2024-03-29T08:04:26.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:26.302+0000 7fd613400700 1 --2- 172.21.15.37:0/2003524102 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd60c0fffa0 0x7fd60c102380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:04:26.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:26.302+0000 7fd613400700 1 --2- 172.21.15.37:0/2003524102 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd60c066240 0x7fd60c0ffa60 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:04:26.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:26.302+0000 7fd613400700 1 -- 172.21.15.37:0/2003524102 >> 172.21.15.37:0/2003524102 conn(0x7fd60c0f7430 msgr2=0x7fd60c0f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:04:26.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:26.302+0000 7fd613400700 1 -- 172.21.15.37:0/2003524102 shutdown_connections 2024-03-29T08:04:26.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:26.302+0000 7fd613400700 1 -- 172.21.15.37:0/2003524102 wait complete. 2024-03-29T08:04:26.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:26.302+0000 7fd613400700 1 Processor -- start 2024-03-29T08:04:26.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:26.303+0000 7fd613400700 1 -- start start 2024-03-29T08:04:26.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:26.303+0000 7fd613400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd60c066240 0x7fd60c10da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:04:26.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:26.303+0000 7fd613400700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd60c0fffa0 0x7fd60c10df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:04:26.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:26.303+0000 7fd613400700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd60c10e540 con 0x7fd60c066240 2024-03-29T08:04:26.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:26.303+0000 7fd613400700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd60c10ed40 con 0x7fd60c0fffa0 2024-03-29T08:04:26.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:26.303+0000 7fd610a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd60c066240 0x7fd60c10da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:04:26.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:26.303+0000 7fd610a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd60c066240 0x7fd60c10da00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57230/0 (socket says 172.21.15.37:57230) 2024-03-29T08:04:26.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:26.303+0000 7fd610a00700 1 -- 172.21.15.37:0/131364581 learned_addr learned my addr 172.21.15.37:0/131364581 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:04:26.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:26.303+0000 7fd60be00700 1 --2- 172.21.15.37:0/131364581 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd60c0fffa0 0x7fd60c10df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:04:26.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:26.304+0000 7fd610a00700 1 -- 172.21.15.37:0/131364581 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd60c0fffa0 msgr2=0x7fd60c10df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:04:26.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:26.304+0000 7fd610a00700 1 --2- 172.21.15.37:0/131364581 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd60c0fffa0 0x7fd60c10df40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:04:26.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:26.304+0000 7fd610a00700 1 -- 172.21.15.37:0/131364581 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd600005040 con 0x7fd60c066240 2024-03-29T08:04:26.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:26.304+0000 7fd610a00700 1 --2- 172.21.15.37:0/131364581 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd60c066240 0x7fd60c10da00 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fd60000b3b0 tx=0x7fd600009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:04:26.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:26.304+0000 7fd608c00700 1 -- 172.21.15.37:0/131364581 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd60000e040 con 0x7fd60c066240 2024-03-29T08:04:26.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:26.304+0000 7fd608c00700 1 -- 172.21.15.37:0/131364581 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd60000bc50 con 0x7fd60c066240 2024-03-29T08:04:26.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:26.304+0000 7fd608c00700 1 -- 172.21.15.37:0/131364581 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd60001c630 con 0x7fd60c066240 2024-03-29T08:04:26.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:26.304+0000 7fd613400700 1 -- 172.21.15.37:0/131364581 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd60c10efc0 con 0x7fd60c066240 2024-03-29T08:04:26.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:26.304+0000 7fd613400700 1 -- 172.21.15.37:0/131364581 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd60c10f500 con 0x7fd60c066240 2024-03-29T08:04:26.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:26.305+0000 7fd613400700 1 -- 172.21.15.37:0/131364581 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd60c05f5f0 con 0x7fd60c066240 2024-03-29T08:04:26.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:26.306+0000 7fd608c00700 1 -- 172.21.15.37:0/131364581 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fd600013420 con 0x7fd60c066240 2024-03-29T08:04:26.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:26.307+0000 7fd608c00700 1 --2- 172.21.15.37:0/131364581 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7fd5f40692b0 0x7fd5f406b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:04:26.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:26.307+0000 7fd608c00700 1 -- 172.21.15.37:0/131364581 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5550+0+0 (secure 0 0 0) 0x7fd600088680 con 0x7fd60c066240 2024-03-29T08:04:26.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:26.307+0000 7fd60be00700 1 --2- 172.21.15.37:0/131364581 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7fd5f40692b0 0x7fd5f406b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:04:26.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:26.308+0000 7fd60be00700 1 --2- 172.21.15.37:0/131364581 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7fd5f40692b0 0x7fd5f406b760 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fd5f8007fd0 tx=0x7fd5f8009040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:04:26.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:26.312+0000 7fd608c00700 1 -- 172.21.15.37:0/131364581 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fd6000583d0 con 0x7fd60c066240 2024-03-29T08:04:26.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:26.518+0000 7fd613400700 1 -- 172.21.15.37:0/131364581 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim}] v 0) v1 -- 0x7fd60c04bed0 con 0x7fd60c066240 2024-03-29T08:04:26.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:26.519+0000 7fd608c00700 1 -- 172.21.15.37:0/131364581 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim}]=0 v21)=0 v21) v1 ==== 155+0+0 (secure 0 0 0) 0x7fd600057f60 con 0x7fd60c066240 2024-03-29T08:04:26.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:26.521+0000 7fd613400700 1 -- 172.21.15.37:0/131364581 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7fd5f40692b0 msgr2=0x7fd5f406b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:04:26.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:26.521+0000 7fd613400700 1 --2- 172.21.15.37:0/131364581 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7fd5f40692b0 0x7fd5f406b760 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fd5f8007fd0 tx=0x7fd5f8009040 comp rx=0 tx=0).stop 2024-03-29T08:04:26.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:26.521+0000 7fd613400700 1 -- 172.21.15.37:0/131364581 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd60c066240 msgr2=0x7fd60c10da00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:04:26.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:26.521+0000 7fd613400700 1 --2- 172.21.15.37:0/131364581 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd60c066240 0x7fd60c10da00 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fd60000b3b0 tx=0x7fd600009e30 comp rx=0 tx=0).stop 2024-03-29T08:04:26.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:26.521+0000 7fd613400700 1 -- 172.21.15.37:0/131364581 shutdown_connections 2024-03-29T08:04:26.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:26.521+0000 7fd613400700 1 --2- 172.21.15.37:0/131364581 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7fd5f40692b0 0x7fd5f406b760 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:04:26.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:26.521+0000 7fd613400700 1 --2- 172.21.15.37:0/131364581 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd60c066240 0x7fd60c10da00 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:04:26.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:26.521+0000 7fd613400700 1 --2- 172.21.15.37:0/131364581 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd60c0fffa0 0x7fd60c10df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:04:26.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:26.521+0000 7fd613400700 1 -- 172.21.15.37:0/131364581 >> 172.21.15.37:0/131364581 conn(0x7fd60c0f7430 msgr2=0x7fd60c0f7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:04:26.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:26.521+0000 7fd613400700 1 -- 172.21.15.37:0/131364581 shutdown_connections 2024-03-29T08:04:26.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:26.521+0000 7fd613400700 1 -- 172.21.15.37:0/131364581 wait complete. 2024-03-29T08:04:26.891 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:04:26 smithi037 ceph-mon[35162]: pgmap v230: 65 pgs: 65 active+clean; 582 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 255 B/s wr, 0 op/s 2024-03-29T08:04:27.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:04:26 smithi148 ceph-mon[44992]: pgmap v230: 65 pgs: 65 active+clean; 582 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 255 B/s wr, 0 op/s 2024-03-29T08:04:27.393 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 99b492c2-eda1-11ee-b647-cb9ed24678a4 -e sha1=d0e392d9aee635f2b65768c460c7a151ead7a5e2 -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force' 2024-03-29T08:04:27.833 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/mon.smithi037/config 2024-03-29T08:04:28.968 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:04:28 smithi037 ceph-mon[35162]: pgmap v231: 65 pgs: 65 active+clean; 587 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 767 B/s wr, 0 op/s 2024-03-29T08:04:29.055 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:04:28 smithi148 ceph-mon[44992]: pgmap v231: 65 pgs: 65 active+clean; 587 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 767 B/s wr, 0 op/s 2024-03-29T08:04:29.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:29.613+0000 7f2e2a200700 1 -- 172.21.15.37:0/225923044 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2e24071e20 msgr2=0x7f2e24070330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:04:29.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:29.613+0000 7f2e2a200700 1 --2- 172.21.15.37:0/225923044 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2e24071e20 0x7f2e24070330 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f2e18004940 tx=0x7f2e18009e30 comp rx=0 tx=0).stop 2024-03-29T08:04:29.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:29.614+0000 7f2e2a200700 1 -- 172.21.15.37:0/225923044 shutdown_connections 2024-03-29T08:04:29.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:29.614+0000 7f2e2a200700 1 --2- 172.21.15.37:0/225923044 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2e24070870 0x7f2e24070ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:04:29.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:29.614+0000 7f2e2a200700 1 --2- 172.21.15.37:0/225923044 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2e24071e20 0x7f2e24070330 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:04:29.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:29.614+0000 7f2e2a200700 1 -- 172.21.15.37:0/225923044 >> 172.21.15.37:0/225923044 conn(0x7f2e240f83a0 msgr2=0x7f2e240fa7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:04:29.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:29.614+0000 7f2e2a200700 1 -- 172.21.15.37:0/225923044 shutdown_connections 2024-03-29T08:04:29.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:29.614+0000 7f2e2a200700 1 -- 172.21.15.37:0/225923044 wait complete. 2024-03-29T08:04:29.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:29.615+0000 7f2e2a200700 1 Processor -- start 2024-03-29T08:04:29.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:29.615+0000 7f2e2a200700 1 -- start start 2024-03-29T08:04:29.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:29.615+0000 7f2e2a200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2e24070870 0x7f2e2410f990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:04:29.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:29.615+0000 7f2e2a200700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2e24071e20 0x7f2e2410fed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:04:29.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:29.615+0000 7f2e2a200700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e241104d0 con 0x7f2e24070870 2024-03-29T08:04:29.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:29.615+0000 7f2e2a200700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e24110d20 con 0x7f2e24071e20 2024-03-29T08:04:29.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:29.615+0000 7f2e28e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2e24070870 0x7f2e2410f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:04:29.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:29.615+0000 7f2e28e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2e24070870 0x7f2e2410f990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57236/0 (socket says 172.21.15.37:57236) 2024-03-29T08:04:29.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:29.615+0000 7f2e28e00700 1 -- 172.21.15.37:0/3572908956 learned_addr learned my addr 172.21.15.37:0/3572908956 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:04:29.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:29.615+0000 7f2e23e00700 1 --2- 172.21.15.37:0/3572908956 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2e24071e20 0x7f2e2410fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:04:29.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:29.616+0000 7f2e28e00700 1 -- 172.21.15.37:0/3572908956 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2e24071e20 msgr2=0x7f2e2410fed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:04:29.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:29.616+0000 7f2e28e00700 1 --2- 172.21.15.37:0/3572908956 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2e24071e20 0x7f2e2410fed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:04:29.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:29.616+0000 7f2e28e00700 1 -- 172.21.15.37:0/3572908956 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e18005040 con 0x7f2e24070870 2024-03-29T08:04:29.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:29.616+0000 7f2e28e00700 1 --2- 172.21.15.37:0/3572908956 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2e24070870 0x7f2e2410f990 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f2e1800b900 tx=0x7f2e18009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:04:29.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:29.616+0000 7f2e20c00700 1 -- 172.21.15.37:0/3572908956 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2e1800e040 con 0x7f2e24070870 2024-03-29T08:04:29.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:29.616+0000 7f2e20c00700 1 -- 172.21.15.37:0/3572908956 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f2e18004290 con 0x7f2e24070870 2024-03-29T08:04:29.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:29.616+0000 7f2e20c00700 1 -- 172.21.15.37:0/3572908956 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2e1801c3f0 con 0x7f2e24070870 2024-03-29T08:04:29.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:29.616+0000 7f2e2a200700 1 -- 172.21.15.37:0/3572908956 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e24110fa0 con 0x7f2e24070870 2024-03-29T08:04:29.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:29.617+0000 7f2e2a200700 1 -- 172.21.15.37:0/3572908956 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e241114e0 con 0x7f2e24070870 2024-03-29T08:04:29.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:29.618+0000 7f2e2a200700 1 -- 172.21.15.37:0/3572908956 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e2405f5f0 con 0x7f2e24070870 2024-03-29T08:04:29.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:29.619+0000 7f2e20c00700 1 -- 172.21.15.37:0/3572908956 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f2e18013420 con 0x7f2e24070870 2024-03-29T08:04:29.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:29.619+0000 7f2e20c00700 1 --2- 172.21.15.37:0/3572908956 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f2e0c0692b0 0x7f2e0c06b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:04:29.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:29.619+0000 7f2e20c00700 1 -- 172.21.15.37:0/3572908956 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f2e180884f0 con 0x7f2e24070870 2024-03-29T08:04:29.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:29.620+0000 7f2e23e00700 1 --2- 172.21.15.37:0/3572908956 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f2e0c0692b0 0x7f2e0c06b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:04:29.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:29.620+0000 7f2e23e00700 1 --2- 172.21.15.37:0/3572908956 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f2e0c0692b0 0x7f2e0c06b760 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f2e10004f40 tx=0x7f2e1000b040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:04:29.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:29.624+0000 7f2e20c00700 1 -- 172.21.15.37:0/3572908956 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f2e180582f0 con 0x7f2e24070870 2024-03-29T08:04:29.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:29.828+0000 7f2e2a200700 1 -- 172.21.15.37:0/3572908956 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim_allowed}] v 0) v1 -- 0x7f2e241118b0 con 0x7f2e24070870 2024-03-29T08:04:29.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:29.828+0000 7f2e20c00700 1 -- 172.21.15.37:0/3572908956 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim_allowed}]=0 v21)=0 v21) v1 ==== 163+0+0 (secure 0 0 0) 0x7f2e18057e80 con 0x7f2e24070870 2024-03-29T08:04:29.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:29.831+0000 7f2e2a200700 1 -- 172.21.15.37:0/3572908956 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f2e0c0692b0 msgr2=0x7f2e0c06b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:04:29.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:29.831+0000 7f2e2a200700 1 --2- 172.21.15.37:0/3572908956 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f2e0c0692b0 0x7f2e0c06b760 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f2e10004f40 tx=0x7f2e1000b040 comp rx=0 tx=0).stop 2024-03-29T08:04:29.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:29.831+0000 7f2e2a200700 1 -- 172.21.15.37:0/3572908956 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2e24070870 msgr2=0x7f2e2410f990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:04:29.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:29.831+0000 7f2e2a200700 1 --2- 172.21.15.37:0/3572908956 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2e24070870 0x7f2e2410f990 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f2e1800b900 tx=0x7f2e18009e30 comp rx=0 tx=0).stop 2024-03-29T08:04:29.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:29.831+0000 7f2e2a200700 1 -- 172.21.15.37:0/3572908956 shutdown_connections 2024-03-29T08:04:29.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:29.831+0000 7f2e2a200700 1 --2- 172.21.15.37:0/3572908956 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f2e0c0692b0 0x7f2e0c06b760 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:04:29.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:29.831+0000 7f2e2a200700 1 --2- 172.21.15.37:0/3572908956 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2e24070870 0x7f2e2410f990 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:04:29.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:29.831+0000 7f2e2a200700 1 --2- 172.21.15.37:0/3572908956 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2e24071e20 0x7f2e2410fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:04:29.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:29.831+0000 7f2e2a200700 1 -- 172.21.15.37:0/3572908956 >> 172.21.15.37:0/3572908956 conn(0x7f2e240f83a0 msgr2=0x7f2e240f9b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:04:29.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:29.831+0000 7f2e2a200700 1 -- 172.21.15.37:0/3572908956 shutdown_connections 2024-03-29T08:04:29.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:29.831+0000 7f2e2a200700 1 -- 172.21.15.37:0/3572908956 wait complete. 2024-03-29T08:04:30.577 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 99b492c2-eda1-11ee-b647-cb9ed24678a4 -e sha1=d0e392d9aee635f2b65768c460c7a151ead7a5e2 -- bash -c 'ceph config set global log_to_journald false --force' 2024-03-29T08:04:30.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:04:30 smithi037 ceph-mon[35162]: pgmap v232: 65 pgs: 65 active+clean; 587 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 767 B/s wr, 0 op/s 2024-03-29T08:04:31.015 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/mon.smithi037/config 2024-03-29T08:04:31.055 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:04:30 smithi148 ceph-mon[44992]: pgmap v232: 65 pgs: 65 active+clean; 587 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 767 B/s wr, 0 op/s 2024-03-29T08:04:32.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:32.879+0000 7f5529c00700 1 -- 172.21.15.37:0/2423956023 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f55240fffc0 msgr2=0x7f55241023a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:04:32.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:32.879+0000 7f5529c00700 1 --2- 172.21.15.37:0/2423956023 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f55240fffc0 0x7f55241023a0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f550c004940 tx=0x7f550c009e30 comp rx=0 tx=0).stop 2024-03-29T08:04:32.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:32.879+0000 7f5529c00700 1 -- 172.21.15.37:0/2423956023 shutdown_connections 2024-03-29T08:04:32.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:32.879+0000 7f5529c00700 1 --2- 172.21.15.37:0/2423956023 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f55240fffc0 0x7f55241023a0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:04:32.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:32.879+0000 7f5529c00700 1 --2- 172.21.15.37:0/2423956023 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5524066220 0x7f55240ffa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:04:32.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:32.879+0000 7f5529c00700 1 -- 172.21.15.37:0/2423956023 >> 172.21.15.37:0/2423956023 conn(0x7f55240f7430 msgr2=0x7f55240f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:04:32.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:32.879+0000 7f5529c00700 1 -- 172.21.15.37:0/2423956023 shutdown_connections 2024-03-29T08:04:32.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:32.879+0000 7f5529c00700 1 -- 172.21.15.37:0/2423956023 wait complete. 2024-03-29T08:04:32.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:32.880+0000 7f5529c00700 1 Processor -- start 2024-03-29T08:04:32.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:32.880+0000 7f5529c00700 1 -- start start 2024-03-29T08:04:32.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:32.881+0000 7f5529c00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5524066220 0x7f552410d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:04:32.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:32.881+0000 7f5529c00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f55240fffc0 0x7f552410df30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:04:32.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:32.881+0000 7f5529c00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f552410e5c0 con 0x7f55240fffc0 2024-03-29T08:04:32.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:32.881+0000 7f5529c00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f552410edb0 con 0x7f5524066220 2024-03-29T08:04:32.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:32.881+0000 7f551be00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f55240fffc0 0x7f552410df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:04:32.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:32.881+0000 7f551be00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f55240fffc0 0x7f552410df30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46446/0 (socket says 172.21.15.37:46446) 2024-03-29T08:04:32.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:32.881+0000 7f551be00700 1 -- 172.21.15.37:0/3175463480 learned_addr learned my addr 172.21.15.37:0/3175463480 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:04:32.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:32.881+0000 7f5522a00700 1 --2- 172.21.15.37:0/3175463480 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5524066220 0x7f552410d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:04:32.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:32.881+0000 7f551be00700 1 -- 172.21.15.37:0/3175463480 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5524066220 msgr2=0x7f552410d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:04:32.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:32.881+0000 7f551be00700 1 --2- 172.21.15.37:0/3175463480 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5524066220 0x7f552410d9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:04:32.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:32.881+0000 7f551be00700 1 -- 172.21.15.37:0/3175463480 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f550c005040 con 0x7f55240fffc0 2024-03-29T08:04:32.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:32.882+0000 7f551be00700 1 --2- 172.21.15.37:0/3175463480 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f55240fffc0 0x7f552410df30 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f550c00b3b0 tx=0x7f550c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:04:32.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:32.882+0000 7f551aa00700 1 -- 172.21.15.37:0/3175463480 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f550c00e040 con 0x7f55240fffc0 2024-03-29T08:04:32.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:32.882+0000 7f551aa00700 1 -- 172.21.15.37:0/3175463480 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f550c004290 con 0x7f55240fffc0 2024-03-29T08:04:32.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:32.882+0000 7f551aa00700 1 -- 172.21.15.37:0/3175463480 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f550c01c3f0 con 0x7f55240fffc0 2024-03-29T08:04:32.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:32.882+0000 7f5529c00700 1 -- 172.21.15.37:0/3175463480 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f552410f030 con 0x7f55240fffc0 2024-03-29T08:04:32.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:32.882+0000 7f5529c00700 1 -- 172.21.15.37:0/3175463480 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f552410f5f0 con 0x7f55240fffc0 2024-03-29T08:04:32.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:32.883+0000 7f5529c00700 1 -- 172.21.15.37:0/3175463480 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f552405f5f0 con 0x7f55240fffc0 2024-03-29T08:04:32.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:32.884+0000 7f551aa00700 1 -- 172.21.15.37:0/3175463480 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f550c00bc50 con 0x7f55240fffc0 2024-03-29T08:04:32.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:32.885+0000 7f551aa00700 1 --2- 172.21.15.37:0/3175463480 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f55080692b0 0x7f550806b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:04:32.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:32.885+0000 7f551aa00700 1 -- 172.21.15.37:0/3175463480 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f550c088550 con 0x7f55240fffc0 2024-03-29T08:04:32.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:32.885+0000 7f5522a00700 1 --2- 172.21.15.37:0/3175463480 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f55080692b0 0x7f550806b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:04:32.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:32.886+0000 7f5522a00700 1 --2- 172.21.15.37:0/3175463480 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f55080692b0 0x7f550806b760 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f5514006fd0 tx=0x7f5514008040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:04:32.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:32.890+0000 7f551aa00700 1 -- 172.21.15.37:0/3175463480 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f550c0582a0 con 0x7f55240fffc0 2024-03-29T08:04:32.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:04:32 smithi037 ceph-mon[35162]: pgmap v233: 65 pgs: 65 active+clean; 587 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 767 B/s wr, 0 op/s 2024-03-29T08:04:33.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:04:32 smithi148 ceph-mon[44992]: pgmap v233: 65 pgs: 65 active+clean; 587 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 767 B/s wr, 0 op/s 2024-03-29T08:04:33.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:33.094+0000 7f5529c00700 1 -- 172.21.15.37:0/3175463480 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command([{prefix=config set, name=log_to_journald}] v 0) v1 -- 0x7f552410f8d0 con 0x7f55240fffc0 2024-03-29T08:04:33.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:33.094+0000 7f551aa00700 1 -- 172.21.15.37:0/3175463480 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{prefix=config set, name=log_to_journald}]=0 v21)=0 v21) v1 ==== 135+0+0 (secure 0 0 0) 0x7f550c057e30 con 0x7f55240fffc0 2024-03-29T08:04:33.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:33.097+0000 7f5529c00700 1 -- 172.21.15.37:0/3175463480 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f55080692b0 msgr2=0x7f550806b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:04:33.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:33.097+0000 7f5529c00700 1 --2- 172.21.15.37:0/3175463480 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f55080692b0 0x7f550806b760 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f5514006fd0 tx=0x7f5514008040 comp rx=0 tx=0).stop 2024-03-29T08:04:33.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:33.097+0000 7f5529c00700 1 -- 172.21.15.37:0/3175463480 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f55240fffc0 msgr2=0x7f552410df30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:04:33.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:33.097+0000 7f5529c00700 1 --2- 172.21.15.37:0/3175463480 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f55240fffc0 0x7f552410df30 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f550c00b3b0 tx=0x7f550c009e30 comp rx=0 tx=0).stop 2024-03-29T08:04:33.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:33.097+0000 7f5529c00700 1 -- 172.21.15.37:0/3175463480 shutdown_connections 2024-03-29T08:04:33.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:33.097+0000 7f5529c00700 1 --2- 172.21.15.37:0/3175463480 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f55080692b0 0x7f550806b760 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:04:33.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:33.097+0000 7f5529c00700 1 --2- 172.21.15.37:0/3175463480 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5524066220 0x7f552410d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:04:33.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:33.097+0000 7f5529c00700 1 --2- 172.21.15.37:0/3175463480 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f55240fffc0 0x7f552410df30 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:04:33.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:33.097+0000 7f5529c00700 1 -- 172.21.15.37:0/3175463480 >> 172.21.15.37:0/3175463480 conn(0x7f55240f7430 msgr2=0x7f55240f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:04:33.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:33.097+0000 7f5529c00700 1 -- 172.21.15.37:0/3175463480 shutdown_connections 2024-03-29T08:04:33.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:33.097+0000 7f5529c00700 1 -- 172.21.15.37:0/3175463480 wait complete. 2024-03-29T08:04:33.828 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 99b492c2-eda1-11ee-b647-cb9ed24678a4 -e sha1=d0e392d9aee635f2b65768c460c7a151ead7a5e2 -- bash -c 'ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1 --daemon-types mgr' 2024-03-29T08:04:34.269 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/mon.smithi037/config 2024-03-29T08:04:34.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:04:34 smithi037 ceph-mon[35162]: pgmap v234: 65 pgs: 65 active+clean; 587 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 767 B/s wr, 0 op/s 2024-03-29T08:04:35.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:04:34 smithi148 ceph-mon[44992]: pgmap v234: 65 pgs: 65 active+clean; 587 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 767 B/s wr, 0 op/s 2024-03-29T08:04:36.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:36.092+0000 7fde7f200700 1 -- 172.21.15.37:0/61623759 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fde78071ea0 msgr2=0x7fde780703b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:04:36.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:36.092+0000 7fde7f200700 1 --2- 172.21.15.37:0/61623759 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fde78071ea0 0x7fde780703b0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fde68004940 tx=0x7fde68009e30 comp rx=0 tx=0).stop 2024-03-29T08:04:36.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:36.092+0000 7fde7f200700 1 -- 172.21.15.37:0/61623759 shutdown_connections 2024-03-29T08:04:36.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:36.092+0000 7fde7f200700 1 --2- 172.21.15.37:0/61623759 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fde780708f0 0x7fde78070d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:04:36.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:36.092+0000 7fde7f200700 1 --2- 172.21.15.37:0/61623759 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fde78071ea0 0x7fde780703b0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:04:36.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:36.092+0000 7fde7f200700 1 -- 172.21.15.37:0/61623759 >> 172.21.15.37:0/61623759 conn(0x7fde780f8550 msgr2=0x7fde780fa960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:04:36.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:36.093+0000 7fde7f200700 1 -- 172.21.15.37:0/61623759 shutdown_connections 2024-03-29T08:04:36.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:36.093+0000 7fde7f200700 1 -- 172.21.15.37:0/61623759 wait complete. 2024-03-29T08:04:36.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:36.094+0000 7fde7f200700 1 Processor -- start 2024-03-29T08:04:36.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:36.094+0000 7fde7f200700 1 -- start start 2024-03-29T08:04:36.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:36.094+0000 7fde7f200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fde780708f0 0x7fde7810fac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:04:36.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:36.095+0000 7fde7f200700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fde78071ea0 0x7fde78110000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:04:36.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:36.095+0000 7fde7f200700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde78110600 con 0x7fde780708f0 2024-03-29T08:04:36.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:36.095+0000 7fde7f200700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde78110770 con 0x7fde78071ea0 2024-03-29T08:04:36.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:36.095+0000 7fde77e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fde780708f0 0x7fde7810fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:04:36.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:36.095+0000 7fde77e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fde780708f0 0x7fde7810fac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46472/0 (socket says 172.21.15.37:46472) 2024-03-29T08:04:36.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:36.095+0000 7fde77e00700 1 -- 172.21.15.37:0/3164566652 learned_addr learned my addr 172.21.15.37:0/3164566652 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:04:36.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:36.095+0000 7fde77400700 1 --2- 172.21.15.37:0/3164566652 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fde78071ea0 0x7fde78110000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:04:36.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:36.095+0000 7fde77e00700 1 -- 172.21.15.37:0/3164566652 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fde78071ea0 msgr2=0x7fde78110000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:04:36.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:36.095+0000 7fde77e00700 1 --2- 172.21.15.37:0/3164566652 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fde78071ea0 0x7fde78110000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:04:36.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:36.095+0000 7fde77e00700 1 -- 172.21.15.37:0/3164566652 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde68005040 con 0x7fde780708f0 2024-03-29T08:04:36.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:36.095+0000 7fde77400700 1 --2- 172.21.15.37:0/3164566652 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fde78071ea0 0x7fde78110000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:04:36.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:36.096+0000 7fde77e00700 1 --2- 172.21.15.37:0/3164566652 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fde780708f0 0x7fde7810fac0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fde68009380 tx=0x7fde68009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:04:36.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:36.096+0000 7fde6fe00700 1 -- 172.21.15.37:0/3164566652 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fde6800e040 con 0x7fde780708f0 2024-03-29T08:04:36.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:36.096+0000 7fde6fe00700 1 -- 172.21.15.37:0/3164566652 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fde68004290 con 0x7fde780708f0 2024-03-29T08:04:36.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:36.096+0000 7fde6fe00700 1 -- 172.21.15.37:0/3164566652 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fde6801c3f0 con 0x7fde780708f0 2024-03-29T08:04:36.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:36.096+0000 7fde7f200700 1 -- 172.21.15.37:0/3164566652 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde78115170 con 0x7fde780708f0 2024-03-29T08:04:36.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:36.096+0000 7fde7f200700 1 -- 172.21.15.37:0/3164566652 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde78115730 con 0x7fde780708f0 2024-03-29T08:04:36.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:36.097+0000 7fde7f200700 1 -- 172.21.15.37:0/3164566652 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde7804c710 con 0x7fde780708f0 2024-03-29T08:04:36.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:36.098+0000 7fde6fe00700 1 -- 172.21.15.37:0/3164566652 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fde68013420 con 0x7fde780708f0 2024-03-29T08:04:36.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:36.099+0000 7fde6fe00700 1 --2- 172.21.15.37:0/3164566652 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7fde5c0692a0 0x7fde5c06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:04:36.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:36.099+0000 7fde6fe00700 1 -- 172.21.15.37:0/3164566652 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5550+0+0 (secure 0 0 0) 0x7fde68088590 con 0x7fde780708f0 2024-03-29T08:04:36.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:36.099+0000 7fde77400700 1 --2- 172.21.15.37:0/3164566652 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7fde5c0692a0 0x7fde5c06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:04:36.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:36.100+0000 7fde77400700 1 --2- 172.21.15.37:0/3164566652 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7fde5c0692a0 0x7fde5c06b750 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fde64009fd0 tx=0x7fde64009ee0 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:04:36.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:36.104+0000 7fde6fe00700 1 -- 172.21.15.37:0/3164566652 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fde680582e0 con 0x7fde780708f0 2024-03-29T08:04:36.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:04:36.325+0000 7fde7f200700 1 -- 172.21.15.37:0/3164566652 --> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] -- mgr_command(tid 0: {"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:d0e392d9aee635f2b65768c460c7a151ead7a5e2", "daemon_types": "mgr", "target": ["mon-mgr", ""]}) v1 -- 0x7fde78103f90 con 0x7fde5c0692a0 2024-03-29T08:04:37.055 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:04:36 smithi148 ceph-mon[44992]: pgmap v235: 65 pgs: 65 active+clean; 587 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 597 B/s wr, 0 op/s 2024-03-29T08:04:37.176 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:04:36 smithi037 ceph-mon[35162]: pgmap v235: 65 pgs: 65 active+clean; 587 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 597 B/s wr, 0 op/s 2024-03-29T08:04:37.997 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:04:37 smithi148 ceph-mon[44992]: from='client.14558 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:d0e392d9aee635f2b65768c460c7a151ead7a5e2", "daemon_types": "mgr", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:04:38.176 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:04:37 smithi037 ceph-mon[35162]: from='client.14558 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:d0e392d9aee635f2b65768c460c7a151ead7a5e2", "daemon_types": "mgr", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:04:39.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:04:38 smithi148 ceph-mon[44992]: pgmap v236: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.1 KiB/s wr, 1 op/s 2024-03-29T08:04:39.176 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:04:38 smithi037 ceph-mon[35162]: pgmap v236: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.1 KiB/s wr, 1 op/s 2024-03-29T08:04:41.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:04:40 smithi148 ceph-mon[44992]: pgmap v237: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 597 B/s wr, 0 op/s 2024-03-29T08:04:41.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:04:40 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:04:41.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:04:40 smithi037 ceph-mon[35162]: pgmap v237: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 597 B/s wr, 0 op/s 2024-03-29T08:04:41.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:04:40 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:04:43.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:04:42 smithi148 ceph-mon[44992]: pgmap v238: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 597 B/s wr, 0 op/s 2024-03-29T08:04:43.176 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:04:42 smithi037 ceph-mon[35162]: pgmap v238: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 597 B/s wr, 0 op/s 2024-03-29T08:04:45.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:04:44 smithi148 ceph-mon[44992]: pgmap v239: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 597 B/s wr, 0 op/s 2024-03-29T08:04:45.176 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:04:44 smithi037 ceph-mon[35162]: pgmap v239: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 597 B/s wr, 0 op/s 2024-03-29T08:04:47.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:04:46 smithi148 ceph-mon[44992]: pgmap v240: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-03-29T08:04:47.176 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:04:46 smithi037 ceph-mon[35162]: pgmap v240: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-03-29T08:04:49.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:04:48 smithi148 ceph-mon[44992]: pgmap v241: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-03-29T08:04:49.176 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:04:48 smithi037 ceph-mon[35162]: pgmap v241: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-03-29T08:04:51.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:04:50 smithi148 ceph-mon[44992]: pgmap v242: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:04:51.176 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:04:50 smithi037 ceph-mon[35162]: pgmap v242: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:04:53.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:04:52 smithi148 ceph-mon[44992]: pgmap v243: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:04:53.176 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:04:52 smithi037 ceph-mon[35162]: pgmap v243: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:04:55.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:04:54 smithi148 ceph-mon[44992]: pgmap v244: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:04:55.176 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:04:54 smithi037 ceph-mon[35162]: pgmap v244: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:04:56.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:04:55 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:04:56.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:04:55 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:04:57.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:04:56 smithi148 ceph-mon[44992]: pgmap v245: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:04:57.176 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:04:56 smithi037 ceph-mon[35162]: pgmap v245: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:04:59.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:04:58 smithi148 ceph-mon[44992]: pgmap v246: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:04:59.176 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:04:58 smithi037 ceph-mon[35162]: pgmap v246: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:05:01.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:05:00 smithi148 ceph-mon[44992]: pgmap v247: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:05:01.176 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:05:00 smithi037 ceph-mon[35162]: pgmap v247: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:05:03.055 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:05:02 smithi148 ceph-mon[44992]: pgmap v248: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:05:03.176 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:05:02 smithi037 ceph-mon[35162]: pgmap v248: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:05:04.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:05:03 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T08:05:04.174 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:05:03 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T08:05:05.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:05:04 smithi148 ceph-mon[44992]: pgmap v249: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:05:05.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:05:04 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T08:05:05.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:05:04 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T08:05:05.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:05:04 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:05:05.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:05:04 smithi037 ceph-mon[35162]: pgmap v249: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:05:05.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:05:04 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T08:05:05.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:05:04 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T08:05:05.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:05:04 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:05:07.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:05:06 smithi148 ceph-mon[44992]: pgmap v250: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:05:07.176 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:05:06 smithi037 ceph-mon[35162]: pgmap v250: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:05:08.450 INFO:teuthology.orchestra.run.smithi037.stdout:Initiating upgrade to quay.ceph.io/ceph-ci/ceph:d0e392d9aee635f2b65768c460c7a151ead7a5e2 2024-03-29T08:05:08.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:08.450+0000 7fde6fe00700 1 -- 172.21.15.37:0/3164566652 <== mgr.14208 v2:172.21.15.37:6800/1802836043 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+89 (secure 0 0 0) 0x7fde78103f90 con 0x7fde5c0692a0 2024-03-29T08:05:08.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:08.453+0000 7fde7f200700 1 -- 172.21.15.37:0/3164566652 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7fde5c0692a0 msgr2=0x7fde5c06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:05:08.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:08.453+0000 7fde7f200700 1 --2- 172.21.15.37:0/3164566652 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7fde5c0692a0 0x7fde5c06b750 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fde64009fd0 tx=0x7fde64009ee0 comp rx=0 tx=0).stop 2024-03-29T08:05:08.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:08.453+0000 7fde7f200700 1 -- 172.21.15.37:0/3164566652 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fde780708f0 msgr2=0x7fde7810fac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:05:08.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:08.453+0000 7fde7f200700 1 --2- 172.21.15.37:0/3164566652 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fde780708f0 0x7fde7810fac0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fde68009380 tx=0x7fde68009e30 comp rx=0 tx=0).stop 2024-03-29T08:05:08.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:08.453+0000 7fde7f200700 1 -- 172.21.15.37:0/3164566652 shutdown_connections 2024-03-29T08:05:08.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:08.453+0000 7fde7f200700 1 --2- 172.21.15.37:0/3164566652 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7fde5c0692a0 0x7fde5c06b750 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:05:08.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:08.453+0000 7fde7f200700 1 --2- 172.21.15.37:0/3164566652 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fde780708f0 0x7fde7810fac0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:05:08.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:08.453+0000 7fde7f200700 1 --2- 172.21.15.37:0/3164566652 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fde78071ea0 0x7fde78110000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:05:08.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:08.453+0000 7fde7f200700 1 -- 172.21.15.37:0/3164566652 >> 172.21.15.37:0/3164566652 conn(0x7fde780f8550 msgr2=0x7fde780f9d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:05:08.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:08.453+0000 7fde7f200700 1 -- 172.21.15.37:0/3164566652 shutdown_connections 2024-03-29T08:05:08.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:08.453+0000 7fde7f200700 1 -- 172.21.15.37:0/3164566652 wait complete. 2024-03-29T08:05:09.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:05:08 smithi148 ceph-mon[44992]: pgmap v251: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:05:09.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:05:08 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:05:09.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:05:08 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T08:05:09.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:05:08 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T08:05:09.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:05:08 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T08:05:09.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:05:08 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:05:09.176 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:05:08 smithi037 ceph-mon[35162]: pgmap v251: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:05:09.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:05:08 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:05:09.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:05:08 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T08:05:09.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:05:08 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T08:05:09.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:05:08 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T08:05:09.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:05:08 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:05:09.223 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 99b492c2-eda1-11ee-b647-cb9ed24678a4 -e sha1=d0e392d9aee635f2b65768c460c7a151ead7a5e2 -- bash -c 'while ceph orch upgrade status | jq '"'"'.in_progress'"'"' | grep true && ! ceph orch upgrade status | jq '"'"'.message'"'"' | grep Error ; do ceph orch ps ; ceph versions ; ceph orch upgrade status ; sleep 30 ; done' 2024-03-29T08:05:09.709 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/mon.smithi037/config 2024-03-29T08:05:10.055 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:05:09 smithi148 ceph-mon[44992]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:d0e392d9aee635f2b65768c460c7a151ead7a5e2 2024-03-29T08:05:10.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:05:09 smithi148 ceph-mon[44992]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:d0e392d9aee635f2b65768c460c7a151ead7a5e2 2024-03-29T08:05:10.093 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:05:09 smithi037 ceph-mon[35162]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:d0e392d9aee635f2b65768c460c7a151ead7a5e2 2024-03-29T08:05:10.093 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:05:09 smithi037 ceph-mon[35162]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:d0e392d9aee635f2b65768c460c7a151ead7a5e2 2024-03-29T08:05:11.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:05:10 smithi148 ceph-mon[44992]: pgmap v252: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:05:11.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:05:10 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:05:11.176 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:05:10 smithi037 ceph-mon[35162]: pgmap v252: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:05:11.176 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:05:10 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:05:13.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:05:12 smithi148 ceph-mon[44992]: pgmap v253: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:05:13.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:05:12 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:05:13.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:05:12 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T08:05:13.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:05:12 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:05:13.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:13.139+0000 7fed38c00700 1 -- 172.21.15.37:0/1271067243 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed34066f50 msgr2=0x7fed340ff9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:05:13.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:13.139+0000 7fed38c00700 1 --2- 172.21.15.37:0/1271067243 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed34066f50 0x7fed340ff9c0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fed24004940 tx=0x7fed24009e30 comp rx=0 tx=0).stop 2024-03-29T08:05:13.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:13.139+0000 7fed38c00700 1 -- 172.21.15.37:0/1271067243 shutdown_connections 2024-03-29T08:05:13.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:13.139+0000 7fed38c00700 1 --2- 172.21.15.37:0/1271067243 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fed340fff00 0x7fed34102320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:05:13.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:13.139+0000 7fed38c00700 1 --2- 172.21.15.37:0/1271067243 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed34066f50 0x7fed340ff9c0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:05:13.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:13.139+0000 7fed38c00700 1 -- 172.21.15.37:0/1271067243 >> 172.21.15.37:0/1271067243 conn(0x7fed340f7450 msgr2=0x7fed340f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:05:13.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:13.139+0000 7fed38c00700 1 -- 172.21.15.37:0/1271067243 shutdown_connections 2024-03-29T08:05:13.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:13.140+0000 7fed38c00700 1 -- 172.21.15.37:0/1271067243 wait complete. 2024-03-29T08:05:13.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:13.140+0000 7fed38c00700 1 Processor -- start 2024-03-29T08:05:13.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:13.141+0000 7fed38c00700 1 -- start start 2024-03-29T08:05:13.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:13.141+0000 7fed38c00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed34066f50 0x7fed3410fb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:05:13.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:13.141+0000 7fed38c00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fed340fff00 0x7fed34110050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:05:13.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:13.141+0000 7fed38c00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed34110650 con 0x7fed34066f50 2024-03-29T08:05:13.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:13.141+0000 7fed38c00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed341107c0 con 0x7fed340fff00 2024-03-29T08:05:13.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:13.141+0000 7fed31e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed34066f50 0x7fed3410fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:05:13.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:13.141+0000 7fed31e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed34066f50 0x7fed3410fb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51820/0 (socket says 172.21.15.37:51820) 2024-03-29T08:05:13.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:13.141+0000 7fed31e00700 1 -- 172.21.15.37:0/2190241503 learned_addr learned my addr 172.21.15.37:0/2190241503 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:05:13.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:13.141+0000 7fed31400700 1 --2- 172.21.15.37:0/2190241503 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fed340fff00 0x7fed34110050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:05:13.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:13.142+0000 7fed31e00700 1 -- 172.21.15.37:0/2190241503 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fed340fff00 msgr2=0x7fed34110050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:05:13.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:13.142+0000 7fed31e00700 1 --2- 172.21.15.37:0/2190241503 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fed340fff00 0x7fed34110050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:05:13.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:13.142+0000 7fed31e00700 1 -- 172.21.15.37:0/2190241503 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed24005040 con 0x7fed34066f50 2024-03-29T08:05:13.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:13.142+0000 7fed31e00700 1 --2- 172.21.15.37:0/2190241503 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed34066f50 0x7fed3410fb10 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fed2400b900 tx=0x7fed24009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:05:13.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:13.142+0000 7fed2a000700 1 -- 172.21.15.37:0/2190241503 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fed2400e040 con 0x7fed34066f50 2024-03-29T08:05:13.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:13.142+0000 7fed2a000700 1 -- 172.21.15.37:0/2190241503 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fed24004290 con 0x7fed34066f50 2024-03-29T08:05:13.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:13.142+0000 7fed38c00700 1 -- 172.21.15.37:0/2190241503 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed34115170 con 0x7fed34066f50 2024-03-29T08:05:13.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:13.142+0000 7fed2a000700 1 -- 172.21.15.37:0/2190241503 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fed2401c3f0 con 0x7fed34066f50 2024-03-29T08:05:13.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:13.142+0000 7fed38c00700 1 -- 172.21.15.37:0/2190241503 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed34115730 con 0x7fed34066f50 2024-03-29T08:05:13.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:13.144+0000 7fed38c00700 1 -- 172.21.15.37:0/2190241503 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed340f8f80 con 0x7fed34066f50 2024-03-29T08:05:13.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:13.145+0000 7fed2a000700 1 -- 172.21.15.37:0/2190241503 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fed24013420 con 0x7fed34066f50 2024-03-29T08:05:13.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:13.146+0000 7fed2a000700 1 --2- 172.21.15.37:0/2190241503 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7fed180692a0 0x7fed1806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:05:13.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:13.146+0000 7fed2a000700 1 -- 172.21.15.37:0/2190241503 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5550+0+0 (secure 0 0 0) 0x7fed240883b0 con 0x7fed34066f50 2024-03-29T08:05:13.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:13.146+0000 7fed31400700 1 --2- 172.21.15.37:0/2190241503 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7fed180692a0 0x7fed1806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:05:13.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:13.147+0000 7fed31400700 1 --2- 172.21.15.37:0/2190241503 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7fed180692a0 0x7fed1806b750 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fed20004f40 tx=0x7fed2000b040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:05:13.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:13.151+0000 7fed2a000700 1 -- 172.21.15.37:0/2190241503 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fed240581c0 con 0x7fed34066f50 2024-03-29T08:05:13.176 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:05:12 smithi037 ceph-mon[35162]: pgmap v253: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:05:13.176 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:05:12 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:05:13.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:05:12 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T08:05:13.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:05:12 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:05:13.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:13.383+0000 7fed38c00700 1 -- 172.21.15.37:0/2190241503 --> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fed3402b340 con 0x7fed180692a0 2024-03-29T08:05:13.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:13.384+0000 7fed2a000700 1 -- 172.21.15.37:0/2190241503 <== mgr.14208 v2:172.21.15.37:6800/1802836043 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+439 (secure 0 0 0) 0x7fed3402b340 con 0x7fed180692a0 2024-03-29T08:05:13.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:13.386+0000 7fed38c00700 1 -- 172.21.15.37:0/2190241503 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7fed180692a0 msgr2=0x7fed1806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:05:13.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:13.386+0000 7fed38c00700 1 --2- 172.21.15.37:0/2190241503 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7fed180692a0 0x7fed1806b750 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fed20004f40 tx=0x7fed2000b040 comp rx=0 tx=0).stop 2024-03-29T08:05:13.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:13.386+0000 7fed38c00700 1 -- 172.21.15.37:0/2190241503 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed34066f50 msgr2=0x7fed3410fb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:05:13.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:13.386+0000 7fed38c00700 1 --2- 172.21.15.37:0/2190241503 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed34066f50 0x7fed3410fb10 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fed2400b900 tx=0x7fed24009e30 comp rx=0 tx=0).stop 2024-03-29T08:05:13.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:13.386+0000 7fed38c00700 1 -- 172.21.15.37:0/2190241503 shutdown_connections 2024-03-29T08:05:13.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:13.386+0000 7fed38c00700 1 --2- 172.21.15.37:0/2190241503 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7fed180692a0 0x7fed1806b750 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:05:13.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:13.386+0000 7fed38c00700 1 --2- 172.21.15.37:0/2190241503 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed34066f50 0x7fed3410fb10 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:05:13.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:13.386+0000 7fed38c00700 1 --2- 172.21.15.37:0/2190241503 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fed340fff00 0x7fed34110050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:05:13.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:13.386+0000 7fed38c00700 1 -- 172.21.15.37:0/2190241503 >> 172.21.15.37:0/2190241503 conn(0x7fed340f7450 msgr2=0x7fed340fbba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:05:13.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:13.387+0000 7fed38c00700 1 -- 172.21.15.37:0/2190241503 shutdown_connections 2024-03-29T08:05:13.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:13.387+0000 7fed38c00700 1 -- 172.21.15.37:0/2190241503 wait complete. 2024-03-29T08:05:13.401 INFO:teuthology.orchestra.run.smithi037.stdout:true 2024-03-29T08:05:13.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:13.539+0000 7f255b000700 1 -- 172.21.15.37:0/345176051 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f25540fff00 msgr2=0x7f2554102320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:05:13.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:13.539+0000 7f255b000700 1 --2- 172.21.15.37:0/345176051 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f25540fff00 0x7f2554102320 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f2540004940 tx=0x7f2540009e30 comp rx=0 tx=0).stop 2024-03-29T08:05:13.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:13.540+0000 7f255b000700 1 -- 172.21.15.37:0/345176051 shutdown_connections 2024-03-29T08:05:13.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:13.540+0000 7f255b000700 1 --2- 172.21.15.37:0/345176051 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f25540fff00 0x7f2554102320 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:05:13.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:13.540+0000 7f255b000700 1 --2- 172.21.15.37:0/345176051 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f25540662e0 0x7f25540ff9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:05:13.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:13.540+0000 7f255b000700 1 -- 172.21.15.37:0/345176051 >> 172.21.15.37:0/345176051 conn(0x7f25540f7430 msgr2=0x7f25540f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:05:13.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:13.540+0000 7f255b000700 1 -- 172.21.15.37:0/345176051 shutdown_connections 2024-03-29T08:05:13.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:13.540+0000 7f255b000700 1 -- 172.21.15.37:0/345176051 wait complete. 2024-03-29T08:05:13.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:13.541+0000 7f255b000700 1 Processor -- start 2024-03-29T08:05:13.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:13.541+0000 7f255b000700 1 -- start start 2024-03-29T08:05:13.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:13.542+0000 7f255b000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f25540662e0 0x7f255410d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:05:13.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:13.542+0000 7f255b000700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f25540fff00 0x7f255410de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:05:13.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:13.542+0000 7f255b000700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f255410e460 con 0x7f25540662e0 2024-03-29T08:05:13.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:13.542+0000 7f255b000700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f255410ec60 con 0x7f25540fff00 2024-03-29T08:05:13.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:13.542+0000 7f2553400700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f25540fff00 0x7f255410de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:05:13.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:13.542+0000 7f2553e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f25540662e0 0x7f255410d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:05:13.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:13.542+0000 7f2553400700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f25540fff00 0x7f255410de60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.37:59300/0 (socket says 172.21.15.37:59300) 2024-03-29T08:05:13.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:13.542+0000 7f2553400700 1 -- 172.21.15.37:0/3117797529 learned_addr learned my addr 172.21.15.37:0/3117797529 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:05:13.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:13.543+0000 7f2553400700 1 -- 172.21.15.37:0/3117797529 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f25540662e0 msgr2=0x7f255410d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:05:13.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:13.543+0000 7f2553400700 1 --2- 172.21.15.37:0/3117797529 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f25540662e0 0x7f255410d920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:05:13.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:13.543+0000 7f2553400700 1 -- 172.21.15.37:0/3117797529 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2540005040 con 0x7f25540fff00 2024-03-29T08:05:13.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:13.543+0000 7f2553e00700 1 --2- 172.21.15.37:0/3117797529 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f25540662e0 0x7f255410d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T08:05:13.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:13.543+0000 7f2553400700 1 --2- 172.21.15.37:0/3117797529 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f25540fff00 0x7f255410de60 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f254000b0d0 tx=0x7f2540009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:05:13.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:13.543+0000 7f254be00700 1 -- 172.21.15.37:0/3117797529 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f254000e070 con 0x7f25540fff00 2024-03-29T08:05:13.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:13.543+0000 7f254be00700 1 -- 172.21.15.37:0/3117797529 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f254000b410 con 0x7f25540fff00 2024-03-29T08:05:13.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:13.544+0000 7f255b000700 1 -- 172.21.15.37:0/3117797529 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f255410eee0 con 0x7f25540fff00 2024-03-29T08:05:13.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:13.544+0000 7f254be00700 1 -- 172.21.15.37:0/3117797529 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2540012600 con 0x7f25540fff00 2024-03-29T08:05:13.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:13.544+0000 7f255b000700 1 -- 172.21.15.37:0/3117797529 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f255410f420 con 0x7f25540fff00 2024-03-29T08:05:13.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:13.545+0000 7f255b000700 1 -- 172.21.15.37:0/3117797529 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f255405f5f0 con 0x7f25540fff00 2024-03-29T08:05:13.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:13.547+0000 7f254be00700 1 -- 172.21.15.37:0/3117797529 <== mon.1 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f254000b9f0 con 0x7f25540fff00 2024-03-29T08:05:13.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:13.548+0000 7f254be00700 1 --2- 172.21.15.37:0/3117797529 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f25380692b0 0x7f253806b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:05:13.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:13.548+0000 7f254be00700 1 -- 172.21.15.37:0/3117797529 <== mon.1 v2:172.21.15.148:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f2540087e30 con 0x7f25540fff00 2024-03-29T08:05:13.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:13.548+0000 7f2553e00700 1 --2- 172.21.15.37:0/3117797529 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f25380692b0 0x7f253806b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:05:13.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:13.549+0000 7f2553e00700 1 --2- 172.21.15.37:0/3117797529 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f25380692b0 0x7f253806b760 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f2544009b80 tx=0x7f254400b040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:05:13.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:13.555+0000 7f254be00700 1 -- 172.21.15.37:0/3117797529 <== mon.1 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f2540057c30 con 0x7f25540fff00 2024-03-29T08:05:13.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:13.799+0000 7f255b000700 1 -- 172.21.15.37:0/3117797529 --> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f255410f7d0 con 0x7f25380692b0 2024-03-29T08:05:13.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:13.800+0000 7f254be00700 1 -- 172.21.15.37:0/3117797529 <== mgr.14208 v2:172.21.15.37:6800/1802836043 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+439 (secure 0 0 0) 0x7f255410f7d0 con 0x7f25380692b0 2024-03-29T08:05:13.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:13.803+0000 7f255b000700 1 -- 172.21.15.37:0/3117797529 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f25380692b0 msgr2=0x7f253806b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:05:13.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:13.803+0000 7f255b000700 1 --2- 172.21.15.37:0/3117797529 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f25380692b0 0x7f253806b760 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f2544009b80 tx=0x7f254400b040 comp rx=0 tx=0).stop 2024-03-29T08:05:13.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:13.803+0000 7f255b000700 1 -- 172.21.15.37:0/3117797529 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f25540fff00 msgr2=0x7f255410de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:05:13.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:13.803+0000 7f255b000700 1 --2- 172.21.15.37:0/3117797529 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f25540fff00 0x7f255410de60 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f254000b0d0 tx=0x7f2540009e30 comp rx=0 tx=0).stop 2024-03-29T08:05:13.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:13.803+0000 7f255b000700 1 -- 172.21.15.37:0/3117797529 shutdown_connections 2024-03-29T08:05:13.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:13.803+0000 7f255b000700 1 --2- 172.21.15.37:0/3117797529 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f25380692b0 0x7f253806b760 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:05:13.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:13.803+0000 7f255b000700 1 --2- 172.21.15.37:0/3117797529 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f25540662e0 0x7f255410d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:05:13.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:13.803+0000 7f255b000700 1 --2- 172.21.15.37:0/3117797529 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f25540fff00 0x7f255410de60 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:05:13.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:13.803+0000 7f255b000700 1 -- 172.21.15.37:0/3117797529 >> 172.21.15.37:0/3117797529 conn(0x7f25540f7430 msgr2=0x7f25540f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:05:13.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:13.803+0000 7f255b000700 1 -- 172.21.15.37:0/3117797529 shutdown_connections 2024-03-29T08:05:13.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:13.803+0000 7f255b000700 1 -- 172.21.15.37:0/3117797529 wait complete. 2024-03-29T08:05:13.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:13.943+0000 7f4a57e00700 1 -- 172.21.15.37:0/4167478672 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4a58100090 msgr2=0x7f4a58100500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:05:13.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:13.943+0000 7f4a57e00700 1 --2- 172.21.15.37:0/4167478672 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4a58100090 0x7f4a58100500 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f4a44004880 tx=0x7f4a44009e30 comp rx=0 tx=0).stop 2024-03-29T08:05:13.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:13.943+0000 7f4a57e00700 1 -- 172.21.15.37:0/4167478672 shutdown_connections 2024-03-29T08:05:13.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:13.943+0000 7f4a57e00700 1 --2- 172.21.15.37:0/4167478672 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4a58100090 0x7f4a58100500 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:05:13.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:13.943+0000 7f4a57e00700 1 --2- 172.21.15.37:0/4167478672 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4a580fee90 0x7f4a580ff2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:05:13.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:13.943+0000 7f4a57e00700 1 -- 172.21.15.37:0/4167478672 >> 172.21.15.37:0/4167478672 conn(0x7f4a580fa790 msgr2=0x7f4a580fcb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:05:13.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:13.943+0000 7f4a57e00700 1 -- 172.21.15.37:0/4167478672 shutdown_connections 2024-03-29T08:05:13.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:13.943+0000 7f4a57e00700 1 -- 172.21.15.37:0/4167478672 wait complete. 2024-03-29T08:05:13.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:13.944+0000 7f4a57e00700 1 Processor -- start 2024-03-29T08:05:13.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:13.944+0000 7f4a57e00700 1 -- start start 2024-03-29T08:05:13.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:13.945+0000 7f4a57e00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4a580fee90 0x7f4a5810f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:05:13.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:13.945+0000 7f4a57e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4a58100090 0x7f4a5810fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:05:13.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:13.945+0000 7f4a57e00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a58110400 con 0x7f4a58100090 2024-03-29T08:05:13.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:13.945+0000 7f4a57e00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a58110c50 con 0x7f4a580fee90 2024-03-29T08:05:13.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:13.945+0000 7f4a56000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4a58100090 0x7f4a5810fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:05:13.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:13.945+0000 7f4a56a00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4a580fee90 0x7f4a5810f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:05:13.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:13.945+0000 7f4a56000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4a58100090 0x7f4a5810fe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51842/0 (socket says 172.21.15.37:51842) 2024-03-29T08:05:13.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:13.945+0000 7f4a56000700 1 -- 172.21.15.37:0/924863733 learned_addr learned my addr 172.21.15.37:0/924863733 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:05:13.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:13.945+0000 7f4a56a00700 1 -- 172.21.15.37:0/924863733 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4a58100090 msgr2=0x7f4a5810fe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:05:13.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:13.945+0000 7f4a56a00700 1 --2- 172.21.15.37:0/924863733 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4a58100090 0x7f4a5810fe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:05:13.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:13.946+0000 7f4a56a00700 1 -- 172.21.15.37:0/924863733 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a44005040 con 0x7f4a580fee90 2024-03-29T08:05:13.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:13.946+0000 7f4a56000700 1 --2- 172.21.15.37:0/924863733 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4a58100090 0x7f4a5810fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T08:05:13.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:13.946+0000 7f4a56a00700 1 --2- 172.21.15.37:0/924863733 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4a580fee90 0x7f4a5810f8c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f4a580ffc00 tx=0x7f4a4800b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:05:13.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:13.946+0000 7f4a4ea00700 1 -- 172.21.15.37:0/924863733 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4a48007cc0 con 0x7f4a580fee90 2024-03-29T08:05:13.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:13.946+0000 7f4a4ea00700 1 -- 172.21.15.37:0/924863733 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f4a4800f040 con 0x7f4a580fee90 2024-03-29T08:05:13.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:13.946+0000 7f4a4ea00700 1 -- 172.21.15.37:0/924863733 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4a4800c8e0 con 0x7f4a580fee90 2024-03-29T08:05:13.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:13.946+0000 7f4a57e00700 1 -- 172.21.15.37:0/924863733 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a58110ed0 con 0x7f4a580fee90 2024-03-29T08:05:13.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:13.946+0000 7f4a57e00700 1 -- 172.21.15.37:0/924863733 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a58111410 con 0x7f4a580fee90 2024-03-29T08:05:13.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:13.948+0000 7f4a57e00700 1 -- 172.21.15.37:0/924863733 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a580fe290 con 0x7f4a580fee90 2024-03-29T08:05:13.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:13.948+0000 7f4a4ea00700 1 -- 172.21.15.37:0/924863733 <== mon.1 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f4a4800ca40 con 0x7f4a580fee90 2024-03-29T08:05:13.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:13.949+0000 7f4a4ea00700 1 --2- 172.21.15.37:0/924863733 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f4a3c0692a0 0x7f4a3c06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:05:13.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:13.949+0000 7f4a4ea00700 1 -- 172.21.15.37:0/924863733 <== mon.1 v2:172.21.15.148:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f4a48086af0 con 0x7f4a580fee90 2024-03-29T08:05:13.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:13.950+0000 7f4a56000700 1 --2- 172.21.15.37:0/924863733 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f4a3c0692a0 0x7f4a3c06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:05:13.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:13.950+0000 7f4a56000700 1 --2- 172.21.15.37:0/924863733 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f4a3c0692a0 0x7f4a3c06b750 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f4a440135d0 tx=0x7f4a44009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:05:13.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:13.954+0000 7f4a4ea00700 1 -- 172.21.15.37:0/924863733 <== mon.1 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f4a4804e190 con 0x7f4a580fee90 2024-03-29T08:05:14.055 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:05:13 smithi148 ceph-mon[44992]: Upgrade: Target is version 18.2.2-1015-gd0e392d9 (unknown) 2024-03-29T08:05:14.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:05:13 smithi148 ceph-mon[44992]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59, digests ['quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59'] 2024-03-29T08:05:14.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:05:13 smithi148 ceph-mon[44992]: Upgrade: Need to upgrade myself (mgr.smithi037.wjikii) 2024-03-29T08:05:14.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:05:13 smithi148 ceph-mon[44992]: Upgrade: Pulling quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59 on smithi148 2024-03-29T08:05:14.176 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:05:13 smithi037 ceph-mon[35162]: Upgrade: Target is version 18.2.2-1015-gd0e392d9 (unknown) 2024-03-29T08:05:14.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:05:13 smithi037 ceph-mon[35162]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59, digests ['quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59'] 2024-03-29T08:05:14.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:05:13 smithi037 ceph-mon[35162]: Upgrade: Need to upgrade myself (mgr.smithi037.wjikii) 2024-03-29T08:05:14.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:05:13 smithi037 ceph-mon[35162]: Upgrade: Pulling quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59 on smithi148 2024-03-29T08:05:14.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:14.226+0000 7f4a57e00700 1 -- 172.21.15.37:0/924863733 --> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f4a581117f0 con 0x7f4a3c0692a0 2024-03-29T08:05:14.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:14.238+0000 7f4a4ea00700 1 -- 172.21.15.37:0/924863733 <== mgr.14208 v2:172.21.15.37:6800/1802836043 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3410 (secure 0 0 0) 0x7f4a581117f0 con 0x7f4a3c0692a0 2024-03-29T08:05:14.238 INFO:teuthology.orchestra.run.smithi037.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-29T08:05:14.238 INFO:teuthology.orchestra.run.smithi037.stdout:alertmanager.smithi037 smithi037 *:9093,9094 running (6m) 71s ago 8m 13.0M - 0.25.0 c8568f914cd2 a6824c801465 2024-03-29T08:05:14.239 INFO:teuthology.orchestra.run.smithi037.stdout:crash.smithi037 smithi037 running (8m) 71s ago 8m 6651k - 17.2.7-789-g4213e17c cf22eb8683be e0343935b25e 2024-03-29T08:05:14.239 INFO:teuthology.orchestra.run.smithi037.stdout:crash.smithi148 smithi148 running (7m) 76s ago 7m 6656k - 17.2.7-789-g4213e17c cf22eb8683be ff5164c552c1 2024-03-29T08:05:14.239 INFO:teuthology.orchestra.run.smithi037.stdout:grafana.smithi037 smithi037 *:3000 running (6m) 71s ago 7m 73.8M - 9.4.7 954c08fa6188 92b77c060e7a 2024-03-29T08:05:14.239 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi037.iifsmh smithi037 running (92s) 71s ago 92s 12.6M - 17.2.7-789-g4213e17c cf22eb8683be 074e1acf331f 2024-03-29T08:05:14.239 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi037.mmwgxe smithi037 running (101s) 71s ago 100s 13.4M - 17.2.7-789-g4213e17c cf22eb8683be 9a5b1e72de5c 2024-03-29T08:05:14.239 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi148.qukpjh smithi148 running (89s) 76s ago 88s 10.5M - 17.2.7-789-g4213e17c cf22eb8683be 8e469782a273 2024-03-29T08:05:14.239 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi148.sqteih smithi148 running (97s) 76s ago 97s 10.4M - 17.2.7-789-g4213e17c cf22eb8683be 8e68f858bb22 2024-03-29T08:05:14.239 INFO:teuthology.orchestra.run.smithi037.stdout:mgr.smithi037.wjikii smithi037 *:9283 running (9m) 71s ago 9m 480M - 17.2.7-789-g4213e17c cf22eb8683be 60172b53efa2 2024-03-29T08:05:14.239 INFO:teuthology.orchestra.run.smithi037.stdout:mgr.smithi148.onhgvm smithi148 *:8443,9283 running (6m) 76s ago 6m 431M - 17.2.7-789-g4213e17c cf22eb8683be a0fe1c4dd2b0 2024-03-29T08:05:14.239 INFO:teuthology.orchestra.run.smithi037.stdout:mon.smithi037 smithi037 running (9m) 71s ago 9m 44.9M 2048M 17.2.7-789-g4213e17c cf22eb8683be bb2101505730 2024-03-29T08:05:14.239 INFO:teuthology.orchestra.run.smithi037.stdout:mon.smithi148 smithi148 running (6m) 76s ago 6m 34.4M 2048M 17.2.7-789-g4213e17c cf22eb8683be 18839ed2952a 2024-03-29T08:05:14.239 INFO:teuthology.orchestra.run.smithi037.stdout:node-exporter.smithi037 smithi037 *:9100 running (8m) 71s ago 8m 7600k - 1.5.0 0da6a335fe13 eb3242f9dc57 2024-03-29T08:05:14.239 INFO:teuthology.orchestra.run.smithi037.stdout:node-exporter.smithi148 smithi148 *:9100 running (6m) 76s ago 6m 7814k - 1.5.0 0da6a335fe13 bb455992ed53 2024-03-29T08:05:14.240 INFO:teuthology.orchestra.run.smithi037.stdout:osd.0 smithi037 running (5m) 71s ago 5m 44.7M 4350M 17.2.7-789-g4213e17c cf22eb8683be 559fccd0ebe1 2024-03-29T08:05:14.240 INFO:teuthology.orchestra.run.smithi037.stdout:osd.1 smithi037 running (5m) 71s ago 5m 45.4M 4350M 17.2.7-789-g4213e17c cf22eb8683be 5582cff11b7a 2024-03-29T08:05:14.240 INFO:teuthology.orchestra.run.smithi037.stdout:osd.2 smithi037 running (4m) 71s ago 4m 42.8M 4350M 17.2.7-789-g4213e17c cf22eb8683be f809bfab13e5 2024-03-29T08:05:14.240 INFO:teuthology.orchestra.run.smithi037.stdout:osd.3 smithi148 running (4m) 76s ago 4m 42.8M 5372M 17.2.7-789-g4213e17c cf22eb8683be a0a8224bb4b0 2024-03-29T08:05:14.240 INFO:teuthology.orchestra.run.smithi037.stdout:osd.4 smithi148 running (3m) 76s ago 3m 41.0M 5372M 17.2.7-789-g4213e17c cf22eb8683be a00b69047ed7 2024-03-29T08:05:14.240 INFO:teuthology.orchestra.run.smithi037.stdout:osd.5 smithi148 running (3m) 76s ago 3m 42.8M 5372M 17.2.7-789-g4213e17c cf22eb8683be 044a3f7db7f8 2024-03-29T08:05:14.240 INFO:teuthology.orchestra.run.smithi037.stdout:prometheus.smithi037 smithi037 *:9095 running (6m) 71s ago 7m 51.7M - 2.43.0 a07b618ecd1d e8c23666a746 2024-03-29T08:05:14.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:14.241+0000 7f4a57e00700 1 -- 172.21.15.37:0/924863733 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f4a3c0692a0 msgr2=0x7f4a3c06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:05:14.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:14.241+0000 7f4a57e00700 1 --2- 172.21.15.37:0/924863733 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f4a3c0692a0 0x7f4a3c06b750 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f4a440135d0 tx=0x7f4a44009e30 comp rx=0 tx=0).stop 2024-03-29T08:05:14.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:14.241+0000 7f4a57e00700 1 -- 172.21.15.37:0/924863733 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4a580fee90 msgr2=0x7f4a5810f8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:05:14.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:14.241+0000 7f4a57e00700 1 --2- 172.21.15.37:0/924863733 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4a580fee90 0x7f4a5810f8c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f4a580ffc00 tx=0x7f4a4800b040 comp rx=0 tx=0).stop 2024-03-29T08:05:14.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:14.241+0000 7f4a57e00700 1 -- 172.21.15.37:0/924863733 shutdown_connections 2024-03-29T08:05:14.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:14.241+0000 7f4a57e00700 1 --2- 172.21.15.37:0/924863733 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f4a3c0692a0 0x7f4a3c06b750 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:05:14.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:14.241+0000 7f4a57e00700 1 --2- 172.21.15.37:0/924863733 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4a580fee90 0x7f4a5810f8c0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:05:14.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:14.241+0000 7f4a57e00700 1 --2- 172.21.15.37:0/924863733 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4a58100090 0x7f4a5810fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:05:14.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:14.241+0000 7f4a57e00700 1 -- 172.21.15.37:0/924863733 >> 172.21.15.37:0/924863733 conn(0x7f4a580fa790 msgr2=0x7f4a580fce30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:05:14.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:14.241+0000 7f4a57e00700 1 -- 172.21.15.37:0/924863733 shutdown_connections 2024-03-29T08:05:14.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:14.241+0000 7f4a57e00700 1 -- 172.21.15.37:0/924863733 wait complete. 2024-03-29T08:05:14.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:14.383+0000 7fc675e00700 1 -- 172.21.15.37:0/42884600 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc6700709a0 msgr2=0x7fc670070e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:05:14.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:14.383+0000 7fc675e00700 1 --2- 172.21.15.37:0/42884600 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc6700709a0 0x7fc670070e10 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fc65c004940 tx=0x7fc65c009e30 comp rx=0 tx=0).stop 2024-03-29T08:05:14.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:14.383+0000 7fc675e00700 1 -- 172.21.15.37:0/42884600 shutdown_connections 2024-03-29T08:05:14.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:14.383+0000 7fc675e00700 1 --2- 172.21.15.37:0/42884600 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc6700709a0 0x7fc670070e10 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:05:14.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:14.383+0000 7fc675e00700 1 --2- 172.21.15.37:0/42884600 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc670071f50 0x7fc670070460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:05:14.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:14.383+0000 7fc675e00700 1 -- 172.21.15.37:0/42884600 >> 172.21.15.37:0/42884600 conn(0x7fc6700f8560 msgr2=0x7fc6700fa970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:05:14.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:14.383+0000 7fc675e00700 1 -- 172.21.15.37:0/42884600 shutdown_connections 2024-03-29T08:05:14.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:14.383+0000 7fc675e00700 1 -- 172.21.15.37:0/42884600 wait complete. 2024-03-29T08:05:14.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:14.384+0000 7fc675e00700 1 Processor -- start 2024-03-29T08:05:14.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:14.384+0000 7fc675e00700 1 -- start start 2024-03-29T08:05:14.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:14.384+0000 7fc675e00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc6700709a0 0x7fc67010faa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:05:14.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:14.385+0000 7fc675e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc670071f50 0x7fc67010ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:05:14.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:14.385+0000 7fc675e00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6701105e0 con 0x7fc670071f50 2024-03-29T08:05:14.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:14.385+0000 7fc675e00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc670110e30 con 0x7fc6700709a0 2024-03-29T08:05:14.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:14.385+0000 7fc66e000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc670071f50 0x7fc67010ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:05:14.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:14.385+0000 7fc66ea00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc6700709a0 0x7fc67010faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:05:14.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:14.385+0000 7fc66e000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc670071f50 0x7fc67010ffe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51870/0 (socket says 172.21.15.37:51870) 2024-03-29T08:05:14.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:14.385+0000 7fc66e000700 1 -- 172.21.15.37:0/380826022 learned_addr learned my addr 172.21.15.37:0/380826022 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:05:14.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:14.385+0000 7fc66ea00700 1 -- 172.21.15.37:0/380826022 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc670071f50 msgr2=0x7fc67010ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:05:14.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:14.385+0000 7fc66ea00700 1 --2- 172.21.15.37:0/380826022 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc670071f50 0x7fc67010ffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:05:14.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:14.385+0000 7fc66ea00700 1 -- 172.21.15.37:0/380826022 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc660005040 con 0x7fc6700709a0 2024-03-29T08:05:14.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:14.385+0000 7fc66e000700 1 --2- 172.21.15.37:0/380826022 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc670071f50 0x7fc67010ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T08:05:14.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:14.386+0000 7fc66ea00700 1 --2- 172.21.15.37:0/380826022 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc6700709a0 0x7fc67010faa0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fc660009fd0 tx=0x7fc660009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:05:14.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:14.386+0000 7fc666a00700 1 -- 172.21.15.37:0/380826022 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc660012570 con 0x7fc6700709a0 2024-03-29T08:05:14.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:14.386+0000 7fc666a00700 1 -- 172.21.15.37:0/380826022 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fc66000f040 con 0x7fc6700709a0 2024-03-29T08:05:14.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:14.386+0000 7fc666a00700 1 -- 172.21.15.37:0/380826022 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc660013b60 con 0x7fc6700709a0 2024-03-29T08:05:14.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:14.386+0000 7fc675e00700 1 -- 172.21.15.37:0/380826022 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc65c005040 con 0x7fc6700709a0 2024-03-29T08:05:14.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:14.386+0000 7fc675e00700 1 -- 172.21.15.37:0/380826022 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc670111470 con 0x7fc6700709a0 2024-03-29T08:05:14.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:14.387+0000 7fc675e00700 1 -- 172.21.15.37:0/380826022 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc67005f5f0 con 0x7fc6700709a0 2024-03-29T08:05:14.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:14.388+0000 7fc666a00700 1 -- 172.21.15.37:0/380826022 <== mon.1 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fc6600126d0 con 0x7fc6700709a0 2024-03-29T08:05:14.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:14.389+0000 7fc666a00700 1 --2- 172.21.15.37:0/380826022 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7fc6540692b0 0x7fc65406b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:05:14.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:14.389+0000 7fc666a00700 1 -- 172.21.15.37:0/380826022 <== mon.1 v2:172.21.15.148:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5550+0+0 (secure 0 0 0) 0x7fc660086fc0 con 0x7fc6700709a0 2024-03-29T08:05:14.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:14.389+0000 7fc66e000700 1 --2- 172.21.15.37:0/380826022 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7fc6540692b0 0x7fc65406b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:05:14.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:14.390+0000 7fc66e000700 1 --2- 172.21.15.37:0/380826022 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7fc6540692b0 0x7fc65406b760 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fc65c013680 tx=0x7fc65c009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:05:14.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:14.397+0000 7fc666a00700 1 -- 172.21.15.37:0/380826022 <== mon.1 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fc660056c10 con 0x7fc6700709a0 2024-03-29T08:05:14.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:14.728+0000 7fc675e00700 1 -- 172.21.15.37:0/380826022 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fc670111840 con 0x7fc6700709a0 2024-03-29T08:05:14.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:14.729+0000 7fc666a00700 1 -- 172.21.15.37:0/380826022 <== mon.1 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+638 (secure 0 0 0) 0x7fc6600567a0 con 0x7fc6700709a0 2024-03-29T08:05:14.729 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-29T08:05:14.729 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": { 2024-03-29T08:05:14.729 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 17.2.7-789-g4213e17c (4213e17c4de67016ed14f55d22503f4923b3a901) quincy (stable)": 2 2024-03-29T08:05:14.729 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:05:14.729 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": { 2024-03-29T08:05:14.729 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 17.2.7-789-g4213e17c (4213e17c4de67016ed14f55d22503f4923b3a901) quincy (stable)": 2 2024-03-29T08:05:14.729 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:05:14.729 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": { 2024-03-29T08:05:14.730 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 17.2.7-789-g4213e17c (4213e17c4de67016ed14f55d22503f4923b3a901) quincy (stable)": 6 2024-03-29T08:05:14.730 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:05:14.730 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": { 2024-03-29T08:05:14.730 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 17.2.7-789-g4213e17c (4213e17c4de67016ed14f55d22503f4923b3a901) quincy (stable)": 4 2024-03-29T08:05:14.730 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:05:14.730 INFO:teuthology.orchestra.run.smithi037.stdout: "overall": { 2024-03-29T08:05:14.730 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 17.2.7-789-g4213e17c (4213e17c4de67016ed14f55d22503f4923b3a901) quincy (stable)": 14 2024-03-29T08:05:14.730 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-29T08:05:14.730 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-29T08:05:14.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:14.732+0000 7fc675e00700 1 -- 172.21.15.37:0/380826022 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7fc6540692b0 msgr2=0x7fc65406b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:05:14.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:14.732+0000 7fc675e00700 1 --2- 172.21.15.37:0/380826022 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7fc6540692b0 0x7fc65406b760 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fc65c013680 tx=0x7fc65c009e30 comp rx=0 tx=0).stop 2024-03-29T08:05:14.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:14.732+0000 7fc675e00700 1 -- 172.21.15.37:0/380826022 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc6700709a0 msgr2=0x7fc67010faa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:05:14.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:14.732+0000 7fc675e00700 1 --2- 172.21.15.37:0/380826022 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc6700709a0 0x7fc67010faa0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fc660009fd0 tx=0x7fc660009e30 comp rx=0 tx=0).stop 2024-03-29T08:05:14.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:14.732+0000 7fc675e00700 1 -- 172.21.15.37:0/380826022 shutdown_connections 2024-03-29T08:05:14.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:14.732+0000 7fc675e00700 1 --2- 172.21.15.37:0/380826022 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7fc6540692b0 0x7fc65406b760 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:05:14.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:14.732+0000 7fc675e00700 1 --2- 172.21.15.37:0/380826022 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc6700709a0 0x7fc67010faa0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:05:14.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:14.732+0000 7fc675e00700 1 --2- 172.21.15.37:0/380826022 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc670071f50 0x7fc67010ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:05:14.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:14.732+0000 7fc675e00700 1 -- 172.21.15.37:0/380826022 >> 172.21.15.37:0/380826022 conn(0x7fc6700f8560 msgr2=0x7fc6700f9c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:05:14.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:14.732+0000 7fc675e00700 1 -- 172.21.15.37:0/380826022 shutdown_connections 2024-03-29T08:05:14.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:14.732+0000 7fc675e00700 1 -- 172.21.15.37:0/380826022 wait complete. 2024-03-29T08:05:14.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:14.892+0000 7fbb88e00700 1 -- 172.21.15.37:0/1831338228 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbb840fdec0 msgr2=0x7fbb840fe330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:05:14.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:14.892+0000 7fbb88e00700 1 --2- 172.21.15.37:0/1831338228 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbb840fdec0 0x7fbb840fe330 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fbb70004940 tx=0x7fbb70009e30 comp rx=0 tx=0).stop 2024-03-29T08:05:14.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:14.892+0000 7fbb88e00700 1 -- 172.21.15.37:0/1831338228 shutdown_connections 2024-03-29T08:05:14.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:14.892+0000 7fbb88e00700 1 --2- 172.21.15.37:0/1831338228 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbb840fdec0 0x7fbb840fe330 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:05:14.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:14.892+0000 7fbb88e00700 1 --2- 172.21.15.37:0/1831338228 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb840fcc10 0x7fbb840fd020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:05:14.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:14.892+0000 7fbb88e00700 1 -- 172.21.15.37:0/1831338228 >> 172.21.15.37:0/1831338228 conn(0x7fbb840f83d0 msgr2=0x7fbb840fa7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:05:14.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:14.892+0000 7fbb88e00700 1 -- 172.21.15.37:0/1831338228 shutdown_connections 2024-03-29T08:05:14.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:14.892+0000 7fbb88e00700 1 -- 172.21.15.37:0/1831338228 wait complete. 2024-03-29T08:05:14.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:14.893+0000 7fbb88e00700 1 Processor -- start 2024-03-29T08:05:14.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:14.893+0000 7fbb88e00700 1 -- start start 2024-03-29T08:05:14.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:14.894+0000 7fbb88e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbb840fcc10 0x7fbb84071e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:05:14.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:14.894+0000 7fbb88e00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb840fdec0 0x7fbb84070310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:05:14.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:14.894+0000 7fbb88e00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb840708c0 con 0x7fbb840fcc10 2024-03-29T08:05:14.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:14.894+0000 7fbb88e00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb84070a30 con 0x7fbb840fdec0 2024-03-29T08:05:14.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:14.894+0000 7fbb83400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbb840fcc10 0x7fbb84071e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:05:14.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:14.894+0000 7fbb83400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbb840fcc10 0x7fbb84071e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51892/0 (socket says 172.21.15.37:51892) 2024-03-29T08:05:14.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:14.894+0000 7fbb83400700 1 -- 172.21.15.37:0/2753910027 learned_addr learned my addr 172.21.15.37:0/2753910027 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:05:14.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:14.894+0000 7fbb82a00700 1 --2- 172.21.15.37:0/2753910027 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb840fdec0 0x7fbb84070310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:05:14.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:14.894+0000 7fbb82a00700 1 -- 172.21.15.37:0/2753910027 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbb840fcc10 msgr2=0x7fbb84071e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:05:14.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:14.894+0000 7fbb82a00700 1 --2- 172.21.15.37:0/2753910027 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbb840fcc10 0x7fbb84071e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:05:14.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:14.894+0000 7fbb82a00700 1 -- 172.21.15.37:0/2753910027 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb70005040 con 0x7fbb840fdec0 2024-03-29T08:05:14.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:14.895+0000 7fbb82a00700 1 --2- 172.21.15.37:0/2753910027 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb840fdec0 0x7fbb84070310 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fbb7000b0d0 tx=0x7fbb70009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:05:14.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:14.895+0000 7fbb7b400700 1 -- 172.21.15.37:0/2753910027 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbb7000e070 con 0x7fbb840fdec0 2024-03-29T08:05:14.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:14.895+0000 7fbb88e00700 1 -- 172.21.15.37:0/2753910027 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb84070cb0 con 0x7fbb840fdec0 2024-03-29T08:05:14.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:14.895+0000 7fbb7b400700 1 -- 172.21.15.37:0/2753910027 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fbb7001d460 con 0x7fbb840fdec0 2024-03-29T08:05:14.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:14.895+0000 7fbb7b400700 1 -- 172.21.15.37:0/2753910027 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbb7000b670 con 0x7fbb840fdec0 2024-03-29T08:05:14.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:14.895+0000 7fbb88e00700 1 -- 172.21.15.37:0/2753910027 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb841156f0 con 0x7fbb840fdec0 2024-03-29T08:05:14.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:14.896+0000 7fbb88e00700 1 -- 172.21.15.37:0/2753910027 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb8405f5f0 con 0x7fbb840fdec0 2024-03-29T08:05:14.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:14.897+0000 7fbb7b400700 1 -- 172.21.15.37:0/2753910027 <== mon.1 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fbb7001da60 con 0x7fbb840fdec0 2024-03-29T08:05:14.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:14.898+0000 7fbb7b400700 1 --2- 172.21.15.37:0/2753910027 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7fbb680692b0 0x7fbb6806b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:05:14.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:14.898+0000 7fbb7b400700 1 -- 172.21.15.37:0/2753910027 <== mon.1 v2:172.21.15.148:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5550+0+0 (secure 0 0 0) 0x7fbb70088300 con 0x7fbb840fdec0 2024-03-29T08:05:14.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:14.898+0000 7fbb83400700 1 --2- 172.21.15.37:0/2753910027 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7fbb680692b0 0x7fbb6806b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:05:14.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:14.899+0000 7fbb83400700 1 --2- 172.21.15.37:0/2753910027 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7fbb680692b0 0x7fbb6806b760 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fbb74008fd0 tx=0x7fbb7400b040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:05:14.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:14.902+0000 7fbb7b400700 1 -- 172.21.15.37:0/2753910027 <== mon.1 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fbb70054540 con 0x7fbb840fdec0 2024-03-29T08:05:15.055 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:05:14 smithi148 ceph-mon[44992]: from='client.14562 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:05:15.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:05:14 smithi148 ceph-mon[44992]: pgmap v254: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:05:15.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:05:14 smithi148 ceph-mon[44992]: from='client.24393 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:05:15.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:05:14 smithi148 ceph-mon[44992]: from='client.? 172.21.15.37:0/380826022' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:05:15.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:15.146+0000 7fbb88e00700 1 -- 172.21.15.37:0/2753910027 --> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fbb84102250 con 0x7fbb680692b0 2024-03-29T08:05:15.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:15.147+0000 7fbb7b400700 1 -- 172.21.15.37:0/2753910027 <== mgr.14208 v2:172.21.15.37:6800/1802836043 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+439 (secure 0 0 0) 0x7fbb84102250 con 0x7fbb680692b0 2024-03-29T08:05:15.147 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-29T08:05:15.147 INFO:teuthology.orchestra.run.smithi037.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59", 2024-03-29T08:05:15.147 INFO:teuthology.orchestra.run.smithi037.stdout: "in_progress": true, 2024-03-29T08:05:15.147 INFO:teuthology.orchestra.run.smithi037.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-03-29T08:05:15.147 INFO:teuthology.orchestra.run.smithi037.stdout: "services_complete": [], 2024-03-29T08:05:15.147 INFO:teuthology.orchestra.run.smithi037.stdout: "progress": "0/2 daemons upgraded", 2024-03-29T08:05:15.147 INFO:teuthology.orchestra.run.smithi037.stdout: "message": "Pulling quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59 image on host smithi148", 2024-03-29T08:05:15.147 INFO:teuthology.orchestra.run.smithi037.stdout: "is_paused": false 2024-03-29T08:05:15.147 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-29T08:05:15.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:15.149+0000 7fbb88e00700 1 -- 172.21.15.37:0/2753910027 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7fbb680692b0 msgr2=0x7fbb6806b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:05:15.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:15.149+0000 7fbb88e00700 1 --2- 172.21.15.37:0/2753910027 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7fbb680692b0 0x7fbb6806b760 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fbb74008fd0 tx=0x7fbb7400b040 comp rx=0 tx=0).stop 2024-03-29T08:05:15.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:15.149+0000 7fbb88e00700 1 -- 172.21.15.37:0/2753910027 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb840fdec0 msgr2=0x7fbb84070310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:05:15.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:15.149+0000 7fbb88e00700 1 --2- 172.21.15.37:0/2753910027 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb840fdec0 0x7fbb84070310 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fbb7000b0d0 tx=0x7fbb70009e30 comp rx=0 tx=0).stop 2024-03-29T08:05:15.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:15.150+0000 7fbb88e00700 1 -- 172.21.15.37:0/2753910027 shutdown_connections 2024-03-29T08:05:15.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:15.150+0000 7fbb88e00700 1 --2- 172.21.15.37:0/2753910027 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7fbb680692b0 0x7fbb6806b760 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:05:15.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:15.150+0000 7fbb88e00700 1 --2- 172.21.15.37:0/2753910027 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbb840fcc10 0x7fbb84071e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:05:15.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:15.150+0000 7fbb88e00700 1 --2- 172.21.15.37:0/2753910027 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb840fdec0 0x7fbb84070310 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:05:15.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:15.150+0000 7fbb88e00700 1 -- 172.21.15.37:0/2753910027 >> 172.21.15.37:0/2753910027 conn(0x7fbb840f83d0 msgr2=0x7fbb840f9af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:05:15.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:15.150+0000 7fbb88e00700 1 -- 172.21.15.37:0/2753910027 shutdown_connections 2024-03-29T08:05:15.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:15.150+0000 7fbb88e00700 1 -- 172.21.15.37:0/2753910027 wait complete. 2024-03-29T08:05:15.176 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:05:14 smithi037 ceph-mon[35162]: from='client.14562 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:05:15.176 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:05:14 smithi037 ceph-mon[35162]: pgmap v254: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:05:15.176 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:05:14 smithi037 ceph-mon[35162]: from='client.24393 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:05:15.176 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:05:14 smithi037 ceph-mon[35162]: from='client.? 172.21.15.37:0/380826022' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:05:16.055 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:05:15 smithi148 ceph-mon[44992]: from='client.24397 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:05:16.176 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:05:15 smithi037 ceph-mon[35162]: from='client.24397 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:05:17.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:05:16 smithi148 ceph-mon[44992]: from='client.24405 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:05:17.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:05:16 smithi148 ceph-mon[44992]: pgmap v255: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:05:17.176 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:05:16 smithi037 ceph-mon[35162]: from='client.24405 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:05:17.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:05:16 smithi037 ceph-mon[35162]: pgmap v255: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:05:19.176 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:05:18 smithi037 ceph-mon[35162]: pgmap v256: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:05:19.305 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:05:18 smithi148 ceph-mon[44992]: pgmap v256: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:05:21.176 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:05:20 smithi037 ceph-mon[35162]: pgmap v257: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:05:21.305 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:05:20 smithi148 ceph-mon[44992]: pgmap v257: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:05:23.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:05:22 smithi037 ceph-mon[35162]: pgmap v258: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:05:23.305 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:05:22 smithi148 ceph-mon[44992]: pgmap v258: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:05:25.176 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:05:24 smithi037 ceph-mon[35162]: pgmap v259: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:05:25.305 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:05:24 smithi148 ceph-mon[44992]: pgmap v259: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:05:26.176 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:05:25 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:05:26.305 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:05:25 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:05:27.176 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:05:26 smithi037 ceph-mon[35162]: pgmap v260: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:05:27.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:05:26 smithi148 ceph-mon[44992]: pgmap v260: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:05:29.176 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:05:28 smithi037 ceph-mon[35162]: pgmap v261: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:05:29.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:05:28 smithi148 ceph-mon[44992]: pgmap v261: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:05:31.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:05:30 smithi037 ceph-mon[35162]: pgmap v262: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:05:31.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:05:30 smithi148 ceph-mon[44992]: pgmap v262: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:05:33.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:05:32 smithi037 ceph-mon[35162]: pgmap v263: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:05:33.305 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:05:32 smithi148 ceph-mon[44992]: pgmap v263: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:05:35.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:05:34 smithi037 ceph-mon[35162]: pgmap v264: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:05:35.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:05:34 smithi148 ceph-mon[44992]: pgmap v264: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:05:37.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:05:36 smithi037 ceph-mon[35162]: pgmap v265: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:05:37.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:05:36 smithi148 ceph-mon[44992]: pgmap v265: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:05:39.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:05:38 smithi037 ceph-mon[35162]: pgmap v266: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:05:39.305 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:05:38 smithi148 ceph-mon[44992]: pgmap v266: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:05:41.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:05:40 smithi037 ceph-mon[35162]: pgmap v267: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:05:41.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:05:40 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:05:41.305 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:05:40 smithi148 ceph-mon[44992]: pgmap v267: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:05:41.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:05:40 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:05:43.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:05:42 smithi037 ceph-mon[35162]: pgmap v268: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:05:43.206 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:05:42 smithi148 ceph-mon[44992]: pgmap v268: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:05:44.845 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:05:44 smithi148 ceph-mon[44992]: pgmap v269: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:05:44.845 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:05:44 smithi148 ceph-mon[44992]: Upgrade: Updating mgr.smithi148.onhgvm 2024-03-29T08:05:44.845 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:05:44 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:05:44.845 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:05:44 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi148.onhgvm", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-29T08:05:44.846 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:05:44 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-29T08:05:44.846 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:05:44 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T08:05:44.846 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:05:44 smithi148 ceph-mon[44992]: Deploying daemon mgr.smithi148.onhgvm on smithi148 2024-03-29T08:05:45.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:05:44 smithi037 ceph-mon[35162]: pgmap v269: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:05:45.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:05:44 smithi037 ceph-mon[35162]: Upgrade: Updating mgr.smithi148.onhgvm 2024-03-29T08:05:45.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:05:44 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:05:45.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:05:44 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi148.onhgvm", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-29T08:05:45.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:05:44 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-29T08:05:45.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:05:44 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T08:05:45.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:05:44 smithi037 ceph-mon[35162]: Deploying daemon mgr.smithi148.onhgvm on smithi148 2024-03-29T08:05:45.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:45.367+0000 7f0afa400700 1 -- 172.21.15.37:0/3422774205 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0af40fcee0 msgr2=0x7f0af40fd310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:05:45.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:45.367+0000 7f0afa400700 1 --2- 172.21.15.37:0/3422774205 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0af40fcee0 0x7f0af40fd310 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f0ae4004970 tx=0x7f0ae4009e30 comp rx=0 tx=0).stop 2024-03-29T08:05:45.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:45.367+0000 7f0afa400700 1 -- 172.21.15.37:0/3422774205 shutdown_connections 2024-03-29T08:05:45.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:45.367+0000 7f0afa400700 1 --2- 172.21.15.37:0/3422774205 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0af40fe0d0 0x7f0af40fe540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:05:45.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:45.367+0000 7f0afa400700 1 --2- 172.21.15.37:0/3422774205 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0af40fcee0 0x7f0af40fd310 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:05:45.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:45.367+0000 7f0afa400700 1 -- 172.21.15.37:0/3422774205 >> 172.21.15.37:0/3422774205 conn(0x7f0af40f8560 msgr2=0x7f0af40fa970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:05:45.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:45.368+0000 7f0afa400700 1 -- 172.21.15.37:0/3422774205 shutdown_connections 2024-03-29T08:05:45.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:45.368+0000 7f0afa400700 1 -- 172.21.15.37:0/3422774205 wait complete. 2024-03-29T08:05:45.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:45.369+0000 7f0afa400700 1 Processor -- start 2024-03-29T08:05:45.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:45.370+0000 7f0afa400700 1 -- start start 2024-03-29T08:05:45.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:45.370+0000 7f0afa400700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0af40fcee0 0x7f0af410d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:05:45.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:45.370+0000 7f0afa400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0af40fe0d0 0x7f0af410de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:05:45.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:45.370+0000 7f0afa400700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0af410e430 con 0x7f0af40fe0d0 2024-03-29T08:05:45.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:45.370+0000 7f0afa400700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0af410ec80 con 0x7f0af40fcee0 2024-03-29T08:05:45.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:45.371+0000 7f0af3400700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0af40fcee0 0x7f0af410d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:05:45.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:45.371+0000 7f0af2a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0af40fe0d0 0x7f0af410de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:05:45.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:45.371+0000 7f0af3400700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0af40fcee0 0x7f0af410d8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.37:58168/0 (socket says 172.21.15.37:58168) 2024-03-29T08:05:45.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:45.371+0000 7f0af3400700 1 -- 172.21.15.37:0/4182122233 learned_addr learned my addr 172.21.15.37:0/4182122233 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:05:45.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:45.371+0000 7f0af3400700 1 -- 172.21.15.37:0/4182122233 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0af40fe0d0 msgr2=0x7f0af410de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:05:45.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:45.371+0000 7f0af3400700 1 --2- 172.21.15.37:0/4182122233 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0af40fe0d0 0x7f0af410de30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:05:45.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:45.371+0000 7f0af3400700 1 -- 172.21.15.37:0/4182122233 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ae4005040 con 0x7f0af40fcee0 2024-03-29T08:05:45.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:45.371+0000 7f0af3400700 1 --2- 172.21.15.37:0/4182122233 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0af40fcee0 0x7f0af410d8f0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f0ae400b370 tx=0x7f0ae4009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:05:45.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:45.372+0000 7f0aeb400700 1 -- 172.21.15.37:0/4182122233 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0ae400e040 con 0x7f0af40fcee0 2024-03-29T08:05:45.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:45.372+0000 7f0aeb400700 1 -- 172.21.15.37:0/4182122233 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f0ae4013420 con 0x7f0af40fcee0 2024-03-29T08:05:45.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:45.372+0000 7f0afa400700 1 -- 172.21.15.37:0/4182122233 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0af410ef00 con 0x7f0af40fcee0 2024-03-29T08:05:45.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:45.372+0000 7f0aeb400700 1 -- 172.21.15.37:0/4182122233 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0ae401c650 con 0x7f0af40fcee0 2024-03-29T08:05:45.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:45.372+0000 7f0afa400700 1 -- 172.21.15.37:0/4182122233 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0af410f440 con 0x7f0af40fcee0 2024-03-29T08:05:45.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:45.374+0000 7f0afa400700 1 -- 172.21.15.37:0/4182122233 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0af405f5f0 con 0x7f0af40fcee0 2024-03-29T08:05:45.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:45.375+0000 7f0aeb400700 1 -- 172.21.15.37:0/4182122233 <== mon.1 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f0ae40139e0 con 0x7f0af40fcee0 2024-03-29T08:05:45.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:45.377+0000 7f0aeb400700 1 --2- 172.21.15.37:0/4182122233 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f0ad80692b0 0x7f0ad806b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:05:45.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:45.377+0000 7f0aeb400700 1 -- 172.21.15.37:0/4182122233 <== mon.1 v2:172.21.15.148:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f0ae4088430 con 0x7f0af40fcee0 2024-03-29T08:05:45.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:45.377+0000 7f0af2a00700 1 --2- 172.21.15.37:0/4182122233 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f0ad80692b0 0x7f0ad806b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:05:45.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:45.378+0000 7f0af2a00700 1 --2- 172.21.15.37:0/4182122233 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f0ad80692b0 0x7f0ad806b760 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f0ae0009b80 tx=0x7f0ae000b040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:05:45.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:45.385+0000 7f0aeb400700 1 -- 172.21.15.37:0/4182122233 <== mon.1 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f0ae4058180 con 0x7f0af40fcee0 2024-03-29T08:05:45.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:45.684+0000 7f0afa400700 1 -- 172.21.15.37:0/4182122233 --> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0af410f810 con 0x7f0ad80692b0 2024-03-29T08:05:45.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:45.686+0000 7f0aeb400700 1 -- 172.21.15.37:0/4182122233 <== mgr.14208 v2:172.21.15.37:6800/1802836043 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+341 (secure 0 0 0) 0x7f0af410f810 con 0x7f0ad80692b0 2024-03-29T08:05:45.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:45.688+0000 7f0afa400700 1 -- 172.21.15.37:0/4182122233 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f0ad80692b0 msgr2=0x7f0ad806b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:05:45.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:45.688+0000 7f0afa400700 1 --2- 172.21.15.37:0/4182122233 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f0ad80692b0 0x7f0ad806b760 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f0ae0009b80 tx=0x7f0ae000b040 comp rx=0 tx=0).stop 2024-03-29T08:05:45.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:45.688+0000 7f0afa400700 1 -- 172.21.15.37:0/4182122233 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0af40fcee0 msgr2=0x7f0af410d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:05:45.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:45.688+0000 7f0afa400700 1 --2- 172.21.15.37:0/4182122233 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0af40fcee0 0x7f0af410d8f0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f0ae400b370 tx=0x7f0ae4009e30 comp rx=0 tx=0).stop 2024-03-29T08:05:45.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:45.688+0000 7f0afa400700 1 -- 172.21.15.37:0/4182122233 shutdown_connections 2024-03-29T08:05:45.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:45.688+0000 7f0afa400700 1 --2- 172.21.15.37:0/4182122233 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f0ad80692b0 0x7f0ad806b760 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:05:45.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:45.688+0000 7f0afa400700 1 --2- 172.21.15.37:0/4182122233 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0af40fcee0 0x7f0af410d8f0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:05:45.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:45.688+0000 7f0afa400700 1 --2- 172.21.15.37:0/4182122233 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0af40fe0d0 0x7f0af410de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:05:45.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:45.688+0000 7f0afa400700 1 -- 172.21.15.37:0/4182122233 >> 172.21.15.37:0/4182122233 conn(0x7f0af40f8560 msgr2=0x7f0af40f9ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:05:45.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:45.688+0000 7f0afa400700 1 -- 172.21.15.37:0/4182122233 shutdown_connections 2024-03-29T08:05:45.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:45.689+0000 7f0afa400700 1 -- 172.21.15.37:0/4182122233 wait complete. 2024-03-29T08:05:45.704 INFO:teuthology.orchestra.run.smithi037.stdout:true 2024-03-29T08:05:45.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:45.829+0000 7fa844600700 1 -- 172.21.15.37:0/4153852572 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa83c0662e0 msgr2=0x7fa83c0ff9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:05:45.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:45.829+0000 7fa844600700 1 --2- 172.21.15.37:0/4153852572 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa83c0662e0 0x7fa83c0ff9c0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fa830004940 tx=0x7fa830009e30 comp rx=0 tx=0).stop 2024-03-29T08:05:45.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:45.830+0000 7fa844600700 1 -- 172.21.15.37:0/4153852572 shutdown_connections 2024-03-29T08:05:45.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:45.830+0000 7fa844600700 1 --2- 172.21.15.37:0/4153852572 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa83c0fff00 0x7fa83c102320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:05:45.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:45.830+0000 7fa844600700 1 --2- 172.21.15.37:0/4153852572 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa83c0662e0 0x7fa83c0ff9c0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:05:45.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:45.830+0000 7fa844600700 1 -- 172.21.15.37:0/4153852572 >> 172.21.15.37:0/4153852572 conn(0x7fa83c0f7430 msgr2=0x7fa83c0f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:05:45.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:45.830+0000 7fa844600700 1 -- 172.21.15.37:0/4153852572 shutdown_connections 2024-03-29T08:05:45.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:45.830+0000 7fa844600700 1 -- 172.21.15.37:0/4153852572 wait complete. 2024-03-29T08:05:45.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:45.830+0000 7fa844600700 1 Processor -- start 2024-03-29T08:05:45.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:45.831+0000 7fa844600700 1 -- start start 2024-03-29T08:05:45.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:45.831+0000 7fa844600700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa83c0662e0 0x7fa83c0fd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:05:45.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:45.831+0000 7fa844600700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa83c0fff00 0x7fa83c0fb8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:05:45.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:45.831+0000 7fa844600700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa83c0fbe70 con 0x7fa83c0662e0 2024-03-29T08:05:45.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:45.831+0000 7fa844600700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa83c0fbfe0 con 0x7fa83c0fff00 2024-03-29T08:05:45.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:45.831+0000 7fa841c00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa83c0662e0 0x7fa83c0fd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:05:45.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:45.831+0000 7fa841c00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa83c0662e0 0x7fa83c0fd3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60654/0 (socket says 172.21.15.37:60654) 2024-03-29T08:05:45.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:45.831+0000 7fa841c00700 1 -- 172.21.15.37:0/3709756972 learned_addr learned my addr 172.21.15.37:0/3709756972 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:05:45.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:45.831+0000 7fa841200700 1 --2- 172.21.15.37:0/3709756972 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa83c0fff00 0x7fa83c0fb8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:05:45.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:45.832+0000 7fa841c00700 1 -- 172.21.15.37:0/3709756972 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa83c0fff00 msgr2=0x7fa83c0fb8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:05:45.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:45.832+0000 7fa841c00700 1 --2- 172.21.15.37:0/3709756972 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa83c0fff00 0x7fa83c0fb8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:05:45.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:45.832+0000 7fa841c00700 1 -- 172.21.15.37:0/3709756972 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa830005040 con 0x7fa83c0662e0 2024-03-29T08:05:45.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:45.832+0000 7fa841c00700 1 --2- 172.21.15.37:0/3709756972 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa83c0662e0 0x7fa83c0fd3f0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fa83000b900 tx=0x7fa830009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:05:45.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:45.832+0000 7fa839600700 1 -- 172.21.15.37:0/3709756972 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa83000e040 con 0x7fa83c0662e0 2024-03-29T08:05:45.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:45.832+0000 7fa839600700 1 -- 172.21.15.37:0/3709756972 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fa830004290 con 0x7fa83c0662e0 2024-03-29T08:05:45.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:45.832+0000 7fa844600700 1 -- 172.21.15.37:0/3709756972 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa83c0fc260 con 0x7fa83c0662e0 2024-03-29T08:05:45.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:45.832+0000 7fa839600700 1 -- 172.21.15.37:0/3709756972 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa83001c3f0 con 0x7fa83c0662e0 2024-03-29T08:05:45.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:45.832+0000 7fa844600700 1 -- 172.21.15.37:0/3709756972 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa83c1159a0 con 0x7fa83c0662e0 2024-03-29T08:05:45.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:45.834+0000 7fa844600700 1 -- 172.21.15.37:0/3709756972 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa83c05f5f0 con 0x7fa83c0662e0 2024-03-29T08:05:45.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:45.834+0000 7fa839600700 1 -- 172.21.15.37:0/3709756972 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fa830013420 con 0x7fa83c0662e0 2024-03-29T08:05:45.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:45.836+0000 7fa839600700 1 --2- 172.21.15.37:0/3709756972 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7fa8240692b0 0x7fa82406b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:05:45.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:45.836+0000 7fa839600700 1 -- 172.21.15.37:0/3709756972 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5550+0+0 (secure 0 0 0) 0x7fa830088540 con 0x7fa83c0662e0 2024-03-29T08:05:45.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:45.836+0000 7fa841200700 1 --2- 172.21.15.37:0/3709756972 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7fa8240692b0 0x7fa82406b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:05:45.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:45.837+0000 7fa841200700 1 --2- 172.21.15.37:0/3709756972 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7fa8240692b0 0x7fa82406b760 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fa828007fd0 tx=0x7fa828009040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:05:45.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:45.840+0000 7fa839600700 1 -- 172.21.15.37:0/3709756972 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fa830054820 con 0x7fa83c0662e0 2024-03-29T08:05:46.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:46.078+0000 7fa844600700 1 -- 172.21.15.37:0/3709756972 --> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa83c0f8f70 con 0x7fa8240692b0 2024-03-29T08:05:46.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:46.079+0000 7fa839600700 1 -- 172.21.15.37:0/3709756972 <== mgr.14208 v2:172.21.15.37:6800/1802836043 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+341 (secure 0 0 0) 0x7fa83c0f8f70 con 0x7fa8240692b0 2024-03-29T08:05:46.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:46.082+0000 7fa844600700 1 -- 172.21.15.37:0/3709756972 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7fa8240692b0 msgr2=0x7fa82406b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:05:46.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:46.082+0000 7fa844600700 1 --2- 172.21.15.37:0/3709756972 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7fa8240692b0 0x7fa82406b760 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fa828007fd0 tx=0x7fa828009040 comp rx=0 tx=0).stop 2024-03-29T08:05:46.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:46.082+0000 7fa844600700 1 -- 172.21.15.37:0/3709756972 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa83c0662e0 msgr2=0x7fa83c0fd3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:05:46.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:46.082+0000 7fa844600700 1 --2- 172.21.15.37:0/3709756972 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa83c0662e0 0x7fa83c0fd3f0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fa83000b900 tx=0x7fa830009e30 comp rx=0 tx=0).stop 2024-03-29T08:05:46.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:46.082+0000 7fa844600700 1 -- 172.21.15.37:0/3709756972 shutdown_connections 2024-03-29T08:05:46.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:46.082+0000 7fa844600700 1 --2- 172.21.15.37:0/3709756972 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7fa8240692b0 0x7fa82406b760 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:05:46.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:46.082+0000 7fa844600700 1 --2- 172.21.15.37:0/3709756972 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa83c0662e0 0x7fa83c0fd3f0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:05:46.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:46.082+0000 7fa844600700 1 --2- 172.21.15.37:0/3709756972 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa83c0fff00 0x7fa83c0fb8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:05:46.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:46.082+0000 7fa844600700 1 -- 172.21.15.37:0/3709756972 >> 172.21.15.37:0/3709756972 conn(0x7fa83c0f7430 msgr2=0x7fa83c0f7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:05:46.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:46.082+0000 7fa844600700 1 -- 172.21.15.37:0/3709756972 shutdown_connections 2024-03-29T08:05:46.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:46.082+0000 7fa844600700 1 -- 172.21.15.37:0/3709756972 wait complete. 2024-03-29T08:05:46.116 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:05:46 smithi148 ceph-mon[44992]: pgmap v270: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:05:46.116 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:05:46 smithi148 ceph-mon[44992]: from='client.24409 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:05:46.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:46.222+0000 7f041ca00700 1 -- 172.21.15.37:0/2719492596 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f04180709a0 msgr2=0x7f0418070e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:05:46.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:46.222+0000 7f041ca00700 1 --2- 172.21.15.37:0/2719492596 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f04180709a0 0x7f0418070e10 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f0404004940 tx=0x7f0404009e30 comp rx=0 tx=0).stop 2024-03-29T08:05:46.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:46.222+0000 7f041ca00700 1 -- 172.21.15.37:0/2719492596 shutdown_connections 2024-03-29T08:05:46.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:46.222+0000 7f041ca00700 1 --2- 172.21.15.37:0/2719492596 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f04180709a0 0x7f0418070e10 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:05:46.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:46.222+0000 7f041ca00700 1 --2- 172.21.15.37:0/2719492596 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0418071f50 0x7f0418070460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:05:46.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:46.222+0000 7f041ca00700 1 -- 172.21.15.37:0/2719492596 >> 172.21.15.37:0/2719492596 conn(0x7f04180f8560 msgr2=0x7f04180fa970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:05:46.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:46.222+0000 7f041ca00700 1 -- 172.21.15.37:0/2719492596 shutdown_connections 2024-03-29T08:05:46.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:46.222+0000 7f041ca00700 1 -- 172.21.15.37:0/2719492596 wait complete. 2024-03-29T08:05:46.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:46.223+0000 7f041ca00700 1 Processor -- start 2024-03-29T08:05:46.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:46.223+0000 7f041ca00700 1 -- start start 2024-03-29T08:05:46.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:46.224+0000 7f041ca00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f04180709a0 0x7f0418113f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:05:46.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:46.224+0000 7f041ca00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0418071f50 0x7f0418114460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:05:46.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:46.224+0000 7f041ca00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0418114a60 con 0x7f0418071f50 2024-03-29T08:05:46.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:46.224+0000 7f041ca00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04181152b0 con 0x7f04180709a0 2024-03-29T08:05:46.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:46.224+0000 7f0415400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0418071f50 0x7f0418114460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:05:46.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:46.224+0000 7f0415400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0418071f50 0x7f0418114460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60674/0 (socket says 172.21.15.37:60674) 2024-03-29T08:05:46.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:46.224+0000 7f0415400700 1 -- 172.21.15.37:0/583322058 learned_addr learned my addr 172.21.15.37:0/583322058 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:05:46.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:46.224+0000 7f0415400700 1 -- 172.21.15.37:0/583322058 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f04180709a0 msgr2=0x7f0418113f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:05:46.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:46.224+0000 7f0415400700 1 --2- 172.21.15.37:0/583322058 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f04180709a0 0x7f0418113f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:05:46.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:46.224+0000 7f0415400700 1 -- 172.21.15.37:0/583322058 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0404005040 con 0x7f0418071f50 2024-03-29T08:05:46.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:46.225+0000 7f0415400700 1 --2- 172.21.15.37:0/583322058 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0418071f50 0x7f0418114460 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f040400b3b0 tx=0x7f0404009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:05:46.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:46.225+0000 7f040e000700 1 -- 172.21.15.37:0/583322058 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f040400e040 con 0x7f0418071f50 2024-03-29T08:05:46.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:46.225+0000 7f040e000700 1 -- 172.21.15.37:0/583322058 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f0404004290 con 0x7f0418071f50 2024-03-29T08:05:46.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:46.225+0000 7f041ca00700 1 -- 172.21.15.37:0/583322058 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0418115530 con 0x7f0418071f50 2024-03-29T08:05:46.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:46.225+0000 7f040e000700 1 -- 172.21.15.37:0/583322058 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f040401c3f0 con 0x7f0418071f50 2024-03-29T08:05:46.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:46.225+0000 7f041ca00700 1 -- 172.21.15.37:0/583322058 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0418115a70 con 0x7f0418071f50 2024-03-29T08:05:46.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:46.226+0000 7f041ca00700 1 -- 172.21.15.37:0/583322058 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f041805f5f0 con 0x7f0418071f50 2024-03-29T08:05:46.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:46.227+0000 7f040e000700 1 -- 172.21.15.37:0/583322058 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f0404013420 con 0x7f0418071f50 2024-03-29T08:05:46.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:46.228+0000 7f040e000700 1 --2- 172.21.15.37:0/583322058 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f03fc0692b0 0x7f03fc06b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:05:46.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:46.228+0000 7f040e000700 1 -- 172.21.15.37:0/583322058 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f0404088550 con 0x7f0418071f50 2024-03-29T08:05:46.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:46.228+0000 7f0415e00700 1 --2- 172.21.15.37:0/583322058 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f03fc0692b0 0x7f03fc06b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:05:46.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:46.229+0000 7f0415e00700 1 --2- 172.21.15.37:0/583322058 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f03fc0692b0 0x7f03fc06b760 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f0418071c90 tx=0x7f0408009ce0 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:05:46.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:46.233+0000 7f040e000700 1 -- 172.21.15.37:0/583322058 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f04040582a0 con 0x7f0418071f50 2024-03-29T08:05:46.426 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:05:46 smithi037 ceph-mon[35162]: pgmap v270: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:05:46.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:05:46 smithi037 ceph-mon[35162]: from='client.24409 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:05:46.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:46.451+0000 7f041ca00700 1 -- 172.21.15.37:0/583322058 --> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f0418115f00 con 0x7f03fc0692b0 2024-03-29T08:05:46.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:46.467+0000 7f040e000700 1 -- 172.21.15.37:0/583322058 <== mgr.14208 v2:172.21.15.37:6800/1802836043 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3366 (secure 0 0 0) 0x7f0418115f00 con 0x7f03fc0692b0 2024-03-29T08:05:46.468 INFO:teuthology.orchestra.run.smithi037.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-29T08:05:46.469 INFO:teuthology.orchestra.run.smithi037.stdout:alertmanager.smithi037 smithi037 *:9093,9094 running (6m) 103s ago 8m 13.0M - 0.25.0 c8568f914cd2 a6824c801465 2024-03-29T08:05:46.469 INFO:teuthology.orchestra.run.smithi037.stdout:crash.smithi037 smithi037 running (8m) 103s ago 8m 6651k - 17.2.7-789-g4213e17c cf22eb8683be e0343935b25e 2024-03-29T08:05:46.469 INFO:teuthology.orchestra.run.smithi037.stdout:crash.smithi148 smithi148 running (7m) 109s ago 7m 6656k - 17.2.7-789-g4213e17c cf22eb8683be ff5164c552c1 2024-03-29T08:05:46.469 INFO:teuthology.orchestra.run.smithi037.stdout:grafana.smithi037 smithi037 *:3000 running (6m) 103s ago 8m 73.8M - 9.4.7 954c08fa6188 92b77c060e7a 2024-03-29T08:05:46.469 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi037.iifsmh smithi037 running (2m) 103s ago 2m 12.6M - 17.2.7-789-g4213e17c cf22eb8683be 074e1acf331f 2024-03-29T08:05:46.469 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi037.mmwgxe smithi037 running (2m) 103s ago 2m 13.4M - 17.2.7-789-g4213e17c cf22eb8683be 9a5b1e72de5c 2024-03-29T08:05:46.469 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi148.qukpjh smithi148 running (2m) 109s ago 2m 10.5M - 17.2.7-789-g4213e17c cf22eb8683be 8e469782a273 2024-03-29T08:05:46.469 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi148.sqteih smithi148 running (2m) 109s ago 2m 10.4M - 17.2.7-789-g4213e17c cf22eb8683be 8e68f858bb22 2024-03-29T08:05:46.469 INFO:teuthology.orchestra.run.smithi037.stdout:mgr.smithi037.wjikii smithi037 *:9283 running (10m) 103s ago 10m 480M - 17.2.7-789-g4213e17c cf22eb8683be 60172b53efa2 2024-03-29T08:05:46.469 INFO:teuthology.orchestra.run.smithi037.stdout:mgr.smithi148.onhgvm smithi148 *:8443,9283 running (7m) 109s ago 7m 431M - 17.2.7-789-g4213e17c cf22eb8683be a0fe1c4dd2b0 2024-03-29T08:05:46.469 INFO:teuthology.orchestra.run.smithi037.stdout:mon.smithi037 smithi037 running (10m) 103s ago 10m 44.9M 2048M 17.2.7-789-g4213e17c cf22eb8683be bb2101505730 2024-03-29T08:05:46.469 INFO:teuthology.orchestra.run.smithi037.stdout:mon.smithi148 smithi148 running (7m) 109s ago 7m 34.4M 2048M 17.2.7-789-g4213e17c cf22eb8683be 18839ed2952a 2024-03-29T08:05:46.469 INFO:teuthology.orchestra.run.smithi037.stdout:node-exporter.smithi037 smithi037 *:9100 running (8m) 103s ago 8m 7600k - 1.5.0 0da6a335fe13 eb3242f9dc57 2024-03-29T08:05:46.469 INFO:teuthology.orchestra.run.smithi037.stdout:node-exporter.smithi148 smithi148 *:9100 running (7m) 109s ago 7m 7814k - 1.5.0 0da6a335fe13 bb455992ed53 2024-03-29T08:05:46.470 INFO:teuthology.orchestra.run.smithi037.stdout:osd.0 smithi037 running (6m) 103s ago 6m 44.7M 4350M 17.2.7-789-g4213e17c cf22eb8683be 559fccd0ebe1 2024-03-29T08:05:46.470 INFO:teuthology.orchestra.run.smithi037.stdout:osd.1 smithi037 running (5m) 103s ago 5m 45.4M 4350M 17.2.7-789-g4213e17c cf22eb8683be 5582cff11b7a 2024-03-29T08:05:46.470 INFO:teuthology.orchestra.run.smithi037.stdout:osd.2 smithi037 running (5m) 103s ago 5m 42.8M 4350M 17.2.7-789-g4213e17c cf22eb8683be f809bfab13e5 2024-03-29T08:05:46.470 INFO:teuthology.orchestra.run.smithi037.stdout:osd.3 smithi148 running (4m) 109s ago 4m 42.8M 5372M 17.2.7-789-g4213e17c cf22eb8683be a0a8224bb4b0 2024-03-29T08:05:46.470 INFO:teuthology.orchestra.run.smithi037.stdout:osd.4 smithi148 running (4m) 109s ago 4m 41.0M 5372M 17.2.7-789-g4213e17c cf22eb8683be a00b69047ed7 2024-03-29T08:05:46.470 INFO:teuthology.orchestra.run.smithi037.stdout:osd.5 smithi148 running (3m) 109s ago 3m 42.8M 5372M 17.2.7-789-g4213e17c cf22eb8683be 044a3f7db7f8 2024-03-29T08:05:46.470 INFO:teuthology.orchestra.run.smithi037.stdout:prometheus.smithi037 smithi037 *:9095 running (6m) 103s ago 8m 51.7M - 2.43.0 a07b618ecd1d e8c23666a746 2024-03-29T08:05:46.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:46.470+0000 7f041ca00700 1 -- 172.21.15.37:0/583322058 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f03fc0692b0 msgr2=0x7f03fc06b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:05:46.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:46.470+0000 7f041ca00700 1 --2- 172.21.15.37:0/583322058 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f03fc0692b0 0x7f03fc06b760 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f0418071c90 tx=0x7f0408009ce0 comp rx=0 tx=0).stop 2024-03-29T08:05:46.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:46.470+0000 7f041ca00700 1 -- 172.21.15.37:0/583322058 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0418071f50 msgr2=0x7f0418114460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:05:46.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:46.470+0000 7f041ca00700 1 --2- 172.21.15.37:0/583322058 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0418071f50 0x7f0418114460 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f040400b3b0 tx=0x7f0404009e30 comp rx=0 tx=0).stop 2024-03-29T08:05:46.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:46.470+0000 7f041ca00700 1 -- 172.21.15.37:0/583322058 shutdown_connections 2024-03-29T08:05:46.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:46.470+0000 7f041ca00700 1 --2- 172.21.15.37:0/583322058 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f03fc0692b0 0x7f03fc06b760 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:05:46.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:46.470+0000 7f041ca00700 1 --2- 172.21.15.37:0/583322058 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f04180709a0 0x7f0418113f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:05:46.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:46.470+0000 7f041ca00700 1 --2- 172.21.15.37:0/583322058 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0418071f50 0x7f0418114460 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:05:46.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:46.470+0000 7f041ca00700 1 -- 172.21.15.37:0/583322058 >> 172.21.15.37:0/583322058 conn(0x7f04180f8560 msgr2=0x7f04180f9ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:05:46.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:46.470+0000 7f041ca00700 1 -- 172.21.15.37:0/583322058 shutdown_connections 2024-03-29T08:05:46.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:46.470+0000 7f041ca00700 1 -- 172.21.15.37:0/583322058 wait complete. 2024-03-29T08:05:46.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:46.608+0000 7efe8c400700 1 -- 172.21.15.37:0/3090645720 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efe84071f50 msgr2=0x7efe84070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:05:46.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:46.608+0000 7efe8c400700 1 --2- 172.21.15.37:0/3090645720 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efe84071f50 0x7efe84070460 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7efe78004940 tx=0x7efe78009e30 comp rx=0 tx=0).stop 2024-03-29T08:05:46.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:46.608+0000 7efe8c400700 1 -- 172.21.15.37:0/3090645720 shutdown_connections 2024-03-29T08:05:46.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:46.608+0000 7efe8c400700 1 --2- 172.21.15.37:0/3090645720 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe840709a0 0x7efe84070e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:05:46.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:46.608+0000 7efe8c400700 1 --2- 172.21.15.37:0/3090645720 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efe84071f50 0x7efe84070460 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:05:46.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:46.608+0000 7efe8c400700 1 -- 172.21.15.37:0/3090645720 >> 172.21.15.37:0/3090645720 conn(0x7efe840f8560 msgr2=0x7efe840fa970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:05:46.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:46.608+0000 7efe8c400700 1 -- 172.21.15.37:0/3090645720 shutdown_connections 2024-03-29T08:05:46.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:46.608+0000 7efe8c400700 1 -- 172.21.15.37:0/3090645720 wait complete. 2024-03-29T08:05:46.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:46.609+0000 7efe8c400700 1 Processor -- start 2024-03-29T08:05:46.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:46.609+0000 7efe8c400700 1 -- start start 2024-03-29T08:05:46.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:46.609+0000 7efe8c400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efe840709a0 0x7efe8410fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:05:46.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:46.609+0000 7efe8c400700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe84071f50 0x7efe84110090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:05:46.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:46.609+0000 7efe8c400700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe84110690 con 0x7efe840709a0 2024-03-29T08:05:46.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:46.609+0000 7efe8c400700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe84110ee0 con 0x7efe84071f50 2024-03-29T08:05:46.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:46.609+0000 7efe89a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efe840709a0 0x7efe8410fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:05:46.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:46.609+0000 7efe89a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efe840709a0 0x7efe8410fb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60696/0 (socket says 172.21.15.37:60696) 2024-03-29T08:05:46.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:46.609+0000 7efe89a00700 1 -- 172.21.15.37:0/1540219558 learned_addr learned my addr 172.21.15.37:0/1540219558 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:05:46.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:46.610+0000 7efe89000700 1 --2- 172.21.15.37:0/1540219558 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe84071f50 0x7efe84110090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:05:46.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:46.610+0000 7efe89a00700 1 -- 172.21.15.37:0/1540219558 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe84071f50 msgr2=0x7efe84110090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:05:46.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:46.610+0000 7efe89a00700 1 --2- 172.21.15.37:0/1540219558 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe84071f50 0x7efe84110090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:05:46.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:46.610+0000 7efe89a00700 1 -- 172.21.15.37:0/1540219558 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe78005040 con 0x7efe840709a0 2024-03-29T08:05:46.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:46.610+0000 7efe89a00700 1 --2- 172.21.15.37:0/1540219558 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efe840709a0 0x7efe8410fb50 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7efe7800b900 tx=0x7efe78009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:05:46.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:46.610+0000 7efe81600700 1 -- 172.21.15.37:0/1540219558 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efe7800e040 con 0x7efe840709a0 2024-03-29T08:05:46.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:46.611+0000 7efe81600700 1 -- 172.21.15.37:0/1540219558 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7efe78004290 con 0x7efe840709a0 2024-03-29T08:05:46.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:46.611+0000 7efe8c400700 1 -- 172.21.15.37:0/1540219558 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe84111160 con 0x7efe840709a0 2024-03-29T08:05:46.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:46.611+0000 7efe81600700 1 -- 172.21.15.37:0/1540219558 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efe7801c3f0 con 0x7efe840709a0 2024-03-29T08:05:46.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:46.611+0000 7efe8c400700 1 -- 172.21.15.37:0/1540219558 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe841116a0 con 0x7efe840709a0 2024-03-29T08:05:46.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:46.612+0000 7efe8c400700 1 -- 172.21.15.37:0/1540219558 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe8405f5f0 con 0x7efe840709a0 2024-03-29T08:05:46.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:46.613+0000 7efe81600700 1 -- 172.21.15.37:0/1540219558 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7efe78013420 con 0x7efe840709a0 2024-03-29T08:05:46.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:46.613+0000 7efe81600700 1 --2- 172.21.15.37:0/1540219558 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7efe6c0692b0 0x7efe6c06b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:05:46.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:46.614+0000 7efe81600700 1 -- 172.21.15.37:0/1540219558 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5550+0+0 (secure 0 0 0) 0x7efe78088520 con 0x7efe840709a0 2024-03-29T08:05:46.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:46.614+0000 7efe89000700 1 --2- 172.21.15.37:0/1540219558 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7efe6c0692b0 0x7efe6c06b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:05:46.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:46.614+0000 7efe89000700 1 --2- 172.21.15.37:0/1540219558 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7efe6c0692b0 0x7efe6c06b760 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7efe70004f40 tx=0x7efe7000b040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:05:46.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:46.618+0000 7efe81600700 1 -- 172.21.15.37:0/1540219558 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7efe780581f0 con 0x7efe840709a0 2024-03-29T08:05:46.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:46.911+0000 7efe8c400700 1 -- 172.21.15.37:0/1540219558 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7efe84111a20 con 0x7efe840709a0 2024-03-29T08:05:46.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:46.911+0000 7efe81600700 1 -- 172.21.15.37:0/1540219558 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+638 (secure 0 0 0) 0x7efe78086e60 con 0x7efe840709a0 2024-03-29T08:05:46.912 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-29T08:05:46.912 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": { 2024-03-29T08:05:46.912 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 17.2.7-789-g4213e17c (4213e17c4de67016ed14f55d22503f4923b3a901) quincy (stable)": 2 2024-03-29T08:05:46.912 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:05:46.913 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": { 2024-03-29T08:05:46.913 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 17.2.7-789-g4213e17c (4213e17c4de67016ed14f55d22503f4923b3a901) quincy (stable)": 2 2024-03-29T08:05:46.913 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:05:46.913 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": { 2024-03-29T08:05:46.913 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 17.2.7-789-g4213e17c (4213e17c4de67016ed14f55d22503f4923b3a901) quincy (stable)": 6 2024-03-29T08:05:46.913 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:05:46.913 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": { 2024-03-29T08:05:46.913 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 17.2.7-789-g4213e17c (4213e17c4de67016ed14f55d22503f4923b3a901) quincy (stable)": 4 2024-03-29T08:05:46.913 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:05:46.913 INFO:teuthology.orchestra.run.smithi037.stdout: "overall": { 2024-03-29T08:05:46.914 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 17.2.7-789-g4213e17c (4213e17c4de67016ed14f55d22503f4923b3a901) quincy (stable)": 14 2024-03-29T08:05:46.914 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-29T08:05:46.914 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-29T08:05:46.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:46.914+0000 7efe8c400700 1 -- 172.21.15.37:0/1540219558 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7efe6c0692b0 msgr2=0x7efe6c06b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:05:46.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:46.914+0000 7efe8c400700 1 --2- 172.21.15.37:0/1540219558 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7efe6c0692b0 0x7efe6c06b760 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7efe70004f40 tx=0x7efe7000b040 comp rx=0 tx=0).stop 2024-03-29T08:05:46.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:46.914+0000 7efe8c400700 1 -- 172.21.15.37:0/1540219558 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efe840709a0 msgr2=0x7efe8410fb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:05:46.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:46.914+0000 7efe8c400700 1 --2- 172.21.15.37:0/1540219558 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efe840709a0 0x7efe8410fb50 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7efe7800b900 tx=0x7efe78009e30 comp rx=0 tx=0).stop 2024-03-29T08:05:46.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:46.914+0000 7efe8c400700 1 -- 172.21.15.37:0/1540219558 shutdown_connections 2024-03-29T08:05:46.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:46.914+0000 7efe8c400700 1 --2- 172.21.15.37:0/1540219558 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7efe6c0692b0 0x7efe6c06b760 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:05:46.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:46.914+0000 7efe8c400700 1 --2- 172.21.15.37:0/1540219558 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efe840709a0 0x7efe8410fb50 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:05:46.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:46.914+0000 7efe8c400700 1 --2- 172.21.15.37:0/1540219558 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe84071f50 0x7efe84110090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:05:46.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:46.914+0000 7efe8c400700 1 -- 172.21.15.37:0/1540219558 >> 172.21.15.37:0/1540219558 conn(0x7efe840f8560 msgr2=0x7efe840f9d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:05:46.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:46.914+0000 7efe8c400700 1 -- 172.21.15.37:0/1540219558 shutdown_connections 2024-03-29T08:05:46.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:46.914+0000 7efe8c400700 1 -- 172.21.15.37:0/1540219558 wait complete. 2024-03-29T08:05:47.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:47.054+0000 7f615fe00700 1 -- 172.21.15.37:0/645931363 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6158071f50 msgr2=0x7f6158070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:05:47.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:47.054+0000 7f615fe00700 1 --2- 172.21.15.37:0/645931363 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6158071f50 0x7f6158070460 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f614c004940 tx=0x7f614c009e30 comp rx=0 tx=0).stop 2024-03-29T08:05:47.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:47.054+0000 7f615fe00700 1 -- 172.21.15.37:0/645931363 shutdown_connections 2024-03-29T08:05:47.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:47.054+0000 7f615fe00700 1 --2- 172.21.15.37:0/645931363 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f61580709a0 0x7f6158070e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:05:47.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:47.054+0000 7f615fe00700 1 --2- 172.21.15.37:0/645931363 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6158071f50 0x7f6158070460 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:05:47.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:47.054+0000 7f615fe00700 1 -- 172.21.15.37:0/645931363 >> 172.21.15.37:0/645931363 conn(0x7f61580f8560 msgr2=0x7f61580fa970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:05:47.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:47.054+0000 7f615fe00700 1 -- 172.21.15.37:0/645931363 shutdown_connections 2024-03-29T08:05:47.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:47.054+0000 7f615fe00700 1 -- 172.21.15.37:0/645931363 wait complete. 2024-03-29T08:05:47.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:47.055+0000 7f615fe00700 1 Processor -- start 2024-03-29T08:05:47.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:47.055+0000 7f615fe00700 1 -- start start 2024-03-29T08:05:47.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:47.055+0000 7f615fe00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f61580709a0 0x7f6158113ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:05:47.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:47.055+0000 7f615fe00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6158071f50 0x7f6158114420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:05:47.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:47.055+0000 7f615fe00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6158114a20 con 0x7f6158071f50 2024-03-29T08:05:47.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:47.055+0000 7f615fe00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6158115270 con 0x7f61580709a0 2024-03-29T08:05:47.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:47.056+0000 7f615ca00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6158071f50 0x7f6158114420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:05:47.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:47.056+0000 7f615d400700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f61580709a0 0x7f6158113ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:05:47.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:47.056+0000 7f615ca00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6158071f50 0x7f6158114420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60716/0 (socket says 172.21.15.37:60716) 2024-03-29T08:05:47.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:47.056+0000 7f615d400700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f61580709a0 0x7f6158113ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.37:58230/0 (socket says 172.21.15.37:58230) 2024-03-29T08:05:47.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:47.056+0000 7f615ca00700 1 -- 172.21.15.37:0/2837710957 learned_addr learned my addr 172.21.15.37:0/2837710957 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:05:47.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:47.056+0000 7f615ca00700 1 -- 172.21.15.37:0/2837710957 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f61580709a0 msgr2=0x7f6158113ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:05:47.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:47.056+0000 7f615ca00700 1 --2- 172.21.15.37:0/2837710957 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f61580709a0 0x7f6158113ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:05:47.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:47.056+0000 7f615ca00700 1 -- 172.21.15.37:0/2837710957 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f614c005040 con 0x7f6158071f50 2024-03-29T08:05:47.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:47.056+0000 7f615ca00700 1 --2- 172.21.15.37:0/2837710957 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6158071f50 0x7f6158114420 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f614400eb40 tx=0x7f614400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:05:47.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:47.057+0000 7f6155600700 1 -- 172.21.15.37:0/2837710957 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6144007cc0 con 0x7f6158071f50 2024-03-29T08:05:47.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:47.057+0000 7f6155600700 1 -- 172.21.15.37:0/2837710957 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f614400f040 con 0x7f6158071f50 2024-03-29T08:05:47.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:47.057+0000 7f615fe00700 1 -- 172.21.15.37:0/2837710957 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61581154f0 con 0x7f6158071f50 2024-03-29T08:05:47.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:47.057+0000 7f6155600700 1 -- 172.21.15.37:0/2837710957 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f614400c990 con 0x7f6158071f50 2024-03-29T08:05:47.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:47.057+0000 7f615fe00700 1 -- 172.21.15.37:0/2837710957 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6158115a30 con 0x7f6158071f50 2024-03-29T08:05:47.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:47.058+0000 7f615fe00700 1 -- 172.21.15.37:0/2837710957 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f615805f5f0 con 0x7f6158071f50 2024-03-29T08:05:47.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:47.059+0000 7f6155600700 1 -- 172.21.15.37:0/2837710957 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f614400caf0 con 0x7f6158071f50 2024-03-29T08:05:47.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:47.060+0000 7f6155600700 1 --2- 172.21.15.37:0/2837710957 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f61400692b0 0x7f614006b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:05:47.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:47.060+0000 7f6155600700 1 -- 172.21.15.37:0/2837710957 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f6144086dc0 con 0x7f6158071f50 2024-03-29T08:05:47.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:47.060+0000 7f615d400700 1 --2- 172.21.15.37:0/2837710957 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f61400692b0 0x7f614006b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:05:47.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:47.061+0000 7f615d400700 1 --2- 172.21.15.37:0/2837710957 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f61400692b0 0x7f614006b760 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f614c00b0d0 tx=0x7f614c009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:05:47.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:47.064+0000 7f6155600700 1 -- 172.21.15.37:0/2837710957 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f614404e190 con 0x7f6158071f50 2024-03-29T08:05:47.186 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:05:47 smithi148 ceph-mon[44992]: from='client.14586 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:05:47.186 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:05:47 smithi148 ceph-mon[44992]: from='client.14590 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:05:47.186 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:05:47 smithi148 ceph-mon[44992]: from='client.? 172.21.15.37:0/1540219558' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:05:47.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:47.287+0000 7f615fe00700 1 -- 172.21.15.37:0/2837710957 --> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6158115dd0 con 0x7f61400692b0 2024-03-29T08:05:47.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:47.289+0000 7f6155600700 1 -- 172.21.15.37:0/2837710957 <== mgr.14208 v2:172.21.15.37:6800/1802836043 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+341 (secure 0 0 0) 0x7f6158115dd0 con 0x7f61400692b0 2024-03-29T08:05:47.290 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-29T08:05:47.290 INFO:teuthology.orchestra.run.smithi037.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59", 2024-03-29T08:05:47.290 INFO:teuthology.orchestra.run.smithi037.stdout: "in_progress": true, 2024-03-29T08:05:47.290 INFO:teuthology.orchestra.run.smithi037.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-03-29T08:05:47.290 INFO:teuthology.orchestra.run.smithi037.stdout: "services_complete": [], 2024-03-29T08:05:47.290 INFO:teuthology.orchestra.run.smithi037.stdout: "progress": "0/2 daemons upgraded", 2024-03-29T08:05:47.290 INFO:teuthology.orchestra.run.smithi037.stdout: "message": "Currently upgrading mgr daemons", 2024-03-29T08:05:47.290 INFO:teuthology.orchestra.run.smithi037.stdout: "is_paused": false 2024-03-29T08:05:47.291 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-29T08:05:47.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:47.291+0000 7f615fe00700 1 -- 172.21.15.37:0/2837710957 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f61400692b0 msgr2=0x7f614006b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:05:47.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:47.291+0000 7f615fe00700 1 --2- 172.21.15.37:0/2837710957 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f61400692b0 0x7f614006b760 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f614c00b0d0 tx=0x7f614c009e30 comp rx=0 tx=0).stop 2024-03-29T08:05:47.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:47.291+0000 7f615fe00700 1 -- 172.21.15.37:0/2837710957 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6158071f50 msgr2=0x7f6158114420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:05:47.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:47.291+0000 7f615fe00700 1 --2- 172.21.15.37:0/2837710957 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6158071f50 0x7f6158114420 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f614400eb40 tx=0x7f614400b040 comp rx=0 tx=0).stop 2024-03-29T08:05:47.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:47.291+0000 7f615fe00700 1 -- 172.21.15.37:0/2837710957 shutdown_connections 2024-03-29T08:05:47.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:47.291+0000 7f615fe00700 1 --2- 172.21.15.37:0/2837710957 >> [v2:172.21.15.37:6800/1802836043,v1:172.21.15.37:6801/1802836043] conn(0x7f61400692b0 0x7f614006b760 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:05:47.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:47.291+0000 7f615fe00700 1 --2- 172.21.15.37:0/2837710957 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f61580709a0 0x7f6158113ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:05:47.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:47.291+0000 7f615fe00700 1 --2- 172.21.15.37:0/2837710957 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6158071f50 0x7f6158114420 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:05:47.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:47.291+0000 7f615fe00700 1 -- 172.21.15.37:0/2837710957 >> 172.21.15.37:0/2837710957 conn(0x7f61580f8560 msgr2=0x7f61580f9ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:05:47.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:47.291+0000 7f615fe00700 1 -- 172.21.15.37:0/2837710957 shutdown_connections 2024-03-29T08:05:47.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:05:47.291+0000 7f615fe00700 1 -- 172.21.15.37:0/2837710957 wait complete. 2024-03-29T08:05:47.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:05:47 smithi037 ceph-mon[35162]: from='client.14586 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:05:47.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:05:47 smithi037 ceph-mon[35162]: from='client.14590 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:05:47.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:05:47 smithi037 ceph-mon[35162]: from='client.? 172.21.15.37:0/1540219558' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:05:48.015 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:05:48 smithi148 ceph-mon[44992]: from='client.14598 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:05:48.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:05:48 smithi148 ceph-mon[44992]: pgmap v271: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:05:48.426 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:05:48 smithi037 ceph-mon[35162]: from='client.14598 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:05:48.426 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:05:48 smithi037 ceph-mon[35162]: pgmap v271: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:05:49.198 INFO:tasks.workunit.client.0.smithi037.stderr:Updating files: 61% (7587/12271) Updating files: 62% (7609/12271) Updating files: 63% (7731/12271) Updating files: 64% (7854/12271) Updating files: 65% (7977/12271) Updating files: 66% (8099/12271) Updating files: 67% (8222/12271) Updating files: 68% (8345/12271) Updating files: 69% (8467/12271) Updating files: 70% (8590/12271) Updating files: 71% (8713/12271) Updating files: 72% (8836/12271) Updating files: 73% (8958/12271) Updating files: 74% (9081/12271) Updating files: 75% (9204/12271) Updating files: 76% (9326/12271) Updating files: 77% (9449/12271) Updating files: 78% (9572/12271) Updating files: 79% (9695/12271) Updating files: 80% (9817/12271) Updating files: 81% (9940/12271) Updating files: 82% (10063/12271) Updating files: 83% (10185/12271) Updating files: 84% (10308/12271) Updating files: 85% (10431/12271) Updating files: 86% (10554/12271) Updating files: 87% (10676/12271) Updating files: 88% (10799/12271) Updating files: 89% (10922/12271) Updating files: 90% (11044/12271) Updating files: 91% (11167/12271) Updating files: 92% (11290/12271) Updating files: 93% (11413/12271) Updating files: 94% (11535/12271) Updating files: 95% (11658/12271) Updating files: 96% (11781/12271) Updating files: 97% (11903/12271) Updating files: 98% (12026/12271) Updating files: 99% (12149/12271) Updating files: 100% (12271/12271) Updating files: 100% (12271/12271), done. 2024-03-29T08:05:50.029 INFO:tasks.workunit.client.0.smithi037.stderr:Note: switching to 'd0e392d9aee635f2b65768c460c7a151ead7a5e2'. 2024-03-29T08:05:50.029 INFO:tasks.workunit.client.0.smithi037.stderr: 2024-03-29T08:05:50.029 INFO:tasks.workunit.client.0.smithi037.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-03-29T08:05:50.029 INFO:tasks.workunit.client.0.smithi037.stderr:changes and commit them, and you can discard any commits you make in this 2024-03-29T08:05:50.029 INFO:tasks.workunit.client.0.smithi037.stderr:state without impacting any branches by switching back to a branch. 2024-03-29T08:05:50.029 INFO:tasks.workunit.client.0.smithi037.stderr: 2024-03-29T08:05:50.029 INFO:tasks.workunit.client.0.smithi037.stderr:If you want to create a new branch to retain commits you create, you may 2024-03-29T08:05:50.029 INFO:tasks.workunit.client.0.smithi037.stderr:do so (now or later) by using -c with the switch command. Example: 2024-03-29T08:05:50.030 INFO:tasks.workunit.client.0.smithi037.stderr: 2024-03-29T08:05:50.030 INFO:tasks.workunit.client.0.smithi037.stderr: git switch -c 2024-03-29T08:05:50.030 INFO:tasks.workunit.client.0.smithi037.stderr: 2024-03-29T08:05:50.030 INFO:tasks.workunit.client.0.smithi037.stderr:Or undo this operation with: 2024-03-29T08:05:50.030 INFO:tasks.workunit.client.0.smithi037.stderr: 2024-03-29T08:05:50.030 INFO:tasks.workunit.client.0.smithi037.stderr: git switch - 2024-03-29T08:05:50.030 INFO:tasks.workunit.client.0.smithi037.stderr: 2024-03-29T08:05:50.030 INFO:tasks.workunit.client.0.smithi037.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-03-29T08:05:50.030 INFO:tasks.workunit.client.0.smithi037.stderr: 2024-03-29T08:05:50.030 INFO:tasks.workunit.client.0.smithi037.stderr:HEAD is now at d0e392d9aee Merge PR #55803 into wip-vshankar-testing-20240326.105515 2024-03-29T08:05:50.044 DEBUG:teuthology.orchestra.run.smithi037:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-03-29T08:05:50.101 INFO:tasks.workunit.client.0.smithi037.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-03-29T08:05:50.106 INFO:tasks.workunit.client.0.smithi037.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-03-29T08:05:50.106 INFO:tasks.workunit.client.0.smithi037.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-03-29T08:05:50.557 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:05:50 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:05:50.557 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:05:50 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:05:50.557 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:05:50 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T08:05:50.557 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:05:50 smithi148 ceph-mon[44992]: pgmap v272: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:05:50.676 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:05:50 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:05:50.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:05:50 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:05:50.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:05:50 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T08:05:50.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:05:50 smithi037 ceph-mon[35162]: pgmap v272: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:05:50.742 INFO:tasks.workunit.client.0.smithi037.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-03-29T08:05:50.829 INFO:tasks.workunit.client.0.smithi037.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-03-29T08:05:50.893 INFO:tasks.workunit.client.0.smithi037.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-03-29T08:05:50.897 INFO:tasks.workunit.client.0.smithi037.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-03-29T08:05:50.897 INFO:tasks.workunit.client.0.smithi037.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-03-29T08:05:50.961 INFO:tasks.workunit.client.0.smithi037.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-03-29T08:05:50.969 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-03-29T08:05:50.969 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-03-29T08:05:51.013 INFO:tasks.workunit:Running workunits matching suites/fsstress.sh on client.0... 2024-03-29T08:05:51.015 INFO:tasks.workunit:Running workunit suites/fsstress.sh... 2024-03-29T08:05:51.015 DEBUG:teuthology.orchestra.run.smithi037:workunit test suites/fsstress.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=d0e392d9aee635f2b65768c460c7a151ead7a5e2 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsstress.sh 2024-03-29T08:05:51.120 INFO:tasks.workunit.client.0.smithi037.stderr:+ mkdir -p fsstress 2024-03-29T08:05:51.124 INFO:tasks.workunit.client.0.smithi037.stderr:+ pushd fsstress 2024-03-29T08:05:51.127 INFO:tasks.workunit.client.0.smithi037.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-03-29T08:05:51.127 INFO:tasks.workunit.client.0.smithi037.stderr:+ wget -q -O ltp-full.tgz http://download.ceph.com/qa/ltp-full-20091231.tgz 2024-03-29T08:05:52.792 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:05:52 smithi148 ceph-mon[44992]: pgmap v273: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:05:52.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:05:52 smithi037 ceph-mon[35162]: pgmap v273: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:05:53.212 INFO:tasks.workunit.client.0.smithi037.stderr:+ tar xzf ltp-full.tgz 2024-03-29T08:05:54.841 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:05:54 smithi148 ceph-mon[44992]: pgmap v274: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:05:54.842 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:05:54 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:05:54.842 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:05:54 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:05:55.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:05:54 smithi037 ceph-mon[35162]: pgmap v274: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:05:55.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:05:54 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:05:55.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:05:54 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:05:56.463 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:05:56 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:05:56.463 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:05:56 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:05:56.463 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:05:56 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:05:56.463 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:05:56 smithi148 ceph-mon[44992]: pgmap v275: 65 pgs: 65 active+clean; 657 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 7.6 KiB/s wr, 2 op/s 2024-03-29T08:05:56.676 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:05:56 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:05:56.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:05:56 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:05:56.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:05:56 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:05:56.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:05:56 smithi037 ceph-mon[35162]: pgmap v275: 65 pgs: 65 active+clean; 657 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 7.6 KiB/s wr, 2 op/s 2024-03-29T08:05:56.692 INFO:tasks.workunit.client.1.smithi148.stderr:Updating files: 55% (6847/12271) Updating files: 56% (6872/12271) Updating files: 57% (6995/12271) Updating files: 58% (7118/12271) Updating files: 59% (7240/12271) Updating files: 60% (7363/12271) Updating files: 61% (7486/12271) Updating files: 62% (7609/12271) Updating files: 63% (7731/12271) Updating files: 64% (7854/12271) Updating files: 65% (7977/12271) Updating files: 66% (8099/12271) Updating files: 67% (8222/12271) Updating files: 68% (8345/12271) Updating files: 69% (8467/12271) Updating files: 70% (8590/12271) Updating files: 71% (8713/12271) Updating files: 72% (8836/12271) Updating files: 73% (8958/12271) Updating files: 74% (9081/12271) Updating files: 75% (9204/12271) Updating files: 76% (9326/12271) Updating files: 77% (9449/12271) Updating files: 78% (9572/12271) Updating files: 79% (9695/12271) Updating files: 80% (9817/12271) Updating files: 81% (9940/12271) Updating files: 82% (10063/12271) Updating files: 83% (10185/12271) Updating files: 84% (10308/12271) Updating files: 85% (10431/12271) Updating files: 86% (10554/12271) Updating files: 87% (10676/12271) Updating files: 87% (10701/12271) Updating files: 88% (10799/12271) Updating files: 89% (10922/12271) Updating files: 90% (11044/12271) Updating files: 91% (11167/12271) Updating files: 92% (11290/12271) Updating files: 93% (11413/12271) Updating files: 94% (11535/12271) Updating files: 95% (11658/12271) Updating files: 96% (11781/12271) Updating files: 97% (11903/12271) Updating files: 98% (12026/12271) Updating files: 99% (12149/12271) Updating files: 100% (12271/12271) Updating files: 100% (12271/12271), done. 2024-03-29T08:05:57.858 INFO:tasks.workunit.client.1.smithi148.stderr:Note: switching to 'd0e392d9aee635f2b65768c460c7a151ead7a5e2'. 2024-03-29T08:05:57.858 INFO:tasks.workunit.client.1.smithi148.stderr: 2024-03-29T08:05:57.858 INFO:tasks.workunit.client.1.smithi148.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-03-29T08:05:57.858 INFO:tasks.workunit.client.1.smithi148.stderr:changes and commit them, and you can discard any commits you make in this 2024-03-29T08:05:57.858 INFO:tasks.workunit.client.1.smithi148.stderr:state without impacting any branches by switching back to a branch. 2024-03-29T08:05:57.858 INFO:tasks.workunit.client.1.smithi148.stderr: 2024-03-29T08:05:57.858 INFO:tasks.workunit.client.1.smithi148.stderr:If you want to create a new branch to retain commits you create, you may 2024-03-29T08:05:57.858 INFO:tasks.workunit.client.1.smithi148.stderr:do so (now or later) by using -c with the switch command. Example: 2024-03-29T08:05:57.858 INFO:tasks.workunit.client.1.smithi148.stderr: 2024-03-29T08:05:57.858 INFO:tasks.workunit.client.1.smithi148.stderr: git switch -c 2024-03-29T08:05:57.859 INFO:tasks.workunit.client.1.smithi148.stderr: 2024-03-29T08:05:57.859 INFO:tasks.workunit.client.1.smithi148.stderr:Or undo this operation with: 2024-03-29T08:05:57.859 INFO:tasks.workunit.client.1.smithi148.stderr: 2024-03-29T08:05:57.859 INFO:tasks.workunit.client.1.smithi148.stderr: git switch - 2024-03-29T08:05:57.859 INFO:tasks.workunit.client.1.smithi148.stderr: 2024-03-29T08:05:57.859 INFO:tasks.workunit.client.1.smithi148.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-03-29T08:05:57.859 INFO:tasks.workunit.client.1.smithi148.stderr: 2024-03-29T08:05:57.859 INFO:tasks.workunit.client.1.smithi148.stderr:HEAD is now at d0e392d9aee Merge PR #55803 into wip-vshankar-testing-20240326.105515 2024-03-29T08:05:57.875 DEBUG:teuthology.orchestra.run.smithi148:> cd -- /home/ubuntu/cephtest/clone.client.1/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.1 2024-03-29T08:05:57.949 INFO:tasks.workunit.client.1.smithi148.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-03-29T08:05:57.954 INFO:tasks.workunit.client.1.smithi148.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/direct_io' 2024-03-29T08:05:57.954 INFO:tasks.workunit.client.1.smithi148.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-03-29T08:05:58.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:05:58 smithi148 ceph-mon[44992]: pgmap v276: 65 pgs: 65 active+clean; 13 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.1 MiB/s wr, 49 op/s 2024-03-29T08:05:58.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:05:58 smithi037 ceph-mon[35162]: pgmap v276: 65 pgs: 65 active+clean; 13 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.1 MiB/s wr, 49 op/s 2024-03-29T08:05:58.944 INFO:tasks.workunit.client.1.smithi148.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-03-29T08:05:59.068 INFO:tasks.workunit.client.1.smithi148.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-03-29T08:05:59.133 INFO:tasks.workunit.client.1.smithi148.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/direct_io' 2024-03-29T08:05:59.138 INFO:tasks.workunit.client.1.smithi148.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/fs' 2024-03-29T08:05:59.138 INFO:tasks.workunit.client.1.smithi148.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-03-29T08:05:59.205 INFO:tasks.workunit.client.1.smithi148.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/fs' 2024-03-29T08:05:59.213 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-03-29T08:05:59.213 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/home/ubuntu/cephtest/workunits.list.client.1 of=/dev/stdout 2024-03-29T08:05:59.267 INFO:tasks.workunit:Running workunits matching suites/fsstress.sh on client.1... 2024-03-29T08:05:59.268 INFO:tasks.workunit:Running workunit suites/fsstress.sh... 2024-03-29T08:05:59.268 DEBUG:teuthology.orchestra.run.smithi148:workunit test suites/fsstress.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && cd -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=d0e392d9aee635f2b65768c460c7a151ead7a5e2 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="1" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.1 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.1 CEPH_MNT=/home/ubuntu/cephtest/mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.1/qa/workunits/suites/fsstress.sh 2024-03-29T08:05:59.347 INFO:tasks.workunit.client.1.smithi148.stderr:+ mkdir -p fsstress 2024-03-29T08:05:59.352 INFO:tasks.workunit.client.1.smithi148.stderr:+ pushd fsstress 2024-03-29T08:05:59.355 INFO:tasks.workunit.client.1.smithi148.stdout:~/cephtest/mnt.1/client.1/tmp/fsstress ~/cephtest/mnt.1/client.1/tmp 2024-03-29T08:05:59.356 INFO:tasks.workunit.client.1.smithi148.stderr:+ wget -q -O ltp-full.tgz http://download.ceph.com/qa/ltp-full-20091231.tgz 2024-03-29T08:06:00.219 INFO:tasks.workunit.client.1.smithi148.stderr:+ tar xzf ltp-full.tgz 2024-03-29T08:06:00.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:06:00 smithi037 ceph-mon[35162]: pgmap v277: 65 pgs: 65 active+clean; 14 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.1 MiB/s wr, 56 op/s 2024-03-29T08:06:00.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:06:00 smithi037 ceph-mon[35162]: Standby manager daemon smithi148.onhgvm restarted 2024-03-29T08:06:00.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:06:00 smithi037 ceph-mon[35162]: Standby manager daemon smithi148.onhgvm started 2024-03-29T08:06:00.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:06:00 smithi037 ceph-mon[35162]: from='mgr.? 172.21.15.148:0/667342922' entity='mgr.smithi148.onhgvm' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi148.onhgvm/crt"}]: dispatch 2024-03-29T08:06:00.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:06:00 smithi037 ceph-mon[35162]: from='mgr.? 172.21.15.148:0/667342922' entity='mgr.smithi148.onhgvm' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-29T08:06:00.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:06:00 smithi037 ceph-mon[35162]: from='mgr.? 172.21.15.148:0/667342922' entity='mgr.smithi148.onhgvm' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi148.onhgvm/key"}]: dispatch 2024-03-29T08:06:00.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:06:00 smithi037 ceph-mon[35162]: from='mgr.? 172.21.15.148:0/667342922' entity='mgr.smithi148.onhgvm' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-29T08:06:01.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:06:00 smithi148 ceph-mon[44992]: pgmap v277: 65 pgs: 65 active+clean; 14 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.1 MiB/s wr, 56 op/s 2024-03-29T08:06:01.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:06:00 smithi148 ceph-mon[44992]: Standby manager daemon smithi148.onhgvm restarted 2024-03-29T08:06:01.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:06:00 smithi148 ceph-mon[44992]: Standby manager daemon smithi148.onhgvm started 2024-03-29T08:06:01.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:06:00 smithi148 ceph-mon[44992]: from='mgr.? 172.21.15.148:0/667342922' entity='mgr.smithi148.onhgvm' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi148.onhgvm/crt"}]: dispatch 2024-03-29T08:06:01.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:06:00 smithi148 ceph-mon[44992]: from='mgr.? 172.21.15.148:0/667342922' entity='mgr.smithi148.onhgvm' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-29T08:06:01.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:06:00 smithi148 ceph-mon[44992]: from='mgr.? 172.21.15.148:0/667342922' entity='mgr.smithi148.onhgvm' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi148.onhgvm/key"}]: dispatch 2024-03-29T08:06:01.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:06:00 smithi148 ceph-mon[44992]: from='mgr.? 172.21.15.148:0/667342922' entity='mgr.smithi148.onhgvm' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-29T08:06:01.880 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:06:01 smithi037 ceph-mon[35162]: mgrmap e20: smithi037.wjikii(active, since 8m), standbys: smithi148.onhgvm 2024-03-29T08:06:01.895 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:06:01 smithi148 ceph-mon[44992]: mgrmap e20: smithi037.wjikii(active, since 8m), standbys: smithi148.onhgvm 2024-03-29T08:06:02.928 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:06:02 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:06:02.928 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:06:02 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:06:02.928 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:06:02 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T08:06:02.928 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:06:02 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T08:06:02.928 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:06:02 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:06:02.928 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:06:02 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T08:06:02.928 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:06:02 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "mgr fail", "who": "smithi037.wjikii"}]: dispatch 2024-03-29T08:06:02.928 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:06:02 smithi037 ceph-mon[35162]: osdmap e48: 6 total, 6 up, 6 in 2024-03-29T08:06:02.928 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:06:02 smithi037 ceph-mon[35162]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd='[{"prefix": "mgr fail", "who": "smithi037.wjikii"}]': finished 2024-03-29T08:06:02.928 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:06:02 smithi037 ceph-mon[35162]: mgrmap e21: smithi148.onhgvm(active, starting, since 0.00994826s) 2024-03-29T08:06:02.929 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:06:02 smithi037 ceph-mon[35162]: from='mgr.14602 172.21.15.148:0/667342922' entity='mgr.smithi148.onhgvm' cmd=[{"prefix": "mon metadata", "id": "smithi037"}]: dispatch 2024-03-29T08:06:02.929 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:06:02 smithi037 ceph-mon[35162]: from='mgr.14602 172.21.15.148:0/667342922' entity='mgr.smithi148.onhgvm' cmd=[{"prefix": "mon metadata", "id": "smithi148"}]: dispatch 2024-03-29T08:06:02.929 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:06:02 smithi037 ceph-mon[35162]: from='mgr.14602 172.21.15.148:0/667342922' entity='mgr.smithi148.onhgvm' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi037.mmwgxe"}]: dispatch 2024-03-29T08:06:02.929 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:06:02 smithi037 ceph-mon[35162]: from='mgr.14602 172.21.15.148:0/667342922' entity='mgr.smithi148.onhgvm' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi037.iifsmh"}]: dispatch 2024-03-29T08:06:02.929 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:06:02 smithi037 ceph-mon[35162]: from='mgr.14602 172.21.15.148:0/667342922' entity='mgr.smithi148.onhgvm' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi148.qukpjh"}]: dispatch 2024-03-29T08:06:02.929 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:06:02 smithi037 ceph-mon[35162]: from='mgr.14602 172.21.15.148:0/667342922' entity='mgr.smithi148.onhgvm' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi148.sqteih"}]: dispatch 2024-03-29T08:06:02.929 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:06:02 smithi037 ceph-mon[35162]: from='mgr.14602 172.21.15.148:0/667342922' entity='mgr.smithi148.onhgvm' cmd=[{"prefix": "mgr metadata", "who": "smithi148.onhgvm", "id": "smithi148.onhgvm"}]: dispatch 2024-03-29T08:06:02.929 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:06:02 smithi037 ceph-mon[35162]: from='mgr.14602 172.21.15.148:0/667342922' entity='mgr.smithi148.onhgvm' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-29T08:06:02.929 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:06:02 smithi037 ceph-mon[35162]: from='mgr.14602 172.21.15.148:0/667342922' entity='mgr.smithi148.onhgvm' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-29T08:06:02.929 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:06:02 smithi037 ceph-mon[35162]: from='mgr.14602 172.21.15.148:0/667342922' entity='mgr.smithi148.onhgvm' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-29T08:06:02.929 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:06:02 smithi037 ceph-mon[35162]: from='mgr.14602 172.21.15.148:0/667342922' entity='mgr.smithi148.onhgvm' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-29T08:06:02.929 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:06:02 smithi037 ceph-mon[35162]: from='mgr.14602 172.21.15.148:0/667342922' entity='mgr.smithi148.onhgvm' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-29T08:06:02.929 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:06:02 smithi037 ceph-mon[35162]: from='mgr.14602 172.21.15.148:0/667342922' entity='mgr.smithi148.onhgvm' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-29T08:06:02.929 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:06:02 smithi037 ceph-mon[35162]: from='mgr.14602 172.21.15.148:0/667342922' entity='mgr.smithi148.onhgvm' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-29T08:06:02.930 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:06:02 smithi037 ceph-mon[35162]: from='mgr.14602 172.21.15.148:0/667342922' entity='mgr.smithi148.onhgvm' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-29T08:06:02.930 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:06:02 smithi037 ceph-mon[35162]: from='mgr.14602 172.21.15.148:0/667342922' entity='mgr.smithi148.onhgvm' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-29T08:06:02.930 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:06:02 smithi037 ceph-mon[35162]: Manager daemon smithi148.onhgvm is now available 2024-03-29T08:06:02.930 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:06:02 smithi037 ceph-mon[35162]: from='mgr.14602 172.21.15.148:0/667342922' entity='mgr.smithi148.onhgvm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T08:06:02.930 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:06:02 smithi037 ceph-mon[35162]: from='mgr.14602 172.21.15.148:0/667342922' entity='mgr.smithi148.onhgvm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:06:02.930 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:06:02 smithi037 ceph-mon[35162]: from='mgr.14602 172.21.15.148:0/667342922' entity='mgr.smithi148.onhgvm' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi148.onhgvm/mirror_snapshot_schedule"}]: dispatch 2024-03-29T08:06:02.930 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:06:02 smithi037 ceph-mon[35162]: from='mgr.14602 ' entity='mgr.smithi148.onhgvm' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi148.onhgvm/mirror_snapshot_schedule"}]: dispatch 2024-03-29T08:06:02.930 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:06:02 smithi037 ceph-mon[35162]: from='mgr.14602 172.21.15.148:0/667342922' entity='mgr.smithi148.onhgvm' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi148.onhgvm/trash_purge_schedule"}]: dispatch 2024-03-29T08:06:02.930 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:06:02 smithi037 ceph-mon[35162]: from='mgr.14602 ' entity='mgr.smithi148.onhgvm' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi148.onhgvm/trash_purge_schedule"}]: dispatch 2024-03-29T08:06:02.930 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:06:02 smithi037 ceph-mon[35162]: from='mgr.14602 ' entity='mgr.smithi148.onhgvm' 2024-03-29T08:06:02.930 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:06:02 smithi037 ceph-mon[35162]: from='mgr.14602 ' entity='mgr.smithi148.onhgvm' 2024-03-29T08:06:03.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:06:02 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:06:03.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:06:02 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:06:03.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:06:02 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T08:06:03.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:06:02 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T08:06:03.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:06:02 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' 2024-03-29T08:06:03.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:06:02 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T08:06:03.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:06:02 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "mgr fail", "who": "smithi037.wjikii"}]: dispatch 2024-03-29T08:06:03.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:06:02 smithi148 ceph-mon[44992]: osdmap e48: 6 total, 6 up, 6 in 2024-03-29T08:06:03.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:06:02 smithi148 ceph-mon[44992]: from='mgr.14208 172.21.15.37:0/4272679143' entity='mgr.smithi037.wjikii' cmd='[{"prefix": "mgr fail", "who": "smithi037.wjikii"}]': finished 2024-03-29T08:06:03.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:06:02 smithi148 ceph-mon[44992]: mgrmap e21: smithi148.onhgvm(active, starting, since 0.00994826s) 2024-03-29T08:06:03.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:06:02 smithi148 ceph-mon[44992]: from='mgr.14602 172.21.15.148:0/667342922' entity='mgr.smithi148.onhgvm' cmd=[{"prefix": "mon metadata", "id": "smithi037"}]: dispatch 2024-03-29T08:06:03.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:06:02 smithi148 ceph-mon[44992]: from='mgr.14602 172.21.15.148:0/667342922' entity='mgr.smithi148.onhgvm' cmd=[{"prefix": "mon metadata", "id": "smithi148"}]: dispatch 2024-03-29T08:06:03.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:06:02 smithi148 ceph-mon[44992]: from='mgr.14602 172.21.15.148:0/667342922' entity='mgr.smithi148.onhgvm' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi037.mmwgxe"}]: dispatch 2024-03-29T08:06:03.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:06:02 smithi148 ceph-mon[44992]: from='mgr.14602 172.21.15.148:0/667342922' entity='mgr.smithi148.onhgvm' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi037.iifsmh"}]: dispatch 2024-03-29T08:06:03.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:06:02 smithi148 ceph-mon[44992]: from='mgr.14602 172.21.15.148:0/667342922' entity='mgr.smithi148.onhgvm' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi148.qukpjh"}]: dispatch 2024-03-29T08:06:03.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:06:02 smithi148 ceph-mon[44992]: from='mgr.14602 172.21.15.148:0/667342922' entity='mgr.smithi148.onhgvm' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi148.sqteih"}]: dispatch 2024-03-29T08:06:03.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:06:02 smithi148 ceph-mon[44992]: from='mgr.14602 172.21.15.148:0/667342922' entity='mgr.smithi148.onhgvm' cmd=[{"prefix": "mgr metadata", "who": "smithi148.onhgvm", "id": "smithi148.onhgvm"}]: dispatch 2024-03-29T08:06:03.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:06:02 smithi148 ceph-mon[44992]: from='mgr.14602 172.21.15.148:0/667342922' entity='mgr.smithi148.onhgvm' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-29T08:06:03.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:06:02 smithi148 ceph-mon[44992]: from='mgr.14602 172.21.15.148:0/667342922' entity='mgr.smithi148.onhgvm' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-29T08:06:03.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:06:02 smithi148 ceph-mon[44992]: from='mgr.14602 172.21.15.148:0/667342922' entity='mgr.smithi148.onhgvm' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-29T08:06:03.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:06:02 smithi148 ceph-mon[44992]: from='mgr.14602 172.21.15.148:0/667342922' entity='mgr.smithi148.onhgvm' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-29T08:06:03.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:06:02 smithi148 ceph-mon[44992]: from='mgr.14602 172.21.15.148:0/667342922' entity='mgr.smithi148.onhgvm' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-29T08:06:03.058 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:06:02 smithi148 ceph-mon[44992]: from='mgr.14602 172.21.15.148:0/667342922' entity='mgr.smithi148.onhgvm' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-29T08:06:03.058 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:06:02 smithi148 ceph-mon[44992]: from='mgr.14602 172.21.15.148:0/667342922' entity='mgr.smithi148.onhgvm' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-29T08:06:03.058 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:06:02 smithi148 ceph-mon[44992]: from='mgr.14602 172.21.15.148:0/667342922' entity='mgr.smithi148.onhgvm' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-29T08:06:03.058 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:06:02 smithi148 ceph-mon[44992]: from='mgr.14602 172.21.15.148:0/667342922' entity='mgr.smithi148.onhgvm' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-29T08:06:03.058 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:06:02 smithi148 ceph-mon[44992]: Manager daemon smithi148.onhgvm is now available 2024-03-29T08:06:03.058 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:06:02 smithi148 ceph-mon[44992]: from='mgr.14602 172.21.15.148:0/667342922' entity='mgr.smithi148.onhgvm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T08:06:03.058 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:06:02 smithi148 ceph-mon[44992]: from='mgr.14602 172.21.15.148:0/667342922' entity='mgr.smithi148.onhgvm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:06:03.058 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:06:02 smithi148 ceph-mon[44992]: from='mgr.14602 172.21.15.148:0/667342922' entity='mgr.smithi148.onhgvm' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi148.onhgvm/mirror_snapshot_schedule"}]: dispatch 2024-03-29T08:06:03.058 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:06:02 smithi148 ceph-mon[44992]: from='mgr.14602 ' entity='mgr.smithi148.onhgvm' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi148.onhgvm/mirror_snapshot_schedule"}]: dispatch 2024-03-29T08:06:03.058 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:06:02 smithi148 ceph-mon[44992]: from='mgr.14602 172.21.15.148:0/667342922' entity='mgr.smithi148.onhgvm' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi148.onhgvm/trash_purge_schedule"}]: dispatch 2024-03-29T08:06:03.058 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:06:02 smithi148 ceph-mon[44992]: from='mgr.14602 ' entity='mgr.smithi148.onhgvm' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi148.onhgvm/trash_purge_schedule"}]: dispatch 2024-03-29T08:06:03.058 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:06:02 smithi148 ceph-mon[44992]: from='mgr.14602 ' entity='mgr.smithi148.onhgvm' 2024-03-29T08:06:03.058 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:06:02 smithi148 ceph-mon[44992]: from='mgr.14602 ' entity='mgr.smithi148.onhgvm' 2024-03-29T08:06:04.178 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:06:03 smithi037 ceph-mon[35162]: Deploying cephadm binary to smithi148 2024-03-29T08:06:04.179 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:06:03 smithi037 ceph-mon[35162]: Deploying cephadm binary to smithi037 2024-03-29T08:06:04.179 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:06:03 smithi037 ceph-mon[35162]: mgrmap e22: smithi148.onhgvm(active, since 1.01517s) 2024-03-29T08:06:04.179 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:06:03 smithi037 ceph-mon[35162]: pgmap v3: 65 pgs: 65 active+clean; 36 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:06:04.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:06:03 smithi148 ceph-mon[44992]: Deploying cephadm binary to smithi148 2024-03-29T08:06:04.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:06:03 smithi148 ceph-mon[44992]: Deploying cephadm binary to smithi037 2024-03-29T08:06:04.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:06:03 smithi148 ceph-mon[44992]: mgrmap e22: smithi148.onhgvm(active, since 1.01517s) 2024-03-29T08:06:04.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:06:03 smithi148 ceph-mon[44992]: pgmap v3: 65 pgs: 65 active+clean; 36 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:06:05.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:06:04 smithi037 ceph-mon[35162]: [29/Mar/2024:08:06:03] ENGINE Bus STARTING 2024-03-29T08:06:05.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:06:04 smithi037 ceph-mon[35162]: [29/Mar/2024:08:06:03] ENGINE Serving on http://172.21.15.148:8765 2024-03-29T08:06:05.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:06:04 smithi037 ceph-mon[35162]: [29/Mar/2024:08:06:03] ENGINE Serving on https://172.21.15.148:7150 2024-03-29T08:06:05.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:06:04 smithi037 ceph-mon[35162]: [29/Mar/2024:08:06:03] ENGINE Bus STARTED 2024-03-29T08:06:05.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:06:04 smithi037 ceph-mon[35162]: mgrmap e23: smithi148.onhgvm(active, since 2s) 2024-03-29T08:06:05.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:06:04 smithi037 ceph-mon[35162]: pgmap v4: 65 pgs: 65 active+clean; 36 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:06:05.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:06:04 smithi148 ceph-mon[44992]: [29/Mar/2024:08:06:03] ENGINE Bus STARTING 2024-03-29T08:06:05.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:06:04 smithi148 ceph-mon[44992]: [29/Mar/2024:08:06:03] ENGINE Serving on http://172.21.15.148:8765 2024-03-29T08:06:05.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:06:04 smithi148 ceph-mon[44992]: [29/Mar/2024:08:06:03] ENGINE Serving on https://172.21.15.148:7150 2024-03-29T08:06:05.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:06:04 smithi148 ceph-mon[44992]: [29/Mar/2024:08:06:03] ENGINE Bus STARTED 2024-03-29T08:06:05.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:06:04 smithi148 ceph-mon[44992]: mgrmap e23: smithi148.onhgvm(active, since 2s) 2024-03-29T08:06:05.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:06:04 smithi148 ceph-mon[44992]: pgmap v4: 65 pgs: 65 active+clean; 36 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:06:07.176 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:06:06 smithi037 ceph-mon[35162]: pgmap v5: 65 pgs: 65 active+clean; 36 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:06:07.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:06:06 smithi037 ceph-mon[35162]: mgrmap e24: smithi148.onhgvm(active, since 4s) 2024-03-29T08:06:07.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:06:06 smithi148 ceph-mon[44992]: pgmap v5: 65 pgs: 65 active+clean; 36 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:06:07.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:06:06 smithi148 ceph-mon[44992]: mgrmap e24: smithi148.onhgvm(active, since 4s) 2024-03-29T08:06:08.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:06:08 smithi037 ceph-mon[35162]: pgmap v6: 65 pgs: 65 active+clean; 38 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail; 484 KiB/s wr, 48 op/s 2024-03-29T08:06:09.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:06:08 smithi148 ceph-mon[44992]: pgmap v6: 65 pgs: 65 active+clean; 38 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail; 484 KiB/s wr, 48 op/s 2024-03-29T08:06:10.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:06:09 smithi037 ceph-mon[35162]: fsmap cephfs:2 {0=cephfs.smithi037.mmwgxe=up:active,1=cephfs.smithi037.iifsmh=up:active} 2 up:standby 2024-03-29T08:06:10.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:06:09 smithi037 ceph-mon[35162]: from='mgr.14602 ' entity='mgr.smithi148.onhgvm' 2024-03-29T08:06:10.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:06:09 smithi037 ceph-mon[35162]: from='mgr.14602 ' entity='mgr.smithi148.onhgvm' 2024-03-29T08:06:10.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:06:09 smithi148 ceph-mon[44992]: fsmap cephfs:2 {0=cephfs.smithi037.mmwgxe=up:active,1=cephfs.smithi037.iifsmh=up:active} 2 up:standby 2024-03-29T08:06:10.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:06:09 smithi148 ceph-mon[44992]: from='mgr.14602 ' entity='mgr.smithi148.onhgvm' 2024-03-29T08:06:10.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:06:09 smithi148 ceph-mon[44992]: from='mgr.14602 ' entity='mgr.smithi148.onhgvm' 2024-03-29T08:06:11.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:06:10 smithi037 ceph-mon[35162]: pgmap v7: 65 pgs: 65 active+clean; 47 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail; 29 KiB/s rd, 1.7 MiB/s wr, 228 op/s 2024-03-29T08:06:11.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:06:10 smithi037 ceph-mon[35162]: from='mgr.14602 ' entity='mgr.smithi148.onhgvm' 2024-03-29T08:06:11.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:06:10 smithi037 ceph-mon[35162]: from='mgr.14602 ' entity='mgr.smithi148.onhgvm' 2024-03-29T08:06:11.305 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:06:10 smithi148 ceph-mon[44992]: pgmap v7: 65 pgs: 65 active+clean; 47 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail; 29 KiB/s rd, 1.7 MiB/s wr, 228 op/s 2024-03-29T08:06:11.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:06:10 smithi148 ceph-mon[44992]: from='mgr.14602 ' entity='mgr.smithi148.onhgvm' 2024-03-29T08:06:11.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:06:10 smithi148 ceph-mon[44992]: from='mgr.14602 ' entity='mgr.smithi148.onhgvm' 2024-03-29T08:06:13.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:06:12 smithi037 ceph-mon[35162]: pgmap v8: 65 pgs: 65 active+clean; 47 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail; 23 KiB/s rd, 1.3 MiB/s wr, 177 op/s 2024-03-29T08:06:13.305 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:06:12 smithi148 ceph-mon[44992]: pgmap v8: 65 pgs: 65 active+clean; 47 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail; 23 KiB/s rd, 1.3 MiB/s wr, 177 op/s 2024-03-29T08:06:15.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:06:14 smithi148 ceph-mon[44992]: pgmap v9: 65 pgs: 65 active+clean; 64 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 19 KiB/s rd, 2.6 MiB/s wr, 273 op/s 2024-03-29T08:06:15.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:06:14 smithi148 ceph-mon[44992]: Standby manager daemon smithi037.wjikii started 2024-03-29T08:06:15.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:06:14 smithi148 ceph-mon[44992]: from='mgr.? 172.21.15.37:0/3220144439' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi037.wjikii/crt"}]: dispatch 2024-03-29T08:06:15.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:06:14 smithi148 ceph-mon[44992]: from='mgr.? 172.21.15.37:0/3220144439' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-29T08:06:15.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:06:14 smithi148 ceph-mon[44992]: from='mgr.? 172.21.15.37:0/3220144439' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi037.wjikii/key"}]: dispatch 2024-03-29T08:06:15.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:06:14 smithi148 ceph-mon[44992]: from='mgr.? 172.21.15.37:0/3220144439' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-29T08:06:15.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:06:14 smithi037 ceph-mon[35162]: pgmap v9: 65 pgs: 65 active+clean; 64 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 19 KiB/s rd, 2.6 MiB/s wr, 273 op/s 2024-03-29T08:06:15.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:06:14 smithi037 ceph-mon[35162]: Standby manager daemon smithi037.wjikii started 2024-03-29T08:06:15.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:06:14 smithi037 ceph-mon[35162]: from='mgr.? 172.21.15.37:0/3220144439' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi037.wjikii/crt"}]: dispatch 2024-03-29T08:06:15.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:06:14 smithi037 ceph-mon[35162]: from='mgr.? 172.21.15.37:0/3220144439' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-29T08:06:15.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:06:14 smithi037 ceph-mon[35162]: from='mgr.? 172.21.15.37:0/3220144439' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi037.wjikii/key"}]: dispatch 2024-03-29T08:06:15.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:06:14 smithi037 ceph-mon[35162]: from='mgr.? 172.21.15.37:0/3220144439' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-29T08:06:16.189 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:06:15 smithi148 ceph-mon[44992]: mgrmap e25: smithi148.onhgvm(active, since 13s), standbys: smithi037.wjikii 2024-03-29T08:06:16.189 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:06:15 smithi148 ceph-mon[44992]: from='mgr.14602 172.21.15.148:0/667342922' entity='mgr.smithi148.onhgvm' cmd=[{"prefix": "mgr metadata", "who": "smithi037.wjikii", "id": "smithi037.wjikii"}]: dispatch 2024-03-29T08:06:16.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:06:15 smithi148 ceph-mon[44992]: from='mgr.14602 ' entity='mgr.smithi148.onhgvm' 2024-03-29T08:06:16.190 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:06:15 smithi148 ceph-mon[44992]: from='mgr.14602 ' entity='mgr.smithi148.onhgvm' 2024-03-29T08:06:16.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:06:15 smithi037 ceph-mon[35162]: mgrmap e25: smithi148.onhgvm(active, since 13s), standbys: smithi037.wjikii 2024-03-29T08:06:16.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:06:15 smithi037 ceph-mon[35162]: from='mgr.14602 172.21.15.148:0/667342922' entity='mgr.smithi148.onhgvm' cmd=[{"prefix": "mgr metadata", "who": "smithi037.wjikii", "id": "smithi037.wjikii"}]: dispatch 2024-03-29T08:06:16.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:06:15 smithi037 ceph-mon[35162]: from='mgr.14602 ' entity='mgr.smithi148.onhgvm' 2024-03-29T08:06:16.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:06:15 smithi037 ceph-mon[35162]: from='mgr.14602 ' entity='mgr.smithi148.onhgvm' 2024-03-29T08:06:17.305 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:06:16 smithi148 ceph-mon[44992]: pgmap v10: 65 pgs: 65 active+clean; 64 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 17 KiB/s rd, 2.4 MiB/s wr, 250 op/s 2024-03-29T08:06:17.426 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:06:16 smithi037 ceph-mon[35162]: pgmap v10: 65 pgs: 65 active+clean; 64 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 17 KiB/s rd, 2.4 MiB/s wr, 250 op/s 2024-03-29T08:06:17.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:17.490+0000 7fcc50400700 1 -- 172.21.15.37:0/1592512635 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcc48102360 msgr2=0x7fcc48104740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:06:17.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:17.490+0000 7fcc50400700 1 --2- 172.21.15.37:0/1592512635 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcc48102360 0x7fcc48104740 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fcc34004940 tx=0x7fcc34009e30 comp rx=0 tx=0).stop 2024-03-29T08:06:17.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:17.491+0000 7fcc50400700 1 -- 172.21.15.37:0/1592512635 shutdown_connections 2024-03-29T08:06:17.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:17.491+0000 7fcc50400700 1 --2- 172.21.15.37:0/1592512635 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcc48102360 0x7fcc48104740 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:06:17.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:17.491+0000 7fcc50400700 1 --2- 172.21.15.37:0/1592512635 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcc48066280 0x7fcc48101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:06:17.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:17.491+0000 7fcc50400700 1 -- 172.21.15.37:0/1592512635 >> 172.21.15.37:0/1592512635 conn(0x7fcc480f7430 msgr2=0x7fcc480f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:06:17.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:17.491+0000 7fcc50400700 1 -- 172.21.15.37:0/1592512635 shutdown_connections 2024-03-29T08:06:17.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:17.491+0000 7fcc50400700 1 -- 172.21.15.37:0/1592512635 wait complete. 2024-03-29T08:06:17.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:17.492+0000 7fcc50400700 1 Processor -- start 2024-03-29T08:06:17.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:17.493+0000 7fcc50400700 1 -- start start 2024-03-29T08:06:17.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:17.493+0000 7fcc50400700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcc48066280 0x7fcc4810fac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:06:17.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:17.493+0000 7fcc50400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcc48102360 0x7fcc48110000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:06:17.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:17.493+0000 7fcc50400700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc48110600 con 0x7fcc48102360 2024-03-29T08:06:17.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:17.493+0000 7fcc50400700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc48110e50 con 0x7fcc48066280 2024-03-29T08:06:17.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:17.493+0000 7fcc4da00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcc48066280 0x7fcc4810fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:06:17.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:17.493+0000 7fcc4da00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcc48066280 0x7fcc4810fac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.37:59170/0 (socket says 172.21.15.37:59170) 2024-03-29T08:06:17.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:17.493+0000 7fcc4da00700 1 -- 172.21.15.37:0/2646740477 learned_addr learned my addr 172.21.15.37:0/2646740477 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:06:17.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:17.494+0000 7fcc4d000700 1 --2- 172.21.15.37:0/2646740477 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcc48102360 0x7fcc48110000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:06:17.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:17.494+0000 7fcc4da00700 1 -- 172.21.15.37:0/2646740477 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcc48102360 msgr2=0x7fcc48110000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:06:17.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:17.494+0000 7fcc4da00700 1 --2- 172.21.15.37:0/2646740477 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcc48102360 0x7fcc48110000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:06:17.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:17.494+0000 7fcc4da00700 1 -- 172.21.15.37:0/2646740477 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc34005040 con 0x7fcc48066280 2024-03-29T08:06:17.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:17.495+0000 7fcc4da00700 1 --2- 172.21.15.37:0/2646740477 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcc48066280 0x7fcc4810fac0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fcc3c00eb70 tx=0x7fcc3c00b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:06:17.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:17.495+0000 7fcc45600700 1 -- 172.21.15.37:0/2646740477 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcc3c007cc0 con 0x7fcc48066280 2024-03-29T08:06:17.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:17.495+0000 7fcc45600700 1 -- 172.21.15.37:0/2646740477 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fcc3c00f040 con 0x7fcc48066280 2024-03-29T08:06:17.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:17.495+0000 7fcc45600700 1 -- 172.21.15.37:0/2646740477 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcc3c00cad0 con 0x7fcc48066280 2024-03-29T08:06:17.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:17.495+0000 7fcc50400700 1 -- 172.21.15.37:0/2646740477 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc481110d0 con 0x7fcc48066280 2024-03-29T08:06:17.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:17.495+0000 7fcc50400700 1 -- 172.21.15.37:0/2646740477 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc48111610 con 0x7fcc48066280 2024-03-29T08:06:17.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:17.497+0000 7fcc50400700 1 -- 172.21.15.37:0/2646740477 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc4805f5f0 con 0x7fcc48066280 2024-03-29T08:06:17.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:17.498+0000 7fcc45600700 1 -- 172.21.15.37:0/2646740477 <== mon.1 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 25) v1 ==== 88766+0+0 (secure 0 0 0) 0x7fcc3c00cc30 con 0x7fcc48066280 2024-03-29T08:06:17.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:17.500+0000 7fcc45600700 1 --2- 172.21.15.37:0/2646740477 >> [v2:172.21.15.148:6828/1134897184,v1:172.21.15.148:6829/1134897184] conn(0x7fcc3006af80 0x7fcc3006d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:06:17.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:17.500+0000 7fcc45600700 1 -- 172.21.15.37:0/2646740477 <== mon.1 v2:172.21.15.148:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5808+0+0 (secure 0 0 0) 0x7fcc3c089320 con 0x7fcc48066280 2024-03-29T08:06:17.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:17.500+0000 7fcc4d000700 1 --2- 172.21.15.37:0/2646740477 >> [v2:172.21.15.148:6828/1134897184,v1:172.21.15.148:6829/1134897184] conn(0x7fcc3006af80 0x7fcc3006d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:06:17.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:17.501+0000 7fcc4d000700 1 --2- 172.21.15.37:0/2646740477 >> [v2:172.21.15.148:6828/1134897184,v1:172.21.15.148:6829/1134897184] conn(0x7fcc3006af80 0x7fcc3006d430 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fcc3400b0d0 tx=0x7fcc34009e30 comp rx=0 tx=0).ready entity=mgr.14602 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:06:17.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:17.509+0000 7fcc45600700 1 -- 172.21.15.37:0/2646740477 <== mon.1 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181579 (secure 0 0 0) 0x7fcc3c04f560 con 0x7fcc48066280 2024-03-29T08:06:17.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:17.852+0000 7fcc50400700 1 -- 172.21.15.37:0/2646740477 --> [v2:172.21.15.148:6828/1134897184,v1:172.21.15.148:6829/1134897184] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fcc48111990 con 0x7fcc3006af80 2024-03-29T08:06:17.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:17.854+0000 7fcc45600700 1 -- 172.21.15.37:0/2646740477 <== mgr.14602 v2:172.21.15.148:6828/1134897184 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+310 (secure 0 0 0) 0x7fcc48111990 con 0x7fcc3006af80 2024-03-29T08:06:17.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:17.857+0000 7fcc50400700 1 -- 172.21.15.37:0/2646740477 >> [v2:172.21.15.148:6828/1134897184,v1:172.21.15.148:6829/1134897184] conn(0x7fcc3006af80 msgr2=0x7fcc3006d430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:06:17.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:17.857+0000 7fcc50400700 1 --2- 172.21.15.37:0/2646740477 >> [v2:172.21.15.148:6828/1134897184,v1:172.21.15.148:6829/1134897184] conn(0x7fcc3006af80 0x7fcc3006d430 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fcc3400b0d0 tx=0x7fcc34009e30 comp rx=0 tx=0).stop 2024-03-29T08:06:17.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:17.857+0000 7fcc50400700 1 -- 172.21.15.37:0/2646740477 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcc48066280 msgr2=0x7fcc4810fac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:06:17.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:17.857+0000 7fcc50400700 1 --2- 172.21.15.37:0/2646740477 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcc48066280 0x7fcc4810fac0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fcc3c00eb70 tx=0x7fcc3c00b040 comp rx=0 tx=0).stop 2024-03-29T08:06:17.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:17.857+0000 7fcc50400700 1 -- 172.21.15.37:0/2646740477 shutdown_connections 2024-03-29T08:06:17.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:17.857+0000 7fcc50400700 1 --2- 172.21.15.37:0/2646740477 >> [v2:172.21.15.148:6828/1134897184,v1:172.21.15.148:6829/1134897184] conn(0x7fcc3006af80 0x7fcc3006d430 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:06:17.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:17.857+0000 7fcc50400700 1 --2- 172.21.15.37:0/2646740477 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcc48066280 0x7fcc4810fac0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:06:17.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:17.857+0000 7fcc50400700 1 --2- 172.21.15.37:0/2646740477 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcc48102360 0x7fcc48110000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:06:17.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:17.857+0000 7fcc50400700 1 -- 172.21.15.37:0/2646740477 >> 172.21.15.37:0/2646740477 conn(0x7fcc480f7430 msgr2=0x7fcc480f7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:06:17.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:17.858+0000 7fcc50400700 1 -- 172.21.15.37:0/2646740477 shutdown_connections 2024-03-29T08:06:17.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:17.858+0000 7fcc50400700 1 -- 172.21.15.37:0/2646740477 wait complete. 2024-03-29T08:06:17.875 INFO:teuthology.orchestra.run.smithi037.stdout:true 2024-03-29T08:06:18.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:18.048+0000 7fa19b200700 1 -- 172.21.15.37:0/123347375 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa1940fe0d0 msgr2=0x7fa1940fe540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:06:18.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:18.048+0000 7fa19b200700 1 --2- 172.21.15.37:0/123347375 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa1940fe0d0 0x7fa1940fe540 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fa180004940 tx=0x7fa180009e30 comp rx=0 tx=0).stop 2024-03-29T08:06:18.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:18.048+0000 7fa19b200700 1 -- 172.21.15.37:0/123347375 shutdown_connections 2024-03-29T08:06:18.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:18.048+0000 7fa19b200700 1 --2- 172.21.15.37:0/123347375 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa1940fe0d0 0x7fa1940fe540 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:06:18.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:18.048+0000 7fa19b200700 1 --2- 172.21.15.37:0/123347375 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa1940fcee0 0x7fa1940fd310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:06:18.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:18.048+0000 7fa19b200700 1 -- 172.21.15.37:0/123347375 >> 172.21.15.37:0/123347375 conn(0x7fa1940f8560 msgr2=0x7fa1940fa970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:06:18.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:18.049+0000 7fa19b200700 1 -- 172.21.15.37:0/123347375 shutdown_connections 2024-03-29T08:06:18.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:18.049+0000 7fa19b200700 1 -- 172.21.15.37:0/123347375 wait complete. 2024-03-29T08:06:18.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:18.050+0000 7fa19b200700 1 Processor -- start 2024-03-29T08:06:18.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:18.050+0000 7fa19b200700 1 -- start start 2024-03-29T08:06:18.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:18.050+0000 7fa19b200700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa1940fcee0 0x7fa19410b690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:06:18.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:18.050+0000 7fa19b200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa1940fe0d0 0x7fa19410bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:06:18.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:18.050+0000 7fa19b200700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa19410c240 con 0x7fa1940fe0d0 2024-03-29T08:06:18.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:18.050+0000 7fa19b200700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa19410ca60 con 0x7fa1940fcee0 2024-03-29T08:06:18.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:18.051+0000 7fa193400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa1940fe0d0 0x7fa19410bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:06:18.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:18.051+0000 7fa193400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa1940fe0d0 0x7fa19410bbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43104/0 (socket says 172.21.15.37:43104) 2024-03-29T08:06:18.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:18.051+0000 7fa193400700 1 -- 172.21.15.37:0/3783976879 learned_addr learned my addr 172.21.15.37:0/3783976879 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:06:18.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:18.051+0000 7fa193e00700 1 --2- 172.21.15.37:0/3783976879 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa1940fcee0 0x7fa19410b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:06:18.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:18.051+0000 7fa193400700 1 -- 172.21.15.37:0/3783976879 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa1940fcee0 msgr2=0x7fa19410b690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:06:18.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:18.051+0000 7fa193400700 1 --2- 172.21.15.37:0/3783976879 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa1940fcee0 0x7fa19410b690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:06:18.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:18.051+0000 7fa193400700 1 -- 172.21.15.37:0/3783976879 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa180005040 con 0x7fa1940fe0d0 2024-03-29T08:06:18.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:18.051+0000 7fa193400700 1 --2- 172.21.15.37:0/3783976879 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa1940fe0d0 0x7fa19410bbf0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fa180008f80 tx=0x7fa180009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:06:18.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:18.052+0000 7fa18be00700 1 -- 172.21.15.37:0/3783976879 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa18000e070 con 0x7fa1940fe0d0 2024-03-29T08:06:18.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:18.052+0000 7fa18be00700 1 -- 172.21.15.37:0/3783976879 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fa18001c400 con 0x7fa1940fe0d0 2024-03-29T08:06:18.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:18.052+0000 7fa18be00700 1 -- 172.21.15.37:0/3783976879 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa18000b650 con 0x7fa1940fe0d0 2024-03-29T08:06:18.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:18.052+0000 7fa19b200700 1 -- 172.21.15.37:0/3783976879 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa19410cce0 con 0x7fa1940fe0d0 2024-03-29T08:06:18.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:18.052+0000 7fa19b200700 1 -- 172.21.15.37:0/3783976879 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa19410d240 con 0x7fa1940fe0d0 2024-03-29T08:06:18.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:18.054+0000 7fa19b200700 1 -- 172.21.15.37:0/3783976879 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa19405f5f0 con 0x7fa1940fe0d0 2024-03-29T08:06:18.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:18.054+0000 7fa18be00700 1 -- 172.21.15.37:0/3783976879 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 25) v1 ==== 88766+0+0 (secure 0 0 0) 0x7fa18001c9c0 con 0x7fa1940fe0d0 2024-03-29T08:06:18.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:18.056+0000 7fa18be00700 1 --2- 172.21.15.37:0/3783976879 >> [v2:172.21.15.148:6828/1134897184,v1:172.21.15.148:6829/1134897184] conn(0x7fa17806aec0 0x7fa17806d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:06:18.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:18.056+0000 7fa18be00700 1 -- 172.21.15.37:0/3783976879 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5808+0+0 (secure 0 0 0) 0x7fa18008aef0 con 0x7fa1940fe0d0 2024-03-29T08:06:18.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:18.056+0000 7fa193e00700 1 --2- 172.21.15.37:0/3783976879 >> [v2:172.21.15.148:6828/1134897184,v1:172.21.15.148:6829/1134897184] conn(0x7fa17806aec0 0x7fa17806d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:06:18.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:18.057+0000 7fa193e00700 1 --2- 172.21.15.37:0/3783976879 >> [v2:172.21.15.148:6828/1134897184,v1:172.21.15.148:6829/1134897184] conn(0x7fa17806aec0 0x7fa17806d370 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fa19410b460 tx=0x7fa18400b040 comp rx=0 tx=0).ready entity=mgr.14602 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:06:18.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:18.062+0000 7fa18be00700 1 -- 172.21.15.37:0/3783976879 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181579 (secure 0 0 0) 0x7fa180055340 con 0x7fa1940fe0d0 2024-03-29T08:06:18.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:06:17 smithi148 ceph-mon[44992]: from='mgr.14602 172.21.15.148:0/667342922' entity='mgr.smithi148.onhgvm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:06:18.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:06:17 smithi148 ceph-mon[44992]: from='mgr.14602 ' entity='mgr.smithi148.onhgvm' 2024-03-29T08:06:18.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:06:17 smithi148 ceph-mon[44992]: from='mgr.14602 ' entity='mgr.smithi148.onhgvm' 2024-03-29T08:06:18.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:06:17 smithi148 ceph-mon[44992]: from='mgr.14602 172.21.15.148:0/667342922' entity='mgr.smithi148.onhgvm' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-29T08:06:18.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:06:17 smithi148 ceph-mon[44992]: from='mgr.14602 ' entity='mgr.smithi148.onhgvm' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-29T08:06:18.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:06:17 smithi148 ceph-mon[44992]: from='mgr.14602 172.21.15.148:0/667342922' entity='mgr.smithi148.onhgvm' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-29T08:06:18.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:06:17 smithi148 ceph-mon[44992]: from='mgr.14602 ' entity='mgr.smithi148.onhgvm' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-29T08:06:18.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:06:17 smithi148 ceph-mon[44992]: from='mgr.14602 172.21.15.148:0/667342922' entity='mgr.smithi148.onhgvm' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-29T08:06:18.307 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:06:17 smithi148 ceph-mon[44992]: from='mgr.14602 ' entity='mgr.smithi148.onhgvm' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-29T08:06:18.307 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:06:17 smithi148 ceph-mon[44992]: from='mgr.14602 ' entity='mgr.smithi148.onhgvm' 2024-03-29T08:06:18.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:18.416+0000 7fa19b200700 1 -- 172.21.15.37:0/3783976879 --> [v2:172.21.15.148:6828/1134897184,v1:172.21.15.148:6829/1134897184] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa19410d520 con 0x7fa17806aec0 2024-03-29T08:06:18.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:18.418+0000 7fa18be00700 1 -- 172.21.15.37:0/3783976879 <== mgr.14602 v2:172.21.15.148:6828/1134897184 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+310 (secure 0 0 0) 0x7fa19410d520 con 0x7fa17806aec0 2024-03-29T08:06:18.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:18.421+0000 7fa19b200700 1 -- 172.21.15.37:0/3783976879 >> [v2:172.21.15.148:6828/1134897184,v1:172.21.15.148:6829/1134897184] conn(0x7fa17806aec0 msgr2=0x7fa17806d370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:06:18.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:18.421+0000 7fa19b200700 1 --2- 172.21.15.37:0/3783976879 >> [v2:172.21.15.148:6828/1134897184,v1:172.21.15.148:6829/1134897184] conn(0x7fa17806aec0 0x7fa17806d370 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fa19410b460 tx=0x7fa18400b040 comp rx=0 tx=0).stop 2024-03-29T08:06:18.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:18.421+0000 7fa19b200700 1 -- 172.21.15.37:0/3783976879 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa1940fe0d0 msgr2=0x7fa19410bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:06:18.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:18.421+0000 7fa19b200700 1 --2- 172.21.15.37:0/3783976879 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa1940fe0d0 0x7fa19410bbf0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fa180008f80 tx=0x7fa180009e30 comp rx=0 tx=0).stop 2024-03-29T08:06:18.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:18.422+0000 7fa19b200700 1 -- 172.21.15.37:0/3783976879 shutdown_connections 2024-03-29T08:06:18.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:18.422+0000 7fa19b200700 1 --2- 172.21.15.37:0/3783976879 >> [v2:172.21.15.148:6828/1134897184,v1:172.21.15.148:6829/1134897184] conn(0x7fa17806aec0 0x7fa17806d370 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:06:18.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:18.422+0000 7fa19b200700 1 --2- 172.21.15.37:0/3783976879 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa1940fcee0 0x7fa19410b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:06:18.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:18.422+0000 7fa19b200700 1 --2- 172.21.15.37:0/3783976879 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa1940fe0d0 0x7fa19410bbf0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:06:18.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:18.422+0000 7fa19b200700 1 -- 172.21.15.37:0/3783976879 >> 172.21.15.37:0/3783976879 conn(0x7fa1940f8560 msgr2=0x7fa1940f9b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:06:18.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:18.422+0000 7fa19b200700 1 -- 172.21.15.37:0/3783976879 shutdown_connections 2024-03-29T08:06:18.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:18.422+0000 7fa19b200700 1 -- 172.21.15.37:0/3783976879 wait complete. 2024-03-29T08:06:18.426 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:06:17 smithi037 ceph-mon[35162]: from='mgr.14602 172.21.15.148:0/667342922' entity='mgr.smithi148.onhgvm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:06:18.426 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:06:17 smithi037 ceph-mon[35162]: from='mgr.14602 ' entity='mgr.smithi148.onhgvm' 2024-03-29T08:06:18.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:06:17 smithi037 ceph-mon[35162]: from='mgr.14602 ' entity='mgr.smithi148.onhgvm' 2024-03-29T08:06:18.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:06:17 smithi037 ceph-mon[35162]: from='mgr.14602 172.21.15.148:0/667342922' entity='mgr.smithi148.onhgvm' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-29T08:06:18.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:06:17 smithi037 ceph-mon[35162]: from='mgr.14602 ' entity='mgr.smithi148.onhgvm' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-29T08:06:18.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:06:17 smithi037 ceph-mon[35162]: from='mgr.14602 172.21.15.148:0/667342922' entity='mgr.smithi148.onhgvm' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-29T08:06:18.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:06:17 smithi037 ceph-mon[35162]: from='mgr.14602 ' entity='mgr.smithi148.onhgvm' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-29T08:06:18.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:06:17 smithi037 ceph-mon[35162]: from='mgr.14602 172.21.15.148:0/667342922' entity='mgr.smithi148.onhgvm' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-29T08:06:18.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:06:17 smithi037 ceph-mon[35162]: from='mgr.14602 ' entity='mgr.smithi148.onhgvm' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-29T08:06:18.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:06:17 smithi037 ceph-mon[35162]: from='mgr.14602 ' entity='mgr.smithi148.onhgvm' 2024-03-29T08:06:18.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:18.591+0000 7f0d5d600700 1 -- 172.21.15.37:0/2816720650 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0d581002a0 msgr2=0x7f0d58100710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:06:18.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:18.591+0000 7f0d5d600700 1 --2- 172.21.15.37:0/2816720650 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0d581002a0 0x7f0d58100710 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f0d44004940 tx=0x7f0d44009e30 comp rx=0 tx=0).stop 2024-03-29T08:06:18.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:18.591+0000 7f0d5d600700 1 -- 172.21.15.37:0/2816720650 shutdown_connections 2024-03-29T08:06:18.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:18.591+0000 7f0d5d600700 1 --2- 172.21.15.37:0/2816720650 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0d581002a0 0x7f0d58100710 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:06:18.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:18.591+0000 7f0d5d600700 1 --2- 172.21.15.37:0/2816720650 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0d580feff0 0x7f0d580ff400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:06:18.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:18.591+0000 7f0d5d600700 1 -- 172.21.15.37:0/2816720650 >> 172.21.15.37:0/2816720650 conn(0x7f0d580fa950 msgr2=0x7f0d580fcd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:06:18.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:18.591+0000 7f0d5d600700 1 -- 172.21.15.37:0/2816720650 shutdown_connections 2024-03-29T08:06:18.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:18.591+0000 7f0d5d600700 1 -- 172.21.15.37:0/2816720650 wait complete. 2024-03-29T08:06:18.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:18.592+0000 7f0d5d600700 1 Processor -- start 2024-03-29T08:06:18.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:18.592+0000 7f0d5d600700 1 -- start start 2024-03-29T08:06:18.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:18.593+0000 7f0d5d600700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0d580feff0 0x7f0d5806e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:06:18.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:18.593+0000 7f0d5d600700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0d581002a0 0x7f0d5806e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:06:18.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:18.593+0000 7f0d5d600700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d5806eee0 con 0x7f0d581002a0 2024-03-29T08:06:18.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:18.593+0000 7f0d5d600700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d5806f020 con 0x7f0d580feff0 2024-03-29T08:06:18.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:18.593+0000 7f0d56000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0d581002a0 0x7f0d5806e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:06:18.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:18.593+0000 7f0d56000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0d581002a0 0x7f0d5806e9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43130/0 (socket says 172.21.15.37:43130) 2024-03-29T08:06:18.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:18.593+0000 7f0d56000700 1 -- 172.21.15.37:0/2120460065 learned_addr learned my addr 172.21.15.37:0/2120460065 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:06:18.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:18.593+0000 7f0d56a00700 1 --2- 172.21.15.37:0/2120460065 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0d580feff0 0x7f0d5806e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:06:18.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:18.594+0000 7f0d56000700 1 -- 172.21.15.37:0/2120460065 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0d580feff0 msgr2=0x7f0d5806e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:06:18.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:18.594+0000 7f0d56000700 1 --2- 172.21.15.37:0/2120460065 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0d580feff0 0x7f0d5806e460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:06:18.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:18.594+0000 7f0d56000700 1 -- 172.21.15.37:0/2120460065 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d44005040 con 0x7f0d581002a0 2024-03-29T08:06:18.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:18.594+0000 7f0d56000700 1 --2- 172.21.15.37:0/2120460065 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0d581002a0 0x7f0d5806e9a0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f0d4400b3b0 tx=0x7f0d44009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:06:18.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:18.594+0000 7f0d4ea00700 1 -- 172.21.15.37:0/2120460065 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0d4400e040 con 0x7f0d581002a0 2024-03-29T08:06:18.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:18.594+0000 7f0d4ea00700 1 -- 172.21.15.37:0/2120460065 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f0d44004290 con 0x7f0d581002a0 2024-03-29T08:06:18.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:18.595+0000 7f0d5d600700 1 -- 172.21.15.37:0/2120460065 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d5806f2a0 con 0x7f0d581002a0 2024-03-29T08:06:18.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:18.595+0000 7f0d4ea00700 1 -- 172.21.15.37:0/2120460065 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0d4401c3f0 con 0x7f0d581002a0 2024-03-29T08:06:18.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:18.595+0000 7f0d5d600700 1 -- 172.21.15.37:0/2120460065 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d5811e170 con 0x7f0d581002a0 2024-03-29T08:06:18.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:18.596+0000 7f0d5d600700 1 -- 172.21.15.37:0/2120460065 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d5805f5f0 con 0x7f0d581002a0 2024-03-29T08:06:18.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:18.598+0000 7f0d4ea00700 1 -- 172.21.15.37:0/2120460065 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 25) v1 ==== 88766+0+0 (secure 0 0 0) 0x7f0d44013420 con 0x7f0d581002a0 2024-03-29T08:06:18.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:18.599+0000 7f0d4ea00700 1 --2- 172.21.15.37:0/2120460065 >> [v2:172.21.15.148:6828/1134897184,v1:172.21.15.148:6829/1134897184] conn(0x7f0d3c06aec0 0x7f0d3c06d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:06:18.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:18.599+0000 7f0d4ea00700 1 -- 172.21.15.37:0/2120460065 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f0d4408aa90 con 0x7f0d581002a0 2024-03-29T08:06:18.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:18.600+0000 7f0d56a00700 1 --2- 172.21.15.37:0/2120460065 >> [v2:172.21.15.148:6828/1134897184,v1:172.21.15.148:6829/1134897184] conn(0x7f0d3c06aec0 0x7f0d3c06d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:06:18.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:18.601+0000 7f0d56a00700 1 --2- 172.21.15.37:0/2120460065 >> [v2:172.21.15.148:6828/1134897184,v1:172.21.15.148:6829/1134897184] conn(0x7f0d3c06aec0 0x7f0d3c06d370 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f0d5806fd20 tx=0x7f0d480091a0 comp rx=0 tx=0).ready entity=mgr.14602 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:06:18.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:18.605+0000 7f0d4ea00700 1 -- 172.21.15.37:0/2120460065 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181579 (secure 0 0 0) 0x7f0d44054fa0 con 0x7f0d581002a0 2024-03-29T08:06:19.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:19.005+0000 7f0d5d600700 1 -- 172.21.15.37:0/2120460065 --> [v2:172.21.15.148:6828/1134897184,v1:172.21.15.148:6829/1134897184] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f0d58104640 con 0x7f0d3c06aec0 2024-03-29T08:06:19.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:19.021+0000 7f0d4ea00700 1 -- 172.21.15.37:0/2120460065 <== mgr.14602 v2:172.21.15.148:6828/1134897184 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3388 (secure 0 0 0) 0x7f0d58104640 con 0x7f0d3c06aec0 2024-03-29T08:06:19.021 INFO:teuthology.orchestra.run.smithi037.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-29T08:06:19.021 INFO:teuthology.orchestra.run.smithi037.stdout:alertmanager.smithi037 smithi037 *:9093,9094 running (7m) 3s ago 9m 13.0M - 0.25.0 c8568f914cd2 a6824c801465 2024-03-29T08:06:19.021 INFO:teuthology.orchestra.run.smithi037.stdout:crash.smithi037 smithi037 running (9m) 3s ago 9m 6651k - 17.2.7-789-g4213e17c cf22eb8683be e0343935b25e 2024-03-29T08:06:19.021 INFO:teuthology.orchestra.run.smithi037.stdout:crash.smithi148 smithi148 running (8m) 9s ago 8m 6656k - 17.2.7-789-g4213e17c cf22eb8683be ff5164c552c1 2024-03-29T08:06:19.021 INFO:teuthology.orchestra.run.smithi037.stdout:grafana.smithi037 smithi037 *:3000 running (7m) 3s ago 8m 73.7M - 9.4.7 954c08fa6188 92b77c060e7a 2024-03-29T08:06:19.021 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi037.iifsmh smithi037 running (2m) 3s ago 2m 17.7M - 17.2.7-789-g4213e17c cf22eb8683be 074e1acf331f 2024-03-29T08:06:19.021 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi037.mmwgxe smithi037 running (2m) 3s ago 2m 64.9M - 17.2.7-789-g4213e17c cf22eb8683be 9a5b1e72de5c 2024-03-29T08:06:19.022 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi148.qukpjh smithi148 running (2m) 9s ago 2m 11.7M - 17.2.7-789-g4213e17c cf22eb8683be 8e469782a273 2024-03-29T08:06:19.022 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi148.sqteih smithi148 running (2m) 9s ago 2m 11.6M - 17.2.7-789-g4213e17c cf22eb8683be 8e68f858bb22 2024-03-29T08:06:19.022 INFO:teuthology.orchestra.run.smithi037.stdout:mgr.smithi037.wjikii smithi037 *:9283 running (10m) 3s ago 10m 156M - 17.2.7-789-g4213e17c cf22eb8683be 60172b53efa2 2024-03-29T08:06:19.022 INFO:teuthology.orchestra.run.smithi037.stdout:mgr.smithi148.onhgvm smithi148 *:8443,9283 running (30s) 9s ago 7m 479M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 0c02ffa63851 2024-03-29T08:06:19.022 INFO:teuthology.orchestra.run.smithi037.stdout:mon.smithi037 smithi037 running (10m) 3s ago 10m 48.6M 2048M 17.2.7-789-g4213e17c cf22eb8683be bb2101505730 2024-03-29T08:06:19.022 INFO:teuthology.orchestra.run.smithi037.stdout:mon.smithi148 smithi148 running (7m) 9s ago 7m 38.5M 2048M 17.2.7-789-g4213e17c cf22eb8683be 18839ed2952a 2024-03-29T08:06:19.022 INFO:teuthology.orchestra.run.smithi037.stdout:node-exporter.smithi037 smithi037 *:9100 running (9m) 3s ago 9m 7750k - 1.5.0 0da6a335fe13 eb3242f9dc57 2024-03-29T08:06:19.022 INFO:teuthology.orchestra.run.smithi037.stdout:node-exporter.smithi148 smithi148 *:9100 running (8m) 9s ago 8m 8014k - 1.5.0 0da6a335fe13 bb455992ed53 2024-03-29T08:06:19.022 INFO:teuthology.orchestra.run.smithi037.stdout:osd.0 smithi037 running (7m) 3s ago 7m 67.4M 1619M 17.2.7-789-g4213e17c cf22eb8683be 559fccd0ebe1 2024-03-29T08:06:19.022 INFO:teuthology.orchestra.run.smithi037.stdout:osd.1 smithi037 running (6m) 3s ago 6m 68.5M 1619M 17.2.7-789-g4213e17c cf22eb8683be 5582cff11b7a 2024-03-29T08:06:19.022 INFO:teuthology.orchestra.run.smithi037.stdout:osd.2 smithi037 running (5m) 3s ago 5m 60.6M 1619M 17.2.7-789-g4213e17c cf22eb8683be f809bfab13e5 2024-03-29T08:06:19.022 INFO:teuthology.orchestra.run.smithi037.stdout:osd.3 smithi148 running (5m) 9s ago 5m 67.0M 5372M 17.2.7-789-g4213e17c cf22eb8683be a0a8224bb4b0 2024-03-29T08:06:19.022 INFO:teuthology.orchestra.run.smithi037.stdout:osd.4 smithi148 running (4m) 9s ago 4m 63.7M 5372M 17.2.7-789-g4213e17c cf22eb8683be a00b69047ed7 2024-03-29T08:06:19.022 INFO:teuthology.orchestra.run.smithi037.stdout:osd.5 smithi148 running (4m) 9s ago 4m 65.5M 5372M 17.2.7-789-g4213e17c cf22eb8683be 044a3f7db7f8 2024-03-29T08:06:19.023 INFO:teuthology.orchestra.run.smithi037.stdout:prometheus.smithi037 smithi037 *:9095 running (7m) 3s ago 8m 53.2M - 2.43.0 a07b618ecd1d e8c23666a746 2024-03-29T08:06:19.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:19.024+0000 7f0d5d600700 1 -- 172.21.15.37:0/2120460065 >> [v2:172.21.15.148:6828/1134897184,v1:172.21.15.148:6829/1134897184] conn(0x7f0d3c06aec0 msgr2=0x7f0d3c06d370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:06:19.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:19.024+0000 7f0d5d600700 1 --2- 172.21.15.37:0/2120460065 >> [v2:172.21.15.148:6828/1134897184,v1:172.21.15.148:6829/1134897184] conn(0x7f0d3c06aec0 0x7f0d3c06d370 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f0d5806fd20 tx=0x7f0d480091a0 comp rx=0 tx=0).stop 2024-03-29T08:06:19.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:19.024+0000 7f0d5d600700 1 -- 172.21.15.37:0/2120460065 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0d581002a0 msgr2=0x7f0d5806e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:06:19.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:19.024+0000 7f0d5d600700 1 --2- 172.21.15.37:0/2120460065 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0d581002a0 0x7f0d5806e9a0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f0d4400b3b0 tx=0x7f0d44009e30 comp rx=0 tx=0).stop 2024-03-29T08:06:19.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:19.025+0000 7f0d5d600700 1 -- 172.21.15.37:0/2120460065 shutdown_connections 2024-03-29T08:06:19.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:19.025+0000 7f0d5d600700 1 --2- 172.21.15.37:0/2120460065 >> [v2:172.21.15.148:6828/1134897184,v1:172.21.15.148:6829/1134897184] conn(0x7f0d3c06aec0 0x7f0d3c06d370 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:06:19.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:19.025+0000 7f0d5d600700 1 --2- 172.21.15.37:0/2120460065 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0d580feff0 0x7f0d5806e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:06:19.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:19.025+0000 7f0d5d600700 1 --2- 172.21.15.37:0/2120460065 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0d581002a0 0x7f0d5806e9a0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:06:19.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:19.025+0000 7f0d5d600700 1 -- 172.21.15.37:0/2120460065 >> 172.21.15.37:0/2120460065 conn(0x7f0d580fa950 msgr2=0x7f0d580fcfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:06:19.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:19.025+0000 7f0d5d600700 1 -- 172.21.15.37:0/2120460065 shutdown_connections 2024-03-29T08:06:19.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:19.025+0000 7f0d5d600700 1 -- 172.21.15.37:0/2120460065 wait complete. 2024-03-29T08:06:19.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:19.242+0000 7f04b0400700 1 -- 172.21.15.37:0/1005731518 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f04a80fcee0 msgr2=0x7f04a80fd310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:06:19.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:19.242+0000 7f04b0400700 1 --2- 172.21.15.37:0/1005731518 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f04a80fcee0 0x7f04a80fd310 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f049c004940 tx=0x7f049c009e30 comp rx=0 tx=0).stop 2024-03-29T08:06:19.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:19.242+0000 7f04b0400700 1 -- 172.21.15.37:0/1005731518 shutdown_connections 2024-03-29T08:06:19.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:19.242+0000 7f04b0400700 1 --2- 172.21.15.37:0/1005731518 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f04a80fe0d0 0x7f04a80fe540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:06:19.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:19.242+0000 7f04b0400700 1 --2- 172.21.15.37:0/1005731518 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f04a80fcee0 0x7f04a80fd310 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:06:19.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:19.242+0000 7f04b0400700 1 -- 172.21.15.37:0/1005731518 >> 172.21.15.37:0/1005731518 conn(0x7f04a80f8560 msgr2=0x7f04a80fa970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:06:19.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:19.242+0000 7f04b0400700 1 -- 172.21.15.37:0/1005731518 shutdown_connections 2024-03-29T08:06:19.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:19.242+0000 7f04b0400700 1 -- 172.21.15.37:0/1005731518 wait complete. 2024-03-29T08:06:19.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:19.244+0000 7f04b0400700 1 Processor -- start 2024-03-29T08:06:19.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:19.244+0000 7f04b0400700 1 -- start start 2024-03-29T08:06:19.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:19.244+0000 7f04b0400700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f04a80fcee0 0x7f04a810fad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:06:19.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:19.244+0000 7f04b0400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f04a80fe0d0 0x7f04a8110010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:06:19.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:19.244+0000 7f04b0400700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04a8110610 con 0x7f04a80fe0d0 2024-03-29T08:06:19.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:19.244+0000 7f04b0400700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04a8110e60 con 0x7f04a80fcee0 2024-03-29T08:06:19.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:19.245+0000 7f04ad000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f04a80fe0d0 0x7f04a8110010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:06:19.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:19.245+0000 7f04ad000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f04a80fe0d0 0x7f04a8110010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43146/0 (socket says 172.21.15.37:43146) 2024-03-29T08:06:19.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:19.245+0000 7f04ad000700 1 -- 172.21.15.37:0/205159782 learned_addr learned my addr 172.21.15.37:0/205159782 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:06:19.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:19.245+0000 7f04ada00700 1 --2- 172.21.15.37:0/205159782 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f04a80fcee0 0x7f04a810fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:06:19.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:19.245+0000 7f04ad000700 1 -- 172.21.15.37:0/205159782 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f04a80fcee0 msgr2=0x7f04a810fad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:06:19.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:19.245+0000 7f04ad000700 1 --2- 172.21.15.37:0/205159782 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f04a80fcee0 0x7f04a810fad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:06:19.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:19.245+0000 7f04ad000700 1 -- 172.21.15.37:0/205159782 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f049c005040 con 0x7f04a80fe0d0 2024-03-29T08:06:19.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:19.246+0000 7f04ad000700 1 --2- 172.21.15.37:0/205159782 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f04a80fe0d0 0x7f04a8110010 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f049400eb40 tx=0x7f049400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:06:19.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:19.246+0000 7f04a5600700 1 -- 172.21.15.37:0/205159782 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0494007cc0 con 0x7f04a80fe0d0 2024-03-29T08:06:19.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:19.246+0000 7f04a5600700 1 -- 172.21.15.37:0/205159782 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f049400f040 con 0x7f04a80fe0d0 2024-03-29T08:06:19.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:19.246+0000 7f04a5600700 1 -- 172.21.15.37:0/205159782 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f049400c990 con 0x7f04a80fe0d0 2024-03-29T08:06:19.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:19.247+0000 7f04b0400700 1 -- 172.21.15.37:0/205159782 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04a81110e0 con 0x7f04a80fe0d0 2024-03-29T08:06:19.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:19.247+0000 7f04b0400700 1 -- 172.21.15.37:0/205159782 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04a8111620 con 0x7f04a80fe0d0 2024-03-29T08:06:19.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:19.248+0000 7f04b0400700 1 -- 172.21.15.37:0/205159782 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04a8101a60 con 0x7f04a80fe0d0 2024-03-29T08:06:19.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:19.250+0000 7f04a5600700 1 -- 172.21.15.37:0/205159782 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 25) v1 ==== 88766+0+0 (secure 0 0 0) 0x7f049400caf0 con 0x7f04a80fe0d0 2024-03-29T08:06:19.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:19.252+0000 7f04a5600700 1 --2- 172.21.15.37:0/205159782 >> [v2:172.21.15.148:6828/1134897184,v1:172.21.15.148:6829/1134897184] conn(0x7f049006aec0 0x7f049006d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:06:19.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:19.252+0000 7f04a5600700 1 -- 172.21.15.37:0/205159782 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f0494089490 con 0x7f04a80fe0d0 2024-03-29T08:06:19.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:19.252+0000 7f04ada00700 1 --2- 172.21.15.37:0/205159782 >> [v2:172.21.15.148:6828/1134897184,v1:172.21.15.148:6829/1134897184] conn(0x7f049006aec0 0x7f049006d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:06:19.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:19.253+0000 7f04ada00700 1 --2- 172.21.15.37:0/205159782 >> [v2:172.21.15.148:6828/1134897184,v1:172.21.15.148:6829/1134897184] conn(0x7f049006aec0 0x7f049006d370 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f049c00b0d0 tx=0x7f049c009e30 comp rx=0 tx=0).ready entity=mgr.14602 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:06:19.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:19.260+0000 7f04a5600700 1 -- 172.21.15.37:0/205159782 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181579 (secure 0 0 0) 0x7f049404f560 con 0x7f04a80fe0d0 2024-03-29T08:06:19.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:06:18 smithi148 ceph-mon[44992]: Adjusting osd_memory_target on smithi037 to 1619M 2024-03-29T08:06:19.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:06:18 smithi148 ceph-mon[44992]: from='client.24453 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:06:19.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:06:18 smithi148 ceph-mon[44992]: pgmap v11: 65 pgs: 65 active+clean; 67 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 17 KiB/s rd, 2.7 MiB/s wr, 275 op/s 2024-03-29T08:06:19.428 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:06:18 smithi037 ceph-mon[35162]: Adjusting osd_memory_target on smithi037 to 1619M 2024-03-29T08:06:19.428 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:06:18 smithi037 ceph-mon[35162]: from='client.24453 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:06:19.428 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:06:18 smithi037 ceph-mon[35162]: pgmap v11: 65 pgs: 65 active+clean; 67 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 17 KiB/s rd, 2.7 MiB/s wr, 275 op/s 2024-03-29T08:06:19.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:19.780+0000 7f04b0400700 1 -- 172.21.15.37:0/205159782 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f04a805f5f0 con 0x7f04a80fe0d0 2024-03-29T08:06:19.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:19.781+0000 7f04a5600700 1 -- 172.21.15.37:0/205159782 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+850 (secure 0 0 0) 0x7f04940573a0 con 0x7f04a80fe0d0 2024-03-29T08:06:19.781 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-29T08:06:19.781 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": { 2024-03-29T08:06:19.781 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 17.2.7-789-g4213e17c (4213e17c4de67016ed14f55d22503f4923b3a901) quincy (stable)": 2 2024-03-29T08:06:19.781 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:06:19.781 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": { 2024-03-29T08:06:19.781 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 17.2.7-789-g4213e17c (4213e17c4de67016ed14f55d22503f4923b3a901) quincy (stable)": 1, 2024-03-29T08:06:19.781 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 1 2024-03-29T08:06:19.781 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:06:19.781 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": { 2024-03-29T08:06:19.781 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 17.2.7-789-g4213e17c (4213e17c4de67016ed14f55d22503f4923b3a901) quincy (stable)": 6 2024-03-29T08:06:19.781 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:06:19.781 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": { 2024-03-29T08:06:19.782 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 17.2.7-789-g4213e17c (4213e17c4de67016ed14f55d22503f4923b3a901) quincy (stable)": 4 2024-03-29T08:06:19.782 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:06:19.782 INFO:teuthology.orchestra.run.smithi037.stdout: "overall": { 2024-03-29T08:06:19.782 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 17.2.7-789-g4213e17c (4213e17c4de67016ed14f55d22503f4923b3a901) quincy (stable)": 13, 2024-03-29T08:06:19.782 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 1 2024-03-29T08:06:19.782 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-29T08:06:19.782 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-29T08:06:19.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:19.784+0000 7f04b0400700 1 -- 172.21.15.37:0/205159782 >> [v2:172.21.15.148:6828/1134897184,v1:172.21.15.148:6829/1134897184] conn(0x7f049006aec0 msgr2=0x7f049006d370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:06:19.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:19.784+0000 7f04b0400700 1 --2- 172.21.15.37:0/205159782 >> [v2:172.21.15.148:6828/1134897184,v1:172.21.15.148:6829/1134897184] conn(0x7f049006aec0 0x7f049006d370 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f049c00b0d0 tx=0x7f049c009e30 comp rx=0 tx=0).stop 2024-03-29T08:06:19.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:19.784+0000 7f04b0400700 1 -- 172.21.15.37:0/205159782 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f04a80fe0d0 msgr2=0x7f04a8110010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:06:19.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:19.784+0000 7f04b0400700 1 --2- 172.21.15.37:0/205159782 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f04a80fe0d0 0x7f04a8110010 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f049400eb40 tx=0x7f049400b040 comp rx=0 tx=0).stop 2024-03-29T08:06:19.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:19.784+0000 7f04b0400700 1 -- 172.21.15.37:0/205159782 shutdown_connections 2024-03-29T08:06:19.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:19.784+0000 7f04b0400700 1 --2- 172.21.15.37:0/205159782 >> [v2:172.21.15.148:6828/1134897184,v1:172.21.15.148:6829/1134897184] conn(0x7f049006aec0 0x7f049006d370 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:06:19.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:19.784+0000 7f04b0400700 1 --2- 172.21.15.37:0/205159782 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f04a80fcee0 0x7f04a810fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:06:19.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:19.784+0000 7f04b0400700 1 --2- 172.21.15.37:0/205159782 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f04a80fe0d0 0x7f04a8110010 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:06:19.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:19.784+0000 7f04b0400700 1 -- 172.21.15.37:0/205159782 >> 172.21.15.37:0/205159782 conn(0x7f04a80f8560 msgr2=0x7f04a80f9c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:06:19.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:19.784+0000 7f04b0400700 1 -- 172.21.15.37:0/205159782 shutdown_connections 2024-03-29T08:06:19.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:19.784+0000 7f04b0400700 1 -- 172.21.15.37:0/205159782 wait complete. 2024-03-29T08:06:19.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:19.982+0000 7fe61a400700 1 -- 172.21.15.37:0/1201202365 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe614071f50 msgr2=0x7fe614070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:06:19.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:19.982+0000 7fe61a400700 1 --2- 172.21.15.37:0/1201202365 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe614071f50 0x7fe614070460 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fe604004940 tx=0x7fe604009e30 comp rx=0 tx=0).stop 2024-03-29T08:06:19.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:19.982+0000 7fe61a400700 1 -- 172.21.15.37:0/1201202365 shutdown_connections 2024-03-29T08:06:19.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:19.982+0000 7fe61a400700 1 --2- 172.21.15.37:0/1201202365 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe6140709a0 0x7fe614070e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:06:19.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:19.982+0000 7fe61a400700 1 --2- 172.21.15.37:0/1201202365 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe614071f50 0x7fe614070460 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:06:19.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:19.982+0000 7fe61a400700 1 -- 172.21.15.37:0/1201202365 >> 172.21.15.37:0/1201202365 conn(0x7fe6140f8560 msgr2=0x7fe6140fa970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:06:19.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:19.982+0000 7fe61a400700 1 -- 172.21.15.37:0/1201202365 shutdown_connections 2024-03-29T08:06:19.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:19.982+0000 7fe61a400700 1 -- 172.21.15.37:0/1201202365 wait complete. 2024-03-29T08:06:19.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:19.983+0000 7fe61a400700 1 Processor -- start 2024-03-29T08:06:19.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:19.983+0000 7fe61a400700 1 -- start start 2024-03-29T08:06:19.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:19.984+0000 7fe61a400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe6140709a0 0x7fe6140fcca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:06:19.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:19.984+0000 7fe61a400700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe614071f50 0x7fe6140fd1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:06:19.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:19.984+0000 7fe61a400700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6140fd830 con 0x7fe6140709a0 2024-03-29T08:06:19.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:19.984+0000 7fe61a400700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6140fe080 con 0x7fe614071f50 2024-03-29T08:06:19.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:19.984+0000 7fe613400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe6140709a0 0x7fe6140fcca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:06:19.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:19.984+0000 7fe613400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe6140709a0 0x7fe6140fcca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58618/0 (socket says 172.21.15.37:58618) 2024-03-29T08:06:19.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:19.984+0000 7fe613400700 1 -- 172.21.15.37:0/3684228026 learned_addr learned my addr 172.21.15.37:0/3684228026 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:06:19.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:19.984+0000 7fe612a00700 1 --2- 172.21.15.37:0/3684228026 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe614071f50 0x7fe6140fd1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:06:19.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:19.985+0000 7fe613400700 1 -- 172.21.15.37:0/3684228026 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe614071f50 msgr2=0x7fe6140fd1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:06:19.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:19.985+0000 7fe613400700 1 --2- 172.21.15.37:0/3684228026 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe614071f50 0x7fe6140fd1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:06:19.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:19.985+0000 7fe613400700 1 -- 172.21.15.37:0/3684228026 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe604005040 con 0x7fe6140709a0 2024-03-29T08:06:19.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:19.985+0000 7fe613400700 1 --2- 172.21.15.37:0/3684228026 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe6140709a0 0x7fe6140fcca0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fe604000c00 tx=0x7fe604009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:06:19.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:19.986+0000 7fe60b400700 1 -- 172.21.15.37:0/3684228026 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe60400e070 con 0x7fe6140709a0 2024-03-29T08:06:19.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:19.986+0000 7fe61a400700 1 -- 172.21.15.37:0/3684228026 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6140fe300 con 0x7fe6140709a0 2024-03-29T08:06:19.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:19.986+0000 7fe60b400700 1 -- 172.21.15.37:0/3684228026 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fe6040093e0 con 0x7fe6140709a0 2024-03-29T08:06:19.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:19.986+0000 7fe60b400700 1 -- 172.21.15.37:0/3684228026 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe60401cd70 con 0x7fe6140709a0 2024-03-29T08:06:19.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:19.986+0000 7fe61a400700 1 -- 172.21.15.37:0/3684228026 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6140fe860 con 0x7fe6140709a0 2024-03-29T08:06:19.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:19.988+0000 7fe61a400700 1 -- 172.21.15.37:0/3684228026 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe61405f5f0 con 0x7fe6140709a0 2024-03-29T08:06:19.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:19.989+0000 7fe60b400700 1 -- 172.21.15.37:0/3684228026 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 25) v1 ==== 88766+0+0 (secure 0 0 0) 0x7fe604013a50 con 0x7fe6140709a0 2024-03-29T08:06:19.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:19.990+0000 7fe60b400700 1 --2- 172.21.15.37:0/3684228026 >> [v2:172.21.15.148:6828/1134897184,v1:172.21.15.148:6829/1134897184] conn(0x7fe5f806aec0 0x7fe5f806d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:06:19.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:19.991+0000 7fe60b400700 1 -- 172.21.15.37:0/3684228026 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5808+0+0 (secure 0 0 0) 0x7fe60408ad60 con 0x7fe6140709a0 2024-03-29T08:06:19.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:19.991+0000 7fe612a00700 1 --2- 172.21.15.37:0/3684228026 >> [v2:172.21.15.148:6828/1134897184,v1:172.21.15.148:6829/1134897184] conn(0x7fe5f806aec0 0x7fe5f806d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:06:19.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:19.992+0000 7fe612a00700 1 --2- 172.21.15.37:0/3684228026 >> [v2:172.21.15.148:6828/1134897184,v1:172.21.15.148:6829/1134897184] conn(0x7fe5f806aec0 0x7fe5f806d370 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fe600009b80 tx=0x7fe60000b040 comp rx=0 tx=0).ready entity=mgr.14602 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:06:20.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:20.004+0000 7fe60b400700 1 -- 172.21.15.37:0/3684228026 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181579 (secure 0 0 0) 0x7fe604055270 con 0x7fe6140709a0 2024-03-29T08:06:20.233 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:06:19 smithi148 ceph-mon[44992]: from='client.14636 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:06:20.233 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:06:19 smithi148 ceph-mon[44992]: from='client.14640 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:06:20.233 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:06:19 smithi148 ceph-mon[44992]: from='client.? 172.21.15.37:0/205159782' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:06:20.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:20.386+0000 7fe61a400700 1 -- 172.21.15.37:0/3684228026 --> [v2:172.21.15.148:6828/1134897184,v1:172.21.15.148:6829/1134897184] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe6140feb40 con 0x7fe5f806aec0 2024-03-29T08:06:20.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:20.388+0000 7fe60b400700 1 -- 172.21.15.37:0/3684228026 <== mgr.14602 v2:172.21.15.148:6828/1134897184 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+310 (secure 0 0 0) 0x7fe6140feb40 con 0x7fe5f806aec0 2024-03-29T08:06:20.388 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-29T08:06:20.388 INFO:teuthology.orchestra.run.smithi037.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59", 2024-03-29T08:06:20.388 INFO:teuthology.orchestra.run.smithi037.stdout: "in_progress": true, 2024-03-29T08:06:20.388 INFO:teuthology.orchestra.run.smithi037.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-03-29T08:06:20.388 INFO:teuthology.orchestra.run.smithi037.stdout: "services_complete": [], 2024-03-29T08:06:20.388 INFO:teuthology.orchestra.run.smithi037.stdout: "progress": "1/2 daemons upgraded", 2024-03-29T08:06:20.388 INFO:teuthology.orchestra.run.smithi037.stdout: "message": "", 2024-03-29T08:06:20.388 INFO:teuthology.orchestra.run.smithi037.stdout: "is_paused": false 2024-03-29T08:06:20.388 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-29T08:06:20.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:20.390+0000 7fe61a400700 1 -- 172.21.15.37:0/3684228026 >> [v2:172.21.15.148:6828/1134897184,v1:172.21.15.148:6829/1134897184] conn(0x7fe5f806aec0 msgr2=0x7fe5f806d370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:06:20.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:20.390+0000 7fe61a400700 1 --2- 172.21.15.37:0/3684228026 >> [v2:172.21.15.148:6828/1134897184,v1:172.21.15.148:6829/1134897184] conn(0x7fe5f806aec0 0x7fe5f806d370 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fe600009b80 tx=0x7fe60000b040 comp rx=0 tx=0).stop 2024-03-29T08:06:20.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:20.390+0000 7fe61a400700 1 -- 172.21.15.37:0/3684228026 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe6140709a0 msgr2=0x7fe6140fcca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:06:20.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:20.390+0000 7fe61a400700 1 --2- 172.21.15.37:0/3684228026 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe6140709a0 0x7fe6140fcca0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fe604000c00 tx=0x7fe604009e30 comp rx=0 tx=0).stop 2024-03-29T08:06:20.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:20.391+0000 7fe61a400700 1 -- 172.21.15.37:0/3684228026 shutdown_connections 2024-03-29T08:06:20.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:20.391+0000 7fe61a400700 1 --2- 172.21.15.37:0/3684228026 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe6140709a0 0x7fe6140fcca0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:06:20.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:20.391+0000 7fe61a400700 1 --2- 172.21.15.37:0/3684228026 >> [v2:172.21.15.148:6828/1134897184,v1:172.21.15.148:6829/1134897184] conn(0x7fe5f806aec0 0x7fe5f806d370 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:06:20.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:20.391+0000 7fe61a400700 1 --2- 172.21.15.37:0/3684228026 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe614071f50 0x7fe6140fd1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:06:20.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:20.391+0000 7fe61a400700 1 -- 172.21.15.37:0/3684228026 >> 172.21.15.37:0/3684228026 conn(0x7fe6140f8560 msgr2=0x7fe6140f9cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:06:20.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:20.391+0000 7fe61a400700 1 -- 172.21.15.37:0/3684228026 shutdown_connections 2024-03-29T08:06:20.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:20.391+0000 7fe61a400700 1 -- 172.21.15.37:0/3684228026 wait complete. 2024-03-29T08:06:20.426 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:06:19 smithi037 ceph-mon[35162]: from='client.14636 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:06:20.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:06:19 smithi037 ceph-mon[35162]: from='client.14640 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:06:20.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:06:19 smithi037 ceph-mon[35162]: from='client.? 172.21.15.37:0/205159782' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:06:21.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:06:20 smithi148 ceph-mon[44992]: pgmap v12: 65 pgs: 65 active+clean; 80 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 17 KiB/s rd, 3.7 MiB/s wr, 390 op/s 2024-03-29T08:06:21.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:06:20 smithi148 ceph-mon[44992]: from='mgr.14602 ' entity='mgr.smithi148.onhgvm' 2024-03-29T08:06:21.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:06:20 smithi148 ceph-mon[44992]: from='mgr.14602 ' entity='mgr.smithi148.onhgvm' 2024-03-29T08:06:21.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:06:20 smithi148 ceph-mon[44992]: from='mgr.14602 172.21.15.148:0/667342922' entity='mgr.smithi148.onhgvm' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-29T08:06:21.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:06:20 smithi148 ceph-mon[44992]: from='mgr.14602 ' entity='mgr.smithi148.onhgvm' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-29T08:06:21.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:06:20 smithi148 ceph-mon[44992]: from='mgr.14602 172.21.15.148:0/667342922' entity='mgr.smithi148.onhgvm' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-29T08:06:21.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:06:20 smithi148 ceph-mon[44992]: from='mgr.14602 ' entity='mgr.smithi148.onhgvm' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-29T08:06:21.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:06:20 smithi148 ceph-mon[44992]: from='mgr.14602 172.21.15.148:0/667342922' entity='mgr.smithi148.onhgvm' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-29T08:06:21.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:06:20 smithi148 ceph-mon[44992]: from='mgr.14602 ' entity='mgr.smithi148.onhgvm' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-29T08:06:21.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:06:20 smithi148 ceph-mon[44992]: from='mgr.14602 ' entity='mgr.smithi148.onhgvm' 2024-03-29T08:06:21.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:06:20 smithi148 ceph-mon[44992]: from='mgr.14602 172.21.15.148:0/667342922' entity='mgr.smithi148.onhgvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T08:06:21.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:06:20 smithi148 ceph-mon[44992]: from='mgr.14602 172.21.15.148:0/667342922' entity='mgr.smithi148.onhgvm' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T08:06:21.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:06:20 smithi037 ceph-mon[35162]: pgmap v12: 65 pgs: 65 active+clean; 80 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 17 KiB/s rd, 3.7 MiB/s wr, 390 op/s 2024-03-29T08:06:21.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:06:20 smithi037 ceph-mon[35162]: from='mgr.14602 ' entity='mgr.smithi148.onhgvm' 2024-03-29T08:06:21.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:06:20 smithi037 ceph-mon[35162]: from='mgr.14602 ' entity='mgr.smithi148.onhgvm' 2024-03-29T08:06:21.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:06:20 smithi037 ceph-mon[35162]: from='mgr.14602 172.21.15.148:0/667342922' entity='mgr.smithi148.onhgvm' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-29T08:06:21.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:06:20 smithi037 ceph-mon[35162]: from='mgr.14602 ' entity='mgr.smithi148.onhgvm' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-29T08:06:21.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:06:20 smithi037 ceph-mon[35162]: from='mgr.14602 172.21.15.148:0/667342922' entity='mgr.smithi148.onhgvm' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-29T08:06:21.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:06:20 smithi037 ceph-mon[35162]: from='mgr.14602 ' entity='mgr.smithi148.onhgvm' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-29T08:06:21.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:06:20 smithi037 ceph-mon[35162]: from='mgr.14602 172.21.15.148:0/667342922' entity='mgr.smithi148.onhgvm' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-29T08:06:21.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:06:20 smithi037 ceph-mon[35162]: from='mgr.14602 ' entity='mgr.smithi148.onhgvm' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-29T08:06:21.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:06:20 smithi037 ceph-mon[35162]: from='mgr.14602 ' entity='mgr.smithi148.onhgvm' 2024-03-29T08:06:21.428 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:06:20 smithi037 ceph-mon[35162]: from='mgr.14602 172.21.15.148:0/667342922' entity='mgr.smithi148.onhgvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T08:06:21.428 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:06:20 smithi037 ceph-mon[35162]: from='mgr.14602 172.21.15.148:0/667342922' entity='mgr.smithi148.onhgvm' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T08:06:22.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:06:21 smithi148 ceph-mon[44992]: from='client.14648 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:06:22.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:06:21 smithi148 ceph-mon[44992]: Adjusting osd_memory_target on smithi148 to 2642M 2024-03-29T08:06:22.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:06:21 smithi148 ceph-mon[44992]: Updating smithi037:/etc/ceph/ceph.conf 2024-03-29T08:06:22.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:06:21 smithi148 ceph-mon[44992]: Updating smithi148:/etc/ceph/ceph.conf 2024-03-29T08:06:22.426 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:06:21 smithi037 ceph-mon[35162]: from='client.14648 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:06:22.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:06:21 smithi037 ceph-mon[35162]: Adjusting osd_memory_target on smithi148 to 2642M 2024-03-29T08:06:22.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:06:21 smithi037 ceph-mon[35162]: Updating smithi037:/etc/ceph/ceph.conf 2024-03-29T08:06:22.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:06:21 smithi037 ceph-mon[35162]: Updating smithi148:/etc/ceph/ceph.conf 2024-03-29T08:06:23.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:06:22 smithi148 ceph-mon[44992]: Updating smithi148:/var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/config/ceph.conf 2024-03-29T08:06:23.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:06:22 smithi148 ceph-mon[44992]: Updating smithi037:/var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/config/ceph.conf 2024-03-29T08:06:23.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:06:22 smithi148 ceph-mon[44992]: pgmap v13: 65 pgs: 65 active+clean; 80 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 2.9 MiB/s wr, 278 op/s 2024-03-29T08:06:23.426 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:06:22 smithi037 ceph-mon[35162]: Updating smithi148:/var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/config/ceph.conf 2024-03-29T08:06:23.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:06:22 smithi037 ceph-mon[35162]: Updating smithi037:/var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/config/ceph.conf 2024-03-29T08:06:23.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:06:22 smithi037 ceph-mon[35162]: pgmap v13: 65 pgs: 65 active+clean; 80 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 2.9 MiB/s wr, 278 op/s 2024-03-29T08:06:24.305 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:06:23 smithi148 ceph-mon[44992]: Updating smithi148:/etc/ceph/ceph.client.admin.keyring 2024-03-29T08:06:24.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:06:23 smithi148 ceph-mon[44992]: Updating smithi037:/etc/ceph/ceph.client.admin.keyring 2024-03-29T08:06:24.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:06:23 smithi148 ceph-mon[44992]: Updating smithi148:/var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/config/ceph.client.admin.keyring 2024-03-29T08:06:24.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:06:23 smithi148 ceph-mon[44992]: from='mgr.14602 ' entity='mgr.smithi148.onhgvm' 2024-03-29T08:06:24.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:06:23 smithi148 ceph-mon[44992]: from='mgr.14602 ' entity='mgr.smithi148.onhgvm' 2024-03-29T08:06:24.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:06:23 smithi148 ceph-mon[44992]: from='mgr.14602 ' entity='mgr.smithi148.onhgvm' 2024-03-29T08:06:24.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:06:23 smithi148 ceph-mon[44992]: from='mgr.14602 ' entity='mgr.smithi148.onhgvm' 2024-03-29T08:06:24.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:06:23 smithi148 ceph-mon[44992]: from='mgr.14602 ' entity='mgr.smithi148.onhgvm' 2024-03-29T08:06:24.426 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:06:23 smithi037 ceph-mon[35162]: Updating smithi148:/etc/ceph/ceph.client.admin.keyring 2024-03-29T08:06:24.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:06:23 smithi037 ceph-mon[35162]: Updating smithi037:/etc/ceph/ceph.client.admin.keyring 2024-03-29T08:06:24.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:06:23 smithi037 ceph-mon[35162]: Updating smithi148:/var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/config/ceph.client.admin.keyring 2024-03-29T08:06:24.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:06:23 smithi037 ceph-mon[35162]: from='mgr.14602 ' entity='mgr.smithi148.onhgvm' 2024-03-29T08:06:24.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:06:23 smithi037 ceph-mon[35162]: from='mgr.14602 ' entity='mgr.smithi148.onhgvm' 2024-03-29T08:06:24.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:06:23 smithi037 ceph-mon[35162]: from='mgr.14602 ' entity='mgr.smithi148.onhgvm' 2024-03-29T08:06:24.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:06:23 smithi037 ceph-mon[35162]: from='mgr.14602 ' entity='mgr.smithi148.onhgvm' 2024-03-29T08:06:24.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:06:23 smithi037 ceph-mon[35162]: from='mgr.14602 ' entity='mgr.smithi148.onhgvm' 2024-03-29T08:06:25.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:06:24 smithi037 ceph-mon[35162]: Updating smithi037:/var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/config/ceph.client.admin.keyring 2024-03-29T08:06:25.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:06:24 smithi037 ceph-mon[35162]: pgmap v14: 65 pgs: 65 active+clean; 98 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 4.5 MiB/s wr, 423 op/s 2024-03-29T08:06:25.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:06:24 smithi037 ceph-mon[35162]: from='mgr.14602 ' entity='mgr.smithi148.onhgvm' 2024-03-29T08:06:25.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:06:24 smithi037 ceph-mon[35162]: from='mgr.14602 ' entity='mgr.smithi148.onhgvm' 2024-03-29T08:06:25.178 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:06:24 smithi037 ceph-mon[35162]: from='mgr.14602 ' entity='mgr.smithi148.onhgvm' 2024-03-29T08:06:25.178 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:06:24 smithi037 ceph-mon[35162]: from='mgr.14602 ' entity='mgr.smithi148.onhgvm' 2024-03-29T08:06:25.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:06:24 smithi148 ceph-mon[44992]: Updating smithi037:/var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/config/ceph.client.admin.keyring 2024-03-29T08:06:25.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:06:24 smithi148 ceph-mon[44992]: pgmap v14: 65 pgs: 65 active+clean; 98 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 4.5 MiB/s wr, 423 op/s 2024-03-29T08:06:25.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:06:24 smithi148 ceph-mon[44992]: from='mgr.14602 ' entity='mgr.smithi148.onhgvm' 2024-03-29T08:06:25.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:06:24 smithi148 ceph-mon[44992]: from='mgr.14602 ' entity='mgr.smithi148.onhgvm' 2024-03-29T08:06:25.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:06:24 smithi148 ceph-mon[44992]: from='mgr.14602 ' entity='mgr.smithi148.onhgvm' 2024-03-29T08:06:25.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:06:24 smithi148 ceph-mon[44992]: from='mgr.14602 ' entity='mgr.smithi148.onhgvm' 2024-03-29T08:06:26.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:06:25 smithi037 ceph-mon[35162]: Reconfiguring node-exporter.smithi037 (dependencies changed)... 2024-03-29T08:06:26.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:06:25 smithi037 ceph-mon[35162]: Reconfiguring daemon node-exporter.smithi037 on smithi037 2024-03-29T08:06:26.305 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:06:25 smithi148 ceph-mon[44992]: Reconfiguring node-exporter.smithi037 (dependencies changed)... 2024-03-29T08:06:26.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:06:25 smithi148 ceph-mon[44992]: Reconfiguring daemon node-exporter.smithi037 on smithi037 2024-03-29T08:06:27.222 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:06:26 smithi037 ceph-mon[35162]: pgmap v15: 65 pgs: 65 active+clean; 98 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 3.0 MiB/s wr, 305 op/s 2024-03-29T08:06:27.305 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:06:26 smithi148 ceph-mon[44992]: pgmap v15: 65 pgs: 65 active+clean; 98 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 3.0 MiB/s wr, 305 op/s 2024-03-29T08:06:28.555 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:06:28 smithi148 ceph-mon[44992]: from='mgr.14602 ' entity='mgr.smithi148.onhgvm' 2024-03-29T08:06:28.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:06:28 smithi148 ceph-mon[44992]: from='mgr.14602 ' entity='mgr.smithi148.onhgvm' 2024-03-29T08:06:28.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:06:28 smithi148 ceph-mon[44992]: Reconfiguring alertmanager.smithi037 (dependencies changed)... 2024-03-29T08:06:28.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:06:28 smithi148 ceph-mon[44992]: Reconfiguring daemon alertmanager.smithi037 on smithi037 2024-03-29T08:06:28.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:06:28 smithi148 ceph-mon[44992]: pgmap v16: 65 pgs: 65 active+clean; 98 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 3.1 MiB/s wr, 327 op/s 2024-03-29T08:06:28.636 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:06:28 smithi037 ceph-mon[35162]: from='mgr.14602 ' entity='mgr.smithi148.onhgvm' 2024-03-29T08:06:28.636 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:06:28 smithi037 ceph-mon[35162]: from='mgr.14602 ' entity='mgr.smithi148.onhgvm' 2024-03-29T08:06:28.636 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:06:28 smithi037 ceph-mon[35162]: Reconfiguring alertmanager.smithi037 (dependencies changed)... 2024-03-29T08:06:28.636 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:06:28 smithi037 ceph-mon[35162]: Reconfiguring daemon alertmanager.smithi037 on smithi037 2024-03-29T08:06:28.636 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:06:28 smithi037 ceph-mon[35162]: pgmap v16: 65 pgs: 65 active+clean; 98 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 3.1 MiB/s wr, 327 op/s 2024-03-29T08:06:31.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:06:30 smithi037 ceph-mon[35162]: pgmap v17: 65 pgs: 65 active+clean; 111 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 3.9 MiB/s wr, 429 op/s 2024-03-29T08:06:31.305 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:06:30 smithi148 ceph-mon[44992]: pgmap v17: 65 pgs: 65 active+clean; 111 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 3.9 MiB/s wr, 429 op/s 2024-03-29T08:06:33.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:06:32 smithi148 ceph-mon[44992]: pgmap v18: 65 pgs: 65 active+clean; 111 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 2.7 MiB/s wr, 293 op/s 2024-03-29T08:06:33.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:06:32 smithi148 ceph-mon[44992]: from='mgr.14602 172.21.15.148:0/667342922' entity='mgr.smithi148.onhgvm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:06:33.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:06:32 smithi037 ceph-mon[35162]: pgmap v18: 65 pgs: 65 active+clean; 111 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 2.7 MiB/s wr, 293 op/s 2024-03-29T08:06:33.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:06:32 smithi037 ceph-mon[35162]: from='mgr.14602 172.21.15.148:0/667342922' entity='mgr.smithi148.onhgvm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:06:35.241 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:06:34 smithi037 ceph-mon[35162]: pgmap v19: 65 pgs: 65 active+clean; 122 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 3.7 MiB/s wr, 451 op/s 2024-03-29T08:06:35.241 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:06:34 smithi037 ceph-mon[35162]: from='mgr.14602 ' entity='mgr.smithi148.onhgvm' 2024-03-29T08:06:35.241 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:06:34 smithi037 ceph-mon[35162]: from='mgr.14602 ' entity='mgr.smithi148.onhgvm' 2024-03-29T08:06:35.305 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:06:34 smithi148 ceph-mon[44992]: pgmap v19: 65 pgs: 65 active+clean; 122 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 3.7 MiB/s wr, 451 op/s 2024-03-29T08:06:35.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:06:34 smithi148 ceph-mon[44992]: from='mgr.14602 ' entity='mgr.smithi148.onhgvm' 2024-03-29T08:06:35.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:06:34 smithi148 ceph-mon[44992]: from='mgr.14602 ' entity='mgr.smithi148.onhgvm' 2024-03-29T08:06:36.154 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:06:35 smithi037 ceph-mon[35162]: Reconfiguring grafana.smithi037 (dependencies changed)... 2024-03-29T08:06:36.154 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:06:35 smithi037 ceph-mon[35162]: Reconfiguring daemon grafana.smithi037 on smithi037 2024-03-29T08:06:36.305 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:06:35 smithi148 ceph-mon[44992]: Reconfiguring grafana.smithi037 (dependencies changed)... 2024-03-29T08:06:36.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:06:35 smithi148 ceph-mon[44992]: Reconfiguring daemon grafana.smithi037 on smithi037 2024-03-29T08:06:37.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:06:36 smithi148 ceph-mon[44992]: pgmap v20: 65 pgs: 65 active+clean; 122 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 2.2 MiB/s wr, 306 op/s 2024-03-29T08:06:37.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:06:36 smithi037 ceph-mon[35162]: pgmap v20: 65 pgs: 65 active+clean; 122 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 2.2 MiB/s wr, 306 op/s 2024-03-29T08:06:39.305 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:06:38 smithi148 ceph-mon[44992]: pgmap v21: 65 pgs: 65 active+clean; 125 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 2.5 MiB/s wr, 351 op/s 2024-03-29T08:06:39.426 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:06:38 smithi037 ceph-mon[35162]: pgmap v21: 65 pgs: 65 active+clean; 125 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 2.5 MiB/s wr, 351 op/s 2024-03-29T08:06:41.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:06:40 smithi148 ceph-mon[44992]: pgmap v22: 65 pgs: 65 active+clean; 139 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 3.6 MiB/s wr, 420 op/s 2024-03-29T08:06:41.426 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:06:40 smithi037 ceph-mon[35162]: pgmap v22: 65 pgs: 65 active+clean; 139 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 3.6 MiB/s wr, 420 op/s 2024-03-29T08:06:43.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:06:42 smithi148 ceph-mon[44992]: pgmap v23: 65 pgs: 65 active+clean; 139 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 2.5 MiB/s wr, 293 op/s 2024-03-29T08:06:43.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:06:42 smithi037 ceph-mon[35162]: pgmap v23: 65 pgs: 65 active+clean; 139 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 2.5 MiB/s wr, 293 op/s 2024-03-29T08:06:45.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:06:44 smithi148 ceph-mon[44992]: pgmap v24: 65 pgs: 65 active+clean; 154 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 3.9 MiB/s wr, 431 op/s 2024-03-29T08:06:45.426 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:06:44 smithi037 ceph-mon[35162]: pgmap v24: 65 pgs: 65 active+clean; 154 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 3.9 MiB/s wr, 431 op/s 2024-03-29T08:06:47.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:06:46 smithi148 ceph-mon[44992]: pgmap v25: 65 pgs: 65 active+clean; 154 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 2.9 MiB/s wr, 273 op/s 2024-03-29T08:06:47.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:06:46 smithi037 ceph-mon[35162]: pgmap v25: 65 pgs: 65 active+clean; 154 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 2.9 MiB/s wr, 273 op/s 2024-03-29T08:06:48.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:06:47 smithi148 ceph-mon[44992]: from='mgr.14602 172.21.15.148:0/667342922' entity='mgr.smithi148.onhgvm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:06:48.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:06:47 smithi037 ceph-mon[35162]: from='mgr.14602 172.21.15.148:0/667342922' entity='mgr.smithi148.onhgvm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:06:49.305 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:06:48 smithi148 ceph-mon[44992]: pgmap v26: 65 pgs: 65 active+clean; 157 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 3.1 MiB/s wr, 316 op/s 2024-03-29T08:06:49.426 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:06:48 smithi037 ceph-mon[35162]: pgmap v26: 65 pgs: 65 active+clean; 157 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 3.1 MiB/s wr, 316 op/s 2024-03-29T08:06:50.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:50.592+0000 7f9d0e000700 1 -- 172.21.15.37:0/884699052 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9d080fff60 msgr2=0x7f9d08102340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:06:50.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:50.592+0000 7f9d0e000700 1 --2- 172.21.15.37:0/884699052 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9d080fff60 0x7f9d08102340 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f9cf4004940 tx=0x7f9cf4009e30 comp rx=0 tx=0).stop 2024-03-29T08:06:50.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:50.593+0000 7f9d0e000700 1 -- 172.21.15.37:0/884699052 shutdown_connections 2024-03-29T08:06:50.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:50.593+0000 7f9d0e000700 1 --2- 172.21.15.37:0/884699052 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9d080fff60 0x7f9d08102340 secure :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f9cf4004940 tx=0x7f9cf4009e30 comp rx=0 tx=0).stop 2024-03-29T08:06:50.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:50.593+0000 7f9d0e000700 1 --2- 172.21.15.37:0/884699052 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9d08066280 0x7f9d080ffa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:06:50.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:50.593+0000 7f9d0e000700 1 -- 172.21.15.37:0/884699052 >> 172.21.15.37:0/884699052 conn(0x7f9d080f7430 msgr2=0x7f9d080f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:06:50.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:50.593+0000 7f9d0e000700 1 -- 172.21.15.37:0/884699052 shutdown_connections 2024-03-29T08:06:50.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:50.593+0000 7f9d0e000700 1 -- 172.21.15.37:0/884699052 wait complete. 2024-03-29T08:06:50.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:50.595+0000 7f9d0e000700 1 Processor -- start 2024-03-29T08:06:50.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:50.596+0000 7f9d0e000700 1 -- start start 2024-03-29T08:06:50.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:50.596+0000 7f9d0e000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9d08066280 0x7f9d0810fd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:06:50.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:50.596+0000 7f9d0e000700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9d081102b0 0x7f9d08110f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:06:50.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:50.596+0000 7f9d0e000700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d08110770 con 0x7f9d08066280 2024-03-29T08:06:50.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:50.596+0000 7f9d0e000700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d081108b0 con 0x7f9d081102b0 2024-03-29T08:06:50.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:50.597+0000 7f9d06a00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9d081102b0 0x7f9d08110f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:06:50.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:50.597+0000 7f9d06a00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9d081102b0 0x7f9d08110f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.37:38582/0 (socket says 172.21.15.37:38582) 2024-03-29T08:06:50.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:50.597+0000 7f9d06a00700 1 -- 172.21.15.37:0/3599284028 learned_addr learned my addr 172.21.15.37:0/3599284028 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:06:50.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:50.598+0000 7f9d07400700 1 --2- 172.21.15.37:0/3599284028 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9d08066280 0x7f9d0810fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:06:50.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:50.598+0000 7f9d06a00700 1 -- 172.21.15.37:0/3599284028 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9d08066280 msgr2=0x7f9d0810fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:06:50.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:50.598+0000 7f9d06a00700 1 --2- 172.21.15.37:0/3599284028 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9d08066280 0x7f9d0810fd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:06:50.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:50.598+0000 7f9d06a00700 1 -- 172.21.15.37:0/3599284028 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cf4005040 con 0x7f9d081102b0 2024-03-29T08:06:50.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:50.598+0000 7f9d07400700 1 --2- 172.21.15.37:0/3599284028 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9d08066280 0x7f9d0810fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:06:50.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:50.598+0000 7f9d06a00700 1 --2- 172.21.15.37:0/3599284028 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9d081102b0 0x7f9d08110f70 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f9cf40137b0 tx=0x7f9cf4009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:06:50.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:50.599+0000 7f9cff400700 1 -- 172.21.15.37:0/3599284028 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9cf4021070 con 0x7f9d081102b0 2024-03-29T08:06:50.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:50.599+0000 7f9d0e000700 1 -- 172.21.15.37:0/3599284028 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d081115f0 con 0x7f9d081102b0 2024-03-29T08:06:50.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:50.599+0000 7f9cff400700 1 -- 172.21.15.37:0/3599284028 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f9cf40093e0 con 0x7f9d081102b0 2024-03-29T08:06:50.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:50.599+0000 7f9cff400700 1 -- 172.21.15.37:0/3599284028 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9cf400e070 con 0x7f9d081102b0 2024-03-29T08:06:50.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:50.599+0000 7f9d0e000700 1 -- 172.21.15.37:0/3599284028 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d08111b50 con 0x7f9d081102b0 2024-03-29T08:06:50.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:50.601+0000 7f9d0e000700 1 -- 172.21.15.37:0/3599284028 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d080fe680 con 0x7f9d081102b0 2024-03-29T08:06:50.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:50.603+0000 7f9cff400700 1 -- 172.21.15.37:0/3599284028 <== mon.1 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 25) v1 ==== 88766+0+0 (secure 0 0 0) 0x7f9cf401c600 con 0x7f9d081102b0 2024-03-29T08:06:50.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:50.604+0000 7f9cff400700 1 --2- 172.21.15.37:0/3599284028 >> [v2:172.21.15.148:6828/1134897184,v1:172.21.15.148:6829/1134897184] conn(0x7f9cec06aec0 0x7f9cec06d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:06:50.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:50.604+0000 7f9cff400700 1 -- 172.21.15.37:0/3599284028 <== mon.1 v2:172.21.15.148:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f9cf408aa90 con 0x7f9d081102b0 2024-03-29T08:06:50.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:50.604+0000 7f9d07400700 1 --2- 172.21.15.37:0/3599284028 >> [v2:172.21.15.148:6828/1134897184,v1:172.21.15.148:6829/1134897184] conn(0x7f9cec06aec0 0x7f9cec06d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:06:50.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:50.605+0000 7f9d07400700 1 --2- 172.21.15.37:0/3599284028 >> [v2:172.21.15.148:6828/1134897184,v1:172.21.15.148:6829/1134897184] conn(0x7f9cec06aec0 0x7f9cec06d370 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f9cf8007fd0 tx=0x7f9cf800b040 comp rx=0 tx=0).ready entity=mgr.14602 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:06:50.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:50.610+0000 7f9cff400700 1 -- 172.21.15.37:0/3599284028 <== mon.1 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181579 (secure 0 0 0) 0x7f9cf4058970 con 0x7f9d081102b0 2024-03-29T08:06:50.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:50.950+0000 7f9d0e000700 1 -- 172.21.15.37:0/3599284028 --> [v2:172.21.15.148:6828/1134897184,v1:172.21.15.148:6829/1134897184] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f9d081057e0 con 0x7f9cec06aec0 2024-03-29T08:06:50.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:50.952+0000 7f9cff400700 1 -- 172.21.15.37:0/3599284028 <== mgr.14602 v2:172.21.15.148:6828/1134897184 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+310 (secure 0 0 0) 0x7f9d081057e0 con 0x7f9cec06aec0 2024-03-29T08:06:50.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:50.955+0000 7f9d0e000700 1 -- 172.21.15.37:0/3599284028 >> [v2:172.21.15.148:6828/1134897184,v1:172.21.15.148:6829/1134897184] conn(0x7f9cec06aec0 msgr2=0x7f9cec06d370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:06:50.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:50.955+0000 7f9d0e000700 1 --2- 172.21.15.37:0/3599284028 >> [v2:172.21.15.148:6828/1134897184,v1:172.21.15.148:6829/1134897184] conn(0x7f9cec06aec0 0x7f9cec06d370 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f9cf8007fd0 tx=0x7f9cf800b040 comp rx=0 tx=0).stop 2024-03-29T08:06:50.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:50.955+0000 7f9d0e000700 1 -- 172.21.15.37:0/3599284028 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9d081102b0 msgr2=0x7f9d08110f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:06:50.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:50.955+0000 7f9d0e000700 1 --2- 172.21.15.37:0/3599284028 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9d081102b0 0x7f9d08110f70 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f9cf40137b0 tx=0x7f9cf4009e30 comp rx=0 tx=0).stop 2024-03-29T08:06:50.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:50.955+0000 7f9d0e000700 1 -- 172.21.15.37:0/3599284028 shutdown_connections 2024-03-29T08:06:50.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:50.955+0000 7f9d0e000700 1 --2- 172.21.15.37:0/3599284028 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9d08066280 0x7f9d0810fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:06:50.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:50.955+0000 7f9d0e000700 1 --2- 172.21.15.37:0/3599284028 >> [v2:172.21.15.148:6828/1134897184,v1:172.21.15.148:6829/1134897184] conn(0x7f9cec06aec0 0x7f9cec06d370 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:06:50.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:50.955+0000 7f9d0e000700 1 --2- 172.21.15.37:0/3599284028 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9d081102b0 0x7f9d08110f70 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:06:50.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:50.955+0000 7f9d0e000700 1 -- 172.21.15.37:0/3599284028 >> 172.21.15.37:0/3599284028 conn(0x7f9d080f7430 msgr2=0x7f9d080fb800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:06:50.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:50.955+0000 7f9d0e000700 1 -- 172.21.15.37:0/3599284028 shutdown_connections 2024-03-29T08:06:50.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:50.955+0000 7f9d0e000700 1 -- 172.21.15.37:0/3599284028 wait complete. 2024-03-29T08:06:50.974 INFO:teuthology.orchestra.run.smithi037.stdout:true 2024-03-29T08:06:51.091 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:06:50 smithi037 ceph-mon[35162]: pgmap v27: 65 pgs: 65 active+clean; 170 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 3.9 MiB/s wr, 356 op/s 2024-03-29T08:06:51.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:51.142+0000 7f49afe00700 1 -- 172.21.15.37:0/3961101518 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f49b00fbb10 msgr2=0x7f49b00fbf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:06:51.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:51.142+0000 7f49afe00700 1 --2- 172.21.15.37:0/3961101518 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f49b00fbb10 0x7f49b00fbf20 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f49a0004940 tx=0x7f49a0009e30 comp rx=0 tx=0).stop 2024-03-29T08:06:51.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:51.143+0000 7f49afe00700 1 -- 172.21.15.37:0/3961101518 shutdown_connections 2024-03-29T08:06:51.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:51.143+0000 7f49afe00700 1 --2- 172.21.15.37:0/3961101518 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f49b00fcdc0 0x7f49b00fd230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:06:51.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:51.143+0000 7f49afe00700 1 --2- 172.21.15.37:0/3961101518 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f49b00fbb10 0x7f49b00fbf20 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:06:51.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:51.143+0000 7f49afe00700 1 -- 172.21.15.37:0/3961101518 >> 172.21.15.37:0/3961101518 conn(0x7f49b00f7270 msgr2=0x7f49b00f96a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:06:51.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:51.143+0000 7f49afe00700 1 -- 172.21.15.37:0/3961101518 shutdown_connections 2024-03-29T08:06:51.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:51.143+0000 7f49afe00700 1 -- 172.21.15.37:0/3961101518 wait complete. 2024-03-29T08:06:51.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:51.144+0000 7f49afe00700 1 Processor -- start 2024-03-29T08:06:51.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:51.144+0000 7f49afe00700 1 -- start start 2024-03-29T08:06:51.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:51.145+0000 7f49afe00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f49b00fbb10 0x7f49b0113d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:06:51.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:51.145+0000 7f49afe00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f49b00fcdc0 0x7f49b01142b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:06:51.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:51.145+0000 7f49afe00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49b01148b0 con 0x7f49b00fcdc0 2024-03-29T08:06:51.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:51.145+0000 7f49afe00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49b0114ff0 con 0x7f49b00fbb10 2024-03-29T08:06:51.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:51.145+0000 7f49aea00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f49b00fbb10 0x7f49b0113d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:06:51.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:51.145+0000 7f49aea00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f49b00fbb10 0x7f49b0113d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.37:38614/0 (socket says 172.21.15.37:38614) 2024-03-29T08:06:51.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:51.145+0000 7f49aea00700 1 -- 172.21.15.37:0/1526394090 learned_addr learned my addr 172.21.15.37:0/1526394090 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:06:51.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:51.146+0000 7f49aea00700 1 -- 172.21.15.37:0/1526394090 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f49b00fcdc0 msgr2=0x7f49b01142b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-29T08:06:51.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:51.146+0000 7f49aea00700 1 --2- 172.21.15.37:0/1526394090 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f49b00fcdc0 0x7f49b01142b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:06:51.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:51.146+0000 7f49aea00700 1 -- 172.21.15.37:0/1526394090 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49a0005040 con 0x7f49b00fbb10 2024-03-29T08:06:51.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:51.147+0000 7f49aea00700 1 --2- 172.21.15.37:0/1526394090 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f49b00fbb10 0x7f49b0113d70 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f49a000b0d0 tx=0x7f49a0009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:06:51.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:51.147+0000 7f49a6a00700 1 -- 172.21.15.37:0/1526394090 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f49a000e070 con 0x7f49b00fbb10 2024-03-29T08:06:51.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:51.147+0000 7f49a6a00700 1 -- 172.21.15.37:0/1526394090 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f49a000b410 con 0x7f49b00fbb10 2024-03-29T08:06:51.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:51.147+0000 7f49afe00700 1 -- 172.21.15.37:0/1526394090 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49b0115270 con 0x7f49b00fbb10 2024-03-29T08:06:51.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:51.148+0000 7f49afe00700 1 -- 172.21.15.37:0/1526394090 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49b0115830 con 0x7f49b00fbb10 2024-03-29T08:06:51.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:51.148+0000 7f49a6a00700 1 -- 172.21.15.37:0/1526394090 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f49a0012600 con 0x7f49b00fbb10 2024-03-29T08:06:51.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:51.150+0000 7f49afe00700 1 -- 172.21.15.37:0/1526394090 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f498c0052e0 con 0x7f49b00fbb10 2024-03-29T08:06:51.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:51.152+0000 7f49a6a00700 1 -- 172.21.15.37:0/1526394090 <== mon.1 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 25) v1 ==== 88766+0+0 (secure 0 0 0) 0x7f49a0013a50 con 0x7f49b00fbb10 2024-03-29T08:06:51.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:51.154+0000 7f49a6a00700 1 --2- 172.21.15.37:0/1526394090 >> [v2:172.21.15.148:6828/1134897184,v1:172.21.15.148:6829/1134897184] conn(0x7f499406b050 0x7f499406d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:06:51.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:51.154+0000 7f49a6a00700 1 -- 172.21.15.37:0/1526394090 <== mon.1 v2:172.21.15.148:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f49a008a440 con 0x7f49b00fbb10 2024-03-29T08:06:51.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:51.155+0000 7f49ae000700 1 --2- 172.21.15.37:0/1526394090 >> [v2:172.21.15.148:6828/1134897184,v1:172.21.15.148:6829/1134897184] conn(0x7f499406b050 0x7f499406d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:06:51.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:51.156+0000 7f49ae000700 1 --2- 172.21.15.37:0/1526394090 >> [v2:172.21.15.148:6828/1134897184,v1:172.21.15.148:6829/1134897184] conn(0x7f499406b050 0x7f499406d500 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f4998009fd0 tx=0x7f499800b040 comp rx=0 tx=0).ready entity=mgr.14602 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:06:51.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:51.158+0000 7f49a6a00700 1 -- 172.21.15.37:0/1526394090 <== mon.1 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181579 (secure 0 0 0) 0x7f49a00583d0 con 0x7f49b00fbb10 2024-03-29T08:06:51.305 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:06:50 smithi148 ceph-mon[44992]: pgmap v27: 65 pgs: 65 active+clean; 170 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 3.9 MiB/s wr, 356 op/s 2024-03-29T08:06:51.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:51.512+0000 7f49afe00700 1 -- 172.21.15.37:0/1526394090 --> [v2:172.21.15.148:6828/1134897184,v1:172.21.15.148:6829/1134897184] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f498c000bf0 con 0x7f499406b050 2024-03-29T08:06:51.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:51.513+0000 7f49a6a00700 1 -- 172.21.15.37:0/1526394090 <== mgr.14602 v2:172.21.15.148:6828/1134897184 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+310 (secure 0 0 0) 0x7f498c000bf0 con 0x7f499406b050 2024-03-29T08:06:51.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:51.516+0000 7f49afe00700 1 -- 172.21.15.37:0/1526394090 >> [v2:172.21.15.148:6828/1134897184,v1:172.21.15.148:6829/1134897184] conn(0x7f499406b050 msgr2=0x7f499406d500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:06:51.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:51.516+0000 7f49afe00700 1 --2- 172.21.15.37:0/1526394090 >> [v2:172.21.15.148:6828/1134897184,v1:172.21.15.148:6829/1134897184] conn(0x7f499406b050 0x7f499406d500 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f4998009fd0 tx=0x7f499800b040 comp rx=0 tx=0).stop 2024-03-29T08:06:51.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:51.516+0000 7f49afe00700 1 -- 172.21.15.37:0/1526394090 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f49b00fbb10 msgr2=0x7f49b0113d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:06:51.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:51.516+0000 7f49afe00700 1 --2- 172.21.15.37:0/1526394090 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f49b00fbb10 0x7f49b0113d70 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f49a000b0d0 tx=0x7f49a0009e30 comp rx=0 tx=0).stop 2024-03-29T08:06:51.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:51.517+0000 7f49afe00700 1 -- 172.21.15.37:0/1526394090 shutdown_connections 2024-03-29T08:06:51.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:51.517+0000 7f49afe00700 1 --2- 172.21.15.37:0/1526394090 >> [v2:172.21.15.148:6828/1134897184,v1:172.21.15.148:6829/1134897184] conn(0x7f499406b050 0x7f499406d500 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:06:51.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:51.517+0000 7f49afe00700 1 --2- 172.21.15.37:0/1526394090 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f49b00fbb10 0x7f49b0113d70 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:06:51.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:51.517+0000 7f49afe00700 1 --2- 172.21.15.37:0/1526394090 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f49b00fcdc0 0x7f49b01142b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:06:51.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:51.517+0000 7f49afe00700 1 -- 172.21.15.37:0/1526394090 >> 172.21.15.37:0/1526394090 conn(0x7f49b00f7270 msgr2=0x7f49b00f8a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:06:51.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:51.517+0000 7f49afe00700 1 -- 172.21.15.37:0/1526394090 shutdown_connections 2024-03-29T08:06:51.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:51.517+0000 7f49afe00700 1 -- 172.21.15.37:0/1526394090 wait complete. 2024-03-29T08:06:51.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:51.727+0000 7f6c6d000700 1 -- 172.21.15.37:0/2887250328 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6c680fff00 msgr2=0x7f6c68102320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:06:51.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:51.727+0000 7f6c6d000700 1 --2- 172.21.15.37:0/2887250328 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6c680fff00 0x7f6c68102320 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f6c50004940 tx=0x7f6c50009e30 comp rx=0 tx=0).stop 2024-03-29T08:06:51.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:51.728+0000 7f6c6d000700 1 -- 172.21.15.37:0/2887250328 shutdown_connections 2024-03-29T08:06:51.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:51.728+0000 7f6c6d000700 1 --2- 172.21.15.37:0/2887250328 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6c680fff00 0x7f6c68102320 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:06:51.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:51.728+0000 7f6c6d000700 1 --2- 172.21.15.37:0/2887250328 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6c680662e0 0x7f6c680ff9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:06:51.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:51.728+0000 7f6c6d000700 1 -- 172.21.15.37:0/2887250328 >> 172.21.15.37:0/2887250328 conn(0x7f6c680f7430 msgr2=0x7f6c680f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:06:51.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:51.728+0000 7f6c6d000700 1 -- 172.21.15.37:0/2887250328 shutdown_connections 2024-03-29T08:06:51.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:51.728+0000 7f6c6d000700 1 -- 172.21.15.37:0/2887250328 wait complete. 2024-03-29T08:06:51.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:51.730+0000 7f6c6d000700 1 Processor -- start 2024-03-29T08:06:51.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:51.731+0000 7f6c6d000700 1 -- start start 2024-03-29T08:06:51.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:51.732+0000 7f6c6d000700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6c680662e0 0x7f6c68106f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:06:51.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:51.732+0000 7f6c6d000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6c680fff00 0x7f6c68107450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:06:51.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:51.732+0000 7f6c6d000700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c68107aa0 con 0x7f6c680fff00 2024-03-29T08:06:51.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:51.732+0000 7f6c6d000700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c681082f0 con 0x7f6c680662e0 2024-03-29T08:06:51.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:51.733+0000 7f6c65e00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6c680662e0 0x7f6c68106f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:06:51.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:51.733+0000 7f6c65e00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6c680662e0 0x7f6c68106f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.37:38620/0 (socket says 172.21.15.37:38620) 2024-03-29T08:06:51.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:51.733+0000 7f6c65e00700 1 -- 172.21.15.37:0/3892364433 learned_addr learned my addr 172.21.15.37:0/3892364433 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:06:51.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:51.733+0000 7f6c65400700 1 --2- 172.21.15.37:0/3892364433 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6c680fff00 0x7f6c68107450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:06:51.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:51.733+0000 7f6c65e00700 1 -- 172.21.15.37:0/3892364433 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6c680fff00 msgr2=0x7f6c68107450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:06:51.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:51.733+0000 7f6c65e00700 1 --2- 172.21.15.37:0/3892364433 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6c680fff00 0x7f6c68107450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:06:51.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:51.734+0000 7f6c65e00700 1 -- 172.21.15.37:0/3892364433 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c50005040 con 0x7f6c680662e0 2024-03-29T08:06:51.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:51.734+0000 7f6c65400700 1 --2- 172.21.15.37:0/3892364433 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6c680fff00 0x7f6c68107450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:06:51.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:51.734+0000 7f6c65e00700 1 --2- 172.21.15.37:0/3892364433 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6c680662e0 0x7f6c68106f10 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f6c6000eb40 tx=0x7f6c6000b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:06:51.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:51.734+0000 7f6c5e000700 1 -- 172.21.15.37:0/3892364433 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6c60007cc0 con 0x7f6c680662e0 2024-03-29T08:06:51.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:51.734+0000 7f6c5e000700 1 -- 172.21.15.37:0/3892364433 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f6c6000f040 con 0x7f6c680662e0 2024-03-29T08:06:51.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:51.734+0000 7f6c5e000700 1 -- 172.21.15.37:0/3892364433 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6c60009570 con 0x7f6c680662e0 2024-03-29T08:06:51.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:51.735+0000 7f6c6d000700 1 -- 172.21.15.37:0/3892364433 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c68108570 con 0x7f6c680662e0 2024-03-29T08:06:51.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:51.735+0000 7f6c6d000700 1 -- 172.21.15.37:0/3892364433 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c68108ae0 con 0x7f6c680662e0 2024-03-29T08:06:51.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:51.736+0000 7f6c6d000700 1 -- 172.21.15.37:0/3892364433 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c680f8fb0 con 0x7f6c680662e0 2024-03-29T08:06:51.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:51.738+0000 7f6c5e000700 1 -- 172.21.15.37:0/3892364433 <== mon.1 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 25) v1 ==== 88766+0+0 (secure 0 0 0) 0x7f6c600096d0 con 0x7f6c680662e0 2024-03-29T08:06:51.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:51.739+0000 7f6c5e000700 1 --2- 172.21.15.37:0/3892364433 >> [v2:172.21.15.148:6828/1134897184,v1:172.21.15.148:6829/1134897184] conn(0x7f6c4c06aec0 0x7f6c4c06d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:06:51.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:51.739+0000 7f6c5e000700 1 -- 172.21.15.37:0/3892364433 <== mon.1 v2:172.21.15.148:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f6c60089680 con 0x7f6c680662e0 2024-03-29T08:06:51.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:51.740+0000 7f6c65400700 1 --2- 172.21.15.37:0/3892364433 >> [v2:172.21.15.148:6828/1134897184,v1:172.21.15.148:6829/1134897184] conn(0x7f6c4c06aec0 0x7f6c4c06d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:06:51.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:51.740+0000 7f6c65400700 1 --2- 172.21.15.37:0/3892364433 >> [v2:172.21.15.148:6828/1134897184,v1:172.21.15.148:6829/1134897184] conn(0x7f6c4c06aec0 0x7f6c4c06d370 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f6c50004560 tx=0x7f6c50009e30 comp rx=0 tx=0).ready entity=mgr.14602 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:06:51.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:51.742+0000 7f6c5e000700 1 -- 172.21.15.37:0/3892364433 <== mon.1 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181579 (secure 0 0 0) 0x7f6c60053b00 con 0x7f6c680662e0 2024-03-29T08:06:52.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:52.091+0000 7f6c6d000700 1 -- 172.21.15.37:0/3892364433 --> [v2:172.21.15.148:6828/1134897184,v1:172.21.15.148:6829/1134897184] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f6c681058d0 con 0x7f6c4c06aec0 2024-03-29T08:06:52.105 INFO:teuthology.orchestra.run.smithi037.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-29T08:06:52.105 INFO:teuthology.orchestra.run.smithi037.stdout:alertmanager.smithi037 smithi037 *:9093,9094 starting - - - - 2024-03-29T08:06:52.105 INFO:teuthology.orchestra.run.smithi037.stdout:crash.smithi037 smithi037 running (9m) 36s ago 9m 6651k - 17.2.7-789-g4213e17c cf22eb8683be e0343935b25e 2024-03-29T08:06:52.105 INFO:teuthology.orchestra.run.smithi037.stdout:crash.smithi148 smithi148 running (8m) 43s ago 8m 6656k - 17.2.7-789-g4213e17c cf22eb8683be ff5164c552c1 2024-03-29T08:06:52.106 INFO:teuthology.orchestra.run.smithi037.stdout:grafana.smithi037 smithi037 *:3000 running (7m) 36s ago 9m 73.7M - 9.4.7 954c08fa6188 92b77c060e7a 2024-03-29T08:06:52.106 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi037.iifsmh smithi037 running (3m) 36s ago 3m 17.7M - 17.2.7-789-g4213e17c cf22eb8683be 074e1acf331f 2024-03-29T08:06:52.106 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi037.mmwgxe smithi037 running (3m) 36s ago 3m 64.9M - 17.2.7-789-g4213e17c cf22eb8683be 9a5b1e72de5c 2024-03-29T08:06:52.106 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi148.qukpjh smithi148 running (3m) 43s ago 3m 11.7M - 17.2.7-789-g4213e17c cf22eb8683be 8e469782a273 2024-03-29T08:06:52.106 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi148.sqteih smithi148 running (3m) 43s ago 3m 11.6M - 17.2.7-789-g4213e17c cf22eb8683be 8e68f858bb22 2024-03-29T08:06:52.106 INFO:teuthology.orchestra.run.smithi037.stdout:mgr.smithi037.wjikii smithi037 *:9283 running (11m) 36s ago 11m 156M - 17.2.7-789-g4213e17c cf22eb8683be 60172b53efa2 2024-03-29T08:06:52.106 INFO:teuthology.orchestra.run.smithi037.stdout:mgr.smithi148.onhgvm smithi148 *:8443,9283 running (63s) 43s ago 8m 479M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 0c02ffa63851 2024-03-29T08:06:52.107 INFO:teuthology.orchestra.run.smithi037.stdout:mon.smithi037 smithi037 running (11m) 36s ago 11m 48.6M 2048M 17.2.7-789-g4213e17c cf22eb8683be bb2101505730 2024-03-29T08:06:52.107 INFO:teuthology.orchestra.run.smithi037.stdout:mon.smithi148 smithi148 running (8m) 43s ago 8m 38.5M 2048M 17.2.7-789-g4213e17c cf22eb8683be 18839ed2952a 2024-03-29T08:06:52.107 INFO:teuthology.orchestra.run.smithi037.stdout:node-exporter.smithi037 smithi037 *:9100 starting - - - - 2024-03-29T08:06:52.107 INFO:teuthology.orchestra.run.smithi037.stdout:node-exporter.smithi148 smithi148 *:9100 running (8m) 43s ago 8m 8014k - 1.5.0 0da6a335fe13 bb455992ed53 2024-03-29T08:06:52.107 INFO:teuthology.orchestra.run.smithi037.stdout:osd.0 smithi037 running (7m) 36s ago 7m 67.4M 1619M 17.2.7-789-g4213e17c cf22eb8683be 559fccd0ebe1 2024-03-29T08:06:52.107 INFO:teuthology.orchestra.run.smithi037.stdout:osd.1 smithi037 running (7m) 36s ago 7m 68.5M 1619M 17.2.7-789-g4213e17c cf22eb8683be 5582cff11b7a 2024-03-29T08:06:52.107 INFO:teuthology.orchestra.run.smithi037.stdout:osd.2 smithi037 running (6m) 36s ago 6m 60.6M 1619M 17.2.7-789-g4213e17c cf22eb8683be f809bfab13e5 2024-03-29T08:06:52.108 INFO:teuthology.orchestra.run.smithi037.stdout:osd.3 smithi148 running (5m) 43s ago 5m 67.0M 2642M 17.2.7-789-g4213e17c cf22eb8683be a0a8224bb4b0 2024-03-29T08:06:52.108 INFO:teuthology.orchestra.run.smithi037.stdout:osd.4 smithi148 running (5m) 43s ago 5m 63.7M 2642M 17.2.7-789-g4213e17c cf22eb8683be a00b69047ed7 2024-03-29T08:06:52.108 INFO:teuthology.orchestra.run.smithi037.stdout:osd.5 smithi148 running (4m) 43s ago 4m 65.5M 2642M 17.2.7-789-g4213e17c cf22eb8683be 044a3f7db7f8 2024-03-29T08:06:52.108 INFO:teuthology.orchestra.run.smithi037.stdout:prometheus.smithi037 smithi037 *:9095 running (7m) 36s ago 9m 53.2M - 2.43.0 a07b618ecd1d e8c23666a746 2024-03-29T08:06:52.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:52.102+0000 7f6c5e000700 1 -- 172.21.15.37:0/3892364433 <== mgr.14602 v2:172.21.15.148:6828/1134897184 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3388 (secure 0 0 0) 0x7f6c681058d0 con 0x7f6c4c06aec0 2024-03-29T08:06:52.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:52.106+0000 7f6c6d000700 1 -- 172.21.15.37:0/3892364433 >> [v2:172.21.15.148:6828/1134897184,v1:172.21.15.148:6829/1134897184] conn(0x7f6c4c06aec0 msgr2=0x7f6c4c06d370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:06:52.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:52.106+0000 7f6c6d000700 1 --2- 172.21.15.37:0/3892364433 >> [v2:172.21.15.148:6828/1134897184,v1:172.21.15.148:6829/1134897184] conn(0x7f6c4c06aec0 0x7f6c4c06d370 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f6c50004560 tx=0x7f6c50009e30 comp rx=0 tx=0).stop 2024-03-29T08:06:52.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:52.106+0000 7f6c6d000700 1 -- 172.21.15.37:0/3892364433 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6c680662e0 msgr2=0x7f6c68106f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:06:52.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:52.106+0000 7f6c6d000700 1 --2- 172.21.15.37:0/3892364433 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6c680662e0 0x7f6c68106f10 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f6c6000eb40 tx=0x7f6c6000b040 comp rx=0 tx=0).stop 2024-03-29T08:06:52.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:52.107+0000 7f6c6d000700 1 -- 172.21.15.37:0/3892364433 shutdown_connections 2024-03-29T08:06:52.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:52.107+0000 7f6c6d000700 1 --2- 172.21.15.37:0/3892364433 >> [v2:172.21.15.148:6828/1134897184,v1:172.21.15.148:6829/1134897184] conn(0x7f6c4c06aec0 0x7f6c4c06d370 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:06:52.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:52.107+0000 7f6c6d000700 1 --2- 172.21.15.37:0/3892364433 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6c680662e0 0x7f6c68106f10 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:06:52.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:52.107+0000 7f6c6d000700 1 --2- 172.21.15.37:0/3892364433 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6c680fff00 0x7f6c68107450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:06:52.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:52.107+0000 7f6c6d000700 1 -- 172.21.15.37:0/3892364433 >> 172.21.15.37:0/3892364433 conn(0x7f6c680f7430 msgr2=0x7f6c680fbb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:06:52.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:52.107+0000 7f6c6d000700 1 -- 172.21.15.37:0/3892364433 shutdown_connections 2024-03-29T08:06:52.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:52.107+0000 7f6c6d000700 1 -- 172.21.15.37:0/3892364433 wait complete. 2024-03-29T08:06:52.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:06:51 smithi037 ceph-mon[35162]: from='client.24471 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:06:52.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:06:51 smithi148 ceph-mon[44992]: from='client.24471 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:06:52.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:52.316+0000 7f2607200700 1 -- 172.21.15.37:0/1504396135 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2600066260 msgr2=0x7f26000ffa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:06:52.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:52.316+0000 7f2607200700 1 --2- 172.21.15.37:0/1504396135 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2600066260 0x7f26000ffa40 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f25f0004940 tx=0x7f25f0009e30 comp rx=0 tx=0).stop 2024-03-29T08:06:52.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:52.316+0000 7f2607200700 1 -- 172.21.15.37:0/1504396135 shutdown_connections 2024-03-29T08:06:52.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:52.316+0000 7f2607200700 1 --2- 172.21.15.37:0/1504396135 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f26000fff80 0x7f2600102360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:06:52.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:52.316+0000 7f2607200700 1 --2- 172.21.15.37:0/1504396135 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2600066260 0x7f26000ffa40 secure :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f25f0004940 tx=0x7f25f0009e30 comp rx=0 tx=0).stop 2024-03-29T08:06:52.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:52.316+0000 7f2607200700 1 -- 172.21.15.37:0/1504396135 >> 172.21.15.37:0/1504396135 conn(0x7f26000f7430 msgr2=0x7f26000f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:06:52.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:52.317+0000 7f2607200700 1 -- 172.21.15.37:0/1504396135 shutdown_connections 2024-03-29T08:06:52.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:52.317+0000 7f2607200700 1 -- 172.21.15.37:0/1504396135 wait complete. 2024-03-29T08:06:52.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:52.319+0000 7f2607200700 1 Processor -- start 2024-03-29T08:06:52.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:52.319+0000 7f2607200700 1 -- start start 2024-03-29T08:06:52.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:52.319+0000 7f2607200700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f26000fff80 0x7f26000fd470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:06:52.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:52.319+0000 7f2607200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f26000fb940 0x7f26000fbdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:06:52.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:52.320+0000 7f2607200700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26000fc2f0 con 0x7f26000fb940 2024-03-29T08:06:52.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:52.320+0000 7f2607200700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26000fc430 con 0x7f26000fff80 2024-03-29T08:06:52.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:52.320+0000 7f25ffe00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f26000fff80 0x7f26000fd470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:06:52.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:52.320+0000 7f25ffe00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f26000fff80 0x7f26000fd470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.37:38632/0 (socket says 172.21.15.37:38632) 2024-03-29T08:06:52.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:52.320+0000 7f25ffe00700 1 -- 172.21.15.37:0/3832027942 learned_addr learned my addr 172.21.15.37:0/3832027942 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:06:52.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:52.320+0000 7f25ff400700 1 --2- 172.21.15.37:0/3832027942 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f26000fb940 0x7f26000fbdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:06:52.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:52.320+0000 7f25ffe00700 1 -- 172.21.15.37:0/3832027942 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f26000fb940 msgr2=0x7f26000fbdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:06:52.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:52.320+0000 7f25ffe00700 1 --2- 172.21.15.37:0/3832027942 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f26000fb940 0x7f26000fbdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:06:52.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:52.321+0000 7f25ffe00700 1 -- 172.21.15.37:0/3832027942 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25f0005040 con 0x7f26000fff80 2024-03-29T08:06:52.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:52.321+0000 7f25ffe00700 1 --2- 172.21.15.37:0/3832027942 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f26000fff80 0x7f26000fd470 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f25f000b900 tx=0x7f25f0009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:06:52.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:52.322+0000 7f25f7e00700 1 -- 172.21.15.37:0/3832027942 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f25f000e040 con 0x7f26000fff80 2024-03-29T08:06:52.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:52.322+0000 7f25f7e00700 1 -- 172.21.15.37:0/3832027942 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f25f0004290 con 0x7f26000fff80 2024-03-29T08:06:52.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:52.322+0000 7f25f7e00700 1 -- 172.21.15.37:0/3832027942 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f25f0013e90 con 0x7f26000fff80 2024-03-29T08:06:52.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:52.322+0000 7f2607200700 1 -- 172.21.15.37:0/3832027942 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26000fc6b0 con 0x7f26000fff80 2024-03-29T08:06:52.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:52.322+0000 7f2607200700 1 -- 172.21.15.37:0/3832027942 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2600115e00 con 0x7f26000fff80 2024-03-29T08:06:52.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:52.323+0000 7f2607200700 1 -- 172.21.15.37:0/3832027942 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f260005f5f0 con 0x7f26000fff80 2024-03-29T08:06:52.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:52.326+0000 7f25f7e00700 1 -- 172.21.15.37:0/3832027942 <== mon.1 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 25) v1 ==== 88766+0+0 (secure 0 0 0) 0x7f25f00136b0 con 0x7f26000fff80 2024-03-29T08:06:52.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:52.328+0000 7f25f7e00700 1 --2- 172.21.15.37:0/3832027942 >> [v2:172.21.15.148:6828/1134897184,v1:172.21.15.148:6829/1134897184] conn(0x7f25e406af90 0x7f25e406d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:06:52.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:52.328+0000 7f25f7e00700 1 -- 172.21.15.37:0/3832027942 <== mon.1 v2:172.21.15.148:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f25f008aa50 con 0x7f26000fff80 2024-03-29T08:06:52.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:52.328+0000 7f25ff400700 1 --2- 172.21.15.37:0/3832027942 >> [v2:172.21.15.148:6828/1134897184,v1:172.21.15.148:6829/1134897184] conn(0x7f25e406af90 0x7f25e406d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:06:52.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:52.329+0000 7f25ff400700 1 --2- 172.21.15.37:0/3832027942 >> [v2:172.21.15.148:6828/1134897184,v1:172.21.15.148:6829/1134897184] conn(0x7f25e406af90 0x7f25e406d440 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f25e8004f40 tx=0x7f25e800b040 comp rx=0 tx=0).ready entity=mgr.14602 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:06:52.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:52.333+0000 7f25f7e00700 1 -- 172.21.15.37:0/3832027942 <== mon.1 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181579 (secure 0 0 0) 0x7f25f0054ea0 con 0x7f26000fff80 2024-03-29T08:06:52.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:52.800+0000 7f2607200700 1 -- 172.21.15.37:0/3832027942 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f2600105770 con 0x7f26000fff80 2024-03-29T08:06:52.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:52.800+0000 7f25f7e00700 1 -- 172.21.15.37:0/3832027942 <== mon.1 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+850 (secure 0 0 0) 0x7f25f00584c0 con 0x7f26000fff80 2024-03-29T08:06:52.801 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-29T08:06:52.801 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": { 2024-03-29T08:06:52.801 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 17.2.7-789-g4213e17c (4213e17c4de67016ed14f55d22503f4923b3a901) quincy (stable)": 2 2024-03-29T08:06:52.801 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:06:52.801 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": { 2024-03-29T08:06:52.801 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 17.2.7-789-g4213e17c (4213e17c4de67016ed14f55d22503f4923b3a901) quincy (stable)": 1, 2024-03-29T08:06:52.802 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 1 2024-03-29T08:06:52.802 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:06:52.802 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": { 2024-03-29T08:06:52.802 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 17.2.7-789-g4213e17c (4213e17c4de67016ed14f55d22503f4923b3a901) quincy (stable)": 6 2024-03-29T08:06:52.802 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:06:52.802 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": { 2024-03-29T08:06:52.802 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 17.2.7-789-g4213e17c (4213e17c4de67016ed14f55d22503f4923b3a901) quincy (stable)": 4 2024-03-29T08:06:52.802 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:06:52.802 INFO:teuthology.orchestra.run.smithi037.stdout: "overall": { 2024-03-29T08:06:52.802 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 17.2.7-789-g4213e17c (4213e17c4de67016ed14f55d22503f4923b3a901) quincy (stable)": 13, 2024-03-29T08:06:52.802 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 1 2024-03-29T08:06:52.802 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-29T08:06:52.802 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-29T08:06:52.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:52.804+0000 7f2607200700 1 -- 172.21.15.37:0/3832027942 >> [v2:172.21.15.148:6828/1134897184,v1:172.21.15.148:6829/1134897184] conn(0x7f25e406af90 msgr2=0x7f25e406d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:06:52.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:52.804+0000 7f2607200700 1 --2- 172.21.15.37:0/3832027942 >> [v2:172.21.15.148:6828/1134897184,v1:172.21.15.148:6829/1134897184] conn(0x7f25e406af90 0x7f25e406d440 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f25e8004f40 tx=0x7f25e800b040 comp rx=0 tx=0).stop 2024-03-29T08:06:52.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:52.804+0000 7f2607200700 1 -- 172.21.15.37:0/3832027942 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f26000fff80 msgr2=0x7f26000fd470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:06:52.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:52.804+0000 7f2607200700 1 --2- 172.21.15.37:0/3832027942 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f26000fff80 0x7f26000fd470 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f25f000b900 tx=0x7f25f0009e30 comp rx=0 tx=0).stop 2024-03-29T08:06:52.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:52.804+0000 7f2607200700 1 -- 172.21.15.37:0/3832027942 shutdown_connections 2024-03-29T08:06:52.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:52.804+0000 7f2607200700 1 --2- 172.21.15.37:0/3832027942 >> [v2:172.21.15.148:6828/1134897184,v1:172.21.15.148:6829/1134897184] conn(0x7f25e406af90 0x7f25e406d440 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:06:52.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:52.804+0000 7f2607200700 1 --2- 172.21.15.37:0/3832027942 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f26000fff80 0x7f26000fd470 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:06:52.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:52.804+0000 7f2607200700 1 --2- 172.21.15.37:0/3832027942 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f26000fb940 0x7f26000fbdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:06:52.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:52.804+0000 7f2607200700 1 -- 172.21.15.37:0/3832027942 >> 172.21.15.37:0/3832027942 conn(0x7f26000f7430 msgr2=0x7f26000f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:06:52.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:52.804+0000 7f2607200700 1 -- 172.21.15.37:0/3832027942 shutdown_connections 2024-03-29T08:06:52.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:52.804+0000 7f2607200700 1 -- 172.21.15.37:0/3832027942 wait complete. 2024-03-29T08:06:52.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:52.995+0000 7f0a16000700 1 -- 172.21.15.37:0/3154487538 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0a100fee90 msgr2=0x7f0a100ff2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:06:52.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:52.995+0000 7f0a16000700 1 --2- 172.21.15.37:0/3154487538 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0a100fee90 0x7f0a100ff2a0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f0a04004940 tx=0x7f0a04009e30 comp rx=0 tx=0).stop 2024-03-29T08:06:52.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:52.995+0000 7f0a16000700 1 -- 172.21.15.37:0/3154487538 shutdown_connections 2024-03-29T08:06:52.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:52.995+0000 7f0a16000700 1 --2- 172.21.15.37:0/3154487538 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0a10100090 0x7f0a10100500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:06:52.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:52.995+0000 7f0a16000700 1 --2- 172.21.15.37:0/3154487538 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0a100fee90 0x7f0a100ff2a0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:06:52.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:52.995+0000 7f0a16000700 1 -- 172.21.15.37:0/3154487538 >> 172.21.15.37:0/3154487538 conn(0x7f0a100fa5d0 msgr2=0x7f0a100fc9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:06:52.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:52.995+0000 7f0a16000700 1 -- 172.21.15.37:0/3154487538 shutdown_connections 2024-03-29T08:06:52.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:52.995+0000 7f0a16000700 1 -- 172.21.15.37:0/3154487538 wait complete. 2024-03-29T08:06:52.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:52.997+0000 7f0a16000700 1 Processor -- start 2024-03-29T08:06:52.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:52.997+0000 7f0a16000700 1 -- start start 2024-03-29T08:06:52.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:52.997+0000 7f0a16000700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0a100fee90 0x7f0a1010f960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:06:52.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:52.997+0000 7f0a16000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0a10100090 0x7f0a1010fea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:06:52.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:52.997+0000 7f0a16000700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a101104a0 con 0x7f0a10100090 2024-03-29T08:06:52.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:52.997+0000 7f0a16000700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a10110cf0 con 0x7f0a100fee90 2024-03-29T08:06:52.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:52.998+0000 7f0a14c00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0a100fee90 0x7f0a1010f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:06:52.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:52.998+0000 7f0a14c00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0a100fee90 0x7f0a1010f960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.37:38650/0 (socket says 172.21.15.37:38650) 2024-03-29T08:06:52.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:52.998+0000 7f0a14c00700 1 -- 172.21.15.37:0/475602924 learned_addr learned my addr 172.21.15.37:0/475602924 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:06:52.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:52.998+0000 7f0a0fe00700 1 --2- 172.21.15.37:0/475602924 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0a10100090 0x7f0a1010fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:06:52.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:52.998+0000 7f0a14c00700 1 -- 172.21.15.37:0/475602924 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0a10100090 msgr2=0x7f0a1010fea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:06:52.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:52.998+0000 7f0a14c00700 1 --2- 172.21.15.37:0/475602924 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0a10100090 0x7f0a1010fea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:06:52.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:52.998+0000 7f0a14c00700 1 -- 172.21.15.37:0/475602924 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a04005040 con 0x7f0a100fee90 2024-03-29T08:06:52.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:52.998+0000 7f0a14c00700 1 --2- 172.21.15.37:0/475602924 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0a100fee90 0x7f0a1010f960 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f0a0400b0d0 tx=0x7f0a04009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:06:52.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:52.999+0000 7f0a0cc00700 1 -- 172.21.15.37:0/475602924 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0a0400e070 con 0x7f0a100fee90 2024-03-29T08:06:52.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:52.999+0000 7f0a0cc00700 1 -- 172.21.15.37:0/475602924 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f0a0401d460 con 0x7f0a100fee90 2024-03-29T08:06:52.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:52.999+0000 7f0a16000700 1 -- 172.21.15.37:0/475602924 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a10110f70 con 0x7f0a100fee90 2024-03-29T08:06:52.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:52.999+0000 7f0a0cc00700 1 -- 172.21.15.37:0/475602924 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0a0400b650 con 0x7f0a100fee90 2024-03-29T08:06:53.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:52.999+0000 7f0a16000700 1 -- 172.21.15.37:0/475602924 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a101114b0 con 0x7f0a100fee90 2024-03-29T08:06:53.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:53.001+0000 7f0a16000700 1 -- 172.21.15.37:0/475602924 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a1005f5f0 con 0x7f0a100fee90 2024-03-29T08:06:53.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:53.001+0000 7f0a0cc00700 1 -- 172.21.15.37:0/475602924 <== mon.1 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 25) v1 ==== 88766+0+0 (secure 0 0 0) 0x7f0a0401da20 con 0x7f0a100fee90 2024-03-29T08:06:53.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:53.003+0000 7f0a0cc00700 1 --2- 172.21.15.37:0/475602924 >> [v2:172.21.15.148:6828/1134897184,v1:172.21.15.148:6829/1134897184] conn(0x7f09f806aec0 0x7f09f806d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:06:53.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:53.003+0000 7f0a0cc00700 1 -- 172.21.15.37:0/475602924 <== mon.1 v2:172.21.15.148:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f0a0408a830 con 0x7f0a100fee90 2024-03-29T08:06:53.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:53.003+0000 7f0a0fe00700 1 --2- 172.21.15.37:0/475602924 >> [v2:172.21.15.148:6828/1134897184,v1:172.21.15.148:6829/1134897184] conn(0x7f09f806aec0 0x7f09f806d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:06:53.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:53.005+0000 7f0a0fe00700 1 --2- 172.21.15.37:0/475602924 >> [v2:172.21.15.148:6828/1134897184,v1:172.21.15.148:6829/1134897184] conn(0x7f09f806aec0 0x7f09f806d370 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f09fc009b80 tx=0x7f09fc00b040 comp rx=0 tx=0).ready entity=mgr.14602 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:06:53.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:53.007+0000 7f0a0cc00700 1 -- 172.21.15.37:0/475602924 <== mon.1 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181579 (secure 0 0 0) 0x7f0a040587c0 con 0x7f0a100fee90 2024-03-29T08:06:53.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:06:53 smithi037 ceph-mon[35162]: from='client.24475 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:06:53.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:06:53 smithi037 ceph-mon[35162]: pgmap v28: 65 pgs: 65 active+clean; 170 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 2.8 MiB/s wr, 265 op/s 2024-03-29T08:06:53.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:06:53 smithi037 ceph-mon[35162]: from='client.? 172.21.15.37:0/3832027942' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:06:53.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:06:53 smithi148 ceph-mon[44992]: from='client.24475 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:06:53.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:06:53 smithi148 ceph-mon[44992]: pgmap v28: 65 pgs: 65 active+clean; 170 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 2.8 MiB/s wr, 265 op/s 2024-03-29T08:06:53.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:06:53 smithi148 ceph-mon[44992]: from='client.? 172.21.15.37:0/3832027942' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:06:53.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:53.429+0000 7f0a16000700 1 -- 172.21.15.37:0/475602924 --> [v2:172.21.15.148:6828/1134897184,v1:172.21.15.148:6829/1134897184] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0a101117e0 con 0x7f09f806aec0 2024-03-29T08:06:53.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:53.431+0000 7f0a0cc00700 1 -- 172.21.15.37:0/475602924 <== mgr.14602 v2:172.21.15.148:6828/1134897184 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+310 (secure 0 0 0) 0x7f0a101117e0 con 0x7f09f806aec0 2024-03-29T08:06:53.430 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-29T08:06:53.431 INFO:teuthology.orchestra.run.smithi037.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59", 2024-03-29T08:06:53.431 INFO:teuthology.orchestra.run.smithi037.stdout: "in_progress": true, 2024-03-29T08:06:53.431 INFO:teuthology.orchestra.run.smithi037.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-03-29T08:06:53.431 INFO:teuthology.orchestra.run.smithi037.stdout: "services_complete": [], 2024-03-29T08:06:53.431 INFO:teuthology.orchestra.run.smithi037.stdout: "progress": "1/2 daemons upgraded", 2024-03-29T08:06:53.431 INFO:teuthology.orchestra.run.smithi037.stdout: "message": "", 2024-03-29T08:06:53.431 INFO:teuthology.orchestra.run.smithi037.stdout: "is_paused": false 2024-03-29T08:06:53.431 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-29T08:06:53.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:53.435+0000 7f0a16000700 1 -- 172.21.15.37:0/475602924 >> [v2:172.21.15.148:6828/1134897184,v1:172.21.15.148:6829/1134897184] conn(0x7f09f806aec0 msgr2=0x7f09f806d370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:06:53.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:53.435+0000 7f0a16000700 1 --2- 172.21.15.37:0/475602924 >> [v2:172.21.15.148:6828/1134897184,v1:172.21.15.148:6829/1134897184] conn(0x7f09f806aec0 0x7f09f806d370 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f09fc009b80 tx=0x7f09fc00b040 comp rx=0 tx=0).stop 2024-03-29T08:06:53.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:53.435+0000 7f0a16000700 1 -- 172.21.15.37:0/475602924 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0a100fee90 msgr2=0x7f0a1010f960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:06:53.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:53.435+0000 7f0a16000700 1 --2- 172.21.15.37:0/475602924 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0a100fee90 0x7f0a1010f960 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f0a0400b0d0 tx=0x7f0a04009e30 comp rx=0 tx=0).stop 2024-03-29T08:06:53.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:53.438+0000 7f0a16000700 1 -- 172.21.15.37:0/475602924 shutdown_connections 2024-03-29T08:06:53.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:53.438+0000 7f0a16000700 1 --2- 172.21.15.37:0/475602924 >> [v2:172.21.15.148:6828/1134897184,v1:172.21.15.148:6829/1134897184] conn(0x7f09f806aec0 0x7f09f806d370 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:06:53.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:53.438+0000 7f0a16000700 1 --2- 172.21.15.37:0/475602924 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0a100fee90 0x7f0a1010f960 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:06:53.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:53.438+0000 7f0a16000700 1 --2- 172.21.15.37:0/475602924 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0a10100090 0x7f0a1010fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:06:53.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:53.438+0000 7f0a16000700 1 -- 172.21.15.37:0/475602924 >> 172.21.15.37:0/475602924 conn(0x7f0a100fa5d0 msgr2=0x7f0a100fbd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:06:53.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:53.438+0000 7f0a16000700 1 -- 172.21.15.37:0/475602924 shutdown_connections 2024-03-29T08:06:53.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:06:53.438+0000 7f0a16000700 1 -- 172.21.15.37:0/475602924 wait complete. 2024-03-29T08:06:54.278 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:06:54 smithi037 ceph-mon[35162]: from='client.24479 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:06:54.305 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:06:54 smithi148 ceph-mon[44992]: from='client.24479 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:06:55.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:06:55 smithi037 ceph-mon[35162]: from='client.24487 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:06:55.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:06:55 smithi037 ceph-mon[35162]: pgmap v29: 65 pgs: 65 active+clean; 183 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 3.9 MiB/s wr, 408 op/s 2024-03-29T08:06:55.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:06:55 smithi037 ceph-mon[35162]: from='mgr.14602 ' entity='mgr.smithi148.onhgvm' 2024-03-29T08:06:55.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:06:55 smithi037 ceph-mon[35162]: from='mgr.14602 ' entity='mgr.smithi148.onhgvm' 2024-03-29T08:06:55.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:06:55 smithi148 ceph-mon[44992]: from='client.24487 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:06:55.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:06:55 smithi148 ceph-mon[44992]: pgmap v29: 65 pgs: 65 active+clean; 183 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 3.9 MiB/s wr, 408 op/s 2024-03-29T08:06:55.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:06:55 smithi148 ceph-mon[44992]: from='mgr.14602 ' entity='mgr.smithi148.onhgvm' 2024-03-29T08:06:55.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:06:55 smithi148 ceph-mon[44992]: from='mgr.14602 ' entity='mgr.smithi148.onhgvm' 2024-03-29T08:06:56.305 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:06:56 smithi037 ceph-mon[35162]: Reconfiguring prometheus.smithi037 (dependencies changed)... 2024-03-29T08:06:56.305 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:06:56 smithi037 ceph-mon[35162]: Reconfiguring daemon prometheus.smithi037 on smithi037 2024-03-29T08:06:56.305 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:06:56 smithi148 ceph-mon[44992]: Reconfiguring prometheus.smithi037 (dependencies changed)... 2024-03-29T08:06:56.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:06:56 smithi148 ceph-mon[44992]: Reconfiguring daemon prometheus.smithi037 on smithi037 2024-03-29T08:06:57.290 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:06:57 smithi037 ceph-mon[35162]: pgmap v30: 65 pgs: 65 active+clean; 183 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 2.5 MiB/s wr, 270 op/s 2024-03-29T08:06:57.305 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:06:57 smithi148 ceph-mon[44992]: pgmap v30: 65 pgs: 65 active+clean; 183 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 2.5 MiB/s wr, 270 op/s 2024-03-29T08:06:58.601 INFO:tasks.workunit.client.0.smithi037.stderr:+ pushd ltp-full-20091231/testcases/kernel/fs/fsstress 2024-03-29T08:06:58.614 INFO:tasks.workunit.client.0.smithi037.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress ~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-03-29T08:06:58.614 INFO:tasks.workunit.client.0.smithi037.stderr:+ make 2024-03-29T08:06:58.821 INFO:tasks.workunit.client.0.smithi037.stdout:cc -DNO_XFS -I/home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress -D_LARGEFILE64_SOURCE -D_GNU_SOURCE -I../../../../include -I../../../../include -L../../../../lib fsstress.c -o fsstress 2024-03-29T08:06:59.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:06:59 smithi148 ceph-mon[44992]: pgmap v31: 65 pgs: 65 active+clean; 191 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 3.2 MiB/s wr, 317 op/s 2024-03-29T08:06:59.426 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:06:59 smithi037 ceph-mon[35162]: pgmap v31: 65 pgs: 65 active+clean; 191 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 3.2 MiB/s wr, 317 op/s 2024-03-29T08:06:59.591 INFO:tasks.workunit.client.0.smithi037.stderr:++ readlink -f fsstress 2024-03-29T08:06:59.595 INFO:tasks.workunit.client.0.smithi037.stderr:+ BIN=/home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress 2024-03-29T08:06:59.595 INFO:tasks.workunit.client.0.smithi037.stderr:+ popd 2024-03-29T08:06:59.597 INFO:tasks.workunit.client.0.smithi037.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-03-29T08:06:59.597 INFO:tasks.workunit.client.0.smithi037.stderr:+ popd 2024-03-29T08:06:59.599 INFO:tasks.workunit.client.0.smithi037.stdout:~/cephtest/mnt.0/client.0/tmp 2024-03-29T08:06:59.599 INFO:tasks.workunit.client.0.smithi037.stderr:++ mktemp -d -p . 2024-03-29T08:06:59.604 INFO:tasks.workunit.client.0.smithi037.stderr:+ T=./tmp.Rc0GmqBkLJ 2024-03-29T08:06:59.604 INFO:tasks.workunit.client.0.smithi037.stderr:+ /home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress -d ./tmp.Rc0GmqBkLJ -l 1 -n 1000 -p 10 -v 2024-03-29T08:06:59.610 INFO:tasks.workunit.client.0.smithi037.stdout:seed = 1712268210 2024-03-29T08:06:59.615 INFO:tasks.workunit.client.0.smithi037.stdout:0/0: write - no filename 2024-03-29T08:06:59.617 INFO:tasks.workunit.client.0.smithi037.stdout:0/1: mkdir d0 0 2024-03-29T08:06:59.617 INFO:tasks.workunit.client.0.smithi037.stdout:0/2: write - no filename 2024-03-29T08:06:59.617 INFO:tasks.workunit.client.0.smithi037.stdout:0/3: readlink - no filename 2024-03-29T08:06:59.618 INFO:tasks.workunit.client.0.smithi037.stdout:0/4: rename d0 to d0/d1 22 2024-03-29T08:06:59.619 INFO:tasks.workunit.client.0.smithi037.stdout:0/5: dwrite - no filename 2024-03-29T08:06:59.620 INFO:tasks.workunit.client.0.smithi037.stdout:0/6: chown d0 94152 1 2024-03-29T08:06:59.620 INFO:tasks.workunit.client.0.smithi037.stdout:0/7: truncate - no filename 2024-03-29T08:06:59.621 INFO:tasks.workunit.client.0.smithi037.stdout:7/0: creat f0 x:0 0 0 2024-03-29T08:06:59.623 INFO:tasks.workunit.client.0.smithi037.stdout:7/1: chown f0 10681641 1 2024-03-29T08:06:59.624 INFO:tasks.workunit.client.0.smithi037.stdout:2/0: dread - no filename 2024-03-29T08:06:59.624 INFO:tasks.workunit.client.0.smithi037.stdout:2/1: dwrite - no filename 2024-03-29T08:06:59.624 INFO:tasks.workunit.client.0.smithi037.stdout:2/2: read - no filename 2024-03-29T08:06:59.624 INFO:tasks.workunit.client.0.smithi037.stdout:2/3: readlink - no filename 2024-03-29T08:06:59.624 INFO:tasks.workunit.client.0.smithi037.stdout:2/4: dwrite - no filename 2024-03-29T08:06:59.624 INFO:tasks.workunit.client.0.smithi037.stdout:1/0: symlink l0 0 2024-03-29T08:06:59.627 INFO:tasks.workunit.client.0.smithi037.stdout:7/2: mknod c1 0 2024-03-29T08:06:59.627 INFO:tasks.workunit.client.0.smithi037.stdout:3/0: dread - no filename 2024-03-29T08:06:59.628 INFO:tasks.workunit.client.0.smithi037.stdout:0/8: rmdir d0 0 2024-03-29T08:06:59.628 INFO:tasks.workunit.client.0.smithi037.stdout:0/9: write - no filename 2024-03-29T08:06:59.628 INFO:tasks.workunit.client.0.smithi037.stdout:0/10: link - no file 2024-03-29T08:06:59.628 INFO:tasks.workunit.client.0.smithi037.stdout:0/11: rename - no filename 2024-03-29T08:06:59.629 INFO:tasks.workunit.client.0.smithi037.stdout:0/12: rmdir - no directory 2024-03-29T08:06:59.629 INFO:tasks.workunit.client.0.smithi037.stdout:0/13: write - no filename 2024-03-29T08:06:59.629 INFO:tasks.workunit.client.0.smithi037.stdout:0/14: truncate - no filename 2024-03-29T08:06:59.629 INFO:tasks.workunit.client.0.smithi037.stdout:0/15: chown . 844 1 2024-03-29T08:06:59.629 INFO:tasks.workunit.client.0.smithi037.stdout:0/16: dread - no filename 2024-03-29T08:06:59.629 INFO:tasks.workunit.client.0.smithi037.stdout:1/1: symlink l1 0 2024-03-29T08:06:59.630 INFO:tasks.workunit.client.0.smithi037.stdout:1/2: dwrite - no filename 2024-03-29T08:06:59.630 INFO:tasks.workunit.client.0.smithi037.stdout:1/3: write - no filename 2024-03-29T08:06:59.630 INFO:tasks.workunit.client.0.smithi037.stdout:1/4: dread - no filename 2024-03-29T08:06:59.630 INFO:tasks.workunit.client.0.smithi037.stdout:1/5: fdatasync - no filename 2024-03-29T08:06:59.630 INFO:tasks.workunit.client.0.smithi037.stdout:1/6: truncate - no filename 2024-03-29T08:06:59.630 INFO:tasks.workunit.client.0.smithi037.stdout:1/7: dwrite - no filename 2024-03-29T08:06:59.630 INFO:tasks.workunit.client.0.smithi037.stdout:7/3: write f0 [98507,97219] 0 2024-03-29T08:06:59.630 INFO:tasks.workunit.client.0.smithi037.stdout:0/17: chown . 119 1 2024-03-29T08:06:59.630 INFO:tasks.workunit.client.0.smithi037.stdout:0/18: fsync - no filename 2024-03-29T08:06:59.630 INFO:tasks.workunit.client.0.smithi037.stdout:0/19: dwrite - no filename 2024-03-29T08:06:59.630 INFO:tasks.workunit.client.0.smithi037.stdout:0/20: write - no filename 2024-03-29T08:06:59.630 INFO:tasks.workunit.client.0.smithi037.stdout:0/21: dwrite - no filename 2024-03-29T08:06:59.630 INFO:tasks.workunit.client.0.smithi037.stdout:0/22: fsync - no filename 2024-03-29T08:06:59.632 INFO:tasks.workunit.client.0.smithi037.stdout:2/5: mkdir d0 0 2024-03-29T08:06:59.633 INFO:tasks.workunit.client.0.smithi037.stdout:1/8: mkdir d2 0 2024-03-29T08:06:59.633 INFO:tasks.workunit.client.0.smithi037.stdout:1/9: write - no filename 2024-03-29T08:06:59.633 INFO:tasks.workunit.client.0.smithi037.stdout:1/10: write - no filename 2024-03-29T08:06:59.633 INFO:tasks.workunit.client.0.smithi037.stdout:2/6: chown d0 147710380 1 2024-03-29T08:06:59.633 INFO:tasks.workunit.client.0.smithi037.stdout:2/7: dwrite - no filename 2024-03-29T08:06:59.635 INFO:tasks.workunit.client.0.smithi037.stdout:7/4: creat f2 x:0 0 0 2024-03-29T08:06:59.635 INFO:tasks.workunit.client.0.smithi037.stdout:4/0: stat - no entries 2024-03-29T08:06:59.636 INFO:tasks.workunit.client.0.smithi037.stdout:0/23: mknod c2 0 2024-03-29T08:06:59.636 INFO:tasks.workunit.client.0.smithi037.stdout:3/1: symlink l0 0 2024-03-29T08:06:59.636 INFO:tasks.workunit.client.0.smithi037.stdout:3/2: write - no filename 2024-03-29T08:06:59.637 INFO:tasks.workunit.client.0.smithi037.stdout:7/5: read f0 [136459,73262] 0 2024-03-29T08:06:59.639 INFO:tasks.workunit.client.0.smithi037.stdout:5/0: fdatasync - no filename 2024-03-29T08:06:59.639 INFO:tasks.workunit.client.0.smithi037.stdout:5/1: write - no filename 2024-03-29T08:06:59.640 INFO:tasks.workunit.client.0.smithi037.stdout:5/2: dwrite - no filename 2024-03-29T08:06:59.640 INFO:tasks.workunit.client.0.smithi037.stdout:5/3: dwrite - no filename 2024-03-29T08:06:59.641 INFO:tasks.workunit.client.0.smithi037.stdout:1/11: rename l1 to d2/l3 0 2024-03-29T08:06:59.641 INFO:tasks.workunit.client.0.smithi037.stdout:1/12: dread - no filename 2024-03-29T08:06:59.644 INFO:tasks.workunit.client.0.smithi037.stdout:3/3: rename l0 to l1 0 2024-03-29T08:06:59.645 INFO:tasks.workunit.client.0.smithi037.stdout:3/4: rmdir - no directory 2024-03-29T08:06:59.645 INFO:tasks.workunit.client.0.smithi037.stdout:3/5: write - no filename 2024-03-29T08:06:59.645 INFO:tasks.workunit.client.0.smithi037.stdout:3/6: read - no filename 2024-03-29T08:06:59.645 INFO:tasks.workunit.client.0.smithi037.stdout:3/7: dread - no filename 2024-03-29T08:06:59.645 INFO:tasks.workunit.client.0.smithi037.stdout:7/6: dread f0 [0,4194304] 0 2024-03-29T08:06:59.647 INFO:tasks.workunit.client.0.smithi037.stdout:2/8: mknod d0/c1 0 2024-03-29T08:06:59.650 INFO:tasks.workunit.client.0.smithi037.stdout:1/13: symlink d2/l4 0 2024-03-29T08:06:59.654 INFO:tasks.workunit.client.0.smithi037.stdout:8/0: fdatasync - no filename 2024-03-29T08:06:59.654 INFO:tasks.workunit.client.0.smithi037.stdout:8/1: dwrite - no filename 2024-03-29T08:06:59.654 INFO:tasks.workunit.client.0.smithi037.stdout:8/2: unlink - no file 2024-03-29T08:06:59.656 INFO:tasks.workunit.client.0.smithi037.stdout:2/9: creat d0/f2 x:0 0 0 2024-03-29T08:06:59.657 INFO:tasks.workunit.client.0.smithi037.stdout:7/7: unlink c1 0 2024-03-29T08:06:59.658 INFO:tasks.workunit.client.0.smithi037.stdout:5/4: symlink l0 0 2024-03-29T08:06:59.658 INFO:tasks.workunit.client.0.smithi037.stdout:5/5: truncate - no filename 2024-03-29T08:06:59.658 INFO:tasks.workunit.client.0.smithi037.stdout:6/0: mkdir d0 0 2024-03-29T08:06:59.658 INFO:tasks.workunit.client.0.smithi037.stdout:1/14: creat d2/f5 x:0 0 0 2024-03-29T08:06:59.663 INFO:tasks.workunit.client.0.smithi037.stdout:2/10: rename d0 to d0/d3 22 2024-03-29T08:06:59.663 INFO:tasks.workunit.client.0.smithi037.stdout:7/8: chown f0 27172785 1 2024-03-29T08:06:59.663 INFO:tasks.workunit.client.0.smithi037.stdout:5/6: symlink l1 0 2024-03-29T08:06:59.665 INFO:tasks.workunit.client.0.smithi037.stdout:8/3: creat f0 x:0 0 0 2024-03-29T08:06:59.666 INFO:tasks.workunit.client.0.smithi037.stdout:9/0: dwrite - no filename 2024-03-29T08:06:59.668 INFO:tasks.workunit.client.0.smithi037.stdout:8/4: truncate f0 788656 0 2024-03-29T08:06:59.668 INFO:tasks.workunit.client.0.smithi037.stdout:7/9: creat f3 x:0 0 0 2024-03-29T08:06:59.670 INFO:tasks.workunit.client.0.smithi037.stdout:1/15: mkdir d2/d6 0 2024-03-29T08:06:59.670 INFO:tasks.workunit.client.0.smithi037.stdout:5/7: unlink l1 0 2024-03-29T08:06:59.670 INFO:tasks.workunit.client.0.smithi037.stdout:5/8: dwrite - no filename 2024-03-29T08:06:59.673 INFO:tasks.workunit.client.0.smithi037.stdout:5/9: chown l0 102520749 1 2024-03-29T08:06:59.673 INFO:tasks.workunit.client.0.smithi037.stdout:5/10: dwrite - no filename 2024-03-29T08:06:59.673 INFO:tasks.workunit.client.0.smithi037.stdout:5/11: rmdir - no directory 2024-03-29T08:06:59.676 INFO:tasks.workunit.client.0.smithi037.stdout:8/5: creat f1 x:0 0 0 2024-03-29T08:06:59.678 INFO:tasks.workunit.client.0.smithi037.stdout:8/6: fdatasync f1 0 2024-03-29T08:06:59.682 INFO:tasks.workunit.client.0.smithi037.stdout:9/1: getdents . 0 2024-03-29T08:06:59.682 INFO:tasks.workunit.client.0.smithi037.stdout:1/16: unlink l0 0 2024-03-29T08:06:59.683 INFO:tasks.workunit.client.0.smithi037.stdout:9/2: chown . 895391 1 2024-03-29T08:06:59.683 INFO:tasks.workunit.client.0.smithi037.stdout:9/3: dread - no filename 2024-03-29T08:06:59.683 INFO:tasks.workunit.client.0.smithi037.stdout:9/4: fsync - no filename 2024-03-29T08:06:59.683 INFO:tasks.workunit.client.0.smithi037.stdout:9/5: rename - no filename 2024-03-29T08:06:59.683 INFO:tasks.workunit.client.0.smithi037.stdout:9/6: rmdir - no directory 2024-03-29T08:06:59.683 INFO:tasks.workunit.client.0.smithi037.stdout:8/7: chown f1 0 1 2024-03-29T08:06:59.687 INFO:tasks.workunit.client.0.smithi037.stdout:1/17: chown d2/l4 453669 1 2024-03-29T08:06:59.688 INFO:tasks.workunit.client.0.smithi037.stdout:5/12: creat f2 x:0 0 0 2024-03-29T08:06:59.692 INFO:tasks.workunit.client.0.smithi037.stdout:8/8: symlink l2 0 2024-03-29T08:06:59.695 INFO:tasks.workunit.client.0.smithi037.stdout:9/7: creat f0 x:0 0 0 2024-03-29T08:06:59.697 INFO:tasks.workunit.client.0.smithi037.stdout:1/18: symlink d2/d6/l7 0 2024-03-29T08:06:59.701 INFO:tasks.workunit.client.0.smithi037.stdout:8/9: mknod c3 0 2024-03-29T08:06:59.701 INFO:tasks.workunit.client.0.smithi037.stdout:9/8: stat f0 0 2024-03-29T08:06:59.704 INFO:tasks.workunit.client.0.smithi037.stdout:1/19: mknod d2/c8 0 2024-03-29T08:06:59.710 INFO:tasks.workunit.client.0.smithi037.stdout:9/9: mkdir d1 0 2024-03-29T08:06:59.720 INFO:tasks.workunit.client.0.smithi037.stdout:1/20: mknod d2/c9 0 2024-03-29T08:06:59.723 INFO:tasks.workunit.client.0.smithi037.stdout:2/11: dwrite d0/f2 [0,4194304] 0 2024-03-29T08:06:59.733 INFO:tasks.workunit.client.0.smithi037.stdout:9/10: mknod d1/c2 0 2024-03-29T08:06:59.738 INFO:tasks.workunit.client.0.smithi037.stdout:1/21: symlink d2/d6/la 0 2024-03-29T08:06:59.742 INFO:tasks.workunit.client.0.smithi037.stdout:9/11: dread - f0 zero size 2024-03-29T08:06:59.748 INFO:tasks.workunit.client.0.smithi037.stdout:7/10: dwrite f0 [0,4194304] 0 2024-03-29T08:06:59.756 INFO:tasks.workunit.client.0.smithi037.stdout:9/12: truncate f0 69698 0 2024-03-29T08:06:59.756 INFO:tasks.workunit.client.0.smithi037.stdout:1/22: mknod d2/d6/cb 0 2024-03-29T08:06:59.761 INFO:tasks.workunit.client.0.smithi037.stdout:7/11: symlink l4 0 2024-03-29T08:06:59.762 INFO:tasks.workunit.client.0.smithi037.stdout:3/8: readlink l1 0 2024-03-29T08:06:59.762 INFO:tasks.workunit.client.0.smithi037.stdout:5/13: dwrite f2 [0,4194304] 0 2024-03-29T08:06:59.767 INFO:tasks.workunit.client.0.smithi037.stdout:3/9: chown l1 166 1 2024-03-29T08:06:59.769 INFO:tasks.workunit.client.0.smithi037.stdout:1/23: mkdir d2/d6/dc 0 2024-03-29T08:06:59.770 INFO:tasks.workunit.client.0.smithi037.stdout:7/12: fsync f3 0 2024-03-29T08:06:59.770 INFO:tasks.workunit.client.0.smithi037.stdout:2/12: fdatasync d0/f2 0 2024-03-29T08:06:59.786 INFO:tasks.workunit.client.0.smithi037.stdout:8/10: dwrite f0 [0,4194304] 0 2024-03-29T08:06:59.787 INFO:tasks.workunit.client.0.smithi037.stdout:7/13: rename f2 to f5 0 2024-03-29T08:06:59.791 INFO:tasks.workunit.client.0.smithi037.stdout:3/10: creat f2 x:0 0 0 2024-03-29T08:06:59.792 INFO:tasks.workunit.client.0.smithi037.stdout:7/14: dread - f3 zero size 2024-03-29T08:06:59.797 INFO:tasks.workunit.client.0.smithi037.stdout:8/11: chown c3 10835 1 2024-03-29T08:06:59.798 INFO:tasks.workunit.client.0.smithi037.stdout:3/11: symlink l3 0 2024-03-29T08:06:59.802 INFO:tasks.workunit.client.0.smithi037.stdout:7/15: dread - f5 zero size 2024-03-29T08:06:59.818 INFO:tasks.workunit.client.0.smithi037.stdout:8/12: write f1 [677553,53091] 0 2024-03-29T08:06:59.840 INFO:tasks.workunit.client.0.smithi037.stdout:1/24: dwrite d2/f5 [0,4194304] 0 2024-03-29T08:06:59.854 INFO:tasks.workunit.client.0.smithi037.stdout:5/14: dwrite f2 [0,4194304] 0 2024-03-29T08:06:59.859 INFO:tasks.workunit.client.0.smithi037.stdout:8/13: truncate f0 1029124 0 2024-03-29T08:06:59.861 INFO:tasks.workunit.client.0.smithi037.stdout:2/13: dwrite d0/f2 [0,4194304] 0 2024-03-29T08:06:59.861 INFO:tasks.workunit.client.0.smithi037.stdout:9/13: dwrite f0 [0,4194304] 0 2024-03-29T08:06:59.864 INFO:tasks.workunit.client.0.smithi037.stdout:8/14: fsync f1 0 2024-03-29T08:06:59.870 INFO:tasks.workunit.client.0.smithi037.stdout:5/15: creat f3 x:0 0 0 2024-03-29T08:06:59.871 INFO:tasks.workunit.client.0.smithi037.stdout:8/15: creat f4 x:0 0 0 2024-03-29T08:06:59.877 INFO:tasks.workunit.client.0.smithi037.stdout:5/16: chown l0 1557 1 2024-03-29T08:06:59.885 INFO:tasks.workunit.client.0.smithi037.stdout:9/14: mkdir d1/d3 0 2024-03-29T08:06:59.886 INFO:tasks.workunit.client.0.smithi037.stdout:3/12: dwrite f2 [0,4194304] 0 2024-03-29T08:06:59.889 INFO:tasks.workunit.client.0.smithi037.stdout:8/16: mkdir d5 0 2024-03-29T08:06:59.891 INFO:tasks.workunit.client.0.smithi037.stdout:3/13: write f2 [308910,57953] 0 2024-03-29T08:06:59.892 INFO:tasks.workunit.client.0.smithi037.stdout:7/16: dwrite f3 [0,4194304] 0 2024-03-29T08:06:59.893 INFO:tasks.workunit.client.0.smithi037.stdout:9/15: mkdir d1/d4 0 2024-03-29T08:06:59.902 INFO:tasks.workunit.client.0.smithi037.stdout:1/25: fsync d2/f5 0 2024-03-29T08:06:59.904 INFO:tasks.workunit.client.0.smithi037.stdout:7/17: mknod c6 0 2024-03-29T08:06:59.905 INFO:tasks.workunit.client.0.smithi037.stdout:9/16: symlink d1/l5 0 2024-03-29T08:06:59.908 INFO:tasks.workunit.client.0.smithi037.stdout:3/14: creat f4 x:0 0 0 2024-03-29T08:06:59.910 INFO:tasks.workunit.client.0.smithi037.stdout:2/14: dread d0/f2 [0,4194304] 0 2024-03-29T08:06:59.915 INFO:tasks.workunit.client.0.smithi037.stdout:9/17: mkdir d1/d6 0 2024-03-29T08:06:59.916 INFO:tasks.workunit.client.0.smithi037.stdout:9/18: chown d1 7909 1 2024-03-29T08:06:59.917 INFO:tasks.workunit.client.0.smithi037.stdout:9/19: chown d1 448392 1 2024-03-29T08:06:59.920 INFO:tasks.workunit.client.0.smithi037.stdout:3/15: creat f5 x:0 0 0 2024-03-29T08:06:59.921 INFO:tasks.workunit.client.0.smithi037.stdout:5/17: getdents . 0 2024-03-29T08:06:59.922 INFO:tasks.workunit.client.0.smithi037.stdout:3/16: chown l3 1311270 1 2024-03-29T08:06:59.922 INFO:tasks.workunit.client.0.smithi037.stdout:9/20: truncate f0 4673576 0 2024-03-29T08:06:59.925 INFO:tasks.workunit.client.0.smithi037.stdout:5/18: truncate f2 4205843 0 2024-03-29T08:06:59.925 INFO:tasks.workunit.client.0.smithi037.stdout:1/26: dread d2/f5 [0,4194304] 0 2024-03-29T08:06:59.926 INFO:tasks.workunit.client.0.smithi037.stdout:9/21: chown f0 95372 1 2024-03-29T08:06:59.926 INFO:tasks.workunit.client.0.smithi037.stdout:5/19: stat l0 0 2024-03-29T08:06:59.929 INFO:tasks.workunit.client.0.smithi037.stdout:5/20: write f3 [283625,98365] 0 2024-03-29T08:06:59.932 INFO:tasks.workunit.client.0.smithi037.stdout:3/17: mknod c6 0 2024-03-29T08:06:59.936 INFO:tasks.workunit.client.0.smithi037.stdout:1/27: creat d2/d6/dc/fd x:0 0 0 2024-03-29T08:06:59.946 INFO:tasks.workunit.client.0.smithi037.stdout:5/21: rename f2 to f4 0 2024-03-29T08:06:59.953 INFO:tasks.workunit.client.0.smithi037.stdout:7/18: truncate f0 2316164 0 2024-03-29T08:06:59.955 INFO:tasks.workunit.client.0.smithi037.stdout:5/22: rename l0 to l5 0 2024-03-29T08:06:59.958 INFO:tasks.workunit.client.0.smithi037.stdout:9/22: dread f0 [0,4194304] 0 2024-03-29T08:06:59.963 INFO:tasks.workunit.client.0.smithi037.stdout:5/23: symlink l6 0 2024-03-29T08:06:59.969 INFO:tasks.workunit.client.0.smithi037.stdout:3/18: dwrite f5 [0,4194304] 0 2024-03-29T08:06:59.975 INFO:tasks.workunit.client.0.smithi037.stdout:2/15: dwrite d0/f2 [0,4194304] 0 2024-03-29T08:06:59.981 INFO:tasks.workunit.client.0.smithi037.stdout:5/24: write f4 [1129395,1801] 0 2024-03-29T08:06:59.985 INFO:tasks.workunit.client.0.smithi037.stdout:2/16: truncate d0/f2 4440133 0 2024-03-29T08:06:59.986 INFO:tasks.workunit.client.0.smithi037.stdout:9/23: mknod d1/d6/c7 0 2024-03-29T08:06:59.992 INFO:tasks.workunit.client.0.smithi037.stdout:1/28: dwrite d2/d6/dc/fd [0,4194304] 0 2024-03-29T08:06:59.995 INFO:tasks.workunit.client.0.smithi037.stdout:2/17: creat d0/f4 x:0 0 0 2024-03-29T08:07:00.003 INFO:tasks.workunit.client.0.smithi037.stdout:9/24: mknod d1/d4/c8 0 2024-03-29T08:07:00.011 INFO:tasks.workunit.client.0.smithi037.stdout:7/19: write f0 [444378,78773] 0 2024-03-29T08:07:00.015 INFO:tasks.workunit.client.0.smithi037.stdout:5/25: dread f3 [0,4194304] 0 2024-03-29T08:07:00.019 INFO:tasks.workunit.client.0.smithi037.stdout:3/19: dread f2 [0,4194304] 0 2024-03-29T08:07:00.020 INFO:tasks.workunit.client.0.smithi037.stdout:5/26: stat f4 0 2024-03-29T08:07:00.024 INFO:tasks.workunit.client.0.smithi037.stdout:9/25: write f0 [2816741,41766] 0 2024-03-29T08:07:00.024 INFO:tasks.workunit.client.0.smithi037.stdout:7/20: write f3 [2478177,54856] 0 2024-03-29T08:07:00.025 INFO:tasks.workunit.client.0.smithi037.stdout:8/17: dwrite f0 [0,4194304] 0 2024-03-29T08:07:00.037 INFO:tasks.workunit.client.0.smithi037.stdout:5/27: dread f3 [0,4194304] 0 2024-03-29T08:07:00.042 INFO:tasks.workunit.client.0.smithi037.stdout:3/20: link l1 l7 0 2024-03-29T08:07:00.043 INFO:tasks.workunit.client.0.smithi037.stdout:8/18: write f4 [678083,41036] 0 2024-03-29T08:07:00.048 INFO:tasks.workunit.client.0.smithi037.stdout:3/21: creat f8 x:0 0 0 2024-03-29T08:07:00.049 INFO:tasks.workunit.client.0.smithi037.stdout:8/19: stat f4 0 2024-03-29T08:07:00.051 INFO:tasks.workunit.client.0.smithi037.stdout:8/20: stat d5 0 2024-03-29T08:07:00.052 INFO:tasks.workunit.client.0.smithi037.stdout:1/29: dwrite d2/f5 [0,4194304] 0 2024-03-29T08:07:00.055 INFO:tasks.workunit.client.0.smithi037.stdout:3/22: write f8 [472385,64426] 0 2024-03-29T08:07:00.061 INFO:tasks.workunit.client.0.smithi037.stdout:1/30: fsync d2/d6/dc/fd 0 2024-03-29T08:07:00.067 INFO:tasks.workunit.client.0.smithi037.stdout:8/21: rename l2 to d5/l6 0 2024-03-29T08:07:00.074 INFO:tasks.workunit.client.0.smithi037.stdout:3/23: mknod c9 0 2024-03-29T08:07:00.081 INFO:tasks.workunit.client.0.smithi037.stdout:2/18: dwrite d0/f4 [0,4194304] 0 2024-03-29T08:07:00.084 INFO:tasks.workunit.client.0.smithi037.stdout:3/24: creat fa x:0 0 0 2024-03-29T08:07:00.093 INFO:tasks.workunit.client.0.smithi037.stdout:3/25: truncate f8 1197377 0 2024-03-29T08:07:00.095 INFO:tasks.workunit.client.0.smithi037.stdout:9/26: dwrite f0 [4194304,4194304] 0 2024-03-29T08:07:00.095 INFO:tasks.workunit.client.0.smithi037.stdout:2/19: mknod d0/c5 0 2024-03-29T08:07:00.095 INFO:tasks.workunit.client.0.smithi037.stdout:5/28: dread f4 [0,4194304] 0 2024-03-29T08:07:00.104 INFO:tasks.workunit.client.0.smithi037.stdout:1/31: truncate d2/d6/dc/fd 157358 0 2024-03-29T08:07:00.106 INFO:tasks.workunit.client.0.smithi037.stdout:7/21: dwrite f5 [0,4194304] 0 2024-03-29T08:07:00.112 INFO:tasks.workunit.client.0.smithi037.stdout:5/29: fdatasync f4 0 2024-03-29T08:07:00.113 INFO:tasks.workunit.client.0.smithi037.stdout:2/20: write d0/f4 [2267262,28684] 0 2024-03-29T08:07:00.113 INFO:tasks.workunit.client.0.smithi037.stdout:3/26: write f2 [4706567,69534] 0 2024-03-29T08:07:00.121 INFO:tasks.workunit.client.0.smithi037.stdout:7/22: creat f7 x:0 0 0 2024-03-29T08:07:00.121 INFO:tasks.workunit.client.0.smithi037.stdout:2/21: fdatasync d0/f2 0 2024-03-29T08:07:00.123 INFO:tasks.workunit.client.0.smithi037.stdout:1/32: rename d2/d6/la to d2/le 0 2024-03-29T08:07:00.125 INFO:tasks.workunit.client.0.smithi037.stdout:8/22: dwrite f4 [0,4194304] 0 2024-03-29T08:07:00.127 INFO:tasks.workunit.client.0.smithi037.stdout:8/23: fdatasync f1 0 2024-03-29T08:07:00.135 INFO:tasks.workunit.client.0.smithi037.stdout:5/30: rename f4 to f7 0 2024-03-29T08:07:00.138 INFO:tasks.workunit.client.0.smithi037.stdout:5/31: chown f7 560 1 2024-03-29T08:07:00.140 INFO:tasks.workunit.client.0.smithi037.stdout:1/33: unlink d2/l4 0 2024-03-29T08:07:00.142 INFO:tasks.workunit.client.0.smithi037.stdout:2/22: rmdir d0 39 2024-03-29T08:07:00.142 INFO:tasks.workunit.client.0.smithi037.stdout:9/27: dread f0 [0,4194304] 0 2024-03-29T08:07:00.146 INFO:tasks.workunit.client.0.smithi037.stdout:5/32: mkdir d8 0 2024-03-29T08:07:00.146 INFO:tasks.workunit.client.0.smithi037.stdout:9/28: chown f0 67 1 2024-03-29T08:07:00.153 INFO:tasks.workunit.client.0.smithi037.stdout:5/33: creat d8/f9 x:0 0 0 2024-03-29T08:07:00.173 INFO:tasks.workunit.client.0.smithi037.stdout:8/24: fdatasync f4 0 2024-03-29T08:07:00.174 INFO:tasks.workunit.client.0.smithi037.stdout:5/34: chown d8 475805150 1 2024-03-29T08:07:00.179 INFO:tasks.workunit.client.0.smithi037.stdout:3/27: dwrite fa [0,4194304] 0 2024-03-29T08:07:00.179 INFO:tasks.workunit.client.0.smithi037.stdout:7/23: getdents . 0 2024-03-29T08:07:00.179 INFO:tasks.workunit.client.0.smithi037.stdout:5/35: chown f7 59 1 2024-03-29T08:07:00.180 INFO:tasks.workunit.client.0.smithi037.stdout:5/36: stat l6 0 2024-03-29T08:07:00.183 INFO:tasks.workunit.client.0.smithi037.stdout:7/24: stat l4 0 2024-03-29T08:07:00.183 INFO:tasks.workunit.client.0.smithi037.stdout:3/28: stat c9 0 2024-03-29T08:07:00.184 INFO:tasks.workunit.client.0.smithi037.stdout:8/25: read f0 [2435733,128924] 0 2024-03-29T08:07:00.186 INFO:tasks.workunit.client.0.smithi037.stdout:3/29: fdatasync f2 0 2024-03-29T08:07:00.192 INFO:tasks.workunit.client.0.smithi037.stdout:3/30: write f5 [1310155,87467] 0 2024-03-29T08:07:00.208 INFO:tasks.workunit.client.0.smithi037.stdout:9/29: dread f0 [4194304,4194304] 0 2024-03-29T08:07:00.213 INFO:tasks.workunit.client.0.smithi037.stdout:1/34: dwrite d2/d6/dc/fd [0,4194304] 0 2024-03-29T08:07:00.226 INFO:tasks.workunit.client.0.smithi037.stdout:7/25: dwrite f3 [0,4194304] 0 2024-03-29T08:07:00.236 INFO:tasks.workunit.client.0.smithi037.stdout:8/26: dwrite f0 [4194304,4194304] 0 2024-03-29T08:07:00.249 INFO:tasks.workunit.client.0.smithi037.stdout:9/30: dread f0 [4194304,4194304] 0 2024-03-29T08:07:00.251 INFO:tasks.workunit.client.0.smithi037.stdout:1/35: dwrite d2/f5 [4194304,4194304] 0 2024-03-29T08:07:00.324 INFO:tasks.workunit.client.0.smithi037.stdout:5/37: unlink f7 0 2024-03-29T08:07:00.327 INFO:tasks.workunit.client.0.smithi037.stdout:3/31: creat fb x:0 0 0 2024-03-29T08:07:00.329 INFO:tasks.workunit.client.0.smithi037.stdout:3/32: truncate fb 420183 0 2024-03-29T08:07:00.331 INFO:tasks.workunit.client.0.smithi037.stdout:8/27: mknod d5/c7 0 2024-03-29T08:07:00.331 INFO:tasks.workunit.client.0.smithi037.stdout:9/31: mknod d1/d6/c9 0 2024-03-29T08:07:00.333 INFO:tasks.workunit.client.0.smithi037.stdout:9/32: chown d1/c2 420796473 1 2024-03-29T08:07:00.333 INFO:tasks.workunit.client.0.smithi037.stdout:1/36: mkdir d2/df 0 2024-03-29T08:07:00.336 INFO:tasks.workunit.client.0.smithi037.stdout:2/23: truncate d0/f2 2736615 0 2024-03-29T08:07:00.344 INFO:tasks.workunit.client.0.smithi037.stdout:8/28: symlink d5/l8 0 2024-03-29T08:07:00.352 INFO:tasks.workunit.client.0.smithi037.stdout:9/33: creat d1/d3/fa x:0 0 0 2024-03-29T08:07:00.356 INFO:tasks.workunit.client.0.smithi037.stdout:9/34: chown d1/d3 5915551 1 2024-03-29T08:07:00.365 INFO:tasks.workunit.client.0.smithi037.stdout:3/33: dwrite fa [0,4194304] 0 2024-03-29T08:07:00.372 INFO:tasks.workunit.client.0.smithi037.stdout:1/37: dwrite d2/d6/dc/fd [0,4194304] 0 2024-03-29T08:07:00.374 INFO:tasks.workunit.client.0.smithi037.stdout:3/34: creat fc x:0 0 0 2024-03-29T08:07:00.376 INFO:tasks.workunit.client.0.smithi037.stdout:5/38: dwrite f3 [0,4194304] 0 2024-03-29T08:07:00.376 INFO:tasks.workunit.client.0.smithi037.stdout:3/35: chown f2 103706558 1 2024-03-29T08:07:00.378 INFO:tasks.workunit.client.0.smithi037.stdout:3/36: read - fc zero size 2024-03-29T08:07:00.378 INFO:tasks.workunit.client.0.smithi037.stdout:8/29: dread f0 [0,4194304] 0 2024-03-29T08:07:00.379 INFO:tasks.workunit.client.0.smithi037.stdout:5/39: write f3 [3697230,123132] 0 2024-03-29T08:07:00.381 INFO:tasks.workunit.client.0.smithi037.stdout:8/30: chown c3 122479 1 2024-03-29T08:07:00.384 INFO:tasks.workunit.client.0.smithi037.stdout:8/31: write f4 [368652,11881] 0 2024-03-29T08:07:00.392 INFO:tasks.workunit.client.0.smithi037.stdout:5/40: rename d8/f9 to d8/fa 0 2024-03-29T08:07:00.394 INFO:tasks.workunit.client.0.smithi037.stdout:5/41: read - d8/fa zero size 2024-03-29T08:07:00.396 INFO:tasks.workunit.client.0.smithi037.stdout:5/42: write f3 [444657,35673] 0 2024-03-29T08:07:00.398 INFO:tasks.workunit.client.0.smithi037.stdout:3/37: dread f5 [0,4194304] 0 2024-03-29T08:07:00.399 INFO:tasks.workunit.client.0.smithi037.stdout:5/43: symlink d8/lb 0 2024-03-29T08:07:00.405 INFO:tasks.workunit.client.0.smithi037.stdout:8/32: dread f0 [4194304,4194304] 0 2024-03-29T08:07:00.411 INFO:tasks.workunit.client.0.smithi037.stdout:5/44: symlink d8/lc 0 2024-03-29T08:07:00.411 INFO:tasks.workunit.client.0.smithi037.stdout:5/45: chown l6 2 1 2024-03-29T08:07:00.416 INFO:tasks.workunit.client.0.smithi037.stdout:3/38: dread f8 [0,4194304] 0 2024-03-29T08:07:00.419 INFO:tasks.workunit.client.0.smithi037.stdout:3/39: mkdir dd 0 2024-03-29T08:07:00.424 INFO:tasks.workunit.client.0.smithi037.stdout:8/33: dread f4 [0,4194304] 0 2024-03-29T08:07:00.425 INFO:tasks.workunit.client.0.smithi037.stdout:3/40: creat dd/fe x:0 0 0 2024-03-29T08:07:00.427 INFO:tasks.workunit.client.0.smithi037.stdout:8/34: symlink d5/l9 0 2024-03-29T08:07:00.429 INFO:tasks.workunit.client.0.smithi037.stdout:8/35: write f4 [1407018,50964] 0 2024-03-29T08:07:00.430 INFO:tasks.workunit.client.0.smithi037.stdout:8/36: stat d5/l8 0 2024-03-29T08:07:00.434 INFO:tasks.workunit.client.0.smithi037.stdout:8/37: creat d5/fa x:0 0 0 2024-03-29T08:07:00.438 INFO:tasks.workunit.client.0.smithi037.stdout:8/38: rename d5/fa to d5/fb 0 2024-03-29T08:07:00.439 INFO:tasks.workunit.client.0.smithi037.stdout:8/39: write f4 [5035713,24315] 0 2024-03-29T08:07:00.449 INFO:tasks.workunit.client.0.smithi037.stdout:7/26: truncate f3 510664 0 2024-03-29T08:07:00.450 INFO:tasks.workunit.client.0.smithi037.stdout:9/35: fsync d1/d3/fa 0 2024-03-29T08:07:00.450 INFO:tasks.workunit.client.0.smithi037.stdout:7/27: dread - f7 zero size 2024-03-29T08:07:00.452 INFO:tasks.workunit.client.0.smithi037.stdout:1/38: truncate d2/f5 2300644 0 2024-03-29T08:07:00.455 INFO:tasks.workunit.client.0.smithi037.stdout:9/36: symlink d1/lb 0 2024-03-29T08:07:00.461 INFO:tasks.workunit.client.0.smithi037.stdout:9/37: symlink d1/d4/lc 0 2024-03-29T08:07:00.462 INFO:tasks.workunit.client.0.smithi037.stdout:8/40: dwrite f1 [0,4194304] 0 2024-03-29T08:07:00.465 INFO:tasks.workunit.client.0.smithi037.stdout:9/38: mknod d1/d4/cd 0 2024-03-29T08:07:00.466 INFO:tasks.workunit.client.0.smithi037.stdout:3/41: getdents dd 0 2024-03-29T08:07:00.468 INFO:tasks.workunit.client.0.smithi037.stdout:2/24: write d0/f2 [2065403,45552] 0 2024-03-29T08:07:00.468 INFO:tasks.workunit.client.0.smithi037.stdout:3/42: dread - dd/fe zero size 2024-03-29T08:07:00.471 INFO:tasks.workunit.client.0.smithi037.stdout:8/41: link f4 d5/fc 0 2024-03-29T08:07:00.473 INFO:tasks.workunit.client.0.smithi037.stdout:7/28: unlink f3 0 2024-03-29T08:07:00.473 INFO:tasks.workunit.client.0.smithi037.stdout:8/42: truncate f1 4662934 0 2024-03-29T08:07:00.475 INFO:tasks.workunit.client.0.smithi037.stdout:1/39: read d2/f5 [2148467,50996] 0 2024-03-29T08:07:00.477 INFO:tasks.workunit.client.0.smithi037.stdout:7/29: read f0 [467376,129945] 0 2024-03-29T08:07:00.484 INFO:tasks.workunit.client.0.smithi037.stdout:8/43: rename c3 to d5/cd 0 2024-03-29T08:07:00.487 INFO:tasks.workunit.client.0.smithi037.stdout:2/25: link d0/f2 d0/f6 0 2024-03-29T08:07:00.494 INFO:tasks.workunit.client.0.smithi037.stdout:1/40: readlink d2/l3 0 2024-03-29T08:07:00.498 INFO:tasks.workunit.client.0.smithi037.stdout:9/39: truncate f0 966860 0 2024-03-29T08:07:00.498 INFO:tasks.workunit.client.0.smithi037.stdout:3/43: dwrite fc [0,4194304] 0 2024-03-29T08:07:00.500 INFO:tasks.workunit.client.0.smithi037.stdout:3/44: chown fc 476418231 1 2024-03-29T08:07:00.501 INFO:tasks.workunit.client.0.smithi037.stdout:2/26: creat d0/f7 x:0 0 0 2024-03-29T08:07:00.501 INFO:tasks.workunit.client.0.smithi037.stdout:9/40: write d1/d3/fa [400304,51396] 0 2024-03-29T08:07:00.503 INFO:tasks.workunit.client.0.smithi037.stdout:1/41: creat d2/d6/f10 x:0 0 0 2024-03-29T08:07:00.506 INFO:tasks.workunit.client.0.smithi037.stdout:3/45: rename fa to dd/ff 0 2024-03-29T08:07:00.510 INFO:tasks.workunit.client.0.smithi037.stdout:7/30: dwrite f5 [4194304,4194304] 0 2024-03-29T08:07:00.510 INFO:tasks.workunit.client.0.smithi037.stdout:3/46: dread - dd/fe zero size 2024-03-29T08:07:00.510 INFO:tasks.workunit.client.0.smithi037.stdout:7/31: rmdir - no directory 2024-03-29T08:07:00.514 INFO:tasks.workunit.client.0.smithi037.stdout:2/27: creat d0/f8 x:0 0 0 2024-03-29T08:07:00.515 INFO:tasks.workunit.client.0.smithi037.stdout:3/47: fsync fb 0 2024-03-29T08:07:00.517 INFO:tasks.workunit.client.0.smithi037.stdout:1/42: creat d2/d6/f11 x:0 0 0 2024-03-29T08:07:00.520 INFO:tasks.workunit.client.0.smithi037.stdout:1/43: dread - d2/d6/f10 zero size 2024-03-29T08:07:00.528 INFO:tasks.workunit.client.0.smithi037.stdout:3/48: rename l1 to dd/l10 0 2024-03-29T08:07:00.533 INFO:tasks.workunit.client.0.smithi037.stdout:3/49: write fc [579453,8375] 0 2024-03-29T08:07:00.536 INFO:tasks.workunit.client.0.smithi037.stdout:7/32: dread f5 [0,4194304] 0 2024-03-29T08:07:00.537 INFO:tasks.workunit.client.0.smithi037.stdout:8/44: dwrite f0 [0,4194304] 0 2024-03-29T08:07:00.541 INFO:tasks.workunit.client.0.smithi037.stdout:3/50: creat dd/f11 x:0 0 0 2024-03-29T08:07:00.542 INFO:tasks.workunit.client.0.smithi037.stdout:1/44: getdents d2/d6/dc 0 2024-03-29T08:07:00.547 INFO:tasks.workunit.client.0.smithi037.stdout:8/45: chown d5/cd 982790 1 2024-03-29T08:07:00.547 INFO:tasks.workunit.client.0.smithi037.stdout:3/51: creat dd/f12 x:0 0 0 2024-03-29T08:07:00.549 INFO:tasks.workunit.client.0.smithi037.stdout:1/45: chown d2/d6/l7 30314 1 2024-03-29T08:07:00.550 INFO:tasks.workunit.client.0.smithi037.stdout:8/46: fsync f4 0 2024-03-29T08:07:00.551 INFO:tasks.workunit.client.0.smithi037.stdout:2/28: dwrite d0/f6 [0,4194304] 0 2024-03-29T08:07:00.566 INFO:tasks.workunit.client.0.smithi037.stdout:3/52: link l3 dd/l13 0 2024-03-29T08:07:00.575 INFO:tasks.workunit.client.0.smithi037.stdout:3/53: mknod dd/c14 0 2024-03-29T08:07:00.575 INFO:tasks.workunit.client.0.smithi037.stdout:7/33: dwrite f5 [4194304,4194304] 0 2024-03-29T08:07:00.585 INFO:tasks.workunit.client.0.smithi037.stdout:2/29: dwrite d0/f6 [0,4194304] 0 2024-03-29T08:07:00.591 INFO:tasks.workunit.client.0.smithi037.stdout:1/46: dwrite d2/d6/f11 [0,4194304] 0 2024-03-29T08:07:00.593 INFO:tasks.workunit.client.0.smithi037.stdout:8/47: dwrite d5/fc [0,4194304] 0 2024-03-29T08:07:00.597 INFO:tasks.workunit.client.0.smithi037.stdout:1/47: unlink d2/d6/f10 0 2024-03-29T08:07:00.600 INFO:tasks.workunit.client.0.smithi037.stdout:8/48: mknod d5/ce 0 2024-03-29T08:07:00.600 INFO:tasks.workunit.client.0.smithi037.stdout:3/54: truncate dd/ff 3578431 0 2024-03-29T08:07:00.601 INFO:tasks.workunit.client.0.smithi037.stdout:1/48: fdatasync d2/f5 0 2024-03-29T08:07:00.601 INFO:tasks.workunit.client.0.smithi037.stdout:3/55: stat dd/fe 0 2024-03-29T08:07:00.601 INFO:tasks.workunit.client.0.smithi037.stdout:7/34: dread f0 [0,4194304] 0 2024-03-29T08:07:00.604 INFO:tasks.workunit.client.0.smithi037.stdout:7/35: write f7 [505200,20462] 0 2024-03-29T08:07:00.605 INFO:tasks.workunit.client.0.smithi037.stdout:3/56: mkdir dd/d15 0 2024-03-29T08:07:00.607 INFO:tasks.workunit.client.0.smithi037.stdout:1/49: unlink d2/d6/f11 0 2024-03-29T08:07:00.607 INFO:tasks.workunit.client.0.smithi037.stdout:7/36: symlink l8 0 2024-03-29T08:07:00.611 INFO:tasks.workunit.client.0.smithi037.stdout:7/37: rename f0 to f9 0 2024-03-29T08:07:00.611 INFO:tasks.workunit.client.0.smithi037.stdout:7/38: readlink l4 0 2024-03-29T08:07:00.613 INFO:tasks.workunit.client.0.smithi037.stdout:7/39: write f5 [740480,97949] 0 2024-03-29T08:07:00.613 INFO:tasks.workunit.client.0.smithi037.stdout:7/40: readlink l8 0 2024-03-29T08:07:00.635 INFO:tasks.workunit.client.0.smithi037.stdout:7/41: dread f7 [0,4194304] 0 2024-03-29T08:07:00.637 INFO:tasks.workunit.client.0.smithi037.stdout:7/42: rename c6 to ca 0 2024-03-29T08:07:00.637 INFO:tasks.workunit.client.0.smithi037.stdout:1/50: write d2/f5 [1028567,39636] 0 2024-03-29T08:07:00.639 INFO:tasks.workunit.client.0.smithi037.stdout:7/43: fdatasync f7 0 2024-03-29T08:07:00.641 INFO:tasks.workunit.client.0.smithi037.stdout:1/51: read d2/d6/dc/fd [1911092,11519] 0 2024-03-29T08:07:00.648 INFO:tasks.workunit.client.0.smithi037.stdout:7/44: rename ca to cb 0 2024-03-29T08:07:00.650 INFO:tasks.workunit.client.0.smithi037.stdout:7/45: symlink lc 0 2024-03-29T08:07:00.652 INFO:tasks.workunit.client.0.smithi037.stdout:7/46: mkdir dd 0 2024-03-29T08:07:00.657 INFO:tasks.workunit.client.0.smithi037.stdout:1/52: dwrite d2/f5 [0,4194304] 0 2024-03-29T08:07:00.660 INFO:tasks.workunit.client.0.smithi037.stdout:1/53: fdatasync d2/d6/dc/fd 0 2024-03-29T08:07:00.664 INFO:tasks.workunit.client.0.smithi037.stdout:1/54: write d2/d6/dc/fd [1643713,29933] 0 2024-03-29T08:07:00.668 INFO:tasks.workunit.client.0.smithi037.stdout:1/55: write d2/f5 [3396727,81087] 0 2024-03-29T08:07:00.671 INFO:tasks.workunit.client.0.smithi037.stdout:1/56: write d2/f5 [1757278,11903] 0 2024-03-29T08:07:00.671 INFO:tasks.workunit.client.0.smithi037.stdout:7/47: dwrite f7 [0,4194304] 0 2024-03-29T08:07:00.674 INFO:tasks.workunit.client.0.smithi037.stdout:1/57: rename d2/d6/dc to d2/d6/dc/d12 22 2024-03-29T08:07:00.679 INFO:tasks.workunit.client.0.smithi037.stdout:1/58: mkdir d2/d6/d13 0 2024-03-29T08:07:00.684 INFO:tasks.workunit.client.0.smithi037.stdout:7/48: dread f9 [0,4194304] 0 2024-03-29T08:07:00.687 INFO:tasks.workunit.client.0.smithi037.stdout:7/49: write f5 [4626969,65643] 0 2024-03-29T08:07:00.691 INFO:tasks.workunit.client.0.smithi037.stdout:7/50: symlink dd/le 0 2024-03-29T08:07:00.695 INFO:tasks.workunit.client.0.smithi037.stdout:1/59: dread d2/d6/dc/fd [0,4194304] 0 2024-03-29T08:07:00.695 INFO:tasks.workunit.client.0.smithi037.stdout:7/51: creat dd/ff x:0 0 0 2024-03-29T08:07:00.697 INFO:tasks.workunit.client.0.smithi037.stdout:1/60: creat d2/f14 x:0 0 0 2024-03-29T08:07:00.701 INFO:tasks.workunit.client.0.smithi037.stdout:1/61: creat d2/d6/dc/f15 x:0 0 0 2024-03-29T08:07:00.703 INFO:tasks.workunit.client.0.smithi037.stdout:1/62: write d2/f14 [78448,76973] 0 2024-03-29T08:07:00.706 INFO:tasks.workunit.client.0.smithi037.stdout:1/63: symlink d2/l16 0 2024-03-29T08:07:00.709 INFO:tasks.workunit.client.0.smithi037.stdout:1/64: symlink d2/l17 0 2024-03-29T08:07:00.710 INFO:tasks.workunit.client.0.smithi037.stdout:1/65: readlink d2/d6/l7 0 2024-03-29T08:07:00.711 INFO:tasks.workunit.client.0.smithi037.stdout:1/66: chown d2 91502698 1 2024-03-29T08:07:00.714 INFO:tasks.workunit.client.0.smithi037.stdout:1/67: mkdir d2/d6/dc/d18 0 2024-03-29T08:07:00.716 INFO:tasks.workunit.client.0.smithi037.stdout:7/52: dwrite f9 [0,4194304] 0 2024-03-29T08:07:00.717 INFO:tasks.workunit.client.0.smithi037.stdout:7/53: dread - dd/ff zero size 2024-03-29T08:07:00.808 INFO:tasks.workunit.client.0.smithi037.stdout:7/54: write dd/ff [17015,87615] 0 2024-03-29T08:07:00.811 INFO:tasks.workunit.client.0.smithi037.stdout:1/68: link d2/f14 d2/d6/dc/f19 0 2024-03-29T08:07:00.811 INFO:tasks.workunit.client.0.smithi037.stdout:7/55: read f7 [1187058,41452] 0 2024-03-29T08:07:00.816 INFO:tasks.workunit.client.0.smithi037.stdout:7/56: symlink dd/l10 0 2024-03-29T08:07:00.816 INFO:tasks.workunit.client.0.smithi037.stdout:1/69: rename d2/c8 to d2/c1a 0 2024-03-29T08:07:00.818 INFO:tasks.workunit.client.0.smithi037.stdout:7/57: write f5 [1376389,21310] 0 2024-03-29T08:07:00.834 INFO:tasks.workunit.client.0.smithi037.stdout:7/58: dwrite f5 [0,4194304] 0 2024-03-29T08:07:00.842 INFO:tasks.workunit.client.0.smithi037.stdout:7/59: truncate f5 4231980 0 2024-03-29T08:07:00.844 INFO:tasks.workunit.client.0.smithi037.stdout:7/60: creat dd/f11 x:0 0 0 2024-03-29T08:07:00.848 INFO:tasks.workunit.client.0.smithi037.stdout:7/61: link f9 dd/f12 0 2024-03-29T08:07:00.851 INFO:tasks.workunit.client.0.smithi037.stdout:7/62: symlink dd/l13 0 2024-03-29T08:07:00.852 INFO:tasks.workunit.client.0.smithi037.stdout:7/63: chown dd/f11 1177930 1 2024-03-29T08:07:00.854 INFO:tasks.workunit.client.0.smithi037.stdout:7/64: creat dd/f14 x:0 0 0 2024-03-29T08:07:00.855 INFO:tasks.workunit.client.0.smithi037.stdout:7/65: truncate f9 4891061 0 2024-03-29T08:07:00.867 INFO:tasks.workunit.client.0.smithi037.stdout:7/66: dread dd/f12 [0,4194304] 0 2024-03-29T08:07:00.869 INFO:tasks.workunit.client.0.smithi037.stdout:7/67: symlink dd/l15 0 2024-03-29T08:07:00.872 INFO:tasks.workunit.client.0.smithi037.stdout:7/68: rename dd/f11 to dd/f16 0 2024-03-29T08:07:00.874 INFO:tasks.workunit.client.0.smithi037.stdout:7/69: creat dd/f17 x:0 0 0 2024-03-29T08:07:00.875 INFO:tasks.workunit.client.0.smithi037.stdout:7/70: truncate dd/f17 497495 0 2024-03-29T08:07:00.877 INFO:tasks.workunit.client.0.smithi037.stdout:7/71: unlink dd/l15 0 2024-03-29T08:07:00.879 INFO:tasks.workunit.client.0.smithi037.stdout:7/72: write f9 [1928083,53485] 0 2024-03-29T08:07:00.881 INFO:tasks.workunit.client.0.smithi037.stdout:7/73: write dd/f12 [5323728,114558] 0 2024-03-29T08:07:00.885 INFO:tasks.workunit.client.0.smithi037.stdout:7/74: rename l4 to dd/l18 0 2024-03-29T08:07:00.886 INFO:tasks.workunit.client.0.smithi037.stdout:7/75: chown f9 215170162 1 2024-03-29T08:07:00.888 INFO:tasks.workunit.client.0.smithi037.stdout:7/76: symlink dd/l19 0 2024-03-29T08:07:00.897 INFO:tasks.workunit.client.0.smithi037.stdout:7/77: getdents dd 0 2024-03-29T08:07:00.912 INFO:tasks.workunit.client.0.smithi037.stdout:7/78: dwrite dd/f16 [0,4194304] 0 2024-03-29T08:07:00.913 INFO:tasks.workunit.client.0.smithi037.stdout:7/79: write dd/f17 [654194,123204] 0 2024-03-29T08:07:00.916 INFO:tasks.workunit.client.0.smithi037.stdout:7/80: chown dd/f12 3216654 1 2024-03-29T08:07:00.918 INFO:tasks.workunit.client.0.smithi037.stdout:7/81: write dd/f14 [625643,112722] 0 2024-03-29T08:07:00.921 INFO:tasks.workunit.client.0.smithi037.stdout:7/82: creat dd/f1a x:0 0 0 2024-03-29T08:07:00.939 INFO:tasks.workunit.client.0.smithi037.stdout:7/83: dwrite dd/f1a [0,4194304] 0 2024-03-29T08:07:00.942 INFO:tasks.workunit.client.0.smithi037.stdout:7/84: creat dd/f1b x:0 0 0 2024-03-29T08:07:00.943 INFO:tasks.workunit.client.0.smithi037.stdout:7/85: chown dd/f1b 353 1 2024-03-29T08:07:00.944 INFO:tasks.workunit.client.0.smithi037.stdout:7/86: read f7 [144562,88300] 0 2024-03-29T08:07:00.945 INFO:tasks.workunit.client.0.smithi037.stdout:7/87: chown dd/l18 343637 1 2024-03-29T08:07:00.960 INFO:tasks.workunit.client.0.smithi037.stdout:7/88: dwrite f5 [0,4194304] 0 2024-03-29T08:07:00.962 INFO:tasks.workunit.client.0.smithi037.stdout:7/89: write dd/f1b [560806,48250] 0 2024-03-29T08:07:00.983 INFO:tasks.workunit.client.0.smithi037.stdout:7/90: dwrite dd/f16 [4194304,4194304] 0 2024-03-29T08:07:00.984 INFO:tasks.workunit.client.0.smithi037.stdout:7/91: chown dd/f14 3021537 1 2024-03-29T08:07:00.987 INFO:tasks.workunit.client.0.smithi037.stdout:7/92: dread dd/ff [0,4194304] 0 2024-03-29T08:07:00.990 INFO:tasks.workunit.client.0.smithi037.stdout:7/93: creat dd/f1c x:0 0 0 2024-03-29T08:07:00.993 INFO:tasks.workunit.client.0.smithi037.stdout:7/94: unlink dd/f1c 0 2024-03-29T08:07:00.994 INFO:tasks.workunit.client.0.smithi037.stdout:7/95: fdatasync f7 0 2024-03-29T08:07:01.016 INFO:tasks.workunit.client.0.smithi037.stdout:7/96: dwrite dd/ff [0,4194304] 0 2024-03-29T08:07:01.022 INFO:tasks.workunit.client.0.smithi037.stdout:7/97: rmdir dd 39 2024-03-29T08:07:01.028 INFO:tasks.workunit.client.0.smithi037.stdout:7/98: getdents dd 0 2024-03-29T08:07:01.030 INFO:tasks.workunit.client.0.smithi037.stdout:7/99: write dd/ff [4074978,90101] 0 2024-03-29T08:07:01.033 INFO:tasks.workunit.client.0.smithi037.stdout:7/100: mkdir dd/d1d 0 2024-03-29T08:07:01.035 INFO:tasks.workunit.client.0.smithi037.stdout:7/101: fsync dd/f1b 0 2024-03-29T08:07:01.045 INFO:tasks.workunit.client.0.smithi037.stdout:7/102: dread dd/f14 [0,4194304] 0 2024-03-29T08:07:01.047 INFO:tasks.workunit.client.0.smithi037.stdout:7/103: rmdir dd 39 2024-03-29T08:07:01.061 INFO:tasks.workunit.client.0.smithi037.stdout:7/104: dwrite dd/f12 [0,4194304] 0 2024-03-29T08:07:01.065 INFO:tasks.workunit.client.0.smithi037.stdout:7/105: unlink dd/l19 0 2024-03-29T08:07:01.066 INFO:tasks.workunit.client.0.smithi037.stdout:7/106: read dd/ff [914027,60155] 0 2024-03-29T08:07:01.068 INFO:tasks.workunit.client.0.smithi037.stdout:7/107: mknod dd/c1e 0 2024-03-29T08:07:01.070 INFO:tasks.workunit.client.0.smithi037.stdout:7/108: creat dd/f1f x:0 0 0 2024-03-29T08:07:01.072 INFO:tasks.workunit.client.0.smithi037.stdout:7/109: mknod dd/c20 0 2024-03-29T08:07:01.073 INFO:tasks.workunit.client.0.smithi037.stdout:7/110: mkdir dd/d21 0 2024-03-29T08:07:01.091 INFO:tasks.workunit.client.0.smithi037.stdout:7/111: dwrite dd/f1f [0,4194304] 0 2024-03-29T08:07:01.093 INFO:tasks.workunit.client.0.smithi037.stdout:7/112: symlink dd/l22 0 2024-03-29T08:07:01.094 INFO:tasks.workunit.client.0.smithi037.stdout:7/113: chown dd/f14 618415487 1 2024-03-29T08:07:01.096 INFO:tasks.workunit.client.0.smithi037.stdout:7/114: mknod dd/c23 0 2024-03-29T08:07:01.111 INFO:tasks.workunit.client.0.smithi037.stdout:7/115: dwrite dd/ff [0,4194304] 0 2024-03-29T08:07:01.131 INFO:tasks.workunit.client.0.smithi037.stdout:7/116: dwrite f9 [4194304,4194304] 0 2024-03-29T08:07:01.142 INFO:tasks.workunit.client.0.smithi037.stdout:7/117: dread dd/f1a [0,4194304] 0 2024-03-29T08:07:01.146 INFO:tasks.workunit.client.0.smithi037.stdout:7/118: creat dd/d1d/f24 x:0 0 0 2024-03-29T08:07:01.153 INFO:tasks.workunit.client.0.smithi037.stdout:7/119: dread dd/f17 [0,4194304] 0 2024-03-29T08:07:01.154 INFO:tasks.workunit.client.0.smithi037.stdout:7/120: chown dd/ff 8874 1 2024-03-29T08:07:01.157 INFO:tasks.workunit.client.0.smithi037.stdout:7/121: unlink dd/d1d/f24 0 2024-03-29T08:07:01.159 INFO:tasks.workunit.client.0.smithi037.stdout:7/122: rename dd/c1e to dd/c25 0 2024-03-29T08:07:01.165 INFO:tasks.workunit.client.0.smithi037.stdout:7/123: rmdir dd 39 2024-03-29T08:07:01.166 INFO:tasks.workunit.client.0.smithi037.stdout:7/124: write f9 [6262266,103763] 0 2024-03-29T08:07:01.168 INFO:tasks.workunit.client.0.smithi037.stdout:7/125: readlink dd/l13 0 2024-03-29T08:07:01.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:07:01 smithi148 ceph-mon[44992]: pgmap v32: 65 pgs: 65 active+clean; 205 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 4.2 MiB/s wr, 356 op/s 2024-03-29T08:07:01.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:07:01 smithi037 ceph-mon[35162]: pgmap v32: 65 pgs: 65 active+clean; 205 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 4.2 MiB/s wr, 356 op/s 2024-03-29T08:07:02.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:07:02 smithi148 ceph-mon[44992]: from='mgr.14602 172.21.15.148:0/667342922' entity='mgr.smithi148.onhgvm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:07:02.426 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:07:02 smithi037 ceph-mon[35162]: from='mgr.14602 172.21.15.148:0/667342922' entity='mgr.smithi148.onhgvm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:07:03.295 INFO:tasks.workunit.client.1.smithi148.stderr:+ pushd ltp-full-20091231/testcases/kernel/fs/fsstress 2024-03-29T08:07:03.305 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:07:03 smithi148 ceph-mon[44992]: pgmap v33: 65 pgs: 65 active+clean; 205 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 3.1 MiB/s wr, 271 op/s 2024-03-29T08:07:03.314 INFO:tasks.workunit.client.1.smithi148.stdout:~/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress ~/cephtest/mnt.1/client.1/tmp/fsstress ~/cephtest/mnt.1/client.1/tmp 2024-03-29T08:07:03.314 INFO:tasks.workunit.client.1.smithi148.stderr:+ make 2024-03-29T08:07:03.410 INFO:tasks.workunit.client.1.smithi148.stdout:cc -DNO_XFS -I/home/ubuntu/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress -D_LARGEFILE64_SOURCE -D_GNU_SOURCE -I../../../../include -I../../../../include -L../../../../lib fsstress.c -o fsstress 2024-03-29T08:07:03.426 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:07:03 smithi037 ceph-mon[35162]: pgmap v33: 65 pgs: 65 active+clean; 205 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 3.1 MiB/s wr, 271 op/s 2024-03-29T08:07:03.965 INFO:tasks.workunit.client.1.smithi148.stderr:++ readlink -f fsstress 2024-03-29T08:07:03.968 INFO:tasks.workunit.client.1.smithi148.stderr:+ BIN=/home/ubuntu/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress 2024-03-29T08:07:03.968 INFO:tasks.workunit.client.1.smithi148.stderr:+ popd 2024-03-29T08:07:03.970 INFO:tasks.workunit.client.1.smithi148.stdout:~/cephtest/mnt.1/client.1/tmp/fsstress ~/cephtest/mnt.1/client.1/tmp 2024-03-29T08:07:03.970 INFO:tasks.workunit.client.1.smithi148.stderr:+ popd 2024-03-29T08:07:03.971 INFO:tasks.workunit.client.1.smithi148.stdout:~/cephtest/mnt.1/client.1/tmp 2024-03-29T08:07:03.972 INFO:tasks.workunit.client.1.smithi148.stderr:++ mktemp -d -p . 2024-03-29T08:07:03.977 INFO:tasks.workunit.client.1.smithi148.stderr:+ T=./tmp.OybOtqWMjH 2024-03-29T08:07:03.977 INFO:tasks.workunit.client.1.smithi148.stderr:+ /home/ubuntu/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress -d ./tmp.OybOtqWMjH -l 1 -n 1000 -p 10 -v 2024-03-29T08:07:03.983 INFO:tasks.workunit.client.1.smithi148.stdout:seed = 1711894831 2024-03-29T08:07:03.988 INFO:tasks.workunit.client.1.smithi148.stdout:0/0: rename - no filename 2024-03-29T08:07:03.990 INFO:tasks.workunit.client.1.smithi148.stdout:0/1: mknod c0 0 2024-03-29T08:07:03.990 INFO:tasks.workunit.client.1.smithi148.stdout:1/0: truncate - no filename 2024-03-29T08:07:03.990 INFO:tasks.workunit.client.1.smithi148.stdout:1/1: chown . 0 1 2024-03-29T08:07:03.991 INFO:tasks.workunit.client.1.smithi148.stdout:0/2: mknod c1 0 2024-03-29T08:07:03.991 INFO:tasks.workunit.client.1.smithi148.stdout:2/0: unlink - no file 2024-03-29T08:07:03.992 INFO:tasks.workunit.client.1.smithi148.stdout:2/1: chown . 235486 1 2024-03-29T08:07:03.992 INFO:tasks.workunit.client.1.smithi148.stdout:1/2: creat f0 x:0 0 0 2024-03-29T08:07:03.993 INFO:tasks.workunit.client.1.smithi148.stdout:1/3: chown f0 11425 1 2024-03-29T08:07:03.993 INFO:tasks.workunit.client.1.smithi148.stdout:3/0: read - no filename 2024-03-29T08:07:03.993 INFO:tasks.workunit.client.1.smithi148.stdout:3/1: dread - no filename 2024-03-29T08:07:03.994 INFO:tasks.workunit.client.1.smithi148.stdout:3/2: read - no filename 2024-03-29T08:07:03.994 INFO:tasks.workunit.client.1.smithi148.stdout:3/3: dwrite - no filename 2024-03-29T08:07:03.994 INFO:tasks.workunit.client.1.smithi148.stdout:3/4: chown . 2504 1 2024-03-29T08:07:03.994 INFO:tasks.workunit.client.1.smithi148.stdout:3/5: dwrite - no filename 2024-03-29T08:07:03.994 INFO:tasks.workunit.client.1.smithi148.stdout:3/6: read - no filename 2024-03-29T08:07:03.994 INFO:tasks.workunit.client.1.smithi148.stdout:3/7: stat - no entries 2024-03-29T08:07:03.995 INFO:tasks.workunit.client.1.smithi148.stdout:1/4: chown f0 23 1 2024-03-29T08:07:03.995 INFO:tasks.workunit.client.1.smithi148.stdout:0/3: creat f2 x:0 0 0 2024-03-29T08:07:03.995 INFO:tasks.workunit.client.1.smithi148.stdout:0/4: stat c0 0 2024-03-29T08:07:03.995 INFO:tasks.workunit.client.1.smithi148.stdout:1/5: truncate f0 835561 0 2024-03-29T08:07:03.996 INFO:tasks.workunit.client.1.smithi148.stdout:2/2: mknod c0 0 2024-03-29T08:07:03.996 INFO:tasks.workunit.client.1.smithi148.stdout:2/3: write - no filename 2024-03-29T08:07:03.997 INFO:tasks.workunit.client.1.smithi148.stdout:2/4: chown c0 298646 1 2024-03-29T08:07:04.000 INFO:tasks.workunit.client.1.smithi148.stdout:3/8: creat f0 x:0 0 0 2024-03-29T08:07:04.001 INFO:tasks.workunit.client.1.smithi148.stdout:2/5: creat f1 x:0 0 0 2024-03-29T08:07:04.002 INFO:tasks.workunit.client.1.smithi148.stdout:3/9: write f0 [568433,39253] 0 2024-03-29T08:07:04.003 INFO:tasks.workunit.client.1.smithi148.stdout:4/0: mknod c0 0 2024-03-29T08:07:04.003 INFO:tasks.workunit.client.1.smithi148.stdout:6/0: truncate - no filename 2024-03-29T08:07:04.003 INFO:tasks.workunit.client.1.smithi148.stdout:6/1: dwrite - no filename 2024-03-29T08:07:04.003 INFO:tasks.workunit.client.1.smithi148.stdout:6/2: write - no filename 2024-03-29T08:07:04.003 INFO:tasks.workunit.client.1.smithi148.stdout:6/3: write - no filename 2024-03-29T08:07:04.003 INFO:tasks.workunit.client.1.smithi148.stdout:6/4: stat - no entries 2024-03-29T08:07:04.004 INFO:tasks.workunit.client.1.smithi148.stdout:5/0: mkdir d0 0 2024-03-29T08:07:04.004 INFO:tasks.workunit.client.1.smithi148.stdout:5/1: dwrite - no filename 2024-03-29T08:07:04.004 INFO:tasks.workunit.client.1.smithi148.stdout:5/2: link - no file 2024-03-29T08:07:04.004 INFO:tasks.workunit.client.1.smithi148.stdout:4/1: chown c0 120117377 1 2024-03-29T08:07:04.009 INFO:tasks.workunit.client.1.smithi148.stdout:4/2: creat f1 x:0 0 0 2024-03-29T08:07:04.009 INFO:tasks.workunit.client.1.smithi148.stdout:3/10: dread f0 [0,4194304] 0 2024-03-29T08:07:04.011 INFO:tasks.workunit.client.1.smithi148.stdout:8/0: unlink - no file 2024-03-29T08:07:04.012 INFO:tasks.workunit.client.1.smithi148.stdout:6/5: creat f0 x:0 0 0 2024-03-29T08:07:04.013 INFO:tasks.workunit.client.1.smithi148.stdout:3/11: creat f1 x:0 0 0 2024-03-29T08:07:04.015 INFO:tasks.workunit.client.1.smithi148.stdout:9/0: write - no filename 2024-03-29T08:07:04.015 INFO:tasks.workunit.client.1.smithi148.stdout:7/0: symlink l0 0 2024-03-29T08:07:04.018 INFO:tasks.workunit.client.1.smithi148.stdout:8/1: creat f0 x:0 0 0 2024-03-29T08:07:04.020 INFO:tasks.workunit.client.1.smithi148.stdout:7/1: mknod c1 0 2024-03-29T08:07:04.020 INFO:tasks.workunit.client.1.smithi148.stdout:7/2: dread - no filename 2024-03-29T08:07:04.020 INFO:tasks.workunit.client.1.smithi148.stdout:7/3: write - no filename 2024-03-29T08:07:04.022 INFO:tasks.workunit.client.1.smithi148.stdout:9/1: mknod c0 0 2024-03-29T08:07:04.024 INFO:tasks.workunit.client.1.smithi148.stdout:8/2: rename f0 to f1 0 2024-03-29T08:07:04.028 INFO:tasks.workunit.client.1.smithi148.stdout:7/4: unlink l0 0 2024-03-29T08:07:04.031 INFO:tasks.workunit.client.1.smithi148.stdout:4/3: dwrite f1 [0,4194304] 0 2024-03-29T08:07:04.034 INFO:tasks.workunit.client.1.smithi148.stdout:7/5: creat f2 x:0 0 0 2024-03-29T08:07:04.036 INFO:tasks.workunit.client.1.smithi148.stdout:4/4: creat f2 x:0 0 0 2024-03-29T08:07:04.038 INFO:tasks.workunit.client.1.smithi148.stdout:7/6: mknod c3 0 2024-03-29T08:07:04.041 INFO:tasks.workunit.client.1.smithi148.stdout:4/5: unlink f2 0 2024-03-29T08:07:04.091 INFO:tasks.workunit.client.1.smithi148.stdout:0/5: fsync f2 0 2024-03-29T08:07:04.093 INFO:tasks.workunit.client.1.smithi148.stdout:0/6: mknod c3 0 2024-03-29T08:07:04.094 INFO:tasks.workunit.client.1.smithi148.stdout:2/6: getdents . 0 2024-03-29T08:07:04.099 INFO:tasks.workunit.client.1.smithi148.stdout:2/7: link c0 c2 0 2024-03-29T08:07:04.099 INFO:tasks.workunit.client.1.smithi148.stdout:6/6: getdents . 0 2024-03-29T08:07:04.100 INFO:tasks.workunit.client.1.smithi148.stdout:3/12: getdents . 0 2024-03-29T08:07:04.102 INFO:tasks.workunit.client.1.smithi148.stdout:9/2: getdents . 0 2024-03-29T08:07:04.102 INFO:tasks.workunit.client.1.smithi148.stdout:9/3: readlink - no filename 2024-03-29T08:07:04.103 INFO:tasks.workunit.client.1.smithi148.stdout:3/13: rename f0 to f2 0 2024-03-29T08:07:04.104 INFO:tasks.workunit.client.1.smithi148.stdout:8/3: rename f1 to f2 0 2024-03-29T08:07:04.106 INFO:tasks.workunit.client.1.smithi148.stdout:4/6: write f1 [5184120,60420] 0 2024-03-29T08:07:04.107 INFO:tasks.workunit.client.1.smithi148.stdout:7/7: fsync f2 0 2024-03-29T08:07:04.108 INFO:tasks.workunit.client.1.smithi148.stdout:9/4: link c0 c1 0 2024-03-29T08:07:04.112 INFO:tasks.workunit.client.1.smithi148.stdout:0/7: getdents . 0 2024-03-29T08:07:04.113 INFO:tasks.workunit.client.1.smithi148.stdout:3/14: rename f2 to f3 0 2024-03-29T08:07:04.114 INFO:tasks.workunit.client.1.smithi148.stdout:4/7: creat f3 x:0 0 0 2024-03-29T08:07:04.116 INFO:tasks.workunit.client.1.smithi148.stdout:9/5: mknod c2 0 2024-03-29T08:07:04.117 INFO:tasks.workunit.client.1.smithi148.stdout:7/8: creat f4 x:0 0 0 2024-03-29T08:07:04.118 INFO:tasks.workunit.client.1.smithi148.stdout:0/8: unlink c1 0 2024-03-29T08:07:04.119 INFO:tasks.workunit.client.1.smithi148.stdout:0/9: rmdir - no directory 2024-03-29T08:07:04.121 INFO:tasks.workunit.client.1.smithi148.stdout:3/15: rename f3 to f4 0 2024-03-29T08:07:04.121 INFO:tasks.workunit.client.1.smithi148.stdout:4/8: chown f3 3186245 1 2024-03-29T08:07:04.124 INFO:tasks.workunit.client.1.smithi148.stdout:7/9: truncate f2 94673 0 2024-03-29T08:07:04.128 INFO:tasks.workunit.client.1.smithi148.stdout:2/8: dwrite f1 [0,4194304] 0 2024-03-29T08:07:04.128 INFO:tasks.workunit.client.1.smithi148.stdout:3/16: rename f4 to f5 0 2024-03-29T08:07:04.128 INFO:tasks.workunit.client.1.smithi148.stdout:4/9: creat f4 x:0 0 0 2024-03-29T08:07:04.129 INFO:tasks.workunit.client.1.smithi148.stdout:0/10: write f2 [473697,3233] 0 2024-03-29T08:07:04.134 INFO:tasks.workunit.client.1.smithi148.stdout:7/10: write f4 [655420,14651] 0 2024-03-29T08:07:04.136 INFO:tasks.workunit.client.1.smithi148.stdout:9/6: getdents . 0 2024-03-29T08:07:04.136 INFO:tasks.workunit.client.1.smithi148.stdout:2/9: stat c2 0 2024-03-29T08:07:04.136 INFO:tasks.workunit.client.1.smithi148.stdout:3/17: symlink l6 0 2024-03-29T08:07:04.137 INFO:tasks.workunit.client.1.smithi148.stdout:0/11: mknod c4 0 2024-03-29T08:07:04.137 INFO:tasks.workunit.client.1.smithi148.stdout:4/10: chown f3 21219311 1 2024-03-29T08:07:04.138 INFO:tasks.workunit.client.1.smithi148.stdout:7/11: unlink c3 0 2024-03-29T08:07:04.139 INFO:tasks.workunit.client.1.smithi148.stdout:3/18: read - f1 zero size 2024-03-29T08:07:04.140 INFO:tasks.workunit.client.1.smithi148.stdout:0/12: symlink l5 0 2024-03-29T08:07:04.140 INFO:tasks.workunit.client.1.smithi148.stdout:2/10: rename c2 to c3 0 2024-03-29T08:07:04.141 INFO:tasks.workunit.client.1.smithi148.stdout:8/4: dwrite f2 [0,4194304] 0 2024-03-29T08:07:04.141 INFO:tasks.workunit.client.1.smithi148.stdout:4/11: creat f5 x:0 0 0 2024-03-29T08:07:04.142 INFO:tasks.workunit.client.1.smithi148.stdout:7/12: creat f5 x:0 0 0 2024-03-29T08:07:04.144 INFO:tasks.workunit.client.1.smithi148.stdout:0/13: creat f6 x:0 0 0 2024-03-29T08:07:04.144 INFO:tasks.workunit.client.1.smithi148.stdout:4/12: write f3 [433862,104798] 0 2024-03-29T08:07:04.147 INFO:tasks.workunit.client.1.smithi148.stdout:0/14: write f6 [479084,55894] 0 2024-03-29T08:07:04.148 INFO:tasks.workunit.client.1.smithi148.stdout:3/19: write f5 [365440,108057] 0 2024-03-29T08:07:04.151 INFO:tasks.workunit.client.1.smithi148.stdout:7/13: creat f6 x:0 0 0 2024-03-29T08:07:04.151 INFO:tasks.workunit.client.1.smithi148.stdout:7/14: readlink - no filename 2024-03-29T08:07:04.153 INFO:tasks.workunit.client.1.smithi148.stdout:3/20: mknod c7 0 2024-03-29T08:07:04.155 INFO:tasks.workunit.client.1.smithi148.stdout:2/11: dread f1 [0,4194304] 0 2024-03-29T08:07:04.161 INFO:tasks.workunit.client.1.smithi148.stdout:0/15: dread f2 [0,4194304] 0 2024-03-29T08:07:04.163 INFO:tasks.workunit.client.1.smithi148.stdout:3/21: creat f8 x:0 0 0 2024-03-29T08:07:04.163 INFO:tasks.workunit.client.1.smithi148.stdout:2/12: symlink l4 0 2024-03-29T08:07:04.165 INFO:tasks.workunit.client.1.smithi148.stdout:8/5: truncate f2 3850501 0 2024-03-29T08:07:04.166 INFO:tasks.workunit.client.1.smithi148.stdout:0/16: creat f7 x:0 0 0 2024-03-29T08:07:04.167 INFO:tasks.workunit.client.1.smithi148.stdout:4/13: truncate f1 3839652 0 2024-03-29T08:07:04.168 INFO:tasks.workunit.client.1.smithi148.stdout:3/22: symlink l9 0 2024-03-29T08:07:04.169 INFO:tasks.workunit.client.1.smithi148.stdout:2/13: fdatasync f1 0 2024-03-29T08:07:04.170 INFO:tasks.workunit.client.1.smithi148.stdout:0/17: chown f2 3055498 1 2024-03-29T08:07:04.170 INFO:tasks.workunit.client.1.smithi148.stdout:4/14: chown c0 66252006 1 2024-03-29T08:07:04.171 INFO:tasks.workunit.client.1.smithi148.stdout:3/23: write f8 [1006814,128414] 0 2024-03-29T08:07:04.173 INFO:tasks.workunit.client.1.smithi148.stdout:4/15: symlink l6 0 2024-03-29T08:07:04.173 INFO:tasks.workunit.client.1.smithi148.stdout:0/18: dread - f7 zero size 2024-03-29T08:07:04.173 INFO:tasks.workunit.client.1.smithi148.stdout:2/14: truncate f1 4753528 0 2024-03-29T08:07:04.175 INFO:tasks.workunit.client.1.smithi148.stdout:3/24: mknod ca 0 2024-03-29T08:07:04.176 INFO:tasks.workunit.client.1.smithi148.stdout:7/15: dwrite f4 [0,4194304] 0 2024-03-29T08:07:04.179 INFO:tasks.workunit.client.1.smithi148.stdout:2/15: chown f1 1165 1 2024-03-29T08:07:04.179 INFO:tasks.workunit.client.1.smithi148.stdout:0/19: rename c0 to c8 0 2024-03-29T08:07:04.180 INFO:tasks.workunit.client.1.smithi148.stdout:3/25: symlink lb 0 2024-03-29T08:07:04.181 INFO:tasks.workunit.client.1.smithi148.stdout:7/16: write f2 [840176,103341] 0 2024-03-29T08:07:04.182 INFO:tasks.workunit.client.1.smithi148.stdout:2/16: write f1 [4772818,68819] 0 2024-03-29T08:07:04.189 INFO:tasks.workunit.client.1.smithi148.stdout:0/20: creat f9 x:0 0 0 2024-03-29T08:07:04.192 INFO:tasks.workunit.client.1.smithi148.stdout:2/17: link f1 f5 0 2024-03-29T08:07:04.198 INFO:tasks.workunit.client.1.smithi148.stdout:8/6: dread f2 [0,4194304] 0 2024-03-29T08:07:04.199 INFO:tasks.workunit.client.1.smithi148.stdout:0/21: truncate f9 710471 0 2024-03-29T08:07:04.199 INFO:tasks.workunit.client.1.smithi148.stdout:0/22: rmdir - no directory 2024-03-29T08:07:04.202 INFO:tasks.workunit.client.1.smithi148.stdout:7/17: fdatasync f4 0 2024-03-29T08:07:04.206 INFO:tasks.workunit.client.1.smithi148.stdout:8/7: mknod c3 0 2024-03-29T08:07:04.211 INFO:tasks.workunit.client.1.smithi148.stdout:0/23: dread f9 [0,4194304] 0 2024-03-29T08:07:04.212 INFO:tasks.workunit.client.1.smithi148.stdout:7/18: read f2 [345865,79114] 0 2024-03-29T08:07:04.212 INFO:tasks.workunit.client.1.smithi148.stdout:4/16: dwrite f4 [0,4194304] 0 2024-03-29T08:07:04.214 INFO:tasks.workunit.client.1.smithi148.stdout:8/8: write f2 [2888360,45981] 0 2024-03-29T08:07:04.215 INFO:tasks.workunit.client.1.smithi148.stdout:0/24: creat fa x:0 0 0 2024-03-29T08:07:04.215 INFO:tasks.workunit.client.1.smithi148.stdout:3/26: dwrite f1 [0,4194304] 0 2024-03-29T08:07:04.217 INFO:tasks.workunit.client.1.smithi148.stdout:7/19: write f2 [1387221,105261] 0 2024-03-29T08:07:04.219 INFO:tasks.workunit.client.1.smithi148.stdout:4/17: creat f7 x:0 0 0 2024-03-29T08:07:04.221 INFO:tasks.workunit.client.1.smithi148.stdout:0/25: write f2 [4162,27934] 0 2024-03-29T08:07:04.223 INFO:tasks.workunit.client.1.smithi148.stdout:0/26: write f2 [1087747,62603] 0 2024-03-29T08:07:04.224 INFO:tasks.workunit.client.1.smithi148.stdout:2/18: dread f1 [0,4194304] 0 2024-03-29T08:07:04.225 INFO:tasks.workunit.client.1.smithi148.stdout:7/20: getdents . 0 2024-03-29T08:07:04.225 INFO:tasks.workunit.client.1.smithi148.stdout:7/21: rmdir - no directory 2024-03-29T08:07:04.226 INFO:tasks.workunit.client.1.smithi148.stdout:7/22: fdatasync f4 0 2024-03-29T08:07:04.226 INFO:tasks.workunit.client.1.smithi148.stdout:2/19: truncate f5 5225728 0 2024-03-29T08:07:04.227 INFO:tasks.workunit.client.1.smithi148.stdout:0/27: mkdir db 0 2024-03-29T08:07:04.230 INFO:tasks.workunit.client.1.smithi148.stdout:0/28: fdatasync f9 0 2024-03-29T08:07:04.233 INFO:tasks.workunit.client.1.smithi148.stdout:2/20: write f5 [4557190,128983] 0 2024-03-29T08:07:04.253 INFO:tasks.workunit.client.1.smithi148.stdout:3/27: dwrite f8 [0,4194304] 0 2024-03-29T08:07:04.253 INFO:tasks.workunit.client.1.smithi148.stdout:8/9: dwrite f2 [0,4194304] 0 2024-03-29T08:07:04.259 INFO:tasks.workunit.client.1.smithi148.stdout:3/28: chown f5 21191 1 2024-03-29T08:07:04.265 INFO:tasks.workunit.client.1.smithi148.stdout:8/10: write f2 [429564,113392] 0 2024-03-29T08:07:04.271 INFO:tasks.workunit.client.1.smithi148.stdout:7/23: dwrite f5 [0,4194304] 0 2024-03-29T08:07:04.272 INFO:tasks.workunit.client.1.smithi148.stdout:8/11: unlink f2 0 2024-03-29T08:07:04.274 INFO:tasks.workunit.client.1.smithi148.stdout:0/29: dwrite f2 [0,4194304] 0 2024-03-29T08:07:04.276 INFO:tasks.workunit.client.1.smithi148.stdout:7/24: creat f7 x:0 0 0 2024-03-29T08:07:04.277 INFO:tasks.workunit.client.1.smithi148.stdout:8/12: rename c3 to c4 0 2024-03-29T08:07:04.280 INFO:tasks.workunit.client.1.smithi148.stdout:7/25: write f7 [263345,79836] 0 2024-03-29T08:07:04.281 INFO:tasks.workunit.client.1.smithi148.stdout:0/30: dread f9 [0,4194304] 0 2024-03-29T08:07:04.288 INFO:tasks.workunit.client.1.smithi148.stdout:0/31: mkdir db/dc 0 2024-03-29T08:07:04.295 INFO:tasks.workunit.client.1.smithi148.stdout:0/32: link c8 db/dc/cd 0 2024-03-29T08:07:04.296 INFO:tasks.workunit.client.1.smithi148.stdout:8/13: link c4 c5 0 2024-03-29T08:07:04.296 INFO:tasks.workunit.client.1.smithi148.stdout:8/14: dwrite - no filename 2024-03-29T08:07:04.298 INFO:tasks.workunit.client.1.smithi148.stdout:8/15: symlink l6 0 2024-03-29T08:07:04.298 INFO:tasks.workunit.client.1.smithi148.stdout:8/16: truncate - no filename 2024-03-29T08:07:04.302 INFO:tasks.workunit.client.1.smithi148.stdout:7/26: dwrite f4 [0,4194304] 0 2024-03-29T08:07:04.305 INFO:tasks.workunit.client.1.smithi148.stdout:7/27: mkdir d8 0 2024-03-29T08:07:04.305 INFO:tasks.workunit.client.1.smithi148.stdout:7/28: readlink - no filename 2024-03-29T08:07:04.308 INFO:tasks.workunit.client.1.smithi148.stdout:8/17: getdents . 0 2024-03-29T08:07:04.309 INFO:tasks.workunit.client.1.smithi148.stdout:7/29: unlink f7 0 2024-03-29T08:07:04.313 INFO:tasks.workunit.client.1.smithi148.stdout:7/30: unlink f5 0 2024-03-29T08:07:04.314 INFO:tasks.workunit.client.1.smithi148.stdout:8/18: chown c4 227321910 1 2024-03-29T08:07:04.314 INFO:tasks.workunit.client.1.smithi148.stdout:8/19: dread - no filename 2024-03-29T08:07:04.314 INFO:tasks.workunit.client.1.smithi148.stdout:8/20: dwrite - no filename 2024-03-29T08:07:04.316 INFO:tasks.workunit.client.1.smithi148.stdout:8/21: creat f7 x:0 0 0 2024-03-29T08:07:04.317 INFO:tasks.workunit.client.1.smithi148.stdout:7/31: mkdir d8/d9 0 2024-03-29T08:07:04.318 INFO:tasks.workunit.client.1.smithi148.stdout:7/32: fdatasync f2 0 2024-03-29T08:07:04.319 INFO:tasks.workunit.client.1.smithi148.stdout:8/22: unlink l6 0 2024-03-29T08:07:04.319 INFO:tasks.workunit.client.1.smithi148.stdout:0/33: dwrite f6 [0,4194304] 0 2024-03-29T08:07:04.324 INFO:tasks.workunit.client.1.smithi148.stdout:8/23: rename c5 to c8 0 2024-03-29T08:07:04.328 INFO:tasks.workunit.client.1.smithi148.stdout:8/24: creat f9 x:0 0 0 2024-03-29T08:07:04.329 INFO:tasks.workunit.client.1.smithi148.stdout:8/25: fsync f7 0 2024-03-29T08:07:04.329 INFO:tasks.workunit.client.1.smithi148.stdout:0/34: dread f2 [0,4194304] 0 2024-03-29T08:07:04.330 INFO:tasks.workunit.client.1.smithi148.stdout:0/35: write fa [650485,120648] 0 2024-03-29T08:07:04.330 INFO:tasks.workunit.client.1.smithi148.stdout:7/33: rmdir d8/d9 0 2024-03-29T08:07:04.333 INFO:tasks.workunit.client.1.smithi148.stdout:0/36: mkdir db/de 0 2024-03-29T08:07:04.334 INFO:tasks.workunit.client.1.smithi148.stdout:7/34: creat d8/fa x:0 0 0 2024-03-29T08:07:04.335 INFO:tasks.workunit.client.1.smithi148.stdout:0/37: mkdir db/de/df 0 2024-03-29T08:07:04.336 INFO:tasks.workunit.client.1.smithi148.stdout:7/35: mknod d8/cb 0 2024-03-29T08:07:04.337 INFO:tasks.workunit.client.1.smithi148.stdout:7/36: dread - d8/fa zero size 2024-03-29T08:07:04.337 INFO:tasks.workunit.client.1.smithi148.stdout:7/37: readlink - no filename 2024-03-29T08:07:04.337 INFO:tasks.workunit.client.1.smithi148.stdout:8/26: fsync f9 0 2024-03-29T08:07:04.338 INFO:tasks.workunit.client.1.smithi148.stdout:0/38: write f2 [1129752,24569] 0 2024-03-29T08:07:04.338 INFO:tasks.workunit.client.1.smithi148.stdout:8/27: dread - f9 zero size 2024-03-29T08:07:04.340 INFO:tasks.workunit.client.1.smithi148.stdout:8/28: chown f9 7160 1 2024-03-29T08:07:04.341 INFO:tasks.workunit.client.1.smithi148.stdout:8/29: mknod ca 0 2024-03-29T08:07:04.342 INFO:tasks.workunit.client.1.smithi148.stdout:8/30: stat f7 0 2024-03-29T08:07:04.345 INFO:tasks.workunit.client.1.smithi148.stdout:8/31: rename c4 to cb 0 2024-03-29T08:07:04.349 INFO:tasks.workunit.client.1.smithi148.stdout:8/32: symlink lc 0 2024-03-29T08:07:04.353 INFO:tasks.workunit.client.1.smithi148.stdout:8/33: truncate f7 311632 0 2024-03-29T08:07:04.356 INFO:tasks.workunit.client.1.smithi148.stdout:8/34: mknod cd 0 2024-03-29T08:07:04.363 INFO:tasks.workunit.client.1.smithi148.stdout:8/35: dread f7 [0,4194304] 0 2024-03-29T08:07:04.364 INFO:tasks.workunit.client.1.smithi148.stdout:7/38: dwrite f6 [0,4194304] 0 2024-03-29T08:07:04.364 INFO:tasks.workunit.client.1.smithi148.stdout:0/39: dwrite f7 [0,4194304] 0 2024-03-29T08:07:04.365 INFO:tasks.workunit.client.1.smithi148.stdout:7/39: read f4 [3396312,41592] 0 2024-03-29T08:07:04.366 INFO:tasks.workunit.client.1.smithi148.stdout:8/36: write f7 [441411,55017] 0 2024-03-29T08:07:04.367 INFO:tasks.workunit.client.1.smithi148.stdout:7/40: fdatasync d8/fa 0 2024-03-29T08:07:04.367 INFO:tasks.workunit.client.1.smithi148.stdout:0/40: creat db/f10 x:0 0 0 2024-03-29T08:07:04.368 INFO:tasks.workunit.client.1.smithi148.stdout:8/37: fsync f9 0 2024-03-29T08:07:04.368 INFO:tasks.workunit.client.1.smithi148.stdout:0/41: readlink l5 0 2024-03-29T08:07:04.373 INFO:tasks.workunit.client.1.smithi148.stdout:8/38: creat fe x:0 0 0 2024-03-29T08:07:04.375 INFO:tasks.workunit.client.1.smithi148.stdout:7/41: rename d8/fa to d8/fc 0 2024-03-29T08:07:04.378 INFO:tasks.workunit.client.1.smithi148.stdout:0/42: getdents db/de 0 2024-03-29T08:07:04.382 INFO:tasks.workunit.client.1.smithi148.stdout:0/43: mkdir db/de/df/d11 0 2024-03-29T08:07:04.384 INFO:tasks.workunit.client.1.smithi148.stdout:0/44: creat db/dc/f12 x:0 0 0 2024-03-29T08:07:04.385 INFO:tasks.workunit.client.1.smithi148.stdout:7/42: dread f2 [0,4194304] 0 2024-03-29T08:07:04.386 INFO:tasks.workunit.client.1.smithi148.stdout:7/43: fdatasync f2 0 2024-03-29T08:07:04.388 INFO:tasks.workunit.client.1.smithi148.stdout:7/44: write d8/fc [595621,4888] 0 2024-03-29T08:07:04.390 INFO:tasks.workunit.client.1.smithi148.stdout:7/45: write f2 [2439573,58299] 0 2024-03-29T08:07:04.390 INFO:tasks.workunit.client.1.smithi148.stdout:7/46: readlink - no filename 2024-03-29T08:07:04.392 INFO:tasks.workunit.client.1.smithi148.stdout:8/39: getdents . 0 2024-03-29T08:07:04.393 INFO:tasks.workunit.client.1.smithi148.stdout:7/47: creat d8/fd x:0 0 0 2024-03-29T08:07:04.395 INFO:tasks.workunit.client.1.smithi148.stdout:0/45: getdents db/dc 0 2024-03-29T08:07:04.396 INFO:tasks.workunit.client.1.smithi148.stdout:7/48: rename d8/fd to d8/fe 0 2024-03-29T08:07:04.397 INFO:tasks.workunit.client.1.smithi148.stdout:7/49: fdatasync d8/fc 0 2024-03-29T08:07:04.400 INFO:tasks.workunit.client.1.smithi148.stdout:0/46: dread fa [0,4194304] 0 2024-03-29T08:07:04.401 INFO:tasks.workunit.client.1.smithi148.stdout:0/47: fdatasync db/f10 0 2024-03-29T08:07:04.403 INFO:tasks.workunit.client.1.smithi148.stdout:0/48: read fa [207663,119735] 0 2024-03-29T08:07:04.406 INFO:tasks.workunit.client.1.smithi148.stdout:0/49: fdatasync f2 0 2024-03-29T08:07:04.413 INFO:tasks.workunit.client.1.smithi148.stdout:8/40: dwrite f7 [0,4194304] 0 2024-03-29T08:07:04.414 INFO:tasks.workunit.client.1.smithi148.stdout:8/41: readlink lc 0 2024-03-29T08:07:04.422 INFO:tasks.workunit.client.1.smithi148.stdout:7/50: dwrite d8/fc [0,4194304] 0 2024-03-29T08:07:04.426 INFO:tasks.workunit.client.1.smithi148.stdout:7/51: symlink d8/lf 0 2024-03-29T08:07:04.430 INFO:tasks.workunit.client.1.smithi148.stdout:7/52: readlink d8/lf 0 2024-03-29T08:07:04.438 INFO:tasks.workunit.client.1.smithi148.stdout:0/50: dwrite db/f10 [0,4194304] 0 2024-03-29T08:07:04.444 INFO:tasks.workunit.client.1.smithi148.stdout:8/42: dwrite fe [0,4194304] 0 2024-03-29T08:07:04.447 INFO:tasks.workunit.client.1.smithi148.stdout:8/43: mknod cf 0 2024-03-29T08:07:04.459 INFO:tasks.workunit.client.1.smithi148.stdout:7/53: dwrite d8/fe [0,4194304] 0 2024-03-29T08:07:04.464 INFO:tasks.workunit.client.1.smithi148.stdout:7/54: truncate d8/fc 4956175 0 2024-03-29T08:07:04.466 INFO:tasks.workunit.client.1.smithi148.stdout:7/55: chown d8/fe 46433225 1 2024-03-29T08:07:04.469 INFO:tasks.workunit.client.1.smithi148.stdout:8/44: dwrite f7 [0,4194304] 0 2024-03-29T08:07:04.472 INFO:tasks.workunit.client.1.smithi148.stdout:7/56: write f4 [4266184,88599] 0 2024-03-29T08:07:04.481 INFO:tasks.workunit.client.1.smithi148.stdout:8/45: dread fe [0,4194304] 0 2024-03-29T08:07:04.489 INFO:tasks.workunit.client.1.smithi148.stdout:7/57: truncate f6 1230940 0 2024-03-29T08:07:04.490 INFO:tasks.workunit.client.1.smithi148.stdout:8/46: dread fe [0,4194304] 0 2024-03-29T08:07:04.491 INFO:tasks.workunit.client.1.smithi148.stdout:7/58: creat d8/f10 x:0 0 0 2024-03-29T08:07:04.492 INFO:tasks.workunit.client.1.smithi148.stdout:7/59: truncate d8/fe 5045804 0 2024-03-29T08:07:04.493 INFO:tasks.workunit.client.1.smithi148.stdout:8/47: creat f10 x:0 0 0 2024-03-29T08:07:04.493 INFO:tasks.workunit.client.1.smithi148.stdout:7/60: chown f4 12324 1 2024-03-29T08:07:04.494 INFO:tasks.workunit.client.1.smithi148.stdout:8/48: write fe [2559511,85920] 0 2024-03-29T08:07:04.495 INFO:tasks.workunit.client.1.smithi148.stdout:8/49: write fe [3702149,10545] 0 2024-03-29T08:07:04.499 INFO:tasks.workunit.client.1.smithi148.stdout:8/50: rename cb to c11 0 2024-03-29T08:07:04.500 INFO:tasks.workunit.client.1.smithi148.stdout:8/51: dread - f9 zero size 2024-03-29T08:07:04.503 INFO:tasks.workunit.client.1.smithi148.stdout:8/52: link cd c12 0 2024-03-29T08:07:04.513 INFO:tasks.workunit.client.1.smithi148.stdout:8/53: dwrite f7 [0,4194304] 0 2024-03-29T08:07:04.515 INFO:tasks.workunit.client.1.smithi148.stdout:8/54: creat f13 x:0 0 0 2024-03-29T08:07:04.523 INFO:tasks.workunit.client.1.smithi148.stdout:8/55: chown c11 0 1 2024-03-29T08:07:04.526 INFO:tasks.workunit.client.1.smithi148.stdout:8/56: rename f9 to f14 0 2024-03-29T08:07:04.529 INFO:tasks.workunit.client.1.smithi148.stdout:8/57: rename f13 to f15 0 2024-03-29T08:07:04.531 INFO:tasks.workunit.client.1.smithi148.stdout:8/58: write f15 [1038451,104569] 0 2024-03-29T08:07:04.532 INFO:tasks.workunit.client.1.smithi148.stdout:8/59: dread - f10 zero size 2024-03-29T08:07:04.534 INFO:tasks.workunit.client.1.smithi148.stdout:8/60: mknod c16 0 2024-03-29T08:07:04.534 INFO:tasks.workunit.client.1.smithi148.stdout:8/61: fdatasync f10 0 2024-03-29T08:07:04.541 INFO:tasks.workunit.client.1.smithi148.stdout:8/62: getdents . 0 2024-03-29T08:07:04.543 INFO:tasks.workunit.client.1.smithi148.stdout:8/63: mkdir d17 0 2024-03-29T08:07:04.543 INFO:tasks.workunit.client.1.smithi148.stdout:8/64: readlink lc 0 2024-03-29T08:07:04.546 INFO:tasks.workunit.client.1.smithi148.stdout:8/65: mkdir d17/d18 0 2024-03-29T08:07:05.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:07:05 smithi148 ceph-mon[44992]: pgmap v34: 65 pgs: 65 active+clean; 315 MiB data, 2.8 GiB used, 534 GiB / 536 GiB avail; 217 KiB/s rd, 21 MiB/s wr, 387 op/s 2024-03-29T08:07:05.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:07:05 smithi037 ceph-mon[35162]: pgmap v34: 65 pgs: 65 active+clean; 315 MiB data, 2.8 GiB used, 534 GiB / 536 GiB avail; 217 KiB/s rd, 21 MiB/s wr, 387 op/s 2024-03-29T08:07:05.438 INFO:tasks.workunit.client.0.smithi037.stdout:6/1: sync 2024-03-29T08:07:05.438 INFO:tasks.workunit.client.0.smithi037.stdout:5/46: sync 2024-03-29T08:07:05.438 INFO:tasks.workunit.client.0.smithi037.stdout:7/126: sync 2024-03-29T08:07:05.438 INFO:tasks.workunit.client.0.smithi037.stdout:4/1: sync 2024-03-29T08:07:05.439 INFO:tasks.workunit.client.0.smithi037.stdout:9/41: sync 2024-03-29T08:07:05.439 INFO:tasks.workunit.client.0.smithi037.stdout:8/49: sync 2024-03-29T08:07:05.439 INFO:tasks.workunit.client.0.smithi037.stdout:3/57: sync 2024-03-29T08:07:05.439 INFO:tasks.workunit.client.0.smithi037.stdout:1/70: sync 2024-03-29T08:07:05.439 INFO:tasks.workunit.client.0.smithi037.stdout:0/24: sync 2024-03-29T08:07:05.439 INFO:tasks.workunit.client.0.smithi037.stdout:2/30: sync 2024-03-29T08:07:05.443 INFO:tasks.workunit.client.0.smithi037.stdout:1/71: dread - d2/d6/dc/f15 zero size 2024-03-29T08:07:05.443 INFO:tasks.workunit.client.0.smithi037.stdout:8/50: read f0 [4205806,3575] 0 2024-03-29T08:07:05.443 INFO:tasks.workunit.client.0.smithi037.stdout:4/2: creat f0 x:0 0 0 2024-03-29T08:07:05.446 INFO:tasks.workunit.client.0.smithi037.stdout:0/25: link c2 c3 0 2024-03-29T08:07:05.446 INFO:tasks.workunit.client.0.smithi037.stdout:0/26: fdatasync - no filename 2024-03-29T08:07:05.448 INFO:tasks.workunit.client.0.smithi037.stdout:7/127: creat dd/d21/f26 x:0 0 0 2024-03-29T08:07:05.449 INFO:tasks.workunit.client.0.smithi037.stdout:6/2: mkdir d0/d1 0 2024-03-29T08:07:05.449 INFO:tasks.workunit.client.0.smithi037.stdout:6/3: dwrite - no filename 2024-03-29T08:07:05.449 INFO:tasks.workunit.client.0.smithi037.stdout:6/4: dread - no filename 2024-03-29T08:07:05.451 INFO:tasks.workunit.client.0.smithi037.stdout:2/31: getdents d0 0 2024-03-29T08:07:05.453 INFO:tasks.workunit.client.0.smithi037.stdout:5/47: dread f3 [0,4194304] 0 2024-03-29T08:07:05.456 INFO:tasks.workunit.client.0.smithi037.stdout:3/58: creat dd/d15/f16 x:0 0 0 2024-03-29T08:07:05.456 INFO:tasks.workunit.client.0.smithi037.stdout:5/48: truncate f3 4790982 0 2024-03-29T08:07:05.456 INFO:tasks.workunit.client.0.smithi037.stdout:4/3: mknod c1 0 2024-03-29T08:07:05.457 INFO:tasks.workunit.client.0.smithi037.stdout:5/49: readlink l6 0 2024-03-29T08:07:05.458 INFO:tasks.workunit.client.0.smithi037.stdout:4/4: fdatasync f0 0 2024-03-29T08:07:05.458 INFO:tasks.workunit.client.0.smithi037.stdout:7/128: dread dd/f17 [0,4194304] 0 2024-03-29T08:07:05.459 INFO:tasks.workunit.client.0.smithi037.stdout:2/32: mknod d0/c9 0 2024-03-29T08:07:05.459 INFO:tasks.workunit.client.0.smithi037.stdout:5/50: dread - d8/fa zero size 2024-03-29T08:07:05.459 INFO:tasks.workunit.client.0.smithi037.stdout:9/42: dread d1/d3/fa [0,4194304] 0 2024-03-29T08:07:05.459 INFO:tasks.workunit.client.0.smithi037.stdout:6/5: symlink d0/l2 0 2024-03-29T08:07:05.459 INFO:tasks.workunit.client.0.smithi037.stdout:6/6: dwrite - no filename 2024-03-29T08:07:05.460 INFO:tasks.workunit.client.0.smithi037.stdout:8/51: rename d5/c7 to d5/cf 0 2024-03-29T08:07:05.461 INFO:tasks.workunit.client.0.smithi037.stdout:5/51: chown d8/lc 22607 1 2024-03-29T08:07:05.462 INFO:tasks.workunit.client.0.smithi037.stdout:0/27: rename c2 to c4 0 2024-03-29T08:07:05.462 INFO:tasks.workunit.client.0.smithi037.stdout:0/28: read - no filename 2024-03-29T08:07:05.462 INFO:tasks.workunit.client.0.smithi037.stdout:0/29: truncate - no filename 2024-03-29T08:07:05.462 INFO:tasks.workunit.client.0.smithi037.stdout:0/30: dread - no filename 2024-03-29T08:07:05.462 INFO:tasks.workunit.client.0.smithi037.stdout:0/31: rmdir - no directory 2024-03-29T08:07:05.462 INFO:tasks.workunit.client.0.smithi037.stdout:4/5: truncate f0 693667 0 2024-03-29T08:07:05.463 INFO:tasks.workunit.client.0.smithi037.stdout:3/59: dread - f4 zero size 2024-03-29T08:07:05.464 INFO:tasks.workunit.client.0.smithi037.stdout:2/33: creat d0/fa x:0 0 0 2024-03-29T08:07:05.465 INFO:tasks.workunit.client.0.smithi037.stdout:7/129: symlink dd/d21/l27 0 2024-03-29T08:07:05.466 INFO:tasks.workunit.client.0.smithi037.stdout:2/34: chown d0/f8 850903 1 2024-03-29T08:07:05.467 INFO:tasks.workunit.client.0.smithi037.stdout:7/130: chown dd/l13 15 1 2024-03-29T08:07:05.467 INFO:tasks.workunit.client.0.smithi037.stdout:9/43: creat d1/d4/fe x:0 0 0 2024-03-29T08:07:05.467 INFO:tasks.workunit.client.0.smithi037.stdout:2/35: write d0/fa [560213,33506] 0 2024-03-29T08:07:05.469 INFO:tasks.workunit.client.0.smithi037.stdout:2/36: chown d0/c1 118 1 2024-03-29T08:07:05.472 INFO:tasks.workunit.client.0.smithi037.stdout:6/7: mknod d0/d1/c3 0 2024-03-29T08:07:05.473 INFO:tasks.workunit.client.0.smithi037.stdout:6/8: truncate - no filename 2024-03-29T08:07:05.473 INFO:tasks.workunit.client.0.smithi037.stdout:5/52: write d8/fa [350367,69822] 0 2024-03-29T08:07:05.474 INFO:tasks.workunit.client.0.smithi037.stdout:9/44: fdatasync d1/d3/fa 0 2024-03-29T08:07:05.475 INFO:tasks.workunit.client.0.smithi037.stdout:1/72: link d2/d6/cb d2/d6/dc/d18/c1b 0 2024-03-29T08:07:05.476 INFO:tasks.workunit.client.0.smithi037.stdout:2/37: creat d0/fb x:0 0 0 2024-03-29T08:07:05.478 INFO:tasks.workunit.client.0.smithi037.stdout:7/131: truncate f7 2500143 0 2024-03-29T08:07:05.480 INFO:tasks.workunit.client.0.smithi037.stdout:6/9: creat d0/f4 x:0 0 0 2024-03-29T08:07:05.480 INFO:tasks.workunit.client.0.smithi037.stdout:2/38: truncate d0/f8 266120 0 2024-03-29T08:07:05.481 INFO:tasks.workunit.client.0.smithi037.stdout:5/53: symlink d8/ld 0 2024-03-29T08:07:05.481 INFO:tasks.workunit.client.0.smithi037.stdout:9/45: mknod d1/d3/cf 0 2024-03-29T08:07:05.482 INFO:tasks.workunit.client.0.smithi037.stdout:4/6: link c1 c2 0 2024-03-29T08:07:05.484 INFO:tasks.workunit.client.0.smithi037.stdout:6/10: dread - d0/f4 zero size 2024-03-29T08:07:05.484 INFO:tasks.workunit.client.0.smithi037.stdout:1/73: dread d2/d6/dc/f19 [0,4194304] 0 2024-03-29T08:07:05.485 INFO:tasks.workunit.client.0.smithi037.stdout:5/54: symlink d8/le 0 2024-03-29T08:07:05.486 INFO:tasks.workunit.client.0.smithi037.stdout:8/52: dwrite d5/fc [0,4194304] 0 2024-03-29T08:07:05.486 INFO:tasks.workunit.client.0.smithi037.stdout:2/39: write d0/f6 [3415915,80860] 0 2024-03-29T08:07:05.486 INFO:tasks.workunit.client.0.smithi037.stdout:4/7: mkdir d3 0 2024-03-29T08:07:05.488 INFO:tasks.workunit.client.0.smithi037.stdout:5/55: fsync d8/fa 0 2024-03-29T08:07:05.489 INFO:tasks.workunit.client.0.smithi037.stdout:9/46: chown d1/l5 94964561 1 2024-03-29T08:07:05.493 INFO:tasks.workunit.client.0.smithi037.stdout:6/11: mkdir d0/d5 0 2024-03-29T08:07:05.494 INFO:tasks.workunit.client.0.smithi037.stdout:4/8: write f0 [789639,86932] 0 2024-03-29T08:07:05.494 INFO:tasks.workunit.client.0.smithi037.stdout:5/56: chown d8/fa 11812 1 2024-03-29T08:07:05.497 INFO:tasks.workunit.client.0.smithi037.stdout:1/74: chown d2/d6/dc/f19 46 1 2024-03-29T08:07:05.497 INFO:tasks.workunit.client.0.smithi037.stdout:9/47: mkdir d1/d6/d10 0 2024-03-29T08:07:05.497 INFO:tasks.workunit.client.0.smithi037.stdout:4/9: chown f0 127 1 2024-03-29T08:07:05.500 INFO:tasks.workunit.client.0.smithi037.stdout:4/10: write f0 [586221,45913] 0 2024-03-29T08:07:05.501 INFO:tasks.workunit.client.0.smithi037.stdout:9/48: mknod d1/d4/c11 0 2024-03-29T08:07:05.509 INFO:tasks.workunit.client.0.smithi037.stdout:1/75: getdents d2/d6 0 2024-03-29T08:07:05.510 INFO:tasks.workunit.client.0.smithi037.stdout:4/11: mkdir d3/d4 0 2024-03-29T08:07:05.515 INFO:tasks.workunit.client.0.smithi037.stdout:9/49: creat d1/d6/d10/f12 x:0 0 0 2024-03-29T08:07:05.515 INFO:tasks.workunit.client.0.smithi037.stdout:4/12: mknod d3/c5 0 2024-03-29T08:07:05.515 INFO:tasks.workunit.client.0.smithi037.stdout:8/53: dread f0 [0,4194304] 0 2024-03-29T08:07:05.522 INFO:tasks.workunit.client.0.smithi037.stdout:2/40: dwrite d0/f8 [0,4194304] 0 2024-03-29T08:07:05.525 INFO:tasks.workunit.client.0.smithi037.stdout:1/76: truncate d2/d6/dc/f15 63326 0 2024-03-29T08:07:05.527 INFO:tasks.workunit.client.0.smithi037.stdout:9/50: mkdir d1/d6/d10/d13 0 2024-03-29T08:07:05.529 INFO:tasks.workunit.client.0.smithi037.stdout:2/41: rename d0/f6 to d0/fc 0 2024-03-29T08:07:05.529 INFO:tasks.workunit.client.0.smithi037.stdout:4/13: link d3/c5 d3/c6 0 2024-03-29T08:07:05.530 INFO:tasks.workunit.client.0.smithi037.stdout:5/57: dwrite d8/fa [0,4194304] 0 2024-03-29T08:07:05.534 INFO:tasks.workunit.client.0.smithi037.stdout:1/77: creat d2/df/f1c x:0 0 0 2024-03-29T08:07:05.535 INFO:tasks.workunit.client.0.smithi037.stdout:2/42: dread - d0/f7 zero size 2024-03-29T08:07:05.536 INFO:tasks.workunit.client.0.smithi037.stdout:9/51: rename d1/d3/cf to d1/d6/c14 0 2024-03-29T08:07:05.536 INFO:tasks.workunit.client.0.smithi037.stdout:1/78: readlink d2/l17 0 2024-03-29T08:07:05.544 INFO:tasks.workunit.client.0.smithi037.stdout:9/52: write d1/d4/fe [540625,64718] 0 2024-03-29T08:07:05.549 INFO:tasks.workunit.client.0.smithi037.stdout:1/79: truncate d2/d6/dc/f15 900575 0 2024-03-29T08:07:05.550 INFO:tasks.workunit.client.0.smithi037.stdout:2/43: link d0/f8 d0/fd 0 2024-03-29T08:07:05.552 INFO:tasks.workunit.client.0.smithi037.stdout:8/54: dwrite f0 [0,4194304] 0 2024-03-29T08:07:05.552 INFO:tasks.workunit.client.0.smithi037.stdout:2/44: chown d0/fb 0 1 2024-03-29T08:07:05.555 INFO:tasks.workunit.client.0.smithi037.stdout:9/53: mknod d1/d3/c15 0 2024-03-29T08:07:05.556 INFO:tasks.workunit.client.0.smithi037.stdout:8/55: write f4 [416442,98938] 0 2024-03-29T08:07:05.557 INFO:tasks.workunit.client.0.smithi037.stdout:9/54: chown d1/d4/c11 1 1 2024-03-29T08:07:05.559 INFO:tasks.workunit.client.0.smithi037.stdout:8/56: read d5/fc [127791,69332] 0 2024-03-29T08:07:05.562 INFO:tasks.workunit.client.0.smithi037.stdout:9/55: mkdir d1/d6/d16 0 2024-03-29T08:07:05.563 INFO:tasks.workunit.client.0.smithi037.stdout:1/80: mkdir d2/d6/dc/d18/d1d 0 2024-03-29T08:07:05.566 INFO:tasks.workunit.client.0.smithi037.stdout:8/57: mkdir d5/d10 0 2024-03-29T08:07:05.570 INFO:tasks.workunit.client.0.smithi037.stdout:8/58: unlink d5/cd 0 2024-03-29T08:07:05.577 INFO:tasks.workunit.client.0.smithi037.stdout:1/81: getdents d2 0 2024-03-29T08:07:05.584 INFO:tasks.workunit.client.0.smithi037.stdout:1/82: write d2/d6/dc/fd [1314951,37280] 0 2024-03-29T08:07:05.586 INFO:tasks.workunit.client.0.smithi037.stdout:9/56: dwrite d1/d3/fa [0,4194304] 0 2024-03-29T08:07:05.590 INFO:tasks.workunit.client.0.smithi037.stdout:7/132: rmdir dd/d21 39 2024-03-29T08:07:05.590 INFO:tasks.workunit.client.0.smithi037.stdout:8/59: dread f0 [4194304,4194304] 0 2024-03-29T08:07:05.590 INFO:tasks.workunit.client.0.smithi037.stdout:0/32: chown c4 0 1 2024-03-29T08:07:05.593 INFO:tasks.workunit.client.0.smithi037.stdout:1/83: dread d2/d6/dc/f15 [0,4194304] 0 2024-03-29T08:07:05.594 INFO:tasks.workunit.client.0.smithi037.stdout:0/33: chown c3 59309 1 2024-03-29T08:07:05.594 INFO:tasks.workunit.client.0.smithi037.stdout:0/34: dwrite - no filename 2024-03-29T08:07:05.594 INFO:tasks.workunit.client.0.smithi037.stdout:7/133: write dd/f1b [185909,126277] 0 2024-03-29T08:07:05.594 INFO:tasks.workunit.client.0.smithi037.stdout:3/60: write f8 [229623,80140] 0 2024-03-29T08:07:05.598 INFO:tasks.workunit.client.0.smithi037.stdout:7/134: fsync dd/ff 0 2024-03-29T08:07:05.598 INFO:tasks.workunit.client.0.smithi037.stdout:9/57: creat d1/d6/d10/d13/f17 x:0 0 0 2024-03-29T08:07:05.600 INFO:tasks.workunit.client.0.smithi037.stdout:6/12: fsync d0/f4 0 2024-03-29T08:07:05.601 INFO:tasks.workunit.client.0.smithi037.stdout:8/60: rename f0 to d5/d10/f11 0 2024-03-29T08:07:05.601 INFO:tasks.workunit.client.0.smithi037.stdout:3/61: chown l7 66 1 2024-03-29T08:07:05.601 INFO:tasks.workunit.client.0.smithi037.stdout:8/61: stat f4 0 2024-03-29T08:07:05.602 INFO:tasks.workunit.client.0.smithi037.stdout:9/58: chown d1/d6/d16 428 1 2024-03-29T08:07:05.602 INFO:tasks.workunit.client.0.smithi037.stdout:0/35: creat f5 x:0 0 0 2024-03-29T08:07:05.607 INFO:tasks.workunit.client.0.smithi037.stdout:1/84: write d2/d6/dc/f15 [1698133,71717] 0 2024-03-29T08:07:05.609 INFO:tasks.workunit.client.0.smithi037.stdout:1/85: dread - d2/df/f1c zero size 2024-03-29T08:07:05.609 INFO:tasks.workunit.client.0.smithi037.stdout:3/62: write f5 [2585626,116113] 0 2024-03-29T08:07:05.610 INFO:tasks.workunit.client.0.smithi037.stdout:0/36: creat f6 x:0 0 0 2024-03-29T08:07:05.615 INFO:tasks.workunit.client.0.smithi037.stdout:7/135: read f7 [1267992,96811] 0 2024-03-29T08:07:05.616 INFO:tasks.workunit.client.0.smithi037.stdout:6/13: creat d0/d5/f6 x:0 0 0 2024-03-29T08:07:05.621 INFO:tasks.workunit.client.0.smithi037.stdout:7/136: rmdir dd 39 2024-03-29T08:07:05.625 INFO:tasks.workunit.client.0.smithi037.stdout:4/14: getdents d3 0 2024-03-29T08:07:05.625 INFO:tasks.workunit.client.0.smithi037.stdout:6/14: creat d0/d5/f7 x:0 0 0 2024-03-29T08:07:05.627 INFO:tasks.workunit.client.0.smithi037.stdout:6/15: rename d0 to d0/d1/d8 22 2024-03-29T08:07:05.629 INFO:tasks.workunit.client.0.smithi037.stdout:4/15: read f0 [607332,78189] 0 2024-03-29T08:07:05.632 INFO:tasks.workunit.client.0.smithi037.stdout:3/63: link dd/l10 dd/l17 0 2024-03-29T08:07:05.632 INFO:tasks.workunit.client.0.smithi037.stdout:5/58: truncate d8/fa 2836678 0 2024-03-29T08:07:05.635 INFO:tasks.workunit.client.0.smithi037.stdout:6/16: dread - d0/d5/f6 zero size 2024-03-29T08:07:05.635 INFO:tasks.workunit.client.0.smithi037.stdout:2/45: rmdir d0 39 2024-03-29T08:07:05.636 INFO:tasks.workunit.client.0.smithi037.stdout:7/137: dread dd/f14 [0,4194304] 0 2024-03-29T08:07:05.638 INFO:tasks.workunit.client.0.smithi037.stdout:6/17: read - d0/d5/f7 zero size 2024-03-29T08:07:05.638 INFO:tasks.workunit.client.0.smithi037.stdout:3/64: chown l3 30636104 1 2024-03-29T08:07:05.638 INFO:tasks.workunit.client.0.smithi037.stdout:8/62: dwrite d5/fb [0,4194304] 0 2024-03-29T08:07:05.641 INFO:tasks.workunit.client.0.smithi037.stdout:5/59: creat d8/ff x:0 0 0 2024-03-29T08:07:05.642 INFO:tasks.workunit.client.0.smithi037.stdout:7/138: mknod dd/d1d/c28 0 2024-03-29T08:07:05.643 INFO:tasks.workunit.client.0.smithi037.stdout:7/139: fdatasync dd/ff 0 2024-03-29T08:07:05.643 INFO:tasks.workunit.client.0.smithi037.stdout:4/16: getdents d3/d4 0 2024-03-29T08:07:05.644 INFO:tasks.workunit.client.0.smithi037.stdout:7/140: chown f7 334504 1 2024-03-29T08:07:05.646 INFO:tasks.workunit.client.0.smithi037.stdout:8/63: fdatasync f4 0 2024-03-29T08:07:05.646 INFO:tasks.workunit.client.0.smithi037.stdout:6/18: rmdir d0/d1 39 2024-03-29T08:07:05.650 INFO:tasks.workunit.client.0.smithi037.stdout:2/46: creat d0/fe x:0 0 0 2024-03-29T08:07:05.651 INFO:tasks.workunit.client.0.smithi037.stdout:4/17: mknod d3/c7 0 2024-03-29T08:07:05.657 INFO:tasks.workunit.client.0.smithi037.stdout:8/64: stat d5/l6 0 2024-03-29T08:07:05.664 INFO:tasks.workunit.client.0.smithi037.stdout:7/141: mknod dd/c29 0 2024-03-29T08:07:05.667 INFO:tasks.workunit.client.0.smithi037.stdout:2/47: rename d0/c1 to d0/cf 0 2024-03-29T08:07:05.668 INFO:tasks.workunit.client.0.smithi037.stdout:9/59: dwrite d1/d3/fa [4194304,4194304] 0 2024-03-29T08:07:05.672 INFO:tasks.workunit.client.0.smithi037.stdout:6/19: rmdir d0/d1 39 2024-03-29T08:07:05.674 INFO:tasks.workunit.client.0.smithi037.stdout:7/142: write dd/f12 [7421883,8313] 0 2024-03-29T08:07:05.675 INFO:tasks.workunit.client.0.smithi037.stdout:3/65: dread dd/ff [0,4194304] 0 2024-03-29T08:07:05.677 INFO:tasks.workunit.client.0.smithi037.stdout:5/60: dwrite d8/ff [0,4194304] 0 2024-03-29T08:07:05.680 INFO:tasks.workunit.client.0.smithi037.stdout:9/60: mkdir d1/d18 0 2024-03-29T08:07:05.680 INFO:tasks.workunit.client.0.smithi037.stdout:8/65: write d5/d10/f11 [4137561,113293] 0 2024-03-29T08:07:05.687 INFO:tasks.workunit.client.0.smithi037.stdout:3/66: write fb [968015,96608] 0 2024-03-29T08:07:05.691 INFO:tasks.workunit.client.0.smithi037.stdout:7/143: chown dd/c25 13050 1 2024-03-29T08:07:05.691 INFO:tasks.workunit.client.0.smithi037.stdout:6/20: creat d0/d5/f9 x:0 0 0 2024-03-29T08:07:05.692 INFO:tasks.workunit.client.0.smithi037.stdout:9/61: rmdir d1/d6 39 2024-03-29T08:07:05.692 INFO:tasks.workunit.client.0.smithi037.stdout:1/86: dwrite d2/d6/dc/f19 [0,4194304] 0 2024-03-29T08:07:05.694 INFO:tasks.workunit.client.0.smithi037.stdout:6/21: dread - d0/f4 zero size 2024-03-29T08:07:05.697 INFO:tasks.workunit.client.0.smithi037.stdout:6/22: dread - d0/d5/f9 zero size 2024-03-29T08:07:05.697 INFO:tasks.workunit.client.0.smithi037.stdout:2/48: link d0/fd d0/f10 0 2024-03-29T08:07:05.698 INFO:tasks.workunit.client.0.smithi037.stdout:8/66: mknod d5/c12 0 2024-03-29T08:07:05.699 INFO:tasks.workunit.client.0.smithi037.stdout:3/67: mkdir dd/d15/d18 0 2024-03-29T08:07:05.702 INFO:tasks.workunit.client.0.smithi037.stdout:6/23: write d0/d5/f7 [852586,57974] 0 2024-03-29T08:07:05.702 INFO:tasks.workunit.client.0.smithi037.stdout:9/62: stat d1/d6/d10/f12 0 2024-03-29T08:07:05.703 INFO:tasks.workunit.client.0.smithi037.stdout:7/144: dread - dd/d21/f26 zero size 2024-03-29T08:07:05.703 INFO:tasks.workunit.client.0.smithi037.stdout:8/67: creat d5/f13 x:0 0 0 2024-03-29T08:07:05.704 INFO:tasks.workunit.client.0.smithi037.stdout:1/87: creat d2/d6/dc/f1e x:0 0 0 2024-03-29T08:07:05.704 INFO:tasks.workunit.client.0.smithi037.stdout:3/68: truncate f2 2288836 0 2024-03-29T08:07:05.705 INFO:tasks.workunit.client.0.smithi037.stdout:8/68: chown d5 7021 1 2024-03-29T08:07:05.707 INFO:tasks.workunit.client.0.smithi037.stdout:2/49: rmdir d0 39 2024-03-29T08:07:05.708 INFO:tasks.workunit.client.0.smithi037.stdout:9/63: mkdir d1/d4/d19 0 2024-03-29T08:07:05.711 INFO:tasks.workunit.client.0.smithi037.stdout:1/88: symlink d2/d6/l1f 0 2024-03-29T08:07:05.712 INFO:tasks.workunit.client.0.smithi037.stdout:5/61: dread d8/fa [0,4194304] 0 2024-03-29T08:07:05.712 INFO:tasks.workunit.client.0.smithi037.stdout:9/64: write d1/d4/fe [1240922,86237] 0 2024-03-29T08:07:05.713 INFO:tasks.workunit.client.0.smithi037.stdout:2/50: chown d0/fc 382203247 1 2024-03-29T08:07:05.715 INFO:tasks.workunit.client.0.smithi037.stdout:3/69: rename f8 to dd/d15/f19 0 2024-03-29T08:07:05.716 INFO:tasks.workunit.client.0.smithi037.stdout:1/89: write d2/f14 [3974429,53819] 0 2024-03-29T08:07:05.716 INFO:tasks.workunit.client.0.smithi037.stdout:6/24: rename d0/d5/f7 to d0/d5/fa 0 2024-03-29T08:07:05.722 INFO:tasks.workunit.client.0.smithi037.stdout:3/70: read dd/ff [1350216,83746] 0 2024-03-29T08:07:05.724 INFO:tasks.workunit.client.0.smithi037.stdout:5/62: mknod d8/c10 0 2024-03-29T08:07:05.727 INFO:tasks.workunit.client.0.smithi037.stdout:2/51: mknod d0/c11 0 2024-03-29T08:07:05.729 INFO:tasks.workunit.client.0.smithi037.stdout:2/52: chown d0/fb 9299 1 2024-03-29T08:07:05.730 INFO:tasks.workunit.client.0.smithi037.stdout:7/145: link f5 dd/d1d/f2a 0 2024-03-29T08:07:05.731 INFO:tasks.workunit.client.0.smithi037.stdout:8/69: dwrite f1 [0,4194304] 0 2024-03-29T08:07:05.733 INFO:tasks.workunit.client.0.smithi037.stdout:1/90: creat d2/df/f20 x:0 0 0 2024-03-29T08:07:05.733 INFO:tasks.workunit.client.0.smithi037.stdout:6/25: creat d0/d5/fb x:0 0 0 2024-03-29T08:07:05.738 INFO:tasks.workunit.client.0.smithi037.stdout:5/63: mknod d8/c11 0 2024-03-29T08:07:05.743 INFO:tasks.workunit.client.0.smithi037.stdout:7/146: creat dd/d21/f2b x:0 0 0 2024-03-29T08:07:05.744 INFO:tasks.workunit.client.0.smithi037.stdout:6/26: symlink d0/d5/lc 0 2024-03-29T08:07:05.746 INFO:tasks.workunit.client.0.smithi037.stdout:2/53: dread d0/f2 [0,4194304] 0 2024-03-29T08:07:05.747 INFO:tasks.workunit.client.0.smithi037.stdout:2/54: chown d0/c9 1 1 2024-03-29T08:07:05.747 INFO:tasks.workunit.client.0.smithi037.stdout:2/55: readlink - no filename 2024-03-29T08:07:05.747 INFO:tasks.workunit.client.0.smithi037.stdout:8/70: dread d5/fb [0,4194304] 0 2024-03-29T08:07:05.748 INFO:tasks.workunit.client.0.smithi037.stdout:3/71: link fc dd/d15/f1a 0 2024-03-29T08:07:05.749 INFO:tasks.workunit.client.0.smithi037.stdout:3/72: dread - dd/fe zero size 2024-03-29T08:07:05.751 INFO:tasks.workunit.client.0.smithi037.stdout:5/64: write f3 [742743,10050] 0 2024-03-29T08:07:05.751 INFO:tasks.workunit.client.0.smithi037.stdout:3/73: dread - dd/fe zero size 2024-03-29T08:07:05.754 INFO:tasks.workunit.client.0.smithi037.stdout:1/91: rename d2/df to d2/d6/dc/d18/d1d/d21 0 2024-03-29T08:07:05.756 INFO:tasks.workunit.client.0.smithi037.stdout:7/147: creat dd/d21/f2c x:0 0 0 2024-03-29T08:07:05.757 INFO:tasks.workunit.client.0.smithi037.stdout:6/27: mkdir d0/d5/dd 0 2024-03-29T08:07:05.760 INFO:tasks.workunit.client.0.smithi037.stdout:5/65: write d8/fa [1591812,73400] 0 2024-03-29T08:07:05.761 INFO:tasks.workunit.client.0.smithi037.stdout:3/74: creat dd/f1b x:0 0 0 2024-03-29T08:07:05.763 INFO:tasks.workunit.client.0.smithi037.stdout:8/71: rename d5/l8 to d5/d10/l14 0 2024-03-29T08:07:05.765 INFO:tasks.workunit.client.0.smithi037.stdout:7/148: symlink dd/d21/l2d 0 2024-03-29T08:07:05.766 INFO:tasks.workunit.client.0.smithi037.stdout:5/66: mknod d8/c12 0 2024-03-29T08:07:05.771 INFO:tasks.workunit.client.0.smithi037.stdout:9/65: dwrite f0 [0,4194304] 0 2024-03-29T08:07:05.771 INFO:tasks.workunit.client.0.smithi037.stdout:7/149: chown dd/le 707041222 1 2024-03-29T08:07:05.772 INFO:tasks.workunit.client.0.smithi037.stdout:3/75: symlink dd/d15/l1c 0 2024-03-29T08:07:05.777 INFO:tasks.workunit.client.0.smithi037.stdout:5/67: creat d8/f13 x:0 0 0 2024-03-29T08:07:05.780 INFO:tasks.workunit.client.0.smithi037.stdout:6/28: symlink d0/d1/le 0 2024-03-29T08:07:05.781 INFO:tasks.workunit.client.0.smithi037.stdout:5/68: write f3 [2662545,70605] 0 2024-03-29T08:07:05.782 INFO:tasks.workunit.client.0.smithi037.stdout:8/72: link d5/d10/f11 d5/f15 0 2024-03-29T08:07:05.784 INFO:tasks.workunit.client.0.smithi037.stdout:2/56: write d0/fd [3211855,33324] 0 2024-03-29T08:07:05.787 INFO:tasks.workunit.client.0.smithi037.stdout:7/150: symlink dd/d21/l2e 0 2024-03-29T08:07:05.792 INFO:tasks.workunit.client.0.smithi037.stdout:9/66: mknod d1/d6/d16/c1a 0 2024-03-29T08:07:05.792 INFO:tasks.workunit.client.0.smithi037.stdout:2/57: fdatasync d0/f2 0 2024-03-29T08:07:05.794 INFO:tasks.workunit.client.0.smithi037.stdout:8/73: write d5/d10/f11 [8311908,17569] 0 2024-03-29T08:07:05.795 INFO:tasks.workunit.client.0.smithi037.stdout:5/69: rename l5 to d8/l14 0 2024-03-29T08:07:05.797 INFO:tasks.workunit.client.0.smithi037.stdout:1/92: dwrite d2/d6/dc/f1e [0,4194304] 0 2024-03-29T08:07:05.798 INFO:tasks.workunit.client.0.smithi037.stdout:7/151: symlink dd/l2f 0 2024-03-29T08:07:05.801 INFO:tasks.workunit.client.0.smithi037.stdout:5/70: read d8/ff [2790462,79110] 0 2024-03-29T08:07:05.802 INFO:tasks.workunit.client.0.smithi037.stdout:3/76: rmdir dd/d15 39 2024-03-29T08:07:05.802 INFO:tasks.workunit.client.0.smithi037.stdout:2/58: read d0/f4 [1442657,66247] 0 2024-03-29T08:07:05.804 INFO:tasks.workunit.client.0.smithi037.stdout:9/67: truncate d1/d3/fa 8863604 0 2024-03-29T08:07:05.807 INFO:tasks.workunit.client.0.smithi037.stdout:3/77: dread - dd/f12 zero size 2024-03-29T08:07:05.809 INFO:tasks.workunit.client.0.smithi037.stdout:2/59: chown d0/f4 75289977 1 2024-03-29T08:07:05.812 INFO:tasks.workunit.client.0.smithi037.stdout:1/93: mkdir d2/d6/dc/d18/d22 0 2024-03-29T08:07:05.812 INFO:tasks.workunit.client.0.smithi037.stdout:5/71: write d8/f13 [757354,36814] 0 2024-03-29T08:07:05.814 INFO:tasks.workunit.client.0.smithi037.stdout:7/152: symlink dd/l30 0 2024-03-29T08:07:05.820 INFO:tasks.workunit.client.0.smithi037.stdout:9/68: creat d1/d6/d10/f1b x:0 0 0 2024-03-29T08:07:05.820 INFO:tasks.workunit.client.0.smithi037.stdout:5/72: rmdir d8 39 2024-03-29T08:07:05.820 INFO:tasks.workunit.client.0.smithi037.stdout:1/94: chown d2/d6/dc/f19 245774323 1 2024-03-29T08:07:05.821 INFO:tasks.workunit.client.0.smithi037.stdout:2/60: symlink d0/l12 0 2024-03-29T08:07:05.823 INFO:tasks.workunit.client.0.smithi037.stdout:7/153: rmdir dd 39 2024-03-29T08:07:05.825 INFO:tasks.workunit.client.0.smithi037.stdout:5/73: readlink d8/lc 0 2024-03-29T08:07:05.827 INFO:tasks.workunit.client.0.smithi037.stdout:2/61: mknod d0/c13 0 2024-03-29T08:07:05.830 INFO:tasks.workunit.client.0.smithi037.stdout:8/74: dwrite d5/f15 [0,4194304] 0 2024-03-29T08:07:05.832 INFO:tasks.workunit.client.0.smithi037.stdout:3/78: rename dd/f12 to dd/f1d 0 2024-03-29T08:07:05.835 INFO:tasks.workunit.client.0.smithi037.stdout:5/74: mknod d8/c15 0 2024-03-29T08:07:05.835 INFO:tasks.workunit.client.0.smithi037.stdout:2/62: rename d0/f4 to d0/f14 0 2024-03-29T08:07:05.837 INFO:tasks.workunit.client.0.smithi037.stdout:7/154: fdatasync dd/f1f 0 2024-03-29T08:07:05.839 INFO:tasks.workunit.client.0.smithi037.stdout:8/75: creat d5/f16 x:0 0 0 2024-03-29T08:07:05.841 INFO:tasks.workunit.client.0.smithi037.stdout:8/76: fdatasync f1 0 2024-03-29T08:07:05.842 INFO:tasks.workunit.client.0.smithi037.stdout:9/69: link d1/d6/c7 d1/d18/c1c 0 2024-03-29T08:07:05.843 INFO:tasks.workunit.client.0.smithi037.stdout:5/75: mknod d8/c16 0 2024-03-29T08:07:05.846 INFO:tasks.workunit.client.0.smithi037.stdout:3/79: dread dd/d15/f19 [0,4194304] 0 2024-03-29T08:07:05.846 INFO:tasks.workunit.client.0.smithi037.stdout:5/76: chown d8/ff 7492006 1 2024-03-29T08:07:05.879 INFO:tasks.workunit.client.0.smithi037.stdout:9/70: mknod d1/d6/d10/c1d 0 2024-03-29T08:07:05.880 INFO:tasks.workunit.client.0.smithi037.stdout:8/77: link d5/fc d5/f17 0 2024-03-29T08:07:05.883 INFO:tasks.workunit.client.0.smithi037.stdout:5/77: creat d8/f17 x:0 0 0 2024-03-29T08:07:05.883 INFO:tasks.workunit.client.0.smithi037.stdout:1/95: dwrite d2/d6/dc/f19 [4194304,4194304] 0 2024-03-29T08:07:05.883 INFO:tasks.workunit.client.0.smithi037.stdout:2/63: getdents d0 0 2024-03-29T08:07:05.883 INFO:tasks.workunit.client.0.smithi037.stdout:3/80: fsync f2 0 2024-03-29T08:07:05.887 INFO:tasks.workunit.client.0.smithi037.stdout:8/78: symlink d5/l18 0 2024-03-29T08:07:05.887 INFO:tasks.workunit.client.0.smithi037.stdout:9/71: write d1/d6/d10/f1b [280066,113342] 0 2024-03-29T08:07:05.889 INFO:tasks.workunit.client.0.smithi037.stdout:2/64: creat d0/f15 x:0 0 0 2024-03-29T08:07:05.890 INFO:tasks.workunit.client.0.smithi037.stdout:9/72: chown d1/d4/fe 30423 1 2024-03-29T08:07:05.890 INFO:tasks.workunit.client.0.smithi037.stdout:8/79: write f1 [2127565,45572] 0 2024-03-29T08:07:05.891 INFO:tasks.workunit.client.0.smithi037.stdout:2/65: chown d0/c5 430 1 2024-03-29T08:07:05.893 INFO:tasks.workunit.client.0.smithi037.stdout:1/96: mknod d2/d6/dc/d18/d1d/d21/c23 0 2024-03-29T08:07:05.893 INFO:tasks.workunit.client.0.smithi037.stdout:3/81: write dd/d15/f19 [2125569,16503] 0 2024-03-29T08:07:05.893 INFO:tasks.workunit.client.0.smithi037.stdout:8/80: chown d5/l9 666 1 2024-03-29T08:07:05.897 INFO:tasks.workunit.client.0.smithi037.stdout:8/81: write d5/f16 [886059,70461] 0 2024-03-29T08:07:05.900 INFO:tasks.workunit.client.0.smithi037.stdout:1/97: chown d2/d6/dc/d18/d1d/d21/f20 21632 1 2024-03-29T08:07:05.902 INFO:tasks.workunit.client.0.smithi037.stdout:3/82: link dd/f1b dd/d15/f1e 0 2024-03-29T08:07:05.904 INFO:tasks.workunit.client.0.smithi037.stdout:8/82: mkdir d5/d10/d19 0 2024-03-29T08:07:05.906 INFO:tasks.workunit.client.0.smithi037.stdout:2/66: rmdir d0 39 2024-03-29T08:07:05.911 INFO:tasks.workunit.client.0.smithi037.stdout:9/73: dread d1/d3/fa [0,4194304] 0 2024-03-29T08:07:05.913 INFO:tasks.workunit.client.0.smithi037.stdout:9/74: dread - d1/d6/d10/d13/f17 zero size 2024-03-29T08:07:05.914 INFO:tasks.workunit.client.0.smithi037.stdout:1/98: getdents d2/d6/dc/d18/d1d 0 2024-03-29T08:07:05.920 INFO:tasks.workunit.client.0.smithi037.stdout:3/83: getdents dd 0 2024-03-29T08:07:05.922 INFO:tasks.workunit.client.0.smithi037.stdout:1/99: unlink d2/d6/dc/d18/d1d/d21/f1c 0 2024-03-29T08:07:05.923 INFO:tasks.workunit.client.0.smithi037.stdout:3/84: dread - dd/d15/f16 zero size 2024-03-29T08:07:05.931 INFO:tasks.workunit.client.0.smithi037.stdout:3/85: creat dd/f1f x:0 0 0 2024-03-29T08:07:05.938 INFO:tasks.workunit.client.0.smithi037.stdout:8/83: dwrite d5/fc [0,4194304] 0 2024-03-29T08:07:05.947 INFO:tasks.workunit.client.0.smithi037.stdout:1/100: dread d2/d6/dc/f15 [0,4194304] 0 2024-03-29T08:07:05.949 INFO:tasks.workunit.client.0.smithi037.stdout:8/84: stat d5/cf 0 2024-03-29T08:07:05.954 INFO:tasks.workunit.client.0.smithi037.stdout:2/67: dwrite d0/f7 [0,4194304] 0 2024-03-29T08:07:05.955 INFO:tasks.workunit.client.0.smithi037.stdout:3/86: symlink dd/d15/d18/l20 0 2024-03-29T08:07:05.957 INFO:tasks.workunit.client.0.smithi037.stdout:8/85: write d5/f17 [6013382,2637] 0 2024-03-29T08:07:05.957 INFO:tasks.workunit.client.0.smithi037.stdout:9/75: dwrite d1/d3/fa [0,4194304] 0 2024-03-29T08:07:05.962 INFO:tasks.workunit.client.0.smithi037.stdout:2/68: write d0/f7 [2261209,32994] 0 2024-03-29T08:07:05.962 INFO:tasks.workunit.client.0.smithi037.stdout:9/76: fdatasync d1/d4/fe 0 2024-03-29T08:07:05.970 INFO:tasks.workunit.client.0.smithi037.stdout:9/77: unlink d1/d4/fe 0 2024-03-29T08:07:05.972 INFO:tasks.workunit.client.0.smithi037.stdout:8/86: symlink d5/d10/d19/l1a 0 2024-03-29T08:07:05.974 INFO:tasks.workunit.client.0.smithi037.stdout:3/87: link f4 dd/d15/f21 0 2024-03-29T08:07:05.976 INFO:tasks.workunit.client.0.smithi037.stdout:8/87: fsync d5/d10/f11 0 2024-03-29T08:07:05.980 INFO:tasks.workunit.client.0.smithi037.stdout:3/88: symlink dd/d15/d18/l22 0 2024-03-29T08:07:05.981 INFO:tasks.workunit.client.0.smithi037.stdout:9/78: chown d1/d6/c14 39572248 1 2024-03-29T08:07:05.982 INFO:tasks.workunit.client.0.smithi037.stdout:8/88: rename d5/fc to d5/f1b 0 2024-03-29T08:07:05.984 INFO:tasks.workunit.client.0.smithi037.stdout:8/89: stat d5/f15 0 2024-03-29T08:07:05.985 INFO:tasks.workunit.client.0.smithi037.stdout:9/79: read d1/d3/fa [4108895,49543] 0 2024-03-29T08:07:05.986 INFO:tasks.workunit.client.0.smithi037.stdout:9/80: chown d1 100688 1 2024-03-29T08:07:05.987 INFO:tasks.workunit.client.0.smithi037.stdout:3/89: fdatasync dd/d15/f1e 0 2024-03-29T08:07:05.992 INFO:tasks.workunit.client.0.smithi037.stdout:9/81: mkdir d1/d4/d19/d1e 0 2024-03-29T08:07:05.995 INFO:tasks.workunit.client.0.smithi037.stdout:9/82: write d1/d6/d10/f12 [377576,32838] 0 2024-03-29T08:07:06.001 INFO:tasks.workunit.client.0.smithi037.stdout:3/90: read - dd/f1b zero size 2024-03-29T08:07:06.001 INFO:tasks.workunit.client.0.smithi037.stdout:9/83: mkdir d1/d1f 0 2024-03-29T08:07:06.002 INFO:tasks.workunit.client.0.smithi037.stdout:8/90: dread f4 [0,4194304] 0 2024-03-29T08:07:06.003 INFO:tasks.workunit.client.0.smithi037.stdout:3/91: rmdir dd/d15 39 2024-03-29T08:07:06.004 INFO:tasks.workunit.client.0.smithi037.stdout:9/84: write d1/d6/d10/f12 [8126,67720] 0 2024-03-29T08:07:06.007 INFO:tasks.workunit.client.0.smithi037.stdout:8/91: creat d5/f1c x:0 0 0 2024-03-29T08:07:06.008 INFO:tasks.workunit.client.0.smithi037.stdout:9/85: mknod d1/d6/d10/c20 0 2024-03-29T08:07:06.009 INFO:tasks.workunit.client.0.smithi037.stdout:8/92: fdatasync d5/f1c 0 2024-03-29T08:07:06.011 INFO:tasks.workunit.client.0.smithi037.stdout:3/92: write dd/f1d [429825,52154] 0 2024-03-29T08:07:06.011 INFO:tasks.workunit.client.0.smithi037.stdout:9/86: truncate d1/d6/d10/d13/f17 799701 0 2024-03-29T08:07:06.012 INFO:tasks.workunit.client.0.smithi037.stdout:8/93: creat d5/d10/f1d x:0 0 0 2024-03-29T08:07:06.014 INFO:tasks.workunit.client.1.smithi148.stdout:1/6: sync 2024-03-29T08:07:06.015 INFO:tasks.workunit.client.0.smithi037.stdout:8/94: creat d5/f1e x:0 0 0 2024-03-29T08:07:06.015 INFO:tasks.workunit.client.0.smithi037.stdout:3/93: fsync dd/ff 0 2024-03-29T08:07:06.017 INFO:tasks.workunit.client.0.smithi037.stdout:3/94: write fb [634115,1509] 0 2024-03-29T08:07:06.018 INFO:tasks.workunit.client.1.smithi148.stdout:1/7: rename f0 to f1 0 2024-03-29T08:07:06.022 INFO:tasks.workunit.client.1.smithi148.stdout:1/8: link f1 f2 0 2024-03-29T08:07:06.024 INFO:tasks.workunit.client.1.smithi148.stdout:1/9: mkdir d3 0 2024-03-29T08:07:06.024 INFO:tasks.workunit.client.0.smithi037.stdout:8/95: truncate d5/f15 8194519 0 2024-03-29T08:07:06.027 INFO:tasks.workunit.client.0.smithi037.stdout:3/95: unlink dd/d15/f1a 0 2024-03-29T08:07:06.030 INFO:tasks.workunit.client.1.smithi148.stdout:1/10: write f2 [940403,70081] 0 2024-03-29T08:07:06.030 INFO:tasks.workunit.client.0.smithi037.stdout:8/96: creat d5/f1f x:0 0 0 2024-03-29T08:07:06.033 INFO:tasks.workunit.client.1.smithi148.stdout:1/11: mknod d3/c4 0 2024-03-29T08:07:06.035 INFO:tasks.workunit.client.0.smithi037.stdout:3/96: creat dd/f23 x:0 0 0 2024-03-29T08:07:06.035 INFO:tasks.workunit.client.1.smithi148.stdout:1/12: mknod d3/c5 0 2024-03-29T08:07:06.036 INFO:tasks.workunit.client.0.smithi037.stdout:8/97: write f4 [5737880,76718] 0 2024-03-29T08:07:06.036 INFO:tasks.workunit.client.0.smithi037.stdout:9/87: dwrite f0 [4194304,4194304] 0 2024-03-29T08:07:06.037 INFO:tasks.workunit.client.1.smithi148.stdout:1/13: symlink d3/l6 0 2024-03-29T08:07:06.040 INFO:tasks.workunit.client.1.smithi148.stdout:1/14: creat d3/f7 x:0 0 0 2024-03-29T08:07:06.044 INFO:tasks.workunit.client.1.smithi148.stdout:1/15: unlink d3/c4 0 2024-03-29T08:07:06.045 INFO:tasks.workunit.client.0.smithi037.stdout:3/97: creat dd/f24 x:0 0 0 2024-03-29T08:07:06.047 INFO:tasks.workunit.client.0.smithi037.stdout:9/88: dread d1/d6/d10/f1b [0,4194304] 0 2024-03-29T08:07:06.053 INFO:tasks.workunit.client.0.smithi037.stdout:3/98: creat dd/f25 x:0 0 0 2024-03-29T08:07:06.053 INFO:tasks.workunit.client.1.smithi148.stdout:1/16: dread f2 [0,4194304] 0 2024-03-29T08:07:06.055 INFO:tasks.workunit.client.0.smithi037.stdout:9/89: write d1/d6/d10/f1b [1149506,77438] 0 2024-03-29T08:07:06.055 INFO:tasks.workunit.client.1.smithi148.stdout:1/17: dread - d3/f7 zero size 2024-03-29T08:07:06.055 INFO:tasks.workunit.client.1.smithi148.stdout:1/18: chown d3/c5 132423947 1 2024-03-29T08:07:06.058 INFO:tasks.workunit.client.0.smithi037.stdout:9/90: creat d1/d3/f21 x:0 0 0 2024-03-29T08:07:06.065 INFO:tasks.workunit.client.0.smithi037.stdout:9/91: mknod d1/d4/d19/d1e/c22 0 2024-03-29T08:07:06.069 INFO:tasks.workunit.client.0.smithi037.stdout:9/92: write d1/d3/fa [7727914,16338] 0 2024-03-29T08:07:06.071 INFO:tasks.workunit.client.0.smithi037.stdout:8/98: dwrite d5/d10/f11 [0,4194304] 0 2024-03-29T08:07:06.075 INFO:tasks.workunit.client.0.smithi037.stdout:9/93: mkdir d1/d6/d10/d23 0 2024-03-29T08:07:06.075 INFO:tasks.workunit.client.0.smithi037.stdout:8/99: symlink d5/d10/l20 0 2024-03-29T08:07:06.077 INFO:tasks.workunit.client.0.smithi037.stdout:9/94: fsync d1/d6/d10/f12 0 2024-03-29T08:07:06.078 INFO:tasks.workunit.client.0.smithi037.stdout:8/100: symlink d5/d10/d19/l21 0 2024-03-29T08:07:06.081 INFO:tasks.workunit.client.0.smithi037.stdout:9/95: getdents d1/d1f 0 2024-03-29T08:07:06.085 INFO:tasks.workunit.client.1.smithi148.stdout:1/19: fsync f1 0 2024-03-29T08:07:06.086 INFO:tasks.workunit.client.0.smithi037.stdout:9/96: rename d1/d6/c14 to d1/d4/d19/d1e/c24 0 2024-03-29T08:07:06.087 INFO:tasks.workunit.client.0.smithi037.stdout:9/97: read - d1/d3/f21 zero size 2024-03-29T08:07:06.088 INFO:tasks.workunit.client.1.smithi148.stdout:1/20: rmdir d3 39 2024-03-29T08:07:06.091 INFO:tasks.workunit.client.1.smithi148.stdout:1/21: symlink d3/l8 0 2024-03-29T08:07:06.093 INFO:tasks.workunit.client.1.smithi148.stdout:1/22: mkdir d3/d9 0 2024-03-29T08:07:06.094 INFO:tasks.workunit.client.0.smithi037.stdout:9/98: read d1/d6/d10/d13/f17 [220093,77086] 0 2024-03-29T08:07:06.096 INFO:tasks.workunit.client.1.smithi148.stdout:1/23: dread f1 [0,4194304] 0 2024-03-29T08:07:06.097 INFO:tasks.workunit.client.1.smithi148.stdout:1/24: truncate d3/f7 542277 0 2024-03-29T08:07:06.098 INFO:tasks.workunit.client.1.smithi148.stdout:1/25: chown f1 76039010 1 2024-03-29T08:07:06.102 INFO:tasks.workunit.client.1.smithi148.stdout:1/26: read d3/f7 [504953,7041] 0 2024-03-29T08:07:06.102 INFO:tasks.workunit.client.0.smithi037.stdout:9/99: getdents d1/d18 0 2024-03-29T08:07:06.103 INFO:tasks.workunit.client.0.smithi037.stdout:4/18: sync 2024-03-29T08:07:06.103 INFO:tasks.workunit.client.0.smithi037.stdout:0/37: sync 2024-03-29T08:07:06.103 INFO:tasks.workunit.client.1.smithi148.stdout:1/27: dread f2 [0,4194304] 0 2024-03-29T08:07:06.104 INFO:tasks.workunit.client.0.smithi037.stdout:8/101: dwrite d5/d10/f1d [0,4194304] 0 2024-03-29T08:07:06.105 INFO:tasks.workunit.client.1.smithi148.stdout:1/28: dread f2 [0,4194304] 0 2024-03-29T08:07:06.107 INFO:tasks.workunit.client.1.smithi148.stdout:1/29: read f2 [702448,107301] 0 2024-03-29T08:07:06.108 INFO:tasks.workunit.client.1.smithi148.stdout:1/30: fdatasync f2 0 2024-03-29T08:07:06.108 INFO:tasks.workunit.client.0.smithi037.stdout:9/100: rename d1/d6 to d1/d4/d19/d1e/d25 0 2024-03-29T08:07:06.108 INFO:tasks.workunit.client.0.smithi037.stdout:0/38: creat f7 x:0 0 0 2024-03-29T08:07:06.109 INFO:tasks.workunit.client.1.smithi148.stdout:1/31: write f2 [1363719,87997] 0 2024-03-29T08:07:06.109 INFO:tasks.workunit.client.0.smithi037.stdout:4/19: write f0 [993317,106740] 0 2024-03-29T08:07:06.110 INFO:tasks.workunit.client.0.smithi037.stdout:9/101: readlink d1/l5 0 2024-03-29T08:07:06.113 INFO:tasks.workunit.client.0.smithi037.stdout:0/39: rename f6 to f8 0 2024-03-29T08:07:06.114 INFO:tasks.workunit.client.1.smithi148.stdout:1/32: rmdir d3/d9 0 2024-03-29T08:07:06.115 INFO:tasks.workunit.client.0.smithi037.stdout:4/20: mkdir d3/d4/d8 0 2024-03-29T08:07:06.118 INFO:tasks.workunit.client.0.smithi037.stdout:9/102: mknod d1/d4/d19/c26 0 2024-03-29T08:07:06.118 INFO:tasks.workunit.client.0.smithi037.stdout:9/103: readlink d1/lb 0 2024-03-29T08:07:06.120 INFO:tasks.workunit.client.0.smithi037.stdout:0/40: unlink c3 0 2024-03-29T08:07:06.121 INFO:tasks.workunit.client.1.smithi148.stdout:1/33: link f1 d3/fa 0 2024-03-29T08:07:06.123 INFO:tasks.workunit.client.0.smithi037.stdout:9/104: creat d1/d18/f27 x:0 0 0 2024-03-29T08:07:06.123 INFO:tasks.workunit.client.0.smithi037.stdout:0/41: write f5 [1031537,128683] 0 2024-03-29T08:07:06.124 INFO:tasks.workunit.client.1.smithi148.stdout:1/34: mknod d3/cb 0 2024-03-29T08:07:06.124 INFO:tasks.workunit.client.0.smithi037.stdout:9/105: readlink d1/l5 0 2024-03-29T08:07:06.128 INFO:tasks.workunit.client.0.smithi037.stdout:0/42: mkdir d9 0 2024-03-29T08:07:06.128 INFO:tasks.workunit.client.0.smithi037.stdout:4/21: dread f0 [0,4194304] 0 2024-03-29T08:07:06.129 INFO:tasks.workunit.client.1.smithi148.stdout:1/35: link d3/cb d3/cc 0 2024-03-29T08:07:06.132 INFO:tasks.workunit.client.0.smithi037.stdout:9/106: rmdir d1/d4 39 2024-03-29T08:07:06.150 INFO:tasks.workunit.client.0.smithi037.stdout:0/43: dwrite f5 [0,4194304] 0 2024-03-29T08:07:06.150 INFO:tasks.workunit.client.1.smithi148.stdout:5/3: sync 2024-03-29T08:07:06.151 INFO:tasks.workunit.client.1.smithi148.stdout:5/4: write - no filename 2024-03-29T08:07:06.151 INFO:tasks.workunit.client.1.smithi148.stdout:5/5: chown d0 597543 1 2024-03-29T08:07:06.151 INFO:tasks.workunit.client.1.smithi148.stdout:5/6: read - no filename 2024-03-29T08:07:06.151 INFO:tasks.workunit.client.1.smithi148.stdout:5/7: unlink - no file 2024-03-29T08:07:06.152 INFO:tasks.workunit.client.1.smithi148.stdout:5/8: rename d0 to d0/d1 22 2024-03-29T08:07:06.153 INFO:tasks.workunit.client.0.smithi037.stdout:9/107: dwrite d1/d3/fa [4194304,4194304] 0 2024-03-29T08:07:06.153 INFO:tasks.workunit.client.1.smithi148.stdout:5/9: rename d0 to d0/d2 22 2024-03-29T08:07:06.153 INFO:tasks.workunit.client.1.smithi148.stdout:5/10: dwrite - no filename 2024-03-29T08:07:06.153 INFO:tasks.workunit.client.1.smithi148.stdout:5/11: dread - no filename 2024-03-29T08:07:06.153 INFO:tasks.workunit.client.1.smithi148.stdout:5/12: dwrite - no filename 2024-03-29T08:07:06.153 INFO:tasks.workunit.client.1.smithi148.stdout:5/13: fdatasync - no filename 2024-03-29T08:07:06.153 INFO:tasks.workunit.client.1.smithi148.stdout:5/14: write - no filename 2024-03-29T08:07:06.154 INFO:tasks.workunit.client.0.smithi037.stdout:0/44: mkdir d9/da 0 2024-03-29T08:07:06.154 INFO:tasks.workunit.client.1.smithi148.stdout:5/15: chown d0 241 1 2024-03-29T08:07:06.155 INFO:tasks.workunit.client.0.smithi037.stdout:0/45: rmdir d9 39 2024-03-29T08:07:06.156 INFO:tasks.workunit.client.1.smithi148.stdout:5/16: getdents d0 0 2024-03-29T08:07:06.159 INFO:tasks.workunit.client.1.smithi148.stdout:5/17: creat d0/f3 x:0 0 0 2024-03-29T08:07:06.160 INFO:tasks.workunit.client.0.smithi037.stdout:0/46: creat d9/fb x:0 0 0 2024-03-29T08:07:06.163 INFO:tasks.workunit.client.1.smithi148.stdout:5/18: symlink d0/l4 0 2024-03-29T08:07:06.166 INFO:tasks.workunit.client.0.smithi037.stdout:0/47: symlink d9/lc 0 2024-03-29T08:07:06.169 INFO:tasks.workunit.client.0.smithi037.stdout:9/108: dread f0 [0,4194304] 0 2024-03-29T08:07:06.171 INFO:tasks.workunit.client.0.smithi037.stdout:9/109: rmdir d1/d3 39 2024-03-29T08:07:06.176 INFO:tasks.workunit.client.0.smithi037.stdout:9/110: creat d1/d4/f28 x:0 0 0 2024-03-29T08:07:06.176 INFO:tasks.workunit.client.1.smithi148.stdout:5/19: dwrite d0/f3 [0,4194304] 0 2024-03-29T08:07:06.177 INFO:tasks.workunit.client.0.smithi037.stdout:0/48: fdatasync f5 0 2024-03-29T08:07:06.180 INFO:tasks.workunit.client.1.smithi148.stdout:5/20: mknod d0/c5 0 2024-03-29T08:07:06.180 INFO:tasks.workunit.client.0.smithi037.stdout:0/49: mkdir d9/da/dd 0 2024-03-29T08:07:06.181 INFO:tasks.workunit.client.1.smithi148.stdout:5/21: chown d0/f3 27211135 1 2024-03-29T08:07:06.185 INFO:tasks.workunit.client.1.smithi148.stdout:5/22: unlink d0/c5 0 2024-03-29T08:07:06.187 INFO:tasks.workunit.client.1.smithi148.stdout:5/23: symlink d0/l6 0 2024-03-29T08:07:06.190 INFO:tasks.workunit.client.0.smithi037.stdout:0/50: getdents d9/da 0 2024-03-29T08:07:06.190 INFO:tasks.workunit.client.1.smithi148.stdout:5/24: unlink d0/l6 0 2024-03-29T08:07:06.191 INFO:tasks.workunit.client.1.smithi148.stdout:5/25: read d0/f3 [2193164,81451] 0 2024-03-29T08:07:06.193 INFO:tasks.workunit.client.0.smithi037.stdout:0/51: truncate f7 957619 0 2024-03-29T08:07:06.194 INFO:tasks.workunit.client.1.smithi148.stdout:5/26: write d0/f3 [3832545,106914] 0 2024-03-29T08:07:06.199 INFO:tasks.workunit.client.0.smithi037.stdout:0/52: rename d9/lc to d9/da/le 0 2024-03-29T08:07:06.201 INFO:tasks.workunit.client.1.smithi148.stdout:5/27: truncate d0/f3 1803248 0 2024-03-29T08:07:06.203 INFO:tasks.workunit.client.0.smithi037.stdout:0/53: creat d9/ff x:0 0 0 2024-03-29T08:07:06.204 INFO:tasks.workunit.client.0.smithi037.stdout:9/111: dwrite d1/d18/f27 [0,4194304] 0 2024-03-29T08:07:06.205 INFO:tasks.workunit.client.0.smithi037.stdout:0/54: chown c4 6165 1 2024-03-29T08:07:06.213 INFO:tasks.workunit.client.0.smithi037.stdout:9/112: dread d1/d4/d19/d1e/d25/d10/d13/f17 [0,4194304] 0 2024-03-29T08:07:06.215 INFO:tasks.workunit.client.0.smithi037.stdout:9/113: rmdir d1/d3 39 2024-03-29T08:07:06.218 INFO:tasks.workunit.client.0.smithi037.stdout:9/114: mknod d1/d4/d19/d1e/d25/d10/c29 0 2024-03-29T08:07:06.220 INFO:tasks.workunit.client.0.smithi037.stdout:9/115: write d1/d3/f21 [160440,29652] 0 2024-03-29T08:07:06.223 INFO:tasks.workunit.client.0.smithi037.stdout:9/116: fsync d1/d4/d19/d1e/d25/d10/d13/f17 0 2024-03-29T08:07:06.226 INFO:tasks.workunit.client.0.smithi037.stdout:9/117: rename d1/d4/d19/d1e/c22 to d1/d4/d19/d1e/c2a 0 2024-03-29T08:07:06.236 INFO:tasks.workunit.client.0.smithi037.stdout:9/118: truncate d1/d4/d19/d1e/d25/d10/f1b 831011 0 2024-03-29T08:07:06.239 INFO:tasks.workunit.client.0.smithi037.stdout:9/119: unlink f0 0 2024-03-29T08:07:06.241 INFO:tasks.workunit.client.0.smithi037.stdout:9/120: creat d1/d4/d19/f2b x:0 0 0 2024-03-29T08:07:06.243 INFO:tasks.workunit.client.0.smithi037.stdout:9/121: write d1/d4/f28 [295171,72634] 0 2024-03-29T08:07:06.257 INFO:tasks.workunit.client.0.smithi037.stdout:9/122: dwrite d1/d18/f27 [0,4194304] 0 2024-03-29T08:07:06.262 INFO:tasks.workunit.client.0.smithi037.stdout:9/123: symlink d1/d4/d19/d1e/d25/d10/d23/l2c 0 2024-03-29T08:07:06.265 INFO:tasks.workunit.client.1.smithi148.stdout:9/7: sync 2024-03-29T08:07:06.265 INFO:tasks.workunit.client.1.smithi148.stdout:6/7: sync 2024-03-29T08:07:06.265 INFO:tasks.workunit.client.1.smithi148.stdout:9/8: dwrite - no filename 2024-03-29T08:07:06.265 INFO:tasks.workunit.client.1.smithi148.stdout:4/18: sync 2024-03-29T08:07:06.265 INFO:tasks.workunit.client.1.smithi148.stdout:2/21: sync 2024-03-29T08:07:06.265 INFO:tasks.workunit.client.1.smithi148.stdout:3/29: sync 2024-03-29T08:07:06.265 INFO:tasks.workunit.client.1.smithi148.stdout:0/51: sync 2024-03-29T08:07:06.265 INFO:tasks.workunit.client.1.smithi148.stdout:7/61: sync 2024-03-29T08:07:06.265 INFO:tasks.workunit.client.1.smithi148.stdout:8/66: sync 2024-03-29T08:07:06.266 INFO:tasks.workunit.client.0.smithi037.stdout:9/124: rename d1/d4/d19 to d1/d4/d19/d1e/d25/d10/d13/d2d 22 2024-03-29T08:07:06.266 INFO:tasks.workunit.client.0.smithi037.stdout:9/125: readlink d1/l5 0 2024-03-29T08:07:06.267 INFO:tasks.workunit.client.1.smithi148.stdout:3/30: fdatasync f8 0 2024-03-29T08:07:06.267 INFO:tasks.workunit.client.1.smithi148.stdout:3/31: rmdir - no directory 2024-03-29T08:07:06.267 INFO:tasks.workunit.client.1.smithi148.stdout:4/19: dread - f5 zero size 2024-03-29T08:07:06.267 INFO:tasks.workunit.client.1.smithi148.stdout:3/32: read f8 [3734708,121717] 0 2024-03-29T08:07:06.268 INFO:tasks.workunit.client.1.smithi148.stdout:9/9: chown c1 976114 1 2024-03-29T08:07:06.268 INFO:tasks.workunit.client.1.smithi148.stdout:6/8: mknod c1 0 2024-03-29T08:07:06.268 INFO:tasks.workunit.client.1.smithi148.stdout:9/10: dread - no filename 2024-03-29T08:07:06.268 INFO:tasks.workunit.client.1.smithi148.stdout:2/22: symlink l6 0 2024-03-29T08:07:06.269 INFO:tasks.workunit.client.0.smithi037.stdout:9/126: mknod d1/d4/d19/c2e 0 2024-03-29T08:07:06.269 INFO:tasks.workunit.client.1.smithi148.stdout:0/52: dread f9 [0,4194304] 0 2024-03-29T08:07:06.269 INFO:tasks.workunit.client.1.smithi148.stdout:6/9: dread - f0 zero size 2024-03-29T08:07:06.269 INFO:tasks.workunit.client.1.smithi148.stdout:6/10: rmdir - no directory 2024-03-29T08:07:06.269 INFO:tasks.workunit.client.1.smithi148.stdout:6/11: rmdir - no directory 2024-03-29T08:07:06.270 INFO:tasks.workunit.client.1.smithi148.stdout:3/33: chown f1 253 1 2024-03-29T08:07:06.270 INFO:tasks.workunit.client.1.smithi148.stdout:6/12: fsync f0 0 2024-03-29T08:07:06.270 INFO:tasks.workunit.client.1.smithi148.stdout:3/34: stat f8 0 2024-03-29T08:07:06.271 INFO:tasks.workunit.client.1.smithi148.stdout:2/23: mkdir d7 0 2024-03-29T08:07:06.272 INFO:tasks.workunit.client.1.smithi148.stdout:0/53: dread - db/dc/f12 zero size 2024-03-29T08:07:06.272 INFO:tasks.workunit.client.1.smithi148.stdout:9/11: getdents . 0 2024-03-29T08:07:06.272 INFO:tasks.workunit.client.1.smithi148.stdout:9/12: truncate - no filename 2024-03-29T08:07:06.273 INFO:tasks.workunit.client.0.smithi037.stdout:9/127: symlink d1/d4/d19/d1e/d25/d10/d13/l2f 0 2024-03-29T08:07:06.273 INFO:tasks.workunit.client.0.smithi037.stdout:9/128: rename d1 to d1/d4/d19/d30 22 2024-03-29T08:07:06.276 INFO:tasks.workunit.client.0.smithi037.stdout:9/129: write d1/d4/d19/d1e/d25/d10/f12 [903332,78161] 0 2024-03-29T08:07:06.292 INFO:tasks.workunit.client.1.smithi148.stdout:6/13: write f0 [325144,65600] 0 2024-03-29T08:07:06.292 INFO:tasks.workunit.client.1.smithi148.stdout:4/20: truncate f4 243698 0 2024-03-29T08:07:06.293 INFO:tasks.workunit.client.1.smithi148.stdout:2/24: write f5 [439401,69072] 0 2024-03-29T08:07:06.293 INFO:tasks.workunit.client.1.smithi148.stdout:9/13: creat f3 x:0 0 0 2024-03-29T08:07:06.295 INFO:tasks.workunit.client.0.smithi037.stdout:9/130: dwrite d1/d4/d19/d1e/d25/d10/f12 [0,4194304] 0 2024-03-29T08:07:06.295 INFO:tasks.workunit.client.1.smithi148.stdout:6/14: fsync f0 0 2024-03-29T08:07:06.296 INFO:tasks.workunit.client.1.smithi148.stdout:4/21: write f5 [36264,100297] 0 2024-03-29T08:07:06.296 INFO:tasks.workunit.client.1.smithi148.stdout:0/54: getdents db/de 0 2024-03-29T08:07:06.300 INFO:tasks.workunit.client.0.smithi037.stdout:9/131: mkdir d1/d4/d19/d1e/d25/d10/d31 0 2024-03-29T08:07:06.303 INFO:tasks.workunit.client.0.smithi037.stdout:9/132: creat d1/d18/f32 x:0 0 0 2024-03-29T08:07:06.305 INFO:tasks.workunit.client.0.smithi037.stdout:9/133: fsync d1/d4/f28 0 2024-03-29T08:07:06.309 INFO:tasks.workunit.client.0.smithi037.stdout:9/134: creat d1/d1f/f33 x:0 0 0 2024-03-29T08:07:06.326 INFO:tasks.workunit.client.1.smithi148.stdout:1/36: sync 2024-03-29T08:07:06.326 INFO:tasks.workunit.client.1.smithi148.stdout:5/28: sync 2024-03-29T08:07:06.329 INFO:tasks.workunit.client.1.smithi148.stdout:9/14: creat f4 x:0 0 0 2024-03-29T08:07:06.331 INFO:tasks.workunit.client.1.smithi148.stdout:0/55: truncate f9 1459880 0 2024-03-29T08:07:06.332 INFO:tasks.workunit.client.0.smithi037.stdout:9/135: dwrite d1/d4/d19/d1e/d25/d10/f1b [0,4194304] 0 2024-03-29T08:07:06.332 INFO:tasks.workunit.client.1.smithi148.stdout:1/37: unlink d3/l8 0 2024-03-29T08:07:06.338 INFO:tasks.workunit.client.1.smithi148.stdout:7/62: dwrite d8/f10 [0,4194304] 0 2024-03-29T08:07:06.338 INFO:tasks.workunit.client.1.smithi148.stdout:6/15: dread f0 [0,4194304] 0 2024-03-29T08:07:06.339 INFO:tasks.workunit.client.0.smithi037.stdout:9/136: rename d1/d4/d19/d1e/d25/d10/f1b to d1/d4/d19/d1e/d25/f34 0 2024-03-29T08:07:06.339 INFO:tasks.workunit.client.0.smithi037.stdout:9/137: readlink d1/d4/lc 0 2024-03-29T08:07:06.341 INFO:tasks.workunit.client.0.smithi037.stdout:9/138: write d1/d4/f28 [262000,24902] 0 2024-03-29T08:07:06.343 INFO:tasks.workunit.client.0.smithi037.stdout:9/139: readlink d1/d4/d19/d1e/d25/d10/d13/l2f 0 2024-03-29T08:07:06.343 INFO:tasks.workunit.client.0.smithi037.stdout:9/140: chown d1/d1f 138144355 1 2024-03-29T08:07:06.345 INFO:tasks.workunit.client.0.smithi037.stdout:9/141: write d1/d18/f27 [3901500,111864] 0 2024-03-29T08:07:06.348 INFO:tasks.workunit.client.1.smithi148.stdout:6/16: write f0 [1342264,23715] 0 2024-03-29T08:07:06.349 INFO:tasks.workunit.client.0.smithi037.stdout:9/142: unlink d1/d4/d19/f2b 0 2024-03-29T08:07:06.352 INFO:tasks.workunit.client.1.smithi148.stdout:7/63: stat d8/f10 0 2024-03-29T08:07:06.355 INFO:tasks.workunit.client.0.smithi037.stdout:9/143: write d1/d4/d19/d1e/d25/f34 [1123039,93471] 0 2024-03-29T08:07:06.357 INFO:tasks.workunit.client.1.smithi148.stdout:2/25: dread f5 [0,4194304] 0 2024-03-29T08:07:06.358 INFO:tasks.workunit.client.0.smithi037.stdout:9/144: symlink d1/d4/d19/l35 0 2024-03-29T08:07:06.363 INFO:tasks.workunit.client.0.smithi037.stdout:9/145: link d1/d4/lc d1/d4/d19/d1e/d25/l36 0 2024-03-29T08:07:06.367 INFO:tasks.workunit.client.0.smithi037.stdout:9/146: creat d1/d4/d19/d1e/d25/d10/d23/f37 x:0 0 0 2024-03-29T08:07:06.370 INFO:tasks.workunit.client.1.smithi148.stdout:4/22: dwrite f7 [0,4194304] 0 2024-03-29T08:07:06.382 INFO:tasks.workunit.client.1.smithi148.stdout:8/67: sync 2024-03-29T08:07:06.383 INFO:tasks.workunit.client.1.smithi148.stdout:4/23: mknod c8 0 2024-03-29T08:07:06.383 INFO:tasks.workunit.client.0.smithi037.stdout:9/147: dread d1/d4/d19/d1e/d25/d10/f12 [0,4194304] 0 2024-03-29T08:07:06.387 INFO:tasks.workunit.client.0.smithi037.stdout:9/148: write d1/d3/f21 [967249,22315] 0 2024-03-29T08:07:06.387 INFO:tasks.workunit.client.1.smithi148.stdout:9/15: dwrite f3 [0,4194304] 0 2024-03-29T08:07:06.388 INFO:tasks.workunit.client.1.smithi148.stdout:0/56: dwrite db/f10 [0,4194304] 0 2024-03-29T08:07:06.390 INFO:tasks.workunit.client.0.smithi037.stdout:9/149: fdatasync d1/d4/d19/d1e/d25/d10/f12 0 2024-03-29T08:07:06.394 INFO:tasks.workunit.client.0.smithi037.stdout:9/150: dread - d1/d4/d19/d1e/d25/d10/d23/f37 zero size 2024-03-29T08:07:06.401 INFO:tasks.workunit.client.1.smithi148.stdout:9/16: mknod c5 0 2024-03-29T08:07:06.402 INFO:tasks.workunit.client.1.smithi148.stdout:5/29: dwrite d0/f3 [0,4194304] 0 2024-03-29T08:07:06.405 INFO:tasks.workunit.client.0.smithi037.stdout:9/151: dread d1/d4/d19/d1e/d25/d10/d13/f17 [0,4194304] 0 2024-03-29T08:07:06.405 INFO:tasks.workunit.client.1.smithi148.stdout:9/17: stat c5 0 2024-03-29T08:07:06.409 INFO:tasks.workunit.client.0.smithi037.stdout:9/152: getdents d1/d3 0 2024-03-29T08:07:06.411 INFO:tasks.workunit.client.1.smithi148.stdout:6/17: dwrite f0 [0,4194304] 0 2024-03-29T08:07:06.412 INFO:tasks.workunit.client.1.smithi148.stdout:0/57: symlink db/de/df/l13 0 2024-03-29T08:07:06.412 INFO:tasks.workunit.client.0.smithi037.stdout:9/153: chown d1/d4/d19/d1e/d25/d10 19794 1 2024-03-29T08:07:06.414 INFO:tasks.workunit.client.1.smithi148.stdout:9/18: chown c1 35478 1 2024-03-29T08:07:06.414 INFO:tasks.workunit.client.0.smithi037.stdout:9/154: write d1/d3/f21 [1229172,90315] 0 2024-03-29T08:07:06.415 INFO:tasks.workunit.client.1.smithi148.stdout:2/26: dread f5 [0,4194304] 0 2024-03-29T08:07:06.416 INFO:tasks.workunit.client.1.smithi148.stdout:5/30: write d0/f3 [193131,87767] 0 2024-03-29T08:07:06.416 INFO:tasks.workunit.client.1.smithi148.stdout:7/64: dwrite f4 [0,4194304] 0 2024-03-29T08:07:06.417 INFO:tasks.workunit.client.0.smithi037.stdout:9/155: chown d1/d18/c1c 1375226041 1 2024-03-29T08:07:06.419 INFO:tasks.workunit.client.1.smithi148.stdout:9/19: dread - f4 zero size 2024-03-29T08:07:06.422 INFO:tasks.workunit.client.1.smithi148.stdout:1/38: dwrite d3/fa [0,4194304] 0 2024-03-29T08:07:06.422 INFO:tasks.workunit.client.1.smithi148.stdout:8/68: dread f15 [0,4194304] 0 2024-03-29T08:07:06.422 INFO:tasks.workunit.client.1.smithi148.stdout:6/18: write f0 [970492,35062] 0 2024-03-29T08:07:06.424 INFO:tasks.workunit.client.1.smithi148.stdout:3/35: sync 2024-03-29T08:07:06.426 INFO:tasks.workunit.client.1.smithi148.stdout:4/24: dread f1 [0,4194304] 0 2024-03-29T08:07:06.426 INFO:tasks.workunit.client.1.smithi148.stdout:9/20: mknod c6 0 2024-03-29T08:07:06.426 INFO:tasks.workunit.client.1.smithi148.stdout:6/19: chown f0 40612 1 2024-03-29T08:07:06.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:07:06 smithi037 ceph-mon[35162]: pgmap v35: 65 pgs: 65 active+clean; 315 MiB data, 2.8 GiB used, 534 GiB / 536 GiB avail; 217 KiB/s rd, 20 MiB/s wr, 244 op/s 2024-03-29T08:07:06.427 INFO:tasks.workunit.client.1.smithi148.stdout:3/36: read f5 [159951,41313] 0 2024-03-29T08:07:06.428 INFO:tasks.workunit.client.1.smithi148.stdout:2/27: chown l6 5656859 1 2024-03-29T08:07:06.430 INFO:tasks.workunit.client.1.smithi148.stdout:0/58: mkdir db/de/df/d14 0 2024-03-29T08:07:06.430 INFO:tasks.workunit.client.1.smithi148.stdout:6/20: read f0 [739509,129400] 0 2024-03-29T08:07:06.433 INFO:tasks.workunit.client.1.smithi148.stdout:7/65: rename d8/lf to d8/l11 0 2024-03-29T08:07:06.434 INFO:tasks.workunit.client.1.smithi148.stdout:9/21: truncate f3 5124146 0 2024-03-29T08:07:06.436 INFO:tasks.workunit.client.1.smithi148.stdout:3/37: unlink l6 0 2024-03-29T08:07:06.438 INFO:tasks.workunit.client.1.smithi148.stdout:4/25: dread f4 [0,4194304] 0 2024-03-29T08:07:06.439 INFO:tasks.workunit.client.1.smithi148.stdout:4/26: rmdir - no directory 2024-03-29T08:07:06.440 INFO:tasks.workunit.client.0.smithi037.stdout:9/156: dwrite d1/d18/f32 [0,4194304] 0 2024-03-29T08:07:06.441 INFO:tasks.workunit.client.1.smithi148.stdout:6/21: symlink l2 0 2024-03-29T08:07:06.443 INFO:tasks.workunit.client.1.smithi148.stdout:3/38: mknod cc 0 2024-03-29T08:07:06.445 INFO:tasks.workunit.client.1.smithi148.stdout:7/66: dread d8/fe [4194304,4194304] 0 2024-03-29T08:07:06.445 INFO:tasks.workunit.client.1.smithi148.stdout:2/28: creat d7/f8 x:0 0 0 2024-03-29T08:07:06.446 INFO:tasks.workunit.client.1.smithi148.stdout:9/22: unlink c0 0 2024-03-29T08:07:06.447 INFO:tasks.workunit.client.0.smithi037.stdout:9/157: creat d1/d4/d19/d1e/d25/d10/d31/f38 x:0 0 0 2024-03-29T08:07:06.448 INFO:tasks.workunit.client.1.smithi148.stdout:3/39: chown c7 62702 1 2024-03-29T08:07:06.450 INFO:tasks.workunit.client.1.smithi148.stdout:4/27: rename l6 to l9 0 2024-03-29T08:07:06.452 INFO:tasks.workunit.client.1.smithi148.stdout:1/39: write d3/f7 [111474,78023] 0 2024-03-29T08:07:06.452 INFO:tasks.workunit.client.0.smithi037.stdout:9/158: creat d1/d4/d19/d1e/d25/d10/d23/f39 x:0 0 0 2024-03-29T08:07:06.455 INFO:tasks.workunit.client.0.smithi037.stdout:9/159: unlink d1/d18/f32 0 2024-03-29T08:07:06.456 INFO:tasks.workunit.client.1.smithi148.stdout:9/23: write f3 [3713279,62186] 0 2024-03-29T08:07:06.458 INFO:tasks.workunit.client.0.smithi037.stdout:9/160: readlink d1/d4/d19/d1e/d25/d10/d23/l2c 0 2024-03-29T08:07:06.459 INFO:tasks.workunit.client.1.smithi148.stdout:5/31: dwrite d0/f3 [0,4194304] 0 2024-03-29T08:07:06.460 INFO:tasks.workunit.client.1.smithi148.stdout:2/29: chown c0 156137 1 2024-03-29T08:07:06.466 INFO:tasks.workunit.client.1.smithi148.stdout:4/28: dread f5 [0,4194304] 0 2024-03-29T08:07:06.468 INFO:tasks.workunit.client.1.smithi148.stdout:8/69: dwrite f15 [0,4194304] 0 2024-03-29T08:07:06.469 INFO:tasks.workunit.client.1.smithi148.stdout:9/24: creat f7 x:0 0 0 2024-03-29T08:07:06.469 INFO:tasks.workunit.client.1.smithi148.stdout:5/32: readlink d0/l4 0 2024-03-29T08:07:06.470 INFO:tasks.workunit.client.0.smithi037.stdout:9/161: rename d1/d18 to d1/d1f/d3a 0 2024-03-29T08:07:06.473 INFO:tasks.workunit.client.0.smithi037.stdout:9/162: creat d1/d4/f3b x:0 0 0 2024-03-29T08:07:06.474 INFO:tasks.workunit.client.1.smithi148.stdout:0/59: truncate f9 404176 0 2024-03-29T08:07:06.479 INFO:tasks.workunit.client.1.smithi148.stdout:0/60: dread - db/dc/f12 zero size 2024-03-29T08:07:06.480 INFO:tasks.workunit.client.1.smithi148.stdout:2/30: creat d7/f9 x:0 0 0 2024-03-29T08:07:06.482 INFO:tasks.workunit.client.1.smithi148.stdout:5/33: write d0/f3 [2472971,47952] 0 2024-03-29T08:07:06.485 INFO:tasks.workunit.client.1.smithi148.stdout:7/67: truncate f2 952652 0 2024-03-29T08:07:06.486 INFO:tasks.workunit.client.1.smithi148.stdout:2/31: read - d7/f9 zero size 2024-03-29T08:07:06.486 INFO:tasks.workunit.client.1.smithi148.stdout:8/70: symlink d17/d18/l19 0 2024-03-29T08:07:06.494 INFO:tasks.workunit.client.0.smithi037.stdout:9/163: dwrite d1/d1f/d3a/f27 [0,4194304] 0 2024-03-29T08:07:06.498 INFO:tasks.workunit.client.1.smithi148.stdout:8/71: chown c16 1029232740 1 2024-03-29T08:07:06.499 INFO:tasks.workunit.client.1.smithi148.stdout:2/32: write d7/f9 [1037122,117983] 0 2024-03-29T08:07:06.500 INFO:tasks.workunit.client.1.smithi148.stdout:6/22: dwrite f0 [0,4194304] 0 2024-03-29T08:07:06.502 INFO:tasks.workunit.client.1.smithi148.stdout:1/40: dread f1 [0,4194304] 0 2024-03-29T08:07:06.507 INFO:tasks.workunit.client.1.smithi148.stdout:3/40: dwrite f1 [0,4194304] 0 2024-03-29T08:07:06.513 INFO:tasks.workunit.client.1.smithi148.stdout:8/72: creat d17/f1a x:0 0 0 2024-03-29T08:07:06.517 INFO:tasks.workunit.client.0.smithi037.stdout:9/164: dwrite d1/d4/d19/d1e/d25/d10/f12 [0,4194304] 0 2024-03-29T08:07:06.517 INFO:tasks.workunit.client.1.smithi148.stdout:6/23: rename c1 to c3 0 2024-03-29T08:07:06.522 INFO:tasks.workunit.client.0.smithi037.stdout:9/165: mknod d1/d3/c3c 0 2024-03-29T08:07:06.524 INFO:tasks.workunit.client.0.smithi037.stdout:9/166: stat d1/d4/d19/d1e/d25/f34 0 2024-03-29T08:07:06.527 INFO:tasks.workunit.client.1.smithi148.stdout:7/68: getdents d8 0 2024-03-29T08:07:06.528 INFO:tasks.workunit.client.0.smithi037.stdout:9/167: creat d1/d4/d19/d1e/d25/d10/f3d x:0 0 0 2024-03-29T08:07:06.531 INFO:tasks.workunit.client.0.smithi037.stdout:9/168: fsync d1/d4/d19/d1e/d25/d10/d23/f39 0 2024-03-29T08:07:06.533 INFO:tasks.workunit.client.0.smithi037.stdout:9/169: write d1/d1f/f33 [930187,3004] 0 2024-03-29T08:07:06.533 INFO:tasks.workunit.client.1.smithi148.stdout:1/41: write d3/f7 [50179,47734] 0 2024-03-29T08:07:06.535 INFO:tasks.workunit.client.1.smithi148.stdout:8/73: getdents d17 0 2024-03-29T08:07:06.535 INFO:tasks.workunit.client.0.smithi037.stdout:9/170: write d1/d4/d19/d1e/d25/d10/f3d [259930,37060] 0 2024-03-29T08:07:06.538 INFO:tasks.workunit.client.1.smithi148.stdout:0/61: dwrite f2 [4194304,4194304] 0 2024-03-29T08:07:06.538 INFO:tasks.workunit.client.1.smithi148.stdout:4/29: dwrite f5 [0,4194304] 0 2024-03-29T08:07:06.545 INFO:tasks.workunit.client.0.smithi037.stdout:7/155: sync 2024-03-29T08:07:06.545 INFO:tasks.workunit.client.0.smithi037.stdout:5/78: sync 2024-03-29T08:07:06.545 INFO:tasks.workunit.client.0.smithi037.stdout:6/29: sync 2024-03-29T08:07:06.546 INFO:tasks.workunit.client.1.smithi148.stdout:6/24: write f0 [1785217,124617] 0 2024-03-29T08:07:06.548 INFO:tasks.workunit.client.0.smithi037.stdout:5/79: write f3 [4283345,65768] 0 2024-03-29T08:07:06.550 INFO:tasks.workunit.client.0.smithi037.stdout:6/30: creat d0/d1/ff x:0 0 0 2024-03-29T08:07:06.551 INFO:tasks.workunit.client.0.smithi037.stdout:7/156: mknod dd/d21/c31 0 2024-03-29T08:07:06.551 INFO:tasks.workunit.client.1.smithi148.stdout:5/34: dwrite d0/f3 [0,4194304] 0 2024-03-29T08:07:06.551 INFO:tasks.workunit.client.1.smithi148.stdout:8/74: chown f10 227 1 2024-03-29T08:07:06.553 INFO:tasks.workunit.client.0.smithi037.stdout:5/80: mkdir d8/d18 0 2024-03-29T08:07:06.555 INFO:tasks.workunit.client.0.smithi037.stdout:6/31: dread - d0/d5/f6 zero size 2024-03-29T08:07:06.556 INFO:tasks.workunit.client.0.smithi037.stdout:6/32: fsync d0/f4 0 2024-03-29T08:07:06.558 INFO:tasks.workunit.client.0.smithi037.stdout:9/171: dwrite d1/d4/d19/d1e/d25/d10/f12 [0,4194304] 0 2024-03-29T08:07:06.559 INFO:tasks.workunit.client.0.smithi037.stdout:5/81: read d8/f13 [725983,24205] 0 2024-03-29T08:07:06.559 INFO:tasks.workunit.client.0.smithi037.stdout:6/33: dread - d0/d5/fb zero size 2024-03-29T08:07:06.561 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:07:06 smithi148 ceph-mon[44992]: pgmap v35: 65 pgs: 65 active+clean; 315 MiB data, 2.8 GiB used, 534 GiB / 536 GiB avail; 217 KiB/s rd, 20 MiB/s wr, 244 op/s 2024-03-29T08:07:06.561 INFO:tasks.workunit.client.1.smithi148.stdout:8/75: fsync f10 0 2024-03-29T08:07:06.561 INFO:tasks.workunit.client.1.smithi148.stdout:9/25: dwrite f3 [4194304,4194304] 0 2024-03-29T08:07:06.563 INFO:tasks.workunit.client.1.smithi148.stdout:5/35: write d0/f3 [3048570,92006] 0 2024-03-29T08:07:06.564 INFO:tasks.workunit.client.0.smithi037.stdout:9/172: chown d1/d3/c3c 1 1 2024-03-29T08:07:06.565 INFO:tasks.workunit.client.1.smithi148.stdout:6/25: fdatasync f0 0 2024-03-29T08:07:06.565 INFO:tasks.workunit.client.0.smithi037.stdout:6/34: fdatasync d0/d5/fb 0 2024-03-29T08:07:06.566 INFO:tasks.workunit.client.1.smithi148.stdout:5/36: chown d0/l4 4422367 1 2024-03-29T08:07:06.569 INFO:tasks.workunit.client.1.smithi148.stdout:9/26: symlink l8 0 2024-03-29T08:07:06.569 INFO:tasks.workunit.client.1.smithi148.stdout:3/41: dread f8 [0,4194304] 0 2024-03-29T08:07:06.570 INFO:tasks.workunit.client.0.smithi037.stdout:9/173: mknod d1/d1f/c3e 0 2024-03-29T08:07:06.570 INFO:tasks.workunit.client.1.smithi148.stdout:5/37: truncate d0/f3 5230455 0 2024-03-29T08:07:06.572 INFO:tasks.workunit.client.1.smithi148.stdout:4/30: write f4 [135785,32496] 0 2024-03-29T08:07:06.572 INFO:tasks.workunit.client.1.smithi148.stdout:2/33: dread f5 [0,4194304] 0 2024-03-29T08:07:06.576 INFO:tasks.workunit.client.1.smithi148.stdout:7/69: dread d8/f10 [0,4194304] 0 2024-03-29T08:07:06.576 INFO:tasks.workunit.client.0.smithi037.stdout:6/35: rename d0/d5/lc to d0/d5/l10 0 2024-03-29T08:07:06.577 INFO:tasks.workunit.client.1.smithi148.stdout:5/38: creat d0/f7 x:0 0 0 2024-03-29T08:07:06.579 INFO:tasks.workunit.client.1.smithi148.stdout:2/34: chown l4 10395938 1 2024-03-29T08:07:06.579 INFO:tasks.workunit.client.0.smithi037.stdout:9/174: creat d1/d4/d19/d1e/f3f x:0 0 0 2024-03-29T08:07:06.583 INFO:tasks.workunit.client.1.smithi148.stdout:7/70: chown d8/l11 54585 1 2024-03-29T08:07:06.584 INFO:tasks.workunit.client.1.smithi148.stdout:4/31: creat fa x:0 0 0 2024-03-29T08:07:06.584 INFO:tasks.workunit.client.0.smithi037.stdout:6/36: link d0/d1/c3 d0/c11 0 2024-03-29T08:07:06.584 INFO:tasks.workunit.client.1.smithi148.stdout:1/42: truncate f1 4002326 0 2024-03-29T08:07:06.585 INFO:tasks.workunit.client.1.smithi148.stdout:5/39: truncate d0/f3 5914812 0 2024-03-29T08:07:06.586 INFO:tasks.workunit.client.1.smithi148.stdout:8/76: truncate f7 2123698 0 2024-03-29T08:07:06.587 INFO:tasks.workunit.client.1.smithi148.stdout:9/27: mkdir d9 0 2024-03-29T08:07:06.589 INFO:tasks.workunit.client.0.smithi037.stdout:6/37: creat d0/d5/dd/f12 x:0 0 0 2024-03-29T08:07:06.591 INFO:tasks.workunit.client.0.smithi037.stdout:7/157: getdents dd/d21 0 2024-03-29T08:07:06.592 INFO:tasks.workunit.client.0.smithi037.stdout:6/38: write d0/d5/f9 [599957,87193] 0 2024-03-29T08:07:06.594 INFO:tasks.workunit.client.0.smithi037.stdout:6/39: chown d0/d5 0 1 2024-03-29T08:07:06.594 INFO:tasks.workunit.client.0.smithi037.stdout:5/82: truncate f3 2139593 0 2024-03-29T08:07:06.596 INFO:tasks.workunit.client.1.smithi148.stdout:4/32: dread f3 [0,4194304] 0 2024-03-29T08:07:06.597 INFO:tasks.workunit.client.0.smithi037.stdout:9/175: truncate d1/d4/d19/d1e/d25/f34 1345009 0 2024-03-29T08:07:06.597 INFO:tasks.workunit.client.0.smithi037.stdout:6/40: chown d0/c11 210856 1 2024-03-29T08:07:06.599 INFO:tasks.workunit.client.1.smithi148.stdout:7/71: rename f2 to d8/f12 0 2024-03-29T08:07:06.601 INFO:tasks.workunit.client.0.smithi037.stdout:5/83: creat d8/d18/f19 x:0 0 0 2024-03-29T08:07:06.601 INFO:tasks.workunit.client.1.smithi148.stdout:0/62: dwrite db/f10 [0,4194304] 0 2024-03-29T08:07:06.602 INFO:tasks.workunit.client.1.smithi148.stdout:3/42: dread f8 [0,4194304] 0 2024-03-29T08:07:06.602 INFO:tasks.workunit.client.1.smithi148.stdout:6/26: dwrite f0 [0,4194304] 0 2024-03-29T08:07:06.603 INFO:tasks.workunit.client.0.smithi037.stdout:9/176: stat d1/d4/d19/d1e/d25/d10/d23/l2c 0 2024-03-29T08:07:06.604 INFO:tasks.workunit.client.1.smithi148.stdout:9/28: rmdir d9 0 2024-03-29T08:07:06.606 INFO:tasks.workunit.client.1.smithi148.stdout:4/33: symlink lb 0 2024-03-29T08:07:06.606 INFO:tasks.workunit.client.0.smithi037.stdout:6/41: unlink d0/d5/l10 0 2024-03-29T08:07:06.607 INFO:tasks.workunit.client.1.smithi148.stdout:5/40: dread d0/f3 [0,4194304] 0 2024-03-29T08:07:06.609 INFO:tasks.workunit.client.0.smithi037.stdout:7/158: dread f5 [0,4194304] 0 2024-03-29T08:07:06.610 INFO:tasks.workunit.client.0.smithi037.stdout:9/177: unlink d1/d4/c8 0 2024-03-29T08:07:06.610 INFO:tasks.workunit.client.1.smithi148.stdout:3/43: mknod cd 0 2024-03-29T08:07:06.611 INFO:tasks.workunit.client.0.smithi037.stdout:6/42: rmdir d0/d1 39 2024-03-29T08:07:06.611 INFO:tasks.workunit.client.1.smithi148.stdout:3/44: fsync f1 0 2024-03-29T08:07:06.612 INFO:tasks.workunit.client.1.smithi148.stdout:3/45: stat ca 0 2024-03-29T08:07:06.612 INFO:tasks.workunit.client.1.smithi148.stdout:0/63: symlink db/dc/l15 0 2024-03-29T08:07:06.615 INFO:tasks.workunit.client.0.smithi037.stdout:5/84: getdents d8 0 2024-03-29T08:07:06.620 INFO:tasks.workunit.client.0.smithi037.stdout:6/43: write d0/f4 [324483,39544] 0 2024-03-29T08:07:06.620 INFO:tasks.workunit.client.0.smithi037.stdout:7/159: truncate f7 1584510 0 2024-03-29T08:07:06.620 INFO:tasks.workunit.client.0.smithi037.stdout:9/178: chown d1/d4/d19/d1e/d25/d10/d23/f37 878929 1 2024-03-29T08:07:06.621 INFO:tasks.workunit.client.0.smithi037.stdout:5/85: mknod d8/d18/c1a 0 2024-03-29T08:07:06.622 INFO:tasks.workunit.client.1.smithi148.stdout:6/27: rename f0 to f4 0 2024-03-29T08:07:06.623 INFO:tasks.workunit.client.0.smithi037.stdout:5/86: chown l6 4035 1 2024-03-29T08:07:06.624 INFO:tasks.workunit.client.1.smithi148.stdout:4/34: chown l9 2734 1 2024-03-29T08:07:06.624 INFO:tasks.workunit.client.0.smithi037.stdout:7/160: rmdir dd/d21 39 2024-03-29T08:07:06.628 INFO:tasks.workunit.client.1.smithi148.stdout:5/41: symlink d0/l8 0 2024-03-29T08:07:06.629 INFO:tasks.workunit.client.0.smithi037.stdout:6/44: mknod d0/c13 0 2024-03-29T08:07:06.629 INFO:tasks.workunit.client.0.smithi037.stdout:9/179: readlink d1/d4/d19/d1e/d25/l36 0 2024-03-29T08:07:06.630 INFO:tasks.workunit.client.1.smithi148.stdout:3/46: truncate f5 648242 0 2024-03-29T08:07:06.631 INFO:tasks.workunit.client.1.smithi148.stdout:5/42: write d0/f7 [7559,28524] 0 2024-03-29T08:07:06.634 INFO:tasks.workunit.client.1.smithi148.stdout:9/29: symlink la 0 2024-03-29T08:07:06.636 INFO:tasks.workunit.client.1.smithi148.stdout:9/30: readlink l8 0 2024-03-29T08:07:06.637 INFO:tasks.workunit.client.1.smithi148.stdout:3/47: mkdir de 0 2024-03-29T08:07:06.637 INFO:tasks.workunit.client.0.smithi037.stdout:6/45: creat d0/d5/f14 x:0 0 0 2024-03-29T08:07:06.639 INFO:tasks.workunit.client.0.smithi037.stdout:7/161: creat dd/d21/f32 x:0 0 0 2024-03-29T08:07:06.640 INFO:tasks.workunit.client.1.smithi148.stdout:0/64: creat db/de/df/d11/f16 x:0 0 0 2024-03-29T08:07:06.640 INFO:tasks.workunit.client.1.smithi148.stdout:9/31: dread - f7 zero size 2024-03-29T08:07:06.640 INFO:tasks.workunit.client.1.smithi148.stdout:1/43: dwrite f1 [0,4194304] 0 2024-03-29T08:07:06.642 INFO:tasks.workunit.client.1.smithi148.stdout:7/72: write f6 [1907321,11201] 0 2024-03-29T08:07:06.643 INFO:tasks.workunit.client.0.smithi037.stdout:9/180: rename d1/d4/d19/d1e/d25/c7 to d1/d4/d19/d1e/d25/c40 0 2024-03-29T08:07:06.644 INFO:tasks.workunit.client.0.smithi037.stdout:5/87: link l6 d8/l1b 0 2024-03-29T08:07:06.644 INFO:tasks.workunit.client.1.smithi148.stdout:3/48: getdents de 0 2024-03-29T08:07:06.644 INFO:tasks.workunit.client.1.smithi148.stdout:6/28: dread f4 [0,4194304] 0 2024-03-29T08:07:06.646 INFO:tasks.workunit.client.1.smithi148.stdout:3/49: chown c7 449890 1 2024-03-29T08:07:06.646 INFO:tasks.workunit.client.1.smithi148.stdout:1/44: mknod d3/cd 0 2024-03-29T08:07:06.646 INFO:tasks.workunit.client.0.smithi037.stdout:6/46: symlink d0/l15 0 2024-03-29T08:07:06.647 INFO:tasks.workunit.client.0.smithi037.stdout:9/181: creat d1/d4/f41 x:0 0 0 2024-03-29T08:07:06.648 INFO:tasks.workunit.client.1.smithi148.stdout:9/32: mkdir db 0 2024-03-29T08:07:06.648 INFO:tasks.workunit.client.1.smithi148.stdout:1/45: chown d3 24482282 1 2024-03-29T08:07:06.649 INFO:tasks.workunit.client.0.smithi037.stdout:6/47: read - d0/d5/f14 zero size 2024-03-29T08:07:06.651 INFO:tasks.workunit.client.1.smithi148.stdout:7/73: creat d8/f13 x:0 0 0 2024-03-29T08:07:06.652 INFO:tasks.workunit.client.1.smithi148.stdout:6/29: creat f5 x:0 0 0 2024-03-29T08:07:06.652 INFO:tasks.workunit.client.1.smithi148.stdout:6/30: rmdir - no directory 2024-03-29T08:07:06.653 INFO:tasks.workunit.client.1.smithi148.stdout:3/50: write f8 [4034667,81298] 0 2024-03-29T08:07:06.654 INFO:tasks.workunit.client.1.smithi148.stdout:1/46: mkdir d3/de 0 2024-03-29T08:07:06.655 INFO:tasks.workunit.client.0.smithi037.stdout:9/182: symlink d1/d4/d19/d1e/d25/d10/d31/l42 0 2024-03-29T08:07:06.657 INFO:tasks.workunit.client.1.smithi148.stdout:0/65: getdents db/de/df 0 2024-03-29T08:07:06.658 INFO:tasks.workunit.client.0.smithi037.stdout:6/48: dread d0/d5/fa [0,4194304] 0 2024-03-29T08:07:06.659 INFO:tasks.workunit.client.1.smithi148.stdout:7/74: mkdir d8/d14 0 2024-03-29T08:07:06.661 INFO:tasks.workunit.client.1.smithi148.stdout:9/33: rename c2 to db/cc 0 2024-03-29T08:07:06.662 INFO:tasks.workunit.client.1.smithi148.stdout:6/31: write f5 [69130,26566] 0 2024-03-29T08:07:06.664 INFO:tasks.workunit.client.1.smithi148.stdout:1/47: write d3/fa [3334530,39447] 0 2024-03-29T08:07:06.668 INFO:tasks.workunit.client.0.smithi037.stdout:9/183: write d1/d4/d19/d1e/d25/d10/d13/f17 [436867,129666] 0 2024-03-29T08:07:06.668 INFO:tasks.workunit.client.0.smithi037.stdout:5/88: write f3 [1044323,25728] 0 2024-03-29T08:07:06.670 INFO:tasks.workunit.client.1.smithi148.stdout:6/32: mkdir d6 0 2024-03-29T08:07:06.671 INFO:tasks.workunit.client.0.smithi037.stdout:5/89: dread - d8/f17 zero size 2024-03-29T08:07:06.671 INFO:tasks.workunit.client.1.smithi148.stdout:1/48: read f2 [400910,129733] 0 2024-03-29T08:07:06.673 INFO:tasks.workunit.client.0.smithi037.stdout:5/90: stat d8/fa 0 2024-03-29T08:07:06.673 INFO:tasks.workunit.client.1.smithi148.stdout:5/43: dwrite d0/f7 [0,4194304] 0 2024-03-29T08:07:06.674 INFO:tasks.workunit.client.1.smithi148.stdout:7/75: creat d8/f15 x:0 0 0 2024-03-29T08:07:06.677 INFO:tasks.workunit.client.1.smithi148.stdout:4/35: dwrite f3 [0,4194304] 0 2024-03-29T08:07:06.678 INFO:tasks.workunit.client.0.smithi037.stdout:5/91: link d8/lb d8/l1c 0 2024-03-29T08:07:06.680 INFO:tasks.workunit.client.0.smithi037.stdout:5/92: fsync d8/d18/f19 0 2024-03-29T08:07:06.683 INFO:tasks.workunit.client.1.smithi148.stdout:2/35: sync 2024-03-29T08:07:06.684 INFO:tasks.workunit.client.0.smithi037.stdout:6/49: dwrite d0/d5/f14 [0,4194304] 0 2024-03-29T08:07:06.685 INFO:tasks.workunit.client.0.smithi037.stdout:5/93: mknod d8/c1d 0 2024-03-29T08:07:06.685 INFO:tasks.workunit.client.0.smithi037.stdout:6/50: chown d0/d5/f6 207209973 1 2024-03-29T08:07:06.687 INFO:tasks.workunit.client.1.smithi148.stdout:5/44: truncate d0/f7 4627892 0 2024-03-29T08:07:06.688 INFO:tasks.workunit.client.1.smithi148.stdout:4/36: creat fc x:0 0 0 2024-03-29T08:07:06.688 INFO:tasks.workunit.client.1.smithi148.stdout:7/76: creat d8/d14/f16 x:0 0 0 2024-03-29T08:07:06.689 INFO:tasks.workunit.client.1.smithi148.stdout:2/36: mkdir d7/da 0 2024-03-29T08:07:06.694 INFO:tasks.workunit.client.1.smithi148.stdout:4/37: chown c8 3 1 2024-03-29T08:07:06.695 INFO:tasks.workunit.client.1.smithi148.stdout:1/49: getdents d3/de 0 2024-03-29T08:07:06.698 INFO:tasks.workunit.client.1.smithi148.stdout:7/77: creat d8/f17 x:0 0 0 2024-03-29T08:07:06.699 INFO:tasks.workunit.client.0.smithi037.stdout:6/51: link d0/d5/fa d0/d5/f16 0 2024-03-29T08:07:06.701 INFO:tasks.workunit.client.1.smithi148.stdout:2/37: mknod d7/cb 0 2024-03-29T08:07:06.702 INFO:tasks.workunit.client.0.smithi037.stdout:5/94: truncate d8/f13 594272 0 2024-03-29T08:07:06.702 INFO:tasks.workunit.client.0.smithi037.stdout:1/101: sync 2024-03-29T08:07:06.703 INFO:tasks.workunit.client.1.smithi148.stdout:4/38: dread f4 [0,4194304] 0 2024-03-29T08:07:06.704 INFO:tasks.workunit.client.0.smithi037.stdout:6/52: fsync d0/d5/dd/f12 0 2024-03-29T08:07:06.705 INFO:tasks.workunit.client.1.smithi148.stdout:1/50: symlink d3/lf 0 2024-03-29T08:07:06.706 INFO:tasks.workunit.client.0.smithi037.stdout:9/184: dwrite d1/d4/f41 [0,4194304] 0 2024-03-29T08:07:06.713 INFO:tasks.workunit.client.0.smithi037.stdout:6/53: fsync d0/d5/f16 0 2024-03-29T08:07:06.714 INFO:tasks.workunit.client.1.smithi148.stdout:0/66: dwrite fa [0,4194304] 0 2024-03-29T08:07:06.717 INFO:tasks.workunit.client.1.smithi148.stdout:7/78: creat d8/f18 x:0 0 0 2024-03-29T08:07:06.719 INFO:tasks.workunit.client.1.smithi148.stdout:1/51: rename d3 to d3/d10 22 2024-03-29T08:07:06.719 INFO:tasks.workunit.client.0.smithi037.stdout:6/54: rename d0/d5/f6 to d0/d1/f17 0 2024-03-29T08:07:06.721 INFO:tasks.workunit.client.1.smithi148.stdout:6/33: dread f4 [0,4194304] 0 2024-03-29T08:07:06.724 INFO:tasks.workunit.client.1.smithi148.stdout:8/77: sync 2024-03-29T08:07:06.725 INFO:tasks.workunit.client.0.smithi037.stdout:6/55: mknod d0/d5/dd/c18 0 2024-03-29T08:07:06.725 INFO:tasks.workunit.client.1.smithi148.stdout:9/34: dwrite f3 [4194304,4194304] 0 2024-03-29T08:07:06.725 INFO:tasks.workunit.client.1.smithi148.stdout:2/38: rename d7/f8 to d7/fc 0 2024-03-29T08:07:06.726 INFO:tasks.workunit.client.1.smithi148.stdout:3/51: dwrite f5 [0,4194304] 0 2024-03-29T08:07:06.728 INFO:tasks.workunit.client.1.smithi148.stdout:9/35: dread - f7 zero size 2024-03-29T08:07:06.730 INFO:tasks.workunit.client.0.smithi037.stdout:6/56: mkdir d0/d19 0 2024-03-29T08:07:06.730 INFO:tasks.workunit.client.1.smithi148.stdout:1/52: symlink d3/l11 0 2024-03-29T08:07:06.730 INFO:tasks.workunit.client.1.smithi148.stdout:8/78: truncate f14 57287 0 2024-03-29T08:07:06.733 INFO:tasks.workunit.client.1.smithi148.stdout:6/34: truncate f5 293602 0 2024-03-29T08:07:06.736 INFO:tasks.workunit.client.0.smithi037.stdout:5/95: write d8/f13 [1404177,54771] 0 2024-03-29T08:07:06.736 INFO:tasks.workunit.client.1.smithi148.stdout:5/45: dread d0/f7 [0,4194304] 0 2024-03-29T08:07:06.738 INFO:tasks.workunit.client.1.smithi148.stdout:2/39: fsync f5 0 2024-03-29T08:07:06.743 INFO:tasks.workunit.client.1.smithi148.stdout:3/52: symlink de/lf 0 2024-03-29T08:07:06.743 INFO:tasks.workunit.client.1.smithi148.stdout:4/39: getdents . 0 2024-03-29T08:07:06.744 INFO:tasks.workunit.client.1.smithi148.stdout:4/40: dread - fa zero size 2024-03-29T08:07:06.745 INFO:tasks.workunit.client.0.smithi037.stdout:6/57: mkdir d0/d5/d1a 0 2024-03-29T08:07:06.745 INFO:tasks.workunit.client.0.smithi037.stdout:5/96: chown d8/f13 2879 1 2024-03-29T08:07:06.745 INFO:tasks.workunit.client.0.smithi037.stdout:1/102: truncate d2/d6/dc/f15 418225 0 2024-03-29T08:07:06.745 INFO:tasks.workunit.client.1.smithi148.stdout:0/67: creat db/de/df/d14/f17 x:0 0 0 2024-03-29T08:07:06.748 INFO:tasks.workunit.client.0.smithi037.stdout:5/97: fdatasync d8/fa 0 2024-03-29T08:07:06.750 INFO:tasks.workunit.client.1.smithi148.stdout:1/53: rename f2 to d3/f12 0 2024-03-29T08:07:06.750 INFO:tasks.workunit.client.0.smithi037.stdout:9/185: dwrite d1/d4/f3b [0,4194304] 0 2024-03-29T08:07:06.754 INFO:tasks.workunit.client.1.smithi148.stdout:2/40: mknod d7/cd 0 2024-03-29T08:07:06.759 INFO:tasks.workunit.client.0.smithi037.stdout:9/186: truncate d1/d4/d19/d1e/d25/d10/f3d 335757 0 2024-03-29T08:07:06.759 INFO:tasks.workunit.client.0.smithi037.stdout:6/58: creat d0/d1/f1b x:0 0 0 2024-03-29T08:07:06.761 INFO:tasks.workunit.client.0.smithi037.stdout:1/103: write d2/d6/dc/d18/d1d/d21/f20 [715967,75627] 0 2024-03-29T08:07:06.765 INFO:tasks.workunit.client.0.smithi037.stdout:9/187: chown d1/d4/d19/d1e/d25/d16/c1a 4381 1 2024-03-29T08:07:06.767 INFO:tasks.workunit.client.0.smithi037.stdout:1/104: chown d2/d6/dc/d18/d1d 7 1 2024-03-29T08:07:06.769 INFO:tasks.workunit.client.0.smithi037.stdout:9/188: unlink d1/c2 0 2024-03-29T08:07:06.770 INFO:tasks.workunit.client.0.smithi037.stdout:5/98: dread d8/fa [0,4194304] 0 2024-03-29T08:07:06.776 INFO:tasks.workunit.client.0.smithi037.stdout:9/189: write d1/d4/d19/d1e/d25/d10/d13/f17 [1531953,84048] 0 2024-03-29T08:07:06.780 INFO:tasks.workunit.client.0.smithi037.stdout:5/99: creat d8/f1e x:0 0 0 2024-03-29T08:07:06.780 INFO:tasks.workunit.client.0.smithi037.stdout:1/105: rename d2/l16 to d2/d6/dc/l24 0 2024-03-29T08:07:06.780 INFO:tasks.workunit.client.1.smithi148.stdout:7/79: dread d8/fc [0,4194304] 0 2024-03-29T08:07:06.781 INFO:tasks.workunit.client.0.smithi037.stdout:9/190: stat d1/l5 0 2024-03-29T08:07:06.781 INFO:tasks.workunit.client.1.smithi148.stdout:5/46: rename d0/f7 to d0/f9 0 2024-03-29T08:07:06.781 INFO:tasks.workunit.client.1.smithi148.stdout:6/35: rename f4 to d6/f7 0 2024-03-29T08:07:06.782 INFO:tasks.workunit.client.0.smithi037.stdout:5/100: chown d8/l14 1671829726 1 2024-03-29T08:07:06.783 INFO:tasks.workunit.client.1.smithi148.stdout:1/54: mkdir d3/d13 0 2024-03-29T08:07:06.784 INFO:tasks.workunit.client.0.smithi037.stdout:9/191: fsync d1/d4/f3b 0 2024-03-29T08:07:06.785 INFO:tasks.workunit.client.0.smithi037.stdout:1/106: fdatasync d2/d6/dc/f19 0 2024-03-29T08:07:06.786 INFO:tasks.workunit.client.1.smithi148.stdout:8/79: dread fe [0,4194304] 0 2024-03-29T08:07:06.787 INFO:tasks.workunit.client.1.smithi148.stdout:5/47: chown d0/l4 196 1 2024-03-29T08:07:06.788 INFO:tasks.workunit.client.0.smithi037.stdout:5/101: chown d8/l1b 211198 1 2024-03-29T08:07:06.793 INFO:tasks.workunit.client.1.smithi148.stdout:9/36: dwrite f7 [0,4194304] 0 2024-03-29T08:07:06.794 INFO:tasks.workunit.client.0.smithi037.stdout:9/192: readlink d1/d4/d19/d1e/d25/d10/d31/l42 0 2024-03-29T08:07:06.794 INFO:tasks.workunit.client.1.smithi148.stdout:4/41: dread f1 [0,4194304] 0 2024-03-29T08:07:06.796 INFO:tasks.workunit.client.0.smithi037.stdout:5/102: stat d8/d18/f19 0 2024-03-29T08:07:06.796 INFO:tasks.workunit.client.0.smithi037.stdout:1/107: creat d2/d6/dc/f25 x:0 0 0 2024-03-29T08:07:06.797 INFO:tasks.workunit.client.1.smithi148.stdout:3/53: dwrite f5 [0,4194304] 0 2024-03-29T08:07:06.803 INFO:tasks.workunit.client.0.smithi037.stdout:6/59: dwrite d0/d1/ff [0,4194304] 0 2024-03-29T08:07:06.803 INFO:tasks.workunit.client.0.smithi037.stdout:1/108: chown d2/d6/cb 575 1 2024-03-29T08:07:06.809 INFO:tasks.workunit.client.0.smithi037.stdout:5/103: creat d8/f1f x:0 0 0 2024-03-29T08:07:06.810 INFO:tasks.workunit.client.0.smithi037.stdout:9/193: unlink d1/d4/d19/d1e/d25/l36 0 2024-03-29T08:07:06.810 INFO:tasks.workunit.client.0.smithi037.stdout:5/104: truncate d8/f1e 1016350 0 2024-03-29T08:07:06.812 INFO:tasks.workunit.client.0.smithi037.stdout:1/109: rename d2/d6/dc/d18/c1b to d2/c26 0 2024-03-29T08:07:06.818 INFO:tasks.workunit.client.1.smithi148.stdout:6/36: dwrite f5 [0,4194304] 0 2024-03-29T08:07:06.822 INFO:tasks.workunit.client.0.smithi037.stdout:6/60: write d0/d5/f16 [1071277,28533] 0 2024-03-29T08:07:06.824 INFO:tasks.workunit.client.1.smithi148.stdout:1/55: dwrite d3/f7 [0,4194304] 0 2024-03-29T08:07:06.829 INFO:tasks.workunit.client.1.smithi148.stdout:4/42: dwrite fc [0,4194304] 0 2024-03-29T08:07:06.831 INFO:tasks.workunit.client.0.smithi037.stdout:5/105: getdents d8 0 2024-03-29T08:07:06.838 INFO:tasks.workunit.client.0.smithi037.stdout:5/106: mknod d8/c20 0 2024-03-29T08:07:06.838 INFO:tasks.workunit.client.0.smithi037.stdout:6/61: dread d0/d5/f9 [0,4194304] 0 2024-03-29T08:07:06.846 INFO:tasks.workunit.client.0.smithi037.stdout:5/107: rename d8/f13 to d8/d18/f21 0 2024-03-29T08:07:06.874 INFO:tasks.workunit.client.0.smithi037.stdout:1/110: dwrite d2/d6/dc/f15 [0,4194304] 0 2024-03-29T08:07:06.884 INFO:tasks.workunit.client.0.smithi037.stdout:6/62: dwrite d0/d5/f16 [0,4194304] 0 2024-03-29T08:07:06.885 INFO:tasks.workunit.client.0.smithi037.stdout:1/111: rename d2/d6/l7 to d2/d6/l27 0 2024-03-29T08:07:06.889 INFO:tasks.workunit.client.0.smithi037.stdout:5/108: dwrite d8/f1e [0,4194304] 0 2024-03-29T08:07:06.899 INFO:tasks.workunit.client.0.smithi037.stdout:6/63: rmdir d0/d5/dd 39 2024-03-29T08:07:06.902 INFO:tasks.workunit.client.0.smithi037.stdout:6/64: read - d0/d1/f1b zero size 2024-03-29T08:07:06.909 INFO:tasks.workunit.client.0.smithi037.stdout:6/65: rename d0 to d0/d5/dd/d1c 22 2024-03-29T08:07:06.916 INFO:tasks.workunit.client.0.smithi037.stdout:6/66: unlink d0/c13 0 2024-03-29T08:07:06.921 INFO:tasks.workunit.client.0.smithi037.stdout:1/112: dwrite d2/d6/dc/f15 [0,4194304] 0 2024-03-29T08:07:06.926 INFO:tasks.workunit.client.0.smithi037.stdout:5/109: dwrite d8/fa [0,4194304] 0 2024-03-29T08:07:06.930 INFO:tasks.workunit.client.0.smithi037.stdout:5/110: mkdir d8/d22 0 2024-03-29T08:07:06.941 INFO:tasks.workunit.client.0.smithi037.stdout:2/69: sync 2024-03-29T08:07:06.941 INFO:tasks.workunit.client.0.smithi037.stdout:3/99: sync 2024-03-29T08:07:06.941 INFO:tasks.workunit.client.1.smithi148.stdout:6/37: sync 2024-03-29T08:07:06.946 INFO:tasks.workunit.client.0.smithi037.stdout:6/67: dwrite d0/d1/f17 [0,4194304] 0 2024-03-29T08:07:06.946 INFO:tasks.workunit.client.0.smithi037.stdout:3/100: chown dd/d15/f16 49 1 2024-03-29T08:07:06.952 INFO:tasks.workunit.client.0.smithi037.stdout:3/101: chown f4 18814991 1 2024-03-29T08:07:06.954 INFO:tasks.workunit.client.0.smithi037.stdout:1/113: dread d2/f5 [0,4194304] 0 2024-03-29T08:07:06.956 INFO:tasks.workunit.client.0.smithi037.stdout:5/111: dread d8/fa [0,4194304] 0 2024-03-29T08:07:06.965 INFO:tasks.workunit.client.1.smithi148.stdout:0/68: rename db/de/df/d14/f17 to db/f18 0 2024-03-29T08:07:06.965 INFO:tasks.workunit.client.1.smithi148.stdout:7/80: mkdir d8/d14/d19 0 2024-03-29T08:07:06.966 INFO:tasks.workunit.client.0.smithi037.stdout:1/114: write d2/d6/dc/f25 [907255,7675] 0 2024-03-29T08:07:06.968 INFO:tasks.workunit.client.1.smithi148.stdout:9/37: chown c5 3 1 2024-03-29T08:07:06.968 INFO:tasks.workunit.client.1.smithi148.stdout:3/54: rmdir de 39 2024-03-29T08:07:06.974 INFO:tasks.workunit.client.1.smithi148.stdout:4/43: rename f5 to fd 0 2024-03-29T08:07:06.975 INFO:tasks.workunit.client.1.smithi148.stdout:6/38: mkdir d6/d8 0 2024-03-29T08:07:06.977 INFO:tasks.workunit.client.1.smithi148.stdout:2/41: creat d7/da/fe x:0 0 0 2024-03-29T08:07:06.980 INFO:tasks.workunit.client.1.smithi148.stdout:8/80: rename lc to d17/l1b 0 2024-03-29T08:07:06.980 INFO:tasks.workunit.client.1.smithi148.stdout:2/42: read f1 [2702390,102069] 0 2024-03-29T08:07:06.983 INFO:tasks.workunit.client.0.smithi037.stdout:1/115: creat d2/d6/d13/f28 x:0 0 0 2024-03-29T08:07:06.985 INFO:tasks.workunit.client.1.smithi148.stdout:9/38: creat db/fd x:0 0 0 2024-03-29T08:07:06.986 INFO:tasks.workunit.client.0.smithi037.stdout:2/70: dwrite d0/fa [0,4194304] 0 2024-03-29T08:07:06.987 INFO:tasks.workunit.client.0.smithi037.stdout:1/116: chown d2 20622 1 2024-03-29T08:07:06.988 INFO:tasks.workunit.client.1.smithi148.stdout:2/43: write f1 [597965,6692] 0 2024-03-29T08:07:06.989 INFO:tasks.workunit.client.1.smithi148.stdout:7/81: creat d8/d14/d19/f1a x:0 0 0 2024-03-29T08:07:06.994 INFO:tasks.workunit.client.1.smithi148.stdout:3/55: mknod de/c10 0 2024-03-29T08:07:06.995 INFO:tasks.workunit.client.0.smithi037.stdout:6/68: dwrite d0/d1/f1b [0,4194304] 0 2024-03-29T08:07:06.997 INFO:tasks.workunit.client.1.smithi148.stdout:9/39: unlink l8 0 2024-03-29T08:07:06.997 INFO:tasks.workunit.client.1.smithi148.stdout:5/48: dwrite d0/f9 [0,4194304] 0 2024-03-29T08:07:07.005 INFO:tasks.workunit.client.1.smithi148.stdout:2/44: creat d7/ff x:0 0 0 2024-03-29T08:07:07.009 INFO:tasks.workunit.client.0.smithi037.stdout:6/69: write d0/d5/f9 [1468879,2284] 0 2024-03-29T08:07:07.011 INFO:tasks.workunit.client.1.smithi148.stdout:5/49: chown d0/f9 852335508 1 2024-03-29T08:07:07.013 INFO:tasks.workunit.client.0.smithi037.stdout:2/71: link d0/c11 d0/c16 0 2024-03-29T08:07:07.013 INFO:tasks.workunit.client.0.smithi037.stdout:5/112: dwrite d8/f17 [0,4194304] 0 2024-03-29T08:07:07.013 INFO:tasks.workunit.client.0.smithi037.stdout:6/70: fsync d0/d5/fa 0 2024-03-29T08:07:07.015 INFO:tasks.workunit.client.0.smithi037.stdout:3/102: dwrite fc [0,4194304] 0 2024-03-29T08:07:07.017 INFO:tasks.workunit.client.0.smithi037.stdout:2/72: fsync d0/fc 0 2024-03-29T08:07:07.017 INFO:tasks.workunit.client.1.smithi148.stdout:6/39: dread d6/f7 [0,4194304] 0 2024-03-29T08:07:07.020 INFO:tasks.workunit.client.1.smithi148.stdout:2/45: write d7/f9 [604051,1238] 0 2024-03-29T08:07:07.020 INFO:tasks.workunit.client.1.smithi148.stdout:1/56: dwrite d3/fa [4194304,4194304] 0 2024-03-29T08:07:07.022 INFO:tasks.workunit.client.1.smithi148.stdout:2/46: chown d7/fc 3 1 2024-03-29T08:07:07.023 INFO:tasks.workunit.client.0.smithi037.stdout:5/113: write d8/d18/f19 [703129,40872] 0 2024-03-29T08:07:07.024 INFO:tasks.workunit.client.1.smithi148.stdout:0/69: sync 2024-03-29T08:07:07.028 INFO:tasks.workunit.client.0.smithi037.stdout:2/73: mknod d0/c17 0 2024-03-29T08:07:07.029 INFO:tasks.workunit.client.1.smithi148.stdout:4/44: dwrite fa [0,4194304] 0 2024-03-29T08:07:07.031 INFO:tasks.workunit.client.0.smithi037.stdout:6/71: write d0/d5/fa [191942,83704] 0 2024-03-29T08:07:07.033 INFO:tasks.workunit.client.1.smithi148.stdout:2/47: read - d7/ff zero size 2024-03-29T08:07:07.034 INFO:tasks.workunit.client.0.smithi037.stdout:3/103: mknod dd/d15/c26 0 2024-03-29T08:07:07.036 INFO:tasks.workunit.client.1.smithi148.stdout:0/70: truncate f7 5004022 0 2024-03-29T08:07:07.037 INFO:tasks.workunit.client.1.smithi148.stdout:8/81: dwrite fe [0,4194304] 0 2024-03-29T08:07:07.040 INFO:tasks.workunit.client.1.smithi148.stdout:0/71: fsync db/dc/f12 0 2024-03-29T08:07:07.045 INFO:tasks.workunit.client.1.smithi148.stdout:8/82: write d17/f1a [993869,1795] 0 2024-03-29T08:07:07.046 INFO:tasks.workunit.client.0.smithi037.stdout:3/104: fdatasync dd/d15/f16 0 2024-03-29T08:07:07.047 INFO:tasks.workunit.client.1.smithi148.stdout:7/82: dread d8/f10 [0,4194304] 0 2024-03-29T08:07:07.048 INFO:tasks.workunit.client.1.smithi148.stdout:0/72: chown f7 27 1 2024-03-29T08:07:07.048 INFO:tasks.workunit.client.1.smithi148.stdout:9/40: dread f3 [0,4194304] 0 2024-03-29T08:07:07.049 INFO:tasks.workunit.client.1.smithi148.stdout:3/56: creat de/f11 x:0 0 0 2024-03-29T08:07:07.050 INFO:tasks.workunit.client.0.smithi037.stdout:2/74: rename d0/fc to d0/f18 0 2024-03-29T08:07:07.052 INFO:tasks.workunit.client.0.smithi037.stdout:1/117: dwrite d2/d6/dc/fd [0,4194304] 0 2024-03-29T08:07:07.052 INFO:tasks.workunit.client.0.smithi037.stdout:6/72: symlink d0/d19/l1d 0 2024-03-29T08:07:07.061 INFO:tasks.workunit.client.1.smithi148.stdout:9/41: write f7 [745932,89809] 0 2024-03-29T08:07:07.061 INFO:tasks.workunit.client.1.smithi148.stdout:5/50: dwrite d0/f9 [0,4194304] 0 2024-03-29T08:07:07.061 INFO:tasks.workunit.client.0.smithi037.stdout:6/73: chown d0/d1/ff 233343934 1 2024-03-29T08:07:07.061 INFO:tasks.workunit.client.0.smithi037.stdout:5/114: getdents d8 0 2024-03-29T08:07:07.061 INFO:tasks.workunit.client.0.smithi037.stdout:3/105: fsync dd/ff 0 2024-03-29T08:07:07.061 INFO:tasks.workunit.client.0.smithi037.stdout:5/115: dread - d8/f1f zero size 2024-03-29T08:07:07.063 INFO:tasks.workunit.client.0.smithi037.stdout:8/102: sync 2024-03-29T08:07:07.067 INFO:tasks.workunit.client.1.smithi148.stdout:4/45: symlink le 0 2024-03-29T08:07:07.069 INFO:tasks.workunit.client.0.smithi037.stdout:8/103: write d5/f16 [655151,92821] 0 2024-03-29T08:07:07.073 INFO:tasks.workunit.client.1.smithi148.stdout:2/48: rename d7/ff to d7/f10 0 2024-03-29T08:07:07.074 INFO:tasks.workunit.client.1.smithi148.stdout:2/49: readlink l6 0 2024-03-29T08:07:07.076 INFO:tasks.workunit.client.1.smithi148.stdout:8/83: rename f10 to d17/d18/f1c 0 2024-03-29T08:07:07.076 INFO:tasks.workunit.client.1.smithi148.stdout:4/46: write fc [1438127,123722] 0 2024-03-29T08:07:07.076 INFO:tasks.workunit.client.1.smithi148.stdout:6/40: sync 2024-03-29T08:07:07.078 INFO:tasks.workunit.client.0.smithi037.stdout:1/118: truncate d2/d6/dc/f19 1610918 0 2024-03-29T08:07:07.079 INFO:tasks.workunit.client.0.smithi037.stdout:3/106: mkdir dd/d15/d27 0 2024-03-29T08:07:07.080 INFO:tasks.workunit.client.0.smithi037.stdout:5/116: symlink d8/d22/l23 0 2024-03-29T08:07:07.081 INFO:tasks.workunit.client.1.smithi148.stdout:3/57: symlink de/l12 0 2024-03-29T08:07:07.081 INFO:tasks.workunit.client.0.smithi037.stdout:1/119: fdatasync d2/d6/d13/f28 0 2024-03-29T08:07:07.084 INFO:tasks.workunit.client.1.smithi148.stdout:7/83: creat d8/d14/f1b x:0 0 0 2024-03-29T08:07:07.084 INFO:tasks.workunit.client.0.smithi037.stdout:5/117: chown d8/ff 178 1 2024-03-29T08:07:07.087 INFO:tasks.workunit.client.1.smithi148.stdout:7/84: fsync d8/f18 0 2024-03-29T08:07:07.087 INFO:tasks.workunit.client.1.smithi148.stdout:5/51: dwrite d0/f9 [0,4194304] 0 2024-03-29T08:07:07.088 INFO:tasks.workunit.client.0.smithi037.stdout:1/120: creat d2/d6/dc/d18/f29 x:0 0 0 2024-03-29T08:07:07.088 INFO:tasks.workunit.client.0.smithi037.stdout:3/107: write dd/fe [995530,94876] 0 2024-03-29T08:07:07.091 INFO:tasks.workunit.client.0.smithi037.stdout:6/74: dwrite d0/d5/fa [0,4194304] 0 2024-03-29T08:07:07.092 INFO:tasks.workunit.client.0.smithi037.stdout:3/108: mknod dd/d15/c28 0 2024-03-29T08:07:07.101 INFO:tasks.workunit.client.1.smithi148.stdout:4/47: mkdir df 0 2024-03-29T08:07:07.102 INFO:tasks.workunit.client.1.smithi148.stdout:8/84: dread fe [0,4194304] 0 2024-03-29T08:07:07.102 INFO:tasks.workunit.client.1.smithi148.stdout:2/50: mknod d7/da/c11 0 2024-03-29T08:07:07.103 INFO:tasks.workunit.client.0.smithi037.stdout:3/109: symlink dd/d15/d27/l29 0 2024-03-29T08:07:07.104 INFO:tasks.workunit.client.0.smithi037.stdout:1/121: link d2/d6/d13/f28 d2/d6/dc/d18/d1d/d21/f2a 0 2024-03-29T08:07:07.106 INFO:tasks.workunit.client.1.smithi148.stdout:6/41: rename d6/d8 to d6/d9 0 2024-03-29T08:07:07.108 INFO:tasks.workunit.client.1.smithi148.stdout:3/58: mkdir de/d13 0 2024-03-29T08:07:07.109 INFO:tasks.workunit.client.0.smithi037.stdout:3/110: mknod dd/d15/c2a 0 2024-03-29T08:07:07.109 INFO:tasks.workunit.client.0.smithi037.stdout:1/122: symlink d2/d6/dc/d18/l2b 0 2024-03-29T08:07:07.111 INFO:tasks.workunit.client.0.smithi037.stdout:8/104: truncate d5/d10/f11 4736301 0 2024-03-29T08:07:07.112 INFO:tasks.workunit.client.1.smithi148.stdout:5/52: mknod d0/ca 0 2024-03-29T08:07:07.114 INFO:tasks.workunit.client.1.smithi148.stdout:7/85: creat d8/d14/f1c x:0 0 0 2024-03-29T08:07:07.116 INFO:tasks.workunit.client.0.smithi037.stdout:8/105: unlink d5/cf 0 2024-03-29T08:07:07.117 INFO:tasks.workunit.client.1.smithi148.stdout:9/42: rename c6 to db/ce 0 2024-03-29T08:07:07.120 INFO:tasks.workunit.client.1.smithi148.stdout:8/85: mkdir d17/d1d 0 2024-03-29T08:07:07.121 INFO:tasks.workunit.client.1.smithi148.stdout:8/86: chown d17/d18/f1c 9 1 2024-03-29T08:07:07.121 INFO:tasks.workunit.client.1.smithi148.stdout:2/51: mknod d7/da/c12 0 2024-03-29T08:07:07.121 INFO:tasks.workunit.client.0.smithi037.stdout:8/106: mkdir d5/d22 0 2024-03-29T08:07:07.124 INFO:tasks.workunit.client.1.smithi148.stdout:6/42: rename d6/f7 to d6/fa 0 2024-03-29T08:07:07.124 INFO:tasks.workunit.client.1.smithi148.stdout:0/73: link db/dc/l15 db/de/df/d11/l19 0 2024-03-29T08:07:07.125 INFO:tasks.workunit.client.0.smithi037.stdout:1/123: link d2/d6/dc/d18/d1d/d21/f20 d2/d6/dc/d18/d1d/f2c 0 2024-03-29T08:07:07.126 INFO:tasks.workunit.client.0.smithi037.stdout:6/75: truncate d0/d5/f16 183170 0 2024-03-29T08:07:07.127 INFO:tasks.workunit.client.0.smithi037.stdout:8/107: creat d5/d10/f23 x:0 0 0 2024-03-29T08:07:07.127 INFO:tasks.workunit.client.1.smithi148.stdout:5/53: mkdir d0/db 0 2024-03-29T08:07:07.129 INFO:tasks.workunit.client.0.smithi037.stdout:6/76: fsync d0/f4 0 2024-03-29T08:07:07.130 INFO:tasks.workunit.client.1.smithi148.stdout:7/86: mknod d8/c1d 0 2024-03-29T08:07:07.130 INFO:tasks.workunit.client.0.smithi037.stdout:3/111: write f4 [254882,82440] 0 2024-03-29T08:07:07.130 INFO:tasks.workunit.client.1.smithi148.stdout:4/48: mkdir df/d10 0 2024-03-29T08:07:07.132 INFO:tasks.workunit.client.0.smithi037.stdout:1/124: mknod d2/d6/dc/d18/d22/c2d 0 2024-03-29T08:07:07.135 INFO:tasks.workunit.client.0.smithi037.stdout:6/77: truncate d0/d5/f9 1665805 0 2024-03-29T08:07:07.136 INFO:tasks.workunit.client.1.smithi148.stdout:3/59: rename lb to de/l14 0 2024-03-29T08:07:07.137 INFO:tasks.workunit.client.1.smithi148.stdout:2/52: mkdir d7/d13 0 2024-03-29T08:07:07.137 INFO:tasks.workunit.client.0.smithi037.stdout:0/55: sync 2024-03-29T08:07:07.137 INFO:tasks.workunit.client.0.smithi037.stdout:4/22: sync 2024-03-29T08:07:07.137 INFO:tasks.workunit.client.0.smithi037.stdout:9/194: sync 2024-03-29T08:07:07.137 INFO:tasks.workunit.client.0.smithi037.stdout:7/162: sync 2024-03-29T08:07:07.139 INFO:tasks.workunit.client.1.smithi148.stdout:4/49: rmdir df 39 2024-03-29T08:07:07.139 INFO:tasks.workunit.client.0.smithi037.stdout:7/163: dread - dd/d21/f2b zero size 2024-03-29T08:07:07.140 INFO:tasks.workunit.client.1.smithi148.stdout:8/87: rename cf to d17/d1d/c1e 0 2024-03-29T08:07:07.141 INFO:tasks.workunit.client.0.smithi037.stdout:4/23: write f0 [1217320,29183] 0 2024-03-29T08:07:07.142 INFO:tasks.workunit.client.0.smithi037.stdout:0/56: rename d9/da/dd to d9/da/dd/d10 22 2024-03-29T08:07:07.144 INFO:tasks.workunit.client.0.smithi037.stdout:1/125: creat d2/d6/d13/f2e x:0 0 0 2024-03-29T08:07:07.144 INFO:tasks.workunit.client.0.smithi037.stdout:4/24: truncate f0 1852050 0 2024-03-29T08:07:07.144 INFO:tasks.workunit.client.0.smithi037.stdout:4/25: readlink - no filename 2024-03-29T08:07:07.146 INFO:tasks.workunit.client.1.smithi148.stdout:3/60: creat de/f15 x:0 0 0 2024-03-29T08:07:07.146 INFO:tasks.workunit.client.1.smithi148.stdout:0/74: dwrite db/dc/f12 [0,4194304] 0 2024-03-29T08:07:07.148 INFO:tasks.workunit.client.1.smithi148.stdout:5/54: dread d0/f9 [0,4194304] 0 2024-03-29T08:07:07.150 INFO:tasks.workunit.client.0.smithi037.stdout:4/26: write f0 [1921754,16943] 0 2024-03-29T08:07:07.150 INFO:tasks.workunit.client.0.smithi037.stdout:1/126: write d2/d6/dc/f25 [1049038,80430] 0 2024-03-29T08:07:07.151 INFO:tasks.workunit.client.0.smithi037.stdout:8/108: dread d5/d10/f1d [0,4194304] 0 2024-03-29T08:07:07.153 INFO:tasks.workunit.client.0.smithi037.stdout:7/164: rename dd/l30 to dd/d1d/l33 0 2024-03-29T08:07:07.153 INFO:tasks.workunit.client.1.smithi148.stdout:2/53: mknod d7/d13/c14 0 2024-03-29T08:07:07.159 INFO:tasks.workunit.client.0.smithi037.stdout:4/27: write f0 [896653,67178] 0 2024-03-29T08:07:07.161 INFO:tasks.workunit.client.0.smithi037.stdout:6/78: rename d0/d5/dd to d0/d5/d1e 0 2024-03-29T08:07:07.162 INFO:tasks.workunit.client.1.smithi148.stdout:7/87: dwrite d8/f17 [0,4194304] 0 2024-03-29T08:07:07.164 INFO:tasks.workunit.client.1.smithi148.stdout:2/54: creat d7/da/f15 x:0 0 0 2024-03-29T08:07:07.170 INFO:tasks.workunit.client.1.smithi148.stdout:7/88: fdatasync d8/f18 0 2024-03-29T08:07:07.171 INFO:tasks.workunit.client.1.smithi148.stdout:2/55: chown d7 24 1 2024-03-29T08:07:07.171 INFO:tasks.workunit.client.0.smithi037.stdout:6/79: fsync d0/d5/fa 0 2024-03-29T08:07:07.173 INFO:tasks.workunit.client.1.smithi148.stdout:0/75: fdatasync db/dc/f12 0 2024-03-29T08:07:07.173 INFO:tasks.workunit.client.1.smithi148.stdout:6/43: dwrite d6/fa [0,4194304] 0 2024-03-29T08:07:07.181 INFO:tasks.workunit.client.0.smithi037.stdout:3/112: dwrite dd/d15/f19 [0,4194304] 0 2024-03-29T08:07:07.182 INFO:tasks.workunit.client.1.smithi148.stdout:7/89: creat d8/d14/f1e x:0 0 0 2024-03-29T08:07:07.185 INFO:tasks.workunit.client.0.smithi037.stdout:6/80: rmdir d0 39 2024-03-29T08:07:07.186 INFO:tasks.workunit.client.1.smithi148.stdout:8/88: dwrite d17/d18/f1c [0,4194304] 0 2024-03-29T08:07:07.187 INFO:tasks.workunit.client.0.smithi037.stdout:1/127: rename d2/d6/dc/d18/l2b to d2/d6/dc/d18/d1d/l2f 0 2024-03-29T08:07:07.188 INFO:tasks.workunit.client.1.smithi148.stdout:6/44: symlink d6/d9/lb 0 2024-03-29T08:07:07.189 INFO:tasks.workunit.client.0.smithi037.stdout:0/57: dwrite f5 [0,4194304] 0 2024-03-29T08:07:07.190 INFO:tasks.workunit.client.0.smithi037.stdout:7/165: write dd/f1a [55401,16204] 0 2024-03-29T08:07:07.191 INFO:tasks.workunit.client.1.smithi148.stdout:6/45: readlink l2 0 2024-03-29T08:07:07.195 INFO:tasks.workunit.client.0.smithi037.stdout:9/195: dwrite d1/d4/d19/d1e/f3f [0,4194304] 0 2024-03-29T08:07:07.196 INFO:tasks.workunit.client.1.smithi148.stdout:7/90: creat d8/f1f x:0 0 0 2024-03-29T08:07:07.196 INFO:tasks.workunit.client.0.smithi037.stdout:4/28: dread f0 [0,4194304] 0 2024-03-29T08:07:07.197 INFO:tasks.workunit.client.1.smithi148.stdout:5/55: dwrite d0/f3 [0,4194304] 0 2024-03-29T08:07:07.199 INFO:tasks.workunit.client.0.smithi037.stdout:0/58: truncate f8 22827 0 2024-03-29T08:07:07.200 INFO:tasks.workunit.client.1.smithi148.stdout:7/91: chown d8/d14/f16 8 1 2024-03-29T08:07:07.200 INFO:tasks.workunit.client.0.smithi037.stdout:3/113: write dd/d15/f19 [4929716,95720] 0 2024-03-29T08:07:07.206 INFO:tasks.workunit.client.1.smithi148.stdout:5/56: symlink d0/db/lc 0 2024-03-29T08:07:07.210 INFO:tasks.workunit.client.0.smithi037.stdout:8/109: dwrite d5/fb [0,4194304] 0 2024-03-29T08:07:07.210 INFO:tasks.workunit.client.0.smithi037.stdout:4/29: creat d3/d4/d8/f9 x:0 0 0 2024-03-29T08:07:07.210 INFO:tasks.workunit.client.0.smithi037.stdout:7/166: write f5 [265501,37002] 0 2024-03-29T08:07:07.211 INFO:tasks.workunit.client.1.smithi148.stdout:6/46: dread f5 [0,4194304] 0 2024-03-29T08:07:07.212 INFO:tasks.workunit.client.0.smithi037.stdout:9/196: unlink d1/d4/d19/d1e/c24 0 2024-03-29T08:07:07.213 INFO:tasks.workunit.client.1.smithi148.stdout:5/57: symlink d0/db/ld 0 2024-03-29T08:07:07.213 INFO:tasks.workunit.client.0.smithi037.stdout:0/59: creat d9/da/f11 x:0 0 0 2024-03-29T08:07:07.215 INFO:tasks.workunit.client.0.smithi037.stdout:3/114: stat dd/l17 0 2024-03-29T08:07:07.221 INFO:tasks.workunit.client.0.smithi037.stdout:7/167: unlink f5 0 2024-03-29T08:07:07.223 INFO:tasks.workunit.client.0.smithi037.stdout:9/197: rmdir d1/d1f/d3a 39 2024-03-29T08:07:07.233 INFO:tasks.workunit.client.0.smithi037.stdout:0/60: symlink d9/da/l12 0 2024-03-29T08:07:07.234 INFO:tasks.workunit.client.0.smithi037.stdout:4/30: rename d3/d4/d8/f9 to d3/d4/fa 0 2024-03-29T08:07:07.241 INFO:tasks.workunit.client.0.smithi037.stdout:3/115: truncate dd/d15/f1e 285789 0 2024-03-29T08:07:07.241 INFO:tasks.workunit.client.0.smithi037.stdout:7/168: mkdir dd/d1d/d34 0 2024-03-29T08:07:07.241 INFO:tasks.workunit.client.0.smithi037.stdout:9/198: mknod d1/d4/d19/d1e/d25/d10/d23/c43 0 2024-03-29T08:07:07.242 INFO:tasks.workunit.client.1.smithi148.stdout:7/92: dwrite d8/fc [0,4194304] 0 2024-03-29T08:07:07.244 INFO:tasks.workunit.client.1.smithi148.stdout:5/58: dread d0/f3 [0,4194304] 0 2024-03-29T08:07:07.247 INFO:tasks.workunit.client.0.smithi037.stdout:0/61: mkdir d9/da/dd/d13 0 2024-03-29T08:07:07.248 INFO:tasks.workunit.client.1.smithi148.stdout:7/93: creat d8/d14/f20 x:0 0 0 2024-03-29T08:07:07.250 INFO:tasks.workunit.client.1.smithi148.stdout:7/94: write d8/d14/f16 [181252,120363] 0 2024-03-29T08:07:07.252 INFO:tasks.workunit.client.0.smithi037.stdout:0/62: truncate d9/da/f11 737842 0 2024-03-29T08:07:07.254 INFO:tasks.workunit.client.0.smithi037.stdout:7/169: mkdir dd/d21/d35 0 2024-03-29T08:07:07.262 INFO:tasks.workunit.client.0.smithi037.stdout:9/199: mkdir d1/d3/d44 0 2024-03-29T08:07:07.264 INFO:tasks.workunit.client.0.smithi037.stdout:8/110: dwrite d5/f1c [0,4194304] 0 2024-03-29T08:07:07.265 INFO:tasks.workunit.client.0.smithi037.stdout:0/63: symlink d9/da/dd/l14 0 2024-03-29T08:07:07.267 INFO:tasks.workunit.client.1.smithi148.stdout:9/43: sync 2024-03-29T08:07:07.269 INFO:tasks.workunit.client.0.smithi037.stdout:8/111: stat d5/d10/l14 0 2024-03-29T08:07:07.271 INFO:tasks.workunit.client.0.smithi037.stdout:4/31: dwrite d3/d4/fa [0,4194304] 0 2024-03-29T08:07:07.272 INFO:tasks.workunit.client.0.smithi037.stdout:1/128: truncate d2/d6/dc/f15 1763069 0 2024-03-29T08:07:07.272 INFO:tasks.workunit.client.1.smithi148.stdout:7/95: dread d8/d14/f16 [0,4194304] 0 2024-03-29T08:07:07.272 INFO:tasks.workunit.client.1.smithi148.stdout:1/57: truncate d3/f7 1147250 0 2024-03-29T08:07:07.273 INFO:tasks.workunit.client.0.smithi037.stdout:8/112: truncate d5/f1e 117007 0 2024-03-29T08:07:07.275 INFO:tasks.workunit.client.1.smithi148.stdout:9/44: truncate db/fd 794938 0 2024-03-29T08:07:07.275 INFO:tasks.workunit.client.1.smithi148.stdout:1/58: write d3/fa [788370,24582] 0 2024-03-29T08:07:07.278 INFO:tasks.workunit.client.1.smithi148.stdout:7/96: creat d8/d14/f21 x:0 0 0 2024-03-29T08:07:07.281 INFO:tasks.workunit.client.1.smithi148.stdout:0/76: rmdir db/de/df/d11 39 2024-03-29T08:07:07.281 INFO:tasks.workunit.client.1.smithi148.stdout:5/59: dwrite d0/f3 [4194304,4194304] 0 2024-03-29T08:07:07.281 INFO:tasks.workunit.client.0.smithi037.stdout:0/64: truncate f7 1127325 0 2024-03-29T08:07:07.281 INFO:tasks.workunit.client.0.smithi037.stdout:8/113: write d5/f17 [3162792,87968] 0 2024-03-29T08:07:07.282 INFO:tasks.workunit.client.1.smithi148.stdout:5/60: chown d0 5 1 2024-03-29T08:07:07.282 INFO:tasks.workunit.client.0.smithi037.stdout:8/114: chown d5/d10/f23 3121 1 2024-03-29T08:07:07.287 INFO:tasks.workunit.client.1.smithi148.stdout:7/97: symlink d8/d14/l22 0 2024-03-29T08:07:07.290 INFO:tasks.workunit.client.0.smithi037.stdout:1/129: rename d2/d6/dc/d18/d22/c2d to d2/d6/dc/c30 0 2024-03-29T08:07:07.294 INFO:tasks.workunit.client.1.smithi148.stdout:5/61: mkdir d0/db/de 0 2024-03-29T08:07:07.294 INFO:tasks.workunit.client.1.smithi148.stdout:0/77: dread db/dc/f12 [0,4194304] 0 2024-03-29T08:07:07.295 INFO:tasks.workunit.client.1.smithi148.stdout:1/59: symlink d3/d13/l14 0 2024-03-29T08:07:07.297 INFO:tasks.workunit.client.0.smithi037.stdout:1/130: dread - d2/d6/dc/d18/d1d/d21/f2a zero size 2024-03-29T08:07:07.304 INFO:tasks.workunit.client.1.smithi148.stdout:7/98: rename d8/d14/l22 to d8/l23 0 2024-03-29T08:07:07.310 INFO:tasks.workunit.client.0.smithi037.stdout:1/131: creat d2/f31 x:0 0 0 2024-03-29T08:07:07.310 INFO:tasks.workunit.client.1.smithi148.stdout:0/78: rename db/de to db/de/df/d1a 22 2024-03-29T08:07:07.312 INFO:tasks.workunit.client.1.smithi148.stdout:0/79: write f2 [5979738,63453] 0 2024-03-29T08:07:07.313 INFO:tasks.workunit.client.1.smithi148.stdout:7/99: symlink d8/l24 0 2024-03-29T08:07:07.313 INFO:tasks.workunit.client.0.smithi037.stdout:0/65: getdents d9/da 0 2024-03-29T08:07:07.315 INFO:tasks.workunit.client.0.smithi037.stdout:4/32: dwrite f0 [0,4194304] 0 2024-03-29T08:07:07.321 INFO:tasks.workunit.client.1.smithi148.stdout:4/50: truncate f7 1489644 0 2024-03-29T08:07:07.322 INFO:tasks.workunit.client.1.smithi148.stdout:5/62: fdatasync d0/f3 0 2024-03-29T08:07:07.324 INFO:tasks.workunit.client.0.smithi037.stdout:1/132: write d2/d6/dc/f25 [604998,29546] 0 2024-03-29T08:07:07.325 INFO:tasks.workunit.client.0.smithi037.stdout:4/33: truncate f0 4222496 0 2024-03-29T08:07:07.326 INFO:tasks.workunit.client.1.smithi148.stdout:7/100: symlink d8/d14/l25 0 2024-03-29T08:07:07.327 INFO:tasks.workunit.client.0.smithi037.stdout:3/116: dwrite dd/f11 [0,4194304] 0 2024-03-29T08:07:07.328 INFO:tasks.workunit.client.1.smithi148.stdout:0/80: dread f7 [0,4194304] 0 2024-03-29T08:07:07.328 INFO:tasks.workunit.client.1.smithi148.stdout:6/47: getdents d6/d9 0 2024-03-29T08:07:07.329 INFO:tasks.workunit.client.0.smithi037.stdout:8/115: dwrite d5/fb [0,4194304] 0 2024-03-29T08:07:07.331 INFO:tasks.workunit.client.1.smithi148.stdout:6/48: write d6/fa [2472550,69156] 0 2024-03-29T08:07:07.333 INFO:tasks.workunit.client.1.smithi148.stdout:2/56: truncate f5 3930648 0 2024-03-29T08:07:07.335 INFO:tasks.workunit.client.0.smithi037.stdout:4/34: write f0 [718954,105743] 0 2024-03-29T08:07:07.335 INFO:tasks.workunit.client.0.smithi037.stdout:3/117: readlink dd/d15/d18/l22 0 2024-03-29T08:07:07.340 INFO:tasks.workunit.client.1.smithi148.stdout:8/89: truncate f15 3185014 0 2024-03-29T08:07:07.340 INFO:tasks.workunit.client.0.smithi037.stdout:3/118: chown fb 99303 1 2024-03-29T08:07:07.344 INFO:tasks.workunit.client.0.smithi037.stdout:8/116: rmdir d5/d22 0 2024-03-29T08:07:07.346 INFO:tasks.workunit.client.0.smithi037.stdout:8/117: readlink d5/d10/l14 0 2024-03-29T08:07:07.347 INFO:tasks.workunit.client.0.smithi037.stdout:4/35: chown d3/c6 36259 1 2024-03-29T08:07:07.347 INFO:tasks.workunit.client.0.smithi037.stdout:1/133: rename d2/d6/cb to d2/d6/dc/d18/c32 0 2024-03-29T08:07:07.349 INFO:tasks.workunit.client.1.smithi148.stdout:6/49: mknod d6/cc 0 2024-03-29T08:07:07.352 INFO:tasks.workunit.client.0.smithi037.stdout:8/118: write d5/f1b [376913,91076] 0 2024-03-29T08:07:07.353 INFO:tasks.workunit.client.1.smithi148.stdout:2/57: creat d7/d13/f16 x:0 0 0 2024-03-29T08:07:07.354 INFO:tasks.workunit.client.0.smithi037.stdout:0/66: dwrite d9/ff [0,4194304] 0 2024-03-29T08:07:07.360 INFO:tasks.workunit.client.0.smithi037.stdout:8/119: mknod d5/c24 0 2024-03-29T08:07:07.362 INFO:tasks.workunit.client.0.smithi037.stdout:0/67: rmdir d9/da/dd 39 2024-03-29T08:07:07.363 INFO:tasks.workunit.client.1.smithi148.stdout:8/90: unlink d17/l1b 0 2024-03-29T08:07:07.364 INFO:tasks.workunit.client.0.smithi037.stdout:8/120: readlink d5/d10/l20 0 2024-03-29T08:07:07.366 INFO:tasks.workunit.client.1.smithi148.stdout:2/58: rename d7/da/f15 to d7/f17 0 2024-03-29T08:07:07.367 INFO:tasks.workunit.client.1.smithi148.stdout:0/81: getdents db/de 0 2024-03-29T08:07:07.368 INFO:tasks.workunit.client.1.smithi148.stdout:8/91: mknod d17/d18/c1f 0 2024-03-29T08:07:07.369 INFO:tasks.workunit.client.1.smithi148.stdout:2/59: write d7/f9 [1732959,34730] 0 2024-03-29T08:07:07.369 INFO:tasks.workunit.client.1.smithi148.stdout:0/82: write f2 [2321588,23155] 0 2024-03-29T08:07:07.370 INFO:tasks.workunit.client.1.smithi148.stdout:8/92: chown ca 13235109 1 2024-03-29T08:07:07.371 INFO:tasks.workunit.client.1.smithi148.stdout:6/50: dread f5 [0,4194304] 0 2024-03-29T08:07:07.371 INFO:tasks.workunit.client.1.smithi148.stdout:2/60: dread - d7/da/fe zero size 2024-03-29T08:07:07.372 INFO:tasks.workunit.client.1.smithi148.stdout:5/63: truncate d0/f3 917985 0 2024-03-29T08:07:07.374 INFO:tasks.workunit.client.1.smithi148.stdout:1/60: truncate d3/f12 1310269 0 2024-03-29T08:07:07.374 INFO:tasks.workunit.client.1.smithi148.stdout:3/61: sync 2024-03-29T08:07:07.380 INFO:tasks.workunit.client.1.smithi148.stdout:6/51: creat d6/fd x:0 0 0 2024-03-29T08:07:07.381 INFO:tasks.workunit.client.0.smithi037.stdout:8/121: fsync d5/f15 0 2024-03-29T08:07:07.383 INFO:tasks.workunit.client.1.smithi148.stdout:2/61: creat d7/d13/f18 x:0 0 0 2024-03-29T08:07:07.383 INFO:tasks.workunit.client.1.smithi148.stdout:6/52: readlink d6/d9/lb 0 2024-03-29T08:07:07.383 INFO:tasks.workunit.client.1.smithi148.stdout:8/93: dread d17/f1a [0,4194304] 0 2024-03-29T08:07:07.385 INFO:tasks.workunit.client.1.smithi148.stdout:4/51: write f1 [3364315,61690] 0 2024-03-29T08:07:07.392 INFO:tasks.workunit.client.0.smithi037.stdout:1/134: dread d2/f5 [0,4194304] 0 2024-03-29T08:07:07.393 INFO:tasks.workunit.client.1.smithi148.stdout:6/53: creat d6/fe x:0 0 0 2024-03-29T08:07:07.397 INFO:tasks.workunit.client.0.smithi037.stdout:4/36: dwrite f0 [4194304,4194304] 0 2024-03-29T08:07:07.397 INFO:tasks.workunit.client.1.smithi148.stdout:8/94: creat d17/d1d/f20 x:0 0 0 2024-03-29T08:07:07.398 INFO:tasks.workunit.client.0.smithi037.stdout:8/122: mkdir d5/d25 0 2024-03-29T08:07:07.400 INFO:tasks.workunit.client.0.smithi037.stdout:3/119: dwrite dd/d15/f16 [0,4194304] 0 2024-03-29T08:07:07.400 INFO:tasks.workunit.client.1.smithi148.stdout:2/62: dwrite d7/f9 [0,4194304] 0 2024-03-29T08:07:07.402 INFO:tasks.workunit.client.1.smithi148.stdout:2/63: dread - d7/f17 zero size 2024-03-29T08:07:07.403 INFO:tasks.workunit.client.0.smithi037.stdout:4/37: chown f0 68468237 1 2024-03-29T08:07:07.406 INFO:tasks.workunit.client.1.smithi148.stdout:0/83: link f6 db/de/df/d11/f1b 0 2024-03-29T08:07:07.406 INFO:tasks.workunit.client.1.smithi148.stdout:3/62: mknod de/d13/c16 0 2024-03-29T08:07:07.407 INFO:tasks.workunit.client.1.smithi148.stdout:5/64: link d0/ca d0/db/de/cf 0 2024-03-29T08:07:07.407 INFO:tasks.workunit.client.0.smithi037.stdout:0/68: dwrite f7 [0,4194304] 0 2024-03-29T08:07:07.407 INFO:tasks.workunit.client.1.smithi148.stdout:2/64: write d7/f9 [2655247,5252] 0 2024-03-29T08:07:07.408 INFO:tasks.workunit.client.0.smithi037.stdout:3/120: write dd/f1f [654897,39551] 0 2024-03-29T08:07:07.409 INFO:tasks.workunit.client.0.smithi037.stdout:8/123: mknod d5/d25/c26 0 2024-03-29T08:07:07.411 INFO:tasks.workunit.client.0.smithi037.stdout:4/38: write f0 [7798227,68775] 0 2024-03-29T08:07:07.418 INFO:tasks.workunit.client.1.smithi148.stdout:8/95: dwrite d17/d1d/f20 [0,4194304] 0 2024-03-29T08:07:07.420 INFO:tasks.workunit.client.1.smithi148.stdout:8/96: rename d17 to d17/d1d/d21 22 2024-03-29T08:07:07.421 INFO:tasks.workunit.client.0.smithi037.stdout:1/135: rename d2/c26 to d2/c33 0 2024-03-29T08:07:07.423 INFO:tasks.workunit.client.1.smithi148.stdout:6/54: getdents d6/d9 0 2024-03-29T08:07:07.427 INFO:tasks.workunit.client.1.smithi148.stdout:6/55: chown d6/d9/lb 37446 1 2024-03-29T08:07:07.429 INFO:tasks.workunit.client.1.smithi148.stdout:4/52: creat df/d10/f11 x:0 0 0 2024-03-29T08:07:07.430 INFO:tasks.workunit.client.0.smithi037.stdout:3/121: mkdir dd/d15/d2b 0 2024-03-29T08:07:07.433 INFO:tasks.workunit.client.0.smithi037.stdout:8/124: unlink d5/d10/d19/l1a 0 2024-03-29T08:07:07.435 INFO:tasks.workunit.client.0.smithi037.stdout:8/125: readlink d5/l9 0 2024-03-29T08:07:07.436 INFO:tasks.workunit.client.1.smithi148.stdout:3/63: dwrite f5 [0,4194304] 0 2024-03-29T08:07:07.436 INFO:tasks.workunit.client.1.smithi148.stdout:8/97: fsync f7 0 2024-03-29T08:07:07.436 INFO:tasks.workunit.client.0.smithi037.stdout:4/39: mkdir d3/d4/d8/db 0 2024-03-29T08:07:07.437 INFO:tasks.workunit.client.1.smithi148.stdout:4/53: creat df/d10/f12 x:0 0 0 2024-03-29T08:07:07.439 INFO:tasks.workunit.client.1.smithi148.stdout:2/65: getdents d7/da 0 2024-03-29T08:07:07.440 INFO:tasks.workunit.client.0.smithi037.stdout:4/40: write f0 [3394178,76276] 0 2024-03-29T08:07:07.441 INFO:tasks.workunit.client.0.smithi037.stdout:8/126: chown d5/f1b 19813 1 2024-03-29T08:07:07.441 INFO:tasks.workunit.client.1.smithi148.stdout:4/54: write f4 [869482,84180] 0 2024-03-29T08:07:07.441 INFO:tasks.workunit.client.1.smithi148.stdout:5/65: dread d0/f3 [0,4194304] 0 2024-03-29T08:07:07.442 INFO:tasks.workunit.client.0.smithi037.stdout:0/69: dwrite f5 [0,4194304] 0 2024-03-29T08:07:07.442 INFO:tasks.workunit.client.0.smithi037.stdout:1/136: readlink d2/d6/l27 0 2024-03-29T08:07:07.443 INFO:tasks.workunit.client.0.smithi037.stdout:4/41: stat d3/c7 0 2024-03-29T08:07:07.443 INFO:tasks.workunit.client.1.smithi148.stdout:5/66: chown d0/db/lc 4434 1 2024-03-29T08:07:07.446 INFO:tasks.workunit.client.0.smithi037.stdout:3/122: creat dd/d15/d2b/f2c x:0 0 0 2024-03-29T08:07:07.447 INFO:tasks.workunit.client.1.smithi148.stdout:6/56: rmdir d6 39 2024-03-29T08:07:07.449 INFO:tasks.workunit.client.0.smithi037.stdout:3/123: chown c6 2850 1 2024-03-29T08:07:07.450 INFO:tasks.workunit.client.1.smithi148.stdout:5/67: dread d0/f3 [0,4194304] 0 2024-03-29T08:07:07.451 INFO:tasks.workunit.client.0.smithi037.stdout:8/127: symlink d5/l27 0 2024-03-29T08:07:07.457 INFO:tasks.workunit.client.0.smithi037.stdout:3/124: mkdir dd/d15/d2b/d2d 0 2024-03-29T08:07:07.458 INFO:tasks.workunit.client.1.smithi148.stdout:9/45: sync 2024-03-29T08:07:07.458 INFO:tasks.workunit.client.1.smithi148.stdout:3/64: dwrite f5 [0,4194304] 0 2024-03-29T08:07:07.459 INFO:tasks.workunit.client.0.smithi037.stdout:3/125: chown dd/f1d 120 1 2024-03-29T08:07:07.460 INFO:tasks.workunit.client.1.smithi148.stdout:5/68: write d0/f9 [4954923,116281] 0 2024-03-29T08:07:07.462 INFO:tasks.workunit.client.0.smithi037.stdout:0/70: mkdir d9/da/d15 0 2024-03-29T08:07:07.462 INFO:tasks.workunit.client.1.smithi148.stdout:9/46: chown db/cc 1485017 1 2024-03-29T08:07:07.463 INFO:tasks.workunit.client.1.smithi148.stdout:9/47: readlink la 0 2024-03-29T08:07:07.464 INFO:tasks.workunit.client.1.smithi148.stdout:1/61: dread d3/f7 [0,4194304] 0 2024-03-29T08:07:07.465 INFO:tasks.workunit.client.0.smithi037.stdout:4/42: dread d3/d4/fa [0,4194304] 0 2024-03-29T08:07:07.466 INFO:tasks.workunit.client.1.smithi148.stdout:5/69: dread d0/f3 [0,4194304] 0 2024-03-29T08:07:07.466 INFO:tasks.workunit.client.1.smithi148.stdout:0/84: truncate fa 2389449 0 2024-03-29T08:07:07.469 INFO:tasks.workunit.client.1.smithi148.stdout:3/65: creat de/f17 x:0 0 0 2024-03-29T08:07:07.469 INFO:tasks.workunit.client.1.smithi148.stdout:9/48: unlink la 0 2024-03-29T08:07:07.470 INFO:tasks.workunit.client.0.smithi037.stdout:3/126: mknod dd/d15/d2b/c2e 0 2024-03-29T08:07:07.470 INFO:tasks.workunit.client.0.smithi037.stdout:4/43: write f0 [2403287,8440] 0 2024-03-29T08:07:07.470 INFO:tasks.workunit.client.1.smithi148.stdout:3/66: chown de/d13 54270516 1 2024-03-29T08:07:07.472 INFO:tasks.workunit.client.0.smithi037.stdout:3/127: read - dd/f24 zero size 2024-03-29T08:07:07.474 INFO:tasks.workunit.client.1.smithi148.stdout:6/57: dread d6/fa [0,4194304] 0 2024-03-29T08:07:07.476 INFO:tasks.workunit.client.0.smithi037.stdout:8/128: write d5/d10/f11 [3862113,18704] 0 2024-03-29T08:07:07.478 INFO:tasks.workunit.client.1.smithi148.stdout:9/49: stat c1 0 2024-03-29T08:07:07.479 INFO:tasks.workunit.client.0.smithi037.stdout:0/71: readlink d9/da/dd/l14 0 2024-03-29T08:07:07.482 INFO:tasks.workunit.client.0.smithi037.stdout:1/137: link d2/c1a d2/d6/d13/c34 0 2024-03-29T08:07:07.483 INFO:tasks.workunit.client.1.smithi148.stdout:3/67: mknod de/d13/c18 0 2024-03-29T08:07:07.483 INFO:tasks.workunit.client.1.smithi148.stdout:4/55: truncate f3 2529818 0 2024-03-29T08:07:07.484 INFO:tasks.workunit.client.1.smithi148.stdout:2/66: dread f5 [0,4194304] 0 2024-03-29T08:07:07.485 INFO:tasks.workunit.client.1.smithi148.stdout:9/50: symlink db/lf 0 2024-03-29T08:07:07.488 INFO:tasks.workunit.client.0.smithi037.stdout:3/128: creat dd/d15/d18/f2f x:0 0 0 2024-03-29T08:07:07.488 INFO:tasks.workunit.client.1.smithi148.stdout:2/67: chown l4 110014 1 2024-03-29T08:07:07.488 INFO:tasks.workunit.client.1.smithi148.stdout:4/56: symlink df/l13 0 2024-03-29T08:07:07.490 INFO:tasks.workunit.client.0.smithi037.stdout:8/129: creat d5/d25/f28 x:0 0 0 2024-03-29T08:07:07.490 INFO:tasks.workunit.client.1.smithi148.stdout:6/58: dread d6/fa [0,4194304] 0 2024-03-29T08:07:07.491 INFO:tasks.workunit.client.0.smithi037.stdout:0/72: mknod d9/da/dd/c16 0 2024-03-29T08:07:07.492 INFO:tasks.workunit.client.1.smithi148.stdout:9/51: mkdir db/d10 0 2024-03-29T08:07:07.493 INFO:tasks.workunit.client.1.smithi148.stdout:0/85: getdents db/de/df/d14 0 2024-03-29T08:07:07.495 INFO:tasks.workunit.client.1.smithi148.stdout:0/86: write f2 [1352626,22706] 0 2024-03-29T08:07:07.495 INFO:tasks.workunit.client.0.smithi037.stdout:1/138: unlink d2/f5 0 2024-03-29T08:07:07.496 INFO:tasks.workunit.client.1.smithi148.stdout:2/68: rename d7/d13/f18 to d7/f19 0 2024-03-29T08:07:07.498 INFO:tasks.workunit.client.1.smithi148.stdout:3/68: dread f5 [0,4194304] 0 2024-03-29T08:07:07.498 INFO:tasks.workunit.client.1.smithi148.stdout:0/87: write db/dc/f12 [986829,14454] 0 2024-03-29T08:07:07.501 INFO:tasks.workunit.client.1.smithi148.stdout:1/62: read d3/f12 [849413,117005] 0 2024-03-29T08:07:07.501 INFO:tasks.workunit.client.0.smithi037.stdout:3/129: mkdir dd/d15/d18/d30 0 2024-03-29T08:07:07.501 INFO:tasks.workunit.client.0.smithi037.stdout:4/44: dwrite f0 [8388608,4194304] 0 2024-03-29T08:07:07.501 INFO:tasks.workunit.client.0.smithi037.stdout:8/130: symlink d5/l29 0 2024-03-29T08:07:07.503 INFO:tasks.workunit.client.1.smithi148.stdout:3/69: chown de/l12 21139271 1 2024-03-29T08:07:07.506 INFO:tasks.workunit.client.1.smithi148.stdout:6/59: dread d6/fa [0,4194304] 0 2024-03-29T08:07:07.508 INFO:tasks.workunit.client.0.smithi037.stdout:3/130: write dd/d15/f21 [1014796,69559] 0 2024-03-29T08:07:07.511 INFO:tasks.workunit.client.1.smithi148.stdout:4/57: dwrite f4 [0,4194304] 0 2024-03-29T08:07:07.511 INFO:tasks.workunit.client.1.smithi148.stdout:6/60: truncate d6/fe 464164 0 2024-03-29T08:07:07.512 INFO:tasks.workunit.client.1.smithi148.stdout:9/52: unlink db/ce 0 2024-03-29T08:07:07.513 INFO:tasks.workunit.client.0.smithi037.stdout:4/45: creat d3/d4/d8/fc x:0 0 0 2024-03-29T08:07:07.516 INFO:tasks.workunit.client.0.smithi037.stdout:3/131: write dd/f23 [253451,117734] 0 2024-03-29T08:07:07.524 INFO:tasks.workunit.client.1.smithi148.stdout:6/61: truncate d6/fd 428129 0 2024-03-29T08:07:07.525 INFO:tasks.workunit.client.1.smithi148.stdout:7/101: sync 2024-03-29T08:07:07.525 INFO:tasks.workunit.client.0.smithi037.stdout:1/139: write d2/d6/d13/f28 [803987,127919] 0 2024-03-29T08:07:07.529 INFO:tasks.workunit.client.0.smithi037.stdout:3/132: mknod dd/c31 0 2024-03-29T08:07:07.531 INFO:tasks.workunit.client.0.smithi037.stdout:0/73: dwrite d9/da/f11 [0,4194304] 0 2024-03-29T08:07:07.531 INFO:tasks.workunit.client.1.smithi148.stdout:6/62: unlink l2 0 2024-03-29T08:07:07.534 INFO:tasks.workunit.client.1.smithi148.stdout:7/102: write d8/f1f [162444,59608] 0 2024-03-29T08:07:07.534 INFO:tasks.workunit.client.1.smithi148.stdout:5/70: dwrite d0/f3 [0,4194304] 0 2024-03-29T08:07:07.535 INFO:tasks.workunit.client.0.smithi037.stdout:1/140: creat d2/d6/dc/f35 x:0 0 0 2024-03-29T08:07:07.536 INFO:tasks.workunit.client.0.smithi037.stdout:0/74: chown d9/da 415931060 1 2024-03-29T08:07:07.538 INFO:tasks.workunit.client.1.smithi148.stdout:6/63: write d6/fe [218526,52979] 0 2024-03-29T08:07:07.539 INFO:tasks.workunit.client.1.smithi148.stdout:0/88: dwrite db/f10 [0,4194304] 0 2024-03-29T08:07:07.541 INFO:tasks.workunit.client.0.smithi037.stdout:0/75: rename d9 to d9/da/dd/d13/d17 22 2024-03-29T08:07:07.541 INFO:tasks.workunit.client.0.smithi037.stdout:3/133: mkdir dd/d32 0 2024-03-29T08:07:07.543 INFO:tasks.workunit.client.0.smithi037.stdout:8/131: dwrite d5/f1c [0,4194304] 0 2024-03-29T08:07:07.543 INFO:tasks.workunit.client.0.smithi037.stdout:3/134: fdatasync dd/fe 0 2024-03-29T08:07:07.544 INFO:tasks.workunit.client.1.smithi148.stdout:5/71: write d0/f9 [3701817,78895] 0 2024-03-29T08:07:07.545 INFO:tasks.workunit.client.0.smithi037.stdout:0/76: write d9/fb [22003,107975] 0 2024-03-29T08:07:07.545 INFO:tasks.workunit.client.1.smithi148.stdout:2/69: dwrite d7/fc [0,4194304] 0 2024-03-29T08:07:07.548 INFO:tasks.workunit.client.0.smithi037.stdout:1/141: creat d2/d6/f36 x:0 0 0 2024-03-29T08:07:07.549 INFO:tasks.workunit.client.1.smithi148.stdout:7/103: readlink d8/l23 0 2024-03-29T08:07:07.550 INFO:tasks.workunit.client.0.smithi037.stdout:8/132: symlink d5/d10/d19/l2a 0 2024-03-29T08:07:07.550 INFO:tasks.workunit.client.1.smithi148.stdout:9/53: dread f7 [0,4194304] 0 2024-03-29T08:07:07.552 INFO:tasks.workunit.client.1.smithi148.stdout:6/64: mkdir d6/df 0 2024-03-29T08:07:07.554 INFO:tasks.workunit.client.1.smithi148.stdout:0/89: symlink db/dc/l1c 0 2024-03-29T08:07:07.555 INFO:tasks.workunit.client.0.smithi037.stdout:3/135: write dd/d15/f21 [1765799,72390] 0 2024-03-29T08:07:07.555 INFO:tasks.workunit.client.0.smithi037.stdout:8/133: chown f1 1983013806 1 2024-03-29T08:07:07.555 INFO:tasks.workunit.client.1.smithi148.stdout:4/58: dread fa [0,4194304] 0 2024-03-29T08:07:07.560 INFO:tasks.workunit.client.0.smithi037.stdout:8/134: write d5/f1b [2852990,21454] 0 2024-03-29T08:07:07.561 INFO:tasks.workunit.client.1.smithi148.stdout:1/63: dwrite d3/fa [0,4194304] 0 2024-03-29T08:07:07.568 INFO:tasks.workunit.client.0.smithi037.stdout:3/136: mkdir dd/d15/d27/d33 0 2024-03-29T08:07:07.570 INFO:tasks.workunit.client.0.smithi037.stdout:4/46: dwrite d3/d4/fa [0,4194304] 0 2024-03-29T08:07:07.573 INFO:tasks.workunit.client.1.smithi148.stdout:5/72: creat d0/db/f10 x:0 0 0 2024-03-29T08:07:07.584 INFO:tasks.workunit.client.1.smithi148.stdout:6/65: dwrite d6/fe [0,4194304] 0 2024-03-29T08:07:07.585 INFO:tasks.workunit.client.0.smithi037.stdout:8/135: symlink d5/d10/l2b 0 2024-03-29T08:07:07.587 INFO:tasks.workunit.client.1.smithi148.stdout:1/64: dwrite f1 [0,4194304] 0 2024-03-29T08:07:07.587 INFO:tasks.workunit.client.1.smithi148.stdout:9/54: rename c5 to db/d10/c11 0 2024-03-29T08:07:07.589 INFO:tasks.workunit.client.0.smithi037.stdout:0/77: creat d9/da/d15/f18 x:0 0 0 2024-03-29T08:07:07.593 INFO:tasks.workunit.client.1.smithi148.stdout:4/59: creat df/f14 x:0 0 0 2024-03-29T08:07:07.600 INFO:tasks.workunit.client.0.smithi037.stdout:4/47: mknod d3/cd 0 2024-03-29T08:07:07.600 INFO:tasks.workunit.client.1.smithi148.stdout:9/55: mkdir db/d12 0 2024-03-29T08:07:07.603 INFO:tasks.workunit.client.0.smithi037.stdout:3/137: mknod dd/d32/c34 0 2024-03-29T08:07:07.609 INFO:tasks.workunit.client.0.smithi037.stdout:3/138: write dd/fe [1129372,117285] 0 2024-03-29T08:07:07.612 INFO:tasks.workunit.client.0.smithi037.stdout:8/136: dwrite d5/f1b [0,4194304] 0 2024-03-29T08:07:07.612 INFO:tasks.workunit.client.0.smithi037.stdout:0/78: dread d9/ff [0,4194304] 0 2024-03-29T08:07:07.613 INFO:tasks.workunit.client.1.smithi148.stdout:2/70: dwrite f5 [4194304,4194304] 0 2024-03-29T08:07:07.613 INFO:tasks.workunit.client.1.smithi148.stdout:1/65: link d3/fa d3/d13/f15 0 2024-03-29T08:07:07.615 INFO:tasks.workunit.client.0.smithi037.stdout:0/79: read - d9/da/d15/f18 zero size 2024-03-29T08:07:07.616 INFO:tasks.workunit.client.1.smithi148.stdout:0/90: fsync db/f10 0 2024-03-29T08:07:07.618 INFO:tasks.workunit.client.0.smithi037.stdout:3/139: creat dd/d15/d18/f35 x:0 0 0 2024-03-29T08:07:07.621 INFO:tasks.workunit.client.0.smithi037.stdout:3/140: stat dd/d15/f16 0 2024-03-29T08:07:07.621 INFO:tasks.workunit.client.1.smithi148.stdout:2/71: chown d7/da/c11 53819 1 2024-03-29T08:07:07.622 INFO:tasks.workunit.client.1.smithi148.stdout:1/66: unlink d3/l11 0 2024-03-29T08:07:07.622 INFO:tasks.workunit.client.1.smithi148.stdout:7/104: getdents d8/d14/d19 0 2024-03-29T08:07:07.622 INFO:tasks.workunit.client.0.smithi037.stdout:1/142: write d2/d6/dc/f19 [801093,79019] 0 2024-03-29T08:07:07.623 INFO:tasks.workunit.client.1.smithi148.stdout:5/73: link d0/db/de/cf d0/db/c11 0 2024-03-29T08:07:07.625 INFO:tasks.workunit.client.1.smithi148.stdout:5/74: stat d0 0 2024-03-29T08:07:07.627 INFO:tasks.workunit.client.0.smithi037.stdout:0/80: symlink d9/da/d15/l19 0 2024-03-29T08:07:07.628 INFO:tasks.workunit.client.0.smithi037.stdout:0/81: chown d9 59481437 1 2024-03-29T08:07:07.630 INFO:tasks.workunit.client.0.smithi037.stdout:4/48: dwrite d3/d4/fa [0,4194304] 0 2024-03-29T08:07:07.631 INFO:tasks.workunit.client.1.smithi148.stdout:5/75: dread - d0/db/f10 zero size 2024-03-29T08:07:07.632 INFO:tasks.workunit.client.1.smithi148.stdout:8/98: sync 2024-03-29T08:07:07.633 INFO:tasks.workunit.client.0.smithi037.stdout:3/141: unlink dd/d15/d18/l22 0 2024-03-29T08:07:07.633 INFO:tasks.workunit.client.1.smithi148.stdout:7/105: creat d8/f26 x:0 0 0 2024-03-29T08:07:07.634 INFO:tasks.workunit.client.1.smithi148.stdout:6/66: dwrite d6/fa [0,4194304] 0 2024-03-29T08:07:07.634 INFO:tasks.workunit.client.1.smithi148.stdout:0/91: symlink db/de/df/d14/l1d 0 2024-03-29T08:07:07.635 INFO:tasks.workunit.client.0.smithi037.stdout:1/143: creat d2/d6/dc/d18/d1d/d21/f37 x:0 0 0 2024-03-29T08:07:07.637 INFO:tasks.workunit.client.0.smithi037.stdout:0/82: write f8 [778336,26792] 0 2024-03-29T08:07:07.639 INFO:tasks.workunit.client.0.smithi037.stdout:4/49: creat d3/d4/fe x:0 0 0 2024-03-29T08:07:07.640 INFO:tasks.workunit.client.1.smithi148.stdout:9/56: dwrite db/fd [0,4194304] 0 2024-03-29T08:07:07.642 INFO:tasks.workunit.client.1.smithi148.stdout:7/106: rename d8 to d8/d14/d19/d27 22 2024-03-29T08:07:07.643 INFO:tasks.workunit.client.1.smithi148.stdout:0/92: creat db/dc/f1e x:0 0 0 2024-03-29T08:07:07.644 INFO:tasks.workunit.client.1.smithi148.stdout:6/67: readlink d6/d9/lb 0 2024-03-29T08:07:07.644 INFO:tasks.workunit.client.1.smithi148.stdout:9/57: chown db/d10 0 1 2024-03-29T08:07:07.644 INFO:tasks.workunit.client.1.smithi148.stdout:8/99: fsync d17/f1a 0 2024-03-29T08:07:07.646 INFO:tasks.workunit.client.1.smithi148.stdout:7/107: stat d8/fe 0 2024-03-29T08:07:07.648 INFO:tasks.workunit.client.1.smithi148.stdout:6/68: read d6/fa [364172,3647] 0 2024-03-29T08:07:07.649 INFO:tasks.workunit.client.0.smithi037.stdout:3/142: rename dd/f25 to dd/d15/d27/f36 0 2024-03-29T08:07:07.650 INFO:tasks.workunit.client.1.smithi148.stdout:1/67: truncate d3/d13/f15 2183474 0 2024-03-29T08:07:07.651 INFO:tasks.workunit.client.1.smithi148.stdout:6/69: chown d6/fe 45436 1 2024-03-29T08:07:07.653 INFO:tasks.workunit.client.0.smithi037.stdout:8/137: getdents d5/d10/d19 0 2024-03-29T08:07:07.656 INFO:tasks.workunit.client.1.smithi148.stdout:3/70: sync 2024-03-29T08:07:07.656 INFO:tasks.workunit.client.1.smithi148.stdout:9/58: write f3 [7359493,58966] 0 2024-03-29T08:07:07.656 INFO:tasks.workunit.client.1.smithi148.stdout:2/72: dread f5 [4194304,4194304] 0 2024-03-29T08:07:07.658 INFO:tasks.workunit.client.0.smithi037.stdout:0/83: write d9/ff [3524734,40280] 0 2024-03-29T08:07:07.659 INFO:tasks.workunit.client.0.smithi037.stdout:2/75: sync 2024-03-29T08:07:07.660 INFO:tasks.workunit.client.1.smithi148.stdout:4/60: truncate fc 3350798 0 2024-03-29T08:07:07.661 INFO:tasks.workunit.client.0.smithi037.stdout:1/144: dread d2/d6/d13/f28 [0,4194304] 0 2024-03-29T08:07:07.662 INFO:tasks.workunit.client.1.smithi148.stdout:4/61: rename df to df/d10/d15 22 2024-03-29T08:07:07.663 INFO:tasks.workunit.client.1.smithi148.stdout:8/100: creat d17/d18/f22 x:0 0 0 2024-03-29T08:07:07.664 INFO:tasks.workunit.client.1.smithi148.stdout:7/108: mknod d8/c28 0 2024-03-29T08:07:07.665 INFO:tasks.workunit.client.0.smithi037.stdout:3/143: creat dd/d15/d2b/f37 x:0 0 0 2024-03-29T08:07:07.665 INFO:tasks.workunit.client.1.smithi148.stdout:0/93: dread db/f10 [0,4194304] 0 2024-03-29T08:07:07.666 INFO:tasks.workunit.client.0.smithi037.stdout:0/84: write f5 [4620585,14496] 0 2024-03-29T08:07:07.667 INFO:tasks.workunit.client.1.smithi148.stdout:0/94: fsync db/f10 0 2024-03-29T08:07:07.671 INFO:tasks.workunit.client.1.smithi148.stdout:9/59: rmdir db/d10 39 2024-03-29T08:07:07.672 INFO:tasks.workunit.client.0.smithi037.stdout:3/144: write dd/d15/d18/f35 [768226,98298] 0 2024-03-29T08:07:07.674 INFO:tasks.workunit.client.1.smithi148.stdout:2/73: mknod d7/d13/c1a 0 2024-03-29T08:07:07.674 INFO:tasks.workunit.client.1.smithi148.stdout:9/60: write f3 [6381329,109251] 0 2024-03-29T08:07:07.675 INFO:tasks.workunit.client.1.smithi148.stdout:8/101: rename d17/d18/f22 to d17/f23 0 2024-03-29T08:07:07.677 INFO:tasks.workunit.client.1.smithi148.stdout:7/109: mkdir d8/d29 0 2024-03-29T08:07:07.678 INFO:tasks.workunit.client.0.smithi037.stdout:1/145: chown d2/d6/dc/d18/d1d/l2f 261598 1 2024-03-29T08:07:07.678 INFO:tasks.workunit.client.0.smithi037.stdout:0/85: mknod d9/c1a 0 2024-03-29T08:07:07.680 INFO:tasks.workunit.client.0.smithi037.stdout:0/86: write d9/fb [1073692,189] 0 2024-03-29T08:07:07.682 INFO:tasks.workunit.client.0.smithi037.stdout:4/50: dwrite d3/d4/fa [0,4194304] 0 2024-03-29T08:07:07.683 INFO:tasks.workunit.client.1.smithi148.stdout:4/62: dwrite df/d10/f12 [0,4194304] 0 2024-03-29T08:07:07.686 INFO:tasks.workunit.client.1.smithi148.stdout:0/95: mkdir db/de/df/d11/d1f 0 2024-03-29T08:07:07.687 INFO:tasks.workunit.client.1.smithi148.stdout:5/76: truncate d0/f9 637251 0 2024-03-29T08:07:07.687 INFO:tasks.workunit.client.1.smithi148.stdout:9/61: fsync f3 0 2024-03-29T08:07:07.689 INFO:tasks.workunit.client.1.smithi148.stdout:0/96: write db/dc/f1e [1019903,96025] 0 2024-03-29T08:07:07.689 INFO:tasks.workunit.client.0.smithi037.stdout:2/76: dread d0/f2 [0,4194304] 0 2024-03-29T08:07:07.691 INFO:tasks.workunit.client.0.smithi037.stdout:8/138: dwrite d5/f13 [0,4194304] 0 2024-03-29T08:07:07.692 INFO:tasks.workunit.client.1.smithi148.stdout:1/68: dread d3/fa [0,4194304] 0 2024-03-29T08:07:07.692 INFO:tasks.workunit.client.1.smithi148.stdout:1/69: readlink d3/lf 0 2024-03-29T08:07:07.693 INFO:tasks.workunit.client.0.smithi037.stdout:8/139: chown d5/d10/l20 0 1 2024-03-29T08:07:07.697 INFO:tasks.workunit.client.1.smithi148.stdout:2/74: creat d7/f1b x:0 0 0 2024-03-29T08:07:07.698 INFO:tasks.workunit.client.1.smithi148.stdout:2/75: stat d7/d13/c14 0 2024-03-29T08:07:07.698 INFO:tasks.workunit.client.1.smithi148.stdout:8/102: symlink d17/d18/l24 0 2024-03-29T08:07:07.700 INFO:tasks.workunit.client.0.smithi037.stdout:0/87: mkdir d9/da/d1b 0 2024-03-29T08:07:07.705 INFO:tasks.workunit.client.1.smithi148.stdout:3/71: link c7 de/d13/c19 0 2024-03-29T08:07:07.707 INFO:tasks.workunit.client.0.smithi037.stdout:4/51: mknod d3/d4/cf 0 2024-03-29T08:07:07.707 INFO:tasks.workunit.client.0.smithi037.stdout:8/140: write d5/f1c [1605205,110923] 0 2024-03-29T08:07:07.708 INFO:tasks.workunit.client.1.smithi148.stdout:4/63: symlink df/l16 0 2024-03-29T08:07:07.709 INFO:tasks.workunit.client.0.smithi037.stdout:2/77: mknod d0/c19 0 2024-03-29T08:07:07.710 INFO:tasks.workunit.client.0.smithi037.stdout:2/78: stat d0/f10 0 2024-03-29T08:07:07.713 INFO:tasks.workunit.client.0.smithi037.stdout:3/145: link dd/d15/f19 dd/d15/d2b/f38 0 2024-03-29T08:07:07.713 INFO:tasks.workunit.client.1.smithi148.stdout:5/77: creat d0/f12 x:0 0 0 2024-03-29T08:07:07.714 INFO:tasks.workunit.client.1.smithi148.stdout:1/70: mknod d3/d13/c16 0 2024-03-29T08:07:07.715 INFO:tasks.workunit.client.0.smithi037.stdout:3/146: chown f4 234 1 2024-03-29T08:07:07.716 INFO:tasks.workunit.client.1.smithi148.stdout:2/76: unlink d7/cb 0 2024-03-29T08:07:07.717 INFO:tasks.workunit.client.0.smithi037.stdout:4/52: mknod d3/c10 0 2024-03-29T08:07:07.718 INFO:tasks.workunit.client.0.smithi037.stdout:2/79: mknod d0/c1a 0 2024-03-29T08:07:07.718 INFO:tasks.workunit.client.1.smithi148.stdout:2/77: dread - d7/f17 zero size 2024-03-29T08:07:07.719 INFO:tasks.workunit.client.1.smithi148.stdout:6/70: link c3 d6/c10 0 2024-03-29T08:07:07.721 INFO:tasks.workunit.client.0.smithi037.stdout:2/80: read d0/f7 [1062924,103543] 0 2024-03-29T08:07:07.721 INFO:tasks.workunit.client.1.smithi148.stdout:6/71: fdatasync d6/fd 0 2024-03-29T08:07:07.724 INFO:tasks.workunit.client.1.smithi148.stdout:3/72: dwrite de/f15 [0,4194304] 0 2024-03-29T08:07:07.727 INFO:tasks.workunit.client.1.smithi148.stdout:4/64: mkdir df/d10/d17 0 2024-03-29T08:07:07.727 INFO:tasks.workunit.client.1.smithi148.stdout:9/62: creat db/d10/f13 x:0 0 0 2024-03-29T08:07:07.727 INFO:tasks.workunit.client.1.smithi148.stdout:3/73: fdatasync f1 0 2024-03-29T08:07:07.728 INFO:tasks.workunit.client.0.smithi037.stdout:4/53: mkdir d3/d11 0 2024-03-29T08:07:07.728 INFO:tasks.workunit.client.0.smithi037.stdout:5/118: sync 2024-03-29T08:07:07.728 INFO:tasks.workunit.client.0.smithi037.stdout:6/81: sync 2024-03-29T08:07:07.730 INFO:tasks.workunit.client.1.smithi148.stdout:1/71: unlink d3/f12 0 2024-03-29T08:07:07.730 INFO:tasks.workunit.client.1.smithi148.stdout:4/65: write df/d10/f11 [426012,1878] 0 2024-03-29T08:07:07.731 INFO:tasks.workunit.client.0.smithi037.stdout:2/81: chown d0/c11 35383 1 2024-03-29T08:07:07.732 INFO:tasks.workunit.client.1.smithi148.stdout:4/66: chown f4 3615 1 2024-03-29T08:07:07.733 INFO:tasks.workunit.client.0.smithi037.stdout:1/146: dwrite d2/d6/d13/f28 [0,4194304] 0 2024-03-29T08:07:07.736 INFO:tasks.workunit.client.1.smithi148.stdout:8/103: dread f15 [0,4194304] 0 2024-03-29T08:07:07.737 INFO:tasks.workunit.client.1.smithi148.stdout:0/97: link db/de/df/l13 db/de/df/d14/l20 0 2024-03-29T08:07:07.739 INFO:tasks.workunit.client.0.smithi037.stdout:2/82: write d0/fd [1070839,130262] 0 2024-03-29T08:07:07.741 INFO:tasks.workunit.client.0.smithi037.stdout:5/119: mkdir d8/d18/d24 0 2024-03-29T08:07:07.741 INFO:tasks.workunit.client.1.smithi148.stdout:9/63: mkdir db/d10/d14 0 2024-03-29T08:07:07.741 INFO:tasks.workunit.client.0.smithi037.stdout:0/88: truncate f5 3936381 0 2024-03-29T08:07:07.742 INFO:tasks.workunit.client.0.smithi037.stdout:8/141: truncate f1 2607780 0 2024-03-29T08:07:07.743 INFO:tasks.workunit.client.1.smithi148.stdout:5/78: symlink d0/db/de/l13 0 2024-03-29T08:07:07.746 INFO:tasks.workunit.client.0.smithi037.stdout:6/82: write d0/d5/f14 [5216342,78313] 0 2024-03-29T08:07:07.749 INFO:tasks.workunit.client.0.smithi037.stdout:4/54: rename d3/d4/fe to d3/d4/d8/db/f12 0 2024-03-29T08:07:07.750 INFO:tasks.workunit.client.1.smithi148.stdout:7/110: truncate d8/fc 1621684 0 2024-03-29T08:07:07.751 INFO:tasks.workunit.client.1.smithi148.stdout:8/104: mknod d17/d1d/c25 0 2024-03-29T08:07:07.752 INFO:tasks.workunit.client.0.smithi037.stdout:6/83: dread - d0/d5/d1e/f12 zero size 2024-03-29T08:07:07.754 INFO:tasks.workunit.client.0.smithi037.stdout:1/147: mkdir d2/d6/dc/d18/d1d/d21/d38 0 2024-03-29T08:07:07.755 INFO:tasks.workunit.client.1.smithi148.stdout:7/111: rename d8/d14 to d8/d14/d2a 22 2024-03-29T08:07:07.756 INFO:tasks.workunit.client.0.smithi037.stdout:2/83: rename d0/fe to d0/f1b 0 2024-03-29T08:07:07.757 INFO:tasks.workunit.client.1.smithi148.stdout:4/67: getdents df/d10/d17 0 2024-03-29T08:07:07.758 INFO:tasks.workunit.client.1.smithi148.stdout:7/112: dread - d8/d14/f20 zero size 2024-03-29T08:07:07.759 INFO:tasks.workunit.client.0.smithi037.stdout:6/84: dread d0/d5/fa [0,4194304] 0 2024-03-29T08:07:07.759 INFO:tasks.workunit.client.0.smithi037.stdout:8/142: mkdir d5/d10/d19/d2c 0 2024-03-29T08:07:07.760 INFO:tasks.workunit.client.1.smithi148.stdout:2/78: dwrite f5 [0,4194304] 0 2024-03-29T08:07:07.761 INFO:tasks.workunit.client.1.smithi148.stdout:9/64: dread f7 [0,4194304] 0 2024-03-29T08:07:07.762 INFO:tasks.workunit.client.0.smithi037.stdout:3/147: dwrite dd/d15/f19 [0,4194304] 0 2024-03-29T08:07:07.763 INFO:tasks.workunit.client.0.smithi037.stdout:8/143: write d5/f1e [502261,31945] 0 2024-03-29T08:07:07.763 INFO:tasks.workunit.client.1.smithi148.stdout:1/72: unlink d3/cc 0 2024-03-29T08:07:07.763 INFO:tasks.workunit.client.1.smithi148.stdout:9/65: fdatasync db/fd 0 2024-03-29T08:07:07.765 INFO:tasks.workunit.client.1.smithi148.stdout:2/79: write d7/f1b [581966,11565] 0 2024-03-29T08:07:07.766 INFO:tasks.workunit.client.0.smithi037.stdout:5/120: rename f3 to d8/d22/f25 0 2024-03-29T08:07:07.766 INFO:tasks.workunit.client.1.smithi148.stdout:3/74: dwrite f5 [4194304,4194304] 0 2024-03-29T08:07:07.767 INFO:tasks.workunit.client.0.smithi037.stdout:1/148: rename d2 to d2/d6/dc/d18/d39 22 2024-03-29T08:07:07.768 INFO:tasks.workunit.client.0.smithi037.stdout:5/121: chown d8/ff 25032 1 2024-03-29T08:07:07.771 INFO:tasks.workunit.client.1.smithi148.stdout:4/68: symlink df/d10/l18 0 2024-03-29T08:07:07.772 INFO:tasks.workunit.client.1.smithi148.stdout:4/69: read fd [982389,50892] 0 2024-03-29T08:07:07.772 INFO:tasks.workunit.client.1.smithi148.stdout:6/72: truncate d6/fd 247695 0 2024-03-29T08:07:07.773 INFO:tasks.workunit.client.1.smithi148.stdout:7/113: truncate d8/d14/f20 357835 0 2024-03-29T08:07:07.773 INFO:tasks.workunit.client.1.smithi148.stdout:4/70: stat f1 0 2024-03-29T08:07:07.775 INFO:tasks.workunit.client.1.smithi148.stdout:7/114: chown f4 155299 1 2024-03-29T08:07:07.775 INFO:tasks.workunit.client.1.smithi148.stdout:9/66: write f7 [4255168,90556] 0 2024-03-29T08:07:07.775 INFO:tasks.workunit.client.1.smithi148.stdout:2/80: write d7/da/fe [361974,74758] 0 2024-03-29T08:07:07.777 INFO:tasks.workunit.client.1.smithi148.stdout:9/67: fdatasync f3 0 2024-03-29T08:07:07.778 INFO:tasks.workunit.client.0.smithi037.stdout:4/55: creat d3/d4/d8/f13 x:0 0 0 2024-03-29T08:07:07.797 INFO:tasks.workunit.client.0.smithi037.stdout:9/200: sync 2024-03-29T08:07:07.797 INFO:tasks.workunit.client.0.smithi037.stdout:7/170: sync 2024-03-29T08:07:07.797 INFO:tasks.workunit.client.0.smithi037.stdout:1/149: creat d2/f3a x:0 0 0 2024-03-29T08:07:07.798 INFO:tasks.workunit.client.1.smithi148.stdout:7/115: fdatasync d8/d14/f21 0 2024-03-29T08:07:07.799 INFO:tasks.workunit.client.0.smithi037.stdout:8/144: dwrite d5/d10/f11 [4194304,4194304] 0 2024-03-29T08:07:07.799 INFO:tasks.workunit.client.1.smithi148.stdout:3/75: rename cc to de/c1a 0 2024-03-29T08:07:07.800 INFO:tasks.workunit.client.1.smithi148.stdout:7/116: chown d8/d29 865022 1 2024-03-29T08:07:07.804 INFO:tasks.workunit.client.1.smithi148.stdout:6/73: creat d6/f11 x:0 0 0 2024-03-29T08:07:07.805 INFO:tasks.workunit.client.1.smithi148.stdout:7/117: read d8/d14/f16 [217903,73782] 0 2024-03-29T08:07:07.807 INFO:tasks.workunit.client.1.smithi148.stdout:5/79: dread d0/f9 [0,4194304] 0 2024-03-29T08:07:07.809 INFO:tasks.workunit.client.0.smithi037.stdout:8/145: write d5/d25/f28 [505894,81481] 0 2024-03-29T08:07:07.810 INFO:tasks.workunit.client.0.smithi037.stdout:7/171: creat dd/d21/f36 x:0 0 0 2024-03-29T08:07:07.814 INFO:tasks.workunit.client.0.smithi037.stdout:9/201: unlink d1/d4/f41 0 2024-03-29T08:07:07.815 INFO:tasks.workunit.client.1.smithi148.stdout:8/105: dwrite f15 [0,4194304] 0 2024-03-29T08:07:07.818 INFO:tasks.workunit.client.0.smithi037.stdout:0/89: getdents d9/da 0 2024-03-29T08:07:07.819 INFO:tasks.workunit.client.0.smithi037.stdout:2/84: link d0/c13 d0/c1c 0 2024-03-29T08:07:07.819 INFO:tasks.workunit.client.1.smithi148.stdout:3/76: dread f5 [4194304,4194304] 0 2024-03-29T08:07:07.822 INFO:tasks.workunit.client.1.smithi148.stdout:3/77: truncate f1 4871227 0 2024-03-29T08:07:07.823 INFO:tasks.workunit.client.0.smithi037.stdout:6/85: truncate d0/d1/ff 1712341 0 2024-03-29T08:07:07.823 INFO:tasks.workunit.client.0.smithi037.stdout:3/148: creat dd/f39 x:0 0 0 2024-03-29T08:07:07.826 INFO:tasks.workunit.client.1.smithi148.stdout:4/71: dwrite fa [0,4194304] 0 2024-03-29T08:07:07.826 INFO:tasks.workunit.client.0.smithi037.stdout:9/202: chown d1/d4/d19/d1e/d25/d10/d31/l42 1157901181 1 2024-03-29T08:07:07.829 INFO:tasks.workunit.client.1.smithi148.stdout:7/118: dwrite d8/f18 [0,4194304] 0 2024-03-29T08:07:07.832 INFO:tasks.workunit.client.0.smithi037.stdout:8/146: mkdir d5/d10/d2d 0 2024-03-29T08:07:07.834 INFO:tasks.workunit.client.0.smithi037.stdout:2/85: rmdir d0 39 2024-03-29T08:07:07.835 INFO:tasks.workunit.client.1.smithi148.stdout:0/98: sync 2024-03-29T08:07:07.840 INFO:tasks.workunit.client.0.smithi037.stdout:8/147: fsync d5/fb 0 2024-03-29T08:07:07.841 INFO:tasks.workunit.client.1.smithi148.stdout:3/78: dwrite f8 [0,4194304] 0 2024-03-29T08:07:07.842 INFO:tasks.workunit.client.1.smithi148.stdout:3/79: write de/f15 [3209809,102313] 0 2024-03-29T08:07:07.844 INFO:tasks.workunit.client.0.smithi037.stdout:9/203: mknod d1/d4/d19/d1e/d25/d10/d23/c45 0 2024-03-29T08:07:07.849 INFO:tasks.workunit.client.0.smithi037.stdout:2/86: write d0/f10 [970585,98664] 0 2024-03-29T08:07:07.849 INFO:tasks.workunit.client.0.smithi037.stdout:5/122: dwrite d8/d22/f25 [0,4194304] 0 2024-03-29T08:07:07.855 INFO:tasks.workunit.client.0.smithi037.stdout:4/56: link c1 d3/d4/c14 0 2024-03-29T08:07:07.858 INFO:tasks.workunit.client.1.smithi148.stdout:3/80: dread f8 [0,4194304] 0 2024-03-29T08:07:07.860 INFO:tasks.workunit.client.1.smithi148.stdout:3/81: write f8 [2119608,91247] 0 2024-03-29T08:07:07.864 INFO:tasks.workunit.client.0.smithi037.stdout:2/87: write d0/f1b [582824,8663] 0 2024-03-29T08:07:07.868 INFO:tasks.workunit.client.0.smithi037.stdout:7/172: creat dd/d1d/d34/f37 x:0 0 0 2024-03-29T08:07:07.868 INFO:tasks.workunit.client.0.smithi037.stdout:0/90: dread d9/fb [0,4194304] 0 2024-03-29T08:07:07.868 INFO:tasks.workunit.client.0.smithi037.stdout:2/88: dread - d0/fb zero size 2024-03-29T08:07:07.869 INFO:tasks.workunit.client.0.smithi037.stdout:4/57: mkdir d3/d4/d15 0 2024-03-29T08:07:07.875 INFO:tasks.workunit.client.0.smithi037.stdout:7/173: mknod dd/c38 0 2024-03-29T08:07:07.878 INFO:tasks.workunit.client.0.smithi037.stdout:6/86: dwrite d0/d5/f9 [0,4194304] 0 2024-03-29T08:07:07.889 INFO:tasks.workunit.client.0.smithi037.stdout:2/89: rename d0/c11 to d0/c1d 0 2024-03-29T08:07:07.889 INFO:tasks.workunit.client.0.smithi037.stdout:4/58: creat d3/d4/d8/db/f16 x:0 0 0 2024-03-29T08:07:07.889 INFO:tasks.workunit.client.0.smithi037.stdout:2/90: readlink d0/l12 0 2024-03-29T08:07:07.891 INFO:tasks.workunit.client.0.smithi037.stdout:0/91: rename d9/da/d15 to d9/da/dd/d1c 0 2024-03-29T08:07:07.891 INFO:tasks.workunit.client.0.smithi037.stdout:9/204: dread d1/d3/fa [4194304,4194304] 0 2024-03-29T08:07:07.891 INFO:tasks.workunit.client.0.smithi037.stdout:2/91: fsync d0/f10 0 2024-03-29T08:07:07.892 INFO:tasks.workunit.client.0.smithi037.stdout:5/123: dwrite d8/f1e [0,4194304] 0 2024-03-29T08:07:07.895 INFO:tasks.workunit.client.0.smithi037.stdout:5/124: dread - d8/f1f zero size 2024-03-29T08:07:07.900 INFO:tasks.workunit.client.0.smithi037.stdout:3/149: dread f5 [0,4194304] 0 2024-03-29T08:07:07.901 INFO:tasks.workunit.client.0.smithi037.stdout:2/92: unlink d0/f10 0 2024-03-29T08:07:07.909 INFO:tasks.workunit.client.0.smithi037.stdout:9/205: creat d1/d1f/d3a/f46 x:0 0 0 2024-03-29T08:07:07.915 INFO:tasks.workunit.client.0.smithi037.stdout:9/206: write d1/d1f/d3a/f27 [1855003,129393] 0 2024-03-29T08:07:07.915 INFO:tasks.workunit.client.0.smithi037.stdout:2/93: mkdir d0/d1e 0 2024-03-29T08:07:07.918 INFO:tasks.workunit.client.0.smithi037.stdout:9/207: stat d1/d4/d19/l35 0 2024-03-29T08:07:07.923 INFO:tasks.workunit.client.0.smithi037.stdout:4/59: dwrite d3/d4/d8/db/f12 [0,4194304] 0 2024-03-29T08:07:07.924 INFO:tasks.workunit.client.0.smithi037.stdout:3/150: mkdir dd/d15/d2b/d2d/d3a 0 2024-03-29T08:07:07.928 INFO:tasks.workunit.client.0.smithi037.stdout:9/208: dread - d1/d1f/d3a/f46 zero size 2024-03-29T08:07:07.937 INFO:tasks.workunit.client.0.smithi037.stdout:4/60: symlink d3/d4/d8/l17 0 2024-03-29T08:07:07.939 INFO:tasks.workunit.client.0.smithi037.stdout:7/174: dread dd/ff [0,4194304] 0 2024-03-29T08:07:07.939 INFO:tasks.workunit.client.0.smithi037.stdout:3/151: mkdir dd/d15/d2b/d2d/d3b 0 2024-03-29T08:07:07.944 INFO:tasks.workunit.client.0.smithi037.stdout:4/61: symlink d3/d4/l18 0 2024-03-29T08:07:07.945 INFO:tasks.workunit.client.0.smithi037.stdout:5/125: dwrite d8/d18/f19 [0,4194304] 0 2024-03-29T08:07:07.947 INFO:tasks.workunit.client.0.smithi037.stdout:2/94: dread d0/f18 [0,4194304] 0 2024-03-29T08:07:07.949 INFO:tasks.workunit.client.0.smithi037.stdout:2/95: readlink d0/l12 0 2024-03-29T08:07:07.950 INFO:tasks.workunit.client.0.smithi037.stdout:9/209: rename d1/d4/lc to d1/d4/l47 0 2024-03-29T08:07:07.954 INFO:tasks.workunit.client.0.smithi037.stdout:9/210: dread - d1/d4/d19/d1e/d25/d10/d23/f39 zero size 2024-03-29T08:07:07.955 INFO:tasks.workunit.client.0.smithi037.stdout:3/152: creat dd/d15/d2b/f3c x:0 0 0 2024-03-29T08:07:07.957 INFO:tasks.workunit.client.0.smithi037.stdout:3/153: stat dd/d15/d18 0 2024-03-29T08:07:07.958 INFO:tasks.workunit.client.0.smithi037.stdout:3/154: chown dd/d15/d27/l29 2 1 2024-03-29T08:07:07.960 INFO:tasks.workunit.client.0.smithi037.stdout:7/175: truncate f7 2033776 0 2024-03-29T08:07:07.968 INFO:tasks.workunit.client.0.smithi037.stdout:9/211: rmdir d1/d1f 39 2024-03-29T08:07:07.970 INFO:tasks.workunit.client.0.smithi037.stdout:3/155: mkdir dd/d15/d2b/d2d/d3d 0 2024-03-29T08:07:07.972 INFO:tasks.workunit.client.0.smithi037.stdout:4/62: creat d3/d4/d15/f19 x:0 0 0 2024-03-29T08:07:07.973 INFO:tasks.workunit.client.0.smithi037.stdout:3/156: write dd/d15/d18/f35 [707445,62310] 0 2024-03-29T08:07:07.977 INFO:tasks.workunit.client.0.smithi037.stdout:0/92: rename d9/da/l12 to d9/da/l1d 0 2024-03-29T08:07:07.980 INFO:tasks.workunit.client.0.smithi037.stdout:9/212: fsync d1/d3/fa 0 2024-03-29T08:07:07.982 INFO:tasks.workunit.client.0.smithi037.stdout:2/96: write d0/f14 [1467219,125976] 0 2024-03-29T08:07:07.982 INFO:tasks.workunit.client.0.smithi037.stdout:3/157: chown dd/d15/l1c 793 1 2024-03-29T08:07:07.989 INFO:tasks.workunit.client.0.smithi037.stdout:2/97: creat d0/f1f x:0 0 0 2024-03-29T08:07:07.989 INFO:tasks.workunit.client.0.smithi037.stdout:7/176: dwrite dd/f1b [0,4194304] 0 2024-03-29T08:07:07.990 INFO:tasks.workunit.client.0.smithi037.stdout:3/158: write dd/d15/d2b/f2c [914593,116945] 0 2024-03-29T08:07:07.991 INFO:tasks.workunit.client.0.smithi037.stdout:0/93: readlink d9/da/l1d 0 2024-03-29T08:07:07.993 INFO:tasks.workunit.client.0.smithi037.stdout:2/98: chown d0/l12 1987142 1 2024-03-29T08:07:07.998 INFO:tasks.workunit.client.0.smithi037.stdout:5/126: dwrite d8/ff [0,4194304] 0 2024-03-29T08:07:08.015 INFO:tasks.workunit.client.0.smithi037.stdout:2/99: write d0/fd [4618181,66185] 0 2024-03-29T08:07:08.020 INFO:tasks.workunit.client.0.smithi037.stdout:9/213: dwrite d1/d4/f28 [0,4194304] 0 2024-03-29T08:07:08.020 INFO:tasks.workunit.client.0.smithi037.stdout:2/100: mkdir d0/d20 0 2024-03-29T08:07:08.022 INFO:tasks.workunit.client.0.smithi037.stdout:0/94: link d9/da/le d9/da/dd/l1e 0 2024-03-29T08:07:08.028 INFO:tasks.workunit.client.0.smithi037.stdout:2/101: getdents d0/d1e 0 2024-03-29T08:07:08.028 INFO:tasks.workunit.client.0.smithi037.stdout:9/214: stat d1/d4/d19/d1e/d25/d10/c20 0 2024-03-29T08:07:08.029 INFO:tasks.workunit.client.0.smithi037.stdout:0/95: symlink d9/da/l1f 0 2024-03-29T08:07:08.031 INFO:tasks.workunit.client.0.smithi037.stdout:3/159: dread f2 [0,4194304] 0 2024-03-29T08:07:08.033 INFO:tasks.workunit.client.0.smithi037.stdout:2/102: stat d0/f2 0 2024-03-29T08:07:08.041 INFO:tasks.workunit.client.0.smithi037.stdout:5/127: dread d8/f17 [0,4194304] 0 2024-03-29T08:07:08.045 INFO:tasks.workunit.client.0.smithi037.stdout:3/160: mkdir dd/d15/d2b/d3e 0 2024-03-29T08:07:08.046 INFO:tasks.workunit.client.0.smithi037.stdout:7/177: dwrite f7 [0,4194304] 0 2024-03-29T08:07:08.049 INFO:tasks.workunit.client.0.smithi037.stdout:5/128: creat d8/d22/f26 x:0 0 0 2024-03-29T08:07:08.050 INFO:tasks.workunit.client.0.smithi037.stdout:3/161: readlink dd/d15/l1c 0 2024-03-29T08:07:08.057 INFO:tasks.workunit.client.0.smithi037.stdout:5/129: creat d8/d22/f27 x:0 0 0 2024-03-29T08:07:08.063 INFO:tasks.workunit.client.0.smithi037.stdout:5/130: readlink d8/ld 0 2024-03-29T08:07:08.064 INFO:tasks.workunit.client.0.smithi037.stdout:7/178: getdents dd 0 2024-03-29T08:07:08.065 INFO:tasks.workunit.client.0.smithi037.stdout:3/162: truncate dd/d15/d2b/f37 249261 0 2024-03-29T08:07:08.074 INFO:tasks.workunit.client.0.smithi037.stdout:9/215: dwrite d1/d4/d19/d1e/d25/d10/d31/f38 [0,4194304] 0 2024-03-29T08:07:08.076 INFO:tasks.workunit.client.0.smithi037.stdout:2/103: dwrite d0/fd [0,4194304] 0 2024-03-29T08:07:08.076 INFO:tasks.workunit.client.0.smithi037.stdout:5/131: creat d8/d18/f28 x:0 0 0 2024-03-29T08:07:08.076 INFO:tasks.workunit.client.1.smithi148.stdout:2/81: mkdir d7/da/d1c 0 2024-03-29T08:07:08.076 INFO:tasks.workunit.client.0.smithi037.stdout:0/96: dwrite d9/da/dd/d1c/f18 [0,4194304] 0 2024-03-29T08:07:08.079 INFO:tasks.workunit.client.0.smithi037.stdout:3/163: creat dd/f3f x:0 0 0 2024-03-29T08:07:08.080 INFO:tasks.workunit.client.0.smithi037.stdout:3/164: fdatasync fc 0 2024-03-29T08:07:08.081 INFO:tasks.workunit.client.0.smithi037.stdout:9/216: chown d1/d4/c11 91325052 1 2024-03-29T08:07:08.081 INFO:tasks.workunit.client.1.smithi148.stdout:6/74: truncate f5 1407198 0 2024-03-29T08:07:08.083 INFO:tasks.workunit.client.1.smithi148.stdout:5/80: symlink d0/db/l14 0 2024-03-29T08:07:08.083 INFO:tasks.workunit.client.0.smithi037.stdout:5/132: mknod d8/d18/d24/c29 0 2024-03-29T08:07:08.084 INFO:tasks.workunit.client.1.smithi148.stdout:8/106: fsync fe 0 2024-03-29T08:07:08.085 INFO:tasks.workunit.client.0.smithi037.stdout:7/179: link dd/d1d/f2a dd/d1d/d34/f39 0 2024-03-29T08:07:08.085 INFO:tasks.workunit.client.0.smithi037.stdout:3/165: write dd/f1f [1147089,55804] 0 2024-03-29T08:07:08.087 INFO:tasks.workunit.client.1.smithi148.stdout:6/75: chown d6/fe 449288938 1 2024-03-29T08:07:08.087 INFO:tasks.workunit.client.1.smithi148.stdout:4/72: rename df/d10/l18 to df/d10/d17/l19 0 2024-03-29T08:07:08.088 INFO:tasks.workunit.client.1.smithi148.stdout:3/82: rename de/d13 to de/d13/d1b 22 2024-03-29T08:07:08.088 INFO:tasks.workunit.client.0.smithi037.stdout:3/166: write fb [124820,79699] 0 2024-03-29T08:07:08.089 INFO:tasks.workunit.client.1.smithi148.stdout:7/119: creat d8/d14/f2b x:0 0 0 2024-03-29T08:07:08.090 INFO:tasks.workunit.client.1.smithi148.stdout:3/83: stat de/d13/c18 0 2024-03-29T08:07:08.091 INFO:tasks.workunit.client.1.smithi148.stdout:3/84: stat cd 0 2024-03-29T08:07:08.093 INFO:tasks.workunit.client.0.smithi037.stdout:9/217: rename d1/d3/fa to d1/d4/d19/d1e/d25/d10/d31/f48 0 2024-03-29T08:07:08.093 INFO:tasks.workunit.client.1.smithi148.stdout:9/68: getdents db/d10/d14 0 2024-03-29T08:07:08.094 INFO:tasks.workunit.client.0.smithi037.stdout:3/167: symlink dd/d15/d2b/l40 0 2024-03-29T08:07:08.095 INFO:tasks.workunit.client.1.smithi148.stdout:1/73: link d3/l6 d3/l17 0 2024-03-29T08:07:08.098 INFO:tasks.workunit.client.0.smithi037.stdout:7/180: rename dd/c25 to dd/d21/c3a 0 2024-03-29T08:07:08.101 INFO:tasks.workunit.client.0.smithi037.stdout:3/168: unlink dd/f1d 0 2024-03-29T08:07:08.101 INFO:tasks.workunit.client.1.smithi148.stdout:9/69: write db/fd [3463703,17924] 0 2024-03-29T08:07:08.103 INFO:tasks.workunit.client.1.smithi148.stdout:7/120: creat d8/d14/f2c x:0 0 0 2024-03-29T08:07:08.103 INFO:tasks.workunit.client.1.smithi148.stdout:4/73: creat df/f1a x:0 0 0 2024-03-29T08:07:08.103 INFO:tasks.workunit.client.1.smithi148.stdout:3/85: creat de/f1c x:0 0 0 2024-03-29T08:07:08.104 INFO:tasks.workunit.client.0.smithi037.stdout:3/169: chown dd/l17 82949878 1 2024-03-29T08:07:08.105 INFO:tasks.workunit.client.1.smithi148.stdout:3/86: dread - de/f1c zero size 2024-03-29T08:07:08.109 INFO:tasks.workunit.client.1.smithi148.stdout:2/82: getdents d7/d13 0 2024-03-29T08:07:08.109 INFO:tasks.workunit.client.1.smithi148.stdout:3/87: readlink de/lf 0 2024-03-29T08:07:08.109 INFO:tasks.workunit.client.1.smithi148.stdout:9/70: rename db/fd to db/d10/f15 0 2024-03-29T08:07:08.110 INFO:tasks.workunit.client.1.smithi148.stdout:4/74: mknod df/d10/c1b 0 2024-03-29T08:07:08.111 INFO:tasks.workunit.client.0.smithi037.stdout:3/170: creat dd/d15/d27/f41 x:0 0 0 2024-03-29T08:07:08.112 INFO:tasks.workunit.client.1.smithi148.stdout:1/74: write d3/d13/f15 [125876,81480] 0 2024-03-29T08:07:08.112 INFO:tasks.workunit.client.1.smithi148.stdout:7/121: creat d8/d14/f2d x:0 0 0 2024-03-29T08:07:08.114 INFO:tasks.workunit.client.1.smithi148.stdout:7/122: write d8/d14/f2c [303112,128260] 0 2024-03-29T08:07:08.114 INFO:tasks.workunit.client.0.smithi037.stdout:5/133: dwrite d8/d22/f26 [0,4194304] 0 2024-03-29T08:07:08.118 INFO:tasks.workunit.client.0.smithi037.stdout:3/171: write dd/d15/d2b/f37 [419174,29455] 0 2024-03-29T08:07:08.119 INFO:tasks.workunit.client.1.smithi148.stdout:1/75: dread d3/fa [0,4194304] 0 2024-03-29T08:07:08.120 INFO:tasks.workunit.client.1.smithi148.stdout:3/88: rename de/l12 to de/l1d 0 2024-03-29T08:07:08.122 INFO:tasks.workunit.client.1.smithi148.stdout:3/89: read f1 [4422747,29416] 0 2024-03-29T08:07:08.124 INFO:tasks.workunit.client.1.smithi148.stdout:2/83: symlink d7/da/l1d 0 2024-03-29T08:07:08.130 INFO:tasks.workunit.client.0.smithi037.stdout:9/218: dwrite d1/d4/d19/d1e/d25/d10/f12 [0,4194304] 0 2024-03-29T08:07:08.131 INFO:tasks.workunit.client.1.smithi148.stdout:9/71: symlink db/l16 0 2024-03-29T08:07:08.132 INFO:tasks.workunit.client.0.smithi037.stdout:7/181: mkdir dd/d21/d35/d3b 0 2024-03-29T08:07:08.136 INFO:tasks.workunit.client.0.smithi037.stdout:3/172: truncate dd/d15/d2b/f37 381150 0 2024-03-29T08:07:08.140 INFO:tasks.workunit.client.1.smithi148.stdout:3/90: unlink ca 0 2024-03-29T08:07:08.141 INFO:tasks.workunit.client.0.smithi037.stdout:5/134: dread d8/fa [0,4194304] 0 2024-03-29T08:07:08.141 INFO:tasks.workunit.client.1.smithi148.stdout:3/91: chown de/lf 0 1 2024-03-29T08:07:08.143 INFO:tasks.workunit.client.1.smithi148.stdout:2/84: symlink d7/d13/l1e 0 2024-03-29T08:07:08.144 INFO:tasks.workunit.client.0.smithi037.stdout:3/173: creat dd/d15/d2b/d2d/f42 x:0 0 0 2024-03-29T08:07:08.144 INFO:tasks.workunit.client.0.smithi037.stdout:5/135: chown d8/d22/f25 876919776 1 2024-03-29T08:07:08.148 INFO:tasks.workunit.client.1.smithi148.stdout:9/72: write db/d10/f15 [2046518,81793] 0 2024-03-29T08:07:08.150 INFO:tasks.workunit.client.0.smithi037.stdout:5/136: fdatasync d8/d18/f19 0 2024-03-29T08:07:08.151 INFO:tasks.workunit.client.0.smithi037.stdout:7/182: dread dd/f17 [0,4194304] 0 2024-03-29T08:07:08.155 INFO:tasks.workunit.client.0.smithi037.stdout:3/174: rename dd/d15/d27/f36 to dd/d32/f43 0 2024-03-29T08:07:08.156 INFO:tasks.workunit.client.1.smithi148.stdout:2/85: link f5 d7/da/f1f 0 2024-03-29T08:07:08.156 INFO:tasks.workunit.client.0.smithi037.stdout:5/137: creat d8/d18/d24/f2a x:0 0 0 2024-03-29T08:07:08.156 INFO:tasks.workunit.client.0.smithi037.stdout:3/175: fdatasync dd/d15/f19 0 2024-03-29T08:07:08.157 INFO:tasks.workunit.client.0.smithi037.stdout:9/219: dwrite d1/d4/d19/d1e/d25/d10/f12 [0,4194304] 0 2024-03-29T08:07:08.158 INFO:tasks.workunit.client.0.smithi037.stdout:5/138: write d8/f1f [529129,128751] 0 2024-03-29T08:07:08.159 INFO:tasks.workunit.client.1.smithi148.stdout:9/73: mknod db/d10/d14/c17 0 2024-03-29T08:07:08.159 INFO:tasks.workunit.client.1.smithi148.stdout:9/74: readlink db/lf 0 2024-03-29T08:07:08.160 INFO:tasks.workunit.client.1.smithi148.stdout:9/75: dread - db/d10/f13 zero size 2024-03-29T08:07:08.161 INFO:tasks.workunit.client.0.smithi037.stdout:3/176: rmdir dd/d32 39 2024-03-29T08:07:08.162 INFO:tasks.workunit.client.0.smithi037.stdout:9/220: dread - d1/d1f/d3a/f46 zero size 2024-03-29T08:07:08.163 INFO:tasks.workunit.client.0.smithi037.stdout:3/177: fdatasync f5 0 2024-03-29T08:07:08.166 INFO:tasks.workunit.client.1.smithi148.stdout:9/76: creat db/d10/d14/f18 x:0 0 0 2024-03-29T08:07:08.168 INFO:tasks.workunit.client.1.smithi148.stdout:2/86: link c3 d7/d13/c20 0 2024-03-29T08:07:08.170 INFO:tasks.workunit.client.0.smithi037.stdout:9/221: mkdir d1/d4/d19/d1e/d25/d49 0 2024-03-29T08:07:08.171 INFO:tasks.workunit.client.1.smithi148.stdout:9/77: symlink db/d10/d14/l19 0 2024-03-29T08:07:08.172 INFO:tasks.workunit.client.1.smithi148.stdout:2/87: mkdir d7/da/d1c/d21 0 2024-03-29T08:07:08.173 INFO:tasks.workunit.client.1.smithi148.stdout:9/78: write db/d10/f15 [142839,57275] 0 2024-03-29T08:07:08.176 INFO:tasks.workunit.client.1.smithi148.stdout:2/88: rmdir d7/da/d1c 39 2024-03-29T08:07:08.177 INFO:tasks.workunit.client.0.smithi037.stdout:9/222: chown d1/d4/d19/d1e/d25/d10/d23/f39 299273 1 2024-03-29T08:07:08.177 INFO:tasks.workunit.client.1.smithi148.stdout:9/79: mkdir db/d10/d1a 0 2024-03-29T08:07:08.177 INFO:tasks.workunit.client.0.smithi037.stdout:3/178: rename dd/d32/f43 to dd/d15/f44 0 2024-03-29T08:07:08.178 INFO:tasks.workunit.client.0.smithi037.stdout:7/183: dwrite dd/d21/f2c [0,4194304] 0 2024-03-29T08:07:08.179 INFO:tasks.workunit.client.0.smithi037.stdout:9/223: readlink d1/l5 0 2024-03-29T08:07:08.180 INFO:tasks.workunit.client.0.smithi037.stdout:5/139: truncate d8/f1f 584622 0 2024-03-29T08:07:08.183 INFO:tasks.workunit.client.0.smithi037.stdout:7/184: write f7 [4315807,27754] 0 2024-03-29T08:07:08.185 INFO:tasks.workunit.client.0.smithi037.stdout:5/140: creat d8/f2b x:0 0 0 2024-03-29T08:07:08.186 INFO:tasks.workunit.client.0.smithi037.stdout:9/224: mkdir d1/d4/d19/d1e/d4a 0 2024-03-29T08:07:08.186 INFO:tasks.workunit.client.1.smithi148.stdout:9/80: rmdir db/d12 0 2024-03-29T08:07:08.191 INFO:tasks.workunit.client.1.smithi148.stdout:9/81: creat db/d10/d1a/f1b x:0 0 0 2024-03-29T08:07:08.193 INFO:tasks.workunit.client.0.smithi037.stdout:7/185: rename f7 to dd/f3c 0 2024-03-29T08:07:08.193 INFO:tasks.workunit.client.1.smithi148.stdout:9/82: write db/d10/f15 [1902467,95100] 0 2024-03-29T08:07:08.198 INFO:tasks.workunit.client.0.smithi037.stdout:3/179: dread f5 [0,4194304] 0 2024-03-29T08:07:08.199 INFO:tasks.workunit.client.0.smithi037.stdout:9/225: rename d1/d4/d19/d1e/d25/d49 to d1/d4/d19/d1e/d25/d10/d4b 0 2024-03-29T08:07:08.200 INFO:tasks.workunit.client.0.smithi037.stdout:7/186: mknod dd/d1d/d34/c3d 0 2024-03-29T08:07:08.204 INFO:tasks.workunit.client.0.smithi037.stdout:5/141: truncate d8/d18/f21 1201578 0 2024-03-29T08:07:08.205 INFO:tasks.workunit.client.0.smithi037.stdout:9/226: dread - d1/d4/d19/d1e/d25/d10/d23/f39 zero size 2024-03-29T08:07:08.207 INFO:tasks.workunit.client.0.smithi037.stdout:5/142: read d8/f1f [380178,33805] 0 2024-03-29T08:07:08.207 INFO:tasks.workunit.client.0.smithi037.stdout:3/180: creat dd/d15/d2b/d2d/d3d/f45 x:0 0 0 2024-03-29T08:07:08.210 INFO:tasks.workunit.client.0.smithi037.stdout:7/187: creat dd/f3e x:0 0 0 2024-03-29T08:07:08.210 INFO:tasks.workunit.client.0.smithi037.stdout:7/188: fdatasync f9 0 2024-03-29T08:07:08.213 INFO:tasks.workunit.client.0.smithi037.stdout:9/227: creat d1/f4c x:0 0 0 2024-03-29T08:07:08.216 INFO:tasks.workunit.client.0.smithi037.stdout:5/143: getdents d8/d18 0 2024-03-29T08:07:08.217 INFO:tasks.workunit.client.0.smithi037.stdout:9/228: chown d1/d4/d19/d1e/d25/d10/f12 1672113 1 2024-03-29T08:07:08.223 INFO:tasks.workunit.client.0.smithi037.stdout:5/144: rename d8/c16 to d8/d18/c2c 0 2024-03-29T08:07:08.223 INFO:tasks.workunit.client.0.smithi037.stdout:9/229: stat d1/d4/d19/d1e/d25/d10/d23 0 2024-03-29T08:07:08.227 INFO:tasks.workunit.client.0.smithi037.stdout:5/145: creat d8/d22/f2d x:0 0 0 2024-03-29T08:07:08.227 INFO:tasks.workunit.client.0.smithi037.stdout:3/181: dwrite f5 [4194304,4194304] 0 2024-03-29T08:07:08.229 INFO:tasks.workunit.client.0.smithi037.stdout:9/230: creat d1/d4/d19/d1e/d25/d10/d31/f4d x:0 0 0 2024-03-29T08:07:08.230 INFO:tasks.workunit.client.1.smithi148.stdout:8/107: sync 2024-03-29T08:07:08.230 INFO:tasks.workunit.client.1.smithi148.stdout:3/92: sync 2024-03-29T08:07:08.230 INFO:tasks.workunit.client.1.smithi148.stdout:7/123: sync 2024-03-29T08:07:08.231 INFO:tasks.workunit.client.0.smithi037.stdout:5/146: mknod d8/d18/d24/c2e 0 2024-03-29T08:07:08.232 INFO:tasks.workunit.client.0.smithi037.stdout:5/147: fsync d8/d18/f28 0 2024-03-29T08:07:08.235 INFO:tasks.workunit.client.0.smithi037.stdout:9/231: rmdir d1/d1f 39 2024-03-29T08:07:08.236 INFO:tasks.workunit.client.1.smithi148.stdout:8/108: truncate f7 871333 0 2024-03-29T08:07:08.237 INFO:tasks.workunit.client.0.smithi037.stdout:5/148: write d8/d22/f25 [3633581,67624] 0 2024-03-29T08:07:08.239 INFO:tasks.workunit.client.0.smithi037.stdout:9/232: fsync d1/d4/d19/d1e/f3f 0 2024-03-29T08:07:08.240 INFO:tasks.workunit.client.1.smithi148.stdout:8/109: mkdir d17/d18/d26 0 2024-03-29T08:07:08.242 INFO:tasks.workunit.client.1.smithi148.stdout:7/124: dread d8/f10 [0,4194304] 0 2024-03-29T08:07:08.243 INFO:tasks.workunit.client.1.smithi148.stdout:7/125: chown d8/fe 1 1 2024-03-29T08:07:08.243 INFO:tasks.workunit.client.0.smithi037.stdout:3/182: rename dd/d15/d2b to dd/d46 0 2024-03-29T08:07:08.247 INFO:tasks.workunit.client.0.smithi037.stdout:9/233: write d1/d4/d19/d1e/d25/d10/f12 [926127,917] 0 2024-03-29T08:07:08.250 INFO:tasks.workunit.client.1.smithi148.stdout:8/110: creat d17/d18/d26/f27 x:0 0 0 2024-03-29T08:07:08.252 INFO:tasks.workunit.client.1.smithi148.stdout:8/111: write d17/d18/f1c [4415645,102635] 0 2024-03-29T08:07:08.252 INFO:tasks.workunit.client.1.smithi148.stdout:7/126: link d8/f1f d8/d29/f2e 0 2024-03-29T08:07:08.253 INFO:tasks.workunit.client.0.smithi037.stdout:5/149: truncate d8/d18/f19 3376663 0 2024-03-29T08:07:08.254 INFO:tasks.workunit.client.0.smithi037.stdout:3/183: mkdir dd/d46/d3e/d47 0 2024-03-29T08:07:08.257 INFO:tasks.workunit.client.1.smithi148.stdout:7/127: mkdir d8/d2f 0 2024-03-29T08:07:08.259 INFO:tasks.workunit.client.0.smithi037.stdout:5/150: rename d8/lc to d8/d22/l2f 0 2024-03-29T08:07:08.259 INFO:tasks.workunit.client.1.smithi148.stdout:8/112: creat d17/f28 x:0 0 0 2024-03-29T08:07:08.262 INFO:tasks.workunit.client.1.smithi148.stdout:7/128: creat d8/d14/f30 x:0 0 0 2024-03-29T08:07:08.262 INFO:tasks.workunit.client.0.smithi037.stdout:9/234: truncate d1/d3/f21 953113 0 2024-03-29T08:07:08.264 INFO:tasks.workunit.client.0.smithi037.stdout:5/151: fdatasync d8/d18/f19 0 2024-03-29T08:07:08.265 INFO:tasks.workunit.client.1.smithi148.stdout:7/129: mknod d8/d14/c31 0 2024-03-29T08:07:08.266 INFO:tasks.workunit.client.1.smithi148.stdout:7/130: write d8/f13 [71309,115129] 0 2024-03-29T08:07:08.268 INFO:tasks.workunit.client.0.smithi037.stdout:5/152: write d8/d22/f27 [258860,92770] 0 2024-03-29T08:07:08.269 INFO:tasks.workunit.client.0.smithi037.stdout:9/235: unlink d1/d4/d19/l35 0 2024-03-29T08:07:08.270 INFO:tasks.workunit.client.0.smithi037.stdout:5/153: dread - d8/d18/d24/f2a zero size 2024-03-29T08:07:08.272 INFO:tasks.workunit.client.0.smithi037.stdout:9/236: chown d1/d4/d19/d1e/d4a 112288776 1 2024-03-29T08:07:08.276 INFO:tasks.workunit.client.1.smithi148.stdout:8/113: dwrite d17/f1a [0,4194304] 0 2024-03-29T08:07:08.277 INFO:tasks.workunit.client.1.smithi148.stdout:8/114: write d17/f1a [656262,16617] 0 2024-03-29T08:07:08.278 INFO:tasks.workunit.client.0.smithi037.stdout:5/154: creat d8/d18/f30 x:0 0 0 2024-03-29T08:07:08.281 INFO:tasks.workunit.client.1.smithi148.stdout:8/115: dread f14 [0,4194304] 0 2024-03-29T08:07:08.282 INFO:tasks.workunit.client.1.smithi148.stdout:8/116: chown c11 30192910 1 2024-03-29T08:07:08.284 INFO:tasks.workunit.client.1.smithi148.stdout:8/117: stat d17/d1d 0 2024-03-29T08:07:08.285 INFO:tasks.workunit.client.0.smithi037.stdout:5/155: rmdir d8/d18/d24 39 2024-03-29T08:07:08.287 INFO:tasks.workunit.client.0.smithi037.stdout:5/156: chown d8/f2b 237810204 1 2024-03-29T08:07:08.288 INFO:tasks.workunit.client.0.smithi037.stdout:5/157: fsync d8/f2b 0 2024-03-29T08:07:08.291 INFO:tasks.workunit.client.0.smithi037.stdout:9/237: link d1/d4/d19/c26 d1/d1f/d3a/c4e 0 2024-03-29T08:07:08.292 INFO:tasks.workunit.client.0.smithi037.stdout:9/238: chown d1/l5 56059751 1 2024-03-29T08:07:08.293 INFO:tasks.workunit.client.0.smithi037.stdout:7/189: dread f9 [4194304,4194304] 0 2024-03-29T08:07:08.295 INFO:tasks.workunit.client.1.smithi148.stdout:8/118: dread d17/d1d/f20 [0,4194304] 0 2024-03-29T08:07:08.299 INFO:tasks.workunit.client.1.smithi148.stdout:8/119: symlink d17/d1d/l29 0 2024-03-29T08:07:08.299 INFO:tasks.workunit.client.0.smithi037.stdout:7/190: fsync dd/d21/f2b 0 2024-03-29T08:07:08.299 INFO:tasks.workunit.client.1.smithi148.stdout:8/120: readlink d17/d1d/l29 0 2024-03-29T08:07:08.301 INFO:tasks.workunit.client.0.smithi037.stdout:9/239: dread d1/d4/d19/d1e/d25/d10/f3d [0,4194304] 0 2024-03-29T08:07:08.301 INFO:tasks.workunit.client.0.smithi037.stdout:7/191: fsync dd/d1d/d34/f37 0 2024-03-29T08:07:08.302 INFO:tasks.workunit.client.0.smithi037.stdout:5/158: getdents d8/d22 0 2024-03-29T08:07:08.306 INFO:tasks.workunit.client.1.smithi148.stdout:5/81: rmdir d0 39 2024-03-29T08:07:08.306 INFO:tasks.workunit.client.0.smithi037.stdout:7/192: creat dd/d1d/d34/f3f x:0 0 0 2024-03-29T08:07:08.310 INFO:tasks.workunit.client.1.smithi148.stdout:5/82: readlink d0/db/lc 0 2024-03-29T08:07:08.312 INFO:tasks.workunit.client.0.smithi037.stdout:7/193: rename dd/f3c to dd/d1d/d34/f40 0 2024-03-29T08:07:08.313 INFO:tasks.workunit.client.0.smithi037.stdout:9/240: link d1/d4/c11 d1/d1f/d3a/c4f 0 2024-03-29T08:07:08.314 INFO:tasks.workunit.client.0.smithi037.stdout:5/159: truncate d8/f17 1143495 0 2024-03-29T08:07:08.315 INFO:tasks.workunit.client.1.smithi148.stdout:3/93: rmdir de 39 2024-03-29T08:07:08.315 INFO:tasks.workunit.client.1.smithi148.stdout:4/75: write f7 [1778842,115144] 0 2024-03-29T08:07:08.316 INFO:tasks.workunit.client.1.smithi148.stdout:8/121: dwrite f15 [0,4194304] 0 2024-03-29T08:07:08.318 INFO:tasks.workunit.client.0.smithi037.stdout:7/194: rename dd/d21/l27 to dd/d21/l41 0 2024-03-29T08:07:08.319 INFO:tasks.workunit.client.1.smithi148.stdout:3/94: dread - de/f1c zero size 2024-03-29T08:07:08.320 INFO:tasks.workunit.client.1.smithi148.stdout:4/76: fsync fd 0 2024-03-29T08:07:08.320 INFO:tasks.workunit.client.0.smithi037.stdout:9/241: chown d1/d1f/d3a/c1c 1018873283 1 2024-03-29T08:07:08.320 INFO:tasks.workunit.client.1.smithi148.stdout:8/122: readlink d17/d18/l24 0 2024-03-29T08:07:08.321 INFO:tasks.workunit.client.1.smithi148.stdout:4/77: readlink l9 0 2024-03-29T08:07:08.321 INFO:tasks.workunit.client.1.smithi148.stdout:0/99: dwrite f7 [0,4194304] 0 2024-03-29T08:07:08.321 INFO:tasks.workunit.client.1.smithi148.stdout:4/78: stat l9 0 2024-03-29T08:07:08.321 INFO:tasks.workunit.client.1.smithi148.stdout:1/76: truncate d3/d13/f15 2070138 0 2024-03-29T08:07:08.324 INFO:tasks.workunit.client.0.smithi037.stdout:7/195: read dd/d1d/f2a [36895,1698] 0 2024-03-29T08:07:08.324 INFO:tasks.workunit.client.1.smithi148.stdout:4/79: write fd [1540429,18461] 0 2024-03-29T08:07:08.325 INFO:tasks.workunit.client.1.smithi148.stdout:8/123: creat d17/d18/f2a x:0 0 0 2024-03-29T08:07:08.327 INFO:tasks.workunit.client.0.smithi037.stdout:9/242: rmdir d1/d4/d19 39 2024-03-29T08:07:08.330 INFO:tasks.workunit.client.1.smithi148.stdout:3/95: rename l9 to de/d13/l1e 0 2024-03-29T08:07:08.330 INFO:tasks.workunit.client.1.smithi148.stdout:0/100: mknod db/de/df/d11/c21 0 2024-03-29T08:07:08.330 INFO:tasks.workunit.client.1.smithi148.stdout:8/124: rename d17 to d17/d1d/d2b 22 2024-03-29T08:07:08.331 INFO:tasks.workunit.client.1.smithi148.stdout:8/125: dread - d17/f28 zero size 2024-03-29T08:07:08.334 INFO:tasks.workunit.client.1.smithi148.stdout:1/77: mkdir d3/d18 0 2024-03-29T08:07:08.334 INFO:tasks.workunit.client.0.smithi037.stdout:7/196: mkdir dd/d21/d35/d42 0 2024-03-29T08:07:08.336 INFO:tasks.workunit.client.1.smithi148.stdout:4/80: creat df/d10/d17/f1c x:0 0 0 2024-03-29T08:07:08.338 INFO:tasks.workunit.client.0.smithi037.stdout:9/243: dread - d1/d4/d19/d1e/d25/d10/d31/f4d zero size 2024-03-29T08:07:08.342 INFO:tasks.workunit.client.1.smithi148.stdout:2/89: truncate d7/f1b 403567 0 2024-03-29T08:07:08.342 INFO:tasks.workunit.client.0.smithi037.stdout:5/160: link d8/d18/f21 d8/d22/f31 0 2024-03-29T08:07:08.343 INFO:tasks.workunit.client.0.smithi037.stdout:7/197: creat dd/d21/d35/d42/f43 x:0 0 0 2024-03-29T08:07:08.343 INFO:tasks.workunit.client.1.smithi148.stdout:8/126: fsync f15 0 2024-03-29T08:07:08.344 INFO:tasks.workunit.client.0.smithi037.stdout:9/244: symlink d1/d4/d19/d1e/d25/d10/d23/l50 0 2024-03-29T08:07:08.345 INFO:tasks.workunit.client.1.smithi148.stdout:0/101: truncate f9 337938 0 2024-03-29T08:07:08.345 INFO:tasks.workunit.client.1.smithi148.stdout:9/83: rmdir db 39 2024-03-29T08:07:08.345 INFO:tasks.workunit.client.1.smithi148.stdout:8/127: chown ca 93 1 2024-03-29T08:07:08.346 INFO:tasks.workunit.client.0.smithi037.stdout:5/161: truncate d8/d22/f25 4815075 0 2024-03-29T08:07:08.346 INFO:tasks.workunit.client.0.smithi037.stdout:7/198: creat dd/d21/f44 x:0 0 0 2024-03-29T08:07:08.347 INFO:tasks.workunit.client.1.smithi148.stdout:0/102: write db/de/df/d11/f16 [473927,29434] 0 2024-03-29T08:07:08.350 INFO:tasks.workunit.client.1.smithi148.stdout:0/103: read f7 [2947841,63015] 0 2024-03-29T08:07:08.354 INFO:tasks.workunit.client.0.smithi037.stdout:5/162: rmdir d8/d18 39 2024-03-29T08:07:08.355 INFO:tasks.workunit.client.0.smithi037.stdout:7/199: truncate dd/f1f 4107595 0 2024-03-29T08:07:08.356 INFO:tasks.workunit.client.0.smithi037.stdout:5/163: write d8/f2b [119896,11227] 0 2024-03-29T08:07:08.357 INFO:tasks.workunit.client.1.smithi148.stdout:7/131: getdents d8 0 2024-03-29T08:07:08.359 INFO:tasks.workunit.client.0.smithi037.stdout:7/200: write dd/d21/f36 [338983,116956] 0 2024-03-29T08:07:08.359 INFO:tasks.workunit.client.0.smithi037.stdout:5/164: symlink d8/d22/l32 0 2024-03-29T08:07:08.362 INFO:tasks.workunit.client.0.smithi037.stdout:9/245: write d1/d3/f21 [1559554,13815] 0 2024-03-29T08:07:08.363 INFO:tasks.workunit.client.1.smithi148.stdout:9/84: rename db/d10/c11 to db/d10/d1a/c1c 0 2024-03-29T08:07:08.363 INFO:tasks.workunit.client.1.smithi148.stdout:7/132: mknod d8/d14/c32 0 2024-03-29T08:07:08.364 INFO:tasks.workunit.client.1.smithi148.stdout:2/90: link d7/da/c11 d7/da/c22 0 2024-03-29T08:07:08.365 INFO:tasks.workunit.client.1.smithi148.stdout:8/128: dwrite d17/d18/d26/f27 [0,4194304] 0 2024-03-29T08:07:08.366 INFO:tasks.workunit.client.1.smithi148.stdout:9/85: truncate f4 16411 0 2024-03-29T08:07:08.369 INFO:tasks.workunit.client.1.smithi148.stdout:6/76: write f5 [1447405,35681] 0 2024-03-29T08:07:08.369 INFO:tasks.workunit.client.1.smithi148.stdout:0/104: dread db/f10 [0,4194304] 0 2024-03-29T08:07:08.370 INFO:tasks.workunit.client.0.smithi037.stdout:9/246: dread d1/d4/d19/d1e/d25/d10/f3d [0,4194304] 0 2024-03-29T08:07:08.371 INFO:tasks.workunit.client.1.smithi148.stdout:7/133: fdatasync d8/fe 0 2024-03-29T08:07:08.373 INFO:tasks.workunit.client.0.smithi037.stdout:5/165: link d8/d18/d24/c29 d8/d18/d24/c33 0 2024-03-29T08:07:08.373 INFO:tasks.workunit.client.1.smithi148.stdout:7/134: chown d8/d29 2 1 2024-03-29T08:07:08.375 INFO:tasks.workunit.client.1.smithi148.stdout:7/135: truncate d8/d14/f21 635581 0 2024-03-29T08:07:08.376 INFO:tasks.workunit.client.0.smithi037.stdout:5/166: dread - d8/d18/f28 zero size 2024-03-29T08:07:08.377 INFO:tasks.workunit.client.1.smithi148.stdout:1/78: dread d3/fa [0,4194304] 0 2024-03-29T08:07:08.377 INFO:tasks.workunit.client.1.smithi148.stdout:8/129: creat d17/f2c x:0 0 0 2024-03-29T08:07:08.378 INFO:tasks.workunit.client.1.smithi148.stdout:9/86: rename c1 to db/d10/d1a/c1d 0 2024-03-29T08:07:08.380 INFO:tasks.workunit.client.0.smithi037.stdout:9/247: rename d1/d4/d19/d1e/d25/d10/c29 to d1/d4/c51 0 2024-03-29T08:07:08.381 INFO:tasks.workunit.client.1.smithi148.stdout:6/77: stat d6/fd 0 2024-03-29T08:07:08.381 INFO:tasks.workunit.client.0.smithi037.stdout:7/201: dwrite f9 [0,4194304] 0 2024-03-29T08:07:08.383 INFO:tasks.workunit.client.1.smithi148.stdout:6/78: dread - d6/f11 zero size 2024-03-29T08:07:08.384 INFO:tasks.workunit.client.0.smithi037.stdout:5/167: mkdir d8/d34 0 2024-03-29T08:07:08.384 INFO:tasks.workunit.client.0.smithi037.stdout:5/168: chown d8/d34 14 1 2024-03-29T08:07:08.385 INFO:tasks.workunit.client.1.smithi148.stdout:6/79: fsync f5 0 2024-03-29T08:07:08.387 INFO:tasks.workunit.client.0.smithi037.stdout:7/202: symlink dd/d1d/l45 0 2024-03-29T08:07:08.389 INFO:tasks.workunit.client.0.smithi037.stdout:5/169: write d8/f1f [483849,116112] 0 2024-03-29T08:07:08.392 INFO:tasks.workunit.client.0.smithi037.stdout:7/203: fdatasync dd/d1d/d34/f3f 0 2024-03-29T08:07:08.392 INFO:tasks.workunit.client.0.smithi037.stdout:5/170: stat d8/d22/l23 0 2024-03-29T08:07:08.394 INFO:tasks.workunit.client.1.smithi148.stdout:3/96: dwrite f5 [4194304,4194304] 0 2024-03-29T08:07:08.394 INFO:tasks.workunit.client.0.smithi037.stdout:7/204: fdatasync dd/d21/f26 0 2024-03-29T08:07:08.396 INFO:tasks.workunit.client.0.smithi037.stdout:7/205: stat dd/d21/d35 0 2024-03-29T08:07:08.397 INFO:tasks.workunit.client.1.smithi148.stdout:3/97: write de/f1c [308080,58280] 0 2024-03-29T08:07:08.397 INFO:tasks.workunit.client.1.smithi148.stdout:7/136: dwrite d8/f18 [4194304,4194304] 0 2024-03-29T08:07:08.398 INFO:tasks.workunit.client.1.smithi148.stdout:9/87: creat db/d10/d1a/f1e x:0 0 0 2024-03-29T08:07:08.402 INFO:tasks.workunit.client.0.smithi037.stdout:7/206: write dd/f1a [2722182,78871] 0 2024-03-29T08:07:08.404 INFO:tasks.workunit.client.0.smithi037.stdout:7/207: read dd/d1d/d34/f40 [2310087,72422] 0 2024-03-29T08:07:08.406 INFO:tasks.workunit.client.1.smithi148.stdout:6/80: getdents d6/df 0 2024-03-29T08:07:08.407 INFO:tasks.workunit.client.1.smithi148.stdout:7/137: mknod d8/d14/d19/c33 0 2024-03-29T08:07:08.408 INFO:tasks.workunit.client.1.smithi148.stdout:2/91: write f1 [2654261,89771] 0 2024-03-29T08:07:08.408 INFO:tasks.workunit.client.1.smithi148.stdout:0/105: link fa db/de/df/d11/f22 0 2024-03-29T08:07:08.409 INFO:tasks.workunit.client.0.smithi037.stdout:9/248: dwrite d1/d4/d19/d1e/f3f [0,4194304] 0 2024-03-29T08:07:08.410 INFO:tasks.workunit.client.1.smithi148.stdout:1/79: getdents d3/d13 0 2024-03-29T08:07:08.414 INFO:tasks.workunit.client.1.smithi148.stdout:1/80: read d3/f7 [935153,61408] 0 2024-03-29T08:07:08.415 INFO:tasks.workunit.client.0.smithi037.stdout:9/249: mknod d1/d4/d19/d1e/d25/d10/d31/c52 0 2024-03-29T08:07:08.417 INFO:tasks.workunit.client.1.smithi148.stdout:8/130: dwrite f14 [0,4194304] 0 2024-03-29T08:07:08.417 INFO:tasks.workunit.client.0.smithi037.stdout:9/250: write d1/d3/f21 [2488933,20904] 0 2024-03-29T08:07:08.418 INFO:tasks.workunit.client.1.smithi148.stdout:1/81: rename d3 to d3/d13/d19 22 2024-03-29T08:07:08.420 INFO:tasks.workunit.client.1.smithi148.stdout:8/131: write d17/d18/f2a [790956,60270] 0 2024-03-29T08:07:08.421 INFO:tasks.workunit.client.0.smithi037.stdout:9/251: write d1/d1f/d3a/f46 [42034,96318] 0 2024-03-29T08:07:08.421 INFO:tasks.workunit.client.1.smithi148.stdout:7/138: creat d8/d14/d19/f34 x:0 0 0 2024-03-29T08:07:08.425 INFO:tasks.workunit.client.1.smithi148.stdout:8/132: read d17/d1d/f20 [1022584,39583] 0 2024-03-29T08:07:08.425 INFO:tasks.workunit.client.1.smithi148.stdout:5/83: sync 2024-03-29T08:07:08.425 INFO:tasks.workunit.client.1.smithi148.stdout:6/81: unlink d6/c10 0 2024-03-29T08:07:08.428 INFO:tasks.workunit.client.0.smithi037.stdout:9/252: read d1/d4/d19/d1e/f3f [1650040,1481] 0 2024-03-29T08:07:08.428 INFO:tasks.workunit.client.1.smithi148.stdout:5/84: read - d0/db/f10 zero size 2024-03-29T08:07:08.430 INFO:tasks.workunit.client.1.smithi148.stdout:2/92: creat d7/da/d1c/f23 x:0 0 0 2024-03-29T08:07:08.431 INFO:tasks.workunit.client.1.smithi148.stdout:0/106: dread f2 [0,4194304] 0 2024-03-29T08:07:08.431 INFO:tasks.workunit.client.1.smithi148.stdout:8/133: creat d17/f2d x:0 0 0 2024-03-29T08:07:08.434 INFO:tasks.workunit.client.1.smithi148.stdout:7/139: creat d8/d29/f35 x:0 0 0 2024-03-29T08:07:08.436 INFO:tasks.workunit.client.0.smithi037.stdout:9/253: dread - d1/d4/d19/d1e/d25/d10/d31/f4d zero size 2024-03-29T08:07:08.440 INFO:tasks.workunit.client.1.smithi148.stdout:8/134: mknod d17/c2e 0 2024-03-29T08:07:08.445 INFO:tasks.workunit.client.1.smithi148.stdout:6/82: rmdir d6/df 0 2024-03-29T08:07:08.445 INFO:tasks.workunit.client.1.smithi148.stdout:8/135: write d17/f28 [421859,104388] 0 2024-03-29T08:07:08.446 INFO:tasks.workunit.client.1.smithi148.stdout:7/140: dread f6 [0,4194304] 0 2024-03-29T08:07:08.447 INFO:tasks.workunit.client.1.smithi148.stdout:0/107: link db/dc/f12 db/f23 0 2024-03-29T08:07:08.448 INFO:tasks.workunit.client.1.smithi148.stdout:8/136: write d17/f28 [765250,25638] 0 2024-03-29T08:07:08.448 INFO:tasks.workunit.client.1.smithi148.stdout:3/98: truncate f5 3910108 0 2024-03-29T08:07:08.450 INFO:tasks.workunit.client.1.smithi148.stdout:6/83: symlink d6/d9/l12 0 2024-03-29T08:07:08.454 INFO:tasks.workunit.client.1.smithi148.stdout:7/141: write d8/f10 [1561479,64876] 0 2024-03-29T08:07:08.454 INFO:tasks.workunit.client.1.smithi148.stdout:0/108: mkdir db/de/d24 0 2024-03-29T08:07:08.456 INFO:tasks.workunit.client.1.smithi148.stdout:3/99: mknod de/d13/c1f 0 2024-03-29T08:07:08.456 INFO:tasks.workunit.client.0.smithi037.stdout:9/254: dwrite d1/d1f/d3a/f46 [0,4194304] 0 2024-03-29T08:07:08.457 INFO:tasks.workunit.client.1.smithi148.stdout:0/109: read f2 [3361307,33938] 0 2024-03-29T08:07:08.457 INFO:tasks.workunit.client.1.smithi148.stdout:2/93: truncate d7/f1b 1258362 0 2024-03-29T08:07:08.459 INFO:tasks.workunit.client.1.smithi148.stdout:5/85: write d0/f9 [1052172,40251] 0 2024-03-29T08:07:08.461 INFO:tasks.workunit.client.1.smithi148.stdout:7/142: creat d8/f36 x:0 0 0 2024-03-29T08:07:08.462 INFO:tasks.workunit.client.0.smithi037.stdout:9/255: creat d1/d4/d19/d1e/d4a/f53 x:0 0 0 2024-03-29T08:07:08.462 INFO:tasks.workunit.client.1.smithi148.stdout:3/100: unlink f1 0 2024-03-29T08:07:08.463 INFO:tasks.workunit.client.1.smithi148.stdout:6/84: truncate f5 848162 0 2024-03-29T08:07:08.464 INFO:tasks.workunit.client.1.smithi148.stdout:0/110: creat db/de/f25 x:0 0 0 2024-03-29T08:07:08.466 INFO:tasks.workunit.client.1.smithi148.stdout:1/82: dwrite d3/d13/f15 [0,4194304] 0 2024-03-29T08:07:08.466 INFO:tasks.workunit.client.1.smithi148.stdout:5/86: symlink d0/l15 0 2024-03-29T08:07:08.468 INFO:tasks.workunit.client.1.smithi148.stdout:7/143: rmdir d8/d29 39 2024-03-29T08:07:08.470 INFO:tasks.workunit.client.1.smithi148.stdout:1/83: write d3/d13/f15 [264475,33567] 0 2024-03-29T08:07:08.471 INFO:tasks.workunit.client.1.smithi148.stdout:3/101: mknod de/d13/c20 0 2024-03-29T08:07:08.474 INFO:tasks.workunit.client.1.smithi148.stdout:0/111: chown c8 10277359 1 2024-03-29T08:07:08.474 INFO:tasks.workunit.client.1.smithi148.stdout:7/144: dread d8/d14/f21 [0,4194304] 0 2024-03-29T08:07:08.475 INFO:tasks.workunit.client.1.smithi148.stdout:6/85: fsync d6/fd 0 2024-03-29T08:07:08.478 INFO:tasks.workunit.client.1.smithi148.stdout:5/87: mkdir d0/db/de/d16 0 2024-03-29T08:07:08.478 INFO:tasks.workunit.client.1.smithi148.stdout:2/94: dwrite f1 [0,4194304] 0 2024-03-29T08:07:08.480 INFO:tasks.workunit.client.1.smithi148.stdout:6/86: dread d6/fd [0,4194304] 0 2024-03-29T08:07:08.481 INFO:tasks.workunit.client.1.smithi148.stdout:1/84: creat d3/d13/f1a x:0 0 0 2024-03-29T08:07:08.482 INFO:tasks.workunit.client.1.smithi148.stdout:2/95: truncate d7/da/fe 1449734 0 2024-03-29T08:07:08.482 INFO:tasks.workunit.client.0.smithi037.stdout:9/256: dwrite d1/d4/f28 [0,4194304] 0 2024-03-29T08:07:08.485 INFO:tasks.workunit.client.1.smithi148.stdout:0/112: dread f9 [0,4194304] 0 2024-03-29T08:07:08.485 INFO:tasks.workunit.client.1.smithi148.stdout:3/102: dread de/f15 [0,4194304] 0 2024-03-29T08:07:08.487 INFO:tasks.workunit.client.1.smithi148.stdout:6/87: write d6/fa [4134900,120777] 0 2024-03-29T08:07:08.489 INFO:tasks.workunit.client.1.smithi148.stdout:2/96: mkdir d7/d24 0 2024-03-29T08:07:08.493 INFO:tasks.workunit.client.1.smithi148.stdout:3/103: creat de/d13/f21 x:0 0 0 2024-03-29T08:07:08.497 INFO:tasks.workunit.client.1.smithi148.stdout:5/88: link d0/db/de/cf d0/db/de/c17 0 2024-03-29T08:07:08.497 INFO:tasks.workunit.client.1.smithi148.stdout:3/104: fdatasync de/d13/f21 0 2024-03-29T08:07:08.501 INFO:tasks.workunit.client.1.smithi148.stdout:6/88: rename d6/f11 to d6/d9/f13 0 2024-03-29T08:07:08.504 INFO:tasks.workunit.client.1.smithi148.stdout:1/85: dread d3/fa [0,4194304] 0 2024-03-29T08:07:08.506 INFO:tasks.workunit.client.1.smithi148.stdout:3/105: mknod de/c22 0 2024-03-29T08:07:08.506 INFO:tasks.workunit.client.1.smithi148.stdout:0/113: dread f2 [0,4194304] 0 2024-03-29T08:07:08.507 INFO:tasks.workunit.client.1.smithi148.stdout:1/86: truncate d3/d13/f1a 274392 0 2024-03-29T08:07:08.508 INFO:tasks.workunit.client.1.smithi148.stdout:3/106: truncate de/f15 4729119 0 2024-03-29T08:07:08.509 INFO:tasks.workunit.client.1.smithi148.stdout:6/89: fdatasync d6/fd 0 2024-03-29T08:07:08.510 INFO:tasks.workunit.client.0.smithi037.stdout:9/257: dwrite d1/d4/d19/d1e/d25/d10/f12 [4194304,4194304] 0 2024-03-29T08:07:08.511 INFO:tasks.workunit.client.1.smithi148.stdout:2/97: creat d7/da/d1c/d21/f25 x:0 0 0 2024-03-29T08:07:08.512 INFO:tasks.workunit.client.1.smithi148.stdout:2/98: dread - d7/f10 zero size 2024-03-29T08:07:08.513 INFO:tasks.workunit.client.1.smithi148.stdout:0/114: unlink db/f10 0 2024-03-29T08:07:08.515 INFO:tasks.workunit.client.1.smithi148.stdout:5/89: getdents d0/db/de/d16 0 2024-03-29T08:07:08.516 INFO:tasks.workunit.client.0.smithi037.stdout:9/258: mknod d1/d4/d19/d1e/d25/d16/c54 0 2024-03-29T08:07:08.521 INFO:tasks.workunit.client.1.smithi148.stdout:1/87: fsync d3/fa 0 2024-03-29T08:07:08.522 INFO:tasks.workunit.client.1.smithi148.stdout:3/107: creat de/d13/f23 x:0 0 0 2024-03-29T08:07:08.523 INFO:tasks.workunit.client.1.smithi148.stdout:6/90: creat d6/f14 x:0 0 0 2024-03-29T08:07:08.524 INFO:tasks.workunit.client.1.smithi148.stdout:3/108: write de/f11 [413638,105672] 0 2024-03-29T08:07:08.525 INFO:tasks.workunit.client.1.smithi148.stdout:6/91: write d6/f14 [851401,34429] 0 2024-03-29T08:07:08.528 INFO:tasks.workunit.client.1.smithi148.stdout:6/92: dread d6/fd [0,4194304] 0 2024-03-29T08:07:08.529 INFO:tasks.workunit.client.1.smithi148.stdout:6/93: chown d6/fe 11 1 2024-03-29T08:07:08.531 INFO:tasks.workunit.client.1.smithi148.stdout:2/99: mknod d7/d24/c26 0 2024-03-29T08:07:08.532 INFO:tasks.workunit.client.1.smithi148.stdout:3/109: write de/f15 [4469027,20603] 0 2024-03-29T08:07:08.534 INFO:tasks.workunit.client.1.smithi148.stdout:1/88: creat d3/de/f1b x:0 0 0 2024-03-29T08:07:08.535 INFO:tasks.workunit.client.1.smithi148.stdout:1/89: chown d3/d13/c16 24 1 2024-03-29T08:07:08.536 INFO:tasks.workunit.client.0.smithi037.stdout:9/259: dwrite d1/d1f/f33 [0,4194304] 0 2024-03-29T08:07:08.537 INFO:tasks.workunit.client.1.smithi148.stdout:4/81: sync 2024-03-29T08:07:08.538 INFO:tasks.workunit.client.1.smithi148.stdout:3/110: truncate f8 5100734 0 2024-03-29T08:07:08.540 INFO:tasks.workunit.client.1.smithi148.stdout:1/90: dread d3/f7 [0,4194304] 0 2024-03-29T08:07:08.541 INFO:tasks.workunit.client.1.smithi148.stdout:3/111: mkdir de/d24 0 2024-03-29T08:07:08.541 INFO:tasks.workunit.client.1.smithi148.stdout:1/91: chown d3 1 1 2024-03-29T08:07:08.544 INFO:tasks.workunit.client.1.smithi148.stdout:3/112: chown de/l14 124 1 2024-03-29T08:07:08.546 INFO:tasks.workunit.client.1.smithi148.stdout:1/92: dread d3/d13/f1a [0,4194304] 0 2024-03-29T08:07:08.546 INFO:tasks.workunit.client.1.smithi148.stdout:6/94: dwrite d6/f14 [0,4194304] 0 2024-03-29T08:07:08.546 INFO:tasks.workunit.client.1.smithi148.stdout:3/113: truncate de/f11 1069212 0 2024-03-29T08:07:08.547 INFO:tasks.workunit.client.1.smithi148.stdout:3/114: dread - de/d13/f23 zero size 2024-03-29T08:07:08.557 INFO:tasks.workunit.client.1.smithi148.stdout:3/115: creat de/d13/f25 x:0 0 0 2024-03-29T08:07:08.557 INFO:tasks.workunit.client.1.smithi148.stdout:2/100: write d7/f1b [1765682,105878] 0 2024-03-29T08:07:08.585 INFO:tasks.workunit.client.1.smithi148.stdout:2/101: stat d7/da 0 2024-03-29T08:07:08.589 INFO:tasks.workunit.client.1.smithi148.stdout:0/115: dwrite db/de/df/d11/f1b [0,4194304] 0 2024-03-29T08:07:08.591 INFO:tasks.workunit.client.1.smithi148.stdout:2/102: read d7/da/f1f [3250176,115589] 0 2024-03-29T08:07:08.591 INFO:tasks.workunit.client.1.smithi148.stdout:3/116: getdents de/d13 0 2024-03-29T08:07:08.597 INFO:tasks.workunit.client.1.smithi148.stdout:2/103: truncate d7/fc 4309817 0 2024-03-29T08:07:08.599 INFO:tasks.workunit.client.1.smithi148.stdout:0/116: mknod db/de/df/d11/d1f/c26 0 2024-03-29T08:07:08.599 INFO:tasks.workunit.client.1.smithi148.stdout:2/104: fdatasync d7/d13/f16 0 2024-03-29T08:07:08.601 INFO:tasks.workunit.client.1.smithi148.stdout:6/95: dwrite d6/fe [0,4194304] 0 2024-03-29T08:07:08.602 INFO:tasks.workunit.client.1.smithi148.stdout:1/93: dwrite d3/fa [4194304,4194304] 0 2024-03-29T08:07:08.603 INFO:tasks.workunit.client.1.smithi148.stdout:0/117: read db/dc/f12 [477749,112818] 0 2024-03-29T08:07:08.605 INFO:tasks.workunit.client.1.smithi148.stdout:2/105: write d7/da/d1c/d21/f25 [1032627,124583] 0 2024-03-29T08:07:08.606 INFO:tasks.workunit.client.1.smithi148.stdout:6/96: mknod d6/c15 0 2024-03-29T08:07:08.611 INFO:tasks.workunit.client.1.smithi148.stdout:1/94: mknod d3/de/c1c 0 2024-03-29T08:07:08.611 INFO:tasks.workunit.client.1.smithi148.stdout:3/117: dread f5 [0,4194304] 0 2024-03-29T08:07:08.611 INFO:tasks.workunit.client.1.smithi148.stdout:2/106: rmdir d7 39 2024-03-29T08:07:08.614 INFO:tasks.workunit.client.1.smithi148.stdout:1/95: mkdir d3/de/d1d 0 2024-03-29T08:07:08.615 INFO:tasks.workunit.client.1.smithi148.stdout:3/118: creat de/f26 x:0 0 0 2024-03-29T08:07:08.616 INFO:tasks.workunit.client.1.smithi148.stdout:6/97: creat d6/d9/f16 x:0 0 0 2024-03-29T08:07:08.618 INFO:tasks.workunit.client.1.smithi148.stdout:2/107: mknod d7/d13/c27 0 2024-03-29T08:07:08.618 INFO:tasks.workunit.client.1.smithi148.stdout:3/119: write de/d13/f21 [395627,35023] 0 2024-03-29T08:07:08.619 INFO:tasks.workunit.client.1.smithi148.stdout:1/96: mknod d3/d13/c1e 0 2024-03-29T08:07:08.620 INFO:tasks.workunit.client.1.smithi148.stdout:2/108: chown d7/d24 0 1 2024-03-29T08:07:08.621 INFO:tasks.workunit.client.1.smithi148.stdout:6/98: mknod d6/c17 0 2024-03-29T08:07:08.622 INFO:tasks.workunit.client.1.smithi148.stdout:2/109: chown d7/d13 31 1 2024-03-29T08:07:08.622 INFO:tasks.workunit.client.1.smithi148.stdout:9/88: sync 2024-03-29T08:07:08.623 INFO:tasks.workunit.client.1.smithi148.stdout:3/120: write de/f11 [529578,4195] 0 2024-03-29T08:07:08.628 INFO:tasks.workunit.client.1.smithi148.stdout:6/99: dread - d6/d9/f16 zero size 2024-03-29T08:07:08.628 INFO:tasks.workunit.client.1.smithi148.stdout:3/121: chown de/lf 65118 1 2024-03-29T08:07:08.628 INFO:tasks.workunit.client.1.smithi148.stdout:0/118: dread f7 [0,4194304] 0 2024-03-29T08:07:08.629 INFO:tasks.workunit.client.1.smithi148.stdout:2/110: creat d7/da/d1c/f28 x:0 0 0 2024-03-29T08:07:08.631 INFO:tasks.workunit.client.1.smithi148.stdout:2/111: creat d7/d13/f29 x:0 0 0 2024-03-29T08:07:08.633 INFO:tasks.workunit.client.1.smithi148.stdout:0/119: mknod db/de/df/d14/c27 0 2024-03-29T08:07:08.636 INFO:tasks.workunit.client.1.smithi148.stdout:2/112: creat d7/d13/f2a x:0 0 0 2024-03-29T08:07:08.637 INFO:tasks.workunit.client.1.smithi148.stdout:0/120: fdatasync db/de/f25 0 2024-03-29T08:07:08.641 INFO:tasks.workunit.client.1.smithi148.stdout:6/100: write f5 [353989,77497] 0 2024-03-29T08:07:08.642 INFO:tasks.workunit.client.1.smithi148.stdout:2/113: write d7/fc [4533188,114094] 0 2024-03-29T08:07:08.647 INFO:tasks.workunit.client.1.smithi148.stdout:0/121: mknod db/de/df/c28 0 2024-03-29T08:07:08.663 INFO:tasks.workunit.client.1.smithi148.stdout:0/122: readlink db/dc/l1c 0 2024-03-29T08:07:08.664 INFO:tasks.workunit.client.1.smithi148.stdout:6/101: rename d6/c15 to d6/d9/c18 0 2024-03-29T08:07:08.668 INFO:tasks.workunit.client.1.smithi148.stdout:3/122: dwrite de/d13/f21 [0,4194304] 0 2024-03-29T08:07:08.671 INFO:tasks.workunit.client.1.smithi148.stdout:9/89: dwrite db/d10/d14/f18 [0,4194304] 0 2024-03-29T08:07:08.678 INFO:tasks.workunit.client.1.smithi148.stdout:9/90: rmdir db/d10 39 2024-03-29T08:07:08.681 INFO:tasks.workunit.client.1.smithi148.stdout:1/97: dwrite d3/d13/f1a [0,4194304] 0 2024-03-29T08:07:08.688 INFO:tasks.workunit.client.1.smithi148.stdout:3/123: fsync de/d13/f21 0 2024-03-29T08:07:08.691 INFO:tasks.workunit.client.1.smithi148.stdout:1/98: write d3/fa [6915983,61392] 0 2024-03-29T08:07:08.699 INFO:tasks.workunit.client.1.smithi148.stdout:0/123: write db/de/df/d11/f22 [936503,83118] 0 2024-03-29T08:07:08.700 INFO:tasks.workunit.client.1.smithi148.stdout:9/91: rename db/d10/d14/l19 to db/l1f 0 2024-03-29T08:07:08.701 INFO:tasks.workunit.client.1.smithi148.stdout:2/114: dwrite d7/f17 [0,4194304] 0 2024-03-29T08:07:08.703 INFO:tasks.workunit.client.1.smithi148.stdout:9/92: readlink db/lf 0 2024-03-29T08:07:08.704 INFO:tasks.workunit.client.1.smithi148.stdout:9/93: chown db/d10/d1a 12668 1 2024-03-29T08:07:08.708 INFO:tasks.workunit.client.1.smithi148.stdout:2/115: symlink d7/d13/l2b 0 2024-03-29T08:07:08.708 INFO:tasks.workunit.client.1.smithi148.stdout:6/102: dwrite d6/f14 [0,4194304] 0 2024-03-29T08:07:08.710 INFO:tasks.workunit.client.1.smithi148.stdout:9/94: write db/d10/d14/f18 [4414879,69922] 0 2024-03-29T08:07:08.712 INFO:tasks.workunit.client.1.smithi148.stdout:0/124: mkdir db/de/df/d29 0 2024-03-29T08:07:08.713 INFO:tasks.workunit.client.1.smithi148.stdout:9/95: chown db/lf 20796 1 2024-03-29T08:07:08.715 INFO:tasks.workunit.client.1.smithi148.stdout:6/103: write d6/d9/f13 [208027,90206] 0 2024-03-29T08:07:08.726 INFO:tasks.workunit.client.0.smithi037.stdout:1/150: sync 2024-03-29T08:07:08.731 INFO:tasks.workunit.client.1.smithi148.stdout:2/116: getdents d7/d13 0 2024-03-29T08:07:08.732 INFO:tasks.workunit.client.1.smithi148.stdout:1/99: truncate d3/d13/f1a 1521158 0 2024-03-29T08:07:08.732 INFO:tasks.workunit.client.1.smithi148.stdout:2/117: chown d7/fc 272697 1 2024-03-29T08:07:08.733 INFO:tasks.workunit.client.1.smithi148.stdout:0/125: getdents db/de/df 0 2024-03-29T08:07:08.734 INFO:tasks.workunit.client.1.smithi148.stdout:1/100: truncate d3/fa 8858181 0 2024-03-29T08:07:08.735 INFO:tasks.workunit.client.1.smithi148.stdout:9/96: truncate f7 4160848 0 2024-03-29T08:07:08.736 INFO:tasks.workunit.client.1.smithi148.stdout:9/97: chown db/d10/d1a/f1e 104178 1 2024-03-29T08:07:08.736 INFO:tasks.workunit.client.1.smithi148.stdout:2/118: rename d7/fc to d7/d24/f2c 0 2024-03-29T08:07:08.737 INFO:tasks.workunit.client.1.smithi148.stdout:6/104: truncate f5 841191 0 2024-03-29T08:07:08.738 INFO:tasks.workunit.client.1.smithi148.stdout:0/126: symlink db/de/df/l2a 0 2024-03-29T08:07:08.741 INFO:tasks.workunit.client.1.smithi148.stdout:2/119: creat d7/f2d x:0 0 0 2024-03-29T08:07:08.744 INFO:tasks.workunit.client.0.smithi037.stdout:1/151: dwrite d2/f31 [0,4194304] 0 2024-03-29T08:07:08.744 INFO:tasks.workunit.client.1.smithi148.stdout:0/127: mkdir db/de/df/d2b 0 2024-03-29T08:07:08.746 INFO:tasks.workunit.client.1.smithi148.stdout:2/120: mkdir d7/d13/d2e 0 2024-03-29T08:07:08.751 INFO:tasks.workunit.client.1.smithi148.stdout:2/121: chown d7/da/c12 91847 1 2024-03-29T08:07:08.753 INFO:tasks.workunit.client.1.smithi148.stdout:7/145: sync 2024-03-29T08:07:08.753 INFO:tasks.workunit.client.1.smithi148.stdout:8/137: sync 2024-03-29T08:07:08.756 INFO:tasks.workunit.client.1.smithi148.stdout:0/128: creat db/de/df/d29/f2c x:0 0 0 2024-03-29T08:07:08.758 INFO:tasks.workunit.client.1.smithi148.stdout:2/122: symlink d7/da/d1c/d21/l2f 0 2024-03-29T08:07:08.761 INFO:tasks.workunit.client.1.smithi148.stdout:1/101: dwrite d3/d13/f15 [0,4194304] 0 2024-03-29T08:07:08.761 INFO:tasks.workunit.client.1.smithi148.stdout:8/138: write d17/f1a [2719827,18277] 0 2024-03-29T08:07:08.762 INFO:tasks.workunit.client.0.smithi037.stdout:1/152: dwrite d2/f14 [0,4194304] 0 2024-03-29T08:07:08.767 INFO:tasks.workunit.client.1.smithi148.stdout:7/146: write f6 [1827131,20091] 0 2024-03-29T08:07:08.767 INFO:tasks.workunit.client.1.smithi148.stdout:2/123: chown d7/da/l1d 404505089 1 2024-03-29T08:07:08.767 INFO:tasks.workunit.client.0.smithi037.stdout:1/153: mknod d2/d6/dc/d18/d22/c3b 0 2024-03-29T08:07:08.768 INFO:tasks.workunit.client.0.smithi037.stdout:1/154: readlink d2/d6/l27 0 2024-03-29T08:07:08.768 INFO:tasks.workunit.client.1.smithi148.stdout:6/105: dwrite d6/d9/f13 [0,4194304] 0 2024-03-29T08:07:08.771 INFO:tasks.workunit.client.1.smithi148.stdout:8/139: chown f14 988903 1 2024-03-29T08:07:08.773 INFO:tasks.workunit.client.0.smithi037.stdout:1/155: dread - d2/d6/dc/d18/d1d/d21/f37 zero size 2024-03-29T08:07:08.774 INFO:tasks.workunit.client.1.smithi148.stdout:1/102: creat d3/de/f1f x:0 0 0 2024-03-29T08:07:08.774 INFO:tasks.workunit.client.1.smithi148.stdout:6/106: creat d6/f19 x:0 0 0 2024-03-29T08:07:08.776 INFO:tasks.workunit.client.1.smithi148.stdout:2/124: truncate d7/da/d1c/f23 604918 0 2024-03-29T08:07:08.777 INFO:tasks.workunit.client.0.smithi037.stdout:1/156: symlink d2/l3c 0 2024-03-29T08:07:08.778 INFO:tasks.workunit.client.1.smithi148.stdout:7/147: mkdir d8/d14/d19/d37 0 2024-03-29T08:07:08.778 INFO:tasks.workunit.client.0.smithi037.stdout:1/157: readlink d2/l3 0 2024-03-29T08:07:08.780 INFO:tasks.workunit.client.1.smithi148.stdout:6/107: creat d6/f1a x:0 0 0 2024-03-29T08:07:08.780 INFO:tasks.workunit.client.1.smithi148.stdout:9/98: dwrite f7 [0,4194304] 0 2024-03-29T08:07:08.780 INFO:tasks.workunit.client.1.smithi148.stdout:4/82: sync 2024-03-29T08:07:08.780 INFO:tasks.workunit.client.1.smithi148.stdout:3/124: sync 2024-03-29T08:07:08.781 INFO:tasks.workunit.client.1.smithi148.stdout:5/90: sync 2024-03-29T08:07:08.786 INFO:tasks.workunit.client.1.smithi148.stdout:3/125: dread - de/f17 zero size 2024-03-29T08:07:08.787 INFO:tasks.workunit.client.1.smithi148.stdout:2/125: write d7/d13/f16 [926609,71449] 0 2024-03-29T08:07:08.789 INFO:tasks.workunit.client.0.smithi037.stdout:1/158: write d2/d6/dc/d18/d1d/f2c [1242982,71138] 0 2024-03-29T08:07:08.791 INFO:tasks.workunit.client.1.smithi148.stdout:5/91: read d0/f9 [452422,5775] 0 2024-03-29T08:07:08.797 INFO:tasks.workunit.client.1.smithi148.stdout:4/83: truncate f3 145833 0 2024-03-29T08:07:08.797 INFO:tasks.workunit.client.1.smithi148.stdout:3/126: chown de/d13/f25 0 1 2024-03-29T08:07:08.814 INFO:tasks.workunit.client.0.smithi037.stdout:1/159: truncate d2/d6/dc/d18/d1d/d21/f2a 1126387 0 2024-03-29T08:07:08.816 INFO:tasks.workunit.client.0.smithi037.stdout:1/160: fsync d2/d6/dc/d18/d1d/f2c 0 2024-03-29T08:07:08.816 INFO:tasks.workunit.client.1.smithi148.stdout:8/140: truncate d17/d18/f1c 85168 0 2024-03-29T08:07:08.819 INFO:tasks.workunit.client.0.smithi037.stdout:1/161: mknod d2/d6/dc/d18/d1d/d21/c3d 0 2024-03-29T08:07:08.819 INFO:tasks.workunit.client.1.smithi148.stdout:0/129: dwrite db/de/df/d11/f1b [0,4194304] 0 2024-03-29T08:07:08.823 INFO:tasks.workunit.client.0.smithi037.stdout:1/162: dread - d2/d6/d13/f2e zero size 2024-03-29T08:07:08.824 INFO:tasks.workunit.client.1.smithi148.stdout:1/103: dwrite d3/d13/f15 [8388608,4194304] 0 2024-03-29T08:07:08.825 INFO:tasks.workunit.client.1.smithi148.stdout:0/130: dread f9 [0,4194304] 0 2024-03-29T08:07:08.826 INFO:tasks.workunit.client.1.smithi148.stdout:5/92: dwrite d0/f12 [0,4194304] 0 2024-03-29T08:07:08.828 INFO:tasks.workunit.client.1.smithi148.stdout:2/126: dread d7/f17 [0,4194304] 0 2024-03-29T08:07:08.829 INFO:tasks.workunit.client.1.smithi148.stdout:2/127: read - d7/d13/f2a zero size 2024-03-29T08:07:08.829 INFO:tasks.workunit.client.0.smithi037.stdout:1/163: creat d2/d6/dc/d18/f3e x:0 0 0 2024-03-29T08:07:08.831 INFO:tasks.workunit.client.0.smithi037.stdout:1/164: write d2/d6/dc/f25 [1429856,40061] 0 2024-03-29T08:07:08.845 INFO:tasks.workunit.client.0.smithi037.stdout:1/165: write d2/d6/dc/f15 [1031742,93320] 0 2024-03-29T08:07:08.852 INFO:tasks.workunit.client.0.smithi037.stdout:1/166: rename d2/d6/d13/f2e to d2/d6/dc/d18/d1d/f3f 0 2024-03-29T08:07:08.861 INFO:tasks.workunit.client.0.smithi037.stdout:8/148: sync 2024-03-29T08:07:08.862 INFO:tasks.workunit.client.0.smithi037.stdout:8/149: dread - d5/d10/f23 zero size 2024-03-29T08:07:08.864 INFO:tasks.workunit.client.0.smithi037.stdout:8/150: chown d5/f1b 76602494 1 2024-03-29T08:07:08.868 INFO:tasks.workunit.client.0.smithi037.stdout:1/167: dwrite d2/d6/dc/f19 [0,4194304] 0 2024-03-29T08:07:08.869 INFO:tasks.workunit.client.0.smithi037.stdout:8/151: rmdir d5/d10/d19/d2c 0 2024-03-29T08:07:08.870 INFO:tasks.workunit.client.0.smithi037.stdout:1/168: readlink d2/d6/l1f 0 2024-03-29T08:07:08.879 INFO:tasks.workunit.client.0.smithi037.stdout:1/169: getdents d2/d6 0 2024-03-29T08:07:08.882 INFO:tasks.workunit.client.0.smithi037.stdout:1/170: write d2/d6/dc/f19 [431060,4192] 0 2024-03-29T08:07:08.883 INFO:tasks.workunit.client.0.smithi037.stdout:1/171: chown d2/d6/dc/f19 1816 1 2024-03-29T08:07:08.884 INFO:tasks.workunit.client.0.smithi037.stdout:1/172: chown d2/f14 3082 1 2024-03-29T08:07:08.885 INFO:tasks.workunit.client.0.smithi037.stdout:8/152: dread d5/d10/f11 [4194304,4194304] 0 2024-03-29T08:07:08.887 INFO:tasks.workunit.client.0.smithi037.stdout:1/173: mkdir d2/d40 0 2024-03-29T08:07:08.888 INFO:tasks.workunit.client.0.smithi037.stdout:8/153: creat d5/d25/f2e x:0 0 0 2024-03-29T08:07:08.889 INFO:tasks.workunit.client.1.smithi148.stdout:6/108: getdents d6 0 2024-03-29T08:07:08.889 INFO:tasks.workunit.client.0.smithi037.stdout:8/154: stat d5/l27 0 2024-03-29T08:07:08.890 INFO:tasks.workunit.client.0.smithi037.stdout:1/174: creat d2/d6/f41 x:0 0 0 2024-03-29T08:07:08.890 INFO:tasks.workunit.client.1.smithi148.stdout:6/109: chown d6/d9/f13 14673317 1 2024-03-29T08:07:08.890 INFO:tasks.workunit.client.1.smithi148.stdout:4/84: creat df/d10/f1d x:0 0 0 2024-03-29T08:07:08.892 INFO:tasks.workunit.client.0.smithi037.stdout:8/155: fsync f1 0 2024-03-29T08:07:08.893 INFO:tasks.workunit.client.0.smithi037.stdout:1/175: symlink d2/l42 0 2024-03-29T08:07:08.894 INFO:tasks.workunit.client.1.smithi148.stdout:0/131: mkdir db/de/df/d11/d2d 0 2024-03-29T08:07:08.894 INFO:tasks.workunit.client.0.smithi037.stdout:8/156: fsync d5/f16 0 2024-03-29T08:07:08.895 INFO:tasks.workunit.client.1.smithi148.stdout:5/93: symlink d0/db/de/l18 0 2024-03-29T08:07:08.897 INFO:tasks.workunit.client.0.smithi037.stdout:1/176: read d2/d6/d13/f28 [589369,96627] 0 2024-03-29T08:07:08.898 INFO:tasks.workunit.client.0.smithi037.stdout:8/157: creat d5/d10/d19/f2f x:0 0 0 2024-03-29T08:07:08.900 INFO:tasks.workunit.client.0.smithi037.stdout:1/177: chown d2/d6/dc/f19 3 1 2024-03-29T08:07:08.900 INFO:tasks.workunit.client.1.smithi148.stdout:3/127: link de/f11 de/d24/f27 0 2024-03-29T08:07:08.900 INFO:tasks.workunit.client.1.smithi148.stdout:3/128: dread - de/d13/f25 zero size 2024-03-29T08:07:08.902 INFO:tasks.workunit.client.1.smithi148.stdout:9/99: rename db/d10/d1a to db/d20 0 2024-03-29T08:07:08.904 INFO:tasks.workunit.client.1.smithi148.stdout:7/148: getdents d8/d14 0 2024-03-29T08:07:08.904 INFO:tasks.workunit.client.0.smithi037.stdout:8/158: truncate d5/fb 2497624 0 2024-03-29T08:07:08.905 INFO:tasks.workunit.client.1.smithi148.stdout:6/110: stat d6/d9/c18 0 2024-03-29T08:07:08.906 INFO:tasks.workunit.client.0.smithi037.stdout:1/178: creat d2/d40/f43 x:0 0 0 2024-03-29T08:07:08.906 INFO:tasks.workunit.client.1.smithi148.stdout:1/104: creat d3/d18/f20 x:0 0 0 2024-03-29T08:07:08.906 INFO:tasks.workunit.client.1.smithi148.stdout:8/141: sync 2024-03-29T08:07:08.908 INFO:tasks.workunit.client.1.smithi148.stdout:0/132: symlink db/dc/l2e 0 2024-03-29T08:07:08.909 INFO:tasks.workunit.client.1.smithi148.stdout:0/133: stat db/de/df/d29/f2c 0 2024-03-29T08:07:08.911 INFO:tasks.workunit.client.1.smithi148.stdout:0/134: write db/dc/f1e [1815921,63672] 0 2024-03-29T08:07:08.912 INFO:tasks.workunit.client.0.smithi037.stdout:8/159: getdents d5/d10/d19 0 2024-03-29T08:07:08.913 INFO:tasks.workunit.client.1.smithi148.stdout:3/129: symlink de/l28 0 2024-03-29T08:07:08.914 INFO:tasks.workunit.client.1.smithi148.stdout:9/100: dread f7 [0,4194304] 0 2024-03-29T08:07:08.919 INFO:tasks.workunit.client.0.smithi037.stdout:8/160: unlink d5/d10/d19/l21 0 2024-03-29T08:07:08.920 INFO:tasks.workunit.client.1.smithi148.stdout:0/135: dread db/de/df/d11/f16 [0,4194304] 0 2024-03-29T08:07:08.921 INFO:tasks.workunit.client.0.smithi037.stdout:2/104: sync 2024-03-29T08:07:08.921 INFO:tasks.workunit.client.0.smithi037.stdout:6/87: sync 2024-03-29T08:07:08.921 INFO:tasks.workunit.client.0.smithi037.stdout:4/63: sync 2024-03-29T08:07:08.923 INFO:tasks.workunit.client.0.smithi037.stdout:8/161: write d5/d25/f28 [1335609,35801] 0 2024-03-29T08:07:08.924 INFO:tasks.workunit.client.1.smithi148.stdout:6/111: symlink d6/d9/l1b 0 2024-03-29T08:07:08.927 INFO:tasks.workunit.client.1.smithi148.stdout:6/112: dread - d6/d9/f16 zero size 2024-03-29T08:07:08.927 INFO:tasks.workunit.client.1.smithi148.stdout:1/105: mkdir d3/de/d21 0 2024-03-29T08:07:08.928 INFO:tasks.workunit.client.0.smithi037.stdout:6/88: chown d0/d5/fa 12409855 1 2024-03-29T08:07:08.928 INFO:tasks.workunit.client.1.smithi148.stdout:2/128: link l6 d7/da/d1c/l30 0 2024-03-29T08:07:08.931 INFO:tasks.workunit.client.1.smithi148.stdout:5/94: symlink d0/db/de/d16/l19 0 2024-03-29T08:07:08.932 INFO:tasks.workunit.client.1.smithi148.stdout:3/130: dread de/d13/f21 [0,4194304] 0 2024-03-29T08:07:08.932 INFO:tasks.workunit.client.1.smithi148.stdout:5/95: dread - d0/db/f10 zero size 2024-03-29T08:07:08.933 INFO:tasks.workunit.client.0.smithi037.stdout:6/89: write d0/d5/f14 [2976039,88178] 0 2024-03-29T08:07:08.934 INFO:tasks.workunit.client.0.smithi037.stdout:1/179: dwrite d2/d6/f36 [0,4194304] 0 2024-03-29T08:07:08.937 INFO:tasks.workunit.client.1.smithi148.stdout:7/149: mkdir d8/d38 0 2024-03-29T08:07:08.937 INFO:tasks.workunit.client.1.smithi148.stdout:7/150: chown d8/d14/f16 9123976 1 2024-03-29T08:07:08.940 INFO:tasks.workunit.client.0.smithi037.stdout:2/105: link d0/f2 d0/f21 0 2024-03-29T08:07:08.940 INFO:tasks.workunit.client.0.smithi037.stdout:6/90: readlink d0/d1/le 0 2024-03-29T08:07:08.940 INFO:tasks.workunit.client.0.smithi037.stdout:1/180: stat d2/f31 0 2024-03-29T08:07:08.941 INFO:tasks.workunit.client.0.smithi037.stdout:6/91: readlink d0/d19/l1d 0 2024-03-29T08:07:08.942 INFO:tasks.workunit.client.0.smithi037.stdout:1/181: read d2/f31 [3117821,127731] 0 2024-03-29T08:07:08.943 INFO:tasks.workunit.client.1.smithi148.stdout:2/129: mkdir d7/da/d1c/d31 0 2024-03-29T08:07:08.945 INFO:tasks.workunit.client.1.smithi148.stdout:3/131: mknod de/c29 0 2024-03-29T08:07:08.945 INFO:tasks.workunit.client.0.smithi037.stdout:6/92: fdatasync d0/d1/f1b 0 2024-03-29T08:07:08.946 INFO:tasks.workunit.client.1.smithi148.stdout:5/96: symlink d0/l1a 0 2024-03-29T08:07:08.953 INFO:tasks.workunit.client.0.smithi037.stdout:6/93: mkdir d0/d5/d1f 0 2024-03-29T08:07:08.954 INFO:tasks.workunit.client.0.smithi037.stdout:1/182: write d2/d6/dc/d18/d1d/d21/f20 [894504,9382] 0 2024-03-29T08:07:08.955 INFO:tasks.workunit.client.0.smithi037.stdout:8/162: dwrite d5/d10/f23 [0,4194304] 0 2024-03-29T08:07:08.956 INFO:tasks.workunit.client.1.smithi148.stdout:1/106: getdents d3/de/d1d 0 2024-03-29T08:07:08.956 INFO:tasks.workunit.client.1.smithi148.stdout:0/136: dwrite f2 [4194304,4194304] 0 2024-03-29T08:07:08.956 INFO:tasks.workunit.client.0.smithi037.stdout:4/64: dread d3/d4/d8/db/f12 [0,4194304] 0 2024-03-29T08:07:08.956 INFO:tasks.workunit.client.0.smithi037.stdout:6/94: chown d0/d1/f1b 56703895 1 2024-03-29T08:07:08.958 INFO:tasks.workunit.client.0.smithi037.stdout:1/183: write d2/d6/f41 [365981,42362] 0 2024-03-29T08:07:08.959 INFO:tasks.workunit.client.1.smithi148.stdout:3/132: dread de/f15 [0,4194304] 0 2024-03-29T08:07:08.961 INFO:tasks.workunit.client.1.smithi148.stdout:9/101: creat db/f21 x:0 0 0 2024-03-29T08:07:08.963 INFO:tasks.workunit.client.1.smithi148.stdout:9/102: dread - db/f21 zero size 2024-03-29T08:07:08.965 INFO:tasks.workunit.client.1.smithi148.stdout:9/103: chown db/cc 2339653 1 2024-03-29T08:07:08.966 INFO:tasks.workunit.client.0.smithi037.stdout:8/163: rename d5/l6 to d5/d10/d19/l30 0 2024-03-29T08:07:08.968 INFO:tasks.workunit.client.1.smithi148.stdout:9/104: write db/d10/d14/f18 [4363712,88045] 0 2024-03-29T08:07:08.968 INFO:tasks.workunit.client.1.smithi148.stdout:3/133: dread de/f11 [0,4194304] 0 2024-03-29T08:07:08.973 INFO:tasks.workunit.client.1.smithi148.stdout:9/105: write db/d10/d14/f18 [1863520,89420] 0 2024-03-29T08:07:08.973 INFO:tasks.workunit.client.1.smithi148.stdout:2/130: symlink d7/da/l32 0 2024-03-29T08:07:08.973 INFO:tasks.workunit.client.1.smithi148.stdout:5/97: creat d0/db/de/d16/f1b x:0 0 0 2024-03-29T08:07:08.974 INFO:tasks.workunit.client.0.smithi037.stdout:8/164: truncate d5/f1e 727335 0 2024-03-29T08:07:08.976 INFO:tasks.workunit.client.0.smithi037.stdout:6/95: getdents d0/d5 0 2024-03-29T08:07:08.977 INFO:tasks.workunit.client.1.smithi148.stdout:0/137: rename c4 to db/c2f 0 2024-03-29T08:07:08.983 INFO:tasks.workunit.client.0.smithi037.stdout:8/165: fdatasync d5/d10/f23 0 2024-03-29T08:07:08.984 INFO:tasks.workunit.client.0.smithi037.stdout:1/184: dread d2/f31 [0,4194304] 0 2024-03-29T08:07:08.988 INFO:tasks.workunit.client.0.smithi037.stdout:8/166: symlink d5/d10/l31 0 2024-03-29T08:07:08.988 INFO:tasks.workunit.client.0.smithi037.stdout:1/185: mknod d2/d40/c44 0 2024-03-29T08:07:08.990 INFO:tasks.workunit.client.0.smithi037.stdout:4/65: dwrite d3/d4/d8/db/f16 [0,4194304] 0 2024-03-29T08:07:08.991 INFO:tasks.workunit.client.1.smithi148.stdout:3/134: rename de/d13/c20 to de/c2a 0 2024-03-29T08:07:08.993 INFO:tasks.workunit.client.1.smithi148.stdout:9/106: dwrite db/d10/f13 [0,4194304] 0 2024-03-29T08:07:08.994 INFO:tasks.workunit.client.0.smithi037.stdout:8/167: creat d5/f32 x:0 0 0 2024-03-29T08:07:08.996 INFO:tasks.workunit.client.1.smithi148.stdout:3/135: chown de/l14 56728 1 2024-03-29T08:07:08.996 INFO:tasks.workunit.client.1.smithi148.stdout:5/98: dwrite d0/db/f10 [0,4194304] 0 2024-03-29T08:07:08.998 INFO:tasks.workunit.client.0.smithi037.stdout:8/168: truncate d5/d25/f2e 715878 0 2024-03-29T08:07:08.999 INFO:tasks.workunit.client.1.smithi148.stdout:2/131: mkdir d7/da/d33 0 2024-03-29T08:07:09.000 INFO:tasks.workunit.client.0.smithi037.stdout:4/66: rmdir d3/d4/d8 39 2024-03-29T08:07:09.004 INFO:tasks.workunit.client.0.smithi037.stdout:8/169: creat d5/f33 x:0 0 0 2024-03-29T08:07:09.005 INFO:tasks.workunit.client.1.smithi148.stdout:0/138: creat db/de/df/f30 x:0 0 0 2024-03-29T08:07:09.012 INFO:tasks.workunit.client.1.smithi148.stdout:3/136: creat de/d24/f2b x:0 0 0 2024-03-29T08:07:09.013 INFO:tasks.workunit.client.0.smithi037.stdout:8/170: mkdir d5/d10/d2d/d34 0 2024-03-29T08:07:09.013 INFO:tasks.workunit.client.0.smithi037.stdout:4/67: truncate d3/d4/d8/db/f16 5021541 0 2024-03-29T08:07:09.016 INFO:tasks.workunit.client.0.smithi037.stdout:1/186: truncate d2/f14 3421272 0 2024-03-29T08:07:09.019 INFO:tasks.workunit.client.1.smithi148.stdout:8/142: dread d17/d18/f1c [0,4194304] 0 2024-03-29T08:07:09.020 INFO:tasks.workunit.client.0.smithi037.stdout:1/187: fsync d2/d40/f43 0 2024-03-29T08:07:09.020 INFO:tasks.workunit.client.0.smithi037.stdout:8/171: creat d5/d10/f35 x:0 0 0 2024-03-29T08:07:09.020 INFO:tasks.workunit.client.0.smithi037.stdout:4/68: creat d3/d4/d8/db/f1a x:0 0 0 2024-03-29T08:07:09.029 INFO:tasks.workunit.client.0.smithi037.stdout:8/172: rename d5/l27 to d5/d10/l36 0 2024-03-29T08:07:09.029 INFO:tasks.workunit.client.1.smithi148.stdout:0/139: creat db/de/df/d11/d2d/f31 x:0 0 0 2024-03-29T08:07:09.029 INFO:tasks.workunit.client.1.smithi148.stdout:9/107: dwrite db/f21 [0,4194304] 0 2024-03-29T08:07:09.030 INFO:tasks.workunit.client.0.smithi037.stdout:1/188: rename d2/d40/c44 to d2/d40/c45 0 2024-03-29T08:07:09.030 INFO:tasks.workunit.client.0.smithi037.stdout:8/173: dread - d5/d10/f35 zero size 2024-03-29T08:07:09.031 INFO:tasks.workunit.client.1.smithi148.stdout:0/140: chown db/dc 1542805 1 2024-03-29T08:07:09.034 INFO:tasks.workunit.client.1.smithi148.stdout:2/132: dwrite d7/da/d1c/f23 [0,4194304] 0 2024-03-29T08:07:09.034 INFO:tasks.workunit.client.1.smithi148.stdout:0/141: read f7 [4654227,52583] 0 2024-03-29T08:07:09.037 INFO:tasks.workunit.client.1.smithi148.stdout:8/143: dread f7 [0,4194304] 0 2024-03-29T08:07:09.037 INFO:tasks.workunit.client.1.smithi148.stdout:4/85: truncate fa 2910624 0 2024-03-29T08:07:09.040 INFO:tasks.workunit.client.0.smithi037.stdout:6/96: dread d0/d1/f17 [0,4194304] 0 2024-03-29T08:07:09.042 INFO:tasks.workunit.client.0.smithi037.stdout:6/97: mkdir d0/d20 0 2024-03-29T08:07:09.042 INFO:tasks.workunit.client.1.smithi148.stdout:0/142: rmdir db/de/df/d29 39 2024-03-29T08:07:09.046 INFO:tasks.workunit.client.1.smithi148.stdout:4/86: rename f1 to df/f1e 0 2024-03-29T08:07:09.047 INFO:tasks.workunit.client.0.smithi037.stdout:6/98: chown d0/d1/le 1948299 1 2024-03-29T08:07:09.048 INFO:tasks.workunit.client.1.smithi148.stdout:8/144: rename d17/d18/d26 to d17/d18/d26/d2f 22 2024-03-29T08:07:09.049 INFO:tasks.workunit.client.0.smithi037.stdout:4/69: dwrite d3/d4/d8/db/f1a [0,4194304] 0 2024-03-29T08:07:09.054 INFO:tasks.workunit.client.1.smithi148.stdout:0/143: symlink db/de/d24/l32 0 2024-03-29T08:07:09.056 INFO:tasks.workunit.client.0.smithi037.stdout:1/189: dread d2/d6/dc/f25 [0,4194304] 0 2024-03-29T08:07:09.057 INFO:tasks.workunit.client.1.smithi148.stdout:6/113: truncate d6/fa 1807943 0 2024-03-29T08:07:09.057 INFO:tasks.workunit.client.0.smithi037.stdout:6/99: mknod d0/d20/c21 0 2024-03-29T08:07:09.060 INFO:tasks.workunit.client.1.smithi148.stdout:9/108: dread f3 [4194304,4194304] 0 2024-03-29T08:07:09.065 INFO:tasks.workunit.client.0.smithi037.stdout:1/190: write d2/d6/dc/f15 [77814,92305] 0 2024-03-29T08:07:09.066 INFO:tasks.workunit.client.1.smithi148.stdout:2/133: dwrite d7/da/fe [0,4194304] 0 2024-03-29T08:07:09.069 INFO:tasks.workunit.client.1.smithi148.stdout:1/107: truncate d3/d13/f1a 839557 0 2024-03-29T08:07:09.073 INFO:tasks.workunit.client.1.smithi148.stdout:6/114: dread f5 [0,4194304] 0 2024-03-29T08:07:09.075 INFO:tasks.workunit.client.0.smithi037.stdout:1/191: symlink d2/d40/l46 0 2024-03-29T08:07:09.076 INFO:tasks.workunit.client.1.smithi148.stdout:8/145: dread fe [0,4194304] 0 2024-03-29T08:07:09.076 INFO:tasks.workunit.client.0.smithi037.stdout:8/174: dread d5/f17 [4194304,4194304] 0 2024-03-29T08:07:09.077 INFO:tasks.workunit.client.1.smithi148.stdout:3/137: getdents de/d24 0 2024-03-29T08:07:09.080 INFO:tasks.workunit.client.0.smithi037.stdout:0/97: sync 2024-03-29T08:07:09.080 INFO:tasks.workunit.client.0.smithi037.stdout:7/208: sync 2024-03-29T08:07:09.080 INFO:tasks.workunit.client.0.smithi037.stdout:5/171: sync 2024-03-29T08:07:09.080 INFO:tasks.workunit.client.0.smithi037.stdout:3/184: sync 2024-03-29T08:07:09.080 INFO:tasks.workunit.client.0.smithi037.stdout:9/260: sync 2024-03-29T08:07:09.081 INFO:tasks.workunit.client.0.smithi037.stdout:1/192: chown d2/d6/dc/d18/f3e 413007203 1 2024-03-29T08:07:09.082 INFO:tasks.workunit.client.1.smithi148.stdout:8/146: write d17/f28 [395573,29843] 0 2024-03-29T08:07:09.083 INFO:tasks.workunit.client.1.smithi148.stdout:7/151: dwrite d8/d29/f2e [0,4194304] 0 2024-03-29T08:07:09.084 INFO:tasks.workunit.client.1.smithi148.stdout:8/147: stat f7 0 2024-03-29T08:07:09.085 INFO:tasks.workunit.client.0.smithi037.stdout:7/209: dread - dd/d21/f32 zero size 2024-03-29T08:07:09.085 INFO:tasks.workunit.client.0.smithi037.stdout:0/98: fsync f8 0 2024-03-29T08:07:09.085 INFO:tasks.workunit.client.1.smithi148.stdout:6/115: symlink d6/l1c 0 2024-03-29T08:07:09.087 INFO:tasks.workunit.client.0.smithi037.stdout:4/70: dwrite f0 [4194304,4194304] 0 2024-03-29T08:07:09.087 INFO:tasks.workunit.client.1.smithi148.stdout:3/138: creat de/f2c x:0 0 0 2024-03-29T08:07:09.089 INFO:tasks.workunit.client.1.smithi148.stdout:7/152: fdatasync d8/d14/f1c 0 2024-03-29T08:07:09.095 INFO:tasks.workunit.client.0.smithi037.stdout:8/175: rename d5/ce to d5/c37 0 2024-03-29T08:07:09.096 INFO:tasks.workunit.client.1.smithi148.stdout:7/153: read f6 [34159,12450] 0 2024-03-29T08:07:09.097 INFO:tasks.workunit.client.0.smithi037.stdout:3/185: symlink dd/d46/l48 0 2024-03-29T08:07:09.099 INFO:tasks.workunit.client.1.smithi148.stdout:0/144: write db/dc/f12 [107494,6275] 0 2024-03-29T08:07:09.100 INFO:tasks.workunit.client.1.smithi148.stdout:4/87: truncate fa 3804797 0 2024-03-29T08:07:09.100 INFO:tasks.workunit.client.1.smithi148.stdout:9/109: dwrite db/f21 [0,4194304] 0 2024-03-29T08:07:09.102 INFO:tasks.workunit.client.0.smithi037.stdout:7/210: write dd/d1d/d34/f3f [132629,7140] 0 2024-03-29T08:07:09.102 INFO:tasks.workunit.client.0.smithi037.stdout:6/100: dread d0/d1/f17 [0,4194304] 0 2024-03-29T08:07:09.103 INFO:tasks.workunit.client.0.smithi037.stdout:1/193: creat d2/d6/dc/d18/d1d/d21/f47 x:0 0 0 2024-03-29T08:07:09.104 INFO:tasks.workunit.client.1.smithi148.stdout:5/99: dwrite d0/f9 [0,4194304] 0 2024-03-29T08:07:09.104 INFO:tasks.workunit.client.1.smithi148.stdout:7/154: fdatasync d8/f26 0 2024-03-29T08:07:09.105 INFO:tasks.workunit.client.0.smithi037.stdout:9/261: mknod d1/d4/c55 0 2024-03-29T08:07:09.106 INFO:tasks.workunit.client.0.smithi037.stdout:4/71: creat d3/f1b x:0 0 0 2024-03-29T08:07:09.107 INFO:tasks.workunit.client.1.smithi148.stdout:7/155: dread - d8/d14/f2b zero size 2024-03-29T08:07:09.108 INFO:tasks.workunit.client.0.smithi037.stdout:3/186: creat dd/d15/f49 x:0 0 0 2024-03-29T08:07:09.108 INFO:tasks.workunit.client.0.smithi037.stdout:6/101: dread d0/d5/f16 [0,4194304] 0 2024-03-29T08:07:09.113 INFO:tasks.workunit.client.0.smithi037.stdout:5/172: truncate d8/d18/f21 671034 0 2024-03-29T08:07:09.116 INFO:tasks.workunit.client.1.smithi148.stdout:4/88: unlink c0 0 2024-03-29T08:07:09.116 INFO:tasks.workunit.client.1.smithi148.stdout:2/134: getdents d7/da/d1c/d21 0 2024-03-29T08:07:09.118 INFO:tasks.workunit.client.0.smithi037.stdout:7/211: write dd/f17 [1269630,117341] 0 2024-03-29T08:07:09.119 INFO:tasks.workunit.client.1.smithi148.stdout:5/100: symlink d0/db/de/l1c 0 2024-03-29T08:07:09.121 INFO:tasks.workunit.client.1.smithi148.stdout:7/156: creat d8/d14/d19/f39 x:0 0 0 2024-03-29T08:07:09.122 INFO:tasks.workunit.client.0.smithi037.stdout:6/102: dread d0/d5/f16 [0,4194304] 0 2024-03-29T08:07:09.123 INFO:tasks.workunit.client.1.smithi148.stdout:9/110: mkdir db/d22 0 2024-03-29T08:07:09.123 INFO:tasks.workunit.client.0.smithi037.stdout:9/262: creat d1/d4/d19/d1e/d25/d16/f56 x:0 0 0 2024-03-29T08:07:09.124 INFO:tasks.workunit.client.0.smithi037.stdout:4/72: unlink c1 0 2024-03-29T08:07:09.126 INFO:tasks.workunit.client.0.smithi037.stdout:3/187: mknod dd/d15/d18/c4a 0 2024-03-29T08:07:09.127 INFO:tasks.workunit.client.0.smithi037.stdout:9/263: stat d1/d3/d44 0 2024-03-29T08:07:09.127 INFO:tasks.workunit.client.1.smithi148.stdout:7/157: dread - d8/f15 zero size 2024-03-29T08:07:09.127 INFO:tasks.workunit.client.1.smithi148.stdout:4/89: creat df/f1f x:0 0 0 2024-03-29T08:07:09.128 INFO:tasks.workunit.client.1.smithi148.stdout:2/135: creat d7/d24/f34 x:0 0 0 2024-03-29T08:07:09.128 INFO:tasks.workunit.client.1.smithi148.stdout:5/101: creat d0/db/f1d x:0 0 0 2024-03-29T08:07:09.130 INFO:tasks.workunit.client.1.smithi148.stdout:6/116: dwrite d6/f19 [0,4194304] 0 2024-03-29T08:07:09.130 INFO:tasks.workunit.client.1.smithi148.stdout:9/111: symlink db/d10/l23 0 2024-03-29T08:07:09.131 INFO:tasks.workunit.client.0.smithi037.stdout:5/173: rename d8/c20 to d8/d18/d24/c35 0 2024-03-29T08:07:09.132 INFO:tasks.workunit.client.1.smithi148.stdout:8/148: dread d17/d18/d26/f27 [0,4194304] 0 2024-03-29T08:07:09.132 INFO:tasks.workunit.client.0.smithi037.stdout:1/194: getdents d2/d6/dc 0 2024-03-29T08:07:09.136 INFO:tasks.workunit.client.1.smithi148.stdout:7/158: dread d8/d14/f2c [0,4194304] 0 2024-03-29T08:07:09.139 INFO:tasks.workunit.client.1.smithi148.stdout:0/145: dwrite db/dc/f12 [0,4194304] 0 2024-03-29T08:07:09.139 INFO:tasks.workunit.client.0.smithi037.stdout:3/188: mkdir dd/d15/d18/d4b 0 2024-03-29T08:07:09.141 INFO:tasks.workunit.client.1.smithi148.stdout:5/102: mkdir d0/db/d1e 0 2024-03-29T08:07:09.141 INFO:tasks.workunit.client.1.smithi148.stdout:8/149: mknod d17/d18/c30 0 2024-03-29T08:07:09.146 INFO:tasks.workunit.client.0.smithi037.stdout:1/195: write d2/f31 [1811893,60563] 0 2024-03-29T08:07:09.151 INFO:tasks.workunit.client.0.smithi037.stdout:7/212: creat dd/d1d/f46 x:0 0 0 2024-03-29T08:07:09.152 INFO:tasks.workunit.client.1.smithi148.stdout:9/112: symlink db/d22/l24 0 2024-03-29T08:07:09.153 INFO:tasks.workunit.client.0.smithi037.stdout:5/174: symlink d8/d18/d24/l36 0 2024-03-29T08:07:09.155 INFO:tasks.workunit.client.1.smithi148.stdout:0/146: dread db/dc/f1e [0,4194304] 0 2024-03-29T08:07:09.156 INFO:tasks.workunit.client.0.smithi037.stdout:5/175: chown d8/d22/l23 0 1 2024-03-29T08:07:09.157 INFO:tasks.workunit.client.0.smithi037.stdout:8/176: dread f1 [0,4194304] 0 2024-03-29T08:07:09.158 INFO:tasks.workunit.client.0.smithi037.stdout:9/264: creat d1/d3/d44/f57 x:0 0 0 2024-03-29T08:07:09.161 INFO:tasks.workunit.client.1.smithi148.stdout:5/103: mknod d0/db/de/c1f 0 2024-03-29T08:07:09.162 INFO:tasks.workunit.client.0.smithi037.stdout:9/265: stat d1/d4/d19/d1e/d4a/f53 0 2024-03-29T08:07:09.163 INFO:tasks.workunit.client.1.smithi148.stdout:3/139: truncate de/f1c 249550 0 2024-03-29T08:07:09.164 INFO:tasks.workunit.client.0.smithi037.stdout:9/266: fsync d1/f4c 0 2024-03-29T08:07:09.166 INFO:tasks.workunit.client.0.smithi037.stdout:6/103: dread d0/d1/f17 [0,4194304] 0 2024-03-29T08:07:09.167 INFO:tasks.workunit.client.0.smithi037.stdout:4/73: link d3/d4/d15/f19 d3/d4/d8/f1c 0 2024-03-29T08:07:09.167 INFO:tasks.workunit.client.1.smithi148.stdout:8/150: mknod d17/d18/d26/c31 0 2024-03-29T08:07:09.167 INFO:tasks.workunit.client.1.smithi148.stdout:9/113: symlink db/d10/l25 0 2024-03-29T08:07:09.168 INFO:tasks.workunit.client.1.smithi148.stdout:8/151: readlink d17/d18/l24 0 2024-03-29T08:07:09.172 INFO:tasks.workunit.client.1.smithi148.stdout:5/104: dread d0/db/f10 [0,4194304] 0 2024-03-29T08:07:09.172 INFO:tasks.workunit.client.0.smithi037.stdout:5/176: mkdir d8/d22/d37 0 2024-03-29T08:07:09.174 INFO:tasks.workunit.client.1.smithi148.stdout:5/105: write d0/f9 [1320669,126328] 0 2024-03-29T08:07:09.177 INFO:tasks.workunit.client.1.smithi148.stdout:2/136: getdents d7/d13 0 2024-03-29T08:07:09.178 INFO:tasks.workunit.client.1.smithi148.stdout:0/147: creat db/de/df/d11/f33 x:0 0 0 2024-03-29T08:07:09.181 INFO:tasks.workunit.client.0.smithi037.stdout:9/267: mknod d1/d4/d19/d1e/d25/c58 0 2024-03-29T08:07:09.182 INFO:tasks.workunit.client.0.smithi037.stdout:1/196: dwrite d2/d6/dc/d18/d1d/f2c [0,4194304] 0 2024-03-29T08:07:09.183 INFO:tasks.workunit.client.0.smithi037.stdout:7/213: symlink dd/l47 0 2024-03-29T08:07:09.197 INFO:tasks.workunit.client.0.smithi037.stdout:9/268: chown d1/d4/d19/d1e/d25/d10/f3d 0 1 2024-03-29T08:07:09.198 INFO:tasks.workunit.client.0.smithi037.stdout:4/74: dread f0 [8388608,4194304] 0 2024-03-29T08:07:09.199 INFO:tasks.workunit.client.0.smithi037.stdout:1/197: dread d2/d6/f41 [0,4194304] 0 2024-03-29T08:07:09.201 INFO:tasks.workunit.client.0.smithi037.stdout:6/104: symlink d0/d5/d1f/l22 0 2024-03-29T08:07:09.202 INFO:tasks.workunit.client.0.smithi037.stdout:9/269: chown d1/d4/d19/c2e 19 1 2024-03-29T08:07:09.203 INFO:tasks.workunit.client.0.smithi037.stdout:3/189: getdents dd/d46/d3e 0 2024-03-29T08:07:09.204 INFO:tasks.workunit.client.1.smithi148.stdout:4/90: truncate fd 3514797 0 2024-03-29T08:07:09.206 INFO:tasks.workunit.client.1.smithi148.stdout:8/152: creat d17/d18/d26/f32 x:0 0 0 2024-03-29T08:07:09.207 INFO:tasks.workunit.client.0.smithi037.stdout:5/177: dwrite d8/d22/f2d [0,4194304] 0 2024-03-29T08:07:09.207 INFO:tasks.workunit.client.1.smithi148.stdout:9/114: chown db/d20/c1c 965835778 1 2024-03-29T08:07:09.208 INFO:tasks.workunit.client.1.smithi148.stdout:7/159: dread d8/fe [0,4194304] 0 2024-03-29T08:07:09.210 INFO:tasks.workunit.client.1.smithi148.stdout:4/91: dread - df/d10/d17/f1c zero size 2024-03-29T08:07:09.210 INFO:tasks.workunit.client.0.smithi037.stdout:6/105: chown d0/d5/f16 0 1 2024-03-29T08:07:09.210 INFO:tasks.workunit.client.0.smithi037.stdout:1/198: write d2/d6/dc/f35 [895912,129497] 0 2024-03-29T08:07:09.211 INFO:tasks.workunit.client.0.smithi037.stdout:1/199: readlink d2/d6/l27 0 2024-03-29T08:07:09.213 INFO:tasks.workunit.client.0.smithi037.stdout:9/270: mknod d1/d4/d19/d1e/c59 0 2024-03-29T08:07:09.213 INFO:tasks.workunit.client.1.smithi148.stdout:9/115: read f4 [10029,75776] 0 2024-03-29T08:07:09.213 INFO:tasks.workunit.client.0.smithi037.stdout:5/178: fdatasync d8/d22/f25 0 2024-03-29T08:07:09.215 INFO:tasks.workunit.client.1.smithi148.stdout:6/117: dwrite d6/f19 [4194304,4194304] 0 2024-03-29T08:07:09.219 INFO:tasks.workunit.client.1.smithi148.stdout:8/153: creat d17/f33 x:0 0 0 2024-03-29T08:07:09.219 INFO:tasks.workunit.client.1.smithi148.stdout:6/118: rename d6 to d6/d9/d1d 22 2024-03-29T08:07:09.219 INFO:tasks.workunit.client.1.smithi148.stdout:7/160: symlink d8/d14/d19/l3a 0 2024-03-29T08:07:09.220 INFO:tasks.workunit.client.1.smithi148.stdout:5/106: dwrite d0/db/f10 [0,4194304] 0 2024-03-29T08:07:09.221 INFO:tasks.workunit.client.1.smithi148.stdout:9/116: write db/d10/f13 [3145301,123508] 0 2024-03-29T08:07:09.223 INFO:tasks.workunit.client.0.smithi037.stdout:7/214: link dd/f14 dd/f48 0 2024-03-29T08:07:09.224 INFO:tasks.workunit.client.1.smithi148.stdout:9/117: write db/d20/f1b [592903,2747] 0 2024-03-29T08:07:09.224 INFO:tasks.workunit.client.0.smithi037.stdout:3/190: getdents dd/d46/d3e/d47 0 2024-03-29T08:07:09.227 INFO:tasks.workunit.client.0.smithi037.stdout:3/191: chown dd/d15/d27/f41 10303 1 2024-03-29T08:07:09.232 INFO:tasks.workunit.client.1.smithi148.stdout:0/148: creat db/de/df/d2b/f34 x:0 0 0 2024-03-29T08:07:09.234 INFO:tasks.workunit.client.0.smithi037.stdout:5/179: symlink d8/l38 0 2024-03-29T08:07:09.234 INFO:tasks.workunit.client.0.smithi037.stdout:6/106: creat d0/d5/d1a/f23 x:0 0 0 2024-03-29T08:07:09.234 INFO:tasks.workunit.client.0.smithi037.stdout:7/215: unlink dd/d1d/f46 0 2024-03-29T08:07:09.235 INFO:tasks.workunit.client.0.smithi037.stdout:3/192: chown dd/d15/c2a 122 1 2024-03-29T08:07:09.236 INFO:tasks.workunit.client.1.smithi148.stdout:0/149: write f2 [4715911,7438] 0 2024-03-29T08:07:09.237 INFO:tasks.workunit.client.1.smithi148.stdout:7/161: symlink d8/d29/l3b 0 2024-03-29T08:07:09.241 INFO:tasks.workunit.client.0.smithi037.stdout:3/193: truncate dd/d46/f38 5185193 0 2024-03-29T08:07:09.241 INFO:tasks.workunit.client.0.smithi037.stdout:4/75: getdents d3/d4/d8 0 2024-03-29T08:07:09.242 INFO:tasks.workunit.client.1.smithi148.stdout:5/107: dwrite d0/f12 [0,4194304] 0 2024-03-29T08:07:09.242 INFO:tasks.workunit.client.0.smithi037.stdout:5/180: truncate d8/d18/d24/f2a 217861 0 2024-03-29T08:07:09.243 INFO:tasks.workunit.client.0.smithi037.stdout:7/216: mkdir dd/d1d/d49 0 2024-03-29T08:07:09.245 INFO:tasks.workunit.client.1.smithi148.stdout:8/154: rename d17/f33 to d17/d18/f34 0 2024-03-29T08:07:09.247 INFO:tasks.workunit.client.1.smithi148.stdout:4/92: write f4 [4320677,59624] 0 2024-03-29T08:07:09.249 INFO:tasks.workunit.client.0.smithi037.stdout:9/271: getdents d1/d4/d19/d1e/d25/d10/d31 0 2024-03-29T08:07:09.250 INFO:tasks.workunit.client.1.smithi148.stdout:9/118: mkdir db/d22/d26 0 2024-03-29T08:07:09.251 INFO:tasks.workunit.client.1.smithi148.stdout:9/119: chown db/d10 17 1 2024-03-29T08:07:09.258 INFO:tasks.workunit.client.1.smithi148.stdout:2/137: getdents d7/d24 0 2024-03-29T08:07:09.261 INFO:tasks.workunit.client.1.smithi148.stdout:8/155: dwrite d17/f28 [0,4194304] 0 2024-03-29T08:07:09.263 INFO:tasks.workunit.client.1.smithi148.stdout:8/156: dread - d17/f2c zero size 2024-03-29T08:07:09.263 INFO:tasks.workunit.client.0.smithi037.stdout:5/181: write d8/f17 [1939340,40885] 0 2024-03-29T08:07:09.266 INFO:tasks.workunit.client.0.smithi037.stdout:1/200: dwrite d2/d6/dc/f1e [0,4194304] 0 2024-03-29T08:07:09.266 INFO:tasks.workunit.client.1.smithi148.stdout:1/108: sync 2024-03-29T08:07:09.268 INFO:tasks.workunit.client.1.smithi148.stdout:1/109: chown d3/de/c1c 302667 1 2024-03-29T08:07:09.269 INFO:tasks.workunit.client.1.smithi148.stdout:1/110: dread - d3/de/f1f zero size 2024-03-29T08:07:09.270 INFO:tasks.workunit.client.0.smithi037.stdout:6/107: dwrite d0/d1/f1b [0,4194304] 0 2024-03-29T08:07:09.270 INFO:tasks.workunit.client.0.smithi037.stdout:1/201: chown d2/d6 0 1 2024-03-29T08:07:09.271 INFO:tasks.workunit.client.1.smithi148.stdout:1/111: dread - d3/de/f1b zero size 2024-03-29T08:07:09.277 INFO:tasks.workunit.client.1.smithi148.stdout:7/162: unlink c1 0 2024-03-29T08:07:09.277 INFO:tasks.workunit.client.1.smithi148.stdout:1/112: dread d3/f7 [0,4194304] 0 2024-03-29T08:07:09.278 INFO:tasks.workunit.client.1.smithi148.stdout:6/119: write d6/fd [631661,62316] 0 2024-03-29T08:07:09.279 INFO:tasks.workunit.client.0.smithi037.stdout:4/76: dread d3/d4/d8/db/f1a [0,4194304] 0 2024-03-29T08:07:09.281 INFO:tasks.workunit.client.1.smithi148.stdout:8/157: dwrite d17/f28 [0,4194304] 0 2024-03-29T08:07:09.281 INFO:tasks.workunit.client.1.smithi148.stdout:5/108: mknod d0/db/c20 0 2024-03-29T08:07:09.283 INFO:tasks.workunit.client.1.smithi148.stdout:8/158: stat d17/f2c 0 2024-03-29T08:07:09.283 INFO:tasks.workunit.client.0.smithi037.stdout:5/182: mknod d8/d18/c39 0 2024-03-29T08:07:09.283 INFO:tasks.workunit.client.0.smithi037.stdout:9/272: dread d1/d3/f21 [0,4194304] 0 2024-03-29T08:07:09.289 INFO:tasks.workunit.client.1.smithi148.stdout:4/93: read fc [1862834,33091] 0 2024-03-29T08:07:09.291 INFO:tasks.workunit.client.1.smithi148.stdout:3/140: sync 2024-03-29T08:07:09.294 INFO:tasks.workunit.client.1.smithi148.stdout:6/120: mkdir d6/d9/d1e 0 2024-03-29T08:07:09.294 INFO:tasks.workunit.client.0.smithi037.stdout:1/202: unlink d2/d6/dc/f19 0 2024-03-29T08:07:09.295 INFO:tasks.workunit.client.0.smithi037.stdout:5/183: chown d8/c15 60998768 1 2024-03-29T08:07:09.296 INFO:tasks.workunit.client.1.smithi148.stdout:4/94: write df/f1a [779323,12457] 0 2024-03-29T08:07:09.299 INFO:tasks.workunit.client.1.smithi148.stdout:5/109: dread d0/f9 [0,4194304] 0 2024-03-29T08:07:09.301 INFO:tasks.workunit.client.0.smithi037.stdout:1/203: readlink d2/d6/dc/d18/d1d/l2f 0 2024-03-29T08:07:09.302 INFO:tasks.workunit.client.1.smithi148.stdout:7/163: dwrite d8/f17 [0,4194304] 0 2024-03-29T08:07:09.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:07:08 smithi148 ceph-mon[44992]: pgmap v36: 65 pgs: 65 active+clean; 418 MiB data, 3.1 GiB used, 533 GiB / 536 GiB avail; 842 KiB/s rd, 41 MiB/s wr, 265 op/s 2024-03-29T08:07:09.306 INFO:tasks.workunit.client.1.smithi148.stdout:7/164: chown d8/d14/f1e 15 1 2024-03-29T08:07:09.308 INFO:tasks.workunit.client.0.smithi037.stdout:6/108: dwrite d0/d5/f14 [0,4194304] 0 2024-03-29T08:07:09.310 INFO:tasks.workunit.client.1.smithi148.stdout:8/159: creat d17/d18/d26/f35 x:0 0 0 2024-03-29T08:07:09.311 INFO:tasks.workunit.client.0.smithi037.stdout:4/77: getdents d3/d4/d8/db 0 2024-03-29T08:07:09.312 INFO:tasks.workunit.client.1.smithi148.stdout:2/138: link d7/da/d1c/f23 d7/da/d33/f35 0 2024-03-29T08:07:09.312 INFO:tasks.workunit.client.1.smithi148.stdout:0/150: write f7 [2917330,104263] 0 2024-03-29T08:07:09.312 INFO:tasks.workunit.client.1.smithi148.stdout:1/113: mkdir d3/de/d21/d22 0 2024-03-29T08:07:09.316 INFO:tasks.workunit.client.0.smithi037.stdout:1/204: creat d2/d6/dc/d18/d22/f48 x:0 0 0 2024-03-29T08:07:09.317 INFO:tasks.workunit.client.1.smithi148.stdout:6/121: dwrite d6/fd [0,4194304] 0 2024-03-29T08:07:09.320 INFO:tasks.workunit.client.0.smithi037.stdout:7/217: dwrite dd/d1d/d34/f39 [0,4194304] 0 2024-03-29T08:07:09.321 INFO:tasks.workunit.client.0.smithi037.stdout:6/109: symlink d0/d1/l24 0 2024-03-29T08:07:09.321 INFO:tasks.workunit.client.1.smithi148.stdout:3/141: mknod de/d13/c2d 0 2024-03-29T08:07:09.323 INFO:tasks.workunit.client.0.smithi037.stdout:5/184: creat d8/d34/f3a x:0 0 0 2024-03-29T08:07:09.323 INFO:tasks.workunit.client.0.smithi037.stdout:1/205: chown d2/d6/l27 128541291 1 2024-03-29T08:07:09.324 INFO:tasks.workunit.client.0.smithi037.stdout:2/106: sync 2024-03-29T08:07:09.327 INFO:tasks.workunit.client.1.smithi148.stdout:7/165: truncate f6 2573408 0 2024-03-29T08:07:09.328 INFO:tasks.workunit.client.1.smithi148.stdout:1/114: dread f1 [0,4194304] 0 2024-03-29T08:07:09.328 INFO:tasks.workunit.client.1.smithi148.stdout:8/160: creat d17/d18/f36 x:0 0 0 2024-03-29T08:07:09.329 INFO:tasks.workunit.client.1.smithi148.stdout:0/151: dread f2 [4194304,4194304] 0 2024-03-29T08:07:09.332 INFO:tasks.workunit.client.1.smithi148.stdout:6/122: creat d6/d9/f1f x:0 0 0 2024-03-29T08:07:09.332 INFO:tasks.workunit.client.1.smithi148.stdout:2/139: write f1 [3514054,57112] 0 2024-03-29T08:07:09.333 INFO:tasks.workunit.client.1.smithi148.stdout:4/95: truncate fd 801427 0 2024-03-29T08:07:09.334 INFO:tasks.workunit.client.1.smithi148.stdout:9/120: getdents db/d10/d14 0 2024-03-29T08:07:09.335 INFO:tasks.workunit.client.1.smithi148.stdout:6/123: chown d6/d9/l1b 53504820 1 2024-03-29T08:07:09.336 INFO:tasks.workunit.client.1.smithi148.stdout:2/140: write d7/f1b [242910,34649] 0 2024-03-29T08:07:09.337 INFO:tasks.workunit.client.1.smithi148.stdout:4/96: write df/d10/f11 [132467,128071] 0 2024-03-29T08:07:09.337 INFO:tasks.workunit.client.1.smithi148.stdout:8/161: mknod d17/d18/c37 0 2024-03-29T08:07:09.337 INFO:tasks.workunit.client.0.smithi037.stdout:6/110: rename d0/d1/l24 to d0/d5/l25 0 2024-03-29T08:07:09.338 INFO:tasks.workunit.client.1.smithi148.stdout:1/115: mkdir d3/d13/d23 0 2024-03-29T08:07:09.340 INFO:tasks.workunit.client.0.smithi037.stdout:9/273: dwrite d1/d4/d19/d1e/f3f [0,4194304] 0 2024-03-29T08:07:09.340 INFO:tasks.workunit.client.0.smithi037.stdout:7/218: mknod dd/d21/d35/d3b/c4a 0 2024-03-29T08:07:09.344 INFO:tasks.workunit.client.1.smithi148.stdout:0/152: creat db/de/df/d2b/f35 x:0 0 0 2024-03-29T08:07:09.345 INFO:tasks.workunit.client.0.smithi037.stdout:2/107: rename d0/d1e to d0/d20/d22 0 2024-03-29T08:07:09.345 INFO:tasks.workunit.client.1.smithi148.stdout:3/142: rename de/f1c to de/f2e 0 2024-03-29T08:07:09.346 INFO:tasks.workunit.client.0.smithi037.stdout:1/206: mknod d2/d6/dc/d18/d1d/d21/d38/c49 0 2024-03-29T08:07:09.346 INFO:tasks.workunit.client.0.smithi037.stdout:7/219: chown dd/f48 50171806 1 2024-03-29T08:07:09.349 INFO:tasks.workunit.client.0.smithi037.stdout:2/108: mkdir d0/d23 0 2024-03-29T08:07:09.351 INFO:tasks.workunit.client.1.smithi148.stdout:7/166: dwrite d8/d14/f30 [0,4194304] 0 2024-03-29T08:07:09.351 INFO:tasks.workunit.client.0.smithi037.stdout:4/78: dread d3/d4/d8/db/f16 [0,4194304] 0 2024-03-29T08:07:09.351 INFO:tasks.workunit.client.0.smithi037.stdout:7/220: write dd/ff [1687070,42501] 0 2024-03-29T08:07:09.353 INFO:tasks.workunit.client.1.smithi148.stdout:2/141: readlink l6 0 2024-03-29T08:07:09.357 INFO:tasks.workunit.client.1.smithi148.stdout:6/124: unlink d6/d9/l12 0 2024-03-29T08:07:09.358 INFO:tasks.workunit.client.1.smithi148.stdout:8/162: mkdir d17/d18/d38 0 2024-03-29T08:07:09.360 INFO:tasks.workunit.client.1.smithi148.stdout:6/125: write d6/d9/f16 [600747,43569] 0 2024-03-29T08:07:09.360 INFO:tasks.workunit.client.0.smithi037.stdout:1/207: creat d2/d6/dc/f4a x:0 0 0 2024-03-29T08:07:09.360 INFO:tasks.workunit.client.0.smithi037.stdout:7/221: write dd/d21/f44 [701085,81013] 0 2024-03-29T08:07:09.361 INFO:tasks.workunit.client.1.smithi148.stdout:1/116: fdatasync d3/fa 0 2024-03-29T08:07:09.365 INFO:tasks.workunit.client.1.smithi148.stdout:4/97: creat df/d10/d17/f20 x:0 0 0 2024-03-29T08:07:09.365 INFO:tasks.workunit.client.0.smithi037.stdout:6/111: fdatasync d0/d5/d1e/f12 0 2024-03-29T08:07:09.366 INFO:tasks.workunit.client.1.smithi148.stdout:6/126: write d6/d9/f13 [4472682,51436] 0 2024-03-29T08:07:09.367 INFO:tasks.workunit.client.1.smithi148.stdout:3/143: creat de/f2f x:0 0 0 2024-03-29T08:07:09.368 INFO:tasks.workunit.client.1.smithi148.stdout:0/153: creat db/de/df/d11/d2d/f36 x:0 0 0 2024-03-29T08:07:09.369 INFO:tasks.workunit.client.1.smithi148.stdout:9/121: symlink db/d22/d26/l27 0 2024-03-29T08:07:09.370 INFO:tasks.workunit.client.0.smithi037.stdout:2/109: link d0/c17 d0/c24 0 2024-03-29T08:07:09.371 INFO:tasks.workunit.client.1.smithi148.stdout:3/144: write de/d13/f23 [992629,104770] 0 2024-03-29T08:07:09.371 INFO:tasks.workunit.client.1.smithi148.stdout:2/142: write d7/f19 [707967,99026] 0 2024-03-29T08:07:09.372 INFO:tasks.workunit.client.0.smithi037.stdout:6/112: mknod d0/d19/c26 0 2024-03-29T08:07:09.372 INFO:tasks.workunit.client.1.smithi148.stdout:3/145: dread - de/f2c zero size 2024-03-29T08:07:09.373 INFO:tasks.workunit.client.1.smithi148.stdout:7/167: rename d8/d2f to d8/d38/d3c 0 2024-03-29T08:07:09.373 INFO:tasks.workunit.client.0.smithi037.stdout:2/110: fdatasync d0/f1f 0 2024-03-29T08:07:09.375 INFO:tasks.workunit.client.0.smithi037.stdout:5/185: dwrite d8/d18/f30 [0,4194304] 0 2024-03-29T08:07:09.376 INFO:tasks.workunit.client.1.smithi148.stdout:3/146: write de/f17 [56014,56288] 0 2024-03-29T08:07:09.381 INFO:tasks.workunit.client.1.smithi148.stdout:4/98: mknod df/d10/c21 0 2024-03-29T08:07:09.382 INFO:tasks.workunit.client.0.smithi037.stdout:9/274: link d1/d4/c51 d1/d4/d19/d1e/d25/d10/d13/c5a 0 2024-03-29T08:07:09.382 INFO:tasks.workunit.client.1.smithi148.stdout:9/122: chown db/d20/c1d 6240653 1 2024-03-29T08:07:09.384 INFO:tasks.workunit.client.0.smithi037.stdout:6/113: dread d0/d5/fa [0,4194304] 0 2024-03-29T08:07:09.388 INFO:tasks.workunit.client.0.smithi037.stdout:5/186: mkdir d8/d34/d3b 0 2024-03-29T08:07:09.390 INFO:tasks.workunit.client.0.smithi037.stdout:7/222: fdatasync dd/d1d/f2a 0 2024-03-29T08:07:09.391 INFO:tasks.workunit.client.0.smithi037.stdout:4/79: dread d3/d4/d8/db/f16 [0,4194304] 0 2024-03-29T08:07:09.393 INFO:tasks.workunit.client.0.smithi037.stdout:5/187: write d8/f2b [827651,61017] 0 2024-03-29T08:07:09.394 INFO:tasks.workunit.client.1.smithi148.stdout:2/143: dwrite d7/da/d1c/f28 [0,4194304] 0 2024-03-29T08:07:09.394 INFO:tasks.workunit.client.0.smithi037.stdout:2/111: symlink d0/d20/l25 0 2024-03-29T08:07:09.395 INFO:tasks.workunit.client.1.smithi148.stdout:4/99: creat df/d10/d17/f22 x:0 0 0 2024-03-29T08:07:09.397 INFO:tasks.workunit.client.1.smithi148.stdout:9/123: symlink db/d10/l28 0 2024-03-29T08:07:09.397 INFO:tasks.workunit.client.1.smithi148.stdout:8/163: dwrite d17/d18/d26/f27 [0,4194304] 0 2024-03-29T08:07:09.398 INFO:tasks.workunit.client.0.smithi037.stdout:5/188: chown d8/f1e 0 1 2024-03-29T08:07:09.399 INFO:tasks.workunit.client.0.smithi037.stdout:5/189: chown d8/f17 58125221 1 2024-03-29T08:07:09.406 INFO:tasks.workunit.client.1.smithi148.stdout:4/100: symlink df/l23 0 2024-03-29T08:07:09.409 INFO:tasks.workunit.client.1.smithi148.stdout:4/101: chown df/f1f 89966 1 2024-03-29T08:07:09.411 INFO:tasks.workunit.client.1.smithi148.stdout:4/102: fsync df/d10/f12 0 2024-03-29T08:07:09.413 INFO:tasks.workunit.client.1.smithi148.stdout:9/124: creat db/d10/f29 x:0 0 0 2024-03-29T08:07:09.413 INFO:tasks.workunit.client.0.smithi037.stdout:9/275: dread d1/d1f/f33 [0,4194304] 0 2024-03-29T08:07:09.416 INFO:tasks.workunit.client.1.smithi148.stdout:4/103: truncate df/d10/d17/f20 501462 0 2024-03-29T08:07:09.418 INFO:tasks.workunit.client.0.smithi037.stdout:4/80: unlink d3/d4/d8/f1c 0 2024-03-29T08:07:09.419 INFO:tasks.workunit.client.1.smithi148.stdout:7/168: write d8/f12 [1319437,103227] 0 2024-03-29T08:07:09.421 INFO:tasks.workunit.client.1.smithi148.stdout:1/117: write d3/d13/f1a [461790,102151] 0 2024-03-29T08:07:09.423 INFO:tasks.workunit.client.1.smithi148.stdout:6/127: truncate d6/f19 4222736 0 2024-03-29T08:07:09.423 INFO:tasks.workunit.client.1.smithi148.stdout:0/154: truncate db/de/df/d11/f1b 566217 0 2024-03-29T08:07:09.424 INFO:tasks.workunit.client.1.smithi148.stdout:6/128: stat d6/d9/f16 0 2024-03-29T08:07:09.425 INFO:tasks.workunit.client.1.smithi148.stdout:1/118: truncate d3/d18/f20 915528 0 2024-03-29T08:07:09.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:07:08 smithi037 ceph-mon[35162]: pgmap v36: 65 pgs: 65 active+clean; 418 MiB data, 3.1 GiB used, 533 GiB / 536 GiB avail; 842 KiB/s rd, 41 MiB/s wr, 265 op/s 2024-03-29T08:07:09.427 INFO:tasks.workunit.client.1.smithi148.stdout:8/164: mkdir d17/d18/d38/d39 0 2024-03-29T08:07:09.427 INFO:tasks.workunit.client.1.smithi148.stdout:9/125: dread db/d10/f13 [0,4194304] 0 2024-03-29T08:07:09.428 INFO:tasks.workunit.client.1.smithi148.stdout:0/155: fdatasync db/de/df/d11/d2d/f31 0 2024-03-29T08:07:09.428 INFO:tasks.workunit.client.1.smithi148.stdout:6/129: read d6/fe [4025210,15664] 0 2024-03-29T08:07:09.429 INFO:tasks.workunit.client.0.smithi037.stdout:9/276: write d1/d3/f21 [3228054,86671] 0 2024-03-29T08:07:09.430 INFO:tasks.workunit.client.1.smithi148.stdout:8/165: write fe [3304045,76001] 0 2024-03-29T08:07:09.430 INFO:tasks.workunit.client.1.smithi148.stdout:6/130: stat d6/f14 0 2024-03-29T08:07:09.431 INFO:tasks.workunit.client.1.smithi148.stdout:3/147: getdents de/d13 0 2024-03-29T08:07:09.431 INFO:tasks.workunit.client.1.smithi148.stdout:4/104: symlink df/d10/d17/l24 0 2024-03-29T08:07:09.432 INFO:tasks.workunit.client.1.smithi148.stdout:8/166: readlink d17/d1d/l29 0 2024-03-29T08:07:09.432 INFO:tasks.workunit.client.1.smithi148.stdout:3/148: chown de/d24 487929636 1 2024-03-29T08:07:09.433 INFO:tasks.workunit.client.1.smithi148.stdout:6/131: write d6/d9/f1f [924923,65061] 0 2024-03-29T08:07:09.434 INFO:tasks.workunit.client.0.smithi037.stdout:4/81: write d3/d4/fa [4548894,118435] 0 2024-03-29T08:07:09.434 INFO:tasks.workunit.client.1.smithi148.stdout:3/149: chown de/f11 101 1 2024-03-29T08:07:09.434 INFO:tasks.workunit.client.1.smithi148.stdout:7/169: mknod d8/d38/c3d 0 2024-03-29T08:07:09.435 INFO:tasks.workunit.client.1.smithi148.stdout:8/167: chown d17/f2d 5 1 2024-03-29T08:07:09.437 INFO:tasks.workunit.client.0.smithi037.stdout:9/277: write d1/d1f/d3a/f46 [2983409,109153] 0 2024-03-29T08:07:09.437 INFO:tasks.workunit.client.0.smithi037.stdout:6/114: dwrite d0/d5/f9 [0,4194304] 0 2024-03-29T08:07:09.437 INFO:tasks.workunit.client.0.smithi037.stdout:5/190: dread d8/d22/f31 [0,4194304] 0 2024-03-29T08:07:09.438 INFO:tasks.workunit.client.1.smithi148.stdout:2/144: write d7/da/d33/f35 [3925375,16224] 0 2024-03-29T08:07:09.442 INFO:tasks.workunit.client.1.smithi148.stdout:9/126: truncate f7 4593424 0 2024-03-29T08:07:09.443 INFO:tasks.workunit.client.1.smithi148.stdout:0/156: creat db/de/df/d11/f37 x:0 0 0 2024-03-29T08:07:09.448 INFO:tasks.workunit.client.1.smithi148.stdout:5/110: sync 2024-03-29T08:07:09.449 INFO:tasks.workunit.client.0.smithi037.stdout:6/115: creat d0/d19/f27 x:0 0 0 2024-03-29T08:07:09.449 INFO:tasks.workunit.client.1.smithi148.stdout:3/150: mkdir de/d13/d30 0 2024-03-29T08:07:09.451 INFO:tasks.workunit.client.0.smithi037.stdout:9/278: symlink d1/d4/d19/d1e/d25/d10/d13/l5b 0 2024-03-29T08:07:09.451 INFO:tasks.workunit.client.1.smithi148.stdout:2/145: creat d7/d24/f36 x:0 0 0 2024-03-29T08:07:09.452 INFO:tasks.workunit.client.0.smithi037.stdout:4/82: rename d3/d4/d8 to d3/d4/d15/d1d 0 2024-03-29T08:07:09.453 INFO:tasks.workunit.client.1.smithi148.stdout:4/105: dwrite df/f14 [0,4194304] 0 2024-03-29T08:07:09.453 INFO:tasks.workunit.client.1.smithi148.stdout:2/146: dread - d7/f10 zero size 2024-03-29T08:07:09.455 INFO:tasks.workunit.client.1.smithi148.stdout:0/157: creat db/de/d24/f38 x:0 0 0 2024-03-29T08:07:09.456 INFO:tasks.workunit.client.1.smithi148.stdout:6/132: truncate d6/fa 2234403 0 2024-03-29T08:07:09.457 INFO:tasks.workunit.client.0.smithi037.stdout:9/279: rmdir d1/d1f 39 2024-03-29T08:07:09.458 INFO:tasks.workunit.client.0.smithi037.stdout:4/83: fsync f0 0 2024-03-29T08:07:09.459 INFO:tasks.workunit.client.0.smithi037.stdout:2/112: dread d0/f14 [0,4194304] 0 2024-03-29T08:07:09.460 INFO:tasks.workunit.client.0.smithi037.stdout:5/191: write d8/d18/f21 [777854,18824] 0 2024-03-29T08:07:09.462 INFO:tasks.workunit.client.0.smithi037.stdout:6/116: rename d0/d1/le to d0/d5/d1e/l28 0 2024-03-29T08:07:09.463 INFO:tasks.workunit.client.1.smithi148.stdout:7/170: mknod d8/d14/d19/d37/c3e 0 2024-03-29T08:07:09.465 INFO:tasks.workunit.client.1.smithi148.stdout:4/106: chown f3 6215 1 2024-03-29T08:07:09.465 INFO:tasks.workunit.client.1.smithi148.stdout:8/168: rmdir d17 39 2024-03-29T08:07:09.468 INFO:tasks.workunit.client.1.smithi148.stdout:7/171: truncate d8/d14/f2d 945408 0 2024-03-29T08:07:09.469 INFO:tasks.workunit.client.1.smithi148.stdout:0/158: mknod db/de/df/c39 0 2024-03-29T08:07:09.470 INFO:tasks.workunit.client.0.smithi037.stdout:7/223: dread dd/f16 [0,4194304] 0 2024-03-29T08:07:09.470 INFO:tasks.workunit.client.1.smithi148.stdout:3/151: dread de/f15 [0,4194304] 0 2024-03-29T08:07:09.471 INFO:tasks.workunit.client.1.smithi148.stdout:6/133: mkdir d6/d9/d1e/d20 0 2024-03-29T08:07:09.473 INFO:tasks.workunit.client.0.smithi037.stdout:9/280: read d1/d1f/d3a/f27 [1486815,25892] 0 2024-03-29T08:07:09.474 INFO:tasks.workunit.client.1.smithi148.stdout:1/119: truncate d3/d18/f20 397752 0 2024-03-29T08:07:09.474 INFO:tasks.workunit.client.1.smithi148.stdout:6/134: stat d6/d9/c18 0 2024-03-29T08:07:09.476 INFO:tasks.workunit.client.0.smithi037.stdout:5/192: symlink d8/d34/l3c 0 2024-03-29T08:07:09.478 INFO:tasks.workunit.client.1.smithi148.stdout:1/120: chown d3/lf 748 1 2024-03-29T08:07:09.480 INFO:tasks.workunit.client.0.smithi037.stdout:7/224: symlink dd/d21/l4b 0 2024-03-29T08:07:09.482 INFO:tasks.workunit.client.1.smithi148.stdout:7/172: creat d8/d38/d3c/f3f x:0 0 0 2024-03-29T08:07:09.482 INFO:tasks.workunit.client.0.smithi037.stdout:2/113: creat d0/d23/f26 x:0 0 0 2024-03-29T08:07:09.483 INFO:tasks.workunit.client.1.smithi148.stdout:9/127: dwrite db/d10/f13 [0,4194304] 0 2024-03-29T08:07:09.483 INFO:tasks.workunit.client.1.smithi148.stdout:4/107: link df/d10/d17/f20 df/d10/d17/f25 0 2024-03-29T08:07:09.486 INFO:tasks.workunit.client.0.smithi037.stdout:9/281: write d1/d4/d19/d1e/d25/d10/d31/f48 [269358,62915] 0 2024-03-29T08:07:09.487 INFO:tasks.workunit.client.0.smithi037.stdout:7/225: creat dd/d21/d35/f4c x:0 0 0 2024-03-29T08:07:09.487 INFO:tasks.workunit.client.1.smithi148.stdout:8/169: creat d17/d18/f3a x:0 0 0 2024-03-29T08:07:09.487 INFO:tasks.workunit.client.1.smithi148.stdout:4/108: stat df/d10/f11 0 2024-03-29T08:07:09.488 INFO:tasks.workunit.client.1.smithi148.stdout:1/121: symlink d3/de/d1d/l24 0 2024-03-29T08:07:09.489 INFO:tasks.workunit.client.1.smithi148.stdout:9/128: dread - db/d20/f1e zero size 2024-03-29T08:07:09.490 INFO:tasks.workunit.client.0.smithi037.stdout:2/114: write d0/f14 [4712888,12596] 0 2024-03-29T08:07:09.492 INFO:tasks.workunit.client.1.smithi148.stdout:6/135: dread d6/d9/f1f [0,4194304] 0 2024-03-29T08:07:09.493 INFO:tasks.workunit.client.0.smithi037.stdout:6/117: readlink d0/d5/d1e/l28 0 2024-03-29T08:07:09.496 INFO:tasks.workunit.client.0.smithi037.stdout:6/118: write d0/f4 [696492,43226] 0 2024-03-29T08:07:09.496 INFO:tasks.workunit.client.1.smithi148.stdout:8/170: chown d17/d18/f34 1 1 2024-03-29T08:07:09.497 INFO:tasks.workunit.client.1.smithi148.stdout:4/109: mkdir df/d10/d26 0 2024-03-29T08:07:09.498 INFO:tasks.workunit.client.0.smithi037.stdout:6/119: write d0/f4 [1416265,29891] 0 2024-03-29T08:07:09.498 INFO:tasks.workunit.client.1.smithi148.stdout:9/129: write f3 [4357617,120530] 0 2024-03-29T08:07:09.500 INFO:tasks.workunit.client.0.smithi037.stdout:2/115: mknod d0/d20/c27 0 2024-03-29T08:07:09.500 INFO:tasks.workunit.client.0.smithi037.stdout:5/193: getdents d8/d22 0 2024-03-29T08:07:09.504 INFO:tasks.workunit.client.0.smithi037.stdout:9/282: getdents d1/d1f/d3a 0 2024-03-29T08:07:09.512 INFO:tasks.workunit.client.1.smithi148.stdout:6/136: link d6/d9/lb d6/l21 0 2024-03-29T08:07:09.512 INFO:tasks.workunit.client.1.smithi148.stdout:5/111: dwrite d0/f9 [0,4194304] 0 2024-03-29T08:07:09.519 INFO:tasks.workunit.client.1.smithi148.stdout:3/152: dwrite de/f15 [0,4194304] 0 2024-03-29T08:07:09.521 INFO:tasks.workunit.client.1.smithi148.stdout:8/171: mkdir d17/d18/d26/d3b 0 2024-03-29T08:07:09.526 INFO:tasks.workunit.client.1.smithi148.stdout:6/137: read f5 [640358,62060] 0 2024-03-29T08:07:09.528 INFO:tasks.workunit.client.0.smithi037.stdout:4/84: dread d3/d4/fa [0,4194304] 0 2024-03-29T08:07:09.529 INFO:tasks.workunit.client.1.smithi148.stdout:0/159: dwrite db/de/df/d11/f37 [0,4194304] 0 2024-03-29T08:07:09.531 INFO:tasks.workunit.client.0.smithi037.stdout:7/226: dwrite dd/d1d/d34/f40 [0,4194304] 0 2024-03-29T08:07:09.531 INFO:tasks.workunit.client.1.smithi148.stdout:6/138: readlink d6/l1c 0 2024-03-29T08:07:09.534 INFO:tasks.workunit.client.1.smithi148.stdout:7/173: dread d8/fe [0,4194304] 0 2024-03-29T08:07:09.536 INFO:tasks.workunit.client.0.smithi037.stdout:7/227: chown dd/d21/f2c 86 1 2024-03-29T08:07:09.537 INFO:tasks.workunit.client.1.smithi148.stdout:3/153: creat de/d24/f31 x:0 0 0 2024-03-29T08:07:09.538 INFO:tasks.workunit.client.0.smithi037.stdout:2/116: dwrite d0/fa [0,4194304] 0 2024-03-29T08:07:09.540 INFO:tasks.workunit.client.1.smithi148.stdout:1/122: dwrite d3/fa [0,4194304] 0 2024-03-29T08:07:09.540 INFO:tasks.workunit.client.0.smithi037.stdout:6/120: dwrite d0/d5/d1a/f23 [0,4194304] 0 2024-03-29T08:07:09.542 INFO:tasks.workunit.client.0.smithi037.stdout:2/117: readlink d0/d20/l25 0 2024-03-29T08:07:09.545 INFO:tasks.workunit.client.0.smithi037.stdout:9/283: dwrite d1/d4/d19/d1e/d25/d10/d31/f48 [4194304,4194304] 0 2024-03-29T08:07:09.545 INFO:tasks.workunit.client.1.smithi148.stdout:0/160: unlink db/de/df/d11/c21 0 2024-03-29T08:07:09.545 INFO:tasks.workunit.client.1.smithi148.stdout:2/147: dread d7/d24/f2c [0,4194304] 0 2024-03-29T08:07:09.548 INFO:tasks.workunit.client.0.smithi037.stdout:6/121: read d0/d5/d1a/f23 [1706573,127568] 0 2024-03-29T08:07:09.548 INFO:tasks.workunit.client.0.smithi037.stdout:7/228: dread dd/d1d/d34/f3f [0,4194304] 0 2024-03-29T08:07:09.549 INFO:tasks.workunit.client.0.smithi037.stdout:2/118: rmdir d0/d23 39 2024-03-29T08:07:09.551 INFO:tasks.workunit.client.1.smithi148.stdout:5/112: dwrite d0/f3 [0,4194304] 0 2024-03-29T08:07:09.552 INFO:tasks.workunit.client.0.smithi037.stdout:5/194: write d8/d18/f19 [3370913,75654] 0 2024-03-29T08:07:09.558 INFO:tasks.workunit.client.1.smithi148.stdout:6/139: mkdir d6/d9/d1e/d22 0 2024-03-29T08:07:09.559 INFO:tasks.workunit.client.0.smithi037.stdout:6/122: mkdir d0/d20/d29 0 2024-03-29T08:07:09.559 INFO:tasks.workunit.client.0.smithi037.stdout:9/284: write d1/d4/d19/d1e/d25/d10/d31/f38 [654113,126530] 0 2024-03-29T08:07:09.560 INFO:tasks.workunit.client.1.smithi148.stdout:6/140: fsync d6/d9/f13 0 2024-03-29T08:07:09.560 INFO:tasks.workunit.client.0.smithi037.stdout:5/195: chown d8/d18/d24/l36 130422237 1 2024-03-29T08:07:09.566 INFO:tasks.workunit.client.0.smithi037.stdout:9/285: rmdir d1/d4 39 2024-03-29T08:07:09.567 INFO:tasks.workunit.client.0.smithi037.stdout:1/208: sync 2024-03-29T08:07:09.567 INFO:tasks.workunit.client.0.smithi037.stdout:0/99: sync 2024-03-29T08:07:09.567 INFO:tasks.workunit.client.0.smithi037.stdout:3/194: sync 2024-03-29T08:07:09.567 INFO:tasks.workunit.client.0.smithi037.stdout:8/177: sync 2024-03-29T08:07:09.568 INFO:tasks.workunit.client.1.smithi148.stdout:4/110: dwrite fa [0,4194304] 0 2024-03-29T08:07:09.568 INFO:tasks.workunit.client.1.smithi148.stdout:7/174: dread d8/f12 [0,4194304] 0 2024-03-29T08:07:09.571 INFO:tasks.workunit.client.0.smithi037.stdout:3/195: stat dd/d15/c28 0 2024-03-29T08:07:09.571 INFO:tasks.workunit.client.0.smithi037.stdout:5/196: mknod d8/d34/c3d 0 2024-03-29T08:07:09.573 INFO:tasks.workunit.client.1.smithi148.stdout:4/111: readlink df/d10/d17/l24 0 2024-03-29T08:07:09.573 INFO:tasks.workunit.client.1.smithi148.stdout:0/161: symlink db/dc/l3a 0 2024-03-29T08:07:09.576 INFO:tasks.workunit.client.0.smithi037.stdout:5/197: readlink d8/d22/l32 0 2024-03-29T08:07:09.576 INFO:tasks.workunit.client.0.smithi037.stdout:1/209: mkdir d2/d4b 0 2024-03-29T08:07:09.577 INFO:tasks.workunit.client.0.smithi037.stdout:3/196: truncate dd/d15/d27/f41 372475 0 2024-03-29T08:07:09.577 INFO:tasks.workunit.client.0.smithi037.stdout:4/85: write d3/d4/d15/f19 [393646,122726] 0 2024-03-29T08:07:09.578 INFO:tasks.workunit.client.1.smithi148.stdout:0/162: chown db/de/df/d11/f33 618360 1 2024-03-29T08:07:09.582 INFO:tasks.workunit.client.1.smithi148.stdout:0/163: truncate db/de/df/d2b/f35 1017204 0 2024-03-29T08:07:09.583 INFO:tasks.workunit.client.0.smithi037.stdout:2/119: getdents d0 0 2024-03-29T08:07:09.585 INFO:tasks.workunit.client.0.smithi037.stdout:4/86: chown d3/c6 922657 1 2024-03-29T08:07:09.586 INFO:tasks.workunit.client.0.smithi037.stdout:2/120: chown d0/f14 18 1 2024-03-29T08:07:09.587 INFO:tasks.workunit.client.0.smithi037.stdout:0/100: mkdir d9/da/dd/d20 0 2024-03-29T08:07:09.587 INFO:tasks.workunit.client.1.smithi148.stdout:6/141: creat d6/d9/f23 x:0 0 0 2024-03-29T08:07:09.590 INFO:tasks.workunit.client.1.smithi148.stdout:6/142: write d6/f14 [1296009,18056] 0 2024-03-29T08:07:09.592 INFO:tasks.workunit.client.0.smithi037.stdout:6/123: creat d0/d5/f2a x:0 0 0 2024-03-29T08:07:09.592 INFO:tasks.workunit.client.1.smithi148.stdout:9/130: sync 2024-03-29T08:07:09.593 INFO:tasks.workunit.client.1.smithi148.stdout:7/175: mknod d8/d38/d3c/c40 0 2024-03-29T08:07:09.594 INFO:tasks.workunit.client.0.smithi037.stdout:5/198: chown d8/d18/d24/c33 0 1 2024-03-29T08:07:09.595 INFO:tasks.workunit.client.1.smithi148.stdout:4/112: creat df/f27 x:0 0 0 2024-03-29T08:07:09.598 INFO:tasks.workunit.client.1.smithi148.stdout:2/148: mkdir d7/d13/d2e/d37 0 2024-03-29T08:07:09.600 INFO:tasks.workunit.client.1.smithi148.stdout:5/113: rename d0/f3 to d0/f21 0 2024-03-29T08:07:09.601 INFO:tasks.workunit.client.0.smithi037.stdout:9/286: mkdir d1/d4/d19/d1e/d4a/d5c 0 2024-03-29T08:07:09.601 INFO:tasks.workunit.client.1.smithi148.stdout:2/149: write d7/d13/f2a [895755,69414] 0 2024-03-29T08:07:09.603 INFO:tasks.workunit.client.1.smithi148.stdout:5/114: dread - d0/db/de/d16/f1b zero size 2024-03-29T08:07:09.605 INFO:tasks.workunit.client.0.smithi037.stdout:2/121: dread d0/f2 [0,4194304] 0 2024-03-29T08:07:09.606 INFO:tasks.workunit.client.1.smithi148.stdout:6/143: creat d6/d9/d1e/d20/f24 x:0 0 0 2024-03-29T08:07:09.608 INFO:tasks.workunit.client.1.smithi148.stdout:5/115: stat d0/ca 0 2024-03-29T08:07:09.608 INFO:tasks.workunit.client.0.smithi037.stdout:8/178: mknod d5/d10/d2d/d34/c38 0 2024-03-29T08:07:09.608 INFO:tasks.workunit.client.0.smithi037.stdout:0/101: write d9/fb [419758,110687] 0 2024-03-29T08:07:09.609 INFO:tasks.workunit.client.0.smithi037.stdout:7/229: dwrite dd/f14 [0,4194304] 0 2024-03-29T08:07:09.610 INFO:tasks.workunit.client.1.smithi148.stdout:6/144: symlink d6/d9/l25 0 2024-03-29T08:07:09.611 INFO:tasks.workunit.client.1.smithi148.stdout:9/131: dwrite f3 [4194304,4194304] 0 2024-03-29T08:07:09.611 INFO:tasks.workunit.client.1.smithi148.stdout:3/154: dread de/f17 [0,4194304] 0 2024-03-29T08:07:09.612 INFO:tasks.workunit.client.1.smithi148.stdout:9/132: stat db/f21 0 2024-03-29T08:07:09.614 INFO:tasks.workunit.client.0.smithi037.stdout:8/179: write d5/d25/f28 [2407099,126305] 0 2024-03-29T08:07:09.615 INFO:tasks.workunit.client.0.smithi037.stdout:1/210: symlink d2/d4b/l4c 0 2024-03-29T08:07:09.615 INFO:tasks.workunit.client.1.smithi148.stdout:5/116: rename d0/db/l14 to d0/db/l22 0 2024-03-29T08:07:09.621 INFO:tasks.workunit.client.0.smithi037.stdout:1/211: dread d2/d6/dc/f25 [0,4194304] 0 2024-03-29T08:07:09.622 INFO:tasks.workunit.client.1.smithi148.stdout:2/150: dwrite d7/f2d [0,4194304] 0 2024-03-29T08:07:09.622 INFO:tasks.workunit.client.0.smithi037.stdout:3/197: mknod dd/d46/d2d/d3b/c4c 0 2024-03-29T08:07:09.625 INFO:tasks.workunit.client.1.smithi148.stdout:9/133: unlink db/d22/l24 0 2024-03-29T08:07:09.628 INFO:tasks.workunit.client.0.smithi037.stdout:4/87: mkdir d3/d11/d1e 0 2024-03-29T08:07:09.628 INFO:tasks.workunit.client.0.smithi037.stdout:5/199: mknod d8/d34/d3b/c3e 0 2024-03-29T08:07:09.629 INFO:tasks.workunit.client.0.smithi037.stdout:6/124: mknod d0/d5/d1f/c2b 0 2024-03-29T08:07:09.631 INFO:tasks.workunit.client.0.smithi037.stdout:1/212: write d2/d6/dc/d18/d22/f48 [846644,18500] 0 2024-03-29T08:07:09.632 INFO:tasks.workunit.client.1.smithi148.stdout:8/172: sync 2024-03-29T08:07:09.633 INFO:tasks.workunit.client.1.smithi148.stdout:7/176: dread d8/f13 [0,4194304] 0 2024-03-29T08:07:09.637 INFO:tasks.workunit.client.1.smithi148.stdout:2/151: creat d7/d13/f38 x:0 0 0 2024-03-29T08:07:09.641 INFO:tasks.workunit.client.0.smithi037.stdout:1/213: chown d2/d6/dc/f4a 1797 1 2024-03-29T08:07:09.641 INFO:tasks.workunit.client.0.smithi037.stdout:8/180: rename d5/d10/l20 to d5/l39 0 2024-03-29T08:07:09.642 INFO:tasks.workunit.client.1.smithi148.stdout:1/123: chown d3/d18/f20 61838 1 2024-03-29T08:07:09.642 INFO:tasks.workunit.client.0.smithi037.stdout:3/198: symlink dd/d46/d2d/l4d 0 2024-03-29T08:07:09.643 INFO:tasks.workunit.client.1.smithi148.stdout:9/134: rename db/d10/f15 to db/d22/d26/f2a 0 2024-03-29T08:07:09.645 INFO:tasks.workunit.client.0.smithi037.stdout:9/287: creat d1/d4/d19/d1e/d4a/d5c/f5d x:0 0 0 2024-03-29T08:07:09.649 INFO:tasks.workunit.client.1.smithi148.stdout:5/117: mkdir d0/db/d1e/d23 0 2024-03-29T08:07:09.650 INFO:tasks.workunit.client.0.smithi037.stdout:2/122: getdents d0/d20/d22 0 2024-03-29T08:07:09.655 INFO:tasks.workunit.client.1.smithi148.stdout:8/173: unlink d17/d1d/f20 0 2024-03-29T08:07:09.655 INFO:tasks.workunit.client.0.smithi037.stdout:8/181: creat d5/f3a x:0 0 0 2024-03-29T08:07:09.655 INFO:tasks.workunit.client.0.smithi037.stdout:7/230: getdents dd/d1d/d49 0 2024-03-29T08:07:09.659 INFO:tasks.workunit.client.0.smithi037.stdout:5/200: link d8/d22/f27 d8/d34/d3b/f3f 0 2024-03-29T08:07:09.660 INFO:tasks.workunit.client.0.smithi037.stdout:4/88: dread d3/d4/d15/d1d/db/f12 [0,4194304] 0 2024-03-29T08:07:09.662 INFO:tasks.workunit.client.0.smithi037.stdout:9/288: mknod d1/d4/d19/d1e/d25/d10/d13/c5e 0 2024-03-29T08:07:09.663 INFO:tasks.workunit.client.1.smithi148.stdout:2/152: mkdir d7/d39 0 2024-03-29T08:07:09.666 INFO:tasks.workunit.client.0.smithi037.stdout:0/102: dwrite f7 [4194304,4194304] 0 2024-03-29T08:07:09.668 INFO:tasks.workunit.client.1.smithi148.stdout:1/124: rmdir d3 39 2024-03-29T08:07:09.670 INFO:tasks.workunit.client.1.smithi148.stdout:8/174: dwrite d17/f2d [0,4194304] 0 2024-03-29T08:07:09.670 INFO:tasks.workunit.client.1.smithi148.stdout:4/113: dread f4 [0,4194304] 0 2024-03-29T08:07:09.671 INFO:tasks.workunit.client.0.smithi037.stdout:4/89: creat d3/d4/d15/f1f x:0 0 0 2024-03-29T08:07:09.672 INFO:tasks.workunit.client.1.smithi148.stdout:8/175: chown d17/d18/d26 1 1 2024-03-29T08:07:09.672 INFO:tasks.workunit.client.0.smithi037.stdout:5/201: write d8/d34/d3b/f3f [696869,29077] 0 2024-03-29T08:07:09.673 INFO:tasks.workunit.client.1.smithi148.stdout:9/135: mkdir db/d10/d14/d2b 0 2024-03-29T08:07:09.676 INFO:tasks.workunit.client.1.smithi148.stdout:4/114: dread df/d10/d17/f20 [0,4194304] 0 2024-03-29T08:07:09.676 INFO:tasks.workunit.client.1.smithi148.stdout:5/118: unlink d0/db/f1d 0 2024-03-29T08:07:09.680 INFO:tasks.workunit.client.0.smithi037.stdout:9/289: write d1/d4/d19/d1e/d25/d10/f12 [1642587,131029] 0 2024-03-29T08:07:09.680 INFO:tasks.workunit.client.1.smithi148.stdout:5/119: chown d0/l15 114 1 2024-03-29T08:07:09.681 INFO:tasks.workunit.client.1.smithi148.stdout:5/120: readlink d0/l4 0 2024-03-29T08:07:09.681 INFO:tasks.workunit.client.1.smithi148.stdout:5/121: readlink d0/l4 0 2024-03-29T08:07:09.682 INFO:tasks.workunit.client.1.smithi148.stdout:8/176: stat d17/d1d/c1e 0 2024-03-29T08:07:09.682 INFO:tasks.workunit.client.0.smithi037.stdout:0/103: rename f8 to d9/da/dd/d13/f21 0 2024-03-29T08:07:09.683 INFO:tasks.workunit.client.1.smithi148.stdout:1/125: write d3/de/f1b [126292,24972] 0 2024-03-29T08:07:09.684 INFO:tasks.workunit.client.1.smithi148.stdout:3/155: getdents de/d13 0 2024-03-29T08:07:09.684 INFO:tasks.workunit.client.1.smithi148.stdout:0/164: dwrite db/de/df/d11/f16 [0,4194304] 0 2024-03-29T08:07:09.684 INFO:tasks.workunit.client.1.smithi148.stdout:5/122: write d0/f12 [3194172,86532] 0 2024-03-29T08:07:09.687 INFO:tasks.workunit.client.1.smithi148.stdout:6/145: write d6/f19 [2759978,80126] 0 2024-03-29T08:07:09.688 INFO:tasks.workunit.client.1.smithi148.stdout:4/115: creat df/f28 x:0 0 0 2024-03-29T08:07:09.688 INFO:tasks.workunit.client.1.smithi148.stdout:2/153: symlink d7/da/l3a 0 2024-03-29T08:07:09.689 INFO:tasks.workunit.client.1.smithi148.stdout:6/146: readlink d6/d9/l1b 0 2024-03-29T08:07:09.689 INFO:tasks.workunit.client.0.smithi037.stdout:1/214: dwrite d2/d6/dc/f35 [0,4194304] 0 2024-03-29T08:07:09.690 INFO:tasks.workunit.client.1.smithi148.stdout:2/154: chown f1 815699 1 2024-03-29T08:07:09.691 INFO:tasks.workunit.client.1.smithi148.stdout:0/165: dread f9 [0,4194304] 0 2024-03-29T08:07:09.693 INFO:tasks.workunit.client.1.smithi148.stdout:4/116: chown df/f1f 1468621 1 2024-03-29T08:07:09.694 INFO:tasks.workunit.client.0.smithi037.stdout:3/199: dread dd/d15/d18/f35 [0,4194304] 0 2024-03-29T08:07:09.694 INFO:tasks.workunit.client.1.smithi148.stdout:4/117: chown l9 15 1 2024-03-29T08:07:09.694 INFO:tasks.workunit.client.1.smithi148.stdout:8/177: creat d17/f3c x:0 0 0 2024-03-29T08:07:09.698 INFO:tasks.workunit.client.0.smithi037.stdout:4/90: dread - d3/d4/d15/d1d/fc zero size 2024-03-29T08:07:09.699 INFO:tasks.workunit.client.0.smithi037.stdout:0/104: write f7 [6775665,79064] 0 2024-03-29T08:07:09.703 INFO:tasks.workunit.client.0.smithi037.stdout:2/123: dread d0/fa [0,4194304] 0 2024-03-29T08:07:09.704 INFO:tasks.workunit.client.1.smithi148.stdout:7/177: getdents d8/d29 0 2024-03-29T08:07:09.704 INFO:tasks.workunit.client.1.smithi148.stdout:5/123: symlink d0/l24 0 2024-03-29T08:07:09.704 INFO:tasks.workunit.client.1.smithi148.stdout:4/118: creat df/d10/d17/f29 x:0 0 0 2024-03-29T08:07:09.705 INFO:tasks.workunit.client.0.smithi037.stdout:1/215: truncate d2/f3a 960088 0 2024-03-29T08:07:09.705 INFO:tasks.workunit.client.0.smithi037.stdout:6/125: dwrite d0/d5/d1e/f12 [0,4194304] 0 2024-03-29T08:07:09.706 INFO:tasks.workunit.client.1.smithi148.stdout:8/178: write d17/f23 [644902,114399] 0 2024-03-29T08:07:09.708 INFO:tasks.workunit.client.1.smithi148.stdout:2/155: link d7/da/fe d7/f3b 0 2024-03-29T08:07:09.709 INFO:tasks.workunit.client.0.smithi037.stdout:3/200: write dd/d15/f19 [3179512,3039] 0 2024-03-29T08:07:09.710 INFO:tasks.workunit.client.1.smithi148.stdout:1/126: dwrite d3/d13/f15 [12582912,4194304] 0 2024-03-29T08:07:09.711 INFO:tasks.workunit.client.1.smithi148.stdout:4/119: dread df/d10/d17/f20 [0,4194304] 0 2024-03-29T08:07:09.711 INFO:tasks.workunit.client.1.smithi148.stdout:5/124: mknod d0/db/de/c25 0 2024-03-29T08:07:09.713 INFO:tasks.workunit.client.0.smithi037.stdout:1/216: chown d2/f31 2898 1 2024-03-29T08:07:09.715 INFO:tasks.workunit.client.1.smithi148.stdout:1/127: write d3/d13/f15 [9040902,48837] 0 2024-03-29T08:07:09.720 INFO:tasks.workunit.client.1.smithi148.stdout:4/120: symlink df/l2a 0 2024-03-29T08:07:09.721 INFO:tasks.workunit.client.0.smithi037.stdout:7/231: dwrite dd/f17 [0,4194304] 0 2024-03-29T08:07:09.722 INFO:tasks.workunit.client.0.smithi037.stdout:4/91: stat d3/d4/c14 0 2024-03-29T08:07:09.725 INFO:tasks.workunit.client.1.smithi148.stdout:0/166: getdents db/de/df/d11 0 2024-03-29T08:07:09.726 INFO:tasks.workunit.client.0.smithi037.stdout:0/105: read f5 [224348,53740] 0 2024-03-29T08:07:09.726 INFO:tasks.workunit.client.0.smithi037.stdout:7/232: read - dd/f3e zero size 2024-03-29T08:07:09.726 INFO:tasks.workunit.client.1.smithi148.stdout:1/128: mkdir d3/de/d25 0 2024-03-29T08:07:09.728 INFO:tasks.workunit.client.0.smithi037.stdout:3/201: rmdir dd/d46/d2d 39 2024-03-29T08:07:09.729 INFO:tasks.workunit.client.0.smithi037.stdout:1/217: rmdir d2/d4b 39 2024-03-29T08:07:09.729 INFO:tasks.workunit.client.0.smithi037.stdout:6/126: fsync d0/d5/f9 0 2024-03-29T08:07:09.735 INFO:tasks.workunit.client.1.smithi148.stdout:0/167: mkdir db/de/df/d2b/d3b 0 2024-03-29T08:07:09.736 INFO:tasks.workunit.client.0.smithi037.stdout:4/92: rmdir d3/d4/d15/d1d/db 39 2024-03-29T08:07:09.742 INFO:tasks.workunit.client.0.smithi037.stdout:1/218: write d2/d6/dc/d18/d1d/f2c [2515839,72759] 0 2024-03-29T08:07:09.743 INFO:tasks.workunit.client.0.smithi037.stdout:9/290: creat d1/d1f/f5f x:0 0 0 2024-03-29T08:07:09.743 INFO:tasks.workunit.client.1.smithi148.stdout:2/156: dwrite f1 [4194304,4194304] 0 2024-03-29T08:07:09.744 INFO:tasks.workunit.client.0.smithi037.stdout:3/202: symlink dd/d15/l4e 0 2024-03-29T08:07:09.744 INFO:tasks.workunit.client.1.smithi148.stdout:5/125: dwrite d0/f12 [0,4194304] 0 2024-03-29T08:07:09.746 INFO:tasks.workunit.client.1.smithi148.stdout:6/147: truncate d6/d9/f1f 95523 0 2024-03-29T08:07:09.748 INFO:tasks.workunit.client.0.smithi037.stdout:1/219: fsync d2/f3a 0 2024-03-29T08:07:09.749 INFO:tasks.workunit.client.1.smithi148.stdout:8/179: dwrite d17/d18/d26/f32 [0,4194304] 0 2024-03-29T08:07:09.749 INFO:tasks.workunit.client.0.smithi037.stdout:5/202: truncate d8/ff 2277563 0 2024-03-29T08:07:09.750 INFO:tasks.workunit.client.1.smithi148.stdout:7/178: truncate f4 3527413 0 2024-03-29T08:07:09.751 INFO:tasks.workunit.client.0.smithi037.stdout:3/203: fdatasync dd/d46/f2c 0 2024-03-29T08:07:09.755 INFO:tasks.workunit.client.0.smithi037.stdout:4/93: rmdir d3/d4/d15/d1d 39 2024-03-29T08:07:09.757 INFO:tasks.workunit.client.1.smithi148.stdout:1/129: dwrite d3/de/f1f [0,4194304] 0 2024-03-29T08:07:09.758 INFO:tasks.workunit.client.0.smithi037.stdout:4/94: fdatasync f0 0 2024-03-29T08:07:09.758 INFO:tasks.workunit.client.0.smithi037.stdout:3/204: chown dd/d15/d18/d30 13264895 1 2024-03-29T08:07:09.758 INFO:tasks.workunit.client.1.smithi148.stdout:7/179: truncate d8/d38/d3c/f3f 421129 0 2024-03-29T08:07:09.759 INFO:tasks.workunit.client.0.smithi037.stdout:2/124: dread d0/fd [0,4194304] 0 2024-03-29T08:07:09.759 INFO:tasks.workunit.client.1.smithi148.stdout:6/148: truncate d6/d9/d1e/d20/f24 61870 0 2024-03-29T08:07:09.759 INFO:tasks.workunit.client.1.smithi148.stdout:1/130: chown d3/d13 44419 1 2024-03-29T08:07:09.760 INFO:tasks.workunit.client.1.smithi148.stdout:4/121: write f3 [119301,79926] 0 2024-03-29T08:07:09.761 INFO:tasks.workunit.client.0.smithi037.stdout:6/127: fsync d0/d5/d1a/f23 0 2024-03-29T08:07:09.764 INFO:tasks.workunit.client.0.smithi037.stdout:9/291: dread - d1/d4/d19/d1e/d25/d10/d31/f4d zero size 2024-03-29T08:07:09.764 INFO:tasks.workunit.client.1.smithi148.stdout:5/126: getdents d0/db 0 2024-03-29T08:07:09.765 INFO:tasks.workunit.client.1.smithi148.stdout:7/180: mkdir d8/d29/d41 0 2024-03-29T08:07:09.766 INFO:tasks.workunit.client.0.smithi037.stdout:4/95: chown d3/d4/d15/f19 35111 1 2024-03-29T08:07:09.767 INFO:tasks.workunit.client.1.smithi148.stdout:6/149: creat d6/d9/d1e/f26 x:0 0 0 2024-03-29T08:07:09.770 INFO:tasks.workunit.client.0.smithi037.stdout:5/203: unlink d8/d34/c3d 0 2024-03-29T08:07:09.771 INFO:tasks.workunit.client.1.smithi148.stdout:1/131: mknod d3/de/d1d/c26 0 2024-03-29T08:07:09.772 INFO:tasks.workunit.client.1.smithi148.stdout:5/127: fdatasync d0/f21 0 2024-03-29T08:07:09.773 INFO:tasks.workunit.client.0.smithi037.stdout:5/204: readlink d8/d22/l2f 0 2024-03-29T08:07:09.773 INFO:tasks.workunit.client.0.smithi037.stdout:1/220: creat d2/d4b/f4d x:0 0 0 2024-03-29T08:07:09.774 INFO:tasks.workunit.client.1.smithi148.stdout:2/157: dwrite d7/f17 [4194304,4194304] 0 2024-03-29T08:07:09.775 INFO:tasks.workunit.client.1.smithi148.stdout:7/181: mknod d8/d14/d19/d37/c42 0 2024-03-29T08:07:09.776 INFO:tasks.workunit.client.0.smithi037.stdout:5/205: readlink d8/le 0 2024-03-29T08:07:09.776 INFO:tasks.workunit.client.1.smithi148.stdout:2/158: read - d7/d24/f34 zero size 2024-03-29T08:07:09.776 INFO:tasks.workunit.client.1.smithi148.stdout:8/180: dread d17/f28 [0,4194304] 0 2024-03-29T08:07:09.779 INFO:tasks.workunit.client.1.smithi148.stdout:6/150: mkdir d6/d9/d1e/d27 0 2024-03-29T08:07:09.781 INFO:tasks.workunit.client.0.smithi037.stdout:5/206: fsync d8/d34/d3b/f3f 0 2024-03-29T08:07:09.784 INFO:tasks.workunit.client.0.smithi037.stdout:2/125: symlink d0/l28 0 2024-03-29T08:07:09.784 INFO:tasks.workunit.client.1.smithi148.stdout:4/122: symlink df/d10/d26/l2b 0 2024-03-29T08:07:09.785 INFO:tasks.workunit.client.0.smithi037.stdout:7/233: link dd/l22 dd/d21/d35/d42/l4d 0 2024-03-29T08:07:09.788 INFO:tasks.workunit.client.1.smithi148.stdout:6/151: dread d6/d9/d1e/d20/f24 [0,4194304] 0 2024-03-29T08:07:09.788 INFO:tasks.workunit.client.0.smithi037.stdout:4/96: rename d3/c10 to d3/d11/c20 0 2024-03-29T08:07:09.790 INFO:tasks.workunit.client.1.smithi148.stdout:6/152: fdatasync d6/d9/f23 0 2024-03-29T08:07:09.790 INFO:tasks.workunit.client.1.smithi148.stdout:1/132: symlink d3/de/d1d/l27 0 2024-03-29T08:07:09.793 INFO:tasks.workunit.client.1.smithi148.stdout:5/128: mkdir d0/d26 0 2024-03-29T08:07:09.801 INFO:tasks.workunit.client.1.smithi148.stdout:2/159: rename d7/da/d1c to d7/da/d33/d3c 0 2024-03-29T08:07:09.804 INFO:tasks.workunit.client.0.smithi037.stdout:1/221: dread d2/d6/dc/f35 [0,4194304] 0 2024-03-29T08:07:09.805 INFO:tasks.workunit.client.1.smithi148.stdout:2/160: write d7/da/d33/d3c/d21/f25 [375750,15630] 0 2024-03-29T08:07:09.805 INFO:tasks.workunit.client.0.smithi037.stdout:9/292: dread d1/d4/f28 [0,4194304] 0 2024-03-29T08:07:09.807 INFO:tasks.workunit.client.1.smithi148.stdout:4/123: write df/f1e [4356700,127221] 0 2024-03-29T08:07:09.809 INFO:tasks.workunit.client.1.smithi148.stdout:5/129: dwrite d0/f12 [0,4194304] 0 2024-03-29T08:07:09.813 INFO:tasks.workunit.client.1.smithi148.stdout:4/124: write df/f27 [695233,99810] 0 2024-03-29T08:07:09.816 INFO:tasks.workunit.client.0.smithi037.stdout:1/222: chown d2/d6/dc/d18/d1d/d21/d38/c49 2407 1 2024-03-29T08:07:09.818 INFO:tasks.workunit.client.0.smithi037.stdout:0/106: dread f7 [0,4194304] 0 2024-03-29T08:07:09.819 INFO:tasks.workunit.client.0.smithi037.stdout:1/223: readlink d2/le 0 2024-03-29T08:07:09.820 INFO:tasks.workunit.client.1.smithi148.stdout:8/181: rmdir d17/d18/d26/d3b 0 2024-03-29T08:07:09.820 INFO:tasks.workunit.client.1.smithi148.stdout:2/161: rename d7/da/f1f to d7/d13/d2e/d37/f3d 0 2024-03-29T08:07:09.821 INFO:tasks.workunit.client.1.smithi148.stdout:2/162: readlink d7/da/l1d 0 2024-03-29T08:07:09.821 INFO:tasks.workunit.client.1.smithi148.stdout:3/156: sync 2024-03-29T08:07:09.821 INFO:tasks.workunit.client.1.smithi148.stdout:2/163: stat d7/cd 0 2024-03-29T08:07:09.822 INFO:tasks.workunit.client.1.smithi148.stdout:3/157: write de/d13/f21 [1938924,97912] 0 2024-03-29T08:07:09.824 INFO:tasks.workunit.client.0.smithi037.stdout:5/207: creat d8/d18/f40 x:0 0 0 2024-03-29T08:07:09.824 INFO:tasks.workunit.client.1.smithi148.stdout:3/158: write de/f15 [4955003,26240] 0 2024-03-29T08:07:09.827 INFO:tasks.workunit.client.0.smithi037.stdout:4/97: rename d3/d4/d15/d1d/f13 to d3/f21 0 2024-03-29T08:07:09.828 INFO:tasks.workunit.client.1.smithi148.stdout:4/125: mkdir df/d2c 0 2024-03-29T08:07:09.831 INFO:tasks.workunit.client.0.smithi037.stdout:0/107: rmdir d9/da/dd/d1c 39 2024-03-29T08:07:09.833 INFO:tasks.workunit.client.0.smithi037.stdout:6/128: dwrite d0/d5/f16 [0,4194304] 0 2024-03-29T08:07:09.833 INFO:tasks.workunit.client.0.smithi037.stdout:1/224: creat d2/d6/dc/d18/d1d/d21/d38/f4e x:0 0 0 2024-03-29T08:07:09.834 INFO:tasks.workunit.client.1.smithi148.stdout:8/182: mkdir d17/d18/d26/d3d 0 2024-03-29T08:07:09.836 INFO:tasks.workunit.client.1.smithi148.stdout:2/164: write d7/d13/d2e/d37/f3d [5906466,90113] 0 2024-03-29T08:07:09.837 INFO:tasks.workunit.client.0.smithi037.stdout:3/205: truncate dd/f23 292431 0 2024-03-29T08:07:09.839 INFO:tasks.workunit.client.0.smithi037.stdout:9/293: rename d1/d4/d19/d1e/d25/d10 to d1/d4/d19/d1e/d4a/d5c/d60 0 2024-03-29T08:07:09.842 INFO:tasks.workunit.client.1.smithi148.stdout:3/159: unlink de/f11 0 2024-03-29T08:07:09.842 INFO:tasks.workunit.client.0.smithi037.stdout:4/98: creat d3/f22 x:0 0 0 2024-03-29T08:07:09.843 INFO:tasks.workunit.client.1.smithi148.stdout:1/133: getdents d3/de 0 2024-03-29T08:07:09.846 INFO:tasks.workunit.client.1.smithi148.stdout:8/183: rename d17/f3c to d17/d18/f3e 0 2024-03-29T08:07:09.851 INFO:tasks.workunit.client.1.smithi148.stdout:5/130: rmdir d0/db/d1e/d23 0 2024-03-29T08:07:09.851 INFO:tasks.workunit.client.1.smithi148.stdout:2/165: creat d7/da/d33/f3e x:0 0 0 2024-03-29T08:07:09.851 INFO:tasks.workunit.client.0.smithi037.stdout:1/225: mkdir d2/d6/dc/d18/d22/d4f 0 2024-03-29T08:07:09.852 INFO:tasks.workunit.client.1.smithi148.stdout:1/134: creat d3/d13/f28 x:0 0 0 2024-03-29T08:07:09.853 INFO:tasks.workunit.client.1.smithi148.stdout:5/131: dread - d0/db/de/d16/f1b zero size 2024-03-29T08:07:09.854 INFO:tasks.workunit.client.1.smithi148.stdout:1/135: write d3/de/f1f [135748,108911] 0 2024-03-29T08:07:09.854 INFO:tasks.workunit.client.1.smithi148.stdout:3/160: link de/d24/f2b de/d13/f32 0 2024-03-29T08:07:09.857 INFO:tasks.workunit.client.1.smithi148.stdout:1/136: read - d3/d13/f28 zero size 2024-03-29T08:07:09.857 INFO:tasks.workunit.client.0.smithi037.stdout:9/294: symlink d1/d3/l61 0 2024-03-29T08:07:09.859 INFO:tasks.workunit.client.1.smithi148.stdout:3/161: mkdir de/d24/d33 0 2024-03-29T08:07:09.860 INFO:tasks.workunit.client.0.smithi037.stdout:6/129: dread d0/d5/f16 [0,4194304] 0 2024-03-29T08:07:09.860 INFO:tasks.workunit.client.0.smithi037.stdout:1/226: write d2/d6/dc/d18/d1d/d21/f20 [4156321,75354] 0 2024-03-29T08:07:09.862 INFO:tasks.workunit.client.1.smithi148.stdout:2/166: creat d7/da/f3f x:0 0 0 2024-03-29T08:07:09.863 INFO:tasks.workunit.client.1.smithi148.stdout:2/167: dread - d7/d24/f36 zero size 2024-03-29T08:07:09.864 INFO:tasks.workunit.client.0.smithi037.stdout:9/295: stat d1/d4/d19/d1e/d4a/d5c/d60/d23/f39 0 2024-03-29T08:07:09.864 INFO:tasks.workunit.client.0.smithi037.stdout:4/99: mkdir d3/d4/d23 0 2024-03-29T08:07:09.866 INFO:tasks.workunit.client.0.smithi037.stdout:0/108: mkdir d9/da/d1b/d22 0 2024-03-29T08:07:09.866 INFO:tasks.workunit.client.0.smithi037.stdout:2/126: rename d0/c9 to d0/d20/d22/c29 0 2024-03-29T08:07:09.867 INFO:tasks.workunit.client.1.smithi148.stdout:1/137: mknod d3/c29 0 2024-03-29T08:07:09.869 INFO:tasks.workunit.client.0.smithi037.stdout:4/100: read - d3/f1b zero size 2024-03-29T08:07:09.873 INFO:tasks.workunit.client.1.smithi148.stdout:5/132: link d0/f21 d0/db/de/d16/f27 0 2024-03-29T08:07:09.874 INFO:tasks.workunit.client.0.smithi037.stdout:9/296: dread - d1/d4/d19/d1e/d4a/f53 zero size 2024-03-29T08:07:09.874 INFO:tasks.workunit.client.0.smithi037.stdout:6/130: rename d0/d19/l1d to d0/d5/d1f/l2c 0 2024-03-29T08:07:09.875 INFO:tasks.workunit.client.0.smithi037.stdout:3/206: dwrite f4 [0,4194304] 0 2024-03-29T08:07:09.875 INFO:tasks.workunit.client.0.smithi037.stdout:1/227: getdents d2/d6 0 2024-03-29T08:07:09.876 INFO:tasks.workunit.client.0.smithi037.stdout:3/207: readlink l7 0 2024-03-29T08:07:09.878 INFO:tasks.workunit.client.0.smithi037.stdout:0/109: unlink d9/da/dd/c16 0 2024-03-29T08:07:09.879 INFO:tasks.workunit.client.1.smithi148.stdout:3/162: getdents de/d24/d33 0 2024-03-29T08:07:09.881 INFO:tasks.workunit.client.0.smithi037.stdout:1/228: fsync d2/d6/dc/d18/d22/f48 0 2024-03-29T08:07:09.881 INFO:tasks.workunit.client.0.smithi037.stdout:6/131: symlink d0/d19/l2d 0 2024-03-29T08:07:09.884 INFO:tasks.workunit.client.0.smithi037.stdout:5/208: dread d8/d18/f19 [0,4194304] 0 2024-03-29T08:07:09.886 INFO:tasks.workunit.client.0.smithi037.stdout:3/208: read - dd/d46/d2d/d3d/f45 zero size 2024-03-29T08:07:09.887 INFO:tasks.workunit.client.1.smithi148.stdout:5/133: dwrite d0/db/f10 [0,4194304] 0 2024-03-29T08:07:09.890 INFO:tasks.workunit.client.1.smithi148.stdout:5/134: stat d0/db/de/d16/f27 0 2024-03-29T08:07:09.890 INFO:tasks.workunit.client.1.smithi148.stdout:2/168: rename d7/f9 to d7/da/f40 0 2024-03-29T08:07:09.891 INFO:tasks.workunit.client.1.smithi148.stdout:5/135: readlink d0/l24 0 2024-03-29T08:07:09.891 INFO:tasks.workunit.client.1.smithi148.stdout:2/169: dread - d7/da/f3f zero size 2024-03-29T08:07:09.892 INFO:tasks.workunit.client.1.smithi148.stdout:4/126: write df/d10/d17/f20 [917522,126437] 0 2024-03-29T08:07:09.893 INFO:tasks.workunit.client.1.smithi148.stdout:2/170: fsync d7/d13/f29 0 2024-03-29T08:07:09.894 INFO:tasks.workunit.client.0.smithi037.stdout:4/101: symlink d3/d11/d1e/l24 0 2024-03-29T08:07:09.897 INFO:tasks.workunit.client.1.smithi148.stdout:5/136: symlink d0/db/d1e/l28 0 2024-03-29T08:07:09.899 INFO:tasks.workunit.client.1.smithi148.stdout:5/137: read d0/f21 [1158473,67459] 0 2024-03-29T08:07:09.900 INFO:tasks.workunit.client.1.smithi148.stdout:5/138: rename d0/db to d0/db/d1e/d29 22 2024-03-29T08:07:09.900 INFO:tasks.workunit.client.1.smithi148.stdout:4/127: mkdir df/d2d 0 2024-03-29T08:07:09.901 INFO:tasks.workunit.client.1.smithi148.stdout:4/128: chown fa 3254 1 2024-03-29T08:07:09.902 INFO:tasks.workunit.client.0.smithi037.stdout:6/132: chown d0/d5/fa 24103 1 2024-03-29T08:07:09.903 INFO:tasks.workunit.client.1.smithi148.stdout:2/171: mkdir d7/d13/d41 0 2024-03-29T08:07:09.904 INFO:tasks.workunit.client.1.smithi148.stdout:2/172: dread - d7/d24/f34 zero size 2024-03-29T08:07:09.907 INFO:tasks.workunit.client.1.smithi148.stdout:2/173: readlink d7/da/l3a 0 2024-03-29T08:07:09.908 INFO:tasks.workunit.client.0.smithi037.stdout:1/229: creat d2/d6/dc/d18/d1d/d21/d38/f50 x:0 0 0 2024-03-29T08:07:09.909 INFO:tasks.workunit.client.0.smithi037.stdout:5/209: symlink d8/d18/d24/l41 0 2024-03-29T08:07:09.909 INFO:tasks.workunit.client.1.smithi148.stdout:2/174: write d7/d13/f2a [1400891,84424] 0 2024-03-29T08:07:09.911 INFO:tasks.workunit.client.1.smithi148.stdout:8/184: dwrite f7 [0,4194304] 0 2024-03-29T08:07:09.913 INFO:tasks.workunit.client.1.smithi148.stdout:8/185: chown d17/d18/d26/f27 0 1 2024-03-29T08:07:09.916 INFO:tasks.workunit.client.1.smithi148.stdout:5/139: creat d0/db/de/f2a x:0 0 0 2024-03-29T08:07:09.917 INFO:tasks.workunit.client.1.smithi148.stdout:8/186: write f7 [3140174,90851] 0 2024-03-29T08:07:09.918 INFO:tasks.workunit.client.1.smithi148.stdout:9/136: sync 2024-03-29T08:07:09.918 INFO:tasks.workunit.client.1.smithi148.stdout:0/168: sync 2024-03-29T08:07:09.919 INFO:tasks.workunit.client.0.smithi037.stdout:9/297: link d1/d4/d19/d1e/d4a/d5c/d60/d31/f48 d1/d4/d19/d1e/d4a/f62 0 2024-03-29T08:07:09.921 INFO:tasks.workunit.client.1.smithi148.stdout:1/138: write d3/f7 [812408,76832] 0 2024-03-29T08:07:09.922 INFO:tasks.workunit.client.0.smithi037.stdout:6/133: fsync d0/d1/f17 0 2024-03-29T08:07:09.928 INFO:tasks.workunit.client.1.smithi148.stdout:5/140: unlink d0/db/de/f2a 0 2024-03-29T08:07:09.930 INFO:tasks.workunit.client.1.smithi148.stdout:8/187: creat d17/d18/f3f x:0 0 0 2024-03-29T08:07:09.932 INFO:tasks.workunit.client.0.smithi037.stdout:9/298: mknod d1/d1f/d3a/c63 0 2024-03-29T08:07:09.933 INFO:tasks.workunit.client.1.smithi148.stdout:3/163: write de/f2e [251676,120160] 0 2024-03-29T08:07:09.934 INFO:tasks.workunit.client.1.smithi148.stdout:9/137: dread db/d10/f13 [0,4194304] 0 2024-03-29T08:07:09.934 INFO:tasks.workunit.client.0.smithi037.stdout:5/210: dread d8/d22/f2d [0,4194304] 0 2024-03-29T08:07:09.937 INFO:tasks.workunit.client.0.smithi037.stdout:5/211: stat d8 0 2024-03-29T08:07:09.938 INFO:tasks.workunit.client.0.smithi037.stdout:0/110: link f5 d9/da/dd/f23 0 2024-03-29T08:07:09.940 INFO:tasks.workunit.client.0.smithi037.stdout:1/230: dwrite d2/d6/dc/d18/d22/f48 [0,4194304] 0 2024-03-29T08:07:09.941 INFO:tasks.workunit.client.1.smithi148.stdout:5/141: dread d0/f9 [0,4194304] 0 2024-03-29T08:07:09.941 INFO:tasks.workunit.client.0.smithi037.stdout:5/212: truncate d8/f17 2640440 0 2024-03-29T08:07:09.942 INFO:tasks.workunit.client.1.smithi148.stdout:1/139: symlink d3/de/d1d/l2a 0 2024-03-29T08:07:09.943 INFO:tasks.workunit.client.0.smithi037.stdout:9/299: fsync d1/d1f/d3a/f27 0 2024-03-29T08:07:09.945 INFO:tasks.workunit.client.1.smithi148.stdout:8/188: dread f7 [0,4194304] 0 2024-03-29T08:07:09.948 INFO:tasks.workunit.client.0.smithi037.stdout:5/213: fdatasync d8/d18/d24/f2a 0 2024-03-29T08:07:09.948 INFO:tasks.workunit.client.1.smithi148.stdout:4/129: dread df/d10/f12 [0,4194304] 0 2024-03-29T08:07:09.949 INFO:tasks.workunit.client.1.smithi148.stdout:0/169: fdatasync db/de/df/d11/f1b 0 2024-03-29T08:07:09.952 INFO:tasks.workunit.client.0.smithi037.stdout:1/231: creat d2/d6/dc/f51 x:0 0 0 2024-03-29T08:07:09.952 INFO:tasks.workunit.client.0.smithi037.stdout:5/214: fsync d8/f2b 0 2024-03-29T08:07:09.953 INFO:tasks.workunit.client.1.smithi148.stdout:1/140: readlink d3/l17 0 2024-03-29T08:07:09.954 INFO:tasks.workunit.client.0.smithi037.stdout:2/127: dwrite d0/f1f [0,4194304] 0 2024-03-29T08:07:09.958 INFO:tasks.workunit.client.0.smithi037.stdout:6/134: rmdir d0/d20/d29 0 2024-03-29T08:07:09.958 INFO:tasks.workunit.client.1.smithi148.stdout:1/141: chown d3/d18/f20 2004149 1 2024-03-29T08:07:09.960 INFO:tasks.workunit.client.1.smithi148.stdout:3/164: dwrite de/f26 [0,4194304] 0 2024-03-29T08:07:09.962 INFO:tasks.workunit.client.1.smithi148.stdout:9/138: symlink db/d10/d14/d2b/l2c 0 2024-03-29T08:07:09.964 INFO:tasks.workunit.client.1.smithi148.stdout:0/170: chown db/c2f 305 1 2024-03-29T08:07:09.964 INFO:tasks.workunit.client.0.smithi037.stdout:9/300: unlink d1/d4/d19/c2e 0 2024-03-29T08:07:09.965 INFO:tasks.workunit.client.1.smithi148.stdout:8/189: symlink d17/d18/d38/d39/l40 0 2024-03-29T08:07:09.966 INFO:tasks.workunit.client.0.smithi037.stdout:4/102: truncate d3/d4/d15/d1d/db/f1a 2070860 0 2024-03-29T08:07:09.968 INFO:tasks.workunit.client.1.smithi148.stdout:8/190: dread - d17/d18/f3a zero size 2024-03-29T08:07:09.968 INFO:tasks.workunit.client.1.smithi148.stdout:4/130: creat df/d2c/f2e x:0 0 0 2024-03-29T08:07:09.968 INFO:tasks.workunit.client.0.smithi037.stdout:1/232: mknod d2/d40/c52 0 2024-03-29T08:07:09.968 INFO:tasks.workunit.client.0.smithi037.stdout:9/301: write d1/d4/d19/d1e/d4a/d5c/d60/d31/f4d [108198,71328] 0 2024-03-29T08:07:09.972 INFO:tasks.workunit.client.1.smithi148.stdout:9/139: rmdir db/d22 39 2024-03-29T08:07:09.972 INFO:tasks.workunit.client.0.smithi037.stdout:6/135: symlink d0/d5/d1e/l2e 0 2024-03-29T08:07:09.973 INFO:tasks.workunit.client.1.smithi148.stdout:5/142: write d0/f21 [2125779,107525] 0 2024-03-29T08:07:09.974 INFO:tasks.workunit.client.0.smithi037.stdout:2/128: unlink d0/c19 0 2024-03-29T08:07:09.974 INFO:tasks.workunit.client.0.smithi037.stdout:9/302: stat d1/d4/d19/d1e/c59 0 2024-03-29T08:07:09.977 INFO:tasks.workunit.client.0.smithi037.stdout:1/233: symlink d2/d4b/l53 0 2024-03-29T08:07:09.979 INFO:tasks.workunit.client.1.smithi148.stdout:1/142: dread d3/d13/f15 [12582912,4194304] 0 2024-03-29T08:07:09.979 INFO:tasks.workunit.client.0.smithi037.stdout:1/234: stat d2/d6/dc/d18/d1d 0 2024-03-29T08:07:09.980 INFO:tasks.workunit.client.1.smithi148.stdout:9/140: mkdir db/d10/d2d 0 2024-03-29T08:07:09.980 INFO:tasks.workunit.client.1.smithi148.stdout:5/143: symlink d0/l2b 0 2024-03-29T08:07:09.983 INFO:tasks.workunit.client.0.smithi037.stdout:4/103: rename d3/f22 to d3/d4/d15/d1d/db/f25 0 2024-03-29T08:07:09.984 INFO:tasks.workunit.client.1.smithi148.stdout:1/143: chown d3/l17 3444 1 2024-03-29T08:07:09.987 INFO:tasks.workunit.client.1.smithi148.stdout:5/144: unlink d0/db/c11 0 2024-03-29T08:07:09.987 INFO:tasks.workunit.client.1.smithi148.stdout:0/171: creat db/de/df/d14/f3c x:0 0 0 2024-03-29T08:07:09.993 INFO:tasks.workunit.client.0.smithi037.stdout:2/129: stat d0/f18 0 2024-03-29T08:07:09.993 INFO:tasks.workunit.client.1.smithi148.stdout:9/141: creat db/d10/d14/d2b/f2e x:0 0 0 2024-03-29T08:07:09.996 INFO:tasks.workunit.client.1.smithi148.stdout:8/191: dwrite d17/d18/d26/f35 [0,4194304] 0 2024-03-29T08:07:09.997 INFO:tasks.workunit.client.1.smithi148.stdout:4/131: dwrite df/d10/f1d [0,4194304] 0 2024-03-29T08:07:09.998 INFO:tasks.workunit.client.0.smithi037.stdout:9/303: creat d1/d4/d19/f64 x:0 0 0 2024-03-29T08:07:10.000 INFO:tasks.workunit.client.0.smithi037.stdout:1/235: fsync d2/d6/dc/f25 0 2024-03-29T08:07:10.000 INFO:tasks.workunit.client.1.smithi148.stdout:7/182: sync 2024-03-29T08:07:10.000 INFO:tasks.workunit.client.1.smithi148.stdout:6/153: sync 2024-03-29T08:07:10.000 INFO:tasks.workunit.client.1.smithi148.stdout:8/192: stat ca 0 2024-03-29T08:07:10.002 INFO:tasks.workunit.client.1.smithi148.stdout:3/165: truncate de/f17 56412 0 2024-03-29T08:07:10.002 INFO:tasks.workunit.client.0.smithi037.stdout:5/215: creat d8/f42 x:0 0 0 2024-03-29T08:07:10.003 INFO:tasks.workunit.client.1.smithi148.stdout:5/145: link d0/f9 d0/f2c 0 2024-03-29T08:07:10.008 INFO:tasks.workunit.client.1.smithi148.stdout:6/154: stat d6/d9/f1f 0 2024-03-29T08:07:10.009 INFO:tasks.workunit.client.1.smithi148.stdout:8/193: write d17/d18/d26/f27 [88454,73572] 0 2024-03-29T08:07:10.009 INFO:tasks.workunit.client.0.smithi037.stdout:6/136: dread d0/d1/ff [0,4194304] 0 2024-03-29T08:07:10.010 INFO:tasks.workunit.client.1.smithi148.stdout:3/166: symlink de/d13/l34 0 2024-03-29T08:07:10.013 INFO:tasks.workunit.client.1.smithi148.stdout:8/194: fsync d17/d18/f36 0 2024-03-29T08:07:10.013 INFO:tasks.workunit.client.1.smithi148.stdout:3/167: stat de/d13/l34 0 2024-03-29T08:07:10.014 INFO:tasks.workunit.client.0.smithi037.stdout:0/111: truncate d9/da/dd/d1c/f18 1363306 0 2024-03-29T08:07:10.015 INFO:tasks.workunit.client.0.smithi037.stdout:5/216: unlink d8/d22/f25 0 2024-03-29T08:07:10.015 INFO:tasks.workunit.client.1.smithi148.stdout:1/144: dread d3/d13/f15 [12582912,4194304] 0 2024-03-29T08:07:10.017 INFO:tasks.workunit.client.0.smithi037.stdout:5/217: chown d8/d18/d24 456938 1 2024-03-29T08:07:10.017 INFO:tasks.workunit.client.1.smithi148.stdout:8/195: chown d17/d18/d38/d39/l40 3288456 1 2024-03-29T08:07:10.017 INFO:tasks.workunit.client.1.smithi148.stdout:1/145: stat d3/de/c1c 0 2024-03-29T08:07:10.019 INFO:tasks.workunit.client.1.smithi148.stdout:8/196: read d17/f23 [743622,122038] 0 2024-03-29T08:07:10.020 INFO:tasks.workunit.client.0.smithi037.stdout:1/236: mknod d2/c54 0 2024-03-29T08:07:10.022 INFO:tasks.workunit.client.1.smithi148.stdout:8/197: write f14 [3231677,14411] 0 2024-03-29T08:07:10.022 INFO:tasks.workunit.client.1.smithi148.stdout:7/183: dwrite d8/f1f [0,4194304] 0 2024-03-29T08:07:10.024 INFO:tasks.workunit.client.0.smithi037.stdout:6/137: rename d0/d19/f27 to d0/d5/d1a/f2f 0 2024-03-29T08:07:10.024 INFO:tasks.workunit.client.1.smithi148.stdout:8/198: readlink d17/d18/l24 0 2024-03-29T08:07:10.030 INFO:tasks.workunit.client.1.smithi148.stdout:4/132: creat df/d2d/f2f x:0 0 0 2024-03-29T08:07:10.030 INFO:tasks.workunit.client.1.smithi148.stdout:5/146: mknod d0/db/de/c2d 0 2024-03-29T08:07:10.033 INFO:tasks.workunit.client.1.smithi148.stdout:0/172: dwrite f6 [0,4194304] 0 2024-03-29T08:07:10.033 INFO:tasks.workunit.client.1.smithi148.stdout:8/199: write d17/d18/d26/f27 [2668852,75898] 0 2024-03-29T08:07:10.034 INFO:tasks.workunit.client.0.smithi037.stdout:4/104: unlink d3/d4/d15/d1d/db/f25 0 2024-03-29T08:07:10.035 INFO:tasks.workunit.client.1.smithi148.stdout:2/175: sync 2024-03-29T08:07:10.035 INFO:tasks.workunit.client.0.smithi037.stdout:1/237: rmdir d2/d6/dc/d18/d1d/d21 39 2024-03-29T08:07:10.036 INFO:tasks.workunit.client.1.smithi148.stdout:5/147: chown d0/f2c 35611 1 2024-03-29T08:07:10.037 INFO:tasks.workunit.client.0.smithi037.stdout:6/138: creat d0/d5/d1f/f30 x:0 0 0 2024-03-29T08:07:10.041 INFO:tasks.workunit.client.1.smithi148.stdout:5/148: chown d0/db/d1e/l28 439 1 2024-03-29T08:07:10.041 INFO:tasks.workunit.client.1.smithi148.stdout:0/173: read - db/de/df/d11/f33 zero size 2024-03-29T08:07:10.042 INFO:tasks.workunit.client.1.smithi148.stdout:3/168: creat de/f35 x:0 0 0 2024-03-29T08:07:10.045 INFO:tasks.workunit.client.0.smithi037.stdout:9/304: getdents d1/d4/d19/d1e/d4a/d5c/d60/d23 0 2024-03-29T08:07:10.045 INFO:tasks.workunit.client.0.smithi037.stdout:5/218: rmdir d8 39 2024-03-29T08:07:10.047 INFO:tasks.workunit.client.1.smithi148.stdout:1/146: dwrite d3/de/f1f [4194304,4194304] 0 2024-03-29T08:07:10.048 INFO:tasks.workunit.client.1.smithi148.stdout:9/142: getdents db/d22/d26 0 2024-03-29T08:07:10.050 INFO:tasks.workunit.client.1.smithi148.stdout:9/143: dread - db/d10/f29 zero size 2024-03-29T08:07:10.051 INFO:tasks.workunit.client.1.smithi148.stdout:6/155: mkdir d6/d9/d1e/d22/d28 0 2024-03-29T08:07:10.051 INFO:tasks.workunit.client.1.smithi148.stdout:1/147: readlink d3/de/d1d/l24 0 2024-03-29T08:07:10.052 INFO:tasks.workunit.client.1.smithi148.stdout:4/133: mkdir df/d2d/d30 0 2024-03-29T08:07:10.053 INFO:tasks.workunit.client.0.smithi037.stdout:2/130: dwrite d0/f15 [0,4194304] 0 2024-03-29T08:07:10.054 INFO:tasks.workunit.client.0.smithi037.stdout:1/238: dread - d2/d6/dc/d18/d1d/d21/d38/f4e zero size 2024-03-29T08:07:10.057 INFO:tasks.workunit.client.1.smithi148.stdout:6/156: chown d6/f14 1443324515 1 2024-03-29T08:07:10.057 INFO:tasks.workunit.client.1.smithi148.stdout:4/134: stat df/d10/d17/l24 0 2024-03-29T08:07:10.059 INFO:tasks.workunit.client.1.smithi148.stdout:2/176: creat d7/da/d33/d3c/f42 x:0 0 0 2024-03-29T08:07:10.062 INFO:tasks.workunit.client.1.smithi148.stdout:6/157: chown d6/f19 3911073 1 2024-03-29T08:07:10.062 INFO:tasks.workunit.client.1.smithi148.stdout:8/200: dwrite d17/d18/d26/f32 [0,4194304] 0 2024-03-29T08:07:10.063 INFO:tasks.workunit.client.1.smithi148.stdout:7/184: fdatasync d8/d29/f2e 0 2024-03-29T08:07:10.064 INFO:tasks.workunit.client.1.smithi148.stdout:6/158: truncate d6/f1a 651730 0 2024-03-29T08:07:10.066 INFO:tasks.workunit.client.0.smithi037.stdout:5/219: chown d8/l1c 107476223 1 2024-03-29T08:07:10.067 INFO:tasks.workunit.client.1.smithi148.stdout:8/201: chown f7 1453 1 2024-03-29T08:07:10.069 INFO:tasks.workunit.client.1.smithi148.stdout:8/202: write d17/f2d [1274032,111890] 0 2024-03-29T08:07:10.070 INFO:tasks.workunit.client.0.smithi037.stdout:0/112: dwrite d9/da/dd/d13/f21 [0,4194304] 0 2024-03-29T08:07:10.070 INFO:tasks.workunit.client.0.smithi037.stdout:6/139: dread d0/d5/f16 [0,4194304] 0 2024-03-29T08:07:10.073 INFO:tasks.workunit.client.0.smithi037.stdout:6/140: fsync d0/f4 0 2024-03-29T08:07:10.076 INFO:tasks.workunit.client.1.smithi148.stdout:5/149: dwrite d0/f21 [0,4194304] 0 2024-03-29T08:07:10.077 INFO:tasks.workunit.client.1.smithi148.stdout:4/135: creat df/d10/f31 x:0 0 0 2024-03-29T08:07:10.077 INFO:tasks.workunit.client.1.smithi148.stdout:2/177: creat d7/d13/d2e/d37/f43 x:0 0 0 2024-03-29T08:07:10.077 INFO:tasks.workunit.client.0.smithi037.stdout:2/131: mknod d0/d20/c2a 0 2024-03-29T08:07:10.079 INFO:tasks.workunit.client.1.smithi148.stdout:2/178: fsync f1 0 2024-03-29T08:07:10.079 INFO:tasks.workunit.client.0.smithi037.stdout:6/141: creat d0/d20/f31 x:0 0 0 2024-03-29T08:07:10.082 INFO:tasks.workunit.client.0.smithi037.stdout:0/113: creat d9/da/f24 x:0 0 0 2024-03-29T08:07:10.084 INFO:tasks.workunit.client.1.smithi148.stdout:6/159: dread d6/f19 [0,4194304] 0 2024-03-29T08:07:10.086 INFO:tasks.workunit.client.0.smithi037.stdout:6/142: mkdir d0/d5/d1f/d32 0 2024-03-29T08:07:10.090 INFO:tasks.workunit.client.0.smithi037.stdout:5/220: dwrite d8/f17 [0,4194304] 0 2024-03-29T08:07:10.092 INFO:tasks.workunit.client.0.smithi037.stdout:9/305: dwrite d1/d4/f28 [0,4194304] 0 2024-03-29T08:07:10.098 INFO:tasks.workunit.client.1.smithi148.stdout:0/174: dwrite db/f18 [0,4194304] 0 2024-03-29T08:07:10.099 INFO:tasks.workunit.client.0.smithi037.stdout:5/221: unlink d8/d18/d24/c29 0 2024-03-29T08:07:10.101 INFO:tasks.workunit.client.0.smithi037.stdout:6/143: creat d0/f33 x:0 0 0 2024-03-29T08:07:10.101 INFO:tasks.workunit.client.1.smithi148.stdout:8/203: mknod d17/c41 0 2024-03-29T08:07:10.103 INFO:tasks.workunit.client.0.smithi037.stdout:5/222: unlink d8/l38 0 2024-03-29T08:07:10.104 INFO:tasks.workunit.client.1.smithi148.stdout:0/175: read db/f23 [2177488,43444] 0 2024-03-29T08:07:10.108 INFO:tasks.workunit.client.0.smithi037.stdout:5/223: chown d8/le 80958482 1 2024-03-29T08:07:10.110 INFO:tasks.workunit.client.0.smithi037.stdout:6/144: mkdir d0/d19/d34 0 2024-03-29T08:07:10.111 INFO:tasks.workunit.client.0.smithi037.stdout:1/239: write d2/f14 [3645854,48030] 0 2024-03-29T08:07:10.115 INFO:tasks.workunit.client.0.smithi037.stdout:0/114: dread d9/da/dd/d13/f21 [0,4194304] 0 2024-03-29T08:07:10.115 INFO:tasks.workunit.client.0.smithi037.stdout:9/306: rename d1/d4/d19/d1e/d4a/d5c/d60/d31/f4d to d1/d4/d19/d1e/d4a/d5c/d60/d23/f65 0 2024-03-29T08:07:10.121 INFO:tasks.workunit.client.0.smithi037.stdout:6/145: creat d0/d5/d1f/f35 x:0 0 0 2024-03-29T08:07:10.121 INFO:tasks.workunit.client.1.smithi148.stdout:2/179: dwrite d7/f19 [0,4194304] 0 2024-03-29T08:07:10.125 INFO:tasks.workunit.client.1.smithi148.stdout:4/136: dwrite df/d10/f12 [0,4194304] 0 2024-03-29T08:07:10.127 INFO:tasks.workunit.client.1.smithi148.stdout:5/150: rmdir d0/db/de/d16 39 2024-03-29T08:07:10.129 INFO:tasks.workunit.client.0.smithi037.stdout:5/224: creat d8/d18/f43 x:0 0 0 2024-03-29T08:07:10.130 INFO:tasks.workunit.client.0.smithi037.stdout:4/105: write d3/d4/d15/d1d/db/f1a [2542715,39538] 0 2024-03-29T08:07:10.130 INFO:tasks.workunit.client.1.smithi148.stdout:2/180: write f5 [3700916,53852] 0 2024-03-29T08:07:10.131 INFO:tasks.workunit.client.1.smithi148.stdout:4/137: dread - df/d2d/f2f zero size 2024-03-29T08:07:10.132 INFO:tasks.workunit.client.0.smithi037.stdout:1/240: creat d2/d6/dc/d18/d1d/d21/f55 x:0 0 0 2024-03-29T08:07:10.132 INFO:tasks.workunit.client.0.smithi037.stdout:6/146: mknod d0/d20/c36 0 2024-03-29T08:07:10.132 INFO:tasks.workunit.client.0.smithi037.stdout:5/225: chown d8/ld 696 1 2024-03-29T08:07:10.133 INFO:tasks.workunit.client.1.smithi148.stdout:7/185: dwrite d8/f13 [0,4194304] 0 2024-03-29T08:07:10.133 INFO:tasks.workunit.client.0.smithi037.stdout:5/226: chown d8 4864 1 2024-03-29T08:07:10.135 INFO:tasks.workunit.client.1.smithi148.stdout:6/160: stat c3 0 2024-03-29T08:07:10.135 INFO:tasks.workunit.client.0.smithi037.stdout:9/307: truncate d1/d4/d19/d1e/d4a/d5c/d60/d31/f48 4985226 0 2024-03-29T08:07:10.136 INFO:tasks.workunit.client.1.smithi148.stdout:9/144: creat db/f2f x:0 0 0 2024-03-29T08:07:10.138 INFO:tasks.workunit.client.1.smithi148.stdout:2/181: truncate d7/d13/f38 620148 0 2024-03-29T08:07:10.140 INFO:tasks.workunit.client.1.smithi148.stdout:0/176: dwrite fa [0,4194304] 0 2024-03-29T08:07:10.140 INFO:tasks.workunit.client.1.smithi148.stdout:2/182: truncate d7/d24/f34 946697 0 2024-03-29T08:07:10.143 INFO:tasks.workunit.client.1.smithi148.stdout:1/148: rmdir d3/de/d21/d22 0 2024-03-29T08:07:10.144 INFO:tasks.workunit.client.0.smithi037.stdout:4/106: truncate f0 5056208 0 2024-03-29T08:07:10.145 INFO:tasks.workunit.client.1.smithi148.stdout:3/169: creat de/d24/f36 x:0 0 0 2024-03-29T08:07:10.145 INFO:tasks.workunit.client.1.smithi148.stdout:1/149: stat d3/lf 0 2024-03-29T08:07:10.146 INFO:tasks.workunit.client.1.smithi148.stdout:1/150: dread - d3/d13/f28 zero size 2024-03-29T08:07:10.154 INFO:tasks.workunit.client.0.smithi037.stdout:5/227: rename d8/ff to d8/d18/d24/f44 0 2024-03-29T08:07:10.156 INFO:tasks.workunit.client.0.smithi037.stdout:5/228: stat d8/d18/f40 0 2024-03-29T08:07:10.158 INFO:tasks.workunit.client.0.smithi037.stdout:9/308: unlink d1/d4/d19/d1e/d4a/d5c/d60/d23/c45 0 2024-03-29T08:07:10.159 INFO:tasks.workunit.client.0.smithi037.stdout:4/107: symlink d3/d4/l26 0 2024-03-29T08:07:10.162 INFO:tasks.workunit.client.0.smithi037.stdout:5/229: write d8/f1e [1948698,22314] 0 2024-03-29T08:07:10.162 INFO:tasks.workunit.client.0.smithi037.stdout:4/108: stat d3/d4/d15/d1d 0 2024-03-29T08:07:10.163 INFO:tasks.workunit.client.1.smithi148.stdout:4/138: symlink df/d2d/l32 0 2024-03-29T08:07:10.170 INFO:tasks.workunit.client.1.smithi148.stdout:7/186: creat d8/d38/f43 x:0 0 0 2024-03-29T08:07:10.174 INFO:tasks.workunit.client.1.smithi148.stdout:7/187: truncate d8/d14/f2b 924634 0 2024-03-29T08:07:10.175 INFO:tasks.workunit.client.0.smithi037.stdout:6/147: dwrite d0/d5/d1a/f2f [0,4194304] 0 2024-03-29T08:07:10.177 INFO:tasks.workunit.client.0.smithi037.stdout:9/309: mknod d1/d4/d19/d1e/d4a/d5c/d60/c66 0 2024-03-29T08:07:10.190 INFO:tasks.workunit.client.0.smithi037.stdout:4/109: write d3/d4/d15/d1d/db/f16 [3578175,118496] 0 2024-03-29T08:07:10.195 INFO:tasks.workunit.client.0.smithi037.stdout:1/241: write d2/d6/dc/d18/d1d/d21/f2a [75440,49787] 0 2024-03-29T08:07:10.197 INFO:tasks.workunit.client.0.smithi037.stdout:4/110: stat d3/d4/fa 0 2024-03-29T08:07:10.202 INFO:tasks.workunit.client.1.smithi148.stdout:5/151: mkdir d0/db/de/d16/d2e 0 2024-03-29T08:07:10.206 INFO:tasks.workunit.client.1.smithi148.stdout:4/139: symlink df/d2d/l33 0 2024-03-29T08:07:10.207 INFO:tasks.workunit.client.1.smithi148.stdout:7/188: mkdir d8/d29/d44 0 2024-03-29T08:07:10.208 INFO:tasks.workunit.client.0.smithi037.stdout:4/111: chown d3/d4/l26 447 1 2024-03-29T08:07:10.209 INFO:tasks.workunit.client.1.smithi148.stdout:7/189: chown d8/f26 79946405 1 2024-03-29T08:07:10.210 INFO:tasks.workunit.client.1.smithi148.stdout:9/145: write db/d10/f13 [5237951,97673] 0 2024-03-29T08:07:10.211 INFO:tasks.workunit.client.0.smithi037.stdout:5/230: dwrite d8/f1f [0,4194304] 0 2024-03-29T08:07:10.211 INFO:tasks.workunit.client.1.smithi148.stdout:2/183: symlink d7/da/l44 0 2024-03-29T08:07:10.212 INFO:tasks.workunit.client.0.smithi037.stdout:9/310: dread d1/d4/d19/d1e/d4a/d5c/d60/d13/f17 [0,4194304] 0 2024-03-29T08:07:10.214 INFO:tasks.workunit.client.1.smithi148.stdout:3/170: mkdir de/d13/d30/d37 0 2024-03-29T08:07:10.215 INFO:tasks.workunit.client.1.smithi148.stdout:1/151: mknod d3/de/d25/c2b 0 2024-03-29T08:07:10.215 INFO:tasks.workunit.client.0.smithi037.stdout:0/115: dwrite f5 [0,4194304] 0 2024-03-29T08:07:10.216 INFO:tasks.workunit.client.1.smithi148.stdout:8/204: link d17/d18/d38/d39/l40 d17/d1d/l42 0 2024-03-29T08:07:10.218 INFO:tasks.workunit.client.0.smithi037.stdout:6/148: dwrite d0/d5/d1a/f2f [0,4194304] 0 2024-03-29T08:07:10.220 INFO:tasks.workunit.client.1.smithi148.stdout:7/190: mkdir d8/d14/d45 0 2024-03-29T08:07:10.220 INFO:tasks.workunit.client.1.smithi148.stdout:0/177: link db/de/df/c39 db/de/df/d11/c3d 0 2024-03-29T08:07:10.224 INFO:tasks.workunit.client.0.smithi037.stdout:9/311: mkdir d1/d1f/d3a/d67 0 2024-03-29T08:07:10.227 INFO:tasks.workunit.client.0.smithi037.stdout:0/116: stat d9/da/dd/d1c/f18 0 2024-03-29T08:07:10.228 INFO:tasks.workunit.client.1.smithi148.stdout:1/152: rmdir d3/de/d25 39 2024-03-29T08:07:10.231 INFO:tasks.workunit.client.0.smithi037.stdout:5/231: mkdir d8/d18/d24/d45 0 2024-03-29T08:07:10.232 INFO:tasks.workunit.client.0.smithi037.stdout:1/242: dwrite d2/d6/dc/f25 [0,4194304] 0 2024-03-29T08:07:10.232 INFO:tasks.workunit.client.1.smithi148.stdout:6/161: dwrite d6/fe [0,4194304] 0 2024-03-29T08:07:10.233 INFO:tasks.workunit.client.1.smithi148.stdout:5/152: dwrite d0/db/f10 [0,4194304] 0 2024-03-29T08:07:10.234 INFO:tasks.workunit.client.1.smithi148.stdout:0/178: mkdir db/de/df/d11/d2d/d3e 0 2024-03-29T08:07:10.237 INFO:tasks.workunit.client.1.smithi148.stdout:3/171: write de/d24/f31 [701360,57317] 0 2024-03-29T08:07:10.237 INFO:tasks.workunit.client.1.smithi148.stdout:4/140: dwrite f7 [0,4194304] 0 2024-03-29T08:07:10.239 INFO:tasks.workunit.client.0.smithi037.stdout:6/149: creat d0/f37 x:0 0 0 2024-03-29T08:07:10.241 INFO:tasks.workunit.client.1.smithi148.stdout:8/205: creat d17/d18/d26/d3d/f43 x:0 0 0 2024-03-29T08:07:10.242 INFO:tasks.workunit.client.1.smithi148.stdout:7/191: symlink d8/d38/d3c/l46 0 2024-03-29T08:07:10.244 INFO:tasks.workunit.client.0.smithi037.stdout:6/150: write d0/d5/fb [1002004,77027] 0 2024-03-29T08:07:10.246 INFO:tasks.workunit.client.1.smithi148.stdout:4/141: write df/d10/f31 [24777,46224] 0 2024-03-29T08:07:10.251 INFO:tasks.workunit.client.0.smithi037.stdout:6/151: chown d0/l15 3 1 2024-03-29T08:07:10.252 INFO:tasks.workunit.client.1.smithi148.stdout:8/206: dread - d17/f2c zero size 2024-03-29T08:07:10.253 INFO:tasks.workunit.client.1.smithi148.stdout:5/153: write d0/db/de/d16/f1b [826280,48663] 0 2024-03-29T08:07:10.255 INFO:tasks.workunit.client.0.smithi037.stdout:6/152: write d0/d20/f31 [421335,58960] 0 2024-03-29T08:07:10.259 INFO:tasks.workunit.client.1.smithi148.stdout:7/192: dread - d8/d14/f1b zero size 2024-03-29T08:07:10.262 INFO:tasks.workunit.client.1.smithi148.stdout:6/162: creat d6/d9/d1e/d22/f29 x:0 0 0 2024-03-29T08:07:10.263 INFO:tasks.workunit.client.1.smithi148.stdout:1/153: dread d3/d18/f20 [0,4194304] 0 2024-03-29T08:07:10.264 INFO:tasks.workunit.client.0.smithi037.stdout:5/232: dwrite d8/f17 [0,4194304] 0 2024-03-29T08:07:10.265 INFO:tasks.workunit.client.0.smithi037.stdout:6/153: rename d0/d5/d1f/d32 to d0/d20/d38 0 2024-03-29T08:07:10.265 INFO:tasks.workunit.client.1.smithi148.stdout:2/184: dwrite d7/da/d33/d3c/d21/f25 [0,4194304] 0 2024-03-29T08:07:10.268 INFO:tasks.workunit.client.1.smithi148.stdout:2/185: stat l4 0 2024-03-29T08:07:10.268 INFO:tasks.workunit.client.1.smithi148.stdout:7/193: chown d8/fe 18 1 2024-03-29T08:07:10.269 INFO:tasks.workunit.client.1.smithi148.stdout:0/179: dread db/de/df/d2b/f35 [0,4194304] 0 2024-03-29T08:07:10.274 INFO:tasks.workunit.client.1.smithi148.stdout:7/194: write d8/d29/f2e [2403740,104087] 0 2024-03-29T08:07:10.274 INFO:tasks.workunit.client.0.smithi037.stdout:1/243: truncate d2/d6/dc/f15 720611 0 2024-03-29T08:07:10.276 INFO:tasks.workunit.client.0.smithi037.stdout:5/233: dread d8/d22/f31 [0,4194304] 0 2024-03-29T08:07:10.276 INFO:tasks.workunit.client.1.smithi148.stdout:0/180: fdatasync db/de/df/d11/f1b 0 2024-03-29T08:07:10.277 INFO:tasks.workunit.client.1.smithi148.stdout:7/195: fsync d8/d14/f30 0 2024-03-29T08:07:10.279 INFO:tasks.workunit.client.1.smithi148.stdout:4/142: rmdir df/d2d/d30 0 2024-03-29T08:07:10.281 INFO:tasks.workunit.client.1.smithi148.stdout:6/163: mknod d6/d9/d1e/d27/c2a 0 2024-03-29T08:07:10.282 INFO:tasks.workunit.client.0.smithi037.stdout:1/244: truncate d2/d6/d13/f28 1175086 0 2024-03-29T08:07:10.285 INFO:tasks.workunit.client.0.smithi037.stdout:6/154: dread d0/d5/f9 [0,4194304] 0 2024-03-29T08:07:10.286 INFO:tasks.workunit.client.1.smithi148.stdout:7/196: creat d8/d14/f47 x:0 0 0 2024-03-29T08:07:10.286 INFO:tasks.workunit.client.0.smithi037.stdout:1/245: write d2/d6/dc/f51 [272288,18548] 0 2024-03-29T08:07:10.287 INFO:tasks.workunit.client.1.smithi148.stdout:4/143: mkdir df/d10/d26/d34 0 2024-03-29T08:07:10.290 INFO:tasks.workunit.client.1.smithi148.stdout:2/186: getdents d7 0 2024-03-29T08:07:10.293 INFO:tasks.workunit.client.0.smithi037.stdout:1/246: rename d2/d6/dc/f51 to d2/d6/dc/d18/d1d/d21/d38/f56 0 2024-03-29T08:07:10.294 INFO:tasks.workunit.client.1.smithi148.stdout:8/207: dwrite fe [0,4194304] 0 2024-03-29T08:07:10.296 INFO:tasks.workunit.client.1.smithi148.stdout:4/144: creat df/d2d/f35 x:0 0 0 2024-03-29T08:07:10.299 INFO:tasks.workunit.client.0.smithi037.stdout:1/247: creat d2/f57 x:0 0 0 2024-03-29T08:07:10.300 INFO:tasks.workunit.client.0.smithi037.stdout:1/248: readlink d2/l42 0 2024-03-29T08:07:10.301 INFO:tasks.workunit.client.1.smithi148.stdout:8/208: mknod d17/c44 0 2024-03-29T08:07:10.301 INFO:tasks.workunit.client.1.smithi148.stdout:9/146: dread f3 [0,4194304] 0 2024-03-29T08:07:10.302 INFO:tasks.workunit.client.1.smithi148.stdout:4/145: dread - df/d10/d17/f22 zero size 2024-03-29T08:07:10.303 INFO:tasks.workunit.client.0.smithi037.stdout:1/249: chown d2/d6/dc/d18/f3e 74837316 1 2024-03-29T08:07:10.303 INFO:tasks.workunit.client.1.smithi148.stdout:6/164: link d6/l1c d6/d9/d1e/d22/d28/l2b 0 2024-03-29T08:07:10.304 INFO:tasks.workunit.client.0.smithi037.stdout:5/234: dwrite d8/d34/d3b/f3f [0,4194304] 0 2024-03-29T08:07:10.306 INFO:tasks.workunit.client.1.smithi148.stdout:2/187: mkdir d7/da/d33/d45 0 2024-03-29T08:07:10.306 INFO:tasks.workunit.client.1.smithi148.stdout:8/209: write d17/f1a [1245290,37530] 0 2024-03-29T08:07:10.308 INFO:tasks.workunit.client.1.smithi148.stdout:0/181: getdents db/de 0 2024-03-29T08:07:10.312 INFO:tasks.workunit.client.0.smithi037.stdout:1/250: mkdir d2/d6/dc/d58 0 2024-03-29T08:07:10.312 INFO:tasks.workunit.client.1.smithi148.stdout:5/154: write d0/f2c [1303017,63038] 0 2024-03-29T08:07:10.313 INFO:tasks.workunit.client.0.smithi037.stdout:5/235: chown d8/d18/d24/c2e 8 1 2024-03-29T08:07:10.316 INFO:tasks.workunit.client.0.smithi037.stdout:1/251: creat d2/d6/dc/d58/f59 x:0 0 0 2024-03-29T08:07:10.317 INFO:tasks.workunit.client.0.smithi037.stdout:5/236: symlink d8/d22/l46 0 2024-03-29T08:07:10.320 INFO:tasks.workunit.client.0.smithi037.stdout:1/252: mknod d2/d6/c5a 0 2024-03-29T08:07:10.320 INFO:tasks.workunit.client.1.smithi148.stdout:1/154: dwrite d3/fa [12582912,4194304] 0 2024-03-29T08:07:10.321 INFO:tasks.workunit.client.1.smithi148.stdout:4/146: creat df/d10/d17/f36 x:0 0 0 2024-03-29T08:07:10.322 INFO:tasks.workunit.client.0.smithi037.stdout:5/237: rename d8/d22/l23 to d8/d22/d37/l47 0 2024-03-29T08:07:10.323 INFO:tasks.workunit.client.1.smithi148.stdout:9/147: rename f7 to db/d22/d26/f30 0 2024-03-29T08:07:10.324 INFO:tasks.workunit.client.1.smithi148.stdout:2/188: rmdir d7/d13/d2e 39 2024-03-29T08:07:10.329 INFO:tasks.workunit.client.0.smithi037.stdout:5/238: rename d8/d18/f19 to d8/d18/d24/f48 0 2024-03-29T08:07:10.331 INFO:tasks.workunit.client.0.smithi037.stdout:5/239: dread - d8/d34/f3a zero size 2024-03-29T08:07:10.333 INFO:tasks.workunit.client.1.smithi148.stdout:2/189: stat d7/da/d33/d3c/f23 0 2024-03-29T08:07:10.334 INFO:tasks.workunit.client.1.smithi148.stdout:7/197: dread d8/d29/f2e [0,4194304] 0 2024-03-29T08:07:10.336 INFO:tasks.workunit.client.1.smithi148.stdout:3/172: dwrite de/d24/f2b [0,4194304] 0 2024-03-29T08:07:10.336 INFO:tasks.workunit.client.0.smithi037.stdout:1/253: dwrite d2/d6/dc/f1e [0,4194304] 0 2024-03-29T08:07:10.337 INFO:tasks.workunit.client.1.smithi148.stdout:9/148: rmdir db/d10/d14 39 2024-03-29T08:07:10.338 INFO:tasks.workunit.client.1.smithi148.stdout:1/155: dread d3/de/f1b [0,4194304] 0 2024-03-29T08:07:10.339 INFO:tasks.workunit.client.1.smithi148.stdout:0/182: link db/de/df/d11/f37 db/de/f3f 0 2024-03-29T08:07:10.340 INFO:tasks.workunit.client.1.smithi148.stdout:3/173: readlink de/lf 0 2024-03-29T08:07:10.341 INFO:tasks.workunit.client.1.smithi148.stdout:7/198: chown d8/d14/f47 0 1 2024-03-29T08:07:10.342 INFO:tasks.workunit.client.1.smithi148.stdout:6/165: dread d6/fd [0,4194304] 0 2024-03-29T08:07:10.343 INFO:tasks.workunit.client.0.smithi037.stdout:1/254: dread d2/d6/dc/d18/d1d/d21/f2a [0,4194304] 0 2024-03-29T08:07:10.343 INFO:tasks.workunit.client.1.smithi148.stdout:0/183: write f7 [4684353,98988] 0 2024-03-29T08:07:10.345 INFO:tasks.workunit.client.0.smithi037.stdout:5/240: dwrite d8/f2b [0,4194304] 0 2024-03-29T08:07:10.346 INFO:tasks.workunit.client.1.smithi148.stdout:1/156: mknod d3/c2c 0 2024-03-29T08:07:10.347 INFO:tasks.workunit.client.1.smithi148.stdout:8/210: getdents d17/d18/d38 0 2024-03-29T08:07:10.348 INFO:tasks.workunit.client.0.smithi037.stdout:1/255: rename d2/d6/dc/d18/d1d/d21/d38/c49 to d2/d4b/c5b 0 2024-03-29T08:07:10.349 INFO:tasks.workunit.client.1.smithi148.stdout:7/199: fsync d8/f12 0 2024-03-29T08:07:10.357 INFO:tasks.workunit.client.1.smithi148.stdout:0/184: fsync db/de/df/d2b/f35 0 2024-03-29T08:07:10.359 INFO:tasks.workunit.client.1.smithi148.stdout:8/211: dread d17/d18/f1c [0,4194304] 0 2024-03-29T08:07:10.360 INFO:tasks.workunit.client.1.smithi148.stdout:4/147: dread f7 [0,4194304] 0 2024-03-29T08:07:10.362 INFO:tasks.workunit.client.0.smithi037.stdout:5/241: dwrite d8/d18/d24/f2a [0,4194304] 0 2024-03-29T08:07:10.362 INFO:tasks.workunit.client.1.smithi148.stdout:5/155: dwrite d0/f9 [0,4194304] 0 2024-03-29T08:07:10.365 INFO:tasks.workunit.client.1.smithi148.stdout:4/148: chown df/d10/d26 846732 1 2024-03-29T08:07:10.367 INFO:tasks.workunit.client.1.smithi148.stdout:5/156: readlink d0/db/lc 0 2024-03-29T08:07:10.367 INFO:tasks.workunit.client.1.smithi148.stdout:6/166: rename d6/d9/d1e/d22/f29 to d6/d9/f2c 0 2024-03-29T08:07:10.375 INFO:tasks.workunit.client.0.smithi037.stdout:5/242: symlink d8/d18/l49 0 2024-03-29T08:07:10.378 INFO:tasks.workunit.client.1.smithi148.stdout:8/212: write d17/d18/f3e [625160,104511] 0 2024-03-29T08:07:10.380 INFO:tasks.workunit.client.1.smithi148.stdout:9/149: dwrite db/d10/d14/d2b/f2e [0,4194304] 0 2024-03-29T08:07:10.383 INFO:tasks.workunit.client.0.smithi037.stdout:5/243: dread d8/d22/f31 [0,4194304] 0 2024-03-29T08:07:10.385 INFO:tasks.workunit.client.0.smithi037.stdout:5/244: dread - d8/d18/f28 zero size 2024-03-29T08:07:10.386 INFO:tasks.workunit.client.1.smithi148.stdout:5/157: creat d0/db/de/d16/f2f x:0 0 0 2024-03-29T08:07:10.390 INFO:tasks.workunit.client.1.smithi148.stdout:1/157: dwrite d3/d13/f15 [8388608,4194304] 0 2024-03-29T08:07:10.391 INFO:tasks.workunit.client.1.smithi148.stdout:9/150: write db/d10/f13 [1851770,37798] 0 2024-03-29T08:07:10.393 INFO:tasks.workunit.client.1.smithi148.stdout:7/200: dwrite d8/f18 [0,4194304] 0 2024-03-29T08:07:10.393 INFO:tasks.workunit.client.1.smithi148.stdout:8/213: readlink d17/d18/d38/d39/l40 0 2024-03-29T08:07:10.395 INFO:tasks.workunit.client.1.smithi148.stdout:7/201: dread - d8/d14/f47 zero size 2024-03-29T08:07:10.397 INFO:tasks.workunit.client.1.smithi148.stdout:5/158: mknod d0/db/de/d16/c30 0 2024-03-29T08:07:10.401 INFO:tasks.workunit.client.1.smithi148.stdout:0/185: dread db/de/df/d11/f22 [0,4194304] 0 2024-03-29T08:07:10.402 INFO:tasks.workunit.client.1.smithi148.stdout:7/202: read - d8/d14/d19/f39 zero size 2024-03-29T08:07:10.404 INFO:tasks.workunit.client.0.smithi037.stdout:5/245: dread d8/f17 [0,4194304] 0 2024-03-29T08:07:10.405 INFO:tasks.workunit.client.0.smithi037.stdout:5/246: dread - d8/f42 zero size 2024-03-29T08:07:10.405 INFO:tasks.workunit.client.1.smithi148.stdout:8/214: creat d17/d18/d26/d3d/f45 x:0 0 0 2024-03-29T08:07:10.406 INFO:tasks.workunit.client.1.smithi148.stdout:6/167: dread d6/fe [0,4194304] 0 2024-03-29T08:07:10.406 INFO:tasks.workunit.client.1.smithi148.stdout:9/151: creat db/d10/d2d/f31 x:0 0 0 2024-03-29T08:07:10.409 INFO:tasks.workunit.client.1.smithi148.stdout:6/168: dread - d6/d9/f23 zero size 2024-03-29T08:07:10.412 INFO:tasks.workunit.client.1.smithi148.stdout:6/169: readlink d6/d9/l1b 0 2024-03-29T08:07:10.415 INFO:tasks.workunit.client.0.smithi037.stdout:5/247: rename d8/d18/d24/f48 to d8/d34/f4a 0 2024-03-29T08:07:10.415 INFO:tasks.workunit.client.1.smithi148.stdout:1/158: getdents d3/d18 0 2024-03-29T08:07:10.415 INFO:tasks.workunit.client.1.smithi148.stdout:8/215: dread d17/d18/f3e [0,4194304] 0 2024-03-29T08:07:10.415 INFO:tasks.workunit.client.1.smithi148.stdout:4/149: rmdir df/d10/d17 39 2024-03-29T08:07:10.417 INFO:tasks.workunit.client.1.smithi148.stdout:4/150: write df/f27 [686564,67889] 0 2024-03-29T08:07:10.417 INFO:tasks.workunit.client.1.smithi148.stdout:1/159: write f1 [6377269,43491] 0 2024-03-29T08:07:10.420 INFO:tasks.workunit.client.0.smithi037.stdout:1/256: rmdir d2/d6/dc/d18/d1d/d21 39 2024-03-29T08:07:10.422 INFO:tasks.workunit.client.1.smithi148.stdout:9/152: mkdir db/d10/d2d/d32 0 2024-03-29T08:07:10.424 INFO:tasks.workunit.client.0.smithi037.stdout:1/257: mknod d2/d6/dc/d18/d22/c5c 0 2024-03-29T08:07:10.424 INFO:tasks.workunit.client.1.smithi148.stdout:9/153: fsync db/d20/f1b 0 2024-03-29T08:07:10.426 INFO:tasks.workunit.client.1.smithi148.stdout:9/154: chown db/d10 56967046 1 2024-03-29T08:07:10.426 INFO:tasks.workunit.client.0.smithi037.stdout:1/258: readlink d2/d6/dc/l24 0 2024-03-29T08:07:10.427 INFO:tasks.workunit.client.1.smithi148.stdout:0/186: dread f2 [4194304,4194304] 0 2024-03-29T08:07:10.427 INFO:tasks.workunit.client.0.smithi037.stdout:1/259: dread - d2/f57 zero size 2024-03-29T08:07:10.429 INFO:tasks.workunit.client.1.smithi148.stdout:7/203: rename d8/d14/f20 to d8/d14/d19/d37/f48 0 2024-03-29T08:07:10.429 INFO:tasks.workunit.client.0.smithi037.stdout:1/260: fsync d2/d6/dc/d58/f59 0 2024-03-29T08:07:10.431 INFO:tasks.workunit.client.0.smithi037.stdout:5/248: dread d8/d22/f2d [0,4194304] 0 2024-03-29T08:07:10.434 INFO:tasks.workunit.client.1.smithi148.stdout:7/204: dread - d8/d14/f1e zero size 2024-03-29T08:07:10.434 INFO:tasks.workunit.client.1.smithi148.stdout:6/170: write d6/fe [3414892,126992] 0 2024-03-29T08:07:10.434 INFO:tasks.workunit.client.1.smithi148.stdout:5/159: getdents d0/db 0 2024-03-29T08:07:10.434 INFO:tasks.workunit.client.0.smithi037.stdout:5/249: mkdir d8/d34/d4b 0 2024-03-29T08:07:10.434 INFO:tasks.workunit.client.0.smithi037.stdout:1/261: chown d2/d6/dc/d18/d1d/d21/f37 21788 1 2024-03-29T08:07:10.434 INFO:tasks.workunit.client.1.smithi148.stdout:8/216: fsync d17/f23 0 2024-03-29T08:07:10.435 INFO:tasks.workunit.client.1.smithi148.stdout:4/151: symlink df/l37 0 2024-03-29T08:07:10.437 INFO:tasks.workunit.client.0.smithi037.stdout:1/262: mkdir d2/d6/dc/d5d 0 2024-03-29T08:07:10.437 INFO:tasks.workunit.client.1.smithi148.stdout:6/171: readlink d6/d9/l1b 0 2024-03-29T08:07:10.438 INFO:tasks.workunit.client.1.smithi148.stdout:5/160: stat d0/db/l22 0 2024-03-29T08:07:10.438 INFO:tasks.workunit.client.1.smithi148.stdout:0/187: rmdir db 39 2024-03-29T08:07:10.440 INFO:tasks.workunit.client.0.smithi037.stdout:1/263: write d2/d6/dc/d18/d1d/f2c [2809965,25487] 0 2024-03-29T08:07:10.440 INFO:tasks.workunit.client.1.smithi148.stdout:9/155: rename db/d10/d14/d2b to db/d10/d2d/d33 0 2024-03-29T08:07:10.446 INFO:tasks.workunit.client.1.smithi148.stdout:1/160: dwrite d3/f7 [0,4194304] 0 2024-03-29T08:07:10.448 INFO:tasks.workunit.client.1.smithi148.stdout:4/152: write df/d10/d17/f36 [276375,5291] 0 2024-03-29T08:07:10.448 INFO:tasks.workunit.client.1.smithi148.stdout:6/172: creat d6/f2d x:0 0 0 2024-03-29T08:07:10.448 INFO:tasks.workunit.client.1.smithi148.stdout:0/188: chown db/de/df/d11/f33 165745590 1 2024-03-29T08:07:10.451 INFO:tasks.workunit.client.1.smithi148.stdout:5/161: unlink d0/l4 0 2024-03-29T08:07:10.454 INFO:tasks.workunit.client.0.smithi037.stdout:1/264: dread d2/d6/dc/d18/d22/f48 [0,4194304] 0 2024-03-29T08:07:10.458 INFO:tasks.workunit.client.1.smithi148.stdout:9/156: rename db/d10/f13 to db/d10/d2d/f34 0 2024-03-29T08:07:10.458 INFO:tasks.workunit.client.1.smithi148.stdout:1/161: creat d3/f2d x:0 0 0 2024-03-29T08:07:10.458 INFO:tasks.workunit.client.1.smithi148.stdout:9/157: chown db/d22/d26/f2a 1426072 1 2024-03-29T08:07:10.460 INFO:tasks.workunit.client.1.smithi148.stdout:6/173: mknod d6/d9/d1e/d27/c2e 0 2024-03-29T08:07:10.462 INFO:tasks.workunit.client.0.smithi037.stdout:5/250: dread d8/d18/d24/f44 [0,4194304] 0 2024-03-29T08:07:10.463 INFO:tasks.workunit.client.1.smithi148.stdout:6/174: read - d6/d9/f2c zero size 2024-03-29T08:07:10.465 INFO:tasks.workunit.client.1.smithi148.stdout:7/205: rmdir d8/d29/d41 0 2024-03-29T08:07:10.465 INFO:tasks.workunit.client.0.smithi037.stdout:5/251: fsync d8/d18/f21 0 2024-03-29T08:07:10.466 INFO:tasks.workunit.client.1.smithi148.stdout:8/217: getdents d17/d18/d38 0 2024-03-29T08:07:10.466 INFO:tasks.workunit.client.1.smithi148.stdout:6/175: write d6/f14 [712239,37568] 0 2024-03-29T08:07:10.467 INFO:tasks.workunit.client.1.smithi148.stdout:9/158: dread db/d10/d2d/f34 [4194304,4194304] 0 2024-03-29T08:07:10.468 INFO:tasks.workunit.client.0.smithi037.stdout:5/252: write d8/d22/f26 [2930518,92365] 0 2024-03-29T08:07:10.469 INFO:tasks.workunit.client.1.smithi148.stdout:1/162: rename d3/de/d1d/l27 to d3/d18/l2e 0 2024-03-29T08:07:10.469 INFO:tasks.workunit.client.1.smithi148.stdout:4/153: dwrite df/d10/f12 [0,4194304] 0 2024-03-29T08:07:10.470 INFO:tasks.workunit.client.1.smithi148.stdout:7/206: truncate d8/d14/f1e 56419 0 2024-03-29T08:07:10.473 INFO:tasks.workunit.client.1.smithi148.stdout:1/163: read d3/de/f1b [15695,107988] 0 2024-03-29T08:07:10.473 INFO:tasks.workunit.client.0.smithi037.stdout:1/265: dwrite d2/d6/dc/d18/f3e [0,4194304] 0 2024-03-29T08:07:10.474 INFO:tasks.workunit.client.1.smithi148.stdout:8/218: write d17/d18/f34 [461640,32381] 0 2024-03-29T08:07:10.475 INFO:tasks.workunit.client.1.smithi148.stdout:6/176: rename d6/c17 to d6/d9/d1e/d27/c2f 0 2024-03-29T08:07:10.477 INFO:tasks.workunit.client.0.smithi037.stdout:5/253: creat d8/f4c x:0 0 0 2024-03-29T08:07:10.477 INFO:tasks.workunit.client.1.smithi148.stdout:0/189: getdents db/de/df 0 2024-03-29T08:07:10.479 INFO:tasks.workunit.client.1.smithi148.stdout:9/159: mkdir db/d10/d14/d35 0 2024-03-29T08:07:10.486 INFO:tasks.workunit.client.0.smithi037.stdout:1/266: mkdir d2/d6/dc/d18/d1d/d5e 0 2024-03-29T08:07:10.487 INFO:tasks.workunit.client.1.smithi148.stdout:8/219: mknod d17/d18/c46 0 2024-03-29T08:07:10.487 INFO:tasks.workunit.client.1.smithi148.stdout:1/164: dread d3/f7 [0,4194304] 0 2024-03-29T08:07:10.489 INFO:tasks.workunit.client.1.smithi148.stdout:4/154: dread fc [0,4194304] 0 2024-03-29T08:07:10.489 INFO:tasks.workunit.client.1.smithi148.stdout:6/177: mkdir d6/d9/d1e/d20/d30 0 2024-03-29T08:07:10.490 INFO:tasks.workunit.client.1.smithi148.stdout:1/165: dread d3/d18/f20 [0,4194304] 0 2024-03-29T08:07:10.491 INFO:tasks.workunit.client.0.smithi037.stdout:1/267: write d2/d6/f41 [746396,33236] 0 2024-03-29T08:07:10.491 INFO:tasks.workunit.client.0.smithi037.stdout:5/254: rmdir d8 39 2024-03-29T08:07:10.493 INFO:tasks.workunit.client.1.smithi148.stdout:7/207: mkdir d8/d29/d44/d49 0 2024-03-29T08:07:10.496 INFO:tasks.workunit.client.1.smithi148.stdout:5/162: truncate d0/f9 2458826 0 2024-03-29T08:07:10.496 INFO:tasks.workunit.client.0.smithi037.stdout:1/268: creat d2/d6/dc/d18/f5f x:0 0 0 2024-03-29T08:07:10.497 INFO:tasks.workunit.client.0.smithi037.stdout:5/255: mkdir d8/d4d 0 2024-03-29T08:07:10.497 INFO:tasks.workunit.client.1.smithi148.stdout:8/220: mknod d17/d18/d38/d39/c47 0 2024-03-29T08:07:10.498 INFO:tasks.workunit.client.1.smithi148.stdout:6/178: creat d6/d9/d1e/d22/f31 x:0 0 0 2024-03-29T08:07:10.498 INFO:tasks.workunit.client.1.smithi148.stdout:8/221: fdatasync d17/d18/d26/d3d/f45 0 2024-03-29T08:07:10.499 INFO:tasks.workunit.client.1.smithi148.stdout:6/179: dread - d6/d9/d1e/f26 zero size 2024-03-29T08:07:10.500 INFO:tasks.workunit.client.1.smithi148.stdout:1/166: symlink d3/d18/l2f 0 2024-03-29T08:07:10.501 INFO:tasks.workunit.client.1.smithi148.stdout:9/160: getdents db/d10/d2d/d32 0 2024-03-29T08:07:10.501 INFO:tasks.workunit.client.1.smithi148.stdout:7/208: mknod d8/d14/d19/c4a 0 2024-03-29T08:07:10.502 INFO:tasks.workunit.client.1.smithi148.stdout:5/163: creat d0/db/d1e/f31 x:0 0 0 2024-03-29T08:07:10.502 INFO:tasks.workunit.client.1.smithi148.stdout:9/161: dread - db/f2f zero size 2024-03-29T08:07:10.503 INFO:tasks.workunit.client.1.smithi148.stdout:4/155: creat df/d10/d26/d34/f38 x:0 0 0 2024-03-29T08:07:10.503 INFO:tasks.workunit.client.1.smithi148.stdout:7/209: chown d8/fc 110 1 2024-03-29T08:07:10.505 INFO:tasks.workunit.client.0.smithi037.stdout:1/269: truncate d2/d6/dc/d18/d1d/d21/d38/f56 968052 0 2024-03-29T08:07:10.505 INFO:tasks.workunit.client.1.smithi148.stdout:5/164: chown d0/db/l22 3235 1 2024-03-29T08:07:10.505 INFO:tasks.workunit.client.1.smithi148.stdout:1/167: creat d3/f30 x:0 0 0 2024-03-29T08:07:10.507 INFO:tasks.workunit.client.0.smithi037.stdout:1/270: dread - d2/d6/dc/d18/d1d/d21/d38/f4e zero size 2024-03-29T08:07:10.508 INFO:tasks.workunit.client.1.smithi148.stdout:6/180: creat d6/d9/d1e/d27/f32 x:0 0 0 2024-03-29T08:07:10.510 INFO:tasks.workunit.client.0.smithi037.stdout:1/271: read - d2/d6/dc/d18/f5f zero size 2024-03-29T08:07:10.511 INFO:tasks.workunit.client.1.smithi148.stdout:7/210: creat d8/d14/d19/f4b x:0 0 0 2024-03-29T08:07:10.512 INFO:tasks.workunit.client.0.smithi037.stdout:5/256: dread d8/d22/f27 [0,4194304] 0 2024-03-29T08:07:10.515 INFO:tasks.workunit.client.0.smithi037.stdout:1/272: creat d2/d6/dc/d18/d1d/d21/f60 x:0 0 0 2024-03-29T08:07:10.516 INFO:tasks.workunit.client.1.smithi148.stdout:6/181: chown d6/d9/lb 38906833 1 2024-03-29T08:07:10.516 INFO:tasks.workunit.client.0.smithi037.stdout:5/257: truncate d8/d34/f3a 570366 0 2024-03-29T08:07:10.517 INFO:tasks.workunit.client.1.smithi148.stdout:5/165: symlink d0/d26/l32 0 2024-03-29T08:07:10.518 INFO:tasks.workunit.client.1.smithi148.stdout:6/182: stat d6/d9/d1e/d27 0 2024-03-29T08:07:10.520 INFO:tasks.workunit.client.0.smithi037.stdout:1/273: write d2/d6/dc/d18/d1d/d21/d38/f4e [208115,82035] 0 2024-03-29T08:07:10.520 INFO:tasks.workunit.client.0.smithi037.stdout:5/258: rename d8/c15 to d8/d22/d37/c4e 0 2024-03-29T08:07:10.521 INFO:tasks.workunit.client.1.smithi148.stdout:9/162: dwrite db/f21 [0,4194304] 0 2024-03-29T08:07:10.522 INFO:tasks.workunit.client.1.smithi148.stdout:5/166: rmdir d0/db/de 39 2024-03-29T08:07:10.524 INFO:tasks.workunit.client.0.smithi037.stdout:5/259: mkdir d8/d22/d4f 0 2024-03-29T08:07:10.525 INFO:tasks.workunit.client.1.smithi148.stdout:2/190: sync 2024-03-29T08:07:10.525 INFO:tasks.workunit.client.1.smithi148.stdout:3/174: sync 2024-03-29T08:07:10.525 INFO:tasks.workunit.client.1.smithi148.stdout:4/156: link df/d10/d26/l2b df/d10/l39 0 2024-03-29T08:07:10.526 INFO:tasks.workunit.client.0.smithi037.stdout:5/260: stat d8/d22/l2f 0 2024-03-29T08:07:10.526 INFO:tasks.workunit.client.0.smithi037.stdout:1/274: symlink d2/d6/dc/d18/d1d/l61 0 2024-03-29T08:07:10.529 INFO:tasks.workunit.client.1.smithi148.stdout:7/211: rename d8/d14/d19/c33 to d8/c4c 0 2024-03-29T08:07:10.530 INFO:tasks.workunit.client.1.smithi148.stdout:2/191: truncate d7/d24/f34 1561936 0 2024-03-29T08:07:10.531 INFO:tasks.workunit.client.0.smithi037.stdout:5/261: creat d8/d18/f50 x:0 0 0 2024-03-29T08:07:10.532 INFO:tasks.workunit.client.1.smithi148.stdout:4/157: chown df/d10/d17/l24 9858372 1 2024-03-29T08:07:10.532 INFO:tasks.workunit.client.1.smithi148.stdout:3/175: unlink f5 0 2024-03-29T08:07:10.532 INFO:tasks.workunit.client.0.smithi037.stdout:1/275: mkdir d2/d6/dc/d5d/d62 0 2024-03-29T08:07:10.534 INFO:tasks.workunit.client.1.smithi148.stdout:9/163: creat db/d10/d2d/d33/f36 x:0 0 0 2024-03-29T08:07:10.535 INFO:tasks.workunit.client.0.smithi037.stdout:1/276: creat d2/d6/dc/d18/f63 x:0 0 0 2024-03-29T08:07:10.536 INFO:tasks.workunit.client.1.smithi148.stdout:2/192: readlink d7/da/l1d 0 2024-03-29T08:07:10.538 INFO:tasks.workunit.client.1.smithi148.stdout:1/168: dwrite d3/de/f1b [0,4194304] 0 2024-03-29T08:07:10.538 INFO:tasks.workunit.client.1.smithi148.stdout:7/212: rmdir d8/d14 39 2024-03-29T08:07:10.540 INFO:tasks.workunit.client.0.smithi037.stdout:5/262: creat d8/d18/d24/d45/f51 x:0 0 0 2024-03-29T08:07:10.540 INFO:tasks.workunit.client.1.smithi148.stdout:8/222: dwrite d17/f23 [0,4194304] 0 2024-03-29T08:07:10.540 INFO:tasks.workunit.client.1.smithi148.stdout:6/183: write d6/d9/f1f [162171,36565] 0 2024-03-29T08:07:10.542 INFO:tasks.workunit.client.0.smithi037.stdout:1/277: write d2/d6/dc/d18/d1d/d21/d38/f50 [857752,118460] 0 2024-03-29T08:07:10.544 INFO:tasks.workunit.client.1.smithi148.stdout:3/176: mknod de/d24/c38 0 2024-03-29T08:07:10.548 INFO:tasks.workunit.client.1.smithi148.stdout:8/223: creat d17/d18/d38/f48 x:0 0 0 2024-03-29T08:07:10.548 INFO:tasks.workunit.client.1.smithi148.stdout:4/158: link df/l37 df/d2d/l3a 0 2024-03-29T08:07:10.550 INFO:tasks.workunit.client.0.smithi037.stdout:5/263: creat d8/f52 x:0 0 0 2024-03-29T08:07:10.550 INFO:tasks.workunit.client.1.smithi148.stdout:9/164: mknod db/d10/d14/d35/c37 0 2024-03-29T08:07:10.550 INFO:tasks.workunit.client.1.smithi148.stdout:7/213: mkdir d8/d14/d45/d4d 0 2024-03-29T08:07:10.553 INFO:tasks.workunit.client.1.smithi148.stdout:7/214: stat d8/f12 0 2024-03-29T08:07:10.554 INFO:tasks.workunit.client.1.smithi148.stdout:6/184: creat d6/d9/d1e/d20/d30/f33 x:0 0 0 2024-03-29T08:07:10.555 INFO:tasks.workunit.client.1.smithi148.stdout:8/224: mkdir d17/d49 0 2024-03-29T08:07:10.556 INFO:tasks.workunit.client.1.smithi148.stdout:4/159: creat df/d10/d26/f3b x:0 0 0 2024-03-29T08:07:10.559 INFO:tasks.workunit.client.1.smithi148.stdout:9/165: creat db/d22/d26/f38 x:0 0 0 2024-03-29T08:07:10.559 INFO:tasks.workunit.client.1.smithi148.stdout:7/215: fdatasync d8/d14/d19/f34 0 2024-03-29T08:07:10.561 INFO:tasks.workunit.client.1.smithi148.stdout:8/225: fdatasync d17/d18/f3f 0 2024-03-29T08:07:10.561 INFO:tasks.workunit.client.1.smithi148.stdout:6/185: truncate d6/f2d 832666 0 2024-03-29T08:07:10.565 INFO:tasks.workunit.client.1.smithi148.stdout:2/193: dwrite d7/da/d33/f35 [4194304,4194304] 0 2024-03-29T08:07:10.567 INFO:tasks.workunit.client.1.smithi148.stdout:9/166: write db/d10/f29 [467229,89688] 0 2024-03-29T08:07:10.567 INFO:tasks.workunit.client.1.smithi148.stdout:1/169: dwrite f1 [12582912,4194304] 0 2024-03-29T08:07:10.567 INFO:tasks.workunit.client.0.smithi037.stdout:5/264: dwrite d8/f1f [0,4194304] 0 2024-03-29T08:07:10.568 INFO:tasks.workunit.client.0.smithi037.stdout:5/265: chown l6 5 1 2024-03-29T08:07:10.569 INFO:tasks.workunit.client.1.smithi148.stdout:1/170: chown d3/d18/f20 0 1 2024-03-29T08:07:10.570 INFO:tasks.workunit.client.0.smithi037.stdout:5/266: chown d8/d18/d24/d45 101 1 2024-03-29T08:07:10.571 INFO:tasks.workunit.client.1.smithi148.stdout:8/226: creat d17/d18/d38/f4a x:0 0 0 2024-03-29T08:07:10.573 INFO:tasks.workunit.client.0.smithi037.stdout:5/267: symlink d8/d34/d3b/l53 0 2024-03-29T08:07:10.574 INFO:tasks.workunit.client.1.smithi148.stdout:7/216: symlink d8/d29/d44/d49/l4e 0 2024-03-29T08:07:10.575 INFO:tasks.workunit.client.0.smithi037.stdout:5/268: chown d8/l1c 689197 1 2024-03-29T08:07:10.575 INFO:tasks.workunit.client.1.smithi148.stdout:2/194: creat d7/d24/f46 x:0 0 0 2024-03-29T08:07:10.577 INFO:tasks.workunit.client.1.smithi148.stdout:7/217: dread - d8/d38/f43 zero size 2024-03-29T08:07:10.578 INFO:tasks.workunit.client.1.smithi148.stdout:2/195: fsync d7/d13/f29 0 2024-03-29T08:07:10.579 INFO:tasks.workunit.client.1.smithi148.stdout:8/227: creat d17/d18/d38/f4b x:0 0 0 2024-03-29T08:07:10.579 INFO:tasks.workunit.client.0.smithi037.stdout:5/269: creat d8/d34/d3b/f54 x:0 0 0 2024-03-29T08:07:10.582 INFO:tasks.workunit.client.1.smithi148.stdout:7/218: chown d8/f26 65050741 1 2024-03-29T08:07:10.582 INFO:tasks.workunit.client.1.smithi148.stdout:1/171: mkdir d3/de/d25/d31 0 2024-03-29T08:07:10.585 INFO:tasks.workunit.client.1.smithi148.stdout:4/160: rmdir df/d10/d26 39 2024-03-29T08:07:10.586 INFO:tasks.workunit.client.1.smithi148.stdout:1/172: chown d3/l17 11 1 2024-03-29T08:07:10.591 INFO:tasks.workunit.client.1.smithi148.stdout:6/186: dwrite d6/d9/f16 [0,4194304] 0 2024-03-29T08:07:10.591 INFO:tasks.workunit.client.1.smithi148.stdout:2/196: creat d7/d13/f47 x:0 0 0 2024-03-29T08:07:10.595 INFO:tasks.workunit.client.1.smithi148.stdout:9/167: truncate f3 5945456 0 2024-03-29T08:07:10.596 INFO:tasks.workunit.client.1.smithi148.stdout:1/173: unlink d3/d13/f28 0 2024-03-29T08:07:10.598 INFO:tasks.workunit.client.1.smithi148.stdout:9/168: chown db/d10/d2d/d33 1 1 2024-03-29T08:07:10.599 INFO:tasks.workunit.client.1.smithi148.stdout:6/187: chown d6/fa 12161166 1 2024-03-29T08:07:10.600 INFO:tasks.workunit.client.1.smithi148.stdout:2/197: creat d7/da/d33/d3c/f48 x:0 0 0 2024-03-29T08:07:10.601 INFO:tasks.workunit.client.0.smithi037.stdout:5/270: dwrite d8/d18/d24/d45/f51 [0,4194304] 0 2024-03-29T08:07:10.602 INFO:tasks.workunit.client.1.smithi148.stdout:8/228: getdents d17/d18/d38/d39 0 2024-03-29T08:07:10.603 INFO:tasks.workunit.client.1.smithi148.stdout:4/161: symlink df/d10/d26/d34/l3c 0 2024-03-29T08:07:10.603 INFO:tasks.workunit.client.1.smithi148.stdout:1/174: mkdir d3/de/d21/d32 0 2024-03-29T08:07:10.604 INFO:tasks.workunit.client.0.smithi037.stdout:5/271: symlink d8/d34/l55 0 2024-03-29T08:07:10.604 INFO:tasks.workunit.client.1.smithi148.stdout:9/169: mkdir db/d22/d39 0 2024-03-29T08:07:10.606 INFO:tasks.workunit.client.1.smithi148.stdout:1/175: chown d3/de/d1d/l24 0 1 2024-03-29T08:07:10.607 INFO:tasks.workunit.client.1.smithi148.stdout:6/188: creat d6/d9/d1e/d22/f34 x:0 0 0 2024-03-29T08:07:10.607 INFO:tasks.workunit.client.0.smithi037.stdout:5/272: mknod d8/c56 0 2024-03-29T08:07:10.611 INFO:tasks.workunit.client.0.smithi037.stdout:5/273: mknod d8/d34/d4b/c57 0 2024-03-29T08:07:10.612 INFO:tasks.workunit.client.1.smithi148.stdout:4/162: creat df/d10/d26/f3d x:0 0 0 2024-03-29T08:07:10.615 INFO:tasks.workunit.client.0.smithi037.stdout:5/274: symlink d8/d34/l58 0 2024-03-29T08:07:10.615 INFO:tasks.workunit.client.1.smithi148.stdout:8/229: rename d17/d1d/l29 to d17/d18/d26/d3d/l4c 0 2024-03-29T08:07:10.616 INFO:tasks.workunit.client.1.smithi148.stdout:0/190: sync 2024-03-29T08:07:10.617 INFO:tasks.workunit.client.1.smithi148.stdout:8/230: readlink d17/d18/l24 0 2024-03-29T08:07:10.618 INFO:tasks.workunit.client.1.smithi148.stdout:8/231: chown c11 2770814 1 2024-03-29T08:07:10.618 INFO:tasks.workunit.client.1.smithi148.stdout:0/191: dread - db/de/f25 zero size 2024-03-29T08:07:10.620 INFO:tasks.workunit.client.1.smithi148.stdout:7/219: truncate d8/f13 2462015 0 2024-03-29T08:07:10.622 INFO:tasks.workunit.client.1.smithi148.stdout:1/176: dread d3/fa [8388608,4194304] 0 2024-03-29T08:07:10.622 INFO:tasks.workunit.client.1.smithi148.stdout:6/189: chown d6/d9/d1e/d27/c2f 7816017 1 2024-03-29T08:07:10.624 INFO:tasks.workunit.client.1.smithi148.stdout:2/198: link d7/f1b d7/da/f49 0 2024-03-29T08:07:10.625 INFO:tasks.workunit.client.1.smithi148.stdout:8/232: rename d17/f2d to d17/d18/f4d 0 2024-03-29T08:07:10.627 INFO:tasks.workunit.client.1.smithi148.stdout:8/233: chown d17/d18/c1f 3231196 1 2024-03-29T08:07:10.628 INFO:tasks.workunit.client.1.smithi148.stdout:7/220: creat d8/d29/f4f x:0 0 0 2024-03-29T08:07:10.631 INFO:tasks.workunit.client.1.smithi148.stdout:0/192: unlink db/de/df/d14/l20 0 2024-03-29T08:07:10.634 INFO:tasks.workunit.client.1.smithi148.stdout:8/234: mkdir d17/d4e 0 2024-03-29T08:07:10.635 INFO:tasks.workunit.client.1.smithi148.stdout:4/163: getdents df/d10/d26 0 2024-03-29T08:07:10.639 INFO:tasks.workunit.client.0.smithi037.stdout:8/182: sync 2024-03-29T08:07:10.641 INFO:tasks.workunit.client.1.smithi148.stdout:0/193: write db/de/df/d11/d2d/f36 [939751,52519] 0 2024-03-29T08:07:10.641 INFO:tasks.workunit.client.0.smithi037.stdout:8/183: fsync d5/d25/f28 0 2024-03-29T08:07:10.644 INFO:tasks.workunit.client.0.smithi037.stdout:8/184: write d5/d10/f35 [742919,105551] 0 2024-03-29T08:07:10.644 INFO:tasks.workunit.client.1.smithi148.stdout:4/164: write df/f14 [4005383,5864] 0 2024-03-29T08:07:10.645 INFO:tasks.workunit.client.1.smithi148.stdout:6/190: dwrite d6/d9/d1e/f26 [0,4194304] 0 2024-03-29T08:07:10.648 INFO:tasks.workunit.client.1.smithi148.stdout:9/170: dwrite f4 [0,4194304] 0 2024-03-29T08:07:10.649 INFO:tasks.workunit.client.0.smithi037.stdout:5/275: dread d8/fa [0,4194304] 0 2024-03-29T08:07:10.650 INFO:tasks.workunit.client.1.smithi148.stdout:7/221: getdents d8/d14/d45 0 2024-03-29T08:07:10.651 INFO:tasks.workunit.client.0.smithi037.stdout:5/276: chown d8/ld 0 1 2024-03-29T08:07:10.652 INFO:tasks.workunit.client.1.smithi148.stdout:4/165: write df/d10/f12 [1918139,22339] 0 2024-03-29T08:07:10.652 INFO:tasks.workunit.client.0.smithi037.stdout:8/185: getdents d5/d25 0 2024-03-29T08:07:10.653 INFO:tasks.workunit.client.1.smithi148.stdout:0/194: mkdir db/de/d40 0 2024-03-29T08:07:10.657 INFO:tasks.workunit.client.0.smithi037.stdout:8/186: symlink d5/d10/l3b 0 2024-03-29T08:07:10.657 INFO:tasks.workunit.client.1.smithi148.stdout:9/171: write db/d10/f29 [95644,120] 0 2024-03-29T08:07:10.659 INFO:tasks.workunit.client.0.smithi037.stdout:8/187: stat d5/d10/f35 0 2024-03-29T08:07:10.660 INFO:tasks.workunit.client.1.smithi148.stdout:7/222: mkdir d8/d29/d44/d50 0 2024-03-29T08:07:10.661 INFO:tasks.workunit.client.0.smithi037.stdout:5/277: getdents d8/d22 0 2024-03-29T08:07:10.665 INFO:tasks.workunit.client.0.smithi037.stdout:5/278: write d8/d22/f27 [4025492,46957] 0 2024-03-29T08:07:10.667 INFO:tasks.workunit.client.1.smithi148.stdout:6/191: link d6/d9/d1e/d20/d30/f33 d6/d9/d1e/d20/d30/f35 0 2024-03-29T08:07:10.667 INFO:tasks.workunit.client.1.smithi148.stdout:8/235: dwrite f15 [0,4194304] 0 2024-03-29T08:07:10.668 INFO:tasks.workunit.client.1.smithi148.stdout:2/199: dwrite d7/f3b [0,4194304] 0 2024-03-29T08:07:10.668 INFO:tasks.workunit.client.0.smithi037.stdout:5/279: mknod d8/d34/c59 0 2024-03-29T08:07:10.669 INFO:tasks.workunit.client.1.smithi148.stdout:9/172: write db/d22/d26/f2a [2744361,94355] 0 2024-03-29T08:07:10.669 INFO:tasks.workunit.client.0.smithi037.stdout:5/280: chown d8/d22/f2d 211057 1 2024-03-29T08:07:10.671 INFO:tasks.workunit.client.1.smithi148.stdout:2/200: read d7/da/d33/d3c/d21/f25 [3090904,28844] 0 2024-03-29T08:07:10.671 INFO:tasks.workunit.client.1.smithi148.stdout:9/173: read db/f21 [1215928,65377] 0 2024-03-29T08:07:10.673 INFO:tasks.workunit.client.1.smithi148.stdout:5/167: sync 2024-03-29T08:07:10.674 INFO:tasks.workunit.client.0.smithi037.stdout:5/281: creat d8/d22/d4f/f5a x:0 0 0 2024-03-29T08:07:10.677 INFO:tasks.workunit.client.1.smithi148.stdout:8/236: write d17/d18/d38/f4a [638530,37753] 0 2024-03-29T08:07:10.678 INFO:tasks.workunit.client.0.smithi037.stdout:8/188: dread d5/f16 [0,4194304] 0 2024-03-29T08:07:10.678 INFO:tasks.workunit.client.1.smithi148.stdout:9/174: write db/d10/d14/f18 [4759819,116420] 0 2024-03-29T08:07:10.680 INFO:tasks.workunit.client.0.smithi037.stdout:8/189: stat d5/d10/d19/l2a 0 2024-03-29T08:07:10.681 INFO:tasks.workunit.client.1.smithi148.stdout:4/166: dread fd [0,4194304] 0 2024-03-29T08:07:10.681 INFO:tasks.workunit.client.1.smithi148.stdout:0/195: getdents db/de/df/d11/d2d 0 2024-03-29T08:07:10.682 INFO:tasks.workunit.client.1.smithi148.stdout:7/223: getdents d8/d38 0 2024-03-29T08:07:10.683 INFO:tasks.workunit.client.0.smithi037.stdout:8/190: chown f4 68 1 2024-03-29T08:07:10.689 INFO:tasks.workunit.client.1.smithi148.stdout:8/237: mkdir d17/d18/d26/d4f 0 2024-03-29T08:07:10.691 INFO:tasks.workunit.client.1.smithi148.stdout:0/196: unlink f2 0 2024-03-29T08:07:10.693 INFO:tasks.workunit.client.1.smithi148.stdout:4/167: mkdir df/d10/d26/d34/d3e 0 2024-03-29T08:07:10.695 INFO:tasks.workunit.client.1.smithi148.stdout:5/168: stat d0/db/de/cf 0 2024-03-29T08:07:10.695 INFO:tasks.workunit.client.0.smithi037.stdout:5/282: dwrite d8/f17 [4194304,4194304] 0 2024-03-29T08:07:10.696 INFO:tasks.workunit.client.1.smithi148.stdout:7/224: rename d8/f12 to d8/d14/d19/d37/f51 0 2024-03-29T08:07:10.697 INFO:tasks.workunit.client.1.smithi148.stdout:6/192: dwrite d6/fe [0,4194304] 0 2024-03-29T08:07:10.701 INFO:tasks.workunit.client.1.smithi148.stdout:3/177: sync 2024-03-29T08:07:10.701 INFO:tasks.workunit.client.1.smithi148.stdout:9/175: link db/d10/d2d/d33/f2e db/d10/d14/d35/f3a 0 2024-03-29T08:07:10.702 INFO:tasks.workunit.client.1.smithi148.stdout:5/169: readlink d0/db/l22 0 2024-03-29T08:07:10.702 INFO:tasks.workunit.client.0.smithi037.stdout:5/283: mkdir d8/d5b 0 2024-03-29T08:07:10.704 INFO:tasks.workunit.client.1.smithi148.stdout:2/201: dwrite d7/d24/f46 [0,4194304] 0 2024-03-29T08:07:10.705 INFO:tasks.workunit.client.1.smithi148.stdout:0/197: creat db/de/df/d11/f41 x:0 0 0 2024-03-29T08:07:10.705 INFO:tasks.workunit.client.1.smithi148.stdout:3/178: fsync de/d13/f21 0 2024-03-29T08:07:10.706 INFO:tasks.workunit.client.0.smithi037.stdout:8/191: dwrite f4 [0,4194304] 0 2024-03-29T08:07:10.707 INFO:tasks.workunit.client.1.smithi148.stdout:5/170: fsync d0/db/de/d16/f27 0 2024-03-29T08:07:10.710 INFO:tasks.workunit.client.0.smithi037.stdout:5/284: mkdir d8/d4d/d5c 0 2024-03-29T08:07:10.712 INFO:tasks.workunit.client.0.smithi037.stdout:8/192: read d5/d10/f23 [169633,107930] 0 2024-03-29T08:07:10.715 INFO:tasks.workunit.client.0.smithi037.stdout:8/193: mknod d5/c3c 0 2024-03-29T08:07:10.718 INFO:tasks.workunit.client.0.smithi037.stdout:8/194: write d5/d25/f2e [556086,66685] 0 2024-03-29T08:07:10.718 INFO:tasks.workunit.client.1.smithi148.stdout:7/225: rename d8/d14/d19/f34 to d8/d29/d44/d50/f52 0 2024-03-29T08:07:10.719 INFO:tasks.workunit.client.0.smithi037.stdout:8/195: dread - d5/f3a zero size 2024-03-29T08:07:10.721 INFO:tasks.workunit.client.1.smithi148.stdout:0/198: mkdir db/de/df/d11/d42 0 2024-03-29T08:07:10.726 INFO:tasks.workunit.client.1.smithi148.stdout:4/168: creat df/d10/d26/d34/d3e/f3f x:0 0 0 2024-03-29T08:07:10.728 INFO:tasks.workunit.client.1.smithi148.stdout:2/202: creat d7/da/d33/d3c/d31/f4a x:0 0 0 2024-03-29T08:07:10.728 INFO:tasks.workunit.client.1.smithi148.stdout:8/238: dwrite d17/d18/d38/f48 [0,4194304] 0 2024-03-29T08:07:10.729 INFO:tasks.workunit.client.1.smithi148.stdout:6/193: dread d6/fe [0,4194304] 0 2024-03-29T08:07:10.729 INFO:tasks.workunit.client.1.smithi148.stdout:7/226: fdatasync d8/d14/d19/f39 0 2024-03-29T08:07:10.731 INFO:tasks.workunit.client.0.smithi037.stdout:8/196: dread d5/f1b [0,4194304] 0 2024-03-29T08:07:10.733 INFO:tasks.workunit.client.1.smithi148.stdout:3/179: mkdir de/d24/d33/d39 0 2024-03-29T08:07:10.733 INFO:tasks.workunit.client.1.smithi148.stdout:0/199: symlink db/de/df/l43 0 2024-03-29T08:07:10.734 INFO:tasks.workunit.client.0.smithi037.stdout:5/285: write d8/d18/d24/f44 [786856,40094] 0 2024-03-29T08:07:10.735 INFO:tasks.workunit.client.1.smithi148.stdout:8/239: truncate f14 4302911 0 2024-03-29T08:07:10.735 INFO:tasks.workunit.client.1.smithi148.stdout:6/194: write d6/f1a [1287318,89770] 0 2024-03-29T08:07:10.735 INFO:tasks.workunit.client.1.smithi148.stdout:4/169: write df/d10/f12 [4890005,85466] 0 2024-03-29T08:07:10.736 INFO:tasks.workunit.client.0.smithi037.stdout:5/286: chown d8/d22/l32 1 1 2024-03-29T08:07:10.737 INFO:tasks.workunit.client.1.smithi148.stdout:3/180: mknod de/d24/c3a 0 2024-03-29T08:07:10.738 INFO:tasks.workunit.client.1.smithi148.stdout:2/203: readlink d7/da/d33/d3c/l30 0 2024-03-29T08:07:10.738 INFO:tasks.workunit.client.0.smithi037.stdout:5/287: rmdir d8/d22/d37 39 2024-03-29T08:07:10.738 INFO:tasks.workunit.client.1.smithi148.stdout:9/176: rename db/d10/d2d/d33 to db/d10/d2d/d32/d3b 0 2024-03-29T08:07:10.740 INFO:tasks.workunit.client.0.smithi037.stdout:5/288: write d8/d34/f4a [4075971,85047] 0 2024-03-29T08:07:10.741 INFO:tasks.workunit.client.1.smithi148.stdout:1/177: sync 2024-03-29T08:07:10.742 INFO:tasks.workunit.client.1.smithi148.stdout:6/195: chown d6/d9/d1e/d22/f34 9715 1 2024-03-29T08:07:10.742 INFO:tasks.workunit.client.1.smithi148.stdout:8/240: unlink c8 0 2024-03-29T08:07:10.744 INFO:tasks.workunit.client.1.smithi148.stdout:9/177: stat db/l1f 0 2024-03-29T08:07:10.744 INFO:tasks.workunit.client.1.smithi148.stdout:5/171: dread d0/f9 [0,4194304] 0 2024-03-29T08:07:10.745 INFO:tasks.workunit.client.0.smithi037.stdout:5/289: unlink d8/d22/l46 0 2024-03-29T08:07:10.746 INFO:tasks.workunit.client.1.smithi148.stdout:3/181: mkdir de/d3b 0 2024-03-29T08:07:10.747 INFO:tasks.workunit.client.1.smithi148.stdout:2/204: mkdir d7/d24/d4b 0 2024-03-29T08:07:10.749 INFO:tasks.workunit.client.0.smithi037.stdout:5/290: symlink d8/l5d 0 2024-03-29T08:07:10.749 INFO:tasks.workunit.client.1.smithi148.stdout:7/227: rename d8/l23 to d8/d29/d44/d50/l53 0 2024-03-29T08:07:10.750 INFO:tasks.workunit.client.1.smithi148.stdout:5/172: write d0/db/d1e/f31 [330045,80030] 0 2024-03-29T08:07:10.752 INFO:tasks.workunit.client.1.smithi148.stdout:1/178: creat d3/de/d1d/f33 x:0 0 0 2024-03-29T08:07:10.754 INFO:tasks.workunit.client.0.smithi037.stdout:5/291: link d8/f52 d8/d22/d4f/f5e 0 2024-03-29T08:07:10.754 INFO:tasks.workunit.client.1.smithi148.stdout:3/182: dread de/f2e [0,4194304] 0 2024-03-29T08:07:10.756 INFO:tasks.workunit.client.1.smithi148.stdout:8/241: rename d17/d18/c30 to d17/c50 0 2024-03-29T08:07:10.756 INFO:tasks.workunit.client.1.smithi148.stdout:1/179: write d3/d13/f1a [1083890,86714] 0 2024-03-29T08:07:10.758 INFO:tasks.workunit.client.1.smithi148.stdout:4/170: link df/f1e df/d10/f40 0 2024-03-29T08:07:10.759 INFO:tasks.workunit.client.0.smithi037.stdout:5/292: link d8/d18/l49 d8/d22/d4f/l5f 0 2024-03-29T08:07:10.760 INFO:tasks.workunit.client.1.smithi148.stdout:7/228: creat d8/d29/f54 x:0 0 0 2024-03-29T08:07:10.760 INFO:tasks.workunit.client.1.smithi148.stdout:8/242: write d17/d18/d38/f48 [5206514,83567] 0 2024-03-29T08:07:10.761 INFO:tasks.workunit.client.0.smithi037.stdout:5/293: write d8/d34/d3b/f54 [713387,71450] 0 2024-03-29T08:07:10.762 INFO:tasks.workunit.client.1.smithi148.stdout:8/243: stat d17/d18 0 2024-03-29T08:07:10.764 INFO:tasks.workunit.client.1.smithi148.stdout:2/205: mkdir d7/da/d4c 0 2024-03-29T08:07:10.768 INFO:tasks.workunit.client.1.smithi148.stdout:3/183: chown de/l1d 20 1 2024-03-29T08:07:10.768 INFO:tasks.workunit.client.1.smithi148.stdout:4/171: mknod df/d2d/c41 0 2024-03-29T08:07:10.769 INFO:tasks.workunit.client.1.smithi148.stdout:6/196: dwrite d6/d9/d1e/d22/f34 [0,4194304] 0 2024-03-29T08:07:10.771 INFO:tasks.workunit.client.1.smithi148.stdout:3/184: chown de/f35 30 1 2024-03-29T08:07:10.773 INFO:tasks.workunit.client.1.smithi148.stdout:3/185: readlink de/l28 0 2024-03-29T08:07:10.774 INFO:tasks.workunit.client.1.smithi148.stdout:8/244: rmdir d17/d18/d26 39 2024-03-29T08:07:10.775 INFO:tasks.workunit.client.1.smithi148.stdout:2/206: fdatasync d7/d13/d2e/d37/f3d 0 2024-03-29T08:07:10.778 INFO:tasks.workunit.client.0.smithi037.stdout:5/294: dwrite d8/d34/d3b/f3f [0,4194304] 0 2024-03-29T08:07:10.779 INFO:tasks.workunit.client.1.smithi148.stdout:9/178: getdents db/d20 0 2024-03-29T08:07:10.780 INFO:tasks.workunit.client.1.smithi148.stdout:1/180: dwrite d3/f2d [0,4194304] 0 2024-03-29T08:07:10.780 INFO:tasks.workunit.client.0.smithi037.stdout:5/295: chown d8/d34/d4b/c57 710 1 2024-03-29T08:07:10.782 INFO:tasks.workunit.client.1.smithi148.stdout:4/172: unlink df/d10/f11 0 2024-03-29T08:07:10.786 INFO:tasks.workunit.client.1.smithi148.stdout:3/186: symlink de/l3c 0 2024-03-29T08:07:10.786 INFO:tasks.workunit.client.1.smithi148.stdout:6/197: write f5 [1006463,75817] 0 2024-03-29T08:07:10.790 INFO:tasks.workunit.client.1.smithi148.stdout:5/173: dwrite d0/f2c [0,4194304] 0 2024-03-29T08:07:10.790 INFO:tasks.workunit.client.1.smithi148.stdout:6/198: chown d6/fd 1840481 1 2024-03-29T08:07:10.792 INFO:tasks.workunit.client.1.smithi148.stdout:9/179: creat db/d10/d2d/d32/d3b/f3c x:0 0 0 2024-03-29T08:07:10.794 INFO:tasks.workunit.client.1.smithi148.stdout:9/180: dread - db/d22/d26/f38 zero size 2024-03-29T08:07:10.795 INFO:tasks.workunit.client.1.smithi148.stdout:0/200: link db/dc/l1c db/de/df/d11/l44 0 2024-03-29T08:07:10.797 INFO:tasks.workunit.client.1.smithi148.stdout:1/181: write d3/fa [9545109,42444] 0 2024-03-29T08:07:10.800 INFO:tasks.workunit.client.0.smithi037.stdout:5/296: dwrite d8/d34/d3b/f3f [4194304,4194304] 0 2024-03-29T08:07:10.802 INFO:tasks.workunit.client.1.smithi148.stdout:5/174: rename d0/l15 to d0/d26/l33 0 2024-03-29T08:07:10.804 INFO:tasks.workunit.client.1.smithi148.stdout:6/199: mkdir d6/d9/d1e/d36 0 2024-03-29T08:07:10.805 INFO:tasks.workunit.client.0.smithi037.stdout:5/297: mkdir d8/d22/d37/d60 0 2024-03-29T08:07:10.810 INFO:tasks.workunit.client.1.smithi148.stdout:0/201: creat db/de/df/d11/d2d/d3e/f45 x:0 0 0 2024-03-29T08:07:10.814 INFO:tasks.workunit.client.0.smithi037.stdout:5/298: dread d8/d18/d24/f44 [0,4194304] 0 2024-03-29T08:07:10.814 INFO:tasks.workunit.client.1.smithi148.stdout:1/182: write d3/de/f1b [2152807,114742] 0 2024-03-29T08:07:10.814 INFO:tasks.workunit.client.1.smithi148.stdout:3/187: link de/d24/f2b de/d24/f3d 0 2024-03-29T08:07:10.817 INFO:tasks.workunit.client.0.smithi037.stdout:5/299: write d8/f52 [957853,68961] 0 2024-03-29T08:07:10.819 INFO:tasks.workunit.client.0.smithi037.stdout:5/300: readlink d8/le 0 2024-03-29T08:07:10.822 INFO:tasks.workunit.client.0.smithi037.stdout:5/301: mkdir d8/d22/d37/d60/d61 0 2024-03-29T08:07:10.823 INFO:tasks.workunit.client.1.smithi148.stdout:5/175: creat d0/db/f34 x:0 0 0 2024-03-29T08:07:10.825 INFO:tasks.workunit.client.0.smithi037.stdout:5/302: truncate d8/d22/d4f/f5a 1022665 0 2024-03-29T08:07:10.825 INFO:tasks.workunit.client.1.smithi148.stdout:6/200: creat d6/d9/d1e/d20/f37 x:0 0 0 2024-03-29T08:07:10.826 INFO:tasks.workunit.client.1.smithi148.stdout:0/202: mkdir db/de/df/d2b/d46 0 2024-03-29T08:07:10.827 INFO:tasks.workunit.client.0.smithi037.stdout:5/303: read d8/d18/d24/d45/f51 [3632245,101133] 0 2024-03-29T08:07:10.829 INFO:tasks.workunit.client.1.smithi148.stdout:4/173: dwrite df/d10/d17/f22 [0,4194304] 0 2024-03-29T08:07:10.830 INFO:tasks.workunit.client.0.smithi037.stdout:5/304: write d8/d18/d24/d45/f51 [4532995,85100] 0 2024-03-29T08:07:10.833 INFO:tasks.workunit.client.1.smithi148.stdout:5/176: stat d0/db/de/cf 0 2024-03-29T08:07:10.834 INFO:tasks.workunit.client.1.smithi148.stdout:2/207: dwrite d7/f10 [0,4194304] 0 2024-03-29T08:07:10.838 INFO:tasks.workunit.client.1.smithi148.stdout:2/208: chown d7/da/l44 13 1 2024-03-29T08:07:10.839 INFO:tasks.workunit.client.1.smithi148.stdout:0/203: truncate db/de/df/f30 567859 0 2024-03-29T08:07:10.840 INFO:tasks.workunit.client.1.smithi148.stdout:6/201: truncate d6/d9/d1e/d20/d30/f33 809875 0 2024-03-29T08:07:10.841 INFO:tasks.workunit.client.0.smithi037.stdout:5/305: rmdir d8/d22 39 2024-03-29T08:07:10.841 INFO:tasks.workunit.client.1.smithi148.stdout:9/181: dread db/d10/d2d/f34 [0,4194304] 0 2024-03-29T08:07:10.843 INFO:tasks.workunit.client.1.smithi148.stdout:9/182: write db/d10/f29 [636937,55066] 0 2024-03-29T08:07:10.844 INFO:tasks.workunit.client.1.smithi148.stdout:1/183: dread d3/de/f1f [0,4194304] 0 2024-03-29T08:07:10.848 INFO:tasks.workunit.client.1.smithi148.stdout:9/183: readlink db/d22/d26/l27 0 2024-03-29T08:07:10.848 INFO:tasks.workunit.client.1.smithi148.stdout:5/177: rmdir d0/db/d1e 39 2024-03-29T08:07:10.848 INFO:tasks.workunit.client.1.smithi148.stdout:4/174: creat df/d10/d17/f42 x:0 0 0 2024-03-29T08:07:10.852 INFO:tasks.workunit.client.1.smithi148.stdout:0/204: write fa [2850690,20603] 0 2024-03-29T08:07:10.852 INFO:tasks.workunit.client.1.smithi148.stdout:1/184: creat d3/d13/f34 x:0 0 0 2024-03-29T08:07:10.855 INFO:tasks.workunit.client.1.smithi148.stdout:5/178: truncate d0/db/f34 456824 0 2024-03-29T08:07:10.859 INFO:tasks.workunit.client.1.smithi148.stdout:1/185: mkdir d3/de/d25/d35 0 2024-03-29T08:07:10.859 INFO:tasks.workunit.client.0.smithi037.stdout:5/306: dwrite d8/d22/f31 [0,4194304] 0 2024-03-29T08:07:10.867 INFO:tasks.workunit.client.1.smithi148.stdout:7/229: sync 2024-03-29T08:07:10.871 INFO:tasks.workunit.client.1.smithi148.stdout:2/209: dwrite d7/f17 [0,4194304] 0 2024-03-29T08:07:10.871 INFO:tasks.workunit.client.1.smithi148.stdout:6/202: fsync d6/d9/d1e/d20/d30/f33 0 2024-03-29T08:07:10.871 INFO:tasks.workunit.client.1.smithi148.stdout:1/186: chown d3/cb 8505706 1 2024-03-29T08:07:10.872 INFO:tasks.workunit.client.1.smithi148.stdout:5/179: truncate d0/f2c 608678 0 2024-03-29T08:07:10.876 INFO:tasks.workunit.client.0.smithi037.stdout:5/307: dwrite d8/f1e [0,4194304] 0 2024-03-29T08:07:10.876 INFO:tasks.workunit.client.1.smithi148.stdout:5/180: fsync d0/db/f34 0 2024-03-29T08:07:10.881 INFO:tasks.workunit.client.0.smithi037.stdout:5/308: creat d8/d4d/d5c/f62 x:0 0 0 2024-03-29T08:07:10.882 INFO:tasks.workunit.client.1.smithi148.stdout:7/230: truncate f6 1682299 0 2024-03-29T08:07:10.883 INFO:tasks.workunit.client.1.smithi148.stdout:5/181: chown d0/db/de/l1c 8 1 2024-03-29T08:07:10.886 INFO:tasks.workunit.client.1.smithi148.stdout:2/210: rename d7/d13 to d7/da/d33/d45/d4d 0 2024-03-29T08:07:10.886 INFO:tasks.workunit.client.1.smithi148.stdout:0/205: dwrite db/de/df/d11/d2d/f36 [0,4194304] 0 2024-03-29T08:07:10.887 INFO:tasks.workunit.client.1.smithi148.stdout:9/184: dwrite db/f21 [0,4194304] 0 2024-03-29T08:07:10.887 INFO:tasks.workunit.client.0.smithi037.stdout:5/309: symlink d8/d22/d4f/l63 0 2024-03-29T08:07:10.896 INFO:tasks.workunit.client.1.smithi148.stdout:2/211: read d7/da/f49 [1465431,92411] 0 2024-03-29T08:07:10.896 INFO:tasks.workunit.client.1.smithi148.stdout:9/185: write db/d22/d26/f2a [52406,77841] 0 2024-03-29T08:07:10.898 INFO:tasks.workunit.client.1.smithi148.stdout:5/182: mkdir d0/db/d1e/d35 0 2024-03-29T08:07:10.900 INFO:tasks.workunit.client.1.smithi148.stdout:0/206: fsync db/de/df/d11/f37 0 2024-03-29T08:07:10.902 INFO:tasks.workunit.client.1.smithi148.stdout:6/203: dread d6/d9/d1e/d22/f34 [0,4194304] 0 2024-03-29T08:07:10.905 INFO:tasks.workunit.client.1.smithi148.stdout:2/212: fsync d7/da/d33/d3c/f23 0 2024-03-29T08:07:10.905 INFO:tasks.workunit.client.1.smithi148.stdout:3/188: dread de/f15 [0,4194304] 0 2024-03-29T08:07:10.906 INFO:tasks.workunit.client.0.smithi037.stdout:3/209: sync 2024-03-29T08:07:10.907 INFO:tasks.workunit.client.0.smithi037.stdout:7/234: sync 2024-03-29T08:07:10.907 INFO:tasks.workunit.client.0.smithi037.stdout:1/278: sync 2024-03-29T08:07:10.907 INFO:tasks.workunit.client.1.smithi148.stdout:7/231: getdents d8/d14/d19/d37 0 2024-03-29T08:07:10.908 INFO:tasks.workunit.client.1.smithi148.stdout:3/189: symlink de/l3e 0 2024-03-29T08:07:10.910 INFO:tasks.workunit.client.1.smithi148.stdout:2/213: write d7/f19 [1430537,75531] 0 2024-03-29T08:07:10.910 INFO:tasks.workunit.client.0.smithi037.stdout:5/310: dwrite d8/d34/f4a [4194304,4194304] 0 2024-03-29T08:07:10.911 INFO:tasks.workunit.client.1.smithi148.stdout:1/187: dwrite d3/de/f1f [4194304,4194304] 0 2024-03-29T08:07:10.911 INFO:tasks.workunit.client.0.smithi037.stdout:1/279: write d2/d6/dc/d18/f5f [47883,4194] 0 2024-03-29T08:07:10.912 INFO:tasks.workunit.client.0.smithi037.stdout:7/235: rmdir dd/d21/d35/d42 39 2024-03-29T08:07:10.913 INFO:tasks.workunit.client.0.smithi037.stdout:5/311: chown d8/d4d/d5c 6304433 1 2024-03-29T08:07:10.914 INFO:tasks.workunit.client.1.smithi148.stdout:7/232: unlink d8/d14/f1b 0 2024-03-29T08:07:10.916 INFO:tasks.workunit.client.0.smithi037.stdout:3/210: creat dd/d46/d2d/d3b/f4f x:0 0 0 2024-03-29T08:07:10.918 INFO:tasks.workunit.client.0.smithi037.stdout:5/312: write d8/d34/d3b/f3f [1375736,130566] 0 2024-03-29T08:07:10.921 INFO:tasks.workunit.client.1.smithi148.stdout:3/190: dread de/f15 [4194304,4194304] 0 2024-03-29T08:07:10.925 INFO:tasks.workunit.client.0.smithi037.stdout:1/280: chown d2/d6/d13/c34 7351494 1 2024-03-29T08:07:10.925 INFO:tasks.workunit.client.0.smithi037.stdout:3/211: write dd/d15/f44 [990949,45259] 0 2024-03-29T08:07:10.928 INFO:tasks.workunit.client.0.smithi037.stdout:7/236: symlink dd/d21/d35/d3b/l4e 0 2024-03-29T08:07:10.929 INFO:tasks.workunit.client.1.smithi148.stdout:2/214: symlink d7/da/d4c/l4e 0 2024-03-29T08:07:10.930 INFO:tasks.workunit.client.0.smithi037.stdout:3/212: creat dd/d15/f50 x:0 0 0 2024-03-29T08:07:10.931 INFO:tasks.workunit.client.0.smithi037.stdout:7/237: chown lc 15476 1 2024-03-29T08:07:10.932 INFO:tasks.workunit.client.0.smithi037.stdout:3/213: chown dd/f39 774987 1 2024-03-29T08:07:10.933 INFO:tasks.workunit.client.0.smithi037.stdout:1/281: mkdir d2/d6/dc/d5d/d64 0 2024-03-29T08:07:10.934 INFO:tasks.workunit.client.0.smithi037.stdout:1/282: stat d2/d6/l1f 0 2024-03-29T08:07:10.935 INFO:tasks.workunit.client.0.smithi037.stdout:3/214: fsync dd/d15/d18/f2f 0 2024-03-29T08:07:10.939 INFO:tasks.workunit.client.1.smithi148.stdout:5/183: read d0/f9 [430882,125425] 0 2024-03-29T08:07:10.944 INFO:tasks.workunit.client.0.smithi037.stdout:1/283: rename d2/c54 to d2/d6/dc/d18/d22/d4f/c65 0 2024-03-29T08:07:10.944 INFO:tasks.workunit.client.0.smithi037.stdout:5/313: dread d8/d34/f4a [0,4194304] 0 2024-03-29T08:07:10.946 INFO:tasks.workunit.client.1.smithi148.stdout:2/215: chown d7/d24/f46 1404 1 2024-03-29T08:07:10.948 INFO:tasks.workunit.client.0.smithi037.stdout:5/314: readlink d8/l1b 0 2024-03-29T08:07:10.948 INFO:tasks.workunit.client.0.smithi037.stdout:1/284: write d2/d40/f43 [584112,44652] 0 2024-03-29T08:07:10.949 INFO:tasks.workunit.client.0.smithi037.stdout:3/215: mknod dd/d46/c51 0 2024-03-29T08:07:10.949 INFO:tasks.workunit.client.0.smithi037.stdout:7/238: read dd/f1a [45798,113213] 0 2024-03-29T08:07:10.950 INFO:tasks.workunit.client.1.smithi148.stdout:8/245: sync 2024-03-29T08:07:10.952 INFO:tasks.workunit.client.0.smithi037.stdout:1/285: creat d2/d4b/f66 x:0 0 0 2024-03-29T08:07:10.953 INFO:tasks.workunit.client.1.smithi148.stdout:9/186: dwrite db/d10/d14/f18 [0,4194304] 0 2024-03-29T08:07:10.956 INFO:tasks.workunit.client.1.smithi148.stdout:6/204: truncate d6/f19 2829828 0 2024-03-29T08:07:10.956 INFO:tasks.workunit.client.1.smithi148.stdout:5/184: truncate d0/f9 1446867 0 2024-03-29T08:07:10.956 INFO:tasks.workunit.client.1.smithi148.stdout:0/207: dwrite db/de/df/d11/f41 [0,4194304] 0 2024-03-29T08:07:10.957 INFO:tasks.workunit.client.0.smithi037.stdout:1/286: truncate d2/d4b/f4d 696897 0 2024-03-29T08:07:10.960 INFO:tasks.workunit.client.0.smithi037.stdout:7/239: symlink dd/d1d/d49/l4f 0 2024-03-29T08:07:10.961 INFO:tasks.workunit.client.1.smithi148.stdout:1/188: dwrite d3/f30 [0,4194304] 0 2024-03-29T08:07:10.964 INFO:tasks.workunit.client.0.smithi037.stdout:1/287: chown d2/d4b/c5b 7 1 2024-03-29T08:07:10.964 INFO:tasks.workunit.client.0.smithi037.stdout:3/216: getdents dd/d46/d2d/d3b 0 2024-03-29T08:07:10.965 INFO:tasks.workunit.client.1.smithi148.stdout:9/187: write db/d10/d2d/f31 [581971,122299] 0 2024-03-29T08:07:10.965 INFO:tasks.workunit.client.1.smithi148.stdout:5/185: write d0/db/d1e/f31 [1205553,69478] 0 2024-03-29T08:07:10.966 INFO:tasks.workunit.client.1.smithi148.stdout:6/205: fsync d6/d9/d1e/d20/d30/f35 0 2024-03-29T08:07:10.966 INFO:tasks.workunit.client.1.smithi148.stdout:2/216: symlink d7/d24/d4b/l4f 0 2024-03-29T08:07:10.967 INFO:tasks.workunit.client.1.smithi148.stdout:7/233: dwrite d8/d14/d19/f4b [0,4194304] 0 2024-03-29T08:07:10.967 INFO:tasks.workunit.client.0.smithi037.stdout:7/240: write dd/d21/f36 [836767,92090] 0 2024-03-29T08:07:10.973 INFO:tasks.workunit.client.0.smithi037.stdout:3/217: dread - dd/d46/d2d/d3d/f45 zero size 2024-03-29T08:07:10.973 INFO:tasks.workunit.client.1.smithi148.stdout:2/217: fdatasync d7/f10 0 2024-03-29T08:07:10.973 INFO:tasks.workunit.client.0.smithi037.stdout:1/288: write d2/d4b/f66 [906356,77287] 0 2024-03-29T08:07:10.976 INFO:tasks.workunit.client.0.smithi037.stdout:5/315: dwrite d8/d18/f28 [0,4194304] 0 2024-03-29T08:07:10.977 INFO:tasks.workunit.client.1.smithi148.stdout:7/234: chown d8/d29/f2e 12 1 2024-03-29T08:07:10.980 INFO:tasks.workunit.client.1.smithi148.stdout:2/218: chown d7/f3b 638 1 2024-03-29T08:07:10.982 INFO:tasks.workunit.client.0.smithi037.stdout:1/289: write d2/d6/dc/d18/d1d/d21/d38/f50 [558327,108610] 0 2024-03-29T08:07:10.984 INFO:tasks.workunit.client.0.smithi037.stdout:3/218: write dd/d46/f3c [56678,68427] 0 2024-03-29T08:07:10.985 INFO:tasks.workunit.client.1.smithi148.stdout:4/175: sync 2024-03-29T08:07:10.986 INFO:tasks.workunit.client.0.smithi037.stdout:1/290: dread - d2/d6/dc/d18/f63 zero size 2024-03-29T08:07:10.988 INFO:tasks.workunit.client.0.smithi037.stdout:5/316: mkdir d8/d18/d24/d64 0 2024-03-29T08:07:10.988 INFO:tasks.workunit.client.1.smithi148.stdout:3/191: dwrite de/d24/f27 [0,4194304] 0 2024-03-29T08:07:10.992 INFO:tasks.workunit.client.0.smithi037.stdout:1/291: mkdir d2/d6/dc/d58/d67 0 2024-03-29T08:07:10.995 INFO:tasks.workunit.client.0.smithi037.stdout:3/219: getdents dd/d46 0 2024-03-29T08:07:10.996 INFO:tasks.workunit.client.1.smithi148.stdout:6/206: creat d6/d9/d1e/d22/d28/f38 x:0 0 0 2024-03-29T08:07:10.999 INFO:tasks.workunit.client.0.smithi037.stdout:3/220: dread - dd/d15/f49 zero size 2024-03-29T08:07:11.001 INFO:tasks.workunit.client.1.smithi148.stdout:4/176: readlink df/d10/l39 0 2024-03-29T08:07:11.003 INFO:tasks.workunit.client.0.smithi037.stdout:1/292: link d2/d6/dc/d18/f63 d2/d6/dc/d5d/d64/f68 0 2024-03-29T08:07:11.003 INFO:tasks.workunit.client.1.smithi148.stdout:0/208: dwrite db/de/df/d2b/f35 [0,4194304] 0 2024-03-29T08:07:11.004 INFO:tasks.workunit.client.1.smithi148.stdout:5/186: getdents d0 0 2024-03-29T08:07:11.005 INFO:tasks.workunit.client.0.smithi037.stdout:1/293: chown d2/c1a 50 1 2024-03-29T08:07:11.005 INFO:tasks.workunit.client.1.smithi148.stdout:2/219: dread d7/f19 [0,4194304] 0 2024-03-29T08:07:11.005 INFO:tasks.workunit.client.1.smithi148.stdout:0/209: chown db/de/df/d11/d1f 577420100 1 2024-03-29T08:07:11.007 INFO:tasks.workunit.client.1.smithi148.stdout:0/210: write f7 [1860033,31702] 0 2024-03-29T08:07:11.008 INFO:tasks.workunit.client.0.smithi037.stdout:5/317: dread d8/f17 [4194304,4194304] 0 2024-03-29T08:07:11.008 INFO:tasks.workunit.client.0.smithi037.stdout:3/221: rename dd/d15/d18/f35 to dd/d46/d2d/d3a/f52 0 2024-03-29T08:07:11.008 INFO:tasks.workunit.client.1.smithi148.stdout:2/220: chown d7/f1b 6 1 2024-03-29T08:07:11.013 INFO:tasks.workunit.client.0.smithi037.stdout:5/318: creat d8/d4d/d5c/f65 x:0 0 0 2024-03-29T08:07:11.016 INFO:tasks.workunit.client.0.smithi037.stdout:1/294: rename d2/f57 to d2/d6/dc/d18/d22/d4f/f69 0 2024-03-29T08:07:11.019 INFO:tasks.workunit.client.0.smithi037.stdout:5/319: mknod d8/d34/c66 0 2024-03-29T08:07:11.020 INFO:tasks.workunit.client.1.smithi148.stdout:1/189: link d3/d18/l2e d3/de/d1d/l36 0 2024-03-29T08:07:11.023 INFO:tasks.workunit.client.0.smithi037.stdout:1/295: rename d2/d6 to d2/d6/dc/d18/d1d/d21/d38/d6a 22 2024-03-29T08:07:11.027 INFO:tasks.workunit.client.1.smithi148.stdout:2/221: dread d7/f19 [0,4194304] 0 2024-03-29T08:07:11.028 INFO:tasks.workunit.client.0.smithi037.stdout:3/222: dwrite dd/d15/f19 [0,4194304] 0 2024-03-29T08:07:11.031 INFO:tasks.workunit.client.1.smithi148.stdout:9/188: dread db/d22/d26/f30 [0,4194304] 0 2024-03-29T08:07:11.031 INFO:tasks.workunit.client.1.smithi148.stdout:4/177: mkdir df/d10/d26/d34/d43 0 2024-03-29T08:07:11.033 INFO:tasks.workunit.client.1.smithi148.stdout:5/187: rmdir d0/db/d1e 39 2024-03-29T08:07:11.034 INFO:tasks.workunit.client.1.smithi148.stdout:0/211: dwrite f7 [0,4194304] 0 2024-03-29T08:07:11.036 INFO:tasks.workunit.client.0.smithi037.stdout:3/223: creat dd/d46/d2d/d3a/f53 x:0 0 0 2024-03-29T08:07:11.041 INFO:tasks.workunit.client.1.smithi148.stdout:3/192: dwrite de/f15 [4194304,4194304] 0 2024-03-29T08:07:11.043 INFO:tasks.workunit.client.0.smithi037.stdout:1/296: getdents d2/d6/dc/d18 0 2024-03-29T08:07:11.046 INFO:tasks.workunit.client.1.smithi148.stdout:9/189: truncate db/d10/d2d/d32/d3b/f36 286149 0 2024-03-29T08:07:11.047 INFO:tasks.workunit.client.0.smithi037.stdout:3/224: getdents dd/d32 0 2024-03-29T08:07:11.048 INFO:tasks.workunit.client.1.smithi148.stdout:4/178: symlink df/d10/d26/l44 0 2024-03-29T08:07:11.049 INFO:tasks.workunit.client.0.smithi037.stdout:5/320: dwrite d8/d18/f50 [0,4194304] 0 2024-03-29T08:07:11.050 INFO:tasks.workunit.client.0.smithi037.stdout:0/117: sync 2024-03-29T08:07:11.051 INFO:tasks.workunit.client.0.smithi037.stdout:9/312: sync 2024-03-29T08:07:11.051 INFO:tasks.workunit.client.0.smithi037.stdout:2/132: sync 2024-03-29T08:07:11.051 INFO:tasks.workunit.client.0.smithi037.stdout:6/155: sync 2024-03-29T08:07:11.051 INFO:tasks.workunit.client.0.smithi037.stdout:4/112: sync 2024-03-29T08:07:11.053 INFO:tasks.workunit.client.1.smithi148.stdout:8/246: dread d17/d18/f4d [0,4194304] 0 2024-03-29T08:07:11.054 INFO:tasks.workunit.client.1.smithi148.stdout:1/190: creat d3/de/d21/d32/f37 x:0 0 0 2024-03-29T08:07:11.058 INFO:tasks.workunit.client.1.smithi148.stdout:6/207: rmdir d6/d9/d1e 39 2024-03-29T08:07:11.060 INFO:tasks.workunit.client.1.smithi148.stdout:1/191: write d3/d13/f1a [1083157,16013] 0 2024-03-29T08:07:11.061 INFO:tasks.workunit.client.1.smithi148.stdout:2/222: dwrite d7/f3b [4194304,4194304] 0 2024-03-29T08:07:11.063 INFO:tasks.workunit.client.1.smithi148.stdout:7/235: dread d8/d14/f30 [0,4194304] 0 2024-03-29T08:07:11.063 INFO:tasks.workunit.client.0.smithi037.stdout:5/321: write d8/d34/d3b/f3f [9378550,96570] 0 2024-03-29T08:07:11.063 INFO:tasks.workunit.client.0.smithi037.stdout:9/313: fdatasync d1/d4/d19/d1e/d25/d16/f56 0 2024-03-29T08:07:11.064 INFO:tasks.workunit.client.1.smithi148.stdout:1/192: chown d3/l6 0 1 2024-03-29T08:07:11.069 INFO:tasks.workunit.client.0.smithi037.stdout:3/225: creat dd/d46/d3e/d47/f54 x:0 0 0 2024-03-29T08:07:11.074 INFO:tasks.workunit.client.0.smithi037.stdout:1/297: truncate d2/d6/dc/f15 1651152 0 2024-03-29T08:07:11.074 INFO:tasks.workunit.client.1.smithi148.stdout:9/190: creat db/d10/d14/f3d x:0 0 0 2024-03-29T08:07:11.078 INFO:tasks.workunit.client.1.smithi148.stdout:4/179: rmdir df/d2d 39 2024-03-29T08:07:11.079 INFO:tasks.workunit.client.1.smithi148.stdout:6/208: dread d6/fe [0,4194304] 0 2024-03-29T08:07:11.082 INFO:tasks.workunit.client.1.smithi148.stdout:6/209: fdatasync d6/d9/f23 0 2024-03-29T08:07:11.082 INFO:tasks.workunit.client.1.smithi148.stdout:1/193: dread d3/f2d [0,4194304] 0 2024-03-29T08:07:11.082 INFO:tasks.workunit.client.0.smithi037.stdout:8/197: sync 2024-03-29T08:07:11.083 INFO:tasks.workunit.client.1.smithi148.stdout:1/194: chown d3/f2d 1 1 2024-03-29T08:07:11.085 INFO:tasks.workunit.client.1.smithi148.stdout:1/195: read d3/fa [13087588,120661] 0 2024-03-29T08:07:11.085 INFO:tasks.workunit.client.0.smithi037.stdout:6/156: creat d0/d20/d38/f39 x:0 0 0 2024-03-29T08:07:11.085 INFO:tasks.workunit.client.1.smithi148.stdout:5/188: mkdir d0/db/d1e/d36 0 2024-03-29T08:07:11.086 INFO:tasks.workunit.client.0.smithi037.stdout:8/198: readlink d5/d10/l2b 0 2024-03-29T08:07:11.086 INFO:tasks.workunit.client.1.smithi148.stdout:1/196: dread - d3/d13/f34 zero size 2024-03-29T08:07:11.088 INFO:tasks.workunit.client.1.smithi148.stdout:1/197: write d3/de/f1b [3904519,31094] 0 2024-03-29T08:07:11.088 INFO:tasks.workunit.client.0.smithi037.stdout:0/118: dread f5 [0,4194304] 0 2024-03-29T08:07:11.090 INFO:tasks.workunit.client.0.smithi037.stdout:7/241: dread dd/f1b [0,4194304] 0 2024-03-29T08:07:11.091 INFO:tasks.workunit.client.1.smithi148.stdout:0/212: rename db/de/df/d29 to db/de/df/d2b/d3b/d47 0 2024-03-29T08:07:11.096 INFO:tasks.workunit.client.1.smithi148.stdout:7/236: creat d8/d14/f55 x:0 0 0 2024-03-29T08:07:11.097 INFO:tasks.workunit.client.0.smithi037.stdout:9/314: write d1/d4/d19/d1e/d4a/d5c/d60/d13/f17 [1902307,37997] 0 2024-03-29T08:07:11.101 INFO:tasks.workunit.client.0.smithi037.stdout:1/298: creat d2/d6/dc/d58/f6b x:0 0 0 2024-03-29T08:07:11.102 INFO:tasks.workunit.client.0.smithi037.stdout:8/199: dwrite d5/d25/f28 [0,4194304] 0 2024-03-29T08:07:11.105 INFO:tasks.workunit.client.1.smithi148.stdout:2/223: link d7/da/d33/d45/d4d/f47 d7/da/d33/d45/d4d/d2e/f50 0 2024-03-29T08:07:11.110 INFO:tasks.workunit.client.0.smithi037.stdout:5/322: creat d8/d5b/f67 x:0 0 0 2024-03-29T08:07:11.112 INFO:tasks.workunit.client.0.smithi037.stdout:3/226: link dd/f39 dd/d46/d2d/d3a/f55 0 2024-03-29T08:07:11.114 INFO:tasks.workunit.client.0.smithi037.stdout:3/227: fsync dd/d15/f16 0 2024-03-29T08:07:11.118 INFO:tasks.workunit.client.0.smithi037.stdout:3/228: chown dd/d15/d27/f41 80766685 1 2024-03-29T08:07:11.120 INFO:tasks.workunit.client.1.smithi148.stdout:5/189: getdents d0/db/d1e/d36 0 2024-03-29T08:07:11.121 INFO:tasks.workunit.client.1.smithi148.stdout:4/180: creat df/f45 x:0 0 0 2024-03-29T08:07:11.122 INFO:tasks.workunit.client.1.smithi148.stdout:4/181: write fa [2861482,38727] 0 2024-03-29T08:07:11.124 INFO:tasks.workunit.client.0.smithi037.stdout:2/133: truncate d0/fd 15685 0 2024-03-29T08:07:11.124 INFO:tasks.workunit.client.0.smithi037.stdout:9/315: dwrite d1/d4/d19/d1e/d4a/d5c/d60/f12 [0,4194304] 0 2024-03-29T08:07:11.127 INFO:tasks.workunit.client.0.smithi037.stdout:6/157: symlink d0/l3a 0 2024-03-29T08:07:11.127 INFO:tasks.workunit.client.0.smithi037.stdout:3/229: chown fc 133129891 1 2024-03-29T08:07:11.129 INFO:tasks.workunit.client.0.smithi037.stdout:8/200: mkdir d5/d10/d19/d3d 0 2024-03-29T08:07:11.134 INFO:tasks.workunit.client.0.smithi037.stdout:1/299: dwrite d2/d6/dc/d18/d1d/d21/f60 [0,4194304] 0 2024-03-29T08:07:11.135 INFO:tasks.workunit.client.0.smithi037.stdout:5/323: rmdir d8/d18/d24 39 2024-03-29T08:07:11.135 INFO:tasks.workunit.client.1.smithi148.stdout:3/193: truncate de/f15 1716077 0 2024-03-29T08:07:11.136 INFO:tasks.workunit.client.1.smithi148.stdout:9/191: getdents db/d10/d2d/d32 0 2024-03-29T08:07:11.136 INFO:tasks.workunit.client.1.smithi148.stdout:3/194: readlink de/lf 0 2024-03-29T08:07:11.137 INFO:tasks.workunit.client.1.smithi148.stdout:4/182: dwrite df/d10/d17/f42 [0,4194304] 0 2024-03-29T08:07:11.139 INFO:tasks.workunit.client.0.smithi037.stdout:0/119: creat d9/da/f25 x:0 0 0 2024-03-29T08:07:11.142 INFO:tasks.workunit.client.1.smithi148.stdout:2/224: write d7/da/d33/d3c/d21/f25 [1666850,42662] 0 2024-03-29T08:07:11.143 INFO:tasks.workunit.client.0.smithi037.stdout:3/230: stat dd/d15/f1e 0 2024-03-29T08:07:11.143 INFO:tasks.workunit.client.0.smithi037.stdout:7/242: creat dd/d21/d35/f50 x:0 0 0 2024-03-29T08:07:11.146 INFO:tasks.workunit.client.0.smithi037.stdout:5/324: fdatasync d8/d22/f2d 0 2024-03-29T08:07:11.147 INFO:tasks.workunit.client.1.smithi148.stdout:5/190: creat d0/db/d1e/f37 x:0 0 0 2024-03-29T08:07:11.148 INFO:tasks.workunit.client.0.smithi037.stdout:8/201: creat d5/d10/d2d/f3e x:0 0 0 2024-03-29T08:07:11.150 INFO:tasks.workunit.client.1.smithi148.stdout:8/247: getdents d17/d18/d26 0 2024-03-29T08:07:11.151 INFO:tasks.workunit.client.0.smithi037.stdout:4/113: dwrite f0 [4194304,4194304] 0 2024-03-29T08:07:11.155 INFO:tasks.workunit.client.0.smithi037.stdout:8/202: dread - d5/f1f zero size 2024-03-29T08:07:11.156 INFO:tasks.workunit.client.1.smithi148.stdout:7/237: rename d8/d14/l25 to d8/l56 0 2024-03-29T08:07:11.156 INFO:tasks.workunit.client.1.smithi148.stdout:9/192: symlink db/d22/l3e 0 2024-03-29T08:07:11.157 INFO:tasks.workunit.client.1.smithi148.stdout:9/193: fsync db/d22/d26/f38 0 2024-03-29T08:07:11.160 INFO:tasks.workunit.client.1.smithi148.stdout:9/194: truncate db/d20/f1e 307271 0 2024-03-29T08:07:11.160 INFO:tasks.workunit.client.1.smithi148.stdout:3/195: creat de/d13/d30/f3f x:0 0 0 2024-03-29T08:07:11.160 INFO:tasks.workunit.client.1.smithi148.stdout:4/183: creat df/d2c/f46 x:0 0 0 2024-03-29T08:07:11.169 INFO:tasks.workunit.client.0.smithi037.stdout:3/231: mknod dd/d46/d2d/d3a/c56 0 2024-03-29T08:07:11.171 INFO:tasks.workunit.client.1.smithi148.stdout:9/195: dread db/d10/d14/f18 [0,4194304] 0 2024-03-29T08:07:11.173 INFO:tasks.workunit.client.1.smithi148.stdout:5/191: symlink d0/d26/l38 0 2024-03-29T08:07:11.174 INFO:tasks.workunit.client.0.smithi037.stdout:5/325: dread d8/f2b [0,4194304] 0 2024-03-29T08:07:11.175 INFO:tasks.workunit.client.1.smithi148.stdout:7/238: truncate d8/d14/f21 646288 0 2024-03-29T08:07:11.175 INFO:tasks.workunit.client.1.smithi148.stdout:5/192: fsync d0/db/de/d16/f2f 0 2024-03-29T08:07:11.175 INFO:tasks.workunit.client.1.smithi148.stdout:7/239: read - d8/d38/f43 zero size 2024-03-29T08:07:11.177 INFO:tasks.workunit.client.0.smithi037.stdout:2/134: creat d0/f2b x:0 0 0 2024-03-29T08:07:11.182 INFO:tasks.workunit.client.0.smithi037.stdout:8/203: fdatasync d5/f1b 0 2024-03-29T08:07:11.183 INFO:tasks.workunit.client.1.smithi148.stdout:3/196: creat de/d13/f40 x:0 0 0 2024-03-29T08:07:11.183 INFO:tasks.workunit.client.0.smithi037.stdout:7/243: mknod dd/c51 0 2024-03-29T08:07:11.184 INFO:tasks.workunit.client.0.smithi037.stdout:6/158: truncate d0/d5/fa 776634 0 2024-03-29T08:07:11.185 INFO:tasks.workunit.client.0.smithi037.stdout:9/316: write d1/d4/d19/d1e/d4a/d5c/d60/d31/f48 [2753936,90665] 0 2024-03-29T08:07:11.186 INFO:tasks.workunit.client.0.smithi037.stdout:0/120: dwrite f5 [0,4194304] 0 2024-03-29T08:07:11.186 INFO:tasks.workunit.client.0.smithi037.stdout:3/232: symlink dd/d15/l57 0 2024-03-29T08:07:11.187 INFO:tasks.workunit.client.1.smithi148.stdout:4/184: creat df/d10/d26/f47 x:0 0 0 2024-03-29T08:07:11.187 INFO:tasks.workunit.client.1.smithi148.stdout:0/213: creat db/de/df/d11/d42/f48 x:0 0 0 2024-03-29T08:07:11.189 INFO:tasks.workunit.client.1.smithi148.stdout:9/196: mkdir db/d22/d3f 0 2024-03-29T08:07:11.189 INFO:tasks.workunit.client.0.smithi037.stdout:3/233: chown dd/f24 3078691 1 2024-03-29T08:07:11.190 INFO:tasks.workunit.client.1.smithi148.stdout:2/225: write d7/da/d33/d45/d4d/f47 [888856,123286] 0 2024-03-29T08:07:11.191 INFO:tasks.workunit.client.1.smithi148.stdout:8/248: mknod d17/d4e/c51 0 2024-03-29T08:07:11.191 INFO:tasks.workunit.client.1.smithi148.stdout:2/226: readlink d7/da/d33/d3c/d21/l2f 0 2024-03-29T08:07:11.192 INFO:tasks.workunit.client.0.smithi037.stdout:2/135: chown d0/c5 0 1 2024-03-29T08:07:11.192 INFO:tasks.workunit.client.0.smithi037.stdout:5/326: creat d8/d22/d4f/f68 x:0 0 0 2024-03-29T08:07:11.192 INFO:tasks.workunit.client.1.smithi148.stdout:5/193: fdatasync d0/f9 0 2024-03-29T08:07:11.193 INFO:tasks.workunit.client.1.smithi148.stdout:7/240: readlink d8/l56 0 2024-03-29T08:07:11.195 INFO:tasks.workunit.client.0.smithi037.stdout:1/300: link d2/d6/dc/f35 d2/f6c 0 2024-03-29T08:07:11.196 INFO:tasks.workunit.client.0.smithi037.stdout:4/114: mkdir d3/d4/d27 0 2024-03-29T08:07:11.199 INFO:tasks.workunit.client.1.smithi148.stdout:4/185: unlink df/d10/f12 0 2024-03-29T08:07:11.200 INFO:tasks.workunit.client.0.smithi037.stdout:6/159: mknod d0/d19/c3b 0 2024-03-29T08:07:11.201 INFO:tasks.workunit.client.0.smithi037.stdout:1/301: chown d2/d6/dc/d58/d67 133 1 2024-03-29T08:07:11.201 INFO:tasks.workunit.client.1.smithi148.stdout:7/241: dread d8/d14/f16 [0,4194304] 0 2024-03-29T08:07:11.202 INFO:tasks.workunit.client.0.smithi037.stdout:7/244: symlink dd/d21/d35/d3b/l52 0 2024-03-29T08:07:11.202 INFO:tasks.workunit.client.1.smithi148.stdout:8/249: rename d17/d18/f34 to d17/d49/f52 0 2024-03-29T08:07:11.207 INFO:tasks.workunit.client.0.smithi037.stdout:9/317: mkdir d1/d4/d19/d1e/d4a/d5c/d60/d68 0 2024-03-29T08:07:11.207 INFO:tasks.workunit.client.0.smithi037.stdout:1/302: readlink d2/d6/l1f 0 2024-03-29T08:07:11.207 INFO:tasks.workunit.client.1.smithi148.stdout:3/197: mkdir de/d3b/d41 0 2024-03-29T08:07:11.207 INFO:tasks.workunit.client.1.smithi148.stdout:8/250: stat d17/d18/c37 0 2024-03-29T08:07:11.207 INFO:tasks.workunit.client.1.smithi148.stdout:2/227: dread d7/da/d33/d45/d4d/f16 [0,4194304] 0 2024-03-29T08:07:11.207 INFO:tasks.workunit.client.1.smithi148.stdout:3/198: fdatasync de/d13/f23 0 2024-03-29T08:07:11.211 INFO:tasks.workunit.client.0.smithi037.stdout:6/160: fsync d0/d5/d1a/f2f 0 2024-03-29T08:07:11.211 INFO:tasks.workunit.client.1.smithi148.stdout:3/199: chown de/d13/f23 13977 1 2024-03-29T08:07:11.212 INFO:tasks.workunit.client.0.smithi037.stdout:2/136: mknod d0/c2c 0 2024-03-29T08:07:11.212 INFO:tasks.workunit.client.0.smithi037.stdout:5/327: read d8/d18/d24/f44 [1219835,75131] 0 2024-03-29T08:07:11.213 INFO:tasks.workunit.client.1.smithi148.stdout:3/200: stat de/d13/f21 0 2024-03-29T08:07:11.214 INFO:tasks.workunit.client.0.smithi037.stdout:4/115: symlink d3/d4/d27/l28 0 2024-03-29T08:07:11.216 INFO:tasks.workunit.client.1.smithi148.stdout:5/194: dwrite d0/f9 [0,4194304] 0 2024-03-29T08:07:11.216 INFO:tasks.workunit.client.1.smithi148.stdout:8/251: symlink d17/d18/d38/l53 0 2024-03-29T08:07:11.219 INFO:tasks.workunit.client.1.smithi148.stdout:0/214: dread db/de/df/d2b/f35 [0,4194304] 0 2024-03-29T08:07:11.224 INFO:tasks.workunit.client.0.smithi037.stdout:3/234: dwrite dd/f3f [0,4194304] 0 2024-03-29T08:07:11.224 INFO:tasks.workunit.client.0.smithi037.stdout:8/204: dread d5/d25/f28 [0,4194304] 0 2024-03-29T08:07:11.224 INFO:tasks.workunit.client.1.smithi148.stdout:9/197: getdents db/d10/d2d/d32/d3b 0 2024-03-29T08:07:11.229 INFO:tasks.workunit.client.0.smithi037.stdout:1/303: symlink d2/d6/dc/d5d/d64/l6d 0 2024-03-29T08:07:11.231 INFO:tasks.workunit.client.0.smithi037.stdout:5/328: fsync d8/d5b/f67 0 2024-03-29T08:07:11.234 INFO:tasks.workunit.client.1.smithi148.stdout:0/215: dwrite f7 [0,4194304] 0 2024-03-29T08:07:11.235 INFO:tasks.workunit.client.1.smithi148.stdout:2/228: dwrite d7/d24/f2c [0,4194304] 0 2024-03-29T08:07:11.236 INFO:tasks.workunit.client.0.smithi037.stdout:8/205: mknod d5/c3f 0 2024-03-29T08:07:11.237 INFO:tasks.workunit.client.1.smithi148.stdout:2/229: stat d7/f19 0 2024-03-29T08:07:11.242 INFO:tasks.workunit.client.0.smithi037.stdout:1/304: creat d2/d6/dc/d18/d22/f6e x:0 0 0 2024-03-29T08:07:11.242 INFO:tasks.workunit.client.1.smithi148.stdout:6/210: sync 2024-03-29T08:07:11.242 INFO:tasks.workunit.client.1.smithi148.stdout:1/198: sync 2024-03-29T08:07:11.244 INFO:tasks.workunit.client.1.smithi148.stdout:6/211: chown d6/l21 17 1 2024-03-29T08:07:11.245 INFO:tasks.workunit.client.1.smithi148.stdout:1/199: chown d3/de/f1f 60 1 2024-03-29T08:07:11.246 INFO:tasks.workunit.client.1.smithi148.stdout:8/252: creat d17/d18/d38/f54 x:0 0 0 2024-03-29T08:07:11.257 INFO:tasks.workunit.client.1.smithi148.stdout:7/242: rename d8/d38/d3c/c40 to d8/c57 0 2024-03-29T08:07:11.259 INFO:tasks.workunit.client.0.smithi037.stdout:5/329: rename d8/d22/f2d to d8/d22/d37/d60/d61/f69 0 2024-03-29T08:07:11.262 INFO:tasks.workunit.client.1.smithi148.stdout:7/243: dread d8/fe [4194304,4194304] 0 2024-03-29T08:07:11.264 INFO:tasks.workunit.client.0.smithi037.stdout:4/116: truncate f0 4091897 0 2024-03-29T08:07:11.264 INFO:tasks.workunit.client.1.smithi148.stdout:7/244: readlink d8/d29/d44/d49/l4e 0 2024-03-29T08:07:11.264 INFO:tasks.workunit.client.0.smithi037.stdout:8/206: creat d5/d10/d2d/f40 x:0 0 0 2024-03-29T08:07:11.265 INFO:tasks.workunit.client.0.smithi037.stdout:1/305: symlink d2/d6/dc/d18/d1d/d5e/l6f 0 2024-03-29T08:07:11.267 INFO:tasks.workunit.client.1.smithi148.stdout:2/230: dread d7/da/d33/d45/d4d/f2a [0,4194304] 0 2024-03-29T08:07:11.271 INFO:tasks.workunit.client.1.smithi148.stdout:6/212: chown d6/d9/d1e/d22/f31 22143269 1 2024-03-29T08:07:11.273 INFO:tasks.workunit.client.0.smithi037.stdout:0/121: dread d9/da/f11 [0,4194304] 0 2024-03-29T08:07:11.273 INFO:tasks.workunit.client.0.smithi037.stdout:5/330: truncate d8/d5b/f67 896470 0 2024-03-29T08:07:11.275 INFO:tasks.workunit.client.0.smithi037.stdout:6/161: dwrite d0/d20/f31 [0,4194304] 0 2024-03-29T08:07:11.275 INFO:tasks.workunit.client.1.smithi148.stdout:5/195: creat d0/db/de/d16/d2e/f39 x:0 0 0 2024-03-29T08:07:11.276 INFO:tasks.workunit.client.0.smithi037.stdout:2/137: dwrite d0/f2b [0,4194304] 0 2024-03-29T08:07:11.276 INFO:tasks.workunit.client.0.smithi037.stdout:5/331: stat d8/d22/d37/d60 0 2024-03-29T08:07:11.277 INFO:tasks.workunit.client.1.smithi148.stdout:2/231: dread d7/da/d33/f35 [4194304,4194304] 0 2024-03-29T08:07:11.277 INFO:tasks.workunit.client.1.smithi148.stdout:2/232: readlink d7/da/l3a 0 2024-03-29T08:07:11.280 INFO:tasks.workunit.client.0.smithi037.stdout:1/306: fsync d2/d6/dc/d18/f5f 0 2024-03-29T08:07:11.281 INFO:tasks.workunit.client.0.smithi037.stdout:8/207: chown d5/f1f 1446948 1 2024-03-29T08:07:11.281 INFO:tasks.workunit.client.1.smithi148.stdout:8/253: creat d17/d18/d26/d3d/f55 x:0 0 0 2024-03-29T08:07:11.284 INFO:tasks.workunit.client.1.smithi148.stdout:4/186: link df/d10/l39 df/l48 0 2024-03-29T08:07:11.288 INFO:tasks.workunit.client.1.smithi148.stdout:0/216: unlink db/de/df/c39 0 2024-03-29T08:07:11.289 INFO:tasks.workunit.client.0.smithi037.stdout:3/235: dread f4 [0,4194304] 0 2024-03-29T08:07:11.295 INFO:tasks.workunit.client.0.smithi037.stdout:9/318: dwrite d1/d4/d19/d1e/d4a/f62 [0,4194304] 0 2024-03-29T08:07:11.296 INFO:tasks.workunit.client.0.smithi037.stdout:5/332: stat d8/d34/d3b/c3e 0 2024-03-29T08:07:11.296 INFO:tasks.workunit.client.1.smithi148.stdout:9/198: dwrite db/d10/d2d/d32/d3b/f2e [0,4194304] 0 2024-03-29T08:07:11.297 INFO:tasks.workunit.client.0.smithi037.stdout:7/245: dread dd/d21/f2c [0,4194304] 0 2024-03-29T08:07:11.297 INFO:tasks.workunit.client.0.smithi037.stdout:1/307: dread - d2/d6/dc/d18/d1d/d21/f55 zero size 2024-03-29T08:07:11.299 INFO:tasks.workunit.client.1.smithi148.stdout:4/187: dwrite fa [0,4194304] 0 2024-03-29T08:07:11.300 INFO:tasks.workunit.client.1.smithi148.stdout:3/201: truncate de/d24/f27 2917329 0 2024-03-29T08:07:11.302 INFO:tasks.workunit.client.1.smithi148.stdout:6/213: write d6/fd [2322537,9703] 0 2024-03-29T08:07:11.305 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:07:10 smithi148 ceph-mon[44992]: pgmap v37: 65 pgs: 65 active+clean; 666 MiB data, 3.6 GiB used, 533 GiB / 536 GiB avail; 1.8 MiB/s rd, 73 MiB/s wr, 283 op/s 2024-03-29T08:07:11.307 INFO:tasks.workunit.client.0.smithi037.stdout:2/138: symlink d0/d23/l2d 0 2024-03-29T08:07:11.310 INFO:tasks.workunit.client.1.smithi148.stdout:5/196: mknod d0/db/de/d16/c3a 0 2024-03-29T08:07:11.312 INFO:tasks.workunit.client.1.smithi148.stdout:2/233: mknod d7/da/d4c/c51 0 2024-03-29T08:07:11.314 INFO:tasks.workunit.client.0.smithi037.stdout:8/208: symlink d5/d10/d19/l41 0 2024-03-29T08:07:11.316 INFO:tasks.workunit.client.0.smithi037.stdout:3/236: symlink dd/d46/d2d/l58 0 2024-03-29T08:07:11.322 INFO:tasks.workunit.client.1.smithi148.stdout:9/199: mknod db/d22/d26/c40 0 2024-03-29T08:07:11.325 INFO:tasks.workunit.client.0.smithi037.stdout:0/122: link c4 d9/c26 0 2024-03-29T08:07:11.327 INFO:tasks.workunit.client.1.smithi148.stdout:6/214: rename d6/d9/d1e/d20/f24 to d6/d9/d1e/d20/f39 0 2024-03-29T08:07:11.328 INFO:tasks.workunit.client.1.smithi148.stdout:1/200: link d3/d13/f15 d3/d13/f38 0 2024-03-29T08:07:11.328 INFO:tasks.workunit.client.1.smithi148.stdout:8/254: symlink d17/d18/l56 0 2024-03-29T08:07:11.330 INFO:tasks.workunit.client.1.smithi148.stdout:1/201: fdatasync d3/d13/f34 0 2024-03-29T08:07:11.330 INFO:tasks.workunit.client.1.smithi148.stdout:8/255: readlink d17/d18/d38/d39/l40 0 2024-03-29T08:07:11.331 INFO:tasks.workunit.client.1.smithi148.stdout:0/217: creat db/de/df/d2b/d3b/f49 x:0 0 0 2024-03-29T08:07:11.331 INFO:tasks.workunit.client.0.smithi037.stdout:9/319: rename d1/d4/d19/d1e/d4a/d5c/d60/d4b to d1/d4/d19/d69 0 2024-03-29T08:07:11.333 INFO:tasks.workunit.client.1.smithi148.stdout:4/188: rename df/f28 to df/d2d/f49 0 2024-03-29T08:07:11.333 INFO:tasks.workunit.client.1.smithi148.stdout:1/202: chown d3/de/d21/d32/f37 14226401 1 2024-03-29T08:07:11.333 INFO:tasks.workunit.client.0.smithi037.stdout:2/139: mkdir d0/d20/d2e 0 2024-03-29T08:07:11.339 INFO:tasks.workunit.client.1.smithi148.stdout:7/245: getdents d8 0 2024-03-29T08:07:11.339 INFO:tasks.workunit.client.1.smithi148.stdout:5/197: creat d0/db/de/f3b x:0 0 0 2024-03-29T08:07:11.339 INFO:tasks.workunit.client.1.smithi148.stdout:1/203: creat d3/d18/f39 x:0 0 0 2024-03-29T08:07:11.340 INFO:tasks.workunit.client.1.smithi148.stdout:4/189: creat df/d10/d26/f4a x:0 0 0 2024-03-29T08:07:11.341 INFO:tasks.workunit.client.0.smithi037.stdout:6/162: dwrite d0/f37 [0,4194304] 0 2024-03-29T08:07:11.341 INFO:tasks.workunit.client.1.smithi148.stdout:7/246: dread - d8/d14/f55 zero size 2024-03-29T08:07:11.341 INFO:tasks.workunit.client.1.smithi148.stdout:1/204: readlink d3/l6 0 2024-03-29T08:07:11.342 INFO:tasks.workunit.client.1.smithi148.stdout:5/198: fsync d0/db/f34 0 2024-03-29T08:07:11.344 INFO:tasks.workunit.client.0.smithi037.stdout:9/320: write d1/d3/f21 [1682740,87170] 0 2024-03-29T08:07:11.345 INFO:tasks.workunit.client.1.smithi148.stdout:1/205: write d3/de/d21/d32/f37 [674840,46077] 0 2024-03-29T08:07:11.348 INFO:tasks.workunit.client.1.smithi148.stdout:4/190: symlink df/l4b 0 2024-03-29T08:07:11.349 INFO:tasks.workunit.client.1.smithi148.stdout:7/247: mkdir d8/d14/d45/d58 0 2024-03-29T08:07:11.350 INFO:tasks.workunit.client.0.smithi037.stdout:2/140: chown d0/fd 0 1 2024-03-29T08:07:11.351 INFO:tasks.workunit.client.0.smithi037.stdout:6/163: write d0/f4 [2055085,78559] 0 2024-03-29T08:07:11.351 INFO:tasks.workunit.client.0.smithi037.stdout:4/117: dread d3/d4/d15/d1d/db/f1a [0,4194304] 0 2024-03-29T08:07:11.352 INFO:tasks.workunit.client.1.smithi148.stdout:8/256: dread d17/d18/f4d [0,4194304] 0 2024-03-29T08:07:11.353 INFO:tasks.workunit.client.1.smithi148.stdout:5/199: symlink d0/db/d1e/d35/l3c 0 2024-03-29T08:07:11.355 INFO:tasks.workunit.client.0.smithi037.stdout:5/333: dwrite d8/d18/f40 [0,4194304] 0 2024-03-29T08:07:11.358 INFO:tasks.workunit.client.0.smithi037.stdout:9/321: chown d1/d4/d19/d1e/d25 3571069 1 2024-03-29T08:07:11.360 INFO:tasks.workunit.client.1.smithi148.stdout:1/206: unlink d3/de/d1d/l36 0 2024-03-29T08:07:11.361 INFO:tasks.workunit.client.1.smithi148.stdout:4/191: mknod df/d10/d26/d34/d3e/c4c 0 2024-03-29T08:07:11.363 INFO:tasks.workunit.client.1.smithi148.stdout:4/192: chown fc 19728 1 2024-03-29T08:07:11.364 INFO:tasks.workunit.client.1.smithi148.stdout:0/218: dwrite db/de/df/d11/f16 [0,4194304] 0 2024-03-29T08:07:11.364 INFO:tasks.workunit.client.1.smithi148.stdout:3/202: write de/d24/f2b [1938472,3748] 0 2024-03-29T08:07:11.364 INFO:tasks.workunit.client.0.smithi037.stdout:5/334: write d8/d18/f30 [576647,128517] 0 2024-03-29T08:07:11.366 INFO:tasks.workunit.client.0.smithi037.stdout:2/141: stat d0/fa 0 2024-03-29T08:07:11.368 INFO:tasks.workunit.client.0.smithi037.stdout:4/118: write d3/d4/d15/f1f [308904,122005] 0 2024-03-29T08:07:11.369 INFO:tasks.workunit.client.1.smithi148.stdout:5/200: creat d0/db/de/d16/f3d x:0 0 0 2024-03-29T08:07:11.369 INFO:tasks.workunit.client.1.smithi148.stdout:1/207: rename d3/c2c to d3/de/d25/d31/c3a 0 2024-03-29T08:07:11.371 INFO:tasks.workunit.client.0.smithi037.stdout:1/308: write d2/d6/dc/f35 [2179226,60696] 0 2024-03-29T08:07:11.372 INFO:tasks.workunit.client.0.smithi037.stdout:9/322: write d1/d4/d19/d1e/d4a/d5c/d60/d23/f37 [542610,3797] 0 2024-03-29T08:07:11.378 INFO:tasks.workunit.client.0.smithi037.stdout:6/164: creat d0/d5/d1f/f3c x:0 0 0 2024-03-29T08:07:11.382 INFO:tasks.workunit.client.0.smithi037.stdout:3/237: dwrite dd/d46/d2d/d3d/f45 [0,4194304] 0 2024-03-29T08:07:11.385 INFO:tasks.workunit.client.1.smithi148.stdout:2/234: dread d7/da/d33/d3c/f28 [0,4194304] 0 2024-03-29T08:07:11.386 INFO:tasks.workunit.client.1.smithi148.stdout:4/193: creat df/f4d x:0 0 0 2024-03-29T08:07:11.388 INFO:tasks.workunit.client.0.smithi037.stdout:6/165: chown d0/d5/d1e/l2e 3 1 2024-03-29T08:07:11.388 INFO:tasks.workunit.client.0.smithi037.stdout:3/238: dread - dd/d15/f49 zero size 2024-03-29T08:07:11.389 INFO:tasks.workunit.client.1.smithi148.stdout:8/257: getdents d17/d18 0 2024-03-29T08:07:11.390 INFO:tasks.workunit.client.1.smithi148.stdout:0/219: dwrite f7 [0,4194304] 0 2024-03-29T08:07:11.390 INFO:tasks.workunit.client.0.smithi037.stdout:5/335: creat d8/d34/f6a x:0 0 0 2024-03-29T08:07:11.393 INFO:tasks.workunit.client.1.smithi148.stdout:4/194: write df/d10/d17/f42 [2118763,99323] 0 2024-03-29T08:07:11.393 INFO:tasks.workunit.client.0.smithi037.stdout:8/209: dwrite d5/d10/f1d [4194304,4194304] 0 2024-03-29T08:07:11.395 INFO:tasks.workunit.client.1.smithi148.stdout:3/203: creat de/d24/f42 x:0 0 0 2024-03-29T08:07:11.395 INFO:tasks.workunit.client.0.smithi037.stdout:3/239: chown dd/d15/f1e 787723002 1 2024-03-29T08:07:11.395 INFO:tasks.workunit.client.0.smithi037.stdout:7/246: truncate dd/d1d/d34/f40 415486 0 2024-03-29T08:07:11.395 INFO:tasks.workunit.client.0.smithi037.stdout:4/119: rmdir d3/d11/d1e 39 2024-03-29T08:07:11.399 INFO:tasks.workunit.client.1.smithi148.stdout:8/258: mknod d17/d18/d26/c57 0 2024-03-29T08:07:11.402 INFO:tasks.workunit.client.0.smithi037.stdout:8/210: read d5/d25/f28 [3568861,129976] 0 2024-03-29T08:07:11.402 INFO:tasks.workunit.client.1.smithi148.stdout:8/259: chown fe 57934 1 2024-03-29T08:07:11.403 INFO:tasks.workunit.client.1.smithi148.stdout:0/220: dread - db/de/df/d14/f3c zero size 2024-03-29T08:07:11.404 INFO:tasks.workunit.client.1.smithi148.stdout:7/248: dread d8/d14/d19/d37/f51 [0,4194304] 0 2024-03-29T08:07:11.405 INFO:tasks.workunit.client.0.smithi037.stdout:2/142: rename d0/c2c to d0/d20/c2f 0 2024-03-29T08:07:11.405 INFO:tasks.workunit.client.1.smithi148.stdout:3/204: read de/d24/f3d [221629,16062] 0 2024-03-29T08:07:11.405 INFO:tasks.workunit.client.1.smithi148.stdout:7/249: readlink d8/l24 0 2024-03-29T08:07:11.406 INFO:tasks.workunit.client.1.smithi148.stdout:2/235: mkdir d7/da/d52 0 2024-03-29T08:07:11.406 INFO:tasks.workunit.client.1.smithi148.stdout:7/250: fdatasync d8/f26 0 2024-03-29T08:07:11.408 INFO:tasks.workunit.client.1.smithi148.stdout:2/236: stat d7/da/d33/d45/d4d/d2e/d37 0 2024-03-29T08:07:11.408 INFO:tasks.workunit.client.1.smithi148.stdout:4/195: unlink df/d10/d17/f22 0 2024-03-29T08:07:11.409 INFO:tasks.workunit.client.0.smithi037.stdout:0/123: truncate f5 1450375 0 2024-03-29T08:07:11.410 INFO:tasks.workunit.client.0.smithi037.stdout:1/309: write d2/d6/dc/d18/d22/f48 [1369675,27260] 0 2024-03-29T08:07:11.412 INFO:tasks.workunit.client.1.smithi148.stdout:2/237: write d7/da/d33/d3c/d21/f25 [5072681,112102] 0 2024-03-29T08:07:11.414 INFO:tasks.workunit.client.0.smithi037.stdout:0/124: truncate d9/da/f25 687458 0 2024-03-29T08:07:11.419 INFO:tasks.workunit.client.1.smithi148.stdout:3/205: symlink de/d3b/l43 0 2024-03-29T08:07:11.422 INFO:tasks.workunit.client.0.smithi037.stdout:7/247: chown dd/d21/c3a 29269577 1 2024-03-29T08:07:11.423 INFO:tasks.workunit.client.0.smithi037.stdout:6/166: dread d0/d5/fb [0,4194304] 0 2024-03-29T08:07:11.423 INFO:tasks.workunit.client.1.smithi148.stdout:3/206: write de/d24/f42 [101459,62877] 0 2024-03-29T08:07:11.423 INFO:tasks.workunit.client.1.smithi148.stdout:8/260: dwrite f14 [0,4194304] 0 2024-03-29T08:07:11.424 INFO:tasks.workunit.client.0.smithi037.stdout:8/211: fdatasync d5/d25/f28 0 2024-03-29T08:07:11.426 INFO:tasks.workunit.client.0.smithi037.stdout:9/323: dread d1/d4/d19/d1e/d4a/d5c/d60/d31/f48 [0,4194304] 0 2024-03-29T08:07:11.428 INFO:tasks.workunit.client.1.smithi148.stdout:3/207: dread - de/d13/f40 zero size 2024-03-29T08:07:11.428 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:07:10 smithi037 ceph-mon[35162]: pgmap v37: 65 pgs: 65 active+clean; 666 MiB data, 3.6 GiB used, 533 GiB / 536 GiB avail; 1.8 MiB/s rd, 73 MiB/s wr, 283 op/s 2024-03-29T08:07:11.428 INFO:tasks.workunit.client.0.smithi037.stdout:8/212: readlink d5/l29 0 2024-03-29T08:07:11.428 INFO:tasks.workunit.client.1.smithi148.stdout:7/251: creat d8/d29/d44/d50/f59 x:0 0 0 2024-03-29T08:07:11.429 INFO:tasks.workunit.client.0.smithi037.stdout:6/167: chown d0/f4 800 1 2024-03-29T08:07:11.430 INFO:tasks.workunit.client.0.smithi037.stdout:4/120: creat d3/d4/d15/f29 x:0 0 0 2024-03-29T08:07:11.432 INFO:tasks.workunit.client.0.smithi037.stdout:5/336: dwrite d8/d22/f31 [0,4194304] 0 2024-03-29T08:07:11.433 INFO:tasks.workunit.client.1.smithi148.stdout:4/196: dwrite df/f4d [0,4194304] 0 2024-03-29T08:07:11.435 INFO:tasks.workunit.client.0.smithi037.stdout:6/168: chown d0/d5/d1f 38603850 1 2024-03-29T08:07:11.435 INFO:tasks.workunit.client.1.smithi148.stdout:4/197: chown df/d10/c1b 0 1 2024-03-29T08:07:11.435 INFO:tasks.workunit.client.1.smithi148.stdout:1/208: dwrite d3/d13/f15 [8388608,4194304] 0 2024-03-29T08:07:11.436 INFO:tasks.workunit.client.0.smithi037.stdout:7/248: readlink dd/l2f 0 2024-03-29T08:07:11.440 INFO:tasks.workunit.client.0.smithi037.stdout:4/121: write d3/d4/d15/f29 [591911,1910] 0 2024-03-29T08:07:11.444 INFO:tasks.workunit.client.0.smithi037.stdout:4/122: dread - d3/f21 zero size 2024-03-29T08:07:11.446 INFO:tasks.workunit.client.1.smithi148.stdout:9/200: sync 2024-03-29T08:07:11.446 INFO:tasks.workunit.client.1.smithi148.stdout:6/215: sync 2024-03-29T08:07:11.447 INFO:tasks.workunit.client.1.smithi148.stdout:6/216: stat d6/cc 0 2024-03-29T08:07:11.449 INFO:tasks.workunit.client.0.smithi037.stdout:2/143: dread d0/f1b [0,4194304] 0 2024-03-29T08:07:11.452 INFO:tasks.workunit.client.1.smithi148.stdout:6/217: write d6/d9/f23 [798229,3329] 0 2024-03-29T08:07:11.453 INFO:tasks.workunit.client.0.smithi037.stdout:9/324: truncate d1/d4/d19/d1e/d25/d16/f56 1039904 0 2024-03-29T08:07:11.454 INFO:tasks.workunit.client.1.smithi148.stdout:7/252: fsync d8/d14/d19/d37/f48 0 2024-03-29T08:07:11.455 INFO:tasks.workunit.client.1.smithi148.stdout:1/209: symlink d3/d18/l3b 0 2024-03-29T08:07:11.455 INFO:tasks.workunit.client.1.smithi148.stdout:8/261: dwrite d17/d18/d38/f48 [0,4194304] 0 2024-03-29T08:07:11.455 INFO:tasks.workunit.client.0.smithi037.stdout:8/213: truncate d5/d10/f23 1218302 0 2024-03-29T08:07:11.460 INFO:tasks.workunit.client.1.smithi148.stdout:8/262: stat d17/d18/d38 0 2024-03-29T08:07:11.461 INFO:tasks.workunit.client.0.smithi037.stdout:1/310: dread d2/d6/dc/d18/d1d/d21/f60 [0,4194304] 0 2024-03-29T08:07:11.461 INFO:tasks.workunit.client.1.smithi148.stdout:6/218: dread - d6/d9/d1e/d20/f37 zero size 2024-03-29T08:07:11.463 INFO:tasks.workunit.client.1.smithi148.stdout:7/253: creat d8/d38/d3c/f5a x:0 0 0 2024-03-29T08:07:11.465 INFO:tasks.workunit.client.0.smithi037.stdout:7/249: rmdir dd/d21/d35/d42 39 2024-03-29T08:07:11.465 INFO:tasks.workunit.client.1.smithi148.stdout:4/198: dwrite df/f45 [0,4194304] 0 2024-03-29T08:07:11.466 INFO:tasks.workunit.client.0.smithi037.stdout:7/250: chown lc 1258 1 2024-03-29T08:07:11.467 INFO:tasks.workunit.client.1.smithi148.stdout:8/263: write d17/d18/d26/d3d/f43 [186682,70405] 0 2024-03-29T08:07:11.467 INFO:tasks.workunit.client.1.smithi148.stdout:7/254: read - d8/d14/f47 zero size 2024-03-29T08:07:11.469 INFO:tasks.workunit.client.0.smithi037.stdout:3/240: truncate dd/d15/f44 984194 0 2024-03-29T08:07:11.471 INFO:tasks.workunit.client.1.smithi148.stdout:4/199: write df/d2c/f46 [951386,26651] 0 2024-03-29T08:07:11.474 INFO:tasks.workunit.client.1.smithi148.stdout:8/264: symlink d17/d18/l58 0 2024-03-29T08:07:11.477 INFO:tasks.workunit.client.0.smithi037.stdout:9/325: symlink d1/d4/d19/d1e/d4a/d5c/l6a 0 2024-03-29T08:07:11.480 INFO:tasks.workunit.client.0.smithi037.stdout:3/241: write dd/d46/d3e/d47/f54 [651240,89027] 0 2024-03-29T08:07:11.483 INFO:tasks.workunit.client.1.smithi148.stdout:3/208: dwrite de/f2e [0,4194304] 0 2024-03-29T08:07:11.485 INFO:tasks.workunit.client.0.smithi037.stdout:9/326: write d1/d1f/d3a/f46 [1269709,28462] 0 2024-03-29T08:07:11.487 INFO:tasks.workunit.client.0.smithi037.stdout:6/169: dwrite d0/d5/fb [0,4194304] 0 2024-03-29T08:07:11.487 INFO:tasks.workunit.client.0.smithi037.stdout:4/123: symlink d3/d4/d15/l2a 0 2024-03-29T08:07:11.487 INFO:tasks.workunit.client.1.smithi148.stdout:3/209: write de/d13/d30/f3f [865580,97596] 0 2024-03-29T08:07:11.489 INFO:tasks.workunit.client.1.smithi148.stdout:1/210: dread d3/d13/f1a [0,4194304] 0 2024-03-29T08:07:11.491 INFO:tasks.workunit.client.0.smithi037.stdout:3/242: write fc [4837594,78830] 0 2024-03-29T08:07:11.491 INFO:tasks.workunit.client.0.smithi037.stdout:1/311: symlink d2/d6/d13/l70 0 2024-03-29T08:07:11.492 INFO:tasks.workunit.client.1.smithi148.stdout:1/211: stat d3/de/d1d 0 2024-03-29T08:07:11.493 INFO:tasks.workunit.client.1.smithi148.stdout:8/265: rmdir d17/d1d 39 2024-03-29T08:07:11.494 INFO:tasks.workunit.client.1.smithi148.stdout:6/219: dwrite d6/d9/d1e/d22/f34 [4194304,4194304] 0 2024-03-29T08:07:11.500 INFO:tasks.workunit.client.0.smithi037.stdout:2/144: link d0/f18 d0/d23/f30 0 2024-03-29T08:07:11.502 INFO:tasks.workunit.client.0.smithi037.stdout:5/337: link d8/d18/c1a d8/d22/d37/d60/d61/c6b 0 2024-03-29T08:07:11.502 INFO:tasks.workunit.client.0.smithi037.stdout:0/125: dread d9/ff [0,4194304] 0 2024-03-29T08:07:11.505 INFO:tasks.workunit.client.0.smithi037.stdout:6/170: mknod d0/d20/c3d 0 2024-03-29T08:07:11.508 INFO:tasks.workunit.client.1.smithi148.stdout:7/255: rename d8/d14/c31 to d8/d14/d19/c5b 0 2024-03-29T08:07:11.508 INFO:tasks.workunit.client.1.smithi148.stdout:3/210: symlink de/d13/l44 0 2024-03-29T08:07:11.510 INFO:tasks.workunit.client.1.smithi148.stdout:1/212: dread d3/de/f1b [0,4194304] 0 2024-03-29T08:07:11.510 INFO:tasks.workunit.client.0.smithi037.stdout:8/214: link d5/d10/d2d/d34/c38 d5/d10/d19/d3d/c42 0 2024-03-29T08:07:11.510 INFO:tasks.workunit.client.0.smithi037.stdout:9/327: chown d1/d4/d19/d1e/d4a/d5c/d60/d13/c5a 0 1 2024-03-29T08:07:11.511 INFO:tasks.workunit.client.1.smithi148.stdout:1/213: chown d3/de/d25 144891 1 2024-03-29T08:07:11.515 INFO:tasks.workunit.client.0.smithi037.stdout:2/145: mknod d0/d20/d22/c31 0 2024-03-29T08:07:11.520 INFO:tasks.workunit.client.0.smithi037.stdout:4/124: mknod d3/d4/c2b 0 2024-03-29T08:07:11.520 INFO:tasks.workunit.client.0.smithi037.stdout:0/126: write f7 [590602,44389] 0 2024-03-29T08:07:11.522 INFO:tasks.workunit.client.1.smithi148.stdout:5/201: sync 2024-03-29T08:07:11.525 INFO:tasks.workunit.client.0.smithi037.stdout:7/251: creat dd/d1d/f53 x:0 0 0 2024-03-29T08:07:11.527 INFO:tasks.workunit.client.1.smithi148.stdout:2/238: dread f1 [0,4194304] 0 2024-03-29T08:07:11.529 INFO:tasks.workunit.client.0.smithi037.stdout:4/125: chown d3/d4/d15/d1d/db/f1a 113643849 1 2024-03-29T08:07:11.529 INFO:tasks.workunit.client.1.smithi148.stdout:6/220: mkdir d6/d9/d1e/d22/d3a 0 2024-03-29T08:07:11.530 INFO:tasks.workunit.client.1.smithi148.stdout:7/256: dwrite d8/d29/f35 [0,4194304] 0 2024-03-29T08:07:11.531 INFO:tasks.workunit.client.0.smithi037.stdout:3/243: creat dd/d15/d18/d30/f59 x:0 0 0 2024-03-29T08:07:11.535 INFO:tasks.workunit.client.1.smithi148.stdout:9/201: sync 2024-03-29T08:07:11.539 INFO:tasks.workunit.client.1.smithi148.stdout:3/211: symlink de/l45 0 2024-03-29T08:07:11.552 INFO:tasks.workunit.client.1.smithi148.stdout:7/257: write d8/d29/f2e [4203141,52853] 0 2024-03-29T08:07:11.553 INFO:tasks.workunit.client.0.smithi037.stdout:8/215: dread d5/d10/f1d [4194304,4194304] 0 2024-03-29T08:07:11.554 INFO:tasks.workunit.client.1.smithi148.stdout:7/258: chown d8/d14/d19/f39 207787 1 2024-03-29T08:07:11.557 INFO:tasks.workunit.client.1.smithi148.stdout:0/221: truncate db/de/df/d11/d2d/f36 4159493 0 2024-03-29T08:07:11.557 INFO:tasks.workunit.client.1.smithi148.stdout:3/212: dwrite de/d13/f40 [0,4194304] 0 2024-03-29T08:07:11.557 INFO:tasks.workunit.client.0.smithi037.stdout:9/328: mknod d1/d1f/c6b 0 2024-03-29T08:07:11.558 INFO:tasks.workunit.client.1.smithi148.stdout:3/213: write de/f35 [57909,88575] 0 2024-03-29T08:07:11.558 INFO:tasks.workunit.client.1.smithi148.stdout:9/202: creat db/d10/d14/f41 x:0 0 0 2024-03-29T08:07:11.559 INFO:tasks.workunit.client.1.smithi148.stdout:9/203: readlink db/d10/l25 0 2024-03-29T08:07:11.567 INFO:tasks.workunit.client.0.smithi037.stdout:4/126: rename d3/cd to d3/d4/d27/c2c 0 2024-03-29T08:07:11.568 INFO:tasks.workunit.client.0.smithi037.stdout:7/252: dwrite dd/d21/d35/f4c [0,4194304] 0 2024-03-29T08:07:11.570 INFO:tasks.workunit.client.0.smithi037.stdout:1/312: dread d2/f31 [0,4194304] 0 2024-03-29T08:07:11.582 INFO:tasks.workunit.client.0.smithi037.stdout:6/171: dread d0/d1/f1b [0,4194304] 0 2024-03-29T08:07:11.584 INFO:tasks.workunit.client.0.smithi037.stdout:8/216: write d5/f15 [8620703,130975] 0 2024-03-29T08:07:11.586 INFO:tasks.workunit.client.1.smithi148.stdout:3/214: unlink de/l1d 0 2024-03-29T08:07:11.588 INFO:tasks.workunit.client.1.smithi148.stdout:3/215: write de/d24/f36 [871369,74134] 0 2024-03-29T08:07:11.588 INFO:tasks.workunit.client.0.smithi037.stdout:6/172: write d0/d5/d1e/f12 [2463046,126443] 0 2024-03-29T08:07:11.590 INFO:tasks.workunit.client.1.smithi148.stdout:3/216: read de/f35 [63708,107405] 0 2024-03-29T08:07:11.591 INFO:tasks.workunit.client.0.smithi037.stdout:1/313: creat d2/d6/dc/d58/f71 x:0 0 0 2024-03-29T08:07:11.593 INFO:tasks.workunit.client.1.smithi148.stdout:8/266: write d17/d18/d26/d3d/f45 [549105,6319] 0 2024-03-29T08:07:11.593 INFO:tasks.workunit.client.0.smithi037.stdout:8/217: dread d5/f16 [0,4194304] 0 2024-03-29T08:07:11.599 INFO:tasks.workunit.client.0.smithi037.stdout:1/314: chown d2/d6/dc/d18/d1d/d21/d38 1511132 1 2024-03-29T08:07:11.599 INFO:tasks.workunit.client.1.smithi148.stdout:5/202: link d0/db/de/d16/f3d d0/db/f3e 0 2024-03-29T08:07:11.601 INFO:tasks.workunit.client.1.smithi148.stdout:5/203: chown d0/ca 238 1 2024-03-29T08:07:11.602 INFO:tasks.workunit.client.1.smithi148.stdout:4/200: truncate df/d10/d17/f20 723157 0 2024-03-29T08:07:11.603 INFO:tasks.workunit.client.1.smithi148.stdout:5/204: dread - d0/db/de/d16/f3d zero size 2024-03-29T08:07:11.603 INFO:tasks.workunit.client.0.smithi037.stdout:4/127: rmdir d3/d4/d23 0 2024-03-29T08:07:11.604 INFO:tasks.workunit.client.1.smithi148.stdout:4/201: write df/d2c/f2e [142752,1047] 0 2024-03-29T08:07:11.604 INFO:tasks.workunit.client.0.smithi037.stdout:1/315: chown d2/d6/dc/d18/f3e 3 1 2024-03-29T08:07:11.605 INFO:tasks.workunit.client.1.smithi148.stdout:0/222: dread db/dc/f12 [0,4194304] 0 2024-03-29T08:07:11.608 INFO:tasks.workunit.client.0.smithi037.stdout:9/329: dwrite d1/d4/d19/d1e/d4a/d5c/d60/d23/f39 [0,4194304] 0 2024-03-29T08:07:11.610 INFO:tasks.workunit.client.0.smithi037.stdout:3/244: rmdir dd/d15/d18 39 2024-03-29T08:07:11.614 INFO:tasks.workunit.client.1.smithi148.stdout:1/214: getdents d3/d13 0 2024-03-29T08:07:11.615 INFO:tasks.workunit.client.1.smithi148.stdout:9/204: creat db/d22/d3f/f42 x:0 0 0 2024-03-29T08:07:11.616 INFO:tasks.workunit.client.0.smithi037.stdout:8/218: creat d5/d10/d19/f43 x:0 0 0 2024-03-29T08:07:11.621 INFO:tasks.workunit.client.1.smithi148.stdout:3/217: rename de/d13 to de/d24/d33/d39/d46 0 2024-03-29T08:07:11.624 INFO:tasks.workunit.client.1.smithi148.stdout:5/205: dwrite d0/f21 [0,4194304] 0 2024-03-29T08:07:11.626 INFO:tasks.workunit.client.1.smithi148.stdout:8/267: readlink d17/d1d/l42 0 2024-03-29T08:07:11.629 INFO:tasks.workunit.client.1.smithi148.stdout:5/206: write d0/db/f10 [1782511,72424] 0 2024-03-29T08:07:11.630 INFO:tasks.workunit.client.0.smithi037.stdout:5/338: truncate d8/d22/f27 7485337 0 2024-03-29T08:07:11.631 INFO:tasks.workunit.client.0.smithi037.stdout:8/219: mknod d5/c44 0 2024-03-29T08:07:11.631 INFO:tasks.workunit.client.1.smithi148.stdout:5/207: chown d0/db/l22 78851 1 2024-03-29T08:07:11.633 INFO:tasks.workunit.client.0.smithi037.stdout:2/146: write d0/f8 [742007,106252] 0 2024-03-29T08:07:11.633 INFO:tasks.workunit.client.1.smithi148.stdout:0/223: dread fa [0,4194304] 0 2024-03-29T08:07:11.634 INFO:tasks.workunit.client.1.smithi148.stdout:2/239: getdents d7/da/d33/d45/d4d/d2e 0 2024-03-29T08:07:11.635 INFO:tasks.workunit.client.0.smithi037.stdout:0/127: dwrite d9/da/dd/d1c/f18 [0,4194304] 0 2024-03-29T08:07:11.635 INFO:tasks.workunit.client.1.smithi148.stdout:6/221: getdents d6/d9/d1e 0 2024-03-29T08:07:11.635 INFO:tasks.workunit.client.0.smithi037.stdout:6/173: rename d0/d20/c21 to d0/c3e 0 2024-03-29T08:07:11.636 INFO:tasks.workunit.client.0.smithi037.stdout:9/330: rename d1/d4/d19 to d1/d4/d19/d6c 22 2024-03-29T08:07:11.637 INFO:tasks.workunit.client.0.smithi037.stdout:6/174: chown d0/d20/d38 0 1 2024-03-29T08:07:11.641 INFO:tasks.workunit.client.0.smithi037.stdout:4/128: creat d3/d11/f2d x:0 0 0 2024-03-29T08:07:11.642 INFO:tasks.workunit.client.0.smithi037.stdout:8/220: mkdir d5/d10/d45 0 2024-03-29T08:07:11.647 INFO:tasks.workunit.client.0.smithi037.stdout:5/339: creat d8/d4d/f6c x:0 0 0 2024-03-29T08:07:11.651 INFO:tasks.workunit.client.1.smithi148.stdout:5/208: dwrite d0/db/de/f3b [0,4194304] 0 2024-03-29T08:07:11.652 INFO:tasks.workunit.client.0.smithi037.stdout:4/129: read d3/d4/fa [2516245,22088] 0 2024-03-29T08:07:11.653 INFO:tasks.workunit.client.1.smithi148.stdout:5/209: stat d0/db/de/cf 0 2024-03-29T08:07:11.656 INFO:tasks.workunit.client.1.smithi148.stdout:1/215: dread d3/f7 [0,4194304] 0 2024-03-29T08:07:11.656 INFO:tasks.workunit.client.1.smithi148.stdout:8/268: write d17/d18/d38/f54 [212469,37280] 0 2024-03-29T08:07:11.658 INFO:tasks.workunit.client.0.smithi037.stdout:9/331: unlink d1/d4/c51 0 2024-03-29T08:07:11.658 INFO:tasks.workunit.client.0.smithi037.stdout:4/130: dread - d3/d11/f2d zero size 2024-03-29T08:07:11.667 INFO:tasks.workunit.client.1.smithi148.stdout:7/259: getdents d8/d29/d44 0 2024-03-29T08:07:11.671 INFO:tasks.workunit.client.1.smithi148.stdout:0/224: write db/de/df/d2b/f35 [1985691,42209] 0 2024-03-29T08:07:11.673 INFO:tasks.workunit.client.0.smithi037.stdout:8/221: mkdir d5/d25/d46 0 2024-03-29T08:07:11.674 INFO:tasks.workunit.client.0.smithi037.stdout:4/131: dread - d3/f1b zero size 2024-03-29T08:07:11.676 INFO:tasks.workunit.client.0.smithi037.stdout:0/128: truncate f5 248870 0 2024-03-29T08:07:11.677 INFO:tasks.workunit.client.1.smithi148.stdout:3/218: fsync de/d24/f27 0 2024-03-29T08:07:11.680 INFO:tasks.workunit.client.1.smithi148.stdout:5/210: symlink d0/db/d1e/l3f 0 2024-03-29T08:07:11.683 INFO:tasks.workunit.client.1.smithi148.stdout:5/211: dread d0/db/f34 [0,4194304] 0 2024-03-29T08:07:11.684 INFO:tasks.workunit.client.1.smithi148.stdout:8/269: write d17/d18/f3a [199673,18981] 0 2024-03-29T08:07:11.686 INFO:tasks.workunit.client.1.smithi148.stdout:3/219: dread de/d24/f27 [0,4194304] 0 2024-03-29T08:07:11.688 INFO:tasks.workunit.client.1.smithi148.stdout:2/240: unlink c3 0 2024-03-29T08:07:11.690 INFO:tasks.workunit.client.0.smithi037.stdout:7/253: dwrite dd/d1d/d34/f40 [0,4194304] 0 2024-03-29T08:07:11.691 INFO:tasks.workunit.client.1.smithi148.stdout:7/260: symlink d8/d14/l5c 0 2024-03-29T08:07:11.693 INFO:tasks.workunit.client.1.smithi148.stdout:9/205: creat db/f43 x:0 0 0 2024-03-29T08:07:11.694 INFO:tasks.workunit.client.1.smithi148.stdout:9/206: dread - db/d10/d14/f3d zero size 2024-03-29T08:07:11.694 INFO:tasks.workunit.client.1.smithi148.stdout:0/225: mkdir db/de/df/d2b/d3b/d4a 0 2024-03-29T08:07:11.695 INFO:tasks.workunit.client.1.smithi148.stdout:8/270: mkdir d17/d1d/d59 0 2024-03-29T08:07:11.696 INFO:tasks.workunit.client.1.smithi148.stdout:1/216: creat d3/de/d25/d35/f3c x:0 0 0 2024-03-29T08:07:11.697 INFO:tasks.workunit.client.1.smithi148.stdout:1/217: chown d3/f7 1 1 2024-03-29T08:07:11.698 INFO:tasks.workunit.client.1.smithi148.stdout:4/202: getdents df/d2d 0 2024-03-29T08:07:11.699 INFO:tasks.workunit.client.0.smithi037.stdout:6/175: dwrite d0/d1/f17 [0,4194304] 0 2024-03-29T08:07:11.699 INFO:tasks.workunit.client.0.smithi037.stdout:0/129: creat d9/f27 x:0 0 0 2024-03-29T08:07:11.700 INFO:tasks.workunit.client.1.smithi148.stdout:6/222: rmdir d6/d9/d1e/d22/d3a 0 2024-03-29T08:07:11.700 INFO:tasks.workunit.client.1.smithi148.stdout:4/203: dread - df/d2d/f35 zero size 2024-03-29T08:07:11.701 INFO:tasks.workunit.client.1.smithi148.stdout:2/241: symlink d7/da/d33/l53 0 2024-03-29T08:07:11.702 INFO:tasks.workunit.client.0.smithi037.stdout:3/245: dwrite dd/f39 [0,4194304] 0 2024-03-29T08:07:11.702 INFO:tasks.workunit.client.1.smithi148.stdout:7/261: symlink d8/d38/d3c/l5d 0 2024-03-29T08:07:11.707 INFO:tasks.workunit.client.0.smithi037.stdout:4/132: rename d3/f1b to d3/d4/d27/f2e 0 2024-03-29T08:07:11.708 INFO:tasks.workunit.client.1.smithi148.stdout:9/207: unlink db/cc 0 2024-03-29T08:07:11.710 INFO:tasks.workunit.client.0.smithi037.stdout:6/176: write d0/f37 [4904896,86211] 0 2024-03-29T08:07:11.711 INFO:tasks.workunit.client.0.smithi037.stdout:8/222: getdents d5 0 2024-03-29T08:07:11.713 INFO:tasks.workunit.client.1.smithi148.stdout:8/271: fdatasync d17/d18/f3f 0 2024-03-29T08:07:11.715 INFO:tasks.workunit.client.0.smithi037.stdout:5/340: dread d8/d18/f40 [0,4194304] 0 2024-03-29T08:07:11.715 INFO:tasks.workunit.client.0.smithi037.stdout:0/130: readlink d9/da/l1d 0 2024-03-29T08:07:11.719 INFO:tasks.workunit.client.1.smithi148.stdout:7/262: unlink d8/f13 0 2024-03-29T08:07:11.724 INFO:tasks.workunit.client.1.smithi148.stdout:4/204: creat df/d10/d26/d34/d43/f4e x:0 0 0 2024-03-29T08:07:11.724 INFO:tasks.workunit.client.1.smithi148.stdout:1/218: dwrite d3/de/d25/d35/f3c [0,4194304] 0 2024-03-29T08:07:11.724 INFO:tasks.workunit.client.1.smithi148.stdout:9/208: chown f3 36115 1 2024-03-29T08:07:11.730 INFO:tasks.workunit.client.1.smithi148.stdout:6/223: dwrite d6/d9/f1f [0,4194304] 0 2024-03-29T08:07:11.730 INFO:tasks.workunit.client.0.smithi037.stdout:6/177: unlink d0/d5/d1e/c18 0 2024-03-29T08:07:11.731 INFO:tasks.workunit.client.0.smithi037.stdout:2/147: dread d0/f1f [0,4194304] 0 2024-03-29T08:07:11.734 INFO:tasks.workunit.client.1.smithi148.stdout:4/205: dread - df/d10/d26/f3b zero size 2024-03-29T08:07:11.734 INFO:tasks.workunit.client.1.smithi148.stdout:2/242: dwrite d7/da/d33/d45/d4d/d2e/f50 [0,4194304] 0 2024-03-29T08:07:11.735 INFO:tasks.workunit.client.1.smithi148.stdout:6/224: fdatasync d6/f2d 0 2024-03-29T08:07:11.735 INFO:tasks.workunit.client.1.smithi148.stdout:1/219: chown d3/d13/c1e 494046 1 2024-03-29T08:07:11.736 INFO:tasks.workunit.client.1.smithi148.stdout:3/220: dread de/d24/f2b [0,4194304] 0 2024-03-29T08:07:11.737 INFO:tasks.workunit.client.1.smithi148.stdout:8/272: write d17/d18/f2a [368277,77580] 0 2024-03-29T08:07:11.737 INFO:tasks.workunit.client.1.smithi148.stdout:0/226: rmdir db/de/df/d2b/d3b/d4a 0 2024-03-29T08:07:11.737 INFO:tasks.workunit.client.0.smithi037.stdout:8/223: rename d5/d25/c26 to d5/d10/c47 0 2024-03-29T08:07:11.738 INFO:tasks.workunit.client.1.smithi148.stdout:3/221: readlink de/l28 0 2024-03-29T08:07:11.738 INFO:tasks.workunit.client.0.smithi037.stdout:4/133: creat d3/d4/f2f x:0 0 0 2024-03-29T08:07:11.743 INFO:tasks.workunit.client.1.smithi148.stdout:9/209: rmdir db/d10/d2d/d32/d3b 39 2024-03-29T08:07:11.745 INFO:tasks.workunit.client.0.smithi037.stdout:9/332: dwrite d1/d4/d19/d1e/d4a/d5c/d60/d13/f17 [0,4194304] 0 2024-03-29T08:07:11.746 INFO:tasks.workunit.client.1.smithi148.stdout:3/222: symlink de/d24/d33/d39/d46/d30/l47 0 2024-03-29T08:07:11.748 INFO:tasks.workunit.client.0.smithi037.stdout:8/224: creat d5/f48 x:0 0 0 2024-03-29T08:07:11.749 INFO:tasks.workunit.client.1.smithi148.stdout:5/212: truncate d0/db/de/d16/f27 2539694 0 2024-03-29T08:07:11.750 INFO:tasks.workunit.client.1.smithi148.stdout:8/273: creat d17/d18/d38/f5a x:0 0 0 2024-03-29T08:07:11.750 INFO:tasks.workunit.client.0.smithi037.stdout:2/148: chown d0/cf 24295 1 2024-03-29T08:07:11.751 INFO:tasks.workunit.client.1.smithi148.stdout:8/274: chown fe 27932 1 2024-03-29T08:07:11.751 INFO:tasks.workunit.client.1.smithi148.stdout:5/213: write d0/f2c [3380489,10402] 0 2024-03-29T08:07:11.754 INFO:tasks.workunit.client.0.smithi037.stdout:6/178: truncate d0/d5/d1f/f3c 745901 0 2024-03-29T08:07:11.756 INFO:tasks.workunit.client.1.smithi148.stdout:1/220: rename d3/lf to d3/de/d25/d31/l3d 0 2024-03-29T08:07:11.758 INFO:tasks.workunit.client.0.smithi037.stdout:2/149: read d0/f2b [2290662,44882] 0 2024-03-29T08:07:11.759 INFO:tasks.workunit.client.1.smithi148.stdout:1/221: write d3/d13/f15 [8044799,112192] 0 2024-03-29T08:07:11.760 INFO:tasks.workunit.client.1.smithi148.stdout:1/222: chown d3/d13/d23 7 1 2024-03-29T08:07:11.761 INFO:tasks.workunit.client.1.smithi148.stdout:1/223: write d3/d13/f15 [5616300,19581] 0 2024-03-29T08:07:11.762 INFO:tasks.workunit.client.1.smithi148.stdout:9/210: creat db/d10/d14/d35/f44 x:0 0 0 2024-03-29T08:07:11.775 INFO:tasks.workunit.client.0.smithi037.stdout:7/254: dwrite dd/d1d/d34/f37 [0,4194304] 0 2024-03-29T08:07:11.779 INFO:tasks.workunit.client.0.smithi037.stdout:6/179: read - d0/d5/d1f/f35 zero size 2024-03-29T08:07:11.782 INFO:tasks.workunit.client.1.smithi148.stdout:3/223: symlink de/d24/d33/d39/l48 0 2024-03-29T08:07:11.783 INFO:tasks.workunit.client.1.smithi148.stdout:8/275: rename d17/f2c to d17/d18/d26/d3d/f5b 0 2024-03-29T08:07:11.784 INFO:tasks.workunit.client.1.smithi148.stdout:7/263: fsync d8/d14/f1c 0 2024-03-29T08:07:11.784 INFO:tasks.workunit.client.1.smithi148.stdout:9/211: unlink db/d22/d26/l27 0 2024-03-29T08:07:11.787 INFO:tasks.workunit.client.0.smithi037.stdout:6/180: dread - d0/d20/d38/f39 zero size 2024-03-29T08:07:11.792 INFO:tasks.workunit.client.0.smithi037.stdout:3/246: dwrite dd/fe [0,4194304] 0 2024-03-29T08:07:11.793 INFO:tasks.workunit.client.1.smithi148.stdout:2/243: getdents d7/da/d33/d3c/d21 0 2024-03-29T08:07:11.799 INFO:tasks.workunit.client.0.smithi037.stdout:7/255: mkdir dd/d1d/d34/d54 0 2024-03-29T08:07:11.802 INFO:tasks.workunit.client.1.smithi148.stdout:5/214: rename d0/db/de/d16/c30 to d0/db/d1e/d35/c40 0 2024-03-29T08:07:11.802 INFO:tasks.workunit.client.0.smithi037.stdout:4/134: dwrite d3/f21 [0,4194304] 0 2024-03-29T08:07:11.807 INFO:tasks.workunit.client.1.smithi148.stdout:8/276: readlink d17/d18/d26/d3d/l4c 0 2024-03-29T08:07:11.807 INFO:tasks.workunit.client.0.smithi037.stdout:8/225: dread d5/f1b [0,4194304] 0 2024-03-29T08:07:11.807 INFO:tasks.workunit.client.0.smithi037.stdout:6/181: write d0/d20/d38/f39 [626784,28183] 0 2024-03-29T08:07:11.808 INFO:tasks.workunit.client.0.smithi037.stdout:3/247: chown dd/d15/l1c 12 1 2024-03-29T08:07:11.814 INFO:tasks.workunit.client.1.smithi148.stdout:3/224: rename de/d3b/d41 to de/d3b/d49 0 2024-03-29T08:07:11.814 INFO:tasks.workunit.client.0.smithi037.stdout:9/333: dwrite d1/d4/f28 [0,4194304] 0 2024-03-29T08:07:11.814 INFO:tasks.workunit.client.0.smithi037.stdout:0/131: dwrite d9/da/dd/d1c/f18 [0,4194304] 0 2024-03-29T08:07:11.819 INFO:tasks.workunit.client.0.smithi037.stdout:8/226: dread - d5/d10/d19/f43 zero size 2024-03-29T08:07:11.819 INFO:tasks.workunit.client.0.smithi037.stdout:7/256: write dd/f12 [2813331,24743] 0 2024-03-29T08:07:11.825 INFO:tasks.workunit.client.0.smithi037.stdout:9/334: stat d1/d4/d19/d1e/d25 0 2024-03-29T08:07:11.826 INFO:tasks.workunit.client.0.smithi037.stdout:8/227: truncate d5/d25/f2e 1090275 0 2024-03-29T08:07:11.829 INFO:tasks.workunit.client.0.smithi037.stdout:8/228: fdatasync d5/f32 0 2024-03-29T08:07:11.829 INFO:tasks.workunit.client.1.smithi148.stdout:3/225: write de/d24/d33/d39/d46/f25 [439238,98945] 0 2024-03-29T08:07:11.829 INFO:tasks.workunit.client.1.smithi148.stdout:6/225: dwrite d6/d9/d1e/d20/f39 [0,4194304] 0 2024-03-29T08:07:11.830 INFO:tasks.workunit.client.1.smithi148.stdout:4/206: dwrite df/d10/d17/f20 [0,4194304] 0 2024-03-29T08:07:11.833 INFO:tasks.workunit.client.1.smithi148.stdout:4/207: fdatasync df/d10/d26/f47 0 2024-03-29T08:07:11.838 INFO:tasks.workunit.client.1.smithi148.stdout:5/215: rename d0/d26 to d0/db/d1e/d36/d41 0 2024-03-29T08:07:11.842 INFO:tasks.workunit.client.0.smithi037.stdout:3/248: dread dd/d46/d2d/d3d/f45 [0,4194304] 0 2024-03-29T08:07:11.845 INFO:tasks.workunit.client.1.smithi148.stdout:7/264: dwrite d8/d14/f2c [0,4194304] 0 2024-03-29T08:07:11.848 INFO:tasks.workunit.client.0.smithi037.stdout:3/249: write dd/d15/f50 [906976,124482] 0 2024-03-29T08:07:11.857 INFO:tasks.workunit.client.1.smithi148.stdout:0/227: sync 2024-03-29T08:07:11.861 INFO:tasks.workunit.client.1.smithi148.stdout:2/244: truncate d7/da/d33/f35 8181704 0 2024-03-29T08:07:11.861 INFO:tasks.workunit.client.1.smithi148.stdout:1/224: getdents d3/de/d25/d31 0 2024-03-29T08:07:11.870 INFO:tasks.workunit.client.0.smithi037.stdout:6/182: rename d0/d1 to d0/d5/d3f 0 2024-03-29T08:07:11.870 INFO:tasks.workunit.client.1.smithi148.stdout:3/226: read de/d24/f42 [58517,45010] 0 2024-03-29T08:07:11.873 INFO:tasks.workunit.client.0.smithi037.stdout:2/150: dwrite d0/f18 [0,4194304] 0 2024-03-29T08:07:11.874 INFO:tasks.workunit.client.1.smithi148.stdout:5/216: mkdir d0/db/d1e/d35/d42 0 2024-03-29T08:07:11.875 INFO:tasks.workunit.client.1.smithi148.stdout:7/265: rmdir d8/d14/d19/d37 39 2024-03-29T08:07:11.877 INFO:tasks.workunit.client.0.smithi037.stdout:7/257: rmdir dd/d1d 39 2024-03-29T08:07:11.879 INFO:tasks.workunit.client.0.smithi037.stdout:8/229: creat d5/d10/d45/f49 x:0 0 0 2024-03-29T08:07:11.881 INFO:tasks.workunit.client.1.smithi148.stdout:9/212: link db/d20/f1e db/d10/d2d/d32/d3b/f45 0 2024-03-29T08:07:11.882 INFO:tasks.workunit.client.0.smithi037.stdout:7/258: creat dd/d21/f55 x:0 0 0 2024-03-29T08:07:11.883 INFO:tasks.workunit.client.1.smithi148.stdout:9/213: chown db/d10/d2d/d32/d3b/f2e 6674 1 2024-03-29T08:07:11.883 INFO:tasks.workunit.client.0.smithi037.stdout:8/230: chown d5/f48 0 1 2024-03-29T08:07:11.884 INFO:tasks.workunit.client.1.smithi148.stdout:9/214: fdatasync db/d10/d14/f3d 0 2024-03-29T08:07:11.885 INFO:tasks.workunit.client.0.smithi037.stdout:8/231: fdatasync d5/f1b 0 2024-03-29T08:07:11.887 INFO:tasks.workunit.client.1.smithi148.stdout:2/245: chown d7/da/c22 1490 1 2024-03-29T08:07:11.887 INFO:tasks.workunit.client.1.smithi148.stdout:1/225: rename f1 to d3/de/d25/d35/f3e 0 2024-03-29T08:07:11.888 INFO:tasks.workunit.client.1.smithi148.stdout:1/226: chown d3/de/d1d 602 1 2024-03-29T08:07:11.888 INFO:tasks.workunit.client.0.smithi037.stdout:2/151: mknod d0/d20/d2e/c32 0 2024-03-29T08:07:11.895 INFO:tasks.workunit.client.0.smithi037.stdout:2/152: fdatasync d0/f1f 0 2024-03-29T08:07:11.897 INFO:tasks.workunit.client.0.smithi037.stdout:8/232: mkdir d5/d10/d19/d3d/d4a 0 2024-03-29T08:07:11.897 INFO:tasks.workunit.client.1.smithi148.stdout:8/277: link d17/d18/d26/d3d/f5b d17/d18/f5c 0 2024-03-29T08:07:11.898 INFO:tasks.workunit.client.0.smithi037.stdout:9/335: write d1/d4/d19/d1e/d4a/d5c/d60/f3d [1186873,109359] 0 2024-03-29T08:07:11.898 INFO:tasks.workunit.client.0.smithi037.stdout:6/183: dwrite d0/d5/f9 [0,4194304] 0 2024-03-29T08:07:11.901 INFO:tasks.workunit.client.0.smithi037.stdout:4/135: truncate d3/d4/d15/f19 503655 0 2024-03-29T08:07:11.903 INFO:tasks.workunit.client.0.smithi037.stdout:0/132: dread f7 [4194304,4194304] 0 2024-03-29T08:07:11.903 INFO:tasks.workunit.client.0.smithi037.stdout:8/233: chown d5/f16 28896201 1 2024-03-29T08:07:11.903 INFO:tasks.workunit.client.0.smithi037.stdout:2/153: symlink d0/d20/d2e/l33 0 2024-03-29T08:07:11.909 INFO:tasks.workunit.client.0.smithi037.stdout:9/336: write d1/d4/d19/f64 [264442,93751] 0 2024-03-29T08:07:11.909 INFO:tasks.workunit.client.1.smithi148.stdout:4/208: dwrite fd [0,4194304] 0 2024-03-29T08:07:11.913 INFO:tasks.workunit.client.0.smithi037.stdout:6/184: creat d0/d19/d34/f40 x:0 0 0 2024-03-29T08:07:11.916 INFO:tasks.workunit.client.0.smithi037.stdout:8/234: mkdir d5/d10/d45/d4b 0 2024-03-29T08:07:11.916 INFO:tasks.workunit.client.0.smithi037.stdout:7/259: dwrite dd/f14 [0,4194304] 0 2024-03-29T08:07:11.923 INFO:tasks.workunit.client.1.smithi148.stdout:9/215: unlink db/f2f 0 2024-03-29T08:07:11.924 INFO:tasks.workunit.client.0.smithi037.stdout:0/133: mknod d9/da/c28 0 2024-03-29T08:07:11.926 INFO:tasks.workunit.client.0.smithi037.stdout:6/185: creat d0/d19/f41 x:0 0 0 2024-03-29T08:07:11.930 INFO:tasks.workunit.client.1.smithi148.stdout:0/228: unlink db/de/df/d11/l44 0 2024-03-29T08:07:11.930 INFO:tasks.workunit.client.0.smithi037.stdout:2/154: link d0/c16 d0/d20/d22/c34 0 2024-03-29T08:07:11.930 INFO:tasks.workunit.client.0.smithi037.stdout:4/136: creat d3/d11/f30 x:0 0 0 2024-03-29T08:07:11.932 INFO:tasks.workunit.client.0.smithi037.stdout:6/186: fdatasync d0/d5/d1e/f12 0 2024-03-29T08:07:11.934 INFO:tasks.workunit.client.0.smithi037.stdout:0/134: creat d9/da/d1b/f29 x:0 0 0 2024-03-29T08:07:11.934 INFO:tasks.workunit.client.0.smithi037.stdout:8/235: mkdir d5/d10/d19/d3d/d4a/d4c 0 2024-03-29T08:07:11.938 INFO:tasks.workunit.client.0.smithi037.stdout:7/260: symlink dd/d1d/d49/l56 0 2024-03-29T08:07:11.938 INFO:tasks.workunit.client.1.smithi148.stdout:9/216: dwrite db/d22/d26/f2a [0,4194304] 0 2024-03-29T08:07:11.939 INFO:tasks.workunit.client.0.smithi037.stdout:9/337: getdents d1/d4/d19/d1e/d4a/d5c 0 2024-03-29T08:07:11.942 INFO:tasks.workunit.client.0.smithi037.stdout:0/135: write d9/da/f25 [1041319,19318] 0 2024-03-29T08:07:11.942 INFO:tasks.workunit.client.1.smithi148.stdout:2/246: mkdir d7/da/d33/d3c/d54 0 2024-03-29T08:07:11.943 INFO:tasks.workunit.client.1.smithi148.stdout:2/247: read - d7/da/d33/f3e zero size 2024-03-29T08:07:11.945 INFO:tasks.workunit.client.0.smithi037.stdout:4/137: mknod d3/d4/d15/d1d/c31 0 2024-03-29T08:07:11.946 INFO:tasks.workunit.client.1.smithi148.stdout:8/278: stat d17/d18/d26/f35 0 2024-03-29T08:07:11.946 INFO:tasks.workunit.client.1.smithi148.stdout:2/248: chown d7/da/d33/d3c/d21 405784 1 2024-03-29T08:07:11.951 INFO:tasks.workunit.client.0.smithi037.stdout:4/138: chown d3/d4/d15/d1d/c31 50389405 1 2024-03-29T08:07:11.951 INFO:tasks.workunit.client.0.smithi037.stdout:0/136: chown d9/da/dd/l14 9005410 1 2024-03-29T08:07:11.952 INFO:tasks.workunit.client.1.smithi148.stdout:3/227: write de/d24/f2b [2229393,87687] 0 2024-03-29T08:07:11.953 INFO:tasks.workunit.client.0.smithi037.stdout:8/236: rename f4 to d5/d10/d19/d3d/f4d 0 2024-03-29T08:07:11.954 INFO:tasks.workunit.client.1.smithi148.stdout:2/249: read - d7/da/d33/d45/d4d/f29 zero size 2024-03-29T08:07:11.955 INFO:tasks.workunit.client.1.smithi148.stdout:3/228: chown de/d24/d33/d39/d46/c1f 10 1 2024-03-29T08:07:11.957 INFO:tasks.workunit.client.1.smithi148.stdout:5/217: creat d0/db/d1e/d35/d42/f43 x:0 0 0 2024-03-29T08:07:11.958 INFO:tasks.workunit.client.1.smithi148.stdout:6/226: getdents d6/d9/d1e/d27 0 2024-03-29T08:07:11.958 INFO:tasks.workunit.client.1.smithi148.stdout:2/250: write d7/da/d33/d45/d4d/f29 [852915,15784] 0 2024-03-29T08:07:11.962 INFO:tasks.workunit.client.1.smithi148.stdout:2/251: dread - d7/da/d33/f3e zero size 2024-03-29T08:07:11.965 INFO:tasks.workunit.client.1.smithi148.stdout:2/252: read d7/da/d33/d45/d4d/f16 [428942,102508] 0 2024-03-29T08:07:11.969 INFO:tasks.workunit.client.0.smithi037.stdout:6/187: getdents d0/d5 0 2024-03-29T08:07:11.970 INFO:tasks.workunit.client.1.smithi148.stdout:8/279: dwrite d17/d18/d26/d3d/f45 [0,4194304] 0 2024-03-29T08:07:11.971 INFO:tasks.workunit.client.1.smithi148.stdout:8/280: dread - d17/d18/f5c zero size 2024-03-29T08:07:11.975 INFO:tasks.workunit.client.0.smithi037.stdout:9/338: getdents d1/d4/d19 0 2024-03-29T08:07:11.976 INFO:tasks.workunit.client.1.smithi148.stdout:5/218: dwrite d0/f9 [0,4194304] 0 2024-03-29T08:07:11.985 INFO:tasks.workunit.client.1.smithi148.stdout:7/266: getdents d8 0 2024-03-29T08:07:11.986 INFO:tasks.workunit.client.0.smithi037.stdout:6/188: mkdir d0/d19/d42 0 2024-03-29T08:07:11.989 INFO:tasks.workunit.client.1.smithi148.stdout:6/227: creat d6/d9/d1e/d22/d28/f3b x:0 0 0 2024-03-29T08:07:11.994 INFO:tasks.workunit.client.1.smithi148.stdout:6/228: truncate d6/d9/d1e/d20/f37 680339 0 2024-03-29T08:07:11.995 INFO:tasks.workunit.client.1.smithi148.stdout:9/217: dwrite db/d10/d14/f18 [4194304,4194304] 0 2024-03-29T08:07:11.996 INFO:tasks.workunit.client.0.smithi037.stdout:4/139: dwrite d3/d4/d15/f29 [0,4194304] 0 2024-03-29T08:07:11.998 INFO:tasks.workunit.client.1.smithi148.stdout:8/281: chown d17/d18/l19 525 1 2024-03-29T08:07:11.998 INFO:tasks.workunit.client.1.smithi148.stdout:6/229: chown d6/d9/d1e/d22/f31 1283814804 1 2024-03-29T08:07:11.999 INFO:tasks.workunit.client.0.smithi037.stdout:7/261: dwrite dd/d1d/d34/f3f [0,4194304] 0 2024-03-29T08:07:12.006 INFO:tasks.workunit.client.0.smithi037.stdout:9/339: creat d1/d1f/d3a/d67/f6d x:0 0 0 2024-03-29T08:07:12.016 INFO:tasks.workunit.client.0.smithi037.stdout:0/137: dwrite d9/da/f24 [0,4194304] 0 2024-03-29T08:07:12.017 INFO:tasks.workunit.client.1.smithi148.stdout:5/219: mknod d0/db/de/d16/c44 0 2024-03-29T08:07:12.018 INFO:tasks.workunit.client.0.smithi037.stdout:8/237: dwrite d5/f17 [0,4194304] 0 2024-03-29T08:07:12.018 INFO:tasks.workunit.client.0.smithi037.stdout:4/140: write d3/d4/d15/f29 [99184,16586] 0 2024-03-29T08:07:12.026 INFO:tasks.workunit.client.1.smithi148.stdout:5/220: dread d0/db/d1e/f31 [0,4194304] 0 2024-03-29T08:07:12.029 INFO:tasks.workunit.client.0.smithi037.stdout:4/141: symlink d3/d4/d27/l32 0 2024-03-29T08:07:12.029 INFO:tasks.workunit.client.0.smithi037.stdout:8/238: write d5/f13 [2632483,20934] 0 2024-03-29T08:07:12.030 INFO:tasks.workunit.client.1.smithi148.stdout:4/209: sync 2024-03-29T08:07:12.031 INFO:tasks.workunit.client.1.smithi148.stdout:4/210: chown df/f4d 12872952 1 2024-03-29T08:07:12.032 INFO:tasks.workunit.client.1.smithi148.stdout:4/211: chown c8 15668 1 2024-03-29T08:07:12.034 INFO:tasks.workunit.client.1.smithi148.stdout:4/212: chown df/d10/d26/f47 2 1 2024-03-29T08:07:12.037 INFO:tasks.workunit.client.0.smithi037.stdout:0/138: write f7 [2296251,22583] 0 2024-03-29T08:07:12.039 INFO:tasks.workunit.client.0.smithi037.stdout:4/142: rmdir d3/d4/d15/d1d 39 2024-03-29T08:07:12.044 INFO:tasks.workunit.client.1.smithi148.stdout:3/229: creat de/d24/d33/d39/d46/d30/d37/f4a x:0 0 0 2024-03-29T08:07:12.045 INFO:tasks.workunit.client.0.smithi037.stdout:7/262: dread dd/f1b [0,4194304] 0 2024-03-29T08:07:12.048 INFO:tasks.workunit.client.1.smithi148.stdout:7/267: stat f6 0 2024-03-29T08:07:12.053 INFO:tasks.workunit.client.1.smithi148.stdout:2/253: write d7/da/f49 [2212561,43431] 0 2024-03-29T08:07:12.053 INFO:tasks.workunit.client.0.smithi037.stdout:8/239: mkdir d5/d25/d46/d4e 0 2024-03-29T08:07:12.054 INFO:tasks.workunit.client.1.smithi148.stdout:1/227: getdents d3/de/d25/d35 0 2024-03-29T08:07:12.055 INFO:tasks.workunit.client.0.smithi037.stdout:4/143: mknod d3/d11/c33 0 2024-03-29T08:07:12.062 INFO:tasks.workunit.client.0.smithi037.stdout:0/139: link d9/da/c28 d9/da/d1b/c2a 0 2024-03-29T08:07:12.063 INFO:tasks.workunit.client.1.smithi148.stdout:3/230: mknod de/d24/d33/d39/c4b 0 2024-03-29T08:07:12.066 INFO:tasks.workunit.client.1.smithi148.stdout:7/268: unlink d8/d29/f4f 0 2024-03-29T08:07:12.067 INFO:tasks.workunit.client.1.smithi148.stdout:7/269: readlink d8/d29/l3b 0 2024-03-29T08:07:12.071 INFO:tasks.workunit.client.1.smithi148.stdout:1/228: symlink d3/de/d25/l3f 0 2024-03-29T08:07:12.072 INFO:tasks.workunit.client.0.smithi037.stdout:8/240: rename d5/d10/d19/l41 to d5/d10/d19/d3d/d4a/d4c/l4f 0 2024-03-29T08:07:12.075 INFO:tasks.workunit.client.0.smithi037.stdout:8/241: write d5/f1e [1695585,99443] 0 2024-03-29T08:07:12.078 INFO:tasks.workunit.client.0.smithi037.stdout:0/140: rmdir d9/da/dd 39 2024-03-29T08:07:12.082 INFO:tasks.workunit.client.0.smithi037.stdout:9/340: truncate d1/d4/d19/d1e/d25/d16/f56 665613 0 2024-03-29T08:07:12.085 INFO:tasks.workunit.client.0.smithi037.stdout:4/144: creat d3/d4/f34 x:0 0 0 2024-03-29T08:07:12.085 INFO:tasks.workunit.client.1.smithi148.stdout:0/229: sync 2024-03-29T08:07:12.086 INFO:tasks.workunit.client.1.smithi148.stdout:3/231: mknod de/d24/d33/c4c 0 2024-03-29T08:07:12.088 INFO:tasks.workunit.client.0.smithi037.stdout:7/263: dwrite dd/d21/f2c [0,4194304] 0 2024-03-29T08:07:12.090 INFO:tasks.workunit.client.1.smithi148.stdout:7/270: unlink d8/d29/d44/d50/f59 0 2024-03-29T08:07:12.091 INFO:tasks.workunit.client.1.smithi148.stdout:7/271: dread - d8/d14/f1c zero size 2024-03-29T08:07:12.093 INFO:tasks.workunit.client.0.smithi037.stdout:8/242: dread d5/d10/f35 [0,4194304] 0 2024-03-29T08:07:12.094 INFO:tasks.workunit.client.0.smithi037.stdout:9/341: mknod d1/d4/d19/c6e 0 2024-03-29T08:07:12.096 INFO:tasks.workunit.client.1.smithi148.stdout:9/218: rename db/d10/d2d/f31 to db/d10/f46 0 2024-03-29T08:07:12.097 INFO:tasks.workunit.client.0.smithi037.stdout:7/264: read - dd/d21/d35/d42/f43 zero size 2024-03-29T08:07:12.097 INFO:tasks.workunit.client.0.smithi037.stdout:4/145: dread - d3/d4/d15/d1d/fc zero size 2024-03-29T08:07:12.097 INFO:tasks.workunit.client.1.smithi148.stdout:9/219: chown db/d10/d2d/d32 71 1 2024-03-29T08:07:12.099 INFO:tasks.workunit.client.1.smithi148.stdout:0/230: chown db/dc/l15 32807 1 2024-03-29T08:07:12.100 INFO:tasks.workunit.client.0.smithi037.stdout:0/141: write d9/da/dd/d13/f21 [2134091,1008] 0 2024-03-29T08:07:12.101 INFO:tasks.workunit.client.1.smithi148.stdout:9/220: chown db/d22/d26/f2a 394 1 2024-03-29T08:07:12.101 INFO:tasks.workunit.client.0.smithi037.stdout:0/142: chown d9/c1a 344309820 1 2024-03-29T08:07:12.104 INFO:tasks.workunit.client.1.smithi148.stdout:9/221: write db/f43 [280918,97699] 0 2024-03-29T08:07:12.104 INFO:tasks.workunit.client.1.smithi148.stdout:7/272: mknod d8/d38/d3c/c5e 0 2024-03-29T08:07:12.105 INFO:tasks.workunit.client.0.smithi037.stdout:0/143: write d9/da/f25 [883852,62558] 0 2024-03-29T08:07:12.105 INFO:tasks.workunit.client.1.smithi148.stdout:1/229: creat d3/d13/d23/f40 x:0 0 0 2024-03-29T08:07:12.108 INFO:tasks.workunit.client.1.smithi148.stdout:8/282: write d17/d18/d26/d3d/f5b [393783,101588] 0 2024-03-29T08:07:12.108 INFO:tasks.workunit.client.0.smithi037.stdout:7/265: mknod dd/d1d/c57 0 2024-03-29T08:07:12.112 INFO:tasks.workunit.client.1.smithi148.stdout:6/230: write d6/f19 [1966401,80461] 0 2024-03-29T08:07:12.113 INFO:tasks.workunit.client.1.smithi148.stdout:1/230: mknod d3/de/d25/d35/c41 0 2024-03-29T08:07:12.115 INFO:tasks.workunit.client.0.smithi037.stdout:4/146: creat d3/d4/f35 x:0 0 0 2024-03-29T08:07:12.116 INFO:tasks.workunit.client.1.smithi148.stdout:7/273: mknod d8/d29/d44/d49/c5f 0 2024-03-29T08:07:12.118 INFO:tasks.workunit.client.1.smithi148.stdout:1/231: chown d3/de/d25/l3f 58202385 1 2024-03-29T08:07:12.120 INFO:tasks.workunit.client.0.smithi037.stdout:4/147: chown d3/d4/d27/l28 305195 1 2024-03-29T08:07:12.121 INFO:tasks.workunit.client.1.smithi148.stdout:8/283: rmdir d17/d18/d26/d3d 39 2024-03-29T08:07:12.121 INFO:tasks.workunit.client.0.smithi037.stdout:7/266: read dd/d1d/f2a [2584585,84780] 0 2024-03-29T08:07:12.125 INFO:tasks.workunit.client.0.smithi037.stdout:9/342: dwrite d1/d4/f28 [0,4194304] 0 2024-03-29T08:07:12.127 INFO:tasks.workunit.client.1.smithi148.stdout:9/222: dread db/d10/f29 [0,4194304] 0 2024-03-29T08:07:12.127 INFO:tasks.workunit.client.1.smithi148.stdout:4/213: dwrite df/f1e [0,4194304] 0 2024-03-29T08:07:12.127 INFO:tasks.workunit.client.1.smithi148.stdout:0/231: dwrite f6 [4194304,4194304] 0 2024-03-29T08:07:12.130 INFO:tasks.workunit.client.0.smithi037.stdout:4/148: write d3/d4/d15/d1d/fc [1040181,47810] 0 2024-03-29T08:07:12.130 INFO:tasks.workunit.client.1.smithi148.stdout:7/274: mkdir d8/d60 0 2024-03-29T08:07:12.132 INFO:tasks.workunit.client.1.smithi148.stdout:3/232: dwrite de/d24/f31 [0,4194304] 0 2024-03-29T08:07:12.135 INFO:tasks.workunit.client.1.smithi148.stdout:4/214: write df/d2c/f2e [634687,122665] 0 2024-03-29T08:07:12.137 INFO:tasks.workunit.client.0.smithi037.stdout:7/267: fdatasync dd/d21/d35/d42/f43 0 2024-03-29T08:07:12.142 INFO:tasks.workunit.client.1.smithi148.stdout:5/221: dwrite d0/f21 [0,4194304] 0 2024-03-29T08:07:12.143 INFO:tasks.workunit.client.1.smithi148.stdout:2/254: truncate d7/da/d33/d45/d4d/f29 839329 0 2024-03-29T08:07:12.143 INFO:tasks.workunit.client.1.smithi148.stdout:6/231: link d6/d9/d1e/d22/f31 d6/d9/d1e/d20/d30/f3c 0 2024-03-29T08:07:12.144 INFO:tasks.workunit.client.0.smithi037.stdout:4/149: write d3/d4/d15/f1f [767677,16909] 0 2024-03-29T08:07:12.147 INFO:tasks.workunit.client.1.smithi148.stdout:9/223: unlink db/d10/d14/d35/f3a 0 2024-03-29T08:07:12.147 INFO:tasks.workunit.client.0.smithi037.stdout:0/144: dwrite f7 [0,4194304] 0 2024-03-29T08:07:12.147 INFO:tasks.workunit.client.1.smithi148.stdout:5/222: write d0/f21 [579165,76665] 0 2024-03-29T08:07:12.147 INFO:tasks.workunit.client.0.smithi037.stdout:4/150: readlink d3/d4/l26 0 2024-03-29T08:07:12.149 INFO:tasks.workunit.client.1.smithi148.stdout:1/232: rename d3/d13 to d3/de/d25/d35/d42 0 2024-03-29T08:07:12.149 INFO:tasks.workunit.client.0.smithi037.stdout:0/145: readlink d9/da/l1d 0 2024-03-29T08:07:12.150 INFO:tasks.workunit.client.1.smithi148.stdout:9/224: rename db/d22/d26 to db/d22/d26/d47 22 2024-03-29T08:07:12.152 INFO:tasks.workunit.client.0.smithi037.stdout:7/268: symlink dd/l58 0 2024-03-29T08:07:12.153 INFO:tasks.workunit.client.1.smithi148.stdout:4/215: creat df/d10/d17/f4f x:0 0 0 2024-03-29T08:07:12.154 INFO:tasks.workunit.client.0.smithi037.stdout:0/146: write d9/da/d1b/f29 [372623,35558] 0 2024-03-29T08:07:12.159 INFO:tasks.workunit.client.1.smithi148.stdout:4/216: fdatasync df/f14 0 2024-03-29T08:07:12.162 INFO:tasks.workunit.client.1.smithi148.stdout:4/217: chown df/d10/c1b 9590360 1 2024-03-29T08:07:12.167 INFO:tasks.workunit.client.1.smithi148.stdout:2/255: rename d7/d24/f46 to d7/d24/d4b/f55 0 2024-03-29T08:07:12.167 INFO:tasks.workunit.client.1.smithi148.stdout:3/233: creat de/d24/f4d x:0 0 0 2024-03-29T08:07:12.170 INFO:tasks.workunit.client.1.smithi148.stdout:0/232: dwrite db/de/df/d11/d2d/d3e/f45 [0,4194304] 0 2024-03-29T08:07:12.170 INFO:tasks.workunit.client.1.smithi148.stdout:5/223: mknod d0/db/de/c45 0 2024-03-29T08:07:12.171 INFO:tasks.workunit.client.1.smithi148.stdout:4/218: write f4 [148280,25894] 0 2024-03-29T08:07:12.172 INFO:tasks.workunit.client.1.smithi148.stdout:9/225: rmdir db/d22/d39 0 2024-03-29T08:07:12.177 INFO:tasks.workunit.client.1.smithi148.stdout:5/224: chown d0/db/de/c1f 7770481 1 2024-03-29T08:07:12.179 INFO:tasks.workunit.client.1.smithi148.stdout:5/225: fsync d0/f9 0 2024-03-29T08:07:12.182 INFO:tasks.workunit.client.1.smithi148.stdout:2/256: rmdir d7/d39 0 2024-03-29T08:07:12.182 INFO:tasks.workunit.client.1.smithi148.stdout:0/233: creat db/de/f4b x:0 0 0 2024-03-29T08:07:12.184 INFO:tasks.workunit.client.1.smithi148.stdout:7/275: symlink d8/d14/d19/d37/l61 0 2024-03-29T08:07:12.190 INFO:tasks.workunit.client.1.smithi148.stdout:0/234: stat db/de/df/d11/d2d 0 2024-03-29T08:07:12.191 INFO:tasks.workunit.client.0.smithi037.stdout:1/316: sync 2024-03-29T08:07:12.194 INFO:tasks.workunit.client.1.smithi148.stdout:1/233: dwrite d3/de/f1f [0,4194304] 0 2024-03-29T08:07:12.194 INFO:tasks.workunit.client.1.smithi148.stdout:5/226: mkdir d0/d46 0 2024-03-29T08:07:12.198 INFO:tasks.workunit.client.1.smithi148.stdout:1/234: read d3/f7 [2898479,84172] 0 2024-03-29T08:07:12.201 INFO:tasks.workunit.client.1.smithi148.stdout:4/219: rmdir df 39 2024-03-29T08:07:12.202 INFO:tasks.workunit.client.1.smithi148.stdout:5/227: creat d0/db/d1e/d36/f47 x:0 0 0 2024-03-29T08:07:12.202 INFO:tasks.workunit.client.1.smithi148.stdout:8/284: dread fe [0,4194304] 0 2024-03-29T08:07:12.204 INFO:tasks.workunit.client.0.smithi037.stdout:9/343: write d1/d1f/d3a/f27 [2530841,106142] 0 2024-03-29T08:07:12.205 INFO:tasks.workunit.client.1.smithi148.stdout:4/220: fsync f4 0 2024-03-29T08:07:12.206 INFO:tasks.workunit.client.1.smithi148.stdout:5/228: write d0/db/f34 [619361,117513] 0 2024-03-29T08:07:12.207 INFO:tasks.workunit.client.0.smithi037.stdout:7/269: truncate dd/f48 3485362 0 2024-03-29T08:07:12.208 INFO:tasks.workunit.client.1.smithi148.stdout:5/229: stat d0/db/lc 0 2024-03-29T08:07:12.208 INFO:tasks.workunit.client.1.smithi148.stdout:9/226: dwrite db/d10/d14/f3d [0,4194304] 0 2024-03-29T08:07:12.210 INFO:tasks.workunit.client.1.smithi148.stdout:0/235: unlink db/de/df/l2a 0 2024-03-29T08:07:12.211 INFO:tasks.workunit.client.1.smithi148.stdout:9/227: write db/d22/d3f/f42 [992941,103962] 0 2024-03-29T08:07:12.211 INFO:tasks.workunit.client.1.smithi148.stdout:5/230: read d0/db/de/f3b [3694519,101554] 0 2024-03-29T08:07:12.212 INFO:tasks.workunit.client.0.smithi037.stdout:4/151: dread f0 [0,4194304] 0 2024-03-29T08:07:12.212 INFO:tasks.workunit.client.1.smithi148.stdout:7/276: creat d8/d29/d44/d49/f62 x:0 0 0 2024-03-29T08:07:12.213 INFO:tasks.workunit.client.0.smithi037.stdout:9/344: mkdir d1/d4/d19/d1e/d4a/d6f 0 2024-03-29T08:07:12.213 INFO:tasks.workunit.client.1.smithi148.stdout:5/231: chown d0/db/d1e 67648109 1 2024-03-29T08:07:12.214 INFO:tasks.workunit.client.1.smithi148.stdout:1/235: creat d3/de/d21/d32/f43 x:0 0 0 2024-03-29T08:07:12.216 INFO:tasks.workunit.client.1.smithi148.stdout:9/228: write db/f21 [1814620,60468] 0 2024-03-29T08:07:12.216 INFO:tasks.workunit.client.1.smithi148.stdout:5/232: chown d0/f21 13663668 1 2024-03-29T08:07:12.217 INFO:tasks.workunit.client.1.smithi148.stdout:1/236: chown d3/f7 13895044 1 2024-03-29T08:07:12.221 INFO:tasks.workunit.client.0.smithi037.stdout:7/270: creat dd/d1d/d34/f59 x:0 0 0 2024-03-29T08:07:12.222 INFO:tasks.workunit.client.1.smithi148.stdout:3/234: write de/d24/f27 [2091918,31409] 0 2024-03-29T08:07:12.222 INFO:tasks.workunit.client.1.smithi148.stdout:3/235: stat de/l3c 0 2024-03-29T08:07:12.225 INFO:tasks.workunit.client.1.smithi148.stdout:8/285: creat d17/d4e/f5d x:0 0 0 2024-03-29T08:07:12.234 INFO:tasks.workunit.client.1.smithi148.stdout:9/229: read db/f21 [121045,7346] 0 2024-03-29T08:07:12.238 INFO:tasks.workunit.client.0.smithi037.stdout:9/345: truncate d1/d4/d19/d1e/d4a/d5c/d60/f12 6374313 0 2024-03-29T08:07:12.238 INFO:tasks.workunit.client.1.smithi148.stdout:4/221: dread df/d2c/f46 [0,4194304] 0 2024-03-29T08:07:12.240 INFO:tasks.workunit.client.0.smithi037.stdout:4/152: dwrite d3/d4/fa [0,4194304] 0 2024-03-29T08:07:12.246 INFO:tasks.workunit.client.1.smithi148.stdout:2/257: fdatasync d7/da/d33/d45/d4d/f29 0 2024-03-29T08:07:12.247 INFO:tasks.workunit.client.1.smithi148.stdout:0/236: dwrite db/de/df/d11/d2d/f31 [0,4194304] 0 2024-03-29T08:07:12.251 INFO:tasks.workunit.client.1.smithi148.stdout:3/236: dwrite de/d24/f4d [0,4194304] 0 2024-03-29T08:07:12.253 INFO:tasks.workunit.client.0.smithi037.stdout:4/153: mknod d3/d11/c36 0 2024-03-29T08:07:12.254 INFO:tasks.workunit.client.1.smithi148.stdout:5/233: dwrite d0/db/de/d16/d2e/f39 [0,4194304] 0 2024-03-29T08:07:12.254 INFO:tasks.workunit.client.1.smithi148.stdout:1/237: dwrite d3/de/d21/d32/f43 [0,4194304] 0 2024-03-29T08:07:12.255 INFO:tasks.workunit.client.1.smithi148.stdout:5/234: fdatasync d0/f12 0 2024-03-29T08:07:12.257 INFO:tasks.workunit.client.1.smithi148.stdout:5/235: readlink d0/db/d1e/l28 0 2024-03-29T08:07:12.258 INFO:tasks.workunit.client.1.smithi148.stdout:1/238: readlink d3/de/d25/d35/d42/l14 0 2024-03-29T08:07:12.258 INFO:tasks.workunit.client.1.smithi148.stdout:2/258: dread d7/da/d33/d45/d4d/f38 [0,4194304] 0 2024-03-29T08:07:12.263 INFO:tasks.workunit.client.1.smithi148.stdout:8/286: unlink d17/c50 0 2024-03-29T08:07:12.264 INFO:tasks.workunit.client.1.smithi148.stdout:9/230: creat db/d10/d14/d35/f48 x:0 0 0 2024-03-29T08:07:12.264 INFO:tasks.workunit.client.0.smithi037.stdout:4/154: chown d3/d4/f2f 23589 1 2024-03-29T08:07:12.265 INFO:tasks.workunit.client.1.smithi148.stdout:4/222: creat df/d2d/f50 x:0 0 0 2024-03-29T08:07:12.268 INFO:tasks.workunit.client.1.smithi148.stdout:7/277: rename d8/d14/f30 to d8/d14/d19/f63 0 2024-03-29T08:07:12.271 INFO:tasks.workunit.client.1.smithi148.stdout:2/259: dread d7/da/d33/d45/d4d/f16 [0,4194304] 0 2024-03-29T08:07:12.271 INFO:tasks.workunit.client.0.smithi037.stdout:4/155: mknod d3/c37 0 2024-03-29T08:07:12.273 INFO:tasks.workunit.client.0.smithi037.stdout:4/156: chown d3/d4/d15 254818 1 2024-03-29T08:07:12.274 INFO:tasks.workunit.client.0.smithi037.stdout:9/346: dread d1/d4/d19/d1e/d4a/f62 [0,4194304] 0 2024-03-29T08:07:12.275 INFO:tasks.workunit.client.1.smithi148.stdout:7/278: chown d8/d14/f2b 578915969 1 2024-03-29T08:07:12.280 INFO:tasks.workunit.client.1.smithi148.stdout:7/279: dread - d8/d29/d44/d49/f62 zero size 2024-03-29T08:07:12.280 INFO:tasks.workunit.client.1.smithi148.stdout:0/237: mkdir db/de/df/d2b/d3b/d47/d4c 0 2024-03-29T08:07:12.280 INFO:tasks.workunit.client.1.smithi148.stdout:3/237: dwrite de/d24/f36 [0,4194304] 0 2024-03-29T08:07:12.283 INFO:tasks.workunit.client.1.smithi148.stdout:4/223: mkdir df/d2c/d51 0 2024-03-29T08:07:12.287 INFO:tasks.workunit.client.0.smithi037.stdout:9/347: getdents d1/d4/d19/d1e/d4a/d5c/d60/d31 0 2024-03-29T08:07:12.289 INFO:tasks.workunit.client.1.smithi148.stdout:3/238: write de/d24/d33/d39/d46/d30/f3f [1047854,20168] 0 2024-03-29T08:07:12.289 INFO:tasks.workunit.client.1.smithi148.stdout:7/280: truncate d8/f36 744381 0 2024-03-29T08:07:12.293 INFO:tasks.workunit.client.0.smithi037.stdout:4/157: rmdir d3/d11 39 2024-03-29T08:07:12.294 INFO:tasks.workunit.client.1.smithi148.stdout:0/238: symlink db/de/df/d11/d2d/l4d 0 2024-03-29T08:07:12.296 INFO:tasks.workunit.client.1.smithi148.stdout:8/287: link d17/d4e/c51 d17/d49/c5e 0 2024-03-29T08:07:12.297 INFO:tasks.workunit.client.1.smithi148.stdout:9/231: link db/d22/d26/f30 db/d10/d2d/d32/d3b/f49 0 2024-03-29T08:07:12.299 INFO:tasks.workunit.client.1.smithi148.stdout:3/239: creat de/d3b/f4e x:0 0 0 2024-03-29T08:07:12.301 INFO:tasks.workunit.client.1.smithi148.stdout:3/240: stat de/l45 0 2024-03-29T08:07:12.302 INFO:tasks.workunit.client.1.smithi148.stdout:4/224: rename df/d10/d26/d34/f38 to df/d10/d26/d34/d3e/f52 0 2024-03-29T08:07:12.303 INFO:tasks.workunit.client.1.smithi148.stdout:7/281: symlink d8/d14/d45/l64 0 2024-03-29T08:07:12.305 INFO:tasks.workunit.client.1.smithi148.stdout:0/239: mknod db/de/df/d2b/d3b/c4e 0 2024-03-29T08:07:12.305 INFO:tasks.workunit.client.1.smithi148.stdout:9/232: rmdir db/d22 39 2024-03-29T08:07:12.307 INFO:tasks.workunit.client.1.smithi148.stdout:3/241: symlink de/d3b/l4f 0 2024-03-29T08:07:12.308 INFO:tasks.workunit.client.1.smithi148.stdout:8/288: mkdir d17/d1d/d59/d5f 0 2024-03-29T08:07:12.312 INFO:tasks.workunit.client.1.smithi148.stdout:6/232: sync 2024-03-29T08:07:12.314 INFO:tasks.workunit.client.1.smithi148.stdout:2/260: dwrite d7/da/d33/d3c/d21/f25 [4194304,4194304] 0 2024-03-29T08:07:12.315 INFO:tasks.workunit.client.0.smithi037.stdout:4/158: dwrite d3/d4/d15/f1f [0,4194304] 0 2024-03-29T08:07:12.315 INFO:tasks.workunit.client.0.smithi037.stdout:9/348: dwrite d1/d4/d19/d1e/d4a/d5c/d60/d31/f48 [0,4194304] 0 2024-03-29T08:07:12.318 INFO:tasks.workunit.client.1.smithi148.stdout:8/289: stat d17/d18/l56 0 2024-03-29T08:07:12.318 INFO:tasks.workunit.client.1.smithi148.stdout:1/239: dwrite d3/de/d25/d35/d42/f38 [16777216,4194304] 0 2024-03-29T08:07:12.319 INFO:tasks.workunit.client.1.smithi148.stdout:5/236: dread d0/db/f10 [0,4194304] 0 2024-03-29T08:07:12.319 INFO:tasks.workunit.client.1.smithi148.stdout:6/233: creat d6/d9/d1e/f3d x:0 0 0 2024-03-29T08:07:12.322 INFO:tasks.workunit.client.1.smithi148.stdout:9/233: truncate db/d22/d26/f38 987462 0 2024-03-29T08:07:12.323 INFO:tasks.workunit.client.1.smithi148.stdout:3/242: truncate de/d24/d33/d39/d46/f21 5061383 0 2024-03-29T08:07:12.324 INFO:tasks.workunit.client.0.smithi037.stdout:9/349: mknod d1/d1f/d3a/c70 0 2024-03-29T08:07:12.327 INFO:tasks.workunit.client.1.smithi148.stdout:5/237: readlink d0/db/de/d16/l19 0 2024-03-29T08:07:12.328 INFO:tasks.workunit.client.1.smithi148.stdout:2/261: mkdir d7/da/d33/d3c/d21/d56 0 2024-03-29T08:07:12.330 INFO:tasks.workunit.client.1.smithi148.stdout:7/282: dwrite d8/f36 [0,4194304] 0 2024-03-29T08:07:12.330 INFO:tasks.workunit.client.1.smithi148.stdout:1/240: mkdir d3/de/d21/d32/d44 0 2024-03-29T08:07:12.330 INFO:tasks.workunit.client.0.smithi037.stdout:9/350: mkdir d1/d4/d19/d1e/d71 0 2024-03-29T08:07:12.333 INFO:tasks.workunit.client.0.smithi037.stdout:9/351: unlink d1/d4/c55 0 2024-03-29T08:07:12.334 INFO:tasks.workunit.client.1.smithi148.stdout:6/234: mknod d6/d9/d1e/d22/c3e 0 2024-03-29T08:07:12.335 INFO:tasks.workunit.client.0.smithi037.stdout:9/352: chown d1/f4c 1078337 1 2024-03-29T08:07:12.335 INFO:tasks.workunit.client.1.smithi148.stdout:8/290: mkdir d17/d60 0 2024-03-29T08:07:12.336 INFO:tasks.workunit.client.1.smithi148.stdout:4/225: dwrite df/d10/d17/f25 [0,4194304] 0 2024-03-29T08:07:12.337 INFO:tasks.workunit.client.1.smithi148.stdout:5/238: truncate d0/db/d1e/d36/f47 320031 0 2024-03-29T08:07:12.339 INFO:tasks.workunit.client.1.smithi148.stdout:1/241: readlink d3/d18/l2e 0 2024-03-29T08:07:12.343 INFO:tasks.workunit.client.1.smithi148.stdout:7/283: mknod d8/d14/d45/c65 0 2024-03-29T08:07:12.346 INFO:tasks.workunit.client.1.smithi148.stdout:8/291: fdatasync d17/d18/d38/f4b 0 2024-03-29T08:07:12.347 INFO:tasks.workunit.client.1.smithi148.stdout:5/239: mknod d0/db/d1e/d35/c48 0 2024-03-29T08:07:12.347 INFO:tasks.workunit.client.1.smithi148.stdout:0/240: write db/f23 [5034779,594] 0 2024-03-29T08:07:12.351 INFO:tasks.workunit.client.1.smithi148.stdout:7/284: readlink d8/d38/d3c/l46 0 2024-03-29T08:07:12.353 INFO:tasks.workunit.client.1.smithi148.stdout:1/242: mknod d3/de/d21/d32/c45 0 2024-03-29T08:07:12.353 INFO:tasks.workunit.client.1.smithi148.stdout:7/285: fdatasync d8/d14/f2c 0 2024-03-29T08:07:12.354 INFO:tasks.workunit.client.1.smithi148.stdout:5/240: chown d0/db/d1e 228979159 1 2024-03-29T08:07:12.356 INFO:tasks.workunit.client.1.smithi148.stdout:3/243: link de/c22 de/d24/d33/d39/c50 0 2024-03-29T08:07:12.357 INFO:tasks.workunit.client.0.smithi037.stdout:9/353: dwrite d1/d4/d19/d1e/d4a/d5c/d60/d23/f37 [0,4194304] 0 2024-03-29T08:07:12.357 INFO:tasks.workunit.client.1.smithi148.stdout:7/286: dread - d8/d38/f43 zero size 2024-03-29T08:07:12.357 INFO:tasks.workunit.client.1.smithi148.stdout:9/234: rename db/d10/d14/c17 to db/d10/d2d/d32/c4a 0 2024-03-29T08:07:12.358 INFO:tasks.workunit.client.1.smithi148.stdout:6/235: creat d6/d9/f3f x:0 0 0 2024-03-29T08:07:12.359 INFO:tasks.workunit.client.1.smithi148.stdout:4/226: creat df/d10/f53 x:0 0 0 2024-03-29T08:07:12.360 INFO:tasks.workunit.client.1.smithi148.stdout:3/244: write de/f2c [343843,114366] 0 2024-03-29T08:07:12.364 INFO:tasks.workunit.client.1.smithi148.stdout:1/243: write d3/d18/f39 [197927,69741] 0 2024-03-29T08:07:12.366 INFO:tasks.workunit.client.0.smithi037.stdout:9/354: creat d1/d4/d19/d1e/d4a/d5c/d60/d23/f72 x:0 0 0 2024-03-29T08:07:12.369 INFO:tasks.workunit.client.1.smithi148.stdout:0/241: getdents db/de 0 2024-03-29T08:07:12.369 INFO:tasks.workunit.client.1.smithi148.stdout:5/241: rename d0/db/d1e/l3f to d0/l49 0 2024-03-29T08:07:12.370 INFO:tasks.workunit.client.0.smithi037.stdout:9/355: stat d1/d1f/d3a/d67 0 2024-03-29T08:07:12.370 INFO:tasks.workunit.client.1.smithi148.stdout:3/245: creat de/f51 x:0 0 0 2024-03-29T08:07:12.370 INFO:tasks.workunit.client.1.smithi148.stdout:7/287: write d8/f1f [971241,49174] 0 2024-03-29T08:07:12.371 INFO:tasks.workunit.client.1.smithi148.stdout:6/236: fsync d6/d9/f2c 0 2024-03-29T08:07:12.373 INFO:tasks.workunit.client.0.smithi037.stdout:4/159: dread d3/d4/d15/d1d/db/f16 [0,4194304] 0 2024-03-29T08:07:12.375 INFO:tasks.workunit.client.1.smithi148.stdout:6/237: stat d6/f19 0 2024-03-29T08:07:12.376 INFO:tasks.workunit.client.1.smithi148.stdout:4/227: rename df/d2d/c41 to df/d2c/c54 0 2024-03-29T08:07:12.379 INFO:tasks.workunit.client.0.smithi037.stdout:4/160: mknod d3/d11/c38 0 2024-03-29T08:07:12.380 INFO:tasks.workunit.client.1.smithi148.stdout:2/262: dread d7/da/fe [0,4194304] 0 2024-03-29T08:07:12.380 INFO:tasks.workunit.client.1.smithi148.stdout:1/244: truncate d3/d18/f20 201329 0 2024-03-29T08:07:12.380 INFO:tasks.workunit.client.1.smithi148.stdout:6/238: fsync d6/d9/d1e/d20/d30/f35 0 2024-03-29T08:07:12.383 INFO:tasks.workunit.client.0.smithi037.stdout:4/161: truncate d3/d4/d27/f2e 979666 0 2024-03-29T08:07:12.384 INFO:tasks.workunit.client.1.smithi148.stdout:5/242: fsync d0/db/de/d16/f3d 0 2024-03-29T08:07:12.389 INFO:tasks.workunit.client.1.smithi148.stdout:7/288: write d8/f10 [3788322,96467] 0 2024-03-29T08:07:12.389 INFO:tasks.workunit.client.1.smithi148.stdout:3/246: symlink de/d3b/d49/l52 0 2024-03-29T08:07:12.390 INFO:tasks.workunit.client.1.smithi148.stdout:9/235: dwrite db/d22/d26/f38 [0,4194304] 0 2024-03-29T08:07:12.397 INFO:tasks.workunit.client.1.smithi148.stdout:5/243: creat d0/db/d1e/f4a x:0 0 0 2024-03-29T08:07:12.397 INFO:tasks.workunit.client.1.smithi148.stdout:1/245: write d3/de/d25/d35/d42/f1a [2008238,125270] 0 2024-03-29T08:07:12.402 INFO:tasks.workunit.client.1.smithi148.stdout:9/236: fdatasync db/d10/d14/d35/f48 0 2024-03-29T08:07:12.402 INFO:tasks.workunit.client.1.smithi148.stdout:2/263: getdents d7/da/d33/d45/d4d/d41 0 2024-03-29T08:07:12.411 INFO:tasks.workunit.client.0.smithi037.stdout:4/162: dwrite f0 [0,4194304] 0 2024-03-29T08:07:12.412 INFO:tasks.workunit.client.0.smithi037.stdout:4/163: chown d3/d11 963 1 2024-03-29T08:07:12.414 INFO:tasks.workunit.client.0.smithi037.stdout:4/164: rmdir d3/d4/d15/d1d 39 2024-03-29T08:07:12.415 INFO:tasks.workunit.client.1.smithi148.stdout:4/228: dwrite df/d10/d17/f42 [4194304,4194304] 0 2024-03-29T08:07:12.416 INFO:tasks.workunit.client.1.smithi148.stdout:6/239: dwrite d6/d9/f3f [0,4194304] 0 2024-03-29T08:07:12.422 INFO:tasks.workunit.client.1.smithi148.stdout:3/247: symlink de/d24/d33/d39/d46/d30/d37/l53 0 2024-03-29T08:07:12.425 INFO:tasks.workunit.client.1.smithi148.stdout:4/229: write df/d10/d26/d34/d43/f4e [192692,80926] 0 2024-03-29T08:07:12.427 INFO:tasks.workunit.client.1.smithi148.stdout:0/242: truncate db/de/df/d11/f16 2044325 0 2024-03-29T08:07:12.431 INFO:tasks.workunit.client.0.smithi037.stdout:4/165: dwrite d3/d4/fa [4194304,4194304] 0 2024-03-29T08:07:12.432 INFO:tasks.workunit.client.1.smithi148.stdout:5/244: link d0/db/de/d16/f2f d0/f4b 0 2024-03-29T08:07:12.437 INFO:tasks.workunit.client.1.smithi148.stdout:4/230: rmdir df/d10/d26/d34/d43 39 2024-03-29T08:07:12.438 INFO:tasks.workunit.client.1.smithi148.stdout:3/248: mknod de/d24/d33/d39/d46/d30/d37/c54 0 2024-03-29T08:07:12.439 INFO:tasks.workunit.client.1.smithi148.stdout:8/292: sync 2024-03-29T08:07:12.440 INFO:tasks.workunit.client.1.smithi148.stdout:0/243: mknod db/de/df/d2b/d3b/d47/c4f 0 2024-03-29T08:07:12.441 INFO:tasks.workunit.client.1.smithi148.stdout:9/237: dwrite db/d22/d26/f2a [0,4194304] 0 2024-03-29T08:07:12.441 INFO:tasks.workunit.client.1.smithi148.stdout:5/245: mknod d0/db/d1e/d35/d42/c4c 0 2024-03-29T08:07:12.443 INFO:tasks.workunit.client.1.smithi148.stdout:3/249: chown de/d24/f2b 3274759 1 2024-03-29T08:07:12.445 INFO:tasks.workunit.client.0.smithi037.stdout:4/166: dread d3/d4/d15/d1d/fc [0,4194304] 0 2024-03-29T08:07:12.446 INFO:tasks.workunit.client.1.smithi148.stdout:9/238: truncate db/d10/f46 1108695 0 2024-03-29T08:07:12.446 INFO:tasks.workunit.client.1.smithi148.stdout:3/250: readlink de/l14 0 2024-03-29T08:07:12.449 INFO:tasks.workunit.client.1.smithi148.stdout:3/251: fsync de/f2e 0 2024-03-29T08:07:12.449 INFO:tasks.workunit.client.0.smithi037.stdout:9/356: truncate d1/d4/d19/d1e/d4a/d5c/d60/f12 532224 0 2024-03-29T08:07:12.452 INFO:tasks.workunit.client.1.smithi148.stdout:8/293: symlink d17/d18/d26/d3d/l61 0 2024-03-29T08:07:12.455 INFO:tasks.workunit.client.1.smithi148.stdout:5/246: rename d0/db/c20 to d0/db/d1e/d36/c4d 0 2024-03-29T08:07:12.457 INFO:tasks.workunit.client.0.smithi037.stdout:4/167: symlink d3/l39 0 2024-03-29T08:07:12.461 INFO:tasks.workunit.client.1.smithi148.stdout:6/240: dread d6/f1a [0,4194304] 0 2024-03-29T08:07:12.462 INFO:tasks.workunit.client.1.smithi148.stdout:0/244: dread db/de/df/f30 [0,4194304] 0 2024-03-29T08:07:12.462 INFO:tasks.workunit.client.1.smithi148.stdout:9/239: mkdir db/d10/d14/d35/d4b 0 2024-03-29T08:07:12.467 INFO:tasks.workunit.client.1.smithi148.stdout:4/231: symlink df/d10/d26/d34/l55 0 2024-03-29T08:07:12.472 INFO:tasks.workunit.client.1.smithi148.stdout:5/247: readlink d0/db/d1e/d36/d41/l33 0 2024-03-29T08:07:12.472 INFO:tasks.workunit.client.1.smithi148.stdout:6/241: rmdir d6/d9/d1e/d22/d28 39 2024-03-29T08:07:12.473 INFO:tasks.workunit.client.0.smithi037.stdout:4/168: dread d3/d4/d15/f29 [0,4194304] 0 2024-03-29T08:07:12.473 INFO:tasks.workunit.client.1.smithi148.stdout:9/240: mknod db/d10/d14/c4c 0 2024-03-29T08:07:12.475 INFO:tasks.workunit.client.0.smithi037.stdout:9/357: dread d1/d3/f21 [0,4194304] 0 2024-03-29T08:07:12.478 INFO:tasks.workunit.client.1.smithi148.stdout:3/252: dwrite de/f2f [0,4194304] 0 2024-03-29T08:07:12.492 INFO:tasks.workunit.client.1.smithi148.stdout:4/232: link df/d2d/f49 df/d10/f56 0 2024-03-29T08:07:12.494 INFO:tasks.workunit.client.1.smithi148.stdout:3/253: symlink de/d3b/l55 0 2024-03-29T08:07:12.496 INFO:tasks.workunit.client.0.smithi037.stdout:4/169: write d3/d4/d15/f19 [155698,51981] 0 2024-03-29T08:07:12.496 INFO:tasks.workunit.client.0.smithi037.stdout:9/358: truncate d1/d4/d19/d1e/d4a/d5c/d60/d31/f48 1537932 0 2024-03-29T08:07:12.496 INFO:tasks.workunit.client.1.smithi148.stdout:9/241: rmdir db 39 2024-03-29T08:07:12.497 INFO:tasks.workunit.client.0.smithi037.stdout:9/359: stat d1/lb 0 2024-03-29T08:07:12.498 INFO:tasks.workunit.client.1.smithi148.stdout:7/289: dwrite d8/fc [0,4194304] 0 2024-03-29T08:07:12.500 INFO:tasks.workunit.client.0.smithi037.stdout:4/170: dread - d3/d11/f2d zero size 2024-03-29T08:07:12.501 INFO:tasks.workunit.client.1.smithi148.stdout:2/264: dwrite d7/da/d33/d45/d4d/f38 [0,4194304] 0 2024-03-29T08:07:12.501 INFO:tasks.workunit.client.1.smithi148.stdout:8/294: dwrite d17/d18/d26/d3d/f43 [0,4194304] 0 2024-03-29T08:07:12.501 INFO:tasks.workunit.client.0.smithi037.stdout:9/360: getdents d1/d4/d19/d1e/d71 0 2024-03-29T08:07:12.502 INFO:tasks.workunit.client.0.smithi037.stdout:5/341: sync 2024-03-29T08:07:12.503 INFO:tasks.workunit.client.1.smithi148.stdout:3/254: fdatasync de/d24/f31 0 2024-03-29T08:07:12.505 INFO:tasks.workunit.client.0.smithi037.stdout:4/171: write d3/f21 [3787859,36839] 0 2024-03-29T08:07:12.509 INFO:tasks.workunit.client.1.smithi148.stdout:1/246: truncate d3/d18/f20 476268 0 2024-03-29T08:07:12.511 INFO:tasks.workunit.client.0.smithi037.stdout:9/361: mknod d1/d1f/c73 0 2024-03-29T08:07:12.515 INFO:tasks.workunit.client.1.smithi148.stdout:9/242: fdatasync db/d22/d26/f38 0 2024-03-29T08:07:12.516 INFO:tasks.workunit.client.0.smithi037.stdout:9/362: creat d1/f74 x:0 0 0 2024-03-29T08:07:12.518 INFO:tasks.workunit.client.1.smithi148.stdout:4/233: creat df/d2c/d51/f57 x:0 0 0 2024-03-29T08:07:12.519 INFO:tasks.workunit.client.1.smithi148.stdout:8/295: mkdir d17/d62 0 2024-03-29T08:07:12.521 INFO:tasks.workunit.client.1.smithi148.stdout:4/234: write fd [4310995,42420] 0 2024-03-29T08:07:12.524 INFO:tasks.workunit.client.0.smithi037.stdout:9/363: mkdir d1/d4/d19/d69/d75 0 2024-03-29T08:07:12.526 INFO:tasks.workunit.client.0.smithi037.stdout:4/172: dread d3/d4/fa [4194304,4194304] 0 2024-03-29T08:07:12.527 INFO:tasks.workunit.client.1.smithi148.stdout:7/290: getdents d8/d14/d45 0 2024-03-29T08:07:12.528 INFO:tasks.workunit.client.1.smithi148.stdout:6/242: dread d6/d9/d1e/d20/f39 [0,4194304] 0 2024-03-29T08:07:12.529 INFO:tasks.workunit.client.1.smithi148.stdout:3/255: creat de/d24/f56 x:0 0 0 2024-03-29T08:07:12.530 INFO:tasks.workunit.client.0.smithi037.stdout:9/364: mknod d1/d4/d19/d69/d75/c76 0 2024-03-29T08:07:12.532 INFO:tasks.workunit.client.1.smithi148.stdout:5/248: dwrite d0/db/d1e/d35/d42/f43 [0,4194304] 0 2024-03-29T08:07:12.535 INFO:tasks.workunit.client.0.smithi037.stdout:4/173: creat d3/f3a x:0 0 0 2024-03-29T08:07:12.536 INFO:tasks.workunit.client.1.smithi148.stdout:7/291: write d8/d14/f47 [916427,70645] 0 2024-03-29T08:07:12.537 INFO:tasks.workunit.client.1.smithi148.stdout:1/247: getdents d3/de/d25/d35/d42 0 2024-03-29T08:07:12.537 INFO:tasks.workunit.client.0.smithi037.stdout:5/342: dwrite d8/d22/d4f/f5e [0,4194304] 0 2024-03-29T08:07:12.537 INFO:tasks.workunit.client.1.smithi148.stdout:0/245: dwrite db/de/df/d11/f37 [0,4194304] 0 2024-03-29T08:07:12.542 INFO:tasks.workunit.client.0.smithi037.stdout:4/174: chown d3/d11/d1e/l24 49883 1 2024-03-29T08:07:12.548 INFO:tasks.workunit.client.1.smithi148.stdout:4/235: creat df/d10/d26/d34/d3e/f58 x:0 0 0 2024-03-29T08:07:12.548 INFO:tasks.workunit.client.1.smithi148.stdout:8/296: rename d17/d62 to d17/d18/d26/d63 0 2024-03-29T08:07:12.550 INFO:tasks.workunit.client.1.smithi148.stdout:1/248: mknod d3/de/d21/d32/c46 0 2024-03-29T08:07:12.551 INFO:tasks.workunit.client.1.smithi148.stdout:2/265: dwrite d7/da/d33/f3e [0,4194304] 0 2024-03-29T08:07:12.552 INFO:tasks.workunit.client.1.smithi148.stdout:0/246: mknod db/de/df/d11/d2d/d3e/c50 0 2024-03-29T08:07:12.552 INFO:tasks.workunit.client.0.smithi037.stdout:4/175: mknod d3/d4/d15/d1d/db/c3b 0 2024-03-29T08:07:12.555 INFO:tasks.workunit.client.1.smithi148.stdout:9/243: dread db/d22/d26/f2a [0,4194304] 0 2024-03-29T08:07:12.558 INFO:tasks.workunit.client.1.smithi148.stdout:3/256: rename de/d3b/l55 to de/d24/d33/d39/d46/l57 0 2024-03-29T08:07:12.559 INFO:tasks.workunit.client.1.smithi148.stdout:7/292: link d8/d14/f2b d8/d14/d45/f66 0 2024-03-29T08:07:12.559 INFO:tasks.workunit.client.1.smithi148.stdout:5/249: link d0/db/d1e/d36/d41/l38 d0/d46/l4e 0 2024-03-29T08:07:12.561 INFO:tasks.workunit.client.1.smithi148.stdout:8/297: mkdir d17/d18/d26/d3d/d64 0 2024-03-29T08:07:12.563 INFO:tasks.workunit.client.0.smithi037.stdout:4/176: unlink d3/d4/fa 0 2024-03-29T08:07:12.565 INFO:tasks.workunit.client.1.smithi148.stdout:0/247: unlink db/dc/f12 0 2024-03-29T08:07:12.565 INFO:tasks.workunit.client.1.smithi148.stdout:2/266: read d7/da/f49 [621871,42632] 0 2024-03-29T08:07:12.566 INFO:tasks.workunit.client.1.smithi148.stdout:1/249: mkdir d3/de/d1d/d47 0 2024-03-29T08:07:12.566 INFO:tasks.workunit.client.1.smithi148.stdout:6/243: dread d6/fa [0,4194304] 0 2024-03-29T08:07:12.567 INFO:tasks.workunit.client.1.smithi148.stdout:4/236: creat df/d10/d26/d34/d43/f59 x:0 0 0 2024-03-29T08:07:12.568 INFO:tasks.workunit.client.0.smithi037.stdout:9/365: rename d1/d4/d19/d1e/d4a/d5c/d60/d31/c52 to d1/d4/d19/d1e/d4a/d5c/d60/d13/c77 0 2024-03-29T08:07:12.568 INFO:tasks.workunit.client.1.smithi148.stdout:3/257: creat de/d3b/f58 x:0 0 0 2024-03-29T08:07:12.570 INFO:tasks.workunit.client.1.smithi148.stdout:9/244: rename db/d10/d14 to db/d10/d2d/d32/d3b/d4d 0 2024-03-29T08:07:12.571 INFO:tasks.workunit.client.1.smithi148.stdout:5/250: symlink d0/d46/l4f 0 2024-03-29T08:07:12.574 INFO:tasks.workunit.client.0.smithi037.stdout:9/366: read d1/d4/d19/d1e/d4a/d5c/d60/f3d [376962,26903] 0 2024-03-29T08:07:12.575 INFO:tasks.workunit.client.1.smithi148.stdout:4/237: truncate df/d10/d17/f25 5227334 0 2024-03-29T08:07:12.576 INFO:tasks.workunit.client.1.smithi148.stdout:0/248: write db/de/df/d2b/d3b/d47/f2c [449595,123691] 0 2024-03-29T08:07:12.577 INFO:tasks.workunit.client.1.smithi148.stdout:9/245: fsync db/d10/d2d/d32/d3b/f36 0 2024-03-29T08:07:12.579 INFO:tasks.workunit.client.1.smithi148.stdout:8/298: link d17/c41 d17/d18/d26/d4f/c65 0 2024-03-29T08:07:12.580 INFO:tasks.workunit.client.0.smithi037.stdout:4/177: rename d3/d4/l26 to d3/d4/d15/d1d/l3c 0 2024-03-29T08:07:12.581 INFO:tasks.workunit.client.1.smithi148.stdout:4/238: write df/f1f [593420,87048] 0 2024-03-29T08:07:12.584 INFO:tasks.workunit.client.0.smithi037.stdout:4/178: mkdir d3/d4/d15/d1d/db/d3d 0 2024-03-29T08:07:12.587 INFO:tasks.workunit.client.1.smithi148.stdout:3/258: rename de/c29 to de/d24/c59 0 2024-03-29T08:07:12.588 INFO:tasks.workunit.client.0.smithi037.stdout:4/179: creat d3/d4/f3e x:0 0 0 2024-03-29T08:07:12.590 INFO:tasks.workunit.client.1.smithi148.stdout:9/246: mkdir db/d10/d2d/d4e 0 2024-03-29T08:07:12.593 INFO:tasks.workunit.client.0.smithi037.stdout:9/367: dread d1/d4/d19/d1e/d4a/d5c/d60/f3d [0,4194304] 0 2024-03-29T08:07:12.593 INFO:tasks.workunit.client.1.smithi148.stdout:1/250: dread d3/de/d25/d35/d42/f1a [0,4194304] 0 2024-03-29T08:07:12.593 INFO:tasks.workunit.client.1.smithi148.stdout:9/247: write db/d22/d26/f38 [2274961,63700] 0 2024-03-29T08:07:12.593 INFO:tasks.workunit.client.1.smithi148.stdout:5/251: creat d0/db/de/f50 x:0 0 0 2024-03-29T08:07:12.594 INFO:tasks.workunit.client.1.smithi148.stdout:6/244: link d6/d9/l25 d6/l40 0 2024-03-29T08:07:12.594 INFO:tasks.workunit.client.0.smithi037.stdout:4/180: creat d3/d4/d15/d1d/f3f x:0 0 0 2024-03-29T08:07:12.595 INFO:tasks.workunit.client.0.smithi037.stdout:4/181: readlink d3/d4/d15/d1d/l17 0 2024-03-29T08:07:12.598 INFO:tasks.workunit.client.1.smithi148.stdout:6/245: read d6/d9/f16 [3979531,101405] 0 2024-03-29T08:07:12.599 INFO:tasks.workunit.client.1.smithi148.stdout:8/299: rename d17/d1d/d59 to d17/d18/d26/d63/d66 0 2024-03-29T08:07:12.601 INFO:tasks.workunit.client.1.smithi148.stdout:0/249: link db/dc/f1e db/f51 0 2024-03-29T08:07:12.601 INFO:tasks.workunit.client.0.smithi037.stdout:4/182: rename d3/d4/d27/c2c to d3/d4/d27/c40 0 2024-03-29T08:07:12.602 INFO:tasks.workunit.client.1.smithi148.stdout:0/250: stat db/dc/l3a 0 2024-03-29T08:07:12.603 INFO:tasks.workunit.client.1.smithi148.stdout:3/259: creat de/d24/d33/d39/f5a x:0 0 0 2024-03-29T08:07:12.604 INFO:tasks.workunit.client.1.smithi148.stdout:0/251: stat db/de/df/d2b/d3b/d47/c4f 0 2024-03-29T08:07:12.607 INFO:tasks.workunit.client.1.smithi148.stdout:1/251: write d3/f7 [2242342,114381] 0 2024-03-29T08:07:12.608 INFO:tasks.workunit.client.1.smithi148.stdout:0/252: fsync db/de/f25 0 2024-03-29T08:07:12.609 INFO:tasks.workunit.client.1.smithi148.stdout:1/252: stat d3/de/d25/l3f 0 2024-03-29T08:07:12.610 INFO:tasks.workunit.client.0.smithi037.stdout:4/183: link d3/c6 d3/d4/d15/d1d/db/c41 0 2024-03-29T08:07:12.611 INFO:tasks.workunit.client.1.smithi148.stdout:9/248: rename db/d10/l28 to db/d10/d2d/d4e/l4f 0 2024-03-29T08:07:12.611 INFO:tasks.workunit.client.1.smithi148.stdout:3/260: dread de/d24/d33/d39/d46/f21 [4194304,4194304] 0 2024-03-29T08:07:12.612 INFO:tasks.workunit.client.0.smithi037.stdout:9/368: write d1/d4/d19/d1e/d25/f34 [2254014,33889] 0 2024-03-29T08:07:12.614 INFO:tasks.workunit.client.1.smithi148.stdout:2/267: write d7/da/d33/d45/d4d/f2a [396866,62926] 0 2024-03-29T08:07:12.615 INFO:tasks.workunit.client.0.smithi037.stdout:4/184: mknod d3/d4/d15/d1d/db/c42 0 2024-03-29T08:07:12.615 INFO:tasks.workunit.client.1.smithi148.stdout:8/300: dread d17/d18/d26/d3d/f45 [0,4194304] 0 2024-03-29T08:07:12.620 INFO:tasks.workunit.client.0.smithi037.stdout:4/185: write d3/d4/d15/f29 [2168489,73111] 0 2024-03-29T08:07:12.620 INFO:tasks.workunit.client.1.smithi148.stdout:5/252: fsync d0/db/de/f50 0 2024-03-29T08:07:12.621 INFO:tasks.workunit.client.0.smithi037.stdout:9/369: link d1/d4/d19/d1e/d4a/d5c/d60/f12 d1/d1f/d3a/f78 0 2024-03-29T08:07:12.621 INFO:tasks.workunit.client.1.smithi148.stdout:1/253: creat d3/de/d25/d35/f48 x:0 0 0 2024-03-29T08:07:12.623 INFO:tasks.workunit.client.0.smithi037.stdout:4/186: chown d3/d4/d27/f2e 38 1 2024-03-29T08:07:12.625 INFO:tasks.workunit.client.1.smithi148.stdout:3/261: rmdir de/d24/d33 39 2024-03-29T08:07:12.626 INFO:tasks.workunit.client.0.smithi037.stdout:9/370: readlink d1/d4/d19/d1e/d4a/d5c/d60/d13/l5b 0 2024-03-29T08:07:12.627 INFO:tasks.workunit.client.1.smithi148.stdout:3/262: write de/d24/f31 [1220243,128069] 0 2024-03-29T08:07:12.629 INFO:tasks.workunit.client.0.smithi037.stdout:9/371: mknod d1/d4/d19/c79 0 2024-03-29T08:07:12.630 INFO:tasks.workunit.client.1.smithi148.stdout:4/239: dwrite df/d10/f56 [0,4194304] 0 2024-03-29T08:07:12.634 INFO:tasks.workunit.client.1.smithi148.stdout:5/253: creat d0/db/d1e/d36/d41/f51 x:0 0 0 2024-03-29T08:07:12.635 INFO:tasks.workunit.client.1.smithi148.stdout:5/254: readlink d0/l8 0 2024-03-29T08:07:12.635 INFO:tasks.workunit.client.1.smithi148.stdout:1/254: symlink d3/de/d25/d35/l49 0 2024-03-29T08:07:12.636 INFO:tasks.workunit.client.1.smithi148.stdout:9/249: mknod db/d10/d2d/d32/c50 0 2024-03-29T08:07:12.640 INFO:tasks.workunit.client.1.smithi148.stdout:5/255: dread d0/db/d1e/d36/f47 [0,4194304] 0 2024-03-29T08:07:12.645 INFO:tasks.workunit.client.0.smithi037.stdout:4/187: dwrite d3/d4/f35 [0,4194304] 0 2024-03-29T08:07:12.647 INFO:tasks.workunit.client.1.smithi148.stdout:4/240: symlink df/d2c/l5a 0 2024-03-29T08:07:12.649 INFO:tasks.workunit.client.1.smithi148.stdout:9/250: dread db/d22/d26/f38 [0,4194304] 0 2024-03-29T08:07:12.650 INFO:tasks.workunit.client.1.smithi148.stdout:9/251: stat db/d20/f1e 0 2024-03-29T08:07:12.651 INFO:tasks.workunit.client.1.smithi148.stdout:0/253: readlink db/dc/l15 0 2024-03-29T08:07:12.654 INFO:tasks.workunit.client.0.smithi037.stdout:4/188: mkdir d3/d4/d15/d43 0 2024-03-29T08:07:12.658 INFO:tasks.workunit.client.1.smithi148.stdout:2/268: creat d7/da/d33/d45/d4d/f57 x:0 0 0 2024-03-29T08:07:12.658 INFO:tasks.workunit.client.0.smithi037.stdout:4/189: creat d3/f44 x:0 0 0 2024-03-29T08:07:12.659 INFO:tasks.workunit.client.1.smithi148.stdout:8/301: creat d17/d18/f67 x:0 0 0 2024-03-29T08:07:12.660 INFO:tasks.workunit.client.0.smithi037.stdout:4/190: readlink d3/d11/d1e/l24 0 2024-03-29T08:07:12.661 INFO:tasks.workunit.client.1.smithi148.stdout:4/241: write f7 [3488013,114751] 0 2024-03-29T08:07:12.661 INFO:tasks.workunit.client.0.smithi037.stdout:4/191: chown d3/d4/d15 82 1 2024-03-29T08:07:12.662 INFO:tasks.workunit.client.1.smithi148.stdout:1/255: mknod d3/de/c4a 0 2024-03-29T08:07:12.663 INFO:tasks.workunit.client.0.smithi037.stdout:4/192: write d3/d4/f2f [117299,5618] 0 2024-03-29T08:07:12.664 INFO:tasks.workunit.client.1.smithi148.stdout:9/252: creat db/d22/f51 x:0 0 0 2024-03-29T08:07:12.664 INFO:tasks.workunit.client.0.smithi037.stdout:9/372: dread d1/d4/f3b [0,4194304] 0 2024-03-29T08:07:12.665 INFO:tasks.workunit.client.0.smithi037.stdout:4/193: dread - d3/d11/f30 zero size 2024-03-29T08:07:12.665 INFO:tasks.workunit.client.1.smithi148.stdout:2/269: mknod d7/da/d33/d45/c58 0 2024-03-29T08:07:12.666 INFO:tasks.workunit.client.1.smithi148.stdout:8/302: truncate d17/d18/d38/f5a 792683 0 2024-03-29T08:07:12.667 INFO:tasks.workunit.client.1.smithi148.stdout:8/303: fdatasync d17/f1a 0 2024-03-29T08:07:12.668 INFO:tasks.workunit.client.0.smithi037.stdout:4/194: write d3/d4/d15/f1f [2094060,50166] 0 2024-03-29T08:07:12.669 INFO:tasks.workunit.client.1.smithi148.stdout:2/270: chown c0 542812 1 2024-03-29T08:07:12.672 INFO:tasks.workunit.client.0.smithi037.stdout:9/373: link d1/d1f/d3a/f27 d1/d4/d19/d1e/d4a/d5c/d60/f7a 0 2024-03-29T08:07:12.674 INFO:tasks.workunit.client.0.smithi037.stdout:9/374: chown d1/d4/d19/f64 153 1 2024-03-29T08:07:12.674 INFO:tasks.workunit.client.0.smithi037.stdout:4/195: rename d3/d11 to d3/d4/d15/d1d/db/d45 0 2024-03-29T08:07:12.676 INFO:tasks.workunit.client.0.smithi037.stdout:4/196: write d3/d4/d15/d1d/f3f [438291,79655] 0 2024-03-29T08:07:12.681 INFO:tasks.workunit.client.0.smithi037.stdout:4/197: dread d3/d4/d15/d1d/fc [0,4194304] 0 2024-03-29T08:07:12.681 INFO:tasks.workunit.client.0.smithi037.stdout:9/375: link d1/d4/d19/c79 d1/d4/d19/d1e/d4a/d5c/d60/d13/c7b 0 2024-03-29T08:07:12.686 INFO:tasks.workunit.client.0.smithi037.stdout:9/376: mkdir d1/d4/d19/d1e/d4a/d5c/d60/d13/d7c 0 2024-03-29T08:07:12.687 INFO:tasks.workunit.client.0.smithi037.stdout:9/377: write d1/d4/d19/f64 [1125348,70912] 0 2024-03-29T08:07:12.691 INFO:tasks.workunit.client.0.smithi037.stdout:9/378: write d1/d1f/d3a/f27 [4582445,105896] 0 2024-03-29T08:07:12.695 INFO:tasks.workunit.client.0.smithi037.stdout:9/379: creat d1/d3/d44/f7d x:0 0 0 2024-03-29T08:07:12.697 INFO:tasks.workunit.client.0.smithi037.stdout:4/198: dwrite d3/f3a [0,4194304] 0 2024-03-29T08:07:12.718 INFO:tasks.workunit.client.1.smithi148.stdout:7/293: sync 2024-03-29T08:07:12.732 INFO:tasks.workunit.client.1.smithi148.stdout:8/304: fsync d17/d18/d26/f35 0 2024-03-29T08:07:12.735 INFO:tasks.workunit.client.1.smithi148.stdout:7/294: stat d8/d29/f35 0 2024-03-29T08:07:12.736 INFO:tasks.workunit.client.1.smithi148.stdout:2/271: creat d7/d24/d4b/f59 x:0 0 0 2024-03-29T08:07:12.738 INFO:tasks.workunit.client.1.smithi148.stdout:0/254: rename db/de/df/d11/d2d/f31 to db/de/df/d11/d42/f52 0 2024-03-29T08:07:12.738 INFO:tasks.workunit.client.1.smithi148.stdout:5/256: dwrite d0/db/f10 [0,4194304] 0 2024-03-29T08:07:12.744 INFO:tasks.workunit.client.1.smithi148.stdout:5/257: readlink d0/db/de/l18 0 2024-03-29T08:07:12.744 INFO:tasks.workunit.client.1.smithi148.stdout:7/295: chown d8/f15 0 1 2024-03-29T08:07:12.748 INFO:tasks.workunit.client.1.smithi148.stdout:1/256: dwrite d3/de/d25/d35/d42/f1a [0,4194304] 0 2024-03-29T08:07:12.749 INFO:tasks.workunit.client.1.smithi148.stdout:5/258: chown d0/db/de/f50 0 1 2024-03-29T08:07:12.749 INFO:tasks.workunit.client.1.smithi148.stdout:9/253: dread db/d22/d26/f38 [0,4194304] 0 2024-03-29T08:07:12.751 INFO:tasks.workunit.client.1.smithi148.stdout:5/259: stat d0/db/de/f3b 0 2024-03-29T08:07:12.752 INFO:tasks.workunit.client.1.smithi148.stdout:2/272: rmdir d7/da/d33/d3c/d21/d56 0 2024-03-29T08:07:12.754 INFO:tasks.workunit.client.1.smithi148.stdout:0/255: getdents db/de/df/d2b 0 2024-03-29T08:07:12.755 INFO:tasks.workunit.client.1.smithi148.stdout:3/263: dread de/d24/d33/d39/d46/f21 [0,4194304] 0 2024-03-29T08:07:12.755 INFO:tasks.workunit.client.1.smithi148.stdout:9/254: mkdir db/d10/d2d/d32/d3b/d52 0 2024-03-29T08:07:12.759 INFO:tasks.workunit.client.1.smithi148.stdout:0/256: chown db/dc 9 1 2024-03-29T08:07:12.759 INFO:tasks.workunit.client.1.smithi148.stdout:1/257: rename d3/de/d25/d35/d42/f34 to d3/d18/f4b 0 2024-03-29T08:07:12.761 INFO:tasks.workunit.client.1.smithi148.stdout:5/260: creat d0/db/de/f52 x:0 0 0 2024-03-29T08:07:12.763 INFO:tasks.workunit.client.1.smithi148.stdout:0/257: dread - db/de/d24/f38 zero size 2024-03-29T08:07:12.764 INFO:tasks.workunit.client.1.smithi148.stdout:3/264: write de/d24/f4d [3441763,111392] 0 2024-03-29T08:07:12.768 INFO:tasks.workunit.client.1.smithi148.stdout:1/258: dread d3/d18/f39 [0,4194304] 0 2024-03-29T08:07:12.769 INFO:tasks.workunit.client.1.smithi148.stdout:8/305: dwrite d17/d18/d26/d3d/f45 [0,4194304] 0 2024-03-29T08:07:12.770 INFO:tasks.workunit.client.1.smithi148.stdout:9/255: rename db/d10/d2d/d4e/l4f to db/d20/l53 0 2024-03-29T08:07:12.771 INFO:tasks.workunit.client.1.smithi148.stdout:5/261: read d0/db/de/f3b [1221663,35970] 0 2024-03-29T08:07:12.773 INFO:tasks.workunit.client.1.smithi148.stdout:3/265: chown de/c22 88 1 2024-03-29T08:07:12.775 INFO:tasks.workunit.client.1.smithi148.stdout:1/259: rmdir d3/de/d25/d35 39 2024-03-29T08:07:12.777 INFO:tasks.workunit.client.1.smithi148.stdout:7/296: dwrite d8/d14/f55 [0,4194304] 0 2024-03-29T08:07:12.785 INFO:tasks.workunit.client.1.smithi148.stdout:3/266: truncate de/f2e 17089 0 2024-03-29T08:07:12.786 INFO:tasks.workunit.client.1.smithi148.stdout:1/260: symlink d3/de/d21/d32/l4c 0 2024-03-29T08:07:12.788 INFO:tasks.workunit.client.1.smithi148.stdout:7/297: mkdir d8/d38/d67 0 2024-03-29T08:07:12.790 INFO:tasks.workunit.client.1.smithi148.stdout:3/267: symlink de/d24/d33/d39/d46/l5b 0 2024-03-29T08:07:12.792 INFO:tasks.workunit.client.1.smithi148.stdout:9/256: truncate db/d10/d2d/d32/d3b/d4d/d35/f44 595942 0 2024-03-29T08:07:12.793 INFO:tasks.workunit.client.1.smithi148.stdout:7/298: fsync d8/d14/d19/f4b 0 2024-03-29T08:07:12.793 INFO:tasks.workunit.client.1.smithi148.stdout:1/261: mkdir d3/d18/d4d 0 2024-03-29T08:07:12.796 INFO:tasks.workunit.client.1.smithi148.stdout:5/262: link d0/db/d1e/f31 d0/db/de/f53 0 2024-03-29T08:07:12.796 INFO:tasks.workunit.client.1.smithi148.stdout:0/258: dwrite db/f18 [0,4194304] 0 2024-03-29T08:07:12.804 INFO:tasks.workunit.client.1.smithi148.stdout:2/273: dread f5 [0,4194304] 0 2024-03-29T08:07:12.806 INFO:tasks.workunit.client.1.smithi148.stdout:9/257: rename db/d10/d2d/d32/d3b to db/d22/d26/d54 0 2024-03-29T08:07:12.806 INFO:tasks.workunit.client.1.smithi148.stdout:8/306: dwrite d17/d18/d26/f35 [0,4194304] 0 2024-03-29T08:07:12.807 INFO:tasks.workunit.client.1.smithi148.stdout:3/268: truncate de/d24/d33/d39/d46/d30/d37/f4a 970167 0 2024-03-29T08:07:12.811 INFO:tasks.workunit.client.1.smithi148.stdout:9/258: chown db/d22/d26/d54/d4d/f18 11464 1 2024-03-29T08:07:12.813 INFO:tasks.workunit.client.1.smithi148.stdout:5/263: rename d0/db/de/d16/l19 to d0/db/de/d16/l54 0 2024-03-29T08:07:12.815 INFO:tasks.workunit.client.1.smithi148.stdout:9/259: readlink db/lf 0 2024-03-29T08:07:12.818 INFO:tasks.workunit.client.0.smithi037.stdout:9/380: dread d1/d4/d19/d1e/d4a/f62 [0,4194304] 0 2024-03-29T08:07:12.820 INFO:tasks.workunit.client.0.smithi037.stdout:9/381: write d1/d1f/f33 [338127,5284] 0 2024-03-29T08:07:12.821 INFO:tasks.workunit.client.1.smithi148.stdout:8/307: rename d17/d49/f52 to d17/f68 0 2024-03-29T08:07:12.822 INFO:tasks.workunit.client.1.smithi148.stdout:5/264: write d0/db/f34 [1100314,18685] 0 2024-03-29T08:07:12.824 INFO:tasks.workunit.client.0.smithi037.stdout:9/382: write d1/d4/d19/d1e/d4a/d5c/f5d [122436,108061] 0 2024-03-29T08:07:12.833 INFO:tasks.workunit.client.1.smithi148.stdout:1/262: dwrite d3/de/d21/d32/f37 [0,4194304] 0 2024-03-29T08:07:12.836 INFO:tasks.workunit.client.0.smithi037.stdout:4/199: truncate d3/d4/d15/d1d/db/f12 104187 0 2024-03-29T08:07:12.837 INFO:tasks.workunit.client.0.smithi037.stdout:9/383: write d1/d4/d19/d1e/d4a/f62 [1689828,48841] 0 2024-03-29T08:07:12.839 INFO:tasks.workunit.client.1.smithi148.stdout:5/265: mkdir d0/db/d1e/d35/d42/d55 0 2024-03-29T08:07:12.840 INFO:tasks.workunit.client.0.smithi037.stdout:4/200: write d3/d4/d15/d1d/db/d45/f30 [718054,12483] 0 2024-03-29T08:07:12.841 INFO:tasks.workunit.client.1.smithi148.stdout:2/274: dwrite d7/d24/d4b/f59 [0,4194304] 0 2024-03-29T08:07:12.844 INFO:tasks.workunit.client.0.smithi037.stdout:9/384: rename d1/d1f/c6b to d1/d4/d19/d1e/d4a/c7e 0 2024-03-29T08:07:12.844 INFO:tasks.workunit.client.1.smithi148.stdout:1/263: write d3/d18/f4b [531804,50778] 0 2024-03-29T08:07:12.846 INFO:tasks.workunit.client.0.smithi037.stdout:4/201: creat d3/d4/f46 x:0 0 0 2024-03-29T08:07:12.848 INFO:tasks.workunit.client.0.smithi037.stdout:4/202: chown d3/d4/f3e 233185157 1 2024-03-29T08:07:12.850 INFO:tasks.workunit.client.1.smithi148.stdout:2/275: creat d7/f5a x:0 0 0 2024-03-29T08:07:12.852 INFO:tasks.workunit.client.1.smithi148.stdout:5/266: read d0/db/d1e/f31 [272399,74384] 0 2024-03-29T08:07:12.855 INFO:tasks.workunit.client.1.smithi148.stdout:7/299: dread d8/f10 [0,4194304] 0 2024-03-29T08:07:12.859 INFO:tasks.workunit.client.1.smithi148.stdout:5/267: fdatasync d0/db/de/f3b 0 2024-03-29T08:07:12.861 INFO:tasks.workunit.client.1.smithi148.stdout:0/259: dread db/de/df/d11/f41 [0,4194304] 0 2024-03-29T08:07:12.863 INFO:tasks.workunit.client.0.smithi037.stdout:9/385: write d1/d1f/d3a/f78 [1268460,97334] 0 2024-03-29T08:07:12.864 INFO:tasks.workunit.client.1.smithi148.stdout:5/268: chown d0/f21 55718464 1 2024-03-29T08:07:12.864 INFO:tasks.workunit.client.1.smithi148.stdout:8/308: dwrite f15 [4194304,4194304] 0 2024-03-29T08:07:12.868 INFO:tasks.workunit.client.0.smithi037.stdout:9/386: creat d1/d4/f7f x:0 0 0 2024-03-29T08:07:12.868 INFO:tasks.workunit.client.1.smithi148.stdout:9/260: dwrite db/d22/d26/d54/d4d/d35/f48 [0,4194304] 0 2024-03-29T08:07:12.874 INFO:tasks.workunit.client.0.smithi037.stdout:9/387: rename d1/d4/d19/d1e/d4a/d5c/d60/d23/l2c to d1/d4/d19/d69/d75/l80 0 2024-03-29T08:07:12.876 INFO:tasks.workunit.client.1.smithi148.stdout:5/269: rmdir d0/db/de/d16 39 2024-03-29T08:07:12.878 INFO:tasks.workunit.client.1.smithi148.stdout:4/242: sync 2024-03-29T08:07:12.880 INFO:tasks.workunit.client.1.smithi148.stdout:0/260: creat db/de/df/d2b/f53 x:0 0 0 2024-03-29T08:07:12.880 INFO:tasks.workunit.client.1.smithi148.stdout:6/246: sync 2024-03-29T08:07:12.881 INFO:tasks.workunit.client.1.smithi148.stdout:8/309: mknod d17/d18/d26/c69 0 2024-03-29T08:07:12.884 INFO:tasks.workunit.client.1.smithi148.stdout:4/243: write fd [5313593,103964] 0 2024-03-29T08:07:12.885 INFO:tasks.workunit.client.0.smithi037.stdout:9/388: dread d1/d4/f3b [0,4194304] 0 2024-03-29T08:07:12.887 INFO:tasks.workunit.client.1.smithi148.stdout:5/270: creat d0/db/de/d16/d2e/f56 x:0 0 0 2024-03-29T08:07:12.888 INFO:tasks.workunit.client.0.smithi037.stdout:9/389: dread - d1/d1f/d3a/d67/f6d zero size 2024-03-29T08:07:12.888 INFO:tasks.workunit.client.1.smithi148.stdout:0/261: symlink db/de/df/d2b/l54 0 2024-03-29T08:07:12.891 INFO:tasks.workunit.client.1.smithi148.stdout:7/300: dwrite d8/d29/f54 [0,4194304] 0 2024-03-29T08:07:12.891 INFO:tasks.workunit.client.1.smithi148.stdout:2/276: dwrite d7/da/d33/d3c/d21/f25 [0,4194304] 0 2024-03-29T08:07:12.893 INFO:tasks.workunit.client.1.smithi148.stdout:8/310: rename d17/d1d/c25 to d17/d18/d38/d39/c6a 0 2024-03-29T08:07:12.894 INFO:tasks.workunit.client.1.smithi148.stdout:5/271: mknod d0/db/de/d16/c57 0 2024-03-29T08:07:12.896 INFO:tasks.workunit.client.1.smithi148.stdout:0/262: chown db/de/df/d14/l1d 98201435 1 2024-03-29T08:07:12.897 INFO:tasks.workunit.client.1.smithi148.stdout:4/244: getdents df/d2c 0 2024-03-29T08:07:12.899 INFO:tasks.workunit.client.0.smithi037.stdout:9/390: dread d1/d4/f28 [0,4194304] 0 2024-03-29T08:07:12.902 INFO:tasks.workunit.client.1.smithi148.stdout:8/311: mkdir d17/d18/d26/d63/d6b 0 2024-03-29T08:07:12.902 INFO:tasks.workunit.client.0.smithi037.stdout:9/391: chown d1/d4/d19/d1e/d4a/d5c/d60/d23/f72 16077 1 2024-03-29T08:07:12.903 INFO:tasks.workunit.client.1.smithi148.stdout:2/277: unlink d7/da/d33/d45/d4d/l1e 0 2024-03-29T08:07:12.905 INFO:tasks.workunit.client.1.smithi148.stdout:5/272: chown d0/db/de/l1c 274 1 2024-03-29T08:07:12.905 INFO:tasks.workunit.client.1.smithi148.stdout:4/245: mknod df/d10/d26/c5b 0 2024-03-29T08:07:12.907 INFO:tasks.workunit.client.1.smithi148.stdout:1/264: dread d3/de/f1b [0,4194304] 0 2024-03-29T08:07:12.909 INFO:tasks.workunit.client.0.smithi037.stdout:9/392: mknod d1/d1f/c81 0 2024-03-29T08:07:12.911 INFO:tasks.workunit.client.1.smithi148.stdout:1/265: read - d3/de/d1d/f33 zero size 2024-03-29T08:07:12.911 INFO:tasks.workunit.client.1.smithi148.stdout:7/301: link d8/d14/d19/f63 d8/d29/d44/d49/f68 0 2024-03-29T08:07:12.912 INFO:tasks.workunit.client.0.smithi037.stdout:9/393: symlink d1/d3/l82 0 2024-03-29T08:07:12.913 INFO:tasks.workunit.client.1.smithi148.stdout:2/278: creat d7/da/d33/f5b x:0 0 0 2024-03-29T08:07:12.914 INFO:tasks.workunit.client.1.smithi148.stdout:3/269: sync 2024-03-29T08:07:12.915 INFO:tasks.workunit.client.1.smithi148.stdout:8/312: read d17/d18/f1c [33713,125397] 0 2024-03-29T08:07:12.916 INFO:tasks.workunit.client.1.smithi148.stdout:9/261: dwrite db/d20/f1b [0,4194304] 0 2024-03-29T08:07:12.917 INFO:tasks.workunit.client.0.smithi037.stdout:9/394: link d1/d1f/f33 d1/d4/d19/d1e/d4a/d6f/f83 0 2024-03-29T08:07:12.917 INFO:tasks.workunit.client.1.smithi148.stdout:5/273: creat d0/db/d1e/d36/d41/f58 x:0 0 0 2024-03-29T08:07:12.918 INFO:tasks.workunit.client.1.smithi148.stdout:1/266: chown d3/c5 94609856 1 2024-03-29T08:07:12.919 INFO:tasks.workunit.client.0.smithi037.stdout:9/395: rmdir d1/d4/d19/d69 39 2024-03-29T08:07:12.920 INFO:tasks.workunit.client.0.smithi037.stdout:9/396: stat d1/d4/d19/d1e/d25 0 2024-03-29T08:07:12.923 INFO:tasks.workunit.client.1.smithi148.stdout:6/247: dwrite d6/d9/f2c [0,4194304] 0 2024-03-29T08:07:12.923 INFO:tasks.workunit.client.1.smithi148.stdout:3/270: rmdir de/d24/d33 39 2024-03-29T08:07:12.927 INFO:tasks.workunit.client.1.smithi148.stdout:5/274: read d0/f21 [2417893,27217] 0 2024-03-29T08:07:12.934 INFO:tasks.workunit.client.0.smithi037.stdout:9/397: dread d1/d4/d19/d1e/d4a/d5c/d60/d23/f37 [0,4194304] 0 2024-03-29T08:07:12.938 INFO:tasks.workunit.client.1.smithi148.stdout:8/313: dread d17/d18/f3a [0,4194304] 0 2024-03-29T08:07:12.939 INFO:tasks.workunit.client.0.smithi037.stdout:9/398: fdatasync d1/d4/d19/d1e/d4a/d5c/d60/d31/f38 0 2024-03-29T08:07:12.941 INFO:tasks.workunit.client.1.smithi148.stdout:0/263: dwrite db/de/f4b [0,4194304] 0 2024-03-29T08:07:12.944 INFO:tasks.workunit.client.1.smithi148.stdout:4/246: dwrite df/f1e [0,4194304] 0 2024-03-29T08:07:12.945 INFO:tasks.workunit.client.0.smithi037.stdout:9/399: mknod d1/d4/d19/d1e/d4a/d5c/d60/d23/c84 0 2024-03-29T08:07:12.949 INFO:tasks.workunit.client.1.smithi148.stdout:3/271: creat de/d24/d33/f5c x:0 0 0 2024-03-29T08:07:12.950 INFO:tasks.workunit.client.1.smithi148.stdout:5/275: write d0/db/de/d16/d2e/f39 [2875993,42935] 0 2024-03-29T08:07:12.953 INFO:tasks.workunit.client.1.smithi148.stdout:8/314: creat d17/d18/d26/d63/d66/d5f/f6c x:0 0 0 2024-03-29T08:07:12.957 INFO:tasks.workunit.client.1.smithi148.stdout:0/264: rename db/dc/l2e to db/de/df/d11/d42/l55 0 2024-03-29T08:07:12.959 INFO:tasks.workunit.client.1.smithi148.stdout:1/267: dwrite d3/de/d25/d35/d42/f1a [4194304,4194304] 0 2024-03-29T08:07:12.960 INFO:tasks.workunit.client.1.smithi148.stdout:3/272: read de/d24/d33/d39/d46/d30/d37/f4a [398753,35288] 0 2024-03-29T08:07:12.962 INFO:tasks.workunit.client.1.smithi148.stdout:8/315: creat d17/d49/f6d x:0 0 0 2024-03-29T08:07:12.962 INFO:tasks.workunit.client.1.smithi148.stdout:9/262: write db/d22/d26/f30 [2181575,2080] 0 2024-03-29T08:07:12.962 INFO:tasks.workunit.client.1.smithi148.stdout:4/247: dread df/d10/f40 [4194304,4194304] 0 2024-03-29T08:07:12.966 INFO:tasks.workunit.client.1.smithi148.stdout:5/276: creat d0/db/d1e/d36/d41/f59 x:0 0 0 2024-03-29T08:07:12.966 INFO:tasks.workunit.client.1.smithi148.stdout:2/279: truncate d7/da/d33/d3c/d21/f25 8024600 0 2024-03-29T08:07:12.967 INFO:tasks.workunit.client.1.smithi148.stdout:6/248: write d6/fa [1897875,124354] 0 2024-03-29T08:07:12.967 INFO:tasks.workunit.client.1.smithi148.stdout:0/265: symlink db/de/df/d14/l56 0 2024-03-29T08:07:12.967 INFO:tasks.workunit.client.1.smithi148.stdout:4/248: write df/d10/d26/f3d [619105,118627] 0 2024-03-29T08:07:12.968 INFO:tasks.workunit.client.0.smithi037.stdout:3/250: sync 2024-03-29T08:07:12.970 INFO:tasks.workunit.client.0.smithi037.stdout:3/251: chown dd/d46/d2d 1377 1 2024-03-29T08:07:12.970 INFO:tasks.workunit.client.1.smithi148.stdout:1/268: creat d3/de/d21/d32/f4e x:0 0 0 2024-03-29T08:07:12.970 INFO:tasks.workunit.client.1.smithi148.stdout:6/249: fdatasync d6/fd 0 2024-03-29T08:07:12.972 INFO:tasks.workunit.client.1.smithi148.stdout:2/280: fdatasync d7/da/d33/d45/d4d/d2e/d37/f3d 0 2024-03-29T08:07:12.972 INFO:tasks.workunit.client.1.smithi148.stdout:9/263: unlink db/d22/d26/d54/d4d/d35/c37 0 2024-03-29T08:07:12.974 INFO:tasks.workunit.client.0.smithi037.stdout:3/252: read dd/d46/f2c [703887,2501] 0 2024-03-29T08:07:12.975 INFO:tasks.workunit.client.1.smithi148.stdout:8/316: creat d17/d18/d38/f6e x:0 0 0 2024-03-29T08:07:12.976 INFO:tasks.workunit.client.1.smithi148.stdout:4/249: write df/d10/d26/d34/d3e/f3f [857416,6440] 0 2024-03-29T08:07:12.976 INFO:tasks.workunit.client.1.smithi148.stdout:5/277: creat d0/db/d1e/d36/f5a x:0 0 0 2024-03-29T08:07:12.978 INFO:tasks.workunit.client.0.smithi037.stdout:9/400: dread d1/d1f/d3a/f27 [0,4194304] 0 2024-03-29T08:07:12.979 INFO:tasks.workunit.client.1.smithi148.stdout:6/250: write d6/f19 [1861814,124256] 0 2024-03-29T08:07:12.979 INFO:tasks.workunit.client.1.smithi148.stdout:9/264: fsync db/d22/d26/d54/d4d/f41 0 2024-03-29T08:07:12.980 INFO:tasks.workunit.client.1.smithi148.stdout:0/266: rename db/dc/l3a to db/de/df/d2b/d3b/l57 0 2024-03-29T08:07:12.982 INFO:tasks.workunit.client.1.smithi148.stdout:1/269: mkdir d3/de/d4f 0 2024-03-29T08:07:12.982 INFO:tasks.workunit.client.1.smithi148.stdout:5/278: dread - d0/db/d1e/d36/d41/f58 zero size 2024-03-29T08:07:12.983 INFO:tasks.workunit.client.1.smithi148.stdout:4/250: write df/d10/d26/d34/d3e/f58 [59820,68041] 0 2024-03-29T08:07:12.984 INFO:tasks.workunit.client.0.smithi037.stdout:9/401: symlink d1/d4/d19/d1e/d4a/d5c/d60/d13/l85 0 2024-03-29T08:07:12.987 INFO:tasks.workunit.client.1.smithi148.stdout:5/279: chown d0/db/de/c17 11428 1 2024-03-29T08:07:12.989 INFO:tasks.workunit.client.1.smithi148.stdout:3/273: dwrite de/d3b/f58 [0,4194304] 0 2024-03-29T08:07:12.990 INFO:tasks.workunit.client.0.smithi037.stdout:9/402: creat d1/d4/d19/d1e/d4a/d6f/f86 x:0 0 0 2024-03-29T08:07:12.990 INFO:tasks.workunit.client.1.smithi148.stdout:8/317: rename d17/d18/d26/d3d to d17/d18/d38/d39/d6f 0 2024-03-29T08:07:12.991 INFO:tasks.workunit.client.1.smithi148.stdout:8/318: stat f15 0 2024-03-29T08:07:12.993 INFO:tasks.workunit.client.1.smithi148.stdout:2/281: fsync d7/da/d33/d45/d4d/f29 0 2024-03-29T08:07:12.993 INFO:tasks.workunit.client.1.smithi148.stdout:0/267: symlink db/de/df/d2b/d3b/d47/l58 0 2024-03-29T08:07:12.995 INFO:tasks.workunit.client.1.smithi148.stdout:1/270: rmdir d3/de/d25/d35/d42/d23 39 2024-03-29T08:07:12.996 INFO:tasks.workunit.client.1.smithi148.stdout:3/274: stat de/d24/d33/d39/d46/f32 0 2024-03-29T08:07:12.997 INFO:tasks.workunit.client.0.smithi037.stdout:9/403: mkdir d1/d4/d19/d1e/d4a/d5c/d60/d31/d87 0 2024-03-29T08:07:12.998 INFO:tasks.workunit.client.1.smithi148.stdout:5/280: mkdir d0/db/d1e/d5b 0 2024-03-29T08:07:12.998 INFO:tasks.workunit.client.1.smithi148.stdout:6/251: rename d6/d9/f3f to d6/f41 0 2024-03-29T08:07:13.003 INFO:tasks.workunit.client.0.smithi037.stdout:9/404: mkdir d1/d4/d19/d1e/d4a/d5c/d60/d31/d87/d88 0 2024-03-29T08:07:13.010 INFO:tasks.workunit.client.0.smithi037.stdout:9/405: mkdir d1/d4/d19/d1e/d71/d89 0 2024-03-29T08:07:13.012 INFO:tasks.workunit.client.1.smithi148.stdout:9/265: dwrite db/d22/d26/f30 [0,4194304] 0 2024-03-29T08:07:13.017 INFO:tasks.workunit.client.1.smithi148.stdout:3/275: unlink de/c1a 0 2024-03-29T08:07:13.019 INFO:tasks.workunit.client.0.smithi037.stdout:3/253: dread dd/d46/f38 [0,4194304] 0 2024-03-29T08:07:13.019 INFO:tasks.workunit.client.0.smithi037.stdout:3/254: fdatasync dd/fe 0 2024-03-29T08:07:13.023 INFO:tasks.workunit.client.1.smithi148.stdout:8/319: dread d17/d18/f2a [0,4194304] 0 2024-03-29T08:07:13.024 INFO:tasks.workunit.client.0.smithi037.stdout:3/255: getdents dd/d32 0 2024-03-29T08:07:13.029 INFO:tasks.workunit.client.1.smithi148.stdout:0/268: rmdir db/de/df/d2b/d46 0 2024-03-29T08:07:13.031 INFO:tasks.workunit.client.0.smithi037.stdout:3/256: link f5 dd/d32/f5a 0 2024-03-29T08:07:13.032 INFO:tasks.workunit.client.1.smithi148.stdout:4/251: dread f7 [0,4194304] 0 2024-03-29T08:07:13.032 INFO:tasks.workunit.client.1.smithi148.stdout:5/281: dwrite d0/db/d1e/d35/d42/f43 [0,4194304] 0 2024-03-29T08:07:13.032 INFO:tasks.workunit.client.1.smithi148.stdout:0/269: dread - db/de/df/d11/d42/f48 zero size 2024-03-29T08:07:13.033 INFO:tasks.workunit.client.1.smithi148.stdout:5/282: stat d0/db/f3e 0 2024-03-29T08:07:13.033 INFO:tasks.workunit.client.1.smithi148.stdout:9/266: dwrite f4 [0,4194304] 0 2024-03-29T08:07:13.034 INFO:tasks.workunit.client.1.smithi148.stdout:4/252: chown df/d2d/f50 28913170 1 2024-03-29T08:07:13.035 INFO:tasks.workunit.client.1.smithi148.stdout:0/270: chown db/de/df/d2b/d3b/f49 232360 1 2024-03-29T08:07:13.036 INFO:tasks.workunit.client.1.smithi148.stdout:0/271: read - db/de/df/d2b/f34 zero size 2024-03-29T08:07:13.036 INFO:tasks.workunit.client.1.smithi148.stdout:8/320: rmdir d17/d18/d26/d63/d66/d5f 39 2024-03-29T08:07:13.038 INFO:tasks.workunit.client.1.smithi148.stdout:8/321: chown d17/d18/d38/d39/d6f/f45 2 1 2024-03-29T08:07:13.046 INFO:tasks.workunit.client.1.smithi148.stdout:0/272: truncate f9 169087 0 2024-03-29T08:07:13.047 INFO:tasks.workunit.client.1.smithi148.stdout:0/273: read - db/de/d24/f38 zero size 2024-03-29T08:07:13.047 INFO:tasks.workunit.client.1.smithi148.stdout:9/267: rmdir db/d22/d26/d54 39 2024-03-29T08:07:13.053 INFO:tasks.workunit.client.0.smithi037.stdout:9/406: dwrite d1/d4/d19/d1e/d4a/d6f/f83 [4194304,4194304] 0 2024-03-29T08:07:13.061 INFO:tasks.workunit.client.1.smithi148.stdout:9/268: fdatasync db/d22/d26/d54/f36 0 2024-03-29T08:07:13.062 INFO:tasks.workunit.client.1.smithi148.stdout:9/269: stat db/d22/d3f/f42 0 2024-03-29T08:07:13.063 INFO:tasks.workunit.client.0.smithi037.stdout:9/407: creat d1/d4/d19/d69/d75/f8a x:0 0 0 2024-03-29T08:07:13.063 INFO:tasks.workunit.client.1.smithi148.stdout:6/252: write d6/d9/d1e/d22/f31 [824949,58171] 0 2024-03-29T08:07:13.064 INFO:tasks.workunit.client.1.smithi148.stdout:9/270: chown f4 3 1 2024-03-29T08:07:13.064 INFO:tasks.workunit.client.0.smithi037.stdout:3/257: dwrite dd/d15/d27/f41 [0,4194304] 0 2024-03-29T08:07:13.065 INFO:tasks.workunit.client.1.smithi148.stdout:6/253: stat d6/d9/l1b 0 2024-03-29T08:07:13.065 INFO:tasks.workunit.client.1.smithi148.stdout:3/276: rename de/d24/c59 to de/d24/c5d 0 2024-03-29T08:07:13.066 INFO:tasks.workunit.client.1.smithi148.stdout:8/322: link d17/d18/d26/c57 d17/d18/d38/c70 0 2024-03-29T08:07:13.067 INFO:tasks.workunit.client.0.smithi037.stdout:3/258: write dd/f1f [925597,50614] 0 2024-03-29T08:07:13.067 INFO:tasks.workunit.client.1.smithi148.stdout:1/271: truncate d3/de/d21/d32/f37 1957754 0 2024-03-29T08:07:13.067 INFO:tasks.workunit.client.0.smithi037.stdout:9/408: creat d1/d4/d19/d69/d75/f8b x:0 0 0 2024-03-29T08:07:13.069 INFO:tasks.workunit.client.1.smithi148.stdout:6/254: dread d6/f2d [0,4194304] 0 2024-03-29T08:07:13.069 INFO:tasks.workunit.client.0.smithi037.stdout:9/409: chown d1/f74 17632 1 2024-03-29T08:07:13.073 INFO:tasks.workunit.client.0.smithi037.stdout:9/410: creat d1/d3/d44/f8c x:0 0 0 2024-03-29T08:07:13.073 INFO:tasks.workunit.client.0.smithi037.stdout:9/411: truncate d1/f4c 972977 0 2024-03-29T08:07:13.084 INFO:tasks.workunit.client.0.smithi037.stdout:2/155: sync 2024-03-29T08:07:13.085 INFO:tasks.workunit.client.1.smithi148.stdout:6/255: dwrite d6/d9/d1e/d27/f32 [0,4194304] 0 2024-03-29T08:07:13.087 INFO:tasks.workunit.client.1.smithi148.stdout:7/302: sync 2024-03-29T08:07:13.088 INFO:tasks.workunit.client.1.smithi148.stdout:4/253: link df/d2c/f46 df/d10/d26/d34/f5c 0 2024-03-29T08:07:13.089 INFO:tasks.workunit.client.1.smithi148.stdout:4/254: chown df/d10/d26 271 1 2024-03-29T08:07:13.090 INFO:tasks.workunit.client.1.smithi148.stdout:4/255: write df/f27 [312533,83951] 0 2024-03-29T08:07:13.091 INFO:tasks.workunit.client.1.smithi148.stdout:0/274: rename db/de/df/d11/f22 to db/dc/f59 0 2024-03-29T08:07:13.092 INFO:tasks.workunit.client.0.smithi037.stdout:3/259: truncate fb 172618 0 2024-03-29T08:07:13.092 INFO:tasks.workunit.client.1.smithi148.stdout:5/283: write d0/f4b [864851,58840] 0 2024-03-29T08:07:13.095 INFO:tasks.workunit.client.0.smithi037.stdout:2/156: mknod d0/d20/d22/c35 0 2024-03-29T08:07:13.096 INFO:tasks.workunit.client.0.smithi037.stdout:3/260: stat dd/d15/c28 0 2024-03-29T08:07:13.096 INFO:tasks.workunit.client.0.smithi037.stdout:9/412: truncate d1/d4/d19/d1e/d4a/d5c/d60/f3d 1296337 0 2024-03-29T08:07:13.100 INFO:tasks.workunit.client.0.smithi037.stdout:3/261: write dd/d46/d2d/d3a/f55 [5133141,114146] 0 2024-03-29T08:07:13.101 INFO:tasks.workunit.client.0.smithi037.stdout:2/157: write d0/fa [425794,114931] 0 2024-03-29T08:07:13.101 INFO:tasks.workunit.client.1.smithi148.stdout:6/256: mknod d6/d9/d1e/d20/c42 0 2024-03-29T08:07:13.104 INFO:tasks.workunit.client.0.smithi037.stdout:9/413: creat d1/f8d x:0 0 0 2024-03-29T08:07:13.104 INFO:tasks.workunit.client.1.smithi148.stdout:5/284: readlink d0/db/de/d16/l54 0 2024-03-29T08:07:13.105 INFO:tasks.workunit.client.0.smithi037.stdout:3/262: symlink dd/d46/d2d/l5b 0 2024-03-29T08:07:13.106 INFO:tasks.workunit.client.1.smithi148.stdout:2/282: sync 2024-03-29T08:07:13.106 INFO:tasks.workunit.client.1.smithi148.stdout:1/272: creat d3/d18/d4d/f50 x:0 0 0 2024-03-29T08:07:13.108 INFO:tasks.workunit.client.1.smithi148.stdout:7/303: mknod d8/c69 0 2024-03-29T08:07:13.112 INFO:tasks.workunit.client.0.smithi037.stdout:2/158: truncate d0/f1f 1336854 0 2024-03-29T08:07:13.112 INFO:tasks.workunit.client.0.smithi037.stdout:3/263: creat dd/d46/d2d/d3d/f5c x:0 0 0 2024-03-29T08:07:13.113 INFO:tasks.workunit.client.0.smithi037.stdout:9/414: chown d1/d4/d19/d69/d75/f8b 253 1 2024-03-29T08:07:13.116 INFO:tasks.workunit.client.1.smithi148.stdout:8/323: getdents d17/d18/d38/d39 0 2024-03-29T08:07:13.117 INFO:tasks.workunit.client.1.smithi148.stdout:1/273: stat d3/de/d25/d35/d42/f1a 0 2024-03-29T08:07:13.119 INFO:tasks.workunit.client.1.smithi148.stdout:2/283: rmdir d7/da/d33/d45 39 2024-03-29T08:07:13.123 INFO:tasks.workunit.client.0.smithi037.stdout:9/415: mknod d1/d1f/c8e 0 2024-03-29T08:07:13.123 INFO:tasks.workunit.client.1.smithi148.stdout:7/304: stat d8/d14/d19/d37/l61 0 2024-03-29T08:07:13.124 INFO:tasks.workunit.client.1.smithi148.stdout:2/284: chown d7/da/d4c 140665 1 2024-03-29T08:07:13.125 INFO:tasks.workunit.client.1.smithi148.stdout:8/324: readlink d17/d18/d38/d39/l40 0 2024-03-29T08:07:13.125 INFO:tasks.workunit.client.1.smithi148.stdout:9/271: dwrite db/d10/d2d/f34 [0,4194304] 0 2024-03-29T08:07:13.127 INFO:tasks.workunit.client.0.smithi037.stdout:9/416: mknod d1/d4/d19/d1e/d4a/d5c/d60/c8f 0 2024-03-29T08:07:13.128 INFO:tasks.workunit.client.1.smithi148.stdout:0/275: dwrite db/de/d24/f38 [0,4194304] 0 2024-03-29T08:07:13.129 INFO:tasks.workunit.client.1.smithi148.stdout:1/274: fdatasync d3/d18/f39 0 2024-03-29T08:07:13.129 INFO:tasks.workunit.client.1.smithi148.stdout:6/257: dread d6/d9/f2c [0,4194304] 0 2024-03-29T08:07:13.131 INFO:tasks.workunit.client.1.smithi148.stdout:0/276: truncate db/de/f3f 4285453 0 2024-03-29T08:07:13.134 INFO:tasks.workunit.client.1.smithi148.stdout:2/285: mknod d7/c5c 0 2024-03-29T08:07:13.134 INFO:tasks.workunit.client.0.smithi037.stdout:9/417: rmdir d1/d4/d19/d1e/d4a/d6f 39 2024-03-29T08:07:13.136 INFO:tasks.workunit.client.1.smithi148.stdout:7/305: rename d8/d14/d45/d58 to d8/d38/d6a 0 2024-03-29T08:07:13.142 INFO:tasks.workunit.client.1.smithi148.stdout:7/306: chown d8/d14/f47 53941153 1 2024-03-29T08:07:13.142 INFO:tasks.workunit.client.1.smithi148.stdout:8/325: rmdir d17/d18/d26/d63/d6b 0 2024-03-29T08:07:13.145 INFO:tasks.workunit.client.0.smithi037.stdout:9/418: symlink d1/d4/d19/d1e/d4a/d5c/d60/d31/l90 0 2024-03-29T08:07:13.145 INFO:tasks.workunit.client.1.smithi148.stdout:2/286: rename f1 to d7/d24/f5d 0 2024-03-29T08:07:13.146 INFO:tasks.workunit.client.0.smithi037.stdout:2/159: dwrite d0/f2b [4194304,4194304] 0 2024-03-29T08:07:13.147 INFO:tasks.workunit.client.0.smithi037.stdout:3/264: dwrite f2 [0,4194304] 0 2024-03-29T08:07:13.149 INFO:tasks.workunit.client.1.smithi148.stdout:8/326: creat d17/d4e/f71 x:0 0 0 2024-03-29T08:07:13.149 INFO:tasks.workunit.client.1.smithi148.stdout:2/287: fsync d7/da/d33/d3c/f48 0 2024-03-29T08:07:13.149 INFO:tasks.workunit.client.1.smithi148.stdout:4/256: dwrite df/d2c/f46 [0,4194304] 0 2024-03-29T08:07:13.150 INFO:tasks.workunit.client.1.smithi148.stdout:3/277: dwrite f8 [4194304,4194304] 0 2024-03-29T08:07:13.152 INFO:tasks.workunit.client.0.smithi037.stdout:9/419: read - d1/d4/d19/d69/d75/f8a zero size 2024-03-29T08:07:13.158 INFO:tasks.workunit.client.1.smithi148.stdout:7/307: creat d8/d14/d45/d4d/f6b x:0 0 0 2024-03-29T08:07:13.160 INFO:tasks.workunit.client.1.smithi148.stdout:0/277: write db/de/df/d11/f16 [1497853,104789] 0 2024-03-29T08:07:13.162 INFO:tasks.workunit.client.0.smithi037.stdout:2/160: creat d0/d20/d2e/f36 x:0 0 0 2024-03-29T08:07:13.166 INFO:tasks.workunit.client.0.smithi037.stdout:2/161: fdatasync d0/f21 0 2024-03-29T08:07:13.166 INFO:tasks.workunit.client.1.smithi148.stdout:4/257: unlink df/d2d/f2f 0 2024-03-29T08:07:13.166 INFO:tasks.workunit.client.1.smithi148.stdout:5/285: dread d0/db/de/d16/f27 [0,4194304] 0 2024-03-29T08:07:13.167 INFO:tasks.workunit.client.1.smithi148.stdout:3/278: mknod de/d24/d33/c5e 0 2024-03-29T08:07:13.168 INFO:tasks.workunit.client.0.smithi037.stdout:9/420: creat d1/d4/d19/d1e/d4a/d6f/f91 x:0 0 0 2024-03-29T08:07:13.169 INFO:tasks.workunit.client.1.smithi148.stdout:3/279: chown de/l28 527587565 1 2024-03-29T08:07:13.170 INFO:tasks.workunit.client.1.smithi148.stdout:7/308: chown d8/d29/f2e 8260278 1 2024-03-29T08:07:13.171 INFO:tasks.workunit.client.1.smithi148.stdout:2/288: mknod d7/da/d33/c5e 0 2024-03-29T08:07:13.172 INFO:tasks.workunit.client.1.smithi148.stdout:5/286: chown d0/db/de/d16/l54 322317713 1 2024-03-29T08:07:13.173 INFO:tasks.workunit.client.1.smithi148.stdout:2/289: stat d7/da/fe 0 2024-03-29T08:07:13.175 INFO:tasks.workunit.client.1.smithi148.stdout:3/280: fsync de/d24/d33/d39/d46/f40 0 2024-03-29T08:07:13.178 INFO:tasks.workunit.client.0.smithi037.stdout:9/421: mkdir d1/d4/d19/d1e/d71/d89/d92 0 2024-03-29T08:07:13.179 INFO:tasks.workunit.client.0.smithi037.stdout:2/162: link d0/c16 d0/d23/c37 0 2024-03-29T08:07:13.180 INFO:tasks.workunit.client.1.smithi148.stdout:7/309: rename d8/d14/d19/d37/c42 to d8/d38/d3c/c6c 0 2024-03-29T08:07:13.184 INFO:tasks.workunit.client.1.smithi148.stdout:0/278: read fa [2216217,32298] 0 2024-03-29T08:07:13.187 INFO:tasks.workunit.client.1.smithi148.stdout:3/281: rename de/f26 to de/d24/d33/d39/d46/f5f 0 2024-03-29T08:07:13.190 INFO:tasks.workunit.client.0.smithi037.stdout:9/422: rename d1/d4/d19/d69/d75/l80 to d1/d4/d19/d1e/d4a/d6f/l93 0 2024-03-29T08:07:13.191 INFO:tasks.workunit.client.1.smithi148.stdout:8/327: dwrite d17/d18/d26/d63/d66/d5f/f6c [0,4194304] 0 2024-03-29T08:07:13.193 INFO:tasks.workunit.client.1.smithi148.stdout:8/328: stat d17/d18/d38/f48 0 2024-03-29T08:07:13.194 INFO:tasks.workunit.client.1.smithi148.stdout:3/282: creat de/d3b/f60 x:0 0 0 2024-03-29T08:07:13.194 INFO:tasks.workunit.client.1.smithi148.stdout:0/279: dread db/de/df/d2b/d3b/d47/f2c [0,4194304] 0 2024-03-29T08:07:13.195 INFO:tasks.workunit.client.1.smithi148.stdout:2/290: rmdir d7/da/d33 39 2024-03-29T08:07:13.195 INFO:tasks.workunit.client.1.smithi148.stdout:8/329: dread - d17/d49/f6d zero size 2024-03-29T08:07:13.196 INFO:tasks.workunit.client.1.smithi148.stdout:2/291: readlink d7/da/l1d 0 2024-03-29T08:07:13.196 INFO:tasks.workunit.client.0.smithi037.stdout:9/423: creat d1/d4/d19/d1e/d71/f94 x:0 0 0 2024-03-29T08:07:13.197 INFO:tasks.workunit.client.1.smithi148.stdout:7/310: rename d8/cb to d8/d29/d44/d50/c6d 0 2024-03-29T08:07:13.198 INFO:tasks.workunit.client.1.smithi148.stdout:8/330: write d17/d18/d26/f27 [340793,123198] 0 2024-03-29T08:07:13.200 INFO:tasks.workunit.client.0.smithi037.stdout:3/265: dread f5 [0,4194304] 0 2024-03-29T08:07:13.200 INFO:tasks.workunit.client.1.smithi148.stdout:4/258: truncate f7 1170879 0 2024-03-29T08:07:13.200 INFO:tasks.workunit.client.0.smithi037.stdout:9/424: mkdir d1/d4/d95 0 2024-03-29T08:07:13.202 INFO:tasks.workunit.client.1.smithi148.stdout:8/331: fdatasync d17/d18/d38/f6e 0 2024-03-29T08:07:13.206 INFO:tasks.workunit.client.0.smithi037.stdout:3/266: creat dd/d32/f5d x:0 0 0 2024-03-29T08:07:13.206 INFO:tasks.workunit.client.1.smithi148.stdout:5/287: write d0/db/de/f53 [1577974,126592] 0 2024-03-29T08:07:13.210 INFO:tasks.workunit.client.1.smithi148.stdout:8/332: dread d17/d18/f5c [0,4194304] 0 2024-03-29T08:07:13.211 INFO:tasks.workunit.client.1.smithi148.stdout:0/280: dread db/de/df/d11/f37 [0,4194304] 0 2024-03-29T08:07:13.212 INFO:tasks.workunit.client.0.smithi037.stdout:9/425: readlink d1/d4/l47 0 2024-03-29T08:07:13.212 INFO:tasks.workunit.client.1.smithi148.stdout:8/333: fsync d17/d18/d38/f5a 0 2024-03-29T08:07:13.215 INFO:tasks.workunit.client.1.smithi148.stdout:3/283: rmdir de 39 2024-03-29T08:07:13.219 INFO:tasks.workunit.client.0.smithi037.stdout:2/163: dread d0/f7 [0,4194304] 0 2024-03-29T08:07:13.221 INFO:tasks.workunit.client.1.smithi148.stdout:5/288: creat d0/db/d1e/d35/f5c x:0 0 0 2024-03-29T08:07:13.222 INFO:tasks.workunit.client.0.smithi037.stdout:2/164: truncate d0/fd 1653609 0 2024-03-29T08:07:13.223 INFO:tasks.workunit.client.1.smithi148.stdout:4/259: mknod df/c5d 0 2024-03-29T08:07:13.224 INFO:tasks.workunit.client.1.smithi148.stdout:5/289: truncate d0/db/de/f50 997503 0 2024-03-29T08:07:13.229 INFO:tasks.workunit.client.1.smithi148.stdout:3/284: dread - de/d24/f56 zero size 2024-03-29T08:07:13.229 INFO:tasks.workunit.client.1.smithi148.stdout:3/285: stat de/lf 0 2024-03-29T08:07:13.230 INFO:tasks.workunit.client.0.smithi037.stdout:9/426: getdents d1/d4 0 2024-03-29T08:07:13.230 INFO:tasks.workunit.client.0.smithi037.stdout:2/165: write d0/f7 [3902885,6311] 0 2024-03-29T08:07:13.231 INFO:tasks.workunit.client.1.smithi148.stdout:2/292: creat d7/da/d33/d45/f5f x:0 0 0 2024-03-29T08:07:13.231 INFO:tasks.workunit.client.1.smithi148.stdout:9/272: sync 2024-03-29T08:07:13.233 INFO:tasks.workunit.client.1.smithi148.stdout:0/281: mkdir db/de/df/d2b/d3b/d47/d4c/d5a 0 2024-03-29T08:07:13.234 INFO:tasks.workunit.client.1.smithi148.stdout:5/290: mkdir d0/db/d1e/d35/d5d 0 2024-03-29T08:07:13.235 INFO:tasks.workunit.client.0.smithi037.stdout:9/427: mkdir d1/d4/d19/d1e/d4a/d6f/d96 0 2024-03-29T08:07:13.236 INFO:tasks.workunit.client.0.smithi037.stdout:2/166: unlink d0/c17 0 2024-03-29T08:07:13.236 INFO:tasks.workunit.client.1.smithi148.stdout:3/286: creat de/f61 x:0 0 0 2024-03-29T08:07:13.238 INFO:tasks.workunit.client.1.smithi148.stdout:3/287: write de/d24/f31 [5028927,119736] 0 2024-03-29T08:07:13.239 INFO:tasks.workunit.client.0.smithi037.stdout:3/267: dread dd/d15/f21 [0,4194304] 0 2024-03-29T08:07:13.239 INFO:tasks.workunit.client.1.smithi148.stdout:0/282: creat db/de/df/d2b/f5b x:0 0 0 2024-03-29T08:07:13.240 INFO:tasks.workunit.client.0.smithi037.stdout:9/428: symlink d1/d4/d19/d1e/d4a/d5c/d60/d31/d87/d88/l97 0 2024-03-29T08:07:13.241 INFO:tasks.workunit.client.1.smithi148.stdout:5/291: creat d0/db/d1e/d35/d42/f5e x:0 0 0 2024-03-29T08:07:13.241 INFO:tasks.workunit.client.1.smithi148.stdout:7/311: link d8/c28 d8/c6e 0 2024-03-29T08:07:13.242 INFO:tasks.workunit.client.1.smithi148.stdout:8/334: rename d17/d49/c5e to d17/d18/d26/c72 0 2024-03-29T08:07:13.245 INFO:tasks.workunit.client.1.smithi148.stdout:9/273: symlink db/d22/d26/d54/d52/l55 0 2024-03-29T08:07:13.246 INFO:tasks.workunit.client.1.smithi148.stdout:0/283: symlink db/de/df/d2b/d3b/d47/d4c/l5c 0 2024-03-29T08:07:13.246 INFO:tasks.workunit.client.1.smithi148.stdout:8/335: write d17/d18/d38/f5a [1685633,30957] 0 2024-03-29T08:07:13.248 INFO:tasks.workunit.client.1.smithi148.stdout:5/292: unlink d0/d46/l4f 0 2024-03-29T08:07:13.249 INFO:tasks.workunit.client.0.smithi037.stdout:3/268: write dd/d46/f37 [894347,39299] 0 2024-03-29T08:07:13.249 INFO:tasks.workunit.client.1.smithi148.stdout:8/336: read d17/d18/f1c [81864,69825] 0 2024-03-29T08:07:13.253 INFO:tasks.workunit.client.0.smithi037.stdout:6/189: sync 2024-03-29T08:07:13.255 INFO:tasks.workunit.client.1.smithi148.stdout:5/293: mknod d0/db/c5f 0 2024-03-29T08:07:13.256 INFO:tasks.workunit.client.0.smithi037.stdout:6/190: fdatasync d0/d5/d1f/f30 0 2024-03-29T08:07:13.256 INFO:tasks.workunit.client.1.smithi148.stdout:0/284: mknod db/de/df/d11/d42/c5d 0 2024-03-29T08:07:13.259 INFO:tasks.workunit.client.1.smithi148.stdout:3/288: dwrite de/f51 [0,4194304] 0 2024-03-29T08:07:13.260 INFO:tasks.workunit.client.1.smithi148.stdout:5/294: chown d0/db/de/l18 15 1 2024-03-29T08:07:13.263 INFO:tasks.workunit.client.1.smithi148.stdout:0/285: chown db/de/df/d11/f41 1 1 2024-03-29T08:07:13.263 INFO:tasks.workunit.client.1.smithi148.stdout:4/260: rename df/d10/d26/d34/d3e/f52 to df/f5e 0 2024-03-29T08:07:13.265 INFO:tasks.workunit.client.0.smithi037.stdout:9/429: dwrite d1/d4/d19/d1e/d4a/d5c/d60/f7a [0,4194304] 0 2024-03-29T08:07:13.265 INFO:tasks.workunit.client.1.smithi148.stdout:1/275: sync 2024-03-29T08:07:13.265 INFO:tasks.workunit.client.1.smithi148.stdout:6/258: sync 2024-03-29T08:07:13.268 INFO:tasks.workunit.client.1.smithi148.stdout:5/295: mknod d0/db/de/d16/d2e/c60 0 2024-03-29T08:07:13.269 INFO:tasks.workunit.client.1.smithi148.stdout:9/274: dread db/d22/d26/f38 [0,4194304] 0 2024-03-29T08:07:13.270 INFO:tasks.workunit.client.1.smithi148.stdout:6/259: chown f5 3295 1 2024-03-29T08:07:13.278 INFO:tasks.workunit.client.1.smithi148.stdout:7/312: rename d8/d14/d19/f1a to d8/d29/d44/d49/f6f 0 2024-03-29T08:07:13.279 INFO:tasks.workunit.client.1.smithi148.stdout:0/286: fsync db/f23 0 2024-03-29T08:07:13.284 INFO:tasks.workunit.client.0.smithi037.stdout:3/269: dread dd/d15/f16 [0,4194304] 0 2024-03-29T08:07:13.287 INFO:tasks.workunit.client.1.smithi148.stdout:5/296: symlink d0/db/de/l61 0 2024-03-29T08:07:13.288 INFO:tasks.workunit.client.1.smithi148.stdout:0/287: chown db/de/df/l43 1 1 2024-03-29T08:07:13.291 INFO:tasks.workunit.client.1.smithi148.stdout:8/337: dread d17/d18/d38/d39/d6f/f43 [0,4194304] 0 2024-03-29T08:07:13.291 INFO:tasks.workunit.client.1.smithi148.stdout:2/293: dwrite d7/da/d33/d3c/d21/f25 [0,4194304] 0 2024-03-29T08:07:13.292 INFO:tasks.workunit.client.0.smithi037.stdout:3/270: rename dd/d46/d2d/d3d to dd/d5e 0 2024-03-29T08:07:13.294 INFO:tasks.workunit.client.1.smithi148.stdout:7/313: link d8/d14/f1c d8/d60/f70 0 2024-03-29T08:07:13.296 INFO:tasks.workunit.client.1.smithi148.stdout:2/294: write d7/da/f3f [356010,48659] 0 2024-03-29T08:07:13.296 INFO:tasks.workunit.client.1.smithi148.stdout:9/275: write db/d22/d26/f38 [2133640,6688] 0 2024-03-29T08:07:13.299 INFO:tasks.workunit.client.1.smithi148.stdout:3/289: truncate de/d24/d33/d39/d46/f25 77497 0 2024-03-29T08:07:13.300 INFO:tasks.workunit.client.1.smithi148.stdout:5/297: dread - d0/db/d1e/d36/d41/f58 zero size 2024-03-29T08:07:13.301 INFO:tasks.workunit.client.1.smithi148.stdout:0/288: stat db/de/df/d11/d2d/d3e/f45 0 2024-03-29T08:07:13.305 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:07:13 smithi148 ceph-mon[44992]: pgmap v38: 65 pgs: 65 active+clean; 666 MiB data, 3.6 GiB used, 533 GiB / 536 GiB avail; 1.8 MiB/s rd, 72 MiB/s wr, 201 op/s 2024-03-29T08:07:13.306 INFO:tasks.workunit.client.0.smithi037.stdout:6/191: dread d0/d5/fb [0,4194304] 0 2024-03-29T08:07:13.307 INFO:tasks.workunit.client.1.smithi148.stdout:8/338: creat d17/d18/d38/d39/f73 x:0 0 0 2024-03-29T08:07:13.308 INFO:tasks.workunit.client.1.smithi148.stdout:4/261: dwrite df/d10/d26/d34/d43/f59 [0,4194304] 0 2024-03-29T08:07:13.310 INFO:tasks.workunit.client.1.smithi148.stdout:6/260: dwrite d6/fa [0,4194304] 0 2024-03-29T08:07:13.313 INFO:tasks.workunit.client.0.smithi037.stdout:3/271: dwrite dd/d15/d18/d30/f59 [0,4194304] 0 2024-03-29T08:07:13.314 INFO:tasks.workunit.client.1.smithi148.stdout:5/298: rename d0/db/d1e/d35 to d0/db/d1e/d35/d42/d62 22 2024-03-29T08:07:13.315 INFO:tasks.workunit.client.1.smithi148.stdout:2/295: creat d7/d24/f60 x:0 0 0 2024-03-29T08:07:13.318 INFO:tasks.workunit.client.1.smithi148.stdout:4/262: chown df/d10/d26/d34/l3c 2550 1 2024-03-29T08:07:13.318 INFO:tasks.workunit.client.1.smithi148.stdout:5/299: readlink d0/db/l22 0 2024-03-29T08:07:13.320 INFO:tasks.workunit.client.1.smithi148.stdout:1/276: dwrite d3/de/f1b [0,4194304] 0 2024-03-29T08:07:13.322 INFO:tasks.workunit.client.1.smithi148.stdout:5/300: write d0/db/de/d16/d2e/f56 [390288,89163] 0 2024-03-29T08:07:13.323 INFO:tasks.workunit.client.1.smithi148.stdout:3/290: mkdir de/d24/d33/d39/d46/d30/d37/d62 0 2024-03-29T08:07:13.324 INFO:tasks.workunit.client.1.smithi148.stdout:4/263: truncate df/d2d/f35 171510 0 2024-03-29T08:07:13.326 INFO:tasks.workunit.client.1.smithi148.stdout:6/261: symlink d6/l43 0 2024-03-29T08:07:13.327 INFO:tasks.workunit.client.1.smithi148.stdout:2/296: mknod d7/c61 0 2024-03-29T08:07:13.329 INFO:tasks.workunit.client.1.smithi148.stdout:1/277: chown d3/de/d21/d32/d44 41461309 1 2024-03-29T08:07:13.330 INFO:tasks.workunit.client.1.smithi148.stdout:7/314: getdents d8/d14/d19 0 2024-03-29T08:07:13.331 INFO:tasks.workunit.client.0.smithi037.stdout:6/192: dwrite d0/d5/d3f/ff [0,4194304] 0 2024-03-29T08:07:13.335 INFO:tasks.workunit.client.1.smithi148.stdout:4/264: readlink df/l37 0 2024-03-29T08:07:13.341 INFO:tasks.workunit.client.1.smithi148.stdout:8/339: rename d17/d18/d38 to d17/d74 0 2024-03-29T08:07:13.341 INFO:tasks.workunit.client.1.smithi148.stdout:7/315: creat d8/d38/f71 x:0 0 0 2024-03-29T08:07:13.341 INFO:tasks.workunit.client.1.smithi148.stdout:6/262: rmdir d6/d9/d1e/d22 39 2024-03-29T08:07:13.343 INFO:tasks.workunit.client.1.smithi148.stdout:1/278: write d3/d18/d4d/f50 [626172,79436] 0 2024-03-29T08:07:13.347 INFO:tasks.workunit.client.1.smithi148.stdout:9/276: dread db/f21 [0,4194304] 0 2024-03-29T08:07:13.348 INFO:tasks.workunit.client.1.smithi148.stdout:2/297: rename d7/cd to d7/da/d33/d45/d4d/d2e/c62 0 2024-03-29T08:07:13.351 INFO:tasks.workunit.client.1.smithi148.stdout:0/289: dwrite db/de/df/d11/f37 [4194304,4194304] 0 2024-03-29T08:07:13.351 INFO:tasks.workunit.client.1.smithi148.stdout:5/301: link d0/db/d1e/d35/d42/f43 d0/db/d1e/d36/d41/f63 0 2024-03-29T08:07:13.353 INFO:tasks.workunit.client.1.smithi148.stdout:8/340: rmdir d17/d60 0 2024-03-29T08:07:13.354 INFO:tasks.workunit.client.1.smithi148.stdout:7/316: getdents d8/d38/d6a 0 2024-03-29T08:07:13.356 INFO:tasks.workunit.client.0.smithi037.stdout:6/193: dwrite d0/d5/d3f/f17 [0,4194304] 0 2024-03-29T08:07:13.359 INFO:tasks.workunit.client.0.smithi037.stdout:6/194: fsync d0/d5/d1f/f3c 0 2024-03-29T08:07:13.360 INFO:tasks.workunit.client.1.smithi148.stdout:1/279: fdatasync d3/de/d25/d35/d42/f38 0 2024-03-29T08:07:13.361 INFO:tasks.workunit.client.1.smithi148.stdout:9/277: symlink db/d22/d26/d54/d52/l56 0 2024-03-29T08:07:13.363 INFO:tasks.workunit.client.0.smithi037.stdout:6/195: mkdir d0/d20/d43 0 2024-03-29T08:07:13.365 INFO:tasks.workunit.client.0.smithi037.stdout:6/196: mkdir d0/d5/d1e/d44 0 2024-03-29T08:07:13.367 INFO:tasks.workunit.client.0.smithi037.stdout:6/197: mknod d0/d20/c45 0 2024-03-29T08:07:13.370 INFO:tasks.workunit.client.0.smithi037.stdout:6/198: rmdir d0/d19/d34 39 2024-03-29T08:07:13.372 INFO:tasks.workunit.client.0.smithi037.stdout:6/199: creat d0/d20/f46 x:0 0 0 2024-03-29T08:07:13.373 INFO:tasks.workunit.client.0.smithi037.stdout:6/200: chown d0/c11 6316 1 2024-03-29T08:07:13.374 INFO:tasks.workunit.client.0.smithi037.stdout:6/201: chown d0/d5/d1f/c2b 688396036 1 2024-03-29T08:07:13.380 INFO:tasks.workunit.client.1.smithi148.stdout:4/265: dread df/f27 [0,4194304] 0 2024-03-29T08:07:13.382 INFO:tasks.workunit.client.1.smithi148.stdout:7/317: truncate d8/d60/f70 301009 0 2024-03-29T08:07:13.385 INFO:tasks.workunit.client.0.smithi037.stdout:6/202: dread d0/d5/d3f/f17 [0,4194304] 0 2024-03-29T08:07:13.390 INFO:tasks.workunit.client.1.smithi148.stdout:0/290: mknod db/de/df/d11/d1f/c5e 0 2024-03-29T08:07:13.390 INFO:tasks.workunit.client.1.smithi148.stdout:3/291: dwrite de/d24/d33/d39/d46/f5f [0,4194304] 0 2024-03-29T08:07:13.390 INFO:tasks.workunit.client.1.smithi148.stdout:7/318: creat d8/d14/f72 x:0 0 0 2024-03-29T08:07:13.391 INFO:tasks.workunit.client.1.smithi148.stdout:4/266: rmdir df/d10/d26 39 2024-03-29T08:07:13.394 INFO:tasks.workunit.client.1.smithi148.stdout:6/263: dread d6/d9/d1e/d27/f32 [0,4194304] 0 2024-03-29T08:07:13.404 INFO:tasks.workunit.client.1.smithi148.stdout:1/280: getdents d3/de/d25/d35/d42/d23 0 2024-03-29T08:07:13.404 INFO:tasks.workunit.client.1.smithi148.stdout:4/267: write df/d10/d26/d34/d43/f4e [568824,113036] 0 2024-03-29T08:07:13.405 INFO:tasks.workunit.client.1.smithi148.stdout:7/319: rename d8/f17 to d8/d29/d44/d49/f73 0 2024-03-29T08:07:13.415 INFO:tasks.workunit.client.1.smithi148.stdout:0/291: getdents db/de/df/d2b 0 2024-03-29T08:07:13.418 INFO:tasks.workunit.client.1.smithi148.stdout:4/268: rename c8 to df/d2c/c5f 0 2024-03-29T08:07:13.421 INFO:tasks.workunit.client.1.smithi148.stdout:2/298: dwrite d7/da/d33/d45/d4d/f47 [0,4194304] 0 2024-03-29T08:07:13.423 INFO:tasks.workunit.client.1.smithi148.stdout:6/264: truncate d6/d9/f1f 1411391 0 2024-03-29T08:07:13.426 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:07:12 smithi037 ceph-mon[35162]: pgmap v38: 65 pgs: 65 active+clean; 666 MiB data, 3.6 GiB used, 533 GiB / 536 GiB avail; 1.8 MiB/s rd, 72 MiB/s wr, 201 op/s 2024-03-29T08:07:13.427 INFO:tasks.workunit.client.1.smithi148.stdout:8/341: dwrite d17/d18/f2a [0,4194304] 0 2024-03-29T08:07:13.430 INFO:tasks.workunit.client.1.smithi148.stdout:5/302: dwrite d0/db/de/d16/f2f [0,4194304] 0 2024-03-29T08:07:13.435 INFO:tasks.workunit.client.1.smithi148.stdout:2/299: fsync d7/f3b 0 2024-03-29T08:07:13.435 INFO:tasks.workunit.client.1.smithi148.stdout:8/342: chown d17/d74/f6e 999880864 1 2024-03-29T08:07:13.436 INFO:tasks.workunit.client.1.smithi148.stdout:7/320: rename d8/d29/d44/d49/f73 to d8/d29/d44/d49/f74 0 2024-03-29T08:07:13.437 INFO:tasks.workunit.client.1.smithi148.stdout:4/269: mkdir df/d60 0 2024-03-29T08:07:13.439 INFO:tasks.workunit.client.1.smithi148.stdout:8/343: dread - d17/d4e/f71 zero size 2024-03-29T08:07:13.441 INFO:tasks.workunit.client.1.smithi148.stdout:6/265: creat d6/d9/d1e/d27/f44 x:0 0 0 2024-03-29T08:07:13.446 INFO:tasks.workunit.client.1.smithi148.stdout:5/303: chown d0/db/d1e/d36/d41/f58 40587 1 2024-03-29T08:07:13.448 INFO:tasks.workunit.client.1.smithi148.stdout:7/321: mknod d8/d29/d44/c75 0 2024-03-29T08:07:13.449 INFO:tasks.workunit.client.1.smithi148.stdout:1/281: dwrite d3/de/f1f [0,4194304] 0 2024-03-29T08:07:13.451 INFO:tasks.workunit.client.1.smithi148.stdout:2/300: creat d7/da/d52/f63 x:0 0 0 2024-03-29T08:07:13.452 INFO:tasks.workunit.client.1.smithi148.stdout:8/344: symlink d17/d18/d26/d63/d66/l75 0 2024-03-29T08:07:13.452 INFO:tasks.workunit.client.1.smithi148.stdout:5/304: rename d0 to d0/db/d1e/d35/d42/d64 22 2024-03-29T08:07:13.453 INFO:tasks.workunit.client.1.smithi148.stdout:7/322: fdatasync d8/d29/d44/d49/f62 0 2024-03-29T08:07:13.453 INFO:tasks.workunit.client.1.smithi148.stdout:3/292: dwrite de/f17 [0,4194304] 0 2024-03-29T08:07:13.454 INFO:tasks.workunit.client.1.smithi148.stdout:6/266: symlink d6/d9/d1e/d20/d30/l45 0 2024-03-29T08:07:13.460 INFO:tasks.workunit.client.1.smithi148.stdout:8/345: creat d17/d74/d39/f76 x:0 0 0 2024-03-29T08:07:13.461 INFO:tasks.workunit.client.1.smithi148.stdout:6/267: symlink d6/l46 0 2024-03-29T08:07:13.464 INFO:tasks.workunit.client.1.smithi148.stdout:2/301: mknod d7/da/c64 0 2024-03-29T08:07:13.465 INFO:tasks.workunit.client.1.smithi148.stdout:3/293: truncate de/d24/f36 5136198 0 2024-03-29T08:07:13.468 INFO:tasks.workunit.client.1.smithi148.stdout:5/305: rename d0/db/de/d16/c44 to d0/db/d1e/d35/d42/c65 0 2024-03-29T08:07:13.471 INFO:tasks.workunit.client.1.smithi148.stdout:1/282: truncate d3/d18/f20 1382128 0 2024-03-29T08:07:13.471 INFO:tasks.workunit.client.1.smithi148.stdout:8/346: rmdir d17/d74/d39/d6f 39 2024-03-29T08:07:13.473 INFO:tasks.workunit.client.1.smithi148.stdout:4/270: dwrite df/f1e [0,4194304] 0 2024-03-29T08:07:13.476 INFO:tasks.workunit.client.1.smithi148.stdout:3/294: fdatasync de/d24/d33/d39/d46/f21 0 2024-03-29T08:07:13.477 INFO:tasks.workunit.client.1.smithi148.stdout:5/306: fsync d0/db/d1e/d35/d42/f43 0 2024-03-29T08:07:13.479 INFO:tasks.workunit.client.1.smithi148.stdout:6/268: truncate d6/fd 1267972 0 2024-03-29T08:07:13.482 INFO:tasks.workunit.client.0.smithi037.stdout:1/317: sync 2024-03-29T08:07:13.483 INFO:tasks.workunit.client.0.smithi037.stdout:5/343: sync 2024-03-29T08:07:13.483 INFO:tasks.workunit.client.0.smithi037.stdout:7/271: sync 2024-03-29T08:07:13.483 INFO:tasks.workunit.client.0.smithi037.stdout:4/203: sync 2024-03-29T08:07:13.483 INFO:tasks.workunit.client.0.smithi037.stdout:8/243: sync 2024-03-29T08:07:13.483 INFO:tasks.workunit.client.0.smithi037.stdout:0/147: sync 2024-03-29T08:07:13.488 INFO:tasks.workunit.client.0.smithi037.stdout:4/204: dread - d3/d4/d15/d1d/db/d45/f2d zero size 2024-03-29T08:07:13.488 INFO:tasks.workunit.client.0.smithi037.stdout:5/344: stat d8/d18/c1a 0 2024-03-29T08:07:13.488 INFO:tasks.workunit.client.0.smithi037.stdout:8/244: fdatasync d5/f1b 0 2024-03-29T08:07:13.489 INFO:tasks.workunit.client.1.smithi148.stdout:5/307: mkdir d0/db/d1e/d5b/d66 0 2024-03-29T08:07:13.491 INFO:tasks.workunit.client.1.smithi148.stdout:4/271: creat df/d10/d26/d34/f61 x:0 0 0 2024-03-29T08:07:13.492 INFO:tasks.workunit.client.0.smithi037.stdout:8/245: fsync d5/f1e 0 2024-03-29T08:07:13.492 INFO:tasks.workunit.client.0.smithi037.stdout:0/148: write d9/da/f24 [2070871,12275] 0 2024-03-29T08:07:13.493 INFO:tasks.workunit.client.1.smithi148.stdout:3/295: mkdir de/d24/d33/d39/d46/d30/d37/d62/d63 0 2024-03-29T08:07:13.493 INFO:tasks.workunit.client.0.smithi037.stdout:1/318: write d2/d6/dc/f35 [216205,1259] 0 2024-03-29T08:07:13.493 INFO:tasks.workunit.client.0.smithi037.stdout:7/272: unlink dd/f12 0 2024-03-29T08:07:13.494 INFO:tasks.workunit.client.1.smithi148.stdout:4/272: chown df/f14 1425275 1 2024-03-29T08:07:13.497 INFO:tasks.workunit.client.1.smithi148.stdout:1/283: dwrite d3/de/d21/d32/f4e [0,4194304] 0 2024-03-29T08:07:13.498 INFO:tasks.workunit.client.0.smithi037.stdout:7/273: chown dd/d1d/d34/d54 50 1 2024-03-29T08:07:13.499 INFO:tasks.workunit.client.1.smithi148.stdout:9/278: sync 2024-03-29T08:07:13.500 INFO:tasks.workunit.client.0.smithi037.stdout:5/345: mknod d8/d4d/c6d 0 2024-03-29T08:07:13.502 INFO:tasks.workunit.client.1.smithi148.stdout:8/347: dwrite d17/d74/f6e [0,4194304] 0 2024-03-29T08:07:13.503 INFO:tasks.workunit.client.0.smithi037.stdout:8/246: mkdir d5/d10/d2d/d34/d50 0 2024-03-29T08:07:13.503 INFO:tasks.workunit.client.0.smithi037.stdout:5/346: chown d8/d34/l3c 147271719 1 2024-03-29T08:07:13.503 INFO:tasks.workunit.client.1.smithi148.stdout:0/292: dread db/de/df/d11/d2d/f36 [0,4194304] 0 2024-03-29T08:07:13.503 INFO:tasks.workunit.client.1.smithi148.stdout:7/323: write d8/d14/f21 [1624448,97556] 0 2024-03-29T08:07:13.507 INFO:tasks.workunit.client.1.smithi148.stdout:2/302: write d7/da/d33/d45/d4d/f29 [60328,12771] 0 2024-03-29T08:07:13.510 INFO:tasks.workunit.client.0.smithi037.stdout:4/205: symlink d3/d4/d15/d1d/db/d45/l47 0 2024-03-29T08:07:13.511 INFO:tasks.workunit.client.1.smithi148.stdout:6/269: getdents d6/d9/d1e/d22/d28 0 2024-03-29T08:07:13.513 INFO:tasks.workunit.client.0.smithi037.stdout:8/247: symlink d5/d10/d2d/d34/l51 0 2024-03-29T08:07:13.516 INFO:tasks.workunit.client.1.smithi148.stdout:4/273: unlink f7 0 2024-03-29T08:07:13.519 INFO:tasks.workunit.client.0.smithi037.stdout:5/347: creat d8/f6e x:0 0 0 2024-03-29T08:07:13.524 INFO:tasks.workunit.client.0.smithi037.stdout:4/206: mknod d3/d4/c48 0 2024-03-29T08:07:13.525 INFO:tasks.workunit.client.1.smithi148.stdout:8/348: truncate d17/f1a 607672 0 2024-03-29T08:07:13.525 INFO:tasks.workunit.client.1.smithi148.stdout:7/324: write d8/f36 [3271859,24791] 0 2024-03-29T08:07:13.526 INFO:tasks.workunit.client.0.smithi037.stdout:8/248: symlink d5/d10/d45/d4b/l52 0 2024-03-29T08:07:13.526 INFO:tasks.workunit.client.1.smithi148.stdout:5/308: dwrite d0/f12 [0,4194304] 0 2024-03-29T08:07:13.527 INFO:tasks.workunit.client.1.smithi148.stdout:8/349: fdatasync d17/d74/d39/f76 0 2024-03-29T08:07:13.530 INFO:tasks.workunit.client.0.smithi037.stdout:5/348: symlink d8/l6f 0 2024-03-29T08:07:13.532 INFO:tasks.workunit.client.1.smithi148.stdout:4/274: mknod df/d10/d26/d34/d43/c62 0 2024-03-29T08:07:13.536 INFO:tasks.workunit.client.1.smithi148.stdout:0/293: getdents db/de/df/d2b/d3b/d47 0 2024-03-29T08:07:13.537 INFO:tasks.workunit.client.0.smithi037.stdout:0/149: dread d9/da/dd/d13/f21 [0,4194304] 0 2024-03-29T08:07:13.538 INFO:tasks.workunit.client.0.smithi037.stdout:1/319: dread d2/d6/dc/f1e [0,4194304] 0 2024-03-29T08:07:13.540 INFO:tasks.workunit.client.0.smithi037.stdout:4/207: creat d3/d4/d15/f49 x:0 0 0 2024-03-29T08:07:13.540 INFO:tasks.workunit.client.1.smithi148.stdout:9/279: dwrite db/d10/f29 [0,4194304] 0 2024-03-29T08:07:13.540 INFO:tasks.workunit.client.1.smithi148.stdout:4/275: dread - df/d2d/f50 zero size 2024-03-29T08:07:13.540 INFO:tasks.workunit.client.1.smithi148.stdout:8/350: write d17/d74/f4a [1468975,20635] 0 2024-03-29T08:07:13.543 INFO:tasks.workunit.client.0.smithi037.stdout:8/249: mknod d5/d10/d2d/d34/d50/c53 0 2024-03-29T08:07:13.545 INFO:tasks.workunit.client.1.smithi148.stdout:4/276: dread - df/d10/d17/f29 zero size 2024-03-29T08:07:13.545 INFO:tasks.workunit.client.0.smithi037.stdout:1/320: rmdir d2/d6/dc/d18 39 2024-03-29T08:07:13.545 INFO:tasks.workunit.client.0.smithi037.stdout:7/274: write dd/f48 [781290,40187] 0 2024-03-29T08:07:13.546 INFO:tasks.workunit.client.1.smithi148.stdout:8/351: fdatasync d17/d74/f48 0 2024-03-29T08:07:13.548 INFO:tasks.workunit.client.0.smithi037.stdout:7/275: chown dd/d21/d35 48 1 2024-03-29T08:07:13.551 INFO:tasks.workunit.client.0.smithi037.stdout:0/150: rename d9/da/c28 to d9/da/d1b/d22/c2b 0 2024-03-29T08:07:13.553 INFO:tasks.workunit.client.1.smithi148.stdout:5/309: unlink d0/db/d1e/d35/c40 0 2024-03-29T08:07:13.555 INFO:tasks.workunit.client.1.smithi148.stdout:2/303: dwrite d7/da/d33/d45/d4d/f38 [0,4194304] 0 2024-03-29T08:07:13.555 INFO:tasks.workunit.client.0.smithi037.stdout:8/250: unlink d5/d10/d2d/d34/l51 0 2024-03-29T08:07:13.558 INFO:tasks.workunit.client.1.smithi148.stdout:9/280: unlink db/d22/d26/d54/d4d/d35/f44 0 2024-03-29T08:07:13.559 INFO:tasks.workunit.client.1.smithi148.stdout:6/270: dwrite d6/d9/d1e/d20/d30/f35 [0,4194304] 0 2024-03-29T08:07:13.561 INFO:tasks.workunit.client.0.smithi037.stdout:7/276: creat dd/d1d/d49/f5a x:0 0 0 2024-03-29T08:07:13.561 INFO:tasks.workunit.client.0.smithi037.stdout:1/321: chown d2/d6/dc/d18/d1d/d21/f55 39 1 2024-03-29T08:07:13.561 INFO:tasks.workunit.client.1.smithi148.stdout:2/304: readlink l6 0 2024-03-29T08:07:13.561 INFO:tasks.workunit.client.0.smithi037.stdout:4/208: rmdir d3/d4/d15/d43 0 2024-03-29T08:07:13.564 INFO:tasks.workunit.client.1.smithi148.stdout:7/325: link d8/d14/d19/f39 d8/d29/d44/d49/f76 0 2024-03-29T08:07:13.566 INFO:tasks.workunit.client.1.smithi148.stdout:4/277: creat df/d10/d26/f63 x:0 0 0 2024-03-29T08:07:13.566 INFO:tasks.workunit.client.1.smithi148.stdout:2/305: readlink d7/da/l1d 0 2024-03-29T08:07:13.567 INFO:tasks.workunit.client.1.smithi148.stdout:5/310: mkdir d0/db/d1e/d35/d42/d67 0 2024-03-29T08:07:13.571 INFO:tasks.workunit.client.0.smithi037.stdout:1/322: mkdir d2/d6/dc/d18/d1d/d5e/d72 0 2024-03-29T08:07:13.574 INFO:tasks.workunit.client.1.smithi148.stdout:3/296: dwrite de/d24/d33/d39/d46/f25 [0,4194304] 0 2024-03-29T08:07:13.575 INFO:tasks.workunit.client.1.smithi148.stdout:5/311: read - d0/db/d1e/f4a zero size 2024-03-29T08:07:13.575 INFO:tasks.workunit.client.1.smithi148.stdout:7/326: readlink d8/d29/l3b 0 2024-03-29T08:07:13.576 INFO:tasks.workunit.client.0.smithi037.stdout:4/209: creat d3/d4/d15/d1d/db/d45/d1e/f4a x:0 0 0 2024-03-29T08:07:13.578 INFO:tasks.workunit.client.0.smithi037.stdout:0/151: getdents d9/da/dd 0 2024-03-29T08:07:13.581 INFO:tasks.workunit.client.0.smithi037.stdout:5/349: truncate d8/d18/f40 2942368 0 2024-03-29T08:07:13.582 INFO:tasks.workunit.client.1.smithi148.stdout:3/297: dread - de/d3b/f4e zero size 2024-03-29T08:07:13.583 INFO:tasks.workunit.client.1.smithi148.stdout:7/327: readlink d8/l24 0 2024-03-29T08:07:13.589 INFO:tasks.workunit.client.0.smithi037.stdout:8/251: dwrite d5/f33 [0,4194304] 0 2024-03-29T08:07:13.594 INFO:tasks.workunit.client.0.smithi037.stdout:8/252: write d5/f33 [3570266,7077] 0 2024-03-29T08:07:13.595 INFO:tasks.workunit.client.1.smithi148.stdout:8/352: dwrite d17/d74/f4a [0,4194304] 0 2024-03-29T08:07:13.595 INFO:tasks.workunit.client.0.smithi037.stdout:5/350: rename d8/d34/d3b to d8/d22/d4f/d70 0 2024-03-29T08:07:13.598 INFO:tasks.workunit.client.1.smithi148.stdout:3/298: chown de/d24/d33/d39/d46/f21 12767221 1 2024-03-29T08:07:13.598 INFO:tasks.workunit.client.1.smithi148.stdout:6/271: getdents d6/d9/d1e/d20 0 2024-03-29T08:07:13.599 INFO:tasks.workunit.client.1.smithi148.stdout:7/328: mknod d8/d38/d3c/c77 0 2024-03-29T08:07:13.605 INFO:tasks.workunit.client.0.smithi037.stdout:8/253: creat d5/d10/d2d/d34/d50/f54 x:0 0 0 2024-03-29T08:07:13.605 INFO:tasks.workunit.client.1.smithi148.stdout:3/299: fdatasync de/f35 0 2024-03-29T08:07:13.608 INFO:tasks.workunit.client.1.smithi148.stdout:0/294: dread db/de/df/d11/f1b [4194304,4194304] 0 2024-03-29T08:07:13.608 INFO:tasks.workunit.client.1.smithi148.stdout:7/329: rmdir d8/d29/d44/d49 39 2024-03-29T08:07:13.608 INFO:tasks.workunit.client.1.smithi148.stdout:9/281: dwrite db/d22/d26/d54/d4d/f18 [4194304,4194304] 0 2024-03-29T08:07:13.609 INFO:tasks.workunit.client.1.smithi148.stdout:6/272: chown d6/d9/f1f 22964669 1 2024-03-29T08:07:13.610 INFO:tasks.workunit.client.1.smithi148.stdout:4/278: dwrite df/f1e [0,4194304] 0 2024-03-29T08:07:13.612 INFO:tasks.workunit.client.1.smithi148.stdout:3/300: write de/d3b/f4e [945803,74623] 0 2024-03-29T08:07:13.616 INFO:tasks.workunit.client.1.smithi148.stdout:1/284: sync 2024-03-29T08:07:13.617 INFO:tasks.workunit.client.0.smithi037.stdout:4/210: dwrite d3/d4/d15/d1d/db/d45/d1e/f4a [0,4194304] 0 2024-03-29T08:07:13.618 INFO:tasks.workunit.client.1.smithi148.stdout:1/285: fsync d3/f7 0 2024-03-29T08:07:13.618 INFO:tasks.workunit.client.1.smithi148.stdout:0/295: creat db/de/df/d2b/d3b/d47/d4c/f5f x:0 0 0 2024-03-29T08:07:13.619 INFO:tasks.workunit.client.0.smithi037.stdout:8/254: symlink d5/d25/d46/d4e/l55 0 2024-03-29T08:07:13.619 INFO:tasks.workunit.client.0.smithi037.stdout:7/277: dread dd/ff [0,4194304] 0 2024-03-29T08:07:13.620 INFO:tasks.workunit.client.1.smithi148.stdout:6/273: creat d6/d9/d1e/d20/d30/f47 x:0 0 0 2024-03-29T08:07:13.621 INFO:tasks.workunit.client.1.smithi148.stdout:7/330: read d8/d29/d44/d49/f68 [510334,107944] 0 2024-03-29T08:07:13.624 INFO:tasks.workunit.client.0.smithi037.stdout:4/211: chown d3/f44 32063 1 2024-03-29T08:07:13.624 INFO:tasks.workunit.client.0.smithi037.stdout:2/167: sync 2024-03-29T08:07:13.628 INFO:tasks.workunit.client.1.smithi148.stdout:8/353: dread d17/d18/d26/f35 [0,4194304] 0 2024-03-29T08:07:13.629 INFO:tasks.workunit.client.1.smithi148.stdout:0/296: dread f9 [0,4194304] 0 2024-03-29T08:07:13.629 INFO:tasks.workunit.client.1.smithi148.stdout:4/279: link df/l2a df/d60/l64 0 2024-03-29T08:07:13.630 INFO:tasks.workunit.client.1.smithi148.stdout:2/306: dread d7/da/d33/d45/d4d/f16 [0,4194304] 0 2024-03-29T08:07:13.633 INFO:tasks.workunit.client.0.smithi037.stdout:4/212: stat d3/c5 0 2024-03-29T08:07:13.634 INFO:tasks.workunit.client.1.smithi148.stdout:0/297: stat db/de/df/d11/d2d/d3e 0 2024-03-29T08:07:13.634 INFO:tasks.workunit.client.1.smithi148.stdout:3/301: truncate de/d24/d33/d39/d46/f21 2762978 0 2024-03-29T08:07:13.636 INFO:tasks.workunit.client.1.smithi148.stdout:7/331: mkdir d8/d14/d45/d4d/d78 0 2024-03-29T08:07:13.637 INFO:tasks.workunit.client.0.smithi037.stdout:4/213: chown d3/d4/d27/l28 199602087 1 2024-03-29T08:07:13.639 INFO:tasks.workunit.client.1.smithi148.stdout:2/307: write d7/d24/f5d [4947617,89241] 0 2024-03-29T08:07:13.639 INFO:tasks.workunit.client.1.smithi148.stdout:9/282: dread db/d22/d26/f38 [0,4194304] 0 2024-03-29T08:07:13.639 INFO:tasks.workunit.client.1.smithi148.stdout:4/280: rmdir df/d10 39 2024-03-29T08:07:13.639 INFO:tasks.workunit.client.0.smithi037.stdout:8/255: unlink d5/d10/c47 0 2024-03-29T08:07:13.640 INFO:tasks.workunit.client.0.smithi037.stdout:0/152: write f5 [1217160,105212] 0 2024-03-29T08:07:13.644 INFO:tasks.workunit.client.0.smithi037.stdout:1/323: dread d2/f14 [0,4194304] 0 2024-03-29T08:07:13.647 INFO:tasks.workunit.client.0.smithi037.stdout:0/153: write d9/f27 [875449,59423] 0 2024-03-29T08:07:13.647 INFO:tasks.workunit.client.1.smithi148.stdout:0/298: write db/de/df/d2b/d3b/d47/f2c [305042,111009] 0 2024-03-29T08:07:13.648 INFO:tasks.workunit.client.0.smithi037.stdout:4/214: chown d3/d4/d15/d1d/db/d45/c33 213895 1 2024-03-29T08:07:13.648 INFO:tasks.workunit.client.0.smithi037.stdout:3/272: sync 2024-03-29T08:07:13.648 INFO:tasks.workunit.client.0.smithi037.stdout:9/430: sync 2024-03-29T08:07:13.649 INFO:tasks.workunit.client.1.smithi148.stdout:4/281: chown df/d10/d26/d34/l55 8 1 2024-03-29T08:07:13.652 INFO:tasks.workunit.client.0.smithi037.stdout:1/324: chown d2/d6/dc/d5d/d62 200 1 2024-03-29T08:07:13.652 INFO:tasks.workunit.client.0.smithi037.stdout:2/168: write d0/f1b [909291,44432] 0 2024-03-29T08:07:13.653 INFO:tasks.workunit.client.1.smithi148.stdout:7/332: rename d8/d29/d44/d49/c5f to d8/d60/c79 0 2024-03-29T08:07:13.653 INFO:tasks.workunit.client.1.smithi148.stdout:6/274: dread d6/d9/f2c [0,4194304] 0 2024-03-29T08:07:13.655 INFO:tasks.workunit.client.0.smithi037.stdout:9/431: dread - d1/d4/d19/d69/d75/f8b zero size 2024-03-29T08:07:13.665 INFO:tasks.workunit.client.1.smithi148.stdout:2/308: rmdir d7/da/d33/d45/d4d/d2e 39 2024-03-29T08:07:13.667 INFO:tasks.workunit.client.1.smithi148.stdout:1/286: getdents d3/de/d21 0 2024-03-29T08:07:13.672 INFO:tasks.workunit.client.1.smithi148.stdout:8/354: getdents d17/d1d 0 2024-03-29T08:07:13.673 INFO:tasks.workunit.client.0.smithi037.stdout:0/154: dread d9/da/dd/f23 [0,4194304] 0 2024-03-29T08:07:13.674 INFO:tasks.workunit.client.1.smithi148.stdout:9/283: dread db/d22/d26/f38 [0,4194304] 0 2024-03-29T08:07:13.676 INFO:tasks.workunit.client.0.smithi037.stdout:5/351: dwrite d8/d22/f27 [0,4194304] 0 2024-03-29T08:07:13.678 INFO:tasks.workunit.client.1.smithi148.stdout:4/282: mkdir df/d2d/d65 0 2024-03-29T08:07:13.678 INFO:tasks.workunit.client.1.smithi148.stdout:9/284: fsync db/d10/f29 0 2024-03-29T08:07:13.679 INFO:tasks.workunit.client.1.smithi148.stdout:8/355: truncate d17/d74/d39/f76 964824 0 2024-03-29T08:07:13.682 INFO:tasks.workunit.client.0.smithi037.stdout:8/256: chown d5/d10/d19/d3d/d4a/d4c/l4f 0 1 2024-03-29T08:07:13.684 INFO:tasks.workunit.client.1.smithi148.stdout:4/283: read df/f1e [2834995,98743] 0 2024-03-29T08:07:13.686 INFO:tasks.workunit.client.1.smithi148.stdout:8/356: write d17/d74/f5a [405194,53261] 0 2024-03-29T08:07:13.690 INFO:tasks.workunit.client.1.smithi148.stdout:7/333: dwrite d8/d38/f71 [0,4194304] 0 2024-03-29T08:07:13.691 INFO:tasks.workunit.client.1.smithi148.stdout:0/299: symlink db/de/df/d2b/d3b/d47/d4c/d5a/l60 0 2024-03-29T08:07:13.696 INFO:tasks.workunit.client.0.smithi037.stdout:0/155: mkdir d9/da/d1b/d22/d2c 0 2024-03-29T08:07:13.696 INFO:tasks.workunit.client.1.smithi148.stdout:3/302: write de/f2e [113356,62047] 0 2024-03-29T08:07:13.699 INFO:tasks.workunit.client.0.smithi037.stdout:0/156: chown d9/da/f24 1222 1 2024-03-29T08:07:13.703 INFO:tasks.workunit.client.0.smithi037.stdout:7/278: rename dd/d1d/l45 to dd/d1d/l5b 0 2024-03-29T08:07:13.703 INFO:tasks.workunit.client.1.smithi148.stdout:5/312: dread d0/db/de/d16/d2e/f39 [0,4194304] 0 2024-03-29T08:07:13.703 INFO:tasks.workunit.client.1.smithi148.stdout:7/334: creat d8/d60/f7a x:0 0 0 2024-03-29T08:07:13.705 INFO:tasks.workunit.client.0.smithi037.stdout:2/169: mkdir d0/d20/d2e/d38 0 2024-03-29T08:07:13.707 INFO:tasks.workunit.client.0.smithi037.stdout:8/257: dwrite d5/f33 [0,4194304] 0 2024-03-29T08:07:13.707 INFO:tasks.workunit.client.0.smithi037.stdout:7/279: chown dd/f14 1622511 1 2024-03-29T08:07:13.708 INFO:tasks.workunit.client.1.smithi148.stdout:8/357: creat d17/d74/d39/f77 x:0 0 0 2024-03-29T08:07:13.710 INFO:tasks.workunit.client.1.smithi148.stdout:5/313: write d0/f12 [2308770,8534] 0 2024-03-29T08:07:13.712 INFO:tasks.workunit.client.0.smithi037.stdout:8/258: dread - d5/d10/d19/f2f zero size 2024-03-29T08:07:13.717 INFO:tasks.workunit.client.0.smithi037.stdout:5/352: dread d8/d18/f50 [0,4194304] 0 2024-03-29T08:07:13.717 INFO:tasks.workunit.client.0.smithi037.stdout:8/259: chown d5/l18 5528 1 2024-03-29T08:07:13.718 INFO:tasks.workunit.client.1.smithi148.stdout:8/358: write d17/d74/d39/f77 [719860,20891] 0 2024-03-29T08:07:13.719 INFO:tasks.workunit.client.0.smithi037.stdout:2/170: fsync d0/f2 0 2024-03-29T08:07:13.720 INFO:tasks.workunit.client.1.smithi148.stdout:9/285: dwrite db/f43 [0,4194304] 0 2024-03-29T08:07:13.721 INFO:tasks.workunit.client.1.smithi148.stdout:6/275: dread d6/f14 [0,4194304] 0 2024-03-29T08:07:13.721 INFO:tasks.workunit.client.0.smithi037.stdout:5/353: write d8/f1f [3090083,63689] 0 2024-03-29T08:07:13.723 INFO:tasks.workunit.client.0.smithi037.stdout:0/157: creat d9/da/d1b/d22/d2c/f2d x:0 0 0 2024-03-29T08:07:13.724 INFO:tasks.workunit.client.0.smithi037.stdout:8/260: stat d5/d10/d19/f2f 0 2024-03-29T08:07:13.726 INFO:tasks.workunit.client.1.smithi148.stdout:5/314: creat d0/db/de/d16/d2e/f68 x:0 0 0 2024-03-29T08:07:13.727 INFO:tasks.workunit.client.1.smithi148.stdout:3/303: getdents de/d3b/d49 0 2024-03-29T08:07:13.728 INFO:tasks.workunit.client.1.smithi148.stdout:4/284: dwrite df/f1f [0,4194304] 0 2024-03-29T08:07:13.729 INFO:tasks.workunit.client.0.smithi037.stdout:2/171: creat d0/d20/d22/f39 x:0 0 0 2024-03-29T08:07:13.729 INFO:tasks.workunit.client.0.smithi037.stdout:5/354: truncate d8/d34/f3a 657422 0 2024-03-29T08:07:13.730 INFO:tasks.workunit.client.0.smithi037.stdout:7/280: truncate dd/f1f 3367038 0 2024-03-29T08:07:13.731 INFO:tasks.workunit.client.1.smithi148.stdout:6/276: chown d6/d9/d1e/d22/f34 338788 1 2024-03-29T08:07:13.733 INFO:tasks.workunit.client.0.smithi037.stdout:5/355: chown d8/d34/f6a 26 1 2024-03-29T08:07:13.735 INFO:tasks.workunit.client.1.smithi148.stdout:8/359: fdatasync d17/d74/d39/f77 0 2024-03-29T08:07:13.738 INFO:tasks.workunit.client.1.smithi148.stdout:6/277: fsync d6/d9/d1e/f26 0 2024-03-29T08:07:13.738 INFO:tasks.workunit.client.0.smithi037.stdout:0/158: mknod d9/da/dd/d1c/c2e 0 2024-03-29T08:07:13.739 INFO:tasks.workunit.client.1.smithi148.stdout:6/278: readlink d6/l46 0 2024-03-29T08:07:13.740 INFO:tasks.workunit.client.0.smithi037.stdout:5/356: chown d8/f1e 1365632 1 2024-03-29T08:07:13.742 INFO:tasks.workunit.client.1.smithi148.stdout:9/286: mknod db/d22/d26/d54/d4d/d35/c57 0 2024-03-29T08:07:13.742 INFO:tasks.workunit.client.0.smithi037.stdout:2/172: fsync d0/f1f 0 2024-03-29T08:07:13.744 INFO:tasks.workunit.client.1.smithi148.stdout:3/304: truncate de/f35 881480 0 2024-03-29T08:07:13.745 INFO:tasks.workunit.client.1.smithi148.stdout:9/287: write db/d22/f51 [1023170,41577] 0 2024-03-29T08:07:13.746 INFO:tasks.workunit.client.0.smithi037.stdout:5/357: readlink d8/d22/d4f/d70/l53 0 2024-03-29T08:07:13.747 INFO:tasks.workunit.client.0.smithi037.stdout:4/215: truncate d3/d4/d15/d1d/db/d45/f30 596256 0 2024-03-29T08:07:13.751 INFO:tasks.workunit.client.0.smithi037.stdout:0/159: mknod d9/da/dd/d13/c2f 0 2024-03-29T08:07:13.751 INFO:tasks.workunit.client.1.smithi148.stdout:1/287: write d3/d18/f20 [1013463,129698] 0 2024-03-29T08:07:13.752 INFO:tasks.workunit.client.1.smithi148.stdout:8/360: rmdir d17/d18/d26/d4f 39 2024-03-29T08:07:13.754 INFO:tasks.workunit.client.0.smithi037.stdout:4/216: dread - d3/d4/d15/d1d/db/d45/f2d zero size 2024-03-29T08:07:13.755 INFO:tasks.workunit.client.1.smithi148.stdout:4/285: dwrite df/d2d/f50 [0,4194304] 0 2024-03-29T08:07:13.755 INFO:tasks.workunit.client.0.smithi037.stdout:7/281: dwrite dd/d21/d35/f4c [0,4194304] 0 2024-03-29T08:07:13.757 INFO:tasks.workunit.client.0.smithi037.stdout:3/273: rename dd/d32 to dd/d46/d2d/d3a/d5f 0 2024-03-29T08:07:13.762 INFO:tasks.workunit.client.0.smithi037.stdout:0/160: truncate d9/da/dd/f23 2097389 0 2024-03-29T08:07:13.767 INFO:tasks.workunit.client.1.smithi148.stdout:3/305: creat de/d3b/d49/f64 x:0 0 0 2024-03-29T08:07:13.768 INFO:tasks.workunit.client.1.smithi148.stdout:2/309: dread d7/da/d33/f3e [0,4194304] 0 2024-03-29T08:07:13.769 INFO:tasks.workunit.client.0.smithi037.stdout:2/173: truncate d0/f18 3211288 0 2024-03-29T08:07:13.770 INFO:tasks.workunit.client.1.smithi148.stdout:8/361: dread - d17/d18/f36 zero size 2024-03-29T08:07:13.772 INFO:tasks.workunit.client.1.smithi148.stdout:5/315: link d0/l8 d0/db/d1e/d35/d42/d55/l69 0 2024-03-29T08:07:13.773 INFO:tasks.workunit.client.1.smithi148.stdout:9/288: mkdir db/d22/d26/d54/d4d/d35/d4b/d58 0 2024-03-29T08:07:13.774 INFO:tasks.workunit.client.0.smithi037.stdout:3/274: creat dd/d46/d2d/f60 x:0 0 0 2024-03-29T08:07:13.775 INFO:tasks.workunit.client.1.smithi148.stdout:6/279: dwrite d6/d9/d1e/d20/f37 [0,4194304] 0 2024-03-29T08:07:13.776 INFO:tasks.workunit.client.1.smithi148.stdout:7/335: dwrite d8/d14/d19/f63 [0,4194304] 0 2024-03-29T08:07:13.777 INFO:tasks.workunit.client.0.smithi037.stdout:7/282: mkdir dd/d21/d35/d42/d5c 0 2024-03-29T08:07:13.777 INFO:tasks.workunit.client.1.smithi148.stdout:2/310: truncate d7/f5a 523856 0 2024-03-29T08:07:13.777 INFO:tasks.workunit.client.0.smithi037.stdout:3/275: read dd/f1f [345338,46092] 0 2024-03-29T08:07:13.778 INFO:tasks.workunit.client.1.smithi148.stdout:2/311: readlink l4 0 2024-03-29T08:07:13.778 INFO:tasks.workunit.client.1.smithi148.stdout:1/288: mkdir d3/de/d25/d51 0 2024-03-29T08:07:13.779 INFO:tasks.workunit.client.1.smithi148.stdout:3/306: write de/d24/f31 [5865650,9181] 0 2024-03-29T08:07:13.781 INFO:tasks.workunit.client.0.smithi037.stdout:7/283: chown dd/d21/d35 1 1 2024-03-29T08:07:13.785 INFO:tasks.workunit.client.1.smithi148.stdout:5/316: fsync d0/db/d1e/f4a 0 2024-03-29T08:07:13.787 INFO:tasks.workunit.client.0.smithi037.stdout:2/174: chown d0/l12 427441215 1 2024-03-29T08:07:13.788 INFO:tasks.workunit.client.1.smithi148.stdout:1/289: chown d3/d18/f4b 4419440 1 2024-03-29T08:07:13.788 INFO:tasks.workunit.client.1.smithi148.stdout:5/317: chown d0/db/de/c2d 1 1 2024-03-29T08:07:13.788 INFO:tasks.workunit.client.0.smithi037.stdout:4/217: dwrite d3/d4/d15/d1d/fc [0,4194304] 0 2024-03-29T08:07:13.791 INFO:tasks.workunit.client.0.smithi037.stdout:5/358: rename d8/d4d/d5c/f65 to d8/d18/f71 0 2024-03-29T08:07:13.792 INFO:tasks.workunit.client.0.smithi037.stdout:2/175: readlink d0/d20/l25 0 2024-03-29T08:07:13.796 INFO:tasks.workunit.client.0.smithi037.stdout:7/284: truncate dd/d1d/d34/f59 416179 0 2024-03-29T08:07:13.796 INFO:tasks.workunit.client.0.smithi037.stdout:3/276: rmdir dd/d46/d2d/d3b 39 2024-03-29T08:07:13.797 INFO:tasks.workunit.client.1.smithi148.stdout:0/300: dread db/de/df/d2b/f35 [0,4194304] 0 2024-03-29T08:07:13.799 INFO:tasks.workunit.client.0.smithi037.stdout:5/359: truncate d8/f42 581990 0 2024-03-29T08:07:13.800 INFO:tasks.workunit.client.1.smithi148.stdout:6/280: mkdir d6/d9/d1e/d27/d48 0 2024-03-29T08:07:13.801 INFO:tasks.workunit.client.0.smithi037.stdout:9/432: symlink d1/d4/d19/l98 0 2024-03-29T08:07:13.802 INFO:tasks.workunit.client.0.smithi037.stdout:3/277: write dd/d46/f3c [1028819,38089] 0 2024-03-29T08:07:13.802 INFO:tasks.workunit.client.0.smithi037.stdout:8/261: dwrite f1 [0,4194304] 0 2024-03-29T08:07:13.802 INFO:tasks.workunit.client.1.smithi148.stdout:2/312: fsync d7/da/d33/d45/d4d/f16 0 2024-03-29T08:07:13.807 INFO:tasks.workunit.client.0.smithi037.stdout:4/218: creat d3/d4/d27/f4b x:0 0 0 2024-03-29T08:07:13.808 INFO:tasks.workunit.client.1.smithi148.stdout:3/307: creat de/d24/d33/d39/d46/f65 x:0 0 0 2024-03-29T08:07:13.815 INFO:tasks.workunit.client.0.smithi037.stdout:0/161: getdents d9/da/dd/d13 0 2024-03-29T08:07:13.816 INFO:tasks.workunit.client.1.smithi148.stdout:5/318: dwrite d0/f12 [0,4194304] 0 2024-03-29T08:07:13.816 INFO:tasks.workunit.client.0.smithi037.stdout:7/285: mkdir dd/d21/d35/d3b/d5d 0 2024-03-29T08:07:13.817 INFO:tasks.workunit.client.0.smithi037.stdout:5/360: mkdir d8/d4d/d72 0 2024-03-29T08:07:13.817 INFO:tasks.workunit.client.1.smithi148.stdout:8/362: write d17/f1a [1306541,129028] 0 2024-03-29T08:07:13.821 INFO:tasks.workunit.client.1.smithi148.stdout:8/363: chown d17/d1d 464039 1 2024-03-29T08:07:13.821 INFO:tasks.workunit.client.1.smithi148.stdout:1/290: read d3/de/d21/d32/f37 [1476991,88935] 0 2024-03-29T08:07:13.830 INFO:tasks.workunit.client.0.smithi037.stdout:3/278: getdents dd/d15/d18/d4b 0 2024-03-29T08:07:13.833 INFO:tasks.workunit.client.1.smithi148.stdout:2/313: creat d7/da/d33/d3c/f65 x:0 0 0 2024-03-29T08:07:13.833 INFO:tasks.workunit.client.1.smithi148.stdout:5/319: mkdir d0/db/d1e/d35/d6a 0 2024-03-29T08:07:13.834 INFO:tasks.workunit.client.0.smithi037.stdout:9/433: dwrite d1/d1f/d3a/d67/f6d [0,4194304] 0 2024-03-29T08:07:13.834 INFO:tasks.workunit.client.0.smithi037.stdout:2/176: getdents d0/d20/d2e 0 2024-03-29T08:07:13.834 INFO:tasks.workunit.client.1.smithi148.stdout:8/364: write d17/d74/f4a [2200899,118416] 0 2024-03-29T08:07:13.836 INFO:tasks.workunit.client.1.smithi148.stdout:0/301: rename db/de/df/l13 to db/de/df/d2b/d3b/d47/d4c/d5a/l61 0 2024-03-29T08:07:13.841 INFO:tasks.workunit.client.1.smithi148.stdout:0/302: chown db/dc/cd 16589 1 2024-03-29T08:07:13.842 INFO:tasks.workunit.client.1.smithi148.stdout:5/320: write d0/db/de/f52 [349745,104589] 0 2024-03-29T08:07:13.844 INFO:tasks.workunit.client.0.smithi037.stdout:4/219: fdatasync d3/d4/d15/d1d/fc 0 2024-03-29T08:07:13.844 INFO:tasks.workunit.client.1.smithi148.stdout:9/289: dread db/d22/d26/d54/d4d/f3d [0,4194304] 0 2024-03-29T08:07:13.845 INFO:tasks.workunit.client.1.smithi148.stdout:2/314: mkdir d7/d24/d4b/d66 0 2024-03-29T08:07:13.846 INFO:tasks.workunit.client.0.smithi037.stdout:0/162: getdents d9/da/d1b/d22 0 2024-03-29T08:07:13.847 INFO:tasks.workunit.client.1.smithi148.stdout:3/308: dwrite de/d24/f56 [0,4194304] 0 2024-03-29T08:07:13.847 INFO:tasks.workunit.client.1.smithi148.stdout:7/336: dread d8/d29/f54 [0,4194304] 0 2024-03-29T08:07:13.849 INFO:tasks.workunit.client.1.smithi148.stdout:9/290: chown db/lf 14 1 2024-03-29T08:07:13.852 INFO:tasks.workunit.client.1.smithi148.stdout:0/303: rmdir db/de/df/d11/d2d 39 2024-03-29T08:07:13.856 INFO:tasks.workunit.client.0.smithi037.stdout:3/279: mkdir dd/d15/d18/d4b/d61 0 2024-03-29T08:07:13.859 INFO:tasks.workunit.client.0.smithi037.stdout:2/177: rename d0/f1f to d0/d20/f3a 0 2024-03-29T08:07:13.861 INFO:tasks.workunit.client.1.smithi148.stdout:3/309: dread f8 [4194304,4194304] 0 2024-03-29T08:07:13.866 INFO:tasks.workunit.client.1.smithi148.stdout:5/321: dwrite d0/f4b [0,4194304] 0 2024-03-29T08:07:13.867 INFO:tasks.workunit.client.1.smithi148.stdout:7/337: symlink d8/d29/l7b 0 2024-03-29T08:07:13.868 INFO:tasks.workunit.client.1.smithi148.stdout:8/365: dwrite d17/f28 [0,4194304] 0 2024-03-29T08:07:13.873 INFO:tasks.workunit.client.0.smithi037.stdout:7/286: dwrite dd/d21/f32 [0,4194304] 0 2024-03-29T08:07:13.873 INFO:tasks.workunit.client.0.smithi037.stdout:8/262: dwrite d5/d10/f1d [0,4194304] 0 2024-03-29T08:07:13.873 INFO:tasks.workunit.client.1.smithi148.stdout:5/322: write d0/db/d1e/d36/d41/f58 [148018,110087] 0 2024-03-29T08:07:13.876 INFO:tasks.workunit.client.0.smithi037.stdout:6/203: sync 2024-03-29T08:07:13.885 INFO:tasks.workunit.client.1.smithi148.stdout:3/310: symlink de/d3b/l66 0 2024-03-29T08:07:13.887 INFO:tasks.workunit.client.1.smithi148.stdout:7/338: mknod d8/d14/d19/d37/c7c 0 2024-03-29T08:07:13.887 INFO:tasks.workunit.client.0.smithi037.stdout:4/220: dwrite d3/d4/f34 [0,4194304] 0 2024-03-29T08:07:13.896 INFO:tasks.workunit.client.1.smithi148.stdout:0/304: rename db/de/df/d11/f33 to db/de/f62 0 2024-03-29T08:07:13.897 INFO:tasks.workunit.client.1.smithi148.stdout:9/291: creat db/f59 x:0 0 0 2024-03-29T08:07:13.899 INFO:tasks.workunit.client.0.smithi037.stdout:0/163: link d9/da/dd/d1c/l19 d9/da/d1b/l30 0 2024-03-29T08:07:13.899 INFO:tasks.workunit.client.0.smithi037.stdout:1/325: sync 2024-03-29T08:07:13.899 INFO:tasks.workunit.client.1.smithi148.stdout:4/286: sync 2024-03-29T08:07:13.899 INFO:tasks.workunit.client.1.smithi148.stdout:6/281: sync 2024-03-29T08:07:13.902 INFO:tasks.workunit.client.1.smithi148.stdout:7/339: dread d8/d14/f16 [0,4194304] 0 2024-03-29T08:07:13.905 INFO:tasks.workunit.client.1.smithi148.stdout:3/311: mknod de/d24/d33/d39/d46/d30/d37/d62/d63/c67 0 2024-03-29T08:07:13.906 INFO:tasks.workunit.client.1.smithi148.stdout:9/292: write f4 [3620663,105532] 0 2024-03-29T08:07:13.906 INFO:tasks.workunit.client.0.smithi037.stdout:3/280: rename dd/d15/f21 to dd/d15/f62 0 2024-03-29T08:07:13.906 INFO:tasks.workunit.client.1.smithi148.stdout:5/323: dwrite d0/db/de/f50 [0,4194304] 0 2024-03-29T08:07:13.907 INFO:tasks.workunit.client.1.smithi148.stdout:7/340: chown d8/d38/f71 197471106 1 2024-03-29T08:07:13.908 INFO:tasks.workunit.client.1.smithi148.stdout:6/282: mknod d6/c49 0 2024-03-29T08:07:13.909 INFO:tasks.workunit.client.1.smithi148.stdout:2/315: dread d7/da/d33/d3c/f23 [4194304,4194304] 0 2024-03-29T08:07:13.911 INFO:tasks.workunit.client.1.smithi148.stdout:0/305: write db/de/df/d11/f1b [8000699,103845] 0 2024-03-29T08:07:13.914 INFO:tasks.workunit.client.1.smithi148.stdout:6/283: write d6/f1a [1993741,87068] 0 2024-03-29T08:07:13.914 INFO:tasks.workunit.client.1.smithi148.stdout:5/324: fsync d0/f4b 0 2024-03-29T08:07:13.917 INFO:tasks.workunit.client.1.smithi148.stdout:6/284: read d6/d9/d1e/d20/f37 [759533,52319] 0 2024-03-29T08:07:13.920 INFO:tasks.workunit.client.1.smithi148.stdout:3/312: read de/d24/f31 [5702417,56076] 0 2024-03-29T08:07:13.921 INFO:tasks.workunit.client.1.smithi148.stdout:6/285: fsync d6/d9/d1e/d22/d28/f3b 0 2024-03-29T08:07:13.922 INFO:tasks.workunit.client.0.smithi037.stdout:5/361: dread d8/fa [0,4194304] 0 2024-03-29T08:07:13.923 INFO:tasks.workunit.client.1.smithi148.stdout:7/341: unlink d8/fc 0 2024-03-29T08:07:13.923 INFO:tasks.workunit.client.0.smithi037.stdout:3/281: write dd/d5e/f5c [149606,30143] 0 2024-03-29T08:07:13.924 INFO:tasks.workunit.client.1.smithi148.stdout:0/306: rmdir db/de/df/d11/d1f 39 2024-03-29T08:07:13.927 INFO:tasks.workunit.client.1.smithi148.stdout:1/291: dread d3/de/d25/d35/d42/f15 [4194304,4194304] 0 2024-03-29T08:07:13.928 INFO:tasks.workunit.client.0.smithi037.stdout:8/263: dwrite d5/f33 [4194304,4194304] 0 2024-03-29T08:07:13.930 INFO:tasks.workunit.client.1.smithi148.stdout:4/287: dwrite df/f27 [0,4194304] 0 2024-03-29T08:07:13.932 INFO:tasks.workunit.client.0.smithi037.stdout:7/287: dwrite dd/d1d/d34/f37 [4194304,4194304] 0 2024-03-29T08:07:13.933 INFO:tasks.workunit.client.1.smithi148.stdout:5/325: creat d0/db/d1e/d35/d42/f6b x:0 0 0 2024-03-29T08:07:13.933 INFO:tasks.workunit.client.1.smithi148.stdout:4/288: readlink df/d10/d26/l44 0 2024-03-29T08:07:13.935 INFO:tasks.workunit.client.1.smithi148.stdout:4/289: truncate df/f14 4348458 0 2024-03-29T08:07:13.935 INFO:tasks.workunit.client.1.smithi148.stdout:5/326: fsync d0/db/d1e/d36/d41/f51 0 2024-03-29T08:07:13.936 INFO:tasks.workunit.client.1.smithi148.stdout:2/316: dwrite d7/da/d33/d45/d4d/d2e/d37/f43 [0,4194304] 0 2024-03-29T08:07:13.937 INFO:tasks.workunit.client.0.smithi037.stdout:7/288: dread - dd/f3e zero size 2024-03-29T08:07:13.941 INFO:tasks.workunit.client.0.smithi037.stdout:4/221: symlink d3/d4/d15/d1d/l4c 0 2024-03-29T08:07:13.948 INFO:tasks.workunit.client.1.smithi148.stdout:7/342: unlink d8/d14/f47 0 2024-03-29T08:07:13.949 INFO:tasks.workunit.client.1.smithi148.stdout:9/293: getdents db 0 2024-03-29T08:07:13.951 INFO:tasks.workunit.client.1.smithi148.stdout:7/343: write d8/d14/d45/d4d/f6b [926835,114602] 0 2024-03-29T08:07:13.955 INFO:tasks.workunit.client.1.smithi148.stdout:8/366: truncate d17/d18/f4d 101024 0 2024-03-29T08:07:13.955 INFO:tasks.workunit.client.1.smithi148.stdout:5/327: write d0/db/de/f50 [4773049,116283] 0 2024-03-29T08:07:13.956 INFO:tasks.workunit.client.0.smithi037.stdout:8/264: symlink d5/d10/d19/d3d/d4a/d4c/l56 0 2024-03-29T08:07:13.961 INFO:tasks.workunit.client.1.smithi148.stdout:1/292: creat d3/de/d21/d32/f52 x:0 0 0 2024-03-29T08:07:13.963 INFO:tasks.workunit.client.0.smithi037.stdout:6/204: getdents d0/d5/d3f 0 2024-03-29T08:07:13.966 INFO:tasks.workunit.client.1.smithi148.stdout:8/367: write d17/d18/d26/f27 [1136292,117470] 0 2024-03-29T08:07:13.967 INFO:tasks.workunit.client.0.smithi037.stdout:8/265: creat d5/d25/d46/d4e/f57 x:0 0 0 2024-03-29T08:07:13.971 INFO:tasks.workunit.client.1.smithi148.stdout:4/290: dread fc [0,4194304] 0 2024-03-29T08:07:13.971 INFO:tasks.workunit.client.0.smithi037.stdout:4/222: symlink d3/d4/d15/l4d 0 2024-03-29T08:07:13.974 INFO:tasks.workunit.client.0.smithi037.stdout:5/362: creat d8/f73 x:0 0 0 2024-03-29T08:07:13.977 INFO:tasks.workunit.client.0.smithi037.stdout:1/326: getdents d2/d6/dc 0 2024-03-29T08:07:13.978 INFO:tasks.workunit.client.1.smithi148.stdout:9/294: fdatasync db/d20/f1e 0 2024-03-29T08:07:13.978 INFO:tasks.workunit.client.1.smithi148.stdout:8/368: dread d17/d74/f4a [0,4194304] 0 2024-03-29T08:07:13.980 INFO:tasks.workunit.client.1.smithi148.stdout:0/307: mknod db/de/df/d11/d2d/c63 0 2024-03-29T08:07:13.982 INFO:tasks.workunit.client.0.smithi037.stdout:9/434: dread d1/d1f/d3a/f46 [0,4194304] 0 2024-03-29T08:07:13.982 INFO:tasks.workunit.client.0.smithi037.stdout:2/178: dwrite d0/d23/f30 [0,4194304] 0 2024-03-29T08:07:13.987 INFO:tasks.workunit.client.0.smithi037.stdout:4/223: stat d3/d4/d15/f49 0 2024-03-29T08:07:13.987 INFO:tasks.workunit.client.1.smithi148.stdout:1/293: unlink d3/de/d21/d32/f4e 0 2024-03-29T08:07:13.987 INFO:tasks.workunit.client.1.smithi148.stdout:4/291: dwrite df/d10/d17/f20 [4194304,4194304] 0 2024-03-29T08:07:13.995 INFO:tasks.workunit.client.1.smithi148.stdout:6/286: rename d6/d9/d1e/d20/d30/f3c to d6/d9/d1e/d22/f4a 0 2024-03-29T08:07:13.997 INFO:tasks.workunit.client.0.smithi037.stdout:0/164: dread d9/da/dd/d1c/f18 [0,4194304] 0 2024-03-29T08:07:13.997 INFO:tasks.workunit.client.1.smithi148.stdout:8/369: dwrite d17/d18/f36 [0,4194304] 0 2024-03-29T08:07:13.999 INFO:tasks.workunit.client.1.smithi148.stdout:6/287: truncate d6/d9/d1e/f3d 760745 0 2024-03-29T08:07:14.005 INFO:tasks.workunit.client.0.smithi037.stdout:6/205: getdents d0/d5/d1f 0 2024-03-29T08:07:14.006 INFO:tasks.workunit.client.0.smithi037.stdout:3/282: dwrite dd/d46/d2d/d3b/f4f [0,4194304] 0 2024-03-29T08:07:14.006 INFO:tasks.workunit.client.0.smithi037.stdout:5/363: creat d8/d18/d24/f74 x:0 0 0 2024-03-29T08:07:14.010 INFO:tasks.workunit.client.0.smithi037.stdout:6/206: write d0/d5/d1f/f3c [1041678,130419] 0 2024-03-29T08:07:14.014 INFO:tasks.workunit.client.0.smithi037.stdout:6/207: write d0/d5/d1a/f2f [4464926,27206] 0 2024-03-29T08:07:14.015 INFO:tasks.workunit.client.0.smithi037.stdout:7/289: dread dd/d1d/d34/f39 [0,4194304] 0 2024-03-29T08:07:14.015 INFO:tasks.workunit.client.0.smithi037.stdout:7/290: readlink dd/l58 0 2024-03-29T08:07:14.016 INFO:tasks.workunit.client.0.smithi037.stdout:8/266: dwrite d5/d25/f2e [0,4194304] 0 2024-03-29T08:07:14.029 INFO:tasks.workunit.client.0.smithi037.stdout:4/224: rename d3/d4/d15/f19 to d3/d4/d15/d1d/db/d45/d1e/f4e 0 2024-03-29T08:07:14.033 INFO:tasks.workunit.client.1.smithi148.stdout:9/295: mkdir db/d22/d26/d54/d4d/d35/d4b/d5a 0 2024-03-29T08:07:14.034 INFO:tasks.workunit.client.1.smithi148.stdout:9/296: stat db/d10/f29 0 2024-03-29T08:07:14.035 INFO:tasks.workunit.client.0.smithi037.stdout:5/364: creat d8/d18/d24/d45/f75 x:0 0 0 2024-03-29T08:07:14.035 INFO:tasks.workunit.client.1.smithi148.stdout:9/297: stat db/d20/c1d 0 2024-03-29T08:07:14.036 INFO:tasks.workunit.client.1.smithi148.stdout:0/308: mknod db/de/df/d2b/c64 0 2024-03-29T08:07:14.039 INFO:tasks.workunit.client.1.smithi148.stdout:0/309: readlink db/de/df/d2b/d3b/d47/d4c/d5a/l60 0 2024-03-29T08:07:14.040 INFO:tasks.workunit.client.0.smithi037.stdout:3/283: symlink dd/d15/d18/d30/l63 0 2024-03-29T08:07:14.040 INFO:tasks.workunit.client.1.smithi148.stdout:1/294: unlink d3/f2d 0 2024-03-29T08:07:14.041 INFO:tasks.workunit.client.0.smithi037.stdout:1/327: truncate d2/f6c 1075489 0 2024-03-29T08:07:14.042 INFO:tasks.workunit.client.1.smithi148.stdout:0/310: fsync db/de/df/d14/f3c 0 2024-03-29T08:07:14.042 INFO:tasks.workunit.client.0.smithi037.stdout:6/208: rmdir d0/d5/d1f 39 2024-03-29T08:07:14.050 INFO:tasks.workunit.client.1.smithi148.stdout:4/292: rmdir df/d2d 39 2024-03-29T08:07:14.052 INFO:tasks.workunit.client.1.smithi148.stdout:4/293: truncate df/d10/f31 753328 0 2024-03-29T08:07:14.058 INFO:tasks.workunit.client.1.smithi148.stdout:7/344: write d8/d14/f1c [928994,112142] 0 2024-03-29T08:07:14.059 INFO:tasks.workunit.client.0.smithi037.stdout:7/291: rename dd/d21/f55 to dd/d21/d35/d42/d5c/f5e 0 2024-03-29T08:07:14.060 INFO:tasks.workunit.client.0.smithi037.stdout:5/365: creat d8/d22/d37/d60/f76 x:0 0 0 2024-03-29T08:07:14.063 INFO:tasks.workunit.client.0.smithi037.stdout:0/165: link d9/da/d1b/d22/d2c/f2d d9/da/dd/d13/f31 0 2024-03-29T08:07:14.065 INFO:tasks.workunit.client.0.smithi037.stdout:4/225: mkdir d3/d4/d15/d1d/d4f 0 2024-03-29T08:07:14.068 INFO:tasks.workunit.client.0.smithi037.stdout:6/209: dwrite d0/d5/d1e/f12 [0,4194304] 0 2024-03-29T08:07:14.070 INFO:tasks.workunit.client.1.smithi148.stdout:3/313: rename de/d24/d33/d39/d46/f40 to de/d3b/d49/f68 0 2024-03-29T08:07:14.073 INFO:tasks.workunit.client.0.smithi037.stdout:6/210: stat d0/d5/fb 0 2024-03-29T08:07:14.073 INFO:tasks.workunit.client.0.smithi037.stdout:4/226: write d3/d4/d15/d1d/db/d45/f2d [800979,40742] 0 2024-03-29T08:07:14.075 INFO:tasks.workunit.client.1.smithi148.stdout:2/317: dread d7/da/fe [4194304,4194304] 0 2024-03-29T08:07:14.076 INFO:tasks.workunit.client.1.smithi148.stdout:9/298: creat db/d22/d26/d54/d4d/f5b x:0 0 0 2024-03-29T08:07:14.077 INFO:tasks.workunit.client.0.smithi037.stdout:6/211: fsync d0/d5/f14 0 2024-03-29T08:07:14.078 INFO:tasks.workunit.client.0.smithi037.stdout:0/166: rmdir d9/da/d1b/d22 39 2024-03-29T08:07:14.081 INFO:tasks.workunit.client.0.smithi037.stdout:8/267: getdents d5/d10/d45 0 2024-03-29T08:07:14.081 INFO:tasks.workunit.client.0.smithi037.stdout:5/366: write d8/d22/d37/d60/d61/f69 [2649064,17420] 0 2024-03-29T08:07:14.083 INFO:tasks.workunit.client.0.smithi037.stdout:9/435: truncate d1/d4/d19/d1e/d4a/f62 154926 0 2024-03-29T08:07:14.084 INFO:tasks.workunit.client.0.smithi037.stdout:7/292: creat dd/d1d/d34/d54/f5f x:0 0 0 2024-03-29T08:07:14.084 INFO:tasks.workunit.client.1.smithi148.stdout:8/370: dread f7 [0,4194304] 0 2024-03-29T08:07:14.089 INFO:tasks.workunit.client.1.smithi148.stdout:7/345: write d8/f1f [194224,56571] 0 2024-03-29T08:07:14.092 INFO:tasks.workunit.client.1.smithi148.stdout:6/288: mknod d6/d9/d1e/d36/c4b 0 2024-03-29T08:07:14.094 INFO:tasks.workunit.client.1.smithi148.stdout:5/328: getdents d0/db/d1e/d5b 0 2024-03-29T08:07:14.096 INFO:tasks.workunit.client.0.smithi037.stdout:3/284: truncate fc 493075 0 2024-03-29T08:07:14.098 INFO:tasks.workunit.client.0.smithi037.stdout:1/328: dwrite d2/f31 [0,4194304] 0 2024-03-29T08:07:14.100 INFO:tasks.workunit.client.1.smithi148.stdout:3/314: rename de/d24/d33/d39/d46/d30/d37/d62/d63/c67 to de/d3b/c69 0 2024-03-29T08:07:14.100 INFO:tasks.workunit.client.0.smithi037.stdout:3/285: read - dd/d15/f49 zero size 2024-03-29T08:07:14.102 INFO:tasks.workunit.client.1.smithi148.stdout:9/299: creat db/f5c x:0 0 0 2024-03-29T08:07:14.105 INFO:tasks.workunit.client.1.smithi148.stdout:2/318: creat d7/da/d33/d45/d4d/d2e/d37/f67 x:0 0 0 2024-03-29T08:07:14.106 INFO:tasks.workunit.client.0.smithi037.stdout:6/212: symlink d0/d5/d1f/l47 0 2024-03-29T08:07:14.106 INFO:tasks.workunit.client.0.smithi037.stdout:4/227: creat d3/d4/d15/f50 x:0 0 0 2024-03-29T08:07:14.107 INFO:tasks.workunit.client.1.smithi148.stdout:2/319: write d7/da/d52/f63 [619087,20312] 0 2024-03-29T08:07:14.108 INFO:tasks.workunit.client.1.smithi148.stdout:7/346: dwrite d8/d14/d19/f63 [4194304,4194304] 0 2024-03-29T08:07:14.108 INFO:tasks.workunit.client.0.smithi037.stdout:3/286: chown dd/d15/f49 465 1 2024-03-29T08:07:14.109 INFO:tasks.workunit.client.1.smithi148.stdout:1/295: creat d3/de/d21/d32/d44/f53 x:0 0 0 2024-03-29T08:07:14.110 INFO:tasks.workunit.client.1.smithi148.stdout:2/320: write d7/da/f3f [124270,62630] 0 2024-03-29T08:07:14.111 INFO:tasks.workunit.client.0.smithi037.stdout:1/329: read - d2/d6/dc/d18/d1d/d21/f37 zero size 2024-03-29T08:07:14.112 INFO:tasks.workunit.client.1.smithi148.stdout:6/289: mknod d6/d9/d1e/d20/c4c 0 2024-03-29T08:07:14.115 INFO:tasks.workunit.client.0.smithi037.stdout:8/268: link d5/d10/l2b d5/d10/l58 0 2024-03-29T08:07:14.116 INFO:tasks.workunit.client.1.smithi148.stdout:3/315: rename de/l3c to de/d24/d33/d39/d46/l6a 0 2024-03-29T08:07:14.117 INFO:tasks.workunit.client.1.smithi148.stdout:7/347: mknod d8/d14/d19/c7d 0 2024-03-29T08:07:14.119 INFO:tasks.workunit.client.0.smithi037.stdout:1/330: fdatasync d2/d6/dc/d58/f71 0 2024-03-29T08:07:14.121 INFO:tasks.workunit.client.0.smithi037.stdout:8/269: fsync d5/f1e 0 2024-03-29T08:07:14.123 INFO:tasks.workunit.client.1.smithi148.stdout:6/290: mkdir d6/d9/d1e/d36/d4d 0 2024-03-29T08:07:14.123 INFO:tasks.workunit.client.0.smithi037.stdout:6/213: creat d0/f48 x:0 0 0 2024-03-29T08:07:14.124 INFO:tasks.workunit.client.1.smithi148.stdout:7/348: write d8/d14/f2d [1517795,113189] 0 2024-03-29T08:07:14.124 INFO:tasks.workunit.client.1.smithi148.stdout:3/316: mkdir de/d24/d33/d6b 0 2024-03-29T08:07:14.125 INFO:tasks.workunit.client.0.smithi037.stdout:0/167: dwrite d9/f27 [0,4194304] 0 2024-03-29T08:07:14.125 INFO:tasks.workunit.client.0.smithi037.stdout:7/293: dwrite dd/d1d/d34/f40 [0,4194304] 0 2024-03-29T08:07:14.128 INFO:tasks.workunit.client.0.smithi037.stdout:1/331: chown d2/d6/l1f 817 1 2024-03-29T08:07:14.130 INFO:tasks.workunit.client.1.smithi148.stdout:1/296: creat d3/de/d25/d51/f54 x:0 0 0 2024-03-29T08:07:14.131 INFO:tasks.workunit.client.0.smithi037.stdout:4/228: creat d3/d4/f51 x:0 0 0 2024-03-29T08:07:14.131 INFO:tasks.workunit.client.0.smithi037.stdout:6/214: readlink d0/l2 0 2024-03-29T08:07:14.133 INFO:tasks.workunit.client.1.smithi148.stdout:1/297: write d3/de/f1f [6316528,103748] 0 2024-03-29T08:07:14.134 INFO:tasks.workunit.client.1.smithi148.stdout:8/371: rename d17/c41 to d17/d74/d39/d6f/d64/c78 0 2024-03-29T08:07:14.137 INFO:tasks.workunit.client.0.smithi037.stdout:0/168: write d9/da/f25 [862663,107475] 0 2024-03-29T08:07:14.139 INFO:tasks.workunit.client.1.smithi148.stdout:5/329: getdents d0/db/de 0 2024-03-29T08:07:14.142 INFO:tasks.workunit.client.1.smithi148.stdout:9/300: creat db/d22/d26/f5d x:0 0 0 2024-03-29T08:07:14.143 INFO:tasks.workunit.client.1.smithi148.stdout:2/321: dwrite d7/da/fe [0,4194304] 0 2024-03-29T08:07:14.144 INFO:tasks.workunit.client.0.smithi037.stdout:1/332: fdatasync d2/d6/dc/d18/d1d/d21/f20 0 2024-03-29T08:07:14.144 INFO:tasks.workunit.client.0.smithi037.stdout:7/294: chown dd/d21/d35/d3b/c4a 756336 1 2024-03-29T08:07:14.144 INFO:tasks.workunit.client.0.smithi037.stdout:0/169: chown d9/da/dd/d13/c2f 1832 1 2024-03-29T08:07:14.144 INFO:tasks.workunit.client.0.smithi037.stdout:4/229: rename d3/d4/d15/d1d to d3/d4/d15/d1d/d52 22 2024-03-29T08:07:14.146 INFO:tasks.workunit.client.0.smithi037.stdout:6/215: mknod d0/d19/c49 0 2024-03-29T08:07:14.146 INFO:tasks.workunit.client.1.smithi148.stdout:8/372: creat d17/d1d/f79 x:0 0 0 2024-03-29T08:07:14.147 INFO:tasks.workunit.client.1.smithi148.stdout:7/349: write d8/f10 [1909215,76268] 0 2024-03-29T08:07:14.149 INFO:tasks.workunit.client.0.smithi037.stdout:7/295: write dd/d1d/d34/f39 [3438666,123419] 0 2024-03-29T08:07:14.149 INFO:tasks.workunit.client.0.smithi037.stdout:4/230: write d3/d4/f34 [4873238,48373] 0 2024-03-29T08:07:14.150 INFO:tasks.workunit.client.1.smithi148.stdout:5/330: mkdir d0/db/d1e/d5b/d66/d6c 0 2024-03-29T08:07:14.151 INFO:tasks.workunit.client.1.smithi148.stdout:0/311: write db/f51 [669729,77353] 0 2024-03-29T08:07:14.154 INFO:tasks.workunit.client.0.smithi037.stdout:1/333: chown d2/d6/dc/d5d/d64/f68 29 1 2024-03-29T08:07:14.155 INFO:tasks.workunit.client.1.smithi148.stdout:4/294: truncate df/d10/d17/f42 7263115 0 2024-03-29T08:07:14.155 INFO:tasks.workunit.client.1.smithi148.stdout:2/322: mkdir d7/d24/d4b/d68 0 2024-03-29T08:07:14.158 INFO:tasks.workunit.client.0.smithi037.stdout:4/231: write d3/d4/f2f [1149815,33038] 0 2024-03-29T08:07:14.160 INFO:tasks.workunit.client.1.smithi148.stdout:3/317: dwrite de/d3b/f60 [0,4194304] 0 2024-03-29T08:07:14.160 INFO:tasks.workunit.client.1.smithi148.stdout:7/350: unlink d8/d14/f1e 0 2024-03-29T08:07:14.161 INFO:tasks.workunit.client.1.smithi148.stdout:4/295: readlink lb 0 2024-03-29T08:07:14.164 INFO:tasks.workunit.client.1.smithi148.stdout:9/301: unlink db/d22/d26/d54/d4d/f3d 0 2024-03-29T08:07:14.165 INFO:tasks.workunit.client.0.smithi037.stdout:5/367: dread d8/f1f [0,4194304] 0 2024-03-29T08:07:14.166 INFO:tasks.workunit.client.1.smithi148.stdout:3/318: stat de/l45 0 2024-03-29T08:07:14.166 INFO:tasks.workunit.client.0.smithi037.stdout:1/334: creat d2/d6/dc/d18/d22/d4f/f73 x:0 0 0 2024-03-29T08:07:14.171 INFO:tasks.workunit.client.0.smithi037.stdout:7/296: mkdir dd/d21/d35/d60 0 2024-03-29T08:07:14.173 INFO:tasks.workunit.client.0.smithi037.stdout:0/170: getdents d9/da/dd 0 2024-03-29T08:07:14.173 INFO:tasks.workunit.client.0.smithi037.stdout:1/335: stat d2/d6/dc/d18/f3e 0 2024-03-29T08:07:14.176 INFO:tasks.workunit.client.0.smithi037.stdout:7/297: dread - dd/d21/d35/d42/f43 zero size 2024-03-29T08:07:14.176 INFO:tasks.workunit.client.1.smithi148.stdout:1/298: dwrite d3/f30 [0,4194304] 0 2024-03-29T08:07:14.176 INFO:tasks.workunit.client.1.smithi148.stdout:4/296: read df/f4d [2858749,87896] 0 2024-03-29T08:07:14.178 INFO:tasks.workunit.client.0.smithi037.stdout:6/216: getdents d0/d19/d34 0 2024-03-29T08:07:14.179 INFO:tasks.workunit.client.1.smithi148.stdout:2/323: write d7/da/d33/f3e [2749798,24392] 0 2024-03-29T08:07:14.179 INFO:tasks.workunit.client.1.smithi148.stdout:0/312: symlink db/de/df/d11/d42/l65 0 2024-03-29T08:07:14.180 INFO:tasks.workunit.client.1.smithi148.stdout:9/302: creat db/d22/d26/d54/d4d/d35/f5e x:0 0 0 2024-03-29T08:07:14.182 INFO:tasks.workunit.client.1.smithi148.stdout:5/331: creat d0/db/d1e/d35/d6a/f6d x:0 0 0 2024-03-29T08:07:14.183 INFO:tasks.workunit.client.1.smithi148.stdout:2/324: read - d7/da/d33/f5b zero size 2024-03-29T08:07:14.183 INFO:tasks.workunit.client.0.smithi037.stdout:4/232: mknod d3/d4/d15/d1d/db/d3d/c53 0 2024-03-29T08:07:14.184 INFO:tasks.workunit.client.1.smithi148.stdout:0/313: dread - db/de/f62 zero size 2024-03-29T08:07:14.185 INFO:tasks.workunit.client.1.smithi148.stdout:5/332: truncate d0/f12 5050980 0 2024-03-29T08:07:14.185 INFO:tasks.workunit.client.1.smithi148.stdout:9/303: write db/d22/d26/d54/d4d/f5b [399513,89156] 0 2024-03-29T08:07:14.189 INFO:tasks.workunit.client.1.smithi148.stdout:8/373: creat d17/d18/d26/d63/d66/f7a x:0 0 0 2024-03-29T08:07:14.193 INFO:tasks.workunit.client.1.smithi148.stdout:4/297: symlink df/d10/d26/d34/d3e/l66 0 2024-03-29T08:07:14.193 INFO:tasks.workunit.client.1.smithi148.stdout:8/374: dread - d17/d74/d39/d6f/f55 zero size 2024-03-29T08:07:14.193 INFO:tasks.workunit.client.1.smithi148.stdout:8/375: readlink d17/d74/l53 0 2024-03-29T08:07:14.196 INFO:tasks.workunit.client.1.smithi148.stdout:1/299: rename d3/de/d21/d32 to d3/d55 0 2024-03-29T08:07:14.196 INFO:tasks.workunit.client.0.smithi037.stdout:1/336: mkdir d2/d6/dc/d58/d67/d74 0 2024-03-29T08:07:14.199 INFO:tasks.workunit.client.1.smithi148.stdout:3/319: mknod de/d24/d33/d6b/c6c 0 2024-03-29T08:07:14.202 INFO:tasks.workunit.client.0.smithi037.stdout:7/298: getdents dd/d21 0 2024-03-29T08:07:14.203 INFO:tasks.workunit.client.1.smithi148.stdout:4/298: creat df/d2c/d51/f67 x:0 0 0 2024-03-29T08:07:14.205 INFO:tasks.workunit.client.0.smithi037.stdout:1/337: write d2/d6/dc/f4a [883005,96055] 0 2024-03-29T08:07:14.205 INFO:tasks.workunit.client.0.smithi037.stdout:6/217: dwrite d0/d5/d1f/f3c [0,4194304] 0 2024-03-29T08:07:14.208 INFO:tasks.workunit.client.0.smithi037.stdout:5/368: dwrite d8/f52 [0,4194304] 0 2024-03-29T08:07:14.209 INFO:tasks.workunit.client.1.smithi148.stdout:5/333: dwrite d0/db/d1e/f31 [0,4194304] 0 2024-03-29T08:07:14.210 INFO:tasks.workunit.client.1.smithi148.stdout:2/325: rename d7/d24/f34 to d7/da/d4c/f69 0 2024-03-29T08:07:14.212 INFO:tasks.workunit.client.1.smithi148.stdout:7/351: dread d8/d14/f55 [0,4194304] 0 2024-03-29T08:07:14.212 INFO:tasks.workunit.client.0.smithi037.stdout:0/171: truncate d9/da/f11 3417998 0 2024-03-29T08:07:14.214 INFO:tasks.workunit.client.0.smithi037.stdout:1/338: stat d2/d6/dc/d18/d1d/d21/f37 0 2024-03-29T08:07:14.215 INFO:tasks.workunit.client.1.smithi148.stdout:9/304: mkdir db/d10/d2d/d4e/d5f 0 2024-03-29T08:07:14.217 INFO:tasks.workunit.client.1.smithi148.stdout:2/326: write d7/da/d33/d45/d4d/d2e/d37/f67 [592515,54467] 0 2024-03-29T08:07:14.218 INFO:tasks.workunit.client.1.smithi148.stdout:3/320: mknod de/d24/d33/c6d 0 2024-03-29T08:07:14.219 INFO:tasks.workunit.client.0.smithi037.stdout:6/218: symlink d0/l4a 0 2024-03-29T08:07:14.220 INFO:tasks.workunit.client.0.smithi037.stdout:4/233: dwrite d3/d4/d15/f49 [0,4194304] 0 2024-03-29T08:07:14.223 INFO:tasks.workunit.client.0.smithi037.stdout:6/219: chown d0/d5/d3f/c3 12131 1 2024-03-29T08:07:14.225 INFO:tasks.workunit.client.1.smithi148.stdout:0/314: rename db/f18 to db/de/df/d2b/d3b/d47/f66 0 2024-03-29T08:07:14.228 INFO:tasks.workunit.client.1.smithi148.stdout:9/305: creat db/d22/d26/d54/f60 x:0 0 0 2024-03-29T08:07:14.229 INFO:tasks.workunit.client.1.smithi148.stdout:3/321: rename de/d24/d33 to de/d24/d33/d39/d6e 22 2024-03-29T08:07:14.230 INFO:tasks.workunit.client.0.smithi037.stdout:5/369: link d8/l1b d8/d34/l77 0 2024-03-29T08:07:14.230 INFO:tasks.workunit.client.1.smithi148.stdout:4/299: mknod df/d2d/c68 0 2024-03-29T08:07:14.231 INFO:tasks.workunit.client.0.smithi037.stdout:7/299: dwrite dd/d1d/d34/f39 [4194304,4194304] 0 2024-03-29T08:07:14.235 INFO:tasks.workunit.client.0.smithi037.stdout:4/234: mkdir d3/d54 0 2024-03-29T08:07:14.236 INFO:tasks.workunit.client.0.smithi037.stdout:7/300: truncate dd/d21/d35/f50 86969 0 2024-03-29T08:07:14.237 INFO:tasks.workunit.client.0.smithi037.stdout:6/220: mknod d0/c4b 0 2024-03-29T08:07:14.238 INFO:tasks.workunit.client.0.smithi037.stdout:5/370: rename d8/d22/l2f to d8/d34/l78 0 2024-03-29T08:07:14.240 INFO:tasks.workunit.client.1.smithi148.stdout:4/300: chown df/d10/d17/l19 3208 1 2024-03-29T08:07:14.240 INFO:tasks.workunit.client.1.smithi148.stdout:3/322: chown de/d24/d33/d39/d46/c19 493 1 2024-03-29T08:07:14.241 INFO:tasks.workunit.client.0.smithi037.stdout:6/221: dread - d0/d5/d1f/f30 zero size 2024-03-29T08:07:14.243 INFO:tasks.workunit.client.1.smithi148.stdout:3/323: chown de/lf 266023 1 2024-03-29T08:07:14.243 INFO:tasks.workunit.client.1.smithi148.stdout:4/301: truncate f3 635390 0 2024-03-29T08:07:14.243 INFO:tasks.workunit.client.1.smithi148.stdout:8/376: dwrite d17/d74/d39/d6f/f43 [0,4194304] 0 2024-03-29T08:07:14.244 INFO:tasks.workunit.client.1.smithi148.stdout:2/327: dread d7/da/d33/d3c/f23 [4194304,4194304] 0 2024-03-29T08:07:14.246 INFO:tasks.workunit.client.0.smithi037.stdout:5/371: mkdir d8/d22/d4f/d70/d79 0 2024-03-29T08:07:14.246 INFO:tasks.workunit.client.1.smithi148.stdout:3/324: write de/f2c [776044,45829] 0 2024-03-29T08:07:14.248 INFO:tasks.workunit.client.1.smithi148.stdout:4/302: symlink df/d10/d17/l69 0 2024-03-29T08:07:14.250 INFO:tasks.workunit.client.0.smithi037.stdout:1/339: dwrite d2/d6/dc/f1e [0,4194304] 0 2024-03-29T08:07:14.251 INFO:tasks.workunit.client.1.smithi148.stdout:4/303: dread - df/d10/f53 zero size 2024-03-29T08:07:14.256 INFO:tasks.workunit.client.0.smithi037.stdout:7/301: rename dd/d21/l2d to dd/d21/d35/d60/l61 0 2024-03-29T08:07:14.257 INFO:tasks.workunit.client.1.smithi148.stdout:5/334: dread d0/f9 [0,4194304] 0 2024-03-29T08:07:14.261 INFO:tasks.workunit.client.0.smithi037.stdout:7/302: fdatasync dd/f16 0 2024-03-29T08:07:14.261 INFO:tasks.workunit.client.1.smithi148.stdout:0/315: dwrite db/de/df/d2b/d3b/d47/f2c [0,4194304] 0 2024-03-29T08:07:14.266 INFO:tasks.workunit.client.1.smithi148.stdout:5/335: mkdir d0/db/d1e/d6e 0 2024-03-29T08:07:14.267 INFO:tasks.workunit.client.0.smithi037.stdout:5/372: rename d8/d5b to d8/d22/d4f/d70/d79/d7a 0 2024-03-29T08:07:14.269 INFO:tasks.workunit.client.1.smithi148.stdout:5/336: chown d0/db/d1e/d36/d41 62 1 2024-03-29T08:07:14.270 INFO:tasks.workunit.client.1.smithi148.stdout:3/325: link de/d24/d33/d39/d46/f5f de/d24/d33/d39/d46/d30/d37/f6f 0 2024-03-29T08:07:14.272 INFO:tasks.workunit.client.1.smithi148.stdout:5/337: stat d0/db/d1e/f37 0 2024-03-29T08:07:14.272 INFO:tasks.workunit.client.0.smithi037.stdout:7/303: fdatasync dd/f16 0 2024-03-29T08:07:14.275 INFO:tasks.workunit.client.1.smithi148.stdout:3/326: chown de/d24/d33/d39/d46/d30/d37/l53 1509317 1 2024-03-29T08:07:14.275 INFO:tasks.workunit.client.1.smithi148.stdout:2/328: dwrite d7/da/d33/d45/d4d/d2e/d37/f67 [0,4194304] 0 2024-03-29T08:07:14.278 INFO:tasks.workunit.client.1.smithi148.stdout:1/300: write d3/d55/f37 [456854,101963] 0 2024-03-29T08:07:14.279 INFO:tasks.workunit.client.0.smithi037.stdout:4/235: dread d3/d4/d15/d1d/db/d45/d1e/f4e [0,4194304] 0 2024-03-29T08:07:14.282 INFO:tasks.workunit.client.0.smithi037.stdout:7/304: getdents dd/d1d 0 2024-03-29T08:07:14.287 INFO:tasks.workunit.client.0.smithi037.stdout:7/305: mknod dd/d1d/d34/d54/c62 0 2024-03-29T08:07:14.289 INFO:tasks.workunit.client.1.smithi148.stdout:0/316: unlink db/de/df/l43 0 2024-03-29T08:07:14.289 INFO:tasks.workunit.client.1.smithi148.stdout:5/338: rename d0 to d0/db/d6f 22 2024-03-29T08:07:14.291 INFO:tasks.workunit.client.1.smithi148.stdout:0/317: fdatasync f6 0 2024-03-29T08:07:14.293 INFO:tasks.workunit.client.1.smithi148.stdout:2/329: creat d7/da/d33/d45/d4d/d41/f6a x:0 0 0 2024-03-29T08:07:14.298 INFO:tasks.workunit.client.0.smithi037.stdout:5/373: dwrite d8/d22/d4f/d70/f3f [4194304,4194304] 0 2024-03-29T08:07:14.299 INFO:tasks.workunit.client.1.smithi148.stdout:7/352: sync 2024-03-29T08:07:14.299 INFO:tasks.workunit.client.1.smithi148.stdout:6/291: sync 2024-03-29T08:07:14.299 INFO:tasks.workunit.client.1.smithi148.stdout:0/318: readlink db/de/df/d2b/d3b/d47/d4c/d5a/l60 0 2024-03-29T08:07:14.300 INFO:tasks.workunit.client.1.smithi148.stdout:1/301: dwrite d3/d55/f37 [0,4194304] 0 2024-03-29T08:07:14.302 INFO:tasks.workunit.client.0.smithi037.stdout:5/374: fsync d8/d22/d37/d60/f76 0 2024-03-29T08:07:14.302 INFO:tasks.workunit.client.0.smithi037.stdout:7/306: rename dd/d21/d35/d3b/c4a to dd/d21/d35/d3b/c63 0 2024-03-29T08:07:14.303 INFO:tasks.workunit.client.1.smithi148.stdout:4/304: truncate f3 268387 0 2024-03-29T08:07:14.305 INFO:tasks.workunit.client.1.smithi148.stdout:2/330: write d7/f1b [1262414,102928] 0 2024-03-29T08:07:14.306 INFO:tasks.workunit.client.1.smithi148.stdout:9/306: dwrite f3 [0,4194304] 0 2024-03-29T08:07:14.306 INFO:tasks.workunit.client.0.smithi037.stdout:4/236: dwrite d3/d4/d15/f1f [0,4194304] 0 2024-03-29T08:07:14.309 INFO:tasks.workunit.client.1.smithi148.stdout:0/319: fsync db/de/df/d11/d2d/d3e/f45 0 2024-03-29T08:07:14.310 INFO:tasks.workunit.client.0.smithi037.stdout:1/340: dread d2/d6/dc/d18/d1d/d21/f2a [0,4194304] 0 2024-03-29T08:07:14.310 INFO:tasks.workunit.client.1.smithi148.stdout:5/339: creat d0/db/d1e/d35/d42/d67/f70 x:0 0 0 2024-03-29T08:07:14.317 INFO:tasks.workunit.client.1.smithi148.stdout:4/305: mkdir df/d10/d17/d6a 0 2024-03-29T08:07:14.318 INFO:tasks.workunit.client.0.smithi037.stdout:7/307: rename dd/d21/d35/d3b/l4e to dd/d21/d35/d42/d5c/l64 0 2024-03-29T08:07:14.318 INFO:tasks.workunit.client.0.smithi037.stdout:5/375: link d8/f17 d8/d22/d4f/d70/d79/f7b 0 2024-03-29T08:07:14.318 INFO:tasks.workunit.client.1.smithi148.stdout:3/327: dwrite de/d3b/d49/f68 [0,4194304] 0 2024-03-29T08:07:14.319 INFO:tasks.workunit.client.0.smithi037.stdout:5/376: stat d8/d22/l32 0 2024-03-29T08:07:14.321 INFO:tasks.workunit.client.0.smithi037.stdout:1/341: chown d2/d40/c45 15 1 2024-03-29T08:07:14.322 INFO:tasks.workunit.client.1.smithi148.stdout:5/340: write d0/db/de/f50 [2093122,108266] 0 2024-03-29T08:07:14.323 INFO:tasks.workunit.client.1.smithi148.stdout:2/331: write d7/da/d33/d45/d4d/f29 [1643,122487] 0 2024-03-29T08:07:14.324 INFO:tasks.workunit.client.1.smithi148.stdout:4/306: write fd [3837412,69360] 0 2024-03-29T08:07:14.325 INFO:tasks.workunit.client.1.smithi148.stdout:9/307: read db/d22/d26/d54/d4d/f18 [2958445,115964] 0 2024-03-29T08:07:14.326 INFO:tasks.workunit.client.0.smithi037.stdout:1/342: fdatasync d2/d6/dc/d58/f71 0 2024-03-29T08:07:14.327 INFO:tasks.workunit.client.1.smithi148.stdout:7/353: readlink d8/d14/d19/d37/l61 0 2024-03-29T08:07:14.331 INFO:tasks.workunit.client.1.smithi148.stdout:2/332: readlink d7/da/l3a 0 2024-03-29T08:07:14.331 INFO:tasks.workunit.client.0.smithi037.stdout:4/237: getdents d3/d4/d15/d1d/db/d45/d1e 0 2024-03-29T08:07:14.334 INFO:tasks.workunit.client.1.smithi148.stdout:3/328: mkdir de/d24/d33/d39/d46/d70 0 2024-03-29T08:07:14.334 INFO:tasks.workunit.client.1.smithi148.stdout:6/292: creat d6/d9/d1e/d22/f4e x:0 0 0 2024-03-29T08:07:14.334 INFO:tasks.workunit.client.1.smithi148.stdout:5/341: read - d0/db/d1e/f4a zero size 2024-03-29T08:07:14.336 INFO:tasks.workunit.client.1.smithi148.stdout:7/354: write d8/f1f [2782246,64458] 0 2024-03-29T08:07:14.336 INFO:tasks.workunit.client.1.smithi148.stdout:0/320: rmdir db/de/d40 0 2024-03-29T08:07:14.337 INFO:tasks.workunit.client.0.smithi037.stdout:4/238: chown d3/d4/d15/d1d/db 59621 1 2024-03-29T08:07:14.337 INFO:tasks.workunit.client.0.smithi037.stdout:1/343: mknod d2/d6/dc/d5d/d64/c75 0 2024-03-29T08:07:14.337 INFO:tasks.workunit.client.0.smithi037.stdout:5/377: link d8/d34/l58 d8/d22/d37/d60/d61/l7c 0 2024-03-29T08:07:14.340 INFO:tasks.workunit.client.1.smithi148.stdout:8/377: sync 2024-03-29T08:07:14.341 INFO:tasks.workunit.client.1.smithi148.stdout:6/293: chown d6/d9/d1e/d20/f39 133936331 1 2024-03-29T08:07:14.341 INFO:tasks.workunit.client.1.smithi148.stdout:4/307: symlink df/d10/d26/d34/d3e/l6b 0 2024-03-29T08:07:14.343 INFO:tasks.workunit.client.1.smithi148.stdout:1/302: dwrite d3/f30 [0,4194304] 0 2024-03-29T08:07:14.343 INFO:tasks.workunit.client.1.smithi148.stdout:3/329: mkdir de/d24/d33/d39/d71 0 2024-03-29T08:07:14.343 INFO:tasks.workunit.client.1.smithi148.stdout:8/378: stat d17/d18/l56 0 2024-03-29T08:07:14.345 INFO:tasks.workunit.client.0.smithi037.stdout:7/308: dread dd/d21/d35/f4c [0,4194304] 0 2024-03-29T08:07:14.347 INFO:tasks.workunit.client.1.smithi148.stdout:5/342: unlink d0/db/de/l1c 0 2024-03-29T08:07:14.347 INFO:tasks.workunit.client.1.smithi148.stdout:7/355: chown d8/f15 41400 1 2024-03-29T08:07:14.349 INFO:tasks.workunit.client.1.smithi148.stdout:5/343: chown d0/db/d1e/d6e 222099 1 2024-03-29T08:07:14.349 INFO:tasks.workunit.client.0.smithi037.stdout:5/378: mknod d8/d22/d4f/d70/d79/d7a/c7d 0 2024-03-29T08:07:14.350 INFO:tasks.workunit.client.0.smithi037.stdout:5/379: stat d8/d18/d24/f74 0 2024-03-29T08:07:14.351 INFO:tasks.workunit.client.1.smithi148.stdout:9/308: rename db/d10/d2d/d32 to db/d22/d3f/d61 0 2024-03-29T08:07:14.352 INFO:tasks.workunit.client.1.smithi148.stdout:1/303: truncate d3/de/d25/d35/f3c 5241700 0 2024-03-29T08:07:14.353 INFO:tasks.workunit.client.1.smithi148.stdout:9/309: chown db/f5c 127 1 2024-03-29T08:07:14.356 INFO:tasks.workunit.client.0.smithi037.stdout:1/344: symlink d2/d6/dc/d18/d1d/d5e/d72/l76 0 2024-03-29T08:07:14.358 INFO:tasks.workunit.client.1.smithi148.stdout:6/294: mkdir d6/d9/d1e/d20/d4f 0 2024-03-29T08:07:14.359 INFO:tasks.workunit.client.0.smithi037.stdout:7/309: fsync dd/f1f 0 2024-03-29T08:07:14.361 INFO:tasks.workunit.client.0.smithi037.stdout:1/345: dread - d2/d6/dc/d18/f29 zero size 2024-03-29T08:07:14.363 INFO:tasks.workunit.client.0.smithi037.stdout:7/310: chown dd/l47 30921 1 2024-03-29T08:07:14.364 INFO:tasks.workunit.client.0.smithi037.stdout:5/380: link d8/d4d/c6d d8/d22/d4f/d70/d79/c7e 0 2024-03-29T08:07:14.366 INFO:tasks.workunit.client.1.smithi148.stdout:9/310: mkdir db/d10/d2d/d62 0 2024-03-29T08:07:14.367 INFO:tasks.workunit.client.0.smithi037.stdout:4/239: dwrite d3/d4/d15/d1d/db/d45/d1e/f4e [0,4194304] 0 2024-03-29T08:07:14.368 INFO:tasks.workunit.client.0.smithi037.stdout:1/346: creat d2/d6/d13/f77 x:0 0 0 2024-03-29T08:07:14.369 INFO:tasks.workunit.client.0.smithi037.stdout:4/240: readlink d3/d4/d15/l4d 0 2024-03-29T08:07:14.369 INFO:tasks.workunit.client.1.smithi148.stdout:0/321: dread db/de/df/d11/f16 [0,4194304] 0 2024-03-29T08:07:14.370 INFO:tasks.workunit.client.1.smithi148.stdout:3/330: creat de/d24/d33/d39/d71/f72 x:0 0 0 2024-03-29T08:07:14.372 INFO:tasks.workunit.client.1.smithi148.stdout:0/322: fsync db/de/df/d2b/f53 0 2024-03-29T08:07:14.373 INFO:tasks.workunit.client.1.smithi148.stdout:1/304: mknod d3/de/d1d/d47/c56 0 2024-03-29T08:07:14.374 INFO:tasks.workunit.client.1.smithi148.stdout:3/331: write de/d24/d33/d39/d46/f23 [2090951,37594] 0 2024-03-29T08:07:14.375 INFO:tasks.workunit.client.0.smithi037.stdout:5/381: rename d8/c11 to d8/d22/d37/d60/c7f 0 2024-03-29T08:07:14.377 INFO:tasks.workunit.client.1.smithi148.stdout:0/323: stat db/de/df/d2b/f35 0 2024-03-29T08:07:14.377 INFO:tasks.workunit.client.1.smithi148.stdout:2/333: dread d7/d24/d4b/f55 [0,4194304] 0 2024-03-29T08:07:14.378 INFO:tasks.workunit.client.1.smithi148.stdout:6/295: unlink d6/d9/l25 0 2024-03-29T08:07:14.379 INFO:tasks.workunit.client.0.smithi037.stdout:5/382: creat d8/d34/f80 x:0 0 0 2024-03-29T08:07:14.380 INFO:tasks.workunit.client.1.smithi148.stdout:0/324: chown db/de/f62 7178188 1 2024-03-29T08:07:14.383 INFO:tasks.workunit.client.0.smithi037.stdout:5/383: write d8/d18/f50 [2385941,29561] 0 2024-03-29T08:07:14.383 INFO:tasks.workunit.client.1.smithi148.stdout:9/311: symlink db/d10/d2d/d62/l63 0 2024-03-29T08:07:14.384 INFO:tasks.workunit.client.0.smithi037.stdout:5/384: chown d8/l5d 7939 1 2024-03-29T08:07:14.387 INFO:tasks.workunit.client.1.smithi148.stdout:4/308: rename f3 to df/d2c/f6c 0 2024-03-29T08:07:14.388 INFO:tasks.workunit.client.1.smithi148.stdout:6/296: fdatasync d6/d9/f2c 0 2024-03-29T08:07:14.389 INFO:tasks.workunit.client.1.smithi148.stdout:4/309: fsync df/d10/d26/f63 0 2024-03-29T08:07:14.391 INFO:tasks.workunit.client.1.smithi148.stdout:9/312: rmdir db/d10/d2d/d4e 39 2024-03-29T08:07:14.395 INFO:tasks.workunit.client.1.smithi148.stdout:4/310: unlink df/l16 0 2024-03-29T08:07:14.397 INFO:tasks.workunit.client.0.smithi037.stdout:7/311: dwrite dd/d21/d35/f4c [0,4194304] 0 2024-03-29T08:07:14.398 INFO:tasks.workunit.client.1.smithi148.stdout:1/305: creat d3/de/f57 x:0 0 0 2024-03-29T08:07:14.401 INFO:tasks.workunit.client.1.smithi148.stdout:6/297: mknod d6/d9/d1e/d22/c50 0 2024-03-29T08:07:14.402 INFO:tasks.workunit.client.0.smithi037.stdout:7/312: chown f9 1775 1 2024-03-29T08:07:14.402 INFO:tasks.workunit.client.1.smithi148.stdout:2/334: dwrite d7/da/d33/d3c/f48 [0,4194304] 0 2024-03-29T08:07:14.403 INFO:tasks.workunit.client.1.smithi148.stdout:9/313: rmdir db/d22/d26/d54/d52 39 2024-03-29T08:07:14.405 INFO:tasks.workunit.client.1.smithi148.stdout:8/379: rename d17/d18/l56 to d17/d18/d26/l7b 0 2024-03-29T08:07:14.405 INFO:tasks.workunit.client.1.smithi148.stdout:0/325: dwrite f7 [4194304,4194304] 0 2024-03-29T08:07:14.405 INFO:tasks.workunit.client.1.smithi148.stdout:3/332: getdents de/d3b/d49 0 2024-03-29T08:07:14.408 INFO:tasks.workunit.client.0.smithi037.stdout:5/385: dwrite d8/d18/f21 [0,4194304] 0 2024-03-29T08:07:14.410 INFO:tasks.workunit.client.0.smithi037.stdout:4/241: dread d3/d4/f35 [0,4194304] 0 2024-03-29T08:07:14.414 INFO:tasks.workunit.client.0.smithi037.stdout:5/386: mknod d8/d18/d24/c81 0 2024-03-29T08:07:14.414 INFO:tasks.workunit.client.1.smithi148.stdout:1/306: symlink d3/de/d25/d35/d42/d23/l58 0 2024-03-29T08:07:14.414 INFO:tasks.workunit.client.1.smithi148.stdout:0/326: chown l5 19977918 1 2024-03-29T08:07:14.420 INFO:tasks.workunit.client.1.smithi148.stdout:7/356: rename d8/d14/d19/d37/f51 to d8/d38/d3c/f7e 0 2024-03-29T08:07:14.421 INFO:tasks.workunit.client.0.smithi037.stdout:5/387: creat d8/d22/d4f/d70/f82 x:0 0 0 2024-03-29T08:07:14.422 INFO:tasks.workunit.client.1.smithi148.stdout:8/380: mknod d17/d18/d26/d63/c7c 0 2024-03-29T08:07:14.422 INFO:tasks.workunit.client.1.smithi148.stdout:1/307: fdatasync d3/d55/d44/f53 0 2024-03-29T08:07:14.424 INFO:tasks.workunit.client.1.smithi148.stdout:6/298: fdatasync d6/d9/d1e/d20/f39 0 2024-03-29T08:07:14.426 INFO:tasks.workunit.client.1.smithi148.stdout:4/311: dwrite df/f14 [0,4194304] 0 2024-03-29T08:07:14.426 INFO:tasks.workunit.client.0.smithi037.stdout:4/242: link d3/d4/d15/d1d/db/d3d/c53 d3/d4/d15/d1d/d4f/c55 0 2024-03-29T08:07:14.426 INFO:tasks.workunit.client.0.smithi037.stdout:5/388: truncate d8/d18/f43 616475 0 2024-03-29T08:07:14.428 INFO:tasks.workunit.client.0.smithi037.stdout:4/243: stat d3/c6 0 2024-03-29T08:07:14.431 INFO:tasks.workunit.client.1.smithi148.stdout:2/335: creat d7/da/d33/d45/f6b x:0 0 0 2024-03-29T08:07:14.436 INFO:tasks.workunit.client.1.smithi148.stdout:5/344: dwrite d0/db/de/d16/d2e/f39 [0,4194304] 0 2024-03-29T08:07:14.436 INFO:tasks.workunit.client.1.smithi148.stdout:7/357: write d8/d29/d44/d49/f62 [721445,99164] 0 2024-03-29T08:07:14.437 INFO:tasks.workunit.client.1.smithi148.stdout:4/312: stat df/d10/d26/f63 0 2024-03-29T08:07:14.438 INFO:tasks.workunit.client.0.smithi037.stdout:5/389: read d8/d18/f28 [2410793,47875] 0 2024-03-29T08:07:14.442 INFO:tasks.workunit.client.1.smithi148.stdout:3/333: rename de/c2a to de/c73 0 2024-03-29T08:07:14.444 INFO:tasks.workunit.client.0.smithi037.stdout:7/313: dread dd/f14 [0,4194304] 0 2024-03-29T08:07:14.445 INFO:tasks.workunit.client.1.smithi148.stdout:8/381: mkdir d17/d18/d26/d63/d7d 0 2024-03-29T08:07:14.446 INFO:tasks.workunit.client.1.smithi148.stdout:4/313: chown df/d10/d17/f4f 13823335 1 2024-03-29T08:07:14.448 INFO:tasks.workunit.client.1.smithi148.stdout:9/314: getdents db/d22/d26/d54/d4d/d35/d4b/d58 0 2024-03-29T08:07:14.451 INFO:tasks.workunit.client.0.smithi037.stdout:7/314: mkdir dd/d21/d65 0 2024-03-29T08:07:14.452 INFO:tasks.workunit.client.1.smithi148.stdout:3/334: unlink de/d24/f42 0 2024-03-29T08:07:14.454 INFO:tasks.workunit.client.1.smithi148.stdout:5/345: mknod d0/db/d1e/d35/d42/d67/c71 0 2024-03-29T08:07:14.454 INFO:tasks.workunit.client.1.smithi148.stdout:7/358: symlink d8/d14/d19/d37/l7f 0 2024-03-29T08:07:14.456 INFO:tasks.workunit.client.0.smithi037.stdout:7/315: getdents dd/d1d/d49 0 2024-03-29T08:07:14.457 INFO:tasks.workunit.client.1.smithi148.stdout:2/336: creat d7/da/d33/d45/d4d/d2e/d37/f6c x:0 0 0 2024-03-29T08:07:14.457 INFO:tasks.workunit.client.0.smithi037.stdout:7/316: chown dd/d21/c31 7022 1 2024-03-29T08:07:14.459 INFO:tasks.workunit.client.1.smithi148.stdout:8/382: truncate d17/d18/f1c 263580 0 2024-03-29T08:07:14.459 INFO:tasks.workunit.client.1.smithi148.stdout:3/335: write de/d24/d33/d39/d71/f72 [948176,22189] 0 2024-03-29T08:07:14.460 INFO:tasks.workunit.client.0.smithi037.stdout:7/317: write dd/f17 [207536,21551] 0 2024-03-29T08:07:14.463 INFO:tasks.workunit.client.1.smithi148.stdout:6/299: fdatasync d6/f14 0 2024-03-29T08:07:14.464 INFO:tasks.workunit.client.0.smithi037.stdout:4/244: write d3/d4/d15/d1d/db/d45/f30 [42367,84552] 0 2024-03-29T08:07:14.465 INFO:tasks.workunit.client.1.smithi148.stdout:5/346: mkdir d0/db/d1e/d5b/d66/d72 0 2024-03-29T08:07:14.466 INFO:tasks.workunit.client.1.smithi148.stdout:1/308: dwrite d3/d55/d44/f53 [0,4194304] 0 2024-03-29T08:07:14.467 INFO:tasks.workunit.client.0.smithi037.stdout:7/318: link dd/d1d/d34/f37 dd/d1d/d34/f66 0 2024-03-29T08:07:14.469 INFO:tasks.workunit.client.0.smithi037.stdout:4/245: creat d3/d4/d15/d1d/db/d3d/f56 x:0 0 0 2024-03-29T08:07:14.470 INFO:tasks.workunit.client.1.smithi148.stdout:1/309: write d3/d55/f52 [451215,113288] 0 2024-03-29T08:07:14.470 INFO:tasks.workunit.client.1.smithi148.stdout:8/383: mkdir d17/d1d/d7e 0 2024-03-29T08:07:14.471 INFO:tasks.workunit.client.0.smithi037.stdout:4/246: chown d3/d4/d15/d1d 434338 1 2024-03-29T08:07:14.472 INFO:tasks.workunit.client.1.smithi148.stdout:3/336: creat de/d3b/f74 x:0 0 0 2024-03-29T08:07:14.473 INFO:tasks.workunit.client.1.smithi148.stdout:6/300: readlink d6/l40 0 2024-03-29T08:07:14.474 INFO:tasks.workunit.client.1.smithi148.stdout:9/315: rename db/d20/c1c to db/d10/d2d/d62/c64 0 2024-03-29T08:07:14.475 INFO:tasks.workunit.client.0.smithi037.stdout:7/319: creat dd/f67 x:0 0 0 2024-03-29T08:07:14.475 INFO:tasks.workunit.client.1.smithi148.stdout:5/347: chown d0/db/de/c2d 0 1 2024-03-29T08:07:14.476 INFO:tasks.workunit.client.1.smithi148.stdout:8/384: symlink d17/d49/l7f 0 2024-03-29T08:07:14.477 INFO:tasks.workunit.client.0.smithi037.stdout:4/247: mkdir d3/d57 0 2024-03-29T08:07:14.477 INFO:tasks.workunit.client.1.smithi148.stdout:1/310: chown d3/de/d25/d35/c41 60267 1 2024-03-29T08:07:14.478 INFO:tasks.workunit.client.0.smithi037.stdout:7/320: write dd/d1d/d34/f39 [7657024,92317] 0 2024-03-29T08:07:14.478 INFO:tasks.workunit.client.0.smithi037.stdout:4/248: chown d3/c6 1032 1 2024-03-29T08:07:14.481 INFO:tasks.workunit.client.0.smithi037.stdout:4/249: chown d3/d4/c48 373488030 1 2024-03-29T08:07:14.481 INFO:tasks.workunit.client.1.smithi148.stdout:8/385: readlink d17/d1d/l42 0 2024-03-29T08:07:14.483 INFO:tasks.workunit.client.1.smithi148.stdout:6/301: mkdir d6/d9/d1e/d20/d30/d51 0 2024-03-29T08:07:14.483 INFO:tasks.workunit.client.0.smithi037.stdout:4/250: truncate d3/f44 344386 0 2024-03-29T08:07:14.492 INFO:tasks.workunit.client.1.smithi148.stdout:2/337: dwrite d7/da/d33/d45/d4d/f57 [0,4194304] 0 2024-03-29T08:07:14.493 INFO:tasks.workunit.client.1.smithi148.stdout:8/386: rmdir d17 39 2024-03-29T08:07:14.500 INFO:tasks.workunit.client.1.smithi148.stdout:9/316: rename db/d22/d26/d54/d4d/f5b to db/d10/d2d/d4e/d5f/f65 0 2024-03-29T08:07:14.500 INFO:tasks.workunit.client.1.smithi148.stdout:0/327: sync 2024-03-29T08:07:14.500 INFO:tasks.workunit.client.1.smithi148.stdout:4/314: sync 2024-03-29T08:07:14.502 INFO:tasks.workunit.client.1.smithi148.stdout:7/359: dwrite d8/fe [4194304,4194304] 0 2024-03-29T08:07:14.502 INFO:tasks.workunit.client.0.smithi037.stdout:4/251: dwrite d3/d4/d15/d1d/db/d45/d1e/f4e [0,4194304] 0 2024-03-29T08:07:14.506 INFO:tasks.workunit.client.1.smithi148.stdout:6/302: creat d6/d9/f52 x:0 0 0 2024-03-29T08:07:14.510 INFO:tasks.workunit.client.1.smithi148.stdout:5/348: dwrite d0/db/f3e [0,4194304] 0 2024-03-29T08:07:14.513 INFO:tasks.workunit.client.1.smithi148.stdout:4/315: dread - df/d10/d17/f1c zero size 2024-03-29T08:07:14.516 INFO:tasks.workunit.client.1.smithi148.stdout:9/317: chown db/d22/d26/f30 531493194 1 2024-03-29T08:07:14.523 INFO:tasks.workunit.client.1.smithi148.stdout:1/311: dwrite d3/de/d25/d35/f3e [4194304,4194304] 0 2024-03-29T08:07:14.529 INFO:tasks.workunit.client.1.smithi148.stdout:5/349: dread d0/f12 [4194304,4194304] 0 2024-03-29T08:07:14.529 INFO:tasks.workunit.client.1.smithi148.stdout:0/328: creat db/de/df/d11/d42/f67 x:0 0 0 2024-03-29T08:07:14.533 INFO:tasks.workunit.client.1.smithi148.stdout:3/337: dwrite de/d24/d33/d39/d46/d30/d37/f6f [0,4194304] 0 2024-03-29T08:07:14.539 INFO:tasks.workunit.client.1.smithi148.stdout:2/338: dwrite d7/da/f3f [0,4194304] 0 2024-03-29T08:07:14.539 INFO:tasks.workunit.client.0.smithi037.stdout:4/252: dwrite d3/d4/d15/d1d/db/f12 [0,4194304] 0 2024-03-29T08:07:14.539 INFO:tasks.workunit.client.0.smithi037.stdout:4/253: write d3/d4/d15/f29 [3028657,91464] 0 2024-03-29T08:07:14.541 INFO:tasks.workunit.client.1.smithi148.stdout:8/387: rename d17/d1d/l42 to d17/d1d/d7e/l80 0 2024-03-29T08:07:14.544 INFO:tasks.workunit.client.1.smithi148.stdout:0/329: chown db/de/df/c28 461517 1 2024-03-29T08:07:14.545 INFO:tasks.workunit.client.1.smithi148.stdout:2/339: chown d7/da/d4c/l4e 19 1 2024-03-29T08:07:14.546 INFO:tasks.workunit.client.1.smithi148.stdout:8/388: creat d17/d74/f81 x:0 0 0 2024-03-29T08:07:14.548 INFO:tasks.workunit.client.1.smithi148.stdout:2/340: read d7/f3b [1322442,104698] 0 2024-03-29T08:07:14.549 INFO:tasks.workunit.client.1.smithi148.stdout:3/338: rename de/l14 to de/d24/d33/d39/d46/d70/l75 0 2024-03-29T08:07:14.549 INFO:tasks.workunit.client.1.smithi148.stdout:6/303: dwrite d6/d9/f13 [0,4194304] 0 2024-03-29T08:07:14.552 INFO:tasks.workunit.client.1.smithi148.stdout:4/316: creat df/d10/d26/d34/f6d x:0 0 0 2024-03-29T08:07:14.553 INFO:tasks.workunit.client.1.smithi148.stdout:4/317: stat df/l37 0 2024-03-29T08:07:14.554 INFO:tasks.workunit.client.1.smithi148.stdout:8/389: truncate d17/d4e/f5d 1038375 0 2024-03-29T08:07:14.559 INFO:tasks.workunit.client.1.smithi148.stdout:7/360: dread d8/d14/d19/f63 [4194304,4194304] 0 2024-03-29T08:07:14.560 INFO:tasks.workunit.client.1.smithi148.stdout:3/339: truncate de/f61 539606 0 2024-03-29T08:07:14.563 INFO:tasks.workunit.client.0.smithi037.stdout:4/254: dwrite d3/d4/f2f [0,4194304] 0 2024-03-29T08:07:14.566 INFO:tasks.workunit.client.0.smithi037.stdout:4/255: write d3/f3a [3383807,39967] 0 2024-03-29T08:07:14.569 INFO:tasks.workunit.client.1.smithi148.stdout:1/312: dwrite d3/d55/f52 [0,4194304] 0 2024-03-29T08:07:14.572 INFO:tasks.workunit.client.1.smithi148.stdout:9/318: rename db/d10/d2d/d62/l63 to db/d10/d2d/d4e/d5f/l66 0 2024-03-29T08:07:14.577 INFO:tasks.workunit.client.1.smithi148.stdout:9/319: chown db/d22/d26/d54/f3c 23 1 2024-03-29T08:07:14.577 INFO:tasks.workunit.client.1.smithi148.stdout:2/341: write d7/da/fe [7040928,107857] 0 2024-03-29T08:07:14.581 INFO:tasks.workunit.client.1.smithi148.stdout:0/330: dwrite db/de/df/d2b/d3b/d47/f66 [0,4194304] 0 2024-03-29T08:07:14.583 INFO:tasks.workunit.client.1.smithi148.stdout:0/331: read db/de/df/d2b/f35 [1816853,70612] 0 2024-03-29T08:07:14.583 INFO:tasks.workunit.client.1.smithi148.stdout:1/313: unlink d3/c29 0 2024-03-29T08:07:14.585 INFO:tasks.workunit.client.1.smithi148.stdout:1/314: fsync d3/de/f1f 0 2024-03-29T08:07:14.585 INFO:tasks.workunit.client.1.smithi148.stdout:5/350: truncate d0/db/f3e 729735 0 2024-03-29T08:07:14.589 INFO:tasks.workunit.client.0.smithi037.stdout:4/256: dwrite d3/f3a [0,4194304] 0 2024-03-29T08:07:14.590 INFO:tasks.workunit.client.1.smithi148.stdout:3/340: rename de/d3b/c69 to de/d3b/c76 0 2024-03-29T08:07:14.593 INFO:tasks.workunit.client.1.smithi148.stdout:5/351: mkdir d0/db/d1e/d5b/d66/d73 0 2024-03-29T08:07:14.594 INFO:tasks.workunit.client.1.smithi148.stdout:6/304: creat d6/d9/d1e/f53 x:0 0 0 2024-03-29T08:07:14.601 INFO:tasks.workunit.client.1.smithi148.stdout:4/318: rename df/d10/d26/f47 to df/d10/f6e 0 2024-03-29T08:07:14.602 INFO:tasks.workunit.client.1.smithi148.stdout:0/332: dwrite db/de/f4b [4194304,4194304] 0 2024-03-29T08:07:14.605 INFO:tasks.workunit.client.1.smithi148.stdout:9/320: creat db/d10/f67 x:0 0 0 2024-03-29T08:07:14.606 INFO:tasks.workunit.client.1.smithi148.stdout:0/333: dread - db/de/f62 zero size 2024-03-29T08:07:14.607 INFO:tasks.workunit.client.1.smithi148.stdout:9/321: read - db/d22/d26/d54/f60 zero size 2024-03-29T08:07:14.610 INFO:tasks.workunit.client.1.smithi148.stdout:6/305: symlink d6/d9/d1e/d20/l54 0 2024-03-29T08:07:14.610 INFO:tasks.workunit.client.1.smithi148.stdout:7/361: truncate d8/d14/f2d 596394 0 2024-03-29T08:07:14.613 INFO:tasks.workunit.client.1.smithi148.stdout:9/322: fsync db/d22/d26/d54/d4d/d35/f5e 0 2024-03-29T08:07:14.614 INFO:tasks.workunit.client.1.smithi148.stdout:9/323: write db/d22/f51 [582366,1595] 0 2024-03-29T08:07:14.616 INFO:tasks.workunit.client.1.smithi148.stdout:4/319: unlink df/d10/d26/d34/d3e/c4c 0 2024-03-29T08:07:14.616 INFO:tasks.workunit.client.1.smithi148.stdout:5/352: creat d0/db/d1e/d5b/d66/f74 x:0 0 0 2024-03-29T08:07:14.618 INFO:tasks.workunit.client.1.smithi148.stdout:8/390: rename ca to d17/d18/d26/d63/d66/d5f/c82 0 2024-03-29T08:07:14.621 INFO:tasks.workunit.client.1.smithi148.stdout:2/342: getdents d7/da 0 2024-03-29T08:07:14.623 INFO:tasks.workunit.client.1.smithi148.stdout:6/306: symlink d6/d9/d1e/d27/l55 0 2024-03-29T08:07:14.626 INFO:tasks.workunit.client.1.smithi148.stdout:6/307: chown d6/f2d 175 1 2024-03-29T08:07:14.627 INFO:tasks.workunit.client.1.smithi148.stdout:4/320: symlink df/d10/l6f 0 2024-03-29T08:07:14.628 INFO:tasks.workunit.client.0.smithi037.stdout:4/257: truncate d3/d4/d15/d1d/db/d45/d1e/f4a 3904498 0 2024-03-29T08:07:14.629 INFO:tasks.workunit.client.0.smithi037.stdout:4/258: chown d3/c7 75202746 1 2024-03-29T08:07:14.630 INFO:tasks.workunit.client.1.smithi148.stdout:6/308: chown d6/d9/d1e/d27/c2f 0 1 2024-03-29T08:07:14.633 INFO:tasks.workunit.client.1.smithi148.stdout:6/309: write d6/d9/d1e/f26 [3454845,1547] 0 2024-03-29T08:07:14.634 INFO:tasks.workunit.client.1.smithi148.stdout:8/391: getdents d17/d4e 0 2024-03-29T08:07:14.634 INFO:tasks.workunit.client.0.smithi037.stdout:4/259: creat d3/d4/d15/d1d/db/d3d/f58 x:0 0 0 2024-03-29T08:07:14.636 INFO:tasks.workunit.client.1.smithi148.stdout:4/321: unlink df/d10/d26/c5b 0 2024-03-29T08:07:14.638 INFO:tasks.workunit.client.1.smithi148.stdout:9/324: link db/d22/d26/d54/f45 db/d10/d2d/f68 0 2024-03-29T08:07:14.640 INFO:tasks.workunit.client.1.smithi148.stdout:4/322: truncate df/d10/f53 312983 0 2024-03-29T08:07:14.640 INFO:tasks.workunit.client.0.smithi037.stdout:4/260: mkdir d3/d4/d15/d1d/db/d45/d1e/d59 0 2024-03-29T08:07:14.640 INFO:tasks.workunit.client.1.smithi148.stdout:8/392: rmdir d17 39 2024-03-29T08:07:14.642 INFO:tasks.workunit.client.1.smithi148.stdout:3/341: dread de/d24/d33/d39/d46/d30/f3f [0,4194304] 0 2024-03-29T08:07:14.643 INFO:tasks.workunit.client.1.smithi148.stdout:6/310: creat d6/d9/d1e/d20/d30/f56 x:0 0 0 2024-03-29T08:07:14.644 INFO:tasks.workunit.client.1.smithi148.stdout:5/353: dread d0/db/f34 [0,4194304] 0 2024-03-29T08:07:14.645 INFO:tasks.workunit.client.1.smithi148.stdout:1/315: truncate d3/de/d25/d35/d42/f38 6517757 0 2024-03-29T08:07:14.647 INFO:tasks.workunit.client.0.smithi037.stdout:4/261: creat d3/d4/d15/d1d/db/d45/f5a x:0 0 0 2024-03-29T08:07:14.647 INFO:tasks.workunit.client.1.smithi148.stdout:4/323: symlink df/d10/l70 0 2024-03-29T08:07:14.648 INFO:tasks.workunit.client.0.smithi037.stdout:4/262: readlink d3/d4/d15/d1d/db/d45/l47 0 2024-03-29T08:07:14.653 INFO:tasks.workunit.client.0.smithi037.stdout:4/263: creat d3/d54/f5b x:0 0 0 2024-03-29T08:07:14.656 INFO:tasks.workunit.client.1.smithi148.stdout:3/342: rename de/f35 to de/d3b/d49/f77 0 2024-03-29T08:07:14.656 INFO:tasks.workunit.client.1.smithi148.stdout:2/343: dwrite d7/da/d4c/f69 [0,4194304] 0 2024-03-29T08:07:14.657 INFO:tasks.workunit.client.1.smithi148.stdout:6/311: rmdir d6/d9/d1e/d20/d4f 0 2024-03-29T08:07:14.660 INFO:tasks.workunit.client.1.smithi148.stdout:7/362: dread d8/f10 [0,4194304] 0 2024-03-29T08:07:14.661 INFO:tasks.workunit.client.1.smithi148.stdout:2/344: dread - d7/d24/f36 zero size 2024-03-29T08:07:14.661 INFO:tasks.workunit.client.1.smithi148.stdout:3/343: write de/d3b/d49/f68 [4180917,59626] 0 2024-03-29T08:07:14.663 INFO:tasks.workunit.client.1.smithi148.stdout:0/334: dread db/de/df/d11/d2d/f36 [0,4194304] 0 2024-03-29T08:07:14.664 INFO:tasks.workunit.client.1.smithi148.stdout:8/393: rename d17/d18/d26/d63/d66/f7a to d17/d1d/d7e/f83 0 2024-03-29T08:07:14.665 INFO:tasks.workunit.client.0.smithi037.stdout:4/264: link d3/d4/d15/d1d/db/d3d/c53 d3/d4/d15/d1d/db/d45/d1e/d59/c5c 0 2024-03-29T08:07:14.669 INFO:tasks.workunit.client.1.smithi148.stdout:6/312: write d6/f1a [2253288,19626] 0 2024-03-29T08:07:14.672 INFO:tasks.workunit.client.1.smithi148.stdout:6/313: write f5 [354817,38253] 0 2024-03-29T08:07:14.674 INFO:tasks.workunit.client.1.smithi148.stdout:5/354: dread d0/db/de/f50 [0,4194304] 0 2024-03-29T08:07:14.677 INFO:tasks.workunit.client.1.smithi148.stdout:6/314: write d6/d9/d1e/f26 [4663206,33077] 0 2024-03-29T08:07:14.677 INFO:tasks.workunit.client.1.smithi148.stdout:3/344: creat de/d24/d33/d39/d46/d30/f78 x:0 0 0 2024-03-29T08:07:14.679 INFO:tasks.workunit.client.1.smithi148.stdout:0/335: unlink db/de/f25 0 2024-03-29T08:07:14.681 INFO:tasks.workunit.client.1.smithi148.stdout:3/345: write de/d24/d33/f5c [200544,37316] 0 2024-03-29T08:07:14.683 INFO:tasks.workunit.client.1.smithi148.stdout:3/346: readlink de/l45 0 2024-03-29T08:07:14.683 INFO:tasks.workunit.client.0.smithi037.stdout:4/265: fsync d3/d4/d27/f2e 0 2024-03-29T08:07:14.686 INFO:tasks.workunit.client.1.smithi148.stdout:7/363: symlink d8/d14/d45/d4d/d78/l80 0 2024-03-29T08:07:14.692 INFO:tasks.workunit.client.1.smithi148.stdout:3/347: rmdir de/d24/d33/d39 39 2024-03-29T08:07:14.694 INFO:tasks.workunit.client.1.smithi148.stdout:5/355: dread d0/f9 [0,4194304] 0 2024-03-29T08:07:14.694 INFO:tasks.workunit.client.1.smithi148.stdout:0/336: mknod db/de/df/c68 0 2024-03-29T08:07:14.694 INFO:tasks.workunit.client.1.smithi148.stdout:7/364: read d8/d29/d44/d49/f74 [1672530,48738] 0 2024-03-29T08:07:14.694 INFO:tasks.workunit.client.1.smithi148.stdout:9/325: truncate db/d22/d26/d54/d4d/f18 5755689 0 2024-03-29T08:07:14.696 INFO:tasks.workunit.client.0.smithi037.stdout:4/266: truncate d3/d4/d15/d1d/db/f12 64979 0 2024-03-29T08:07:14.696 INFO:tasks.workunit.client.1.smithi148.stdout:3/348: truncate de/d3b/f74 766893 0 2024-03-29T08:07:14.697 INFO:tasks.workunit.client.1.smithi148.stdout:4/324: dread df/d10/d26/d34/f5c [0,4194304] 0 2024-03-29T08:07:14.697 INFO:tasks.workunit.client.1.smithi148.stdout:1/316: truncate d3/d55/d44/f53 1201707 0 2024-03-29T08:07:14.700 INFO:tasks.workunit.client.0.smithi037.stdout:4/267: unlink d3/d4/d15/d1d/fc 0 2024-03-29T08:07:14.701 INFO:tasks.workunit.client.1.smithi148.stdout:5/356: symlink d0/db/d1e/d35/d42/d67/l75 0 2024-03-29T08:07:14.701 INFO:tasks.workunit.client.1.smithi148.stdout:1/317: read d3/de/d25/d35/d42/f1a [5816311,82652] 0 2024-03-29T08:07:14.701 INFO:tasks.workunit.client.1.smithi148.stdout:4/325: dread df/d10/d26/d34/d3e/f58 [0,4194304] 0 2024-03-29T08:07:14.702 INFO:tasks.workunit.client.1.smithi148.stdout:5/357: stat d0/db/d1e/d5b/d66/d6c 0 2024-03-29T08:07:14.703 INFO:tasks.workunit.client.1.smithi148.stdout:7/365: mknod d8/d14/d19/c81 0 2024-03-29T08:07:14.703 INFO:tasks.workunit.client.1.smithi148.stdout:4/326: fdatasync df/d10/f1d 0 2024-03-29T08:07:14.704 INFO:tasks.workunit.client.0.smithi037.stdout:4/268: creat d3/f5d x:0 0 0 2024-03-29T08:07:14.705 INFO:tasks.workunit.client.1.smithi148.stdout:5/358: stat d0/db/d1e/d35/d42/f5e 0 2024-03-29T08:07:14.709 INFO:tasks.workunit.client.1.smithi148.stdout:3/349: mkdir de/d24/d33/d39/d46/d30/d37/d79 0 2024-03-29T08:07:14.712 INFO:tasks.workunit.client.0.smithi037.stdout:2/179: sync 2024-03-29T08:07:14.713 INFO:tasks.workunit.client.1.smithi148.stdout:4/327: unlink df/d10/f31 0 2024-03-29T08:07:14.715 INFO:tasks.workunit.client.1.smithi148.stdout:2/345: dwrite d7/da/d33/d45/d4d/d2e/d37/f67 [4194304,4194304] 0 2024-03-29T08:07:14.718 INFO:tasks.workunit.client.1.smithi148.stdout:9/326: fsync db/d22/d26/d54/d4d/f18 0 2024-03-29T08:07:14.719 INFO:tasks.workunit.client.0.smithi037.stdout:4/269: write d3/d4/d15/d1d/db/f12 [215926,60155] 0 2024-03-29T08:07:14.723 INFO:tasks.workunit.client.0.smithi037.stdout:4/270: read - d3/d4/d27/f4b zero size 2024-03-29T08:07:14.725 INFO:tasks.workunit.client.0.smithi037.stdout:2/180: rename d0/d20/d22/c34 to d0/d20/d2e/d38/c3b 0 2024-03-29T08:07:14.727 INFO:tasks.workunit.client.0.smithi037.stdout:4/271: mkdir d3/d4/d5e 0 2024-03-29T08:07:14.728 INFO:tasks.workunit.client.0.smithi037.stdout:2/181: creat d0/d20/d22/f3c x:0 0 0 2024-03-29T08:07:14.731 INFO:tasks.workunit.client.1.smithi148.stdout:5/359: unlink d0/db/de/l13 0 2024-03-29T08:07:14.731 INFO:tasks.workunit.client.0.smithi037.stdout:4/272: chown d3/d4/d15/d1d/l3c 28179301 1 2024-03-29T08:07:14.732 INFO:tasks.workunit.client.0.smithi037.stdout:2/182: creat d0/d20/f3d x:0 0 0 2024-03-29T08:07:14.736 INFO:tasks.workunit.client.0.smithi037.stdout:4/273: mkdir d3/d4/d15/d1d/db/d45/d1e/d5f 0 2024-03-29T08:07:14.737 INFO:tasks.workunit.client.1.smithi148.stdout:7/366: fdatasync d8/d14/f2d 0 2024-03-29T08:07:14.738 INFO:tasks.workunit.client.0.smithi037.stdout:4/274: dread - d3/d4/d15/d1d/db/d3d/f58 zero size 2024-03-29T08:07:14.738 INFO:tasks.workunit.client.1.smithi148.stdout:4/328: dwrite df/d10/d26/d34/f61 [0,4194304] 0 2024-03-29T08:07:14.741 INFO:tasks.workunit.client.0.smithi037.stdout:4/275: chown d3/d4/d15/d1d/db/d45/f30 3 1 2024-03-29T08:07:14.742 INFO:tasks.workunit.client.1.smithi148.stdout:2/346: dwrite d7/da/d33/d45/d4d/f2a [0,4194304] 0 2024-03-29T08:07:14.746 INFO:tasks.workunit.client.1.smithi148.stdout:1/318: dread d3/d55/f43 [0,4194304] 0 2024-03-29T08:07:14.746 INFO:tasks.workunit.client.1.smithi148.stdout:2/347: rmdir d7/da/d33 39 2024-03-29T08:07:14.746 INFO:tasks.workunit.client.1.smithi148.stdout:9/327: dwrite db/d22/d26/f30 [4194304,4194304] 0 2024-03-29T08:07:14.748 INFO:tasks.workunit.client.0.smithi037.stdout:4/276: mkdir d3/d4/d15/d1d/db/d45/d1e/d59/d60 0 2024-03-29T08:07:14.751 INFO:tasks.workunit.client.1.smithi148.stdout:1/319: creat d3/de/d25/d35/d42/d23/f59 x:0 0 0 2024-03-29T08:07:14.753 INFO:tasks.workunit.client.0.smithi037.stdout:4/277: dread d3/f44 [0,4194304] 0 2024-03-29T08:07:14.754 INFO:tasks.workunit.client.1.smithi148.stdout:3/350: truncate de/d24/f2b 2870300 0 2024-03-29T08:07:14.755 INFO:tasks.workunit.client.1.smithi148.stdout:9/328: rename db/d22/d3f/d61/c50 to db/d10/d2d/d4e/d5f/c69 0 2024-03-29T08:07:14.760 INFO:tasks.workunit.client.0.smithi037.stdout:4/278: symlink d3/d4/l61 0 2024-03-29T08:07:14.761 INFO:tasks.workunit.client.1.smithi148.stdout:7/367: dwrite d8/d14/f72 [0,4194304] 0 2024-03-29T08:07:14.762 INFO:tasks.workunit.client.1.smithi148.stdout:3/351: creat de/d24/d33/d39/d46/d30/d37/f7a x:0 0 0 2024-03-29T08:07:14.762 INFO:tasks.workunit.client.1.smithi148.stdout:2/348: truncate d7/da/d33/d3c/f23 4090419 0 2024-03-29T08:07:14.765 INFO:tasks.workunit.client.1.smithi148.stdout:3/352: mknod de/d3b/d49/c7b 0 2024-03-29T08:07:14.769 INFO:tasks.workunit.client.1.smithi148.stdout:7/368: write d8/d29/d44/d49/f74 [1563973,70740] 0 2024-03-29T08:07:14.769 INFO:tasks.workunit.client.0.smithi037.stdout:4/279: getdents d3/d4/d15/d1d/db/d3d 0 2024-03-29T08:07:14.773 INFO:tasks.workunit.client.1.smithi148.stdout:3/353: read de/d24/d33/d39/d46/f25 [4045514,61846] 0 2024-03-29T08:07:14.773 INFO:tasks.workunit.client.1.smithi148.stdout:1/320: write d3/d55/f37 [4542621,100830] 0 2024-03-29T08:07:14.777 INFO:tasks.workunit.client.1.smithi148.stdout:7/369: creat d8/d29/d44/d49/f82 x:0 0 0 2024-03-29T08:07:14.777 INFO:tasks.workunit.client.1.smithi148.stdout:3/354: write de/f2c [820336,62641] 0 2024-03-29T08:07:14.781 INFO:tasks.workunit.client.1.smithi148.stdout:7/370: creat d8/d29/d44/d49/f83 x:0 0 0 2024-03-29T08:07:14.784 INFO:tasks.workunit.client.0.smithi037.stdout:4/280: rmdir d3/d4/d15/d1d 39 2024-03-29T08:07:14.785 INFO:tasks.workunit.client.1.smithi148.stdout:2/349: dread d7/da/fe [0,4194304] 0 2024-03-29T08:07:14.787 INFO:tasks.workunit.client.1.smithi148.stdout:7/371: symlink d8/l84 0 2024-03-29T08:07:14.787 INFO:tasks.workunit.client.1.smithi148.stdout:9/329: dwrite db/f5c [0,4194304] 0 2024-03-29T08:07:14.791 INFO:tasks.workunit.client.0.smithi037.stdout:4/281: creat d3/d4/d15/d1d/db/d45/d1e/d5f/f62 x:0 0 0 2024-03-29T08:07:14.791 INFO:tasks.workunit.client.1.smithi148.stdout:7/372: mkdir d8/d29/d44/d50/d85 0 2024-03-29T08:07:14.794 INFO:tasks.workunit.client.0.smithi037.stdout:4/282: rmdir d3/d4/d15/d1d/db/d3d 39 2024-03-29T08:07:14.795 INFO:tasks.workunit.client.1.smithi148.stdout:3/355: dread de/f15 [0,4194304] 0 2024-03-29T08:07:14.796 INFO:tasks.workunit.client.1.smithi148.stdout:9/330: rename db/d20/c1d to db/d22/d3f/d61/c6a 0 2024-03-29T08:07:14.797 INFO:tasks.workunit.client.1.smithi148.stdout:9/331: stat db/d22/d26/f2a 0 2024-03-29T08:07:14.800 INFO:tasks.workunit.client.1.smithi148.stdout:9/332: creat db/d10/d2d/d4e/f6b x:0 0 0 2024-03-29T08:07:14.803 INFO:tasks.workunit.client.1.smithi148.stdout:9/333: creat db/d10/d2d/d4e/f6c x:0 0 0 2024-03-29T08:07:14.803 INFO:tasks.workunit.client.0.smithi037.stdout:4/283: rmdir d3/d4/d15 39 2024-03-29T08:07:14.807 INFO:tasks.workunit.client.1.smithi148.stdout:3/356: dread de/d3b/d49/f77 [0,4194304] 0 2024-03-29T08:07:14.808 INFO:tasks.workunit.client.1.smithi148.stdout:9/334: rename db/d10/d2d/d62 to db/d10/d6d 0 2024-03-29T08:07:14.808 INFO:tasks.workunit.client.0.smithi037.stdout:4/284: creat d3/d4/d15/f63 x:0 0 0 2024-03-29T08:07:14.810 INFO:tasks.workunit.client.1.smithi148.stdout:9/335: mkdir db/d10/d2d/d4e/d5f/d6e 0 2024-03-29T08:07:14.813 INFO:tasks.workunit.client.1.smithi148.stdout:9/336: rmdir db/d22/d26/d54/d4d/d35/d4b 39 2024-03-29T08:07:14.814 INFO:tasks.workunit.client.1.smithi148.stdout:9/337: fdatasync db/d22/f51 0 2024-03-29T08:07:14.817 INFO:tasks.workunit.client.1.smithi148.stdout:9/338: symlink db/d10/d2d/l6f 0 2024-03-29T08:07:14.822 INFO:tasks.workunit.client.0.smithi037.stdout:4/285: dwrite d3/f5d [0,4194304] 0 2024-03-29T08:07:14.822 INFO:tasks.workunit.client.1.smithi148.stdout:9/339: link db/f5c db/d20/f70 0 2024-03-29T08:07:14.824 INFO:tasks.workunit.client.0.smithi037.stdout:4/286: readlink d3/d4/d15/d1d/db/d45/l47 0 2024-03-29T08:07:14.826 INFO:tasks.workunit.client.0.smithi037.stdout:4/287: write d3/d4/d27/f2e [400174,57421] 0 2024-03-29T08:07:14.827 INFO:tasks.workunit.client.1.smithi148.stdout:3/357: dread de/d24/d33/d39/d46/f25 [0,4194304] 0 2024-03-29T08:07:14.842 INFO:tasks.workunit.client.1.smithi148.stdout:8/394: sync 2024-03-29T08:07:14.844 INFO:tasks.workunit.client.1.smithi148.stdout:9/340: dwrite db/d10/d2d/d4e/f6c [0,4194304] 0 2024-03-29T08:07:14.846 INFO:tasks.workunit.client.0.smithi037.stdout:4/288: dread d3/d4/d15/d1d/db/f1a [0,4194304] 0 2024-03-29T08:07:14.846 INFO:tasks.workunit.client.1.smithi148.stdout:8/395: write d17/d18/f36 [3204445,67489] 0 2024-03-29T08:07:14.846 INFO:tasks.workunit.client.1.smithi148.stdout:3/358: dwrite de/d24/d33/d39/d46/d30/f78 [0,4194304] 0 2024-03-29T08:07:14.849 INFO:tasks.workunit.client.0.smithi037.stdout:4/289: write d3/d4/d15/f49 [2109946,61575] 0 2024-03-29T08:07:14.853 INFO:tasks.workunit.client.0.smithi037.stdout:4/290: fsync d3/d4/d15/d1d/db/d3d/f58 0 2024-03-29T08:07:14.855 INFO:tasks.workunit.client.1.smithi148.stdout:8/396: dread - d17/d18/f3f zero size 2024-03-29T08:07:14.855 INFO:tasks.workunit.client.1.smithi148.stdout:9/341: mknod db/d10/d2d/d4e/d5f/c71 0 2024-03-29T08:07:14.858 INFO:tasks.workunit.client.1.smithi148.stdout:8/397: write f15 [7734016,41470] 0 2024-03-29T08:07:14.859 INFO:tasks.workunit.client.0.smithi037.stdout:4/291: creat d3/d4/d15/d1d/f64 x:0 0 0 2024-03-29T08:07:14.860 INFO:tasks.workunit.client.1.smithi148.stdout:3/359: creat de/d24/d33/d39/d46/d30/d37/d79/f7c x:0 0 0 2024-03-29T08:07:14.867 INFO:tasks.workunit.client.1.smithi148.stdout:3/360: rename de/d24/d33/d39/d46/l34 to de/d24/d33/d39/d46/d70/l7d 0 2024-03-29T08:07:14.867 INFO:tasks.workunit.client.0.smithi037.stdout:4/292: mknod d3/d4/d15/c65 0 2024-03-29T08:07:14.873 INFO:tasks.workunit.client.1.smithi148.stdout:3/361: getdents de/d24/d33/d39/d71 0 2024-03-29T08:07:14.874 INFO:tasks.workunit.client.1.smithi148.stdout:9/342: dwrite db/d22/d26/d54/d4d/f41 [0,4194304] 0 2024-03-29T08:07:14.876 INFO:tasks.workunit.client.1.smithi148.stdout:8/398: dwrite d17/f28 [0,4194304] 0 2024-03-29T08:07:14.879 INFO:tasks.workunit.client.0.smithi037.stdout:4/293: dread d3/d4/d15/d1d/db/d45/d1e/f4a [0,4194304] 0 2024-03-29T08:07:14.882 INFO:tasks.workunit.client.0.smithi037.stdout:4/294: rmdir d3/d4/d15/d1d/db/d45/d1e 39 2024-03-29T08:07:14.886 INFO:tasks.workunit.client.1.smithi148.stdout:8/399: mknod d17/d1d/c84 0 2024-03-29T08:07:14.887 INFO:tasks.workunit.client.0.smithi037.stdout:4/295: rmdir d3/d4/d5e 0 2024-03-29T08:07:14.889 INFO:tasks.workunit.client.1.smithi148.stdout:8/400: creat d17/d1d/d7e/f85 x:0 0 0 2024-03-29T08:07:14.891 INFO:tasks.workunit.client.1.smithi148.stdout:3/362: dwrite de/d3b/d49/f64 [0,4194304] 0 2024-03-29T08:07:14.891 INFO:tasks.workunit.client.1.smithi148.stdout:8/401: mkdir d17/d74/d86 0 2024-03-29T08:07:14.892 INFO:tasks.workunit.client.0.smithi037.stdout:4/296: creat d3/f66 x:0 0 0 2024-03-29T08:07:14.893 INFO:tasks.workunit.client.0.smithi037.stdout:4/297: chown d3/d4/d15/d1d/db/d3d 14 1 2024-03-29T08:07:14.896 INFO:tasks.workunit.client.1.smithi148.stdout:3/363: truncate de/d24/d33/d39/d46/d30/d37/f7a 79469 0 2024-03-29T08:07:14.898 INFO:tasks.workunit.client.1.smithi148.stdout:8/402: dread d17/d74/d39/d6f/f5b [0,4194304] 0 2024-03-29T08:07:14.903 INFO:tasks.workunit.client.1.smithi148.stdout:8/403: getdents d17/d18/d26/d63/d7d 0 2024-03-29T08:07:14.907 INFO:tasks.workunit.client.1.smithi148.stdout:8/404: rename d17/d18/c46 to d17/d18/d26/d63/c87 0 2024-03-29T08:07:14.912 INFO:tasks.workunit.client.1.smithi148.stdout:3/364: dwrite de/d24/f31 [0,4194304] 0 2024-03-29T08:07:14.916 INFO:tasks.workunit.client.1.smithi148.stdout:3/365: mknod de/d24/d33/d39/d46/c7e 0 2024-03-29T08:07:14.919 INFO:tasks.workunit.client.1.smithi148.stdout:3/366: chown de/d24/d33/c4c 5882020 1 2024-03-29T08:07:14.921 INFO:tasks.workunit.client.0.smithi037.stdout:3/287: sync 2024-03-29T08:07:14.921 INFO:tasks.workunit.client.0.smithi037.stdout:9/436: sync 2024-03-29T08:07:14.923 INFO:tasks.workunit.client.1.smithi148.stdout:3/367: mkdir de/d24/d33/d39/d7f 0 2024-03-29T08:07:14.926 INFO:tasks.workunit.client.0.smithi037.stdout:9/437: symlink d1/d4/d19/d69/d75/l99 0 2024-03-29T08:07:14.929 INFO:tasks.workunit.client.0.smithi037.stdout:9/438: chown d1/d4/d19/d1e/d4a/d5c/d60/d31/d87/d88 0 1 2024-03-29T08:07:14.930 INFO:tasks.workunit.client.0.smithi037.stdout:3/288: rmdir dd/d15/d18/d4b/d61 0 2024-03-29T08:07:14.931 INFO:tasks.workunit.client.1.smithi148.stdout:8/405: dread d17/d74/f6e [0,4194304] 0 2024-03-29T08:07:14.932 INFO:tasks.workunit.client.0.smithi037.stdout:9/439: chown d1/d4/d19/d1e/d25 0 1 2024-03-29T08:07:14.935 INFO:tasks.workunit.client.1.smithi148.stdout:8/406: creat d17/d4e/f88 x:0 0 0 2024-03-29T08:07:14.935 INFO:tasks.workunit.client.1.smithi148.stdout:3/368: dread de/d24/d33/d39/d46/f5f [0,4194304] 0 2024-03-29T08:07:14.937 INFO:tasks.workunit.client.1.smithi148.stdout:8/407: read d17/d18/f3a [77479,33394] 0 2024-03-29T08:07:14.939 INFO:tasks.workunit.client.0.smithi037.stdout:9/440: getdents d1/d4/d19/d1e/d4a 0 2024-03-29T08:07:14.940 INFO:tasks.workunit.client.1.smithi148.stdout:3/369: dread de/d24/f36 [4194304,4194304] 0 2024-03-29T08:07:14.940 INFO:tasks.workunit.client.1.smithi148.stdout:8/408: mknod d17/d1d/d7e/c89 0 2024-03-29T08:07:14.941 INFO:tasks.workunit.client.1.smithi148.stdout:3/370: chown cd 5236 1 2024-03-29T08:07:14.943 INFO:tasks.workunit.client.0.smithi037.stdout:9/441: creat d1/d4/d19/d1e/d4a/d5c/d60/d23/f9a x:0 0 0 2024-03-29T08:07:14.944 INFO:tasks.workunit.client.0.smithi037.stdout:3/289: dread dd/f1f [0,4194304] 0 2024-03-29T08:07:14.944 INFO:tasks.workunit.client.1.smithi148.stdout:3/371: creat de/f80 x:0 0 0 2024-03-29T08:07:14.945 INFO:tasks.workunit.client.1.smithi148.stdout:3/372: chown de 1166504295 1 2024-03-29T08:07:14.946 INFO:tasks.workunit.client.0.smithi037.stdout:3/290: write dd/d15/d27/f41 [119981,28143] 0 2024-03-29T08:07:14.948 INFO:tasks.workunit.client.0.smithi037.stdout:9/442: mkdir d1/d1f/d3a/d9b 0 2024-03-29T08:07:14.948 INFO:tasks.workunit.client.1.smithi148.stdout:8/409: link d17/d74/d39/f76 d17/d18/f8a 0 2024-03-29T08:07:14.950 INFO:tasks.workunit.client.1.smithi148.stdout:3/373: mkdir de/d24/d81 0 2024-03-29T08:07:14.951 INFO:tasks.workunit.client.1.smithi148.stdout:8/410: readlink d17/d18/d26/l7b 0 2024-03-29T08:07:14.953 INFO:tasks.workunit.client.0.smithi037.stdout:9/443: unlink d1/d4/d19/d1e/d25/c40 0 2024-03-29T08:07:14.954 INFO:tasks.workunit.client.1.smithi148.stdout:8/411: mknod d17/d18/d26/c8b 0 2024-03-29T08:07:14.956 INFO:tasks.workunit.client.0.smithi037.stdout:3/291: getdents dd 0 2024-03-29T08:07:14.956 INFO:tasks.workunit.client.1.smithi148.stdout:8/412: write d17/d74/f6e [962611,82965] 0 2024-03-29T08:07:14.959 INFO:tasks.workunit.client.0.smithi037.stdout:3/292: rmdir dd/d46 39 2024-03-29T08:07:14.960 INFO:tasks.workunit.client.0.smithi037.stdout:9/444: read d1/d4/d19/d1e/d25/d16/f56 [503197,17088] 0 2024-03-29T08:07:14.962 INFO:tasks.workunit.client.1.smithi148.stdout:8/413: rename d17/d1d/d7e/c89 to d17/d18/d26/d63/d7d/c8c 0 2024-03-29T08:07:14.962 INFO:tasks.workunit.client.0.smithi037.stdout:9/445: truncate d1/f4c 1617530 0 2024-03-29T08:07:14.962 INFO:tasks.workunit.client.0.smithi037.stdout:9/446: fdatasync d1/f8d 0 2024-03-29T08:07:14.965 INFO:tasks.workunit.client.1.smithi148.stdout:3/374: dwrite de/d24/d33/d39/d46/d30/f3f [0,4194304] 0 2024-03-29T08:07:14.965 INFO:tasks.workunit.client.0.smithi037.stdout:9/447: write d1/d1f/d3a/f46 [826706,46472] 0 2024-03-29T08:07:14.967 INFO:tasks.workunit.client.1.smithi148.stdout:3/375: truncate de/f2c 1717038 0 2024-03-29T08:07:14.970 INFO:tasks.workunit.client.0.smithi037.stdout:9/448: creat d1/d4/d19/d69/f9c x:0 0 0 2024-03-29T08:07:14.970 INFO:tasks.workunit.client.1.smithi148.stdout:3/376: readlink de/d24/d33/d39/d46/d70/l75 0 2024-03-29T08:07:14.974 INFO:tasks.workunit.client.0.smithi037.stdout:9/449: getdents d1/d4/d19/d1e/d71/d89/d92 0 2024-03-29T08:07:14.974 INFO:tasks.workunit.client.1.smithi148.stdout:3/377: truncate de/d24/d33/d39/d46/f65 12483 0 2024-03-29T08:07:14.975 INFO:tasks.workunit.client.1.smithi148.stdout:3/378: stat de/d3b/d49 0 2024-03-29T08:07:14.978 INFO:tasks.workunit.client.1.smithi148.stdout:6/315: sync 2024-03-29T08:07:14.978 INFO:tasks.workunit.client.1.smithi148.stdout:0/337: sync 2024-03-29T08:07:14.978 INFO:tasks.workunit.client.1.smithi148.stdout:3/379: symlink de/d24/d33/d6b/l82 0 2024-03-29T08:07:14.979 INFO:tasks.workunit.client.0.smithi037.stdout:9/450: mknod d1/d4/d19/d1e/d4a/d5c/d60/d13/c9d 0 2024-03-29T08:07:14.979 INFO:tasks.workunit.client.1.smithi148.stdout:6/316: stat d6/d9/d1e/d36/c4b 0 2024-03-29T08:07:14.980 INFO:tasks.workunit.client.1.smithi148.stdout:3/380: chown de/d24/d33/d39/d46/d30/d37/d79 72853265 1 2024-03-29T08:07:14.981 INFO:tasks.workunit.client.0.smithi037.stdout:9/451: dread - d1/d4/d19/d1e/d4a/d6f/f91 zero size 2024-03-29T08:07:14.983 INFO:tasks.workunit.client.1.smithi148.stdout:6/317: creat d6/d9/d1e/d36/d4d/f57 x:0 0 0 2024-03-29T08:07:14.983 INFO:tasks.workunit.client.1.smithi148.stdout:3/381: readlink de/d24/d33/d39/d46/l57 0 2024-03-29T08:07:14.986 INFO:tasks.workunit.client.1.smithi148.stdout:3/382: creat de/d24/d33/d39/d46/d70/f83 x:0 0 0 2024-03-29T08:07:14.988 INFO:tasks.workunit.client.1.smithi148.stdout:6/318: rename d6/d9/d1e/d27/c2a to d6/d9/d1e/d36/c58 0 2024-03-29T08:07:14.994 INFO:tasks.workunit.client.1.smithi148.stdout:6/319: unlink d6/fa 0 2024-03-29T08:07:14.994 INFO:tasks.workunit.client.1.smithi148.stdout:0/338: dwrite db/f23 [4194304,4194304] 0 2024-03-29T08:07:14.995 INFO:tasks.workunit.client.0.smithi037.stdout:9/452: dwrite d1/d1f/f5f [0,4194304] 0 2024-03-29T08:07:14.995 INFO:tasks.workunit.client.1.smithi148.stdout:3/383: link de/d3b/d49/c7b de/d24/d33/d39/d71/c84 0 2024-03-29T08:07:14.996 INFO:tasks.workunit.client.1.smithi148.stdout:6/320: write d6/f2d [1081139,81010] 0 2024-03-29T08:07:15.000 INFO:tasks.workunit.client.1.smithi148.stdout:3/384: symlink de/l85 0 2024-03-29T08:07:15.000 INFO:tasks.workunit.client.0.smithi037.stdout:9/453: chown d1/d4/d19/d1e/d4a/d5c/d60/d13/c7b 7 1 2024-03-29T08:07:15.001 INFO:tasks.workunit.client.1.smithi148.stdout:0/339: symlink db/l69 0 2024-03-29T08:07:15.003 INFO:tasks.workunit.client.1.smithi148.stdout:3/385: chown de/f2f 203023990 1 2024-03-29T08:07:15.007 INFO:tasks.workunit.client.0.smithi037.stdout:9/454: creat d1/d4/d19/d1e/d25/d16/f9e x:0 0 0 2024-03-29T08:07:15.007 INFO:tasks.workunit.client.1.smithi148.stdout:6/321: write d6/fe [1449163,109486] 0 2024-03-29T08:07:15.011 INFO:tasks.workunit.client.0.smithi037.stdout:9/455: creat d1/d3/d44/f9f x:0 0 0 2024-03-29T08:07:15.016 INFO:tasks.workunit.client.1.smithi148.stdout:6/322: rename d6/d9/d1e/d20/f37 to d6/d9/d1e/d27/f59 0 2024-03-29T08:07:15.018 INFO:tasks.workunit.client.0.smithi037.stdout:9/456: unlink d1/d1f/d3a/c4f 0 2024-03-29T08:07:15.021 INFO:tasks.workunit.client.1.smithi148.stdout:6/323: symlink d6/d9/d1e/d20/l5a 0 2024-03-29T08:07:15.021 INFO:tasks.workunit.client.0.smithi037.stdout:9/457: creat d1/d4/d19/d1e/d4a/fa0 x:0 0 0 2024-03-29T08:07:15.023 INFO:tasks.workunit.client.1.smithi148.stdout:0/340: dwrite db/de/df/d11/d42/f67 [0,4194304] 0 2024-03-29T08:07:15.026 INFO:tasks.workunit.client.1.smithi148.stdout:0/341: rmdir db/de/d24 39 2024-03-29T08:07:15.035 INFO:tasks.workunit.client.1.smithi148.stdout:6/324: rmdir d6/d9/d1e 39 2024-03-29T08:07:15.042 INFO:tasks.workunit.client.1.smithi148.stdout:3/386: dwrite de/d24/d33/d39/d46/d30/d37/f4a [0,4194304] 0 2024-03-29T08:07:15.044 INFO:tasks.workunit.client.1.smithi148.stdout:6/325: rename d6/d9/d1e/d27/l55 to d6/d9/d1e/d27/d48/l5b 0 2024-03-29T08:07:15.045 INFO:tasks.workunit.client.0.smithi037.stdout:9/458: dread d1/d4/d19/d1e/d4a/d5c/d60/f7a [0,4194304] 0 2024-03-29T08:07:15.047 INFO:tasks.workunit.client.1.smithi148.stdout:0/342: dwrite db/de/d24/f38 [0,4194304] 0 2024-03-29T08:07:15.047 INFO:tasks.workunit.client.1.smithi148.stdout:6/326: write d6/d9/f13 [5070246,55803] 0 2024-03-29T08:07:15.054 INFO:tasks.workunit.client.0.smithi037.stdout:9/459: symlink d1/d4/d19/d1e/d71/d89/d92/la1 0 2024-03-29T08:07:15.054 INFO:tasks.workunit.client.1.smithi148.stdout:3/387: truncate de/f61 284093 0 2024-03-29T08:07:15.056 INFO:tasks.workunit.client.1.smithi148.stdout:0/343: rename db/de/df/d2b/d3b/d47/f66 to db/de/df/d2b/d3b/d47/d4c/d5a/f6a 0 2024-03-29T08:07:15.059 INFO:tasks.workunit.client.1.smithi148.stdout:0/344: mknod db/c6b 0 2024-03-29T08:07:15.062 INFO:tasks.workunit.client.1.smithi148.stdout:0/345: write db/de/df/d11/d42/f67 [1591029,46362] 0 2024-03-29T08:07:15.063 INFO:tasks.workunit.client.1.smithi148.stdout:3/388: link de/d24/d33/d39/d46/f23 de/d24/d33/d39/d46/d30/f86 0 2024-03-29T08:07:15.064 INFO:tasks.workunit.client.1.smithi148.stdout:6/327: dwrite d6/d9/d1e/d27/f32 [0,4194304] 0 2024-03-29T08:07:15.066 INFO:tasks.workunit.client.1.smithi148.stdout:5/360: sync 2024-03-29T08:07:15.066 INFO:tasks.workunit.client.1.smithi148.stdout:4/329: sync 2024-03-29T08:07:15.067 INFO:tasks.workunit.client.1.smithi148.stdout:5/361: chown d0/f4b 196043 1 2024-03-29T08:07:15.068 INFO:tasks.workunit.client.0.smithi037.stdout:9/460: dread d1/d4/d19/d1e/d4a/f62 [0,4194304] 0 2024-03-29T08:07:15.072 INFO:tasks.workunit.client.1.smithi148.stdout:3/389: unlink de/d24/d33/d39/d46/d30/f78 0 2024-03-29T08:07:15.075 INFO:tasks.workunit.client.1.smithi148.stdout:6/328: symlink d6/d9/l5c 0 2024-03-29T08:07:15.078 INFO:tasks.workunit.client.0.smithi037.stdout:9/461: getdents d1/d4/d19/d1e/d25/d16 0 2024-03-29T08:07:15.078 INFO:tasks.workunit.client.1.smithi148.stdout:6/329: fdatasync d6/d9/d1e/d20/d30/f35 0 2024-03-29T08:07:15.080 INFO:tasks.workunit.client.1.smithi148.stdout:4/330: mkdir df/d71 0 2024-03-29T08:07:15.084 INFO:tasks.workunit.client.1.smithi148.stdout:9/343: sync 2024-03-29T08:07:15.084 INFO:tasks.workunit.client.1.smithi148.stdout:2/350: sync 2024-03-29T08:07:15.084 INFO:tasks.workunit.client.1.smithi148.stdout:7/373: sync 2024-03-29T08:07:15.084 INFO:tasks.workunit.client.1.smithi148.stdout:8/414: sync 2024-03-29T08:07:15.084 INFO:tasks.workunit.client.1.smithi148.stdout:1/321: sync 2024-03-29T08:07:15.086 INFO:tasks.workunit.client.0.smithi037.stdout:9/462: rename d1/d4/d19/d1e/d4a/d5c/d60 to d1/d4/d19/d1e/d4a/d6f/da2 0 2024-03-29T08:07:15.086 INFO:tasks.workunit.client.1.smithi148.stdout:6/330: unlink d6/d9/f2c 0 2024-03-29T08:07:15.088 INFO:tasks.workunit.client.1.smithi148.stdout:1/322: write d3/de/f1f [2125214,41075] 0 2024-03-29T08:07:15.088 INFO:tasks.workunit.client.1.smithi148.stdout:6/331: chown d6/d9/d1e/d20 458764191 1 2024-03-29T08:07:15.091 INFO:tasks.workunit.client.0.smithi037.stdout:9/463: read d1/d4/d19/d1e/d4a/d6f/da2/d23/f39 [3764593,54598] 0 2024-03-29T08:07:15.092 INFO:tasks.workunit.client.1.smithi148.stdout:5/362: rmdir d0/db/d1e/d35/d5d 0 2024-03-29T08:07:15.093 INFO:tasks.workunit.client.1.smithi148.stdout:0/346: truncate db/de/d24/f38 2810075 0 2024-03-29T08:07:15.094 INFO:tasks.workunit.client.0.smithi037.stdout:9/464: mknod d1/d4/d19/d1e/ca3 0 2024-03-29T08:07:15.096 INFO:tasks.workunit.client.1.smithi148.stdout:0/347: stat db/de/df/d2b/d3b/d47 0 2024-03-29T08:07:15.098 INFO:tasks.workunit.client.1.smithi148.stdout:7/374: rename d8/l24 to d8/d14/d45/d4d/d78/l86 0 2024-03-29T08:07:15.099 INFO:tasks.workunit.client.1.smithi148.stdout:6/332: fsync d6/d9/d1e/d22/f4a 0 2024-03-29T08:07:15.099 INFO:tasks.workunit.client.1.smithi148.stdout:0/348: chown db/dc/l15 439309607 1 2024-03-29T08:07:15.099 INFO:tasks.workunit.client.0.smithi037.stdout:9/465: creat d1/d4/d19/d1e/d4a/d6f/d96/fa4 x:0 0 0 2024-03-29T08:07:15.101 INFO:tasks.workunit.client.0.smithi037.stdout:9/466: write d1/f4c [634962,114201] 0 2024-03-29T08:07:15.101 INFO:tasks.workunit.client.1.smithi148.stdout:1/323: unlink d3/de/d25/d31/l3d 0 2024-03-29T08:07:15.102 INFO:tasks.workunit.client.1.smithi148.stdout:3/390: truncate de/d24/f4d 307745 0 2024-03-29T08:07:15.102 INFO:tasks.workunit.client.0.smithi037.stdout:9/467: chown d1/d1f/f5f 124330184 1 2024-03-29T08:07:15.106 INFO:tasks.workunit.client.1.smithi148.stdout:8/415: dwrite d17/d4e/f5d [0,4194304] 0 2024-03-29T08:07:15.106 INFO:tasks.workunit.client.1.smithi148.stdout:2/351: write d7/f2d [3647299,58808] 0 2024-03-29T08:07:15.112 INFO:tasks.workunit.client.0.smithi037.stdout:9/468: link d1/d1f/d3a/c70 d1/d4/d19/d1e/d4a/d6f/da2/d68/ca5 0 2024-03-29T08:07:15.112 INFO:tasks.workunit.client.0.smithi037.stdout:8/270: sync 2024-03-29T08:07:15.113 INFO:tasks.workunit.client.1.smithi148.stdout:0/349: symlink db/de/df/l6c 0 2024-03-29T08:07:15.114 INFO:tasks.workunit.client.1.smithi148.stdout:8/416: write d17/d74/f6e [1692069,66681] 0 2024-03-29T08:07:15.117 INFO:tasks.workunit.client.1.smithi148.stdout:4/331: dread df/d10/d17/f25 [0,4194304] 0 2024-03-29T08:07:15.120 INFO:tasks.workunit.client.1.smithi148.stdout:2/352: creat d7/da/d33/d45/d4d/d2e/d37/f6d x:0 0 0 2024-03-29T08:07:15.125 INFO:tasks.workunit.client.1.smithi148.stdout:6/333: creat d6/d9/d1e/f5d x:0 0 0 2024-03-29T08:07:15.125 INFO:tasks.workunit.client.1.smithi148.stdout:9/344: truncate db/d22/d26/d54/f49 4914887 0 2024-03-29T08:07:15.125 INFO:tasks.workunit.client.1.smithi148.stdout:1/324: link d3/cd d3/d18/c5a 0 2024-03-29T08:07:15.127 INFO:tasks.workunit.client.1.smithi148.stdout:0/350: creat db/de/df/d11/d2d/d3e/f6d x:0 0 0 2024-03-29T08:07:15.127 INFO:tasks.workunit.client.1.smithi148.stdout:5/363: truncate d0/db/f3e 512964 0 2024-03-29T08:07:15.131 INFO:tasks.workunit.client.0.smithi037.stdout:8/271: dwrite d5/d10/f1d [0,4194304] 0 2024-03-29T08:07:15.132 INFO:tasks.workunit.client.1.smithi148.stdout:2/353: rename d7/da/d33/d45/d4d/c14 to d7/da/d33/d45/c6e 0 2024-03-29T08:07:15.133 INFO:tasks.workunit.client.1.smithi148.stdout:6/334: mkdir d6/d9/d1e/d36/d4d/d5e 0 2024-03-29T08:07:15.134 INFO:tasks.workunit.client.0.smithi037.stdout:8/272: chown d5/l18 1452730 1 2024-03-29T08:07:15.136 INFO:tasks.workunit.client.1.smithi148.stdout:8/417: creat d17/d74/d86/f8d x:0 0 0 2024-03-29T08:07:15.136 INFO:tasks.workunit.client.1.smithi148.stdout:1/325: creat d3/de/d25/d31/f5b x:0 0 0 2024-03-29T08:07:15.136 INFO:tasks.workunit.client.1.smithi148.stdout:9/345: mkdir db/d10/d72 0 2024-03-29T08:07:15.136 INFO:tasks.workunit.client.1.smithi148.stdout:4/332: write df/d10/d26/d34/f5c [3375886,17095] 0 2024-03-29T08:07:15.137 INFO:tasks.workunit.client.0.smithi037.stdout:8/273: mknod d5/d10/d45/c59 0 2024-03-29T08:07:15.137 INFO:tasks.workunit.client.0.smithi037.stdout:9/469: dwrite d1/d4/d19/d1e/d4a/d6f/da2/d23/f72 [0,4194304] 0 2024-03-29T08:07:15.140 INFO:tasks.workunit.client.0.smithi037.stdout:8/274: rmdir d5/d25 39 2024-03-29T08:07:15.142 INFO:tasks.workunit.client.0.smithi037.stdout:9/470: mkdir d1/d4/d19/d1e/d71/d89/da6 0 2024-03-29T08:07:15.143 INFO:tasks.workunit.client.1.smithi148.stdout:7/375: dread d8/d14/f55 [0,4194304] 0 2024-03-29T08:07:15.143 INFO:tasks.workunit.client.0.smithi037.stdout:8/275: mknod d5/d10/c5a 0 2024-03-29T08:07:15.146 INFO:tasks.workunit.client.1.smithi148.stdout:3/391: dread f8 [0,4194304] 0 2024-03-29T08:07:15.150 INFO:tasks.workunit.client.0.smithi037.stdout:8/276: creat d5/f5b x:0 0 0 2024-03-29T08:07:15.152 INFO:tasks.workunit.client.1.smithi148.stdout:9/346: mkdir db/d10/d6d/d73 0 2024-03-29T08:07:15.153 INFO:tasks.workunit.client.0.smithi037.stdout:9/471: creat d1/d4/d19/d1e/d4a/fa7 x:0 0 0 2024-03-29T08:07:15.158 INFO:tasks.workunit.client.0.smithi037.stdout:8/277: getdents d5/d10/d45/d4b 0 2024-03-29T08:07:15.164 INFO:tasks.workunit.client.0.smithi037.stdout:8/278: truncate d5/d25/f28 4978526 0 2024-03-29T08:07:15.166 INFO:tasks.workunit.client.0.smithi037.stdout:8/279: chown d5/d25/f28 0 1 2024-03-29T08:07:15.166 INFO:tasks.workunit.client.1.smithi148.stdout:9/347: unlink db/d10/f29 0 2024-03-29T08:07:15.168 INFO:tasks.workunit.client.1.smithi148.stdout:6/335: dwrite d6/d9/d1e/d20/d30/f47 [0,4194304] 0 2024-03-29T08:07:15.170 INFO:tasks.workunit.client.1.smithi148.stdout:7/376: rename d8/d38/f43 to d8/d29/d44/d50/f87 0 2024-03-29T08:07:15.180 INFO:tasks.workunit.client.1.smithi148.stdout:9/348: dread - db/f59 zero size 2024-03-29T08:07:15.181 INFO:tasks.workunit.client.1.smithi148.stdout:1/326: dwrite d3/de/f1f [8388608,4194304] 0 2024-03-29T08:07:15.182 INFO:tasks.workunit.client.1.smithi148.stdout:2/354: dwrite d7/da/d33/d3c/f65 [0,4194304] 0 2024-03-29T08:07:15.183 INFO:tasks.workunit.client.1.smithi148.stdout:8/418: truncate d17/d18/d26/f35 3815433 0 2024-03-29T08:07:15.186 INFO:tasks.workunit.client.1.smithi148.stdout:3/392: rename de/d3b/d49/f77 to de/d24/d33/d39/d71/f87 0 2024-03-29T08:07:15.186 INFO:tasks.workunit.client.1.smithi148.stdout:4/333: truncate df/d10/d26/d34/f61 3807439 0 2024-03-29T08:07:15.188 INFO:tasks.workunit.client.0.smithi037.stdout:8/280: dwrite d5/d10/d19/f2f [0,4194304] 0 2024-03-29T08:07:15.190 INFO:tasks.workunit.client.1.smithi148.stdout:5/364: dwrite d0/db/d1e/d36/d41/f63 [4194304,4194304] 0 2024-03-29T08:07:15.192 INFO:tasks.workunit.client.1.smithi148.stdout:2/355: read - d7/da/d33/d45/f5f zero size 2024-03-29T08:07:15.193 INFO:tasks.workunit.client.0.smithi037.stdout:8/281: read d5/f16 [129033,72586] 0 2024-03-29T08:07:15.194 INFO:tasks.workunit.client.1.smithi148.stdout:0/351: dread db/de/df/d11/f1b [0,4194304] 0 2024-03-29T08:07:15.198 INFO:tasks.workunit.client.1.smithi148.stdout:8/419: read - d17/d1d/d7e/f83 zero size 2024-03-29T08:07:15.199 INFO:tasks.workunit.client.1.smithi148.stdout:2/356: write d7/da/d33/d3c/d21/f25 [5202157,119908] 0 2024-03-29T08:07:15.204 INFO:tasks.workunit.client.1.smithi148.stdout:1/327: symlink d3/d18/d4d/l5c 0 2024-03-29T08:07:15.205 INFO:tasks.workunit.client.1.smithi148.stdout:7/377: unlink d8/d29/d44/d50/l53 0 2024-03-29T08:07:15.206 INFO:tasks.workunit.client.1.smithi148.stdout:9/349: write db/f43 [2456079,80443] 0 2024-03-29T08:07:15.210 INFO:tasks.workunit.client.1.smithi148.stdout:3/393: unlink de/d24/d33/d39/d46/d70/l75 0 2024-03-29T08:07:15.211 INFO:tasks.workunit.client.1.smithi148.stdout:6/336: dwrite d6/d9/d1e/d22/f4e [0,4194304] 0 2024-03-29T08:07:15.212 INFO:tasks.workunit.client.1.smithi148.stdout:0/352: creat db/de/df/d11/d42/f6e x:0 0 0 2024-03-29T08:07:15.214 INFO:tasks.workunit.client.1.smithi148.stdout:3/394: readlink de/d24/d33/d39/l48 0 2024-03-29T08:07:15.215 INFO:tasks.workunit.client.1.smithi148.stdout:7/378: chown d8/d29/f54 1935755 1 2024-03-29T08:07:15.215 INFO:tasks.workunit.client.0.smithi037.stdout:9/472: dread d1/d4/d19/d1e/d4a/d6f/da2/d13/f17 [0,4194304] 0 2024-03-29T08:07:15.217 INFO:tasks.workunit.client.1.smithi148.stdout:4/334: creat df/d71/f72 x:0 0 0 2024-03-29T08:07:15.218 INFO:tasks.workunit.client.1.smithi148.stdout:5/365: rename d0/db/d1e/d35/d42/d55 to d0/db/d1e/d35/d42/d67/d76 0 2024-03-29T08:07:15.219 INFO:tasks.workunit.client.0.smithi037.stdout:9/473: write d1/d3/f21 [54363,114284] 0 2024-03-29T08:07:15.220 INFO:tasks.workunit.client.1.smithi148.stdout:7/379: fsync d8/d29/d44/d49/f62 0 2024-03-29T08:07:15.220 INFO:tasks.workunit.client.1.smithi148.stdout:9/350: unlink db/d10/f46 0 2024-03-29T08:07:15.223 INFO:tasks.workunit.client.1.smithi148.stdout:2/357: symlink d7/da/d33/d45/d4d/d2e/l6f 0 2024-03-29T08:07:15.224 INFO:tasks.workunit.client.1.smithi148.stdout:6/337: write d6/d9/d1e/f26 [3619318,121345] 0 2024-03-29T08:07:15.231 INFO:tasks.workunit.client.0.smithi037.stdout:8/282: dwrite d5/d10/f23 [0,4194304] 0 2024-03-29T08:07:15.234 INFO:tasks.workunit.client.0.smithi037.stdout:8/283: write d5/f13 [750140,37419] 0 2024-03-29T08:07:15.236 INFO:tasks.workunit.client.1.smithi148.stdout:8/420: dwrite d17/d74/d39/d6f/f45 [0,4194304] 0 2024-03-29T08:07:15.236 INFO:tasks.workunit.client.1.smithi148.stdout:5/366: write d0/db/de/d16/d2e/f68 [47942,92639] 0 2024-03-29T08:07:15.236 INFO:tasks.workunit.client.1.smithi148.stdout:3/395: creat de/d24/d33/d39/d46/d30/d37/f88 x:0 0 0 2024-03-29T08:07:15.238 INFO:tasks.workunit.client.0.smithi037.stdout:9/474: dread d1/d4/d19/d1e/d4a/d6f/da2/f3d [0,4194304] 0 2024-03-29T08:07:15.240 INFO:tasks.workunit.client.1.smithi148.stdout:4/335: mkdir df/d10/d17/d73 0 2024-03-29T08:07:15.240 INFO:tasks.workunit.client.0.smithi037.stdout:9/475: dread - d1/d4/d19/d69/f9c zero size 2024-03-29T08:07:15.240 INFO:tasks.workunit.client.0.smithi037.stdout:8/284: rename d5/l18 to d5/d10/d2d/d34/d50/l5c 0 2024-03-29T08:07:15.242 INFO:tasks.workunit.client.0.smithi037.stdout:8/285: dread - d5/d10/d19/f43 zero size 2024-03-29T08:07:15.244 INFO:tasks.workunit.client.0.smithi037.stdout:8/286: chown d5/d25/d46 106089761 1 2024-03-29T08:07:15.244 INFO:tasks.workunit.client.0.smithi037.stdout:9/476: write d1/d4/d19/d1e/d25/d16/f9e [874950,120277] 0 2024-03-29T08:07:15.244 INFO:tasks.workunit.client.1.smithi148.stdout:6/338: rmdir d6/d9/d1e/d20 39 2024-03-29T08:07:15.246 INFO:tasks.workunit.client.0.smithi037.stdout:8/287: fdatasync d5/d10/f11 0 2024-03-29T08:07:15.247 INFO:tasks.workunit.client.0.smithi037.stdout:8/288: stat d5/c37 0 2024-03-29T08:07:15.249 INFO:tasks.workunit.client.1.smithi148.stdout:7/380: rename d8/d14/d45/d4d/d78/l80 to d8/d38/d67/l88 0 2024-03-29T08:07:15.250 INFO:tasks.workunit.client.0.smithi037.stdout:8/289: chown d5/d10/d45/d4b 800832 1 2024-03-29T08:07:15.250 INFO:tasks.workunit.client.0.smithi037.stdout:9/477: symlink d1/d4/d19/d1e/d4a/d5c/la8 0 2024-03-29T08:07:15.254 INFO:tasks.workunit.client.1.smithi148.stdout:8/421: creat d17/d18/d26/d63/d66/d5f/f8e x:0 0 0 2024-03-29T08:07:15.256 INFO:tasks.workunit.client.1.smithi148.stdout:0/353: write db/dc/f59 [4683938,128380] 0 2024-03-29T08:07:15.257 INFO:tasks.workunit.client.0.smithi037.stdout:8/290: rename d5/f33 to d5/d10/d19/d3d/f5d 0 2024-03-29T08:07:15.258 INFO:tasks.workunit.client.1.smithi148.stdout:2/358: mkdir d7/da/d33/d3c/d54/d70 0 2024-03-29T08:07:15.260 INFO:tasks.workunit.client.0.smithi037.stdout:9/478: write d1/d4/d19/d1e/d4a/d6f/da2/d23/f37 [620368,46864] 0 2024-03-29T08:07:15.261 INFO:tasks.workunit.client.0.smithi037.stdout:8/291: creat d5/d10/d2d/d34/d50/f5e x:0 0 0 2024-03-29T08:07:15.262 INFO:tasks.workunit.client.1.smithi148.stdout:4/336: rename df/d10/c1b to df/d10/d17/c74 0 2024-03-29T08:07:15.264 INFO:tasks.workunit.client.1.smithi148.stdout:4/337: readlink df/l23 0 2024-03-29T08:07:15.264 INFO:tasks.workunit.client.1.smithi148.stdout:1/328: truncate d3/de/f1f 9819491 0 2024-03-29T08:07:15.264 INFO:tasks.workunit.client.0.smithi037.stdout:8/292: truncate d5/f13 5225984 0 2024-03-29T08:07:15.266 INFO:tasks.workunit.client.1.smithi148.stdout:5/367: dwrite d0/db/f34 [0,4194304] 0 2024-03-29T08:07:15.266 INFO:tasks.workunit.client.1.smithi148.stdout:0/354: mknod db/de/d24/c6f 0 2024-03-29T08:07:15.268 INFO:tasks.workunit.client.0.smithi037.stdout:9/479: mknod d1/d4/d19/d1e/d4a/d6f/d96/ca9 0 2024-03-29T08:07:15.269 INFO:tasks.workunit.client.1.smithi148.stdout:9/351: mkdir db/d22/d26/d54/d4d/d35/d4b/d5a/d74 0 2024-03-29T08:07:15.274 INFO:tasks.workunit.client.1.smithi148.stdout:1/329: creat d3/f5d x:0 0 0 2024-03-29T08:07:15.276 INFO:tasks.workunit.client.0.smithi037.stdout:8/293: creat d5/d25/f5f x:0 0 0 2024-03-29T08:07:15.278 INFO:tasks.workunit.client.0.smithi037.stdout:8/294: dread - d5/d25/f5f zero size 2024-03-29T08:07:15.280 INFO:tasks.workunit.client.1.smithi148.stdout:9/352: stat db/d10/d2d/d4e/d5f/l66 0 2024-03-29T08:07:15.281 INFO:tasks.workunit.client.1.smithi148.stdout:1/330: write d3/de/d25/d35/f48 [377474,26874] 0 2024-03-29T08:07:15.281 INFO:tasks.workunit.client.1.smithi148.stdout:2/359: dwrite d7/da/d33/d45/d4d/d2e/f50 [4194304,4194304] 0 2024-03-29T08:07:15.282 INFO:tasks.workunit.client.0.smithi037.stdout:8/295: truncate d5/d10/d19/f2f 4799906 0 2024-03-29T08:07:15.289 INFO:tasks.workunit.client.1.smithi148.stdout:7/381: link d8/l56 d8/d38/l89 0 2024-03-29T08:07:15.290 INFO:tasks.workunit.client.1.smithi148.stdout:3/396: truncate de/d24/f56 3466417 0 2024-03-29T08:07:15.292 INFO:tasks.workunit.client.0.smithi037.stdout:9/480: mkdir d1/d4/d19/d1e/d4a/d6f/da2/d31/daa 0 2024-03-29T08:07:15.293 INFO:tasks.workunit.client.0.smithi037.stdout:8/296: creat d5/d10/d2d/f60 x:0 0 0 2024-03-29T08:07:15.293 INFO:tasks.workunit.client.1.smithi148.stdout:4/338: dwrite df/d10/d17/f29 [0,4194304] 0 2024-03-29T08:07:15.295 INFO:tasks.workunit.client.0.smithi037.stdout:9/481: write d1/d3/f21 [1551836,345] 0 2024-03-29T08:07:15.297 INFO:tasks.workunit.client.0.smithi037.stdout:9/482: dread - d1/d3/d44/f8c zero size 2024-03-29T08:07:15.298 INFO:tasks.workunit.client.1.smithi148.stdout:9/353: dread db/f5c [0,4194304] 0 2024-03-29T08:07:15.299 INFO:tasks.workunit.client.1.smithi148.stdout:0/355: creat db/de/f70 x:0 0 0 2024-03-29T08:07:15.300 INFO:tasks.workunit.client.0.smithi037.stdout:8/297: creat d5/d10/d19/d3d/d4a/f61 x:0 0 0 2024-03-29T08:07:15.301 INFO:tasks.workunit.client.1.smithi148.stdout:2/360: mknod d7/da/d33/d45/d4d/d2e/c71 0 2024-03-29T08:07:15.302 INFO:tasks.workunit.client.1.smithi148.stdout:5/368: link d0/l49 d0/db/d1e/l77 0 2024-03-29T08:07:15.304 INFO:tasks.workunit.client.1.smithi148.stdout:7/382: mknod d8/d29/d44/d49/c8a 0 2024-03-29T08:07:15.304 INFO:tasks.workunit.client.1.smithi148.stdout:5/369: dread - d0/db/d1e/d35/f5c zero size 2024-03-29T08:07:15.305 INFO:tasks.workunit.client.0.smithi037.stdout:8/298: creat d5/d10/d2d/f62 x:0 0 0 2024-03-29T08:07:15.305 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:07:15 smithi148 ceph-mon[44992]: pgmap v39: 65 pgs: 65 active+clean; 1.3 GiB data, 5.4 GiB used, 531 GiB / 536 GiB avail; 22 MiB/s rd, 180 MiB/s wr, 373 op/s 2024-03-29T08:07:15.306 INFO:tasks.workunit.client.0.smithi037.stdout:9/483: symlink d1/d4/d19/d1e/d4a/d6f/da2/d68/lab 0 2024-03-29T08:07:15.306 INFO:tasks.workunit.client.1.smithi148.stdout:3/397: symlink de/d24/d33/d39/d46/d70/l89 0 2024-03-29T08:07:15.309 INFO:tasks.workunit.client.1.smithi148.stdout:4/339: rename df/f1f to df/d10/d26/d34/d3e/f75 0 2024-03-29T08:07:15.309 INFO:tasks.workunit.client.0.smithi037.stdout:8/299: fsync d5/d10/d2d/d34/d50/f54 0 2024-03-29T08:07:15.310 INFO:tasks.workunit.client.1.smithi148.stdout:3/398: chown de/d24/d33/d39/d46/d30/d37/d79 6988185 1 2024-03-29T08:07:15.312 INFO:tasks.workunit.client.0.smithi037.stdout:8/300: dread - d5/d10/d19/d3d/d4a/f61 zero size 2024-03-29T08:07:15.316 INFO:tasks.workunit.client.1.smithi148.stdout:4/340: read df/f45 [3349041,23124] 0 2024-03-29T08:07:15.316 INFO:tasks.workunit.client.1.smithi148.stdout:0/356: dwrite db/de/df/d2b/d3b/d47/f2c [0,4194304] 0 2024-03-29T08:07:15.316 INFO:tasks.workunit.client.0.smithi037.stdout:8/301: write d5/d10/d2d/f62 [971878,57947] 0 2024-03-29T08:07:15.326 INFO:tasks.workunit.client.1.smithi148.stdout:2/361: rmdir d7/da/d52 39 2024-03-29T08:07:15.329 INFO:tasks.workunit.client.0.smithi037.stdout:9/484: dwrite d1/d4/d19/d1e/d71/f94 [0,4194304] 0 2024-03-29T08:07:15.330 INFO:tasks.workunit.client.1.smithi148.stdout:9/354: creat db/d22/d26/d54/d4d/d35/d4b/d5a/d74/f75 x:0 0 0 2024-03-29T08:07:15.330 INFO:tasks.workunit.client.1.smithi148.stdout:5/370: symlink d0/db/d1e/d5b/d66/d73/l78 0 2024-03-29T08:07:15.332 INFO:tasks.workunit.client.0.smithi037.stdout:9/485: write d1/d4/d19/d1e/d4a/fa7 [251931,34220] 0 2024-03-29T08:07:15.334 INFO:tasks.workunit.client.1.smithi148.stdout:3/399: getdents de/d24/d81 0 2024-03-29T08:07:15.335 INFO:tasks.workunit.client.1.smithi148.stdout:1/331: getdents d3/de/d25/d31 0 2024-03-29T08:07:15.338 INFO:tasks.workunit.client.1.smithi148.stdout:4/341: symlink df/l76 0 2024-03-29T08:07:15.339 INFO:tasks.workunit.client.0.smithi037.stdout:8/302: dwrite d5/d10/d2d/f60 [0,4194304] 0 2024-03-29T08:07:15.340 INFO:tasks.workunit.client.1.smithi148.stdout:3/400: chown f8 4305 1 2024-03-29T08:07:15.342 INFO:tasks.workunit.client.1.smithi148.stdout:1/332: mkdir d3/de/d21/d5e 0 2024-03-29T08:07:15.345 INFO:tasks.workunit.client.1.smithi148.stdout:0/357: creat db/de/df/d11/f71 x:0 0 0 2024-03-29T08:07:15.346 INFO:tasks.workunit.client.0.smithi037.stdout:8/303: mknod d5/d10/d19/d3d/c63 0 2024-03-29T08:07:15.348 INFO:tasks.workunit.client.1.smithi148.stdout:1/333: stat d3/de/d25/d35/f3c 0 2024-03-29T08:07:15.349 INFO:tasks.workunit.client.1.smithi148.stdout:4/342: rename df/d10 to df/d2c/d51/d77 0 2024-03-29T08:07:15.354 INFO:tasks.workunit.client.1.smithi148.stdout:2/362: dwrite d7/da/d33/d45/d4d/f47 [4194304,4194304] 0 2024-03-29T08:07:15.354 INFO:tasks.workunit.client.1.smithi148.stdout:9/355: dwrite db/d10/d2d/d4e/f6b [0,4194304] 0 2024-03-29T08:07:15.355 INFO:tasks.workunit.client.1.smithi148.stdout:0/358: creat db/de/df/d2b/f72 x:0 0 0 2024-03-29T08:07:15.356 INFO:tasks.workunit.client.0.smithi037.stdout:9/486: dwrite d1/d1f/d3a/f46 [4194304,4194304] 0 2024-03-29T08:07:15.360 INFO:tasks.workunit.client.1.smithi148.stdout:1/334: mkdir d3/de/d21/d5f 0 2024-03-29T08:07:15.363 INFO:tasks.workunit.client.1.smithi148.stdout:2/363: read d7/da/d33/d45/d4d/f47 [3917363,114137] 0 2024-03-29T08:07:15.366 INFO:tasks.workunit.client.0.smithi037.stdout:8/304: write d5/fb [849641,69942] 0 2024-03-29T08:07:15.367 INFO:tasks.workunit.client.1.smithi148.stdout:3/401: rename de/d24/d33/d39/d71/f87 to de/d3b/d49/f8a 0 2024-03-29T08:07:15.367 INFO:tasks.workunit.client.1.smithi148.stdout:2/364: readlink d7/da/d33/d45/d4d/d2e/l6f 0 2024-03-29T08:07:15.368 INFO:tasks.workunit.client.0.smithi037.stdout:8/305: fdatasync d5/f48 0 2024-03-29T08:07:15.371 INFO:tasks.workunit.client.0.smithi037.stdout:8/306: stat d5/d10/d19/d3d/d4a/d4c 0 2024-03-29T08:07:15.372 INFO:tasks.workunit.client.1.smithi148.stdout:2/365: dread - d7/da/d33/d45/f5f zero size 2024-03-29T08:07:15.372 INFO:tasks.workunit.client.1.smithi148.stdout:9/356: chown db/l16 4 1 2024-03-29T08:07:15.374 INFO:tasks.workunit.client.1.smithi148.stdout:9/357: write f3 [4387210,65136] 0 2024-03-29T08:07:15.377 INFO:tasks.workunit.client.0.smithi037.stdout:8/307: rename d5/d10/d19/d3d/d4a/d4c/l4f to d5/d10/d45/l64 0 2024-03-29T08:07:15.378 INFO:tasks.workunit.client.1.smithi148.stdout:0/359: dwrite db/de/f70 [0,4194304] 0 2024-03-29T08:07:15.378 INFO:tasks.workunit.client.1.smithi148.stdout:5/371: truncate d0/db/de/f53 2245987 0 2024-03-29T08:07:15.380 INFO:tasks.workunit.client.1.smithi148.stdout:3/402: creat de/d24/d33/d39/f8b x:0 0 0 2024-03-29T08:07:15.380 INFO:tasks.workunit.client.0.smithi037.stdout:8/308: rmdir d5/d10/d19/d3d 39 2024-03-29T08:07:15.384 INFO:tasks.workunit.client.0.smithi037.stdout:8/309: unlink d5/d10/d2d/f62 0 2024-03-29T08:07:15.386 INFO:tasks.workunit.client.1.smithi148.stdout:4/343: getdents df/d2c/d51/d77 0 2024-03-29T08:07:15.386 INFO:tasks.workunit.client.1.smithi148.stdout:5/372: mkdir d0/db/d1e/d79 0 2024-03-29T08:07:15.389 INFO:tasks.workunit.client.1.smithi148.stdout:4/344: readlink df/d2d/l32 0 2024-03-29T08:07:15.390 INFO:tasks.workunit.client.1.smithi148.stdout:9/358: mknod db/d10/c76 0 2024-03-29T08:07:15.391 INFO:tasks.workunit.client.0.smithi037.stdout:8/310: creat d5/d10/d19/d3d/d4a/d4c/f65 x:0 0 0 2024-03-29T08:07:15.391 INFO:tasks.workunit.client.1.smithi148.stdout:3/403: symlink de/d24/d33/d39/d46/d70/l8c 0 2024-03-29T08:07:15.392 INFO:tasks.workunit.client.1.smithi148.stdout:2/366: rename d7/da/d4c/c51 to d7/da/d4c/c72 0 2024-03-29T08:07:15.394 INFO:tasks.workunit.client.1.smithi148.stdout:9/359: readlink db/d10/d2d/d4e/d5f/l66 0 2024-03-29T08:07:15.394 INFO:tasks.workunit.client.1.smithi148.stdout:6/339: sync 2024-03-29T08:07:15.397 INFO:tasks.workunit.client.1.smithi148.stdout:4/345: chown df/d2c/d51/d77/d26/l2b 1 1 2024-03-29T08:07:15.402 INFO:tasks.workunit.client.1.smithi148.stdout:2/367: creat d7/da/d33/d3c/d21/f73 x:0 0 0 2024-03-29T08:07:15.402 INFO:tasks.workunit.client.1.smithi148.stdout:4/346: chown df/d2c/d51/d77/f6e 1705436 1 2024-03-29T08:07:15.403 INFO:tasks.workunit.client.1.smithi148.stdout:4/347: readlink df/l13 0 2024-03-29T08:07:15.403 INFO:tasks.workunit.client.1.smithi148.stdout:0/360: dread db/de/df/d11/d42/f67 [0,4194304] 0 2024-03-29T08:07:15.404 INFO:tasks.workunit.client.1.smithi148.stdout:2/368: stat d7/da/d33/d45/d4d/f2a 0 2024-03-29T08:07:15.404 INFO:tasks.workunit.client.1.smithi148.stdout:9/360: fsync db/d10/d2d/f68 0 2024-03-29T08:07:15.406 INFO:tasks.workunit.client.0.smithi037.stdout:8/311: dwrite d5/d10/d19/d3d/f5d [4194304,4194304] 0 2024-03-29T08:07:15.407 INFO:tasks.workunit.client.1.smithi148.stdout:4/348: readlink df/l2a 0 2024-03-29T08:07:15.408 INFO:tasks.workunit.client.1.smithi148.stdout:2/369: chown l6 41311 1 2024-03-29T08:07:15.408 INFO:tasks.workunit.client.0.smithi037.stdout:8/312: dread - d5/d10/d2d/d34/d50/f54 zero size 2024-03-29T08:07:15.410 INFO:tasks.workunit.client.1.smithi148.stdout:9/361: dread - db/d22/d26/d54/f3c zero size 2024-03-29T08:07:15.411 INFO:tasks.workunit.client.0.smithi037.stdout:8/313: write d5/d10/d19/f2f [622067,63709] 0 2024-03-29T08:07:15.411 INFO:tasks.workunit.client.1.smithi148.stdout:6/340: rename d6/l1c to d6/d9/d1e/d22/l5f 0 2024-03-29T08:07:15.413 INFO:tasks.workunit.client.1.smithi148.stdout:1/335: dread d3/f30 [0,4194304] 0 2024-03-29T08:07:15.414 INFO:tasks.workunit.client.0.smithi037.stdout:8/314: read - d5/d10/d2d/f40 zero size 2024-03-29T08:07:15.414 INFO:tasks.workunit.client.1.smithi148.stdout:0/361: write db/de/df/d11/d42/f48 [234613,110295] 0 2024-03-29T08:07:15.414 INFO:tasks.workunit.client.1.smithi148.stdout:2/370: write d7/f10 [1902004,2868] 0 2024-03-29T08:07:15.417 INFO:tasks.workunit.client.1.smithi148.stdout:4/349: mknod df/d60/c78 0 2024-03-29T08:07:15.421 INFO:tasks.workunit.client.1.smithi148.stdout:6/341: dread - d6/d9/d1e/d36/d4d/f57 zero size 2024-03-29T08:07:15.425 INFO:tasks.workunit.client.1.smithi148.stdout:2/371: write d7/f10 [797668,76498] 0 2024-03-29T08:07:15.426 INFO:tasks.workunit.client.1.smithi148.stdout:0/362: dread - db/de/df/d2b/f53 zero size 2024-03-29T08:07:15.426 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:07:15 smithi037 ceph-mon[35162]: pgmap v39: 65 pgs: 65 active+clean; 1.3 GiB data, 5.4 GiB used, 531 GiB / 536 GiB avail; 22 MiB/s rd, 180 MiB/s wr, 373 op/s 2024-03-29T08:07:15.429 INFO:tasks.workunit.client.0.smithi037.stdout:8/315: dwrite d5/d10/d2d/f60 [0,4194304] 0 2024-03-29T08:07:15.430 INFO:tasks.workunit.client.1.smithi148.stdout:1/336: rename d3/de/d1d/l24 to d3/de/d21/l60 0 2024-03-29T08:07:15.431 INFO:tasks.workunit.client.1.smithi148.stdout:5/373: dwrite d0/db/de/d16/f27 [0,4194304] 0 2024-03-29T08:07:15.433 INFO:tasks.workunit.client.1.smithi148.stdout:6/342: creat d6/f60 x:0 0 0 2024-03-29T08:07:15.433 INFO:tasks.workunit.client.0.smithi037.stdout:8/316: creat d5/d25/d46/d4e/f66 x:0 0 0 2024-03-29T08:07:15.434 INFO:tasks.workunit.client.0.smithi037.stdout:8/317: chown d5/c3f 5735 1 2024-03-29T08:07:15.435 INFO:tasks.workunit.client.1.smithi148.stdout:3/404: dwrite de/d24/f4d [0,4194304] 0 2024-03-29T08:07:15.437 INFO:tasks.workunit.client.1.smithi148.stdout:2/372: mknod d7/da/d33/d45/d4d/c74 0 2024-03-29T08:07:15.437 INFO:tasks.workunit.client.0.smithi037.stdout:8/318: mkdir d5/d10/d19/d3d/d4a/d4c/d67 0 2024-03-29T08:07:15.444 INFO:tasks.workunit.client.1.smithi148.stdout:0/363: fsync db/de/df/d11/f41 0 2024-03-29T08:07:15.447 INFO:tasks.workunit.client.1.smithi148.stdout:3/405: readlink de/d3b/d49/l52 0 2024-03-29T08:07:15.449 INFO:tasks.workunit.client.1.smithi148.stdout:5/374: rename d0/db/de/d16/f1b to d0/db/de/d16/f7a 0 2024-03-29T08:07:15.450 INFO:tasks.workunit.client.1.smithi148.stdout:3/406: dread - de/f80 zero size 2024-03-29T08:07:15.450 INFO:tasks.workunit.client.1.smithi148.stdout:0/364: dread - db/de/df/d2b/f5b zero size 2024-03-29T08:07:15.454 INFO:tasks.workunit.client.1.smithi148.stdout:0/365: chown db/de/df/d11 3 1 2024-03-29T08:07:15.456 INFO:tasks.workunit.client.1.smithi148.stdout:4/350: dwrite df/d2c/d51/d77/d26/d34/d3e/f75 [4194304,4194304] 0 2024-03-29T08:07:15.460 INFO:tasks.workunit.client.1.smithi148.stdout:0/366: write db/de/df/d2b/f34 [681433,77220] 0 2024-03-29T08:07:15.463 INFO:tasks.workunit.client.1.smithi148.stdout:9/362: truncate db/d22/d3f/f42 543420 0 2024-03-29T08:07:15.468 INFO:tasks.workunit.client.1.smithi148.stdout:9/363: chown db/d22/d26/d54/d4d/d35/d4b/d58 0 1 2024-03-29T08:07:15.468 INFO:tasks.workunit.client.1.smithi148.stdout:6/343: dwrite d6/d9/d1e/d20/d30/f35 [4194304,4194304] 0 2024-03-29T08:07:15.469 INFO:tasks.workunit.client.1.smithi148.stdout:0/367: symlink db/de/df/d2b/d3b/d47/d4c/d5a/l73 0 2024-03-29T08:07:15.471 INFO:tasks.workunit.client.1.smithi148.stdout:4/351: link df/l48 df/d2c/d51/d77/d26/d34/d43/l79 0 2024-03-29T08:07:15.474 INFO:tasks.workunit.client.1.smithi148.stdout:0/368: symlink db/de/l74 0 2024-03-29T08:07:15.475 INFO:tasks.workunit.client.1.smithi148.stdout:5/375: rename d0/db/de to d0/db/d7b 0 2024-03-29T08:07:15.477 INFO:tasks.workunit.client.1.smithi148.stdout:6/344: rename d6/l21 to d6/d9/d1e/d27/l61 0 2024-03-29T08:07:15.478 INFO:tasks.workunit.client.1.smithi148.stdout:1/337: truncate d3/d18/f20 320478 0 2024-03-29T08:07:15.481 INFO:tasks.workunit.client.1.smithi148.stdout:0/369: write db/de/df/d11/d42/f6e [432740,17507] 0 2024-03-29T08:07:15.481 INFO:tasks.workunit.client.1.smithi148.stdout:5/376: unlink d0/db/d1e/d36/d41/f51 0 2024-03-29T08:07:15.482 INFO:tasks.workunit.client.1.smithi148.stdout:6/345: chown d6/d9/f1f 71 1 2024-03-29T08:07:15.485 INFO:tasks.workunit.client.1.smithi148.stdout:1/338: fdatasync d3/de/d25/d35/f3c 0 2024-03-29T08:07:15.492 INFO:tasks.workunit.client.1.smithi148.stdout:1/339: mkdir d3/de/d25/d35/d42/d23/d61 0 2024-03-29T08:07:15.492 INFO:tasks.workunit.client.1.smithi148.stdout:7/383: sync 2024-03-29T08:07:15.492 INFO:tasks.workunit.client.1.smithi148.stdout:8/422: sync 2024-03-29T08:07:15.492 INFO:tasks.workunit.client.1.smithi148.stdout:5/377: write d0/db/d7b/d16/d2e/f68 [1047682,116479] 0 2024-03-29T08:07:15.493 INFO:tasks.workunit.client.1.smithi148.stdout:0/370: chown db/de/df/d2b/d3b/d47/d4c/d5a 184819 1 2024-03-29T08:07:15.495 INFO:tasks.workunit.client.1.smithi148.stdout:6/346: symlink d6/d9/d1e/d36/d4d/d5e/l62 0 2024-03-29T08:07:15.496 INFO:tasks.workunit.client.1.smithi148.stdout:8/423: chown d17/d74/f6e 1616795984 1 2024-03-29T08:07:15.499 INFO:tasks.workunit.client.0.smithi037.stdout:8/319: dread d5/d10/f11 [0,4194304] 0 2024-03-29T08:07:15.500 INFO:tasks.workunit.client.1.smithi148.stdout:7/384: readlink d8/d38/d67/l88 0 2024-03-29T08:07:15.501 INFO:tasks.workunit.client.1.smithi148.stdout:9/364: dwrite db/d22/d26/d54/d4d/d35/f5e [0,4194304] 0 2024-03-29T08:07:15.504 INFO:tasks.workunit.client.0.smithi037.stdout:8/320: creat d5/d10/d2d/d34/d50/f68 x:0 0 0 2024-03-29T08:07:15.504 INFO:tasks.workunit.client.1.smithi148.stdout:9/365: read db/f5c [2226915,43613] 0 2024-03-29T08:07:15.505 INFO:tasks.workunit.client.1.smithi148.stdout:5/378: mkdir d0/db/d1e/d36/d7c 0 2024-03-29T08:07:15.505 INFO:tasks.workunit.client.1.smithi148.stdout:8/424: mknod d17/d4e/c8f 0 2024-03-29T08:07:15.506 INFO:tasks.workunit.client.1.smithi148.stdout:4/352: write df/d2c/d51/d77/d26/d34/f61 [1428240,25031] 0 2024-03-29T08:07:15.507 INFO:tasks.workunit.client.1.smithi148.stdout:6/347: symlink d6/d9/d1e/d36/d4d/l63 0 2024-03-29T08:07:15.509 INFO:tasks.workunit.client.1.smithi148.stdout:7/385: creat d8/d29/d44/d50/f8b x:0 0 0 2024-03-29T08:07:15.510 INFO:tasks.workunit.client.1.smithi148.stdout:8/425: readlink d17/d74/d39/d6f/l4c 0 2024-03-29T08:07:15.511 INFO:tasks.workunit.client.1.smithi148.stdout:4/353: write df/d71/f72 [609579,58060] 0 2024-03-29T08:07:15.511 INFO:tasks.workunit.client.0.smithi037.stdout:8/321: link d5/c3f d5/d10/d19/d3d/d4a/c69 0 2024-03-29T08:07:15.512 INFO:tasks.workunit.client.1.smithi148.stdout:1/340: getdents d3/de/d25/d31 0 2024-03-29T08:07:15.515 INFO:tasks.workunit.client.1.smithi148.stdout:1/341: stat d3/f7 0 2024-03-29T08:07:15.515 INFO:tasks.workunit.client.1.smithi148.stdout:8/426: write f14 [4177719,111581] 0 2024-03-29T08:07:15.516 INFO:tasks.workunit.client.0.smithi037.stdout:8/322: rename d5/d10/d45/f49 to d5/d25/d46/d4e/f6a 0 2024-03-29T08:07:15.518 INFO:tasks.workunit.client.1.smithi148.stdout:7/386: write d8/d14/f2c [747587,93461] 0 2024-03-29T08:07:15.518 INFO:tasks.workunit.client.1.smithi148.stdout:8/427: stat d17/d49/f6d 0 2024-03-29T08:07:15.525 INFO:tasks.workunit.client.1.smithi148.stdout:4/354: creat df/d2c/d51/d77/d17/d73/f7a x:0 0 0 2024-03-29T08:07:15.526 INFO:tasks.workunit.client.0.smithi037.stdout:8/323: getdents d5/d10/d45 0 2024-03-29T08:07:15.528 INFO:tasks.workunit.client.1.smithi148.stdout:8/428: read d17/d74/f5a [20664,112755] 0 2024-03-29T08:07:15.529 INFO:tasks.workunit.client.1.smithi148.stdout:1/342: getdents d3/de 0 2024-03-29T08:07:15.531 INFO:tasks.workunit.client.0.smithi037.stdout:8/324: link d5/d10/d19/f2f d5/d10/d19/d3d/d4a/f6b 0 2024-03-29T08:07:15.531 INFO:tasks.workunit.client.1.smithi148.stdout:7/387: creat d8/d29/f8c x:0 0 0 2024-03-29T08:07:15.536 INFO:tasks.workunit.client.1.smithi148.stdout:0/371: dwrite db/de/f4b [4194304,4194304] 0 2024-03-29T08:07:15.537 INFO:tasks.workunit.client.0.smithi037.stdout:8/325: link d5/f17 d5/d10/d45/d4b/f6c 0 2024-03-29T08:07:15.541 INFO:tasks.workunit.client.1.smithi148.stdout:8/429: unlink d17/d74/f81 0 2024-03-29T08:07:15.542 INFO:tasks.workunit.client.1.smithi148.stdout:5/379: dread d0/f2c [0,4194304] 0 2024-03-29T08:07:15.542 INFO:tasks.workunit.client.1.smithi148.stdout:1/343: chown d3/de/d25/d35/d42/d23 1182 1 2024-03-29T08:07:15.544 INFO:tasks.workunit.client.1.smithi148.stdout:9/366: dwrite db/d22/f51 [0,4194304] 0 2024-03-29T08:07:15.549 INFO:tasks.workunit.client.1.smithi148.stdout:6/348: dwrite d6/d9/d1e/d20/d30/f56 [0,4194304] 0 2024-03-29T08:07:15.551 INFO:tasks.workunit.client.1.smithi148.stdout:8/430: rmdir d17/d74/d39/d6f 39 2024-03-29T08:07:15.552 INFO:tasks.workunit.client.1.smithi148.stdout:7/388: dread d8/d60/f70 [0,4194304] 0 2024-03-29T08:07:15.556 INFO:tasks.workunit.client.0.smithi037.stdout:8/326: dwrite d5/d10/d19/d3d/d4a/f6b [4194304,4194304] 0 2024-03-29T08:07:15.560 INFO:tasks.workunit.client.0.smithi037.stdout:8/327: symlink d5/d10/d2d/d34/l6d 0 2024-03-29T08:07:15.568 INFO:tasks.workunit.client.1.smithi148.stdout:3/407: sync 2024-03-29T08:07:15.570 INFO:tasks.workunit.client.0.smithi037.stdout:0/172: sync 2024-03-29T08:07:15.571 INFO:tasks.workunit.client.0.smithi037.stdout:6/222: sync 2024-03-29T08:07:15.571 INFO:tasks.workunit.client.1.smithi148.stdout:2/373: sync 2024-03-29T08:07:15.573 INFO:tasks.workunit.client.1.smithi148.stdout:9/367: dwrite db/d22/d26/d54/f36 [0,4194304] 0 2024-03-29T08:07:15.575 INFO:tasks.workunit.client.0.smithi037.stdout:6/223: unlink d0/c4b 0 2024-03-29T08:07:15.578 INFO:tasks.workunit.client.1.smithi148.stdout:8/431: rmdir d17/d4e 39 2024-03-29T08:07:15.580 INFO:tasks.workunit.client.1.smithi148.stdout:6/349: creat d6/d9/d1e/d27/f64 x:0 0 0 2024-03-29T08:07:15.583 INFO:tasks.workunit.client.0.smithi037.stdout:6/224: rmdir d0/d5/d1a 39 2024-03-29T08:07:15.584 INFO:tasks.workunit.client.1.smithi148.stdout:2/374: creat d7/da/d33/d3c/d54/f75 x:0 0 0 2024-03-29T08:07:15.585 INFO:tasks.workunit.client.0.smithi037.stdout:8/328: dread d5/d10/d19/d3d/d4a/f6b [4194304,4194304] 0 2024-03-29T08:07:15.590 INFO:tasks.workunit.client.1.smithi148.stdout:8/432: readlink d17/d74/d39/d6f/l61 0 2024-03-29T08:07:15.590 INFO:tasks.workunit.client.1.smithi148.stdout:0/372: dwrite f9 [0,4194304] 0 2024-03-29T08:07:15.593 INFO:tasks.workunit.client.1.smithi148.stdout:3/408: unlink de/d24/f2b 0 2024-03-29T08:07:15.595 INFO:tasks.workunit.client.0.smithi037.stdout:6/225: creat d0/d5/d1f/f4c x:0 0 0 2024-03-29T08:07:15.597 INFO:tasks.workunit.client.1.smithi148.stdout:3/409: fsync de/d24/f4d 0 2024-03-29T08:07:15.600 INFO:tasks.workunit.client.0.smithi037.stdout:6/226: creat d0/d20/f4d x:0 0 0 2024-03-29T08:07:15.603 INFO:tasks.workunit.client.1.smithi148.stdout:4/355: dread df/d2d/f49 [0,4194304] 0 2024-03-29T08:07:15.603 INFO:tasks.workunit.client.1.smithi148.stdout:5/380: getdents d0/db/d1e/d5b/d66 0 2024-03-29T08:07:15.604 INFO:tasks.workunit.client.0.smithi037.stdout:6/227: fsync d0/f33 0 2024-03-29T08:07:15.605 INFO:tasks.workunit.client.1.smithi148.stdout:0/373: mkdir db/de/d75 0 2024-03-29T08:07:15.606 INFO:tasks.workunit.client.0.smithi037.stdout:0/173: dwrite d9/da/dd/f23 [0,4194304] 0 2024-03-29T08:07:15.607 INFO:tasks.workunit.client.1.smithi148.stdout:1/344: truncate d3/d55/f52 2228518 0 2024-03-29T08:07:15.610 INFO:tasks.workunit.client.1.smithi148.stdout:1/345: fsync d3/d18/f39 0 2024-03-29T08:07:15.610 INFO:tasks.workunit.client.0.smithi037.stdout:0/174: mknod d9/da/dd/d20/c32 0 2024-03-29T08:07:15.614 INFO:tasks.workunit.client.1.smithi148.stdout:0/374: creat db/de/df/d2b/f76 x:0 0 0 2024-03-29T08:07:15.614 INFO:tasks.workunit.client.0.smithi037.stdout:0/175: mkdir d9/da/d1b/d33 0 2024-03-29T08:07:15.619 INFO:tasks.workunit.client.0.smithi037.stdout:0/176: creat d9/da/f34 x:0 0 0 2024-03-29T08:07:15.620 INFO:tasks.workunit.client.1.smithi148.stdout:8/433: getdents d17/d49 0 2024-03-29T08:07:15.623 INFO:tasks.workunit.client.1.smithi148.stdout:1/346: mknod d3/de/d25/d35/c62 0 2024-03-29T08:07:15.627 INFO:tasks.workunit.client.1.smithi148.stdout:3/410: write de/d24/d33/d39/d46/f23 [1192146,69388] 0 2024-03-29T08:07:15.629 INFO:tasks.workunit.client.1.smithi148.stdout:7/389: dwrite d8/d14/d19/f39 [0,4194304] 0 2024-03-29T08:07:15.630 INFO:tasks.workunit.client.0.smithi037.stdout:6/228: dwrite d0/d5/f14 [4194304,4194304] 0 2024-03-29T08:07:15.631 INFO:tasks.workunit.client.1.smithi148.stdout:2/375: dread d7/da/f3f [0,4194304] 0 2024-03-29T08:07:15.634 INFO:tasks.workunit.client.1.smithi148.stdout:0/375: link db/de/f62 db/de/df/d2b/f77 0 2024-03-29T08:07:15.635 INFO:tasks.workunit.client.1.smithi148.stdout:5/381: truncate d0/db/f10 1092176 0 2024-03-29T08:07:15.636 INFO:tasks.workunit.client.0.smithi037.stdout:6/229: chown d0/c3e 32208557 1 2024-03-29T08:07:15.639 INFO:tasks.workunit.client.0.smithi037.stdout:6/230: mkdir d0/d19/d34/d4e 0 2024-03-29T08:07:15.644 INFO:tasks.workunit.client.1.smithi148.stdout:0/376: read - db/de/df/d11/d2d/d3e/f6d zero size 2024-03-29T08:07:15.645 INFO:tasks.workunit.client.1.smithi148.stdout:9/368: dread db/d22/d26/d54/f2e [0,4194304] 0 2024-03-29T08:07:15.648 INFO:tasks.workunit.client.1.smithi148.stdout:7/390: unlink d8/c4c 0 2024-03-29T08:07:15.653 INFO:tasks.workunit.client.1.smithi148.stdout:8/434: dwrite d17/d74/d39/f77 [0,4194304] 0 2024-03-29T08:07:15.655 INFO:tasks.workunit.client.1.smithi148.stdout:3/411: chown de/d24/d33/f5c 24 1 2024-03-29T08:07:15.658 INFO:tasks.workunit.client.1.smithi148.stdout:4/356: dread df/f27 [0,4194304] 0 2024-03-29T08:07:15.658 INFO:tasks.workunit.client.1.smithi148.stdout:0/377: mkdir db/dc/d78 0 2024-03-29T08:07:15.659 INFO:tasks.workunit.client.1.smithi148.stdout:5/382: dread d0/db/d7b/f52 [0,4194304] 0 2024-03-29T08:07:15.660 INFO:tasks.workunit.client.1.smithi148.stdout:9/369: creat db/d22/d26/d54/d4d/d35/d4b/d5a/f77 x:0 0 0 2024-03-29T08:07:15.661 INFO:tasks.workunit.client.1.smithi148.stdout:1/347: write d3/de/d25/d35/d42/f15 [4034458,52715] 0 2024-03-29T08:07:15.664 INFO:tasks.workunit.client.1.smithi148.stdout:5/383: fdatasync d0/db/d1e/d36/d41/f58 0 2024-03-29T08:07:15.665 INFO:tasks.workunit.client.1.smithi148.stdout:1/348: chown d3/fa 0 1 2024-03-29T08:07:15.665 INFO:tasks.workunit.client.1.smithi148.stdout:4/357: creat df/d71/f7b x:0 0 0 2024-03-29T08:07:15.669 INFO:tasks.workunit.client.1.smithi148.stdout:1/349: stat d3/de/d25/d35/d42/l14 0 2024-03-29T08:07:15.670 INFO:tasks.workunit.client.1.smithi148.stdout:2/376: link d7/da/l44 d7/d24/d4b/l76 0 2024-03-29T08:07:15.672 INFO:tasks.workunit.client.1.smithi148.stdout:2/377: dread - d7/d24/f60 zero size 2024-03-29T08:07:15.673 INFO:tasks.workunit.client.1.smithi148.stdout:0/378: rmdir db/de/df/d11/d2d/d3e 39 2024-03-29T08:07:15.674 INFO:tasks.workunit.client.0.smithi037.stdout:6/231: dwrite d0/d5/d1a/f23 [4194304,4194304] 0 2024-03-29T08:07:15.677 INFO:tasks.workunit.client.1.smithi148.stdout:4/358: mkdir df/d2d/d7c 0 2024-03-29T08:07:15.678 INFO:tasks.workunit.client.0.smithi037.stdout:6/232: mkdir d0/d4f 0 2024-03-29T08:07:15.679 INFO:tasks.workunit.client.1.smithi148.stdout:5/384: rename d0/db/d1e/d36/d41/l32 to d0/db/d1e/d35/d42/l7d 0 2024-03-29T08:07:15.680 INFO:tasks.workunit.client.1.smithi148.stdout:2/378: write d7/d24/f5d [4827781,82776] 0 2024-03-29T08:07:15.682 INFO:tasks.workunit.client.1.smithi148.stdout:8/435: dwrite d17/d4e/f5d [4194304,4194304] 0 2024-03-29T08:07:15.686 INFO:tasks.workunit.client.0.smithi037.stdout:6/233: read d0/d5/d1a/f2f [1716706,118044] 0 2024-03-29T08:07:15.689 INFO:tasks.workunit.client.0.smithi037.stdout:6/234: dread - d0/d20/f4d zero size 2024-03-29T08:07:15.691 INFO:tasks.workunit.client.1.smithi148.stdout:0/379: read db/de/df/d11/d42/f52 [462558,55748] 0 2024-03-29T08:07:15.692 INFO:tasks.workunit.client.0.smithi037.stdout:6/235: write d0/d5/fb [2432127,28872] 0 2024-03-29T08:07:15.692 INFO:tasks.workunit.client.1.smithi148.stdout:4/359: read df/d71/f72 [406875,42343] 0 2024-03-29T08:07:15.693 INFO:tasks.workunit.client.1.smithi148.stdout:1/350: rmdir d3/de/d25/d51 39 2024-03-29T08:07:15.694 INFO:tasks.workunit.client.1.smithi148.stdout:8/436: dread - d17/d18/d26/d63/d66/d5f/f8e zero size 2024-03-29T08:07:15.694 INFO:tasks.workunit.client.1.smithi148.stdout:7/391: dwrite d8/d14/f2d [0,4194304] 0 2024-03-29T08:07:15.696 INFO:tasks.workunit.client.1.smithi148.stdout:5/385: rmdir d0/db/d7b/d16/d2e 39 2024-03-29T08:07:15.696 INFO:tasks.workunit.client.1.smithi148.stdout:9/370: write db/d22/d3f/f42 [1067743,37689] 0 2024-03-29T08:07:15.696 INFO:tasks.workunit.client.0.smithi037.stdout:6/236: creat d0/d19/d34/f50 x:0 0 0 2024-03-29T08:07:15.697 INFO:tasks.workunit.client.1.smithi148.stdout:4/360: unlink le 0 2024-03-29T08:07:15.699 INFO:tasks.workunit.client.1.smithi148.stdout:2/379: mknod d7/da/d33/d45/d4d/d2e/c77 0 2024-03-29T08:07:15.701 INFO:tasks.workunit.client.1.smithi148.stdout:5/386: creat d0/db/d1e/f7e x:0 0 0 2024-03-29T08:07:15.702 INFO:tasks.workunit.client.1.smithi148.stdout:8/437: mknod d17/d18/d26/d4f/c90 0 2024-03-29T08:07:15.702 INFO:tasks.workunit.client.1.smithi148.stdout:0/380: creat db/dc/d78/f79 x:0 0 0 2024-03-29T08:07:15.703 INFO:tasks.workunit.client.1.smithi148.stdout:4/361: rmdir df 39 2024-03-29T08:07:15.704 INFO:tasks.workunit.client.0.smithi037.stdout:6/237: link d0/d20/c3d d0/d20/d38/c51 0 2024-03-29T08:07:15.706 INFO:tasks.workunit.client.1.smithi148.stdout:9/371: symlink db/d10/d2d/d4e/d5f/l78 0 2024-03-29T08:07:15.706 INFO:tasks.workunit.client.1.smithi148.stdout:1/351: rename d3/d55/f52 to d3/d18/d4d/f63 0 2024-03-29T08:07:15.707 INFO:tasks.workunit.client.0.smithi037.stdout:5/390: sync 2024-03-29T08:07:15.707 INFO:tasks.workunit.client.0.smithi037.stdout:1/347: sync 2024-03-29T08:07:15.707 INFO:tasks.workunit.client.0.smithi037.stdout:6/238: unlink d0/d19/c3b 0 2024-03-29T08:07:15.709 INFO:tasks.workunit.client.1.smithi148.stdout:2/380: mknod d7/da/d33/d3c/c78 0 2024-03-29T08:07:15.710 INFO:tasks.workunit.client.0.smithi037.stdout:1/348: fdatasync d2/d6/dc/d18/f3e 0 2024-03-29T08:07:15.710 INFO:tasks.workunit.client.0.smithi037.stdout:6/239: read - d0/d20/f46 zero size 2024-03-29T08:07:15.712 INFO:tasks.workunit.client.1.smithi148.stdout:8/438: write d17/d74/f5a [572469,44249] 0 2024-03-29T08:07:15.713 INFO:tasks.workunit.client.1.smithi148.stdout:1/352: symlink d3/de/d21/l64 0 2024-03-29T08:07:15.713 INFO:tasks.workunit.client.0.smithi037.stdout:5/391: unlink d8/d34/l55 0 2024-03-29T08:07:15.714 INFO:tasks.workunit.client.1.smithi148.stdout:9/372: creat db/d22/d26/d54/d4d/f79 x:0 0 0 2024-03-29T08:07:15.716 INFO:tasks.workunit.client.1.smithi148.stdout:2/381: creat d7/da/d4c/f79 x:0 0 0 2024-03-29T08:07:15.718 INFO:tasks.workunit.client.0.smithi037.stdout:5/392: symlink d8/d34/d4b/l83 0 2024-03-29T08:07:15.718 INFO:tasks.workunit.client.1.smithi148.stdout:5/387: getdents d0/db/d7b 0 2024-03-29T08:07:15.720 INFO:tasks.workunit.client.1.smithi148.stdout:4/362: creat df/d2c/d51/d77/d26/f7d x:0 0 0 2024-03-29T08:07:15.720 INFO:tasks.workunit.client.1.smithi148.stdout:7/392: dwrite d8/d14/d45/d4d/f6b [0,4194304] 0 2024-03-29T08:07:15.721 INFO:tasks.workunit.client.1.smithi148.stdout:1/353: symlink d3/de/d1d/l65 0 2024-03-29T08:07:15.721 INFO:tasks.workunit.client.0.smithi037.stdout:1/349: rmdir d2/d6/dc/d58/d67/d74 0 2024-03-29T08:07:15.723 INFO:tasks.workunit.client.1.smithi148.stdout:7/393: dread - d8/d29/d44/d50/f52 zero size 2024-03-29T08:07:15.725 INFO:tasks.workunit.client.1.smithi148.stdout:6/350: sync 2024-03-29T08:07:15.725 INFO:tasks.workunit.client.0.smithi037.stdout:1/350: write d2/f31 [3052681,52389] 0 2024-03-29T08:07:15.728 INFO:tasks.workunit.client.1.smithi148.stdout:5/388: truncate d0/db/d7b/f52 904734 0 2024-03-29T08:07:15.728 INFO:tasks.workunit.client.1.smithi148.stdout:0/381: write db/de/f62 [844481,43909] 0 2024-03-29T08:07:15.729 INFO:tasks.workunit.client.0.smithi037.stdout:5/393: dread d8/d22/d4f/d70/f54 [0,4194304] 0 2024-03-29T08:07:15.732 INFO:tasks.workunit.client.0.smithi037.stdout:1/351: rename d2/d6/dc/d58/f6b to d2/d40/f78 0 2024-03-29T08:07:15.732 INFO:tasks.workunit.client.1.smithi148.stdout:1/354: symlink d3/de/d21/d5f/l66 0 2024-03-29T08:07:15.733 INFO:tasks.workunit.client.1.smithi148.stdout:2/382: creat d7/da/d33/d45/d4d/f7a x:0 0 0 2024-03-29T08:07:15.735 INFO:tasks.workunit.client.0.smithi037.stdout:1/352: rmdir d2/d6/dc 39 2024-03-29T08:07:15.736 INFO:tasks.workunit.client.0.smithi037.stdout:1/353: readlink d2/l42 0 2024-03-29T08:07:15.738 INFO:tasks.workunit.client.1.smithi148.stdout:0/382: mkdir db/de/df/d2b/d3b/d47/d4c/d5a/d7a 0 2024-03-29T08:07:15.739 INFO:tasks.workunit.client.1.smithi148.stdout:1/355: truncate d3/de/d25/d51/f54 963019 0 2024-03-29T08:07:15.740 INFO:tasks.workunit.client.1.smithi148.stdout:2/383: creat d7/da/d33/d3c/d54/d70/f7b x:0 0 0 2024-03-29T08:07:15.744 INFO:tasks.workunit.client.1.smithi148.stdout:9/373: dwrite db/d22/d26/f2a [0,4194304] 0 2024-03-29T08:07:15.744 INFO:tasks.workunit.client.1.smithi148.stdout:0/383: rmdir db/de/df/d14 39 2024-03-29T08:07:15.749 INFO:tasks.workunit.client.1.smithi148.stdout:6/351: dwrite d6/d9/d1e/d27/f44 [0,4194304] 0 2024-03-29T08:07:15.750 INFO:tasks.workunit.client.1.smithi148.stdout:8/439: truncate d17/d4e/f5d 6922898 0 2024-03-29T08:07:15.750 INFO:tasks.workunit.client.0.smithi037.stdout:5/394: dwrite d8/d18/d24/f44 [0,4194304] 0 2024-03-29T08:07:15.751 INFO:tasks.workunit.client.0.smithi037.stdout:1/354: truncate d2/d4b/f4d 573951 0 2024-03-29T08:07:15.751 INFO:tasks.workunit.client.1.smithi148.stdout:8/440: chown d17/d18/d26 53371 1 2024-03-29T08:07:15.753 INFO:tasks.workunit.client.1.smithi148.stdout:1/356: creat d3/de/d25/d31/f67 x:0 0 0 2024-03-29T08:07:15.755 INFO:tasks.workunit.client.1.smithi148.stdout:8/441: dread - d17/d4e/f88 zero size 2024-03-29T08:07:15.756 INFO:tasks.workunit.client.1.smithi148.stdout:5/389: dwrite d0/db/d1e/d35/d6a/f6d [0,4194304] 0 2024-03-29T08:07:15.757 INFO:tasks.workunit.client.0.smithi037.stdout:5/395: stat d8/d22/d4f/d70/l53 0 2024-03-29T08:07:15.760 INFO:tasks.workunit.client.1.smithi148.stdout:6/352: truncate d6/d9/d1e/d36/d4d/f57 856364 0 2024-03-29T08:07:15.760 INFO:tasks.workunit.client.1.smithi148.stdout:1/357: truncate d3/d18/f4b 1528041 0 2024-03-29T08:07:15.762 INFO:tasks.workunit.client.0.smithi037.stdout:1/355: dread - d2/d6/dc/d5d/d64/f68 zero size 2024-03-29T08:07:15.764 INFO:tasks.workunit.client.0.smithi037.stdout:5/396: rename d8/f6e to d8/d18/d24/d45/f84 0 2024-03-29T08:07:15.764 INFO:tasks.workunit.client.1.smithi148.stdout:2/384: rename d7/da/d33/d45/d4d/f2a to d7/f7c 0 2024-03-29T08:07:15.767 INFO:tasks.workunit.client.1.smithi148.stdout:2/385: write d7/f17 [5248852,4657] 0 2024-03-29T08:07:15.771 INFO:tasks.workunit.client.1.smithi148.stdout:1/358: chown d3/d18/c5a 267151015 1 2024-03-29T08:07:15.773 INFO:tasks.workunit.client.1.smithi148.stdout:5/390: getdents d0/db/d1e/d6e 0 2024-03-29T08:07:15.780 INFO:tasks.workunit.client.1.smithi148.stdout:5/391: creat d0/db/d1e/d36/d41/f7f x:0 0 0 2024-03-29T08:07:15.781 INFO:tasks.workunit.client.1.smithi148.stdout:8/442: creat d17/d18/d26/d63/d66/f91 x:0 0 0 2024-03-29T08:07:15.783 INFO:tasks.workunit.client.0.smithi037.stdout:5/397: dread d8/d18/d24/f44 [0,4194304] 0 2024-03-29T08:07:15.784 INFO:tasks.workunit.client.1.smithi148.stdout:8/443: write d17/d1d/f79 [554465,92912] 0 2024-03-29T08:07:15.784 INFO:tasks.workunit.client.1.smithi148.stdout:0/384: rename db/de/df/d2b/f35 to db/f7b 0 2024-03-29T08:07:15.785 INFO:tasks.workunit.client.1.smithi148.stdout:4/363: dread df/f45 [0,4194304] 0 2024-03-29T08:07:15.785 INFO:tasks.workunit.client.1.smithi148.stdout:6/353: getdents d6/d9/d1e/d36 0 2024-03-29T08:07:15.786 INFO:tasks.workunit.client.1.smithi148.stdout:1/359: truncate d3/d18/d4d/f63 273868 0 2024-03-29T08:07:15.787 INFO:tasks.workunit.client.0.smithi037.stdout:5/398: mkdir d8/d22/d4f/d70/d79/d7a/d85 0 2024-03-29T08:07:15.787 INFO:tasks.workunit.client.1.smithi148.stdout:2/386: getdents d7/da 0 2024-03-29T08:07:15.789 INFO:tasks.workunit.client.1.smithi148.stdout:8/444: creat d17/d1d/f92 x:0 0 0 2024-03-29T08:07:15.793 INFO:tasks.workunit.client.1.smithi148.stdout:9/374: rename db/d10/d72 to db/d10/d2d/d4e/d5f/d6e/d7a 0 2024-03-29T08:07:15.794 INFO:tasks.workunit.client.1.smithi148.stdout:8/445: dread - d17/d18/f67 zero size 2024-03-29T08:07:15.795 INFO:tasks.workunit.client.0.smithi037.stdout:5/399: rename d8/d22/d4f/d70/l53 to d8/l86 0 2024-03-29T08:07:15.795 INFO:tasks.workunit.client.1.smithi148.stdout:8/446: stat d17/f1a 0 2024-03-29T08:07:15.796 INFO:tasks.workunit.client.1.smithi148.stdout:1/360: unlink d3/de/d25/d35/f3e 0 2024-03-29T08:07:15.797 INFO:tasks.workunit.client.0.smithi037.stdout:1/356: write d2/d6/f36 [3081780,99546] 0 2024-03-29T08:07:15.798 INFO:tasks.workunit.client.0.smithi037.stdout:5/400: stat d8/d18/d24/l36 0 2024-03-29T08:07:15.801 INFO:tasks.workunit.client.1.smithi148.stdout:4/364: rename df/d2c/d51/d77/l6f to df/d2c/d51/d77/d17/l7e 0 2024-03-29T08:07:15.802 INFO:tasks.workunit.client.0.smithi037.stdout:5/401: unlink d8/f1e 0 2024-03-29T08:07:15.803 INFO:tasks.workunit.client.1.smithi148.stdout:0/385: rmdir db/de/df/d2b 39 2024-03-29T08:07:15.804 INFO:tasks.workunit.client.0.smithi037.stdout:5/402: truncate d8/d34/f3a 1669135 0 2024-03-29T08:07:15.807 INFO:tasks.workunit.client.1.smithi148.stdout:6/354: dwrite d6/d9/d1e/f26 [4194304,4194304] 0 2024-03-29T08:07:15.808 INFO:tasks.workunit.client.0.smithi037.stdout:1/357: getdents d2/d6/dc/d18/d1d/d21/d38 0 2024-03-29T08:07:15.808 INFO:tasks.workunit.client.0.smithi037.stdout:5/403: write d8/f52 [1359131,40371] 0 2024-03-29T08:07:15.810 INFO:tasks.workunit.client.0.smithi037.stdout:1/358: stat d2/d6/dc/d18/d1d/d21/d38 0 2024-03-29T08:07:15.810 INFO:tasks.workunit.client.1.smithi148.stdout:3/412: sync 2024-03-29T08:07:15.812 INFO:tasks.workunit.client.1.smithi148.stdout:1/361: dread d3/f30 [0,4194304] 0 2024-03-29T08:07:15.813 INFO:tasks.workunit.client.0.smithi037.stdout:5/404: symlink d8/d22/d4f/d70/d79/d7a/d85/l87 0 2024-03-29T08:07:15.815 INFO:tasks.workunit.client.1.smithi148.stdout:9/375: creat db/d10/d2d/f7b x:0 0 0 2024-03-29T08:07:15.818 INFO:tasks.workunit.client.1.smithi148.stdout:2/387: getdents d7/da/d33/d45/d4d/d2e/d37 0 2024-03-29T08:07:15.819 INFO:tasks.workunit.client.1.smithi148.stdout:0/386: mkdir db/de/df/d2b/d3b/d47/d7c 0 2024-03-29T08:07:15.820 INFO:tasks.workunit.client.1.smithi148.stdout:0/387: chown db 1935712 1 2024-03-29T08:07:15.820 INFO:tasks.workunit.client.1.smithi148.stdout:5/392: dwrite d0/db/d1e/d36/f47 [0,4194304] 0 2024-03-29T08:07:15.823 INFO:tasks.workunit.client.0.smithi037.stdout:1/359: getdents d2/d6/dc/d5d 0 2024-03-29T08:07:15.823 INFO:tasks.workunit.client.1.smithi148.stdout:3/413: mkdir de/d24/d33/d39/d46/d8d 0 2024-03-29T08:07:15.828 INFO:tasks.workunit.client.1.smithi148.stdout:0/388: chown db/de/df/d2b/d3b/d47/d4c/l5c 0 1 2024-03-29T08:07:15.828 INFO:tasks.workunit.client.1.smithi148.stdout:7/394: sync 2024-03-29T08:07:15.828 INFO:tasks.workunit.client.1.smithi148.stdout:1/362: mknod d3/de/d25/c68 0 2024-03-29T08:07:15.828 INFO:tasks.workunit.client.1.smithi148.stdout:0/389: chown db/de/df/d11/f1b 67 1 2024-03-29T08:07:15.829 INFO:tasks.workunit.client.1.smithi148.stdout:6/355: dwrite d6/d9/d1e/f53 [0,4194304] 0 2024-03-29T08:07:15.831 INFO:tasks.workunit.client.0.smithi037.stdout:5/405: dread d8/f52 [0,4194304] 0 2024-03-29T08:07:15.836 INFO:tasks.workunit.client.0.smithi037.stdout:5/406: fsync d8/d18/f71 0 2024-03-29T08:07:15.841 INFO:tasks.workunit.client.0.smithi037.stdout:5/407: unlink d8/d22/d4f/d70/d79/d7a/f67 0 2024-03-29T08:07:15.845 INFO:tasks.workunit.client.1.smithi148.stdout:2/388: mkdir d7/d24/d4b/d66/d7d 0 2024-03-29T08:07:15.846 INFO:tasks.workunit.client.1.smithi148.stdout:9/376: write db/d10/d2d/f68 [141249,51665] 0 2024-03-29T08:07:15.846 INFO:tasks.workunit.client.1.smithi148.stdout:1/363: rename d3/de/d1d/l2a to d3/de/d1d/l69 0 2024-03-29T08:07:15.851 INFO:tasks.workunit.client.0.smithi037.stdout:5/408: getdents d8/d22/d4f/d70/d79/d7a/d85 0 2024-03-29T08:07:15.853 INFO:tasks.workunit.client.0.smithi037.stdout:5/409: stat d8/d18/d24/f44 0 2024-03-29T08:07:15.859 INFO:tasks.workunit.client.1.smithi148.stdout:5/393: link d0/db/d1e/d36/f47 d0/db/d1e/d36/d7c/f80 0 2024-03-29T08:07:15.860 INFO:tasks.workunit.client.1.smithi148.stdout:0/390: dwrite db/de/df/d11/f1b [4194304,4194304] 0 2024-03-29T08:07:15.862 INFO:tasks.workunit.client.1.smithi148.stdout:6/356: symlink d6/d9/d1e/l65 0 2024-03-29T08:07:15.863 INFO:tasks.workunit.client.1.smithi148.stdout:7/395: dwrite d8/d14/d45/f66 [0,4194304] 0 2024-03-29T08:07:15.865 INFO:tasks.workunit.client.1.smithi148.stdout:2/389: unlink d7/d24/d4b/f59 0 2024-03-29T08:07:15.866 INFO:tasks.workunit.client.0.smithi037.stdout:7/321: sync 2024-03-29T08:07:15.866 INFO:tasks.workunit.client.0.smithi037.stdout:2/183: sync 2024-03-29T08:07:15.869 INFO:tasks.workunit.client.1.smithi148.stdout:7/396: stat d8/d60/f70 0 2024-03-29T08:07:15.872 INFO:tasks.workunit.client.1.smithi148.stdout:4/365: dwrite df/d2c/d51/d77/f1d [0,4194304] 0 2024-03-29T08:07:15.872 INFO:tasks.workunit.client.0.smithi037.stdout:5/410: dwrite d8/d22/d4f/d70/d79/f7b [4194304,4194304] 0 2024-03-29T08:07:15.872 INFO:tasks.workunit.client.0.smithi037.stdout:2/184: mknod d0/d20/d22/c3e 0 2024-03-29T08:07:15.873 INFO:tasks.workunit.client.1.smithi148.stdout:3/414: link de/d24/f36 de/d24/d33/f8e 0 2024-03-29T08:07:15.875 INFO:tasks.workunit.client.1.smithi148.stdout:8/447: dwrite d17/d4e/f5d [4194304,4194304] 0 2024-03-29T08:07:15.876 INFO:tasks.workunit.client.1.smithi148.stdout:9/377: creat db/d22/d26/d54/d4d/d35/d4b/d58/f7c x:0 0 0 2024-03-29T08:07:15.876 INFO:tasks.workunit.client.0.smithi037.stdout:2/185: fdatasync d0/f18 0 2024-03-29T08:07:15.880 INFO:tasks.workunit.client.0.smithi037.stdout:5/411: mkdir d8/d22/d37/d60/d88 0 2024-03-29T08:07:15.880 INFO:tasks.workunit.client.1.smithi148.stdout:8/448: fdatasync d17/d74/d39/d6f/f55 0 2024-03-29T08:07:15.881 INFO:tasks.workunit.client.1.smithi148.stdout:6/357: unlink d6/d9/d1e/d20/d30/l45 0 2024-03-29T08:07:15.885 INFO:tasks.workunit.client.0.smithi037.stdout:2/186: creat d0/d20/d22/f3f x:0 0 0 2024-03-29T08:07:15.885 INFO:tasks.workunit.client.1.smithi148.stdout:0/391: mknod db/c7d 0 2024-03-29T08:07:15.888 INFO:tasks.workunit.client.1.smithi148.stdout:7/397: creat d8/d38/f8d x:0 0 0 2024-03-29T08:07:15.891 INFO:tasks.workunit.client.1.smithi148.stdout:7/398: dread - d8/d29/d44/d50/f52 zero size 2024-03-29T08:07:15.892 INFO:tasks.workunit.client.1.smithi148.stdout:2/390: dwrite d7/da/d33/f5b [0,4194304] 0 2024-03-29T08:07:15.893 INFO:tasks.workunit.client.1.smithi148.stdout:9/378: dread db/d10/d2d/d4e/f6b [0,4194304] 0 2024-03-29T08:07:15.893 INFO:tasks.workunit.client.0.smithi037.stdout:2/187: fdatasync d0/d20/f3a 0 2024-03-29T08:07:15.895 INFO:tasks.workunit.client.1.smithi148.stdout:7/399: dread - d8/d29/f8c zero size 2024-03-29T08:07:15.897 INFO:tasks.workunit.client.1.smithi148.stdout:8/449: mknod d17/d1d/c93 0 2024-03-29T08:07:15.900 INFO:tasks.workunit.client.0.smithi037.stdout:7/322: dwrite dd/d21/f2c [0,4194304] 0 2024-03-29T08:07:15.901 INFO:tasks.workunit.client.0.smithi037.stdout:2/188: dread - d0/d23/f26 zero size 2024-03-29T08:07:15.904 INFO:tasks.workunit.client.1.smithi148.stdout:0/392: unlink f6 0 2024-03-29T08:07:15.906 INFO:tasks.workunit.client.1.smithi148.stdout:0/393: write db/de/df/d2b/f5b [63252,14553] 0 2024-03-29T08:07:15.908 INFO:tasks.workunit.client.1.smithi148.stdout:0/394: stat db/de/df/d11/f41 0 2024-03-29T08:07:15.912 INFO:tasks.workunit.client.1.smithi148.stdout:0/395: write db/de/df/d11/d42/f6e [983870,52785] 0 2024-03-29T08:07:15.912 INFO:tasks.workunit.client.1.smithi148.stdout:1/364: symlink d3/de/d25/d35/d42/d23/d61/l6a 0 2024-03-29T08:07:15.914 INFO:tasks.workunit.client.1.smithi148.stdout:1/365: chown d3/f7 0 1 2024-03-29T08:07:15.915 INFO:tasks.workunit.client.1.smithi148.stdout:0/396: fdatasync db/de/df/d11/f37 0 2024-03-29T08:07:15.920 INFO:tasks.workunit.client.0.smithi037.stdout:7/323: mkdir dd/d68 0 2024-03-29T08:07:15.920 INFO:tasks.workunit.client.1.smithi148.stdout:7/400: rmdir d8/d14 39 2024-03-29T08:07:15.922 INFO:tasks.workunit.client.1.smithi148.stdout:2/391: mknod d7/da/d33/d3c/d31/c7e 0 2024-03-29T08:07:15.922 INFO:tasks.workunit.client.1.smithi148.stdout:9/379: rename db/d22 to db/d22/d26/d54/d52/d7d 22 2024-03-29T08:07:15.924 INFO:tasks.workunit.client.1.smithi148.stdout:2/392: readlink d7/da/l1d 0 2024-03-29T08:07:15.927 INFO:tasks.workunit.client.1.smithi148.stdout:6/358: dread d6/f41 [0,4194304] 0 2024-03-29T08:07:15.928 INFO:tasks.workunit.client.1.smithi148.stdout:2/393: chown d7/da/d33/d45/d4d/c1a 0 1 2024-03-29T08:07:15.928 INFO:tasks.workunit.client.1.smithi148.stdout:6/359: chown d6/d9/c18 39789598 1 2024-03-29T08:07:15.929 INFO:tasks.workunit.client.0.smithi037.stdout:5/412: truncate d8/d22/d4f/d70/d79/f7b 5735266 0 2024-03-29T08:07:15.930 INFO:tasks.workunit.client.1.smithi148.stdout:2/394: write d7/da/d33/f5b [4002299,107604] 0 2024-03-29T08:07:15.931 INFO:tasks.workunit.client.1.smithi148.stdout:1/366: creat d3/d55/d44/f6b x:0 0 0 2024-03-29T08:07:15.937 INFO:tasks.workunit.client.0.smithi037.stdout:7/324: creat dd/d21/d35/d42/d5c/f69 x:0 0 0 2024-03-29T08:07:15.938 INFO:tasks.workunit.client.1.smithi148.stdout:9/380: rmdir db/d22/d26 39 2024-03-29T08:07:15.939 INFO:tasks.workunit.client.0.smithi037.stdout:2/189: link d0/f8 d0/f40 0 2024-03-29T08:07:15.940 INFO:tasks.workunit.client.1.smithi148.stdout:4/366: getdents df/d2d 0 2024-03-29T08:07:15.941 INFO:tasks.workunit.client.0.smithi037.stdout:5/413: rmdir d8/d22/d37 39 2024-03-29T08:07:15.942 INFO:tasks.workunit.client.1.smithi148.stdout:2/395: dread d7/da/f3f [0,4194304] 0 2024-03-29T08:07:15.943 INFO:tasks.workunit.client.0.smithi037.stdout:7/325: mknod dd/d68/c6a 0 2024-03-29T08:07:15.943 INFO:tasks.workunit.client.1.smithi148.stdout:2/396: read d7/da/f3f [3581566,71197] 0 2024-03-29T08:07:15.944 INFO:tasks.workunit.client.0.smithi037.stdout:2/190: write d0/f1b [933414,61736] 0 2024-03-29T08:07:15.947 INFO:tasks.workunit.client.1.smithi148.stdout:3/415: truncate de/d3b/f58 2172343 0 2024-03-29T08:07:15.947 INFO:tasks.workunit.client.0.smithi037.stdout:5/414: truncate d8/d22/d4f/d70/f82 439588 0 2024-03-29T08:07:15.948 INFO:tasks.workunit.client.1.smithi148.stdout:3/416: chown de/l3e 3214 1 2024-03-29T08:07:15.951 INFO:tasks.workunit.client.1.smithi148.stdout:6/360: fsync d6/fd 0 2024-03-29T08:07:15.952 INFO:tasks.workunit.client.1.smithi148.stdout:4/367: truncate fc 71943 0 2024-03-29T08:07:15.952 INFO:tasks.workunit.client.1.smithi148.stdout:3/417: read de/d3b/f60 [3570133,43760] 0 2024-03-29T08:07:15.952 INFO:tasks.workunit.client.0.smithi037.stdout:5/415: mknod d8/d4d/d5c/c89 0 2024-03-29T08:07:15.953 INFO:tasks.workunit.client.1.smithi148.stdout:2/397: creat d7/da/f7f x:0 0 0 2024-03-29T08:07:15.955 INFO:tasks.workunit.client.1.smithi148.stdout:1/367: symlink d3/de/d4f/l6c 0 2024-03-29T08:07:15.957 INFO:tasks.workunit.client.1.smithi148.stdout:6/361: write d6/d9/f23 [211358,127522] 0 2024-03-29T08:07:15.957 INFO:tasks.workunit.client.1.smithi148.stdout:9/381: dwrite db/d10/d2d/f34 [0,4194304] 0 2024-03-29T08:07:15.959 INFO:tasks.workunit.client.1.smithi148.stdout:4/368: chown df/d2c/d51/f67 6163 1 2024-03-29T08:07:15.960 INFO:tasks.workunit.client.1.smithi148.stdout:3/418: stat de/d24/d33/d39/d46/f25 0 2024-03-29T08:07:15.962 INFO:tasks.workunit.client.0.smithi037.stdout:2/191: rename d0/d20/d2e/d38/c3b to d0/d20/d22/c41 0 2024-03-29T08:07:15.964 INFO:tasks.workunit.client.1.smithi148.stdout:5/394: dread d0/db/f10 [0,4194304] 0 2024-03-29T08:07:15.969 INFO:tasks.workunit.client.0.smithi037.stdout:2/192: dread - d0/d20/f3d zero size 2024-03-29T08:07:15.971 INFO:tasks.workunit.client.1.smithi148.stdout:3/419: fsync de/d24/f27 0 2024-03-29T08:07:15.972 INFO:tasks.workunit.client.1.smithi148.stdout:7/401: dwrite d8/d14/d19/f4b [0,4194304] 0 2024-03-29T08:07:15.973 INFO:tasks.workunit.client.0.smithi037.stdout:7/326: link dd/d21/f2b dd/d21/d35/f6b 0 2024-03-29T08:07:15.977 INFO:tasks.workunit.client.1.smithi148.stdout:6/362: dread d6/d9/f23 [0,4194304] 0 2024-03-29T08:07:15.979 INFO:tasks.workunit.client.1.smithi148.stdout:6/363: read f5 [644541,5218] 0 2024-03-29T08:07:15.982 INFO:tasks.workunit.client.1.smithi148.stdout:8/450: dwrite d17/d18/f8a [0,4194304] 0 2024-03-29T08:07:15.983 INFO:tasks.workunit.client.1.smithi148.stdout:5/395: mkdir d0/db/d1e/d35/d42/d67/d76/d81 0 2024-03-29T08:07:15.984 INFO:tasks.workunit.client.1.smithi148.stdout:1/368: getdents d3/d55/d44 0 2024-03-29T08:07:15.986 INFO:tasks.workunit.client.0.smithi037.stdout:5/416: dwrite d8/fa [0,4194304] 0 2024-03-29T08:07:15.986 INFO:tasks.workunit.client.1.smithi148.stdout:1/369: chown d3/d18/l3b 127 1 2024-03-29T08:07:15.989 INFO:tasks.workunit.client.1.smithi148.stdout:0/397: dread db/de/df/d11/d42/f52 [0,4194304] 0 2024-03-29T08:07:15.992 INFO:tasks.workunit.client.1.smithi148.stdout:4/369: rename df/d2c/d51/d77/d26/d34/d3e/l6b to df/d2d/d7c/l7f 0 2024-03-29T08:07:15.998 INFO:tasks.workunit.client.1.smithi148.stdout:2/398: symlink d7/da/d33/d3c/l80 0 2024-03-29T08:07:16.000 INFO:tasks.workunit.client.0.smithi037.stdout:7/327: getdents dd/d21/d35/d42/d5c 0 2024-03-29T08:07:16.002 INFO:tasks.workunit.client.1.smithi148.stdout:8/451: chown d17/d74/d39/l40 3316249 1 2024-03-29T08:07:16.003 INFO:tasks.workunit.client.1.smithi148.stdout:8/452: write f14 [4254413,10598] 0 2024-03-29T08:07:16.004 INFO:tasks.workunit.client.1.smithi148.stdout:8/453: chown d17/d49 128367277 1 2024-03-29T08:07:16.006 INFO:tasks.workunit.client.1.smithi148.stdout:5/396: write d0/db/d7b/f50 [2496571,60094] 0 2024-03-29T08:07:16.006 INFO:tasks.workunit.client.0.smithi037.stdout:2/193: write d0/fb [703930,6316] 0 2024-03-29T08:07:16.010 INFO:tasks.workunit.client.1.smithi148.stdout:0/398: stat db/de/df/d14/l56 0 2024-03-29T08:07:16.010 INFO:tasks.workunit.client.1.smithi148.stdout:6/364: rename d6/d9/d1e/d27/c2f to d6/d9/d1e/d20/d30/d51/c66 0 2024-03-29T08:07:16.012 INFO:tasks.workunit.client.1.smithi148.stdout:4/370: fdatasync df/d2c/d51/d77/f53 0 2024-03-29T08:07:16.012 INFO:tasks.workunit.client.0.smithi037.stdout:5/417: dread d8/d22/f31 [0,4194304] 0 2024-03-29T08:07:16.013 INFO:tasks.workunit.client.1.smithi148.stdout:5/397: dread d0/db/f10 [0,4194304] 0 2024-03-29T08:07:16.013 INFO:tasks.workunit.client.1.smithi148.stdout:2/399: symlink d7/d24/d4b/l81 0 2024-03-29T08:07:16.015 INFO:tasks.workunit.client.1.smithi148.stdout:8/454: mkdir d17/d18/d26/d4f/d94 0 2024-03-29T08:07:16.016 INFO:tasks.workunit.client.1.smithi148.stdout:7/402: link d8/d29/l7b d8/d60/l8e 0 2024-03-29T08:07:16.017 INFO:tasks.workunit.client.1.smithi148.stdout:1/370: truncate d3/d18/f20 1259904 0 2024-03-29T08:07:16.019 INFO:tasks.workunit.client.1.smithi148.stdout:7/403: read d8/d14/f2c [406218,111325] 0 2024-03-29T08:07:16.020 INFO:tasks.workunit.client.0.smithi037.stdout:5/418: truncate d8/d18/d24/d45/f84 598660 0 2024-03-29T08:07:16.021 INFO:tasks.workunit.client.1.smithi148.stdout:3/420: rename de/d24/d33/c6d to de/d24/d81/c8f 0 2024-03-29T08:07:16.024 INFO:tasks.workunit.client.1.smithi148.stdout:7/404: dread d8/d14/d19/d37/f48 [0,4194304] 0 2024-03-29T08:07:16.024 INFO:tasks.workunit.client.0.smithi037.stdout:5/419: fsync d8/d34/f4a 0 2024-03-29T08:07:16.024 INFO:tasks.workunit.client.0.smithi037.stdout:2/194: dread d0/fb [0,4194304] 0 2024-03-29T08:07:16.025 INFO:tasks.workunit.client.1.smithi148.stdout:2/400: chown d7/da/d52 6 1 2024-03-29T08:07:16.026 INFO:tasks.workunit.client.0.smithi037.stdout:7/328: truncate dd/d1d/d34/f3f 1197372 0 2024-03-29T08:07:16.027 INFO:tasks.workunit.client.1.smithi148.stdout:8/455: unlink d17/d18/f3a 0 2024-03-29T08:07:16.027 INFO:tasks.workunit.client.1.smithi148.stdout:5/398: dread d0/db/d1e/d35/d6a/f6d [0,4194304] 0 2024-03-29T08:07:16.028 INFO:tasks.workunit.client.1.smithi148.stdout:6/365: mknod d6/c67 0 2024-03-29T08:07:16.028 INFO:tasks.workunit.client.1.smithi148.stdout:5/399: read d0/db/f34 [570225,42137] 0 2024-03-29T08:07:16.029 INFO:tasks.workunit.client.1.smithi148.stdout:1/371: write d3/f30 [3347951,14685] 0 2024-03-29T08:07:16.031 INFO:tasks.workunit.client.0.smithi037.stdout:5/420: truncate d8/d22/d4f/d70/f54 1206842 0 2024-03-29T08:07:16.031 INFO:tasks.workunit.client.0.smithi037.stdout:2/195: symlink d0/l42 0 2024-03-29T08:07:16.033 INFO:tasks.workunit.client.0.smithi037.stdout:2/196: fdatasync d0/f1b 0 2024-03-29T08:07:16.033 INFO:tasks.workunit.client.0.smithi037.stdout:7/329: mkdir dd/d21/d35/d42/d6c 0 2024-03-29T08:07:16.034 INFO:tasks.workunit.client.0.smithi037.stdout:5/421: readlink l6 0 2024-03-29T08:07:16.034 INFO:tasks.workunit.client.1.smithi148.stdout:0/399: dwrite db/de/d24/f38 [0,4194304] 0 2024-03-29T08:07:16.035 INFO:tasks.workunit.client.0.smithi037.stdout:2/197: fdatasync d0/fb 0 2024-03-29T08:07:16.036 INFO:tasks.workunit.client.1.smithi148.stdout:3/421: rename de/d24/d33/d39/d46/f5f to de/d3b/d49/f90 0 2024-03-29T08:07:16.038 INFO:tasks.workunit.client.1.smithi148.stdout:3/422: chown de/d24/d33/d39/d46/c7e 201356056 1 2024-03-29T08:07:16.039 INFO:tasks.workunit.client.0.smithi037.stdout:5/422: mkdir d8/d4d/d8a 0 2024-03-29T08:07:16.039 INFO:tasks.workunit.client.1.smithi148.stdout:2/401: chown d7/da/d4c 51 1 2024-03-29T08:07:16.040 INFO:tasks.workunit.client.0.smithi037.stdout:7/330: truncate dd/d21/f26 901995 0 2024-03-29T08:07:16.041 INFO:tasks.workunit.client.0.smithi037.stdout:7/331: write dd/f67 [150324,60814] 0 2024-03-29T08:07:16.041 INFO:tasks.workunit.client.1.smithi148.stdout:5/400: dread d0/db/d7b/d16/f27 [0,4194304] 0 2024-03-29T08:07:16.042 INFO:tasks.workunit.client.0.smithi037.stdout:2/198: rmdir d0/d20/d2e 39 2024-03-29T08:07:16.043 INFO:tasks.workunit.client.1.smithi148.stdout:7/405: symlink d8/d38/l8f 0 2024-03-29T08:07:16.043 INFO:tasks.workunit.client.1.smithi148.stdout:8/456: symlink d17/d49/l95 0 2024-03-29T08:07:16.044 INFO:tasks.workunit.client.1.smithi148.stdout:5/401: write d0/db/d1e/d36/d41/f58 [833501,119839] 0 2024-03-29T08:07:16.048 INFO:tasks.workunit.client.1.smithi148.stdout:5/402: read - d0/db/d1e/d36/d41/f59 zero size 2024-03-29T08:07:16.049 INFO:tasks.workunit.client.0.smithi037.stdout:7/332: creat dd/d21/f6d x:0 0 0 2024-03-29T08:07:16.049 INFO:tasks.workunit.client.1.smithi148.stdout:5/403: chown d0/db/f10 16403083 1 2024-03-29T08:07:16.050 INFO:tasks.workunit.client.1.smithi148.stdout:1/372: mkdir d3/de/d1d/d47/d6d 0 2024-03-29T08:07:16.052 INFO:tasks.workunit.client.0.smithi037.stdout:7/333: truncate dd/f67 762682 0 2024-03-29T08:07:16.052 INFO:tasks.workunit.client.1.smithi148.stdout:0/400: dwrite db/de/df/d2b/d3b/d47/f2c [0,4194304] 0 2024-03-29T08:07:16.055 INFO:tasks.workunit.client.0.smithi037.stdout:2/199: truncate d0/fb 613960 0 2024-03-29T08:07:16.056 INFO:tasks.workunit.client.1.smithi148.stdout:3/423: mknod de/d3b/d49/c91 0 2024-03-29T08:07:16.058 INFO:tasks.workunit.client.0.smithi037.stdout:7/334: readlink dd/l10 0 2024-03-29T08:07:16.060 INFO:tasks.workunit.client.0.smithi037.stdout:7/335: fsync dd/f17 0 2024-03-29T08:07:16.062 INFO:tasks.workunit.client.1.smithi148.stdout:2/402: readlink d7/da/l44 0 2024-03-29T08:07:16.067 INFO:tasks.workunit.client.0.smithi037.stdout:5/423: rename d8/d22/d37/l47 to d8/d22/d37/d60/l8b 0 2024-03-29T08:07:16.073 INFO:tasks.workunit.client.0.smithi037.stdout:5/424: chown d8/d22/d4f/d70/c3e 748034353 1 2024-03-29T08:07:16.074 INFO:tasks.workunit.client.1.smithi148.stdout:4/371: write df/f4d [1248670,24540] 0 2024-03-29T08:07:16.074 INFO:tasks.workunit.client.0.smithi037.stdout:3/293: sync 2024-03-29T08:07:16.074 INFO:tasks.workunit.client.0.smithi037.stdout:4/298: sync 2024-03-29T08:07:16.077 INFO:tasks.workunit.client.1.smithi148.stdout:1/373: unlink d3/de/d25/c2b 0 2024-03-29T08:07:16.079 INFO:tasks.workunit.client.1.smithi148.stdout:5/404: rename d0/ca to d0/db/d1e/d5b/d66/c82 0 2024-03-29T08:07:16.080 INFO:tasks.workunit.client.0.smithi037.stdout:4/299: write d3/d4/d15/d1d/f3f [1190047,67672] 0 2024-03-29T08:07:16.082 INFO:tasks.workunit.client.1.smithi148.stdout:7/406: dread d8/d29/d44/d49/f68 [4194304,4194304] 0 2024-03-29T08:07:16.082 INFO:tasks.workunit.client.0.smithi037.stdout:2/200: dwrite d0/d20/d22/f3c [0,4194304] 0 2024-03-29T08:07:16.082 INFO:tasks.workunit.client.0.smithi037.stdout:7/336: unlink dd/d21/f2b 0 2024-03-29T08:07:16.084 INFO:tasks.workunit.client.1.smithi148.stdout:1/374: mkdir d3/de/d6e 0 2024-03-29T08:07:16.085 INFO:tasks.workunit.client.1.smithi148.stdout:0/401: stat db/de/df/d11/d1f/c5e 0 2024-03-29T08:07:16.087 INFO:tasks.workunit.client.1.smithi148.stdout:2/403: mkdir d7/da/d33/d45/d4d/d82 0 2024-03-29T08:07:16.090 INFO:tasks.workunit.client.1.smithi148.stdout:4/372: dread df/f27 [0,4194304] 0 2024-03-29T08:07:16.093 INFO:tasks.workunit.client.1.smithi148.stdout:7/407: creat d8/d29/d44/d50/f90 x:0 0 0 2024-03-29T08:07:16.094 INFO:tasks.workunit.client.0.smithi037.stdout:5/425: symlink d8/d4d/d5c/l8c 0 2024-03-29T08:07:16.095 INFO:tasks.workunit.client.0.smithi037.stdout:8/329: sync 2024-03-29T08:07:16.096 INFO:tasks.workunit.client.0.smithi037.stdout:9/487: sync 2024-03-29T08:07:16.096 INFO:tasks.workunit.client.0.smithi037.stdout:1/360: sync 2024-03-29T08:07:16.096 INFO:tasks.workunit.client.0.smithi037.stdout:0/177: sync 2024-03-29T08:07:16.096 INFO:tasks.workunit.client.0.smithi037.stdout:6/240: sync 2024-03-29T08:07:16.098 INFO:tasks.workunit.client.0.smithi037.stdout:9/488: fsync d1/f8d 0 2024-03-29T08:07:16.098 INFO:tasks.workunit.client.0.smithi037.stdout:0/178: fsync d9/da/f24 0 2024-03-29T08:07:16.098 INFO:tasks.workunit.client.1.smithi148.stdout:5/405: rename d0/d46/l4e to d0/d46/l83 0 2024-03-29T08:07:16.099 INFO:tasks.workunit.client.1.smithi148.stdout:3/424: dread de/d3b/d49/f64 [0,4194304] 0 2024-03-29T08:07:16.101 INFO:tasks.workunit.client.1.smithi148.stdout:2/404: mkdir d7/d24/d4b/d83 0 2024-03-29T08:07:16.102 INFO:tasks.workunit.client.0.smithi037.stdout:0/179: chown d9/da/f34 28 1 2024-03-29T08:07:16.104 INFO:tasks.workunit.client.1.smithi148.stdout:5/406: write d0/f9 [2000044,98023] 0 2024-03-29T08:07:16.106 INFO:tasks.workunit.client.0.smithi037.stdout:1/361: fdatasync d2/d6/dc/d18/d1d/d21/f20 0 2024-03-29T08:07:16.106 INFO:tasks.workunit.client.1.smithi148.stdout:4/373: chown df/d2c/d51/d77/f6e 111 1 2024-03-29T08:07:16.109 INFO:tasks.workunit.client.1.smithi148.stdout:1/375: dwrite d3/d55/d44/f6b [0,4194304] 0 2024-03-29T08:07:16.112 INFO:tasks.workunit.client.0.smithi037.stdout:2/201: readlink d0/d20/d2e/l33 0 2024-03-29T08:07:16.116 INFO:tasks.workunit.client.1.smithi148.stdout:0/402: dwrite db/de/f62 [0,4194304] 0 2024-03-29T08:07:16.116 INFO:tasks.workunit.client.1.smithi148.stdout:4/374: mkdir df/d2c/d80 0 2024-03-29T08:07:16.116 INFO:tasks.workunit.client.0.smithi037.stdout:7/337: unlink dd/d21/f44 0 2024-03-29T08:07:16.118 INFO:tasks.workunit.client.1.smithi148.stdout:7/408: dwrite d8/d29/d44/d49/f82 [0,4194304] 0 2024-03-29T08:07:16.122 INFO:tasks.workunit.client.1.smithi148.stdout:5/407: mkdir d0/db/d84 0 2024-03-29T08:07:16.127 INFO:tasks.workunit.client.0.smithi037.stdout:3/294: chown dd/l17 41828 1 2024-03-29T08:07:16.130 INFO:tasks.workunit.client.1.smithi148.stdout:4/375: symlink df/d2c/d51/d77/d17/d6a/l81 0 2024-03-29T08:07:16.132 INFO:tasks.workunit.client.0.smithi037.stdout:5/426: dwrite d8/d22/f27 [0,4194304] 0 2024-03-29T08:07:16.133 INFO:tasks.workunit.client.1.smithi148.stdout:0/403: truncate db/de/df/d11/d42/f67 4937518 0 2024-03-29T08:07:16.133 INFO:tasks.workunit.client.0.smithi037.stdout:8/330: unlink d5/d10/d19/d3d/f4d 0 2024-03-29T08:07:16.135 INFO:tasks.workunit.client.1.smithi148.stdout:2/405: dread d7/da/f3f [0,4194304] 0 2024-03-29T08:07:16.135 INFO:tasks.workunit.client.1.smithi148.stdout:6/366: truncate d6/d9/d1e/d27/f44 4120172 0 2024-03-29T08:07:16.136 INFO:tasks.workunit.client.1.smithi148.stdout:5/408: dread d0/db/d7b/f52 [0,4194304] 0 2024-03-29T08:07:16.138 INFO:tasks.workunit.client.1.smithi148.stdout:6/367: stat d6/d9/d1e/f5d 0 2024-03-29T08:07:16.142 INFO:tasks.workunit.client.0.smithi037.stdout:1/362: mknod d2/d6/dc/d18/d1d/d5e/d72/c79 0 2024-03-29T08:07:16.148 INFO:tasks.workunit.client.1.smithi148.stdout:7/409: getdents d8/d14 0 2024-03-29T08:07:16.150 INFO:tasks.workunit.client.1.smithi148.stdout:7/410: fdatasync d8/d14/d19/f39 0 2024-03-29T08:07:16.150 INFO:tasks.workunit.client.0.smithi037.stdout:7/338: dwrite dd/d21/f32 [4194304,4194304] 0 2024-03-29T08:07:16.151 INFO:tasks.workunit.client.0.smithi037.stdout:2/202: symlink d0/d20/d22/l43 0 2024-03-29T08:07:16.152 INFO:tasks.workunit.client.1.smithi148.stdout:7/411: chown d8/d29/d44/d49/f62 113 1 2024-03-29T08:07:16.153 INFO:tasks.workunit.client.1.smithi148.stdout:0/404: creat db/de/df/d2b/d3b/d47/d4c/d5a/f7e x:0 0 0 2024-03-29T08:07:16.155 INFO:tasks.workunit.client.0.smithi037.stdout:6/241: dread d0/d5/d1a/f2f [0,4194304] 0 2024-03-29T08:07:16.159 INFO:tasks.workunit.client.1.smithi148.stdout:5/409: unlink d0/db/d7b/d16/c3a 0 2024-03-29T08:07:16.163 INFO:tasks.workunit.client.1.smithi148.stdout:9/382: sync 2024-03-29T08:07:16.163 INFO:tasks.workunit.client.1.smithi148.stdout:5/410: write d0/db/d1e/d35/d42/d67/f70 [871425,76944] 0 2024-03-29T08:07:16.167 INFO:tasks.workunit.client.1.smithi148.stdout:2/406: rename d7/da/d33/d45/d4d/d2e/l6f to d7/da/d33/d3c/d21/l84 0 2024-03-29T08:07:16.170 INFO:tasks.workunit.client.0.smithi037.stdout:4/300: link d3/f5d d3/d57/f67 0 2024-03-29T08:07:16.171 INFO:tasks.workunit.client.1.smithi148.stdout:7/412: dwrite d8/d29/f35 [0,4194304] 0 2024-03-29T08:07:16.174 INFO:tasks.workunit.client.0.smithi037.stdout:5/427: mkdir d8/d22/d37/d8d 0 2024-03-29T08:07:16.174 INFO:tasks.workunit.client.0.smithi037.stdout:8/331: creat d5/d10/d19/d3d/d4a/d4c/f6e x:0 0 0 2024-03-29T08:07:16.177 INFO:tasks.workunit.client.0.smithi037.stdout:3/295: fdatasync dd/d46/d3e/d47/f54 0 2024-03-29T08:07:16.178 INFO:tasks.workunit.client.0.smithi037.stdout:9/489: rename d1/d1f/d3a/d67/f6d to d1/d4/d19/d1e/d4a/d6f/da2/d31/fac 0 2024-03-29T08:07:16.178 INFO:tasks.workunit.client.1.smithi148.stdout:9/383: creat db/d10/d2d/d4e/d5f/d6e/d7a/f7e x:0 0 0 2024-03-29T08:07:16.179 INFO:tasks.workunit.client.0.smithi037.stdout:8/332: readlink d5/d10/d19/l2a 0 2024-03-29T08:07:16.181 INFO:tasks.workunit.client.1.smithi148.stdout:3/425: dwrite de/d24/f36 [0,4194304] 0 2024-03-29T08:07:16.184 INFO:tasks.workunit.client.1.smithi148.stdout:7/413: mkdir d8/d38/d6a/d91 0 2024-03-29T08:07:16.187 INFO:tasks.workunit.client.1.smithi148.stdout:0/405: write db/de/df/f30 [48186,34458] 0 2024-03-29T08:07:16.189 INFO:tasks.workunit.client.0.smithi037.stdout:6/242: mknod d0/d5/d1f/c52 0 2024-03-29T08:07:16.191 INFO:tasks.workunit.client.1.smithi148.stdout:5/411: symlink d0/db/d1e/d36/d7c/l85 0 2024-03-29T08:07:16.191 INFO:tasks.workunit.client.0.smithi037.stdout:6/243: chown d0/d5/d1e/l2e 156 1 2024-03-29T08:07:16.193 INFO:tasks.workunit.client.1.smithi148.stdout:1/376: unlink d3/de/d25/d31/c3a 0 2024-03-29T08:07:16.197 INFO:tasks.workunit.client.1.smithi148.stdout:4/376: dwrite df/d2c/d51/d77/d17/f42 [4194304,4194304] 0 2024-03-29T08:07:16.197 INFO:tasks.workunit.client.1.smithi148.stdout:6/368: rename d6/d9/d1e/d36/d4d/d5e to d6/d9/d68 0 2024-03-29T08:07:16.199 INFO:tasks.workunit.client.0.smithi037.stdout:4/301: unlink d3/d4/c2b 0 2024-03-29T08:07:16.200 INFO:tasks.workunit.client.1.smithi148.stdout:5/412: dread - d0/db/d1e/d35/d42/f5e zero size 2024-03-29T08:07:16.201 INFO:tasks.workunit.client.1.smithi148.stdout:1/377: fsync d3/d55/d44/f53 0 2024-03-29T08:07:16.203 INFO:tasks.workunit.client.1.smithi148.stdout:6/369: chown d6/f14 0 1 2024-03-29T08:07:16.205 INFO:tasks.workunit.client.1.smithi148.stdout:0/406: mknod db/de/df/d2b/d3b/d47/d4c/d5a/d7a/c7f 0 2024-03-29T08:07:16.206 INFO:tasks.workunit.client.1.smithi148.stdout:1/378: chown d3/l17 3823 1 2024-03-29T08:07:16.209 INFO:tasks.workunit.client.0.smithi037.stdout:5/428: rename d8/l1c to d8/d34/l8e 0 2024-03-29T08:07:16.210 INFO:tasks.workunit.client.1.smithi148.stdout:5/413: chown d0/db/d1e/d35/l3c 4 1 2024-03-29T08:07:16.210 INFO:tasks.workunit.client.1.smithi148.stdout:6/370: chown d6/d9/d1e/d22/f34 4 1 2024-03-29T08:07:16.214 INFO:tasks.workunit.client.1.smithi148.stdout:4/377: truncate df/d2c/d51/d77/d17/d73/f7a 766255 0 2024-03-29T08:07:16.215 INFO:tasks.workunit.client.0.smithi037.stdout:5/429: dread - d8/d34/f6a zero size 2024-03-29T08:07:16.216 INFO:tasks.workunit.client.1.smithi148.stdout:2/407: dwrite d7/d24/d4b/f55 [0,4194304] 0 2024-03-29T08:07:16.216 INFO:tasks.workunit.client.0.smithi037.stdout:9/490: unlink d1/d4/d19/d1e/d71/d89/d92/la1 0 2024-03-29T08:07:16.218 INFO:tasks.workunit.client.0.smithi037.stdout:8/333: creat d5/d10/d19/d3d/d4a/f6f x:0 0 0 2024-03-29T08:07:16.220 INFO:tasks.workunit.client.0.smithi037.stdout:6/244: symlink d0/d5/d3f/l53 0 2024-03-29T08:07:16.223 INFO:tasks.workunit.client.0.smithi037.stdout:6/245: stat d0/l3a 0 2024-03-29T08:07:16.224 INFO:tasks.workunit.client.0.smithi037.stdout:7/339: readlink dd/d21/d35/d42/d5c/l64 0 2024-03-29T08:07:16.226 INFO:tasks.workunit.client.1.smithi148.stdout:6/371: dread d6/d9/d1e/d20/d30/f56 [0,4194304] 0 2024-03-29T08:07:16.227 INFO:tasks.workunit.client.0.smithi037.stdout:0/180: getdents d9/da/d1b 0 2024-03-29T08:07:16.228 INFO:tasks.workunit.client.0.smithi037.stdout:6/246: write d0/d19/d34/f50 [795342,25855] 0 2024-03-29T08:07:16.230 INFO:tasks.workunit.client.1.smithi148.stdout:1/379: truncate d3/de/f1f 9051033 0 2024-03-29T08:07:16.234 INFO:tasks.workunit.client.1.smithi148.stdout:5/414: dwrite d0/f9 [0,4194304] 0 2024-03-29T08:07:16.235 INFO:tasks.workunit.client.1.smithi148.stdout:1/380: chown d3/de/d25/d35/c62 2020 1 2024-03-29T08:07:16.237 INFO:tasks.workunit.client.0.smithi037.stdout:6/247: dread - d0/d20/f46 zero size 2024-03-29T08:07:16.238 INFO:tasks.workunit.client.1.smithi148.stdout:1/381: read - d3/de/d25/d35/d42/d23/f40 zero size 2024-03-29T08:07:16.240 INFO:tasks.workunit.client.1.smithi148.stdout:2/408: creat d7/da/d33/d3c/d54/d70/f85 x:0 0 0 2024-03-29T08:07:16.240 INFO:tasks.workunit.client.0.smithi037.stdout:9/491: rename d1/d4/d19/d1e/d4a/fa7 to d1/d4/d95/fad 0 2024-03-29T08:07:16.241 INFO:tasks.workunit.client.1.smithi148.stdout:1/382: chown d3/de/d25/d35/d42 7451 1 2024-03-29T08:07:16.244 INFO:tasks.workunit.client.1.smithi148.stdout:3/426: dread de/d24/f27 [0,4194304] 0 2024-03-29T08:07:16.244 INFO:tasks.workunit.client.0.smithi037.stdout:7/340: fdatasync dd/f48 0 2024-03-29T08:07:16.248 INFO:tasks.workunit.client.0.smithi037.stdout:1/363: getdents d2/d6/dc/d18/d22 0 2024-03-29T08:07:16.252 INFO:tasks.workunit.client.1.smithi148.stdout:5/415: mkdir d0/d46/d86 0 2024-03-29T08:07:16.252 INFO:tasks.workunit.client.0.smithi037.stdout:4/302: mknod d3/d4/d15/d1d/db/d45/d1e/c68 0 2024-03-29T08:07:16.254 INFO:tasks.workunit.client.0.smithi037.stdout:2/203: dwrite d0/f15 [4194304,4194304] 0 2024-03-29T08:07:16.254 INFO:tasks.workunit.client.0.smithi037.stdout:5/430: dwrite d8/fa [0,4194304] 0 2024-03-29T08:07:16.256 INFO:tasks.workunit.client.1.smithi148.stdout:2/409: mkdir d7/d86 0 2024-03-29T08:07:16.257 INFO:tasks.workunit.client.1.smithi148.stdout:1/383: symlink d3/d55/d44/l6f 0 2024-03-29T08:07:16.258 INFO:tasks.workunit.client.0.smithi037.stdout:0/181: rename d9/da/l1f to d9/da/dd/d13/l35 0 2024-03-29T08:07:16.258 INFO:tasks.workunit.client.1.smithi148.stdout:4/378: dread fa [0,4194304] 0 2024-03-29T08:07:16.261 INFO:tasks.workunit.client.1.smithi148.stdout:0/407: link db/de/df/d11/d1f/c5e db/dc/d78/c80 0 2024-03-29T08:07:16.261 INFO:tasks.workunit.client.1.smithi148.stdout:6/372: mkdir d6/d69 0 2024-03-29T08:07:16.261 INFO:tasks.workunit.client.0.smithi037.stdout:8/334: write d5/d25/f28 [3462751,121843] 0 2024-03-29T08:07:16.261 INFO:tasks.workunit.client.0.smithi037.stdout:7/341: write dd/d1d/d49/f5a [905656,101881] 0 2024-03-29T08:07:16.263 INFO:tasks.workunit.client.1.smithi148.stdout:5/416: unlink d0/db/d1e/d35/d42/f43 0 2024-03-29T08:07:16.264 INFO:tasks.workunit.client.1.smithi148.stdout:3/427: truncate de/d24/f31 6174953 0 2024-03-29T08:07:16.268 INFO:tasks.workunit.client.1.smithi148.stdout:4/379: rmdir df/d2c/d51/d77/d26/d34/d3e 39 2024-03-29T08:07:16.272 INFO:tasks.workunit.client.0.smithi037.stdout:3/296: dread dd/fe [0,4194304] 0 2024-03-29T08:07:16.272 INFO:tasks.workunit.client.1.smithi148.stdout:6/373: mknod d6/c6a 0 2024-03-29T08:07:16.273 INFO:tasks.workunit.client.1.smithi148.stdout:1/384: link d3/cd d3/d55/c70 0 2024-03-29T08:07:16.274 INFO:tasks.workunit.client.1.smithi148.stdout:6/374: chown d6/fd 3375 1 2024-03-29T08:07:16.275 INFO:tasks.workunit.client.0.smithi037.stdout:1/364: symlink d2/d6/dc/d5d/d62/l7a 0 2024-03-29T08:07:16.276 INFO:tasks.workunit.client.1.smithi148.stdout:0/408: rename db/de/df/d2b/d3b/d47/d4c/d5a/l73 to db/de/df/d2b/d3b/l81 0 2024-03-29T08:07:16.277 INFO:tasks.workunit.client.1.smithi148.stdout:5/417: mkdir d0/db/d7b/d16/d87 0 2024-03-29T08:07:16.277 INFO:tasks.workunit.client.1.smithi148.stdout:8/457: sync 2024-03-29T08:07:16.279 INFO:tasks.workunit.client.1.smithi148.stdout:6/375: chown d6/d9/d1e/d20/d30/f33 44600 1 2024-03-29T08:07:16.280 INFO:tasks.workunit.client.1.smithi148.stdout:2/410: dwrite d7/da/d33/d3c/f48 [0,4194304] 0 2024-03-29T08:07:16.283 INFO:tasks.workunit.client.1.smithi148.stdout:5/418: chown d0/db/d1e/f7e 1528042 1 2024-03-29T08:07:16.283 INFO:tasks.workunit.client.1.smithi148.stdout:0/409: chown db/de/df/d2b/d3b/d47/l58 12514849 1 2024-03-29T08:07:16.285 INFO:tasks.workunit.client.1.smithi148.stdout:2/411: write d7/da/d33/d45/d4d/f29 [1341212,41314] 0 2024-03-29T08:07:16.287 INFO:tasks.workunit.client.0.smithi037.stdout:1/365: stat d2/d6/dc/c30 0 2024-03-29T08:07:16.287 INFO:tasks.workunit.client.0.smithi037.stdout:9/492: truncate d1/d4/d19/d1e/d25/d16/f56 915453 0 2024-03-29T08:07:16.287 INFO:tasks.workunit.client.1.smithi148.stdout:4/380: fsync df/d2c/d51/d77/f40 0 2024-03-29T08:07:16.299 INFO:tasks.workunit.client.0.smithi037.stdout:4/303: truncate d3/f3a 3654875 0 2024-03-29T08:07:16.302 INFO:tasks.workunit.client.0.smithi037.stdout:0/182: write d9/da/d1b/d22/d2c/f2d [777814,53961] 0 2024-03-29T08:07:16.305 INFO:tasks.workunit.client.0.smithi037.stdout:8/335: truncate d5/d10/d2d/d34/d50/f68 722481 0 2024-03-29T08:07:16.305 INFO:tasks.workunit.client.1.smithi148.stdout:1/385: dread d3/f7 [0,4194304] 0 2024-03-29T08:07:16.306 INFO:tasks.workunit.client.0.smithi037.stdout:7/342: dwrite dd/d21/f6d [0,4194304] 0 2024-03-29T08:07:16.306 INFO:tasks.workunit.client.1.smithi148.stdout:0/410: write db/de/df/d11/d42/f52 [2926109,117898] 0 2024-03-29T08:07:16.313 INFO:tasks.workunit.client.1.smithi148.stdout:4/381: creat df/d2c/d51/d77/d26/d34/d43/f82 x:0 0 0 2024-03-29T08:07:16.316 INFO:tasks.workunit.client.1.smithi148.stdout:4/382: readlink df/d2c/d51/d77/d26/d34/l55 0 2024-03-29T08:07:16.319 INFO:tasks.workunit.client.1.smithi148.stdout:2/412: rename d7/da/d33/d45/d4d/f29 to d7/d24/d4b/f87 0 2024-03-29T08:07:16.322 INFO:tasks.workunit.client.1.smithi148.stdout:4/383: truncate df/d2c/d51/d77/d26/f3b 828985 0 2024-03-29T08:07:16.326 INFO:tasks.workunit.client.0.smithi037.stdout:1/366: fdatasync d2/d6/dc/fd 0 2024-03-29T08:07:16.326 INFO:tasks.workunit.client.0.smithi037.stdout:2/204: dread d0/f7 [0,4194304] 0 2024-03-29T08:07:16.326 INFO:tasks.workunit.client.1.smithi148.stdout:1/386: dwrite d3/de/d25/d51/f54 [0,4194304] 0 2024-03-29T08:07:16.327 INFO:tasks.workunit.client.0.smithi037.stdout:6/248: dread d0/d5/f9 [0,4194304] 0 2024-03-29T08:07:16.329 INFO:tasks.workunit.client.1.smithi148.stdout:1/387: dread - d3/de/d25/d31/f67 zero size 2024-03-29T08:07:16.331 INFO:tasks.workunit.client.1.smithi148.stdout:1/388: chown d3/de/d25/d35/d42/d23/f40 29280 1 2024-03-29T08:07:16.333 INFO:tasks.workunit.client.1.smithi148.stdout:1/389: chown d3/d18/f4b 314 1 2024-03-29T08:07:16.333 INFO:tasks.workunit.client.0.smithi037.stdout:9/493: creat d1/d4/d19/d1e/fae x:0 0 0 2024-03-29T08:07:16.338 INFO:tasks.workunit.client.0.smithi037.stdout:5/431: link d8/l14 d8/d22/d4f/d70/d79/d7a/l8f 0 2024-03-29T08:07:16.338 INFO:tasks.workunit.client.0.smithi037.stdout:8/336: write d5/d10/d2d/f3e [260522,7091] 0 2024-03-29T08:07:16.339 INFO:tasks.workunit.client.0.smithi037.stdout:0/183: dwrite f7 [8388608,4194304] 0 2024-03-29T08:07:16.340 INFO:tasks.workunit.client.1.smithi148.stdout:5/419: write d0/f12 [1314984,126945] 0 2024-03-29T08:07:16.343 INFO:tasks.workunit.client.1.smithi148.stdout:1/390: dread d3/d18/f39 [0,4194304] 0 2024-03-29T08:07:16.344 INFO:tasks.workunit.client.1.smithi148.stdout:1/391: chown d3/l6 33702 1 2024-03-29T08:07:16.348 INFO:tasks.workunit.client.1.smithi148.stdout:2/413: creat d7/da/d33/d45/f88 x:0 0 0 2024-03-29T08:07:16.349 INFO:tasks.workunit.client.1.smithi148.stdout:2/414: readlink l6 0 2024-03-29T08:07:16.350 INFO:tasks.workunit.client.0.smithi037.stdout:2/205: mkdir d0/d23/d44 0 2024-03-29T08:07:16.356 INFO:tasks.workunit.client.0.smithi037.stdout:6/249: rename d0/f48 to d0/d19/d34/d4e/f54 0 2024-03-29T08:07:16.357 INFO:tasks.workunit.client.0.smithi037.stdout:9/494: write d1/d1f/d3a/f78 [968531,72216] 0 2024-03-29T08:07:16.358 INFO:tasks.workunit.client.0.smithi037.stdout:7/343: fsync dd/d21/d35/d42/d5c/f5e 0 2024-03-29T08:07:16.360 INFO:tasks.workunit.client.1.smithi148.stdout:8/458: write d17/d18/f3e [679073,20820] 0 2024-03-29T08:07:16.366 INFO:tasks.workunit.client.1.smithi148.stdout:6/376: getdents d6/d9/d1e 0 2024-03-29T08:07:16.367 INFO:tasks.workunit.client.0.smithi037.stdout:5/432: mkdir d8/d22/d4f/d70/d79/d7a/d85/d90 0 2024-03-29T08:07:16.368 INFO:tasks.workunit.client.1.smithi148.stdout:6/377: write d6/d9/d1e/d22/f34 [3362228,100858] 0 2024-03-29T08:07:16.369 INFO:tasks.workunit.client.1.smithi148.stdout:9/384: sync 2024-03-29T08:07:16.369 INFO:tasks.workunit.client.1.smithi148.stdout:5/420: dread d0/db/d7b/d16/f2f [0,4194304] 0 2024-03-29T08:07:16.370 INFO:tasks.workunit.client.0.smithi037.stdout:0/184: creat d9/da/dd/d13/f36 x:0 0 0 2024-03-29T08:07:16.372 INFO:tasks.workunit.client.0.smithi037.stdout:5/433: write d8/d22/d4f/d70/f82 [742686,69136] 0 2024-03-29T08:07:16.373 INFO:tasks.workunit.client.1.smithi148.stdout:2/415: readlink d7/da/l1d 0 2024-03-29T08:07:16.375 INFO:tasks.workunit.client.1.smithi148.stdout:0/411: rmdir db/de/d75 0 2024-03-29T08:07:16.378 INFO:tasks.workunit.client.0.smithi037.stdout:9/495: rmdir d1/d4/d19/d1e/d25/d16 39 2024-03-29T08:07:16.380 INFO:tasks.workunit.client.1.smithi148.stdout:6/378: mkdir d6/d9/d1e/d20/d6b 0 2024-03-29T08:07:16.380 INFO:tasks.workunit.client.0.smithi037.stdout:5/434: fdatasync d8/f42 0 2024-03-29T08:07:16.380 INFO:tasks.workunit.client.0.smithi037.stdout:8/337: fsync d5/f1b 0 2024-03-29T08:07:16.382 INFO:tasks.workunit.client.1.smithi148.stdout:4/384: creat df/d2c/d51/d77/f83 x:0 0 0 2024-03-29T08:07:16.385 INFO:tasks.workunit.client.1.smithi148.stdout:6/379: write d6/d9/d1e/d27/f64 [559455,122476] 0 2024-03-29T08:07:16.385 INFO:tasks.workunit.client.1.smithi148.stdout:0/412: symlink db/de/df/d14/l82 0 2024-03-29T08:07:16.388 INFO:tasks.workunit.client.0.smithi037.stdout:8/338: dread - d5/d25/d46/d4e/f66 zero size 2024-03-29T08:07:16.389 INFO:tasks.workunit.client.0.smithi037.stdout:7/344: dwrite dd/f3e [0,4194304] 0 2024-03-29T08:07:16.391 INFO:tasks.workunit.client.1.smithi148.stdout:5/421: dread d0/db/d7b/d16/f2f [0,4194304] 0 2024-03-29T08:07:16.391 INFO:tasks.workunit.client.1.smithi148.stdout:0/413: dread db/de/df/d2b/f5b [0,4194304] 0 2024-03-29T08:07:16.393 INFO:tasks.workunit.client.0.smithi037.stdout:2/206: rename d0/fb to d0/d23/d44/f45 0 2024-03-29T08:07:16.394 INFO:tasks.workunit.client.1.smithi148.stdout:0/414: chown db/de/df/d2b/f76 16 1 2024-03-29T08:07:16.398 INFO:tasks.workunit.client.1.smithi148.stdout:1/392: write d3/fa [3987968,52964] 0 2024-03-29T08:07:16.399 INFO:tasks.workunit.client.1.smithi148.stdout:6/380: mknod d6/d69/c6c 0 2024-03-29T08:07:16.403 INFO:tasks.workunit.client.1.smithi148.stdout:9/385: dwrite db/d22/d26/f2a [0,4194304] 0 2024-03-29T08:07:16.404 INFO:tasks.workunit.client.1.smithi148.stdout:8/459: truncate d17/d18/f3e 294671 0 2024-03-29T08:07:16.405 INFO:tasks.workunit.client.0.smithi037.stdout:8/339: write d5/d10/d2d/f3e [142315,71726] 0 2024-03-29T08:07:16.406 INFO:tasks.workunit.client.0.smithi037.stdout:3/297: truncate dd/d46/d2d/d3b/f4f 1227138 0 2024-03-29T08:07:16.406 INFO:tasks.workunit.client.1.smithi148.stdout:2/416: dread d7/da/d33/d45/d4d/f38 [0,4194304] 0 2024-03-29T08:07:16.406 INFO:tasks.workunit.client.0.smithi037.stdout:4/304: write d3/d4/d15/d1d/db/d45/d1e/f4a [3947576,58816] 0 2024-03-29T08:07:16.408 INFO:tasks.workunit.client.1.smithi148.stdout:4/385: rename fc to df/d2c/d51/d77/d26/f84 0 2024-03-29T08:07:16.410 INFO:tasks.workunit.client.1.smithi148.stdout:0/415: mknod db/c83 0 2024-03-29T08:07:16.411 INFO:tasks.workunit.client.1.smithi148.stdout:1/393: mkdir d3/d18/d71 0 2024-03-29T08:07:16.413 INFO:tasks.workunit.client.1.smithi148.stdout:6/381: mknod d6/d9/d1e/d20/d30/d51/c6d 0 2024-03-29T08:07:16.414 INFO:tasks.workunit.client.1.smithi148.stdout:1/394: truncate d3/d18/f39 946677 0 2024-03-29T08:07:16.414 INFO:tasks.workunit.client.0.smithi037.stdout:0/185: dwrite d9/da/dd/f23 [4194304,4194304] 0 2024-03-29T08:07:16.414 INFO:tasks.workunit.client.0.smithi037.stdout:6/250: rename d0/d5/d1f/f35 to d0/d20/d38/f55 0 2024-03-29T08:07:16.416 INFO:tasks.workunit.client.1.smithi148.stdout:1/395: fdatasync d3/d55/d44/f6b 0 2024-03-29T08:07:16.422 INFO:tasks.workunit.client.1.smithi148.stdout:6/382: creat d6/d9/d1e/d27/f6e x:0 0 0 2024-03-29T08:07:16.423 INFO:tasks.workunit.client.1.smithi148.stdout:4/386: stat df/d2c/d51/d77/d17/f1c 0 2024-03-29T08:07:16.423 INFO:tasks.workunit.client.0.smithi037.stdout:2/207: creat d0/d23/d44/f46 x:0 0 0 2024-03-29T08:07:16.425 INFO:tasks.workunit.client.0.smithi037.stdout:5/435: mknod d8/c91 0 2024-03-29T08:07:16.426 INFO:tasks.workunit.client.1.smithi148.stdout:2/417: dwrite d7/da/d33/d3c/d54/d70/f7b [0,4194304] 0 2024-03-29T08:07:16.427 INFO:tasks.workunit.client.1.smithi148.stdout:9/386: symlink db/d10/d2d/l7f 0 2024-03-29T08:07:16.429 INFO:tasks.workunit.client.0.smithi037.stdout:1/367: dwrite d2/d6/dc/d18/d1d/d21/d38/f50 [0,4194304] 0 2024-03-29T08:07:16.430 INFO:tasks.workunit.client.0.smithi037.stdout:0/186: chown d9/da/d1b/d33 185597189 1 2024-03-29T08:07:16.430 INFO:tasks.workunit.client.1.smithi148.stdout:0/416: link db/de/df/d14/f3c db/de/df/d2b/d3b/f84 0 2024-03-29T08:07:16.434 INFO:tasks.workunit.client.1.smithi148.stdout:4/387: creat df/d60/f85 x:0 0 0 2024-03-29T08:07:16.437 INFO:tasks.workunit.client.1.smithi148.stdout:2/418: write d7/da/f3f [2006802,49003] 0 2024-03-29T08:07:16.439 INFO:tasks.workunit.client.0.smithi037.stdout:3/298: write f4 [5233011,35952] 0 2024-03-29T08:07:16.441 INFO:tasks.workunit.client.0.smithi037.stdout:9/496: dwrite d1/d4/d19/f64 [0,4194304] 0 2024-03-29T08:07:16.442 INFO:tasks.workunit.client.1.smithi148.stdout:8/460: read d17/d18/f4d [39770,22537] 0 2024-03-29T08:07:16.443 INFO:tasks.workunit.client.1.smithi148.stdout:9/387: read db/d20/f1e [224488,101349] 0 2024-03-29T08:07:16.446 INFO:tasks.workunit.client.1.smithi148.stdout:1/396: mkdir d3/de/d1d/d47/d6d/d72 0 2024-03-29T08:07:16.446 INFO:tasks.workunit.client.0.smithi037.stdout:1/368: stat d2/d6/dc/d18/d1d/d5e/d72 0 2024-03-29T08:07:16.447 INFO:tasks.workunit.client.1.smithi148.stdout:1/397: chown d3/d18/d4d 490937406 1 2024-03-29T08:07:16.449 INFO:tasks.workunit.client.0.smithi037.stdout:7/345: creat dd/d21/d35/d60/f6e x:0 0 0 2024-03-29T08:07:16.452 INFO:tasks.workunit.client.0.smithi037.stdout:6/251: symlink d0/d20/l56 0 2024-03-29T08:07:16.453 INFO:tasks.workunit.client.1.smithi148.stdout:9/388: unlink db/d22/d26/f5d 0 2024-03-29T08:07:16.456 INFO:tasks.workunit.client.0.smithi037.stdout:9/497: write d1/d1f/d3a/f46 [2465110,130789] 0 2024-03-29T08:07:16.457 INFO:tasks.workunit.client.0.smithi037.stdout:4/305: rename d3/d4/d15/l2a to d3/d4/d15/d1d/db/d45/d1e/d59/l69 0 2024-03-29T08:07:16.458 INFO:tasks.workunit.client.1.smithi148.stdout:1/398: creat d3/de/d25/d51/f73 x:0 0 0 2024-03-29T08:07:16.461 INFO:tasks.workunit.client.0.smithi037.stdout:2/208: creat d0/d20/d22/f47 x:0 0 0 2024-03-29T08:07:16.461 INFO:tasks.workunit.client.0.smithi037.stdout:3/299: read - dd/d46/d2d/f42 zero size 2024-03-29T08:07:16.464 INFO:tasks.workunit.client.1.smithi148.stdout:8/461: mknod d17/d74/d39/c96 0 2024-03-29T08:07:16.464 INFO:tasks.workunit.client.0.smithi037.stdout:2/209: readlink d0/l42 0 2024-03-29T08:07:16.465 INFO:tasks.workunit.client.1.smithi148.stdout:7/414: sync 2024-03-29T08:07:16.468 INFO:tasks.workunit.client.0.smithi037.stdout:8/340: dread d5/f16 [0,4194304] 0 2024-03-29T08:07:16.474 INFO:tasks.workunit.client.1.smithi148.stdout:4/388: truncate df/d2c/d51/d77/d26/d34/d3e/f58 270640 0 2024-03-29T08:07:16.475 INFO:tasks.workunit.client.1.smithi148.stdout:1/399: creat d3/d18/d4d/f74 x:0 0 0 2024-03-29T08:07:16.475 INFO:tasks.workunit.client.1.smithi148.stdout:0/417: creat db/de/df/d11/d2d/f85 x:0 0 0 2024-03-29T08:07:16.475 INFO:tasks.workunit.client.0.smithi037.stdout:4/306: write d3/d4/f35 [4670964,131065] 0 2024-03-29T08:07:16.478 INFO:tasks.workunit.client.0.smithi037.stdout:3/300: mkdir dd/d46/d3e/d64 0 2024-03-29T08:07:16.480 INFO:tasks.workunit.client.0.smithi037.stdout:2/210: creat d0/d20/d22/f48 x:0 0 0 2024-03-29T08:07:16.482 INFO:tasks.workunit.client.0.smithi037.stdout:4/307: dread - d3/d4/d15/d1d/db/d45/f5a zero size 2024-03-29T08:07:16.482 INFO:tasks.workunit.client.1.smithi148.stdout:8/462: read d17/f1a [1291548,99273] 0 2024-03-29T08:07:16.485 INFO:tasks.workunit.client.0.smithi037.stdout:7/346: link dd/d21/f32 dd/d21/d65/f6f 0 2024-03-29T08:07:16.486 INFO:tasks.workunit.client.1.smithi148.stdout:9/389: symlink db/d22/d26/d54/l80 0 2024-03-29T08:07:16.487 INFO:tasks.workunit.client.1.smithi148.stdout:9/390: dread - db/d10/d2d/f7b zero size 2024-03-29T08:07:16.490 INFO:tasks.workunit.client.0.smithi037.stdout:9/498: creat d1/d4/d19/d1e/d4a/d6f/da2/d13/d7c/faf x:0 0 0 2024-03-29T08:07:16.490 INFO:tasks.workunit.client.0.smithi037.stdout:8/341: write d5/d25/d46/d4e/f6a [65464,15270] 0 2024-03-29T08:07:16.491 INFO:tasks.workunit.client.1.smithi148.stdout:4/389: creat df/d2c/f86 x:0 0 0 2024-03-29T08:07:16.491 INFO:tasks.workunit.client.1.smithi148.stdout:9/391: dread - db/d22/d26/d54/f60 zero size 2024-03-29T08:07:16.492 INFO:tasks.workunit.client.0.smithi037.stdout:2/211: rmdir d0/d23/d44 39 2024-03-29T08:07:16.493 INFO:tasks.workunit.client.1.smithi148.stdout:6/383: dwrite f5 [0,4194304] 0 2024-03-29T08:07:16.494 INFO:tasks.workunit.client.0.smithi037.stdout:7/347: mkdir dd/d21/d70 0 2024-03-29T08:07:16.495 INFO:tasks.workunit.client.1.smithi148.stdout:0/418: chown db/dc/d78/c80 0 1 2024-03-29T08:07:16.496 INFO:tasks.workunit.client.0.smithi037.stdout:8/342: chown d5/d10/f1d 137111 1 2024-03-29T08:07:16.497 INFO:tasks.workunit.client.0.smithi037.stdout:6/252: getdents d0/d5/d1e 0 2024-03-29T08:07:16.500 INFO:tasks.workunit.client.0.smithi037.stdout:4/308: truncate d3/d4/d15/d1d/f64 661749 0 2024-03-29T08:07:16.505 INFO:tasks.workunit.client.1.smithi148.stdout:8/463: mknod d17/d18/d26/d63/c97 0 2024-03-29T08:07:16.506 INFO:tasks.workunit.client.0.smithi037.stdout:9/499: fdatasync d1/d4/f28 0 2024-03-29T08:07:16.507 INFO:tasks.workunit.client.1.smithi148.stdout:0/419: dread db/de/d24/f38 [0,4194304] 0 2024-03-29T08:07:16.510 INFO:tasks.workunit.client.1.smithi148.stdout:9/392: readlink db/l1f 0 2024-03-29T08:07:16.510 INFO:tasks.workunit.client.0.smithi037.stdout:6/253: write d0/d5/d1e/f12 [1383581,21321] 0 2024-03-29T08:07:16.514 INFO:tasks.workunit.client.1.smithi148.stdout:4/390: mknod df/d2c/d51/d77/d26/d34/d43/c87 0 2024-03-29T08:07:16.519 INFO:tasks.workunit.client.0.smithi037.stdout:1/369: dwrite d2/d6/dc/f15 [0,4194304] 0 2024-03-29T08:07:16.522 INFO:tasks.workunit.client.0.smithi037.stdout:5/436: dread d8/d34/f4a [0,4194304] 0 2024-03-29T08:07:16.524 INFO:tasks.workunit.client.1.smithi148.stdout:3/428: sync 2024-03-29T08:07:16.528 INFO:tasks.workunit.client.1.smithi148.stdout:1/400: creat d3/de/d25/f75 x:0 0 0 2024-03-29T08:07:16.528 INFO:tasks.workunit.client.0.smithi037.stdout:2/212: creat d0/d20/f49 x:0 0 0 2024-03-29T08:07:16.529 INFO:tasks.workunit.client.1.smithi148.stdout:9/393: creat db/d10/d2d/d4e/f81 x:0 0 0 2024-03-29T08:07:16.535 INFO:tasks.workunit.client.1.smithi148.stdout:7/415: truncate d8/d29/f35 2864013 0 2024-03-29T08:07:16.536 INFO:tasks.workunit.client.0.smithi037.stdout:4/309: mkdir d3/d57/d6a 0 2024-03-29T08:07:16.539 INFO:tasks.workunit.client.0.smithi037.stdout:6/254: fdatasync d0/d20/f31 0 2024-03-29T08:07:16.539 INFO:tasks.workunit.client.0.smithi037.stdout:8/343: mkdir d5/d10/d19/d3d/d4a/d4c/d67/d70 0 2024-03-29T08:07:16.540 INFO:tasks.workunit.client.0.smithi037.stdout:6/255: stat d0/d4f 0 2024-03-29T08:07:16.542 INFO:tasks.workunit.client.0.smithi037.stdout:1/370: write d2/d6/dc/d18/d22/d4f/f69 [38563,61174] 0 2024-03-29T08:07:16.546 INFO:tasks.workunit.client.1.smithi148.stdout:0/420: rename db/de/df/d11/d2d/c63 to db/de/df/d11/d2d/c86 0 2024-03-29T08:07:16.546 INFO:tasks.workunit.client.0.smithi037.stdout:1/371: chown d2/d6/dc/d18/d22/c3b 306 1 2024-03-29T08:07:16.550 INFO:tasks.workunit.client.0.smithi037.stdout:3/301: dread dd/d15/d27/f41 [0,4194304] 0 2024-03-29T08:07:16.552 INFO:tasks.workunit.client.0.smithi037.stdout:4/310: mkdir d3/d57/d6b 0 2024-03-29T08:07:16.554 INFO:tasks.workunit.client.1.smithi148.stdout:8/464: dwrite d17/d18/f5c [0,4194304] 0 2024-03-29T08:07:16.554 INFO:tasks.workunit.client.0.smithi037.stdout:8/344: symlink d5/d10/d2d/l71 0 2024-03-29T08:07:16.554 INFO:tasks.workunit.client.0.smithi037.stdout:2/213: rename d0/d20/d2e/l33 to d0/d20/d2e/l4a 0 2024-03-29T08:07:16.554 INFO:tasks.workunit.client.0.smithi037.stdout:3/302: fdatasync dd/d46/d2d/d3a/d5f/f5d 0 2024-03-29T08:07:16.557 INFO:tasks.workunit.client.1.smithi148.stdout:0/421: fsync db/de/d24/f38 0 2024-03-29T08:07:16.558 INFO:tasks.workunit.client.0.smithi037.stdout:6/256: rename d0/d5/d1a/f23 to d0/d4f/f57 0 2024-03-29T08:07:16.558 INFO:tasks.workunit.client.0.smithi037.stdout:1/372: rmdir d2/d6/dc/d18 39 2024-03-29T08:07:16.561 INFO:tasks.workunit.client.0.smithi037.stdout:6/257: chown d0/l2 457343 1 2024-03-29T08:07:16.561 INFO:tasks.workunit.client.1.smithi148.stdout:3/429: dwrite de/d3b/d49/f90 [0,4194304] 0 2024-03-29T08:07:16.562 INFO:tasks.workunit.client.0.smithi037.stdout:2/214: symlink d0/d20/d2e/d38/l4b 0 2024-03-29T08:07:16.563 INFO:tasks.workunit.client.1.smithi148.stdout:3/430: dread - de/f80 zero size 2024-03-29T08:07:16.563 INFO:tasks.workunit.client.0.smithi037.stdout:4/311: mknod d3/d4/d15/d1d/db/c6c 0 2024-03-29T08:07:16.564 INFO:tasks.workunit.client.1.smithi148.stdout:9/394: dread db/d10/d2d/d4e/f6c [0,4194304] 0 2024-03-29T08:07:16.569 INFO:tasks.workunit.client.0.smithi037.stdout:8/345: mknod d5/d10/d19/d3d/d4a/d4c/d67/c72 0 2024-03-29T08:07:16.571 INFO:tasks.workunit.client.0.smithi037.stdout:4/312: dread - d3/d4/d15/d1d/db/d3d/f58 zero size 2024-03-29T08:07:16.571 INFO:tasks.workunit.client.1.smithi148.stdout:3/431: dread de/d24/d33/f5c [0,4194304] 0 2024-03-29T08:07:16.573 INFO:tasks.workunit.client.0.smithi037.stdout:4/313: read - d3/d4/f46 zero size 2024-03-29T08:07:16.575 INFO:tasks.workunit.client.1.smithi148.stdout:6/384: dread d6/d9/d1e/d20/f39 [0,4194304] 0 2024-03-29T08:07:16.576 INFO:tasks.workunit.client.0.smithi037.stdout:6/258: symlink d0/d4f/l58 0 2024-03-29T08:07:16.578 INFO:tasks.workunit.client.1.smithi148.stdout:3/432: chown de/d24/d33/d39/d46/d30/d37/f6f 55687 1 2024-03-29T08:07:16.580 INFO:tasks.workunit.client.0.smithi037.stdout:5/437: dwrite d8/d18/d24/d45/f84 [0,4194304] 0 2024-03-29T08:07:16.580 INFO:tasks.workunit.client.1.smithi148.stdout:6/385: write f5 [3650909,18087] 0 2024-03-29T08:07:16.581 INFO:tasks.workunit.client.1.smithi148.stdout:8/465: dwrite fe [0,4194304] 0 2024-03-29T08:07:16.584 INFO:tasks.workunit.client.0.smithi037.stdout:4/314: write d3/d4/d15/d1d/f64 [1008278,47157] 0 2024-03-29T08:07:16.585 INFO:tasks.workunit.client.1.smithi148.stdout:9/395: rmdir db/d22 39 2024-03-29T08:07:16.588 INFO:tasks.workunit.client.0.smithi037.stdout:1/373: dread d2/d6/d13/f28 [0,4194304] 0 2024-03-29T08:07:16.590 INFO:tasks.workunit.client.0.smithi037.stdout:8/346: creat d5/d10/d19/d3d/d4a/d4c/f73 x:0 0 0 2024-03-29T08:07:16.597 INFO:tasks.workunit.client.0.smithi037.stdout:6/259: symlink d0/d19/d34/l59 0 2024-03-29T08:07:16.600 INFO:tasks.workunit.client.1.smithi148.stdout:4/391: dread df/f1e [0,4194304] 0 2024-03-29T08:07:16.600 INFO:tasks.workunit.client.1.smithi148.stdout:0/422: dwrite db/de/df/d11/f16 [0,4194304] 0 2024-03-29T08:07:16.601 INFO:tasks.workunit.client.1.smithi148.stdout:9/396: fdatasync db/d22/f51 0 2024-03-29T08:07:16.605 INFO:tasks.workunit.client.0.smithi037.stdout:5/438: creat d8/d22/d4f/d70/d79/f92 x:0 0 0 2024-03-29T08:07:16.607 INFO:tasks.workunit.client.1.smithi148.stdout:1/401: dwrite d3/d18/f20 [0,4194304] 0 2024-03-29T08:07:16.607 INFO:tasks.workunit.client.0.smithi037.stdout:9/500: truncate d1/d1f/d3a/f27 2254043 0 2024-03-29T08:07:16.607 INFO:tasks.workunit.client.1.smithi148.stdout:6/386: creat d6/d9/d1e/d36/f6f x:0 0 0 2024-03-29T08:07:16.608 INFO:tasks.workunit.client.1.smithi148.stdout:2/419: sync 2024-03-29T08:07:16.608 INFO:tasks.workunit.client.1.smithi148.stdout:5/422: sync 2024-03-29T08:07:16.609 INFO:tasks.workunit.client.0.smithi037.stdout:2/215: rename d0/d20/c2a to d0/d23/d44/c4c 0 2024-03-29T08:07:16.611 INFO:tasks.workunit.client.1.smithi148.stdout:3/433: rename de/d3b/f4e to de/d24/d33/d39/d46/d30/d37/d62/d63/f92 0 2024-03-29T08:07:16.613 INFO:tasks.workunit.client.0.smithi037.stdout:9/501: readlink d1/d4/d19/d1e/d4a/d5c/l6a 0 2024-03-29T08:07:16.613 INFO:tasks.workunit.client.0.smithi037.stdout:3/303: link dd/f3f dd/d5e/f65 0 2024-03-29T08:07:16.613 INFO:tasks.workunit.client.1.smithi148.stdout:7/416: dwrite f4 [0,4194304] 0 2024-03-29T08:07:16.614 INFO:tasks.workunit.client.0.smithi037.stdout:8/347: mknod d5/d10/d2d/d34/d50/c74 0 2024-03-29T08:07:16.614 INFO:tasks.workunit.client.1.smithi148.stdout:2/420: chown d7/f10 597 1 2024-03-29T08:07:16.615 INFO:tasks.workunit.client.1.smithi148.stdout:3/434: stat de/l3e 0 2024-03-29T08:07:16.616 INFO:tasks.workunit.client.1.smithi148.stdout:9/397: write db/d10/d2d/f34 [690728,54993] 0 2024-03-29T08:07:16.617 INFO:tasks.workunit.client.0.smithi037.stdout:6/260: unlink d0/d5/d1f/c2b 0 2024-03-29T08:07:16.619 INFO:tasks.workunit.client.1.smithi148.stdout:6/387: readlink d6/l43 0 2024-03-29T08:07:16.620 INFO:tasks.workunit.client.1.smithi148.stdout:5/423: readlink d0/db/d1e/l77 0 2024-03-29T08:07:16.621 INFO:tasks.workunit.client.1.smithi148.stdout:8/466: creat d17/d18/d26/f98 x:0 0 0 2024-03-29T08:07:16.621 INFO:tasks.workunit.client.0.smithi037.stdout:4/315: truncate d3/f3a 855354 0 2024-03-29T08:07:16.627 INFO:tasks.workunit.client.0.smithi037.stdout:1/374: fdatasync d2/d6/dc/d18/d1d/d21/f47 0 2024-03-29T08:07:16.629 INFO:tasks.workunit.client.1.smithi148.stdout:7/417: read - d8/d29/d44/d49/f6f zero size 2024-03-29T08:07:16.634 INFO:tasks.workunit.client.0.smithi037.stdout:3/304: chown dd/f1b 713616 1 2024-03-29T08:07:16.635 INFO:tasks.workunit.client.0.smithi037.stdout:6/261: rename d0/d20/d38 to d0/d19/d34/d5a 0 2024-03-29T08:07:16.637 INFO:tasks.workunit.client.1.smithi148.stdout:3/435: creat de/d3b/f93 x:0 0 0 2024-03-29T08:07:16.637 INFO:tasks.workunit.client.1.smithi148.stdout:4/392: dread df/d2c/f6c [0,4194304] 0 2024-03-29T08:07:16.637 INFO:tasks.workunit.client.1.smithi148.stdout:9/398: write db/f21 [1608347,127988] 0 2024-03-29T08:07:16.639 INFO:tasks.workunit.client.0.smithi037.stdout:5/439: creat d8/d22/f93 x:0 0 0 2024-03-29T08:07:16.641 INFO:tasks.workunit.client.0.smithi037.stdout:9/502: truncate d1/d4/d19/d1e/d4a/d6f/da2/d31/f48 291924 0 2024-03-29T08:07:16.642 INFO:tasks.workunit.client.1.smithi148.stdout:5/424: chown d0/db/d1e/f31 64940 1 2024-03-29T08:07:16.646 INFO:tasks.workunit.client.0.smithi037.stdout:3/305: write dd/d15/f19 [919734,18134] 0 2024-03-29T08:07:16.647 INFO:tasks.workunit.client.1.smithi148.stdout:4/393: creat df/d71/f88 x:0 0 0 2024-03-29T08:07:16.647 INFO:tasks.workunit.client.0.smithi037.stdout:1/375: creat d2/d6/dc/d18/d1d/d5e/d72/f7b x:0 0 0 2024-03-29T08:07:16.649 INFO:tasks.workunit.client.1.smithi148.stdout:2/421: truncate d7/da/d33/d45/d4d/f16 582933 0 2024-03-29T08:07:16.649 INFO:tasks.workunit.client.0.smithi037.stdout:3/306: readlink dd/d15/l1c 0 2024-03-29T08:07:16.649 INFO:tasks.workunit.client.1.smithi148.stdout:6/388: creat d6/f70 x:0 0 0 2024-03-29T08:07:16.650 INFO:tasks.workunit.client.0.smithi037.stdout:5/440: creat d8/d4d/d5c/f94 x:0 0 0 2024-03-29T08:07:16.656 INFO:tasks.workunit.client.1.smithi148.stdout:0/423: truncate db/de/df/d11/f16 988730 0 2024-03-29T08:07:16.657 INFO:tasks.workunit.client.0.smithi037.stdout:8/348: dwrite d5/d10/d2d/f60 [0,4194304] 0 2024-03-29T08:07:16.657 INFO:tasks.workunit.client.0.smithi037.stdout:2/216: write d0/d23/f26 [835886,117123] 0 2024-03-29T08:07:16.658 INFO:tasks.workunit.client.0.smithi037.stdout:3/307: write dd/d46/f2c [805644,70743] 0 2024-03-29T08:07:16.658 INFO:tasks.workunit.client.0.smithi037.stdout:9/503: rmdir d1/d4/d19/d1e/d4a/d6f/da2/d68 39 2024-03-29T08:07:16.659 INFO:tasks.workunit.client.1.smithi148.stdout:4/394: chown df/d2c/d51/d77/d26/f63 57640224 1 2024-03-29T08:07:16.660 INFO:tasks.workunit.client.1.smithi148.stdout:3/436: creat de/d24/d33/f94 x:0 0 0 2024-03-29T08:07:16.663 INFO:tasks.workunit.client.0.smithi037.stdout:1/376: mkdir d2/d6/dc/d18/d22/d4f/d7c 0 2024-03-29T08:07:16.663 INFO:tasks.workunit.client.1.smithi148.stdout:9/399: fsync db/d22/d26/d54/f49 0 2024-03-29T08:07:16.665 INFO:tasks.workunit.client.0.smithi037.stdout:8/349: readlink d5/l29 0 2024-03-29T08:07:16.665 INFO:tasks.workunit.client.1.smithi148.stdout:6/389: mknod d6/d9/d1e/d36/c71 0 2024-03-29T08:07:16.668 INFO:tasks.workunit.client.1.smithi148.stdout:7/418: dwrite d8/f36 [0,4194304] 0 2024-03-29T08:07:16.670 INFO:tasks.workunit.client.0.smithi037.stdout:3/308: chown dd/fe 43 1 2024-03-29T08:07:16.672 INFO:tasks.workunit.client.1.smithi148.stdout:6/390: stat d6/d9 0 2024-03-29T08:07:16.673 INFO:tasks.workunit.client.1.smithi148.stdout:5/425: rename d0/db/d7b/f53 to d0/db/d1e/d35/d6a/f88 0 2024-03-29T08:07:16.673 INFO:tasks.workunit.client.1.smithi148.stdout:8/467: dwrite d17/d1d/f79 [0,4194304] 0 2024-03-29T08:07:16.674 INFO:tasks.workunit.client.0.smithi037.stdout:8/350: truncate d5/f48 458833 0 2024-03-29T08:07:16.676 INFO:tasks.workunit.client.1.smithi148.stdout:2/422: write d7/da/d33/d45/d4d/f38 [2411492,98500] 0 2024-03-29T08:07:16.677 INFO:tasks.workunit.client.0.smithi037.stdout:9/504: rmdir d1/d4/d19/d1e/d4a 39 2024-03-29T08:07:16.678 INFO:tasks.workunit.client.0.smithi037.stdout:2/217: truncate d0/f8 466031 0 2024-03-29T08:07:16.680 INFO:tasks.workunit.client.1.smithi148.stdout:4/395: fdatasync df/d2c/d51/d77/d26/f3b 0 2024-03-29T08:07:16.681 INFO:tasks.workunit.client.1.smithi148.stdout:0/424: rmdir db/de/df/d11/d2d 39 2024-03-29T08:07:16.682 INFO:tasks.workunit.client.0.smithi037.stdout:6/262: write d0/f33 [653807,17733] 0 2024-03-29T08:07:16.684 INFO:tasks.workunit.client.0.smithi037.stdout:3/309: symlink dd/d46/d2d/d3a/d5f/l66 0 2024-03-29T08:07:16.686 INFO:tasks.workunit.client.0.smithi037.stdout:6/263: chown d0/d5/d3f 2187745 1 2024-03-29T08:07:16.690 INFO:tasks.workunit.client.0.smithi037.stdout:3/310: chown dd/d15/d18/f2f 1318 1 2024-03-29T08:07:16.692 INFO:tasks.workunit.client.0.smithi037.stdout:4/316: dread d3/d4/d15/f1f [0,4194304] 0 2024-03-29T08:07:16.693 INFO:tasks.workunit.client.1.smithi148.stdout:4/396: fsync df/d2c/d51/d77/d26/d34/f61 0 2024-03-29T08:07:16.693 INFO:tasks.workunit.client.1.smithi148.stdout:9/400: symlink db/d22/d3f/l82 0 2024-03-29T08:07:16.694 INFO:tasks.workunit.client.1.smithi148.stdout:1/402: dread d3/d55/f37 [0,4194304] 0 2024-03-29T08:07:16.697 INFO:tasks.workunit.client.1.smithi148.stdout:0/425: mknod db/dc/c87 0 2024-03-29T08:07:16.700 INFO:tasks.workunit.client.1.smithi148.stdout:5/426: symlink d0/db/d1e/d79/l89 0 2024-03-29T08:07:16.701 INFO:tasks.workunit.client.0.smithi037.stdout:6/264: mknod d0/d19/d34/d4e/c5b 0 2024-03-29T08:07:16.702 INFO:tasks.workunit.client.0.smithi037.stdout:4/317: dread d3/d4/d15/d1d/db/f12 [0,4194304] 0 2024-03-29T08:07:16.702 INFO:tasks.workunit.client.1.smithi148.stdout:2/423: write d7/f7c [2293408,85989] 0 2024-03-29T08:07:16.703 INFO:tasks.workunit.client.0.smithi037.stdout:3/311: creat dd/d15/d18/d4b/f67 x:0 0 0 2024-03-29T08:07:16.707 INFO:tasks.workunit.client.1.smithi148.stdout:4/397: mkdir df/d89 0 2024-03-29T08:07:16.708 INFO:tasks.workunit.client.1.smithi148.stdout:2/424: write d7/da/d33/d3c/f48 [4942908,22605] 0 2024-03-29T08:07:16.709 INFO:tasks.workunit.client.0.smithi037.stdout:4/318: chown d3/d4/d15/d1d/d4f 12 1 2024-03-29T08:07:16.714 INFO:tasks.workunit.client.0.smithi037.stdout:9/505: dread d1/d4/d19/d1e/d4a/d6f/da2/d31/f48 [0,4194304] 0 2024-03-29T08:07:16.715 INFO:tasks.workunit.client.1.smithi148.stdout:3/437: dwrite de/d3b/f74 [0,4194304] 0 2024-03-29T08:07:16.716 INFO:tasks.workunit.client.0.smithi037.stdout:4/319: fsync d3/d4/d15/d1d/db/d3d/f56 0 2024-03-29T08:07:16.716 INFO:tasks.workunit.client.1.smithi148.stdout:7/419: write d8/d29/d44/d49/f6f [812100,57498] 0 2024-03-29T08:07:16.716 INFO:tasks.workunit.client.1.smithi148.stdout:6/391: dwrite d6/d9/d1e/d22/f4e [0,4194304] 0 2024-03-29T08:07:16.717 INFO:tasks.workunit.client.1.smithi148.stdout:4/398: unlink df/d71/f72 0 2024-03-29T08:07:16.718 INFO:tasks.workunit.client.1.smithi148.stdout:8/468: dread d17/d74/f48 [4194304,4194304] 0 2024-03-29T08:07:16.719 INFO:tasks.workunit.client.1.smithi148.stdout:5/427: truncate d0/db/d1e/d35/d42/d67/f70 1717699 0 2024-03-29T08:07:16.720 INFO:tasks.workunit.client.0.smithi037.stdout:5/441: dwrite d8/d18/d24/d45/f51 [4194304,4194304] 0 2024-03-29T08:07:16.721 INFO:tasks.workunit.client.1.smithi148.stdout:0/426: chown db/de/df/d11/d2d/d3e 129 1 2024-03-29T08:07:16.722 INFO:tasks.workunit.client.1.smithi148.stdout:2/425: rmdir d7/da/d4c 39 2024-03-29T08:07:16.723 INFO:tasks.workunit.client.0.smithi037.stdout:4/320: dread - d3/d4/d15/f63 zero size 2024-03-29T08:07:16.725 INFO:tasks.workunit.client.1.smithi148.stdout:4/399: unlink df/f45 0 2024-03-29T08:07:16.726 INFO:tasks.workunit.client.1.smithi148.stdout:7/420: mknod d8/d38/d6a/c92 0 2024-03-29T08:07:16.729 INFO:tasks.workunit.client.1.smithi148.stdout:6/392: symlink d6/d9/d1e/d20/d30/l72 0 2024-03-29T08:07:16.729 INFO:tasks.workunit.client.1.smithi148.stdout:5/428: dread - d0/db/d1e/d35/d42/f5e zero size 2024-03-29T08:07:16.732 INFO:tasks.workunit.client.1.smithi148.stdout:8/469: mknod d17/d18/d26/d63/d66/d5f/c99 0 2024-03-29T08:07:16.736 INFO:tasks.workunit.client.1.smithi148.stdout:6/393: write d6/d9/f52 [287265,63361] 0 2024-03-29T08:07:16.737 INFO:tasks.workunit.client.1.smithi148.stdout:1/403: dwrite d3/d55/d44/f53 [0,4194304] 0 2024-03-29T08:07:16.742 INFO:tasks.workunit.client.0.smithi037.stdout:8/351: dwrite d5/f1c [0,4194304] 0 2024-03-29T08:07:16.744 INFO:tasks.workunit.client.0.smithi037.stdout:4/321: mknod d3/d4/d15/d1d/db/c6d 0 2024-03-29T08:07:16.745 INFO:tasks.workunit.client.0.smithi037.stdout:8/352: fsync d5/d25/f28 0 2024-03-29T08:07:16.745 INFO:tasks.workunit.client.0.smithi037.stdout:6/265: dwrite d0/d19/d34/d5a/f39 [0,4194304] 0 2024-03-29T08:07:16.748 INFO:tasks.workunit.client.1.smithi148.stdout:8/470: chown d17/d18/d26/d63/d66/d5f/c99 25094441 1 2024-03-29T08:07:16.750 INFO:tasks.workunit.client.1.smithi148.stdout:8/471: write f15 [4945243,61280] 0 2024-03-29T08:07:16.751 INFO:tasks.workunit.client.0.smithi037.stdout:5/442: link d8/d22/d4f/l5f d8/d22/d37/d8d/l95 0 2024-03-29T08:07:16.752 INFO:tasks.workunit.client.0.smithi037.stdout:6/266: chown d0/d20/f31 3755065 1 2024-03-29T08:07:16.753 INFO:tasks.workunit.client.1.smithi148.stdout:0/427: dwrite db/de/d24/f38 [0,4194304] 0 2024-03-29T08:07:16.755 INFO:tasks.workunit.client.1.smithi148.stdout:1/404: mkdir d3/d18/d4d/d76 0 2024-03-29T08:07:16.756 INFO:tasks.workunit.client.1.smithi148.stdout:0/428: readlink db/de/d24/l32 0 2024-03-29T08:07:16.756 INFO:tasks.workunit.client.0.smithi037.stdout:8/353: mknod d5/d25/c75 0 2024-03-29T08:07:16.757 INFO:tasks.workunit.client.0.smithi037.stdout:6/267: fdatasync d0/f37 0 2024-03-29T08:07:16.757 INFO:tasks.workunit.client.1.smithi148.stdout:6/394: mkdir d6/d9/d1e/d27/d48/d73 0 2024-03-29T08:07:16.758 INFO:tasks.workunit.client.0.smithi037.stdout:5/443: chown d8/d22/d4f/d70/d79/f92 0 1 2024-03-29T08:07:16.761 INFO:tasks.workunit.client.0.smithi037.stdout:4/322: symlink d3/d4/d15/l6e 0 2024-03-29T08:07:16.761 INFO:tasks.workunit.client.1.smithi148.stdout:5/429: truncate d0/db/d7b/d16/f3d 208207 0 2024-03-29T08:07:16.763 INFO:tasks.workunit.client.1.smithi148.stdout:8/472: mkdir d17/d49/d9a 0 2024-03-29T08:07:16.767 INFO:tasks.workunit.client.1.smithi148.stdout:1/405: chown d3/de/d21/d5f/l66 0 1 2024-03-29T08:07:16.769 INFO:tasks.workunit.client.1.smithi148.stdout:0/429: mknod db/de/df/d2b/d3b/d47/d4c/c88 0 2024-03-29T08:07:16.770 INFO:tasks.workunit.client.0.smithi037.stdout:8/354: unlink d5/l39 0 2024-03-29T08:07:16.770 INFO:tasks.workunit.client.0.smithi037.stdout:6/268: mkdir d0/d19/d5c 0 2024-03-29T08:07:16.772 INFO:tasks.workunit.client.0.smithi037.stdout:6/269: chown d0/d19/d5c 8531790 1 2024-03-29T08:07:16.773 INFO:tasks.workunit.client.1.smithi148.stdout:6/395: creat d6/d9/d1e/d22/d28/f74 x:0 0 0 2024-03-29T08:07:16.774 INFO:tasks.workunit.client.0.smithi037.stdout:5/444: mknod d8/d22/d37/d8d/c96 0 2024-03-29T08:07:16.777 INFO:tasks.workunit.client.1.smithi148.stdout:2/426: dwrite d7/da/d33/d45/d4d/d2e/d37/f67 [0,4194304] 0 2024-03-29T08:07:16.779 INFO:tasks.workunit.client.0.smithi037.stdout:8/355: symlink d5/d10/d19/d3d/d4a/d4c/l76 0 2024-03-29T08:07:16.784 INFO:tasks.workunit.client.0.smithi037.stdout:6/270: rename d0/d5/d1f/f3c to d0/d5/d1a/f5d 0 2024-03-29T08:07:16.785 INFO:tasks.workunit.client.0.smithi037.stdout:5/445: creat d8/d18/d24/d45/f97 x:0 0 0 2024-03-29T08:07:16.791 INFO:tasks.workunit.client.0.smithi037.stdout:8/356: creat d5/d10/d19/d3d/d4a/f77 x:0 0 0 2024-03-29T08:07:16.791 INFO:tasks.workunit.client.0.smithi037.stdout:4/323: dwrite d3/f21 [0,4194304] 0 2024-03-29T08:07:16.791 INFO:tasks.workunit.client.1.smithi148.stdout:0/430: creat db/de/df/d2b/d3b/d47/d4c/f89 x:0 0 0 2024-03-29T08:07:16.791 INFO:tasks.workunit.client.1.smithi148.stdout:1/406: symlink d3/de/d25/d35/d42/l77 0 2024-03-29T08:07:16.792 INFO:tasks.workunit.client.1.smithi148.stdout:9/401: dread db/d22/d26/f38 [0,4194304] 0 2024-03-29T08:07:16.792 INFO:tasks.workunit.client.1.smithi148.stdout:6/396: creat d6/d9/f75 x:0 0 0 2024-03-29T08:07:16.794 INFO:tasks.workunit.client.0.smithi037.stdout:6/271: read d0/f33 [268182,109482] 0 2024-03-29T08:07:16.797 INFO:tasks.workunit.client.1.smithi148.stdout:2/427: creat d7/da/f89 x:0 0 0 2024-03-29T08:07:16.799 INFO:tasks.workunit.client.0.smithi037.stdout:8/357: mkdir d5/d78 0 2024-03-29T08:07:16.801 INFO:tasks.workunit.client.1.smithi148.stdout:9/402: read db/d22/d26/f2a [185898,24038] 0 2024-03-29T08:07:16.801 INFO:tasks.workunit.client.0.smithi037.stdout:5/446: symlink d8/d22/d37/d60/d88/l98 0 2024-03-29T08:07:16.801 INFO:tasks.workunit.client.1.smithi148.stdout:5/430: dwrite d0/db/d1e/d36/f5a [0,4194304] 0 2024-03-29T08:07:16.804 INFO:tasks.workunit.client.0.smithi037.stdout:6/272: fsync d0/d19/d34/d4e/f54 0 2024-03-29T08:07:16.806 INFO:tasks.workunit.client.1.smithi148.stdout:5/431: chown d0/db/d1e/d36/f5a 780 1 2024-03-29T08:07:16.807 INFO:tasks.workunit.client.0.smithi037.stdout:3/312: dread dd/f39 [0,4194304] 0 2024-03-29T08:07:16.808 INFO:tasks.workunit.client.0.smithi037.stdout:5/447: chown d8/d34/d4b/c57 200 1 2024-03-29T08:07:16.809 INFO:tasks.workunit.client.1.smithi148.stdout:2/428: write d7/d24/f5d [6622475,6334] 0 2024-03-29T08:07:16.811 INFO:tasks.workunit.client.1.smithi148.stdout:6/397: truncate d6/d9/d1e/d27/f6e 969909 0 2024-03-29T08:07:16.811 INFO:tasks.workunit.client.1.smithi148.stdout:9/403: symlink db/d22/d3f/l83 0 2024-03-29T08:07:16.812 INFO:tasks.workunit.client.0.smithi037.stdout:6/273: creat d0/d5/d1e/f5e x:0 0 0 2024-03-29T08:07:16.813 INFO:tasks.workunit.client.1.smithi148.stdout:0/431: truncate db/de/df/d11/d2d/d3e/f45 1063335 0 2024-03-29T08:07:16.818 INFO:tasks.workunit.client.0.smithi037.stdout:3/313: unlink dd/f1f 0 2024-03-29T08:07:16.819 INFO:tasks.workunit.client.1.smithi148.stdout:1/407: dread d3/d18/f20 [0,4194304] 0 2024-03-29T08:07:16.824 INFO:tasks.workunit.client.0.smithi037.stdout:5/448: mknod d8/d22/d37/d60/d88/c99 0 2024-03-29T08:07:16.824 INFO:tasks.workunit.client.0.smithi037.stdout:6/274: rmdir d0/d5 39 2024-03-29T08:07:16.824 INFO:tasks.workunit.client.1.smithi148.stdout:2/429: mknod d7/da/d33/c8a 0 2024-03-29T08:07:16.825 INFO:tasks.workunit.client.0.smithi037.stdout:5/449: chown d8/d4d/d72 97201395 1 2024-03-29T08:07:16.826 INFO:tasks.workunit.client.1.smithi148.stdout:3/438: sync 2024-03-29T08:07:16.827 INFO:tasks.workunit.client.1.smithi148.stdout:7/421: dread d8/d14/d45/f66 [0,4194304] 0 2024-03-29T08:07:16.827 INFO:tasks.workunit.client.1.smithi148.stdout:4/400: sync 2024-03-29T08:07:16.828 INFO:tasks.workunit.client.0.smithi037.stdout:3/314: creat dd/d46/d2d/d3a/f68 x:0 0 0 2024-03-29T08:07:16.828 INFO:tasks.workunit.client.1.smithi148.stdout:0/432: rename db/de/df/d2b/f5b to db/de/df/d11/d1f/f8a 0 2024-03-29T08:07:16.828 INFO:tasks.workunit.client.1.smithi148.stdout:1/408: symlink d3/de/d1d/d47/d6d/l78 0 2024-03-29T08:07:16.830 INFO:tasks.workunit.client.1.smithi148.stdout:9/404: creat db/f84 x:0 0 0 2024-03-29T08:07:16.831 INFO:tasks.workunit.client.0.smithi037.stdout:5/450: readlink d8/d22/d4f/d70/d79/d7a/d85/l87 0 2024-03-29T08:07:16.834 INFO:tasks.workunit.client.0.smithi037.stdout:5/451: write d8/d34/f3a [1181556,20360] 0 2024-03-29T08:07:16.836 INFO:tasks.workunit.client.1.smithi148.stdout:0/433: write db/de/df/d2b/d3b/d47/f2c [79163,92682] 0 2024-03-29T08:07:16.837 INFO:tasks.workunit.client.1.smithi148.stdout:8/473: dread d17/d74/d39/d6f/f45 [0,4194304] 0 2024-03-29T08:07:16.837 INFO:tasks.workunit.client.0.smithi037.stdout:9/506: dread d1/d4/d19/d1e/f3f [0,4194304] 0 2024-03-29T08:07:16.846 INFO:tasks.workunit.client.0.smithi037.stdout:5/452: mknod d8/d22/d37/d60/c9a 0 2024-03-29T08:07:16.850 INFO:tasks.workunit.client.1.smithi148.stdout:0/434: write db/f23 [927783,81644] 0 2024-03-29T08:07:16.850 INFO:tasks.workunit.client.0.smithi037.stdout:5/453: write d8/f73 [687683,20061] 0 2024-03-29T08:07:16.852 INFO:tasks.workunit.client.0.smithi037.stdout:8/358: write d5/f17 [1222854,22175] 0 2024-03-29T08:07:16.853 INFO:tasks.workunit.client.1.smithi148.stdout:4/401: getdents df/d2c 0 2024-03-29T08:07:16.854 INFO:tasks.workunit.client.1.smithi148.stdout:3/439: dread de/f2c [0,4194304] 0 2024-03-29T08:07:16.856 INFO:tasks.workunit.client.0.smithi037.stdout:4/324: dread f0 [0,4194304] 0 2024-03-29T08:07:16.859 INFO:tasks.workunit.client.0.smithi037.stdout:5/454: symlink d8/d22/d37/d8d/l9b 0 2024-03-29T08:07:16.861 INFO:tasks.workunit.client.1.smithi148.stdout:5/432: dwrite d0/db/d7b/d16/f27 [4194304,4194304] 0 2024-03-29T08:07:16.862 INFO:tasks.workunit.client.0.smithi037.stdout:8/359: dread - d5/d10/d19/d3d/d4a/f6f zero size 2024-03-29T08:07:16.862 INFO:tasks.workunit.client.0.smithi037.stdout:4/325: dread - d3/d4/d27/f4b zero size 2024-03-29T08:07:16.865 INFO:tasks.workunit.client.0.smithi037.stdout:9/507: write d1/d4/d19/d1e/d4a/d6f/da2/f3d [895833,96915] 0 2024-03-29T08:07:16.865 INFO:tasks.workunit.client.1.smithi148.stdout:6/398: dread d6/d9/d1e/f53 [0,4194304] 0 2024-03-29T08:07:16.866 INFO:tasks.workunit.client.0.smithi037.stdout:3/315: dwrite dd/d46/d2d/d3a/d5f/f5a [8388608,4194304] 0 2024-03-29T08:07:16.868 INFO:tasks.workunit.client.1.smithi148.stdout:9/405: symlink db/d22/d26/d54/d4d/d35/d4b/d5a/l85 0 2024-03-29T08:07:16.870 INFO:tasks.workunit.client.1.smithi148.stdout:7/422: rename d8/f26 to d8/d29/d44/f93 0 2024-03-29T08:07:16.871 INFO:tasks.workunit.client.0.smithi037.stdout:8/360: creat d5/d10/d2d/d34/f79 x:0 0 0 2024-03-29T08:07:16.871 INFO:tasks.workunit.client.1.smithi148.stdout:0/435: creat db/dc/d78/f8b x:0 0 0 2024-03-29T08:07:16.872 INFO:tasks.workunit.client.0.smithi037.stdout:4/326: write d3/d4/d27/f4b [425630,87203] 0 2024-03-29T08:07:16.876 INFO:tasks.workunit.client.1.smithi148.stdout:5/433: mknod d0/db/d1e/d35/c8a 0 2024-03-29T08:07:16.877 INFO:tasks.workunit.client.1.smithi148.stdout:6/399: creat d6/d9/d1e/d20/d30/d51/f76 x:0 0 0 2024-03-29T08:07:16.877 INFO:tasks.workunit.client.0.smithi037.stdout:6/275: dwrite d0/d5/d1a/f5d [0,4194304] 0 2024-03-29T08:07:16.879 INFO:tasks.workunit.client.1.smithi148.stdout:2/430: getdents d7/da/d4c 0 2024-03-29T08:07:16.881 INFO:tasks.workunit.client.1.smithi148.stdout:9/406: mkdir db/d22/d26/d54/d4d/d35/d4b/d5a/d74/d86 0 2024-03-29T08:07:16.881 INFO:tasks.workunit.client.1.smithi148.stdout:2/431: fdatasync d7/da/d33/d3c/d21/f73 0 2024-03-29T08:07:16.884 INFO:tasks.workunit.client.1.smithi148.stdout:9/407: write db/d10/d2d/f7b [815202,81090] 0 2024-03-29T08:07:16.884 INFO:tasks.workunit.client.1.smithi148.stdout:8/474: rename f14 to d17/d18/d26/d4f/d94/f9b 0 2024-03-29T08:07:16.887 INFO:tasks.workunit.client.0.smithi037.stdout:3/316: creat dd/f69 x:0 0 0 2024-03-29T08:07:16.887 INFO:tasks.workunit.client.1.smithi148.stdout:1/409: dread d3/de/d25/d35/d42/f38 [4194304,4194304] 0 2024-03-29T08:07:16.888 INFO:tasks.workunit.client.1.smithi148.stdout:7/423: mkdir d8/d29/d44/d50/d94 0 2024-03-29T08:07:16.889 INFO:tasks.workunit.client.1.smithi148.stdout:4/402: mkdir df/d2c/d51/d77/d26/d34/d8a 0 2024-03-29T08:07:16.889 INFO:tasks.workunit.client.1.smithi148.stdout:0/436: mknod db/de/df/d11/d1f/c8c 0 2024-03-29T08:07:16.891 INFO:tasks.workunit.client.1.smithi148.stdout:5/434: creat d0/db/d1e/d5b/d66/d73/f8b x:0 0 0 2024-03-29T08:07:16.892 INFO:tasks.workunit.client.0.smithi037.stdout:6/276: write d0/d5/f9 [3492272,118850] 0 2024-03-29T08:07:16.893 INFO:tasks.workunit.client.0.smithi037.stdout:4/327: fdatasync d3/d4/d15/d1d/db/f16 0 2024-03-29T08:07:16.895 INFO:tasks.workunit.client.1.smithi148.stdout:5/435: chown d0/db/d1e/d5b/d66/d72 62 1 2024-03-29T08:07:16.896 INFO:tasks.workunit.client.1.smithi148.stdout:5/436: write d0/db/d1e/d35/f5c [259429,18455] 0 2024-03-29T08:07:16.898 INFO:tasks.workunit.client.1.smithi148.stdout:3/440: rename de/d24/d33/c4c to de/d24/d33/d39/d46/d30/d37/d62/c95 0 2024-03-29T08:07:16.898 INFO:tasks.workunit.client.0.smithi037.stdout:9/508: write d1/d4/d19/d1e/f3f [2413703,65268] 0 2024-03-29T08:07:16.898 INFO:tasks.workunit.client.0.smithi037.stdout:5/455: dread d8/d18/d24/f44 [0,4194304] 0 2024-03-29T08:07:16.899 INFO:tasks.workunit.client.0.smithi037.stdout:3/317: fsync dd/d15/f1e 0 2024-03-29T08:07:16.900 INFO:tasks.workunit.client.1.smithi148.stdout:6/400: symlink d6/l77 0 2024-03-29T08:07:16.901 INFO:tasks.workunit.client.1.smithi148.stdout:4/403: stat df/d2d/d7c/l7f 0 2024-03-29T08:07:16.902 INFO:tasks.workunit.client.0.smithi037.stdout:6/277: write d0/d20/f46 [986594,10303] 0 2024-03-29T08:07:16.902 INFO:tasks.workunit.client.1.smithi148.stdout:8/475: write d17/d18/d26/d4f/d94/f9b [4940873,112689] 0 2024-03-29T08:07:16.904 INFO:tasks.workunit.client.0.smithi037.stdout:9/509: stat d1/d4/d19/d1e 0 2024-03-29T08:07:16.909 INFO:tasks.workunit.client.1.smithi148.stdout:0/437: mkdir db/de/df/d2b/d3b/d47/d4c/d5a/d8d 0 2024-03-29T08:07:16.910 INFO:tasks.workunit.client.0.smithi037.stdout:8/361: dwrite d5/d10/d19/f43 [0,4194304] 0 2024-03-29T08:07:16.911 INFO:tasks.workunit.client.1.smithi148.stdout:4/404: stat df/d2c/d51/d77/d17/d6a/l81 0 2024-03-29T08:07:16.911 INFO:tasks.workunit.client.1.smithi148.stdout:9/408: symlink db/d22/d26/d54/l87 0 2024-03-29T08:07:16.912 INFO:tasks.workunit.client.1.smithi148.stdout:3/441: unlink de/d3b/f74 0 2024-03-29T08:07:16.912 INFO:tasks.workunit.client.0.smithi037.stdout:5/456: rmdir d8/d22 39 2024-03-29T08:07:16.913 INFO:tasks.workunit.client.1.smithi148.stdout:5/437: symlink d0/db/d1e/d5b/d66/l8c 0 2024-03-29T08:07:16.913 INFO:tasks.workunit.client.1.smithi148.stdout:7/424: unlink d8/d14/d45/d4d/d78/l86 0 2024-03-29T08:07:16.914 INFO:tasks.workunit.client.0.smithi037.stdout:6/278: read d0/d5/d3f/f17 [4152928,104064] 0 2024-03-29T08:07:16.915 INFO:tasks.workunit.client.1.smithi148.stdout:1/410: dwrite d3/d18/f4b [0,4194304] 0 2024-03-29T08:07:16.916 INFO:tasks.workunit.client.1.smithi148.stdout:6/401: symlink d6/d9/d1e/d27/l78 0 2024-03-29T08:07:16.918 INFO:tasks.workunit.client.1.smithi148.stdout:9/409: dread - db/f59 zero size 2024-03-29T08:07:16.919 INFO:tasks.workunit.client.1.smithi148.stdout:3/442: write de/d24/d33/d39/d71/f72 [116469,98823] 0 2024-03-29T08:07:16.924 INFO:tasks.workunit.client.0.smithi037.stdout:8/362: write d5/f32 [273237,27776] 0 2024-03-29T08:07:16.926 INFO:tasks.workunit.client.0.smithi037.stdout:6/279: creat d0/d5/d1e/f5f x:0 0 0 2024-03-29T08:07:16.926 INFO:tasks.workunit.client.1.smithi148.stdout:4/405: write df/d2c/d51/d77/d17/d73/f7a [506862,14337] 0 2024-03-29T08:07:16.927 INFO:tasks.workunit.client.0.smithi037.stdout:4/328: mkdir d3/d4/d15/d1d/db/d45/d1e/d59/d60/d6f 0 2024-03-29T08:07:16.929 INFO:tasks.workunit.client.0.smithi037.stdout:4/329: dread - d3/f66 zero size 2024-03-29T08:07:16.929 INFO:tasks.workunit.client.0.smithi037.stdout:9/510: mknod d1/d4/d19/d1e/d71/d89/da6/cb0 0 2024-03-29T08:07:16.930 INFO:tasks.workunit.client.0.smithi037.stdout:6/280: truncate d0/d19/f41 1030447 0 2024-03-29T08:07:16.931 INFO:tasks.workunit.client.1.smithi148.stdout:6/402: mkdir d6/d9/d1e/d27/d79 0 2024-03-29T08:07:16.931 INFO:tasks.workunit.client.1.smithi148.stdout:1/411: write d3/d55/f43 [4022513,6865] 0 2024-03-29T08:07:16.933 INFO:tasks.workunit.client.1.smithi148.stdout:0/438: dread db/de/df/f30 [0,4194304] 0 2024-03-29T08:07:16.934 INFO:tasks.workunit.client.1.smithi148.stdout:2/432: dwrite d7/da/f3f [0,4194304] 0 2024-03-29T08:07:16.935 INFO:tasks.workunit.client.0.smithi037.stdout:5/457: rename d8/d18/d24/d45 to d8/d22/d4f/d70/d9c 0 2024-03-29T08:07:16.941 INFO:tasks.workunit.client.1.smithi148.stdout:3/443: creat de/d24/d33/f96 x:0 0 0 2024-03-29T08:07:16.942 INFO:tasks.workunit.client.1.smithi148.stdout:6/403: chown d6/d69/c6c 0 1 2024-03-29T08:07:16.944 INFO:tasks.workunit.client.0.smithi037.stdout:4/330: fdatasync d3/d4/d15/d1d/db/f12 0 2024-03-29T08:07:16.945 INFO:tasks.workunit.client.0.smithi037.stdout:3/318: link dd/f11 dd/d46/d2d/d3a/d5f/f6a 0 2024-03-29T08:07:16.949 INFO:tasks.workunit.client.0.smithi037.stdout:8/363: dread d5/d10/f23 [0,4194304] 0 2024-03-29T08:07:16.949 INFO:tasks.workunit.client.1.smithi148.stdout:1/412: creat d3/de/d6e/f79 x:0 0 0 2024-03-29T08:07:16.952 INFO:tasks.workunit.client.0.smithi037.stdout:8/364: chown d5/f1c 14143 1 2024-03-29T08:07:16.955 INFO:tasks.workunit.client.1.smithi148.stdout:8/476: dwrite d17/d74/d39/f73 [0,4194304] 0 2024-03-29T08:07:16.955 INFO:tasks.workunit.client.1.smithi148.stdout:4/406: unlink df/d2c/d51/d77/d26/d34/f61 0 2024-03-29T08:07:16.958 INFO:tasks.workunit.client.1.smithi148.stdout:2/433: symlink d7/da/d33/d45/d4d/d2e/l8b 0 2024-03-29T08:07:16.959 INFO:tasks.workunit.client.0.smithi037.stdout:6/281: mkdir d0/d19/d42/d60 0 2024-03-29T08:07:16.959 INFO:tasks.workunit.client.0.smithi037.stdout:9/511: truncate d1/d4/d19/d1e/d25/d16/f9e 1383849 0 2024-03-29T08:07:16.959 INFO:tasks.workunit.client.0.smithi037.stdout:4/331: truncate d3/d4/d15/d1d/db/d45/d1e/f4e 4569561 0 2024-03-29T08:07:16.962 INFO:tasks.workunit.client.0.smithi037.stdout:8/365: rmdir d5/d10/d19 39 2024-03-29T08:07:16.962 INFO:tasks.workunit.client.0.smithi037.stdout:6/282: write d0/d20/f4d [751481,34072] 0 2024-03-29T08:07:16.964 INFO:tasks.workunit.client.0.smithi037.stdout:4/332: chown d3/d4/f2f 1 1 2024-03-29T08:07:16.966 INFO:tasks.workunit.client.1.smithi148.stdout:1/413: mkdir d3/de/d25/d35/d42/d23/d61/d7a 0 2024-03-29T08:07:16.967 INFO:tasks.workunit.client.0.smithi037.stdout:3/319: symlink dd/d46/d2d/l6b 0 2024-03-29T08:07:16.970 INFO:tasks.workunit.client.1.smithi148.stdout:8/477: dread - d17/d74/d39/d6f/f55 zero size 2024-03-29T08:07:16.971 INFO:tasks.workunit.client.0.smithi037.stdout:8/366: read d5/f32 [152937,118721] 0 2024-03-29T08:07:16.972 INFO:tasks.workunit.client.0.smithi037.stdout:5/458: getdents d8/d18/d24 0 2024-03-29T08:07:16.972 INFO:tasks.workunit.client.1.smithi148.stdout:2/434: stat d7/da/d33/d3c/f65 0 2024-03-29T08:07:16.975 INFO:tasks.workunit.client.0.smithi037.stdout:3/320: fdatasync dd/f1b 0 2024-03-29T08:07:16.976 INFO:tasks.workunit.client.1.smithi148.stdout:7/425: dwrite d8/d29/d44/d50/f52 [0,4194304] 0 2024-03-29T08:07:16.980 INFO:tasks.workunit.client.1.smithi148.stdout:9/410: dwrite db/f43 [4194304,4194304] 0 2024-03-29T08:07:16.981 INFO:tasks.workunit.client.0.smithi037.stdout:8/367: mkdir d5/d7a 0 2024-03-29T08:07:16.984 INFO:tasks.workunit.client.0.smithi037.stdout:8/368: stat d5/d25 0 2024-03-29T08:07:16.991 INFO:tasks.workunit.client.0.smithi037.stdout:5/459: mknod d8/d4d/d72/c9d 0 2024-03-29T08:07:16.992 INFO:tasks.workunit.client.1.smithi148.stdout:5/438: truncate d0/db/d7b/d16/d2e/f68 644087 0 2024-03-29T08:07:16.993 INFO:tasks.workunit.client.1.smithi148.stdout:8/478: creat d17/d49/f9c x:0 0 0 2024-03-29T08:07:16.993 INFO:tasks.workunit.client.1.smithi148.stdout:6/404: dwrite d6/d9/f75 [0,4194304] 0 2024-03-29T08:07:16.994 INFO:tasks.workunit.client.0.smithi037.stdout:9/512: dwrite d1/d4/d19/d1e/d4a/fa0 [0,4194304] 0 2024-03-29T08:07:16.995 INFO:tasks.workunit.client.0.smithi037.stdout:6/283: getdents d0/d20 0 2024-03-29T08:07:17.001 INFO:tasks.workunit.client.1.smithi148.stdout:4/407: dwrite df/d2c/d51/d77/d17/f4f [0,4194304] 0 2024-03-29T08:07:17.002 INFO:tasks.workunit.client.0.smithi037.stdout:5/460: creat d8/d22/d4f/d70/d79/d7a/d85/f9e x:0 0 0 2024-03-29T08:07:17.002 INFO:tasks.workunit.client.0.smithi037.stdout:6/284: mkdir d0/d5/d1a/d61 0 2024-03-29T08:07:17.004 INFO:tasks.workunit.client.1.smithi148.stdout:0/439: truncate db/de/df/d11/f37 7727189 0 2024-03-29T08:07:17.005 INFO:tasks.workunit.client.0.smithi037.stdout:5/461: chown d8/c56 1 1 2024-03-29T08:07:17.010 INFO:tasks.workunit.client.0.smithi037.stdout:6/285: write d0/d5/d1a/f5d [3261429,98997] 0 2024-03-29T08:07:17.010 INFO:tasks.workunit.client.0.smithi037.stdout:5/462: stat d8/d34/l77 0 2024-03-29T08:07:17.017 INFO:tasks.workunit.client.0.smithi037.stdout:9/513: fsync d1/d4/d19/d1e/d4a/d6f/da2/d31/fac 0 2024-03-29T08:07:17.025 INFO:tasks.workunit.client.0.smithi037.stdout:3/321: write dd/ff [4272741,43406] 0 2024-03-29T08:07:17.029 INFO:tasks.workunit.client.0.smithi037.stdout:5/463: creat d8/d22/d4f/d70/f9f x:0 0 0 2024-03-29T08:07:17.035 INFO:tasks.workunit.client.0.smithi037.stdout:5/464: readlink d8/d22/d4f/l63 0 2024-03-29T08:07:17.036 INFO:tasks.workunit.client.0.smithi037.stdout:4/333: dread d3/d4/d15/d1d/db/f16 [0,4194304] 0 2024-03-29T08:07:17.037 INFO:tasks.workunit.client.1.smithi148.stdout:1/414: symlink d3/d18/d4d/d76/l7b 0 2024-03-29T08:07:17.040 INFO:tasks.workunit.client.0.smithi037.stdout:8/369: dread d5/d25/f28 [0,4194304] 0 2024-03-29T08:07:17.044 INFO:tasks.workunit.client.0.smithi037.stdout:3/322: mkdir dd/d46/d2d/d3a/d6c 0 2024-03-29T08:07:17.046 INFO:tasks.workunit.client.0.smithi037.stdout:6/286: dwrite d0/f4 [0,4194304] 0 2024-03-29T08:07:17.050 INFO:tasks.workunit.client.0.smithi037.stdout:6/287: readlink d0/d5/d1f/l47 0 2024-03-29T08:07:17.057 INFO:tasks.workunit.client.0.smithi037.stdout:3/323: readlink dd/l13 0 2024-03-29T08:07:17.062 INFO:tasks.workunit.client.0.smithi037.stdout:6/288: mkdir d0/d19/d42/d60/d62 0 2024-03-29T08:07:17.070 INFO:tasks.workunit.client.0.smithi037.stdout:9/514: dwrite d1/d4/d19/d1e/d25/f34 [0,4194304] 0 2024-03-29T08:07:17.079 INFO:tasks.workunit.client.0.smithi037.stdout:3/324: getdents dd/d46/d3e 0 2024-03-29T08:07:17.079 INFO:tasks.workunit.client.1.smithi148.stdout:2/435: fdatasync d7/da/fe 0 2024-03-29T08:07:17.083 INFO:tasks.workunit.client.0.smithi037.stdout:5/465: dwrite d8/d34/f4a [4194304,4194304] 0 2024-03-29T08:07:17.083 INFO:tasks.workunit.client.1.smithi148.stdout:2/436: chown d7/da/d33/d3c 25154 1 2024-03-29T08:07:17.086 INFO:tasks.workunit.client.1.smithi148.stdout:9/411: creat db/d22/d26/d54/d4d/d35/d4b/f88 x:0 0 0 2024-03-29T08:07:17.090 INFO:tasks.workunit.client.0.smithi037.stdout:4/334: dread d3/d4/d15/d1d/db/f1a [0,4194304] 0 2024-03-29T08:07:17.090 INFO:tasks.workunit.client.1.smithi148.stdout:9/412: write db/d22/d26/d54/d4d/d35/d4b/d5a/f77 [335470,77565] 0 2024-03-29T08:07:17.091 INFO:tasks.workunit.client.0.smithi037.stdout:3/325: creat dd/d5e/f6d x:0 0 0 2024-03-29T08:07:17.092 INFO:tasks.workunit.client.0.smithi037.stdout:9/515: truncate d1/d4/d19/d69/d75/f8b 387035 0 2024-03-29T08:07:17.092 INFO:tasks.workunit.client.1.smithi148.stdout:9/413: read - db/d22/d26/d54/f3c zero size 2024-03-29T08:07:17.096 INFO:tasks.workunit.client.0.smithi037.stdout:8/370: dwrite d5/d10/d19/d3d/f5d [0,4194304] 0 2024-03-29T08:07:17.097 INFO:tasks.workunit.client.0.smithi037.stdout:3/326: write dd/f69 [465964,29247] 0 2024-03-29T08:07:17.102 INFO:tasks.workunit.client.0.smithi037.stdout:8/371: chown d5/d10/d45/l64 84666981 1 2024-03-29T08:07:17.103 INFO:tasks.workunit.client.1.smithi148.stdout:3/444: getdents de/d24/d33/d39/d46/d30/d37/d62/d63 0 2024-03-29T08:07:17.104 INFO:tasks.workunit.client.0.smithi037.stdout:5/466: dread d8/f42 [0,4194304] 0 2024-03-29T08:07:17.104 INFO:tasks.workunit.client.0.smithi037.stdout:3/327: dread - dd/d15/d18/d4b/f67 zero size 2024-03-29T08:07:17.105 INFO:tasks.workunit.client.0.smithi037.stdout:9/516: mknod d1/d4/d95/cb1 0 2024-03-29T08:07:17.105 INFO:tasks.workunit.client.1.smithi148.stdout:3/445: write de/d24/d33/f96 [630178,118596] 0 2024-03-29T08:07:17.106 INFO:tasks.workunit.client.0.smithi037.stdout:6/289: dwrite d0/d5/f9 [0,4194304] 0 2024-03-29T08:07:17.108 INFO:tasks.workunit.client.0.smithi037.stdout:6/290: readlink d0/l2 0 2024-03-29T08:07:17.109 INFO:tasks.workunit.client.1.smithi148.stdout:5/439: sync 2024-03-29T08:07:17.111 INFO:tasks.workunit.client.1.smithi148.stdout:6/405: creat d6/d9/d1e/d36/d4d/f7a x:0 0 0 2024-03-29T08:07:17.115 INFO:tasks.workunit.client.0.smithi037.stdout:3/328: dread dd/f69 [0,4194304] 0 2024-03-29T08:07:17.116 INFO:tasks.workunit.client.0.smithi037.stdout:4/335: rename d3/d4/d15/d1d/db/d45/d1e/c68 to d3/d4/c70 0 2024-03-29T08:07:17.121 INFO:tasks.workunit.client.0.smithi037.stdout:9/517: creat d1/d4/d95/fb2 x:0 0 0 2024-03-29T08:07:17.121 INFO:tasks.workunit.client.0.smithi037.stdout:8/372: getdents d5/d7a 0 2024-03-29T08:07:17.126 INFO:tasks.workunit.client.0.smithi037.stdout:8/373: chown d5/d10/d2d/f60 501890 1 2024-03-29T08:07:17.126 INFO:tasks.workunit.client.0.smithi037.stdout:3/329: unlink dd/f39 0 2024-03-29T08:07:17.126 INFO:tasks.workunit.client.0.smithi037.stdout:3/330: chown dd/d15/d18/d4b 1016 1 2024-03-29T08:07:17.128 INFO:tasks.workunit.client.0.smithi037.stdout:8/374: write f1 [2178241,25054] 0 2024-03-29T08:07:17.131 INFO:tasks.workunit.client.0.smithi037.stdout:8/375: truncate d5/f32 602463 0 2024-03-29T08:07:17.136 INFO:tasks.workunit.client.0.smithi037.stdout:8/376: write d5/d10/d19/d3d/d4a/d4c/f65 [1001995,121173] 0 2024-03-29T08:07:17.137 INFO:tasks.workunit.client.0.smithi037.stdout:5/467: rename d8/d34/f3a to d8/fa0 0 2024-03-29T08:07:17.137 INFO:tasks.workunit.client.1.smithi148.stdout:3/446: stat de/d24/c38 0 2024-03-29T08:07:17.140 INFO:tasks.workunit.client.0.smithi037.stdout:4/336: creat d3/d4/d15/d1d/f71 x:0 0 0 2024-03-29T08:07:17.141 INFO:tasks.workunit.client.1.smithi148.stdout:8/479: symlink d17/d49/d9a/l9d 0 2024-03-29T08:07:17.142 INFO:tasks.workunit.client.0.smithi037.stdout:3/331: mknod dd/d15/d18/c6e 0 2024-03-29T08:07:17.145 INFO:tasks.workunit.client.0.smithi037.stdout:5/468: write d8/d22/d4f/d70/f3f [1804767,75107] 0 2024-03-29T08:07:17.145 INFO:tasks.workunit.client.0.smithi037.stdout:4/337: chown d3/d4/d27/l32 7257 1 2024-03-29T08:07:17.149 INFO:tasks.workunit.client.0.smithi037.stdout:6/291: rename d0/d5/d3f/ff to d0/d5/d1f/f63 0 2024-03-29T08:07:17.152 INFO:tasks.workunit.client.1.smithi148.stdout:0/440: getdents db/de/df/d2b/d3b/d47/d7c 0 2024-03-29T08:07:17.153 INFO:tasks.workunit.client.1.smithi148.stdout:1/415: write d3/de/d25/d35/d42/f15 [444816,18454] 0 2024-03-29T08:07:17.157 INFO:tasks.workunit.client.0.smithi037.stdout:7/348: sync 2024-03-29T08:07:17.157 INFO:tasks.workunit.client.0.smithi037.stdout:0/187: sync 2024-03-29T08:07:17.161 INFO:tasks.workunit.client.0.smithi037.stdout:8/377: dwrite d5/d25/f2e [0,4194304] 0 2024-03-29T08:07:17.164 INFO:tasks.workunit.client.0.smithi037.stdout:7/349: read - dd/d21/d35/d42/d5c/f5e zero size 2024-03-29T08:07:17.164 INFO:tasks.workunit.client.0.smithi037.stdout:9/518: write d1/d4/f3b [482981,61613] 0 2024-03-29T08:07:17.166 INFO:tasks.workunit.client.1.smithi148.stdout:5/440: truncate d0/db/d1e/d36/d41/f7f 427709 0 2024-03-29T08:07:17.166 INFO:tasks.workunit.client.0.smithi037.stdout:8/378: chown d5/f48 0 1 2024-03-29T08:07:17.168 INFO:tasks.workunit.client.0.smithi037.stdout:8/379: fsync d5/d10/f11 0 2024-03-29T08:07:17.169 INFO:tasks.workunit.client.0.smithi037.stdout:9/519: write d1/d4/d95/fb2 [986878,71498] 0 2024-03-29T08:07:17.170 INFO:tasks.workunit.client.0.smithi037.stdout:5/469: unlink d8/d22/d4f/d70/d9c/f51 0 2024-03-29T08:07:17.173 INFO:tasks.workunit.client.1.smithi148.stdout:4/408: rmdir df/d2d/d65 0 2024-03-29T08:07:17.174 INFO:tasks.workunit.client.0.smithi037.stdout:6/292: dread - d0/d5/f2a zero size 2024-03-29T08:07:17.177 INFO:tasks.workunit.client.0.smithi037.stdout:3/332: fsync dd/f23 0 2024-03-29T08:07:17.178 INFO:tasks.workunit.client.0.smithi037.stdout:4/338: mkdir d3/d4/d15/d1d/db/d72 0 2024-03-29T08:07:17.181 INFO:tasks.workunit.client.0.smithi037.stdout:4/339: write d3/d4/d15/d1d/f71 [733483,87523] 0 2024-03-29T08:07:17.181 INFO:tasks.workunit.client.0.smithi037.stdout:0/188: mknod d9/da/d1b/c37 0 2024-03-29T08:07:17.186 INFO:tasks.workunit.client.0.smithi037.stdout:4/340: dread d3/d4/d15/d1d/db/f12 [0,4194304] 0 2024-03-29T08:07:17.188 INFO:tasks.workunit.client.1.smithi148.stdout:1/416: unlink d3/de/d25/d35/d42/c1e 0 2024-03-29T08:07:17.191 INFO:tasks.workunit.client.1.smithi148.stdout:0/441: mkdir db/de/d8e 0 2024-03-29T08:07:17.194 INFO:tasks.workunit.client.1.smithi148.stdout:4/409: creat df/d2d/f8b x:0 0 0 2024-03-29T08:07:17.194 INFO:tasks.workunit.client.1.smithi148.stdout:0/442: write db/de/df/d2b/f34 [1142659,74472] 0 2024-03-29T08:07:17.195 INFO:tasks.workunit.client.1.smithi148.stdout:2/437: getdents d7/da/d33/d45/d4d/d41 0 2024-03-29T08:07:17.196 INFO:tasks.workunit.client.0.smithi037.stdout:5/470: truncate d8/d22/f26 2600753 0 2024-03-29T08:07:17.198 INFO:tasks.workunit.client.1.smithi148.stdout:5/441: mknod d0/db/d1e/d36/d41/c8d 0 2024-03-29T08:07:17.199 INFO:tasks.workunit.client.0.smithi037.stdout:5/471: read d8/f17 [4248196,20275] 0 2024-03-29T08:07:17.201 INFO:tasks.workunit.client.1.smithi148.stdout:0/443: write db/de/df/d11/d42/f67 [689658,124218] 0 2024-03-29T08:07:17.201 INFO:tasks.workunit.client.1.smithi148.stdout:1/417: dread d3/d55/f37 [0,4194304] 0 2024-03-29T08:07:17.205 INFO:tasks.workunit.client.0.smithi037.stdout:0/189: mkdir d9/da/dd/d20/d38 0 2024-03-29T08:07:17.208 INFO:tasks.workunit.client.0.smithi037.stdout:5/472: dread d8/f17 [4194304,4194304] 0 2024-03-29T08:07:17.208 INFO:tasks.workunit.client.1.smithi148.stdout:4/410: creat df/d2c/d51/d77/d26/d34/d8a/f8c x:0 0 0 2024-03-29T08:07:17.212 INFO:tasks.workunit.client.0.smithi037.stdout:4/341: dwrite d3/d4/f46 [0,4194304] 0 2024-03-29T08:07:17.220 INFO:tasks.workunit.client.1.smithi148.stdout:2/438: getdents d7/da/d33/d45/d4d/d2e 0 2024-03-29T08:07:17.221 INFO:tasks.workunit.client.0.smithi037.stdout:0/190: creat d9/da/dd/d1c/f39 x:0 0 0 2024-03-29T08:07:17.222 INFO:tasks.workunit.client.1.smithi148.stdout:1/418: mknod d3/de/d25/d35/d42/c7c 0 2024-03-29T08:07:17.225 INFO:tasks.workunit.client.1.smithi148.stdout:5/442: dwrite d0/db/d1e/d35/d6a/f6d [0,4194304] 0 2024-03-29T08:07:17.225 INFO:tasks.workunit.client.1.smithi148.stdout:2/439: mkdir d7/d8c 0 2024-03-29T08:07:17.226 INFO:tasks.workunit.client.1.smithi148.stdout:1/419: fsync d3/de/d25/d51/f73 0 2024-03-29T08:07:17.229 INFO:tasks.workunit.client.0.smithi037.stdout:9/520: link d1/d3/c15 d1/d4/d19/d69/d75/cb3 0 2024-03-29T08:07:17.229 INFO:tasks.workunit.client.1.smithi148.stdout:9/414: rename db/d22/d26/d54/d4d/d35/d4b/d5a/d74/f75 to db/d22/f89 0 2024-03-29T08:07:17.233 INFO:tasks.workunit.client.0.smithi037.stdout:0/191: mknod d9/c3a 0 2024-03-29T08:07:17.236 INFO:tasks.workunit.client.1.smithi148.stdout:7/426: truncate d8/d14/f2b 2282063 0 2024-03-29T08:07:17.237 INFO:tasks.workunit.client.1.smithi148.stdout:3/447: unlink de/d24/f3d 0 2024-03-29T08:07:17.237 INFO:tasks.workunit.client.1.smithi148.stdout:1/420: mknod d3/d55/d44/c7d 0 2024-03-29T08:07:17.239 INFO:tasks.workunit.client.0.smithi037.stdout:8/380: dwrite d5/f1f [0,4194304] 0 2024-03-29T08:07:17.239 INFO:tasks.workunit.client.0.smithi037.stdout:3/333: mkdir dd/d46/d2d/d3a/d6c/d6f 0 2024-03-29T08:07:17.241 INFO:tasks.workunit.client.0.smithi037.stdout:7/350: getdents dd/d21/d35/d3b 0 2024-03-29T08:07:17.243 INFO:tasks.workunit.client.1.smithi148.stdout:2/440: mkdir d7/da/d33/d45/d4d/d82/d8d 0 2024-03-29T08:07:17.243 INFO:tasks.workunit.client.0.smithi037.stdout:5/473: link d8/d22/d4f/f5a d8/d22/d37/d60/fa1 0 2024-03-29T08:07:17.243 INFO:tasks.workunit.client.0.smithi037.stdout:9/521: fsync d1/d4/d19/d1e/d4a/d6f/da2/d23/f65 0 2024-03-29T08:07:17.246 INFO:tasks.workunit.client.1.smithi148.stdout:0/444: dread f7 [4194304,4194304] 0 2024-03-29T08:07:17.247 INFO:tasks.workunit.client.1.smithi148.stdout:9/415: creat db/d22/d26/f8a x:0 0 0 2024-03-29T08:07:17.250 INFO:tasks.workunit.client.0.smithi037.stdout:3/334: chown dd/c14 2414308 1 2024-03-29T08:07:17.251 INFO:tasks.workunit.client.0.smithi037.stdout:6/293: dwrite d0/d5/f16 [0,4194304] 0 2024-03-29T08:07:17.252 INFO:tasks.workunit.client.0.smithi037.stdout:8/381: mknod d5/d10/d19/d3d/c7b 0 2024-03-29T08:07:17.253 INFO:tasks.workunit.client.1.smithi148.stdout:2/441: mkdir d7/d86/d8e 0 2024-03-29T08:07:17.255 INFO:tasks.workunit.client.1.smithi148.stdout:0/445: symlink db/de/df/d11/d2d/d3e/l8f 0 2024-03-29T08:07:17.257 INFO:tasks.workunit.client.0.smithi037.stdout:5/474: symlink d8/d18/d24/la2 0 2024-03-29T08:07:17.260 INFO:tasks.workunit.client.1.smithi148.stdout:9/416: creat db/d22/d26/d54/d4d/d35/d4b/d58/f8b x:0 0 0 2024-03-29T08:07:17.262 INFO:tasks.workunit.client.0.smithi037.stdout:8/382: write d5/d25/d46/d4e/f57 [183610,70670] 0 2024-03-29T08:07:17.262 INFO:tasks.workunit.client.1.smithi148.stdout:7/427: link d8/d29/d44/d49/c8a d8/d29/d44/d50/c95 0 2024-03-29T08:07:17.266 INFO:tasks.workunit.client.1.smithi148.stdout:3/448: dwrite de/d24/d33/d39/d46/d30/d37/f7a [0,4194304] 0 2024-03-29T08:07:17.267 INFO:tasks.workunit.client.1.smithi148.stdout:6/406: rename d6/f60 to d6/d9/d1e/d27/d48/f7b 0 2024-03-29T08:07:17.269 INFO:tasks.workunit.client.0.smithi037.stdout:3/335: fdatasync dd/d46/d2d/d3a/f53 0 2024-03-29T08:07:17.270 INFO:tasks.workunit.client.1.smithi148.stdout:3/449: fdatasync de/d24/d33/d39/d46/d70/f83 0 2024-03-29T08:07:17.274 INFO:tasks.workunit.client.1.smithi148.stdout:7/428: creat d8/d38/d3c/f96 x:0 0 0 2024-03-29T08:07:17.275 INFO:tasks.workunit.client.0.smithi037.stdout:8/383: creat d5/f7c x:0 0 0 2024-03-29T08:07:17.276 INFO:tasks.workunit.client.1.smithi148.stdout:7/429: chown d8/d14/d45/d4d/d78 50450667 1 2024-03-29T08:07:17.277 INFO:tasks.workunit.client.1.smithi148.stdout:2/442: dread d7/d24/d4b/f87 [0,4194304] 0 2024-03-29T08:07:17.278 INFO:tasks.workunit.client.1.smithi148.stdout:2/443: readlink d7/d24/d4b/l4f 0 2024-03-29T08:07:17.279 INFO:tasks.workunit.client.0.smithi037.stdout:1/377: sync 2024-03-29T08:07:17.279 INFO:tasks.workunit.client.0.smithi037.stdout:2/218: sync 2024-03-29T08:07:17.279 INFO:tasks.workunit.client.0.smithi037.stdout:5/475: getdents d8/d22/d4f/d70 0 2024-03-29T08:07:17.282 INFO:tasks.workunit.client.1.smithi148.stdout:5/443: rename d0/db/d1e/f37 to d0/d46/f8e 0 2024-03-29T08:07:17.283 INFO:tasks.workunit.client.1.smithi148.stdout:8/480: link d17/d49/d9a/l9d d17/l9e 0 2024-03-29T08:07:17.283 INFO:tasks.workunit.client.1.smithi148.stdout:6/407: creat d6/d9/d1e/d20/d30/d51/f7c x:0 0 0 2024-03-29T08:07:17.283 INFO:tasks.workunit.client.1.smithi148.stdout:4/411: write df/d2c/d51/d77/d26/f84 [492034,115424] 0 2024-03-29T08:07:17.287 INFO:tasks.workunit.client.1.smithi148.stdout:3/450: mknod de/d24/d33/d39/d46/d30/d37/c97 0 2024-03-29T08:07:17.288 INFO:tasks.workunit.client.1.smithi148.stdout:7/430: readlink d8/d38/l89 0 2024-03-29T08:07:17.288 INFO:tasks.workunit.client.1.smithi148.stdout:4/412: dread - df/f5e zero size 2024-03-29T08:07:17.289 INFO:tasks.workunit.client.0.smithi037.stdout:3/336: rmdir dd 39 2024-03-29T08:07:17.291 INFO:tasks.workunit.client.1.smithi148.stdout:5/444: write d0/db/d7b/f50 [5163310,19842] 0 2024-03-29T08:07:17.295 INFO:tasks.workunit.client.1.smithi148.stdout:9/417: creat db/d22/d26/d54/d4d/d35/f8c x:0 0 0 2024-03-29T08:07:17.297 INFO:tasks.workunit.client.0.smithi037.stdout:0/192: truncate d9/da/dd/d1c/f18 3741019 0 2024-03-29T08:07:17.297 INFO:tasks.workunit.client.0.smithi037.stdout:6/294: dwrite d0/d19/f41 [0,4194304] 0 2024-03-29T08:07:17.299 INFO:tasks.workunit.client.1.smithi148.stdout:1/421: rename d3/de/d25/d31 to d3/d55/d7e 0 2024-03-29T08:07:17.301 INFO:tasks.workunit.client.1.smithi148.stdout:2/444: dwrite d7/f7c [0,4194304] 0 2024-03-29T08:07:17.303 INFO:tasks.workunit.client.0.smithi037.stdout:9/522: dwrite d1/f74 [0,4194304] 0 2024-03-29T08:07:17.303 INFO:tasks.workunit.client.1.smithi148.stdout:2/445: fdatasync d7/da/d33/d45/f6b 0 2024-03-29T08:07:17.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:07:17 smithi148 ceph-mon[44992]: pgmap v40: 65 pgs: 65 active+clean; 1.3 GiB data, 5.4 GiB used, 531 GiB / 536 GiB avail; 22 MiB/s rd, 162 MiB/s wr, 258 op/s 2024-03-29T08:07:17.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:07:17 smithi148 ceph-mon[44992]: from='mgr.14602 ' entity='mgr.smithi148.onhgvm' 2024-03-29T08:07:17.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:07:17 smithi148 ceph-mon[44992]: from='mgr.14602 ' entity='mgr.smithi148.onhgvm' 2024-03-29T08:07:17.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:07:17 smithi148 ceph-mon[44992]: from='mgr.14602 172.21.15.148:0/667342922' entity='mgr.smithi148.onhgvm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:07:17.307 INFO:tasks.workunit.client.1.smithi148.stdout:1/422: write d3/de/d6e/f79 [862054,33286] 0 2024-03-29T08:07:17.308 INFO:tasks.workunit.client.1.smithi148.stdout:7/431: mknod d8/d14/d19/d37/c97 0 2024-03-29T08:07:17.313 INFO:tasks.workunit.client.0.smithi037.stdout:5/476: truncate d8/d22/d4f/d70/d79/d7a/d85/f9e 215838 0 2024-03-29T08:07:17.325 INFO:tasks.workunit.client.0.smithi037.stdout:7/351: dwrite dd/f16 [0,4194304] 0 2024-03-29T08:07:17.328 INFO:tasks.workunit.client.0.smithi037.stdout:1/378: dwrite d2/d6/d13/f77 [0,4194304] 0 2024-03-29T08:07:17.329 INFO:tasks.workunit.client.1.smithi148.stdout:1/423: mkdir d3/de/d21/d7f 0 2024-03-29T08:07:17.333 INFO:tasks.workunit.client.0.smithi037.stdout:8/384: symlink d5/d10/d19/d3d/d4a/d4c/d67/l7d 0 2024-03-29T08:07:17.336 INFO:tasks.workunit.client.0.smithi037.stdout:0/193: dwrite d9/f27 [0,4194304] 0 2024-03-29T08:07:17.337 INFO:tasks.workunit.client.0.smithi037.stdout:2/219: mknod d0/d20/c4d 0 2024-03-29T08:07:17.343 INFO:tasks.workunit.client.1.smithi148.stdout:2/446: creat d7/d24/d4b/d66/d7d/f8f x:0 0 0 2024-03-29T08:07:17.345 INFO:tasks.workunit.client.1.smithi148.stdout:2/447: readlink d7/da/d33/d3c/d21/l2f 0 2024-03-29T08:07:17.346 INFO:tasks.workunit.client.1.smithi148.stdout:6/408: dread d6/d9/d1e/d20/d30/f35 [4194304,4194304] 0 2024-03-29T08:07:17.347 INFO:tasks.workunit.client.1.smithi148.stdout:1/424: creat d3/de/d21/f80 x:0 0 0 2024-03-29T08:07:17.347 INFO:tasks.workunit.client.1.smithi148.stdout:5/445: link d0/db/d7b/f50 d0/db/d7b/d16/f8f 0 2024-03-29T08:07:17.348 INFO:tasks.workunit.client.0.smithi037.stdout:8/385: creat d5/d10/d19/d3d/f7e x:0 0 0 2024-03-29T08:07:17.351 INFO:tasks.workunit.client.0.smithi037.stdout:2/220: mkdir d0/d20/d22/d4e 0 2024-03-29T08:07:17.352 INFO:tasks.workunit.client.1.smithi148.stdout:6/409: chown d6/f41 4718 1 2024-03-29T08:07:17.352 INFO:tasks.workunit.client.1.smithi148.stdout:4/413: getdents df/d60 0 2024-03-29T08:07:17.359 INFO:tasks.workunit.client.0.smithi037.stdout:9/523: link d1/d4/d19/d1e/d25/f34 d1/d4/d19/d1e/d4a/d6f/da2/d13/d7c/fb4 0 2024-03-29T08:07:17.360 INFO:tasks.workunit.client.1.smithi148.stdout:2/448: unlink d7/f7c 0 2024-03-29T08:07:17.361 INFO:tasks.workunit.client.1.smithi148.stdout:9/418: getdents db/d22/d26/d54/d4d/d35/d4b/d58 0 2024-03-29T08:07:17.364 INFO:tasks.workunit.client.1.smithi148.stdout:0/446: rename db/de/df/d11/d1f/f8a to db/de/df/f90 0 2024-03-29T08:07:17.364 INFO:tasks.workunit.client.1.smithi148.stdout:2/449: dread - d7/da/d33/d45/d4d/d2e/d37/f6d zero size 2024-03-29T08:07:17.364 INFO:tasks.workunit.client.1.smithi148.stdout:7/432: truncate d8/fe 4124015 0 2024-03-29T08:07:17.366 INFO:tasks.workunit.client.0.smithi037.stdout:6/295: dwrite d0/d5/d3f/f17 [4194304,4194304] 0 2024-03-29T08:07:17.366 INFO:tasks.workunit.client.0.smithi037.stdout:0/194: mknod d9/da/dd/c3b 0 2024-03-29T08:07:17.370 INFO:tasks.workunit.client.1.smithi148.stdout:7/433: write d8/d29/d44/d50/f90 [615209,130350] 0 2024-03-29T08:07:17.371 INFO:tasks.workunit.client.0.smithi037.stdout:3/337: creat dd/d15/d27/d33/f70 x:0 0 0 2024-03-29T08:07:17.373 INFO:tasks.workunit.client.0.smithi037.stdout:3/338: readlink dd/d46/d2d/l6b 0 2024-03-29T08:07:17.375 INFO:tasks.workunit.client.1.smithi148.stdout:1/425: dwrite d3/de/d25/d35/d42/d23/f40 [0,4194304] 0 2024-03-29T08:07:17.384 INFO:tasks.workunit.client.0.smithi037.stdout:1/379: creat d2/d6/f7d x:0 0 0 2024-03-29T08:07:17.388 INFO:tasks.workunit.client.1.smithi148.stdout:9/419: write db/d22/d26/f38 [4128364,99453] 0 2024-03-29T08:07:17.388 INFO:tasks.workunit.client.1.smithi148.stdout:6/410: mkdir d6/d9/d1e/d22/d7d 0 2024-03-29T08:07:17.389 INFO:tasks.workunit.client.0.smithi037.stdout:7/352: creat dd/d21/d35/f71 x:0 0 0 2024-03-29T08:07:17.391 INFO:tasks.workunit.client.1.smithi148.stdout:6/411: chown d6/d9/d1e/l65 3631997 1 2024-03-29T08:07:17.394 INFO:tasks.workunit.client.0.smithi037.stdout:6/296: rmdir d0/d20 39 2024-03-29T08:07:17.395 INFO:tasks.workunit.client.1.smithi148.stdout:8/481: rename d17/d18/d26/l7b to d17/d74/d39/d6f/d64/l9f 0 2024-03-29T08:07:17.396 INFO:tasks.workunit.client.1.smithi148.stdout:5/446: dread d0/db/d1e/d36/d7c/f80 [0,4194304] 0 2024-03-29T08:07:17.397 INFO:tasks.workunit.client.1.smithi148.stdout:5/447: chown d0/d46/d86 4915522 1 2024-03-29T08:07:17.398 INFO:tasks.workunit.client.0.smithi037.stdout:0/195: stat d9/c26 0 2024-03-29T08:07:17.398 INFO:tasks.workunit.client.0.smithi037.stdout:7/353: write dd/d21/f2c [284152,8581] 0 2024-03-29T08:07:17.399 INFO:tasks.workunit.client.1.smithi148.stdout:0/447: symlink db/dc/d78/l91 0 2024-03-29T08:07:17.400 INFO:tasks.workunit.client.0.smithi037.stdout:2/221: truncate d0/f8 1025181 0 2024-03-29T08:07:17.402 INFO:tasks.workunit.client.0.smithi037.stdout:9/524: dwrite d1/d4/d19/d1e/d4a/d6f/da2/f3d [0,4194304] 0 2024-03-29T08:07:17.404 INFO:tasks.workunit.client.0.smithi037.stdout:7/354: readlink dd/l2f 0 2024-03-29T08:07:17.405 INFO:tasks.workunit.client.0.smithi037.stdout:5/477: dwrite d8/d18/d24/f44 [4194304,4194304] 0 2024-03-29T08:07:17.407 INFO:tasks.workunit.client.1.smithi148.stdout:4/414: write df/d2c/d51/d77/d26/f4a [137400,22738] 0 2024-03-29T08:07:17.411 INFO:tasks.workunit.client.0.smithi037.stdout:1/380: mkdir d2/d6/dc/d7e 0 2024-03-29T08:07:17.419 INFO:tasks.workunit.client.0.smithi037.stdout:0/196: creat d9/da/f3c x:0 0 0 2024-03-29T08:07:17.421 INFO:tasks.workunit.client.0.smithi037.stdout:5/478: mknod d8/d22/d4f/d70/d9c/ca3 0 2024-03-29T08:07:17.424 INFO:tasks.workunit.client.0.smithi037.stdout:9/525: fdatasync d1/d1f/d3a/f27 0 2024-03-29T08:07:17.428 INFO:tasks.workunit.client.0.smithi037.stdout:1/381: creat d2/d6/dc/d18/d22/d4f/d7c/f7f x:0 0 0 2024-03-29T08:07:17.428 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:07:17 smithi037 ceph-mon[35162]: pgmap v40: 65 pgs: 65 active+clean; 1.3 GiB data, 5.4 GiB used, 531 GiB / 536 GiB avail; 22 MiB/s rd, 162 MiB/s wr, 258 op/s 2024-03-29T08:07:17.428 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:07:17 smithi037 ceph-mon[35162]: from='mgr.14602 ' entity='mgr.smithi148.onhgvm' 2024-03-29T08:07:17.428 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:07:17 smithi037 ceph-mon[35162]: from='mgr.14602 ' entity='mgr.smithi148.onhgvm' 2024-03-29T08:07:17.428 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:07:17 smithi037 ceph-mon[35162]: from='mgr.14602 172.21.15.148:0/667342922' entity='mgr.smithi148.onhgvm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:07:17.429 INFO:tasks.workunit.client.1.smithi148.stdout:1/426: mknod d3/de/d1d/d47/d6d/c81 0 2024-03-29T08:07:17.432 INFO:tasks.workunit.client.0.smithi037.stdout:6/297: dread d0/d20/f4d [0,4194304] 0 2024-03-29T08:07:17.435 INFO:tasks.workunit.client.0.smithi037.stdout:5/479: chown d8/d22/d4f/d70/d9c/f75 11008 1 2024-03-29T08:07:17.436 INFO:tasks.workunit.client.1.smithi148.stdout:2/450: symlink d7/da/d33/d45/d4d/d82/d8d/l90 0 2024-03-29T08:07:17.437 INFO:tasks.workunit.client.0.smithi037.stdout:2/222: dwrite d0/d23/f26 [0,4194304] 0 2024-03-29T08:07:17.437 INFO:tasks.workunit.client.1.smithi148.stdout:2/451: dread - d7/da/d4c/f79 zero size 2024-03-29T08:07:17.438 INFO:tasks.workunit.client.0.smithi037.stdout:6/298: dread - d0/d5/f2a zero size 2024-03-29T08:07:17.453 INFO:tasks.workunit.client.0.smithi037.stdout:8/386: truncate d5/d10/d19/f43 1285357 0 2024-03-29T08:07:17.457 INFO:tasks.workunit.client.0.smithi037.stdout:7/355: dread dd/d21/d65/f6f [0,4194304] 0 2024-03-29T08:07:17.457 INFO:tasks.workunit.client.0.smithi037.stdout:1/382: mkdir d2/d4b/d80 0 2024-03-29T08:07:17.458 INFO:tasks.workunit.client.0.smithi037.stdout:5/480: symlink d8/d22/d37/d60/d61/la4 0 2024-03-29T08:07:17.460 INFO:tasks.workunit.client.0.smithi037.stdout:5/481: chown d8/d34/l3c 1 1 2024-03-29T08:07:17.461 INFO:tasks.workunit.client.0.smithi037.stdout:7/356: fsync dd/d1d/d34/f39 0 2024-03-29T08:07:17.466 INFO:tasks.workunit.client.0.smithi037.stdout:8/387: fsync d5/f13 0 2024-03-29T08:07:17.477 INFO:tasks.workunit.client.0.smithi037.stdout:1/383: dwrite d2/d6/d13/f77 [4194304,4194304] 0 2024-03-29T08:07:17.477 INFO:tasks.workunit.client.0.smithi037.stdout:6/299: symlink d0/d20/d43/l64 0 2024-03-29T08:07:17.478 INFO:tasks.workunit.client.0.smithi037.stdout:1/384: readlink d2/d6/l27 0 2024-03-29T08:07:17.479 INFO:tasks.workunit.client.0.smithi037.stdout:5/482: mknod d8/d22/d4f/d70/ca5 0 2024-03-29T08:07:17.482 INFO:tasks.workunit.client.0.smithi037.stdout:0/197: creat d9/da/d1b/f3d x:0 0 0 2024-03-29T08:07:17.483 INFO:tasks.workunit.client.1.smithi148.stdout:6/412: mkdir d6/d9/d1e/d27/d48/d73/d7e 0 2024-03-29T08:07:17.484 INFO:tasks.workunit.client.0.smithi037.stdout:0/198: stat d9/da/dd/d13 0 2024-03-29T08:07:17.485 INFO:tasks.workunit.client.1.smithi148.stdout:6/413: chown d6/d9/d1e/d22/d28/f38 344906 1 2024-03-29T08:07:17.486 INFO:tasks.workunit.client.1.smithi148.stdout:3/451: rename de/d24/c5d to de/d24/d33/d39/d46/d8d/c98 0 2024-03-29T08:07:17.487 INFO:tasks.workunit.client.0.smithi037.stdout:2/223: truncate d0/f40 1312792 0 2024-03-29T08:07:17.488 INFO:tasks.workunit.client.1.smithi148.stdout:5/448: creat d0/db/d7b/d16/d87/f90 x:0 0 0 2024-03-29T08:07:17.555 INFO:tasks.workunit.client.1.smithi148.stdout:3/452: read de/f2c [1002587,65507] 0 2024-03-29T08:07:17.556 INFO:tasks.workunit.client.1.smithi148.stdout:6/414: unlink d6/d9/d1e/d20/d30/d51/c6d 0 2024-03-29T08:07:17.558 INFO:tasks.workunit.client.1.smithi148.stdout:5/449: unlink d0/db/d7b/f50 0 2024-03-29T08:07:17.558 INFO:tasks.workunit.client.1.smithi148.stdout:0/448: write db/de/df/d11/d2d/d3e/f45 [1405292,80064] 0 2024-03-29T08:07:17.560 INFO:tasks.workunit.client.1.smithi148.stdout:9/420: rename db/d22/d26/d54/d4d/d35/d4b/f88 to db/d22/d26/d54/d4d/f8d 0 2024-03-29T08:07:17.562 INFO:tasks.workunit.client.1.smithi148.stdout:9/421: write db/d22/d26/d54/f3c [127599,64927] 0 2024-03-29T08:07:17.563 INFO:tasks.workunit.client.1.smithi148.stdout:4/415: truncate df/d2d/f49 2225951 0 2024-03-29T08:07:17.564 INFO:tasks.workunit.client.1.smithi148.stdout:7/434: link d8/d29/f35 d8/d60/f98 0 2024-03-29T08:07:17.566 INFO:tasks.workunit.client.1.smithi148.stdout:4/416: truncate df/d71/f7b 943320 0 2024-03-29T08:07:17.566 INFO:tasks.workunit.client.1.smithi148.stdout:7/435: chown d8/d14/d19/c81 3843972 1 2024-03-29T08:07:17.567 INFO:tasks.workunit.client.1.smithi148.stdout:9/422: write db/d22/d26/d54/d4d/d35/d4b/d58/f8b [871818,19302] 0 2024-03-29T08:07:17.568 INFO:tasks.workunit.client.1.smithi148.stdout:8/482: rename d17/d18/d26/d63/d66/f91 to d17/d18/d26/d63/fa0 0 2024-03-29T08:07:17.570 INFO:tasks.workunit.client.1.smithi148.stdout:3/453: creat de/d24/d33/d39/d46/d30/d37/d62/d63/f99 x:0 0 0 2024-03-29T08:07:17.570 INFO:tasks.workunit.client.1.smithi148.stdout:7/436: write d8/d29/d44/d50/f8b [559377,35881] 0 2024-03-29T08:07:17.576 INFO:tasks.workunit.client.1.smithi148.stdout:5/450: dread d0/db/d7b/d16/d2e/f68 [0,4194304] 0 2024-03-29T08:07:17.581 INFO:tasks.workunit.client.1.smithi148.stdout:2/452: truncate d7/da/d33/d45/d4d/d2e/d37/f3d 1896887 0 2024-03-29T08:07:17.581 INFO:tasks.workunit.client.1.smithi148.stdout:1/427: rename d3/de/d1d/d47/d6d/l78 to d3/de/d4f/l82 0 2024-03-29T08:07:17.582 INFO:tasks.workunit.client.1.smithi148.stdout:2/453: dread - d7/d24/f60 zero size 2024-03-29T08:07:17.583 INFO:tasks.workunit.client.1.smithi148.stdout:1/428: stat d3/de/d25/d35/d42 0 2024-03-29T08:07:17.585 INFO:tasks.workunit.client.1.smithi148.stdout:1/429: fdatasync d3/de/d25/d35/f3c 0 2024-03-29T08:07:17.586 INFO:tasks.workunit.client.1.smithi148.stdout:9/423: mkdir db/d22/d26/d54/d4d/d35/d4b/d8e 0 2024-03-29T08:07:17.586 INFO:tasks.workunit.client.1.smithi148.stdout:2/454: truncate d7/da/d33/d45/f6b 600649 0 2024-03-29T08:07:17.586 INFO:tasks.workunit.client.1.smithi148.stdout:5/451: creat d0/db/d7b/f91 x:0 0 0 2024-03-29T08:07:17.587 INFO:tasks.workunit.client.1.smithi148.stdout:2/455: fsync d7/da/f3f 0 2024-03-29T08:07:17.589 INFO:tasks.workunit.client.1.smithi148.stdout:8/483: mknod d17/d74/d39/ca1 0 2024-03-29T08:07:17.590 INFO:tasks.workunit.client.1.smithi148.stdout:8/484: stat f7 0 2024-03-29T08:07:17.593 INFO:tasks.workunit.client.1.smithi148.stdout:3/454: dread de/f2f [0,4194304] 0 2024-03-29T08:07:17.593 INFO:tasks.workunit.client.1.smithi148.stdout:6/415: rename d6/d9/f13 to d6/d9/d1e/d27/f7f 0 2024-03-29T08:07:17.600 INFO:tasks.workunit.client.1.smithi148.stdout:1/430: symlink d3/de/d25/d35/d42/d23/l83 0 2024-03-29T08:07:17.604 INFO:tasks.workunit.client.1.smithi148.stdout:9/424: mknod db/d10/d2d/d4e/d5f/d6e/d7a/c8f 0 2024-03-29T08:07:17.606 INFO:tasks.workunit.client.1.smithi148.stdout:9/425: chown db/d22/d26/d54 0 1 2024-03-29T08:07:17.608 INFO:tasks.workunit.client.0.smithi037.stdout:9/526: getdents d1/d3 0 2024-03-29T08:07:17.609 INFO:tasks.workunit.client.1.smithi148.stdout:6/416: dwrite f5 [0,4194304] 0 2024-03-29T08:07:17.610 INFO:tasks.workunit.client.0.smithi037.stdout:9/527: chown d1/d1f/d3a/d67 3494 1 2024-03-29T08:07:17.610 INFO:tasks.workunit.client.1.smithi148.stdout:8/485: unlink d17/f28 0 2024-03-29T08:07:17.616 INFO:tasks.workunit.client.1.smithi148.stdout:3/455: readlink de/d24/d33/d39/d46/l5b 0 2024-03-29T08:07:17.619 INFO:tasks.workunit.client.0.smithi037.stdout:7/357: mkdir dd/d72 0 2024-03-29T08:07:17.619 INFO:tasks.workunit.client.1.smithi148.stdout:5/452: creat d0/db/d1e/d6e/f92 x:0 0 0 2024-03-29T08:07:17.622 INFO:tasks.workunit.client.1.smithi148.stdout:1/431: dwrite d3/de/d21/f80 [0,4194304] 0 2024-03-29T08:07:17.623 INFO:tasks.workunit.client.1.smithi148.stdout:9/426: symlink db/d22/d26/d54/d4d/l90 0 2024-03-29T08:07:17.623 INFO:tasks.workunit.client.1.smithi148.stdout:4/417: rename df/d2c/d51/d77/f6e to df/d2c/f8d 0 2024-03-29T08:07:17.624 INFO:tasks.workunit.client.0.smithi037.stdout:5/483: readlink d8/d22/d37/d8d/l95 0 2024-03-29T08:07:17.624 INFO:tasks.workunit.client.0.smithi037.stdout:0/199: unlink d9/c26 0 2024-03-29T08:07:17.625 INFO:tasks.workunit.client.0.smithi037.stdout:5/484: readlink d8/l5d 0 2024-03-29T08:07:17.627 INFO:tasks.workunit.client.0.smithi037.stdout:0/200: chown d9/da/dd/d20/c32 1 1 2024-03-29T08:07:17.630 INFO:tasks.workunit.client.0.smithi037.stdout:5/485: truncate d8/d22/d4f/d70/d9c/f97 307684 0 2024-03-29T08:07:17.633 INFO:tasks.workunit.client.0.smithi037.stdout:5/486: write d8/d18/f43 [117114,16095] 0 2024-03-29T08:07:17.633 INFO:tasks.workunit.client.1.smithi148.stdout:7/437: rename d8/d14/d19/c81 to d8/d38/d3c/c99 0 2024-03-29T08:07:17.633 INFO:tasks.workunit.client.1.smithi148.stdout:9/427: write db/d22/d26/f2a [1907679,98734] 0 2024-03-29T08:07:17.635 INFO:tasks.workunit.client.1.smithi148.stdout:5/453: truncate d0/db/d1e/d36/d41/f63 5260504 0 2024-03-29T08:07:17.636 INFO:tasks.workunit.client.1.smithi148.stdout:8/486: getdents d17/d18/d26/d63/d66 0 2024-03-29T08:07:17.638 INFO:tasks.workunit.client.1.smithi148.stdout:8/487: fsync f15 0 2024-03-29T08:07:17.640 INFO:tasks.workunit.client.1.smithi148.stdout:9/428: mknod db/d22/c91 0 2024-03-29T08:07:17.642 INFO:tasks.workunit.client.0.smithi037.stdout:4/342: sync 2024-03-29T08:07:17.644 INFO:tasks.workunit.client.0.smithi037.stdout:4/343: fsync d3/d4/d15/f63 0 2024-03-29T08:07:17.646 INFO:tasks.workunit.client.1.smithi148.stdout:1/432: dread d3/d18/f4b [0,4194304] 0 2024-03-29T08:07:17.646 INFO:tasks.workunit.client.1.smithi148.stdout:8/488: truncate d17/f23 4096879 0 2024-03-29T08:07:17.647 INFO:tasks.workunit.client.0.smithi037.stdout:6/300: creat d0/d5/d1a/d61/f65 x:0 0 0 2024-03-29T08:07:17.649 INFO:tasks.workunit.client.1.smithi148.stdout:8/489: readlink d17/d74/l53 0 2024-03-29T08:07:17.650 INFO:tasks.workunit.client.0.smithi037.stdout:0/201: rmdir d9/da/dd/d20 39 2024-03-29T08:07:17.650 INFO:tasks.workunit.client.0.smithi037.stdout:6/301: read d0/d5/f9 [3845629,114379] 0 2024-03-29T08:07:17.652 INFO:tasks.workunit.client.0.smithi037.stdout:0/202: write f5 [2441435,47074] 0 2024-03-29T08:07:17.654 INFO:tasks.workunit.client.0.smithi037.stdout:2/224: creat d0/d20/d22/d4e/f4f x:0 0 0 2024-03-29T08:07:17.655 INFO:tasks.workunit.client.1.smithi148.stdout:4/418: dwrite df/d2c/f2e [0,4194304] 0 2024-03-29T08:07:17.655 INFO:tasks.workunit.client.0.smithi037.stdout:4/344: mkdir d3/d4/d15/d1d/db/d45/d1e/d59/d73 0 2024-03-29T08:07:17.656 INFO:tasks.workunit.client.1.smithi148.stdout:1/433: mkdir d3/de/d1d/d47/d6d/d84 0 2024-03-29T08:07:17.658 INFO:tasks.workunit.client.1.smithi148.stdout:8/490: creat d17/d18/d26/d4f/d94/fa2 x:0 0 0 2024-03-29T08:07:17.661 INFO:tasks.workunit.client.1.smithi148.stdout:7/438: dread d8/d60/f70 [0,4194304] 0 2024-03-29T08:07:17.662 INFO:tasks.workunit.client.1.smithi148.stdout:8/491: readlink d17/d1d/d7e/l80 0 2024-03-29T08:07:17.663 INFO:tasks.workunit.client.1.smithi148.stdout:3/456: truncate de/f15 132604 0 2024-03-29T08:07:17.664 INFO:tasks.workunit.client.1.smithi148.stdout:2/456: dread d7/da/d33/d3c/f23 [0,4194304] 0 2024-03-29T08:07:17.664 INFO:tasks.workunit.client.1.smithi148.stdout:4/419: dread df/d2d/f35 [0,4194304] 0 2024-03-29T08:07:17.666 INFO:tasks.workunit.client.0.smithi037.stdout:7/358: truncate dd/f17 3769741 0 2024-03-29T08:07:17.667 INFO:tasks.workunit.client.0.smithi037.stdout:6/302: fsync d0/d20/f4d 0 2024-03-29T08:07:17.669 INFO:tasks.workunit.client.1.smithi148.stdout:1/434: mkdir d3/d18/d4d/d76/d85 0 2024-03-29T08:07:17.669 INFO:tasks.workunit.client.1.smithi148.stdout:9/429: write db/d22/d26/f30 [90942,41106] 0 2024-03-29T08:07:17.670 INFO:tasks.workunit.client.0.smithi037.stdout:2/225: creat d0/d20/d22/f50 x:0 0 0 2024-03-29T08:07:17.671 INFO:tasks.workunit.client.1.smithi148.stdout:5/454: dwrite d0/db/d7b/d16/f2f [0,4194304] 0 2024-03-29T08:07:17.672 INFO:tasks.workunit.client.1.smithi148.stdout:1/435: dread - d3/f5d zero size 2024-03-29T08:07:17.676 INFO:tasks.workunit.client.1.smithi148.stdout:4/420: readlink df/d2c/d51/d77/d26/l44 0 2024-03-29T08:07:17.678 INFO:tasks.workunit.client.1.smithi148.stdout:9/430: chown db/d22/d3f/l83 106031 1 2024-03-29T08:07:17.678 INFO:tasks.workunit.client.1.smithi148.stdout:1/436: chown d3/de/d25/l3f 442146 1 2024-03-29T08:07:17.678 INFO:tasks.workunit.client.1.smithi148.stdout:1/437: stat d3/cb 0 2024-03-29T08:07:17.684 INFO:tasks.workunit.client.1.smithi148.stdout:4/421: chown df/d2c/d51/d77/d17/f29 2030256 1 2024-03-29T08:07:17.688 INFO:tasks.workunit.client.1.smithi148.stdout:4/422: stat df/d2c/d51/d77/d26/f63 0 2024-03-29T08:07:17.691 INFO:tasks.workunit.client.1.smithi148.stdout:4/423: write df/f4d [982248,56787] 0 2024-03-29T08:07:17.691 INFO:tasks.workunit.client.0.smithi037.stdout:0/203: dwrite f5 [0,4194304] 0 2024-03-29T08:07:17.692 INFO:tasks.workunit.client.1.smithi148.stdout:2/457: dread d7/da/d33/d45/d4d/d2e/d37/f67 [0,4194304] 0 2024-03-29T08:07:17.698 INFO:tasks.workunit.client.0.smithi037.stdout:0/204: dread - d9/da/f34 zero size 2024-03-29T08:07:17.713 INFO:tasks.workunit.client.1.smithi148.stdout:9/431: dread db/d22/d26/d54/f36 [0,4194304] 0 2024-03-29T08:07:17.717 INFO:tasks.workunit.client.1.smithi148.stdout:9/432: write db/d10/d2d/f7b [1543197,58908] 0 2024-03-29T08:07:17.718 INFO:tasks.workunit.client.1.smithi148.stdout:9/433: readlink db/d10/l25 0 2024-03-29T08:07:17.719 INFO:tasks.workunit.client.1.smithi148.stdout:9/434: read db/f43 [6718751,70643] 0 2024-03-29T08:07:17.730 INFO:tasks.workunit.client.1.smithi148.stdout:8/492: dread d17/d18/f8a [0,4194304] 0 2024-03-29T08:07:17.731 INFO:tasks.workunit.client.1.smithi148.stdout:8/493: chown f15 735430589 1 2024-03-29T08:07:17.739 INFO:tasks.workunit.client.1.smithi148.stdout:2/458: dread d7/da/d33/d45/d4d/d2e/f50 [4194304,4194304] 0 2024-03-29T08:07:17.833 INFO:tasks.workunit.client.1.smithi148.stdout:5/455: symlink d0/db/d1e/d5b/d66/l93 0 2024-03-29T08:07:17.836 INFO:tasks.workunit.client.1.smithi148.stdout:4/424: stat df/d2c/d51/d77/l39 0 2024-03-29T08:07:17.842 INFO:tasks.workunit.client.1.smithi148.stdout:9/435: creat db/d22/d26/d54/d4d/d35/d4b/d58/f92 x:0 0 0 2024-03-29T08:07:17.843 INFO:tasks.workunit.client.1.smithi148.stdout:8/494: stat d17/l9e 0 2024-03-29T08:07:17.846 INFO:tasks.workunit.client.1.smithi148.stdout:2/459: rename d7/da/d33/d3c/d21 to d7/d8c/d91 0 2024-03-29T08:07:17.846 INFO:tasks.workunit.client.1.smithi148.stdout:8/495: write d17/d18/d26/d63/d66/d5f/f6c [4094317,69547] 0 2024-03-29T08:07:17.846 INFO:tasks.workunit.client.1.smithi148.stdout:2/460: fsync d7/da/f89 0 2024-03-29T08:07:17.847 INFO:tasks.workunit.client.1.smithi148.stdout:8/496: chown d17/d18/l58 64638995 1 2024-03-29T08:07:17.853 INFO:tasks.workunit.client.1.smithi148.stdout:5/456: dread - d0/d46/f8e zero size 2024-03-29T08:07:17.855 INFO:tasks.workunit.client.0.smithi037.stdout:1/385: sync 2024-03-29T08:07:17.856 INFO:tasks.workunit.client.0.smithi037.stdout:3/339: sync 2024-03-29T08:07:17.856 INFO:tasks.workunit.client.0.smithi037.stdout:8/388: sync 2024-03-29T08:07:17.857 INFO:tasks.workunit.client.1.smithi148.stdout:9/436: fdatasync db/d22/d26/d54/f2e 0 2024-03-29T08:07:17.858 INFO:tasks.workunit.client.0.smithi037.stdout:8/389: fdatasync d5/d10/d19/d3d/f5d 0 2024-03-29T08:07:17.861 INFO:tasks.workunit.client.1.smithi148.stdout:8/497: dread d17/d74/d39/f73 [0,4194304] 0 2024-03-29T08:07:17.864 INFO:tasks.workunit.client.1.smithi148.stdout:3/457: link de/d24/d33/f5c de/d24/d33/f9a 0 2024-03-29T08:07:17.865 INFO:tasks.workunit.client.1.smithi148.stdout:2/461: dread - d7/da/d33/d3c/d31/f4a zero size 2024-03-29T08:07:17.866 INFO:tasks.workunit.client.1.smithi148.stdout:1/438: creat d3/de/f86 x:0 0 0 2024-03-29T08:07:17.868 INFO:tasks.workunit.client.1.smithi148.stdout:4/425: symlink df/d2c/d80/l8e 0 2024-03-29T08:07:17.869 INFO:tasks.workunit.client.1.smithi148.stdout:5/457: mknod d0/db/d1e/d35/c94 0 2024-03-29T08:07:17.871 INFO:tasks.workunit.client.1.smithi148.stdout:3/458: creat de/d24/d33/d39/d71/f9b x:0 0 0 2024-03-29T08:07:17.874 INFO:tasks.workunit.client.1.smithi148.stdout:2/462: rmdir d7/da/d33/d3c/d54/d70 39 2024-03-29T08:07:17.874 INFO:tasks.workunit.client.1.smithi148.stdout:1/439: chown d3/de/d25/d35/d42/d23/d61 58 1 2024-03-29T08:07:17.875 INFO:tasks.workunit.client.1.smithi148.stdout:8/498: dwrite d17/d18/f67 [0,4194304] 0 2024-03-29T08:07:17.879 INFO:tasks.workunit.client.1.smithi148.stdout:9/437: mknod db/d22/d26/d54/d4d/d35/c93 0 2024-03-29T08:07:17.880 INFO:tasks.workunit.client.1.smithi148.stdout:4/426: fdatasync df/d2c/d51/d77/d17/f25 0 2024-03-29T08:07:17.881 INFO:tasks.workunit.client.0.smithi037.stdout:9/528: getdents d1/d1f 0 2024-03-29T08:07:17.882 INFO:tasks.workunit.client.1.smithi148.stdout:9/438: write db/d22/d26/d54/f49 [2615894,117491] 0 2024-03-29T08:07:17.882 INFO:tasks.workunit.client.1.smithi148.stdout:4/427: chown df/l23 3552 1 2024-03-29T08:07:17.882 INFO:tasks.workunit.client.1.smithi148.stdout:3/459: read de/d24/d33/d39/d46/d30/f86 [2117612,125013] 0 2024-03-29T08:07:17.882 INFO:tasks.workunit.client.0.smithi037.stdout:4/345: rmdir d3 39 2024-03-29T08:07:17.887 INFO:tasks.workunit.client.1.smithi148.stdout:6/417: sync 2024-03-29T08:07:17.890 INFO:tasks.workunit.client.1.smithi148.stdout:2/463: dwrite d7/da/f89 [0,4194304] 0 2024-03-29T08:07:17.890 INFO:tasks.workunit.client.1.smithi148.stdout:8/499: mknod d17/d74/d39/d6f/d64/ca3 0 2024-03-29T08:07:17.892 INFO:tasks.workunit.client.1.smithi148.stdout:6/418: truncate d6/d9/d1e/d22/f4a 1356415 0 2024-03-29T08:07:17.895 INFO:tasks.workunit.client.1.smithi148.stdout:5/458: mkdir d0/db/d1e/d5b/d66/d72/d95 0 2024-03-29T08:07:17.896 INFO:tasks.workunit.client.1.smithi148.stdout:4/428: creat df/d2c/d51/d77/d17/f8f x:0 0 0 2024-03-29T08:07:17.896 INFO:tasks.workunit.client.0.smithi037.stdout:0/205: mknod d9/c3e 0 2024-03-29T08:07:17.897 INFO:tasks.workunit.client.0.smithi037.stdout:5/487: dwrite d8/d4d/d5c/f62 [0,4194304] 0 2024-03-29T08:07:17.898 INFO:tasks.workunit.client.1.smithi148.stdout:3/460: chown de/d24/d33/d39/d46/d30/d37/d62/c95 49 1 2024-03-29T08:07:17.900 INFO:tasks.workunit.client.1.smithi148.stdout:5/459: readlink d0/db/d1e/d5b/d66/d73/l78 0 2024-03-29T08:07:17.901 INFO:tasks.workunit.client.1.smithi148.stdout:3/461: fdatasync de/d24/d33/d39/d71/f72 0 2024-03-29T08:07:17.903 INFO:tasks.workunit.client.1.smithi148.stdout:9/439: dwrite db/d10/d2d/d4e/f81 [0,4194304] 0 2024-03-29T08:07:17.904 INFO:tasks.workunit.client.1.smithi148.stdout:1/440: mknod d3/de/d25/d35/d42/c87 0 2024-03-29T08:07:17.905 INFO:tasks.workunit.client.0.smithi037.stdout:3/340: rename f4 to dd/d46/f71 0 2024-03-29T08:07:17.905 INFO:tasks.workunit.client.0.smithi037.stdout:8/390: creat d5/d10/d45/f7f x:0 0 0 2024-03-29T08:07:17.906 INFO:tasks.workunit.client.1.smithi148.stdout:9/440: dread - db/d22/d26/d54/d4d/f79 zero size 2024-03-29T08:07:17.906 INFO:tasks.workunit.client.1.smithi148.stdout:9/441: stat db/l1f 0 2024-03-29T08:07:17.906 INFO:tasks.workunit.client.1.smithi148.stdout:1/441: fdatasync d3/de/d25/d35/d42/d23/f59 0 2024-03-29T08:07:17.907 INFO:tasks.workunit.client.0.smithi037.stdout:5/488: fdatasync d8/d22/d4f/d70/f3f 0 2024-03-29T08:07:17.909 INFO:tasks.workunit.client.1.smithi148.stdout:1/442: fsync d3/de/d25/d35/d42/d23/f40 0 2024-03-29T08:07:17.911 INFO:tasks.workunit.client.0.smithi037.stdout:6/303: getdents d0/d19/d5c 0 2024-03-29T08:07:17.912 INFO:tasks.workunit.client.1.smithi148.stdout:8/500: mknod d17/d4e/ca4 0 2024-03-29T08:07:17.912 INFO:tasks.workunit.client.0.smithi037.stdout:1/386: mknod d2/d6/c81 0 2024-03-29T08:07:17.912 INFO:tasks.workunit.client.1.smithi148.stdout:1/443: truncate d3/de/d25/d51/f73 799398 0 2024-03-29T08:07:17.913 INFO:tasks.workunit.client.0.smithi037.stdout:8/391: write d5/d10/d2d/f60 [2164893,87307] 0 2024-03-29T08:07:17.917 INFO:tasks.workunit.client.0.smithi037.stdout:4/346: write d3/d4/d27/f2e [1359466,129648] 0 2024-03-29T08:07:17.920 INFO:tasks.workunit.client.0.smithi037.stdout:6/304: stat d0/d5/d1a/f2f 0 2024-03-29T08:07:17.920 INFO:tasks.workunit.client.1.smithi148.stdout:5/460: mkdir d0/db/d1e/d35/d42/d96 0 2024-03-29T08:07:17.921 INFO:tasks.workunit.client.0.smithi037.stdout:3/341: creat dd/d5e/f72 x:0 0 0 2024-03-29T08:07:17.924 INFO:tasks.workunit.client.1.smithi148.stdout:8/501: dread d17/d74/d39/f76 [0,4194304] 0 2024-03-29T08:07:17.924 INFO:tasks.workunit.client.1.smithi148.stdout:3/462: dread de/f15 [0,4194304] 0 2024-03-29T08:07:17.925 INFO:tasks.workunit.client.0.smithi037.stdout:0/206: symlink d9/da/dd/d13/l3f 0 2024-03-29T08:07:17.926 INFO:tasks.workunit.client.1.smithi148.stdout:1/444: write d3/de/f57 [256297,129128] 0 2024-03-29T08:07:17.929 INFO:tasks.workunit.client.1.smithi148.stdout:9/442: mkdir db/d22/d3f/d61/d94 0 2024-03-29T08:07:17.931 INFO:tasks.workunit.client.0.smithi037.stdout:9/529: rename d1/d1f/d3a/c1c to d1/d4/d19/d1e/d4a/d6f/cb5 0 2024-03-29T08:07:17.932 INFO:tasks.workunit.client.0.smithi037.stdout:5/489: write d8/d18/d24/f74 [46098,21597] 0 2024-03-29T08:07:17.933 INFO:tasks.workunit.client.1.smithi148.stdout:8/502: chown d17/d18/l24 1602 1 2024-03-29T08:07:17.933 INFO:tasks.workunit.client.0.smithi037.stdout:7/359: dwrite dd/f1f [0,4194304] 0 2024-03-29T08:07:17.934 INFO:tasks.workunit.client.0.smithi037.stdout:3/342: mknod dd/d46/c73 0 2024-03-29T08:07:17.935 INFO:tasks.workunit.client.1.smithi148.stdout:8/503: chown d17/d1d/d7e/l80 24 1 2024-03-29T08:07:17.935 INFO:tasks.workunit.client.1.smithi148.stdout:8/504: fsync fe 0 2024-03-29T08:07:17.936 INFO:tasks.workunit.client.0.smithi037.stdout:4/347: symlink d3/d4/d15/d1d/db/d45/d1e/l74 0 2024-03-29T08:07:17.937 INFO:tasks.workunit.client.0.smithi037.stdout:6/305: creat d0/d19/d34/f66 x:0 0 0 2024-03-29T08:07:17.938 INFO:tasks.workunit.client.1.smithi148.stdout:8/505: read - d17/d18/d26/d63/fa0 zero size 2024-03-29T08:07:17.939 INFO:tasks.workunit.client.0.smithi037.stdout:4/348: fdatasync d3/f66 0 2024-03-29T08:07:17.939 INFO:tasks.workunit.client.0.smithi037.stdout:6/306: chown d0/d5/d1f/l22 9216149 1 2024-03-29T08:07:17.943 INFO:tasks.workunit.client.0.smithi037.stdout:6/307: read d0/d5/fb [2053503,15887] 0 2024-03-29T08:07:17.945 INFO:tasks.workunit.client.1.smithi148.stdout:3/463: dread de/d24/d33/d39/d46/d30/d37/d62/d63/f92 [0,4194304] 0 2024-03-29T08:07:17.953 INFO:tasks.workunit.client.0.smithi037.stdout:0/207: unlink d9/da/f34 0 2024-03-29T08:07:17.957 INFO:tasks.workunit.client.0.smithi037.stdout:1/387: truncate d2/d6/dc/d18/d1d/d21/d38/f56 283893 0 2024-03-29T08:07:17.960 INFO:tasks.workunit.client.1.smithi148.stdout:3/464: dwrite de/d24/d33/d39/d46/d30/d37/d79/f7c [0,4194304] 0 2024-03-29T08:07:17.961 INFO:tasks.workunit.client.1.smithi148.stdout:3/465: stat de/d24/f36 0 2024-03-29T08:07:17.966 INFO:tasks.workunit.client.1.smithi148.stdout:4/429: creat df/d2c/d51/d77/f90 x:0 0 0 2024-03-29T08:07:17.967 INFO:tasks.workunit.client.0.smithi037.stdout:3/343: fsync dd/d15/f62 0 2024-03-29T08:07:17.968 INFO:tasks.workunit.client.1.smithi148.stdout:4/430: read - df/d2c/d51/d77/f90 zero size 2024-03-29T08:07:17.969 INFO:tasks.workunit.client.0.smithi037.stdout:3/344: chown dd/d46/d3e 243862885 1 2024-03-29T08:07:17.971 INFO:tasks.workunit.client.0.smithi037.stdout:8/392: creat d5/f80 x:0 0 0 2024-03-29T08:07:17.979 INFO:tasks.workunit.client.1.smithi148.stdout:9/443: symlink db/d22/d26/d54/d4d/d35/d4b/d58/l95 0 2024-03-29T08:07:17.983 INFO:tasks.workunit.client.0.smithi037.stdout:2/226: dwrite d0/fd [0,4194304] 0 2024-03-29T08:07:17.984 INFO:tasks.workunit.client.1.smithi148.stdout:8/506: mkdir d17/d74/d39/d6f/d64/da5 0 2024-03-29T08:07:17.986 INFO:tasks.workunit.client.0.smithi037.stdout:1/388: chown d2/f6c 4 1 2024-03-29T08:07:17.990 INFO:tasks.workunit.client.1.smithi148.stdout:3/466: rmdir de/d24/d33/d39/d71 39 2024-03-29T08:07:17.993 INFO:tasks.workunit.client.1.smithi148.stdout:3/467: fdatasync de/d24/d33/d39/d46/d30/d37/f7a 0 2024-03-29T08:07:17.996 INFO:tasks.workunit.client.1.smithi148.stdout:5/461: symlink d0/db/d1e/d35/d42/d67/d76/d81/l97 0 2024-03-29T08:07:17.998 INFO:tasks.workunit.client.1.smithi148.stdout:4/431: mkdir df/d2c/d51/d77/d26/d34/d43/d91 0 2024-03-29T08:07:18.000 INFO:tasks.workunit.client.1.smithi148.stdout:5/462: chown d0/db/d7b/c25 149 1 2024-03-29T08:07:18.000 INFO:tasks.workunit.client.1.smithi148.stdout:9/444: rename db/d22/d26/d54/d4d/c4c to db/d22/d26/d54/d52/c96 0 2024-03-29T08:07:18.002 INFO:tasks.workunit.client.1.smithi148.stdout:6/419: getdents d6/d9/d1e 0 2024-03-29T08:07:18.010 INFO:tasks.workunit.client.1.smithi148.stdout:6/420: symlink d6/d9/d1e/d20/d6b/l80 0 2024-03-29T08:07:18.013 INFO:tasks.workunit.client.1.smithi148.stdout:6/421: mknod d6/d9/d1e/d22/c81 0 2024-03-29T08:07:18.014 INFO:tasks.workunit.client.0.smithi037.stdout:2/227: dwrite d0/d20/d22/f3c [0,4194304] 0 2024-03-29T08:07:18.016 INFO:tasks.workunit.client.0.smithi037.stdout:9/530: creat d1/fb6 x:0 0 0 2024-03-29T08:07:18.017 INFO:tasks.workunit.client.0.smithi037.stdout:5/490: creat d8/d18/d24/fa6 x:0 0 0 2024-03-29T08:07:18.020 INFO:tasks.workunit.client.1.smithi148.stdout:4/432: read fa [158185,81078] 0 2024-03-29T08:07:18.020 INFO:tasks.workunit.client.0.smithi037.stdout:8/393: dread d5/d10/f1d [0,4194304] 0 2024-03-29T08:07:18.021 INFO:tasks.workunit.client.1.smithi148.stdout:4/433: stat df/d2c/d51/f67 0 2024-03-29T08:07:18.024 INFO:tasks.workunit.client.1.smithi148.stdout:4/434: write df/d2c/d51/f67 [99274,75394] 0 2024-03-29T08:07:18.024 INFO:tasks.workunit.client.0.smithi037.stdout:4/349: dread d3/d57/f67 [0,4194304] 0 2024-03-29T08:07:18.029 INFO:tasks.workunit.client.1.smithi148.stdout:0/449: dwrite db/de/d24/f38 [0,4194304] 0 2024-03-29T08:07:18.029 INFO:tasks.workunit.client.1.smithi148.stdout:3/468: truncate de/f51 1581762 0 2024-03-29T08:07:18.030 INFO:tasks.workunit.client.1.smithi148.stdout:0/450: dread - db/dc/d78/f8b zero size 2024-03-29T08:07:18.031 INFO:tasks.workunit.client.0.smithi037.stdout:1/389: symlink d2/d6/dc/d18/d1d/d21/d38/l82 0 2024-03-29T08:07:18.032 INFO:tasks.workunit.client.1.smithi148.stdout:4/435: mkdir df/d92 0 2024-03-29T08:07:18.033 INFO:tasks.workunit.client.1.smithi148.stdout:9/445: truncate db/d10/d2d/f34 1992463 0 2024-03-29T08:07:18.033 INFO:tasks.workunit.client.1.smithi148.stdout:3/469: read de/d24/d33/f8e [1616314,92531] 0 2024-03-29T08:07:18.036 INFO:tasks.workunit.client.1.smithi148.stdout:9/446: write db/d20/f1b [4776354,76196] 0 2024-03-29T08:07:18.037 INFO:tasks.workunit.client.0.smithi037.stdout:3/345: rmdir dd/d46/d2d/d3a/d5f 39 2024-03-29T08:07:18.040 INFO:tasks.workunit.client.1.smithi148.stdout:3/470: symlink de/d3b/d49/l9c 0 2024-03-29T08:07:18.040 INFO:tasks.workunit.client.0.smithi037.stdout:3/346: chown dd/d15/d18/d4b/f67 2139979799 1 2024-03-29T08:07:18.044 INFO:tasks.workunit.client.1.smithi148.stdout:6/422: write d6/d9/d1e/d27/d48/f7b [53654,71676] 0 2024-03-29T08:07:18.047 INFO:tasks.workunit.client.1.smithi148.stdout:9/447: rmdir db/d10/d2d/d4e/d5f/d6e 39 2024-03-29T08:07:18.049 INFO:tasks.workunit.client.1.smithi148.stdout:4/436: dwrite df/d2c/d51/d77/d26/f3d [0,4194304] 0 2024-03-29T08:07:18.050 INFO:tasks.workunit.client.1.smithi148.stdout:3/471: symlink de/d24/d33/d6b/l9d 0 2024-03-29T08:07:18.050 INFO:tasks.workunit.client.0.smithi037.stdout:2/228: chown d0/c24 1980 1 2024-03-29T08:07:18.051 INFO:tasks.workunit.client.1.smithi148.stdout:7/439: sync 2024-03-29T08:07:18.051 INFO:tasks.workunit.client.1.smithi148.stdout:3/472: stat de/l28 0 2024-03-29T08:07:18.051 INFO:tasks.workunit.client.1.smithi148.stdout:3/473: fdatasync de/d24/f31 0 2024-03-29T08:07:18.054 INFO:tasks.workunit.client.0.smithi037.stdout:0/208: dwrite d9/fb [0,4194304] 0 2024-03-29T08:07:18.056 INFO:tasks.workunit.client.0.smithi037.stdout:2/229: dread - d0/d20/d22/f3f zero size 2024-03-29T08:07:18.056 INFO:tasks.workunit.client.1.smithi148.stdout:9/448: creat db/d22/d26/d54/d4d/d35/d4b/d8e/f97 x:0 0 0 2024-03-29T08:07:18.056 INFO:tasks.workunit.client.1.smithi148.stdout:3/474: symlink de/d24/d33/d39/d46/d8d/l9e 0 2024-03-29T08:07:18.059 INFO:tasks.workunit.client.0.smithi037.stdout:7/360: link dd/le dd/d21/d35/d42/d5c/l73 0 2024-03-29T08:07:18.059 INFO:tasks.workunit.client.0.smithi037.stdout:4/350: dwrite d3/d4/d15/d1d/db/f16 [0,4194304] 0 2024-03-29T08:07:18.060 INFO:tasks.workunit.client.1.smithi148.stdout:3/475: write de/d24/d33/d39/d46/d30/d37/d62/d63/f99 [553385,33295] 0 2024-03-29T08:07:18.065 INFO:tasks.workunit.client.0.smithi037.stdout:4/351: write d3/d4/d15/d1d/f71 [1308324,65906] 0 2024-03-29T08:07:18.071 INFO:tasks.workunit.client.0.smithi037.stdout:9/531: symlink d1/d4/d19/d1e/d4a/d6f/da2/d31/d87/lb7 0 2024-03-29T08:07:18.076 INFO:tasks.workunit.client.0.smithi037.stdout:9/532: truncate d1/f8d 903033 0 2024-03-29T08:07:18.076 INFO:tasks.workunit.client.1.smithi148.stdout:9/449: dread db/d22/d26/d54/d4d/f18 [4194304,4194304] 0 2024-03-29T08:07:18.079 INFO:tasks.workunit.client.0.smithi037.stdout:9/533: chown d1/d4/f28 5318292 1 2024-03-29T08:07:18.079 INFO:tasks.workunit.client.0.smithi037.stdout:5/491: symlink d8/d22/d4f/d70/d9c/la7 0 2024-03-29T08:07:18.081 INFO:tasks.workunit.client.0.smithi037.stdout:5/492: readlink d8/d22/l32 0 2024-03-29T08:07:18.092 INFO:tasks.workunit.client.0.smithi037.stdout:6/308: getdents d0/d19/d42/d60 0 2024-03-29T08:07:18.094 INFO:tasks.workunit.client.0.smithi037.stdout:3/347: symlink dd/d15/d27/l74 0 2024-03-29T08:07:18.103 INFO:tasks.workunit.client.0.smithi037.stdout:2/230: read d0/fa [1422377,98718] 0 2024-03-29T08:07:18.103 INFO:tasks.workunit.client.0.smithi037.stdout:7/361: creat dd/d21/d35/d42/d5c/f74 x:0 0 0 2024-03-29T08:07:18.103 INFO:tasks.workunit.client.0.smithi037.stdout:8/394: dwrite d5/d10/f23 [4194304,4194304] 0 2024-03-29T08:07:18.104 INFO:tasks.workunit.client.1.smithi148.stdout:4/437: write df/d2c/d51/f57 [785242,125412] 0 2024-03-29T08:07:18.106 INFO:tasks.workunit.client.1.smithi148.stdout:4/438: fdatasync df/d2c/d51/d77/f83 0 2024-03-29T08:07:18.106 INFO:tasks.workunit.client.1.smithi148.stdout:9/450: fsync db/d22/d26/d54/d4d/f8d 0 2024-03-29T08:07:18.107 INFO:tasks.workunit.client.1.smithi148.stdout:3/476: link de/d24/d33/d39/d46/c16 de/d24/d33/d39/d46/d30/c9f 0 2024-03-29T08:07:18.107 INFO:tasks.workunit.client.1.smithi148.stdout:9/451: readlink db/l1f 0 2024-03-29T08:07:18.108 INFO:tasks.workunit.client.0.smithi037.stdout:2/231: write d0/d20/d2e/f36 [76520,15328] 0 2024-03-29T08:07:18.111 INFO:tasks.workunit.client.0.smithi037.stdout:4/352: unlink d3/d4/d15/d1d/db/d45/f30 0 2024-03-29T08:07:18.111 INFO:tasks.workunit.client.1.smithi148.stdout:0/451: getdents db/de/df/d2b/d3b/d47 0 2024-03-29T08:07:18.112 INFO:tasks.workunit.client.0.smithi037.stdout:1/390: dread d2/d4b/f66 [0,4194304] 0 2024-03-29T08:07:18.113 INFO:tasks.workunit.client.1.smithi148.stdout:0/452: dread - db/de/df/d2b/f72 zero size 2024-03-29T08:07:18.116 INFO:tasks.workunit.client.0.smithi037.stdout:3/348: mkdir dd/d15/d27/d33/d75 0 2024-03-29T08:07:18.123 INFO:tasks.workunit.client.0.smithi037.stdout:9/534: dread d1/d4/d19/d1e/d25/d16/f56 [0,4194304] 0 2024-03-29T08:07:18.123 INFO:tasks.workunit.client.0.smithi037.stdout:0/209: symlink d9/da/dd/l40 0 2024-03-29T08:07:18.126 INFO:tasks.workunit.client.1.smithi148.stdout:9/452: rmdir db/d22/d3f/d61 39 2024-03-29T08:07:18.128 INFO:tasks.workunit.client.1.smithi148.stdout:3/477: symlink de/d24/d33/d39/d46/d70/la0 0 2024-03-29T08:07:18.128 INFO:tasks.workunit.client.1.smithi148.stdout:0/453: creat db/de/df/d11/d1f/f92 x:0 0 0 2024-03-29T08:07:18.129 INFO:tasks.workunit.client.1.smithi148.stdout:4/439: dread df/f14 [0,4194304] 0 2024-03-29T08:07:18.130 INFO:tasks.workunit.client.1.smithi148.stdout:3/478: readlink de/d24/d33/d39/d46/d70/l89 0 2024-03-29T08:07:18.132 INFO:tasks.workunit.client.1.smithi148.stdout:4/440: unlink df/d2c/f6c 0 2024-03-29T08:07:18.132 INFO:tasks.workunit.client.1.smithi148.stdout:3/479: chown de/d24/d33/d39/c50 1 1 2024-03-29T08:07:18.132 INFO:tasks.workunit.client.1.smithi148.stdout:9/453: mknod db/d22/d26/d54/c98 0 2024-03-29T08:07:18.135 INFO:tasks.workunit.client.1.smithi148.stdout:9/454: rmdir db/d10/d2d 39 2024-03-29T08:07:18.137 INFO:tasks.workunit.client.1.smithi148.stdout:4/441: rename df/d2c/d51/d77/l39 to df/d2c/l93 0 2024-03-29T08:07:18.138 INFO:tasks.workunit.client.1.smithi148.stdout:3/480: truncate de/d3b/f60 4641229 0 2024-03-29T08:07:18.138 INFO:tasks.workunit.client.0.smithi037.stdout:6/309: rename d0/d19/f41 to d0/d5/d1e/f67 0 2024-03-29T08:07:18.139 INFO:tasks.workunit.client.1.smithi148.stdout:9/455: creat db/d22/d26/d54/d4d/d35/d4b/d8e/f99 x:0 0 0 2024-03-29T08:07:18.139 INFO:tasks.workunit.client.1.smithi148.stdout:4/442: creat df/d2d/f94 x:0 0 0 2024-03-29T08:07:18.140 INFO:tasks.workunit.client.1.smithi148.stdout:0/454: creat db/de/df/d2b/f93 x:0 0 0 2024-03-29T08:07:18.141 INFO:tasks.workunit.client.1.smithi148.stdout:9/456: chown db/d22/d26/d54/f45 62 1 2024-03-29T08:07:18.141 INFO:tasks.workunit.client.1.smithi148.stdout:3/481: readlink de/d24/d33/d39/d46/l6a 0 2024-03-29T08:07:18.143 INFO:tasks.workunit.client.1.smithi148.stdout:0/455: dread - db/de/df/d11/d2d/f85 zero size 2024-03-29T08:07:18.143 INFO:tasks.workunit.client.1.smithi148.stdout:9/457: chown db/lf 51041 1 2024-03-29T08:07:18.146 INFO:tasks.workunit.client.1.smithi148.stdout:4/443: rmdir df/d2c/d51/d77/d26/d34/d43/d91 0 2024-03-29T08:07:18.146 INFO:tasks.workunit.client.1.smithi148.stdout:9/458: chown db/d22/d26/d54/d4d/d35/d4b/d8e/f97 110 1 2024-03-29T08:07:18.146 INFO:tasks.workunit.client.0.smithi037.stdout:7/362: creat dd/d21/f75 x:0 0 0 2024-03-29T08:07:18.147 INFO:tasks.workunit.client.1.smithi148.stdout:3/482: write de/d24/d33/d39/d46/d30/d37/d62/d63/f92 [897408,97925] 0 2024-03-29T08:07:18.148 INFO:tasks.workunit.client.1.smithi148.stdout:4/444: write df/d2d/f94 [1252,59723] 0 2024-03-29T08:07:18.149 INFO:tasks.workunit.client.0.smithi037.stdout:7/363: rename dd to dd/d1d/d34/d54/d76 22 2024-03-29T08:07:18.149 INFO:tasks.workunit.client.0.smithi037.stdout:2/232: chown d0/d23/c37 103 1 2024-03-29T08:07:18.150 INFO:tasks.workunit.client.1.smithi148.stdout:9/459: mkdir db/d22/d3f/d9a 0 2024-03-29T08:07:18.152 INFO:tasks.workunit.client.1.smithi148.stdout:7/440: fdatasync d8/d29/d44/d50/f87 0 2024-03-29T08:07:18.152 INFO:tasks.workunit.client.0.smithi037.stdout:8/395: chown d5/d10/d2d/d34/d50/l5c 3498 1 2024-03-29T08:07:18.153 INFO:tasks.workunit.client.0.smithi037.stdout:8/396: write d5/f17 [6729456,9032] 0 2024-03-29T08:07:18.154 INFO:tasks.workunit.client.1.smithi148.stdout:4/445: creat df/d2c/d51/d77/d26/d34/d8a/f95 x:0 0 0 2024-03-29T08:07:18.154 INFO:tasks.workunit.client.1.smithi148.stdout:9/460: mkdir db/d10/d9b 0 2024-03-29T08:07:18.155 INFO:tasks.workunit.client.0.smithi037.stdout:8/397: chown d5/d78 5296055 1 2024-03-29T08:07:18.157 INFO:tasks.workunit.client.1.smithi148.stdout:4/446: mknod df/d2c/d51/c96 0 2024-03-29T08:07:18.158 INFO:tasks.workunit.client.1.smithi148.stdout:9/461: readlink db/d20/l53 0 2024-03-29T08:07:18.158 INFO:tasks.workunit.client.1.smithi148.stdout:3/483: link de/d24/d33/d39/d46/d70/la0 de/d24/d33/d39/d71/la1 0 2024-03-29T08:07:18.162 INFO:tasks.workunit.client.0.smithi037.stdout:4/353: creat d3/d4/d15/d1d/db/d45/d1e/d59/d60/f75 x:0 0 0 2024-03-29T08:07:18.163 INFO:tasks.workunit.client.1.smithi148.stdout:3/484: creat de/d24/d33/d39/d46/d30/fa2 x:0 0 0 2024-03-29T08:07:18.165 INFO:tasks.workunit.client.1.smithi148.stdout:3/485: chown de/d24/d33/d39/d71/f9b 31184859 1 2024-03-29T08:07:18.166 INFO:tasks.workunit.client.1.smithi148.stdout:3/486: read de/d24/f27 [351281,30965] 0 2024-03-29T08:07:18.167 INFO:tasks.workunit.client.0.smithi037.stdout:3/349: creat dd/d46/d3e/d47/f76 x:0 0 0 2024-03-29T08:07:18.169 INFO:tasks.workunit.client.1.smithi148.stdout:7/441: rename d8/d14/d19/c4a to d8/d29/d44/d50/d94/c9a 0 2024-03-29T08:07:18.169 INFO:tasks.workunit.client.0.smithi037.stdout:2/233: dwrite d0/d23/f30 [0,4194304] 0 2024-03-29T08:07:18.169 INFO:tasks.workunit.client.0.smithi037.stdout:3/350: chown dd/d15/f50 14 1 2024-03-29T08:07:18.170 INFO:tasks.workunit.client.1.smithi148.stdout:0/456: link db/de/df/f90 db/dc/d78/f94 0 2024-03-29T08:07:18.170 INFO:tasks.workunit.client.1.smithi148.stdout:9/462: dread db/f43 [4194304,4194304] 0 2024-03-29T08:07:18.171 INFO:tasks.workunit.client.0.smithi037.stdout:2/234: fdatasync d0/d20/d22/f50 0 2024-03-29T08:07:18.172 INFO:tasks.workunit.client.0.smithi037.stdout:3/351: write dd/d46/f38 [2989930,54904] 0 2024-03-29T08:07:18.177 INFO:tasks.workunit.client.1.smithi148.stdout:7/442: fsync d8/d29/f35 0 2024-03-29T08:07:18.184 INFO:tasks.workunit.client.1.smithi148.stdout:3/487: dwrite de/d24/d33/d39/d46/d30/d37/d62/d63/f92 [0,4194304] 0 2024-03-29T08:07:18.186 INFO:tasks.workunit.client.1.smithi148.stdout:3/488: stat de/d24/c38 0 2024-03-29T08:07:18.188 INFO:tasks.workunit.client.1.smithi148.stdout:3/489: chown de/d24/d33/d39/d7f 1215303128 1 2024-03-29T08:07:18.195 INFO:tasks.workunit.client.1.smithi148.stdout:4/447: rename df/d2c/d51/d77/d17/l24 to df/d2c/d51/d77/d26/d34/d3e/l97 0 2024-03-29T08:07:18.195 INFO:tasks.workunit.client.1.smithi148.stdout:7/443: dwrite d8/f36 [0,4194304] 0 2024-03-29T08:07:18.197 INFO:tasks.workunit.client.1.smithi148.stdout:9/463: mkdir db/d22/d26/d54/d52/d9c 0 2024-03-29T08:07:18.199 INFO:tasks.workunit.client.1.smithi148.stdout:4/448: unlink df/d2c/d51/d77/d17/f1c 0 2024-03-29T08:07:18.200 INFO:tasks.workunit.client.0.smithi037.stdout:6/310: mknod d0/d19/d34/d4e/c68 0 2024-03-29T08:07:18.205 INFO:tasks.workunit.client.1.smithi148.stdout:3/490: mkdir de/d24/d33/d39/da3 0 2024-03-29T08:07:18.206 INFO:tasks.workunit.client.1.smithi148.stdout:0/457: unlink db/de/df/d11/f1b 0 2024-03-29T08:07:18.206 INFO:tasks.workunit.client.1.smithi148.stdout:9/464: mkdir db/d10/d9d 0 2024-03-29T08:07:18.206 INFO:tasks.workunit.client.0.smithi037.stdout:8/398: dread - d5/f3a zero size 2024-03-29T08:07:18.206 INFO:tasks.workunit.client.1.smithi148.stdout:4/449: mkdir df/d2c/d51/d77/d26/d34/d8a/d98 0 2024-03-29T08:07:18.210 INFO:tasks.workunit.client.1.smithi148.stdout:4/450: dread df/d2c/d51/f67 [0,4194304] 0 2024-03-29T08:07:18.210 INFO:tasks.workunit.client.0.smithi037.stdout:4/354: mknod d3/d4/d15/d1d/db/d45/d1e/d59/d60/c76 0 2024-03-29T08:07:18.211 INFO:tasks.workunit.client.1.smithi148.stdout:0/458: truncate db/de/df/d11/f71 271788 0 2024-03-29T08:07:18.211 INFO:tasks.workunit.client.1.smithi148.stdout:4/451: readlink df/l13 0 2024-03-29T08:07:18.212 INFO:tasks.workunit.client.1.smithi148.stdout:4/452: stat df/d71/f88 0 2024-03-29T08:07:18.213 INFO:tasks.workunit.client.0.smithi037.stdout:8/399: stat d5/d10/d19/d3d/d4a/d4c 0 2024-03-29T08:07:18.213 INFO:tasks.workunit.client.0.smithi037.stdout:2/235: mkdir d0/d20/d22/d4e/d51 0 2024-03-29T08:07:18.214 INFO:tasks.workunit.client.1.smithi148.stdout:3/491: getdents de/d3b/d49 0 2024-03-29T08:07:18.216 INFO:tasks.workunit.client.0.smithi037.stdout:2/236: write d0/f1b [1774748,30343] 0 2024-03-29T08:07:18.217 INFO:tasks.workunit.client.0.smithi037.stdout:9/535: mknod d1/d4/d19/d1e/d4a/d6f/da2/d68/cb8 0 2024-03-29T08:07:18.218 INFO:tasks.workunit.client.1.smithi148.stdout:4/453: write df/d2c/d51/d77/d26/d34/d43/f4e [828443,22198] 0 2024-03-29T08:07:18.220 INFO:tasks.workunit.client.1.smithi148.stdout:8/507: chown d17/d18/d26/d63/d66/d5f/f8e 15795 1 2024-03-29T08:07:18.220 INFO:tasks.workunit.client.0.smithi037.stdout:1/391: link d2/l42 d2/d6/dc/d5d/d62/l83 0 2024-03-29T08:07:18.220 INFO:tasks.workunit.client.1.smithi148.stdout:2/464: sync 2024-03-29T08:07:18.221 INFO:tasks.workunit.client.1.smithi148.stdout:1/445: sync 2024-03-29T08:07:18.221 INFO:tasks.workunit.client.1.smithi148.stdout:6/423: sync 2024-03-29T08:07:18.221 INFO:tasks.workunit.client.1.smithi148.stdout:5/463: sync 2024-03-29T08:07:18.221 INFO:tasks.workunit.client.1.smithi148.stdout:4/454: mknod df/d60/c99 0 2024-03-29T08:07:18.223 INFO:tasks.workunit.client.1.smithi148.stdout:0/459: rename db/de/df/d11/d42/f48 to db/de/df/f95 0 2024-03-29T08:07:18.225 INFO:tasks.workunit.client.1.smithi148.stdout:1/446: write d3/d18/d4d/f50 [874603,122966] 0 2024-03-29T08:07:18.225 INFO:tasks.workunit.client.1.smithi148.stdout:2/465: write d7/da/d4c/f69 [911921,44115] 0 2024-03-29T08:07:18.230 INFO:tasks.workunit.client.0.smithi037.stdout:0/210: unlink d9/da/d1b/c2a 0 2024-03-29T08:07:18.231 INFO:tasks.workunit.client.1.smithi148.stdout:4/455: unlink df/d2c/d51/d77/d26/d34/d43/f82 0 2024-03-29T08:07:18.233 INFO:tasks.workunit.client.1.smithi148.stdout:8/508: symlink d17/d74/d39/la6 0 2024-03-29T08:07:18.236 INFO:tasks.workunit.client.0.smithi037.stdout:9/536: fsync d1/d3/d44/f57 0 2024-03-29T08:07:18.237 INFO:tasks.workunit.client.0.smithi037.stdout:2/237: mknod d0/d20/d2e/c52 0 2024-03-29T08:07:18.238 INFO:tasks.workunit.client.1.smithi148.stdout:8/509: truncate d17/d1d/d7e/f85 188432 0 2024-03-29T08:07:18.238 INFO:tasks.workunit.client.1.smithi148.stdout:1/447: creat d3/de/d6e/f88 x:0 0 0 2024-03-29T08:07:18.241 INFO:tasks.workunit.client.1.smithi148.stdout:6/424: dwrite d6/f70 [0,4194304] 0 2024-03-29T08:07:18.241 INFO:tasks.workunit.client.1.smithi148.stdout:1/448: write d3/d55/d7e/f67 [935700,71338] 0 2024-03-29T08:07:18.242 INFO:tasks.workunit.client.1.smithi148.stdout:3/492: rename de/d24/d33/d39/d46/d70/l89 to de/d24/d33/d39/d46/la4 0 2024-03-29T08:07:18.245 INFO:tasks.workunit.client.1.smithi148.stdout:7/444: write d8/d14/d45/f66 [3066188,86448] 0 2024-03-29T08:07:18.245 INFO:tasks.workunit.client.1.smithi148.stdout:5/464: mkdir d0/db/d1e/d35/d42/d67/d76/d81/d98 0 2024-03-29T08:07:18.247 INFO:tasks.workunit.client.0.smithi037.stdout:7/364: getdents dd/d21/d35/d60 0 2024-03-29T08:07:18.249 INFO:tasks.workunit.client.0.smithi037.stdout:8/400: dwrite d5/d25/d46/d4e/f57 [0,4194304] 0 2024-03-29T08:07:18.249 INFO:tasks.workunit.client.0.smithi037.stdout:7/365: readlink dd/l58 0 2024-03-29T08:07:18.251 INFO:tasks.workunit.client.0.smithi037.stdout:1/392: dwrite d2/d6/dc/d18/d1d/d21/f37 [0,4194304] 0 2024-03-29T08:07:18.253 INFO:tasks.workunit.client.1.smithi148.stdout:7/445: rename d8/d14/d19/d37/l61 to d8/d29/d44/d50/d94/l9b 0 2024-03-29T08:07:18.254 INFO:tasks.workunit.client.1.smithi148.stdout:7/446: stat d8/d38/d3c/f5a 0 2024-03-29T08:07:18.257 INFO:tasks.workunit.client.0.smithi037.stdout:3/352: getdents dd/d46/d2d 0 2024-03-29T08:07:18.259 INFO:tasks.workunit.client.0.smithi037.stdout:0/211: rmdir d9/da/dd/d20 39 2024-03-29T08:07:18.260 INFO:tasks.workunit.client.1.smithi148.stdout:6/425: fdatasync d6/d9/d1e/d27/f7f 0 2024-03-29T08:07:18.263 INFO:tasks.workunit.client.1.smithi148.stdout:6/426: truncate d6/d9/d1e/d36/d4d/f7a 783881 0 2024-03-29T08:07:18.264 INFO:tasks.workunit.client.0.smithi037.stdout:0/212: dread - d9/da/f3c zero size 2024-03-29T08:07:18.264 INFO:tasks.workunit.client.0.smithi037.stdout:4/355: dwrite d3/d4/d15/d1d/db/f1a [0,4194304] 0 2024-03-29T08:07:18.265 INFO:tasks.workunit.client.0.smithi037.stdout:2/238: mkdir d0/d53 0 2024-03-29T08:07:18.265 INFO:tasks.workunit.client.1.smithi148.stdout:3/493: mkdir de/d3b/d49/da5 0 2024-03-29T08:07:18.266 INFO:tasks.workunit.client.1.smithi148.stdout:1/449: mknod d3/de/d25/d35/d42/c89 0 2024-03-29T08:07:18.267 INFO:tasks.workunit.client.0.smithi037.stdout:4/356: stat d3/d4/f34 0 2024-03-29T08:07:18.275 INFO:tasks.workunit.client.0.smithi037.stdout:6/311: dwrite d0/d5/f9 [0,4194304] 0 2024-03-29T08:07:18.278 INFO:tasks.workunit.client.0.smithi037.stdout:4/357: write d3/d4/f3e [206326,78459] 0 2024-03-29T08:07:18.279 INFO:tasks.workunit.client.0.smithi037.stdout:9/537: truncate d1/d4/d19/d1e/d4a/f53 384036 0 2024-03-29T08:07:18.280 INFO:tasks.workunit.client.1.smithi148.stdout:9/465: dwrite db/d20/f70 [0,4194304] 0 2024-03-29T08:07:18.281 INFO:tasks.workunit.client.0.smithi037.stdout:8/401: mknod d5/d10/d19/d3d/d4a/d4c/c81 0 2024-03-29T08:07:18.281 INFO:tasks.workunit.client.1.smithi148.stdout:4/456: creat df/f9a x:0 0 0 2024-03-29T08:07:18.282 INFO:tasks.workunit.client.0.smithi037.stdout:2/239: unlink d0/f8 0 2024-03-29T08:07:18.286 INFO:tasks.workunit.client.0.smithi037.stdout:3/353: creat dd/d46/d2d/d3a/d5f/f77 x:0 0 0 2024-03-29T08:07:18.286 INFO:tasks.workunit.client.0.smithi037.stdout:4/358: write d3/d4/f3e [637075,125354] 0 2024-03-29T08:07:18.286 INFO:tasks.workunit.client.1.smithi148.stdout:3/494: dwrite de/d24/d33/d39/d46/d30/d37/f7a [4194304,4194304] 0 2024-03-29T08:07:18.289 INFO:tasks.workunit.client.1.smithi148.stdout:4/457: write df/d2c/f86 [1008766,97837] 0 2024-03-29T08:07:18.291 INFO:tasks.workunit.client.1.smithi148.stdout:3/495: chown de/d24/d33/d39/c50 29695507 1 2024-03-29T08:07:18.294 INFO:tasks.workunit.client.1.smithi148.stdout:7/447: dread d8/d29/d44/d49/f76 [0,4194304] 0 2024-03-29T08:07:18.295 INFO:tasks.workunit.client.1.smithi148.stdout:6/427: symlink d6/l82 0 2024-03-29T08:07:18.296 INFO:tasks.workunit.client.1.smithi148.stdout:4/458: chown df/d2d/f35 0 1 2024-03-29T08:07:18.297 INFO:tasks.workunit.client.0.smithi037.stdout:7/366: dwrite dd/f3e [0,4194304] 0 2024-03-29T08:07:18.299 INFO:tasks.workunit.client.1.smithi148.stdout:6/428: write d6/f14 [1701952,5925] 0 2024-03-29T08:07:18.301 INFO:tasks.workunit.client.0.smithi037.stdout:3/354: write dd/d46/f3c [1220960,21986] 0 2024-03-29T08:07:18.302 INFO:tasks.workunit.client.1.smithi148.stdout:3/496: creat de/d24/d33/d6b/fa6 x:0 0 0 2024-03-29T08:07:18.304 INFO:tasks.workunit.client.0.smithi037.stdout:0/213: creat d9/da/d1b/f41 x:0 0 0 2024-03-29T08:07:18.304 INFO:tasks.workunit.client.0.smithi037.stdout:2/240: getdents d0/d53 0 2024-03-29T08:07:18.307 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:07:18 smithi148 ceph-mon[44992]: Reconfiguring node-exporter.smithi148 (dependencies changed)... 2024-03-29T08:07:18.307 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:07:18 smithi148 ceph-mon[44992]: Reconfiguring daemon node-exporter.smithi148 on smithi148 2024-03-29T08:07:18.307 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:07:18 smithi148 ceph-mon[44992]: fsmap cephfs:2 {0=cephfs.smithi037.mmwgxe=up:active,1=cephfs.smithi037.iifsmh=up:active} 2 up:standby 2024-03-29T08:07:18.307 INFO:tasks.workunit.client.1.smithi148.stdout:5/465: getdents d0/db/d7b 0 2024-03-29T08:07:18.308 INFO:tasks.workunit.client.0.smithi037.stdout:7/367: write dd/d21/f6d [2452451,83299] 0 2024-03-29T08:07:18.308 INFO:tasks.workunit.client.0.smithi037.stdout:8/402: truncate d5/d10/d2d/d34/d50/f68 1610034 0 2024-03-29T08:07:18.309 INFO:tasks.workunit.client.0.smithi037.stdout:2/241: chown d0/d20/f3d 92 1 2024-03-29T08:07:18.309 INFO:tasks.workunit.client.1.smithi148.stdout:1/450: creat d3/de/d1d/d47/d6d/d72/f8a x:0 0 0 2024-03-29T08:07:18.312 INFO:tasks.workunit.client.1.smithi148.stdout:4/459: mkdir df/d2c/d51/d77/d26/d9b 0 2024-03-29T08:07:18.312 INFO:tasks.workunit.client.1.smithi148.stdout:9/466: dwrite db/d22/d26/f2a [4194304,4194304] 0 2024-03-29T08:07:18.312 INFO:tasks.workunit.client.0.smithi037.stdout:4/359: creat d3/d57/d6a/f77 x:0 0 0 2024-03-29T08:07:18.314 INFO:tasks.workunit.client.1.smithi148.stdout:3/497: chown de/d24/d33/d39/d46/d30/l47 787834 1 2024-03-29T08:07:18.315 INFO:tasks.workunit.client.0.smithi037.stdout:9/538: rename d1/d4/d19/d1e/f3f to d1/d4/d19/fb9 0 2024-03-29T08:07:18.315 INFO:tasks.workunit.client.1.smithi148.stdout:4/460: fsync df/d2d/f50 0 2024-03-29T08:07:18.315 INFO:tasks.workunit.client.0.smithi037.stdout:8/403: write d5/d25/f5f [281546,129268] 0 2024-03-29T08:07:18.316 INFO:tasks.workunit.client.1.smithi148.stdout:2/466: dwrite d7/f19 [0,4194304] 0 2024-03-29T08:07:18.317 INFO:tasks.workunit.client.1.smithi148.stdout:7/448: creat d8/d14/d19/d37/f9c x:0 0 0 2024-03-29T08:07:18.319 INFO:tasks.workunit.client.0.smithi037.stdout:0/214: fdatasync d9/da/d1b/d22/d2c/f2d 0 2024-03-29T08:07:18.320 INFO:tasks.workunit.client.1.smithi148.stdout:9/467: dread - db/d22/d26/d54/d4d/d35/d4b/d8e/f97 zero size 2024-03-29T08:07:18.320 INFO:tasks.workunit.client.0.smithi037.stdout:8/404: fdatasync d5/d25/f28 0 2024-03-29T08:07:18.322 INFO:tasks.workunit.client.1.smithi148.stdout:9/468: chown db/d22/d3f 899407 1 2024-03-29T08:07:18.323 INFO:tasks.workunit.client.1.smithi148.stdout:4/461: write df/d2c/d51/d77/d26/d34/d8a/f95 [791255,48208] 0 2024-03-29T08:07:18.324 INFO:tasks.workunit.client.1.smithi148.stdout:4/462: fsync fd 0 2024-03-29T08:07:18.326 INFO:tasks.workunit.client.0.smithi037.stdout:0/215: write f5 [9032356,64124] 0 2024-03-29T08:07:18.327 INFO:tasks.workunit.client.0.smithi037.stdout:7/368: creat dd/f77 x:0 0 0 2024-03-29T08:07:18.334 INFO:tasks.workunit.client.0.smithi037.stdout:2/242: link d0/c5 d0/c54 0 2024-03-29T08:07:18.334 INFO:tasks.workunit.client.0.smithi037.stdout:0/216: creat d9/da/d1b/d22/d2c/f42 x:0 0 0 2024-03-29T08:07:18.335 INFO:tasks.workunit.client.1.smithi148.stdout:6/429: rename d6/d9/d1e/d27/f44 to d6/d9/d1e/d27/d48/d73/f83 0 2024-03-29T08:07:18.335 INFO:tasks.workunit.client.1.smithi148.stdout:3/498: unlink de/d24/d33/d39/d46/d30/f86 0 2024-03-29T08:07:18.341 INFO:tasks.workunit.client.1.smithi148.stdout:1/451: truncate d3/d18/d4d/f63 786265 0 2024-03-29T08:07:18.341 INFO:tasks.workunit.client.0.smithi037.stdout:2/243: write d0/f2 [3182506,46505] 0 2024-03-29T08:07:18.342 INFO:tasks.workunit.client.1.smithi148.stdout:7/449: dwrite d8/d14/f1c [0,4194304] 0 2024-03-29T08:07:18.343 INFO:tasks.workunit.client.1.smithi148.stdout:7/450: readlink d8/d14/d45/l64 0 2024-03-29T08:07:18.344 INFO:tasks.workunit.client.1.smithi148.stdout:7/451: fdatasync d8/d38/f8d 0 2024-03-29T08:07:18.345 INFO:tasks.workunit.client.1.smithi148.stdout:7/452: stat d8/d14/d45/d4d 0 2024-03-29T08:07:18.346 INFO:tasks.workunit.client.0.smithi037.stdout:2/244: chown d0/l42 38667 1 2024-03-29T08:07:18.347 INFO:tasks.workunit.client.1.smithi148.stdout:7/453: write d8/d29/d44/d50/f52 [1463068,34345] 0 2024-03-29T08:07:18.347 INFO:tasks.workunit.client.0.smithi037.stdout:1/393: write d2/d6/dc/d18/d1d/d21/f47 [1010499,91629] 0 2024-03-29T08:07:18.348 INFO:tasks.workunit.client.1.smithi148.stdout:4/463: mkdir df/d2c/d51/d77/d26/d9c 0 2024-03-29T08:07:18.354 INFO:tasks.workunit.client.1.smithi148.stdout:3/499: stat de/d3b/f60 0 2024-03-29T08:07:18.354 INFO:tasks.workunit.client.0.smithi037.stdout:4/360: dwrite d3/d4/d15/d1d/db/d45/d1e/f4e [0,4194304] 0 2024-03-29T08:07:18.367 INFO:tasks.workunit.client.1.smithi148.stdout:2/467: creat d7/da/d33/f92 x:0 0 0 2024-03-29T08:07:18.374 INFO:tasks.workunit.client.1.smithi148.stdout:7/454: dwrite d8/d14/f16 [0,4194304] 0 2024-03-29T08:07:18.375 INFO:tasks.workunit.client.1.smithi148.stdout:3/500: dwrite de/d3b/d49/f90 [0,4194304] 0 2024-03-29T08:07:18.380 INFO:tasks.workunit.client.1.smithi148.stdout:4/464: unlink df/d2c/d51/d77/c21 0 2024-03-29T08:07:18.380 INFO:tasks.workunit.client.1.smithi148.stdout:9/469: creat db/d22/d3f/d61/d94/f9e x:0 0 0 2024-03-29T08:07:18.380 INFO:tasks.workunit.client.0.smithi037.stdout:8/405: dwrite d5/d10/d19/d3d/d4a/d4c/f73 [0,4194304] 0 2024-03-29T08:07:18.387 INFO:tasks.workunit.client.1.smithi148.stdout:8/510: stat fe 0 2024-03-29T08:07:18.389 INFO:tasks.workunit.client.0.smithi037.stdout:0/217: dwrite d9/fb [0,4194304] 0 2024-03-29T08:07:18.396 INFO:tasks.workunit.client.1.smithi148.stdout:2/468: unlink d7/da/d33/d3c/c78 0 2024-03-29T08:07:18.396 INFO:tasks.workunit.client.0.smithi037.stdout:9/539: dread d1/d1f/f5f [0,4194304] 0 2024-03-29T08:07:18.398 INFO:tasks.workunit.client.1.smithi148.stdout:8/511: unlink d17/d49/l95 0 2024-03-29T08:07:18.404 INFO:tasks.workunit.client.0.smithi037.stdout:0/218: chown d9/da/dd/d13/f21 4824040 1 2024-03-29T08:07:18.405 INFO:tasks.workunit.client.0.smithi037.stdout:4/361: dwrite d3/d4/f34 [4194304,4194304] 0 2024-03-29T08:07:18.406 INFO:tasks.workunit.client.0.smithi037.stdout:1/394: creat d2/f84 x:0 0 0 2024-03-29T08:07:18.407 INFO:tasks.workunit.client.1.smithi148.stdout:4/465: mknod df/d2c/d51/d77/d26/d34/d3e/c9d 0 2024-03-29T08:07:18.408 INFO:tasks.workunit.client.0.smithi037.stdout:2/245: dwrite d0/d23/f26 [0,4194304] 0 2024-03-29T08:07:18.412 INFO:tasks.workunit.client.1.smithi148.stdout:2/469: unlink d7/da/d33/d3c/f48 0 2024-03-29T08:07:18.415 INFO:tasks.workunit.client.0.smithi037.stdout:9/540: chown d1/d4/d19/d1e/d4a/d6f/da2/d31/f38 6817765 1 2024-03-29T08:07:18.416 INFO:tasks.workunit.client.0.smithi037.stdout:3/355: write dd/d15/f1e [969433,50070] 0 2024-03-29T08:07:18.419 INFO:tasks.workunit.client.1.smithi148.stdout:9/470: mknod db/d10/d6d/d73/c9f 0 2024-03-29T08:07:18.420 INFO:tasks.workunit.client.1.smithi148.stdout:9/471: stat db/d22/d26/d54/d52/l56 0 2024-03-29T08:07:18.423 INFO:tasks.workunit.client.1.smithi148.stdout:4/466: chown df/d2c/d51/d77/d26/l2b 21 1 2024-03-29T08:07:18.426 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:07:18 smithi037 ceph-mon[35162]: Reconfiguring node-exporter.smithi148 (dependencies changed)... 2024-03-29T08:07:18.426 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:07:18 smithi037 ceph-mon[35162]: Reconfiguring daemon node-exporter.smithi148 on smithi148 2024-03-29T08:07:18.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:07:18 smithi037 ceph-mon[35162]: fsmap cephfs:2 {0=cephfs.smithi037.mmwgxe=up:active,1=cephfs.smithi037.iifsmh=up:active} 2 up:standby 2024-03-29T08:07:18.431 INFO:tasks.workunit.client.0.smithi037.stdout:4/362: dread d3/d4/d15/d1d/db/d45/d1e/f4e [0,4194304] 0 2024-03-29T08:07:18.431 INFO:tasks.workunit.client.1.smithi148.stdout:6/430: dread d6/d9/d1e/f26 [0,4194304] 0 2024-03-29T08:07:18.433 INFO:tasks.workunit.client.1.smithi148.stdout:6/431: chown d6/d9/d1e 3 1 2024-03-29T08:07:18.447 INFO:tasks.workunit.client.0.smithi037.stdout:3/356: symlink dd/d46/l78 0 2024-03-29T08:07:18.448 INFO:tasks.workunit.client.1.smithi148.stdout:9/472: symlink db/d22/d26/d54/d4d/d35/d4b/d8e/la0 0 2024-03-29T08:07:18.450 INFO:tasks.workunit.client.0.smithi037.stdout:8/406: read d5/d10/d19/f2f [1586971,130207] 0 2024-03-29T08:07:18.458 INFO:tasks.workunit.client.0.smithi037.stdout:0/219: rename d9/da/le to d9/da/dd/d1c/l43 0 2024-03-29T08:07:18.465 INFO:tasks.workunit.client.1.smithi148.stdout:6/432: symlink d6/d9/l84 0 2024-03-29T08:07:18.466 INFO:tasks.workunit.client.0.smithi037.stdout:9/541: mkdir d1/d4/d19/d1e/d4a/d6f/da2/d31/daa/dba 0 2024-03-29T08:07:18.468 INFO:tasks.workunit.client.1.smithi148.stdout:3/501: link de/d24/d33/d39/d46/c1f de/d24/d33/d39/d46/ca7 0 2024-03-29T08:07:18.469 INFO:tasks.workunit.client.0.smithi037.stdout:4/363: mknod d3/d4/d15/d1d/d4f/c78 0 2024-03-29T08:07:18.470 INFO:tasks.workunit.client.1.smithi148.stdout:9/473: mknod db/d10/ca1 0 2024-03-29T08:07:18.472 INFO:tasks.workunit.client.0.smithi037.stdout:9/542: truncate d1/d4/d95/fad 1001747 0 2024-03-29T08:07:18.472 INFO:tasks.workunit.client.0.smithi037.stdout:8/407: rename d5/l29 to d5/d10/d2d/d34/d50/l82 0 2024-03-29T08:07:18.473 INFO:tasks.workunit.client.1.smithi148.stdout:6/433: fdatasync d6/f41 0 2024-03-29T08:07:18.473 INFO:tasks.workunit.client.1.smithi148.stdout:2/470: rmdir d7/da/d33 39 2024-03-29T08:07:18.473 INFO:tasks.workunit.client.1.smithi148.stdout:3/502: chown de/d24/d33/d39/c4b 3167885 1 2024-03-29T08:07:18.475 INFO:tasks.workunit.client.1.smithi148.stdout:6/434: stat d6/d9/d1e/d27/d48 0 2024-03-29T08:07:18.476 INFO:tasks.workunit.client.0.smithi037.stdout:9/543: readlink d1/d4/d19/d1e/d4a/d6f/da2/d13/l85 0 2024-03-29T08:07:18.477 INFO:tasks.workunit.client.1.smithi148.stdout:7/455: write d8/d29/d44/f93 [198333,64958] 0 2024-03-29T08:07:18.477 INFO:tasks.workunit.client.0.smithi037.stdout:3/357: symlink dd/d46/d2d/d3a/l79 0 2024-03-29T08:07:18.478 INFO:tasks.workunit.client.1.smithi148.stdout:6/435: write d6/fe [1926569,65667] 0 2024-03-29T08:07:18.479 INFO:tasks.workunit.client.1.smithi148.stdout:0/460: sync 2024-03-29T08:07:18.480 INFO:tasks.workunit.client.0.smithi037.stdout:4/364: mkdir d3/d4/d15/d1d/db/d45/d1e/d59/d79 0 2024-03-29T08:07:18.481 INFO:tasks.workunit.client.1.smithi148.stdout:5/466: sync 2024-03-29T08:07:18.483 INFO:tasks.workunit.client.1.smithi148.stdout:8/512: write d17/d18/f4d [684912,124453] 0 2024-03-29T08:07:18.483 INFO:tasks.workunit.client.0.smithi037.stdout:0/220: rename d9/da/dd/l40 to d9/da/dd/d1c/l44 0 2024-03-29T08:07:18.485 INFO:tasks.workunit.client.1.smithi148.stdout:6/436: write d6/d9/d1e/f5d [82599,104338] 0 2024-03-29T08:07:18.486 INFO:tasks.workunit.client.1.smithi148.stdout:2/471: rmdir d7/da/d33/d45/d4d/d2e 39 2024-03-29T08:07:18.489 INFO:tasks.workunit.client.1.smithi148.stdout:1/452: dread d3/de/d25/d35/d42/f1a [4194304,4194304] 0 2024-03-29T08:07:18.490 INFO:tasks.workunit.client.1.smithi148.stdout:7/456: creat d8/d14/d45/d4d/f9d x:0 0 0 2024-03-29T08:07:18.491 INFO:tasks.workunit.client.1.smithi148.stdout:5/467: creat d0/db/d1e/d5b/d66/d72/f99 x:0 0 0 2024-03-29T08:07:18.495 INFO:tasks.workunit.client.1.smithi148.stdout:0/461: creat db/de/df/d2b/d3b/d47/f96 x:0 0 0 2024-03-29T08:07:18.495 INFO:tasks.workunit.client.1.smithi148.stdout:1/453: truncate d3/f30 4335857 0 2024-03-29T08:07:18.496 INFO:tasks.workunit.client.1.smithi148.stdout:2/472: creat d7/da/d33/d3c/d54/f93 x:0 0 0 2024-03-29T08:07:18.496 INFO:tasks.workunit.client.1.smithi148.stdout:4/467: dwrite df/d2c/d51/d77/f40 [0,4194304] 0 2024-03-29T08:07:18.498 INFO:tasks.workunit.client.1.smithi148.stdout:9/474: dwrite db/f5c [0,4194304] 0 2024-03-29T08:07:18.500 INFO:tasks.workunit.client.1.smithi148.stdout:1/454: write d3/fa [3521525,117454] 0 2024-03-29T08:07:18.502 INFO:tasks.workunit.client.0.smithi037.stdout:4/365: creat d3/d57/f7a x:0 0 0 2024-03-29T08:07:18.502 INFO:tasks.workunit.client.1.smithi148.stdout:5/468: unlink d0/db/d1e/d36/f5a 0 2024-03-29T08:07:18.503 INFO:tasks.workunit.client.1.smithi148.stdout:6/437: dread d6/d9/d1e/d22/f31 [0,4194304] 0 2024-03-29T08:07:18.504 INFO:tasks.workunit.client.1.smithi148.stdout:8/513: unlink d17/d74/f6e 0 2024-03-29T08:07:18.505 INFO:tasks.workunit.client.0.smithi037.stdout:8/408: rename d5/d25 to d5/d10/d2d/d34/d50/d83 0 2024-03-29T08:07:18.506 INFO:tasks.workunit.client.1.smithi148.stdout:4/468: unlink df/d2c/d51/d77/d26/l44 0 2024-03-29T08:07:18.507 INFO:tasks.workunit.client.1.smithi148.stdout:6/438: chown d6/d9/d1e/d27/d48/f7b 140292835 1 2024-03-29T08:07:18.508 INFO:tasks.workunit.client.0.smithi037.stdout:1/395: link d2/d6/dc/d5d/d62/l83 d2/d6/l85 0 2024-03-29T08:07:18.508 INFO:tasks.workunit.client.1.smithi148.stdout:5/469: chown d0/db/d1e/d36/d41/c8d 61627692 1 2024-03-29T08:07:18.510 INFO:tasks.workunit.client.1.smithi148.stdout:9/475: getdents db/d22/d26/d54/d4d/d35/d4b/d5a/d74/d86 0 2024-03-29T08:07:18.512 INFO:tasks.workunit.client.0.smithi037.stdout:2/246: dread - d0/d23/d44/f46 zero size 2024-03-29T08:07:18.512 INFO:tasks.workunit.client.1.smithi148.stdout:6/439: dread - d6/d9/d1e/d22/d28/f74 zero size 2024-03-29T08:07:18.514 INFO:tasks.workunit.client.1.smithi148.stdout:4/469: creat df/d2c/d51/d77/d26/d9b/f9e x:0 0 0 2024-03-29T08:07:18.520 INFO:tasks.workunit.client.1.smithi148.stdout:8/514: mknod d17/d74/d39/d6f/ca7 0 2024-03-29T08:07:18.522 INFO:tasks.workunit.client.1.smithi148.stdout:7/457: mknod d8/c9e 0 2024-03-29T08:07:18.523 INFO:tasks.workunit.client.1.smithi148.stdout:5/470: rename d0/db/d1e/d35/d6a to d0/db/d1e/d35/d42/d67/d9a 0 2024-03-29T08:07:18.526 INFO:tasks.workunit.client.1.smithi148.stdout:4/470: mknod df/d2c/d51/d77/d26/d34/d8a/c9f 0 2024-03-29T08:07:18.528 INFO:tasks.workunit.client.0.smithi037.stdout:0/221: truncate d9/da/f24 2962542 0 2024-03-29T08:07:18.529 INFO:tasks.workunit.client.1.smithi148.stdout:1/455: dwrite d3/de/d25/f75 [0,4194304] 0 2024-03-29T08:07:18.529 INFO:tasks.workunit.client.1.smithi148.stdout:9/476: rename db/d22/d26/d54/d52/l56 to db/d22/d26/d54/d4d/d35/d4b/d5a/d74/la2 0 2024-03-29T08:07:18.530 INFO:tasks.workunit.client.0.smithi037.stdout:8/409: symlink d5/d10/d2d/d34/d50/l84 0 2024-03-29T08:07:18.532 INFO:tasks.workunit.client.1.smithi148.stdout:4/471: chown df/d2d/f49 37391486 1 2024-03-29T08:07:18.532 INFO:tasks.workunit.client.0.smithi037.stdout:1/396: mkdir d2/d6/dc/d18/d22/d4f/d86 0 2024-03-29T08:07:18.533 INFO:tasks.workunit.client.1.smithi148.stdout:3/503: dread de/d24/f56 [0,4194304] 0 2024-03-29T08:07:18.536 INFO:tasks.workunit.client.0.smithi037.stdout:1/397: chown d2/d6/dc/d18/d1d/d21/c3d 0 1 2024-03-29T08:07:18.536 INFO:tasks.workunit.client.1.smithi148.stdout:2/473: dread d7/da/d33/d45/d4d/d2e/f50 [0,4194304] 0 2024-03-29T08:07:18.536 INFO:tasks.workunit.client.1.smithi148.stdout:1/456: write d3/de/d25/d35/d42/d23/f40 [2920030,103670] 0 2024-03-29T08:07:18.537 INFO:tasks.workunit.client.1.smithi148.stdout:7/458: rmdir d8/d38/d3c 39 2024-03-29T08:07:18.537 INFO:tasks.workunit.client.1.smithi148.stdout:9/477: mkdir db/d22/da3 0 2024-03-29T08:07:18.540 INFO:tasks.workunit.client.0.smithi037.stdout:0/222: creat d9/da/dd/d13/f45 x:0 0 0 2024-03-29T08:07:18.541 INFO:tasks.workunit.client.1.smithi148.stdout:0/462: write db/de/df/d14/f3c [903064,80924] 0 2024-03-29T08:07:18.543 INFO:tasks.workunit.client.1.smithi148.stdout:4/472: unlink df/d2c/d51/c96 0 2024-03-29T08:07:18.543 INFO:tasks.workunit.client.0.smithi037.stdout:0/223: dread - d9/da/f3c zero size 2024-03-29T08:07:18.543 INFO:tasks.workunit.client.1.smithi148.stdout:1/457: mknod d3/d55/c8b 0 2024-03-29T08:07:18.545 INFO:tasks.workunit.client.1.smithi148.stdout:1/458: fsync d3/f5d 0 2024-03-29T08:07:18.545 INFO:tasks.workunit.client.0.smithi037.stdout:3/358: truncate f5 3191002 0 2024-03-29T08:07:18.546 INFO:tasks.workunit.client.0.smithi037.stdout:9/544: getdents d1/d4/d19/d1e/d4a/d6f/da2/d68 0 2024-03-29T08:07:18.548 INFO:tasks.workunit.client.0.smithi037.stdout:3/359: chown dd/d46/d2d/f60 21 1 2024-03-29T08:07:18.548 INFO:tasks.workunit.client.0.smithi037.stdout:4/366: rename d3/d4/f51 to d3/d4/f7b 0 2024-03-29T08:07:18.549 INFO:tasks.workunit.client.1.smithi148.stdout:4/473: fsync df/d2c/d51/d77/d26/d34/d43/f59 0 2024-03-29T08:07:18.549 INFO:tasks.workunit.client.1.smithi148.stdout:2/474: getdents d7/d24/d4b/d83 0 2024-03-29T08:07:18.549 INFO:tasks.workunit.client.0.smithi037.stdout:2/247: mkdir d0/d20/d22/d55 0 2024-03-29T08:07:18.551 INFO:tasks.workunit.client.0.smithi037.stdout:4/367: readlink d3/d4/d15/l6e 0 2024-03-29T08:07:18.551 INFO:tasks.workunit.client.0.smithi037.stdout:1/398: mkdir d2/d6/dc/d5d/d64/d87 0 2024-03-29T08:07:18.551 INFO:tasks.workunit.client.1.smithi148.stdout:1/459: dread - d3/de/d1d/d47/d6d/d72/f8a zero size 2024-03-29T08:07:18.554 INFO:tasks.workunit.client.1.smithi148.stdout:6/440: write d6/fd [951483,81572] 0 2024-03-29T08:07:18.554 INFO:tasks.workunit.client.1.smithi148.stdout:5/471: dwrite d0/db/d1e/d35/d42/d67/f70 [0,4194304] 0 2024-03-29T08:07:18.554 INFO:tasks.workunit.client.1.smithi148.stdout:7/459: mknod d8/d14/d45/d4d/d78/c9f 0 2024-03-29T08:07:18.557 INFO:tasks.workunit.client.0.smithi037.stdout:4/368: truncate d3/d4/d15/f50 1046309 0 2024-03-29T08:07:18.558 INFO:tasks.workunit.client.0.smithi037.stdout:3/360: rename dd/d46/d2d/d3b to dd/d5e/d7a 0 2024-03-29T08:07:18.559 INFO:tasks.workunit.client.1.smithi148.stdout:6/441: fdatasync d6/d9/f75 0 2024-03-29T08:07:18.561 INFO:tasks.workunit.client.1.smithi148.stdout:4/474: dread df/d2c/d51/d77/f53 [0,4194304] 0 2024-03-29T08:07:18.561 INFO:tasks.workunit.client.1.smithi148.stdout:2/475: creat d7/d86/d8e/f94 x:0 0 0 2024-03-29T08:07:18.562 INFO:tasks.workunit.client.0.smithi037.stdout:4/369: symlink d3/d54/l7c 0 2024-03-29T08:07:18.562 INFO:tasks.workunit.client.1.smithi148.stdout:6/442: readlink d6/l43 0 2024-03-29T08:07:18.563 INFO:tasks.workunit.client.1.smithi148.stdout:3/504: getdents de/d24/d33/d39/d46/d30/d37/d62/d63 0 2024-03-29T08:07:18.564 INFO:tasks.workunit.client.0.smithi037.stdout:0/224: dread d9/da/d1b/f29 [0,4194304] 0 2024-03-29T08:07:18.564 INFO:tasks.workunit.client.1.smithi148.stdout:5/472: symlink d0/db/d1e/d35/d42/l9b 0 2024-03-29T08:07:18.564 INFO:tasks.workunit.client.1.smithi148.stdout:0/463: rename db/de/df/f90 to db/f97 0 2024-03-29T08:07:18.565 INFO:tasks.workunit.client.0.smithi037.stdout:8/410: getdents d5/d10/d45/d4b 0 2024-03-29T08:07:18.565 INFO:tasks.workunit.client.1.smithi148.stdout:9/478: dwrite db/d22/d26/f8a [0,4194304] 0 2024-03-29T08:07:18.566 INFO:tasks.workunit.client.0.smithi037.stdout:9/545: mkdir d1/d4/d19/d1e/d4a/d6f/da2/d31/daa/dba/dbb 0 2024-03-29T08:07:18.567 INFO:tasks.workunit.client.1.smithi148.stdout:1/460: dread d3/de/d25/d51/f73 [0,4194304] 0 2024-03-29T08:07:18.569 INFO:tasks.workunit.client.0.smithi037.stdout:1/399: symlink d2/d6/dc/d7e/l88 0 2024-03-29T08:07:18.570 INFO:tasks.workunit.client.0.smithi037.stdout:2/248: rmdir d0/d20/d22/d4e/d51 0 2024-03-29T08:07:18.572 INFO:tasks.workunit.client.1.smithi148.stdout:2/476: unlink d7/da/d33/f92 0 2024-03-29T08:07:18.573 INFO:tasks.workunit.client.0.smithi037.stdout:8/411: truncate d5/f3a 507244 0 2024-03-29T08:07:18.574 INFO:tasks.workunit.client.0.smithi037.stdout:2/249: dread - d0/d20/f3d zero size 2024-03-29T08:07:18.574 INFO:tasks.workunit.client.1.smithi148.stdout:2/477: truncate d7/da/fe 9024711 0 2024-03-29T08:07:18.575 INFO:tasks.workunit.client.1.smithi148.stdout:0/464: unlink db/de/df/d14/c27 0 2024-03-29T08:07:18.576 INFO:tasks.workunit.client.0.smithi037.stdout:9/546: symlink d1/d4/d95/lbc 0 2024-03-29T08:07:18.579 INFO:tasks.workunit.client.0.smithi037.stdout:1/400: mknod d2/d6/dc/d18/d1d/d21/d38/c89 0 2024-03-29T08:07:18.580 INFO:tasks.workunit.client.0.smithi037.stdout:1/401: readlink d2/d6/dc/d7e/l88 0 2024-03-29T08:07:18.583 INFO:tasks.workunit.client.0.smithi037.stdout:2/250: mknod d0/d20/d2e/c56 0 2024-03-29T08:07:18.584 INFO:tasks.workunit.client.0.smithi037.stdout:8/412: mkdir d5/d10/d19/d3d/d4a/d4c/d67/d85 0 2024-03-29T08:07:18.586 INFO:tasks.workunit.client.0.smithi037.stdout:0/225: truncate d9/da/dd/d1c/f18 2898102 0 2024-03-29T08:07:18.586 INFO:tasks.workunit.client.0.smithi037.stdout:4/370: getdents d3/d4/d15/d1d 0 2024-03-29T08:07:18.588 INFO:tasks.workunit.client.0.smithi037.stdout:1/402: symlink d2/d6/dc/d5d/d64/d87/l8a 0 2024-03-29T08:07:18.590 INFO:tasks.workunit.client.0.smithi037.stdout:8/413: write d5/f5b [862897,12723] 0 2024-03-29T08:07:18.591 INFO:tasks.workunit.client.1.smithi148.stdout:1/461: fdatasync d3/de/d25/d51/f73 0 2024-03-29T08:07:18.592 INFO:tasks.workunit.client.0.smithi037.stdout:4/371: chown d3/d4/d15/l4d 42 1 2024-03-29T08:07:18.592 INFO:tasks.workunit.client.1.smithi148.stdout:4/475: rmdir df/d92 0 2024-03-29T08:07:18.593 INFO:tasks.workunit.client.1.smithi148.stdout:3/505: mknod de/d24/d33/d39/d7f/ca8 0 2024-03-29T08:07:18.596 INFO:tasks.workunit.client.1.smithi148.stdout:0/465: rmdir db/dc/d78 39 2024-03-29T08:07:18.596 INFO:tasks.workunit.client.0.smithi037.stdout:0/226: rename d9/da/dd/d13/c2f to d9/c46 0 2024-03-29T08:07:18.598 INFO:tasks.workunit.client.1.smithi148.stdout:9/479: rmdir db/d10/d9b 0 2024-03-29T08:07:18.599 INFO:tasks.workunit.client.1.smithi148.stdout:6/443: dwrite d6/d9/d1e/d22/d28/f74 [0,4194304] 0 2024-03-29T08:07:18.601 INFO:tasks.workunit.client.1.smithi148.stdout:3/506: write de/d24/d33/d39/d46/d30/fa2 [780235,35784] 0 2024-03-29T08:07:18.606 INFO:tasks.workunit.client.1.smithi148.stdout:8/515: dread d17/f23 [0,4194304] 0 2024-03-29T08:07:18.606 INFO:tasks.workunit.client.1.smithi148.stdout:6/444: chown d6/d9/f1f 14 1 2024-03-29T08:07:18.606 INFO:tasks.workunit.client.1.smithi148.stdout:4/476: write df/d2c/d51/d77/d17/d73/f7a [1559652,102412] 0 2024-03-29T08:07:18.607 INFO:tasks.workunit.client.0.smithi037.stdout:1/403: rename d2/d6/dc/d18/d1d/f2c to d2/d4b/d80/f8b 0 2024-03-29T08:07:18.609 INFO:tasks.workunit.client.1.smithi148.stdout:7/460: getdents d8/d14/d45/d4d/d78 0 2024-03-29T08:07:18.610 INFO:tasks.workunit.client.0.smithi037.stdout:0/227: creat d9/da/d1b/d22/f47 x:0 0 0 2024-03-29T08:07:18.612 INFO:tasks.workunit.client.1.smithi148.stdout:6/445: write d6/d9/f52 [696376,54681] 0 2024-03-29T08:07:18.613 INFO:tasks.workunit.client.0.smithi037.stdout:9/547: dwrite d1/d3/d44/f8c [0,4194304] 0 2024-03-29T08:07:18.615 INFO:tasks.workunit.client.1.smithi148.stdout:0/466: creat db/de/df/d2b/d3b/d47/d4c/d5a/f98 x:0 0 0 2024-03-29T08:07:18.616 INFO:tasks.workunit.client.1.smithi148.stdout:9/480: unlink db/l1f 0 2024-03-29T08:07:18.617 INFO:tasks.workunit.client.1.smithi148.stdout:2/478: creat d7/da/d33/d3c/d54/f95 x:0 0 0 2024-03-29T08:07:18.620 INFO:tasks.workunit.client.1.smithi148.stdout:3/507: creat de/d24/d33/d39/d46/d70/fa9 x:0 0 0 2024-03-29T08:07:18.623 INFO:tasks.workunit.client.1.smithi148.stdout:0/467: readlink db/de/df/d14/l1d 0 2024-03-29T08:07:18.623 INFO:tasks.workunit.client.0.smithi037.stdout:9/548: symlink d1/d4/d19/d1e/d71/d89/lbd 0 2024-03-29T08:07:18.627 INFO:tasks.workunit.client.1.smithi148.stdout:2/479: mkdir d7/da/d52/d96 0 2024-03-29T08:07:18.627 INFO:tasks.workunit.client.1.smithi148.stdout:1/462: getdents d3/de/d25 0 2024-03-29T08:07:18.629 INFO:tasks.workunit.client.1.smithi148.stdout:6/446: dread d6/d9/d1e/d27/f6e [0,4194304] 0 2024-03-29T08:07:18.629 INFO:tasks.workunit.client.1.smithi148.stdout:4/477: rename df/d2c/d51/d77/d26/d9c to df/da0 0 2024-03-29T08:07:18.630 INFO:tasks.workunit.client.0.smithi037.stdout:9/549: mkdir d1/d4/d19/d1e/d71/d89/dbe 0 2024-03-29T08:07:18.633 INFO:tasks.workunit.client.1.smithi148.stdout:2/480: dread - d7/d24/f60 zero size 2024-03-29T08:07:18.634 INFO:tasks.workunit.client.1.smithi148.stdout:0/468: write db/de/df/d2b/f93 [541414,22385] 0 2024-03-29T08:07:18.635 INFO:tasks.workunit.client.1.smithi148.stdout:8/516: dwrite d17/d49/f9c [0,4194304] 0 2024-03-29T08:07:18.636 INFO:tasks.workunit.client.0.smithi037.stdout:0/228: rename d9/da/d1b/d22/c2b to d9/da/dd/c48 0 2024-03-29T08:07:18.638 INFO:tasks.workunit.client.1.smithi148.stdout:1/463: write d3/de/d25/d51/f54 [1159693,76115] 0 2024-03-29T08:07:18.639 INFO:tasks.workunit.client.1.smithi148.stdout:3/508: creat de/d24/d33/d39/d46/d30/d37/faa x:0 0 0 2024-03-29T08:07:18.641 INFO:tasks.workunit.client.1.smithi148.stdout:7/461: truncate d8/d14/f16 1987472 0 2024-03-29T08:07:18.642 INFO:tasks.workunit.client.1.smithi148.stdout:6/447: mknod d6/d9/d1e/d27/c85 0 2024-03-29T08:07:18.644 INFO:tasks.workunit.client.0.smithi037.stdout:4/372: dread d3/d4/d15/d1d/db/d45/d1e/f4a [0,4194304] 0 2024-03-29T08:07:18.645 INFO:tasks.workunit.client.1.smithi148.stdout:3/509: write de/d24/d33/d39/d46/d30/d37/faa [189115,50537] 0 2024-03-29T08:07:18.645 INFO:tasks.workunit.client.0.smithi037.stdout:1/404: rename d2/d6/dc/d18/d1d/d5e/d72/f7b to d2/d6/dc/d18/d1d/d21/d38/f8c 0 2024-03-29T08:07:18.646 INFO:tasks.workunit.client.1.smithi148.stdout:6/448: read - d6/d9/d1e/d20/d30/d51/f76 zero size 2024-03-29T08:07:18.648 INFO:tasks.workunit.client.1.smithi148.stdout:3/510: fsync de/d24/d33/f94 0 2024-03-29T08:07:18.648 INFO:tasks.workunit.client.0.smithi037.stdout:9/550: dread d1/d4/d19/d1e/d4a/f53 [0,4194304] 0 2024-03-29T08:07:18.649 INFO:tasks.workunit.client.1.smithi148.stdout:9/481: dread db/d22/d26/d54/f36 [0,4194304] 0 2024-03-29T08:07:18.650 INFO:tasks.workunit.client.1.smithi148.stdout:1/464: unlink d3/de/d21/l60 0 2024-03-29T08:07:18.651 INFO:tasks.workunit.client.0.smithi037.stdout:0/229: chown d9/da/dd/d20/c32 1 1 2024-03-29T08:07:18.653 INFO:tasks.workunit.client.1.smithi148.stdout:8/517: chown f15 5 1 2024-03-29T08:07:18.657 INFO:tasks.workunit.client.0.smithi037.stdout:0/230: stat d9/da/dd/d1c/c2e 0 2024-03-29T08:07:18.659 INFO:tasks.workunit.client.1.smithi148.stdout:2/481: dwrite d7/da/d33/d45/d4d/d2e/d37/f6c [0,4194304] 0 2024-03-29T08:07:18.660 INFO:tasks.workunit.client.1.smithi148.stdout:4/478: dread df/d2c/d51/f57 [0,4194304] 0 2024-03-29T08:07:18.660 INFO:tasks.workunit.client.1.smithi148.stdout:6/449: unlink d6/f2d 0 2024-03-29T08:07:18.662 INFO:tasks.workunit.client.1.smithi148.stdout:3/511: mknod de/d24/d33/d39/d46/d30/cab 0 2024-03-29T08:07:18.663 INFO:tasks.workunit.client.1.smithi148.stdout:2/482: truncate d7/da/d4c/f79 874630 0 2024-03-29T08:07:18.665 INFO:tasks.workunit.client.1.smithi148.stdout:1/465: chown d3/de/d1d/l69 521504841 1 2024-03-29T08:07:18.669 INFO:tasks.workunit.client.1.smithi148.stdout:4/479: symlink df/d2c/d51/d77/d26/d9b/la1 0 2024-03-29T08:07:18.671 INFO:tasks.workunit.client.1.smithi148.stdout:6/450: rename d6/d9/d1e/d20/c42 to d6/d9/d1e/c86 0 2024-03-29T08:07:18.673 INFO:tasks.workunit.client.1.smithi148.stdout:3/512: creat de/d24/d33/d39/d7f/fac x:0 0 0 2024-03-29T08:07:18.676 INFO:tasks.workunit.client.1.smithi148.stdout:2/483: creat d7/da/d33/d45/d4d/d2e/f97 x:0 0 0 2024-03-29T08:07:18.678 INFO:tasks.workunit.client.1.smithi148.stdout:1/466: rmdir d3/d18 39 2024-03-29T08:07:18.681 INFO:tasks.workunit.client.1.smithi148.stdout:6/451: creat d6/d9/d1e/d36/f87 x:0 0 0 2024-03-29T08:07:18.683 INFO:tasks.workunit.client.1.smithi148.stdout:2/484: unlink d7/da/d33/d3c/f23 0 2024-03-29T08:07:18.684 INFO:tasks.workunit.client.1.smithi148.stdout:4/480: symlink df/d2c/d51/d77/la2 0 2024-03-29T08:07:18.686 INFO:tasks.workunit.client.1.smithi148.stdout:1/467: mkdir d3/de/d1d/d47/d6d/d84/d8c 0 2024-03-29T08:07:18.687 INFO:tasks.workunit.client.1.smithi148.stdout:6/452: read d6/d9/d1e/d22/f4a [372166,35655] 0 2024-03-29T08:07:18.690 INFO:tasks.workunit.client.1.smithi148.stdout:4/481: rmdir df/d2c/d80 39 2024-03-29T08:07:18.691 INFO:tasks.workunit.client.1.smithi148.stdout:0/469: creat db/de/df/d2b/d3b/d47/d4c/d5a/d8d/f99 x:0 0 0 2024-03-29T08:07:18.697 INFO:tasks.workunit.client.1.smithi148.stdout:1/468: rename d3/d18/l2f to d3/d55/d7e/l8d 0 2024-03-29T08:07:18.699 INFO:tasks.workunit.client.1.smithi148.stdout:4/482: write df/d2c/f8d [483299,46185] 0 2024-03-29T08:07:18.701 INFO:tasks.workunit.client.1.smithi148.stdout:9/482: truncate db/d20/f70 3275927 0 2024-03-29T08:07:18.703 INFO:tasks.workunit.client.1.smithi148.stdout:3/513: write de/d3b/d49/f8a [1794834,64193] 0 2024-03-29T08:07:18.704 INFO:tasks.workunit.client.1.smithi148.stdout:2/485: dwrite d7/da/f3f [0,4194304] 0 2024-03-29T08:07:18.705 INFO:tasks.workunit.client.1.smithi148.stdout:1/469: creat d3/d18/d71/f8e x:0 0 0 2024-03-29T08:07:18.705 INFO:tasks.workunit.client.1.smithi148.stdout:9/483: dread - db/f84 zero size 2024-03-29T08:07:18.708 INFO:tasks.workunit.client.1.smithi148.stdout:3/514: chown de/d24/d33 255973849 1 2024-03-29T08:07:18.709 INFO:tasks.workunit.client.1.smithi148.stdout:6/453: dwrite d6/f41 [0,4194304] 0 2024-03-29T08:07:18.711 INFO:tasks.workunit.client.0.smithi037.stdout:1/405: creat d2/d6/dc/d18/f8d x:0 0 0 2024-03-29T08:07:18.712 INFO:tasks.workunit.client.1.smithi148.stdout:3/515: write de/d24/d33/d39/d71/f9b [172847,10139] 0 2024-03-29T08:07:18.713 INFO:tasks.workunit.client.1.smithi148.stdout:4/483: unlink df/d2c/d51/d77/d26/d9b/f9e 0 2024-03-29T08:07:18.713 INFO:tasks.workunit.client.0.smithi037.stdout:4/373: mkdir d3/d4/d15/d1d/db/d45/d1e/d59/d7d 0 2024-03-29T08:07:18.715 INFO:tasks.workunit.client.1.smithi148.stdout:3/516: read de/d24/f27 [359244,5261] 0 2024-03-29T08:07:18.718 INFO:tasks.workunit.client.1.smithi148.stdout:1/470: rename d3/d55/d44 to d3/d18/d4d/d8f 0 2024-03-29T08:07:18.720 INFO:tasks.workunit.client.1.smithi148.stdout:9/484: chown db/d22/d26/d54/d52/c96 245 1 2024-03-29T08:07:18.725 INFO:tasks.workunit.client.1.smithi148.stdout:8/518: write f7 [1790058,106935] 0 2024-03-29T08:07:18.726 INFO:tasks.workunit.client.1.smithi148.stdout:1/471: truncate d3/de/d6e/f88 594572 0 2024-03-29T08:07:18.727 INFO:tasks.workunit.client.1.smithi148.stdout:3/517: dread de/d24/d33/d39/d46/d30/d37/faa [0,4194304] 0 2024-03-29T08:07:18.727 INFO:tasks.workunit.client.1.smithi148.stdout:4/484: mknod df/d2c/d51/d77/d26/d34/ca3 0 2024-03-29T08:07:18.728 INFO:tasks.workunit.client.1.smithi148.stdout:8/519: write d17/d18/f3f [576158,7271] 0 2024-03-29T08:07:18.730 INFO:tasks.workunit.client.1.smithi148.stdout:4/485: fsync df/d71/f88 0 2024-03-29T08:07:18.730 INFO:tasks.workunit.client.1.smithi148.stdout:6/454: rmdir d6/d9/d1e/d27/d79 0 2024-03-29T08:07:18.731 INFO:tasks.workunit.client.1.smithi148.stdout:2/486: getdents d7/da/d52 0 2024-03-29T08:07:18.731 INFO:tasks.workunit.client.1.smithi148.stdout:0/470: creat db/dc/d78/f9a x:0 0 0 2024-03-29T08:07:18.733 INFO:tasks.workunit.client.1.smithi148.stdout:1/472: rename d3/de/d25/d35/l49 to d3/de/d21/l90 0 2024-03-29T08:07:18.735 INFO:tasks.workunit.client.1.smithi148.stdout:4/486: symlink df/da0/la4 0 2024-03-29T08:07:18.736 INFO:tasks.workunit.client.0.smithi037.stdout:1/406: write d2/d4b/d80/f8b [4960275,100516] 0 2024-03-29T08:07:18.736 INFO:tasks.workunit.client.1.smithi148.stdout:9/485: symlink db/d10/d2d/d4e/d5f/d6e/d7a/la4 0 2024-03-29T08:07:18.737 INFO:tasks.workunit.client.1.smithi148.stdout:6/455: rename d6/c67 to d6/d9/d1e/d36/d4d/c88 0 2024-03-29T08:07:18.738 INFO:tasks.workunit.client.1.smithi148.stdout:2/487: rename d7/da/d33/d45/d4d/d82 to d7/da/d33/d45/d4d/d82/d98 22 2024-03-29T08:07:18.738 INFO:tasks.workunit.client.1.smithi148.stdout:4/487: chown df/d2c/d51/d77/f90 81827 1 2024-03-29T08:07:18.739 INFO:tasks.workunit.client.1.smithi148.stdout:9/486: symlink db/d20/la5 0 2024-03-29T08:07:18.740 INFO:tasks.workunit.client.1.smithi148.stdout:8/520: creat d17/d18/d26/d63/d66/d5f/fa8 x:0 0 0 2024-03-29T08:07:18.741 INFO:tasks.workunit.client.1.smithi148.stdout:1/473: creat d3/de/f91 x:0 0 0 2024-03-29T08:07:18.743 INFO:tasks.workunit.client.1.smithi148.stdout:2/488: unlink d7/da/d33/d45/f88 0 2024-03-29T08:07:18.744 INFO:tasks.workunit.client.1.smithi148.stdout:6/456: mknod d6/d9/d1e/d22/d7d/c89 0 2024-03-29T08:07:18.744 INFO:tasks.workunit.client.1.smithi148.stdout:0/471: creat db/de/df/d11/d1f/f9b x:0 0 0 2024-03-29T08:07:18.745 INFO:tasks.workunit.client.1.smithi148.stdout:9/487: rmdir db/d10/d6d 39 2024-03-29T08:07:18.747 INFO:tasks.workunit.client.1.smithi148.stdout:1/474: rmdir d3/d18/d4d/d76 39 2024-03-29T08:07:18.747 INFO:tasks.workunit.client.1.smithi148.stdout:9/488: fdatasync db/d22/d26/f38 0 2024-03-29T08:07:18.749 INFO:tasks.workunit.client.1.smithi148.stdout:3/518: dwrite de/d3b/d49/f68 [0,4194304] 0 2024-03-29T08:07:18.750 INFO:tasks.workunit.client.1.smithi148.stdout:1/475: chown d3/de/d6e/f88 170122 1 2024-03-29T08:07:18.751 INFO:tasks.workunit.client.0.smithi037.stdout:1/407: fdatasync d2/d4b/f66 0 2024-03-29T08:07:18.751 INFO:tasks.workunit.client.1.smithi148.stdout:6/457: creat d6/d9/d1e/f8a x:0 0 0 2024-03-29T08:07:18.753 INFO:tasks.workunit.client.1.smithi148.stdout:2/489: symlink d7/da/d33/d3c/d54/d70/l99 0 2024-03-29T08:07:18.755 INFO:tasks.workunit.client.0.smithi037.stdout:1/408: chown d2/d6/dc/d5d/d64/f68 21 1 2024-03-29T08:07:18.756 INFO:tasks.workunit.client.1.smithi148.stdout:0/472: dread - db/de/df/d2b/d3b/f49 zero size 2024-03-29T08:07:18.757 INFO:tasks.workunit.client.1.smithi148.stdout:9/489: mkdir db/d10/d2d/d4e/da6 0 2024-03-29T08:07:18.760 INFO:tasks.workunit.client.1.smithi148.stdout:6/458: read d6/d9/f16 [788628,13506] 0 2024-03-29T08:07:18.761 INFO:tasks.workunit.client.0.smithi037.stdout:0/231: link d9/da/d1b/f29 d9/da/d1b/d33/f49 0 2024-03-29T08:07:18.761 INFO:tasks.workunit.client.1.smithi148.stdout:2/490: mkdir d7/da/d4c/d9a 0 2024-03-29T08:07:18.762 INFO:tasks.workunit.client.0.smithi037.stdout:1/409: fsync d2/d6/dc/f25 0 2024-03-29T08:07:18.764 INFO:tasks.workunit.client.0.smithi037.stdout:1/410: chown d2/d6/f36 44449067 1 2024-03-29T08:07:18.764 INFO:tasks.workunit.client.1.smithi148.stdout:8/521: creat d17/d18/fa9 x:0 0 0 2024-03-29T08:07:18.765 INFO:tasks.workunit.client.1.smithi148.stdout:2/491: dread - d7/da/d33/d45/d4d/f7a zero size 2024-03-29T08:07:18.771 INFO:tasks.workunit.client.1.smithi148.stdout:3/519: dwrite de/d24/d33/d39/d46/d30/d37/f6f [0,4194304] 0 2024-03-29T08:07:18.772 INFO:tasks.workunit.client.1.smithi148.stdout:0/473: symlink db/de/df/d2b/d3b/d47/d7c/l9c 0 2024-03-29T08:07:18.776 INFO:tasks.workunit.client.1.smithi148.stdout:3/520: write de/d24/d33/d39/d46/d30/d37/f7a [53571,10856] 0 2024-03-29T08:07:18.778 INFO:tasks.workunit.client.1.smithi148.stdout:8/522: creat d17/d18/d26/d63/d7d/faa x:0 0 0 2024-03-29T08:07:18.778 INFO:tasks.workunit.client.1.smithi148.stdout:3/521: chown de/d24/d33/d39/d46/d8d 4 1 2024-03-29T08:07:18.780 INFO:tasks.workunit.client.1.smithi148.stdout:5/473: sync 2024-03-29T08:07:18.783 INFO:tasks.workunit.client.1.smithi148.stdout:3/522: fdatasync de/d24/d33/d39/d46/d30/d37/d62/d63/f99 0 2024-03-29T08:07:18.785 INFO:tasks.workunit.client.1.smithi148.stdout:2/492: dwrite d7/f19 [0,4194304] 0 2024-03-29T08:07:18.792 INFO:tasks.workunit.client.1.smithi148.stdout:9/490: dread db/d22/d26/d54/d4d/d35/f5e [0,4194304] 0 2024-03-29T08:07:18.796 INFO:tasks.workunit.client.1.smithi148.stdout:0/474: write f9 [1635513,130785] 0 2024-03-29T08:07:18.799 INFO:tasks.workunit.client.1.smithi148.stdout:6/459: rename d6/d9/d1e/d20/d30/d51/c66 to d6/d69/c8b 0 2024-03-29T08:07:18.801 INFO:tasks.workunit.client.1.smithi148.stdout:0/475: mknod db/de/df/d2b/d3b/c9d 0 2024-03-29T08:07:18.803 INFO:tasks.workunit.client.0.smithi037.stdout:0/232: dread f5 [4194304,4194304] 0 2024-03-29T08:07:18.803 INFO:tasks.workunit.client.1.smithi148.stdout:0/476: write db/de/df/d14/f3c [1398860,52162] 0 2024-03-29T08:07:18.804 INFO:tasks.workunit.client.1.smithi148.stdout:0/477: dread - db/dc/d78/f79 zero size 2024-03-29T08:07:18.806 INFO:tasks.workunit.client.0.smithi037.stdout:1/411: dwrite d2/d6/dc/d18/d1d/f3f [0,4194304] 0 2024-03-29T08:07:18.806 INFO:tasks.workunit.client.0.smithi037.stdout:0/233: chown d9/ff 362973 1 2024-03-29T08:07:18.807 INFO:tasks.workunit.client.0.smithi037.stdout:9/551: write d1/d4/d19/d1e/d4a/d6f/f83 [7156895,103343] 0 2024-03-29T08:07:18.811 INFO:tasks.workunit.client.1.smithi148.stdout:0/478: mknod db/de/df/d2b/d3b/d47/d4c/d5a/d8d/c9e 0 2024-03-29T08:07:18.815 INFO:tasks.workunit.client.1.smithi148.stdout:5/474: chown d0/db/d7b/d16/f8f 9979103 1 2024-03-29T08:07:18.815 INFO:tasks.workunit.client.0.smithi037.stdout:0/234: rmdir d9/da/d1b/d22 39 2024-03-29T08:07:18.817 INFO:tasks.workunit.client.0.smithi037.stdout:1/412: write d2/d6/dc/f4a [1912600,94765] 0 2024-03-29T08:07:18.818 INFO:tasks.workunit.client.1.smithi148.stdout:5/475: chown d0/db/ld 16240978 1 2024-03-29T08:07:18.819 INFO:tasks.workunit.client.1.smithi148.stdout:2/493: unlink d7/da/d4c/l4e 0 2024-03-29T08:07:18.821 INFO:tasks.workunit.client.1.smithi148.stdout:4/488: truncate df/f27 612862 0 2024-03-29T08:07:18.822 INFO:tasks.workunit.client.1.smithi148.stdout:0/479: mkdir db/de/df/d2b/d3b/d47/d9f 0 2024-03-29T08:07:18.826 INFO:tasks.workunit.client.0.smithi037.stdout:9/552: mknod d1/d4/d19/d1e/d4a/d6f/da2/d68/cbf 0 2024-03-29T08:07:18.827 INFO:tasks.workunit.client.1.smithi148.stdout:5/476: truncate d0/db/f10 2090893 0 2024-03-29T08:07:18.829 INFO:tasks.workunit.client.1.smithi148.stdout:5/477: write d0/f2c [792083,86292] 0 2024-03-29T08:07:18.830 INFO:tasks.workunit.client.0.smithi037.stdout:1/413: mknod d2/d6/dc/d58/c8e 0 2024-03-29T08:07:18.831 INFO:tasks.workunit.client.0.smithi037.stdout:0/235: symlink d9/da/l4a 0 2024-03-29T08:07:18.834 INFO:tasks.workunit.client.1.smithi148.stdout:9/491: rename db/d22/d26/f8a to db/fa7 0 2024-03-29T08:07:18.838 INFO:tasks.workunit.client.1.smithi148.stdout:6/460: link d6/d9/d1e/d36/d4d/l63 d6/d9/d1e/d22/d28/l8c 0 2024-03-29T08:07:18.841 INFO:tasks.workunit.client.0.smithi037.stdout:0/236: creat d9/da/dd/d13/f4b x:0 0 0 2024-03-29T08:07:18.842 INFO:tasks.workunit.client.1.smithi148.stdout:0/480: getdents db/de 0 2024-03-29T08:07:18.842 INFO:tasks.workunit.client.1.smithi148.stdout:6/461: write d6/d9/d1e/d36/f6f [982941,99141] 0 2024-03-29T08:07:18.843 INFO:tasks.workunit.client.1.smithi148.stdout:5/478: chown d0/db/d1e/d36/d41/f58 51388379 1 2024-03-29T08:07:18.844 INFO:tasks.workunit.client.0.smithi037.stdout:9/553: dread d1/d4/d19/d1e/d4a/fa0 [0,4194304] 0 2024-03-29T08:07:18.848 INFO:tasks.workunit.client.1.smithi148.stdout:0/481: mkdir db/de/df/d11/d42/da0 0 2024-03-29T08:07:18.849 INFO:tasks.workunit.client.0.smithi037.stdout:9/554: chown d1/d4/d19/d1e/d4a/d6f/da2/c1d 741442710 1 2024-03-29T08:07:18.851 INFO:tasks.workunit.client.0.smithi037.stdout:4/374: symlink d3/d4/d15/d1d/db/l7e 0 2024-03-29T08:07:18.852 INFO:tasks.workunit.client.1.smithi148.stdout:4/489: dread fd [0,4194304] 0 2024-03-29T08:07:18.852 INFO:tasks.workunit.client.0.smithi037.stdout:9/555: write d1/f4c [1232496,128234] 0 2024-03-29T08:07:18.856 INFO:tasks.workunit.client.1.smithi148.stdout:2/494: symlink d7/da/d33/d45/d4d/d2e/l9b 0 2024-03-29T08:07:18.858 INFO:tasks.workunit.client.1.smithi148.stdout:6/462: dread d6/d9/d1e/d36/f6f [0,4194304] 0 2024-03-29T08:07:18.859 INFO:tasks.workunit.client.0.smithi037.stdout:4/375: rmdir d3/d4/d15/d1d/db/d45 39 2024-03-29T08:07:18.862 INFO:tasks.workunit.client.1.smithi148.stdout:8/523: write d17/d18/d26/f35 [1900236,13590] 0 2024-03-29T08:07:18.863 INFO:tasks.workunit.client.1.smithi148.stdout:4/490: dread df/d2c/f8d [0,4194304] 0 2024-03-29T08:07:18.863 INFO:tasks.workunit.client.0.smithi037.stdout:4/376: creat d3/d4/d27/f7f x:0 0 0 2024-03-29T08:07:18.865 INFO:tasks.workunit.client.1.smithi148.stdout:9/492: dwrite db/fa7 [0,4194304] 0 2024-03-29T08:07:18.868 INFO:tasks.workunit.client.1.smithi148.stdout:9/493: fdatasync db/d22/d26/d54/d4d/d35/d4b/d5a/f77 0 2024-03-29T08:07:18.869 INFO:tasks.workunit.client.1.smithi148.stdout:5/479: mkdir d0/db/d1e/d5b/d66/d72/d95/d9c 0 2024-03-29T08:07:18.873 INFO:tasks.workunit.client.1.smithi148.stdout:6/463: chown d6/d9/d1e/d22/l5f 3859508 1 2024-03-29T08:07:18.877 INFO:tasks.workunit.client.0.smithi037.stdout:0/237: dread d9/da/f11 [0,4194304] 0 2024-03-29T08:07:18.877 INFO:tasks.workunit.client.1.smithi148.stdout:4/491: read df/d2c/d51/d77/d26/d34/d3e/f75 [4149599,93370] 0 2024-03-29T08:07:18.881 INFO:tasks.workunit.client.1.smithi148.stdout:5/480: rmdir d0/db/d1e/d35/d42/d67/d76/d81 39 2024-03-29T08:07:18.882 INFO:tasks.workunit.client.0.smithi037.stdout:4/377: dread d3/d4/d15/d1d/db/f16 [0,4194304] 0 2024-03-29T08:07:18.883 INFO:tasks.workunit.client.1.smithi148.stdout:0/482: truncate db/f7b 3384631 0 2024-03-29T08:07:18.886 INFO:tasks.workunit.client.1.smithi148.stdout:6/464: getdents d6/d9/d1e/d22/d7d 0 2024-03-29T08:07:18.888 INFO:tasks.workunit.client.1.smithi148.stdout:0/483: unlink c3 0 2024-03-29T08:07:18.888 INFO:tasks.workunit.client.0.smithi037.stdout:5/493: sync 2024-03-29T08:07:18.888 INFO:tasks.workunit.client.0.smithi037.stdout:0/238: read d9/da/d1b/f29 [49944,75597] 0 2024-03-29T08:07:18.889 INFO:tasks.workunit.client.1.smithi148.stdout:9/494: dread db/f21 [0,4194304] 0 2024-03-29T08:07:18.892 INFO:tasks.workunit.client.1.smithi148.stdout:6/465: write d6/d9/d1e/d20/d30/f47 [3153750,81057] 0 2024-03-29T08:07:18.892 INFO:tasks.workunit.client.1.smithi148.stdout:5/481: link d0/db/d7b/f91 d0/db/d1e/d5b/d66/d72/d95/f9d 0 2024-03-29T08:07:18.895 INFO:tasks.workunit.client.1.smithi148.stdout:5/482: dread - d0/db/d1e/d6e/f92 zero size 2024-03-29T08:07:18.895 INFO:tasks.workunit.client.1.smithi148.stdout:7/462: sync 2024-03-29T08:07:18.896 INFO:tasks.workunit.client.0.smithi037.stdout:1/414: dwrite d2/d6/dc/d5d/d64/f68 [0,4194304] 0 2024-03-29T08:07:18.897 INFO:tasks.workunit.client.1.smithi148.stdout:2/495: dwrite d7/da/d33/f3e [0,4194304] 0 2024-03-29T08:07:18.899 INFO:tasks.workunit.client.0.smithi037.stdout:1/415: readlink d2/d4b/l53 0 2024-03-29T08:07:18.900 INFO:tasks.workunit.client.1.smithi148.stdout:2/496: write d7/da/f7f [811462,111039] 0 2024-03-29T08:07:18.903 INFO:tasks.workunit.client.0.smithi037.stdout:5/494: rename d8/d22/d37/d60/d61/f69 to d8/d34/d4b/fa8 0 2024-03-29T08:07:18.904 INFO:tasks.workunit.client.1.smithi148.stdout:6/466: symlink d6/d9/l8d 0 2024-03-29T08:07:18.915 INFO:tasks.workunit.client.0.smithi037.stdout:5/495: getdents d8/d22/d37/d8d 0 2024-03-29T08:07:18.917 INFO:tasks.workunit.client.1.smithi148.stdout:5/483: unlink d0/db/d7b/d16/d2e/f39 0 2024-03-29T08:07:18.918 INFO:tasks.workunit.client.0.smithi037.stdout:5/496: fsync d8/d34/d4b/fa8 0 2024-03-29T08:07:18.920 INFO:tasks.workunit.client.0.smithi037.stdout:5/497: chown d8/d34/f4a 133510453 1 2024-03-29T08:07:18.921 INFO:tasks.workunit.client.1.smithi148.stdout:4/492: dwrite f4 [0,4194304] 0 2024-03-29T08:07:18.923 INFO:tasks.workunit.client.1.smithi148.stdout:6/467: getdents d6/d9/d1e/d22/d28 0 2024-03-29T08:07:18.923 INFO:tasks.workunit.client.1.smithi148.stdout:9/495: dwrite db/d22/d26/d54/d4d/d35/d4b/d58/f92 [0,4194304] 0 2024-03-29T08:07:18.924 INFO:tasks.workunit.client.0.smithi037.stdout:5/498: write d8/d22/f93 [238507,52757] 0 2024-03-29T08:07:18.929 INFO:tasks.workunit.client.1.smithi148.stdout:9/496: write db/f84 [90348,97642] 0 2024-03-29T08:07:18.930 INFO:tasks.workunit.client.1.smithi148.stdout:4/493: symlink df/d60/la5 0 2024-03-29T08:07:18.931 INFO:tasks.workunit.client.1.smithi148.stdout:2/497: dwrite d7/d24/f60 [0,4194304] 0 2024-03-29T08:07:18.932 INFO:tasks.workunit.client.1.smithi148.stdout:6/468: chown d6/d9/d1e/d36/d4d/f57 35582 1 2024-03-29T08:07:18.936 INFO:tasks.workunit.client.1.smithi148.stdout:2/498: chown d7/da/d33/d45/d4d/c27 94813969 1 2024-03-29T08:07:18.940 INFO:tasks.workunit.client.1.smithi148.stdout:0/484: dread db/de/f62 [0,4194304] 0 2024-03-29T08:07:18.943 INFO:tasks.workunit.client.1.smithi148.stdout:9/497: unlink db/d22/d26/d54/d4d/d35/d4b/d8e/f99 0 2024-03-29T08:07:18.943 INFO:tasks.workunit.client.1.smithi148.stdout:5/484: getdents d0/db/d7b/d16/d87 0 2024-03-29T08:07:18.945 INFO:tasks.workunit.client.1.smithi148.stdout:5/485: dread - d0/db/d7b/d16/d87/f90 zero size 2024-03-29T08:07:18.945 INFO:tasks.workunit.client.0.smithi037.stdout:1/416: dread d2/d6/dc/d18/f3e [0,4194304] 0 2024-03-29T08:07:18.947 INFO:tasks.workunit.client.1.smithi148.stdout:6/469: link d6/c6a d6/d9/d1e/d22/d28/c8e 0 2024-03-29T08:07:18.948 INFO:tasks.workunit.client.1.smithi148.stdout:5/486: write d0/db/d1e/d5b/d66/f74 [889717,17919] 0 2024-03-29T08:07:18.951 INFO:tasks.workunit.client.1.smithi148.stdout:6/470: unlink d6/d9/d1e/d20/f39 0 2024-03-29T08:07:18.952 INFO:tasks.workunit.client.1.smithi148.stdout:5/487: mkdir d0/d46/d9e 0 2024-03-29T08:07:18.954 INFO:tasks.workunit.client.0.smithi037.stdout:0/239: dread f7 [8388608,4194304] 0 2024-03-29T08:07:18.954 INFO:tasks.workunit.client.1.smithi148.stdout:5/488: chown d0/db/d1e/d35/d42/f6b 189637 1 2024-03-29T08:07:18.955 INFO:tasks.workunit.client.1.smithi148.stdout:6/471: mknod d6/d9/c8f 0 2024-03-29T08:07:18.958 INFO:tasks.workunit.client.1.smithi148.stdout:2/499: dwrite d7/da/d33/d3c/d54/f93 [0,4194304] 0 2024-03-29T08:07:18.965 INFO:tasks.workunit.client.1.smithi148.stdout:6/472: link d6/d9/d1e/d27/f64 d6/d9/d1e/d27/f90 0 2024-03-29T08:07:18.970 INFO:tasks.workunit.client.1.smithi148.stdout:6/473: mknod d6/d9/d1e/d22/d28/c91 0 2024-03-29T08:07:18.974 INFO:tasks.workunit.client.0.smithi037.stdout:0/240: dwrite d9/da/f25 [0,4194304] 0 2024-03-29T08:07:18.977 INFO:tasks.workunit.client.0.smithi037.stdout:0/241: mknod d9/da/d1b/d22/c4c 0 2024-03-29T08:07:18.980 INFO:tasks.workunit.client.0.smithi037.stdout:0/242: read d9/ff [1222037,85986] 0 2024-03-29T08:07:18.982 INFO:tasks.workunit.client.1.smithi148.stdout:5/489: truncate d0/f9 2441260 0 2024-03-29T08:07:18.984 INFO:tasks.workunit.client.1.smithi148.stdout:6/474: write d6/d9/f16 [2999853,125300] 0 2024-03-29T08:07:18.986 INFO:tasks.workunit.client.1.smithi148.stdout:5/490: creat d0/db/d1e/d36/d41/f9f x:0 0 0 2024-03-29T08:07:18.992 INFO:tasks.workunit.client.0.smithi037.stdout:0/243: dwrite d9/f27 [0,4194304] 0 2024-03-29T08:07:18.996 INFO:tasks.workunit.client.1.smithi148.stdout:6/475: dwrite d6/d9/d1e/d22/f34 [4194304,4194304] 0 2024-03-29T08:07:19.003 INFO:tasks.workunit.client.1.smithi148.stdout:6/476: truncate d6/d9/d1e/d36/f87 698178 0 2024-03-29T08:07:19.008 INFO:tasks.workunit.client.0.smithi037.stdout:5/499: write d8/d22/d4f/f68 [177002,93148] 0 2024-03-29T08:07:19.008 INFO:tasks.workunit.client.0.smithi037.stdout:1/417: write d2/d6/dc/d18/d1d/d21/f55 [421768,58966] 0 2024-03-29T08:07:19.009 INFO:tasks.workunit.client.0.smithi037.stdout:5/500: chown d8/d22/d4f 1 1 2024-03-29T08:07:19.012 INFO:tasks.workunit.client.1.smithi148.stdout:0/485: fdatasync db/f7b 0 2024-03-29T08:07:19.014 INFO:tasks.workunit.client.0.smithi037.stdout:5/501: fsync d8/f17 0 2024-03-29T08:07:19.014 INFO:tasks.workunit.client.1.smithi148.stdout:6/477: unlink d6/d9/d1e/c86 0 2024-03-29T08:07:19.016 INFO:tasks.workunit.client.1.smithi148.stdout:0/486: chown db/de/df/d2b/d3b/f84 29928124 1 2024-03-29T08:07:19.016 INFO:tasks.workunit.client.0.smithi037.stdout:1/418: rmdir d2/d6/dc/d18/d22/d4f/d7c 39 2024-03-29T08:07:19.017 INFO:tasks.workunit.client.0.smithi037.stdout:0/244: dread f7 [4194304,4194304] 0 2024-03-29T08:07:19.018 INFO:tasks.workunit.client.1.smithi148.stdout:0/487: readlink l5 0 2024-03-29T08:07:19.021 INFO:tasks.workunit.client.1.smithi148.stdout:6/478: dread d6/d9/d1e/d22/f31 [0,4194304] 0 2024-03-29T08:07:19.022 INFO:tasks.workunit.client.0.smithi037.stdout:5/502: truncate d8/d34/d4b/fa8 3989731 0 2024-03-29T08:07:19.023 INFO:tasks.workunit.client.0.smithi037.stdout:0/245: truncate d9/da/dd/d1c/f39 256871 0 2024-03-29T08:07:19.026 INFO:tasks.workunit.client.0.smithi037.stdout:1/419: symlink d2/d6/dc/d18/d22/d4f/d7c/l8f 0 2024-03-29T08:07:19.026 INFO:tasks.workunit.client.1.smithi148.stdout:6/479: rmdir d6/d9/d1e/d20/d6b 39 2024-03-29T08:07:19.027 INFO:tasks.workunit.client.0.smithi037.stdout:0/246: fdatasync d9/da/dd/d13/f31 0 2024-03-29T08:07:19.029 INFO:tasks.workunit.client.0.smithi037.stdout:1/420: mknod d2/d6/dc/d18/c90 0 2024-03-29T08:07:19.029 INFO:tasks.workunit.client.1.smithi148.stdout:0/488: dread db/de/df/f95 [0,4194304] 0 2024-03-29T08:07:19.034 INFO:tasks.workunit.client.1.smithi148.stdout:0/489: unlink db/de/df/d14/l56 0 2024-03-29T08:07:19.034 INFO:tasks.workunit.client.1.smithi148.stdout:6/480: link d6/fd d6/d9/d1e/d27/d48/d73/d7e/f92 0 2024-03-29T08:07:19.036 INFO:tasks.workunit.client.1.smithi148.stdout:6/481: write d6/d9/d1e/d36/f87 [1224281,54106] 0 2024-03-29T08:07:19.038 INFO:tasks.workunit.client.1.smithi148.stdout:6/482: read - d6/d9/d1e/d20/d30/d51/f76 zero size 2024-03-29T08:07:19.050 INFO:tasks.workunit.client.0.smithi037.stdout:1/421: truncate d2/d6/dc/d18/d1d/d21/f37 2876756 0 2024-03-29T08:07:19.051 INFO:tasks.workunit.client.0.smithi037.stdout:5/503: dread d8/d18/f30 [0,4194304] 0 2024-03-29T08:07:19.052 INFO:tasks.workunit.client.1.smithi148.stdout:6/483: dwrite d6/d9/f16 [4194304,4194304] 0 2024-03-29T08:07:19.054 INFO:tasks.workunit.client.0.smithi037.stdout:1/422: truncate d2/d6/dc/f25 2713922 0 2024-03-29T08:07:19.056 INFO:tasks.workunit.client.0.smithi037.stdout:5/504: creat d8/d22/d37/d60/fa9 x:0 0 0 2024-03-29T08:07:19.057 INFO:tasks.workunit.client.1.smithi148.stdout:6/484: creat d6/d9/d1e/d20/d30/f93 x:0 0 0 2024-03-29T08:07:19.057 INFO:tasks.workunit.client.0.smithi037.stdout:1/423: chown d2/d6/dc/d18/d1d/f3f 13462 1 2024-03-29T08:07:19.058 INFO:tasks.workunit.client.1.smithi148.stdout:6/485: fsync d6/d9/d1e/d22/f4e 0 2024-03-29T08:07:19.059 INFO:tasks.workunit.client.1.smithi148.stdout:0/490: dwrite db/de/df/d11/d2d/f36 [0,4194304] 0 2024-03-29T08:07:19.061 INFO:tasks.workunit.client.0.smithi037.stdout:5/505: dread d8/d18/d24/f74 [0,4194304] 0 2024-03-29T08:07:19.063 INFO:tasks.workunit.client.0.smithi037.stdout:1/424: creat d2/d6/dc/d7e/f91 x:0 0 0 2024-03-29T08:07:19.067 INFO:tasks.workunit.client.0.smithi037.stdout:0/247: dread d9/ff [0,4194304] 0 2024-03-29T08:07:19.073 INFO:tasks.workunit.client.0.smithi037.stdout:1/425: link d2/d6/dc/d18/d22/d4f/f73 d2/d40/f92 0 2024-03-29T08:07:19.077 INFO:tasks.workunit.client.1.smithi148.stdout:6/486: dwrite d6/d9/f16 [0,4194304] 0 2024-03-29T08:07:19.082 INFO:tasks.workunit.client.1.smithi148.stdout:0/491: truncate db/de/f4b 7895448 0 2024-03-29T08:07:19.082 INFO:tasks.workunit.client.0.smithi037.stdout:0/248: link d9/da/dd/d1c/l43 d9/da/dd/d13/l4d 0 2024-03-29T08:07:19.083 INFO:tasks.workunit.client.1.smithi148.stdout:6/487: write d6/d9/d1e/d20/d30/f56 [10443,77065] 0 2024-03-29T08:07:19.083 INFO:tasks.workunit.client.0.smithi037.stdout:0/249: stat d9/da/dd/d1c 0 2024-03-29T08:07:19.085 INFO:tasks.workunit.client.1.smithi148.stdout:0/492: fdatasync db/dc/d78/f79 0 2024-03-29T08:07:19.085 INFO:tasks.workunit.client.0.smithi037.stdout:0/250: dread - d9/da/dd/d13/f4b zero size 2024-03-29T08:07:19.089 INFO:tasks.workunit.client.0.smithi037.stdout:0/251: creat d9/da/dd/d20/f4e x:0 0 0 2024-03-29T08:07:19.089 INFO:tasks.workunit.client.1.smithi148.stdout:1/476: sync 2024-03-29T08:07:19.090 INFO:tasks.workunit.client.1.smithi148.stdout:3/523: sync 2024-03-29T08:07:19.093 INFO:tasks.workunit.client.0.smithi037.stdout:0/252: symlink d9/da/d1b/d22/l4f 0 2024-03-29T08:07:19.095 INFO:tasks.workunit.client.1.smithi148.stdout:1/477: creat d3/de/d1d/d47/d6d/d84/f92 x:0 0 0 2024-03-29T08:07:19.097 INFO:tasks.workunit.client.0.smithi037.stdout:1/426: dwrite d2/d6/dc/f1e [0,4194304] 0 2024-03-29T08:07:19.097 INFO:tasks.workunit.client.0.smithi037.stdout:0/253: write d9/da/d1b/d22/d2c/f42 [466991,109837] 0 2024-03-29T08:07:19.104 INFO:tasks.workunit.client.1.smithi148.stdout:6/488: dwrite d6/d9/d1e/d36/d4d/f57 [0,4194304] 0 2024-03-29T08:07:19.104 INFO:tasks.workunit.client.1.smithi148.stdout:3/524: dread de/d24/d33/d39/d46/d30/d37/f6f [0,4194304] 0 2024-03-29T08:07:19.106 INFO:tasks.workunit.client.0.smithi037.stdout:6/312: sync 2024-03-29T08:07:19.108 INFO:tasks.workunit.client.1.smithi148.stdout:1/478: link d3/de/d25/d51/f73 d3/f93 0 2024-03-29T08:07:19.110 INFO:tasks.workunit.client.1.smithi148.stdout:1/479: readlink d3/de/d21/d5f/l66 0 2024-03-29T08:07:19.110 INFO:tasks.workunit.client.0.smithi037.stdout:1/427: truncate d2/d6/dc/d18/d1d/d21/d38/f8c 309561 0 2024-03-29T08:07:19.113 INFO:tasks.workunit.client.0.smithi037.stdout:5/506: dread d8/d22/f31 [0,4194304] 0 2024-03-29T08:07:19.115 INFO:tasks.workunit.client.0.smithi037.stdout:5/507: stat d8/d22/d4f 0 2024-03-29T08:07:19.116 INFO:tasks.workunit.client.1.smithi148.stdout:3/525: symlink de/d24/d33/lad 0 2024-03-29T08:07:19.116 INFO:tasks.workunit.client.1.smithi148.stdout:3/526: chown de/d24/d33 0 1 2024-03-29T08:07:19.117 INFO:tasks.workunit.client.0.smithi037.stdout:6/313: mkdir d0/d19/d34/d4e/d69 0 2024-03-29T08:07:19.119 INFO:tasks.workunit.client.0.smithi037.stdout:5/508: symlink d8/d22/d37/d60/laa 0 2024-03-29T08:07:19.119 INFO:tasks.workunit.client.0.smithi037.stdout:1/428: creat d2/d6/d13/f93 x:0 0 0 2024-03-29T08:07:19.120 INFO:tasks.workunit.client.0.smithi037.stdout:0/254: dwrite d9/f27 [0,4194304] 0 2024-03-29T08:07:19.122 INFO:tasks.workunit.client.0.smithi037.stdout:1/429: stat d2/d6/dc/d7e/l88 0 2024-03-29T08:07:19.123 INFO:tasks.workunit.client.0.smithi037.stdout:5/509: chown d8/d22/d37/d60/d61/la4 1894281105 1 2024-03-29T08:07:19.125 INFO:tasks.workunit.client.0.smithi037.stdout:6/314: mkdir d0/d6a 0 2024-03-29T08:07:19.128 INFO:tasks.workunit.client.0.smithi037.stdout:1/430: readlink d2/d6/dc/d18/d1d/d5e/d72/l76 0 2024-03-29T08:07:19.132 INFO:tasks.workunit.client.0.smithi037.stdout:0/255: rmdir d9/da 39 2024-03-29T08:07:19.133 INFO:tasks.workunit.client.0.smithi037.stdout:5/510: rmdir d8/d22/d4f/d70/d79/d7a/d85 39 2024-03-29T08:07:19.133 INFO:tasks.workunit.client.0.smithi037.stdout:5/511: chown d8 1730 1 2024-03-29T08:07:19.134 INFO:tasks.workunit.client.0.smithi037.stdout:6/315: rmdir d0/d19/d42/d60 39 2024-03-29T08:07:19.135 INFO:tasks.workunit.client.1.smithi148.stdout:3/527: link de/d24/f4d de/d24/d33/d39/d46/d30/d37/fae 0 2024-03-29T08:07:19.136 INFO:tasks.workunit.client.0.smithi037.stdout:1/431: fdatasync d2/d6/dc/fd 0 2024-03-29T08:07:19.142 INFO:tasks.workunit.client.1.smithi148.stdout:6/489: write d6/d9/d1e/d27/f59 [4795351,70141] 0 2024-03-29T08:07:19.146 INFO:tasks.workunit.client.0.smithi037.stdout:1/432: rmdir d2 39 2024-03-29T08:07:19.147 INFO:tasks.workunit.client.1.smithi148.stdout:3/528: getdents de/d24/d33/d39/d71 0 2024-03-29T08:07:19.147 INFO:tasks.workunit.client.1.smithi148.stdout:0/493: dwrite db/de/f4b [4194304,4194304] 0 2024-03-29T08:07:19.150 INFO:tasks.workunit.client.1.smithi148.stdout:3/529: dread - de/d24/d33/f94 zero size 2024-03-29T08:07:19.151 INFO:tasks.workunit.client.0.smithi037.stdout:5/512: rmdir d8/d18/d24/d64 0 2024-03-29T08:07:19.152 INFO:tasks.workunit.client.0.smithi037.stdout:1/433: chown d2/le 2186962 1 2024-03-29T08:07:19.154 INFO:tasks.workunit.client.1.smithi148.stdout:6/490: getdents d6/d9/d1e/d20 0 2024-03-29T08:07:19.154 INFO:tasks.workunit.client.1.smithi148.stdout:1/480: dread d3/fa [0,4194304] 0 2024-03-29T08:07:19.158 INFO:tasks.workunit.client.0.smithi037.stdout:1/434: creat d2/f94 x:0 0 0 2024-03-29T08:07:19.159 INFO:tasks.workunit.client.1.smithi148.stdout:6/491: creat d6/d9/d1e/d27/d48/f94 x:0 0 0 2024-03-29T08:07:19.159 INFO:tasks.workunit.client.1.smithi148.stdout:1/481: chown d3/d55/c8b 91734274 1 2024-03-29T08:07:19.161 INFO:tasks.workunit.client.0.smithi037.stdout:5/513: rename d8/d18/c39 to d8/d18/cab 0 2024-03-29T08:07:19.165 INFO:tasks.workunit.client.1.smithi148.stdout:1/482: write d3/de/d25/d35/d42/d23/f40 [3211898,112044] 0 2024-03-29T08:07:19.166 INFO:tasks.workunit.client.0.smithi037.stdout:0/256: dwrite d9/da/d1b/f3d [0,4194304] 0 2024-03-29T08:07:19.166 INFO:tasks.workunit.client.1.smithi148.stdout:3/530: getdents de/d24/d33/d6b 0 2024-03-29T08:07:19.170 INFO:tasks.workunit.client.1.smithi148.stdout:6/492: rename d6/d9/d1e/d36/c71 to d6/d9/d1e/d20/d6b/c95 0 2024-03-29T08:07:19.172 INFO:tasks.workunit.client.1.smithi148.stdout:0/494: dwrite db/de/df/d11/d42/f52 [0,4194304] 0 2024-03-29T08:07:19.172 INFO:tasks.workunit.client.1.smithi148.stdout:8/524: sync 2024-03-29T08:07:19.172 INFO:tasks.workunit.client.1.smithi148.stdout:4/494: sync 2024-03-29T08:07:19.172 INFO:tasks.workunit.client.1.smithi148.stdout:7/463: sync 2024-03-29T08:07:19.173 INFO:tasks.workunit.client.0.smithi037.stdout:0/257: dread d9/da/d1b/d33/f49 [0,4194304] 0 2024-03-29T08:07:19.176 INFO:tasks.workunit.client.1.smithi148.stdout:6/493: symlink d6/d9/l96 0 2024-03-29T08:07:19.176 INFO:tasks.workunit.client.1.smithi148.stdout:7/464: write d8/d14/d19/d37/f9c [547849,120027] 0 2024-03-29T08:07:19.179 INFO:tasks.workunit.client.0.smithi037.stdout:5/514: truncate d8/d34/d4b/fa8 1191089 0 2024-03-29T08:07:19.179 INFO:tasks.workunit.client.1.smithi148.stdout:1/483: getdents d3/d55/d7e 0 2024-03-29T08:07:19.180 INFO:tasks.workunit.client.0.smithi037.stdout:1/435: dwrite d2/d6/dc/f1e [0,4194304] 0 2024-03-29T08:07:19.180 INFO:tasks.workunit.client.0.smithi037.stdout:6/316: dread d0/f4 [0,4194304] 0 2024-03-29T08:07:19.182 INFO:tasks.workunit.client.0.smithi037.stdout:1/436: readlink d2/d6/dc/d5d/d62/l7a 0 2024-03-29T08:07:19.183 INFO:tasks.workunit.client.0.smithi037.stdout:5/515: dread d8/d18/d24/f74 [0,4194304] 0 2024-03-29T08:07:19.184 INFO:tasks.workunit.client.1.smithi148.stdout:8/525: symlink d17/d74/d86/lab 0 2024-03-29T08:07:19.189 INFO:tasks.workunit.client.1.smithi148.stdout:6/494: getdents d6/d9/d1e/d20/d30/d51 0 2024-03-29T08:07:19.192 INFO:tasks.workunit.client.1.smithi148.stdout:0/495: dwrite db/de/df/d11/d42/f6e [0,4194304] 0 2024-03-29T08:07:19.197 INFO:tasks.workunit.client.1.smithi148.stdout:6/495: mknod d6/d9/d1e/d27/c97 0 2024-03-29T08:07:19.199 INFO:tasks.workunit.client.1.smithi148.stdout:6/496: chown d6/d9/d1e/d27/l78 725155 1 2024-03-29T08:07:19.199 INFO:tasks.workunit.client.0.smithi037.stdout:0/258: creat d9/f50 x:0 0 0 2024-03-29T08:07:19.199 INFO:tasks.workunit.client.1.smithi148.stdout:6/497: fdatasync d6/d9/f16 0 2024-03-29T08:07:19.204 INFO:tasks.workunit.client.1.smithi148.stdout:8/526: creat d17/d74/d39/fac x:0 0 0 2024-03-29T08:07:19.206 INFO:tasks.workunit.client.1.smithi148.stdout:1/484: dread d3/d55/d7e/f67 [0,4194304] 0 2024-03-29T08:07:19.206 INFO:tasks.workunit.client.0.smithi037.stdout:5/516: dread d8/d18/f21 [0,4194304] 0 2024-03-29T08:07:19.207 INFO:tasks.workunit.client.0.smithi037.stdout:1/437: dwrite d2/f84 [0,4194304] 0 2024-03-29T08:07:19.211 INFO:tasks.workunit.client.1.smithi148.stdout:1/485: creat d3/d55/d7e/f94 x:0 0 0 2024-03-29T08:07:19.213 INFO:tasks.workunit.client.0.smithi037.stdout:1/438: mkdir d2/d6/dc/d58/d67/d95 0 2024-03-29T08:07:19.215 INFO:tasks.workunit.client.1.smithi148.stdout:1/486: fsync d3/de/d1d/d47/d6d/d84/f92 0 2024-03-29T08:07:19.215 INFO:tasks.workunit.client.1.smithi148.stdout:8/527: unlink d17/f1a 0 2024-03-29T08:07:19.217 INFO:tasks.workunit.client.1.smithi148.stdout:0/496: dwrite db/de/f70 [0,4194304] 0 2024-03-29T08:07:19.219 INFO:tasks.workunit.client.1.smithi148.stdout:1/487: fdatasync d3/de/d1d/d47/d6d/d72/f8a 0 2024-03-29T08:07:19.220 INFO:tasks.workunit.client.0.smithi037.stdout:1/439: unlink d2/d6/dc/l24 0 2024-03-29T08:07:19.222 INFO:tasks.workunit.client.0.smithi037.stdout:6/317: truncate d0/d5/d1e/f67 3169085 0 2024-03-29T08:07:19.225 INFO:tasks.workunit.client.0.smithi037.stdout:6/318: chown d0/d19/d42 388737687 1 2024-03-29T08:07:19.226 INFO:tasks.workunit.client.0.smithi037.stdout:1/440: rename d2/f31 to d2/d6/dc/d58/d67/d95/f96 0 2024-03-29T08:07:19.227 INFO:tasks.workunit.client.0.smithi037.stdout:0/259: write d9/da/f11 [3815132,94249] 0 2024-03-29T08:07:19.227 INFO:tasks.workunit.client.1.smithi148.stdout:6/498: dwrite d6/d9/d1e/f53 [0,4194304] 0 2024-03-29T08:07:19.232 INFO:tasks.workunit.client.0.smithi037.stdout:6/319: unlink d0/d20/f4d 0 2024-03-29T08:07:19.233 INFO:tasks.workunit.client.1.smithi148.stdout:9/498: sync 2024-03-29T08:07:19.234 INFO:tasks.workunit.client.1.smithi148.stdout:2/500: sync 2024-03-29T08:07:19.234 INFO:tasks.workunit.client.1.smithi148.stdout:5/491: sync 2024-03-29T08:07:19.236 INFO:tasks.workunit.client.0.smithi037.stdout:0/260: symlink d9/da/dd/d13/l51 0 2024-03-29T08:07:19.236 INFO:tasks.workunit.client.0.smithi037.stdout:1/441: chown d2/d6/dc/d18/d22/d4f/c65 4182628 1 2024-03-29T08:07:19.238 INFO:tasks.workunit.client.0.smithi037.stdout:1/442: chown d2/d6/dc/d58 6 1 2024-03-29T08:07:19.238 INFO:tasks.workunit.client.0.smithi037.stdout:6/320: dread - d0/d19/d34/f40 zero size 2024-03-29T08:07:19.241 INFO:tasks.workunit.client.1.smithi148.stdout:7/465: dwrite d8/d29/f54 [4194304,4194304] 0 2024-03-29T08:07:19.241 INFO:tasks.workunit.client.0.smithi037.stdout:0/261: truncate d9/da/dd/d13/f36 927739 0 2024-03-29T08:07:19.243 INFO:tasks.workunit.client.1.smithi148.stdout:5/492: fsync d0/f21 0 2024-03-29T08:07:19.245 INFO:tasks.workunit.client.0.smithi037.stdout:6/321: fdatasync d0/d5/d3f/f1b 0 2024-03-29T08:07:19.248 INFO:tasks.workunit.client.0.smithi037.stdout:1/443: creat d2/d6/dc/d5d/d62/f97 x:0 0 0 2024-03-29T08:07:19.248 INFO:tasks.workunit.client.1.smithi148.stdout:4/495: dread df/d2c/d51/d77/d17/f25 [4194304,4194304] 0 2024-03-29T08:07:19.250 INFO:tasks.workunit.client.0.smithi037.stdout:0/262: symlink d9/da/d1b/d22/l52 0 2024-03-29T08:07:19.251 INFO:tasks.workunit.client.1.smithi148.stdout:6/499: rmdir d6/d9/d1e/d36/d4d 39 2024-03-29T08:07:19.251 INFO:tasks.workunit.client.0.smithi037.stdout:0/263: fdatasync d9/f27 0 2024-03-29T08:07:19.252 INFO:tasks.workunit.client.0.smithi037.stdout:6/322: creat d0/d19/d34/f6b x:0 0 0 2024-03-29T08:07:19.253 INFO:tasks.workunit.client.1.smithi148.stdout:9/499: chown db/d22/d26/d54/d4d/d35/d4b/d58/f7c 5104 1 2024-03-29T08:07:19.256 INFO:tasks.workunit.client.0.smithi037.stdout:6/323: creat d0/d5/d1a/f6c x:0 0 0 2024-03-29T08:07:19.257 INFO:tasks.workunit.client.1.smithi148.stdout:3/531: dread de/f17 [0,4194304] 0 2024-03-29T08:07:19.257 INFO:tasks.workunit.client.0.smithi037.stdout:0/264: write d9/da/dd/d1c/f39 [786878,27441] 0 2024-03-29T08:07:19.261 INFO:tasks.workunit.client.1.smithi148.stdout:0/497: dwrite db/de/df/d2b/d3b/d47/d4c/d5a/f98 [0,4194304] 0 2024-03-29T08:07:19.261 INFO:tasks.workunit.client.1.smithi148.stdout:5/493: rename d0/d46/f8e to d0/db/d7b/d16/d2e/fa0 0 2024-03-29T08:07:19.262 INFO:tasks.workunit.client.0.smithi037.stdout:1/444: rename d2/f6c to d2/d6/dc/d18/d1d/d21/d38/f98 0 2024-03-29T08:07:19.262 INFO:tasks.workunit.client.0.smithi037.stdout:0/265: chown d9/da/d1b/d22/l4f 10153724 1 2024-03-29T08:07:19.262 INFO:tasks.workunit.client.1.smithi148.stdout:7/466: creat d8/d14/d45/d4d/d78/fa0 x:0 0 0 2024-03-29T08:07:19.263 INFO:tasks.workunit.client.0.smithi037.stdout:6/324: symlink d0/d20/d43/l6d 0 2024-03-29T08:07:19.263 INFO:tasks.workunit.client.1.smithi148.stdout:1/488: dwrite d3/de/d25/d35/d42/d23/f40 [0,4194304] 0 2024-03-29T08:07:19.264 INFO:tasks.workunit.client.1.smithi148.stdout:4/496: mkdir df/d2c/d80/da6 0 2024-03-29T08:07:19.265 INFO:tasks.workunit.client.0.smithi037.stdout:6/325: chown d0/d5/d1f/l22 9717 1 2024-03-29T08:07:19.266 INFO:tasks.workunit.client.1.smithi148.stdout:6/500: mkdir d6/d9/d1e/d22/d28/d98 0 2024-03-29T08:07:19.270 INFO:tasks.workunit.client.0.smithi037.stdout:1/445: unlink d2/d6/dc/d18/d22/d4f/d7c/l8f 0 2024-03-29T08:07:19.272 INFO:tasks.workunit.client.1.smithi148.stdout:1/489: mknod d3/d55/c95 0 2024-03-29T08:07:19.272 INFO:tasks.workunit.client.1.smithi148.stdout:0/498: rmdir db/de/df/d2b/d3b 39 2024-03-29T08:07:19.274 INFO:tasks.workunit.client.0.smithi037.stdout:6/326: mkdir d0/d19/d42/d6e 0 2024-03-29T08:07:19.274 INFO:tasks.workunit.client.1.smithi148.stdout:5/494: readlink d0/db/d1e/d36/d41/l38 0 2024-03-29T08:07:19.278 INFO:tasks.workunit.client.1.smithi148.stdout:3/532: mknod de/d24/d33/d39/d46/caf 0 2024-03-29T08:07:19.281 INFO:tasks.workunit.client.1.smithi148.stdout:4/497: fsync df/d2c/d51/d77/d17/f20 0 2024-03-29T08:07:19.282 INFO:tasks.workunit.client.1.smithi148.stdout:7/467: unlink d8/d14/f72 0 2024-03-29T08:07:19.282 INFO:tasks.workunit.client.1.smithi148.stdout:0/499: symlink db/de/la1 0 2024-03-29T08:07:19.286 INFO:tasks.workunit.client.0.smithi037.stdout:0/266: dwrite d9/fb [4194304,4194304] 0 2024-03-29T08:07:19.286 INFO:tasks.workunit.client.1.smithi148.stdout:2/501: dwrite d7/da/d33/d3c/f28 [4194304,4194304] 0 2024-03-29T08:07:19.287 INFO:tasks.workunit.client.1.smithi148.stdout:4/498: write df/d2d/f94 [785917,43913] 0 2024-03-29T08:07:19.289 INFO:tasks.workunit.client.1.smithi148.stdout:5/495: dread d0/db/d7b/f52 [0,4194304] 0 2024-03-29T08:07:19.289 INFO:tasks.workunit.client.0.smithi037.stdout:6/327: unlink d0/d5/l25 0 2024-03-29T08:07:19.289 INFO:tasks.workunit.client.1.smithi148.stdout:6/501: creat d6/d9/d1e/d22/d28/d98/f99 x:0 0 0 2024-03-29T08:07:19.290 INFO:tasks.workunit.client.1.smithi148.stdout:9/500: dwrite db/d22/d26/d54/d4d/d35/d4b/d58/f92 [0,4194304] 0 2024-03-29T08:07:19.292 INFO:tasks.workunit.client.0.smithi037.stdout:6/328: chown d0/d4f/f57 3 1 2024-03-29T08:07:19.292 INFO:tasks.workunit.client.1.smithi148.stdout:6/502: write d6/d9/f75 [4540286,86068] 0 2024-03-29T08:07:19.293 INFO:tasks.workunit.client.1.smithi148.stdout:5/496: dread - d0/db/d1e/d5b/d66/d72/d95/f9d zero size 2024-03-29T08:07:19.294 INFO:tasks.workunit.client.1.smithi148.stdout:9/501: read - db/d22/d3f/d61/d94/f9e zero size 2024-03-29T08:07:19.295 INFO:tasks.workunit.client.1.smithi148.stdout:3/533: rename de/d24/d33/d39/d46/d30/d37/c54 to de/d24/d81/cb0 0 2024-03-29T08:07:19.297 INFO:tasks.workunit.client.1.smithi148.stdout:2/502: mknod d7/da/d4c/c9c 0 2024-03-29T08:07:19.297 INFO:tasks.workunit.client.1.smithi148.stdout:0/500: write db/de/f62 [4778187,53739] 0 2024-03-29T08:07:19.305 INFO:tasks.workunit.client.0.smithi037.stdout:6/329: write d0/d19/d34/d4e/f54 [791534,80393] 0 2024-03-29T08:07:19.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:07:19 smithi148 ceph-mon[44992]: pgmap v41: 65 pgs: 65 active+clean; 1.6 GiB data, 6.2 GiB used, 530 GiB / 536 GiB avail; 39 MiB/s rd, 216 MiB/s wr, 345 op/s 2024-03-29T08:07:19.309 INFO:tasks.workunit.client.0.smithi037.stdout:6/330: write d0/d19/d34/d4e/f54 [1534576,76810] 0 2024-03-29T08:07:19.311 INFO:tasks.workunit.client.1.smithi148.stdout:3/534: read de/d24/d33/d39/d46/d30/d37/faa [21650,27556] 0 2024-03-29T08:07:19.315 INFO:tasks.workunit.client.1.smithi148.stdout:9/502: dread db/fa7 [0,4194304] 0 2024-03-29T08:07:19.315 INFO:tasks.workunit.client.1.smithi148.stdout:4/499: dwrite df/d2c/f86 [0,4194304] 0 2024-03-29T08:07:19.316 INFO:tasks.workunit.client.0.smithi037.stdout:1/446: rename d2/d6/dc/d18/d1d/d21/d38 to d2/d6/dc/d18/d1d/d21/d99 0 2024-03-29T08:07:19.317 INFO:tasks.workunit.client.1.smithi148.stdout:3/535: write f8 [5820669,3805] 0 2024-03-29T08:07:19.318 INFO:tasks.workunit.client.1.smithi148.stdout:5/497: rmdir d0/db/d1e/d5b/d66/d6c 0 2024-03-29T08:07:19.320 INFO:tasks.workunit.client.1.smithi148.stdout:4/500: readlink df/d2c/d51/d77/la2 0 2024-03-29T08:07:19.322 INFO:tasks.workunit.client.1.smithi148.stdout:3/536: write de/d24/d33/d39/d7f/fac [343201,104593] 0 2024-03-29T08:07:19.323 INFO:tasks.workunit.client.1.smithi148.stdout:5/498: symlink d0/db/d7b/la1 0 2024-03-29T08:07:19.330 INFO:tasks.workunit.client.1.smithi148.stdout:4/501: rename df/d2d to df/d2c/d51/d77/d26/d9b/da7 0 2024-03-29T08:07:19.330 INFO:tasks.workunit.client.0.smithi037.stdout:7/369: sync 2024-03-29T08:07:19.330 INFO:tasks.workunit.client.0.smithi037.stdout:3/361: sync 2024-03-29T08:07:19.331 INFO:tasks.workunit.client.1.smithi148.stdout:6/503: truncate d6/fe 1115282 0 2024-03-29T08:07:19.332 INFO:tasks.workunit.client.1.smithi148.stdout:6/504: stat d6/d9/d1e/d27/c2e 0 2024-03-29T08:07:19.334 INFO:tasks.workunit.client.1.smithi148.stdout:2/503: truncate d7/da/d33/d45/d4d/f47 5468524 0 2024-03-29T08:07:19.335 INFO:tasks.workunit.client.1.smithi148.stdout:5/499: mknod d0/d46/d86/ca2 0 2024-03-29T08:07:19.338 INFO:tasks.workunit.client.1.smithi148.stdout:9/503: write db/f5c [1028796,23950] 0 2024-03-29T08:07:19.339 INFO:tasks.workunit.client.1.smithi148.stdout:5/500: chown d0/f4b 37 1 2024-03-29T08:07:19.340 INFO:tasks.workunit.client.1.smithi148.stdout:2/504: read d7/da/fe [4243942,83738] 0 2024-03-29T08:07:19.342 INFO:tasks.workunit.client.1.smithi148.stdout:7/468: write d8/d38/f71 [2144033,41168] 0 2024-03-29T08:07:19.343 INFO:tasks.workunit.client.0.smithi037.stdout:0/267: mkdir d9/da/dd/d20/d38/d53 0 2024-03-29T08:07:19.345 INFO:tasks.workunit.client.1.smithi148.stdout:4/502: truncate df/d2c/d51/d77/d26/d34/d43/f59 3111892 0 2024-03-29T08:07:19.348 INFO:tasks.workunit.client.0.smithi037.stdout:6/331: creat d0/d5/d3f/f6f x:0 0 0 2024-03-29T08:07:19.348 INFO:tasks.workunit.client.1.smithi148.stdout:4/503: write df/d2c/d51/d77/d17/f20 [2029971,68968] 0 2024-03-29T08:07:19.348 INFO:tasks.workunit.client.1.smithi148.stdout:6/505: mknod d6/d9/d1e/d27/d48/d73/d7e/c9a 0 2024-03-29T08:07:19.349 INFO:tasks.workunit.client.0.smithi037.stdout:0/268: write d9/da/d1b/d22/d2c/f42 [152395,85341] 0 2024-03-29T08:07:19.351 INFO:tasks.workunit.client.0.smithi037.stdout:7/370: creat dd/d68/f78 x:0 0 0 2024-03-29T08:07:19.351 INFO:tasks.workunit.client.1.smithi148.stdout:4/504: write df/d2c/d51/d77/d17/f20 [4853848,128347] 0 2024-03-29T08:07:19.352 INFO:tasks.workunit.client.0.smithi037.stdout:3/362: symlink dd/d46/d3e/d47/l7b 0 2024-03-29T08:07:19.353 INFO:tasks.workunit.client.1.smithi148.stdout:3/537: truncate de/d3b/d49/f64 2182323 0 2024-03-29T08:07:19.353 INFO:tasks.workunit.client.1.smithi148.stdout:4/505: chown df/d71 89 1 2024-03-29T08:07:19.353 INFO:tasks.workunit.client.1.smithi148.stdout:5/501: mknod d0/db/d1e/d36/d7c/ca3 0 2024-03-29T08:07:19.357 INFO:tasks.workunit.client.1.smithi148.stdout:6/506: creat d6/d9/d1e/d27/f9b x:0 0 0 2024-03-29T08:07:19.357 INFO:tasks.workunit.client.1.smithi148.stdout:9/504: mkdir db/d10/d6d/d73/da8 0 2024-03-29T08:07:19.360 INFO:tasks.workunit.client.1.smithi148.stdout:9/505: write f4 [1628465,13321] 0 2024-03-29T08:07:19.361 INFO:tasks.workunit.client.1.smithi148.stdout:0/501: truncate db/de/df/d2b/f77 584351 0 2024-03-29T08:07:19.362 INFO:tasks.workunit.client.1.smithi148.stdout:4/506: dread df/d2c/d51/d77/d26/d34/d43/f4e [0,4194304] 0 2024-03-29T08:07:19.363 INFO:tasks.workunit.client.1.smithi148.stdout:5/502: fsync d0/db/d7b/d16/f27 0 2024-03-29T08:07:19.365 INFO:tasks.workunit.client.1.smithi148.stdout:2/505: creat d7/da/f9d x:0 0 0 2024-03-29T08:07:19.367 INFO:tasks.workunit.client.1.smithi148.stdout:6/507: creat d6/d9/d1e/d22/d28/d98/f9c x:0 0 0 2024-03-29T08:07:19.367 INFO:tasks.workunit.client.0.smithi037.stdout:3/363: rename dd/d15/d18/d30/f59 to dd/d46/d3e/f7c 0 2024-03-29T08:07:19.368 INFO:tasks.workunit.client.1.smithi148.stdout:5/503: truncate d0/db/d1e/f7e 497605 0 2024-03-29T08:07:19.369 INFO:tasks.workunit.client.0.smithi037.stdout:0/269: mkdir d9/da/dd/d54 0 2024-03-29T08:07:19.370 INFO:tasks.workunit.client.0.smithi037.stdout:1/447: link d2/d6/dc/d18/d1d/d21/d99/c89 d2/d6/dc/c9a 0 2024-03-29T08:07:19.370 INFO:tasks.workunit.client.1.smithi148.stdout:9/506: mknod db/d10/d2d/d4e/d5f/d6e/ca9 0 2024-03-29T08:07:19.371 INFO:tasks.workunit.client.1.smithi148.stdout:6/508: truncate d6/d9/d1e/d27/d48/f94 993901 0 2024-03-29T08:07:19.372 INFO:tasks.workunit.client.1.smithi148.stdout:4/507: chown df/d2c/d51/d77/f1d 11036 1 2024-03-29T08:07:19.373 INFO:tasks.workunit.client.0.smithi037.stdout:7/371: dread dd/f1f [0,4194304] 0 2024-03-29T08:07:19.373 INFO:tasks.workunit.client.0.smithi037.stdout:0/270: write d9/f27 [4614504,43707] 0 2024-03-29T08:07:19.375 INFO:tasks.workunit.client.0.smithi037.stdout:3/364: symlink dd/d5e/d7a/l7d 0 2024-03-29T08:07:19.376 INFO:tasks.workunit.client.1.smithi148.stdout:6/509: unlink d6/f14 0 2024-03-29T08:07:19.378 INFO:tasks.workunit.client.1.smithi148.stdout:3/538: getdents de/d24/d81 0 2024-03-29T08:07:19.379 INFO:tasks.workunit.client.1.smithi148.stdout:8/528: sync 2024-03-29T08:07:19.379 INFO:tasks.workunit.client.1.smithi148.stdout:1/490: sync 2024-03-29T08:07:19.379 INFO:tasks.workunit.client.0.smithi037.stdout:1/448: mknod d2/d6/dc/c9b 0 2024-03-29T08:07:19.379 INFO:tasks.workunit.client.1.smithi148.stdout:2/506: symlink d7/da/d33/d3c/d54/l9e 0 2024-03-29T08:07:19.380 INFO:tasks.workunit.client.0.smithi037.stdout:3/365: readlink dd/d15/d18/l20 0 2024-03-29T08:07:19.381 INFO:tasks.workunit.client.1.smithi148.stdout:4/508: creat df/d2c/d51/d77/d26/d9b/fa8 x:0 0 0 2024-03-29T08:07:19.385 INFO:tasks.workunit.client.0.smithi037.stdout:1/449: rename d2/d6 to d2/d6/dc/d18/d22/d4f/d7c/d9c 22 2024-03-29T08:07:19.388 INFO:tasks.workunit.client.0.smithi037.stdout:6/332: dwrite d0/f33 [0,4194304] 0 2024-03-29T08:07:19.388 INFO:tasks.workunit.client.1.smithi148.stdout:4/509: readlink df/d2c/d51/d77/d17/l19 0 2024-03-29T08:07:19.388 INFO:tasks.workunit.client.1.smithi148.stdout:6/510: read d6/d9/d1e/f5d [71441,114030] 0 2024-03-29T08:07:19.389 INFO:tasks.workunit.client.0.smithi037.stdout:7/372: creat dd/d21/d35/f79 x:0 0 0 2024-03-29T08:07:19.393 INFO:tasks.workunit.client.1.smithi148.stdout:0/502: dread db/dc/f59 [0,4194304] 0 2024-03-29T08:07:19.396 INFO:tasks.workunit.client.0.smithi037.stdout:3/366: rename dd/f11 to dd/d46/d3e/f7e 0 2024-03-29T08:07:19.397 INFO:tasks.workunit.client.0.smithi037.stdout:7/373: symlink dd/d1d/l7a 0 2024-03-29T08:07:19.398 INFO:tasks.workunit.client.1.smithi148.stdout:0/503: stat db/dc/d78/f8b 0 2024-03-29T08:07:19.399 INFO:tasks.workunit.client.1.smithi148.stdout:7/469: truncate f4 3653526 0 2024-03-29T08:07:19.399 INFO:tasks.workunit.client.0.smithi037.stdout:1/450: mknod d2/d6/d13/c9d 0 2024-03-29T08:07:19.400 INFO:tasks.workunit.client.1.smithi148.stdout:5/504: dwrite d0/db/d1e/d36/d41/f9f [0,4194304] 0 2024-03-29T08:07:19.400 INFO:tasks.workunit.client.1.smithi148.stdout:0/504: stat db/de/l74 0 2024-03-29T08:07:19.402 INFO:tasks.workunit.client.1.smithi148.stdout:4/510: truncate df/d2c/d51/d77/f83 183119 0 2024-03-29T08:07:19.404 INFO:tasks.workunit.client.0.smithi037.stdout:3/367: creat dd/d46/d3e/d47/f7f x:0 0 0 2024-03-29T08:07:19.410 INFO:tasks.workunit.client.1.smithi148.stdout:7/470: fdatasync d8/d38/d3c/f96 0 2024-03-29T08:07:19.411 INFO:tasks.workunit.client.0.smithi037.stdout:7/374: rename dd/l13 to dd/d1d/d49/l7b 0 2024-03-29T08:07:19.411 INFO:tasks.workunit.client.0.smithi037.stdout:2/251: sync 2024-03-29T08:07:19.411 INFO:tasks.workunit.client.0.smithi037.stdout:8/414: sync 2024-03-29T08:07:19.411 INFO:tasks.workunit.client.0.smithi037.stdout:4/378: sync 2024-03-29T08:07:19.411 INFO:tasks.workunit.client.0.smithi037.stdout:9/556: sync 2024-03-29T08:07:19.412 INFO:tasks.workunit.client.1.smithi148.stdout:5/505: mknod d0/d46/d9e/ca4 0 2024-03-29T08:07:19.412 INFO:tasks.workunit.client.1.smithi148.stdout:3/539: write de/d24/f56 [1937310,127745] 0 2024-03-29T08:07:19.412 INFO:tasks.workunit.client.0.smithi037.stdout:3/368: read - dd/d15/d18/f2f zero size 2024-03-29T08:07:19.413 INFO:tasks.workunit.client.1.smithi148.stdout:9/507: dread db/d20/f1b [0,4194304] 0 2024-03-29T08:07:19.414 INFO:tasks.workunit.client.1.smithi148.stdout:9/508: fdatasync db/d20/f70 0 2024-03-29T08:07:19.416 INFO:tasks.workunit.client.1.smithi148.stdout:7/471: chown d8/d38/d6a/d91 5592 1 2024-03-29T08:07:19.416 INFO:tasks.workunit.client.1.smithi148.stdout:0/505: write db/de/df/f95 [443383,61137] 0 2024-03-29T08:07:19.417 INFO:tasks.workunit.client.0.smithi037.stdout:2/252: dread - d0/d20/d22/f39 zero size 2024-03-29T08:07:19.417 INFO:tasks.workunit.client.1.smithi148.stdout:2/507: truncate d7/d24/d4b/f87 754380 0 2024-03-29T08:07:19.417 INFO:tasks.workunit.client.1.smithi148.stdout:1/491: dread d3/de/d25/d35/d42/d23/f40 [0,4194304] 0 2024-03-29T08:07:19.418 INFO:tasks.workunit.client.1.smithi148.stdout:8/529: link d17/d1d/c93 d17/d74/d39/d6f/d64/da5/cad 0 2024-03-29T08:07:19.419 INFO:tasks.workunit.client.1.smithi148.stdout:6/511: write d6/d9/d1e/d27/d48/d73/d7e/f92 [1549232,98757] 0 2024-03-29T08:07:19.421 INFO:tasks.workunit.client.1.smithi148.stdout:1/492: readlink d3/l17 0 2024-03-29T08:07:19.421 INFO:tasks.workunit.client.0.smithi037.stdout:4/379: stat d3/d4/d15/d1d/db/d45/l47 0 2024-03-29T08:07:19.423 INFO:tasks.workunit.client.1.smithi148.stdout:0/506: stat db/de/df/d11/d42/da0 0 2024-03-29T08:07:19.425 INFO:tasks.workunit.client.1.smithi148.stdout:1/493: fsync d3/f5d 0 2024-03-29T08:07:19.425 INFO:tasks.workunit.client.1.smithi148.stdout:2/508: chown d7/da/d33/d3c/f42 215 1 2024-03-29T08:07:19.425 INFO:tasks.workunit.client.1.smithi148.stdout:9/509: symlink db/d10/d2d/d4e/d5f/d6e/laa 0 2024-03-29T08:07:19.427 INFO:tasks.workunit.client.1.smithi148.stdout:9/510: chown db 80663 1 2024-03-29T08:07:19.428 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:07:19 smithi037 ceph-mon[35162]: pgmap v41: 65 pgs: 65 active+clean; 1.6 GiB data, 6.2 GiB used, 530 GiB / 536 GiB avail; 39 MiB/s rd, 216 MiB/s wr, 345 op/s 2024-03-29T08:07:19.428 INFO:tasks.workunit.client.0.smithi037.stdout:4/380: write d3/d4/f46 [3983886,27612] 0 2024-03-29T08:07:19.430 INFO:tasks.workunit.client.0.smithi037.stdout:8/415: symlink d5/d10/d19/d3d/d4a/d4c/l86 0 2024-03-29T08:07:19.431 INFO:tasks.workunit.client.1.smithi148.stdout:8/530: read d17/f68 [161791,7225] 0 2024-03-29T08:07:19.432 INFO:tasks.workunit.client.1.smithi148.stdout:1/494: chown d3/de/d1d/d47/d6d/c81 421577883 1 2024-03-29T08:07:19.432 INFO:tasks.workunit.client.0.smithi037.stdout:6/333: dwrite d0/d5/d1a/f5d [0,4194304] 0 2024-03-29T08:07:19.432 INFO:tasks.workunit.client.0.smithi037.stdout:4/381: dread - d3/d54/f5b zero size 2024-03-29T08:07:19.434 INFO:tasks.workunit.client.1.smithi148.stdout:5/506: rename d0/db/d1e/d36/c4d to d0/db/d1e/d35/d42/d96/ca5 0 2024-03-29T08:07:19.436 INFO:tasks.workunit.client.1.smithi148.stdout:5/507: write d0/f4b [3992550,31473] 0 2024-03-29T08:07:19.437 INFO:tasks.workunit.client.1.smithi148.stdout:4/511: dwrite df/d2c/d51/d77/d26/f3b [0,4194304] 0 2024-03-29T08:07:19.437 INFO:tasks.workunit.client.0.smithi037.stdout:0/271: dread d9/da/dd/d13/f21 [0,4194304] 0 2024-03-29T08:07:19.438 INFO:tasks.workunit.client.1.smithi148.stdout:2/509: creat d7/da/d33/d45/d4d/d82/f9f x:0 0 0 2024-03-29T08:07:19.440 INFO:tasks.workunit.client.1.smithi148.stdout:4/512: chown df/d71/f7b 61400632 1 2024-03-29T08:07:19.441 INFO:tasks.workunit.client.1.smithi148.stdout:5/508: fsync d0/db/d1e/d5b/d66/d73/f8b 0 2024-03-29T08:07:19.443 INFO:tasks.workunit.client.0.smithi037.stdout:2/253: symlink d0/d53/l57 0 2024-03-29T08:07:19.445 INFO:tasks.workunit.client.1.smithi148.stdout:9/511: rename db/d22/d3f/d9a to db/d22/d26/d54/d52/dab 0 2024-03-29T08:07:19.447 INFO:tasks.workunit.client.1.smithi148.stdout:2/510: mknod d7/da/d33/d45/d4d/d2e/ca0 0 2024-03-29T08:07:19.447 INFO:tasks.workunit.client.0.smithi037.stdout:6/334: unlink d0/d5/f9 0 2024-03-29T08:07:19.451 INFO:tasks.workunit.client.1.smithi148.stdout:8/531: dwrite f15 [0,4194304] 0 2024-03-29T08:07:19.451 INFO:tasks.workunit.client.1.smithi148.stdout:3/540: write de/d24/d33/d39/d46/f23 [504157,107768] 0 2024-03-29T08:07:19.451 INFO:tasks.workunit.client.1.smithi148.stdout:5/509: creat d0/d46/d9e/fa6 x:0 0 0 2024-03-29T08:07:19.452 INFO:tasks.workunit.client.0.smithi037.stdout:9/557: dwrite d1/d4/d19/d1e/d25/d16/f9e [0,4194304] 0 2024-03-29T08:07:19.454 INFO:tasks.workunit.client.1.smithi148.stdout:4/513: symlink df/d2c/d51/d77/d26/d34/d3e/la9 0 2024-03-29T08:07:19.455 INFO:tasks.workunit.client.1.smithi148.stdout:8/532: write d17/d74/f5a [2184343,3279] 0 2024-03-29T08:07:19.456 INFO:tasks.workunit.client.1.smithi148.stdout:4/514: fdatasync df/d2c/f86 0 2024-03-29T08:07:19.456 INFO:tasks.workunit.client.1.smithi148.stdout:6/512: rename d6/d9/d1e/d22 to d6/d9/d1e/d36/d4d/d9d 0 2024-03-29T08:07:19.457 INFO:tasks.workunit.client.0.smithi037.stdout:5/517: sync 2024-03-29T08:07:19.457 INFO:tasks.workunit.client.0.smithi037.stdout:8/416: mkdir d5/d10/d2d/d34/d50/d87 0 2024-03-29T08:07:19.463 INFO:tasks.workunit.client.1.smithi148.stdout:3/541: rmdir de/d24/d33/d39/d7f 39 2024-03-29T08:07:19.466 INFO:tasks.workunit.client.1.smithi148.stdout:5/510: creat d0/db/d1e/fa7 x:0 0 0 2024-03-29T08:07:19.470 INFO:tasks.workunit.client.1.smithi148.stdout:8/533: dread d17/d74/f48 [4194304,4194304] 0 2024-03-29T08:07:19.470 INFO:tasks.workunit.client.0.smithi037.stdout:6/335: dread d0/d20/f46 [0,4194304] 0 2024-03-29T08:07:19.470 INFO:tasks.workunit.client.0.smithi037.stdout:6/336: chown d0/d19 1028 1 2024-03-29T08:07:19.471 INFO:tasks.workunit.client.1.smithi148.stdout:9/512: rename db/d22/d26/d54/d4d/d35/d4b/d58/f7c to db/d10/d2d/d4e/da6/fac 0 2024-03-29T08:07:19.474 INFO:tasks.workunit.client.1.smithi148.stdout:1/495: write d3/d18/d4d/f63 [1061619,129234] 0 2024-03-29T08:07:19.474 INFO:tasks.workunit.client.0.smithi037.stdout:0/272: mkdir d9/da/dd/d20/d55 0 2024-03-29T08:07:19.475 INFO:tasks.workunit.client.1.smithi148.stdout:2/511: creat d7/d24/d4b/d83/fa1 x:0 0 0 2024-03-29T08:07:19.476 INFO:tasks.workunit.client.1.smithi148.stdout:5/511: dread d0/db/d7b/d16/f8f [4194304,4194304] 0 2024-03-29T08:07:19.477 INFO:tasks.workunit.client.0.smithi037.stdout:2/254: rmdir d0/d53 39 2024-03-29T08:07:19.483 INFO:tasks.workunit.client.0.smithi037.stdout:4/382: dwrite d3/d4/d15/d1d/f3f [0,4194304] 0 2024-03-29T08:07:19.483 INFO:tasks.workunit.client.1.smithi148.stdout:9/513: mknod db/d22/d26/d54/d4d/d35/d4b/d5a/cad 0 2024-03-29T08:07:19.483 INFO:tasks.workunit.client.0.smithi037.stdout:5/518: write d8/fa0 [2716220,66571] 0 2024-03-29T08:07:19.486 INFO:tasks.workunit.client.1.smithi148.stdout:5/512: mkdir d0/db/d1e/d6e/da8 0 2024-03-29T08:07:19.487 INFO:tasks.workunit.client.1.smithi148.stdout:6/513: link c3 d6/d9/d1e/d36/d4d/c9e 0 2024-03-29T08:07:19.488 INFO:tasks.workunit.client.0.smithi037.stdout:3/369: write dd/d5e/f65 [384765,89061] 0 2024-03-29T08:07:19.492 INFO:tasks.workunit.client.1.smithi148.stdout:6/514: write d6/d9/d1e/f53 [2612453,130920] 0 2024-03-29T08:07:19.493 INFO:tasks.workunit.client.1.smithi148.stdout:8/534: rmdir d17/d74 39 2024-03-29T08:07:19.493 INFO:tasks.workunit.client.0.smithi037.stdout:7/375: write dd/d1d/d34/f66 [256082,76574] 0 2024-03-29T08:07:19.497 INFO:tasks.workunit.client.1.smithi148.stdout:3/542: getdents de/d24/d33/d39/d46/d30 0 2024-03-29T08:07:19.497 INFO:tasks.workunit.client.1.smithi148.stdout:2/512: dwrite d7/da/f7f [0,4194304] 0 2024-03-29T08:07:19.497 INFO:tasks.workunit.client.0.smithi037.stdout:5/519: write d8/d18/d24/fa6 [81341,18016] 0 2024-03-29T08:07:19.498 INFO:tasks.workunit.client.0.smithi037.stdout:3/370: write dd/d46/f2c [928911,22457] 0 2024-03-29T08:07:19.499 INFO:tasks.workunit.client.0.smithi037.stdout:2/255: mknod d0/d23/d44/c58 0 2024-03-29T08:07:19.501 INFO:tasks.workunit.client.1.smithi148.stdout:0/507: truncate db/de/f70 1160937 0 2024-03-29T08:07:19.502 INFO:tasks.workunit.client.1.smithi148.stdout:6/515: fsync d6/d9/d1e/d20/d30/f33 0 2024-03-29T08:07:19.503 INFO:tasks.workunit.client.1.smithi148.stdout:0/508: readlink db/de/l74 0 2024-03-29T08:07:19.507 INFO:tasks.workunit.client.0.smithi037.stdout:8/417: dwrite d5/f80 [0,4194304] 0 2024-03-29T08:07:19.511 INFO:tasks.workunit.client.0.smithi037.stdout:0/273: creat d9/da/dd/d54/f56 x:0 0 0 2024-03-29T08:07:19.513 INFO:tasks.workunit.client.0.smithi037.stdout:8/418: truncate d5/f5b 1490687 0 2024-03-29T08:07:19.515 INFO:tasks.workunit.client.0.smithi037.stdout:9/558: creat d1/d1f/d3a/fc0 x:0 0 0 2024-03-29T08:07:19.516 INFO:tasks.workunit.client.1.smithi148.stdout:2/513: write d7/da/f49 [3218429,21870] 0 2024-03-29T08:07:19.517 INFO:tasks.workunit.client.1.smithi148.stdout:6/516: mkdir d6/d9/d1e/d27/d48/d9f 0 2024-03-29T08:07:19.520 INFO:tasks.workunit.client.1.smithi148.stdout:0/509: dread db/f97 [0,4194304] 0 2024-03-29T08:07:19.522 INFO:tasks.workunit.client.1.smithi148.stdout:6/517: fdatasync d6/d9/d1e/d27/f7f 0 2024-03-29T08:07:19.522 INFO:tasks.workunit.client.1.smithi148.stdout:9/514: dwrite db/d10/d2d/d4e/d5f/d6e/d7a/f7e [0,4194304] 0 2024-03-29T08:07:19.523 INFO:tasks.workunit.client.0.smithi037.stdout:2/256: write d0/f21 [4598948,452] 0 2024-03-29T08:07:19.524 INFO:tasks.workunit.client.0.smithi037.stdout:7/376: mknod dd/d1d/d34/d54/c7c 0 2024-03-29T08:07:19.525 INFO:tasks.workunit.client.1.smithi148.stdout:1/496: dread d3/f7 [0,4194304] 0 2024-03-29T08:07:19.529 INFO:tasks.workunit.client.1.smithi148.stdout:2/514: truncate d7/f5a 111323 0 2024-03-29T08:07:19.531 INFO:tasks.workunit.client.0.smithi037.stdout:4/383: dwrite d3/d4/d15/f63 [0,4194304] 0 2024-03-29T08:07:19.531 INFO:tasks.workunit.client.0.smithi037.stdout:9/559: stat d1/d4/d19/d1e/d4a/d6f/d96/fa4 0 2024-03-29T08:07:19.533 INFO:tasks.workunit.client.1.smithi148.stdout:1/497: fsync d3/de/d6e/f88 0 2024-03-29T08:07:19.534 INFO:tasks.workunit.client.1.smithi148.stdout:4/515: dwrite df/d2c/d51/d77/d26/d34/d43/f59 [0,4194304] 0 2024-03-29T08:07:19.535 INFO:tasks.workunit.client.1.smithi148.stdout:2/515: write d7/da/d4c/f79 [139083,19697] 0 2024-03-29T08:07:19.535 INFO:tasks.workunit.client.0.smithi037.stdout:8/419: dread d5/f48 [0,4194304] 0 2024-03-29T08:07:19.536 INFO:tasks.workunit.client.1.smithi148.stdout:3/543: dread de/f2c [0,4194304] 0 2024-03-29T08:07:19.538 INFO:tasks.workunit.client.1.smithi148.stdout:6/518: creat d6/d9/d1e/d27/d48/d9f/fa0 x:0 0 0 2024-03-29T08:07:19.539 INFO:tasks.workunit.client.1.smithi148.stdout:2/516: chown d7/da/d33/d3c/d31 980509 1 2024-03-29T08:07:19.541 INFO:tasks.workunit.client.0.smithi037.stdout:7/377: write dd/d1d/d49/f5a [1898959,119445] 0 2024-03-29T08:07:19.542 INFO:tasks.workunit.client.1.smithi148.stdout:2/517: dread - d7/d24/f36 zero size 2024-03-29T08:07:19.546 INFO:tasks.workunit.client.0.smithi037.stdout:4/384: dread - d3/d4/d15/d1d/db/d3d/f56 zero size 2024-03-29T08:07:19.551 INFO:tasks.workunit.client.1.smithi148.stdout:1/498: symlink d3/de/d4f/l96 0 2024-03-29T08:07:19.551 INFO:tasks.workunit.client.1.smithi148.stdout:9/515: dwrite db/d22/d26/f2a [0,4194304] 0 2024-03-29T08:07:19.554 INFO:tasks.workunit.client.1.smithi148.stdout:1/499: dread - d3/d18/d4d/f74 zero size 2024-03-29T08:07:19.555 INFO:tasks.workunit.client.0.smithi037.stdout:1/451: dread d2/d6/dc/d18/d1d/d21/f60 [0,4194304] 0 2024-03-29T08:07:19.555 INFO:tasks.workunit.client.1.smithi148.stdout:6/519: mknod d6/d9/d1e/d27/d48/d73/d7e/ca1 0 2024-03-29T08:07:19.563 INFO:tasks.workunit.client.1.smithi148.stdout:3/544: symlink de/d24/d33/d39/d7f/lb1 0 2024-03-29T08:07:19.567 INFO:tasks.workunit.client.1.smithi148.stdout:2/518: mknod d7/da/d4c/d9a/ca2 0 2024-03-29T08:07:19.569 INFO:tasks.workunit.client.0.smithi037.stdout:6/337: truncate d0/d19/d34/d5a/f39 3273566 0 2024-03-29T08:07:19.573 INFO:tasks.workunit.client.1.smithi148.stdout:5/513: dread d0/db/d1e/d36/d41/f63 [0,4194304] 0 2024-03-29T08:07:19.573 INFO:tasks.workunit.client.1.smithi148.stdout:1/500: dwrite d3/de/f91 [0,4194304] 0 2024-03-29T08:07:19.574 INFO:tasks.workunit.client.1.smithi148.stdout:2/519: mkdir d7/da/d52/da3 0 2024-03-29T08:07:19.577 INFO:tasks.workunit.client.1.smithi148.stdout:2/520: chown d7/da/d33/d45/d4d/d2e/c71 14069 1 2024-03-29T08:07:19.577 INFO:tasks.workunit.client.0.smithi037.stdout:6/338: write d0/d5/d1a/f5d [1350316,114970] 0 2024-03-29T08:07:19.577 INFO:tasks.workunit.client.1.smithi148.stdout:1/501: write d3/d18/d71/f8e [506871,35400] 0 2024-03-29T08:07:19.580 INFO:tasks.workunit.client.1.smithi148.stdout:8/535: rename d17/d74/d39/fac to d17/fae 0 2024-03-29T08:07:19.581 INFO:tasks.workunit.client.1.smithi148.stdout:3/545: dread de/d3b/d49/f90 [0,4194304] 0 2024-03-29T08:07:19.582 INFO:tasks.workunit.client.0.smithi037.stdout:7/378: dwrite dd/d21/d35/f79 [0,4194304] 0 2024-03-29T08:07:19.587 INFO:tasks.workunit.client.1.smithi148.stdout:1/502: write d3/de/d25/d35/d42/f15 [5949,26064] 0 2024-03-29T08:07:19.590 INFO:tasks.workunit.client.1.smithi148.stdout:3/546: fdatasync de/d3b/f60 0 2024-03-29T08:07:19.595 INFO:tasks.workunit.client.1.smithi148.stdout:6/520: rename d6/d9/d1e/d27/l61 to d6/d9/d1e/d20/d30/la2 0 2024-03-29T08:07:19.595 INFO:tasks.workunit.client.1.smithi148.stdout:1/503: mkdir d3/de/d1d/d97 0 2024-03-29T08:07:19.597 INFO:tasks.workunit.client.1.smithi148.stdout:0/510: link db/de/d24/l32 db/de/df/d2b/d3b/d47/d4c/la2 0 2024-03-29T08:07:19.599 INFO:tasks.workunit.client.1.smithi148.stdout:7/472: sync 2024-03-29T08:07:19.600 INFO:tasks.workunit.client.1.smithi148.stdout:2/521: rename d7/d86 to d7/d8c/da4 0 2024-03-29T08:07:19.602 INFO:tasks.workunit.client.0.smithi037.stdout:0/274: link d9/da/f11 d9/da/d1b/d22/d2c/f57 0 2024-03-29T08:07:19.607 INFO:tasks.workunit.client.0.smithi037.stdout:2/257: getdents d0/d20/d22/d55 0 2024-03-29T08:07:19.610 INFO:tasks.workunit.client.0.smithi037.stdout:5/520: mknod d8/d22/d4f/d70/d79/d7a/d85/d90/cac 0 2024-03-29T08:07:19.612 INFO:tasks.workunit.client.1.smithi148.stdout:8/536: getdents d17/d18/d26/d4f 0 2024-03-29T08:07:19.613 INFO:tasks.workunit.client.0.smithi037.stdout:4/385: mkdir d3/d80 0 2024-03-29T08:07:19.616 INFO:tasks.workunit.client.0.smithi037.stdout:5/521: truncate d8/d18/d24/fa6 915577 0 2024-03-29T08:07:19.617 INFO:tasks.workunit.client.1.smithi148.stdout:2/522: read d7/da/d33/d45/d4d/f38 [391361,57197] 0 2024-03-29T08:07:19.618 INFO:tasks.workunit.client.1.smithi148.stdout:0/511: rmdir db/de/df/d2b/d3b/d47/d4c/d5a/d7a 39 2024-03-29T08:07:19.619 INFO:tasks.workunit.client.0.smithi037.stdout:5/522: chown d8/d22/d4f/f5e 24 1 2024-03-29T08:07:19.621 INFO:tasks.workunit.client.1.smithi148.stdout:4/516: dwrite df/d2c/d51/d77/d17/f36 [0,4194304] 0 2024-03-29T08:07:19.621 INFO:tasks.workunit.client.0.smithi037.stdout:8/420: rename d5/d10/d2d/d34/d50/l82 to d5/d10/d2d/d34/d50/d83/d46/d4e/l88 0 2024-03-29T08:07:19.624 INFO:tasks.workunit.client.0.smithi037.stdout:8/421: write d5/f17 [1602080,94442] 0 2024-03-29T08:07:19.624 INFO:tasks.workunit.client.1.smithi148.stdout:3/547: rename de/d24/d33/d39/d46/l1e to de/d24/d33/d39/d46/d30/d37/d62/d63/lb2 0 2024-03-29T08:07:19.625 INFO:tasks.workunit.client.0.smithi037.stdout:3/371: write dd/d46/d2d/f42 [9434,81108] 0 2024-03-29T08:07:19.628 INFO:tasks.workunit.client.0.smithi037.stdout:8/422: stat d5/d10/f23 0 2024-03-29T08:07:19.629 INFO:tasks.workunit.client.1.smithi148.stdout:7/473: dread d8/d29/d44/d50/f8b [0,4194304] 0 2024-03-29T08:07:19.630 INFO:tasks.workunit.client.1.smithi148.stdout:4/517: mkdir df/d2c/d80/daa 0 2024-03-29T08:07:19.630 INFO:tasks.workunit.client.1.smithi148.stdout:1/504: dwrite d3/d18/d4d/f74 [0,4194304] 0 2024-03-29T08:07:19.637 INFO:tasks.workunit.client.1.smithi148.stdout:3/548: mknod de/d24/d33/d39/d71/cb3 0 2024-03-29T08:07:19.638 INFO:tasks.workunit.client.0.smithi037.stdout:6/339: creat d0/d6a/f70 x:0 0 0 2024-03-29T08:07:19.640 INFO:tasks.workunit.client.1.smithi148.stdout:9/516: dread f4 [0,4194304] 0 2024-03-29T08:07:19.642 INFO:tasks.workunit.client.1.smithi148.stdout:4/518: symlink df/d2c/d51/d77/d26/d9b/lab 0 2024-03-29T08:07:19.643 INFO:tasks.workunit.client.1.smithi148.stdout:3/549: mknod de/d24/d33/d39/d46/d30/d37/d79/cb4 0 2024-03-29T08:07:19.645 INFO:tasks.workunit.client.1.smithi148.stdout:4/519: fsync df/d2c/d51/d77/f83 0 2024-03-29T08:07:19.645 INFO:tasks.workunit.client.0.smithi037.stdout:2/258: chown d0/l42 2686729 1 2024-03-29T08:07:19.646 INFO:tasks.workunit.client.1.smithi148.stdout:6/521: dread d6/f70 [0,4194304] 0 2024-03-29T08:07:19.647 INFO:tasks.workunit.client.0.smithi037.stdout:2/259: dread - d0/d23/d44/f46 zero size 2024-03-29T08:07:19.648 INFO:tasks.workunit.client.1.smithi148.stdout:3/550: readlink de/d24/d33/d39/d46/d8d/l9e 0 2024-03-29T08:07:19.648 INFO:tasks.workunit.client.1.smithi148.stdout:7/474: mknod d8/d29/d44/d50/d94/ca1 0 2024-03-29T08:07:19.649 INFO:tasks.workunit.client.1.smithi148.stdout:6/522: write d6/fd [458217,28008] 0 2024-03-29T08:07:19.651 INFO:tasks.workunit.client.0.smithi037.stdout:9/560: creat d1/d4/d19/d1e/d25/fc1 x:0 0 0 2024-03-29T08:07:19.651 INFO:tasks.workunit.client.1.smithi148.stdout:6/523: fdatasync f5 0 2024-03-29T08:07:19.656 INFO:tasks.workunit.client.1.smithi148.stdout:5/514: dread d0/f12 [0,4194304] 0 2024-03-29T08:07:19.656 INFO:tasks.workunit.client.1.smithi148.stdout:6/524: truncate d6/d9/d1e/f8a 443979 0 2024-03-29T08:07:19.657 INFO:tasks.workunit.client.1.smithi148.stdout:3/551: creat de/d24/d33/d39/d46/d30/d37/fb5 x:0 0 0 2024-03-29T08:07:19.658 INFO:tasks.workunit.client.1.smithi148.stdout:0/512: mkdir db/de/df/d11/d2d/da3 0 2024-03-29T08:07:19.658 INFO:tasks.workunit.client.1.smithi148.stdout:3/552: chown de/d3b 132589084 1 2024-03-29T08:07:19.660 INFO:tasks.workunit.client.1.smithi148.stdout:4/520: link l9 df/d2c/d51/d77/d26/d34/d3e/lac 0 2024-03-29T08:07:19.661 INFO:tasks.workunit.client.0.smithi037.stdout:1/452: dwrite d2/d6/dc/d18/d1d/d21/d99/f98 [0,4194304] 0 2024-03-29T08:07:19.662 INFO:tasks.workunit.client.1.smithi148.stdout:9/517: write db/d22/f89 [532005,89598] 0 2024-03-29T08:07:19.662 INFO:tasks.workunit.client.1.smithi148.stdout:6/525: chown d6/d9/d1e/d36/d4d/d9d/d28/f38 181981281 1 2024-03-29T08:07:19.664 INFO:tasks.workunit.client.1.smithi148.stdout:9/518: chown db/d22/d26/d54/d4d 498262542 1 2024-03-29T08:07:19.666 INFO:tasks.workunit.client.1.smithi148.stdout:9/519: chown db/d10/d2d/d4e 75047476 1 2024-03-29T08:07:19.666 INFO:tasks.workunit.client.1.smithi148.stdout:3/553: chown de/d24/d33/d39/d46/l5b 24012 1 2024-03-29T08:07:19.666 INFO:tasks.workunit.client.1.smithi148.stdout:6/526: fsync d6/d9/d1e/d27/f90 0 2024-03-29T08:07:19.667 INFO:tasks.workunit.client.1.smithi148.stdout:7/475: rmdir d8/d29/d44/d49 39 2024-03-29T08:07:19.667 INFO:tasks.workunit.client.1.smithi148.stdout:2/523: dread d7/d24/f5d [0,4194304] 0 2024-03-29T08:07:19.667 INFO:tasks.workunit.client.1.smithi148.stdout:0/513: mknod db/de/df/d2b/d3b/d47/d7c/ca4 0 2024-03-29T08:07:19.668 INFO:tasks.workunit.client.1.smithi148.stdout:8/537: link d17/d74/d39/d6f/d64/da5/cad d17/d74/d39/d6f/d64/caf 0 2024-03-29T08:07:19.669 INFO:tasks.workunit.client.1.smithi148.stdout:4/521: rename df/d2c/d51/d77/d26/d9b/da7/d7c/l7f to df/d2c/d80/daa/lad 0 2024-03-29T08:07:19.671 INFO:tasks.workunit.client.0.smithi037.stdout:8/423: symlink d5/d10/d2d/d34/d50/d83/l89 0 2024-03-29T08:07:19.674 INFO:tasks.workunit.client.0.smithi037.stdout:7/379: creat dd/f7d x:0 0 0 2024-03-29T08:07:19.676 INFO:tasks.workunit.client.1.smithi148.stdout:3/554: stat de/d24/d33/d39/d46/c18 0 2024-03-29T08:07:19.677 INFO:tasks.workunit.client.1.smithi148.stdout:7/476: symlink d8/d38/d6a/d91/la2 0 2024-03-29T08:07:19.677 INFO:tasks.workunit.client.0.smithi037.stdout:1/453: rmdir d2/d6/dc/d18/d1d 39 2024-03-29T08:07:19.677 INFO:tasks.workunit.client.0.smithi037.stdout:8/424: chown d5/d10/d45/c59 19 1 2024-03-29T08:07:19.678 INFO:tasks.workunit.client.1.smithi148.stdout:8/538: chown d17/d74/d39/d6f/f45 5376 1 2024-03-29T08:07:19.679 INFO:tasks.workunit.client.0.smithi037.stdout:6/340: mkdir d0/d5/d1e/d44/d71 0 2024-03-29T08:07:19.680 INFO:tasks.workunit.client.1.smithi148.stdout:2/524: fdatasync d7/da/d33/d45/d4d/f38 0 2024-03-29T08:07:19.681 INFO:tasks.workunit.client.1.smithi148.stdout:6/527: rename d6/f1a to d6/d9/d1e/d27/d48/d9f/fa3 0 2024-03-29T08:07:19.682 INFO:tasks.workunit.client.0.smithi037.stdout:5/523: creat d8/d18/d24/fad x:0 0 0 2024-03-29T08:07:19.682 INFO:tasks.workunit.client.1.smithi148.stdout:0/514: fsync db/de/df/d2b/d3b/d47/d4c/d5a/d8d/f99 0 2024-03-29T08:07:19.685 INFO:tasks.workunit.client.0.smithi037.stdout:3/372: fdatasync dd/d46/d2d/d3a/f52 0 2024-03-29T08:07:19.687 INFO:tasks.workunit.client.1.smithi148.stdout:3/555: chown de/d24/d33/d39/d46/d70/l8c 259081 1 2024-03-29T08:07:19.687 INFO:tasks.workunit.client.1.smithi148.stdout:0/515: fdatasync db/de/df/d11/d42/f52 0 2024-03-29T08:07:19.687 INFO:tasks.workunit.client.1.smithi148.stdout:9/520: creat db/d22/d26/d54/fae x:0 0 0 2024-03-29T08:07:19.688 INFO:tasks.workunit.client.0.smithi037.stdout:2/260: dwrite d0/d20/d22/f48 [0,4194304] 0 2024-03-29T08:07:19.688 INFO:tasks.workunit.client.1.smithi148.stdout:1/505: dread d3/de/d25/d51/f54 [0,4194304] 0 2024-03-29T08:07:19.689 INFO:tasks.workunit.client.0.smithi037.stdout:6/341: write d0/d5/d1a/d61/f65 [557639,70611] 0 2024-03-29T08:07:19.689 INFO:tasks.workunit.client.0.smithi037.stdout:8/425: write d5/f15 [1096136,11527] 0 2024-03-29T08:07:19.689 INFO:tasks.workunit.client.1.smithi148.stdout:5/515: dwrite d0/db/d1e/d5b/d66/d73/f8b [0,4194304] 0 2024-03-29T08:07:19.691 INFO:tasks.workunit.client.1.smithi148.stdout:8/539: symlink d17/d18/d26/d63/lb0 0 2024-03-29T08:07:19.696 INFO:tasks.workunit.client.0.smithi037.stdout:9/561: dwrite d1/d4/d19/d1e/d4a/d6f/da2/d23/f37 [4194304,4194304] 0 2024-03-29T08:07:19.697 INFO:tasks.workunit.client.0.smithi037.stdout:2/261: dread - d0/d23/d44/f46 zero size 2024-03-29T08:07:19.699 INFO:tasks.workunit.client.0.smithi037.stdout:8/426: dread - d5/d10/d19/d3d/d4a/d4c/f6e zero size 2024-03-29T08:07:19.699 INFO:tasks.workunit.client.1.smithi148.stdout:6/528: rmdir d6/d9/d1e/d27/d48/d73 39 2024-03-29T08:07:19.703 INFO:tasks.workunit.client.1.smithi148.stdout:2/525: dwrite d7/da/f3f [0,4194304] 0 2024-03-29T08:07:19.704 INFO:tasks.workunit.client.1.smithi148.stdout:7/477: rename d8/d29/d44/d49/f6f to d8/d29/d44/d49/fa3 0 2024-03-29T08:07:19.706 INFO:tasks.workunit.client.0.smithi037.stdout:5/524: symlink d8/d22/d37/d60/d61/lae 0 2024-03-29T08:07:19.707 INFO:tasks.workunit.client.1.smithi148.stdout:8/540: creat d17/d74/fb1 x:0 0 0 2024-03-29T08:07:19.708 INFO:tasks.workunit.client.1.smithi148.stdout:2/526: chown d7/da/d33/d45/d4d/d2e/d37/f43 1324470 1 2024-03-29T08:07:19.711 INFO:tasks.workunit.client.0.smithi037.stdout:4/386: dwrite d3/d4/d15/d1d/db/f12 [0,4194304] 0 2024-03-29T08:07:19.711 INFO:tasks.workunit.client.1.smithi148.stdout:2/527: readlink d7/da/d33/d3c/l30 0 2024-03-29T08:07:19.714 INFO:tasks.workunit.client.0.smithi037.stdout:5/525: write d8/d22/f27 [8882817,130526] 0 2024-03-29T08:07:19.721 INFO:tasks.workunit.client.1.smithi148.stdout:7/478: symlink d8/d29/d44/d50/d85/la4 0 2024-03-29T08:07:19.723 INFO:tasks.workunit.client.1.smithi148.stdout:6/529: symlink d6/d9/d1e/d20/la4 0 2024-03-29T08:07:19.723 INFO:tasks.workunit.client.1.smithi148.stdout:4/522: link df/d2c/d51/d77/d26/d34/d3e/l66 df/d2c/d80/lae 0 2024-03-29T08:07:19.724 INFO:tasks.workunit.client.1.smithi148.stdout:0/516: getdents db/de/df/d2b/d3b/d47/d4c/d5a 0 2024-03-29T08:07:19.727 INFO:tasks.workunit.client.0.smithi037.stdout:4/387: dread - d3/d4/d15/d1d/db/d45/d1e/d59/d60/f75 zero size 2024-03-29T08:07:19.728 INFO:tasks.workunit.client.0.smithi037.stdout:7/380: truncate dd/d21/d35/f6b 435823 0 2024-03-29T08:07:19.729 INFO:tasks.workunit.client.1.smithi148.stdout:9/521: creat db/d10/faf x:0 0 0 2024-03-29T08:07:19.730 INFO:tasks.workunit.client.1.smithi148.stdout:7/479: mkdir d8/d38/d3c/da5 0 2024-03-29T08:07:19.731 INFO:tasks.workunit.client.1.smithi148.stdout:6/530: rmdir d6/d9/d1e/d36/d4d 39 2024-03-29T08:07:19.732 INFO:tasks.workunit.client.0.smithi037.stdout:8/427: readlink d5/d10/l2b 0 2024-03-29T08:07:19.732 INFO:tasks.workunit.client.1.smithi148.stdout:4/523: write df/d2c/d51/d77/f1d [882162,70665] 0 2024-03-29T08:07:19.733 INFO:tasks.workunit.client.0.smithi037.stdout:3/373: dread fc [0,4194304] 0 2024-03-29T08:07:19.733 INFO:tasks.workunit.client.0.smithi037.stdout:5/526: creat d8/d22/d37/d60/faf x:0 0 0 2024-03-29T08:07:19.736 INFO:tasks.workunit.client.1.smithi148.stdout:9/522: stat db/d22/d26/d54/d4d/f79 0 2024-03-29T08:07:19.738 INFO:tasks.workunit.client.1.smithi148.stdout:2/528: dwrite d7/da/d33/d45/d4d/d41/f6a [0,4194304] 0 2024-03-29T08:07:19.744 INFO:tasks.workunit.client.1.smithi148.stdout:2/529: mkdir d7/da/d33/d3c/da5 0 2024-03-29T08:07:19.744 INFO:tasks.workunit.client.1.smithi148.stdout:5/516: rename d0/db/d1e/d35/d42/l7d to d0/db/la9 0 2024-03-29T08:07:19.744 INFO:tasks.workunit.client.0.smithi037.stdout:2/262: dread d0/d20/d22/f48 [0,4194304] 0 2024-03-29T08:07:19.745 INFO:tasks.workunit.client.0.smithi037.stdout:5/527: write d8/d22/d4f/d70/d79/f92 [364751,254] 0 2024-03-29T08:07:19.746 INFO:tasks.workunit.client.0.smithi037.stdout:8/428: unlink d5/d10/f11 0 2024-03-29T08:07:19.751 INFO:tasks.workunit.client.0.smithi037.stdout:1/454: write d2/d6/dc/d18/d22/f6e [16143,109109] 0 2024-03-29T08:07:19.754 INFO:tasks.workunit.client.0.smithi037.stdout:9/562: dwrite d1/f4c [0,4194304] 0 2024-03-29T08:07:19.755 INFO:tasks.workunit.client.0.smithi037.stdout:3/374: creat dd/d5e/f80 x:0 0 0 2024-03-29T08:07:19.756 INFO:tasks.workunit.client.1.smithi148.stdout:3/556: dread de/d3b/f58 [0,4194304] 0 2024-03-29T08:07:19.756 INFO:tasks.workunit.client.1.smithi148.stdout:4/524: read df/f4d [945156,39243] 0 2024-03-29T08:07:19.757 INFO:tasks.workunit.client.1.smithi148.stdout:5/517: creat d0/db/d1e/d79/faa x:0 0 0 2024-03-29T08:07:19.759 INFO:tasks.workunit.client.1.smithi148.stdout:1/506: dread d3/d18/d4d/d8f/f6b [0,4194304] 0 2024-03-29T08:07:19.759 INFO:tasks.workunit.client.1.smithi148.stdout:2/530: mknod d7/d24/d4b/d66/ca6 0 2024-03-29T08:07:19.761 INFO:tasks.workunit.client.0.smithi037.stdout:5/528: mknod d8/d34/cb0 0 2024-03-29T08:07:19.767 INFO:tasks.workunit.client.1.smithi148.stdout:2/531: write d7/d8c/d91/f73 [957098,77222] 0 2024-03-29T08:07:19.768 INFO:tasks.workunit.client.1.smithi148.stdout:8/541: write d17/d74/d39/f73 [3391200,11961] 0 2024-03-29T08:07:19.770 INFO:tasks.workunit.client.1.smithi148.stdout:2/532: fsync d7/f2d 0 2024-03-29T08:07:19.773 INFO:tasks.workunit.client.0.smithi037.stdout:9/563: creat d1/d4/d19/d69/fc2 x:0 0 0 2024-03-29T08:07:19.773 INFO:tasks.workunit.client.1.smithi148.stdout:4/525: dread df/d2c/d51/d77/d26/f4a [0,4194304] 0 2024-03-29T08:07:19.773 INFO:tasks.workunit.client.1.smithi148.stdout:3/557: fsync de/d24/d33/d39/f5a 0 2024-03-29T08:07:19.774 INFO:tasks.workunit.client.1.smithi148.stdout:1/507: symlink d3/de/d25/d51/l98 0 2024-03-29T08:07:19.774 INFO:tasks.workunit.client.0.smithi037.stdout:4/388: dwrite d3/d4/d15/d1d/f3f [0,4194304] 0 2024-03-29T08:07:19.776 INFO:tasks.workunit.client.1.smithi148.stdout:0/517: write db/f97 [1103647,78638] 0 2024-03-29T08:07:19.776 INFO:tasks.workunit.client.1.smithi148.stdout:6/531: dwrite d6/d9/d1e/d36/d4d/d9d/d28/f74 [4194304,4194304] 0 2024-03-29T08:07:19.780 INFO:tasks.workunit.client.0.smithi037.stdout:1/455: symlink d2/d6/dc/d18/d1d/d21/l9e 0 2024-03-29T08:07:19.789 INFO:tasks.workunit.client.1.smithi148.stdout:5/518: link d0/db/d7b/c25 d0/db/d1e/d5b/d66/d72/d95/cab 0 2024-03-29T08:07:19.792 INFO:tasks.workunit.client.0.smithi037.stdout:3/375: creat dd/d46/d2d/d3a/f81 x:0 0 0 2024-03-29T08:07:19.793 INFO:tasks.workunit.client.0.smithi037.stdout:2/263: dread d0/f18 [0,4194304] 0 2024-03-29T08:07:19.793 INFO:tasks.workunit.client.1.smithi148.stdout:2/533: mknod d7/da/d33/d3c/da5/ca7 0 2024-03-29T08:07:19.794 INFO:tasks.workunit.client.1.smithi148.stdout:1/508: truncate d3/de/d1d/d47/d6d/d84/f92 488287 0 2024-03-29T08:07:19.796 INFO:tasks.workunit.client.1.smithi148.stdout:5/519: stat d0/db/d1e/d79/l89 0 2024-03-29T08:07:19.797 INFO:tasks.workunit.client.0.smithi037.stdout:9/564: rmdir d1/d4/d95 39 2024-03-29T08:07:19.797 INFO:tasks.workunit.client.1.smithi148.stdout:8/542: creat d17/d74/d39/d6f/d64/da5/fb2 x:0 0 0 2024-03-29T08:07:19.797 INFO:tasks.workunit.client.1.smithi148.stdout:7/480: dwrite d8/d60/f98 [0,4194304] 0 2024-03-29T08:07:19.798 INFO:tasks.workunit.client.0.smithi037.stdout:5/529: dread d8/d18/f21 [0,4194304] 0 2024-03-29T08:07:19.799 INFO:tasks.workunit.client.0.smithi037.stdout:7/381: getdents dd/d1d/d49 0 2024-03-29T08:07:19.799 INFO:tasks.workunit.client.0.smithi037.stdout:1/456: unlink d2/d6/dc/d5d/d64/c75 0 2024-03-29T08:07:19.800 INFO:tasks.workunit.client.1.smithi148.stdout:1/509: fsync d3/d55/d7e/f5b 0 2024-03-29T08:07:19.805 INFO:tasks.workunit.client.0.smithi037.stdout:8/429: dwrite d5/d10/d2d/f60 [0,4194304] 0 2024-03-29T08:07:19.805 INFO:tasks.workunit.client.1.smithi148.stdout:0/518: truncate db/de/df/d2b/d3b/d47/d4c/d5a/d8d/f99 625683 0 2024-03-29T08:07:19.805 INFO:tasks.workunit.client.0.smithi037.stdout:5/530: chown d8 185 1 2024-03-29T08:07:19.807 INFO:tasks.workunit.client.1.smithi148.stdout:1/510: fdatasync d3/de/f86 0 2024-03-29T08:07:19.809 INFO:tasks.workunit.client.1.smithi148.stdout:2/534: rename d7/da/d33/d3c/d54/f75 to d7/d24/fa8 0 2024-03-29T08:07:19.809 INFO:tasks.workunit.client.0.smithi037.stdout:9/565: dread - d1/d3/d44/f9f zero size 2024-03-29T08:07:19.813 INFO:tasks.workunit.client.1.smithi148.stdout:4/526: dwrite df/d2c/d51/d77/d17/f29 [4194304,4194304] 0 2024-03-29T08:07:19.813 INFO:tasks.workunit.client.0.smithi037.stdout:7/382: fdatasync dd/d21/f75 0 2024-03-29T08:07:19.815 INFO:tasks.workunit.client.0.smithi037.stdout:6/342: dwrite d0/d5/f2a [0,4194304] 0 2024-03-29T08:07:19.816 INFO:tasks.workunit.client.1.smithi148.stdout:8/543: creat d17/d49/fb3 x:0 0 0 2024-03-29T08:07:19.816 INFO:tasks.workunit.client.1.smithi148.stdout:0/519: write db/de/df/f95 [850236,74650] 0 2024-03-29T08:07:19.818 INFO:tasks.workunit.client.1.smithi148.stdout:6/532: dread d6/d9/d1e/d36/f6f [0,4194304] 0 2024-03-29T08:07:19.822 INFO:tasks.workunit.client.1.smithi148.stdout:1/511: write d3/de/d1d/d47/d6d/d84/f92 [1233105,105869] 0 2024-03-29T08:07:19.822 INFO:tasks.workunit.client.0.smithi037.stdout:2/264: mkdir d0/d23/d59 0 2024-03-29T08:07:19.824 INFO:tasks.workunit.client.0.smithi037.stdout:4/389: dwrite d3/d4/f34 [4194304,4194304] 0 2024-03-29T08:07:19.827 INFO:tasks.workunit.client.0.smithi037.stdout:5/531: unlink d8/d18/c2c 0 2024-03-29T08:07:19.828 INFO:tasks.workunit.client.1.smithi148.stdout:9/523: dread db/d10/d2d/d4e/f81 [0,4194304] 0 2024-03-29T08:07:19.830 INFO:tasks.workunit.client.0.smithi037.stdout:7/383: dread - dd/d1d/f53 zero size 2024-03-29T08:07:19.831 INFO:tasks.workunit.client.0.smithi037.stdout:5/532: write d8/d22/d4f/f68 [256304,95008] 0 2024-03-29T08:07:19.837 INFO:tasks.workunit.client.0.smithi037.stdout:6/343: mkdir d0/d5/d1e/d44/d72 0 2024-03-29T08:07:19.838 INFO:tasks.workunit.client.1.smithi148.stdout:2/535: rename d7/d8c/da4/d8e/f94 to d7/da/d33/d45/d4d/d82/fa9 0 2024-03-29T08:07:19.839 INFO:tasks.workunit.client.0.smithi037.stdout:1/457: write d2/f14 [2011670,47936] 0 2024-03-29T08:07:19.842 INFO:tasks.workunit.client.1.smithi148.stdout:7/481: dwrite d8/d29/d44/d49/f83 [0,4194304] 0 2024-03-29T08:07:19.842 INFO:tasks.workunit.client.1.smithi148.stdout:5/520: dwrite d0/d46/d9e/fa6 [0,4194304] 0 2024-03-29T08:07:19.847 INFO:tasks.workunit.client.0.smithi037.stdout:3/376: dwrite dd/d5e/f5c [0,4194304] 0 2024-03-29T08:07:19.848 INFO:tasks.workunit.client.1.smithi148.stdout:9/524: mkdir db/d10/d2d/d4e/d5f/d6e/db0 0 2024-03-29T08:07:19.856 INFO:tasks.workunit.client.1.smithi148.stdout:8/544: rename d17/d1d/f79 to d17/d74/fb4 0 2024-03-29T08:07:19.856 INFO:tasks.workunit.client.1.smithi148.stdout:6/533: mknod d6/d9/d1e/d20/d30/d51/ca5 0 2024-03-29T08:07:19.858 INFO:tasks.workunit.client.0.smithi037.stdout:5/533: creat d8/d22/d37/fb1 x:0 0 0 2024-03-29T08:07:19.864 INFO:tasks.workunit.client.0.smithi037.stdout:9/566: chown d1/d4/d19/d1e/d4a/c7e 239336183 1 2024-03-29T08:07:19.866 INFO:tasks.workunit.client.0.smithi037.stdout:5/534: write d8/d18/d24/f44 [3672310,25612] 0 2024-03-29T08:07:19.867 INFO:tasks.workunit.client.1.smithi148.stdout:5/521: symlink d0/db/d1e/d6e/da8/lac 0 2024-03-29T08:07:19.867 INFO:tasks.workunit.client.1.smithi148.stdout:4/527: dwrite df/d2c/d51/d77/d26/f3d [0,4194304] 0 2024-03-29T08:07:19.874 INFO:tasks.workunit.client.0.smithi037.stdout:3/377: dwrite dd/fe [4194304,4194304] 0 2024-03-29T08:07:19.876 INFO:tasks.workunit.client.0.smithi037.stdout:2/265: getdents d0/d20/d22/d55 0 2024-03-29T08:07:19.877 INFO:tasks.workunit.client.1.smithi148.stdout:4/528: rmdir df/d2c/d51/d77/d17 39 2024-03-29T08:07:19.878 INFO:tasks.workunit.client.0.smithi037.stdout:4/390: mkdir d3/d80/d81 0 2024-03-29T08:07:19.883 INFO:tasks.workunit.client.0.smithi037.stdout:8/430: mkdir d5/d10/d2d/d34/d50/d83/d46/d8a 0 2024-03-29T08:07:19.886 INFO:tasks.workunit.client.1.smithi148.stdout:3/558: dread de/d24/d33/d39/d46/d30/d37/f7a [4194304,4194304] 0 2024-03-29T08:07:19.889 INFO:tasks.workunit.client.1.smithi148.stdout:4/529: truncate df/d2c/d51/d77/d26/d34/d3e/f3f 792156 0 2024-03-29T08:07:19.892 INFO:tasks.workunit.client.1.smithi148.stdout:1/512: write d3/de/d1d/f33 [979137,83918] 0 2024-03-29T08:07:19.893 INFO:tasks.workunit.client.1.smithi148.stdout:8/545: fdatasync d17/d4e/f5d 0 2024-03-29T08:07:19.897 INFO:tasks.workunit.client.1.smithi148.stdout:1/513: read - d3/d55/d7e/f94 zero size 2024-03-29T08:07:19.898 INFO:tasks.workunit.client.1.smithi148.stdout:7/482: rename d8/c69 to d8/d14/d45/d4d/ca6 0 2024-03-29T08:07:19.900 INFO:tasks.workunit.client.1.smithi148.stdout:9/525: truncate db/fa7 3877855 0 2024-03-29T08:07:19.901 INFO:tasks.workunit.client.0.smithi037.stdout:9/567: creat d1/d4/d19/d1e/d4a/d6f/da2/d13/d7c/fc3 x:0 0 0 2024-03-29T08:07:19.901 INFO:tasks.workunit.client.1.smithi148.stdout:6/534: write d6/f19 [3516826,36842] 0 2024-03-29T08:07:19.905 INFO:tasks.workunit.client.0.smithi037.stdout:4/391: dwrite d3/d4/d15/f49 [4194304,4194304] 0 2024-03-29T08:07:19.907 INFO:tasks.workunit.client.1.smithi148.stdout:4/530: mknod df/d2c/d51/d77/d26/d9b/da7/caf 0 2024-03-29T08:07:19.908 INFO:tasks.workunit.client.0.smithi037.stdout:0/275: sync 2024-03-29T08:07:19.908 INFO:tasks.workunit.client.0.smithi037.stdout:3/378: rmdir dd/d15/d18/d30 39 2024-03-29T08:07:19.908 INFO:tasks.workunit.client.1.smithi148.stdout:7/483: write d8/d29/d44/d50/f8b [1253441,106176] 0 2024-03-29T08:07:19.909 INFO:tasks.workunit.client.1.smithi148.stdout:8/546: symlink d17/d74/d39/d6f/d64/da5/lb5 0 2024-03-29T08:07:19.909 INFO:tasks.workunit.client.1.smithi148.stdout:2/536: dwrite d7/da/d33/d45/d4d/d2e/f50 [0,4194304] 0 2024-03-29T08:07:19.909 INFO:tasks.workunit.client.1.smithi148.stdout:9/526: truncate db/d22/d26/d54/d4d/d35/d4b/d8e/f97 741771 0 2024-03-29T08:07:19.910 INFO:tasks.workunit.client.1.smithi148.stdout:7/484: chown d8/d38/d3c/da5 3 1 2024-03-29T08:07:19.912 INFO:tasks.workunit.client.1.smithi148.stdout:8/547: write d17/d74/f48 [3695011,76568] 0 2024-03-29T08:07:19.913 INFO:tasks.workunit.client.0.smithi037.stdout:7/384: rename dd/d21/d35/f71 to dd/f7e 0 2024-03-29T08:07:19.913 INFO:tasks.workunit.client.1.smithi148.stdout:9/527: dread - db/f59 zero size 2024-03-29T08:07:19.913 INFO:tasks.workunit.client.1.smithi148.stdout:1/514: rmdir d3/de/d6e 39 2024-03-29T08:07:19.916 INFO:tasks.workunit.client.0.smithi037.stdout:6/344: rmdir d0/d19/d42/d6e 0 2024-03-29T08:07:19.918 INFO:tasks.workunit.client.0.smithi037.stdout:5/535: mknod d8/cb2 0 2024-03-29T08:07:19.920 INFO:tasks.workunit.client.0.smithi037.stdout:9/568: mknod d1/d4/d19/d1e/d4a/d6f/da2/d31/d87/cc4 0 2024-03-29T08:07:19.921 INFO:tasks.workunit.client.0.smithi037.stdout:2/266: symlink d0/d23/l5a 0 2024-03-29T08:07:19.925 INFO:tasks.workunit.client.0.smithi037.stdout:4/392: creat d3/d4/d15/d1d/db/d45/d1e/d5f/f82 x:0 0 0 2024-03-29T08:07:19.928 INFO:tasks.workunit.client.1.smithi148.stdout:5/522: rename d0/f2c to d0/db/d7b/d16/d87/fad 0 2024-03-29T08:07:19.928 INFO:tasks.workunit.client.1.smithi148.stdout:0/520: dread db/f23 [0,4194304] 0 2024-03-29T08:07:19.931 INFO:tasks.workunit.client.1.smithi148.stdout:7/485: symlink d8/d29/d44/d50/d85/la7 0 2024-03-29T08:07:19.931 INFO:tasks.workunit.client.1.smithi148.stdout:1/515: getdents d3/de/d21/d7f 0 2024-03-29T08:07:19.935 INFO:tasks.workunit.client.1.smithi148.stdout:2/537: read d7/da/d33/d45/d4d/f57 [2527310,128813] 0 2024-03-29T08:07:19.935 INFO:tasks.workunit.client.1.smithi148.stdout:5/523: write d0/db/d1e/d35/d42/f5e [11745,86199] 0 2024-03-29T08:07:19.935 INFO:tasks.workunit.client.0.smithi037.stdout:7/385: creat dd/d21/d35/d60/f7f x:0 0 0 2024-03-29T08:07:19.935 INFO:tasks.workunit.client.0.smithi037.stdout:3/379: dwrite dd/d5e/f72 [0,4194304] 0 2024-03-29T08:07:19.937 INFO:tasks.workunit.client.1.smithi148.stdout:0/521: write db/de/df/d2b/d3b/d47/f96 [939061,62096] 0 2024-03-29T08:07:19.938 INFO:tasks.workunit.client.1.smithi148.stdout:7/486: mkdir d8/d14/da8 0 2024-03-29T08:07:19.939 INFO:tasks.workunit.client.0.smithi037.stdout:5/536: rmdir d8/d22/d37/d60/d88 39 2024-03-29T08:07:19.939 INFO:tasks.workunit.client.0.smithi037.stdout:5/537: readlink d8/l1b 0 2024-03-29T08:07:19.940 INFO:tasks.workunit.client.1.smithi148.stdout:9/528: mknod db/d10/d6d/d73/da8/cb1 0 2024-03-29T08:07:19.942 INFO:tasks.workunit.client.1.smithi148.stdout:6/535: link c3 d6/d9/d1e/d20/d30/ca6 0 2024-03-29T08:07:19.946 INFO:tasks.workunit.client.1.smithi148.stdout:1/516: creat d3/de/d21/d5e/f99 x:0 0 0 2024-03-29T08:07:19.946 INFO:tasks.workunit.client.1.smithi148.stdout:7/487: stat d8/d29/d44/d50/d85/la7 0 2024-03-29T08:07:19.946 INFO:tasks.workunit.client.0.smithi037.stdout:9/569: dread d1/d4/d19/d1e/d4a/d6f/da2/f12 [0,4194304] 0 2024-03-29T08:07:19.948 INFO:tasks.workunit.client.1.smithi148.stdout:3/559: rename f8 to de/d24/d33/d39/fb6 0 2024-03-29T08:07:19.948 INFO:tasks.workunit.client.1.smithi148.stdout:0/522: write db/de/df/d2b/d3b/d47/f2c [4904896,403] 0 2024-03-29T08:07:19.951 INFO:tasks.workunit.client.1.smithi148.stdout:5/524: fsync d0/db/d7b/d16/d2e/f68 0 2024-03-29T08:07:19.954 INFO:tasks.workunit.client.1.smithi148.stdout:7/488: write d8/d60/f70 [4803919,27570] 0 2024-03-29T08:07:19.957 INFO:tasks.workunit.client.0.smithi037.stdout:8/431: creat d5/d78/f8b x:0 0 0 2024-03-29T08:07:19.957 INFO:tasks.workunit.client.1.smithi148.stdout:2/538: creat d7/da/d33/d3c/d31/faa x:0 0 0 2024-03-29T08:07:19.958 INFO:tasks.workunit.client.0.smithi037.stdout:1/458: write d2/d6/dc/f25 [2485551,8934] 0 2024-03-29T08:07:19.959 INFO:tasks.workunit.client.1.smithi148.stdout:3/560: symlink de/d24/d33/d39/d7f/lb7 0 2024-03-29T08:07:19.963 INFO:tasks.workunit.client.0.smithi037.stdout:2/267: truncate d0/f40 3765381 0 2024-03-29T08:07:19.965 INFO:tasks.workunit.client.0.smithi037.stdout:5/538: mkdir d8/d22/d37/d8d/db3 0 2024-03-29T08:07:19.966 INFO:tasks.workunit.client.1.smithi148.stdout:1/517: rename d3/de/d25/d35/d42/c87 to d3/d55/d7e/c9a 0 2024-03-29T08:07:19.969 INFO:tasks.workunit.client.1.smithi148.stdout:0/523: creat db/de/df/d2b/d3b/d47/d9f/fa5 x:0 0 0 2024-03-29T08:07:19.970 INFO:tasks.workunit.client.0.smithi037.stdout:4/393: creat d3/d4/d15/d1d/db/d45/d1e/d59/d73/f83 x:0 0 0 2024-03-29T08:07:19.970 INFO:tasks.workunit.client.0.smithi037.stdout:1/459: write d2/d6/dc/f25 [2735222,25739] 0 2024-03-29T08:07:19.972 INFO:tasks.workunit.client.1.smithi148.stdout:5/525: mkdir d0/db/dae 0 2024-03-29T08:07:19.975 INFO:tasks.workunit.client.1.smithi148.stdout:5/526: stat d0/db/d7b/la1 0 2024-03-29T08:07:19.977 INFO:tasks.workunit.client.0.smithi037.stdout:6/345: creat d0/d5/f73 x:0 0 0 2024-03-29T08:07:19.977 INFO:tasks.workunit.client.0.smithi037.stdout:9/570: dread d1/f4c [0,4194304] 0 2024-03-29T08:07:19.977 INFO:tasks.workunit.client.0.smithi037.stdout:4/394: chown d3/d80/d81 33920675 1 2024-03-29T08:07:19.977 INFO:tasks.workunit.client.0.smithi037.stdout:3/380: dwrite dd/d46/d3e/d47/f54 [0,4194304] 0 2024-03-29T08:07:19.979 INFO:tasks.workunit.client.1.smithi148.stdout:9/529: dwrite db/d10/d2d/d4e/f6c [0,4194304] 0 2024-03-29T08:07:19.979 INFO:tasks.workunit.client.1.smithi148.stdout:5/527: chown d0/l49 32421 1 2024-03-29T08:07:19.979 INFO:tasks.workunit.client.1.smithi148.stdout:8/548: dread d17/d18/f8a [0,4194304] 0 2024-03-29T08:07:19.986 INFO:tasks.workunit.client.1.smithi148.stdout:2/539: dread d7/da/d33/d45/d4d/f16 [0,4194304] 0 2024-03-29T08:07:19.987 INFO:tasks.workunit.client.0.smithi037.stdout:4/395: chown d3/c7 26975831 1 2024-03-29T08:07:19.988 INFO:tasks.workunit.client.0.smithi037.stdout:5/539: symlink d8/d22/lb4 0 2024-03-29T08:07:19.989 INFO:tasks.workunit.client.1.smithi148.stdout:6/536: link d6/l46 d6/d9/d1e/d20/d30/la7 0 2024-03-29T08:07:19.991 INFO:tasks.workunit.client.0.smithi037.stdout:9/571: fdatasync d1/d4/d19/d69/d75/f8b 0 2024-03-29T08:07:19.992 INFO:tasks.workunit.client.1.smithi148.stdout:0/524: creat db/de/df/fa6 x:0 0 0 2024-03-29T08:07:19.992 INFO:tasks.workunit.client.1.smithi148.stdout:3/561: dwrite de/d3b/d49/f8a [0,4194304] 0 2024-03-29T08:07:19.992 INFO:tasks.workunit.client.1.smithi148.stdout:4/531: truncate df/d2c/d51/d77/d17/f20 4016426 0 2024-03-29T08:07:19.994 INFO:tasks.workunit.client.0.smithi037.stdout:0/276: truncate d9/da/f25 4149175 0 2024-03-29T08:07:19.995 INFO:tasks.workunit.client.0.smithi037.stdout:5/540: rename d8 to d8/d4d/d8a/db5 22 2024-03-29T08:07:19.997 INFO:tasks.workunit.client.1.smithi148.stdout:2/540: fdatasync d7/da/d33/d3c/d54/f95 0 2024-03-29T08:07:20.001 INFO:tasks.workunit.client.1.smithi148.stdout:1/518: dwrite d3/d55/f43 [0,4194304] 0 2024-03-29T08:07:20.004 INFO:tasks.workunit.client.1.smithi148.stdout:9/530: unlink db/d10/d2d/d4e/d5f/d6e/d7a/la4 0 2024-03-29T08:07:20.004 INFO:tasks.workunit.client.1.smithi148.stdout:0/525: rmdir db/de/df/d11/d42 39 2024-03-29T08:07:20.004 INFO:tasks.workunit.client.1.smithi148.stdout:1/519: readlink d3/l17 0 2024-03-29T08:07:20.005 INFO:tasks.workunit.client.1.smithi148.stdout:5/528: mkdir d0/db/d84/daf 0 2024-03-29T08:07:20.006 INFO:tasks.workunit.client.1.smithi148.stdout:2/541: mknod d7/da/d4c/cab 0 2024-03-29T08:07:20.008 INFO:tasks.workunit.client.1.smithi148.stdout:0/526: fdatasync db/dc/d78/f9a 0 2024-03-29T08:07:20.008 INFO:tasks.workunit.client.0.smithi037.stdout:2/268: creat d0/d20/d22/d55/f5b x:0 0 0 2024-03-29T08:07:20.010 INFO:tasks.workunit.client.0.smithi037.stdout:7/386: dwrite dd/d21/d35/f4c [0,4194304] 0 2024-03-29T08:07:20.011 INFO:tasks.workunit.client.1.smithi148.stdout:2/542: stat d7/da/d33/d45/d4d/d2e/d37/f43 0 2024-03-29T08:07:20.015 INFO:tasks.workunit.client.0.smithi037.stdout:6/346: creat d0/d5/d1e/d44/d71/f74 x:0 0 0 2024-03-29T08:07:20.016 INFO:tasks.workunit.client.0.smithi037.stdout:8/432: dwrite d5/fb [0,4194304] 0 2024-03-29T08:07:20.017 INFO:tasks.workunit.client.1.smithi148.stdout:4/532: creat df/d2c/d51/d77/d26/d34/d8a/d98/fb0 x:0 0 0 2024-03-29T08:07:20.017 INFO:tasks.workunit.client.0.smithi037.stdout:5/541: unlink d8/d18/f43 0 2024-03-29T08:07:20.020 INFO:tasks.workunit.client.1.smithi148.stdout:6/537: rename d6/d9/d1e/d36/d4d/d9d/c50 to d6/d9/d1e/d20/d6b/ca8 0 2024-03-29T08:07:20.021 INFO:tasks.workunit.client.1.smithi148.stdout:4/533: chown df/d2c/d51/d77/d26/d9b/da7/l3a 5608 1 2024-03-29T08:07:20.022 INFO:tasks.workunit.client.1.smithi148.stdout:0/527: mknod db/dc/ca7 0 2024-03-29T08:07:20.024 INFO:tasks.workunit.client.0.smithi037.stdout:7/387: chown dd/d21/d35/f79 93 1 2024-03-29T08:07:20.024 INFO:tasks.workunit.client.1.smithi148.stdout:7/489: dwrite d8/f10 [0,4194304] 0 2024-03-29T08:07:20.030 INFO:tasks.workunit.client.0.smithi037.stdout:4/396: mknod d3/d4/d15/d1d/db/d45/c84 0 2024-03-29T08:07:20.035 INFO:tasks.workunit.client.0.smithi037.stdout:1/460: rename d2/d6/dc/d58/d67/d95 to d2/d6/dc/d18/d22/d4f/d9f 0 2024-03-29T08:07:20.035 INFO:tasks.workunit.client.1.smithi148.stdout:1/520: getdents d3/d18/d4d 0 2024-03-29T08:07:20.043 INFO:tasks.workunit.client.1.smithi148.stdout:9/531: link db/d22/d3f/d61/c6a db/d22/d26/d54/d4d/d35/d4b/d58/cb2 0 2024-03-29T08:07:20.043 INFO:tasks.workunit.client.1.smithi148.stdout:8/549: truncate d17/d74/fb4 788689 0 2024-03-29T08:07:20.044 INFO:tasks.workunit.client.0.smithi037.stdout:6/347: creat d0/d5/d1e/d44/d71/f75 x:0 0 0 2024-03-29T08:07:20.047 INFO:tasks.workunit.client.1.smithi148.stdout:6/538: creat d6/d9/d1e/d27/d48/d73/d7e/fa9 x:0 0 0 2024-03-29T08:07:20.048 INFO:tasks.workunit.client.0.smithi037.stdout:1/461: truncate d2/d6/f36 5230531 0 2024-03-29T08:07:20.048 INFO:tasks.workunit.client.0.smithi037.stdout:6/348: readlink d0/l2 0 2024-03-29T08:07:20.049 INFO:tasks.workunit.client.0.smithi037.stdout:4/397: chown d3/d4/d15/d1d/db/d45/f5a 506626 1 2024-03-29T08:07:20.049 INFO:tasks.workunit.client.1.smithi148.stdout:1/521: read d3/fa [6141155,62644] 0 2024-03-29T08:07:20.049 INFO:tasks.workunit.client.1.smithi148.stdout:8/550: write d17/d49/f9c [3113656,121637] 0 2024-03-29T08:07:20.053 INFO:tasks.workunit.client.0.smithi037.stdout:6/349: stat d0/d20 0 2024-03-29T08:07:20.056 INFO:tasks.workunit.client.1.smithi148.stdout:0/528: dwrite db/de/d24/f38 [4194304,4194304] 0 2024-03-29T08:07:20.059 INFO:tasks.workunit.client.1.smithi148.stdout:1/522: creat d3/d18/d71/f9b x:0 0 0 2024-03-29T08:07:20.060 INFO:tasks.workunit.client.1.smithi148.stdout:9/532: dread db/d22/d26/d54/f3c [0,4194304] 0 2024-03-29T08:07:20.066 INFO:tasks.workunit.client.0.smithi037.stdout:0/277: dwrite d9/da/dd/d13/f4b [0,4194304] 0 2024-03-29T08:07:20.067 INFO:tasks.workunit.client.0.smithi037.stdout:8/433: dread d5/fb [0,4194304] 0 2024-03-29T08:07:20.068 INFO:tasks.workunit.client.1.smithi148.stdout:2/543: dread d7/da/d33/d45/d4d/f57 [0,4194304] 0 2024-03-29T08:07:20.068 INFO:tasks.workunit.client.0.smithi037.stdout:7/388: dwrite dd/f48 [0,4194304] 0 2024-03-29T08:07:20.074 INFO:tasks.workunit.client.1.smithi148.stdout:8/551: dwrite d17/d18/f5c [0,4194304] 0 2024-03-29T08:07:20.074 INFO:tasks.workunit.client.1.smithi148.stdout:2/544: truncate d7/da/d33/d45/d4d/d2e/f50 5889838 0 2024-03-29T08:07:20.087 INFO:tasks.workunit.client.0.smithi037.stdout:0/278: fsync d9/da/dd/d13/f36 0 2024-03-29T08:07:20.087 INFO:tasks.workunit.client.0.smithi037.stdout:2/269: creat d0/d23/f5c x:0 0 0 2024-03-29T08:07:20.089 INFO:tasks.workunit.client.1.smithi148.stdout:9/533: mknod db/d22/d26/d54/d4d/d35/cb3 0 2024-03-29T08:07:20.097 INFO:tasks.workunit.client.1.smithi148.stdout:2/545: symlink d7/da/d33/d3c/d54/d70/lac 0 2024-03-29T08:07:20.100 INFO:tasks.workunit.client.1.smithi148.stdout:6/539: link d6/d9/d1e/d36/d4d/d9d/f31 d6/d9/d1e/d27/d48/d9f/faa 0 2024-03-29T08:07:20.103 INFO:tasks.workunit.client.0.smithi037.stdout:3/381: truncate dd/d15/d27/f41 713220 0 2024-03-29T08:07:20.104 INFO:tasks.workunit.client.1.smithi148.stdout:9/534: creat db/d22/d26/d54/d4d/d35/d4b/fb4 x:0 0 0 2024-03-29T08:07:20.105 INFO:tasks.workunit.client.1.smithi148.stdout:9/535: stat db/lf 0 2024-03-29T08:07:20.106 INFO:tasks.workunit.client.1.smithi148.stdout:3/562: truncate de/d24/d33/d39/d7f/fac 446913 0 2024-03-29T08:07:20.106 INFO:tasks.workunit.client.0.smithi037.stdout:3/382: chown dd/d46/l48 914 1 2024-03-29T08:07:20.109 INFO:tasks.workunit.client.1.smithi148.stdout:2/546: mkdir d7/da/d4c/d9a/dad 0 2024-03-29T08:07:20.110 INFO:tasks.workunit.client.0.smithi037.stdout:9/572: rename d1/d3/c15 to d1/d4/d19/d1e/d25/cc5 0 2024-03-29T08:07:20.110 INFO:tasks.workunit.client.1.smithi148.stdout:6/540: symlink d6/d9/d1e/d36/d4d/lab 0 2024-03-29T08:07:20.112 INFO:tasks.workunit.client.1.smithi148.stdout:6/541: fdatasync d6/d9/f75 0 2024-03-29T08:07:20.114 INFO:tasks.workunit.client.0.smithi037.stdout:9/573: write d1/d4/d19/d1e/d25/fc1 [704029,112032] 0 2024-03-29T08:07:20.116 INFO:tasks.workunit.client.1.smithi148.stdout:7/490: rename d8/d14/d19 to d8/d38/d3c/da9 0 2024-03-29T08:07:20.121 INFO:tasks.workunit.client.1.smithi148.stdout:0/529: link db/c6b db/de/df/ca8 0 2024-03-29T08:07:20.123 INFO:tasks.workunit.client.1.smithi148.stdout:2/547: read - d7/da/d33/d3c/f42 zero size 2024-03-29T08:07:20.125 INFO:tasks.workunit.client.1.smithi148.stdout:5/529: dwrite d0/db/d1e/d5b/d66/d72/d95/f9d [0,4194304] 0 2024-03-29T08:07:20.127 INFO:tasks.workunit.client.1.smithi148.stdout:0/530: write db/de/df/d2b/f93 [587692,22154] 0 2024-03-29T08:07:20.128 INFO:tasks.workunit.client.0.smithi037.stdout:7/389: symlink dd/d21/l80 0 2024-03-29T08:07:20.128 INFO:tasks.workunit.client.1.smithi148.stdout:6/542: creat d6/d9/d1e/d20/d30/fac x:0 0 0 2024-03-29T08:07:20.130 INFO:tasks.workunit.client.0.smithi037.stdout:7/390: chown dd/d1d/l7a 43 1 2024-03-29T08:07:20.130 INFO:tasks.workunit.client.1.smithi148.stdout:4/534: dwrite df/d2c/d51/d77/d26/d9b/da7/f35 [0,4194304] 0 2024-03-29T08:07:20.133 INFO:tasks.workunit.client.1.smithi148.stdout:0/531: read - db/de/df/d2b/f53 zero size 2024-03-29T08:07:20.134 INFO:tasks.workunit.client.1.smithi148.stdout:1/523: rename d3/d18/f4b to d3/de/d25/f9c 0 2024-03-29T08:07:20.134 INFO:tasks.workunit.client.1.smithi148.stdout:4/535: chown df/f5e 829457 1 2024-03-29T08:07:20.134 INFO:tasks.workunit.client.0.smithi037.stdout:8/434: symlink d5/d10/d19/d3d/d4a/d4c/d67/l8c 0 2024-03-29T08:07:20.136 INFO:tasks.workunit.client.1.smithi148.stdout:0/532: fdatasync db/f97 0 2024-03-29T08:07:20.137 INFO:tasks.workunit.client.1.smithi148.stdout:9/536: rmdir db/d22/d26/d54/d4d/d35/d4b/d5a/d74/d86 0 2024-03-29T08:07:20.139 INFO:tasks.workunit.client.1.smithi148.stdout:7/491: mknod d8/d38/d3c/da5/caa 0 2024-03-29T08:07:20.140 INFO:tasks.workunit.client.1.smithi148.stdout:4/536: fdatasync df/f14 0 2024-03-29T08:07:20.145 INFO:tasks.workunit.client.1.smithi148.stdout:2/548: write d7/da/f40 [4279374,33907] 0 2024-03-29T08:07:20.150 INFO:tasks.workunit.client.0.smithi037.stdout:6/350: link d0/d19/d34/l59 d0/d20/l76 0 2024-03-29T08:07:20.150 INFO:tasks.workunit.client.0.smithi037.stdout:1/462: dread d2/d6/dc/f15 [0,4194304] 0 2024-03-29T08:07:20.151 INFO:tasks.workunit.client.1.smithi148.stdout:5/530: dwrite d0/db/d1e/d79/faa [0,4194304] 0 2024-03-29T08:07:20.152 INFO:tasks.workunit.client.1.smithi148.stdout:6/543: dread d6/d9/d1e/d36/d4d/d9d/d28/f74 [4194304,4194304] 0 2024-03-29T08:07:20.156 INFO:tasks.workunit.client.1.smithi148.stdout:8/552: write d17/d74/fb4 [976611,124971] 0 2024-03-29T08:07:20.157 INFO:tasks.workunit.client.1.smithi148.stdout:2/549: mknod d7/d24/d4b/cae 0 2024-03-29T08:07:20.160 INFO:tasks.workunit.client.0.smithi037.stdout:0/279: symlink d9/da/dd/l58 0 2024-03-29T08:07:20.161 INFO:tasks.workunit.client.1.smithi148.stdout:5/531: dread d0/db/d1e/d35/f5c [0,4194304] 0 2024-03-29T08:07:20.162 INFO:tasks.workunit.client.1.smithi148.stdout:6/544: creat d6/d9/d68/fad x:0 0 0 2024-03-29T08:07:20.163 INFO:tasks.workunit.client.0.smithi037.stdout:4/398: truncate d3/d4/d15/d1d/db/f12 3501368 0 2024-03-29T08:07:20.164 INFO:tasks.workunit.client.0.smithi037.stdout:7/391: mkdir dd/d1d/d49/d81 0 2024-03-29T08:07:20.168 INFO:tasks.workunit.client.1.smithi148.stdout:2/550: unlink d7/da/d33/f3e 0 2024-03-29T08:07:20.170 INFO:tasks.workunit.client.1.smithi148.stdout:2/551: chown d7/da/d33/d45/d4d/d2e/c77 11 1 2024-03-29T08:07:20.171 INFO:tasks.workunit.client.1.smithi148.stdout:8/553: dread d17/d18/f5c [0,4194304] 0 2024-03-29T08:07:20.171 INFO:tasks.workunit.client.1.smithi148.stdout:5/532: creat d0/db/d1e/d5b/d66/d72/d95/fb0 x:0 0 0 2024-03-29T08:07:20.172 INFO:tasks.workunit.client.0.smithi037.stdout:2/270: write d0/d20/f3a [1796618,120086] 0 2024-03-29T08:07:20.173 INFO:tasks.workunit.client.1.smithi148.stdout:2/552: write d7/da/f49 [848821,117427] 0 2024-03-29T08:07:20.175 INFO:tasks.workunit.client.1.smithi148.stdout:7/492: creat d8/d29/d44/fab x:0 0 0 2024-03-29T08:07:20.176 INFO:tasks.workunit.client.1.smithi148.stdout:3/563: write de/f51 [1851121,40456] 0 2024-03-29T08:07:20.180 INFO:tasks.workunit.client.0.smithi037.stdout:9/574: symlink d1/d4/lc6 0 2024-03-29T08:07:20.183 INFO:tasks.workunit.client.1.smithi148.stdout:4/537: write df/d2c/d51/d77/d26/d9b/da7/f49 [2858382,889] 0 2024-03-29T08:07:20.183 INFO:tasks.workunit.client.0.smithi037.stdout:1/463: mkdir d2/d6/dc/d18/d22/da0 0 2024-03-29T08:07:20.184 INFO:tasks.workunit.client.1.smithi148.stdout:0/533: dwrite fa [0,4194304] 0 2024-03-29T08:07:20.184 INFO:tasks.workunit.client.1.smithi148.stdout:8/554: mknod d17/d18/cb6 0 2024-03-29T08:07:20.192 INFO:tasks.workunit.client.1.smithi148.stdout:9/537: truncate db/d22/d26/d54/d4d/d35/f5e 4163382 0 2024-03-29T08:07:20.193 INFO:tasks.workunit.client.1.smithi148.stdout:2/553: creat d7/da/d33/d3c/d31/faf x:0 0 0 2024-03-29T08:07:20.193 INFO:tasks.workunit.client.0.smithi037.stdout:7/392: write dd/d21/f32 [58993,73967] 0 2024-03-29T08:07:20.193 INFO:tasks.workunit.client.0.smithi037.stdout:6/351: dwrite d0/f33 [0,4194304] 0 2024-03-29T08:07:20.195 INFO:tasks.workunit.client.1.smithi148.stdout:5/533: mkdir d0/db/d7b/d16/db1 0 2024-03-29T08:07:20.197 INFO:tasks.workunit.client.1.smithi148.stdout:7/493: creat d8/d38/d3c/da9/fac x:0 0 0 2024-03-29T08:07:20.198 INFO:tasks.workunit.client.1.smithi148.stdout:4/538: write df/f4d [4874754,32247] 0 2024-03-29T08:07:20.201 INFO:tasks.workunit.client.1.smithi148.stdout:6/545: link d6/d9/d1e/d20/c4c d6/d9/d1e/d20/d6b/cae 0 2024-03-29T08:07:20.202 INFO:tasks.workunit.client.1.smithi148.stdout:0/534: mkdir db/de/d8e/da9 0 2024-03-29T08:07:20.203 INFO:tasks.workunit.client.0.smithi037.stdout:7/393: chown dd/d68/f78 2086741 1 2024-03-29T08:07:20.204 INFO:tasks.workunit.client.1.smithi148.stdout:2/554: creat d7/d8c/da4/fb0 x:0 0 0 2024-03-29T08:07:20.205 INFO:tasks.workunit.client.0.smithi037.stdout:9/575: symlink d1/d4/d19/d69/lc7 0 2024-03-29T08:07:20.206 INFO:tasks.workunit.client.1.smithi148.stdout:5/534: creat d0/db/d1e/fb2 x:0 0 0 2024-03-29T08:07:20.206 INFO:tasks.workunit.client.1.smithi148.stdout:3/564: truncate de/d24/d33/d39/d46/d30/d37/d62/d63/f99 451135 0 2024-03-29T08:07:20.207 INFO:tasks.workunit.client.1.smithi148.stdout:0/535: dread - db/dc/d78/f9a zero size 2024-03-29T08:07:20.209 INFO:tasks.workunit.client.0.smithi037.stdout:4/399: dwrite d3/d4/d15/d1d/db/d3d/f56 [0,4194304] 0 2024-03-29T08:07:20.209 INFO:tasks.workunit.client.0.smithi037.stdout:3/383: write f5 [1434880,58379] 0 2024-03-29T08:07:20.210 INFO:tasks.workunit.client.1.smithi148.stdout:7/494: dread d8/d29/d44/d50/f8b [0,4194304] 0 2024-03-29T08:07:20.211 INFO:tasks.workunit.client.1.smithi148.stdout:8/555: creat d17/d18/d26/d4f/fb7 x:0 0 0 2024-03-29T08:07:20.211 INFO:tasks.workunit.client.1.smithi148.stdout:6/546: write d6/d9/d1e/d27/f32 [967255,28236] 0 2024-03-29T08:07:20.211 INFO:tasks.workunit.client.1.smithi148.stdout:4/539: truncate df/d2c/d51/f57 1655817 0 2024-03-29T08:07:20.215 INFO:tasks.workunit.client.1.smithi148.stdout:5/535: dread - d0/db/d1e/fa7 zero size 2024-03-29T08:07:20.217 INFO:tasks.workunit.client.0.smithi037.stdout:6/352: rmdir d0/d4f 39 2024-03-29T08:07:20.221 INFO:tasks.workunit.client.0.smithi037.stdout:1/464: rename d2/d6/dc/d18/d22/d4f/d9f to d2/d4b/da1 0 2024-03-29T08:07:20.223 INFO:tasks.workunit.client.0.smithi037.stdout:7/394: creat dd/d21/d35/d42/d5c/f82 x:0 0 0 2024-03-29T08:07:20.224 INFO:tasks.workunit.client.1.smithi148.stdout:8/556: fsync d17/d74/d39/f77 0 2024-03-29T08:07:20.224 INFO:tasks.workunit.client.0.smithi037.stdout:4/400: rmdir d3/d4 39 2024-03-29T08:07:20.225 INFO:tasks.workunit.client.0.smithi037.stdout:0/280: rename d9/da/dd/d1c to d9/da/dd/d59 0 2024-03-29T08:07:20.227 INFO:tasks.workunit.client.0.smithi037.stdout:0/281: truncate d9/f50 111788 0 2024-03-29T08:07:20.227 INFO:tasks.workunit.client.1.smithi148.stdout:2/555: symlink d7/da/d33/d45/d4d/d82/d8d/lb1 0 2024-03-29T08:07:20.228 INFO:tasks.workunit.client.1.smithi148.stdout:6/547: creat d6/d9/d1e/d20/d30/d51/faf x:0 0 0 2024-03-29T08:07:20.231 INFO:tasks.workunit.client.1.smithi148.stdout:7/495: truncate d8/fe 284839 0 2024-03-29T08:07:20.232 INFO:tasks.workunit.client.0.smithi037.stdout:6/353: mkdir d0/d5/d1e/d44/d72/d77 0 2024-03-29T08:07:20.234 INFO:tasks.workunit.client.1.smithi148.stdout:8/557: mkdir d17/d49/d9a/db8 0 2024-03-29T08:07:20.234 INFO:tasks.workunit.client.0.smithi037.stdout:7/395: dread - dd/d1d/d34/d54/f5f zero size 2024-03-29T08:07:20.235 INFO:tasks.workunit.client.0.smithi037.stdout:1/465: truncate d2/f3a 363748 0 2024-03-29T08:07:20.235 INFO:tasks.workunit.client.1.smithi148.stdout:3/565: rmdir de/d24/d33/d39/d71 39 2024-03-29T08:07:20.238 INFO:tasks.workunit.client.1.smithi148.stdout:2/556: stat d7/da/c11 0 2024-03-29T08:07:20.239 INFO:tasks.workunit.client.1.smithi148.stdout:8/558: write d17/d74/f48 [2116587,90531] 0 2024-03-29T08:07:20.239 INFO:tasks.workunit.client.1.smithi148.stdout:0/536: rename db/de/df/d2b/d3b to db/dc/daa 0 2024-03-29T08:07:20.239 INFO:tasks.workunit.client.0.smithi037.stdout:3/384: link dd/d46/d2d/d3a/d5f/c34 dd/d46/c82 0 2024-03-29T08:07:20.241 INFO:tasks.workunit.client.1.smithi148.stdout:9/538: dwrite db/d22/d26/d54/f45 [0,4194304] 0 2024-03-29T08:07:20.241 INFO:tasks.workunit.client.1.smithi148.stdout:6/548: write d6/d9/d68/fad [291529,49348] 0 2024-03-29T08:07:20.244 INFO:tasks.workunit.client.0.smithi037.stdout:0/282: chown d9/da/d1b/l30 7871944 1 2024-03-29T08:07:20.245 INFO:tasks.workunit.client.1.smithi148.stdout:7/496: creat d8/d38/d3c/da5/fad x:0 0 0 2024-03-29T08:07:20.249 INFO:tasks.workunit.client.1.smithi148.stdout:5/536: read d0/db/d7b/d16/f7a [779022,80956] 0 2024-03-29T08:07:20.250 INFO:tasks.workunit.client.0.smithi037.stdout:8/435: truncate d5/f80 337771 0 2024-03-29T08:07:20.250 INFO:tasks.workunit.client.0.smithi037.stdout:5/542: sync 2024-03-29T08:07:20.251 INFO:tasks.workunit.client.0.smithi037.stdout:9/576: rename d1/d1f/c73 to d1/d4/d19/cc8 0 2024-03-29T08:07:20.259 INFO:tasks.workunit.client.0.smithi037.stdout:5/543: write d8/f42 [748978,31343] 0 2024-03-29T08:07:20.259 INFO:tasks.workunit.client.0.smithi037.stdout:8/436: readlink d5/d10/d19/d3d/d4a/d4c/l56 0 2024-03-29T08:07:20.259 INFO:tasks.workunit.client.0.smithi037.stdout:2/271: dread d0/fd [0,4194304] 0 2024-03-29T08:07:20.262 INFO:tasks.workunit.client.1.smithi148.stdout:5/537: stat d0/db/d7b/d16/d2e/f56 0 2024-03-29T08:07:20.262 INFO:tasks.workunit.client.1.smithi148.stdout:3/566: rmdir de/d3b/d49/da5 0 2024-03-29T08:07:20.264 INFO:tasks.workunit.client.1.smithi148.stdout:1/524: dwrite d3/de/d25/d51/f73 [0,4194304] 0 2024-03-29T08:07:20.265 INFO:tasks.workunit.client.1.smithi148.stdout:6/549: creat d6/fb0 x:0 0 0 2024-03-29T08:07:20.267 INFO:tasks.workunit.client.1.smithi148.stdout:2/557: creat d7/da/d33/d3c/d54/fb2 x:0 0 0 2024-03-29T08:07:20.268 INFO:tasks.workunit.client.1.smithi148.stdout:3/567: stat de/f61 0 2024-03-29T08:07:20.268 INFO:tasks.workunit.client.0.smithi037.stdout:1/466: creat d2/d6/dc/d58/fa2 x:0 0 0 2024-03-29T08:07:20.268 INFO:tasks.workunit.client.0.smithi037.stdout:4/401: dwrite d3/d4/d27/f4b [0,4194304] 0 2024-03-29T08:07:20.269 INFO:tasks.workunit.client.1.smithi148.stdout:5/538: fdatasync d0/db/d1e/d35/d42/d67/f70 0 2024-03-29T08:07:20.269 INFO:tasks.workunit.client.0.smithi037.stdout:5/544: fdatasync d8/d22/d4f/d70/d79/f92 0 2024-03-29T08:07:20.274 INFO:tasks.workunit.client.1.smithi148.stdout:6/550: stat d6/d9/d1e/d20/d30/l72 0 2024-03-29T08:07:20.277 INFO:tasks.workunit.client.1.smithi148.stdout:2/558: write d7/f1b [3549510,97588] 0 2024-03-29T08:07:20.279 INFO:tasks.workunit.client.0.smithi037.stdout:7/396: symlink dd/d72/l83 0 2024-03-29T08:07:20.280 INFO:tasks.workunit.client.0.smithi037.stdout:2/272: symlink d0/d20/d2e/l5d 0 2024-03-29T08:07:20.281 INFO:tasks.workunit.client.1.smithi148.stdout:0/537: dread db/de/df/d11/d2d/d3e/f45 [0,4194304] 0 2024-03-29T08:07:20.282 INFO:tasks.workunit.client.1.smithi148.stdout:5/539: unlink d0/db/d1e/d35/f5c 0 2024-03-29T08:07:20.291 INFO:tasks.workunit.client.1.smithi148.stdout:7/497: dwrite d8/d29/d44/d49/f76 [0,4194304] 0 2024-03-29T08:07:20.291 INFO:tasks.workunit.client.1.smithi148.stdout:3/568: dread de/d24/d33/d39/d71/f9b [0,4194304] 0 2024-03-29T08:07:20.292 INFO:tasks.workunit.client.1.smithi148.stdout:2/559: chown d7/da/d33/d3c/l80 652 1 2024-03-29T08:07:20.293 INFO:tasks.workunit.client.0.smithi037.stdout:1/467: unlink d2/d6/dc/d18/d22/d4f/c65 0 2024-03-29T08:07:20.293 INFO:tasks.workunit.client.0.smithi037.stdout:6/354: dread d0/d5/d3f/f1b [0,4194304] 0 2024-03-29T08:07:20.301 INFO:tasks.workunit.client.1.smithi148.stdout:3/569: rmdir de/d24/d33/d39/d46 39 2024-03-29T08:07:20.306 INFO:tasks.workunit.client.1.smithi148.stdout:7/498: creat d8/d38/d3c/da5/fae x:0 0 0 2024-03-29T08:07:20.306 INFO:tasks.workunit.client.1.smithi148.stdout:2/560: symlink d7/da/d33/lb3 0 2024-03-29T08:07:20.306 INFO:tasks.workunit.client.1.smithi148.stdout:9/539: dwrite db/f43 [4194304,4194304] 0 2024-03-29T08:07:20.308 INFO:tasks.workunit.client.0.smithi037.stdout:0/283: write d9/da/d1b/d22/d2c/f57 [3144524,27414] 0 2024-03-29T08:07:20.314 INFO:tasks.workunit.client.1.smithi148.stdout:8/559: dwrite d17/d4e/f71 [0,4194304] 0 2024-03-29T08:07:20.315 INFO:tasks.workunit.client.1.smithi148.stdout:3/570: creat de/d3b/fb8 x:0 0 0 2024-03-29T08:07:20.315 INFO:tasks.workunit.client.1.smithi148.stdout:9/540: dread - db/d10/faf zero size 2024-03-29T08:07:20.317 INFO:tasks.workunit.client.1.smithi148.stdout:2/561: write d7/da/d33/d45/d4d/d2e/d37/f6d [832073,18141] 0 2024-03-29T08:07:20.319 INFO:tasks.workunit.client.1.smithi148.stdout:4/540: dread df/f4d [0,4194304] 0 2024-03-29T08:07:20.320 INFO:tasks.workunit.client.1.smithi148.stdout:7/499: truncate d8/d14/d45/f66 3907720 0 2024-03-29T08:07:20.320 INFO:tasks.workunit.client.1.smithi148.stdout:8/560: chown d17/d4e/ca4 411 1 2024-03-29T08:07:20.323 INFO:tasks.workunit.client.1.smithi148.stdout:1/525: truncate d3/de/d25/d35/d42/d23/f40 2024887 0 2024-03-29T08:07:20.325 INFO:tasks.workunit.client.1.smithi148.stdout:5/540: dread d0/db/d7b/d16/d2e/f68 [0,4194304] 0 2024-03-29T08:07:20.326 INFO:tasks.workunit.client.0.smithi037.stdout:3/385: dwrite dd/d15/f16 [4194304,4194304] 0 2024-03-29T08:07:20.329 INFO:tasks.workunit.client.1.smithi148.stdout:4/541: fdatasync df/d2c/d51/d77/f83 0 2024-03-29T08:07:20.332 INFO:tasks.workunit.client.1.smithi148.stdout:9/541: unlink db/d22/d26/f2a 0 2024-03-29T08:07:20.333 INFO:tasks.workunit.client.0.smithi037.stdout:9/577: dread d1/d4/d19/d1e/d4a/d6f/da2/f3d [0,4194304] 0 2024-03-29T08:07:20.337 INFO:tasks.workunit.client.1.smithi148.stdout:0/538: rmdir db/de/df/d11/d42/da0 0 2024-03-29T08:07:20.340 INFO:tasks.workunit.client.0.smithi037.stdout:5/545: dwrite d8/f2b [0,4194304] 0 2024-03-29T08:07:20.341 INFO:tasks.workunit.client.1.smithi148.stdout:4/542: symlink df/d2c/d51/d77/d17/d6a/lb1 0 2024-03-29T08:07:20.342 INFO:tasks.workunit.client.1.smithi148.stdout:2/562: getdents d7/da/d33/d45/d4d/d41 0 2024-03-29T08:07:20.342 INFO:tasks.workunit.client.1.smithi148.stdout:7/500: fsync d8/d29/d44/d49/f76 0 2024-03-29T08:07:20.345 INFO:tasks.workunit.client.1.smithi148.stdout:8/561: unlink d17/d18/d26/d4f/c90 0 2024-03-29T08:07:20.347 INFO:tasks.workunit.client.1.smithi148.stdout:1/526: creat d3/de/d25/f9d x:0 0 0 2024-03-29T08:07:20.347 INFO:tasks.workunit.client.1.smithi148.stdout:3/571: dwrite de/d24/f31 [4194304,4194304] 0 2024-03-29T08:07:20.351 INFO:tasks.workunit.client.0.smithi037.stdout:8/437: dwrite d5/f48 [0,4194304] 0 2024-03-29T08:07:20.353 INFO:tasks.workunit.client.1.smithi148.stdout:2/563: write d7/d24/d4b/d66/d7d/f8f [8683,98514] 0 2024-03-29T08:07:20.354 INFO:tasks.workunit.client.1.smithi148.stdout:0/539: unlink db/dc/daa/d47/d4c/d5a/f7e 0 2024-03-29T08:07:20.356 INFO:tasks.workunit.client.1.smithi148.stdout:7/501: truncate d8/d14/f2b 4503951 0 2024-03-29T08:07:20.358 INFO:tasks.workunit.client.0.smithi037.stdout:4/402: rmdir d3/d80/d81 0 2024-03-29T08:07:20.359 INFO:tasks.workunit.client.0.smithi037.stdout:3/386: dread dd/d15/f1e [0,4194304] 0 2024-03-29T08:07:20.360 INFO:tasks.workunit.client.0.smithi037.stdout:7/397: mkdir dd/d1d/d49/d81/d84 0 2024-03-29T08:07:20.363 INFO:tasks.workunit.client.0.smithi037.stdout:2/273: mkdir d0/d23/d59/d5e 0 2024-03-29T08:07:20.363 INFO:tasks.workunit.client.1.smithi148.stdout:3/572: creat de/d24/d33/d39/d71/fb9 x:0 0 0 2024-03-29T08:07:20.367 INFO:tasks.workunit.client.1.smithi148.stdout:3/573: readlink de/l45 0 2024-03-29T08:07:20.367 INFO:tasks.workunit.client.0.smithi037.stdout:6/355: creat d0/d4f/f78 x:0 0 0 2024-03-29T08:07:20.371 INFO:tasks.workunit.client.1.smithi148.stdout:7/502: creat d8/d38/d3c/da9/d37/faf x:0 0 0 2024-03-29T08:07:20.372 INFO:tasks.workunit.client.0.smithi037.stdout:3/387: chown dd/d46/d2d/d3a/d5f/f77 126110215 1 2024-03-29T08:07:20.372 INFO:tasks.workunit.client.0.smithi037.stdout:6/356: fsync d0/f33 0 2024-03-29T08:07:20.374 INFO:tasks.workunit.client.1.smithi148.stdout:6/551: dread d6/f19 [0,4194304] 0 2024-03-29T08:07:20.375 INFO:tasks.workunit.client.1.smithi148.stdout:2/564: creat d7/d24/d4b/d66/d7d/fb4 x:0 0 0 2024-03-29T08:07:20.377 INFO:tasks.workunit.client.0.smithi037.stdout:4/403: mkdir d3/d57/d85 0 2024-03-29T08:07:20.378 INFO:tasks.workunit.client.0.smithi037.stdout:0/284: mknod d9/da/dd/d20/d38/d53/c5a 0 2024-03-29T08:07:20.379 INFO:tasks.workunit.client.0.smithi037.stdout:8/438: write d5/f16 [535696,103977] 0 2024-03-29T08:07:20.379 INFO:tasks.workunit.client.0.smithi037.stdout:6/357: dread - d0/d19/d34/f40 zero size 2024-03-29T08:07:20.381 INFO:tasks.workunit.client.1.smithi148.stdout:6/552: chown d6/d9/d1e/d27/f6e 833957 1 2024-03-29T08:07:20.382 INFO:tasks.workunit.client.1.smithi148.stdout:1/527: rename d3/de/d25/d35/d42/d23 to d3/d18/d4d/d9e 0 2024-03-29T08:07:20.382 INFO:tasks.workunit.client.1.smithi148.stdout:5/541: dread d0/db/d7b/d16/d87/fad [0,4194304] 0 2024-03-29T08:07:20.384 INFO:tasks.workunit.client.0.smithi037.stdout:5/546: mkdir d8/d22/db6 0 2024-03-29T08:07:20.385 INFO:tasks.workunit.client.1.smithi148.stdout:8/562: dwrite d17/d18/f4d [0,4194304] 0 2024-03-29T08:07:20.385 INFO:tasks.workunit.client.0.smithi037.stdout:3/388: mkdir dd/d15/d27/d33/d83 0 2024-03-29T08:07:20.386 INFO:tasks.workunit.client.0.smithi037.stdout:6/358: read d0/f4 [3017579,49314] 0 2024-03-29T08:07:20.389 INFO:tasks.workunit.client.1.smithi148.stdout:9/542: truncate db/d22/d26/d54/f3c 110916 0 2024-03-29T08:07:20.395 INFO:tasks.workunit.client.0.smithi037.stdout:9/578: rename d1/d4/d19/d1e/c59 to d1/d4/cc9 0 2024-03-29T08:07:20.395 INFO:tasks.workunit.client.1.smithi148.stdout:1/528: mknod d3/de/d1d/d47/d6d/c9f 0 2024-03-29T08:07:20.397 INFO:tasks.workunit.client.1.smithi148.stdout:4/543: truncate df/f4d 607645 0 2024-03-29T08:07:20.398 INFO:tasks.workunit.client.0.smithi037.stdout:2/274: dwrite d0/d20/f49 [0,4194304] 0 2024-03-29T08:07:20.398 INFO:tasks.workunit.client.1.smithi148.stdout:4/544: readlink df/d2c/l5a 0 2024-03-29T08:07:20.401 INFO:tasks.workunit.client.1.smithi148.stdout:5/542: mknod d0/db/dae/cb3 0 2024-03-29T08:07:20.404 INFO:tasks.workunit.client.0.smithi037.stdout:2/275: write d0/d23/d44/f46 [1007003,18206] 0 2024-03-29T08:07:20.408 INFO:tasks.workunit.client.0.smithi037.stdout:1/468: getdents d2/d6/dc/d5d/d64/d87 0 2024-03-29T08:07:20.408 INFO:tasks.workunit.client.1.smithi148.stdout:1/529: fsync d3/de/d6e/f79 0 2024-03-29T08:07:20.412 INFO:tasks.workunit.client.1.smithi148.stdout:4/545: chown df/d2c/d51/d77/f56 11899971 1 2024-03-29T08:07:20.415 INFO:tasks.workunit.client.1.smithi148.stdout:5/543: creat d0/fb4 x:0 0 0 2024-03-29T08:07:20.416 INFO:tasks.workunit.client.0.smithi037.stdout:6/359: dread d0/d5/f2a [0,4194304] 0 2024-03-29T08:07:20.416 INFO:tasks.workunit.client.0.smithi037.stdout:0/285: dread d9/da/dd/d13/f21 [0,4194304] 0 2024-03-29T08:07:20.416 INFO:tasks.workunit.client.1.smithi148.stdout:5/544: read d0/db/d1e/d35/d42/f5e [96907,72629] 0 2024-03-29T08:07:20.418 INFO:tasks.workunit.client.1.smithi148.stdout:7/503: rename d8/d29/d44/d50/f52 to d8/d38/fb0 0 2024-03-29T08:07:20.418 INFO:tasks.workunit.client.1.smithi148.stdout:9/543: creat db/d22/d3f/d61/fb5 x:0 0 0 2024-03-29T08:07:20.421 INFO:tasks.workunit.client.0.smithi037.stdout:4/404: dwrite d3/d4/f34 [0,4194304] 0 2024-03-29T08:07:20.422 INFO:tasks.workunit.client.1.smithi148.stdout:1/530: creat d3/de/d21/d7f/fa0 x:0 0 0 2024-03-29T08:07:20.423 INFO:tasks.workunit.client.0.smithi037.stdout:0/286: dread - d9/da/dd/d13/f45 zero size 2024-03-29T08:07:20.425 INFO:tasks.workunit.client.0.smithi037.stdout:3/389: mkdir dd/d15/d27/d33/d84 0 2024-03-29T08:07:20.426 INFO:tasks.workunit.client.1.smithi148.stdout:5/545: dread d0/f12 [0,4194304] 0 2024-03-29T08:07:20.428 INFO:tasks.workunit.client.1.smithi148.stdout:5/546: dread - d0/db/d1e/fb2 zero size 2024-03-29T08:07:20.428 INFO:tasks.workunit.client.1.smithi148.stdout:7/504: symlink d8/d29/d44/d50/d94/lb1 0 2024-03-29T08:07:20.429 INFO:tasks.workunit.client.0.smithi037.stdout:0/287: dread d9/f50 [0,4194304] 0 2024-03-29T08:07:20.431 INFO:tasks.workunit.client.0.smithi037.stdout:8/439: mkdir d5/d10/d19/d8d 0 2024-03-29T08:07:20.431 INFO:tasks.workunit.client.1.smithi148.stdout:5/547: chown d0/db/d1e/d36/d7c/f80 8 1 2024-03-29T08:07:20.432 INFO:tasks.workunit.client.1.smithi148.stdout:4/546: mkdir df/d2c/db2 0 2024-03-29T08:07:20.433 INFO:tasks.workunit.client.0.smithi037.stdout:9/579: dread - d1/d4/d19/d1e/d4a/d6f/f86 zero size 2024-03-29T08:07:20.436 INFO:tasks.workunit.client.1.smithi148.stdout:7/505: read d8/f36 [2499939,66577] 0 2024-03-29T08:07:20.436 INFO:tasks.workunit.client.1.smithi148.stdout:4/547: chown df/d2c/d51/d77/la2 3 1 2024-03-29T08:07:20.436 INFO:tasks.workunit.client.0.smithi037.stdout:2/276: chown d0/d53/l57 60672 1 2024-03-29T08:07:20.437 INFO:tasks.workunit.client.1.smithi148.stdout:1/531: readlink d3/de/d4f/l82 0 2024-03-29T08:07:20.442 INFO:tasks.workunit.client.1.smithi148.stdout:2/565: write d7/da/d33/f35 [3603785,45432] 0 2024-03-29T08:07:20.442 INFO:tasks.workunit.client.1.smithi148.stdout:5/548: symlink d0/db/d1e/d5b/lb5 0 2024-03-29T08:07:20.445 INFO:tasks.workunit.client.1.smithi148.stdout:1/532: readlink d3/de/d21/l90 0 2024-03-29T08:07:20.446 INFO:tasks.workunit.client.1.smithi148.stdout:6/553: rename d6/d9/d1e/d20/d30/d51 to d6/d9/d1e/d36/db1 0 2024-03-29T08:07:20.448 INFO:tasks.workunit.client.1.smithi148.stdout:1/533: readlink d3/d18/l2e 0 2024-03-29T08:07:20.449 INFO:tasks.workunit.client.1.smithi148.stdout:3/574: dwrite de/d24/d33/d39/d7f/fac [0,4194304] 0 2024-03-29T08:07:20.449 INFO:tasks.workunit.client.1.smithi148.stdout:7/506: symlink d8/d38/d3c/da9/d37/lb2 0 2024-03-29T08:07:20.450 INFO:tasks.workunit.client.0.smithi037.stdout:4/405: creat d3/d4/d27/f86 x:0 0 0 2024-03-29T08:07:20.451 INFO:tasks.workunit.client.1.smithi148.stdout:4/548: chown df/d2c/d51/d77/d26/f7d 1 1 2024-03-29T08:07:20.453 INFO:tasks.workunit.client.1.smithi148.stdout:7/507: dread - d8/d38/f8d zero size 2024-03-29T08:07:20.454 INFO:tasks.workunit.client.1.smithi148.stdout:8/563: rename d17/d1d/d7e/f85 to d17/d74/d39/d6f/fb9 0 2024-03-29T08:07:20.454 INFO:tasks.workunit.client.1.smithi148.stdout:5/549: write d0/db/d1e/d79/faa [3575149,30469] 0 2024-03-29T08:07:20.455 INFO:tasks.workunit.client.0.smithi037.stdout:8/440: write d5/f1f [2980203,102348] 0 2024-03-29T08:07:20.457 INFO:tasks.workunit.client.1.smithi148.stdout:0/540: creat db/dc/daa/d47/d4c/d5a/fab x:0 0 0 2024-03-29T08:07:20.460 INFO:tasks.workunit.client.1.smithi148.stdout:4/549: write df/d2c/d51/d77/d26/f84 [1596948,122253] 0 2024-03-29T08:07:20.464 INFO:tasks.workunit.client.0.smithi037.stdout:1/469: link d2/f84 d2/d6/dc/d58/d67/fa3 0 2024-03-29T08:07:20.466 INFO:tasks.workunit.client.1.smithi148.stdout:9/544: rename db/d22/d26/f30 to db/d10/d9d/fb6 0 2024-03-29T08:07:20.468 INFO:tasks.workunit.client.0.smithi037.stdout:2/277: mkdir d0/d20/d2e/d5f 0 2024-03-29T08:07:20.471 INFO:tasks.workunit.client.0.smithi037.stdout:7/398: write dd/d1d/d34/f59 [1423404,118395] 0 2024-03-29T08:07:20.474 INFO:tasks.workunit.client.1.smithi148.stdout:7/508: mkdir d8/d38/d6a/db3 0 2024-03-29T08:07:20.474 INFO:tasks.workunit.client.0.smithi037.stdout:5/547: link d8/d22/d37/d8d/l95 d8/d22/d4f/lb7 0 2024-03-29T08:07:20.475 INFO:tasks.workunit.client.1.smithi148.stdout:2/566: dread d7/da/f3f [0,4194304] 0 2024-03-29T08:07:20.477 INFO:tasks.workunit.client.1.smithi148.stdout:1/534: rename d3/de/d25/d35/f3c to d3/de/d1d/fa1 0 2024-03-29T08:07:20.479 INFO:tasks.workunit.client.1.smithi148.stdout:5/550: link d0/db/d1e/d79/faa d0/db/d7b/fb6 0 2024-03-29T08:07:20.480 INFO:tasks.workunit.client.1.smithi148.stdout:8/564: creat d17/d49/d9a/db8/fba x:0 0 0 2024-03-29T08:07:20.481 INFO:tasks.workunit.client.1.smithi148.stdout:6/554: truncate d6/d9/d1e/d36/d4d/d9d/f4a 1838627 0 2024-03-29T08:07:20.482 INFO:tasks.workunit.client.0.smithi037.stdout:3/390: mkdir dd/d46/d2d/d3a/d85 0 2024-03-29T08:07:20.483 INFO:tasks.workunit.client.1.smithi148.stdout:2/567: mkdir d7/d24/db5 0 2024-03-29T08:07:20.484 INFO:tasks.workunit.client.1.smithi148.stdout:3/575: dwrite de/d24/d33/d39/d46/f23 [0,4194304] 0 2024-03-29T08:07:20.484 INFO:tasks.workunit.client.1.smithi148.stdout:4/550: mkdir df/d2c/d51/d77/d26/d34/db3 0 2024-03-29T08:07:20.485 INFO:tasks.workunit.client.1.smithi148.stdout:9/545: creat db/d22/d26/d54/d52/d9c/fb7 x:0 0 0 2024-03-29T08:07:20.485 INFO:tasks.workunit.client.1.smithi148.stdout:7/509: mkdir d8/d38/d3c/da5/db4 0 2024-03-29T08:07:20.486 INFO:tasks.workunit.client.1.smithi148.stdout:5/551: chown d0/db/d1e/l77 1200875 1 2024-03-29T08:07:20.488 INFO:tasks.workunit.client.1.smithi148.stdout:4/551: readlink df/d2c/d51/d77/d26/d34/l3c 0 2024-03-29T08:07:20.490 INFO:tasks.workunit.client.0.smithi037.stdout:9/580: dread d1/d3/f21 [0,4194304] 0 2024-03-29T08:07:20.492 INFO:tasks.workunit.client.0.smithi037.stdout:9/581: fsync d1/d4/d19/d1e/fae 0 2024-03-29T08:07:20.496 INFO:tasks.workunit.client.1.smithi148.stdout:9/546: fsync db/d22/d26/d54/d4d/f18 0 2024-03-29T08:07:20.496 INFO:tasks.workunit.client.1.smithi148.stdout:0/541: mkdir db/de/df/d11/d2d/da3/dac 0 2024-03-29T08:07:20.502 INFO:tasks.workunit.client.0.smithi037.stdout:0/288: getdents d9/da/dd/d20/d55 0 2024-03-29T08:07:20.504 INFO:tasks.workunit.client.1.smithi148.stdout:9/547: symlink db/d20/lb8 0 2024-03-29T08:07:20.505 INFO:tasks.workunit.client.0.smithi037.stdout:0/289: chown d9/da/dd/l58 254517075 1 2024-03-29T08:07:20.509 INFO:tasks.workunit.client.1.smithi148.stdout:1/535: dread d3/fa [4194304,4194304] 0 2024-03-29T08:07:20.509 INFO:tasks.workunit.client.1.smithi148.stdout:2/568: dwrite d7/f19 [0,4194304] 0 2024-03-29T08:07:20.511 INFO:tasks.workunit.client.1.smithi148.stdout:6/555: getdents d6/d9/d1e/d20/d30 0 2024-03-29T08:07:20.511 INFO:tasks.workunit.client.1.smithi148.stdout:7/510: creat d8/d14/d45/d4d/d78/fb5 x:0 0 0 2024-03-29T08:07:20.514 INFO:tasks.workunit.client.0.smithi037.stdout:6/360: truncate d0/d5/d3f/f17 2333893 0 2024-03-29T08:07:20.514 INFO:tasks.workunit.client.1.smithi148.stdout:0/542: dread - db/dc/daa/d47/d4c/f5f zero size 2024-03-29T08:07:20.515 INFO:tasks.workunit.client.1.smithi148.stdout:5/552: getdents d0/db/d1e/d36 0 2024-03-29T08:07:20.517 INFO:tasks.workunit.client.1.smithi148.stdout:0/543: chown db/dc/c87 3483632 1 2024-03-29T08:07:20.520 INFO:tasks.workunit.client.0.smithi037.stdout:9/582: unlink d1/d4/d19/d69/d75/f8a 0 2024-03-29T08:07:20.520 INFO:tasks.workunit.client.1.smithi148.stdout:6/556: readlink d6/d9/d1e/d20/l54 0 2024-03-29T08:07:20.525 INFO:tasks.workunit.client.1.smithi148.stdout:2/569: mkdir d7/da/d33/d3c/d54/db6 0 2024-03-29T08:07:20.525 INFO:tasks.workunit.client.1.smithi148.stdout:6/557: dread - d6/d9/d1e/d20/d30/f93 zero size 2024-03-29T08:07:20.527 INFO:tasks.workunit.client.1.smithi148.stdout:0/544: mknod db/de/df/d11/d1f/cad 0 2024-03-29T08:07:20.527 INFO:tasks.workunit.client.1.smithi148.stdout:7/511: rmdir d8/d38 39 2024-03-29T08:07:20.528 INFO:tasks.workunit.client.1.smithi148.stdout:9/548: creat db/d22/d3f/d61/fb9 x:0 0 0 2024-03-29T08:07:20.529 INFO:tasks.workunit.client.1.smithi148.stdout:8/565: write d17/d18/f4d [4249402,20055] 0 2024-03-29T08:07:20.529 INFO:tasks.workunit.client.0.smithi037.stdout:8/441: creat d5/d10/d19/d3d/d4a/d4c/d67/d85/f8e x:0 0 0 2024-03-29T08:07:20.531 INFO:tasks.workunit.client.0.smithi037.stdout:2/278: creat d0/d23/d59/d5e/f60 x:0 0 0 2024-03-29T08:07:20.532 INFO:tasks.workunit.client.1.smithi148.stdout:0/545: stat db/de/df/d14 0 2024-03-29T08:07:20.534 INFO:tasks.workunit.client.0.smithi037.stdout:5/548: creat d8/d22/d37/d8d/db3/fb8 x:0 0 0 2024-03-29T08:07:20.536 INFO:tasks.workunit.client.0.smithi037.stdout:3/391: symlink dd/d15/d27/d33/d84/l86 0 2024-03-29T08:07:20.537 INFO:tasks.workunit.client.1.smithi148.stdout:2/570: mkdir d7/da/d33/d45/d4d/db7 0 2024-03-29T08:07:20.537 INFO:tasks.workunit.client.1.smithi148.stdout:6/558: rmdir d6/d9/d1e/d36/d4d/d9d/d28 39 2024-03-29T08:07:20.538 INFO:tasks.workunit.client.0.smithi037.stdout:6/361: fdatasync d0/d19/d34/f40 0 2024-03-29T08:07:20.538 INFO:tasks.workunit.client.0.smithi037.stdout:2/279: write d0/d20/f3d [679933,41881] 0 2024-03-29T08:07:20.538 INFO:tasks.workunit.client.1.smithi148.stdout:7/512: write d8/d29/d44/d49/f76 [4546071,7310] 0 2024-03-29T08:07:20.541 INFO:tasks.workunit.client.1.smithi148.stdout:4/552: write df/f4d [244629,125964] 0 2024-03-29T08:07:20.542 INFO:tasks.workunit.client.1.smithi148.stdout:1/536: dwrite d3/d55/f37 [0,4194304] 0 2024-03-29T08:07:20.542 INFO:tasks.workunit.client.1.smithi148.stdout:8/566: fdatasync d17/d18/f36 0 2024-03-29T08:07:20.545 INFO:tasks.workunit.client.0.smithi037.stdout:4/406: dwrite d3/d4/f2f [4194304,4194304] 0 2024-03-29T08:07:20.545 INFO:tasks.workunit.client.0.smithi037.stdout:0/290: rename d9/f50 to d9/da/dd/f5b 0 2024-03-29T08:07:20.546 INFO:tasks.workunit.client.0.smithi037.stdout:6/362: write d0/d19/d34/f66 [817212,67869] 0 2024-03-29T08:07:20.547 INFO:tasks.workunit.client.1.smithi148.stdout:9/549: creat db/d22/d26/d54/d4d/d35/d4b/d5a/d74/fba x:0 0 0 2024-03-29T08:07:20.547 INFO:tasks.workunit.client.1.smithi148.stdout:5/553: write d0/db/f3e [318449,88994] 0 2024-03-29T08:07:20.550 INFO:tasks.workunit.client.0.smithi037.stdout:7/399: getdents dd/d1d/d34/d54 0 2024-03-29T08:07:20.552 INFO:tasks.workunit.client.1.smithi148.stdout:6/559: write d6/d9/d1e/d27/f7f [5699387,61270] 0 2024-03-29T08:07:20.553 INFO:tasks.workunit.client.0.smithi037.stdout:6/363: dread - d0/d19/d34/f6b zero size 2024-03-29T08:07:20.557 INFO:tasks.workunit.client.1.smithi148.stdout:8/567: write d17/d74/d39/d6f/f45 [4357929,63191] 0 2024-03-29T08:07:20.559 INFO:tasks.workunit.client.1.smithi148.stdout:2/571: getdents d7/d8c/da4/d8e 0 2024-03-29T08:07:20.561 INFO:tasks.workunit.client.1.smithi148.stdout:1/537: link d3/de/d1d/l69 d3/de/d6e/la2 0 2024-03-29T08:07:20.562 INFO:tasks.workunit.client.0.smithi037.stdout:6/364: truncate d0/d6a/f70 123559 0 2024-03-29T08:07:20.562 INFO:tasks.workunit.client.0.smithi037.stdout:4/407: creat d3/d4/d15/d1d/d4f/f87 x:0 0 0 2024-03-29T08:07:20.563 INFO:tasks.workunit.client.1.smithi148.stdout:3/576: dwrite de/d3b/d49/f64 [0,4194304] 0 2024-03-29T08:07:20.563 INFO:tasks.workunit.client.1.smithi148.stdout:4/553: write df/d2c/d51/d77/d26/d34/d8a/f8c [216445,34987] 0 2024-03-29T08:07:20.565 INFO:tasks.workunit.client.0.smithi037.stdout:6/365: chown d0/d5/d1e/d44/d71 20506 1 2024-03-29T08:07:20.567 INFO:tasks.workunit.client.0.smithi037.stdout:7/400: rename dd/d21/d35/d42/d6c to dd/d21/d85 0 2024-03-29T08:07:20.568 INFO:tasks.workunit.client.0.smithi037.stdout:3/392: link dd/l17 dd/d46/d2d/d3a/d6c/l87 0 2024-03-29T08:07:20.568 INFO:tasks.workunit.client.1.smithi148.stdout:5/554: creat d0/db/d1e/d5b/d66/fb7 x:0 0 0 2024-03-29T08:07:20.568 INFO:tasks.workunit.client.1.smithi148.stdout:2/572: readlink d7/da/d33/d3c/l30 0 2024-03-29T08:07:20.573 INFO:tasks.workunit.client.1.smithi148.stdout:1/538: creat d3/de/d21/d5f/fa3 x:0 0 0 2024-03-29T08:07:20.573 INFO:tasks.workunit.client.1.smithi148.stdout:3/577: dread - de/d24/d33/d39/d46/d70/f83 zero size 2024-03-29T08:07:20.575 INFO:tasks.workunit.client.0.smithi037.stdout:4/408: fsync d3/f44 0 2024-03-29T08:07:20.579 INFO:tasks.workunit.client.1.smithi148.stdout:4/554: fsync df/d2c/d51/d77/d26/d34/f5c 0 2024-03-29T08:07:20.579 INFO:tasks.workunit.client.1.smithi148.stdout:8/568: fdatasync d17/d74/f54 0 2024-03-29T08:07:20.581 INFO:tasks.workunit.client.0.smithi037.stdout:2/280: link d0/c1d d0/d20/d22/c61 0 2024-03-29T08:07:20.586 INFO:tasks.workunit.client.1.smithi148.stdout:0/546: dwrite db/de/df/d11/d1f/f9b [0,4194304] 0 2024-03-29T08:07:20.586 INFO:tasks.workunit.client.0.smithi037.stdout:0/291: dwrite d9/da/dd/d20/f4e [0,4194304] 0 2024-03-29T08:07:20.588 INFO:tasks.workunit.client.0.smithi037.stdout:9/583: write d1/d4/d19/d1e/d4a/d6f/d96/fa4 [53129,129537] 0 2024-03-29T08:07:20.589 INFO:tasks.workunit.client.1.smithi148.stdout:6/560: rename d6/f41 to d6/d9/d1e/d36/d4d/d9d/fb2 0 2024-03-29T08:07:20.591 INFO:tasks.workunit.client.0.smithi037.stdout:7/401: read dd/d1d/d34/f3f [780479,34369] 0 2024-03-29T08:07:20.592 INFO:tasks.workunit.client.1.smithi148.stdout:1/539: chown d3/d18/d4d/l5c 6 1 2024-03-29T08:07:20.592 INFO:tasks.workunit.client.0.smithi037.stdout:9/584: chown d1/d4/d19/d69/d75/f8b 9547 1 2024-03-29T08:07:20.594 INFO:tasks.workunit.client.1.smithi148.stdout:3/578: creat de/d24/d33/d39/d46/d30/d37/fba x:0 0 0 2024-03-29T08:07:20.595 INFO:tasks.workunit.client.0.smithi037.stdout:6/366: mkdir d0/d19/d34/d4e/d69/d79 0 2024-03-29T08:07:20.595 INFO:tasks.workunit.client.1.smithi148.stdout:5/555: dread d0/f9 [0,4194304] 0 2024-03-29T08:07:20.595 INFO:tasks.workunit.client.1.smithi148.stdout:4/555: read df/d2c/d51/d77/d26/d9b/da7/f94 [422279,6510] 0 2024-03-29T08:07:20.603 INFO:tasks.workunit.client.1.smithi148.stdout:6/561: mkdir d6/d9/d1e/d36/d4d/d9d/d28/db3 0 2024-03-29T08:07:20.603 INFO:tasks.workunit.client.1.smithi148.stdout:0/547: mkdir db/de/df/d2b/dae 0 2024-03-29T08:07:20.605 INFO:tasks.workunit.client.0.smithi037.stdout:8/442: truncate d5/d10/d2d/f60 744688 0 2024-03-29T08:07:20.609 INFO:tasks.workunit.client.1.smithi148.stdout:4/556: symlink df/d2c/d51/d77/d26/d34/d3e/lb4 0 2024-03-29T08:07:20.610 INFO:tasks.workunit.client.1.smithi148.stdout:5/556: symlink d0/db/d84/daf/lb8 0 2024-03-29T08:07:20.611 INFO:tasks.workunit.client.0.smithi037.stdout:4/409: dwrite d3/d4/d15/d1d/db/d3d/f58 [0,4194304] 0 2024-03-29T08:07:20.612 INFO:tasks.workunit.client.0.smithi037.stdout:9/585: mkdir d1/d4/d19/d1e/d4a/d5c/dca 0 2024-03-29T08:07:20.619 INFO:tasks.workunit.client.1.smithi148.stdout:7/513: truncate d8/d38/f71 1048671 0 2024-03-29T08:07:20.621 INFO:tasks.workunit.client.1.smithi148.stdout:4/557: creat df/d2c/d80/fb5 x:0 0 0 2024-03-29T08:07:20.623 INFO:tasks.workunit.client.1.smithi148.stdout:3/579: link de/d24/d33/d39/d46/d30/c9f de/d24/d33/d39/d46/d70/cbb 0 2024-03-29T08:07:20.625 INFO:tasks.workunit.client.0.smithi037.stdout:6/367: creat d0/d5/f7a x:0 0 0 2024-03-29T08:07:20.626 INFO:tasks.workunit.client.0.smithi037.stdout:7/402: link dd/d1d/c28 dd/d21/d35/d3b/c86 0 2024-03-29T08:07:20.627 INFO:tasks.workunit.client.0.smithi037.stdout:4/410: truncate d3/d4/d15/d1d/db/d45/f2d 1350290 0 2024-03-29T08:07:20.629 INFO:tasks.workunit.client.1.smithi148.stdout:0/548: write db/dc/daa/d47/d4c/d5a/f6a [827483,119778] 0 2024-03-29T08:07:20.630 INFO:tasks.workunit.client.0.smithi037.stdout:3/393: truncate dd/d46/f2c 335227 0 2024-03-29T08:07:20.631 INFO:tasks.workunit.client.0.smithi037.stdout:2/281: dread d0/f1b [0,4194304] 0 2024-03-29T08:07:20.633 INFO:tasks.workunit.client.1.smithi148.stdout:9/550: dread db/d22/d26/d54/d4d/d35/f48 [0,4194304] 0 2024-03-29T08:07:20.633 INFO:tasks.workunit.client.0.smithi037.stdout:9/586: creat d1/d4/d19/d1e/d25/fcb x:0 0 0 2024-03-29T08:07:20.634 INFO:tasks.workunit.client.1.smithi148.stdout:1/540: dwrite d3/de/d21/d5e/f99 [0,4194304] 0 2024-03-29T08:07:20.635 INFO:tasks.workunit.client.1.smithi148.stdout:2/573: write d7/da/d33/d45/d4d/f57 [3030127,124259] 0 2024-03-29T08:07:20.637 INFO:tasks.workunit.client.0.smithi037.stdout:4/411: mkdir d3/d54/d88 0 2024-03-29T08:07:20.639 INFO:tasks.workunit.client.0.smithi037.stdout:0/292: dwrite f5 [0,4194304] 0 2024-03-29T08:07:20.640 INFO:tasks.workunit.client.0.smithi037.stdout:3/394: rmdir dd/d15/d27 39 2024-03-29T08:07:20.641 INFO:tasks.workunit.client.1.smithi148.stdout:1/541: chown d3/d18/d4d/l5c 24266492 1 2024-03-29T08:07:20.641 INFO:tasks.workunit.client.1.smithi148.stdout:3/580: mkdir de/d24/d33/d39/da3/dbc 0 2024-03-29T08:07:20.644 INFO:tasks.workunit.client.0.smithi037.stdout:2/282: mkdir d0/d23/d44/d62 0 2024-03-29T08:07:20.645 INFO:tasks.workunit.client.1.smithi148.stdout:5/557: creat d0/db/d1e/d35/d42/d67/d76/d81/d98/fb9 x:0 0 0 2024-03-29T08:07:20.648 INFO:tasks.workunit.client.0.smithi037.stdout:4/412: truncate d3/d4/d15/f63 4312336 0 2024-03-29T08:07:20.654 INFO:tasks.workunit.client.1.smithi148.stdout:6/562: dwrite d6/d9/d1e/d20/d30/f47 [0,4194304] 0 2024-03-29T08:07:20.654 INFO:tasks.workunit.client.1.smithi148.stdout:5/558: readlink d0/db/d1e/d5b/lb5 0 2024-03-29T08:07:20.654 INFO:tasks.workunit.client.1.smithi148.stdout:9/551: chown db/d10/d2d/d4e/d5f/c69 7378042 1 2024-03-29T08:07:20.658 INFO:tasks.workunit.client.1.smithi148.stdout:1/542: symlink d3/de/d1d/la4 0 2024-03-29T08:07:20.658 INFO:tasks.workunit.client.1.smithi148.stdout:3/581: dread - de/d24/d33/d39/d46/d30/d37/f88 zero size 2024-03-29T08:07:20.659 INFO:tasks.workunit.client.1.smithi148.stdout:2/574: mknod d7/da/d33/d3c/cb8 0 2024-03-29T08:07:20.660 INFO:tasks.workunit.client.1.smithi148.stdout:5/559: chown d0/db/d1e 1819 1 2024-03-29T08:07:20.661 INFO:tasks.workunit.client.0.smithi037.stdout:6/368: rename d0/d5/d1e/d44/d72 to d0/d6a/d7b 0 2024-03-29T08:07:20.663 INFO:tasks.workunit.client.1.smithi148.stdout:9/552: chown db/d22/d26/d54/l80 697369849 1 2024-03-29T08:07:20.664 INFO:tasks.workunit.client.1.smithi148.stdout:1/543: stat d3/de/d1d/d47/d6d/c9f 0 2024-03-29T08:07:20.667 INFO:tasks.workunit.client.1.smithi148.stdout:6/563: mkdir d6/d9/d1e/d27/d48/d73/d7e/db4 0 2024-03-29T08:07:20.667 INFO:tasks.workunit.client.1.smithi148.stdout:8/569: dread d17/d18/f36 [0,4194304] 0 2024-03-29T08:07:20.668 INFO:tasks.workunit.client.0.smithi037.stdout:0/293: creat d9/da/dd/d13/f5c x:0 0 0 2024-03-29T08:07:20.672 INFO:tasks.workunit.client.1.smithi148.stdout:3/582: creat de/fbd x:0 0 0 2024-03-29T08:07:20.677 INFO:tasks.workunit.client.0.smithi037.stdout:3/395: dwrite dd/d15/f62 [0,4194304] 0 2024-03-29T08:07:20.677 INFO:tasks.workunit.client.0.smithi037.stdout:2/283: creat d0/d20/d2e/d5f/f63 x:0 0 0 2024-03-29T08:07:20.679 INFO:tasks.workunit.client.1.smithi148.stdout:5/560: dread - d0/db/d1e/f4a zero size 2024-03-29T08:07:20.680 INFO:tasks.workunit.client.1.smithi148.stdout:2/575: mknod d7/da/d4c/d9a/cb9 0 2024-03-29T08:07:20.682 INFO:tasks.workunit.client.0.smithi037.stdout:4/413: mkdir d3/d4/d15/d89 0 2024-03-29T08:07:20.683 INFO:tasks.workunit.client.1.smithi148.stdout:0/549: rename db/dc/daa/d47/d4c/d5a/d8d to db/dc/daa/d47/daf 0 2024-03-29T08:07:20.685 INFO:tasks.workunit.client.1.smithi148.stdout:6/564: fsync d6/d9/d1e/d27/d48/d9f/faa 0 2024-03-29T08:07:20.685 INFO:tasks.workunit.client.1.smithi148.stdout:2/576: stat d7/da/f9d 0 2024-03-29T08:07:20.686 INFO:tasks.workunit.client.0.smithi037.stdout:0/294: mknod d9/da/dd/d59/c5d 0 2024-03-29T08:07:20.688 INFO:tasks.workunit.client.1.smithi148.stdout:8/570: mknod d17/d74/d86/cbb 0 2024-03-29T08:07:20.689 INFO:tasks.workunit.client.0.smithi037.stdout:3/396: chown fb 1162 1 2024-03-29T08:07:20.691 INFO:tasks.workunit.client.1.smithi148.stdout:9/553: fdatasync db/d22/d26/d54/d4d/d35/f5e 0 2024-03-29T08:07:20.692 INFO:tasks.workunit.client.1.smithi148.stdout:4/558: dwrite df/d2c/d51/d77/d26/d9b/fa8 [0,4194304] 0 2024-03-29T08:07:20.694 INFO:tasks.workunit.client.0.smithi037.stdout:9/587: link d1/d4/cc9 d1/d4/d19/d1e/d4a/d6f/da2/d31/daa/dba/dbb/ccc 0 2024-03-29T08:07:20.695 INFO:tasks.workunit.client.0.smithi037.stdout:6/369: dwrite d0/d5/f14 [4194304,4194304] 0 2024-03-29T08:07:20.696 INFO:tasks.workunit.client.0.smithi037.stdout:0/295: write d9/da/dd/d59/f39 [186659,76778] 0 2024-03-29T08:07:20.700 INFO:tasks.workunit.client.1.smithi148.stdout:6/565: rmdir d6/d9/d1e/d27/d48/d73 39 2024-03-29T08:07:20.702 INFO:tasks.workunit.client.1.smithi148.stdout:0/550: mkdir db/de/df/d14/db0 0 2024-03-29T08:07:20.705 INFO:tasks.workunit.client.0.smithi037.stdout:2/284: dread d0/d23/d44/f45 [0,4194304] 0 2024-03-29T08:07:20.705 INFO:tasks.workunit.client.1.smithi148.stdout:1/544: dwrite d3/f93 [0,4194304] 0 2024-03-29T08:07:20.707 INFO:tasks.workunit.client.1.smithi148.stdout:2/577: write d7/da/d33/d3c/d54/d70/f85 [167291,7968] 0 2024-03-29T08:07:20.710 INFO:tasks.workunit.client.0.smithi037.stdout:3/397: dread dd/d46/f37 [0,4194304] 0 2024-03-29T08:07:20.712 INFO:tasks.workunit.client.0.smithi037.stdout:9/588: chown d1/d1f/c81 0 1 2024-03-29T08:07:20.713 INFO:tasks.workunit.client.1.smithi148.stdout:2/578: chown d7/da/d33/d45/d4d/d2e 293125 1 2024-03-29T08:07:20.715 INFO:tasks.workunit.client.1.smithi148.stdout:8/571: symlink d17/d74/d39/d6f/d64/lbc 0 2024-03-29T08:07:20.716 INFO:tasks.workunit.client.1.smithi148.stdout:2/579: fdatasync d7/d24/f36 0 2024-03-29T08:07:20.717 INFO:tasks.workunit.client.0.smithi037.stdout:6/370: mknod d0/d19/d42/d60/c7c 0 2024-03-29T08:07:20.723 INFO:tasks.workunit.client.0.smithi037.stdout:7/403: dread f9 [4194304,4194304] 0 2024-03-29T08:07:20.724 INFO:tasks.workunit.client.1.smithi148.stdout:8/572: mkdir d17/d18/dbd 0 2024-03-29T08:07:20.724 INFO:tasks.workunit.client.0.smithi037.stdout:9/589: creat d1/d4/d19/d1e/d4a/d6f/fcd x:0 0 0 2024-03-29T08:07:20.725 INFO:tasks.workunit.client.1.smithi148.stdout:7/514: dwrite f4 [0,4194304] 0 2024-03-29T08:07:20.727 INFO:tasks.workunit.client.0.smithi037.stdout:2/285: rename d0/d20/d2e/l4a to d0/d20/d22/l64 0 2024-03-29T08:07:20.728 INFO:tasks.workunit.client.1.smithi148.stdout:5/561: dread d0/d46/d9e/fa6 [0,4194304] 0 2024-03-29T08:07:20.731 INFO:tasks.workunit.client.0.smithi037.stdout:7/404: creat dd/d1d/d34/d54/f87 x:0 0 0 2024-03-29T08:07:20.732 INFO:tasks.workunit.client.0.smithi037.stdout:2/286: fsync d0/f1b 0 2024-03-29T08:07:20.733 INFO:tasks.workunit.client.0.smithi037.stdout:2/287: stat d0/d53 0 2024-03-29T08:07:20.736 INFO:tasks.workunit.client.0.smithi037.stdout:9/590: truncate d1/d4/d19/d1e/d4a/d6f/da2/f7a 344136 0 2024-03-29T08:07:20.737 INFO:tasks.workunit.client.1.smithi148.stdout:2/580: creat d7/da/d52/fba x:0 0 0 2024-03-29T08:07:20.741 INFO:tasks.workunit.client.1.smithi148.stdout:0/551: mkdir db/dc/daa/d47/d4c/d5a/d7a/db1 0 2024-03-29T08:07:20.742 INFO:tasks.workunit.client.1.smithi148.stdout:1/545: creat d3/de/d25/d35/d42/fa5 x:0 0 0 2024-03-29T08:07:20.743 INFO:tasks.workunit.client.0.smithi037.stdout:9/591: fsync d1/d4/d19/d1e/d4a/d6f/da2/d23/f72 0 2024-03-29T08:07:20.746 INFO:tasks.workunit.client.0.smithi037.stdout:4/414: write d3/f3a [505166,121710] 0 2024-03-29T08:07:20.747 INFO:tasks.workunit.client.0.smithi037.stdout:2/288: creat d0/d23/f65 x:0 0 0 2024-03-29T08:07:20.748 INFO:tasks.workunit.client.1.smithi148.stdout:1/546: dread - d3/de/f86 zero size 2024-03-29T08:07:20.749 INFO:tasks.workunit.client.0.smithi037.stdout:0/296: write d9/da/d1b/f29 [347372,54149] 0 2024-03-29T08:07:20.750 INFO:tasks.workunit.client.1.smithi148.stdout:4/559: link df/l23 df/d2c/d51/d77/d26/d34/lb6 0 2024-03-29T08:07:20.750 INFO:tasks.workunit.client.0.smithi037.stdout:9/592: creat d1/d4/d19/d69/fce x:0 0 0 2024-03-29T08:07:20.750 INFO:tasks.workunit.client.0.smithi037.stdout:4/415: write d3/d4/f3e [745884,42039] 0 2024-03-29T08:07:20.756 INFO:tasks.workunit.client.0.smithi037.stdout:0/297: creat d9/da/dd/d13/f5e x:0 0 0 2024-03-29T08:07:20.757 INFO:tasks.workunit.client.1.smithi148.stdout:1/547: readlink d3/de/d1d/la4 0 2024-03-29T08:07:20.758 INFO:tasks.workunit.client.0.smithi037.stdout:0/298: fdatasync d9/da/f11 0 2024-03-29T08:07:20.758 INFO:tasks.workunit.client.1.smithi148.stdout:9/554: dwrite db/d22/d3f/f42 [0,4194304] 0 2024-03-29T08:07:20.758 INFO:tasks.workunit.client.1.smithi148.stdout:2/581: unlink d7/da/d33/d45/d4d/d2e/d37/f6d 0 2024-03-29T08:07:20.759 INFO:tasks.workunit.client.0.smithi037.stdout:2/289: mknod d0/d20/d2e/d5f/c66 0 2024-03-29T08:07:20.760 INFO:tasks.workunit.client.0.smithi037.stdout:9/593: mknod d1/d1f/ccf 0 2024-03-29T08:07:20.761 INFO:tasks.workunit.client.0.smithi037.stdout:0/299: fdatasync d9/da/d1b/d22/d2c/f42 0 2024-03-29T08:07:20.764 INFO:tasks.workunit.client.0.smithi037.stdout:0/300: chown d9/da/dd/c3b 46872735 1 2024-03-29T08:07:20.767 INFO:tasks.workunit.client.1.smithi148.stdout:0/552: write db/de/df/d11/d2d/d3e/f45 [2492300,11030] 0 2024-03-29T08:07:20.769 INFO:tasks.workunit.client.0.smithi037.stdout:9/594: chown d1/d4/d19/d1e/d4a/d6f/da2/d31/l90 34797139 1 2024-03-29T08:07:20.769 INFO:tasks.workunit.client.0.smithi037.stdout:0/301: fsync d9/da/dd/d13/f5c 0 2024-03-29T08:07:20.773 INFO:tasks.workunit.client.0.smithi037.stdout:0/302: chown d9/c3a 3 1 2024-03-29T08:07:20.773 INFO:tasks.workunit.client.0.smithi037.stdout:4/416: link d3/d4/d15/d1d/l3c d3/d4/d15/d1d/db/d45/d1e/d59/d7d/l8a 0 2024-03-29T08:07:20.777 INFO:tasks.workunit.client.1.smithi148.stdout:3/583: dread de/d24/d33/d39/d46/d30/d37/f6f [0,4194304] 0 2024-03-29T08:07:20.777 INFO:tasks.workunit.client.1.smithi148.stdout:9/555: dread db/d22/f89 [0,4194304] 0 2024-03-29T08:07:20.780 INFO:tasks.workunit.client.1.smithi148.stdout:1/548: stat d3/d18/d4d/d9e/f59 0 2024-03-29T08:07:20.782 INFO:tasks.workunit.client.1.smithi148.stdout:6/566: dwrite d6/d9/d1e/d27/f9b [0,4194304] 0 2024-03-29T08:07:20.783 INFO:tasks.workunit.client.1.smithi148.stdout:2/582: rmdir d7/da/d33/d3c/d31 39 2024-03-29T08:07:20.783 INFO:tasks.workunit.client.0.smithi037.stdout:7/405: dread dd/d1d/d34/f40 [0,4194304] 0 2024-03-29T08:07:20.785 INFO:tasks.workunit.client.0.smithi037.stdout:2/290: dread d0/f18 [0,4194304] 0 2024-03-29T08:07:20.787 INFO:tasks.workunit.client.1.smithi148.stdout:7/515: dwrite d8/d29/d44/d50/f90 [0,4194304] 0 2024-03-29T08:07:20.788 INFO:tasks.workunit.client.1.smithi148.stdout:9/556: readlink db/d10/l23 0 2024-03-29T08:07:20.789 INFO:tasks.workunit.client.0.smithi037.stdout:4/417: mknod d3/d4/d15/d89/c8b 0 2024-03-29T08:07:20.791 INFO:tasks.workunit.client.0.smithi037.stdout:0/303: mkdir d9/da/dd/d20/d55/d5f 0 2024-03-29T08:07:20.792 INFO:tasks.workunit.client.1.smithi148.stdout:8/573: dwrite d17/d74/d86/f8d [0,4194304] 0 2024-03-29T08:07:20.792 INFO:tasks.workunit.client.1.smithi148.stdout:2/583: chown d7/da/f3f 165 1 2024-03-29T08:07:20.794 INFO:tasks.workunit.client.1.smithi148.stdout:7/516: dread - d8/d14/d45/d4d/f9d zero size 2024-03-29T08:07:20.798 INFO:tasks.workunit.client.0.smithi037.stdout:0/304: stat d9/da/dd/d59/f39 0 2024-03-29T08:07:20.799 INFO:tasks.workunit.client.0.smithi037.stdout:7/406: mknod dd/d1d/d49/d81/d84/c88 0 2024-03-29T08:07:20.800 INFO:tasks.workunit.client.1.smithi148.stdout:3/584: rmdir de/d24/d33/d39/d7f 39 2024-03-29T08:07:20.800 INFO:tasks.workunit.client.1.smithi148.stdout:8/574: write d17/d18/fa9 [801917,72180] 0 2024-03-29T08:07:20.800 INFO:tasks.workunit.client.1.smithi148.stdout:2/584: chown d7/d24/d4b/l4f 57722615 1 2024-03-29T08:07:20.800 INFO:tasks.workunit.client.0.smithi037.stdout:0/305: dread - d9/da/dd/d13/f5c zero size 2024-03-29T08:07:20.802 INFO:tasks.workunit.client.1.smithi148.stdout:4/560: getdents df/d2c/d51/d77/d26/d34/d8a/d98 0 2024-03-29T08:07:20.803 INFO:tasks.workunit.client.0.smithi037.stdout:9/595: dread d1/d4/d19/d1e/d4a/d6f/da2/d23/f37 [4194304,4194304] 0 2024-03-29T08:07:20.806 INFO:tasks.workunit.client.0.smithi037.stdout:4/418: symlink d3/d4/d15/d1d/db/d45/d1e/d59/d60/l8c 0 2024-03-29T08:07:20.806 INFO:tasks.workunit.client.1.smithi148.stdout:7/517: read - d8/d38/d3c/f96 zero size 2024-03-29T08:07:20.806 INFO:tasks.workunit.client.1.smithi148.stdout:6/567: symlink d6/d9/d1e/d27/d48/lb5 0 2024-03-29T08:07:20.808 INFO:tasks.workunit.client.1.smithi148.stdout:9/557: mkdir db/d22/d26/d54/d4d/dbb 0 2024-03-29T08:07:20.810 INFO:tasks.workunit.client.1.smithi148.stdout:2/585: dread - d7/d24/f36 zero size 2024-03-29T08:07:20.810 INFO:tasks.workunit.client.1.smithi148.stdout:4/561: chown fa 0 1 2024-03-29T08:07:20.811 INFO:tasks.workunit.client.0.smithi037.stdout:7/407: read dd/d21/d65/f6f [3638436,67163] 0 2024-03-29T08:07:20.813 INFO:tasks.workunit.client.0.smithi037.stdout:9/596: stat d1/d4/d19/d1e/d4a/d6f/d96/fa4 0 2024-03-29T08:07:20.816 INFO:tasks.workunit.client.1.smithi148.stdout:0/553: dread db/de/df/d11/d2d/d3e/f45 [0,4194304] 0 2024-03-29T08:07:20.818 INFO:tasks.workunit.client.0.smithi037.stdout:7/408: chown dd/le 26410 1 2024-03-29T08:07:20.818 INFO:tasks.workunit.client.1.smithi148.stdout:3/585: creat de/d24/d33/d39/da3/dbc/fbe x:0 0 0 2024-03-29T08:07:20.819 INFO:tasks.workunit.client.1.smithi148.stdout:5/562: dread d0/db/d1e/f31 [0,4194304] 0 2024-03-29T08:07:20.820 INFO:tasks.workunit.client.0.smithi037.stdout:4/419: truncate d3/d4/d15/f29 2243352 0 2024-03-29T08:07:20.820 INFO:tasks.workunit.client.0.smithi037.stdout:2/291: dwrite d0/d20/d22/d4e/f4f [0,4194304] 0 2024-03-29T08:07:20.823 INFO:tasks.workunit.client.0.smithi037.stdout:7/409: dread - dd/d21/d35/d60/f6e zero size 2024-03-29T08:07:20.823 INFO:tasks.workunit.client.0.smithi037.stdout:4/420: chown d3/d4/d27/l28 2903746 1 2024-03-29T08:07:20.827 INFO:tasks.workunit.client.0.smithi037.stdout:4/421: fdatasync d3/d57/d6a/f77 0 2024-03-29T08:07:20.828 INFO:tasks.workunit.client.1.smithi148.stdout:4/562: write df/d2c/d51/d77/d26/d34/d8a/f8c [1274757,4057] 0 2024-03-29T08:07:20.828 INFO:tasks.workunit.client.0.smithi037.stdout:4/422: stat f0 0 2024-03-29T08:07:20.830 INFO:tasks.workunit.client.1.smithi148.stdout:1/549: dwrite d3/d18/d4d/f63 [0,4194304] 0 2024-03-29T08:07:20.830 INFO:tasks.workunit.client.0.smithi037.stdout:2/292: creat d0/d23/d44/f67 x:0 0 0 2024-03-29T08:07:20.830 INFO:tasks.workunit.client.1.smithi148.stdout:8/575: rename d17/d74/d39 to d17/d49/d9a/dbe 0 2024-03-29T08:07:20.832 INFO:tasks.workunit.client.0.smithi037.stdout:7/410: mkdir dd/d21/d89 0 2024-03-29T08:07:20.833 INFO:tasks.workunit.client.1.smithi148.stdout:3/586: readlink de/d24/d33/d39/d46/d30/l47 0 2024-03-29T08:07:20.835 INFO:tasks.workunit.client.0.smithi037.stdout:2/293: write d0/d20/d22/f50 [511096,113143] 0 2024-03-29T08:07:20.843 INFO:tasks.workunit.client.0.smithi037.stdout:9/597: write d1/d3/d44/f57 [140184,56593] 0 2024-03-29T08:07:20.846 INFO:tasks.workunit.client.0.smithi037.stdout:7/411: creat dd/d1d/d49/d81/d84/f8a x:0 0 0 2024-03-29T08:07:20.847 INFO:tasks.workunit.client.1.smithi148.stdout:7/518: rename d8/c57 to d8/d38/d3c/da9/d37/cb6 0 2024-03-29T08:07:20.849 INFO:tasks.workunit.client.0.smithi037.stdout:2/294: mkdir d0/d20/d2e/d38/d68 0 2024-03-29T08:07:20.850 INFO:tasks.workunit.client.1.smithi148.stdout:4/563: rmdir df/d2c/d51/d77 39 2024-03-29T08:07:20.851 INFO:tasks.workunit.client.0.smithi037.stdout:9/598: symlink d1/d3/d44/ld0 0 2024-03-29T08:07:20.853 INFO:tasks.workunit.client.0.smithi037.stdout:4/423: link d3/d4/d15/d1d/db/d45/d1e/d5f/f62 d3/d54/d88/f8d 0 2024-03-29T08:07:20.854 INFO:tasks.workunit.client.0.smithi037.stdout:7/412: write dd/f7d [541241,61122] 0 2024-03-29T08:07:20.854 INFO:tasks.workunit.client.0.smithi037.stdout:2/295: truncate d0/f1b 2186506 0 2024-03-29T08:07:20.857 INFO:tasks.workunit.client.1.smithi148.stdout:1/550: creat d3/de/d21/d5e/fa6 x:0 0 0 2024-03-29T08:07:20.859 INFO:tasks.workunit.client.0.smithi037.stdout:0/306: dwrite d9/da/dd/d13/f21 [0,4194304] 0 2024-03-29T08:07:20.861 INFO:tasks.workunit.client.1.smithi148.stdout:3/587: mkdir de/d24/d33/d39/d46/d30/d37/d62/d63/dbf 0 2024-03-29T08:07:20.862 INFO:tasks.workunit.client.0.smithi037.stdout:9/599: read d1/f74 [885522,17475] 0 2024-03-29T08:07:20.863 INFO:tasks.workunit.client.0.smithi037.stdout:4/424: creat d3/d57/f8e x:0 0 0 2024-03-29T08:07:20.863 INFO:tasks.workunit.client.1.smithi148.stdout:6/568: dread d6/f70 [0,4194304] 0 2024-03-29T08:07:20.871 INFO:tasks.workunit.client.0.smithi037.stdout:4/425: write d3/d4/d15/d1d/f71 [110030,45727] 0 2024-03-29T08:07:20.874 INFO:tasks.workunit.client.1.smithi148.stdout:9/558: dread db/d20/f1b [0,4194304] 0 2024-03-29T08:07:20.874 INFO:tasks.workunit.client.1.smithi148.stdout:0/554: dwrite db/de/df/d2b/f53 [0,4194304] 0 2024-03-29T08:07:20.874 INFO:tasks.workunit.client.0.smithi037.stdout:0/307: symlink d9/da/d1b/d22/l60 0 2024-03-29T08:07:20.875 INFO:tasks.workunit.client.0.smithi037.stdout:9/600: chown d1/d4/d19/d1e/d4a/d6f/da2/f12 43147 1 2024-03-29T08:07:20.878 INFO:tasks.workunit.client.1.smithi148.stdout:2/586: dwrite d7/da/d33/d45/d4d/f47 [0,4194304] 0 2024-03-29T08:07:20.880 INFO:tasks.workunit.client.0.smithi037.stdout:7/413: dread dd/d1d/d34/f59 [0,4194304] 0 2024-03-29T08:07:20.883 INFO:tasks.workunit.client.1.smithi148.stdout:4/564: readlink df/d2c/d51/d77/d26/d34/d3e/lb4 0 2024-03-29T08:07:20.884 INFO:tasks.workunit.client.1.smithi148.stdout:0/555: readlink db/de/df/d14/l1d 0 2024-03-29T08:07:20.885 INFO:tasks.workunit.client.0.smithi037.stdout:4/426: mkdir d3/d4/d15/d1d/db/d3d/d8f 0 2024-03-29T08:07:20.886 INFO:tasks.workunit.client.0.smithi037.stdout:2/296: dread d0/f40 [0,4194304] 0 2024-03-29T08:07:20.886 INFO:tasks.workunit.client.1.smithi148.stdout:5/563: dwrite d0/db/d1e/f4a [0,4194304] 0 2024-03-29T08:07:20.889 INFO:tasks.workunit.client.1.smithi148.stdout:1/551: creat d3/d18/d71/fa7 x:0 0 0 2024-03-29T08:07:20.894 INFO:tasks.workunit.client.1.smithi148.stdout:4/565: rmdir df/d2c/d51/d77/d17 39 2024-03-29T08:07:20.894 INFO:tasks.workunit.client.0.smithi037.stdout:7/414: dread dd/d21/d35/f6b [0,4194304] 0 2024-03-29T08:07:20.896 INFO:tasks.workunit.client.0.smithi037.stdout:7/415: dread - dd/d68/f78 zero size 2024-03-29T08:07:20.898 INFO:tasks.workunit.client.1.smithi148.stdout:8/576: dwrite d17/d49/d9a/dbe/d6f/fb9 [0,4194304] 0 2024-03-29T08:07:20.898 INFO:tasks.workunit.client.0.smithi037.stdout:7/416: write dd/f3e [1193611,11099] 0 2024-03-29T08:07:20.899 INFO:tasks.workunit.client.1.smithi148.stdout:9/559: creat db/d22/d3f/fbc x:0 0 0 2024-03-29T08:07:20.901 INFO:tasks.workunit.client.0.smithi037.stdout:2/297: rename d0/f2 to d0/d20/d2e/f69 0 2024-03-29T08:07:20.903 INFO:tasks.workunit.client.0.smithi037.stdout:2/298: write d0/d20/d22/f50 [184786,47381] 0 2024-03-29T08:07:20.905 INFO:tasks.workunit.client.1.smithi148.stdout:4/566: readlink df/d2c/l93 0 2024-03-29T08:07:20.906 INFO:tasks.workunit.client.1.smithi148.stdout:9/560: dread - db/d22/d26/d54/d4d/d35/d4b/d5a/d74/fba zero size 2024-03-29T08:07:20.907 INFO:tasks.workunit.client.1.smithi148.stdout:7/519: rmdir d8/d38/d6a/db3 0 2024-03-29T08:07:20.908 INFO:tasks.workunit.client.1.smithi148.stdout:1/552: rmdir d3/de/d1d/d47/d6d/d72 39 2024-03-29T08:07:20.910 INFO:tasks.workunit.client.0.smithi037.stdout:0/308: dread d9/da/d1b/d22/d2c/f2d [0,4194304] 0 2024-03-29T08:07:20.912 INFO:tasks.workunit.client.0.smithi037.stdout:0/309: readlink d9/da/l4a 0 2024-03-29T08:07:20.912 INFO:tasks.workunit.client.1.smithi148.stdout:3/588: rename de/d24/d33/d39/d46/d30/d37/fb5 to de/d24/d33/d39/d46/d30/d37/d79/fc0 0 2024-03-29T08:07:20.913 INFO:tasks.workunit.client.0.smithi037.stdout:0/310: fsync d9/da/f3c 0 2024-03-29T08:07:20.915 INFO:tasks.workunit.client.1.smithi148.stdout:9/561: write db/d10/d2d/f7b [668679,98848] 0 2024-03-29T08:07:20.916 INFO:tasks.workunit.client.1.smithi148.stdout:8/577: rename d17/d18/d26/d4f to d17/d18/d26/d4f/d94/dbf 22 2024-03-29T08:07:20.919 INFO:tasks.workunit.client.0.smithi037.stdout:2/299: symlink d0/d23/l6a 0 2024-03-29T08:07:20.924 INFO:tasks.workunit.client.1.smithi148.stdout:6/569: getdents d6/d9/d1e/d36/d4d/d9d 0 2024-03-29T08:07:20.925 INFO:tasks.workunit.client.0.smithi037.stdout:2/300: chown d0/d20/d22/c61 5336 1 2024-03-29T08:07:20.926 INFO:tasks.workunit.client.1.smithi148.stdout:7/520: mkdir d8/d38/d3c/da9/db7 0 2024-03-29T08:07:20.928 INFO:tasks.workunit.client.1.smithi148.stdout:5/564: dwrite d0/db/d1e/d36/d41/f58 [0,4194304] 0 2024-03-29T08:07:20.930 INFO:tasks.workunit.client.0.smithi037.stdout:2/301: write d0/d23/d59/d5e/f60 [267807,71956] 0 2024-03-29T08:07:20.936 INFO:tasks.workunit.client.0.smithi037.stdout:2/302: creat d0/d20/d2e/d38/f6b x:0 0 0 2024-03-29T08:07:20.938 INFO:tasks.workunit.client.0.smithi037.stdout:9/601: dread d1/d4/d19/d1e/d25/f34 [0,4194304] 0 2024-03-29T08:07:20.940 INFO:tasks.workunit.client.0.smithi037.stdout:4/427: dread d3/d4/d15/d1d/db/f1a [0,4194304] 0 2024-03-29T08:07:20.941 INFO:tasks.workunit.client.0.smithi037.stdout:7/417: dwrite f9 [4194304,4194304] 0 2024-03-29T08:07:20.941 INFO:tasks.workunit.client.0.smithi037.stdout:9/602: chown d1/d4/d19/d1e/d4a/d6f/da2/d13/l5b 286870553 1 2024-03-29T08:07:20.944 INFO:tasks.workunit.client.0.smithi037.stdout:7/418: chown dd/d21/d35/d60/f7f 43413 1 2024-03-29T08:07:20.947 INFO:tasks.workunit.client.0.smithi037.stdout:9/603: read d1/d1f/f5f [3419946,58970] 0 2024-03-29T08:07:20.950 INFO:tasks.workunit.client.0.smithi037.stdout:7/419: chown dd/l22 78126 1 2024-03-29T08:07:20.952 INFO:tasks.workunit.client.0.smithi037.stdout:7/420: readlink dd/l22 0 2024-03-29T08:07:20.953 INFO:tasks.workunit.client.0.smithi037.stdout:9/604: read d1/d4/d19/d1e/d4a/d6f/da2/f3d [2543967,55041] 0 2024-03-29T08:07:20.957 INFO:tasks.workunit.client.0.smithi037.stdout:2/303: dwrite d0/d20/f3a [0,4194304] 0 2024-03-29T08:07:20.957 INFO:tasks.workunit.client.0.smithi037.stdout:7/421: write dd/d1d/d49/f5a [1432077,105863] 0 2024-03-29T08:07:20.960 INFO:tasks.workunit.client.0.smithi037.stdout:9/605: fsync d1/d4/d19/d1e/d4a/d6f/da2/f12 0 2024-03-29T08:07:20.960 INFO:tasks.workunit.client.0.smithi037.stdout:4/428: truncate d3/d4/d15/d1d/f64 1041459 0 2024-03-29T08:07:20.962 INFO:tasks.workunit.client.0.smithi037.stdout:4/429: fdatasync d3/f21 0 2024-03-29T08:07:20.963 INFO:tasks.workunit.client.0.smithi037.stdout:2/304: rmdir d0/d20/d22 39 2024-03-29T08:07:20.968 INFO:tasks.workunit.client.0.smithi037.stdout:9/606: symlink d1/d4/d19/d69/ld1 0 2024-03-29T08:07:20.972 INFO:tasks.workunit.client.0.smithi037.stdout:4/430: creat d3/d4/d15/d1d/db/d3d/f90 x:0 0 0 2024-03-29T08:07:20.973 INFO:tasks.workunit.client.0.smithi037.stdout:7/422: truncate dd/d21/d35/f50 770327 0 2024-03-29T08:07:20.975 INFO:tasks.workunit.client.0.smithi037.stdout:2/305: mknod d0/d20/d22/d4e/c6c 0 2024-03-29T08:07:20.975 INFO:tasks.workunit.client.0.smithi037.stdout:7/423: readlink dd/d1d/d49/l7b 0 2024-03-29T08:07:20.977 INFO:tasks.workunit.client.0.smithi037.stdout:9/607: write d1/d4/d19/fb9 [2603941,58159] 0 2024-03-29T08:07:20.979 INFO:tasks.workunit.client.0.smithi037.stdout:2/306: symlink d0/d20/d22/l6d 0 2024-03-29T08:07:20.981 INFO:tasks.workunit.client.0.smithi037.stdout:9/608: write d1/d1f/d3a/f78 [1776925,119767] 0 2024-03-29T08:07:20.983 INFO:tasks.workunit.client.0.smithi037.stdout:9/609: stat d1/d1f/d3a 0 2024-03-29T08:07:20.985 INFO:tasks.workunit.client.0.smithi037.stdout:2/307: rename d0/d23/l6a to d0/l6e 0 2024-03-29T08:07:20.986 INFO:tasks.workunit.client.0.smithi037.stdout:9/610: dread - d1/d4/f7f zero size 2024-03-29T08:07:20.987 INFO:tasks.workunit.client.0.smithi037.stdout:4/431: dread d3/d4/d15/d1d/db/f12 [0,4194304] 0 2024-03-29T08:07:20.990 INFO:tasks.workunit.client.0.smithi037.stdout:9/611: mknod d1/d1f/d3a/cd2 0 2024-03-29T08:07:20.991 INFO:tasks.workunit.client.0.smithi037.stdout:4/432: symlink d3/d4/d27/l91 0 2024-03-29T08:07:20.992 INFO:tasks.workunit.client.0.smithi037.stdout:4/433: stat d3/d4/c48 0 2024-03-29T08:07:20.998 INFO:tasks.workunit.client.0.smithi037.stdout:9/612: dread d1/d4/d19/d1e/d4a/d6f/da2/d23/f65 [0,4194304] 0 2024-03-29T08:07:21.003 INFO:tasks.workunit.client.0.smithi037.stdout:9/613: mkdir d1/d4/d19/d1e/d4a/d6f/da2/d13/d7c/dd3 0 2024-03-29T08:07:21.004 INFO:tasks.workunit.client.1.smithi148.stdout:7/521: stat d8/d38/d3c/da5/caa 0 2024-03-29T08:07:21.005 INFO:tasks.workunit.client.1.smithi148.stdout:5/565: stat d0/db/f34 0 2024-03-29T08:07:21.006 INFO:tasks.workunit.client.0.smithi037.stdout:9/614: readlink d1/d3/l61 0 2024-03-29T08:07:21.008 INFO:tasks.workunit.client.0.smithi037.stdout:9/615: rename d1/d4/d19 to d1/d4/d19/d69/dd4 22 2024-03-29T08:07:21.012 INFO:tasks.workunit.client.0.smithi037.stdout:2/308: write d0/d20/d22/f39 [651924,109164] 0 2024-03-29T08:07:21.012 INFO:tasks.workunit.client.0.smithi037.stdout:4/434: write d3/f44 [825894,70361] 0 2024-03-29T08:07:21.013 INFO:tasks.workunit.client.0.smithi037.stdout:2/309: dread - d0/d23/f5c zero size 2024-03-29T08:07:21.015 INFO:tasks.workunit.client.0.smithi037.stdout:9/616: creat d1/d4/d19/d1e/d4a/d6f/da2/d31/d87/fd5 x:0 0 0 2024-03-29T08:07:21.016 INFO:tasks.workunit.client.0.smithi037.stdout:2/310: chown d0/d20/d22/f48 0 1 2024-03-29T08:07:21.018 INFO:tasks.workunit.client.1.smithi148.stdout:9/562: dread db/d10/d2d/d4e/d5f/f65 [0,4194304] 0 2024-03-29T08:07:21.019 INFO:tasks.workunit.client.0.smithi037.stdout:4/435: chown d3/d4/d15/d1d/db/d45/c20 13938542 1 2024-03-29T08:07:21.022 INFO:tasks.workunit.client.0.smithi037.stdout:9/617: symlink d1/d4/d19/d1e/d71/d89/d92/ld6 0 2024-03-29T08:07:21.022 INFO:tasks.workunit.client.1.smithi148.stdout:2/587: dwrite d7/da/d33/d3c/d31/faa [0,4194304] 0 2024-03-29T08:07:21.026 INFO:tasks.workunit.client.0.smithi037.stdout:2/311: rename d0/d23/d59 to d0/d20/d2e/d5f/d6f 0 2024-03-29T08:07:21.033 INFO:tasks.workunit.client.0.smithi037.stdout:9/618: creat d1/d4/d19/d1e/d71/d89/dbe/fd7 x:0 0 0 2024-03-29T08:07:21.035 INFO:tasks.workunit.client.1.smithi148.stdout:4/567: dwrite df/d2c/d51/d77/d26/f84 [0,4194304] 0 2024-03-29T08:07:21.038 INFO:tasks.workunit.client.1.smithi148.stdout:5/566: dread - d0/db/d1e/d35/d42/d67/d76/d81/d98/fb9 zero size 2024-03-29T08:07:21.038 INFO:tasks.workunit.client.1.smithi148.stdout:0/556: dwrite db/de/df/d11/d2d/d3e/f45 [0,4194304] 0 2024-03-29T08:07:21.038 INFO:tasks.workunit.client.0.smithi037.stdout:2/312: getdents d0/d20/d22/d4e 0 2024-03-29T08:07:21.040 INFO:tasks.workunit.client.0.smithi037.stdout:9/619: unlink d1/f4c 0 2024-03-29T08:07:21.041 INFO:tasks.workunit.client.1.smithi148.stdout:9/563: unlink db/d10/d9d/fb6 0 2024-03-29T08:07:21.043 INFO:tasks.workunit.client.1.smithi148.stdout:3/589: dwrite de/d24/d33/f96 [0,4194304] 0 2024-03-29T08:07:21.043 INFO:tasks.workunit.client.0.smithi037.stdout:2/313: mkdir d0/d23/d70 0 2024-03-29T08:07:21.045 INFO:tasks.workunit.client.1.smithi148.stdout:8/578: dwrite fe [0,4194304] 0 2024-03-29T08:07:21.049 INFO:tasks.workunit.client.1.smithi148.stdout:2/588: chown d7/d24/d4b/f87 1653949 1 2024-03-29T08:07:21.050 INFO:tasks.workunit.client.1.smithi148.stdout:9/564: truncate db/d10/faf 348396 0 2024-03-29T08:07:21.052 INFO:tasks.workunit.client.1.smithi148.stdout:9/565: fdatasync db/f59 0 2024-03-29T08:07:21.053 INFO:tasks.workunit.client.0.smithi037.stdout:2/314: fsync d0/d20/d22/f48 0 2024-03-29T08:07:21.054 INFO:tasks.workunit.client.1.smithi148.stdout:6/570: getdents d6/d9/d1e/d27/d48/d73/d7e 0 2024-03-29T08:07:21.057 INFO:tasks.workunit.client.0.smithi037.stdout:2/315: stat d0/d23/d44/d62 0 2024-03-29T08:07:21.059 INFO:tasks.workunit.client.0.smithi037.stdout:2/316: fsync d0/d20/d22/f47 0 2024-03-29T08:07:21.060 INFO:tasks.workunit.client.0.smithi037.stdout:7/424: dread dd/f3e [0,4194304] 0 2024-03-29T08:07:21.060 INFO:tasks.workunit.client.1.smithi148.stdout:8/579: mknod d17/d18/cc0 0 2024-03-29T08:07:21.065 INFO:tasks.workunit.client.1.smithi148.stdout:3/590: creat de/d24/d33/d39/da3/fc1 x:0 0 0 2024-03-29T08:07:21.067 INFO:tasks.workunit.client.1.smithi148.stdout:2/589: mkdir d7/da/d33/d3c/d31/dbb 0 2024-03-29T08:07:21.070 INFO:tasks.workunit.client.1.smithi148.stdout:7/522: dwrite d8/d29/d44/d49/f76 [0,4194304] 0 2024-03-29T08:07:21.070 INFO:tasks.workunit.client.1.smithi148.stdout:9/566: mkdir db/d10/d2d/d4e/d5f/d6e/d7a/dbd 0 2024-03-29T08:07:21.071 INFO:tasks.workunit.client.1.smithi148.stdout:8/580: symlink d17/d18/lc1 0 2024-03-29T08:07:21.073 INFO:tasks.workunit.client.1.smithi148.stdout:3/591: chown de/d24/f27 1102383 1 2024-03-29T08:07:21.077 INFO:tasks.workunit.client.1.smithi148.stdout:5/567: link d0/fb4 d0/db/d1e/d35/d42/d67/d76/fba 0 2024-03-29T08:07:21.077 INFO:tasks.workunit.client.0.smithi037.stdout:7/425: dwrite dd/d21/f75 [0,4194304] 0 2024-03-29T08:07:21.079 INFO:tasks.workunit.client.0.smithi037.stdout:9/620: write d1/d4/d19/d1e/d4a/d6f/da2/d23/f72 [733862,104267] 0 2024-03-29T08:07:21.079 INFO:tasks.workunit.client.1.smithi148.stdout:2/590: chown d7/da/d33/d45/d4d/d2e/ca0 222 1 2024-03-29T08:07:21.081 INFO:tasks.workunit.client.0.smithi037.stdout:1/470: sync 2024-03-29T08:07:21.082 INFO:tasks.workunit.client.0.smithi037.stdout:9/621: stat d1/d4/d19/d69/f9c 0 2024-03-29T08:07:21.082 INFO:tasks.workunit.client.0.smithi037.stdout:7/426: write dd/d1d/d34/f37 [3759132,36146] 0 2024-03-29T08:07:21.083 INFO:tasks.workunit.client.1.smithi148.stdout:9/567: creat db/d10/d6d/fbe x:0 0 0 2024-03-29T08:07:21.083 INFO:tasks.workunit.client.1.smithi148.stdout:7/523: mkdir d8/d29/db8 0 2024-03-29T08:07:21.083 INFO:tasks.workunit.client.1.smithi148.stdout:8/581: truncate d17/d18/f2a 2010331 0 2024-03-29T08:07:21.084 INFO:tasks.workunit.client.1.smithi148.stdout:3/592: creat de/d24/d33/d39/d46/d8d/fc2 x:0 0 0 2024-03-29T08:07:21.085 INFO:tasks.workunit.client.1.smithi148.stdout:5/568: chown d0/db/d7b/f3b 74 1 2024-03-29T08:07:21.085 INFO:tasks.workunit.client.1.smithi148.stdout:4/568: dwrite df/d2c/d51/d77/d26/f3b [0,4194304] 0 2024-03-29T08:07:21.086 INFO:tasks.workunit.client.1.smithi148.stdout:6/571: truncate d6/d9/d1e/f26 6968092 0 2024-03-29T08:07:21.086 INFO:tasks.workunit.client.0.smithi037.stdout:9/622: chown d1/d4/d19/d1e/d25/fc1 0 1 2024-03-29T08:07:21.089 INFO:tasks.workunit.client.0.smithi037.stdout:9/623: read - d1/d4/d19/d69/fce zero size 2024-03-29T08:07:21.089 INFO:tasks.workunit.client.1.smithi148.stdout:4/569: chown df/d71/f88 764150829 1 2024-03-29T08:07:21.090 INFO:tasks.workunit.client.1.smithi148.stdout:2/591: creat d7/da/d33/d45/d4d/d2e/fbc x:0 0 0 2024-03-29T08:07:21.090 INFO:tasks.workunit.client.0.smithi037.stdout:7/427: rmdir dd/d21 39 2024-03-29T08:07:21.090 INFO:tasks.workunit.client.0.smithi037.stdout:1/471: truncate d2/d6/dc/d18/f3e 4355320 0 2024-03-29T08:07:21.091 INFO:tasks.workunit.client.1.smithi148.stdout:3/593: dread - de/d24/d33/d39/da3/fc1 zero size 2024-03-29T08:07:21.092 INFO:tasks.workunit.client.1.smithi148.stdout:9/568: symlink db/d10/d2d/lbf 0 2024-03-29T08:07:21.096 INFO:tasks.workunit.client.0.smithi037.stdout:9/624: mkdir d1/d4/d19/d1e/d71/d89/da6/dd8 0 2024-03-29T08:07:21.096 INFO:tasks.workunit.client.1.smithi148.stdout:2/592: chown d7/d8c 6738 1 2024-03-29T08:07:21.097 INFO:tasks.workunit.client.1.smithi148.stdout:8/582: symlink d17/d1d/lc2 0 2024-03-29T08:07:21.097 INFO:tasks.workunit.client.1.smithi148.stdout:5/569: fsync d0/db/d7b/d16/f3d 0 2024-03-29T08:07:21.097 INFO:tasks.workunit.client.0.smithi037.stdout:7/428: rename dd/d1d/d49/l56 to dd/d1d/d34/d54/l8b 0 2024-03-29T08:07:21.099 INFO:tasks.workunit.client.1.smithi148.stdout:4/570: rmdir df/d2c/d51/d77/d26/d34/d43 39 2024-03-29T08:07:21.099 INFO:tasks.workunit.client.1.smithi148.stdout:8/583: chown d17/d49/fb3 4715 1 2024-03-29T08:07:21.101 INFO:tasks.workunit.client.0.smithi037.stdout:1/472: truncate d2/d6/d13/f28 1267951 0 2024-03-29T08:07:21.101 INFO:tasks.workunit.client.1.smithi148.stdout:2/593: write d7/da/d33/d45/d4d/d82/f9f [963290,58782] 0 2024-03-29T08:07:21.102 INFO:tasks.workunit.client.1.smithi148.stdout:3/594: creat de/d24/d33/d39/d46/d30/d37/fc3 x:0 0 0 2024-03-29T08:07:21.105 INFO:tasks.workunit.client.1.smithi148.stdout:9/569: mkdir db/d22/d26/d54/d4d/dbb/dc0 0 2024-03-29T08:07:21.105 INFO:tasks.workunit.client.1.smithi148.stdout:7/524: dread d8/d60/f70 [4194304,4194304] 0 2024-03-29T08:07:21.109 INFO:tasks.workunit.client.1.smithi148.stdout:8/584: mknod d17/d18/d26/d4f/d94/cc3 0 2024-03-29T08:07:21.112 INFO:tasks.workunit.client.0.smithi037.stdout:1/473: symlink d2/d6/la4 0 2024-03-29T08:07:21.113 INFO:tasks.workunit.client.1.smithi148.stdout:4/571: write df/d2c/d51/d77/d26/d9b/fa8 [945260,50098] 0 2024-03-29T08:07:21.113 INFO:tasks.workunit.client.1.smithi148.stdout:5/570: dread d0/db/d1e/d5b/d66/f74 [0,4194304] 0 2024-03-29T08:07:21.115 INFO:tasks.workunit.client.1.smithi148.stdout:2/594: truncate d7/da/fe 9066926 0 2024-03-29T08:07:21.116 INFO:tasks.workunit.client.1.smithi148.stdout:0/557: dwrite db/dc/daa/d47/d4c/f5f [0,4194304] 0 2024-03-29T08:07:21.116 INFO:tasks.workunit.client.1.smithi148.stdout:8/585: symlink d17/d1d/d7e/lc4 0 2024-03-29T08:07:21.119 INFO:tasks.workunit.client.1.smithi148.stdout:7/525: mknod d8/cb9 0 2024-03-29T08:07:21.123 INFO:tasks.workunit.client.1.smithi148.stdout:5/571: fdatasync d0/f12 0 2024-03-29T08:07:21.123 INFO:tasks.workunit.client.1.smithi148.stdout:4/572: mknod df/d2c/d80/daa/cb7 0 2024-03-29T08:07:21.124 INFO:tasks.workunit.client.1.smithi148.stdout:6/572: write d6/d9/d1e/d27/f6e [1725891,26800] 0 2024-03-29T08:07:21.124 INFO:tasks.workunit.client.1.smithi148.stdout:8/586: write d17/d49/d9a/db8/fba [910614,41977] 0 2024-03-29T08:07:21.125 INFO:tasks.workunit.client.0.smithi037.stdout:7/429: dwrite dd/f67 [0,4194304] 0 2024-03-29T08:07:21.128 INFO:tasks.workunit.client.1.smithi148.stdout:2/595: creat d7/da/d4c/d9a/fbd x:0 0 0 2024-03-29T08:07:21.130 INFO:tasks.workunit.client.1.smithi148.stdout:0/558: fdatasync db/de/df/d11/f16 0 2024-03-29T08:07:21.132 INFO:tasks.workunit.client.1.smithi148.stdout:9/570: rename db/d22/d26/d54/l2c to db/d22/d26/d54/d52/d9c/lc1 0 2024-03-29T08:07:21.132 INFO:tasks.workunit.client.1.smithi148.stdout:8/587: chown d17/d1d/d7e/lc4 1 1 2024-03-29T08:07:21.136 INFO:tasks.workunit.client.1.smithi148.stdout:5/572: fdatasync d0/db/d1e/d79/faa 0 2024-03-29T08:07:21.136 INFO:tasks.workunit.client.1.smithi148.stdout:0/559: chown db/dc/c87 847106332 1 2024-03-29T08:07:21.137 INFO:tasks.workunit.client.0.smithi037.stdout:9/625: write d1/d4/d19/d1e/d4a/d6f/da2/d31/f48 [49249,115940] 0 2024-03-29T08:07:21.138 INFO:tasks.workunit.client.1.smithi148.stdout:3/595: dwrite de/d24/d33/d39/d46/d30/fa2 [0,4194304] 0 2024-03-29T08:07:21.139 INFO:tasks.workunit.client.1.smithi148.stdout:8/588: write d17/d18/f67 [4752278,90534] 0 2024-03-29T08:07:21.140 INFO:tasks.workunit.client.1.smithi148.stdout:2/596: creat d7/d24/d4b/d66/d7d/fbe x:0 0 0 2024-03-29T08:07:21.141 INFO:tasks.workunit.client.0.smithi037.stdout:1/474: dwrite d2/d6/f36 [0,4194304] 0 2024-03-29T08:07:21.145 INFO:tasks.workunit.client.1.smithi148.stdout:7/526: write d8/d29/d44/d49/f62 [993941,515] 0 2024-03-29T08:07:21.146 INFO:tasks.workunit.client.1.smithi148.stdout:2/597: creat d7/fbf x:0 0 0 2024-03-29T08:07:21.150 INFO:tasks.workunit.client.1.smithi148.stdout:2/598: chown d7/da/d33/d45 3 1 2024-03-29T08:07:21.151 INFO:tasks.workunit.client.1.smithi148.stdout:8/589: write d17/d49/d9a/dbe/d6f/f45 [3363747,19528] 0 2024-03-29T08:07:21.154 INFO:tasks.workunit.client.1.smithi148.stdout:9/571: rename db/d10/d2d/d4e/d5f/c69 to db/d10/d2d/d4e/da6/cc2 0 2024-03-29T08:07:21.157 INFO:tasks.workunit.client.1.smithi148.stdout:2/599: write d7/f19 [372424,114146] 0 2024-03-29T08:07:21.157 INFO:tasks.workunit.client.1.smithi148.stdout:6/573: getdents d6/d9/d68 0 2024-03-29T08:07:21.160 INFO:tasks.workunit.client.1.smithi148.stdout:3/596: creat de/d24/d33/fc4 x:0 0 0 2024-03-29T08:07:21.164 INFO:tasks.workunit.client.0.smithi037.stdout:1/475: truncate d2/d4b/da1/f96 3372737 0 2024-03-29T08:07:21.165 INFO:tasks.workunit.client.1.smithi148.stdout:8/590: write d17/d4e/f71 [2311414,7854] 0 2024-03-29T08:07:21.173 INFO:tasks.workunit.client.0.smithi037.stdout:1/476: rmdir d2/d6/dc/d18 39 2024-03-29T08:07:21.173 INFO:tasks.workunit.client.0.smithi037.stdout:9/626: dwrite d1/d4/d19/d1e/d4a/d6f/da2/d23/f9a [0,4194304] 0 2024-03-29T08:07:21.174 INFO:tasks.workunit.client.1.smithi148.stdout:3/597: write de/d24/d33/f94 [58077,49641] 0 2024-03-29T08:07:21.176 INFO:tasks.workunit.client.1.smithi148.stdout:2/600: mknod d7/d24/d4b/d66/cc0 0 2024-03-29T08:07:21.176 INFO:tasks.workunit.client.1.smithi148.stdout:4/573: dwrite df/d2c/d51/d77/d26/d34/d8a/d98/fb0 [0,4194304] 0 2024-03-29T08:07:21.177 INFO:tasks.workunit.client.0.smithi037.stdout:1/477: creat d2/d40/fa5 x:0 0 0 2024-03-29T08:07:21.179 INFO:tasks.workunit.client.1.smithi148.stdout:0/560: dread db/de/df/d11/d2d/d3e/f45 [0,4194304] 0 2024-03-29T08:07:21.180 INFO:tasks.workunit.client.0.smithi037.stdout:9/627: chown d1/d4/d19/d1e/d4a/d6f/da2/d23/f39 62 1 2024-03-29T08:07:21.181 INFO:tasks.workunit.client.1.smithi148.stdout:9/572: rename db/d22/d3f/d61 to db/d22/d26/d54/d4d/d35/d4b/dc3 0 2024-03-29T08:07:21.183 INFO:tasks.workunit.client.1.smithi148.stdout:3/598: read de/d3b/d49/f8a [4133902,104564] 0 2024-03-29T08:07:21.188 INFO:tasks.workunit.client.1.smithi148.stdout:7/527: dwrite d8/d14/f2b [0,4194304] 0 2024-03-29T08:07:21.192 INFO:tasks.workunit.client.0.smithi037.stdout:9/628: rename d1/d4/d19/d1e/d4a/d6f/da2/d31/d87/cc4 to d1/d4/d19/d1e/d4a/d5c/cd9 0 2024-03-29T08:07:21.196 INFO:tasks.workunit.client.1.smithi148.stdout:7/528: creat d8/d29/fba x:0 0 0 2024-03-29T08:07:21.196 INFO:tasks.workunit.client.1.smithi148.stdout:8/591: dread d17/d4e/f71 [0,4194304] 0 2024-03-29T08:07:21.197 INFO:tasks.workunit.client.1.smithi148.stdout:9/573: write f4 [4282606,83697] 0 2024-03-29T08:07:21.198 INFO:tasks.workunit.client.0.smithi037.stdout:9/629: creat d1/d4/d19/d69/fda x:0 0 0 2024-03-29T08:07:21.200 INFO:tasks.workunit.client.1.smithi148.stdout:0/561: creat db/de/df/d11/d2d/fb2 x:0 0 0 2024-03-29T08:07:21.200 INFO:tasks.workunit.client.1.smithi148.stdout:6/574: dwrite d6/d9/d1e/d27/f6e [0,4194304] 0 2024-03-29T08:07:21.205 INFO:tasks.workunit.client.1.smithi148.stdout:4/574: fsync df/d2c/d51/d77/d26/d34/d3e/f58 0 2024-03-29T08:07:21.208 INFO:tasks.workunit.client.1.smithi148.stdout:2/601: rmdir d7/da/d52/d96 0 2024-03-29T08:07:21.209 INFO:tasks.workunit.client.1.smithi148.stdout:8/592: mknod d17/d49/cc5 0 2024-03-29T08:07:21.210 INFO:tasks.workunit.client.1.smithi148.stdout:7/529: symlink d8/d38/lbb 0 2024-03-29T08:07:21.210 INFO:tasks.workunit.client.1.smithi148.stdout:9/574: rmdir db/d20 39 2024-03-29T08:07:21.214 INFO:tasks.workunit.client.1.smithi148.stdout:4/575: read df/d2c/d51/d77/d26/f3d [3749634,29880] 0 2024-03-29T08:07:21.215 INFO:tasks.workunit.client.1.smithi148.stdout:7/530: dread - d8/d38/d3c/da5/fae zero size 2024-03-29T08:07:21.217 INFO:tasks.workunit.client.0.smithi037.stdout:9/630: dwrite d1/d4/d19/d1e/d4a/d6f/da2/d23/f9a [0,4194304] 0 2024-03-29T08:07:21.217 INFO:tasks.workunit.client.1.smithi148.stdout:2/602: write d7/da/d33/d45/d4d/f47 [5585746,35143] 0 2024-03-29T08:07:21.218 INFO:tasks.workunit.client.1.smithi148.stdout:8/593: mkdir d17/d18/d26/d63/d7d/dc6 0 2024-03-29T08:07:21.224 INFO:tasks.workunit.client.0.smithi037.stdout:9/631: read d1/d4/d19/d1e/d4a/d6f/da2/f12 [541469,97343] 0 2024-03-29T08:07:21.227 INFO:tasks.workunit.client.0.smithi037.stdout:9/632: fsync d1/d4/d19/d1e/d4a/d6f/da2/d13/d7c/faf 0 2024-03-29T08:07:21.255 INFO:tasks.workunit.client.1.smithi148.stdout:3/599: dwrite de/f2f [0,4194304] 0 2024-03-29T08:07:21.256 INFO:tasks.workunit.client.1.smithi148.stdout:8/594: stat d17/d49/d9a/db8/fba 0 2024-03-29T08:07:21.256 INFO:tasks.workunit.client.1.smithi148.stdout:2/603: mknod d7/d8c/da4/d8e/cc1 0 2024-03-29T08:07:21.257 INFO:tasks.workunit.client.1.smithi148.stdout:7/531: read d8/d14/f2b [3226331,52666] 0 2024-03-29T08:07:21.257 INFO:tasks.workunit.client.1.smithi148.stdout:4/576: chown df/d2c/d51/d77/d26/d34/db3 2 1 2024-03-29T08:07:21.258 INFO:tasks.workunit.client.0.smithi037.stdout:5/549: sync 2024-03-29T08:07:21.261 INFO:tasks.workunit.client.0.smithi037.stdout:5/550: mkdir d8/db9 0 2024-03-29T08:07:21.263 INFO:tasks.workunit.client.0.smithi037.stdout:5/551: creat d8/d34/fba x:0 0 0 2024-03-29T08:07:21.267 INFO:tasks.workunit.client.1.smithi148.stdout:2/604: creat d7/d24/d4b/fc2 x:0 0 0 2024-03-29T08:07:21.269 INFO:tasks.workunit.client.1.smithi148.stdout:8/595: write d17/d18/d26/d4f/d94/fa2 [863672,75460] 0 2024-03-29T08:07:21.273 INFO:tasks.workunit.client.1.smithi148.stdout:4/577: chown df/d2c/d51/d77/d26/d34 5 1 2024-03-29T08:07:21.274 INFO:tasks.workunit.client.1.smithi148.stdout:0/562: dwrite db/de/f62 [0,4194304] 0 2024-03-29T08:07:21.275 INFO:tasks.workunit.client.1.smithi148.stdout:7/532: dread d8/d38/d3c/f3f [0,4194304] 0 2024-03-29T08:07:21.280 INFO:tasks.workunit.client.1.smithi148.stdout:3/600: chown de/d24/d33/d39/d46/d30/d37/d62/d63/f92 91407 1 2024-03-29T08:07:21.281 INFO:tasks.workunit.client.0.smithi037.stdout:9/633: dwrite d1/d4/d19/d1e/d4a/d6f/f86 [0,4194304] 0 2024-03-29T08:07:21.285 INFO:tasks.workunit.client.1.smithi148.stdout:9/575: dread db/d20/f1e [0,4194304] 0 2024-03-29T08:07:21.289 INFO:tasks.workunit.client.0.smithi037.stdout:5/552: dwrite d8/d4d/d5c/f94 [0,4194304] 0 2024-03-29T08:07:21.291 INFO:tasks.workunit.client.1.smithi148.stdout:3/601: fdatasync de/d24/d33/f5c 0 2024-03-29T08:07:21.291 INFO:tasks.workunit.client.0.smithi037.stdout:9/634: unlink d1/d3/d44/f7d 0 2024-03-29T08:07:21.292 INFO:tasks.workunit.client.1.smithi148.stdout:9/576: fsync db/d22/f89 0 2024-03-29T08:07:21.293 INFO:tasks.workunit.client.1.smithi148.stdout:6/575: dwrite d6/d9/d1e/f26 [0,4194304] 0 2024-03-29T08:07:21.293 INFO:tasks.workunit.client.0.smithi037.stdout:1/478: dread d2/d4b/da1/f96 [0,4194304] 0 2024-03-29T08:07:21.298 INFO:tasks.workunit.client.0.smithi037.stdout:9/635: symlink d1/d1f/ldb 0 2024-03-29T08:07:21.299 INFO:tasks.workunit.client.0.smithi037.stdout:1/479: write d2/d6/dc/f15 [4123328,48409] 0 2024-03-29T08:07:21.304 INFO:tasks.workunit.client.1.smithi148.stdout:3/602: chown de/d24/d33/d39/d46/d30/f3f 142343 1 2024-03-29T08:07:21.306 INFO:tasks.workunit.client.1.smithi148.stdout:2/605: dwrite d7/da/d33/d45/d4d/f7a [0,4194304] 0 2024-03-29T08:07:21.310 INFO:tasks.workunit.client.0.smithi037.stdout:5/553: dread d8/d22/d4f/f5a [0,4194304] 0 2024-03-29T08:07:21.310 INFO:tasks.workunit.client.1.smithi148.stdout:2/606: chown d7/da/d33/d45/d4d/f47 1 1 2024-03-29T08:07:21.312 INFO:tasks.workunit.client.1.smithi148.stdout:2/607: dread - d7/da/d52/fba zero size 2024-03-29T08:07:21.314 INFO:tasks.workunit.client.1.smithi148.stdout:9/577: creat db/d10/d2d/d4e/d5f/d6e/db0/fc4 x:0 0 0 2024-03-29T08:07:21.317 INFO:tasks.workunit.client.1.smithi148.stdout:9/578: truncate db/d20/f1e 4288987 0 2024-03-29T08:07:21.319 INFO:tasks.workunit.client.0.smithi037.stdout:9/636: symlink d1/d4/d19/d1e/d4a/d6f/da2/d31/daa/dba/ldc 0 2024-03-29T08:07:21.320 INFO:tasks.workunit.client.1.smithi148.stdout:9/579: fdatasync db/d10/d2d/d4e/d5f/f65 0 2024-03-29T08:07:21.321 INFO:tasks.workunit.client.0.smithi037.stdout:5/554: mknod d8/d22/d4f/d70/d79/d7a/d85/d90/cbb 0 2024-03-29T08:07:21.325 INFO:tasks.workunit.client.0.smithi037.stdout:9/637: write d1/d4/d19/d1e/d4a/d6f/da2/d23/f37 [693281,129870] 0 2024-03-29T08:07:21.326 INFO:tasks.workunit.client.0.smithi037.stdout:5/555: truncate d8/d18/d24/f74 452208 0 2024-03-29T08:07:21.327 INFO:tasks.workunit.client.1.smithi148.stdout:9/580: write db/d10/d2d/d4e/d5f/d6e/d7a/f7e [149323,7606] 0 2024-03-29T08:07:21.328 INFO:tasks.workunit.client.0.smithi037.stdout:1/480: link d2/d6/dc/d58/fa2 d2/d6/dc/d18/d22/d4f/d7c/fa6 0 2024-03-29T08:07:21.332 INFO:tasks.workunit.client.1.smithi148.stdout:0/563: write db/de/f70 [1509038,122338] 0 2024-03-29T08:07:21.334 INFO:tasks.workunit.client.0.smithi037.stdout:1/481: fdatasync d2/d6/dc/d5d/d62/f97 0 2024-03-29T08:07:21.337 INFO:tasks.workunit.client.0.smithi037.stdout:9/638: mkdir d1/d1f/d3a/d9b/ddd 0 2024-03-29T08:07:21.338 INFO:tasks.workunit.client.1.smithi148.stdout:8/596: truncate d17/d49/d9a/dbe/d6f/f5b 2404664 0 2024-03-29T08:07:21.340 INFO:tasks.workunit.client.1.smithi148.stdout:3/603: dwrite de/f80 [0,4194304] 0 2024-03-29T08:07:21.340 INFO:tasks.workunit.client.1.smithi148.stdout:2/608: dwrite d7/d24/d4b/d66/d7d/fbe [0,4194304] 0 2024-03-29T08:07:21.341 INFO:tasks.workunit.client.0.smithi037.stdout:9/639: write d1/d4/f3b [3299778,68297] 0 2024-03-29T08:07:21.341 INFO:tasks.workunit.client.0.smithi037.stdout:1/482: read d2/d6/dc/d18/d1d/d21/f60 [363794,111605] 0 2024-03-29T08:07:21.343 INFO:tasks.workunit.client.1.smithi148.stdout:0/564: stat db/de/d24 0 2024-03-29T08:07:21.347 INFO:tasks.workunit.client.1.smithi148.stdout:4/578: dread df/d2c/d51/d77/d17/f29 [4194304,4194304] 0 2024-03-29T08:07:21.350 INFO:tasks.workunit.client.0.smithi037.stdout:9/640: creat d1/d4/fde x:0 0 0 2024-03-29T08:07:21.350 INFO:tasks.workunit.client.1.smithi148.stdout:4/579: stat df/d60/f85 0 2024-03-29T08:07:21.351 INFO:tasks.workunit.client.1.smithi148.stdout:2/609: chown d7/da/d4c/f69 5 1 2024-03-29T08:07:21.353 INFO:tasks.workunit.client.0.smithi037.stdout:9/641: chown d1/d4/d19/d1e/d4a/fa0 1124 1 2024-03-29T08:07:21.354 INFO:tasks.workunit.client.1.smithi148.stdout:8/597: mknod d17/d49/d9a/dbe/d6f/d64/cc7 0 2024-03-29T08:07:21.357 INFO:tasks.workunit.client.1.smithi148.stdout:3/604: chown de/d24/d33/d39/d46/d70 513664 1 2024-03-29T08:07:21.358 INFO:tasks.workunit.client.1.smithi148.stdout:4/580: getdents df/d2c/d80/da6 0 2024-03-29T08:07:21.358 INFO:tasks.workunit.client.0.smithi037.stdout:5/556: rename d8/d22/d4f/d70/d79/d7a to d8/d22/d37/dbc 0 2024-03-29T08:07:21.359 INFO:tasks.workunit.client.1.smithi148.stdout:7/533: dwrite d8/d38/d3c/f5a [0,4194304] 0 2024-03-29T08:07:21.361 INFO:tasks.workunit.client.0.smithi037.stdout:9/642: read d1/d4/d19/d1e/d25/fc1 [697447,34681] 0 2024-03-29T08:07:21.365 INFO:tasks.workunit.client.1.smithi148.stdout:8/598: dread - d17/d49/d9a/dbe/d6f/d64/da5/fb2 zero size 2024-03-29T08:07:21.366 INFO:tasks.workunit.client.0.smithi037.stdout:8/443: sync 2024-03-29T08:07:21.366 INFO:tasks.workunit.client.0.smithi037.stdout:3/398: sync 2024-03-29T08:07:21.366 INFO:tasks.workunit.client.0.smithi037.stdout:1/483: write d2/d6/dc/d18/d1d/d21/d99/f4e [1107905,57282] 0 2024-03-29T08:07:21.369 INFO:tasks.workunit.client.1.smithi148.stdout:2/610: creat d7/da/d33/d45/d4d/fc3 x:0 0 0 2024-03-29T08:07:21.372 INFO:tasks.workunit.client.0.smithi037.stdout:8/444: write d5/d10/d2d/d34/d50/f68 [303263,67478] 0 2024-03-29T08:07:21.376 INFO:tasks.workunit.client.1.smithi148.stdout:2/611: read d7/da/f89 [2259537,110093] 0 2024-03-29T08:07:21.376 INFO:tasks.workunit.client.1.smithi148.stdout:9/581: dwrite db/d22/d26/d54/d4d/f79 [0,4194304] 0 2024-03-29T08:07:21.377 INFO:tasks.workunit.client.1.smithi148.stdout:4/581: read df/d2c/d51/d77/d26/d34/d43/f59 [132082,43777] 0 2024-03-29T08:07:21.380 INFO:tasks.workunit.client.1.smithi148.stdout:7/534: rename d8/d38/d3c/f3f to d8/d14/d45/d4d/d78/fbc 0 2024-03-29T08:07:21.380 INFO:tasks.workunit.client.1.smithi148.stdout:3/605: write de/d3b/f60 [4030073,3163] 0 2024-03-29T08:07:21.383 INFO:tasks.workunit.client.0.smithi037.stdout:3/399: fdatasync dd/d5e/d7a/f4f 0 2024-03-29T08:07:21.383 INFO:tasks.workunit.client.1.smithi148.stdout:2/612: mkdir d7/da/d4c/dc4 0 2024-03-29T08:07:21.386 INFO:tasks.workunit.client.1.smithi148.stdout:9/582: write db/d10/d2d/d4e/d5f/f65 [1467368,89866] 0 2024-03-29T08:07:21.387 INFO:tasks.workunit.client.1.smithi148.stdout:4/582: read - df/d2c/d51/d77/d26/f63 zero size 2024-03-29T08:07:21.388 INFO:tasks.workunit.client.0.smithi037.stdout:1/484: fdatasync d2/d6/d13/f28 0 2024-03-29T08:07:21.389 INFO:tasks.workunit.client.0.smithi037.stdout:8/445: creat d5/d10/d19/d3d/d4a/d4c/f8f x:0 0 0 2024-03-29T08:07:21.389 INFO:tasks.workunit.client.0.smithi037.stdout:5/557: dread d8/d22/f26 [0,4194304] 0 2024-03-29T08:07:21.390 INFO:tasks.workunit.client.1.smithi148.stdout:7/535: rename d8/d14/d45/f66 to d8/d38/d6a/fbd 0 2024-03-29T08:07:21.391 INFO:tasks.workunit.client.0.smithi037.stdout:9/643: write d1/d4/d19/d1e/d4a/d5c/f5d [559126,129133] 0 2024-03-29T08:07:21.393 INFO:tasks.workunit.client.1.smithi148.stdout:8/599: mknod d17/d49/d9a/dbe/cc8 0 2024-03-29T08:07:21.393 INFO:tasks.workunit.client.1.smithi148.stdout:2/613: rmdir d7/da/d33/d45/d4d 39 2024-03-29T08:07:21.394 INFO:tasks.workunit.client.0.smithi037.stdout:1/485: write d2/d6/dc/f35 [4581414,52255] 0 2024-03-29T08:07:21.394 INFO:tasks.workunit.client.0.smithi037.stdout:5/558: write d8/d4d/d5c/f94 [1789369,46317] 0 2024-03-29T08:07:21.395 INFO:tasks.workunit.client.1.smithi148.stdout:6/576: dread d6/d9/d1e/d20/d30/f35 [0,4194304] 0 2024-03-29T08:07:21.398 INFO:tasks.workunit.client.0.smithi037.stdout:3/400: stat dd/d46/d2d/d3a/d5f/c34 0 2024-03-29T08:07:21.402 INFO:tasks.workunit.client.1.smithi148.stdout:2/614: write d7/da/d33/d45/d4d/d41/f6a [689077,98092] 0 2024-03-29T08:07:21.403 INFO:tasks.workunit.client.0.smithi037.stdout:5/559: truncate d8/d22/d37/d60/faf 716349 0 2024-03-29T08:07:21.404 INFO:tasks.workunit.client.1.smithi148.stdout:4/583: mknod df/d2c/db2/cb8 0 2024-03-29T08:07:21.405 INFO:tasks.workunit.client.1.smithi148.stdout:9/583: symlink db/d22/d26/d54/d4d/dbb/dc0/lc5 0 2024-03-29T08:07:21.405 INFO:tasks.workunit.client.1.smithi148.stdout:6/577: chown d6/c49 1 1 2024-03-29T08:07:21.410 INFO:tasks.workunit.client.0.smithi037.stdout:7/430: sync 2024-03-29T08:07:21.410 INFO:tasks.workunit.client.0.smithi037.stdout:6/371: sync 2024-03-29T08:07:21.411 INFO:tasks.workunit.client.0.smithi037.stdout:0/311: sync 2024-03-29T08:07:21.411 INFO:tasks.workunit.client.0.smithi037.stdout:8/446: dwrite d5/d10/d19/d3d/d4a/d4c/f73 [4194304,4194304] 0 2024-03-29T08:07:21.411 INFO:tasks.workunit.client.0.smithi037.stdout:4/436: sync 2024-03-29T08:07:21.411 INFO:tasks.workunit.client.0.smithi037.stdout:2/317: sync 2024-03-29T08:07:21.412 INFO:tasks.workunit.client.1.smithi148.stdout:4/584: symlink df/d2c/d80/daa/lb9 0 2024-03-29T08:07:21.412 INFO:tasks.workunit.client.1.smithi148.stdout:7/536: dwrite d8/d38/f8d [0,4194304] 0 2024-03-29T08:07:21.415 INFO:tasks.workunit.client.0.smithi037.stdout:4/437: write d3/d4/d15/f50 [299554,2611] 0 2024-03-29T08:07:21.415 INFO:tasks.workunit.client.1.smithi148.stdout:4/585: chown df/d2c/d51/d77/d26/f4a 15 1 2024-03-29T08:07:21.416 INFO:tasks.workunit.client.1.smithi148.stdout:8/600: dwrite d17/d49/d9a/dbe/d6f/f45 [0,4194304] 0 2024-03-29T08:07:21.417 INFO:tasks.workunit.client.0.smithi037.stdout:8/447: write d5/d10/d2d/d34/d50/f68 [2112767,100256] 0 2024-03-29T08:07:21.418 INFO:tasks.workunit.client.0.smithi037.stdout:4/438: stat d3/c5 0 2024-03-29T08:07:21.419 INFO:tasks.workunit.client.0.smithi037.stdout:9/644: mkdir d1/d3/ddf 0 2024-03-29T08:07:21.419 INFO:tasks.workunit.client.1.smithi148.stdout:8/601: chown d17/d49/d9a/dbe/d6f/d64 1 1 2024-03-29T08:07:21.420 INFO:tasks.workunit.client.1.smithi148.stdout:8/602: read fe [1243452,38489] 0 2024-03-29T08:07:21.421 INFO:tasks.workunit.client.0.smithi037.stdout:5/560: dread d8/d22/f26 [0,4194304] 0 2024-03-29T08:07:21.423 INFO:tasks.workunit.client.1.smithi148.stdout:0/565: dread db/de/df/d11/d42/f67 [0,4194304] 0 2024-03-29T08:07:21.424 INFO:tasks.workunit.client.0.smithi037.stdout:8/448: dread - d5/d10/d19/d3d/d4a/d4c/d67/d85/f8e zero size 2024-03-29T08:07:21.425 INFO:tasks.workunit.client.0.smithi037.stdout:1/486: fdatasync d2/d6/dc/d18/f29 0 2024-03-29T08:07:21.426 INFO:tasks.workunit.client.1.smithi148.stdout:9/584: mknod db/d10/d2d/d4e/d5f/d6e/d7a/dbd/cc6 0 2024-03-29T08:07:21.426 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:07:21 smithi037 ceph-mon[35162]: pgmap v42: 65 pgs: 65 active+clean; 1.8 GiB data, 7.1 GiB used, 529 GiB / 536 GiB avail; 59 MiB/s rd, 241 MiB/s wr, 434 op/s 2024-03-29T08:07:21.427 INFO:tasks.workunit.client.0.smithi037.stdout:5/561: truncate d8/d22/f93 929539 0 2024-03-29T08:07:21.429 INFO:tasks.workunit.client.0.smithi037.stdout:5/562: chown d8/ld 31434 1 2024-03-29T08:07:21.432 INFO:tasks.workunit.client.1.smithi148.stdout:8/603: dread d17/d18/f3f [0,4194304] 0 2024-03-29T08:07:21.438 INFO:tasks.workunit.client.1.smithi148.stdout:9/585: symlink db/d22/d26/d54/d4d/dbb/dc0/lc7 0 2024-03-29T08:07:21.442 INFO:tasks.workunit.client.1.smithi148.stdout:6/578: write d6/d9/d1e/d27/f64 [756270,108057] 0 2024-03-29T08:07:21.444 INFO:tasks.workunit.client.1.smithi148.stdout:9/586: unlink db/d10/d6d/d73/c9f 0 2024-03-29T08:07:21.446 INFO:tasks.workunit.client.1.smithi148.stdout:6/579: mkdir d6/d9/d1e/d27/d48/d73/d7e/db6 0 2024-03-29T08:07:21.449 INFO:tasks.workunit.client.0.smithi037.stdout:6/372: creat d0/d19/d42/d60/f7d x:0 0 0 2024-03-29T08:07:21.454 INFO:tasks.workunit.client.1.smithi148.stdout:2/615: dwrite d7/da/d33/d45/d4d/d2e/d37/f43 [4194304,4194304] 0 2024-03-29T08:07:21.457 INFO:tasks.workunit.client.1.smithi148.stdout:6/580: mkdir d6/d9/d1e/d20/db7 0 2024-03-29T08:07:21.461 INFO:tasks.workunit.client.0.smithi037.stdout:3/401: dwrite dd/d46/d2d/d3a/f53 [0,4194304] 0 2024-03-29T08:07:21.462 INFO:tasks.workunit.client.0.smithi037.stdout:9/645: write d1/d4/d19/d1e/d4a/fa0 [4044541,11871] 0 2024-03-29T08:07:21.463 INFO:tasks.workunit.client.1.smithi148.stdout:2/616: chown d7/da/d33/d45/d4d/d82/f9f 4 1 2024-03-29T08:07:21.464 INFO:tasks.workunit.client.0.smithi037.stdout:7/431: mkdir dd/d21/d35/d42/d5c/d8c 0 2024-03-29T08:07:21.466 INFO:tasks.workunit.client.1.smithi148.stdout:9/587: dread db/d22/d3f/f42 [0,4194304] 0 2024-03-29T08:07:21.471 INFO:tasks.workunit.client.1.smithi148.stdout:4/586: dread df/f14 [0,4194304] 0 2024-03-29T08:07:21.471 INFO:tasks.workunit.client.1.smithi148.stdout:0/566: dwrite db/de/df/d11/d42/f67 [4194304,4194304] 0 2024-03-29T08:07:21.472 INFO:tasks.workunit.client.0.smithi037.stdout:2/318: write d0/f40 [3691787,18559] 0 2024-03-29T08:07:21.472 INFO:tasks.workunit.client.1.smithi148.stdout:6/581: rename d6/d9/d1e/d27/c85 to d6/d9/d1e/d20/db7/cb8 0 2024-03-29T08:07:21.473 INFO:tasks.workunit.client.0.smithi037.stdout:0/312: dread f5 [4194304,4194304] 0 2024-03-29T08:07:21.477 INFO:tasks.workunit.client.1.smithi148.stdout:9/588: unlink db/d22/d26/d54/d4d/f8d 0 2024-03-29T08:07:21.478 INFO:tasks.workunit.client.1.smithi148.stdout:9/589: stat db/d22/d26/d54/d52/dab 0 2024-03-29T08:07:21.479 INFO:tasks.workunit.client.1.smithi148.stdout:4/587: fdatasync df/d2c/d51/d77/d26/f63 0 2024-03-29T08:07:21.482 INFO:tasks.workunit.client.0.smithi037.stdout:6/373: rmdir d0 39 2024-03-29T08:07:21.482 INFO:tasks.workunit.client.1.smithi148.stdout:4/588: dread - df/d2c/d51/d77/d26/d9b/da7/f8b zero size 2024-03-29T08:07:21.485 INFO:tasks.workunit.client.1.smithi148.stdout:8/604: link d17/d1d/d7e/lc4 d17/d18/d26/d63/d66/lc9 0 2024-03-29T08:07:21.485 INFO:tasks.workunit.client.1.smithi148.stdout:6/582: symlink d6/d9/d1e/d20/lb9 0 2024-03-29T08:07:21.486 INFO:tasks.workunit.client.0.smithi037.stdout:3/402: dread dd/d46/f3c [0,4194304] 0 2024-03-29T08:07:21.489 INFO:tasks.workunit.client.1.smithi148.stdout:2/617: rename c0 to d7/da/cc5 0 2024-03-29T08:07:21.490 INFO:tasks.workunit.client.1.smithi148.stdout:0/567: unlink db/dc/daa/d47/d4c/d5a/fab 0 2024-03-29T08:07:21.492 INFO:tasks.workunit.client.0.smithi037.stdout:2/319: dread d0/d20/d22/f50 [0,4194304] 0 2024-03-29T08:07:21.494 INFO:tasks.workunit.client.1.smithi148.stdout:9/590: creat db/d22/d26/d54/d4d/d35/d4b/fc8 x:0 0 0 2024-03-29T08:07:21.496 INFO:tasks.workunit.client.1.smithi148.stdout:4/589: symlink df/d2c/d51/d77/d26/d9b/da7/d7c/lba 0 2024-03-29T08:07:21.498 INFO:tasks.workunit.client.0.smithi037.stdout:8/449: rename d5/d10/d2d/f60 to d5/d10/d19/d8d/f90 0 2024-03-29T08:07:21.498 INFO:tasks.workunit.client.1.smithi148.stdout:6/583: mkdir d6/d9/d1e/d36/d4d/d9d/d28/dba 0 2024-03-29T08:07:21.500 INFO:tasks.workunit.client.1.smithi148.stdout:7/537: dwrite f6 [0,4194304] 0 2024-03-29T08:07:21.501 INFO:tasks.workunit.client.0.smithi037.stdout:7/432: dwrite dd/d21/d35/d42/d5c/f74 [0,4194304] 0 2024-03-29T08:07:21.508 INFO:tasks.workunit.client.1.smithi148.stdout:7/538: rmdir d8/d38/d67 39 2024-03-29T08:07:21.508 INFO:tasks.workunit.client.1.smithi148.stdout:6/584: rename d6/d9/f1f to d6/d9/d1e/fbb 0 2024-03-29T08:07:21.509 INFO:tasks.workunit.client.0.smithi037.stdout:2/320: stat d0/c13 0 2024-03-29T08:07:21.510 INFO:tasks.workunit.client.0.smithi037.stdout:2/321: stat d0/d20/f49 0 2024-03-29T08:07:21.511 INFO:tasks.workunit.client.0.smithi037.stdout:9/646: mkdir d1/d4/d19/d1e/d4a/d6f/da2/d13/de0 0 2024-03-29T08:07:21.513 INFO:tasks.workunit.client.0.smithi037.stdout:8/450: read - d5/d10/d2d/d34/d50/f54 zero size 2024-03-29T08:07:21.517 INFO:tasks.workunit.client.0.smithi037.stdout:0/313: link d9/da/dd/d13/f45 d9/da/dd/d13/f61 0 2024-03-29T08:07:21.519 INFO:tasks.workunit.client.0.smithi037.stdout:1/487: dwrite d2/d6/dc/d18/d1d/d21/f37 [0,4194304] 0 2024-03-29T08:07:21.520 INFO:tasks.workunit.client.1.smithi148.stdout:7/539: creat d8/d14/d45/fbe x:0 0 0 2024-03-29T08:07:21.521 INFO:tasks.workunit.client.0.smithi037.stdout:4/439: getdents d3/d80 0 2024-03-29T08:07:21.521 INFO:tasks.workunit.client.1.smithi148.stdout:0/568: dwrite db/dc/daa/d47/daf/f99 [0,4194304] 0 2024-03-29T08:07:21.524 INFO:tasks.workunit.client.0.smithi037.stdout:5/563: getdents d8/d22 0 2024-03-29T08:07:21.524 INFO:tasks.workunit.client.0.smithi037.stdout:0/314: dread - d9/da/dd/d13/f5c zero size 2024-03-29T08:07:21.525 INFO:tasks.workunit.client.0.smithi037.stdout:2/322: chown d0/l6e 121134 1 2024-03-29T08:07:21.528 INFO:tasks.workunit.client.0.smithi037.stdout:8/451: mknod d5/d10/d19/d3d/d4a/c91 0 2024-03-29T08:07:21.529 INFO:tasks.workunit.client.1.smithi148.stdout:2/618: dwrite d7/da/d33/d45/d4d/d2e/f50 [0,4194304] 0 2024-03-29T08:07:21.530 INFO:tasks.workunit.client.0.smithi037.stdout:0/315: write d9/f27 [5451559,27862] 0 2024-03-29T08:07:21.530 INFO:tasks.workunit.client.0.smithi037.stdout:4/440: truncate d3/d54/f5b 902362 0 2024-03-29T08:07:21.532 INFO:tasks.workunit.client.0.smithi037.stdout:9/647: symlink d1/d4/d19/d1e/le1 0 2024-03-29T08:07:21.533 INFO:tasks.workunit.client.1.smithi148.stdout:9/591: dwrite db/d22/d26/d54/d4d/d35/d4b/d8e/f97 [0,4194304] 0 2024-03-29T08:07:21.533 INFO:tasks.workunit.client.1.smithi148.stdout:8/605: dread d17/d18/d26/f35 [0,4194304] 0 2024-03-29T08:07:21.536 INFO:tasks.workunit.client.0.smithi037.stdout:4/441: fsync d3/d57/f8e 0 2024-03-29T08:07:21.537 INFO:tasks.workunit.client.0.smithi037.stdout:3/403: mknod dd/d15/d27/d33/d75/c88 0 2024-03-29T08:07:21.539 INFO:tasks.workunit.client.1.smithi148.stdout:6/585: getdents d6/d9/d1e/d36/d4d/d9d 0 2024-03-29T08:07:21.540 INFO:tasks.workunit.client.1.smithi148.stdout:8/606: fdatasync d17/d74/fb1 0 2024-03-29T08:07:21.546 INFO:tasks.workunit.client.1.smithi148.stdout:6/586: creat d6/d9/d1e/fbc x:0 0 0 2024-03-29T08:07:21.547 INFO:tasks.workunit.client.0.smithi037.stdout:6/374: dwrite d0/d5/d1e/d44/d71/f74 [0,4194304] 0 2024-03-29T08:07:21.547 INFO:tasks.workunit.client.1.smithi148.stdout:0/569: readlink db/dc/daa/d47/d4c/d5a/l60 0 2024-03-29T08:07:21.549 INFO:tasks.workunit.client.0.smithi037.stdout:0/316: chown d9/da/dd/d59/f39 0 1 2024-03-29T08:07:21.555 INFO:tasks.workunit.client.1.smithi148.stdout:8/607: unlink d17/d74/f4a 0 2024-03-29T08:07:21.555 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:07:21 smithi148 ceph-mon[44992]: pgmap v42: 65 pgs: 65 active+clean; 1.8 GiB data, 7.1 GiB used, 529 GiB / 536 GiB avail; 59 MiB/s rd, 241 MiB/s wr, 434 op/s 2024-03-29T08:07:21.559 INFO:tasks.workunit.client.0.smithi037.stdout:4/442: symlink d3/d4/d15/d1d/l92 0 2024-03-29T08:07:21.565 INFO:tasks.workunit.client.1.smithi148.stdout:4/590: truncate df/d2c/d51/d77/d17/f4f 1771444 0 2024-03-29T08:07:21.566 INFO:tasks.workunit.client.0.smithi037.stdout:3/404: creat dd/d15/d27/f89 x:0 0 0 2024-03-29T08:07:21.567 INFO:tasks.workunit.client.1.smithi148.stdout:0/570: truncate db/de/df/d11/d2d/d3e/f6d 451685 0 2024-03-29T08:07:21.567 INFO:tasks.workunit.client.1.smithi148.stdout:7/540: dwrite d8/d29/d44/d49/fa3 [0,4194304] 0 2024-03-29T08:07:21.568 INFO:tasks.workunit.client.1.smithi148.stdout:2/619: dwrite d7/da/d33/d3c/d54/fb2 [0,4194304] 0 2024-03-29T08:07:21.568 INFO:tasks.workunit.client.1.smithi148.stdout:4/591: stat df/l2a 0 2024-03-29T08:07:21.571 INFO:tasks.workunit.client.0.smithi037.stdout:2/323: mkdir d0/d23/d70/d71 0 2024-03-29T08:07:21.574 INFO:tasks.workunit.client.0.smithi037.stdout:7/433: dwrite dd/d1d/f53 [0,4194304] 0 2024-03-29T08:07:21.574 INFO:tasks.workunit.client.0.smithi037.stdout:2/324: chown d0/d20/c27 772 1 2024-03-29T08:07:21.574 INFO:tasks.workunit.client.1.smithi148.stdout:2/620: truncate d7/da/d4c/d9a/fbd 587056 0 2024-03-29T08:07:21.579 INFO:tasks.workunit.client.1.smithi148.stdout:0/571: mknod db/de/d8e/cb3 0 2024-03-29T08:07:21.583 INFO:tasks.workunit.client.1.smithi148.stdout:4/592: creat df/d2c/d51/d77/d26/d9b/da7/fbb x:0 0 0 2024-03-29T08:07:21.584 INFO:tasks.workunit.client.1.smithi148.stdout:2/621: stat d7/da/d33/d45/c6e 0 2024-03-29T08:07:21.588 INFO:tasks.workunit.client.1.smithi148.stdout:7/541: creat d8/d38/d3c/da9/db7/fbf x:0 0 0 2024-03-29T08:07:21.588 INFO:tasks.workunit.client.1.smithi148.stdout:9/592: dread db/d22/d26/d54/d4d/f41 [0,4194304] 0 2024-03-29T08:07:21.589 INFO:tasks.workunit.client.1.smithi148.stdout:2/622: truncate d7/d24/d4b/fc2 19489 0 2024-03-29T08:07:21.590 INFO:tasks.workunit.client.0.smithi037.stdout:6/375: mknod d0/d19/d42/c7e 0 2024-03-29T08:07:21.594 INFO:tasks.workunit.client.1.smithi148.stdout:4/593: mkdir df/d2c/d80/daa/dbc 0 2024-03-29T08:07:21.594 INFO:tasks.workunit.client.1.smithi148.stdout:0/572: rename db/de/df/d11/d2d/d3e/f45 to db/dc/daa/d47/d7c/fb4 0 2024-03-29T08:07:21.596 INFO:tasks.workunit.client.1.smithi148.stdout:2/623: mkdir d7/da/d4c/dc4/dc6 0 2024-03-29T08:07:21.598 INFO:tasks.workunit.client.0.smithi037.stdout:8/452: rename f1 to d5/d10/f92 0 2024-03-29T08:07:21.600 INFO:tasks.workunit.client.1.smithi148.stdout:9/593: getdents db/d22/d26/d54/d4d/d35/d4b/d5a/d74 0 2024-03-29T08:07:21.602 INFO:tasks.workunit.client.1.smithi148.stdout:4/594: link df/d2c/d51/d77/d26/f4a df/d2c/d51/d77/d17/fbd 0 2024-03-29T08:07:21.602 INFO:tasks.workunit.client.1.smithi148.stdout:7/542: mknod d8/d14/d45/d4d/d78/cc0 0 2024-03-29T08:07:21.602 INFO:tasks.workunit.client.1.smithi148.stdout:2/624: truncate d7/da/d33/d45/d4d/d2e/d37/f67 3081816 0 2024-03-29T08:07:21.604 INFO:tasks.workunit.client.1.smithi148.stdout:9/594: stat db/d22/d26/d54/f2e 0 2024-03-29T08:07:21.606 INFO:tasks.workunit.client.1.smithi148.stdout:2/625: dread d7/d24/d4b/d66/d7d/f8f [0,4194304] 0 2024-03-29T08:07:21.608 INFO:tasks.workunit.client.1.smithi148.stdout:9/595: write db/d20/f1b [2186150,114153] 0 2024-03-29T08:07:21.609 INFO:tasks.workunit.client.0.smithi037.stdout:0/317: link d9/f27 d9/da/dd/d20/d38/d53/f62 0 2024-03-29T08:07:21.609 INFO:tasks.workunit.client.0.smithi037.stdout:1/488: dread d2/d4b/d80/f8b [0,4194304] 0 2024-03-29T08:07:21.610 INFO:tasks.workunit.client.1.smithi148.stdout:0/573: fdatasync db/de/df/d2b/f76 0 2024-03-29T08:07:21.617 INFO:tasks.workunit.client.0.smithi037.stdout:3/405: write dd/d5e/f45 [4665405,39697] 0 2024-03-29T08:07:21.619 INFO:tasks.workunit.client.1.smithi148.stdout:0/574: rename db to db/de/df/d11/d2d/db5 22 2024-03-29T08:07:21.619 INFO:tasks.workunit.client.1.smithi148.stdout:8/608: dread d17/d18/f5c [0,4194304] 0 2024-03-29T08:07:21.619 INFO:tasks.workunit.client.1.smithi148.stdout:2/626: rmdir d7/d24/db5 0 2024-03-29T08:07:21.620 INFO:tasks.workunit.client.0.smithi037.stdout:9/648: dwrite d1/d4/d19/d1e/d4a/d6f/da2/d23/f39 [0,4194304] 0 2024-03-29T08:07:21.621 INFO:tasks.workunit.client.1.smithi148.stdout:2/627: readlink d7/da/d33/d3c/l30 0 2024-03-29T08:07:21.623 INFO:tasks.workunit.client.1.smithi148.stdout:7/543: mkdir d8/d38/d67/dc1 0 2024-03-29T08:07:21.623 INFO:tasks.workunit.client.1.smithi148.stdout:9/596: creat db/d22/d26/d54/fc9 x:0 0 0 2024-03-29T08:07:21.629 INFO:tasks.workunit.client.1.smithi148.stdout:9/597: mkdir db/d22/d26/d54/d4d/dbb/dc0/dca 0 2024-03-29T08:07:21.636 INFO:tasks.workunit.client.1.smithi148.stdout:9/598: chown db/d22/d26/d54/f49 17628 1 2024-03-29T08:07:21.638 INFO:tasks.workunit.client.1.smithi148.stdout:4/595: dread fa [0,4194304] 0 2024-03-29T08:07:21.639 INFO:tasks.workunit.client.1.smithi148.stdout:2/628: getdents d7/d24 0 2024-03-29T08:07:21.640 INFO:tasks.workunit.client.0.smithi037.stdout:7/434: write dd/d21/d35/d42/f43 [603486,123407] 0 2024-03-29T08:07:21.640 INFO:tasks.workunit.client.1.smithi148.stdout:9/599: mkdir db/d10/d2d/d4e/d5f/d6e/dcb 0 2024-03-29T08:07:21.643 INFO:tasks.workunit.client.0.smithi037.stdout:1/489: creat d2/d6/dc/d18/d22/fa7 x:0 0 0 2024-03-29T08:07:21.644 INFO:tasks.workunit.client.0.smithi037.stdout:2/325: creat d0/d23/d70/d71/f72 x:0 0 0 2024-03-29T08:07:21.644 INFO:tasks.workunit.client.1.smithi148.stdout:0/575: creat db/dc/daa/d47/d4c/d5a/fb6 x:0 0 0 2024-03-29T08:07:21.646 INFO:tasks.workunit.client.0.smithi037.stdout:3/406: readlink dd/d46/d2d/l5b 0 2024-03-29T08:07:21.649 INFO:tasks.workunit.client.1.smithi148.stdout:0/576: read db/dc/daa/d47/f2c [680370,55163] 0 2024-03-29T08:07:21.650 INFO:tasks.workunit.client.0.smithi037.stdout:3/407: fdatasync dd/d46/d2d/d3a/f81 0 2024-03-29T08:07:21.650 INFO:tasks.workunit.client.0.smithi037.stdout:8/453: dread d5/f1e [0,4194304] 0 2024-03-29T08:07:21.650 INFO:tasks.workunit.client.1.smithi148.stdout:7/544: dwrite d8/d29/d44/d50/f90 [0,4194304] 0 2024-03-29T08:07:21.656 INFO:tasks.workunit.client.0.smithi037.stdout:0/318: fsync d9/da/f24 0 2024-03-29T08:07:21.658 INFO:tasks.workunit.client.1.smithi148.stdout:8/609: link d17/d49/d9a/dbe/d6f/d64/c78 d17/d4e/cca 0 2024-03-29T08:07:21.659 INFO:tasks.workunit.client.0.smithi037.stdout:0/319: chown d9/da/f3c 1 1 2024-03-29T08:07:21.659 INFO:tasks.workunit.client.1.smithi148.stdout:7/545: fsync d8/d38/d3c/f7e 0 2024-03-29T08:07:21.660 INFO:tasks.workunit.client.1.smithi148.stdout:9/600: rmdir db 39 2024-03-29T08:07:21.660 INFO:tasks.workunit.client.1.smithi148.stdout:8/610: write d17/d74/f5a [2591790,82283] 0 2024-03-29T08:07:21.664 INFO:tasks.workunit.client.1.smithi148.stdout:4/596: mkdir df/d2c/d51/d77/dbe 0 2024-03-29T08:07:21.666 INFO:tasks.workunit.client.1.smithi148.stdout:0/577: link db/de/df/d11/f16 db/dc/daa/d47/daf/fb7 0 2024-03-29T08:07:21.672 INFO:tasks.workunit.client.1.smithi148.stdout:9/601: mknod db/d22/d26/d54/d52/ccc 0 2024-03-29T08:07:21.673 INFO:tasks.workunit.client.0.smithi037.stdout:9/649: symlink d1/d4/d19/d1e/d4a/d6f/da2/d31/daa/dba/dbb/le2 0 2024-03-29T08:07:21.673 INFO:tasks.workunit.client.0.smithi037.stdout:3/408: dread - dd/f24 zero size 2024-03-29T08:07:21.674 INFO:tasks.workunit.client.0.smithi037.stdout:8/454: symlink d5/d78/l93 0 2024-03-29T08:07:21.676 INFO:tasks.workunit.client.0.smithi037.stdout:2/326: dwrite d0/d20/d22/d4e/f4f [0,4194304] 0 2024-03-29T08:07:21.677 INFO:tasks.workunit.client.1.smithi148.stdout:9/602: fsync db/d22/d26/d54/d4d/f41 0 2024-03-29T08:07:21.678 INFO:tasks.workunit.client.1.smithi148.stdout:9/603: chown db/f5c 81325 1 2024-03-29T08:07:21.679 INFO:tasks.workunit.client.1.smithi148.stdout:9/604: chown db/d10/d2d/l6f 17 1 2024-03-29T08:07:21.681 INFO:tasks.workunit.client.0.smithi037.stdout:7/435: dwrite dd/f1f [0,4194304] 0 2024-03-29T08:07:21.681 INFO:tasks.workunit.client.1.smithi148.stdout:7/546: rename d8/d29/d44/d49/f62 to d8/d14/d45/d4d/d78/fc2 0 2024-03-29T08:07:21.681 INFO:tasks.workunit.client.0.smithi037.stdout:4/443: mkdir d3/d4/d15/d1d/db/d45/d93 0 2024-03-29T08:07:21.682 INFO:tasks.workunit.client.1.smithi148.stdout:0/578: mknod db/de/df/d11/d2d/da3/cb8 0 2024-03-29T08:07:21.683 INFO:tasks.workunit.client.0.smithi037.stdout:9/650: write d1/d4/d19/d1e/d4a/d6f/da2/d23/f65 [903109,93643] 0 2024-03-29T08:07:21.683 INFO:tasks.workunit.client.0.smithi037.stdout:7/436: chown dd/d1d/d49 1 1 2024-03-29T08:07:21.685 INFO:tasks.workunit.client.0.smithi037.stdout:4/444: fsync d3/d4/d27/f86 0 2024-03-29T08:07:21.688 INFO:tasks.workunit.client.1.smithi148.stdout:2/629: dread d7/da/d33/d3c/f65 [0,4194304] 0 2024-03-29T08:07:21.690 INFO:tasks.workunit.client.1.smithi148.stdout:2/630: chown d7/f1b 7 1 2024-03-29T08:07:21.693 INFO:tasks.workunit.client.1.smithi148.stdout:2/631: readlink d7/da/l3a 0 2024-03-29T08:07:21.695 INFO:tasks.workunit.client.1.smithi148.stdout:4/597: dwrite df/d2c/d51/d77/d17/f20 [4194304,4194304] 0 2024-03-29T08:07:21.697 INFO:tasks.workunit.client.1.smithi148.stdout:5/573: sync 2024-03-29T08:07:21.697 INFO:tasks.workunit.client.1.smithi148.stdout:1/553: sync 2024-03-29T08:07:21.697 INFO:tasks.workunit.client.1.smithi148.stdout:3/606: sync 2024-03-29T08:07:21.697 INFO:tasks.workunit.client.1.smithi148.stdout:6/587: sync 2024-03-29T08:07:21.698 INFO:tasks.workunit.client.1.smithi148.stdout:4/598: fdatasync df/d2c/d51/d77/d26/f3b 0 2024-03-29T08:07:21.700 INFO:tasks.workunit.client.1.smithi148.stdout:2/632: write d7/da/d33/d3c/d54/f93 [4529768,108842] 0 2024-03-29T08:07:21.704 INFO:tasks.workunit.client.1.smithi148.stdout:9/605: dread db/d22/d26/d54/d4d/d35/f48 [0,4194304] 0 2024-03-29T08:07:21.705 INFO:tasks.workunit.client.1.smithi148.stdout:8/611: unlink d17/d18/l58 0 2024-03-29T08:07:21.706 INFO:tasks.workunit.client.1.smithi148.stdout:3/607: rmdir de/d24/d33/d39/d46/d8d 39 2024-03-29T08:07:21.707 INFO:tasks.workunit.client.1.smithi148.stdout:9/606: write db/f5c [3211966,118800] 0 2024-03-29T08:07:21.708 INFO:tasks.workunit.client.1.smithi148.stdout:8/612: write d17/d18/f67 [1319974,87110] 0 2024-03-29T08:07:21.708 INFO:tasks.workunit.client.1.smithi148.stdout:7/547: mkdir d8/d29/d44/dc3 0 2024-03-29T08:07:21.709 INFO:tasks.workunit.client.1.smithi148.stdout:2/633: fdatasync f5 0 2024-03-29T08:07:21.713 INFO:tasks.workunit.client.1.smithi148.stdout:8/613: readlink d17/d49/d9a/l9d 0 2024-03-29T08:07:21.716 INFO:tasks.workunit.client.0.smithi037.stdout:3/409: creat dd/d15/d18/d4b/f8a x:0 0 0 2024-03-29T08:07:21.716 INFO:tasks.workunit.client.0.smithi037.stdout:2/327: symlink d0/d20/d2e/d38/l73 0 2024-03-29T08:07:21.716 INFO:tasks.workunit.client.0.smithi037.stdout:7/437: mknod dd/d21/c8d 0 2024-03-29T08:07:21.716 INFO:tasks.workunit.client.0.smithi037.stdout:1/490: link d2/d6/dc/d58/f59 d2/d4b/d80/fa8 0 2024-03-29T08:07:21.716 INFO:tasks.workunit.client.0.smithi037.stdout:9/651: chown d1/d4/d19/d1e/d4a/d6f/cb5 553377 1 2024-03-29T08:07:21.717 INFO:tasks.workunit.client.1.smithi148.stdout:7/548: creat d8/d38/d3c/da9/db7/fc4 x:0 0 0 2024-03-29T08:07:21.717 INFO:tasks.workunit.client.0.smithi037.stdout:4/445: write d3/d4/d15/d1d/db/f16 [5993355,25572] 0 2024-03-29T08:07:21.718 INFO:tasks.workunit.client.1.smithi148.stdout:9/607: creat db/d22/da3/fcd x:0 0 0 2024-03-29T08:07:21.719 INFO:tasks.workunit.client.0.smithi037.stdout:9/652: stat d1/d4/d19/d1e/d4a/d6f/f91 0 2024-03-29T08:07:21.719 INFO:tasks.workunit.client.1.smithi148.stdout:3/608: symlink de/d24/d33/d39/d46/d30/d37/lc5 0 2024-03-29T08:07:21.733 INFO:tasks.workunit.client.1.smithi148.stdout:9/608: mknod db/d20/cce 0 2024-03-29T08:07:21.733 INFO:tasks.workunit.client.0.smithi037.stdout:0/320: truncate d9/da/d1b/d22/d2c/f57 3533830 0 2024-03-29T08:07:21.734 INFO:tasks.workunit.client.1.smithi148.stdout:6/588: dread d6/d9/d1e/f53 [0,4194304] 0 2024-03-29T08:07:21.734 INFO:tasks.workunit.client.1.smithi148.stdout:3/609: symlink de/d24/d33/d39/d46/d30/lc6 0 2024-03-29T08:07:21.735 INFO:tasks.workunit.client.0.smithi037.stdout:3/410: creat dd/d46/d2d/d3a/d6c/f8b x:0 0 0 2024-03-29T08:07:21.736 INFO:tasks.workunit.client.0.smithi037.stdout:0/321: stat d9/da/d1b/d33 0 2024-03-29T08:07:21.736 INFO:tasks.workunit.client.0.smithi037.stdout:3/411: chown dd 14935 1 2024-03-29T08:07:21.736 INFO:tasks.workunit.client.1.smithi148.stdout:2/634: creat d7/da/d4c/d9a/dad/fc7 x:0 0 0 2024-03-29T08:07:21.737 INFO:tasks.workunit.client.1.smithi148.stdout:7/549: fsync d8/d14/f55 0 2024-03-29T08:07:21.737 INFO:tasks.workunit.client.1.smithi148.stdout:3/610: read - de/d24/d33/d39/d46/d30/d37/fba zero size 2024-03-29T08:07:21.739 INFO:tasks.workunit.client.1.smithi148.stdout:9/609: symlink db/d22/d26/d54/d4d/dbb/lcf 0 2024-03-29T08:07:21.739 INFO:tasks.workunit.client.1.smithi148.stdout:1/554: write d3/d18/d4d/d9e/f40 [1164767,21264] 0 2024-03-29T08:07:21.740 INFO:tasks.workunit.client.1.smithi148.stdout:6/589: mknod d6/d9/d1e/d27/cbd 0 2024-03-29T08:07:21.740 INFO:tasks.workunit.client.0.smithi037.stdout:7/438: creat dd/d21/d35/d42/f8e x:0 0 0 2024-03-29T08:07:21.741 INFO:tasks.workunit.client.1.smithi148.stdout:2/635: dread - d7/da/d4c/d9a/dad/fc7 zero size 2024-03-29T08:07:21.742 INFO:tasks.workunit.client.0.smithi037.stdout:7/439: chown dd/f16 22 1 2024-03-29T08:07:21.743 INFO:tasks.workunit.client.1.smithi148.stdout:2/636: readlink d7/da/l3a 0 2024-03-29T08:07:21.743 INFO:tasks.workunit.client.1.smithi148.stdout:3/611: mknod de/d24/d33/d39/cc7 0 2024-03-29T08:07:21.744 INFO:tasks.workunit.client.0.smithi037.stdout:2/328: creat d0/d20/d2e/d5f/d6f/d5e/f74 x:0 0 0 2024-03-29T08:07:21.746 INFO:tasks.workunit.client.1.smithi148.stdout:1/555: mkdir d3/d18/da8 0 2024-03-29T08:07:21.747 INFO:tasks.workunit.client.1.smithi148.stdout:8/614: creat d17/d49/d9a/dbe/fcb x:0 0 0 2024-03-29T08:07:21.747 INFO:tasks.workunit.client.1.smithi148.stdout:6/590: write d6/f70 [3028862,74009] 0 2024-03-29T08:07:21.750 INFO:tasks.workunit.client.1.smithi148.stdout:3/612: read de/d24/d33/d39/d46/d30/d37/fae [3284013,36215] 0 2024-03-29T08:07:21.750 INFO:tasks.workunit.client.0.smithi037.stdout:2/329: dread - d0/d20/d2e/d5f/f63 zero size 2024-03-29T08:07:21.751 INFO:tasks.workunit.client.1.smithi148.stdout:1/556: rmdir d3/d55/d7e 39 2024-03-29T08:07:21.752 INFO:tasks.workunit.client.1.smithi148.stdout:0/579: write db/dc/daa/f49 [699506,36698] 0 2024-03-29T08:07:21.755 INFO:tasks.workunit.client.1.smithi148.stdout:9/610: link db/d10/d6d/d73/da8/cb1 db/d22/d26/d54/d4d/dbb/dc0/cd0 0 2024-03-29T08:07:21.756 INFO:tasks.workunit.client.0.smithi037.stdout:9/653: mkdir d1/d4/d19/d1e/d4a/d6f/da2/d31/daa/dba/de3 0 2024-03-29T08:07:21.758 INFO:tasks.workunit.client.1.smithi148.stdout:6/591: unlink d6/d69/c8b 0 2024-03-29T08:07:21.760 INFO:tasks.workunit.client.1.smithi148.stdout:9/611: chown db/d22/d26/d54/d52/d9c 18029252 1 2024-03-29T08:07:21.761 INFO:tasks.workunit.client.1.smithi148.stdout:8/615: creat d17/d49/d9a/dbe/d6f/fcc x:0 0 0 2024-03-29T08:07:21.762 INFO:tasks.workunit.client.0.smithi037.stdout:0/322: mknod d9/da/dd/d13/c63 0 2024-03-29T08:07:21.763 INFO:tasks.workunit.client.1.smithi148.stdout:6/592: fsync d6/d9/d1e/d27/f59 0 2024-03-29T08:07:21.763 INFO:tasks.workunit.client.0.smithi037.stdout:4/446: dread d3/d4/f2f [0,4194304] 0 2024-03-29T08:07:21.764 INFO:tasks.workunit.client.1.smithi148.stdout:7/550: dwrite d8/d38/d3c/da9/db7/fbf [0,4194304] 0 2024-03-29T08:07:21.766 INFO:tasks.workunit.client.1.smithi148.stdout:6/593: chown d6/d69/c6c 13753 1 2024-03-29T08:07:21.766 INFO:tasks.workunit.client.1.smithi148.stdout:2/637: dwrite d7/d24/d4b/fc2 [0,4194304] 0 2024-03-29T08:07:21.768 INFO:tasks.workunit.client.1.smithi148.stdout:0/580: unlink f9 0 2024-03-29T08:07:21.769 INFO:tasks.workunit.client.1.smithi148.stdout:1/557: truncate d3/de/d25/d35/f48 1118011 0 2024-03-29T08:07:21.774 INFO:tasks.workunit.client.1.smithi148.stdout:8/616: mknod d17/d18/d26/ccd 0 2024-03-29T08:07:21.774 INFO:tasks.workunit.client.0.smithi037.stdout:3/412: fdatasync dd/d46/d2d/d3a/f55 0 2024-03-29T08:07:21.775 INFO:tasks.workunit.client.0.smithi037.stdout:4/447: stat d3/d4/c70 0 2024-03-29T08:07:21.778 INFO:tasks.workunit.client.1.smithi148.stdout:2/638: mkdir d7/d8c/da4/dc8 0 2024-03-29T08:07:21.781 INFO:tasks.workunit.client.0.smithi037.stdout:4/448: chown d3/d4/d15/d1d/db/d45/d1e/d59/d7d 208667583 1 2024-03-29T08:07:21.783 INFO:tasks.workunit.client.1.smithi148.stdout:6/594: rename d6/d9/d1e/d27/f7f to d6/d9/d1e/d27/d48/d73/d7e/fbe 0 2024-03-29T08:07:21.786 INFO:tasks.workunit.client.1.smithi148.stdout:1/558: fdatasync d3/de/d25/f75 0 2024-03-29T08:07:21.787 INFO:tasks.workunit.client.0.smithi037.stdout:1/491: creat d2/d6/dc/d18/d1d/fa9 x:0 0 0 2024-03-29T08:07:21.788 INFO:tasks.workunit.client.1.smithi148.stdout:2/639: truncate d7/d24/d4b/f87 1707618 0 2024-03-29T08:07:21.788 INFO:tasks.workunit.client.1.smithi148.stdout:1/559: dread - d3/de/d21/d5f/fa3 zero size 2024-03-29T08:07:21.788 INFO:tasks.workunit.client.0.smithi037.stdout:4/449: write d3/d4/d15/d1d/db/d3d/f56 [3180487,99186] 0 2024-03-29T08:07:21.788 INFO:tasks.workunit.client.0.smithi037.stdout:8/455: fsync d5/f32 0 2024-03-29T08:07:21.789 INFO:tasks.workunit.client.1.smithi148.stdout:1/560: fdatasync d3/de/d21/f80 0 2024-03-29T08:07:21.789 INFO:tasks.workunit.client.1.smithi148.stdout:8/617: mkdir d17/dce 0 2024-03-29T08:07:21.791 INFO:tasks.workunit.client.1.smithi148.stdout:6/595: creat d6/d9/d1e/d36/d4d/fbf x:0 0 0 2024-03-29T08:07:21.793 INFO:tasks.workunit.client.0.smithi037.stdout:9/654: mkdir d1/d4/d95/de4 0 2024-03-29T08:07:21.793 INFO:tasks.workunit.client.1.smithi148.stdout:6/596: write d6/f70 [1554359,82899] 0 2024-03-29T08:07:21.794 INFO:tasks.workunit.client.0.smithi037.stdout:8/456: chown d5/d10/d2d/d34/d50/d83/d46/d4e/f6a 800 1 2024-03-29T08:07:21.795 INFO:tasks.workunit.client.0.smithi037.stdout:0/323: rename d9/da/dd/d13/l35 to d9/da/dd/d20/l64 0 2024-03-29T08:07:21.797 INFO:tasks.workunit.client.1.smithi148.stdout:8/618: dread - d17/d49/f6d zero size 2024-03-29T08:07:21.797 INFO:tasks.workunit.client.1.smithi148.stdout:1/561: rename d3/f30 to d3/d18/da8/fa9 0 2024-03-29T08:07:21.801 INFO:tasks.workunit.client.1.smithi148.stdout:6/597: dread d6/d9/f23 [0,4194304] 0 2024-03-29T08:07:21.801 INFO:tasks.workunit.client.1.smithi148.stdout:1/562: truncate d3/de/d21/d5e/fa6 605719 0 2024-03-29T08:07:21.805 INFO:tasks.workunit.client.1.smithi148.stdout:0/581: write db/dc/daa/d47/daf/fb7 [467483,4538] 0 2024-03-29T08:07:21.805 INFO:tasks.workunit.client.0.smithi037.stdout:3/413: creat dd/d15/d27/d33/d75/f8c x:0 0 0 2024-03-29T08:07:21.807 INFO:tasks.workunit.client.0.smithi037.stdout:7/440: creat dd/d21/d35/f8f x:0 0 0 2024-03-29T08:07:21.807 INFO:tasks.workunit.client.1.smithi148.stdout:1/563: chown d3/de/d25/d51 2417 1 2024-03-29T08:07:21.808 INFO:tasks.workunit.client.1.smithi148.stdout:8/619: symlink d17/d74/d86/lcf 0 2024-03-29T08:07:21.809 INFO:tasks.workunit.client.1.smithi148.stdout:3/613: write de/d24/d33/d39/d46/f65 [195945,130416] 0 2024-03-29T08:07:21.810 INFO:tasks.workunit.client.1.smithi148.stdout:6/598: mkdir d6/d9/d1e/d36/d4d/d9d/d28/dba/dc0 0 2024-03-29T08:07:21.811 INFO:tasks.workunit.client.1.smithi148.stdout:1/564: write d3/de/f86 [146251,58282] 0 2024-03-29T08:07:21.813 INFO:tasks.workunit.client.0.smithi037.stdout:4/450: rename d3/d4/d27/f4b to d3/d4/d15/d89/f94 0 2024-03-29T08:07:21.815 INFO:tasks.workunit.client.1.smithi148.stdout:9/612: dread db/d10/d2d/d4e/f81 [0,4194304] 0 2024-03-29T08:07:21.817 INFO:tasks.workunit.client.0.smithi037.stdout:9/655: truncate d1/d4/d19/d1e/d4a/d6f/da2/d13/f17 2746125 0 2024-03-29T08:07:21.819 INFO:tasks.workunit.client.1.smithi148.stdout:8/620: mkdir d17/d18/d26/d63/d66/d5f/dd0 0 2024-03-29T08:07:21.821 INFO:tasks.workunit.client.0.smithi037.stdout:3/414: creat dd/d15/d27/d33/d75/f8d x:0 0 0 2024-03-29T08:07:21.822 INFO:tasks.workunit.client.1.smithi148.stdout:8/621: fdatasync d17/d18/d26/d63/d66/d5f/fa8 0 2024-03-29T08:07:21.822 INFO:tasks.workunit.client.0.smithi037.stdout:7/441: rmdir dd/d21/d65 39 2024-03-29T08:07:21.823 INFO:tasks.workunit.client.1.smithi148.stdout:0/582: dread db/dc/daa/d47/daf/f99 [0,4194304] 0 2024-03-29T08:07:21.824 INFO:tasks.workunit.client.0.smithi037.stdout:8/457: dwrite d5/d10/d2d/d34/d50/d83/d46/d4e/f6a [0,4194304] 0 2024-03-29T08:07:21.825 INFO:tasks.workunit.client.0.smithi037.stdout:1/492: dread d2/d6/dc/d58/d67/fa3 [0,4194304] 0 2024-03-29T08:07:21.829 INFO:tasks.workunit.client.1.smithi148.stdout:2/640: write d7/f5a [853527,15002] 0 2024-03-29T08:07:21.830 INFO:tasks.workunit.client.0.smithi037.stdout:0/324: unlink d9/da/dd/d13/f4b 0 2024-03-29T08:07:21.830 INFO:tasks.workunit.client.1.smithi148.stdout:1/565: getdents d3/de/d6e 0 2024-03-29T08:07:21.832 INFO:tasks.workunit.client.1.smithi148.stdout:2/641: chown d7/da/d33 867797453 1 2024-03-29T08:07:21.833 INFO:tasks.workunit.client.1.smithi148.stdout:3/614: dwrite de/d24/d33/f94 [0,4194304] 0 2024-03-29T08:07:21.834 INFO:tasks.workunit.client.1.smithi148.stdout:1/566: fdatasync d3/de/d21/d5e/fa6 0 2024-03-29T08:07:21.837 INFO:tasks.workunit.client.1.smithi148.stdout:8/622: symlink d17/d49/d9a/dbe/d6f/d64/ld1 0 2024-03-29T08:07:21.838 INFO:tasks.workunit.client.1.smithi148.stdout:9/613: dwrite db/d20/f1b [0,4194304] 0 2024-03-29T08:07:21.840 INFO:tasks.workunit.client.0.smithi037.stdout:8/458: chown d5/d7a 13090 1 2024-03-29T08:07:21.841 INFO:tasks.workunit.client.1.smithi148.stdout:1/567: write d3/d18/d71/f8e [922852,34311] 0 2024-03-29T08:07:21.842 INFO:tasks.workunit.client.1.smithi148.stdout:3/615: dread - de/d24/d33/d39/da3/dbc/fbe zero size 2024-03-29T08:07:21.844 INFO:tasks.workunit.client.1.smithi148.stdout:2/642: symlink d7/da/lc9 0 2024-03-29T08:07:21.846 INFO:tasks.workunit.client.1.smithi148.stdout:6/599: dread d6/d9/d1e/d27/f32 [0,4194304] 0 2024-03-29T08:07:21.848 INFO:tasks.workunit.client.0.smithi037.stdout:9/656: symlink d1/d4/d19/d1e/d4a/d6f/da2/d13/le5 0 2024-03-29T08:07:21.850 INFO:tasks.workunit.client.0.smithi037.stdout:7/442: dwrite dd/d21/d35/d42/f43 [0,4194304] 0 2024-03-29T08:07:21.850 INFO:tasks.workunit.client.1.smithi148.stdout:6/600: chown d6/d9/d1e/d20/d30/f47 504 1 2024-03-29T08:07:21.853 INFO:tasks.workunit.client.0.smithi037.stdout:3/415: mkdir dd/d15/d18/d8e 0 2024-03-29T08:07:21.854 INFO:tasks.workunit.client.1.smithi148.stdout:9/614: mkdir db/d10/d2d/d4e/da6/dd1 0 2024-03-29T08:07:21.855 INFO:tasks.workunit.client.1.smithi148.stdout:3/616: rmdir de/d24/d33/d39/d71 39 2024-03-29T08:07:21.855 INFO:tasks.workunit.client.1.smithi148.stdout:8/623: mknod d17/d18/d26/cd2 0 2024-03-29T08:07:21.856 INFO:tasks.workunit.client.0.smithi037.stdout:1/493: rename d2/l42 to d2/d6/dc/d5d/d62/laa 0 2024-03-29T08:07:21.860 INFO:tasks.workunit.client.1.smithi148.stdout:9/615: readlink db/d22/d26/d54/d4d/dbb/dc0/lc7 0 2024-03-29T08:07:21.861 INFO:tasks.workunit.client.1.smithi148.stdout:6/601: symlink d6/d9/d1e/d20/d30/lc1 0 2024-03-29T08:07:21.864 INFO:tasks.workunit.client.0.smithi037.stdout:3/416: read fb [150799,105703] 0 2024-03-29T08:07:21.865 INFO:tasks.workunit.client.1.smithi148.stdout:8/624: read d17/d18/d26/d4f/d94/f9b [4813986,105898] 0 2024-03-29T08:07:21.865 INFO:tasks.workunit.client.1.smithi148.stdout:6/602: read - d6/d9/d1e/d36/db1/faf zero size 2024-03-29T08:07:21.869 INFO:tasks.workunit.client.0.smithi037.stdout:4/451: getdents d3/d57 0 2024-03-29T08:07:21.869 INFO:tasks.workunit.client.1.smithi148.stdout:6/603: chown d6/d9/d1e/d36/d4d/lab 56 1 2024-03-29T08:07:21.871 INFO:tasks.workunit.client.0.smithi037.stdout:7/443: truncate dd/d21/f36 703654 0 2024-03-29T08:07:21.872 INFO:tasks.workunit.client.0.smithi037.stdout:0/325: getdents d9/da/dd 0 2024-03-29T08:07:21.874 INFO:tasks.workunit.client.0.smithi037.stdout:1/494: unlink d2/d6/dc/d18/d1d/d21/d99/f50 0 2024-03-29T08:07:21.876 INFO:tasks.workunit.client.1.smithi148.stdout:8/625: mknod d17/d1d/d7e/cd3 0 2024-03-29T08:07:21.876 INFO:tasks.workunit.client.1.smithi148.stdout:9/616: rename db/d22/d26/d54/d4d/f41 to db/d10/d6d/fd2 0 2024-03-29T08:07:21.880 INFO:tasks.workunit.client.1.smithi148.stdout:0/583: dread db/de/df/d11/d1f/f9b [0,4194304] 0 2024-03-29T08:07:21.882 INFO:tasks.workunit.client.1.smithi148.stdout:3/617: dread de/d24/d33/d39/d46/d30/fa2 [0,4194304] 0 2024-03-29T08:07:21.883 INFO:tasks.workunit.client.1.smithi148.stdout:6/604: dread d6/d9/d1e/f3d [0,4194304] 0 2024-03-29T08:07:21.886 INFO:tasks.workunit.client.0.smithi037.stdout:7/444: dread dd/d21/d35/f6b [0,4194304] 0 2024-03-29T08:07:21.886 INFO:tasks.workunit.client.1.smithi148.stdout:3/618: dread de/f15 [0,4194304] 0 2024-03-29T08:07:21.889 INFO:tasks.workunit.client.0.smithi037.stdout:9/657: dwrite d1/d4/f3b [0,4194304] 0 2024-03-29T08:07:21.892 INFO:tasks.workunit.client.1.smithi148.stdout:2/643: truncate d7/da/d33/d45/d4d/d2e/d37/f43 4531379 0 2024-03-29T08:07:21.894 INFO:tasks.workunit.client.0.smithi037.stdout:0/326: mknod d9/da/dd/d20/d55/c65 0 2024-03-29T08:07:21.895 INFO:tasks.workunit.client.0.smithi037.stdout:8/459: dread d5/f1f [0,4194304] 0 2024-03-29T08:07:21.897 INFO:tasks.workunit.client.1.smithi148.stdout:9/617: symlink db/d10/d2d/d4e/d5f/d6e/db0/ld3 0 2024-03-29T08:07:21.899 INFO:tasks.workunit.client.1.smithi148.stdout:6/605: write d6/d9/d1e/d27/d48/d73/d7e/fbe [5300774,86239] 0 2024-03-29T08:07:21.901 INFO:tasks.workunit.client.1.smithi148.stdout:3/619: truncate de/d24/d33/d39/d46/d30/d37/faa 748817 0 2024-03-29T08:07:21.901 INFO:tasks.workunit.client.1.smithi148.stdout:0/584: readlink db/dc/l1c 0 2024-03-29T08:07:21.902 INFO:tasks.workunit.client.1.smithi148.stdout:2/644: creat d7/da/d33/d3c/d54/d70/fca x:0 0 0 2024-03-29T08:07:21.903 INFO:tasks.workunit.client.0.smithi037.stdout:7/445: mknod dd/d21/d35/d42/c90 0 2024-03-29T08:07:21.907 INFO:tasks.workunit.client.0.smithi037.stdout:7/446: write dd/d21/d35/d42/f8e [887189,86204] 0 2024-03-29T08:07:21.908 INFO:tasks.workunit.client.0.smithi037.stdout:4/452: mkdir d3/d4/d15/d1d/db/d45/d1e/d59/d79/d95 0 2024-03-29T08:07:21.908 INFO:tasks.workunit.client.1.smithi148.stdout:9/618: rename db/d20 to db/d10/d2d/d4e/dd4 0 2024-03-29T08:07:21.912 INFO:tasks.workunit.client.0.smithi037.stdout:3/417: write dd/d46/f2c [801691,70162] 0 2024-03-29T08:07:21.912 INFO:tasks.workunit.client.1.smithi148.stdout:3/620: unlink de/d3b/f93 0 2024-03-29T08:07:21.913 INFO:tasks.workunit.client.1.smithi148.stdout:2/645: dread d7/da/d4c/f79 [0,4194304] 0 2024-03-29T08:07:21.915 INFO:tasks.workunit.client.1.smithi148.stdout:3/621: fsync de/d24/d33/d39/d71/fb9 0 2024-03-29T08:07:21.916 INFO:tasks.workunit.client.0.smithi037.stdout:7/447: mknod dd/d1d/d49/c91 0 2024-03-29T08:07:21.916 INFO:tasks.workunit.client.1.smithi148.stdout:9/619: mkdir db/d10/d2d/d4e/d5f/dd5 0 2024-03-29T08:07:21.917 INFO:tasks.workunit.client.1.smithi148.stdout:3/622: dread - de/d24/d33/d6b/fa6 zero size 2024-03-29T08:07:21.917 INFO:tasks.workunit.client.1.smithi148.stdout:2/646: creat d7/da/d33/d45/d4d/d41/fcb x:0 0 0 2024-03-29T08:07:21.918 INFO:tasks.workunit.client.1.smithi148.stdout:9/620: readlink db/d10/d2d/l6f 0 2024-03-29T08:07:21.918 INFO:tasks.workunit.client.0.smithi037.stdout:9/658: dwrite d1/d4/d19/d1e/d4a/d6f/f83 [0,4194304] 0 2024-03-29T08:07:21.920 INFO:tasks.workunit.client.0.smithi037.stdout:8/460: truncate d5/f80 330133 0 2024-03-29T08:07:21.920 INFO:tasks.workunit.client.0.smithi037.stdout:7/448: chown dd/d1d/d49 4 1 2024-03-29T08:07:21.924 INFO:tasks.workunit.client.1.smithi148.stdout:0/585: dwrite db/de/df/f30 [0,4194304] 0 2024-03-29T08:07:21.926 INFO:tasks.workunit.client.1.smithi148.stdout:2/647: mkdir d7/da/dcc 0 2024-03-29T08:07:21.927 INFO:tasks.workunit.client.0.smithi037.stdout:7/449: chown dd/d21/l80 13 1 2024-03-29T08:07:21.929 INFO:tasks.workunit.client.0.smithi037.stdout:3/418: mkdir dd/d15/d27/d33/d75/d8f 0 2024-03-29T08:07:21.930 INFO:tasks.workunit.client.0.smithi037.stdout:8/461: stat d5/d10/d2d/d34/l6d 0 2024-03-29T08:07:21.935 INFO:tasks.workunit.client.0.smithi037.stdout:0/327: dwrite d9/da/dd/d13/f5c [0,4194304] 0 2024-03-29T08:07:21.937 INFO:tasks.workunit.client.1.smithi148.stdout:3/623: link de/d24/d33/d39/d46/d30/d37/f7a de/d24/d33/d39/d46/d30/d37/d62/d63/dbf/fc8 0 2024-03-29T08:07:21.938 INFO:tasks.workunit.client.0.smithi037.stdout:1/495: dwrite d2/d40/f43 [0,4194304] 0 2024-03-29T08:07:21.938 INFO:tasks.workunit.client.0.smithi037.stdout:9/659: readlink d1/d4/d19/d1e/d4a/d6f/da2/d13/l2f 0 2024-03-29T08:07:21.944 INFO:tasks.workunit.client.1.smithi148.stdout:2/648: link d7/da/f89 d7/da/d4c/d9a/dad/fcd 0 2024-03-29T08:07:21.950 INFO:tasks.workunit.client.1.smithi148.stdout:2/649: creat d7/da/d33/d3c/da5/fce x:0 0 0 2024-03-29T08:07:21.952 INFO:tasks.workunit.client.1.smithi148.stdout:6/606: truncate d6/d9/f52 713712 0 2024-03-29T08:07:21.953 INFO:tasks.workunit.client.0.smithi037.stdout:3/419: creat dd/d15/d27/d33/f90 x:0 0 0 2024-03-29T08:07:21.955 INFO:tasks.workunit.client.1.smithi148.stdout:2/650: symlink d7/lcf 0 2024-03-29T08:07:21.955 INFO:tasks.workunit.client.0.smithi037.stdout:0/328: chown d9/da/f25 9 1 2024-03-29T08:07:21.958 INFO:tasks.workunit.client.0.smithi037.stdout:9/660: write d1/d4/d19/d1e/d4a/d6f/da2/d23/f72 [2996402,52014] 0 2024-03-29T08:07:21.958 INFO:tasks.workunit.client.0.smithi037.stdout:3/420: chown dd/d5e/f80 161 1 2024-03-29T08:07:21.959 INFO:tasks.workunit.client.1.smithi148.stdout:0/586: getdents db/de/df/d11/d2d/d3e 0 2024-03-29T08:07:21.961 INFO:tasks.workunit.client.0.smithi037.stdout:0/329: dread d9/da/dd/f5b [0,4194304] 0 2024-03-29T08:07:21.962 INFO:tasks.workunit.client.1.smithi148.stdout:2/651: mknod d7/d24/d4b/d83/cd0 0 2024-03-29T08:07:21.964 INFO:tasks.workunit.client.0.smithi037.stdout:8/462: creat d5/d10/d2d/d34/d50/d87/f94 x:0 0 0 2024-03-29T08:07:21.966 INFO:tasks.workunit.client.1.smithi148.stdout:6/607: truncate d6/d9/d1e/d36/d4d/d9d/f31 246118 0 2024-03-29T08:07:21.968 INFO:tasks.workunit.client.0.smithi037.stdout:3/421: truncate dd/d46/f71 5911698 0 2024-03-29T08:07:21.970 INFO:tasks.workunit.client.1.smithi148.stdout:6/608: write d6/d9/d1e/d36/d4d/d9d/f34 [7719230,73031] 0 2024-03-29T08:07:21.970 INFO:tasks.workunit.client.1.smithi148.stdout:0/587: mkdir db/de/df/d2b/db9 0 2024-03-29T08:07:21.971 INFO:tasks.workunit.client.1.smithi148.stdout:9/621: dwrite f3 [0,4194304] 0 2024-03-29T08:07:21.972 INFO:tasks.workunit.client.0.smithi037.stdout:0/330: creat d9/da/d1b/d22/d2c/f66 x:0 0 0 2024-03-29T08:07:21.973 INFO:tasks.workunit.client.1.smithi148.stdout:6/609: dread - d6/d9/d1e/d20/d30/fac zero size 2024-03-29T08:07:21.973 INFO:tasks.workunit.client.0.smithi037.stdout:4/453: dread d3/f21 [0,4194304] 0 2024-03-29T08:07:21.975 INFO:tasks.workunit.client.1.smithi148.stdout:6/610: readlink d6/d9/d1e/d36/d4d/d9d/l5f 0 2024-03-29T08:07:21.979 INFO:tasks.workunit.client.0.smithi037.stdout:9/661: rename d1/d1f/d3a/f78 to d1/d4/d19/d69/fe6 0 2024-03-29T08:07:21.981 INFO:tasks.workunit.client.1.smithi148.stdout:6/611: mkdir d6/d9/d1e/d36/d4d/d9d/d28/d98/dc2 0 2024-03-29T08:07:21.985 INFO:tasks.workunit.client.0.smithi037.stdout:3/422: mknod dd/d15/d27/c91 0 2024-03-29T08:07:21.986 INFO:tasks.workunit.client.1.smithi148.stdout:6/612: chown d6/d9/d1e/d36/c58 611454 1 2024-03-29T08:07:21.986 INFO:tasks.workunit.client.1.smithi148.stdout:0/588: rename db/dc/l1c to db/de/df/d2b/db9/lba 0 2024-03-29T08:07:21.987 INFO:tasks.workunit.client.1.smithi148.stdout:2/652: truncate d7/da/d33/d45/d4d/f7a 1258282 0 2024-03-29T08:07:21.989 INFO:tasks.workunit.client.1.smithi148.stdout:2/653: write d7/f2d [4181263,40572] 0 2024-03-29T08:07:21.990 INFO:tasks.workunit.client.0.smithi037.stdout:8/463: rename d5/d10/d19/d3d/d4a/c91 to d5/d10/d19/d3d/d4a/d4c/c95 0 2024-03-29T08:07:21.993 INFO:tasks.workunit.client.1.smithi148.stdout:2/654: mkdir d7/da/d4c/dd1 0 2024-03-29T08:07:21.995 INFO:tasks.workunit.client.1.smithi148.stdout:2/655: chown d7/da/d33/d45/d4d/d2e/d37 773 1 2024-03-29T08:07:21.996 INFO:tasks.workunit.client.1.smithi148.stdout:0/589: mknod db/dc/daa/d47/cbb 0 2024-03-29T08:07:21.996 INFO:tasks.workunit.client.0.smithi037.stdout:7/450: dread dd/d21/d65/f6f [0,4194304] 0 2024-03-29T08:07:22.001 INFO:tasks.workunit.client.0.smithi037.stdout:1/496: truncate d2/d6/dc/d18/d1d/d21/d99/f8c 60940 0 2024-03-29T08:07:22.003 INFO:tasks.workunit.client.0.smithi037.stdout:7/451: chown dd/d1d/d34/c3d 81 1 2024-03-29T08:07:22.005 INFO:tasks.workunit.client.0.smithi037.stdout:7/452: fdatasync dd/d68/f78 0 2024-03-29T08:07:22.006 INFO:tasks.workunit.client.1.smithi148.stdout:6/613: link c3 d6/d9/d1e/d20/d30/cc3 0 2024-03-29T08:07:22.010 INFO:tasks.workunit.client.1.smithi148.stdout:6/614: fdatasync d6/d9/d1e/f8a 0 2024-03-29T08:07:22.011 INFO:tasks.workunit.client.0.smithi037.stdout:4/454: mkdir d3/d4/d15/d1d/db/d45/d93/d96 0 2024-03-29T08:07:22.013 INFO:tasks.workunit.client.1.smithi148.stdout:6/615: write d6/d9/d1e/f8a [1460728,62200] 0 2024-03-29T08:07:22.015 INFO:tasks.workunit.client.0.smithi037.stdout:9/662: creat d1/d4/d19/fe7 x:0 0 0 2024-03-29T08:07:22.016 INFO:tasks.workunit.client.0.smithi037.stdout:0/331: link d9/da/dd/d59/l19 d9/da/dd/d13/l67 0 2024-03-29T08:07:22.018 INFO:tasks.workunit.client.0.smithi037.stdout:7/453: creat dd/d21/d70/f92 x:0 0 0 2024-03-29T08:07:22.034 INFO:tasks.workunit.client.0.smithi037.stdout:4/455: mkdir d3/d4/d15/d1d/db/d45/d1e/d59/d7d/d97 0 2024-03-29T08:07:22.037 INFO:tasks.workunit.client.0.smithi037.stdout:0/332: rename d9/da/l4a to d9/da/dd/d20/l68 0 2024-03-29T08:07:22.040 INFO:tasks.workunit.client.1.smithi148.stdout:9/622: dread db/d10/d2d/d4e/d5f/d6e/d7a/f7e [0,4194304] 0 2024-03-29T08:07:22.040 INFO:tasks.workunit.client.0.smithi037.stdout:4/456: read d3/d4/d15/d89/f94 [1980424,99523] 0 2024-03-29T08:07:22.041 INFO:tasks.workunit.client.1.smithi148.stdout:9/623: write db/f5c [3155330,48423] 0 2024-03-29T08:07:22.044 INFO:tasks.workunit.client.0.smithi037.stdout:4/457: write d3/d57/f7a [692876,39681] 0 2024-03-29T08:07:22.053 INFO:tasks.workunit.client.0.smithi037.stdout:3/423: dread dd/d46/d3e/f7c [0,4194304] 0 2024-03-29T08:07:22.066 INFO:tasks.workunit.client.1.smithi148.stdout:2/656: getdents d7/d24/d4b/d83 0 2024-03-29T08:07:22.066 INFO:tasks.workunit.client.0.smithi037.stdout:3/424: symlink dd/d15/d27/d33/d84/l92 0 2024-03-29T08:07:22.073 INFO:tasks.workunit.client.0.smithi037.stdout:3/425: dread - dd/d46/d2d/f60 zero size 2024-03-29T08:07:22.078 INFO:tasks.workunit.client.0.smithi037.stdout:3/426: dread - dd/d15/f49 zero size 2024-03-29T08:07:22.080 INFO:tasks.workunit.client.1.smithi148.stdout:9/624: link db/d10/d6d/fd2 db/d22/d26/d54/d52/dab/fd6 0 2024-03-29T08:07:22.090 INFO:tasks.workunit.client.0.smithi037.stdout:3/427: mknod dd/d15/d18/d8e/c93 0 2024-03-29T08:07:22.090 INFO:tasks.workunit.client.0.smithi037.stdout:7/454: write dd/d21/f36 [367437,36211] 0 2024-03-29T08:07:22.091 INFO:tasks.workunit.client.1.smithi148.stdout:9/625: creat db/d22/fd7 x:0 0 0 2024-03-29T08:07:22.092 INFO:tasks.workunit.client.0.smithi037.stdout:7/455: stat dd/f67 0 2024-03-29T08:07:22.097 INFO:tasks.workunit.client.0.smithi037.stdout:9/663: dwrite d1/d4/d19/d1e/d4a/d6f/f91 [0,4194304] 0 2024-03-29T08:07:22.104 INFO:tasks.workunit.client.1.smithi148.stdout:9/626: dwrite db/d22/f89 [0,4194304] 0 2024-03-29T08:07:22.109 INFO:tasks.workunit.client.0.smithi037.stdout:0/333: dwrite d9/da/f24 [0,4194304] 0 2024-03-29T08:07:22.115 INFO:tasks.workunit.client.1.smithi148.stdout:9/627: creat db/d10/d6d/d73/da8/fd8 x:0 0 0 2024-03-29T08:07:22.120 INFO:tasks.workunit.client.0.smithi037.stdout:0/334: readlink d9/da/dd/d20/l68 0 2024-03-29T08:07:22.123 INFO:tasks.workunit.client.1.smithi148.stdout:9/628: mkdir db/d10/d2d/d4e/da6/dd1/dd9 0 2024-03-29T08:07:22.128 INFO:tasks.workunit.client.0.smithi037.stdout:0/335: fsync d9/da/d1b/d22/d2c/f2d 0 2024-03-29T08:07:22.130 INFO:tasks.workunit.client.1.smithi148.stdout:2/657: dwrite d7/d8c/d91/f25 [4194304,4194304] 0 2024-03-29T08:07:22.132 INFO:tasks.workunit.client.0.smithi037.stdout:7/456: dwrite dd/d1d/d34/f39 [4194304,4194304] 0 2024-03-29T08:07:22.135 INFO:tasks.workunit.client.1.smithi148.stdout:6/616: rename d6/d9/d1e/d27/f59 to d6/d9/d1e/d27/d48/d73/d7e/fc4 0 2024-03-29T08:07:22.138 INFO:tasks.workunit.client.0.smithi037.stdout:3/428: dwrite dd/d46/d2d/d3a/d5f/f5d [0,4194304] 0 2024-03-29T08:07:22.140 INFO:tasks.workunit.client.1.smithi148.stdout:6/617: mkdir d6/d9/dc5 0 2024-03-29T08:07:22.141 INFO:tasks.workunit.client.0.smithi037.stdout:3/429: chown dd/d46/c2e 10 1 2024-03-29T08:07:22.141 INFO:tasks.workunit.client.1.smithi148.stdout:2/658: getdents d7/d8c/da4 0 2024-03-29T08:07:22.142 INFO:tasks.workunit.client.1.smithi148.stdout:2/659: fdatasync d7/fbf 0 2024-03-29T08:07:22.142 INFO:tasks.workunit.client.1.smithi148.stdout:9/629: dwrite db/d10/d2d/f68 [0,4194304] 0 2024-03-29T08:07:22.149 INFO:tasks.workunit.client.0.smithi037.stdout:0/336: unlink d9/da/d1b/d22/d2c/f66 0 2024-03-29T08:07:22.151 INFO:tasks.workunit.client.1.smithi148.stdout:2/660: symlink d7/da/d33/d45/d4d/d2e/d37/ld2 0 2024-03-29T08:07:22.155 INFO:tasks.workunit.client.1.smithi148.stdout:6/618: truncate d6/d9/d1e/d27/d48/d73/f83 2660544 0 2024-03-29T08:07:22.156 INFO:tasks.workunit.client.0.smithi037.stdout:7/457: rename dd/d21/c31 to dd/d1d/d34/d54/c93 0 2024-03-29T08:07:22.158 INFO:tasks.workunit.client.1.smithi148.stdout:2/661: chown d7/da/d33/d45/d4d/d2e/d37/f43 74260844 1 2024-03-29T08:07:22.162 INFO:tasks.workunit.client.1.smithi148.stdout:5/574: sync 2024-03-29T08:07:22.164 INFO:tasks.workunit.client.1.smithi148.stdout:2/662: creat d7/da/d33/fd3 x:0 0 0 2024-03-29T08:07:22.165 INFO:tasks.workunit.client.0.smithi037.stdout:7/458: mkdir dd/d21/d35/d60/d94 0 2024-03-29T08:07:22.165 INFO:tasks.workunit.client.0.smithi037.stdout:0/337: creat d9/da/dd/d20/d38/f69 x:0 0 0 2024-03-29T08:07:22.166 INFO:tasks.workunit.client.1.smithi148.stdout:5/575: chown d0/db/d84 984959 1 2024-03-29T08:07:22.168 INFO:tasks.workunit.client.1.smithi148.stdout:2/663: stat d7/d8c/d91/l84 0 2024-03-29T08:07:22.169 INFO:tasks.workunit.client.1.smithi148.stdout:2/664: chown d7/d8c/d91/l2f 73 1 2024-03-29T08:07:22.170 INFO:tasks.workunit.client.0.smithi037.stdout:0/338: write d9/da/dd/d13/f21 [5043610,2158] 0 2024-03-29T08:07:22.170 INFO:tasks.workunit.client.1.smithi148.stdout:2/665: dread - d7/d8c/da4/fb0 zero size 2024-03-29T08:07:22.171 INFO:tasks.workunit.client.0.smithi037.stdout:7/459: symlink dd/d21/d35/d3b/d5d/l95 0 2024-03-29T08:07:22.171 INFO:tasks.workunit.client.0.smithi037.stdout:9/664: dread d1/f74 [0,4194304] 0 2024-03-29T08:07:22.172 INFO:tasks.workunit.client.1.smithi148.stdout:2/666: chown d7/da/d33/d45/d4d/d41/f6a 134809 1 2024-03-29T08:07:22.173 INFO:tasks.workunit.client.0.smithi037.stdout:3/430: dread dd/d15/f44 [0,4194304] 0 2024-03-29T08:07:22.174 INFO:tasks.workunit.client.0.smithi037.stdout:7/460: dread - dd/d21/d35/d60/f7f zero size 2024-03-29T08:07:22.177 INFO:tasks.workunit.client.0.smithi037.stdout:0/339: creat d9/da/dd/d20/d38/d53/f6a x:0 0 0 2024-03-29T08:07:22.178 INFO:tasks.workunit.client.1.smithi148.stdout:5/576: dread d0/db/d7b/d16/d2e/f68 [0,4194304] 0 2024-03-29T08:07:22.179 INFO:tasks.workunit.client.0.smithi037.stdout:5/564: sync 2024-03-29T08:07:22.179 INFO:tasks.workunit.client.0.smithi037.stdout:6/376: sync 2024-03-29T08:07:22.179 INFO:tasks.workunit.client.1.smithi148.stdout:2/667: rmdir d7/d24/d4b/d68 0 2024-03-29T08:07:22.180 INFO:tasks.workunit.client.0.smithi037.stdout:7/461: write dd/f3e [529119,11802] 0 2024-03-29T08:07:22.181 INFO:tasks.workunit.client.0.smithi037.stdout:9/665: mknod d1/d1f/d3a/d9b/ddd/ce8 0 2024-03-29T08:07:22.181 INFO:tasks.workunit.client.1.smithi148.stdout:0/590: sync 2024-03-29T08:07:22.181 INFO:tasks.workunit.client.1.smithi148.stdout:4/599: sync 2024-03-29T08:07:22.181 INFO:tasks.workunit.client.1.smithi148.stdout:7/551: sync 2024-03-29T08:07:22.181 INFO:tasks.workunit.client.1.smithi148.stdout:8/626: sync 2024-03-29T08:07:22.181 INFO:tasks.workunit.client.1.smithi148.stdout:3/624: sync 2024-03-29T08:07:22.181 INFO:tasks.workunit.client.1.smithi148.stdout:1/568: sync 2024-03-29T08:07:22.184 INFO:tasks.workunit.client.0.smithi037.stdout:5/565: chown d8/d18/d24/f74 23925027 1 2024-03-29T08:07:22.185 INFO:tasks.workunit.client.1.smithi148.stdout:8/627: readlink d17/d1d/lc2 0 2024-03-29T08:07:22.185 INFO:tasks.workunit.client.1.smithi148.stdout:6/619: write d6/d9/d1e/fbb [2214066,7855] 0 2024-03-29T08:07:22.186 INFO:tasks.workunit.client.0.smithi037.stdout:7/462: write dd/f16 [3544700,101162] 0 2024-03-29T08:07:22.189 INFO:tasks.workunit.client.1.smithi148.stdout:2/668: mknod d7/da/cd4 0 2024-03-29T08:07:22.189 INFO:tasks.workunit.client.1.smithi148.stdout:1/569: stat d3/d55/d7e/l8d 0 2024-03-29T08:07:22.189 INFO:tasks.workunit.client.0.smithi037.stdout:6/377: creat d0/d5/d1e/f7f x:0 0 0 2024-03-29T08:07:22.191 INFO:tasks.workunit.client.1.smithi148.stdout:7/552: creat d8/d14/da8/fc5 x:0 0 0 2024-03-29T08:07:22.191 INFO:tasks.workunit.client.1.smithi148.stdout:0/591: rmdir db/dc/daa/d47/daf 39 2024-03-29T08:07:22.193 INFO:tasks.workunit.client.1.smithi148.stdout:9/630: dwrite db/d22/d26/d54/f36 [0,4194304] 0 2024-03-29T08:07:22.194 INFO:tasks.workunit.client.0.smithi037.stdout:5/566: fsync d8/f4c 0 2024-03-29T08:07:22.194 INFO:tasks.workunit.client.1.smithi148.stdout:8/628: fsync fe 0 2024-03-29T08:07:22.196 INFO:tasks.workunit.client.0.smithi037.stdout:9/666: rename d1/d4/d19/d1e/d25/d16/f56 to d1/d4/d19/d1e/d71/d89/da6/dd8/fe9 0 2024-03-29T08:07:22.196 INFO:tasks.workunit.client.1.smithi148.stdout:7/553: chown d8/d14/da8 32091 1 2024-03-29T08:07:22.198 INFO:tasks.workunit.client.1.smithi148.stdout:2/669: mkdir d7/d8c/d91/dd5 0 2024-03-29T08:07:22.198 INFO:tasks.workunit.client.0.smithi037.stdout:7/463: symlink dd/d21/d35/d3b/d5d/l96 0 2024-03-29T08:07:22.199 INFO:tasks.workunit.client.1.smithi148.stdout:8/629: readlink d17/d18/lc1 0 2024-03-29T08:07:22.200 INFO:tasks.workunit.client.0.smithi037.stdout:3/431: dread dd/d46/d2d/d3a/d5f/f5d [0,4194304] 0 2024-03-29T08:07:22.203 INFO:tasks.workunit.client.1.smithi148.stdout:6/620: creat d6/d9/d1e/d20/d30/fc6 x:0 0 0 2024-03-29T08:07:22.205 INFO:tasks.workunit.client.1.smithi148.stdout:1/570: fsync d3/de/f1f 0 2024-03-29T08:07:22.208 INFO:tasks.workunit.client.0.smithi037.stdout:6/378: rename d0/d5/d1f/l47 to d0/d19/d42/d60/d62/l80 0 2024-03-29T08:07:22.209 INFO:tasks.workunit.client.1.smithi148.stdout:5/577: dwrite d0/f4b [0,4194304] 0 2024-03-29T08:07:22.211 INFO:tasks.workunit.client.1.smithi148.stdout:2/670: creat d7/d8c/da4/d8e/fd6 x:0 0 0 2024-03-29T08:07:22.211 INFO:tasks.workunit.client.0.smithi037.stdout:6/379: dread - d0/d19/d42/d60/f7d zero size 2024-03-29T08:07:22.214 INFO:tasks.workunit.client.1.smithi148.stdout:6/621: mkdir d6/d9/dc7 0 2024-03-29T08:07:22.216 INFO:tasks.workunit.client.0.smithi037.stdout:0/340: dwrite d9/da/d1b/d22/d2c/f42 [0,4194304] 0 2024-03-29T08:07:22.216 INFO:tasks.workunit.client.0.smithi037.stdout:9/667: unlink d1/d4/d19/d69/d75/l99 0 2024-03-29T08:07:22.217 INFO:tasks.workunit.client.1.smithi148.stdout:2/671: dread - d7/d24/fa8 zero size 2024-03-29T08:07:22.217 INFO:tasks.workunit.client.1.smithi148.stdout:1/571: chown d3/d55/c70 299721767 1 2024-03-29T08:07:22.218 INFO:tasks.workunit.client.1.smithi148.stdout:9/631: truncate db/d22/d26/d54/d4d/d35/d4b/d5a/d74/fba 639525 0 2024-03-29T08:07:22.219 INFO:tasks.workunit.client.1.smithi148.stdout:0/592: mknod db/de/df/d11/d2d/da3/cbc 0 2024-03-29T08:07:22.219 INFO:tasks.workunit.client.0.smithi037.stdout:9/668: chown d1/d4/d19/d1e/d71/d89/da6 356764 1 2024-03-29T08:07:22.220 INFO:tasks.workunit.client.1.smithi148.stdout:4/600: dwrite df/d2c/d51/d77/d26/d34/d8a/d98/fb0 [4194304,4194304] 0 2024-03-29T08:07:22.220 INFO:tasks.workunit.client.0.smithi037.stdout:0/341: write d9/da/d1b/d22/f47 [1013203,89309] 0 2024-03-29T08:07:22.222 INFO:tasks.workunit.client.1.smithi148.stdout:5/578: symlink d0/db/d1e/lbb 0 2024-03-29T08:07:22.222 INFO:tasks.workunit.client.0.smithi037.stdout:0/342: fsync d9/da/dd/d54/f56 0 2024-03-29T08:07:22.222 INFO:tasks.workunit.client.1.smithi148.stdout:3/625: dwrite de/d24/d33/f8e [0,4194304] 0 2024-03-29T08:07:22.223 INFO:tasks.workunit.client.1.smithi148.stdout:0/593: chown db/dc 36 1 2024-03-29T08:07:22.224 INFO:tasks.workunit.client.0.smithi037.stdout:0/343: truncate d9/da/f24 4605686 0 2024-03-29T08:07:22.224 INFO:tasks.workunit.client.0.smithi037.stdout:9/669: chown d1/d4/d19/d1e/d25/c58 46237 1 2024-03-29T08:07:22.229 INFO:tasks.workunit.client.1.smithi148.stdout:5/579: chown d0/db/d7b/f52 51528665 1 2024-03-29T08:07:22.230 INFO:tasks.workunit.client.1.smithi148.stdout:7/554: creat d8/d38/d3c/da5/db4/fc6 x:0 0 0 2024-03-29T08:07:22.230 INFO:tasks.workunit.client.1.smithi148.stdout:2/672: write d7/da/d33/d45/d4d/d41/f6a [268969,32270] 0 2024-03-29T08:07:22.230 INFO:tasks.workunit.client.0.smithi037.stdout:9/670: read d1/d4/d19/d1e/d4a/d6f/da2/d31/f48 [14355,8799] 0 2024-03-29T08:07:22.232 INFO:tasks.workunit.client.1.smithi148.stdout:9/632: chown db/d10/d2d/d4e/f81 1605452 1 2024-03-29T08:07:22.232 INFO:tasks.workunit.client.0.smithi037.stdout:9/671: read - d1/fb6 zero size 2024-03-29T08:07:22.234 INFO:tasks.workunit.client.1.smithi148.stdout:5/580: readlink d0/db/d7b/l18 0 2024-03-29T08:07:22.235 INFO:tasks.workunit.client.0.smithi037.stdout:9/672: read - d1/d4/d19/d1e/d71/d89/dbe/fd7 zero size 2024-03-29T08:07:22.235 INFO:tasks.workunit.client.1.smithi148.stdout:2/673: fdatasync d7/da/d33/fd3 0 2024-03-29T08:07:22.236 INFO:tasks.workunit.client.1.smithi148.stdout:6/622: creat d6/d9/d1e/d36/d4d/d9d/d28/d98/dc2/fc8 x:0 0 0 2024-03-29T08:07:22.237 INFO:tasks.workunit.client.0.smithi037.stdout:3/432: creat dd/d46/d2d/d3a/d6c/d6f/f94 x:0 0 0 2024-03-29T08:07:22.240 INFO:tasks.workunit.client.1.smithi148.stdout:4/601: mknod df/d2c/d51/d77/d26/cbf 0 2024-03-29T08:07:22.241 INFO:tasks.workunit.client.0.smithi037.stdout:0/344: dread d9/da/d1b/d22/d2c/f42 [0,4194304] 0 2024-03-29T08:07:22.241 INFO:tasks.workunit.client.0.smithi037.stdout:5/567: rename d8/d34/l58 to d8/d34/d4b/lbd 0 2024-03-29T08:07:22.243 INFO:tasks.workunit.client.1.smithi148.stdout:8/630: link d17/d49/d9a/dbe/d6f/d64/ld1 d17/d18/dbd/ld4 0 2024-03-29T08:07:22.244 INFO:tasks.workunit.client.1.smithi148.stdout:7/555: creat d8/d14/fc7 x:0 0 0 2024-03-29T08:07:22.245 INFO:tasks.workunit.client.1.smithi148.stdout:2/674: chown d7/d24/d4b/d66/d7d 10267645 1 2024-03-29T08:07:22.245 INFO:tasks.workunit.client.0.smithi037.stdout:0/345: readlink d9/da/dd/l58 0 2024-03-29T08:07:22.246 INFO:tasks.workunit.client.1.smithi148.stdout:5/581: mknod d0/db/d1e/d35/d42/d67/cbc 0 2024-03-29T08:07:22.247 INFO:tasks.workunit.client.0.smithi037.stdout:9/673: creat d1/d4/fea x:0 0 0 2024-03-29T08:07:22.247 INFO:tasks.workunit.client.1.smithi148.stdout:4/602: write df/d2c/f86 [1468836,91328] 0 2024-03-29T08:07:22.248 INFO:tasks.workunit.client.1.smithi148.stdout:1/572: dread d3/de/d1d/f33 [0,4194304] 0 2024-03-29T08:07:22.250 INFO:tasks.workunit.client.0.smithi037.stdout:3/433: rename dd/d46/d2d/d3a/f68 to dd/d46/d3e/d47/f95 0 2024-03-29T08:07:22.251 INFO:tasks.workunit.client.0.smithi037.stdout:0/346: symlink d9/da/d1b/l6b 0 2024-03-29T08:07:22.255 INFO:tasks.workunit.client.1.smithi148.stdout:8/631: truncate d17/d18/d26/d4f/fb7 416572 0 2024-03-29T08:07:22.255 INFO:tasks.workunit.client.0.smithi037.stdout:9/674: mknod d1/d4/d95/de4/ceb 0 2024-03-29T08:07:22.257 INFO:tasks.workunit.client.0.smithi037.stdout:0/347: chown d9/da/dd/d13/l3f 191641443 1 2024-03-29T08:07:22.257 INFO:tasks.workunit.client.1.smithi148.stdout:3/626: rename de/d3b/d49/c91 to de/d24/d33/d39/d46/d30/d37/d62/d63/cc9 0 2024-03-29T08:07:22.258 INFO:tasks.workunit.client.0.smithi037.stdout:3/434: rename dd/d5e/f72 to dd/d15/d27/d33/f96 0 2024-03-29T08:07:22.260 INFO:tasks.workunit.client.1.smithi148.stdout:8/632: symlink d17/d1d/d7e/ld5 0 2024-03-29T08:07:22.261 INFO:tasks.workunit.client.0.smithi037.stdout:9/675: write d1/d4/d19/d1e/d4a/d6f/da2/d23/f9a [4721332,83840] 0 2024-03-29T08:07:22.262 INFO:tasks.workunit.client.1.smithi148.stdout:7/556: getdents d8/d38/d3c/da5 0 2024-03-29T08:07:22.263 INFO:tasks.workunit.client.0.smithi037.stdout:3/435: write dd/d46/d2d/d3a/d6c/f8b [91158,98644] 0 2024-03-29T08:07:22.264 INFO:tasks.workunit.client.0.smithi037.stdout:0/348: creat d9/f6c x:0 0 0 2024-03-29T08:07:22.264 INFO:tasks.workunit.client.1.smithi148.stdout:1/573: creat d3/de/d25/d35/faa x:0 0 0 2024-03-29T08:07:22.269 INFO:tasks.workunit.client.0.smithi037.stdout:9/676: unlink d1/d4/d19/d1e/d25/f34 0 2024-03-29T08:07:22.269 INFO:tasks.workunit.client.0.smithi037.stdout:9/677: readlink d1/d1f/ldb 0 2024-03-29T08:07:22.269 INFO:tasks.workunit.client.0.smithi037.stdout:0/349: write d9/da/d1b/f41 [407346,98103] 0 2024-03-29T08:07:22.273 INFO:tasks.workunit.client.1.smithi148.stdout:7/557: read d8/d29/d44/d50/f8b [171812,38812] 0 2024-03-29T08:07:22.280 INFO:tasks.workunit.client.1.smithi148.stdout:3/627: getdents de/d24/d33/d39/d46/d30/d37 0 2024-03-29T08:07:22.285 INFO:tasks.workunit.client.0.smithi037.stdout:3/436: dwrite dd/d46/f2c [0,4194304] 0 2024-03-29T08:07:22.286 INFO:tasks.workunit.client.1.smithi148.stdout:3/628: dread - de/d3b/fb8 zero size 2024-03-29T08:07:22.287 INFO:tasks.workunit.client.1.smithi148.stdout:1/574: link d3/de/d25/f9d d3/d18/da8/fab 0 2024-03-29T08:07:22.288 INFO:tasks.workunit.client.1.smithi148.stdout:2/675: dwrite d7/da/f40 [0,4194304] 0 2024-03-29T08:07:22.289 INFO:tasks.workunit.client.1.smithi148.stdout:5/582: dwrite d0/db/d1e/f7e [0,4194304] 0 2024-03-29T08:07:22.291 INFO:tasks.workunit.client.1.smithi148.stdout:7/558: mkdir d8/d38/d3c/da9/d37/dc8 0 2024-03-29T08:07:22.297 INFO:tasks.workunit.client.1.smithi148.stdout:3/629: chown de/d24/d33 56436340 1 2024-03-29T08:07:22.298 INFO:tasks.workunit.client.1.smithi148.stdout:7/559: chown d8/d14/da8 1641 1 2024-03-29T08:07:22.298 INFO:tasks.workunit.client.1.smithi148.stdout:6/623: dread d6/f19 [0,4194304] 0 2024-03-29T08:07:22.298 INFO:tasks.workunit.client.0.smithi037.stdout:3/437: mkdir dd/d15/d27/d33/d97 0 2024-03-29T08:07:22.299 INFO:tasks.workunit.client.1.smithi148.stdout:5/583: unlink d0/f12 0 2024-03-29T08:07:22.301 INFO:tasks.workunit.client.1.smithi148.stdout:7/560: write d8/d14/d45/d4d/f9d [765232,55916] 0 2024-03-29T08:07:22.302 INFO:tasks.workunit.client.1.smithi148.stdout:3/630: chown de/d24/d33/d39/d71/f9b 2 1 2024-03-29T08:07:22.304 INFO:tasks.workunit.client.1.smithi148.stdout:2/676: creat d7/da/d33/d3c/d31/fd7 x:0 0 0 2024-03-29T08:07:22.306 INFO:tasks.workunit.client.0.smithi037.stdout:9/678: dwrite d1/d4/f3b [0,4194304] 0 2024-03-29T08:07:22.306 INFO:tasks.workunit.client.0.smithi037.stdout:6/380: dwrite d0/d5/d1f/f30 [0,4194304] 0 2024-03-29T08:07:22.313 INFO:tasks.workunit.client.1.smithi148.stdout:3/631: fdatasync de/d24/d33/f9a 0 2024-03-29T08:07:22.316 INFO:tasks.workunit.client.1.smithi148.stdout:0/594: dwrite db/de/df/d11/f37 [0,4194304] 0 2024-03-29T08:07:22.320 INFO:tasks.workunit.client.0.smithi037.stdout:3/438: fsync fb 0 2024-03-29T08:07:22.322 INFO:tasks.workunit.client.1.smithi148.stdout:2/677: fsync f5 0 2024-03-29T08:07:22.322 INFO:tasks.workunit.client.0.smithi037.stdout:6/381: dread - d0/d5/d1a/f6c zero size 2024-03-29T08:07:22.324 INFO:tasks.workunit.client.1.smithi148.stdout:1/575: dread d3/d18/f39 [0,4194304] 0 2024-03-29T08:07:22.329 INFO:tasks.workunit.client.1.smithi148.stdout:6/624: rename d6/d9/d1e/d27/c97 to d6/d9/d1e/d27/d48/d9f/cc9 0 2024-03-29T08:07:22.333 INFO:tasks.workunit.client.0.smithi037.stdout:0/350: dread d9/da/d1b/f3d [0,4194304] 0 2024-03-29T08:07:22.337 INFO:tasks.workunit.client.1.smithi148.stdout:2/678: symlink d7/da/d33/d3c/d54/d70/ld8 0 2024-03-29T08:07:22.339 INFO:tasks.workunit.client.1.smithi148.stdout:2/679: chown d7/da/d33/lb3 56793 1 2024-03-29T08:07:22.344 INFO:tasks.workunit.client.1.smithi148.stdout:7/561: rename d8/d14/d45/l64 to d8/d14/lc9 0 2024-03-29T08:07:22.344 INFO:tasks.workunit.client.1.smithi148.stdout:0/595: dwrite db/de/df/d2b/f77 [0,4194304] 0 2024-03-29T08:07:22.344 INFO:tasks.workunit.client.0.smithi037.stdout:5/568: dread d8/f52 [0,4194304] 0 2024-03-29T08:07:22.344 INFO:tasks.workunit.client.1.smithi148.stdout:3/632: symlink de/d24/d33/d39/d46/d30/d37/d62/d63/dbf/lca 0 2024-03-29T08:07:22.346 INFO:tasks.workunit.client.0.smithi037.stdout:5/569: readlink d8/l6f 0 2024-03-29T08:07:22.346 INFO:tasks.workunit.client.1.smithi148.stdout:6/625: getdents d6/d9/dc7 0 2024-03-29T08:07:22.349 INFO:tasks.workunit.client.0.smithi037.stdout:9/679: dwrite d1/d4/d19/d1e/d4a/d6f/da2/d23/f9a [0,4194304] 0 2024-03-29T08:07:22.353 INFO:tasks.workunit.client.1.smithi148.stdout:1/576: rename d3/d18/d4d/d9e/d61 to d3/de/d21/d5e/dac 0 2024-03-29T08:07:22.354 INFO:tasks.workunit.client.1.smithi148.stdout:7/562: stat d8/d38/d3c/da9/d37/c97 0 2024-03-29T08:07:22.361 INFO:tasks.workunit.client.1.smithi148.stdout:5/584: dwrite d0/db/d7b/d16/d87/fad [0,4194304] 0 2024-03-29T08:07:22.362 INFO:tasks.workunit.client.1.smithi148.stdout:3/633: creat de/d24/d33/d39/d46/d30/d37/d62/d63/fcb x:0 0 0 2024-03-29T08:07:22.364 INFO:tasks.workunit.client.1.smithi148.stdout:1/577: creat d3/d55/d7e/fad x:0 0 0 2024-03-29T08:07:22.364 INFO:tasks.workunit.client.1.smithi148.stdout:6/626: write d6/d9/d1e/d27/d48/d73/d7e/f92 [302813,82589] 0 2024-03-29T08:07:22.367 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:07:22 smithi148 ceph-mon[44992]: pgmap v43: 65 pgs: 65 active+clean; 1.8 GiB data, 7.1 GiB used, 529 GiB / 536 GiB avail; 58 MiB/s rd, 207 MiB/s wr, 369 op/s 2024-03-29T08:07:22.367 INFO:tasks.workunit.client.1.smithi148.stdout:0/596: truncate db/dc/daa/d47/d4c/d5a/fb6 879839 0 2024-03-29T08:07:22.373 INFO:tasks.workunit.client.1.smithi148.stdout:6/627: mkdir d6/d9/d68/dca 0 2024-03-29T08:07:22.373 INFO:tasks.workunit.client.1.smithi148.stdout:1/578: truncate d3/d18/f20 1990379 0 2024-03-29T08:07:22.374 INFO:tasks.workunit.client.0.smithi037.stdout:2/330: sync 2024-03-29T08:07:22.374 INFO:tasks.workunit.client.1.smithi148.stdout:0/597: chown db/f51 6891410 1 2024-03-29T08:07:22.375 INFO:tasks.workunit.client.0.smithi037.stdout:6/382: rename d0/d19/l2d to d0/d6a/l81 0 2024-03-29T08:07:22.377 INFO:tasks.workunit.client.1.smithi148.stdout:0/598: write db/de/df/d11/d2d/f85 [1039593,7553] 0 2024-03-29T08:07:22.381 INFO:tasks.workunit.client.0.smithi037.stdout:5/570: truncate d8/d22/d4f/f5a 1574323 0 2024-03-29T08:07:22.382 INFO:tasks.workunit.client.1.smithi148.stdout:6/628: creat d6/d9/d1e/d20/d30/fcb x:0 0 0 2024-03-29T08:07:22.383 INFO:tasks.workunit.client.1.smithi148.stdout:6/629: chown d6/d9/d1e 678 1 2024-03-29T08:07:22.384 INFO:tasks.workunit.client.0.smithi037.stdout:6/383: dread - d0/d5/d1e/f5f zero size 2024-03-29T08:07:22.387 INFO:tasks.workunit.client.0.smithi037.stdout:6/384: write d0/d5/d1a/f5d [4457967,81221] 0 2024-03-29T08:07:22.389 INFO:tasks.workunit.client.0.smithi037.stdout:5/571: creat d8/d22/d4f/d70/fbe x:0 0 0 2024-03-29T08:07:22.392 INFO:tasks.workunit.client.0.smithi037.stdout:9/680: unlink d1/d4/d19/d1e/d4a/d6f/l93 0 2024-03-29T08:07:22.393 INFO:tasks.workunit.client.1.smithi148.stdout:6/630: mkdir d6/d9/d1e/d36/d4d/d9d/d28/d98/dcc 0 2024-03-29T08:07:22.396 INFO:tasks.workunit.client.1.smithi148.stdout:2/680: dwrite d7/da/d4c/d9a/dad/fcd [0,4194304] 0 2024-03-29T08:07:22.399 INFO:tasks.workunit.client.0.smithi037.stdout:5/572: mkdir d8/d4d/d5c/dbf 0 2024-03-29T08:07:22.399 INFO:tasks.workunit.client.0.smithi037.stdout:9/681: write d1/d4/d19/d1e/d4a/d6f/da2/d23/f37 [5585488,10178] 0 2024-03-29T08:07:22.399 INFO:tasks.workunit.client.0.smithi037.stdout:2/331: creat d0/d20/f75 x:0 0 0 2024-03-29T08:07:22.399 INFO:tasks.workunit.client.1.smithi148.stdout:6/631: write d6/f19 [1917876,105155] 0 2024-03-29T08:07:22.403 INFO:tasks.workunit.client.1.smithi148.stdout:0/599: dwrite fa [0,4194304] 0 2024-03-29T08:07:22.403 INFO:tasks.workunit.client.0.smithi037.stdout:5/573: mknod d8/d4d/cc0 0 2024-03-29T08:07:22.405 INFO:tasks.workunit.client.1.smithi148.stdout:2/681: mknod d7/da/dcc/cd9 0 2024-03-29T08:07:22.407 INFO:tasks.workunit.client.0.smithi037.stdout:5/574: mkdir d8/d22/dc1 0 2024-03-29T08:07:22.408 INFO:tasks.workunit.client.1.smithi148.stdout:2/682: write d7/da/f49 [812628,129931] 0 2024-03-29T08:07:22.408 INFO:tasks.workunit.client.1.smithi148.stdout:6/632: rename d6/d9/l5c to d6/d9/d1e/d36/d4d/d9d/d28/d98/lcd 0 2024-03-29T08:07:22.409 INFO:tasks.workunit.client.0.smithi037.stdout:5/575: chown d8/d18/f28 0 1 2024-03-29T08:07:22.412 INFO:tasks.workunit.client.0.smithi037.stdout:9/682: getdents d1/d4 0 2024-03-29T08:07:22.413 INFO:tasks.workunit.client.1.smithi148.stdout:9/633: sync 2024-03-29T08:07:22.413 INFO:tasks.workunit.client.1.smithi148.stdout:2/683: readlink d7/da/d33/d3c/d54/d70/l99 0 2024-03-29T08:07:22.415 INFO:tasks.workunit.client.1.smithi148.stdout:6/633: write d6/f19 [98303,12320] 0 2024-03-29T08:07:22.418 INFO:tasks.workunit.client.1.smithi148.stdout:9/634: fdatasync db/d22/d26/d54/d4d/d35/d4b/dc3/d94/f9e 0 2024-03-29T08:07:22.418 INFO:tasks.workunit.client.0.smithi037.stdout:9/683: creat d1/d4/d19/d1e/d71/fec x:0 0 0 2024-03-29T08:07:22.419 INFO:tasks.workunit.client.1.smithi148.stdout:2/684: mkdir d7/d8c/da4/d8e/dda 0 2024-03-29T08:07:22.420 INFO:tasks.workunit.client.1.smithi148.stdout:7/563: truncate d8/d14/d45/d4d/d78/fc2 541154 0 2024-03-29T08:07:22.420 INFO:tasks.workunit.client.0.smithi037.stdout:3/439: write dd/d15/f50 [2043502,119077] 0 2024-03-29T08:07:22.423 INFO:tasks.workunit.client.1.smithi148.stdout:5/585: write d0/db/d1e/d36/d41/f58 [4756420,6504] 0 2024-03-29T08:07:22.425 INFO:tasks.workunit.client.0.smithi037.stdout:9/684: fdatasync d1/d4/d19/d1e/d25/fc1 0 2024-03-29T08:07:22.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:07:22 smithi037 ceph-mon[35162]: pgmap v43: 65 pgs: 65 active+clean; 1.8 GiB data, 7.1 GiB used, 529 GiB / 536 GiB avail; 58 MiB/s rd, 207 MiB/s wr, 369 op/s 2024-03-29T08:07:22.427 INFO:tasks.workunit.client.1.smithi148.stdout:3/634: write de/d24/d33/d39/d46/f25 [5144836,50872] 0 2024-03-29T08:07:22.430 INFO:tasks.workunit.client.0.smithi037.stdout:3/440: chown dd/d46/d2d/d3a/d5f/f6a 28 1 2024-03-29T08:07:22.430 INFO:tasks.workunit.client.1.smithi148.stdout:7/564: dread d8/d38/d3c/da9/d37/f48 [0,4194304] 0 2024-03-29T08:07:22.433 INFO:tasks.workunit.client.1.smithi148.stdout:0/600: dwrite db/dc/daa/d47/d4c/d5a/f98 [0,4194304] 0 2024-03-29T08:07:22.434 INFO:tasks.workunit.client.1.smithi148.stdout:7/565: chown d8/d38/f8d 2815 1 2024-03-29T08:07:22.437 INFO:tasks.workunit.client.1.smithi148.stdout:0/601: readlink db/dc/daa/d47/d4c/l5c 0 2024-03-29T08:07:22.437 INFO:tasks.workunit.client.1.smithi148.stdout:1/579: dread d3/de/d25/d51/f54 [0,4194304] 0 2024-03-29T08:07:22.440 INFO:tasks.workunit.client.0.smithi037.stdout:3/441: rename dd/d46/c82 to dd/d46/d2d/d3a/d5f/c98 0 2024-03-29T08:07:22.441 INFO:tasks.workunit.client.1.smithi148.stdout:2/685: dread d7/da/d33/d45/d4d/d2e/f50 [0,4194304] 0 2024-03-29T08:07:22.442 INFO:tasks.workunit.client.0.smithi037.stdout:0/351: dwrite d9/da/dd/d20/d38/d53/f62 [0,4194304] 0 2024-03-29T08:07:22.444 INFO:tasks.workunit.client.1.smithi148.stdout:1/580: mkdir d3/d18/dae 0 2024-03-29T08:07:22.444 INFO:tasks.workunit.client.1.smithi148.stdout:9/635: write db/d22/d26/d54/d4d/d35/f5e [1758653,67840] 0 2024-03-29T08:07:22.445 INFO:tasks.workunit.client.0.smithi037.stdout:9/685: getdents d1/d4 0 2024-03-29T08:07:22.449 INFO:tasks.workunit.client.1.smithi148.stdout:2/686: symlink d7/da/d33/d3c/d54/ldb 0 2024-03-29T08:07:22.450 INFO:tasks.workunit.client.1.smithi148.stdout:5/586: dread d0/db/d7b/d16/f2f [0,4194304] 0 2024-03-29T08:07:22.451 INFO:tasks.workunit.client.1.smithi148.stdout:9/636: rmdir db/d10/d2d/d4e/d5f 39 2024-03-29T08:07:22.453 INFO:tasks.workunit.client.1.smithi148.stdout:0/602: getdents db/de/df/d11 0 2024-03-29T08:07:22.453 INFO:tasks.workunit.client.0.smithi037.stdout:8/464: sync 2024-03-29T08:07:22.453 INFO:tasks.workunit.client.0.smithi037.stdout:1/497: sync 2024-03-29T08:07:22.454 INFO:tasks.workunit.client.1.smithi148.stdout:9/637: readlink db/lf 0 2024-03-29T08:07:22.455 INFO:tasks.workunit.client.1.smithi148.stdout:5/587: creat d0/db/d1e/fbd x:0 0 0 2024-03-29T08:07:22.456 INFO:tasks.workunit.client.0.smithi037.stdout:5/576: rmdir d8/d22/d37/d8d/db3 39 2024-03-29T08:07:22.456 INFO:tasks.workunit.client.1.smithi148.stdout:2/687: dread - d7/da/d33/d3c/d54/f95 zero size 2024-03-29T08:07:22.459 INFO:tasks.workunit.client.1.smithi148.stdout:3/635: dwrite de/d24/d33/d39/d46/d30/fa2 [0,4194304] 0 2024-03-29T08:07:22.462 INFO:tasks.workunit.client.0.smithi037.stdout:9/686: unlink d1/d4/d19/d69/d75/cb3 0 2024-03-29T08:07:22.463 INFO:tasks.workunit.client.1.smithi148.stdout:9/638: mknod db/d22/da3/cda 0 2024-03-29T08:07:22.464 INFO:tasks.workunit.client.0.smithi037.stdout:0/352: creat d9/da/d1b/d33/f6d x:0 0 0 2024-03-29T08:07:22.467 INFO:tasks.workunit.client.1.smithi148.stdout:2/688: symlink d7/da/dcc/ldc 0 2024-03-29T08:07:22.467 INFO:tasks.workunit.client.1.smithi148.stdout:5/588: mknod d0/db/d1e/d35/d42/cbe 0 2024-03-29T08:07:22.468 INFO:tasks.workunit.client.0.smithi037.stdout:2/332: dread d0/f7 [0,4194304] 0 2024-03-29T08:07:22.468 INFO:tasks.workunit.client.0.smithi037.stdout:1/498: symlink d2/d6/dc/d18/d22/da0/lab 0 2024-03-29T08:07:22.469 INFO:tasks.workunit.client.1.smithi148.stdout:7/566: dread d8/d14/d45/d4d/d78/fc2 [0,4194304] 0 2024-03-29T08:07:22.470 INFO:tasks.workunit.client.0.smithi037.stdout:6/385: dwrite d0/d5/fb [0,4194304] 0 2024-03-29T08:07:22.472 INFO:tasks.workunit.client.1.smithi148.stdout:9/639: dread - db/d22/d26/d54/f60 zero size 2024-03-29T08:07:22.474 INFO:tasks.workunit.client.0.smithi037.stdout:8/465: mkdir d5/d10/d2d/d34/d50/d83/d46/d8a/d96 0 2024-03-29T08:07:22.475 INFO:tasks.workunit.client.0.smithi037.stdout:3/442: dwrite dd/d15/f19 [0,4194304] 0 2024-03-29T08:07:22.475 INFO:tasks.workunit.client.0.smithi037.stdout:2/333: fsync d0/f1b 0 2024-03-29T08:07:22.480 INFO:tasks.workunit.client.0.smithi037.stdout:6/386: chown d0/d19/d34/d4e/c68 0 1 2024-03-29T08:07:22.481 INFO:tasks.workunit.client.0.smithi037.stdout:9/687: mkdir d1/d4/d19/d1e/d4a/d6f/da2/d23/ded 0 2024-03-29T08:07:22.483 INFO:tasks.workunit.client.1.smithi148.stdout:1/581: dwrite d3/de/d25/d35/d42/f15 [4194304,4194304] 0 2024-03-29T08:07:22.486 INFO:tasks.workunit.client.1.smithi148.stdout:5/589: unlink d0/db/d1e/d5b/d66/l93 0 2024-03-29T08:07:22.487 INFO:tasks.workunit.client.0.smithi037.stdout:9/688: write d1/d1f/d3a/fc0 [934571,26450] 0 2024-03-29T08:07:22.488 INFO:tasks.workunit.client.1.smithi148.stdout:7/567: symlink d8/d38/d67/lca 0 2024-03-29T08:07:22.490 INFO:tasks.workunit.client.1.smithi148.stdout:2/689: unlink d7/da/d33/d45/d4d/d2e/c77 0 2024-03-29T08:07:22.492 INFO:tasks.workunit.client.0.smithi037.stdout:2/334: creat d0/d23/d70/f76 x:0 0 0 2024-03-29T08:07:22.492 INFO:tasks.workunit.client.0.smithi037.stdout:1/499: mknod d2/d6/dc/d18/d1d/d21/cac 0 2024-03-29T08:07:22.494 INFO:tasks.workunit.client.0.smithi037.stdout:6/387: rename d0/d5/f16 to d0/d19/d42/d60/d62/f82 0 2024-03-29T08:07:22.494 INFO:tasks.workunit.client.1.smithi148.stdout:0/603: dwrite db/de/df/d11/f16 [0,4194304] 0 2024-03-29T08:07:22.497 INFO:tasks.workunit.client.0.smithi037.stdout:3/443: creat dd/d15/d27/d33/d75/d8f/f99 x:0 0 0 2024-03-29T08:07:22.500 INFO:tasks.workunit.client.1.smithi148.stdout:0/604: fdatasync db/de/f3f 0 2024-03-29T08:07:22.503 INFO:tasks.workunit.client.1.smithi148.stdout:5/590: fsync d0/db/d7b/fb6 0 2024-03-29T08:07:22.506 INFO:tasks.workunit.client.0.smithi037.stdout:5/577: dwrite d8/f42 [0,4194304] 0 2024-03-29T08:07:22.506 INFO:tasks.workunit.client.1.smithi148.stdout:7/568: rename d8/d14/f2c to d8/d38/d3c/da9/d37/dc8/fcb 0 2024-03-29T08:07:22.512 INFO:tasks.workunit.client.0.smithi037.stdout:3/444: unlink dd/d46/d3e/d47/l7b 0 2024-03-29T08:07:22.513 INFO:tasks.workunit.client.0.smithi037.stdout:0/353: dread d9/da/d1b/d22/d2c/f2d [0,4194304] 0 2024-03-29T08:07:22.514 INFO:tasks.workunit.client.0.smithi037.stdout:3/445: chown dd/d5e 1019029 1 2024-03-29T08:07:22.515 INFO:tasks.workunit.client.1.smithi148.stdout:0/605: dread - db/de/df/d11/d2d/fb2 zero size 2024-03-29T08:07:22.515 INFO:tasks.workunit.client.1.smithi148.stdout:9/640: dwrite db/d22/d26/d54/d4d/d35/f8c [0,4194304] 0 2024-03-29T08:07:22.515 INFO:tasks.workunit.client.1.smithi148.stdout:2/690: rename d7/da/f49 to d7/da/d4c/dc4/fdd 0 2024-03-29T08:07:22.516 INFO:tasks.workunit.client.0.smithi037.stdout:3/446: chown l3 19 1 2024-03-29T08:07:22.516 INFO:tasks.workunit.client.0.smithi037.stdout:0/354: dread - d9/da/dd/d13/f61 zero size 2024-03-29T08:07:22.518 INFO:tasks.workunit.client.0.smithi037.stdout:1/500: rename d2/d6/dc/d5d/d64 to d2/d6/dc/d18/d1d/dad 0 2024-03-29T08:07:22.520 INFO:tasks.workunit.client.1.smithi148.stdout:4/603: sync 2024-03-29T08:07:22.521 INFO:tasks.workunit.client.0.smithi037.stdout:0/355: readlink d9/da/dd/d13/l51 0 2024-03-29T08:07:22.521 INFO:tasks.workunit.client.1.smithi148.stdout:1/582: dwrite d3/d18/d71/f8e [0,4194304] 0 2024-03-29T08:07:22.521 INFO:tasks.workunit.client.1.smithi148.stdout:8/633: sync 2024-03-29T08:07:22.524 INFO:tasks.workunit.client.1.smithi148.stdout:2/691: chown d7/d8c/d91/l2f 1357 1 2024-03-29T08:07:22.527 INFO:tasks.workunit.client.0.smithi037.stdout:5/578: creat d8/d4d/d5c/dbf/fc2 x:0 0 0 2024-03-29T08:07:22.529 INFO:tasks.workunit.client.1.smithi148.stdout:4/604: creat df/d2c/d51/d77/d17/d73/fc0 x:0 0 0 2024-03-29T08:07:22.530 INFO:tasks.workunit.client.0.smithi037.stdout:0/356: mknod d9/da/d1b/d22/d2c/c6e 0 2024-03-29T08:07:22.530 INFO:tasks.workunit.client.1.smithi148.stdout:2/692: unlink d7/da/d4c/c72 0 2024-03-29T08:07:22.535 INFO:tasks.workunit.client.1.smithi148.stdout:9/641: mknod db/d10/cdb 0 2024-03-29T08:07:22.536 INFO:tasks.workunit.client.0.smithi037.stdout:3/447: link dd/d46/d2d/d3a/d5f/f5a dd/d46/d2d/d3a/d5f/f9a 0 2024-03-29T08:07:22.536 INFO:tasks.workunit.client.1.smithi148.stdout:1/583: creat d3/d18/faf x:0 0 0 2024-03-29T08:07:22.538 INFO:tasks.workunit.client.1.smithi148.stdout:0/606: dread db/f97 [0,4194304] 0 2024-03-29T08:07:22.538 INFO:tasks.workunit.client.1.smithi148.stdout:4/605: rmdir df/d2c/d51 39 2024-03-29T08:07:22.539 INFO:tasks.workunit.client.1.smithi148.stdout:1/584: readlink d3/de/d21/d5e/dac/l6a 0 2024-03-29T08:07:22.540 INFO:tasks.workunit.client.0.smithi037.stdout:2/335: dwrite d0/d20/d2e/f36 [0,4194304] 0 2024-03-29T08:07:22.541 INFO:tasks.workunit.client.0.smithi037.stdout:5/579: creat d8/d4d/d5c/fc3 x:0 0 0 2024-03-29T08:07:22.542 INFO:tasks.workunit.client.1.smithi148.stdout:5/591: read d0/db/d7b/d16/d2e/f56 [13708,2291] 0 2024-03-29T08:07:22.543 INFO:tasks.workunit.client.0.smithi037.stdout:0/357: truncate d9/da/dd/d20/d38/f69 14630 0 2024-03-29T08:07:22.544 INFO:tasks.workunit.client.0.smithi037.stdout:2/336: readlink d0/d20/d2e/l5d 0 2024-03-29T08:07:22.546 INFO:tasks.workunit.client.0.smithi037.stdout:3/448: truncate dd/d15/d18/d4b/f67 358437 0 2024-03-29T08:07:22.548 INFO:tasks.workunit.client.1.smithi148.stdout:9/642: write db/d22/d26/d54/f45 [628023,33522] 0 2024-03-29T08:07:22.550 INFO:tasks.workunit.client.1.smithi148.stdout:2/693: getdents d7/d8c/da4/d8e/dda 0 2024-03-29T08:07:22.552 INFO:tasks.workunit.client.0.smithi037.stdout:3/449: write dd/d15/d27/d33/d75/f8d [683922,37098] 0 2024-03-29T08:07:22.552 INFO:tasks.workunit.client.1.smithi148.stdout:4/606: dread - df/d2c/d51/d77/d26/f7d zero size 2024-03-29T08:07:22.554 INFO:tasks.workunit.client.1.smithi148.stdout:0/607: rename db/de/df/d11/f71 to db/de/df/d2b/dae/fbd 0 2024-03-29T08:07:22.555 INFO:tasks.workunit.client.1.smithi148.stdout:3/636: dread de/d24/d33/d39/d7f/fac [0,4194304] 0 2024-03-29T08:07:22.556 INFO:tasks.workunit.client.0.smithi037.stdout:3/450: chown dd/d46/d2d/d3a/f52 0 1 2024-03-29T08:07:22.557 INFO:tasks.workunit.client.1.smithi148.stdout:9/643: dread - db/d22/d26/d54/fae zero size 2024-03-29T08:07:22.559 INFO:tasks.workunit.client.1.smithi148.stdout:3/637: readlink de/d24/d33/d39/d46/l6a 0 2024-03-29T08:07:22.559 INFO:tasks.workunit.client.1.smithi148.stdout:2/694: creat d7/da/d4c/d9a/dad/fde x:0 0 0 2024-03-29T08:07:22.562 INFO:tasks.workunit.client.1.smithi148.stdout:9/644: chown db/d10/d2d/d4e/dd4/f1e 97282 1 2024-03-29T08:07:22.563 INFO:tasks.workunit.client.1.smithi148.stdout:2/695: write d7/da/d33/d3c/d54/f93 [103175,115390] 0 2024-03-29T08:07:22.564 INFO:tasks.workunit.client.1.smithi148.stdout:9/645: fdatasync db/d10/d2d/d4e/dd4/f70 0 2024-03-29T08:07:22.565 INFO:tasks.workunit.client.1.smithi148.stdout:9/646: fdatasync db/d10/faf 0 2024-03-29T08:07:22.566 INFO:tasks.workunit.client.0.smithi037.stdout:0/358: dwrite d9/da/d1b/d22/d2c/f2d [0,4194304] 0 2024-03-29T08:07:22.567 INFO:tasks.workunit.client.1.smithi148.stdout:5/592: rename d0/db/d1e/f4a to d0/db/d1e/d5b/d66/d72/d95/d9c/fbf 0 2024-03-29T08:07:22.567 INFO:tasks.workunit.client.1.smithi148.stdout:9/647: write db/d10/d6d/d73/da8/fd8 [848237,68770] 0 2024-03-29T08:07:22.568 INFO:tasks.workunit.client.0.smithi037.stdout:1/501: getdents d2/d6/dc/d18/d22/d4f/d7c 0 2024-03-29T08:07:22.569 INFO:tasks.workunit.client.1.smithi148.stdout:9/648: write db/f43 [7233067,88538] 0 2024-03-29T08:07:22.570 INFO:tasks.workunit.client.1.smithi148.stdout:7/569: rmdir d8/d38/d3c/da9/d37 39 2024-03-29T08:07:22.571 INFO:tasks.workunit.client.1.smithi148.stdout:4/607: creat df/d2c/d51/d77/d26/d9b/fc1 x:0 0 0 2024-03-29T08:07:22.571 INFO:tasks.workunit.client.1.smithi148.stdout:0/608: rmdir db/dc/daa/d47/d4c 39 2024-03-29T08:07:22.571 INFO:tasks.workunit.client.0.smithi037.stdout:0/359: write d9/da/d1b/d22/d2c/f2d [2403612,69633] 0 2024-03-29T08:07:22.574 INFO:tasks.workunit.client.1.smithi148.stdout:4/608: readlink df/d2c/d51/d77/d26/d9b/da7/l32 0 2024-03-29T08:07:22.575 INFO:tasks.workunit.client.1.smithi148.stdout:7/570: truncate d8/d38/d3c/da9/db7/fc4 1035942 0 2024-03-29T08:07:22.576 INFO:tasks.workunit.client.1.smithi148.stdout:3/638: creat de/d24/d33/d39/da3/fcc x:0 0 0 2024-03-29T08:07:22.577 INFO:tasks.workunit.client.0.smithi037.stdout:9/689: dread d1/d4/d19/fb9 [0,4194304] 0 2024-03-29T08:07:22.582 INFO:tasks.workunit.client.0.smithi037.stdout:6/388: unlink d0/d19/d42/d60/d62/f82 0 2024-03-29T08:07:22.583 INFO:tasks.workunit.client.1.smithi148.stdout:0/609: rename db/de/l74 to db/de/d24/lbe 0 2024-03-29T08:07:22.584 INFO:tasks.workunit.client.1.smithi148.stdout:0/610: write fa [1630125,102452] 0 2024-03-29T08:07:22.585 INFO:tasks.workunit.client.1.smithi148.stdout:5/593: symlink d0/db/d84/daf/lc0 0 2024-03-29T08:07:22.589 INFO:tasks.workunit.client.1.smithi148.stdout:7/571: creat d8/d14/fcc x:0 0 0 2024-03-29T08:07:22.589 INFO:tasks.workunit.client.1.smithi148.stdout:2/696: unlink d7/da/l1d 0 2024-03-29T08:07:22.590 INFO:tasks.workunit.client.0.smithi037.stdout:2/337: unlink d0/f1b 0 2024-03-29T08:07:22.590 INFO:tasks.workunit.client.1.smithi148.stdout:0/611: write db/dc/daa/d47/d9f/fa5 [339007,100181] 0 2024-03-29T08:07:22.593 INFO:tasks.workunit.client.0.smithi037.stdout:3/451: mknod dd/d15/c9b 0 2024-03-29T08:07:22.594 INFO:tasks.workunit.client.1.smithi148.stdout:3/639: rename de/d24/d33/d39/d46/d30/f3f to de/d24/d33/d39/da3/fcd 0 2024-03-29T08:07:22.594 INFO:tasks.workunit.client.1.smithi148.stdout:8/634: dwrite d17/d18/f5c [0,4194304] 0 2024-03-29T08:07:22.597 INFO:tasks.workunit.client.0.smithi037.stdout:0/360: rename d9/da/d1b/d22/f47 to d9/da/dd/d13/f6f 0 2024-03-29T08:07:22.599 INFO:tasks.workunit.client.0.smithi037.stdout:5/580: symlink d8/d22/db6/lc4 0 2024-03-29T08:07:22.599 INFO:tasks.workunit.client.0.smithi037.stdout:1/502: read d2/d6/d13/f28 [931820,18652] 0 2024-03-29T08:07:22.602 INFO:tasks.workunit.client.1.smithi148.stdout:4/609: getdents df/da0 0 2024-03-29T08:07:22.602 INFO:tasks.workunit.client.0.smithi037.stdout:0/361: chown d9/da/dd/d20/l68 137 1 2024-03-29T08:07:22.605 INFO:tasks.workunit.client.1.smithi148.stdout:8/635: dread - d17/d18/d26/f98 zero size 2024-03-29T08:07:22.607 INFO:tasks.workunit.client.1.smithi148.stdout:8/636: stat d17/d49/d9a/dbe/d6f/ca7 0 2024-03-29T08:07:22.613 INFO:tasks.workunit.client.1.smithi148.stdout:0/612: dread db/dc/f1e [0,4194304] 0 2024-03-29T08:07:22.614 INFO:tasks.workunit.client.1.smithi148.stdout:7/572: rename d8/d29/d44/d50/d94/c9a to d8/d14/da8/ccd 0 2024-03-29T08:07:22.615 INFO:tasks.workunit.client.0.smithi037.stdout:2/338: symlink d0/d20/d2e/d38/l77 0 2024-03-29T08:07:22.617 INFO:tasks.workunit.client.1.smithi148.stdout:8/637: mknod d17/d49/d9a/dbe/d6f/d64/cd6 0 2024-03-29T08:07:22.620 INFO:tasks.workunit.client.1.smithi148.stdout:2/697: rename d7/da/d33/d45/d4d/c74 to d7/da/d33/d45/d4d/d82/d8d/cdf 0 2024-03-29T08:07:22.620 INFO:tasks.workunit.client.1.smithi148.stdout:4/610: dwrite df/d2c/d51/d77/d26/d34/d8a/f95 [0,4194304] 0 2024-03-29T08:07:22.621 INFO:tasks.workunit.client.0.smithi037.stdout:3/452: unlink dd/d15/d18/l20 0 2024-03-29T08:07:22.621 INFO:tasks.workunit.client.1.smithi148.stdout:9/649: truncate db/d10/d6d/d73/da8/fd8 507605 0 2024-03-29T08:07:22.622 INFO:tasks.workunit.client.0.smithi037.stdout:1/503: mkdir d2/d6/d13/dae 0 2024-03-29T08:07:22.623 INFO:tasks.workunit.client.1.smithi148.stdout:4/611: stat df/d60/c78 0 2024-03-29T08:07:22.625 INFO:tasks.workunit.client.1.smithi148.stdout:9/650: fsync db/d22/d26/d54/d4d/d35/f8c 0 2024-03-29T08:07:22.628 INFO:tasks.workunit.client.1.smithi148.stdout:3/640: rename de/d24/d33/f5c to de/d24/d33/d39/d46/d30/d37/d62/d63/fce 0 2024-03-29T08:07:22.629 INFO:tasks.workunit.client.0.smithi037.stdout:9/690: write d1/d4/d19/d1e/d4a/d6f/da2/d13/d7c/fc3 [42368,78990] 0 2024-03-29T08:07:22.629 INFO:tasks.workunit.client.0.smithi037.stdout:5/581: dwrite d8/d34/fba [0,4194304] 0 2024-03-29T08:07:22.629 INFO:tasks.workunit.client.1.smithi148.stdout:0/613: chown db/de/df/d11/d42/f52 141730441 1 2024-03-29T08:07:22.630 INFO:tasks.workunit.client.0.smithi037.stdout:2/339: truncate d0/d23/d44/f46 1939351 0 2024-03-29T08:07:22.631 INFO:tasks.workunit.client.1.smithi148.stdout:4/612: creat df/da0/fc2 x:0 0 0 2024-03-29T08:07:22.632 INFO:tasks.workunit.client.1.smithi148.stdout:0/614: readlink db/dc/d78/l91 0 2024-03-29T08:07:22.633 INFO:tasks.workunit.client.1.smithi148.stdout:8/638: creat d17/d18/d26/fd7 x:0 0 0 2024-03-29T08:07:22.633 INFO:tasks.workunit.client.0.smithi037.stdout:4/458: sync 2024-03-29T08:07:22.633 INFO:tasks.workunit.client.0.smithi037.stdout:7/464: sync 2024-03-29T08:07:22.636 INFO:tasks.workunit.client.1.smithi148.stdout:5/594: dwrite d0/db/d1e/d36/d41/f63 [0,4194304] 0 2024-03-29T08:07:22.636 INFO:tasks.workunit.client.1.smithi148.stdout:2/698: symlink d7/da/d33/d3c/le0 0 2024-03-29T08:07:22.637 INFO:tasks.workunit.client.0.smithi037.stdout:4/459: chown d3/d57/f7a 214562327 1 2024-03-29T08:07:22.639 INFO:tasks.workunit.client.1.smithi148.stdout:3/641: rmdir de/d24/d33/d39/d7f 39 2024-03-29T08:07:22.639 INFO:tasks.workunit.client.0.smithi037.stdout:2/340: chown d0/d23/d44/d62 0 1 2024-03-29T08:07:22.640 INFO:tasks.workunit.client.1.smithi148.stdout:5/595: creat d0/d46/d9e/fc1 x:0 0 0 2024-03-29T08:07:22.641 INFO:tasks.workunit.client.1.smithi148.stdout:2/699: mkdir d7/da/d33/d3c/d31/de1 0 2024-03-29T08:07:22.644 INFO:tasks.workunit.client.1.smithi148.stdout:0/615: truncate f7 8860186 0 2024-03-29T08:07:22.644 INFO:tasks.workunit.client.0.smithi037.stdout:1/504: rmdir d2/d6/dc/d18/d1d/dad 39 2024-03-29T08:07:22.649 INFO:tasks.workunit.client.0.smithi037.stdout:9/691: mknod d1/d4/d95/cee 0 2024-03-29T08:07:22.650 INFO:tasks.workunit.client.1.smithi148.stdout:3/642: creat de/d24/d33/d39/d46/d30/fcf x:0 0 0 2024-03-29T08:07:22.651 INFO:tasks.workunit.client.1.smithi148.stdout:4/613: dwrite df/d2c/d51/d77/d17/d73/fc0 [0,4194304] 0 2024-03-29T08:07:22.651 INFO:tasks.workunit.client.0.smithi037.stdout:0/362: dread d9/fb [0,4194304] 0 2024-03-29T08:07:22.654 INFO:tasks.workunit.client.0.smithi037.stdout:0/363: readlink d9/da/dd/d20/l68 0 2024-03-29T08:07:22.658 INFO:tasks.workunit.client.0.smithi037.stdout:7/465: rmdir dd/d21/d35 39 2024-03-29T08:07:22.660 INFO:tasks.workunit.client.1.smithi148.stdout:8/639: truncate d17/d18/f3e 779160 0 2024-03-29T08:07:22.662 INFO:tasks.workunit.client.1.smithi148.stdout:5/596: creat d0/db/d84/daf/fc2 x:0 0 0 2024-03-29T08:07:22.664 INFO:tasks.workunit.client.0.smithi037.stdout:7/466: write dd/d1d/d34/f66 [6047822,123738] 0 2024-03-29T08:07:22.666 INFO:tasks.workunit.client.0.smithi037.stdout:9/692: creat d1/d4/d19/d1e/d4a/d6f/da2/d31/daa/fef x:0 0 0 2024-03-29T08:07:22.669 INFO:tasks.workunit.client.0.smithi037.stdout:4/460: dwrite d3/d57/f8e [0,4194304] 0 2024-03-29T08:07:22.672 INFO:tasks.workunit.client.1.smithi148.stdout:8/640: dread d17/d74/f5a [0,4194304] 0 2024-03-29T08:07:22.673 INFO:tasks.workunit.client.1.smithi148.stdout:2/700: rename d7/da/d33/d3c/da5/fce to d7/da/d33/d3c/d31/dbb/fe2 0 2024-03-29T08:07:22.676 INFO:tasks.workunit.client.1.smithi148.stdout:2/701: chown d7/da/d33/d3c/d54/d70/f85 13 1 2024-03-29T08:07:22.681 INFO:tasks.workunit.client.1.smithi148.stdout:9/651: link db/d22/d26/d54/c98 db/d22/d26/d54/d4d/d35/d4b/dc3/d94/cdc 0 2024-03-29T08:07:22.683 INFO:tasks.workunit.client.1.smithi148.stdout:3/643: creat de/d24/d33/d39/da3/dbc/fd0 x:0 0 0 2024-03-29T08:07:22.689 INFO:tasks.workunit.client.0.smithi037.stdout:3/453: dread dd/d15/f16 [4194304,4194304] 0 2024-03-29T08:07:22.690 INFO:tasks.workunit.client.1.smithi148.stdout:2/702: dread d7/da/d33/d45/d4d/f16 [0,4194304] 0 2024-03-29T08:07:22.694 INFO:tasks.workunit.client.1.smithi148.stdout:9/652: dread db/f84 [0,4194304] 0 2024-03-29T08:07:22.699 INFO:tasks.workunit.client.1.smithi148.stdout:5/597: mknod d0/cc3 0 2024-03-29T08:07:22.699 INFO:tasks.workunit.client.1.smithi148.stdout:8/641: creat d17/d74/d86/fd8 x:0 0 0 2024-03-29T08:07:22.699 INFO:tasks.workunit.client.1.smithi148.stdout:0/616: getdents db/de/df/d11/d2d/da3 0 2024-03-29T08:07:22.700 INFO:tasks.workunit.client.1.smithi148.stdout:3/644: rename de/d24/d33/fc4 to de/d24/d33/d39/d46/d30/d37/d62/d63/dbf/fd1 0 2024-03-29T08:07:22.702 INFO:tasks.workunit.client.0.smithi037.stdout:1/505: mkdir d2/d6/dc/d18/d1d/dad/daf 0 2024-03-29T08:07:22.702 INFO:tasks.workunit.client.1.smithi148.stdout:7/573: write d8/d38/d3c/da9/d37/f48 [1142752,113288] 0 2024-03-29T08:07:22.707 INFO:tasks.workunit.client.0.smithi037.stdout:0/364: mknod d9/da/c70 0 2024-03-29T08:07:22.708 INFO:tasks.workunit.client.0.smithi037.stdout:2/341: dread d0/d23/d44/f46 [0,4194304] 0 2024-03-29T08:07:22.709 INFO:tasks.workunit.client.1.smithi148.stdout:4/614: dwrite df/d2c/d51/d77/d26/d34/d3e/f75 [4194304,4194304] 0 2024-03-29T08:07:22.710 INFO:tasks.workunit.client.0.smithi037.stdout:6/389: dwrite d0/d19/d34/d5a/f39 [0,4194304] 0 2024-03-29T08:07:22.710 INFO:tasks.workunit.client.0.smithi037.stdout:3/454: mknod dd/d15/d18/d8e/c9c 0 2024-03-29T08:07:22.711 INFO:tasks.workunit.client.1.smithi148.stdout:2/703: dwrite d7/da/d4c/d9a/dad/fcd [0,4194304] 0 2024-03-29T08:07:22.712 INFO:tasks.workunit.client.1.smithi148.stdout:7/574: mknod d8/d38/d67/cce 0 2024-03-29T08:07:22.716 INFO:tasks.workunit.client.1.smithi148.stdout:9/653: dwrite db/d22/d26/d54/d4d/d35/d4b/d58/f8b [0,4194304] 0 2024-03-29T08:07:22.716 INFO:tasks.workunit.client.1.smithi148.stdout:5/598: creat d0/db/d1e/d35/d42/d67/d76/d81/d98/fc4 x:0 0 0 2024-03-29T08:07:22.717 INFO:tasks.workunit.client.0.smithi037.stdout:0/365: write d9/da/dd/d13/f5e [825180,117614] 0 2024-03-29T08:07:22.718 INFO:tasks.workunit.client.0.smithi037.stdout:5/582: link d8/d22/f31 d8/fc5 0 2024-03-29T08:07:22.718 INFO:tasks.workunit.client.1.smithi148.stdout:4/615: write df/d2c/f86 [3103865,108510] 0 2024-03-29T08:07:22.719 INFO:tasks.workunit.client.0.smithi037.stdout:8/466: sync 2024-03-29T08:07:22.720 INFO:tasks.workunit.client.0.smithi037.stdout:6/390: readlink d0/d4f/l58 0 2024-03-29T08:07:22.721 INFO:tasks.workunit.client.1.smithi148.stdout:3/645: rmdir de 39 2024-03-29T08:07:22.724 INFO:tasks.workunit.client.0.smithi037.stdout:4/461: mkdir d3/d4/d15/d1d/db/d72/d98 0 2024-03-29T08:07:22.725 INFO:tasks.workunit.client.0.smithi037.stdout:6/391: dread - d0/d4f/f78 zero size 2024-03-29T08:07:22.726 INFO:tasks.workunit.client.0.smithi037.stdout:1/506: unlink d2/d6/dc/f35 0 2024-03-29T08:07:22.727 INFO:tasks.workunit.client.0.smithi037.stdout:7/467: rename dd/d21/d35/d42/d5c/f74 to dd/d21/d35/d3b/f97 0 2024-03-29T08:07:22.730 INFO:tasks.workunit.client.1.smithi148.stdout:0/617: dwrite db/de/df/d2b/f77 [4194304,4194304] 0 2024-03-29T08:07:22.731 INFO:tasks.workunit.client.0.smithi037.stdout:6/392: stat d0/d5/d1e/d44/d71 0 2024-03-29T08:07:22.731 INFO:tasks.workunit.client.0.smithi037.stdout:2/342: fsync d0/f7 0 2024-03-29T08:07:22.734 INFO:tasks.workunit.client.0.smithi037.stdout:2/343: readlink d0/d23/l2d 0 2024-03-29T08:07:22.737 INFO:tasks.workunit.client.1.smithi148.stdout:4/616: mknod df/da0/cc3 0 2024-03-29T08:07:22.740 INFO:tasks.workunit.client.1.smithi148.stdout:2/704: getdents d7/da/d33/d3c/d31/de1 0 2024-03-29T08:07:22.741 INFO:tasks.workunit.client.1.smithi148.stdout:9/654: rename db/d10/d2d/l6f to db/d10/d6d/d73/da8/ldd 0 2024-03-29T08:07:22.742 INFO:tasks.workunit.client.0.smithi037.stdout:9/693: rmdir d1/d3/ddf 0 2024-03-29T08:07:22.742 INFO:tasks.workunit.client.0.smithi037.stdout:8/467: creat d5/d10/d19/d8d/f97 x:0 0 0 2024-03-29T08:07:22.746 INFO:tasks.workunit.client.0.smithi037.stdout:0/366: creat d9/da/dd/d20/d38/d53/f71 x:0 0 0 2024-03-29T08:07:22.749 INFO:tasks.workunit.client.0.smithi037.stdout:1/507: mkdir d2/d4b/d80/db0 0 2024-03-29T08:07:22.751 INFO:tasks.workunit.client.1.smithi148.stdout:4/617: creat df/d2c/d51/d77/d26/d34/d8a/fc4 x:0 0 0 2024-03-29T08:07:22.752 INFO:tasks.workunit.client.0.smithi037.stdout:4/462: dread d3/d57/f7a [0,4194304] 0 2024-03-29T08:07:22.753 INFO:tasks.workunit.client.0.smithi037.stdout:3/455: creat dd/d15/d18/d30/f9d x:0 0 0 2024-03-29T08:07:22.755 INFO:tasks.workunit.client.1.smithi148.stdout:5/599: dwrite d0/d46/d9e/fa6 [0,4194304] 0 2024-03-29T08:07:22.759 INFO:tasks.workunit.client.0.smithi037.stdout:2/344: mkdir d0/d23/d70/d71/d78 0 2024-03-29T08:07:22.759 INFO:tasks.workunit.client.1.smithi148.stdout:9/655: getdents db/d22/d26/d54/d4d/d35/d4b/dc3/d94 0 2024-03-29T08:07:22.760 INFO:tasks.workunit.client.1.smithi148.stdout:2/705: dwrite d7/d24/f36 [0,4194304] 0 2024-03-29T08:07:22.760 INFO:tasks.workunit.client.0.smithi037.stdout:9/694: mknod d1/d4/d19/d1e/d4a/d6f/d96/cf0 0 2024-03-29T08:07:22.761 INFO:tasks.workunit.client.1.smithi148.stdout:5/600: write d0/db/d1e/d6e/f92 [95022,63755] 0 2024-03-29T08:07:22.762 INFO:tasks.workunit.client.0.smithi037.stdout:1/508: write d2/f94 [114798,106480] 0 2024-03-29T08:07:22.763 INFO:tasks.workunit.client.1.smithi148.stdout:4/618: symlink df/d2c/d80/da6/lc5 0 2024-03-29T08:07:22.764 INFO:tasks.workunit.client.0.smithi037.stdout:8/468: rename d5/d78/f8b to d5/d10/d2d/d34/d50/d83/d46/f98 0 2024-03-29T08:07:22.766 INFO:tasks.workunit.client.1.smithi148.stdout:9/656: dread db/d10/faf [0,4194304] 0 2024-03-29T08:07:22.768 INFO:tasks.workunit.client.0.smithi037.stdout:7/468: link dd/d21/d35/f6b dd/d72/f98 0 2024-03-29T08:07:22.768 INFO:tasks.workunit.client.0.smithi037.stdout:4/463: creat d3/d54/f99 x:0 0 0 2024-03-29T08:07:22.775 INFO:tasks.workunit.client.0.smithi037.stdout:2/345: chown d0/c16 11379 1 2024-03-29T08:07:22.778 INFO:tasks.workunit.client.0.smithi037.stdout:3/456: creat dd/d46/d2d/d3a/d85/f9e x:0 0 0 2024-03-29T08:07:22.779 INFO:tasks.workunit.client.1.smithi148.stdout:9/657: dread db/d10/d2d/d4e/dd4/f1b [4194304,4194304] 0 2024-03-29T08:07:22.779 INFO:tasks.workunit.client.0.smithi037.stdout:6/393: dwrite d0/d5/d1a/f5d [4194304,4194304] 0 2024-03-29T08:07:22.779 INFO:tasks.workunit.client.1.smithi148.stdout:7/575: write d8/d14/d45/d4d/f6b [1435682,63432] 0 2024-03-29T08:07:22.782 INFO:tasks.workunit.client.1.smithi148.stdout:6/634: sync 2024-03-29T08:07:22.783 INFO:tasks.workunit.client.0.smithi037.stdout:5/583: dwrite d8/d22/d4f/d70/d9c/f97 [0,4194304] 0 2024-03-29T08:07:22.784 INFO:tasks.workunit.client.0.smithi037.stdout:9/695: rmdir d1/d4/d19/d1e/d25/d16 39 2024-03-29T08:07:22.784 INFO:tasks.workunit.client.1.smithi148.stdout:7/576: mknod d8/d38/d6a/ccf 0 2024-03-29T08:07:22.785 INFO:tasks.workunit.client.0.smithi037.stdout:2/346: stat d0/d23/d44/d62 0 2024-03-29T08:07:22.786 INFO:tasks.workunit.client.1.smithi148.stdout:0/618: write db/de/df/d2b/f76 [400736,65845] 0 2024-03-29T08:07:22.788 INFO:tasks.workunit.client.0.smithi037.stdout:5/584: dread - d8/d22/d37/fb1 zero size 2024-03-29T08:07:22.789 INFO:tasks.workunit.client.0.smithi037.stdout:4/464: chown d3/d4/d15/d1d/db/d45/d1e/d59/d73/f83 26134 1 2024-03-29T08:07:22.790 INFO:tasks.workunit.client.1.smithi148.stdout:2/706: getdents d7 0 2024-03-29T08:07:22.791 INFO:tasks.workunit.client.1.smithi148.stdout:3/646: dread de/d24/d33/d39/fb6 [0,4194304] 0 2024-03-29T08:07:22.793 INFO:tasks.workunit.client.0.smithi037.stdout:6/394: symlink d0/d6a/l83 0 2024-03-29T08:07:22.795 INFO:tasks.workunit.client.1.smithi148.stdout:3/647: write de/d24/d33/d39/da3/dbc/fd0 [903661,119491] 0 2024-03-29T08:07:22.797 INFO:tasks.workunit.client.1.smithi148.stdout:6/635: chown d6/d9/d1e/d36/d4d/c88 6038187 1 2024-03-29T08:07:22.800 INFO:tasks.workunit.client.1.smithi148.stdout:5/601: getdents d0/d46/d9e 0 2024-03-29T08:07:22.800 INFO:tasks.workunit.client.1.smithi148.stdout:3/648: write de/d24/d33/d39/d46/d30/fa2 [2118604,64009] 0 2024-03-29T08:07:22.800 INFO:tasks.workunit.client.1.smithi148.stdout:5/602: readlink d0/l1a 0 2024-03-29T08:07:22.802 INFO:tasks.workunit.client.0.smithi037.stdout:7/469: creat dd/d21/d89/f99 x:0 0 0 2024-03-29T08:07:22.803 INFO:tasks.workunit.client.1.smithi148.stdout:2/707: rename d7/da/d33/d3c/d31/f4a to d7/da/d33/d45/d4d/db7/fe3 0 2024-03-29T08:07:22.805 INFO:tasks.workunit.client.1.smithi148.stdout:0/619: mkdir db/de/d24/dbf 0 2024-03-29T08:07:22.806 INFO:tasks.workunit.client.0.smithi037.stdout:2/347: dread d0/d20/d2e/f36 [0,4194304] 0 2024-03-29T08:07:22.806 INFO:tasks.workunit.client.0.smithi037.stdout:8/469: rmdir d5/d7a 0 2024-03-29T08:07:22.807 INFO:tasks.workunit.client.1.smithi148.stdout:3/649: truncate de/f17 1073095 0 2024-03-29T08:07:22.808 INFO:tasks.workunit.client.1.smithi148.stdout:4/619: truncate fa 1487944 0 2024-03-29T08:07:22.809 INFO:tasks.workunit.client.1.smithi148.stdout:3/650: chown de/d24/d81 3 1 2024-03-29T08:07:22.813 INFO:tasks.workunit.client.1.smithi148.stdout:9/658: getdents db/d22/d3f 0 2024-03-29T08:07:22.815 INFO:tasks.workunit.client.1.smithi148.stdout:2/708: unlink d7/da/d4c/d9a/dad/fde 0 2024-03-29T08:07:22.816 INFO:tasks.workunit.client.1.smithi148.stdout:4/620: mknod df/da0/cc6 0 2024-03-29T08:07:22.818 INFO:tasks.workunit.client.1.smithi148.stdout:2/709: write d7/da/d33/f35 [497607,130691] 0 2024-03-29T08:07:22.818 INFO:tasks.workunit.client.1.smithi148.stdout:3/651: chown de/d3b/c76 51532 1 2024-03-29T08:07:22.820 INFO:tasks.workunit.client.0.smithi037.stdout:1/509: write d2/d6/dc/d58/f71 [408595,22293] 0 2024-03-29T08:07:22.820 INFO:tasks.workunit.client.1.smithi148.stdout:4/621: write df/d2c/d51/d77/d26/d9b/fa8 [252352,89668] 0 2024-03-29T08:07:22.828 INFO:tasks.workunit.client.1.smithi148.stdout:2/710: write d7/da/d33/d45/d4d/d82/fa9 [211865,54509] 0 2024-03-29T08:07:22.829 INFO:tasks.workunit.client.0.smithi037.stdout:1/510: chown d2/d6/dc/f1e 15055 1 2024-03-29T08:07:22.830 INFO:tasks.workunit.client.0.smithi037.stdout:9/696: dwrite d1/d4/d19/d1e/d71/d89/dbe/fd7 [0,4194304] 0 2024-03-29T08:07:22.832 INFO:tasks.workunit.client.0.smithi037.stdout:7/470: dread dd/d1d/d49/f5a [0,4194304] 0 2024-03-29T08:07:22.833 INFO:tasks.workunit.client.1.smithi148.stdout:1/585: sync 2024-03-29T08:07:22.834 INFO:tasks.workunit.client.1.smithi148.stdout:6/636: dread d6/d9/d1e/d27/d48/d73/d7e/fbe [0,4194304] 0 2024-03-29T08:07:22.834 INFO:tasks.workunit.client.1.smithi148.stdout:6/637: dread - d6/fb0 zero size 2024-03-29T08:07:22.835 INFO:tasks.workunit.client.0.smithi037.stdout:5/585: symlink d8/d22/d37/d8d/lc6 0 2024-03-29T08:07:22.838 INFO:tasks.workunit.client.0.smithi037.stdout:0/367: dwrite d9/da/dd/f5b [0,4194304] 0 2024-03-29T08:07:22.841 INFO:tasks.workunit.client.1.smithi148.stdout:5/603: write d0/db/d1e/d35/d42/f6b [76173,43129] 0 2024-03-29T08:07:22.843 INFO:tasks.workunit.client.1.smithi148.stdout:7/577: rmdir d8/d60 39 2024-03-29T08:07:22.843 INFO:tasks.workunit.client.0.smithi037.stdout:2/348: creat d0/d23/d70/f79 x:0 0 0 2024-03-29T08:07:22.843 INFO:tasks.workunit.client.1.smithi148.stdout:9/659: mkdir db/d10/d2d/d4e/d5f/d6e/dcb/dde 0 2024-03-29T08:07:22.846 INFO:tasks.workunit.client.1.smithi148.stdout:2/711: mknod d7/da/d33/d3c/d31/ce4 0 2024-03-29T08:07:22.847 INFO:tasks.workunit.client.0.smithi037.stdout:6/395: write d0/d5/d1a/f2f [113272,122671] 0 2024-03-29T08:07:22.850 INFO:tasks.workunit.client.0.smithi037.stdout:8/470: chown d5/d10/d19/f43 87058527 1 2024-03-29T08:07:22.851 INFO:tasks.workunit.client.0.smithi037.stdout:9/697: write d1/d4/d19/d1e/d4a/d6f/da2/d13/d7c/fc3 [29566,25099] 0 2024-03-29T08:07:22.851 INFO:tasks.workunit.client.0.smithi037.stdout:1/511: rename d2/d6/d13/c9d to d2/d4b/cb1 0 2024-03-29T08:07:22.852 INFO:tasks.workunit.client.0.smithi037.stdout:8/471: stat d5/f32 0 2024-03-29T08:07:22.852 INFO:tasks.workunit.client.1.smithi148.stdout:7/578: symlink d8/d29/ld0 0 2024-03-29T08:07:22.855 INFO:tasks.workunit.client.1.smithi148.stdout:1/586: write d3/de/d1d/fa1 [1916459,64816] 0 2024-03-29T08:07:22.856 INFO:tasks.workunit.client.0.smithi037.stdout:2/349: creat d0/d53/f7a x:0 0 0 2024-03-29T08:07:22.857 INFO:tasks.workunit.client.0.smithi037.stdout:0/368: mkdir d9/da/dd/d20/d72 0 2024-03-29T08:07:22.857 INFO:tasks.workunit.client.0.smithi037.stdout:6/396: dread - d0/d19/d34/f40 zero size 2024-03-29T08:07:22.858 INFO:tasks.workunit.client.1.smithi148.stdout:2/712: symlink d7/d8c/da4/le5 0 2024-03-29T08:07:22.859 INFO:tasks.workunit.client.0.smithi037.stdout:2/350: fdatasync d0/d23/f26 0 2024-03-29T08:07:22.860 INFO:tasks.workunit.client.1.smithi148.stdout:6/638: dwrite d6/d9/d1e/d36/d4d/f7a [0,4194304] 0 2024-03-29T08:07:22.860 INFO:tasks.workunit.client.0.smithi037.stdout:4/465: getdents d3/d4/d15/d89 0 2024-03-29T08:07:22.860 INFO:tasks.workunit.client.1.smithi148.stdout:7/579: write d8/d29/d44/fab [22508,15520] 0 2024-03-29T08:07:22.868 INFO:tasks.workunit.client.0.smithi037.stdout:9/698: mknod d1/d1f/cf1 0 2024-03-29T08:07:22.868 INFO:tasks.workunit.client.0.smithi037.stdout:2/351: dread - d0/d53/f7a zero size 2024-03-29T08:07:22.868 INFO:tasks.workunit.client.0.smithi037.stdout:4/466: write d3/d4/d27/f2e [1809078,111111] 0 2024-03-29T08:07:22.869 INFO:tasks.workunit.client.1.smithi148.stdout:6/639: write d6/d9/d1e/d27/d48/f94 [1090914,59836] 0 2024-03-29T08:07:22.869 INFO:tasks.workunit.client.1.smithi148.stdout:1/587: rmdir d3/d55/d7e 39 2024-03-29T08:07:22.869 INFO:tasks.workunit.client.1.smithi148.stdout:2/713: unlink d7/da/d52/fba 0 2024-03-29T08:07:22.872 INFO:tasks.workunit.client.0.smithi037.stdout:5/586: rename d8/l5d to d8/d18/lc7 0 2024-03-29T08:07:22.873 INFO:tasks.workunit.client.1.smithi148.stdout:1/588: readlink d3/d18/d4d/d9e/l83 0 2024-03-29T08:07:22.874 INFO:tasks.workunit.client.1.smithi148.stdout:5/604: dwrite d0/db/d1e/d36/d41/f58 [0,4194304] 0 2024-03-29T08:07:22.875 INFO:tasks.workunit.client.0.smithi037.stdout:1/512: unlink d2/d6/dc/d18/d1d/d21/f55 0 2024-03-29T08:07:22.875 INFO:tasks.workunit.client.1.smithi148.stdout:2/714: chown d7/da/dcc/cd9 32 1 2024-03-29T08:07:22.879 INFO:tasks.workunit.client.0.smithi037.stdout:7/471: getdents dd/d1d/d34/d54 0 2024-03-29T08:07:22.883 INFO:tasks.workunit.client.0.smithi037.stdout:0/369: mknod d9/da/dd/d20/c73 0 2024-03-29T08:07:22.885 INFO:tasks.workunit.client.0.smithi037.stdout:0/370: readlink d9/da/d1b/d22/l52 0 2024-03-29T08:07:22.887 INFO:tasks.workunit.client.1.smithi148.stdout:2/715: dread - d7/da/d33/fd3 zero size 2024-03-29T08:07:22.887 INFO:tasks.workunit.client.1.smithi148.stdout:4/622: write df/f27 [231221,78755] 0 2024-03-29T08:07:22.889 INFO:tasks.workunit.client.0.smithi037.stdout:2/352: symlink d0/d20/d2e/d5f/d6f/d5e/l7b 0 2024-03-29T08:07:22.890 INFO:tasks.workunit.client.0.smithi037.stdout:0/371: dread - d9/da/f3c zero size 2024-03-29T08:07:22.890 INFO:tasks.workunit.client.1.smithi148.stdout:7/580: dread d8/d14/d45/d4d/d78/fbc [0,4194304] 0 2024-03-29T08:07:22.891 INFO:tasks.workunit.client.1.smithi148.stdout:0/620: dread db/de/df/d11/d2d/f36 [0,4194304] 0 2024-03-29T08:07:22.892 INFO:tasks.workunit.client.1.smithi148.stdout:3/652: dread de/d24/d33/d39/d46/f23 [0,4194304] 0 2024-03-29T08:07:22.896 INFO:tasks.workunit.client.1.smithi148.stdout:9/660: write db/d10/d2d/d4e/dd4/f1b [5230875,15475] 0 2024-03-29T08:07:22.897 INFO:tasks.workunit.client.1.smithi148.stdout:2/716: creat d7/d8c/fe6 x:0 0 0 2024-03-29T08:07:22.899 INFO:tasks.workunit.client.0.smithi037.stdout:9/699: creat d1/d4/d19/d1e/d4a/d6f/da2/d31/daa/dba/dbb/ff2 x:0 0 0 2024-03-29T08:07:22.900 INFO:tasks.workunit.client.1.smithi148.stdout:0/621: unlink db/de/df/d11/d2d/da3/cbc 0 2024-03-29T08:07:22.900 INFO:tasks.workunit.client.1.smithi148.stdout:5/605: rename d0/db/d1e/d35/d42/d67/f70 to d0/db/d1e/fc5 0 2024-03-29T08:07:22.900 INFO:tasks.workunit.client.1.smithi148.stdout:4/623: symlink df/d2c/d51/d77/d26/d9b/lc7 0 2024-03-29T08:07:22.901 INFO:tasks.workunit.client.0.smithi037.stdout:6/397: rmdir d0/d19/d5c 0 2024-03-29T08:07:22.902 INFO:tasks.workunit.client.1.smithi148.stdout:0/622: dread - db/de/df/d2b/f72 zero size 2024-03-29T08:07:22.902 INFO:tasks.workunit.client.1.smithi148.stdout:4/624: stat df/l4b 0 2024-03-29T08:07:22.902 INFO:tasks.workunit.client.1.smithi148.stdout:5/606: write d0/db/d1e/f7e [4253919,104043] 0 2024-03-29T08:07:22.903 INFO:tasks.workunit.client.1.smithi148.stdout:0/623: fdatasync db/de/f62 0 2024-03-29T08:07:22.904 INFO:tasks.workunit.client.0.smithi037.stdout:9/700: write d1/d4/d19/d69/d75/f8b [1302847,90770] 0 2024-03-29T08:07:22.905 INFO:tasks.workunit.client.1.smithi148.stdout:7/581: truncate d8/d38/f71 1133307 0 2024-03-29T08:07:22.906 INFO:tasks.workunit.client.0.smithi037.stdout:8/472: mknod d5/d10/d19/d3d/d4a/d4c/d67/d70/c99 0 2024-03-29T08:07:22.907 INFO:tasks.workunit.client.0.smithi037.stdout:5/587: dread d8/d22/f93 [0,4194304] 0 2024-03-29T08:07:22.908 INFO:tasks.workunit.client.1.smithi148.stdout:0/624: chown db/f23 939404547 1 2024-03-29T08:07:22.908 INFO:tasks.workunit.client.0.smithi037.stdout:1/513: rename d2/d6/dc/d18/d1d/d21/f47 to d2/d6/dc/d18/d1d/d5e/fb2 0 2024-03-29T08:07:22.909 INFO:tasks.workunit.client.1.smithi148.stdout:1/589: creat d3/d18/d4d/d76/d85/fb0 x:0 0 0 2024-03-29T08:07:22.911 INFO:tasks.workunit.client.0.smithi037.stdout:8/473: read d5/f16 [925320,55045] 0 2024-03-29T08:07:22.914 INFO:tasks.workunit.client.1.smithi148.stdout:7/582: mknod d8/d38/d3c/da5/db4/cd1 0 2024-03-29T08:07:22.914 INFO:tasks.workunit.client.1.smithi148.stdout:5/607: truncate d0/db/d1e/d36/d41/f7f 1127110 0 2024-03-29T08:07:22.915 INFO:tasks.workunit.client.0.smithi037.stdout:2/353: unlink d0/d20/d2e/f69 0 2024-03-29T08:07:22.916 INFO:tasks.workunit.client.1.smithi148.stdout:1/590: rename d3/f93 to d3/d18/fb1 0 2024-03-29T08:07:22.919 INFO:tasks.workunit.client.0.smithi037.stdout:4/467: getdents d3/d57/d6b 0 2024-03-29T08:07:22.920 INFO:tasks.workunit.client.1.smithi148.stdout:7/583: mknod d8/d14/d45/cd2 0 2024-03-29T08:07:22.921 INFO:tasks.workunit.client.1.smithi148.stdout:5/608: symlink d0/db/d1e/d36/d41/lc6 0 2024-03-29T08:07:22.922 INFO:tasks.workunit.client.0.smithi037.stdout:6/398: mkdir d0/d5/d1e/d84 0 2024-03-29T08:07:22.923 INFO:tasks.workunit.client.1.smithi148.stdout:4/625: dwrite df/d2c/d51/d77/d26/d9b/da7/f50 [0,4194304] 0 2024-03-29T08:07:22.925 INFO:tasks.workunit.client.0.smithi037.stdout:6/399: read d0/d6a/f70 [33971,34236] 0 2024-03-29T08:07:22.926 INFO:tasks.workunit.client.1.smithi148.stdout:1/591: rename d3/d18/d4d/d8f/f53 to d3/d18/d71/fb2 0 2024-03-29T08:07:22.927 INFO:tasks.workunit.client.1.smithi148.stdout:3/653: write de/d24/d33/d39/d46/d30/d37/f4a [4710680,121099] 0 2024-03-29T08:07:22.929 INFO:tasks.workunit.client.1.smithi148.stdout:9/661: truncate db/d10/d2d/f68 1479926 0 2024-03-29T08:07:22.930 INFO:tasks.workunit.client.1.smithi148.stdout:0/625: dwrite db/de/df/d2b/f93 [0,4194304] 0 2024-03-29T08:07:22.932 INFO:tasks.workunit.client.1.smithi148.stdout:5/609: unlink d0/db/d1e/d79/faa 0 2024-03-29T08:07:22.935 INFO:tasks.workunit.client.1.smithi148.stdout:9/662: fdatasync db/d22/da3/fcd 0 2024-03-29T08:07:22.935 INFO:tasks.workunit.client.1.smithi148.stdout:1/592: write d3/d18/fb1 [216413,37993] 0 2024-03-29T08:07:22.937 INFO:tasks.workunit.client.0.smithi037.stdout:8/474: creat d5/d10/d19/d3d/f9a x:0 0 0 2024-03-29T08:07:22.940 INFO:tasks.workunit.client.1.smithi148.stdout:7/584: link d8/d38/d3c/f96 d8/d29/d44/d50/d85/fd3 0 2024-03-29T08:07:22.941 INFO:tasks.workunit.client.1.smithi148.stdout:5/610: mkdir d0/db/d1e/d5b/d66/dc7 0 2024-03-29T08:07:22.943 INFO:tasks.workunit.client.1.smithi148.stdout:2/717: dwrite d7/d24/d4b/f55 [4194304,4194304] 0 2024-03-29T08:07:22.943 INFO:tasks.workunit.client.1.smithi148.stdout:3/654: creat de/d24/d33/d39/d46/d30/d37/d62/d63/dbf/fd2 x:0 0 0 2024-03-29T08:07:22.944 INFO:tasks.workunit.client.0.smithi037.stdout:2/354: rmdir d0/d20/d2e/d5f/d6f/d5e 39 2024-03-29T08:07:22.949 INFO:tasks.workunit.client.0.smithi037.stdout:2/355: chown d0/d20/d2e/c56 8008217 1 2024-03-29T08:07:22.951 INFO:tasks.workunit.client.0.smithi037.stdout:4/468: read d3/d4/d15/f29 [1747509,91224] 0 2024-03-29T08:07:22.953 INFO:tasks.workunit.client.1.smithi148.stdout:2/718: creat d7/d24/d4b/fe7 x:0 0 0 2024-03-29T08:07:22.955 INFO:tasks.workunit.client.1.smithi148.stdout:5/611: mkdir d0/db/d1e/d5b/d66/d72/d95/d9c/dc8 0 2024-03-29T08:07:22.958 INFO:tasks.workunit.client.1.smithi148.stdout:3/655: symlink de/d24/d33/d39/d46/ld3 0 2024-03-29T08:07:22.959 INFO:tasks.workunit.client.1.smithi148.stdout:8/642: sync 2024-03-29T08:07:22.960 INFO:tasks.workunit.client.1.smithi148.stdout:4/626: write df/d2c/d51/d77/d26/d34/d3e/f58 [1079852,98101] 0 2024-03-29T08:07:22.965 INFO:tasks.workunit.client.1.smithi148.stdout:2/719: creat d7/da/d33/d3c/da5/fe8 x:0 0 0 2024-03-29T08:07:22.965 INFO:tasks.workunit.client.1.smithi148.stdout:4/627: fdatasync df/f14 0 2024-03-29T08:07:22.968 INFO:tasks.workunit.client.1.smithi148.stdout:1/593: dwrite d3/de/d25/d35/d42/f15 [0,4194304] 0 2024-03-29T08:07:22.972 INFO:tasks.workunit.client.0.smithi037.stdout:0/372: dwrite d9/ff [0,4194304] 0 2024-03-29T08:07:22.972 INFO:tasks.workunit.client.0.smithi037.stdout:8/475: creat d5/d10/d2d/f9b x:0 0 0 2024-03-29T08:07:22.974 INFO:tasks.workunit.client.1.smithi148.stdout:3/656: readlink de/d24/d33/d39/d46/d30/d37/d62/d63/dbf/lca 0 2024-03-29T08:07:22.974 INFO:tasks.workunit.client.1.smithi148.stdout:7/585: dwrite d8/d14/da8/fc5 [0,4194304] 0 2024-03-29T08:07:22.977 INFO:tasks.workunit.client.0.smithi037.stdout:7/472: truncate dd/d21/d65/f6f 7754720 0 2024-03-29T08:07:22.978 INFO:tasks.workunit.client.1.smithi148.stdout:0/626: rename db/dc/daa/d47/d4c/d5a/fb6 to db/de/df/d11/d2d/fc0 0 2024-03-29T08:07:22.978 INFO:tasks.workunit.client.1.smithi148.stdout:2/720: unlink d7/da/d33/d3c/d54/l9e 0 2024-03-29T08:07:22.979 INFO:tasks.workunit.client.1.smithi148.stdout:9/663: dwrite db/d22/d26/d54/d4d/d35/d4b/dc3/fb9 [0,4194304] 0 2024-03-29T08:07:22.979 INFO:tasks.workunit.client.1.smithi148.stdout:7/586: readlink d8/d29/d44/d50/d85/la4 0 2024-03-29T08:07:22.983 INFO:tasks.workunit.client.0.smithi037.stdout:7/473: chown dd/d21/f36 5 1 2024-03-29T08:07:22.983 INFO:tasks.workunit.client.1.smithi148.stdout:1/594: symlink d3/d18/lb3 0 2024-03-29T08:07:22.987 INFO:tasks.workunit.client.1.smithi148.stdout:8/643: creat d17/d49/d9a/dbe/d6f/d64/fd9 x:0 0 0 2024-03-29T08:07:22.988 INFO:tasks.workunit.client.0.smithi037.stdout:4/469: dwrite d3/d4/d27/f7f [0,4194304] 0 2024-03-29T08:07:22.989 INFO:tasks.workunit.client.1.smithi148.stdout:3/657: dread de/d24/d33/d39/d71/f72 [0,4194304] 0 2024-03-29T08:07:22.990 INFO:tasks.workunit.client.1.smithi148.stdout:4/628: creat df/d2c/d51/d77/d26/d34/d43/fc8 x:0 0 0 2024-03-29T08:07:22.991 INFO:tasks.workunit.client.1.smithi148.stdout:0/627: dread db/dc/d78/f94 [0,4194304] 0 2024-03-29T08:07:22.992 INFO:tasks.workunit.client.1.smithi148.stdout:5/612: mkdir d0/db/d7b/dc9 0 2024-03-29T08:07:22.992 INFO:tasks.workunit.client.0.smithi037.stdout:9/701: dwrite d1/d4/d19/d1e/d4a/f53 [0,4194304] 0 2024-03-29T08:07:22.993 INFO:tasks.workunit.client.0.smithi037.stdout:4/470: chown d3/d4/d15/d1d/db/d45/d1e/d59/d60 3 1 2024-03-29T08:07:22.994 INFO:tasks.workunit.client.1.smithi148.stdout:4/629: stat df/da0/la4 0 2024-03-29T08:07:22.994 INFO:tasks.workunit.client.1.smithi148.stdout:7/587: creat d8/d29/fd4 x:0 0 0 2024-03-29T08:07:22.995 INFO:tasks.workunit.client.1.smithi148.stdout:1/595: symlink d3/de/d25/lb4 0 2024-03-29T08:07:23.003 INFO:tasks.workunit.client.1.smithi148.stdout:1/596: read d3/de/d25/f75 [1230701,99275] 0 2024-03-29T08:07:23.007 INFO:tasks.workunit.client.1.smithi148.stdout:3/658: creat de/d24/d33/fd4 x:0 0 0 2024-03-29T08:07:23.008 INFO:tasks.workunit.client.1.smithi148.stdout:2/721: getdents d7/d24/d4b/d66/d7d 0 2024-03-29T08:07:23.011 INFO:tasks.workunit.client.1.smithi148.stdout:6/640: sync 2024-03-29T08:07:23.012 INFO:tasks.workunit.client.1.smithi148.stdout:5/613: rename d0/db/d1e/d36/d41/c8d to d0/db/d1e/d5b/d66/dc7/cca 0 2024-03-29T08:07:23.012 INFO:tasks.workunit.client.1.smithi148.stdout:8/644: chown d17/d18/f2a 45364652 1 2024-03-29T08:07:23.013 INFO:tasks.workunit.client.0.smithi037.stdout:3/457: sync 2024-03-29T08:07:23.014 INFO:tasks.workunit.client.1.smithi148.stdout:3/659: fsync de/d24/d33/f96 0 2024-03-29T08:07:23.014 INFO:tasks.workunit.client.1.smithi148.stdout:2/722: mknod d7/d24/d4b/ce9 0 2024-03-29T08:07:23.018 INFO:tasks.workunit.client.0.smithi037.stdout:5/588: creat d8/d22/d37/dbc/fc8 x:0 0 0 2024-03-29T08:07:23.019 INFO:tasks.workunit.client.1.smithi148.stdout:6/641: creat d6/d9/d1e/d36/d4d/d9d/fce x:0 0 0 2024-03-29T08:07:23.021 INFO:tasks.workunit.client.1.smithi148.stdout:4/630: getdents df/d2c/d51/d77/d17 0 2024-03-29T08:07:23.024 INFO:tasks.workunit.client.1.smithi148.stdout:2/723: creat d7/d8c/da4/d8e/dda/fea x:0 0 0 2024-03-29T08:07:23.027 INFO:tasks.workunit.client.0.smithi037.stdout:6/400: dwrite d0/d5/d1f/f4c [0,4194304] 0 2024-03-29T08:07:23.029 INFO:tasks.workunit.client.0.smithi037.stdout:0/373: dwrite d9/da/dd/d13/f31 [4194304,4194304] 0 2024-03-29T08:07:23.031 INFO:tasks.workunit.client.1.smithi148.stdout:6/642: truncate d6/d9/d1e/d27/d48/d73/d7e/fa9 754616 0 2024-03-29T08:07:23.037 INFO:tasks.workunit.client.1.smithi148.stdout:4/631: rename df/d2c/d51/d77/d17/d6a to df/d2c/d80/daa/dbc/dc9 0 2024-03-29T08:07:23.049 INFO:tasks.workunit.client.0.smithi037.stdout:7/474: dwrite dd/d21/d35/d42/f8e [0,4194304] 0 2024-03-29T08:07:23.049 INFO:tasks.workunit.client.0.smithi037.stdout:6/401: fdatasync d0/d19/d34/d4e/f54 0 2024-03-29T08:07:23.052 INFO:tasks.workunit.client.0.smithi037.stdout:9/702: dwrite d1/d4/d19/d1e/d4a/d6f/da2/d23/f9a [0,4194304] 0 2024-03-29T08:07:23.055 INFO:tasks.workunit.client.1.smithi148.stdout:5/614: write d0/db/d1e/d35/d42/d67/d9a/f6d [1401871,46907] 0 2024-03-29T08:07:23.055 INFO:tasks.workunit.client.1.smithi148.stdout:7/588: unlink d8/d60/c79 0 2024-03-29T08:07:23.056 INFO:tasks.workunit.client.0.smithi037.stdout:8/476: mkdir d5/d10/d2d/d9c 0 2024-03-29T08:07:23.056 INFO:tasks.workunit.client.0.smithi037.stdout:6/402: write d0/d19/d34/f6b [622527,23267] 0 2024-03-29T08:07:23.058 INFO:tasks.workunit.client.1.smithi148.stdout:0/628: link db/dc/daa/d47/d4c/d5a/l60 db/de/d24/lc1 0 2024-03-29T08:07:23.060 INFO:tasks.workunit.client.0.smithi037.stdout:1/514: getdents d2/d6 0 2024-03-29T08:07:23.073 INFO:tasks.workunit.client.1.smithi148.stdout:3/660: dwrite de/d24/d33/d39/d46/d30/d37/d62/d63/dbf/fc8 [0,4194304] 0 2024-03-29T08:07:23.073 INFO:tasks.workunit.client.0.smithi037.stdout:4/471: truncate d3/d4/d15/f29 2293138 0 2024-03-29T08:07:23.077 INFO:tasks.workunit.client.1.smithi148.stdout:0/629: stat db/de/df/d11/d1f/c8c 0 2024-03-29T08:07:23.078 INFO:tasks.workunit.client.0.smithi037.stdout:3/458: mknod dd/d15/d18/d4b/c9f 0 2024-03-29T08:07:23.081 INFO:tasks.workunit.client.0.smithi037.stdout:5/589: mkdir d8/d22/d37/d60/d61/dc9 0 2024-03-29T08:07:23.082 INFO:tasks.workunit.client.0.smithi037.stdout:8/477: unlink d5/d10/l14 0 2024-03-29T08:07:23.086 INFO:tasks.workunit.client.0.smithi037.stdout:8/478: chown d5/d10/d19/d3d/d4a/d4c 1102 1 2024-03-29T08:07:23.087 INFO:tasks.workunit.client.1.smithi148.stdout:8/645: getdents d17/d1d 0 2024-03-29T08:07:23.088 INFO:tasks.workunit.client.0.smithi037.stdout:0/374: mkdir d9/d74 0 2024-03-29T08:07:23.090 INFO:tasks.workunit.client.0.smithi037.stdout:8/479: chown d5/d10/f92 29709552 1 2024-03-29T08:07:23.090 INFO:tasks.workunit.client.1.smithi148.stdout:4/632: dwrite df/d2c/d51/d77/d26/d34/d43/f59 [0,4194304] 0 2024-03-29T08:07:23.091 INFO:tasks.workunit.client.1.smithi148.stdout:5/615: dread d0/db/d1e/d36/d41/f7f [0,4194304] 0 2024-03-29T08:07:23.092 INFO:tasks.workunit.client.1.smithi148.stdout:2/724: dread d7/d24/f5d [0,4194304] 0 2024-03-29T08:07:23.092 INFO:tasks.workunit.client.1.smithi148.stdout:9/664: dread db/d22/d26/d54/d4d/f18 [0,4194304] 0 2024-03-29T08:07:23.092 INFO:tasks.workunit.client.0.smithi037.stdout:7/475: symlink dd/d21/d35/l9a 0 2024-03-29T08:07:23.092 INFO:tasks.workunit.client.0.smithi037.stdout:6/403: unlink d0/d20/l76 0 2024-03-29T08:07:23.095 INFO:tasks.workunit.client.1.smithi148.stdout:4/633: chown df/f27 162 1 2024-03-29T08:07:23.096 INFO:tasks.workunit.client.0.smithi037.stdout:7/476: fsync dd/d1d/f2a 0 2024-03-29T08:07:23.096 INFO:tasks.workunit.client.1.smithi148.stdout:0/630: rmdir db/de/df/d11/d1f 39 2024-03-29T08:07:23.100 INFO:tasks.workunit.client.1.smithi148.stdout:2/725: read d7/da/d4c/f79 [284924,104226] 0 2024-03-29T08:07:23.101 INFO:tasks.workunit.client.0.smithi037.stdout:8/480: write d5/d10/d19/d3d/d4a/d4c/d67/d85/f8e [180303,11022] 0 2024-03-29T08:07:23.102 INFO:tasks.workunit.client.1.smithi148.stdout:1/597: dread d3/de/f1f [0,4194304] 0 2024-03-29T08:07:23.103 INFO:tasks.workunit.client.1.smithi148.stdout:5/616: rmdir d0/db/d84/daf 39 2024-03-29T08:07:23.108 INFO:tasks.workunit.client.1.smithi148.stdout:3/661: dread de/d24/d33/d39/da3/dbc/fd0 [0,4194304] 0 2024-03-29T08:07:23.108 INFO:tasks.workunit.client.1.smithi148.stdout:2/726: read d7/da/f3f [3768797,66469] 0 2024-03-29T08:07:23.109 INFO:tasks.workunit.client.1.smithi148.stdout:4/634: truncate df/d2c/d51/d77/f53 1257938 0 2024-03-29T08:07:23.111 INFO:tasks.workunit.client.1.smithi148.stdout:5/617: mknod d0/ccb 0 2024-03-29T08:07:23.112 INFO:tasks.workunit.client.1.smithi148.stdout:6/643: dread d6/d9/d1e/d36/d4d/d9d/d28/f74 [4194304,4194304] 0 2024-03-29T08:07:23.114 INFO:tasks.workunit.client.1.smithi148.stdout:1/598: dread d3/de/d21/d5e/fa6 [0,4194304] 0 2024-03-29T08:07:23.114 INFO:tasks.workunit.client.1.smithi148.stdout:3/662: mknod de/d24/d33/d39/cd5 0 2024-03-29T08:07:23.116 INFO:tasks.workunit.client.1.smithi148.stdout:2/727: mknod d7/d24/d4b/d66/ceb 0 2024-03-29T08:07:23.125 INFO:tasks.workunit.client.0.smithi037.stdout:3/459: rmdir dd/d46/d3e/d64 0 2024-03-29T08:07:23.128 INFO:tasks.workunit.client.1.smithi148.stdout:6/644: symlink d6/d9/d1e/d27/d48/d73/lcf 0 2024-03-29T08:07:23.128 INFO:tasks.workunit.client.0.smithi037.stdout:3/460: fsync dd/d46/d2d/d3a/d5f/f77 0 2024-03-29T08:07:23.129 INFO:tasks.workunit.client.1.smithi148.stdout:3/663: symlink de/d24/d33/d39/da3/dbc/ld6 0 2024-03-29T08:07:23.131 INFO:tasks.workunit.client.1.smithi148.stdout:5/618: mknod d0/db/d1e/d36/d41/ccc 0 2024-03-29T08:07:23.134 INFO:tasks.workunit.client.1.smithi148.stdout:4/635: getdents df/d2c/d80/daa/dbc 0 2024-03-29T08:07:23.135 INFO:tasks.workunit.client.0.smithi037.stdout:9/703: dread d1/d4/d19/d1e/d4a/d6f/da2/d31/fac [0,4194304] 0 2024-03-29T08:07:23.135 INFO:tasks.workunit.client.1.smithi148.stdout:5/619: stat d0/db/d1e/d5b 0 2024-03-29T08:07:23.135 INFO:tasks.workunit.client.1.smithi148.stdout:0/631: dwrite db/de/df/d2b/f72 [0,4194304] 0 2024-03-29T08:07:23.136 INFO:tasks.workunit.client.0.smithi037.stdout:5/590: write d8/d22/d37/d60/f76 [296498,88674] 0 2024-03-29T08:07:23.138 INFO:tasks.workunit.client.1.smithi148.stdout:9/665: dwrite db/d22/d26/d54/d4d/d35/d4b/d5a/d74/fba [0,4194304] 0 2024-03-29T08:07:23.138 INFO:tasks.workunit.client.1.smithi148.stdout:8/646: creat d17/d49/d9a/dbe/d6f/d64/da5/fda x:0 0 0 2024-03-29T08:07:23.139 INFO:tasks.workunit.client.0.smithi037.stdout:5/591: read - d8/d4d/d5c/fc3 zero size 2024-03-29T08:07:23.139 INFO:tasks.workunit.client.0.smithi037.stdout:1/515: truncate d2/d6/d13/f77 1243566 0 2024-03-29T08:07:23.140 INFO:tasks.workunit.client.1.smithi148.stdout:2/728: rename d7/d24/d4b/d66/d7d to d7/da/d33/d3c/d54/dec 0 2024-03-29T08:07:23.142 INFO:tasks.workunit.client.0.smithi037.stdout:5/592: chown d8/f73 707141900 1 2024-03-29T08:07:23.142 INFO:tasks.workunit.client.1.smithi148.stdout:0/632: readlink db/de/df/d14/l1d 0 2024-03-29T08:07:23.144 INFO:tasks.workunit.client.1.smithi148.stdout:3/664: mknod de/d24/d33/d39/da3/dbc/cd7 0 2024-03-29T08:07:23.145 INFO:tasks.workunit.client.0.smithi037.stdout:0/375: write d9/da/dd/d13/f45 [379903,80238] 0 2024-03-29T08:07:23.145 INFO:tasks.workunit.client.1.smithi148.stdout:8/647: chown d17/d18/d26/c31 1537 1 2024-03-29T08:07:23.151 INFO:tasks.workunit.client.0.smithi037.stdout:4/472: truncate d3/d4/d15/d1d/f3f 222874 0 2024-03-29T08:07:23.152 INFO:tasks.workunit.client.0.smithi037.stdout:6/404: getdents d0/d19/d34/d4e/d69/d79 0 2024-03-29T08:07:23.152 INFO:tasks.workunit.client.1.smithi148.stdout:9/666: dread - db/d10/f67 zero size 2024-03-29T08:07:23.152 INFO:tasks.workunit.client.1.smithi148.stdout:7/589: dread d8/d29/d44/d49/f68 [4194304,4194304] 0 2024-03-29T08:07:23.154 INFO:tasks.workunit.client.0.smithi037.stdout:9/704: mkdir d1/d4/d19/d69/df3 0 2024-03-29T08:07:23.155 INFO:tasks.workunit.client.1.smithi148.stdout:1/599: dwrite d3/de/d1d/fa1 [0,4194304] 0 2024-03-29T08:07:23.157 INFO:tasks.workunit.client.0.smithi037.stdout:7/477: write dd/d21/d35/f6b [29813,87815] 0 2024-03-29T08:07:23.159 INFO:tasks.workunit.client.0.smithi037.stdout:5/593: stat d8/d22/lb4 0 2024-03-29T08:07:23.161 INFO:tasks.workunit.client.1.smithi148.stdout:6/645: link d6/d9/c18 d6/d9/d1e/d20/d30/cd0 0 2024-03-29T08:07:23.164 INFO:tasks.workunit.client.0.smithi037.stdout:0/376: rename d9/da/dd/d59/c5d to d9/da/d1b/d22/c75 0 2024-03-29T08:07:23.164 INFO:tasks.workunit.client.1.smithi148.stdout:2/729: truncate d7/da/d33/d45/d4d/d41/fcb 673357 0 2024-03-29T08:07:23.164 INFO:tasks.workunit.client.1.smithi148.stdout:1/600: stat d3/d18/d4d 0 2024-03-29T08:07:23.165 INFO:tasks.workunit.client.1.smithi148.stdout:3/665: mknod de/d24/cd8 0 2024-03-29T08:07:23.165 INFO:tasks.workunit.client.0.smithi037.stdout:2/356: dwrite d0/f18 [0,4194304] 0 2024-03-29T08:07:23.166 INFO:tasks.workunit.client.1.smithi148.stdout:0/633: getdents db/de/df/d11/d2d/d3e 0 2024-03-29T08:07:23.166 INFO:tasks.workunit.client.0.smithi037.stdout:7/478: truncate dd/d21/d35/d60/f7f 575327 0 2024-03-29T08:07:23.168 INFO:tasks.workunit.client.1.smithi148.stdout:1/601: fdatasync d3/d55/d7e/fad 0 2024-03-29T08:07:23.170 INFO:tasks.workunit.client.0.smithi037.stdout:9/705: chown d1/d1f/d3a/f27 1802 1 2024-03-29T08:07:23.170 INFO:tasks.workunit.client.1.smithi148.stdout:9/667: mkdir db/d22/d26/d54/d4d/d35/d4b/d8e/ddf 0 2024-03-29T08:07:23.171 INFO:tasks.workunit.client.1.smithi148.stdout:7/590: creat d8/d38/d3c/da5/fd5 x:0 0 0 2024-03-29T08:07:23.172 INFO:tasks.workunit.client.0.smithi037.stdout:6/405: unlink d0/d5/f7a 0 2024-03-29T08:07:23.174 INFO:tasks.workunit.client.1.smithi148.stdout:7/591: chown d8/d38/d3c/l46 6 1 2024-03-29T08:07:23.174 INFO:tasks.workunit.client.1.smithi148.stdout:3/666: symlink de/d24/d33/d39/d46/d30/d37/d62/d63/dbf/ld9 0 2024-03-29T08:07:23.174 INFO:tasks.workunit.client.1.smithi148.stdout:8/648: rename d17/d49/d9a/dbe/c96 to d17/d49/d9a/dbe/cdb 0 2024-03-29T08:07:23.178 INFO:tasks.workunit.client.0.smithi037.stdout:1/516: rmdir d2/d6/dc/d18/d1d/d21/d99 39 2024-03-29T08:07:23.178 INFO:tasks.workunit.client.1.smithi148.stdout:3/667: write de/d24/d33/d39/d46/f65 [203059,29440] 0 2024-03-29T08:07:23.181 INFO:tasks.workunit.client.1.smithi148.stdout:5/620: dwrite d0/db/d7b/d16/f2f [0,4194304] 0 2024-03-29T08:07:23.181 INFO:tasks.workunit.client.0.smithi037.stdout:9/706: chown d1/d4/d19/d1e/d71/d89/d92 1 1 2024-03-29T08:07:23.181 INFO:tasks.workunit.client.1.smithi148.stdout:0/634: unlink db/de/df/d11/d2d/d3e/l8f 0 2024-03-29T08:07:23.185 INFO:tasks.workunit.client.1.smithi148.stdout:8/649: creat d17/d4e/fdc x:0 0 0 2024-03-29T08:07:23.186 INFO:tasks.workunit.client.0.smithi037.stdout:0/377: creat d9/da/dd/f76 x:0 0 0 2024-03-29T08:07:23.186 INFO:tasks.workunit.client.1.smithi148.stdout:7/592: mknod d8/d38/d6a/d91/cd6 0 2024-03-29T08:07:23.189 INFO:tasks.workunit.client.0.smithi037.stdout:7/479: mkdir dd/d21/d35/d42/d5c/d9b 0 2024-03-29T08:07:23.190 INFO:tasks.workunit.client.0.smithi037.stdout:1/517: fdatasync d2/f84 0 2024-03-29T08:07:23.190 INFO:tasks.workunit.client.1.smithi148.stdout:0/635: dread db/de/df/d2b/dae/fbd [0,4194304] 0 2024-03-29T08:07:23.192 INFO:tasks.workunit.client.1.smithi148.stdout:2/730: rename d7/da/d33/d45/d4d to d7/da/d33/ded 0 2024-03-29T08:07:23.193 INFO:tasks.workunit.client.0.smithi037.stdout:3/461: dwrite dd/d5e/d7a/f4f [0,4194304] 0 2024-03-29T08:07:23.197 INFO:tasks.workunit.client.0.smithi037.stdout:2/357: creat d0/d23/d70/f7c x:0 0 0 2024-03-29T08:07:23.197 INFO:tasks.workunit.client.0.smithi037.stdout:5/594: creat d8/fca x:0 0 0 2024-03-29T08:07:23.201 INFO:tasks.workunit.client.0.smithi037.stdout:1/518: mknod d2/d4b/da1/cb3 0 2024-03-29T08:07:23.202 INFO:tasks.workunit.client.1.smithi148.stdout:8/650: creat d17/d74/d86/fdd x:0 0 0 2024-03-29T08:07:23.203 INFO:tasks.workunit.client.1.smithi148.stdout:3/668: symlink de/lda 0 2024-03-29T08:07:23.205 INFO:tasks.workunit.client.0.smithi037.stdout:7/480: unlink dd/d1d/d34/d54/l8b 0 2024-03-29T08:07:23.208 INFO:tasks.workunit.client.1.smithi148.stdout:0/636: dread db/dc/f1e [0,4194304] 0 2024-03-29T08:07:23.209 INFO:tasks.workunit.client.0.smithi037.stdout:5/595: fdatasync d8/d22/d4f/d70/d9c/f97 0 2024-03-29T08:07:23.211 INFO:tasks.workunit.client.1.smithi148.stdout:1/602: getdents d3/d55/d7e 0 2024-03-29T08:07:23.215 INFO:tasks.workunit.client.0.smithi037.stdout:4/473: dread d3/d4/d15/f49 [4194304,4194304] 0 2024-03-29T08:07:23.217 INFO:tasks.workunit.client.1.smithi148.stdout:2/731: dread d7/da/d33/d3c/d54/dec/f8f [0,4194304] 0 2024-03-29T08:07:23.217 INFO:tasks.workunit.client.1.smithi148.stdout:0/637: fsync db/de/df/d2b/f72 0 2024-03-29T08:07:23.219 INFO:tasks.workunit.client.1.smithi148.stdout:5/621: link d0/db/d1e/d35/l3c d0/db/d1e/d5b/d66/d72/d95/d9c/lcd 0 2024-03-29T08:07:23.224 INFO:tasks.workunit.client.1.smithi148.stdout:6/646: rename d6/d9/d1e/d20/d6b/ca8 to d6/d9/d1e/d20/cd1 0 2024-03-29T08:07:23.224 INFO:tasks.workunit.client.0.smithi037.stdout:9/707: dwrite d1/d4/d19/d1e/d25/fcb [0,4194304] 0 2024-03-29T08:07:23.229 INFO:tasks.workunit.client.1.smithi148.stdout:2/732: mknod d7/da/d4c/dc4/cee 0 2024-03-29T08:07:23.229 INFO:tasks.workunit.client.1.smithi148.stdout:7/593: dwrite d8/d38/d3c/da5/fae [0,4194304] 0 2024-03-29T08:07:23.231 INFO:tasks.workunit.client.1.smithi148.stdout:1/603: creat d3/d55/d7e/fb5 x:0 0 0 2024-03-29T08:07:23.231 INFO:tasks.workunit.client.1.smithi148.stdout:5/622: rmdir d0/db/d7b 39 2024-03-29T08:07:23.231 INFO:tasks.workunit.client.0.smithi037.stdout:4/474: chown d3/d4/d15/d1d/db/d45/d1e/d59/c5c 147823 1 2024-03-29T08:07:23.232 INFO:tasks.workunit.client.1.smithi148.stdout:4/636: dwrite df/d2c/d51/d77/d26/d34/d3e/f3f [0,4194304] 0 2024-03-29T08:07:23.233 INFO:tasks.workunit.client.0.smithi037.stdout:4/475: chown d3/d4/d15/f50 393323674 1 2024-03-29T08:07:23.236 INFO:tasks.workunit.client.0.smithi037.stdout:0/378: truncate d9/da/dd/f76 164428 0 2024-03-29T08:07:23.236 INFO:tasks.workunit.client.0.smithi037.stdout:5/596: unlink d8/d22/d37/d60/c7f 0 2024-03-29T08:07:23.236 INFO:tasks.workunit.client.1.smithi148.stdout:9/668: rename db to db/d22/de0 22 2024-03-29T08:07:23.241 INFO:tasks.workunit.client.0.smithi037.stdout:9/708: rename d1/d4/d19/fe7 to d1/d4/d19/d69/ff4 0 2024-03-29T08:07:23.243 INFO:tasks.workunit.client.0.smithi037.stdout:3/462: getdents dd/d15 0 2024-03-29T08:07:23.247 INFO:tasks.workunit.client.1.smithi148.stdout:3/669: getdents de/d24/d33/d39/d46/d8d 0 2024-03-29T08:07:23.249 INFO:tasks.workunit.client.1.smithi148.stdout:8/651: dread d17/d49/d9a/dbe/d6f/f5b [0,4194304] 0 2024-03-29T08:07:23.249 INFO:tasks.workunit.client.1.smithi148.stdout:0/638: write db/de/df/d11/f16 [2823488,12812] 0 2024-03-29T08:07:23.251 INFO:tasks.workunit.client.1.smithi148.stdout:7/594: chown d8/d38/d6a/d91/cd6 366 1 2024-03-29T08:07:23.253 INFO:tasks.workunit.client.1.smithi148.stdout:4/637: write df/d2c/d51/d77/d17/f8f [781589,2129] 0 2024-03-29T08:07:23.257 INFO:tasks.workunit.client.0.smithi037.stdout:3/463: rmdir dd/d5e 39 2024-03-29T08:07:23.260 INFO:tasks.workunit.client.1.smithi148.stdout:5/623: symlink d0/db/lce 0 2024-03-29T08:07:23.260 INFO:tasks.workunit.client.0.smithi037.stdout:6/406: dwrite d0/f37 [0,4194304] 0 2024-03-29T08:07:23.264 INFO:tasks.workunit.client.1.smithi148.stdout:9/669: dread db/d22/d26/d54/d4d/f18 [4194304,4194304] 0 2024-03-29T08:07:23.265 INFO:tasks.workunit.client.0.smithi037.stdout:3/464: dread - dd/d15/d27/f89 zero size 2024-03-29T08:07:23.265 INFO:tasks.workunit.client.0.smithi037.stdout:0/379: mknod d9/da/dd/d59/c77 0 2024-03-29T08:07:23.266 INFO:tasks.workunit.client.1.smithi148.stdout:3/670: stat de/d24/d33/d39/d46/d30/d37/fba 0 2024-03-29T08:07:23.267 INFO:tasks.workunit.client.0.smithi037.stdout:6/407: fdatasync d0/d4f/f78 0 2024-03-29T08:07:23.267 INFO:tasks.workunit.client.0.smithi037.stdout:9/709: mkdir d1/d1f/df5 0 2024-03-29T08:07:23.268 INFO:tasks.workunit.client.1.smithi148.stdout:8/652: mkdir d17/d18/d26/d4f/dde 0 2024-03-29T08:07:23.269 INFO:tasks.workunit.client.0.smithi037.stdout:0/380: stat d9/fb 0 2024-03-29T08:07:23.271 INFO:tasks.workunit.client.1.smithi148.stdout:6/647: truncate d6/d9/d1e/d36/d4d/d9d/f34 6981475 0 2024-03-29T08:07:23.273 INFO:tasks.workunit.client.0.smithi037.stdout:5/597: dwrite d8/d34/fba [0,4194304] 0 2024-03-29T08:07:23.273 INFO:tasks.workunit.client.1.smithi148.stdout:3/671: stat de/d24/d33/d39/d46/f23 0 2024-03-29T08:07:23.276 INFO:tasks.workunit.client.0.smithi037.stdout:6/408: write d0/d5/d3f/f6f [246467,33051] 0 2024-03-29T08:07:23.277 INFO:tasks.workunit.client.0.smithi037.stdout:3/465: symlink dd/d15/d27/la0 0 2024-03-29T08:07:23.279 INFO:tasks.workunit.client.0.smithi037.stdout:1/519: dwrite d2/d6/dc/d18/d1d/d21/f2a [0,4194304] 0 2024-03-29T08:07:23.281 INFO:tasks.workunit.client.0.smithi037.stdout:3/466: stat dd/d15/d18/d30/f9d 0 2024-03-29T08:07:23.284 INFO:tasks.workunit.client.1.smithi148.stdout:1/604: dwrite d3/de/d1d/d47/d6d/d84/f92 [0,4194304] 0 2024-03-29T08:07:23.287 INFO:tasks.workunit.client.0.smithi037.stdout:1/520: dread - d2/d6/f7d zero size 2024-03-29T08:07:23.290 INFO:tasks.workunit.client.0.smithi037.stdout:6/409: fsync d0/d5/d1e/f67 0 2024-03-29T08:07:23.294 INFO:tasks.workunit.client.0.smithi037.stdout:5/598: truncate d8/fc5 2172704 0 2024-03-29T08:07:23.294 INFO:tasks.workunit.client.1.smithi148.stdout:0/639: dread db/de/df/d11/f16 [0,4194304] 0 2024-03-29T08:07:23.295 INFO:tasks.workunit.client.1.smithi148.stdout:8/653: mkdir d17/d18/ddf 0 2024-03-29T08:07:23.297 INFO:tasks.workunit.client.1.smithi148.stdout:3/672: rename de/d3b/l66 to de/d24/d33/d39/d46/d8d/ldb 0 2024-03-29T08:07:23.297 INFO:tasks.workunit.client.0.smithi037.stdout:1/521: mknod d2/d6/dc/d18/d1d/d5e/cb4 0 2024-03-29T08:07:23.301 INFO:tasks.workunit.client.1.smithi148.stdout:7/595: dwrite d8/d29/d44/d49/f82 [0,4194304] 0 2024-03-29T08:07:23.302 INFO:tasks.workunit.client.0.smithi037.stdout:1/522: write d2/d6/f7d [31217,2969] 0 2024-03-29T08:07:23.302 INFO:tasks.workunit.client.0.smithi037.stdout:3/467: getdents dd/d15/d27/d33 0 2024-03-29T08:07:23.302 INFO:tasks.workunit.client.1.smithi148.stdout:5/624: getdents d0/d46 0 2024-03-29T08:07:23.303 INFO:tasks.workunit.client.1.smithi148.stdout:1/605: symlink d3/de/d1d/d47/lb6 0 2024-03-29T08:07:23.308 INFO:tasks.workunit.client.1.smithi148.stdout:0/640: fsync db/de/df/d11/d42/f6e 0 2024-03-29T08:07:23.309 INFO:tasks.workunit.client.0.smithi037.stdout:1/523: fsync d2/f3a 0 2024-03-29T08:07:23.309 INFO:tasks.workunit.client.1.smithi148.stdout:8/654: unlink d17/d49/d9a/l9d 0 2024-03-29T08:07:23.314 INFO:tasks.workunit.client.1.smithi148.stdout:2/733: dread d7/da/d33/ded/d41/f6a [0,4194304] 0 2024-03-29T08:07:23.315 INFO:tasks.workunit.client.1.smithi148.stdout:8/655: fdatasync d17/d18/f36 0 2024-03-29T08:07:23.319 INFO:tasks.workunit.client.1.smithi148.stdout:4/638: dread df/d2c/f86 [0,4194304] 0 2024-03-29T08:07:23.322 INFO:tasks.workunit.client.0.smithi037.stdout:1/524: symlink d2/d6/dc/d5d/d62/lb5 0 2024-03-29T08:07:23.324 INFO:tasks.workunit.client.1.smithi148.stdout:5/625: chown d0/db/d7b/d16/d2e 52 1 2024-03-29T08:07:23.325 INFO:tasks.workunit.client.1.smithi148.stdout:0/641: mkdir db/de/df/d11/d42/dc2 0 2024-03-29T08:07:23.325 INFO:tasks.workunit.client.0.smithi037.stdout:6/410: dwrite d0/d19/d34/d5a/f39 [4194304,4194304] 0 2024-03-29T08:07:23.327 INFO:tasks.workunit.client.1.smithi148.stdout:7/596: rename d8/d29/d44/f93 to d8/d38/d67/fd7 0 2024-03-29T08:07:23.327 INFO:tasks.workunit.client.0.smithi037.stdout:1/525: dread - d2/d6/dc/d18/d22/d4f/f73 zero size 2024-03-29T08:07:23.328 INFO:tasks.workunit.client.1.smithi148.stdout:8/656: dread - d17/d74/f4b zero size 2024-03-29T08:07:23.330 INFO:tasks.workunit.client.0.smithi037.stdout:4/476: dread d3/d4/d15/d1d/db/d3d/f58 [0,4194304] 0 2024-03-29T08:07:23.331 INFO:tasks.workunit.client.1.smithi148.stdout:2/734: creat d7/d8c/da4/d8e/fef x:0 0 0 2024-03-29T08:07:23.334 INFO:tasks.workunit.client.1.smithi148.stdout:1/606: symlink d3/lb7 0 2024-03-29T08:07:23.338 INFO:tasks.workunit.client.1.smithi148.stdout:5/626: mknod d0/db/d1e/d5b/d66/dc7/ccf 0 2024-03-29T08:07:23.340 INFO:tasks.workunit.client.0.smithi037.stdout:6/411: dread d0/d19/d34/f6b [0,4194304] 0 2024-03-29T08:07:23.341 INFO:tasks.workunit.client.1.smithi148.stdout:6/648: dwrite d6/fe [0,4194304] 0 2024-03-29T08:07:23.343 INFO:tasks.workunit.client.1.smithi148.stdout:2/735: stat d7/da/d33/d3c/d54/d70/ld8 0 2024-03-29T08:07:23.345 INFO:tasks.workunit.client.1.smithi148.stdout:3/673: dwrite de/d3b/d49/f64 [0,4194304] 0 2024-03-29T08:07:23.348 INFO:tasks.workunit.client.1.smithi148.stdout:0/642: rename db/dc/daa/d47/d4c/d5a/f6a to db/de/fc3 0 2024-03-29T08:07:23.352 INFO:tasks.workunit.client.1.smithi148.stdout:9/670: dread db/d22/d26/d54/d4d/d35/d4b/d58/f92 [0,4194304] 0 2024-03-29T08:07:23.355 INFO:tasks.workunit.client.1.smithi148.stdout:5/627: symlink d0/db/ld0 0 2024-03-29T08:07:23.356 INFO:tasks.workunit.client.0.smithi037.stdout:6/412: dread d0/f37 [4194304,4194304] 0 2024-03-29T08:07:23.359 INFO:tasks.workunit.client.1.smithi148.stdout:4/639: write fa [1328712,21612] 0 2024-03-29T08:07:23.367 INFO:tasks.workunit.client.0.smithi037.stdout:6/413: creat d0/d6a/f85 x:0 0 0 2024-03-29T08:07:23.372 INFO:tasks.workunit.client.1.smithi148.stdout:4/640: fdatasync df/d2c/d80/fb5 0 2024-03-29T08:07:23.375 INFO:tasks.workunit.client.1.smithi148.stdout:5/628: mkdir d0/db/d1e/d5b/d66/d72/d95/dd1 0 2024-03-29T08:07:23.375 INFO:tasks.workunit.client.0.smithi037.stdout:1/526: link d2/d6/dc/d18/d1d/dad/l6d d2/d6/dc/d18/d22/d4f/lb6 0 2024-03-29T08:07:23.378 INFO:tasks.workunit.client.0.smithi037.stdout:6/414: mknod d0/d6a/d7b/d77/c86 0 2024-03-29T08:07:23.379 INFO:tasks.workunit.client.1.smithi148.stdout:9/671: truncate db/d22/d26/d54/d4d/f18 3148971 0 2024-03-29T08:07:23.380 INFO:tasks.workunit.client.1.smithi148.stdout:7/597: getdents d8/d38/d3c/da5/db4 0 2024-03-29T08:07:23.380 INFO:tasks.workunit.client.0.smithi037.stdout:1/527: unlink d2/d6/dc/d18/d1d/d21/f2a 0 2024-03-29T08:07:23.380 INFO:tasks.workunit.client.1.smithi148.stdout:1/607: dwrite d3/de/d1d/d47/d6d/d84/f92 [0,4194304] 0 2024-03-29T08:07:23.382 INFO:tasks.workunit.client.1.smithi148.stdout:4/641: stat df/d2c/d51/d77/d26/d9b/da7/d7c 0 2024-03-29T08:07:23.382 INFO:tasks.workunit.client.0.smithi037.stdout:6/415: creat d0/d5/d1e/d84/f87 x:0 0 0 2024-03-29T08:07:23.383 INFO:tasks.workunit.client.0.smithi037.stdout:6/416: readlink d0/d20/l56 0 2024-03-29T08:07:23.388 INFO:tasks.workunit.client.0.smithi037.stdout:1/528: creat d2/d6/dc/d18/d1d/d21/fb7 x:0 0 0 2024-03-29T08:07:23.389 INFO:tasks.workunit.client.1.smithi148.stdout:7/598: fdatasync d8/d29/f2e 0 2024-03-29T08:07:23.390 INFO:tasks.workunit.client.0.smithi037.stdout:6/417: unlink d0/d5/d1e/d84/f87 0 2024-03-29T08:07:23.390 INFO:tasks.workunit.client.0.smithi037.stdout:1/529: read - d2/d40/f78 zero size 2024-03-29T08:07:23.392 INFO:tasks.workunit.client.1.smithi148.stdout:1/608: mkdir d3/d18/d4d/d9e/db8 0 2024-03-29T08:07:23.392 INFO:tasks.workunit.client.1.smithi148.stdout:2/736: dread d7/da/d33/ded/f57 [0,4194304] 0 2024-03-29T08:07:23.392 INFO:tasks.workunit.client.1.smithi148.stdout:8/657: dwrite d17/d49/d9a/dbe/d6f/f5b [0,4194304] 0 2024-03-29T08:07:23.392 INFO:tasks.workunit.client.1.smithi148.stdout:9/672: mknod db/d10/d2d/d4e/d5f/d6e/dcb/ce1 0 2024-03-29T08:07:23.394 INFO:tasks.workunit.client.1.smithi148.stdout:7/599: write d8/d38/d3c/da9/d37/faf [465337,127920] 0 2024-03-29T08:07:23.394 INFO:tasks.workunit.client.0.smithi037.stdout:1/530: chown d2/d6/dc/d18/d1d/d5e/cb4 738549 1 2024-03-29T08:07:23.396 INFO:tasks.workunit.client.1.smithi148.stdout:4/642: rename df/d2c/d80/l8e to df/d2c/d51/d77/d26/d34/db3/lca 0 2024-03-29T08:07:23.397 INFO:tasks.workunit.client.0.smithi037.stdout:1/531: chown d2/d6/dc/d58/fa2 161928 1 2024-03-29T08:07:23.397 INFO:tasks.workunit.client.1.smithi148.stdout:1/609: fsync d3/de/d1d/fa1 0 2024-03-29T08:07:23.398 INFO:tasks.workunit.client.1.smithi148.stdout:2/737: unlink d7/f19 0 2024-03-29T08:07:23.399 INFO:tasks.workunit.client.1.smithi148.stdout:9/673: stat db/d22/d26/d54/d4d/dbb/dc0/dca 0 2024-03-29T08:07:23.401 INFO:tasks.workunit.client.0.smithi037.stdout:8/481: sync 2024-03-29T08:07:23.404 INFO:tasks.workunit.client.1.smithi148.stdout:9/674: readlink db/d10/d2d/d4e/d5f/d6e/db0/ld3 0 2024-03-29T08:07:23.405 INFO:tasks.workunit.client.1.smithi148.stdout:7/600: chown d8/d38/d3c/da5/db4/fc6 2675 1 2024-03-29T08:07:23.407 INFO:tasks.workunit.client.0.smithi037.stdout:4/477: dwrite d3/d4/d15/d1d/db/d45/d1e/f4e [0,4194304] 0 2024-03-29T08:07:23.408 INFO:tasks.workunit.client.1.smithi148.stdout:3/674: dread de/d24/f36 [0,4194304] 0 2024-03-29T08:07:23.412 INFO:tasks.workunit.client.0.smithi037.stdout:1/532: unlink d2/d6/dc/d18/d1d/d21/d99/f56 0 2024-03-29T08:07:23.412 INFO:tasks.workunit.client.1.smithi148.stdout:1/610: creat d3/de/d1d/fb9 x:0 0 0 2024-03-29T08:07:23.413 INFO:tasks.workunit.client.1.smithi148.stdout:4/643: creat df/d2c/d80/daa/fcb x:0 0 0 2024-03-29T08:07:23.414 INFO:tasks.workunit.client.1.smithi148.stdout:2/738: creat d7/d24/d4b/d66/ff0 x:0 0 0 2024-03-29T08:07:23.415 INFO:tasks.workunit.client.1.smithi148.stdout:9/675: mknod db/d22/d26/d54/d52/dab/ce2 0 2024-03-29T08:07:23.417 INFO:tasks.workunit.client.0.smithi037.stdout:8/482: link d5/d10/d19/f43 d5/d10/d2d/d34/d50/d83/f9d 0 2024-03-29T08:07:23.417 INFO:tasks.workunit.client.1.smithi148.stdout:3/675: symlink de/d24/d33/d39/d46/d70/ldc 0 2024-03-29T08:07:23.418 INFO:tasks.workunit.client.0.smithi037.stdout:4/478: creat d3/d54/d88/f9a x:0 0 0 2024-03-29T08:07:23.419 INFO:tasks.workunit.client.1.smithi148.stdout:3/676: stat de/f15 0 2024-03-29T08:07:23.421 INFO:tasks.workunit.client.0.smithi037.stdout:8/483: symlink d5/d10/d19/l9e 0 2024-03-29T08:07:23.424 INFO:tasks.workunit.client.1.smithi148.stdout:7/601: dread d8/d38/d3c/da9/db7/fc4 [0,4194304] 0 2024-03-29T08:07:23.427 INFO:tasks.workunit.client.1.smithi148.stdout:9/676: fdatasync db/d10/d2d/d4e/da6/fac 0 2024-03-29T08:07:23.427 INFO:tasks.workunit.client.0.smithi037.stdout:4/479: creat d3/d4/d15/d1d/db/d45/d1e/d59/d79/d95/f9b x:0 0 0 2024-03-29T08:07:23.429 INFO:tasks.workunit.client.1.smithi148.stdout:1/611: creat d3/de/d1d/d47/d6d/d72/fba x:0 0 0 2024-03-29T08:07:23.432 INFO:tasks.workunit.client.1.smithi148.stdout:0/643: dwrite db/de/df/d2b/f34 [0,4194304] 0 2024-03-29T08:07:23.432 INFO:tasks.workunit.client.1.smithi148.stdout:5/629: dread d0/db/f10 [0,4194304] 0 2024-03-29T08:07:23.436 INFO:tasks.workunit.client.1.smithi148.stdout:9/677: write db/d22/d26/d54/fae [839407,79461] 0 2024-03-29T08:07:23.439 INFO:tasks.workunit.client.0.smithi037.stdout:6/418: dread d0/d4f/f57 [0,4194304] 0 2024-03-29T08:07:23.442 INFO:tasks.workunit.client.1.smithi148.stdout:3/677: link de/f61 de/d3b/d49/fdd 0 2024-03-29T08:07:23.444 INFO:tasks.workunit.client.1.smithi148.stdout:1/612: chown d3/de/d25/d35/d42/c16 497195908 1 2024-03-29T08:07:23.445 INFO:tasks.workunit.client.1.smithi148.stdout:9/678: mknod db/d22/d3f/ce3 0 2024-03-29T08:07:23.446 INFO:tasks.workunit.client.0.smithi037.stdout:4/480: creat d3/d4/d15/d1d/db/d45/d1e/d59/d60/f9c x:0 0 0 2024-03-29T08:07:23.448 INFO:tasks.workunit.client.0.smithi037.stdout:1/533: truncate d2/d40/f43 2566647 0 2024-03-29T08:07:23.448 INFO:tasks.workunit.client.1.smithi148.stdout:4/644: link df/d2c/d51/d77/d26/cbf df/d2c/d51/d77/d17/d73/ccc 0 2024-03-29T08:07:23.450 INFO:tasks.workunit.client.0.smithi037.stdout:4/481: fsync d3/d4/d27/f2e 0 2024-03-29T08:07:23.450 INFO:tasks.workunit.client.1.smithi148.stdout:8/658: dread d17/d49/f9c [0,4194304] 0 2024-03-29T08:07:23.452 INFO:tasks.workunit.client.1.smithi148.stdout:7/602: rmdir d8/d38/d3c/da9 39 2024-03-29T08:07:23.455 INFO:tasks.workunit.client.1.smithi148.stdout:9/679: symlink db/d22/d26/le4 0 2024-03-29T08:07:23.456 INFO:tasks.workunit.client.1.smithi148.stdout:4/645: dread df/d2c/d51/d77/d26/f4a [0,4194304] 0 2024-03-29T08:07:23.458 INFO:tasks.workunit.client.1.smithi148.stdout:7/603: symlink d8/d14/d45/ld8 0 2024-03-29T08:07:23.458 INFO:tasks.workunit.client.0.smithi037.stdout:6/419: getdents d0/d6a 0 2024-03-29T08:07:23.461 INFO:tasks.workunit.client.1.smithi148.stdout:0/644: dwrite db/de/df/d2b/f34 [0,4194304] 0 2024-03-29T08:07:23.462 INFO:tasks.workunit.client.0.smithi037.stdout:1/534: rename d2/d40/f92 to d2/fb8 0 2024-03-29T08:07:23.463 INFO:tasks.workunit.client.0.smithi037.stdout:4/482: truncate d3/d4/d15/d1d/f3f 1176751 0 2024-03-29T08:07:23.465 INFO:tasks.workunit.client.1.smithi148.stdout:9/680: mknod db/d10/d2d/d4e/d5f/d6e/dcb/dde/ce5 0 2024-03-29T08:07:23.465 INFO:tasks.workunit.client.1.smithi148.stdout:4/646: mknod df/d2c/d51/ccd 0 2024-03-29T08:07:23.466 INFO:tasks.workunit.client.1.smithi148.stdout:3/678: dwrite de/d24/f36 [0,4194304] 0 2024-03-29T08:07:23.467 INFO:tasks.workunit.client.0.smithi037.stdout:6/420: symlink d0/d4f/l88 0 2024-03-29T08:07:23.467 INFO:tasks.workunit.client.1.smithi148.stdout:4/647: stat df/d2c/c54 0 2024-03-29T08:07:23.468 INFO:tasks.workunit.client.1.smithi148.stdout:9/681: chown db/d10/d2d/d4e/d5f/d6e 0 1 2024-03-29T08:07:23.471 INFO:tasks.workunit.client.0.smithi037.stdout:1/535: fsync d2/d6/dc/d18/d22/d4f/f69 0 2024-03-29T08:07:23.471 INFO:tasks.workunit.client.1.smithi148.stdout:4/648: write df/d2c/d51/d77/d26/d34/d43/fc8 [8692,26739] 0 2024-03-29T08:07:23.471 INFO:tasks.workunit.client.1.smithi148.stdout:7/604: creat d8/d29/d44/dc3/fd9 x:0 0 0 2024-03-29T08:07:23.472 INFO:tasks.workunit.client.1.smithi148.stdout:0/645: creat db/dc/daa/d47/d7c/fc4 x:0 0 0 2024-03-29T08:07:23.475 INFO:tasks.workunit.client.0.smithi037.stdout:4/483: symlink d3/d57/d85/l9d 0 2024-03-29T08:07:23.480 INFO:tasks.workunit.client.1.smithi148.stdout:4/649: dread - df/d2c/d51/d77/d26/d34/d8a/fc4 zero size 2024-03-29T08:07:23.482 INFO:tasks.workunit.client.1.smithi148.stdout:9/682: rename db/d22/d26/d54/d4d/d35/d4b/dc3/d94/f9e to db/d10/d2d/d4e/dd4/fe6 0 2024-03-29T08:07:23.483 INFO:tasks.workunit.client.0.smithi037.stdout:4/484: write d3/d4/d15/d1d/db/d45/d1e/d59/d73/f83 [416095,87076] 0 2024-03-29T08:07:23.483 INFO:tasks.workunit.client.0.smithi037.stdout:6/421: rename d0/d19/d42/d60/c7c to d0/d5/d1a/d61/c89 0 2024-03-29T08:07:23.484 INFO:tasks.workunit.client.1.smithi148.stdout:2/739: dread d7/da/fe [4194304,4194304] 0 2024-03-29T08:07:23.492 INFO:tasks.workunit.client.1.smithi148.stdout:9/683: rmdir db/d22/d26/d54/d4d/d35/d4b/dc3 39 2024-03-29T08:07:23.494 INFO:tasks.workunit.client.1.smithi148.stdout:7/605: read - d8/d38/d3c/da9/fac zero size 2024-03-29T08:07:23.498 INFO:tasks.workunit.client.1.smithi148.stdout:2/740: rename d7/da/d33/ded/db7 to d7/d24/d4b/d66/df1 0 2024-03-29T08:07:23.500 INFO:tasks.workunit.client.1.smithi148.stdout:5/630: dread d0/db/d7b/fb6 [0,4194304] 0 2024-03-29T08:07:23.504 INFO:tasks.workunit.client.1.smithi148.stdout:7/606: creat d8/d29/d44/d50/d85/fda x:0 0 0 2024-03-29T08:07:23.507 INFO:tasks.workunit.client.1.smithi148.stdout:4/650: dwrite df/d2c/f86 [0,4194304] 0 2024-03-29T08:07:23.510 INFO:tasks.workunit.client.1.smithi148.stdout:0/646: mkdir db/de/df/d11/d1f/dc5 0 2024-03-29T08:07:23.518 INFO:tasks.workunit.client.1.smithi148.stdout:0/647: dread db/de/df/d11/d2d/d3e/f6d [0,4194304] 0 2024-03-29T08:07:23.524 INFO:tasks.workunit.client.0.smithi037.stdout:6/422: fdatasync d0/d5/d3f/f17 0 2024-03-29T08:07:23.535 INFO:tasks.workunit.client.0.smithi037.stdout:1/536: write d2/d6/dc/d18/d22/d4f/d7c/f7f [371966,80930] 0 2024-03-29T08:07:23.543 INFO:tasks.workunit.client.0.smithi037.stdout:4/485: dread d3/d4/d15/d1d/db/d3d/f56 [0,4194304] 0 2024-03-29T08:07:23.544 INFO:tasks.workunit.client.0.smithi037.stdout:4/486: stat d3/d57/d85 0 2024-03-29T08:07:23.549 INFO:tasks.workunit.client.0.smithi037.stdout:1/537: mknod d2/d6/dc/d18/d1d/d21/cb9 0 2024-03-29T08:07:23.550 INFO:tasks.workunit.client.0.smithi037.stdout:4/487: creat d3/d57/f9e x:0 0 0 2024-03-29T08:07:23.561 INFO:tasks.workunit.client.0.smithi037.stdout:6/423: dwrite d0/d19/d34/f50 [0,4194304] 0 2024-03-29T08:07:23.561 INFO:tasks.workunit.client.0.smithi037.stdout:1/538: write d2/d6/dc/d18/d22/d4f/f73 [344863,111214] 0 2024-03-29T08:07:23.563 INFO:tasks.workunit.client.0.smithi037.stdout:4/488: link d3/f5d d3/d4/d15/d1d/db/d45/d1e/d59/d7d/f9f 0 2024-03-29T08:07:23.566 INFO:tasks.workunit.client.0.smithi037.stdout:6/424: creat d0/d5/d1f/f8a x:0 0 0 2024-03-29T08:07:23.571 INFO:tasks.workunit.client.0.smithi037.stdout:1/539: symlink d2/d6/dc/d18/d1d/d5e/d72/lba 0 2024-03-29T08:07:23.573 INFO:tasks.workunit.client.0.smithi037.stdout:4/489: mkdir d3/d4/d15/d1d/db/d45/d93/d96/da0 0 2024-03-29T08:07:23.578 INFO:tasks.workunit.client.0.smithi037.stdout:2/358: sync 2024-03-29T08:07:23.578 INFO:tasks.workunit.client.0.smithi037.stdout:7/481: sync 2024-03-29T08:07:23.583 INFO:tasks.workunit.client.1.smithi148.stdout:0/648: dread - db/dc/daa/d47/d4c/f89 zero size 2024-03-29T08:07:23.583 INFO:tasks.workunit.client.1.smithi148.stdout:6/649: sync 2024-03-29T08:07:23.583 INFO:tasks.workunit.client.1.smithi148.stdout:4/651: dwrite df/d2c/d80/daa/fcb [0,4194304] 0 2024-03-29T08:07:23.583 INFO:tasks.workunit.client.0.smithi037.stdout:6/425: dread d0/f37 [0,4194304] 0 2024-03-29T08:07:23.584 INFO:tasks.workunit.client.1.smithi148.stdout:3/679: dwrite de/f15 [0,4194304] 0 2024-03-29T08:07:23.584 INFO:tasks.workunit.client.0.smithi037.stdout:4/490: rmdir d3/d57/d6a 39 2024-03-29T08:07:23.586 INFO:tasks.workunit.client.1.smithi148.stdout:7/607: write d8/f36 [495071,70825] 0 2024-03-29T08:07:23.586 INFO:tasks.workunit.client.0.smithi037.stdout:7/482: mkdir dd/d1d/d34/d54/d9c 0 2024-03-29T08:07:23.587 INFO:tasks.workunit.client.0.smithi037.stdout:6/426: chown d0/d5/d1a/f6c 70087044 1 2024-03-29T08:07:23.589 INFO:tasks.workunit.client.1.smithi148.stdout:4/652: chown df/d2c/d51/d77/dbe 220883 1 2024-03-29T08:07:23.594 INFO:tasks.workunit.client.1.smithi148.stdout:7/608: write d8/d14/d45/d4d/d78/fa0 [200879,89995] 0 2024-03-29T08:07:23.594 INFO:tasks.workunit.client.1.smithi148.stdout:2/741: dwrite d7/da/d33/ded/d2e/d37/f67 [0,4194304] 0 2024-03-29T08:07:23.597 INFO:tasks.workunit.client.0.smithi037.stdout:7/483: creat dd/d21/d35/d42/d5c/f9d x:0 0 0 2024-03-29T08:07:23.599 INFO:tasks.workunit.client.0.smithi037.stdout:2/359: mkdir d0/d20/d7d 0 2024-03-29T08:07:23.600 INFO:tasks.workunit.client.1.smithi148.stdout:3/680: mkdir de/d24/d33/d39/d46/d30/d37/d62/dde 0 2024-03-29T08:07:23.602 INFO:tasks.workunit.client.1.smithi148.stdout:6/650: mkdir d6/d9/d1e/d36/d4d/d9d/d28/db3/dd2 0 2024-03-29T08:07:23.603 INFO:tasks.workunit.client.1.smithi148.stdout:3/681: write de/f2f [382709,92550] 0 2024-03-29T08:07:23.606 INFO:tasks.workunit.client.1.smithi148.stdout:2/742: rmdir d7/da/d33/d3c 39 2024-03-29T08:07:23.611 INFO:tasks.workunit.client.1.smithi148.stdout:9/684: dread db/d22/d26/d54/d4d/d35/d4b/d8e/f97 [0,4194304] 0 2024-03-29T08:07:23.611 INFO:tasks.workunit.client.0.smithi037.stdout:6/427: link d0/d5/d1a/f5d d0/d19/d42/d60/f8b 0 2024-03-29T08:07:23.613 INFO:tasks.workunit.client.1.smithi148.stdout:6/651: mkdir d6/d9/dc5/dd3 0 2024-03-29T08:07:23.614 INFO:tasks.workunit.client.1.smithi148.stdout:6/652: fdatasync d6/fe 0 2024-03-29T08:07:23.616 INFO:tasks.workunit.client.1.smithi148.stdout:0/649: dwrite db/de/df/d11/d2d/f36 [0,4194304] 0 2024-03-29T08:07:23.618 INFO:tasks.workunit.client.1.smithi148.stdout:9/685: read db/d22/d26/d54/f2e [1723018,5209] 0 2024-03-29T08:07:23.623 INFO:tasks.workunit.client.1.smithi148.stdout:6/653: unlink d6/d9/d1e/d20/d30/f47 0 2024-03-29T08:07:23.625 INFO:tasks.workunit.client.0.smithi037.stdout:1/540: dread d2/d6/dc/f25 [0,4194304] 0 2024-03-29T08:07:23.627 INFO:tasks.workunit.client.0.smithi037.stdout:7/484: rename dd/d1d/d34/c3d to dd/d21/d35/d42/d5c/c9e 0 2024-03-29T08:07:23.627 INFO:tasks.workunit.client.1.smithi148.stdout:2/743: link d7/da/c11 d7/d24/d4b/cf2 0 2024-03-29T08:07:23.629 INFO:tasks.workunit.client.1.smithi148.stdout:4/653: dwrite df/d2c/d51/d77/d17/f42 [0,4194304] 0 2024-03-29T08:07:23.632 INFO:tasks.workunit.client.1.smithi148.stdout:4/654: read - df/d2c/d51/d77/d26/d9b/fc1 zero size 2024-03-29T08:07:23.632 INFO:tasks.workunit.client.1.smithi148.stdout:6/654: rename d6/d9/d1e/d36/d4d/d9d/c81 to d6/d69/cd4 0 2024-03-29T08:07:23.636 INFO:tasks.workunit.client.1.smithi148.stdout:6/655: fsync d6/fb0 0 2024-03-29T08:07:23.636 INFO:tasks.workunit.client.1.smithi148.stdout:2/744: dread - d7/da/d33/d3c/da5/fe8 zero size 2024-03-29T08:07:23.639 INFO:tasks.workunit.client.1.smithi148.stdout:0/650: mknod db/dc/daa/d47/d4c/cc6 0 2024-03-29T08:07:23.643 INFO:tasks.workunit.client.1.smithi148.stdout:4/655: rename df/d2c/d51/d77/f40 to df/d71/fce 0 2024-03-29T08:07:23.644 INFO:tasks.workunit.client.0.smithi037.stdout:2/360: symlink d0/d20/d7d/l7e 0 2024-03-29T08:07:23.645 INFO:tasks.workunit.client.1.smithi148.stdout:7/609: write d8/d29/d44/d50/d85/fd3 [752791,15220] 0 2024-03-29T08:07:23.646 INFO:tasks.workunit.client.1.smithi148.stdout:0/651: chown db/de/df/d11/d2d/f36 2063 1 2024-03-29T08:07:23.647 INFO:tasks.workunit.client.0.smithi037.stdout:7/485: unlink dd/d1d/d34/f59 0 2024-03-29T08:07:23.648 INFO:tasks.workunit.client.1.smithi148.stdout:3/682: dwrite de/d24/d33/d39/d46/d30/d37/faa [0,4194304] 0 2024-03-29T08:07:23.649 INFO:tasks.workunit.client.0.smithi037.stdout:1/541: mkdir d2/d6/dc/d18/d1d/dad/d87/dbb 0 2024-03-29T08:07:23.650 INFO:tasks.workunit.client.1.smithi148.stdout:9/686: dread db/d10/d2d/d4e/d5f/f65 [0,4194304] 0 2024-03-29T08:07:23.651 INFO:tasks.workunit.client.1.smithi148.stdout:4/656: dread - df/d2c/d51/d77/d26/d9b/da7/f8b zero size 2024-03-29T08:07:23.651 INFO:tasks.workunit.client.0.smithi037.stdout:4/491: link d3/d4/d15/d1d/db/d3d/c53 d3/d4/d15/d1d/db/d45/d1e/d59/d60/ca1 0 2024-03-29T08:07:23.653 INFO:tasks.workunit.client.1.smithi148.stdout:6/656: mknod d6/d9/d1e/d27/d48/d73/d7e/cd5 0 2024-03-29T08:07:23.655 INFO:tasks.workunit.client.1.smithi148.stdout:2/745: mkdir d7/d8c/d91/dd5/df3 0 2024-03-29T08:07:23.655 INFO:tasks.workunit.client.1.smithi148.stdout:0/652: write db/f23 [3220388,52405] 0 2024-03-29T08:07:23.660 INFO:tasks.workunit.client.0.smithi037.stdout:2/361: read d0/f15 [3495788,66553] 0 2024-03-29T08:07:23.662 INFO:tasks.workunit.client.1.smithi148.stdout:6/657: dread - d6/d9/d1e/d20/d30/fcb zero size 2024-03-29T08:07:23.663 INFO:tasks.workunit.client.0.smithi037.stdout:1/542: read d2/d6/dc/d58/d67/fa3 [1857591,105721] 0 2024-03-29T08:07:23.668 INFO:tasks.workunit.client.1.smithi148.stdout:4/657: creat df/d89/fcf x:0 0 0 2024-03-29T08:07:23.669 INFO:tasks.workunit.client.1.smithi148.stdout:4/658: fdatasync df/d71/f88 0 2024-03-29T08:07:23.669 INFO:tasks.workunit.client.1.smithi148.stdout:7/610: dread d8/d38/d3c/da9/f63 [4194304,4194304] 0 2024-03-29T08:07:23.671 INFO:tasks.workunit.client.1.smithi148.stdout:7/611: stat d8/d38/d3c/da5/caa 0 2024-03-29T08:07:23.674 INFO:tasks.workunit.client.1.smithi148.stdout:9/687: symlink db/d10/d2d/d4e/d5f/dd5/le7 0 2024-03-29T08:07:23.674 INFO:tasks.workunit.client.1.smithi148.stdout:6/658: dread d6/d9/d1e/d27/f64 [0,4194304] 0 2024-03-29T08:07:23.675 INFO:tasks.workunit.client.1.smithi148.stdout:7/612: write d8/d38/d3c/da9/f39 [4164358,98661] 0 2024-03-29T08:07:23.677 INFO:tasks.workunit.client.1.smithi148.stdout:4/659: symlink df/d89/ld0 0 2024-03-29T08:07:23.678 INFO:tasks.workunit.client.1.smithi148.stdout:3/683: link de/d24/d33/d39/d46/f32 de/d24/d33/d39/d46/d30/fdf 0 2024-03-29T08:07:23.682 INFO:tasks.workunit.client.1.smithi148.stdout:7/613: write d8/d29/d44/d49/fa3 [4105075,80467] 0 2024-03-29T08:07:23.683 INFO:tasks.workunit.client.1.smithi148.stdout:5/631: dread d0/db/d7b/d16/f8f [0,4194304] 0 2024-03-29T08:07:23.684 INFO:tasks.workunit.client.1.smithi148.stdout:7/614: write d8/d38/d3c/da5/fd5 [382990,34208] 0 2024-03-29T08:07:23.685 INFO:tasks.workunit.client.1.smithi148.stdout:6/659: fdatasync d6/d9/d1e/d27/d48/d9f/fa3 0 2024-03-29T08:07:23.686 INFO:tasks.workunit.client.1.smithi148.stdout:2/746: write d7/da/d33/d3c/d54/d70/f7b [1481587,125555] 0 2024-03-29T08:07:23.688 INFO:tasks.workunit.client.1.smithi148.stdout:4/660: creat df/d2c/d51/d77/d26/d34/d3e/fd1 x:0 0 0 2024-03-29T08:07:23.688 INFO:tasks.workunit.client.1.smithi148.stdout:3/684: mknod de/d24/d33/d39/d46/d30/d37/ce0 0 2024-03-29T08:07:23.691 INFO:tasks.workunit.client.1.smithi148.stdout:9/688: creat db/d10/d2d/d4e/d5f/fe8 x:0 0 0 2024-03-29T08:07:23.692 INFO:tasks.workunit.client.1.smithi148.stdout:3/685: readlink de/d24/d33/d39/da3/dbc/ld6 0 2024-03-29T08:07:23.693 INFO:tasks.workunit.client.1.smithi148.stdout:2/747: dread d7/da/d33/d3c/d54/dec/f8f [0,4194304] 0 2024-03-29T08:07:23.693 INFO:tasks.workunit.client.1.smithi148.stdout:4/661: rename lb to df/d71/ld2 0 2024-03-29T08:07:23.695 INFO:tasks.workunit.client.1.smithi148.stdout:6/660: creat d6/d9/d1e/d36/d4d/d9d/d28/d98/dcc/fd6 x:0 0 0 2024-03-29T08:07:23.696 INFO:tasks.workunit.client.1.smithi148.stdout:4/662: read - df/da0/fc2 zero size 2024-03-29T08:07:23.698 INFO:tasks.workunit.client.1.smithi148.stdout:2/748: unlink d7/d8c/da4/fb0 0 2024-03-29T08:07:23.700 INFO:tasks.workunit.client.1.smithi148.stdout:4/663: write df/d2c/d51/d77/d17/f20 [8398046,85283] 0 2024-03-29T08:07:23.702 INFO:tasks.workunit.client.1.smithi148.stdout:4/664: dread - df/d2c/d51/d77/d26/d9b/fc1 zero size 2024-03-29T08:07:23.705 INFO:tasks.workunit.client.1.smithi148.stdout:3/686: mkdir de/d24/d33/d39/d46/d30/d37/d62/de1 0 2024-03-29T08:07:23.707 INFO:tasks.workunit.client.1.smithi148.stdout:2/749: symlink d7/da/d33/d3c/d54/dec/lf4 0 2024-03-29T08:07:23.710 INFO:tasks.workunit.client.1.smithi148.stdout:3/687: creat de/d24/d33/d39/d46/d70/fe2 x:0 0 0 2024-03-29T08:07:23.712 INFO:tasks.workunit.client.1.smithi148.stdout:9/689: creat db/d22/d26/d54/d4d/d35/d4b/dc3/fe9 x:0 0 0 2024-03-29T08:07:23.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:23.711+0000 7f4dfb000700 1 -- 172.21.15.37:0/976438145 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4df4072820 msgr2=0x7f4df4072c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:07:23.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:23.712+0000 7f4df2a00700 1 -- 172.21.15.37:0/976438145 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4ddc00e070 con 0x7f4df4072820 2024-03-29T08:07:23.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:23.712+0000 7f4dfb000700 1 --2- 172.21.15.37:0/976438145 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4df4072820 0x7f4df4072c30 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f4ddc004940 tx=0x7f4ddc009e30 comp rx=0 tx=0).stop 2024-03-29T08:07:23.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:23.712+0000 7f4dfb000700 1 -- 172.21.15.37:0/976438145 shutdown_connections 2024-03-29T08:07:23.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:23.712+0000 7f4dfb000700 1 --2- 172.21.15.37:0/976438145 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4df4073170 0x7f4df4073b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:07:23.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:23.712+0000 7f4dfb000700 1 --2- 172.21.15.37:0/976438145 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4df4072820 0x7f4df4072c30 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:07:23.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:23.712+0000 7f4dfb000700 1 -- 172.21.15.37:0/976438145 >> 172.21.15.37:0/976438145 conn(0x7f4df40fa950 msgr2=0x7f4df40fcd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:07:23.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:23.713+0000 7f4dfb000700 1 -- 172.21.15.37:0/976438145 shutdown_connections 2024-03-29T08:07:23.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:23.713+0000 7f4dfb000700 1 -- 172.21.15.37:0/976438145 wait complete. 2024-03-29T08:07:23.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:23.714+0000 7f4dfb000700 1 Processor -- start 2024-03-29T08:07:23.715 INFO:tasks.workunit.client.0.smithi037.stdout:5/599: sync 2024-03-29T08:07:23.715 INFO:tasks.workunit.client.0.smithi037.stdout:9/710: sync 2024-03-29T08:07:23.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:23.715+0000 7f4dfb000700 1 -- start start 2024-03-29T08:07:23.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:23.715+0000 7f4dfb000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4df4072820 0x7f4df4113de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:07:23.716 INFO:tasks.workunit.client.1.smithi148.stdout:3/688: creat de/d24/d33/d39/d46/d30/d37/d79/fe3 x:0 0 0 2024-03-29T08:07:23.716 INFO:tasks.workunit.client.1.smithi148.stdout:5/632: dwrite d0/db/d1e/d36/d41/f7f [0,4194304] 0 2024-03-29T08:07:23.717 INFO:tasks.workunit.client.1.smithi148.stdout:1/613: sync 2024-03-29T08:07:23.717 INFO:tasks.workunit.client.0.smithi037.stdout:3/468: sync 2024-03-29T08:07:23.717 INFO:tasks.workunit.client.0.smithi037.stdout:8/484: sync 2024-03-29T08:07:23.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:23.716+0000 7f4df3e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4df4072820 0x7f4df4113de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:07:23.717 INFO:tasks.workunit.client.0.smithi037.stdout:0/381: sync 2024-03-29T08:07:23.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:23.716+0000 7f4dfb000700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4df4114320 0x7f4df4114fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:07:23.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:23.716+0000 7f4dfb000700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4df41147e0 con 0x7f4df4072820 2024-03-29T08:07:23.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:23.716+0000 7f4dfb000700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4df4114920 con 0x7f4df4114320 2024-03-29T08:07:23.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:23.717+0000 7f4df3e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4df4072820 0x7f4df4113de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46406/0 (socket says 172.21.15.37:46406) 2024-03-29T08:07:23.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:23.717+0000 7f4df3e00700 1 -- 172.21.15.37:0/1240753908 learned_addr learned my addr 172.21.15.37:0/1240753908 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:07:23.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:23.718+0000 7f4df3e00700 1 -- 172.21.15.37:0/1240753908 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4df4114320 msgr2=0x7f4df4114fe0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-03-29T08:07:23.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:23.718+0000 7f4df3e00700 1 --2- 172.21.15.37:0/1240753908 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4df4114320 0x7f4df4114fe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:07:23.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:23.718+0000 7f4df3e00700 1 -- 172.21.15.37:0/1240753908 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ddc005040 con 0x7f4df4072820 2024-03-29T08:07:23.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:23.719+0000 7f4df3e00700 1 --2- 172.21.15.37:0/1240753908 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4df4072820 0x7f4df4113de0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f4ddc000c00 tx=0x7f4ddc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:07:23.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:23.720+0000 7f4debe00700 1 -- 172.21.15.37:0/1240753908 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4ddc00e040 con 0x7f4df4072820 2024-03-29T08:07:23.719 INFO:tasks.workunit.client.1.smithi148.stdout:9/690: mknod db/d22/d26/cea 0 2024-03-29T08:07:23.720 INFO:tasks.workunit.client.1.smithi148.stdout:3/689: symlink de/le4 0 2024-03-29T08:07:23.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:23.720+0000 7f4debe00700 1 -- 172.21.15.37:0/1240753908 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f4ddc013ab0 con 0x7f4df4072820 2024-03-29T08:07:23.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:23.720+0000 7f4debe00700 1 -- 172.21.15.37:0/1240753908 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4ddc01c7c0 con 0x7f4df4072820 2024-03-29T08:07:23.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:23.721+0000 7f4dfb000700 1 -- 172.21.15.37:0/1240753908 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4df4115660 con 0x7f4df4072820 2024-03-29T08:07:23.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:23.721+0000 7f4dfb000700 1 -- 172.21.15.37:0/1240753908 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4df4115b70 con 0x7f4df4072820 2024-03-29T08:07:23.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:23.724+0000 7f4debe00700 1 -- 172.21.15.37:0/1240753908 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 25) v1 ==== 88766+0+0 (secure 0 0 0) 0x7f4ddc013c20 con 0x7f4df4072820 2024-03-29T08:07:23.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:23.724+0000 7f4dfb000700 1 -- 172.21.15.37:0/1240753908 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4df410c5b0 con 0x7f4df4072820 2024-03-29T08:07:23.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:23.726+0000 7f4debe00700 1 --2- 172.21.15.37:0/1240753908 >> [v2:172.21.15.148:6828/1134897184,v1:172.21.15.148:6829/1134897184] conn(0x7f4dd806afe0 0x7f4dd806d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:07:23.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:23.726+0000 7f4debe00700 1 -- 172.21.15.37:0/1240753908 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f4ddc08a7f0 con 0x7f4df4072820 2024-03-29T08:07:23.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:23.727+0000 7f4df3400700 1 --2- 172.21.15.37:0/1240753908 >> [v2:172.21.15.148:6828/1134897184,v1:172.21.15.148:6829/1134897184] conn(0x7f4dd806afe0 0x7f4dd806d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:07:23.727 INFO:tasks.workunit.client.1.smithi148.stdout:9/691: unlink db/d10/l25 0 2024-03-29T08:07:23.729 INFO:tasks.workunit.client.1.smithi148.stdout:3/690: unlink de/d3b/f60 0 2024-03-29T08:07:23.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:23.732+0000 7f4debe00700 1 -- 172.21.15.37:0/1240753908 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181579 (secure 0 0 0) 0x7f4ddc0586d0 con 0x7f4df4072820 2024-03-29T08:07:23.734 INFO:tasks.workunit.client.1.smithi148.stdout:3/691: write de/d24/d33/d39/da3/fc1 [84088,21419] 0 2024-03-29T08:07:23.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:23.736+0000 7f4df3400700 1 --2- 172.21.15.37:0/1240753908 >> [v2:172.21.15.148:6828/1134897184,v1:172.21.15.148:6829/1134897184] conn(0x7f4dd806afe0 0x7f4dd806d490 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f4de4004d00 tx=0x7f4de4009ce0 comp rx=0 tx=0).ready entity=mgr.14602 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:07:23.739 INFO:tasks.workunit.client.1.smithi148.stdout:1/614: dwrite d3/de/d21/d5e/fa6 [0,4194304] 0 2024-03-29T08:07:23.742 INFO:tasks.workunit.client.1.smithi148.stdout:3/692: creat de/d24/d33/d39/d46/d70/fe5 x:0 0 0 2024-03-29T08:07:23.742 INFO:tasks.workunit.client.1.smithi148.stdout:6/661: write d6/d9/d1e/d36/d4d/d9d/f31 [1244900,91422] 0 2024-03-29T08:07:23.745 INFO:tasks.workunit.client.0.smithi037.stdout:5/600: dwrite d8/d4d/d5c/fc3 [0,4194304] 0 2024-03-29T08:07:23.747 INFO:tasks.workunit.client.1.smithi148.stdout:1/615: creat d3/d18/d4d/d76/fbb x:0 0 0 2024-03-29T08:07:23.759 INFO:tasks.workunit.client.1.smithi148.stdout:6/662: dread d6/d9/d1e/d27/d48/d73/d7e/fa9 [0,4194304] 0 2024-03-29T08:07:23.763 INFO:tasks.workunit.client.1.smithi148.stdout:2/750: dwrite d7/f3b [0,4194304] 0 2024-03-29T08:07:23.773 INFO:tasks.workunit.client.1.smithi148.stdout:1/616: dread d3/fa [0,4194304] 0 2024-03-29T08:07:23.773 INFO:tasks.workunit.client.1.smithi148.stdout:5/633: dwrite d0/db/d1e/d36/d7c/f80 [0,4194304] 0 2024-03-29T08:07:23.774 INFO:tasks.workunit.client.0.smithi037.stdout:6/428: dwrite d0/f37 [0,4194304] 0 2024-03-29T08:07:23.777 INFO:tasks.workunit.client.0.smithi037.stdout:6/429: readlink d0/d4f/l88 0 2024-03-29T08:07:23.778 INFO:tasks.workunit.client.1.smithi148.stdout:5/634: stat d0/db/d1e/d36/d41 0 2024-03-29T08:07:23.780 INFO:tasks.workunit.client.0.smithi037.stdout:1/543: creat d2/d40/fbc x:0 0 0 2024-03-29T08:07:23.780 INFO:tasks.workunit.client.0.smithi037.stdout:6/430: fdatasync d0/d4f/f78 0 2024-03-29T08:07:23.781 INFO:tasks.workunit.client.1.smithi148.stdout:2/751: mknod d7/da/d33/ded/cf5 0 2024-03-29T08:07:23.787 INFO:tasks.workunit.client.1.smithi148.stdout:3/693: getdents de/d24/d33/d39/d46/d70 0 2024-03-29T08:07:23.789 INFO:tasks.workunit.client.1.smithi148.stdout:1/617: symlink d3/de/d1d/d47/lbc 0 2024-03-29T08:07:23.792 INFO:tasks.workunit.client.1.smithi148.stdout:9/692: dwrite db/d10/d2d/f34 [0,4194304] 0 2024-03-29T08:07:23.792 INFO:tasks.workunit.client.1.smithi148.stdout:2/752: fsync d7/da/d4c/f79 0 2024-03-29T08:07:23.793 INFO:tasks.workunit.client.1.smithi148.stdout:3/694: read de/d3b/d49/f68 [2719530,18920] 0 2024-03-29T08:07:23.795 INFO:tasks.workunit.client.1.smithi148.stdout:0/653: dwrite f7 [4194304,4194304] 0 2024-03-29T08:07:23.796 INFO:tasks.workunit.client.1.smithi148.stdout:2/753: write d7/da/d4c/f69 [4839309,62015] 0 2024-03-29T08:07:23.797 INFO:tasks.workunit.client.1.smithi148.stdout:5/635: dread d0/db/d1e/d35/d42/f5e [0,4194304] 0 2024-03-29T08:07:23.800 INFO:tasks.workunit.client.1.smithi148.stdout:6/663: getdents d6/d9/d1e/d27/d48/d73/d7e 0 2024-03-29T08:07:23.803 INFO:tasks.workunit.client.1.smithi148.stdout:2/754: write d7/d8c/d91/f25 [8229878,111840] 0 2024-03-29T08:07:23.807 INFO:tasks.workunit.client.1.smithi148.stdout:7/615: chown d8/f15 1614 1 2024-03-29T08:07:23.809 INFO:tasks.workunit.client.1.smithi148.stdout:0/654: truncate db/de/df/d2b/f53 5135330 0 2024-03-29T08:07:23.815 INFO:tasks.workunit.client.1.smithi148.stdout:0/655: chown db/de/df/d11/d1f/c26 20669 1 2024-03-29T08:07:23.822 INFO:tasks.workunit.client.1.smithi148.stdout:1/618: fdatasync d3/de/f1b 0 2024-03-29T08:07:23.823 INFO:tasks.workunit.client.0.smithi037.stdout:3/469: read dd/d15/d27/f41 [230595,21850] 0 2024-03-29T08:07:23.830 INFO:tasks.workunit.client.1.smithi148.stdout:9/693: rename db/d22/d26/d54/d4d/d35/d4b/dc3/d94 to db/d10/d6d/d73/da8/deb 0 2024-03-29T08:07:23.830 INFO:tasks.workunit.client.0.smithi037.stdout:2/362: dread d0/fd [0,4194304] 0 2024-03-29T08:07:23.831 INFO:tasks.workunit.client.1.smithi148.stdout:9/694: fdatasync db/d10/d2d/d4e/d5f/fe8 0 2024-03-29T08:07:23.839 INFO:tasks.workunit.client.0.smithi037.stdout:8/485: dwrite d5/f5b [0,4194304] 0 2024-03-29T08:07:23.852 INFO:tasks.workunit.client.1.smithi148.stdout:5/636: dread d0/d46/d9e/fa6 [0,4194304] 0 2024-03-29T08:07:23.853 INFO:tasks.workunit.client.1.smithi148.stdout:0/656: unlink db/de/df/d11/d2d/da3/cb8 0 2024-03-29T08:07:23.860 INFO:tasks.workunit.client.1.smithi148.stdout:7/616: rename d8/d38/d3c to d8/d38/d3c/ddb 22 2024-03-29T08:07:23.861 INFO:tasks.workunit.client.0.smithi037.stdout:6/431: dread d0/d19/d42/d60/f8b [0,4194304] 0 2024-03-29T08:07:23.862 INFO:tasks.workunit.client.1.smithi148.stdout:7/617: chown d8/d29/d44/d50/d94/lb1 3532720 1 2024-03-29T08:07:23.862 INFO:tasks.workunit.client.1.smithi148.stdout:5/637: link d0/d46/d9e/fa6 d0/db/d1e/d5b/d66/fd2 0 2024-03-29T08:07:23.869 INFO:tasks.workunit.client.1.smithi148.stdout:7/618: creat d8/d14/da8/fdc x:0 0 0 2024-03-29T08:07:23.869 INFO:tasks.workunit.client.1.smithi148.stdout:9/695: rename db/d10/d2d/d4e/d5f/d6e/dcb to db/d10/d6d/d73/da8/deb/dec 0 2024-03-29T08:07:23.870 INFO:tasks.workunit.client.1.smithi148.stdout:2/755: write d7/da/d52/f63 [1504306,84542] 0 2024-03-29T08:07:23.871 INFO:tasks.workunit.client.1.smithi148.stdout:3/695: write de/d24/d33/d39/d46/d30/d37/d62/d63/f99 [110165,122359] 0 2024-03-29T08:07:23.874 INFO:tasks.workunit.client.1.smithi148.stdout:5/638: creat d0/db/d1e/d79/fd3 x:0 0 0 2024-03-29T08:07:23.875 INFO:tasks.workunit.client.1.smithi148.stdout:7/619: chown d8/d29/d44/c75 9142786 1 2024-03-29T08:07:23.880 INFO:tasks.workunit.client.1.smithi148.stdout:3/696: mknod de/d24/d33/d39/da3/dbc/ce6 0 2024-03-29T08:07:23.881 INFO:tasks.workunit.client.1.smithi148.stdout:5/639: symlink d0/db/d1e/d5b/d66/d72/d95/d9c/ld4 0 2024-03-29T08:07:23.883 INFO:tasks.workunit.client.1.smithi148.stdout:7/620: chown d8/d29/d44/d50/d94/ca1 524529118 1 2024-03-29T08:07:23.887 INFO:tasks.workunit.client.0.smithi037.stdout:3/470: symlink dd/d15/d27/d33/la1 0 2024-03-29T08:07:23.889 INFO:tasks.workunit.client.1.smithi148.stdout:2/756: creat d7/da/d33/d3c/ff6 x:0 0 0 2024-03-29T08:07:23.890 INFO:tasks.workunit.client.1.smithi148.stdout:1/619: dwrite d3/de/f1f [0,4194304] 0 2024-03-29T08:07:23.891 INFO:tasks.workunit.client.0.smithi037.stdout:2/363: fdatasync d0/d23/d44/f46 0 2024-03-29T08:07:23.894 INFO:tasks.workunit.client.1.smithi148.stdout:1/620: write d3/d55/f37 [5357776,118061] 0 2024-03-29T08:07:23.900 INFO:tasks.workunit.client.1.smithi148.stdout:0/657: dread db/dc/daa/d47/daf/f99 [0,4194304] 0 2024-03-29T08:07:23.908 INFO:tasks.workunit.client.0.smithi037.stdout:8/486: mknod d5/d10/d2d/d34/d50/d83/d46/c9f 0 2024-03-29T08:07:23.910 INFO:tasks.workunit.client.1.smithi148.stdout:5/640: truncate d0/db/d7b/fb6 836017 0 2024-03-29T08:07:23.910 INFO:tasks.workunit.client.1.smithi148.stdout:9/696: dwrite db/d22/d26/d54/f45 [0,4194304] 0 2024-03-29T08:07:23.910 INFO:tasks.workunit.client.1.smithi148.stdout:3/697: dwrite de/d24/d33/d39/d46/d30/d37/d79/fe3 [0,4194304] 0 2024-03-29T08:07:23.911 INFO:tasks.workunit.client.1.smithi148.stdout:1/621: symlink d3/d18/dae/lbd 0 2024-03-29T08:07:23.913 INFO:tasks.workunit.client.1.smithi148.stdout:8/659: sync 2024-03-29T08:07:23.919 INFO:tasks.workunit.client.1.smithi148.stdout:9/697: mkdir db/d10/d6d/d73/da8/deb/dec/dde/ded 0 2024-03-29T08:07:23.924 INFO:tasks.workunit.client.1.smithi148.stdout:9/698: mkdir db/d22/d26/dee 0 2024-03-29T08:07:23.924 INFO:tasks.workunit.client.0.smithi037.stdout:6/432: unlink d0/d5/d1f/f8a 0 2024-03-29T08:07:23.926 INFO:tasks.workunit.client.1.smithi148.stdout:9/699: read - db/d10/d2d/d4e/da6/fac zero size 2024-03-29T08:07:23.926 INFO:tasks.workunit.client.0.smithi037.stdout:6/433: chown d0/d19/d34/d4e 211715 1 2024-03-29T08:07:23.942 INFO:tasks.workunit.client.0.smithi037.stdout:5/601: mkdir d8/d22/d37/d8d/db3/dcb 0 2024-03-29T08:07:23.944 INFO:tasks.workunit.client.1.smithi148.stdout:1/622: dread d3/d18/f39 [0,4194304] 0 2024-03-29T08:07:23.944 INFO:tasks.workunit.client.1.smithi148.stdout:3/698: dread de/d3b/d49/f68 [0,4194304] 0 2024-03-29T08:07:23.948 INFO:tasks.workunit.client.1.smithi148.stdout:9/700: write db/d10/d2d/d4e/f6b [1797308,64790] 0 2024-03-29T08:07:23.958 INFO:tasks.workunit.client.1.smithi148.stdout:8/660: symlink d17/d49/d9a/dbe/d6f/d64/da5/le0 0 2024-03-29T08:07:23.959 INFO:tasks.workunit.client.1.smithi148.stdout:1/623: write d3/d18/f20 [2438473,54047] 0 2024-03-29T08:07:23.959 INFO:tasks.workunit.client.1.smithi148.stdout:3/699: link de/l85 de/d24/le7 0 2024-03-29T08:07:23.960 INFO:tasks.workunit.client.1.smithi148.stdout:0/658: rename db/dc/daa/d47/d4c/la2 to db/dc/daa/d47/d4c/d5a/d7a/db1/lc7 0 2024-03-29T08:07:23.963 INFO:tasks.workunit.client.1.smithi148.stdout:8/661: truncate d17/d49/d9a/dbe/d6f/fcc 97881 0 2024-03-29T08:07:23.966 INFO:tasks.workunit.client.1.smithi148.stdout:5/641: rename d0/db/d1e/d5b/d66/d72/d95 to d0/db/d1e/d35/d42/d67/d76/d81/d98/dd5 0 2024-03-29T08:07:23.967 INFO:tasks.workunit.client.1.smithi148.stdout:9/701: dwrite db/d10/d2d/d4e/f6c [0,4194304] 0 2024-03-29T08:07:23.969 INFO:tasks.workunit.client.1.smithi148.stdout:3/700: unlink de/d24/d33/f96 0 2024-03-29T08:07:23.972 INFO:tasks.workunit.client.1.smithi148.stdout:0/659: write db/dc/daa/d47/d7c/fb4 [4716368,116834] 0 2024-03-29T08:07:23.974 INFO:tasks.workunit.client.1.smithi148.stdout:5/642: creat d0/db/d1e/d6e/da8/fd6 x:0 0 0 2024-03-29T08:07:23.975 INFO:tasks.workunit.client.1.smithi148.stdout:5/643: readlink d0/db/d1e/l77 0 2024-03-29T08:07:23.979 INFO:tasks.workunit.client.1.smithi148.stdout:8/662: unlink d17/d4e/fdc 0 2024-03-29T08:07:23.980 INFO:tasks.workunit.client.1.smithi148.stdout:9/702: mkdir db/d22/d26/d54/d4d/dbb/def 0 2024-03-29T08:07:23.980 INFO:tasks.workunit.client.1.smithi148.stdout:3/701: stat de/f61 0 2024-03-29T08:07:23.982 INFO:tasks.workunit.client.1.smithi148.stdout:9/703: dread - db/d10/d2d/d4e/da6/fac zero size 2024-03-29T08:07:23.983 INFO:tasks.workunit.client.0.smithi037.stdout:7/486: link dd/l10 dd/d21/d35/d42/l9f 0 2024-03-29T08:07:23.984 INFO:tasks.workunit.client.1.smithi148.stdout:9/704: chown db/d22/d26/d54/d4d/d35/d4b/d8e 209466 1 2024-03-29T08:07:23.984 INFO:tasks.workunit.client.1.smithi148.stdout:3/702: creat de/d24/d33/d6b/fe8 x:0 0 0 2024-03-29T08:07:23.986 INFO:tasks.workunit.client.0.smithi037.stdout:7/487: chown dd/d21/d35/d42/d5c/f69 0 1 2024-03-29T08:07:23.986 INFO:tasks.workunit.client.1.smithi148.stdout:5/644: truncate d0/db/d7b/f3b 3881363 0 2024-03-29T08:07:23.986 INFO:tasks.workunit.client.1.smithi148.stdout:3/703: stat de/d24/d81/cb0 0 2024-03-29T08:07:23.989 INFO:tasks.workunit.client.0.smithi037.stdout:7/488: truncate dd/f48 4945363 0 2024-03-29T08:07:23.991 INFO:tasks.workunit.client.1.smithi148.stdout:5/645: mkdir d0/db/d1e/d35/d42/d67/d9a/dd7 0 2024-03-29T08:07:23.992 INFO:tasks.workunit.client.0.smithi037.stdout:7/489: write dd/d21/d35/d60/f6e [535080,4432] 0 2024-03-29T08:07:23.992 INFO:tasks.workunit.client.1.smithi148.stdout:3/704: symlink de/d24/d33/le9 0 2024-03-29T08:07:23.993 INFO:tasks.workunit.client.0.smithi037.stdout:6/434: creat d0/d5/d1e/d44/d71/f8c x:0 0 0 2024-03-29T08:07:23.993 INFO:tasks.workunit.client.1.smithi148.stdout:5/646: dread - d0/d46/d9e/fc1 zero size 2024-03-29T08:07:23.995 INFO:tasks.workunit.client.1.smithi148.stdout:5/647: read - d0/db/d1e/d35/d42/d67/d76/fba zero size 2024-03-29T08:07:23.996 INFO:tasks.workunit.client.0.smithi037.stdout:5/602: creat d8/d4d/d72/fcc x:0 0 0 2024-03-29T08:07:23.997 INFO:tasks.workunit.client.1.smithi148.stdout:9/705: dread db/d22/d26/d54/f45 [0,4194304] 0 2024-03-29T08:07:23.999 INFO:tasks.workunit.client.1.smithi148.stdout:3/705: fsync de/f61 0 2024-03-29T08:07:23.999 INFO:tasks.workunit.client.1.smithi148.stdout:0/660: link db/dc/d78/f8b db/dc/daa/fc8 0 2024-03-29T08:07:24.005 INFO:tasks.workunit.client.1.smithi148.stdout:8/663: creat d17/d18/ddf/fe1 x:0 0 0 2024-03-29T08:07:24.005 INFO:tasks.workunit.client.1.smithi148.stdout:8/664: fdatasync d17/d18/f67 0 2024-03-29T08:07:24.007 INFO:tasks.workunit.client.1.smithi148.stdout:3/706: dread de/d24/d33/d39/d46/f65 [0,4194304] 0 2024-03-29T08:07:24.008 INFO:tasks.workunit.client.0.smithi037.stdout:2/364: creat d0/d53/f7f x:0 0 0 2024-03-29T08:07:24.010 INFO:tasks.workunit.client.1.smithi148.stdout:0/661: mknod db/de/d24/cc9 0 2024-03-29T08:07:24.014 INFO:tasks.workunit.client.0.smithi037.stdout:0/382: getdents d9/da/d1b/d33 0 2024-03-29T08:07:24.020 INFO:tasks.workunit.client.0.smithi037.stdout:6/435: creat d0/d20/f8d x:0 0 0 2024-03-29T08:07:24.023 INFO:tasks.workunit.client.0.smithi037.stdout:7/490: symlink dd/d21/d35/d42/d5c/la0 0 2024-03-29T08:07:24.027 INFO:tasks.workunit.client.1.smithi148.stdout:9/706: symlink db/d22/d26/d54/d4d/d35/lf0 0 2024-03-29T08:07:24.028 INFO:tasks.workunit.client.1.smithi148.stdout:3/707: creat de/d24/d33/d39/d46/d30/d37/fea x:0 0 0 2024-03-29T08:07:24.030 INFO:tasks.workunit.client.0.smithi037.stdout:2/365: creat d0/d20/d7d/f80 x:0 0 0 2024-03-29T08:07:24.030 INFO:tasks.workunit.client.0.smithi037.stdout:6/436: chown d0/d19/d34/d4e/c5b 38 1 2024-03-29T08:07:24.033 INFO:tasks.workunit.client.0.smithi037.stdout:9/711: dwrite d1/f8d [0,4194304] 0 2024-03-29T08:07:24.034 INFO:tasks.workunit.client.0.smithi037.stdout:0/383: fdatasync d9/da/dd/f23 0 2024-03-29T08:07:24.038 INFO:tasks.workunit.client.0.smithi037.stdout:2/366: stat d0/c1c 0 2024-03-29T08:07:24.039 INFO:tasks.workunit.client.1.smithi148.stdout:5/648: creat d0/db/d7b/fd8 x:0 0 0 2024-03-29T08:07:24.041 INFO:tasks.workunit.client.1.smithi148.stdout:3/708: dwrite de/d24/d33/d6b/fe8 [0,4194304] 0 2024-03-29T08:07:24.041 INFO:tasks.workunit.client.1.smithi148.stdout:9/707: mkdir db/d22/d26/d54/d4d/d35/d4b/d58/df1 0 2024-03-29T08:07:24.044 INFO:tasks.workunit.client.0.smithi037.stdout:2/367: chown d0/d23/d44/d62 0 1 2024-03-29T08:07:24.047 INFO:tasks.workunit.client.0.smithi037.stdout:9/712: readlink d1/d4/d19/d1e/d4a/d6f/da2/d68/lab 0 2024-03-29T08:07:24.048 INFO:tasks.workunit.client.0.smithi037.stdout:1/544: mkdir d2/d6/dc/d18/dbd 0 2024-03-29T08:07:24.050 INFO:tasks.workunit.client.0.smithi037.stdout:4/492: dwrite d3/d4/d15/f29 [0,4194304] 0 2024-03-29T08:07:24.052 INFO:tasks.workunit.client.1.smithi148.stdout:5/649: rmdir d0/db/d7b/d16/db1 0 2024-03-29T08:07:24.054 INFO:tasks.workunit.client.1.smithi148.stdout:0/662: rmdir db/dc 39 2024-03-29T08:07:24.055 INFO:tasks.workunit.client.1.smithi148.stdout:9/708: mkdir db/d22/d26/d54/d4d/d35/d4b/d58/df1/df2 0 2024-03-29T08:07:24.058 INFO:tasks.workunit.client.1.smithi148.stdout:3/709: dwrite de/d24/d33/d39/d46/d30/d37/d62/d63/dbf/fd2 [0,4194304] 0 2024-03-29T08:07:24.059 INFO:tasks.workunit.client.1.smithi148.stdout:0/663: fsync db/de/df/d11/f16 0 2024-03-29T08:07:24.064 INFO:tasks.workunit.client.1.smithi148.stdout:9/709: write db/f21 [2405421,43922] 0 2024-03-29T08:07:24.066 INFO:tasks.workunit.client.1.smithi148.stdout:4/665: sync 2024-03-29T08:07:24.066 INFO:tasks.workunit.client.1.smithi148.stdout:6/664: sync 2024-03-29T08:07:24.066 INFO:tasks.workunit.client.1.smithi148.stdout:9/710: fsync db/d22/d26/d54/d4d/d35/d4b/dc3/fe9 0 2024-03-29T08:07:24.067 INFO:tasks.workunit.client.1.smithi148.stdout:3/710: symlink de/d24/d33/d39/d46/d70/leb 0 2024-03-29T08:07:24.070 INFO:tasks.workunit.client.1.smithi148.stdout:4/666: chown df/d2c/d51/d77/d17/d73/f7a 0 1 2024-03-29T08:07:24.070 INFO:tasks.workunit.client.1.smithi148.stdout:6/665: chown d6/d9/d1e/d27/d48/d9f/faa 1443002 1 2024-03-29T08:07:24.072 INFO:tasks.workunit.client.0.smithi037.stdout:2/368: unlink d0/f2b 0 2024-03-29T08:07:24.073 INFO:tasks.workunit.client.0.smithi037.stdout:4/493: mknod d3/d4/d15/d1d/db/d45/d1e/d59/d7d/ca2 0 2024-03-29T08:07:24.075 INFO:tasks.workunit.client.1.smithi148.stdout:4/667: fsync df/d71/f7b 0 2024-03-29T08:07:24.077 INFO:tasks.workunit.client.0.smithi037.stdout:2/369: fsync d0/d23/f26 0 2024-03-29T08:07:24.079 INFO:tasks.workunit.client.1.smithi148.stdout:0/664: creat db/de/d8e/da9/fca x:0 0 0 2024-03-29T08:07:24.080 INFO:tasks.workunit.client.0.smithi037.stdout:2/370: write d0/d20/f75 [956372,40641] 0 2024-03-29T08:07:24.080 INFO:tasks.workunit.client.0.smithi037.stdout:4/494: readlink d3/d4/d15/d1d/db/d45/l47 0 2024-03-29T08:07:24.081 INFO:tasks.workunit.client.1.smithi148.stdout:0/665: chown db/de/d8e/da9 11583 1 2024-03-29T08:07:24.084 INFO:tasks.workunit.client.0.smithi037.stdout:2/371: fdatasync d0/d23/f26 0 2024-03-29T08:07:24.084 INFO:tasks.workunit.client.0.smithi037.stdout:9/713: mkdir d1/d1f/df6 0 2024-03-29T08:07:24.088 INFO:tasks.workunit.client.1.smithi148.stdout:6/666: getdents d6/d9/d1e/d36/d4d/d9d/d28/d98 0 2024-03-29T08:07:24.092 INFO:tasks.workunit.client.1.smithi148.stdout:4/668: dwrite df/d2c/d51/d77/d17/f36 [0,4194304] 0 2024-03-29T08:07:24.096 INFO:tasks.workunit.client.0.smithi037.stdout:0/384: rename d9/da/dd/d59/l43 to d9/da/dd/d20/d38/l78 0 2024-03-29T08:07:24.096 INFO:tasks.workunit.client.1.smithi148.stdout:6/667: unlink d6/d9/d1e/d27/d48/d9f/fa0 0 2024-03-29T08:07:24.099 INFO:tasks.workunit.client.1.smithi148.stdout:3/711: dread de/d24/d33/d39/d46/f21 [0,4194304] 0 2024-03-29T08:07:24.102 INFO:tasks.workunit.client.1.smithi148.stdout:3/712: write de/d24/d33/d39/d46/d30/d37/fc3 [838600,91571] 0 2024-03-29T08:07:24.104 INFO:tasks.workunit.client.0.smithi037.stdout:9/714: mknod d1/d4/d19/d1e/d4a/d6f/da2/d31/daa/cf7 0 2024-03-29T08:07:24.104 INFO:tasks.workunit.client.1.smithi148.stdout:0/666: symlink db/dc/daa/d47/daf/lcb 0 2024-03-29T08:07:24.106 INFO:tasks.workunit.client.0.smithi037.stdout:9/715: stat d1/d3/c3c 0 2024-03-29T08:07:24.106 INFO:tasks.workunit.client.1.smithi148.stdout:0/667: write db/dc/d78/f94 [1902945,10780] 0 2024-03-29T08:07:24.108 INFO:tasks.workunit.client.0.smithi037.stdout:1/545: creat d2/d6/dc/d18/d1d/d21/d99/fbe x:0 0 0 2024-03-29T08:07:24.113 INFO:tasks.workunit.client.0.smithi037.stdout:6/437: dread d0/d5/d1e/f12 [0,4194304] 0 2024-03-29T08:07:24.117 INFO:tasks.workunit.client.0.smithi037.stdout:1/546: write d2/d6/dc/f15 [3936314,110734] 0 2024-03-29T08:07:24.127 INFO:tasks.workunit.client.1.smithi148.stdout:5/650: dwrite d0/db/d7b/d16/d2e/f56 [0,4194304] 0 2024-03-29T08:07:24.129 INFO:tasks.workunit.client.0.smithi037.stdout:4/495: creat d3/d4/d15/d1d/db/fa3 x:0 0 0 2024-03-29T08:07:24.135 INFO:tasks.workunit.client.0.smithi037.stdout:0/385: dwrite d9/da/dd/d13/f45 [0,4194304] 0 2024-03-29T08:07:24.136 INFO:tasks.workunit.client.0.smithi037.stdout:1/547: fdatasync d2/d6/dc/d18/d1d/d21/d99/f98 0 2024-03-29T08:07:24.136 INFO:tasks.workunit.client.1.smithi148.stdout:3/713: dwrite de/f80 [0,4194304] 0 2024-03-29T08:07:24.137 INFO:tasks.workunit.client.1.smithi148.stdout:5/651: chown d0/db/d1e/d5b/d66/d72 440845977 1 2024-03-29T08:07:24.139 INFO:tasks.workunit.client.0.smithi037.stdout:9/716: rename d1/d4/d19/d69/f9c to d1/d4/d19/d1e/d25/ff8 0 2024-03-29T08:07:24.139 INFO:tasks.workunit.client.1.smithi148.stdout:6/668: dwrite d6/d9/f23 [0,4194304] 0 2024-03-29T08:07:24.140 INFO:tasks.workunit.client.1.smithi148.stdout:0/668: dwrite db/dc/d78/f79 [0,4194304] 0 2024-03-29T08:07:24.144 INFO:tasks.workunit.client.0.smithi037.stdout:1/548: write d2/d6/dc/d18/d22/d4f/d7c/f7f [435571,48237] 0 2024-03-29T08:07:24.144 INFO:tasks.workunit.client.1.smithi148.stdout:0/669: chown db/de/df/d2b/dae 4309 1 2024-03-29T08:07:24.147 INFO:tasks.workunit.client.1.smithi148.stdout:7/621: dwrite d8/d14/f2d [0,4194304] 0 2024-03-29T08:07:24.148 INFO:tasks.workunit.client.1.smithi148.stdout:3/714: creat de/d24/d33/d39/da3/dbc/fec x:0 0 0 2024-03-29T08:07:24.150 INFO:tasks.workunit.client.1.smithi148.stdout:7/622: stat d8/d29/d44/d50/d94/lb1 0 2024-03-29T08:07:24.152 INFO:tasks.workunit.client.0.smithi037.stdout:0/386: creat d9/da/dd/d20/d38/d53/f79 x:0 0 0 2024-03-29T08:07:24.153 INFO:tasks.workunit.client.0.smithi037.stdout:3/471: dwrite dd/d46/f3c [0,4194304] 0 2024-03-29T08:07:24.154 INFO:tasks.workunit.client.1.smithi148.stdout:5/652: write d0/db/d1e/d35/d42/d67/d76/d81/d98/dd5/f9d [823812,50129] 0 2024-03-29T08:07:24.154 INFO:tasks.workunit.client.1.smithi148.stdout:3/715: chown de/d24/d33/d39/d46/d30/cab 366614143 1 2024-03-29T08:07:24.157 INFO:tasks.workunit.client.1.smithi148.stdout:6/669: dread d6/d9/d1e/d27/d48/d73/d7e/fa9 [0,4194304] 0 2024-03-29T08:07:24.159 INFO:tasks.workunit.client.1.smithi148.stdout:7/623: truncate d8/d38/d3c/da5/fd5 861859 0 2024-03-29T08:07:24.160 INFO:tasks.workunit.client.0.smithi037.stdout:0/387: rename d9 to d9/da/dd/d20/d38/d7a 22 2024-03-29T08:07:24.162 INFO:tasks.workunit.client.1.smithi148.stdout:4/669: truncate df/d71/fce 678185 0 2024-03-29T08:07:24.165 INFO:tasks.workunit.client.1.smithi148.stdout:4/670: stat df/f9a 0 2024-03-29T08:07:24.165 INFO:tasks.workunit.client.0.smithi037.stdout:8/487: dwrite d5/f1e [0,4194304] 0 2024-03-29T08:07:24.166 INFO:tasks.workunit.client.1.smithi148.stdout:0/670: mknod db/de/d8e/da9/ccc 0 2024-03-29T08:07:24.169 INFO:tasks.workunit.client.1.smithi148.stdout:3/716: readlink de/d24/d33/d39/d46/d30/d37/d62/d63/lb2 0 2024-03-29T08:07:24.170 INFO:tasks.workunit.client.1.smithi148.stdout:7/624: unlink d8/d14/f21 0 2024-03-29T08:07:24.178 INFO:tasks.workunit.client.1.smithi148.stdout:6/670: rmdir d6/d9/d1e/d36/d4d/d9d/d28/db3 39 2024-03-29T08:07:24.184 INFO:tasks.workunit.client.1.smithi148.stdout:2/757: sync 2024-03-29T08:07:24.185 INFO:tasks.workunit.client.1.smithi148.stdout:0/671: symlink db/de/df/d11/d42/dc2/lcd 0 2024-03-29T08:07:24.188 INFO:tasks.workunit.client.1.smithi148.stdout:4/671: dwrite df/d2c/d80/daa/fcb [0,4194304] 0 2024-03-29T08:07:24.188 INFO:tasks.workunit.client.1.smithi148.stdout:0/672: write db/dc/daa/f49 [657333,82678] 0 2024-03-29T08:07:24.188 INFO:tasks.workunit.client.1.smithi148.stdout:3/717: creat de/d24/d33/d39/d71/fed x:0 0 0 2024-03-29T08:07:24.190 INFO:tasks.workunit.client.0.smithi037.stdout:8/488: rename d5/c12 to d5/d10/d19/d8d/ca0 0 2024-03-29T08:07:24.196 INFO:tasks.workunit.client.0.smithi037.stdout:7/491: dwrite dd/d21/d35/d42/d5c/f69 [0,4194304] 0 2024-03-29T08:07:24.200 INFO:tasks.workunit.client.1.smithi148.stdout:3/718: mknod de/d24/d33/d39/d71/cee 0 2024-03-29T08:07:24.204 INFO:tasks.workunit.client.1.smithi148.stdout:6/671: link d6/d9/d1e/d36/d4d/d9d/f4a d6/d9/d68/fd7 0 2024-03-29T08:07:24.205 INFO:tasks.workunit.client.0.smithi037.stdout:3/472: dwrite dd/f3f [0,4194304] 0 2024-03-29T08:07:24.205 INFO:tasks.workunit.client.0.smithi037.stdout:6/438: write d0/d5/d1a/f5d [9341954,26] 0 2024-03-29T08:07:24.208 INFO:tasks.workunit.client.0.smithi037.stdout:6/439: fdatasync d0/d5/f14 0 2024-03-29T08:07:24.210 INFO:tasks.workunit.client.1.smithi148.stdout:4/672: rename df/d2c/d80/da6/lc5 to df/d60/ld3 0 2024-03-29T08:07:24.212 INFO:tasks.workunit.client.1.smithi148.stdout:2/758: creat d7/d8c/d91/dd5/df3/ff7 x:0 0 0 2024-03-29T08:07:24.212 INFO:tasks.workunit.client.1.smithi148.stdout:3/719: dread de/d3b/d49/f64 [0,4194304] 0 2024-03-29T08:07:24.213 INFO:tasks.workunit.client.0.smithi037.stdout:8/489: dread - d5/d10/d19/d3d/d4a/f61 zero size 2024-03-29T08:07:24.216 INFO:tasks.workunit.client.1.smithi148.stdout:4/673: truncate df/d2c/d51/d77/d26/d9b/fc1 604206 0 2024-03-29T08:07:24.219 INFO:tasks.workunit.client.1.smithi148.stdout:7/625: getdents d8 0 2024-03-29T08:07:24.219 INFO:tasks.workunit.client.1.smithi148.stdout:4/674: chown df/f14 13831 1 2024-03-29T08:07:24.220 INFO:tasks.workunit.client.1.smithi148.stdout:4/675: readlink df/l37 0 2024-03-29T08:07:24.222 INFO:tasks.workunit.client.1.smithi148.stdout:0/673: dread db/dc/daa/d47/d7c/fb4 [0,4194304] 0 2024-03-29T08:07:24.222 INFO:tasks.workunit.client.1.smithi148.stdout:3/720: mkdir de/d24/d33/d39/d46/d70/def 0 2024-03-29T08:07:24.225 INFO:tasks.workunit.client.1.smithi148.stdout:3/721: chown de/d24/d33/d39/da3/dbc/fbe 4084 1 2024-03-29T08:07:24.226 INFO:tasks.workunit.client.0.smithi037.stdout:3/473: stat dd/d5e/f6d 0 2024-03-29T08:07:24.226 INFO:tasks.workunit.client.1.smithi148.stdout:0/674: fsync db/de/df/d11/d2d/f36 0 2024-03-29T08:07:24.226 INFO:tasks.workunit.client.1.smithi148.stdout:7/626: creat d8/d38/d3c/da9/fdd x:0 0 0 2024-03-29T08:07:24.229 INFO:tasks.workunit.client.0.smithi037.stdout:6/440: creat d0/d6a/d7b/d77/f8e x:0 0 0 2024-03-29T08:07:24.231 INFO:tasks.workunit.client.1.smithi148.stdout:2/759: rename d7/da/d4c/cab to d7/da/d33/d3c/d31/de1/cf8 0 2024-03-29T08:07:24.234 INFO:tasks.workunit.client.0.smithi037.stdout:4/496: write d3/d4/d15/d1d/db/f12 [2056313,72436] 0 2024-03-29T08:07:24.236 INFO:tasks.workunit.client.0.smithi037.stdout:1/549: dread d2/d40/f43 [0,4194304] 0 2024-03-29T08:07:24.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:24.237+0000 7f4dfb000700 1 -- 172.21.15.37:0/1240753908 --> [v2:172.21.15.148:6828/1134897184,v1:172.21.15.148:6829/1134897184] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4df4073170 con 0x7f4dd806afe0 2024-03-29T08:07:24.237 INFO:tasks.workunit.client.0.smithi037.stdout:6/441: symlink d0/d19/d34/l8f 0 2024-03-29T08:07:24.239 INFO:tasks.workunit.client.1.smithi148.stdout:7/627: dwrite d8/f36 [0,4194304] 0 2024-03-29T08:07:24.240 INFO:tasks.workunit.client.0.smithi037.stdout:8/490: mkdir d5/d10/d2d/d9c/da1 0 2024-03-29T08:07:24.242 INFO:tasks.workunit.client.1.smithi148.stdout:7/628: chown d8/c1d 332028188 1 2024-03-29T08:07:24.244 INFO:tasks.workunit.client.0.smithi037.stdout:9/717: truncate d1/d4/d19/d1e/d71/d89/dbe/fd7 3654645 0 2024-03-29T08:07:24.245 INFO:tasks.workunit.client.1.smithi148.stdout:4/676: mknod df/d2c/d51/d77/d26/cd4 0 2024-03-29T08:07:24.247 INFO:tasks.workunit.client.1.smithi148.stdout:4/677: chown fd 84687892 1 2024-03-29T08:07:24.247 INFO:tasks.workunit.client.1.smithi148.stdout:7/629: creat d8/d29/d44/d49/fde x:0 0 0 2024-03-29T08:07:24.247 INFO:tasks.workunit.client.1.smithi148.stdout:2/760: dwrite d7/f5a [0,4194304] 0 2024-03-29T08:07:24.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:24.244+0000 7f4debe00700 1 -- 172.21.15.37:0/1240753908 <== mgr.14602 v2:172.21.15.148:6828/1134897184 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+341 (secure 0 0 0) 0x7f4df4073170 con 0x7f4dd806afe0 2024-03-29T08:07:24.249 INFO:tasks.workunit.client.1.smithi148.stdout:7/630: fsync d8/d14/da8/fdc 0 2024-03-29T08:07:24.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:24.249+0000 7f4dfb000700 1 -- 172.21.15.37:0/1240753908 >> [v2:172.21.15.148:6828/1134897184,v1:172.21.15.148:6829/1134897184] conn(0x7f4dd806afe0 msgr2=0x7f4dd806d490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:07:24.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:24.249+0000 7f4dfb000700 1 --2- 172.21.15.37:0/1240753908 >> [v2:172.21.15.148:6828/1134897184,v1:172.21.15.148:6829/1134897184] conn(0x7f4dd806afe0 0x7f4dd806d490 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f4de4004d00 tx=0x7f4de4009ce0 comp rx=0 tx=0).stop 2024-03-29T08:07:24.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:24.249+0000 7f4dfb000700 1 -- 172.21.15.37:0/1240753908 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4df4072820 msgr2=0x7f4df4113de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:07:24.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:24.249+0000 7f4dfb000700 1 --2- 172.21.15.37:0/1240753908 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4df4072820 0x7f4df4113de0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f4ddc000c00 tx=0x7f4ddc009e30 comp rx=0 tx=0).stop 2024-03-29T08:07:24.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:24.249+0000 7f4dfb000700 1 -- 172.21.15.37:0/1240753908 shutdown_connections 2024-03-29T08:07:24.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:24.249+0000 7f4dfb000700 1 --2- 172.21.15.37:0/1240753908 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4df4072820 0x7f4df4113de0 secure :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f4ddc000c00 tx=0x7f4ddc009e30 comp rx=0 tx=0).stop 2024-03-29T08:07:24.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:24.249+0000 7f4dfb000700 1 --2- 172.21.15.37:0/1240753908 >> [v2:172.21.15.148:6828/1134897184,v1:172.21.15.148:6829/1134897184] conn(0x7f4dd806afe0 0x7f4dd806d490 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:07:24.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:24.249+0000 7f4dfb000700 1 --2- 172.21.15.37:0/1240753908 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4df4114320 0x7f4df4114fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:07:24.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:24.249+0000 7f4dfb000700 1 -- 172.21.15.37:0/1240753908 >> 172.21.15.37:0/1240753908 conn(0x7f4df40fa950 msgr2=0x7f4df40fcdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:07:24.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:24.249+0000 7f4dfb000700 1 -- 172.21.15.37:0/1240753908 shutdown_connections 2024-03-29T08:07:24.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:24.250+0000 7f4dfb000700 1 -- 172.21.15.37:0/1240753908 wait complete. 2024-03-29T08:07:24.251 INFO:tasks.workunit.client.0.smithi037.stdout:6/442: readlink d0/d4f/l88 0 2024-03-29T08:07:24.255 INFO:tasks.workunit.client.1.smithi148.stdout:7/631: symlink d8/d38/d67/ldf 0 2024-03-29T08:07:24.257 INFO:tasks.workunit.client.0.smithi037.stdout:7/492: dread dd/d21/d35/d42/d5c/f69 [0,4194304] 0 2024-03-29T08:07:24.260 INFO:tasks.workunit.client.1.smithi148.stdout:1/624: sync 2024-03-29T08:07:24.260 INFO:tasks.workunit.client.1.smithi148.stdout:8/665: sync 2024-03-29T08:07:24.261 INFO:tasks.workunit.client.1.smithi148.stdout:0/675: write db/de/df/d11/d2d/d3e/f6d [868559,93619] 0 2024-03-29T08:07:24.262 INFO:tasks.workunit.client.1.smithi148.stdout:3/722: creat de/d24/d33/d39/d46/d30/d37/d62/d63/dbf/ff0 x:0 0 0 2024-03-29T08:07:24.264 INFO:tasks.workunit.client.1.smithi148.stdout:7/632: mkdir d8/d29/db8/de0 0 2024-03-29T08:07:24.265 INFO:tasks.workunit.client.1.smithi148.stdout:8/666: dread - d17/d49/d9a/dbe/d6f/d64/fd9 zero size 2024-03-29T08:07:24.267 INFO:tasks.workunit.client.1.smithi148.stdout:8/667: write d17/d74/d86/f8d [4775231,309] 0 2024-03-29T08:07:24.268 INFO:tasks.workunit.client.0.smithi037.stdout:6/443: symlink d0/d6a/d7b/d77/l90 0 2024-03-29T08:07:24.269 INFO:tasks.workunit.client.0.smithi037.stdout:7/493: write dd/d1d/d49/d81/d84/f8a [881507,5017] 0 2024-03-29T08:07:24.271 INFO:tasks.workunit.client.0.smithi037.stdout:0/388: dread d9/da/dd/f23 [0,4194304] 0 2024-03-29T08:07:24.274 INFO:tasks.workunit.client.0.smithi037.stdout:7/494: write dd/d21/d35/f8f [740803,128720] 0 2024-03-29T08:07:24.275 INFO:tasks.workunit.client.1.smithi148.stdout:6/672: link d6/d9/d1e/d36/d4d/d9d/d28/d98/lcd d6/d9/d68/ld8 0 2024-03-29T08:07:24.279 INFO:teuthology.orchestra.run.smithi037.stdout:true 2024-03-29T08:07:24.280 INFO:tasks.workunit.client.0.smithi037.stdout:3/474: rename dd/d46/d2d/d3a/d6c/d6f to dd/d15/d18/d4b/da2 0 2024-03-29T08:07:24.284 INFO:tasks.workunit.client.0.smithi037.stdout:8/491: dwrite d5/f1b [4194304,4194304] 0 2024-03-29T08:07:24.284 INFO:tasks.workunit.client.1.smithi148.stdout:5/653: write d0/db/d1e/d35/d42/d67/d76/fba [514301,11977] 0 2024-03-29T08:07:24.286 INFO:tasks.workunit.client.1.smithi148.stdout:4/678: readlink df/d2c/d51/d77/d26/d34/d3e/l66 0 2024-03-29T08:07:24.288 INFO:tasks.workunit.client.0.smithi037.stdout:9/718: mknod d1/d4/d95/cf9 0 2024-03-29T08:07:24.288 INFO:tasks.workunit.client.1.smithi148.stdout:4/679: dread - df/d2c/d51/d77/d26/f7d zero size 2024-03-29T08:07:24.290 INFO:tasks.workunit.client.1.smithi148.stdout:0/676: creat db/de/df/d11/d2d/d3e/fce x:0 0 0 2024-03-29T08:07:24.292 INFO:tasks.workunit.client.1.smithi148.stdout:6/673: mknod d6/d9/d1e/d36/d4d/d9d/d28/d98/dc2/cd9 0 2024-03-29T08:07:24.292 INFO:tasks.workunit.client.1.smithi148.stdout:0/677: read - db/dc/daa/d47/d7c/fc4 zero size 2024-03-29T08:07:24.294 INFO:tasks.workunit.client.1.smithi148.stdout:5/654: dread - d0/db/d7b/d16/d2e/fa0 zero size 2024-03-29T08:07:24.296 INFO:tasks.workunit.client.1.smithi148.stdout:3/723: symlink de/d24/d33/d39/d46/lf1 0 2024-03-29T08:07:24.297 INFO:tasks.workunit.client.0.smithi037.stdout:7/495: mknod dd/d68/ca1 0 2024-03-29T08:07:24.298 INFO:tasks.workunit.client.1.smithi148.stdout:7/633: write d8/d14/d45/d4d/d78/fc2 [1151182,19608] 0 2024-03-29T08:07:24.301 INFO:tasks.workunit.client.1.smithi148.stdout:5/655: dread - d0/db/d1e/d36/d41/f59 zero size 2024-03-29T08:07:24.301 INFO:tasks.workunit.client.0.smithi037.stdout:8/492: creat d5/d10/d2d/d34/d50/fa2 x:0 0 0 2024-03-29T08:07:24.302 INFO:tasks.workunit.client.0.smithi037.stdout:4/497: creat d3/d4/d15/d1d/db/d45/d1e/d59/d60/d6f/fa4 x:0 0 0 2024-03-29T08:07:24.304 INFO:tasks.workunit.client.0.smithi037.stdout:1/550: dwrite d2/d6/dc/d5d/d62/f97 [0,4194304] 0 2024-03-29T08:07:24.305 INFO:tasks.workunit.client.0.smithi037.stdout:3/475: truncate dd/d15/d27/f41 1003025 0 2024-03-29T08:07:24.307 INFO:tasks.workunit.client.1.smithi148.stdout:3/724: rmdir de/d24/d33/d39/d46/d30/d37/d79 39 2024-03-29T08:07:24.307 INFO:tasks.workunit.client.1.smithi148.stdout:9/711: sync 2024-03-29T08:07:24.311 INFO:tasks.workunit.client.1.smithi148.stdout:1/625: dwrite d3/de/d25/d51/f54 [0,4194304] 0 2024-03-29T08:07:24.316 INFO:tasks.workunit.client.0.smithi037.stdout:7/496: unlink dd/d1d/c28 0 2024-03-29T08:07:24.318 INFO:tasks.workunit.client.1.smithi148.stdout:6/674: dwrite d6/d9/d1e/d20/d30/f93 [0,4194304] 0 2024-03-29T08:07:24.322 INFO:tasks.workunit.client.0.smithi037.stdout:7/497: fdatasync dd/d21/d35/d42/f8e 0 2024-03-29T08:07:24.322 INFO:tasks.workunit.client.1.smithi148.stdout:9/712: mknod db/d22/d26/d54/d52/d9c/cf3 0 2024-03-29T08:07:24.323 INFO:tasks.workunit.client.1.smithi148.stdout:1/626: symlink d3/d18/d4d/d76/lbe 0 2024-03-29T08:07:24.323 INFO:tasks.workunit.client.1.smithi148.stdout:6/675: stat d6/d9/d1e/d20/d30/fac 0 2024-03-29T08:07:24.328 INFO:tasks.workunit.client.1.smithi148.stdout:5/656: dwrite d0/db/d1e/d36/d41/f58 [0,4194304] 0 2024-03-29T08:07:24.332 INFO:tasks.workunit.client.1.smithi148.stdout:9/713: fsync db/d22/d26/d54/f49 0 2024-03-29T08:07:24.333 INFO:tasks.workunit.client.0.smithi037.stdout:4/498: unlink d3/d4/d15/d1d/db/c41 0 2024-03-29T08:07:24.333 INFO:tasks.workunit.client.1.smithi148.stdout:5/657: write d0/f9 [3106484,61073] 0 2024-03-29T08:07:24.338 INFO:tasks.workunit.client.1.smithi148.stdout:3/725: dread de/d24/d33/d39/d46/d30/d37/d62/d63/dbf/fd2 [0,4194304] 0 2024-03-29T08:07:24.343 INFO:tasks.workunit.client.0.smithi037.stdout:7/498: mkdir dd/d21/d65/da2 0 2024-03-29T08:07:24.347 INFO:tasks.workunit.client.1.smithi148.stdout:9/714: mknod db/d10/d2d/d4e/d5f/cf4 0 2024-03-29T08:07:24.348 INFO:tasks.workunit.client.1.smithi148.stdout:5/658: getdents d0/d46/d9e 0 2024-03-29T08:07:24.350 INFO:tasks.workunit.client.1.smithi148.stdout:1/627: link d3/d55/c8b d3/de/d1d/d47/d6d/d72/cbf 0 2024-03-29T08:07:24.351 INFO:tasks.workunit.client.1.smithi148.stdout:5/659: stat d0/db/d1e/fc5 0 2024-03-29T08:07:24.353 INFO:tasks.workunit.client.1.smithi148.stdout:6/676: rename d6/d9/d1e/d27/d48/d73/d7e to d6/d9/d1e/d36/d4d/d9d/d28/dda 0 2024-03-29T08:07:24.357 INFO:tasks.workunit.client.1.smithi148.stdout:6/677: mknod d6/d9/d1e/d27/d48/d9f/cdb 0 2024-03-29T08:07:24.358 INFO:tasks.workunit.client.0.smithi037.stdout:8/493: dwrite d5/d10/d19/d3d/d4a/d4c/f8f [0,4194304] 0 2024-03-29T08:07:24.359 INFO:tasks.workunit.client.1.smithi148.stdout:5/660: truncate d0/db/d7b/fb6 854457 0 2024-03-29T08:07:24.363 INFO:tasks.workunit.client.1.smithi148.stdout:1/628: creat d3/de/fc0 x:0 0 0 2024-03-29T08:07:24.365 INFO:tasks.workunit.client.1.smithi148.stdout:6/678: write d6/d9/d1e/d36/d4d/d9d/d28/dda/fbe [6347350,7798] 0 2024-03-29T08:07:24.368 INFO:tasks.workunit.client.1.smithi148.stdout:9/715: dwrite db/d22/d26/d54/d4d/d35/d4b/d58/f92 [0,4194304] 0 2024-03-29T08:07:24.371 INFO:tasks.workunit.client.1.smithi148.stdout:1/629: creat d3/de/d21/fc1 x:0 0 0 2024-03-29T08:07:24.372 INFO:tasks.workunit.client.0.smithi037.stdout:3/476: dread fc [0,4194304] 0 2024-03-29T08:07:24.377 INFO:tasks.workunit.client.1.smithi148.stdout:9/716: mkdir db/d10/d6d/d73/da8/deb/df5 0 2024-03-29T08:07:24.382 INFO:tasks.workunit.client.1.smithi148.stdout:6/679: dread d6/d9/d1e/d27/d48/d9f/faa [0,4194304] 0 2024-03-29T08:07:24.383 INFO:tasks.workunit.client.1.smithi148.stdout:1/630: rename d3/de/d21/d5e to d3/de/d25/d35/dc2 0 2024-03-29T08:07:24.384 INFO:tasks.workunit.client.1.smithi148.stdout:5/661: dwrite d0/db/d1e/d35/d42/d67/d76/fba [0,4194304] 0 2024-03-29T08:07:24.387 INFO:tasks.workunit.client.1.smithi148.stdout:4/680: write df/d2c/d51/d77/d17/d73/f7a [1206926,104150] 0 2024-03-29T08:07:24.387 INFO:tasks.workunit.client.1.smithi148.stdout:6/680: readlink d6/d9/d1e/d20/l54 0 2024-03-29T08:07:24.387 INFO:tasks.workunit.client.0.smithi037.stdout:3/477: creat dd/d46/d3e/fa3 x:0 0 0 2024-03-29T08:07:24.388 INFO:tasks.workunit.client.1.smithi148.stdout:5/662: chown d0/db/d1e/d5b/d66/d72 122 1 2024-03-29T08:07:24.391 INFO:tasks.workunit.client.1.smithi148.stdout:4/681: write df/d2c/d51/d77/f83 [531841,121753] 0 2024-03-29T08:07:24.392 INFO:tasks.workunit.client.0.smithi037.stdout:8/494: creat d5/d10/d2d/d9c/da1/fa3 x:0 0 0 2024-03-29T08:07:24.392 INFO:tasks.workunit.client.1.smithi148.stdout:1/631: creat d3/de/d1d/d47/fc3 x:0 0 0 2024-03-29T08:07:24.394 INFO:tasks.workunit.client.0.smithi037.stdout:3/478: fdatasync dd/d46/d3e/d47/f7f 0 2024-03-29T08:07:24.396 INFO:tasks.workunit.client.0.smithi037.stdout:8/495: write d5/f32 [1464896,97770] 0 2024-03-29T08:07:24.398 INFO:tasks.workunit.client.1.smithi148.stdout:2/761: dwrite d7/da/d33/ded/f16 [0,4194304] 0 2024-03-29T08:07:24.399 INFO:tasks.workunit.client.0.smithi037.stdout:1/551: dread d2/d6/d13/f77 [0,4194304] 0 2024-03-29T08:07:24.399 INFO:tasks.workunit.client.1.smithi148.stdout:5/663: mknod d0/db/d7b/cd9 0 2024-03-29T08:07:24.402 INFO:tasks.workunit.client.0.smithi037.stdout:4/499: getdents d3/d4/d15 0 2024-03-29T08:07:24.405 INFO:tasks.workunit.client.1.smithi148.stdout:4/682: mkdir df/d2c/d51/d77/d26/d34/db3/dd5 0 2024-03-29T08:07:24.406 INFO:tasks.workunit.client.1.smithi148.stdout:2/762: symlink d7/da/d52/lf9 0 2024-03-29T08:07:24.408 INFO:tasks.workunit.client.0.smithi037.stdout:3/479: creat dd/d15/d27/fa4 x:0 0 0 2024-03-29T08:07:24.410 INFO:tasks.workunit.client.1.smithi148.stdout:5/664: readlink d0/db/d84/daf/lc0 0 2024-03-29T08:07:24.411 INFO:tasks.workunit.client.1.smithi148.stdout:8/668: truncate d17/d74/d86/f8d 4291425 0 2024-03-29T08:07:24.413 INFO:tasks.workunit.client.0.smithi037.stdout:5/603: sync 2024-03-29T08:07:24.415 INFO:tasks.workunit.client.1.smithi148.stdout:5/665: fsync d0/db/d7b/d16/d2e/f56 0 2024-03-29T08:07:24.419 INFO:tasks.workunit.client.1.smithi148.stdout:5/666: chown d0/db/d84/daf/lb8 24 1 2024-03-29T08:07:24.420 INFO:tasks.workunit.client.1.smithi148.stdout:1/632: dwrite d3/d18/d4d/d76/d85/fb0 [0,4194304] 0 2024-03-29T08:07:24.421 INFO:tasks.workunit.client.1.smithi148.stdout:7/634: sync 2024-03-29T08:07:24.424 INFO:tasks.workunit.client.1.smithi148.stdout:6/681: dread d6/d9/d1e/d36/f6f [0,4194304] 0 2024-03-29T08:07:24.424 INFO:tasks.workunit.client.1.smithi148.stdout:0/678: truncate db/dc/daa/d47/f96 786054 0 2024-03-29T08:07:24.427 INFO:tasks.workunit.client.1.smithi148.stdout:9/717: dread db/d10/d6d/fd2 [0,4194304] 0 2024-03-29T08:07:24.430 INFO:tasks.workunit.client.0.smithi037.stdout:4/500: write d3/d4/d15/d89/f94 [2289409,44509] 0 2024-03-29T08:07:24.430 INFO:tasks.workunit.client.1.smithi148.stdout:8/669: rename d17/d18/d26/f98 to d17/d49/d9a/fe2 0 2024-03-29T08:07:24.431 INFO:tasks.workunit.client.1.smithi148.stdout:4/683: link df/d2c/d51/d77/d17/l7e df/d2c/d51/d77/d26/d34/d8a/d98/ld6 0 2024-03-29T08:07:24.435 INFO:tasks.workunit.client.1.smithi148.stdout:4/684: dread - df/d2c/d80/fb5 zero size 2024-03-29T08:07:24.436 INFO:tasks.workunit.client.0.smithi037.stdout:3/480: chown dd/l10 230361 1 2024-03-29T08:07:24.441 INFO:tasks.workunit.client.0.smithi037.stdout:5/604: mknod d8/d22/d37/d60/d61/ccd 0 2024-03-29T08:07:24.443 INFO:tasks.workunit.client.1.smithi148.stdout:5/667: link d0/db/d1e/d36/d41/f9f d0/db/d1e/d5b/fda 0 2024-03-29T08:07:24.446 INFO:tasks.workunit.client.1.smithi148.stdout:8/670: creat d17/d18/d26/d4f/d94/fe3 x:0 0 0 2024-03-29T08:07:24.447 INFO:tasks.workunit.client.0.smithi037.stdout:1/552: fdatasync d2/d6/d13/f28 0 2024-03-29T08:07:24.447 INFO:tasks.workunit.client.1.smithi148.stdout:6/682: creat d6/d9/fdc x:0 0 0 2024-03-29T08:07:24.447 INFO:tasks.workunit.client.1.smithi148.stdout:9/718: dwrite db/d10/d2d/f34 [0,4194304] 0 2024-03-29T08:07:24.448 INFO:tasks.workunit.client.1.smithi148.stdout:3/726: dread de/d24/d33/d39/d46/f23 [0,4194304] 0 2024-03-29T08:07:24.450 INFO:tasks.workunit.client.0.smithi037.stdout:1/553: fsync d2/fb8 0 2024-03-29T08:07:24.452 INFO:tasks.workunit.client.1.smithi148.stdout:9/719: stat db/d22/d26/d54/d4d/d35/d4b/d58/f8b 0 2024-03-29T08:07:24.453 INFO:tasks.workunit.client.1.smithi148.stdout:4/685: creat df/d2c/d80/daa/dbc/fd7 x:0 0 0 2024-03-29T08:07:24.454 INFO:tasks.workunit.client.1.smithi148.stdout:5/668: stat d0/db/d7b/l18 0 2024-03-29T08:07:24.457 INFO:tasks.workunit.client.0.smithi037.stdout:7/499: truncate dd/d21/f75 1010763 0 2024-03-29T08:07:24.459 INFO:tasks.workunit.client.1.smithi148.stdout:6/683: mkdir d6/d9/d1e/d27/d48/d73/ddd 0 2024-03-29T08:07:24.460 INFO:tasks.workunit.client.1.smithi148.stdout:0/679: truncate db/de/df/d11/d42/f52 3552349 0 2024-03-29T08:07:24.463 INFO:tasks.workunit.client.0.smithi037.stdout:4/501: write d3/d57/d6a/f77 [699208,39656] 0 2024-03-29T08:07:24.463 INFO:tasks.workunit.client.1.smithi148.stdout:0/680: write db/de/df/d11/d42/f67 [7776328,121171] 0 2024-03-29T08:07:24.465 INFO:tasks.workunit.client.1.smithi148.stdout:8/671: unlink d17/d49/f6d 0 2024-03-29T08:07:24.467 INFO:tasks.workunit.client.1.smithi148.stdout:4/686: mkdir df/d2c/d51/d77/d26/d34/d3e/dd8 0 2024-03-29T08:07:24.470 INFO:tasks.workunit.client.1.smithi148.stdout:4/687: write df/d2c/d51/d77/f83 [403246,47119] 0 2024-03-29T08:07:24.475 INFO:tasks.workunit.client.1.smithi148.stdout:5/669: truncate d0/db/f3e 115128 0 2024-03-29T08:07:24.476 INFO:tasks.workunit.client.1.smithi148.stdout:3/727: link de/d3b/l4f de/d3b/d49/lf2 0 2024-03-29T08:07:24.477 INFO:tasks.workunit.client.1.smithi148.stdout:8/672: chown d17/d18/dbd/ld4 94105697 1 2024-03-29T08:07:24.477 INFO:tasks.workunit.client.1.smithi148.stdout:7/635: dread d8/d14/da8/fc5 [0,4194304] 0 2024-03-29T08:07:24.477 INFO:tasks.workunit.client.1.smithi148.stdout:0/681: truncate db/dc/daa/d47/f2c 5451471 0 2024-03-29T08:07:24.480 INFO:tasks.workunit.client.1.smithi148.stdout:6/684: rmdir d6/d9/d1e/d36/d4d/d9d/d28/dba/dc0 0 2024-03-29T08:07:24.482 INFO:tasks.workunit.client.1.smithi148.stdout:8/673: read d17/d4e/f5d [5544276,74787] 0 2024-03-29T08:07:24.482 INFO:tasks.workunit.client.1.smithi148.stdout:5/670: mkdir d0/d46/d9e/ddb 0 2024-03-29T08:07:24.483 INFO:tasks.workunit.client.0.smithi037.stdout:8/496: link d5/f80 d5/d10/d45/d4b/fa4 0 2024-03-29T08:07:24.486 INFO:tasks.workunit.client.1.smithi148.stdout:3/728: rmdir de/d24/d33/d39/d46/d30/d37/d62 39 2024-03-29T08:07:24.487 INFO:tasks.workunit.client.1.smithi148.stdout:4/688: link df/d2c/d51/d77/d26/d34/d43/c87 df/d2c/d51/d77/d26/cd9 0 2024-03-29T08:07:24.489 INFO:tasks.workunit.client.1.smithi148.stdout:5/671: rmdir d0/db/d84 39 2024-03-29T08:07:24.489 INFO:tasks.workunit.client.0.smithi037.stdout:5/605: mkdir d8/d22/d37/d60/d88/dce 0 2024-03-29T08:07:24.490 INFO:tasks.workunit.client.1.smithi148.stdout:6/685: dread - d6/d9/d1e/d36/d4d/d9d/d28/f3b zero size 2024-03-29T08:07:24.491 INFO:tasks.workunit.client.0.smithi037.stdout:4/502: dwrite d3/d4/d15/d1d/db/d45/d1e/f4e [0,4194304] 0 2024-03-29T08:07:24.493 INFO:tasks.workunit.client.1.smithi148.stdout:9/720: write db/fa7 [2383562,123869] 0 2024-03-29T08:07:24.494 INFO:tasks.workunit.client.1.smithi148.stdout:0/682: dread db/dc/daa/d47/d7c/fb4 [0,4194304] 0 2024-03-29T08:07:24.494 INFO:tasks.workunit.client.0.smithi037.stdout:4/503: write d3/f44 [251635,16264] 0 2024-03-29T08:07:24.496 INFO:tasks.workunit.client.0.smithi037.stdout:4/504: chown d3/d4/f2f 290023 1 2024-03-29T08:07:24.497 INFO:tasks.workunit.client.1.smithi148.stdout:7/636: dwrite d8/d14/d45/d4d/f6b [0,4194304] 0 2024-03-29T08:07:24.502 INFO:tasks.workunit.client.0.smithi037.stdout:1/554: truncate d2/d6/dc/d18/d1d/d21/d99/f8c 139892 0 2024-03-29T08:07:24.503 INFO:tasks.workunit.client.0.smithi037.stdout:8/497: fdatasync d5/d10/f92 0 2024-03-29T08:07:24.512 INFO:tasks.workunit.client.0.smithi037.stdout:3/481: getdents dd/d46/d2d 0 2024-03-29T08:07:24.514 INFO:tasks.workunit.client.0.smithi037.stdout:4/505: mknod d3/d4/d15/d1d/db/d45/d93/d96/da0/ca5 0 2024-03-29T08:07:24.515 INFO:tasks.workunit.client.1.smithi148.stdout:5/672: rename d0/f21 to d0/db/d1e/d6e/da8/fdc 0 2024-03-29T08:07:24.515 INFO:tasks.workunit.client.1.smithi148.stdout:6/686: fdatasync d6/d9/d1e/d36/d4d/d9d/d28/dda/fbe 0 2024-03-29T08:07:24.518 INFO:tasks.workunit.client.0.smithi037.stdout:5/606: getdents d8/d22/db6 0 2024-03-29T08:07:24.523 INFO:tasks.workunit.client.1.smithi148.stdout:4/689: getdents df/d2c 0 2024-03-29T08:07:24.525 INFO:tasks.workunit.client.1.smithi148.stdout:8/674: dwrite d17/d74/fb4 [0,4194304] 0 2024-03-29T08:07:24.527 INFO:tasks.workunit.client.0.smithi037.stdout:8/498: dread d5/d10/d45/d4b/fa4 [0,4194304] 0 2024-03-29T08:07:24.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:24.530+0000 7efcdbe00700 1 -- 172.21.15.37:0/1943260071 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efcd40fff00 msgr2=0x7efcd4102320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:07:24.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:24.530+0000 7efcdbe00700 1 --2- 172.21.15.37:0/1943260071 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efcd40fff00 0x7efcd4102320 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7efcc0004940 tx=0x7efcc0009e30 comp rx=0 tx=0).stop 2024-03-29T08:07:24.530 INFO:tasks.workunit.client.1.smithi148.stdout:4/690: chown df/d2c/d51/d77/d26/d34/d8a/f95 1008 1 2024-03-29T08:07:24.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:24.533+0000 7efcdbe00700 1 -- 172.21.15.37:0/1943260071 shutdown_connections 2024-03-29T08:07:24.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:24.533+0000 7efcdbe00700 1 --2- 172.21.15.37:0/1943260071 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efcd40fff00 0x7efcd4102320 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:07:24.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:24.533+0000 7efcdbe00700 1 --2- 172.21.15.37:0/1943260071 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efcd40662e0 0x7efcd40ff9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:07:24.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:24.533+0000 7efcdbe00700 1 -- 172.21.15.37:0/1943260071 >> 172.21.15.37:0/1943260071 conn(0x7efcd40f7430 msgr2=0x7efcd40f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:07:24.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:24.533+0000 7efcdbe00700 1 -- 172.21.15.37:0/1943260071 shutdown_connections 2024-03-29T08:07:24.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:24.534+0000 7efcdbe00700 1 -- 172.21.15.37:0/1943260071 wait complete. 2024-03-29T08:07:24.534 INFO:tasks.workunit.client.0.smithi037.stdout:7/500: dwrite dd/d21/d35/f50 [0,4194304] 0 2024-03-29T08:07:24.534 INFO:tasks.workunit.client.0.smithi037.stdout:4/506: creat d3/d4/d15/d1d/db/d45/d1e/d59/d79/fa6 x:0 0 0 2024-03-29T08:07:24.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:24.535+0000 7efcdbe00700 1 Processor -- start 2024-03-29T08:07:24.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:24.537+0000 7efcdbe00700 1 -- start start 2024-03-29T08:07:24.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:24.537+0000 7efcdbe00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efcd40662e0 0x7efcd410b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:07:24.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:24.537+0000 7efcdbe00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efcd40fff00 0x7efcd410bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:07:24.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:24.537+0000 7efcdbe00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcd410c1e0 con 0x7efcd40fff00 2024-03-29T08:07:24.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:24.537+0000 7efcdbe00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcd410ca30 con 0x7efcd40662e0 2024-03-29T08:07:24.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:24.538+0000 7efcd8a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efcd40fff00 0x7efcd410bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:07:24.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:24.538+0000 7efcd9400700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efcd40662e0 0x7efcd410b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:07:24.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:24.538+0000 7efcd9400700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efcd40662e0 0x7efcd410b650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.37:43850/0 (socket says 172.21.15.37:43850) 2024-03-29T08:07:24.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:24.538+0000 7efcd9400700 1 -- 172.21.15.37:0/3747444018 learned_addr learned my addr 172.21.15.37:0/3747444018 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:07:24.539 INFO:tasks.workunit.client.1.smithi148.stdout:4/691: stat df/d2c/db2 0 2024-03-29T08:07:24.539 INFO:tasks.workunit.client.0.smithi037.stdout:1/555: dwrite d2/d6/f36 [4194304,4194304] 0 2024-03-29T08:07:24.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:24.540+0000 7efcd9400700 1 -- 172.21.15.37:0/3747444018 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efcd40fff00 msgr2=0x7efcd410bb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:07:24.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:24.540+0000 7efcd9400700 1 --2- 172.21.15.37:0/3747444018 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efcd40fff00 0x7efcd410bb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:07:24.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:24.540+0000 7efcd9400700 1 -- 172.21.15.37:0/3747444018 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcc0005040 con 0x7efcd40662e0 2024-03-29T08:07:24.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:24.540+0000 7efcd9400700 1 --2- 172.21.15.37:0/3747444018 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efcd40662e0 0x7efcd410b650 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7efcc800eb40 tx=0x7efcc800b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:07:24.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:24.544+0000 7efcd1600700 1 -- 172.21.15.37:0/3747444018 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efcc8007cc0 con 0x7efcd40662e0 2024-03-29T08:07:24.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:24.544+0000 7efcd1600700 1 -- 172.21.15.37:0/3747444018 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7efcc800f040 con 0x7efcd40662e0 2024-03-29T08:07:24.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:24.544+0000 7efcd1600700 1 -- 172.21.15.37:0/3747444018 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efcc80094d0 con 0x7efcd40662e0 2024-03-29T08:07:24.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:24.544+0000 7efcdbe00700 1 -- 172.21.15.37:0/3747444018 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efcd410ccb0 con 0x7efcd40662e0 2024-03-29T08:07:24.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:24.545+0000 7efcdbe00700 1 -- 172.21.15.37:0/3747444018 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcd410d270 con 0x7efcd40662e0 2024-03-29T08:07:24.547 INFO:tasks.workunit.client.0.smithi037.stdout:7/501: stat dd/le 0 2024-03-29T08:07:24.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:24.547+0000 7efcdbe00700 1 -- 172.21.15.37:0/3747444018 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efcd405f5f0 con 0x7efcd40662e0 2024-03-29T08:07:24.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:24.549+0000 7efcd1600700 1 -- 172.21.15.37:0/3747444018 <== mon.1 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 25) v1 ==== 88766+0+0 (secure 0 0 0) 0x7efcc8009630 con 0x7efcd40662e0 2024-03-29T08:07:24.550 INFO:tasks.workunit.client.0.smithi037.stdout:8/499: creat d5/d10/d45/d4b/fa5 x:0 0 0 2024-03-29T08:07:24.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:24.551+0000 7efcd1600700 1 --2- 172.21.15.37:0/3747444018 >> [v2:172.21.15.148:6828/1134897184,v1:172.21.15.148:6829/1134897184] conn(0x7efcbc06aec0 0x7efcbc06d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:07:24.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:24.551+0000 7efcd1600700 1 -- 172.21.15.37:0/3747444018 <== mon.1 v2:172.21.15.148:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5808+0+0 (secure 0 0 0) 0x7efcc8089350 con 0x7efcd40662e0 2024-03-29T08:07:24.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:24.552+0000 7efcd8a00700 1 --2- 172.21.15.37:0/3747444018 >> [v2:172.21.15.148:6828/1134897184,v1:172.21.15.148:6829/1134897184] conn(0x7efcbc06aec0 0x7efcbc06d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:07:24.552 INFO:tasks.workunit.client.0.smithi037.stdout:7/502: fdatasync dd/d21/f36 0 2024-03-29T08:07:24.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:24.554+0000 7efcd8a00700 1 --2- 172.21.15.37:0/3747444018 >> [v2:172.21.15.148:6828/1134897184,v1:172.21.15.148:6829/1134897184] conn(0x7efcbc06aec0 0x7efcbc06d370 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7efcc000b0d0 tx=0x7efcc0009e30 comp rx=0 tx=0).ready entity=mgr.14602 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:07:24.555 INFO:tasks.workunit.client.1.smithi148.stdout:5/673: dwrite d0/db/d84/daf/fc2 [0,4194304] 0 2024-03-29T08:07:24.555 INFO:tasks.workunit.client.1.smithi148.stdout:9/721: link db/d22/d26/d54/d4d/dbb/dc0/lc7 db/d22/lf6 0 2024-03-29T08:07:24.556 INFO:tasks.workunit.client.1.smithi148.stdout:8/675: link d17/d18/d26/d63/d66/d5f/f8e d17/d18/fe4 0 2024-03-29T08:07:24.557 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:07:24 smithi148 ceph-mon[44992]: from='mgr.14602 ' entity='mgr.smithi148.onhgvm' 2024-03-29T08:07:24.557 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:07:24 smithi148 ceph-mon[44992]: from='mgr.14602 ' entity='mgr.smithi148.onhgvm' 2024-03-29T08:07:24.557 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:07:24 smithi148 ceph-mon[44992]: from='mgr.14602 172.21.15.148:0/667342922' entity='mgr.smithi148.onhgvm' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-29T08:07:24.557 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:07:24 smithi148 ceph-mon[44992]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-29T08:07:24.557 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:07:24 smithi148 ceph-mon[44992]: from='mgr.14602 172.21.15.148:0/667342922' entity='mgr.smithi148.onhgvm' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-29T08:07:24.557 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:07:24 smithi148 ceph-mon[44992]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-29T08:07:24.557 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:07:24 smithi148 ceph-mon[44992]: from='mgr.14602 172.21.15.148:0/667342922' entity='mgr.smithi148.onhgvm' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-29T08:07:24.557 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:07:24 smithi148 ceph-mon[44992]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-29T08:07:24.557 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:07:24 smithi148 ceph-mon[44992]: from='mgr.14602 172.21.15.148:0/667342922' entity='mgr.smithi148.onhgvm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T08:07:24.557 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:07:24 smithi148 ceph-mon[44992]: Upgrade: Need to upgrade myself (mgr.smithi148.onhgvm) 2024-03-29T08:07:24.558 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:07:24 smithi148 ceph-mon[44992]: Upgrade: Need to upgrade myself (mgr.smithi148.onhgvm) 2024-03-29T08:07:24.558 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:07:24 smithi148 ceph-mon[44992]: pgmap v44: 65 pgs: 65 active+clean; 2.4 GiB data, 8.5 GiB used, 528 GiB / 536 GiB avail; 99 MiB/s rd, 308 MiB/s wr, 582 op/s 2024-03-29T08:07:24.558 INFO:tasks.workunit.client.0.smithi037.stdout:7/503: symlink dd/d72/la3 0 2024-03-29T08:07:24.558 INFO:tasks.workunit.client.1.smithi148.stdout:6/687: dwrite d6/d9/d1e/d36/d4d/d9d/d28/d98/dc2/fc8 [0,4194304] 0 2024-03-29T08:07:24.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:24.560+0000 7efcd1600700 1 -- 172.21.15.37:0/3747444018 <== mon.1 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181579 (secure 0 0 0) 0x7efcc804f560 con 0x7efcd40662e0 2024-03-29T08:07:24.562 INFO:tasks.workunit.client.1.smithi148.stdout:2/763: sync 2024-03-29T08:07:24.562 INFO:tasks.workunit.client.1.smithi148.stdout:8/676: write d17/d18/f5c [3339280,91468] 0 2024-03-29T08:07:24.562 INFO:tasks.workunit.client.1.smithi148.stdout:9/722: readlink db/d10/d2d/lbf 0 2024-03-29T08:07:24.563 INFO:tasks.workunit.client.0.smithi037.stdout:1/556: read d2/d6/dc/d18/d22/d4f/f73 [166139,110154] 0 2024-03-29T08:07:24.566 INFO:tasks.workunit.client.1.smithi148.stdout:0/683: dread db/dc/daa/d47/daf/fb7 [0,4194304] 0 2024-03-29T08:07:24.566 INFO:tasks.workunit.client.1.smithi148.stdout:6/688: write d6/d9/d1e/d36/f87 [1916497,53767] 0 2024-03-29T08:07:24.567 INFO:tasks.workunit.client.1.smithi148.stdout:7/637: dread f4 [0,4194304] 0 2024-03-29T08:07:24.567 INFO:tasks.workunit.client.1.smithi148.stdout:9/723: truncate db/d10/faf 1213825 0 2024-03-29T08:07:24.570 INFO:tasks.workunit.client.1.smithi148.stdout:6/689: dread - d6/fb0 zero size 2024-03-29T08:07:24.570 INFO:tasks.workunit.client.1.smithi148.stdout:8/677: truncate d17/d18/f1c 728736 0 2024-03-29T08:07:24.571 INFO:tasks.workunit.client.0.smithi037.stdout:8/500: link d5/d10/d19/d3d/d4a/d4c/d67/l8c d5/d10/d2d/d9c/la6 0 2024-03-29T08:07:24.571 INFO:tasks.workunit.client.1.smithi148.stdout:4/692: dread df/d2c/d51/d77/d26/d34/d43/f4e [0,4194304] 0 2024-03-29T08:07:24.572 INFO:tasks.workunit.client.1.smithi148.stdout:5/674: fdatasync d0/db/d1e/d5b/d66/d72/f99 0 2024-03-29T08:07:24.576 INFO:tasks.workunit.client.1.smithi148.stdout:5/675: chown d0/db/d1e/d35/d42/d67/c71 677264526 1 2024-03-29T08:07:24.576 INFO:tasks.workunit.client.1.smithi148.stdout:7/638: rmdir d8/d38/d6a 39 2024-03-29T08:07:24.578 INFO:tasks.workunit.client.0.smithi037.stdout:7/504: truncate dd/d21/d35/d42/f8e 2130459 0 2024-03-29T08:07:24.580 INFO:tasks.workunit.client.0.smithi037.stdout:1/557: symlink d2/d6/dc/d18/d22/d4f/d7c/lbf 0 2024-03-29T08:07:24.581 INFO:tasks.workunit.client.1.smithi148.stdout:4/693: read df/d2c/d51/d77/d26/d34/d8a/f95 [442839,14178] 0 2024-03-29T08:07:24.581 INFO:tasks.workunit.client.0.smithi037.stdout:8/501: mkdir d5/d10/d2d/da7 0 2024-03-29T08:07:24.583 INFO:tasks.workunit.client.1.smithi148.stdout:8/678: rename d17/d18/d26/f27 to d17/d74/d86/fe5 0 2024-03-29T08:07:24.586 INFO:tasks.workunit.client.0.smithi037.stdout:5/607: dwrite d8/d18/f40 [0,4194304] 0 2024-03-29T08:07:24.588 INFO:tasks.workunit.client.1.smithi148.stdout:8/679: dread - d17/d49/d9a/dbe/d6f/d64/da5/fb2 zero size 2024-03-29T08:07:24.588 INFO:tasks.workunit.client.1.smithi148.stdout:5/676: mkdir d0/db/d7b/d16/d2e/ddd 0 2024-03-29T08:07:24.589 INFO:tasks.workunit.client.0.smithi037.stdout:1/558: creat d2/d6/dc/d58/fc0 x:0 0 0 2024-03-29T08:07:24.591 INFO:tasks.workunit.client.0.smithi037.stdout:7/505: read dd/d21/d35/d60/f7f [351645,50155] 0 2024-03-29T08:07:24.591 INFO:tasks.workunit.client.0.smithi037.stdout:8/502: chown d5/d10/d2d/d34/d50/f54 624 1 2024-03-29T08:07:24.594 INFO:tasks.workunit.client.0.smithi037.stdout:7/506: fsync dd/f16 0 2024-03-29T08:07:24.594 INFO:tasks.workunit.client.0.smithi037.stdout:8/503: readlink d5/d10/d45/d4b/l52 0 2024-03-29T08:07:24.594 INFO:tasks.workunit.client.0.smithi037.stdout:3/482: dread dd/d15/d27/d33/f96 [0,4194304] 0 2024-03-29T08:07:24.599 INFO:tasks.workunit.client.0.smithi037.stdout:1/559: read - d2/d6/dc/d18/d1d/d21/d99/fbe zero size 2024-03-29T08:07:24.600 INFO:tasks.workunit.client.1.smithi148.stdout:0/684: dwrite db/de/d8e/da9/fca [0,4194304] 0 2024-03-29T08:07:24.601 INFO:tasks.workunit.client.0.smithi037.stdout:8/504: write d5/f1e [1516770,101450] 0 2024-03-29T08:07:24.602 INFO:tasks.workunit.client.0.smithi037.stdout:5/608: truncate d8/d34/f6a 160670 0 2024-03-29T08:07:24.604 INFO:tasks.workunit.client.1.smithi148.stdout:9/724: dwrite db/d22/d26/d54/d4d/d35/d4b/dc3/fe9 [0,4194304] 0 2024-03-29T08:07:24.605 INFO:tasks.workunit.client.0.smithi037.stdout:3/483: symlink dd/d46/d2d/la5 0 2024-03-29T08:07:24.607 INFO:tasks.workunit.client.1.smithi148.stdout:7/639: dwrite d8/d38/d3c/da5/fad [0,4194304] 0 2024-03-29T08:07:24.614 INFO:tasks.workunit.client.0.smithi037.stdout:4/507: dread d3/d4/f3e [0,4194304] 0 2024-03-29T08:07:24.617 INFO:tasks.workunit.client.1.smithi148.stdout:5/677: creat d0/db/d1e/d35/d42/fde x:0 0 0 2024-03-29T08:07:24.619 INFO:tasks.workunit.client.0.smithi037.stdout:8/505: fdatasync d5/f15 0 2024-03-29T08:07:24.620 INFO:tasks.workunit.client.1.smithi148.stdout:6/690: link d6/d9/d1e/d27/c2e d6/d9/d1e/d36/d4d/d9d/d7d/cde 0 2024-03-29T08:07:24.621 INFO:tasks.workunit.client.1.smithi148.stdout:7/640: symlink d8/d38/d67/le1 0 2024-03-29T08:07:24.622 INFO:tasks.workunit.client.1.smithi148.stdout:7/641: stat d8/d14/c32 0 2024-03-29T08:07:24.623 INFO:tasks.workunit.client.1.smithi148.stdout:9/725: creat db/d10/d2d/d4e/d5f/d6e/d7a/ff7 x:0 0 0 2024-03-29T08:07:24.625 INFO:tasks.workunit.client.1.smithi148.stdout:7/642: dread - d8/d14/da8/fdc zero size 2024-03-29T08:07:24.627 INFO:tasks.workunit.client.1.smithi148.stdout:6/691: read d6/d9/d1e/d27/d48/f94 [1118503,113052] 0 2024-03-29T08:07:24.628 INFO:tasks.workunit.client.1.smithi148.stdout:6/692: stat d6/d9/d1e/d27/d48/d73/ddd 0 2024-03-29T08:07:24.633 INFO:tasks.workunit.client.0.smithi037.stdout:4/508: symlink d3/d4/d15/d1d/db/d72/la7 0 2024-03-29T08:07:24.633 INFO:tasks.workunit.client.0.smithi037.stdout:1/560: dread d2/d6/dc/d18/d1d/d21/d99/f8c [0,4194304] 0 2024-03-29T08:07:24.634 INFO:tasks.workunit.client.1.smithi148.stdout:4/694: rename df/d60/l64 to df/d2c/d51/d77/lda 0 2024-03-29T08:07:24.640 INFO:tasks.workunit.client.0.smithi037.stdout:8/506: unlink d5/f1b 0 2024-03-29T08:07:24.645 INFO:tasks.workunit.client.1.smithi148.stdout:6/693: symlink d6/d9/d1e/d36/db1/ldf 0 2024-03-29T08:07:24.648 INFO:tasks.workunit.client.0.smithi037.stdout:1/561: mknod d2/d4b/da1/cc1 0 2024-03-29T08:07:24.649 INFO:tasks.workunit.client.1.smithi148.stdout:4/695: unlink df/d2c/d51/d77/d26/d34/d3e/f3f 0 2024-03-29T08:07:24.654 INFO:tasks.workunit.client.0.smithi037.stdout:4/509: rmdir d3/d4/d15/d1d/db/d45/d1e/d59/d60 39 2024-03-29T08:07:24.655 INFO:tasks.workunit.client.1.smithi148.stdout:7/643: creat d8/d38/d6a/d91/fe2 x:0 0 0 2024-03-29T08:07:24.658 INFO:tasks.workunit.client.1.smithi148.stdout:7/644: fdatasync d8/d38/d3c/da9/d37/f48 0 2024-03-29T08:07:24.662 INFO:tasks.workunit.client.1.smithi148.stdout:9/726: creat db/d22/d26/d54/d4d/d35/d4b/dc3/ff8 x:0 0 0 2024-03-29T08:07:24.662 INFO:tasks.workunit.client.0.smithi037.stdout:7/507: dread dd/f14 [0,4194304] 0 2024-03-29T08:07:24.664 INFO:tasks.workunit.client.1.smithi148.stdout:6/694: link d6/d9/d1e/d36/d4d/d9d/d28/d98/f9c d6/d9/d1e/d36/db1/fe0 0 2024-03-29T08:07:24.665 INFO:tasks.workunit.client.1.smithi148.stdout:9/727: chown db/d22/d26/d54/f2e 1 1 2024-03-29T08:07:24.666 INFO:tasks.workunit.client.1.smithi148.stdout:6/695: readlink d6/l82 0 2024-03-29T08:07:24.666 INFO:tasks.workunit.client.0.smithi037.stdout:1/562: fsync d2/d6/dc/fd 0 2024-03-29T08:07:24.667 INFO:tasks.workunit.client.1.smithi148.stdout:5/678: dwrite d0/db/d7b/d16/d2e/fa0 [0,4194304] 0 2024-03-29T08:07:24.669 INFO:tasks.workunit.client.1.smithi148.stdout:7/645: mkdir d8/d38/d3c/da5/de3 0 2024-03-29T08:07:24.671 INFO:tasks.workunit.client.1.smithi148.stdout:3/729: sync 2024-03-29T08:07:24.671 INFO:tasks.workunit.client.1.smithi148.stdout:1/633: sync 2024-03-29T08:07:24.671 INFO:tasks.workunit.client.1.smithi148.stdout:4/696: rename df/d2c/d51/d77/f53 to df/fdb 0 2024-03-29T08:07:24.671 INFO:tasks.workunit.client.1.smithi148.stdout:0/685: truncate db/de/df/d11/d2d/f85 1018215 0 2024-03-29T08:07:24.673 INFO:tasks.workunit.client.1.smithi148.stdout:9/728: read - db/d22/d26/d54/d4d/d35/d4b/fc8 zero size 2024-03-29T08:07:24.674 INFO:tasks.workunit.client.1.smithi148.stdout:8/680: dwrite d17/d18/fe4 [0,4194304] 0 2024-03-29T08:07:24.675 INFO:tasks.workunit.client.1.smithi148.stdout:6/696: creat d6/d9/dc5/fe1 x:0 0 0 2024-03-29T08:07:24.676 INFO:tasks.workunit.client.0.smithi037.stdout:5/609: dwrite d8/d22/d4f/d70/f9f [0,4194304] 0 2024-03-29T08:07:24.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:07:24 smithi037 ceph-mon[35162]: from='mgr.14602 ' entity='mgr.smithi148.onhgvm' 2024-03-29T08:07:24.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:07:24 smithi037 ceph-mon[35162]: from='mgr.14602 ' entity='mgr.smithi148.onhgvm' 2024-03-29T08:07:24.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:07:24 smithi037 ceph-mon[35162]: from='mgr.14602 172.21.15.148:0/667342922' entity='mgr.smithi148.onhgvm' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-29T08:07:24.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:07:24 smithi037 ceph-mon[35162]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-29T08:07:24.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:07:24 smithi037 ceph-mon[35162]: from='mgr.14602 172.21.15.148:0/667342922' entity='mgr.smithi148.onhgvm' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-29T08:07:24.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:07:24 smithi037 ceph-mon[35162]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-29T08:07:24.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:07:24 smithi037 ceph-mon[35162]: from='mgr.14602 172.21.15.148:0/667342922' entity='mgr.smithi148.onhgvm' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-29T08:07:24.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:07:24 smithi037 ceph-mon[35162]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-29T08:07:24.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:07:24 smithi037 ceph-mon[35162]: from='mgr.14602 172.21.15.148:0/667342922' entity='mgr.smithi148.onhgvm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T08:07:24.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:07:24 smithi037 ceph-mon[35162]: Upgrade: Need to upgrade myself (mgr.smithi148.onhgvm) 2024-03-29T08:07:24.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:07:24 smithi037 ceph-mon[35162]: Upgrade: Need to upgrade myself (mgr.smithi148.onhgvm) 2024-03-29T08:07:24.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:07:24 smithi037 ceph-mon[35162]: pgmap v44: 65 pgs: 65 active+clean; 2.4 GiB data, 8.5 GiB used, 528 GiB / 536 GiB avail; 99 MiB/s rd, 308 MiB/s wr, 582 op/s 2024-03-29T08:07:24.678 INFO:tasks.workunit.client.1.smithi148.stdout:0/686: dread - db/de/df/d11/d2d/d3e/fce zero size 2024-03-29T08:07:24.680 INFO:tasks.workunit.client.1.smithi148.stdout:7/646: chown d8/c28 51 1 2024-03-29T08:07:24.682 INFO:tasks.workunit.client.1.smithi148.stdout:0/687: write db/de/df/d11/d2d/d3e/fce [697651,9939] 0 2024-03-29T08:07:24.683 INFO:tasks.workunit.client.1.smithi148.stdout:5/679: unlink d0/db/d1e/d35/d42/d67/d76/d81/l97 0 2024-03-29T08:07:24.685 INFO:tasks.workunit.client.0.smithi037.stdout:8/507: link d5/f48 d5/d10/d19/d8d/fa8 0 2024-03-29T08:07:24.685 INFO:tasks.workunit.client.1.smithi148.stdout:1/634: symlink d3/de/d21/d7f/lc4 0 2024-03-29T08:07:24.687 INFO:tasks.workunit.client.1.smithi148.stdout:8/681: symlink d17/d18/d26/d63/d7d/le6 0 2024-03-29T08:07:24.691 INFO:tasks.workunit.client.1.smithi148.stdout:4/697: creat df/d2c/d51/d77/d26/d9b/fdc x:0 0 0 2024-03-29T08:07:24.694 INFO:tasks.workunit.client.0.smithi037.stdout:1/563: fsync d2/d4b/da1/f96 0 2024-03-29T08:07:24.697 INFO:tasks.workunit.client.1.smithi148.stdout:0/688: creat db/de/df/d11/d42/dc2/fcf x:0 0 0 2024-03-29T08:07:24.697 INFO:tasks.workunit.client.1.smithi148.stdout:9/729: symlink db/d10/d6d/d73/da8/deb/dec/lf9 0 2024-03-29T08:07:24.702 INFO:tasks.workunit.client.1.smithi148.stdout:3/730: dwrite de/d24/d33/d39/d46/d30/d37/fc3 [0,4194304] 0 2024-03-29T08:07:24.702 INFO:tasks.workunit.client.1.smithi148.stdout:1/635: dread d3/de/f86 [0,4194304] 0 2024-03-29T08:07:24.703 INFO:tasks.workunit.client.1.smithi148.stdout:4/698: fdatasync df/d2c/d51/d77/d17/fbd 0 2024-03-29T08:07:24.704 INFO:tasks.workunit.client.1.smithi148.stdout:7/647: unlink d8/d14/d45/d4d/ca6 0 2024-03-29T08:07:24.705 INFO:tasks.workunit.client.1.smithi148.stdout:9/730: truncate db/d10/d2d/d4e/f6b 4980171 0 2024-03-29T08:07:24.706 INFO:tasks.workunit.client.1.smithi148.stdout:1/636: dread - d3/d55/d7e/f94 zero size 2024-03-29T08:07:24.706 INFO:tasks.workunit.client.1.smithi148.stdout:5/680: dwrite d0/db/d84/daf/fc2 [0,4194304] 0 2024-03-29T08:07:24.708 INFO:tasks.workunit.client.1.smithi148.stdout:0/689: creat db/de/df/d11/d42/fd0 x:0 0 0 2024-03-29T08:07:24.710 INFO:tasks.workunit.client.1.smithi148.stdout:3/731: write de/d24/d33/d39/da3/dbc/fbe [127061,79806] 0 2024-03-29T08:07:24.710 INFO:tasks.workunit.client.1.smithi148.stdout:4/699: dread - df/d2c/d51/d77/d26/f7d zero size 2024-03-29T08:07:24.712 INFO:tasks.workunit.client.1.smithi148.stdout:7/648: fdatasync f4 0 2024-03-29T08:07:24.713 INFO:tasks.workunit.client.1.smithi148.stdout:8/682: rmdir d17/d49/d9a/dbe/d6f/d64 39 2024-03-29T08:07:24.713 INFO:tasks.workunit.client.0.smithi037.stdout:7/508: symlink dd/la4 0 2024-03-29T08:07:24.713 INFO:tasks.workunit.client.1.smithi148.stdout:5/681: creat d0/db/d1e/d35/d42/d96/fdf x:0 0 0 2024-03-29T08:07:24.715 INFO:tasks.workunit.client.1.smithi148.stdout:3/732: write de/fbd [627000,81073] 0 2024-03-29T08:07:24.718 INFO:tasks.workunit.client.1.smithi148.stdout:1/637: mknod d3/de/d1d/d47/cc5 0 2024-03-29T08:07:24.719 INFO:tasks.workunit.client.1.smithi148.stdout:0/690: mknod db/de/cd1 0 2024-03-29T08:07:24.719 INFO:tasks.workunit.client.1.smithi148.stdout:5/682: chown d0/db/d1e/d36/d41/ccc 3 1 2024-03-29T08:07:24.719 INFO:tasks.workunit.client.0.smithi037.stdout:1/564: dwrite d2/d6/dc/d58/f71 [0,4194304] 0 2024-03-29T08:07:24.720 INFO:tasks.workunit.client.0.smithi037.stdout:5/610: rmdir d8/d22/d4f/d70 39 2024-03-29T08:07:24.723 INFO:tasks.workunit.client.1.smithi148.stdout:6/697: getdents d6/d9/dc5 0 2024-03-29T08:07:24.724 INFO:tasks.workunit.client.1.smithi148.stdout:0/691: write db/dc/daa/d47/daf/fb7 [3631169,83751] 0 2024-03-29T08:07:24.726 INFO:tasks.workunit.client.0.smithi037.stdout:1/565: write d2/d6/dc/d18/d22/fa7 [584295,95266] 0 2024-03-29T08:07:24.729 INFO:tasks.workunit.client.1.smithi148.stdout:8/683: rename d17/d1d/d7e/l80 to d17/d18/d26/d4f/d94/le7 0 2024-03-29T08:07:24.729 INFO:tasks.workunit.client.1.smithi148.stdout:6/698: truncate d6/d9/d1e/f5d 1027791 0 2024-03-29T08:07:24.730 INFO:tasks.workunit.client.1.smithi148.stdout:3/733: chown de/d24/d33/d39/d46/d70/cbb 1739386 1 2024-03-29T08:07:24.735 INFO:tasks.workunit.client.0.smithi037.stdout:4/510: getdents d3 0 2024-03-29T08:07:24.735 INFO:tasks.workunit.client.0.smithi037.stdout:7/509: truncate dd/d21/d35/d42/d5c/f5e 753968 0 2024-03-29T08:07:24.737 INFO:tasks.workunit.client.1.smithi148.stdout:3/734: stat de/c73 0 2024-03-29T08:07:24.740 INFO:tasks.workunit.client.0.smithi037.stdout:4/511: chown d3/d4/d15/d1d/db/d45/d1e/d59/d79 550 1 2024-03-29T08:07:24.740 INFO:tasks.workunit.client.0.smithi037.stdout:5/611: creat d8/d34/fcf x:0 0 0 2024-03-29T08:07:24.741 INFO:tasks.workunit.client.1.smithi148.stdout:1/638: rename d3/de/d25/c68 to d3/de/d25/d35/dc2/cc6 0 2024-03-29T08:07:24.741 INFO:tasks.workunit.client.1.smithi148.stdout:5/683: rmdir d0/d46/d9e/ddb 0 2024-03-29T08:07:24.742 INFO:tasks.workunit.client.0.smithi037.stdout:7/510: truncate dd/d21/d35/d42/d5c/f9d 919497 0 2024-03-29T08:07:24.743 INFO:tasks.workunit.client.1.smithi148.stdout:7/649: dwrite d8/d38/d3c/da9/fdd [0,4194304] 0 2024-03-29T08:07:24.745 INFO:tasks.workunit.client.1.smithi148.stdout:5/684: dread - d0/db/d1e/d5b/d66/d72/f99 zero size 2024-03-29T08:07:24.748 INFO:tasks.workunit.client.1.smithi148.stdout:3/735: unlink de/d24/d33/d39/da3/dbc/fec 0 2024-03-29T08:07:24.748 INFO:tasks.workunit.client.1.smithi148.stdout:5/685: dread - d0/db/d7b/fd8 zero size 2024-03-29T08:07:24.753 INFO:tasks.workunit.client.1.smithi148.stdout:6/699: getdents d6/d9/d1e/d36/d4d/d9d/d28/db3 0 2024-03-29T08:07:24.753 INFO:tasks.workunit.client.1.smithi148.stdout:1/639: symlink d3/de/d25/d35/d42/lc7 0 2024-03-29T08:07:24.754 INFO:tasks.workunit.client.1.smithi148.stdout:3/736: mkdir de/d24/d33/d39/d46/d8d/df3 0 2024-03-29T08:07:24.756 INFO:tasks.workunit.client.1.smithi148.stdout:8/684: mkdir d17/dce/de8 0 2024-03-29T08:07:24.758 INFO:tasks.workunit.client.1.smithi148.stdout:1/640: mknod d3/d55/d7e/cc8 0 2024-03-29T08:07:24.759 INFO:tasks.workunit.client.0.smithi037.stdout:1/566: mknod d2/d6/dc/d18/d22/d4f/d86/cc2 0 2024-03-29T08:07:24.759 INFO:tasks.workunit.client.1.smithi148.stdout:6/700: fsync d6/d9/d1e/d36/d4d/d9d/d28/d98/dcc/fd6 0 2024-03-29T08:07:24.760 INFO:tasks.workunit.client.1.smithi148.stdout:5/686: mknod d0/db/d1e/d35/d42/d67/d76/d81/d98/dd5/ce0 0 2024-03-29T08:07:24.765 INFO:tasks.workunit.client.1.smithi148.stdout:6/701: chown d6/d9/d1e/d36/d4d/d9d/d28/dda 54 1 2024-03-29T08:07:24.767 INFO:tasks.workunit.client.1.smithi148.stdout:8/685: rename d17/d74/d86/cbb to d17/d1d/d7e/ce9 0 2024-03-29T08:07:24.767 INFO:tasks.workunit.client.1.smithi148.stdout:0/692: dread db/de/df/d2b/f53 [0,4194304] 0 2024-03-29T08:07:24.768 INFO:tasks.workunit.client.1.smithi148.stdout:9/731: dread db/d10/d2d/d4e/dd4/f70 [0,4194304] 0 2024-03-29T08:07:24.774 INFO:tasks.workunit.client.0.smithi037.stdout:7/511: symlink dd/la5 0 2024-03-29T08:07:24.777 INFO:tasks.workunit.client.0.smithi037.stdout:1/567: creat d2/d4b/d80/fc3 x:0 0 0 2024-03-29T08:07:24.777 INFO:tasks.workunit.client.1.smithi148.stdout:9/732: truncate db/d22/d26/d54/f2e 338771 0 2024-03-29T08:07:24.778 INFO:tasks.workunit.client.1.smithi148.stdout:8/686: mknod d17/d49/cea 0 2024-03-29T08:07:24.780 INFO:tasks.workunit.client.0.smithi037.stdout:5/612: creat d8/d18/fd0 x:0 0 0 2024-03-29T08:07:24.780 INFO:tasks.workunit.client.1.smithi148.stdout:2/764: sync 2024-03-29T08:07:24.780 INFO:tasks.workunit.client.1.smithi148.stdout:7/650: link d8/d38/d67/l88 d8/d14/d45/le4 0 2024-03-29T08:07:24.785 INFO:tasks.workunit.client.1.smithi148.stdout:0/693: mkdir db/de/df/d14/db0/dd2 0 2024-03-29T08:07:24.785 INFO:tasks.workunit.client.1.smithi148.stdout:7/651: dread - d8/d29/fd4 zero size 2024-03-29T08:07:24.788 INFO:tasks.workunit.client.1.smithi148.stdout:2/765: unlink d7/da/f89 0 2024-03-29T08:07:24.790 INFO:tasks.workunit.client.0.smithi037.stdout:1/568: creat d2/d4b/fc4 x:0 0 0 2024-03-29T08:07:24.790 INFO:tasks.workunit.client.0.smithi037.stdout:5/613: mknod d8/d22/d37/d60/d61/cd1 0 2024-03-29T08:07:24.790 INFO:tasks.workunit.client.1.smithi148.stdout:5/687: dwrite d0/db/d1e/d35/d42/fde [0,4194304] 0 2024-03-29T08:07:24.793 INFO:tasks.workunit.client.1.smithi148.stdout:2/766: write d7/f2d [881641,16682] 0 2024-03-29T08:07:24.794 INFO:tasks.workunit.client.1.smithi148.stdout:8/687: chown d17/d18/d26/d63/d7d/c8c 302973 1 2024-03-29T08:07:24.794 INFO:tasks.workunit.client.1.smithi148.stdout:9/733: write db/d22/d26/d54/d4d/d35/d4b/dc3/ff8 [5316,110595] 0 2024-03-29T08:07:24.795 INFO:tasks.workunit.client.1.smithi148.stdout:6/702: dread d6/f70 [0,4194304] 0 2024-03-29T08:07:24.804 INFO:tasks.workunit.client.1.smithi148.stdout:3/737: dwrite de/d24/d33/d39/f5a [0,4194304] 0 2024-03-29T08:07:24.804 INFO:tasks.workunit.client.1.smithi148.stdout:1/641: dread d3/d18/d4d/d8f/f6b [0,4194304] 0 2024-03-29T08:07:24.805 INFO:tasks.workunit.client.0.smithi037.stdout:4/512: dread d3/d4/d15/f49 [0,4194304] 0 2024-03-29T08:07:24.806 INFO:tasks.workunit.client.0.smithi037.stdout:8/508: dwrite d5/d10/d2d/d34/d50/d83/d46/f98 [0,4194304] 0 2024-03-29T08:07:24.807 INFO:tasks.workunit.client.1.smithi148.stdout:5/688: write d0/db/d1e/d35/d42/f6b [774851,4192] 0 2024-03-29T08:07:24.809 INFO:tasks.workunit.client.1.smithi148.stdout:9/734: read db/d22/d26/d54/d4d/d35/d4b/dc3/ff8 [76942,4767] 0 2024-03-29T08:07:24.815 INFO:tasks.workunit.client.1.smithi148.stdout:7/652: creat d8/d38/d3c/da9/fe5 x:0 0 0 2024-03-29T08:07:24.815 INFO:tasks.workunit.client.1.smithi148.stdout:2/767: mkdir d7/d24/d4b/d66/dfa 0 2024-03-29T08:07:24.815 INFO:tasks.workunit.client.1.smithi148.stdout:0/694: dread db/de/d8e/da9/fca [0,4194304] 0 2024-03-29T08:07:24.816 INFO:tasks.workunit.client.1.smithi148.stdout:7/653: chown d8/d38/f8d 53163553 1 2024-03-29T08:07:24.827 INFO:tasks.workunit.client.0.smithi037.stdout:7/512: dwrite dd/d21/f2c [0,4194304] 0 2024-03-29T08:07:24.830 INFO:tasks.workunit.client.1.smithi148.stdout:6/703: symlink d6/d9/d68/dca/le2 0 2024-03-29T08:07:24.830 INFO:tasks.workunit.client.1.smithi148.stdout:9/735: write db/d10/d2d/d4e/d5f/f65 [2227401,126636] 0 2024-03-29T08:07:24.832 INFO:tasks.workunit.client.1.smithi148.stdout:3/738: fsync de/d24/d33/d39/d46/d30/d37/d79/fc0 0 2024-03-29T08:07:24.836 INFO:tasks.workunit.client.1.smithi148.stdout:0/695: symlink db/ld3 0 2024-03-29T08:07:24.841 INFO:tasks.workunit.client.0.smithi037.stdout:5/614: dwrite d8/d18/d24/f2a [0,4194304] 0 2024-03-29T08:07:24.843 INFO:tasks.workunit.client.1.smithi148.stdout:0/696: rmdir db/de 39 2024-03-29T08:07:24.843 INFO:tasks.workunit.client.1.smithi148.stdout:9/736: chown db/d22/d26/d54/d4d/f18 1160822288 1 2024-03-29T08:07:24.843 INFO:tasks.workunit.client.0.smithi037.stdout:7/513: creat dd/d21/d85/fa6 x:0 0 0 2024-03-29T08:07:24.844 INFO:tasks.workunit.client.0.smithi037.stdout:1/569: link d2/d4b/l53 d2/d6/dc/d18/d1d/d21/lc5 0 2024-03-29T08:07:24.849 INFO:tasks.workunit.client.0.smithi037.stdout:7/514: symlink dd/d68/la7 0 2024-03-29T08:07:24.851 INFO:tasks.workunit.client.0.smithi037.stdout:1/570: chown d2/d6/dc/d58/d67 12386822 1 2024-03-29T08:07:24.852 INFO:tasks.workunit.client.0.smithi037.stdout:5/615: creat d8/d22/dc1/fd2 x:0 0 0 2024-03-29T08:07:24.852 INFO:tasks.workunit.client.1.smithi148.stdout:6/704: fsync d6/d9/d1e/d36/d4d/d9d/f31 0 2024-03-29T08:07:24.858 INFO:tasks.workunit.client.0.smithi037.stdout:7/515: creat dd/d21/d35/d60/d94/fa8 x:0 0 0 2024-03-29T08:07:24.860 INFO:tasks.workunit.client.1.smithi148.stdout:3/739: creat de/d24/d33/d39/d46/d30/ff4 x:0 0 0 2024-03-29T08:07:24.860 INFO:tasks.workunit.client.0.smithi037.stdout:5/616: symlink d8/d22/d37/d8d/db3/dcb/ld3 0 2024-03-29T08:07:24.864 INFO:tasks.workunit.client.0.smithi037.stdout:1/571: dread d2/d4b/f66 [0,4194304] 0 2024-03-29T08:07:24.866 INFO:tasks.workunit.client.0.smithi037.stdout:7/516: mknod dd/d72/ca9 0 2024-03-29T08:07:24.871 INFO:tasks.workunit.client.0.smithi037.stdout:1/572: write d2/d6/f7d [445001,15881] 0 2024-03-29T08:07:24.875 INFO:tasks.workunit.client.1.smithi148.stdout:3/740: truncate de/d24/d33/d39/d46/d30/d37/d62/d63/dbf/ff0 844651 0 2024-03-29T08:07:24.876 INFO:tasks.workunit.client.0.smithi037.stdout:4/513: dwrite d3/d4/d15/d1d/db/d45/d1e/f4a [0,4194304] 0 2024-03-29T08:07:24.877 INFO:tasks.workunit.client.0.smithi037.stdout:7/517: rename dd/d1d/d49/l7b to dd/d68/laa 0 2024-03-29T08:07:24.879 INFO:tasks.workunit.client.0.smithi037.stdout:1/573: dread - d2/d6/dc/d58/f59 zero size 2024-03-29T08:07:24.879 INFO:tasks.workunit.client.1.smithi148.stdout:1/642: link d3/de/c1c d3/de/d25/d35/d42/cc9 0 2024-03-29T08:07:24.882 INFO:tasks.workunit.client.0.smithi037.stdout:5/617: creat d8/d22/d37/dbc/fd4 x:0 0 0 2024-03-29T08:07:24.883 INFO:tasks.workunit.client.0.smithi037.stdout:7/518: creat dd/d21/d85/fab x:0 0 0 2024-03-29T08:07:24.883 INFO:tasks.workunit.client.1.smithi148.stdout:8/688: rename d17/l9e to d17/d18/ddf/leb 0 2024-03-29T08:07:24.883 INFO:tasks.workunit.client.1.smithi148.stdout:3/741: symlink de/d24/d33/d39/lf5 0 2024-03-29T08:07:24.886 INFO:tasks.workunit.client.1.smithi148.stdout:3/742: stat de/d24/d33/d39/d46/d30/d37/fba 0 2024-03-29T08:07:24.888 INFO:tasks.workunit.client.0.smithi037.stdout:7/519: creat dd/d72/fac x:0 0 0 2024-03-29T08:07:24.889 INFO:tasks.workunit.client.0.smithi037.stdout:6/444: sync 2024-03-29T08:07:24.889 INFO:tasks.workunit.client.0.smithi037.stdout:0/389: sync 2024-03-29T08:07:24.890 INFO:tasks.workunit.client.0.smithi037.stdout:2/372: sync 2024-03-29T08:07:24.895 INFO:tasks.workunit.client.0.smithi037.stdout:8/509: dwrite d5/d10/f35 [0,4194304] 0 2024-03-29T08:07:24.895 INFO:tasks.workunit.client.0.smithi037.stdout:7/520: dread - dd/f77 zero size 2024-03-29T08:07:24.900 INFO:tasks.workunit.client.0.smithi037.stdout:7/521: chown dd/d1d/f53 12114 1 2024-03-29T08:07:24.900 INFO:tasks.workunit.client.0.smithi037.stdout:0/390: write d9/da/d1b/d22/d2c/f42 [5084661,73702] 0 2024-03-29T08:07:24.900 INFO:tasks.workunit.client.0.smithi037.stdout:2/373: unlink d0/d53/f7f 0 2024-03-29T08:07:24.902 INFO:tasks.workunit.client.0.smithi037.stdout:0/391: chown d9/da/d1b/d33 2505162 1 2024-03-29T08:07:24.913 INFO:tasks.workunit.client.0.smithi037.stdout:4/514: dwrite d3/d4/f2f [8388608,4194304] 0 2024-03-29T08:07:24.916 INFO:tasks.workunit.client.0.smithi037.stdout:6/445: rename d0/d4f/l88 to d0/d5/d1e/d84/l91 0 2024-03-29T08:07:24.921 INFO:tasks.workunit.client.1.smithi148.stdout:3/743: chown de/d24/d33/d39/d46/d30/d37/d62/d63/fcb 132473782 1 2024-03-29T08:07:24.922 INFO:tasks.workunit.client.0.smithi037.stdout:8/510: mknod d5/ca9 0 2024-03-29T08:07:24.926 INFO:tasks.workunit.client.1.smithi148.stdout:1/643: dwrite d3/de/f86 [0,4194304] 0 2024-03-29T08:07:24.929 INFO:tasks.workunit.client.0.smithi037.stdout:0/392: mkdir d9/da/dd/d13/d7b 0 2024-03-29T08:07:24.940 INFO:tasks.workunit.client.0.smithi037.stdout:6/446: rename d0/d4f/f57 to d0/d5/d3f/f92 0 2024-03-29T08:07:24.941 INFO:tasks.workunit.client.0.smithi037.stdout:4/515: dread d3/d4/d15/d1d/db/d45/d1e/f4e [0,4194304] 0 2024-03-29T08:07:24.948 INFO:tasks.workunit.client.0.smithi037.stdout:6/447: rename d0/c11 to d0/d19/d34/d5a/c93 0 2024-03-29T08:07:24.950 INFO:tasks.workunit.client.0.smithi037.stdout:7/522: rmdir dd/d21/d35/d42/d5c/d9b 0 2024-03-29T08:07:24.951 INFO:tasks.workunit.client.1.smithi148.stdout:8/689: creat d17/d18/d26/d4f/d94/fec x:0 0 0 2024-03-29T08:07:24.952 INFO:tasks.workunit.client.1.smithi148.stdout:2/768: rename d7/da/d33/ded/d41/fcb to d7/da/d33/d3c/ffb 0 2024-03-29T08:07:24.953 INFO:tasks.workunit.client.0.smithi037.stdout:4/516: rmdir d3/d4/d15/d1d/db/d45/d1e/d59/d7d 39 2024-03-29T08:07:24.954 INFO:tasks.workunit.client.0.smithi037.stdout:4/517: readlink d3/d54/l7c 0 2024-03-29T08:07:24.954 INFO:tasks.workunit.client.1.smithi148.stdout:3/744: creat de/d24/d33/d39/da3/ff6 x:0 0 0 2024-03-29T08:07:24.955 INFO:tasks.workunit.client.0.smithi037.stdout:6/448: unlink d0/f4 0 2024-03-29T08:07:24.957 INFO:tasks.workunit.client.1.smithi148.stdout:2/769: mknod d7/da/d4c/dc4/cfc 0 2024-03-29T08:07:24.958 INFO:tasks.workunit.client.1.smithi148.stdout:1/644: mkdir d3/d55/dca 0 2024-03-29T08:07:24.959 INFO:tasks.workunit.client.0.smithi037.stdout:7/523: mknod dd/d1d/d49/d81/d84/cad 0 2024-03-29T08:07:24.960 INFO:tasks.workunit.client.1.smithi148.stdout:3/745: creat de/d24/d33/d39/d46/d70/def/ff7 x:0 0 0 2024-03-29T08:07:24.962 INFO:tasks.workunit.client.1.smithi148.stdout:3/746: write de/f2f [4827287,41219] 0 2024-03-29T08:07:24.963 INFO:tasks.workunit.client.1.smithi148.stdout:7/654: rename d8/d14/d45/cd2 to d8/d38/d3c/da9/d37/ce6 0 2024-03-29T08:07:24.965 INFO:tasks.workunit.client.1.smithi148.stdout:2/770: truncate d7/d24/d4b/d66/df1/fe3 885583 0 2024-03-29T08:07:24.967 INFO:tasks.workunit.client.1.smithi148.stdout:2/771: chown d7/da/d33/d3c/d31/de1 3445 1 2024-03-29T08:07:24.968 INFO:tasks.workunit.client.0.smithi037.stdout:4/518: unlink d3/d4/f7b 0 2024-03-29T08:07:24.972 INFO:tasks.workunit.client.1.smithi148.stdout:2/772: rmdir d7/da/d4c/d9a 39 2024-03-29T08:07:24.976 INFO:tasks.workunit.client.0.smithi037.stdout:6/449: mkdir d0/d19/d34/d4e/d94 0 2024-03-29T08:07:24.976 INFO:tasks.workunit.client.1.smithi148.stdout:2/773: creat d7/da/d33/ded/d2e/d37/ffd x:0 0 0 2024-03-29T08:07:24.976 INFO:tasks.workunit.client.0.smithi037.stdout:2/374: write d0/fa [514444,17987] 0 2024-03-29T08:07:24.983 INFO:tasks.workunit.client.1.smithi148.stdout:7/655: symlink d8/d38/d3c/da5/db4/le7 0 2024-03-29T08:07:24.987 INFO:tasks.workunit.client.0.smithi037.stdout:8/511: rename d5/d10/d2d/d34/f79 to d5/d10/d19/d3d/d4a/d4c/d67/faa 0 2024-03-29T08:07:24.991 INFO:tasks.workunit.client.1.smithi148.stdout:2/774: dwrite d7/da/d33/d3c/d54/dec/fb4 [0,4194304] 0 2024-03-29T08:07:24.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:24.992+0000 7efcdbe00700 1 -- 172.21.15.37:0/3747444018 --> [v2:172.21.15.148:6828/1134897184,v1:172.21.15.148:6829/1134897184] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7efcd410d4e0 con 0x7efcbc06aec0 2024-03-29T08:07:24.994 INFO:tasks.workunit.client.1.smithi148.stdout:3/747: dread de/d24/d33/d39/d46/d30/d37/f7a [0,4194304] 0 2024-03-29T08:07:24.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:24.994+0000 7efcd1600700 1 -- 172.21.15.37:0/3747444018 <== mgr.14602 v2:172.21.15.148:6828/1134897184 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+341 (secure 0 0 0) 0x7efcd410d4e0 con 0x7efcbc06aec0 2024-03-29T08:07:24.995 INFO:tasks.workunit.client.1.smithi148.stdout:2/775: mknod d7/d24/d4b/cfe 0 2024-03-29T08:07:24.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:24.997+0000 7efcdbe00700 1 -- 172.21.15.37:0/3747444018 >> [v2:172.21.15.148:6828/1134897184,v1:172.21.15.148:6829/1134897184] conn(0x7efcbc06aec0 msgr2=0x7efcbc06d370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:07:24.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:24.997+0000 7efcdbe00700 1 --2- 172.21.15.37:0/3747444018 >> [v2:172.21.15.148:6828/1134897184,v1:172.21.15.148:6829/1134897184] conn(0x7efcbc06aec0 0x7efcbc06d370 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7efcc000b0d0 tx=0x7efcc0009e30 comp rx=0 tx=0).stop 2024-03-29T08:07:24.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:24.997+0000 7efcdbe00700 1 -- 172.21.15.37:0/3747444018 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efcd40662e0 msgr2=0x7efcd410b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:07:24.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:24.997+0000 7efcdbe00700 1 --2- 172.21.15.37:0/3747444018 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efcd40662e0 0x7efcd410b650 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7efcc800eb40 tx=0x7efcc800b040 comp rx=0 tx=0).stop 2024-03-29T08:07:24.998 INFO:tasks.workunit.client.0.smithi037.stdout:4/519: dwrite d3/d4/d15/d1d/db/d45/d1e/d59/d79/fa6 [0,4194304] 0 2024-03-29T08:07:24.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:24.998+0000 7efcdbe00700 1 -- 172.21.15.37:0/3747444018 shutdown_connections 2024-03-29T08:07:24.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:24.998+0000 7efcdbe00700 1 --2- 172.21.15.37:0/3747444018 >> [v2:172.21.15.148:6828/1134897184,v1:172.21.15.148:6829/1134897184] conn(0x7efcbc06aec0 0x7efcbc06d370 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:07:24.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:24.998+0000 7efcdbe00700 1 --2- 172.21.15.37:0/3747444018 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efcd40662e0 0x7efcd410b650 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:07:24.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:24.998+0000 7efcdbe00700 1 --2- 172.21.15.37:0/3747444018 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efcd40fff00 0x7efcd410bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:07:24.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:24.998+0000 7efcdbe00700 1 -- 172.21.15.37:0/3747444018 >> 172.21.15.37:0/3747444018 conn(0x7efcd40f7430 msgr2=0x7efcd40f7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:07:24.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:24.998+0000 7efcdbe00700 1 -- 172.21.15.37:0/3747444018 shutdown_connections 2024-03-29T08:07:24.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:24.999+0000 7efcdbe00700 1 -- 172.21.15.37:0/3747444018 wait complete. 2024-03-29T08:07:25.004 INFO:tasks.workunit.client.0.smithi037.stdout:7/524: dwrite dd/f48 [0,4194304] 0 2024-03-29T08:07:25.006 INFO:tasks.workunit.client.0.smithi037.stdout:2/375: link d0/d20/c4d d0/d20/d22/d4e/c81 0 2024-03-29T08:07:25.007 INFO:tasks.workunit.client.1.smithi148.stdout:3/748: rename de/d24/d33/d39/d46/d30/d37/d62/d63/dbf/ld9 to de/d24/d33/d39/d46/d70/def/lf8 0 2024-03-29T08:07:25.011 INFO:tasks.workunit.client.0.smithi037.stdout:7/525: write dd/d21/d35/f8f [1876358,33265] 0 2024-03-29T08:07:25.015 INFO:tasks.workunit.client.1.smithi148.stdout:3/749: read - de/d24/d33/d39/d46/d70/fa9 zero size 2024-03-29T08:07:25.016 INFO:tasks.workunit.client.1.smithi148.stdout:2/776: dwrite d7/d8c/fe6 [0,4194304] 0 2024-03-29T08:07:25.016 INFO:tasks.workunit.client.0.smithi037.stdout:7/526: read - dd/d1d/d34/d54/f5f zero size 2024-03-29T08:07:25.017 INFO:tasks.workunit.client.1.smithi148.stdout:7/656: dwrite d8/d14/f1c [0,4194304] 0 2024-03-29T08:07:25.030 INFO:tasks.workunit.client.1.smithi148.stdout:2/777: write d7/da/f7f [602464,108960] 0 2024-03-29T08:07:25.031 INFO:tasks.workunit.client.0.smithi037.stdout:4/520: mknod d3/d4/d15/d1d/db/d72/d98/ca8 0 2024-03-29T08:07:25.031 INFO:tasks.workunit.client.1.smithi148.stdout:4/700: sync 2024-03-29T08:07:25.031 INFO:tasks.workunit.client.1.smithi148.stdout:5/689: sync 2024-03-29T08:07:25.034 INFO:tasks.workunit.client.1.smithi148.stdout:4/701: write df/d2c/d51/d77/d17/f42 [6697303,128893] 0 2024-03-29T08:07:25.036 INFO:tasks.workunit.client.0.smithi037.stdout:4/521: write d3/d4/d15/d89/f94 [4966724,38460] 0 2024-03-29T08:07:25.038 INFO:tasks.workunit.client.1.smithi148.stdout:5/690: mkdir d0/db/d84/daf/de1 0 2024-03-29T08:07:25.042 INFO:tasks.workunit.client.1.smithi148.stdout:4/702: mkdir df/d89/ddd 0 2024-03-29T08:07:25.050 INFO:tasks.workunit.client.1.smithi148.stdout:5/691: truncate d0/db/f3e 1149716 0 2024-03-29T08:07:25.052 INFO:tasks.workunit.client.1.smithi148.stdout:7/657: unlink d8/d38/d3c/da9/d37/cb6 0 2024-03-29T08:07:25.052 INFO:tasks.workunit.client.1.smithi148.stdout:2/778: getdents d7/da/d4c/dc4 0 2024-03-29T08:07:25.054 INFO:tasks.workunit.client.1.smithi148.stdout:3/750: dread de/f17 [0,4194304] 0 2024-03-29T08:07:25.057 INFO:tasks.workunit.client.1.smithi148.stdout:8/690: write d17/d74/d86/fe5 [3269114,15990] 0 2024-03-29T08:07:25.057 INFO:tasks.workunit.client.0.smithi037.stdout:6/450: write d0/d5/d1e/f5f [684317,129473] 0 2024-03-29T08:07:25.058 INFO:tasks.workunit.client.1.smithi148.stdout:5/692: link d0/db/ld0 d0/d46/le2 0 2024-03-29T08:07:25.061 INFO:tasks.workunit.client.0.smithi037.stdout:6/451: chown d0/d19/d34/l8f 1813643 1 2024-03-29T08:07:25.062 INFO:tasks.workunit.client.1.smithi148.stdout:7/658: unlink d8/d38/d6a/d91/cd6 0 2024-03-29T08:07:25.062 INFO:tasks.workunit.client.1.smithi148.stdout:3/751: symlink de/d24/d33/d39/d46/d30/d37/d62/d63/dbf/lf9 0 2024-03-29T08:07:25.068 INFO:tasks.workunit.client.1.smithi148.stdout:3/752: mknod de/d24/d33/d39/d46/d70/def/cfa 0 2024-03-29T08:07:25.068 INFO:tasks.workunit.client.0.smithi037.stdout:4/522: mknod d3/d4/d15/d1d/db/d45/d1e/d59/ca9 0 2024-03-29T08:07:25.070 INFO:tasks.workunit.client.1.smithi148.stdout:7/659: mknod d8/d38/d3c/da5/db4/ce8 0 2024-03-29T08:07:25.070 INFO:tasks.workunit.client.1.smithi148.stdout:5/693: creat d0/db/d7b/fe3 x:0 0 0 2024-03-29T08:07:25.070 INFO:tasks.workunit.client.1.smithi148.stdout:2/779: write d7/da/d33/d3c/f42 [603182,49339] 0 2024-03-29T08:07:25.071 INFO:tasks.workunit.client.1.smithi148.stdout:7/660: chown d8/d60 52448911 1 2024-03-29T08:07:25.072 INFO:tasks.workunit.client.0.smithi037.stdout:7/527: write dd/d21/d35/d3b/f97 [445651,75681] 0 2024-03-29T08:07:25.076 INFO:tasks.workunit.client.1.smithi148.stdout:8/691: chown d17/d49/d9a/dbe/d6f/f55 3 1 2024-03-29T08:07:25.082 INFO:tasks.workunit.client.0.smithi037.stdout:6/452: mkdir d0/d20/d95 0 2024-03-29T08:07:25.084 INFO:tasks.workunit.client.1.smithi148.stdout:3/753: creat de/d24/d33/d39/ffb x:0 0 0 2024-03-29T08:07:25.086 INFO:tasks.workunit.client.0.smithi037.stdout:7/528: rename dd/d21/d89/f99 to dd/d72/fae 0 2024-03-29T08:07:25.088 INFO:tasks.workunit.client.0.smithi037.stdout:4/523: creat d3/d57/d6b/faa x:0 0 0 2024-03-29T08:07:25.091 INFO:tasks.workunit.client.1.smithi148.stdout:8/692: creat d17/dce/de8/fed x:0 0 0 2024-03-29T08:07:25.091 INFO:tasks.workunit.client.1.smithi148.stdout:3/754: rmdir de/d24/d33/d6b 39 2024-03-29T08:07:25.091 INFO:tasks.workunit.client.0.smithi037.stdout:8/512: dwrite d5/f16 [0,4194304] 0 2024-03-29T08:07:25.092 INFO:tasks.workunit.client.0.smithi037.stdout:7/529: fdatasync dd/d21/d35/f8f 0 2024-03-29T08:07:25.095 INFO:tasks.workunit.client.1.smithi148.stdout:2/780: dread d7/d24/f36 [0,4194304] 0 2024-03-29T08:07:25.095 INFO:tasks.workunit.client.0.smithi037.stdout:6/453: creat d0/d5/d3f/f96 x:0 0 0 2024-03-29T08:07:25.097 INFO:tasks.workunit.client.1.smithi148.stdout:0/697: sync 2024-03-29T08:07:25.097 INFO:tasks.workunit.client.1.smithi148.stdout:9/737: sync 2024-03-29T08:07:25.097 INFO:tasks.workunit.client.1.smithi148.stdout:6/705: sync 2024-03-29T08:07:25.097 INFO:tasks.workunit.client.1.smithi148.stdout:2/781: stat d7/da/d33/ded/d82/fa9 0 2024-03-29T08:07:25.098 INFO:tasks.workunit.client.1.smithi148.stdout:8/693: chown d17/d18/fe4 1 1 2024-03-29T08:07:25.099 INFO:tasks.workunit.client.0.smithi037.stdout:2/376: dwrite d0/d20/d2e/f36 [0,4194304] 0 2024-03-29T08:07:25.100 INFO:tasks.workunit.client.0.smithi037.stdout:8/513: read - d5/d10/d2d/d34/d50/fa2 zero size 2024-03-29T08:07:25.101 INFO:tasks.workunit.client.1.smithi148.stdout:6/706: dread - d6/d9/d1e/d20/d30/fc6 zero size 2024-03-29T08:07:25.102 INFO:tasks.workunit.client.0.smithi037.stdout:7/530: write dd/d1d/d34/f66 [5659725,53777] 0 2024-03-29T08:07:25.105 INFO:tasks.workunit.client.1.smithi148.stdout:3/755: mkdir de/d24/d33/d39/d46/d30/d37/d62/d63/dbf/dfc 0 2024-03-29T08:07:25.108 INFO:tasks.workunit.client.1.smithi148.stdout:5/694: write d0/db/d7b/d16/d87/f90 [689312,99947] 0 2024-03-29T08:07:25.110 INFO:tasks.workunit.client.1.smithi148.stdout:6/707: creat d6/d9/d1e/d20/d6b/fe3 x:0 0 0 2024-03-29T08:07:25.111 INFO:tasks.workunit.client.1.smithi148.stdout:8/694: chown d17/d18/d26/d63/d66/l75 127490092 1 2024-03-29T08:07:25.112 INFO:tasks.workunit.client.1.smithi148.stdout:4/703: dread df/d2c/d51/d77/d26/d34/d43/f59 [0,4194304] 0 2024-03-29T08:07:25.113 INFO:tasks.workunit.client.1.smithi148.stdout:3/756: write de/d24/d33/d39/d46/d30/fa2 [3742325,20747] 0 2024-03-29T08:07:25.117 INFO:tasks.workunit.client.0.smithi037.stdout:6/454: rename d0/f37 to d0/d4f/f97 0 2024-03-29T08:07:25.117 INFO:tasks.workunit.client.1.smithi148.stdout:0/698: creat db/de/fd4 x:0 0 0 2024-03-29T08:07:25.121 INFO:tasks.workunit.client.0.smithi037.stdout:2/377: creat d0/d20/d2e/f82 x:0 0 0 2024-03-29T08:07:25.121 INFO:tasks.workunit.client.1.smithi148.stdout:6/708: chown d6/d9/lb 203229558 1 2024-03-29T08:07:25.122 INFO:tasks.workunit.client.1.smithi148.stdout:9/738: dwrite db/d22/d26/d54/d4d/d35/d4b/d5a/d74/fba [4194304,4194304] 0 2024-03-29T08:07:25.125 INFO:tasks.workunit.client.0.smithi037.stdout:2/378: write d0/d20/d2e/f82 [402150,18059] 0 2024-03-29T08:07:25.125 INFO:tasks.workunit.client.1.smithi148.stdout:5/695: creat d0/db/d84/daf/de1/fe4 x:0 0 0 2024-03-29T08:07:25.128 INFO:tasks.workunit.client.1.smithi148.stdout:0/699: unlink db/de/f3f 0 2024-03-29T08:07:25.128 INFO:tasks.workunit.client.1.smithi148.stdout:4/704: symlink df/d2c/d51/d77/d17/lde 0 2024-03-29T08:07:25.130 INFO:tasks.workunit.client.0.smithi037.stdout:2/379: dread - d0/d23/d70/f79 zero size 2024-03-29T08:07:25.131 INFO:tasks.workunit.client.1.smithi148.stdout:3/757: fdatasync de/d24/f56 0 2024-03-29T08:07:25.131 INFO:tasks.workunit.client.1.smithi148.stdout:5/696: write d0/db/d7b/fe3 [597595,76639] 0 2024-03-29T08:07:25.132 INFO:tasks.workunit.client.1.smithi148.stdout:1/645: sync 2024-03-29T08:07:25.133 INFO:tasks.workunit.client.1.smithi148.stdout:7/661: dwrite d8/d29/d44/d49/f68 [0,4194304] 0 2024-03-29T08:07:25.135 INFO:tasks.workunit.client.0.smithi037.stdout:8/514: dwrite d5/d10/d19/d3d/d4a/d4c/d67/d85/f8e [0,4194304] 0 2024-03-29T08:07:25.137 INFO:tasks.workunit.client.0.smithi037.stdout:6/455: link d0/d20/d43/l6d d0/d19/d34/d4e/l98 0 2024-03-29T08:07:25.138 INFO:tasks.workunit.client.1.smithi148.stdout:6/709: creat d6/d9/d1e/d27/d48/d73/ddd/fe4 x:0 0 0 2024-03-29T08:07:25.139 INFO:tasks.workunit.client.0.smithi037.stdout:8/515: fdatasync d5/d10/d2d/d34/d50/fa2 0 2024-03-29T08:07:25.141 INFO:tasks.workunit.client.1.smithi148.stdout:4/705: creat df/d2c/db2/fdf x:0 0 0 2024-03-29T08:07:25.147 INFO:tasks.workunit.client.1.smithi148.stdout:1/646: mkdir d3/de/d25/d35/dcb 0 2024-03-29T08:07:25.151 INFO:tasks.workunit.client.1.smithi148.stdout:2/782: write d7/da/d33/ded/d2e/d37/f3d [2664782,123232] 0 2024-03-29T08:07:25.152 INFO:tasks.workunit.client.1.smithi148.stdout:5/697: chown d0/db/d7b/l61 37238225 1 2024-03-29T08:07:25.153 INFO:tasks.workunit.client.1.smithi148.stdout:6/710: symlink d6/d9/d68/le5 0 2024-03-29T08:07:25.153 INFO:tasks.workunit.client.1.smithi148.stdout:1/647: chown d3/de/d6e 1 1 2024-03-29T08:07:25.154 INFO:tasks.workunit.client.1.smithi148.stdout:4/706: mkdir df/d2c/d51/d77/d26/d34/d43/de0 0 2024-03-29T08:07:25.159 INFO:tasks.workunit.client.1.smithi148.stdout:4/707: unlink df/f4d 0 2024-03-29T08:07:25.162 INFO:tasks.workunit.client.1.smithi148.stdout:2/783: chown d7/da/d33/f5b 1 1 2024-03-29T08:07:25.166 INFO:tasks.workunit.client.1.smithi148.stdout:9/739: dwrite db/d22/d26/d54/f45 [0,4194304] 0 2024-03-29T08:07:25.170 INFO:tasks.workunit.client.0.smithi037.stdout:7/531: truncate dd/f3e 3837804 0 2024-03-29T08:07:25.173 INFO:tasks.workunit.client.1.smithi148.stdout:8/695: dwrite d17/d18/f36 [4194304,4194304] 0 2024-03-29T08:07:25.174 INFO:tasks.workunit.client.0.smithi037.stdout:4/524: dread d3/d4/d15/d89/f94 [0,4194304] 0 2024-03-29T08:07:25.176 INFO:tasks.workunit.client.0.smithi037.stdout:7/532: mknod dd/caf 0 2024-03-29T08:07:25.179 INFO:tasks.workunit.client.1.smithi148.stdout:4/708: write df/d2c/d51/d77/d26/d34/d43/fc8 [851367,50539] 0 2024-03-29T08:07:25.179 INFO:tasks.workunit.client.0.smithi037.stdout:3/484: sync 2024-03-29T08:07:25.179 INFO:tasks.workunit.client.0.smithi037.stdout:9/719: sync 2024-03-29T08:07:25.183 INFO:tasks.workunit.client.1.smithi148.stdout:7/662: dwrite d8/d38/d3c/da9/f39 [4194304,4194304] 0 2024-03-29T08:07:25.184 INFO:tasks.workunit.client.0.smithi037.stdout:3/485: dread - dd/d15/f49 zero size 2024-03-29T08:07:25.188 INFO:tasks.workunit.client.0.smithi037.stdout:8/516: dwrite d5/d10/d2d/d34/d50/f5e [0,4194304] 0 2024-03-29T08:07:25.191 INFO:tasks.workunit.client.1.smithi148.stdout:8/696: write d17/d18/d26/fd7 [112679,84978] 0 2024-03-29T08:07:25.192 INFO:tasks.workunit.client.0.smithi037.stdout:3/486: fdatasync dd/d5e/f65 0 2024-03-29T08:07:25.194 INFO:tasks.workunit.client.1.smithi148.stdout:9/740: read db/d10/d2d/d4e/f6b [372229,49383] 0 2024-03-29T08:07:25.195 INFO:tasks.workunit.client.1.smithi148.stdout:7/663: write d8/d38/f8d [627724,93760] 0 2024-03-29T08:07:25.201 INFO:tasks.workunit.client.0.smithi037.stdout:7/533: dwrite dd/d21/d35/f4c [0,4194304] 0 2024-03-29T08:07:25.203 INFO:tasks.workunit.client.1.smithi148.stdout:3/758: dwrite de/d24/d33/d39/f8b [0,4194304] 0 2024-03-29T08:07:25.206 INFO:tasks.workunit.client.1.smithi148.stdout:9/741: chown db/d22/f51 22890314 1 2024-03-29T08:07:25.208 INFO:tasks.workunit.client.1.smithi148.stdout:4/709: creat df/d2c/d51/d77/d17/fe1 x:0 0 0 2024-03-29T08:07:25.208 INFO:tasks.workunit.client.1.smithi148.stdout:1/648: dwrite d3/d18/d71/fb2 [0,4194304] 0 2024-03-29T08:07:25.209 INFO:tasks.workunit.client.1.smithi148.stdout:3/759: write de/d24/f31 [6932119,90230] 0 2024-03-29T08:07:25.210 INFO:tasks.workunit.client.1.smithi148.stdout:0/700: dread db/dc/daa/d47/f2c [0,4194304] 0 2024-03-29T08:07:25.211 INFO:tasks.workunit.client.0.smithi037.stdout:8/517: unlink d5/d10/d2d/d34/d50/l84 0 2024-03-29T08:07:25.214 INFO:tasks.workunit.client.1.smithi148.stdout:4/710: chown df/d2c/d51/d77/d26/d9b/lab 1 1 2024-03-29T08:07:25.218 INFO:tasks.workunit.client.1.smithi148.stdout:2/784: dwrite d7/da/d33/d45/f5f [0,4194304] 0 2024-03-29T08:07:25.219 INFO:tasks.workunit.client.1.smithi148.stdout:4/711: chown df/d2c/d80/daa/dbc/dc9 2261561 1 2024-03-29T08:07:25.221 INFO:tasks.workunit.client.1.smithi148.stdout:2/785: chown d7/d8c 397234709 1 2024-03-29T08:07:25.223 INFO:tasks.workunit.client.1.smithi148.stdout:7/664: mkdir d8/d38/d3c/da5/db4/de9 0 2024-03-29T08:07:25.226 INFO:tasks.workunit.client.1.smithi148.stdout:3/760: read de/d3b/f58 [154997,9338] 0 2024-03-29T08:07:25.226 INFO:tasks.workunit.client.1.smithi148.stdout:2/786: write d7/da/d33/d3c/d54/f93 [2953007,14116] 0 2024-03-29T08:07:25.228 INFO:tasks.workunit.client.1.smithi148.stdout:2/787: fdatasync d7/da/d33/fd3 0 2024-03-29T08:07:25.232 INFO:tasks.workunit.client.1.smithi148.stdout:2/788: chown d7/da/d4c/c9c 28155 1 2024-03-29T08:07:25.234 INFO:tasks.workunit.client.1.smithi148.stdout:1/649: dwrite d3/de/d25/f75 [0,4194304] 0 2024-03-29T08:07:25.240 INFO:tasks.workunit.client.0.smithi037.stdout:7/534: rmdir dd 39 2024-03-29T08:07:25.240 INFO:tasks.workunit.client.1.smithi148.stdout:5/698: sync 2024-03-29T08:07:25.241 INFO:tasks.workunit.client.1.smithi148.stdout:3/761: chown de/d24/d33/d39/d71/c84 1746 1 2024-03-29T08:07:25.242 INFO:tasks.workunit.client.1.smithi148.stdout:9/742: link db/d22/lf6 db/d10/d2d/d4e/d5f/dd5/lfa 0 2024-03-29T08:07:25.244 INFO:tasks.workunit.client.1.smithi148.stdout:6/711: dread d6/d9/d1e/d27/f32 [0,4194304] 0 2024-03-29T08:07:25.246 INFO:tasks.workunit.client.1.smithi148.stdout:5/699: stat d0/db/d1e/d35/d42/f6b 0 2024-03-29T08:07:25.247 INFO:tasks.workunit.client.1.smithi148.stdout:9/743: stat db/d22/d26/d54/d4d/d35 0 2024-03-29T08:07:25.247 INFO:tasks.workunit.client.0.smithi037.stdout:9/720: truncate d1/d4/d19/d1e/d4a/d6f/da2/d13/f17 3600400 0 2024-03-29T08:07:25.248 INFO:tasks.workunit.client.1.smithi148.stdout:7/665: dread - d8/d60/f7a zero size 2024-03-29T08:07:25.253 INFO:tasks.workunit.client.1.smithi148.stdout:3/762: chown de/d24/d33/d39/da3/dbc/fbe 25876 1 2024-03-29T08:07:25.253 INFO:tasks.workunit.client.0.smithi037.stdout:7/535: chown dd/d21/f2c 1292 1 2024-03-29T08:07:25.255 INFO:tasks.workunit.client.0.smithi037.stdout:7/536: chown dd/d72/f98 34 1 2024-03-29T08:07:25.257 INFO:tasks.workunit.client.1.smithi148.stdout:1/650: symlink d3/de/d1d/d47/d6d/d84/lcc 0 2024-03-29T08:07:25.265 INFO:tasks.workunit.client.1.smithi148.stdout:5/700: unlink d0/db/d1e/d36/d7c/l85 0 2024-03-29T08:07:25.266 INFO:tasks.workunit.client.1.smithi148.stdout:0/701: dread db/de/df/d14/f3c [0,4194304] 0 2024-03-29T08:07:25.267 INFO:tasks.workunit.client.1.smithi148.stdout:3/763: mknod de/d24/d33/d39/d46/d30/d37/d62/d63/dbf/cfd 0 2024-03-29T08:07:25.267 INFO:tasks.workunit.client.0.smithi037.stdout:9/721: creat d1/d4/d19/d1e/d71/ffa x:0 0 0 2024-03-29T08:07:25.268 INFO:tasks.workunit.client.1.smithi148.stdout:6/712: dread d6/f70 [0,4194304] 0 2024-03-29T08:07:25.269 INFO:tasks.workunit.client.0.smithi037.stdout:4/525: write d3/d54/d88/f8d [818039,20583] 0 2024-03-29T08:07:25.270 INFO:tasks.workunit.client.1.smithi148.stdout:8/697: write f15 [5463122,95062] 0 2024-03-29T08:07:25.272 INFO:tasks.workunit.client.0.smithi037.stdout:3/487: creat dd/d46/d2d/d3a/fa6 x:0 0 0 2024-03-29T08:07:25.272 INFO:tasks.workunit.client.1.smithi148.stdout:2/789: link d7/da/d33/d3c/f28 d7/d8c/d91/fff 0 2024-03-29T08:07:25.276 INFO:tasks.workunit.client.1.smithi148.stdout:5/701: chown d0/db/d1e/d35/d42/d67/d9a 695 1 2024-03-29T08:07:25.276 INFO:tasks.workunit.client.1.smithi148.stdout:4/712: dwrite df/d2c/d51/d77/d17/fbd [0,4194304] 0 2024-03-29T08:07:25.280 INFO:tasks.workunit.client.1.smithi148.stdout:7/666: dwrite d8/d14/da8/fdc [0,4194304] 0 2024-03-29T08:07:25.280 INFO:tasks.workunit.client.1.smithi148.stdout:3/764: mkdir de/d24/d33/d39/da3/dfe 0 2024-03-29T08:07:25.281 INFO:tasks.workunit.client.1.smithi148.stdout:0/702: symlink db/de/df/d2b/dae/ld5 0 2024-03-29T08:07:25.282 INFO:tasks.workunit.client.1.smithi148.stdout:0/703: chown db/dc 280287 1 2024-03-29T08:07:25.285 INFO:tasks.workunit.client.1.smithi148.stdout:0/704: fsync db/de/f70 0 2024-03-29T08:07:25.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:25.285+0000 7f48cec00700 1 -- 172.21.15.37:0/3110823432 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f48c806e9a0 msgr2=0x7f48c806ee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:07:25.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:25.285+0000 7f48cec00700 1 --2- 172.21.15.37:0/3110823432 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f48c806e9a0 0x7f48c806ee10 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f48b4004940 tx=0x7f48b4009e30 comp rx=0 tx=0).stop 2024-03-29T08:07:25.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:25.285+0000 7f48cec00700 1 -- 172.21.15.37:0/3110823432 shutdown_connections 2024-03-29T08:07:25.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:25.285+0000 7f48cec00700 1 --2- 172.21.15.37:0/3110823432 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f48c806e9a0 0x7f48c806ee10 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:07:25.286 INFO:tasks.workunit.client.0.smithi037.stdout:7/537: dwrite dd/d1d/f53 [0,4194304] 0 2024-03-29T08:07:25.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:25.286+0000 7f48cec00700 1 --2- 172.21.15.37:0/3110823432 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f48c806f9d0 0x7f48c806e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:07:25.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:25.286+0000 7f48cec00700 1 -- 172.21.15.37:0/3110823432 >> 172.21.15.37:0/3110823432 conn(0x7f48c8069de0 msgr2=0x7f48c806c210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:07:25.287 INFO:tasks.workunit.client.0.smithi037.stdout:9/722: truncate d1/d4/d19/d1e/d4a/d6f/da2/d31/daa/dba/dbb/ff2 508038 0 2024-03-29T08:07:25.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:25.287+0000 7f48cec00700 1 -- 172.21.15.37:0/3110823432 shutdown_connections 2024-03-29T08:07:25.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:25.288+0000 7f48cec00700 1 -- 172.21.15.37:0/3110823432 wait complete. 2024-03-29T08:07:25.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:25.290+0000 7f48cec00700 1 Processor -- start 2024-03-29T08:07:25.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:25.291+0000 7f48cec00700 1 -- start start 2024-03-29T08:07:25.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:25.291+0000 7f48cec00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f48c806e9a0 0x7f48c8113ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:07:25.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:25.291+0000 7f48cec00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f48c806f9d0 0x7f48c8114430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:07:25.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:25.291+0000 7f48cec00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48c8114ac0 con 0x7f48c806f9d0 2024-03-29T08:07:25.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:25.291+0000 7f48cec00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48c8117450 con 0x7f48c806e9a0 2024-03-29T08:07:25.292 INFO:tasks.workunit.client.1.smithi148.stdout:7/667: write d8/d14/d45/d4d/d78/fb5 [196436,39739] 0 2024-03-29T08:07:25.292 INFO:tasks.workunit.client.1.smithi148.stdout:2/790: truncate d7/da/d33/d3c/d54/f95 949083 0 2024-03-29T08:07:25.292 INFO:tasks.workunit.client.1.smithi148.stdout:0/705: fdatasync db/dc/daa/d47/d4c/d5a/f98 0 2024-03-29T08:07:25.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:25.292+0000 7f48c7e00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f48c806e9a0 0x7f48c8113ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:07:25.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:25.292+0000 7f48c7400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f48c806f9d0 0x7f48c8114430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:07:25.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:25.292+0000 7f48c7e00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f48c806e9a0 0x7f48c8113ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.37:43852/0 (socket says 172.21.15.37:43852) 2024-03-29T08:07:25.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:25.292+0000 7f48c7e00700 1 -- 172.21.15.37:0/2779142792 learned_addr learned my addr 172.21.15.37:0/2779142792 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:07:25.293 INFO:tasks.workunit.client.1.smithi148.stdout:6/713: unlink d6/d9/d1e/d36/d4d/d9d/d28/d98/f99 0 2024-03-29T08:07:25.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:25.294+0000 7f48c7400700 1 -- 172.21.15.37:0/2779142792 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f48c806e9a0 msgr2=0x7f48c8113ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:07:25.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:25.294+0000 7f48c7400700 1 --2- 172.21.15.37:0/2779142792 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f48c806e9a0 0x7f48c8113ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:07:25.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:25.294+0000 7f48c7400700 1 -- 172.21.15.37:0/2779142792 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48b4005040 con 0x7f48c806f9d0 2024-03-29T08:07:25.294 INFO:tasks.workunit.client.1.smithi148.stdout:2/791: stat d7/lcf 0 2024-03-29T08:07:25.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:25.295+0000 7f48c7400700 1 --2- 172.21.15.37:0/2779142792 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f48c806f9d0 0x7f48c8114430 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f48b4004890 tx=0x7f48b4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:07:25.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:25.296+0000 7f48bfe00700 1 -- 172.21.15.37:0/2779142792 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f48b400e030 con 0x7f48c806f9d0 2024-03-29T08:07:25.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:25.297+0000 7f48bfe00700 1 -- 172.21.15.37:0/2779142792 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f48b40093b0 con 0x7f48c806f9d0 2024-03-29T08:07:25.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:25.297+0000 7f48bfe00700 1 -- 172.21.15.37:0/2779142792 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f48b4013d90 con 0x7f48c806f9d0 2024-03-29T08:07:25.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:25.297+0000 7f48cec00700 1 -- 172.21.15.37:0/2779142792 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48c81176d0 con 0x7f48c806f9d0 2024-03-29T08:07:25.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:25.297+0000 7f48cec00700 1 -- 172.21.15.37:0/2779142792 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48c8117c90 con 0x7f48c806f9d0 2024-03-29T08:07:25.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:25.299+0000 7f48cec00700 1 -- 172.21.15.37:0/2779142792 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48c804c710 con 0x7f48c806f9d0 2024-03-29T08:07:25.300 INFO:tasks.workunit.client.0.smithi037.stdout:7/538: truncate dd/d1d/d34/f39 2802406 0 2024-03-29T08:07:25.300 INFO:tasks.workunit.client.1.smithi148.stdout:5/702: mkdir d0/db/d1e/d35/d42/d67/d76/d81/d98/de5 0 2024-03-29T08:07:25.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:25.301+0000 7f48bfe00700 1 -- 172.21.15.37:0/2779142792 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 25) v1 ==== 88766+0+0 (secure 0 0 0) 0x7f48b4013620 con 0x7f48c806f9d0 2024-03-29T08:07:25.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:25.303+0000 7f48bfe00700 1 --2- 172.21.15.37:0/2779142792 >> [v2:172.21.15.148:6828/1134897184,v1:172.21.15.148:6829/1134897184] conn(0x7f48ac06afd0 0x7f48ac06d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:07:25.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:25.303+0000 7f48bfe00700 1 -- 172.21.15.37:0/2779142792 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f48b408b490 con 0x7f48c806f9d0 2024-03-29T08:07:25.303 INFO:tasks.workunit.client.1.smithi148.stdout:5/703: write d0/db/d1e/fbd [241283,104396] 0 2024-03-29T08:07:25.304 INFO:tasks.workunit.client.1.smithi148.stdout:7/668: fdatasync d8/d29/d44/d49/f74 0 2024-03-29T08:07:25.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:25.305+0000 7f48c7e00700 1 --2- 172.21.15.37:0/2779142792 >> [v2:172.21.15.148:6828/1134897184,v1:172.21.15.148:6829/1134897184] conn(0x7f48ac06afd0 0x7f48ac06d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:07:25.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:25.307+0000 7f48c7e00700 1 --2- 172.21.15.37:0/2779142792 >> [v2:172.21.15.148:6828/1134897184,v1:172.21.15.148:6829/1134897184] conn(0x7f48ac06afd0 0x7f48ac06d480 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f48b8009c10 tx=0x7f48b8000f90 comp rx=0 tx=0).ready entity=mgr.14602 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:07:25.307 INFO:tasks.workunit.client.1.smithi148.stdout:4/713: creat df/d2c/d51/fe2 x:0 0 0 2024-03-29T08:07:25.307 INFO:tasks.workunit.client.1.smithi148.stdout:3/765: dwrite de/d24/d33/d39/d46/d30/ff4 [0,4194304] 0 2024-03-29T08:07:25.309 INFO:tasks.workunit.client.1.smithi148.stdout:0/706: symlink db/de/df/d11/d42/ld6 0 2024-03-29T08:07:25.310 INFO:tasks.workunit.client.1.smithi148.stdout:6/714: mknod d6/ce6 0 2024-03-29T08:07:25.310 INFO:tasks.workunit.client.1.smithi148.stdout:7/669: chown d8/d14/d45/d4d/f6b 3122 1 2024-03-29T08:07:25.311 INFO:tasks.workunit.client.1.smithi148.stdout:0/707: write db/dc/f59 [3402880,71745] 0 2024-03-29T08:07:25.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:25.313+0000 7f48bfe00700 1 -- 172.21.15.37:0/2779142792 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181579 (secure 0 0 0) 0x7f48b4059430 con 0x7f48c806f9d0 2024-03-29T08:07:25.313 INFO:tasks.workunit.client.1.smithi148.stdout:7/670: chown d8/d29/d44/fab 413328 1 2024-03-29T08:07:25.315 INFO:tasks.workunit.client.1.smithi148.stdout:5/704: stat d0/db/d7b/f91 0 2024-03-29T08:07:25.315 INFO:tasks.workunit.client.1.smithi148.stdout:1/651: write d3/d18/da8/fab [528101,85447] 0 2024-03-29T08:07:25.317 INFO:tasks.workunit.client.0.smithi037.stdout:8/518: dwrite d5/d10/d19/d3d/d4a/f6f [0,4194304] 0 2024-03-29T08:07:25.323 INFO:tasks.workunit.client.1.smithi148.stdout:3/766: rename de/d24/d33/d39/d46/d30/d37/f7a to de/d24/d33/d39/d46/d30/d37/d62/fff 0 2024-03-29T08:07:25.325 INFO:tasks.workunit.client.1.smithi148.stdout:6/715: creat d6/d9/d1e/d36/d4d/d9d/d28/db3/dd2/fe7 x:0 0 0 2024-03-29T08:07:25.326 INFO:tasks.workunit.client.0.smithi037.stdout:7/539: dread dd/d21/d35/f8f [0,4194304] 0 2024-03-29T08:07:25.327 INFO:tasks.workunit.client.1.smithi148.stdout:3/767: write de/d24/d33/d39/d46/d30/ff4 [2848815,91272] 0 2024-03-29T08:07:25.329 INFO:tasks.workunit.client.1.smithi148.stdout:0/708: mkdir db/dd7 0 2024-03-29T08:07:25.336 INFO:tasks.workunit.client.1.smithi148.stdout:7/671: dwrite d8/d38/d3c/da9/db7/fbf [0,4194304] 0 2024-03-29T08:07:25.337 INFO:tasks.workunit.client.1.smithi148.stdout:4/714: truncate df/d71/fce 1240037 0 2024-03-29T08:07:25.339 INFO:tasks.workunit.client.0.smithi037.stdout:8/519: creat d5/d78/fab x:0 0 0 2024-03-29T08:07:25.343 INFO:tasks.workunit.client.1.smithi148.stdout:4/715: mknod df/d2c/d51/d77/d17/ce3 0 2024-03-29T08:07:25.348 INFO:tasks.workunit.client.1.smithi148.stdout:0/709: rmdir db/dd7 0 2024-03-29T08:07:25.350 INFO:tasks.workunit.client.1.smithi148.stdout:5/705: dwrite d0/db/d1e/d35/d42/d67/d76/d81/d98/dd5/f9d [4194304,4194304] 0 2024-03-29T08:07:25.352 INFO:tasks.workunit.client.1.smithi148.stdout:7/672: symlink d8/d29/d44/d50/d94/lea 0 2024-03-29T08:07:25.356 INFO:tasks.workunit.client.1.smithi148.stdout:9/744: sync 2024-03-29T08:07:25.356 INFO:tasks.workunit.client.1.smithi148.stdout:8/698: dread d17/f23 [0,4194304] 0 2024-03-29T08:07:25.359 INFO:tasks.workunit.client.1.smithi148.stdout:3/768: dwrite de/d24/d33/d39/da3/dbc/fbe [0,4194304] 0 2024-03-29T08:07:25.359 INFO:tasks.workunit.client.1.smithi148.stdout:5/706: mkdir d0/db/d1e/d36/de6 0 2024-03-29T08:07:25.364 INFO:tasks.workunit.client.0.smithi037.stdout:4/526: write d3/d4/d15/d1d/db/d45/d1e/d59/d7d/f9f [2779359,122109] 0 2024-03-29T08:07:25.366 INFO:tasks.workunit.client.1.smithi148.stdout:5/707: fsync d0/db/d1e/d36/d41/f59 0 2024-03-29T08:07:25.366 INFO:tasks.workunit.client.1.smithi148.stdout:2/792: write d7/da/d33/f5b [2806133,62181] 0 2024-03-29T08:07:25.368 INFO:tasks.workunit.client.0.smithi037.stdout:3/488: dwrite dd/d46/d3e/f7e [0,4194304] 0 2024-03-29T08:07:25.370 INFO:tasks.workunit.client.0.smithi037.stdout:3/489: readlink dd/d46/l78 0 2024-03-29T08:07:25.371 INFO:tasks.workunit.client.1.smithi148.stdout:0/710: rename db/de/cd1 to db/dc/d78/cd8 0 2024-03-29T08:07:25.373 INFO:tasks.workunit.client.1.smithi148.stdout:4/716: getdents df/d2c/d80/da6 0 2024-03-29T08:07:25.373 INFO:tasks.workunit.client.0.smithi037.stdout:3/490: chown dd/d46/f3c 7130179 1 2024-03-29T08:07:25.375 INFO:tasks.workunit.client.1.smithi148.stdout:7/673: mknod d8/d29/ceb 0 2024-03-29T08:07:25.378 INFO:tasks.workunit.client.1.smithi148.stdout:6/716: dread d6/d9/d1e/f53 [0,4194304] 0 2024-03-29T08:07:25.379 INFO:tasks.workunit.client.1.smithi148.stdout:9/745: dwrite db/fa7 [0,4194304] 0 2024-03-29T08:07:25.382 INFO:tasks.workunit.client.1.smithi148.stdout:5/708: unlink d0/db/d7b/d16/f27 0 2024-03-29T08:07:25.382 INFO:tasks.workunit.client.1.smithi148.stdout:8/699: stat d17/d18/d26/c57 0 2024-03-29T08:07:25.384 INFO:tasks.workunit.client.0.smithi037.stdout:9/723: dwrite d1/d4/d95/fad [0,4194304] 0 2024-03-29T08:07:25.395 INFO:tasks.workunit.client.1.smithi148.stdout:3/769: rename de/d24/d33/d39/d46/d70/l7d to de/d24/d33/d39/d46/d30/d37/d79/l100 0 2024-03-29T08:07:25.396 INFO:tasks.workunit.client.1.smithi148.stdout:0/711: mknod db/de/df/cd9 0 2024-03-29T08:07:25.399 INFO:tasks.workunit.client.1.smithi148.stdout:0/712: stat db/dc/d78/f79 0 2024-03-29T08:07:25.399 INFO:tasks.workunit.client.1.smithi148.stdout:9/746: mknod db/d22/d26/d54/d52/d9c/cfb 0 2024-03-29T08:07:25.406 INFO:tasks.workunit.client.1.smithi148.stdout:6/717: fdatasync d6/d9/d1e/d36/d4d/d9d/fb2 0 2024-03-29T08:07:25.407 INFO:tasks.workunit.client.0.smithi037.stdout:8/520: mkdir d5/d78/dac 0 2024-03-29T08:07:25.408 INFO:tasks.workunit.client.1.smithi148.stdout:7/674: rename d8/d38/d3c/da9/d37/c3e to d8/d38/d3c/da9/d37/dc8/cec 0 2024-03-29T08:07:25.409 INFO:tasks.workunit.client.1.smithi148.stdout:8/700: truncate d17/d4e/f88 443079 0 2024-03-29T08:07:25.410 INFO:tasks.workunit.client.0.smithi037.stdout:3/491: creat dd/d15/d18/d8e/fa7 x:0 0 0 2024-03-29T08:07:25.412 INFO:tasks.workunit.client.1.smithi148.stdout:7/675: fsync d8/d38/d3c/da9/fac 0 2024-03-29T08:07:25.415 INFO:tasks.workunit.client.0.smithi037.stdout:4/527: symlink d3/d57/lab 0 2024-03-29T08:07:25.415 INFO:tasks.workunit.client.1.smithi148.stdout:7/676: write d8/d14/fcc [477831,32996] 0 2024-03-29T08:07:25.418 INFO:tasks.workunit.client.1.smithi148.stdout:1/652: dread d3/de/f1f [4194304,4194304] 0 2024-03-29T08:07:25.422 INFO:tasks.workunit.client.1.smithi148.stdout:0/713: dread db/de/df/d11/d2d/f85 [0,4194304] 0 2024-03-29T08:07:25.426 INFO:tasks.workunit.client.1.smithi148.stdout:5/709: link d0/db/ld d0/db/d1e/d36/d7c/le7 0 2024-03-29T08:07:25.427 INFO:tasks.workunit.client.1.smithi148.stdout:0/714: write db/de/df/d2b/f72 [329550,120364] 0 2024-03-29T08:07:25.430 INFO:tasks.workunit.client.1.smithi148.stdout:0/715: write fa [5699263,110495] 0 2024-03-29T08:07:25.431 INFO:tasks.workunit.client.1.smithi148.stdout:2/793: truncate d7/d24/d4b/fc2 932579 0 2024-03-29T08:07:25.432 INFO:tasks.workunit.client.1.smithi148.stdout:6/718: rename d6/d9/d1e/d27/d48/d9f/cc9 to d6/d9/d1e/d27/d48/d73/ce8 0 2024-03-29T08:07:25.434 INFO:tasks.workunit.client.0.smithi037.stdout:9/724: dwrite d1/f74 [0,4194304] 0 2024-03-29T08:07:25.434 INFO:tasks.workunit.client.1.smithi148.stdout:5/710: write d0/db/d1e/d35/d42/fde [1498358,74627] 0 2024-03-29T08:07:25.435 INFO:tasks.workunit.client.0.smithi037.stdout:4/528: dwrite d3/d54/f99 [0,4194304] 0 2024-03-29T08:07:25.436 INFO:tasks.workunit.client.0.smithi037.stdout:9/725: readlink d1/d4/l47 0 2024-03-29T08:07:25.439 INFO:tasks.workunit.client.1.smithi148.stdout:2/794: chown d7/da/d33/d3c/d31 790629 1 2024-03-29T08:07:25.441 INFO:tasks.workunit.client.1.smithi148.stdout:7/677: dwrite d8/d14/fc7 [0,4194304] 0 2024-03-29T08:07:25.442 INFO:tasks.workunit.client.1.smithi148.stdout:9/747: getdents db 0 2024-03-29T08:07:25.445 INFO:tasks.workunit.client.1.smithi148.stdout:6/719: symlink d6/d9/d1e/d20/d30/le9 0 2024-03-29T08:07:25.445 INFO:tasks.workunit.client.1.smithi148.stdout:8/701: rename d17/d4e/f71 to d17/d18/d26/d4f/fee 0 2024-03-29T08:07:25.450 INFO:tasks.workunit.client.1.smithi148.stdout:2/795: read d7/f1b [3335343,80737] 0 2024-03-29T08:07:25.451 INFO:tasks.workunit.client.1.smithi148.stdout:7/678: stat d8/d38/d3c/l5d 0 2024-03-29T08:07:25.454 INFO:tasks.workunit.client.1.smithi148.stdout:7/679: dread - d8/f15 zero size 2024-03-29T08:07:25.455 INFO:tasks.workunit.client.0.smithi037.stdout:4/529: unlink d3/d4/d15/d1d/d4f/c78 0 2024-03-29T08:07:25.457 INFO:tasks.workunit.client.0.smithi037.stdout:9/726: getdents d1/d4 0 2024-03-29T08:07:25.457 INFO:tasks.workunit.client.0.smithi037.stdout:4/530: stat d3/d4/d15/d1d/db/c3b 0 2024-03-29T08:07:25.458 INFO:tasks.workunit.client.1.smithi148.stdout:2/796: fsync d7/da/f7f 0 2024-03-29T08:07:25.458 INFO:tasks.workunit.client.1.smithi148.stdout:4/717: dwrite df/f1e [0,4194304] 0 2024-03-29T08:07:25.458 INFO:tasks.workunit.client.0.smithi037.stdout:7/540: write dd/d21/d35/d42/d5c/f5e [74997,1521] 0 2024-03-29T08:07:25.465 INFO:tasks.workunit.client.1.smithi148.stdout:4/718: fdatasync df/d2c/f46 0 2024-03-29T08:07:25.465 INFO:tasks.workunit.client.1.smithi148.stdout:9/748: read db/d22/d26/d54/f49 [900636,10609] 0 2024-03-29T08:07:25.466 INFO:tasks.workunit.client.1.smithi148.stdout:3/770: dread de/d24/d33/d39/d46/d30/d37/d79/fe3 [0,4194304] 0 2024-03-29T08:07:25.468 INFO:tasks.workunit.client.1.smithi148.stdout:1/653: dwrite d3/de/d25/d35/dc2/f99 [0,4194304] 0 2024-03-29T08:07:25.469 INFO:tasks.workunit.client.1.smithi148.stdout:9/749: fdatasync f3 0 2024-03-29T08:07:25.469 INFO:tasks.workunit.client.1.smithi148.stdout:2/797: unlink d7/da/d33/d3c/d54/d70/fca 0 2024-03-29T08:07:25.470 INFO:tasks.workunit.client.0.smithi037.stdout:7/541: rmdir dd/d21/d35/d3b 39 2024-03-29T08:07:25.474 INFO:tasks.workunit.client.1.smithi148.stdout:2/798: write d7/f2d [4574305,25940] 0 2024-03-29T08:07:25.474 INFO:tasks.workunit.client.0.smithi037.stdout:7/542: dread - dd/d1d/d34/d54/f5f zero size 2024-03-29T08:07:25.480 INFO:tasks.workunit.client.1.smithi148.stdout:3/771: unlink de/d24/d33/d39/d46/d30/d37/fba 0 2024-03-29T08:07:25.488 INFO:tasks.workunit.client.1.smithi148.stdout:9/750: mkdir db/d10/d6d/dfc 0 2024-03-29T08:07:25.490 INFO:tasks.workunit.client.1.smithi148.stdout:9/751: chown db/d22/d26/d54/d4d/dbb/dc0/lc5 689753739 1 2024-03-29T08:07:25.494 INFO:tasks.workunit.client.0.smithi037.stdout:4/531: truncate d3/d4/d15/d89/f94 4821437 0 2024-03-29T08:07:25.494 INFO:tasks.workunit.client.1.smithi148.stdout:7/680: truncate d8/f10 2797886 0 2024-03-29T08:07:25.495 INFO:tasks.workunit.client.1.smithi148.stdout:1/654: creat d3/d18/d4d/d76/d85/fcd x:0 0 0 2024-03-29T08:07:25.497 INFO:tasks.workunit.client.0.smithi037.stdout:3/492: dwrite dd/f23 [0,4194304] 0 2024-03-29T08:07:25.499 INFO:tasks.workunit.client.1.smithi148.stdout:2/799: creat d7/d8c/da4/d8e/dda/f100 x:0 0 0 2024-03-29T08:07:25.501 INFO:tasks.workunit.client.1.smithi148.stdout:2/800: write d7/da/d33/ded/f16 [4433070,62843] 0 2024-03-29T08:07:25.501 INFO:tasks.workunit.client.0.smithi037.stdout:4/532: stat d3/d4/d15/d1d/db/d45/d93/d96/da0/ca5 0 2024-03-29T08:07:25.502 INFO:tasks.workunit.client.1.smithi148.stdout:7/681: rename d8/d14/da8/fc5 to d8/d38/d3c/da5/db4/de9/fed 0 2024-03-29T08:07:25.502 INFO:tasks.workunit.client.1.smithi148.stdout:6/720: dwrite d6/d9/d1e/d36/d4d/d9d/f31 [0,4194304] 0 2024-03-29T08:07:25.504 INFO:tasks.workunit.client.1.smithi148.stdout:3/772: readlink de/l85 0 2024-03-29T08:07:25.505 INFO:tasks.workunit.client.0.smithi037.stdout:7/543: dwrite dd/d1d/d34/f37 [0,4194304] 0 2024-03-29T08:07:25.508 INFO:tasks.workunit.client.1.smithi148.stdout:7/682: creat d8/d29/db8/fee x:0 0 0 2024-03-29T08:07:25.509 INFO:tasks.workunit.client.0.smithi037.stdout:3/493: rename dd/d15/f19 to dd/d15/d18/d8e/fa8 0 2024-03-29T08:07:25.510 INFO:tasks.workunit.client.1.smithi148.stdout:7/683: chown d8/d38/d3c/da9/f39 876 1 2024-03-29T08:07:25.512 INFO:tasks.workunit.client.1.smithi148.stdout:7/684: stat d8/d38/d3c/da9 0 2024-03-29T08:07:25.513 INFO:tasks.workunit.client.0.smithi037.stdout:4/533: write d3/d4/d15/d1d/db/d45/d1e/f4e [2536739,40943] 0 2024-03-29T08:07:25.516 INFO:tasks.workunit.client.1.smithi148.stdout:8/702: dread d17/d18/d26/d4f/fee [0,4194304] 0 2024-03-29T08:07:25.517 INFO:tasks.workunit.client.0.smithi037.stdout:4/534: chown d3/d4/d15/d1d/db/d45/d93/d96/da0 61525 1 2024-03-29T08:07:25.518 INFO:tasks.workunit.client.0.smithi037.stdout:7/544: creat dd/d21/d35/d60/d94/fb0 x:0 0 0 2024-03-29T08:07:25.524 INFO:tasks.workunit.client.1.smithi148.stdout:8/703: creat d17/d4e/fef x:0 0 0 2024-03-29T08:07:25.524 INFO:tasks.workunit.client.1.smithi148.stdout:7/685: mknod d8/d38/d67/dc1/cef 0 2024-03-29T08:07:25.529 INFO:tasks.workunit.client.1.smithi148.stdout:1/655: creat d3/de/d1d/d47/d6d/d84/d8c/fce x:0 0 0 2024-03-29T08:07:25.532 INFO:tasks.workunit.client.1.smithi148.stdout:0/716: sync 2024-03-29T08:07:25.532 INFO:tasks.workunit.client.1.smithi148.stdout:5/711: sync 2024-03-29T08:07:25.532 INFO:tasks.workunit.client.1.smithi148.stdout:1/656: read d3/de/d25/d35/dc2/f99 [2857792,93413] 0 2024-03-29T08:07:25.533 INFO:tasks.workunit.client.1.smithi148.stdout:3/773: write de/d24/d33/d39/d46/d30/d37/d62/d63/dbf/fd1 [487511,23941] 0 2024-03-29T08:07:25.534 INFO:tasks.workunit.client.1.smithi148.stdout:1/657: chown d3/d18/dae 1 1 2024-03-29T08:07:25.536 INFO:tasks.workunit.client.1.smithi148.stdout:8/704: truncate d17/d49/d9a/dbe/f73 1425989 0 2024-03-29T08:07:25.538 INFO:tasks.workunit.client.1.smithi148.stdout:4/719: getdents df/d2c/d51/d77/d26/d34/d8a/d98 0 2024-03-29T08:07:25.538 INFO:tasks.workunit.client.1.smithi148.stdout:0/717: creat db/de/df/d11/d42/fda x:0 0 0 2024-03-29T08:07:25.540 INFO:tasks.workunit.client.1.smithi148.stdout:9/752: write db/d22/f51 [1973603,44339] 0 2024-03-29T08:07:25.541 INFO:tasks.workunit.client.1.smithi148.stdout:2/801: creat d7/da/d33/d3c/f101 x:0 0 0 2024-03-29T08:07:25.543 INFO:tasks.workunit.client.1.smithi148.stdout:0/718: stat db/de/d8e/da9 0 2024-03-29T08:07:25.543 INFO:tasks.workunit.client.0.smithi037.stdout:4/535: dwrite d3/d4/d27/f7f [4194304,4194304] 0 2024-03-29T08:07:25.545 INFO:tasks.workunit.client.1.smithi148.stdout:1/658: creat d3/d18/d4d/d8f/fcf x:0 0 0 2024-03-29T08:07:25.548 INFO:tasks.workunit.client.0.smithi037.stdout:3/494: chown dd/d15/d27/d33/d75 14 1 2024-03-29T08:07:25.549 INFO:tasks.workunit.client.1.smithi148.stdout:8/705: chown c16 3338 1 2024-03-29T08:07:25.550 INFO:tasks.workunit.client.0.smithi037.stdout:4/536: symlink d3/d4/d15/d1d/db/d72/lac 0 2024-03-29T08:07:25.558 INFO:tasks.workunit.client.0.smithi037.stdout:4/537: chown d3/d4/d15/d1d/d4f/c55 104841785 1 2024-03-29T08:07:25.559 INFO:tasks.workunit.client.1.smithi148.stdout:4/720: rename df/d2c/d51/d77/d26/d9b/da7/f49 to df/d2c/d51/d77/d26/d9b/fe4 0 2024-03-29T08:07:25.560 INFO:tasks.workunit.client.1.smithi148.stdout:9/753: mknod db/d22/d26/d54/d4d/d35/d4b/d58/df1/cfd 0 2024-03-29T08:07:25.564 INFO:tasks.workunit.client.1.smithi148.stdout:0/719: creat db/de/df/d2b/dae/fdb x:0 0 0 2024-03-29T08:07:25.565 INFO:tasks.workunit.client.0.smithi037.stdout:3/495: write dd/f1b [380608,40894] 0 2024-03-29T08:07:25.566 INFO:tasks.workunit.client.1.smithi148.stdout:4/721: stat df/d2c/f8d 0 2024-03-29T08:07:25.567 INFO:tasks.workunit.client.1.smithi148.stdout:6/721: dwrite d6/d9/d1e/d36/db1/f7c [0,4194304] 0 2024-03-29T08:07:25.569 INFO:tasks.workunit.client.1.smithi148.stdout:5/712: dwrite d0/d46/d9e/fc1 [0,4194304] 0 2024-03-29T08:07:25.573 INFO:tasks.workunit.client.1.smithi148.stdout:8/706: read d17/d18/d26/d63/d66/d5f/f6c [938459,24212] 0 2024-03-29T08:07:25.574 INFO:tasks.workunit.client.0.smithi037.stdout:3/496: creat dd/fa9 x:0 0 0 2024-03-29T08:07:25.577 INFO:tasks.workunit.client.1.smithi148.stdout:1/659: rename d3/de/d25/d35/dc2/f99 to d3/de/d25/d35/fd0 0 2024-03-29T08:07:25.578 INFO:tasks.workunit.client.1.smithi148.stdout:6/722: chown d6/d9/d1e/d27/cbd 45 1 2024-03-29T08:07:25.586 INFO:tasks.workunit.client.1.smithi148.stdout:7/686: dwrite d8/f15 [0,4194304] 0 2024-03-29T08:07:25.592 INFO:tasks.workunit.client.1.smithi148.stdout:1/660: symlink d3/d18/d4d/d76/d85/ld1 0 2024-03-29T08:07:25.593 INFO:tasks.workunit.client.1.smithi148.stdout:3/774: dwrite de/f2c [0,4194304] 0 2024-03-29T08:07:25.593 INFO:tasks.workunit.client.1.smithi148.stdout:9/754: unlink db/d10/d2d/d4e/d5f/dd5/lfa 0 2024-03-29T08:07:25.596 INFO:tasks.workunit.client.1.smithi148.stdout:6/723: rmdir d6/d9/d1e/d36/d4d/d9d 39 2024-03-29T08:07:25.596 INFO:tasks.workunit.client.1.smithi148.stdout:2/802: dwrite d7/da/d4c/dc4/fdd [0,4194304] 0 2024-03-29T08:07:25.598 INFO:tasks.workunit.client.1.smithi148.stdout:1/661: chown d3/d55/c45 505514834 1 2024-03-29T08:07:25.602 INFO:tasks.workunit.client.1.smithi148.stdout:4/722: write df/d71/f7b [1484096,5034] 0 2024-03-29T08:07:25.602 INFO:tasks.workunit.client.1.smithi148.stdout:3/775: mkdir de/d24/d33/d39/d46/d70/d101 0 2024-03-29T08:07:25.603 INFO:tasks.workunit.client.1.smithi148.stdout:9/755: symlink db/d10/d2d/d4e/d5f/d6e/d7a/dbd/lfe 0 2024-03-29T08:07:25.607 INFO:tasks.workunit.client.0.smithi037.stdout:3/497: dwrite dd/d46/f38 [4194304,4194304] 0 2024-03-29T08:07:25.607 INFO:tasks.workunit.client.1.smithi148.stdout:8/707: write d17/d74/f54 [141670,6756] 0 2024-03-29T08:07:25.607 INFO:tasks.workunit.client.1.smithi148.stdout:1/662: truncate d3/f5d 219464 0 2024-03-29T08:07:25.621 INFO:tasks.workunit.client.1.smithi148.stdout:1/663: symlink d3/de/d25/d51/ld2 0 2024-03-29T08:07:25.623 INFO:tasks.workunit.client.1.smithi148.stdout:3/776: creat de/d24/d33/d39/d46/d30/d37/d62/d63/f102 x:0 0 0 2024-03-29T08:07:25.629 INFO:tasks.workunit.client.1.smithi148.stdout:4/723: creat df/d2c/d51/d77/d26/d34/d8a/d98/fe5 x:0 0 0 2024-03-29T08:07:25.630 INFO:tasks.workunit.client.1.smithi148.stdout:5/713: dwrite d0/db/d1e/d6e/da8/fd6 [0,4194304] 0 2024-03-29T08:07:25.630 INFO:tasks.workunit.client.1.smithi148.stdout:1/664: mknod d3/de/d25/d35/dc2/cd3 0 2024-03-29T08:07:25.631 INFO:tasks.workunit.client.1.smithi148.stdout:2/803: dread d7/da/d52/f63 [0,4194304] 0 2024-03-29T08:07:25.633 INFO:tasks.workunit.client.1.smithi148.stdout:4/724: rmdir df/d89 39 2024-03-29T08:07:25.635 INFO:tasks.workunit.client.1.smithi148.stdout:3/777: chown de/d24/d33/d39/d46/d30/d37/d62/d63/dbf/ff0 8016 1 2024-03-29T08:07:25.636 INFO:tasks.workunit.client.1.smithi148.stdout:0/720: dwrite db/de/df/d11/d1f/f92 [0,4194304] 0 2024-03-29T08:07:25.638 INFO:tasks.workunit.client.1.smithi148.stdout:2/804: write d7/da/f40 [4536071,42693] 0 2024-03-29T08:07:25.638 INFO:tasks.workunit.client.1.smithi148.stdout:1/665: creat d3/d18/dae/fd4 x:0 0 0 2024-03-29T08:07:25.639 INFO:tasks.workunit.client.1.smithi148.stdout:3/778: write de/f80 [1842634,18881] 0 2024-03-29T08:07:25.645 INFO:tasks.workunit.client.1.smithi148.stdout:7/687: write d8/d38/d3c/f7e [2429218,28109] 0 2024-03-29T08:07:25.647 INFO:tasks.workunit.client.1.smithi148.stdout:8/708: dwrite d17/d74/f48 [0,4194304] 0 2024-03-29T08:07:25.649 INFO:tasks.workunit.client.1.smithi148.stdout:5/714: creat d0/db/d1e/d35/d42/d96/fe8 x:0 0 0 2024-03-29T08:07:25.649 INFO:tasks.workunit.client.1.smithi148.stdout:7/688: stat d8/d38/d6a/d91 0 2024-03-29T08:07:25.652 INFO:tasks.workunit.client.1.smithi148.stdout:2/805: readlink d7/da/l44 0 2024-03-29T08:07:25.653 INFO:tasks.workunit.client.1.smithi148.stdout:1/666: creat d3/de/d1d/fd5 x:0 0 0 2024-03-29T08:07:25.653 INFO:tasks.workunit.client.1.smithi148.stdout:7/689: write f4 [98662,48764] 0 2024-03-29T08:07:25.654 INFO:tasks.workunit.client.0.smithi037.stdout:1/574: sync 2024-03-29T08:07:25.655 INFO:tasks.workunit.client.0.smithi037.stdout:3/498: dread dd/d5e/d7a/f4f [0,4194304] 0 2024-03-29T08:07:25.658 INFO:tasks.workunit.client.1.smithi148.stdout:3/779: mknod de/d24/d33/c103 0 2024-03-29T08:07:25.661 INFO:tasks.workunit.client.0.smithi037.stdout:1/575: symlink d2/d6/dc/d18/d22/d4f/d7c/lc6 0 2024-03-29T08:07:25.664 INFO:tasks.workunit.client.1.smithi148.stdout:4/725: dread df/d2c/f8d [0,4194304] 0 2024-03-29T08:07:25.668 INFO:tasks.workunit.client.1.smithi148.stdout:4/726: fsync df/d2c/d51/d77/d26/d34/d43/fc8 0 2024-03-29T08:07:25.669 INFO:tasks.workunit.client.1.smithi148.stdout:8/709: mkdir d17/d18/d26/d63/d66/d5f/dd0/df0 0 2024-03-29T08:07:25.671 INFO:tasks.workunit.client.1.smithi148.stdout:6/724: dwrite d6/d9/d1e/d20/d30/f33 [4194304,4194304] 0 2024-03-29T08:07:25.671 INFO:tasks.workunit.client.1.smithi148.stdout:2/806: rename d7/da/d33/ded/d2e/ca0 to d7/da/d33/d3c/da5/c102 0 2024-03-29T08:07:25.671 INFO:tasks.workunit.client.0.smithi037.stdout:3/499: dread dd/d46/d2d/d3a/d5f/f6a [0,4194304] 0 2024-03-29T08:07:25.679 INFO:tasks.workunit.client.0.smithi037.stdout:1/576: truncate d2/d6/dc/d58/f71 3756818 0 2024-03-29T08:07:25.682 INFO:tasks.workunit.client.0.smithi037.stdout:1/577: creat d2/d6/d13/dae/fc7 x:0 0 0 2024-03-29T08:07:25.687 INFO:tasks.workunit.client.1.smithi148.stdout:8/710: fsync d17/d18/fe4 0 2024-03-29T08:07:25.688 INFO:tasks.workunit.client.1.smithi148.stdout:4/727: symlink df/le6 0 2024-03-29T08:07:25.692 INFO:tasks.workunit.client.1.smithi148.stdout:2/807: creat d7/d8c/f103 x:0 0 0 2024-03-29T08:07:25.693 INFO:tasks.workunit.client.0.smithi037.stdout:1/578: link d2/d6/d13/l70 d2/d4b/da1/lc8 0 2024-03-29T08:07:25.694 INFO:tasks.workunit.client.1.smithi148.stdout:9/756: dread db/d22/f89 [0,4194304] 0 2024-03-29T08:07:25.696 INFO:tasks.workunit.client.0.smithi037.stdout:1/579: read - d2/d6/dc/d7e/f91 zero size 2024-03-29T08:07:25.699 INFO:tasks.workunit.client.1.smithi148.stdout:4/728: write df/d2c/d51/fe2 [387623,113505] 0 2024-03-29T08:07:25.701 INFO:tasks.workunit.client.1.smithi148.stdout:5/715: dwrite d0/db/d7b/d16/d2e/f56 [0,4194304] 0 2024-03-29T08:07:25.704 INFO:tasks.workunit.client.0.smithi037.stdout:3/500: dread dd/d15/f62 [0,4194304] 0 2024-03-29T08:07:25.708 INFO:tasks.workunit.client.0.smithi037.stdout:1/580: link d2/d6/dc/d18/d1d/d5e/cb4 d2/d6/d13/cc9 0 2024-03-29T08:07:25.711 INFO:tasks.workunit.client.0.smithi037.stdout:1/581: rmdir d2/d6/dc/d18/d22/d4f 39 2024-03-29T08:07:25.713 INFO:tasks.workunit.client.1.smithi148.stdout:1/667: dwrite d3/d18/d4d/d76/d85/fb0 [0,4194304] 0 2024-03-29T08:07:25.714 INFO:tasks.workunit.client.1.smithi148.stdout:0/721: dread db/de/f4b [0,4194304] 0 2024-03-29T08:07:25.714 INFO:tasks.workunit.client.1.smithi148.stdout:2/808: getdents d7/d8c 0 2024-03-29T08:07:25.718 INFO:tasks.workunit.client.1.smithi148.stdout:9/757: creat db/d10/d2d/d4e/da6/dd1/dd9/fff x:0 0 0 2024-03-29T08:07:25.719 INFO:tasks.workunit.client.0.smithi037.stdout:1/582: write d2/d6/dc/d58/fa2 [695486,62510] 0 2024-03-29T08:07:25.721 INFO:tasks.workunit.client.1.smithi148.stdout:3/780: dwrite de/d24/d33/d39/da3/fcd [4194304,4194304] 0 2024-03-29T08:07:25.724 INFO:tasks.workunit.client.0.smithi037.stdout:1/583: symlink d2/d6/dc/d18/d1d/d5e/d72/lca 0 2024-03-29T08:07:25.724 INFO:tasks.workunit.client.1.smithi148.stdout:2/809: write d7/da/d33/ded/f16 [634880,46690] 0 2024-03-29T08:07:25.725 INFO:tasks.workunit.client.1.smithi148.stdout:8/711: dwrite f7 [0,4194304] 0 2024-03-29T08:07:25.728 INFO:tasks.workunit.client.1.smithi148.stdout:1/668: creat d3/fd6 x:0 0 0 2024-03-29T08:07:25.728 INFO:tasks.workunit.client.1.smithi148.stdout:4/729: fsync df/d2c/d51/d77/d26/d9b/da7/f8b 0 2024-03-29T08:07:25.729 INFO:tasks.workunit.client.0.smithi037.stdout:1/584: chown d2/d6/dc/d5d/d62/f97 4296 1 2024-03-29T08:07:25.731 INFO:tasks.workunit.client.0.smithi037.stdout:1/585: dread - d2/d4b/d80/fc3 zero size 2024-03-29T08:07:25.734 INFO:tasks.workunit.client.1.smithi148.stdout:9/758: symlink db/d22/d26/d54/d4d/d35/d4b/d5a/l100 0 2024-03-29T08:07:25.736 INFO:tasks.workunit.client.1.smithi148.stdout:3/781: truncate de/d24/d33/d39/d46/d70/fe5 192292 0 2024-03-29T08:07:25.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:25.738+0000 7f48cec00700 1 -- 172.21.15.37:0/2779142792 --> [v2:172.21.15.148:6828/1134897184,v1:172.21.15.148:6829/1134897184] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f48c81180b0 con 0x7f48ac06afd0 2024-03-29T08:07:25.739 INFO:tasks.workunit.client.1.smithi148.stdout:6/725: dread d6/d9/d1e/d36/d4d/d9d/f4e [0,4194304] 0 2024-03-29T08:07:25.740 INFO:tasks.workunit.client.1.smithi148.stdout:4/730: unlink df/d2c/d51/d77/d26/d9b/da7/l33 0 2024-03-29T08:07:25.745 INFO:tasks.workunit.client.1.smithi148.stdout:4/731: write df/f14 [5367700,2564] 0 2024-03-29T08:07:25.750 INFO:tasks.workunit.client.1.smithi148.stdout:5/716: truncate d0/db/d1e/d6e/f92 7022 0 2024-03-29T08:07:25.751 INFO:tasks.workunit.client.0.smithi037.stdout:1/586: dwrite d2/d6/dc/d58/d67/fa3 [0,4194304] 0 2024-03-29T08:07:25.751 INFO:tasks.workunit.client.1.smithi148.stdout:0/722: rename db/de/df/d2b/dae/ld5 to db/de/df/d2b/ldc 0 2024-03-29T08:07:25.752 INFO:tasks.workunit.client.1.smithi148.stdout:9/759: symlink db/d10/l101 0 2024-03-29T08:07:25.754 INFO:tasks.workunit.client.1.smithi148.stdout:1/669: write d3/de/d6e/f88 [201734,116708] 0 2024-03-29T08:07:25.754 INFO:tasks.workunit.client.1.smithi148.stdout:6/726: chown d6/d9/d1e/d27/f9b 1088985979 1 2024-03-29T08:07:25.758 INFO:tasks.workunit.client.1.smithi148.stdout:3/782: creat de/d24/d33/d39/d46/d70/d101/f104 x:0 0 0 2024-03-29T08:07:25.758 INFO:tasks.workunit.client.1.smithi148.stdout:7/690: sync 2024-03-29T08:07:25.760 INFO:tasks.workunit.client.1.smithi148.stdout:2/810: dwrite d7/d8c/da4/d8e/dda/f100 [0,4194304] 0 2024-03-29T08:07:25.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:25.763+0000 7f48bfe00700 1 -- 172.21.15.37:0/2779142792 <== mgr.14602 v2:172.21.15.148:6828/1134897184 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3388 (secure 0 0 0) 0x7f48c81180b0 con 0x7f48ac06afd0 2024-03-29T08:07:25.763 INFO:teuthology.orchestra.run.smithi037.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-29T08:07:25.763 INFO:teuthology.orchestra.run.smithi037.stdout:alertmanager.smithi037 smithi037 *:9093,9094 starting - - - - 2024-03-29T08:07:25.763 INFO:teuthology.orchestra.run.smithi037.stdout:crash.smithi037 smithi037 running (10m) 70s ago 10m 6651k - 17.2.7-789-g4213e17c cf22eb8683be e0343935b25e 2024-03-29T08:07:25.763 INFO:teuthology.orchestra.run.smithi037.stdout:crash.smithi148 smithi148 running (9m) 76s ago 9m 6656k - 17.2.7-789-g4213e17c cf22eb8683be ff5164c552c1 2024-03-29T08:07:25.763 INFO:teuthology.orchestra.run.smithi037.stdout:grafana.smithi037 smithi037 *:3000 starting - - - - 2024-03-29T08:07:25.763 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi037.iifsmh smithi037 running (3m) 70s ago 3m 17.7M - 17.2.7-789-g4213e17c cf22eb8683be 074e1acf331f 2024-03-29T08:07:25.763 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi037.mmwgxe smithi037 running (3m) 70s ago 3m 64.9M - 17.2.7-789-g4213e17c cf22eb8683be 9a5b1e72de5c 2024-03-29T08:07:25.763 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi148.qukpjh smithi148 running (3m) 76s ago 3m 11.7M - 17.2.7-789-g4213e17c cf22eb8683be 8e469782a273 2024-03-29T08:07:25.763 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi148.sqteih smithi148 running (3m) 76s ago 3m 11.6M - 17.2.7-789-g4213e17c cf22eb8683be 8e68f858bb22 2024-03-29T08:07:25.763 INFO:teuthology.orchestra.run.smithi037.stdout:mgr.smithi037.wjikii smithi037 *:9283 running (11m) 70s ago 11m 156M - 17.2.7-789-g4213e17c cf22eb8683be 60172b53efa2 2024-03-29T08:07:25.763 INFO:teuthology.orchestra.run.smithi037.stdout:mgr.smithi148.onhgvm smithi148 *:8443,9283 running (97s) 76s ago 9m 479M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 0c02ffa63851 2024-03-29T08:07:25.764 INFO:teuthology.orchestra.run.smithi037.stdout:mon.smithi037 smithi037 running (11m) 70s ago 12m 48.6M 2048M 17.2.7-789-g4213e17c cf22eb8683be bb2101505730 2024-03-29T08:07:25.764 INFO:teuthology.orchestra.run.smithi037.stdout:mon.smithi148 smithi148 running (8m) 76s ago 8m 38.5M 2048M 17.2.7-789-g4213e17c cf22eb8683be 18839ed2952a 2024-03-29T08:07:25.764 INFO:teuthology.orchestra.run.smithi037.stdout:node-exporter.smithi037 smithi037 *:9100 starting - - - - 2024-03-29T08:07:25.764 INFO:teuthology.orchestra.run.smithi037.stdout:node-exporter.smithi148 smithi148 *:9100 starting - - - - 2024-03-29T08:07:25.764 INFO:teuthology.orchestra.run.smithi037.stdout:osd.0 smithi037 running (8m) 70s ago 8m 67.4M 1619M 17.2.7-789-g4213e17c cf22eb8683be 559fccd0ebe1 2024-03-29T08:07:25.764 INFO:teuthology.orchestra.run.smithi037.stdout:osd.1 smithi037 running (7m) 70s ago 7m 68.5M 1619M 17.2.7-789-g4213e17c cf22eb8683be 5582cff11b7a 2024-03-29T08:07:25.764 INFO:teuthology.orchestra.run.smithi037.stdout:osd.2 smithi037 running (7m) 70s ago 7m 60.6M 1619M 17.2.7-789-g4213e17c cf22eb8683be f809bfab13e5 2024-03-29T08:07:25.764 INFO:teuthology.orchestra.run.smithi037.stdout:osd.3 smithi148 running (6m) 76s ago 6m 67.0M 2642M 17.2.7-789-g4213e17c cf22eb8683be a0a8224bb4b0 2024-03-29T08:07:25.764 INFO:teuthology.orchestra.run.smithi037.stdout:osd.4 smithi148 running (6m) 76s ago 5m 63.7M 2642M 17.2.7-789-g4213e17c cf22eb8683be a00b69047ed7 2024-03-29T08:07:25.764 INFO:teuthology.orchestra.run.smithi037.stdout:osd.5 smithi148 running (5m) 76s ago 5m 65.5M 2642M 17.2.7-789-g4213e17c cf22eb8683be 044a3f7db7f8 2024-03-29T08:07:25.764 INFO:teuthology.orchestra.run.smithi037.stdout:prometheus.smithi037 smithi037 *:9095 starting - - - - 2024-03-29T08:07:25.765 INFO:tasks.workunit.client.1.smithi148.stdout:9/760: dread - db/d10/d2d/d4e/da6/fac zero size 2024-03-29T08:07:25.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:25.767+0000 7f48cec00700 1 -- 172.21.15.37:0/2779142792 >> [v2:172.21.15.148:6828/1134897184,v1:172.21.15.148:6829/1134897184] conn(0x7f48ac06afd0 msgr2=0x7f48ac06d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:07:25.767 INFO:tasks.workunit.client.1.smithi148.stdout:4/732: write df/d2c/d51/d77/d26/d9b/fc1 [690638,17792] 0 2024-03-29T08:07:25.767 INFO:tasks.workunit.client.0.smithi037.stdout:1/587: getdents d2/d6/dc/d18/d1d/d5e/d72 0 2024-03-29T08:07:25.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:25.767+0000 7f48cec00700 1 --2- 172.21.15.37:0/2779142792 >> [v2:172.21.15.148:6828/1134897184,v1:172.21.15.148:6829/1134897184] conn(0x7f48ac06afd0 0x7f48ac06d480 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f48b8009c10 tx=0x7f48b8000f90 comp rx=0 tx=0).stop 2024-03-29T08:07:25.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:25.767+0000 7f48cec00700 1 -- 172.21.15.37:0/2779142792 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f48c806f9d0 msgr2=0x7f48c8114430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:07:25.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:25.767+0000 7f48cec00700 1 --2- 172.21.15.37:0/2779142792 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f48c806f9d0 0x7f48c8114430 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f48b4004890 tx=0x7f48b4009e30 comp rx=0 tx=0).stop 2024-03-29T08:07:25.769 INFO:tasks.workunit.client.1.smithi148.stdout:1/670: creat d3/d55/d7e/fd7 x:0 0 0 2024-03-29T08:07:25.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:25.767+0000 7f48cec00700 1 -- 172.21.15.37:0/2779142792 shutdown_connections 2024-03-29T08:07:25.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:25.767+0000 7f48cec00700 1 --2- 172.21.15.37:0/2779142792 >> [v2:172.21.15.148:6828/1134897184,v1:172.21.15.148:6829/1134897184] conn(0x7f48ac06afd0 0x7f48ac06d480 secure :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f48b8009c10 tx=0x7f48b8000f90 comp rx=0 tx=0).stop 2024-03-29T08:07:25.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:25.767+0000 7f48cec00700 1 --2- 172.21.15.37:0/2779142792 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f48c806e9a0 0x7f48c8113ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:07:25.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:25.767+0000 7f48cec00700 1 --2- 172.21.15.37:0/2779142792 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f48c806f9d0 0x7f48c8114430 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:07:25.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:25.767+0000 7f48cec00700 1 -- 172.21.15.37:0/2779142792 >> 172.21.15.37:0/2779142792 conn(0x7f48c8069de0 msgr2=0x7f48c806b8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:07:25.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:25.767+0000 7f48cec00700 1 -- 172.21.15.37:0/2779142792 shutdown_connections 2024-03-29T08:07:25.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:25.767+0000 7f48cec00700 1 -- 172.21.15.37:0/2779142792 wait complete. 2024-03-29T08:07:25.772 INFO:tasks.workunit.client.1.smithi148.stdout:3/783: symlink de/d3b/l105 0 2024-03-29T08:07:25.775 INFO:tasks.workunit.client.1.smithi148.stdout:1/671: write d3/de/d21/d7f/fa0 [74634,26158] 0 2024-03-29T08:07:25.776 INFO:tasks.workunit.client.0.smithi037.stdout:1/588: rename d2/d6/dc/f25 to d2/d6/dc/fcb 0 2024-03-29T08:07:25.779 INFO:tasks.workunit.client.0.smithi037.stdout:1/589: dread - d2/d6/d13/f93 zero size 2024-03-29T08:07:25.780 INFO:tasks.workunit.client.1.smithi148.stdout:2/811: symlink d7/da/d33/d3c/d54/d70/l104 0 2024-03-29T08:07:25.791 INFO:tasks.workunit.client.1.smithi148.stdout:4/733: creat df/d89/ddd/fe7 x:0 0 0 2024-03-29T08:07:25.791 INFO:tasks.workunit.client.1.smithi148.stdout:8/712: dread d17/d74/d86/fe5 [0,4194304] 0 2024-03-29T08:07:25.792 INFO:tasks.workunit.client.1.smithi148.stdout:0/723: dwrite db/dc/d78/f8b [0,4194304] 0 2024-03-29T08:07:25.797 INFO:tasks.workunit.client.1.smithi148.stdout:1/672: creat d3/de/d25/d35/dc2/dac/fd8 x:0 0 0 2024-03-29T08:07:25.801 INFO:tasks.workunit.client.1.smithi148.stdout:9/761: write db/d22/d26/d54/d4d/f18 [983158,69333] 0 2024-03-29T08:07:25.803 INFO:tasks.workunit.client.1.smithi148.stdout:7/691: dwrite d8/d29/d44/d49/f68 [4194304,4194304] 0 2024-03-29T08:07:25.804 INFO:tasks.workunit.client.1.smithi148.stdout:6/727: dwrite d6/d9/d1e/d36/d4d/d9d/fce [0,4194304] 0 2024-03-29T08:07:25.806 INFO:tasks.workunit.client.1.smithi148.stdout:2/812: mknod d7/d8c/da4/dc8/c105 0 2024-03-29T08:07:25.807 INFO:tasks.workunit.client.1.smithi148.stdout:4/734: creat df/d2c/d51/d77/d17/fe8 x:0 0 0 2024-03-29T08:07:25.810 INFO:tasks.workunit.client.1.smithi148.stdout:8/713: mkdir d17/d49/d9a/dbe/d6f/d64/df1 0 2024-03-29T08:07:25.814 INFO:tasks.workunit.client.1.smithi148.stdout:1/673: symlink d3/de/d25/d51/ld9 0 2024-03-29T08:07:25.818 INFO:tasks.workunit.client.1.smithi148.stdout:6/728: fdatasync d6/d9/d1e/d20/d6b/fe3 0 2024-03-29T08:07:25.821 INFO:tasks.workunit.client.1.smithi148.stdout:5/717: dwrite d0/db/d1e/d35/d42/d67/d9a/f88 [0,4194304] 0 2024-03-29T08:07:25.824 INFO:tasks.workunit.client.1.smithi148.stdout:7/692: creat d8/d38/ff0 x:0 0 0 2024-03-29T08:07:25.825 INFO:tasks.workunit.client.1.smithi148.stdout:1/674: mkdir d3/d18/dae/dda 0 2024-03-29T08:07:25.825 INFO:tasks.workunit.client.1.smithi148.stdout:2/813: chown d7/da/d33/d3c/d54/f93 54 1 2024-03-29T08:07:25.827 INFO:tasks.workunit.client.1.smithi148.stdout:0/724: dread db/dc/daa/d47/d7c/fb4 [4194304,4194304] 0 2024-03-29T08:07:25.827 INFO:tasks.workunit.client.1.smithi148.stdout:5/718: stat d0/db/f3e 0 2024-03-29T08:07:25.837 INFO:tasks.workunit.client.1.smithi148.stdout:6/729: chown d6/d9/d1e/fbb 0 1 2024-03-29T08:07:25.837 INFO:tasks.workunit.client.1.smithi148.stdout:2/814: creat d7/da/dcc/f106 x:0 0 0 2024-03-29T08:07:25.838 INFO:tasks.workunit.client.1.smithi148.stdout:5/719: chown d0/db/d1e/d79 134 1 2024-03-29T08:07:25.838 INFO:tasks.workunit.client.1.smithi148.stdout:3/784: dwrite de/d24/d33/d39/d46/d30/d37/d79/f7c [4194304,4194304] 0 2024-03-29T08:07:25.841 INFO:tasks.workunit.client.1.smithi148.stdout:8/714: getdents d17/d49/d9a/dbe/d6f/d64/da5 0 2024-03-29T08:07:25.843 INFO:tasks.workunit.client.1.smithi148.stdout:3/785: chown de/f15 32 1 2024-03-29T08:07:25.843 INFO:tasks.workunit.client.1.smithi148.stdout:9/762: link db/d22/d26/d54/f45 db/d10/d6d/d73/da8/deb/dec/dde/f102 0 2024-03-29T08:07:25.843 INFO:tasks.workunit.client.1.smithi148.stdout:0/725: symlink db/dc/daa/d47/d4c/ldd 0 2024-03-29T08:07:25.849 INFO:tasks.workunit.client.1.smithi148.stdout:2/815: creat d7/da/d33/d3c/d31/f107 x:0 0 0 2024-03-29T08:07:25.852 INFO:tasks.workunit.client.1.smithi148.stdout:4/735: dwrite df/d2c/d51/d77/f56 [0,4194304] 0 2024-03-29T08:07:25.855 INFO:tasks.workunit.client.1.smithi148.stdout:3/786: write de/d24/d33/d39/da3/fcd [6970387,57976] 0 2024-03-29T08:07:25.856 INFO:tasks.workunit.client.1.smithi148.stdout:1/675: rename d3/de/d1d/d47/d6d/d72/cbf to d3/de/d1d/d47/d6d/d72/cdb 0 2024-03-29T08:07:25.858 INFO:tasks.workunit.client.1.smithi148.stdout:8/715: link d17/d49/d9a/dbe/f76 d17/d18/ff2 0 2024-03-29T08:07:25.860 INFO:tasks.workunit.client.1.smithi148.stdout:7/693: dwrite d8/d29/f54 [4194304,4194304] 0 2024-03-29T08:07:25.860 INFO:tasks.workunit.client.1.smithi148.stdout:6/730: dread d6/d9/d1e/d27/d48/f7b [0,4194304] 0 2024-03-29T08:07:25.860 INFO:tasks.workunit.client.1.smithi148.stdout:2/816: write d7/d8c/d91/dd5/df3/ff7 [265762,11279] 0 2024-03-29T08:07:25.861 INFO:tasks.workunit.client.1.smithi148.stdout:5/720: mkdir d0/db/d7b/d16/d2e/ddd/de9 0 2024-03-29T08:07:25.866 INFO:tasks.workunit.client.0.smithi037.stdout:5/618: sync 2024-03-29T08:07:25.869 INFO:tasks.workunit.client.1.smithi148.stdout:1/676: truncate d3/d18/d71/fb2 4563465 0 2024-03-29T08:07:25.872 INFO:tasks.workunit.client.1.smithi148.stdout:2/817: write d7/da/d33/d3c/d54/f95 [1447296,125412] 0 2024-03-29T08:07:25.873 INFO:tasks.workunit.client.1.smithi148.stdout:3/787: fdatasync de/d24/d33/d39/d46/d30/fcf 0 2024-03-29T08:07:25.875 INFO:tasks.workunit.client.1.smithi148.stdout:8/716: creat d17/d49/d9a/dbe/d6f/ff3 x:0 0 0 2024-03-29T08:07:25.877 INFO:tasks.workunit.client.1.smithi148.stdout:4/736: write df/d2c/d51/f67 [888358,119823] 0 2024-03-29T08:07:25.881 INFO:tasks.workunit.client.1.smithi148.stdout:8/717: stat d17/d18/dbd/ld4 0 2024-03-29T08:07:25.884 INFO:tasks.workunit.client.1.smithi148.stdout:6/731: creat d6/d9/d1e/d36/d4d/d9d/d28/fea x:0 0 0 2024-03-29T08:07:25.884 INFO:tasks.workunit.client.1.smithi148.stdout:0/726: dwrite db/dc/daa/d47/f96 [0,4194304] 0 2024-03-29T08:07:25.885 INFO:tasks.workunit.client.1.smithi148.stdout:5/721: write d0/db/d1e/d35/d42/d67/d76/d81/d98/dd5/f9d [1549849,119011] 0 2024-03-29T08:07:25.885 INFO:tasks.workunit.client.1.smithi148.stdout:7/694: rename d8/d38/d3c/da5/db4/cd1 to d8/d38/d3c/da5/de3/cf1 0 2024-03-29T08:07:25.888 INFO:tasks.workunit.client.1.smithi148.stdout:0/727: fsync db/de/df/d2b/f93 0 2024-03-29T08:07:25.889 INFO:tasks.workunit.client.1.smithi148.stdout:3/788: creat de/d24/d33/d39/d46/d30/f106 x:0 0 0 2024-03-29T08:07:25.892 INFO:tasks.workunit.client.1.smithi148.stdout:2/818: fsync d7/da/d33/ded/d2e/d37/f43 0 2024-03-29T08:07:25.896 INFO:tasks.workunit.client.1.smithi148.stdout:9/763: truncate db/d22/d26/d54/d4d/d35/d4b/dc3/fe9 1709981 0 2024-03-29T08:07:25.898 INFO:tasks.workunit.client.1.smithi148.stdout:7/695: link d8/d14/f1c d8/d29/db8/de0/ff2 0 2024-03-29T08:07:25.903 INFO:tasks.workunit.client.1.smithi148.stdout:9/764: write db/d10/d2d/d4e/da6/fac [226737,87755] 0 2024-03-29T08:07:25.906 INFO:tasks.workunit.client.1.smithi148.stdout:3/789: write de/f2e [410404,40546] 0 2024-03-29T08:07:25.906 INFO:tasks.workunit.client.1.smithi148.stdout:5/722: getdents d0/db/d7b/d16/d2e 0 2024-03-29T08:07:25.910 INFO:tasks.workunit.client.1.smithi148.stdout:0/728: write db/de/f4b [5859075,74502] 0 2024-03-29T08:07:25.912 INFO:tasks.workunit.client.1.smithi148.stdout:2/819: link d7/da/d4c/d9a/cb9 d7/da/d33/ded/d2e/c108 0 2024-03-29T08:07:25.914 INFO:tasks.workunit.client.0.smithi037.stdout:5/619: dread d8/d22/d4f/d70/f3f [4194304,4194304] 0 2024-03-29T08:07:25.918 INFO:tasks.workunit.client.0.smithi037.stdout:5/620: creat d8/d22/d37/d8d/db3/dcb/fd5 x:0 0 0 2024-03-29T08:07:25.918 INFO:tasks.workunit.client.1.smithi148.stdout:5/723: fdatasync d0/db/d1e/d5b/d66/fb7 0 2024-03-29T08:07:25.919 INFO:tasks.workunit.client.0.smithi037.stdout:5/621: readlink d8/d4d/d5c/l8c 0 2024-03-29T08:07:25.924 INFO:tasks.workunit.client.0.smithi037.stdout:5/622: stat d8/d18/d24/c81 0 2024-03-29T08:07:25.925 INFO:tasks.workunit.client.1.smithi148.stdout:3/790: fsync de/d24/d33/d39/d46/d30/f106 0 2024-03-29T08:07:25.929 INFO:tasks.workunit.client.1.smithi148.stdout:8/718: dwrite d17/d74/d86/f8d [0,4194304] 0 2024-03-29T08:07:25.930 INFO:tasks.workunit.client.1.smithi148.stdout:2/820: chown d7/d24/d4b/l81 172822320 1 2024-03-29T08:07:25.930 INFO:tasks.workunit.client.1.smithi148.stdout:4/737: dwrite df/fdb [0,4194304] 0 2024-03-29T08:07:25.932 INFO:tasks.workunit.client.1.smithi148.stdout:6/732: dwrite d6/d9/d1e/d27/f32 [0,4194304] 0 2024-03-29T08:07:25.944 INFO:tasks.workunit.client.1.smithi148.stdout:8/719: write d17/d49/d9a/dbe/f77 [4064068,117561] 0 2024-03-29T08:07:25.949 INFO:tasks.workunit.client.1.smithi148.stdout:4/738: creat df/d2c/d80/daa/dbc/dc9/fe9 x:0 0 0 2024-03-29T08:07:25.951 INFO:tasks.workunit.client.1.smithi148.stdout:3/791: write de/d24/d33/d39/d46/d70/f83 [711010,78219] 0 2024-03-29T08:07:25.951 INFO:tasks.workunit.client.0.smithi037.stdout:5/623: dwrite d8/d22/d4f/d70/d9c/f97 [0,4194304] 0 2024-03-29T08:07:25.953 INFO:tasks.workunit.client.1.smithi148.stdout:6/733: mknod d6/d9/d1e/d36/ceb 0 2024-03-29T08:07:25.956 INFO:tasks.workunit.client.0.smithi037.stdout:5/624: symlink d8/d4d/d8a/ld6 0 2024-03-29T08:07:25.959 INFO:tasks.workunit.client.1.smithi148.stdout:5/724: link d0/db/d1e/d5b/d66/dc7/ccf d0/db/d7b/d16/d2e/ddd/cea 0 2024-03-29T08:07:25.961 INFO:tasks.workunit.client.1.smithi148.stdout:0/729: dwrite db/de/df/f95 [0,4194304] 0 2024-03-29T08:07:25.963 INFO:tasks.workunit.client.1.smithi148.stdout:9/765: dwrite db/d22/d26/d54/d4d/d35/d4b/d58/f8b [0,4194304] 0 2024-03-29T08:07:25.964 INFO:tasks.workunit.client.1.smithi148.stdout:4/739: creat df/fea x:0 0 0 2024-03-29T08:07:25.965 INFO:tasks.workunit.client.1.smithi148.stdout:7/696: dread d8/d14/f55 [0,4194304] 0 2024-03-29T08:07:25.970 INFO:tasks.workunit.client.1.smithi148.stdout:4/740: chown df/d2c/d51/d77/d26/d9b/da7/d7c 12063 1 2024-03-29T08:07:25.971 INFO:tasks.workunit.client.1.smithi148.stdout:7/697: chown d8/fe 61461 1 2024-03-29T08:07:25.971 INFO:tasks.workunit.client.1.smithi148.stdout:5/725: chown d0/db/ld 39 1 2024-03-29T08:07:25.972 INFO:tasks.workunit.client.1.smithi148.stdout:2/821: dwrite d7/f2d [0,4194304] 0 2024-03-29T08:07:25.972 INFO:tasks.workunit.client.1.smithi148.stdout:6/734: getdents d6/d9/d68/dca 0 2024-03-29T08:07:25.975 INFO:tasks.workunit.client.1.smithi148.stdout:9/766: chown db/d10/d2d/d4e/d5f/d6e/d7a/ff7 4 1 2024-03-29T08:07:25.975 INFO:tasks.workunit.client.1.smithi148.stdout:0/730: mkdir db/de/df/d11/d2d/dde 0 2024-03-29T08:07:25.976 INFO:tasks.workunit.client.1.smithi148.stdout:7/698: chown d8/d14/fc7 767 1 2024-03-29T08:07:25.977 INFO:tasks.workunit.client.1.smithi148.stdout:8/720: fdatasync d17/d74/d86/f8d 0 2024-03-29T08:07:25.986 INFO:tasks.workunit.client.1.smithi148.stdout:2/822: creat d7/da/d33/d3c/d31/de1/f109 x:0 0 0 2024-03-29T08:07:25.992 INFO:tasks.workunit.client.0.smithi037.stdout:5/625: dwrite d8/d18/f30 [0,4194304] 0 2024-03-29T08:07:25.993 INFO:tasks.workunit.client.1.smithi148.stdout:6/735: creat d6/d9/dc5/dd3/fec x:0 0 0 2024-03-29T08:07:25.995 INFO:tasks.workunit.client.1.smithi148.stdout:2/823: chown d7/da/d33/ded/d2e/l8b 64753083 1 2024-03-29T08:07:25.996 INFO:tasks.workunit.client.1.smithi148.stdout:9/767: mknod db/d10/d2d/d4e/d5f/c103 0 2024-03-29T08:07:25.996 INFO:tasks.workunit.client.1.smithi148.stdout:0/731: creat db/de/df/d11/d2d/fdf x:0 0 0 2024-03-29T08:07:25.997 INFO:tasks.workunit.client.1.smithi148.stdout:3/792: dwrite de/d24/d33/d39/d46/d30/d37/d62/fff [0,4194304] 0 2024-03-29T08:07:25.997 INFO:tasks.workunit.client.1.smithi148.stdout:1/677: sync 2024-03-29T08:07:26.006 INFO:tasks.workunit.client.1.smithi148.stdout:6/736: readlink d6/d9/d1e/d36/d4d/l63 0 2024-03-29T08:07:26.009 INFO:tasks.workunit.client.1.smithi148.stdout:0/732: fdatasync db/de/df/d11/d42/f6e 0 2024-03-29T08:07:26.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:26.013+0000 7f976aa00700 1 -- 172.21.15.37:0/3808458996 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f97641046d0 msgr2=0x7f9764104b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:07:26.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:26.013+0000 7f976aa00700 1 --2- 172.21.15.37:0/3808458996 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f97641046d0 0x7f9764104b40 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f9750004880 tx=0x7f9750009e30 comp rx=0 tx=0).stop 2024-03-29T08:07:26.014 INFO:tasks.workunit.client.1.smithi148.stdout:4/741: dwrite df/d2c/d51/d77/d26/d34/d8a/d98/fe5 [0,4194304] 0 2024-03-29T08:07:26.014 INFO:tasks.workunit.client.1.smithi148.stdout:3/793: rmdir de/d24/d33/d39/d71 39 2024-03-29T08:07:26.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:26.014+0000 7f976aa00700 1 -- 172.21.15.37:0/3808458996 shutdown_connections 2024-03-29T08:07:26.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:26.014+0000 7f976aa00700 1 --2- 172.21.15.37:0/3808458996 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f97641046d0 0x7f9764104b40 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:07:26.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:26.014+0000 7f976aa00700 1 --2- 172.21.15.37:0/3808458996 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9764103420 0x7f9764103830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:07:26.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:26.014+0000 7f976aa00700 1 -- 172.21.15.37:0/3808458996 >> 172.21.15.37:0/3808458996 conn(0x7f97640725a0 msgr2=0x7f97640749d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:07:26.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:26.014+0000 7f976aa00700 1 -- 172.21.15.37:0/3808458996 shutdown_connections 2024-03-29T08:07:26.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:26.014+0000 7f976aa00700 1 -- 172.21.15.37:0/3808458996 wait complete. 2024-03-29T08:07:26.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:26.017+0000 7f976aa00700 1 Processor -- start 2024-03-29T08:07:26.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:26.017+0000 7f976aa00700 1 -- start start 2024-03-29T08:07:26.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:26.018+0000 7f976aa00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9764103420 0x7f976406e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:07:26.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:26.018+0000 7f976aa00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f97641046d0 0x7f976406e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:07:26.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:26.018+0000 7f976aa00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f976406fe30 con 0x7f9764103420 2024-03-29T08:07:26.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:26.018+0000 7f976aa00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f976406ffa0 con 0x7f97641046d0 2024-03-29T08:07:26.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:26.019+0000 7f9763e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9764103420 0x7f976406e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:07:26.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:26.019+0000 7f9763e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9764103420 0x7f976406e420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46462/0 (socket says 172.21.15.37:46462) 2024-03-29T08:07:26.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:26.019+0000 7f9763e00700 1 -- 172.21.15.37:0/3065520488 learned_addr learned my addr 172.21.15.37:0/3065520488 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:07:26.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:26.019+0000 7f9763400700 1 --2- 172.21.15.37:0/3065520488 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f97641046d0 0x7f976406e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:07:26.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:26.020+0000 7f9763e00700 1 -- 172.21.15.37:0/3065520488 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f97641046d0 msgr2=0x7f976406e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:07:26.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:26.020+0000 7f9763e00700 1 --2- 172.21.15.37:0/3065520488 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f97641046d0 0x7f976406e960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:07:26.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:26.020+0000 7f9763e00700 1 -- 172.21.15.37:0/3065520488 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9750005040 con 0x7f9764103420 2024-03-29T08:07:26.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:26.021+0000 7f9763e00700 1 --2- 172.21.15.37:0/3065520488 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9764103420 0x7f976406e420 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f97580041d0 tx=0x7f975800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:07:26.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:26.022+0000 7f9757e00700 1 -- 172.21.15.37:0/3065520488 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9758007cc0 con 0x7f9764103420 2024-03-29T08:07:26.022 INFO:tasks.workunit.client.1.smithi148.stdout:6/737: creat d6/d9/d1e/d20/db7/fed x:0 0 0 2024-03-29T08:07:26.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:26.022+0000 7f976aa00700 1 -- 172.21.15.37:0/3065520488 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f976406ef00 con 0x7f9764103420 2024-03-29T08:07:26.024 INFO:tasks.workunit.client.1.smithi148.stdout:2/824: mkdir d7/da/d33/d3c/d54/db6/d10a 0 2024-03-29T08:07:26.025 INFO:tasks.workunit.client.1.smithi148.stdout:5/726: dread d0/db/d1e/d35/d42/fde [0,4194304] 0 2024-03-29T08:07:26.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:26.025+0000 7f976aa00700 1 -- 172.21.15.37:0/3065520488 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9764122250 con 0x7f9764103420 2024-03-29T08:07:26.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:26.025+0000 7f9757e00700 1 -- 172.21.15.37:0/3065520488 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f975800f040 con 0x7f9764103420 2024-03-29T08:07:26.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:26.025+0000 7f9757e00700 1 -- 172.21.15.37:0/3065520488 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f975800c8c0 con 0x7f9764103420 2024-03-29T08:07:26.026 INFO:tasks.workunit.client.1.smithi148.stdout:4/742: creat df/d2c/d51/d77/d26/d9b/da7/d7c/feb x:0 0 0 2024-03-29T08:07:26.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:26.027+0000 7f976aa00700 1 -- 172.21.15.37:0/3065520488 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97440052e0 con 0x7f9764103420 2024-03-29T08:07:26.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:26.028+0000 7f9757e00700 1 -- 172.21.15.37:0/3065520488 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 25) v1 ==== 88766+0+0 (secure 0 0 0) 0x7f975800ca20 con 0x7f9764103420 2024-03-29T08:07:26.028 INFO:tasks.workunit.client.0.smithi037.stdout:5/626: truncate d8/d22/d37/d60/faf 251483 0 2024-03-29T08:07:26.029 INFO:tasks.workunit.client.1.smithi148.stdout:0/733: rename db/dc/daa/d47/d7c/fc4 to db/dc/daa/fe0 0 2024-03-29T08:07:26.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:26.029+0000 7f9757e00700 1 --2- 172.21.15.37:0/3065520488 >> [v2:172.21.15.148:6828/1134897184,v1:172.21.15.148:6829/1134897184] conn(0x7f974c06af00 0x7f974c06d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:07:26.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:26.030+0000 7f9757e00700 1 -- 172.21.15.37:0/3065520488 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f9758007e20 con 0x7f9764103420 2024-03-29T08:07:26.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:26.031+0000 7f9763400700 1 --2- 172.21.15.37:0/3065520488 >> [v2:172.21.15.148:6828/1134897184,v1:172.21.15.148:6829/1134897184] conn(0x7f974c06af00 0x7f974c06d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:07:26.032 INFO:tasks.workunit.client.1.smithi148.stdout:8/721: write d17/d49/d9a/dbe/f73 [875911,46163] 0 2024-03-29T08:07:26.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:26.032+0000 7f9763400700 1 --2- 172.21.15.37:0/3065520488 >> [v2:172.21.15.148:6828/1134897184,v1:172.21.15.148:6829/1134897184] conn(0x7f974c06af00 0x7f974c06d3b0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f9750005010 tx=0x7f9750009e30 comp rx=0 tx=0).ready entity=mgr.14602 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:07:26.040 INFO:tasks.workunit.client.1.smithi148.stdout:5/727: fsync d0/db/d1e/d36/d41/f63 0 2024-03-29T08:07:26.041 INFO:tasks.workunit.client.1.smithi148.stdout:8/722: chown d17/d18/f2a 14466 1 2024-03-29T08:07:26.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:26.043+0000 7f9757e00700 1 -- 172.21.15.37:0/3065520488 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181579 (secure 0 0 0) 0x7f9758053f00 con 0x7f9764103420 2024-03-29T08:07:26.047 INFO:tasks.workunit.client.1.smithi148.stdout:1/678: dwrite d3/d18/faf [0,4194304] 0 2024-03-29T08:07:26.048 INFO:tasks.workunit.client.0.smithi037.stdout:5/627: dwrite d8/d22/d37/d60/f76 [0,4194304] 0 2024-03-29T08:07:26.055 INFO:tasks.workunit.client.1.smithi148.stdout:3/794: truncate de/d24/d33/d39/d46/d30/d37/d62/d63/f92 1560756 0 2024-03-29T08:07:26.055 INFO:tasks.workunit.client.1.smithi148.stdout:9/768: dwrite db/d10/d2d/d4e/d5f/d6e/d7a/f7e [0,4194304] 0 2024-03-29T08:07:26.056 INFO:tasks.workunit.client.0.smithi037.stdout:5/628: creat d8/d22/d4f/d70/d79/fd7 x:0 0 0 2024-03-29T08:07:26.058 INFO:tasks.workunit.client.1.smithi148.stdout:1/679: mkdir d3/d18/d71/ddc 0 2024-03-29T08:07:26.060 INFO:tasks.workunit.client.1.smithi148.stdout:7/699: dwrite d8/d38/d3c/da9/d37/dc8/fcb [0,4194304] 0 2024-03-29T08:07:26.061 INFO:tasks.workunit.client.1.smithi148.stdout:0/734: rename db/de/d8e/cb3 to db/de/df/d2b/ce1 0 2024-03-29T08:07:26.063 INFO:tasks.workunit.client.1.smithi148.stdout:5/728: chown d0/l8 688 1 2024-03-29T08:07:26.065 INFO:tasks.workunit.client.1.smithi148.stdout:1/680: chown d3/de/d1d/fd5 139 1 2024-03-29T08:07:26.066 INFO:tasks.workunit.client.1.smithi148.stdout:9/769: chown db/d22/d26/d54/f36 0 1 2024-03-29T08:07:26.068 INFO:tasks.workunit.client.1.smithi148.stdout:1/681: readlink d3/d18/l2e 0 2024-03-29T08:07:26.069 INFO:tasks.workunit.client.0.smithi037.stdout:5/629: creat d8/d18/d24/fd8 x:0 0 0 2024-03-29T08:07:26.071 INFO:tasks.workunit.client.1.smithi148.stdout:8/723: symlink d17/d18/ddf/lf4 0 2024-03-29T08:07:26.071 INFO:tasks.workunit.client.1.smithi148.stdout:4/743: dwrite df/d2c/f86 [0,4194304] 0 2024-03-29T08:07:26.075 INFO:tasks.workunit.client.1.smithi148.stdout:3/795: write de/d24/d33/d39/d46/d70/def/ff7 [629122,20525] 0 2024-03-29T08:07:26.078 INFO:tasks.workunit.client.1.smithi148.stdout:0/735: unlink db/dc/daa/f49 0 2024-03-29T08:07:26.078 INFO:tasks.workunit.client.1.smithi148.stdout:7/700: mkdir d8/df3 0 2024-03-29T08:07:26.079 INFO:tasks.workunit.client.0.smithi037.stdout:0/393: sync 2024-03-29T08:07:26.080 INFO:tasks.workunit.client.0.smithi037.stdout:0/394: stat d9/d74 0 2024-03-29T08:07:26.083 INFO:tasks.workunit.client.1.smithi148.stdout:3/796: unlink de/d24/d33/c103 0 2024-03-29T08:07:26.083 INFO:tasks.workunit.client.1.smithi148.stdout:6/738: dwrite d6/f70 [0,4194304] 0 2024-03-29T08:07:26.085 INFO:tasks.workunit.client.1.smithi148.stdout:3/797: chown de/d3b/d49/l52 567 1 2024-03-29T08:07:26.092 INFO:tasks.workunit.client.1.smithi148.stdout:7/701: mkdir d8/df4 0 2024-03-29T08:07:26.094 INFO:tasks.workunit.client.1.smithi148.stdout:9/770: creat db/d22/d26/d54/d4d/dbb/def/f104 x:0 0 0 2024-03-29T08:07:26.103 INFO:tasks.workunit.client.1.smithi148.stdout:4/744: dread df/fdb [0,4194304] 0 2024-03-29T08:07:26.105 INFO:tasks.workunit.client.1.smithi148.stdout:7/702: creat d8/d38/d3c/da9/ff5 x:0 0 0 2024-03-29T08:07:26.106 INFO:tasks.workunit.client.1.smithi148.stdout:3/798: creat de/d24/d33/d39/d46/d30/d37/d62/dde/f107 x:0 0 0 2024-03-29T08:07:26.109 INFO:tasks.workunit.client.1.smithi148.stdout:4/745: rmdir df/d89/ddd 39 2024-03-29T08:07:26.109 INFO:tasks.workunit.client.1.smithi148.stdout:3/799: fdatasync de/d24/d33/f94 0 2024-03-29T08:07:26.112 INFO:tasks.workunit.client.1.smithi148.stdout:2/825: dread d7/da/d33/ded/d2e/f50 [4194304,4194304] 0 2024-03-29T08:07:26.113 INFO:tasks.workunit.client.1.smithi148.stdout:7/703: write d8/d14/d45/d4d/f6b [3774353,15999] 0 2024-03-29T08:07:26.114 INFO:tasks.workunit.client.1.smithi148.stdout:3/800: chown de/d24/d33/d39/c50 1 1 2024-03-29T08:07:26.115 INFO:tasks.workunit.client.1.smithi148.stdout:4/746: mknod df/d2c/d80/cec 0 2024-03-29T08:07:26.121 INFO:tasks.workunit.client.1.smithi148.stdout:9/771: dwrite db/d10/d2d/d4e/dd4/f1b [0,4194304] 0 2024-03-29T08:07:26.121 INFO:tasks.workunit.client.1.smithi148.stdout:2/826: truncate d7/da/f3f 4687599 0 2024-03-29T08:07:26.128 INFO:tasks.workunit.client.0.smithi037.stdout:0/395: truncate d9/da/d1b/d22/d2c/f42 2742438 0 2024-03-29T08:07:26.128 INFO:tasks.workunit.client.1.smithi148.stdout:8/724: write d17/f68 [1033856,58875] 0 2024-03-29T08:07:26.130 INFO:tasks.workunit.client.1.smithi148.stdout:3/801: mknod de/d24/d33/d39/d46/d30/d37/d62/dde/c108 0 2024-03-29T08:07:26.135 INFO:tasks.workunit.client.1.smithi148.stdout:7/704: symlink d8/d60/lf6 0 2024-03-29T08:07:26.135 INFO:tasks.workunit.client.0.smithi037.stdout:0/396: mknod d9/da/d1b/d22/c7c 0 2024-03-29T08:07:26.136 INFO:tasks.workunit.client.1.smithi148.stdout:2/827: write d7/da/d33/ded/f57 [2832366,50953] 0 2024-03-29T08:07:26.138 INFO:tasks.workunit.client.0.smithi037.stdout:0/397: chown d9/da/d1b/d33/f6d 4040 1 2024-03-29T08:07:26.138 INFO:tasks.workunit.client.0.smithi037.stdout:5/630: dread d8/d22/d4f/d70/d9c/f84 [0,4194304] 0 2024-03-29T08:07:26.138 INFO:tasks.workunit.client.1.smithi148.stdout:5/729: truncate d0/db/d1e/d36/d41/f7f 2375749 0 2024-03-29T08:07:26.139 INFO:tasks.workunit.client.1.smithi148.stdout:3/802: rmdir de/d24/d33/d39/da3 39 2024-03-29T08:07:26.142 INFO:tasks.workunit.client.0.smithi037.stdout:5/631: creat d8/d22/d37/fd9 x:0 0 0 2024-03-29T08:07:26.144 INFO:tasks.workunit.client.1.smithi148.stdout:6/739: dread d6/d9/d1e/d27/f6e [0,4194304] 0 2024-03-29T08:07:26.145 INFO:tasks.workunit.client.1.smithi148.stdout:7/705: mkdir d8/d14/d45/d4d/df7 0 2024-03-29T08:07:26.146 INFO:tasks.workunit.client.1.smithi148.stdout:0/736: dread db/f23 [0,4194304] 0 2024-03-29T08:07:26.146 INFO:tasks.workunit.client.1.smithi148.stdout:5/730: stat d0/db/d1e/d35/d42/l9b 0 2024-03-29T08:07:26.148 INFO:tasks.workunit.client.0.smithi037.stdout:0/398: read d9/da/d1b/d22/d2c/f2d [154566,43778] 0 2024-03-29T08:07:26.150 INFO:tasks.workunit.client.0.smithi037.stdout:0/399: write d9/ff [1830924,8454] 0 2024-03-29T08:07:26.151 INFO:tasks.workunit.client.1.smithi148.stdout:4/747: dwrite df/d2c/d51/d77/d26/d34/f5c [0,4194304] 0 2024-03-29T08:07:26.151 INFO:tasks.workunit.client.1.smithi148.stdout:6/740: dread - d6/d9/d1e/d20/d30/fc6 zero size 2024-03-29T08:07:26.152 INFO:tasks.workunit.client.1.smithi148.stdout:7/706: rmdir d8/d38/d3c/da5/de3 39 2024-03-29T08:07:26.153 INFO:tasks.workunit.client.1.smithi148.stdout:3/803: mkdir de/d24/d33/d39/d46/d109 0 2024-03-29T08:07:26.154 INFO:tasks.workunit.client.1.smithi148.stdout:9/772: write db/d22/d26/d54/d4d/d35/d4b/dc3/fb5 [300692,118296] 0 2024-03-29T08:07:26.158 INFO:tasks.workunit.client.1.smithi148.stdout:9/773: fsync db/d22/f51 0 2024-03-29T08:07:26.159 INFO:tasks.workunit.client.0.smithi037.stdout:5/632: rename d8/d22/d37/d8d/l95 to d8/d22/d37/d60/lda 0 2024-03-29T08:07:26.160 INFO:tasks.workunit.client.1.smithi148.stdout:5/731: readlink d0/db/d7b/d16/l54 0 2024-03-29T08:07:26.161 INFO:tasks.workunit.client.0.smithi037.stdout:0/400: mkdir d9/d74/d7d 0 2024-03-29T08:07:26.163 INFO:tasks.workunit.client.1.smithi148.stdout:9/774: dread - db/d10/d2d/d4e/dd4/fe6 zero size 2024-03-29T08:07:26.163 INFO:tasks.workunit.client.1.smithi148.stdout:2/828: write d7/da/d33/ded/d2e/d37/f43 [1223730,62202] 0 2024-03-29T08:07:26.163 INFO:tasks.workunit.client.0.smithi037.stdout:0/401: readlink d9/da/d1b/d22/l60 0 2024-03-29T08:07:26.165 INFO:tasks.workunit.client.1.smithi148.stdout:5/732: chown d0/db/d1e/d5b/d66/d73/l78 3009289 1 2024-03-29T08:07:26.165 INFO:tasks.workunit.client.1.smithi148.stdout:4/748: mknod df/d2c/d51/d77/dbe/ced 0 2024-03-29T08:07:26.166 INFO:tasks.workunit.client.0.smithi037.stdout:5/633: fsync d8/d18/f71 0 2024-03-29T08:07:26.167 INFO:tasks.workunit.client.1.smithi148.stdout:1/682: sync 2024-03-29T08:07:26.169 INFO:tasks.workunit.client.1.smithi148.stdout:0/737: creat db/de/df/d14/fe2 x:0 0 0 2024-03-29T08:07:26.171 INFO:tasks.workunit.client.1.smithi148.stdout:3/804: truncate de/d24/d33/d39/d46/d30/d37/d62/d63/fcb 802569 0 2024-03-29T08:07:26.174 INFO:tasks.workunit.client.1.smithi148.stdout:1/683: chown d3/de/f57 71 1 2024-03-29T08:07:26.174 INFO:tasks.workunit.client.1.smithi148.stdout:0/738: unlink db/de/la1 0 2024-03-29T08:07:26.176 INFO:tasks.workunit.client.1.smithi148.stdout:2/829: unlink d7/d24/fa8 0 2024-03-29T08:07:26.177 INFO:tasks.workunit.client.1.smithi148.stdout:4/749: write df/d2c/d51/d77/d26/f84 [3095107,84040] 0 2024-03-29T08:07:26.180 INFO:tasks.workunit.client.1.smithi148.stdout:1/684: stat d3/d55/d7e/c9a 0 2024-03-29T08:07:26.181 INFO:tasks.workunit.client.1.smithi148.stdout:8/725: dread d17/d4e/f5d [0,4194304] 0 2024-03-29T08:07:26.182 INFO:tasks.workunit.client.0.smithi037.stdout:0/402: fdatasync d9/da/d1b/d22/d2c/f42 0 2024-03-29T08:07:26.182 INFO:tasks.workunit.client.1.smithi148.stdout:6/741: truncate d6/d9/d1e/f8a 1143369 0 2024-03-29T08:07:26.183 INFO:tasks.workunit.client.0.smithi037.stdout:5/634: write d8/fa0 [124422,42452] 0 2024-03-29T08:07:26.186 INFO:tasks.workunit.client.0.smithi037.stdout:5/635: fsync d8/d18/f40 0 2024-03-29T08:07:26.187 INFO:tasks.workunit.client.1.smithi148.stdout:4/750: write df/d2c/d51/d77/d26/f84 [768093,33871] 0 2024-03-29T08:07:26.188 INFO:tasks.workunit.client.1.smithi148.stdout:8/726: unlink d17/d18/f36 0 2024-03-29T08:07:26.188 INFO:tasks.workunit.client.0.smithi037.stdout:0/403: mkdir d9/d74/d7d/d7e 0 2024-03-29T08:07:26.192 INFO:tasks.workunit.client.0.smithi037.stdout:5/636: chown d8/d22/f27 1126599 1 2024-03-29T08:07:26.193 INFO:tasks.workunit.client.0.smithi037.stdout:0/404: symlink d9/da/dd/d20/d38/l7f 0 2024-03-29T08:07:26.194 INFO:tasks.workunit.client.1.smithi148.stdout:7/707: dwrite d8/d14/f2b [0,4194304] 0 2024-03-29T08:07:26.197 INFO:tasks.workunit.client.0.smithi037.stdout:5/637: rmdir d8/d22/d4f/d70/d79 39 2024-03-29T08:07:26.199 INFO:tasks.workunit.client.1.smithi148.stdout:5/733: unlink d0/db/d1e/d35/d42/d67/d76/d81/d98/dd5/d9c/ld4 0 2024-03-29T08:07:26.203 INFO:tasks.workunit.client.1.smithi148.stdout:1/685: truncate d3/de/d6e/f79 192946 0 2024-03-29T08:07:26.204 INFO:tasks.workunit.client.1.smithi148.stdout:4/751: truncate fd 2843514 0 2024-03-29T08:07:26.206 INFO:tasks.workunit.client.1.smithi148.stdout:2/830: mknod d7/da/d33/d3c/d54/c10b 0 2024-03-29T08:07:26.207 INFO:tasks.workunit.client.1.smithi148.stdout:0/739: dwrite db/dc/f1e [0,4194304] 0 2024-03-29T08:07:26.211 INFO:tasks.workunit.client.1.smithi148.stdout:9/775: getdents db/d10/d2d/d4e/d5f/d6e/db0 0 2024-03-29T08:07:26.212 INFO:tasks.workunit.client.1.smithi148.stdout:1/686: write d3/de/d21/f80 [3458981,9275] 0 2024-03-29T08:07:26.215 INFO:tasks.workunit.client.1.smithi148.stdout:0/740: symlink db/de/d8e/da9/le3 0 2024-03-29T08:07:26.215 INFO:tasks.workunit.client.1.smithi148.stdout:5/734: mknod d0/db/d7b/ceb 0 2024-03-29T08:07:26.216 INFO:tasks.workunit.client.0.smithi037.stdout:0/405: dwrite d9/da/dd/d13/f5e [0,4194304] 0 2024-03-29T08:07:26.217 INFO:tasks.workunit.client.1.smithi148.stdout:4/752: write df/d2c/d51/d77/d17/d73/f7a [1463342,6887] 0 2024-03-29T08:07:26.219 INFO:tasks.workunit.client.0.smithi037.stdout:5/638: dwrite d8/d4d/d5c/fc3 [0,4194304] 0 2024-03-29T08:07:26.224 INFO:tasks.workunit.client.0.smithi037.stdout:5/639: chown d8/d22/d37/d8d/db3/dcb/fd5 17 1 2024-03-29T08:07:26.224 INFO:tasks.workunit.client.1.smithi148.stdout:8/727: dwrite d17/d4e/f5d [4194304,4194304] 0 2024-03-29T08:07:26.225 INFO:tasks.workunit.client.0.smithi037.stdout:0/406: mknod d9/da/dd/d20/d38/c80 0 2024-03-29T08:07:26.226 INFO:tasks.workunit.client.1.smithi148.stdout:0/741: write db/de/df/d11/d2d/fdf [691926,122985] 0 2024-03-29T08:07:26.230 INFO:tasks.workunit.client.0.smithi037.stdout:5/640: read d8/d34/fba [3428838,35725] 0 2024-03-29T08:07:26.231 INFO:tasks.workunit.client.1.smithi148.stdout:4/753: creat df/da0/fee x:0 0 0 2024-03-29T08:07:26.232 INFO:tasks.workunit.client.0.smithi037.stdout:5/641: stat d8/d34/d4b/c57 0 2024-03-29T08:07:26.242 INFO:tasks.workunit.client.0.smithi037.stdout:5/642: creat d8/d34/d4b/fdb x:0 0 0 2024-03-29T08:07:26.247 INFO:tasks.workunit.client.0.smithi037.stdout:5/643: write d8/d18/d24/f74 [431932,84437] 0 2024-03-29T08:07:26.249 INFO:tasks.workunit.client.1.smithi148.stdout:0/742: write db/de/df/f95 [2677598,24987] 0 2024-03-29T08:07:26.257 INFO:tasks.workunit.client.1.smithi148.stdout:6/742: dread d6/d9/d1e/f26 [4194304,4194304] 0 2024-03-29T08:07:26.258 INFO:tasks.workunit.client.0.smithi037.stdout:5/644: link d8/c56 d8/d22/d4f/d70/d9c/cdc 0 2024-03-29T08:07:26.260 INFO:tasks.workunit.client.0.smithi037.stdout:0/407: dwrite d9/da/f3c [0,4194304] 0 2024-03-29T08:07:26.262 INFO:tasks.workunit.client.1.smithi148.stdout:7/708: dwrite d8/d14/f1c [0,4194304] 0 2024-03-29T08:07:26.263 INFO:tasks.workunit.client.0.smithi037.stdout:5/645: dread - d8/d4d/d5c/dbf/fc2 zero size 2024-03-29T08:07:26.264 INFO:tasks.workunit.client.1.smithi148.stdout:4/754: mkdir df/d2c/d51/d77/def 0 2024-03-29T08:07:26.271 INFO:tasks.workunit.client.1.smithi148.stdout:1/687: dwrite d3/d18/faf [0,4194304] 0 2024-03-29T08:07:26.276 INFO:tasks.workunit.client.1.smithi148.stdout:9/776: dwrite db/d10/d2d/d4e/d5f/fe8 [0,4194304] 0 2024-03-29T08:07:26.276 INFO:tasks.workunit.client.0.smithi037.stdout:0/408: link d9/da/dd/d59/f18 d9/da/d1b/d33/f81 0 2024-03-29T08:07:26.276 INFO:tasks.workunit.client.0.smithi037.stdout:5/646: link d8/cb2 d8/d22/db6/cdd 0 2024-03-29T08:07:26.278 INFO:tasks.workunit.client.1.smithi148.stdout:3/805: dread de/d24/d33/d39/d7f/fac [0,4194304] 0 2024-03-29T08:07:26.279 INFO:tasks.workunit.client.1.smithi148.stdout:8/728: dwrite d17/f68 [0,4194304] 0 2024-03-29T08:07:26.282 INFO:tasks.workunit.client.1.smithi148.stdout:0/743: readlink db/dc/daa/d47/d4c/d5a/l60 0 2024-03-29T08:07:26.289 INFO:tasks.workunit.client.1.smithi148.stdout:2/831: dwrite d7/d8c/d91/fff [0,4194304] 0 2024-03-29T08:07:26.289 INFO:tasks.workunit.client.1.smithi148.stdout:6/743: stat d6/d9/d1e/d36/d4d/d9d/d28/d98/dc2/fc8 0 2024-03-29T08:07:26.290 INFO:tasks.workunit.client.1.smithi148.stdout:5/735: dwrite d0/db/d1e/d35/d42/d67/d76/d81/d98/fb9 [0,4194304] 0 2024-03-29T08:07:26.290 INFO:tasks.workunit.client.0.smithi037.stdout:5/647: mkdir d8/d22/dde 0 2024-03-29T08:07:26.292 INFO:tasks.workunit.client.1.smithi148.stdout:3/806: readlink de/d24/d33/d39/d7f/lb7 0 2024-03-29T08:07:26.295 INFO:tasks.workunit.client.1.smithi148.stdout:8/729: write d17/d18/d26/d63/d66/d5f/f8e [2613436,82765] 0 2024-03-29T08:07:26.295 INFO:tasks.workunit.client.1.smithi148.stdout:2/832: fdatasync d7/d24/f5d 0 2024-03-29T08:07:26.296 INFO:tasks.workunit.client.0.smithi037.stdout:0/409: getdents d9 0 2024-03-29T08:07:26.296 INFO:tasks.workunit.client.0.smithi037.stdout:4/538: sync 2024-03-29T08:07:26.297 INFO:tasks.workunit.client.0.smithi037.stdout:3/501: sync 2024-03-29T08:07:26.297 INFO:tasks.workunit.client.0.smithi037.stdout:8/521: sync 2024-03-29T08:07:26.297 INFO:tasks.workunit.client.0.smithi037.stdout:1/590: sync 2024-03-29T08:07:26.298 INFO:tasks.workunit.client.0.smithi037.stdout:2/380: sync 2024-03-29T08:07:26.298 INFO:tasks.workunit.client.0.smithi037.stdout:7/545: sync 2024-03-29T08:07:26.298 INFO:tasks.workunit.client.0.smithi037.stdout:9/727: sync 2024-03-29T08:07:26.298 INFO:tasks.workunit.client.0.smithi037.stdout:6/456: sync 2024-03-29T08:07:26.299 INFO:tasks.workunit.client.1.smithi148.stdout:4/755: symlink df/d2c/d80/da6/lf0 0 2024-03-29T08:07:26.300 INFO:tasks.workunit.client.0.smithi037.stdout:5/648: mkdir d8/d4d/ddf 0 2024-03-29T08:07:26.303 INFO:tasks.workunit.client.1.smithi148.stdout:8/730: truncate d17/d18/d26/d4f/d94/fec 848219 0 2024-03-29T08:07:26.305 INFO:tasks.workunit.client.1.smithi148.stdout:7/709: creat d8/d38/d3c/da9/d37/dc8/ff8 x:0 0 0 2024-03-29T08:07:26.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:07:26 smithi148 ceph-mon[44992]: from='client.14666 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:07:26.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:07:26 smithi148 ceph-mon[44992]: from='client.24493 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:07:26.308 INFO:tasks.workunit.client.1.smithi148.stdout:7/710: readlink d8/d60/lf6 0 2024-03-29T08:07:26.310 INFO:tasks.workunit.client.1.smithi148.stdout:1/688: dwrite d3/de/d25/d35/faa [0,4194304] 0 2024-03-29T08:07:26.310 INFO:tasks.workunit.client.1.smithi148.stdout:6/744: rmdir d6/d9/d1e/d20/db7 39 2024-03-29T08:07:26.312 INFO:tasks.workunit.client.0.smithi037.stdout:1/591: write d2/d6/dc/f15 [2215378,82393] 0 2024-03-29T08:07:26.312 INFO:tasks.workunit.client.0.smithi037.stdout:2/381: read d0/d20/f75 [135054,38362] 0 2024-03-29T08:07:26.313 INFO:tasks.workunit.client.0.smithi037.stdout:3/502: creat dd/d5e/faa x:0 0 0 2024-03-29T08:07:26.317 INFO:tasks.workunit.client.1.smithi148.stdout:0/744: fdatasync db/de/df/d14/f3c 0 2024-03-29T08:07:26.323 INFO:tasks.workunit.client.0.smithi037.stdout:4/539: mkdir d3/d4/d15/d1d/dad 0 2024-03-29T08:07:26.326 INFO:tasks.workunit.client.0.smithi037.stdout:4/540: write d3/d4/d15/f29 [1716460,47078] 0 2024-03-29T08:07:26.331 INFO:tasks.workunit.client.1.smithi148.stdout:8/731: dwrite d17/d49/d9a/dbe/d6f/f5b [0,4194304] 0 2024-03-29T08:07:26.333 INFO:tasks.workunit.client.1.smithi148.stdout:5/736: symlink d0/db/d1e/d35/d42/d67/d76/lec 0 2024-03-29T08:07:26.337 INFO:tasks.workunit.client.1.smithi148.stdout:4/756: link df/d2c/f2e df/d2c/d51/d77/d26/d34/d8a/ff1 0 2024-03-29T08:07:26.340 INFO:tasks.workunit.client.1.smithi148.stdout:7/711: creat d8/d29/d44/d50/ff9 x:0 0 0 2024-03-29T08:07:26.341 INFO:tasks.workunit.client.1.smithi148.stdout:9/777: dread db/d22/d26/f38 [0,4194304] 0 2024-03-29T08:07:26.341 INFO:tasks.workunit.client.1.smithi148.stdout:1/689: getdents d3/de/d21 0 2024-03-29T08:07:26.343 INFO:tasks.workunit.client.1.smithi148.stdout:4/757: symlink df/d2c/lf2 0 2024-03-29T08:07:26.347 INFO:tasks.workunit.client.0.smithi037.stdout:9/728: chown d1/d4/d19/d1e/d25/ff8 7058 1 2024-03-29T08:07:26.347 INFO:tasks.workunit.client.1.smithi148.stdout:2/833: dwrite d7/da/d33/ded/d2e/f50 [0,4194304] 0 2024-03-29T08:07:26.351 INFO:tasks.workunit.client.1.smithi148.stdout:5/737: creat d0/db/d1e/d35/d42/d67/d9a/dd7/fed x:0 0 0 2024-03-29T08:07:26.352 INFO:tasks.workunit.client.1.smithi148.stdout:6/745: dread d6/d9/f23 [0,4194304] 0 2024-03-29T08:07:26.356 INFO:tasks.workunit.client.1.smithi148.stdout:6/746: creat d6/d9/d1e/fee x:0 0 0 2024-03-29T08:07:26.356 INFO:tasks.workunit.client.0.smithi037.stdout:9/729: read d1/d1f/d3a/fc0 [738295,72609] 0 2024-03-29T08:07:26.358 INFO:tasks.workunit.client.1.smithi148.stdout:8/732: dread d17/d18/fa9 [0,4194304] 0 2024-03-29T08:07:26.359 INFO:tasks.workunit.client.0.smithi037.stdout:8/522: creat d5/d10/d2d/d34/d50/fad x:0 0 0 2024-03-29T08:07:26.367 INFO:tasks.workunit.client.0.smithi037.stdout:1/592: truncate d2/d6/dc/d18/d1d/dad/f68 4975341 0 2024-03-29T08:07:26.379 INFO:tasks.workunit.client.1.smithi148.stdout:5/738: link d0/db/d1e/d5b/d66/l8c d0/db/d1e/d35/d42/lee 0 2024-03-29T08:07:26.380 INFO:tasks.workunit.client.1.smithi148.stdout:8/733: symlink d17/d74/d86/lf5 0 2024-03-29T08:07:26.381 INFO:tasks.workunit.client.1.smithi148.stdout:6/747: rename d6/d9/d1e/d36/d4d/f57 to d6/d9/d1e/d36/d4d/d9d/d28/db3/dd2/fef 0 2024-03-29T08:07:26.384 INFO:tasks.workunit.client.1.smithi148.stdout:9/778: dwrite db/d22/da3/fcd [0,4194304] 0 2024-03-29T08:07:26.386 INFO:tasks.workunit.client.1.smithi148.stdout:6/748: rmdir d6/d9/dc5 39 2024-03-29T08:07:26.386 INFO:tasks.workunit.client.1.smithi148.stdout:8/734: chown d17/d18/dbd/ld4 1649276 1 2024-03-29T08:07:26.392 INFO:tasks.workunit.client.1.smithi148.stdout:1/690: dwrite d3/de/d1d/f33 [0,4194304] 0 2024-03-29T08:07:26.395 INFO:tasks.workunit.client.1.smithi148.stdout:6/749: stat d6/d9/d1e/d20/d6b/l80 0 2024-03-29T08:07:26.396 INFO:tasks.workunit.client.1.smithi148.stdout:5/739: mkdir d0/db/d7b/d16/d2e/ddd/de9/def 0 2024-03-29T08:07:26.405 INFO:tasks.workunit.client.1.smithi148.stdout:0/745: write db/de/df/d11/d42/f52 [4531954,101567] 0 2024-03-29T08:07:26.405 INFO:tasks.workunit.client.1.smithi148.stdout:5/740: read d0/d46/d9e/fa6 [2550869,118118] 0 2024-03-29T08:07:26.405 INFO:tasks.workunit.client.1.smithi148.stdout:6/750: chown d6/d9/d1e/d36/d4d/d9d/d28/dda/db6 15 1 2024-03-29T08:07:26.405 INFO:tasks.workunit.client.1.smithi148.stdout:4/758: dwrite df/d60/f85 [0,4194304] 0 2024-03-29T08:07:26.407 INFO:tasks.workunit.client.1.smithi148.stdout:3/807: dread de/d24/d33/d39/d46/d30/d37/f6f [0,4194304] 0 2024-03-29T08:07:26.410 INFO:tasks.workunit.client.1.smithi148.stdout:5/741: fsync d0/d46/d9e/fc1 0 2024-03-29T08:07:26.416 INFO:tasks.workunit.client.1.smithi148.stdout:8/735: mkdir d17/d49/d9a/dbe/d6f/d64/df1/df6 0 2024-03-29T08:07:26.417 INFO:tasks.workunit.client.1.smithi148.stdout:2/834: dread d7/da/d33/ded/d2e/d37/f3d [0,4194304] 0 2024-03-29T08:07:26.418 INFO:tasks.workunit.client.1.smithi148.stdout:0/746: write db/dc/daa/d47/d4c/d5a/f98 [1741386,74467] 0 2024-03-29T08:07:26.420 INFO:tasks.workunit.client.0.smithi037.stdout:2/382: dwrite d0/d20/d22/f50 [0,4194304] 0 2024-03-29T08:07:26.421 INFO:tasks.workunit.client.1.smithi148.stdout:6/751: rename d6/d9/d1e/d20/d30/le9 to d6/d9/d1e/lf0 0 2024-03-29T08:07:26.422 INFO:tasks.workunit.client.0.smithi037.stdout:3/503: dwrite dd/d46/f37 [0,4194304] 0 2024-03-29T08:07:26.423 INFO:tasks.workunit.client.1.smithi148.stdout:3/808: truncate de/d24/d33/fd4 607944 0 2024-03-29T08:07:26.425 INFO:tasks.workunit.client.1.smithi148.stdout:4/759: stat df/d2c/d51/d77/d26/d34/d8a/fc4 0 2024-03-29T08:07:26.426 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:07:26 smithi037 ceph-mon[35162]: from='client.14666 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:07:26.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:07:26 smithi037 ceph-mon[35162]: from='client.24493 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:07:26.432 INFO:tasks.workunit.client.1.smithi148.stdout:9/779: dwrite db/f84 [0,4194304] 0 2024-03-29T08:07:26.432 INFO:tasks.workunit.client.1.smithi148.stdout:2/835: mkdir d7/d8c/da4/d10c 0 2024-03-29T08:07:26.440 INFO:tasks.workunit.client.1.smithi148.stdout:4/760: write df/d2c/d51/d77/d17/f8f [279028,57353] 0 2024-03-29T08:07:26.441 INFO:tasks.workunit.client.1.smithi148.stdout:3/809: symlink de/d24/d33/d39/d46/d109/l10a 0 2024-03-29T08:07:26.441 INFO:tasks.workunit.client.1.smithi148.stdout:2/836: readlink d7/d24/d4b/l4f 0 2024-03-29T08:07:26.441 INFO:tasks.workunit.client.1.smithi148.stdout:7/712: dread d8/f18 [4194304,4194304] 0 2024-03-29T08:07:26.441 INFO:tasks.workunit.client.1.smithi148.stdout:5/742: write d0/db/d7b/d16/f3d [344548,39598] 0 2024-03-29T08:07:26.449 INFO:tasks.workunit.client.1.smithi148.stdout:3/810: write de/d24/d33/d39/d46/d70/f83 [326421,10609] 0 2024-03-29T08:07:26.449 INFO:tasks.workunit.client.0.smithi037.stdout:9/730: mkdir d1/d4/d19/d1e/d71/d89/da6/dfb 0 2024-03-29T08:07:26.452 INFO:tasks.workunit.client.0.smithi037.stdout:8/523: rmdir d5/d10/d19/d8d 39 2024-03-29T08:07:26.453 INFO:tasks.workunit.client.1.smithi148.stdout:3/811: chown de/d3b/fb8 248869 1 2024-03-29T08:07:26.453 INFO:tasks.workunit.client.1.smithi148.stdout:2/837: symlink d7/da/d33/d45/l10d 0 2024-03-29T08:07:26.453 INFO:tasks.workunit.client.1.smithi148.stdout:6/752: dread d6/d9/d1e/d27/f90 [0,4194304] 0 2024-03-29T08:07:26.456 INFO:tasks.workunit.client.1.smithi148.stdout:5/743: truncate d0/db/d1e/d35/d42/d67/d9a/f6d 4635084 0 2024-03-29T08:07:26.456 INFO:tasks.workunit.client.0.smithi037.stdout:8/524: dread d5/d10/d45/d4b/fa4 [0,4194304] 0 2024-03-29T08:07:26.457 INFO:tasks.workunit.client.0.smithi037.stdout:5/649: rename d8/d4d/d72/c9d to d8/d22/d37/d60/d88/dce/ce0 0 2024-03-29T08:07:26.457 INFO:tasks.workunit.client.1.smithi148.stdout:6/753: readlink d6/l40 0 2024-03-29T08:07:26.460 INFO:tasks.workunit.client.1.smithi148.stdout:3/812: chown de/f61 14774205 1 2024-03-29T08:07:26.460 INFO:tasks.workunit.client.0.smithi037.stdout:6/457: creat d0/d19/f99 x:0 0 0 2024-03-29T08:07:26.461 INFO:tasks.workunit.client.1.smithi148.stdout:9/780: link db/d22/d26/d54/d4d/d35/c93 db/d22/d26/d54/d52/d9c/c105 0 2024-03-29T08:07:26.463 INFO:tasks.workunit.client.1.smithi148.stdout:1/691: dread d3/de/d25/d35/d42/f1a [0,4194304] 0 2024-03-29T08:07:26.464 INFO:tasks.workunit.client.1.smithi148.stdout:6/754: dread - d6/d9/d1e/d36/d4d/d9d/d28/d98/dcc/fd6 zero size 2024-03-29T08:07:26.466 INFO:tasks.workunit.client.1.smithi148.stdout:9/781: stat db/d10/d2d/d4e/d5f/d6e/db0/fc4 0 2024-03-29T08:07:26.467 INFO:tasks.workunit.client.1.smithi148.stdout:5/744: mknod d0/db/d1e/d35/d42/d67/d76/d81/cf0 0 2024-03-29T08:07:26.470 INFO:tasks.workunit.client.1.smithi148.stdout:2/838: rename d7/d24/d4b/d66/ca6 to d7/da/d4c/c10e 0 2024-03-29T08:07:26.471 INFO:tasks.workunit.client.1.smithi148.stdout:1/692: mknod d3/de/d21/cdd 0 2024-03-29T08:07:26.475 INFO:tasks.workunit.client.1.smithi148.stdout:9/782: truncate db/d10/d2d/d4e/da6/fac 1285112 0 2024-03-29T08:07:26.477 INFO:tasks.workunit.client.1.smithi148.stdout:5/745: mknod d0/db/d1e/d5b/cf1 0 2024-03-29T08:07:26.477 INFO:tasks.workunit.client.1.smithi148.stdout:1/693: truncate d3/de/d1d/fd5 930667 0 2024-03-29T08:07:26.481 INFO:tasks.workunit.client.0.smithi037.stdout:0/410: link d9/c1a d9/da/dd/d20/d55/d5f/c82 0 2024-03-29T08:07:26.489 INFO:tasks.workunit.client.1.smithi148.stdout:3/813: dwrite de/d24/f36 [0,4194304] 0 2024-03-29T08:07:26.491 INFO:tasks.workunit.client.1.smithi148.stdout:9/783: mknod db/d10/c106 0 2024-03-29T08:07:26.492 INFO:tasks.workunit.client.0.smithi037.stdout:5/650: creat d8/d22/d37/d8d/db3/dcb/fe1 x:0 0 0 2024-03-29T08:07:26.493 INFO:tasks.workunit.client.1.smithi148.stdout:5/746: mkdir d0/db/df2 0 2024-03-29T08:07:26.495 INFO:tasks.workunit.client.1.smithi148.stdout:1/694: mkdir d3/d55/dde 0 2024-03-29T08:07:26.497 INFO:tasks.workunit.client.0.smithi037.stdout:6/458: creat d0/d19/d42/d60/d62/f9a x:0 0 0 2024-03-29T08:07:26.499 INFO:tasks.workunit.client.0.smithi037.stdout:7/546: getdents dd/d72 0 2024-03-29T08:07:26.502 INFO:tasks.workunit.client.1.smithi148.stdout:4/761: dwrite df/d2c/d51/d77/d26/d34/d43/f4e [0,4194304] 0 2024-03-29T08:07:26.504 INFO:tasks.workunit.client.1.smithi148.stdout:5/747: creat d0/db/d1e/d79/ff3 x:0 0 0 2024-03-29T08:07:26.506 INFO:tasks.workunit.client.1.smithi148.stdout:0/747: dread db/dc/daa/d47/d4c/d5a/f98 [0,4194304] 0 2024-03-29T08:07:26.512 INFO:tasks.workunit.client.1.smithi148.stdout:9/784: write db/d10/d2d/d4e/da6/fac [11233,82375] 0 2024-03-29T08:07:26.515 INFO:tasks.workunit.client.1.smithi148.stdout:2/839: dread d7/da/d33/d3c/d31/faa [0,4194304] 0 2024-03-29T08:07:26.517 INFO:tasks.workunit.client.0.smithi037.stdout:8/525: dwrite d5/d10/d19/d8d/f97 [0,4194304] 0 2024-03-29T08:07:26.519 INFO:tasks.workunit.client.1.smithi148.stdout:8/736: dwrite d17/d1d/d7e/f83 [0,4194304] 0 2024-03-29T08:07:26.521 INFO:tasks.workunit.client.0.smithi037.stdout:6/459: truncate d0/d5/d1e/f67 3340634 0 2024-03-29T08:07:26.523 INFO:tasks.workunit.client.1.smithi148.stdout:0/748: dread - db/dc/daa/d47/d4c/f89 zero size 2024-03-29T08:07:26.523 INFO:tasks.workunit.client.1.smithi148.stdout:4/762: mkdir df/d2c/d51/d77/d26/d34/d3e/df3 0 2024-03-29T08:07:26.523 INFO:tasks.workunit.client.1.smithi148.stdout:9/785: read - db/d22/fd7 zero size 2024-03-29T08:07:26.526 INFO:tasks.workunit.client.0.smithi037.stdout:9/731: creat d1/d4/d19/d1e/d4a/d6f/da2/ffc x:0 0 0 2024-03-29T08:07:26.527 INFO:tasks.workunit.client.1.smithi148.stdout:7/713: dwrite d8/d29/d44/d49/f83 [0,4194304] 0 2024-03-29T08:07:26.529 INFO:tasks.workunit.client.1.smithi148.stdout:2/840: write d7/da/d33/ded/d2e/d37/f43 [2524496,36367] 0 2024-03-29T08:07:26.530 INFO:tasks.workunit.client.0.smithi037.stdout:1/593: rename d2/d40/l46 to d2/d6/dc/d18/d1d/d21/lcc 0 2024-03-29T08:07:26.531 INFO:tasks.workunit.client.1.smithi148.stdout:6/755: dread d6/d9/d1e/d20/d30/f93 [0,4194304] 0 2024-03-29T08:07:26.533 INFO:tasks.workunit.client.0.smithi037.stdout:3/504: dwrite f2 [0,4194304] 0 2024-03-29T08:07:26.534 INFO:tasks.workunit.client.1.smithi148.stdout:5/748: getdents d0/db/d84/daf/de1 0 2024-03-29T08:07:26.536 INFO:tasks.workunit.client.0.smithi037.stdout:4/541: getdents d3/d4/d15/d1d/db/d45/d1e/d59 0 2024-03-29T08:07:26.538 INFO:tasks.workunit.client.0.smithi037.stdout:2/383: write d0/f7 [4865980,621] 0 2024-03-29T08:07:26.540 INFO:tasks.workunit.client.1.smithi148.stdout:7/714: read d8/d14/d45/d4d/f6b [1425908,50777] 0 2024-03-29T08:07:26.541 INFO:tasks.workunit.client.1.smithi148.stdout:9/786: rmdir db/d10 39 2024-03-29T08:07:26.543 INFO:tasks.workunit.client.1.smithi148.stdout:4/763: chown df/d2c/d51/d77/d26/d34/d8a/fc4 50640 1 2024-03-29T08:07:26.543 INFO:tasks.workunit.client.0.smithi037.stdout:7/547: unlink dd/d21/f26 0 2024-03-29T08:07:26.544 INFO:tasks.workunit.client.1.smithi148.stdout:6/756: symlink d6/d9/d1e/d36/d4d/d9d/lf1 0 2024-03-29T08:07:26.547 INFO:tasks.workunit.client.1.smithi148.stdout:8/737: mknod d17/d49/d9a/dbe/d6f/d64/df1/cf7 0 2024-03-29T08:07:26.547 INFO:tasks.workunit.client.0.smithi037.stdout:8/526: mknod d5/d10/d19/d3d/d4a/d4c/d67/cae 0 2024-03-29T08:07:26.551 INFO:tasks.workunit.client.0.smithi037.stdout:6/460: truncate d0/d19/d34/f66 1013143 0 2024-03-29T08:07:26.556 INFO:tasks.workunit.client.0.smithi037.stdout:1/594: creat d2/d6/dc/d18/d22/da0/fcd x:0 0 0 2024-03-29T08:07:26.559 INFO:tasks.workunit.client.1.smithi148.stdout:3/814: dread de/d24/d33/d39/d46/d30/d37/d62/d63/f92 [0,4194304] 0 2024-03-29T08:07:26.561 INFO:tasks.workunit.client.1.smithi148.stdout:8/738: readlink d17/d49/d9a/dbe/d6f/d64/ld1 0 2024-03-29T08:07:26.561 INFO:tasks.workunit.client.1.smithi148.stdout:7/715: dread d8/d38/d3c/f96 [0,4194304] 0 2024-03-29T08:07:26.562 INFO:tasks.workunit.client.0.smithi037.stdout:7/548: unlink dd/d21/d35/f79 0 2024-03-29T08:07:26.565 INFO:tasks.workunit.client.0.smithi037.stdout:2/384: chown d0/f40 390 1 2024-03-29T08:07:26.565 INFO:tasks.workunit.client.0.smithi037.stdout:9/732: mkdir d1/d1f/dfd 0 2024-03-29T08:07:26.565 INFO:tasks.workunit.client.1.smithi148.stdout:4/764: dread - df/d2c/d80/daa/dbc/dc9/fe9 zero size 2024-03-29T08:07:26.568 INFO:tasks.workunit.client.1.smithi148.stdout:5/749: getdents d0/db/d1e/d35/d42/d67/d76/d81/d98 0 2024-03-29T08:07:26.570 INFO:tasks.workunit.client.1.smithi148.stdout:1/695: dwrite d3/f5d [0,4194304] 0 2024-03-29T08:07:26.575 INFO:tasks.workunit.client.1.smithi148.stdout:0/749: dwrite db/de/df/d11/f16 [0,4194304] 0 2024-03-29T08:07:26.576 INFO:tasks.workunit.client.1.smithi148.stdout:7/716: creat d8/d38/d6a/ffa x:0 0 0 2024-03-29T08:07:26.579 INFO:tasks.workunit.client.0.smithi037.stdout:5/651: dwrite d8/fa [4194304,4194304] 0 2024-03-29T08:07:26.579 INFO:tasks.workunit.client.0.smithi037.stdout:0/411: write d9/da/d1b/d22/d2c/f57 [54406,125574] 0 2024-03-29T08:07:26.587 INFO:tasks.workunit.client.1.smithi148.stdout:9/787: dread db/d22/d26/d54/d4d/d35/d4b/d58/f8b [0,4194304] 0 2024-03-29T08:07:26.587 INFO:tasks.workunit.client.0.smithi037.stdout:3/505: getdents dd/d46/d2d 0 2024-03-29T08:07:26.589 INFO:tasks.workunit.client.0.smithi037.stdout:4/542: creat d3/d4/d15/d1d/db/d45/fae x:0 0 0 2024-03-29T08:07:26.589 INFO:tasks.workunit.client.0.smithi037.stdout:6/461: dread d0/d5/d1a/d61/f65 [0,4194304] 0 2024-03-29T08:07:26.593 INFO:tasks.workunit.client.1.smithi148.stdout:7/717: unlink d8/d14/c32 0 2024-03-29T08:07:26.595 INFO:tasks.workunit.client.1.smithi148.stdout:9/788: creat db/d22/d26/d54/d4d/dbb/def/f107 x:0 0 0 2024-03-29T08:07:26.596 INFO:tasks.workunit.client.1.smithi148.stdout:6/757: getdents d6/d9/d1e/d27/d48 0 2024-03-29T08:07:26.598 INFO:tasks.workunit.client.0.smithi037.stdout:7/549: truncate dd/d21/d35/d60/d94/fa8 871868 0 2024-03-29T08:07:26.598 INFO:tasks.workunit.client.1.smithi148.stdout:7/718: read d8/d14/f2b [1172671,26773] 0 2024-03-29T08:07:26.599 INFO:tasks.workunit.client.1.smithi148.stdout:5/750: dwrite d0/db/d7b/d16/d87/fad [0,4194304] 0 2024-03-29T08:07:26.602 INFO:tasks.workunit.client.1.smithi148.stdout:0/750: fdatasync db/dc/daa/d47/d4c/d5a/f98 0 2024-03-29T08:07:26.604 INFO:tasks.workunit.client.0.smithi037.stdout:1/595: dread d2/d6/dc/d18/d22/d4f/d7c/f7f [0,4194304] 0 2024-03-29T08:07:26.604 INFO:tasks.workunit.client.0.smithi037.stdout:5/652: symlink d8/d34/le2 0 2024-03-29T08:07:26.604 INFO:tasks.workunit.client.1.smithi148.stdout:2/841: dread d7/f17 [4194304,4194304] 0 2024-03-29T08:07:26.604 INFO:tasks.workunit.client.1.smithi148.stdout:3/815: link de/d24/d33/d39/da3/ff6 de/d24/f10b 0 2024-03-29T08:07:26.605 INFO:tasks.workunit.client.0.smithi037.stdout:5/653: fdatasync d8/d22/d37/fd9 0 2024-03-29T08:07:26.606 INFO:tasks.workunit.client.1.smithi148.stdout:4/765: write df/d2c/d51/d77/f90 [603804,40077] 0 2024-03-29T08:07:26.609 INFO:tasks.workunit.client.0.smithi037.stdout:0/412: unlink d9/f6c 0 2024-03-29T08:07:26.609 INFO:tasks.workunit.client.1.smithi148.stdout:2/842: write d7/da/d33/ded/fc3 [260270,90704] 0 2024-03-29T08:07:26.610 INFO:tasks.workunit.client.1.smithi148.stdout:1/696: truncate d3/fa 6375784 0 2024-03-29T08:07:26.610 INFO:tasks.workunit.client.0.smithi037.stdout:3/506: truncate dd/d46/d2d/d3a/d5f/f77 928166 0 2024-03-29T08:07:26.611 INFO:tasks.workunit.client.1.smithi148.stdout:5/751: mknod d0/db/d7b/d16/d2e/ddd/cf4 0 2024-03-29T08:07:26.611 INFO:tasks.workunit.client.1.smithi148.stdout:1/697: chown d3/d18/f39 31 1 2024-03-29T08:07:26.614 INFO:tasks.workunit.client.1.smithi148.stdout:1/698: dread - d3/d55/d7e/fd7 zero size 2024-03-29T08:07:26.616 INFO:tasks.workunit.client.1.smithi148.stdout:6/758: creat d6/d9/d1e/d36/d4d/d9d/d28/dda/db4/ff2 x:0 0 0 2024-03-29T08:07:26.619 INFO:tasks.workunit.client.1.smithi148.stdout:5/752: unlink d0/db/d1e/d79/fd3 0 2024-03-29T08:07:26.620 INFO:tasks.workunit.client.1.smithi148.stdout:5/753: chown d0/db/d7b/l61 59 1 2024-03-29T08:07:26.620 INFO:tasks.workunit.client.0.smithi037.stdout:4/543: symlink d3/d4/d15/d1d/db/d45/d1e/d59/d79/laf 0 2024-03-29T08:07:26.621 INFO:tasks.workunit.client.0.smithi037.stdout:1/596: write d2/d6/dc/d58/d67/fa3 [413154,18189] 0 2024-03-29T08:07:26.625 INFO:tasks.workunit.client.1.smithi148.stdout:9/789: truncate db/d10/d2d/f68 2130225 0 2024-03-29T08:07:26.629 INFO:tasks.workunit.client.1.smithi148.stdout:3/816: dread de/d24/d33/d39/d71/f9b [0,4194304] 0 2024-03-29T08:07:26.634 INFO:tasks.workunit.client.1.smithi148.stdout:2/843: mkdir d7/da/d4c/dc4/dc6/d10f 0 2024-03-29T08:07:26.637 INFO:tasks.workunit.client.0.smithi037.stdout:6/462: dwrite d0/d5/d1a/f2f [0,4194304] 0 2024-03-29T08:07:26.643 INFO:tasks.workunit.client.0.smithi037.stdout:5/654: symlink d8/d4d/d72/le3 0 2024-03-29T08:07:26.644 INFO:tasks.workunit.client.0.smithi037.stdout:8/527: write d5/f80 [768595,10931] 0 2024-03-29T08:07:26.644 INFO:tasks.workunit.client.1.smithi148.stdout:0/751: link db/dc/daa/d47/d4c/ldd db/de/df/d11/d42/le4 0 2024-03-29T08:07:26.648 INFO:tasks.workunit.client.1.smithi148.stdout:2/844: dwrite d7/d8c/d91/f25 [4194304,4194304] 0 2024-03-29T08:07:26.648 INFO:tasks.workunit.client.0.smithi037.stdout:8/528: readlink d5/d10/d19/l9e 0 2024-03-29T08:07:26.650 INFO:tasks.workunit.client.1.smithi148.stdout:1/699: mkdir d3/d55/ddf 0 2024-03-29T08:07:26.650 INFO:tasks.workunit.client.1.smithi148.stdout:9/790: readlink db/lf 0 2024-03-29T08:07:26.652 INFO:tasks.workunit.client.1.smithi148.stdout:3/817: symlink de/d24/d33/d39/d46/d30/d37/d79/l10c 0 2024-03-29T08:07:26.652 INFO:tasks.workunit.client.1.smithi148.stdout:7/719: rename d8/d29 to d8/dfb 0 2024-03-29T08:07:26.654 INFO:tasks.workunit.client.1.smithi148.stdout:1/700: write d3/d55/d7e/fb5 [543164,46983] 0 2024-03-29T08:07:26.655 INFO:tasks.workunit.client.1.smithi148.stdout:0/752: creat db/de/d8e/da9/fe5 x:0 0 0 2024-03-29T08:07:26.656 INFO:tasks.workunit.client.0.smithi037.stdout:1/597: truncate d2/d6/dc/d18/f29 873776 0 2024-03-29T08:07:26.658 INFO:tasks.workunit.client.1.smithi148.stdout:1/701: chown d3/de/d25/d51/f54 15429 1 2024-03-29T08:07:26.658 INFO:tasks.workunit.client.0.smithi037.stdout:2/385: link d0/d20/l25 d0/d23/d44/l83 0 2024-03-29T08:07:26.660 INFO:tasks.workunit.client.1.smithi148.stdout:4/766: creat df/d2c/d51/ff4 x:0 0 0 2024-03-29T08:07:26.662 INFO:tasks.workunit.client.1.smithi148.stdout:4/767: dread - df/d2c/d51/d77/d26/f63 zero size 2024-03-29T08:07:26.667 INFO:tasks.workunit.client.1.smithi148.stdout:2/845: dwrite d7/d24/d4b/f55 [4194304,4194304] 0 2024-03-29T08:07:26.667 INFO:tasks.workunit.client.0.smithi037.stdout:5/655: creat d8/d4d/d5c/dbf/fe4 x:0 0 0 2024-03-29T08:07:26.669 INFO:tasks.workunit.client.1.smithi148.stdout:1/702: creat d3/de/d1d/d47/fe0 x:0 0 0 2024-03-29T08:07:26.670 INFO:tasks.workunit.client.1.smithi148.stdout:0/753: mknod db/dc/daa/d47/daf/ce6 0 2024-03-29T08:07:26.671 INFO:tasks.workunit.client.1.smithi148.stdout:9/791: symlink db/d10/l108 0 2024-03-29T08:07:26.672 INFO:tasks.workunit.client.1.smithi148.stdout:3/818: dwrite de/d24/d33/d39/d46/d30/fa2 [4194304,4194304] 0 2024-03-29T08:07:26.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:26.674+0000 7f976aa00700 1 -- 172.21.15.37:0/3065520488 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f9744005860 con 0x7f9764103420 2024-03-29T08:07:26.674 INFO:tasks.workunit.client.0.smithi037.stdout:9/733: write d1/d4/d19/d1e/d25/d16/f9e [3393508,106491] 0 2024-03-29T08:07:26.676 INFO:tasks.workunit.client.1.smithi148.stdout:0/754: write db/de/df/d2b/f76 [558779,52278] 0 2024-03-29T08:07:26.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:26.676+0000 7f9757e00700 1 -- 172.21.15.37:0/3065520488 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+850 (secure 0 0 0) 0x7f9758019070 con 0x7f9764103420 2024-03-29T08:07:26.677 INFO:tasks.workunit.client.0.smithi037.stdout:4/544: truncate d3/d4/d15/d1d/db/d45/d1e/d5f/f82 145177 0 2024-03-29T08:07:26.677 INFO:tasks.workunit.client.0.smithi037.stdout:7/550: write dd/f1a [5008807,41980] 0 2024-03-29T08:07:26.679 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-29T08:07:26.679 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": { 2024-03-29T08:07:26.679 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 17.2.7-789-g4213e17c (4213e17c4de67016ed14f55d22503f4923b3a901) quincy (stable)": 2 2024-03-29T08:07:26.679 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:07:26.680 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": { 2024-03-29T08:07:26.680 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 17.2.7-789-g4213e17c (4213e17c4de67016ed14f55d22503f4923b3a901) quincy (stable)": 1, 2024-03-29T08:07:26.680 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 1 2024-03-29T08:07:26.680 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:07:26.680 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": { 2024-03-29T08:07:26.680 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 17.2.7-789-g4213e17c (4213e17c4de67016ed14f55d22503f4923b3a901) quincy (stable)": 6 2024-03-29T08:07:26.680 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:07:26.680 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": { 2024-03-29T08:07:26.680 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 17.2.7-789-g4213e17c (4213e17c4de67016ed14f55d22503f4923b3a901) quincy (stable)": 4 2024-03-29T08:07:26.680 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:07:26.680 INFO:teuthology.orchestra.run.smithi037.stdout: "overall": { 2024-03-29T08:07:26.680 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 17.2.7-789-g4213e17c (4213e17c4de67016ed14f55d22503f4923b3a901) quincy (stable)": 13, 2024-03-29T08:07:26.680 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 1 2024-03-29T08:07:26.680 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-29T08:07:26.680 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-29T08:07:26.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:26.681+0000 7f976aa00700 1 -- 172.21.15.37:0/3065520488 >> [v2:172.21.15.148:6828/1134897184,v1:172.21.15.148:6829/1134897184] conn(0x7f974c06af00 msgr2=0x7f974c06d3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:07:26.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:26.681+0000 7f976aa00700 1 --2- 172.21.15.37:0/3065520488 >> [v2:172.21.15.148:6828/1134897184,v1:172.21.15.148:6829/1134897184] conn(0x7f974c06af00 0x7f974c06d3b0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f9750005010 tx=0x7f9750009e30 comp rx=0 tx=0).stop 2024-03-29T08:07:26.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:26.681+0000 7f976aa00700 1 -- 172.21.15.37:0/3065520488 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9764103420 msgr2=0x7f976406e420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:07:26.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:26.681+0000 7f976aa00700 1 --2- 172.21.15.37:0/3065520488 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9764103420 0x7f976406e420 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f97580041d0 tx=0x7f975800b040 comp rx=0 tx=0).stop 2024-03-29T08:07:26.682 INFO:tasks.workunit.client.1.smithi148.stdout:4/768: write df/d89/ddd/fe7 [803688,38095] 0 2024-03-29T08:07:26.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:26.682+0000 7f976aa00700 1 -- 172.21.15.37:0/3065520488 shutdown_connections 2024-03-29T08:07:26.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:26.682+0000 7f976aa00700 1 --2- 172.21.15.37:0/3065520488 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9764103420 0x7f976406e420 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:07:26.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:26.682+0000 7f976aa00700 1 --2- 172.21.15.37:0/3065520488 >> [v2:172.21.15.148:6828/1134897184,v1:172.21.15.148:6829/1134897184] conn(0x7f974c06af00 0x7f974c06d3b0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:07:26.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:26.682+0000 7f976aa00700 1 --2- 172.21.15.37:0/3065520488 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f97641046d0 0x7f976406e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:07:26.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:26.682+0000 7f976aa00700 1 -- 172.21.15.37:0/3065520488 >> 172.21.15.37:0/3065520488 conn(0x7f97640725a0 msgr2=0x7f9764073c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:07:26.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:26.682+0000 7f976aa00700 1 -- 172.21.15.37:0/3065520488 shutdown_connections 2024-03-29T08:07:26.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:26.682+0000 7f976aa00700 1 -- 172.21.15.37:0/3065520488 wait complete. 2024-03-29T08:07:26.683 INFO:tasks.workunit.client.0.smithi037.stdout:1/598: stat d2/d6/dc/d18/d1d/dad/f68 0 2024-03-29T08:07:26.686 INFO:tasks.workunit.client.1.smithi148.stdout:7/720: rename d8/d38/d3c/da9/d37/f48 to d8/d38/ffc 0 2024-03-29T08:07:26.686 INFO:tasks.workunit.client.1.smithi148.stdout:2/846: mkdir d7/da/d33/d3c/d31/de1/d110 0 2024-03-29T08:07:26.690 INFO:tasks.workunit.client.0.smithi037.stdout:7/551: fsync dd/d1d/d49/d81/d84/f8a 0 2024-03-29T08:07:26.690 INFO:tasks.workunit.client.1.smithi148.stdout:3/819: mkdir de/d24/d33/d39/d46/d30/d10d 0 2024-03-29T08:07:26.691 INFO:tasks.workunit.client.1.smithi148.stdout:2/847: chown d7/da/d33/ded/d82/d8d 62617593 1 2024-03-29T08:07:26.691 INFO:tasks.workunit.client.1.smithi148.stdout:6/759: dwrite d6/d9/d68/fad [0,4194304] 0 2024-03-29T08:07:26.693 INFO:tasks.workunit.client.0.smithi037.stdout:1/599: chown d2/d4b/d80/db0 59508 1 2024-03-29T08:07:26.695 INFO:tasks.workunit.client.1.smithi148.stdout:9/792: mknod db/d10/d2d/d4e/da6/dd1/dd9/c109 0 2024-03-29T08:07:26.696 INFO:tasks.workunit.client.1.smithi148.stdout:0/755: dread db/f23 [4194304,4194304] 0 2024-03-29T08:07:26.697 INFO:tasks.workunit.client.1.smithi148.stdout:7/721: creat d8/d38/d67/dc1/ffd x:0 0 0 2024-03-29T08:07:26.700 INFO:tasks.workunit.client.0.smithi037.stdout:6/463: creat d0/d19/f9b x:0 0 0 2024-03-29T08:07:26.702 INFO:tasks.workunit.client.1.smithi148.stdout:7/722: write d8/d14/d45/d4d/d78/fc2 [262623,6519] 0 2024-03-29T08:07:26.704 INFO:tasks.workunit.client.1.smithi148.stdout:5/754: truncate d0/db/d1e/d35/d42/d67/d76/d81/d98/dd5/d9c/fbf 569875 0 2024-03-29T08:07:26.705 INFO:tasks.workunit.client.0.smithi037.stdout:2/386: dwrite d0/f7 [0,4194304] 0 2024-03-29T08:07:26.705 INFO:tasks.workunit.client.0.smithi037.stdout:4/545: creat d3/d4/d15/d1d/db/d45/d1e/d59/d7d/d97/fb0 x:0 0 0 2024-03-29T08:07:26.707 INFO:tasks.workunit.client.0.smithi037.stdout:5/656: rename d8/d4d/d5c/l8c to d8/d22/le5 0 2024-03-29T08:07:26.707 INFO:tasks.workunit.client.1.smithi148.stdout:3/820: unlink de/d3b/d49/f64 0 2024-03-29T08:07:26.708 INFO:tasks.workunit.client.1.smithi148.stdout:5/755: write d0/db/d1e/d5b/d66/fb7 [69276,87602] 0 2024-03-29T08:07:26.711 INFO:tasks.workunit.client.1.smithi148.stdout:1/703: dread d3/de/d25/f75 [0,4194304] 0 2024-03-29T08:07:26.712 INFO:tasks.workunit.client.1.smithi148.stdout:2/848: write d7/da/d33/d3c/d54/dec/f8f [603067,24106] 0 2024-03-29T08:07:26.714 INFO:tasks.workunit.client.1.smithi148.stdout:3/821: write de/d24/d33/d39/d46/d30/d37/d62/d63/fcb [172114,14712] 0 2024-03-29T08:07:26.716 INFO:tasks.workunit.client.1.smithi148.stdout:9/793: fdatasync db/d10/d2d/d4e/dd4/fe6 0 2024-03-29T08:07:26.716 INFO:tasks.workunit.client.1.smithi148.stdout:4/769: link df/d2c/d51/d77/d17/fbd df/d2c/d51/d77/d26/d34/d43/ff5 0 2024-03-29T08:07:26.716 INFO:tasks.workunit.client.0.smithi037.stdout:6/464: mknod d0/d4f/c9c 0 2024-03-29T08:07:26.718 INFO:tasks.workunit.client.1.smithi148.stdout:1/704: dread - d3/de/d1d/d47/fe0 zero size 2024-03-29T08:07:26.718 INFO:tasks.workunit.client.0.smithi037.stdout:7/552: mknod dd/d21/d35/cb1 0 2024-03-29T08:07:26.721 INFO:tasks.workunit.client.1.smithi148.stdout:3/822: write de/d24/d33/d39/d46/d30/d37/faa [4745174,95661] 0 2024-03-29T08:07:26.727 INFO:tasks.workunit.client.0.smithi037.stdout:9/734: rename d1/d3/c3c to d1/d1f/d3a/d9b/cfe 0 2024-03-29T08:07:26.729 INFO:tasks.workunit.client.0.smithi037.stdout:9/735: fdatasync d1/fb6 0 2024-03-29T08:07:26.733 INFO:tasks.workunit.client.0.smithi037.stdout:1/600: dwrite d2/d6/dc/d18/d1d/fa9 [0,4194304] 0 2024-03-29T08:07:26.734 INFO:tasks.workunit.client.1.smithi148.stdout:6/760: rename d6/d9/dc5 to d6/d9/d1e/d36/d4d/d9d/d28/dda/db6/df3 0 2024-03-29T08:07:26.735 INFO:tasks.workunit.client.1.smithi148.stdout:0/756: truncate db/dc/daa/d47/d4c/d5a/f98 1037444 0 2024-03-29T08:07:26.737 INFO:tasks.workunit.client.1.smithi148.stdout:7/723: creat d8/d38/d3c/da9/ffe x:0 0 0 2024-03-29T08:07:26.738 INFO:tasks.workunit.client.1.smithi148.stdout:2/849: mkdir d7/d8c/d91/dd5/d111 0 2024-03-29T08:07:26.739 INFO:tasks.workunit.client.1.smithi148.stdout:7/724: dread - d8/dfb/d44/d50/ff9 zero size 2024-03-29T08:07:26.742 INFO:tasks.workunit.client.1.smithi148.stdout:4/770: mkdir df/d2c/d51/d77/d26/d34/d43/df6 0 2024-03-29T08:07:26.742 INFO:tasks.workunit.client.1.smithi148.stdout:8/739: sync 2024-03-29T08:07:26.743 INFO:tasks.workunit.client.1.smithi148.stdout:7/725: mkdir d8/d38/d6a/d91/dff 0 2024-03-29T08:07:26.744 INFO:tasks.workunit.client.0.smithi037.stdout:0/413: rename d9/da/d1b/d33 to d9/da/dd/d13/d83 0 2024-03-29T08:07:26.745 INFO:tasks.workunit.client.1.smithi148.stdout:7/726: write d8/d38/d3c/da9/f63 [2714908,28205] 0 2024-03-29T08:07:26.747 INFO:tasks.workunit.client.1.smithi148.stdout:1/705: mknod d3/d18/dae/ce1 0 2024-03-29T08:07:26.748 INFO:tasks.workunit.client.1.smithi148.stdout:5/756: truncate d0/db/d7b/f3b 1950650 0 2024-03-29T08:07:26.751 INFO:tasks.workunit.client.1.smithi148.stdout:5/757: chown d0/db/d7b/ceb 0 1 2024-03-29T08:07:26.751 INFO:tasks.workunit.client.1.smithi148.stdout:9/794: mkdir db/d22/d26/d54/d4d/d35/d4b/d5a/d74/d10a 0 2024-03-29T08:07:26.751 INFO:tasks.workunit.client.0.smithi037.stdout:5/657: rmdir d8/d22/d37/d60/d61/dc9 0 2024-03-29T08:07:26.754 INFO:tasks.workunit.client.1.smithi148.stdout:3/823: fsync de/d24/d33/d39/d46/d30/d37/f88 0 2024-03-29T08:07:26.755 INFO:tasks.workunit.client.1.smithi148.stdout:8/740: mknod d17/d49/d9a/dbe/d6f/d64/df1/cf8 0 2024-03-29T08:07:26.758 INFO:tasks.workunit.client.1.smithi148.stdout:6/761: mkdir d6/d9/d1e/d36/d4d/d9d/d28/dda/db6/df3/df4 0 2024-03-29T08:07:26.758 INFO:tasks.workunit.client.0.smithi037.stdout:7/553: rename dd/d21/f2c to dd/d21/d70/fb2 0 2024-03-29T08:07:26.762 INFO:tasks.workunit.client.0.smithi037.stdout:0/414: creat d9/da/dd/d20/d38/f84 x:0 0 0 2024-03-29T08:07:26.763 INFO:tasks.workunit.client.0.smithi037.stdout:9/736: link d1/d3/d44/f9f d1/d4/d95/fff 0 2024-03-29T08:07:26.763 INFO:tasks.workunit.client.1.smithi148.stdout:6/762: chown d6/d9/d1e/d27/f90 12748346 1 2024-03-29T08:07:26.766 INFO:tasks.workunit.client.0.smithi037.stdout:5/658: read d8/d34/fba [1149782,102566] 0 2024-03-29T08:07:26.767 INFO:tasks.workunit.client.1.smithi148.stdout:7/727: stat d8/dfb/d44/d50/d94/l9b 0 2024-03-29T08:07:26.768 INFO:tasks.workunit.client.0.smithi037.stdout:2/387: dread d0/f40 [0,4194304] 0 2024-03-29T08:07:26.770 INFO:tasks.workunit.client.0.smithi037.stdout:6/465: dwrite d0/d19/d34/d5a/f55 [0,4194304] 0 2024-03-29T08:07:26.771 INFO:tasks.workunit.client.1.smithi148.stdout:8/741: mknod d17/d18/d26/d4f/cf9 0 2024-03-29T08:07:26.771 INFO:tasks.workunit.client.1.smithi148.stdout:7/728: dread - d8/dfb/db8/fee zero size 2024-03-29T08:07:26.771 INFO:tasks.workunit.client.1.smithi148.stdout:4/771: dwrite df/d2c/d51/d77/d26/d9b/da7/f50 [0,4194304] 0 2024-03-29T08:07:26.774 INFO:tasks.workunit.client.1.smithi148.stdout:1/706: unlink d3/de/d25/d35/d42/l14 0 2024-03-29T08:07:26.778 INFO:tasks.workunit.client.1.smithi148.stdout:3/824: mkdir de/d24/d33/d39/d46/d30/d37/d79/d10e 0 2024-03-29T08:07:26.782 INFO:tasks.workunit.client.1.smithi148.stdout:0/757: dread db/de/df/d11/d42/f6e [0,4194304] 0 2024-03-29T08:07:26.782 INFO:tasks.workunit.client.1.smithi148.stdout:5/758: dread d0/db/d1e/d35/d42/d67/d76/d81/d98/fb9 [0,4194304] 0 2024-03-29T08:07:26.783 INFO:tasks.workunit.client.0.smithi037.stdout:2/388: mkdir d0/d20/d22/d4e/d84 0 2024-03-29T08:07:26.787 INFO:tasks.workunit.client.1.smithi148.stdout:6/763: creat d6/d9/d1e/d36/d4d/d9d/d28/d98/ff5 x:0 0 0 2024-03-29T08:07:26.793 INFO:tasks.workunit.client.0.smithi037.stdout:6/466: rename d0/d19/d34/f50 to d0/d19/d42/d60/f9d 0 2024-03-29T08:07:26.794 INFO:tasks.workunit.client.0.smithi037.stdout:1/601: getdents d2/d6/dc/d58 0 2024-03-29T08:07:26.796 INFO:tasks.workunit.client.1.smithi148.stdout:2/850: dwrite d7/da/d33/d3c/ffb [0,4194304] 0 2024-03-29T08:07:26.798 INFO:tasks.workunit.client.1.smithi148.stdout:0/758: truncate db/de/df/d2b/dae/fbd 1133885 0 2024-03-29T08:07:26.801 INFO:tasks.workunit.client.1.smithi148.stdout:2/851: chown d7/da/l3a 32082764 1 2024-03-29T08:07:26.801 INFO:tasks.workunit.client.1.smithi148.stdout:8/742: chown d17/d49/d9a/dbe/d6f/d64/da5/cad 390071 1 2024-03-29T08:07:26.807 INFO:tasks.workunit.client.1.smithi148.stdout:5/759: rmdir d0/db/d1e/d35/d42/d67/d76/d81/d98 39 2024-03-29T08:07:26.807 INFO:tasks.workunit.client.1.smithi148.stdout:5/760: write d0/db/d7b/d16/d87/fad [3798363,22796] 0 2024-03-29T08:07:26.807 INFO:tasks.workunit.client.1.smithi148.stdout:9/795: dread db/d22/d26/d54/d4d/d35/f48 [0,4194304] 0 2024-03-29T08:07:26.810 INFO:tasks.workunit.client.1.smithi148.stdout:2/852: mkdir d7/da/d4c/dc4/dc6/d112 0 2024-03-29T08:07:26.811 INFO:tasks.workunit.client.1.smithi148.stdout:1/707: dread d3/d55/d7e/fb5 [0,4194304] 0 2024-03-29T08:07:26.812 INFO:tasks.workunit.client.1.smithi148.stdout:7/729: rename d8/d38/d3c/f7e to d8/dfb/d44/dc3/f100 0 2024-03-29T08:07:26.815 INFO:tasks.workunit.client.1.smithi148.stdout:6/764: creat d6/d9/d1e/d36/ff6 x:0 0 0 2024-03-29T08:07:26.816 INFO:tasks.workunit.client.1.smithi148.stdout:4/772: write df/d2c/d51/d77/d26/d34/d43/ff5 [629961,130415] 0 2024-03-29T08:07:26.817 INFO:tasks.workunit.client.1.smithi148.stdout:0/759: symlink db/de/d8e/da9/le7 0 2024-03-29T08:07:26.817 INFO:tasks.workunit.client.1.smithi148.stdout:9/796: write db/f59 [220981,117063] 0 2024-03-29T08:07:26.820 INFO:tasks.workunit.client.1.smithi148.stdout:4/773: chown df/d2c/d51/d77/d26/d34/d8a/f8c 39 1 2024-03-29T08:07:26.822 INFO:tasks.workunit.client.1.smithi148.stdout:2/853: creat d7/da/d52/f113 x:0 0 0 2024-03-29T08:07:26.823 INFO:tasks.workunit.client.0.smithi037.stdout:2/389: mkdir d0/d23/d44/d85 0 2024-03-29T08:07:26.823 INFO:tasks.workunit.client.1.smithi148.stdout:1/708: fsync d3/de/f1f 0 2024-03-29T08:07:26.824 INFO:tasks.workunit.client.1.smithi148.stdout:6/765: mkdir d6/d9/d1e/d27/d48/df7 0 2024-03-29T08:07:26.827 INFO:tasks.workunit.client.1.smithi148.stdout:4/774: chown df/d2c/d51/d77/d26/d9b/da7/caf 403 1 2024-03-29T08:07:26.827 INFO:tasks.workunit.client.1.smithi148.stdout:5/761: mknod d0/db/d7b/cf5 0 2024-03-29T08:07:26.827 INFO:tasks.workunit.client.0.smithi037.stdout:7/554: write dd/f1b [1306825,32444] 0 2024-03-29T08:07:26.828 INFO:tasks.workunit.client.1.smithi148.stdout:9/797: mknod db/d22/d26/d54/d4d/d35/d4b/c10b 0 2024-03-29T08:07:26.829 INFO:tasks.workunit.client.1.smithi148.stdout:2/854: chown d7/da/d33/d45/l10d 55854 1 2024-03-29T08:07:26.831 INFO:tasks.workunit.client.1.smithi148.stdout:3/825: truncate de/d24/d33/d39/d7f/fac 833496 0 2024-03-29T08:07:26.834 INFO:tasks.workunit.client.1.smithi148.stdout:9/798: chown db/d10/d6d/d73/da8/fd8 18497 1 2024-03-29T08:07:26.835 INFO:tasks.workunit.client.0.smithi037.stdout:6/467: creat d0/d6a/d7b/f9e x:0 0 0 2024-03-29T08:07:26.835 INFO:tasks.workunit.client.0.smithi037.stdout:9/737: write d1/d4/d19/d69/fda [630319,104102] 0 2024-03-29T08:07:26.836 INFO:tasks.workunit.client.1.smithi148.stdout:7/730: dwrite d8/dfb/d44/fab [0,4194304] 0 2024-03-29T08:07:26.838 INFO:tasks.workunit.client.0.smithi037.stdout:1/602: creat d2/d6/dc/d18/d1d/dad/d87/fce x:0 0 0 2024-03-29T08:07:26.839 INFO:tasks.workunit.client.1.smithi148.stdout:3/826: write de/d24/d33/d39/d46/d70/def/ff7 [374204,85030] 0 2024-03-29T08:07:26.842 INFO:tasks.workunit.client.1.smithi148.stdout:9/799: mknod db/d22/d26/d54/d52/c10c 0 2024-03-29T08:07:26.843 INFO:tasks.workunit.client.0.smithi037.stdout:5/659: creat d8/d22/d37/d60/fe6 x:0 0 0 2024-03-29T08:07:26.848 INFO:tasks.workunit.client.1.smithi148.stdout:0/760: rename db/de/df/d11/d2d/d3e/f6d to db/de/df/d11/d1f/fe8 0 2024-03-29T08:07:26.849 INFO:tasks.workunit.client.1.smithi148.stdout:6/766: dwrite d6/d9/d1e/f26 [4194304,4194304] 0 2024-03-29T08:07:26.850 INFO:tasks.workunit.client.0.smithi037.stdout:0/415: link d9/ff d9/da/dd/d20/d72/f85 0 2024-03-29T08:07:26.854 INFO:tasks.workunit.client.1.smithi148.stdout:1/709: truncate d3/d18/d4d/d9e/f40 1356681 0 2024-03-29T08:07:26.855 INFO:tasks.workunit.client.1.smithi148.stdout:5/762: rename d0/db/d1e/d35/d42/d67/cbc to d0/db/d1e/d35/d42/d67/d76/d81/d98/dd5/cf6 0 2024-03-29T08:07:26.856 INFO:tasks.workunit.client.1.smithi148.stdout:8/743: mknod d17/d18/cfa 0 2024-03-29T08:07:26.860 INFO:tasks.workunit.client.0.smithi037.stdout:2/390: truncate d0/d20/f49 2909129 0 2024-03-29T08:07:26.860 INFO:tasks.workunit.client.0.smithi037.stdout:9/738: dwrite d1/d4/d19/d1e/d25/d16/f9e [0,4194304] 0 2024-03-29T08:07:26.861 INFO:tasks.workunit.client.1.smithi148.stdout:4/775: dwrite df/d2c/d51/d77/d17/d73/fc0 [4194304,4194304] 0 2024-03-29T08:07:26.862 INFO:tasks.workunit.client.1.smithi148.stdout:8/744: fdatasync d17/d18/d26/d4f/d94/fe3 0 2024-03-29T08:07:26.865 INFO:tasks.workunit.client.1.smithi148.stdout:8/745: chown d17/d18/f4d 32583 1 2024-03-29T08:07:26.868 INFO:tasks.workunit.client.1.smithi148.stdout:5/763: mkdir d0/db/df7 0 2024-03-29T08:07:26.875 INFO:tasks.workunit.client.0.smithi037.stdout:6/468: dwrite d0/d19/d34/d5a/f55 [0,4194304] 0 2024-03-29T08:07:26.877 INFO:tasks.workunit.client.1.smithi148.stdout:9/800: dwrite db/d22/d26/d54/d4d/f79 [0,4194304] 0 2024-03-29T08:07:26.877 INFO:tasks.workunit.client.1.smithi148.stdout:3/827: dread de/d24/d33/d39/fb6 [0,4194304] 0 2024-03-29T08:07:26.880 INFO:tasks.workunit.client.1.smithi148.stdout:6/767: dread d6/d9/d1e/d20/d30/f93 [0,4194304] 0 2024-03-29T08:07:26.886 INFO:tasks.workunit.client.1.smithi148.stdout:1/710: write d3/d18/d4d/f50 [1549379,47880] 0 2024-03-29T08:07:26.887 INFO:tasks.workunit.client.0.smithi037.stdout:1/603: truncate d2/d6/dc/d58/f71 2884819 0 2024-03-29T08:07:26.890 INFO:tasks.workunit.client.0.smithi037.stdout:0/416: creat d9/da/d1b/d22/d2c/f86 x:0 0 0 2024-03-29T08:07:26.891 INFO:tasks.workunit.client.1.smithi148.stdout:5/764: write d0/db/d1e/d5b/d66/fd2 [2813160,65416] 0 2024-03-29T08:07:26.892 INFO:tasks.workunit.client.1.smithi148.stdout:9/801: creat db/d10/d6d/d73/da8/deb/dec/f10d x:0 0 0 2024-03-29T08:07:26.894 INFO:tasks.workunit.client.0.smithi037.stdout:2/391: truncate d0/d20/f3d 319905 0 2024-03-29T08:07:26.894 INFO:tasks.workunit.client.1.smithi148.stdout:1/711: rmdir d3/de/d1d/d47/d6d 39 2024-03-29T08:07:26.894 INFO:tasks.workunit.client.0.smithi037.stdout:1/604: chown d2/d6/dc/d18/d1d/dad/d87/l8a 1236653 1 2024-03-29T08:07:26.903 INFO:tasks.workunit.client.1.smithi148.stdout:8/746: dwrite d17/d18/d26/fd7 [0,4194304] 0 2024-03-29T08:07:26.903 INFO:tasks.workunit.client.1.smithi148.stdout:6/768: mknod d6/d9/d1e/d36/d4d/d9d/d28/dda/db6/df3/cf8 0 2024-03-29T08:07:26.904 INFO:tasks.workunit.client.0.smithi037.stdout:7/555: link dd/d1d/l5b dd/d21/d35/d60/d94/lb3 0 2024-03-29T08:07:26.907 INFO:tasks.workunit.client.0.smithi037.stdout:5/660: dread d8/d22/d4f/d70/d9c/f84 [0,4194304] 0 2024-03-29T08:07:26.909 INFO:tasks.workunit.client.1.smithi148.stdout:4/776: dwrite df/d89/fcf [0,4194304] 0 2024-03-29T08:07:26.910 INFO:tasks.workunit.client.0.smithi037.stdout:6/469: getdents d0/d20/d95 0 2024-03-29T08:07:26.911 INFO:tasks.workunit.client.0.smithi037.stdout:2/392: chown d0/d20/d22/l64 11 1 2024-03-29T08:07:26.913 INFO:tasks.workunit.client.0.smithi037.stdout:9/739: creat d1/d4/d19/d1e/d4a/d6f/da2/f100 x:0 0 0 2024-03-29T08:07:26.917 INFO:tasks.workunit.client.0.smithi037.stdout:6/470: readlink d0/d4f/l58 0 2024-03-29T08:07:26.917 INFO:tasks.workunit.client.0.smithi037.stdout:0/417: creat d9/d74/d7d/f87 x:0 0 0 2024-03-29T08:07:26.922 INFO:tasks.workunit.client.1.smithi148.stdout:9/802: rename db/d22/d26/d54/f45 to db/d22/d26/f10e 0 2024-03-29T08:07:26.927 INFO:tasks.workunit.client.0.smithi037.stdout:2/393: fdatasync d0/f15 0 2024-03-29T08:07:26.933 INFO:tasks.workunit.client.1.smithi148.stdout:5/765: creat d0/db/d1e/d35/d42/d67/d76/d81/ff8 x:0 0 0 2024-03-29T08:07:26.933 INFO:tasks.workunit.client.0.smithi037.stdout:9/740: creat d1/d4/d95/f101 x:0 0 0 2024-03-29T08:07:26.934 INFO:tasks.workunit.client.1.smithi148.stdout:4/777: mknod df/d2c/d51/d77/d26/d34/d3e/df3/cf7 0 2024-03-29T08:07:26.934 INFO:tasks.workunit.client.1.smithi148.stdout:6/769: getdents d6/d69 0 2024-03-29T08:07:26.935 INFO:tasks.workunit.client.0.smithi037.stdout:7/556: dwrite dd/d1d/d34/f66 [4194304,4194304] 0 2024-03-29T08:07:26.936 INFO:tasks.workunit.client.1.smithi148.stdout:8/747: dwrite d17/dce/de8/fed [0,4194304] 0 2024-03-29T08:07:26.938 INFO:tasks.workunit.client.1.smithi148.stdout:0/761: dread db/de/df/d11/f37 [4194304,4194304] 0 2024-03-29T08:07:26.941 INFO:tasks.workunit.client.0.smithi037.stdout:7/557: write dd/d72/fac [123895,43424] 0 2024-03-29T08:07:26.943 INFO:tasks.workunit.client.1.smithi148.stdout:7/731: sync 2024-03-29T08:07:26.943 INFO:tasks.workunit.client.1.smithi148.stdout:2/855: sync 2024-03-29T08:07:26.944 INFO:tasks.workunit.client.0.smithi037.stdout:6/471: rmdir d0/d6a/d7b 39 2024-03-29T08:07:26.948 INFO:tasks.workunit.client.0.smithi037.stdout:1/605: creat d2/d6/dc/d18/d22/fcf x:0 0 0 2024-03-29T08:07:26.948 INFO:tasks.workunit.client.1.smithi148.stdout:3/828: truncate de/d24/d33/d39/d46/d30/d37/faa 15231 0 2024-03-29T08:07:26.951 INFO:tasks.workunit.client.1.smithi148.stdout:4/778: dwrite df/d60/f85 [4194304,4194304] 0 2024-03-29T08:07:26.952 INFO:tasks.workunit.client.1.smithi148.stdout:8/748: dwrite fe [0,4194304] 0 2024-03-29T08:07:26.955 INFO:tasks.workunit.client.1.smithi148.stdout:4/779: chown df/d2c/d51/d77/d26/d34/l55 155063 1 2024-03-29T08:07:26.963 INFO:tasks.workunit.client.1.smithi148.stdout:9/803: unlink db/d10/d2d/d4e/f6c 0 2024-03-29T08:07:26.965 INFO:tasks.workunit.client.1.smithi148.stdout:1/712: dread d3/de/d25/d51/f54 [0,4194304] 0 2024-03-29T08:07:26.967 INFO:tasks.workunit.client.1.smithi148.stdout:6/770: fdatasync d6/d9/d1e/d36/d4d/d9d/d28/f74 0 2024-03-29T08:07:26.967 INFO:tasks.workunit.client.1.smithi148.stdout:1/713: chown d3/de/d1d/d47 16020 1 2024-03-29T08:07:26.968 INFO:tasks.workunit.client.0.smithi037.stdout:7/558: mkdir dd/d1d/d34/d54/db4 0 2024-03-29T08:07:26.969 INFO:tasks.workunit.client.0.smithi037.stdout:1/606: creat d2/d4b/fd0 x:0 0 0 2024-03-29T08:07:26.970 INFO:tasks.workunit.client.1.smithi148.stdout:1/714: chown d3/d18/d4d/l5c 235917 1 2024-03-29T08:07:26.972 INFO:tasks.workunit.client.0.smithi037.stdout:7/559: chown dd/d21/d65/da2 168217135 1 2024-03-29T08:07:26.972 INFO:tasks.workunit.client.1.smithi148.stdout:2/856: fsync d7/da/d33/d3c/d54/dec/fbe 0 2024-03-29T08:07:26.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:26.975+0000 7f35c4400700 1 -- 172.21.15.37:0/3431686870 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f35bc0fdfe0 msgr2=0x7f35bc0fe450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:07:26.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:26.975+0000 7f35c4400700 1 --2- 172.21.15.37:0/3431686870 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f35bc0fdfe0 0x7f35bc0fe450 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f35a8004940 tx=0x7f35a8009e30 comp rx=0 tx=0).stop 2024-03-29T08:07:26.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:26.975+0000 7f35c4400700 1 -- 172.21.15.37:0/3431686870 shutdown_connections 2024-03-29T08:07:26.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:26.975+0000 7f35c4400700 1 --2- 172.21.15.37:0/3431686870 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f35bc0fdfe0 0x7f35bc0fe450 secure :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f35a8004940 tx=0x7f35a8009e30 comp rx=0 tx=0).stop 2024-03-29T08:07:26.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:26.975+0000 7f35c4400700 1 --2- 172.21.15.37:0/3431686870 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f35bc0fce70 0x7f35bc0fd280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:07:26.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:26.975+0000 7f35c4400700 1 -- 172.21.15.37:0/3431686870 >> 172.21.15.37:0/3431686870 conn(0x7f35bc0f8510 msgr2=0x7f35bc0fa960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:07:26.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:26.976+0000 7f35c4400700 1 -- 172.21.15.37:0/3431686870 shutdown_connections 2024-03-29T08:07:26.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:26.976+0000 7f35c4400700 1 -- 172.21.15.37:0/3431686870 wait complete. 2024-03-29T08:07:26.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:26.977+0000 7f35c4400700 1 Processor -- start 2024-03-29T08:07:26.977 INFO:tasks.workunit.client.0.smithi037.stdout:7/560: read dd/d21/f36 [415309,114062] 0 2024-03-29T08:07:26.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:26.978+0000 7f35c4400700 1 -- start start 2024-03-29T08:07:26.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:26.978+0000 7f35c4400700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f35bc0fce70 0x7f35bc114010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:07:26.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:26.978+0000 7f35c4400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f35bc114550 0x7f35bc119510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:07:26.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:26.978+0000 7f35c4400700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35bc114a10 con 0x7f35bc114550 2024-03-29T08:07:26.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:26.978+0000 7f35c4400700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35bc114b50 con 0x7f35bc0fce70 2024-03-29T08:07:26.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:26.979+0000 7f35c1a00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f35bc0fce70 0x7f35bc114010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:07:26.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:26.979+0000 7f35c1a00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f35bc0fce70 0x7f35bc114010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.37:43870/0 (socket says 172.21.15.37:43870) 2024-03-29T08:07:26.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:26.979+0000 7f35c1a00700 1 -- 172.21.15.37:0/282127810 learned_addr learned my addr 172.21.15.37:0/282127810 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:07:26.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:26.979+0000 7f35c1000700 1 --2- 172.21.15.37:0/282127810 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f35bc114550 0x7f35bc119510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:07:26.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:26.980+0000 7f35c1000700 1 -- 172.21.15.37:0/282127810 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f35bc0fce70 msgr2=0x7f35bc114010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:07:26.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:26.980+0000 7f35c1000700 1 --2- 172.21.15.37:0/282127810 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f35bc0fce70 0x7f35bc114010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:07:26.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:26.980+0000 7f35c1000700 1 -- 172.21.15.37:0/282127810 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35a8005040 con 0x7f35bc114550 2024-03-29T08:07:26.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:26.980+0000 7f35c1000700 1 --2- 172.21.15.37:0/282127810 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f35bc114550 0x7f35bc119510 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f35a8008000 tx=0x7f35a8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:07:26.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:26.982+0000 7f35b9600700 1 -- 172.21.15.37:0/282127810 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f35a8020070 con 0x7f35bc114550 2024-03-29T08:07:26.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:26.982+0000 7f35b9600700 1 -- 172.21.15.37:0/282127810 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f35a8013d20 con 0x7f35bc114550 2024-03-29T08:07:26.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:26.982+0000 7f35b9600700 1 -- 172.21.15.37:0/282127810 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f35a800e070 con 0x7f35bc114550 2024-03-29T08:07:26.983 INFO:tasks.workunit.client.1.smithi148.stdout:5/766: dwrite d0/db/d1e/fc5 [0,4194304] 0 2024-03-29T08:07:26.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:26.984+0000 7f35c4400700 1 -- 172.21.15.37:0/282127810 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35bc0679e0 con 0x7f35bc114550 2024-03-29T08:07:26.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:26.985+0000 7f35c4400700 1 -- 172.21.15.37:0/282127810 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35bc067ec0 con 0x7f35bc114550 2024-03-29T08:07:26.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:26.988+0000 7f35c4400700 1 -- 172.21.15.37:0/282127810 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35bc103c80 con 0x7f35bc114550 2024-03-29T08:07:26.989 INFO:tasks.workunit.client.1.smithi148.stdout:9/804: rmdir db/d22/d26/d54/d52/d9c 39 2024-03-29T08:07:26.991 INFO:tasks.workunit.client.1.smithi148.stdout:9/805: write db/d22/d26/d54/f49 [1770606,82423] 0 2024-03-29T08:07:26.991 INFO:tasks.workunit.client.1.smithi148.stdout:6/771: mknod d6/d9/d1e/d36/d4d/d9d/d28/dba/cf9 0 2024-03-29T08:07:26.993 INFO:tasks.workunit.client.0.smithi037.stdout:2/394: rename d0/d20/d22/f50 to d0/d20/d2e/d5f/d6f/d5e/f86 0 2024-03-29T08:07:26.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:26.992+0000 7f35b9600700 1 -- 172.21.15.37:0/282127810 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 25) v1 ==== 88766+0+0 (secure 0 0 0) 0x7f35a801c400 con 0x7f35bc114550 2024-03-29T08:07:26.995 INFO:tasks.workunit.client.1.smithi148.stdout:6/772: write d6/d9/d1e/d27/f32 [2090879,3074] 0 2024-03-29T08:07:26.996 INFO:tasks.workunit.client.1.smithi148.stdout:9/806: chown db/d22/d26/d54/d4d/dbb/lcf 0 1 2024-03-29T08:07:26.997 INFO:tasks.workunit.client.1.smithi148.stdout:1/715: mknod d3/de/d21/d5f/ce2 0 2024-03-29T08:07:27.002 INFO:tasks.workunit.client.1.smithi148.stdout:1/716: chown d3/d55/d7e/l8d 344678 1 2024-03-29T08:07:27.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:26.996+0000 7f35b9600700 1 --2- 172.21.15.37:0/282127810 >> [v2:172.21.15.148:6828/1134897184,v1:172.21.15.148:6829/1134897184] conn(0x7f35a406af80 0x7f35a406d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:07:27.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:26.996+0000 7f35b9600700 1 -- 172.21.15.37:0/282127810 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f35a808ae20 con 0x7f35bc114550 2024-03-29T08:07:27.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:26.997+0000 7f35c1a00700 1 --2- 172.21.15.37:0/282127810 >> [v2:172.21.15.148:6828/1134897184,v1:172.21.15.148:6829/1134897184] conn(0x7f35a406af80 0x7f35a406d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:07:27.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:26.999+0000 7f35c1a00700 1 --2- 172.21.15.37:0/282127810 >> [v2:172.21.15.148:6828/1134897184,v1:172.21.15.148:6829/1134897184] conn(0x7f35a406af80 0x7f35a406d430 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f35b0008fd0 tx=0x7f35b000b040 comp rx=0 tx=0).ready entity=mgr.14602 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:07:27.003 INFO:tasks.workunit.client.1.smithi148.stdout:5/767: fdatasync d0/db/d7b/f3b 0 2024-03-29T08:07:27.003 INFO:tasks.workunit.client.0.smithi037.stdout:1/607: fdatasync d2/d4b/d80/fa8 0 2024-03-29T08:07:27.005 INFO:tasks.workunit.client.1.smithi148.stdout:9/807: fdatasync db/d10/d2d/d4e/dd4/fe6 0 2024-03-29T08:07:27.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:27.008+0000 7f35b9600700 1 -- 172.21.15.37:0/282127810 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181579 (secure 0 0 0) 0x7f35a8058dc0 con 0x7f35bc114550 2024-03-29T08:07:27.009 INFO:tasks.workunit.client.0.smithi037.stdout:7/561: creat dd/d21/d35/d60/d94/fb5 x:0 0 0 2024-03-29T08:07:27.010 INFO:tasks.workunit.client.1.smithi148.stdout:2/857: rename d7/da/d33/ded/d2e/c71 to d7/da/d33/c114 0 2024-03-29T08:07:27.016 INFO:tasks.workunit.client.0.smithi037.stdout:2/395: symlink d0/d20/d2e/d5f/d6f/d5e/l87 0 2024-03-29T08:07:27.018 INFO:tasks.workunit.client.1.smithi148.stdout:7/732: mknod d8/d14/d45/c101 0 2024-03-29T08:07:27.024 INFO:tasks.workunit.client.1.smithi148.stdout:8/749: unlink d17/d49/d9a/dbe/f77 0 2024-03-29T08:07:27.029 INFO:tasks.workunit.client.1.smithi148.stdout:2/858: dwrite d7/da/d33/d3c/f42 [0,4194304] 0 2024-03-29T08:07:27.032 INFO:tasks.workunit.client.0.smithi037.stdout:5/661: write d8/d18/f28 [4083899,27094] 0 2024-03-29T08:07:27.036 INFO:tasks.workunit.client.0.smithi037.stdout:7/562: symlink dd/d1d/d34/lb6 0 2024-03-29T08:07:27.037 INFO:tasks.workunit.client.0.smithi037.stdout:2/396: creat d0/d53/f88 x:0 0 0 2024-03-29T08:07:27.040 INFO:tasks.workunit.client.0.smithi037.stdout:2/397: dread - d0/d53/f7a zero size 2024-03-29T08:07:27.056 INFO:tasks.workunit.client.0.smithi037.stdout:5/662: mknod d8/d22/ce7 0 2024-03-29T08:07:27.062 INFO:tasks.workunit.client.0.smithi037.stdout:5/663: mkdir d8/d22/d37/de8 0 2024-03-29T08:07:27.067 INFO:tasks.workunit.client.0.smithi037.stdout:2/398: rmdir d0/d23/d44/d85 0 2024-03-29T08:07:27.069 INFO:tasks.workunit.client.0.smithi037.stdout:0/418: truncate d9/da/dd/d20/d38/d53/f62 1786105 0 2024-03-29T08:07:27.074 INFO:tasks.workunit.client.0.smithi037.stdout:9/741: truncate d1/d4/d19/d69/d75/f8b 1074410 0 2024-03-29T08:07:27.080 INFO:tasks.workunit.client.0.smithi037.stdout:5/664: read d8/d22/d37/d60/fa1 [1473083,31952] 0 2024-03-29T08:07:27.080 INFO:tasks.workunit.client.0.smithi037.stdout:0/419: truncate d9/da/d1b/f29 1166757 0 2024-03-29T08:07:27.081 INFO:tasks.workunit.client.0.smithi037.stdout:5/665: chown d8/d22/d37/dbc/fd4 12668626 1 2024-03-29T08:07:27.082 INFO:tasks.workunit.client.0.smithi037.stdout:0/420: chown d9/da/dd/d20/d38 6819973 1 2024-03-29T08:07:27.084 INFO:tasks.workunit.client.1.smithi148.stdout:5/768: symlink d0/db/d1e/d35/d42/d67/d76/d81/d98/dd5/d9c/lf9 0 2024-03-29T08:07:27.085 INFO:tasks.workunit.client.0.smithi037.stdout:0/421: readlink d9/da/dd/d13/l3f 0 2024-03-29T08:07:27.092 INFO:tasks.workunit.client.1.smithi148.stdout:0/762: truncate db/dc/d78/f94 1662559 0 2024-03-29T08:07:27.092 INFO:tasks.workunit.client.1.smithi148.stdout:2/859: dread - d7/da/d33/d3c/d31/faf zero size 2024-03-29T08:07:27.094 INFO:tasks.workunit.client.0.smithi037.stdout:1/608: write d2/d6/d13/f93 [289497,60786] 0 2024-03-29T08:07:27.096 INFO:tasks.workunit.client.0.smithi037.stdout:2/399: dwrite d0/d23/f30 [0,4194304] 0 2024-03-29T08:07:27.096 INFO:tasks.workunit.client.1.smithi148.stdout:6/773: link d6/d9/d1e/d36/ff6 d6/d9/d1e/d36/d4d/d9d/d28/dda/db6/ffa 0 2024-03-29T08:07:27.100 INFO:tasks.workunit.client.1.smithi148.stdout:6/774: write d6/d9/d1e/d36/d4d/d9d/d28/d98/ff5 [356923,3041] 0 2024-03-29T08:07:27.101 INFO:tasks.workunit.client.0.smithi037.stdout:5/666: creat d8/d4d/ddf/fe9 x:0 0 0 2024-03-29T08:07:27.103 INFO:tasks.workunit.client.0.smithi037.stdout:5/667: write d8/fa0 [954581,117887] 0 2024-03-29T08:07:27.105 INFO:tasks.workunit.client.1.smithi148.stdout:6/775: write d6/d9/d1e/d20/d30/fac [1026564,116073] 0 2024-03-29T08:07:27.112 INFO:tasks.workunit.client.1.smithi148.stdout:9/808: mknod db/d10/d6d/d73/da8/deb/dec/dde/ded/c10f 0 2024-03-29T08:07:27.112 INFO:tasks.workunit.client.1.smithi148.stdout:5/769: getdents d0/db/df7 0 2024-03-29T08:07:27.115 INFO:tasks.workunit.client.1.smithi148.stdout:8/750: dwrite d17/d18/d26/fd7 [4194304,4194304] 0 2024-03-29T08:07:27.116 INFO:tasks.workunit.client.1.smithi148.stdout:0/763: dwrite db/de/df/d2b/f93 [0,4194304] 0 2024-03-29T08:07:27.123 INFO:tasks.workunit.client.1.smithi148.stdout:7/733: getdents d8/d60 0 2024-03-29T08:07:27.125 INFO:tasks.workunit.client.1.smithi148.stdout:8/751: chown d17/d18/l24 188 1 2024-03-29T08:07:27.125 INFO:tasks.workunit.client.1.smithi148.stdout:9/809: creat db/d10/d9d/f110 x:0 0 0 2024-03-29T08:07:27.129 INFO:tasks.workunit.client.0.smithi037.stdout:1/609: creat d2/d6/dc/d18/d1d/dad/daf/fd1 x:0 0 0 2024-03-29T08:07:27.131 INFO:tasks.workunit.client.1.smithi148.stdout:4/780: write df/d2c/d51/d77/d26/f7d [685614,6606] 0 2024-03-29T08:07:27.131 INFO:tasks.workunit.client.0.smithi037.stdout:0/422: creat d9/da/f88 x:0 0 0 2024-03-29T08:07:27.132 INFO:tasks.workunit.client.1.smithi148.stdout:1/717: dread d3/de/d25/d35/d42/f15 [0,4194304] 0 2024-03-29T08:07:27.135 INFO:tasks.workunit.client.1.smithi148.stdout:9/810: fdatasync db/d22/d26/d54/d4d/d35/d4b/d58/f8b 0 2024-03-29T08:07:27.135 INFO:tasks.workunit.client.1.smithi148.stdout:4/781: fdatasync df/f5e 0 2024-03-29T08:07:27.139 INFO:tasks.workunit.client.0.smithi037.stdout:9/742: dwrite d1/d3/d44/f57 [0,4194304] 0 2024-03-29T08:07:27.141 INFO:tasks.workunit.client.1.smithi148.stdout:1/718: truncate d3/de/f1b 2339432 0 2024-03-29T08:07:27.143 INFO:tasks.workunit.client.1.smithi148.stdout:0/764: symlink db/dc/daa/d47/d4c/d5a/d7a/db1/le9 0 2024-03-29T08:07:27.147 INFO:tasks.workunit.client.1.smithi148.stdout:3/829: dwrite de/f51 [0,4194304] 0 2024-03-29T08:07:27.147 INFO:tasks.workunit.client.1.smithi148.stdout:8/752: creat d17/d18/d26/d63/ffb x:0 0 0 2024-03-29T08:07:27.150 INFO:tasks.workunit.client.1.smithi148.stdout:8/753: chown d17/d18/l24 28886080 1 2024-03-29T08:07:27.151 INFO:tasks.workunit.client.0.smithi037.stdout:1/610: mkdir d2/d6/dc/d7e/dd2 0 2024-03-29T08:07:27.153 INFO:tasks.workunit.client.1.smithi148.stdout:9/811: creat db/d10/d2d/d4e/d5f/dd5/f111 x:0 0 0 2024-03-29T08:07:27.154 INFO:tasks.workunit.client.1.smithi148.stdout:4/782: mknod df/d2c/d51/d77/d17/d73/cf8 0 2024-03-29T08:07:27.155 INFO:tasks.workunit.client.1.smithi148.stdout:0/765: truncate db/de/df/d11/d42/dc2/fcf 980844 0 2024-03-29T08:07:27.156 INFO:tasks.workunit.client.1.smithi148.stdout:7/734: creat d8/d38/d3c/da5/de3/f102 x:0 0 0 2024-03-29T08:07:27.160 INFO:tasks.workunit.client.0.smithi037.stdout:0/423: chown d9/da/d1b/l30 434216406 1 2024-03-29T08:07:27.166 INFO:tasks.workunit.client.1.smithi148.stdout:8/754: rename d17/d18/d26/d63/d66/l75 to d17/d18/d26/d63/d7d/lfc 0 2024-03-29T08:07:27.169 INFO:tasks.workunit.client.1.smithi148.stdout:1/719: dwrite d3/d55/d7e/fad [0,4194304] 0 2024-03-29T08:07:27.171 INFO:tasks.workunit.client.0.smithi037.stdout:0/424: truncate d9/da/dd/d13/f5e 4666746 0 2024-03-29T08:07:27.172 INFO:tasks.workunit.client.0.smithi037.stdout:9/743: unlink d1/d4/d19/d1e/d71/d89/da6/dd8/fe9 0 2024-03-29T08:07:27.174 INFO:tasks.workunit.client.1.smithi148.stdout:5/770: rmdir d0/d46/d86 39 2024-03-29T08:07:27.175 INFO:tasks.workunit.client.0.smithi037.stdout:2/400: dwrite d0/d20/d22/d55/f5b [0,4194304] 0 2024-03-29T08:07:27.175 INFO:tasks.workunit.client.1.smithi148.stdout:4/783: creat df/d2c/d51/d77/d26/d9b/da7/d7c/ff9 x:0 0 0 2024-03-29T08:07:27.177 INFO:tasks.workunit.client.0.smithi037.stdout:9/744: chown d1/d3/d44/f57 67914164 1 2024-03-29T08:07:27.178 INFO:tasks.workunit.client.1.smithi148.stdout:4/784: chown df/d2c/d51/d77/d26/d34/d3e/l66 303 1 2024-03-29T08:07:27.183 INFO:tasks.workunit.client.0.smithi037.stdout:9/745: write d1/d4/d19/d1e/d4a/d6f/da2/d31/daa/fef [707452,114232] 0 2024-03-29T08:07:27.183 INFO:tasks.workunit.client.0.smithi037.stdout:5/668: write d8/d22/d4f/d70/f82 [187522,54183] 0 2024-03-29T08:07:27.184 INFO:tasks.workunit.client.1.smithi148.stdout:1/720: link d3/de/d21/cdd d3/d18/dae/ce3 0 2024-03-29T08:07:27.187 INFO:tasks.workunit.client.1.smithi148.stdout:4/785: chown df/d2c/db2/fdf 3 1 2024-03-29T08:07:27.189 INFO:tasks.workunit.client.1.smithi148.stdout:4/786: write df/d2c/d51/d77/d26/f4a [3537585,106959] 0 2024-03-29T08:07:27.190 INFO:tasks.workunit.client.1.smithi148.stdout:0/766: rename db/de/df/d11/d42/dc2 to db/de/df/d14/dea 0 2024-03-29T08:07:27.196 INFO:tasks.workunit.client.1.smithi148.stdout:4/787: chown df/d2c/d80/daa/dbc/dc9/l81 137239114 1 2024-03-29T08:07:27.200 INFO:tasks.workunit.client.1.smithi148.stdout:2/860: sync 2024-03-29T08:07:27.202 INFO:tasks.workunit.client.0.smithi037.stdout:0/425: mknod d9/da/d1b/d22/d2c/c89 0 2024-03-29T08:07:27.202 INFO:tasks.workunit.client.1.smithi148.stdout:2/861: read - d7/da/d33/fd3 zero size 2024-03-29T08:07:27.202 INFO:tasks.workunit.client.1.smithi148.stdout:1/721: mkdir d3/de/d1d/d47/de4 0 2024-03-29T08:07:27.203 INFO:tasks.workunit.client.1.smithi148.stdout:0/767: readlink db/de/df/d2b/ldc 0 2024-03-29T08:07:27.204 INFO:tasks.workunit.client.1.smithi148.stdout:4/788: creat df/d2c/d51/d77/d26/d34/d43/ffa x:0 0 0 2024-03-29T08:07:27.208 INFO:tasks.workunit.client.1.smithi148.stdout:4/789: mknod df/d2c/db2/cfb 0 2024-03-29T08:07:27.211 INFO:tasks.workunit.client.0.smithi037.stdout:5/669: symlink d8/d22/d37/d60/d88/lea 0 2024-03-29T08:07:27.215 INFO:tasks.workunit.client.1.smithi148.stdout:3/830: unlink de/d24/d33/d39/d46/f32 0 2024-03-29T08:07:27.217 INFO:tasks.workunit.client.0.smithi037.stdout:0/426: unlink d9/da/d1b/l6b 0 2024-03-29T08:07:27.219 INFO:tasks.workunit.client.1.smithi148.stdout:1/722: dwrite d3/de/d1d/f33 [0,4194304] 0 2024-03-29T08:07:27.219 INFO:tasks.workunit.client.1.smithi148.stdout:0/768: link db/dc/daa/d47/d9f/fa5 db/dc/daa/d47/d4c/d5a/feb 0 2024-03-29T08:07:27.220 INFO:tasks.workunit.client.1.smithi148.stdout:3/831: write de/d24/d33/d39/f5a [46065,1184] 0 2024-03-29T08:07:27.222 INFO:tasks.workunit.client.1.smithi148.stdout:0/769: chown db/de/d8e/da9/le3 153854 1 2024-03-29T08:07:27.223 INFO:tasks.workunit.client.0.smithi037.stdout:0/427: chown d9/da/d1b/d22/d2c/c89 2607810 1 2024-03-29T08:07:27.223 INFO:tasks.workunit.client.1.smithi148.stdout:7/735: dread d8/d14/d45/d4d/f6b [0,4194304] 0 2024-03-29T08:07:27.224 INFO:tasks.workunit.client.1.smithi148.stdout:0/770: chown db/dc/daa/d47 442 1 2024-03-29T08:07:27.229 INFO:tasks.workunit.client.1.smithi148.stdout:7/736: mknod d8/dfb/db8/c103 0 2024-03-29T08:07:27.231 INFO:tasks.workunit.client.1.smithi148.stdout:6/776: write d6/d9/d1e/d27/f9b [5192138,29311] 0 2024-03-29T08:07:27.232 INFO:tasks.workunit.client.0.smithi037.stdout:5/670: dwrite d8/d18/f28 [0,4194304] 0 2024-03-29T08:07:27.233 INFO:tasks.workunit.client.0.smithi037.stdout:2/401: write d0/d20/d22/f39 [708020,70467] 0 2024-03-29T08:07:27.235 INFO:tasks.workunit.client.1.smithi148.stdout:8/755: rmdir d17/d18/d26/d4f 39 2024-03-29T08:07:27.237 INFO:tasks.workunit.client.0.smithi037.stdout:1/611: truncate d2/d6/dc/d18/d1d/d21/f37 4054507 0 2024-03-29T08:07:27.238 INFO:tasks.workunit.client.1.smithi148.stdout:1/723: dwrite d3/de/d25/f9d [0,4194304] 0 2024-03-29T08:07:27.240 INFO:tasks.workunit.client.1.smithi148.stdout:8/756: write d17/d18/d26/d63/d66/d5f/f8e [3832084,24030] 0 2024-03-29T08:07:27.242 INFO:tasks.workunit.client.0.smithi037.stdout:2/402: readlink d0/d53/l57 0 2024-03-29T08:07:27.246 INFO:tasks.workunit.client.0.smithi037.stdout:5/671: rename d8/d22/d37/d60 to d8/d22/d37/d8d/db3/dcb/deb 0 2024-03-29T08:07:27.249 INFO:tasks.workunit.client.1.smithi148.stdout:0/771: link db/de/df/d11/d2d/l4d db/dc/daa/d47/lec 0 2024-03-29T08:07:27.249 INFO:tasks.workunit.client.1.smithi148.stdout:8/757: read d17/f68 [3924411,22352] 0 2024-03-29T08:07:27.254 INFO:tasks.workunit.client.0.smithi037.stdout:2/403: mkdir d0/d53/d89 0 2024-03-29T08:07:27.256 INFO:tasks.workunit.client.1.smithi148.stdout:0/772: truncate db/de/df/d11/d2d/fdf 1201641 0 2024-03-29T08:07:27.257 INFO:tasks.workunit.client.1.smithi148.stdout:7/737: dread d8/f10 [0,4194304] 0 2024-03-29T08:07:27.261 INFO:tasks.workunit.client.1.smithi148.stdout:6/777: dread d6/d9/d1e/d36/d4d/d9d/fb2 [0,4194304] 0 2024-03-29T08:07:27.264 INFO:tasks.workunit.client.1.smithi148.stdout:6/778: dread - d6/d9/d1e/d36/d4d/d9d/d28/db3/dd2/fe7 zero size 2024-03-29T08:07:27.270 INFO:tasks.workunit.client.1.smithi148.stdout:0/773: write db/de/df/d11/d1f/fe8 [775691,2032] 0 2024-03-29T08:07:27.271 INFO:tasks.workunit.client.0.smithi037.stdout:2/404: read d0/d20/f3a [2864343,100378] 0 2024-03-29T08:07:27.272 INFO:tasks.workunit.client.1.smithi148.stdout:0/774: stat db/de/d8e/da9/fe5 0 2024-03-29T08:07:27.274 INFO:tasks.workunit.client.1.smithi148.stdout:0/775: chown db/dc/daa/c4e 3257606 1 2024-03-29T08:07:27.275 INFO:tasks.workunit.client.1.smithi148.stdout:0/776: stat db/dc/d78/c80 0 2024-03-29T08:07:27.277 INFO:tasks.workunit.client.0.smithi037.stdout:5/672: dwrite d8/fa [4194304,4194304] 0 2024-03-29T08:07:27.280 INFO:tasks.workunit.client.1.smithi148.stdout:0/777: dread - db/de/df/d11/d42/fd0 zero size 2024-03-29T08:07:27.280 INFO:tasks.workunit.client.1.smithi148.stdout:7/738: mknod d8/dfb/d44/dc3/c104 0 2024-03-29T08:07:27.283 INFO:tasks.workunit.client.1.smithi148.stdout:7/739: readlink d8/dfb/l3b 0 2024-03-29T08:07:27.283 INFO:tasks.workunit.client.1.smithi148.stdout:8/758: link d17/d74/d86/f8d d17/d74/d86/ffd 0 2024-03-29T08:07:27.283 INFO:tasks.workunit.client.0.smithi037.stdout:0/428: dwrite d9/da/dd/d13/d83/f49 [0,4194304] 0 2024-03-29T08:07:27.284 INFO:tasks.workunit.client.1.smithi148.stdout:6/779: symlink d6/d9/d1e/d36/d4d/d9d/d28/dda/db6/df3/df4/lfb 0 2024-03-29T08:07:27.290 INFO:tasks.workunit.client.1.smithi148.stdout:6/780: truncate d6/d9/d1e/d27/f90 462736 0 2024-03-29T08:07:27.293 INFO:tasks.workunit.client.0.smithi037.stdout:2/405: rmdir d0/d20/d7d 39 2024-03-29T08:07:27.301 INFO:tasks.workunit.client.1.smithi148.stdout:6/781: mknod d6/d9/d1e/d36/cfc 0 2024-03-29T08:07:27.304 INFO:tasks.workunit.client.0.smithi037.stdout:0/429: fsync d9/da/dd/d54/f56 0 2024-03-29T08:07:27.305 INFO:tasks.workunit.client.1.smithi148.stdout:6/782: write d6/d9/d1e/d36/d4d/d9d/f31 [1973297,121682] 0 2024-03-29T08:07:27.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:07:27 smithi148 ceph-mon[44992]: from='client.14674 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:07:27.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:07:27 smithi148 ceph-mon[44992]: pgmap v45: 65 pgs: 65 active+clean; 2.4 GiB data, 8.5 GiB used, 528 GiB / 536 GiB avail; 79 MiB/s rd, 200 MiB/s wr, 409 op/s 2024-03-29T08:07:27.307 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:07:27 smithi148 ceph-mon[44992]: from='client.? 172.21.15.37:0/3065520488' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:07:27.308 INFO:tasks.workunit.client.0.smithi037.stdout:5/673: truncate d8/d22/f93 1972139 0 2024-03-29T08:07:27.309 INFO:tasks.workunit.client.1.smithi148.stdout:6/783: readlink d6/d9/d1e/d36/d4d/d9d/lf1 0 2024-03-29T08:07:27.310 INFO:tasks.workunit.client.0.smithi037.stdout:5/674: readlink d8/l6f 0 2024-03-29T08:07:27.313 INFO:tasks.workunit.client.0.smithi037.stdout:0/430: mkdir d9/da/dd/d20/d38/d8a 0 2024-03-29T08:07:27.313 INFO:tasks.workunit.client.1.smithi148.stdout:9/812: dwrite db/d22/d3f/f42 [0,4194304] 0 2024-03-29T08:07:27.315 INFO:tasks.workunit.client.0.smithi037.stdout:5/675: chown d8/d22/d37/d8d/db3/dcb/deb/d88/l98 13565874 1 2024-03-29T08:07:27.316 INFO:tasks.workunit.client.0.smithi037.stdout:2/406: creat d0/f8a x:0 0 0 2024-03-29T08:07:27.325 INFO:tasks.workunit.client.0.smithi037.stdout:0/431: mknod d9/da/d1b/c8b 0 2024-03-29T08:07:27.326 INFO:tasks.workunit.client.1.smithi148.stdout:8/759: write d17/d18/d26/d63/d66/d5f/f6c [4508756,119473] 0 2024-03-29T08:07:27.327 INFO:tasks.workunit.client.1.smithi148.stdout:9/813: getdents db/d10/d2d/d4e/dd4 0 2024-03-29T08:07:27.331 INFO:tasks.workunit.client.1.smithi148.stdout:9/814: write db/d22/d26/d54/d4d/d35/f5e [3166627,58014] 0 2024-03-29T08:07:27.331 INFO:tasks.workunit.client.0.smithi037.stdout:5/676: getdents d8/d22/d37/dbc/d85/d90 0 2024-03-29T08:07:27.332 INFO:tasks.workunit.client.0.smithi037.stdout:0/432: write d9/da/f11 [2534526,88481] 0 2024-03-29T08:07:27.336 INFO:tasks.workunit.client.1.smithi148.stdout:8/760: creat d17/d1d/d7e/ffe x:0 0 0 2024-03-29T08:07:27.336 INFO:tasks.workunit.client.0.smithi037.stdout:0/433: dread - d9/da/dd/d20/d38/f84 zero size 2024-03-29T08:07:27.337 INFO:tasks.workunit.client.1.smithi148.stdout:5/771: rename d0/db/d7b/fd8 to d0/db/d84/daf/ffa 0 2024-03-29T08:07:27.338 INFO:tasks.workunit.client.0.smithi037.stdout:5/677: dread - d8/d22/d37/dbc/fd4 zero size 2024-03-29T08:07:27.338 INFO:tasks.workunit.client.1.smithi148.stdout:0/778: dread db/de/df/d2b/f72 [0,4194304] 0 2024-03-29T08:07:27.340 INFO:tasks.workunit.client.0.smithi037.stdout:2/407: write d0/d23/d44/f46 [1505211,33458] 0 2024-03-29T08:07:27.348 INFO:tasks.workunit.client.1.smithi148.stdout:2/862: rename d7/da/l32 to d7/d24/d4b/d66/l115 0 2024-03-29T08:07:27.349 INFO:tasks.workunit.client.0.smithi037.stdout:0/434: symlink d9/da/d1b/d22/l8c 0 2024-03-29T08:07:27.350 INFO:tasks.workunit.client.1.smithi148.stdout:5/772: dread d0/db/d7b/f3b [0,4194304] 0 2024-03-29T08:07:27.350 INFO:tasks.workunit.client.0.smithi037.stdout:5/678: readlink d8/d22/d37/d8d/db3/dcb/deb/l8b 0 2024-03-29T08:07:27.351 INFO:tasks.workunit.client.1.smithi148.stdout:8/761: dwrite d17/d18/f5c [0,4194304] 0 2024-03-29T08:07:27.351 INFO:tasks.workunit.client.0.smithi037.stdout:2/408: fsync d0/f21 0 2024-03-29T08:07:27.352 INFO:tasks.workunit.client.1.smithi148.stdout:5/773: stat d0/d46/d9e 0 2024-03-29T08:07:27.353 INFO:tasks.workunit.client.0.smithi037.stdout:2/409: chown d0/d23/d44/f67 163106589 1 2024-03-29T08:07:27.354 INFO:tasks.workunit.client.1.smithi148.stdout:7/740: dread d8/d38/f8d [0,4194304] 0 2024-03-29T08:07:27.356 INFO:tasks.workunit.client.1.smithi148.stdout:7/741: chown d8/d38/d3c/da9/db7/fbf 3705 1 2024-03-29T08:07:27.357 INFO:tasks.workunit.client.1.smithi148.stdout:4/790: rename df/d2c/f8d to df/d89/ddd/ffc 0 2024-03-29T08:07:27.359 INFO:tasks.workunit.client.1.smithi148.stdout:4/791: fdatasync df/d2c/d80/fb5 0 2024-03-29T08:07:27.363 INFO:tasks.workunit.client.1.smithi148.stdout:2/863: creat d7/da/d33/d3c/f116 x:0 0 0 2024-03-29T08:07:27.367 INFO:tasks.workunit.client.1.smithi148.stdout:3/832: rename de/d24/d33/d39/d46/l5b to de/d24/d33/d39/d46/d8d/l10f 0 2024-03-29T08:07:27.369 INFO:tasks.workunit.client.1.smithi148.stdout:4/792: unlink df/d2c/d51/d77/f90 0 2024-03-29T08:07:27.370 INFO:tasks.workunit.client.1.smithi148.stdout:2/864: mknod d7/da/d52/c117 0 2024-03-29T08:07:27.373 INFO:tasks.workunit.client.1.smithi148.stdout:2/865: read - d7/d24/d4b/d66/ff0 zero size 2024-03-29T08:07:27.373 INFO:tasks.workunit.client.1.smithi148.stdout:8/762: dwrite d17/d49/d9a/dbe/d6f/d64/da5/fda [0,4194304] 0 2024-03-29T08:07:27.374 INFO:tasks.workunit.client.1.smithi148.stdout:8/763: write f7 [2785260,17669] 0 2024-03-29T08:07:27.376 INFO:tasks.workunit.client.1.smithi148.stdout:2/866: read d7/da/d4c/f69 [231229,5002] 0 2024-03-29T08:07:27.383 INFO:tasks.workunit.client.1.smithi148.stdout:4/793: dread df/d2c/d51/d77/d26/f7d [0,4194304] 0 2024-03-29T08:07:27.384 INFO:tasks.workunit.client.1.smithi148.stdout:1/724: rename d3/de/d25/d35/dc2/cc6 to d3/d18/d4d/d76/ce5 0 2024-03-29T08:07:27.385 INFO:tasks.workunit.client.1.smithi148.stdout:8/764: chown d17/d18/d26/d4f/cf9 4281793 1 2024-03-29T08:07:27.390 INFO:tasks.workunit.client.1.smithi148.stdout:6/784: rename d6/d9/d1e/fbc to d6/d9/d1e/d36/d4d/d9d/d28/d98/dc2/ffd 0 2024-03-29T08:07:27.390 INFO:tasks.workunit.client.1.smithi148.stdout:5/774: dwrite d0/db/d1e/fb2 [0,4194304] 0 2024-03-29T08:07:27.393 INFO:tasks.workunit.client.1.smithi148.stdout:8/765: mkdir d17/d49/d9a/db8/dff 0 2024-03-29T08:07:27.395 INFO:tasks.workunit.client.1.smithi148.stdout:7/742: dwrite d8/d38/d3c/f5a [0,4194304] 0 2024-03-29T08:07:27.397 INFO:tasks.workunit.client.1.smithi148.stdout:0/779: rename db/dc/cd to db/dc/daa/d47/daf/ced 0 2024-03-29T08:07:27.397 INFO:tasks.workunit.client.1.smithi148.stdout:8/766: read d17/dce/de8/fed [491663,122013] 0 2024-03-29T08:07:27.405 INFO:tasks.workunit.client.1.smithi148.stdout:2/867: link d7/da/d33/d3c/d54/c10b d7/d8c/da4/d8e/dda/c118 0 2024-03-29T08:07:27.407 INFO:tasks.workunit.client.1.smithi148.stdout:4/794: truncate df/d2c/d51/d77/d26/d9b/da7/f94 1174424 0 2024-03-29T08:07:27.408 INFO:tasks.workunit.client.1.smithi148.stdout:1/725: rmdir d3/d55/dde 0 2024-03-29T08:07:27.412 INFO:tasks.workunit.client.1.smithi148.stdout:2/868: creat d7/f119 x:0 0 0 2024-03-29T08:07:27.417 INFO:tasks.workunit.client.1.smithi148.stdout:1/726: rmdir d3/d18/d4d/d76/d85 39 2024-03-29T08:07:27.422 INFO:tasks.workunit.client.1.smithi148.stdout:7/743: dwrite d8/d38/d3c/da9/d37/faf [0,4194304] 0 2024-03-29T08:07:27.422 INFO:tasks.workunit.client.1.smithi148.stdout:5/775: rename d0/db/d7b/c17 to d0/db/d1e/d35/d42/d67/d76/d81/cfb 0 2024-03-29T08:07:27.425 INFO:tasks.workunit.client.1.smithi148.stdout:6/785: dwrite d6/d9/f23 [0,4194304] 0 2024-03-29T08:07:27.426 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:07:27 smithi037 ceph-mon[35162]: from='client.14674 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:07:27.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:07:27 smithi037 ceph-mon[35162]: pgmap v45: 65 pgs: 65 active+clean; 2.4 GiB data, 8.5 GiB used, 528 GiB / 536 GiB avail; 79 MiB/s rd, 200 MiB/s wr, 409 op/s 2024-03-29T08:07:27.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:07:27 smithi037 ceph-mon[35162]: from='client.? 172.21.15.37:0/3065520488' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:07:27.433 INFO:tasks.workunit.client.0.smithi037.stdout:5/679: symlink d8/d4d/lec 0 2024-03-29T08:07:27.435 INFO:tasks.workunit.client.0.smithi037.stdout:5/680: readlink d8/d22/d37/d8d/lc6 0 2024-03-29T08:07:27.435 INFO:tasks.workunit.client.1.smithi148.stdout:8/767: rename d17/d18/d26/c8b to d17/d74/c100 0 2024-03-29T08:07:27.436 INFO:tasks.workunit.client.0.smithi037.stdout:2/410: symlink d0/d20/d2e/l8b 0 2024-03-29T08:07:27.437 INFO:tasks.workunit.client.0.smithi037.stdout:0/435: creat d9/da/f8d x:0 0 0 2024-03-29T08:07:27.439 INFO:tasks.workunit.client.1.smithi148.stdout:2/869: creat d7/da/d33/ded/f11a x:0 0 0 2024-03-29T08:07:27.445 INFO:tasks.workunit.client.1.smithi148.stdout:0/780: dread f7 [4194304,4194304] 0 2024-03-29T08:07:27.446 INFO:tasks.workunit.client.1.smithi148.stdout:3/833: dread de/d24/d33/d39/d46/d30/d37/fae [0,4194304] 0 2024-03-29T08:07:27.446 INFO:tasks.workunit.client.0.smithi037.stdout:2/411: link d0/d20/f3a d0/f8c 0 2024-03-29T08:07:27.447 INFO:tasks.workunit.client.1.smithi148.stdout:4/795: rename df/d2c/d80/daa/dbc/dc9/lb1 to df/d2c/d51/d77/d17/lfd 0 2024-03-29T08:07:27.447 INFO:tasks.workunit.client.0.smithi037.stdout:0/436: creat d9/da/dd/d20/d38/d8a/f8e x:0 0 0 2024-03-29T08:07:27.449 INFO:tasks.workunit.client.0.smithi037.stdout:2/412: read - d0/d23/d44/f67 zero size 2024-03-29T08:07:27.450 INFO:tasks.workunit.client.1.smithi148.stdout:0/781: write db/f51 [4165656,86607] 0 2024-03-29T08:07:27.455 INFO:tasks.workunit.client.1.smithi148.stdout:1/727: dwrite d3/de/d25/d35/d42/fa5 [0,4194304] 0 2024-03-29T08:07:27.455 INFO:tasks.workunit.client.1.smithi148.stdout:9/815: sync 2024-03-29T08:07:27.455 INFO:tasks.workunit.client.1.smithi148.stdout:4/796: truncate df/d71/f88 364321 0 2024-03-29T08:07:27.457 INFO:tasks.workunit.client.0.smithi037.stdout:2/413: chown d0/d23/d44/c4c 22 1 2024-03-29T08:07:27.459 INFO:tasks.workunit.client.1.smithi148.stdout:5/776: link d0/db/d1e/d36/d41/f7f d0/db/d7b/d16/d2e/ddd/ffc 0 2024-03-29T08:07:27.460 INFO:tasks.workunit.client.1.smithi148.stdout:0/782: chown db/c7d 6717 1 2024-03-29T08:07:27.460 INFO:tasks.workunit.client.1.smithi148.stdout:3/834: write de/d24/d33/d39/d46/f23 [1402529,24574] 0 2024-03-29T08:07:27.461 INFO:tasks.workunit.client.1.smithi148.stdout:7/744: dread d8/dfb/f2e [0,4194304] 0 2024-03-29T08:07:27.461 INFO:tasks.workunit.client.1.smithi148.stdout:1/728: symlink d3/d18/d71/le6 0 2024-03-29T08:07:27.463 INFO:tasks.workunit.client.1.smithi148.stdout:5/777: fsync d0/db/d7b/d16/d2e/fa0 0 2024-03-29T08:07:27.464 INFO:tasks.workunit.client.0.smithi037.stdout:5/681: write d8/d22/d4f/d70/d79/f92 [2541,7324] 0 2024-03-29T08:07:27.467 INFO:tasks.workunit.client.1.smithi148.stdout:0/783: chown db/dc/daa/d47/d9f/fa5 6086869 1 2024-03-29T08:07:27.468 INFO:tasks.workunit.client.0.smithi037.stdout:0/437: write d9/da/dd/d13/f21 [1466294,60146] 0 2024-03-29T08:07:27.468 INFO:tasks.workunit.client.1.smithi148.stdout:3/835: fsync de/d24/d33/d39/d46/d30/d37/fea 0 2024-03-29T08:07:27.469 INFO:tasks.workunit.client.1.smithi148.stdout:5/778: chown d0/db/d1e/d5b/d66/d73 134 1 2024-03-29T08:07:27.471 INFO:tasks.workunit.client.1.smithi148.stdout:1/729: chown d3/de/d21/d5f/fa3 127046925 1 2024-03-29T08:07:27.473 INFO:tasks.workunit.client.1.smithi148.stdout:7/745: write d8/dfb/d44/d49/f76 [5481380,4508] 0 2024-03-29T08:07:27.474 INFO:tasks.workunit.client.1.smithi148.stdout:9/816: creat db/d10/d2d/d4e/d5f/dd5/f112 x:0 0 0 2024-03-29T08:07:27.475 INFO:tasks.workunit.client.0.smithi037.stdout:0/438: creat d9/da/dd/d20/d38/d53/f8f x:0 0 0 2024-03-29T08:07:27.476 INFO:tasks.workunit.client.1.smithi148.stdout:4/797: mkdir df/d2c/d51/d77/dfe 0 2024-03-29T08:07:27.476 INFO:tasks.workunit.client.1.smithi148.stdout:0/784: write db/de/df/d2b/f77 [7668549,87968] 0 2024-03-29T08:07:27.476 INFO:tasks.workunit.client.1.smithi148.stdout:2/870: link d7/d8c/da4/d8e/dda/c118 d7/d24/d4b/d66/df1/c11b 0 2024-03-29T08:07:27.482 INFO:tasks.workunit.client.0.smithi037.stdout:2/414: truncate d0/d23/d44/f46 617004 0 2024-03-29T08:07:27.483 INFO:tasks.workunit.client.1.smithi148.stdout:1/730: dread - d3/d18/d71/fa7 zero size 2024-03-29T08:07:27.483 INFO:tasks.workunit.client.1.smithi148.stdout:7/746: chown d8/d14/da8/ccd 1256 1 2024-03-29T08:07:27.483 INFO:tasks.workunit.client.1.smithi148.stdout:6/786: dwrite d6/d9/f52 [0,4194304] 0 2024-03-29T08:07:27.483 INFO:tasks.workunit.client.0.smithi037.stdout:5/682: truncate d8/d18/f28 391095 0 2024-03-29T08:07:27.488 INFO:tasks.workunit.client.1.smithi148.stdout:2/871: write d7/da/d33/ded/d2e/f50 [3527021,81529] 0 2024-03-29T08:07:27.488 INFO:tasks.workunit.client.1.smithi148.stdout:9/817: mkdir db/d10/d2d/d4e/d113 0 2024-03-29T08:07:27.489 INFO:tasks.workunit.client.1.smithi148.stdout:0/785: read db/dc/d78/f8b [3926231,78785] 0 2024-03-29T08:07:27.489 INFO:tasks.workunit.client.1.smithi148.stdout:1/731: readlink d3/de/d1d/d47/lbc 0 2024-03-29T08:07:27.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:27.490+0000 7f35c4400700 1 -- 172.21.15.37:0/282127810 --> [v2:172.21.15.148:6828/1134897184,v1:172.21.15.148:6829/1134897184] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f35bc02b450 con 0x7f35a406af80 2024-03-29T08:07:27.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:27.494+0000 7f35b9600700 1 -- 172.21.15.37:0/282127810 <== mgr.14602 v2:172.21.15.148:6828/1134897184 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+341 (secure 0 0 0) 0x7f35bc02b450 con 0x7f35a406af80 2024-03-29T08:07:27.494 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-29T08:07:27.494 INFO:teuthology.orchestra.run.smithi037.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59", 2024-03-29T08:07:27.494 INFO:teuthology.orchestra.run.smithi037.stdout: "in_progress": true, 2024-03-29T08:07:27.494 INFO:teuthology.orchestra.run.smithi037.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-03-29T08:07:27.495 INFO:teuthology.orchestra.run.smithi037.stdout: "services_complete": [], 2024-03-29T08:07:27.495 INFO:teuthology.orchestra.run.smithi037.stdout: "progress": "1/2 daemons upgraded", 2024-03-29T08:07:27.495 INFO:teuthology.orchestra.run.smithi037.stdout: "message": "Currently upgrading mgr daemons", 2024-03-29T08:07:27.495 INFO:teuthology.orchestra.run.smithi037.stdout: "is_paused": false 2024-03-29T08:07:27.495 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-29T08:07:27.495 INFO:tasks.workunit.client.1.smithi148.stdout:7/747: symlink d8/d38/l105 0 2024-03-29T08:07:27.496 INFO:tasks.workunit.client.1.smithi148.stdout:6/787: write d6/d9/d1e/d36/d4d/d9d/d28/db3/dd2/fe7 [466969,122978] 0 2024-03-29T08:07:27.498 INFO:tasks.workunit.client.0.smithi037.stdout:0/439: dwrite d9/da/f8d [0,4194304] 0 2024-03-29T08:07:27.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:27.498+0000 7f35c4400700 1 -- 172.21.15.37:0/282127810 >> [v2:172.21.15.148:6828/1134897184,v1:172.21.15.148:6829/1134897184] conn(0x7f35a406af80 msgr2=0x7f35a406d430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:07:27.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:27.498+0000 7f35c4400700 1 --2- 172.21.15.37:0/282127810 >> [v2:172.21.15.148:6828/1134897184,v1:172.21.15.148:6829/1134897184] conn(0x7f35a406af80 0x7f35a406d430 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f35b0008fd0 tx=0x7f35b000b040 comp rx=0 tx=0).stop 2024-03-29T08:07:27.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:27.498+0000 7f35c4400700 1 -- 172.21.15.37:0/282127810 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f35bc114550 msgr2=0x7f35bc119510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:07:27.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:27.498+0000 7f35c4400700 1 --2- 172.21.15.37:0/282127810 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f35bc114550 0x7f35bc119510 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f35a8008000 tx=0x7f35a8009e30 comp rx=0 tx=0).stop 2024-03-29T08:07:27.499 INFO:tasks.workunit.client.1.smithi148.stdout:6/788: fsync d6/fe 0 2024-03-29T08:07:27.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:27.499+0000 7f35c4400700 1 -- 172.21.15.37:0/282127810 shutdown_connections 2024-03-29T08:07:27.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:27.499+0000 7f35c4400700 1 --2- 172.21.15.37:0/282127810 >> [v2:172.21.15.148:6828/1134897184,v1:172.21.15.148:6829/1134897184] conn(0x7f35a406af80 0x7f35a406d430 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:07:27.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:27.499+0000 7f35c4400700 1 --2- 172.21.15.37:0/282127810 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f35bc0fce70 0x7f35bc114010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:07:27.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:27.499+0000 7f35c4400700 1 --2- 172.21.15.37:0/282127810 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f35bc114550 0x7f35bc119510 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:07:27.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:27.499+0000 7f35c4400700 1 -- 172.21.15.37:0/282127810 >> 172.21.15.37:0/282127810 conn(0x7f35bc0f8510 msgr2=0x7f35bc0f9d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:07:27.502 INFO:tasks.workunit.client.1.smithi148.stdout:0/786: mkdir db/de/df/d11/d2d/da3/dee 0 2024-03-29T08:07:27.502 INFO:tasks.workunit.client.0.smithi037.stdout:2/415: dread d0/fd [0,4194304] 0 2024-03-29T08:07:27.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:27.503+0000 7f35c4400700 1 -- 172.21.15.37:0/282127810 shutdown_connections 2024-03-29T08:07:27.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:27.503+0000 7f35c4400700 1 -- 172.21.15.37:0/282127810 wait complete. 2024-03-29T08:07:27.504 INFO:tasks.workunit.client.0.smithi037.stdout:0/440: truncate d9/da/f8d 4499683 0 2024-03-29T08:07:27.505 INFO:tasks.workunit.client.0.smithi037.stdout:2/416: dread - d0/d23/d70/f7c zero size 2024-03-29T08:07:27.505 INFO:tasks.workunit.client.1.smithi148.stdout:7/748: mkdir d8/dfb/d44/d50/d106 0 2024-03-29T08:07:27.509 INFO:tasks.workunit.client.1.smithi148.stdout:6/789: creat d6/d9/d1e/d36/d4d/d9d/d28/d98/dcc/ffe x:0 0 0 2024-03-29T08:07:27.510 INFO:tasks.workunit.client.0.smithi037.stdout:0/441: chown d9/da/d1b/d22/l52 0 1 2024-03-29T08:07:27.515 INFO:tasks.workunit.client.1.smithi148.stdout:5/779: dwrite d0/db/d1e/d35/d42/d67/d9a/dd7/fed [0,4194304] 0 2024-03-29T08:07:27.516 INFO:tasks.workunit.client.0.smithi037.stdout:0/442: mknod d9/da/d1b/d22/d2c/c90 0 2024-03-29T08:07:27.520 INFO:tasks.workunit.client.1.smithi148.stdout:9/818: rename db/d22/d26/d54/d4d/d35/d4b/fc8 to db/d10/d2d/d4e/d5f/f114 0 2024-03-29T08:07:27.521 INFO:tasks.workunit.client.0.smithi037.stdout:0/443: symlink d9/da/dd/d59/l91 0 2024-03-29T08:07:27.521 INFO:tasks.workunit.client.0.smithi037.stdout:2/417: link d0/d23/d70/d71/f72 d0/d20/d2e/d5f/f8d 0 2024-03-29T08:07:27.521 INFO:tasks.workunit.client.1.smithi148.stdout:3/836: dwrite de/d24/d33/f9a [0,4194304] 0 2024-03-29T08:07:27.524 INFO:tasks.workunit.client.1.smithi148.stdout:1/732: dwrite d3/d55/f37 [4194304,4194304] 0 2024-03-29T08:07:27.525 INFO:tasks.workunit.client.0.smithi037.stdout:2/418: dread - d0/d20/d7d/f80 zero size 2024-03-29T08:07:27.527 INFO:tasks.workunit.client.1.smithi148.stdout:7/749: truncate d8/d38/d3c/da9/d37/dc8/ff8 218841 0 2024-03-29T08:07:27.527 INFO:tasks.workunit.client.0.smithi037.stdout:0/444: read d9/da/dd/d20/d38/f69 [9718,57694] 0 2024-03-29T08:07:27.527 INFO:tasks.workunit.client.1.smithi148.stdout:9/819: chown db/d22/d26/d54/d4d/d35/c57 117 1 2024-03-29T08:07:27.528 INFO:tasks.workunit.client.1.smithi148.stdout:3/837: creat de/f110 x:0 0 0 2024-03-29T08:07:27.529 INFO:tasks.workunit.client.1.smithi148.stdout:1/733: chown d3/d55/f43 215724539 1 2024-03-29T08:07:27.529 INFO:tasks.workunit.client.0.smithi037.stdout:2/419: chown d0/d20/d2e/f36 6969 1 2024-03-29T08:07:27.531 INFO:tasks.workunit.client.1.smithi148.stdout:5/780: dread d0/db/d1e/d6e/f92 [0,4194304] 0 2024-03-29T08:07:27.533 INFO:tasks.workunit.client.1.smithi148.stdout:1/734: write d3/d18/d71/f8e [2239989,54616] 0 2024-03-29T08:07:27.534 INFO:tasks.workunit.client.1.smithi148.stdout:6/790: creat d6/d9/d1e/d27/d48/d73/fff x:0 0 0 2024-03-29T08:07:27.536 INFO:tasks.workunit.client.1.smithi148.stdout:9/820: chown db/d22/d26/d54/d52/d9c/cf3 827543 1 2024-03-29T08:07:27.537 INFO:tasks.workunit.client.1.smithi148.stdout:0/787: rename db/dc/daa/d47/d7c/l9c to db/de/df/d11/d1f/dc5/lef 0 2024-03-29T08:07:27.538 INFO:tasks.workunit.client.1.smithi148.stdout:1/735: truncate d3/d18/d4d/d8f/fcf 1014800 0 2024-03-29T08:07:27.539 INFO:tasks.workunit.client.1.smithi148.stdout:3/838: fdatasync de/d24/d33/d39/da3/dbc/fd0 0 2024-03-29T08:07:27.543 INFO:tasks.workunit.client.1.smithi148.stdout:6/791: creat d6/d9/d1e/d20/d30/f100 x:0 0 0 2024-03-29T08:07:27.543 INFO:tasks.workunit.client.0.smithi037.stdout:0/445: mkdir d9/da/dd/d54/d92 0 2024-03-29T08:07:27.544 INFO:tasks.workunit.client.1.smithi148.stdout:9/821: write db/d22/d26/d54/d4d/d35/d4b/d58/f92 [1355922,87881] 0 2024-03-29T08:07:27.546 INFO:tasks.workunit.client.1.smithi148.stdout:7/750: symlink d8/l107 0 2024-03-29T08:07:27.547 INFO:tasks.workunit.client.1.smithi148.stdout:0/788: mknod db/de/df/d11/d2d/dde/cf0 0 2024-03-29T08:07:27.547 INFO:tasks.workunit.client.1.smithi148.stdout:5/781: creat d0/db/d1e/d36/de6/ffd x:0 0 0 2024-03-29T08:07:27.547 INFO:tasks.workunit.client.1.smithi148.stdout:9/822: readlink db/d22/l3e 0 2024-03-29T08:07:27.549 INFO:tasks.workunit.client.1.smithi148.stdout:3/839: readlink de/d24/d33/d39/d46/d70/def/lf8 0 2024-03-29T08:07:27.549 INFO:tasks.workunit.client.0.smithi037.stdout:0/446: mknod d9/da/d1b/d22/c93 0 2024-03-29T08:07:27.550 INFO:tasks.workunit.client.1.smithi148.stdout:2/872: dread d7/d24/f2c [0,4194304] 0 2024-03-29T08:07:27.550 INFO:tasks.workunit.client.1.smithi148.stdout:0/789: write fa [3775403,99933] 0 2024-03-29T08:07:27.552 INFO:tasks.workunit.client.1.smithi148.stdout:7/751: creat d8/dfb/f108 x:0 0 0 2024-03-29T08:07:27.553 INFO:tasks.workunit.client.0.smithi037.stdout:2/420: dwrite d0/d23/f30 [4194304,4194304] 0 2024-03-29T08:07:27.554 INFO:tasks.workunit.client.1.smithi148.stdout:7/752: chown d8/dfb/db8 538680 1 2024-03-29T08:07:27.555 INFO:tasks.workunit.client.1.smithi148.stdout:0/790: readlink db/de/d24/l32 0 2024-03-29T08:07:27.555 INFO:tasks.workunit.client.0.smithi037.stdout:0/447: creat d9/d74/d7d/f94 x:0 0 0 2024-03-29T08:07:27.556 INFO:tasks.workunit.client.1.smithi148.stdout:7/753: creat d8/dfb/f109 x:0 0 0 2024-03-29T08:07:27.557 INFO:tasks.workunit.client.1.smithi148.stdout:0/791: mkdir db/de/d8e/df1 0 2024-03-29T08:07:27.557 INFO:tasks.workunit.client.1.smithi148.stdout:5/782: mknod d0/db/d7b/d16/d2e/ddd/de9/cfe 0 2024-03-29T08:07:27.560 INFO:tasks.workunit.client.1.smithi148.stdout:0/792: creat db/dc/daa/ff2 x:0 0 0 2024-03-29T08:07:27.562 INFO:tasks.workunit.client.1.smithi148.stdout:3/840: fsync de/d24/d33/d39/d46/d30/ff4 0 2024-03-29T08:07:27.565 INFO:tasks.workunit.client.0.smithi037.stdout:0/448: rename d9/da/dd/d13/d83/f49 to d9/da/dd/d13/f95 0 2024-03-29T08:07:27.566 INFO:tasks.workunit.client.1.smithi148.stdout:2/873: rename d7/da/d33/ded/d82 to d7/da/dcc/d11c 0 2024-03-29T08:07:27.568 INFO:tasks.workunit.client.1.smithi148.stdout:2/874: dread - d7/d8c/da4/d8e/fef zero size 2024-03-29T08:07:27.569 INFO:tasks.workunit.client.1.smithi148.stdout:8/768: sync 2024-03-29T08:07:27.570 INFO:tasks.workunit.client.1.smithi148.stdout:7/754: link d8/c1d d8/d38/d3c/da5/c10a 0 2024-03-29T08:07:27.570 INFO:tasks.workunit.client.1.smithi148.stdout:1/736: write d3/d55/f43 [2580132,754] 0 2024-03-29T08:07:27.571 INFO:tasks.workunit.client.0.smithi037.stdout:2/421: dread d0/fd [0,4194304] 0 2024-03-29T08:07:27.573 INFO:tasks.workunit.client.1.smithi148.stdout:0/793: read db/de/df/d11/d42/f52 [2972068,81667] 0 2024-03-29T08:07:27.575 INFO:tasks.workunit.client.1.smithi148.stdout:8/769: creat d17/d18/d26/d63/d7d/f101 x:0 0 0 2024-03-29T08:07:27.575 INFO:tasks.workunit.client.1.smithi148.stdout:1/737: write d3/de/d25/d51/f73 [863833,26] 0 2024-03-29T08:07:27.576 INFO:tasks.workunit.client.1.smithi148.stdout:7/755: stat d8/dfb/d44/d50/c95 0 2024-03-29T08:07:27.576 INFO:tasks.workunit.client.1.smithi148.stdout:5/783: getdents d0/db/d1e/d5b/d66/d72 0 2024-03-29T08:07:27.578 INFO:tasks.workunit.client.0.smithi037.stdout:2/422: creat d0/f8e x:0 0 0 2024-03-29T08:07:27.578 INFO:tasks.workunit.client.1.smithi148.stdout:8/770: chown d17/d74/d86/fe5 28294338 1 2024-03-29T08:07:27.579 INFO:tasks.workunit.client.1.smithi148.stdout:7/756: write d8/dfb/f54 [6850193,86929] 0 2024-03-29T08:07:27.582 INFO:tasks.workunit.client.1.smithi148.stdout:1/738: mkdir d3/d18/dae/de7 0 2024-03-29T08:07:27.582 INFO:tasks.workunit.client.1.smithi148.stdout:2/875: rename d7/d24/d4b/l76 to d7/da/d4c/dc4/dc6/d10f/l11d 0 2024-03-29T08:07:27.584 INFO:tasks.workunit.client.0.smithi037.stdout:2/423: creat d0/d20/d7d/f8f x:0 0 0 2024-03-29T08:07:27.585 INFO:tasks.workunit.client.1.smithi148.stdout:8/771: fsync d17/d1d/f92 0 2024-03-29T08:07:27.587 INFO:tasks.workunit.client.1.smithi148.stdout:1/739: stat d3/de/d21/d7f 0 2024-03-29T08:07:27.587 INFO:tasks.workunit.client.1.smithi148.stdout:2/876: dread - d7/da/d33/d3c/d31/faf zero size 2024-03-29T08:07:27.588 INFO:tasks.workunit.client.0.smithi037.stdout:2/424: symlink d0/d20/d22/l90 0 2024-03-29T08:07:27.589 INFO:tasks.workunit.client.1.smithi148.stdout:2/877: chown d7/da/d4c/dd1 40424740 1 2024-03-29T08:07:27.589 INFO:tasks.workunit.client.1.smithi148.stdout:7/757: mknod d8/d14/d45/d4d/c10b 0 2024-03-29T08:07:27.591 INFO:tasks.workunit.client.1.smithi148.stdout:9/823: write db/d22/d26/d54/d4d/d35/d4b/fb4 [837597,4297] 0 2024-03-29T08:07:27.594 INFO:tasks.workunit.client.1.smithi148.stdout:8/772: unlink d17/d1d/d7e/f83 0 2024-03-29T08:07:27.595 INFO:tasks.workunit.client.1.smithi148.stdout:2/878: mkdir d7/da/d4c/dc4/dc6/d112/d11e 0 2024-03-29T08:07:27.595 INFO:tasks.workunit.client.1.smithi148.stdout:1/740: mkdir d3/de/d25/de8 0 2024-03-29T08:07:27.596 INFO:tasks.workunit.client.1.smithi148.stdout:9/824: mknod db/d10/d9d/c115 0 2024-03-29T08:07:27.599 INFO:tasks.workunit.client.1.smithi148.stdout:9/825: chown db/d22/d3f/ce3 16291521 1 2024-03-29T08:07:27.601 INFO:tasks.workunit.client.1.smithi148.stdout:2/879: unlink d7/da/d4c/dc4/cee 0 2024-03-29T08:07:27.601 INFO:tasks.workunit.client.1.smithi148.stdout:7/758: readlink d8/d38/d67/l88 0 2024-03-29T08:07:27.601 INFO:tasks.workunit.client.0.smithi037.stdout:2/425: link d0/d20/d22/f47 d0/d20/d2e/f91 0 2024-03-29T08:07:27.603 INFO:tasks.workunit.client.1.smithi148.stdout:8/773: readlink d17/d18/d26/d63/d66/lc9 0 2024-03-29T08:07:27.603 INFO:tasks.workunit.client.0.smithi037.stdout:0/449: dread d9/ff [0,4194304] 0 2024-03-29T08:07:27.606 INFO:tasks.workunit.client.0.smithi037.stdout:2/426: rename d0/d20/d22/d4e/d84 to d0/d20/d22/d55/d92 0 2024-03-29T08:07:27.607 INFO:tasks.workunit.client.1.smithi148.stdout:0/794: dread db/de/f4b [4194304,4194304] 0 2024-03-29T08:07:27.607 INFO:tasks.workunit.client.1.smithi148.stdout:1/741: truncate d3/de/d1d/d47/d6d/d84/d8c/fce 713565 0 2024-03-29T08:07:27.607 INFO:tasks.workunit.client.0.smithi037.stdout:0/450: creat d9/da/dd/d13/f96 x:0 0 0 2024-03-29T08:07:27.608 INFO:tasks.workunit.client.1.smithi148.stdout:8/774: write d17/d18/ddf/fe1 [736536,27548] 0 2024-03-29T08:07:27.608 INFO:tasks.workunit.client.0.smithi037.stdout:2/427: write d0/d20/d22/f39 [990064,48845] 0 2024-03-29T08:07:27.610 INFO:tasks.workunit.client.0.smithi037.stdout:0/451: unlink f7 0 2024-03-29T08:07:27.611 INFO:tasks.workunit.client.1.smithi148.stdout:5/784: dwrite d0/db/d1e/d5b/d66/f74 [0,4194304] 0 2024-03-29T08:07:27.611 INFO:tasks.workunit.client.1.smithi148.stdout:9/826: truncate db/d22/d26/d54/d4d/d35/d4b/dc3/fb5 512537 0 2024-03-29T08:07:27.611 INFO:tasks.workunit.client.1.smithi148.stdout:8/775: chown d17/d18/ddf/fe1 1374508 1 2024-03-29T08:07:27.619 INFO:tasks.workunit.client.0.smithi037.stdout:3/507: sync 2024-03-29T08:07:27.619 INFO:tasks.workunit.client.0.smithi037.stdout:8/529: sync 2024-03-29T08:07:27.619 INFO:tasks.workunit.client.0.smithi037.stdout:4/546: sync 2024-03-29T08:07:27.620 INFO:tasks.workunit.client.1.smithi148.stdout:8/776: rmdir d17/d49/d9a/dbe 39 2024-03-29T08:07:27.626 INFO:tasks.workunit.client.0.smithi037.stdout:4/547: chown d3/c5 72490715 1 2024-03-29T08:07:27.626 INFO:tasks.workunit.client.0.smithi037.stdout:8/530: write d5/d10/d2d/d34/d50/fad [531124,36601] 0 2024-03-29T08:07:27.627 INFO:tasks.workunit.client.1.smithi148.stdout:5/785: creat d0/db/d1e/d35/d42/d67/d76/d81/fff x:0 0 0 2024-03-29T08:07:27.629 INFO:tasks.workunit.client.1.smithi148.stdout:9/827: truncate db/d10/d2d/d4e/dd4/f70 2590413 0 2024-03-29T08:07:27.631 INFO:tasks.workunit.client.0.smithi037.stdout:3/508: write dd/d46/d2d/f42 [832433,115064] 0 2024-03-29T08:07:27.632 INFO:tasks.workunit.client.0.smithi037.stdout:2/428: dwrite d0/d20/d22/f39 [0,4194304] 0 2024-03-29T08:07:27.632 INFO:tasks.workunit.client.1.smithi148.stdout:9/828: readlink db/d10/l23 0 2024-03-29T08:07:27.634 INFO:tasks.workunit.client.0.smithi037.stdout:4/548: rename d3/d4/d15/l4d to d3/d4/d15/d1d/d4f/lb1 0 2024-03-29T08:07:27.634 INFO:tasks.workunit.client.1.smithi148.stdout:2/880: dwrite d7/da/d33/ded/f16 [0,4194304] 0 2024-03-29T08:07:27.635 INFO:tasks.workunit.client.1.smithi148.stdout:5/786: dread - d0/db/d1e/d35/d42/d96/fdf zero size 2024-03-29T08:07:27.639 INFO:tasks.workunit.client.1.smithi148.stdout:7/759: dwrite d8/f18 [8388608,4194304] 0 2024-03-29T08:07:27.642 INFO:tasks.workunit.client.0.smithi037.stdout:8/531: getdents d5/d10/d45 0 2024-03-29T08:07:27.644 INFO:tasks.workunit.client.0.smithi037.stdout:3/509: truncate dd/d46/d2d/d3a/f52 1312230 0 2024-03-29T08:07:27.644 INFO:tasks.workunit.client.0.smithi037.stdout:4/549: getdents d3/d80 0 2024-03-29T08:07:27.645 INFO:tasks.workunit.client.1.smithi148.stdout:7/760: dread - d8/d38/d3c/da9/ffe zero size 2024-03-29T08:07:27.650 INFO:tasks.workunit.client.1.smithi148.stdout:0/795: dread db/f51 [0,4194304] 0 2024-03-29T08:07:27.651 INFO:tasks.workunit.client.0.smithi037.stdout:3/510: creat dd/d15/d27/d33/d75/fab x:0 0 0 2024-03-29T08:07:27.651 INFO:tasks.workunit.client.0.smithi037.stdout:8/532: dread - d5/d10/d19/d3d/d4a/d4c/f6e zero size 2024-03-29T08:07:27.653 INFO:tasks.workunit.client.0.smithi037.stdout:8/533: chown d5/d10/d2d/d34 0 1 2024-03-29T08:07:27.657 INFO:tasks.workunit.client.1.smithi148.stdout:1/742: dread d3/d18/f20 [0,4194304] 0 2024-03-29T08:07:27.657 INFO:tasks.workunit.client.1.smithi148.stdout:8/777: dwrite d17/d18/d26/d4f/d94/fe3 [0,4194304] 0 2024-03-29T08:07:27.660 INFO:tasks.workunit.client.0.smithi037.stdout:4/550: chown d3/d4/d15/d1d/db 950860709 1 2024-03-29T08:07:27.660 INFO:tasks.workunit.client.0.smithi037.stdout:8/534: dread - d5/d10/d2d/f40 zero size 2024-03-29T08:07:27.661 INFO:tasks.workunit.client.1.smithi148.stdout:7/761: rename d8/d14/fc7 to d8/d38/d6a/d91/dff/f10c 0 2024-03-29T08:07:27.661 INFO:tasks.workunit.client.1.smithi148.stdout:1/743: readlink d3/de/d1d/d47/d6d/d84/lcc 0 2024-03-29T08:07:27.664 INFO:tasks.workunit.client.0.smithi037.stdout:3/511: rename dd/d15/c28 to dd/d15/d27/d33/d83/cac 0 2024-03-29T08:07:27.666 INFO:tasks.workunit.client.1.smithi148.stdout:7/762: stat d8/d14/f2b 0 2024-03-29T08:07:27.667 INFO:tasks.workunit.client.0.smithi037.stdout:2/429: dwrite d0/f14 [0,4194304] 0 2024-03-29T08:07:27.668 INFO:tasks.workunit.client.1.smithi148.stdout:8/778: unlink d17/d49/f9c 0 2024-03-29T08:07:27.670 INFO:tasks.workunit.client.1.smithi148.stdout:5/787: dwrite d0/db/d7b/d16/d2e/fa0 [0,4194304] 0 2024-03-29T08:07:27.671 INFO:tasks.workunit.client.1.smithi148.stdout:4/798: sync 2024-03-29T08:07:27.673 INFO:tasks.workunit.client.1.smithi148.stdout:9/829: dwrite db/d22/d26/d54/d4d/d35/f5e [0,4194304] 0 2024-03-29T08:07:27.674 INFO:tasks.workunit.client.0.smithi037.stdout:8/535: creat d5/d10/d19/d3d/d4a/d4c/d67/faf x:0 0 0 2024-03-29T08:07:27.676 INFO:tasks.workunit.client.0.smithi037.stdout:2/430: creat d0/d20/d2e/d38/f93 x:0 0 0 2024-03-29T08:07:27.686 INFO:tasks.workunit.client.0.smithi037.stdout:2/431: rename d0/d20/d2e/f82 to d0/d20/d2e/d38/f94 0 2024-03-29T08:07:27.688 INFO:tasks.workunit.client.1.smithi148.stdout:7/763: rename d8/d14/d45/c101 to d8/d38/d6a/c10d 0 2024-03-29T08:07:27.690 INFO:tasks.workunit.client.1.smithi148.stdout:1/744: dread d3/d18/da8/fab [0,4194304] 0 2024-03-29T08:07:27.696 INFO:tasks.workunit.client.1.smithi148.stdout:9/830: truncate db/d22/d26/d54/d4d/d35/d4b/d5a/f77 1330537 0 2024-03-29T08:07:27.696 INFO:tasks.workunit.client.0.smithi037.stdout:4/551: write d3/d54/f5b [1418730,77143] 0 2024-03-29T08:07:27.696 INFO:tasks.workunit.client.1.smithi148.stdout:8/779: getdents d17/d49/d9a/dbe/d6f/d64/da5 0 2024-03-29T08:07:27.696 INFO:tasks.workunit.client.1.smithi148.stdout:0/796: write db/de/d24/f38 [347567,73011] 0 2024-03-29T08:07:27.700 INFO:tasks.workunit.client.0.smithi037.stdout:2/432: link d0/d20/d2e/c32 d0/d20/d22/d55/d92/c95 0 2024-03-29T08:07:27.702 INFO:tasks.workunit.client.0.smithi037.stdout:3/512: fdatasync dd/d15/d27/d33/d75/f8c 0 2024-03-29T08:07:27.704 INFO:tasks.workunit.client.0.smithi037.stdout:4/552: mkdir d3/d4/d15/d1d/db/d45/d93/d96/da0/db2 0 2024-03-29T08:07:27.706 INFO:tasks.workunit.client.1.smithi148.stdout:5/788: dwrite d0/db/d1e/d36/f47 [0,4194304] 0 2024-03-29T08:07:27.708 INFO:tasks.workunit.client.1.smithi148.stdout:9/831: getdents db/d10/d6d/d73/da8/deb/dec/dde 0 2024-03-29T08:07:27.712 INFO:tasks.workunit.client.1.smithi148.stdout:8/780: truncate d17/d18/f8a 328873 0 2024-03-29T08:07:27.717 INFO:tasks.workunit.client.0.smithi037.stdout:8/536: dwrite d5/d10/d2d/d34/d50/d83/f28 [0,4194304] 0 2024-03-29T08:07:27.717 INFO:tasks.workunit.client.1.smithi148.stdout:0/797: creat db/de/d8e/df1/ff3 x:0 0 0 2024-03-29T08:07:27.720 INFO:tasks.workunit.client.1.smithi148.stdout:4/799: truncate df/d2c/d51/d77/d17/f42 5707206 0 2024-03-29T08:07:27.721 INFO:tasks.workunit.client.0.smithi037.stdout:3/513: creat dd/d15/d18/d4b/fad x:0 0 0 2024-03-29T08:07:27.721 INFO:tasks.workunit.client.1.smithi148.stdout:0/798: chown db/dc/d78/f8b 12372559 1 2024-03-29T08:07:27.722 INFO:tasks.workunit.client.1.smithi148.stdout:5/789: dwrite d0/db/d84/daf/fc2 [0,4194304] 0 2024-03-29T08:07:27.725 INFO:tasks.workunit.client.1.smithi148.stdout:1/745: write d3/d18/d4d/f63 [4414534,71689] 0 2024-03-29T08:07:27.725 INFO:tasks.workunit.client.1.smithi148.stdout:7/764: link d8/d38/d3c/da9/l3a d8/d38/d6a/d91/dff/l10e 0 2024-03-29T08:07:27.726 INFO:tasks.workunit.client.0.smithi037.stdout:2/433: link d0/d20/l25 d0/d23/d44/l96 0 2024-03-29T08:07:27.726 INFO:tasks.workunit.client.0.smithi037.stdout:3/514: truncate dd/d46/d3e/d47/f95 307803 0 2024-03-29T08:07:27.727 INFO:tasks.workunit.client.1.smithi148.stdout:9/832: truncate db/d22/d26/d54/d4d/f18 1247507 0 2024-03-29T08:07:27.728 INFO:tasks.workunit.client.0.smithi037.stdout:8/537: truncate d5/d10/d19/d3d/f5d 1937032 0 2024-03-29T08:07:27.728 INFO:tasks.workunit.client.1.smithi148.stdout:8/781: mknod d17/d18/d26/d4f/dde/c102 0 2024-03-29T08:07:27.729 INFO:tasks.workunit.client.0.smithi037.stdout:3/515: chown dd/d46/d2d 112 1 2024-03-29T08:07:27.731 INFO:tasks.workunit.client.0.smithi037.stdout:2/434: write d0/d20/d2e/d38/f94 [17912,124619] 0 2024-03-29T08:07:27.732 INFO:tasks.workunit.client.1.smithi148.stdout:4/800: unlink df/d2c/d51/d77/d26/d34/d3e/f75 0 2024-03-29T08:07:27.733 INFO:tasks.workunit.client.0.smithi037.stdout:3/516: chown dd/d15/f49 414032 1 2024-03-29T08:07:27.733 INFO:tasks.workunit.client.0.smithi037.stdout:8/538: chown d5/d10/d2d/d34/l6d 29528 1 2024-03-29T08:07:27.734 INFO:tasks.workunit.client.1.smithi148.stdout:8/782: chown d17/d49/d9a/dbe/d6f/d64/ld1 15516338 1 2024-03-29T08:07:27.736 INFO:tasks.workunit.client.0.smithi037.stdout:3/517: stat dd/d5e/f6d 0 2024-03-29T08:07:27.736 INFO:tasks.workunit.client.1.smithi148.stdout:1/746: unlink d3/d18/d4d/d9e/l83 0 2024-03-29T08:07:27.739 INFO:tasks.workunit.client.1.smithi148.stdout:7/765: truncate d8/dfb/d44/d50/f87 496944 0 2024-03-29T08:07:27.740 INFO:tasks.workunit.client.0.smithi037.stdout:8/539: write d5/d10/d19/d8d/f97 [4043590,100855] 0 2024-03-29T08:07:27.743 INFO:tasks.workunit.client.1.smithi148.stdout:8/783: mkdir d17/d74/d103 0 2024-03-29T08:07:27.745 INFO:tasks.workunit.client.1.smithi148.stdout:8/784: truncate fe 4571384 0 2024-03-29T08:07:27.745 INFO:tasks.workunit.client.1.smithi148.stdout:9/833: truncate db/d22/d26/f10e 1112577 0 2024-03-29T08:07:27.747 INFO:tasks.workunit.client.1.smithi148.stdout:9/834: readlink db/d10/l101 0 2024-03-29T08:07:27.749 INFO:tasks.workunit.client.1.smithi148.stdout:7/766: mknod d8/d38/d3c/da5/db4/c10f 0 2024-03-29T08:07:27.751 INFO:tasks.workunit.client.1.smithi148.stdout:4/801: rename df/d2c/d80/lae to df/d2c/d51/d77/d26/d34/d3e/df3/lff 0 2024-03-29T08:07:27.752 INFO:tasks.workunit.client.1.smithi148.stdout:5/790: write d0/db/d1e/fa7 [639983,5389] 0 2024-03-29T08:07:27.754 INFO:tasks.workunit.client.1.smithi148.stdout:7/767: dread - d8/dfb/fba zero size 2024-03-29T08:07:27.757 INFO:tasks.workunit.client.0.smithi037.stdout:8/540: write d5/f5b [1486717,111235] 0 2024-03-29T08:07:27.758 INFO:tasks.workunit.client.1.smithi148.stdout:5/791: creat d0/db/d1e/d6e/da8/f100 x:0 0 0 2024-03-29T08:07:27.760 INFO:tasks.workunit.client.1.smithi148.stdout:1/747: write d3/de/d25/d35/d42/f1a [5268697,87559] 0 2024-03-29T08:07:27.760 INFO:tasks.workunit.client.1.smithi148.stdout:7/768: write d8/d38/d3c/da9/ffe [477874,77713] 0 2024-03-29T08:07:27.761 INFO:tasks.workunit.client.0.smithi037.stdout:4/553: dwrite d3/d4/d15/d1d/f3f [0,4194304] 0 2024-03-29T08:07:27.763 INFO:tasks.workunit.client.1.smithi148.stdout:4/802: mkdir df/d2c/d51/d77/d26/d34/d43/df6/d100 0 2024-03-29T08:07:27.764 INFO:tasks.workunit.client.1.smithi148.stdout:7/769: readlink d8/dfb/l3b 0 2024-03-29T08:07:27.766 INFO:tasks.workunit.client.1.smithi148.stdout:7/770: read - d8/dfb/f109 zero size 2024-03-29T08:07:27.766 INFO:tasks.workunit.client.0.smithi037.stdout:2/435: dwrite d0/f14 [0,4194304] 0 2024-03-29T08:07:27.769 INFO:tasks.workunit.client.1.smithi148.stdout:8/785: rename d17/d18/d26/c69 to d17/d49/d9a/dbe/d6f/c104 0 2024-03-29T08:07:27.769 INFO:tasks.workunit.client.1.smithi148.stdout:1/748: getdents d3/d18/d4d/d76/d85 0 2024-03-29T08:07:27.774 INFO:tasks.workunit.client.0.smithi037.stdout:3/518: write dd/d15/f44 [915238,35688] 0 2024-03-29T08:07:27.777 INFO:tasks.workunit.client.1.smithi148.stdout:8/786: mknod d17/dce/c105 0 2024-03-29T08:07:27.781 INFO:tasks.workunit.client.0.smithi037.stdout:8/541: write d5/d10/d2d/d34/d50/d83/d46/d4e/f57 [4145873,116904] 0 2024-03-29T08:07:27.782 INFO:tasks.workunit.client.1.smithi148.stdout:8/787: dread - d17/fae zero size 2024-03-29T08:07:27.783 INFO:tasks.workunit.client.0.smithi037.stdout:3/519: write dd/d46/d3e/f7e [3927847,86473] 0 2024-03-29T08:07:27.786 INFO:tasks.workunit.client.0.smithi037.stdout:2/436: creat d0/d20/d2e/d5f/d6f/f97 x:0 0 0 2024-03-29T08:07:27.788 INFO:tasks.workunit.client.1.smithi148.stdout:7/771: write d8/dfb/d44/d49/f74 [244654,57997] 0 2024-03-29T08:07:27.791 INFO:tasks.workunit.client.0.smithi037.stdout:4/554: dwrite d3/d4/d27/f2e [0,4194304] 0 2024-03-29T08:07:27.791 INFO:tasks.workunit.client.1.smithi148.stdout:4/803: dwrite df/d2c/d51/d77/d17/fe1 [0,4194304] 0 2024-03-29T08:07:27.793 INFO:tasks.workunit.client.1.smithi148.stdout:4/804: chown df/d2c/d51/d77/def 112 1 2024-03-29T08:07:27.794 INFO:tasks.workunit.client.0.smithi037.stdout:4/555: creat d3/d80/fb3 x:0 0 0 2024-03-29T08:07:27.795 INFO:tasks.workunit.client.1.smithi148.stdout:6/792: sync 2024-03-29T08:07:27.795 INFO:tasks.workunit.client.1.smithi148.stdout:4/805: stat df/d60/la5 0 2024-03-29T08:07:27.796 INFO:tasks.workunit.client.1.smithi148.stdout:7/772: creat d8/d14/d45/d4d/df7/f110 x:0 0 0 2024-03-29T08:07:27.798 INFO:tasks.workunit.client.1.smithi148.stdout:4/806: readlink df/d2c/d51/d77/d17/l69 0 2024-03-29T08:07:27.798 INFO:tasks.workunit.client.1.smithi148.stdout:6/793: mknod d6/c101 0 2024-03-29T08:07:27.803 INFO:tasks.workunit.client.1.smithi148.stdout:4/807: truncate fd 1801944 0 2024-03-29T08:07:27.804 INFO:tasks.workunit.client.1.smithi148.stdout:7/773: mkdir d8/df4/d111 0 2024-03-29T08:07:27.808 INFO:tasks.workunit.client.1.smithi148.stdout:7/774: read d8/d14/f55 [1440749,65216] 0 2024-03-29T08:07:27.809 INFO:tasks.workunit.client.1.smithi148.stdout:6/794: truncate d6/d9/d1e/d36/d4d/d9d/d28/d98/dcc/fd6 93046 0 2024-03-29T08:07:27.810 INFO:tasks.workunit.client.0.smithi037.stdout:2/437: dwrite d0/d53/f88 [0,4194304] 0 2024-03-29T08:07:27.815 INFO:tasks.workunit.client.1.smithi148.stdout:8/788: dread d17/d74/f5a [0,4194304] 0 2024-03-29T08:07:27.818 INFO:tasks.workunit.client.1.smithi148.stdout:6/795: dread d6/d9/d1e/d27/d48/f7b [0,4194304] 0 2024-03-29T08:07:27.822 INFO:tasks.workunit.client.1.smithi148.stdout:5/792: dread d0/db/d7b/f91 [0,4194304] 0 2024-03-29T08:07:27.828 INFO:tasks.workunit.client.0.smithi037.stdout:2/438: getdents d0/d23 0 2024-03-29T08:07:27.835 INFO:tasks.workunit.client.0.smithi037.stdout:4/556: dwrite d3/d4/d15/d1d/db/d45/d1e/f4a [0,4194304] 0 2024-03-29T08:07:27.836 INFO:tasks.workunit.client.0.smithi037.stdout:2/439: symlink d0/d23/l98 0 2024-03-29T08:07:27.836 INFO:tasks.workunit.client.1.smithi148.stdout:4/808: write df/f9a [998790,14322] 0 2024-03-29T08:07:27.838 INFO:tasks.workunit.client.0.smithi037.stdout:3/520: dread f5 [0,4194304] 0 2024-03-29T08:07:27.839 INFO:tasks.workunit.client.0.smithi037.stdout:8/542: dwrite d5/d10/d2d/f3e [0,4194304] 0 2024-03-29T08:07:27.841 INFO:tasks.workunit.client.1.smithi148.stdout:6/796: dwrite d6/d9/d1e/d36/d4d/d9d/fce [0,4194304] 0 2024-03-29T08:07:27.843 INFO:tasks.workunit.client.0.smithi037.stdout:4/557: write d3/d4/d15/d1d/db/d3d/f56 [3645432,114965] 0 2024-03-29T08:07:27.844 INFO:tasks.workunit.client.0.smithi037.stdout:8/543: dread - d5/f7c zero size 2024-03-29T08:07:27.846 INFO:tasks.workunit.client.0.smithi037.stdout:4/558: chown d3/d57/f7a 4 1 2024-03-29T08:07:27.848 INFO:tasks.workunit.client.1.smithi148.stdout:4/809: truncate df/d89/ddd/ffc 1240278 0 2024-03-29T08:07:27.850 INFO:tasks.workunit.client.0.smithi037.stdout:3/521: creat dd/d46/d2d/d3a/d5f/fae x:0 0 0 2024-03-29T08:07:27.850 INFO:tasks.workunit.client.1.smithi148.stdout:6/797: mknod d6/d9/d1e/d36/d4d/d9d/d28/c102 0 2024-03-29T08:07:27.852 INFO:tasks.workunit.client.0.smithi037.stdout:4/559: creat d3/d54/d88/fb4 x:0 0 0 2024-03-29T08:07:27.854 INFO:tasks.workunit.client.1.smithi148.stdout:4/810: mkdir df/d2c/d51/d101 0 2024-03-29T08:07:27.855 INFO:tasks.workunit.client.1.smithi148.stdout:5/793: dread d0/db/d7b/d16/d2e/f56 [0,4194304] 0 2024-03-29T08:07:27.856 INFO:tasks.workunit.client.1.smithi148.stdout:6/798: creat d6/d9/d1e/d36/d4d/d9d/d28/dda/db4/f103 x:0 0 0 2024-03-29T08:07:27.858 INFO:tasks.workunit.client.1.smithi148.stdout:4/811: symlink df/d60/l102 0 2024-03-29T08:07:27.861 INFO:tasks.workunit.client.0.smithi037.stdout:3/522: rmdir dd/d15/d18/d4b 39 2024-03-29T08:07:27.861 INFO:tasks.workunit.client.1.smithi148.stdout:4/812: creat df/d71/f103 x:0 0 0 2024-03-29T08:07:27.863 INFO:tasks.workunit.client.0.smithi037.stdout:2/440: creat d0/d23/d70/f99 x:0 0 0 2024-03-29T08:07:27.865 INFO:tasks.workunit.client.0.smithi037.stdout:4/560: mkdir d3/d4/d15/d1d/db/d45/d1e/d59/db5 0 2024-03-29T08:07:27.865 INFO:tasks.workunit.client.0.smithi037.stdout:3/523: write dd/d15/d18/d8e/fa7 [336710,87013] 0 2024-03-29T08:07:27.868 INFO:tasks.workunit.client.1.smithi148.stdout:4/813: write df/d2c/d51/d77/d26/d9b/da7/d7c/ff9 [313207,122718] 0 2024-03-29T08:07:27.868 INFO:tasks.workunit.client.1.smithi148.stdout:6/799: getdents d6/d9/d1e/d36/db1 0 2024-03-29T08:07:27.870 INFO:tasks.workunit.client.0.smithi037.stdout:2/441: chown d0/d20/d2e/d5f/f8d 7 1 2024-03-29T08:07:27.870 INFO:tasks.workunit.client.0.smithi037.stdout:2/442: symlink d0/d20/d22/d55/l9a 2 2024-03-29T08:07:27.872 INFO:tasks.workunit.client.0.smithi037.stdout:4/561: dread - d3/d4/d15/d1d/db/d45/f5a zero size 2024-03-29T08:07:27.874 INFO:tasks.workunit.client.1.smithi148.stdout:4/814: creat df/d2c/d51/d77/d26/d34/d8a/f104 x:0 0 0 2024-03-29T08:07:27.874 INFO:tasks.workunit.client.0.smithi037.stdout:2/443: stat d0/f8c 0 2024-03-29T08:07:27.874 INFO:tasks.workunit.client.1.smithi148.stdout:6/800: creat d6/d9/d1e/d36/d4d/d9d/d28/dda/db4/f104 x:0 0 0 2024-03-29T08:07:27.876 INFO:tasks.workunit.client.0.smithi037.stdout:4/562: stat d3/d4/d15/d1d/db/d45/d93/d96/da0/db2 0 2024-03-29T08:07:27.877 INFO:tasks.workunit.client.0.smithi037.stdout:3/524: rename dd/d46/d2d/d3a/f55 to dd/d15/d18/faf 0 2024-03-29T08:07:27.878 INFO:tasks.workunit.client.0.smithi037.stdout:2/444: fsync d0/d20/d22/f3f 0 2024-03-29T08:07:27.879 INFO:tasks.workunit.client.1.smithi148.stdout:6/801: fdatasync d6/d9/d1e/f8a 0 2024-03-29T08:07:27.882 INFO:tasks.workunit.client.1.smithi148.stdout:4/815: creat df/f105 x:0 0 0 2024-03-29T08:07:27.885 INFO:tasks.workunit.client.1.smithi148.stdout:6/802: mkdir d6/d9/d1e/d36/d4d/d9d/d28/dda/db6/df3/df4/d105 0 2024-03-29T08:07:27.887 INFO:tasks.workunit.client.1.smithi148.stdout:4/816: symlink df/d2c/d80/daa/dbc/l106 0 2024-03-29T08:07:27.888 INFO:tasks.workunit.client.0.smithi037.stdout:2/445: rename d0/d23/l98 to d0/d23/d70/d71/d78/l9b 0 2024-03-29T08:07:27.891 INFO:tasks.workunit.client.0.smithi037.stdout:4/563: link d3/c6 d3/d4/d15/d1d/db/d45/d93/d96/cb6 0 2024-03-29T08:07:27.892 INFO:tasks.workunit.client.1.smithi148.stdout:4/817: chown df/d2c/f46 98 1 2024-03-29T08:07:27.892 INFO:tasks.workunit.client.0.smithi037.stdout:2/446: fdatasync d0/d20/d22/d55/f5b 0 2024-03-29T08:07:27.893 INFO:tasks.workunit.client.1.smithi148.stdout:6/803: creat d6/d9/d1e/d27/d48/d9f/f106 x:0 0 0 2024-03-29T08:07:27.895 INFO:tasks.workunit.client.0.smithi037.stdout:8/544: dread d5/d10/d19/d3d/f5d [0,4194304] 0 2024-03-29T08:07:27.896 INFO:tasks.workunit.client.1.smithi148.stdout:4/818: creat df/f107 x:0 0 0 2024-03-29T08:07:27.896 INFO:tasks.workunit.client.1.smithi148.stdout:6/804: dread - d6/d9/d1e/d36/db1/faf zero size 2024-03-29T08:07:27.901 INFO:tasks.workunit.client.1.smithi148.stdout:6/805: creat d6/d9/d1e/d27/d48/d73/f107 x:0 0 0 2024-03-29T08:07:27.902 INFO:tasks.workunit.client.1.smithi148.stdout:5/794: dwrite d0/db/d1e/d6e/f92 [0,4194304] 0 2024-03-29T08:07:27.903 INFO:tasks.workunit.client.0.smithi037.stdout:3/525: dwrite dd/f24 [0,4194304] 0 2024-03-29T08:07:27.905 INFO:tasks.workunit.client.1.smithi148.stdout:6/806: read d6/d9/d1e/f53 [2212805,32359] 0 2024-03-29T08:07:27.905 INFO:tasks.workunit.client.0.smithi037.stdout:4/564: mkdir d3/d4/d15/d1d/db/d45/d1e/d59/d60/db7 0 2024-03-29T08:07:27.905 INFO:tasks.workunit.client.0.smithi037.stdout:2/447: truncate d0/f7 2569887 0 2024-03-29T08:07:27.908 INFO:tasks.workunit.client.0.smithi037.stdout:3/526: symlink dd/d46/d2d/d3a/lb0 0 2024-03-29T08:07:27.909 INFO:tasks.workunit.client.0.smithi037.stdout:2/448: mkdir d0/d20/d2e/d9c 0 2024-03-29T08:07:27.909 INFO:tasks.workunit.client.1.smithi148.stdout:4/819: link df/d2c/d51/d77/d26/d34/d8a/d98/fe5 df/d2c/d51/d77/d26/d34/d43/f108 0 2024-03-29T08:07:27.910 INFO:tasks.workunit.client.1.smithi148.stdout:6/807: chown d6/d9/d1e/d36/d4d/d9d/d28/dda/db6/df3/dd3/fec 343 1 2024-03-29T08:07:27.910 INFO:tasks.workunit.client.0.smithi037.stdout:2/449: chown d0/f14 246651 1 2024-03-29T08:07:27.910 INFO:tasks.workunit.client.0.smithi037.stdout:6/472: sync 2024-03-29T08:07:27.910 INFO:tasks.workunit.client.0.smithi037.stdout:7/563: sync 2024-03-29T08:07:27.911 INFO:tasks.workunit.client.0.smithi037.stdout:4/565: creat d3/d4/d15/d1d/db/d45/d93/fb8 x:0 0 0 2024-03-29T08:07:27.913 INFO:tasks.workunit.client.1.smithi148.stdout:5/795: mknod d0/db/d1e/d35/d42/d67/d76/d81/c101 0 2024-03-29T08:07:27.914 INFO:tasks.workunit.client.1.smithi148.stdout:2/881: sync 2024-03-29T08:07:27.915 INFO:tasks.workunit.client.1.smithi148.stdout:3/841: sync 2024-03-29T08:07:27.915 INFO:tasks.workunit.client.0.smithi037.stdout:2/450: rmdir d0/d20/d22 39 2024-03-29T08:07:27.917 INFO:tasks.workunit.client.1.smithi148.stdout:5/796: dread - d0/db/d84/daf/de1/fe4 zero size 2024-03-29T08:07:27.918 INFO:tasks.workunit.client.0.smithi037.stdout:7/564: truncate dd/f67 3230508 0 2024-03-29T08:07:27.923 INFO:tasks.workunit.client.1.smithi148.stdout:2/882: creat d7/da/d4c/d9a/dad/f11f x:0 0 0 2024-03-29T08:07:27.924 INFO:tasks.workunit.client.0.smithi037.stdout:6/473: symlink d0/d19/d34/d4e/d69/d79/l9f 0 2024-03-29T08:07:27.924 INFO:tasks.workunit.client.1.smithi148.stdout:2/883: read - d7/da/dcc/f106 zero size 2024-03-29T08:07:27.925 INFO:tasks.workunit.client.1.smithi148.stdout:5/797: dread - d0/db/d1e/d35/d42/d67/d76/d81/d98/dd5/fb0 zero size 2024-03-29T08:07:27.927 INFO:tasks.workunit.client.0.smithi037.stdout:2/451: dread d0/d23/d44/f45 [0,4194304] 0 2024-03-29T08:07:27.927 INFO:tasks.workunit.client.0.smithi037.stdout:7/565: rename dd/d1d/d34/d54/c62 to dd/d1d/d49/d81/cb7 0 2024-03-29T08:07:27.927 INFO:tasks.workunit.client.1.smithi148.stdout:2/884: write d7/da/f3f [3983263,91067] 0 2024-03-29T08:07:27.929 INFO:tasks.workunit.client.0.smithi037.stdout:7/566: fsync dd/d1d/f53 0 2024-03-29T08:07:27.931 INFO:tasks.workunit.client.0.smithi037.stdout:6/474: creat d0/d19/d34/d4e/d69/d79/fa0 x:0 0 0 2024-03-29T08:07:27.936 INFO:tasks.workunit.client.0.smithi037.stdout:2/452: rename d0/c5 to d0/d20/d2e/d5f/c9d 0 2024-03-29T08:07:27.940 INFO:tasks.workunit.client.1.smithi148.stdout:6/808: write d6/d9/d1e/d36/d4d/d9d/d28/f38 [676987,56023] 0 2024-03-29T08:07:27.946 INFO:tasks.workunit.client.1.smithi148.stdout:2/885: dread d7/da/d52/f63 [0,4194304] 0 2024-03-29T08:07:27.949 INFO:tasks.workunit.client.1.smithi148.stdout:2/886: readlink d7/da/d33/d3c/le0 0 2024-03-29T08:07:27.952 INFO:tasks.workunit.client.1.smithi148.stdout:4/820: dwrite df/d2c/d51/d77/d17/f29 [4194304,4194304] 0 2024-03-29T08:07:27.954 INFO:tasks.workunit.client.0.smithi037.stdout:4/566: dread d3/d4/d15/d1d/db/f16 [0,4194304] 0 2024-03-29T08:07:27.955 INFO:tasks.workunit.client.0.smithi037.stdout:7/567: write dd/d1d/d34/f3f [1723869,111668] 0 2024-03-29T08:07:27.956 INFO:tasks.workunit.client.1.smithi148.stdout:5/798: dwrite d0/db/d7b/d16/f8f [0,4194304] 0 2024-03-29T08:07:27.956 INFO:tasks.workunit.client.0.smithi037.stdout:4/567: chown d3/f44 244060763 1 2024-03-29T08:07:27.957 INFO:tasks.workunit.client.1.smithi148.stdout:4/821: readlink df/da0/la4 0 2024-03-29T08:07:27.960 INFO:tasks.workunit.client.1.smithi148.stdout:5/799: stat d0/db/la9 0 2024-03-29T08:07:27.961 INFO:tasks.workunit.client.1.smithi148.stdout:2/887: dread - d7/da/d33/ded/d2e/f97 zero size 2024-03-29T08:07:27.965 INFO:tasks.workunit.client.1.smithi148.stdout:3/842: dread de/d24/d33/d39/f8b [0,4194304] 0 2024-03-29T08:07:27.965 INFO:tasks.workunit.client.0.smithi037.stdout:7/568: creat dd/d21/d35/d3b/fb8 x:0 0 0 2024-03-29T08:07:27.966 INFO:tasks.workunit.client.1.smithi148.stdout:5/800: unlink d0/db/d1e/d35/d42/f6b 0 2024-03-29T08:07:27.968 INFO:tasks.workunit.client.0.smithi037.stdout:7/569: rmdir dd/d21/d70 39 2024-03-29T08:07:27.969 INFO:tasks.workunit.client.1.smithi148.stdout:6/809: dwrite d6/d9/d1e/d36/d4d/d9d/d28/dda/db6/df3/dd3/fec [0,4194304] 0 2024-03-29T08:07:27.970 INFO:tasks.workunit.client.0.smithi037.stdout:7/570: dread - dd/f7e zero size 2024-03-29T08:07:27.972 INFO:tasks.workunit.client.1.smithi148.stdout:2/888: unlink d7/da/d52/f63 0 2024-03-29T08:07:27.976 INFO:tasks.workunit.client.1.smithi148.stdout:3/843: unlink de/d24/f56 0 2024-03-29T08:07:27.976 INFO:tasks.workunit.client.1.smithi148.stdout:6/810: truncate d6/d9/d1e/d36/d4d/fbf 628688 0 2024-03-29T08:07:27.977 INFO:tasks.workunit.client.1.smithi148.stdout:3/844: fsync de/f110 0 2024-03-29T08:07:27.978 INFO:tasks.workunit.client.1.smithi148.stdout:2/889: mkdir d7/d8c/da4/d8e/dda/d120 0 2024-03-29T08:07:27.980 INFO:tasks.workunit.client.1.smithi148.stdout:4/822: getdents df/d2c/db2 0 2024-03-29T08:07:27.983 INFO:tasks.workunit.client.1.smithi148.stdout:3/845: stat de/d24/d33/d39/d46/d30/d37/d79/f7c 0 2024-03-29T08:07:27.993 INFO:tasks.workunit.client.1.smithi148.stdout:3/846: creat de/d24/d33/d39/d46/d70/def/f111 x:0 0 0 2024-03-29T08:07:27.993 INFO:tasks.workunit.client.0.smithi037.stdout:7/571: truncate dd/d1d/d34/f3f 909340 0 2024-03-29T08:07:27.995 INFO:tasks.workunit.client.1.smithi148.stdout:5/801: dwrite d0/db/d1e/d36/de6/ffd [0,4194304] 0 2024-03-29T08:07:27.997 INFO:tasks.workunit.client.0.smithi037.stdout:7/572: rmdir dd/d21/d35/d3b 39 2024-03-29T08:07:27.998 INFO:tasks.workunit.client.1.smithi148.stdout:6/811: dwrite d6/fe [0,4194304] 0 2024-03-29T08:07:28.001 INFO:tasks.workunit.client.0.smithi037.stdout:7/573: creat dd/d68/fb9 x:0 0 0 2024-03-29T08:07:28.002 INFO:tasks.workunit.client.0.smithi037.stdout:7/574: fsync dd/d21/d35/f8f 0 2024-03-29T08:07:28.005 INFO:tasks.workunit.client.1.smithi148.stdout:3/847: mknod de/d24/d33/d39/d46/d30/d37/d79/d10e/c112 0 2024-03-29T08:07:28.009 INFO:tasks.workunit.client.1.smithi148.stdout:2/890: write d7/da/f9d [816906,114692] 0 2024-03-29T08:07:28.010 INFO:tasks.workunit.client.1.smithi148.stdout:4/823: write df/f1a [367943,84034] 0 2024-03-29T08:07:28.012 INFO:tasks.workunit.client.1.smithi148.stdout:2/891: readlink d7/da/d33/lb3 0 2024-03-29T08:07:28.014 INFO:tasks.workunit.client.0.smithi037.stdout:4/568: dread d3/d4/d15/d1d/db/f1a [0,4194304] 0 2024-03-29T08:07:28.015 INFO:tasks.workunit.client.0.smithi037.stdout:7/575: write f9 [819076,10287] 0 2024-03-29T08:07:28.016 INFO:tasks.workunit.client.1.smithi148.stdout:4/824: dread - df/d2c/d51/d77/d26/d34/d8a/f104 zero size 2024-03-29T08:07:28.018 INFO:tasks.workunit.client.0.smithi037.stdout:4/569: write d3/d54/d88/f8d [1825775,44658] 0 2024-03-29T08:07:28.019 INFO:tasks.workunit.client.1.smithi148.stdout:2/892: write d7/da/d33/d3c/d54/f95 [630927,27568] 0 2024-03-29T08:07:28.019 INFO:tasks.workunit.client.0.smithi037.stdout:4/570: chown d3/f21 0 1 2024-03-29T08:07:28.024 INFO:tasks.workunit.client.1.smithi148.stdout:6/812: dwrite d6/d9/d1e/d27/d48/d9f/faa [0,4194304] 0 2024-03-29T08:07:28.025 INFO:tasks.workunit.client.1.smithi148.stdout:4/825: dread - df/d2c/d51/d77/d26/d34/f6d zero size 2024-03-29T08:07:28.026 INFO:tasks.workunit.client.0.smithi037.stdout:4/571: dread d3/d57/d6a/f77 [0,4194304] 0 2024-03-29T08:07:28.027 INFO:tasks.workunit.client.1.smithi148.stdout:4/826: chown df/d2c/d51/d77/f1d 1732557 1 2024-03-29T08:07:28.027 INFO:tasks.workunit.client.1.smithi148.stdout:2/893: symlink d7/d24/d4b/d66/df1/l121 0 2024-03-29T08:07:28.028 INFO:tasks.workunit.client.0.smithi037.stdout:4/572: chown d3/d4/d27/l28 3821 1 2024-03-29T08:07:28.030 INFO:tasks.workunit.client.0.smithi037.stdout:4/573: chown d3/d4/f35 56480 1 2024-03-29T08:07:28.032 INFO:tasks.workunit.client.1.smithi148.stdout:4/827: symlink df/d2c/d51/d77/l109 0 2024-03-29T08:07:28.036 INFO:tasks.workunit.client.0.smithi037.stdout:4/574: mkdir d3/d4/d15/d1d/db/d45/d1e/d59/d60/d6f/db9 0 2024-03-29T08:07:28.037 INFO:tasks.workunit.client.1.smithi148.stdout:4/828: rmdir df/d2c/d51/d77/d26/d34/d3e 39 2024-03-29T08:07:28.038 INFO:tasks.workunit.client.1.smithi148.stdout:3/848: dwrite de/d24/d33/d39/d46/d30/d37/faa [0,4194304] 0 2024-03-29T08:07:28.038 INFO:tasks.workunit.client.1.smithi148.stdout:6/813: link d6/d9/d1e/d27/d48/d73/ce8 d6/d9/d1e/d36/d4d/d9d/d28/dda/db6/c108 0 2024-03-29T08:07:28.040 INFO:tasks.workunit.client.0.smithi037.stdout:4/575: readlink d3/d4/d15/d1d/db/d45/d1e/d59/l69 0 2024-03-29T08:07:28.040 INFO:tasks.workunit.client.1.smithi148.stdout:3/849: stat de/d3b/d49/f8a 0 2024-03-29T08:07:28.041 INFO:tasks.workunit.client.1.smithi148.stdout:6/814: dread - d6/d9/d1e/d20/d6b/fe3 zero size 2024-03-29T08:07:28.042 INFO:tasks.workunit.client.0.smithi037.stdout:7/576: dread dd/f1b [0,4194304] 0 2024-03-29T08:07:28.043 INFO:tasks.workunit.client.0.smithi037.stdout:4/576: chown d3/d4/d15/d1d/db/d3d/f58 155390 1 2024-03-29T08:07:28.043 INFO:tasks.workunit.client.1.smithi148.stdout:3/850: fdatasync de/d24/d33/d39/d46/f23 0 2024-03-29T08:07:28.043 INFO:tasks.workunit.client.1.smithi148.stdout:4/829: mkdir df/d2c/d51/d77/d26/d9b/d10a 0 2024-03-29T08:07:28.046 INFO:tasks.workunit.client.1.smithi148.stdout:4/830: write df/f14 [5565130,83296] 0 2024-03-29T08:07:28.047 INFO:tasks.workunit.client.0.smithi037.stdout:4/577: dread - d3/d4/d15/d1d/db/d45/d1e/d59/d60/d6f/fa4 zero size 2024-03-29T08:07:28.048 INFO:tasks.workunit.client.0.smithi037.stdout:4/578: chown d3/d57/f8e 6143628 1 2024-03-29T08:07:28.048 INFO:tasks.workunit.client.1.smithi148.stdout:3/851: chown de/d24/d33/d39/d46/f65 58435 1 2024-03-29T08:07:28.049 INFO:tasks.workunit.client.1.smithi148.stdout:4/831: chown df/d2c/d51/d77/d26/f3b 0 1 2024-03-29T08:07:28.051 INFO:tasks.workunit.client.1.smithi148.stdout:3/852: symlink de/d24/d33/d39/d46/d109/l113 0 2024-03-29T08:07:28.051 INFO:tasks.workunit.client.1.smithi148.stdout:4/832: stat df/d2c/d51/d77/d17/d73/fc0 0 2024-03-29T08:07:28.052 INFO:tasks.workunit.client.1.smithi148.stdout:4/833: write fa [1520718,47340] 0 2024-03-29T08:07:28.053 INFO:tasks.workunit.client.1.smithi148.stdout:6/815: rename d6/d9/d1e/d36/d4d/d9d/d28/d98/lcd to d6/d9/d1e/d36/d4d/d9d/d28/db3/l109 0 2024-03-29T08:07:28.054 INFO:tasks.workunit.client.1.smithi148.stdout:4/834: fsync df/d2c/d51/d77/f83 0 2024-03-29T08:07:28.056 INFO:tasks.workunit.client.0.smithi037.stdout:4/579: rename d3/f44 to d3/d4/d15/d1d/dad/fba 0 2024-03-29T08:07:28.063 INFO:tasks.workunit.client.0.smithi037.stdout:4/580: link d3/d4/d15/d1d/dad/fba d3/d4/d15/d1d/db/d72/fbb 0 2024-03-29T08:07:28.070 INFO:tasks.workunit.client.1.smithi148.stdout:4/835: dwrite df/d71/f103 [0,4194304] 0 2024-03-29T08:07:28.071 INFO:tasks.workunit.client.0.smithi037.stdout:7/577: dread dd/f67 [0,4194304] 0 2024-03-29T08:07:28.071 INFO:tasks.workunit.client.1.smithi148.stdout:6/816: dwrite d6/d9/d1e/d36/d4d/d9d/f31 [4194304,4194304] 0 2024-03-29T08:07:28.073 INFO:tasks.workunit.client.1.smithi148.stdout:4/836: write df/d2c/d51/d77/d26/d34/d8a/f104 [358052,76045] 0 2024-03-29T08:07:28.077 INFO:tasks.workunit.client.1.smithi148.stdout:4/837: unlink df/d2c/d51/d77/d17/fe1 0 2024-03-29T08:07:28.086 INFO:tasks.workunit.client.1.smithi148.stdout:4/838: getdents df/d2c/d51/d77/d26/d34/d8a/d98 0 2024-03-29T08:07:28.088 INFO:tasks.workunit.client.1.smithi148.stdout:6/817: dwrite d6/d9/d1e/fee [0,4194304] 0 2024-03-29T08:07:28.090 INFO:tasks.workunit.client.1.smithi148.stdout:4/839: mkdir df/d2c/d10b 0 2024-03-29T08:07:28.091 INFO:tasks.workunit.client.1.smithi148.stdout:4/840: fsync df/f9a 0 2024-03-29T08:07:28.098 INFO:tasks.workunit.client.0.smithi037.stdout:7/578: dread dd/d21/d35/f4c [0,4194304] 0 2024-03-29T08:07:28.099 INFO:tasks.workunit.client.1.smithi148.stdout:6/818: truncate d6/d9/d1e/d36/d4d/d9d/d28/dda/fa9 291941 0 2024-03-29T08:07:28.100 INFO:tasks.workunit.client.1.smithi148.stdout:4/841: rmdir df/d2c/d51/d77/d26/d34/db3/dd5 0 2024-03-29T08:07:28.102 INFO:tasks.workunit.client.0.smithi037.stdout:7/579: dread - dd/d21/d35/d60/d94/fb5 zero size 2024-03-29T08:07:28.104 INFO:tasks.workunit.client.1.smithi148.stdout:4/842: rmdir df/d89 39 2024-03-29T08:07:28.107 INFO:tasks.workunit.client.1.smithi148.stdout:4/843: fsync df/d2c/d51/d77/d26/d9b/da7/d7c/ff9 0 2024-03-29T08:07:28.107 INFO:tasks.workunit.client.0.smithi037.stdout:7/580: symlink dd/d21/d35/d60/d94/lba 0 2024-03-29T08:07:28.108 INFO:tasks.workunit.client.1.smithi148.stdout:3/853: write de/d24/d33/d39/da3/ff6 [431913,62324] 0 2024-03-29T08:07:28.113 INFO:tasks.workunit.client.1.smithi148.stdout:6/819: dwrite d6/f70 [0,4194304] 0 2024-03-29T08:07:28.117 INFO:tasks.workunit.client.1.smithi148.stdout:4/844: rmdir df/d2c/d51/d77/d26/d9b/da7/d7c 39 2024-03-29T08:07:28.117 INFO:tasks.workunit.client.0.smithi037.stdout:7/581: link dd/d21/d35/d3b/fb8 dd/d1d/d34/fbb 0 2024-03-29T08:07:28.118 INFO:tasks.workunit.client.1.smithi148.stdout:4/845: readlink df/l48 0 2024-03-29T08:07:28.119 INFO:tasks.workunit.client.0.smithi037.stdout:7/582: read - dd/d21/d85/fa6 zero size 2024-03-29T08:07:28.119 INFO:tasks.workunit.client.0.smithi037.stdout:4/581: dwrite d3/d4/d15/d1d/db/d45/d1e/d59/d60/f75 [0,4194304] 0 2024-03-29T08:07:28.120 INFO:tasks.workunit.client.1.smithi148.stdout:6/820: write d6/d9/d1e/d36/d4d/d9d/fce [2017979,4009] 0 2024-03-29T08:07:28.123 INFO:tasks.workunit.client.0.smithi037.stdout:4/582: creat d3/d4/d27/fbc x:0 0 0 2024-03-29T08:07:28.125 INFO:tasks.workunit.client.0.smithi037.stdout:4/583: readlink d3/d4/d27/l28 0 2024-03-29T08:07:28.126 INFO:tasks.workunit.client.0.smithi037.stdout:7/583: rename dd/d21/d35/f8f to dd/d1d/d49/fbc 0 2024-03-29T08:07:28.126 INFO:tasks.workunit.client.1.smithi148.stdout:6/821: write d6/d9/d1e/d36/d4d/d9d/fb2 [373126,3792] 0 2024-03-29T08:07:28.127 INFO:tasks.workunit.client.1.smithi148.stdout:3/854: dread de/d24/d33/d39/d46/d30/d37/fae [0,4194304] 0 2024-03-29T08:07:28.129 INFO:tasks.workunit.client.0.smithi037.stdout:4/584: write d3/d4/d15/d1d/db/fa3 [421373,53700] 0 2024-03-29T08:07:28.129 INFO:tasks.workunit.client.1.smithi148.stdout:7/775: sync 2024-03-29T08:07:28.130 INFO:tasks.workunit.client.1.smithi148.stdout:1/749: sync 2024-03-29T08:07:28.130 INFO:tasks.workunit.client.1.smithi148.stdout:0/799: sync 2024-03-29T08:07:28.130 INFO:tasks.workunit.client.1.smithi148.stdout:9/835: sync 2024-03-29T08:07:28.133 INFO:tasks.workunit.client.0.smithi037.stdout:7/584: creat dd/d21/d35/d3b/fbd x:0 0 0 2024-03-29T08:07:28.137 INFO:tasks.workunit.client.1.smithi148.stdout:7/776: unlink d8/d38/d3c/f96 0 2024-03-29T08:07:28.138 INFO:tasks.workunit.client.1.smithi148.stdout:6/822: creat d6/d9/d1e/d36/d4d/d9d/d28/d98/dc2/f10a x:0 0 0 2024-03-29T08:07:28.138 INFO:tasks.workunit.client.0.smithi037.stdout:7/585: mknod dd/d21/d35/d60/cbe 0 2024-03-29T08:07:28.139 INFO:tasks.workunit.client.1.smithi148.stdout:1/750: write d3/d18/d4d/d76/d85/fcd [703577,33786] 0 2024-03-29T08:07:28.141 INFO:tasks.workunit.client.1.smithi148.stdout:3/855: symlink de/d24/d33/d39/d46/d30/d10d/l114 0 2024-03-29T08:07:28.141 INFO:tasks.workunit.client.0.smithi037.stdout:7/586: write dd/d1d/d34/f37 [1217299,80607] 0 2024-03-29T08:07:28.142 INFO:tasks.workunit.client.1.smithi148.stdout:0/800: creat db/de/df/d11/ff4 x:0 0 0 2024-03-29T08:07:28.143 INFO:tasks.workunit.client.1.smithi148.stdout:6/823: fdatasync d6/d9/d1e/d27/d48/d9f/f106 0 2024-03-29T08:07:28.146 INFO:tasks.workunit.client.1.smithi148.stdout:9/836: rmdir db/d22/d26/d54/d4d/d35/d4b/d58/df1/df2 0 2024-03-29T08:07:28.147 INFO:tasks.workunit.client.1.smithi148.stdout:7/777: truncate d8/d14/d45/d4d/d78/fb5 1098343 0 2024-03-29T08:07:28.147 INFO:tasks.workunit.client.0.smithi037.stdout:9/746: sync 2024-03-29T08:07:28.147 INFO:tasks.workunit.client.1.smithi148.stdout:0/801: chown db/dc/daa/d47/f96 188 1 2024-03-29T08:07:28.149 INFO:tasks.workunit.client.1.smithi148.stdout:3/856: chown de/d24/d33/d39/d46/d30/d37/d62/d63/dbf/lca 5 1 2024-03-29T08:07:28.149 INFO:tasks.workunit.client.1.smithi148.stdout:4/846: dwrite df/d2c/d51/d77/d26/d34/d3e/f58 [0,4194304] 0 2024-03-29T08:07:28.154 INFO:tasks.workunit.client.0.smithi037.stdout:9/747: rename d1/d4/d19/d1e/d4a/d5c/l6a to d1/d4/d19/d1e/d4a/d6f/d96/l102 0 2024-03-29T08:07:28.155 INFO:tasks.workunit.client.1.smithi148.stdout:0/802: write db/de/df/d11/f16 [3117986,97377] 0 2024-03-29T08:07:28.155 INFO:tasks.workunit.client.1.smithi148.stdout:6/824: fdatasync d6/d9/d1e/d36/d4d/d9d/d28/dda/db6/ffa 0 2024-03-29T08:07:28.159 INFO:tasks.workunit.client.1.smithi148.stdout:1/751: truncate d3/de/d25/d35/fd0 2254453 0 2024-03-29T08:07:28.159 INFO:tasks.workunit.client.1.smithi148.stdout:4/847: dread - df/d2c/d51/d77/d26/d9b/da7/fbb zero size 2024-03-29T08:07:28.162 INFO:tasks.workunit.client.1.smithi148.stdout:4/848: write df/fea [148576,92607] 0 2024-03-29T08:07:28.162 INFO:tasks.workunit.client.1.smithi148.stdout:1/752: stat d3/d55/c8b 0 2024-03-29T08:07:28.163 INFO:tasks.workunit.client.0.smithi037.stdout:9/748: dread d1/d4/d19/d1e/d4a/d6f/da2/d31/daa/dba/dbb/ff2 [0,4194304] 0 2024-03-29T08:07:28.164 INFO:tasks.workunit.client.1.smithi148.stdout:6/825: mknod d6/d9/d1e/d36/d4d/d9d/d28/db3/c10b 0 2024-03-29T08:07:28.166 INFO:tasks.workunit.client.1.smithi148.stdout:0/803: mkdir db/de/df/d11/d1f/dc5/df5 0 2024-03-29T08:07:28.168 INFO:tasks.workunit.client.1.smithi148.stdout:3/857: read de/d24/d33/d39/d46/f23 [2175977,108926] 0 2024-03-29T08:07:28.169 INFO:tasks.workunit.client.0.smithi037.stdout:9/749: symlink d1/d4/d19/d1e/d71/d89/l103 0 2024-03-29T08:07:28.171 INFO:tasks.workunit.client.1.smithi148.stdout:9/837: dwrite db/d10/d2d/d4e/dd4/f1b [4194304,4194304] 0 2024-03-29T08:07:28.171 INFO:tasks.workunit.client.1.smithi148.stdout:0/804: chown db/de/df/d2b/f72 61 1 2024-03-29T08:07:28.172 INFO:tasks.workunit.client.0.smithi037.stdout:7/587: dread dd/f1f [0,4194304] 0 2024-03-29T08:07:28.173 INFO:tasks.workunit.client.0.smithi037.stdout:9/750: fdatasync d1/d4/d19/d1e/d4a/d6f/da2/d31/f38 0 2024-03-29T08:07:28.176 INFO:tasks.workunit.client.0.smithi037.stdout:9/751: dread - d1/d4/d95/fff zero size 2024-03-29T08:07:28.178 INFO:tasks.workunit.client.0.smithi037.stdout:7/588: rename dd/d21/d85/fa6 to dd/d21/d35/d60/fbf 0 2024-03-29T08:07:28.182 INFO:tasks.workunit.client.1.smithi148.stdout:6/826: link d6/c6a d6/d9/d1e/d27/d48/df7/c10c 0 2024-03-29T08:07:28.183 INFO:tasks.workunit.client.1.smithi148.stdout:7/778: dwrite d8/d38/d3c/da5/db4/de9/fed [0,4194304] 0 2024-03-29T08:07:28.188 INFO:tasks.workunit.client.0.smithi037.stdout:7/589: creat dd/d21/d35/fc0 x:0 0 0 2024-03-29T08:07:28.195 INFO:tasks.workunit.client.1.smithi148.stdout:7/779: fsync d8/d38/d3c/da9/db7/fc4 0 2024-03-29T08:07:28.196 INFO:tasks.workunit.client.0.smithi037.stdout:7/590: creat dd/d1d/d49/d81/fc1 x:0 0 0 2024-03-29T08:07:28.197 INFO:tasks.workunit.client.1.smithi148.stdout:1/753: dread d3/d18/d4d/d9e/f40 [0,4194304] 0 2024-03-29T08:07:28.199 INFO:tasks.workunit.client.1.smithi148.stdout:4/849: dwrite df/d2c/d80/daa/dbc/dc9/fe9 [0,4194304] 0 2024-03-29T08:07:28.203 INFO:tasks.workunit.client.1.smithi148.stdout:8/789: sync 2024-03-29T08:07:28.209 INFO:tasks.workunit.client.1.smithi148.stdout:6/827: link d6/d9/d1e/d36/d4d/d9d/d28/dda/f92 d6/d9/d1e/f10d 0 2024-03-29T08:07:28.214 INFO:tasks.workunit.client.1.smithi148.stdout:0/805: dwrite db/dc/f1e [4194304,4194304] 0 2024-03-29T08:07:28.215 INFO:tasks.workunit.client.0.smithi037.stdout:9/752: dwrite d1/d4/d19/d1e/d4a/d6f/da2/d23/f65 [0,4194304] 0 2024-03-29T08:07:28.215 INFO:tasks.workunit.client.1.smithi148.stdout:3/858: dwrite de/d24/d33/d39/d46/d70/fe5 [0,4194304] 0 2024-03-29T08:07:28.216 INFO:tasks.workunit.client.0.smithi037.stdout:7/591: getdents dd/d1d/d49/d81 0 2024-03-29T08:07:28.218 INFO:tasks.workunit.client.0.smithi037.stdout:7/592: chown dd/d68/fb9 2808919 1 2024-03-29T08:07:28.224 INFO:tasks.workunit.client.1.smithi148.stdout:0/806: write db/dc/daa/d47/daf/fb7 [4756140,65070] 0 2024-03-29T08:07:28.224 INFO:tasks.workunit.client.1.smithi148.stdout:4/850: creat df/d2c/d51/d77/d26/d34/f10c x:0 0 0 2024-03-29T08:07:28.224 INFO:tasks.workunit.client.0.smithi037.stdout:9/753: getdents d1/d1f/df6 0 2024-03-29T08:07:28.228 INFO:tasks.workunit.client.0.smithi037.stdout:7/593: link dd/d21/l80 dd/d1d/d49/d81/d84/lc2 0 2024-03-29T08:07:28.232 INFO:tasks.workunit.client.1.smithi148.stdout:3/859: mkdir de/d24/d33/d39/d46/d70/d101/d115 0 2024-03-29T08:07:28.238 INFO:tasks.workunit.client.1.smithi148.stdout:8/790: rename d17/d49/d9a/dbe/d6f/d64/fd9 to d17/d49/d9a/dbe/d6f/d64/df1/f106 0 2024-03-29T08:07:28.238 INFO:tasks.workunit.client.1.smithi148.stdout:9/838: dwrite db/d22/d26/d54/d4d/d35/d4b/d5a/f77 [0,4194304] 0 2024-03-29T08:07:28.240 INFO:tasks.workunit.client.0.smithi037.stdout:7/594: mkdir dd/dc3 0 2024-03-29T08:07:28.242 INFO:tasks.workunit.client.1.smithi148.stdout:4/851: creat df/d2c/d51/d77/def/f10d x:0 0 0 2024-03-29T08:07:28.242 INFO:tasks.workunit.client.1.smithi148.stdout:1/754: dwrite d3/de/d1d/fd5 [0,4194304] 0 2024-03-29T08:07:28.242 INFO:tasks.workunit.client.1.smithi148.stdout:0/807: creat db/de/df/d11/d2d/da3/dee/ff6 x:0 0 0 2024-03-29T08:07:28.250 INFO:tasks.workunit.client.0.smithi037.stdout:9/754: dwrite d1/d4/d19/d1e/d25/d16/f9e [0,4194304] 0 2024-03-29T08:07:28.251 INFO:tasks.workunit.client.1.smithi148.stdout:4/852: truncate df/f27 1104061 0 2024-03-29T08:07:28.256 INFO:tasks.workunit.client.0.smithi037.stdout:7/595: link dd/d68/la7 dd/d21/d89/lc4 0 2024-03-29T08:07:28.257 INFO:tasks.workunit.client.1.smithi148.stdout:9/839: fsync db/d10/d6d/d73/da8/deb/dec/dde/f102 0 2024-03-29T08:07:28.261 INFO:tasks.workunit.client.1.smithi148.stdout:2/894: sync 2024-03-29T08:07:28.261 INFO:tasks.workunit.client.1.smithi148.stdout:5/802: sync 2024-03-29T08:07:28.262 INFO:tasks.workunit.client.1.smithi148.stdout:1/755: creat d3/de/d4f/fe9 x:0 0 0 2024-03-29T08:07:28.262 INFO:tasks.workunit.client.1.smithi148.stdout:3/860: dwrite de/f51 [0,4194304] 0 2024-03-29T08:07:28.265 INFO:tasks.workunit.client.1.smithi148.stdout:6/828: dwrite d6/d9/d1e/d36/db1/f76 [0,4194304] 0 2024-03-29T08:07:28.265 INFO:tasks.workunit.client.1.smithi148.stdout:7/780: dread d8/d38/d3c/da9/f63 [4194304,4194304] 0 2024-03-29T08:07:28.268 INFO:tasks.workunit.client.0.smithi037.stdout:7/596: getdents dd 0 2024-03-29T08:07:28.269 INFO:tasks.workunit.client.1.smithi148.stdout:8/791: truncate d17/d4e/f5d 8366791 0 2024-03-29T08:07:28.270 INFO:tasks.workunit.client.1.smithi148.stdout:0/808: write db/f7b [2566492,93437] 0 2024-03-29T08:07:28.271 INFO:tasks.workunit.client.1.smithi148.stdout:5/803: chown d0/db/d1e/d35/c48 42837 1 2024-03-29T08:07:28.272 INFO:tasks.workunit.client.1.smithi148.stdout:2/895: write d7/da/dcc/f106 [339036,80206] 0 2024-03-29T08:07:28.273 INFO:tasks.workunit.client.1.smithi148.stdout:4/853: write df/d2c/d51/d77/d26/f63 [294279,48306] 0 2024-03-29T08:07:28.275 INFO:tasks.workunit.client.0.smithi037.stdout:7/597: creat dd/d21/d35/fc5 x:0 0 0 2024-03-29T08:07:28.275 INFO:tasks.workunit.client.1.smithi148.stdout:8/792: mkdir d17/d4e/d107 0 2024-03-29T08:07:28.277 INFO:tasks.workunit.client.1.smithi148.stdout:6/829: write d6/d9/d1e/d36/d4d/d9d/d28/dda/db6/df3/fe1 [968040,102516] 0 2024-03-29T08:07:28.279 INFO:tasks.workunit.client.0.smithi037.stdout:7/598: mknod dd/dc3/cc6 0 2024-03-29T08:07:28.281 INFO:tasks.workunit.client.1.smithi148.stdout:9/840: read db/d10/d2d/d4e/f81 [2328741,108077] 0 2024-03-29T08:07:28.282 INFO:tasks.workunit.client.1.smithi148.stdout:5/804: fsync d0/db/d1e/d35/d42/d67/d9a/f6d 0 2024-03-29T08:07:28.282 INFO:tasks.workunit.client.1.smithi148.stdout:8/793: symlink d17/d49/d9a/db8/l108 0 2024-03-29T08:07:28.291 INFO:tasks.workunit.client.1.smithi148.stdout:0/809: creat db/dc/daa/d47/d4c/d5a/d7a/ff7 x:0 0 0 2024-03-29T08:07:28.291 INFO:tasks.workunit.client.1.smithi148.stdout:1/756: link d3/d18/d4d/d76/d85/ld1 d3/de/d25/d35/dc2/dac/d7a/lea 0 2024-03-29T08:07:28.292 INFO:tasks.workunit.client.1.smithi148.stdout:8/794: symlink d17/d18/d26/d4f/dde/l109 0 2024-03-29T08:07:28.293 INFO:tasks.workunit.client.1.smithi148.stdout:9/841: fdatasync db/d10/d2d/d4e/d5f/f114 0 2024-03-29T08:07:28.295 INFO:tasks.workunit.client.1.smithi148.stdout:1/757: chown d3/c5 790013332 1 2024-03-29T08:07:28.296 INFO:tasks.workunit.client.1.smithi148.stdout:8/795: dread - d17/d74/f4b zero size 2024-03-29T08:07:28.298 INFO:tasks.workunit.client.1.smithi148.stdout:5/805: rmdir d0/d46/d86 39 2024-03-29T08:07:28.298 INFO:tasks.workunit.client.1.smithi148.stdout:4/854: dread df/d2c/d51/d77/d26/d9b/fc1 [0,4194304] 0 2024-03-29T08:07:28.299 INFO:tasks.workunit.client.0.smithi037.stdout:1/612: sync 2024-03-29T08:07:28.299 INFO:tasks.workunit.client.1.smithi148.stdout:6/830: mknod d6/d9/d1e/d36/d4d/d9d/d28/d98/c10e 0 2024-03-29T08:07:28.300 INFO:tasks.workunit.client.1.smithi148.stdout:9/842: unlink f4 0 2024-03-29T08:07:28.300 INFO:tasks.workunit.client.1.smithi148.stdout:2/896: link d7/da/d33/ded/d2e/l8b d7/da/d33/d3c/d31/l122 0 2024-03-29T08:07:28.301 INFO:tasks.workunit.client.1.smithi148.stdout:0/810: mknod db/dc/daa/d47/cf8 0 2024-03-29T08:07:28.302 INFO:tasks.workunit.client.1.smithi148.stdout:8/796: stat d17/d49/d9a/dbe/cdb 0 2024-03-29T08:07:28.303 INFO:tasks.workunit.client.1.smithi148.stdout:5/806: chown d0/db/d1e/f31 47856980 1 2024-03-29T08:07:28.303 INFO:tasks.workunit.client.1.smithi148.stdout:6/831: read - d6/d9/d1e/d36/ff6 zero size 2024-03-29T08:07:28.303 INFO:tasks.workunit.client.1.smithi148.stdout:2/897: stat d7/d8c/f103 0 2024-03-29T08:07:28.304 INFO:tasks.workunit.client.0.smithi037.stdout:1/613: symlink d2/d4b/d80/ld3 0 2024-03-29T08:07:28.309 INFO:tasks.workunit.client.1.smithi148.stdout:0/811: symlink db/de/df/lf9 0 2024-03-29T08:07:28.310 INFO:tasks.workunit.client.1.smithi148.stdout:0/812: readlink db/de/df/d2b/ldc 0 2024-03-29T08:07:28.311 INFO:tasks.workunit.client.1.smithi148.stdout:5/807: dread - d0/db/d1e/d35/d42/d67/d76/d81/fff zero size 2024-03-29T08:07:28.313 INFO:tasks.workunit.client.1.smithi148.stdout:2/898: truncate d7/d24/f5d 1687 0 2024-03-29T08:07:28.327 INFO:tasks.workunit.client.0.smithi037.stdout:7/599: write dd/f67 [136268,18084] 0 2024-03-29T08:07:28.330 INFO:tasks.workunit.client.0.smithi037.stdout:9/755: dread d1/d4/d19/d1e/d4a/d6f/da2/d13/d7c/fb4 [0,4194304] 0 2024-03-29T08:07:28.331 INFO:tasks.workunit.client.1.smithi148.stdout:0/813: truncate db/de/df/d11/f41 3418077 0 2024-03-29T08:07:28.331 INFO:tasks.workunit.client.1.smithi148.stdout:9/843: dread db/d22/d26/d54/d4d/d35/d4b/d5a/f77 [0,4194304] 0 2024-03-29T08:07:28.333 INFO:tasks.workunit.client.1.smithi148.stdout:6/832: truncate d6/d9/d1e/d27/f9b 766913 0 2024-03-29T08:07:28.336 INFO:tasks.workunit.client.1.smithi148.stdout:7/781: dread d8/d38/f8d [0,4194304] 0 2024-03-29T08:07:28.336 INFO:tasks.workunit.client.1.smithi148.stdout:8/797: dwrite d17/d18/ddf/fe1 [0,4194304] 0 2024-03-29T08:07:28.338 INFO:tasks.workunit.client.1.smithi148.stdout:2/899: rename d7/da/d4c/d9a/dad to d7/da/d33/d3c/d31/d123 0 2024-03-29T08:07:28.339 INFO:tasks.workunit.client.0.smithi037.stdout:1/614: dread d2/d6/dc/d18/d1d/d21/f37 [0,4194304] 0 2024-03-29T08:07:28.339 INFO:tasks.workunit.client.1.smithi148.stdout:9/844: stat db/d10/d2d/d4e/da6/dd1/dd9/fff 0 2024-03-29T08:07:28.344 INFO:tasks.workunit.client.1.smithi148.stdout:4/855: dwrite df/f27 [0,4194304] 0 2024-03-29T08:07:28.344 INFO:tasks.workunit.client.1.smithi148.stdout:5/808: truncate d0/db/d1e/d35/d42/d67/d9a/f6d 3275061 0 2024-03-29T08:07:28.344 INFO:tasks.workunit.client.1.smithi148.stdout:9/845: stat db/d22/d26/d54/d4d/d35/d4b/c10b 0 2024-03-29T08:07:28.345 INFO:tasks.workunit.client.0.smithi037.stdout:1/615: chown d2/d6/dc/d18/d22/d4f/d86/cc2 243177123 1 2024-03-29T08:07:28.346 INFO:tasks.workunit.client.1.smithi148.stdout:5/809: read - d0/db/d84/daf/ffa zero size 2024-03-29T08:07:28.346 INFO:tasks.workunit.client.0.smithi037.stdout:7/600: creat dd/d21/d35/d3b/d5d/fc7 x:0 0 0 2024-03-29T08:07:28.348 INFO:tasks.workunit.client.1.smithi148.stdout:2/900: rmdir d7/d24/d4b/d66/df1 39 2024-03-29T08:07:28.349 INFO:tasks.workunit.client.1.smithi148.stdout:5/810: chown d0/db/dae 3982 1 2024-03-29T08:07:28.350 INFO:tasks.workunit.client.1.smithi148.stdout:4/856: symlink df/d2c/d51/d77/d26/d34/db3/l10e 0 2024-03-29T08:07:28.353 INFO:tasks.workunit.client.1.smithi148.stdout:8/798: mkdir d17/d74/d10a 0 2024-03-29T08:07:28.354 INFO:tasks.workunit.client.1.smithi148.stdout:1/758: dread d3/de/d25/d35/d42/f38 [0,4194304] 0 2024-03-29T08:07:28.355 INFO:tasks.workunit.client.1.smithi148.stdout:1/759: stat d3/d18/d4d/d76 0 2024-03-29T08:07:28.356 INFO:tasks.workunit.client.1.smithi148.stdout:9/846: mkdir db/d22/d26/d54/d4d/d35/d4b/dc3/d116 0 2024-03-29T08:07:28.359 INFO:tasks.workunit.client.1.smithi148.stdout:2/901: mknod d7/da/dcc/d11c/c124 0 2024-03-29T08:07:28.361 INFO:tasks.workunit.client.1.smithi148.stdout:5/811: creat d0/db/d7b/d16/d2e/ddd/de9/f102 x:0 0 0 2024-03-29T08:07:28.361 INFO:tasks.workunit.client.0.smithi037.stdout:9/756: creat d1/d4/d19/d1e/d4a/d6f/da2/d31/daa/dba/de3/f104 x:0 0 0 2024-03-29T08:07:28.362 INFO:tasks.workunit.client.1.smithi148.stdout:4/857: creat df/d2c/d51/d77/def/f10f x:0 0 0 2024-03-29T08:07:28.363 INFO:tasks.workunit.client.1.smithi148.stdout:1/760: mkdir d3/d18/d4d/d9e/deb 0 2024-03-29T08:07:28.363 INFO:tasks.workunit.client.1.smithi148.stdout:7/782: creat d8/d14/da8/f112 x:0 0 0 2024-03-29T08:07:28.364 INFO:tasks.workunit.client.1.smithi148.stdout:4/858: fsync df/d2c/d51/d77/d26/d9b/fe4 0 2024-03-29T08:07:28.366 INFO:tasks.workunit.client.0.smithi037.stdout:7/601: mkdir dd/d1d/d34/dc8 0 2024-03-29T08:07:28.366 INFO:tasks.workunit.client.1.smithi148.stdout:2/902: mknod d7/d8c/c125 0 2024-03-29T08:07:28.367 INFO:tasks.workunit.client.1.smithi148.stdout:4/859: write df/d2c/db2/fdf [607301,28741] 0 2024-03-29T08:07:28.368 INFO:tasks.workunit.client.1.smithi148.stdout:5/812: creat d0/db/d1e/f103 x:0 0 0 2024-03-29T08:07:28.373 INFO:tasks.workunit.client.0.smithi037.stdout:1/616: truncate d2/d6/dc/d18/d22/d4f/f69 759018 0 2024-03-29T08:07:28.374 INFO:tasks.workunit.client.1.smithi148.stdout:0/814: mkdir db/de/df/d2b/dfa 0 2024-03-29T08:07:28.374 INFO:tasks.workunit.client.1.smithi148.stdout:2/903: symlink d7/d8c/da4/dc8/l126 0 2024-03-29T08:07:28.375 INFO:tasks.workunit.client.1.smithi148.stdout:5/813: fdatasync d0/db/f10 0 2024-03-29T08:07:28.375 INFO:tasks.workunit.client.1.smithi148.stdout:9/847: dwrite db/d22/da3/fcd [0,4194304] 0 2024-03-29T08:07:28.379 INFO:tasks.workunit.client.1.smithi148.stdout:3/861: sync 2024-03-29T08:07:28.379 INFO:tasks.workunit.client.1.smithi148.stdout:7/783: chown d8/c1d 5863 1 2024-03-29T08:07:28.379 INFO:tasks.workunit.client.1.smithi148.stdout:4/860: truncate df/d2c/d51/d77/d26/d34/d43/fc8 987821 0 2024-03-29T08:07:28.380 INFO:tasks.workunit.client.1.smithi148.stdout:1/761: symlink d3/de/lec 0 2024-03-29T08:07:28.382 INFO:tasks.workunit.client.1.smithi148.stdout:6/833: dwrite d6/d9/d1e/d36/d4d/f7a [0,4194304] 0 2024-03-29T08:07:28.385 INFO:tasks.workunit.client.1.smithi148.stdout:5/814: creat d0/db/d1e/d36/f104 x:0 0 0 2024-03-29T08:07:28.386 INFO:tasks.workunit.client.1.smithi148.stdout:8/799: link d17/d49/d9a/dbe/cc8 d17/dce/c10b 0 2024-03-29T08:07:28.389 INFO:tasks.workunit.client.1.smithi148.stdout:3/862: chown de/d24/d33/d39/da3/dbc/ce6 37 1 2024-03-29T08:07:28.392 INFO:tasks.workunit.client.1.smithi148.stdout:2/904: mkdir d7/da/dcc/d11c/d8d/d127 0 2024-03-29T08:07:28.393 INFO:tasks.workunit.client.1.smithi148.stdout:0/815: rmdir db/de/df/d11/d2d/d3e 39 2024-03-29T08:07:28.394 INFO:tasks.workunit.client.1.smithi148.stdout:7/784: truncate d8/f10 1429306 0 2024-03-29T08:07:28.398 INFO:tasks.workunit.client.1.smithi148.stdout:8/800: write d17/d18/d26/d63/d66/d5f/f8e [84594,35915] 0 2024-03-29T08:07:28.401 INFO:tasks.workunit.client.1.smithi148.stdout:7/785: write d8/d38/d3c/da9/ffe [805389,31231] 0 2024-03-29T08:07:28.401 INFO:tasks.workunit.client.1.smithi148.stdout:0/816: readlink db/de/df/d11/d42/ld6 0 2024-03-29T08:07:28.402 INFO:tasks.workunit.client.1.smithi148.stdout:2/905: read d7/da/d33/d3c/ffb [1220799,45867] 0 2024-03-29T08:07:28.403 INFO:tasks.workunit.client.1.smithi148.stdout:1/762: rename d3/d18/d4d/d76/d85/fb0 to d3/de/d21/d5f/fed 0 2024-03-29T08:07:28.406 INFO:tasks.workunit.client.1.smithi148.stdout:6/834: dread d6/d9/d1e/d36/d4d/d9d/d28/dda/db6/df3/fe1 [0,4194304] 0 2024-03-29T08:07:28.407 INFO:tasks.workunit.client.1.smithi148.stdout:3/863: unlink de/d24/d33/d39/ffb 0 2024-03-29T08:07:28.409 INFO:tasks.workunit.client.1.smithi148.stdout:8/801: dread - d17/d74/d86/fd8 zero size 2024-03-29T08:07:28.411 INFO:tasks.workunit.client.1.smithi148.stdout:8/802: chown d17/d18/d26/d63/d7d/le6 1 1 2024-03-29T08:07:28.412 INFO:tasks.workunit.client.1.smithi148.stdout:5/815: rename d0/db/d1e/d6e/da8/fd6 to d0/db/d1e/d35/d42/d67/d76/d81/d98/dd5/d9c/f105 0 2024-03-29T08:07:28.416 INFO:tasks.workunit.client.1.smithi148.stdout:1/763: mkdir d3/de/d1d/dee 0 2024-03-29T08:07:28.422 INFO:tasks.workunit.client.1.smithi148.stdout:3/864: symlink de/d24/d33/d39/d46/d8d/l116 0 2024-03-29T08:07:28.424 INFO:tasks.workunit.client.1.smithi148.stdout:9/848: dread db/d22/d26/d54/d4d/d35/d4b/dc3/fb9 [0,4194304] 0 2024-03-29T08:07:28.426 INFO:tasks.workunit.client.1.smithi148.stdout:7/786: dwrite d8/dfb/d44/d49/f68 [0,4194304] 0 2024-03-29T08:07:28.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:07:28 smithi037 ceph-mon[35162]: from='client.14682 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:07:28.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:07:28 smithi037 ceph-mon[35162]: pgmap v46: 65 pgs: 65 active+clean; 2.7 GiB data, 9.3 GiB used, 527 GiB / 536 GiB avail; 97 MiB/s rd, 256 MiB/s wr, 506 op/s 2024-03-29T08:07:28.428 INFO:tasks.workunit.client.1.smithi148.stdout:8/803: getdents d17/d49/d9a/dbe/d6f/d64/df1 0 2024-03-29T08:07:28.434 INFO:tasks.workunit.client.1.smithi148.stdout:1/764: mkdir d3/de/d25/def 0 2024-03-29T08:07:28.434 INFO:tasks.workunit.client.1.smithi148.stdout:2/906: link d7/d24/d4b/f87 d7/d24/f128 0 2024-03-29T08:07:28.435 INFO:tasks.workunit.client.1.smithi148.stdout:8/804: fsync d17/d74/f54 0 2024-03-29T08:07:28.435 INFO:tasks.workunit.client.1.smithi148.stdout:6/835: write d6/d9/d1e/fbb [2619134,17251] 0 2024-03-29T08:07:28.435 INFO:tasks.workunit.client.1.smithi148.stdout:7/787: fdatasync d8/d14/d45/fbe 0 2024-03-29T08:07:28.438 INFO:tasks.workunit.client.0.smithi037.stdout:1/617: creat d2/d6/dc/d18/d1d/dad/daf/fd4 x:0 0 0 2024-03-29T08:07:28.438 INFO:tasks.workunit.client.1.smithi148.stdout:1/765: mkdir d3/d55/d7e/df0 0 2024-03-29T08:07:28.438 INFO:tasks.workunit.client.1.smithi148.stdout:2/907: write d7/da/d33/f5b [1297638,116653] 0 2024-03-29T08:07:28.439 INFO:tasks.workunit.client.1.smithi148.stdout:8/805: chown d17/d18/d26/d4f/d94/fa2 52883113 1 2024-03-29T08:07:28.439 INFO:tasks.workunit.client.0.smithi037.stdout:3/527: sync 2024-03-29T08:07:28.439 INFO:tasks.workunit.client.0.smithi037.stdout:2/453: sync 2024-03-29T08:07:28.439 INFO:tasks.workunit.client.0.smithi037.stdout:4/585: sync 2024-03-29T08:07:28.440 INFO:tasks.workunit.client.0.smithi037.stdout:5/683: sync 2024-03-29T08:07:28.440 INFO:tasks.workunit.client.0.smithi037.stdout:8/545: sync 2024-03-29T08:07:28.440 INFO:tasks.workunit.client.0.smithi037.stdout:0/452: sync 2024-03-29T08:07:28.441 INFO:tasks.workunit.client.0.smithi037.stdout:6/475: sync 2024-03-29T08:07:28.445 INFO:tasks.workunit.client.1.smithi148.stdout:6/836: chown d6/d9/d1e/d36/d4d/d9d/d28/d98/dc2/cd9 116 1 2024-03-29T08:07:28.446 INFO:tasks.workunit.client.1.smithi148.stdout:1/766: fdatasync d3/d18/d4d/d76/fbb 0 2024-03-29T08:07:28.447 INFO:tasks.workunit.client.1.smithi148.stdout:0/817: dread db/de/df/d2b/f93 [0,4194304] 0 2024-03-29T08:07:28.448 INFO:tasks.workunit.client.1.smithi148.stdout:3/865: dwrite de/d24/d33/d39/da3/fcd [4194304,4194304] 0 2024-03-29T08:07:28.448 INFO:tasks.workunit.client.1.smithi148.stdout:2/908: creat d7/d8c/f129 x:0 0 0 2024-03-29T08:07:28.449 INFO:tasks.workunit.client.0.smithi037.stdout:3/528: mkdir dd/d46/d2d/d3a/d6c/db1 0 2024-03-29T08:07:28.452 INFO:tasks.workunit.client.1.smithi148.stdout:6/837: dread - d6/d9/d1e/d36/d4d/d9d/d28/dda/db4/ff2 zero size 2024-03-29T08:07:28.452 INFO:tasks.workunit.client.0.smithi037.stdout:3/529: fsync dd/d15/d18/d8e/fa7 0 2024-03-29T08:07:28.453 INFO:tasks.workunit.client.1.smithi148.stdout:9/849: dread db/d22/d26/f10e [0,4194304] 0 2024-03-29T08:07:28.453 INFO:tasks.workunit.client.1.smithi148.stdout:7/788: mknod d8/d38/d3c/da5/db4/c113 0 2024-03-29T08:07:28.454 INFO:tasks.workunit.client.1.smithi148.stdout:1/767: write d3/d18/d71/fb2 [4035876,91392] 0 2024-03-29T08:07:28.455 INFO:tasks.workunit.client.0.smithi037.stdout:7/602: link dd/d72/l83 dd/d21/d35/lc9 0 2024-03-29T08:07:28.457 INFO:tasks.workunit.client.1.smithi148.stdout:3/866: chown de/d24/d33/d39/d46/d30/d37/d62/d63/dbf/ff0 636 1 2024-03-29T08:07:28.458 INFO:tasks.workunit.client.0.smithi037.stdout:1/618: rename d2/d6/f36 to d2/d6/dc/d5d/fd5 0 2024-03-29T08:07:28.459 INFO:tasks.workunit.client.1.smithi148.stdout:3/867: readlink de/d24/le7 0 2024-03-29T08:07:28.459 INFO:tasks.workunit.client.1.smithi148.stdout:2/909: mknod d7/d8c/d91/dd5/df3/c12a 0 2024-03-29T08:07:28.461 INFO:tasks.workunit.client.1.smithi148.stdout:6/838: mknod d6/d9/d1e/d27/d48/d9f/c10f 0 2024-03-29T08:07:28.463 INFO:tasks.workunit.client.1.smithi148.stdout:7/789: chown d8/d38/d3c/c6c 169625 1 2024-03-29T08:07:28.463 INFO:tasks.workunit.client.1.smithi148.stdout:6/839: dread - d6/d9/d1e/d36/ff6 zero size 2024-03-29T08:07:28.464 INFO:tasks.workunit.client.1.smithi148.stdout:1/768: mkdir d3/de/d25/df1 0 2024-03-29T08:07:28.467 INFO:tasks.workunit.client.1.smithi148.stdout:9/850: rename db/d22/d26/d54/d4d/d35/d4b/d5a/l85 to db/d10/d9d/l117 0 2024-03-29T08:07:28.468 INFO:tasks.workunit.client.0.smithi037.stdout:8/546: creat d5/d10/d2d/d34/d50/d83/d46/d4e/fb0 x:0 0 0 2024-03-29T08:07:28.468 INFO:tasks.workunit.client.1.smithi148.stdout:3/868: creat de/d24/d33/d39/d46/d70/f117 x:0 0 0 2024-03-29T08:07:28.469 INFO:tasks.workunit.client.1.smithi148.stdout:0/818: dread db/f51 [4194304,4194304] 0 2024-03-29T08:07:28.470 INFO:tasks.workunit.client.1.smithi148.stdout:2/910: mkdir d7/da/d33/d3c/d54/dec/d12b 0 2024-03-29T08:07:28.471 INFO:tasks.workunit.client.1.smithi148.stdout:1/769: creat d3/d18/da8/ff2 x:0 0 0 2024-03-29T08:07:28.471 INFO:tasks.workunit.client.1.smithi148.stdout:6/840: dread d6/d9/d1e/d36/d4d/d9d/d28/dda/db6/df3/fe1 [0,4194304] 0 2024-03-29T08:07:28.473 INFO:tasks.workunit.client.0.smithi037.stdout:5/684: dwrite d8/d18/d24/f2a [0,4194304] 0 2024-03-29T08:07:28.473 INFO:tasks.workunit.client.0.smithi037.stdout:8/547: dread - d5/d10/d45/f7f zero size 2024-03-29T08:07:28.478 INFO:tasks.workunit.client.1.smithi148.stdout:3/869: rename de/d3b/d49/lf2 to de/d24/d33/d39/d7f/l118 0 2024-03-29T08:07:28.479 INFO:tasks.workunit.client.0.smithi037.stdout:3/530: read dd/d15/f62 [5452782,19103] 0 2024-03-29T08:07:28.480 INFO:tasks.workunit.client.0.smithi037.stdout:2/454: dwrite d0/d20/d22/d55/f5b [0,4194304] 0 2024-03-29T08:07:28.481 INFO:tasks.workunit.client.1.smithi148.stdout:3/870: fdatasync de/d24/d33/d39/da3/dbc/fd0 0 2024-03-29T08:07:28.484 INFO:tasks.workunit.client.0.smithi037.stdout:8/548: chown d5/d10/d19/d3d/c63 93936 1 2024-03-29T08:07:28.484 INFO:tasks.workunit.client.0.smithi037.stdout:0/453: rename d9/da/dd/d20/d38/d53/f71 to d9/d74/f97 0 2024-03-29T08:07:28.485 INFO:tasks.workunit.client.1.smithi148.stdout:1/770: rename d3/de/d25/d35 to d3/d18/dae/dda/df3 0 2024-03-29T08:07:28.487 INFO:tasks.workunit.client.1.smithi148.stdout:0/819: fdatasync db/de/df/d11/f41 0 2024-03-29T08:07:28.489 INFO:tasks.workunit.client.1.smithi148.stdout:3/871: readlink de/d24/d33/d39/d46/d8d/ldb 0 2024-03-29T08:07:28.489 INFO:tasks.workunit.client.1.smithi148.stdout:2/911: getdents d7/da/d33/ded/d2e/d37 0 2024-03-29T08:07:28.494 INFO:tasks.workunit.client.1.smithi148.stdout:3/872: write de/d24/d33/d39/d46/d30/d37/d62/d63/dbf/fc8 [7890794,29633] 0 2024-03-29T08:07:28.494 INFO:tasks.workunit.client.1.smithi148.stdout:9/851: dread db/d22/d26/d54/d4d/d35/d4b/d58/f92 [0,4194304] 0 2024-03-29T08:07:28.494 INFO:tasks.workunit.client.1.smithi148.stdout:8/806: write d17/d18/d26/f32 [4667043,35443] 0 2024-03-29T08:07:28.495 INFO:tasks.workunit.client.0.smithi037.stdout:7/603: mkdir dd/d1d/d49/d81/d84/dca 0 2024-03-29T08:07:28.499 INFO:tasks.workunit.client.1.smithi148.stdout:9/852: write db/d10/faf [265232,77465] 0 2024-03-29T08:07:28.505 INFO:tasks.workunit.client.1.smithi148.stdout:9/853: fsync db/d22/d26/d54/d4d/d35/d4b/d58/f8b 0 2024-03-29T08:07:28.506 INFO:tasks.workunit.client.0.smithi037.stdout:5/685: rename d8/d34/c66 to d8/d4d/d8a/ced 0 2024-03-29T08:07:28.508 INFO:tasks.workunit.client.1.smithi148.stdout:1/771: dwrite d3/d55/d7e/fad [0,4194304] 0 2024-03-29T08:07:28.509 INFO:tasks.workunit.client.0.smithi037.stdout:9/757: dwrite d1/d3/d44/f8c [0,4194304] 0 2024-03-29T08:07:28.509 INFO:tasks.workunit.client.0.smithi037.stdout:5/686: write d8/d18/f40 [4538053,71839] 0 2024-03-29T08:07:28.513 INFO:tasks.workunit.client.0.smithi037.stdout:8/549: mknod d5/d10/d2d/d9c/da1/cb1 0 2024-03-29T08:07:28.515 INFO:tasks.workunit.client.1.smithi148.stdout:0/820: dwrite db/de/df/d2b/dae/fbd [0,4194304] 0 2024-03-29T08:07:28.516 INFO:tasks.workunit.client.1.smithi148.stdout:1/772: chown d3/d18/d4d/f74 1 1 2024-03-29T08:07:28.519 INFO:tasks.workunit.client.1.smithi148.stdout:7/790: write d8/d38/d6a/fbd [5167112,113682] 0 2024-03-29T08:07:28.519 INFO:tasks.workunit.client.1.smithi148.stdout:6/841: write d6/d9/d1e/d36/d4d/d9d/d28/f3b [153180,38345] 0 2024-03-29T08:07:28.523 INFO:tasks.workunit.client.1.smithi148.stdout:9/854: creat db/d22/d26/f118 x:0 0 0 2024-03-29T08:07:28.525 INFO:tasks.workunit.client.1.smithi148.stdout:1/773: symlink d3/d55/ddf/lf4 0 2024-03-29T08:07:28.526 INFO:tasks.workunit.client.1.smithi148.stdout:8/807: creat d17/d18/d26/d63/d7d/dc6/f10c x:0 0 0 2024-03-29T08:07:28.526 INFO:tasks.workunit.client.0.smithi037.stdout:5/687: chown d8/d4d/d5c/dbf/fe4 91196 1 2024-03-29T08:07:28.526 INFO:tasks.workunit.client.1.smithi148.stdout:4/861: sync 2024-03-29T08:07:28.527 INFO:tasks.workunit.client.1.smithi148.stdout:9/855: readlink db/d22/d26/d54/d4d/d35/d4b/d8e/la0 0 2024-03-29T08:07:28.530 INFO:tasks.workunit.client.1.smithi148.stdout:0/821: dread - db/de/df/d11/d2d/fb2 zero size 2024-03-29T08:07:28.531 INFO:tasks.workunit.client.0.smithi037.stdout:2/455: symlink d0/d20/l9e 0 2024-03-29T08:07:28.532 INFO:tasks.workunit.client.1.smithi148.stdout:1/774: creat d3/de/d1d/d47/d6d/d72/ff5 x:0 0 0 2024-03-29T08:07:28.533 INFO:tasks.workunit.client.0.smithi037.stdout:4/586: dread d3/d4/d15/d1d/db/d45/d1e/d59/d7d/f9f [0,4194304] 0 2024-03-29T08:07:28.533 INFO:tasks.workunit.client.1.smithi148.stdout:9/856: unlink db/d22/d26/d54/d52/c10c 0 2024-03-29T08:07:28.534 INFO:tasks.workunit.client.1.smithi148.stdout:7/791: rmdir d8/dfb/d44/d50/d94 39 2024-03-29T08:07:28.536 INFO:tasks.workunit.client.1.smithi148.stdout:4/862: truncate df/d2c/d80/daa/fcb 4668011 0 2024-03-29T08:07:28.536 INFO:tasks.workunit.client.1.smithi148.stdout:0/822: dread - db/de/df/d2b/dae/fdb zero size 2024-03-29T08:07:28.538 INFO:tasks.workunit.client.1.smithi148.stdout:9/857: stat db/d10/d2d/d4e/dd4/f1b 0 2024-03-29T08:07:28.538 INFO:tasks.workunit.client.1.smithi148.stdout:8/808: creat d17/d74/d86/f10d x:0 0 0 2024-03-29T08:07:28.539 INFO:tasks.workunit.client.0.smithi037.stdout:6/476: getdents d0/d19/d42/d60 0 2024-03-29T08:07:28.542 INFO:tasks.workunit.client.0.smithi037.stdout:4/587: chown d3/d4/d15/d1d/db/f12 323524 1 2024-03-29T08:07:28.544 INFO:tasks.workunit.client.1.smithi148.stdout:6/842: dwrite d6/d9/d1e/d27/d48/d9f/faa [8388608,4194304] 0 2024-03-29T08:07:28.548 INFO:tasks.workunit.client.0.smithi037.stdout:7/604: dwrite dd/d21/d35/d3b/f97 [0,4194304] 0 2024-03-29T08:07:28.548 INFO:tasks.workunit.client.1.smithi148.stdout:6/843: unlink d6/d69/cd4 0 2024-03-29T08:07:28.551 INFO:tasks.workunit.client.0.smithi037.stdout:3/531: creat dd/d15/d18/fb2 x:0 0 0 2024-03-29T08:07:28.554 INFO:tasks.workunit.client.1.smithi148.stdout:4/863: getdents df/d2c/d51/d77/d26/d34/d3e/df3 0 2024-03-29T08:07:28.555 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:07:28 smithi148 ceph-mon[44992]: from='client.14682 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:07:28.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:07:28 smithi148 ceph-mon[44992]: pgmap v46: 65 pgs: 65 active+clean; 2.7 GiB data, 9.3 GiB used, 527 GiB / 536 GiB avail; 97 MiB/s rd, 256 MiB/s wr, 506 op/s 2024-03-29T08:07:28.556 INFO:tasks.workunit.client.0.smithi037.stdout:4/588: stat d3/d4/d15/d1d/db/c42 0 2024-03-29T08:07:28.561 INFO:tasks.workunit.client.0.smithi037.stdout:0/454: creat d9/da/f98 x:0 0 0 2024-03-29T08:07:28.564 INFO:tasks.workunit.client.0.smithi037.stdout:8/550: symlink d5/d10/d2d/lb2 0 2024-03-29T08:07:28.565 INFO:tasks.workunit.client.1.smithi148.stdout:1/775: write d3/de/f57 [93066,32921] 0 2024-03-29T08:07:28.567 INFO:tasks.workunit.client.1.smithi148.stdout:9/858: write db/d10/d2d/d4e/f81 [4590957,73512] 0 2024-03-29T08:07:28.569 INFO:tasks.workunit.client.1.smithi148.stdout:6/844: dread d6/d9/d1e/d36/d4d/d9d/d28/dda/f92 [0,4194304] 0 2024-03-29T08:07:28.574 INFO:tasks.workunit.client.1.smithi148.stdout:9/859: chown db/d22/d26/d54/d4d/dbb/def/f104 2547 1 2024-03-29T08:07:28.577 INFO:tasks.workunit.client.1.smithi148.stdout:6/845: mknod d6/d9/d1e/d36/d4d/c110 0 2024-03-29T08:07:28.578 INFO:tasks.workunit.client.1.smithi148.stdout:9/860: chown db/d10/d6d/d73/da8/deb/df5 21 1 2024-03-29T08:07:28.578 INFO:tasks.workunit.client.1.smithi148.stdout:0/823: mknod db/dc/daa/d47/cfb 0 2024-03-29T08:07:28.581 INFO:tasks.workunit.client.0.smithi037.stdout:1/619: mknod d2/d6/dc/d18/d1d/dad/daf/cd6 0 2024-03-29T08:07:28.582 INFO:tasks.workunit.client.1.smithi148.stdout:0/824: write db/de/d8e/df1/ff3 [349020,121147] 0 2024-03-29T08:07:28.584 INFO:tasks.workunit.client.1.smithi148.stdout:6/846: symlink d6/d9/d1e/d36/d4d/d9d/d28/dda/db4/l111 0 2024-03-29T08:07:28.585 INFO:tasks.workunit.client.0.smithi037.stdout:0/455: dread d9/da/f3c [0,4194304] 0 2024-03-29T08:07:28.586 INFO:tasks.workunit.client.1.smithi148.stdout:8/809: write d17/d18/f3f [1380722,70545] 0 2024-03-29T08:07:28.586 INFO:tasks.workunit.client.1.smithi148.stdout:1/776: dwrite d3/f5d [0,4194304] 0 2024-03-29T08:07:28.588 INFO:tasks.workunit.client.0.smithi037.stdout:0/456: chown d9/da/dd/d13/f5e 3336888 1 2024-03-29T08:07:28.590 INFO:tasks.workunit.client.0.smithi037.stdout:8/551: mknod d5/d10/d19/d3d/d4a/d4c/d67/d70/cb3 0 2024-03-29T08:07:28.591 INFO:tasks.workunit.client.1.smithi148.stdout:1/777: mkdir d3/d18/dae/df6 0 2024-03-29T08:07:28.591 INFO:tasks.workunit.client.1.smithi148.stdout:7/792: write d8/d38/f71 [1653032,52425] 0 2024-03-29T08:07:28.593 INFO:tasks.workunit.client.1.smithi148.stdout:0/825: symlink db/de/df/d11/d1f/dc5/lfc 0 2024-03-29T08:07:28.597 INFO:tasks.workunit.client.1.smithi148.stdout:8/810: creat d17/d49/d9a/dbe/d6f/d64/f10e x:0 0 0 2024-03-29T08:07:28.597 INFO:tasks.workunit.client.1.smithi148.stdout:1/778: mkdir d3/d55/d7e/df0/df7 0 2024-03-29T08:07:28.599 INFO:tasks.workunit.client.0.smithi037.stdout:0/457: write d9/da/dd/d13/f45 [947672,27828] 0 2024-03-29T08:07:28.599 INFO:tasks.workunit.client.0.smithi037.stdout:1/620: rmdir d2/d6/dc 39 2024-03-29T08:07:28.600 INFO:tasks.workunit.client.1.smithi148.stdout:0/826: write db/dc/daa/d47/f96 [635727,8718] 0 2024-03-29T08:07:28.601 INFO:tasks.workunit.client.0.smithi037.stdout:6/477: creat d0/d6a/d7b/fa1 x:0 0 0 2024-03-29T08:07:28.601 INFO:tasks.workunit.client.0.smithi037.stdout:3/532: symlink dd/d15/d18/d4b/da2/lb3 0 2024-03-29T08:07:28.603 INFO:tasks.workunit.client.0.smithi037.stdout:4/589: mknod d3/d4/d15/d1d/db/d45/d1e/d59/cbd 0 2024-03-29T08:07:28.604 INFO:tasks.workunit.client.0.smithi037.stdout:9/758: dwrite d1/d4/d19/d1e/d4a/d6f/da2/f7a [0,4194304] 0 2024-03-29T08:07:28.605 INFO:tasks.workunit.client.0.smithi037.stdout:8/552: mknod d5/d10/d19/d8d/cb4 0 2024-03-29T08:07:28.608 INFO:tasks.workunit.client.0.smithi037.stdout:0/458: dread - d9/da/dd/d20/d38/d8a/f8e zero size 2024-03-29T08:07:28.608 INFO:tasks.workunit.client.1.smithi148.stdout:1/779: dread - d3/d18/d71/fa7 zero size 2024-03-29T08:07:28.612 INFO:tasks.workunit.client.0.smithi037.stdout:5/688: dwrite d8/d18/d24/fa6 [0,4194304] 0 2024-03-29T08:07:28.613 INFO:tasks.workunit.client.0.smithi037.stdout:2/456: write d0/d23/d70/f76 [815711,98742] 0 2024-03-29T08:07:28.613 INFO:tasks.workunit.client.0.smithi037.stdout:3/533: rmdir dd/d15/d27/d33/d83 39 2024-03-29T08:07:28.613 INFO:tasks.workunit.client.1.smithi148.stdout:6/847: dwrite d6/fe [4194304,4194304] 0 2024-03-29T08:07:28.615 INFO:tasks.workunit.client.0.smithi037.stdout:6/478: symlink d0/d6a/d7b/d77/la2 0 2024-03-29T08:07:28.618 INFO:tasks.workunit.client.1.smithi148.stdout:1/780: read - d3/de/d1d/d47/d6d/d72/ff5 zero size 2024-03-29T08:07:28.620 INFO:tasks.workunit.client.0.smithi037.stdout:9/759: symlink d1/d3/d44/l105 0 2024-03-29T08:07:28.620 INFO:tasks.workunit.client.1.smithi148.stdout:9/861: dwrite db/d22/d26/f118 [0,4194304] 0 2024-03-29T08:07:28.622 INFO:tasks.workunit.client.0.smithi037.stdout:8/553: symlink d5/d10/d2d/d34/d50/lb5 0 2024-03-29T08:07:28.624 INFO:tasks.workunit.client.0.smithi037.stdout:5/689: unlink d8/d4d/d5c/f62 0 2024-03-29T08:07:28.626 INFO:tasks.workunit.client.1.smithi148.stdout:4/864: dread df/d2c/d51/d77/d26/d9b/da7/f35 [0,4194304] 0 2024-03-29T08:07:28.627 INFO:tasks.workunit.client.1.smithi148.stdout:6/848: symlink d6/d9/d1e/d27/d48/d9f/l112 0 2024-03-29T08:07:28.628 INFO:tasks.workunit.client.0.smithi037.stdout:3/534: symlink dd/d5e/d7a/lb4 0 2024-03-29T08:07:28.629 INFO:tasks.workunit.client.1.smithi148.stdout:9/862: creat db/d10/d2d/d4e/d5f/d6e/f119 x:0 0 0 2024-03-29T08:07:28.631 INFO:tasks.workunit.client.1.smithi148.stdout:1/781: chown d3/de/d25/d51/ld2 3266 1 2024-03-29T08:07:28.633 INFO:tasks.workunit.client.0.smithi037.stdout:6/479: mkdir d0/d4f/da3 0 2024-03-29T08:07:28.633 INFO:tasks.workunit.client.1.smithi148.stdout:7/793: dwrite d8/d14/d45/d4d/df7/f110 [0,4194304] 0 2024-03-29T08:07:28.635 INFO:tasks.workunit.client.0.smithi037.stdout:9/760: truncate d1/d1f/f5f 1115151 0 2024-03-29T08:07:28.637 INFO:tasks.workunit.client.1.smithi148.stdout:6/849: mknod d6/d9/d1e/d20/d30/c113 0 2024-03-29T08:07:28.638 INFO:tasks.workunit.client.0.smithi037.stdout:5/690: mkdir d8/d22/d37/d8d/db3/dee 0 2024-03-29T08:07:28.639 INFO:tasks.workunit.client.1.smithi148.stdout:1/782: truncate d3/d55/d7e/fd7 706595 0 2024-03-29T08:07:28.639 INFO:tasks.workunit.client.0.smithi037.stdout:9/761: fsync d1/d4/d19/d1e/d71/ffa 0 2024-03-29T08:07:28.640 INFO:tasks.workunit.client.0.smithi037.stdout:8/554: creat d5/d10/d2d/d34/d50/d83/d46/d4e/fb6 x:0 0 0 2024-03-29T08:07:28.641 INFO:tasks.workunit.client.0.smithi037.stdout:1/621: creat d2/d6/fd7 x:0 0 0 2024-03-29T08:07:28.645 INFO:tasks.workunit.client.1.smithi148.stdout:9/863: creat db/d10/d2d/d4e/da6/f11a x:0 0 0 2024-03-29T08:07:28.646 INFO:tasks.workunit.client.0.smithi037.stdout:0/459: creat d9/da/dd/f99 x:0 0 0 2024-03-29T08:07:28.650 INFO:tasks.workunit.client.0.smithi037.stdout:6/480: mknod d0/d5/d1a/d61/ca4 0 2024-03-29T08:07:28.650 INFO:tasks.workunit.client.1.smithi148.stdout:1/783: creat d3/d18/d71/ddc/ff8 x:0 0 0 2024-03-29T08:07:28.650 INFO:tasks.workunit.client.1.smithi148.stdout:6/850: mkdir d6/d9/d1e/d36/d4d/d9d/d28/db3/d114 0 2024-03-29T08:07:28.651 INFO:tasks.workunit.client.1.smithi148.stdout:9/864: fsync db/d22/d26/d54/d4d/d35/d4b/dc3/fb5 0 2024-03-29T08:07:28.652 INFO:tasks.workunit.client.0.smithi037.stdout:4/590: getdents d3/d4/d15/d1d/d4f 0 2024-03-29T08:07:28.652 INFO:tasks.workunit.client.0.smithi037.stdout:9/762: rename d1/d4/d19/d1e/fae to d1/d1f/d3a/d9b/ddd/f106 0 2024-03-29T08:07:28.653 INFO:tasks.workunit.client.1.smithi148.stdout:1/784: fdatasync d3/d18/d4d/d8f/f6b 0 2024-03-29T08:07:28.657 INFO:tasks.workunit.client.0.smithi037.stdout:4/591: write d3/d54/f99 [2043375,10470] 0 2024-03-29T08:07:28.658 INFO:tasks.workunit.client.0.smithi037.stdout:6/481: mknod d0/d6a/d7b/d77/ca5 0 2024-03-29T08:07:28.661 INFO:tasks.workunit.client.0.smithi037.stdout:6/482: chown d0/d6a/f85 169106 1 2024-03-29T08:07:28.663 INFO:tasks.workunit.client.0.smithi037.stdout:6/483: stat d0/d19/d42/c7e 0 2024-03-29T08:07:28.665 INFO:tasks.workunit.client.1.smithi148.stdout:7/794: dwrite d8/dfb/f109 [0,4194304] 0 2024-03-29T08:07:28.673 INFO:tasks.workunit.client.1.smithi148.stdout:5/816: sync 2024-03-29T08:07:28.674 INFO:tasks.workunit.client.1.smithi148.stdout:4/865: dwrite df/d2c/d51/f57 [0,4194304] 0 2024-03-29T08:07:28.679 INFO:tasks.workunit.client.1.smithi148.stdout:8/811: dread f7 [0,4194304] 0 2024-03-29T08:07:28.680 INFO:tasks.workunit.client.1.smithi148.stdout:0/827: dread db/dc/daa/d47/daf/f99 [0,4194304] 0 2024-03-29T08:07:28.686 INFO:tasks.workunit.client.1.smithi148.stdout:4/866: creat df/d60/f110 x:0 0 0 2024-03-29T08:07:28.686 INFO:tasks.workunit.client.1.smithi148.stdout:5/817: dread - d0/db/d1e/d6e/da8/f100 zero size 2024-03-29T08:07:28.692 INFO:tasks.workunit.client.1.smithi148.stdout:1/785: dwrite d3/d55/f43 [0,4194304] 0 2024-03-29T08:07:28.700 INFO:tasks.workunit.client.1.smithi148.stdout:4/867: write df/d2c/d51/d77/d26/d34/d3e/f58 [4507689,60199] 0 2024-03-29T08:07:28.705 INFO:tasks.workunit.client.1.smithi148.stdout:9/865: dwrite db/d10/d2d/d4e/dd4/f1e [0,4194304] 0 2024-03-29T08:07:28.713 INFO:tasks.workunit.client.1.smithi148.stdout:4/868: mknod df/d2c/d51/d77/d26/d9b/d10a/c111 0 2024-03-29T08:07:28.719 INFO:tasks.workunit.client.1.smithi148.stdout:9/866: creat db/d22/d26/d54/d4d/dbb/f11b x:0 0 0 2024-03-29T08:07:28.720 INFO:tasks.workunit.client.0.smithi037.stdout:0/460: rmdir d9/da/dd/d59 39 2024-03-29T08:07:28.722 INFO:tasks.workunit.client.0.smithi037.stdout:0/461: write d9/da/f8d [2386967,27711] 0 2024-03-29T08:07:28.724 INFO:tasks.workunit.client.1.smithi148.stdout:1/786: dwrite d3/d18/da8/ff2 [0,4194304] 0 2024-03-29T08:07:28.726 INFO:tasks.workunit.client.1.smithi148.stdout:5/818: dwrite d0/db/d7b/d16/f7a [0,4194304] 0 2024-03-29T08:07:28.729 INFO:tasks.workunit.client.1.smithi148.stdout:7/795: rename d8/d38/d67/dc1 to d8/d38/d3c/da5/db4/de9/d114 0 2024-03-29T08:07:28.730 INFO:tasks.workunit.client.0.smithi037.stdout:9/763: chown d1/d4/d19/d1e/d4a/d6f/da2/d13/c5a 0 1 2024-03-29T08:07:28.731 INFO:tasks.workunit.client.1.smithi148.stdout:5/819: mknod d0/c106 0 2024-03-29T08:07:28.733 INFO:tasks.workunit.client.1.smithi148.stdout:0/828: creat db/de/df/d11/d2d/da3/ffd x:0 0 0 2024-03-29T08:07:28.735 INFO:tasks.workunit.client.1.smithi148.stdout:1/787: mkdir d3/de/d6e/df9 0 2024-03-29T08:07:28.737 INFO:tasks.workunit.client.1.smithi148.stdout:1/788: stat d3/de/d1d/d47/d6d/d84/lcc 0 2024-03-29T08:07:28.739 INFO:tasks.workunit.client.0.smithi037.stdout:2/457: write d0/f7 [1536379,11614] 0 2024-03-29T08:07:28.740 INFO:tasks.workunit.client.1.smithi148.stdout:8/812: creat d17/d18/d26/d63/f10f x:0 0 0 2024-03-29T08:07:28.742 INFO:tasks.workunit.client.1.smithi148.stdout:1/789: dread - d3/fd6 zero size 2024-03-29T08:07:28.743 INFO:tasks.workunit.client.1.smithi148.stdout:8/813: fsync d17/d18/fe4 0 2024-03-29T08:07:28.748 INFO:tasks.workunit.client.0.smithi037.stdout:3/535: write dd/d15/d27/d33/f70 [24247,107839] 0 2024-03-29T08:07:28.748 INFO:tasks.workunit.client.1.smithi148.stdout:8/814: stat d17/d18/d26/d63/d66/d5f 0 2024-03-29T08:07:28.749 INFO:tasks.workunit.client.1.smithi148.stdout:5/820: link d0/f9 d0/db/d84/daf/de1/f107 0 2024-03-29T08:07:28.750 INFO:tasks.workunit.client.0.smithi037.stdout:6/484: creat d0/d5/d1a/fa6 x:0 0 0 2024-03-29T08:07:28.753 INFO:tasks.workunit.client.1.smithi148.stdout:5/821: chown d0/l2b 376 1 2024-03-29T08:07:28.754 INFO:tasks.workunit.client.0.smithi037.stdout:5/691: write d8/d22/d4f/d70/f3f [5083434,60487] 0 2024-03-29T08:07:28.754 INFO:tasks.workunit.client.1.smithi148.stdout:2/912: sync 2024-03-29T08:07:28.754 INFO:tasks.workunit.client.1.smithi148.stdout:3/873: sync 2024-03-29T08:07:28.757 INFO:tasks.workunit.client.0.smithi037.stdout:8/555: link d5/d10/d2d/d34/d50/d83/c75 d5/d10/d19/d3d/d4a/d4c/cb7 0 2024-03-29T08:07:28.757 INFO:tasks.workunit.client.0.smithi037.stdout:5/692: write d8/d22/d4f/d70/f3f [9423480,109160] 0 2024-03-29T08:07:28.758 INFO:tasks.workunit.client.1.smithi148.stdout:9/867: dwrite db/d10/d2d/d4e/d5f/d6e/d7a/ff7 [0,4194304] 0 2024-03-29T08:07:28.759 INFO:tasks.workunit.client.1.smithi148.stdout:8/815: fdatasync d17/d18/d26/d63/d7d/dc6/f10c 0 2024-03-29T08:07:28.760 INFO:tasks.workunit.client.0.smithi037.stdout:3/536: dread dd/f1b [0,4194304] 0 2024-03-29T08:07:28.766 INFO:tasks.workunit.client.1.smithi148.stdout:9/868: dread - db/d10/d2d/d4e/d5f/dd5/f111 zero size 2024-03-29T08:07:28.767 INFO:tasks.workunit.client.1.smithi148.stdout:7/796: write d8/dfb/d44/d50/d85/fda [299368,115556] 0 2024-03-29T08:07:28.768 INFO:tasks.workunit.client.1.smithi148.stdout:2/913: mknod d7/da/d4c/dd1/c12c 0 2024-03-29T08:07:28.771 INFO:tasks.workunit.client.1.smithi148.stdout:5/822: symlink d0/db/d1e/d36/l108 0 2024-03-29T08:07:28.771 INFO:tasks.workunit.client.1.smithi148.stdout:0/829: mkdir db/de/df/d2b/dfa/dfe 0 2024-03-29T08:07:28.774 INFO:tasks.workunit.client.1.smithi148.stdout:2/914: chown d7/da/d33/d45/f6b 1947546 1 2024-03-29T08:07:28.775 INFO:tasks.workunit.client.1.smithi148.stdout:9/869: unlink db/d22/d26/d54/f49 0 2024-03-29T08:07:28.775 INFO:tasks.workunit.client.0.smithi037.stdout:4/592: mkdir d3/d4/d15/d1d/db/d45/d1e/d59/db5/dbe 0 2024-03-29T08:07:28.777 INFO:tasks.workunit.client.1.smithi148.stdout:8/816: mkdir d17/dce/d110 0 2024-03-29T08:07:28.779 INFO:tasks.workunit.client.0.smithi037.stdout:4/593: dread - d3/d54/d88/f9a zero size 2024-03-29T08:07:28.781 INFO:tasks.workunit.client.0.smithi037.stdout:5/693: dwrite d8/d18/d24/fd8 [0,4194304] 0 2024-03-29T08:07:28.781 INFO:tasks.workunit.client.1.smithi148.stdout:4/869: dread df/d2c/d51/d77/d26/f3d [0,4194304] 0 2024-03-29T08:07:28.783 INFO:tasks.workunit.client.1.smithi148.stdout:2/915: mknod d7/da/dcc/d11c/d8d/c12d 0 2024-03-29T08:07:28.787 INFO:tasks.workunit.client.1.smithi148.stdout:5/823: rename d0/db/d1e/d36/d41/ccc to d0/db/df2/c109 0 2024-03-29T08:07:28.789 INFO:tasks.workunit.client.1.smithi148.stdout:5/824: chown d0/db/d7b/d16/d2e 241 1 2024-03-29T08:07:28.791 INFO:tasks.workunit.client.0.smithi037.stdout:2/458: mknod d0/d23/d70/d71/d78/c9f 0 2024-03-29T08:07:28.791 INFO:tasks.workunit.client.1.smithi148.stdout:4/870: fdatasync df/d2c/d51/d77/d17/fbd 0 2024-03-29T08:07:28.797 INFO:tasks.workunit.client.1.smithi148.stdout:9/870: mknod db/d22/d26/c11c 0 2024-03-29T08:07:28.798 INFO:tasks.workunit.client.1.smithi148.stdout:4/871: chown df/d2c/d51/d77/d26/d34/db3 533278 1 2024-03-29T08:07:28.800 INFO:tasks.workunit.client.0.smithi037.stdout:9/764: rename d1/d4/d19/d1e/d4a/d6f/da2/d13/c7b to d1/d4/d19/d1e/d4a/d6f/da2/d31/daa/dba/c107 0 2024-03-29T08:07:28.803 INFO:tasks.workunit.client.1.smithi148.stdout:2/916: mknod d7/da/d33/d3c/c12e 0 2024-03-29T08:07:28.807 INFO:tasks.workunit.client.0.smithi037.stdout:4/594: dread d3/d4/d15/d1d/db/d45/d1e/d5f/f62 [0,4194304] 0 2024-03-29T08:07:28.807 INFO:tasks.workunit.client.1.smithi148.stdout:9/871: chown db/d10/d2d/lbf 26 1 2024-03-29T08:07:28.810 INFO:tasks.workunit.client.1.smithi148.stdout:5/825: rename d0/db/d1e/d35/d42/d96/ca5 to d0/db/c10a 0 2024-03-29T08:07:28.813 INFO:tasks.workunit.client.1.smithi148.stdout:3/874: dwrite de/d3b/fb8 [0,4194304] 0 2024-03-29T08:07:28.816 INFO:tasks.workunit.client.1.smithi148.stdout:0/830: link db/de/df/d11/d42/l65 db/de/df/d11/d2d/dde/lff 0 2024-03-29T08:07:28.816 INFO:tasks.workunit.client.1.smithi148.stdout:3/875: fdatasync de/d24/d33/f94 0 2024-03-29T08:07:28.818 INFO:tasks.workunit.client.1.smithi148.stdout:5/826: mkdir d0/db/d1e/d35/d42/d10b 0 2024-03-29T08:07:28.818 INFO:tasks.workunit.client.0.smithi037.stdout:0/462: creat d9/da/dd/d20/d38/d8a/f9a x:0 0 0 2024-03-29T08:07:28.819 INFO:tasks.workunit.client.1.smithi148.stdout:0/831: stat db/dc/daa/d47/f2c 0 2024-03-29T08:07:28.823 INFO:tasks.workunit.client.1.smithi148.stdout:4/872: truncate df/d2c/d51/d77/d26/f4a 149556 0 2024-03-29T08:07:28.825 INFO:tasks.workunit.client.1.smithi148.stdout:4/873: readlink df/d60/la5 0 2024-03-29T08:07:28.826 INFO:tasks.workunit.client.1.smithi148.stdout:7/797: getdents d8/dfb 0 2024-03-29T08:07:28.827 INFO:tasks.workunit.client.0.smithi037.stdout:8/556: dread d5/d10/d19/d3d/d4a/f6f [0,4194304] 0 2024-03-29T08:07:28.827 INFO:tasks.workunit.client.1.smithi148.stdout:5/827: mkdir d0/db/df7/d10c 0 2024-03-29T08:07:28.828 INFO:tasks.workunit.client.1.smithi148.stdout:6/851: sync 2024-03-29T08:07:28.828 INFO:tasks.workunit.client.0.smithi037.stdout:5/694: dread - d8/d22/d37/d8d/db3/fb8 zero size 2024-03-29T08:07:28.830 INFO:tasks.workunit.client.1.smithi148.stdout:0/832: rmdir db/dc/daa/d47 39 2024-03-29T08:07:28.831 INFO:tasks.workunit.client.1.smithi148.stdout:3/876: mkdir de/d24/d119 0 2024-03-29T08:07:28.833 INFO:tasks.workunit.client.1.smithi148.stdout:9/872: write db/d22/d26/f38 [1701430,19455] 0 2024-03-29T08:07:28.835 INFO:tasks.workunit.client.0.smithi037.stdout:1/622: fsync d2/d6/dc/d58/f59 0 2024-03-29T08:07:28.837 INFO:tasks.workunit.client.1.smithi148.stdout:6/852: rmdir d6/d9 39 2024-03-29T08:07:28.840 INFO:tasks.workunit.client.1.smithi148.stdout:7/798: mkdir d8/dfb/db8/d115 0 2024-03-29T08:07:28.840 INFO:tasks.workunit.client.1.smithi148.stdout:0/833: truncate db/de/df/d14/fe2 409787 0 2024-03-29T08:07:28.840 INFO:tasks.workunit.client.1.smithi148.stdout:9/873: chown db/d22/d26/d54/d4d/d35/d4b/dc3 51835 1 2024-03-29T08:07:28.840 INFO:tasks.workunit.client.1.smithi148.stdout:3/877: dread - de/d24/d33/d39/d46/d70/f117 zero size 2024-03-29T08:07:28.843 INFO:tasks.workunit.client.0.smithi037.stdout:1/623: read d2/d4b/f66 [510731,113084] 0 2024-03-29T08:07:28.844 INFO:tasks.workunit.client.1.smithi148.stdout:4/874: unlink df/d2c/d51/d77/d26/d9b/da7/d7c/feb 0 2024-03-29T08:07:28.846 INFO:tasks.workunit.client.1.smithi148.stdout:1/790: sync 2024-03-29T08:07:28.846 INFO:tasks.workunit.client.0.smithi037.stdout:0/463: dwrite d9/d74/d7d/f87 [0,4194304] 0 2024-03-29T08:07:28.847 INFO:tasks.workunit.client.0.smithi037.stdout:2/459: rename d0/d20/d7d/f8f to d0/d20/d22/fa0 0 2024-03-29T08:07:28.853 INFO:tasks.workunit.client.0.smithi037.stdout:6/485: dread d0/d5/fa [0,4194304] 0 2024-03-29T08:07:28.855 INFO:tasks.workunit.client.1.smithi148.stdout:3/878: truncate de/d24/d33/d39/d46/f65 882634 0 2024-03-29T08:07:28.856 INFO:tasks.workunit.client.1.smithi148.stdout:1/791: read d3/d18/d4d/f50 [1503726,58350] 0 2024-03-29T08:07:28.856 INFO:tasks.workunit.client.0.smithi037.stdout:6/486: chown d0/d5/d3f/f96 9603348 1 2024-03-29T08:07:28.857 INFO:tasks.workunit.client.1.smithi148.stdout:6/853: fsync d6/d9/d1e/d36/db1/fe0 0 2024-03-29T08:07:28.861 INFO:tasks.workunit.client.0.smithi037.stdout:9/765: symlink d1/d4/d19/d1e/d4a/d6f/da2/d31/d87/d88/l108 0 2024-03-29T08:07:28.861 INFO:tasks.workunit.client.1.smithi148.stdout:9/874: rename db/d22/d26/d54/d4d/d35/d4b/d58/df1/cfd to db/d22/d26/d54/d52/d9c/c11d 0 2024-03-29T08:07:28.861 INFO:tasks.workunit.client.1.smithi148.stdout:0/834: dread db/de/df/d11/d1f/fe8 [0,4194304] 0 2024-03-29T08:07:28.872 INFO:tasks.workunit.client.1.smithi148.stdout:1/792: mkdir d3/de/d6e/df9/dfa 0 2024-03-29T08:07:28.873 INFO:tasks.workunit.client.0.smithi037.stdout:5/695: mknod d8/d4d/d8a/cef 0 2024-03-29T08:07:28.874 INFO:tasks.workunit.client.1.smithi148.stdout:2/917: dread d7/da/d33/f35 [0,4194304] 0 2024-03-29T08:07:28.875 INFO:tasks.workunit.client.1.smithi148.stdout:4/875: truncate df/d2c/d51/d77/d26/d9b/da7/f94 1015911 0 2024-03-29T08:07:28.877 INFO:tasks.workunit.client.1.smithi148.stdout:6/854: write d6/d9/d1e/d36/db1/f7c [5119656,54203] 0 2024-03-29T08:07:28.881 INFO:tasks.workunit.client.1.smithi148.stdout:1/793: readlink d3/d18/d71/le6 0 2024-03-29T08:07:28.883 INFO:tasks.workunit.client.1.smithi148.stdout:3/879: link de/d24/d33/d39/d46/d30/d37/d62/fff de/d24/d33/d39/da3/dbc/f11a 0 2024-03-29T08:07:28.890 INFO:tasks.workunit.client.0.smithi037.stdout:0/464: dread d9/da/dd/d59/f39 [0,4194304] 0 2024-03-29T08:07:28.893 INFO:tasks.workunit.client.0.smithi037.stdout:5/696: creat d8/d22/d4f/d70/ff0 x:0 0 0 2024-03-29T08:07:28.894 INFO:tasks.workunit.client.1.smithi148.stdout:5/828: dwrite d0/db/d1e/d5b/d66/d73/f8b [0,4194304] 0 2024-03-29T08:07:28.895 INFO:tasks.workunit.client.0.smithi037.stdout:6/487: link d0/d5/f2a d0/d5/fa7 0 2024-03-29T08:07:28.895 INFO:tasks.workunit.client.1.smithi148.stdout:3/880: creat de/d24/d119/f11b x:0 0 0 2024-03-29T08:07:28.897 INFO:tasks.workunit.client.1.smithi148.stdout:1/794: chown d3/d18/dae/dda/df3/d42/l77 857 1 2024-03-29T08:07:28.901 INFO:tasks.workunit.client.0.smithi037.stdout:5/697: mkdir d8/d22/db6/df1 0 2024-03-29T08:07:28.903 INFO:tasks.workunit.client.1.smithi148.stdout:0/835: dread db/de/f70 [0,4194304] 0 2024-03-29T08:07:28.906 INFO:tasks.workunit.client.1.smithi148.stdout:9/875: dwrite db/d10/f67 [0,4194304] 0 2024-03-29T08:07:28.908 INFO:tasks.workunit.client.1.smithi148.stdout:5/829: chown d0/c106 2921731 1 2024-03-29T08:07:28.917 INFO:tasks.workunit.client.0.smithi037.stdout:8/557: symlink d5/d10/d19/d3d/d4a/lb8 0 2024-03-29T08:07:28.923 INFO:tasks.workunit.client.1.smithi148.stdout:2/918: dwrite d7/da/d33/d3c/f28 [8388608,4194304] 0 2024-03-29T08:07:28.923 INFO:tasks.workunit.client.0.smithi037.stdout:6/488: dread d0/d19/d34/d4e/f54 [0,4194304] 0 2024-03-29T08:07:28.927 INFO:tasks.workunit.client.1.smithi148.stdout:5/830: unlink d0/db/d7b/d16/d87/f90 0 2024-03-29T08:07:28.927 INFO:tasks.workunit.client.1.smithi148.stdout:8/817: truncate d17/d49/d9a/dbe/d6f/d64/da5/fda 2800031 0 2024-03-29T08:07:28.930 INFO:tasks.workunit.client.0.smithi037.stdout:8/558: symlink d5/d10/lb9 0 2024-03-29T08:07:28.930 INFO:tasks.workunit.client.1.smithi148.stdout:4/876: dwrite df/d2c/d51/d77/d26/f3b [0,4194304] 0 2024-03-29T08:07:28.930 INFO:tasks.workunit.client.1.smithi148.stdout:8/818: chown d17/d18/d26/d63/d7d/dc6 318688423 1 2024-03-29T08:07:28.933 INFO:tasks.workunit.client.1.smithi148.stdout:8/819: readlink d17/d74/d86/lcf 0 2024-03-29T08:07:28.937 INFO:tasks.workunit.client.0.smithi037.stdout:6/489: rename d0/d4f to d0/d5/d1f/da8 0 2024-03-29T08:07:28.937 INFO:tasks.workunit.client.1.smithi148.stdout:9/876: mkdir db/d22/d26/d54/d4d/d35/d4b/d8e/ddf/d11e 0 2024-03-29T08:07:28.938 INFO:tasks.workunit.client.1.smithi148.stdout:2/919: dread d7/da/d33/d3c/d54/dec/f8f [0,4194304] 0 2024-03-29T08:07:28.938 INFO:tasks.workunit.client.1.smithi148.stdout:4/877: symlink df/d2c/d80/da6/l112 0 2024-03-29T08:07:28.939 INFO:tasks.workunit.client.0.smithi037.stdout:9/766: dread d1/d4/d19/d1e/d4a/d6f/da2/d23/f37 [4194304,4194304] 0 2024-03-29T08:07:28.940 INFO:tasks.workunit.client.0.smithi037.stdout:2/460: symlink d0/d23/d70/la1 0 2024-03-29T08:07:28.948 INFO:tasks.workunit.client.1.smithi148.stdout:0/836: getdents db/dc/daa/d47/d4c 0 2024-03-29T08:07:28.948 INFO:tasks.workunit.client.1.smithi148.stdout:3/881: getdents de/d24/d33/d39/d46/d70/def 0 2024-03-29T08:07:28.949 INFO:tasks.workunit.client.1.smithi148.stdout:2/920: stat d7/da/d52/lf9 0 2024-03-29T08:07:28.949 INFO:tasks.workunit.client.0.smithi037.stdout:6/490: mknod d0/ca9 0 2024-03-29T08:07:28.950 INFO:tasks.workunit.client.1.smithi148.stdout:5/831: mkdir d0/db/d1e/d35/d42/d67/d76/d81/d98/dd5/d9c/dc8/d10d 0 2024-03-29T08:07:28.950 INFO:tasks.workunit.client.1.smithi148.stdout:8/820: write d17/d49/d9a/dbe/d6f/d64/df1/f106 [410296,124158] 0 2024-03-29T08:07:28.950 INFO:tasks.workunit.client.1.smithi148.stdout:7/799: truncate d8/d14/da8/fdc 2298134 0 2024-03-29T08:07:28.951 INFO:tasks.workunit.client.1.smithi148.stdout:4/878: stat df/d2c/d51/d77/d17/f4f 0 2024-03-29T08:07:28.952 INFO:tasks.workunit.client.0.smithi037.stdout:2/461: fdatasync d0/d20/d2e/d5f/d6f/d5e/f74 0 2024-03-29T08:07:28.957 INFO:tasks.workunit.client.1.smithi148.stdout:4/879: readlink df/d2c/d80/daa/lb9 0 2024-03-29T08:07:28.959 INFO:tasks.workunit.client.0.smithi037.stdout:1/624: truncate d2/d6/dc/d5d/d62/f97 3562102 0 2024-03-29T08:07:28.959 INFO:tasks.workunit.client.1.smithi148.stdout:7/800: truncate d8/dfb/d44/d50/ff9 58563 0 2024-03-29T08:07:28.960 INFO:tasks.workunit.client.0.smithi037.stdout:4/595: dwrite d3/d4/d27/f86 [0,4194304] 0 2024-03-29T08:07:28.965 INFO:tasks.workunit.client.0.smithi037.stdout:1/625: write d2/d6/dc/d18/d22/da0/fcd [899872,127920] 0 2024-03-29T08:07:28.965 INFO:tasks.workunit.client.1.smithi148.stdout:9/877: mknod db/d22/d26/d54/d4d/dbb/def/c11f 0 2024-03-29T08:07:28.967 INFO:tasks.workunit.client.1.smithi148.stdout:9/878: chown db/d10/d2d/d4e/dd4/f1e 61 1 2024-03-29T08:07:28.968 INFO:tasks.workunit.client.1.smithi148.stdout:9/879: fsync f3 0 2024-03-29T08:07:28.969 INFO:tasks.workunit.client.1.smithi148.stdout:3/882: rename de/d24/d33/d39/da3/fcc to de/d24/d33/d39/da3/f11c 0 2024-03-29T08:07:28.971 INFO:tasks.workunit.client.0.smithi037.stdout:9/767: link d1/d1f/ccf d1/d4/d19/d1e/d4a/d6f/da2/d31/d87/c109 0 2024-03-29T08:07:28.971 INFO:tasks.workunit.client.1.smithi148.stdout:2/921: mkdir d7/da/dcc/d11c/d8d/d127/d12f 0 2024-03-29T08:07:28.974 INFO:tasks.workunit.client.1.smithi148.stdout:9/880: fdatasync db/d22/d26/d54/d4d/d35/d4b/dc3/fb5 0 2024-03-29T08:07:28.975 INFO:tasks.workunit.client.1.smithi148.stdout:0/837: chown db/de/df/d11/d42/le4 25 1 2024-03-29T08:07:28.975 INFO:tasks.workunit.client.1.smithi148.stdout:5/832: getdents d0/db/d1e/d5b/d66 0 2024-03-29T08:07:28.976 INFO:tasks.workunit.client.1.smithi148.stdout:4/880: getdents df/d60 0 2024-03-29T08:07:28.977 INFO:tasks.workunit.client.1.smithi148.stdout:8/821: creat d17/d18/d26/d63/d7d/dc6/f111 x:0 0 0 2024-03-29T08:07:28.977 INFO:tasks.workunit.client.0.smithi037.stdout:4/596: creat d3/d4/d15/d1d/db/d72/d98/fbf x:0 0 0 2024-03-29T08:07:28.977 INFO:tasks.workunit.client.0.smithi037.stdout:9/768: stat d1/d4/d19/d1e/d4a/d6f/d96/fa4 0 2024-03-29T08:07:28.978 INFO:tasks.workunit.client.1.smithi148.stdout:1/795: dread d3/d18/dae/dda/df3/fd0 [0,4194304] 0 2024-03-29T08:07:28.980 INFO:tasks.workunit.client.1.smithi148.stdout:7/801: symlink d8/df4/l116 0 2024-03-29T08:07:28.981 INFO:tasks.workunit.client.1.smithi148.stdout:4/881: fsync df/d2c/d51/d77/d26/d9b/fdc 0 2024-03-29T08:07:28.981 INFO:tasks.workunit.client.0.smithi037.stdout:6/491: getdents d0/d19/d34/d4e/d94 0 2024-03-29T08:07:28.982 INFO:tasks.workunit.client.0.smithi037.stdout:4/597: chown d3/d4/d15/d1d/d4f 463971 1 2024-03-29T08:07:28.983 INFO:tasks.workunit.client.0.smithi037.stdout:1/626: mkdir d2/d6/dd8 0 2024-03-29T08:07:28.986 INFO:tasks.workunit.client.0.smithi037.stdout:9/769: readlink d1/d4/d19/d1e/d4a/d6f/da2/d31/d87/d88/l108 0 2024-03-29T08:07:28.986 INFO:tasks.workunit.client.1.smithi148.stdout:9/881: rename db/d10/d2d/d4e/d5f/d6e/laa to db/d10/d6d/l120 0 2024-03-29T08:07:28.987 INFO:tasks.workunit.client.0.smithi037.stdout:6/492: fsync d0/d19/d34/d4e/d69/d79/fa0 0 2024-03-29T08:07:28.987 INFO:tasks.workunit.client.1.smithi148.stdout:9/882: write f3 [4618989,112754] 0 2024-03-29T08:07:28.989 INFO:tasks.workunit.client.0.smithi037.stdout:8/559: mknod d5/d10/d19/d3d/cba 0 2024-03-29T08:07:28.989 INFO:tasks.workunit.client.1.smithi148.stdout:9/883: fsync db/d10/faf 0 2024-03-29T08:07:28.989 INFO:tasks.workunit.client.0.smithi037.stdout:0/465: dwrite d9/da/dd/d20/d38/f69 [0,4194304] 0 2024-03-29T08:07:28.990 INFO:tasks.workunit.client.1.smithi148.stdout:5/833: creat d0/db/d1e/d35/d42/d67/d76/d81/d98/dd5/f10e x:0 0 0 2024-03-29T08:07:28.992 INFO:tasks.workunit.client.1.smithi148.stdout:1/796: symlink d3/d18/d4d/d8f/lfb 0 2024-03-29T08:07:28.995 INFO:tasks.workunit.client.1.smithi148.stdout:3/883: symlink de/d24/d33/d39/d46/d30/d37/d62/d63/dbf/dfc/l11d 0 2024-03-29T08:07:28.995 INFO:tasks.workunit.client.1.smithi148.stdout:0/838: link db/de/f70 db/de/d24/dbf/f100 0 2024-03-29T08:07:28.996 INFO:tasks.workunit.client.0.smithi037.stdout:0/466: dread - d9/da/dd/d20/d38/d53/f8f zero size 2024-03-29T08:07:28.996 INFO:tasks.workunit.client.0.smithi037.stdout:6/493: truncate d0/d19/f9b 940482 0 2024-03-29T08:07:28.997 INFO:tasks.workunit.client.1.smithi148.stdout:2/922: getdents d7/da/d4c/dc4/dc6 0 2024-03-29T08:07:28.998 INFO:tasks.workunit.client.0.smithi037.stdout:6/494: dread - d0/d6a/d7b/fa1 zero size 2024-03-29T08:07:29.001 INFO:tasks.workunit.client.0.smithi037.stdout:0/467: write d9/d74/d7d/f87 [4774248,109320] 0 2024-03-29T08:07:29.003 INFO:tasks.workunit.client.0.smithi037.stdout:4/598: rename d3/f66 to d3/d54/fc0 0 2024-03-29T08:07:29.003 INFO:tasks.workunit.client.0.smithi037.stdout:6/495: read d0/d19/d42/d60/f8b [6687611,49977] 0 2024-03-29T08:07:29.006 INFO:tasks.workunit.client.1.smithi148.stdout:2/923: dread d7/da/d4c/d9a/fbd [0,4194304] 0 2024-03-29T08:07:29.007 INFO:tasks.workunit.client.1.smithi148.stdout:3/884: unlink de/d24/d33/d39/d46/c1f 0 2024-03-29T08:07:29.010 INFO:tasks.workunit.client.1.smithi148.stdout:7/802: rename d8/dfb/l3b to d8/dfb/db8/de0/l117 0 2024-03-29T08:07:29.013 INFO:tasks.workunit.client.1.smithi148.stdout:6/855: sync 2024-03-29T08:07:29.016 INFO:tasks.workunit.client.0.smithi037.stdout:1/627: unlink d2/d6/dc/d18/d22/d4f/d7c/fa6 0 2024-03-29T08:07:29.017 INFO:tasks.workunit.client.1.smithi148.stdout:7/803: chown d8/dfb/d44/d49/f83 234 1 2024-03-29T08:07:29.017 INFO:tasks.workunit.client.1.smithi148.stdout:0/839: dwrite db/de/df/f95 [0,4194304] 0 2024-03-29T08:07:29.019 INFO:tasks.workunit.client.1.smithi148.stdout:4/882: rename df/d2c/d51/d77/d26/d9b/da7/d7c/lba to df/d2c/d51/d77/d17/d73/l113 0 2024-03-29T08:07:29.022 INFO:tasks.workunit.client.1.smithi148.stdout:3/885: creat de/d24/d33/d39/d46/d30/d37/d62/f11e x:0 0 0 2024-03-29T08:07:29.025 INFO:tasks.workunit.client.1.smithi148.stdout:7/804: chown d8/d14/d45/d4d/d78/fbc 174269 1 2024-03-29T08:07:29.025 INFO:tasks.workunit.client.1.smithi148.stdout:9/884: dwrite db/d22/f89 [0,4194304] 0 2024-03-29T08:07:29.025 INFO:tasks.workunit.client.1.smithi148.stdout:4/883: dread - df/da0/fee zero size 2024-03-29T08:07:29.029 INFO:tasks.workunit.client.1.smithi148.stdout:5/834: rename d0/db/d7b/d16/d2e/f68 to d0/db/d84/f10f 0 2024-03-29T08:07:29.033 INFO:tasks.workunit.client.1.smithi148.stdout:8/822: getdents d17/d18/d26/d63/d66/d5f 0 2024-03-29T08:07:29.034 INFO:tasks.workunit.client.1.smithi148.stdout:9/885: unlink db/d10/c106 0 2024-03-29T08:07:29.040 INFO:tasks.workunit.client.0.smithi037.stdout:1/628: mknod d2/d6/dc/d18/d22/d4f/d86/cd9 0 2024-03-29T08:07:29.047 INFO:tasks.workunit.client.1.smithi148.stdout:1/797: truncate d3/d18/d4d/f63 3858563 0 2024-03-29T08:07:29.049 INFO:tasks.workunit.client.0.smithi037.stdout:0/468: dread d9/da/dd/d13/f36 [0,4194304] 0 2024-03-29T08:07:29.051 INFO:tasks.workunit.client.1.smithi148.stdout:2/924: write d7/d8c/da4/d8e/fd6 [550853,104993] 0 2024-03-29T08:07:29.052 INFO:tasks.workunit.client.1.smithi148.stdout:6/856: dwrite d6/d9/d1e/d20/d6b/fe3 [0,4194304] 0 2024-03-29T08:07:29.054 INFO:tasks.workunit.client.0.smithi037.stdout:6/496: dread d0/d5/d3f/f92 [0,4194304] 0 2024-03-29T08:07:29.054 INFO:tasks.workunit.client.0.smithi037.stdout:1/629: mkdir d2/d6/dc/d18/dbd/dda 0 2024-03-29T08:07:29.057 INFO:tasks.workunit.client.0.smithi037.stdout:9/770: write d1/d4/d19/d1e/d4a/d6f/da2/d31/d87/fd5 [390720,59669] 0 2024-03-29T08:07:29.060 INFO:tasks.workunit.client.1.smithi148.stdout:5/835: link d0/db/d1e/d36/d41/f58 d0/db/d1e/d35/d42/d67/d76/d81/d98/de5/f110 0 2024-03-29T08:07:29.060 INFO:tasks.workunit.client.1.smithi148.stdout:8/823: getdents d17/d49/d9a 0 2024-03-29T08:07:29.064 INFO:tasks.workunit.client.0.smithi037.stdout:4/599: dread d3/f3a [0,4194304] 0 2024-03-29T08:07:29.065 INFO:tasks.workunit.client.1.smithi148.stdout:6/857: stat d6/d9/d1e/d27/d48/df7 0 2024-03-29T08:07:29.065 INFO:tasks.workunit.client.1.smithi148.stdout:9/886: link db/d10/d2d/l7f db/d10/d2d/d4e/da6/dd1/dd9/l121 0 2024-03-29T08:07:29.066 INFO:tasks.workunit.client.0.smithi037.stdout:0/469: write d9/da/d1b/d22/d2c/f2d [3564779,44633] 0 2024-03-29T08:07:29.067 INFO:tasks.workunit.client.1.smithi148.stdout:7/805: dwrite d8/d38/d3c/da9/fe5 [0,4194304] 0 2024-03-29T08:07:29.068 INFO:tasks.workunit.client.1.smithi148.stdout:0/840: dwrite db/de/df/d2b/f72 [0,4194304] 0 2024-03-29T08:07:29.068 INFO:tasks.workunit.client.1.smithi148.stdout:8/824: stat d17/d18/d26/d63/d66/d5f/dd0 0 2024-03-29T08:07:29.069 INFO:tasks.workunit.client.0.smithi037.stdout:6/497: symlink d0/d5/d1e/d44/d71/laa 0 2024-03-29T08:07:29.070 INFO:tasks.workunit.client.1.smithi148.stdout:5/836: fsync d0/db/d1e/d5b/d66/fd2 0 2024-03-29T08:07:29.075 INFO:tasks.workunit.client.1.smithi148.stdout:1/798: mkdir d3/de/d25/dfc 0 2024-03-29T08:07:29.078 INFO:tasks.workunit.client.1.smithi148.stdout:4/884: dread fa [0,4194304] 0 2024-03-29T08:07:29.079 INFO:tasks.workunit.client.1.smithi148.stdout:1/799: stat d3/d18/dae/dda/df3/c62 0 2024-03-29T08:07:29.079 INFO:tasks.workunit.client.1.smithi148.stdout:7/806: creat d8/dfb/db8/d115/f118 x:0 0 0 2024-03-29T08:07:29.080 INFO:tasks.workunit.client.0.smithi037.stdout:9/771: rename d1/d4/d19/d1e/d4a/d6f/da2/c20 to d1/d4/d19/d1e/d4a/d6f/da2/d13/de0/c10a 0 2024-03-29T08:07:29.082 INFO:tasks.workunit.client.1.smithi148.stdout:4/885: fsync df/d2c/d51/d77/d26/d34/d43/ffa 0 2024-03-29T08:07:29.082 INFO:tasks.workunit.client.1.smithi148.stdout:7/807: dread - d8/d38/d3c/da5/de3/f102 zero size 2024-03-29T08:07:29.091 INFO:tasks.workunit.client.0.smithi037.stdout:1/630: dwrite d2/d6/dc/d18/d1d/dad/daf/fd4 [0,4194304] 0 2024-03-29T08:07:29.091 INFO:tasks.workunit.client.0.smithi037.stdout:0/470: mknod d9/da/dd/d59/c9b 0 2024-03-29T08:07:29.093 INFO:tasks.workunit.client.0.smithi037.stdout:8/560: dwrite d5/d10/d45/d4b/f6c [4194304,4194304] 0 2024-03-29T08:07:29.096 INFO:tasks.workunit.client.0.smithi037.stdout:8/561: readlink d5/d10/l58 0 2024-03-29T08:07:29.097 INFO:tasks.workunit.client.0.smithi037.stdout:4/600: mkdir d3/d4/d15/d1d/db/d45/d1e/d59/dc1 0 2024-03-29T08:07:29.100 INFO:tasks.workunit.client.0.smithi037.stdout:6/498: write d0/f33 [4570343,77073] 0 2024-03-29T08:07:29.102 INFO:tasks.workunit.client.0.smithi037.stdout:6/499: chown d0/d19/d34 169 1 2024-03-29T08:07:29.108 INFO:tasks.workunit.client.0.smithi037.stdout:0/471: link d9/d74/d7d/f94 d9/da/dd/d20/d55/f9c 0 2024-03-29T08:07:29.112 INFO:tasks.workunit.client.1.smithi148.stdout:2/925: write d7/d24/f60 [4837365,33150] 0 2024-03-29T08:07:29.115 INFO:tasks.workunit.client.0.smithi037.stdout:1/631: creat d2/d6/dc/d18/d1d/d21/fdb x:0 0 0 2024-03-29T08:07:29.118 INFO:tasks.workunit.client.1.smithi148.stdout:2/926: dread - d7/da/d33/d3c/d31/d123/f11f zero size 2024-03-29T08:07:29.119 INFO:tasks.workunit.client.0.smithi037.stdout:6/500: write d0/d5/d3f/f92 [9099278,81491] 0 2024-03-29T08:07:29.124 INFO:tasks.workunit.client.0.smithi037.stdout:0/472: creat d9/da/d1b/f9d x:0 0 0 2024-03-29T08:07:29.125 INFO:tasks.workunit.client.1.smithi148.stdout:2/927: creat d7/da/d33/d3c/d54/db6/f130 x:0 0 0 2024-03-29T08:07:29.126 INFO:tasks.workunit.client.0.smithi037.stdout:1/632: truncate d2/d4b/f66 888868 0 2024-03-29T08:07:29.127 INFO:tasks.workunit.client.1.smithi148.stdout:7/808: dwrite d8/d14/d45/d4d/d78/fbc [0,4194304] 0 2024-03-29T08:07:29.128 INFO:tasks.workunit.client.0.smithi037.stdout:6/501: symlink d0/d19/d34/d4e/d69/d79/lab 0 2024-03-29T08:07:29.130 INFO:tasks.workunit.client.0.smithi037.stdout:0/473: creat d9/da/dd/d20/f9e x:0 0 0 2024-03-29T08:07:29.132 INFO:tasks.workunit.client.1.smithi148.stdout:9/887: dread f3 [0,4194304] 0 2024-03-29T08:07:29.133 INFO:tasks.workunit.client.0.smithi037.stdout:1/633: mkdir d2/d6/dc/d18/d22/d4f/d7c/ddc 0 2024-03-29T08:07:29.133 INFO:tasks.workunit.client.1.smithi148.stdout:7/809: chown d8/d14/da8/f112 390708 1 2024-03-29T08:07:29.134 INFO:tasks.workunit.client.1.smithi148.stdout:5/837: dwrite d0/db/d84/daf/ffa [0,4194304] 0 2024-03-29T08:07:29.135 INFO:tasks.workunit.client.0.smithi037.stdout:1/634: write d2/d40/fbc [705391,131012] 0 2024-03-29T08:07:29.141 INFO:tasks.workunit.client.1.smithi148.stdout:9/888: fsync db/d10/d6d/d73/da8/deb/dec/dde/f102 0 2024-03-29T08:07:29.145 INFO:tasks.workunit.client.1.smithi148.stdout:3/886: sync 2024-03-29T08:07:29.145 INFO:tasks.workunit.client.1.smithi148.stdout:9/889: mkdir db/d10/d6d/d73/d122 0 2024-03-29T08:07:29.149 INFO:tasks.workunit.client.0.smithi037.stdout:0/474: dwrite d9/da/dd/d54/f56 [0,4194304] 0 2024-03-29T08:07:29.150 INFO:tasks.workunit.client.1.smithi148.stdout:9/890: chown db/d22/d26/d54/d52/d9c 100204 1 2024-03-29T08:07:29.156 INFO:tasks.workunit.client.1.smithi148.stdout:9/891: write db/d10/d6d/d73/da8/deb/dec/dde/f102 [1625911,67113] 0 2024-03-29T08:07:29.158 INFO:tasks.workunit.client.0.smithi037.stdout:0/475: mkdir d9/da/dd/d20/d55/d5f/d9f 0 2024-03-29T08:07:29.158 INFO:tasks.workunit.client.0.smithi037.stdout:9/772: write d1/d4/d19/d69/fc2 [639143,7050] 0 2024-03-29T08:07:29.163 INFO:tasks.workunit.client.1.smithi148.stdout:5/838: dwrite d0/db/d1e/d35/d42/d96/fdf [0,4194304] 0 2024-03-29T08:07:29.165 INFO:tasks.workunit.client.0.smithi037.stdout:0/476: write d9/da/dd/d13/f31 [7246152,5035] 0 2024-03-29T08:07:29.167 INFO:tasks.workunit.client.1.smithi148.stdout:9/892: write db/f84 [2721737,125858] 0 2024-03-29T08:07:29.170 INFO:tasks.workunit.client.0.smithi037.stdout:9/773: symlink d1/d4/d19/d1e/d4a/d6f/da2/d13/d7c/l10b 0 2024-03-29T08:07:29.173 INFO:tasks.workunit.client.0.smithi037.stdout:0/477: rename d9/da/dd/d59/l44 to d9/da/dd/d20/d55/d5f/d9f/la0 0 2024-03-29T08:07:29.174 INFO:tasks.workunit.client.0.smithi037.stdout:4/601: dread f0 [0,4194304] 0 2024-03-29T08:07:29.174 INFO:tasks.workunit.client.0.smithi037.stdout:9/774: dread - d1/d4/d19/d1e/d4a/d6f/da2/d31/daa/dba/de3/f104 zero size 2024-03-29T08:07:29.176 INFO:tasks.workunit.client.0.smithi037.stdout:0/478: creat d9/d74/fa1 x:0 0 0 2024-03-29T08:07:29.178 INFO:tasks.workunit.client.0.smithi037.stdout:4/602: mkdir d3/d4/d15/d1d/dc2 0 2024-03-29T08:07:29.179 INFO:tasks.workunit.client.0.smithi037.stdout:9/775: getdents d1/d4/d19/d69/df3 0 2024-03-29T08:07:29.180 INFO:tasks.workunit.client.1.smithi148.stdout:9/893: dwrite db/d22/fd7 [0,4194304] 0 2024-03-29T08:07:29.182 INFO:tasks.workunit.client.0.smithi037.stdout:0/479: write d9/da/dd/d20/d55/f9c [810371,62193] 0 2024-03-29T08:07:29.185 INFO:tasks.workunit.client.1.smithi148.stdout:9/894: mkdir db/d22/d26/d54/d52/dab/d123 0 2024-03-29T08:07:29.189 INFO:tasks.workunit.client.1.smithi148.stdout:1/800: dwrite d3/de/d6e/f79 [0,4194304] 0 2024-03-29T08:07:29.189 INFO:tasks.workunit.client.0.smithi037.stdout:9/776: rename d1/d4/d19/d1e/d4a/d6f/da2/d31/l42 to d1/d3/d44/l10c 0 2024-03-29T08:07:29.190 INFO:tasks.workunit.client.0.smithi037.stdout:0/480: rmdir d9/da/d1b/d22 39 2024-03-29T08:07:29.199 INFO:tasks.workunit.client.1.smithi148.stdout:9/895: rename db/d22/d26/d54/d4d/d35/c93 to db/d10/d2d/d4e/d5f/d6e/d7a/c124 0 2024-03-29T08:07:29.204 INFO:tasks.workunit.client.0.smithi037.stdout:9/777: rmdir d1/d4/d19/d1e/d71/d89/da6/dfb 0 2024-03-29T08:07:29.207 INFO:tasks.workunit.client.1.smithi148.stdout:1/801: dwrite d3/f5d [0,4194304] 0 2024-03-29T08:07:29.207 INFO:tasks.workunit.client.0.smithi037.stdout:4/603: dwrite d3/d4/d15/f49 [0,4194304] 0 2024-03-29T08:07:29.207 INFO:tasks.workunit.client.1.smithi148.stdout:9/896: creat db/d22/d26/d54/d52/d9c/f125 x:0 0 0 2024-03-29T08:07:29.208 INFO:tasks.workunit.client.0.smithi037.stdout:9/778: dread - d1/d4/d19/d1e/d4a/d6f/da2/d31/daa/dba/de3/f104 zero size 2024-03-29T08:07:29.210 INFO:tasks.workunit.client.0.smithi037.stdout:4/604: creat d3/d57/d6b/fc3 x:0 0 0 2024-03-29T08:07:29.214 INFO:tasks.workunit.client.0.smithi037.stdout:4/605: read d3/d54/f99 [1367254,62039] 0 2024-03-29T08:07:29.221 INFO:tasks.workunit.client.0.smithi037.stdout:6/502: dread d0/d5/d1e/d44/d71/f74 [0,4194304] 0 2024-03-29T08:07:29.223 INFO:tasks.workunit.client.1.smithi148.stdout:4/886: truncate df/d2c/d80/daa/fcb 4185807 0 2024-03-29T08:07:29.223 INFO:tasks.workunit.client.0.smithi037.stdout:4/606: symlink d3/d4/d15/d1d/db/d45/d93/lc4 0 2024-03-29T08:07:29.226 INFO:tasks.workunit.client.0.smithi037.stdout:8/562: dwrite d5/f15 [4194304,4194304] 0 2024-03-29T08:07:29.229 INFO:tasks.workunit.client.0.smithi037.stdout:4/607: read d3/d4/d27/f7f [7239319,42983] 0 2024-03-29T08:07:29.230 INFO:tasks.workunit.client.1.smithi148.stdout:4/887: mknod df/d2c/d51/d101/c114 0 2024-03-29T08:07:29.232 INFO:tasks.workunit.client.1.smithi148.stdout:9/897: dwrite db/d22/d26/d54/d4d/f79 [4194304,4194304] 0 2024-03-29T08:07:29.234 INFO:tasks.workunit.client.1.smithi148.stdout:4/888: chown df/d2c/d80/da6 107 1 2024-03-29T08:07:29.235 INFO:tasks.workunit.client.1.smithi148.stdout:1/802: dread d3/d18/dae/dda/df3/f48 [0,4194304] 0 2024-03-29T08:07:29.235 INFO:tasks.workunit.client.0.smithi037.stdout:4/608: symlink d3/d4/d15/d1d/db/d45/d1e/d59/d60/d6f/lc5 0 2024-03-29T08:07:29.239 INFO:tasks.workunit.client.1.smithi148.stdout:6/858: sync 2024-03-29T08:07:29.242 INFO:tasks.workunit.client.1.smithi148.stdout:6/859: readlink d6/l82 0 2024-03-29T08:07:29.246 INFO:tasks.workunit.client.1.smithi148.stdout:4/889: creat df/d2c/d51/d77/d17/f115 x:0 0 0 2024-03-29T08:07:29.247 INFO:tasks.workunit.client.1.smithi148.stdout:1/803: creat d3/d55/ffd x:0 0 0 2024-03-29T08:07:29.252 INFO:tasks.workunit.client.1.smithi148.stdout:4/890: chown df/le6 46175173 1 2024-03-29T08:07:29.252 INFO:tasks.workunit.client.1.smithi148.stdout:1/804: mknod d3/de/d1d/cfe 0 2024-03-29T08:07:29.255 INFO:tasks.workunit.client.1.smithi148.stdout:7/810: dwrite d8/dfb/d44/dc3/fd9 [0,4194304] 0 2024-03-29T08:07:29.256 INFO:tasks.workunit.client.1.smithi148.stdout:2/928: dwrite d7/fbf [0,4194304] 0 2024-03-29T08:07:29.257 INFO:tasks.workunit.client.1.smithi148.stdout:6/860: creat d6/d9/d1e/d36/d4d/f115 x:0 0 0 2024-03-29T08:07:29.259 INFO:tasks.workunit.client.1.smithi148.stdout:1/805: mknod d3/de/d6e/df9/cff 0 2024-03-29T08:07:29.264 INFO:tasks.workunit.client.1.smithi148.stdout:2/929: truncate d7/da/d33/ded/d2e/d37/ffd 461326 0 2024-03-29T08:07:29.265 INFO:tasks.workunit.client.1.smithi148.stdout:1/806: rmdir d3/de/d1d/d47/d6d/d84/d8c 39 2024-03-29T08:07:29.272 INFO:tasks.workunit.client.0.smithi037.stdout:8/563: dread d5/d10/f23 [4194304,4194304] 0 2024-03-29T08:07:29.275 INFO:tasks.workunit.client.1.smithi148.stdout:1/807: unlink d3/d55/d7e/f5b 0 2024-03-29T08:07:29.276 INFO:tasks.workunit.client.1.smithi148.stdout:4/891: dread df/d2c/d51/d77/f83 [0,4194304] 0 2024-03-29T08:07:29.278 INFO:tasks.workunit.client.1.smithi148.stdout:3/887: write de/d24/d33/d39/d46/d70/fa9 [899456,76154] 0 2024-03-29T08:07:29.279 INFO:tasks.workunit.client.1.smithi148.stdout:2/930: rename d7/da/d33/d3c/d31/de1 to d7/da/d33/d3c/d31/dbb/d131 0 2024-03-29T08:07:29.282 INFO:tasks.workunit.client.1.smithi148.stdout:6/861: dwrite d6/d9/d1e/f5d [0,4194304] 0 2024-03-29T08:07:29.282 INFO:tasks.workunit.client.1.smithi148.stdout:1/808: symlink d3/de/d1d/d47/d6d/d72/l100 0 2024-03-29T08:07:29.282 INFO:tasks.workunit.client.1.smithi148.stdout:5/839: write d0/db/f34 [549545,32954] 0 2024-03-29T08:07:29.288 INFO:tasks.workunit.client.0.smithi037.stdout:7/605: sync 2024-03-29T08:07:29.288 INFO:tasks.workunit.client.0.smithi037.stdout:3/537: sync 2024-03-29T08:07:29.289 INFO:tasks.workunit.client.1.smithi148.stdout:8/825: sync 2024-03-29T08:07:29.290 INFO:tasks.workunit.client.1.smithi148.stdout:7/811: dread d8/dfb/d44/d50/f8b [0,4194304] 0 2024-03-29T08:07:29.291 INFO:tasks.workunit.client.1.smithi148.stdout:3/888: creat de/d24/d33/d39/d46/d30/d10d/f11f x:0 0 0 2024-03-29T08:07:29.294 INFO:tasks.workunit.client.0.smithi037.stdout:8/564: dwrite d5/d10/d19/d3d/d4a/d4c/f8f [0,4194304] 0 2024-03-29T08:07:29.294 INFO:tasks.workunit.client.1.smithi148.stdout:3/889: chown de/lf 110010 1 2024-03-29T08:07:29.295 INFO:tasks.workunit.client.1.smithi148.stdout:2/931: mknod d7/da/dcc/d11c/d8d/d127/d12f/c132 0 2024-03-29T08:07:29.299 INFO:tasks.workunit.client.1.smithi148.stdout:5/840: rename d0/db/d1e/d35/d42/d67/d9a/f88 to d0/db/d1e/d35/d42/d67/d76/d81/d98/f111 0 2024-03-29T08:07:29.304 INFO:tasks.workunit.client.1.smithi148.stdout:7/812: chown d8/fe 21550721 1 2024-03-29T08:07:29.305 INFO:tasks.workunit.client.1.smithi148.stdout:8/826: mkdir d17/d49/d9a/dbe/d112 0 2024-03-29T08:07:29.306 INFO:tasks.workunit.client.1.smithi148.stdout:2/932: rename d7/da/d33/ded/l2b to d7/d8c/da4/d8e/l133 0 2024-03-29T08:07:29.307 INFO:tasks.workunit.client.0.smithi037.stdout:8/565: creat d5/d10/d2d/d34/d50/d83/d46/fbb x:0 0 0 2024-03-29T08:07:29.308 INFO:tasks.workunit.client.0.smithi037.stdout:8/566: write d5/f80 [1822457,42230] 0 2024-03-29T08:07:29.309 INFO:tasks.workunit.client.1.smithi148.stdout:0/841: sync 2024-03-29T08:07:29.314 INFO:tasks.workunit.client.1.smithi148.stdout:6/862: dwrite d6/d9/d1e/d36/d4d/f115 [0,4194304] 0 2024-03-29T08:07:29.315 INFO:tasks.workunit.client.0.smithi037.stdout:0/481: write d9/da/dd/d13/d83/f81 [748048,76638] 0 2024-03-29T08:07:29.317 INFO:tasks.workunit.client.0.smithi037.stdout:0/482: dread - d9/da/d1b/d22/d2c/f86 zero size 2024-03-29T08:07:29.319 INFO:tasks.workunit.client.0.smithi037.stdout:0/483: chown d9/da/dd/d59/c2e 73 1 2024-03-29T08:07:29.320 INFO:tasks.workunit.client.1.smithi148.stdout:9/898: dread db/d22/d26/d54/d4d/f79 [0,4194304] 0 2024-03-29T08:07:29.321 INFO:tasks.workunit.client.1.smithi148.stdout:0/842: stat db/dc/daa/d47/d7c/ca4 0 2024-03-29T08:07:29.321 INFO:tasks.workunit.client.0.smithi037.stdout:9/779: write d1/d4/d19/d1e/d4a/d6f/da2/d31/f48 [796621,2300] 0 2024-03-29T08:07:29.324 INFO:tasks.workunit.client.0.smithi037.stdout:6/503: write d0/d19/d42/d60/f7d [142008,16398] 0 2024-03-29T08:07:29.325 INFO:tasks.workunit.client.1.smithi148.stdout:2/933: read d7/da/d33/d3c/f65 [2070643,2116] 0 2024-03-29T08:07:29.327 INFO:tasks.workunit.client.0.smithi037.stdout:9/780: creat d1/d4/d19/d1e/d71/d89/f10d x:0 0 0 2024-03-29T08:07:29.328 INFO:tasks.workunit.client.1.smithi148.stdout:6/863: fsync d6/d9/d1e/d36/d4d/d9d/d28/f38 0 2024-03-29T08:07:29.330 INFO:tasks.workunit.client.0.smithi037.stdout:6/504: fsync d0/d5/d1e/d44/d71/f74 0 2024-03-29T08:07:29.334 INFO:tasks.workunit.client.1.smithi148.stdout:9/899: truncate db/d10/d6d/d73/da8/deb/dec/f10d 204685 0 2024-03-29T08:07:29.337 INFO:tasks.workunit.client.1.smithi148.stdout:8/827: dread d17/d49/d9a/dbe/f76 [0,4194304] 0 2024-03-29T08:07:29.345 INFO:tasks.workunit.client.0.smithi037.stdout:7/606: dwrite dd/d1d/d34/d54/f87 [0,4194304] 0 2024-03-29T08:07:29.347 INFO:tasks.workunit.client.1.smithi148.stdout:6/864: chown d6/d9/d1e/d36/d4d/d9d/d28/dda/c9a 2283 1 2024-03-29T08:07:29.349 INFO:tasks.workunit.client.0.smithi037.stdout:3/538: dwrite dd/d46/d2d/d3a/f52 [0,4194304] 0 2024-03-29T08:07:29.350 INFO:tasks.workunit.client.1.smithi148.stdout:2/934: dread d7/d24/d4b/d66/df1/fe3 [0,4194304] 0 2024-03-29T08:07:29.350 INFO:tasks.workunit.client.1.smithi148.stdout:7/813: dwrite d8/d14/d45/d4d/f6b [0,4194304] 0 2024-03-29T08:07:29.352 INFO:tasks.workunit.client.1.smithi148.stdout:0/843: dwrite db/dc/daa/d47/d4c/d5a/d7a/ff7 [0,4194304] 0 2024-03-29T08:07:29.352 INFO:tasks.workunit.client.0.smithi037.stdout:3/539: dread - dd/d5e/faa zero size 2024-03-29T08:07:29.355 INFO:tasks.workunit.client.0.smithi037.stdout:3/540: fsync dd/f24 0 2024-03-29T08:07:29.356 INFO:tasks.workunit.client.0.smithi037.stdout:6/505: dwrite d0/d19/d42/d60/f9d [0,4194304] 0 2024-03-29T08:07:29.357 INFO:tasks.workunit.client.1.smithi148.stdout:8/828: truncate d17/d18/d26/d63/d7d/faa 886422 0 2024-03-29T08:07:29.358 INFO:tasks.workunit.client.1.smithi148.stdout:9/900: creat db/d22/d26/d54/d4d/d35/d4b/d5a/d74/d10a/f126 x:0 0 0 2024-03-29T08:07:29.358 INFO:tasks.workunit.client.1.smithi148.stdout:0/844: dread - db/de/df/d11/d2d/fb2 zero size 2024-03-29T08:07:29.368 INFO:tasks.workunit.client.1.smithi148.stdout:4/892: sync 2024-03-29T08:07:29.369 INFO:tasks.workunit.client.1.smithi148.stdout:9/901: fdatasync db/d22/d3f/f42 0 2024-03-29T08:07:29.369 INFO:tasks.workunit.client.1.smithi148.stdout:2/935: truncate d7/da/dcc/d11c/f9f 1026178 0 2024-03-29T08:07:29.369 INFO:tasks.workunit.client.1.smithi148.stdout:8/829: write d17/d18/d26/d63/d66/d5f/f6c [424289,67563] 0 2024-03-29T08:07:29.370 INFO:tasks.workunit.client.0.smithi037.stdout:8/567: write d5/d10/d19/d3d/f5d [889715,124040] 0 2024-03-29T08:07:29.371 INFO:tasks.workunit.client.0.smithi037.stdout:0/484: write d9/da/dd/f5b [2109230,19806] 0 2024-03-29T08:07:29.372 INFO:tasks.workunit.client.0.smithi037.stdout:3/541: write dd/d5e/d7a/f4f [4009336,67988] 0 2024-03-29T08:07:29.373 INFO:tasks.workunit.client.1.smithi148.stdout:8/830: readlink d17/d18/lc1 0 2024-03-29T08:07:29.378 INFO:tasks.workunit.client.1.smithi148.stdout:6/865: dread d6/d9/d1e/d36/d4d/fbf [0,4194304] 0 2024-03-29T08:07:29.379 INFO:tasks.workunit.client.0.smithi037.stdout:7/607: dread dd/d72/fac [0,4194304] 0 2024-03-29T08:07:29.381 INFO:tasks.workunit.client.0.smithi037.stdout:9/781: dread d1/d4/d19/d1e/d4a/d6f/da2/d23/f37 [0,4194304] 0 2024-03-29T08:07:29.382 INFO:tasks.workunit.client.1.smithi148.stdout:8/831: readlink d17/d49/d9a/dbe/l40 0 2024-03-29T08:07:29.384 INFO:tasks.workunit.client.1.smithi148.stdout:3/890: dwrite de/d24/d33/d39/d46/d8d/fc2 [0,4194304] 0 2024-03-29T08:07:29.385 INFO:tasks.workunit.client.1.smithi148.stdout:4/893: truncate df/d2c/d51/d77/d26/d34/d43/f4e 4454815 0 2024-03-29T08:07:29.386 INFO:tasks.workunit.client.0.smithi037.stdout:7/608: truncate dd/d1d/d49/d81/fc1 437014 0 2024-03-29T08:07:29.386 INFO:tasks.workunit.client.1.smithi148.stdout:5/841: dread d0/db/d1e/d6e/f92 [0,4194304] 0 2024-03-29T08:07:29.387 INFO:tasks.workunit.client.1.smithi148.stdout:1/809: dread d3/d18/dae/dda/df3/d42/fa5 [0,4194304] 0 2024-03-29T08:07:29.390 INFO:tasks.workunit.client.1.smithi148.stdout:3/891: chown de/d24/d33/d39/d7f/fac 10930215 1 2024-03-29T08:07:29.391 INFO:tasks.workunit.client.1.smithi148.stdout:7/814: getdents d8/d14/d45/d4d 0 2024-03-29T08:07:29.393 INFO:tasks.workunit.client.1.smithi148.stdout:4/894: mkdir df/d2c/d51/d77/d26/d34/d3e/d116 0 2024-03-29T08:07:29.393 INFO:tasks.workunit.client.1.smithi148.stdout:5/842: symlink d0/db/d1e/d6e/da8/l112 0 2024-03-29T08:07:29.395 INFO:tasks.workunit.client.1.smithi148.stdout:0/845: rename db/dc/daa/d47/d4c/d5a/feb to db/de/d8e/f101 0 2024-03-29T08:07:29.397 INFO:tasks.workunit.client.1.smithi148.stdout:6/866: link d6/d9/d1e/d27/d48/d9f/f106 d6/d9/d1e/d27/f116 0 2024-03-29T08:07:29.398 INFO:tasks.workunit.client.1.smithi148.stdout:5/843: dread - d0/db/d1e/d79/ff3 zero size 2024-03-29T08:07:29.398 INFO:tasks.workunit.client.1.smithi148.stdout:6/867: readlink d6/l82 0 2024-03-29T08:07:29.400 INFO:tasks.workunit.client.1.smithi148.stdout:7/815: readlink d8/dfb/db8/de0/l117 0 2024-03-29T08:07:29.401 INFO:tasks.workunit.client.1.smithi148.stdout:9/902: dread db/d22/d26/d54/f2e [0,4194304] 0 2024-03-29T08:07:29.404 INFO:tasks.workunit.client.1.smithi148.stdout:5/844: unlink d0/db/d1e/fc5 0 2024-03-29T08:07:29.405 INFO:tasks.workunit.client.1.smithi148.stdout:4/895: fsync df/d2c/d51/d77/d26/d9b/fa8 0 2024-03-29T08:07:29.406 INFO:tasks.workunit.client.1.smithi148.stdout:0/846: rename db/de/df/d11/d42/ld6 to db/de/df/d11/d2d/da3/l102 0 2024-03-29T08:07:29.407 INFO:tasks.workunit.client.1.smithi148.stdout:8/832: getdents d17/d1d 0 2024-03-29T08:07:29.408 INFO:tasks.workunit.client.1.smithi148.stdout:5/845: chown d0/db/d1e/f31 1 1 2024-03-29T08:07:29.408 INFO:tasks.workunit.client.1.smithi148.stdout:7/816: creat d8/dfb/d44/d49/f119 x:0 0 0 2024-03-29T08:07:29.409 INFO:tasks.workunit.client.0.smithi037.stdout:6/506: creat d0/d20/fac x:0 0 0 2024-03-29T08:07:29.410 INFO:tasks.workunit.client.1.smithi148.stdout:3/892: truncate de/d24/d33/d39/d46/d30/d37/d62/d63/f102 404161 0 2024-03-29T08:07:29.410 INFO:tasks.workunit.client.0.smithi037.stdout:0/485: dread - d9/da/dd/d13/d83/f6d zero size 2024-03-29T08:07:29.411 INFO:tasks.workunit.client.1.smithi148.stdout:0/847: write db/f23 [2266260,97245] 0 2024-03-29T08:07:29.411 INFO:tasks.workunit.client.1.smithi148.stdout:8/833: chown d17/d18/d26/d4f/cf9 721265 1 2024-03-29T08:07:29.411 INFO:tasks.workunit.client.1.smithi148.stdout:4/896: creat df/d2c/d51/d77/dbe/f117 x:0 0 0 2024-03-29T08:07:29.412 INFO:tasks.workunit.client.0.smithi037.stdout:3/542: symlink dd/d15/d27/d33/d75/d8f/lb5 0 2024-03-29T08:07:29.414 INFO:tasks.workunit.client.1.smithi148.stdout:7/817: chown d8/dfb/l7b 79 1 2024-03-29T08:07:29.415 INFO:tasks.workunit.client.0.smithi037.stdout:9/782: rmdir d1/d4/d19 39 2024-03-29T08:07:29.415 INFO:tasks.workunit.client.0.smithi037.stdout:7/609: rmdir dd/d21/d35/d3b 39 2024-03-29T08:07:29.417 INFO:tasks.workunit.client.1.smithi148.stdout:6/868: link d6/d9/d1e/d27/c2e d6/d9/d1e/d36/d4d/d9d/d28/db3/c117 0 2024-03-29T08:07:29.418 INFO:tasks.workunit.client.0.smithi037.stdout:3/543: chown dd/d46/d2d/d3a/d85/f9e 9 1 2024-03-29T08:07:29.418 INFO:tasks.workunit.client.1.smithi148.stdout:0/848: write db/de/df/f95 [2018294,121125] 0 2024-03-29T08:07:29.419 INFO:tasks.workunit.client.1.smithi148.stdout:4/897: symlink df/l118 0 2024-03-29T08:07:29.420 INFO:tasks.workunit.client.0.smithi037.stdout:0/486: write d9/da/dd/d20/d72/f85 [4884355,41082] 0 2024-03-29T08:07:29.424 INFO:tasks.workunit.client.1.smithi148.stdout:7/818: creat d8/d38/d3c/da9/d37/f11a x:0 0 0 2024-03-29T08:07:29.424 INFO:tasks.workunit.client.1.smithi148.stdout:7/819: readlink d8/d14/d45/le4 0 2024-03-29T08:07:29.424 INFO:tasks.workunit.client.0.smithi037.stdout:3/544: fsync dd/f1b 0 2024-03-29T08:07:29.424 INFO:tasks.workunit.client.0.smithi037.stdout:9/783: chown d1/d4/d19/d1e/d4a/d6f/da2/d31/daa/fef 11622 1 2024-03-29T08:07:29.425 INFO:tasks.workunit.client.1.smithi148.stdout:4/898: symlink df/d2c/d51/d77/d26/l119 0 2024-03-29T08:07:29.427 INFO:tasks.workunit.client.1.smithi148.stdout:0/849: symlink db/de/df/d2b/db9/l103 0 2024-03-29T08:07:29.427 INFO:tasks.workunit.client.0.smithi037.stdout:7/610: creat dd/d21/d35/fcb x:0 0 0 2024-03-29T08:07:29.427 INFO:tasks.workunit.client.0.smithi037.stdout:5/698: sync 2024-03-29T08:07:29.431 INFO:tasks.workunit.client.0.smithi037.stdout:7/611: read dd/f14 [2286334,26292] 0 2024-03-29T08:07:29.436 INFO:tasks.workunit.client.1.smithi148.stdout:2/936: dread d7/da/d33/ded/f16 [0,4194304] 0 2024-03-29T08:07:29.437 INFO:tasks.workunit.client.0.smithi037.stdout:3/545: creat dd/d46/d2d/d3a/fb6 x:0 0 0 2024-03-29T08:07:29.437 INFO:tasks.workunit.client.1.smithi148.stdout:4/899: dread df/fea [0,4194304] 0 2024-03-29T08:07:29.441 INFO:tasks.workunit.client.1.smithi148.stdout:8/834: dwrite d17/d74/d86/f10d [0,4194304] 0 2024-03-29T08:07:29.443 INFO:tasks.workunit.client.0.smithi037.stdout:7/612: mknod dd/d1d/d49/ccc 0 2024-03-29T08:07:29.446 INFO:tasks.workunit.client.1.smithi148.stdout:4/900: write df/d2c/d51/d77/d26/f3b [1663668,9902] 0 2024-03-29T08:07:29.446 INFO:tasks.workunit.client.1.smithi148.stdout:6/869: dwrite d6/d9/d1e/fee [0,4194304] 0 2024-03-29T08:07:29.450 INFO:tasks.workunit.client.0.smithi037.stdout:7/613: fdatasync dd/d21/d35/d42/d5c/f5e 0 2024-03-29T08:07:29.452 INFO:tasks.workunit.client.1.smithi148.stdout:2/937: truncate d7/da/d33/d3c/ff6 295073 0 2024-03-29T08:07:29.453 INFO:tasks.workunit.client.0.smithi037.stdout:9/784: dwrite d1/d4/d19/d69/fc2 [0,4194304] 0 2024-03-29T08:07:29.455 INFO:tasks.workunit.client.0.smithi037.stdout:5/699: dwrite d8/d4d/d5c/dbf/fe4 [0,4194304] 0 2024-03-29T08:07:29.458 INFO:tasks.workunit.client.0.smithi037.stdout:7/614: symlink dd/d72/lcd 0 2024-03-29T08:07:29.460 INFO:tasks.workunit.client.0.smithi037.stdout:5/700: write d8/d22/d4f/d70/f3f [2178428,128359] 0 2024-03-29T08:07:29.462 INFO:tasks.workunit.client.1.smithi148.stdout:6/870: write d6/d9/d1e/d20/d6b/fe3 [4879428,107838] 0 2024-03-29T08:07:29.462 INFO:tasks.workunit.client.1.smithi148.stdout:8/835: creat d17/d18/d26/d63/d7d/dc6/f113 x:0 0 0 2024-03-29T08:07:29.463 INFO:tasks.workunit.client.0.smithi037.stdout:4/609: sync 2024-03-29T08:07:29.465 INFO:tasks.workunit.client.0.smithi037.stdout:1/635: sync 2024-03-29T08:07:29.465 INFO:tasks.workunit.client.0.smithi037.stdout:2/462: sync 2024-03-29T08:07:29.467 INFO:tasks.workunit.client.1.smithi148.stdout:1/810: dwrite d3/d18/dae/dda/df3/fd0 [0,4194304] 0 2024-03-29T08:07:29.474 INFO:tasks.workunit.client.1.smithi148.stdout:0/850: dwrite db/dc/daa/d47/d7c/fb4 [0,4194304] 0 2024-03-29T08:07:29.478 INFO:tasks.workunit.client.1.smithi148.stdout:6/871: creat d6/d9/d1e/d20/d6b/f118 x:0 0 0 2024-03-29T08:07:29.478 INFO:tasks.workunit.client.1.smithi148.stdout:5/846: dread d0/db/d1e/d36/d41/f63 [0,4194304] 0 2024-03-29T08:07:29.479 INFO:tasks.workunit.client.0.smithi037.stdout:2/463: mknod d0/d23/d70/d71/ca2 0 2024-03-29T08:07:29.485 INFO:tasks.workunit.client.0.smithi037.stdout:0/487: dread d9/da/dd/d13/f5e [0,4194304] 0 2024-03-29T08:07:29.486 INFO:tasks.workunit.client.1.smithi148.stdout:4/901: dread df/d2c/d51/d77/d26/f63 [0,4194304] 0 2024-03-29T08:07:29.488 INFO:tasks.workunit.client.1.smithi148.stdout:6/872: symlink d6/d9/d1e/d20/db7/l119 0 2024-03-29T08:07:29.488 INFO:tasks.workunit.client.0.smithi037.stdout:4/610: fdatasync d3/d4/d15/d1d/f64 0 2024-03-29T08:07:29.489 INFO:tasks.workunit.client.0.smithi037.stdout:1/636: write d2/d6/dc/d5d/fd5 [5128147,62141] 0 2024-03-29T08:07:29.490 INFO:tasks.workunit.client.0.smithi037.stdout:2/464: write d0/d20/d22/d55/f5b [4788926,96972] 0 2024-03-29T08:07:29.492 INFO:tasks.workunit.client.1.smithi148.stdout:8/836: truncate d17/d49/d9a/dbe/d6f/f45 5411109 0 2024-03-29T08:07:29.494 INFO:tasks.workunit.client.1.smithi148.stdout:6/873: creat d6/d9/d1e/d36/d4d/d9d/d28/dba/f11a x:0 0 0 2024-03-29T08:07:29.495 INFO:tasks.workunit.client.1.smithi148.stdout:1/811: dwrite d3/f5d [0,4194304] 0 2024-03-29T08:07:29.498 INFO:tasks.workunit.client.0.smithi037.stdout:3/546: write dd/d46/d2d/d3a/f53 [550943,58181] 0 2024-03-29T08:07:29.498 INFO:tasks.workunit.client.1.smithi148.stdout:6/874: read - d6/d9/d1e/d20/d30/fcb zero size 2024-03-29T08:07:29.500 INFO:tasks.workunit.client.0.smithi037.stdout:7/615: mkdir dd/d21/d35/d3b/dce 0 2024-03-29T08:07:29.501 INFO:tasks.workunit.client.1.smithi148.stdout:6/875: write d6/d9/d1e/d20/d30/f33 [4486589,59891] 0 2024-03-29T08:07:29.503 INFO:tasks.workunit.client.1.smithi148.stdout:7/820: rename d8/d38/d3c/da9 to d8/dfb/d11b 0 2024-03-29T08:07:29.504 INFO:tasks.workunit.client.0.smithi037.stdout:8/568: dwrite d5/d10/d2d/d34/d50/d83/d46/d4e/f6a [0,4194304] 0 2024-03-29T08:07:29.504 INFO:tasks.workunit.client.0.smithi037.stdout:6/507: dwrite d0/d5/d1e/f7f [0,4194304] 0 2024-03-29T08:07:29.506 INFO:tasks.workunit.client.0.smithi037.stdout:9/785: truncate d1/d4/d19/d1e/d4a/d6f/da2/d13/d7c/faf 835475 0 2024-03-29T08:07:29.508 INFO:tasks.workunit.client.1.smithi148.stdout:9/903: sync 2024-03-29T08:07:29.510 INFO:tasks.workunit.client.1.smithi148.stdout:4/902: rename df/d2c/d51/d77/d26/d34/d8a/ff1 to df/d2c/d51/d77/d26/d34/d8a/d98/f11a 0 2024-03-29T08:07:29.514 INFO:tasks.workunit.client.0.smithi037.stdout:8/569: write d5/d10/d19/d3d/d4a/d4c/f8f [3482605,18992] 0 2024-03-29T08:07:29.515 INFO:tasks.workunit.client.0.smithi037.stdout:4/611: rmdir d3 39 2024-03-29T08:07:29.517 INFO:tasks.workunit.client.1.smithi148.stdout:0/851: creat db/de/df/f104 x:0 0 0 2024-03-29T08:07:29.518 INFO:tasks.workunit.client.1.smithi148.stdout:1/812: rename d3/d18/dae/dda/df3/dc2 to d3/de/d1d/d101 0 2024-03-29T08:07:29.520 INFO:tasks.workunit.client.1.smithi148.stdout:7/821: unlink d8/dfb/d11b/d37/dc8/fcb 0 2024-03-29T08:07:29.520 INFO:tasks.workunit.client.0.smithi037.stdout:8/570: dread - d5/d10/d19/d3d/d4a/d4c/f6e zero size 2024-03-29T08:07:29.520 INFO:tasks.workunit.client.1.smithi148.stdout:9/904: creat db/f127 x:0 0 0 2024-03-29T08:07:29.521 INFO:tasks.workunit.client.0.smithi037.stdout:2/465: rmdir d0/d20/d2e/d5f 39 2024-03-29T08:07:29.523 INFO:tasks.workunit.client.0.smithi037.stdout:1/637: dread - d2/d6/d13/dae/fc7 zero size 2024-03-29T08:07:29.523 INFO:tasks.workunit.client.0.smithi037.stdout:8/571: chown d5/d10/f1d 15835712 1 2024-03-29T08:07:29.524 INFO:tasks.workunit.client.1.smithi148.stdout:1/813: mkdir d3/d18/d71/d102 0 2024-03-29T08:07:29.526 INFO:tasks.workunit.client.0.smithi037.stdout:0/488: symlink d9/da/dd/d20/d55/d5f/d9f/la2 0 2024-03-29T08:07:29.526 INFO:tasks.workunit.client.0.smithi037.stdout:3/547: creat dd/d46/d3e/fb7 x:0 0 0 2024-03-29T08:07:29.528 INFO:tasks.workunit.client.0.smithi037.stdout:6/508: truncate d0/d5/d1f/f30 3784472 0 2024-03-29T08:07:29.528 INFO:tasks.workunit.client.1.smithi148.stdout:0/852: fdatasync db/de/d8e/da9/fca 0 2024-03-29T08:07:29.532 INFO:tasks.workunit.client.0.smithi037.stdout:8/572: mkdir d5/d10/d2d/d9c/da1/dbc 0 2024-03-29T08:07:29.534 INFO:tasks.workunit.client.0.smithi037.stdout:8/573: readlink d5/d10/d2d/l71 0 2024-03-29T08:07:29.534 INFO:tasks.workunit.client.0.smithi037.stdout:1/638: truncate d2/d6/dc/d18/d1d/dad/daf/fd4 4963862 0 2024-03-29T08:07:29.534 INFO:tasks.workunit.client.1.smithi148.stdout:9/905: stat db/d10/d6d/d73/da8/deb/dec/f10d 0 2024-03-29T08:07:29.535 INFO:tasks.workunit.client.1.smithi148.stdout:6/876: dwrite d6/d9/d1e/d36/d4d/d9d/d28/db3/dd2/fe7 [0,4194304] 0 2024-03-29T08:07:29.537 INFO:tasks.workunit.client.0.smithi037.stdout:0/489: symlink d9/d74/la3 0 2024-03-29T08:07:29.539 INFO:tasks.workunit.client.0.smithi037.stdout:3/548: unlink dd/d46/c51 0 2024-03-29T08:07:29.539 INFO:tasks.workunit.client.1.smithi148.stdout:4/903: rename df/d2c/d51/d77/d26/d34/d3e/l97 to df/d2c/d51/d77/d26/d34/d3e/dd8/l11b 0 2024-03-29T08:07:29.543 INFO:tasks.workunit.client.0.smithi037.stdout:0/490: read - d9/da/dd/d13/f96 zero size 2024-03-29T08:07:29.544 INFO:tasks.workunit.client.1.smithi148.stdout:8/837: write d17/d74/fb1 [494389,67578] 0 2024-03-29T08:07:29.545 INFO:tasks.workunit.client.0.smithi037.stdout:4/612: read d3/f21 [403703,86135] 0 2024-03-29T08:07:29.546 INFO:tasks.workunit.client.0.smithi037.stdout:3/549: write dd/d5e/faa [125277,92058] 0 2024-03-29T08:07:29.547 INFO:tasks.workunit.client.1.smithi148.stdout:2/938: dread d7/f1b [0,4194304] 0 2024-03-29T08:07:29.547 INFO:tasks.workunit.client.0.smithi037.stdout:6/509: creat d0/d5/d1e/d44/fad x:0 0 0 2024-03-29T08:07:29.548 INFO:tasks.workunit.client.1.smithi148.stdout:6/877: symlink d6/l11b 0 2024-03-29T08:07:29.551 INFO:tasks.workunit.client.0.smithi037.stdout:4/613: chown d3/d4/f34 41446 1 2024-03-29T08:07:29.551 INFO:tasks.workunit.client.1.smithi148.stdout:4/904: creat df/d2c/db2/f11c x:0 0 0 2024-03-29T08:07:29.553 INFO:tasks.workunit.client.0.smithi037.stdout:8/574: mkdir d5/d10/d2d/d34/d50/d87/dbd 0 2024-03-29T08:07:29.556 INFO:tasks.workunit.client.1.smithi148.stdout:0/853: dwrite db/de/df/d11/d2d/f36 [0,4194304] 0 2024-03-29T08:07:29.557 INFO:tasks.workunit.client.0.smithi037.stdout:4/614: fsync d3/d4/d15/d1d/db/d45/d1e/d59/d60/f75 0 2024-03-29T08:07:29.558 INFO:tasks.workunit.client.0.smithi037.stdout:3/550: read dd/d46/d3e/f7e [878675,48743] 0 2024-03-29T08:07:29.560 INFO:tasks.workunit.client.0.smithi037.stdout:2/466: symlink d0/d20/d2e/d5f/d6f/la3 0 2024-03-29T08:07:29.561 INFO:tasks.workunit.client.1.smithi148.stdout:7/822: link d8/d38/d3c/da5/db4/ce8 d8/dfb/d44/c11c 0 2024-03-29T08:07:29.561 INFO:tasks.workunit.client.1.smithi148.stdout:2/939: dread d7/da/d4c/f79 [0,4194304] 0 2024-03-29T08:07:29.562 INFO:tasks.workunit.client.1.smithi148.stdout:9/906: mknod db/d22/d26/d54/d4d/d35/d4b/c128 0 2024-03-29T08:07:29.564 INFO:tasks.workunit.client.0.smithi037.stdout:0/491: mkdir d9/da/dd/d13/da4 0 2024-03-29T08:07:29.565 INFO:tasks.workunit.client.1.smithi148.stdout:6/878: creat d6/d9/dc7/f11c x:0 0 0 2024-03-29T08:07:29.565 INFO:tasks.workunit.client.1.smithi148.stdout:4/905: mkdir df/d2c/d80/d11d 0 2024-03-29T08:07:29.567 INFO:tasks.workunit.client.1.smithi148.stdout:6/879: read d6/d9/d1e/d36/d4d/f115 [3567644,128879] 0 2024-03-29T08:07:29.568 INFO:tasks.workunit.client.1.smithi148.stdout:2/940: rename d7/lcf to d7/d24/d4b/d66/df1/l134 0 2024-03-29T08:07:29.572 INFO:tasks.workunit.client.0.smithi037.stdout:7/616: dread dd/d21/d65/f6f [0,4194304] 0 2024-03-29T08:07:29.573 INFO:tasks.workunit.client.1.smithi148.stdout:2/941: mknod d7/da/d4c/dc4/c135 0 2024-03-29T08:07:29.573 INFO:tasks.workunit.client.1.smithi148.stdout:6/880: write d6/d9/d1e/d36/d4d/d9d/d28/fea [548659,71665] 0 2024-03-29T08:07:29.577 INFO:tasks.workunit.client.0.smithi037.stdout:6/510: symlink d0/d5/d1e/lae 0 2024-03-29T08:07:29.584 INFO:tasks.workunit.client.1.smithi148.stdout:2/942: unlink d7/d24/d4b/d66/df1/fe3 0 2024-03-29T08:07:29.584 INFO:tasks.workunit.client.1.smithi148.stdout:0/854: dread db/dc/d78/f94 [0,4194304] 0 2024-03-29T08:07:29.586 INFO:tasks.workunit.client.0.smithi037.stdout:1/639: rename d2/d6/c81 to d2/d6/d13/cdd 0 2024-03-29T08:07:29.588 INFO:tasks.workunit.client.1.smithi148.stdout:7/823: mkdir d8/d11d 0 2024-03-29T08:07:29.592 INFO:tasks.workunit.client.1.smithi148.stdout:4/906: dwrite df/d2c/d51/d77/d26/d34/f10c [0,4194304] 0 2024-03-29T08:07:29.594 INFO:tasks.workunit.client.0.smithi037.stdout:9/786: dwrite d1/d3/f21 [0,4194304] 0 2024-03-29T08:07:29.595 INFO:tasks.workunit.client.1.smithi148.stdout:2/943: read d7/da/dcc/d11c/f9f [73600,68221] 0 2024-03-29T08:07:29.597 INFO:tasks.workunit.client.1.smithi148.stdout:6/881: dwrite d6/f70 [0,4194304] 0 2024-03-29T08:07:29.597 INFO:tasks.workunit.client.1.smithi148.stdout:8/838: write d17/d49/d9a/dbe/d6f/f43 [3946557,91215] 0 2024-03-29T08:07:29.599 INFO:tasks.workunit.client.1.smithi148.stdout:8/839: stat d17/d18 0 2024-03-29T08:07:29.600 INFO:tasks.workunit.client.1.smithi148.stdout:2/944: write d7/da/d33/d3c/f42 [276728,24948] 0 2024-03-29T08:07:29.600 INFO:tasks.workunit.client.1.smithi148.stdout:3/893: sync 2024-03-29T08:07:29.601 INFO:tasks.workunit.client.0.smithi037.stdout:4/615: dwrite d3/d4/d15/d1d/db/d45/d1e/d59/d7d/d97/fb0 [0,4194304] 0 2024-03-29T08:07:29.602 INFO:tasks.workunit.client.1.smithi148.stdout:7/824: mkdir d8/dfb/db8/d11e 0 2024-03-29T08:07:29.602 INFO:tasks.workunit.client.1.smithi148.stdout:1/814: dread d3/d18/da8/fa9 [0,4194304] 0 2024-03-29T08:07:29.602 INFO:tasks.workunit.client.0.smithi037.stdout:3/551: read - dd/d15/d18/d4b/da2/f94 zero size 2024-03-29T08:07:29.606 INFO:tasks.workunit.client.1.smithi148.stdout:1/815: readlink d3/de/d25/d51/ld2 0 2024-03-29T08:07:29.606 INFO:tasks.workunit.client.1.smithi148.stdout:7/825: write d8/d38/d6a/d91/fe2 [361290,44677] 0 2024-03-29T08:07:29.615 INFO:tasks.workunit.client.1.smithi148.stdout:6/882: creat d6/d9/d1e/d20/d6b/f11d x:0 0 0 2024-03-29T08:07:29.619 INFO:tasks.workunit.client.1.smithi148.stdout:0/855: rename db/de/df/d11/d42/fda to db/dc/daa/d47/f105 0 2024-03-29T08:07:29.621 INFO:tasks.workunit.client.1.smithi148.stdout:1/816: fsync d3/de/d1d/d47/d6d/d72/f8a 0 2024-03-29T08:07:29.621 INFO:tasks.workunit.client.0.smithi037.stdout:6/511: creat d0/d19/d34/d4e/faf x:0 0 0 2024-03-29T08:07:29.622 INFO:tasks.workunit.client.0.smithi037.stdout:6/512: dread - d0/d19/d34/f40 zero size 2024-03-29T08:07:29.623 INFO:tasks.workunit.client.0.smithi037.stdout:0/492: dread d9/da/dd/d13/f61 [0,4194304] 0 2024-03-29T08:07:29.627 INFO:tasks.workunit.client.1.smithi148.stdout:6/883: mkdir d6/d9/d1e/d27/d48/d73/ddd/d11e 0 2024-03-29T08:07:29.627 INFO:tasks.workunit.client.1.smithi148.stdout:8/840: fsync d17/d18/f3e 0 2024-03-29T08:07:29.628 INFO:tasks.workunit.client.1.smithi148.stdout:2/945: rename d7/da/c12 to d7/d8c/da4/d10c/c136 0 2024-03-29T08:07:29.630 INFO:tasks.workunit.client.0.smithi037.stdout:8/575: creat d5/d78/dac/fbe x:0 0 0 2024-03-29T08:07:29.630 INFO:tasks.workunit.client.1.smithi148.stdout:1/817: mkdir d3/d18/d4d/d76/d85/d103 0 2024-03-29T08:07:29.631 INFO:tasks.workunit.client.1.smithi148.stdout:7/826: mknod d8/df3/c11f 0 2024-03-29T08:07:29.632 INFO:tasks.workunit.client.0.smithi037.stdout:2/467: mkdir d0/d20/d2e/da4 0 2024-03-29T08:07:29.633 INFO:tasks.workunit.client.1.smithi148.stdout:0/856: fsync db/de/f4b 0 2024-03-29T08:07:29.634 INFO:tasks.workunit.client.1.smithi148.stdout:7/827: readlink d8/d38/l8f 0 2024-03-29T08:07:29.634 INFO:tasks.workunit.client.1.smithi148.stdout:1/818: creat d3/d18/d71/f104 x:0 0 0 2024-03-29T08:07:29.634 INFO:tasks.workunit.client.0.smithi037.stdout:9/787: fdatasync d1/d4/d19/d1e/d25/fc1 0 2024-03-29T08:07:29.636 INFO:tasks.workunit.client.0.smithi037.stdout:3/552: creat dd/d15/d27/d33/d84/fb8 x:0 0 0 2024-03-29T08:07:29.638 INFO:tasks.workunit.client.0.smithi037.stdout:4/616: unlink d3/d4/d15/d1d/db/d45/d1e/d59/d60/c76 0 2024-03-29T08:07:29.639 INFO:tasks.workunit.client.0.smithi037.stdout:7/617: mkdir dd/d21/d35/dcf 0 2024-03-29T08:07:29.640 INFO:tasks.workunit.client.0.smithi037.stdout:0/493: symlink d9/da/d1b/d22/la5 0 2024-03-29T08:07:29.642 INFO:tasks.workunit.client.1.smithi148.stdout:1/819: mknod d3/d18/d71/c105 0 2024-03-29T08:07:29.643 INFO:tasks.workunit.client.1.smithi148.stdout:7/828: write d8/d38/d6a/d91/fe2 [666998,2569] 0 2024-03-29T08:07:29.644 INFO:tasks.workunit.client.0.smithi037.stdout:3/553: dread - dd/d15/d27/d33/f90 zero size 2024-03-29T08:07:29.649 INFO:tasks.workunit.client.1.smithi148.stdout:4/907: dwrite df/d2c/d51/d77/d26/f4a [0,4194304] 0 2024-03-29T08:07:29.650 INFO:tasks.workunit.client.0.smithi037.stdout:9/788: dread d1/d4/d19/d1e/d4a/d6f/da2/d13/d7c/fc3 [0,4194304] 0 2024-03-29T08:07:29.651 INFO:tasks.workunit.client.0.smithi037.stdout:4/617: mknod d3/d4/cc6 0 2024-03-29T08:07:29.653 INFO:tasks.workunit.client.1.smithi148.stdout:7/829: creat d8/d14/d45/d4d/f120 x:0 0 0 2024-03-29T08:07:29.655 INFO:tasks.workunit.client.0.smithi037.stdout:7/618: mknod dd/d21/d35/d42/d5c/d8c/cd0 0 2024-03-29T08:07:29.657 INFO:tasks.workunit.client.0.smithi037.stdout:0/494: creat d9/da/dd/d13/d7b/fa6 x:0 0 0 2024-03-29T08:07:29.660 INFO:tasks.workunit.client.1.smithi148.stdout:6/884: truncate d6/d9/d1e/fee 999383 0 2024-03-29T08:07:29.664 INFO:tasks.workunit.client.0.smithi037.stdout:7/619: creat dd/d21/d35/d60/fd1 x:0 0 0 2024-03-29T08:07:29.665 INFO:tasks.workunit.client.0.smithi037.stdout:9/789: write d1/d4/d19/d1e/d25/fc1 [331965,46407] 0 2024-03-29T08:07:29.666 INFO:tasks.workunit.client.1.smithi148.stdout:3/894: dwrite de/d24/f27 [0,4194304] 0 2024-03-29T08:07:29.667 INFO:tasks.workunit.client.1.smithi148.stdout:7/830: creat d8/d38/d3c/da5/f121 x:0 0 0 2024-03-29T08:07:29.668 INFO:tasks.workunit.client.0.smithi037.stdout:4/618: rename d3/d4/d15/d1d/db/d45/d1e/d59/d60/f75 to d3/d4/d15/d1d/db/d45/d1e/d59/d7d/fc7 0 2024-03-29T08:07:29.669 INFO:tasks.workunit.client.1.smithi148.stdout:8/841: write d17/d18/f4d [3027203,49078] 0 2024-03-29T08:07:29.672 INFO:tasks.workunit.client.0.smithi037.stdout:3/554: link f2 dd/d15/d27/d33/d83/fb9 0 2024-03-29T08:07:29.673 INFO:tasks.workunit.client.1.smithi148.stdout:6/885: dread d6/d9/d1e/d36/d4d/d9d/d28/d98/ff5 [0,4194304] 0 2024-03-29T08:07:29.674 INFO:tasks.workunit.client.0.smithi037.stdout:0/495: creat d9/d74/d7d/d7e/fa7 x:0 0 0 2024-03-29T08:07:29.675 INFO:tasks.workunit.client.0.smithi037.stdout:3/555: readlink dd/d15/d27/d33/d75/d8f/lb5 0 2024-03-29T08:07:29.677 INFO:tasks.workunit.client.0.smithi037.stdout:3/556: stat dd/d46/d2d 0 2024-03-29T08:07:29.677 INFO:tasks.workunit.client.1.smithi148.stdout:2/946: dwrite d7/d24/d4b/f87 [0,4194304] 0 2024-03-29T08:07:29.677 INFO:tasks.workunit.client.0.smithi037.stdout:7/620: mknod dd/d72/cd2 0 2024-03-29T08:07:29.678 INFO:tasks.workunit.client.0.smithi037.stdout:6/513: write d0/d6a/f85 [91945,107693] 0 2024-03-29T08:07:29.683 INFO:tasks.workunit.client.1.smithi148.stdout:1/820: dwrite d3/de/d1d/d47/d6d/d72/f8a [0,4194304] 0 2024-03-29T08:07:29.683 INFO:tasks.workunit.client.0.smithi037.stdout:8/576: write d5/f3a [662576,80856] 0 2024-03-29T08:07:29.683 INFO:tasks.workunit.client.0.smithi037.stdout:7/621: dread - dd/d21/d35/fc0 zero size 2024-03-29T08:07:29.685 INFO:tasks.workunit.client.0.smithi037.stdout:4/619: rename d3/d4/d15/d1d/db/d45/d1e/d59/d60/l8c to d3/d4/d15/d1d/db/d45/d1e/d59/d60/lc8 0 2024-03-29T08:07:29.686 INFO:tasks.workunit.client.1.smithi148.stdout:6/886: chown d6/d9/d1e/d36/d4d/d9d/d28/dda/db4 522761 1 2024-03-29T08:07:29.686 INFO:tasks.workunit.client.0.smithi037.stdout:3/557: readlink dd/d15/l4e 0 2024-03-29T08:07:29.688 INFO:tasks.workunit.client.1.smithi148.stdout:4/908: dwrite df/d71/fce [0,4194304] 0 2024-03-29T08:07:29.690 INFO:tasks.workunit.client.0.smithi037.stdout:3/558: write dd/d5e/faa [884731,98130] 0 2024-03-29T08:07:29.692 INFO:tasks.workunit.client.1.smithi148.stdout:2/947: write d7/da/d33/fd3 [199545,99730] 0 2024-03-29T08:07:29.695 INFO:tasks.workunit.client.0.smithi037.stdout:1/640: dwrite d2/d6/dc/d5d/d62/f97 [0,4194304] 0 2024-03-29T08:07:29.695 INFO:tasks.workunit.client.0.smithi037.stdout:9/790: rename d1/d4/d19/d1e/d4a/d6f/da2/d31/daa/dba/ldc to d1/d1f/d3a/l10e 0 2024-03-29T08:07:29.696 INFO:tasks.workunit.client.1.smithi148.stdout:7/831: rename d8/dfb/d11b/l3a to d8/dfb/d44/d49/l122 0 2024-03-29T08:07:29.698 INFO:tasks.workunit.client.1.smithi148.stdout:1/821: creat d3/d18/d4d/f106 x:0 0 0 2024-03-29T08:07:29.698 INFO:tasks.workunit.client.1.smithi148.stdout:4/909: rmdir df/d2c/d51/d77/d26/d34/d3e/df3 39 2024-03-29T08:07:29.703 INFO:tasks.workunit.client.0.smithi037.stdout:3/559: dread - dd/d5e/f80 zero size 2024-03-29T08:07:29.703 INFO:tasks.workunit.client.1.smithi148.stdout:3/895: rename de/d24/d33/d39/d46/d30/d37/d79/f7c to de/d24/d33/d39/d46/d70/def/f120 0 2024-03-29T08:07:29.705 INFO:tasks.workunit.client.1.smithi148.stdout:4/910: readlink df/d2c/d51/d77/d26/d9b/lc7 0 2024-03-29T08:07:29.708 INFO:tasks.workunit.client.1.smithi148.stdout:8/842: write d17/d18/f3e [174556,75804] 0 2024-03-29T08:07:29.708 INFO:tasks.workunit.client.1.smithi148.stdout:2/948: truncate d7/da/d33/ded/d2e/f97 74556 0 2024-03-29T08:07:29.709 INFO:tasks.workunit.client.0.smithi037.stdout:0/496: rename d9/da/dd/d13/da4 to d9/da/dd/d54/da8 0 2024-03-29T08:07:29.713 INFO:tasks.workunit.client.1.smithi148.stdout:1/822: mkdir d3/de/d1d/d47/d6d/d84/d107 0 2024-03-29T08:07:29.714 INFO:tasks.workunit.client.1.smithi148.stdout:3/896: fsync de/d24/d33/d39/d46/f65 0 2024-03-29T08:07:29.715 INFO:tasks.workunit.client.0.smithi037.stdout:6/514: write d0/d19/d42/d60/f8b [9632739,42520] 0 2024-03-29T08:07:29.716 INFO:tasks.workunit.client.1.smithi148.stdout:4/911: unlink df/d2c/d51/d77/d26/d34/d43/c87 0 2024-03-29T08:07:29.719 INFO:tasks.workunit.client.1.smithi148.stdout:1/823: write d3/d18/d4d/f106 [120022,67549] 0 2024-03-29T08:07:29.720 INFO:tasks.workunit.client.1.smithi148.stdout:6/887: truncate d6/d9/d1e/d20/d6b/fe3 2689128 0 2024-03-29T08:07:29.721 INFO:tasks.workunit.client.0.smithi037.stdout:0/497: creat d9/d74/fa9 x:0 0 0 2024-03-29T08:07:29.722 INFO:tasks.workunit.client.1.smithi148.stdout:5/847: sync 2024-03-29T08:07:29.723 INFO:tasks.workunit.client.0.smithi037.stdout:6/515: fsync d0/d19/d34/f6b 0 2024-03-29T08:07:29.724 INFO:tasks.workunit.client.0.smithi037.stdout:8/577: link d5/d10/d19/d3d/c42 d5/d10/d2d/d34/d50/d83/d46/d8a/cbf 0 2024-03-29T08:07:29.725 INFO:tasks.workunit.client.1.smithi148.stdout:6/888: readlink d6/l46 0 2024-03-29T08:07:29.726 INFO:tasks.workunit.client.0.smithi037.stdout:9/791: creat d1/d4/d19/d1e/d4a/d6f/da2/d13/d7c/f10f x:0 0 0 2024-03-29T08:07:29.726 INFO:tasks.workunit.client.1.smithi148.stdout:3/897: stat de/d24/d33/d39/d46/d30/d37/d62/d63/dbf/dfc/l11d 0 2024-03-29T08:07:29.727 INFO:tasks.workunit.client.0.smithi037.stdout:3/560: dread dd/d15/d27/d33/d75/f8d [0,4194304] 0 2024-03-29T08:07:29.728 INFO:tasks.workunit.client.1.smithi148.stdout:8/843: rename d17/d18/d26/d4f/dde/c102 to d17/d18/c114 0 2024-03-29T08:07:29.728 INFO:tasks.workunit.client.0.smithi037.stdout:0/498: creat d9/da/d1b/d22/faa x:0 0 0 2024-03-29T08:07:29.731 INFO:tasks.workunit.client.1.smithi148.stdout:5/848: symlink d0/db/d1e/d5b/d66/d73/l113 0 2024-03-29T08:07:29.731 INFO:tasks.workunit.client.0.smithi037.stdout:6/516: symlink d0/d20/d43/lb0 0 2024-03-29T08:07:29.732 INFO:tasks.workunit.client.1.smithi148.stdout:1/824: symlink d3/de/d25/df1/l108 0 2024-03-29T08:07:29.734 INFO:tasks.workunit.client.1.smithi148.stdout:3/898: mknod de/d24/d33/d39/c121 0 2024-03-29T08:07:29.735 INFO:tasks.workunit.client.1.smithi148.stdout:5/849: mkdir d0/d46/d9e/d114 0 2024-03-29T08:07:29.736 INFO:tasks.workunit.client.0.smithi037.stdout:8/578: creat d5/d10/d2d/d34/d50/fc0 x:0 0 0 2024-03-29T08:07:29.736 INFO:tasks.workunit.client.0.smithi037.stdout:0/499: mkdir d9/da/dd/d20/d55/d5f/dab 0 2024-03-29T08:07:29.737 INFO:tasks.workunit.client.1.smithi148.stdout:3/899: fdatasync de/d24/d33/f8e 0 2024-03-29T08:07:29.737 INFO:tasks.workunit.client.0.smithi037.stdout:4/620: write d3/d4/d15/d1d/db/d45/d1e/d59/d79/d95/f9b [550634,122141] 0 2024-03-29T08:07:29.738 INFO:tasks.workunit.client.1.smithi148.stdout:5/850: stat d0/db/d1e/d36/d41/lc6 0 2024-03-29T08:07:29.741 INFO:tasks.workunit.client.0.smithi037.stdout:6/517: creat d0/d5/d3f/fb1 x:0 0 0 2024-03-29T08:07:29.742 INFO:tasks.workunit.client.1.smithi148.stdout:8/844: link d17/d49/d9a/dbe/d6f/f43 d17/d18/d26/d63/d7d/f115 0 2024-03-29T08:07:29.744 INFO:tasks.workunit.client.0.smithi037.stdout:4/621: fsync d3/d4/d15/f29 0 2024-03-29T08:07:29.745 INFO:tasks.workunit.client.0.smithi037.stdout:8/579: rmdir d5/d10/d19/d3d/d4a/d4c/d67/d85 39 2024-03-29T08:07:29.746 INFO:tasks.workunit.client.1.smithi148.stdout:5/851: symlink d0/db/d7b/dc9/l115 0 2024-03-29T08:07:29.747 INFO:tasks.workunit.client.0.smithi037.stdout:0/500: mkdir d9/da/dd/d20/d55/dac 0 2024-03-29T08:07:29.750 INFO:tasks.workunit.client.1.smithi148.stdout:2/949: dread d7/d24/f36 [0,4194304] 0 2024-03-29T08:07:29.751 INFO:tasks.workunit.client.1.smithi148.stdout:7/832: dread d8/dfb/d44/d49/f82 [0,4194304] 0 2024-03-29T08:07:29.752 INFO:tasks.workunit.client.0.smithi037.stdout:6/518: mknod d0/d19/d42/d60/cb2 0 2024-03-29T08:07:29.753 INFO:tasks.workunit.client.0.smithi037.stdout:0/501: readlink d9/da/dd/d13/l67 0 2024-03-29T08:07:29.754 INFO:tasks.workunit.client.1.smithi148.stdout:8/845: mkdir d17/d49/d9a/db8/dff/d116 0 2024-03-29T08:07:29.754 INFO:tasks.workunit.client.1.smithi148.stdout:7/833: chown d8/d14/fcc 57719305 1 2024-03-29T08:07:29.754 INFO:tasks.workunit.client.1.smithi148.stdout:4/912: dread df/d2c/d51/d77/d26/d34/d8a/d98/f11a [0,4194304] 0 2024-03-29T08:07:29.755 INFO:tasks.workunit.client.1.smithi148.stdout:5/852: mknod d0/db/d1e/d5b/d66/d73/c116 0 2024-03-29T08:07:29.755 INFO:tasks.workunit.client.0.smithi037.stdout:3/561: dwrite dd/d15/d27/d33/d75/fab [0,4194304] 0 2024-03-29T08:07:29.757 INFO:tasks.workunit.client.1.smithi148.stdout:8/846: chown d17/d18/d26/d4f/d94/cc3 262364099 1 2024-03-29T08:07:29.757 INFO:tasks.workunit.client.0.smithi037.stdout:9/792: link d1/d1f/d3a/l10e d1/d3/d44/l110 0 2024-03-29T08:07:29.758 INFO:tasks.workunit.client.0.smithi037.stdout:0/502: write d9/da/f11 [3733941,31952] 0 2024-03-29T08:07:29.759 INFO:tasks.workunit.client.1.smithi148.stdout:2/950: unlink d7/da/f40 0 2024-03-29T08:07:29.765 INFO:tasks.workunit.client.1.smithi148.stdout:9/907: sync 2024-03-29T08:07:29.765 INFO:tasks.workunit.client.1.smithi148.stdout:0/857: sync 2024-03-29T08:07:29.765 INFO:tasks.workunit.client.1.smithi148.stdout:1/825: dwrite d3/de/d1d/d47/d6d/d72/fba [0,4194304] 0 2024-03-29T08:07:29.765 INFO:tasks.workunit.client.1.smithi148.stdout:8/847: mknod d17/dce/de8/c117 0 2024-03-29T08:07:29.770 INFO:tasks.workunit.client.1.smithi148.stdout:3/900: link de/d24/d33/d39/d46/d30/d37/d62/d63/dbf/cfd de/d24/d81/c122 0 2024-03-29T08:07:29.772 INFO:tasks.workunit.client.1.smithi148.stdout:7/834: mknod d8/d38/d3c/da5/c123 0 2024-03-29T08:07:29.773 INFO:tasks.workunit.client.0.smithi037.stdout:6/519: mknod d0/d5/cb3 0 2024-03-29T08:07:29.775 INFO:tasks.workunit.client.1.smithi148.stdout:5/853: dread - d0/db/d1e/d35/d42/d67/d76/d81/d98/fc4 zero size 2024-03-29T08:07:29.776 INFO:tasks.workunit.client.0.smithi037.stdout:3/562: fsync dd/d15/d27/d33/d83/fb9 0 2024-03-29T08:07:29.779 INFO:tasks.workunit.client.0.smithi037.stdout:9/793: symlink d1/d4/d19/d1e/d4a/d6f/da2/d23/l111 0 2024-03-29T08:07:29.782 INFO:tasks.workunit.client.1.smithi148.stdout:3/901: creat de/d24/d33/d39/d7f/f123 x:0 0 0 2024-03-29T08:07:29.782 INFO:tasks.workunit.client.1.smithi148.stdout:8/848: dread d17/d74/fb1 [0,4194304] 0 2024-03-29T08:07:29.783 INFO:tasks.workunit.client.0.smithi037.stdout:0/503: dread d9/da/dd/d54/f56 [0,4194304] 0 2024-03-29T08:07:29.784 INFO:tasks.workunit.client.0.smithi037.stdout:6/520: creat d0/d19/d34/d4e/fb4 x:0 0 0 2024-03-29T08:07:29.784 INFO:tasks.workunit.client.1.smithi148.stdout:2/951: creat d7/da/d33/f137 x:0 0 0 2024-03-29T08:07:29.785 INFO:tasks.workunit.client.0.smithi037.stdout:9/794: fsync d1/d4/d19/d1e/d4a/d6f/da2/d31/d87/fd5 0 2024-03-29T08:07:29.786 INFO:tasks.workunit.client.1.smithi148.stdout:7/835: getdents d8/d11d 0 2024-03-29T08:07:29.787 INFO:tasks.workunit.client.1.smithi148.stdout:9/908: dwrite db/d10/d2d/f34 [0,4194304] 0 2024-03-29T08:07:29.788 INFO:tasks.workunit.client.0.smithi037.stdout:8/580: truncate d5/f15 1910044 0 2024-03-29T08:07:29.791 INFO:tasks.workunit.client.1.smithi148.stdout:7/836: dread - d8/d38/ff0 zero size 2024-03-29T08:07:29.793 INFO:tasks.workunit.client.0.smithi037.stdout:6/521: creat d0/d5/d1e/fb5 x:0 0 0 2024-03-29T08:07:29.796 INFO:tasks.workunit.client.1.smithi148.stdout:3/902: write de/d24/d33/d39/da3/dbc/fbe [3718785,26059] 0 2024-03-29T08:07:29.797 INFO:tasks.workunit.client.0.smithi037.stdout:9/795: creat d1/d1f/d3a/d9b/ddd/f112 x:0 0 0 2024-03-29T08:07:29.798 INFO:tasks.workunit.client.0.smithi037.stdout:3/563: getdents dd/d46/d2d/d3a/d5f 0 2024-03-29T08:07:29.798 INFO:tasks.workunit.client.1.smithi148.stdout:4/913: getdents df/d2c/d51/d77/d26 0 2024-03-29T08:07:29.799 INFO:tasks.workunit.client.0.smithi037.stdout:3/564: fdatasync dd/f3f 0 2024-03-29T08:07:29.800 INFO:tasks.workunit.client.1.smithi148.stdout:7/837: mkdir d8/d14/d124 0 2024-03-29T08:07:29.801 INFO:tasks.workunit.client.0.smithi037.stdout:6/522: mknod d0/d5/d1a/cb6 0 2024-03-29T08:07:29.801 INFO:tasks.workunit.client.1.smithi148.stdout:1/826: dwrite d3/de/d25/d51/f54 [0,4194304] 0 2024-03-29T08:07:29.803 INFO:tasks.workunit.client.1.smithi148.stdout:2/952: symlink d7/da/d4c/dc4/dc6/d112/d11e/l138 0 2024-03-29T08:07:29.805 INFO:tasks.workunit.client.0.smithi037.stdout:9/796: rename d1/d4/l47 to d1/d4/d19/d69/l113 0 2024-03-29T08:07:29.808 INFO:tasks.workunit.client.1.smithi148.stdout:8/849: rmdir d17/d74 39 2024-03-29T08:07:29.808 INFO:tasks.workunit.client.0.smithi037.stdout:4/622: dread d3/d4/d15/f1f [0,4194304] 0 2024-03-29T08:07:29.808 INFO:tasks.workunit.client.0.smithi037.stdout:6/523: truncate d0/d5/d3f/f96 643434 0 2024-03-29T08:07:29.809 INFO:tasks.workunit.client.0.smithi037.stdout:8/581: link d5/d10/d2d/d34/d50/fa2 d5/d10/d2d/d34/d50/d83/d46/d8a/fc1 0 2024-03-29T08:07:29.812 INFO:tasks.workunit.client.0.smithi037.stdout:8/582: dread - d5/d78/fab zero size 2024-03-29T08:07:29.814 INFO:tasks.workunit.client.0.smithi037.stdout:0/504: write d9/da/d1b/d22/d2c/f42 [1197992,75104] 0 2024-03-29T08:07:29.814 INFO:tasks.workunit.client.1.smithi148.stdout:3/903: creat de/d24/d33/d39/d46/d30/d37/d62/d63/dbf/f124 x:0 0 0 2024-03-29T08:07:29.814 INFO:tasks.workunit.client.1.smithi148.stdout:7/838: creat d8/d38/d3c/da5/de3/f125 x:0 0 0 2024-03-29T08:07:29.816 INFO:tasks.workunit.client.0.smithi037.stdout:3/565: dread dd/d46/f71 [4194304,4194304] 0 2024-03-29T08:07:29.818 INFO:tasks.workunit.client.1.smithi148.stdout:1/827: unlink d3/d18/dae/lbd 0 2024-03-29T08:07:29.818 INFO:tasks.workunit.client.1.smithi148.stdout:5/854: dwrite d0/db/d7b/d16/d2e/ddd/ffc [0,4194304] 0 2024-03-29T08:07:29.818 INFO:tasks.workunit.client.0.smithi037.stdout:4/623: mkdir d3/d4/d15/d1d/db/d45/d93/d96/da0/dc9 0 2024-03-29T08:07:29.819 INFO:tasks.workunit.client.0.smithi037.stdout:6/524: chown d0/d5/d1f/f30 655063 1 2024-03-29T08:07:29.822 INFO:tasks.workunit.client.0.smithi037.stdout:8/583: creat d5/d10/d45/fc2 x:0 0 0 2024-03-29T08:07:29.823 INFO:tasks.workunit.client.1.smithi148.stdout:9/909: dread f3 [0,4194304] 0 2024-03-29T08:07:29.823 INFO:tasks.workunit.client.1.smithi148.stdout:2/953: write d7/da/d4c/f79 [844618,71551] 0 2024-03-29T08:07:29.823 INFO:tasks.workunit.client.1.smithi148.stdout:3/904: chown de/d24/d33/d39/d46/d70/def/ff7 3 1 2024-03-29T08:07:29.826 INFO:tasks.workunit.client.1.smithi148.stdout:4/914: link df/d2c/c5f df/d2c/d51/d77/dfe/c11e 0 2024-03-29T08:07:29.826 INFO:tasks.workunit.client.1.smithi148.stdout:1/828: symlink d3/de/d25/d51/l109 0 2024-03-29T08:07:29.828 INFO:tasks.workunit.client.0.smithi037.stdout:0/505: rmdir d9/da/d1b/d22/d2c 39 2024-03-29T08:07:29.828 INFO:tasks.workunit.client.1.smithi148.stdout:7/839: rename d8/d14/d124 to d8/df4/d126 0 2024-03-29T08:07:29.833 INFO:tasks.workunit.client.0.smithi037.stdout:9/797: dread d1/d4/d19/d1e/d4a/d6f/da2/d23/f65 [0,4194304] 0 2024-03-29T08:07:29.834 INFO:tasks.workunit.client.1.smithi148.stdout:3/905: mknod de/d24/d33/d39/d46/d70/d101/d115/c125 0 2024-03-29T08:07:29.835 INFO:tasks.workunit.client.0.smithi037.stdout:6/525: symlink d0/d20/lb7 0 2024-03-29T08:07:29.835 INFO:tasks.workunit.client.1.smithi148.stdout:9/910: write db/d22/d26/d54/d4d/d35/d4b/d58/f8b [2453326,1530] 0 2024-03-29T08:07:29.835 INFO:tasks.workunit.client.1.smithi148.stdout:5/855: rename d0/db/d1e/d5b/d66/d73 to d0/db/d7b/d16/d87/d117 0 2024-03-29T08:07:29.840 INFO:tasks.workunit.client.1.smithi148.stdout:7/840: truncate d8/d38/d3c/da5/de3/f125 1046681 0 2024-03-29T08:07:29.840 INFO:tasks.workunit.client.1.smithi148.stdout:8/850: chown d17/d18/fa9 141 1 2024-03-29T08:07:29.841 INFO:tasks.workunit.client.1.smithi148.stdout:4/915: mknod df/d2c/d51/d77/d26/d34/d43/c11f 0 2024-03-29T08:07:29.847 INFO:tasks.workunit.client.1.smithi148.stdout:9/911: mknod db/d22/d26/d54/d4d/d35/d4b/dc3/d116/c129 0 2024-03-29T08:07:29.847 INFO:tasks.workunit.client.1.smithi148.stdout:0/858: dread db/de/d24/f38 [4194304,4194304] 0 2024-03-29T08:07:29.849 INFO:tasks.workunit.client.1.smithi148.stdout:5/856: rmdir d0/db/d1e/d35/d42/d67/d76/d81/d98/dd5/d9c/dc8 39 2024-03-29T08:07:29.852 INFO:tasks.workunit.client.1.smithi148.stdout:4/916: mknod df/d2c/d51/d77/d26/d34/d3e/dd8/c120 0 2024-03-29T08:07:29.853 INFO:tasks.workunit.client.1.smithi148.stdout:3/906: dread de/d24/d33/d39/d46/d30/d37/d62/d63/dbf/fd1 [0,4194304] 0 2024-03-29T08:07:29.855 INFO:tasks.workunit.client.0.smithi037.stdout:4/624: write d3/f21 [802950,49986] 0 2024-03-29T08:07:29.856 INFO:tasks.workunit.client.1.smithi148.stdout:3/907: dread - de/f110 zero size 2024-03-29T08:07:29.857 INFO:tasks.workunit.client.0.smithi037.stdout:3/566: link dd/d15/d18/d4b/da2/f94 dd/d15/d27/d33/d97/fba 0 2024-03-29T08:07:29.857 INFO:tasks.workunit.client.0.smithi037.stdout:6/526: write d0/d5/fb [1819884,106874] 0 2024-03-29T08:07:29.859 INFO:tasks.workunit.client.1.smithi148.stdout:4/917: chown df/f1e 2275 1 2024-03-29T08:07:29.864 INFO:tasks.workunit.client.0.smithi037.stdout:0/506: rename d9/d74/d7d/d7e/fa7 to d9/da/d1b/d22/fad 0 2024-03-29T08:07:29.865 INFO:tasks.workunit.client.1.smithi148.stdout:2/954: dwrite d7/da/d33/f5b [0,4194304] 0 2024-03-29T08:07:29.866 INFO:tasks.workunit.client.1.smithi148.stdout:0/859: creat db/de/df/d11/d2d/da3/dee/f106 x:0 0 0 2024-03-29T08:07:29.869 INFO:tasks.workunit.client.1.smithi148.stdout:3/908: creat de/f126 x:0 0 0 2024-03-29T08:07:29.870 INFO:tasks.workunit.client.0.smithi037.stdout:4/625: creat d3/d4/d15/d1d/db/d45/d1e/d59/d7d/fca x:0 0 0 2024-03-29T08:07:29.873 INFO:tasks.workunit.client.0.smithi037.stdout:8/584: dwrite d5/f32 [0,4194304] 0 2024-03-29T08:07:29.876 INFO:tasks.workunit.client.0.smithi037.stdout:3/567: getdents dd/d5e 0 2024-03-29T08:07:29.877 INFO:tasks.workunit.client.1.smithi148.stdout:8/851: mknod d17/d49/d9a/db8/dff/d116/c118 0 2024-03-29T08:07:29.878 INFO:tasks.workunit.client.1.smithi148.stdout:1/829: dwrite d3/d18/d71/fa7 [0,4194304] 0 2024-03-29T08:07:29.880 INFO:tasks.workunit.client.1.smithi148.stdout:7/841: getdents d8/dfb/db8 0 2024-03-29T08:07:29.886 INFO:tasks.workunit.client.1.smithi148.stdout:4/918: mknod df/d2c/d51/d77/d26/d34/d3e/c121 0 2024-03-29T08:07:29.888 INFO:tasks.workunit.client.1.smithi148.stdout:8/852: unlink d17/d18/cc0 0 2024-03-29T08:07:29.889 INFO:tasks.workunit.client.1.smithi148.stdout:3/909: mknod de/d24/d33/d39/d46/d30/d37/d62/de1/c127 0 2024-03-29T08:07:29.891 INFO:tasks.workunit.client.1.smithi148.stdout:2/955: read d7/da/d33/d3c/d54/d70/f7b [638278,31566] 0 2024-03-29T08:07:29.892 INFO:tasks.workunit.client.0.smithi037.stdout:9/798: rename d1/d1f/d3a/d9b/cfe to d1/d4/d19/d1e/d4a/d6f/da2/d13/c114 0 2024-03-29T08:07:29.896 INFO:tasks.workunit.client.0.smithi037.stdout:8/585: symlink d5/d10/d2d/d34/d50/d83/lc3 0 2024-03-29T08:07:29.897 INFO:tasks.workunit.client.1.smithi148.stdout:3/910: mkdir de/d3b/d49/d128 0 2024-03-29T08:07:29.898 INFO:tasks.workunit.client.0.smithi037.stdout:0/507: truncate d9/da/dd/d20/d38/d53/f62 2190946 0 2024-03-29T08:07:29.898 INFO:tasks.workunit.client.1.smithi148.stdout:8/853: read - d17/d74/d86/fd8 zero size 2024-03-29T08:07:29.901 INFO:tasks.workunit.client.1.smithi148.stdout:1/830: link d3/de/d21/l64 d3/de/d21/d5f/l10a 0 2024-03-29T08:07:29.901 INFO:tasks.workunit.client.1.smithi148.stdout:5/857: dread d0/db/d1e/d5b/fda [0,4194304] 0 2024-03-29T08:07:29.902 INFO:tasks.workunit.client.1.smithi148.stdout:9/912: dread db/d10/d2d/d4e/dd4/f1b [0,4194304] 0 2024-03-29T08:07:29.903 INFO:tasks.workunit.client.0.smithi037.stdout:4/626: creat d3/d4/d15/d1d/db/d45/d93/d96/da0/db2/fcb x:0 0 0 2024-03-29T08:07:29.904 INFO:tasks.workunit.client.0.smithi037.stdout:0/508: write d9/d74/d7d/f87 [4380072,33727] 0 2024-03-29T08:07:29.904 INFO:tasks.workunit.client.0.smithi037.stdout:6/527: rename d0/d5/d1e/d44/d71/laa to d0/d19/d42/d60/lb8 0 2024-03-29T08:07:29.905 INFO:tasks.workunit.client.0.smithi037.stdout:9/799: read d1/d4/d19/d1e/d4a/d6f/da2/d31/f38 [3254154,71662] 0 2024-03-29T08:07:29.908 INFO:tasks.workunit.client.1.smithi148.stdout:2/956: link d7/d8c/da4/d8e/fd6 d7/d8c/f139 0 2024-03-29T08:07:29.909 INFO:tasks.workunit.client.0.smithi037.stdout:0/509: symlink d9/da/dd/d20/d38/d8a/lae 0 2024-03-29T08:07:29.910 INFO:tasks.workunit.client.1.smithi148.stdout:3/911: stat de/d24/d33/d39/d46/d30/d37/ce0 0 2024-03-29T08:07:29.912 INFO:tasks.workunit.client.1.smithi148.stdout:1/831: creat d3/d18/d4d/d76/d85/f10b x:0 0 0 2024-03-29T08:07:29.912 INFO:tasks.workunit.client.1.smithi148.stdout:4/919: dwrite df/d2c/d51/d77/f83 [0,4194304] 0 2024-03-29T08:07:29.913 INFO:tasks.workunit.client.1.smithi148.stdout:2/957: chown d7/fbf 649543 1 2024-03-29T08:07:29.914 INFO:tasks.workunit.client.0.smithi037.stdout:4/627: dread d3/d4/d15/d1d/db/fa3 [0,4194304] 0 2024-03-29T08:07:29.920 INFO:tasks.workunit.client.1.smithi148.stdout:8/854: mkdir d17/d18/d26/d63/d66/d5f/dd0/d119 0 2024-03-29T08:07:29.920 INFO:tasks.workunit.client.1.smithi148.stdout:7/842: link d8/f36 d8/dfb/d44/d50/d94/f127 0 2024-03-29T08:07:29.920 INFO:tasks.workunit.client.1.smithi148.stdout:3/912: mknod de/d24/d33/d39/d71/c129 0 2024-03-29T08:07:29.921 INFO:tasks.workunit.client.0.smithi037.stdout:9/800: symlink d1/l115 0 2024-03-29T08:07:29.922 INFO:tasks.workunit.client.0.smithi037.stdout:4/628: read d3/d4/d15/d1d/db/d45/d1e/d59/d7d/fc7 [1119314,24602] 0 2024-03-29T08:07:29.922 INFO:tasks.workunit.client.0.smithi037.stdout:8/586: getdents d5/d10/d2d/d34/d50 0 2024-03-29T08:07:29.923 INFO:tasks.workunit.client.1.smithi148.stdout:6/889: dread f5 [0,4194304] 0 2024-03-29T08:07:29.926 INFO:tasks.workunit.client.0.smithi037.stdout:3/568: dwrite dd/d46/d3e/f7e [0,4194304] 0 2024-03-29T08:07:29.927 INFO:tasks.workunit.client.0.smithi037.stdout:9/801: unlink d1/d4/d19/d1e/d4a/fa0 0 2024-03-29T08:07:29.928 INFO:tasks.workunit.client.0.smithi037.stdout:4/629: symlink d3/d4/d15/d1d/d4f/lcc 0 2024-03-29T08:07:29.932 INFO:tasks.workunit.client.0.smithi037.stdout:4/630: write d3/d4/d15/f49 [5257195,56996] 0 2024-03-29T08:07:29.933 INFO:tasks.workunit.client.0.smithi037.stdout:9/802: readlink d1/d4/d19/d1e/d4a/d6f/da2/d13/d7c/l10b 0 2024-03-29T08:07:29.935 INFO:tasks.workunit.client.1.smithi148.stdout:7/843: rename d8/dfb/d11b/d37/lb2 to d8/d14/da8/l128 0 2024-03-29T08:07:29.935 INFO:tasks.workunit.client.1.smithi148.stdout:8/855: chown d17/d18/d26/d63/d7d/le6 292344 1 2024-03-29T08:07:29.936 INFO:tasks.workunit.client.0.smithi037.stdout:4/631: readlink d3/d4/d15/d1d/db/l7e 0 2024-03-29T08:07:29.936 INFO:tasks.workunit.client.0.smithi037.stdout:9/803: chown d1/d1f/d3a/d9b/ddd/ce8 7222554 1 2024-03-29T08:07:29.937 INFO:tasks.workunit.client.0.smithi037.stdout:3/569: getdents dd/d46/d3e 0 2024-03-29T08:07:29.940 INFO:tasks.workunit.client.1.smithi148.stdout:5/858: dread d0/db/d7b/f3b [0,4194304] 0 2024-03-29T08:07:29.941 INFO:tasks.workunit.client.0.smithi037.stdout:4/632: stat d3/d4/d15/d1d/db/d45/d1e/d59/db5 0 2024-03-29T08:07:29.942 INFO:tasks.workunit.client.0.smithi037.stdout:9/804: dread - d1/d4/d19/d1e/d71/d89/f10d zero size 2024-03-29T08:07:29.944 INFO:tasks.workunit.client.1.smithi148.stdout:7/844: chown d8/d38/d6a/d91 228359 1 2024-03-29T08:07:29.946 INFO:tasks.workunit.client.0.smithi037.stdout:3/570: rename dd/f23 to dd/d5e/fbb 0 2024-03-29T08:07:29.947 INFO:tasks.workunit.client.0.smithi037.stdout:9/805: write d1/d4/d19/d1e/d4a/d6f/da2/f100 [989562,48322] 0 2024-03-29T08:07:29.948 INFO:tasks.workunit.client.1.smithi148.stdout:0/860: dwrite db/de/df/d2b/f53 [0,4194304] 0 2024-03-29T08:07:29.950 INFO:tasks.workunit.client.0.smithi037.stdout:3/571: rmdir dd/d15 39 2024-03-29T08:07:29.952 INFO:tasks.workunit.client.1.smithi148.stdout:3/913: symlink de/d24/d33/d39/d46/d70/def/l12a 0 2024-03-29T08:07:29.957 INFO:tasks.workunit.client.0.smithi037.stdout:4/633: rename d3/d4/d15/d1d/db/d3d to d3/d80/dcd 0 2024-03-29T08:07:29.958 INFO:tasks.workunit.client.0.smithi037.stdout:9/806: mkdir d1/d4/d19/d1e/d71/d116 0 2024-03-29T08:07:29.960 INFO:tasks.workunit.client.0.smithi037.stdout:0/510: dwrite d9/da/dd/f23 [4194304,4194304] 0 2024-03-29T08:07:29.962 INFO:tasks.workunit.client.1.smithi148.stdout:7/845: symlink d8/d14/l129 0 2024-03-29T08:07:29.963 INFO:tasks.workunit.client.0.smithi037.stdout:4/634: chown d3/d4/d15/d1d/f64 0 1 2024-03-29T08:07:29.966 INFO:tasks.workunit.client.0.smithi037.stdout:0/511: fsync d9/da/f8d 0 2024-03-29T08:07:29.966 INFO:tasks.workunit.client.1.smithi148.stdout:1/832: dwrite d3/d18/fb1 [0,4194304] 0 2024-03-29T08:07:29.967 INFO:tasks.workunit.client.1.smithi148.stdout:7/846: dread - d8/dfb/db8/fee zero size 2024-03-29T08:07:29.969 INFO:tasks.workunit.client.1.smithi148.stdout:8/856: dread d17/d74/f54 [0,4194304] 0 2024-03-29T08:07:29.970 INFO:tasks.workunit.client.1.smithi148.stdout:6/890: truncate d6/d9/d1e/d36/d4d/d9d/d28/dba/f11a 170905 0 2024-03-29T08:07:29.971 INFO:tasks.workunit.client.1.smithi148.stdout:9/913: link db/d22/d26/le4 db/d22/d26/d54/d4d/d35/d4b/dc3/l12a 0 2024-03-29T08:07:29.973 INFO:tasks.workunit.client.1.smithi148.stdout:7/847: dread - d8/dfb/d11b/d37/f11a zero size 2024-03-29T08:07:29.973 INFO:tasks.workunit.client.1.smithi148.stdout:3/914: chown de/d24/d33/d39/d46/d30/d37/d79/fe3 242 1 2024-03-29T08:07:29.973 INFO:tasks.workunit.client.1.smithi148.stdout:5/859: creat d0/d46/d9e/d114/f118 x:0 0 0 2024-03-29T08:07:29.975 INFO:tasks.workunit.client.0.smithi037.stdout:9/807: chown d1/d4/d19/d1e/d4a/d6f/da2/d31/daa/dba/dbb/ccc 384984857 1 2024-03-29T08:07:29.980 INFO:tasks.workunit.client.1.smithi148.stdout:2/958: dwrite d7/da/d33/ded/d2e/d37/f6c [4194304,4194304] 0 2024-03-29T08:07:29.983 INFO:tasks.workunit.client.0.smithi037.stdout:4/635: creat d3/d4/d15/d1d/db/d45/fce x:0 0 0 2024-03-29T08:07:29.983 INFO:tasks.workunit.client.0.smithi037.stdout:9/808: dread - d1/d1f/d3a/d9b/ddd/f106 zero size 2024-03-29T08:07:29.985 INFO:tasks.workunit.client.1.smithi148.stdout:2/959: readlink d7/da/d33/lb3 0 2024-03-29T08:07:29.986 INFO:tasks.workunit.client.1.smithi148.stdout:9/914: dread - db/d22/d26/d54/d52/d9c/f125 zero size 2024-03-29T08:07:29.987 INFO:tasks.workunit.client.0.smithi037.stdout:9/809: chown d1/d1f/cf1 347 1 2024-03-29T08:07:29.987 INFO:tasks.workunit.client.1.smithi148.stdout:8/857: creat d17/d18/d26/d63/d7d/f11a x:0 0 0 2024-03-29T08:07:29.991 INFO:tasks.workunit.client.1.smithi148.stdout:4/920: dread df/d2c/d51/d77/d26/d9b/da7/f50 [0,4194304] 0 2024-03-29T08:07:29.996 INFO:tasks.workunit.client.1.smithi148.stdout:6/891: creat d6/d9/d1e/d36/d4d/d9d/d28/dda/db6/df3/dd3/f11f x:0 0 0 2024-03-29T08:07:29.998 INFO:tasks.workunit.client.0.smithi037.stdout:4/636: mknod d3/d80/dcd/ccf 0 2024-03-29T08:07:30.002 INFO:tasks.workunit.client.1.smithi148.stdout:0/861: creat db/de/df/d2b/f107 x:0 0 0 2024-03-29T08:07:30.004 INFO:tasks.workunit.client.1.smithi148.stdout:8/858: symlink d17/d1d/d7e/l11b 0 2024-03-29T08:07:30.005 INFO:tasks.workunit.client.0.smithi037.stdout:3/572: write dd/d15/f50 [853567,64970] 0 2024-03-29T08:07:30.008 INFO:tasks.workunit.client.1.smithi148.stdout:0/862: rename db/de/df/d2b/dfa to db/de/df/d2b/dfa/dfe/d108 22 2024-03-29T08:07:30.011 INFO:tasks.workunit.client.0.smithi037.stdout:3/573: unlink dd/d46/d2d/d3a/f52 0 2024-03-29T08:07:30.012 INFO:tasks.workunit.client.1.smithi148.stdout:4/921: symlink df/d2c/d51/d77/d26/d9b/da7/l122 0 2024-03-29T08:07:30.012 INFO:tasks.workunit.client.1.smithi148.stdout:0/863: fdatasync db/de/df/d11/ff4 0 2024-03-29T08:07:30.015 INFO:tasks.workunit.client.0.smithi037.stdout:0/512: dwrite d9/da/d1b/f3d [0,4194304] 0 2024-03-29T08:07:30.016 INFO:tasks.workunit.client.1.smithi148.stdout:4/922: mkdir df/d71/d123 0 2024-03-29T08:07:30.017 INFO:tasks.workunit.client.1.smithi148.stdout:1/833: dread d3/d18/f20 [0,4194304] 0 2024-03-29T08:07:30.017 INFO:tasks.workunit.client.1.smithi148.stdout:8/859: chown d17/d4e/c51 27512 1 2024-03-29T08:07:30.018 INFO:tasks.workunit.client.0.smithi037.stdout:4/637: mkdir d3/d4/d15/d1d/db/d45/d1e/d59/d60/dd0 0 2024-03-29T08:07:30.019 INFO:tasks.workunit.client.1.smithi148.stdout:0/864: write db/de/df/d11/d2d/da3/dee/f106 [445822,2162] 0 2024-03-29T08:07:30.025 INFO:tasks.workunit.client.1.smithi148.stdout:1/834: readlink d3/de/d25/df1/l108 0 2024-03-29T08:07:30.025 INFO:tasks.workunit.client.1.smithi148.stdout:7/848: write d8/dfb/f8c [206993,71498] 0 2024-03-29T08:07:30.028 INFO:tasks.workunit.client.1.smithi148.stdout:4/923: write df/d2c/d51/d77/dbe/f117 [182345,23588] 0 2024-03-29T08:07:30.028 INFO:tasks.workunit.client.0.smithi037.stdout:4/638: rename d3/d4/d15/d1d/db/d45/d1e/d59/d7d/ca2 to d3/d4/d15/d1d/db/d72/d98/cd1 0 2024-03-29T08:07:30.030 INFO:tasks.workunit.client.1.smithi148.stdout:8/860: symlink d17/d18/d26/d63/d66/d5f/dd0/l11c 0 2024-03-29T08:07:30.033 INFO:tasks.workunit.client.1.smithi148.stdout:3/915: write de/d24/d33/d39/da3/dbc/f11a [5106118,4387] 0 2024-03-29T08:07:30.033 INFO:tasks.workunit.client.1.smithi148.stdout:5/860: write d0/db/d1e/d35/d42/d67/d76/d81/d98/dd5/d9c/fbf [1012340,207] 0 2024-03-29T08:07:30.035 INFO:tasks.workunit.client.0.smithi037.stdout:3/574: dwrite dd/d46/d2d/d3a/f53 [0,4194304] 0 2024-03-29T08:07:30.036 INFO:tasks.workunit.client.1.smithi148.stdout:1/835: dread - d3/de/d1d/d101/dac/fd8 zero size 2024-03-29T08:07:30.036 INFO:tasks.workunit.client.1.smithi148.stdout:6/892: dwrite d6/d9/d1e/d36/d4d/f7a [0,4194304] 0 2024-03-29T08:07:30.038 INFO:tasks.workunit.client.0.smithi037.stdout:9/810: dwrite d1/f8d [4194304,4194304] 0 2024-03-29T08:07:30.041 INFO:tasks.workunit.client.1.smithi148.stdout:4/924: truncate df/f9a 1287217 0 2024-03-29T08:07:30.047 INFO:tasks.workunit.client.1.smithi148.stdout:5/861: mkdir d0/db/d1e/d6e/d119 0 2024-03-29T08:07:30.048 INFO:tasks.workunit.client.1.smithi148.stdout:3/916: dread - de/d24/d33/d39/d46/d30/d37/f88 zero size 2024-03-29T08:07:30.049 INFO:tasks.workunit.client.1.smithi148.stdout:5/862: readlink d0/l2b 0 2024-03-29T08:07:30.054 INFO:tasks.workunit.client.1.smithi148.stdout:9/915: dread db/d22/d26/d54/d4d/d35/f48 [0,4194304] 0 2024-03-29T08:07:30.054 INFO:tasks.workunit.client.1.smithi148.stdout:0/865: dread db/de/df/d11/d1f/fe8 [0,4194304] 0 2024-03-29T08:07:30.054 INFO:tasks.workunit.client.1.smithi148.stdout:2/960: dread d7/d24/d4b/f55 [0,4194304] 0 2024-03-29T08:07:30.057 INFO:tasks.workunit.client.0.smithi037.stdout:4/639: dwrite d3/d4/d27/fbc [0,4194304] 0 2024-03-29T08:07:30.058 INFO:tasks.workunit.client.1.smithi148.stdout:6/893: symlink d6/d9/d68/dca/l120 0 2024-03-29T08:07:30.062 INFO:tasks.workunit.client.0.smithi037.stdout:4/640: write d3/d57/d6b/faa [529988,116282] 0 2024-03-29T08:07:30.063 INFO:tasks.workunit.client.1.smithi148.stdout:4/925: creat df/d2c/d80/daa/f124 x:0 0 0 2024-03-29T08:07:30.064 INFO:tasks.workunit.client.1.smithi148.stdout:6/894: fdatasync d6/d9/d1e/fbb 0 2024-03-29T08:07:30.064 INFO:tasks.workunit.client.1.smithi148.stdout:1/836: write d3/d18/f39 [865289,58287] 0 2024-03-29T08:07:30.064 INFO:tasks.workunit.client.1.smithi148.stdout:0/866: fdatasync db/de/d8e/df1/ff3 0 2024-03-29T08:07:30.064 INFO:tasks.workunit.client.1.smithi148.stdout:7/849: rename d8/d38/c3d to d8/dfb/d44/d50/d85/c12a 0 2024-03-29T08:07:30.064 INFO:tasks.workunit.client.0.smithi037.stdout:3/575: dwrite dd/d15/f44 [0,4194304] 0 2024-03-29T08:07:30.068 INFO:tasks.workunit.client.1.smithi148.stdout:8/861: truncate d17/d18/d26/d63/d66/d5f/f8e 235528 0 2024-03-29T08:07:30.075 INFO:tasks.workunit.client.1.smithi148.stdout:9/916: mkdir db/d22/d26/d54/d52/d9c/d12b 0 2024-03-29T08:07:30.077 INFO:tasks.workunit.client.0.smithi037.stdout:4/641: fsync d3/d4/d15/d1d/db/f1a 0 2024-03-29T08:07:30.079 INFO:tasks.workunit.client.0.smithi037.stdout:9/811: write d1/d1f/d3a/d9b/ddd/f106 [776711,22157] 0 2024-03-29T08:07:30.079 INFO:tasks.workunit.client.0.smithi037.stdout:3/576: creat dd/d46/d2d/d3a/d6c/db1/fbc x:0 0 0 2024-03-29T08:07:30.081 INFO:tasks.workunit.client.1.smithi148.stdout:2/961: unlink d7/da/d33/ded/d2e/l8b 0 2024-03-29T08:07:30.081 INFO:tasks.workunit.client.1.smithi148.stdout:5/863: mknod d0/db/d1e/d35/d42/d67/d76/d81/d98/dd5/d9c/dc8/c11a 0 2024-03-29T08:07:30.083 INFO:tasks.workunit.client.1.smithi148.stdout:6/895: truncate d6/d9/d1e/f8a 596993 0 2024-03-29T08:07:30.083 INFO:tasks.workunit.client.1.smithi148.stdout:6/896: stat d6 0 2024-03-29T08:07:30.085 INFO:tasks.workunit.client.1.smithi148.stdout:8/862: mkdir d17/dce/d11d 0 2024-03-29T08:07:30.088 INFO:tasks.workunit.client.1.smithi148.stdout:0/867: write db/de/df/d11/d42/f67 [2953018,8834] 0 2024-03-29T08:07:30.089 INFO:tasks.workunit.client.1.smithi148.stdout:3/917: rename de/d24/d33/d39/da3/dbc/f11a to de/d24/f12b 0 2024-03-29T08:07:30.089 INFO:tasks.workunit.client.0.smithi037.stdout:0/513: dread d9/da/dd/d13/f95 [0,4194304] 0 2024-03-29T08:07:30.091 INFO:tasks.workunit.client.1.smithi148.stdout:8/863: write d17/d18/d26/d63/d7d/dc6/f113 [516640,84373] 0 2024-03-29T08:07:30.092 INFO:tasks.workunit.client.1.smithi148.stdout:1/837: dread d3/d18/d4d/d76/d85/fcd [0,4194304] 0 2024-03-29T08:07:30.093 INFO:tasks.workunit.client.1.smithi148.stdout:5/864: mknod d0/db/d1e/d5b/d66/d72/c11b 0 2024-03-29T08:07:30.094 INFO:tasks.workunit.client.0.smithi037.stdout:4/642: rmdir d3/d4/d15/d1d/db/d45/d93/d96/da0/db2 39 2024-03-29T08:07:30.097 INFO:tasks.workunit.client.0.smithi037.stdout:4/643: dread - d3/d4/d15/d1d/d4f/f87 zero size 2024-03-29T08:07:30.097 INFO:tasks.workunit.client.0.smithi037.stdout:9/812: mknod d1/d4/d19/d1e/d4a/d6f/da2/d68/c117 0 2024-03-29T08:07:30.098 INFO:tasks.workunit.client.1.smithi148.stdout:6/897: fdatasync d6/d9/d1e/d36/d4d/d9d/d28/dda/fc4 0 2024-03-29T08:07:30.099 INFO:tasks.workunit.client.1.smithi148.stdout:9/917: mkdir db/d10/d2d/d4e/d113/d12c 0 2024-03-29T08:07:30.100 INFO:tasks.workunit.client.1.smithi148.stdout:7/850: link d8/l107 d8/d38/d3c/da5/db4/de9/d114/l12b 0 2024-03-29T08:07:30.100 INFO:tasks.workunit.client.1.smithi148.stdout:1/838: chown d3/d55/ddf/lf4 1663442690 1 2024-03-29T08:07:30.100 INFO:tasks.workunit.client.0.smithi037.stdout:0/514: mknod d9/da/dd/d59/caf 0 2024-03-29T08:07:30.100 INFO:tasks.workunit.client.1.smithi148.stdout:3/918: symlink de/d24/d33/d39/d46/d70/d101/d115/l12c 0 2024-03-29T08:07:30.101 INFO:tasks.workunit.client.0.smithi037.stdout:9/813: read d1/d4/d19/d69/fc2 [190823,114991] 0 2024-03-29T08:07:30.101 INFO:tasks.workunit.client.1.smithi148.stdout:5/865: symlink d0/db/d1e/d5b/d66/dc7/l11c 0 2024-03-29T08:07:30.108 INFO:tasks.workunit.client.1.smithi148.stdout:8/864: mknod d17/d74/d103/c11e 0 2024-03-29T08:07:30.109 INFO:tasks.workunit.client.1.smithi148.stdout:6/898: write d6/d9/d1e/d36/d4d/d9d/f4a [7401525,69178] 0 2024-03-29T08:07:30.114 INFO:tasks.workunit.client.1.smithi148.stdout:1/839: read d3/de/f1b [1371975,46158] 0 2024-03-29T08:07:30.116 INFO:tasks.workunit.client.0.smithi037.stdout:4/644: mknod d3/d4/d15/d1d/db/d45/d1e/d59/d60/cd2 0 2024-03-29T08:07:30.121 INFO:tasks.workunit.client.1.smithi148.stdout:0/868: dwrite db/de/d24/dbf/f100 [0,4194304] 0 2024-03-29T08:07:30.125 INFO:tasks.workunit.client.1.smithi148.stdout:8/865: getdents d17/d1d 0 2024-03-29T08:07:30.126 INFO:tasks.workunit.client.0.smithi037.stdout:0/515: creat d9/da/d1b/fb0 x:0 0 0 2024-03-29T08:07:30.127 INFO:tasks.workunit.client.1.smithi148.stdout:9/918: rmdir db/d10/d2d/d4e/d5f/dd5 39 2024-03-29T08:07:30.128 INFO:tasks.workunit.client.0.smithi037.stdout:3/577: dwrite dd/d15/d18/d4b/f67 [0,4194304] 0 2024-03-29T08:07:30.130 INFO:tasks.workunit.client.1.smithi148.stdout:5/866: mknod d0/db/d1e/d36/c11d 0 2024-03-29T08:07:30.131 INFO:tasks.workunit.client.0.smithi037.stdout:0/516: stat d9/da/dd/l1e 0 2024-03-29T08:07:30.132 INFO:tasks.workunit.client.1.smithi148.stdout:0/869: fdatasync db/de/df/d11/d2d/fb2 0 2024-03-29T08:07:30.132 INFO:tasks.workunit.client.1.smithi148.stdout:6/899: dread - d6/d9/fdc zero size 2024-03-29T08:07:30.137 INFO:tasks.workunit.client.0.smithi037.stdout:9/814: write d1/d4/d19/d1e/d4a/d6f/da2/f3d [2977888,75133] 0 2024-03-29T08:07:30.139 INFO:tasks.workunit.client.0.smithi037.stdout:0/517: fdatasync d9/da/dd/d20/d38/d53/f6a 0 2024-03-29T08:07:30.141 INFO:tasks.workunit.client.1.smithi148.stdout:2/962: dwrite d7/da/dcc/d11c/fa9 [0,4194304] 0 2024-03-29T08:07:30.141 INFO:tasks.workunit.client.1.smithi148.stdout:1/840: dread d3/d18/dae/dda/df3/fd0 [0,4194304] 0 2024-03-29T08:07:30.141 INFO:tasks.workunit.client.0.smithi037.stdout:9/815: symlink d1/d4/d19/d1e/l118 0 2024-03-29T08:07:30.143 INFO:tasks.workunit.client.1.smithi148.stdout:5/867: write d0/db/d7b/d16/f7a [3278689,72908] 0 2024-03-29T08:07:30.143 INFO:tasks.workunit.client.0.smithi037.stdout:0/518: dread - d9/da/dd/d13/d83/f6d zero size 2024-03-29T08:07:30.148 INFO:tasks.workunit.client.0.smithi037.stdout:9/816: rename d1/d1f/d3a/d9b/ddd/f112 to d1/d4/d95/f119 0 2024-03-29T08:07:30.149 INFO:tasks.workunit.client.1.smithi148.stdout:6/900: chown d6/d9/d1e/d36/d4d/d9d/d28/f74 4751968 1 2024-03-29T08:07:30.152 INFO:tasks.workunit.client.0.smithi037.stdout:0/519: link d9/da/c70 d9/da/dd/d13/cb1 0 2024-03-29T08:07:30.152 INFO:tasks.workunit.client.1.smithi148.stdout:0/870: truncate db/de/df/d14/fe2 911527 0 2024-03-29T08:07:30.153 INFO:tasks.workunit.client.1.smithi148.stdout:5/868: stat d0/db/d1e/d35/d42/cbe 0 2024-03-29T08:07:30.155 INFO:tasks.workunit.client.1.smithi148.stdout:3/919: link de/d24/d33/d39/d46/d30/l47 de/d24/d33/d39/d46/d30/d37/d62/d63/dbf/l12d 0 2024-03-29T08:07:30.156 INFO:tasks.workunit.client.0.smithi037.stdout:4/645: truncate d3/d4/d15/f49 1527185 0 2024-03-29T08:07:30.159 INFO:tasks.workunit.client.0.smithi037.stdout:0/520: unlink d9/da/dd/d20/d38/d53/f79 0 2024-03-29T08:07:30.159 INFO:tasks.workunit.client.1.smithi148.stdout:2/963: creat d7/da/dcc/d11c/f13a x:0 0 0 2024-03-29T08:07:30.161 INFO:tasks.workunit.client.1.smithi148.stdout:1/841: unlink d3/de/d21/l64 0 2024-03-29T08:07:30.162 INFO:tasks.workunit.client.1.smithi148.stdout:0/871: read db/f7b [3051499,33525] 0 2024-03-29T08:07:30.166 INFO:tasks.workunit.client.1.smithi148.stdout:3/920: stat de/d24/d33/d39/d46/d70/fa9 0 2024-03-29T08:07:30.166 INFO:tasks.workunit.client.1.smithi148.stdout:8/866: dwrite d17/d18/d26/d4f/fee [0,4194304] 0 2024-03-29T08:07:30.170 INFO:tasks.workunit.client.1.smithi148.stdout:0/872: read - db/dc/daa/fe0 zero size 2024-03-29T08:07:30.170 INFO:tasks.workunit.client.0.smithi037.stdout:0/521: dread d9/da/dd/d13/f45 [0,4194304] 0 2024-03-29T08:07:30.171 INFO:tasks.workunit.client.1.smithi148.stdout:1/842: creat d3/de/d25/df1/f10c x:0 0 0 2024-03-29T08:07:30.175 INFO:tasks.workunit.client.1.smithi148.stdout:2/964: getdents d7/d24/d4b/d66 0 2024-03-29T08:07:30.177 INFO:tasks.workunit.client.1.smithi148.stdout:0/873: fsync db/de/df/d11/d2d/f36 0 2024-03-29T08:07:30.178 INFO:tasks.workunit.client.1.smithi148.stdout:7/851: dread d8/dfb/db8/de0/ff2 [0,4194304] 0 2024-03-29T08:07:30.179 INFO:tasks.workunit.client.0.smithi037.stdout:0/522: rename d9/da/f24 to d9/da/dd/d20/d55/dac/fb2 0 2024-03-29T08:07:30.179 INFO:tasks.workunit.client.0.smithi037.stdout:5/701: sync 2024-03-29T08:07:30.179 INFO:tasks.workunit.client.1.smithi148.stdout:1/843: rmdir d3/d18/d4d/d76 39 2024-03-29T08:07:30.179 INFO:tasks.workunit.client.1.smithi148.stdout:5/869: getdents d0/db/d7b/d16/d2e/ddd/de9 0 2024-03-29T08:07:30.181 INFO:tasks.workunit.client.0.smithi037.stdout:9/817: write d1/d4/d19/f64 [5024604,90089] 0 2024-03-29T08:07:30.181 INFO:tasks.workunit.client.1.smithi148.stdout:7/852: fdatasync d8/dfb/f108 0 2024-03-29T08:07:30.183 INFO:tasks.workunit.client.0.smithi037.stdout:9/818: chown d1/d4/d19/d1e/d71 30849 1 2024-03-29T08:07:30.184 INFO:tasks.workunit.client.1.smithi148.stdout:7/853: stat d8/dfb/d11b/fe5 0 2024-03-29T08:07:30.185 INFO:tasks.workunit.client.1.smithi148.stdout:2/965: unlink d7/da/d52/lf9 0 2024-03-29T08:07:30.186 INFO:tasks.workunit.client.1.smithi148.stdout:0/874: read db/de/df/d11/d2d/fc0 [644005,70189] 0 2024-03-29T08:07:30.186 INFO:tasks.workunit.client.0.smithi037.stdout:9/819: chown d1/d1f/d3a/f46 54 1 2024-03-29T08:07:30.187 INFO:tasks.workunit.client.0.smithi037.stdout:5/702: mkdir d8/d4d/ddf/df2 0 2024-03-29T08:07:30.188 INFO:tasks.workunit.client.1.smithi148.stdout:3/921: rename de/d24/d33/d39/d46/f25 to de/d24/d33/d39/d46/d109/f12e 0 2024-03-29T08:07:30.190 INFO:tasks.workunit.client.1.smithi148.stdout:5/870: read d0/db/d1e/d36/d7c/f80 [3380438,75505] 0 2024-03-29T08:07:30.190 INFO:tasks.workunit.client.1.smithi148.stdout:2/966: readlink d7/da/d33/ded/d2e/d37/ld2 0 2024-03-29T08:07:30.191 INFO:tasks.workunit.client.1.smithi148.stdout:7/854: unlink d8/f18 0 2024-03-29T08:07:30.195 INFO:tasks.workunit.client.1.smithi148.stdout:0/875: fdatasync db/de/d24/dbf/f100 0 2024-03-29T08:07:30.195 INFO:tasks.workunit.client.0.smithi037.stdout:9/820: unlink d1/d4/d19/d1e/d4a/d6f/d96/fa4 0 2024-03-29T08:07:30.196 INFO:tasks.workunit.client.0.smithi037.stdout:4/646: dwrite d3/d4/d15/f63 [4194304,4194304] 0 2024-03-29T08:07:30.197 INFO:tasks.workunit.client.1.smithi148.stdout:1/844: rename d3/d18/d4d/d8f/c7d to d3/d18/dae/dda/df3/c10d 0 2024-03-29T08:07:30.199 INFO:tasks.workunit.client.0.smithi037.stdout:4/647: chown d3/d57/d85/l9d 28556570 1 2024-03-29T08:07:30.199 INFO:tasks.workunit.client.0.smithi037.stdout:5/703: creat d8/d22/d37/dbc/ff3 x:0 0 0 2024-03-29T08:07:30.204 INFO:tasks.workunit.client.1.smithi148.stdout:2/967: rmdir d7/da/d4c/dc4/dc6/d10f 39 2024-03-29T08:07:30.205 INFO:tasks.workunit.client.0.smithi037.stdout:5/704: mknod d8/d4d/ddf/df2/cf4 0 2024-03-29T08:07:30.208 INFO:tasks.workunit.client.0.smithi037.stdout:5/705: write d8/d22/d4f/d70/f3f [974212,54225] 0 2024-03-29T08:07:30.212 INFO:tasks.workunit.client.1.smithi148.stdout:8/867: dwrite d17/d49/d9a/dbe/d6f/d64/da5/fb2 [0,4194304] 0 2024-03-29T08:07:30.212 INFO:tasks.workunit.client.0.smithi037.stdout:4/648: link d3/d4/d15/d1d/db/d45/d1e/d59/d7d/fca d3/d4/d15/d1d/db/d45/d1e/d59/dc1/fd3 0 2024-03-29T08:07:30.214 INFO:tasks.workunit.client.0.smithi037.stdout:4/649: dread - d3/d4/d15/d1d/db/d45/fce zero size 2024-03-29T08:07:30.215 INFO:tasks.workunit.client.0.smithi037.stdout:5/706: symlink d8/d4d/ddf/lf5 0 2024-03-29T08:07:30.215 INFO:tasks.workunit.client.1.smithi148.stdout:2/968: chown d7/d24/d4b/l81 30116 1 2024-03-29T08:07:30.216 INFO:tasks.workunit.client.0.smithi037.stdout:4/650: chown d3/f3a 1325 1 2024-03-29T08:07:30.219 INFO:tasks.workunit.client.0.smithi037.stdout:5/707: mkdir d8/d22/d37/dbc/d85/df6 0 2024-03-29T08:07:30.221 INFO:tasks.workunit.client.0.smithi037.stdout:4/651: unlink d3/d4/f35 0 2024-03-29T08:07:30.221 INFO:tasks.workunit.client.1.smithi148.stdout:9/919: dread db/f21 [0,4194304] 0 2024-03-29T08:07:30.221 INFO:tasks.workunit.client.1.smithi148.stdout:3/922: link de/d24/d33/d39/d46/ld3 de/d24/d81/l12f 0 2024-03-29T08:07:30.221 INFO:tasks.workunit.client.1.smithi148.stdout:6/901: dwrite d6/d9/fdc [0,4194304] 0 2024-03-29T08:07:30.222 INFO:tasks.workunit.client.1.smithi148.stdout:5/871: getdents d0/db/d1e/d35/d42/d67 0 2024-03-29T08:07:30.224 INFO:tasks.workunit.client.1.smithi148.stdout:8/868: fdatasync d17/d49/d9a/fe2 0 2024-03-29T08:07:30.224 INFO:tasks.workunit.client.1.smithi148.stdout:1/845: rename d3/d18/d4d/d76/d85/d103 to d3/de/d25/dfc/d10e 0 2024-03-29T08:07:30.225 INFO:tasks.workunit.client.0.smithi037.stdout:9/821: write d1/d4/d19/d1e/d4a/d6f/f86 [5224541,12904] 0 2024-03-29T08:07:30.226 INFO:tasks.workunit.client.1.smithi148.stdout:0/876: link db/de/d8e/da9/fca db/f109 0 2024-03-29T08:07:30.227 INFO:tasks.workunit.client.0.smithi037.stdout:5/708: creat d8/d22/dde/ff7 x:0 0 0 2024-03-29T08:07:30.229 INFO:tasks.workunit.client.1.smithi148.stdout:6/902: creat d6/d9/d1e/d20/d6b/f121 x:0 0 0 2024-03-29T08:07:30.230 INFO:tasks.workunit.client.0.smithi037.stdout:5/709: fsync d8/d22/d37/d8d/db3/dcb/fe1 0 2024-03-29T08:07:30.231 INFO:tasks.workunit.client.0.smithi037.stdout:4/652: creat d3/d4/d15/d1d/db/d45/d1e/d59/d73/fd4 x:0 0 0 2024-03-29T08:07:30.232 INFO:tasks.workunit.client.1.smithi148.stdout:0/877: stat db/de/df/d11/d1f 0 2024-03-29T08:07:30.232 INFO:tasks.workunit.client.1.smithi148.stdout:9/920: symlink db/d22/d26/d54/d4d/d35/d4b/d5a/d74/d10a/l12d 0 2024-03-29T08:07:30.234 INFO:tasks.workunit.client.0.smithi037.stdout:9/822: truncate d1/d4/d19/d1e/d4a/d5c/f5d 922379 0 2024-03-29T08:07:30.237 INFO:tasks.workunit.client.1.smithi148.stdout:0/878: write db/de/f70 [1041425,123277] 0 2024-03-29T08:07:30.238 INFO:tasks.workunit.client.1.smithi148.stdout:1/846: write d3/de/d1d/d47/d6d/d84/d8c/fce [226901,113390] 0 2024-03-29T08:07:30.238 INFO:tasks.workunit.client.0.smithi037.stdout:9/823: chown d1/d4/d19/d1e/d4a/d6f/da2/f7a 146015856 1 2024-03-29T08:07:30.246 INFO:tasks.workunit.client.0.smithi037.stdout:4/653: read d3/d4/d15/d1d/f71 [1307865,46470] 0 2024-03-29T08:07:30.247 INFO:tasks.workunit.client.0.smithi037.stdout:9/824: rename d1/f8d to d1/d4/d19/d1e/d4a/d6f/da2/d31/d87/d88/f11a 0 2024-03-29T08:07:30.248 INFO:tasks.workunit.client.1.smithi148.stdout:8/869: write d17/d49/d9a/dbe/d6f/f55 [934186,49684] 0 2024-03-29T08:07:30.252 INFO:tasks.workunit.client.1.smithi148.stdout:7/855: dwrite d8/dfb/fba [0,4194304] 0 2024-03-29T08:07:30.256 INFO:tasks.workunit.client.1.smithi148.stdout:6/903: write d6/d9/d1e/d36/d4d/d9d/d28/d98/dc2/ffd [117016,103786] 0 2024-03-29T08:07:30.259 INFO:tasks.workunit.client.0.smithi037.stdout:4/654: link d3/d4/d15/l6e d3/d4/d27/ld5 0 2024-03-29T08:07:30.260 INFO:tasks.workunit.client.1.smithi148.stdout:8/870: fdatasync d17/d74/fb4 0 2024-03-29T08:07:30.261 INFO:tasks.workunit.client.1.smithi148.stdout:7/856: symlink d8/d38/d6a/d91/l12c 0 2024-03-29T08:07:30.261 INFO:tasks.workunit.client.1.smithi148.stdout:1/847: rename d3/de/d1d/d47/d6d/d84/d107 to d3/de/d1d/d47/d6d/d72/d10f 0 2024-03-29T08:07:30.264 INFO:tasks.workunit.client.0.smithi037.stdout:4/655: rmdir d3/d4/d15/d1d/db/d72 39 2024-03-29T08:07:30.270 INFO:tasks.workunit.client.1.smithi148.stdout:8/871: chown d17/d49/d9a/dbe/d6f/fb9 4 1 2024-03-29T08:07:30.270 INFO:tasks.workunit.client.1.smithi148.stdout:7/857: symlink d8/df4/d126/l12d 0 2024-03-29T08:07:30.272 INFO:tasks.workunit.client.1.smithi148.stdout:1/848: mkdir d3/d18/dae/dda/df3/d110 0 2024-03-29T08:07:30.272 INFO:tasks.workunit.client.1.smithi148.stdout:0/879: write db/de/df/d11/f41 [2345162,17635] 0 2024-03-29T08:07:30.275 INFO:tasks.workunit.client.1.smithi148.stdout:7/858: stat d8/d38/d6a 0 2024-03-29T08:07:30.276 INFO:tasks.workunit.client.1.smithi148.stdout:6/904: write d6/d9/d1e/f8a [1185656,18370] 0 2024-03-29T08:07:30.277 INFO:tasks.workunit.client.1.smithi148.stdout:9/921: dwrite db/d22/d3f/fbc [0,4194304] 0 2024-03-29T08:07:30.279 INFO:tasks.workunit.client.0.smithi037.stdout:5/710: dread d8/d34/f4a [4194304,4194304] 0 2024-03-29T08:07:30.279 INFO:tasks.workunit.client.1.smithi148.stdout:8/872: creat d17/d1d/f11f x:0 0 0 2024-03-29T08:07:30.281 INFO:tasks.workunit.client.1.smithi148.stdout:5/872: dread d0/db/d1e/d35/d42/fde [0,4194304] 0 2024-03-29T08:07:30.282 INFO:tasks.workunit.client.0.smithi037.stdout:9/825: dwrite d1/d4/d19/d1e/d4a/d6f/da2/d13/d7c/fc3 [0,4194304] 0 2024-03-29T08:07:30.283 INFO:tasks.workunit.client.0.smithi037.stdout:5/711: rmdir d8/d4d 39 2024-03-29T08:07:30.284 INFO:tasks.workunit.client.1.smithi148.stdout:3/923: dread de/d24/d33/d39/d46/d30/d37/fc3 [0,4194304] 0 2024-03-29T08:07:30.285 INFO:tasks.workunit.client.1.smithi148.stdout:4/926: sync 2024-03-29T08:07:30.285 INFO:tasks.workunit.client.1.smithi148.stdout:0/880: chown db/de/df/d2b/db9/l103 0 1 2024-03-29T08:07:30.287 INFO:tasks.workunit.client.1.smithi148.stdout:1/849: mkdir d3/de/d1d/d97/d111 0 2024-03-29T08:07:30.288 INFO:tasks.workunit.client.1.smithi148.stdout:7/859: chown d8/dfb/db8/de0/l117 229413 1 2024-03-29T08:07:30.289 INFO:tasks.workunit.client.1.smithi148.stdout:8/873: read d17/d18/f8a [59010,111651] 0 2024-03-29T08:07:30.289 INFO:tasks.workunit.client.0.smithi037.stdout:5/712: creat d8/db9/ff8 x:0 0 0 2024-03-29T08:07:30.291 INFO:tasks.workunit.client.1.smithi148.stdout:6/905: chown d6/d9/d1e/d20/cd1 54 1 2024-03-29T08:07:30.292 INFO:tasks.workunit.client.1.smithi148.stdout:5/873: stat d0/d46/d86 0 2024-03-29T08:07:30.292 INFO:tasks.workunit.client.0.smithi037.stdout:4/656: dwrite d3/d4/d15/d1d/db/d45/d1e/d59/d7d/d97/fb0 [0,4194304] 0 2024-03-29T08:07:30.294 INFO:tasks.workunit.client.0.smithi037.stdout:4/657: fsync d3/d54/f5b 0 2024-03-29T08:07:30.295 INFO:tasks.workunit.client.1.smithi148.stdout:4/927: readlink df/d2c/d51/d77/d26/d9b/da7/l122 0 2024-03-29T08:07:30.296 INFO:tasks.workunit.client.0.smithi037.stdout:5/713: write d8/d22/d4f/d70/f9f [4912841,30520] 0 2024-03-29T08:07:30.296 INFO:tasks.workunit.client.1.smithi148.stdout:1/850: chown d3/de/d21/d7f/lc4 0 1 2024-03-29T08:07:30.298 INFO:tasks.workunit.client.1.smithi148.stdout:5/874: symlink d0/l11e 0 2024-03-29T08:07:30.299 INFO:tasks.workunit.client.1.smithi148.stdout:7/860: mknod d8/d60/c12e 0 2024-03-29T08:07:30.300 INFO:tasks.workunit.client.1.smithi148.stdout:0/881: creat db/de/df/d14/dea/f10a x:0 0 0 2024-03-29T08:07:30.303 INFO:tasks.workunit.client.1.smithi148.stdout:1/851: chown d3/de/d21/d7f 0 1 2024-03-29T08:07:30.304 INFO:tasks.workunit.client.0.smithi037.stdout:4/658: creat d3/d4/d15/d1d/db/d45/d1e/d59/d73/fd6 x:0 0 0 2024-03-29T08:07:30.306 INFO:tasks.workunit.client.0.smithi037.stdout:5/714: fsync d8/f52 0 2024-03-29T08:07:30.307 INFO:tasks.workunit.client.1.smithi148.stdout:6/906: readlink d6/d9/d1e/d27/d48/lb5 0 2024-03-29T08:07:30.311 INFO:tasks.workunit.client.1.smithi148.stdout:3/924: symlink de/d24/d33/d39/d46/d30/d37/d62/d63/dbf/dfc/l130 0 2024-03-29T08:07:30.314 INFO:tasks.workunit.client.1.smithi148.stdout:0/882: creat db/dc/daa/d47/d9f/f10b x:0 0 0 2024-03-29T08:07:30.314 INFO:tasks.workunit.client.1.smithi148.stdout:7/861: truncate d8/f36 3992915 0 2024-03-29T08:07:30.315 INFO:tasks.workunit.client.0.smithi037.stdout:9/826: dwrite d1/d4/d19/d1e/d4a/d6f/da2/d31/d87/d88/f11a [0,4194304] 0 2024-03-29T08:07:30.316 INFO:tasks.workunit.client.1.smithi148.stdout:5/875: mknod d0/db/d1e/d36/d41/c11f 0 2024-03-29T08:07:30.316 INFO:tasks.workunit.client.0.smithi037.stdout:5/715: fdatasync d8/d22/d4f/d70/f54 0 2024-03-29T08:07:30.321 INFO:tasks.workunit.client.1.smithi148.stdout:0/883: fsync db/de/d24/dbf/f100 0 2024-03-29T08:07:30.321 INFO:tasks.workunit.client.1.smithi148.stdout:9/922: dwrite db/d22/d3f/fbc [0,4194304] 0 2024-03-29T08:07:30.321 INFO:tasks.workunit.client.1.smithi148.stdout:3/925: creat de/d24/d33/d39/d46/d70/d101/f131 x:0 0 0 2024-03-29T08:07:30.324 INFO:tasks.workunit.client.1.smithi148.stdout:7/862: write d8/d14/f2b [4691665,128501] 0 2024-03-29T08:07:30.324 INFO:tasks.workunit.client.0.smithi037.stdout:9/827: creat d1/d4/d19/d1e/d71/d89/d92/f11b x:0 0 0 2024-03-29T08:07:30.326 INFO:tasks.workunit.client.1.smithi148.stdout:8/874: truncate d17/d18/d26/d4f/fee 485399 0 2024-03-29T08:07:30.328 INFO:tasks.workunit.client.0.smithi037.stdout:9/828: stat d1/d4/d19/d1e/d71/d89/d92/ld6 0 2024-03-29T08:07:30.331 INFO:tasks.workunit.client.0.smithi037.stdout:9/829: readlink d1/d4/d19/d69/ld1 0 2024-03-29T08:07:30.335 INFO:tasks.workunit.client.1.smithi148.stdout:6/907: write d6/d9/d1e/d20/d30/fcb [500675,48367] 0 2024-03-29T08:07:30.337 INFO:tasks.workunit.client.1.smithi148.stdout:4/928: dread fa [0,4194304] 0 2024-03-29T08:07:30.340 INFO:tasks.workunit.client.0.smithi037.stdout:4/659: dwrite d3/d54/fc0 [0,4194304] 0 2024-03-29T08:07:30.340 INFO:tasks.workunit.client.1.smithi148.stdout:7/863: write d8/d38/d3c/da5/db4/de9/fed [1836766,111877] 0 2024-03-29T08:07:30.340 INFO:tasks.workunit.client.1.smithi148.stdout:9/923: stat db/d10/d2d/d4e/d5f/dd5/f112 0 2024-03-29T08:07:30.341 INFO:tasks.workunit.client.1.smithi148.stdout:5/876: dread d0/db/d84/f10f [0,4194304] 0 2024-03-29T08:07:30.343 INFO:tasks.workunit.client.0.smithi037.stdout:9/830: mknod d1/d4/d19/d1e/d4a/d6f/da2/d31/d87/d88/c11c 0 2024-03-29T08:07:30.350 INFO:tasks.workunit.client.0.smithi037.stdout:9/831: symlink d1/d1f/d3a/l11d 0 2024-03-29T08:07:30.352 INFO:tasks.workunit.client.1.smithi148.stdout:8/875: chown d17/d49/d9a/dbe/cc8 118600150 1 2024-03-29T08:07:30.352 INFO:tasks.workunit.client.1.smithi148.stdout:6/908: read d6/fe [4837253,48735] 0 2024-03-29T08:07:30.353 INFO:tasks.workunit.client.1.smithi148.stdout:4/929: mkdir df/d2c/d10b/d125 0 2024-03-29T08:07:30.353 INFO:tasks.workunit.client.0.smithi037.stdout:5/716: dread d8/d18/f21 [0,4194304] 0 2024-03-29T08:07:30.356 INFO:tasks.workunit.client.0.smithi037.stdout:9/832: creat d1/d3/d44/f11e x:0 0 0 2024-03-29T08:07:30.357 INFO:tasks.workunit.client.0.smithi037.stdout:5/717: truncate d8/d22/dc1/fd2 44559 0 2024-03-29T08:07:30.357 INFO:tasks.workunit.client.1.smithi148.stdout:1/852: dwrite d3/d18/da8/ff2 [4194304,4194304] 0 2024-03-29T08:07:30.360 INFO:tasks.workunit.client.0.smithi037.stdout:9/833: truncate d1/d4/f28 701685 0 2024-03-29T08:07:30.361 INFO:tasks.workunit.client.1.smithi148.stdout:6/909: write d6/d9/dc7/f11c [742906,117126] 0 2024-03-29T08:07:30.363 INFO:tasks.workunit.client.0.smithi037.stdout:5/718: mkdir d8/d22/d37/d8d/db3/df9 0 2024-03-29T08:07:30.363 INFO:tasks.workunit.client.0.smithi037.stdout:9/834: chown d1/d4/fea 3546 1 2024-03-29T08:07:30.363 INFO:tasks.workunit.client.1.smithi148.stdout:3/926: dread de/d24/d33/d39/d46/d30/d37/fc3 [0,4194304] 0 2024-03-29T08:07:30.366 INFO:tasks.workunit.client.1.smithi148.stdout:9/924: rename db/d22/d26/d54/d4d/dbb/def/c11f to db/d10/d6d/d73/c12e 0 2024-03-29T08:07:30.367 INFO:tasks.workunit.client.1.smithi148.stdout:2/969: sync 2024-03-29T08:07:30.368 INFO:tasks.workunit.client.0.smithi037.stdout:9/835: unlink d1/d4/d19/d1e/d4a/d6f/da2/d13/l85 0 2024-03-29T08:07:30.372 INFO:tasks.workunit.client.1.smithi148.stdout:8/876: creat d17/d18/dbd/f120 x:0 0 0 2024-03-29T08:07:30.374 INFO:tasks.workunit.client.0.smithi037.stdout:5/719: mkdir d8/d22/d37/d8d/db3/dcb/deb/dfa 0 2024-03-29T08:07:30.376 INFO:tasks.workunit.client.0.smithi037.stdout:7/622: sync 2024-03-29T08:07:30.376 INFO:tasks.workunit.client.0.smithi037.stdout:2/468: sync 2024-03-29T08:07:30.378 INFO:tasks.workunit.client.1.smithi148.stdout:3/927: creat de/d24/d33/d39/d46/d109/f132 x:0 0 0 2024-03-29T08:07:30.379 INFO:tasks.workunit.client.1.smithi148.stdout:1/853: link d3/d55/f43 d3/d18/d71/ddc/f112 0 2024-03-29T08:07:30.381 INFO:tasks.workunit.client.1.smithi148.stdout:4/930: dread df/d2c/d51/d77/d26/d34/d43/fc8 [0,4194304] 0 2024-03-29T08:07:30.385 INFO:tasks.workunit.client.1.smithi148.stdout:2/970: getdents d7/da/dcc/d11c/d8d/d127/d12f 0 2024-03-29T08:07:30.389 INFO:tasks.workunit.client.1.smithi148.stdout:1/854: truncate d3/d18/f39 1118875 0 2024-03-29T08:07:30.395 INFO:tasks.workunit.client.1.smithi148.stdout:4/931: creat df/d60/f126 x:0 0 0 2024-03-29T08:07:30.398 INFO:tasks.workunit.client.1.smithi148.stdout:0/884: dread db/de/df/d2b/f34 [0,4194304] 0 2024-03-29T08:07:30.399 INFO:tasks.workunit.client.1.smithi148.stdout:2/971: creat d7/da/d4c/dd1/f13b x:0 0 0 2024-03-29T08:07:30.399 INFO:tasks.workunit.client.1.smithi148.stdout:1/855: stat d3/de/d21/d5f/fa3 0 2024-03-29T08:07:30.401 INFO:tasks.workunit.client.1.smithi148.stdout:5/877: dread d0/db/d7b/d16/d2e/f56 [0,4194304] 0 2024-03-29T08:07:30.402 INFO:tasks.workunit.client.1.smithi148.stdout:9/925: dwrite db/d10/d2d/d4e/da6/fac [0,4194304] 0 2024-03-29T08:07:30.404 INFO:tasks.workunit.client.1.smithi148.stdout:3/928: link de/d24/d33/d39/d46/d8d/l116 de/d24/d33/d39/da3/dbc/l133 0 2024-03-29T08:07:30.405 INFO:tasks.workunit.client.1.smithi148.stdout:7/864: dread d8/d38/d3c/da5/fae [0,4194304] 0 2024-03-29T08:07:30.406 INFO:tasks.workunit.client.1.smithi148.stdout:4/932: symlink df/d2c/d51/d101/l127 0 2024-03-29T08:07:30.408 INFO:tasks.workunit.client.1.smithi148.stdout:0/885: unlink db/dc/daa/d47/f105 0 2024-03-29T08:07:30.409 INFO:tasks.workunit.client.0.smithi037.stdout:5/720: mknod d8/d22/d4f/cfb 0 2024-03-29T08:07:30.409 INFO:tasks.workunit.client.1.smithi148.stdout:5/878: symlink d0/db/d7b/d16/d87/d117/l120 0 2024-03-29T08:07:30.411 INFO:tasks.workunit.client.1.smithi148.stdout:1/856: symlink d3/l113 0 2024-03-29T08:07:30.411 INFO:tasks.workunit.client.1.smithi148.stdout:2/972: creat d7/da/d52/da3/f13c x:0 0 0 2024-03-29T08:07:30.412 INFO:tasks.workunit.client.0.smithi037.stdout:2/469: unlink d0/cf 0 2024-03-29T08:07:30.415 INFO:tasks.workunit.client.1.smithi148.stdout:6/910: dread d6/d9/d68/fad [0,4194304] 0 2024-03-29T08:07:30.416 INFO:tasks.workunit.client.1.smithi148.stdout:9/926: creat db/d22/d26/d54/d4d/d35/d4b/d5a/d74/d10a/f12f x:0 0 0 2024-03-29T08:07:30.417 INFO:tasks.workunit.client.1.smithi148.stdout:0/886: creat db/de/df/d11/d2d/da3/f10c x:0 0 0 2024-03-29T08:07:30.420 INFO:tasks.workunit.client.1.smithi148.stdout:2/973: unlink d7/da/lc9 0 2024-03-29T08:07:30.421 INFO:tasks.workunit.client.0.smithi037.stdout:2/470: dread d0/d23/d44/f45 [0,4194304] 0 2024-03-29T08:07:30.421 INFO:tasks.workunit.client.1.smithi148.stdout:9/927: symlink db/d22/d26/d54/d52/dab/l130 0 2024-03-29T08:07:30.421 INFO:tasks.workunit.client.1.smithi148.stdout:5/879: getdents d0/db/d7b/d16/d2e/ddd/de9/def 0 2024-03-29T08:07:30.424 INFO:tasks.workunit.client.0.smithi037.stdout:5/721: link d8/d4d/d5c/c89 d8/d4d/d5c/cfc 0 2024-03-29T08:07:30.424 INFO:tasks.workunit.client.1.smithi148.stdout:2/974: write d7/da/d4c/f79 [1688738,25711] 0 2024-03-29T08:07:30.426 INFO:tasks.workunit.client.1.smithi148.stdout:7/865: creat d8/d38/d3c/da5/f12f x:0 0 0 2024-03-29T08:07:30.428 INFO:tasks.workunit.client.1.smithi148.stdout:9/928: mkdir db/d22/d131 0 2024-03-29T08:07:30.431 INFO:tasks.workunit.client.0.smithi037.stdout:5/722: unlink d8/d22/lb4 0 2024-03-29T08:07:30.433 INFO:tasks.workunit.client.1.smithi148.stdout:7/866: dread - d8/d14/d45/fbe zero size 2024-03-29T08:07:30.433 INFO:tasks.workunit.client.1.smithi148.stdout:4/933: rmdir df/d2c/d51 39 2024-03-29T08:07:30.434 INFO:tasks.workunit.client.0.smithi037.stdout:5/723: truncate d8/d18/d24/f74 780967 0 2024-03-29T08:07:30.434 INFO:tasks.workunit.client.1.smithi148.stdout:3/929: write de/d3b/d49/fdd [532586,89919] 0 2024-03-29T08:07:30.436 INFO:tasks.workunit.client.0.smithi037.stdout:5/724: read - d8/d22/d37/dbc/fd4 zero size 2024-03-29T08:07:30.436 INFO:tasks.workunit.client.0.smithi037.stdout:9/836: write d1/d4/fde [247119,70699] 0 2024-03-29T08:07:30.437 INFO:tasks.workunit.client.1.smithi148.stdout:5/880: getdents d0/db/d1e/d35/d42/d10b 0 2024-03-29T08:07:30.438 INFO:tasks.workunit.client.1.smithi148.stdout:2/975: creat d7/da/d33/d3c/d54/db6/f13d x:0 0 0 2024-03-29T08:07:30.440 INFO:tasks.workunit.client.1.smithi148.stdout:3/930: mknod de/c134 0 2024-03-29T08:07:30.440 INFO:tasks.workunit.client.1.smithi148.stdout:9/929: mknod db/d22/d26/d54/d4d/d35/d4b/dc3/d116/c132 0 2024-03-29T08:07:30.441 INFO:tasks.workunit.client.1.smithi148.stdout:4/934: chown df/d60/f110 35 1 2024-03-29T08:07:30.442 INFO:tasks.workunit.client.1.smithi148.stdout:7/867: mkdir d8/df4/d130 0 2024-03-29T08:07:30.443 INFO:tasks.workunit.client.1.smithi148.stdout:1/857: dwrite d3/de/d1d/f33 [0,4194304] 0 2024-03-29T08:07:30.447 INFO:tasks.workunit.client.0.smithi037.stdout:9/837: symlink d1/d4/d19/d1e/d4a/d5c/dca/l11f 0 2024-03-29T08:07:30.448 INFO:tasks.workunit.client.0.smithi037.stdout:2/471: write d0/f40 [240677,42455] 0 2024-03-29T08:07:30.451 INFO:tasks.workunit.client.0.smithi037.stdout:7/623: dwrite dd/d21/f75 [0,4194304] 0 2024-03-29T08:07:30.453 INFO:tasks.workunit.client.1.smithi148.stdout:7/868: mkdir d8/df4/d131 0 2024-03-29T08:07:30.453 INFO:tasks.workunit.client.1.smithi148.stdout:8/877: dwrite d17/d18/d26/d63/d7d/faa [0,4194304] 0 2024-03-29T08:07:30.456 INFO:tasks.workunit.client.0.smithi037.stdout:9/838: write d1/d4/d19/d1e/d4a/d6f/da2/f100 [1342667,81133] 0 2024-03-29T08:07:30.456 INFO:tasks.workunit.client.0.smithi037.stdout:5/725: link d8/d22/le5 d8/d22/d37/dbc/lfd 0 2024-03-29T08:07:30.458 INFO:tasks.workunit.client.0.smithi037.stdout:7/624: fsync dd/d72/f98 0 2024-03-29T08:07:30.460 INFO:tasks.workunit.client.1.smithi148.stdout:7/869: chown d8/d38/l105 0 1 2024-03-29T08:07:30.460 INFO:tasks.workunit.client.1.smithi148.stdout:0/887: dwrite db/de/fc3 [0,4194304] 0 2024-03-29T08:07:30.460 INFO:tasks.workunit.client.0.smithi037.stdout:2/472: truncate d0/d20/d22/f48 4730282 0 2024-03-29T08:07:30.465 INFO:tasks.workunit.client.1.smithi148.stdout:1/858: creat d3/de/d1d/d47/d6d/d72/f114 x:0 0 0 2024-03-29T08:07:30.465 INFO:tasks.workunit.client.1.smithi148.stdout:9/930: creat db/d22/d26/d54/d4d/d35/d4b/d58/f133 x:0 0 0 2024-03-29T08:07:30.466 INFO:tasks.workunit.client.0.smithi037.stdout:7/625: symlink dd/d21/d35/d60/ld3 0 2024-03-29T08:07:30.468 INFO:tasks.workunit.client.0.smithi037.stdout:2/473: mkdir d0/d20/d2e/d5f/da5 0 2024-03-29T08:07:30.468 INFO:tasks.workunit.client.1.smithi148.stdout:0/888: dread - db/de/df/d11/d2d/fb2 zero size 2024-03-29T08:07:30.469 INFO:tasks.workunit.client.0.smithi037.stdout:7/626: fsync dd/d72/fac 0 2024-03-29T08:07:30.474 INFO:tasks.workunit.client.1.smithi148.stdout:7/870: mknod d8/dfb/c132 0 2024-03-29T08:07:30.475 INFO:tasks.workunit.client.1.smithi148.stdout:5/881: rename d0/db/d1e/d36/de6 to d0/db/d1e/d35/d42/d67/d76/d81/d98/dd5/dd1/d121 0 2024-03-29T08:07:30.479 INFO:tasks.workunit.client.1.smithi148.stdout:2/976: creat d7/da/d33/d3c/d31/d123/f13e x:0 0 0 2024-03-29T08:07:30.481 INFO:tasks.workunit.client.1.smithi148.stdout:1/859: mknod d3/d55/dca/c115 0 2024-03-29T08:07:30.482 INFO:tasks.workunit.client.0.smithi037.stdout:2/474: symlink d0/d20/d2e/d9c/la6 0 2024-03-29T08:07:30.483 INFO:tasks.workunit.client.0.smithi037.stdout:7/627: truncate dd/d1d/f2a 842297 0 2024-03-29T08:07:30.484 INFO:tasks.workunit.client.0.smithi037.stdout:9/839: dwrite d1/d4/d19/d1e/d4a/d6f/da2/f7a [0,4194304] 0 2024-03-29T08:07:30.484 INFO:tasks.workunit.client.0.smithi037.stdout:5/726: read d8/d22/d37/dbc/d85/f9e [98320,98543] 0 2024-03-29T08:07:30.487 INFO:tasks.workunit.client.1.smithi148.stdout:8/878: dread d17/d49/d9a/dbe/d6f/d64/df1/f106 [0,4194304] 0 2024-03-29T08:07:30.487 INFO:tasks.workunit.client.0.smithi037.stdout:5/727: chown d8/d34/l77 230 1 2024-03-29T08:07:30.489 INFO:tasks.workunit.client.1.smithi148.stdout:2/977: dread - d7/da/d33/d3c/f116 zero size 2024-03-29T08:07:30.491 INFO:tasks.workunit.client.0.smithi037.stdout:5/728: chown d8/d22/d37/dbc/ff3 531051559 1 2024-03-29T08:07:30.491 INFO:tasks.workunit.client.1.smithi148.stdout:0/889: creat db/dc/daa/d47/d4c/d5a/d7a/f10d x:0 0 0 2024-03-29T08:07:30.495 INFO:tasks.workunit.client.0.smithi037.stdout:2/475: read d0/d20/f49 [506156,58176] 0 2024-03-29T08:07:30.496 INFO:tasks.workunit.client.1.smithi148.stdout:4/935: rename df/d2c/d51/d77/d26/cbf to df/d89/c128 0 2024-03-29T08:07:30.497 INFO:tasks.workunit.client.1.smithi148.stdout:1/860: creat d3/d55/d7e/df0/f116 x:0 0 0 2024-03-29T08:07:30.502 INFO:tasks.workunit.client.0.smithi037.stdout:5/729: creat d8/d34/ffe x:0 0 0 2024-03-29T08:07:30.502 INFO:tasks.workunit.client.0.smithi037.stdout:2/476: chown d0/d53/f88 16343567 1 2024-03-29T08:07:30.503 INFO:tasks.workunit.client.1.smithi148.stdout:0/890: read db/de/df/d11/d42/f52 [3070565,94962] 0 2024-03-29T08:07:30.503 INFO:tasks.workunit.client.1.smithi148.stdout:3/931: dwrite de/d24/d33/d39/d46/d70/fe2 [0,4194304] 0 2024-03-29T08:07:30.503 INFO:tasks.workunit.client.1.smithi148.stdout:8/879: truncate d17/d18/fa9 671027 0 2024-03-29T08:07:30.506 INFO:tasks.workunit.client.1.smithi148.stdout:4/936: dread - df/d2c/d51/d77/def/f10f zero size 2024-03-29T08:07:30.507 INFO:tasks.workunit.client.1.smithi148.stdout:5/882: link d0/db/d7b/d16/d87/d117/l113 d0/db/d1e/d6e/d119/l122 0 2024-03-29T08:07:30.510 INFO:tasks.workunit.client.1.smithi148.stdout:1/861: creat d3/de/d1d/f117 x:0 0 0 2024-03-29T08:07:30.514 INFO:tasks.workunit.client.1.smithi148.stdout:9/931: dwrite db/d22/d26/d54/fae [0,4194304] 0 2024-03-29T08:07:30.515 INFO:tasks.workunit.client.0.smithi037.stdout:7/628: dwrite dd/d21/d35/d3b/fbd [0,4194304] 0 2024-03-29T08:07:30.516 INFO:tasks.workunit.client.0.smithi037.stdout:9/840: write d1/d4/d19/d1e/d25/fcb [3288057,53736] 0 2024-03-29T08:07:30.516 INFO:tasks.workunit.client.1.smithi148.stdout:5/883: readlink d0/db/d1e/d5b/d66/dc7/l11c 0 2024-03-29T08:07:30.516 INFO:tasks.workunit.client.1.smithi148.stdout:4/937: unlink df/d71/f88 0 2024-03-29T08:07:30.520 INFO:tasks.workunit.client.1.smithi148.stdout:4/938: write df/f107 [73192,39475] 0 2024-03-29T08:07:30.520 INFO:tasks.workunit.client.0.smithi037.stdout:9/841: chown d1/d4/d19/d69/ld1 22655771 1 2024-03-29T08:07:30.521 INFO:tasks.workunit.client.0.smithi037.stdout:5/730: rename d8/d22/d37/d8d/db3/dcb/deb/c9a to d8/d22/d37/d8d/db3/dcb/cff 0 2024-03-29T08:07:30.521 INFO:tasks.workunit.client.1.smithi148.stdout:9/932: chown db/d10/d2d/f7b 23848228 1 2024-03-29T08:07:30.523 INFO:tasks.workunit.client.1.smithi148.stdout:3/932: mkdir de/d24/d33/d39/d46/d30/d37/d79/d10e/d135 0 2024-03-29T08:07:30.523 INFO:tasks.workunit.client.0.smithi037.stdout:2/477: getdents d0/d20/d22/d4e 0 2024-03-29T08:07:30.525 INFO:tasks.workunit.client.0.smithi037.stdout:9/842: chown d1/d4/d19/d1e/d4a/d6f/da2/d68/lab 4897485 1 2024-03-29T08:07:30.526 INFO:tasks.workunit.client.1.smithi148.stdout:2/978: dwrite d7/da/d52/da3/f13c [0,4194304] 0 2024-03-29T08:07:30.529 INFO:tasks.workunit.client.1.smithi148.stdout:2/979: readlink d7/da/d33/d3c/l80 0 2024-03-29T08:07:30.531 INFO:tasks.workunit.client.1.smithi148.stdout:3/933: creat de/d24/d33/d39/d46/d70/def/f136 x:0 0 0 2024-03-29T08:07:30.534 INFO:tasks.workunit.client.0.smithi037.stdout:2/478: truncate d0/d20/d2e/d5f/d6f/d5e/f60 953223 0 2024-03-29T08:07:30.536 INFO:tasks.workunit.client.0.smithi037.stdout:2/479: dread - d0/d20/d22/fa0 zero size 2024-03-29T08:07:30.537 INFO:tasks.workunit.client.1.smithi148.stdout:2/980: write d7/da/d33/d3c/f116 [102558,14911] 0 2024-03-29T08:07:30.537 INFO:tasks.workunit.client.1.smithi148.stdout:1/862: dread d3/d55/d7e/fb5 [0,4194304] 0 2024-03-29T08:07:30.537 INFO:tasks.workunit.client.1.smithi148.stdout:3/934: creat de/f137 x:0 0 0 2024-03-29T08:07:30.540 INFO:tasks.workunit.client.1.smithi148.stdout:1/863: write d3/de/d1d/f33 [2618420,30084] 0 2024-03-29T08:07:30.542 INFO:tasks.workunit.client.0.smithi037.stdout:9/843: dread d1/d3/f21 [0,4194304] 0 2024-03-29T08:07:30.542 INFO:tasks.workunit.client.1.smithi148.stdout:1/864: chown d3 50 1 2024-03-29T08:07:30.544 INFO:tasks.workunit.client.0.smithi037.stdout:2/480: creat d0/d20/d2e/fa7 x:0 0 0 2024-03-29T08:07:30.545 INFO:tasks.workunit.client.1.smithi148.stdout:8/880: write d17/d1d/f92 [193754,129657] 0 2024-03-29T08:07:30.545 INFO:tasks.workunit.client.1.smithi148.stdout:2/981: dread - d7/da/d33/ded/d2e/fbc zero size 2024-03-29T08:07:30.549 INFO:tasks.workunit.client.0.smithi037.stdout:2/481: dread - d0/d20/d2e/fa7 zero size 2024-03-29T08:07:30.550 INFO:tasks.workunit.client.1.smithi148.stdout:0/891: dwrite db/de/df/fa6 [0,4194304] 0 2024-03-29T08:07:30.555 INFO:tasks.workunit.client.1.smithi148.stdout:1/865: creat d3/d18/dae/dda/f118 x:0 0 0 2024-03-29T08:07:30.555 INFO:tasks.workunit.client.0.smithi037.stdout:2/482: rmdir d0/d20/d22/d4e 39 2024-03-29T08:07:30.557 INFO:tasks.workunit.client.0.smithi037.stdout:9/844: truncate d1/d4/d19/d1e/d4a/d6f/f91 1689152 0 2024-03-29T08:07:30.560 INFO:tasks.workunit.client.0.smithi037.stdout:5/731: dwrite d8/d34/fcf [0,4194304] 0 2024-03-29T08:07:30.564 INFO:tasks.workunit.client.0.smithi037.stdout:5/732: stat d8/d22/d37/d8d/db3/dee 0 2024-03-29T08:07:30.564 INFO:tasks.workunit.client.1.smithi148.stdout:2/982: unlink d7/da/d33/d3c/d54/dec/fb4 0 2024-03-29T08:07:30.565 INFO:tasks.workunit.client.1.smithi148.stdout:5/884: dwrite d0/db/d1e/d35/d42/d67/d76/d81/fff [0,4194304] 0 2024-03-29T08:07:30.567 INFO:tasks.workunit.client.0.smithi037.stdout:9/845: write d1/d4/d19/d1e/d4a/d6f/da2/d13/d7c/f10f [811006,67596] 0 2024-03-29T08:07:30.569 INFO:tasks.workunit.client.0.smithi037.stdout:7/629: dread dd/d1d/d34/f39 [0,4194304] 0 2024-03-29T08:07:30.569 INFO:tasks.workunit.client.0.smithi037.stdout:2/483: fdatasync d0/d20/d22/f47 0 2024-03-29T08:07:30.572 INFO:tasks.workunit.client.0.smithi037.stdout:2/484: dread - d0/d53/f7a zero size 2024-03-29T08:07:30.572 INFO:tasks.workunit.client.1.smithi148.stdout:5/885: chown d0/db/d7b/d16/d2e/fa0 14370 1 2024-03-29T08:07:30.573 INFO:tasks.workunit.client.1.smithi148.stdout:1/866: symlink d3/de/d25/l119 0 2024-03-29T08:07:30.573 INFO:tasks.workunit.client.1.smithi148.stdout:3/935: dwrite de/f137 [0,4194304] 0 2024-03-29T08:07:30.578 INFO:tasks.workunit.client.0.smithi037.stdout:7/630: unlink dd/d21/d35/f4c 0 2024-03-29T08:07:30.579 INFO:tasks.workunit.client.0.smithi037.stdout:2/485: mknod d0/d23/d44/ca8 0 2024-03-29T08:07:30.583 INFO:tasks.workunit.client.0.smithi037.stdout:5/733: getdents d8/d22/dde 0 2024-03-29T08:07:30.586 INFO:tasks.workunit.client.1.smithi148.stdout:1/867: symlink d3/d18/d4d/d9e/db8/l11a 0 2024-03-29T08:07:30.588 INFO:tasks.workunit.client.0.smithi037.stdout:5/734: symlink d8/d34/l100 0 2024-03-29T08:07:30.589 INFO:tasks.workunit.client.1.smithi148.stdout:9/933: dread db/d10/d2d/f68 [0,4194304] 0 2024-03-29T08:07:30.595 INFO:tasks.workunit.client.0.smithi037.stdout:7/631: link dd/d72/la3 dd/d1d/d34/d54/d9c/ld4 0 2024-03-29T08:07:30.597 INFO:tasks.workunit.client.1.smithi148.stdout:0/892: write db/dc/daa/d47/d9f/fa5 [123750,52628] 0 2024-03-29T08:07:30.601 INFO:tasks.workunit.client.0.smithi037.stdout:5/735: truncate d8/d22/d37/dbc/fc8 641542 0 2024-03-29T08:07:30.602 INFO:tasks.workunit.client.1.smithi148.stdout:1/868: mkdir d3/d18/d71/d11b 0 2024-03-29T08:07:30.603 INFO:tasks.workunit.client.0.smithi037.stdout:5/736: fsync d8/d18/d24/fa6 0 2024-03-29T08:07:30.604 INFO:tasks.workunit.client.1.smithi148.stdout:0/893: symlink db/de/d8e/da9/l10e 0 2024-03-29T08:07:30.606 INFO:tasks.workunit.client.1.smithi148.stdout:0/894: fdatasync db/dc/daa/d47/d9f/f10b 0 2024-03-29T08:07:30.609 INFO:tasks.workunit.client.0.smithi037.stdout:7/632: mknod dd/d1d/d49/d81/d84/dca/cd5 0 2024-03-29T08:07:30.611 INFO:tasks.workunit.client.1.smithi148.stdout:2/983: write f5 [596546,49293] 0 2024-03-29T08:07:30.611 INFO:tasks.workunit.client.0.smithi037.stdout:5/737: readlink d8/d34/d4b/lbd 0 2024-03-29T08:07:30.612 INFO:tasks.workunit.client.1.smithi148.stdout:1/869: creat d3/de/d1d/d47/de4/f11c x:0 0 0 2024-03-29T08:07:30.613 INFO:tasks.workunit.client.1.smithi148.stdout:6/911: sync 2024-03-29T08:07:30.616 INFO:tasks.workunit.client.0.smithi037.stdout:2/486: dwrite d0/d20/f3a [0,4194304] 0 2024-03-29T08:07:30.617 INFO:tasks.workunit.client.1.smithi148.stdout:2/984: write d7/da/d33/d3c/d31/d123/f11f [274464,70334] 0 2024-03-29T08:07:30.619 INFO:tasks.workunit.client.0.smithi037.stdout:5/738: creat d8/db9/f101 x:0 0 0 2024-03-29T08:07:30.622 INFO:tasks.workunit.client.0.smithi037.stdout:9/846: dread d1/d1f/d3a/f46 [0,4194304] 0 2024-03-29T08:07:30.622 INFO:tasks.workunit.client.1.smithi148.stdout:5/886: truncate d0/db/d1e/d5b/fda 2795202 0 2024-03-29T08:07:30.623 INFO:tasks.workunit.client.1.smithi148.stdout:3/936: write de/d24/d33/d39/d46/d30/d37/d62/d63/dbf/ff0 [1399370,94127] 0 2024-03-29T08:07:30.623 INFO:tasks.workunit.client.0.smithi037.stdout:5/739: creat d8/d22/d4f/d70/d9c/f102 x:0 0 0 2024-03-29T08:07:30.627 INFO:tasks.workunit.client.0.smithi037.stdout:5/740: fsync d8/d22/d4f/d70/f54 0 2024-03-29T08:07:30.628 INFO:tasks.workunit.client.1.smithi148.stdout:4/939: dread df/d2c/d51/d77/d26/f3d [0,4194304] 0 2024-03-29T08:07:30.628 INFO:tasks.workunit.client.0.smithi037.stdout:9/847: creat d1/d4/d19/d1e/d71/d116/f120 x:0 0 0 2024-03-29T08:07:30.630 INFO:tasks.workunit.client.0.smithi037.stdout:7/633: truncate dd/d1d/d34/d54/f87 1654322 0 2024-03-29T08:07:30.630 INFO:tasks.workunit.client.0.smithi037.stdout:5/741: mkdir d8/d22/db6/d103 0 2024-03-29T08:07:30.632 INFO:tasks.workunit.client.0.smithi037.stdout:7/634: chown dd/d21/d35/d60/d94/fa8 2705 1 2024-03-29T08:07:30.635 INFO:tasks.workunit.client.0.smithi037.stdout:5/742: unlink d8/d22/d4f/f68 0 2024-03-29T08:07:30.636 INFO:tasks.workunit.client.1.smithi148.stdout:6/912: mkdir d6/d9/d1e/d36/db1/d122 0 2024-03-29T08:07:30.636 INFO:tasks.workunit.client.1.smithi148.stdout:3/937: fsync de/d24/d33/d39/d46/d30/d37/d62/d63/fcb 0 2024-03-29T08:07:30.637 INFO:tasks.workunit.client.0.smithi037.stdout:7/635: truncate dd/ff 1324993 0 2024-03-29T08:07:30.641 INFO:tasks.workunit.client.1.smithi148.stdout:3/938: truncate de/f126 37293 0 2024-03-29T08:07:30.641 INFO:tasks.workunit.client.1.smithi148.stdout:6/913: write d6/d9/fdc [4265642,47750] 0 2024-03-29T08:07:30.641 INFO:tasks.workunit.client.1.smithi148.stdout:5/887: symlink d0/db/d1e/d5b/d66/l123 0 2024-03-29T08:07:30.645 INFO:tasks.workunit.client.1.smithi148.stdout:0/895: dwrite db/dc/daa/d47/d4c/d5a/d7a/f10d [0,4194304] 0 2024-03-29T08:07:30.647 INFO:tasks.workunit.client.1.smithi148.stdout:9/934: dread db/d22/d26/f38 [0,4194304] 0 2024-03-29T08:07:30.648 INFO:tasks.workunit.client.0.smithi037.stdout:9/848: truncate d1/d4/d19/d1e/d4a/d6f/da2/f3d 1042495 0 2024-03-29T08:07:30.649 INFO:tasks.workunit.client.1.smithi148.stdout:5/888: dread - d0/db/d1e/d6e/da8/f100 zero size 2024-03-29T08:07:30.649 INFO:tasks.workunit.client.0.smithi037.stdout:7/636: fsync dd/f14 0 2024-03-29T08:07:30.651 INFO:tasks.workunit.client.0.smithi037.stdout:7/637: chown dd/d1d/d34/f37 4675783 1 2024-03-29T08:07:30.652 INFO:tasks.workunit.client.1.smithi148.stdout:5/889: readlink d0/db/d7b/l18 0 2024-03-29T08:07:30.653 INFO:tasks.workunit.client.1.smithi148.stdout:1/870: rename d3/de/d1d to d3/d18/d4d/d76/d85/d11d 0 2024-03-29T08:07:30.655 INFO:tasks.workunit.client.0.smithi037.stdout:7/638: dread - dd/d72/fae zero size 2024-03-29T08:07:30.655 INFO:tasks.workunit.client.0.smithi037.stdout:7/639: stat dd/d21/d35/d60/fbf 0 2024-03-29T08:07:30.655 INFO:tasks.workunit.client.1.smithi148.stdout:1/871: stat d3/d55/ddf/lf4 0 2024-03-29T08:07:30.657 INFO:tasks.workunit.client.1.smithi148.stdout:0/896: mknod db/de/df/d11/c10f 0 2024-03-29T08:07:30.659 INFO:tasks.workunit.client.1.smithi148.stdout:9/935: fsync db/d22/d26/f10e 0 2024-03-29T08:07:30.660 INFO:tasks.workunit.client.1.smithi148.stdout:6/914: mkdir d6/d9/d1e/d20/d123 0 2024-03-29T08:07:30.661 INFO:tasks.workunit.client.1.smithi148.stdout:4/940: rename df/f1a to df/d2c/d51/d77/d26/d34/d3e/dd8/f129 0 2024-03-29T08:07:30.662 INFO:tasks.workunit.client.1.smithi148.stdout:9/936: chown db/d10/cdb 8 1 2024-03-29T08:07:30.663 INFO:tasks.workunit.client.0.smithi037.stdout:7/640: getdents dd/d1d/d34/dc8 0 2024-03-29T08:07:30.663 INFO:tasks.workunit.client.1.smithi148.stdout:2/985: truncate d7/da/dcc/d11c/fa9 4006319 0 2024-03-29T08:07:30.663 INFO:tasks.workunit.client.0.smithi037.stdout:2/487: fsync d0/f8c 0 2024-03-29T08:07:30.665 INFO:tasks.workunit.client.0.smithi037.stdout:7/641: symlink dd/d1d/ld6 0 2024-03-29T08:07:30.667 INFO:tasks.workunit.client.1.smithi148.stdout:9/937: fdatasync db/d22/fd7 0 2024-03-29T08:07:30.669 INFO:tasks.workunit.client.0.smithi037.stdout:7/642: creat dd/dc3/fd7 x:0 0 0 2024-03-29T08:07:30.669 INFO:tasks.workunit.client.1.smithi148.stdout:0/897: mkdir db/dc/daa/d47/d4c/d5a/d7a/db1/d110 0 2024-03-29T08:07:30.671 INFO:tasks.workunit.client.0.smithi037.stdout:2/488: getdents d0/d20/d2e 0 2024-03-29T08:07:30.671 INFO:tasks.workunit.client.1.smithi148.stdout:6/915: read d6/d9/d1e/d36/d4d/d9d/d28/fea [586432,41041] 0 2024-03-29T08:07:30.671 INFO:tasks.workunit.client.1.smithi148.stdout:3/939: getdents de/d24/d33/d39/d46/d8d 0 2024-03-29T08:07:30.672 INFO:tasks.workunit.client.0.smithi037.stdout:7/643: dread dd/d72/fac [0,4194304] 0 2024-03-29T08:07:30.672 INFO:tasks.workunit.client.1.smithi148.stdout:9/938: chown db/d10/l101 403 1 2024-03-29T08:07:30.673 INFO:tasks.workunit.client.0.smithi037.stdout:2/489: readlink d0/d20/d2e/l5d 0 2024-03-29T08:07:30.674 INFO:tasks.workunit.client.0.smithi037.stdout:5/743: write d8/d22/f31 [1847243,119727] 0 2024-03-29T08:07:30.674 INFO:tasks.workunit.client.1.smithi148.stdout:1/872: write d3/d18/d4d/d9e/f40 [218636,80023] 0 2024-03-29T08:07:30.676 INFO:tasks.workunit.client.1.smithi148.stdout:2/986: mkdir d7/d13f 0 2024-03-29T08:07:30.676 INFO:tasks.workunit.client.0.smithi037.stdout:7/644: mkdir dd/d1d/d49/d81/dd8 0 2024-03-29T08:07:30.678 INFO:tasks.workunit.client.1.smithi148.stdout:4/941: symlink df/d2c/d51/d77/d26/d34/d3e/d116/l12a 0 2024-03-29T08:07:30.681 INFO:tasks.workunit.client.1.smithi148.stdout:1/873: chown d3/de/d6e 57597191 1 2024-03-29T08:07:30.683 INFO:tasks.workunit.client.1.smithi148.stdout:5/890: dwrite d0/db/d1e/d36/d41/f7f [0,4194304] 0 2024-03-29T08:07:30.686 INFO:tasks.workunit.client.1.smithi148.stdout:1/874: write d3/d55/f37 [1829812,128170] 0 2024-03-29T08:07:30.687 INFO:tasks.workunit.client.0.smithi037.stdout:7/645: symlink dd/d21/d70/ld9 0 2024-03-29T08:07:30.687 INFO:tasks.workunit.client.1.smithi148.stdout:0/898: write db/dc/daa/fe0 [784797,73007] 0 2024-03-29T08:07:30.688 INFO:tasks.workunit.client.1.smithi148.stdout:4/942: unlink df/fdb 0 2024-03-29T08:07:30.692 INFO:tasks.workunit.client.0.smithi037.stdout:7/646: mknod dd/d21/d70/cda 0 2024-03-29T08:07:30.693 INFO:tasks.workunit.client.0.smithi037.stdout:9/849: dwrite d1/d4/d19/d69/fce [0,4194304] 0 2024-03-29T08:07:30.693 INFO:tasks.workunit.client.1.smithi148.stdout:9/939: rename db/l16 to db/d22/d26/d54/d52/d9c/d12b/l134 0 2024-03-29T08:07:30.697 INFO:tasks.workunit.client.0.smithi037.stdout:7/647: mkdir dd/d21/d35/d42/ddb 0 2024-03-29T08:07:30.699 INFO:tasks.workunit.client.0.smithi037.stdout:9/850: mknod d1/d1f/d3a/d9b/c121 0 2024-03-29T08:07:30.703 INFO:tasks.workunit.client.1.smithi148.stdout:4/943: write df/d2c/d51/d77/d17/fbd [4216483,43403] 0 2024-03-29T08:07:30.707 INFO:tasks.workunit.client.1.smithi148.stdout:1/875: symlink d3/d18/d4d/d9e/db8/l11e 0 2024-03-29T08:07:30.708 INFO:tasks.workunit.client.1.smithi148.stdout:5/891: rmdir d0/db/d1e/d35/d42/d10b 0 2024-03-29T08:07:30.710 INFO:tasks.workunit.client.0.smithi037.stdout:5/744: dwrite d8/d34/f4a [4194304,4194304] 0 2024-03-29T08:07:30.717 INFO:tasks.workunit.client.1.smithi148.stdout:0/899: rename db/de/df/d11/d42/f67 to db/dc/daa/f111 0 2024-03-29T08:07:30.717 INFO:tasks.workunit.client.1.smithi148.stdout:2/987: dwrite d7/da/d33/ded/d2e/d37/ffd [0,4194304] 0 2024-03-29T08:07:30.717 INFO:tasks.workunit.client.1.smithi148.stdout:4/944: symlink df/d2c/d51/d77/l12b 0 2024-03-29T08:07:30.718 INFO:tasks.workunit.client.0.smithi037.stdout:5/745: getdents d8/d4d/ddf 0 2024-03-29T08:07:30.721 INFO:tasks.workunit.client.0.smithi037.stdout:2/490: dread d0/f40 [0,4194304] 0 2024-03-29T08:07:30.725 INFO:tasks.workunit.client.1.smithi148.stdout:3/940: dread de/d24/d33/d39/d46/d30/d37/f6f [0,4194304] 0 2024-03-29T08:07:30.725 INFO:tasks.workunit.client.1.smithi148.stdout:9/940: rename db/d22/d26/d54/d4d/d35/d4b/d58/f8b to db/d10/d6d/d73/da8/deb/f135 0 2024-03-29T08:07:30.726 INFO:tasks.workunit.client.0.smithi037.stdout:5/746: rename d8/d22/f26 to d8/d22/d37/dbc/d85/df6/f104 0 2024-03-29T08:07:30.727 INFO:tasks.workunit.client.0.smithi037.stdout:2/491: write d0/d23/f30 [3479293,47449] 0 2024-03-29T08:07:30.729 INFO:tasks.workunit.client.1.smithi148.stdout:1/876: mknod d3/d18/d4d/d9e/deb/c11f 0 2024-03-29T08:07:30.730 INFO:tasks.workunit.client.0.smithi037.stdout:2/492: stat d0/d20/d22/l64 0 2024-03-29T08:07:30.732 INFO:tasks.workunit.client.1.smithi148.stdout:2/988: read d7/da/d33/ded/d2e/f97 [60164,52682] 0 2024-03-29T08:07:30.732 INFO:tasks.workunit.client.1.smithi148.stdout:9/941: fsync db/d10/d2d/d4e/d5f/d6e/f119 0 2024-03-29T08:07:30.733 INFO:tasks.workunit.client.1.smithi148.stdout:6/916: truncate d6/d9/d1e/d27/d48/d9f/faa 8814737 0 2024-03-29T08:07:30.733 INFO:tasks.workunit.client.1.smithi148.stdout:4/945: dread df/d2c/d51/fe2 [0,4194304] 0 2024-03-29T08:07:30.734 INFO:tasks.workunit.client.0.smithi037.stdout:9/851: write d1/d4/d19/d1e/d4a/d6f/da2/f12 [1610189,130480] 0 2024-03-29T08:07:30.734 INFO:tasks.workunit.client.1.smithi148.stdout:5/892: write d0/db/d1e/f7e [4421268,16586] 0 2024-03-29T08:07:30.739 INFO:tasks.workunit.client.1.smithi148.stdout:4/946: write df/d2c/f46 [341673,95139] 0 2024-03-29T08:07:30.741 INFO:tasks.workunit.client.1.smithi148.stdout:7/871: sync 2024-03-29T08:07:30.741 INFO:tasks.workunit.client.1.smithi148.stdout:0/900: rename db/de/df/d11/d2d/da3/dee/ff6 to db/de/df/d2b/f112 0 2024-03-29T08:07:30.742 INFO:tasks.workunit.client.1.smithi148.stdout:2/989: mknod d7/da/d52/c140 0 2024-03-29T08:07:30.743 INFO:tasks.workunit.client.1.smithi148.stdout:9/942: chown db/d10/d2d/d4e/d5f/cf4 2 1 2024-03-29T08:07:30.746 INFO:tasks.workunit.client.0.smithi037.stdout:6/528: sync 2024-03-29T08:07:30.746 INFO:tasks.workunit.client.0.smithi037.stdout:2/493: creat d0/d20/d2e/d5f/d6f/fa9 x:0 0 0 2024-03-29T08:07:30.746 INFO:tasks.workunit.client.0.smithi037.stdout:1/641: sync 2024-03-29T08:07:30.748 INFO:tasks.workunit.client.0.smithi037.stdout:7/648: dwrite dd/d68/f78 [0,4194304] 0 2024-03-29T08:07:30.751 INFO:tasks.workunit.client.1.smithi148.stdout:9/943: write db/d10/d6d/d73/da8/deb/dec/dde/f102 [111521,89135] 0 2024-03-29T08:07:30.751 INFO:tasks.workunit.client.0.smithi037.stdout:6/529: chown d0/d5/d3f/f96 8119 1 2024-03-29T08:07:30.752 INFO:tasks.workunit.client.0.smithi037.stdout:2/494: creat d0/d20/d2e/d5f/faa x:0 0 0 2024-03-29T08:07:30.754 INFO:tasks.workunit.client.1.smithi148.stdout:6/917: getdents d6/d9 0 2024-03-29T08:07:30.755 INFO:tasks.workunit.client.0.smithi037.stdout:1/642: creat d2/d6/dc/d7e/dd2/fde x:0 0 0 2024-03-29T08:07:30.757 INFO:tasks.workunit.client.0.smithi037.stdout:2/495: chown d0/d20/c4d 1655337 1 2024-03-29T08:07:30.759 INFO:tasks.workunit.client.1.smithi148.stdout:6/918: chown d6/d9/d1e/f3d 1508 1 2024-03-29T08:07:30.759 INFO:tasks.workunit.client.0.smithi037.stdout:6/530: write d0/d5/d1f/f63 [4377157,31695] 0 2024-03-29T08:07:30.761 INFO:tasks.workunit.client.0.smithi037.stdout:7/649: mknod dd/d21/d35/dcf/cdc 0 2024-03-29T08:07:30.762 INFO:tasks.workunit.client.1.smithi148.stdout:2/990: getdents d7/da/d33/d3c/d31/d123 0 2024-03-29T08:07:30.763 INFO:tasks.workunit.client.0.smithi037.stdout:5/747: write d8/d22/d37/dbc/d85/df6/f104 [964479,23680] 0 2024-03-29T08:07:30.763 INFO:tasks.workunit.client.0.smithi037.stdout:6/531: stat d0/d5/f73 0 2024-03-29T08:07:30.764 INFO:tasks.workunit.client.1.smithi148.stdout:9/944: creat db/d22/d131/f136 x:0 0 0 2024-03-29T08:07:30.764 INFO:tasks.workunit.client.0.smithi037.stdout:1/643: creat d2/d6/dc/d18/d1d/dad/daf/fdf x:0 0 0 2024-03-29T08:07:30.766 INFO:tasks.workunit.client.0.smithi037.stdout:2/496: symlink d0/d20/d22/d4e/lab 0 2024-03-29T08:07:30.768 INFO:tasks.workunit.client.1.smithi148.stdout:5/893: write d0/db/d84/f10f [338732,48200] 0 2024-03-29T08:07:30.769 INFO:tasks.workunit.client.0.smithi037.stdout:2/497: dread - d0/f8a zero size 2024-03-29T08:07:30.770 INFO:tasks.workunit.client.0.smithi037.stdout:1/644: fdatasync d2/d6/dc/d18/d1d/dad/daf/fd4 0 2024-03-29T08:07:30.772 INFO:tasks.workunit.client.1.smithi148.stdout:1/877: truncate d3/d18/d71/f8e 3270861 0 2024-03-29T08:07:30.773 INFO:tasks.workunit.client.0.smithi037.stdout:5/748: creat d8/d22/d37/d8d/db3/dcb/f105 x:0 0 0 2024-03-29T08:07:30.774 INFO:tasks.workunit.client.1.smithi148.stdout:6/919: chown d6/d9/d1e/d36/d4d/d9d/d28/dda/db6/df3/dd3/f11f 9079484 1 2024-03-29T08:07:30.775 INFO:tasks.workunit.client.1.smithi148.stdout:9/945: fdatasync db/d10/d2d/d4e/d5f/dd5/f111 0 2024-03-29T08:07:30.777 INFO:tasks.workunit.client.0.smithi037.stdout:6/532: symlink d0/d6a/d7b/d77/lb9 0 2024-03-29T08:07:30.780 INFO:tasks.workunit.client.1.smithi148.stdout:4/947: dwrite df/d2c/d51/d77/d17/fbd [4194304,4194304] 0 2024-03-29T08:07:30.782 INFO:tasks.workunit.client.1.smithi148.stdout:8/881: sync 2024-03-29T08:07:30.783 INFO:tasks.workunit.client.1.smithi148.stdout:1/878: write d3/d18/d4d/d8f/fcf [467327,15380] 0 2024-03-29T08:07:30.783 INFO:tasks.workunit.client.0.smithi037.stdout:5/749: chown d8/d4d/c6d 218255 1 2024-03-29T08:07:30.784 INFO:tasks.workunit.client.1.smithi148.stdout:5/894: unlink d0/db/d1e/d6e/da8/lac 0 2024-03-29T08:07:30.786 INFO:tasks.workunit.client.0.smithi037.stdout:6/533: symlink d0/d19/d42/lba 0 2024-03-29T08:07:30.786 INFO:tasks.workunit.client.1.smithi148.stdout:7/872: dwrite d8/d14/d45/d4d/d78/fb5 [0,4194304] 0 2024-03-29T08:07:30.787 INFO:tasks.workunit.client.0.smithi037.stdout:5/750: dread - d8/d22/d4f/d70/d79/fd7 zero size 2024-03-29T08:07:30.792 INFO:tasks.workunit.client.1.smithi148.stdout:7/873: dread - d8/d38/ff0 zero size 2024-03-29T08:07:30.794 INFO:tasks.workunit.client.1.smithi148.stdout:9/946: truncate db/d10/d9d/f110 227109 0 2024-03-29T08:07:30.794 INFO:tasks.workunit.client.1.smithi148.stdout:3/941: dread de/d24/d33/d39/d46/d30/d37/d62/d63/dbf/fc8 [4194304,4194304] 0 2024-03-29T08:07:30.796 INFO:tasks.workunit.client.0.smithi037.stdout:5/751: unlink d8/d34/l3c 0 2024-03-29T08:07:30.797 INFO:tasks.workunit.client.1.smithi148.stdout:2/991: dread d7/da/d33/d3c/d54/d70/f85 [0,4194304] 0 2024-03-29T08:07:30.797 INFO:tasks.workunit.client.0.smithi037.stdout:9/852: dread d1/d4/d19/d1e/d25/fcb [0,4194304] 0 2024-03-29T08:07:30.799 INFO:tasks.workunit.client.1.smithi148.stdout:5/895: fsync d0/db/d1e/d79/ff3 0 2024-03-29T08:07:30.800 INFO:tasks.workunit.client.1.smithi148.stdout:0/901: dread db/f7b [0,4194304] 0 2024-03-29T08:07:30.802 INFO:tasks.workunit.client.0.smithi037.stdout:2/498: write d0/d20/d2e/d5f/d6f/d5e/f74 [809529,122645] 0 2024-03-29T08:07:30.803 INFO:tasks.workunit.client.1.smithi148.stdout:4/948: link df/d2c/d51/ff4 df/d2c/d80/daa/dbc/f12c 0 2024-03-29T08:07:30.803 INFO:tasks.workunit.client.1.smithi148.stdout:8/882: truncate d17/d49/d9a/dbe/d6f/fb9 133326 0 2024-03-29T08:07:30.805 INFO:tasks.workunit.client.1.smithi148.stdout:9/947: fdatasync db/d10/d6d/fd2 0 2024-03-29T08:07:30.806 INFO:tasks.workunit.client.0.smithi037.stdout:9/853: truncate d1/d4/d19/d69/fda 1386664 0 2024-03-29T08:07:30.810 INFO:tasks.workunit.client.1.smithi148.stdout:8/883: readlink d17/d18/d26/d63/lb0 0 2024-03-29T08:07:30.811 INFO:tasks.workunit.client.0.smithi037.stdout:6/534: write d0/d5/d1f/f4c [5050614,124010] 0 2024-03-29T08:07:30.811 INFO:tasks.workunit.client.0.smithi037.stdout:5/752: creat d8/d4d/d8a/f106 x:0 0 0 2024-03-29T08:07:30.813 INFO:tasks.workunit.client.1.smithi148.stdout:0/902: write db/de/df/d2b/f93 [1696603,113880] 0 2024-03-29T08:07:30.814 INFO:tasks.workunit.client.1.smithi148.stdout:5/896: unlink d0/db/d7b/d16/d2e/fa0 0 2024-03-29T08:07:30.816 INFO:tasks.workunit.client.0.smithi037.stdout:6/535: mkdir d0/d5/d1e/d44/dbb 0 2024-03-29T08:07:30.820 INFO:tasks.workunit.client.0.smithi037.stdout:6/536: mkdir d0/d5/d1e/d84/dbc 0 2024-03-29T08:07:30.821 INFO:tasks.workunit.client.0.smithi037.stdout:1/645: dwrite d2/d40/f43 [0,4194304] 0 2024-03-29T08:07:30.821 INFO:tasks.workunit.client.1.smithi148.stdout:1/879: rename d3/d18/d71/f8e to d3/de/f120 0 2024-03-29T08:07:30.821 INFO:tasks.workunit.client.1.smithi148.stdout:6/920: dwrite d6/d9/d1e/f3d [0,4194304] 0 2024-03-29T08:07:30.822 INFO:tasks.workunit.client.1.smithi148.stdout:5/897: chown d0/db/d7b/d16/d87/d117/l78 33160692 1 2024-03-29T08:07:30.823 INFO:tasks.workunit.client.1.smithi148.stdout:0/903: chown db/de/d24/dbf/f100 780459 1 2024-03-29T08:07:30.825 INFO:tasks.workunit.client.0.smithi037.stdout:6/537: read d0/d5/fa7 [3065867,89382] 0 2024-03-29T08:07:30.826 INFO:tasks.workunit.client.1.smithi148.stdout:8/884: read d17/d18/d26/d4f/fb7 [75286,15310] 0 2024-03-29T08:07:30.826 INFO:tasks.workunit.client.1.smithi148.stdout:3/942: link de/f17 de/d24/d33/d39/d46/d30/d37/d62/f138 0 2024-03-29T08:07:30.827 INFO:tasks.workunit.client.0.smithi037.stdout:1/646: getdents d2/d6/dc/d18/d1d/d5e/d72 0 2024-03-29T08:07:30.828 INFO:tasks.workunit.client.1.smithi148.stdout:5/898: readlink d0/db/d1e/d6e/da8/l112 0 2024-03-29T08:07:30.831 INFO:tasks.workunit.client.0.smithi037.stdout:6/538: creat d0/d5/d1a/d61/fbd x:0 0 0 2024-03-29T08:07:30.836 INFO:tasks.workunit.client.0.smithi037.stdout:6/539: fdatasync d0/d5/d3f/f17 0 2024-03-29T08:07:30.841 INFO:tasks.workunit.client.0.smithi037.stdout:1/647: symlink d2/d6/dc/d18/dbd/dda/le0 0 2024-03-29T08:07:30.842 INFO:tasks.workunit.client.1.smithi148.stdout:8/885: fdatasync d17/d74/d86/fdd 0 2024-03-29T08:07:30.843 INFO:tasks.workunit.client.0.smithi037.stdout:6/540: mknod d0/d5/d1e/cbe 0 2024-03-29T08:07:30.844 INFO:tasks.workunit.client.0.smithi037.stdout:6/541: readlink d0/l2 0 2024-03-29T08:07:30.844 INFO:tasks.workunit.client.1.smithi148.stdout:0/904: creat db/de/df/d11/d2d/d3e/f113 x:0 0 0 2024-03-29T08:07:30.846 INFO:tasks.workunit.client.0.smithi037.stdout:1/648: chown d2/d6/d13/cc9 43821 1 2024-03-29T08:07:30.846 INFO:tasks.workunit.client.1.smithi148.stdout:3/943: symlink de/d24/d33/d39/l139 0 2024-03-29T08:07:30.847 INFO:tasks.workunit.client.1.smithi148.stdout:1/880: rename d3/de/d25/de8 to d3/d18/d4d/d121 0 2024-03-29T08:07:30.848 INFO:tasks.workunit.client.0.smithi037.stdout:6/542: mkdir d0/d19/d34/d5a/dbf 0 2024-03-29T08:07:30.851 INFO:tasks.workunit.client.1.smithi148.stdout:4/949: write df/f5e [829691,45013] 0 2024-03-29T08:07:30.851 INFO:tasks.workunit.client.1.smithi148.stdout:6/921: write d6/d9/d1e/d36/ff6 [336555,66932] 0 2024-03-29T08:07:30.851 INFO:tasks.workunit.client.0.smithi037.stdout:1/649: creat d2/d6/fe1 x:0 0 0 2024-03-29T08:07:30.852 INFO:tasks.workunit.client.0.smithi037.stdout:1/650: dread - d2/d6/fd7 zero size 2024-03-29T08:07:30.858 INFO:tasks.workunit.client.1.smithi148.stdout:7/874: dread d8/d14/d45/d4d/df7/f110 [0,4194304] 0 2024-03-29T08:07:30.859 INFO:tasks.workunit.client.1.smithi148.stdout:9/948: dwrite db/d10/d2d/d4e/da6/f11a [0,4194304] 0 2024-03-29T08:07:30.860 INFO:tasks.workunit.client.0.smithi037.stdout:1/651: link d2/d4b/l53 d2/d4b/d80/db0/le2 0 2024-03-29T08:07:30.861 INFO:tasks.workunit.client.1.smithi148.stdout:8/886: rmdir d17/dce/d11d 0 2024-03-29T08:07:30.861 INFO:tasks.workunit.client.1.smithi148.stdout:4/950: chown df/d2c/d51/d77/dfe 0 1 2024-03-29T08:07:30.869 INFO:tasks.workunit.client.1.smithi148.stdout:7/875: rmdir d8/d38/d6a 39 2024-03-29T08:07:30.870 INFO:tasks.workunit.client.1.smithi148.stdout:2/992: dread d7/da/d33/d3c/ffb [0,4194304] 0 2024-03-29T08:07:30.870 INFO:tasks.workunit.client.1.smithi148.stdout:0/905: dread db/dc/daa/d47/d9f/fa5 [0,4194304] 0 2024-03-29T08:07:30.870 INFO:tasks.workunit.client.1.smithi148.stdout:6/922: chown d6/d9/d1e/d20/d30/f35 10708 1 2024-03-29T08:07:30.872 INFO:tasks.workunit.client.1.smithi148.stdout:5/899: dwrite d0/db/d7b/d16/f7a [0,4194304] 0 2024-03-29T08:07:30.873 INFO:tasks.workunit.client.1.smithi148.stdout:3/944: symlink de/d24/d33/d39/d46/d30/d37/d62/d63/dbf/l13a 0 2024-03-29T08:07:30.876 INFO:tasks.workunit.client.0.smithi037.stdout:1/652: dwrite d2/d6/dc/d18/d22/fcf [0,4194304] 0 2024-03-29T08:07:30.878 INFO:tasks.workunit.client.1.smithi148.stdout:9/949: creat db/d22/d26/d54/d4d/d35/d4b/d5a/f137 x:0 0 0 2024-03-29T08:07:30.883 INFO:tasks.workunit.client.1.smithi148.stdout:4/951: creat df/d2c/d51/d77/d26/d34/d3e/df3/f12d x:0 0 0 2024-03-29T08:07:30.886 INFO:tasks.workunit.client.1.smithi148.stdout:3/945: write de/d24/d33/d39/d46/d30/d37/d62/f11e [306308,57213] 0 2024-03-29T08:07:30.887 INFO:tasks.workunit.client.1.smithi148.stdout:8/887: truncate d17/d18/d26/d4f/d94/fa2 996476 0 2024-03-29T08:07:30.888 INFO:tasks.workunit.client.1.smithi148.stdout:1/881: dwrite d3/d18/dae/dda/df3/faa [0,4194304] 0 2024-03-29T08:07:30.889 INFO:tasks.workunit.client.1.smithi148.stdout:2/993: symlink d7/da/d33/d45/l141 0 2024-03-29T08:07:30.890 INFO:tasks.workunit.client.1.smithi148.stdout:6/923: mkdir d6/d9/d1e/d36/d124 0 2024-03-29T08:07:30.890 INFO:tasks.workunit.client.0.smithi037.stdout:1/653: creat d2/d6/fe3 x:0 0 0 2024-03-29T08:07:30.891 INFO:tasks.workunit.client.0.smithi037.stdout:6/543: dwrite d0/d20/f31 [0,4194304] 0 2024-03-29T08:07:30.892 INFO:tasks.workunit.client.1.smithi148.stdout:5/900: dread d0/db/d7b/d16/f3d [0,4194304] 0 2024-03-29T08:07:30.896 INFO:tasks.workunit.client.0.smithi037.stdout:1/654: creat d2/d6/dc/fe4 x:0 0 0 2024-03-29T08:07:30.896 INFO:tasks.workunit.client.1.smithi148.stdout:0/906: link db/c7d db/dc/daa/d47/d4c/d5a/d7a/db1/d110/c114 0 2024-03-29T08:07:30.897 INFO:tasks.workunit.client.1.smithi148.stdout:1/882: truncate d3/d18/f39 1304888 0 2024-03-29T08:07:30.900 INFO:tasks.workunit.client.0.smithi037.stdout:1/655: creat d2/d6/d13/fe5 x:0 0 0 2024-03-29T08:07:30.904 INFO:tasks.workunit.client.1.smithi148.stdout:8/888: mknod d17/d18/d26/c121 0 2024-03-29T08:07:30.905 INFO:tasks.workunit.client.0.smithi037.stdout:1/656: creat d2/d6/dc/d18/d1d/d5e/fe6 x:0 0 0 2024-03-29T08:07:30.906 INFO:tasks.workunit.client.1.smithi148.stdout:1/883: chown d3/d18/d4d/d9e/db8/l11a 2806 1 2024-03-29T08:07:30.910 INFO:tasks.workunit.client.1.smithi148.stdout:3/946: truncate de/d24/d33/d39/d46/d30/d37/d62/d63/dbf/fd2 4304945 0 2024-03-29T08:07:30.910 INFO:tasks.workunit.client.1.smithi148.stdout:7/876: dwrite d8/dfb/db8/fee [0,4194304] 0 2024-03-29T08:07:30.917 INFO:tasks.workunit.client.1.smithi148.stdout:8/889: mkdir d17/d18/d26/d63/d66/d5f/dd0/d119/d122 0 2024-03-29T08:07:30.921 INFO:tasks.workunit.client.1.smithi148.stdout:7/877: creat d8/df4/d111/f133 x:0 0 0 2024-03-29T08:07:30.927 INFO:tasks.workunit.client.1.smithi148.stdout:8/890: mknod d17/d18/d26/d4f/dde/c123 0 2024-03-29T08:07:30.929 INFO:tasks.workunit.client.0.smithi037.stdout:6/544: dread d0/d19/d42/d60/f8b [4194304,4194304] 0 2024-03-29T08:07:30.930 INFO:tasks.workunit.client.1.smithi148.stdout:7/878: unlink d8/dfb/d11b/f4b 0 2024-03-29T08:07:30.933 INFO:tasks.workunit.client.0.smithi037.stdout:6/545: symlink d0/lc0 0 2024-03-29T08:07:30.935 INFO:tasks.workunit.client.1.smithi148.stdout:2/994: dwrite d7/da/d33/d3c/d54/f95 [0,4194304] 0 2024-03-29T08:07:30.936 INFO:tasks.workunit.client.0.smithi037.stdout:6/546: rmdir d0/d20 39 2024-03-29T08:07:30.940 INFO:tasks.workunit.client.1.smithi148.stdout:2/995: fdatasync d7/d24/d4b/f55 0 2024-03-29T08:07:30.941 INFO:tasks.workunit.client.0.smithi037.stdout:1/657: write d2/d6/dc/d18/d1d/d21/d99/f8c [727964,7590] 0 2024-03-29T08:07:30.944 INFO:tasks.workunit.client.0.smithi037.stdout:6/547: creat d0/d19/d34/d4e/d94/fc1 x:0 0 0 2024-03-29T08:07:30.947 INFO:tasks.workunit.client.1.smithi148.stdout:1/884: write d3/fa [6289160,104047] 0 2024-03-29T08:07:30.948 INFO:tasks.workunit.client.0.smithi037.stdout:6/548: write d0/d19/d42/d60/d62/f9a [970610,86284] 0 2024-03-29T08:07:30.950 INFO:tasks.workunit.client.0.smithi037.stdout:1/658: rename d2/f84 to d2/d6/dc/d18/d1d/fe7 0 2024-03-29T08:07:30.951 INFO:tasks.workunit.client.0.smithi037.stdout:6/549: chown d0/d6a/f70 1 1 2024-03-29T08:07:30.952 INFO:tasks.workunit.client.1.smithi148.stdout:0/907: write db/de/df/d11/f37 [8139408,24791] 0 2024-03-29T08:07:30.953 INFO:tasks.workunit.client.0.smithi037.stdout:6/550: chown d0/l4a 320 1 2024-03-29T08:07:30.958 INFO:tasks.workunit.client.0.smithi037.stdout:1/659: rename d2/d6/dc/d18/d1d/dad to d2/d40/de8 0 2024-03-29T08:07:30.960 INFO:tasks.workunit.client.1.smithi148.stdout:2/996: dread d7/d8c/f139 [0,4194304] 0 2024-03-29T08:07:30.961 INFO:tasks.workunit.client.0.smithi037.stdout:6/551: creat d0/d5/d1e/fc2 x:0 0 0 2024-03-29T08:07:30.962 INFO:tasks.workunit.client.0.smithi037.stdout:4/660: sync 2024-03-29T08:07:30.962 INFO:tasks.workunit.client.0.smithi037.stdout:0/523: sync 2024-03-29T08:07:30.962 INFO:tasks.workunit.client.0.smithi037.stdout:8/587: sync 2024-03-29T08:07:30.962 INFO:tasks.workunit.client.0.smithi037.stdout:3/578: sync 2024-03-29T08:07:30.963 INFO:tasks.workunit.client.0.smithi037.stdout:1/660: symlink d2/d6/dc/d18/d1d/le9 0 2024-03-29T08:07:30.969 INFO:tasks.workunit.client.0.smithi037.stdout:6/552: creat d0/d5/d1f/fc3 x:0 0 0 2024-03-29T08:07:30.972 INFO:tasks.workunit.client.0.smithi037.stdout:6/553: readlink d0/d19/d34/d4e/d69/d79/lab 0 2024-03-29T08:07:30.972 INFO:tasks.workunit.client.1.smithi148.stdout:8/891: write d17/d18/fa9 [899424,126557] 0 2024-03-29T08:07:30.973 INFO:tasks.workunit.client.1.smithi148.stdout:5/901: dwrite d0/db/d7b/f52 [0,4194304] 0 2024-03-29T08:07:30.974 INFO:tasks.workunit.client.1.smithi148.stdout:4/952: dwrite fd [0,4194304] 0 2024-03-29T08:07:30.974 INFO:tasks.workunit.client.1.smithi148.stdout:1/885: write d3/d18/d4d/d8f/f6b [3657568,78592] 0 2024-03-29T08:07:30.975 INFO:tasks.workunit.client.0.smithi037.stdout:3/579: creat dd/d15/d27/d33/d84/fbd x:0 0 0 2024-03-29T08:07:30.977 INFO:tasks.workunit.client.1.smithi148.stdout:7/879: dwrite d8/d38/d3c/da5/f121 [0,4194304] 0 2024-03-29T08:07:30.983 INFO:tasks.workunit.client.1.smithi148.stdout:0/908: read db/de/df/d11/d42/f52 [2298220,43727] 0 2024-03-29T08:07:30.983 INFO:tasks.workunit.client.0.smithi037.stdout:0/524: rename d9/da/dd/d13/l3f to d9/d74/d7d/lb3 0 2024-03-29T08:07:30.984 INFO:tasks.workunit.client.0.smithi037.stdout:0/525: fdatasync d9/da/d1b/f9d 0 2024-03-29T08:07:30.985 INFO:tasks.workunit.client.1.smithi148.stdout:3/947: dwrite de/d24/d33/d39/d46/d30/d37/fc3 [0,4194304] 0 2024-03-29T08:07:30.986 INFO:tasks.workunit.client.1.smithi148.stdout:8/892: chown d17/d49/d9a/dbe/d6f/d64/da5/lb5 1139599 1 2024-03-29T08:07:30.990 INFO:tasks.workunit.client.0.smithi037.stdout:8/588: mknod d5/d10/d2d/d34/d50/d83/d46/d8a/d96/cc4 0 2024-03-29T08:07:30.991 INFO:tasks.workunit.client.1.smithi148.stdout:0/909: write db/dc/daa/d47/daf/fb7 [2611746,89583] 0 2024-03-29T08:07:30.992 INFO:tasks.workunit.client.1.smithi148.stdout:0/910: readlink db/ld3 0 2024-03-29T08:07:30.997 INFO:tasks.workunit.client.0.smithi037.stdout:4/661: link d3/d57/d6b/faa d3/d4/d15/d1d/db/d45/d1e/d59/db5/dbe/fd7 0 2024-03-29T08:07:31.000 INFO:tasks.workunit.client.0.smithi037.stdout:0/526: creat d9/da/dd/d59/fb4 x:0 0 0 2024-03-29T08:07:31.001 INFO:tasks.workunit.client.1.smithi148.stdout:7/880: readlink d8/dfb/d44/d49/l122 0 2024-03-29T08:07:31.003 INFO:tasks.workunit.client.1.smithi148.stdout:1/886: symlink d3/d18/d4d/d8f/l122 0 2024-03-29T08:07:31.004 INFO:tasks.workunit.client.0.smithi037.stdout:4/662: read d3/f3a [704351,116612] 0 2024-03-29T08:07:31.006 INFO:tasks.workunit.client.1.smithi148.stdout:7/881: stat d8/d38/d3c/da5/de3/f102 0 2024-03-29T08:07:31.008 INFO:tasks.workunit.client.0.smithi037.stdout:6/554: creat d0/d19/fc4 x:0 0 0 2024-03-29T08:07:31.009 INFO:tasks.workunit.client.1.smithi148.stdout:1/887: chown d3/de/d6e/f88 7656300 1 2024-03-29T08:07:31.010 INFO:tasks.workunit.client.1.smithi148.stdout:7/882: chown d8/dfb/d11b 364 1 2024-03-29T08:07:31.010 INFO:tasks.workunit.client.0.smithi037.stdout:4/663: chown d3/d4/d15/d1d/db/d45/d93/fb8 5048086 1 2024-03-29T08:07:31.010 INFO:tasks.workunit.client.0.smithi037.stdout:0/527: dread d9/da/dd/d13/f36 [0,4194304] 0 2024-03-29T08:07:31.011 INFO:tasks.workunit.client.0.smithi037.stdout:1/661: getdents d2/d4b/d80 0 2024-03-29T08:07:31.011 INFO:tasks.workunit.client.1.smithi148.stdout:5/902: dwrite d0/db/d7b/f52 [0,4194304] 0 2024-03-29T08:07:31.012 INFO:tasks.workunit.client.1.smithi148.stdout:8/893: mknod d17/d49/d9a/dbe/d112/c124 0 2024-03-29T08:07:31.015 INFO:tasks.workunit.client.0.smithi037.stdout:3/580: link dd/d5e/f5c dd/d46/d2d/d3a/d6c/db1/fbe 0 2024-03-29T08:07:31.020 INFO:tasks.workunit.client.1.smithi148.stdout:0/911: dwrite db/de/df/d11/d42/fd0 [0,4194304] 0 2024-03-29T08:07:31.020 INFO:tasks.workunit.client.1.smithi148.stdout:3/948: link de/d24/d33/d39/d46/d30/d37/faa de/d24/d33/d39/f13b 0 2024-03-29T08:07:31.022 INFO:tasks.workunit.client.0.smithi037.stdout:6/555: dread - d0/d6a/d7b/d77/f8e zero size 2024-03-29T08:07:31.023 INFO:tasks.workunit.client.1.smithi148.stdout:1/888: symlink d3/d18/dae/dda/df3/l123 0 2024-03-29T08:07:31.025 INFO:tasks.workunit.client.0.smithi037.stdout:0/528: mkdir d9/da/dd/d54/db5 0 2024-03-29T08:07:31.035 INFO:tasks.workunit.client.1.smithi148.stdout:1/889: link d3/d18/d4d/l5c d3/d55/ddf/l124 0 2024-03-29T08:07:31.035 INFO:tasks.workunit.client.1.smithi148.stdout:1/890: chown d3/d18/c5a 12 1 2024-03-29T08:07:31.036 INFO:tasks.workunit.client.0.smithi037.stdout:1/662: symlink d2/d6/dc/d18/d22/lea 0 2024-03-29T08:07:31.036 INFO:tasks.workunit.client.1.smithi148.stdout:4/953: write df/d2c/d51/d77/d26/d9b/fa8 [1626040,65345] 0 2024-03-29T08:07:31.038 INFO:tasks.workunit.client.0.smithi037.stdout:8/589: dread d5/d10/f1d [4194304,4194304] 0 2024-03-29T08:07:31.038 INFO:tasks.workunit.client.1.smithi148.stdout:5/903: write d0/db/d1e/fbd [909399,88698] 0 2024-03-29T08:07:31.039 INFO:tasks.workunit.client.1.smithi148.stdout:1/891: dread - d3/de/fc0 zero size 2024-03-29T08:07:31.040 INFO:tasks.workunit.client.1.smithi148.stdout:5/904: stat d0/db/d1e/d35/c48 0 2024-03-29T08:07:31.041 INFO:tasks.workunit.client.0.smithi037.stdout:6/556: unlink d0/d5/fb 0 2024-03-29T08:07:31.042 INFO:tasks.workunit.client.1.smithi148.stdout:8/894: link d17/d18/d26/d4f/dde/l109 d17/d18/d26/d63/d66/d5f/dd0/d119/l125 0 2024-03-29T08:07:31.042 INFO:tasks.workunit.client.1.smithi148.stdout:9/950: sync 2024-03-29T08:07:31.046 INFO:tasks.workunit.client.1.smithi148.stdout:4/954: truncate df/d2c/d51/d77/d26/d34/d8a/f8c 1551680 0 2024-03-29T08:07:31.046 INFO:tasks.workunit.client.1.smithi148.stdout:5/905: symlink d0/db/d7b/d16/d87/d117/l124 0 2024-03-29T08:07:31.047 INFO:tasks.workunit.client.1.smithi148.stdout:0/912: rename db/dc/daa/d47/d9f to db/de/df/d115 0 2024-03-29T08:07:31.050 INFO:tasks.workunit.client.1.smithi148.stdout:1/892: mkdir d3/d18/d4d/d76/d85/d11d/d97/d111/d125 0 2024-03-29T08:07:31.051 INFO:tasks.workunit.client.1.smithi148.stdout:0/913: dread - db/de/df/d2b/f107 zero size 2024-03-29T08:07:31.051 INFO:tasks.workunit.client.0.smithi037.stdout:4/664: mkdir d3/d4/d15/d1d/db/d45/d93/d96/da0/dc9/dd8 0 2024-03-29T08:07:31.052 INFO:tasks.workunit.client.1.smithi148.stdout:4/955: chown df/d2c/d51/d77/d26/d34/ca3 0 1 2024-03-29T08:07:31.052 INFO:tasks.workunit.client.0.smithi037.stdout:8/590: stat d5/f1c 0 2024-03-29T08:07:31.053 INFO:tasks.workunit.client.1.smithi148.stdout:3/949: rename de/f110 to de/d24/d33/d39/d46/d30/d37/d62/d63/dbf/f13c 0 2024-03-29T08:07:31.054 INFO:tasks.workunit.client.1.smithi148.stdout:7/883: dread d8/d38/d3c/f5a [0,4194304] 0 2024-03-29T08:07:31.055 INFO:tasks.workunit.client.0.smithi037.stdout:8/591: chown d5/d10/d2d/d34/d50/fad 41175131 1 2024-03-29T08:07:31.057 INFO:tasks.workunit.client.1.smithi148.stdout:1/893: chown d3/d18/dae/dda/df3/d42/f15 1328 1 2024-03-29T08:07:31.058 INFO:tasks.workunit.client.1.smithi148.stdout:4/956: mknod df/d2c/d51/d77/d26/d34/d8a/d98/c12e 0 2024-03-29T08:07:31.058 INFO:tasks.workunit.client.0.smithi037.stdout:6/557: dwrite d0/d19/d42/d60/f9d [4194304,4194304] 0 2024-03-29T08:07:31.058 INFO:tasks.workunit.client.1.smithi148.stdout:0/914: mkdir db/de/df/d2b/dae/d116 0 2024-03-29T08:07:31.060 INFO:tasks.workunit.client.0.smithi037.stdout:3/581: creat dd/d46/d2d/d3a/fbf x:0 0 0 2024-03-29T08:07:31.062 INFO:tasks.workunit.client.1.smithi148.stdout:3/950: dread de/d24/f36 [4194304,4194304] 0 2024-03-29T08:07:31.063 INFO:tasks.workunit.client.0.smithi037.stdout:4/665: creat d3/d54/d88/fd9 x:0 0 0 2024-03-29T08:07:31.063 INFO:tasks.workunit.client.0.smithi037.stdout:0/529: write d9/da/dd/d13/f5c [2483511,69255] 0 2024-03-29T08:07:31.065 INFO:tasks.workunit.client.1.smithi148.stdout:9/951: dread db/d22/d3f/fbc [0,4194304] 0 2024-03-29T08:07:31.067 INFO:tasks.workunit.client.0.smithi037.stdout:6/558: creat d0/d5/d1f/da8/fc5 x:0 0 0 2024-03-29T08:07:31.068 INFO:tasks.workunit.client.1.smithi148.stdout:1/894: mknod d3/d18/d71/c126 0 2024-03-29T08:07:31.069 INFO:tasks.workunit.client.0.smithi037.stdout:3/582: rmdir dd/d5e 39 2024-03-29T08:07:31.071 INFO:tasks.workunit.client.0.smithi037.stdout:0/530: rmdir d9 39 2024-03-29T08:07:31.072 INFO:tasks.workunit.client.0.smithi037.stdout:1/663: write d2/d6/dc/d18/d1d/d21/f37 [4072819,42278] 0 2024-03-29T08:07:31.073 INFO:tasks.workunit.client.1.smithi148.stdout:8/895: dwrite d17/d18/f5c [4194304,4194304] 0 2024-03-29T08:07:31.073 INFO:tasks.workunit.client.0.smithi037.stdout:3/583: chown dd/d15/d18/d8e/fa8 97 1 2024-03-29T08:07:31.076 INFO:tasks.workunit.client.1.smithi148.stdout:3/951: stat de/d24/d33/d39/d46/d30/d37/d79/d10e/c112 0 2024-03-29T08:07:31.077 INFO:tasks.workunit.client.1.smithi148.stdout:5/906: dwrite d0/db/d1e/d79/ff3 [0,4194304] 0 2024-03-29T08:07:31.077 INFO:tasks.workunit.client.0.smithi037.stdout:4/666: creat d3/d4/d15/d1d/db/d45/d1e/d59/d7d/fda x:0 0 0 2024-03-29T08:07:31.078 INFO:tasks.workunit.client.0.smithi037.stdout:6/559: creat d0/d19/d42/d60/d62/fc6 x:0 0 0 2024-03-29T08:07:31.081 INFO:tasks.workunit.client.0.smithi037.stdout:4/667: fdatasync d3/d4/d27/f2e 0 2024-03-29T08:07:31.083 INFO:tasks.workunit.client.0.smithi037.stdout:3/584: mkdir dd/d46/d2d/d3a/d6c/db1/dc0 0 2024-03-29T08:07:31.084 INFO:tasks.workunit.client.1.smithi148.stdout:7/884: creat d8/f134 x:0 0 0 2024-03-29T08:07:31.085 INFO:tasks.workunit.client.1.smithi148.stdout:1/895: creat d3/de/d25/def/f127 x:0 0 0 2024-03-29T08:07:31.085 INFO:tasks.workunit.client.1.smithi148.stdout:7/885: stat d8/dfb/d11b/f39 0 2024-03-29T08:07:31.085 INFO:tasks.workunit.client.1.smithi148.stdout:0/915: dread db/de/df/fa6 [0,4194304] 0 2024-03-29T08:07:31.089 INFO:tasks.workunit.client.0.smithi037.stdout:8/592: dread d5/d10/f23 [4194304,4194304] 0 2024-03-29T08:07:31.090 INFO:tasks.workunit.client.0.smithi037.stdout:6/560: creat d0/d5/fc7 x:0 0 0 2024-03-29T08:07:31.091 INFO:tasks.workunit.client.0.smithi037.stdout:4/668: mkdir d3/d4/d15/d1d/db/d45/d93/ddb 0 2024-03-29T08:07:31.092 INFO:tasks.workunit.client.1.smithi148.stdout:0/916: read db/f23 [4145606,26716] 0 2024-03-29T08:07:31.096 INFO:tasks.workunit.client.1.smithi148.stdout:1/896: creat d3/d18/d4d/d76/d85/d11d/d47/de4/f128 x:0 0 0 2024-03-29T08:07:31.098 INFO:tasks.workunit.client.1.smithi148.stdout:7/886: rmdir d8/df4/d126 39 2024-03-29T08:07:31.100 INFO:tasks.workunit.client.0.smithi037.stdout:4/669: write d3/d4/d15/d1d/db/d45/d1e/d59/d79/d95/f9b [1154609,32156] 0 2024-03-29T08:07:31.100 INFO:tasks.workunit.client.0.smithi037.stdout:0/531: link d9/da/f3c d9/da/dd/d20/d55/fb6 0 2024-03-29T08:07:31.101 INFO:tasks.workunit.client.1.smithi148.stdout:3/952: write de/d3b/d49/f90 [1345334,9098] 0 2024-03-29T08:07:31.105 INFO:tasks.workunit.client.0.smithi037.stdout:4/670: fdatasync d3/d54/d88/fd9 0 2024-03-29T08:07:31.106 INFO:tasks.workunit.client.0.smithi037.stdout:0/532: write d9/d74/fa9 [214524,26680] 0 2024-03-29T08:07:31.107 INFO:tasks.workunit.client.1.smithi148.stdout:1/897: mkdir d3/d18/d4d/d76/d85/d11d/d47/d6d/d84/d8c/d129 0 2024-03-29T08:07:31.109 INFO:tasks.workunit.client.1.smithi148.stdout:5/907: dwrite d0/db/d84/daf/de1/fe4 [0,4194304] 0 2024-03-29T08:07:31.111 INFO:tasks.workunit.client.1.smithi148.stdout:4/957: dread df/d2c/d51/d77/d26/f84 [0,4194304] 0 2024-03-29T08:07:31.114 INFO:tasks.workunit.client.1.smithi148.stdout:3/953: mkdir de/d24/d33/d39/d46/d30/d13d 0 2024-03-29T08:07:31.114 INFO:tasks.workunit.client.1.smithi148.stdout:7/887: creat d8/dfb/db8/d115/f135 x:0 0 0 2024-03-29T08:07:31.114 INFO:tasks.workunit.client.0.smithi037.stdout:4/671: symlink d3/d80/dcd/ldc 0 2024-03-29T08:07:31.116 INFO:tasks.workunit.client.1.smithi148.stdout:1/898: dread - d3/d18/d71/f9b zero size 2024-03-29T08:07:31.119 INFO:tasks.workunit.client.1.smithi148.stdout:0/917: rename db/de/d24/lbe to db/de/df/d11/d1f/dc5/l117 0 2024-03-29T08:07:31.120 INFO:tasks.workunit.client.0.smithi037.stdout:6/561: dwrite d0/d19/d42/d60/d62/fc6 [0,4194304] 0 2024-03-29T08:07:31.122 INFO:tasks.workunit.client.1.smithi148.stdout:5/908: chown d0/db/d1e/d35/d42/d67/d76/d81/d98/dd5/d9c/dc8 268 1 2024-03-29T08:07:31.123 INFO:tasks.workunit.client.1.smithi148.stdout:9/952: dread db/d22/d26/d54/d4d/d35/d4b/d5a/f77 [0,4194304] 0 2024-03-29T08:07:31.123 INFO:tasks.workunit.client.1.smithi148.stdout:4/958: truncate df/d2c/d51/d77/d26/d34/f5c 4552677 0 2024-03-29T08:07:31.126 INFO:tasks.workunit.client.1.smithi148.stdout:1/899: symlink d3/l12a 0 2024-03-29T08:07:31.130 INFO:tasks.workunit.client.0.smithi037.stdout:4/672: creat d3/d4/d15/d1d/d4f/fdd x:0 0 0 2024-03-29T08:07:31.130 INFO:tasks.workunit.client.1.smithi148.stdout:3/954: mkdir de/d24/d33/d39/da3/dfe/d13e 0 2024-03-29T08:07:31.130 INFO:tasks.workunit.client.1.smithi148.stdout:5/909: dread - d0/db/d1e/d35/d42/d67/d76/d81/d98/dd5/fb0 zero size 2024-03-29T08:07:31.132 INFO:tasks.workunit.client.1.smithi148.stdout:5/910: read d0/db/d1e/d35/d42/f5e [67084,25876] 0 2024-03-29T08:07:31.133 INFO:tasks.workunit.client.1.smithi148.stdout:9/953: mkdir db/d10/d6d/d73/d122/d138 0 2024-03-29T08:07:31.135 INFO:tasks.workunit.client.1.smithi148.stdout:7/888: rename d8/d14/d45/d4d/df7/f110 to d8/f136 0 2024-03-29T08:07:31.135 INFO:tasks.workunit.client.0.smithi037.stdout:6/562: symlink d0/d19/d34/d4e/lc8 0 2024-03-29T08:07:31.137 INFO:tasks.workunit.client.0.smithi037.stdout:4/673: creat d3/d57/d85/fde x:0 0 0 2024-03-29T08:07:31.137 INFO:tasks.workunit.client.1.smithi148.stdout:3/955: write de/d24/d33/d39/d46/d30/d37/d62/d63/f102 [1184480,73373] 0 2024-03-29T08:07:31.140 INFO:tasks.workunit.client.1.smithi148.stdout:5/911: mknod d0/d46/d9e/d114/c125 0 2024-03-29T08:07:31.143 INFO:tasks.workunit.client.0.smithi037.stdout:8/593: dwrite d5/f15 [0,4194304] 0 2024-03-29T08:07:31.145 INFO:tasks.workunit.client.1.smithi148.stdout:9/954: rmdir db/d22/d26/d54/d4d/d35/d4b/d5a 39 2024-03-29T08:07:31.146 INFO:tasks.workunit.client.0.smithi037.stdout:6/563: mknod d0/d19/d42/d60/cc9 0 2024-03-29T08:07:31.146 INFO:tasks.workunit.client.1.smithi148.stdout:3/956: write de/d24/d33/d39/d46/d8d/fc2 [2527998,66382] 0 2024-03-29T08:07:31.147 INFO:tasks.workunit.client.0.smithi037.stdout:4/674: mknod d3/d4/d15/d1d/cdf 0 2024-03-29T08:07:31.149 INFO:tasks.workunit.client.0.smithi037.stdout:4/675: stat d3/d4/d15/d1d/f3f 0 2024-03-29T08:07:31.149 INFO:tasks.workunit.client.1.smithi148.stdout:5/912: mknod d0/db/d1e/d35/d42/d67/d9a/c126 0 2024-03-29T08:07:31.150 INFO:tasks.workunit.client.1.smithi148.stdout:0/918: rename db/dc/daa/d47/d4c/c88 to db/dc/c118 0 2024-03-29T08:07:31.153 INFO:tasks.workunit.client.1.smithi148.stdout:4/959: dwrite df/d2c/d51/d77/d26/d9b/da7/d7c/ff9 [0,4194304] 0 2024-03-29T08:07:31.155 INFO:tasks.workunit.client.0.smithi037.stdout:0/533: dwrite d9/da/dd/d20/d55/dac/fb2 [0,4194304] 0 2024-03-29T08:07:31.156 INFO:tasks.workunit.client.1.smithi148.stdout:9/955: write db/d10/d2d/f34 [445062,84325] 0 2024-03-29T08:07:31.157 INFO:tasks.workunit.client.0.smithi037.stdout:4/676: rename d3/f21 to d3/d4/d15/d1d/db/d45/d1e/d59/d60/dd0/fe0 0 2024-03-29T08:07:31.160 INFO:tasks.workunit.client.0.smithi037.stdout:0/534: fsync d9/da/dd/d20/d55/fb6 0 2024-03-29T08:07:31.161 INFO:tasks.workunit.client.0.smithi037.stdout:4/677: dread - d3/d57/d85/fde zero size 2024-03-29T08:07:31.162 INFO:tasks.workunit.client.0.smithi037.stdout:0/535: read - d9/da/f88 zero size 2024-03-29T08:07:31.164 INFO:tasks.workunit.client.1.smithi148.stdout:3/957: write de/d24/d33/d39/d46/d30/d37/d62/dde/f107 [816565,31562] 0 2024-03-29T08:07:31.165 INFO:tasks.workunit.client.0.smithi037.stdout:4/678: mknod d3/d57/d6b/ce1 0 2024-03-29T08:07:31.167 INFO:tasks.workunit.client.0.smithi037.stdout:8/594: write d5/f16 [3338845,33184] 0 2024-03-29T08:07:31.167 INFO:tasks.workunit.client.0.smithi037.stdout:0/536: mknod d9/d74/d7d/d7e/cb7 0 2024-03-29T08:07:31.170 INFO:tasks.workunit.client.1.smithi148.stdout:7/889: dread d8/f10 [0,4194304] 0 2024-03-29T08:07:31.171 INFO:tasks.workunit.client.1.smithi148.stdout:9/956: getdents db/d10/d2d 0 2024-03-29T08:07:31.176 INFO:tasks.workunit.client.0.smithi037.stdout:6/564: write d0/d5/d1e/f5e [600302,65868] 0 2024-03-29T08:07:31.178 INFO:tasks.workunit.client.1.smithi148.stdout:3/958: symlink de/d3b/d49/d128/l13f 0 2024-03-29T08:07:31.182 INFO:tasks.workunit.client.0.smithi037.stdout:6/565: symlink d0/d5/d1a/lca 0 2024-03-29T08:07:31.182 INFO:tasks.workunit.client.1.smithi148.stdout:4/960: link df/d2c/db2/cfb df/d2c/d51/d77/d26/d34/d3e/c12f 0 2024-03-29T08:07:31.184 INFO:tasks.workunit.client.0.smithi037.stdout:0/537: dread d9/da/d1b/f3d [0,4194304] 0 2024-03-29T08:07:31.184 INFO:tasks.workunit.client.1.smithi148.stdout:0/919: write db/de/d8e/da9/fca [4334106,92431] 0 2024-03-29T08:07:31.185 INFO:tasks.workunit.client.0.smithi037.stdout:6/566: readlink d0/d19/d34/d4e/d69/d79/lab 0 2024-03-29T08:07:31.191 INFO:tasks.workunit.client.0.smithi037.stdout:0/538: mknod d9/da/dd/d20/d38/d8a/cb8 0 2024-03-29T08:07:31.191 INFO:tasks.workunit.client.1.smithi148.stdout:3/959: getdents de/d24/d33/d39/da3/dbc 0 2024-03-29T08:07:31.192 INFO:tasks.workunit.client.1.smithi148.stdout:5/913: dwrite d0/db/d1e/d35/d42/d67/d76/d81/d98/dd5/f10e [0,4194304] 0 2024-03-29T08:07:31.195 INFO:tasks.workunit.client.1.smithi148.stdout:4/961: unlink df/d2c/d51/d77/dbe/ced 0 2024-03-29T08:07:31.197 INFO:tasks.workunit.client.1.smithi148.stdout:2/997: sync 2024-03-29T08:07:31.197 INFO:tasks.workunit.client.1.smithi148.stdout:6/924: sync 2024-03-29T08:07:31.197 INFO:tasks.workunit.client.0.smithi037.stdout:0/539: fdatasync d9/d74/f97 0 2024-03-29T08:07:31.199 INFO:tasks.workunit.client.1.smithi148.stdout:7/890: dread d8/dfb/d11b/d37/f9c [0,4194304] 0 2024-03-29T08:07:31.204 INFO:tasks.workunit.client.0.smithi037.stdout:8/595: write d5/d10/d19/d3d/d4a/d4c/f65 [1839477,15451] 0 2024-03-29T08:07:31.207 INFO:tasks.workunit.client.1.smithi148.stdout:1/900: dread d3/de/d25/f9c [0,4194304] 0 2024-03-29T08:07:31.209 INFO:tasks.workunit.client.1.smithi148.stdout:4/962: mknod df/d2c/d51/d77/d26/d9b/c130 0 2024-03-29T08:07:31.210 INFO:tasks.workunit.client.0.smithi037.stdout:6/567: dwrite d0/d5/d1e/f7f [0,4194304] 0 2024-03-29T08:07:31.214 INFO:tasks.workunit.client.0.smithi037.stdout:8/596: write d5/d10/d2d/d34/d50/d83/d46/d4e/f57 [5259850,33818] 0 2024-03-29T08:07:31.216 INFO:tasks.workunit.client.0.smithi037.stdout:4/679: dread d3/d4/d15/d1d/db/d45/d1e/d59/d7d/f9f [0,4194304] 0 2024-03-29T08:07:31.216 INFO:tasks.workunit.client.1.smithi148.stdout:2/998: write d7/d24/d4b/f55 [3518092,73854] 0 2024-03-29T08:07:31.219 INFO:tasks.workunit.client.0.smithi037.stdout:0/540: getdents d9/da/d1b/d22 0 2024-03-29T08:07:31.220 INFO:tasks.workunit.client.1.smithi148.stdout:9/957: dwrite db/d22/fd7 [4194304,4194304] 0 2024-03-29T08:07:31.222 INFO:tasks.workunit.client.0.smithi037.stdout:8/597: mkdir d5/d10/d2d/d34/dc5 0 2024-03-29T08:07:31.222 INFO:tasks.workunit.client.1.smithi148.stdout:6/925: write d6/d9/d1e/d27/d48/d73/f83 [3593382,110113] 0 2024-03-29T08:07:31.226 INFO:tasks.workunit.client.0.smithi037.stdout:4/680: mknod d3/d4/d15/d1d/db/d45/d1e/d59/d73/ce2 0 2024-03-29T08:07:31.230 INFO:tasks.workunit.client.1.smithi148.stdout:1/901: mkdir d3/d18/d4d/d76/d85/d11d/d47/d6d/d72/d12b 0 2024-03-29T08:07:31.232 INFO:tasks.workunit.client.0.smithi037.stdout:0/541: creat d9/d74/d7d/d7e/fb9 x:0 0 0 2024-03-29T08:07:31.232 INFO:tasks.workunit.client.1.smithi148.stdout:7/891: fsync d8/d38/d6a/d91/dff/f10c 0 2024-03-29T08:07:31.236 INFO:tasks.workunit.client.1.smithi148.stdout:9/958: chown db/d22/d26/f10e 514 1 2024-03-29T08:07:31.236 INFO:tasks.workunit.client.1.smithi148.stdout:8/896: sync 2024-03-29T08:07:31.236 INFO:tasks.workunit.client.1.smithi148.stdout:2/999: mknod d7/da/d33/d3c/d54/c142 0 2024-03-29T08:07:31.238 INFO:tasks.workunit.client.1.smithi148.stdout:0/920: truncate db/de/df/d11/d2d/fdf 382735 0 2024-03-29T08:07:31.240 INFO:tasks.workunit.client.0.smithi037.stdout:0/542: unlink d9/da/d1b/d22/fad 0 2024-03-29T08:07:31.241 INFO:tasks.workunit.client.1.smithi148.stdout:1/902: mknod d3/de/d6e/c12c 0 2024-03-29T08:07:31.242 INFO:tasks.workunit.client.1.smithi148.stdout:3/960: dwrite de/d3b/d49/fdd [0,4194304] 0 2024-03-29T08:07:31.243 INFO:tasks.workunit.client.0.smithi037.stdout:8/598: write d5/d10/d19/f2f [2768879,5617] 0 2024-03-29T08:07:31.246 INFO:tasks.workunit.client.0.smithi037.stdout:0/543: mknod d9/da/dd/d20/d55/d5f/d9f/cba 0 2024-03-29T08:07:31.249 INFO:tasks.workunit.client.0.smithi037.stdout:8/599: creat d5/d10/d2d/fc6 x:0 0 0 2024-03-29T08:07:31.251 INFO:tasks.workunit.client.1.smithi148.stdout:6/926: truncate d6/d9/d1e/d27/f9b 300776 0 2024-03-29T08:07:31.251 INFO:tasks.workunit.client.1.smithi148.stdout:1/903: stat d3/de/c4a 0 2024-03-29T08:07:31.252 INFO:tasks.workunit.client.1.smithi148.stdout:8/897: mkdir d17/d18/d26/d63/d126 0 2024-03-29T08:07:31.253 INFO:tasks.workunit.client.0.smithi037.stdout:4/681: dread d3/d4/d15/f63 [4194304,4194304] 0 2024-03-29T08:07:31.254 INFO:tasks.workunit.client.0.smithi037.stdout:8/600: dread - d5/d10/d19/d3d/d4a/f61 zero size 2024-03-29T08:07:31.255 INFO:tasks.workunit.client.1.smithi148.stdout:5/914: dwrite d0/db/d1e/d35/d42/d67/d76/fba [4194304,4194304] 0 2024-03-29T08:07:31.255 INFO:tasks.workunit.client.1.smithi148.stdout:4/963: dwrite df/f27 [0,4194304] 0 2024-03-29T08:07:31.259 INFO:tasks.workunit.client.0.smithi037.stdout:6/568: dread d0/d5/d1f/f4c [0,4194304] 0 2024-03-29T08:07:31.259 INFO:tasks.workunit.client.0.smithi037.stdout:4/682: creat d3/fe3 x:0 0 0 2024-03-29T08:07:31.261 INFO:tasks.workunit.client.1.smithi148.stdout:1/904: truncate d3/d18/d4d/f106 577111 0 2024-03-29T08:07:31.263 INFO:tasks.workunit.client.1.smithi148.stdout:7/892: link d8/d38/d6a/d91/la2 d8/df4/d111/l137 0 2024-03-29T08:07:31.263 INFO:tasks.workunit.client.1.smithi148.stdout:9/959: fdatasync db/d22/fd7 0 2024-03-29T08:07:31.266 INFO:tasks.workunit.client.1.smithi148.stdout:6/927: write d6/d9/d1e/d36/d4d/d9d/d28/dda/db6/ffa [328447,36603] 0 2024-03-29T08:07:31.268 INFO:tasks.workunit.client.1.smithi148.stdout:0/921: rename db/c2f to db/de/df/d14/dea/c119 0 2024-03-29T08:07:31.268 INFO:tasks.workunit.client.1.smithi148.stdout:3/961: rmdir de/d24/d33/d39/d46/d30/d37/d62/d63/dbf/dfc 39 2024-03-29T08:07:31.273 INFO:tasks.workunit.client.1.smithi148.stdout:8/898: getdents d17/d49/d9a/dbe/d6f/d64/df1/df6 0 2024-03-29T08:07:31.274 INFO:tasks.workunit.client.1.smithi148.stdout:0/922: chown db/de/df/d11/d2d/da3/dee 287925838 1 2024-03-29T08:07:31.274 INFO:tasks.workunit.client.0.smithi037.stdout:4/683: creat d3/d4/d15/d1d/db/d45/d1e/d59/d60/d6f/db9/fe4 x:0 0 0 2024-03-29T08:07:31.278 INFO:tasks.workunit.client.1.smithi148.stdout:6/928: write d6/fe [6637326,28114] 0 2024-03-29T08:07:31.280 INFO:tasks.workunit.client.1.smithi148.stdout:9/960: symlink db/d22/d26/d54/l139 0 2024-03-29T08:07:31.281 INFO:tasks.workunit.client.1.smithi148.stdout:0/923: creat db/dc/daa/d47/daf/f11a x:0 0 0 2024-03-29T08:07:31.281 INFO:tasks.workunit.client.1.smithi148.stdout:9/961: chown db/d10 14154 1 2024-03-29T08:07:31.286 INFO:tasks.workunit.client.0.smithi037.stdout:4/684: mknod d3/d4/d15/d1d/db/d45/d1e/d59/d7d/d97/ce5 0 2024-03-29T08:07:31.287 INFO:tasks.workunit.client.1.smithi148.stdout:8/899: creat d17/d49/d9a/dbe/d6f/f127 x:0 0 0 2024-03-29T08:07:31.287 INFO:tasks.workunit.client.1.smithi148.stdout:1/905: rename d3/d18/dae/dda/df3/c62 to d3/de/d25/df1/c12d 0 2024-03-29T08:07:31.288 INFO:tasks.workunit.client.0.smithi037.stdout:8/601: dwrite d5/d10/d2d/d34/d50/d83/d46/d4e/f6a [0,4194304] 0 2024-03-29T08:07:31.288 INFO:tasks.workunit.client.1.smithi148.stdout:0/924: mknod db/de/df/d2b/dfa/c11b 0 2024-03-29T08:07:31.288 INFO:tasks.workunit.client.1.smithi148.stdout:3/962: mkdir de/d24/d33/d39/d46/d30/d37/d62/dde/d140 0 2024-03-29T08:07:31.293 INFO:tasks.workunit.client.0.smithi037.stdout:8/602: write d5/d10/d2d/f3e [688927,87070] 0 2024-03-29T08:07:31.293 INFO:tasks.workunit.client.1.smithi148.stdout:0/925: fdatasync db/de/df/d115/f10b 0 2024-03-29T08:07:31.293 INFO:tasks.workunit.client.1.smithi148.stdout:8/900: truncate d17/d74/d86/f8d 5106214 0 2024-03-29T08:07:31.294 INFO:tasks.workunit.client.0.smithi037.stdout:4/685: symlink d3/d4/d15/d1d/db/d45/d1e/d59/db5/le6 0 2024-03-29T08:07:31.294 INFO:tasks.workunit.client.1.smithi148.stdout:6/929: unlink d6/d9/d1e/d36/d4d/d9d/d28/dda/db6/df3/cf8 0 2024-03-29T08:07:31.294 INFO:tasks.workunit.client.1.smithi148.stdout:1/906: write d3/de/d25/d51/f73 [2015600,121265] 0 2024-03-29T08:07:31.299 INFO:tasks.workunit.client.0.smithi037.stdout:8/603: mknod d5/d10/d45/d4b/cc7 0 2024-03-29T08:07:31.299 INFO:tasks.workunit.client.1.smithi148.stdout:9/962: mkdir db/d10/d2d/d4e/d5f/d6e/d7a/d13a 0 2024-03-29T08:07:31.299 INFO:tasks.workunit.client.1.smithi148.stdout:0/926: dread - db/dc/daa/d47/daf/f11a zero size 2024-03-29T08:07:31.300 INFO:tasks.workunit.client.1.smithi148.stdout:4/964: link df/d2c/d51/d77/d26/d9b/lab df/d2c/d51/d77/d26/d9b/da7/d7c/l131 0 2024-03-29T08:07:31.301 INFO:tasks.workunit.client.1.smithi148.stdout:7/893: rename d8/d14/da8/f112 to d8/dfb/d44/d50/f138 0 2024-03-29T08:07:31.302 INFO:tasks.workunit.client.1.smithi148.stdout:5/915: dread d0/db/d1e/d35/d42/d96/fdf [0,4194304] 0 2024-03-29T08:07:31.302 INFO:tasks.workunit.client.1.smithi148.stdout:8/901: symlink d17/d18/d26/d4f/d94/l128 0 2024-03-29T08:07:31.304 INFO:tasks.workunit.client.1.smithi148.stdout:1/907: read d3/d18/dae/dda/df3/f48 [445781,103286] 0 2024-03-29T08:07:31.304 INFO:tasks.workunit.client.1.smithi148.stdout:4/965: chown df/d2c/d10b/d125 2545 1 2024-03-29T08:07:31.305 INFO:tasks.workunit.client.1.smithi148.stdout:0/927: rmdir db/dc 39 2024-03-29T08:07:31.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:07:30 smithi148 ceph-mon[44992]: pgmap v47: 65 pgs: 65 active+clean; 2.9 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 103 MiB/s rd, 247 MiB/s wr, 528 op/s 2024-03-29T08:07:31.307 INFO:tasks.workunit.client.1.smithi148.stdout:6/930: read d6/d9/d1e/d27/f32 [3929171,37696] 0 2024-03-29T08:07:31.307 INFO:tasks.workunit.client.1.smithi148.stdout:7/894: rmdir d8/df4 39 2024-03-29T08:07:31.311 INFO:tasks.workunit.client.1.smithi148.stdout:5/916: unlink d0/db/d1e/f103 0 2024-03-29T08:07:31.312 INFO:tasks.workunit.client.1.smithi148.stdout:8/902: symlink d17/d49/d9a/db8/dff/d116/l129 0 2024-03-29T08:07:31.312 INFO:tasks.workunit.client.1.smithi148.stdout:0/928: write db/de/fc3 [933335,37898] 0 2024-03-29T08:07:31.314 INFO:tasks.workunit.client.1.smithi148.stdout:6/931: write d6/d9/d1e/f5d [3408031,8362] 0 2024-03-29T08:07:31.322 INFO:tasks.workunit.client.0.smithi037.stdout:6/569: dread d0/d5/d1a/f2f [0,4194304] 0 2024-03-29T08:07:31.327 INFO:tasks.workunit.client.1.smithi148.stdout:5/917: write d0/db/d1e/d35/d42/d67/d76/d81/d98/dd5/d9c/fbf [684838,13962] 0 2024-03-29T08:07:31.328 INFO:tasks.workunit.client.0.smithi037.stdout:4/686: dwrite d3/d4/f3e [0,4194304] 0 2024-03-29T08:07:31.329 INFO:tasks.workunit.client.1.smithi148.stdout:5/918: chown d0/l49 1242 1 2024-03-29T08:07:31.331 INFO:tasks.workunit.client.0.smithi037.stdout:6/570: rmdir d0/d5/d1a/d61 39 2024-03-29T08:07:31.332 INFO:tasks.workunit.client.1.smithi148.stdout:5/919: stat d0/db/f3e 0 2024-03-29T08:07:31.332 INFO:tasks.workunit.client.1.smithi148.stdout:3/963: truncate de/d24/d33/d39/d46/d30/d37/d62/f11e 321477 0 2024-03-29T08:07:31.334 INFO:tasks.workunit.client.1.smithi148.stdout:1/908: getdents d3/d18/d4d/d76/d85/d11d/d47/d6d/d84/d8c 0 2024-03-29T08:07:31.336 INFO:tasks.workunit.client.1.smithi148.stdout:8/903: rename d17/d74/f5a to d17/d18/d26/d63/d66/d5f/f12a 0 2024-03-29T08:07:31.337 INFO:tasks.workunit.client.0.smithi037.stdout:4/687: mkdir d3/de7 0 2024-03-29T08:07:31.340 INFO:tasks.workunit.client.0.smithi037.stdout:4/688: dread - d3/d57/d6b/fc3 zero size 2024-03-29T08:07:31.341 INFO:tasks.workunit.client.0.smithi037.stdout:6/571: rename d0/d19/d34/d4e/c68 to d0/d20/ccb 0 2024-03-29T08:07:31.342 INFO:tasks.workunit.client.0.smithi037.stdout:6/572: readlink d0/d20/lb7 0 2024-03-29T08:07:31.347 INFO:tasks.workunit.client.0.smithi037.stdout:6/573: mkdir d0/d19/d34/d4e/d69/d79/dcc 0 2024-03-29T08:07:31.348 INFO:tasks.workunit.client.1.smithi148.stdout:3/964: creat de/d24/d33/d39/d46/d70/def/f141 x:0 0 0 2024-03-29T08:07:31.350 INFO:tasks.workunit.client.0.smithi037.stdout:6/574: chown d0/d19/d34/d4e/d69/d79/lab 12266230 1 2024-03-29T08:07:31.351 INFO:tasks.workunit.client.1.smithi148.stdout:7/895: dwrite d8/d14/d45/d4d/d78/fa0 [0,4194304] 0 2024-03-29T08:07:31.351 INFO:tasks.workunit.client.1.smithi148.stdout:1/909: mknod d3/d18/dae/de7/c12e 0 2024-03-29T08:07:31.355 INFO:tasks.workunit.client.0.smithi037.stdout:6/575: mkdir d0/d19/d34/d4e/d69/d79/dcd 0 2024-03-29T08:07:31.356 INFO:tasks.workunit.client.0.smithi037.stdout:4/689: dwrite d3/d4/d27/f86 [0,4194304] 0 2024-03-29T08:07:31.356 INFO:tasks.workunit.client.1.smithi148.stdout:4/966: dwrite df/d2c/d51/f67 [0,4194304] 0 2024-03-29T08:07:31.357 INFO:tasks.workunit.client.1.smithi148.stdout:7/896: chown d8/d38/d6a/fbd 119556404 1 2024-03-29T08:07:31.361 INFO:tasks.workunit.client.1.smithi148.stdout:3/965: symlink de/d24/d33/d39/d46/d70/def/l142 0 2024-03-29T08:07:31.362 INFO:tasks.workunit.client.1.smithi148.stdout:5/920: write d0/db/d1e/d35/d42/d67/d76/d81/d98/dd5/fb0 [519467,41241] 0 2024-03-29T08:07:31.365 INFO:tasks.workunit.client.0.smithi037.stdout:4/690: mkdir d3/d4/d15/d1d/db/d45/d1e/d59/d7d/de8 0 2024-03-29T08:07:31.365 INFO:tasks.workunit.client.1.smithi148.stdout:5/921: chown d0/db/la9 635385 1 2024-03-29T08:07:31.367 INFO:tasks.workunit.client.1.smithi148.stdout:9/963: dread db/d22/d26/d54/d4d/d35/d4b/d58/f92 [0,4194304] 0 2024-03-29T08:07:31.368 INFO:tasks.workunit.client.1.smithi148.stdout:5/922: chown d0/db/ld 28602 1 2024-03-29T08:07:31.373 INFO:tasks.workunit.client.1.smithi148.stdout:8/904: write d17/d18/d26/d4f/fb7 [389608,73349] 0 2024-03-29T08:07:31.379 INFO:tasks.workunit.client.1.smithi148.stdout:5/923: read d0/db/d84/daf/ffa [2785354,3659] 0 2024-03-29T08:07:31.385 INFO:tasks.workunit.client.1.smithi148.stdout:1/910: write d3/d55/f43 [1141175,102691] 0 2024-03-29T08:07:31.391 INFO:tasks.workunit.client.1.smithi148.stdout:8/905: truncate d17/d18/d26/d63/d66/d5f/f6c 5453369 0 2024-03-29T08:07:31.392 INFO:tasks.workunit.client.0.smithi037.stdout:7/650: sync 2024-03-29T08:07:31.394 INFO:tasks.workunit.client.0.smithi037.stdout:4/691: write d3/d80/dcd/f90 [274473,61616] 0 2024-03-29T08:07:31.395 INFO:tasks.workunit.client.1.smithi148.stdout:6/932: dread d6/d9/d1e/d36/d4d/d9d/d28/db3/dd2/fef [0,4194304] 0 2024-03-29T08:07:31.395 INFO:tasks.workunit.client.1.smithi148.stdout:0/929: dread db/dc/daa/d47/d4c/d5a/d7a/ff7 [0,4194304] 0 2024-03-29T08:07:31.398 INFO:tasks.workunit.client.0.smithi037.stdout:7/651: fsync dd/d1d/d49/f5a 0 2024-03-29T08:07:31.400 INFO:tasks.workunit.client.0.smithi037.stdout:4/692: chown d3/d4/d15/d1d/db/d45/d1e/d59/d7d/fca 1234022 1 2024-03-29T08:07:31.400 INFO:tasks.workunit.client.1.smithi148.stdout:4/967: dwrite df/f27 [0,4194304] 0 2024-03-29T08:07:31.402 INFO:tasks.workunit.client.0.smithi037.stdout:7/652: fsync dd/d21/d35/fc5 0 2024-03-29T08:07:31.403 INFO:tasks.workunit.client.0.smithi037.stdout:6/576: dwrite d0/d19/d34/d4e/f54 [0,4194304] 0 2024-03-29T08:07:31.404 INFO:tasks.workunit.client.1.smithi148.stdout:8/906: fsync d17/d18/d26/d4f/d94/f9b 0 2024-03-29T08:07:31.405 INFO:tasks.workunit.client.1.smithi148.stdout:6/933: mknod d6/c125 0 2024-03-29T08:07:31.406 INFO:tasks.workunit.client.0.smithi037.stdout:7/653: symlink dd/d21/ldd 0 2024-03-29T08:07:31.412 INFO:tasks.workunit.client.1.smithi148.stdout:5/924: link d0/db/d1e/d5b/d66/l8c d0/db/d1e/d35/d42/d67/d76/d81/d98/dd5/d9c/l127 0 2024-03-29T08:07:31.413 INFO:tasks.workunit.client.1.smithi148.stdout:0/930: readlink db/de/df/d11/d1f/dc5/l117 0 2024-03-29T08:07:31.414 INFO:tasks.workunit.client.0.smithi037.stdout:4/693: creat d3/d4/d15/d1d/db/d45/d1e/d59/fe9 x:0 0 0 2024-03-29T08:07:31.416 INFO:tasks.workunit.client.1.smithi148.stdout:6/934: creat d6/d9/d1e/f126 x:0 0 0 2024-03-29T08:07:31.418 INFO:tasks.workunit.client.0.smithi037.stdout:7/654: dread dd/d72/f98 [0,4194304] 0 2024-03-29T08:07:31.418 INFO:tasks.workunit.client.0.smithi037.stdout:6/577: creat d0/d19/fce x:0 0 0 2024-03-29T08:07:31.419 INFO:tasks.workunit.client.1.smithi148.stdout:4/968: mknod df/d2c/d51/d77/d26/d34/d43/c132 0 2024-03-29T08:07:31.420 INFO:tasks.workunit.client.0.smithi037.stdout:4/694: fsync d3/d4/d15/d1d/f64 0 2024-03-29T08:07:31.422 INFO:tasks.workunit.client.1.smithi148.stdout:9/964: dwrite db/d22/d26/d54/fc9 [0,4194304] 0 2024-03-29T08:07:31.425 INFO:tasks.workunit.client.1.smithi148.stdout:8/907: mknod d17/d18/d26/d63/d7d/dc6/c12b 0 2024-03-29T08:07:31.426 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:07:30 smithi037 ceph-mon[35162]: pgmap v47: 65 pgs: 65 active+clean; 2.9 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 103 MiB/s rd, 247 MiB/s wr, 528 op/s 2024-03-29T08:07:31.431 INFO:tasks.workunit.client.0.smithi037.stdout:6/578: rename d0/d5/d1e/d44/d71/f74 to d0/fcf 0 2024-03-29T08:07:31.435 INFO:tasks.workunit.client.0.smithi037.stdout:4/695: rename d3/d80/dcd/ccf to d3/d4/d15/d1d/db/d72/d98/cea 0 2024-03-29T08:07:31.435 INFO:tasks.workunit.client.1.smithi148.stdout:4/969: creat df/d2c/d51/d77/d26/d9b/f133 x:0 0 0 2024-03-29T08:07:31.436 INFO:tasks.workunit.client.0.smithi037.stdout:7/655: dread dd/d1d/d34/f3f [0,4194304] 0 2024-03-29T08:07:31.436 INFO:tasks.workunit.client.0.smithi037.stdout:6/579: mkdir d0/d5/d1f/da8/dd0 0 2024-03-29T08:07:31.437 INFO:tasks.workunit.client.1.smithi148.stdout:1/911: dwrite d3/d18/d4d/d76/d85/d11d/d47/fe0 [0,4194304] 0 2024-03-29T08:07:31.439 INFO:tasks.workunit.client.1.smithi148.stdout:6/935: write d6/d9/d1e/d36/d4d/d9d/d28/d98/dc2/ffd [559031,116000] 0 2024-03-29T08:07:31.440 INFO:tasks.workunit.client.1.smithi148.stdout:3/966: dread de/d3b/fb8 [0,4194304] 0 2024-03-29T08:07:31.443 INFO:tasks.workunit.client.1.smithi148.stdout:8/908: chown d17/f23 950540 1 2024-03-29T08:07:31.444 INFO:tasks.workunit.client.1.smithi148.stdout:0/931: rename db/de/df/d11/f16 to db/dc/daa/d47/d4c/d5a/d7a/db1/d110/f11c 0 2024-03-29T08:07:31.447 INFO:tasks.workunit.client.1.smithi148.stdout:7/897: dread d8/d38/d6a/d91/dff/f10c [0,4194304] 0 2024-03-29T08:07:31.447 INFO:tasks.workunit.client.1.smithi148.stdout:9/965: read db/d10/d2d/d4e/dd4/f70 [367435,1385] 0 2024-03-29T08:07:31.448 INFO:tasks.workunit.client.1.smithi148.stdout:6/936: mkdir d6/d9/dc7/d127 0 2024-03-29T08:07:31.454 INFO:tasks.workunit.client.1.smithi148.stdout:3/967: creat de/d24/d33/d39/d46/d30/d37/d79/d10e/f143 x:0 0 0 2024-03-29T08:07:31.454 INFO:tasks.workunit.client.1.smithi148.stdout:1/912: read d3/d18/d4d/d76/d85/d11d/d47/d6d/d72/fba [269452,34116] 0 2024-03-29T08:07:31.456 INFO:tasks.workunit.client.1.smithi148.stdout:0/932: symlink db/de/df/d11/d2d/da3/l11d 0 2024-03-29T08:07:31.459 INFO:tasks.workunit.client.1.smithi148.stdout:7/898: write d8/dfb/d44/d50/f138 [598120,45461] 0 2024-03-29T08:07:31.460 INFO:tasks.workunit.client.1.smithi148.stdout:9/966: chown db/d10/d9d/l117 25678962 1 2024-03-29T08:07:31.463 INFO:tasks.workunit.client.1.smithi148.stdout:8/909: write d17/d74/d86/fdd [1042033,123421] 0 2024-03-29T08:07:31.466 INFO:tasks.workunit.client.1.smithi148.stdout:3/968: write de/d24/d33/d39/d46/d30/d37/d62/d63/fce [70279,10994] 0 2024-03-29T08:07:31.467 INFO:tasks.workunit.client.0.smithi037.stdout:7/656: dwrite dd/d21/f32 [4194304,4194304] 0 2024-03-29T08:07:31.470 INFO:tasks.workunit.client.1.smithi148.stdout:0/933: creat db/de/df/d2b/dae/d116/f11e x:0 0 0 2024-03-29T08:07:31.470 INFO:tasks.workunit.client.1.smithi148.stdout:9/967: chown db/d22/d26/d54/d4d/d35/d4b/c128 7526 1 2024-03-29T08:07:31.470 INFO:tasks.workunit.client.1.smithi148.stdout:1/913: write d3/d18/d4d/d9e/f59 [579472,90675] 0 2024-03-29T08:07:31.472 INFO:tasks.workunit.client.1.smithi148.stdout:8/910: symlink d17/d18/d26/d63/d126/l12c 0 2024-03-29T08:07:31.473 INFO:tasks.workunit.client.1.smithi148.stdout:7/899: mkdir d8/df4/d126/d139 0 2024-03-29T08:07:31.478 INFO:tasks.workunit.client.0.smithi037.stdout:7/657: write dd/f17 [769205,124725] 0 2024-03-29T08:07:31.478 INFO:tasks.workunit.client.1.smithi148.stdout:5/925: dwrite d0/db/d84/daf/de1/f107 [4194304,4194304] 0 2024-03-29T08:07:31.480 INFO:tasks.workunit.client.1.smithi148.stdout:1/914: mkdir d3/d18/d4d/d9e/d12f 0 2024-03-29T08:07:31.480 INFO:tasks.workunit.client.0.smithi037.stdout:7/658: fsync dd/d21/d35/d3b/d5d/fc7 0 2024-03-29T08:07:31.480 INFO:tasks.workunit.client.1.smithi148.stdout:7/900: mkdir d8/df3/d13a 0 2024-03-29T08:07:31.480 INFO:tasks.workunit.client.1.smithi148.stdout:9/968: mkdir db/d22/d3f/d13b 0 2024-03-29T08:07:31.484 INFO:tasks.workunit.client.1.smithi148.stdout:9/969: dread - db/d22/d26/d54/f60 zero size 2024-03-29T08:07:31.491 INFO:tasks.workunit.client.1.smithi148.stdout:1/915: rename d3/d18/d4d/d76/d85/d11d/d47/cc5 to d3/de/d21/c130 0 2024-03-29T08:07:31.493 INFO:tasks.workunit.client.1.smithi148.stdout:7/901: creat d8/df4/d111/f13b x:0 0 0 2024-03-29T08:07:31.494 INFO:tasks.workunit.client.1.smithi148.stdout:5/926: write d0/db/d7b/f3b [1968704,35257] 0 2024-03-29T08:07:31.494 INFO:tasks.workunit.client.1.smithi148.stdout:6/937: dread d6/d9/f23 [0,4194304] 0 2024-03-29T08:07:31.494 INFO:tasks.workunit.client.1.smithi148.stdout:0/934: rmdir db/de/df/d11/d1f/dc5/df5 0 2024-03-29T08:07:31.495 INFO:tasks.workunit.client.1.smithi148.stdout:9/970: write db/d10/d2d/d4e/dd4/f1b [3580796,44444] 0 2024-03-29T08:07:31.501 INFO:tasks.workunit.client.1.smithi148.stdout:9/971: write db/d10/d2d/d4e/d5f/dd5/f112 [281527,99902] 0 2024-03-29T08:07:31.503 INFO:tasks.workunit.client.1.smithi148.stdout:1/916: creat d3/f131 x:0 0 0 2024-03-29T08:07:31.504 INFO:tasks.workunit.client.0.smithi037.stdout:7/659: dwrite dd/ff [0,4194304] 0 2024-03-29T08:07:31.505 INFO:tasks.workunit.client.1.smithi148.stdout:0/935: truncate db/de/df/d14/f3c 343649 0 2024-03-29T08:07:31.510 INFO:tasks.workunit.client.0.smithi037.stdout:7/660: mknod dd/d21/d35/dcf/cde 0 2024-03-29T08:07:31.513 INFO:tasks.workunit.client.1.smithi148.stdout:0/936: symlink db/de/d8e/l11f 0 2024-03-29T08:07:31.513 INFO:tasks.workunit.client.1.smithi148.stdout:9/972: rename db/d22/d26/d54/d4d/d35/d4b/d5a/d74/d10a to db/d22/d26/d54/d4d/dbb/def/d13c 0 2024-03-29T08:07:31.515 INFO:tasks.workunit.client.0.smithi037.stdout:7/661: read dd/d1d/d49/fbc [663720,21101] 0 2024-03-29T08:07:31.517 INFO:tasks.workunit.client.1.smithi148.stdout:7/902: dwrite d8/d38/d6a/d91/fe2 [0,4194304] 0 2024-03-29T08:07:31.518 INFO:tasks.workunit.client.1.smithi148.stdout:6/938: dread d6/d9/d1e/d36/d4d/d9d/d28/fea [0,4194304] 0 2024-03-29T08:07:31.522 INFO:tasks.workunit.client.0.smithi037.stdout:7/662: getdents dd/d21/d35/d3b 0 2024-03-29T08:07:31.524 INFO:tasks.workunit.client.1.smithi148.stdout:8/911: dread d17/dce/de8/fed [0,4194304] 0 2024-03-29T08:07:31.524 INFO:tasks.workunit.client.1.smithi148.stdout:7/903: creat d8/dfb/d44/dc3/f13c x:0 0 0 2024-03-29T08:07:31.526 INFO:tasks.workunit.client.1.smithi148.stdout:9/973: write db/d22/d26/d54/d52/d9c/fb7 [758546,32437] 0 2024-03-29T08:07:31.527 INFO:tasks.workunit.client.0.smithi037.stdout:7/663: creat dd/d21/d35/dcf/fdf x:0 0 0 2024-03-29T08:07:31.527 INFO:tasks.workunit.client.1.smithi148.stdout:8/912: dread - d17/d49/d9a/dbe/d6f/d64/f10e zero size 2024-03-29T08:07:31.528 INFO:tasks.workunit.client.0.smithi037.stdout:7/664: chown dd/d21/d35 330636 1 2024-03-29T08:07:31.532 INFO:tasks.workunit.client.1.smithi148.stdout:0/937: link db/de/df/d11/d42/f6e db/dc/daa/d47/d4c/d5a/f120 0 2024-03-29T08:07:31.534 INFO:tasks.workunit.client.0.smithi037.stdout:7/665: mkdir dd/d21/d35/d3b/dce/de0 0 2024-03-29T08:07:31.534 INFO:tasks.workunit.client.1.smithi148.stdout:6/939: write d6/d9/d1e/d27/f90 [1189207,104308] 0 2024-03-29T08:07:31.535 INFO:tasks.workunit.client.1.smithi148.stdout:8/913: creat d17/d4e/f12d x:0 0 0 2024-03-29T08:07:31.536 INFO:tasks.workunit.client.0.smithi037.stdout:7/666: chown dd/d1d/d49/d81/cb7 31961100 1 2024-03-29T08:07:31.539 INFO:tasks.workunit.client.0.smithi037.stdout:7/667: read dd/d21/d70/fb2 [1923633,86889] 0 2024-03-29T08:07:31.539 INFO:tasks.workunit.client.1.smithi148.stdout:8/914: mkdir d17/d4e/d107/d12e 0 2024-03-29T08:07:31.539 INFO:tasks.workunit.client.1.smithi148.stdout:6/940: stat d6/d9/d1e/d36/d4d/d9d/d28/d98/dc2/f10a 0 2024-03-29T08:07:31.540 INFO:tasks.workunit.client.0.smithi037.stdout:7/668: stat dd/d21/d35 0 2024-03-29T08:07:31.542 INFO:tasks.workunit.client.1.smithi148.stdout:0/938: getdents db/dc/daa/d47/d7c 0 2024-03-29T08:07:31.543 INFO:tasks.workunit.client.1.smithi148.stdout:6/941: unlink d6/fd 0 2024-03-29T08:07:31.548 INFO:tasks.workunit.client.1.smithi148.stdout:0/939: write db/de/fc3 [3290032,47472] 0 2024-03-29T08:07:31.548 INFO:tasks.workunit.client.1.smithi148.stdout:7/904: dread d8/d38/d3c/da5/f121 [0,4194304] 0 2024-03-29T08:07:31.550 INFO:tasks.workunit.client.0.smithi037.stdout:7/669: write dd/d1d/d34/d54/f87 [1878317,108704] 0 2024-03-29T08:07:31.552 INFO:tasks.workunit.client.0.smithi037.stdout:7/670: stat dd/d68/laa 0 2024-03-29T08:07:31.553 INFO:tasks.workunit.client.1.smithi148.stdout:0/940: chown db/de/df/d2b/dae/d116/f11e 4849 1 2024-03-29T08:07:31.554 INFO:tasks.workunit.client.1.smithi148.stdout:6/942: read d6/d9/d1e/d36/d4d/d9d/d28/dda/f92 [200788,57883] 0 2024-03-29T08:07:31.555 INFO:tasks.workunit.client.0.smithi037.stdout:7/671: unlink dd/c29 0 2024-03-29T08:07:31.556 INFO:tasks.workunit.client.1.smithi148.stdout:9/974: dwrite db/d22/d26/d54/d4d/d35/d4b/dc3/ff8 [0,4194304] 0 2024-03-29T08:07:31.558 INFO:tasks.workunit.client.0.smithi037.stdout:7/672: mknod dd/d72/ce1 0 2024-03-29T08:07:31.560 INFO:tasks.workunit.client.0.smithi037.stdout:7/673: unlink dd/d21/l4b 0 2024-03-29T08:07:31.563 INFO:tasks.workunit.client.1.smithi148.stdout:7/905: link d8/dfb/d11b/d37/c97 d8/df4/d111/c13d 0 2024-03-29T08:07:31.564 INFO:tasks.workunit.client.0.smithi037.stdout:7/674: unlink dd/d21/d35/d3b/d5d/l96 0 2024-03-29T08:07:31.566 INFO:tasks.workunit.client.1.smithi148.stdout:6/943: chown d6/d9/d1e/d20/d30/fcb 17 1 2024-03-29T08:07:31.568 INFO:tasks.workunit.client.0.smithi037.stdout:7/675: mkdir dd/d21/d35/d42/ddb/de2 0 2024-03-29T08:07:31.569 INFO:tasks.workunit.client.0.smithi037.stdout:7/676: chown dd/la5 1408122 1 2024-03-29T08:07:31.571 INFO:tasks.workunit.client.1.smithi148.stdout:8/915: dwrite d17/d49/d9a/dbe/fcb [0,4194304] 0 2024-03-29T08:07:31.575 INFO:tasks.workunit.client.1.smithi148.stdout:6/944: fdatasync d6/d9/d1e/d36/db1/f76 0 2024-03-29T08:07:31.575 INFO:tasks.workunit.client.0.smithi037.stdout:7/677: creat dd/fe3 x:0 0 0 2024-03-29T08:07:31.576 INFO:tasks.workunit.client.1.smithi148.stdout:6/945: chown d6/fe 38516287 1 2024-03-29T08:07:31.580 INFO:tasks.workunit.client.1.smithi148.stdout:7/906: rename d8/d38/d6a/c92 to d8/dfb/d44/d50/d85/c13e 0 2024-03-29T08:07:31.581 INFO:tasks.workunit.client.1.smithi148.stdout:9/975: dwrite db/f43 [0,4194304] 0 2024-03-29T08:07:31.581 INFO:tasks.workunit.client.1.smithi148.stdout:7/907: write d8/f134 [564370,130041] 0 2024-03-29T08:07:31.583 INFO:tasks.workunit.client.0.smithi037.stdout:7/678: creat dd/d21/d35/fe4 x:0 0 0 2024-03-29T08:07:31.586 INFO:tasks.workunit.client.1.smithi148.stdout:3/969: sync 2024-03-29T08:07:31.586 INFO:tasks.workunit.client.1.smithi148.stdout:1/917: sync 2024-03-29T08:07:31.586 INFO:tasks.workunit.client.1.smithi148.stdout:4/970: sync 2024-03-29T08:07:31.586 INFO:tasks.workunit.client.1.smithi148.stdout:5/927: sync 2024-03-29T08:07:31.586 INFO:tasks.workunit.client.1.smithi148.stdout:8/916: rename d17/d49/d9a/dbe/d6f/d64/cd6 to d17/dce/de8/c12f 0 2024-03-29T08:07:31.586 INFO:tasks.workunit.client.1.smithi148.stdout:7/908: read - d8/d38/d3c/da5/db4/de9/d114/ffd zero size 2024-03-29T08:07:31.586 INFO:tasks.workunit.client.0.smithi037.stdout:7/679: write dd/d21/f36 [1607328,84348] 0 2024-03-29T08:07:31.588 INFO:tasks.workunit.client.1.smithi148.stdout:9/976: read - db/d10/d2d/d4e/d5f/d6e/db0/fc4 zero size 2024-03-29T08:07:31.589 INFO:tasks.workunit.client.0.smithi037.stdout:7/680: symlink dd/d21/d35/d60/d94/le5 0 2024-03-29T08:07:31.592 INFO:tasks.workunit.client.0.smithi037.stdout:7/681: write dd/d21/d35/d3b/d5d/fc7 [795697,71593] 0 2024-03-29T08:07:31.593 INFO:tasks.workunit.client.1.smithi148.stdout:8/917: fdatasync d17/d18/f1c 0 2024-03-29T08:07:31.594 INFO:tasks.workunit.client.1.smithi148.stdout:7/909: creat d8/df3/d13a/f13f x:0 0 0 2024-03-29T08:07:31.596 INFO:tasks.workunit.client.1.smithi148.stdout:0/941: dread db/f51 [0,4194304] 0 2024-03-29T08:07:31.598 INFO:tasks.workunit.client.1.smithi148.stdout:1/918: chown d3/d18/d4d/f63 3368 1 2024-03-29T08:07:31.602 INFO:tasks.workunit.client.1.smithi148.stdout:5/928: write d0/db/d1e/d35/d42/d67/d76/d81/d98/f111 [2574717,97163] 0 2024-03-29T08:07:31.602 INFO:tasks.workunit.client.1.smithi148.stdout:7/910: mknod d8/d14/da8/c140 0 2024-03-29T08:07:31.604 INFO:tasks.workunit.client.1.smithi148.stdout:4/971: link df/d2c/d51/d77/d17/l7e df/d2c/d10b/d125/l134 0 2024-03-29T08:07:31.604 INFO:tasks.workunit.client.1.smithi148.stdout:9/977: chown db/d22/d26/d54/l80 6 1 2024-03-29T08:07:31.605 INFO:tasks.workunit.client.1.smithi148.stdout:1/919: chown d3/de/d25/d51/ld2 13319791 1 2024-03-29T08:07:31.610 INFO:tasks.workunit.client.1.smithi148.stdout:5/929: fsync d0/db/d1e/fbd 0 2024-03-29T08:07:31.611 INFO:tasks.workunit.client.1.smithi148.stdout:6/946: link d6/d9/d1e/d36/d4d/d9d/d28/dda/db6/df3/dd3/f11f d6/d9/d1e/d20/f128 0 2024-03-29T08:07:31.613 INFO:tasks.workunit.client.0.smithi037.stdout:9/854: sync 2024-03-29T08:07:31.613 INFO:tasks.workunit.client.0.smithi037.stdout:2/499: sync 2024-03-29T08:07:31.613 INFO:tasks.workunit.client.1.smithi148.stdout:5/930: stat d0/db/lc 0 2024-03-29T08:07:31.614 INFO:tasks.workunit.client.0.smithi037.stdout:7/682: write dd/d1d/d34/f3f [1814454,90165] 0 2024-03-29T08:07:31.616 INFO:tasks.workunit.client.0.smithi037.stdout:7/683: fsync dd/fe3 0 2024-03-29T08:07:31.616 INFO:tasks.workunit.client.1.smithi148.stdout:0/942: getdents db/de/df/d14/db0 0 2024-03-29T08:07:31.616 INFO:tasks.workunit.client.1.smithi148.stdout:9/978: unlink db/fa7 0 2024-03-29T08:07:31.619 INFO:tasks.workunit.client.1.smithi148.stdout:8/918: link d17/d1d/d7e/ce9 d17/d18/c130 0 2024-03-29T08:07:31.619 INFO:tasks.workunit.client.1.smithi148.stdout:1/920: symlink d3/d18/d4d/d76/d85/d11d/d97/d111/d125/l132 0 2024-03-29T08:07:31.619 INFO:tasks.workunit.client.1.smithi148.stdout:6/947: chown d6/d9/d1e/d27/cbd 0 1 2024-03-29T08:07:31.620 INFO:tasks.workunit.client.0.smithi037.stdout:2/500: creat d0/d20/d2e/d38/d68/fac x:0 0 0 2024-03-29T08:07:31.620 INFO:tasks.workunit.client.0.smithi037.stdout:7/684: truncate dd/d21/d35/d3b/fb8 655235 0 2024-03-29T08:07:31.622 INFO:tasks.workunit.client.1.smithi148.stdout:0/943: stat db/de/df/d11/d2d/d3e/c50 0 2024-03-29T08:07:31.622 INFO:tasks.workunit.client.1.smithi148.stdout:4/972: truncate df/d2c/d51/d77/d26/d34/d8a/d98/fb0 5809843 0 2024-03-29T08:07:31.623 INFO:tasks.workunit.client.0.smithi037.stdout:2/501: fdatasync d0/d20/d2e/f36 0 2024-03-29T08:07:31.624 INFO:tasks.workunit.client.0.smithi037.stdout:2/502: readlink d0/l28 0 2024-03-29T08:07:31.625 INFO:tasks.workunit.client.1.smithi148.stdout:8/919: chown d17/d18/d26/d4f/d94/cc3 653 1 2024-03-29T08:07:31.626 INFO:tasks.workunit.client.1.smithi148.stdout:6/948: read d6/d9/d1e/fbb [1586276,125451] 0 2024-03-29T08:07:31.626 INFO:tasks.workunit.client.1.smithi148.stdout:9/979: mkdir db/d22/d26/d54/d4d/d35/d4b/d8e/ddf/d13d 0 2024-03-29T08:07:31.627 INFO:tasks.workunit.client.0.smithi037.stdout:7/685: rename dd/d21/d35/dcf/fdf to dd/d1d/d34/dc8/fe6 0 2024-03-29T08:07:31.627 INFO:tasks.workunit.client.1.smithi148.stdout:4/973: fdatasync df/d60/f110 0 2024-03-29T08:07:31.627 INFO:tasks.workunit.client.0.smithi037.stdout:2/503: write d0/f21 [377996,80250] 0 2024-03-29T08:07:31.628 INFO:tasks.workunit.client.1.smithi148.stdout:1/921: mknod d3/d18/d4d/d76/c133 0 2024-03-29T08:07:31.632 INFO:tasks.workunit.client.1.smithi148.stdout:7/911: dwrite d8/dfb/db8/de0/ff2 [0,4194304] 0 2024-03-29T08:07:31.632 INFO:tasks.workunit.client.1.smithi148.stdout:8/920: fdatasync d17/d74/fb4 0 2024-03-29T08:07:31.634 INFO:tasks.workunit.client.1.smithi148.stdout:9/980: chown db/d10/d2d/d4e/f6b 6478 1 2024-03-29T08:07:31.634 INFO:tasks.workunit.client.0.smithi037.stdout:9/855: dwrite d1/d4/d19/d1e/d71/d116/f120 [0,4194304] 0 2024-03-29T08:07:31.638 INFO:tasks.workunit.client.0.smithi037.stdout:9/856: chown d1/d4/cc9 114221007 1 2024-03-29T08:07:31.638 INFO:tasks.workunit.client.0.smithi037.stdout:2/504: fsync d0/d23/f65 0 2024-03-29T08:07:31.639 INFO:tasks.workunit.client.0.smithi037.stdout:7/686: truncate dd/d1d/f53 3312278 0 2024-03-29T08:07:31.639 INFO:tasks.workunit.client.1.smithi148.stdout:3/970: dread de/d24/d33/d39/d46/d30/d37/faa [0,4194304] 0 2024-03-29T08:07:31.644 INFO:tasks.workunit.client.1.smithi148.stdout:5/931: truncate d0/db/d1e/d36/d7c/f80 1150657 0 2024-03-29T08:07:31.644 INFO:tasks.workunit.client.1.smithi148.stdout:6/949: creat d6/d9/d1e/d27/d48/d73/ddd/f129 x:0 0 0 2024-03-29T08:07:31.644 INFO:tasks.workunit.client.1.smithi148.stdout:9/981: mknod db/d10/d9d/c13e 0 2024-03-29T08:07:31.645 INFO:tasks.workunit.client.1.smithi148.stdout:7/912: chown d8/dfb/c132 119 1 2024-03-29T08:07:31.648 INFO:tasks.workunit.client.1.smithi148.stdout:1/922: unlink d3/de/d21/cdd 0 2024-03-29T08:07:31.648 INFO:tasks.workunit.client.0.smithi037.stdout:2/505: rmdir d0/d23/d70/d71/d78 39 2024-03-29T08:07:31.649 INFO:tasks.workunit.client.0.smithi037.stdout:7/687: mknod dd/d21/d89/ce7 0 2024-03-29T08:07:31.650 INFO:tasks.workunit.client.1.smithi148.stdout:4/974: rename fa to df/d2c/d80/daa/dbc/dc9/f135 0 2024-03-29T08:07:31.651 INFO:tasks.workunit.client.1.smithi148.stdout:1/923: fsync d3/de/d25/df1/f10c 0 2024-03-29T08:07:31.652 INFO:tasks.workunit.client.1.smithi148.stdout:7/913: fsync d8/d38/d3c/da5/db4/fc6 0 2024-03-29T08:07:31.652 INFO:tasks.workunit.client.0.smithi037.stdout:2/506: write d0/d20/d2e/d5f/d6f/f97 [76878,130847] 0 2024-03-29T08:07:31.653 INFO:tasks.workunit.client.1.smithi148.stdout:4/975: dread - df/d2c/d80/daa/f124 zero size 2024-03-29T08:07:31.656 INFO:tasks.workunit.client.0.smithi037.stdout:2/507: write d0/d20/d7d/f80 [536286,50905] 0 2024-03-29T08:07:31.657 INFO:tasks.workunit.client.1.smithi148.stdout:7/914: write d8/dfb/db8/d115/f118 [90109,63031] 0 2024-03-29T08:07:31.658 INFO:tasks.workunit.client.1.smithi148.stdout:3/971: creat de/d24/d33/d39/d46/d70/d101/f144 x:0 0 0 2024-03-29T08:07:31.659 INFO:tasks.workunit.client.0.smithi037.stdout:7/688: dread dd/d21/d35/d60/f6e [0,4194304] 0 2024-03-29T08:07:31.661 INFO:tasks.workunit.client.1.smithi148.stdout:6/950: symlink d6/d9/d1e/d36/d4d/l12a 0 2024-03-29T08:07:31.663 INFO:tasks.workunit.client.0.smithi037.stdout:2/508: symlink d0/d23/d70/d71/lad 0 2024-03-29T08:07:31.668 INFO:tasks.workunit.client.1.smithi148.stdout:7/915: mknod d8/df4/d111/c141 0 2024-03-29T08:07:31.668 INFO:tasks.workunit.client.0.smithi037.stdout:2/509: write d0/d20/d2e/d5f/d6f/d5e/f86 [3124378,70116] 0 2024-03-29T08:07:31.673 INFO:tasks.workunit.client.1.smithi148.stdout:8/921: write d17/d18/d26/d63/d7d/f115 [538394,17241] 0 2024-03-29T08:07:31.674 INFO:tasks.workunit.client.1.smithi148.stdout:0/944: dwrite db/dc/daa/d47/d4c/d5a/d7a/ff7 [0,4194304] 0 2024-03-29T08:07:31.675 INFO:tasks.workunit.client.1.smithi148.stdout:4/976: mkdir df/d2c/d51/d77/d26/d34/d43/df6/d136 0 2024-03-29T08:07:31.677 INFO:tasks.workunit.client.1.smithi148.stdout:7/916: dread - d8/dfb/f108 zero size 2024-03-29T08:07:31.680 INFO:tasks.workunit.client.1.smithi148.stdout:5/932: dwrite d0/d46/d9e/d114/f118 [0,4194304] 0 2024-03-29T08:07:31.681 INFO:tasks.workunit.client.0.smithi037.stdout:9/857: dread d1/d4/d19/f64 [0,4194304] 0 2024-03-29T08:07:31.683 INFO:tasks.workunit.client.1.smithi148.stdout:4/977: read df/d2c/d51/d77/f56 [2175388,41203] 0 2024-03-29T08:07:31.687 INFO:tasks.workunit.client.0.smithi037.stdout:7/689: dwrite dd/f7d [0,4194304] 0 2024-03-29T08:07:31.688 INFO:tasks.workunit.client.1.smithi148.stdout:9/982: dwrite db/d10/d2d/d4e/da6/f11a [0,4194304] 0 2024-03-29T08:07:31.692 INFO:tasks.workunit.client.0.smithi037.stdout:7/690: truncate dd/d1d/d34/dc8/fe6 209650 0 2024-03-29T08:07:31.692 INFO:tasks.workunit.client.1.smithi148.stdout:8/922: read d17/d18/d26/d63/d66/d5f/f8e [234864,12577] 0 2024-03-29T08:07:31.696 INFO:tasks.workunit.client.1.smithi148.stdout:6/951: dread d6/d9/d1e/f8a [0,4194304] 0 2024-03-29T08:07:31.699 INFO:tasks.workunit.client.0.smithi037.stdout:7/691: mknod dd/d21/d35/d3b/dce/ce8 0 2024-03-29T08:07:31.703 INFO:tasks.workunit.client.0.smithi037.stdout:7/692: write dd/d21/d35/d42/d5c/f5e [838099,66399] 0 2024-03-29T08:07:31.704 INFO:tasks.workunit.client.1.smithi148.stdout:4/978: read df/d2c/f2e [1852927,61994] 0 2024-03-29T08:07:31.708 INFO:tasks.workunit.client.0.smithi037.stdout:9/858: dread d1/d4/d19/d1e/d4a/d6f/da2/d13/d7c/fc3 [0,4194304] 0 2024-03-29T08:07:31.717 INFO:tasks.workunit.client.0.smithi037.stdout:2/510: dread d0/d20/d22/d55/f5b [0,4194304] 0 2024-03-29T08:07:31.718 INFO:tasks.workunit.client.0.smithi037.stdout:9/859: read d1/d4/d19/d1e/d25/fc1 [708572,111836] 0 2024-03-29T08:07:31.718 INFO:tasks.workunit.client.1.smithi148.stdout:8/923: dread d17/d49/d9a/dbe/d6f/fb9 [0,4194304] 0 2024-03-29T08:07:31.723 INFO:tasks.workunit.client.0.smithi037.stdout:2/511: truncate d0/f15 856086 0 2024-03-29T08:07:31.723 INFO:tasks.workunit.client.1.smithi148.stdout:4/979: truncate df/d2c/d51/d77/d26/d34/d3e/dd8/f129 1196641 0 2024-03-29T08:07:31.723 INFO:tasks.workunit.client.0.smithi037.stdout:9/860: mkdir d1/d4/d19/d1e/d4a/d6f/da2/d13/de0/d122 0 2024-03-29T08:07:31.724 INFO:tasks.workunit.client.1.smithi148.stdout:6/952: link d6/d9/d1e/d20/d6b/fe3 d6/d9/d68/f12b 0 2024-03-29T08:07:31.727 INFO:tasks.workunit.client.0.smithi037.stdout:9/861: fdatasync d1/d4/d19/d1e/d71/ffa 0 2024-03-29T08:07:31.730 INFO:tasks.workunit.client.0.smithi037.stdout:2/512: getdents d0/d23/d70 0 2024-03-29T08:07:31.732 INFO:tasks.workunit.client.0.smithi037.stdout:2/513: fdatasync d0/d23/d70/f76 0 2024-03-29T08:07:31.735 INFO:tasks.workunit.client.1.smithi148.stdout:4/980: chown df/d2c/d51/d77/d17/d73/fc0 193532085 1 2024-03-29T08:07:31.735 INFO:tasks.workunit.client.1.smithi148.stdout:9/983: dwrite db/d22/d26/d54/f60 [0,4194304] 0 2024-03-29T08:07:31.735 INFO:tasks.workunit.client.1.smithi148.stdout:5/933: dwrite d0/db/d7b/fe3 [0,4194304] 0 2024-03-29T08:07:31.735 INFO:tasks.workunit.client.0.smithi037.stdout:2/514: chown d0/d23/f65 120 1 2024-03-29T08:07:31.735 INFO:tasks.workunit.client.0.smithi037.stdout:7/693: dwrite dd/f77 [0,4194304] 0 2024-03-29T08:07:31.737 INFO:tasks.workunit.client.0.smithi037.stdout:7/694: readlink dd/l47 0 2024-03-29T08:07:31.737 INFO:tasks.workunit.client.0.smithi037.stdout:2/515: dread - d0/d20/d2e/fa7 zero size 2024-03-29T08:07:31.738 INFO:tasks.workunit.client.1.smithi148.stdout:6/953: truncate d6/d9/d1e/d20/f128 261234 0 2024-03-29T08:07:31.744 INFO:tasks.workunit.client.1.smithi148.stdout:0/945: dread db/de/fc3 [0,4194304] 0 2024-03-29T08:07:31.744 INFO:tasks.workunit.client.1.smithi148.stdout:3/972: dread de/d24/d33/d39/d46/d109/f12e [0,4194304] 0 2024-03-29T08:07:31.746 INFO:tasks.workunit.client.1.smithi148.stdout:9/984: write db/f43 [37082,102140] 0 2024-03-29T08:07:31.748 INFO:tasks.workunit.client.1.smithi148.stdout:8/924: truncate d17/d18/ddf/fe1 3669162 0 2024-03-29T08:07:31.750 INFO:tasks.workunit.client.1.smithi148.stdout:4/981: creat df/d2c/d51/d77/d26/f137 x:0 0 0 2024-03-29T08:07:31.753 INFO:tasks.workunit.client.1.smithi148.stdout:5/934: mkdir d0/db/d1e/d35/d42/d67/d76/d81/d98/dd5/d9c/dc8/d128 0 2024-03-29T08:07:31.755 INFO:tasks.workunit.client.1.smithi148.stdout:9/985: read db/d10/d2d/f68 [3411602,84960] 0 2024-03-29T08:07:31.761 INFO:tasks.workunit.client.1.smithi148.stdout:8/925: rename d17/d18/d26/d63/d7d/f101 to d17/d49/f131 0 2024-03-29T08:07:31.761 INFO:tasks.workunit.client.1.smithi148.stdout:0/946: rename db/de/df/d11/d2d/fc0 to db/de/df/d11/d2d/f121 0 2024-03-29T08:07:31.761 INFO:tasks.workunit.client.0.smithi037.stdout:9/862: write d1/d4/d19/d1e/d4a/d6f/da2/d13/d7c/fb4 [169452,28938] 0 2024-03-29T08:07:31.761 INFO:tasks.workunit.client.1.smithi148.stdout:3/973: getdents de/d3b 0 2024-03-29T08:07:31.763 INFO:tasks.workunit.client.1.smithi148.stdout:4/982: getdents df/d2c 0 2024-03-29T08:07:31.764 INFO:tasks.workunit.client.0.smithi037.stdout:9/863: fdatasync d1/d4/d19/d1e/d4a/d6f/da2/d31/daa/dba/de3/f104 0 2024-03-29T08:07:31.768 INFO:tasks.workunit.client.0.smithi037.stdout:9/864: fsync d1/d4/d19/d1e/d4a/f53 0 2024-03-29T08:07:31.769 INFO:tasks.workunit.client.1.smithi148.stdout:4/983: chown df/f1e 26 1 2024-03-29T08:07:31.772 INFO:tasks.workunit.client.1.smithi148.stdout:8/926: creat d17/d49/d9a/db8/f132 x:0 0 0 2024-03-29T08:07:31.774 INFO:tasks.workunit.client.0.smithi037.stdout:9/865: rename d1/d4/d19/d1e/d4a/d6f/da2/d13/c9d to d1/d4/d19/d1e/d71/d89/da6/dd8/c123 0 2024-03-29T08:07:31.776 INFO:tasks.workunit.client.1.smithi148.stdout:4/984: stat df/d2c/d51/d77/d26/d9b/fa8 0 2024-03-29T08:07:31.777 INFO:tasks.workunit.client.0.smithi037.stdout:7/695: truncate dd/d21/d35/d42/d5c/f5e 150356 0 2024-03-29T08:07:31.778 INFO:tasks.workunit.client.0.smithi037.stdout:7/696: stat dd/d21/d89 0 2024-03-29T08:07:31.778 INFO:tasks.workunit.client.0.smithi037.stdout:9/866: unlink d1/f74 0 2024-03-29T08:07:31.779 INFO:tasks.workunit.client.1.smithi148.stdout:9/986: getdents db/d22/d26/d54/d52/dab 0 2024-03-29T08:07:31.779 INFO:tasks.workunit.client.1.smithi148.stdout:8/927: fdatasync d17/d49/d9a/dbe/fcb 0 2024-03-29T08:07:31.779 INFO:tasks.workunit.client.1.smithi148.stdout:5/935: link d0/db/d7b/d16/d2e/ddd/de9/f102 d0/db/d1e/d35/d42/d67/f129 0 2024-03-29T08:07:31.779 INFO:tasks.workunit.client.0.smithi037.stdout:7/697: fsync dd/d21/f36 0 2024-03-29T08:07:31.779 INFO:tasks.workunit.client.1.smithi148.stdout:6/954: dwrite d6/d9/d1e/d36/d4d/d9d/d28/dda/db4/ff2 [0,4194304] 0 2024-03-29T08:07:31.780 INFO:tasks.workunit.client.0.smithi037.stdout:7/698: fdatasync dd/f1a 0 2024-03-29T08:07:31.782 INFO:tasks.workunit.client.0.smithi037.stdout:2/516: truncate d0/d20/d22/d55/f5b 4340126 0 2024-03-29T08:07:31.783 INFO:tasks.workunit.client.0.smithi037.stdout:7/699: mknod dd/d21/d65/da2/ce9 0 2024-03-29T08:07:31.785 INFO:tasks.workunit.client.1.smithi148.stdout:8/928: chown d17/d49/d9a/dbe/d6f/d64/ca3 1 1 2024-03-29T08:07:31.786 INFO:tasks.workunit.client.0.smithi037.stdout:2/517: unlink d0/d20/d22/l43 0 2024-03-29T08:07:31.787 INFO:tasks.workunit.client.0.smithi037.stdout:7/700: creat dd/d21/d85/fea x:0 0 0 2024-03-29T08:07:31.796 INFO:tasks.workunit.client.1.smithi148.stdout:4/985: creat df/d2c/d51/d77/d26/d34/d3e/f138 x:0 0 0 2024-03-29T08:07:31.796 INFO:tasks.workunit.client.1.smithi148.stdout:6/955: creat d6/d9/d1e/d36/d4d/d9d/d28/dda/db6/df3/f12c x:0 0 0 2024-03-29T08:07:31.802 INFO:tasks.workunit.client.1.smithi148.stdout:5/936: creat d0/db/d1e/d35/d42/d67/d76/d81/d98/de5/f12a x:0 0 0 2024-03-29T08:07:31.802 INFO:tasks.workunit.client.1.smithi148.stdout:8/929: mkdir d17/d18/d26/d63/d66/d5f/dd0/d119/d122/d133 0 2024-03-29T08:07:31.803 INFO:tasks.workunit.client.1.smithi148.stdout:0/947: dwrite db/de/f70 [4194304,4194304] 0 2024-03-29T08:07:31.805 INFO:tasks.workunit.client.0.smithi037.stdout:7/701: dwrite dd/d72/fae [0,4194304] 0 2024-03-29T08:07:31.806 INFO:tasks.workunit.client.1.smithi148.stdout:4/986: write df/d2c/f46 [4274768,121443] 0 2024-03-29T08:07:31.809 INFO:tasks.workunit.client.0.smithi037.stdout:7/702: read dd/f48 [3534949,33942] 0 2024-03-29T08:07:31.811 INFO:tasks.workunit.client.1.smithi148.stdout:4/987: readlink df/l118 0 2024-03-29T08:07:31.812 INFO:tasks.workunit.client.1.smithi148.stdout:3/974: dwrite de/d24/d33/d39/d46/d30/d37/d79/fc0 [0,4194304] 0 2024-03-29T08:07:31.814 INFO:tasks.workunit.client.1.smithi148.stdout:6/956: unlink d6/d9/d1e/d36/d4d/d9d/d28/db3/dd2/fef 0 2024-03-29T08:07:31.814 INFO:tasks.workunit.client.1.smithi148.stdout:0/948: chown db/dc/d78/l91 11 1 2024-03-29T08:07:31.814 INFO:tasks.workunit.client.0.smithi037.stdout:7/703: rmdir dd/d1d/d49/d81/d84/dca 39 2024-03-29T08:07:31.816 INFO:tasks.workunit.client.1.smithi148.stdout:9/987: dwrite db/d22/d26/d54/fc9 [0,4194304] 0 2024-03-29T08:07:31.819 INFO:tasks.workunit.client.1.smithi148.stdout:3/975: chown de/d24/d33/d39/da3/ff6 2136 1 2024-03-29T08:07:31.822 INFO:tasks.workunit.client.1.smithi148.stdout:4/988: unlink df/d2c/d51/d77/d26/f3d 0 2024-03-29T08:07:31.824 INFO:tasks.workunit.client.0.smithi037.stdout:7/704: rename dd/d21/d35/d60/fbf to dd/d1d/d34/d54/feb 0 2024-03-29T08:07:31.826 INFO:tasks.workunit.client.1.smithi148.stdout:6/957: mkdir d6/d9/d1e/d36/d124/d12d 0 2024-03-29T08:07:31.829 INFO:tasks.workunit.client.1.smithi148.stdout:0/949: rmdir db 39 2024-03-29T08:07:31.829 INFO:tasks.workunit.client.0.smithi037.stdout:2/518: dwrite d0/d20/d2e/d5f/d6f/d5e/f60 [0,4194304] 0 2024-03-29T08:07:31.831 INFO:tasks.workunit.client.1.smithi148.stdout:9/988: creat db/d22/d26/d54/d4d/d35/f13f x:0 0 0 2024-03-29T08:07:31.833 INFO:tasks.workunit.client.1.smithi148.stdout:4/989: symlink df/l139 0 2024-03-29T08:07:31.838 INFO:tasks.workunit.client.0.smithi037.stdout:1/664: sync 2024-03-29T08:07:31.838 INFO:tasks.workunit.client.0.smithi037.stdout:5/753: sync 2024-03-29T08:07:31.839 INFO:tasks.workunit.client.1.smithi148.stdout:9/989: creat db/d10/d2d/d4e/d5f/dd5/f140 x:0 0 0 2024-03-29T08:07:31.840 INFO:tasks.workunit.client.1.smithi148.stdout:6/958: creat d6/d9/d1e/d36/d4d/d9d/d28/dda/db6/df3/df4/f12e x:0 0 0 2024-03-29T08:07:31.842 INFO:tasks.workunit.client.0.smithi037.stdout:2/519: getdents d0/d20/d2e/d5f/d6f 0 2024-03-29T08:07:31.842 INFO:tasks.workunit.client.1.smithi148.stdout:5/937: dwrite d0/db/d1e/fa7 [0,4194304] 0 2024-03-29T08:07:31.842 INFO:tasks.workunit.client.0.smithi037.stdout:1/665: stat d2/d6/dc/d18/d22/d4f/d7c/f7f 0 2024-03-29T08:07:31.847 INFO:tasks.workunit.client.1.smithi148.stdout:5/938: read d0/db/d7b/fe3 [3676239,51218] 0 2024-03-29T08:07:31.848 INFO:tasks.workunit.client.1.smithi148.stdout:6/959: symlink d6/d9/d68/l12f 0 2024-03-29T08:07:31.857 INFO:tasks.workunit.client.0.smithi037.stdout:5/754: dread d8/d22/d4f/d70/f54 [0,4194304] 0 2024-03-29T08:07:31.857 INFO:tasks.workunit.client.0.smithi037.stdout:2/520: symlink d0/d20/d2e/da4/lae 0 2024-03-29T08:07:31.860 INFO:tasks.workunit.client.1.smithi148.stdout:9/990: rename db/d22/d26/d54/d4d/d35/f13f to db/d22/d26/d54/d52/dab/f141 0 2024-03-29T08:07:31.861 INFO:tasks.workunit.client.1.smithi148.stdout:0/950: mkdir db/de/df/d14/dea/d122 0 2024-03-29T08:07:31.869 INFO:tasks.workunit.client.1.smithi148.stdout:3/976: dread de/d24/d33/d39/da3/fcd [0,4194304] 0 2024-03-29T08:07:31.869 INFO:tasks.workunit.client.0.smithi037.stdout:1/666: dwrite d2/d40/fbc [0,4194304] 0 2024-03-29T08:07:31.870 INFO:tasks.workunit.client.1.smithi148.stdout:9/991: mknod db/d10/d6d/d73/da8/deb/dec/c142 0 2024-03-29T08:07:31.870 INFO:tasks.workunit.client.0.smithi037.stdout:5/755: creat d8/d22/db6/f107 x:0 0 0 2024-03-29T08:07:31.870 INFO:tasks.workunit.client.1.smithi148.stdout:0/951: mknod db/de/df/d11/d2d/da3/dee/c123 0 2024-03-29T08:07:31.872 INFO:tasks.workunit.client.0.smithi037.stdout:5/756: stat d8/c10 0 2024-03-29T08:07:31.873 INFO:tasks.workunit.client.0.smithi037.stdout:2/521: mkdir d0/d23/daf 0 2024-03-29T08:07:31.874 INFO:tasks.workunit.client.1.smithi148.stdout:9/992: chown db/d10/d2d/d4e/dd4/f70 23 1 2024-03-29T08:07:31.875 INFO:tasks.workunit.client.0.smithi037.stdout:1/667: dread - d2/d6/dc/d7e/f91 zero size 2024-03-29T08:07:31.875 INFO:tasks.workunit.client.1.smithi148.stdout:0/952: mkdir db/dc/d78/d124 0 2024-03-29T08:07:31.876 INFO:tasks.workunit.client.0.smithi037.stdout:5/757: dread - d8/f4c zero size 2024-03-29T08:07:31.878 INFO:tasks.workunit.client.0.smithi037.stdout:2/522: mknod d0/d20/d22/d55/cb0 0 2024-03-29T08:07:31.879 INFO:tasks.workunit.client.1.smithi148.stdout:1/924: sync 2024-03-29T08:07:31.880 INFO:tasks.workunit.client.0.smithi037.stdout:1/668: write d2/d6/dc/fe4 [956722,103101] 0 2024-03-29T08:07:31.880 INFO:tasks.workunit.client.1.smithi148.stdout:0/953: mkdir db/de/df/d2b/dae/d116/d125 0 2024-03-29T08:07:31.882 INFO:tasks.workunit.client.1.smithi148.stdout:4/990: truncate df/d2c/d51/d77/d26/d34/f5c 3292640 0 2024-03-29T08:07:31.885 INFO:tasks.workunit.client.1.smithi148.stdout:0/954: read - db/de/df/d14/dea/f10a zero size 2024-03-29T08:07:31.885 INFO:tasks.workunit.client.0.smithi037.stdout:7/705: dread dd/d1d/d34/f37 [4194304,4194304] 0 2024-03-29T08:07:31.885 INFO:tasks.workunit.client.0.smithi037.stdout:2/523: creat d0/d20/d2e/d5f/d6f/d5e/fb1 x:0 0 0 2024-03-29T08:07:31.886 INFO:tasks.workunit.client.0.smithi037.stdout:1/669: fsync d2/d6/dc/d58/fa2 0 2024-03-29T08:07:31.888 INFO:tasks.workunit.client.0.smithi037.stdout:1/670: chown d2/c9 13742032 1 2024-03-29T08:07:31.888 INFO:tasks.workunit.client.0.smithi037.stdout:2/524: write d0/d53/f88 [5192985,126545] 0 2024-03-29T08:07:31.890 INFO:tasks.workunit.client.1.smithi148.stdout:1/925: rmdir d3/d18/d4d/d76/d85/d11d/d47/d6d/d72 39 2024-03-29T08:07:31.890 INFO:tasks.workunit.client.1.smithi148.stdout:6/960: dread d6/d9/d1e/d27/f6e [0,4194304] 0 2024-03-29T08:07:31.891 INFO:tasks.workunit.client.0.smithi037.stdout:5/758: link d8/d22/d4f/d70/d79/f92 d8/d22/d37/de8/f108 0 2024-03-29T08:07:31.893 INFO:tasks.workunit.client.0.smithi037.stdout:7/706: creat dd/d21/fec x:0 0 0 2024-03-29T08:07:31.894 INFO:tasks.workunit.client.0.smithi037.stdout:5/759: mkdir d8/d4d/d109 0 2024-03-29T08:07:31.895 INFO:tasks.workunit.client.0.smithi037.stdout:2/525: symlink d0/d20/d22/d4e/lb2 0 2024-03-29T08:07:31.896 INFO:tasks.workunit.client.1.smithi148.stdout:5/939: dwrite d0/db/d1e/d36/d41/f58 [0,4194304] 0 2024-03-29T08:07:31.896 INFO:tasks.workunit.client.1.smithi148.stdout:0/955: chown db/dc/daa/d47/daf/f11a 6741 1 2024-03-29T08:07:31.899 INFO:tasks.workunit.client.0.smithi037.stdout:1/671: getdents d2/d4b/d80/db0 0 2024-03-29T08:07:31.899 INFO:tasks.workunit.client.1.smithi148.stdout:0/956: stat db/dc/daa/fe0 0 2024-03-29T08:07:31.900 INFO:tasks.workunit.client.1.smithi148.stdout:6/961: stat d6/d9/d1e/d36/d4d/d9d/d28/d98/dc2/ffd 0 2024-03-29T08:07:31.901 INFO:tasks.workunit.client.0.smithi037.stdout:5/760: mknod d8/d22/d37/dbc/d85/c10a 0 2024-03-29T08:07:31.901 INFO:tasks.workunit.client.1.smithi148.stdout:3/977: truncate de/d3b/d49/fdd 676228 0 2024-03-29T08:07:31.904 INFO:tasks.workunit.client.1.smithi148.stdout:4/991: rename df/c5d to df/d2c/d51/d77/d26/d34/d43/df6/c13a 0 2024-03-29T08:07:31.905 INFO:tasks.workunit.client.0.smithi037.stdout:5/761: rmdir d8/d4d/d5c/dbf 39 2024-03-29T08:07:31.911 INFO:tasks.workunit.client.0.smithi037.stdout:5/762: creat d8/d22/dc1/f10b x:0 0 0 2024-03-29T08:07:31.911 INFO:tasks.workunit.client.1.smithi148.stdout:5/940: getdents d0/db/df2 0 2024-03-29T08:07:31.912 INFO:tasks.workunit.client.1.smithi148.stdout:0/957: write db/de/df/d2b/dae/d116/f11e [508233,47440] 0 2024-03-29T08:07:31.913 INFO:tasks.workunit.client.1.smithi148.stdout:3/978: chown de/d24/d33/d39/d46/d70/cbb 0 1 2024-03-29T08:07:31.916 INFO:tasks.workunit.client.1.smithi148.stdout:9/993: dwrite db/d22/d26/d54/d4d/d35/d4b/dc3/fb9 [4194304,4194304] 0 2024-03-29T08:07:31.916 INFO:tasks.workunit.client.0.smithi037.stdout:5/763: chown d8/d22/d37/d8d/db3/dcb/deb/d61/ccd 177876 1 2024-03-29T08:07:31.917 INFO:tasks.workunit.client.1.smithi148.stdout:7/917: sync 2024-03-29T08:07:31.918 INFO:tasks.workunit.client.1.smithi148.stdout:6/962: creat d6/d9/d1e/d36/d4d/d9d/d28/db3/d114/f130 x:0 0 0 2024-03-29T08:07:31.918 INFO:tasks.workunit.client.0.smithi037.stdout:5/764: chown d8/d22/d4f/d70/f82 16333327 1 2024-03-29T08:07:31.919 INFO:tasks.workunit.client.1.smithi148.stdout:5/941: chown d0/db/d1e/d35/d42/d67/d76/fba 34 1 2024-03-29T08:07:31.920 INFO:tasks.workunit.client.0.smithi037.stdout:1/672: link d2/d6/dc/d5d/d62/laa d2/d6/leb 0 2024-03-29T08:07:31.922 INFO:tasks.workunit.client.1.smithi148.stdout:0/958: readlink db/dc/daa/l57 0 2024-03-29T08:07:31.923 INFO:tasks.workunit.client.1.smithi148.stdout:6/963: write d6/d9/d1e/d36/d4d/d9d/d28/d98/dc2/ffd [815769,42464] 0 2024-03-29T08:07:31.928 INFO:tasks.workunit.client.1.smithi148.stdout:3/979: mkdir de/d24/d33/d39/d46/d30/d37/d79/d10e/d145 0 2024-03-29T08:07:31.929 INFO:tasks.workunit.client.1.smithi148.stdout:7/918: truncate d8/fe 493177 0 2024-03-29T08:07:31.931 INFO:tasks.workunit.client.1.smithi148.stdout:0/959: creat db/de/d8e/da9/f126 x:0 0 0 2024-03-29T08:07:31.932 INFO:tasks.workunit.client.1.smithi148.stdout:1/926: dread d3/d55/f43 [0,4194304] 0 2024-03-29T08:07:31.934 INFO:tasks.workunit.client.1.smithi148.stdout:1/927: readlink d3/de/d25/d51/ld2 0 2024-03-29T08:07:31.934 INFO:tasks.workunit.client.1.smithi148.stdout:0/960: fsync db/de/df/d11/d2d/da3/dee/f106 0 2024-03-29T08:07:31.938 INFO:tasks.workunit.client.0.smithi037.stdout:5/765: getdents d8/d22/d37/dbc 0 2024-03-29T08:07:31.938 INFO:tasks.workunit.client.0.smithi037.stdout:7/707: write dd/d21/d35/d42/f8e [2674737,73566] 0 2024-03-29T08:07:31.938 INFO:tasks.workunit.client.1.smithi148.stdout:9/994: write db/d22/d26/f38 [1412177,123895] 0 2024-03-29T08:07:31.941 INFO:tasks.workunit.client.1.smithi148.stdout:6/964: rmdir d6/d9/dc7 39 2024-03-29T08:07:31.942 INFO:tasks.workunit.client.1.smithi148.stdout:0/961: rename db/de/df/d11/d1f/fe8 to db/de/df/d11/d1f/f127 0 2024-03-29T08:07:31.943 INFO:tasks.workunit.client.0.smithi037.stdout:5/766: creat d8/d4d/ddf/f10c x:0 0 0 2024-03-29T08:07:31.944 INFO:tasks.workunit.client.1.smithi148.stdout:4/992: link df/d2c/d51/d77/d26/d34/ca3 df/d2c/d51/d77/d26/d34/d8a/d98/c13b 0 2024-03-29T08:07:31.946 INFO:tasks.workunit.client.0.smithi037.stdout:1/673: dwrite d2/d40/fbc [0,4194304] 0 2024-03-29T08:07:31.948 INFO:tasks.workunit.client.0.smithi037.stdout:5/767: dread - d8/d22/d37/dbc/ff3 zero size 2024-03-29T08:07:31.949 INFO:tasks.workunit.client.1.smithi148.stdout:3/980: creat de/d24/d33/f146 x:0 0 0 2024-03-29T08:07:31.953 INFO:tasks.workunit.client.1.smithi148.stdout:6/965: rmdir d6/d9/d1e/d36/d4d/d9d/d28/dda/db4 39 2024-03-29T08:07:31.953 INFO:tasks.workunit.client.1.smithi148.stdout:8/930: sync 2024-03-29T08:07:31.953 INFO:tasks.workunit.client.0.smithi037.stdout:2/526: dread d0/f18 [4194304,4194304] 0 2024-03-29T08:07:31.955 INFO:tasks.workunit.client.0.smithi037.stdout:5/768: write d8/d22/d37/d8d/db3/dcb/deb/fe6 [419502,27416] 0 2024-03-29T08:07:31.955 INFO:tasks.workunit.client.1.smithi148.stdout:5/942: dwrite d0/db/d1e/d6e/f92 [0,4194304] 0 2024-03-29T08:07:31.956 INFO:tasks.workunit.client.1.smithi148.stdout:4/993: dread - df/d2c/d80/daa/f124 zero size 2024-03-29T08:07:31.963 INFO:tasks.workunit.client.1.smithi148.stdout:5/943: mknod d0/d46/d9e/c12b 0 2024-03-29T08:07:31.964 INFO:tasks.workunit.client.0.smithi037.stdout:7/708: dwrite dd/d72/fae [4194304,4194304] 0 2024-03-29T08:07:31.966 INFO:tasks.workunit.client.0.smithi037.stdout:7/709: write dd/d1d/d34/f3f [1999490,72796] 0 2024-03-29T08:07:31.967 INFO:tasks.workunit.client.0.smithi037.stdout:7/710: stat dd/dc3/fd7 0 2024-03-29T08:07:31.968 INFO:tasks.workunit.client.1.smithi148.stdout:7/919: dread d8/d14/f16 [0,4194304] 0 2024-03-29T08:07:31.969 INFO:tasks.workunit.client.1.smithi148.stdout:6/966: symlink d6/d9/d1e/d36/d4d/d9d/d28/d98/l131 0 2024-03-29T08:07:31.976 INFO:tasks.workunit.client.0.smithi037.stdout:1/674: write d2/d6/dc/d18/d1d/d21/fb7 [415310,91683] 0 2024-03-29T08:07:31.976 INFO:tasks.workunit.client.0.smithi037.stdout:2/527: write d0/d23/d44/f67 [60110,126908] 0 2024-03-29T08:07:31.978 INFO:tasks.workunit.client.1.smithi148.stdout:3/981: creat de/d24/d33/d39/d46/d30/d37/d62/de1/f147 x:0 0 0 2024-03-29T08:07:31.980 INFO:tasks.workunit.client.1.smithi148.stdout:1/928: write d3/d18/d71/f9b [503567,32423] 0 2024-03-29T08:07:31.980 INFO:tasks.workunit.client.1.smithi148.stdout:9/995: write db/d22/d26/d54/d4d/d35/d4b/d8e/f97 [3344413,96469] 0 2024-03-29T08:07:31.983 INFO:tasks.workunit.client.0.smithi037.stdout:2/528: mknod d0/d20/d7d/cb3 0 2024-03-29T08:07:31.984 INFO:tasks.workunit.client.0.smithi037.stdout:1/675: fsync d2/d6/dc/d58/fa2 0 2024-03-29T08:07:31.989 INFO:tasks.workunit.client.1.smithi148.stdout:0/962: dwrite db/de/df/d2b/f34 [0,4194304] 0 2024-03-29T08:07:31.990 INFO:tasks.workunit.client.0.smithi037.stdout:2/529: fdatasync d0/f40 0 2024-03-29T08:07:31.994 INFO:tasks.workunit.client.0.smithi037.stdout:7/711: write dd/d21/d35/f6b [819988,41631] 0 2024-03-29T08:07:31.995 INFO:tasks.workunit.client.0.smithi037.stdout:5/769: dwrite d8/d4d/d72/fcc [0,4194304] 0 2024-03-29T08:07:31.996 INFO:tasks.workunit.client.0.smithi037.stdout:2/530: chown d0/d20/d22/l64 7 1 2024-03-29T08:07:31.997 INFO:tasks.workunit.client.1.smithi148.stdout:7/920: link d8/d38/d6a/d91/dff/l10e d8/dfb/d44/d50/l142 0 2024-03-29T08:07:31.998 INFO:tasks.workunit.client.0.smithi037.stdout:1/676: read d2/d6/dc/d58/fa2 [682821,82547] 0 2024-03-29T08:07:32.001 INFO:tasks.workunit.client.1.smithi148.stdout:7/921: chown d8/dfb/d44/dc3 0 1 2024-03-29T08:07:32.003 INFO:tasks.workunit.client.0.smithi037.stdout:2/531: fdatasync d0/d20/f75 0 2024-03-29T08:07:32.003 INFO:tasks.workunit.client.0.smithi037.stdout:7/712: truncate dd/d1d/d34/d54/feb 246208 0 2024-03-29T08:07:32.004 INFO:tasks.workunit.client.1.smithi148.stdout:8/931: dwrite d17/d74/fb4 [0,4194304] 0 2024-03-29T08:07:32.006 INFO:tasks.workunit.client.0.smithi037.stdout:7/713: stat dd/d21/d35/d42/f8e 0 2024-03-29T08:07:32.008 INFO:tasks.workunit.client.1.smithi148.stdout:7/922: chown d8/dfb/d44/d50/f138 42294056 1 2024-03-29T08:07:32.011 INFO:tasks.workunit.client.1.smithi148.stdout:1/929: dwrite d3/d55/f37 [0,4194304] 0 2024-03-29T08:07:32.011 INFO:tasks.workunit.client.0.smithi037.stdout:2/532: dread d0/d23/d44/f67 [0,4194304] 0 2024-03-29T08:07:32.013 INFO:tasks.workunit.client.1.smithi148.stdout:7/923: dread - d8/df4/d111/f13b zero size 2024-03-29T08:07:32.014 INFO:tasks.workunit.client.0.smithi037.stdout:5/770: fdatasync d8/d22/d37/d8d/db3/dcb/deb/fa9 0 2024-03-29T08:07:32.022 INFO:tasks.workunit.client.0.smithi037.stdout:1/677: symlink d2/d6/dc/d18/d22/da0/lec 0 2024-03-29T08:07:32.026 INFO:tasks.workunit.client.1.smithi148.stdout:7/924: chown d8/d38/d3c/da5/c123 3 1 2024-03-29T08:07:32.035 INFO:tasks.workunit.client.0.smithi037.stdout:3/585: sync 2024-03-29T08:07:32.038 INFO:tasks.workunit.client.0.smithi037.stdout:3/586: truncate dd/d15/f50 2275423 0 2024-03-29T08:07:32.039 INFO:tasks.workunit.client.1.smithi148.stdout:8/932: dwrite d17/d74/fb1 [0,4194304] 0 2024-03-29T08:07:32.039 INFO:tasks.workunit.client.1.smithi148.stdout:4/994: creat df/d2c/d51/d77/f13c x:0 0 0 2024-03-29T08:07:32.040 INFO:tasks.workunit.client.0.smithi037.stdout:2/533: dwrite d0/d20/d2e/fa7 [0,4194304] 0 2024-03-29T08:07:32.041 INFO:tasks.workunit.client.1.smithi148.stdout:4/995: chown df/f105 9369911 1 2024-03-29T08:07:32.042 INFO:tasks.workunit.client.1.smithi148.stdout:8/933: stat d17/d18/d26/d63/d66/d5f 0 2024-03-29T08:07:32.045 INFO:tasks.workunit.client.1.smithi148.stdout:8/934: readlink d17/d1d/lc2 0 2024-03-29T08:07:32.046 INFO:tasks.workunit.client.1.smithi148.stdout:8/935: chown d17/f23 2 1 2024-03-29T08:07:32.048 INFO:tasks.workunit.client.0.smithi037.stdout:3/587: dread - dd/d46/d3e/fa3 zero size 2024-03-29T08:07:32.048 INFO:tasks.workunit.client.0.smithi037.stdout:5/771: dwrite d8/d22/dc1/fd2 [0,4194304] 0 2024-03-29T08:07:32.050 INFO:tasks.workunit.client.1.smithi148.stdout:7/925: dread d8/d14/fcc [0,4194304] 0 2024-03-29T08:07:32.053 INFO:tasks.workunit.client.1.smithi148.stdout:8/936: creat d17/d18/d26/d63/d66/d5f/f134 x:0 0 0 2024-03-29T08:07:32.059 INFO:tasks.workunit.client.0.smithi037.stdout:5/772: write d8/d34/fcf [172268,8034] 0 2024-03-29T08:07:32.059 INFO:tasks.workunit.client.0.smithi037.stdout:0/544: sync 2024-03-29T08:07:32.059 INFO:tasks.workunit.client.0.smithi037.stdout:4/696: sync 2024-03-29T08:07:32.059 INFO:tasks.workunit.client.0.smithi037.stdout:9/867: sync 2024-03-29T08:07:32.060 INFO:tasks.workunit.client.0.smithi037.stdout:6/580: sync 2024-03-29T08:07:32.060 INFO:tasks.workunit.client.0.smithi037.stdout:8/604: sync 2024-03-29T08:07:32.066 INFO:tasks.workunit.client.1.smithi148.stdout:7/926: read d8/d14/d45/d4d/d78/fbc [3886699,36206] 0 2024-03-29T08:07:32.067 INFO:tasks.workunit.client.0.smithi037.stdout:2/534: creat d0/d20/d2e/da4/fb4 x:0 0 0 2024-03-29T08:07:32.067 INFO:tasks.workunit.client.0.smithi037.stdout:6/581: chown d0/d5/d1e/d44 1906413574 1 2024-03-29T08:07:32.067 INFO:tasks.workunit.client.0.smithi037.stdout:5/773: chown d8/d22/d4f/d70/d9c 2489 1 2024-03-29T08:07:32.069 INFO:tasks.workunit.client.1.smithi148.stdout:5/944: fsync d0/db/d1e/d35/d42/d67/d76/d81/d98/fb9 0 2024-03-29T08:07:32.071 INFO:tasks.workunit.client.1.smithi148.stdout:9/996: sync 2024-03-29T08:07:32.071 INFO:tasks.workunit.client.0.smithi037.stdout:8/605: fsync d5/f15 0 2024-03-29T08:07:32.071 INFO:tasks.workunit.client.0.smithi037.stdout:4/697: readlink d3/d4/d15/d1d/db/d45/d1e/l24 0 2024-03-29T08:07:32.075 INFO:tasks.workunit.client.1.smithi148.stdout:8/937: creat d17/d18/d26/d63/d66/d5f/dd0/d119/d122/f135 x:0 0 0 2024-03-29T08:07:32.076 INFO:tasks.workunit.client.0.smithi037.stdout:3/588: write dd/d15/f62 [5260394,64378] 0 2024-03-29T08:07:32.076 INFO:tasks.workunit.client.0.smithi037.stdout:8/606: stat d5/d10/d19 0 2024-03-29T08:07:32.083 INFO:tasks.workunit.client.0.smithi037.stdout:4/698: write d3/d54/fc0 [506747,122357] 0 2024-03-29T08:07:32.091 INFO:tasks.workunit.client.0.smithi037.stdout:4/699: write d3/d54/fc0 [1929779,18994] 0 2024-03-29T08:07:32.093 INFO:tasks.workunit.client.1.smithi148.stdout:8/938: read d17/d18/f2a [912374,79121] 0 2024-03-29T08:07:32.094 INFO:tasks.workunit.client.0.smithi037.stdout:7/714: dwrite dd/d21/d35/d60/f6e [0,4194304] 0 2024-03-29T08:07:32.096 INFO:tasks.workunit.client.0.smithi037.stdout:6/582: rmdir d0/d19/d34/d5a 39 2024-03-29T08:07:32.100 INFO:tasks.workunit.client.0.smithi037.stdout:2/535: rmdir d0/d23/d70/d71 39 2024-03-29T08:07:32.100 INFO:tasks.workunit.client.0.smithi037.stdout:1/678: dread d2/d6/d13/f28 [0,4194304] 0 2024-03-29T08:07:32.106 INFO:tasks.workunit.client.0.smithi037.stdout:2/536: read d0/d53/f88 [2658429,110185] 0 2024-03-29T08:07:32.109 INFO:tasks.workunit.client.0.smithi037.stdout:2/537: fdatasync d0/f14 0 2024-03-29T08:07:32.113 INFO:tasks.workunit.client.0.smithi037.stdout:9/868: unlink d1/d4/d19/d1e/d4a/d6f/da2/d31/daa/cf7 0 2024-03-29T08:07:32.115 INFO:tasks.workunit.client.1.smithi148.stdout:1/930: creat d3/d18/d4d/d76/d85/d11d/d47/d6d/d72/d12b/f134 x:0 0 0 2024-03-29T08:07:32.115 INFO:tasks.workunit.client.0.smithi037.stdout:4/700: dwrite d3/d4/d27/f86 [0,4194304] 0 2024-03-29T08:07:32.117 INFO:tasks.workunit.client.0.smithi037.stdout:8/607: mkdir d5/d10/d19/d3d/d4a/dc8 0 2024-03-29T08:07:32.123 INFO:tasks.workunit.client.1.smithi148.stdout:1/931: symlink d3/d18/d4d/d9e/d12f/l135 0 2024-03-29T08:07:32.124 INFO:tasks.workunit.client.0.smithi037.stdout:0/545: mkdir d9/da/dd/d20/d55/d5f/dab/dbb 0 2024-03-29T08:07:32.124 INFO:tasks.workunit.client.0.smithi037.stdout:7/715: mknod dd/d1d/ced 0 2024-03-29T08:07:32.125 INFO:tasks.workunit.client.0.smithi037.stdout:5/774: write d8/d22/d4f/d70/d79/f7b [3415234,3648] 0 2024-03-29T08:07:32.129 INFO:tasks.workunit.client.1.smithi148.stdout:7/927: dwrite d8/d38/d3c/da5/db4/fc6 [0,4194304] 0 2024-03-29T08:07:32.133 INFO:tasks.workunit.client.1.smithi148.stdout:1/932: write d3/d18/d4d/d76/d85/d11d/d47/d6d/d72/ff5 [604261,33890] 0 2024-03-29T08:07:32.133 INFO:tasks.workunit.client.0.smithi037.stdout:5/775: write d8/fa [4936378,92714] 0 2024-03-29T08:07:32.134 INFO:tasks.workunit.client.0.smithi037.stdout:3/589: dwrite dd/d15/d18/f2f [0,4194304] 0 2024-03-29T08:07:32.138 INFO:tasks.workunit.client.1.smithi148.stdout:8/939: dwrite d17/d18/ddf/fe1 [0,4194304] 0 2024-03-29T08:07:32.143 INFO:tasks.workunit.client.0.smithi037.stdout:5/776: write d8/d18/d24/f74 [1502929,97468] 0 2024-03-29T08:07:32.143 INFO:tasks.workunit.client.0.smithi037.stdout:5/777: fdatasync d8/fc5 0 2024-03-29T08:07:32.143 INFO:tasks.workunit.client.0.smithi037.stdout:5/778: write d8/fc5 [1801059,111807] 0 2024-03-29T08:07:32.145 INFO:tasks.workunit.client.1.smithi148.stdout:1/933: symlink d3/d55/ddf/l136 0 2024-03-29T08:07:32.147 INFO:tasks.workunit.client.0.smithi037.stdout:9/869: rmdir d1/d4/d19/d1e/d4a/d6f/da2 39 2024-03-29T08:07:32.149 INFO:tasks.workunit.client.1.smithi148.stdout:8/940: mknod d17/d74/c136 0 2024-03-29T08:07:32.149 INFO:tasks.workunit.client.0.smithi037.stdout:4/701: creat d3/d80/feb x:0 0 0 2024-03-29T08:07:32.151 INFO:tasks.workunit.client.0.smithi037.stdout:8/608: rmdir d5/d10/d45 39 2024-03-29T08:07:32.153 INFO:tasks.workunit.client.1.smithi148.stdout:7/928: link d8/dfb/d11b/f39 d8/dfb/db8/d11e/f143 0 2024-03-29T08:07:32.154 INFO:tasks.workunit.client.1.smithi148.stdout:8/941: chown d17/d49/d9a/dbe/la6 21 1 2024-03-29T08:07:32.156 INFO:tasks.workunit.client.0.smithi037.stdout:7/716: creat dd/d21/d35/d42/ddb/fee x:0 0 0 2024-03-29T08:07:32.156 INFO:tasks.workunit.client.1.smithi148.stdout:4/996: mknod df/d2c/d51/d77/d26/d9b/da7/c13d 0 2024-03-29T08:07:32.157 INFO:tasks.workunit.client.1.smithi148.stdout:8/942: chown d17/d74/fb4 18877975 1 2024-03-29T08:07:32.159 INFO:tasks.workunit.client.1.smithi148.stdout:4/997: stat df/d2c/d51/d77/dbe/f117 0 2024-03-29T08:07:32.160 INFO:tasks.workunit.client.0.smithi037.stdout:6/583: fdatasync d0/fcf 0 2024-03-29T08:07:32.163 INFO:tasks.workunit.client.0.smithi037.stdout:2/538: creat d0/d23/d44/d62/fb5 x:0 0 0 2024-03-29T08:07:32.164 INFO:tasks.workunit.client.1.smithi148.stdout:4/998: write df/d2c/d51/d77/def/f10d [114663,113520] 0 2024-03-29T08:07:32.164 INFO:tasks.workunit.client.0.smithi037.stdout:5/779: creat d8/d4d/f10d x:0 0 0 2024-03-29T08:07:32.164 INFO:tasks.workunit.client.0.smithi037.stdout:5/780: stat d8/fa0 0 2024-03-29T08:07:32.169 INFO:tasks.workunit.client.0.smithi037.stdout:7/717: rename dd/d21/d35/d3b/d5d/l95 to dd/d21/d65/lef 0 2024-03-29T08:07:32.170 INFO:tasks.workunit.client.0.smithi037.stdout:4/702: mkdir d3/d57/dec 0 2024-03-29T08:07:32.171 INFO:tasks.workunit.client.0.smithi037.stdout:9/870: symlink d1/d4/d19/l124 0 2024-03-29T08:07:32.171 INFO:tasks.workunit.client.0.smithi037.stdout:4/703: stat d3/d4/d27/l91 0 2024-03-29T08:07:32.173 INFO:tasks.workunit.client.0.smithi037.stdout:5/781: mknod d8/d22/d37/dbc/d85/d90/c10e 0 2024-03-29T08:07:32.179 INFO:tasks.workunit.client.0.smithi037.stdout:5/782: read - d8/d22/d4f/d70/d9c/f102 zero size 2024-03-29T08:07:32.180 INFO:tasks.workunit.client.0.smithi037.stdout:2/539: link d0/d20/d2e/d5f/faa d0/d20/d2e/d38/fb6 0 2024-03-29T08:07:32.193 INFO:tasks.workunit.client.1.smithi148.stdout:9/997: mkdir db/d22/d26/d54/d4d/d35/d4b/d58/d143 0 2024-03-29T08:07:32.196 INFO:tasks.workunit.client.1.smithi148.stdout:9/998: read db/d10/d2d/d4e/dd4/f1b [3384189,235] 0 2024-03-29T08:07:32.200 INFO:tasks.workunit.client.0.smithi037.stdout:9/871: mkdir d1/d4/d19/d1e/d4a/d6f/da2/d13/de0/d125 0 2024-03-29T08:07:32.202 INFO:tasks.workunit.client.1.smithi148.stdout:8/943: fsync d17/d74/d86/fe5 0 2024-03-29T08:07:32.209 INFO:tasks.workunit.client.0.smithi037.stdout:7/718: dwrite dd/d21/d35/d3b/d5d/fc7 [0,4194304] 0 2024-03-29T08:07:32.219 INFO:tasks.workunit.client.0.smithi037.stdout:2/540: creat d0/d20/d2e/d5f/d6f/fb7 x:0 0 0 2024-03-29T08:07:32.220 INFO:tasks.workunit.client.0.smithi037.stdout:1/679: dwrite d2/d6/d13/f77 [0,4194304] 0 2024-03-29T08:07:32.224 INFO:tasks.workunit.client.1.smithi148.stdout:8/944: dwrite d17/d18/d26/d63/d7d/f115 [4194304,4194304] 0 2024-03-29T08:07:32.228 INFO:tasks.workunit.client.0.smithi037.stdout:9/872: unlink d1/d4/d19/d1e/d25/c58 0 2024-03-29T08:07:32.235 INFO:tasks.workunit.client.1.smithi148.stdout:6/967: rename d6/d9/d1e/d36/f6f to d6/f132 0 2024-03-29T08:07:32.235 INFO:tasks.workunit.client.0.smithi037.stdout:0/546: truncate f5 5695017 0 2024-03-29T08:07:32.236 INFO:tasks.workunit.client.1.smithi148.stdout:4/999: write df/d2c/d51/d77/d26/d34/d8a/d98/fb0 [2267098,44854] 0 2024-03-29T08:07:32.239 INFO:tasks.workunit.client.1.smithi148.stdout:9/999: write db/d22/d26/d54/f36 [848180,122278] 0 2024-03-29T08:07:32.239 INFO:tasks.workunit.client.1.smithi148.stdout:8/945: fdatasync d17/d49/d9a/dbe/f76 0 2024-03-29T08:07:32.241 INFO:tasks.workunit.client.0.smithi037.stdout:7/719: mkdir dd/d1d/d49/d81/df0 0 2024-03-29T08:07:32.244 INFO:tasks.workunit.client.0.smithi037.stdout:1/680: truncate d2/f94 1264254 0 2024-03-29T08:07:32.245 INFO:tasks.workunit.client.0.smithi037.stdout:2/541: stat d0/d23/d70/d71/lad 0 2024-03-29T08:07:32.246 INFO:tasks.workunit.client.1.smithi148.stdout:5/945: unlink d0/db/d1e/d35/d42/d67/d76/d81/cf0 0 2024-03-29T08:07:32.247 INFO:tasks.workunit.client.1.smithi148.stdout:6/968: unlink d6/d9/d1e/d27/c2e 0 2024-03-29T08:07:32.247 INFO:tasks.workunit.client.0.smithi037.stdout:8/609: dread d5/d10/d2d/d34/d50/f5e [0,4194304] 0 2024-03-29T08:07:32.249 INFO:tasks.workunit.client.1.smithi148.stdout:5/946: write d0/db/d1e/d36/d41/f7f [4965279,38532] 0 2024-03-29T08:07:32.250 INFO:tasks.workunit.client.1.smithi148.stdout:0/963: rename db/dc/daa/d47/cbb to db/dc/d78/c128 0 2024-03-29T08:07:32.251 INFO:tasks.workunit.client.0.smithi037.stdout:4/704: write d3/d4/d15/d1d/db/d45/d1e/d5f/f62 [673332,95270] 0 2024-03-29T08:07:32.253 INFO:tasks.workunit.client.1.smithi148.stdout:0/964: dread - db/de/df/d11/ff4 zero size 2024-03-29T08:07:32.254 INFO:tasks.workunit.client.1.smithi148.stdout:6/969: dread d6/d9/d1e/d36/d4d/fbf [0,4194304] 0 2024-03-29T08:07:32.254 INFO:tasks.workunit.client.1.smithi148.stdout:3/982: rename de/d24/d33/d39/d46/d30/d37/d62/d63/dbf/lf9 to de/d24/d33/d39/d46/d30/d37/d79/d10e/d145/l148 0 2024-03-29T08:07:32.257 INFO:tasks.workunit.client.0.smithi037.stdout:3/590: dwrite dd/d15/d27/d33/f90 [0,4194304] 0 2024-03-29T08:07:32.257 INFO:tasks.workunit.client.0.smithi037.stdout:5/783: write d8/d22/d37/de8/f108 [491986,12557] 0 2024-03-29T08:07:32.258 INFO:tasks.workunit.client.1.smithi148.stdout:7/929: write d8/fe [1459011,21022] 0 2024-03-29T08:07:32.259 INFO:tasks.workunit.client.0.smithi037.stdout:7/720: mknod dd/d1d/d34/d54/cf1 0 2024-03-29T08:07:32.259 INFO:tasks.workunit.client.1.smithi148.stdout:5/947: mkdir d0/db/d1e/d5b/d66/d72/d12c 0 2024-03-29T08:07:32.262 INFO:tasks.workunit.client.0.smithi037.stdout:9/873: truncate d1/d4/d19/d1e/d4a/d6f/f83 889439 0 2024-03-29T08:07:32.268 INFO:tasks.workunit.client.1.smithi148.stdout:1/934: rename d3/de/d6e/la2 to d3/d18/d4d/d9e/l137 0 2024-03-29T08:07:32.269 INFO:tasks.workunit.client.1.smithi148.stdout:8/946: dread d17/d18/d26/d4f/d94/fe3 [0,4194304] 0 2024-03-29T08:07:32.269 INFO:tasks.workunit.client.1.smithi148.stdout:7/930: creat d8/d38/d6a/d91/dff/f144 x:0 0 0 2024-03-29T08:07:32.270 INFO:tasks.workunit.client.1.smithi148.stdout:5/948: getdents d0/db/dae 0 2024-03-29T08:07:32.272 INFO:tasks.workunit.client.1.smithi148.stdout:7/931: chown d8/d38/l89 194 1 2024-03-29T08:07:32.275 INFO:tasks.workunit.client.1.smithi148.stdout:0/965: dwrite db/de/df/fa6 [0,4194304] 0 2024-03-29T08:07:32.275 INFO:tasks.workunit.client.1.smithi148.stdout:6/970: rename d6/d9/d1e/d36/d4d/c88 to d6/d9/d1e/d36/d4d/d9d/d28/dda/db6/df3/c133 0 2024-03-29T08:07:32.275 INFO:tasks.workunit.client.1.smithi148.stdout:7/932: write d8/fe [1098288,75866] 0 2024-03-29T08:07:32.276 INFO:tasks.workunit.client.0.smithi037.stdout:1/681: dwrite d2/d6/d13/f77 [0,4194304] 0 2024-03-29T08:07:32.283 INFO:tasks.workunit.client.1.smithi148.stdout:3/983: rename de/d24/d33/d39/d46/d30/d37/d79/fc0 to de/d24/d33/d39/d46/d30/d37/d79/d10e/f149 0 2024-03-29T08:07:32.286 INFO:tasks.workunit.client.1.smithi148.stdout:6/971: symlink d6/d69/l134 0 2024-03-29T08:07:32.292 INFO:tasks.workunit.client.0.smithi037.stdout:0/547: write d9/da/d1b/f41 [212215,2976] 0 2024-03-29T08:07:32.294 INFO:tasks.workunit.client.0.smithi037.stdout:3/591: dread - dd/d46/d2d/d3a/fa6 zero size 2024-03-29T08:07:32.294 INFO:tasks.workunit.client.1.smithi148.stdout:5/949: rename d0/db/d7b/d16/d87/d117 to d0/db/d1e/d5b/d66/d72/d12d 0 2024-03-29T08:07:32.296 INFO:tasks.workunit.client.1.smithi148.stdout:5/950: chown d0/d46/l83 197 1 2024-03-29T08:07:32.296 INFO:tasks.workunit.client.1.smithi148.stdout:7/933: unlink d8/d38/d3c/c99 0 2024-03-29T08:07:32.298 INFO:tasks.workunit.client.1.smithi148.stdout:7/934: chown d8/dfb/d44/d49/f76 8260 1 2024-03-29T08:07:32.299 INFO:tasks.workunit.client.0.smithi037.stdout:8/610: mkdir d5/d10/d45/d4b/dc9 0 2024-03-29T08:07:32.304 INFO:tasks.workunit.client.0.smithi037.stdout:1/682: fdatasync d2/d6/dc/d58/fc0 0 2024-03-29T08:07:32.309 INFO:tasks.workunit.client.0.smithi037.stdout:7/721: write dd/d1d/f2a [390135,48577] 0 2024-03-29T08:07:32.313 INFO:tasks.workunit.client.0.smithi037.stdout:5/784: mkdir d8/d22/d4f/d70/d10f 0 2024-03-29T08:07:32.314 INFO:tasks.workunit.client.1.smithi148.stdout:1/935: write d3/de/d21/f80 [4503697,70750] 0 2024-03-29T08:07:32.316 INFO:tasks.workunit.client.0.smithi037.stdout:5/785: stat d8/d34/cb0 0 2024-03-29T08:07:32.317 INFO:tasks.workunit.client.0.smithi037.stdout:2/542: dwrite d0/d20/d2e/d5f/d6f/d5e/f60 [4194304,4194304] 0 2024-03-29T08:07:32.321 INFO:tasks.workunit.client.0.smithi037.stdout:3/592: truncate dd/d46/d2d/d3a/d6c/f8b 572949 0 2024-03-29T08:07:32.325 INFO:tasks.workunit.client.1.smithi148.stdout:7/935: getdents d8/df4/d111 0 2024-03-29T08:07:32.328 INFO:tasks.workunit.client.1.smithi148.stdout:3/984: write de/d3b/d49/fdd [1344585,111587] 0 2024-03-29T08:07:32.328 INFO:tasks.workunit.client.1.smithi148.stdout:7/936: fdatasync d8/dfb/d44/d50/ff9 0 2024-03-29T08:07:32.329 INFO:tasks.workunit.client.0.smithi037.stdout:9/874: creat d1/d1f/f126 x:0 0 0 2024-03-29T08:07:32.336 INFO:tasks.workunit.client.1.smithi148.stdout:5/951: dread d0/db/d1e/d35/d42/d67/d9a/dd7/fed [0,4194304] 0 2024-03-29T08:07:32.339 INFO:tasks.workunit.client.1.smithi148.stdout:0/966: dread db/de/df/f30 [0,4194304] 0 2024-03-29T08:07:32.341 INFO:tasks.workunit.client.1.smithi148.stdout:1/936: mknod d3/d18/dae/dda/df3/d110/c138 0 2024-03-29T08:07:32.342 INFO:tasks.workunit.client.1.smithi148.stdout:7/937: readlink d8/d38/d3c/da5/db4/de9/d114/l12b 0 2024-03-29T08:07:32.342 INFO:tasks.workunit.client.1.smithi148.stdout:6/972: dread d6/f19 [0,4194304] 0 2024-03-29T08:07:32.343 INFO:tasks.workunit.client.1.smithi148.stdout:5/952: creat d0/db/d1e/d35/d42/f12e x:0 0 0 2024-03-29T08:07:32.343 INFO:tasks.workunit.client.0.smithi037.stdout:8/611: write d5/f32 [4763007,88730] 0 2024-03-29T08:07:32.344 INFO:tasks.workunit.client.1.smithi148.stdout:3/985: getdents de/d24/d33/d39/da3/dfe/d13e 0 2024-03-29T08:07:32.345 INFO:tasks.workunit.client.1.smithi148.stdout:0/967: symlink db/de/df/d11/d2d/da3/l129 0 2024-03-29T08:07:32.345 INFO:tasks.workunit.client.0.smithi037.stdout:3/593: symlink dd/d46/d3e/lc1 0 2024-03-29T08:07:32.345 INFO:tasks.workunit.client.0.smithi037.stdout:2/543: fsync d0/d20/d2e/d5f/f63 0 2024-03-29T08:07:32.346 INFO:tasks.workunit.client.1.smithi148.stdout:6/973: chown d6/d9/f23 0 1 2024-03-29T08:07:32.349 INFO:tasks.workunit.client.1.smithi148.stdout:5/953: chown d0/db/d1e/d35/d42/d67/d76/d81/d98/dd5/d9c/fbf 9164 1 2024-03-29T08:07:32.351 INFO:tasks.workunit.client.1.smithi148.stdout:1/937: getdents d3/de/d6e/df9/dfa 0 2024-03-29T08:07:32.353 INFO:tasks.workunit.client.0.smithi037.stdout:9/875: mkdir d1/d1f/dfd/d127 0 2024-03-29T08:07:32.353 INFO:tasks.workunit.client.1.smithi148.stdout:1/938: stat d3/de/d21/d5f/fa3 0 2024-03-29T08:07:32.354 INFO:tasks.workunit.client.1.smithi148.stdout:5/954: rmdir d0/db/d1e/d5b/d66/d72/d12d 39 2024-03-29T08:07:32.355 INFO:tasks.workunit.client.0.smithi037.stdout:8/612: stat d5/d10/d19/d8d/ca0 0 2024-03-29T08:07:32.355 INFO:tasks.workunit.client.1.smithi148.stdout:1/939: write d3/fd6 [1027310,116354] 0 2024-03-29T08:07:32.356 INFO:tasks.workunit.client.1.smithi148.stdout:6/974: truncate d6/d9/d1e/d20/d6b/fe3 2177350 0 2024-03-29T08:07:32.356 INFO:tasks.workunit.client.0.smithi037.stdout:4/705: dread d3/d4/d15/d89/f94 [0,4194304] 0 2024-03-29T08:07:32.357 INFO:tasks.workunit.client.0.smithi037.stdout:5/786: write d8/d18/d24/f44 [7832958,73751] 0 2024-03-29T08:07:32.363 INFO:tasks.workunit.client.1.smithi148.stdout:0/968: creat db/de/df/d11/f12a x:0 0 0 2024-03-29T08:07:32.364 INFO:tasks.workunit.client.0.smithi037.stdout:3/594: link dd/d15/d27/la0 dd/d46/lc2 0 2024-03-29T08:07:32.364 INFO:tasks.workunit.client.1.smithi148.stdout:3/986: dwrite de/d24/d119/f11b [0,4194304] 0 2024-03-29T08:07:32.364 INFO:tasks.workunit.client.1.smithi148.stdout:5/955: readlink d0/db/d1e/d35/d42/d67/d76/d81/d98/dd5/d9c/l127 0 2024-03-29T08:07:32.366 INFO:tasks.workunit.client.0.smithi037.stdout:0/548: dread d9/da/dd/d20/f4e [0,4194304] 0 2024-03-29T08:07:32.368 INFO:tasks.workunit.client.1.smithi148.stdout:3/987: read de/d3b/f58 [237988,70335] 0 2024-03-29T08:07:32.370 INFO:tasks.workunit.client.1.smithi148.stdout:1/940: mknod d3/d18/d4d/d76/d85/d11d/d47/c139 0 2024-03-29T08:07:32.370 INFO:tasks.workunit.client.1.smithi148.stdout:7/938: dwrite d8/dfb/d44/fab [4194304,4194304] 0 2024-03-29T08:07:32.372 INFO:tasks.workunit.client.0.smithi037.stdout:5/787: write d8/d22/d4f/d70/f82 [40476,54214] 0 2024-03-29T08:07:32.373 INFO:tasks.workunit.client.0.smithi037.stdout:4/706: rename d3/d4/d15 to d3/d57/dec/ded 0 2024-03-29T08:07:32.379 INFO:tasks.workunit.client.0.smithi037.stdout:7/722: dwrite dd/d1d/d34/d54/f5f [0,4194304] 0 2024-03-29T08:07:32.380 INFO:tasks.workunit.client.1.smithi148.stdout:6/975: mknod d6/d9/d1e/d36/d4d/d9d/d28/dda/db6/df3/df4/d105/c135 0 2024-03-29T08:07:32.382 INFO:tasks.workunit.client.1.smithi148.stdout:6/976: write d6/d9/d1e/d27/f90 [1464526,3240] 0 2024-03-29T08:07:32.385 INFO:tasks.workunit.client.1.smithi148.stdout:3/988: unlink de/d24/d33/d39/d46/d30/d37/d79/d10e/c112 0 2024-03-29T08:07:32.391 INFO:tasks.workunit.client.1.smithi148.stdout:6/977: write d6/d9/d1e/fbb [579759,26327] 0 2024-03-29T08:07:32.391 INFO:tasks.workunit.client.1.smithi148.stdout:7/939: readlink d8/d14/da8/l128 0 2024-03-29T08:07:32.391 INFO:tasks.workunit.client.1.smithi148.stdout:1/941: dwrite d3/de/d21/f80 [4194304,4194304] 0 2024-03-29T08:07:32.394 INFO:tasks.workunit.client.0.smithi037.stdout:2/544: link d0/d20/d2e/d5f/d6f/fa9 d0/d23/d44/d62/fb8 0 2024-03-29T08:07:32.396 INFO:tasks.workunit.client.1.smithi148.stdout:5/956: dread d0/db/d1e/d36/d41/f63 [4194304,4194304] 0 2024-03-29T08:07:32.399 INFO:tasks.workunit.client.0.smithi037.stdout:1/683: dwrite d2/d6/dc/d18/f8d [0,4194304] 0 2024-03-29T08:07:32.399 INFO:tasks.workunit.client.1.smithi148.stdout:6/978: fdatasync d6/d9/d1e/d27/d48/d9f/f106 0 2024-03-29T08:07:32.400 INFO:tasks.workunit.client.1.smithi148.stdout:3/989: unlink de/d24/d33/d39/d46/d30/d37/d79/d10e/f149 0 2024-03-29T08:07:32.400 INFO:tasks.workunit.client.1.smithi148.stdout:5/957: write d0/db/d1e/d6e/da8/f100 [231798,79080] 0 2024-03-29T08:07:32.401 INFO:tasks.workunit.client.1.smithi148.stdout:1/942: creat d3/d55/d7e/f13a x:0 0 0 2024-03-29T08:07:32.402 INFO:tasks.workunit.client.0.smithi037.stdout:5/788: dread - d8/d18/d24/fad zero size 2024-03-29T08:07:32.402 INFO:tasks.workunit.client.1.smithi148.stdout:0/969: truncate db/f109 1489891 0 2024-03-29T08:07:32.404 INFO:tasks.workunit.client.1.smithi148.stdout:1/943: write d3/fd6 [2166296,108558] 0 2024-03-29T08:07:32.408 INFO:tasks.workunit.client.1.smithi148.stdout:1/944: chown d3/d18/d4d/d76/d85/d11d/d97/d111/d125 57 1 2024-03-29T08:07:32.409 INFO:tasks.workunit.client.0.smithi037.stdout:1/684: chown d2/c1a 763 1 2024-03-29T08:07:32.409 INFO:tasks.workunit.client.0.smithi037.stdout:2/545: chown d0/d20/d22/c3e 1620 1 2024-03-29T08:07:32.412 INFO:tasks.workunit.client.1.smithi148.stdout:6/979: mkdir d6/d9/d1e/d136 0 2024-03-29T08:07:32.412 INFO:tasks.workunit.client.1.smithi148.stdout:5/958: rename d0/db/d7b/d16/d2e/ddd/de9/cfe to d0/d46/d9e/c12f 0 2024-03-29T08:07:32.415 INFO:tasks.workunit.client.1.smithi148.stdout:6/980: write d6/d9/d1e/d20/d6b/f11d [54899,81652] 0 2024-03-29T08:07:32.417 INFO:tasks.workunit.client.1.smithi148.stdout:7/940: getdents d8/dfb/db8/d11e 0 2024-03-29T08:07:32.419 INFO:tasks.workunit.client.1.smithi148.stdout:6/981: write d6/f70 [38225,73974] 0 2024-03-29T08:07:32.420 INFO:tasks.workunit.client.1.smithi148.stdout:1/945: mkdir d3/d18/dae/dda/df3/d13b 0 2024-03-29T08:07:32.420 INFO:tasks.workunit.client.0.smithi037.stdout:0/549: dwrite d9/da/dd/d20/f9e [0,4194304] 0 2024-03-29T08:07:32.423 INFO:tasks.workunit.client.1.smithi148.stdout:1/946: chown d3/d18/d4d/f63 17 1 2024-03-29T08:07:32.425 INFO:tasks.workunit.client.1.smithi148.stdout:6/982: stat d6/d9/d1e/f3d 0 2024-03-29T08:07:32.426 INFO:tasks.workunit.client.1.smithi148.stdout:7/941: chown d8/c6e 0 1 2024-03-29T08:07:32.427 INFO:tasks.workunit.client.1.smithi148.stdout:3/990: dwrite de/d24/d33/d39/d46/d30/d37/d62/fff [0,4194304] 0 2024-03-29T08:07:32.436 INFO:tasks.workunit.client.1.smithi148.stdout:3/991: symlink de/d24/d33/d39/d46/d30/d37/d79/d10e/d135/l14a 0 2024-03-29T08:07:32.438 INFO:tasks.workunit.client.0.smithi037.stdout:9/876: dread d1/d4/d19/d1e/d25/fcb [0,4194304] 0 2024-03-29T08:07:32.439 INFO:tasks.workunit.client.1.smithi148.stdout:3/992: chown de/d24/d33/d39/d46/d70 0 1 2024-03-29T08:07:32.443 INFO:tasks.workunit.client.1.smithi148.stdout:1/947: dwrite d3/d18/d4d/f106 [0,4194304] 0 2024-03-29T08:07:32.459 INFO:tasks.workunit.client.1.smithi148.stdout:3/993: write de/d24/d33/d39/d46/d30/d37/d62/d63/dbf/fd1 [561584,97262] 0 2024-03-29T08:07:32.460 INFO:tasks.workunit.client.1.smithi148.stdout:1/948: fsync d3/de/d21/fc1 0 2024-03-29T08:07:32.463 INFO:tasks.workunit.client.1.smithi148.stdout:8/947: sync 2024-03-29T08:07:32.463 INFO:tasks.workunit.client.1.smithi148.stdout:5/959: truncate d0/db/d1e/d6e/f92 195045 0 2024-03-29T08:07:32.467 INFO:tasks.workunit.client.0.smithi037.stdout:7/723: rmdir dd/d1d/d34/dc8 39 2024-03-29T08:07:32.468 INFO:tasks.workunit.client.0.smithi037.stdout:9/877: unlink d1/d3/d44/f57 0 2024-03-29T08:07:32.471 INFO:tasks.workunit.client.0.smithi037.stdout:5/789: write d8/d4d/ddf/fe9 [88505,123339] 0 2024-03-29T08:07:32.471 INFO:tasks.workunit.client.0.smithi037.stdout:3/595: creat dd/d15/fc3 x:0 0 0 2024-03-29T08:07:32.473 INFO:tasks.workunit.client.0.smithi037.stdout:4/707: creat d3/d57/fee x:0 0 0 2024-03-29T08:07:32.474 INFO:tasks.workunit.client.0.smithi037.stdout:0/550: symlink d9/lbc 0 2024-03-29T08:07:32.475 INFO:tasks.workunit.client.1.smithi148.stdout:6/983: dwrite d6/d9/d1e/d20/f128 [0,4194304] 0 2024-03-29T08:07:32.477 INFO:tasks.workunit.client.0.smithi037.stdout:8/613: getdents d5/d10/d2d/d34/d50 0 2024-03-29T08:07:32.478 INFO:tasks.workunit.client.0.smithi037.stdout:5/790: chown d8/f73 67562 1 2024-03-29T08:07:32.479 INFO:tasks.workunit.client.1.smithi148.stdout:1/949: truncate d3/d18/dae/dda/df3/faa 886750 0 2024-03-29T08:07:32.481 INFO:tasks.workunit.client.1.smithi148.stdout:6/984: symlink d6/d9/d1e/d36/d124/d12d/l137 0 2024-03-29T08:07:32.481 INFO:tasks.workunit.client.1.smithi148.stdout:1/950: dread - d3/de/d21/fc1 zero size 2024-03-29T08:07:32.482 INFO:tasks.workunit.client.0.smithi037.stdout:7/724: creat dd/d21/d35/d42/ddb/de2/ff2 x:0 0 0 2024-03-29T08:07:32.482 INFO:tasks.workunit.client.0.smithi037.stdout:8/614: write d5/d10/d2d/fc6 [199851,75214] 0 2024-03-29T08:07:32.485 INFO:tasks.workunit.client.0.smithi037.stdout:3/596: read dd/d46/f2c [1736396,61529] 0 2024-03-29T08:07:32.485 INFO:tasks.workunit.client.0.smithi037.stdout:0/551: fsync f5 0 2024-03-29T08:07:32.489 INFO:tasks.workunit.client.0.smithi037.stdout:5/791: dread - d8/d34/f80 zero size 2024-03-29T08:07:32.489 INFO:tasks.workunit.client.1.smithi148.stdout:5/960: dwrite d0/db/d1e/d35/d42/d67/d9a/dd7/fed [0,4194304] 0 2024-03-29T08:07:32.490 INFO:tasks.workunit.client.0.smithi037.stdout:4/708: symlink d3/d4/lef 0 2024-03-29T08:07:32.495 INFO:tasks.workunit.client.1.smithi148.stdout:1/951: creat d3/d18/d4d/d76/d85/d11d/d47/d6d/d72/d12b/f13c x:0 0 0 2024-03-29T08:07:32.501 INFO:tasks.workunit.client.1.smithi148.stdout:3/994: dread de/d24/d33/d6b/fe8 [0,4194304] 0 2024-03-29T08:07:32.503 INFO:tasks.workunit.client.1.smithi148.stdout:3/995: fsync de/d24/d33/f9a 0 2024-03-29T08:07:32.504 INFO:tasks.workunit.client.0.smithi037.stdout:7/725: creat dd/d21/d35/d42/d5c/d8c/ff3 x:0 0 0 2024-03-29T08:07:32.506 INFO:tasks.workunit.client.1.smithi148.stdout:1/952: rename d3/d18/d4d/d76/d85/d11d/d47/fe0 to d3/de/d25/df1/f13d 0 2024-03-29T08:07:32.507 INFO:tasks.workunit.client.1.smithi148.stdout:8/948: dread d17/d49/d9a/dbe/d6f/d64/da5/fb2 [0,4194304] 0 2024-03-29T08:07:32.507 INFO:tasks.workunit.client.0.smithi037.stdout:3/597: fsync dd/d46/d2d/d3a/d5f/f5d 0 2024-03-29T08:07:32.511 INFO:tasks.workunit.client.1.smithi148.stdout:6/985: dwrite d6/d9/d1e/d36/d4d/d9d/d28/db3/dd2/fe7 [0,4194304] 0 2024-03-29T08:07:32.515 INFO:tasks.workunit.client.1.smithi148.stdout:0/970: truncate db/de/df/d115/fa5 321845 0 2024-03-29T08:07:32.516 INFO:tasks.workunit.client.0.smithi037.stdout:5/792: truncate d8/d4d/d5c/f94 1846185 0 2024-03-29T08:07:32.519 INFO:tasks.workunit.client.1.smithi148.stdout:3/996: rename cd to de/d24/d33/d39/d46/d30/d37/d79/d10e/d145/c14b 0 2024-03-29T08:07:32.520 INFO:tasks.workunit.client.0.smithi037.stdout:4/709: rename d3/d57/d6b/faa to d3/d57/dec/ded/d1d/db/d72/ff0 0 2024-03-29T08:07:32.520 INFO:tasks.workunit.client.1.smithi148.stdout:5/961: link d0/db/d1e/d35/d42/d67/d9a/c126 d0/db/d7b/c130 0 2024-03-29T08:07:32.522 INFO:tasks.workunit.client.1.smithi148.stdout:8/949: dread - d17/d4e/fef zero size 2024-03-29T08:07:32.523 INFO:tasks.workunit.client.1.smithi148.stdout:6/986: symlink d6/d9/d1e/d27/d48/l138 0 2024-03-29T08:07:32.526 INFO:tasks.workunit.client.1.smithi148.stdout:3/997: mkdir de/d24/d33/d39/da3/dfe/d14c 0 2024-03-29T08:07:32.530 INFO:tasks.workunit.client.0.smithi037.stdout:3/598: dwrite dd/d15/d27/d33/f90 [0,4194304] 0 2024-03-29T08:07:32.533 INFO:tasks.workunit.client.1.smithi148.stdout:1/953: getdents d3/d18/dae/dda/df3 0 2024-03-29T08:07:32.535 INFO:tasks.workunit.client.0.smithi037.stdout:4/710: write d3/d57/dec/ded/d1d/db/fa3 [1036768,123785] 0 2024-03-29T08:07:32.535 INFO:tasks.workunit.client.1.smithi148.stdout:8/950: mknod d17/d18/d26/d63/d66/d5f/dd0/d119/d122/d133/c137 0 2024-03-29T08:07:32.538 INFO:tasks.workunit.client.1.smithi148.stdout:3/998: mknod de/d24/d33/d39/d46/d70/def/c14d 0 2024-03-29T08:07:32.539 INFO:tasks.workunit.client.0.smithi037.stdout:3/599: mknod dd/d15/d18/d4b/da2/cc4 0 2024-03-29T08:07:32.542 INFO:tasks.workunit.client.0.smithi037.stdout:5/793: getdents d8/d4d/d109 0 2024-03-29T08:07:32.543 INFO:tasks.workunit.client.1.smithi148.stdout:5/962: dwrite d0/db/d1e/d35/d42/d67/d9a/dd7/fed [0,4194304] 0 2024-03-29T08:07:32.544 INFO:tasks.workunit.client.0.smithi037.stdout:4/711: mknod d3/d57/dec/ded/d1d/db/d45/d1e/d59/db5/cf1 0 2024-03-29T08:07:32.546 INFO:tasks.workunit.client.0.smithi037.stdout:3/600: rename dd/d46/d2d/d3a/lb0 to dd/d46/d2d/d3a/d6c/lc5 0 2024-03-29T08:07:32.548 INFO:tasks.workunit.client.1.smithi148.stdout:6/987: dwrite d6/d9/d1e/d36/d4d/d9d/d28/d98/dc2/ffd [0,4194304] 0 2024-03-29T08:07:32.548 INFO:tasks.workunit.client.1.smithi148.stdout:1/954: rename d3/d18/dae/dda/df3/c10d to d3/de/d25/dfc/c13e 0 2024-03-29T08:07:32.549 INFO:tasks.workunit.client.0.smithi037.stdout:3/601: readlink dd/d15/d27/l74 0 2024-03-29T08:07:32.551 INFO:tasks.workunit.client.1.smithi148.stdout:7/942: dwrite d8/dfb/db8/d11e/f143 [0,4194304] 0 2024-03-29T08:07:32.552 INFO:tasks.workunit.client.0.smithi037.stdout:4/712: fsync f0 0 2024-03-29T08:07:32.553 INFO:tasks.workunit.client.1.smithi148.stdout:3/999: chown de/d24/d33/d39/d46/d30/d37/d79/d10e/d145 3 1 2024-03-29T08:07:32.554 INFO:tasks.workunit.client.0.smithi037.stdout:7/726: dread dd/d1d/d34/f37 [0,4194304] 0 2024-03-29T08:07:32.560 INFO:tasks.workunit.client.1.smithi148.stdout:5/963: rename d0/db/d7b/d16/d87 to d0/db/d84/daf/de1/d131 0 2024-03-29T08:07:32.561 INFO:tasks.workunit.client.0.smithi037.stdout:4/713: creat d3/d57/dec/ff2 x:0 0 0 2024-03-29T08:07:32.561 INFO:tasks.workunit.client.1.smithi148.stdout:1/955: symlink d3/de/d6e/df9/l13f 0 2024-03-29T08:07:32.563 INFO:tasks.workunit.client.0.smithi037.stdout:9/878: write d1/d4/d19/d1e/d4a/d6f/da2/d13/d7c/faf [1643596,75233] 0 2024-03-29T08:07:32.563 INFO:tasks.workunit.client.1.smithi148.stdout:6/988: write d6/d9/d1e/d36/d4d/d9d/d28/d98/dc2/f10a [691633,19574] 0 2024-03-29T08:07:32.564 INFO:tasks.workunit.client.0.smithi037.stdout:4/714: read d3/d57/dec/ded/f63 [7337790,79118] 0 2024-03-29T08:07:32.565 INFO:tasks.workunit.client.1.smithi148.stdout:1/956: stat d3/d18/da8 0 2024-03-29T08:07:32.565 INFO:tasks.workunit.client.0.smithi037.stdout:7/727: truncate dd/d21/d35/f50 4336196 0 2024-03-29T08:07:32.569 INFO:tasks.workunit.client.0.smithi037.stdout:7/728: write dd/d72/f98 [1876568,16922] 0 2024-03-29T08:07:32.569 INFO:tasks.workunit.client.1.smithi148.stdout:6/989: chown d6/d9/d1e/d20/d30/fc6 1386 1 2024-03-29T08:07:32.569 INFO:tasks.workunit.client.1.smithi148.stdout:0/971: dread db/de/df/d11/f41 [0,4194304] 0 2024-03-29T08:07:32.570 INFO:tasks.workunit.client.0.smithi037.stdout:0/552: dread - d9/da/dd/d13/f96 zero size 2024-03-29T08:07:32.572 INFO:tasks.workunit.client.0.smithi037.stdout:5/794: rename d8/d34/d4b to d8/d22/d4f/d70/d110 0 2024-03-29T08:07:32.572 INFO:tasks.workunit.client.1.smithi148.stdout:5/964: creat d0/db/d1e/d5b/d66/f132 x:0 0 0 2024-03-29T08:07:32.574 INFO:tasks.workunit.client.1.smithi148.stdout:6/990: readlink d6/d9/d1e/d27/d48/d73/lcf 0 2024-03-29T08:07:32.575 INFO:tasks.workunit.client.0.smithi037.stdout:5/795: write d8/db9/f101 [857635,47250] 0 2024-03-29T08:07:32.575 INFO:tasks.workunit.client.1.smithi148.stdout:7/943: truncate d8/dfb/d44/d50/f87 1157985 0 2024-03-29T08:07:32.576 INFO:tasks.workunit.client.0.smithi037.stdout:0/553: dread - d9/da/d1b/fb0 zero size 2024-03-29T08:07:32.576 INFO:tasks.workunit.client.1.smithi148.stdout:5/965: readlink d0/db/d7b/d16/l54 0 2024-03-29T08:07:32.578 INFO:tasks.workunit.client.0.smithi037.stdout:5/796: dread - d8/d22/db6/f107 zero size 2024-03-29T08:07:32.579 INFO:tasks.workunit.client.1.smithi148.stdout:8/951: dread d17/d18/d26/d4f/d94/f9b [4194304,4194304] 0 2024-03-29T08:07:32.581 INFO:tasks.workunit.client.0.smithi037.stdout:7/729: creat dd/d1d/d34/d54/ff4 x:0 0 0 2024-03-29T08:07:32.584 INFO:tasks.workunit.client.1.smithi148.stdout:6/991: creat d6/d9/d1e/d36/d4d/d9d/d28/dda/db6/df3/f139 x:0 0 0 2024-03-29T08:07:32.586 INFO:tasks.workunit.client.0.smithi037.stdout:0/554: read d9/da/dd/d54/f56 [2519570,108163] 0 2024-03-29T08:07:32.587 INFO:tasks.workunit.client.1.smithi148.stdout:0/972: rename db/dc/daa/d47/d4c/d5a/d7a/c7f to db/de/df/d11/d42/c12b 0 2024-03-29T08:07:32.589 INFO:tasks.workunit.client.1.smithi148.stdout:0/973: write db/de/d24/dbf/f100 [5896872,44793] 0 2024-03-29T08:07:32.590 INFO:tasks.workunit.client.0.smithi037.stdout:5/797: creat d8/d22/d37/dbc/f111 x:0 0 0 2024-03-29T08:07:32.592 INFO:tasks.workunit.client.1.smithi148.stdout:1/957: truncate d3/d18/dae/dda/df3/d42/f1a 707842 0 2024-03-29T08:07:32.592 INFO:tasks.workunit.client.0.smithi037.stdout:7/730: chown dd/d72/l83 56547 1 2024-03-29T08:07:32.595 INFO:tasks.workunit.client.1.smithi148.stdout:6/992: creat d6/d9/d1e/d20/db7/f13a x:0 0 0 2024-03-29T08:07:32.597 INFO:tasks.workunit.client.0.smithi037.stdout:8/615: truncate d5/f5b 3929066 0 2024-03-29T08:07:32.599 INFO:tasks.workunit.client.0.smithi037.stdout:5/798: write d8/d4d/d5c/dbf/fe4 [1705626,112918] 0 2024-03-29T08:07:32.600 INFO:tasks.workunit.client.1.smithi148.stdout:5/966: dwrite d0/db/d7b/fe3 [0,4194304] 0 2024-03-29T08:07:32.600 INFO:tasks.workunit.client.1.smithi148.stdout:1/958: creat d3/d18/d4d/d76/f140 x:0 0 0 2024-03-29T08:07:32.601 INFO:tasks.workunit.client.1.smithi148.stdout:6/993: write d6/d9/d1e/d36/d4d/d9d/d28/dda/db6/ffa [50590,64717] 0 2024-03-29T08:07:32.603 INFO:tasks.workunit.client.1.smithi148.stdout:6/994: fsync d6/d9/fdc 0 2024-03-29T08:07:32.603 INFO:tasks.workunit.client.0.smithi037.stdout:8/616: dread - d5/d10/d19/d3d/d4a/f61 zero size 2024-03-29T08:07:32.604 INFO:tasks.workunit.client.1.smithi148.stdout:8/952: creat d17/d18/d26/d63/d66/f138 x:0 0 0 2024-03-29T08:07:32.606 INFO:tasks.workunit.client.1.smithi148.stdout:5/967: read d0/db/f3e [853,17169] 0 2024-03-29T08:07:32.610 INFO:tasks.workunit.client.1.smithi148.stdout:5/968: write d0/db/d1e/d35/d42/d67/d76/d81/d98/f111 [1635436,32275] 0 2024-03-29T08:07:32.610 INFO:tasks.workunit.client.1.smithi148.stdout:1/959: symlink d3/d18/d4d/d76/d85/d11d/d47/d6d/d84/d8c/d129/l141 0 2024-03-29T08:07:32.611 INFO:tasks.workunit.client.0.smithi037.stdout:3/602: write dd/d15/d18/faf [2549944,116678] 0 2024-03-29T08:07:32.617 INFO:tasks.workunit.client.0.smithi037.stdout:9/879: dread d1/d3/f21 [0,4194304] 0 2024-03-29T08:07:32.617 INFO:tasks.workunit.client.0.smithi037.stdout:8/617: symlink d5/d10/d19/d3d/d4a/d4c/d67/d85/lca 0 2024-03-29T08:07:32.620 INFO:tasks.workunit.client.1.smithi148.stdout:5/969: symlink d0/db/d7b/d16/d2e/ddd/l133 0 2024-03-29T08:07:32.622 INFO:tasks.workunit.client.1.smithi148.stdout:6/995: link d6/f19 d6/d9/d1e/d36/d124/f13b 0 2024-03-29T08:07:32.622 INFO:tasks.workunit.client.1.smithi148.stdout:7/944: write d8/dfb/d11b/ff5 [876440,61032] 0 2024-03-29T08:07:32.622 INFO:tasks.workunit.client.0.smithi037.stdout:3/603: fdatasync dd/ff 0 2024-03-29T08:07:32.627 INFO:tasks.workunit.client.1.smithi148.stdout:6/996: unlink d6/d9/f52 0 2024-03-29T08:07:32.630 INFO:tasks.workunit.client.0.smithi037.stdout:0/555: dread d9/da/f8d [0,4194304] 0 2024-03-29T08:07:32.630 INFO:tasks.workunit.client.1.smithi148.stdout:5/970: read d0/d46/d9e/fc1 [3814655,105216] 0 2024-03-29T08:07:32.630 INFO:tasks.workunit.client.1.smithi148.stdout:1/960: dwrite d3/d18/d4d/d76/d85/d11d/d47/d6d/d84/d8c/fce [0,4194304] 0 2024-03-29T08:07:32.632 INFO:tasks.workunit.client.0.smithi037.stdout:0/556: dread - d9/da/dd/d13/f96 zero size 2024-03-29T08:07:32.636 INFO:tasks.workunit.client.1.smithi148.stdout:6/997: creat d6/d9/d1e/d36/d4d/d9d/d28/dda/db6/f13c x:0 0 0 2024-03-29T08:07:32.637 INFO:tasks.workunit.client.1.smithi148.stdout:7/945: mkdir d8/d38/d3c/da5/db4/de9/d114/d145 0 2024-03-29T08:07:32.638 INFO:tasks.workunit.client.1.smithi148.stdout:1/961: stat d3/d18/d4d/d76/d85/d11d/d101/fa6 0 2024-03-29T08:07:32.642 INFO:tasks.workunit.client.1.smithi148.stdout:5/971: creat d0/db/d1e/d35/d42/d67/d76/f134 x:0 0 0 2024-03-29T08:07:32.643 INFO:tasks.workunit.client.1.smithi148.stdout:6/998: creat d6/d9/d1e/d20/d6b/f13d x:0 0 0 2024-03-29T08:07:32.645 INFO:tasks.workunit.client.1.smithi148.stdout:5/972: fsync d0/db/d1e/d6e/da8/f100 0 2024-03-29T08:07:32.645 INFO:tasks.workunit.client.1.smithi148.stdout:1/962: creat d3/d18/d4d/d9e/d12f/f142 x:0 0 0 2024-03-29T08:07:32.645 INFO:tasks.workunit.client.1.smithi148.stdout:0/974: dwrite db/de/df/d11/d42/f6e [0,4194304] 0 2024-03-29T08:07:32.645 INFO:tasks.workunit.client.0.smithi037.stdout:0/557: rmdir d9/da/dd/d13/d7b 39 2024-03-29T08:07:32.648 INFO:tasks.workunit.client.1.smithi148.stdout:5/973: rmdir d0/db/d1e/d6e 39 2024-03-29T08:07:32.652 INFO:tasks.workunit.client.0.smithi037.stdout:9/880: rename d1/d4/d19/d1e/d4a/d6f/da2/d31/daa/dba/c107 to d1/d4/d19/d1e/d4a/d6f/da2/d68/c128 0 2024-03-29T08:07:32.652 INFO:tasks.workunit.client.0.smithi037.stdout:7/731: dread dd/d21/d35/d3b/f97 [0,4194304] 0 2024-03-29T08:07:32.654 INFO:tasks.workunit.client.1.smithi148.stdout:6/999: fsync d6/d9/d1e/d36/d4d/d9d/d28/dda/f92 0 2024-03-29T08:07:32.654 INFO:tasks.workunit.client.0.smithi037.stdout:3/604: truncate dd/d15/d27/f41 1851976 0 2024-03-29T08:07:32.656 INFO:tasks.workunit.client.1.smithi148.stdout:7/946: write d8/dfb/d11b/d37/f9c [1099704,113060] 0 2024-03-29T08:07:32.658 INFO:tasks.workunit.client.1.smithi148.stdout:5/974: truncate d0/db/d1e/d6e/da8/f100 522261 0 2024-03-29T08:07:32.658 INFO:tasks.workunit.client.0.smithi037.stdout:5/799: creat d8/d22/d37/dbc/f112 x:0 0 0 2024-03-29T08:07:32.659 INFO:tasks.workunit.client.0.smithi037.stdout:0/558: symlink d9/da/dd/d13/lbd 0 2024-03-29T08:07:32.661 INFO:tasks.workunit.client.0.smithi037.stdout:8/618: dwrite d5/d10/d2d/d9c/da1/fa3 [0,4194304] 0 2024-03-29T08:07:32.661 INFO:tasks.workunit.client.0.smithi037.stdout:9/881: chown d1/d4/d19/d1e/d4a/d6f/cb5 1023225178 1 2024-03-29T08:07:32.662 INFO:tasks.workunit.client.1.smithi148.stdout:5/975: stat d0/db/d1e/d35/d42/fde 0 2024-03-29T08:07:32.665 INFO:tasks.workunit.client.1.smithi148.stdout:5/976: rmdir d0/db/d1e/d5b/d66 39 2024-03-29T08:07:32.665 INFO:tasks.workunit.client.1.smithi148.stdout:0/975: dwrite db/dc/daa/ff2 [0,4194304] 0 2024-03-29T08:07:32.666 INFO:tasks.workunit.client.0.smithi037.stdout:3/605: symlink dd/d15/d27/d33/d84/lc6 0 2024-03-29T08:07:32.669 INFO:tasks.workunit.client.1.smithi148.stdout:7/947: dread d8/f10 [0,4194304] 0 2024-03-29T08:07:32.672 INFO:tasks.workunit.client.1.smithi148.stdout:7/948: truncate d8/dfb/f8c 1313914 0 2024-03-29T08:07:32.674 INFO:tasks.workunit.client.1.smithi148.stdout:0/976: creat db/de/df/d11/d1f/dc5/f12c x:0 0 0 2024-03-29T08:07:32.674 INFO:tasks.workunit.client.0.smithi037.stdout:0/559: symlink d9/da/dd/d54/da8/lbe 0 2024-03-29T08:07:32.676 INFO:tasks.workunit.client.1.smithi148.stdout:5/977: chown d0/db/d1e/d6e/da8/fdc 250194147 1 2024-03-29T08:07:32.678 INFO:tasks.workunit.client.0.smithi037.stdout:3/606: truncate dd/d15/d27/d33/d83/fb9 1418945 0 2024-03-29T08:07:32.679 INFO:tasks.workunit.client.1.smithi148.stdout:7/949: creat d8/d38/d3c/da5/db4/de9/d114/f146 x:0 0 0 2024-03-29T08:07:32.682 INFO:tasks.workunit.client.0.smithi037.stdout:8/619: link d5/d10/d2d/d9c/da1/fa3 d5/d10/d2d/fcb 0 2024-03-29T08:07:32.683 INFO:tasks.workunit.client.1.smithi148.stdout:0/977: mkdir db/de/df/d11/d2d/dde/d12d 0 2024-03-29T08:07:32.686 INFO:tasks.workunit.client.0.smithi037.stdout:7/732: dwrite dd/d21/d70/fb2 [0,4194304] 0 2024-03-29T08:07:32.687 INFO:tasks.workunit.client.0.smithi037.stdout:0/560: creat d9/d74/d7d/d7e/fbf x:0 0 0 2024-03-29T08:07:32.694 INFO:tasks.workunit.client.1.smithi148.stdout:1/963: link d3/d18/dae/dda/df3/d42/f1a d3/d18/d4d/d9e/d12f/f143 0 2024-03-29T08:07:32.694 INFO:tasks.workunit.client.0.smithi037.stdout:8/620: creat d5/d10/d19/d3d/d4a/d4c/d67/d70/fcc x:0 0 0 2024-03-29T08:07:32.695 INFO:tasks.workunit.client.1.smithi148.stdout:0/978: read db/de/d8e/da9/fca [461580,129445] 0 2024-03-29T08:07:32.695 INFO:tasks.workunit.client.1.smithi148.stdout:8/953: sync 2024-03-29T08:07:32.696 INFO:tasks.workunit.client.0.smithi037.stdout:3/607: creat dd/d15/fc7 x:0 0 0 2024-03-29T08:07:32.697 INFO:tasks.workunit.client.0.smithi037.stdout:6/584: sync 2024-03-29T08:07:32.697 INFO:tasks.workunit.client.1.smithi148.stdout:5/978: dwrite d0/db/d1e/d35/d42/fde [0,4194304] 0 2024-03-29T08:07:32.698 INFO:tasks.workunit.client.0.smithi037.stdout:8/621: readlink d5/d10/d19/d3d/d4a/d4c/l56 0 2024-03-29T08:07:32.700 INFO:tasks.workunit.client.0.smithi037.stdout:8/622: readlink d5/d10/d2d/d34/d50/d83/d46/d4e/l55 0 2024-03-29T08:07:32.705 INFO:tasks.workunit.client.1.smithi148.stdout:1/964: rename d3/d18/l3b to d3/d18/d71/ddc/l144 0 2024-03-29T08:07:32.707 INFO:tasks.workunit.client.1.smithi148.stdout:5/979: creat d0/db/d1e/d35/d42/d67/d76/d81/d98/dd5/dd1/d121/f135 x:0 0 0 2024-03-29T08:07:32.708 INFO:tasks.workunit.client.1.smithi148.stdout:0/979: mknod db/de/df/d11/d42/c12e 0 2024-03-29T08:07:32.712 INFO:tasks.workunit.client.1.smithi148.stdout:1/965: creat d3/de/d6e/df9/dfa/f145 x:0 0 0 2024-03-29T08:07:32.712 INFO:tasks.workunit.client.0.smithi037.stdout:9/882: link d1/d4/d19/d1e/d4a/d6f/da2/d31/d87/c109 d1/d4/d19/d1e/d4a/d6f/da2/d31/d87/c129 0 2024-03-29T08:07:32.713 INFO:tasks.workunit.client.1.smithi148.stdout:5/980: unlink d0/db/d84/daf/lc0 0 2024-03-29T08:07:32.717 INFO:tasks.workunit.client.0.smithi037.stdout:8/623: creat d5/d10/d19/d3d/d4a/d4c/fcd x:0 0 0 2024-03-29T08:07:32.718 INFO:tasks.workunit.client.1.smithi148.stdout:8/954: dwrite d17/d18/d26/d4f/fb7 [0,4194304] 0 2024-03-29T08:07:32.720 INFO:tasks.workunit.client.1.smithi148.stdout:5/981: read d0/db/d7b/d16/f2f [3718850,109320] 0 2024-03-29T08:07:32.722 INFO:tasks.workunit.client.1.smithi148.stdout:8/955: read - d17/d18/d26/d63/d66/f138 zero size 2024-03-29T08:07:32.723 INFO:tasks.workunit.client.0.smithi037.stdout:5/800: dwrite d8/d22/d4f/d70/d9c/f97 [0,4194304] 0 2024-03-29T08:07:32.728 INFO:tasks.workunit.client.1.smithi148.stdout:5/982: dread - d0/db/d1e/d36/d41/f59 zero size 2024-03-29T08:07:32.730 INFO:tasks.workunit.client.0.smithi037.stdout:0/561: rename d9/da/d1b/d22/c75 to d9/da/dd/d20/cc0 0 2024-03-29T08:07:32.731 INFO:tasks.workunit.client.1.smithi148.stdout:8/956: symlink d17/d18/d26/d4f/l139 0 2024-03-29T08:07:32.732 INFO:tasks.workunit.client.0.smithi037.stdout:7/733: link dd/d1d/d49/ccc dd/d1d/d34/d54/db4/cf5 0 2024-03-29T08:07:32.733 INFO:tasks.workunit.client.0.smithi037.stdout:0/562: chown d9/da/dd/d20/d55 31 1 2024-03-29T08:07:32.736 INFO:tasks.workunit.client.1.smithi148.stdout:1/966: dwrite d3/de/d21/f80 [0,4194304] 0 2024-03-29T08:07:32.737 INFO:tasks.workunit.client.1.smithi148.stdout:7/950: dwrite d8/dfb/d44/d50/f8b [0,4194304] 0 2024-03-29T08:07:32.739 INFO:tasks.workunit.client.0.smithi037.stdout:6/585: write d0/d5/f73 [124034,84727] 0 2024-03-29T08:07:32.740 INFO:tasks.workunit.client.1.smithi148.stdout:5/983: dread d0/db/d1e/d36/d41/f63 [4194304,4194304] 0 2024-03-29T08:07:32.746 INFO:tasks.workunit.client.0.smithi037.stdout:6/586: creat d0/d6a/d7b/fd1 x:0 0 0 2024-03-29T08:07:32.750 INFO:tasks.workunit.client.1.smithi148.stdout:1/967: creat d3/de/d4f/f146 x:0 0 0 2024-03-29T08:07:32.750 INFO:tasks.workunit.client.0.smithi037.stdout:0/563: symlink d9/da/dd/d13/d7b/lc1 0 2024-03-29T08:07:32.751 INFO:tasks.workunit.client.1.smithi148.stdout:7/951: mkdir d8/d11d/d147 0 2024-03-29T08:07:32.753 INFO:tasks.workunit.client.0.smithi037.stdout:8/624: dwrite d5/d10/d45/fc2 [0,4194304] 0 2024-03-29T08:07:32.754 INFO:tasks.workunit.client.1.smithi148.stdout:5/984: mknod d0/d46/c136 0 2024-03-29T08:07:32.756 INFO:tasks.workunit.client.1.smithi148.stdout:1/968: creat d3/de/d4f/f147 x:0 0 0 2024-03-29T08:07:32.758 INFO:tasks.workunit.client.1.smithi148.stdout:7/952: truncate d8/fe 2268373 0 2024-03-29T08:07:32.759 INFO:tasks.workunit.client.0.smithi037.stdout:0/564: dread - d9/da/dd/f99 zero size 2024-03-29T08:07:32.759 INFO:tasks.workunit.client.0.smithi037.stdout:5/801: write d8/d22/d37/d8d/db3/dcb/deb/fa9 [588575,54024] 0 2024-03-29T08:07:32.760 INFO:tasks.workunit.client.1.smithi148.stdout:0/980: dwrite db/dc/daa/d47/f2c [4194304,4194304] 0 2024-03-29T08:07:32.761 INFO:tasks.workunit.client.0.smithi037.stdout:6/587: creat d0/d5/d1a/fd2 x:0 0 0 2024-03-29T08:07:32.762 INFO:tasks.workunit.client.0.smithi037.stdout:5/802: chown d8/d4d/d72 405198 1 2024-03-29T08:07:32.765 INFO:tasks.workunit.client.0.smithi037.stdout:3/608: dread dd/d46/d2d/d3a/d6c/db1/fbe [0,4194304] 0 2024-03-29T08:07:32.765 INFO:tasks.workunit.client.0.smithi037.stdout:0/565: dread - d9/da/dd/d20/d38/d53/f6a zero size 2024-03-29T08:07:32.766 INFO:tasks.workunit.client.1.smithi148.stdout:1/969: symlink d3/d18/d71/d11b/l148 0 2024-03-29T08:07:32.770 INFO:tasks.workunit.client.1.smithi148.stdout:7/953: link d8/l107 d8/dfb/d11b/d37/dc8/l148 0 2024-03-29T08:07:32.775 INFO:tasks.workunit.client.1.smithi148.stdout:7/954: fsync d8/d38/f71 0 2024-03-29T08:07:32.775 INFO:tasks.workunit.client.1.smithi148.stdout:1/970: creat d3/d18/dae/dda/f149 x:0 0 0 2024-03-29T08:07:32.776 INFO:tasks.workunit.client.0.smithi037.stdout:8/625: symlink d5/d10/d2d/d34/d50/d83/d46/d4e/lce 0 2024-03-29T08:07:32.777 INFO:tasks.workunit.client.1.smithi148.stdout:8/957: write d17/d74/d86/fe5 [3843413,107007] 0 2024-03-29T08:07:32.780 INFO:tasks.workunit.client.0.smithi037.stdout:3/609: truncate dd/d46/d2d/f60 232288 0 2024-03-29T08:07:32.782 INFO:tasks.workunit.client.0.smithi037.stdout:9/883: dread d1/d3/d44/f8c [0,4194304] 0 2024-03-29T08:07:32.783 INFO:tasks.workunit.client.1.smithi148.stdout:1/971: unlink d3/d55/d7e/fd7 0 2024-03-29T08:07:32.783 INFO:tasks.workunit.client.0.smithi037.stdout:7/734: dread dd/d1d/d49/f5a [0,4194304] 0 2024-03-29T08:07:32.786 INFO:tasks.workunit.client.1.smithi148.stdout:7/955: creat d8/dfb/db8/d115/f149 x:0 0 0 2024-03-29T08:07:32.786 INFO:tasks.workunit.client.1.smithi148.stdout:5/985: dwrite d0/db/d1e/d35/d42/d67/d76/d81/d98/dd5/dd1/d121/f135 [0,4194304] 0 2024-03-29T08:07:32.789 INFO:tasks.workunit.client.1.smithi148.stdout:0/981: dwrite db/de/df/d11/d42/f52 [0,4194304] 0 2024-03-29T08:07:32.789 INFO:tasks.workunit.client.0.smithi037.stdout:7/735: chown dd/d21/d85/fab 24714922 1 2024-03-29T08:07:32.789 INFO:tasks.workunit.client.1.smithi148.stdout:5/986: readlink d0/db/d1e/d36/d7c/le7 0 2024-03-29T08:07:32.790 INFO:tasks.workunit.client.0.smithi037.stdout:8/626: unlink d5/c24 0 2024-03-29T08:07:32.791 INFO:tasks.workunit.client.0.smithi037.stdout:5/803: dwrite d8/d18/f21 [0,4194304] 0 2024-03-29T08:07:32.792 INFO:tasks.workunit.client.0.smithi037.stdout:0/566: link d9/da/dd/d59/caf d9/da/dd/d20/d38/cc2 0 2024-03-29T08:07:32.807 INFO:tasks.workunit.client.1.smithi148.stdout:0/982: creat db/de/df/d2b/dae/d116/d125/f12f x:0 0 0 2024-03-29T08:07:32.810 INFO:tasks.workunit.client.0.smithi037.stdout:6/588: truncate d0/d19/d42/d60/f8b 8986734 0 2024-03-29T08:07:32.810 INFO:tasks.workunit.client.1.smithi148.stdout:0/983: readlink db/dc/daa/l57 0 2024-03-29T08:07:32.811 INFO:tasks.workunit.client.1.smithi148.stdout:5/987: dwrite d0/db/d1e/d35/d42/d67/d76/d81/d98/f111 [0,4194304] 0 2024-03-29T08:07:32.812 INFO:tasks.workunit.client.1.smithi148.stdout:8/958: write f15 [6435980,94863] 0 2024-03-29T08:07:32.813 INFO:tasks.workunit.client.1.smithi148.stdout:1/972: dwrite d3/d18/dae/dda/f118 [0,4194304] 0 2024-03-29T08:07:32.813 INFO:tasks.workunit.client.0.smithi037.stdout:8/627: mknod d5/ccf 0 2024-03-29T08:07:32.816 INFO:tasks.workunit.client.0.smithi037.stdout:9/884: getdents d1/d1f/d3a/d9b/ddd 0 2024-03-29T08:07:32.818 INFO:tasks.workunit.client.0.smithi037.stdout:3/610: write dd/fe [2676846,12901] 0 2024-03-29T08:07:32.820 INFO:tasks.workunit.client.0.smithi037.stdout:7/736: link dd/d21/d35/d60/d94/fb5 dd/d1d/d49/d81/d84/dca/ff6 0 2024-03-29T08:07:32.820 INFO:tasks.workunit.client.1.smithi148.stdout:1/973: mkdir d3/d55/ddf/d14a 0 2024-03-29T08:07:32.821 INFO:tasks.workunit.client.1.smithi148.stdout:5/988: fdatasync d0/db/d1e/d36/f47 0 2024-03-29T08:07:32.824 INFO:tasks.workunit.client.0.smithi037.stdout:5/804: link d8/c12 d8/d22/dde/c113 0 2024-03-29T08:07:32.827 INFO:tasks.workunit.client.0.smithi037.stdout:5/805: readlink d8/d22/d37/d8d/db3/dcb/deb/d61/la4 0 2024-03-29T08:07:32.828 INFO:tasks.workunit.client.1.smithi148.stdout:1/974: mknod d3/d18/dae/c14b 0 2024-03-29T08:07:32.830 INFO:tasks.workunit.client.0.smithi037.stdout:9/885: rmdir d1/d4/d19/d1e/d71/d89/d92 39 2024-03-29T08:07:32.831 INFO:tasks.workunit.client.0.smithi037.stdout:5/806: readlink d8/d22/d37/d8d/db3/dcb/deb/d88/l98 0 2024-03-29T08:07:32.832 INFO:tasks.workunit.client.1.smithi148.stdout:1/975: write d3/d18/d4d/d8f/f6b [3175435,31672] 0 2024-03-29T08:07:32.832 INFO:tasks.workunit.client.0.smithi037.stdout:0/567: dwrite d9/da/dd/d13/f95 [0,4194304] 0 2024-03-29T08:07:32.837 INFO:tasks.workunit.client.0.smithi037.stdout:7/737: mknod dd/d21/d89/cf7 0 2024-03-29T08:07:32.837 INFO:tasks.workunit.client.1.smithi148.stdout:0/984: link db/de/df/d11/f41 db/de/df/d2b/f130 0 2024-03-29T08:07:32.838 INFO:tasks.workunit.client.0.smithi037.stdout:3/611: symlink dd/d46/d2d/d3a/d6c/db1/dc0/lc8 0 2024-03-29T08:07:32.838 INFO:tasks.workunit.client.1.smithi148.stdout:5/989: rename d0/db/d1e/d36/f47 to d0/db/d84/daf/f137 0 2024-03-29T08:07:32.838 INFO:tasks.workunit.client.0.smithi037.stdout:8/628: symlink d5/d10/ld0 0 2024-03-29T08:07:32.839 INFO:tasks.workunit.client.0.smithi037.stdout:7/738: chown dd/d72/ca9 69784 1 2024-03-29T08:07:32.839 INFO:tasks.workunit.client.0.smithi037.stdout:5/807: creat d8/d4d/ddf/df2/f114 x:0 0 0 2024-03-29T08:07:32.844 INFO:tasks.workunit.client.1.smithi148.stdout:0/985: mkdir db/de/df/d14/dea/d122/d131 0 2024-03-29T08:07:32.845 INFO:tasks.workunit.client.0.smithi037.stdout:7/739: dread - dd/d21/d35/d42/ddb/fee zero size 2024-03-29T08:07:32.846 INFO:tasks.workunit.client.1.smithi148.stdout:0/986: chown db/de/d8e/da9/f126 128443668 1 2024-03-29T08:07:32.847 INFO:tasks.workunit.client.0.smithi037.stdout:7/740: stat dd/c23 0 2024-03-29T08:07:32.847 INFO:tasks.workunit.client.1.smithi148.stdout:1/976: mknod d3/d18/d4d/c14c 0 2024-03-29T08:07:32.848 INFO:tasks.workunit.client.0.smithi037.stdout:8/629: creat d5/d10/d2d/da7/fd1 x:0 0 0 2024-03-29T08:07:32.854 INFO:tasks.workunit.client.0.smithi037.stdout:0/568: getdents d9/da/dd/d20/d38 0 2024-03-29T08:07:32.859 INFO:tasks.workunit.client.0.smithi037.stdout:7/741: rename dd/d1d/d34/d54 to dd/d21/d65/da2/df8 0 2024-03-29T08:07:32.861 INFO:tasks.workunit.client.1.smithi148.stdout:8/959: write d17/d18/f1c [77110,11789] 0 2024-03-29T08:07:32.862 INFO:tasks.workunit.client.0.smithi037.stdout:1/685: sync 2024-03-29T08:07:32.863 INFO:tasks.workunit.client.0.smithi037.stdout:8/630: rmdir d5/d10/d2d/d34/d50/d83/d46/d4e 39 2024-03-29T08:07:32.866 INFO:tasks.workunit.client.0.smithi037.stdout:0/569: truncate d9/d74/f97 962067 0 2024-03-29T08:07:32.867 INFO:tasks.workunit.client.1.smithi148.stdout:8/960: rmdir d17/d49/d9a/db8/dff/d116 39 2024-03-29T08:07:32.871 INFO:tasks.workunit.client.0.smithi037.stdout:1/686: creat d2/d6/dc/d58/fed x:0 0 0 2024-03-29T08:07:32.873 INFO:tasks.workunit.client.1.smithi148.stdout:5/990: dread d0/db/d1e/d5b/fda [0,4194304] 0 2024-03-29T08:07:32.875 INFO:tasks.workunit.client.1.smithi148.stdout:0/987: rename db/de/df/d14/dea/d122 to db/de/d132 0 2024-03-29T08:07:32.876 INFO:tasks.workunit.client.0.smithi037.stdout:8/631: mknod d5/d10/d2d/d34/d50/d83/d46/d8a/d96/cd2 0 2024-03-29T08:07:32.876 INFO:tasks.workunit.client.1.smithi148.stdout:5/991: creat d0/db/d1e/d35/d42/f138 x:0 0 0 2024-03-29T08:07:32.879 INFO:tasks.workunit.client.1.smithi148.stdout:5/992: stat d0/db/d1e/d36/d7c/le7 0 2024-03-29T08:07:32.879 INFO:tasks.workunit.client.0.smithi037.stdout:0/570: creat d9/da/dd/d20/d55/d5f/fc3 x:0 0 0 2024-03-29T08:07:32.880 INFO:tasks.workunit.client.0.smithi037.stdout:9/886: dread d1/d4/d19/d1e/d4a/d6f/da2/d23/f9a [0,4194304] 0 2024-03-29T08:07:32.883 INFO:tasks.workunit.client.1.smithi148.stdout:5/993: write d0/db/d84/daf/ffa [4346588,84101] 0 2024-03-29T08:07:32.883 INFO:tasks.workunit.client.1.smithi148.stdout:1/977: dread d3/d18/d4d/d76/d85/d11d/fa1 [0,4194304] 0 2024-03-29T08:07:32.884 INFO:tasks.workunit.client.1.smithi148.stdout:8/961: dwrite d17/d49/d9a/db8/f132 [0,4194304] 0 2024-03-29T08:07:32.884 INFO:tasks.workunit.client.0.smithi037.stdout:6/589: dwrite d0/d19/d34/d5a/f39 [4194304,4194304] 0 2024-03-29T08:07:32.887 INFO:tasks.workunit.client.0.smithi037.stdout:9/887: dread - d1/d4/d95/f119 zero size 2024-03-29T08:07:32.888 INFO:tasks.workunit.client.0.smithi037.stdout:1/687: symlink d2/d6/dc/d18/lee 0 2024-03-29T08:07:32.891 INFO:tasks.workunit.client.0.smithi037.stdout:5/808: dread d8/d22/f27 [0,4194304] 0 2024-03-29T08:07:32.892 INFO:tasks.workunit.client.0.smithi037.stdout:7/742: rmdir dd/d21/d65/da2/df8 39 2024-03-29T08:07:32.895 INFO:tasks.workunit.client.0.smithi037.stdout:7/743: fsync dd/d21/d85/fea 0 2024-03-29T08:07:32.900 INFO:tasks.workunit.client.1.smithi148.stdout:8/962: unlink d17/d49/d9a/dbe/la6 0 2024-03-29T08:07:32.900 INFO:tasks.workunit.client.0.smithi037.stdout:3/612: dwrite dd/d5e/f5c [4194304,4194304] 0 2024-03-29T08:07:32.902 INFO:tasks.workunit.client.1.smithi148.stdout:0/988: link db/de/d8e/df1/ff3 db/de/df/d2b/dae/f133 0 2024-03-29T08:07:32.903 INFO:tasks.workunit.client.1.smithi148.stdout:5/994: dwrite d0/db/d7b/d16/f7a [0,4194304] 0 2024-03-29T08:07:32.903 INFO:tasks.workunit.client.1.smithi148.stdout:0/989: stat db/dc/daa/c4e 0 2024-03-29T08:07:32.904 INFO:tasks.workunit.client.0.smithi037.stdout:3/613: write dd/d15/fc7 [39734,48334] 0 2024-03-29T08:07:32.907 INFO:tasks.workunit.client.0.smithi037.stdout:8/632: truncate d5/d10/d19/d8d/f90 1704404 0 2024-03-29T08:07:32.907 INFO:tasks.workunit.client.1.smithi148.stdout:0/990: stat db/de/df/d11/d1f/c8c 0 2024-03-29T08:07:32.911 INFO:tasks.workunit.client.1.smithi148.stdout:5/995: fdatasync d0/db/d7b/d16/d2e/f56 0 2024-03-29T08:07:32.912 INFO:tasks.workunit.client.0.smithi037.stdout:5/809: rmdir d8/d4d 39 2024-03-29T08:07:32.915 INFO:tasks.workunit.client.0.smithi037.stdout:8/633: write d5/d10/d45/fc2 [2686362,106480] 0 2024-03-29T08:07:32.917 INFO:tasks.workunit.client.1.smithi148.stdout:8/963: dread d17/d49/d9a/dbe/d6f/fcc [0,4194304] 0 2024-03-29T08:07:32.918 INFO:tasks.workunit.client.0.smithi037.stdout:5/810: write d8/d34/fcf [2812477,81682] 0 2024-03-29T08:07:32.919 INFO:tasks.workunit.client.1.smithi148.stdout:5/996: truncate d0/db/d1e/d35/d42/f12e 984370 0 2024-03-29T08:07:32.920 INFO:tasks.workunit.client.1.smithi148.stdout:8/964: rename d17 to d17/d74/d103/d13a 22 2024-03-29T08:07:32.921 INFO:tasks.workunit.client.1.smithi148.stdout:0/991: rmdir db/de/df/d11 39 2024-03-29T08:07:32.921 INFO:tasks.workunit.client.1.smithi148.stdout:7/956: sync 2024-03-29T08:07:32.923 INFO:tasks.workunit.client.0.smithi037.stdout:0/571: mknod d9/da/dd/d54/d92/cc4 0 2024-03-29T08:07:32.924 INFO:tasks.workunit.client.0.smithi037.stdout:6/590: mknod d0/d5/d1f/da8/dd0/cd3 0 2024-03-29T08:07:32.925 INFO:tasks.workunit.client.1.smithi148.stdout:7/957: write d8/f134 [1157368,125930] 0 2024-03-29T08:07:32.926 INFO:tasks.workunit.client.0.smithi037.stdout:1/688: mknod d2/d6/dc/cef 0 2024-03-29T08:07:32.926 INFO:tasks.workunit.client.0.smithi037.stdout:6/591: chown d0/fcf 3683 1 2024-03-29T08:07:32.927 INFO:tasks.workunit.client.1.smithi148.stdout:5/997: dread - d0/db/d1e/d5b/d66/f132 zero size 2024-03-29T08:07:32.927 INFO:tasks.workunit.client.1.smithi148.stdout:1/978: dwrite d3/d18/d4d/d76/d85/d11d/d47/d6d/d84/f92 [0,4194304] 0 2024-03-29T08:07:32.928 INFO:tasks.workunit.client.0.smithi037.stdout:3/614: dread dd/f1b [0,4194304] 0 2024-03-29T08:07:32.930 INFO:tasks.workunit.client.0.smithi037.stdout:8/634: chown d5/c3c 3507 1 2024-03-29T08:07:32.931 INFO:tasks.workunit.client.1.smithi148.stdout:0/992: read db/de/df/d11/d42/fd0 [1898699,108936] 0 2024-03-29T08:07:32.931 INFO:tasks.workunit.client.1.smithi148.stdout:7/958: truncate d8/dfb/fd4 418397 0 2024-03-29T08:07:32.934 INFO:tasks.workunit.client.1.smithi148.stdout:7/959: stat d8/dfb/d44/d50/d94/lb1 0 2024-03-29T08:07:32.935 INFO:tasks.workunit.client.0.smithi037.stdout:7/744: rename dd/d21/d35/d3b/fbd to dd/d21/d65/da2/df8/ff9 0 2024-03-29T08:07:32.938 INFO:tasks.workunit.client.0.smithi037.stdout:6/592: symlink d0/d19/d42/d60/ld4 0 2024-03-29T08:07:32.942 INFO:tasks.workunit.client.0.smithi037.stdout:6/593: truncate d0/d20/f31 4644559 0 2024-03-29T08:07:32.945 INFO:tasks.workunit.client.1.smithi148.stdout:1/979: dread d3/d18/dae/dda/f118 [0,4194304] 0 2024-03-29T08:07:32.945 INFO:tasks.workunit.client.0.smithi037.stdout:6/594: chown d0/d5/f14 836058 1 2024-03-29T08:07:32.948 INFO:tasks.workunit.client.1.smithi148.stdout:5/998: dwrite d0/db/d1e/d35/d42/d96/fdf [0,4194304] 0 2024-03-29T08:07:32.948 INFO:tasks.workunit.client.0.smithi037.stdout:3/615: write dd/d5e/fbb [3400098,1950] 0 2024-03-29T08:07:32.951 INFO:tasks.workunit.client.1.smithi148.stdout:0/993: rename db/de/df/d2b/db9 to db/de/df/d11/d2d/dde/d12d/d134 0 2024-03-29T08:07:32.956 INFO:tasks.workunit.client.0.smithi037.stdout:0/572: rename d9/da/dd/d59/l19 to d9/da/dd/d54/d92/lc5 0 2024-03-29T08:07:32.957 INFO:tasks.workunit.client.1.smithi148.stdout:5/999: creat d0/db/d1e/d36/f139 x:0 0 0 2024-03-29T08:07:32.957 INFO:tasks.workunit.client.1.smithi148.stdout:1/980: link d3/d18/d4d/d9e/l58 d3/d55/d7e/df0/l14d 0 2024-03-29T08:07:32.962 INFO:tasks.workunit.client.1.smithi148.stdout:1/981: write d3/de/d4f/fe9 [238532,42718] 0 2024-03-29T08:07:32.964 INFO:tasks.workunit.client.1.smithi148.stdout:7/960: link d8/d38/d3c/l5d d8/df4/d131/l14a 0 2024-03-29T08:07:32.966 INFO:tasks.workunit.client.1.smithi148.stdout:8/965: getdents d17/d18/d26/d4f 0 2024-03-29T08:07:32.969 INFO:tasks.workunit.client.1.smithi148.stdout:1/982: mkdir d3/d18/d4d/d76/d85/d11d/d47/d6d/d72/d14e 0 2024-03-29T08:07:32.971 INFO:tasks.workunit.client.0.smithi037.stdout:5/811: truncate d8/fa 6965802 0 2024-03-29T08:07:32.973 INFO:tasks.workunit.client.1.smithi148.stdout:1/983: mkdir d3/d18/d4d/d9e/d14f 0 2024-03-29T08:07:32.975 INFO:tasks.workunit.client.1.smithi148.stdout:1/984: unlink d3/d18/d71/d11b/l148 0 2024-03-29T08:07:32.980 INFO:tasks.workunit.client.0.smithi037.stdout:8/635: dread d5/d10/f1d [0,4194304] 0 2024-03-29T08:07:32.983 INFO:tasks.workunit.client.0.smithi037.stdout:3/616: fsync f2 0 2024-03-29T08:07:32.983 INFO:tasks.workunit.client.0.smithi037.stdout:7/745: creat dd/d21/d65/da2/ffa x:0 0 0 2024-03-29T08:07:32.983 INFO:tasks.workunit.client.1.smithi148.stdout:8/966: mknod d17/d49/d9a/c13b 0 2024-03-29T08:07:32.986 INFO:tasks.workunit.client.0.smithi037.stdout:8/636: write d5/d10/d2d/fc6 [625391,90573] 0 2024-03-29T08:07:32.988 INFO:tasks.workunit.client.0.smithi037.stdout:1/689: rename d2/d6/dc/d18/d22/d4f/d7c/lc6 to d2/lf0 0 2024-03-29T08:07:32.990 INFO:tasks.workunit.client.1.smithi148.stdout:7/961: write d8/dfb/d44/d49/f82 [3035656,7317] 0 2024-03-29T08:07:32.993 INFO:tasks.workunit.client.1.smithi148.stdout:7/962: fsync d8/d38/d6a/d91/dff/f144 0 2024-03-29T08:07:32.994 INFO:tasks.workunit.client.1.smithi148.stdout:0/994: dread db/dc/daa/d47/daf/f99 [0,4194304] 0 2024-03-29T08:07:32.998 INFO:tasks.workunit.client.0.smithi037.stdout:5/812: dwrite d8/db9/ff8 [0,4194304] 0 2024-03-29T08:07:33.003 INFO:tasks.workunit.client.0.smithi037.stdout:6/595: link d0/ca9 d0/d19/d34/cd5 0 2024-03-29T08:07:33.009 INFO:tasks.workunit.client.1.smithi148.stdout:7/963: dwrite d8/d38/d6a/d91/fe2 [0,4194304] 0 2024-03-29T08:07:33.010 INFO:tasks.workunit.client.0.smithi037.stdout:3/617: creat dd/d46/d2d/fc9 x:0 0 0 2024-03-29T08:07:33.021 INFO:tasks.workunit.client.0.smithi037.stdout:8/637: mknod d5/d10/d2d/d34/d50/d83/d46/d8a/cd3 0 2024-03-29T08:07:33.026 INFO:tasks.workunit.client.0.smithi037.stdout:0/573: getdents d9/da/dd/d20/d72 0 2024-03-29T08:07:33.032 INFO:tasks.workunit.client.1.smithi148.stdout:1/985: dread d3/de/f91 [0,4194304] 0 2024-03-29T08:07:33.032 INFO:tasks.workunit.client.0.smithi037.stdout:0/574: chown d9/da/dd/d13/f6f 0 1 2024-03-29T08:07:33.034 INFO:tasks.workunit.client.0.smithi037.stdout:7/746: dread dd/d21/f32 [4194304,4194304] 0 2024-03-29T08:07:33.036 INFO:tasks.workunit.client.0.smithi037.stdout:3/618: dwrite dd/d15/d27/d33/d84/fbd [0,4194304] 0 2024-03-29T08:07:33.038 INFO:tasks.workunit.client.0.smithi037.stdout:7/747: readlink dd/d21/d35/l9a 0 2024-03-29T08:07:33.052 INFO:tasks.workunit.client.0.smithi037.stdout:6/596: read - d0/d20/f8d zero size 2024-03-29T08:07:33.056 INFO:tasks.workunit.client.0.smithi037.stdout:6/597: fdatasync d0/d19/f9b 0 2024-03-29T08:07:33.058 INFO:tasks.workunit.client.1.smithi148.stdout:8/967: symlink d17/d74/l13c 0 2024-03-29T08:07:33.060 INFO:tasks.workunit.client.1.smithi148.stdout:0/995: mknod db/de/df/d14/dea/c135 0 2024-03-29T08:07:33.062 INFO:tasks.workunit.client.1.smithi148.stdout:0/996: chown db/de/df/d2b/f76 389034 1 2024-03-29T08:07:33.065 INFO:tasks.workunit.client.1.smithi148.stdout:0/997: read db/dc/daa/d47/f96 [2921696,71892] 0 2024-03-29T08:07:33.066 INFO:tasks.workunit.client.0.smithi037.stdout:1/690: dread d2/d6/dc/d18/d1d/fa9 [0,4194304] 0 2024-03-29T08:07:33.067 INFO:tasks.workunit.client.1.smithi148.stdout:0/998: dread - db/dc/daa/d47/d4c/f89 zero size 2024-03-29T08:07:33.067 INFO:tasks.workunit.client.0.smithi037.stdout:5/813: dwrite d8/d22/d4f/f5e [0,4194304] 0 2024-03-29T08:07:33.068 INFO:tasks.workunit.client.0.smithi037.stdout:6/598: rmdir d0/d6a/d7b/d77 39 2024-03-29T08:07:33.071 INFO:tasks.workunit.client.0.smithi037.stdout:6/599: truncate d0/d5/fc7 668057 0 2024-03-29T08:07:33.071 INFO:tasks.workunit.client.0.smithi037.stdout:3/619: fsync dd/d15/d27/d33/d97/fba 0 2024-03-29T08:07:33.076 INFO:tasks.workunit.client.0.smithi037.stdout:6/600: write d0/d19/d34/d4e/f54 [1439515,59892] 0 2024-03-29T08:07:33.077 INFO:tasks.workunit.client.0.smithi037.stdout:0/575: creat d9/da/fc6 x:0 0 0 2024-03-29T08:07:33.083 INFO:tasks.workunit.client.0.smithi037.stdout:8/638: write d5/d10/d2d/d34/d50/fa2 [64562,84026] 0 2024-03-29T08:07:33.083 INFO:tasks.workunit.client.1.smithi148.stdout:1/986: dread d3/d18/d71/fa7 [0,4194304] 0 2024-03-29T08:07:33.085 INFO:tasks.workunit.client.0.smithi037.stdout:6/601: mkdir d0/d6a/dd6 0 2024-03-29T08:07:33.088 INFO:tasks.workunit.client.0.smithi037.stdout:6/602: chown d0/d19/d42/lba 33271056 1 2024-03-29T08:07:33.089 INFO:tasks.workunit.client.0.smithi037.stdout:5/814: rename d8/d22/d37/dbc/d85/d90/c10e to d8/d22/d37/d8d/db3/c115 0 2024-03-29T08:07:33.090 INFO:tasks.workunit.client.1.smithi148.stdout:1/987: creat d3/de/d25/f150 x:0 0 0 2024-03-29T08:07:33.090 INFO:tasks.workunit.client.0.smithi037.stdout:7/748: write dd/f3e [4757239,53074] 0 2024-03-29T08:07:33.091 INFO:tasks.workunit.client.1.smithi148.stdout:8/968: write d17/d18/ff2 [1312332,125592] 0 2024-03-29T08:07:33.094 INFO:tasks.workunit.client.0.smithi037.stdout:7/749: truncate dd/f7e 854900 0 2024-03-29T08:07:33.096 INFO:tasks.workunit.client.1.smithi148.stdout:7/964: sync 2024-03-29T08:07:33.098 INFO:tasks.workunit.client.0.smithi037.stdout:8/639: creat d5/d10/d19/d3d/d4a/d4c/d67/fd4 x:0 0 0 2024-03-29T08:07:33.099 INFO:tasks.workunit.client.1.smithi148.stdout:1/988: write d3/de/f86 [2956595,24199] 0 2024-03-29T08:07:33.101 INFO:tasks.workunit.client.0.smithi037.stdout:1/691: write d2/f3a [443800,59399] 0 2024-03-29T08:07:33.101 INFO:tasks.workunit.client.0.smithi037.stdout:7/750: read - dd/d21/d35/d42/ddb/fee zero size 2024-03-29T08:07:33.104 INFO:tasks.workunit.client.0.smithi037.stdout:5/815: truncate d8/d22/d37/d8d/db3/dcb/fe1 944763 0 2024-03-29T08:07:33.104 INFO:tasks.workunit.client.1.smithi148.stdout:8/969: truncate d17/d18/d26/d4f/d94/fa2 1282060 0 2024-03-29T08:07:33.105 INFO:tasks.workunit.client.0.smithi037.stdout:4/715: sync 2024-03-29T08:07:33.105 INFO:tasks.workunit.client.0.smithi037.stdout:2/546: sync 2024-03-29T08:07:33.106 INFO:tasks.workunit.client.1.smithi148.stdout:0/999: dwrite db/de/d8e/da9/fca [0,4194304] 0 2024-03-29T08:07:33.109 INFO:tasks.workunit.client.1.smithi148.stdout:7/965: rmdir d8/d14/d45/d4d/df7 0 2024-03-29T08:07:33.112 INFO:tasks.workunit.client.1.smithi148.stdout:1/989: truncate d3/d18/d4d/d76/d85/d11d/d47/d6d/d72/d12b/f13c 425033 0 2024-03-29T08:07:33.112 INFO:tasks.workunit.client.0.smithi037.stdout:4/716: chown d3/d54/d88/fd9 4299 1 2024-03-29T08:07:33.115 INFO:tasks.workunit.client.1.smithi148.stdout:7/966: symlink d8/d11d/d147/l14b 0 2024-03-29T08:07:33.116 INFO:tasks.workunit.client.1.smithi148.stdout:8/970: mkdir d17/d18/d26/d63/d66/d13d 0 2024-03-29T08:07:33.117 INFO:tasks.workunit.client.0.smithi037.stdout:6/603: getdents d0/d5/d1e/d44 0 2024-03-29T08:07:33.119 INFO:tasks.workunit.client.1.smithi148.stdout:1/990: unlink d3/de/d6e/df9/dfa/f145 0 2024-03-29T08:07:33.121 INFO:tasks.workunit.client.0.smithi037.stdout:0/576: dread d9/da/dd/d59/f18 [0,4194304] 0 2024-03-29T08:07:33.122 INFO:tasks.workunit.client.1.smithi148.stdout:1/991: creat d3/d18/dae/dda/f151 x:0 0 0 2024-03-29T08:07:33.126 INFO:tasks.workunit.client.0.smithi037.stdout:2/547: rename d0/d53/l57 to d0/d53/d89/lb9 0 2024-03-29T08:07:33.127 INFO:tasks.workunit.client.1.smithi148.stdout:7/967: symlink d8/dfb/d44/d50/d85/l14c 0 2024-03-29T08:07:33.128 INFO:tasks.workunit.client.0.smithi037.stdout:4/717: truncate d3/d4/f2f 5466524 0 2024-03-29T08:07:33.129 INFO:tasks.workunit.client.0.smithi037.stdout:0/577: truncate d9/da/dd/d13/f96 488829 0 2024-03-29T08:07:33.129 INFO:tasks.workunit.client.1.smithi148.stdout:7/968: readlink d8/d11d/d147/l14b 0 2024-03-29T08:07:33.129 INFO:tasks.workunit.client.0.smithi037.stdout:3/620: dread dd/d5e/f45 [0,4194304] 0 2024-03-29T08:07:33.130 INFO:tasks.workunit.client.0.smithi037.stdout:7/751: mkdir dd/d1d/d49/d81/df0/dfb 0 2024-03-29T08:07:33.131 INFO:tasks.workunit.client.0.smithi037.stdout:6/604: creat d0/d19/d34/d5a/fd7 x:0 0 0 2024-03-29T08:07:33.132 INFO:tasks.workunit.client.1.smithi148.stdout:7/969: chown d8/d38/d3c/da5/caa 63 1 2024-03-29T08:07:33.132 INFO:tasks.workunit.client.0.smithi037.stdout:7/752: fdatasync dd/ff 0 2024-03-29T08:07:33.138 INFO:tasks.workunit.client.1.smithi148.stdout:8/971: dwrite d17/d49/f131 [0,4194304] 0 2024-03-29T08:07:33.138 INFO:tasks.workunit.client.0.smithi037.stdout:7/753: rename dd to dd/d21/d35/d42/d5c/d8c/dfc 22 2024-03-29T08:07:33.141 INFO:tasks.workunit.client.1.smithi148.stdout:8/972: write f15 [4836315,24671] 0 2024-03-29T08:07:33.144 INFO:tasks.workunit.client.0.smithi037.stdout:0/578: mknod d9/da/dd/d20/d38/d8a/cc7 0 2024-03-29T08:07:33.144 INFO:tasks.workunit.client.0.smithi037.stdout:6/605: stat d0/d6a/d7b/d77/c86 0 2024-03-29T08:07:33.145 INFO:tasks.workunit.client.1.smithi148.stdout:1/992: dread d3/de/d25/df1/f13d [0,4194304] 0 2024-03-29T08:07:33.146 INFO:tasks.workunit.client.0.smithi037.stdout:1/692: dread d2/d6/dc/f4a [0,4194304] 0 2024-03-29T08:07:33.146 INFO:tasks.workunit.client.0.smithi037.stdout:4/718: creat d3/d57/dec/ded/d1d/db/d72/d98/ff3 x:0 0 0 2024-03-29T08:07:33.146 INFO:tasks.workunit.client.1.smithi148.stdout:1/993: chown d3/d18/d4d 10970 1 2024-03-29T08:07:33.149 INFO:tasks.workunit.client.0.smithi037.stdout:2/548: chown d0/d53/d89/lb9 2725 1 2024-03-29T08:07:33.151 INFO:tasks.workunit.client.0.smithi037.stdout:5/816: dread d8/fa0 [0,4194304] 0 2024-03-29T08:07:33.151 INFO:tasks.workunit.client.1.smithi148.stdout:1/994: chown d3/d18/d4d/d9e/l137 13486 1 2024-03-29T08:07:33.152 INFO:tasks.workunit.client.0.smithi037.stdout:0/579: readlink d9/da/dd/l58 0 2024-03-29T08:07:33.153 INFO:tasks.workunit.client.0.smithi037.stdout:1/693: dread - d2/d6/d13/fe5 zero size 2024-03-29T08:07:33.154 INFO:tasks.workunit.client.0.smithi037.stdout:2/549: fsync d0/d20/d2e/da4/fb4 0 2024-03-29T08:07:33.158 INFO:tasks.workunit.client.1.smithi148.stdout:1/995: creat d3/de/d25/d51/f152 x:0 0 0 2024-03-29T08:07:33.158 INFO:tasks.workunit.client.1.smithi148.stdout:7/970: write f6 [90916,23314] 0 2024-03-29T08:07:33.158 INFO:tasks.workunit.client.0.smithi037.stdout:8/640: truncate d5/d10/d19/d3d/d4a/f6f 17321 0 2024-03-29T08:07:33.162 INFO:tasks.workunit.client.1.smithi148.stdout:8/973: fsync d17/d49/d9a/dbe/d6f/d64/da5/fda 0 2024-03-29T08:07:33.164 INFO:tasks.workunit.client.0.smithi037.stdout:7/754: truncate dd/f48 1283584 0 2024-03-29T08:07:33.165 INFO:tasks.workunit.client.0.smithi037.stdout:8/641: fsync d5/d10/d19/d3d/d4a/f6b 0 2024-03-29T08:07:33.165 INFO:tasks.workunit.client.0.smithi037.stdout:0/580: chown d9/da/dd/d20/d38/l7f 99565 1 2024-03-29T08:07:33.166 INFO:tasks.workunit.client.0.smithi037.stdout:4/719: write d3/d57/dec/ded/d1d/db/d72/d98/fbf [10061,33814] 0 2024-03-29T08:07:33.167 INFO:tasks.workunit.client.1.smithi148.stdout:7/971: fdatasync d8/dfb/d44/d49/fa3 0 2024-03-29T08:07:33.169 INFO:tasks.workunit.client.1.smithi148.stdout:1/996: write d3/d18/d4d/d9e/d12f/f143 [516269,20549] 0 2024-03-29T08:07:33.171 INFO:tasks.workunit.client.0.smithi037.stdout:1/694: write d2/d6/dc/fe4 [811175,18901] 0 2024-03-29T08:07:33.176 INFO:tasks.workunit.client.1.smithi148.stdout:1/997: creat d3/d18/dae/de7/f153 x:0 0 0 2024-03-29T08:07:33.177 INFO:tasks.workunit.client.0.smithi037.stdout:3/621: dread dd/d5e/fbb [0,4194304] 0 2024-03-29T08:07:33.179 INFO:tasks.workunit.client.1.smithi148.stdout:8/974: creat d17/d18/d26/d63/d66/d5f/dd0/d119/d122/d133/f13e x:0 0 0 2024-03-29T08:07:33.181 INFO:tasks.workunit.client.1.smithi148.stdout:1/998: unlink d3/c5 0 2024-03-29T08:07:33.183 INFO:tasks.workunit.client.1.smithi148.stdout:8/975: chown d17/d18/d26/d63/d66/d5f/dd0/d119/d122/d133/f13e 247669 1 2024-03-29T08:07:33.183 INFO:tasks.workunit.client.1.smithi148.stdout:1/999: dread - d3/d18/d4d/d76/f140 zero size 2024-03-29T08:07:33.183 INFO:tasks.workunit.client.0.smithi037.stdout:6/606: dwrite d0/d5/d3f/f17 [0,4194304] 0 2024-03-29T08:07:33.184 INFO:tasks.workunit.client.1.smithi148.stdout:8/976: write d17/d49/f131 [4738655,41526] 0 2024-03-29T08:07:33.185 INFO:tasks.workunit.client.0.smithi037.stdout:2/550: fdatasync d0/d20/f75 0 2024-03-29T08:07:33.195 INFO:tasks.workunit.client.0.smithi037.stdout:7/755: mkdir dd/d21/d35/d42/d5c/dfd 0 2024-03-29T08:07:33.197 INFO:tasks.workunit.client.1.smithi148.stdout:8/977: symlink d17/dce/de8/l13f 0 2024-03-29T08:07:33.200 INFO:tasks.workunit.client.0.smithi037.stdout:8/642: dread d5/d10/d19/d3d/f5d [0,4194304] 0 2024-03-29T08:07:33.203 INFO:tasks.workunit.client.0.smithi037.stdout:8/643: chown d5/d10/d2d/lb2 0 1 2024-03-29T08:07:33.204 INFO:tasks.workunit.client.0.smithi037.stdout:5/817: write d8/d22/d4f/d70/d110/fdb [605955,49338] 0 2024-03-29T08:07:33.211 INFO:tasks.workunit.client.0.smithi037.stdout:2/551: write d0/d20/d2e/d5f/d6f/f97 [1194075,97724] 0 2024-03-29T08:07:33.217 INFO:tasks.workunit.client.0.smithi037.stdout:7/756: write dd/f1b [5069754,43434] 0 2024-03-29T08:07:33.222 INFO:tasks.workunit.client.1.smithi148.stdout:7/972: truncate d8/dfb/d11b/d37/f9c 517781 0 2024-03-29T08:07:33.223 INFO:tasks.workunit.client.0.smithi037.stdout:3/622: chown dd/d46/d2d/d3a/d6c/db1/fbc 12217939 1 2024-03-29T08:07:33.227 INFO:tasks.workunit.client.0.smithi037.stdout:1/695: getdents d2/d6/dc/d18/d22/da0 0 2024-03-29T08:07:33.227 INFO:tasks.workunit.client.1.smithi148.stdout:7/973: fdatasync d8/d38/d3c/da5/fd5 0 2024-03-29T08:07:33.236 INFO:tasks.workunit.client.0.smithi037.stdout:8/644: dread d5/d10/d2d/d34/d50/d83/f5f [0,4194304] 0 2024-03-29T08:07:33.236 INFO:tasks.workunit.client.0.smithi037.stdout:3/623: creat dd/d46/d2d/d3a/d5f/fca x:0 0 0 2024-03-29T08:07:33.242 INFO:tasks.workunit.client.0.smithi037.stdout:6/607: dread d0/d5/fa7 [0,4194304] 0 2024-03-29T08:07:33.244 INFO:tasks.workunit.client.0.smithi037.stdout:7/757: link dd/d21/d35/d42/d5c/f9d dd/d21/d35/d42/ffe 0 2024-03-29T08:07:33.245 INFO:tasks.workunit.client.1.smithi148.stdout:8/978: dread d17/d18/d26/d63/d7d/f115 [0,4194304] 0 2024-03-29T08:07:33.248 INFO:tasks.workunit.client.1.smithi148.stdout:8/979: creat d17/d18/d26/d63/f140 x:0 0 0 2024-03-29T08:07:33.249 INFO:tasks.workunit.client.0.smithi037.stdout:5/818: dwrite d8/d22/d37/d8d/db3/dcb/fd5 [0,4194304] 0 2024-03-29T08:07:33.249 INFO:tasks.workunit.client.0.smithi037.stdout:4/720: dwrite d3/d57/dec/ded/d1d/db/d45/d1e/d59/d79/fa6 [0,4194304] 0 2024-03-29T08:07:33.252 INFO:tasks.workunit.client.0.smithi037.stdout:3/624: creat dd/d46/d2d/fcb x:0 0 0 2024-03-29T08:07:33.253 INFO:tasks.workunit.client.1.smithi148.stdout:8/980: dread d17/d18/d26/d63/d66/d5f/f8e [0,4194304] 0 2024-03-29T08:07:33.253 INFO:tasks.workunit.client.0.smithi037.stdout:1/696: write d2/d40/de8/daf/fd1 [187469,61156] 0 2024-03-29T08:07:33.257 INFO:tasks.workunit.client.0.smithi037.stdout:7/758: chown dd/d1d/d49/ccc 195 1 2024-03-29T08:07:33.257 INFO:tasks.workunit.client.0.smithi037.stdout:7/759: readlink dd/d72/l83 0 2024-03-29T08:07:33.258 INFO:tasks.workunit.client.1.smithi148.stdout:8/981: mknod d17/d49/c141 0 2024-03-29T08:07:33.260 INFO:tasks.workunit.client.0.smithi037.stdout:2/552: dread d0/d20/d2e/fa7 [0,4194304] 0 2024-03-29T08:07:33.261 INFO:tasks.workunit.client.0.smithi037.stdout:5/819: write d8/d22/d37/d8d/db3/dcb/deb/fa9 [1316174,100675] 0 2024-03-29T08:07:33.261 INFO:tasks.workunit.client.0.smithi037.stdout:3/625: mknod dd/d15/d27/d33/d75/ccc 0 2024-03-29T08:07:33.263 INFO:tasks.workunit.client.1.smithi148.stdout:8/982: rmdir d17/d49/d9a/dbe/d112 39 2024-03-29T08:07:33.265 INFO:tasks.workunit.client.0.smithi037.stdout:8/645: rename d5/d10/d19/d3d/d4a/d4c/f73 to d5/d10/d2d/d34/d50/d83/d46/fd5 0 2024-03-29T08:07:33.268 INFO:tasks.workunit.client.0.smithi037.stdout:2/553: dread - d0/d23/d70/f79 zero size 2024-03-29T08:07:33.269 INFO:tasks.workunit.client.0.smithi037.stdout:7/760: unlink dd/d1d/d34/fbb 0 2024-03-29T08:07:33.273 INFO:tasks.workunit.client.0.smithi037.stdout:1/697: creat d2/d6/dc/d18/d22/d4f/ff1 x:0 0 0 2024-03-29T08:07:33.275 INFO:tasks.workunit.client.1.smithi148.stdout:7/974: dread d8/d14/d45/d4d/d78/fbc [0,4194304] 0 2024-03-29T08:07:33.276 INFO:tasks.workunit.client.1.smithi148.stdout:8/983: dwrite f15 [4194304,4194304] 0 2024-03-29T08:07:33.277 INFO:tasks.workunit.client.0.smithi037.stdout:3/626: mknod dd/d46/d2d/d3a/d85/ccd 0 2024-03-29T08:07:33.278 INFO:tasks.workunit.client.0.smithi037.stdout:2/554: write d0/d20/d2e/d5f/d6f/d5e/f60 [7584392,117786] 0 2024-03-29T08:07:33.278 INFO:tasks.workunit.client.0.smithi037.stdout:5/820: fdatasync d8/d4d/d5c/fc3 0 2024-03-29T08:07:33.284 INFO:tasks.workunit.client.1.smithi148.stdout:8/984: mkdir d17/d1d/d7e/d142 0 2024-03-29T08:07:33.289 INFO:tasks.workunit.client.0.smithi037.stdout:1/698: creat d2/d6/dc/d5d/ff2 x:0 0 0 2024-03-29T08:07:33.289 INFO:tasks.workunit.client.0.smithi037.stdout:8/646: getdents d5/d10/d45/d4b 0 2024-03-29T08:07:33.293 INFO:tasks.workunit.client.1.smithi148.stdout:7/975: dwrite d8/dfb/fba [0,4194304] 0 2024-03-29T08:07:33.299 INFO:tasks.workunit.client.0.smithi037.stdout:4/721: dwrite d3/d57/dec/ded/d1d/db/d45/d1e/d59/d79/fa6 [0,4194304] 0 2024-03-29T08:07:33.304 INFO:tasks.workunit.client.0.smithi037.stdout:8/647: mkdir d5/d10/d2d/d34/d50/d87/dd6 0 2024-03-29T08:07:33.305 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:07:32 smithi148 ceph-mon[44992]: pgmap v48: 65 pgs: 65 active+clean; 2.9 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 81 MiB/s rd, 201 MiB/s wr, 418 op/s 2024-03-29T08:07:33.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:07:32 smithi148 ceph-mon[44992]: from='mgr.14602 172.21.15.148:0/667342922' entity='mgr.smithi148.onhgvm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:07:33.308 INFO:tasks.workunit.client.0.smithi037.stdout:4/722: chown d3/d57/dec/ded/d1d/db 11972171 1 2024-03-29T08:07:33.311 INFO:tasks.workunit.client.0.smithi037.stdout:5/821: creat d8/d22/d37/d8d/f116 x:0 0 0 2024-03-29T08:07:33.312 INFO:tasks.workunit.client.0.smithi037.stdout:8/648: stat d5/ca9 0 2024-03-29T08:07:33.316 INFO:tasks.workunit.client.0.smithi037.stdout:3/627: dwrite dd/d46/d2d/d3a/f53 [0,4194304] 0 2024-03-29T08:07:33.317 INFO:tasks.workunit.client.0.smithi037.stdout:7/761: write dd/d1d/d49/d81/d84/dca/ff6 [300329,96526] 0 2024-03-29T08:07:33.317 INFO:tasks.workunit.client.1.smithi148.stdout:7/976: rename d8/dfb/d11b/c7d to d8/d38/d67/c14d 0 2024-03-29T08:07:33.318 INFO:tasks.workunit.client.1.smithi148.stdout:8/985: dread d17/d49/d9a/dbe/fcb [0,4194304] 0 2024-03-29T08:07:33.319 INFO:tasks.workunit.client.0.smithi037.stdout:4/723: readlink d3/d57/dec/ded/d1d/db/d45/d1e/d59/d60/d6f/lc5 0 2024-03-29T08:07:33.321 INFO:tasks.workunit.client.0.smithi037.stdout:8/649: stat d5/d10/d19/d3d/d4a/d4c/d67/l7d 0 2024-03-29T08:07:33.323 INFO:tasks.workunit.client.1.smithi148.stdout:7/977: creat d8/d11d/f14e x:0 0 0 2024-03-29T08:07:33.325 INFO:tasks.workunit.client.1.smithi148.stdout:8/986: write d17/d49/d9a/dbe/d6f/d64/df1/f106 [692186,91605] 0 2024-03-29T08:07:33.327 INFO:tasks.workunit.client.0.smithi037.stdout:1/699: write d2/f14 [858686,57947] 0 2024-03-29T08:07:33.328 INFO:tasks.workunit.client.1.smithi148.stdout:7/978: mkdir d8/d38/d67/d14f 0 2024-03-29T08:07:33.330 INFO:tasks.workunit.client.0.smithi037.stdout:7/762: creat dd/d21/d35/d60/d94/fff x:0 0 0 2024-03-29T08:07:33.330 INFO:tasks.workunit.client.0.smithi037.stdout:4/724: fsync d3/d4/f2f 0 2024-03-29T08:07:33.332 INFO:tasks.workunit.client.0.smithi037.stdout:6/608: dread d0/d5/d1e/f7f [0,4194304] 0 2024-03-29T08:07:33.333 INFO:tasks.workunit.client.0.smithi037.stdout:3/628: rename dd/d46/d2d/l6b to dd/d15/d27/d33/d75/lce 0 2024-03-29T08:07:33.334 INFO:tasks.workunit.client.1.smithi148.stdout:8/987: creat d17/d1d/d7e/d142/f143 x:0 0 0 2024-03-29T08:07:33.335 INFO:tasks.workunit.client.0.smithi037.stdout:8/650: truncate d5/d10/f92 1105264 0 2024-03-29T08:07:33.346 INFO:tasks.workunit.client.0.smithi037.stdout:7/763: symlink dd/d21/d85/l100 0 2024-03-29T08:07:33.350 INFO:tasks.workunit.client.1.smithi148.stdout:8/988: dwrite d17/d49/d9a/dbe/f76 [0,4194304] 0 2024-03-29T08:07:33.352 INFO:tasks.workunit.client.0.smithi037.stdout:3/629: mkdir dd/d15/d18/d30/dcf 0 2024-03-29T08:07:33.352 INFO:tasks.workunit.client.0.smithi037.stdout:2/555: dwrite d0/d20/d22/f48 [0,4194304] 0 2024-03-29T08:07:33.352 INFO:tasks.workunit.client.0.smithi037.stdout:6/609: creat d0/d19/d34/d4e/d69/d79/dcc/fd8 x:0 0 0 2024-03-29T08:07:33.352 INFO:tasks.workunit.client.1.smithi148.stdout:8/989: chown d17/d18/ddf/fe1 3366 1 2024-03-29T08:07:33.355 INFO:tasks.workunit.client.1.smithi148.stdout:7/979: dread d8/d14/d45/d4d/d78/fc2 [0,4194304] 0 2024-03-29T08:07:33.357 INFO:tasks.workunit.client.1.smithi148.stdout:7/980: fsync d8/d60/f70 0 2024-03-29T08:07:33.358 INFO:tasks.workunit.client.1.smithi148.stdout:8/990: creat d17/d74/d86/f144 x:0 0 0 2024-03-29T08:07:33.360 INFO:tasks.workunit.client.1.smithi148.stdout:7/981: read d8/f134 [282817,10087] 0 2024-03-29T08:07:33.361 INFO:tasks.workunit.client.0.smithi037.stdout:5/822: dwrite d8/d22/d37/de8/f108 [0,4194304] 0 2024-03-29T08:07:33.363 INFO:tasks.workunit.client.0.smithi037.stdout:4/725: rename d3/d4/d27/c40 to d3/d57/dec/ded/d1d/cf4 0 2024-03-29T08:07:33.369 INFO:tasks.workunit.client.0.smithi037.stdout:5/823: write d8/d22/db6/f107 [512676,56348] 0 2024-03-29T08:07:33.371 INFO:tasks.workunit.client.1.smithi148.stdout:8/991: readlink d17/d18/d26/d63/d7d/lfc 0 2024-03-29T08:07:33.375 INFO:tasks.workunit.client.1.smithi148.stdout:8/992: read d17/d18/d26/d4f/d94/fe3 [3925990,122954] 0 2024-03-29T08:07:33.377 INFO:tasks.workunit.client.1.smithi148.stdout:7/982: rename d8/d14/f2b to d8/df4/d111/f150 0 2024-03-29T08:07:33.379 INFO:tasks.workunit.client.1.smithi148.stdout:8/993: write d17/d49/d9a/dbe/d6f/d64/f10e [248268,66270] 0 2024-03-29T08:07:33.385 INFO:tasks.workunit.client.0.smithi037.stdout:7/764: creat dd/d21/d35/d3b/d5d/f101 x:0 0 0 2024-03-29T08:07:33.386 INFO:tasks.workunit.client.1.smithi148.stdout:7/983: symlink d8/d38/d3c/l151 0 2024-03-29T08:07:33.389 INFO:tasks.workunit.client.0.smithi037.stdout:5/824: mknod d8/d22/d37/d8d/db3/c117 0 2024-03-29T08:07:33.390 INFO:tasks.workunit.client.1.smithi148.stdout:7/984: read d8/d38/f8d [1259215,71807] 0 2024-03-29T08:07:33.391 INFO:tasks.workunit.client.0.smithi037.stdout:1/700: dwrite d2/d6/dc/d18/d22/d4f/f73 [0,4194304] 0 2024-03-29T08:07:33.395 INFO:tasks.workunit.client.0.smithi037.stdout:8/651: rename d5/d10/d2d/d34/d50/d83/d46/fd5 to d5/d10/d2d/d34/d50/d87/fd7 0 2024-03-29T08:07:33.399 INFO:tasks.workunit.client.1.smithi148.stdout:7/985: symlink d8/d38/d3c/l152 0 2024-03-29T08:07:33.400 INFO:tasks.workunit.client.0.smithi037.stdout:5/825: read d8/d18/f21 [2519012,112251] 0 2024-03-29T08:07:33.402 INFO:tasks.workunit.client.0.smithi037.stdout:2/556: creat d0/d20/fba x:0 0 0 2024-03-29T08:07:33.406 INFO:tasks.workunit.client.1.smithi148.stdout:7/986: symlink d8/d38/d3c/da5/de3/l153 0 2024-03-29T08:07:33.407 INFO:tasks.workunit.client.0.smithi037.stdout:4/726: write d3/d57/dec/ded/d1d/d4f/f87 [333974,125305] 0 2024-03-29T08:07:33.407 INFO:tasks.workunit.client.1.smithi148.stdout:8/994: write d17/d18/d26/fd7 [1915915,95983] 0 2024-03-29T08:07:33.410 INFO:tasks.workunit.client.0.smithi037.stdout:9/888: sync 2024-03-29T08:07:33.412 INFO:tasks.workunit.client.0.smithi037.stdout:4/727: dread - d3/d57/dec/ded/d1d/db/d45/d1e/d59/d7d/fda zero size 2024-03-29T08:07:33.416 INFO:tasks.workunit.client.1.smithi148.stdout:8/995: creat d17/d18/d26/d63/d126/f145 x:0 0 0 2024-03-29T08:07:33.420 INFO:tasks.workunit.client.0.smithi037.stdout:1/701: dwrite d2/d6/dc/d58/fed [0,4194304] 0 2024-03-29T08:07:33.426 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:07:32 smithi037 ceph-mon[35162]: pgmap v48: 65 pgs: 65 active+clean; 2.9 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 81 MiB/s rd, 201 MiB/s wr, 418 op/s 2024-03-29T08:07:33.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:07:32 smithi037 ceph-mon[35162]: from='mgr.14602 172.21.15.148:0/667342922' entity='mgr.smithi148.onhgvm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:07:33.433 INFO:tasks.workunit.client.0.smithi037.stdout:3/630: rename fc to dd/d15/d27/d33/d84/fd0 0 2024-03-29T08:07:33.436 INFO:tasks.workunit.client.1.smithi148.stdout:7/987: dread d8/dfb/d44/d49/f74 [0,4194304] 0 2024-03-29T08:07:33.436 INFO:tasks.workunit.client.0.smithi037.stdout:7/765: symlink dd/d21/d35/d3b/dce/l102 0 2024-03-29T08:07:33.438 INFO:tasks.workunit.client.0.smithi037.stdout:7/766: rename dd to dd/d72/d103 22 2024-03-29T08:07:33.440 INFO:tasks.workunit.client.0.smithi037.stdout:5/826: creat d8/d4d/d72/f118 x:0 0 0 2024-03-29T08:07:33.440 INFO:tasks.workunit.client.1.smithi148.stdout:8/996: dread d17/d18/d26/d63/d7d/faa [0,4194304] 0 2024-03-29T08:07:33.441 INFO:tasks.workunit.client.0.smithi037.stdout:7/767: chown dd/d21/d35/d60/d94/fb5 7 1 2024-03-29T08:07:33.448 INFO:tasks.workunit.client.0.smithi037.stdout:7/768: read dd/f17 [3390231,50568] 0 2024-03-29T08:07:33.449 INFO:tasks.workunit.client.0.smithi037.stdout:4/728: truncate d3/d57/dec/ded/d1d/db/d45/d1e/d59/d60/f9c 333197 0 2024-03-29T08:07:33.452 INFO:tasks.workunit.client.0.smithi037.stdout:0/581: sync 2024-03-29T08:07:33.454 INFO:tasks.workunit.client.0.smithi037.stdout:3/631: mknod dd/d46/d2d/d3a/d5f/cd1 0 2024-03-29T08:07:33.455 INFO:tasks.workunit.client.1.smithi148.stdout:8/997: link d17/d74/d86/lf5 d17/d74/d10a/l146 0 2024-03-29T08:07:33.456 INFO:tasks.workunit.client.0.smithi037.stdout:6/610: link d0/d5/d1a/d61/c89 d0/cd9 0 2024-03-29T08:07:33.459 INFO:tasks.workunit.client.0.smithi037.stdout:7/769: symlink dd/d21/d35/d3b/dce/l104 0 2024-03-29T08:07:33.460 INFO:tasks.workunit.client.0.smithi037.stdout:6/611: write d0/d19/d34/d4e/fb4 [1046049,54328] 0 2024-03-29T08:07:33.460 INFO:tasks.workunit.client.1.smithi148.stdout:8/998: truncate d17/d49/d9a/dbe/f73 2246208 0 2024-03-29T08:07:33.462 INFO:tasks.workunit.client.0.smithi037.stdout:5/827: dwrite d8/d22/d37/d8d/f116 [0,4194304] 0 2024-03-29T08:07:33.463 INFO:tasks.workunit.client.0.smithi037.stdout:1/702: symlink d2/d6/dc/d18/d22/lf3 0 2024-03-29T08:07:33.465 INFO:tasks.workunit.client.0.smithi037.stdout:6/612: chown d0/d19/c49 3328 1 2024-03-29T08:07:33.466 INFO:tasks.workunit.client.0.smithi037.stdout:4/729: mknod d3/d80/dcd/cf5 0 2024-03-29T08:07:33.468 INFO:tasks.workunit.client.0.smithi037.stdout:4/730: fdatasync d3/d80/feb 0 2024-03-29T08:07:33.473 INFO:tasks.workunit.client.1.smithi148.stdout:8/999: mknod d17/dce/d110/c147 0 2024-03-29T08:07:33.478 INFO:tasks.workunit.client.0.smithi037.stdout:0/582: rename d9/da/dd/d13/c63 to d9/da/dd/d20/d55/d5f/cc8 0 2024-03-29T08:07:33.481 INFO:tasks.workunit.client.0.smithi037.stdout:5/828: mkdir d8/d4d/d8a/d119 0 2024-03-29T08:07:33.483 INFO:tasks.workunit.client.0.smithi037.stdout:8/652: dwrite d5/d10/d19/d8d/f97 [0,4194304] 0 2024-03-29T08:07:33.485 INFO:tasks.workunit.client.0.smithi037.stdout:1/703: mknod d2/d6/dc/d18/dbd/dda/cf4 0 2024-03-29T08:07:33.488 INFO:tasks.workunit.client.0.smithi037.stdout:0/583: write d9/da/d1b/f41 [1064882,88638] 0 2024-03-29T08:07:33.488 INFO:tasks.workunit.client.0.smithi037.stdout:2/557: getdents d0/d20/d2e/da4 0 2024-03-29T08:07:33.492 INFO:tasks.workunit.client.0.smithi037.stdout:3/632: truncate dd/d46/d2d/f60 1151330 0 2024-03-29T08:07:33.493 INFO:tasks.workunit.client.0.smithi037.stdout:9/889: dwrite d1/d4/d19/d1e/d4a/d6f/da2/ffc [0,4194304] 0 2024-03-29T08:07:33.496 INFO:tasks.workunit.client.0.smithi037.stdout:7/770: write dd/d1d/d49/d81/d84/f8a [1908059,120384] 0 2024-03-29T08:07:33.498 INFO:tasks.workunit.client.0.smithi037.stdout:7/771: readlink dd/d1d/d49/l4f 0 2024-03-29T08:07:33.501 INFO:tasks.workunit.client.0.smithi037.stdout:5/829: rename d8/d18/f21 to d8/db9/f11a 0 2024-03-29T08:07:33.502 INFO:tasks.workunit.client.0.smithi037.stdout:6/613: creat d0/d5/fda x:0 0 0 2024-03-29T08:07:33.504 INFO:tasks.workunit.client.0.smithi037.stdout:9/890: write d1/d4/d19/f64 [3166039,112327] 0 2024-03-29T08:07:33.505 INFO:tasks.workunit.client.0.smithi037.stdout:5/830: stat d8/d22/d37/d8d/db3/dcb/deb/l8b 0 2024-03-29T08:07:33.505 INFO:tasks.workunit.client.0.smithi037.stdout:1/704: symlink d2/d6/dc/lf5 0 2024-03-29T08:07:33.507 INFO:tasks.workunit.client.0.smithi037.stdout:1/705: chown d2/d4b/c5b 7843 1 2024-03-29T08:07:33.509 INFO:tasks.workunit.client.0.smithi037.stdout:5/831: readlink d8/d22/d37/d8d/db3/dcb/deb/d61/l7c 0 2024-03-29T08:07:33.511 INFO:tasks.workunit.client.0.smithi037.stdout:7/772: mknod dd/d21/d35/d60/d94/c105 0 2024-03-29T08:07:33.511 INFO:tasks.workunit.client.0.smithi037.stdout:8/653: creat d5/d10/d2d/d34/d50/d83/d46/d4e/fd8 x:0 0 0 2024-03-29T08:07:33.518 INFO:tasks.workunit.client.0.smithi037.stdout:5/832: fdatasync d8/d22/d37/d8d/f116 0 2024-03-29T08:07:33.518 INFO:tasks.workunit.client.0.smithi037.stdout:6/614: mkdir d0/d5/d1e/d44/dbb/ddb 0 2024-03-29T08:07:33.519 INFO:tasks.workunit.client.0.smithi037.stdout:1/706: mkdir d2/d6/dc/d58/d67/df6 0 2024-03-29T08:07:33.521 INFO:tasks.workunit.client.0.smithi037.stdout:2/558: rename d0/d23/f5c to d0/d20/d22/d4e/fbb 0 2024-03-29T08:07:33.525 INFO:tasks.workunit.client.0.smithi037.stdout:1/707: fsync d2/d6/d13/f77 0 2024-03-29T08:07:33.528 INFO:tasks.workunit.client.0.smithi037.stdout:3/633: dwrite dd/d15/d27/d33/f70 [0,4194304] 0 2024-03-29T08:07:33.531 INFO:tasks.workunit.client.0.smithi037.stdout:9/891: creat d1/d1f/d3a/f12a x:0 0 0 2024-03-29T08:07:33.531 INFO:tasks.workunit.client.0.smithi037.stdout:0/584: truncate d9/da/d1b/f3d 1696678 0 2024-03-29T08:07:33.534 INFO:tasks.workunit.client.0.smithi037.stdout:3/634: read dd/d15/d27/d33/f70 [848479,62772] 0 2024-03-29T08:07:33.539 INFO:tasks.workunit.client.0.smithi037.stdout:6/615: creat d0/d5/d1f/fdc x:0 0 0 2024-03-29T08:07:33.543 INFO:tasks.workunit.client.0.smithi037.stdout:4/731: dread d3/d57/dec/ded/d1d/db/d45/d1e/d59/d60/dd0/fe0 [0,4194304] 0 2024-03-29T08:07:33.544 INFO:tasks.workunit.client.0.smithi037.stdout:9/892: read d1/d4/d19/d1e/d4a/d6f/da2/d31/d87/fd5 [171552,90816] 0 2024-03-29T08:07:33.545 INFO:tasks.workunit.client.0.smithi037.stdout:2/559: symlink d0/d20/d2e/d5f/d6f/lbc 0 2024-03-29T08:07:33.545 INFO:tasks.workunit.client.0.smithi037.stdout:7/773: creat dd/d1d/d49/d81/d84/f106 x:0 0 0 2024-03-29T08:07:33.549 INFO:tasks.workunit.client.0.smithi037.stdout:6/616: mkdir d0/d20/ddd 0 2024-03-29T08:07:33.551 INFO:tasks.workunit.client.0.smithi037.stdout:5/833: creat d8/d18/d24/f11b x:0 0 0 2024-03-29T08:07:33.554 INFO:tasks.workunit.client.1.smithi148.stdout:7/988: sync 2024-03-29T08:07:33.554 INFO:tasks.workunit.client.0.smithi037.stdout:9/893: mknod d1/d1f/d3a/d9b/c12b 0 2024-03-29T08:07:33.559 INFO:tasks.workunit.client.0.smithi037.stdout:3/635: write dd/ff [4242872,115969] 0 2024-03-29T08:07:33.562 INFO:tasks.workunit.client.0.smithi037.stdout:0/585: rename d9/da/dd/d13/cb1 to d9/da/dd/d13/d7b/cc9 0 2024-03-29T08:07:33.562 INFO:tasks.workunit.client.0.smithi037.stdout:1/708: dwrite d2/d6/dc/d18/d1d/d21/fdb [0,4194304] 0 2024-03-29T08:07:33.566 INFO:tasks.workunit.client.0.smithi037.stdout:7/774: mkdir dd/d1d/d49/d81/dd8/d107 0 2024-03-29T08:07:33.566 INFO:tasks.workunit.client.0.smithi037.stdout:0/586: chown d9/da/f3c 1886 1 2024-03-29T08:07:33.566 INFO:tasks.workunit.client.0.smithi037.stdout:8/654: dwrite d5/f1f [0,4194304] 0 2024-03-29T08:07:33.572 INFO:tasks.workunit.client.0.smithi037.stdout:2/560: symlink d0/d20/d2e/d5f/da5/lbd 0 2024-03-29T08:07:33.578 INFO:tasks.workunit.client.0.smithi037.stdout:6/617: truncate d0/d5/fa7 2752987 0 2024-03-29T08:07:33.578 INFO:tasks.workunit.client.0.smithi037.stdout:1/709: read d2/d6/dc/d58/fa2 [578685,61300] 0 2024-03-29T08:07:33.579 INFO:tasks.workunit.client.1.smithi148.stdout:7/989: dwrite d8/dfb/f35 [0,4194304] 0 2024-03-29T08:07:33.579 INFO:tasks.workunit.client.0.smithi037.stdout:9/894: creat d1/d4/d19/d69/d75/f12c x:0 0 0 2024-03-29T08:07:33.579 INFO:tasks.workunit.client.0.smithi037.stdout:3/636: mknod dd/d46/d3e/cd2 0 2024-03-29T08:07:33.580 INFO:tasks.workunit.client.1.smithi148.stdout:7/990: stat d8/d38/d3c/c5e 0 2024-03-29T08:07:33.581 INFO:tasks.workunit.client.0.smithi037.stdout:7/775: creat dd/f108 x:0 0 0 2024-03-29T08:07:33.583 INFO:tasks.workunit.client.1.smithi148.stdout:7/991: write d8/dfb/db8/d115/f118 [1589,73346] 0 2024-03-29T08:07:33.588 INFO:tasks.workunit.client.0.smithi037.stdout:8/655: mknod d5/cd9 0 2024-03-29T08:07:33.590 INFO:tasks.workunit.client.0.smithi037.stdout:7/776: chown dd/d21/d65/da2/df8/cf1 19 1 2024-03-29T08:07:33.590 INFO:tasks.workunit.client.0.smithi037.stdout:9/895: dread d1/d4/fde [0,4194304] 0 2024-03-29T08:07:33.598 INFO:tasks.workunit.client.1.smithi148.stdout:7/992: creat d8/d60/f154 x:0 0 0 2024-03-29T08:07:33.599 INFO:tasks.workunit.client.1.smithi148.stdout:7/993: chown d8/dfb/d44/d50 29194 1 2024-03-29T08:07:33.606 INFO:tasks.workunit.client.0.smithi037.stdout:2/561: mkdir d0/d23/d70/d71/d78/dbe 0 2024-03-29T08:07:33.607 INFO:tasks.workunit.client.0.smithi037.stdout:7/777: mkdir dd/d1d/d109 0 2024-03-29T08:07:33.608 INFO:tasks.workunit.client.0.smithi037.stdout:9/896: fdatasync d1/d4/d19/d1e/d4a/d6f/da2/d23/f9a 0 2024-03-29T08:07:33.610 INFO:tasks.workunit.client.0.smithi037.stdout:8/656: creat d5/d10/d2d/d34/d50/d83/d46/d8a/d96/fda x:0 0 0 2024-03-29T08:07:33.610 INFO:tasks.workunit.client.0.smithi037.stdout:6/618: mkdir d0/d19/d34/d5a/dbf/dde 0 2024-03-29T08:07:33.611 INFO:tasks.workunit.client.0.smithi037.stdout:4/732: getdents d3/d57/dec/ded/d1d/db/d45/d1e/d59/d79 0 2024-03-29T08:07:33.612 INFO:tasks.workunit.client.0.smithi037.stdout:3/637: symlink dd/d15/d27/d33/d97/ld3 0 2024-03-29T08:07:33.613 INFO:tasks.workunit.client.1.smithi148.stdout:7/994: getdents d8/df3 0 2024-03-29T08:07:33.618 INFO:tasks.workunit.client.0.smithi037.stdout:1/710: write d2/d6/dc/d18/d22/d4f/d7c/f7f [921630,118487] 0 2024-03-29T08:07:33.620 INFO:tasks.workunit.client.0.smithi037.stdout:4/733: read d3/d57/dec/ded/d1d/db/d45/d1e/d59/d60/dd0/fe0 [1373073,66952] 0 2024-03-29T08:07:33.623 INFO:tasks.workunit.client.1.smithi148.stdout:7/995: symlink d8/d38/l155 0 2024-03-29T08:07:33.624 INFO:tasks.workunit.client.0.smithi037.stdout:0/587: dwrite f5 [0,4194304] 0 2024-03-29T08:07:33.626 INFO:tasks.workunit.client.0.smithi037.stdout:1/711: write d2/d6/dc/d18/d22/d4f/ff1 [682475,8010] 0 2024-03-29T08:07:33.627 INFO:tasks.workunit.client.1.smithi148.stdout:7/996: dread d8/dfb/d11b/db7/fc4 [0,4194304] 0 2024-03-29T08:07:33.629 INFO:tasks.workunit.client.0.smithi037.stdout:2/562: mkdir d0/d23/daf/dbf 0 2024-03-29T08:07:33.630 INFO:tasks.workunit.client.1.smithi148.stdout:7/997: mknod d8/d38/d6a/d91/dff/c156 0 2024-03-29T08:07:33.632 INFO:tasks.workunit.client.1.smithi148.stdout:7/998: chown d8/f136 398116585 1 2024-03-29T08:07:33.636 INFO:tasks.workunit.client.0.smithi037.stdout:7/778: dwrite dd/d21/d35/d42/d5c/d8c/ff3 [0,4194304] 0 2024-03-29T08:07:33.642 INFO:tasks.workunit.client.0.smithi037.stdout:8/657: write d5/d10/d19/d3d/d4a/d4c/d67/d85/f8e [2167318,108480] 0 2024-03-29T08:07:33.645 INFO:tasks.workunit.client.0.smithi037.stdout:4/734: rename d3/d57/dec/ded/d1d/db/d45/d1e/d59/d60/dd0 to d3/d80/df6 0 2024-03-29T08:07:33.646 INFO:tasks.workunit.client.0.smithi037.stdout:4/735: readlink d3/d57/lab 0 2024-03-29T08:07:33.650 INFO:tasks.workunit.client.1.smithi148.stdout:7/999: creat d8/df4/f157 x:0 0 0 2024-03-29T08:07:33.656 INFO:tasks.workunit.client.0.smithi037.stdout:9/897: truncate d1/d4/d19/d1e/d4a/d5c/f5d 793563 0 2024-03-29T08:07:33.658 INFO:tasks.workunit.client.1.smithi148.stderr:+ rm -rf -- ./tmp.OybOtqWMjH 2024-03-29T08:07:33.660 INFO:tasks.workunit.client.0.smithi037.stdout:2/563: creat d0/d20/d2e/d5f/d6f/d5e/fc0 x:0 0 0 2024-03-29T08:07:33.663 INFO:tasks.workunit.client.0.smithi037.stdout:9/898: chown d1/d4/f7f 148 1 2024-03-29T08:07:33.666 INFO:tasks.workunit.client.0.smithi037.stdout:3/638: dread dd/d46/d2d/d3a/d5f/f6a [0,4194304] 0 2024-03-29T08:07:33.669 INFO:tasks.workunit.client.0.smithi037.stdout:9/899: dread - d1/d4/d19/d1e/d25/ff8 zero size 2024-03-29T08:07:33.673 INFO:tasks.workunit.client.0.smithi037.stdout:6/619: dread d0/d5/fa7 [0,4194304] 0 2024-03-29T08:07:33.677 INFO:tasks.workunit.client.0.smithi037.stdout:9/900: write d1/d4/d19/d1e/d4a/d6f/da2/d31/daa/dba/de3/f104 [1032804,10616] 0 2024-03-29T08:07:33.733 INFO:tasks.workunit.client.0.smithi037.stdout:8/658: fdatasync d5/d10/d2d/d34/d50/d87/fd7 0 2024-03-29T08:07:33.737 INFO:tasks.workunit.client.0.smithi037.stdout:8/659: write d5/d10/d19/f2f [1782566,119386] 0 2024-03-29T08:07:33.745 INFO:tasks.workunit.client.0.smithi037.stdout:2/564: creat d0/d20/d2e/d5f/d6f/d5e/fc1 x:0 0 0 2024-03-29T08:07:33.751 INFO:tasks.workunit.client.0.smithi037.stdout:7/779: dwrite f9 [0,4194304] 0 2024-03-29T08:07:33.759 INFO:tasks.workunit.client.0.smithi037.stdout:6/620: creat d0/d5/d1f/da8/fdf x:0 0 0 2024-03-29T08:07:33.760 INFO:tasks.workunit.client.0.smithi037.stdout:6/621: chown d0/d5/d1f/da8 11 1 2024-03-29T08:07:33.774 INFO:tasks.workunit.client.0.smithi037.stdout:2/565: mkdir d0/d20/d2e/d5f/d6f/d5e/dc2 0 2024-03-29T08:07:33.776 INFO:tasks.workunit.client.0.smithi037.stdout:2/566: write d0/d53/f7a [463352,88791] 0 2024-03-29T08:07:33.778 INFO:tasks.workunit.client.0.smithi037.stdout:6/622: truncate d0/d5/fa 833111 0 2024-03-29T08:07:33.781 INFO:tasks.workunit.client.0.smithi037.stdout:6/623: dread - d0/d5/d1a/d61/fbd zero size 2024-03-29T08:07:33.787 INFO:tasks.workunit.client.0.smithi037.stdout:6/624: symlink d0/d20/d43/le0 0 2024-03-29T08:07:33.794 INFO:tasks.workunit.client.0.smithi037.stdout:6/625: symlink d0/d19/d34/d5a/dbf/le1 0 2024-03-29T08:07:33.796 INFO:tasks.workunit.client.0.smithi037.stdout:6/626: chown d0/d6a/d7b/d77 63098 1 2024-03-29T08:07:33.802 INFO:tasks.workunit.client.0.smithi037.stdout:6/627: creat d0/d19/d34/d4e/d94/fe2 x:0 0 0 2024-03-29T08:07:33.806 INFO:tasks.workunit.client.0.smithi037.stdout:6/628: mkdir d0/d5/d1f/da8/de3 0 2024-03-29T08:07:33.809 INFO:tasks.workunit.client.0.smithi037.stdout:6/629: chown d0/d19/d42/lba 25 1 2024-03-29T08:07:33.813 INFO:tasks.workunit.client.0.smithi037.stdout:6/630: truncate d0/d19/d34/d4e/faf 307931 0 2024-03-29T08:07:33.815 INFO:tasks.workunit.client.0.smithi037.stdout:6/631: chown d0/d20/lb7 1461855 1 2024-03-29T08:07:33.821 INFO:tasks.workunit.client.0.smithi037.stdout:5/834: sync 2024-03-29T08:07:33.824 INFO:tasks.workunit.client.0.smithi037.stdout:1/712: creat d2/d6/dc/d18/d22/d4f/ff7 x:0 0 0 2024-03-29T08:07:33.834 INFO:tasks.workunit.client.0.smithi037.stdout:1/713: link d2/d6/dc/f15 d2/d6/dc/d5d/d62/ff8 0 2024-03-29T08:07:33.840 INFO:tasks.workunit.client.0.smithi037.stdout:1/714: read - d2/d4b/d80/fc3 zero size 2024-03-29T08:07:33.847 INFO:tasks.workunit.client.0.smithi037.stdout:1/715: read d2/d4b/d80/f8b [2176875,15086] 0 2024-03-29T08:07:33.847 INFO:tasks.workunit.client.0.smithi037.stdout:6/632: dread d0/d19/d42/d60/d62/fc6 [0,4194304] 0 2024-03-29T08:07:33.852 INFO:tasks.workunit.client.0.smithi037.stdout:1/716: fsync d2/d6/dc/d58/f59 0 2024-03-29T08:07:33.852 INFO:tasks.workunit.client.0.smithi037.stdout:5/835: dread d8/f42 [0,4194304] 0 2024-03-29T08:07:33.857 INFO:tasks.workunit.client.0.smithi037.stdout:1/717: write d2/d6/dc/d18/d1d/d5e/fe6 [608363,116665] 0 2024-03-29T08:07:33.869 INFO:tasks.workunit.client.0.smithi037.stdout:3/639: dwrite dd/d46/d2d/d3a/d5f/f5a [0,4194304] 0 2024-03-29T08:07:33.871 INFO:tasks.workunit.client.0.smithi037.stdout:1/718: rename d2/d6/dc/d18/d22/f48 to d2/d6/dc/d18/d22/ff9 0 2024-03-29T08:07:33.872 INFO:tasks.workunit.client.0.smithi037.stdout:8/660: write d5/d10/d19/d3d/d4a/d4c/d67/faf [730485,80149] 0 2024-03-29T08:07:33.878 INFO:tasks.workunit.client.0.smithi037.stdout:3/640: mkdir dd/d5e/d7a/dd4 0 2024-03-29T08:07:33.883 INFO:tasks.workunit.client.0.smithi037.stdout:1/719: creat d2/d6/dc/d7e/dd2/ffa x:0 0 0 2024-03-29T08:07:33.886 INFO:tasks.workunit.client.0.smithi037.stdout:1/720: dread - d2/d40/fa5 zero size 2024-03-29T08:07:33.886 INFO:tasks.workunit.client.0.smithi037.stdout:2/567: write d0/d20/d22/f3c [241193,122318] 0 2024-03-29T08:07:33.914 INFO:tasks.workunit.client.0.smithi037.stdout:4/736: rename d3/d57/f7a to d3/d57/dec/ded/d1d/db/d45/d1e/d59/ff7 0 2024-03-29T08:07:33.919 INFO:tasks.workunit.client.0.smithi037.stdout:2/568: dread d0/d53/f88 [0,4194304] 0 2024-03-29T08:07:33.919 INFO:tasks.workunit.client.0.smithi037.stdout:0/588: rename d9/da/d1b/d22/d2c/c6e to d9/da/dd/d13/d7b/cca 0 2024-03-29T08:07:33.925 INFO:tasks.workunit.client.0.smithi037.stdout:2/569: mknod d0/d20/d7d/cc3 0 2024-03-29T08:07:33.925 INFO:tasks.workunit.client.0.smithi037.stdout:0/589: dread d9/da/dd/d13/f5e [4194304,4194304] 0 2024-03-29T08:07:33.930 INFO:tasks.workunit.client.0.smithi037.stdout:9/901: rename d1/d4/d19/d1e/d71/d89/da6/dd8 to d1/d4/d19/d1e/d4a/d6f/da2/d13/d12d 0 2024-03-29T08:07:33.931 INFO:tasks.workunit.client.0.smithi037.stdout:2/570: unlink d0/l28 0 2024-03-29T08:07:33.934 INFO:tasks.workunit.client.0.smithi037.stdout:6/633: dwrite d0/d5/d3f/f6f [0,4194304] 0 2024-03-29T08:07:33.938 INFO:tasks.workunit.client.0.smithi037.stdout:5/836: dwrite d8/f73 [0,4194304] 0 2024-03-29T08:07:33.945 INFO:tasks.workunit.client.0.smithi037.stdout:2/571: chown d0/d20/c2f 2 1 2024-03-29T08:07:33.949 INFO:tasks.workunit.client.0.smithi037.stdout:2/572: write d0/d20/d2e/d5f/d6f/d5e/f86 [3285929,57149] 0 2024-03-29T08:07:33.951 INFO:tasks.workunit.client.0.smithi037.stdout:0/590: mkdir d9/da/dd/d54/db5/dcb 0 2024-03-29T08:07:33.953 INFO:tasks.workunit.client.0.smithi037.stdout:0/591: readlink d9/da/dd/d54/da8/lbe 0 2024-03-29T08:07:33.954 INFO:tasks.workunit.client.0.smithi037.stdout:5/837: truncate d8/f1f 110483 0 2024-03-29T08:07:33.957 INFO:tasks.workunit.client.0.smithi037.stdout:5/838: chown d8/d18/f40 7332444 1 2024-03-29T08:07:33.959 INFO:tasks.workunit.client.0.smithi037.stdout:6/634: mknod d0/d6a/dd6/ce4 0 2024-03-29T08:07:33.964 INFO:tasks.workunit.client.0.smithi037.stdout:2/573: unlink d0/d23/d70/f7c 0 2024-03-29T08:07:33.965 INFO:tasks.workunit.client.0.smithi037.stdout:7/780: rename dd/d21/d35/d60/d94/le5 to dd/d21/d35/d3b/l10a 0 2024-03-29T08:07:33.966 INFO:tasks.workunit.client.0.smithi037.stdout:5/839: truncate d8/fca 762841 0 2024-03-29T08:07:33.967 INFO:tasks.workunit.client.0.smithi037.stdout:6/635: symlink d0/d5/d1e/d84/le5 0 2024-03-29T08:07:33.968 INFO:tasks.workunit.client.0.smithi037.stdout:2/574: creat d0/d20/d2e/d9c/fc4 x:0 0 0 2024-03-29T08:07:33.970 INFO:tasks.workunit.client.0.smithi037.stdout:8/661: rename d5/d10/d2d/d34/d50/d83/d46/c9f to d5/d10/d19/d8d/cdb 0 2024-03-29T08:07:33.974 INFO:tasks.workunit.client.0.smithi037.stdout:5/840: write d8/d22/d4f/d70/f82 [1654693,104348] 0 2024-03-29T08:07:33.978 INFO:tasks.workunit.client.0.smithi037.stdout:1/721: rename d2/d6/dd8 to d2/d4b/d80/dfb 0 2024-03-29T08:07:33.979 INFO:tasks.workunit.client.0.smithi037.stdout:0/592: dwrite d9/da/dd/f99 [0,4194304] 0 2024-03-29T08:07:33.980 INFO:tasks.workunit.client.0.smithi037.stdout:2/575: creat d0/d23/d44/fc5 x:0 0 0 2024-03-29T08:07:33.983 INFO:tasks.workunit.client.0.smithi037.stdout:8/662: mkdir d5/d10/d2d/d34/d50/ddc 0 2024-03-29T08:07:33.988 INFO:tasks.workunit.client.0.smithi037.stdout:8/663: write d5/d10/d19/d3d/d4a/d4c/d67/d70/fcc [348035,119689] 0 2024-03-29T08:07:33.989 INFO:tasks.workunit.client.0.smithi037.stdout:6/636: truncate d0/d5/d1a/f6c 1012168 0 2024-03-29T08:07:33.992 INFO:tasks.workunit.client.0.smithi037.stdout:8/664: stat d5/d10/d2d/d34/d50/d87/f94 0 2024-03-29T08:07:33.993 INFO:tasks.workunit.client.0.smithi037.stdout:5/841: mkdir d8/d4d/ddf/d11c 0 2024-03-29T08:07:34.000 INFO:tasks.workunit.client.0.smithi037.stdout:4/737: rename d3/d57/dec/ded/d1d/f71 to d3/d57/dec/ded/d1d/db/d72/ff8 0 2024-03-29T08:07:34.007 INFO:tasks.workunit.client.0.smithi037.stdout:7/781: write dd/d21/d35/d60/d94/fb0 [529616,52640] 0 2024-03-29T08:07:34.018 INFO:tasks.workunit.client.0.smithi037.stdout:2/576: unlink d0/d20/d2e/d5f/d6f/d5e/l7b 0 2024-03-29T08:07:34.019 INFO:tasks.workunit.client.0.smithi037.stdout:5/842: unlink d8/d4d/d8a/f106 0 2024-03-29T08:07:34.027 INFO:tasks.workunit.client.0.smithi037.stdout:6/637: mknod d0/d20/ddd/ce6 0 2024-03-29T08:07:34.031 INFO:tasks.workunit.client.0.smithi037.stdout:8/665: write d5/d10/d2d/d34/d50/f68 [1391645,18895] 0 2024-03-29T08:07:34.035 INFO:tasks.workunit.client.0.smithi037.stdout:4/738: dread d3/f5d [0,4194304] 0 2024-03-29T08:07:34.038 INFO:tasks.workunit.client.0.smithi037.stdout:2/577: chown d0/d20/d22/d55/f5b 4625708 1 2024-03-29T08:07:34.039 INFO:tasks.workunit.client.0.smithi037.stdout:0/593: creat d9/da/d1b/d22/fcc x:0 0 0 2024-03-29T08:07:34.048 INFO:tasks.workunit.client.0.smithi037.stdout:2/578: symlink d0/d20/d2e/da4/lc6 0 2024-03-29T08:07:34.052 INFO:tasks.workunit.client.0.smithi037.stdout:7/782: creat dd/d1d/d34/f10b x:0 0 0 2024-03-29T08:07:34.060 INFO:tasks.workunit.client.0.smithi037.stdout:7/783: truncate dd/d21/d35/d42/d5c/f69 4690531 0 2024-03-29T08:07:34.065 INFO:tasks.workunit.client.0.smithi037.stdout:6/638: write d0/d5/d1f/f30 [3026379,45102] 0 2024-03-29T08:07:34.066 INFO:tasks.workunit.client.0.smithi037.stdout:4/739: dwrite d3/d57/dec/ded/d1d/db/d45/d1e/d59/d73/fd6 [0,4194304] 0 2024-03-29T08:07:34.067 INFO:tasks.workunit.client.0.smithi037.stdout:8/666: write d5/d10/d19/d8d/fa8 [4727061,5390] 0 2024-03-29T08:07:34.068 INFO:tasks.workunit.client.0.smithi037.stdout:7/784: write dd/d21/d65/da2/df8/f87 [1395864,4358] 0 2024-03-29T08:07:34.073 INFO:tasks.workunit.client.0.smithi037.stdout:7/785: read - dd/fe3 zero size 2024-03-29T08:07:34.078 INFO:tasks.workunit.client.0.smithi037.stdout:6/639: read d0/d5/f73 [100015,114740] 0 2024-03-29T08:07:34.081 INFO:tasks.workunit.client.0.smithi037.stdout:2/579: dwrite d0/d20/d2e/d5f/d6f/d5e/f74 [0,4194304] 0 2024-03-29T08:07:34.082 INFO:tasks.workunit.client.0.smithi037.stdout:0/594: dread d9/da/f8d [0,4194304] 0 2024-03-29T08:07:34.086 INFO:tasks.workunit.client.0.smithi037.stdout:5/843: dwrite d8/fca [0,4194304] 0 2024-03-29T08:07:34.086 INFO:tasks.workunit.client.0.smithi037.stdout:8/667: symlink d5/d78/ldd 0 2024-03-29T08:07:34.086 INFO:tasks.workunit.client.0.smithi037.stdout:7/786: creat dd/d21/d85/f10c x:0 0 0 2024-03-29T08:07:34.087 INFO:tasks.workunit.client.0.smithi037.stdout:2/580: fsync d0/d20/d2e/d5f/d6f/fb7 0 2024-03-29T08:07:34.087 INFO:tasks.workunit.client.0.smithi037.stdout:6/640: readlink d0/d6a/d7b/d77/lb9 0 2024-03-29T08:07:34.091 INFO:tasks.workunit.client.0.smithi037.stdout:0/595: write d9/ff [5315674,41428] 0 2024-03-29T08:07:34.097 INFO:tasks.workunit.client.0.smithi037.stdout:8/668: mknod d5/d10/d2d/d9c/da1/cde 0 2024-03-29T08:07:34.099 INFO:tasks.workunit.client.0.smithi037.stdout:2/581: fdatasync d0/f15 0 2024-03-29T08:07:34.103 INFO:tasks.workunit.client.0.smithi037.stdout:0/596: write d9/da/dd/d13/f6f [230838,24868] 0 2024-03-29T08:07:34.104 INFO:tasks.workunit.client.0.smithi037.stdout:6/641: symlink d0/d5/le7 0 2024-03-29T08:07:34.109 INFO:tasks.workunit.client.0.smithi037.stdout:5/844: creat d8/d22/d4f/f11d x:0 0 0 2024-03-29T08:07:34.112 INFO:tasks.workunit.client.0.smithi037.stdout:8/669: creat d5/d10/d2d/d34/d50/ddc/fdf x:0 0 0 2024-03-29T08:07:34.115 INFO:tasks.workunit.client.0.smithi037.stdout:0/597: unlink d9/da/d1b/d22/fcc 0 2024-03-29T08:07:34.115 INFO:tasks.workunit.client.0.smithi037.stdout:7/787: dwrite dd/d21/d35/d42/f8e [0,4194304] 0 2024-03-29T08:07:34.120 INFO:tasks.workunit.client.0.smithi037.stdout:5/845: symlink d8/d4d/ddf/l11e 0 2024-03-29T08:07:34.122 INFO:tasks.workunit.client.0.smithi037.stdout:8/670: stat d5/d10/d19/d3d/d4a/c69 0 2024-03-29T08:07:34.125 INFO:tasks.workunit.client.0.smithi037.stdout:6/642: truncate d0/d19/d42/d60/f8b 60023 0 2024-03-29T08:07:34.127 INFO:tasks.workunit.client.0.smithi037.stdout:2/582: write d0/d23/d44/f45 [123192,33671] 0 2024-03-29T08:07:34.128 INFO:tasks.workunit.client.0.smithi037.stdout:0/598: creat d9/da/dd/d20/d72/fcd x:0 0 0 2024-03-29T08:07:34.129 INFO:tasks.workunit.client.0.smithi037.stdout:5/846: read d8/d22/d4f/d70/d9c/f97 [3353836,70812] 0 2024-03-29T08:07:34.132 INFO:tasks.workunit.client.0.smithi037.stdout:6/643: creat d0/d20/ddd/fe8 x:0 0 0 2024-03-29T08:07:34.132 INFO:tasks.workunit.client.0.smithi037.stdout:0/599: write d9/da/dd/d59/fb4 [28093,93061] 0 2024-03-29T08:07:34.135 INFO:tasks.workunit.client.0.smithi037.stdout:2/583: fdatasync d0/d20/d2e/f91 0 2024-03-29T08:07:34.142 INFO:tasks.workunit.client.0.smithi037.stdout:0/600: chown d9/da/dd/d13/f61 526539 1 2024-03-29T08:07:34.142 INFO:tasks.workunit.client.0.smithi037.stdout:8/671: getdents d5/d10/d19/d3d/d4a 0 2024-03-29T08:07:34.147 INFO:tasks.workunit.client.0.smithi037.stdout:8/672: chown d5/d10/d19/d3d/f9a 104 1 2024-03-29T08:07:34.154 INFO:tasks.workunit.client.0.smithi037.stdout:7/788: dread dd/d21/d65/da2/df8/f87 [0,4194304] 0 2024-03-29T08:07:34.154 INFO:tasks.workunit.client.0.smithi037.stdout:0/601: creat d9/da/d1b/d22/fce x:0 0 0 2024-03-29T08:07:34.155 INFO:tasks.workunit.client.0.smithi037.stdout:2/584: dread d0/d20/d2e/fa7 [0,4194304] 0 2024-03-29T08:07:34.157 INFO:tasks.workunit.client.0.smithi037.stdout:8/673: dread - d5/d10/d19/d3d/d4a/d4c/f6e zero size 2024-03-29T08:07:34.163 INFO:tasks.workunit.client.0.smithi037.stdout:7/789: creat dd/d21/d35/d60/d94/f10d x:0 0 0 2024-03-29T08:07:34.164 INFO:tasks.workunit.client.0.smithi037.stdout:6/644: dwrite d0/d5/d1f/da8/f97 [4194304,4194304] 0 2024-03-29T08:07:34.174 INFO:tasks.workunit.client.0.smithi037.stdout:2/585: getdents d0/d20/d22/d55/d92 0 2024-03-29T08:07:34.174 INFO:tasks.workunit.client.0.smithi037.stdout:5/847: dwrite d8/d22/d37/d8d/db3/fb8 [0,4194304] 0 2024-03-29T08:07:34.174 INFO:tasks.workunit.client.0.smithi037.stdout:7/790: fdatasync dd/d21/d35/d3b/fb8 0 2024-03-29T08:07:34.175 INFO:tasks.workunit.client.0.smithi037.stdout:0/602: dread d9/da/dd/d59/f39 [0,4194304] 0 2024-03-29T08:07:34.177 INFO:tasks.workunit.client.0.smithi037.stdout:6/645: symlink d0/d5/d1f/da8/da3/le9 0 2024-03-29T08:07:34.182 INFO:tasks.workunit.client.0.smithi037.stdout:6/646: chown d0/d20/ddd/ce6 314567064 1 2024-03-29T08:07:34.184 INFO:tasks.workunit.client.0.smithi037.stdout:6/647: fsync d0/d19/d42/d60/f7d 0 2024-03-29T08:07:34.191 INFO:tasks.workunit.client.0.smithi037.stdout:7/791: mkdir dd/d21/d65/da2/df8/d9c/d10e 0 2024-03-29T08:07:34.193 INFO:tasks.workunit.client.0.smithi037.stdout:7/792: stat dd/d21/d35/d3b/dce/l104 0 2024-03-29T08:07:34.194 INFO:tasks.workunit.client.0.smithi037.stdout:6/648: fsync d0/d19/f99 0 2024-03-29T08:07:34.195 INFO:tasks.workunit.client.0.smithi037.stdout:7/793: fsync dd/d21/d35/fcb 0 2024-03-29T08:07:34.197 INFO:tasks.workunit.client.0.smithi037.stdout:2/586: creat d0/d20/d2e/d5f/d6f/d5e/dc2/fc7 x:0 0 0 2024-03-29T08:07:34.202 INFO:tasks.workunit.client.0.smithi037.stdout:5/848: write d8/f1f [376156,64487] 0 2024-03-29T08:07:34.203 INFO:tasks.workunit.client.0.smithi037.stdout:6/649: creat d0/d19/d34/d5a/dbf/fea x:0 0 0 2024-03-29T08:07:34.205 INFO:tasks.workunit.client.0.smithi037.stdout:7/794: symlink dd/d1d/d49/d81/df0/l10f 0 2024-03-29T08:07:34.206 INFO:tasks.workunit.client.0.smithi037.stdout:6/650: dread - d0/d6a/d7b/d77/f8e zero size 2024-03-29T08:07:34.206 INFO:tasks.workunit.client.0.smithi037.stdout:7/795: readlink lc 0 2024-03-29T08:07:34.208 INFO:tasks.workunit.client.0.smithi037.stdout:6/651: fsync d0/d6a/d7b/fa1 0 2024-03-29T08:07:34.209 INFO:tasks.workunit.client.0.smithi037.stdout:5/849: rmdir d8/d4d/d72 39 2024-03-29T08:07:34.210 INFO:tasks.workunit.client.0.smithi037.stdout:0/603: truncate d9/da/dd/d20/d72/f85 2113330 0 2024-03-29T08:07:34.214 INFO:tasks.workunit.client.0.smithi037.stdout:0/604: chown d9/da/d1b/d22/d2c/f86 1105 1 2024-03-29T08:07:34.218 INFO:tasks.workunit.client.0.smithi037.stdout:5/850: fdatasync d8/d18/f50 0 2024-03-29T08:07:34.220 INFO:tasks.workunit.client.0.smithi037.stdout:8/674: dread d5/d10/d2d/d34/d50/d83/f28 [0,4194304] 0 2024-03-29T08:07:34.225 INFO:tasks.workunit.client.0.smithi037.stdout:7/796: creat dd/d1d/d49/d81/dd8/d107/f110 x:0 0 0 2024-03-29T08:07:34.228 INFO:tasks.workunit.client.0.smithi037.stdout:0/605: rename d9/da/d1b/d22/c93 to d9/da/dd/d54/ccf 0 2024-03-29T08:07:34.231 INFO:tasks.workunit.client.0.smithi037.stdout:2/587: dwrite d0/d23/d44/f46 [0,4194304] 0 2024-03-29T08:07:34.235 INFO:tasks.workunit.client.0.smithi037.stdout:6/652: getdents d0/d20 0 2024-03-29T08:07:34.235 INFO:tasks.workunit.client.0.smithi037.stdout:7/797: creat dd/d1d/d49/d81/dd8/d107/f111 x:0 0 0 2024-03-29T08:07:34.239 INFO:tasks.workunit.client.0.smithi037.stdout:8/675: getdents d5 0 2024-03-29T08:07:34.240 INFO:tasks.workunit.client.0.smithi037.stdout:6/653: read d0/d5/d3f/f6f [2071442,40898] 0 2024-03-29T08:07:34.241 INFO:tasks.workunit.client.0.smithi037.stdout:5/851: dread d8/d22/d37/d8d/f116 [0,4194304] 0 2024-03-29T08:07:34.243 INFO:tasks.workunit.client.0.smithi037.stdout:6/654: dread - d0/d5/d3f/fb1 zero size 2024-03-29T08:07:34.244 INFO:tasks.workunit.client.0.smithi037.stdout:7/798: mkdir dd/d68/d112 0 2024-03-29T08:07:34.244 INFO:tasks.workunit.client.0.smithi037.stdout:8/676: chown d5/d10/d2d/d34/d50/ddc 11366015 1 2024-03-29T08:07:34.249 INFO:tasks.workunit.client.0.smithi037.stdout:6/655: chown d0/d19/fc4 130887 1 2024-03-29T08:07:34.255 INFO:tasks.workunit.client.0.smithi037.stdout:6/656: write d0/d19/d42/d60/d62/f9a [1205288,116696] 0 2024-03-29T08:07:34.257 INFO:tasks.workunit.client.0.smithi037.stdout:8/677: dread d5/d10/d2d/d34/d50/d83/d46/d8a/fc1 [0,4194304] 0 2024-03-29T08:07:34.259 INFO:tasks.workunit.client.0.smithi037.stdout:2/588: fdatasync d0/d23/d44/f46 0 2024-03-29T08:07:34.271 INFO:tasks.workunit.client.0.smithi037.stdout:8/678: truncate d5/d10/d2d/d34/d50/f5e 2125986 0 2024-03-29T08:07:34.278 INFO:tasks.workunit.client.0.smithi037.stdout:0/606: dread d9/da/dd/d13/f45 [0,4194304] 0 2024-03-29T08:07:34.279 INFO:tasks.workunit.client.0.smithi037.stdout:8/679: chown d5/d10/d19/d3d/d4a/d4c/cb7 126 1 2024-03-29T08:07:34.282 INFO:tasks.workunit.client.0.smithi037.stdout:8/680: chown d5/d10/d2d/d34/d50/f68 613228 1 2024-03-29T08:07:34.285 INFO:tasks.workunit.client.0.smithi037.stdout:5/852: write d8/d22/d4f/d70/d9c/f84 [4213368,75472] 0 2024-03-29T08:07:34.287 INFO:tasks.workunit.client.0.smithi037.stdout:5/853: dread - d8/d22/dc1/f10b zero size 2024-03-29T08:07:34.289 INFO:tasks.workunit.client.0.smithi037.stdout:8/681: mkdir d5/d10/d2d/d9c/de0 0 2024-03-29T08:07:34.290 INFO:tasks.workunit.client.0.smithi037.stdout:2/589: dread d0/d20/f3a [0,4194304] 0 2024-03-29T08:07:34.294 INFO:tasks.workunit.client.0.smithi037.stdout:7/799: dread dd/d21/d35/d3b/d5d/fc7 [0,4194304] 0 2024-03-29T08:07:34.301 INFO:tasks.workunit.client.0.smithi037.stdout:6/657: dwrite d0/d5/d1e/d44/d71/f8c [0,4194304] 0 2024-03-29T08:07:34.302 INFO:tasks.workunit.client.0.smithi037.stdout:8/682: creat d5/d10/d2d/d9c/fe1 x:0 0 0 2024-03-29T08:07:34.303 INFO:tasks.workunit.client.0.smithi037.stdout:6/658: readlink d0/d6a/l83 0 2024-03-29T08:07:34.304 INFO:tasks.workunit.client.0.smithi037.stdout:5/854: unlink d8/d18/d24/c81 0 2024-03-29T08:07:34.304 INFO:tasks.workunit.client.0.smithi037.stdout:6/659: dread - d0/d5/d1a/fa6 zero size 2024-03-29T08:07:34.318 INFO:tasks.workunit.client.0.smithi037.stdout:2/590: write d0/d20/f49 [1216275,119223] 0 2024-03-29T08:07:34.320 INFO:tasks.workunit.client.0.smithi037.stdout:8/683: dread d5/d10/d2d/fc6 [0,4194304] 0 2024-03-29T08:07:34.322 INFO:tasks.workunit.client.0.smithi037.stdout:0/607: getdents d9/da/dd/d13/d7b 0 2024-03-29T08:07:34.323 INFO:tasks.workunit.client.0.smithi037.stdout:8/684: read - d5/d10/d2d/da7/fd1 zero size 2024-03-29T08:07:34.328 INFO:tasks.workunit.client.0.smithi037.stdout:5/855: symlink d8/l11f 0 2024-03-29T08:07:34.329 INFO:tasks.workunit.client.0.smithi037.stdout:6/660: link d0/d20/ddd/fe8 d0/d19/d34/d4e/d69/d79/feb 0 2024-03-29T08:07:34.331 INFO:tasks.workunit.client.0.smithi037.stdout:7/800: link dd/d21/d35/d3b/dce/l104 dd/d1d/d49/d81/d84/dca/l113 0 2024-03-29T08:07:34.333 INFO:tasks.workunit.client.0.smithi037.stdout:8/685: mkdir d5/d10/d2d/d34/d50/d83/d46/d4e/de2 0 2024-03-29T08:07:34.336 INFO:tasks.workunit.client.0.smithi037.stdout:5/856: creat d8/d22/d37/d8d/db3/dcb/f120 x:0 0 0 2024-03-29T08:07:34.337 INFO:tasks.workunit.client.0.smithi037.stdout:6/661: symlink d0/d19/d34/d5a/lec 0 2024-03-29T08:07:34.337 INFO:tasks.workunit.client.0.smithi037.stdout:5/857: chown d8/d18/d24/f74 130375681 1 2024-03-29T08:07:34.339 INFO:tasks.workunit.client.0.smithi037.stdout:7/801: creat dd/d21/d35/d42/ddb/f114 x:0 0 0 2024-03-29T08:07:34.340 INFO:tasks.workunit.client.0.smithi037.stdout:8/686: truncate d5/d10/d45/f7f 200917 0 2024-03-29T08:07:34.343 INFO:tasks.workunit.client.0.smithi037.stdout:0/608: link d9/da/d1b/d22/d2c/f2d d9/da/dd/fd0 0 2024-03-29T08:07:34.345 INFO:tasks.workunit.client.0.smithi037.stdout:6/662: creat d0/d19/d34/d4e/d94/fed x:0 0 0 2024-03-29T08:07:34.345 INFO:tasks.workunit.client.0.smithi037.stdout:2/591: link d0/d20/d2e/d5f/da5/lbd d0/lc8 0 2024-03-29T08:07:34.351 INFO:tasks.workunit.client.0.smithi037.stdout:6/663: unlink d0/lc0 0 2024-03-29T08:07:34.355 INFO:tasks.workunit.client.0.smithi037.stdout:2/592: unlink d0/d20/d7d/f80 0 2024-03-29T08:07:34.356 INFO:tasks.workunit.client.0.smithi037.stdout:2/593: stat d0/d23 0 2024-03-29T08:07:34.364 INFO:tasks.workunit.client.0.smithi037.stdout:5/858: link d8/d18/l49 d8/d4d/ddf/d11c/l121 0 2024-03-29T08:07:34.366 INFO:tasks.workunit.client.0.smithi037.stdout:5/859: chown d8/d22/d37/dbc/d85/l87 1441 1 2024-03-29T08:07:34.368 INFO:tasks.workunit.client.0.smithi037.stdout:2/594: symlink d0/d53/lc9 0 2024-03-29T08:07:34.369 INFO:tasks.workunit.client.0.smithi037.stdout:8/687: link d5/d10/d19/d3d/d4a/f61 d5/d10/d19/d3d/d4a/dc8/fe3 0 2024-03-29T08:07:34.373 INFO:tasks.workunit.client.0.smithi037.stdout:0/609: write d9/da/dd/d13/d83/f6d [465590,127286] 0 2024-03-29T08:07:34.375 INFO:tasks.workunit.client.0.smithi037.stdout:7/802: getdents dd/d1d/d49/d81/dd8/d107 0 2024-03-29T08:07:34.377 INFO:tasks.workunit.client.0.smithi037.stdout:0/610: chown d9/da/dd/d59/c9b 1 1 2024-03-29T08:07:34.377 INFO:tasks.workunit.client.0.smithi037.stdout:7/803: truncate dd/d72/fae 9131406 0 2024-03-29T08:07:34.378 INFO:tasks.workunit.client.0.smithi037.stdout:0/611: fsync d9/da/dd/f99 0 2024-03-29T08:07:34.380 INFO:tasks.workunit.client.0.smithi037.stdout:0/612: dread - d9/d74/fa1 zero size 2024-03-29T08:07:34.383 INFO:tasks.workunit.client.0.smithi037.stdout:0/613: write d9/da/d1b/f29 [654030,123341] 0 2024-03-29T08:07:34.385 INFO:tasks.workunit.client.0.smithi037.stdout:5/860: dread d8/d18/f28 [0,4194304] 0 2024-03-29T08:07:34.387 INFO:tasks.workunit.client.0.smithi037.stdout:0/614: write d9/da/dd/d20/d55/dac/fb2 [161754,105742] 0 2024-03-29T08:07:34.391 INFO:tasks.workunit.client.0.smithi037.stdout:2/595: write d0/d20/f75 [757744,49324] 0 2024-03-29T08:07:34.403 INFO:tasks.workunit.client.0.smithi037.stdout:0/615: mknod d9/da/dd/d20/d55/dac/cd1 0 2024-03-29T08:07:34.405 INFO:tasks.workunit.client.0.smithi037.stdout:7/804: symlink dd/d21/d65/da2/l115 0 2024-03-29T08:07:34.406 INFO:tasks.workunit.client.0.smithi037.stdout:6/664: dwrite d0/d5/d1e/f67 [0,4194304] 0 2024-03-29T08:07:34.409 INFO:tasks.workunit.client.0.smithi037.stdout:6/665: fdatasync d0/d6a/f85 0 2024-03-29T08:07:34.433 INFO:tasks.workunit.client.0.smithi037.stdout:8/688: link d5/d10/c5a d5/ce4 0 2024-03-29T08:07:34.435 INFO:tasks.workunit.client.0.smithi037.stdout:0/616: mkdir d9/da/dd/d20/d55/dd2 0 2024-03-29T08:07:34.436 INFO:tasks.workunit.client.0.smithi037.stdout:7/805: mknod dd/d21/d35/d42/ddb/de2/c116 0 2024-03-29T08:07:34.438 INFO:tasks.workunit.client.0.smithi037.stdout:6/666: mkdir d0/d19/d34/d4e/d69/dee 0 2024-03-29T08:07:34.442 INFO:tasks.workunit.client.0.smithi037.stdout:5/861: rename d8/d22/dc1/f10b to d8/d22/d37/d8d/db3/dcb/deb/f122 0 2024-03-29T08:07:34.444 INFO:tasks.workunit.client.0.smithi037.stdout:0/617: chown d9/da/d1b/d22/l8c 451676 1 2024-03-29T08:07:34.446 INFO:tasks.workunit.client.0.smithi037.stdout:6/667: symlink d0/d5/d1e/d44/dbb/lef 0 2024-03-29T08:07:34.451 INFO:tasks.workunit.client.0.smithi037.stdout:6/668: truncate d0/d6a/d7b/fa1 647827 0 2024-03-29T08:07:34.453 INFO:tasks.workunit.client.0.smithi037.stdout:2/596: rename d0/d20 to d0/d23/d44/dca 0 2024-03-29T08:07:34.454 INFO:tasks.workunit.client.0.smithi037.stdout:7/806: dwrite dd/d21/f36 [0,4194304] 0 2024-03-29T08:07:34.456 INFO:tasks.workunit.client.0.smithi037.stdout:5/862: creat d8/d22/d37/dbc/d85/d90/f123 x:0 0 0 2024-03-29T08:07:34.457 INFO:tasks.workunit.client.0.smithi037.stdout:8/689: mknod d5/d10/d2d/d34/d50/d83/d46/ce5 0 2024-03-29T08:07:34.463 INFO:tasks.workunit.client.0.smithi037.stdout:0/618: rename d9/d74/d7d/f94 to d9/da/dd/d20/d55/d5f/d9f/fd3 0 2024-03-29T08:07:34.471 INFO:tasks.workunit.client.0.smithi037.stdout:6/669: dwrite d0/d5/d1f/fc3 [0,4194304] 0 2024-03-29T08:07:34.471 INFO:tasks.workunit.client.0.smithi037.stdout:5/863: mkdir d8/d22/d37/dbc/d85/d124 0 2024-03-29T08:07:34.472 INFO:tasks.workunit.client.0.smithi037.stdout:2/597: symlink d0/d23/d70/d71/lcb 0 2024-03-29T08:07:34.477 INFO:tasks.workunit.client.0.smithi037.stdout:0/619: mkdir d9/da/dd/d54/d92/dd4 0 2024-03-29T08:07:34.477 INFO:tasks.workunit.client.0.smithi037.stdout:6/670: chown d0/d5/d1a/d61/f65 60812 1 2024-03-29T08:07:34.481 INFO:tasks.workunit.client.0.smithi037.stdout:6/671: write d0/d19/d42/d60/d62/f9a [103334,129299] 0 2024-03-29T08:07:34.483 INFO:tasks.workunit.client.0.smithi037.stdout:5/864: mkdir d8/d34/d125 0 2024-03-29T08:07:34.487 INFO:tasks.workunit.client.0.smithi037.stdout:6/672: truncate d0/d5/d1f/f4c 5321007 0 2024-03-29T08:07:34.493 INFO:tasks.workunit.client.0.smithi037.stdout:7/807: getdents dd/d21/d65/da2 0 2024-03-29T08:07:34.495 INFO:tasks.workunit.client.0.smithi037.stdout:7/808: write dd/f77 [2753381,85016] 0 2024-03-29T08:07:34.497 INFO:tasks.workunit.client.0.smithi037.stdout:5/865: read d8/d18/d24/fd8 [1042926,11365] 0 2024-03-29T08:07:34.497 INFO:tasks.workunit.client.0.smithi037.stdout:2/598: write d0/d53/f88 [5561610,34076] 0 2024-03-29T08:07:34.498 INFO:tasks.workunit.client.0.smithi037.stdout:8/690: dread d5/d10/d2d/fcb [0,4194304] 0 2024-03-29T08:07:34.502 INFO:tasks.workunit.client.0.smithi037.stdout:0/620: unlink d9/c46 0 2024-03-29T08:07:34.502 INFO:tasks.workunit.client.0.smithi037.stdout:8/691: chown d5/d10/d19/d3d/c7b 168 1 2024-03-29T08:07:34.502 INFO:tasks.workunit.client.0.smithi037.stdout:6/673: mkdir d0/d5/d1f/da8/de3/df0 0 2024-03-29T08:07:34.504 INFO:tasks.workunit.client.0.smithi037.stdout:6/674: chown d0/d19/d34/d4e/d69/d79/fa0 1011705052 1 2024-03-29T08:07:34.504 INFO:tasks.workunit.client.0.smithi037.stdout:6/675: stat d0/d19/c49 0 2024-03-29T08:07:34.524 INFO:tasks.workunit.client.0.smithi037.stdout:2/599: creat d0/d23/d44/dca/d7d/fcc x:0 0 0 2024-03-29T08:07:34.527 INFO:tasks.workunit.client.0.smithi037.stdout:8/692: fsync d5/d10/d19/d3d/d4a/d4c/f6e 0 2024-03-29T08:07:34.527 INFO:tasks.workunit.client.0.smithi037.stdout:7/809: dwrite dd/d21/d35/fc5 [0,4194304] 0 2024-03-29T08:07:34.527 INFO:tasks.workunit.client.0.smithi037.stdout:0/621: dread - d9/da/dd/d20/d38/d53/f6a zero size 2024-03-29T08:07:34.530 INFO:tasks.workunit.client.0.smithi037.stdout:5/866: mkdir d8/d18/d126 0 2024-03-29T08:07:34.531 INFO:tasks.workunit.client.0.smithi037.stdout:0/622: rename d9/da to d9/da/dd/d20/d55/d5f/dab/dd5 22 2024-03-29T08:07:34.538 INFO:tasks.workunit.client.0.smithi037.stdout:0/623: truncate d9/da/dd/d54/f56 4326957 0 2024-03-29T08:07:34.539 INFO:tasks.workunit.client.0.smithi037.stdout:8/693: write d5/f15 [762616,84868] 0 2024-03-29T08:07:34.540 INFO:tasks.workunit.client.0.smithi037.stdout:6/676: truncate d0/d5/fa 1761385 0 2024-03-29T08:07:34.541 INFO:tasks.workunit.client.0.smithi037.stdout:8/694: fdatasync d5/d10/d2d/f3e 0 2024-03-29T08:07:34.544 INFO:tasks.workunit.client.0.smithi037.stdout:8/695: dread - d5/d10/d2d/d34/d50/ddc/fdf zero size 2024-03-29T08:07:34.546 INFO:tasks.workunit.client.0.smithi037.stdout:6/677: dread d0/d6a/f85 [0,4194304] 0 2024-03-29T08:07:34.549 INFO:tasks.workunit.client.0.smithi037.stdout:0/624: mknod d9/da/dd/d20/d55/d5f/cd6 0 2024-03-29T08:07:34.550 INFO:tasks.workunit.client.0.smithi037.stdout:7/810: link dd/f7e dd/d21/d65/da2/f117 0 2024-03-29T08:07:34.550 INFO:tasks.workunit.client.0.smithi037.stdout:5/867: symlink d8/d18/d126/l127 0 2024-03-29T08:07:34.552 INFO:tasks.workunit.client.0.smithi037.stdout:2/600: creat d0/fcd x:0 0 0 2024-03-29T08:07:34.553 INFO:tasks.workunit.client.0.smithi037.stdout:6/678: symlink d0/d5/d1e/d44/d71/lf1 0 2024-03-29T08:07:34.558 INFO:tasks.workunit.client.0.smithi037.stdout:7/811: write dd/d21/f32 [6685794,18747] 0 2024-03-29T08:07:34.558 INFO:tasks.workunit.client.0.smithi037.stdout:6/679: read d0/d19/d34/d4e/fb4 [114639,45358] 0 2024-03-29T08:07:34.560 INFO:tasks.workunit.client.0.smithi037.stdout:2/601: fdatasync d0/d23/d44/dca/d22/f39 0 2024-03-29T08:07:34.562 INFO:tasks.workunit.client.0.smithi037.stdout:6/680: stat d0/d5/d1e/f5e 0 2024-03-29T08:07:34.564 INFO:tasks.workunit.client.0.smithi037.stdout:6/681: chown d0/l3a 723739 1 2024-03-29T08:07:34.564 INFO:tasks.workunit.client.0.smithi037.stdout:2/602: dread - d0/d23/d70/d71/f72 zero size 2024-03-29T08:07:34.566 INFO:tasks.workunit.client.0.smithi037.stdout:6/682: read d0/d19/d42/d60/d62/f9a [1079659,40582] 0 2024-03-29T08:07:34.569 INFO:tasks.workunit.client.0.smithi037.stdout:0/625: link d9/da/d1b/d22/l4f d9/da/dd/d13/d7b/ld7 0 2024-03-29T08:07:34.570 INFO:tasks.workunit.client.0.smithi037.stdout:0/626: fdatasync d9/da/d1b/d22/fce 0 2024-03-29T08:07:34.572 INFO:tasks.workunit.client.0.smithi037.stdout:7/812: mkdir dd/d1d/d109/d118 0 2024-03-29T08:07:34.573 INFO:tasks.workunit.client.0.smithi037.stdout:2/603: dread - d0/d23/d44/dca/d2e/d5f/d6f/fa9 zero size 2024-03-29T08:07:34.579 INFO:tasks.workunit.client.0.smithi037.stdout:8/696: write d5/d10/d19/f43 [910590,86511] 0 2024-03-29T08:07:34.580 INFO:tasks.workunit.client.0.smithi037.stdout:5/868: write d8/d22/d37/dbc/d85/f9e [451787,127376] 0 2024-03-29T08:07:34.582 INFO:tasks.workunit.client.0.smithi037.stdout:3/641: sync 2024-03-29T08:07:34.590 INFO:tasks.workunit.client.0.smithi037.stdout:3/642: dread dd/f1b [0,4194304] 0 2024-03-29T08:07:34.590 INFO:tasks.workunit.client.0.smithi037.stdout:6/683: write d0/d19/d34/d5a/f55 [1050408,54303] 0 2024-03-29T08:07:34.592 INFO:tasks.workunit.client.0.smithi037.stdout:3/643: read - dd/d46/d3e/fb7 zero size 2024-03-29T08:07:34.596 INFO:tasks.workunit.client.0.smithi037.stdout:0/627: creat d9/da/d1b/d22/fd8 x:0 0 0 2024-03-29T08:07:34.598 INFO:tasks.workunit.client.0.smithi037.stdout:8/697: mkdir d5/d10/d19/d3d/d4a/dc8/de6 0 2024-03-29T08:07:34.599 INFO:tasks.workunit.client.0.smithi037.stdout:3/644: mknod dd/d46/d3e/d47/cd5 0 2024-03-29T08:07:34.600 INFO:tasks.workunit.client.0.smithi037.stdout:2/604: rename d0/f21 to d0/d23/d44/dca/fce 0 2024-03-29T08:07:34.602 INFO:tasks.workunit.client.0.smithi037.stdout:7/813: symlink dd/d21/d65/da2/df8/d9c/l119 0 2024-03-29T08:07:34.610 INFO:tasks.workunit.client.0.smithi037.stdout:2/605: creat d0/d23/d44/dca/d22/d55/d92/fcf x:0 0 0 2024-03-29T08:07:34.611 INFO:tasks.workunit.client.0.smithi037.stdout:0/628: mknod d9/da/d1b/cd9 0 2024-03-29T08:07:34.619 INFO:tasks.workunit.client.0.smithi037.stdout:3/645: write dd/d15/d18/d4b/f8a [847975,30140] 0 2024-03-29T08:07:34.620 INFO:tasks.workunit.client.0.smithi037.stdout:8/698: write d5/d10/d2d/d34/d50/d83/f28 [532026,72803] 0 2024-03-29T08:07:34.630 INFO:tasks.workunit.client.0.smithi037.stdout:7/814: getdents dd/d21/d35/d3b/dce/de0 0 2024-03-29T08:07:34.633 INFO:tasks.workunit.client.0.smithi037.stdout:7/815: chown dd/d21/d35/d3b/fb8 123897 1 2024-03-29T08:07:34.633 INFO:tasks.workunit.client.0.smithi037.stdout:0/629: symlink d9/d74/d7d/lda 0 2024-03-29T08:07:34.636 INFO:tasks.workunit.client.0.smithi037.stdout:5/869: rename d8/d18/d24/l36 to d8/d4d/l128 0 2024-03-29T08:07:34.640 INFO:tasks.workunit.client.0.smithi037.stdout:0/630: symlink d9/d74/ldb 0 2024-03-29T08:07:34.642 INFO:tasks.workunit.client.0.smithi037.stdout:6/684: rename d0/d5/d1a/d61 to d0/d19/d34/d4e/df2 0 2024-03-29T08:07:34.647 INFO:tasks.workunit.client.0.smithi037.stdout:8/699: write d5/f1c [3770599,6126] 0 2024-03-29T08:07:34.649 INFO:tasks.workunit.client.0.smithi037.stdout:2/606: dwrite d0/d23/d44/dca/d2e/d5f/d6f/fa9 [0,4194304] 0 2024-03-29T08:07:34.651 INFO:tasks.workunit.client.0.smithi037.stdout:7/816: rename dd/f17 to dd/d21/d35/d42/d5c/d8c/f11a 0 2024-03-29T08:07:34.655 INFO:tasks.workunit.client.0.smithi037.stdout:6/685: write d0/d5/f73 [555394,49413] 0 2024-03-29T08:07:34.656 INFO:tasks.workunit.client.0.smithi037.stdout:3/646: dwrite dd/d46/d2d/d3a/fa6 [0,4194304] 0 2024-03-29T08:07:34.656 INFO:tasks.workunit.client.0.smithi037.stdout:0/631: mknod d9/da/dd/d20/d38/d8a/cdc 0 2024-03-29T08:07:34.661 INFO:tasks.workunit.client.0.smithi037.stdout:5/870: creat d8/d4d/d5c/f129 x:0 0 0 2024-03-29T08:07:34.661 INFO:tasks.workunit.client.0.smithi037.stdout:8/700: creat d5/d10/d2d/d9c/da1/fe7 x:0 0 0 2024-03-29T08:07:34.665 INFO:tasks.workunit.client.0.smithi037.stdout:1/722: sync 2024-03-29T08:07:34.665 INFO:tasks.workunit.client.0.smithi037.stdout:9/902: sync 2024-03-29T08:07:34.665 INFO:tasks.workunit.client.0.smithi037.stdout:4/740: sync 2024-03-29T08:07:34.671 INFO:tasks.workunit.client.0.smithi037.stdout:0/632: rename d9/da/dd/d20 to d9/da/dd/d20/d55/d5f/d9f/ddd 22 2024-03-29T08:07:34.673 INFO:tasks.workunit.client.0.smithi037.stdout:3/647: mkdir dd/d15/d27/d33/d97/dd6 0 2024-03-29T08:07:34.673 INFO:tasks.workunit.client.0.smithi037.stdout:5/871: chown d8/d22/db6/lc4 533 1 2024-03-29T08:07:34.673 INFO:tasks.workunit.client.0.smithi037.stdout:2/607: mknod d0/d23/d44/dca/d2e/d5f/cd0 0 2024-03-29T08:07:34.675 INFO:tasks.workunit.client.0.smithi037.stdout:6/686: truncate d0/d5/d1e/f5f 1152151 0 2024-03-29T08:07:34.680 INFO:tasks.workunit.client.0.smithi037.stdout:2/608: truncate d0/fcd 255283 0 2024-03-29T08:07:34.681 INFO:tasks.workunit.client.0.smithi037.stdout:1/723: rmdir d2/d6/dc/d7e 39 2024-03-29T08:07:34.681 INFO:tasks.workunit.client.0.smithi037.stdout:9/903: creat d1/d1f/df6/f12e x:0 0 0 2024-03-29T08:07:34.681 INFO:tasks.workunit.client.0.smithi037.stdout:8/701: chown d5/d10/d19/d3d/d4a/d4c/d67/d70/cb3 954592 1 2024-03-29T08:07:34.684 INFO:tasks.workunit.client.0.smithi037.stdout:5/872: symlink d8/d4d/ddf/l12a 0 2024-03-29T08:07:34.684 INFO:tasks.workunit.client.0.smithi037.stdout:3/648: readlink dd/d15/d27/la0 0 2024-03-29T08:07:34.688 INFO:tasks.workunit.client.0.smithi037.stdout:1/724: dread - d2/d4b/d80/fa8 zero size 2024-03-29T08:07:34.689 INFO:tasks.workunit.client.0.smithi037.stdout:6/687: truncate d0/d5/d1a/f2f 5474679 0 2024-03-29T08:07:34.690 INFO:tasks.workunit.client.0.smithi037.stdout:0/633: write d9/da/dd/d54/f56 [2818789,4627] 0 2024-03-29T08:07:34.692 INFO:tasks.workunit.client.0.smithi037.stdout:3/649: chown dd/d5e/f65 31315 1 2024-03-29T08:07:34.694 INFO:tasks.workunit.client.0.smithi037.stdout:8/702: fsync d5/d10/d2d/d34/d50/d83/d46/d8a/d96/fda 0 2024-03-29T08:07:34.695 INFO:tasks.workunit.client.0.smithi037.stdout:2/609: stat d0/d23/d44/dca/d22/d4e/c6c 0 2024-03-29T08:07:34.697 INFO:tasks.workunit.client.0.smithi037.stdout:4/741: symlink d3/d57/dec/ded/d1d/db/d45/d93/d96/da0/db2/lf9 0 2024-03-29T08:07:34.698 INFO:tasks.workunit.client.0.smithi037.stdout:7/817: dwrite dd/d21/d65/da2/df8/f87 [0,4194304] 0 2024-03-29T08:07:34.703 INFO:tasks.workunit.client.0.smithi037.stdout:1/725: readlink d2/d6/dc/d18/d22/d4f/d7c/lbf 0 2024-03-29T08:07:34.706 INFO:tasks.workunit.client.0.smithi037.stdout:1/726: chown d2/d6/dc/d18/d22/d4f/ff7 51 1 2024-03-29T08:07:34.713 INFO:tasks.workunit.client.0.smithi037.stdout:2/610: rmdir d0/d23/d44/dca/d22/d4e 39 2024-03-29T08:07:34.715 INFO:tasks.workunit.client.0.smithi037.stdout:0/634: dread d9/da/dd/f99 [0,4194304] 0 2024-03-29T08:07:34.721 INFO:tasks.workunit.client.0.smithi037.stdout:5/873: write d8/d18/f40 [2913603,128366] 0 2024-03-29T08:07:34.728 INFO:tasks.workunit.client.0.smithi037.stdout:4/742: dwrite d3/d54/d88/f8d [0,4194304] 0 2024-03-29T08:07:34.730 INFO:tasks.workunit.client.0.smithi037.stdout:8/703: creat d5/d78/dac/fe8 x:0 0 0 2024-03-29T08:07:34.732 INFO:tasks.workunit.client.0.smithi037.stdout:1/727: dwrite d2/d6/d13/dae/fc7 [0,4194304] 0 2024-03-29T08:07:34.738 INFO:tasks.workunit.client.0.smithi037.stdout:7/818: getdents dd/d68/d112 0 2024-03-29T08:07:34.749 INFO:tasks.workunit.client.0.smithi037.stdout:9/904: link d1/d1f/d3a/fc0 d1/d1f/dfd/d127/f12f 0 2024-03-29T08:07:34.749 INFO:tasks.workunit.client.0.smithi037.stdout:4/743: dread d3/d57/f67 [0,4194304] 0 2024-03-29T08:07:34.751 INFO:tasks.workunit.client.0.smithi037.stdout:5/874: chown d8/d18/d24/l41 6414171 1 2024-03-29T08:07:34.753 INFO:tasks.workunit.client.0.smithi037.stdout:6/688: truncate d0/d5/d1e/f5e 611286 0 2024-03-29T08:07:34.756 INFO:tasks.workunit.client.0.smithi037.stdout:8/704: creat d5/d10/d45/fe9 x:0 0 0 2024-03-29T08:07:34.758 INFO:tasks.workunit.client.0.smithi037.stdout:2/611: mknod d0/d23/d44/dca/d22/d4e/cd1 0 2024-03-29T08:07:34.761 INFO:tasks.workunit.client.0.smithi037.stdout:1/728: chown d2/lf0 11993 1 2024-03-29T08:07:34.761 INFO:tasks.workunit.client.0.smithi037.stdout:3/650: creat dd/d46/d2d/fd7 x:0 0 0 2024-03-29T08:07:34.763 INFO:tasks.workunit.client.0.smithi037.stdout:4/744: rmdir d3/d57/dec 39 2024-03-29T08:07:34.767 INFO:tasks.workunit.client.0.smithi037.stdout:5/875: rename d8/fa0 to d8/d34/f12b 0 2024-03-29T08:07:34.771 INFO:tasks.workunit.client.0.smithi037.stdout:7/819: creat dd/d1d/d34/dc8/f11b x:0 0 0 2024-03-29T08:07:34.772 INFO:tasks.workunit.client.0.smithi037.stdout:2/612: truncate d0/d23/f65 144177 0 2024-03-29T08:07:34.773 INFO:tasks.workunit.client.0.smithi037.stdout:7/820: fsync dd/d68/f78 0 2024-03-29T08:07:34.774 INFO:tasks.workunit.client.0.smithi037.stdout:8/705: truncate d5/d10/d2d/d34/d50/d83/f5f 272865 0 2024-03-29T08:07:34.776 INFO:tasks.workunit.client.0.smithi037.stdout:8/706: chown d5/d10/d2d/d34/d50/fad 51658391 1 2024-03-29T08:07:34.782 INFO:tasks.workunit.client.0.smithi037.stdout:6/689: creat d0/d19/d34/d4e/d69/dee/ff3 x:0 0 0 2024-03-29T08:07:34.787 INFO:tasks.workunit.client.0.smithi037.stdout:1/729: mkdir d2/d6/dc/d18/d1d/dfc 0 2024-03-29T08:07:34.790 INFO:tasks.workunit.client.0.smithi037.stdout:1/730: write d2/d40/fbc [939387,20562] 0 2024-03-29T08:07:34.790 INFO:tasks.workunit.client.0.smithi037.stdout:9/905: link d1/d4/d95/cb1 d1/d3/d44/c130 0 2024-03-29T08:07:34.795 INFO:tasks.workunit.client.0.smithi037.stdout:8/707: creat d5/d10/d2d/d34/d50/fea x:0 0 0 2024-03-29T08:07:34.798 INFO:tasks.workunit.client.0.smithi037.stdout:3/651: write dd/d46/d2d/d3a/d5f/f77 [610777,25788] 0 2024-03-29T08:07:34.800 INFO:tasks.workunit.client.0.smithi037.stdout:5/876: symlink d8/d4d/d5c/l12c 0 2024-03-29T08:07:34.802 INFO:tasks.workunit.client.0.smithi037.stdout:3/652: write dd/d15/d18/d4b/f8a [413012,71396] 0 2024-03-29T08:07:34.804 INFO:tasks.workunit.client.0.smithi037.stdout:4/745: dread d3/d4/f34 [0,4194304] 0 2024-03-29T08:07:34.808 INFO:tasks.workunit.client.0.smithi037.stdout:7/821: symlink dd/d1d/d49/d81/d84/l11c 0 2024-03-29T08:07:34.812 INFO:tasks.workunit.client.0.smithi037.stdout:1/731: creat d2/d4b/ffd x:0 0 0 2024-03-29T08:07:34.813 INFO:tasks.workunit.client.0.smithi037.stdout:7/822: truncate dd/d21/d65/da2/df8/ff4 1010488 0 2024-03-29T08:07:34.814 INFO:tasks.workunit.client.0.smithi037.stdout:9/906: creat d1/d1f/dfd/f131 x:0 0 0 2024-03-29T08:07:34.815 INFO:tasks.workunit.client.0.smithi037.stdout:7/823: write dd/d1d/f2a [1174849,7183] 0 2024-03-29T08:07:34.819 INFO:tasks.workunit.client.0.smithi037.stdout:6/690: creat d0/d19/d34/d5a/dbf/dde/ff4 x:0 0 0 2024-03-29T08:07:34.822 INFO:tasks.workunit.client.0.smithi037.stdout:3/653: creat dd/d15/d27/d33/d84/fd8 x:0 0 0 2024-03-29T08:07:34.831 INFO:tasks.workunit.client.0.smithi037.stdout:5/877: dread d8/d4d/d5c/f94 [0,4194304] 0 2024-03-29T08:07:34.832 INFO:tasks.workunit.client.0.smithi037.stdout:4/746: dread d3/d57/dec/ded/d1d/db/d45/d1e/d59/db5/dbe/fd7 [0,4194304] 0 2024-03-29T08:07:34.832 INFO:tasks.workunit.client.0.smithi037.stdout:1/732: fsync d2/d6/dc/d18/f5f 0 2024-03-29T08:07:34.834 INFO:tasks.workunit.client.0.smithi037.stdout:8/708: rename d5/d10/d19/d8d/cdb to d5/d10/d2d/d34/d50/ddc/ceb 0 2024-03-29T08:07:34.836 INFO:tasks.workunit.client.0.smithi037.stdout:9/907: creat d1/d1f/df6/f132 x:0 0 0 2024-03-29T08:07:34.840 INFO:tasks.workunit.client.0.smithi037.stdout:2/613: dwrite d0/d23/f65 [0,4194304] 0 2024-03-29T08:07:34.842 INFO:tasks.workunit.client.0.smithi037.stdout:3/654: dread - dd/d46/d3e/d47/f7f zero size 2024-03-29T08:07:34.852 INFO:tasks.workunit.client.0.smithi037.stdout:7/824: mkdir dd/d21/d65/da2/d11d 0 2024-03-29T08:07:34.852 INFO:tasks.workunit.client.0.smithi037.stdout:8/709: rmdir d5/d10/d19/d3d 39 2024-03-29T08:07:34.860 INFO:tasks.workunit.client.0.smithi037.stdout:9/908: unlink d1/d4/d19/d1e/d4a/d6f/da2/ffc 0 2024-03-29T08:07:34.866 INFO:tasks.workunit.client.0.smithi037.stdout:1/733: getdents d2/d6/dc/d18/d1d/dfc 0 2024-03-29T08:07:34.868 INFO:tasks.workunit.client.0.smithi037.stdout:5/878: getdents d8/d22/d37/d8d/db3/df9 0 2024-03-29T08:07:34.870 INFO:tasks.workunit.client.0.smithi037.stdout:4/747: symlink d3/d57/dec/ded/d1d/dc2/lfa 0 2024-03-29T08:07:34.873 INFO:tasks.workunit.client.0.smithi037.stdout:1/734: dread - d2/d40/fa5 zero size 2024-03-29T08:07:34.881 INFO:tasks.workunit.client.0.smithi037.stdout:6/691: dwrite d0/d19/d34/d4e/d69/d79/fa0 [0,4194304] 0 2024-03-29T08:07:34.889 INFO:tasks.workunit.client.0.smithi037.stdout:3/655: dread dd/d46/d2d/d3a/d6c/f8b [0,4194304] 0 2024-03-29T08:07:34.891 INFO:tasks.workunit.client.0.smithi037.stdout:7/825: write dd/d1d/f53 [4103016,112361] 0 2024-03-29T08:07:34.892 INFO:tasks.workunit.client.0.smithi037.stdout:8/710: dwrite d5/d10/d2d/d34/d50/d83/d46/d4e/fd8 [0,4194304] 0 2024-03-29T08:07:34.895 INFO:tasks.workunit.client.0.smithi037.stdout:7/826: stat dd/d21/d35/d3b/fb8 0 2024-03-29T08:07:34.900 INFO:tasks.workunit.client.0.smithi037.stdout:1/735: rename d2/d6/dc/d18/d22/fa7 to d2/d6/dc/d5d/ffe 0 2024-03-29T08:07:34.903 INFO:tasks.workunit.client.0.smithi037.stdout:5/879: getdents d8/d22/d37/d8d/db3/dcb/deb/d88 0 2024-03-29T08:07:34.905 INFO:tasks.workunit.client.0.smithi037.stdout:6/692: creat d0/d20/d95/ff5 x:0 0 0 2024-03-29T08:07:34.908 INFO:tasks.workunit.client.0.smithi037.stdout:1/736: fdatasync d2/d6/dc/fe4 0 2024-03-29T08:07:34.909 INFO:tasks.workunit.client.0.smithi037.stdout:3/656: dread - dd/d15/d27/f89 zero size 2024-03-29T08:07:34.911 INFO:tasks.workunit.client.0.smithi037.stdout:4/748: rename d3/d57/dec/ded/d1d/db/d45/d1e/d59/db5/dbe to d3/d57/dec/ded/d1d/dad/dfb 0 2024-03-29T08:07:34.912 INFO:tasks.workunit.client.0.smithi037.stdout:7/827: creat dd/f11e x:0 0 0 2024-03-29T08:07:34.920 INFO:tasks.workunit.client.0.smithi037.stdout:6/693: write d0/d5/d3f/fb1 [30182,85958] 0 2024-03-29T08:07:34.930 INFO:tasks.workunit.client.0.smithi037.stdout:9/909: dwrite d1/d4/d19/d1e/d4a/d6f/da2/d31/daa/fef [0,4194304] 0 2024-03-29T08:07:34.936 INFO:tasks.workunit.client.0.smithi037.stdout:6/694: chown d0/cd9 313921737 1 2024-03-29T08:07:34.941 INFO:tasks.workunit.client.0.smithi037.stdout:3/657: dread dd/d15/d18/d4b/f8a [0,4194304] 0 2024-03-29T08:07:34.949 INFO:tasks.workunit.client.0.smithi037.stdout:8/711: dwrite d5/d10/d19/d3d/f5d [0,4194304] 0 2024-03-29T08:07:34.954 INFO:tasks.workunit.client.0.smithi037.stdout:9/910: mknod d1/d4/d19/d69/d75/c133 0 2024-03-29T08:07:34.956 INFO:tasks.workunit.client.0.smithi037.stdout:4/749: fdatasync d3/d57/dec/ded/d1d/db/d45/d1e/d59/ff7 0 2024-03-29T08:07:34.959 INFO:tasks.workunit.client.0.smithi037.stdout:5/880: dread d8/d34/f12b [0,4194304] 0 2024-03-29T08:07:34.960 INFO:tasks.workunit.client.0.smithi037.stdout:3/658: symlink dd/d46/d2d/d3a/d85/ld9 0 2024-03-29T08:07:34.963 INFO:tasks.workunit.client.0.smithi037.stdout:9/911: mknod d1/d1f/df6/c134 0 2024-03-29T08:07:34.967 INFO:tasks.workunit.client.0.smithi037.stdout:5/881: chown d8/d22/d4f/d70/d79/fd7 16 1 2024-03-29T08:07:34.969 INFO:tasks.workunit.client.0.smithi037.stdout:7/828: dread dd/d21/d35/d42/d5c/d8c/ff3 [0,4194304] 0 2024-03-29T08:07:34.970 INFO:tasks.workunit.client.0.smithi037.stdout:4/750: readlink d3/d57/dec/ded/d1d/db/d72/lac 0 2024-03-29T08:07:34.971 INFO:tasks.workunit.client.0.smithi037.stdout:1/737: dwrite d2/d6/dc/d18/f3e [0,4194304] 0 2024-03-29T08:07:34.977 INFO:tasks.workunit.client.0.smithi037.stdout:3/659: rename dd/d15/d27 to dd/d46/d2d/d3a/d6c/db1/dc0/dda 0 2024-03-29T08:07:34.978 INFO:tasks.workunit.client.0.smithi037.stdout:6/695: write d0/d6a/d7b/f9e [166281,40494] 0 2024-03-29T08:07:34.979 INFO:tasks.workunit.client.0.smithi037.stdout:5/882: read - d8/d18/fd0 zero size 2024-03-29T08:07:34.980 INFO:tasks.workunit.client.0.smithi037.stdout:1/738: readlink d2/d6/dc/d18/d22/d4f/d7c/lbf 0 2024-03-29T08:07:34.981 INFO:tasks.workunit.client.0.smithi037.stdout:9/912: creat d1/d4/d19/d1e/d71/f135 x:0 0 0 2024-03-29T08:07:34.986 INFO:tasks.workunit.client.0.smithi037.stdout:3/660: write dd/d46/d2d/d3a/fbf [323011,62972] 0 2024-03-29T08:07:34.988 INFO:tasks.workunit.client.0.smithi037.stdout:9/913: write d1/d1f/f126 [63205,35814] 0 2024-03-29T08:07:34.992 INFO:tasks.workunit.client.0.smithi037.stdout:6/696: fdatasync d0/d19/d34/d4e/df2/f65 0 2024-03-29T08:07:35.001 INFO:tasks.workunit.client.0.smithi037.stdout:5/883: write d8/d22/d37/dbc/ff3 [711567,49331] 0 2024-03-29T08:07:35.006 INFO:tasks.workunit.client.0.smithi037.stdout:7/829: dwrite dd/f77 [0,4194304] 0 2024-03-29T08:07:35.020 INFO:tasks.workunit.client.0.smithi037.stdout:3/661: rename dd/d46/d2d/d3a/d5f to dd/d46/d2d/d3a/d6c/db1/dc0/dda/ddb 0 2024-03-29T08:07:35.023 INFO:tasks.workunit.client.0.smithi037.stdout:3/662: chown c9 68484265 1 2024-03-29T08:07:35.030 INFO:tasks.workunit.client.0.smithi037.stdout:4/751: getdents d3/d57/dec/ded/d1d/db/d45/d93/d96 0 2024-03-29T08:07:35.037 INFO:tasks.workunit.client.0.smithi037.stdout:9/914: rename d1/d4/d19/d1e/d4a/d6f/da2/c8f to d1/d4/d19/d1e/d4a/d6f/da2/d31/daa/dba/de3/c136 0 2024-03-29T08:07:35.041 INFO:tasks.workunit.client.0.smithi037.stdout:6/697: dwrite d0/d19/d34/d4e/f54 [0,4194304] 0 2024-03-29T08:07:35.041 INFO:tasks.workunit.client.0.smithi037.stdout:3/663: truncate dd/d46/d2d/d3a/fb6 1022364 0 2024-03-29T08:07:35.043 INFO:tasks.workunit.client.0.smithi037.stdout:8/712: dwrite d5/d10/d19/d3d/d4a/dc8/fe3 [0,4194304] 0 2024-03-29T08:07:35.043 INFO:tasks.workunit.client.0.smithi037.stdout:1/739: getdents d2/d6/dc/d18/d1d/d5e 0 2024-03-29T08:07:35.044 INFO:tasks.workunit.client.0.smithi037.stdout:4/752: write d3/fe3 [263635,48066] 0 2024-03-29T08:07:35.045 INFO:tasks.workunit.client.0.smithi037.stdout:0/635: sync 2024-03-29T08:07:35.046 INFO:tasks.workunit.client.0.smithi037.stdout:5/884: dwrite d8/d4d/ddf/fe9 [0,4194304] 0 2024-03-29T08:07:35.052 INFO:tasks.workunit.client.0.smithi037.stdout:7/830: mkdir dd/d21/d35/d42/d5c/dfd/d11f 0 2024-03-29T08:07:35.056 INFO:tasks.workunit.client.0.smithi037.stdout:9/915: creat d1/d3/d44/f137 x:0 0 0 2024-03-29T08:07:35.058 INFO:tasks.workunit.client.0.smithi037.stdout:0/636: dread - d9/da/dd/d13/d7b/fa6 zero size 2024-03-29T08:07:35.060 INFO:tasks.workunit.client.0.smithi037.stdout:6/698: rename d0/d5/d1e/cbe to d0/d20/cf6 0 2024-03-29T08:07:35.067 INFO:tasks.workunit.client.0.smithi037.stdout:7/831: mkdir dd/d1d/d49/d81/d120 0 2024-03-29T08:07:35.070 INFO:tasks.workunit.client.0.smithi037.stdout:7/832: fdatasync dd/dc3/fd7 0 2024-03-29T08:07:35.071 INFO:tasks.workunit.client.0.smithi037.stdout:6/699: rmdir d0/d19/d34/d4e/d94 39 2024-03-29T08:07:35.073 INFO:tasks.workunit.client.0.smithi037.stdout:7/833: fsync dd/d72/f98 0 2024-03-29T08:07:35.073 INFO:tasks.workunit.client.0.smithi037.stdout:4/753: creat d3/d57/dec/ded/d1d/db/d45/d93/ffc x:0 0 0 2024-03-29T08:07:35.077 INFO:tasks.workunit.client.0.smithi037.stdout:3/664: truncate dd/d46/d2d/d3a/d6c/db1/dc0/dda/d33/f96 3248122 0 2024-03-29T08:07:35.078 INFO:tasks.workunit.client.0.smithi037.stdout:1/740: creat d2/d6/dc/d18/d1d/fff x:0 0 0 2024-03-29T08:07:35.082 INFO:tasks.workunit.client.0.smithi037.stdout:5/885: creat d8/d22/d37/d8d/db3/dee/f12d x:0 0 0 2024-03-29T08:07:35.086 INFO:tasks.workunit.client.0.smithi037.stdout:1/741: write d2/fb8 [305679,45868] 0 2024-03-29T08:07:35.086 INFO:tasks.workunit.client.0.smithi037.stdout:6/700: mknod d0/d6a/d7b/d77/cf7 0 2024-03-29T08:07:35.087 INFO:tasks.workunit.client.0.smithi037.stdout:0/637: rename d9/da/d1b/d22/la5 to d9/da/dd/d20/d55/dac/lde 0 2024-03-29T08:07:35.091 INFO:tasks.workunit.client.0.smithi037.stdout:0/638: fdatasync d9/d74/fa1 0 2024-03-29T08:07:35.093 INFO:tasks.workunit.client.0.smithi037.stdout:8/713: getdents d5/d10/d19/d3d/d4a/d4c/d67/d85 0 2024-03-29T08:07:35.094 INFO:tasks.workunit.client.0.smithi037.stdout:7/834: mknod dd/d21/c121 0 2024-03-29T08:07:35.095 INFO:tasks.workunit.client.0.smithi037.stdout:0/639: dread - d9/da/dd/d20/d72/fcd zero size 2024-03-29T08:07:35.099 INFO:tasks.workunit.client.0.smithi037.stdout:4/754: creat d3/d57/dec/ded/d1d/db/d45/d1e/d59/d79/d95/ffd x:0 0 0 2024-03-29T08:07:35.100 INFO:tasks.workunit.client.0.smithi037.stdout:5/886: symlink d8/d4d/ddf/l12e 0 2024-03-29T08:07:35.103 INFO:tasks.workunit.client.0.smithi037.stdout:3/665: mknod dd/cdc 0 2024-03-29T08:07:35.109 INFO:tasks.workunit.client.0.smithi037.stdout:7/835: mkdir dd/d21/d35/d42/ddb/de2/d122 0 2024-03-29T08:07:35.110 INFO:tasks.workunit.client.0.smithi037.stdout:4/755: creat d3/d4/d27/ffe x:0 0 0 2024-03-29T08:07:35.113 INFO:tasks.workunit.client.0.smithi037.stdout:5/887: creat d8/d22/d37/d8d/db3/dcb/f12f x:0 0 0 2024-03-29T08:07:35.114 INFO:tasks.workunit.client.0.smithi037.stdout:7/836: read - dd/d21/d85/fea zero size 2024-03-29T08:07:35.117 INFO:tasks.workunit.client.0.smithi037.stdout:8/714: getdents d5/d78/dac 0 2024-03-29T08:07:35.122 INFO:tasks.workunit.client.0.smithi037.stdout:4/756: write d3/d57/dec/ded/d1d/db/d45/d1e/d59/d79/fa6 [2251811,42283] 0 2024-03-29T08:07:35.124 INFO:tasks.workunit.client.0.smithi037.stdout:7/837: write dd/d21/d35/d60/fd1 [1036525,97201] 0 2024-03-29T08:07:35.132 INFO:tasks.workunit.client.0.smithi037.stdout:5/888: dread d8/d22/d37/d8d/db3/dcb/deb/fe6 [0,4194304] 0 2024-03-29T08:07:35.132 INFO:tasks.workunit.client.0.smithi037.stdout:4/757: creat d3/de7/fff x:0 0 0 2024-03-29T08:07:35.145 INFO:tasks.workunit.client.0.smithi037.stdout:5/889: mknod d8/d22/db6/d103/c130 0 2024-03-29T08:07:35.146 INFO:tasks.workunit.client.0.smithi037.stdout:3/666: write dd/d5e/f80 [534295,119159] 0 2024-03-29T08:07:35.146 INFO:tasks.workunit.client.0.smithi037.stdout:7/838: dwrite dd/d1d/d34/f3f [0,4194304] 0 2024-03-29T08:07:35.150 INFO:tasks.workunit.client.0.smithi037.stdout:0/640: truncate d9/da/dd/d13/f5e 919683 0 2024-03-29T08:07:35.158 INFO:tasks.workunit.client.0.smithi037.stdout:8/715: write d5/f13 [5707086,124153] 0 2024-03-29T08:07:35.158 INFO:tasks.workunit.client.0.smithi037.stdout:0/641: write d9/da/dd/f23 [4188859,63965] 0 2024-03-29T08:07:35.160 INFO:tasks.workunit.client.0.smithi037.stdout:1/742: dwrite d2/d4b/fc4 [0,4194304] 0 2024-03-29T08:07:35.166 INFO:tasks.workunit.client.0.smithi037.stdout:4/758: write d3/d57/f8e [4018687,119365] 0 2024-03-29T08:07:35.169 INFO:tasks.workunit.client.0.smithi037.stdout:7/839: rename dd/d21/d65/da2/df8/f87 to dd/d21/d65/da2/df8/f123 0 2024-03-29T08:07:35.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:07:35 smithi037 ceph-mon[35162]: pgmap v49: 65 pgs: 65 active+clean; 3.5 GiB data, 12 GiB used, 525 GiB / 536 GiB avail; 131 MiB/s rd, 300 MiB/s wr, 662 op/s 2024-03-29T08:07:35.178 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:07:35 smithi037 ceph-mon[35162]: mgrmap e26: smithi148.onhgvm(active, since 92s), standbys: smithi037.wjikii 2024-03-29T08:07:35.180 INFO:tasks.workunit.client.0.smithi037.stdout:8/716: creat d5/d78/fec x:0 0 0 2024-03-29T08:07:35.181 INFO:tasks.workunit.client.0.smithi037.stdout:3/667: mknod dd/d46/d2d/d3a/d6c/db1/dc0/dda/d33/d97/dd6/cdd 0 2024-03-29T08:07:35.198 INFO:tasks.workunit.client.0.smithi037.stdout:5/890: rename d8/d22/db6/d103 to d8/d4d/d5c/d131 0 2024-03-29T08:07:35.198 INFO:tasks.workunit.client.0.smithi037.stdout:3/668: mknod dd/d46/d2d/d3a/d6c/db1/dc0/dda/d33/d97/cde 0 2024-03-29T08:07:35.200 INFO:tasks.workunit.client.0.smithi037.stdout:1/743: fdatasync d2/d6/dc/d18/d22/ff9 0 2024-03-29T08:07:35.204 INFO:tasks.workunit.client.0.smithi037.stdout:8/717: unlink d5/d10/d19/d3d/d4a/d4c/fcd 0 2024-03-29T08:07:35.205 INFO:tasks.workunit.client.0.smithi037.stdout:4/759: dwrite d3/d4/d27/fbc [0,4194304] 0 2024-03-29T08:07:35.206 INFO:tasks.workunit.client.0.smithi037.stdout:3/669: stat dd/d46/d2d/d3a/d6c/db1/dc0/dda/d33/d75 0 2024-03-29T08:07:35.215 INFO:tasks.workunit.client.0.smithi037.stdout:7/840: rename dd/f11e to dd/d21/d35/d42/d5c/d8c/f124 0 2024-03-29T08:07:35.219 INFO:tasks.workunit.client.0.smithi037.stdout:7/841: fsync dd/d21/d35/fe4 0 2024-03-29T08:07:35.219 INFO:tasks.workunit.client.0.smithi037.stdout:0/642: dwrite d9/da/dd/d20/d38/d8a/f9a [0,4194304] 0 2024-03-29T08:07:35.230 INFO:tasks.workunit.client.0.smithi037.stdout:5/891: truncate d8/d18/f50 3611512 0 2024-03-29T08:07:35.230 INFO:tasks.workunit.client.0.smithi037.stdout:3/670: mknod dd/d46/d2d/d3a/d6c/cdf 0 2024-03-29T08:07:35.231 INFO:tasks.workunit.client.0.smithi037.stdout:1/744: creat d2/d6/dc/d18/dbd/f100 x:0 0 0 2024-03-29T08:07:35.232 INFO:tasks.workunit.client.0.smithi037.stdout:8/718: chown d5/c3f 115 1 2024-03-29T08:07:35.236 INFO:tasks.workunit.client.0.smithi037.stdout:5/892: chown d8/d22/d37/de8/f108 1833 1 2024-03-29T08:07:35.237 INFO:tasks.workunit.client.0.smithi037.stdout:3/671: write dd/d5e/f5c [1829497,40211] 0 2024-03-29T08:07:35.237 INFO:tasks.workunit.client.0.smithi037.stdout:0/643: unlink d9/da/dd/d13/d83/f81 0 2024-03-29T08:07:35.241 INFO:tasks.workunit.client.0.smithi037.stdout:4/760: mknod d3/d57/dec/ded/d1d/db/d45/d1e/d59/d60/db7/c100 0 2024-03-29T08:07:35.242 INFO:tasks.workunit.client.0.smithi037.stdout:5/893: fsync d8/d22/d4f/f5e 0 2024-03-29T08:07:35.242 INFO:tasks.workunit.client.0.smithi037.stdout:0/644: write d9/d74/fa9 [32031,122575] 0 2024-03-29T08:07:35.245 INFO:tasks.workunit.client.0.smithi037.stdout:8/719: creat d5/d10/d19/d8d/fed x:0 0 0 2024-03-29T08:07:35.250 INFO:tasks.workunit.client.0.smithi037.stdout:5/894: stat d8/d4d/d5c/dbf/fc2 0 2024-03-29T08:07:35.259 INFO:tasks.workunit.client.0.smithi037.stdout:8/720: mknod d5/d10/d2d/d34/d50/d83/d46/d4e/cee 0 2024-03-29T08:07:35.262 INFO:tasks.workunit.client.0.smithi037.stdout:3/672: dwrite dd/f3f [4194304,4194304] 0 2024-03-29T08:07:35.266 INFO:tasks.workunit.client.0.smithi037.stdout:2/614: sync 2024-03-29T08:07:35.269 INFO:tasks.workunit.client.0.smithi037.stdout:0/645: rename d9/da/dd/d59/f39 to d9/d74/d7d/fdf 0 2024-03-29T08:07:35.273 INFO:tasks.workunit.client.0.smithi037.stdout:2/615: write d0/d23/d44/dca/d2e/d5f/d6f/d5e/fb1 [227813,112421] 0 2024-03-29T08:07:35.275 INFO:tasks.workunit.client.0.smithi037.stdout:7/842: write dd/f67 [1097912,120662] 0 2024-03-29T08:07:35.279 INFO:tasks.workunit.client.0.smithi037.stdout:1/745: dread d2/d6/dc/d18/d1d/d21/fdb [0,4194304] 0 2024-03-29T08:07:35.282 INFO:tasks.workunit.client.0.smithi037.stdout:3/673: creat dd/d46/fe0 x:0 0 0 2024-03-29T08:07:35.284 INFO:tasks.workunit.client.0.smithi037.stdout:1/746: write d2/d6/dc/d18/d22/d4f/ff7 [759321,49747] 0 2024-03-29T08:07:35.288 INFO:tasks.workunit.client.0.smithi037.stdout:2/616: chown d0/l12 5 1 2024-03-29T08:07:35.295 INFO:tasks.workunit.client.0.smithi037.stdout:7/843: mkdir dd/d1d/d49/d81/dd8/d125 0 2024-03-29T08:07:35.299 INFO:tasks.workunit.client.0.smithi037.stdout:1/747: rmdir d2/d6/dc/d18/d22 39 2024-03-29T08:07:35.302 INFO:tasks.workunit.client.0.smithi037.stdout:4/761: write d3/d4/f2f [2345685,48552] 0 2024-03-29T08:07:35.305 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:07:35 smithi148 ceph-mon[44992]: pgmap v49: 65 pgs: 65 active+clean; 3.5 GiB data, 12 GiB used, 525 GiB / 536 GiB avail; 131 MiB/s rd, 300 MiB/s wr, 662 op/s 2024-03-29T08:07:35.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:07:35 smithi148 ceph-mon[44992]: mgrmap e26: smithi148.onhgvm(active, since 92s), standbys: smithi037.wjikii 2024-03-29T08:07:35.306 INFO:tasks.workunit.client.0.smithi037.stdout:7/844: creat dd/d21/d35/d42/d5c/f126 x:0 0 0 2024-03-29T08:07:35.307 INFO:tasks.workunit.client.0.smithi037.stdout:1/748: symlink d2/d6/dc/d58/l101 0 2024-03-29T08:07:35.309 INFO:tasks.workunit.client.0.smithi037.stdout:7/845: read dd/d21/f75 [830413,19296] 0 2024-03-29T08:07:35.312 INFO:tasks.workunit.client.0.smithi037.stdout:1/749: truncate d2/f3a 542674 0 2024-03-29T08:07:35.318 INFO:tasks.workunit.client.0.smithi037.stdout:8/721: write d5/d10/f23 [4092974,28900] 0 2024-03-29T08:07:35.327 INFO:tasks.workunit.client.0.smithi037.stdout:7/846: rename dd/d21/d65/da2/df8/d9c to dd/d68/d112/d127 0 2024-03-29T08:07:35.332 INFO:tasks.workunit.client.0.smithi037.stdout:1/750: creat d2/d4b/d80/dfb/f102 x:0 0 0 2024-03-29T08:07:35.336 INFO:tasks.workunit.client.0.smithi037.stdout:1/751: write d2/d4b/ffd [511399,99703] 0 2024-03-29T08:07:35.342 INFO:tasks.workunit.client.0.smithi037.stdout:4/762: link d3/d57/d6a/f77 d3/d57/dec/ded/d1d/db/d45/d93/d96/da0/db2/f101 0 2024-03-29T08:07:35.346 INFO:tasks.workunit.client.0.smithi037.stdout:7/847: dread dd/f77 [0,4194304] 0 2024-03-29T08:07:35.352 INFO:tasks.workunit.client.0.smithi037.stdout:3/674: dwrite dd/d46/d2d/d3a/d6c/f8b [0,4194304] 0 2024-03-29T08:07:35.358 INFO:tasks.workunit.client.0.smithi037.stdout:2/617: dwrite d0/d23/d44/dca/d2e/f91 [0,4194304] 0 2024-03-29T08:07:35.365 INFO:tasks.workunit.client.0.smithi037.stdout:8/722: rmdir d5/d10/d2d/d34/d50/d83/d46/d4e/de2 0 2024-03-29T08:07:35.370 INFO:tasks.workunit.client.0.smithi037.stdout:8/723: stat d5/d10/d2d/d34/d50/lb5 0 2024-03-29T08:07:35.376 INFO:tasks.workunit.client.0.smithi037.stdout:1/752: dwrite d2/d4b/d80/dfb/f102 [0,4194304] 0 2024-03-29T08:07:35.378 INFO:tasks.workunit.client.0.smithi037.stdout:0/646: dwrite d9/d74/d7d/fdf [0,4194304] 0 2024-03-29T08:07:35.383 INFO:tasks.workunit.client.0.smithi037.stdout:7/848: creat dd/d21/d35/d42/d5c/d8c/f128 x:0 0 0 2024-03-29T08:07:35.385 INFO:tasks.workunit.client.0.smithi037.stdout:0/647: chown d9/da/dd 24892229 1 2024-03-29T08:07:35.386 INFO:tasks.workunit.client.0.smithi037.stdout:2/618: fdatasync d0/d23/d70/f99 0 2024-03-29T08:07:35.390 INFO:tasks.workunit.client.0.smithi037.stdout:1/753: chown d2/d6/dc/d18/d22/ff9 198730716 1 2024-03-29T08:07:35.392 INFO:tasks.workunit.client.0.smithi037.stdout:7/849: creat dd/d68/f129 x:0 0 0 2024-03-29T08:07:35.393 INFO:tasks.workunit.client.0.smithi037.stdout:0/648: write d9/da/dd/d54/f56 [3159656,16781] 0 2024-03-29T08:07:35.398 INFO:tasks.workunit.client.0.smithi037.stdout:7/850: chown dd/d1d/d109/d118 14515 1 2024-03-29T08:07:35.401 INFO:tasks.workunit.client.0.smithi037.stdout:8/724: dread d5/d10/d19/f43 [0,4194304] 0 2024-03-29T08:07:35.411 INFO:tasks.workunit.client.0.smithi037.stdout:7/851: write dd/f3e [519388,111386] 0 2024-03-29T08:07:35.412 INFO:tasks.workunit.client.0.smithi037.stdout:0/649: creat d9/da/dd/d20/d55/d5f/fe0 x:0 0 0 2024-03-29T08:07:35.412 INFO:tasks.workunit.client.0.smithi037.stdout:2/619: link d0/d53/f88 d0/d23/d44/dca/d2e/fd2 0 2024-03-29T08:07:35.413 INFO:tasks.workunit.client.0.smithi037.stdout:8/725: chown d5/d10/d45/d4b 18 1 2024-03-29T08:07:35.418 INFO:tasks.workunit.client.0.smithi037.stdout:7/852: readlink dd/d1d/d34/lb6 0 2024-03-29T08:07:35.418 INFO:tasks.workunit.client.0.smithi037.stdout:2/620: chown d0/d23/d44/dca/d22 15579 1 2024-03-29T08:07:35.426 INFO:tasks.workunit.client.0.smithi037.stdout:4/763: dwrite d3/d57/dec/ded/d1d/db/f1a [4194304,4194304] 0 2024-03-29T08:07:35.432 INFO:tasks.workunit.client.0.smithi037.stdout:3/675: dread dd/d46/f3c [0,4194304] 0 2024-03-29T08:07:35.456 INFO:tasks.workunit.client.0.smithi037.stdout:8/726: creat d5/d10/d2d/d34/d50/d83/d46/d8a/d96/fef x:0 0 0 2024-03-29T08:07:35.459 INFO:tasks.workunit.client.0.smithi037.stdout:7/853: rename dd/d21/d65/da2/df8/cf1 to dd/d21/d35/d42/d5c/dfd/c12a 0 2024-03-29T08:07:35.460 INFO:tasks.workunit.client.0.smithi037.stdout:4/764: mkdir d3/d57/dec/ded/d1d/db/d45/d1e/d102 0 2024-03-29T08:07:35.463 INFO:tasks.workunit.client.0.smithi037.stdout:3/676: fdatasync dd/d46/d3e/f7e 0 2024-03-29T08:07:35.464 INFO:tasks.workunit.client.0.smithi037.stdout:1/754: link d2/fb8 d2/d6/dc/d18/d1d/d5e/d72/f103 0 2024-03-29T08:07:35.466 INFO:tasks.workunit.client.0.smithi037.stdout:0/650: symlink d9/le1 0 2024-03-29T08:07:35.468 INFO:tasks.workunit.client.0.smithi037.stdout:0/651: stat d9/da/dd/d20/d38/d8a/cb8 0 2024-03-29T08:07:35.469 INFO:tasks.workunit.client.0.smithi037.stdout:7/854: write dd/d21/d35/d42/d5c/d8c/f124 [154534,28597] 0 2024-03-29T08:07:35.471 INFO:tasks.workunit.client.0.smithi037.stdout:1/755: symlink d2/d40/de8/l104 0 2024-03-29T08:07:35.474 INFO:tasks.workunit.client.0.smithi037.stdout:4/765: symlink d3/d80/dcd/d8f/l103 0 2024-03-29T08:07:35.480 INFO:tasks.workunit.client.0.smithi037.stdout:1/756: creat d2/d6/dc/d18/d22/d4f/d7c/f105 x:0 0 0 2024-03-29T08:07:35.482 INFO:tasks.workunit.client.0.smithi037.stdout:4/766: creat d3/f104 x:0 0 0 2024-03-29T08:07:35.484 INFO:tasks.workunit.client.0.smithi037.stdout:7/855: link dd/d21/d85/fab dd/d1d/d109/d118/f12b 0 2024-03-29T08:07:35.485 INFO:tasks.workunit.client.0.smithi037.stdout:7/856: stat dd/d21/d35/d60/f7f 0 2024-03-29T08:07:35.490 INFO:tasks.workunit.client.0.smithi037.stdout:7/857: truncate dd/d21/d70/f92 482553 0 2024-03-29T08:07:35.491 INFO:tasks.workunit.client.0.smithi037.stdout:7/858: readlink dd/d21/d35/d60/d94/lba 0 2024-03-29T08:07:35.492 INFO:tasks.workunit.client.0.smithi037.stdout:2/621: write d0/f8e [48218,31467] 0 2024-03-29T08:07:35.501 INFO:tasks.workunit.client.0.smithi037.stdout:7/859: getdents dd/d1d/d49/d81/df0/dfb 0 2024-03-29T08:07:35.504 INFO:tasks.workunit.client.0.smithi037.stdout:8/727: truncate d5/d10/d19/d3d/d4a/d4c/d67/faf 334306 0 2024-03-29T08:07:35.513 INFO:tasks.workunit.client.0.smithi037.stdout:2/622: write d0/d23/f26 [4978401,20721] 0 2024-03-29T08:07:35.517 INFO:tasks.workunit.client.0.smithi037.stdout:7/860: truncate dd/d1d/d109/d118/f12b 226481 0 2024-03-29T08:07:35.518 INFO:tasks.workunit.client.0.smithi037.stdout:0/652: dwrite d9/f27 [0,4194304] 0 2024-03-29T08:07:35.518 INFO:tasks.workunit.client.0.smithi037.stdout:2/623: chown d0/d23/d44/d62/fb8 20 1 2024-03-29T08:07:35.521 INFO:tasks.workunit.client.0.smithi037.stdout:8/728: dread d5/d10/d2d/d34/d50/d83/d46/d4e/f57 [4194304,4194304] 0 2024-03-29T08:07:35.523 INFO:tasks.workunit.client.0.smithi037.stdout:0/653: read - d9/da/dd/d20/d55/d5f/fe0 zero size 2024-03-29T08:07:35.523 INFO:tasks.workunit.client.0.smithi037.stdout:2/624: dread - d0/d23/d44/dca/d2e/d5f/f8d zero size 2024-03-29T08:07:35.526 INFO:tasks.workunit.client.0.smithi037.stdout:7/861: symlink dd/d1d/d49/d81/d84/l12c 0 2024-03-29T08:07:35.529 INFO:tasks.workunit.client.0.smithi037.stdout:0/654: creat d9/da/dd/d54/fe2 x:0 0 0 2024-03-29T08:07:35.532 INFO:tasks.workunit.client.0.smithi037.stdout:2/625: fsync d0/d53/f88 0 2024-03-29T08:07:35.536 INFO:tasks.workunit.client.0.smithi037.stdout:7/862: rename dd/d1d/d34/dc8 to dd/d68/d112/d127/d12d 0 2024-03-29T08:07:35.539 INFO:tasks.workunit.client.0.smithi037.stdout:7/863: chown dd/d21/d85/fea 1721 1 2024-03-29T08:07:35.540 INFO:tasks.workunit.client.0.smithi037.stdout:8/729: creat d5/ff0 x:0 0 0 2024-03-29T08:07:35.544 INFO:tasks.workunit.client.0.smithi037.stdout:2/626: creat d0/d23/d44/dca/d2e/d5f/da5/fd3 x:0 0 0 2024-03-29T08:07:35.553 INFO:tasks.workunit.client.0.smithi037.stdout:2/627: fdatasync d0/d23/d44/dca/d2e/d5f/faa 0 2024-03-29T08:07:35.555 INFO:tasks.workunit.client.0.smithi037.stdout:0/655: dwrite d9/da/d1b/fb0 [0,4194304] 0 2024-03-29T08:07:35.557 INFO:tasks.workunit.client.0.smithi037.stdout:7/864: dread dd/d21/d35/d42/d5c/f9d [0,4194304] 0 2024-03-29T08:07:35.559 INFO:tasks.workunit.client.0.smithi037.stdout:7/865: readlink dd/d21/l41 0 2024-03-29T08:07:35.564 INFO:tasks.workunit.client.0.smithi037.stdout:7/866: write dd/f67 [43772,36968] 0 2024-03-29T08:07:35.566 INFO:tasks.workunit.client.0.smithi037.stdout:2/628: dread d0/f15 [0,4194304] 0 2024-03-29T08:07:35.569 INFO:tasks.workunit.client.0.smithi037.stdout:8/730: dwrite d5/d10/d45/fe9 [0,4194304] 0 2024-03-29T08:07:35.575 INFO:tasks.workunit.client.0.smithi037.stdout:2/629: symlink d0/d23/daf/ld4 0 2024-03-29T08:07:35.575 INFO:tasks.workunit.client.0.smithi037.stdout:7/867: symlink dd/d21/d35/d42/ddb/de2/l12e 0 2024-03-29T08:07:35.586 INFO:tasks.workunit.client.0.smithi037.stdout:7/868: dread dd/d1d/d109/d118/f12b [0,4194304] 0 2024-03-29T08:07:35.589 INFO:tasks.workunit.client.0.smithi037.stdout:8/731: dread d5/d10/d2d/d34/d50/f68 [0,4194304] 0 2024-03-29T08:07:35.589 INFO:tasks.workunit.client.0.smithi037.stdout:7/869: stat dd/d1d/d49/d81/dd8/d125 0 2024-03-29T08:07:35.592 INFO:tasks.workunit.client.0.smithi037.stdout:7/870: write dd/d21/d70/fb2 [530265,18856] 0 2024-03-29T08:07:35.592 INFO:tasks.workunit.client.0.smithi037.stdout:2/630: link d0/d23/d44/dca/d2e/l5d d0/d23/d44/dca/d2e/d5f/ld5 0 2024-03-29T08:07:35.598 INFO:tasks.workunit.client.0.smithi037.stdout:8/732: creat d5/d10/d2d/d34/d50/d83/d46/d8a/ff1 x:0 0 0 2024-03-29T08:07:35.599 INFO:tasks.workunit.client.0.smithi037.stdout:2/631: mknod d0/d23/d44/dca/d7d/cd6 0 2024-03-29T08:07:35.606 INFO:tasks.workunit.client.0.smithi037.stdout:2/632: creat d0/d23/d44/dca/d2e/d5f/d6f/fd7 x:0 0 0 2024-03-29T08:07:35.613 INFO:tasks.workunit.client.0.smithi037.stdout:2/633: unlink d0/d23/d44/dca/d2e/c52 0 2024-03-29T08:07:35.620 INFO:tasks.workunit.client.0.smithi037.stdout:7/871: dwrite dd/d21/d35/fcb [0,4194304] 0 2024-03-29T08:07:35.626 INFO:tasks.workunit.client.0.smithi037.stdout:8/733: dwrite d5/d10/d2d/d9c/fe1 [0,4194304] 0 2024-03-29T08:07:35.637 INFO:tasks.workunit.client.0.smithi037.stdout:8/734: mknod d5/d78/cf2 0 2024-03-29T08:07:35.638 INFO:tasks.workunit.client.0.smithi037.stdout:2/634: rmdir d0/d23/d70/d71/d78/dbe 0 2024-03-29T08:07:35.643 INFO:tasks.workunit.client.0.smithi037.stdout:8/735: creat d5/d10/d19/d3d/d4a/d4c/d67/d85/ff3 x:0 0 0 2024-03-29T08:07:35.644 INFO:tasks.workunit.client.0.smithi037.stdout:2/635: symlink d0/d23/d44/dca/d2e/d5f/ld8 0 2024-03-29T08:07:35.651 INFO:tasks.workunit.client.0.smithi037.stdout:2/636: mknod d0/d23/d44/dca/d2e/d5f/da5/cd9 0 2024-03-29T08:07:35.653 INFO:tasks.workunit.client.0.smithi037.stdout:7/872: write dd/d21/d35/d42/d5c/f82 [782813,83917] 0 2024-03-29T08:07:35.656 INFO:tasks.workunit.client.0.smithi037.stdout:2/637: mkdir d0/d23/d44/dca/d2e/d5f/da5/dda 0 2024-03-29T08:07:35.658 INFO:tasks.workunit.client.0.smithi037.stdout:7/873: creat dd/d21/d35/d60/f12f x:0 0 0 2024-03-29T08:07:35.659 INFO:tasks.workunit.client.0.smithi037.stdout:8/736: write d5/d10/d19/d3d/f7e [403165,19674] 0 2024-03-29T08:07:35.663 INFO:tasks.workunit.client.0.smithi037.stdout:7/874: truncate dd/d68/fb9 800777 0 2024-03-29T08:07:35.665 INFO:tasks.workunit.client.0.smithi037.stdout:8/737: symlink d5/d10/d19/d3d/lf4 0 2024-03-29T08:07:35.668 INFO:tasks.workunit.client.0.smithi037.stdout:8/738: truncate d5/d78/fec 466647 0 2024-03-29T08:07:35.669 INFO:tasks.workunit.client.0.smithi037.stdout:7/875: rename dd/d21/c3a to dd/d21/d35/d3b/d5d/c130 0 2024-03-29T08:07:35.669 INFO:tasks.workunit.client.0.smithi037.stdout:2/638: link d0/d23/d44/dca/d22/c61 d0/d23/d44/dca/d22/d55/cdb 0 2024-03-29T08:07:35.671 INFO:tasks.workunit.client.0.smithi037.stdout:8/739: chown d5/d10/d2d/d34/d50/d83/d46/d8a/cd3 16103 1 2024-03-29T08:07:35.674 INFO:tasks.workunit.client.0.smithi037.stdout:7/876: truncate dd/d68/d112/d127/d12d/fe6 1167632 0 2024-03-29T08:07:35.675 INFO:tasks.workunit.client.0.smithi037.stdout:7/877: chown dd/d21/d35/dcf/cde 17821039 1 2024-03-29T08:07:35.679 INFO:tasks.workunit.client.0.smithi037.stdout:8/740: mkdir d5/d10/d2d/df5 0 2024-03-29T08:07:35.680 INFO:tasks.workunit.client.0.smithi037.stdout:7/878: write dd/d1d/d49/d81/d84/f8a [37214,114627] 0 2024-03-29T08:07:35.688 INFO:tasks.workunit.client.0.smithi037.stdout:7/879: dread dd/d21/d35/d60/f7f [0,4194304] 0 2024-03-29T08:07:35.692 INFO:tasks.workunit.client.0.smithi037.stdout:7/880: write dd/d21/d35/d42/d5c/d8c/f128 [738266,29094] 0 2024-03-29T08:07:35.694 INFO:tasks.workunit.client.0.smithi037.stdout:7/881: stat dd/d68/laa 0 2024-03-29T08:07:35.697 INFO:tasks.workunit.client.0.smithi037.stdout:7/882: mkdir dd/d1d/d109/d131 0 2024-03-29T08:07:35.701 INFO:tasks.workunit.client.0.smithi037.stdout:2/639: truncate d0/d23/f65 450625 0 2024-03-29T08:07:35.705 INFO:tasks.workunit.client.0.smithi037.stdout:7/883: rename dd/d21/d35/d42/ddb/f114 to dd/d68/d112/f132 0 2024-03-29T08:07:35.705 INFO:tasks.workunit.client.0.smithi037.stdout:2/640: fdatasync d0/d23/d44/dca/d22/d55/d92/fcf 0 2024-03-29T08:07:35.717 INFO:tasks.workunit.client.0.smithi037.stdout:2/641: rename d0/d23/d44/d62/fb8 to d0/d23/d44/dca/d7d/fdc 0 2024-03-29T08:07:35.735 INFO:tasks.workunit.client.0.smithi037.stdout:2/642: rename d0/d23/d44/dca/d2e/d5f/c9d to d0/d23/d44/dca/d2e/d38/d68/cdd 0 2024-03-29T08:07:35.735 INFO:tasks.workunit.client.0.smithi037.stdout:7/884: dwrite dd/d21/d35/d42/d5c/f9d [0,4194304] 0 2024-03-29T08:07:35.750 INFO:tasks.workunit.client.0.smithi037.stdout:2/643: creat d0/d23/daf/dbf/fde x:0 0 0 2024-03-29T08:07:35.750 INFO:tasks.workunit.client.0.smithi037.stdout:7/885: creat dd/d1d/d49/d81/dd8/d125/f133 x:0 0 0 2024-03-29T08:07:35.754 INFO:tasks.workunit.client.0.smithi037.stdout:7/886: creat dd/d68/f134 x:0 0 0 2024-03-29T08:07:35.758 INFO:tasks.workunit.client.0.smithi037.stdout:7/887: write dd/d1d/d49/d81/d84/dca/ff6 [1139166,55860] 0 2024-03-29T08:07:35.760 INFO:tasks.workunit.client.0.smithi037.stdout:2/644: getdents d0/d23/d44/dca/d2e/d9c 0 2024-03-29T08:07:35.763 INFO:tasks.workunit.client.0.smithi037.stdout:7/888: chown dd/d21/d65/da2/l115 643 1 2024-03-29T08:07:35.765 INFO:tasks.workunit.client.0.smithi037.stdout:7/889: stat dd/d21/d35/d42/ddb/de2 0 2024-03-29T08:07:35.767 INFO:tasks.workunit.client.0.smithi037.stdout:9/916: sync 2024-03-29T08:07:35.769 INFO:tasks.workunit.client.0.smithi037.stdout:6/701: sync 2024-03-29T08:07:35.771 INFO:tasks.workunit.client.0.smithi037.stdout:6/702: chown d0/d6a/d7b/d77/c86 1 1 2024-03-29T08:07:35.772 INFO:tasks.workunit.client.0.smithi037.stdout:7/890: rename dd/d1d/d34/f3f to dd/d21/d65/f135 0 2024-03-29T08:07:35.779 INFO:tasks.workunit.client.0.smithi037.stdout:2/645: truncate d0/d53/f7a 118418 0 2024-03-29T08:07:35.786 INFO:tasks.workunit.client.0.smithi037.stdout:2/646: mknod d0/d23/d44/dca/d2e/d5f/d6f/d5e/dc2/cdf 0 2024-03-29T08:07:35.791 INFO:tasks.workunit.client.0.smithi037.stdout:2/647: fsync d0/d23/d44/dca/d2e/da4/fb4 0 2024-03-29T08:07:35.792 INFO:tasks.workunit.client.0.smithi037.stdout:9/917: getdents d1/d4/d19/d1e/d4a 0 2024-03-29T08:07:35.792 INFO:tasks.workunit.client.0.smithi037.stdout:7/891: write dd/d68/fb9 [970980,16674] 0 2024-03-29T08:07:35.792 INFO:tasks.workunit.client.0.smithi037.stdout:6/703: dwrite d0/d5/d1f/da8/fdf [0,4194304] 0 2024-03-29T08:07:35.806 INFO:tasks.workunit.client.0.smithi037.stdout:6/704: truncate d0/d5/d1e/d44/d71/f75 710320 0 2024-03-29T08:07:35.809 INFO:tasks.workunit.client.0.smithi037.stdout:7/892: dread dd/d21/d35/fc5 [0,4194304] 0 2024-03-29T08:07:35.813 INFO:tasks.workunit.client.0.smithi037.stdout:9/918: getdents d1/d4/d19/d1e/d4a/d6f/da2/d31/daa/dba/dbb 0 2024-03-29T08:07:35.815 INFO:tasks.workunit.client.0.smithi037.stdout:7/893: mkdir dd/d21/d35/d3b/d136 0 2024-03-29T08:07:35.821 INFO:tasks.workunit.client.0.smithi037.stdout:9/919: chown d1/d4/d19/d1e/d4a/d6f/da2/d23/f39 1 1 2024-03-29T08:07:35.823 INFO:tasks.workunit.client.0.smithi037.stdout:2/648: dwrite d0/d23/d44/dca/d2e/d5f/faa [0,4194304] 0 2024-03-29T08:07:35.835 INFO:tasks.workunit.client.0.smithi037.stdout:2/649: rename d0/d23/d44/f45 to d0/d23/d44/dca/d2e/da4/fe0 0 2024-03-29T08:07:35.836 INFO:tasks.workunit.client.0.smithi037.stdout:6/705: dwrite d0/d5/d1f/f63 [0,4194304] 0 2024-03-29T08:07:35.840 INFO:tasks.workunit.client.0.smithi037.stdout:2/650: write d0/d23/d44/dca/f3a [213420,119731] 0 2024-03-29T08:07:35.841 INFO:tasks.workunit.client.0.smithi037.stdout:9/920: mkdir d1/d4/d19/d1e/d4a/d6f/da2/d13/de0/d125/d138 0 2024-03-29T08:07:35.846 INFO:tasks.workunit.client.0.smithi037.stdout:2/651: creat d0/d23/d44/dca/d2e/d5f/fe1 x:0 0 0 2024-03-29T08:07:35.846 INFO:tasks.workunit.client.0.smithi037.stdout:7/894: dwrite dd/d21/d35/d3b/fb8 [0,4194304] 0 2024-03-29T08:07:35.851 INFO:tasks.workunit.client.0.smithi037.stdout:9/921: chown d1/d4/d19/d1e/d4a/d6f/da2/d13/c5e 1 1 2024-03-29T08:07:35.855 INFO:tasks.workunit.client.0.smithi037.stdout:9/922: write d1/d4/d19/d69/fce [2137635,52477] 0 2024-03-29T08:07:35.855 INFO:tasks.workunit.client.0.smithi037.stdout:2/652: mknod d0/d23/d44/dca/d2e/d38/ce2 0 2024-03-29T08:07:35.862 INFO:tasks.workunit.client.0.smithi037.stdout:2/653: creat d0/d23/d44/dca/d2e/da4/fe3 x:0 0 0 2024-03-29T08:07:35.867 INFO:tasks.workunit.client.0.smithi037.stdout:2/654: read - d0/d23/d44/dca/d2e/d38/f93 zero size 2024-03-29T08:07:35.871 INFO:tasks.workunit.client.0.smithi037.stdout:9/923: truncate d1/d4/d19/f64 4646712 0 2024-03-29T08:07:35.873 INFO:tasks.workunit.client.0.smithi037.stdout:2/655: mknod d0/d23/daf/ce4 0 2024-03-29T08:07:35.876 INFO:tasks.workunit.client.0.smithi037.stdout:9/924: symlink d1/d4/d19/d69/df3/l139 0 2024-03-29T08:07:35.878 INFO:tasks.workunit.client.0.smithi037.stdout:9/925: read - d1/d4/d19/d1e/d71/ffa zero size 2024-03-29T08:07:35.878 INFO:tasks.workunit.client.0.smithi037.stdout:2/656: mknod d0/d23/d44/dca/d22/d4e/ce5 0 2024-03-29T08:07:35.882 INFO:tasks.workunit.client.0.smithi037.stdout:2/657: dread d0/f15 [0,4194304] 0 2024-03-29T08:07:35.883 INFO:tasks.workunit.client.0.smithi037.stdout:7/895: dread dd/d1d/d34/f66 [4194304,4194304] 0 2024-03-29T08:07:35.888 INFO:tasks.workunit.client.0.smithi037.stdout:7/896: write dd/d21/d35/d60/d94/f10d [150193,27805] 0 2024-03-29T08:07:35.889 INFO:tasks.workunit.client.0.smithi037.stdout:7/897: fdatasync dd/d21/d85/f10c 0 2024-03-29T08:07:35.893 INFO:tasks.workunit.client.0.smithi037.stdout:7/898: truncate dd/d72/fae 9832203 0 2024-03-29T08:07:35.905 INFO:tasks.workunit.client.0.smithi037.stdout:9/926: dread d1/d3/f21 [0,4194304] 0 2024-03-29T08:07:35.906 INFO:tasks.workunit.client.0.smithi037.stdout:2/658: dwrite d0/d23/d44/dca/d2e/d5f/d6f/fb7 [0,4194304] 0 2024-03-29T08:07:35.914 INFO:tasks.workunit.client.0.smithi037.stdout:2/659: symlink d0/d23/d44/dca/d2e/da4/le6 0 2024-03-29T08:07:35.915 INFO:tasks.workunit.client.0.smithi037.stdout:7/899: write dd/f48 [1519960,35861] 0 2024-03-29T08:07:35.920 INFO:tasks.workunit.client.0.smithi037.stdout:7/900: creat dd/d68/d112/d127/d12d/f137 x:0 0 0 2024-03-29T08:07:35.921 INFO:tasks.workunit.client.0.smithi037.stdout:7/901: write dd/d21/fec [752172,60970] 0 2024-03-29T08:07:35.923 INFO:tasks.workunit.client.0.smithi037.stdout:7/902: chown dd/d21/fec 78774284 1 2024-03-29T08:07:35.926 INFO:tasks.workunit.client.0.smithi037.stdout:7/903: mknod dd/d21/d35/d60/c138 0 2024-03-29T08:07:35.930 INFO:tasks.workunit.client.0.smithi037.stdout:9/927: dwrite d1/d4/d19/d1e/d4a/d6f/da2/d23/f9a [0,4194304] 0 2024-03-29T08:07:35.941 INFO:tasks.workunit.client.0.smithi037.stdout:9/928: read d1/d4/d19/d1e/d4a/d6f/da2/d31/d87/d88/f11a [6242318,89346] 0 2024-03-29T08:07:35.947 INFO:tasks.workunit.client.0.smithi037.stdout:9/929: creat d1/d4/d19/d1e/d4a/d6f/da2/d13/de0/d125/d138/f13a x:0 0 0 2024-03-29T08:07:35.955 INFO:tasks.workunit.client.0.smithi037.stdout:9/930: rename d1/d4/d19/d1e/d71/d89/da6 to d1/d4/d19/d1e/d4a/d6f/da2/d13/d7c/dd3/d13b 0 2024-03-29T08:07:35.964 INFO:tasks.workunit.client.0.smithi037.stdout:9/931: dread d1/d4/d19/d1e/d4a/d6f/da2/d31/f48 [0,4194304] 0 2024-03-29T08:07:35.967 INFO:tasks.workunit.client.0.smithi037.stdout:9/932: symlink d1/d1f/d3a/d67/l13c 0 2024-03-29T08:07:35.968 INFO:tasks.workunit.client.0.smithi037.stdout:9/933: chown d1/d4/d19/d1e/d4a/d5c/dca 30933 1 2024-03-29T08:07:35.974 INFO:tasks.workunit.client.0.smithi037.stdout:9/934: mknod d1/d1f/c13d 0 2024-03-29T08:07:36.009 INFO:tasks.workunit.client.0.smithi037.stdout:9/935: truncate d1/d4/d19/d1e/d71/f94 1668073 0 2024-03-29T08:07:36.009 INFO:tasks.workunit.client.0.smithi037.stdout:9/936: link d1/d4/d19/d1e/d4a/d6f/da2/d13/d7c/fb4 d1/d1f/df6/f13e 0 2024-03-29T08:07:36.009 INFO:tasks.workunit.client.0.smithi037.stdout:9/937: creat d1/d3/d44/f13f x:0 0 0 2024-03-29T08:07:36.090 INFO:tasks.workunit.client.0.smithi037.stdout:9/938: creat d1/d1f/d3a/d9b/f140 x:0 0 0 2024-03-29T08:07:36.093 INFO:tasks.workunit.client.0.smithi037.stdout:9/939: chown d1/d4/d19/c6e 51199 1 2024-03-29T08:07:36.098 INFO:tasks.workunit.client.0.smithi037.stdout:9/940: mkdir d1/d4/d95/d141 0 2024-03-29T08:07:36.105 INFO:tasks.workunit.client.0.smithi037.stdout:9/941: read d1/d4/d19/d1e/d4a/d6f/da2/d13/f17 [523068,67706] 0 2024-03-29T08:07:36.106 INFO:tasks.workunit.client.0.smithi037.stdout:9/942: readlink d1/d4/d19/d1e/d4a/d6f/da2/d68/lab 0 2024-03-29T08:07:36.110 INFO:tasks.workunit.client.0.smithi037.stdout:9/943: read d1/d1f/f33 [41314,76119] 0 2024-03-29T08:07:36.115 INFO:tasks.workunit.client.0.smithi037.stdout:9/944: creat d1/d4/d19/d1e/d4a/d6f/da2/d23/f142 x:0 0 0 2024-03-29T08:07:36.119 INFO:tasks.workunit.client.0.smithi037.stdout:9/945: symlink d1/d4/d19/d1e/d4a/d6f/da2/d13/d7c/dd3/d13b/l143 0 2024-03-29T08:07:36.142 INFO:tasks.workunit.client.0.smithi037.stdout:9/946: dwrite d1/d1f/f126 [0,4194304] 0 2024-03-29T08:07:36.162 INFO:tasks.workunit.client.0.smithi037.stdout:9/947: dread d1/d4/d19/d69/fe6 [0,4194304] 0 2024-03-29T08:07:36.169 INFO:tasks.workunit.client.0.smithi037.stdout:9/948: dread d1/d4/d19/d1e/d4a/d6f/da2/d31/f48 [0,4194304] 0 2024-03-29T08:07:36.171 INFO:tasks.workunit.client.0.smithi037.stdout:9/949: chown d1/d4/d19/d69/fce 138870 1 2024-03-29T08:07:36.175 INFO:tasks.workunit.client.0.smithi037.stdout:9/950: creat d1/d4/d19/d1e/d4a/d6f/da2/d31/daa/dba/dbb/f144 x:0 0 0 2024-03-29T08:07:36.199 INFO:tasks.workunit.client.0.smithi037.stdout:9/951: dread d1/d4/d19/d1e/d4a/d6f/da2/d31/f38 [0,4194304] 0 2024-03-29T08:07:36.202 INFO:tasks.workunit.client.0.smithi037.stdout:5/895: sync 2024-03-29T08:07:36.203 INFO:tasks.workunit.client.0.smithi037.stdout:9/952: mkdir d1/d1f/dfd/d127/d145 0 2024-03-29T08:07:36.206 INFO:tasks.workunit.client.0.smithi037.stdout:9/953: write d1/d1f/d3a/d9b/f140 [726130,126342] 0 2024-03-29T08:07:36.206 INFO:tasks.workunit.client.0.smithi037.stdout:5/896: creat d8/d22/d4f/d70/d9c/f132 x:0 0 0 2024-03-29T08:07:36.215 INFO:tasks.workunit.client.0.smithi037.stdout:5/897: dread d8/d22/d4f/d70/f54 [0,4194304] 0 2024-03-29T08:07:36.222 INFO:tasks.workunit.client.0.smithi037.stdout:5/898: mkdir d8/d22/d37/d8d/db3/dcb/deb/d88/d133 0 2024-03-29T08:07:36.223 INFO:tasks.workunit.client.0.smithi037.stdout:9/954: dwrite d1/d1f/d3a/f12a [0,4194304] 0 2024-03-29T08:07:36.224 INFO:tasks.workunit.client.0.smithi037.stdout:5/899: chown d8/d22/d37/d8d/db3/dcb/deb/dfa 19 1 2024-03-29T08:07:36.234 INFO:tasks.workunit.client.0.smithi037.stdout:9/955: rename d1/d4/d19/d1e/d71/d116/f120 to d1/d4/d19/d1e/d4a/d6f/da2/d13/d7c/f146 0 2024-03-29T08:07:36.242 INFO:tasks.workunit.client.0.smithi037.stdout:9/956: unlink d1/d4/d19/d69/fda 0 2024-03-29T08:07:36.252 INFO:tasks.workunit.client.0.smithi037.stdout:5/900: dwrite d8/d22/d37/d8d/db3/dcb/deb/fa1 [0,4194304] 0 2024-03-29T08:07:36.263 INFO:tasks.workunit.client.0.smithi037.stdout:9/957: dwrite d1/d4/d19/d1e/d4a/d6f/da2/d31/daa/fef [4194304,4194304] 0 2024-03-29T08:07:36.265 INFO:tasks.workunit.client.0.smithi037.stdout:5/901: link d8/l1b d8/d22/d4f/l134 0 2024-03-29T08:07:36.287 INFO:tasks.workunit.client.0.smithi037.stdout:9/958: write d1/d4/d19/d1e/d4a/d6f/da2/d31/d87/d88/f11a [5761412,48253] 0 2024-03-29T08:07:36.304 INFO:tasks.workunit.client.0.smithi037.stdout:5/902: dread d8/f52 [0,4194304] 0 2024-03-29T08:07:36.305 INFO:tasks.workunit.client.0.smithi037.stdout:9/959: dwrite d1/d4/d19/d1e/d4a/d6f/da2/d13/d7c/fc3 [0,4194304] 0 2024-03-29T08:07:36.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:07:36 smithi148 ceph-mon[44992]: Upgrade: Updating mgr.smithi037.wjikii 2024-03-29T08:07:36.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:07:36 smithi148 ceph-mon[44992]: from='mgr.14602 ' entity='mgr.smithi148.onhgvm' 2024-03-29T08:07:36.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:07:36 smithi148 ceph-mon[44992]: from='mgr.14602 172.21.15.148:0/667342922' entity='mgr.smithi148.onhgvm' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi037.wjikii", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-29T08:07:36.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:07:36 smithi148 ceph-mon[44992]: from='mgr.14602 ' entity='mgr.smithi148.onhgvm' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi037.wjikii", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-29T08:07:36.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:07:36 smithi148 ceph-mon[44992]: from='mgr.14602 172.21.15.148:0/667342922' entity='mgr.smithi148.onhgvm' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-29T08:07:36.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:07:36 smithi148 ceph-mon[44992]: from='mgr.14602 172.21.15.148:0/667342922' entity='mgr.smithi148.onhgvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T08:07:36.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:07:36 smithi148 ceph-mon[44992]: Deploying daemon mgr.smithi037.wjikii on smithi037 2024-03-29T08:07:36.306 INFO:tasks.workunit.client.0.smithi037.stdout:5/903: readlink d8/d22/d37/d8d/db3/dcb/ld3 0 2024-03-29T08:07:36.310 INFO:tasks.workunit.client.0.smithi037.stdout:5/904: symlink d8/d22/d37/de8/l135 0 2024-03-29T08:07:36.313 INFO:tasks.workunit.client.0.smithi037.stdout:9/960: truncate d1/d4/d19/d1e/d4a/d5c/f5d 654204 0 2024-03-29T08:07:36.314 INFO:tasks.workunit.client.0.smithi037.stdout:5/905: truncate d8/d22/d37/dbc/f111 443868 0 2024-03-29T08:07:36.317 INFO:tasks.workunit.client.0.smithi037.stdout:9/961: mknod d1/d1f/d3a/c147 0 2024-03-29T08:07:36.327 INFO:tasks.workunit.client.0.smithi037.stdout:5/906: write d8/d22/d4f/d70/fbe [967293,82279] 0 2024-03-29T08:07:36.331 INFO:tasks.workunit.client.0.smithi037.stdout:9/962: getdents d1/d4/d19/d1e/d4a/d6f 0 2024-03-29T08:07:36.335 INFO:tasks.workunit.client.0.smithi037.stdout:9/963: write d1/d4/d19/d69/ff4 [736964,102622] 0 2024-03-29T08:07:36.340 INFO:tasks.workunit.client.0.smithi037.stdout:9/964: dread - d1/d4/d19/d1e/d4a/d6f/da2/d13/de0/d125/d138/f13a zero size 2024-03-29T08:07:36.341 INFO:tasks.workunit.client.0.smithi037.stdout:5/907: dread d8/d22/d4f/d70/d110/fa8 [0,4194304] 0 2024-03-29T08:07:36.342 INFO:tasks.workunit.client.0.smithi037.stdout:9/965: truncate d1/d3/f21 4363126 0 2024-03-29T08:07:36.343 INFO:tasks.workunit.client.0.smithi037.stdout:5/908: truncate d8/f1f 811844 0 2024-03-29T08:07:36.344 INFO:tasks.workunit.client.0.smithi037.stdout:5/909: chown d8/d22/d4f/d70/f3f 1935908 1 2024-03-29T08:07:36.353 INFO:tasks.workunit.client.0.smithi037.stdout:3/677: sync 2024-03-29T08:07:36.359 INFO:tasks.workunit.client.0.smithi037.stdout:3/678: creat dd/d46/d2d/d3a/d6c/db1/dc0/dda/d33/d97/dd6/fe1 x:0 0 0 2024-03-29T08:07:36.363 INFO:tasks.workunit.client.0.smithi037.stdout:3/679: mknod dd/d15/ce2 0 2024-03-29T08:07:36.367 INFO:tasks.workunit.client.0.smithi037.stdout:9/966: dwrite d1/d1f/f5f [0,4194304] 0 2024-03-29T08:07:36.376 INFO:tasks.workunit.client.0.smithi037.stdout:5/910: dread d8/db9/f11a [0,4194304] 0 2024-03-29T08:07:36.379 INFO:tasks.workunit.client.0.smithi037.stdout:9/967: mknod d1/d4/d19/d1e/d4a/d6f/da2/d13/d7c/c148 0 2024-03-29T08:07:36.386 INFO:tasks.workunit.client.0.smithi037.stdout:5/911: symlink d8/d22/d37/d8d/db3/dcb/deb/dfa/l136 0 2024-03-29T08:07:36.389 INFO:tasks.workunit.client.0.smithi037.stdout:5/912: chown d8/l1b 0 1 2024-03-29T08:07:36.390 INFO:tasks.workunit.client.0.smithi037.stdout:9/968: rename d1/d4/d19/d1e/d4a/d6f/f91 to d1/d4/d19/d1e/d4a/d6f/da2/d13/de0/d125/f149 0 2024-03-29T08:07:36.393 INFO:tasks.workunit.client.0.smithi037.stdout:3/680: dwrite dd/d46/d2d/d3a/d6c/db1/dc0/dda/d33/d84/fd0 [0,4194304] 0 2024-03-29T08:07:36.395 INFO:tasks.workunit.client.0.smithi037.stdout:3/681: chown dd/d46/d3e/fb7 1886 1 2024-03-29T08:07:36.397 INFO:tasks.workunit.client.0.smithi037.stdout:9/969: creat d1/d1f/f14a x:0 0 0 2024-03-29T08:07:36.398 INFO:tasks.workunit.client.0.smithi037.stdout:5/913: truncate d8/d18/f50 1075083 0 2024-03-29T08:07:36.400 INFO:tasks.workunit.client.0.smithi037.stdout:5/914: fsync d8/f1f 0 2024-03-29T08:07:36.400 INFO:tasks.workunit.client.0.smithi037.stdout:3/682: creat dd/d46/d3e/fe3 x:0 0 0 2024-03-29T08:07:36.401 INFO:tasks.workunit.client.0.smithi037.stdout:5/915: chown d8/db9 1065555574 1 2024-03-29T08:07:36.403 INFO:tasks.workunit.client.0.smithi037.stdout:9/970: fsync d1/d1f/d3a/f46 0 2024-03-29T08:07:36.403 INFO:tasks.workunit.client.0.smithi037.stdout:0/656: sync 2024-03-29T08:07:36.404 INFO:tasks.workunit.client.0.smithi037.stdout:1/757: sync 2024-03-29T08:07:36.404 INFO:tasks.workunit.client.0.smithi037.stdout:7/904: sync 2024-03-29T08:07:36.404 INFO:tasks.workunit.client.0.smithi037.stdout:8/741: sync 2024-03-29T08:07:36.405 INFO:tasks.workunit.client.0.smithi037.stdout:5/916: chown d8/d34/f80 20947 1 2024-03-29T08:07:36.405 INFO:tasks.workunit.client.0.smithi037.stdout:4/767: sync 2024-03-29T08:07:36.405 INFO:tasks.workunit.client.0.smithi037.stdout:2/660: sync 2024-03-29T08:07:36.410 INFO:tasks.workunit.client.0.smithi037.stdout:6/706: sync 2024-03-29T08:07:36.422 INFO:tasks.workunit.client.0.smithi037.stdout:3/683: truncate dd/d46/d2d/d3a/f81 20888 0 2024-03-29T08:07:36.426 INFO:tasks.workunit.client.0.smithi037.stdout:0/657: write d9/da/dd/d20/d38/d53/f62 [2211454,77999] 0 2024-03-29T08:07:36.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:07:36 smithi037 ceph-mon[35162]: Upgrade: Updating mgr.smithi037.wjikii 2024-03-29T08:07:36.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:07:36 smithi037 ceph-mon[35162]: from='mgr.14602 ' entity='mgr.smithi148.onhgvm' 2024-03-29T08:07:36.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:07:36 smithi037 ceph-mon[35162]: from='mgr.14602 172.21.15.148:0/667342922' entity='mgr.smithi148.onhgvm' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi037.wjikii", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-29T08:07:36.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:07:36 smithi037 ceph-mon[35162]: from='mgr.14602 ' entity='mgr.smithi148.onhgvm' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi037.wjikii", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-29T08:07:36.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:07:36 smithi037 ceph-mon[35162]: from='mgr.14602 172.21.15.148:0/667342922' entity='mgr.smithi148.onhgvm' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-29T08:07:36.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:07:36 smithi037 ceph-mon[35162]: from='mgr.14602 172.21.15.148:0/667342922' entity='mgr.smithi148.onhgvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T08:07:36.428 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:07:36 smithi037 ceph-mon[35162]: Deploying daemon mgr.smithi037.wjikii on smithi037 2024-03-29T08:07:36.428 INFO:tasks.workunit.client.0.smithi037.stdout:3/684: chown dd/d15/d18/d30/f9d 248674814 1 2024-03-29T08:07:36.428 INFO:tasks.workunit.client.0.smithi037.stdout:3/685: chown dd/d46/d2d 16848294 1 2024-03-29T08:07:36.434 INFO:tasks.workunit.client.0.smithi037.stdout:5/917: dread d8/d22/d37/dbc/d85/df6/f104 [0,4194304] 0 2024-03-29T08:07:36.437 INFO:tasks.workunit.client.0.smithi037.stdout:5/918: write d8/d22/d4f/d70/d79/f7b [4864495,55903] 0 2024-03-29T08:07:36.465 INFO:tasks.workunit.client.0.smithi037.stdout:4/768: creat d3/d57/dec/ded/d1d/db/d45/d1e/d59/d60/d6f/db9/f105 x:0 0 0 2024-03-29T08:07:36.470 INFO:tasks.workunit.client.0.smithi037.stdout:8/742: rename d5/d10/d2d/d9c/da1/cb1 to d5/d10/d45/d4b/cf6 0 2024-03-29T08:07:36.476 INFO:tasks.workunit.client.0.smithi037.stdout:0/658: creat d9/da/dd/d20/d55/fe3 x:0 0 0 2024-03-29T08:07:36.477 INFO:tasks.workunit.client.0.smithi037.stdout:0/659: readlink d9/da/dd/d20/l68 0 2024-03-29T08:07:36.480 INFO:tasks.workunit.client.0.smithi037.stdout:9/971: dread - d1/d4/d19/d1e/d4a/d6f/fcd zero size 2024-03-29T08:07:36.483 INFO:tasks.workunit.client.0.smithi037.stdout:7/905: mkdir dd/d1d/d34/d139 0 2024-03-29T08:07:36.484 INFO:tasks.workunit.client.0.smithi037.stdout:7/906: chown dd/f1b 5293 1 2024-03-29T08:07:36.486 INFO:tasks.workunit.client.0.smithi037.stdout:4/769: truncate d3/d4/d27/f7f 7358044 0 2024-03-29T08:07:36.488 INFO:tasks.workunit.client.0.smithi037.stdout:8/743: unlink d5/d10/d2d/d34/d50/d83/d46/d8a/fc1 0 2024-03-29T08:07:36.501 INFO:tasks.workunit.client.0.smithi037.stdout:0/660: truncate d9/da/dd/d13/f31 2948074 0 2024-03-29T08:07:36.503 INFO:tasks.workunit.client.0.smithi037.stdout:9/972: mkdir d1/d4/d14b 0 2024-03-29T08:07:36.507 INFO:tasks.workunit.client.0.smithi037.stdout:4/770: mknod d3/d57/dec/ded/d1d/db/d45/d1e/d102/c106 0 2024-03-29T08:07:36.511 INFO:tasks.workunit.client.0.smithi037.stdout:7/907: rename dd/d21/d65/da2/df8/f5f to dd/d21/f13a 0 2024-03-29T08:07:36.511 INFO:tasks.workunit.client.0.smithi037.stdout:8/744: dread d5/d10/d19/d3d/d4a/d4c/f65 [0,4194304] 0 2024-03-29T08:07:36.516 INFO:tasks.workunit.client.0.smithi037.stdout:4/771: rmdir d3/d57/dec/ded/d1d/db/d45/d1e/d59/d60 39 2024-03-29T08:07:36.524 INFO:tasks.workunit.client.0.smithi037.stdout:5/919: write d8/d22/d4f/d70/d79/fd7 [537140,28921] 0 2024-03-29T08:07:36.525 INFO:tasks.workunit.client.0.smithi037.stdout:1/758: dwrite d2/d6/dc/d58/f71 [0,4194304] 0 2024-03-29T08:07:36.531 INFO:tasks.workunit.client.0.smithi037.stdout:1/759: write d2/d6/dc/d18/d22/d4f/d7c/f7f [1737542,129069] 0 2024-03-29T08:07:36.534 INFO:tasks.workunit.client.0.smithi037.stdout:8/745: symlink d5/d10/d2d/d34/d50/d83/d46/lf7 0 2024-03-29T08:07:36.537 INFO:tasks.workunit.client.0.smithi037.stdout:5/920: mkdir d8/d4d/d5c/d131/d137 0 2024-03-29T08:07:36.542 INFO:tasks.workunit.client.0.smithi037.stdout:6/707: dwrite d0/d5/d1e/f5f [0,4194304] 0 2024-03-29T08:07:36.543 INFO:tasks.workunit.client.0.smithi037.stdout:9/973: rename d1/d4/d19/d1e/d25 to d1/d4/d19/d1e/d4a/d6f/d14c 0 2024-03-29T08:07:36.552 INFO:tasks.workunit.client.0.smithi037.stdout:5/921: symlink d8/d4d/ddf/l138 0 2024-03-29T08:07:36.552 INFO:tasks.workunit.client.0.smithi037.stdout:0/661: truncate d9/da/dd/d13/f95 1992908 0 2024-03-29T08:07:36.556 INFO:tasks.workunit.client.0.smithi037.stdout:0/662: chown d9/da/dd/d54/d92/dd4 34 1 2024-03-29T08:07:36.557 INFO:tasks.workunit.client.0.smithi037.stdout:7/908: rename dd/d21/d35/d60/f6e to dd/d68/d112/d127/f13b 0 2024-03-29T08:07:36.561 INFO:tasks.workunit.client.0.smithi037.stdout:9/974: chown d1/d4/d19/c79 1328 1 2024-03-29T08:07:36.562 INFO:tasks.workunit.client.0.smithi037.stdout:4/772: write d3/d54/f5b [2518354,27151] 0 2024-03-29T08:07:36.562 INFO:tasks.workunit.client.0.smithi037.stdout:6/708: dwrite d0/d6a/d7b/fa1 [0,4194304] 0 2024-03-29T08:07:36.572 INFO:tasks.workunit.client.0.smithi037.stdout:8/746: rename d5/d10/d19/d3d/d4a to d5/d10/d2d/d9c/df8 0 2024-03-29T08:07:36.580 INFO:tasks.workunit.client.0.smithi037.stdout:9/975: mkdir d1/d1f/dfd/d14d 0 2024-03-29T08:07:36.581 INFO:tasks.workunit.client.0.smithi037.stdout:4/773: mknod d3/d57/d85/c107 0 2024-03-29T08:07:36.588 INFO:tasks.workunit.client.0.smithi037.stdout:4/774: dread d3/d57/dec/ded/d1d/db/d72/d98/fbf [0,4194304] 0 2024-03-29T08:07:36.589 INFO:tasks.workunit.client.0.smithi037.stdout:6/709: truncate d0/d19/d42/d60/d62/f9a 1480723 0 2024-03-29T08:07:36.590 INFO:tasks.workunit.client.0.smithi037.stdout:0/663: symlink d9/da/d1b/le4 0 2024-03-29T08:07:36.594 INFO:tasks.workunit.client.0.smithi037.stdout:0/664: chown d9/da/d1b/c37 83521067 1 2024-03-29T08:07:36.596 INFO:tasks.workunit.client.0.smithi037.stdout:0/665: chown d9/d74/d7d/d7e 203788022 1 2024-03-29T08:07:36.596 INFO:tasks.workunit.client.0.smithi037.stdout:7/909: fsync dd/d21/d35/d42/d5c/f5e 0 2024-03-29T08:07:36.598 INFO:tasks.workunit.client.0.smithi037.stdout:1/760: dwrite d2/d6/dc/d5d/ffe [0,4194304] 0 2024-03-29T08:07:36.598 INFO:tasks.workunit.client.0.smithi037.stdout:9/976: symlink d1/d1f/dfd/d127/l14e 0 2024-03-29T08:07:36.601 INFO:tasks.workunit.client.0.smithi037.stdout:7/910: chown dd/d21/d35/d42/d5c/la0 2047997 1 2024-03-29T08:07:36.603 INFO:tasks.workunit.client.0.smithi037.stdout:1/761: readlink d2/d6/dc/d18/dbd/dda/le0 0 2024-03-29T08:07:36.605 INFO:tasks.workunit.client.0.smithi037.stdout:5/922: truncate d8/f17 220246 0 2024-03-29T08:07:36.609 INFO:tasks.workunit.client.0.smithi037.stdout:8/747: fsync d5/d10/d19/d8d/f90 0 2024-03-29T08:07:36.614 INFO:tasks.workunit.client.0.smithi037.stdout:0/666: dread d9/da/dd/d13/f6f [0,4194304] 0 2024-03-29T08:07:36.616 INFO:tasks.workunit.client.0.smithi037.stdout:6/710: mkdir d0/d19/df8 0 2024-03-29T08:07:36.620 INFO:tasks.workunit.client.0.smithi037.stdout:7/911: rename dd/d72/l83 to dd/d68/d112/d127/d12d/l13c 0 2024-03-29T08:07:36.624 INFO:tasks.workunit.client.0.smithi037.stdout:5/923: symlink d8/d22/d37/d8d/db3/dcb/l139 0 2024-03-29T08:07:36.628 INFO:tasks.workunit.client.0.smithi037.stdout:1/762: truncate d2/d6/dc/d18/f5f 918256 0 2024-03-29T08:07:36.633 INFO:tasks.workunit.client.0.smithi037.stdout:6/711: rmdir d0/d19/d34/d4e/df2 39 2024-03-29T08:07:36.634 INFO:tasks.workunit.client.0.smithi037.stdout:4/775: dwrite d3/d57/dec/ded/d1d/db/d45/d1e/d59/dc1/fd3 [0,4194304] 0 2024-03-29T08:07:36.636 INFO:tasks.workunit.client.0.smithi037.stdout:4/776: chown d3/d4/f46 41 1 2024-03-29T08:07:36.639 INFO:tasks.workunit.client.0.smithi037.stdout:0/667: rename d9/d74/d7d/fdf to d9/da/dd/d54/db5/dcb/fe5 0 2024-03-29T08:07:36.642 INFO:tasks.workunit.client.0.smithi037.stdout:5/924: mkdir d8/d34/d13a 0 2024-03-29T08:07:36.644 INFO:tasks.workunit.client.0.smithi037.stdout:1/763: rmdir d2/d6/dc/d5d/d62 39 2024-03-29T08:07:36.650 INFO:tasks.workunit.client.0.smithi037.stdout:6/712: fsync d0/d5/d1e/fc2 0 2024-03-29T08:07:36.656 INFO:tasks.workunit.client.0.smithi037.stdout:4/777: creat d3/d57/dec/ded/d1d/db/d45/d1e/f108 x:0 0 0 2024-03-29T08:07:36.658 INFO:tasks.workunit.client.0.smithi037.stdout:0/668: mkdir d9/d74/de6 0 2024-03-29T08:07:36.663 INFO:tasks.workunit.client.0.smithi037.stdout:5/925: fsync d8/d18/f71 0 2024-03-29T08:07:36.668 INFO:tasks.workunit.client.0.smithi037.stdout:9/977: getdents d1/d4/d19/d1e/d71/d89 0 2024-03-29T08:07:36.673 INFO:tasks.workunit.client.0.smithi037.stdout:6/713: mknod d0/d5/d3f/cf9 0 2024-03-29T08:07:36.674 INFO:tasks.workunit.client.0.smithi037.stdout:7/912: rename dd/d68/d112/d127/ld4 to dd/d21/d35/d42/d5c/l13d 0 2024-03-29T08:07:36.676 INFO:tasks.workunit.client.0.smithi037.stdout:1/764: write d2/d6/dc/d18/f63 [4267885,57198] 0 2024-03-29T08:07:36.676 INFO:tasks.workunit.client.0.smithi037.stdout:7/913: truncate dd/d68/fb9 1071809 0 2024-03-29T08:07:36.677 INFO:tasks.workunit.client.0.smithi037.stdout:4/778: creat d3/f109 x:0 0 0 2024-03-29T08:07:36.678 INFO:tasks.workunit.client.0.smithi037.stdout:0/669: creat d9/d74/fe7 x:0 0 0 2024-03-29T08:07:36.679 INFO:tasks.workunit.client.0.smithi037.stdout:8/748: link d5/f17 d5/d10/d2d/d34/d50/ddc/ff9 0 2024-03-29T08:07:36.687 INFO:tasks.workunit.client.0.smithi037.stdout:6/714: creat d0/d19/d34/d4e/d69/dee/ffa x:0 0 0 2024-03-29T08:07:36.687 INFO:tasks.workunit.client.0.smithi037.stdout:5/926: write d8/d22/d37/d8d/db3/dcb/deb/f76 [1490582,123306] 0 2024-03-29T08:07:36.689 INFO:tasks.workunit.client.0.smithi037.stdout:7/914: chown dd/d21/d89/lc4 193092400 1 2024-03-29T08:07:36.691 INFO:tasks.workunit.client.0.smithi037.stdout:4/779: symlink d3/d54/d88/l10a 0 2024-03-29T08:07:36.692 INFO:tasks.workunit.client.0.smithi037.stdout:1/765: creat d2/d6/dc/d18/dbd/dda/f106 x:0 0 0 2024-03-29T08:07:36.694 INFO:tasks.workunit.client.0.smithi037.stdout:1/766: chown d2/d4b/c5b 0 1 2024-03-29T08:07:36.694 INFO:tasks.workunit.client.0.smithi037.stdout:8/749: dread - d5/d10/d2d/d34/d50/d83/d46/fbb zero size 2024-03-29T08:07:36.696 INFO:tasks.workunit.client.0.smithi037.stdout:8/750: chown d5/d78/cf2 4933727 1 2024-03-29T08:07:36.700 INFO:tasks.workunit.client.0.smithi037.stdout:6/715: read - d0/d19/fc4 zero size 2024-03-29T08:07:36.703 INFO:tasks.workunit.client.0.smithi037.stdout:7/915: creat dd/d68/d112/f13e x:0 0 0 2024-03-29T08:07:36.704 INFO:tasks.workunit.client.0.smithi037.stdout:7/916: fdatasync dd/d1d/f2a 0 2024-03-29T08:07:36.707 INFO:tasks.workunit.client.0.smithi037.stdout:0/670: creat d9/da/dd/d54/d92/dd4/fe8 x:0 0 0 2024-03-29T08:07:36.709 INFO:tasks.workunit.client.0.smithi037.stdout:9/978: write d1/d4/d19/d1e/d4a/d5c/f5d [45358,129503] 0 2024-03-29T08:07:36.716 INFO:tasks.workunit.client.0.smithi037.stdout:0/671: creat d9/da/dd/d20/d38/d8a/fe9 x:0 0 0 2024-03-29T08:07:36.719 INFO:tasks.workunit.client.0.smithi037.stdout:8/751: mkdir d5/d10/dfa 0 2024-03-29T08:07:36.721 INFO:tasks.workunit.client.0.smithi037.stdout:4/780: dread d3/d57/dec/ded/d1d/db/f16 [0,4194304] 0 2024-03-29T08:07:36.721 INFO:tasks.workunit.client.0.smithi037.stdout:5/927: dwrite d8/fc5 [0,4194304] 0 2024-03-29T08:07:36.721 INFO:tasks.workunit.client.0.smithi037.stdout:6/716: creat d0/d5/d1e/d44/dbb/ddb/ffb x:0 0 0 2024-03-29T08:07:36.723 INFO:tasks.workunit.client.0.smithi037.stdout:1/767: write d2/d6/dc/d5d/d62/f97 [432188,20099] 0 2024-03-29T08:07:36.724 INFO:tasks.workunit.client.0.smithi037.stdout:9/979: truncate d1/d4/d95/f101 471471 0 2024-03-29T08:07:36.726 INFO:tasks.workunit.client.0.smithi037.stdout:7/917: creat dd/d1d/d34/f13f x:0 0 0 2024-03-29T08:07:36.727 INFO:tasks.workunit.client.0.smithi037.stdout:5/928: fdatasync d8/d22/d37/dbc/f111 0 2024-03-29T08:07:36.727 INFO:tasks.workunit.client.0.smithi037.stdout:8/752: mkdir d5/d10/d2d/d9c/dfb 0 2024-03-29T08:07:36.729 INFO:tasks.workunit.client.0.smithi037.stdout:9/980: stat d1/d4/d19/d1e/d4a/f53 0 2024-03-29T08:07:36.731 INFO:tasks.workunit.client.0.smithi037.stdout:4/781: creat d3/d80/f10b x:0 0 0 2024-03-29T08:07:36.732 INFO:tasks.workunit.client.0.smithi037.stdout:1/768: creat d2/d4b/d80/f107 x:0 0 0 2024-03-29T08:07:36.734 INFO:tasks.workunit.client.0.smithi037.stdout:6/717: symlink d0/d5/lfc 0 2024-03-29T08:07:36.736 INFO:tasks.workunit.client.0.smithi037.stdout:5/929: readlink d8/d22/db6/lc4 0 2024-03-29T08:07:36.737 INFO:tasks.workunit.client.0.smithi037.stdout:7/918: mkdir dd/d1d/d49/d81/df0/d140 0 2024-03-29T08:07:36.739 INFO:tasks.workunit.client.0.smithi037.stdout:8/753: creat d5/d10/d45/d4b/ffc x:0 0 0 2024-03-29T08:07:36.741 INFO:tasks.workunit.client.0.smithi037.stdout:6/718: chown d0/d20/fac 196 1 2024-03-29T08:07:36.742 INFO:tasks.workunit.client.0.smithi037.stdout:7/919: fsync dd/d1d/d34/f39 0 2024-03-29T08:07:36.749 INFO:tasks.workunit.client.0.smithi037.stdout:8/754: fsync d5/d10/d19/f43 0 2024-03-29T08:07:36.749 INFO:tasks.workunit.client.0.smithi037.stdout:9/981: truncate d1/d4/d19/d1e/d71/d89/f10d 732680 0 2024-03-29T08:07:36.749 INFO:tasks.workunit.client.0.smithi037.stdout:6/719: chown d0/d19/d34/d5a/dbf/fea 216 1 2024-03-29T08:07:36.752 INFO:tasks.workunit.client.0.smithi037.stdout:5/930: rename d8/d22/d37/d8d/db3/dcb/cff to d8/d22/dc1/c13b 0 2024-03-29T08:07:36.753 INFO:tasks.workunit.client.0.smithi037.stdout:7/920: fdatasync dd/d21/d35/d42/d5c/d8c/f124 0 2024-03-29T08:07:36.754 INFO:tasks.workunit.client.0.smithi037.stdout:6/720: readlink d0/d20/d43/l6d 0 2024-03-29T08:07:36.755 INFO:tasks.workunit.client.0.smithi037.stdout:0/672: dread d9/da/dd/d20/d55/fb6 [0,4194304] 0 2024-03-29T08:07:36.760 INFO:tasks.workunit.client.0.smithi037.stdout:9/982: fsync d1/d4/d19/d1e/d4a/d6f/f83 0 2024-03-29T08:07:36.761 INFO:tasks.workunit.client.0.smithi037.stdout:6/721: chown d0/d5/d1f/da8/f78 17280 1 2024-03-29T08:07:36.764 INFO:tasks.workunit.client.0.smithi037.stdout:4/782: dwrite d3/d57/dec/ded/d1d/d4f/f87 [0,4194304] 0 2024-03-29T08:07:36.764 INFO:tasks.workunit.client.0.smithi037.stdout:7/921: write dd/d21/f13a [1909129,5454] 0 2024-03-29T08:07:36.768 INFO:tasks.workunit.client.0.smithi037.stdout:6/722: write d0/d5/d3f/fb1 [152481,64235] 0 2024-03-29T08:07:36.778 INFO:tasks.workunit.client.0.smithi037.stdout:5/931: getdents d8/d22/d4f 0 2024-03-29T08:07:36.784 INFO:tasks.workunit.client.0.smithi037.stdout:9/983: rmdir d1/d4/d19/d1e/d71/d116 0 2024-03-29T08:07:36.785 INFO:tasks.workunit.client.0.smithi037.stdout:1/769: dread d2/d40/fbc [0,4194304] 0 2024-03-29T08:07:36.786 INFO:tasks.workunit.client.0.smithi037.stdout:5/932: fsync d8/d22/d37/d8d/db3/fb8 0 2024-03-29T08:07:36.787 INFO:tasks.workunit.client.0.smithi037.stdout:6/723: mknod d0/cfd 0 2024-03-29T08:07:36.788 INFO:tasks.workunit.client.0.smithi037.stdout:0/673: dwrite d9/f27 [0,4194304] 0 2024-03-29T08:07:36.801 INFO:tasks.workunit.client.0.smithi037.stdout:0/674: readlink d9/d74/d7d/lb3 0 2024-03-29T08:07:36.802 INFO:tasks.workunit.client.0.smithi037.stdout:0/675: readlink d9/da/dd/d20/d38/l78 0 2024-03-29T08:07:36.804 INFO:tasks.workunit.client.0.smithi037.stdout:1/770: creat d2/d6/dc/d7e/f108 x:0 0 0 2024-03-29T08:07:36.813 INFO:tasks.workunit.client.0.smithi037.stdout:7/922: dread dd/d21/d65/da2/df8/ff9 [0,4194304] 0 2024-03-29T08:07:36.815 INFO:tasks.workunit.client.0.smithi037.stdout:5/933: write d8/d34/f12b [3679466,65174] 0 2024-03-29T08:07:36.818 INFO:tasks.workunit.client.0.smithi037.stdout:0/676: rmdir d9/da/dd/d13/d83 39 2024-03-29T08:07:36.819 INFO:tasks.workunit.client.0.smithi037.stdout:1/771: rmdir d2/d6/dc/d58/d67 39 2024-03-29T08:07:36.820 INFO:tasks.workunit.client.0.smithi037.stdout:4/783: link d3/d4/d27/f2e d3/d57/dec/ded/d1d/db/d45/d1e/d59/d60/db7/f10c 0 2024-03-29T08:07:36.823 INFO:tasks.workunit.client.0.smithi037.stdout:5/934: creat d8/d22/d4f/d70/d79/f13c x:0 0 0 2024-03-29T08:07:36.824 INFO:tasks.workunit.client.0.smithi037.stdout:0/677: chown d9/da/dd/f23 397 1 2024-03-29T08:07:36.826 INFO:tasks.workunit.client.0.smithi037.stdout:7/923: rename dd/d68/d112/d127/f13b to dd/d1d/f141 0 2024-03-29T08:07:36.826 INFO:tasks.workunit.client.0.smithi037.stdout:0/678: read d9/f27 [3407880,31190] 0 2024-03-29T08:07:36.833 INFO:tasks.workunit.client.0.smithi037.stdout:7/924: chown dd/d21/d35/d42/ffe 5 1 2024-03-29T08:07:36.833 INFO:tasks.workunit.client.0.smithi037.stdout:1/772: rename d2/d4b to d2/d6/dc/d18/d1d/dfc/d109 0 2024-03-29T08:07:36.833 INFO:tasks.workunit.client.0.smithi037.stdout:9/984: dwrite d1/d4/d19/d1e/d4a/d6f/da2/d31/fac [4194304,4194304] 0 2024-03-29T08:07:36.838 INFO:tasks.workunit.client.0.smithi037.stdout:7/925: chown dd/d1d/f2a 243554 1 2024-03-29T08:07:36.839 INFO:tasks.workunit.client.0.smithi037.stdout:7/926: stat dd/d1d 0 2024-03-29T08:07:36.839 INFO:tasks.workunit.client.0.smithi037.stdout:6/724: dwrite d0/d5/d1e/d44/d71/f75 [0,4194304] 0 2024-03-29T08:07:36.842 INFO:tasks.workunit.client.0.smithi037.stdout:7/927: chown dd/d21/d65/f6f 6438677 1 2024-03-29T08:07:36.844 INFO:tasks.workunit.client.0.smithi037.stdout:9/985: symlink d1/d1f/d3a/d9b/ddd/l14f 0 2024-03-29T08:07:36.845 INFO:tasks.workunit.client.0.smithi037.stdout:1/773: mknod d2/d6/dc/c10a 0 2024-03-29T08:07:36.846 INFO:tasks.workunit.client.0.smithi037.stdout:7/928: write dd/d68/d112/f13e [790212,117235] 0 2024-03-29T08:07:36.847 INFO:tasks.workunit.client.0.smithi037.stdout:6/725: write d0/d20/d95/ff5 [128352,112683] 0 2024-03-29T08:07:36.847 INFO:tasks.workunit.client.0.smithi037.stdout:5/935: link d8/d18/f28 d8/d22/d37/d8d/db3/dcb/deb/dfa/f13d 0 2024-03-29T08:07:36.854 INFO:tasks.workunit.client.0.smithi037.stdout:0/679: write d9/da/dd/d20/d55/d5f/d9f/fd3 [1661930,118188] 0 2024-03-29T08:07:36.862 INFO:tasks.workunit.client.0.smithi037.stdout:5/936: fdatasync d8/d22/d37/d8d/db3/dcb/deb/dfa/f13d 0 2024-03-29T08:07:36.862 INFO:tasks.workunit.client.0.smithi037.stdout:7/929: truncate dd/d21/d35/d42/d5c/d8c/f128 1177439 0 2024-03-29T08:07:36.866 INFO:tasks.workunit.client.0.smithi037.stdout:6/726: symlink d0/d19/d34/d4e/d69/d79/dcd/lfe 0 2024-03-29T08:07:36.866 INFO:tasks.workunit.client.0.smithi037.stdout:7/930: read dd/d21/d35/d3b/d5d/fc7 [313241,123294] 0 2024-03-29T08:07:36.869 INFO:tasks.workunit.client.0.smithi037.stdout:1/774: write d2/d40/de8/daf/fdf [993612,117298] 0 2024-03-29T08:07:36.869 INFO:tasks.workunit.client.0.smithi037.stdout:9/986: dread d1/d4/d19/d1e/d4a/d6f/da2/d31/fac [4194304,4194304] 0 2024-03-29T08:07:36.871 INFO:tasks.workunit.client.0.smithi037.stdout:5/937: dread d8/f1f [0,4194304] 0 2024-03-29T08:07:36.876 INFO:tasks.workunit.client.0.smithi037.stdout:1/775: mknod d2/d6/d13/c10b 0 2024-03-29T08:07:36.881 INFO:tasks.workunit.client.0.smithi037.stdout:5/938: rmdir d8/d22/d37/d8d/db3/dcb/deb 39 2024-03-29T08:07:36.882 INFO:tasks.workunit.client.0.smithi037.stdout:0/680: dwrite d9/da/dd/d54/d92/dd4/fe8 [0,4194304] 0 2024-03-29T08:07:36.882 INFO:tasks.workunit.client.0.smithi037.stdout:6/727: write d0/d5/d1e/f5e [129518,55436] 0 2024-03-29T08:07:36.883 INFO:tasks.workunit.client.0.smithi037.stdout:7/931: getdents dd/d1d/d49 0 2024-03-29T08:07:36.884 INFO:tasks.workunit.client.0.smithi037.stdout:9/987: fdatasync d1/d4/d19/d1e/d4a/d6f/da2/d31/fac 0 2024-03-29T08:07:36.886 INFO:tasks.workunit.client.0.smithi037.stdout:5/939: fsync d8/d22/d4f/d70/d9c/f102 0 2024-03-29T08:07:36.887 INFO:tasks.workunit.client.0.smithi037.stdout:1/776: fdatasync d2/d6/dc/d7e/dd2/fde 0 2024-03-29T08:07:36.893 INFO:tasks.workunit.client.0.smithi037.stdout:9/988: fsync d1/d4/d19/d1e/d4a/d6f/da2/f7a 0 2024-03-29T08:07:36.899 INFO:tasks.workunit.client.0.smithi037.stdout:6/728: rename d0/d19/d34/d4e/d69/dee/ff3 to d0/d6a/d7b/fff 0 2024-03-29T08:07:36.900 INFO:tasks.workunit.client.0.smithi037.stdout:7/932: write dd/f7e [1356259,25117] 0 2024-03-29T08:07:36.903 INFO:tasks.workunit.client.0.smithi037.stdout:1/777: truncate d2/d6/dc/d18/d1d/dfc/d109/da1/f96 2498904 0 2024-03-29T08:07:36.904 INFO:tasks.workunit.client.0.smithi037.stdout:5/940: getdents d8/d22/d37/d8d/db3/dcb/deb/d88/d133 0 2024-03-29T08:07:36.906 INFO:tasks.workunit.client.0.smithi037.stdout:6/729: symlink d0/d5/d3f/l100 0 2024-03-29T08:07:36.907 INFO:tasks.workunit.client.0.smithi037.stdout:1/778: mknod d2/d40/c10c 0 2024-03-29T08:07:36.912 INFO:tasks.workunit.client.0.smithi037.stdout:7/933: creat dd/d21/d65/da2/f142 x:0 0 0 2024-03-29T08:07:36.917 INFO:tasks.workunit.client.0.smithi037.stdout:6/730: rename d0/d5/d1e/f5f to d0/d5/d1e/d44/f101 0 2024-03-29T08:07:36.917 INFO:tasks.workunit.client.0.smithi037.stdout:0/681: dread d9/d74/d7d/f87 [0,4194304] 0 2024-03-29T08:07:36.919 INFO:tasks.workunit.client.0.smithi037.stdout:6/731: chown d0/d19/f9b 7381 1 2024-03-29T08:07:36.920 INFO:tasks.workunit.client.0.smithi037.stdout:9/989: link d1/d4/d19/d1e/d4a/c7e d1/d4/d19/d1e/d4a/d6f/da2/c150 0 2024-03-29T08:07:36.923 INFO:tasks.workunit.client.0.smithi037.stdout:6/732: truncate d0/d5/d1f/f63 4911228 0 2024-03-29T08:07:36.929 INFO:tasks.workunit.client.0.smithi037.stdout:0/682: creat d9/da/dd/d20/d55/dac/fea x:0 0 0 2024-03-29T08:07:36.931 INFO:tasks.workunit.client.0.smithi037.stdout:0/683: chown d9/da/dd/d20/d55/dd2 146141692 1 2024-03-29T08:07:36.932 INFO:tasks.workunit.client.0.smithi037.stdout:7/934: symlink dd/d21/l143 0 2024-03-29T08:07:36.935 INFO:tasks.workunit.client.0.smithi037.stdout:5/941: write d8/d22/d37/d8d/db3/dcb/f105 [975229,40869] 0 2024-03-29T08:07:36.938 INFO:tasks.workunit.client.0.smithi037.stdout:9/990: unlink d1/d4/d19/d1e/d4a/d6f/da2/d13/d7c/f146 0 2024-03-29T08:07:36.939 INFO:tasks.workunit.client.0.smithi037.stdout:0/684: symlink d9/da/dd/d20/d38/leb 0 2024-03-29T08:07:36.943 INFO:tasks.workunit.client.0.smithi037.stdout:1/779: dwrite d2/d6/dc/d18/d22/ff9 [0,4194304] 0 2024-03-29T08:07:36.946 INFO:tasks.workunit.client.0.smithi037.stdout:9/991: write d1/d4/d19/d1e/d4a/d6f/da2/d31/daa/dba/de3/f104 [146003,782] 0 2024-03-29T08:07:36.947 INFO:tasks.workunit.client.0.smithi037.stdout:6/733: dwrite d0/d19/d34/d5a/f39 [4194304,4194304] 0 2024-03-29T08:07:36.951 INFO:tasks.workunit.client.0.smithi037.stdout:6/734: stat d0/d19/d34/d4e/fb4 0 2024-03-29T08:07:36.955 INFO:tasks.workunit.client.0.smithi037.stdout:5/942: creat d8/d4d/f13e x:0 0 0 2024-03-29T08:07:36.955 INFO:tasks.workunit.client.0.smithi037.stdout:7/935: dread dd/d21/d35/d3b/fb8 [0,4194304] 0 2024-03-29T08:07:36.958 INFO:tasks.workunit.client.0.smithi037.stdout:1/780: rename d2/d6/dc/d18/dbd/dda to d2/d6/dc/d18/d22/d4f/d86/d10d 0 2024-03-29T08:07:36.963 INFO:tasks.workunit.client.0.smithi037.stdout:9/992: truncate d1/d1f/dfd/d127/f12f 1814977 0 2024-03-29T08:07:36.964 INFO:tasks.workunit.client.0.smithi037.stdout:7/936: creat dd/dc3/f144 x:0 0 0 2024-03-29T08:07:36.964 INFO:tasks.workunit.client.0.smithi037.stdout:1/781: creat d2/d40/de8/f10e x:0 0 0 2024-03-29T08:07:36.968 INFO:tasks.workunit.client.0.smithi037.stdout:0/685: dread d9/da/d1b/f41 [0,4194304] 0 2024-03-29T08:07:36.968 INFO:tasks.workunit.client.0.smithi037.stdout:5/943: dread d8/d22/d4f/d70/d110/fdb [0,4194304] 0 2024-03-29T08:07:36.969 INFO:tasks.workunit.client.0.smithi037.stdout:9/993: mkdir d1/d1f/d3a/d151 0 2024-03-29T08:07:36.978 INFO:tasks.workunit.client.0.smithi037.stdout:7/937: getdents dd/d21/d85 0 2024-03-29T08:07:36.980 INFO:tasks.workunit.client.0.smithi037.stdout:0/686: creat d9/da/dd/d20/d55/d5f/d9f/fec x:0 0 0 2024-03-29T08:07:36.981 INFO:tasks.workunit.client.0.smithi037.stdout:5/944: read d8/d22/d37/d8d/db3/dcb/deb/f76 [3190871,105763] 0 2024-03-29T08:07:36.983 INFO:tasks.workunit.client.0.smithi037.stdout:6/735: dwrite d0/d6a/d7b/fff [0,4194304] 0 2024-03-29T08:07:36.992 INFO:tasks.workunit.client.0.smithi037.stdout:7/938: creat dd/d21/d35/d3b/dce/de0/f145 x:0 0 0 2024-03-29T08:07:36.992 INFO:tasks.workunit.client.0.smithi037.stdout:1/782: write d2/d40/de8/d87/fce [482882,12871] 0 2024-03-29T08:07:36.998 INFO:tasks.workunit.client.0.smithi037.stdout:7/939: chown dd/d21/d65/f135 3034 1 2024-03-29T08:07:37.007 INFO:tasks.workunit.client.0.smithi037.stdout:0/687: dwrite d9/da/dd/d13/d7b/fa6 [0,4194304] 0 2024-03-29T08:07:37.012 INFO:tasks.workunit.client.0.smithi037.stdout:9/994: dwrite d1/d1f/f33 [0,4194304] 0 2024-03-29T08:07:37.013 INFO:tasks.workunit.client.0.smithi037.stdout:0/688: chown d9/da/dd/d20/d38/f69 1534 1 2024-03-29T08:07:37.023 INFO:tasks.workunit.client.0.smithi037.stdout:1/783: getdents d2/d6/dc/d18/d1d/dfc/d109/d80/db0 0 2024-03-29T08:07:37.027 INFO:tasks.workunit.client.0.smithi037.stdout:6/736: link d0/d19/d42/lba d0/d19/d34/d4e/d94/l102 0 2024-03-29T08:07:37.027 INFO:tasks.workunit.client.0.smithi037.stdout:2/661: sync 2024-03-29T08:07:37.027 INFO:tasks.workunit.client.0.smithi037.stdout:3/686: sync 2024-03-29T08:07:37.033 INFO:tasks.workunit.client.0.smithi037.stdout:2/662: fsync d0/d23/d44/dca/d2e/d5f/d6f/d5e/f60 0 2024-03-29T08:07:37.037 INFO:tasks.workunit.client.0.smithi037.stdout:9/995: symlink d1/d4/d19/d1e/d4a/d6f/da2/d23/l152 0 2024-03-29T08:07:37.041 INFO:tasks.workunit.client.0.smithi037.stdout:0/689: rename d9/da/dd/d20/c32 to d9/da/dd/d54/db5/dcb/ced 0 2024-03-29T08:07:37.044 INFO:tasks.workunit.client.0.smithi037.stdout:6/737: mknod d0/d19/d34/d5a/dbf/c103 0 2024-03-29T08:07:37.045 INFO:tasks.workunit.client.0.smithi037.stdout:6/738: chown d0 532 1 2024-03-29T08:07:37.052 INFO:tasks.workunit.client.0.smithi037.stdout:2/663: creat d0/d23/d44/dca/d2e/fe7 x:0 0 0 2024-03-29T08:07:37.054 INFO:tasks.workunit.client.0.smithi037.stdout:3/687: dwrite dd/ff [4194304,4194304] 0 2024-03-29T08:07:37.056 INFO:tasks.workunit.client.0.smithi037.stdout:6/739: mkdir d0/d19/d34/d4e/d69/dee/d104 0 2024-03-29T08:07:37.064 INFO:tasks.workunit.client.0.smithi037.stdout:6/740: fsync d0/d5/d3f/f17 0 2024-03-29T08:07:37.065 INFO:tasks.workunit.client.0.smithi037.stdout:7/940: getdents dd/d21/d35/dcf 0 2024-03-29T08:07:37.067 INFO:tasks.workunit.client.0.smithi037.stdout:0/690: mkdir d9/da/d1b/d22/dee 0 2024-03-29T08:07:37.070 INFO:tasks.workunit.client.0.smithi037.stdout:1/784: dread d2/d6/d13/f77 [0,4194304] 0 2024-03-29T08:07:37.073 INFO:tasks.workunit.client.0.smithi037.stdout:7/941: symlink dd/d21/d35/d42/ddb/de2/l146 0 2024-03-29T08:07:37.076 INFO:tasks.workunit.client.0.smithi037.stdout:0/691: creat d9/d74/d7d/d7e/fef x:0 0 0 2024-03-29T08:07:37.085 INFO:tasks.workunit.client.0.smithi037.stdout:1/785: unlink d2/d6/d13/fe5 0 2024-03-29T08:07:37.086 INFO:tasks.workunit.client.0.smithi037.stdout:2/664: dwrite d0/d23/d44/dca/d2e/d5f/fe1 [0,4194304] 0 2024-03-29T08:07:37.098 INFO:tasks.workunit.client.0.smithi037.stdout:7/942: dwrite dd/dc3/f144 [0,4194304] 0 2024-03-29T08:07:37.110 INFO:tasks.workunit.client.0.smithi037.stdout:9/996: dwrite d1/d1f/d3a/fc0 [0,4194304] 0 2024-03-29T08:07:37.112 INFO:tasks.workunit.client.0.smithi037.stdout:0/692: symlink d9/da/lf0 0 2024-03-29T08:07:37.127 INFO:tasks.workunit.client.0.smithi037.stdout:3/688: dwrite dd/d15/d18/d4b/f67 [0,4194304] 0 2024-03-29T08:07:37.133 INFO:tasks.workunit.client.0.smithi037.stdout:0/693: dread d9/da/d1b/f41 [0,4194304] 0 2024-03-29T08:07:37.136 INFO:tasks.workunit.client.0.smithi037.stdout:7/943: mkdir dd/d68/d147 0 2024-03-29T08:07:37.137 INFO:tasks.workunit.client.0.smithi037.stdout:6/741: dwrite d0/d6a/f70 [0,4194304] 0 2024-03-29T08:07:37.146 INFO:tasks.workunit.client.0.smithi037.stdout:9/997: rename d1/d1f/ldb to d1/d4/d19/d1e/d4a/d6f/da2/d23/l153 0 2024-03-29T08:07:37.147 INFO:tasks.workunit.client.0.smithi037.stdout:9/998: stat d1/d4 0 2024-03-29T08:07:37.152 INFO:tasks.workunit.client.0.smithi037.stdout:2/665: write d0/d23/d44/dca/d22/fa0 [183136,82670] 0 2024-03-29T08:07:37.155 INFO:tasks.workunit.client.0.smithi037.stdout:7/944: write dd/d1d/d49/fbc [1456339,105272] 0 2024-03-29T08:07:37.158 INFO:tasks.workunit.client.0.smithi037.stdout:1/786: creat d2/d6/dc/d18/d1d/d21/d99/f10f x:0 0 0 2024-03-29T08:07:37.172 INFO:tasks.workunit.client.0.smithi037.stdout:1/787: chown d2/d6/dc/d18/d1d/dfc/d109/d80/db0/le2 383316 1 2024-03-29T08:07:37.173 INFO:tasks.workunit.client.0.smithi037.stdout:9/999: mkdir d1/d4/d19/d1e/d4a/d6f/da2/d23/ded/d154 0 2024-03-29T08:07:37.173 INFO:tasks.workunit.client.0.smithi037.stdout:7/945: mkdir dd/d21/d35/d42/ddb/de2/d122/d148 0 2024-03-29T08:07:37.175 INFO:tasks.workunit.client.0.smithi037.stdout:3/689: getdents dd/d46/d2d/d3a 0 2024-03-29T08:07:37.187 INFO:tasks.workunit.client.0.smithi037.stdout:3/690: unlink dd/d15/d18/d8e/fa8 0 2024-03-29T08:07:37.194 INFO:tasks.workunit.client.0.smithi037.stdout:7/946: dread dd/d21/d35/d42/d5c/f9d [0,4194304] 0 2024-03-29T08:07:37.196 INFO:tasks.workunit.client.0.smithi037.stdout:2/666: dwrite d0/d53/f88 [0,4194304] 0 2024-03-29T08:07:37.198 INFO:tasks.workunit.client.0.smithi037.stdout:3/691: fdatasync dd/d46/d2d/d3a/d6c/db1/dc0/dda/d33/d75/f8d 0 2024-03-29T08:07:37.200 INFO:tasks.workunit.client.0.smithi037.stdout:7/947: write dd/d68/f129 [552407,20278] 0 2024-03-29T08:07:37.200 INFO:tasks.workunit.client.0.smithi037.stdout:2/667: readlink d0/d53/d89/lb9 0 2024-03-29T08:07:37.203 INFO:tasks.workunit.client.0.smithi037.stdout:7/948: stat dd/d21/d35/d60/d94/fff 0 2024-03-29T08:07:37.207 INFO:tasks.workunit.client.0.smithi037.stdout:2/668: symlink d0/d23/daf/le8 0 2024-03-29T08:07:37.208 INFO:tasks.workunit.client.0.smithi037.stdout:7/949: write dd/d21/d35/d60/d94/fb5 [1450841,111729] 0 2024-03-29T08:07:37.209 INFO:tasks.workunit.client.0.smithi037.stdout:1/788: dread d2/d6/dc/d18/d1d/dfc/d109/fc4 [0,4194304] 0 2024-03-29T08:07:37.212 INFO:tasks.workunit.client.0.smithi037.stdout:2/669: dread - d0/d23/d44/dca/d2e/d5f/d6f/d5e/dc2/fc7 zero size 2024-03-29T08:07:37.213 INFO:tasks.workunit.client.0.smithi037.stdout:1/789: dread - d2/d6/dc/d18/d22/d4f/d86/d10d/f106 zero size 2024-03-29T08:07:37.216 INFO:tasks.workunit.client.0.smithi037.stdout:1/790: stat d2/d6/dc/d58/fed 0 2024-03-29T08:07:37.220 INFO:tasks.workunit.client.0.smithi037.stdout:1/791: write d2/d6/dc/d18/d1d/d5e/fe6 [1538201,121259] 0 2024-03-29T08:07:37.224 INFO:tasks.workunit.client.0.smithi037.stdout:2/670: rmdir d0/d23/d44/dca/d2e/d5f/d6f 39 2024-03-29T08:07:37.226 INFO:tasks.workunit.client.0.smithi037.stdout:3/692: dwrite dd/d46/d2d/d3a/d6c/db1/dc0/dda/ddb/f5a [0,4194304] 0 2024-03-29T08:07:37.237 INFO:tasks.workunit.client.0.smithi037.stdout:1/792: mknod d2/d6/dc/d18/d1d/d5e/d72/c110 0 2024-03-29T08:07:37.241 INFO:tasks.workunit.client.0.smithi037.stdout:3/693: truncate dd/d15/d18/d4b/fad 788866 0 2024-03-29T08:07:37.259 INFO:tasks.workunit.client.0.smithi037.stdout:3/694: rename l3 to dd/d46/d3e/d47/le4 0 2024-03-29T08:07:37.264 INFO:tasks.workunit.client.0.smithi037.stdout:3/695: fdatasync dd/d15/d18/f2f 0 2024-03-29T08:07:37.275 INFO:tasks.workunit.client.0.smithi037.stdout:2/671: dwrite d0/d23/d44/f67 [0,4194304] 0 2024-03-29T08:07:37.278 INFO:tasks.workunit.client.0.smithi037.stdout:8/755: sync 2024-03-29T08:07:37.281 INFO:tasks.workunit.client.0.smithi037.stdout:2/672: chown d0/d23/d44/dca/d2e/c32 9045 1 2024-03-29T08:07:37.286 INFO:tasks.workunit.client.0.smithi037.stdout:3/696: dwrite dd/d46/d2d/d3a/f53 [0,4194304] 0 2024-03-29T08:07:37.293 INFO:tasks.workunit.client.0.smithi037.stdout:3/697: mknod dd/d5e/ce5 0 2024-03-29T08:07:37.294 INFO:tasks.workunit.client.0.smithi037.stdout:2/673: mknod d0/d23/d44/dca/d2e/d5f/ce9 0 2024-03-29T08:07:37.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:07:37 smithi148 ceph-mon[44992]: pgmap v50: 65 pgs: 65 active+clean; 3.5 GiB data, 12 GiB used, 525 GiB / 536 GiB avail; 90 MiB/s rd, 199 MiB/s wr, 449 op/s 2024-03-29T08:07:37.307 INFO:tasks.workunit.client.0.smithi037.stdout:3/698: creat dd/d15/fe6 x:0 0 0 2024-03-29T08:07:37.309 INFO:tasks.workunit.client.0.smithi037.stdout:3/699: write dd/d46/d2d/fcb [365368,70881] 0 2024-03-29T08:07:37.325 INFO:tasks.workunit.client.0.smithi037.stdout:2/674: write d0/d23/d44/dca/d22/f3f [819364,126776] 0 2024-03-29T08:07:37.325 INFO:tasks.workunit.client.0.smithi037.stdout:8/756: dread d5/d10/d19/d8d/fa8 [0,4194304] 0 2024-03-29T08:07:37.329 INFO:tasks.workunit.client.0.smithi037.stdout:2/675: mknod d0/d23/d44/dca/d22/cea 0 2024-03-29T08:07:37.334 INFO:tasks.workunit.client.0.smithi037.stdout:2/676: write d0/d23/d44/dca/d22/f48 [1009158,26680] 0 2024-03-29T08:07:37.342 INFO:tasks.workunit.client.0.smithi037.stdout:2/677: write d0/d23/d44/dca/d2e/d38/fb6 [1609433,68449] 0 2024-03-29T08:07:37.353 INFO:tasks.workunit.client.0.smithi037.stdout:8/757: dwrite d5/d10/d2d/d34/d50/d83/d46/f98 [0,4194304] 0 2024-03-29T08:07:37.356 INFO:tasks.workunit.client.0.smithi037.stdout:2/678: truncate d0/d23/d44/dca/d2e/d5f/d6f/d5e/dc2/fc7 755393 0 2024-03-29T08:07:37.364 INFO:tasks.workunit.client.0.smithi037.stdout:2/679: creat d0/d23/d44/dca/d2e/d5f/d6f/d5e/dc2/feb x:0 0 0 2024-03-29T08:07:37.365 INFO:tasks.workunit.client.0.smithi037.stdout:2/680: fdatasync d0/f8e 0 2024-03-29T08:07:37.374 INFO:tasks.workunit.client.0.smithi037.stdout:2/681: rename d0/d53/f88 to d0/d23/d44/dca/d2e/da4/fec 0 2024-03-29T08:07:37.377 INFO:tasks.workunit.client.0.smithi037.stdout:2/682: chown d0/d23/d44/dca/d22/d55/d92 5487246 1 2024-03-29T08:07:37.388 INFO:tasks.workunit.client.0.smithi037.stdout:2/683: link d0/d23/d44/dca/d7d/cd6 d0/d23/d44/dca/d2e/d5f/ced 0 2024-03-29T08:07:37.391 INFO:tasks.workunit.client.0.smithi037.stdout:2/684: read d0/d23/d44/dca/f49 [2598656,60321] 0 2024-03-29T08:07:37.395 INFO:tasks.workunit.client.0.smithi037.stdout:2/685: symlink d0/d23/d44/d62/lee 0 2024-03-29T08:07:37.403 INFO:tasks.workunit.client.0.smithi037.stdout:2/686: link d0/lc8 d0/d23/d44/dca/d7d/lef 0 2024-03-29T08:07:37.404 INFO:tasks.workunit.client.0.smithi037.stdout:2/687: write d0/fd [2860297,98624] 0 2024-03-29T08:07:37.426 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:07:37 smithi037 ceph-mon[35162]: pgmap v50: 65 pgs: 65 active+clean; 3.5 GiB data, 12 GiB used, 525 GiB / 536 GiB avail; 90 MiB/s rd, 199 MiB/s wr, 449 op/s 2024-03-29T08:07:37.433 INFO:tasks.workunit.client.0.smithi037.stdout:2/688: dread d0/d23/d44/dca/fce [4194304,4194304] 0 2024-03-29T08:07:37.445 INFO:tasks.workunit.client.0.smithi037.stdout:2/689: mkdir d0/d23/d44/dca/d22/df0 0 2024-03-29T08:07:37.449 INFO:tasks.workunit.client.0.smithi037.stdout:2/690: chown d0/d23/d44/dca/d2e/d5f/da5/cd9 5463827 1 2024-03-29T08:07:37.456 INFO:tasks.workunit.client.0.smithi037.stdout:2/691: write d0/d53/f7a [842087,122755] 0 2024-03-29T08:07:37.457 INFO:tasks.workunit.client.0.smithi037.stdout:2/692: chown d0/d23/daf/dbf 1 1 2024-03-29T08:07:37.462 INFO:tasks.workunit.client.0.smithi037.stdout:2/693: getdents d0/d23/d70/d71/d78 0 2024-03-29T08:07:37.484 INFO:tasks.workunit.client.0.smithi037.stdout:2/694: dread d0/d23/d44/dca/d2e/f36 [0,4194304] 0 2024-03-29T08:07:37.488 INFO:tasks.workunit.client.0.smithi037.stdout:2/695: unlink d0/d23/d70/d71/ca2 0 2024-03-29T08:07:37.491 INFO:tasks.workunit.client.0.smithi037.stdout:2/696: chown d0/d23/d44/dca/d2e/l5d 18 1 2024-03-29T08:07:37.496 INFO:tasks.workunit.client.0.smithi037.stdout:2/697: rename d0/d53/f7a to d0/d23/d44/dca/ff1 0 2024-03-29T08:07:37.499 INFO:tasks.workunit.client.0.smithi037.stdout:2/698: creat d0/d23/d44/dca/d22/d55/d92/ff2 x:0 0 0 2024-03-29T08:07:37.501 INFO:tasks.workunit.client.0.smithi037.stdout:2/699: fdatasync d0/f7 0 2024-03-29T08:07:37.509 INFO:tasks.workunit.client.0.smithi037.stdout:2/700: dread d0/d23/d44/dca/d2e/d38/f94 [0,4194304] 0 2024-03-29T08:07:37.514 INFO:tasks.workunit.client.0.smithi037.stdout:2/701: rename d0/d23/d70/d71 to d0/d23/d44/dca/d2e/d9c/df3 0 2024-03-29T08:07:37.530 INFO:tasks.workunit.client.0.smithi037.stdout:2/702: dwrite d0/d23/d44/dca/d2e/da4/fe3 [0,4194304] 0 2024-03-29T08:07:37.533 INFO:tasks.workunit.client.0.smithi037.stdout:2/703: read d0/d23/d44/dca/d22/d4e/f4f [2763589,12174] 0 2024-03-29T08:07:37.535 INFO:tasks.workunit.client.0.smithi037.stdout:4/784: sync 2024-03-29T08:07:37.535 INFO:tasks.workunit.client.0.smithi037.stdout:5/945: sync 2024-03-29T08:07:37.535 INFO:tasks.workunit.client.0.smithi037.stdout:2/704: stat d0/d23/d44/dca/d2e/d38/f6b 0 2024-03-29T08:07:37.537 INFO:tasks.workunit.client.0.smithi037.stdout:2/705: read - d0/d23/d44/dca/fba zero size 2024-03-29T08:07:37.541 INFO:tasks.workunit.client.0.smithi037.stdout:4/785: unlink d3/d57/dec/ded/d1d/db/d45/c36 0 2024-03-29T08:07:37.542 INFO:tasks.workunit.client.0.smithi037.stdout:2/706: mknod d0/d23/d44/dca/d7d/cf4 0 2024-03-29T08:07:37.545 INFO:tasks.workunit.client.0.smithi037.stdout:4/786: creat d3/d57/d85/f10d x:0 0 0 2024-03-29T08:07:37.548 INFO:tasks.workunit.client.0.smithi037.stdout:4/787: symlink d3/d80/dcd/l10e 0 2024-03-29T08:07:37.551 INFO:tasks.workunit.client.0.smithi037.stdout:2/707: dread d0/d23/d44/dca/f3d [0,4194304] 0 2024-03-29T08:07:37.555 INFO:tasks.workunit.client.0.smithi037.stdout:5/946: dwrite d8/d22/db6/f107 [0,4194304] 0 2024-03-29T08:07:37.558 INFO:tasks.workunit.client.0.smithi037.stdout:5/947: fsync d8/d22/d4f/d70/f82 0 2024-03-29T08:07:37.560 INFO:tasks.workunit.client.0.smithi037.stdout:5/948: stat d8/d22/d37/d8d/db3/dcb/fe1 0 2024-03-29T08:07:37.560 INFO:tasks.workunit.client.0.smithi037.stdout:4/788: unlink d3/d57/dec/ded/d1d/db/d72/d98/cea 0 2024-03-29T08:07:37.564 INFO:tasks.workunit.client.0.smithi037.stdout:2/708: rename d0/c1c to d0/d23/d44/dca/d2e/d5f/d6f/d5e/cf5 0 2024-03-29T08:07:37.565 INFO:tasks.workunit.client.0.smithi037.stdout:2/709: chown d0/d23/d44/dca/d7d/fcc 1 1 2024-03-29T08:07:37.570 INFO:tasks.workunit.client.0.smithi037.stdout:4/789: creat d3/d57/dec/ded/d1d/db/d45/d1e/d59/d73/f10f x:0 0 0 2024-03-29T08:07:37.573 INFO:tasks.workunit.client.0.smithi037.stdout:2/710: unlink d0/fd 0 2024-03-29T08:07:37.579 INFO:tasks.workunit.client.0.smithi037.stdout:4/790: dread d3/d57/d6a/f77 [0,4194304] 0 2024-03-29T08:07:37.593 INFO:tasks.workunit.client.0.smithi037.stdout:2/711: dread d0/d23/d44/dca/d22/f39 [0,4194304] 0 2024-03-29T08:07:37.780 INFO:tasks.workunit.client.0.smithi037.stdout:0/694: sync 2024-03-29T08:07:37.813 INFO:tasks.workunit.client.0.smithi037.stdout:0/695: read d9/da/dd/f76 [29341,52266] 0 2024-03-29T08:07:37.816 INFO:tasks.workunit.client.0.smithi037.stdout:0/696: fsync d9/da/dd/f5b 0 2024-03-29T08:07:37.833 INFO:tasks.workunit.client.0.smithi037.stdout:0/697: dwrite d9/da/dd/d20/d72/f85 [0,4194304] 0 2024-03-29T08:07:37.838 INFO:tasks.workunit.client.0.smithi037.stdout:0/698: creat d9/da/dd/d20/d72/ff1 x:0 0 0 2024-03-29T08:07:37.844 INFO:tasks.workunit.client.0.smithi037.stdout:0/699: rename d9/d74/d7d/d7e/fbf to d9/da/d1b/d22/d2c/ff2 0 2024-03-29T08:07:37.848 INFO:tasks.workunit.client.0.smithi037.stdout:0/700: creat d9/da/dd/d20/d55/d5f/dab/ff3 x:0 0 0 2024-03-29T08:07:37.858 INFO:tasks.workunit.client.0.smithi037.stdout:0/701: rename d9/da/dd/d20/d55/dd2 to d9/da/d1b/d22/dee/df4 0 2024-03-29T08:07:37.859 INFO:tasks.workunit.client.0.smithi037.stdout:0/702: truncate d9/da/dd/d20/d55/d5f/fe0 540208 0 2024-03-29T08:07:37.861 INFO:tasks.workunit.client.0.smithi037.stdout:0/703: write d9/da/dd/f23 [2178912,52693] 0 2024-03-29T08:07:37.867 INFO:tasks.workunit.client.0.smithi037.stdout:0/704: getdents d9/da/dd/d20/d38/d53 0 2024-03-29T08:07:37.871 INFO:tasks.workunit.client.0.smithi037.stdout:0/705: fdatasync d9/da/dd/d13/f36 0 2024-03-29T08:07:37.877 INFO:tasks.workunit.client.0.smithi037.stdout:0/706: rename d9/da/d1b/d22/d2c/c89 to d9/da/dd/d13/d7b/cf5 0 2024-03-29T08:07:37.882 INFO:tasks.workunit.client.0.smithi037.stdout:0/707: rename d9/da/dd/d54/da8 to d9/da/dd/d54/db5/df6 0 2024-03-29T08:07:37.889 INFO:tasks.workunit.client.0.smithi037.stdout:0/708: getdents d9/da/d1b/d22/dee 0 2024-03-29T08:07:37.905 INFO:tasks.workunit.client.0.smithi037.stdout:0/709: dwrite d9/da/dd/d20/f9e [0,4194304] 0 2024-03-29T08:07:37.922 INFO:tasks.workunit.client.0.smithi037.stdout:0/710: dwrite d9/da/dd/d13/f45 [0,4194304] 0 2024-03-29T08:07:37.927 INFO:tasks.workunit.client.0.smithi037.stdout:0/711: write d9/da/dd/d20/d38/d8a/f9a [4179887,57609] 0 2024-03-29T08:07:37.927 INFO:tasks.workunit.client.0.smithi037.stdout:6/742: sync 2024-03-29T08:07:37.934 INFO:tasks.workunit.client.0.smithi037.stdout:0/712: mkdir d9/da/dd/d20/d72/df7 0 2024-03-29T08:07:37.938 INFO:tasks.workunit.client.0.smithi037.stdout:0/713: mknod d9/da/dd/d20/d55/d5f/cf8 0 2024-03-29T08:07:37.942 INFO:tasks.workunit.client.0.smithi037.stdout:6/743: rename d0/d19/d34/d4e/d69/d79/dcc to d0/d19/d34/d4e/df2/d105 0 2024-03-29T08:07:37.950 INFO:tasks.workunit.client.0.smithi037.stdout:6/744: creat d0/d19/d34/d4e/d69/dee/f106 x:0 0 0 2024-03-29T08:07:37.953 INFO:tasks.workunit.client.0.smithi037.stdout:0/714: dread d9/da/dd/d20/d55/fb6 [0,4194304] 0 2024-03-29T08:07:37.953 INFO:tasks.workunit.client.0.smithi037.stdout:6/745: mknod d0/d6a/dd6/c107 0 2024-03-29T08:07:37.955 INFO:tasks.workunit.client.0.smithi037.stdout:6/746: stat d0/d19/d34/d5a/dbf/dde 0 2024-03-29T08:07:37.957 INFO:tasks.workunit.client.0.smithi037.stdout:6/747: truncate d0/d6a/f85 1004688 0 2024-03-29T08:07:37.960 INFO:tasks.workunit.client.0.smithi037.stdout:6/748: unlink d0/d19/d42/d60/ld4 0 2024-03-29T08:07:37.965 INFO:tasks.workunit.client.0.smithi037.stdout:0/715: dread d9/d74/d7d/f87 [0,4194304] 0 2024-03-29T08:07:37.967 INFO:tasks.workunit.client.0.smithi037.stdout:6/749: creat d0/d19/f108 x:0 0 0 2024-03-29T08:07:37.969 INFO:tasks.workunit.client.0.smithi037.stdout:6/750: dread - d0/d5/d1f/fdc zero size 2024-03-29T08:07:37.969 INFO:tasks.workunit.client.0.smithi037.stdout:0/716: creat d9/da/dd/d20/ff9 x:0 0 0 2024-03-29T08:07:37.972 INFO:tasks.workunit.client.0.smithi037.stdout:0/717: fdatasync d9/da/dd/d20/d38/f84 0 2024-03-29T08:07:37.983 INFO:tasks.workunit.client.0.smithi037.stdout:0/718: dread d9/d74/f97 [0,4194304] 0 2024-03-29T08:07:37.986 INFO:tasks.workunit.client.0.smithi037.stdout:0/719: chown d9/da/dd/d20/d38/d8a/cc7 16270 1 2024-03-29T08:07:37.988 INFO:tasks.workunit.client.0.smithi037.stdout:6/751: dwrite d0/d19/d34/d4e/d69/dee/ffa [0,4194304] 0 2024-03-29T08:07:37.995 INFO:tasks.workunit.client.0.smithi037.stdout:0/720: rmdir d9/da/dd/d20/d55/d5f/d9f 39 2024-03-29T08:07:37.998 INFO:tasks.workunit.client.0.smithi037.stdout:6/752: symlink d0/d20/d43/l109 0 2024-03-29T08:07:37.999 INFO:tasks.workunit.client.0.smithi037.stdout:0/721: creat d9/da/dd/d20/d55/dac/ffa x:0 0 0 2024-03-29T08:07:38.015 INFO:tasks.workunit.client.0.smithi037.stdout:0/722: creat d9/da/dd/d13/ffb x:0 0 0 2024-03-29T08:07:38.023 INFO:tasks.workunit.client.0.smithi037.stdout:7/950: sync 2024-03-29T08:07:38.024 INFO:tasks.workunit.client.0.smithi037.stdout:6/753: write d0/d19/d42/d60/f8b [349846,34577] 0 2024-03-29T08:07:38.028 INFO:tasks.workunit.client.0.smithi037.stdout:0/723: truncate d9/da/d1b/f3d 586742 0 2024-03-29T08:07:38.030 INFO:tasks.workunit.client.0.smithi037.stdout:6/754: symlink d0/d19/d34/d5a/dbf/dde/l10a 0 2024-03-29T08:07:38.032 INFO:tasks.workunit.client.0.smithi037.stdout:7/951: symlink dd/d72/l149 0 2024-03-29T08:07:38.042 INFO:tasks.workunit.client.0.smithi037.stdout:6/755: mknod d0/d19/d34/d5a/dbf/dde/c10b 0 2024-03-29T08:07:38.048 INFO:tasks.workunit.client.0.smithi037.stdout:6/756: rmdir d0/d5/d1f/da8/dd0 39 2024-03-29T08:07:38.056 INFO:tasks.workunit.client.0.smithi037.stdout:6/757: mknod d0/d19/d34/c10c 0 2024-03-29T08:07:38.058 INFO:tasks.workunit.client.0.smithi037.stdout:7/952: dwrite dd/d21/d35/d42/ddb/de2/ff2 [0,4194304] 0 2024-03-29T08:07:38.058 INFO:tasks.workunit.client.0.smithi037.stdout:6/758: readlink d0/d19/d34/l8f 0 2024-03-29T08:07:38.064 INFO:tasks.workunit.client.0.smithi037.stdout:3/700: sync 2024-03-29T08:07:38.064 INFO:tasks.workunit.client.0.smithi037.stdout:1/793: sync 2024-03-29T08:07:38.065 INFO:tasks.workunit.client.0.smithi037.stdout:8/758: sync 2024-03-29T08:07:38.065 INFO:tasks.workunit.client.0.smithi037.stdout:2/712: sync 2024-03-29T08:07:38.066 INFO:tasks.workunit.client.0.smithi037.stdout:4/791: sync 2024-03-29T08:07:38.066 INFO:tasks.workunit.client.0.smithi037.stdout:5/949: sync 2024-03-29T08:07:38.076 INFO:tasks.workunit.client.0.smithi037.stdout:4/792: creat d3/d57/dec/ded/d1d/dad/dfb/f110 x:0 0 0 2024-03-29T08:07:38.079 INFO:tasks.workunit.client.0.smithi037.stdout:1/794: write d2/d6/dc/d18/f63 [2114599,46543] 0 2024-03-29T08:07:38.080 INFO:tasks.workunit.client.0.smithi037.stdout:3/701: symlink dd/d15/d18/d8e/le7 0 2024-03-29T08:07:38.087 INFO:tasks.workunit.client.0.smithi037.stdout:4/793: write d3/d4/d27/ffe [200432,34392] 0 2024-03-29T08:07:38.088 INFO:tasks.workunit.client.0.smithi037.stdout:7/953: truncate dd/d1d/d49/f5a 2036065 0 2024-03-29T08:07:38.094 INFO:tasks.workunit.client.0.smithi037.stdout:7/954: fdatasync dd/d1d/d34/f10b 0 2024-03-29T08:07:38.095 INFO:tasks.workunit.client.0.smithi037.stdout:5/950: mknod d8/d4d/d5c/c13f 0 2024-03-29T08:07:38.097 INFO:tasks.workunit.client.0.smithi037.stdout:6/759: dwrite d0/d5/d1a/f5d [0,4194304] 0 2024-03-29T08:07:38.099 INFO:tasks.workunit.client.0.smithi037.stdout:1/795: creat d2/d6/dc/d18/d1d/dfc/d109/d80/db0/f111 x:0 0 0 2024-03-29T08:07:38.101 INFO:tasks.workunit.client.0.smithi037.stdout:3/702: truncate dd/d46/d2d/d3a/fb6 1101089 0 2024-03-29T08:07:38.108 INFO:tasks.workunit.client.0.smithi037.stdout:0/724: dwrite d9/da/dd/d13/f5c [0,4194304] 0 2024-03-29T08:07:38.109 INFO:tasks.workunit.client.0.smithi037.stdout:6/760: dread - d0/d19/d34/d4e/df2/fbd zero size 2024-03-29T08:07:38.124 INFO:tasks.workunit.client.0.smithi037.stdout:5/951: write d8/d22/d37/fd9 [258823,122691] 0 2024-03-29T08:07:38.131 INFO:tasks.workunit.client.0.smithi037.stdout:8/759: rename d5/d10/d2d/d34/d50/d83/d46/d4e/l88 to d5/d10/d2d/d9c/de0/lfd 0 2024-03-29T08:07:38.131 INFO:tasks.workunit.client.0.smithi037.stdout:2/713: dread d0/d23/d44/dca/d2e/d5f/d6f/fa9 [0,4194304] 0 2024-03-29T08:07:38.135 INFO:tasks.workunit.client.0.smithi037.stdout:6/761: truncate d0/d5/d1e/f12 2954500 0 2024-03-29T08:07:38.135 INFO:tasks.workunit.client.0.smithi037.stdout:1/796: dread d2/f94 [0,4194304] 0 2024-03-29T08:07:38.136 INFO:tasks.workunit.client.0.smithi037.stdout:4/794: getdents d3/d57/dec/ded/d1d/db/d45/d93/ddb 0 2024-03-29T08:07:38.139 INFO:tasks.workunit.client.0.smithi037.stdout:6/762: write d0/d5/d1a/f5d [733358,47560] 0 2024-03-29T08:07:38.140 INFO:tasks.workunit.client.0.smithi037.stdout:7/955: rename dd/c51 to dd/d1d/d49/d81/d120/c14a 0 2024-03-29T08:07:38.142 INFO:tasks.workunit.client.0.smithi037.stdout:8/760: unlink d5/d10/d2d/d34/d50/d83/d46/d8a/d96/cd2 0 2024-03-29T08:07:38.143 INFO:tasks.workunit.client.0.smithi037.stdout:4/795: chown d3/d57/dec/ded/d1d/db/d45/d1e/d59/d60/d6f/db9/fe4 867375 1 2024-03-29T08:07:38.151 INFO:tasks.workunit.client.0.smithi037.stdout:5/952: mknod d8/d22/d37/d8d/db3/dcb/c140 0 2024-03-29T08:07:38.160 INFO:tasks.workunit.client.0.smithi037.stdout:7/956: dwrite dd/d21/d35/d60/d94/fb0 [0,4194304] 0 2024-03-29T08:07:38.162 INFO:tasks.workunit.client.0.smithi037.stdout:4/796: rmdir d3 39 2024-03-29T08:07:38.164 INFO:tasks.workunit.client.0.smithi037.stdout:8/761: rmdir d5/d10/d2d/d34/d50/d83/d46/d4e 39 2024-03-29T08:07:38.171 INFO:tasks.workunit.client.0.smithi037.stdout:3/703: rename dd/d15/c2a to dd/d46/d2d/d3a/d6c/db1/dc0/dda/d33/d97/ce8 0 2024-03-29T08:07:38.186 INFO:tasks.workunit.client.0.smithi037.stdout:4/797: symlink d3/d57/dec/ded/d1d/db/d45/d1e/d59/d7d/d97/l111 0 2024-03-29T08:07:38.187 INFO:tasks.workunit.client.0.smithi037.stdout:8/762: getdents d5/d78/dac 0 2024-03-29T08:07:38.192 INFO:tasks.workunit.client.0.smithi037.stdout:6/763: write d0/d20/f46 [562835,125766] 0 2024-03-29T08:07:38.200 INFO:tasks.workunit.client.0.smithi037.stdout:0/725: rename d9/da/dd/d20/d55/dac/fea to d9/da/dd/d13/ffc 0 2024-03-29T08:07:38.203 INFO:tasks.workunit.client.0.smithi037.stdout:1/797: write d2/d6/dc/d18/f29 [998719,46597] 0 2024-03-29T08:07:38.213 INFO:tasks.workunit.client.0.smithi037.stdout:0/726: mkdir d9/da/dd/d20/d38/d8a/dfd 0 2024-03-29T08:07:38.217 INFO:tasks.workunit.client.0.smithi037.stdout:8/763: creat d5/ffe x:0 0 0 2024-03-29T08:07:38.224 INFO:tasks.workunit.client.0.smithi037.stdout:8/764: write d5/d10/d2d/d9c/df8/f6b [4870563,4321] 0 2024-03-29T08:07:38.224 INFO:tasks.workunit.client.0.smithi037.stdout:5/953: dwrite d8/d34/fba [0,4194304] 0 2024-03-29T08:07:38.228 INFO:tasks.workunit.client.0.smithi037.stdout:1/798: unlink d2/d40/c52 0 2024-03-29T08:07:38.235 INFO:tasks.workunit.client.0.smithi037.stdout:7/957: dwrite dd/d21/d35/d60/f7f [0,4194304] 0 2024-03-29T08:07:38.240 INFO:tasks.workunit.client.0.smithi037.stdout:0/727: creat d9/da/dd/d54/ffe x:0 0 0 2024-03-29T08:07:38.249 INFO:tasks.workunit.client.0.smithi037.stdout:4/798: rename d3/d57/dec/ff2 to d3/d57/dec/ded/f112 0 2024-03-29T08:07:38.251 INFO:tasks.workunit.client.0.smithi037.stdout:1/799: creat d2/d6/dc/d18/d1d/dfc/d109/d80/db0/f112 x:0 0 0 2024-03-29T08:07:38.255 INFO:tasks.workunit.client.0.smithi037.stdout:3/704: dwrite dd/d46/d2d/d3a/d6c/db1/dc0/dda/f41 [0,4194304] 0 2024-03-29T08:07:38.262 INFO:tasks.workunit.client.0.smithi037.stdout:0/728: creat d9/da/dd/d20/d38/d53/fff x:0 0 0 2024-03-29T08:07:38.265 INFO:tasks.workunit.client.0.smithi037.stdout:5/954: dwrite d8/d22/d4f/f5e [0,4194304] 0 2024-03-29T08:07:38.267 INFO:tasks.workunit.client.0.smithi037.stdout:4/799: creat d3/d57/dec/f113 x:0 0 0 2024-03-29T08:07:38.270 INFO:tasks.workunit.client.0.smithi037.stdout:0/729: dread - d9/da/dd/d20/d55/fe3 zero size 2024-03-29T08:07:38.273 INFO:tasks.workunit.client.0.smithi037.stdout:6/764: write d0/d19/d34/f6b [1242336,110710] 0 2024-03-29T08:07:38.276 INFO:tasks.workunit.client.0.smithi037.stdout:8/765: dread d5/d10/d45/d4b/f6c [4194304,4194304] 0 2024-03-29T08:07:38.278 INFO:tasks.workunit.client.0.smithi037.stdout:5/955: dread - d8/d22/d37/dbc/f112 zero size 2024-03-29T08:07:38.279 INFO:tasks.workunit.client.0.smithi037.stdout:7/958: dwrite dd/d21/d70/fb2 [0,4194304] 0 2024-03-29T08:07:38.286 INFO:tasks.workunit.client.0.smithi037.stdout:1/800: truncate d2/d6/dc/d18/d1d/d21/f37 4247741 0 2024-03-29T08:07:38.289 INFO:tasks.workunit.client.0.smithi037.stdout:6/765: unlink d0/d19/d34/f6b 0 2024-03-29T08:07:38.290 INFO:tasks.workunit.client.0.smithi037.stdout:6/766: fdatasync d0/d6a/d7b/f9e 0 2024-03-29T08:07:38.294 INFO:tasks.workunit.client.0.smithi037.stdout:0/730: dread - d9/da/d1b/d22/d2c/f86 zero size 2024-03-29T08:07:38.297 INFO:tasks.workunit.client.0.smithi037.stdout:7/959: mkdir dd/d1d/d49/d81/dd8/d107/d14b 0 2024-03-29T08:07:38.298 INFO:tasks.workunit.client.0.smithi037.stdout:8/766: creat d5/d10/d2d/d9c/de0/fff x:0 0 0 2024-03-29T08:07:38.302 INFO:tasks.workunit.client.0.smithi037.stdout:4/800: fdatasync d3/d57/dec/ded/d1d/db/d45/d1e/d59/d79/d95/f9b 0 2024-03-29T08:07:38.305 INFO:tasks.workunit.client.0.smithi037.stdout:8/767: chown d5/d10/d2d/d9c/df8/dc8/fe3 2 1 2024-03-29T08:07:38.306 INFO:tasks.workunit.client.0.smithi037.stdout:1/801: mkdir d2/d6/dc/d7e/dd2/d113 0 2024-03-29T08:07:38.308 INFO:tasks.workunit.client.0.smithi037.stdout:4/801: write d3/d54/f5b [364334,37254] 0 2024-03-29T08:07:38.312 INFO:tasks.workunit.client.0.smithi037.stdout:8/768: read d5/d10/d2d/d9c/df8/d4c/f65 [657947,52437] 0 2024-03-29T08:07:38.320 INFO:tasks.workunit.client.0.smithi037.stdout:7/960: write dd/d21/d35/d3b/fb8 [1266084,38900] 0 2024-03-29T08:07:38.328 INFO:tasks.workunit.client.0.smithi037.stdout:8/769: rmdir d5/d10/d19/d3d 39 2024-03-29T08:07:38.334 INFO:tasks.workunit.client.0.smithi037.stdout:1/802: symlink d2/d6/dc/d58/l114 0 2024-03-29T08:07:38.335 INFO:tasks.workunit.client.0.smithi037.stdout:7/961: mknod dd/d21/d35/c14c 0 2024-03-29T08:07:38.339 INFO:tasks.workunit.client.0.smithi037.stdout:4/802: dwrite d3/d57/dec/ded/d1d/dad/dfb/f110 [0,4194304] 0 2024-03-29T08:07:38.342 INFO:tasks.workunit.client.0.smithi037.stdout:1/803: creat d2/d6/dc/d7e/f115 x:0 0 0 2024-03-29T08:07:38.351 INFO:tasks.workunit.client.0.smithi037.stdout:7/962: rename dd/d21/d35/d3b/dce/l102 to dd/dc3/l14d 0 2024-03-29T08:07:38.353 INFO:tasks.workunit.client.0.smithi037.stdout:7/963: fdatasync dd/d68/f78 0 2024-03-29T08:07:38.355 INFO:tasks.workunit.client.0.smithi037.stdout:5/956: dread d8/d22/d37/d8d/db3/dcb/deb/f76 [0,4194304] 0 2024-03-29T08:07:38.357 INFO:tasks.workunit.client.0.smithi037.stdout:4/803: creat d3/d57/dec/ded/d1d/db/d45/d1e/d5f/f114 x:0 0 0 2024-03-29T08:07:38.361 INFO:tasks.workunit.client.0.smithi037.stdout:1/804: fdatasync d2/d6/dc/d18/d1d/dfc/d109/d80/fc3 0 2024-03-29T08:07:38.374 INFO:tasks.workunit.client.0.smithi037.stdout:1/805: mknod d2/d6/d13/c116 0 2024-03-29T08:07:38.375 INFO:tasks.workunit.client.0.smithi037.stdout:5/957: write d8/d22/d4f/d70/f3f [8482847,14008] 0 2024-03-29T08:07:38.384 INFO:tasks.workunit.client.0.smithi037.stdout:0/731: write d9/da/f98 [681322,83561] 0 2024-03-29T08:07:38.384 INFO:tasks.workunit.client.0.smithi037.stdout:4/804: mknod d3/d57/dec/ded/d1d/db/d45/d1e/d59/d73/c115 0 2024-03-29T08:07:38.393 INFO:tasks.workunit.client.0.smithi037.stdout:8/770: dwrite d5/d10/d2d/d34/d50/fa2 [0,4194304] 0 2024-03-29T08:07:38.395 INFO:tasks.workunit.client.0.smithi037.stdout:3/705: dwrite dd/d46/d2d/d3a/d6c/db1/dc0/dda/d33/d75/d8f/f99 [0,4194304] 0 2024-03-29T08:07:38.400 INFO:tasks.workunit.client.0.smithi037.stdout:1/806: fsync d2/d6/dc/d18/d1d/dfc/d109/d80/db0/f111 0 2024-03-29T08:07:38.401 INFO:tasks.workunit.client.0.smithi037.stdout:0/732: chown d9/da/dd/d20/d38/d53/f62 992 1 2024-03-29T08:07:38.411 INFO:tasks.workunit.client.0.smithi037.stdout:7/964: dwrite dd/d21/d35/f6b [0,4194304] 0 2024-03-29T08:07:38.413 INFO:tasks.workunit.client.0.smithi037.stdout:3/706: readlink dd/d46/d2d/d3a/d6c/db1/dc0/dda/d33/d84/l86 0 2024-03-29T08:07:38.421 INFO:tasks.workunit.client.0.smithi037.stdout:7/965: chown dd/d1d/d49/d81/d84/f8a 869746039 1 2024-03-29T08:07:38.424 INFO:tasks.workunit.client.0.smithi037.stdout:0/733: creat d9/da/dd/d13/f100 x:0 0 0 2024-03-29T08:07:38.427 INFO:tasks.workunit.client.0.smithi037.stdout:1/807: getdents d2/d40/de8 0 2024-03-29T08:07:38.428 INFO:tasks.workunit.client.0.smithi037.stdout:1/808: stat d2/l3 0 2024-03-29T08:07:38.428 INFO:tasks.workunit.client.0.smithi037.stdout:0/734: readlink d9/da/lf0 0 2024-03-29T08:07:38.438 INFO:tasks.workunit.client.0.smithi037.stdout:7/966: write dd/d1d/d49/d81/d84/dca/ff6 [1914707,31728] 0 2024-03-29T08:07:38.439 INFO:tasks.workunit.client.0.smithi037.stdout:0/735: stat d9/da/dd/d20/d72/df7 0 2024-03-29T08:07:38.440 INFO:tasks.workunit.client.0.smithi037.stdout:8/771: truncate d5/d10/d2d/d34/d50/d83/f28 3918498 0 2024-03-29T08:07:38.442 INFO:tasks.workunit.client.0.smithi037.stdout:0/736: read d9/da/dd/d59/fb4 [78676,21807] 0 2024-03-29T08:07:38.444 INFO:tasks.workunit.client.0.smithi037.stdout:5/958: dread d8/d4d/d5c/fc3 [0,4194304] 0 2024-03-29T08:07:38.444 INFO:tasks.workunit.client.0.smithi037.stdout:1/809: chown d2/d6/dc/d18/d1d/dfc/d109/d80/ld3 26821430 1 2024-03-29T08:07:38.449 INFO:tasks.workunit.client.0.smithi037.stdout:1/810: read d2/d6/dc/d5d/ffe [726169,123828] 0 2024-03-29T08:07:38.460 INFO:tasks.workunit.client.0.smithi037.stdout:4/805: dread d3/d57/dec/ded/d1d/db/d45/d1e/f4a [0,4194304] 0 2024-03-29T08:07:38.461 INFO:tasks.workunit.client.0.smithi037.stdout:4/806: stat d3/d4/c70 0 2024-03-29T08:07:38.464 INFO:tasks.workunit.client.0.smithi037.stdout:4/807: write d3/d57/dec/ded/d1d/db/d45/d1e/f108 [574350,4555] 0 2024-03-29T08:07:38.482 INFO:tasks.workunit.client.0.smithi037.stdout:3/707: truncate dd/d46/d2d/d3a/d6c/db1/dc0/dda/d33/d83/fb9 1735139 0 2024-03-29T08:07:38.484 INFO:tasks.workunit.client.0.smithi037.stdout:7/967: creat dd/d21/d35/d42/d5c/d8c/f14e x:0 0 0 2024-03-29T08:07:38.490 INFO:tasks.workunit.client.0.smithi037.stdout:8/772: dread d5/d10/d2d/fc6 [0,4194304] 0 2024-03-29T08:07:38.493 INFO:tasks.workunit.client.0.smithi037.stdout:4/808: dread - d3/d57/dec/ded/d1d/db/d45/fae zero size 2024-03-29T08:07:38.496 INFO:tasks.workunit.client.0.smithi037.stdout:3/708: mknod dd/d5e/d7a/ce9 0 2024-03-29T08:07:38.498 INFO:tasks.workunit.client.0.smithi037.stdout:7/968: mkdir dd/d21/d35/d3b/dce/de0/d14f 0 2024-03-29T08:07:38.500 INFO:tasks.workunit.client.0.smithi037.stdout:5/959: mkdir d8/d22/d37/dbc/d141 0 2024-03-29T08:07:38.507 INFO:tasks.workunit.client.0.smithi037.stdout:3/709: chown dd/d46/d2d/f60 1289382 1 2024-03-29T08:07:38.510 INFO:tasks.workunit.client.0.smithi037.stdout:4/809: rename d3/d57/dec/ded/d89/c8b to d3/d57/dec/ded/d1d/db/d45/d1e/d59/d60/db7/c116 0 2024-03-29T08:07:38.511 INFO:tasks.workunit.client.0.smithi037.stdout:3/710: dread - dd/d46/d2d/d3a/d6c/db1/dc0/dda/d33/d97/dd6/fe1 zero size 2024-03-29T08:07:38.514 INFO:tasks.workunit.client.0.smithi037.stdout:5/960: mknod d8/d4d/d5c/d131/c142 0 2024-03-29T08:07:38.516 INFO:tasks.workunit.client.0.smithi037.stdout:7/969: mknod dd/d1d/d49/d81/dd8/c150 0 2024-03-29T08:07:38.519 INFO:tasks.workunit.client.0.smithi037.stdout:0/737: link d9/d74/d7d/lda d9/da/dd/d54/db5/dcb/l101 0 2024-03-29T08:07:38.526 INFO:tasks.workunit.client.0.smithi037.stdout:8/773: dread d5/d10/f35 [0,4194304] 0 2024-03-29T08:07:38.527 INFO:tasks.workunit.client.0.smithi037.stdout:7/970: chown dd/d21/d35/d3b/l10a 0 1 2024-03-29T08:07:38.528 INFO:tasks.workunit.client.0.smithi037.stdout:0/738: mknod d9/da/dd/d20/c102 0 2024-03-29T08:07:38.530 INFO:tasks.workunit.client.0.smithi037.stdout:8/774: chown d5/d10/d2d/d9c/df8/d4c/f65 498105560 1 2024-03-29T08:07:38.534 INFO:tasks.workunit.client.0.smithi037.stdout:0/739: dread - d9/da/dd/d13/ffb zero size 2024-03-29T08:07:38.536 INFO:tasks.workunit.client.0.smithi037.stdout:3/711: dwrite dd/d46/d3e/fb7 [0,4194304] 0 2024-03-29T08:07:38.542 INFO:tasks.workunit.client.0.smithi037.stdout:5/961: write d8/d4d/d5c/f94 [1305306,70440] 0 2024-03-29T08:07:38.543 INFO:tasks.workunit.client.0.smithi037.stdout:8/775: rename d5/d10/d19/f43 to d5/d10/d45/d4b/f100 0 2024-03-29T08:07:38.548 INFO:tasks.workunit.client.0.smithi037.stdout:1/811: dwrite d2/d6/dc/d58/fa2 [0,4194304] 0 2024-03-29T08:07:38.548 INFO:tasks.workunit.client.0.smithi037.stdout:0/740: dread d9/d74/f97 [0,4194304] 0 2024-03-29T08:07:38.551 INFO:tasks.workunit.client.0.smithi037.stdout:3/712: symlink dd/d46/d3e/lea 0 2024-03-29T08:07:38.551 INFO:tasks.workunit.client.0.smithi037.stdout:7/971: truncate dd/d21/d65/f135 738156 0 2024-03-29T08:07:38.556 INFO:tasks.workunit.client.0.smithi037.stdout:5/962: fsync d8/d18/d24/f74 0 2024-03-29T08:07:38.557 INFO:tasks.workunit.client.0.smithi037.stdout:8/776: unlink d5/d10/d19/d8d/cb4 0 2024-03-29T08:07:38.559 INFO:tasks.workunit.client.0.smithi037.stdout:7/972: unlink dd/d21/d85/l100 0 2024-03-29T08:07:38.562 INFO:tasks.workunit.client.0.smithi037.stdout:4/810: dwrite d3/d4/f46 [4194304,4194304] 0 2024-03-29T08:07:38.563 INFO:tasks.workunit.client.0.smithi037.stdout:3/713: dread dd/d46/d3e/d47/f95 [0,4194304] 0 2024-03-29T08:07:38.563 INFO:tasks.workunit.client.0.smithi037.stdout:2/714: sync 2024-03-29T08:07:38.563 INFO:tasks.workunit.client.0.smithi037.stdout:6/767: sync 2024-03-29T08:07:38.567 INFO:tasks.workunit.client.0.smithi037.stdout:8/777: dread - d5/d10/d2d/d34/d50/d83/d46/fbb zero size 2024-03-29T08:07:38.574 INFO:tasks.workunit.client.0.smithi037.stdout:2/715: write d0/d23/d44/dca/d2e/d5f/da5/fd3 [250861,129992] 0 2024-03-29T08:07:38.582 INFO:tasks.workunit.client.0.smithi037.stdout:5/963: mknod d8/d18/d24/c143 0 2024-03-29T08:07:38.587 INFO:tasks.workunit.client.0.smithi037.stdout:4/811: symlink d3/d57/dec/ded/d1d/db/d45/d93/l117 0 2024-03-29T08:07:38.590 INFO:tasks.workunit.client.0.smithi037.stdout:7/973: dread dd/d21/d35/d60/d94/fb0 [0,4194304] 0 2024-03-29T08:07:38.592 INFO:tasks.workunit.client.0.smithi037.stdout:0/741: rename d9/da/dd/d20/d55/d5f/dab/ff3 to d9/da/d1b/d22/f103 0 2024-03-29T08:07:38.598 INFO:tasks.workunit.client.0.smithi037.stdout:7/974: fdatasync dd/d21/d65/da2/df8/ff4 0 2024-03-29T08:07:38.604 INFO:tasks.workunit.client.0.smithi037.stdout:1/812: dwrite d2/d6/fe3 [0,4194304] 0 2024-03-29T08:07:38.605 INFO:tasks.workunit.client.0.smithi037.stdout:6/768: getdents d0/d5 0 2024-03-29T08:07:38.605 INFO:tasks.workunit.client.0.smithi037.stdout:3/714: dread dd/d15/f50 [0,4194304] 0 2024-03-29T08:07:38.611 INFO:tasks.workunit.client.0.smithi037.stdout:5/964: rename d8/d34/d125 to d8/d22/d37/dbc/d141/d144 0 2024-03-29T08:07:38.612 INFO:tasks.workunit.client.0.smithi037.stdout:2/716: dwrite d0/d23/d44/dca/d22/d55/d92/fcf [0,4194304] 0 2024-03-29T08:07:38.619 INFO:tasks.workunit.client.0.smithi037.stdout:2/717: chown d0/d23/d44/dca/d2e/d38/fb6 101495 1 2024-03-29T08:07:38.624 INFO:tasks.workunit.client.0.smithi037.stdout:4/812: symlink d3/d57/dec/ded/d1d/db/d45/d1e/d59/d7d/d97/l118 0 2024-03-29T08:07:38.627 INFO:tasks.workunit.client.0.smithi037.stdout:4/813: chown d3/d80/dcd/d8f 6 1 2024-03-29T08:07:38.633 INFO:tasks.workunit.client.0.smithi037.stdout:6/769: write d0/d5/d3f/f92 [2273522,29498] 0 2024-03-29T08:07:38.637 INFO:tasks.workunit.client.0.smithi037.stdout:6/770: write d0/d5/d1a/fa6 [482960,51678] 0 2024-03-29T08:07:38.637 INFO:tasks.workunit.client.0.smithi037.stdout:5/965: write d8/d22/d4f/d70/ff0 [79323,109605] 0 2024-03-29T08:07:38.639 INFO:tasks.workunit.client.0.smithi037.stdout:2/718: unlink d0/d23/d44/dca/d2e/d38/l77 0 2024-03-29T08:07:38.640 INFO:tasks.workunit.client.0.smithi037.stdout:8/778: getdents d5/d10/d2d/d9c/df8/d4c/d67/d70 0 2024-03-29T08:07:38.644 INFO:tasks.workunit.client.0.smithi037.stdout:1/813: symlink d2/d6/dc/l117 0 2024-03-29T08:07:38.645 INFO:tasks.workunit.client.0.smithi037.stdout:2/719: stat d0/d23/d44/dca/d2e/c32 0 2024-03-29T08:07:38.647 INFO:tasks.workunit.client.0.smithi037.stdout:6/771: dread d0/d6a/d7b/f9e [0,4194304] 0 2024-03-29T08:07:38.647 INFO:tasks.workunit.client.0.smithi037.stdout:4/814: write d3/d57/dec/ded/d1d/f3f [3976022,17826] 0 2024-03-29T08:07:38.649 INFO:tasks.workunit.client.0.smithi037.stdout:7/975: dwrite dd/d21/d35/d42/d5c/f5e [0,4194304] 0 2024-03-29T08:07:38.650 INFO:tasks.workunit.client.0.smithi037.stdout:2/720: fsync d0/d23/d44/dca/d22/fa0 0 2024-03-29T08:07:38.653 INFO:tasks.workunit.client.0.smithi037.stdout:6/772: dread - d0/d19/d34/d5a/dbf/dde/ff4 zero size 2024-03-29T08:07:38.653 INFO:tasks.workunit.client.0.smithi037.stdout:5/966: mknod d8/d22/d37/dbc/d85/df6/c145 0 2024-03-29T08:07:38.655 INFO:tasks.workunit.client.0.smithi037.stdout:3/715: link dd/d46/d2d/d3a/d6c/db1/dc0/dda/ddb/fae dd/d15/d18/d4b/feb 0 2024-03-29T08:07:38.655 INFO:tasks.workunit.client.0.smithi037.stdout:7/976: read dd/d21/d35/fc5 [2418892,50875] 0 2024-03-29T08:07:38.656 INFO:tasks.workunit.client.0.smithi037.stdout:5/967: readlink d8/d18/d24/la2 0 2024-03-29T08:07:38.741 INFO:tasks.workunit.client.0.smithi037.stdout:0/742: link d9/da/dd/d54/ccf d9/da/dd/d20/d55/d5f/dab/dbb/c104 0 2024-03-29T08:07:38.757 INFO:tasks.workunit.client.0.smithi037.stdout:4/815: unlink d3/d57/dec/ded/d1d/db/d45/d1e/l74 0 2024-03-29T08:07:38.769 INFO:tasks.workunit.client.0.smithi037.stdout:8/779: creat d5/d10/d19/d3d/f101 x:0 0 0 2024-03-29T08:07:38.779 INFO:tasks.workunit.client.0.smithi037.stdout:0/743: creat d9/da/dd/d20/d38/d53/f105 x:0 0 0 2024-03-29T08:07:38.782 INFO:tasks.workunit.client.0.smithi037.stdout:4/816: fsync d3/d57/dec/ded/d1d/db/d45/d93/d96/da0/db2/f101 0 2024-03-29T08:07:38.785 INFO:tasks.workunit.client.0.smithi037.stdout:4/817: chown d3/d57/dec/ded/d1d/db/d45/d1e/d59/d73/c115 714997 1 2024-03-29T08:07:38.801 INFO:tasks.workunit.client.0.smithi037.stdout:3/716: creat dd/d15/d18/d30/dcf/fec x:0 0 0 2024-03-29T08:07:38.805 INFO:tasks.workunit.client.0.smithi037.stdout:1/814: creat d2/d6/dc/d18/d1d/f118 x:0 0 0 2024-03-29T08:07:38.819 INFO:tasks.workunit.client.0.smithi037.stdout:5/968: link d8/l14 d8/d22/d37/d8d/db3/dee/l146 0 2024-03-29T08:07:38.827 INFO:tasks.workunit.client.0.smithi037.stdout:5/969: rmdir d8/d4d/d5c/d131 39 2024-03-29T08:07:38.840 INFO:tasks.workunit.client.0.smithi037.stdout:8/780: mkdir d5/d10/d2d/d9c/df8/d102 0 2024-03-29T08:07:38.841 INFO:tasks.workunit.client.0.smithi037.stdout:8/781: fsync d5/ffe 0 2024-03-29T08:07:38.857 INFO:tasks.workunit.client.0.smithi037.stdout:5/970: dread d8/d22/d37/d8d/db3/dcb/fd5 [0,4194304] 0 2024-03-29T08:07:38.859 INFO:tasks.workunit.client.0.smithi037.stdout:6/773: dwrite d0/d19/fc4 [0,4194304] 0 2024-03-29T08:07:38.861 INFO:tasks.workunit.client.0.smithi037.stdout:8/782: dwrite d5/d10/d2d/d9c/de0/fff [0,4194304] 0 2024-03-29T08:07:38.863 INFO:tasks.workunit.client.0.smithi037.stdout:5/971: chown d8/d22/d37/d8d/db3/dcb/deb/d61/l7c 1 1 2024-03-29T08:07:38.863 INFO:tasks.workunit.client.0.smithi037.stdout:8/783: chown d5/d10/d2d/d9c/da1/fe7 208144347 1 2024-03-29T08:07:38.864 INFO:tasks.workunit.client.0.smithi037.stdout:5/972: dread - d8/d18/d24/fad zero size 2024-03-29T08:07:38.867 INFO:tasks.workunit.client.0.smithi037.stdout:6/774: unlink d0/d5/le7 0 2024-03-29T08:07:38.873 INFO:tasks.workunit.client.0.smithi037.stdout:8/784: unlink d5/d10/l31 0 2024-03-29T08:07:38.884 INFO:tasks.workunit.client.0.smithi037.stdout:5/973: dread d8/d22/d4f/d70/f82 [0,4194304] 0 2024-03-29T08:07:38.888 INFO:tasks.workunit.client.0.smithi037.stdout:8/785: unlink d5/d10/d2d/d34/d50/d83/d46/d4e/fb0 0 2024-03-29T08:07:38.893 INFO:tasks.workunit.client.0.smithi037.stdout:7/977: rename dd/d21/d35/d3b/d5d/fc7 to dd/d21/d65/da2/f151 0 2024-03-29T08:07:38.894 INFO:tasks.workunit.client.0.smithi037.stdout:5/974: mkdir d8/d4d/d8a/d147 0 2024-03-29T08:07:38.894 INFO:tasks.workunit.client.0.smithi037.stdout:0/744: dwrite d9/da/dd/f5b [0,4194304] 0 2024-03-29T08:07:38.896 INFO:tasks.workunit.client.0.smithi037.stdout:4/818: dwrite d3/d57/dec/ded/d1d/db/d45/fce [0,4194304] 0 2024-03-29T08:07:38.910 INFO:tasks.workunit.client.0.smithi037.stdout:0/745: fdatasync d9/da/dd/d13/f36 0 2024-03-29T08:07:38.916 INFO:tasks.workunit.client.0.smithi037.stdout:4/819: symlink d3/d80/dcd/d8f/l119 0 2024-03-29T08:07:38.918 INFO:tasks.workunit.client.0.smithi037.stdout:8/786: truncate d5/f1e 3364021 0 2024-03-29T08:07:38.922 INFO:tasks.workunit.client.0.smithi037.stdout:0/746: dread - d9/da/dd/d20/d38/d53/f8f zero size 2024-03-29T08:07:38.923 INFO:tasks.workunit.client.0.smithi037.stdout:3/717: dwrite dd/d46/d2d/d3a/d6c/db1/dc0/dda/d33/f90 [0,4194304] 0 2024-03-29T08:07:38.925 INFO:tasks.workunit.client.0.smithi037.stdout:1/815: dwrite d2/d40/de8/daf/fd1 [0,4194304] 0 2024-03-29T08:07:38.944 INFO:tasks.workunit.client.0.smithi037.stdout:1/816: truncate d2/d6/dc/d5d/ff2 901921 0 2024-03-29T08:07:38.946 INFO:tasks.workunit.client.0.smithi037.stdout:3/718: symlink dd/d15/d18/d4b/da2/led 0 2024-03-29T08:07:38.946 INFO:tasks.workunit.client.0.smithi037.stdout:2/721: mkdir d0/d23/d44/df6 0 2024-03-29T08:07:38.949 INFO:tasks.workunit.client.0.smithi037.stdout:4/820: mkdir d3/d57/dec/ded/d1d/d11a 0 2024-03-29T08:07:38.954 INFO:tasks.workunit.client.0.smithi037.stdout:3/719: creat dd/d5e/d7a/fee x:0 0 0 2024-03-29T08:07:38.956 INFO:tasks.workunit.client.0.smithi037.stdout:5/975: rename d8/d4d/d5c/dbf to d8/d22/d37/d8d/db3/dcb/deb/d148 0 2024-03-29T08:07:38.957 INFO:tasks.workunit.client.0.smithi037.stdout:4/821: fsync d3/d57/dec/ded/d1d/db/d72/d98/ff3 0 2024-03-29T08:07:38.959 INFO:tasks.workunit.client.0.smithi037.stdout:4/822: readlink d3/d54/l7c 0 2024-03-29T08:07:38.965 INFO:tasks.workunit.client.0.smithi037.stdout:4/823: stat d3/d80/dcd/d8f/l103 0 2024-03-29T08:07:38.966 INFO:tasks.workunit.client.0.smithi037.stdout:6/775: write d0/d19/d34/f66 [1816821,96892] 0 2024-03-29T08:07:38.969 INFO:tasks.workunit.client.0.smithi037.stdout:5/976: fdatasync d8/d22/d4f/d70/f54 0 2024-03-29T08:07:38.979 INFO:tasks.workunit.client.0.smithi037.stdout:3/720: mknod dd/d5e/d7a/dd4/cef 0 2024-03-29T08:07:38.984 INFO:tasks.workunit.client.0.smithi037.stdout:7/978: truncate dd/dc3/f144 1276970 0 2024-03-29T08:07:38.993 INFO:tasks.workunit.client.0.smithi037.stdout:0/747: write d9/da/dd/d20/d38/d53/f62 [5032889,42238] 0 2024-03-29T08:07:38.995 INFO:tasks.workunit.client.0.smithi037.stdout:2/722: link d0/c54 d0/d23/d44/dca/d2e/d5f/d6f/d5e/cf7 0 2024-03-29T08:07:38.997 INFO:tasks.workunit.client.0.smithi037.stdout:3/721: mkdir dd/d46/d2d/d3a/d6c/db1/dc0/dda/ddb/df0 0 2024-03-29T08:07:39.007 INFO:tasks.workunit.client.0.smithi037.stdout:4/824: write d3/d57/dec/ded/d1d/db/f16 [486553,48785] 0 2024-03-29T08:07:39.009 INFO:tasks.workunit.client.0.smithi037.stdout:0/748: read d9/da/d1b/d22/d2c/f57 [1512547,55576] 0 2024-03-29T08:07:39.096 INFO:tasks.workunit.client.0.smithi037.stdout:2/723: creat d0/d23/d44/dca/d2e/d5f/da5/ff8 x:0 0 0 2024-03-29T08:07:39.102 INFO:tasks.workunit.client.0.smithi037.stdout:5/977: write d8/d22/d37/d8d/db3/dcb/deb/dfa/f13d [1300975,71122] 0 2024-03-29T08:07:39.111 INFO:tasks.workunit.client.0.smithi037.stdout:7/979: symlink dd/d68/d112/d127/d10e/l152 0 2024-03-29T08:07:39.111 INFO:tasks.workunit.client.0.smithi037.stdout:3/722: dread dd/d5e/faa [0,4194304] 0 2024-03-29T08:07:39.150 INFO:tasks.workunit.client.0.smithi037.stdout:0/749: dread d9/da/dd/d13/d83/f6d [0,4194304] 0 2024-03-29T08:07:39.153 INFO:tasks.workunit.client.0.smithi037.stdout:2/724: truncate d0/d23/d44/dca/f3d 87145 0 2024-03-29T08:07:39.154 INFO:tasks.workunit.client.0.smithi037.stdout:5/978: rmdir d8/d22/d37/dbc/d85 39 2024-03-29T08:07:39.157 INFO:tasks.workunit.client.0.smithi037.stdout:6/776: link d0/d5/d1e/f67 d0/d19/d34/d4e/df2/d105/f10d 0 2024-03-29T08:07:39.158 INFO:tasks.workunit.client.0.smithi037.stdout:3/723: dread - dd/d46/d2d/d3a/d6c/db1/fbc zero size 2024-03-29T08:07:39.160 INFO:tasks.workunit.client.0.smithi037.stdout:0/750: mkdir d9/da/dd/d54/db5/d106 0 2024-03-29T08:07:39.162 INFO:tasks.workunit.client.0.smithi037.stdout:2/725: symlink d0/d23/d44/lf9 0 2024-03-29T08:07:39.164 INFO:tasks.workunit.client.0.smithi037.stdout:3/724: dread dd/d5e/faa [0,4194304] 0 2024-03-29T08:07:39.165 INFO:tasks.workunit.client.0.smithi037.stdout:2/726: readlink d0/d23/d44/dca/d22/l90 0 2024-03-29T08:07:39.166 INFO:tasks.workunit.client.0.smithi037.stdout:0/751: creat d9/da/d1b/d22/d2c/f107 x:0 0 0 2024-03-29T08:07:39.170 INFO:tasks.workunit.client.0.smithi037.stdout:6/777: getdents d0/d5/d1f/da8/de3 0 2024-03-29T08:07:39.171 INFO:tasks.workunit.client.0.smithi037.stdout:0/752: chown d9/da/dd/d54/db5/df6/lbe 32393734 1 2024-03-29T08:07:39.172 INFO:tasks.workunit.client.0.smithi037.stdout:2/727: creat d0/d23/d44/df6/ffa x:0 0 0 2024-03-29T08:07:39.174 INFO:tasks.workunit.client.0.smithi037.stdout:6/778: stat d0/d19/d34/d5a/f39 0 2024-03-29T08:07:39.176 INFO:tasks.workunit.client.0.smithi037.stdout:0/753: unlink d9/da/dd/d20/d55/d5f/fe0 0 2024-03-29T08:07:39.177 INFO:tasks.workunit.client.0.smithi037.stdout:3/725: truncate dd/d46/d2d/d3a/d6c/db1/dc0/dda/d33/d97/dd6/fe1 926009 0 2024-03-29T08:07:39.179 INFO:tasks.workunit.client.0.smithi037.stdout:6/779: chown d0/d5/d1a/cb6 9706 1 2024-03-29T08:07:39.183 INFO:tasks.workunit.client.0.smithi037.stdout:0/754: mkdir d9/da/dd/d20/d55/d5f/d9f/d108 0 2024-03-29T08:07:39.185 INFO:tasks.workunit.client.0.smithi037.stdout:3/726: creat dd/d46/d2d/d3a/d6c/db1/dc0/dda/d33/d97/ff1 x:0 0 0 2024-03-29T08:07:39.190 INFO:tasks.workunit.client.0.smithi037.stdout:3/727: symlink dd/d46/d3e/d47/lf2 0 2024-03-29T08:07:39.195 INFO:tasks.workunit.client.0.smithi037.stdout:3/728: mknod dd/d46/d3e/cf3 0 2024-03-29T08:07:39.197 INFO:tasks.workunit.client.0.smithi037.stdout:0/755: link d9/da/d1b/c8b d9/da/dd/d20/d55/c109 0 2024-03-29T08:07:39.199 INFO:tasks.workunit.client.0.smithi037.stdout:2/728: dread d0/d23/d44/dca/d22/f47 [0,4194304] 0 2024-03-29T08:07:39.209 INFO:tasks.workunit.client.0.smithi037.stdout:6/780: dread d0/d5/d1e/d44/d71/f75 [0,4194304] 0 2024-03-29T08:07:39.215 INFO:tasks.workunit.client.0.smithi037.stdout:3/729: write dd/d15/f62 [2716060,111483] 0 2024-03-29T08:07:39.217 INFO:tasks.workunit.client.0.smithi037.stdout:6/781: chown d0/d5/d3f/f96 72792775 1 2024-03-29T08:07:39.225 INFO:tasks.workunit.client.0.smithi037.stdout:6/782: creat d0/d20/d95/f10e x:0 0 0 2024-03-29T08:07:39.226 INFO:tasks.workunit.client.0.smithi037.stdout:3/730: dread dd/d46/d2d/f60 [0,4194304] 0 2024-03-29T08:07:39.230 INFO:tasks.workunit.client.0.smithi037.stdout:0/756: dwrite d9/da/dd/d13/f31 [0,4194304] 0 2024-03-29T08:07:39.231 INFO:tasks.workunit.client.0.smithi037.stdout:3/731: write dd/d46/f71 [4438284,15289] 0 2024-03-29T08:07:39.231 INFO:tasks.workunit.client.0.smithi037.stdout:2/729: dread d0/d23/d44/dca/d2e/d5f/d6f/d5e/f74 [0,4194304] 0 2024-03-29T08:07:39.233 INFO:tasks.workunit.client.0.smithi037.stdout:6/783: rename d0/d19/d34/d4e/faf to d0/d20/d43/f10f 0 2024-03-29T08:07:39.234 INFO:tasks.workunit.client.0.smithi037.stdout:0/757: truncate f5 6707956 0 2024-03-29T08:07:39.235 INFO:tasks.workunit.client.0.smithi037.stdout:2/730: fsync d0/d23/d44/dca/d2e/d5f/da5/ff8 0 2024-03-29T08:07:39.236 INFO:tasks.workunit.client.0.smithi037.stdout:6/784: stat d0/d6a/d7b/d77 0 2024-03-29T08:07:39.242 INFO:tasks.workunit.client.0.smithi037.stdout:0/758: truncate d9/da/dd/d20/d72/fcd 722942 0 2024-03-29T08:07:39.244 INFO:tasks.workunit.client.0.smithi037.stdout:6/785: dread - d0/d20/fac zero size 2024-03-29T08:07:39.251 INFO:tasks.workunit.client.0.smithi037.stdout:0/759: chown d9/da/dd/d20/d55/dac/ffa 33 1 2024-03-29T08:07:39.251 INFO:tasks.workunit.client.0.smithi037.stdout:6/786: getdents d0/d5/d1f/da8 0 2024-03-29T08:07:39.257 INFO:tasks.workunit.client.0.smithi037.stdout:0/760: write d9/da/dd/d20/d38/d53/fff [911381,105167] 0 2024-03-29T08:07:39.259 INFO:tasks.workunit.client.0.smithi037.stdout:2/731: dread d0/d23/d44/dca/d2e/d5f/d6f/fa9 [0,4194304] 0 2024-03-29T08:07:39.265 INFO:tasks.workunit.client.0.smithi037.stdout:3/732: dwrite dd/d46/d2d/d3a/d6c/db1/dc0/dda/ddb/f77 [0,4194304] 0 2024-03-29T08:07:39.269 INFO:tasks.workunit.client.0.smithi037.stdout:3/733: fsync dd/d5e/f65 0 2024-03-29T08:07:39.289 INFO:tasks.workunit.client.0.smithi037.stdout:6/787: mkdir d0/d19/d34/d5a/dbf/dde/d110 0 2024-03-29T08:07:39.292 INFO:tasks.workunit.client.0.smithi037.stdout:2/732: write d0/d23/d44/dca/d7d/fdc [36868,2991] 0 2024-03-29T08:07:39.296 INFO:tasks.workunit.client.0.smithi037.stdout:0/761: rename d9/da/dd/d59/c9b to d9/da/dd/d20/d55/d5f/dab/dbb/c10a 0 2024-03-29T08:07:39.300 INFO:tasks.workunit.client.0.smithi037.stdout:3/734: creat dd/d46/d2d/d3a/d6c/db1/dc0/dda/ddb/ff4 x:0 0 0 2024-03-29T08:07:39.305 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:07:39 smithi148 ceph-mon[44992]: pgmap v51: 65 pgs: 65 active+clean; 3.6 GiB data, 12 GiB used, 525 GiB / 536 GiB avail; 102 MiB/s rd, 220 MiB/s wr, 503 op/s 2024-03-29T08:07:39.306 INFO:tasks.workunit.client.0.smithi037.stdout:3/735: readlink dd/d46/d2d/d3a/d6c/db1/dc0/dda/l29 0 2024-03-29T08:07:39.335 INFO:tasks.workunit.client.0.smithi037.stdout:0/762: symlink d9/da/d1b/d22/dee/l10b 0 2024-03-29T08:07:39.336 INFO:tasks.workunit.client.0.smithi037.stdout:3/736: truncate dd/fa9 744722 0 2024-03-29T08:07:39.337 INFO:tasks.workunit.client.0.smithi037.stdout:0/763: write d9/f27 [4019752,85488] 0 2024-03-29T08:07:39.340 INFO:tasks.workunit.client.0.smithi037.stdout:6/788: link d0/d19/d34/d4e/l98 d0/d5/d1f/l111 0 2024-03-29T08:07:39.341 INFO:tasks.workunit.client.0.smithi037.stdout:0/764: write d9/da/dd/d20/d55/d5f/d9f/fd3 [2726315,84402] 0 2024-03-29T08:07:39.342 INFO:tasks.workunit.client.0.smithi037.stdout:3/737: mkdir dd/d46/d2d/d3a/d6c/db1/dc0/dda/d33/d97/dd6/df5 0 2024-03-29T08:07:39.343 INFO:tasks.workunit.client.0.smithi037.stdout:0/765: readlink d9/da/dd/d13/l51 0 2024-03-29T08:07:39.349 INFO:tasks.workunit.client.0.smithi037.stdout:0/766: rmdir d9/da/dd/d20/d55/dac 39 2024-03-29T08:07:39.358 INFO:tasks.workunit.client.0.smithi037.stdout:0/767: symlink d9/da/dd/d54/d92/dd4/l10c 0 2024-03-29T08:07:39.359 INFO:tasks.workunit.client.0.smithi037.stdout:2/733: write d0/d23/d44/dca/d2e/d9c/df3/f72 [108912,82225] 0 2024-03-29T08:07:39.367 INFO:tasks.workunit.client.0.smithi037.stdout:2/734: mkdir d0/d23/d44/dca/d7d/dfb 0 2024-03-29T08:07:39.378 INFO:tasks.workunit.client.0.smithi037.stdout:0/768: truncate d9/da/dd/d59/f18 1282933 0 2024-03-29T08:07:39.378 INFO:tasks.workunit.client.0.smithi037.stdout:2/735: symlink d0/d23/d44/dca/d7d/lfc 0 2024-03-29T08:07:39.381 INFO:tasks.workunit.client.0.smithi037.stdout:6/789: dread d0/d20/f31 [0,4194304] 0 2024-03-29T08:07:39.398 INFO:tasks.workunit.client.0.smithi037.stdout:0/769: creat d9/f10d x:0 0 0 2024-03-29T08:07:39.399 INFO:tasks.workunit.client.0.smithi037.stdout:3/738: dwrite dd/d5e/f6d [0,4194304] 0 2024-03-29T08:07:39.410 INFO:tasks.workunit.client.0.smithi037.stdout:0/770: creat d9/da/dd/d13/d83/f10e x:0 0 0 2024-03-29T08:07:39.415 INFO:tasks.workunit.client.0.smithi037.stdout:7/980: creat dd/d1d/d109/d131/f153 x:0 0 0 2024-03-29T08:07:39.415 INFO:tasks.workunit.client.0.smithi037.stdout:3/739: readlink dd/d46/d2d/l58 0 2024-03-29T08:07:39.421 INFO:tasks.workunit.client.0.smithi037.stdout:6/790: dwrite d0/d5/d1a/fa6 [0,4194304] 0 2024-03-29T08:07:39.423 INFO:tasks.workunit.client.0.smithi037.stdout:7/981: dread - dd/d1d/d49/d81/d84/f106 zero size 2024-03-29T08:07:39.426 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:07:39 smithi037 ceph-mon[35162]: pgmap v51: 65 pgs: 65 active+clean; 3.6 GiB data, 12 GiB used, 525 GiB / 536 GiB avail; 102 MiB/s rd, 220 MiB/s wr, 503 op/s 2024-03-29T08:07:39.429 INFO:tasks.workunit.client.0.smithi037.stdout:2/736: dwrite d0/d23/d44/dca/d2e/da4/fe0 [0,4194304] 0 2024-03-29T08:07:39.429 INFO:tasks.workunit.client.0.smithi037.stdout:3/740: rename dd/d46/d3e/d47/cd5 to dd/d46/d3e/d47/cf6 0 2024-03-29T08:07:39.437 INFO:tasks.workunit.client.0.smithi037.stdout:6/791: rename d0/d19/d34/d5a/dbf/dde/ff4 to d0/d5/d1e/d84/f112 0 2024-03-29T08:07:39.454 INFO:tasks.workunit.client.0.smithi037.stdout:3/741: write fb [717262,111891] 0 2024-03-29T08:07:39.457 INFO:tasks.workunit.client.0.smithi037.stdout:7/982: dwrite dd/d21/f36 [0,4194304] 0 2024-03-29T08:07:39.467 INFO:tasks.workunit.client.0.smithi037.stdout:7/983: link dd/d21/d35/f6b dd/d21/d35/d3b/f154 0 2024-03-29T08:07:39.475 INFO:tasks.workunit.client.0.smithi037.stdout:7/984: fsync dd/d1d/d109/d118/f12b 0 2024-03-29T08:07:39.479 INFO:tasks.workunit.client.0.smithi037.stdout:0/771: dread d9/da/d1b/d22/d2c/f57 [0,4194304] 0 2024-03-29T08:07:39.483 INFO:tasks.workunit.client.0.smithi037.stdout:6/792: unlink d0/d6a/d7b/fff 0 2024-03-29T08:07:39.494 INFO:tasks.workunit.client.0.smithi037.stdout:6/793: mkdir d0/d5/d1f/da8/da3/d113 0 2024-03-29T08:07:39.502 INFO:tasks.workunit.client.0.smithi037.stdout:7/985: dread dd/d21/d35/d60/d94/fb5 [0,4194304] 0 2024-03-29T08:07:39.503 INFO:tasks.workunit.client.0.smithi037.stdout:6/794: link d0/d5/d1f/da8/dd0/cd3 d0/d19/d34/d5a/dbf/c114 0 2024-03-29T08:07:39.505 INFO:tasks.workunit.client.0.smithi037.stdout:7/986: symlink dd/d1d/d49/d81/l155 0 2024-03-29T08:07:39.506 INFO:tasks.workunit.client.0.smithi037.stdout:6/795: dread - d0/d6a/d7b/fd1 zero size 2024-03-29T08:07:39.507 INFO:tasks.workunit.client.0.smithi037.stdout:7/987: mkdir dd/d1d/d49/d81/d84/dca/d156 0 2024-03-29T08:07:39.508 INFO:tasks.workunit.client.0.smithi037.stdout:6/796: readlink d0/d5/d1e/l2e 0 2024-03-29T08:07:39.509 INFO:tasks.workunit.client.0.smithi037.stdout:7/988: fsync dd/d1d/f53 0 2024-03-29T08:07:39.509 INFO:tasks.workunit.client.0.smithi037.stdout:2/737: write d0/d23/d44/dca/d2e/d5f/d6f/f97 [1649227,126169] 0 2024-03-29T08:07:39.514 INFO:tasks.workunit.client.0.smithi037.stdout:6/797: creat d0/d5/d1e/d84/f115 x:0 0 0 2024-03-29T08:07:39.521 INFO:tasks.workunit.client.0.smithi037.stdout:2/738: link d0/d23/d44/dca/d2e/da4/le6 d0/d23/daf/dbf/lfd 0 2024-03-29T08:07:39.527 INFO:tasks.workunit.client.0.smithi037.stdout:6/798: link d0/d20/d95/f10e d0/d19/d34/d4e/d94/f116 0 2024-03-29T08:07:39.533 INFO:tasks.workunit.client.0.smithi037.stdout:2/739: creat d0/d23/d44/dca/d2e/d38/ffe x:0 0 0 2024-03-29T08:07:39.536 INFO:tasks.workunit.client.0.smithi037.stdout:3/742: dwrite dd/f1b [0,4194304] 0 2024-03-29T08:07:39.536 INFO:tasks.workunit.client.0.smithi037.stdout:7/989: dwrite dd/d21/d35/d60/f7f [0,4194304] 0 2024-03-29T08:07:39.536 INFO:tasks.workunit.client.0.smithi037.stdout:0/772: dwrite d9/da/dd/d13/f95 [0,4194304] 0 2024-03-29T08:07:39.539 INFO:tasks.workunit.client.0.smithi037.stdout:3/743: dread - dd/d46/d2d/fd7 zero size 2024-03-29T08:07:39.539 INFO:tasks.workunit.client.0.smithi037.stdout:6/799: creat d0/d19/d34/d4e/df2/d105/f117 x:0 0 0 2024-03-29T08:07:39.541 INFO:tasks.workunit.client.0.smithi037.stdout:7/990: truncate dd/d21/d85/fea 22436 0 2024-03-29T08:07:39.542 INFO:tasks.workunit.client.0.smithi037.stdout:2/740: dread - d0/d23/d44/dca/d2e/d9c/fc4 zero size 2024-03-29T08:07:39.544 INFO:tasks.workunit.client.0.smithi037.stdout:7/991: fsync dd/d21/d35/d60/f12f 0 2024-03-29T08:07:39.548 INFO:tasks.workunit.client.0.smithi037.stdout:7/992: mkdir dd/d1d/d109/d131/d157 0 2024-03-29T08:07:39.550 INFO:tasks.workunit.client.0.smithi037.stdout:7/993: dread - dd/fe3 zero size 2024-03-29T08:07:39.552 INFO:tasks.workunit.client.0.smithi037.stdout:6/800: fsync d0/d19/d42/d60/f9d 0 2024-03-29T08:07:39.558 INFO:tasks.workunit.client.0.smithi037.stdout:3/744: dread dd/f1b [0,4194304] 0 2024-03-29T08:07:39.561 INFO:tasks.workunit.client.0.smithi037.stdout:6/801: link d0/d20/l56 d0/d5/d1f/da8/de3/l118 0 2024-03-29T08:07:39.562 INFO:tasks.workunit.client.0.smithi037.stdout:3/745: mknod dd/d15/d18/d8e/cf7 0 2024-03-29T08:07:39.568 INFO:tasks.workunit.client.0.smithi037.stdout:7/994: write dd/f16 [7695183,110113] 0 2024-03-29T08:07:39.572 INFO:tasks.workunit.client.0.smithi037.stdout:0/773: truncate d9/da/d1b/fb0 3636746 0 2024-03-29T08:07:39.573 INFO:tasks.workunit.client.0.smithi037.stdout:6/802: truncate d0/d19/d34/d4e/d94/fed 682614 0 2024-03-29T08:07:39.577 INFO:tasks.workunit.client.0.smithi037.stdout:0/774: chown d9/da/dd/d13/f6f 3100661 1 2024-03-29T08:07:39.577 INFO:tasks.workunit.client.0.smithi037.stdout:2/741: dwrite d0/d23/d44/dca/d22/f47 [0,4194304] 0 2024-03-29T08:07:39.578 INFO:tasks.workunit.client.0.smithi037.stdout:3/746: getdents dd/d15/d18/d30/dcf 0 2024-03-29T08:07:39.586 INFO:tasks.workunit.client.0.smithi037.stdout:0/775: rename d9/d74/ldb to d9/da/dd/d59/l10f 0 2024-03-29T08:07:39.589 INFO:tasks.workunit.client.0.smithi037.stdout:3/747: getdents dd/d5e/d7a 0 2024-03-29T08:07:39.590 INFO:tasks.workunit.client.0.smithi037.stdout:0/776: dread - d9/da/fc6 zero size 2024-03-29T08:07:39.599 INFO:tasks.workunit.client.0.smithi037.stdout:7/995: truncate dd/d68/d112/d127/d12d/fe6 613617 0 2024-03-29T08:07:39.602 INFO:tasks.workunit.client.0.smithi037.stdout:3/748: rename dd/d15/d18/c4a to dd/d46/d3e/cf8 0 2024-03-29T08:07:39.607 INFO:tasks.workunit.client.0.smithi037.stdout:3/749: chown dd/d46/d2d/d3a/d6c 3 1 2024-03-29T08:07:39.610 INFO:tasks.workunit.client.0.smithi037.stdout:2/742: dread d0/d23/d44/dca/d2e/d5f/d6f/d5e/dc2/fc7 [0,4194304] 0 2024-03-29T08:07:39.611 INFO:tasks.workunit.client.0.smithi037.stdout:0/777: write d9/d74/d7d/d7e/fb9 [78653,101930] 0 2024-03-29T08:07:39.614 INFO:tasks.workunit.client.0.smithi037.stdout:3/750: creat dd/d15/d18/d4b/ff9 x:0 0 0 2024-03-29T08:07:39.615 INFO:tasks.workunit.client.0.smithi037.stdout:3/751: stat dd/l10 0 2024-03-29T08:07:39.619 INFO:tasks.workunit.client.0.smithi037.stdout:0/778: write d9/da/dd/d13/d7b/fa6 [5220583,121200] 0 2024-03-29T08:07:39.621 INFO:tasks.workunit.client.0.smithi037.stdout:6/803: dwrite d0/d5/d1e/f7f [0,4194304] 0 2024-03-29T08:07:39.625 INFO:tasks.workunit.client.0.smithi037.stdout:2/743: creat d0/d23/d44/dca/d2e/d9c/df3/fff x:0 0 0 2024-03-29T08:07:39.632 INFO:tasks.workunit.client.0.smithi037.stdout:6/804: fsync d0/d19/f9b 0 2024-03-29T08:07:39.638 INFO:tasks.workunit.client.0.smithi037.stdout:3/752: getdents dd/d46/d2d/d3a/d6c/db1/dc0/dda/d33/d84 0 2024-03-29T08:07:39.639 INFO:tasks.workunit.client.0.smithi037.stdout:6/805: write d0/d5/d1e/d44/d71/f75 [1980140,1799] 0 2024-03-29T08:07:39.643 INFO:tasks.workunit.client.0.smithi037.stdout:3/753: dread dd/d46/d2d/d3a/f81 [0,4194304] 0 2024-03-29T08:07:39.646 INFO:tasks.workunit.client.0.smithi037.stdout:6/806: creat d0/d19/d34/d4e/d94/f119 x:0 0 0 2024-03-29T08:07:39.648 INFO:tasks.workunit.client.0.smithi037.stdout:3/754: rmdir dd/d15/d18/d30 39 2024-03-29T08:07:39.651 INFO:tasks.workunit.client.0.smithi037.stdout:0/779: dwrite d9/d74/f97 [0,4194304] 0 2024-03-29T08:07:39.653 INFO:tasks.workunit.client.0.smithi037.stdout:2/744: dwrite d0/d23/d44/dca/d2e/d5f/d6f/d5e/dc2/feb [0,4194304] 0 2024-03-29T08:07:39.655 INFO:tasks.workunit.client.0.smithi037.stdout:6/807: symlink d0/d20/ddd/l11a 0 2024-03-29T08:07:39.668 INFO:tasks.workunit.client.0.smithi037.stdout:0/780: getdents d9/da/d1b/d22/dee 0 2024-03-29T08:07:39.676 INFO:tasks.workunit.client.0.smithi037.stdout:0/781: mkdir d9/da/d1b/d22/dee/d110 0 2024-03-29T08:07:39.679 INFO:tasks.workunit.client.0.smithi037.stdout:2/745: dwrite d0/d23/d44/f67 [0,4194304] 0 2024-03-29T08:07:39.682 INFO:tasks.workunit.client.0.smithi037.stdout:3/755: dwrite dd/d46/d3e/f7e [0,4194304] 0 2024-03-29T08:07:39.690 INFO:tasks.workunit.client.0.smithi037.stdout:3/756: fsync dd/d46/d3e/f7c 0 2024-03-29T08:07:39.691 INFO:tasks.workunit.client.0.smithi037.stdout:2/746: mknod d0/d23/d44/dca/d7d/dfb/c100 0 2024-03-29T08:07:39.693 INFO:tasks.workunit.client.0.smithi037.stdout:2/747: chown d0/d23/d44/c4c 1911838006 1 2024-03-29T08:07:39.697 INFO:tasks.workunit.client.0.smithi037.stdout:3/757: readlink dd/d46/lc2 0 2024-03-29T08:07:39.697 INFO:tasks.workunit.client.0.smithi037.stdout:6/808: dread d0/d5/d3f/f92 [4194304,4194304] 0 2024-03-29T08:07:39.697 INFO:tasks.workunit.client.0.smithi037.stdout:0/782: truncate d9/da/dd/d13/f31 3887470 0 2024-03-29T08:07:39.702 INFO:tasks.workunit.client.0.smithi037.stdout:2/748: stat d0/d23/d44/dca/l25 0 2024-03-29T08:07:39.702 INFO:tasks.workunit.client.0.smithi037.stdout:3/758: fsync dd/f24 0 2024-03-29T08:07:39.702 INFO:tasks.workunit.client.0.smithi037.stdout:6/809: stat d0/d19/d34/d4e/df2/f65 0 2024-03-29T08:07:39.710 INFO:tasks.workunit.client.0.smithi037.stdout:2/749: rename d0/d23/d44/dca/d2e/d5f/da5/fd3 to d0/d23/daf/dbf/f101 0 2024-03-29T08:07:39.713 INFO:tasks.workunit.client.0.smithi037.stdout:6/810: truncate d0/d5/f14 1861653 0 2024-03-29T08:07:39.714 INFO:tasks.workunit.client.0.smithi037.stdout:2/750: stat d0/d23/d44/dca/d7d/dfb 0 2024-03-29T08:07:39.717 INFO:tasks.workunit.client.0.smithi037.stdout:6/811: write d0/d5/d1f/f63 [4772321,130026] 0 2024-03-29T08:07:39.720 INFO:tasks.workunit.client.0.smithi037.stdout:0/783: dwrite d9/da/d1b/f29 [0,4194304] 0 2024-03-29T08:07:39.732 INFO:tasks.workunit.client.0.smithi037.stdout:0/784: rename d9/da/dd/d13/f5c to d9/da/dd/d54/db5/d106/f111 0 2024-03-29T08:07:39.739 INFO:tasks.workunit.client.0.smithi037.stdout:6/812: dread d0/d19/d34/d4e/df2/f65 [0,4194304] 0 2024-03-29T08:07:39.745 INFO:tasks.workunit.client.0.smithi037.stdout:2/751: dwrite d0/d23/d44/f67 [0,4194304] 0 2024-03-29T08:07:39.747 INFO:tasks.workunit.client.0.smithi037.stdout:6/813: chown d0/d5/d1f/da8/de3/l118 143189464 1 2024-03-29T08:07:39.750 INFO:tasks.workunit.client.0.smithi037.stdout:3/759: dwrite dd/f69 [0,4194304] 0 2024-03-29T08:07:39.753 INFO:tasks.workunit.client.0.smithi037.stdout:6/814: symlink d0/d19/d34/d5a/l11b 0 2024-03-29T08:07:39.755 INFO:tasks.workunit.client.0.smithi037.stdout:2/752: mknod d0/d23/daf/dbf/c102 0 2024-03-29T08:07:39.763 INFO:tasks.workunit.client.0.smithi037.stdout:2/753: rmdir d0/d23/d44/d62 39 2024-03-29T08:07:39.763 INFO:tasks.workunit.client.0.smithi037.stdout:0/785: dwrite d9/f10d [0,4194304] 0 2024-03-29T08:07:39.766 INFO:tasks.workunit.client.0.smithi037.stdout:3/760: getdents dd/d5e/d7a/dd4 0 2024-03-29T08:07:39.768 INFO:tasks.workunit.client.0.smithi037.stdout:3/761: chown dd/d5e/fbb 0 1 2024-03-29T08:07:39.774 INFO:tasks.workunit.client.0.smithi037.stdout:0/786: rmdir d9/da/dd/d54/d92/dd4 39 2024-03-29T08:07:39.774 INFO:tasks.workunit.client.0.smithi037.stdout:0/787: chown d9/d74/d7d/d7e/fef 291 1 2024-03-29T08:07:39.774 INFO:tasks.workunit.client.0.smithi037.stdout:2/754: rename d0/d23/d44/dca/f49 to d0/d23/d44/dca/d2e/da4/f103 0 2024-03-29T08:07:39.779 INFO:tasks.workunit.client.0.smithi037.stdout:0/788: truncate d9/da/dd/d13/ffb 23976 0 2024-03-29T08:07:39.782 INFO:tasks.workunit.client.0.smithi037.stdout:2/755: dread d0/d23/d44/dca/d2e/d5f/f8d [0,4194304] 0 2024-03-29T08:07:39.782 INFO:tasks.workunit.client.0.smithi037.stdout:0/789: chown d9/da/d1b/d22/d2c/ff2 522649 1 2024-03-29T08:07:39.784 INFO:tasks.workunit.client.0.smithi037.stdout:3/762: write dd/d15/d18/d4b/da2/f94 [20869,48037] 0 2024-03-29T08:07:39.786 INFO:tasks.workunit.client.0.smithi037.stdout:2/756: mknod d0/d53/c104 0 2024-03-29T08:07:39.790 INFO:tasks.workunit.client.0.smithi037.stdout:3/763: truncate dd/d46/fe0 199294 0 2024-03-29T08:07:39.790 INFO:tasks.workunit.client.0.smithi037.stdout:0/790: symlink d9/da/dd/d20/d55/d5f/dab/l112 0 2024-03-29T08:07:39.797 INFO:tasks.workunit.client.0.smithi037.stdout:3/764: rmdir dd/d46/d2d/d3a/d6c/db1 39 2024-03-29T08:07:39.798 INFO:tasks.workunit.client.0.smithi037.stdout:2/757: getdents d0/d23/d44/dca/d2e/d5f/da5 0 2024-03-29T08:07:39.808 INFO:tasks.workunit.client.0.smithi037.stdout:2/758: unlink d0/d23/d44/dca/d22/c35 0 2024-03-29T08:07:39.810 INFO:tasks.workunit.client.0.smithi037.stdout:3/765: creat dd/d46/d2d/d3a/d6c/db1/dc0/dda/d33/d84/ffa x:0 0 0 2024-03-29T08:07:39.812 INFO:tasks.workunit.client.0.smithi037.stdout:0/791: dwrite d9/f27 [0,4194304] 0 2024-03-29T08:07:39.814 INFO:tasks.workunit.client.0.smithi037.stdout:6/815: dread d0/d19/d42/d60/f9d [0,4194304] 0 2024-03-29T08:07:39.817 INFO:tasks.workunit.client.0.smithi037.stdout:2/759: creat d0/d23/d44/f105 x:0 0 0 2024-03-29T08:07:39.823 INFO:tasks.workunit.client.0.smithi037.stdout:6/816: truncate d0/fcf 2794021 0 2024-03-29T08:07:39.826 INFO:tasks.workunit.client.0.smithi037.stdout:3/766: mkdir dd/d46/d2d/d3a/d6c/db1/dc0/dda/d33/dfb 0 2024-03-29T08:07:39.827 INFO:tasks.workunit.client.0.smithi037.stdout:0/792: creat d9/da/dd/d20/d38/d8a/dfd/f113 x:0 0 0 2024-03-29T08:07:39.829 INFO:tasks.workunit.client.0.smithi037.stdout:2/760: mkdir d0/d23/d44/d106 0 2024-03-29T08:07:39.830 INFO:tasks.workunit.client.0.smithi037.stdout:0/793: readlink d9/d74/d7d/lb3 0 2024-03-29T08:07:39.834 INFO:tasks.workunit.client.0.smithi037.stdout:2/761: chown d0/d23/d44/dca/d2e/f91 21 1 2024-03-29T08:07:39.834 INFO:tasks.workunit.client.0.smithi037.stdout:3/767: symlink dd/d15/lfc 0 2024-03-29T08:07:39.839 INFO:tasks.workunit.client.0.smithi037.stdout:3/768: write dd/d46/d2d/fcb [700659,63304] 0 2024-03-29T08:07:39.842 INFO:tasks.workunit.client.0.smithi037.stdout:2/762: mkdir d0/d23/d44/dca/d2e/d9c/d107 0 2024-03-29T08:07:39.856 INFO:tasks.workunit.client.0.smithi037.stdout:0/794: dwrite d9/da/dd/f99 [0,4194304] 0 2024-03-29T08:07:39.856 INFO:tasks.workunit.client.0.smithi037.stdout:2/763: rename d0/d23/d44/dca/d2e/d5f/d6f/la3 to d0/d23/d44/dca/d22/d55/l108 0 2024-03-29T08:07:39.858 INFO:tasks.workunit.client.0.smithi037.stdout:3/769: truncate dd/d46/d2d/d3a/d6c/db1/dc0/dda/d33/d83/fb9 2242080 0 2024-03-29T08:07:39.859 INFO:tasks.workunit.client.0.smithi037.stdout:6/817: dwrite d0/d5/d1f/da8/fc5 [0,4194304] 0 2024-03-29T08:07:39.872 INFO:tasks.workunit.client.0.smithi037.stdout:3/770: creat dd/ffd x:0 0 0 2024-03-29T08:07:39.875 INFO:tasks.workunit.client.0.smithi037.stdout:3/771: chown dd/f1b 49934 1 2024-03-29T08:07:39.877 INFO:tasks.workunit.client.0.smithi037.stdout:2/764: write d0/d23/d44/dca/d22/f3c [2258303,12794] 0 2024-03-29T08:07:39.880 INFO:tasks.workunit.client.0.smithi037.stdout:0/795: dread d9/da/dd/f99 [0,4194304] 0 2024-03-29T08:07:39.883 INFO:tasks.workunit.client.0.smithi037.stdout:3/772: mknod dd/d46/d2d/d3a/d6c/db1/dc0/dda/d33/d83/cfe 0 2024-03-29T08:07:39.887 INFO:tasks.workunit.client.0.smithi037.stdout:8/787: sync 2024-03-29T08:07:39.887 INFO:tasks.workunit.client.0.smithi037.stdout:6/818: dwrite d0/d5/d1f/da8/fc5 [0,4194304] 0 2024-03-29T08:07:39.890 INFO:tasks.workunit.client.0.smithi037.stdout:3/773: read - dd/d46/d2d/d3a/d6c/db1/dc0/dda/ddb/ff4 zero size 2024-03-29T08:07:39.892 INFO:tasks.workunit.client.0.smithi037.stdout:6/819: chown d0/d5/d3f/f17 41960 1 2024-03-29T08:07:39.895 INFO:tasks.workunit.client.0.smithi037.stdout:0/796: rename d9/da/dd/d20/d55/d5f/dab/dbb/c104 to d9/d74/de6/c114 0 2024-03-29T08:07:39.896 INFO:tasks.workunit.client.0.smithi037.stdout:2/765: truncate d0/d23/d44/dca/d22/d4e/fbb 659559 0 2024-03-29T08:07:39.897 INFO:tasks.workunit.client.0.smithi037.stdout:3/774: dread - dd/d15/d18/fb2 zero size 2024-03-29T08:07:39.900 INFO:tasks.workunit.client.0.smithi037.stdout:3/775: chown dd/d46/fe0 2 1 2024-03-29T08:07:39.901 INFO:tasks.workunit.client.0.smithi037.stdout:8/788: creat d5/d10/d2d/df5/f103 x:0 0 0 2024-03-29T08:07:39.906 INFO:tasks.workunit.client.0.smithi037.stdout:6/820: link d0/d5/d3f/f92 d0/d6a/dd6/f11c 0 2024-03-29T08:07:39.906 INFO:tasks.workunit.client.0.smithi037.stdout:2/766: mknod d0/c109 0 2024-03-29T08:07:39.911 INFO:tasks.workunit.client.0.smithi037.stdout:8/789: read d5/d10/d2d/d9c/df8/d4c/f8f [2906493,114808] 0 2024-03-29T08:07:39.912 INFO:tasks.workunit.client.0.smithi037.stdout:6/821: truncate d0/d5/f2a 2008867 0 2024-03-29T08:07:39.917 INFO:tasks.workunit.client.0.smithi037.stdout:3/776: rename dd/d46/d2d/d3a/d6c/db1/dc0/dda/d33/d97/ce8 to dd/d15/d18/d4b/da2/cff 0 2024-03-29T08:07:39.922 INFO:tasks.workunit.client.0.smithi037.stdout:6/822: creat d0/d20/d95/f11d x:0 0 0 2024-03-29T08:07:39.923 INFO:tasks.workunit.client.0.smithi037.stdout:2/767: write d0/f40 [4661781,125794] 0 2024-03-29T08:07:39.923 INFO:tasks.workunit.client.0.smithi037.stdout:6/823: rmdir d0/d19 39 2024-03-29T08:07:39.924 INFO:tasks.workunit.client.0.smithi037.stdout:0/797: dwrite d9/da/dd/d20/d55/f9c [0,4194304] 0 2024-03-29T08:07:39.928 INFO:tasks.workunit.client.0.smithi037.stdout:3/777: creat dd/d15/d18/d30/f100 x:0 0 0 2024-03-29T08:07:39.934 INFO:tasks.workunit.client.0.smithi037.stdout:0/798: mkdir d9/d74/d7d/d7e/d115 0 2024-03-29T08:07:39.941 INFO:tasks.workunit.client.0.smithi037.stdout:2/768: getdents d0/d53/d89 0 2024-03-29T08:07:39.942 INFO:tasks.workunit.client.0.smithi037.stdout:6/824: write d0/d19/d42/d60/f7d [1103964,25394] 0 2024-03-29T08:07:39.945 INFO:tasks.workunit.client.0.smithi037.stdout:0/799: creat d9/da/d1b/f116 x:0 0 0 2024-03-29T08:07:39.947 INFO:tasks.workunit.client.0.smithi037.stdout:3/778: getdents dd/d46/d2d/d3a/d6c/db1 0 2024-03-29T08:07:39.949 INFO:tasks.workunit.client.0.smithi037.stdout:6/825: creat d0/d19/d42/d60/d62/f11e x:0 0 0 2024-03-29T08:07:39.952 INFO:tasks.workunit.client.0.smithi037.stdout:0/800: symlink d9/da/dd/d54/db5/dcb/l117 0 2024-03-29T08:07:39.953 INFO:tasks.workunit.client.0.smithi037.stdout:8/790: dread d5/d10/d2d/fcb [0,4194304] 0 2024-03-29T08:07:39.955 INFO:tasks.workunit.client.0.smithi037.stdout:6/826: mkdir d0/d6a/d7b/d11f 0 2024-03-29T08:07:39.960 INFO:tasks.workunit.client.0.smithi037.stdout:0/801: creat d9/da/dd/d54/db5/dcb/f118 x:0 0 0 2024-03-29T08:07:39.961 INFO:tasks.workunit.client.0.smithi037.stdout:6/827: mknod d0/d5/d1e/d84/c120 0 2024-03-29T08:07:39.961 INFO:tasks.workunit.client.0.smithi037.stdout:5/979: sync 2024-03-29T08:07:39.961 INFO:tasks.workunit.client.0.smithi037.stdout:4/825: sync 2024-03-29T08:07:39.961 INFO:tasks.workunit.client.0.smithi037.stdout:7/996: sync 2024-03-29T08:07:39.962 INFO:tasks.workunit.client.0.smithi037.stdout:2/769: dread d0/d23/d44/dca/d2e/d5f/d6f/f97 [0,4194304] 0 2024-03-29T08:07:39.962 INFO:tasks.workunit.client.0.smithi037.stdout:1/817: sync 2024-03-29T08:07:39.963 INFO:tasks.workunit.client.0.smithi037.stdout:3/779: truncate dd/d46/d2d/d3a/d6c/db1/dc0/dda/d33/d84/fb8 403444 0 2024-03-29T08:07:39.964 INFO:tasks.workunit.client.0.smithi037.stdout:8/791: getdents d5/d10/d2d/d9c/df8/d102 0 2024-03-29T08:07:39.967 INFO:tasks.workunit.client.0.smithi037.stdout:0/802: write d9/da/dd/d20/d38/d53/fff [309100,28361] 0 2024-03-29T08:07:39.968 INFO:tasks.workunit.client.0.smithi037.stdout:6/828: write d0/d6a/d7b/fa1 [2602474,87823] 0 2024-03-29T08:07:39.969 INFO:tasks.workunit.client.0.smithi037.stdout:1/818: mknod d2/d6/dc/d18/dbd/c119 0 2024-03-29T08:07:39.974 INFO:tasks.workunit.client.0.smithi037.stdout:5/980: getdents d8/d18 0 2024-03-29T08:07:39.976 INFO:tasks.workunit.client.0.smithi037.stdout:7/997: rename dd/d21/d35/d42/f8e to dd/d21/d35/d3b/dce/f158 0 2024-03-29T08:07:39.977 INFO:tasks.workunit.client.0.smithi037.stdout:8/792: dread - d5/d10/d2d/d34/d50/d83/d46/d4e/fb6 zero size 2024-03-29T08:07:39.978 INFO:tasks.workunit.client.0.smithi037.stdout:4/826: readlink d3/d57/dec/ded/d1d/db/d45/d1e/d59/db5/le6 0 2024-03-29T08:07:39.981 INFO:tasks.workunit.client.0.smithi037.stdout:2/770: rmdir d0/d23/d44/dca/d2e/d5f/d6f/d5e/dc2 39 2024-03-29T08:07:39.982 INFO:tasks.workunit.client.0.smithi037.stdout:8/793: fdatasync d5/d10/d2d/d34/d50/d83/d46/d8a/d96/fda 0 2024-03-29T08:07:39.982 INFO:tasks.workunit.client.0.smithi037.stdout:6/829: fsync d0/d5/d1e/fc2 0 2024-03-29T08:07:39.987 INFO:tasks.workunit.client.0.smithi037.stdout:5/981: creat d8/d22/d4f/f149 x:0 0 0 2024-03-29T08:07:39.989 INFO:tasks.workunit.client.0.smithi037.stdout:0/803: rename d9/da/dd/d20/d38/d8a/cb8 to d9/da/dd/d20/d55/d5f/dab/c119 0 2024-03-29T08:07:39.989 INFO:tasks.workunit.client.0.smithi037.stdout:5/982: chown d8/d4d/ddf/l138 1705122 1 2024-03-29T08:07:39.992 INFO:tasks.workunit.client.0.smithi037.stdout:0/804: read d9/da/dd/d13/f6f [763167,75852] 0 2024-03-29T08:07:39.996 INFO:tasks.workunit.client.0.smithi037.stdout:1/819: mknod d2/d6/dc/d18/d22/d4f/c11a 0 2024-03-29T08:07:39.998 INFO:tasks.workunit.client.0.smithi037.stdout:6/830: unlink d0/d20/ddd/ce6 0 2024-03-29T08:07:40.000 INFO:tasks.workunit.client.0.smithi037.stdout:2/771: mkdir d0/d23/d44/dca/d22/d55/d92/d10a 0 2024-03-29T08:07:40.005 INFO:tasks.workunit.client.0.smithi037.stdout:5/983: unlink d8/d22/d37/d8d/db3/dcb/f12f 0 2024-03-29T08:07:40.006 INFO:tasks.workunit.client.0.smithi037.stdout:8/794: write d5/d10/d19/d8d/f90 [1712840,15347] 0 2024-03-29T08:07:40.010 INFO:tasks.workunit.client.0.smithi037.stdout:2/772: symlink d0/d23/d44/dca/d2e/d9c/df3/d78/l10b 0 2024-03-29T08:07:40.011 INFO:tasks.workunit.client.0.smithi037.stdout:8/795: stat d5/d78/dac/fe8 0 2024-03-29T08:07:40.015 INFO:tasks.workunit.client.0.smithi037.stdout:6/831: write d0/d5/d3f/f96 [377064,59913] 0 2024-03-29T08:07:40.015 INFO:tasks.workunit.client.0.smithi037.stdout:0/805: write d9/da/dd/d20/f4e [2688644,74745] 0 2024-03-29T08:07:40.016 INFO:tasks.workunit.client.0.smithi037.stdout:1/820: truncate d2/d6/dc/d18/d1d/dfc/d109/d80/fc3 979757 0 2024-03-29T08:07:40.019 INFO:tasks.workunit.client.0.smithi037.stdout:5/984: chown d8/d22/d37/d8d/db3/dcb/deb/d88/l98 63 1 2024-03-29T08:07:40.021 INFO:tasks.workunit.client.0.smithi037.stdout:4/827: dwrite d3/d57/dec/ded/d1d/f64 [0,4194304] 0 2024-03-29T08:07:40.023 INFO:tasks.workunit.client.0.smithi037.stdout:8/796: creat d5/d10/d2d/f104 x:0 0 0 2024-03-29T08:07:40.030 INFO:tasks.workunit.client.0.smithi037.stdout:6/832: symlink d0/d5/d1f/da8/da3/d113/l121 0 2024-03-29T08:07:40.035 INFO:tasks.workunit.client.0.smithi037.stdout:4/828: creat d3/d57/dec/ded/d1d/db/d45/d93/d96/da0/dc9/f11b x:0 0 0 2024-03-29T08:07:40.039 INFO:tasks.workunit.client.0.smithi037.stdout:8/797: getdents d5/d10/d2d/d9c/da1/dbc 0 2024-03-29T08:07:40.039 INFO:tasks.workunit.client.0.smithi037.stdout:1/821: getdents d2/d6/dc/d18 0 2024-03-29T08:07:40.039 INFO:tasks.workunit.client.0.smithi037.stdout:2/773: truncate d0/d23/d44/dca/d22/f48 589398 0 2024-03-29T08:07:40.046 INFO:tasks.workunit.client.0.smithi037.stdout:5/985: rename d8/d22/d37/d8d/db3/dcb/deb/f122 to d8/d22/d37/d8d/db3/dcb/f14a 0 2024-03-29T08:07:40.052 INFO:tasks.workunit.client.0.smithi037.stdout:2/774: fsync d0/d23/d44/dca/d22/d55/d92/ff2 0 2024-03-29T08:07:40.054 INFO:tasks.workunit.client.0.smithi037.stdout:6/833: dread d0/f33 [4194304,4194304] 0 2024-03-29T08:07:40.054 INFO:tasks.workunit.client.0.smithi037.stdout:5/986: chown d8/d22/d37/d8d/db3/dcb/deb/d61/l7c 42 1 2024-03-29T08:07:40.060 INFO:tasks.workunit.client.0.smithi037.stdout:1/822: truncate d2/d6/dc/d18/d1d/d21/d99/f4e 292666 0 2024-03-29T08:07:40.060 INFO:tasks.workunit.client.0.smithi037.stdout:8/798: write d5/d10/d45/d4b/fa4 [2249422,17717] 0 2024-03-29T08:07:40.062 INFO:tasks.workunit.client.0.smithi037.stdout:6/834: unlink d0/d5/d1a/fd2 0 2024-03-29T08:07:40.063 INFO:tasks.workunit.client.0.smithi037.stdout:0/806: dread f5 [0,4194304] 0 2024-03-29T08:07:40.067 INFO:tasks.workunit.client.0.smithi037.stdout:4/829: rename d3/d57/dec/ded/d1d/db/d45/d1e/d59/d60/f9c to d3/d80/dcd/d8f/f11c 0 2024-03-29T08:07:40.068 INFO:tasks.workunit.client.0.smithi037.stdout:2/775: symlink d0/d23/d44/dca/d2e/d9c/d107/l10c 0 2024-03-29T08:07:40.070 INFO:tasks.workunit.client.0.smithi037.stdout:8/799: write d5/d10/d2d/f3e [4812314,3143] 0 2024-03-29T08:07:40.072 INFO:tasks.workunit.client.0.smithi037.stdout:1/823: write d2/d6/dc/d18/d1d/dfc/d109/d80/db0/f111 [973537,28756] 0 2024-03-29T08:07:40.075 INFO:tasks.workunit.client.0.smithi037.stdout:0/807: write d9/da/dd/d20/d55/fe3 [442881,126910] 0 2024-03-29T08:07:40.076 INFO:tasks.workunit.client.0.smithi037.stdout:5/987: link d8/d22/d37/d8d/db3/dcb/deb/f76 d8/d4d/ddf/d11c/f14b 0 2024-03-29T08:07:40.077 INFO:tasks.workunit.client.0.smithi037.stdout:1/824: write d2/f3a [781711,86794] 0 2024-03-29T08:07:40.077 INFO:tasks.workunit.client.0.smithi037.stdout:6/835: fsync d0/d19/d34/d4e/fb4 0 2024-03-29T08:07:40.084 INFO:tasks.workunit.client.0.smithi037.stdout:1/825: chown d2/d6/dc/d18/d22/d4f/d86 1 1 2024-03-29T08:07:40.085 INFO:tasks.workunit.client.0.smithi037.stdout:6/836: write d0/d5/d3f/fb1 [204227,3990] 0 2024-03-29T08:07:40.088 INFO:tasks.workunit.client.0.smithi037.stdout:8/800: getdents d5/d10/d2d/d9c/df8/d4c 0 2024-03-29T08:07:40.091 INFO:tasks.workunit.client.0.smithi037.stdout:0/808: creat d9/d74/de6/f11a x:0 0 0 2024-03-29T08:07:40.098 INFO:tasks.workunit.client.0.smithi037.stdout:5/988: write d8/f2b [4104039,95023] 0 2024-03-29T08:07:40.098 INFO:tasks.workunit.client.0.smithi037.stdout:6/837: dread - d0/d19/d34/d5a/fd7 zero size 2024-03-29T08:07:40.100 INFO:tasks.workunit.client.0.smithi037.stdout:1/826: read - d2/d6/fd7 zero size 2024-03-29T08:07:40.101 INFO:tasks.workunit.client.0.smithi037.stdout:8/801: write d5/d10/d2d/d34/d50/f68 [157439,61425] 0 2024-03-29T08:07:40.106 INFO:tasks.workunit.client.0.smithi037.stdout:2/776: dwrite d0/f8c [4194304,4194304] 0 2024-03-29T08:07:40.112 INFO:tasks.workunit.client.0.smithi037.stdout:0/809: write d9/da/dd/d13/f5e [914919,53428] 0 2024-03-29T08:07:40.114 INFO:tasks.workunit.client.0.smithi037.stdout:1/827: mkdir d2/d6/dc/d58/d67/d11b 0 2024-03-29T08:07:40.115 INFO:tasks.workunit.client.0.smithi037.stdout:5/989: write d8/d18/d24/fad [486792,53383] 0 2024-03-29T08:07:40.117 INFO:tasks.workunit.client.0.smithi037.stdout:5/990: chown d8/d4d/d5c 376435 1 2024-03-29T08:07:40.117 INFO:tasks.workunit.client.0.smithi037.stdout:6/838: symlink d0/d19/d34/d4e/df2/l122 0 2024-03-29T08:07:40.121 INFO:tasks.workunit.client.0.smithi037.stdout:0/810: fdatasync d9/da/dd/d13/f96 0 2024-03-29T08:07:40.121 INFO:tasks.workunit.client.0.smithi037.stdout:1/828: stat d2/d6/dc/d18/d1d/dfc/d109/d80/db0 0 2024-03-29T08:07:40.121 INFO:tasks.workunit.client.0.smithi037.stdout:2/777: creat d0/d23/d44/dca/d22/d55/d92/d10a/f10d x:0 0 0 2024-03-29T08:07:40.130 INFO:tasks.workunit.client.0.smithi037.stdout:2/778: stat d0/d23/d44/dca/d2e/d5f/cd0 0 2024-03-29T08:07:40.132 INFO:tasks.workunit.client.0.smithi037.stdout:0/811: creat d9/da/f11b x:0 0 0 2024-03-29T08:07:40.137 INFO:tasks.workunit.client.0.smithi037.stdout:2/779: chown d0/d23/d44/dca/d2e/da4/fe3 437487172 1 2024-03-29T08:07:40.138 INFO:tasks.workunit.client.0.smithi037.stdout:5/991: link d8/d22/d37/d8d/db3/dcb/deb/fe6 d8/d22/d37/d8d/db3/df9/f14c 0 2024-03-29T08:07:40.144 INFO:tasks.workunit.client.0.smithi037.stdout:2/780: chown d0/d23/d44/df6/ffa 5886653 1 2024-03-29T08:07:40.144 INFO:tasks.workunit.client.0.smithi037.stdout:1/829: truncate d2/d6/fe3 3054546 0 2024-03-29T08:07:40.145 INFO:tasks.workunit.client.0.smithi037.stdout:0/812: dread - d9/da/dd/d20/d55/dac/ffa zero size 2024-03-29T08:07:40.146 INFO:tasks.workunit.client.0.smithi037.stdout:5/992: mkdir d8/d4d/ddf/df2/d14d 0 2024-03-29T08:07:40.151 INFO:tasks.workunit.client.0.smithi037.stdout:1/830: symlink d2/d6/d13/l11c 0 2024-03-29T08:07:40.152 INFO:tasks.workunit.client.0.smithi037.stdout:6/839: dread d0/d5/d1f/da8/fc5 [0,4194304] 0 2024-03-29T08:07:40.156 INFO:tasks.workunit.client.0.smithi037.stdout:2/781: mknod d0/d23/d44/dca/d22/c10e 0 2024-03-29T08:07:40.156 INFO:tasks.workunit.client.0.smithi037.stdout:8/802: dwrite d5/d10/d2d/d34/d50/d83/f5f [0,4194304] 0 2024-03-29T08:07:40.171 INFO:tasks.workunit.client.0.smithi037.stdout:1/831: mknod d2/d6/dc/d18/d22/d4f/d7c/ddc/c11d 0 2024-03-29T08:07:40.174 INFO:tasks.workunit.client.0.smithi037.stdout:0/813: dread d9/da/dd/d20/d38/d53/f62 [0,4194304] 0 2024-03-29T08:07:40.175 INFO:tasks.workunit.client.0.smithi037.stdout:2/782: rename d0/d23/d44/dca/d22 to d0/d23/daf/dbf/d10f 0 2024-03-29T08:07:40.176 INFO:tasks.workunit.client.0.smithi037.stdout:1/832: write d2/d6/dc/d18/d1d/dfc/d109/d80/db0/f112 [158037,20434] 0 2024-03-29T08:07:40.181 INFO:tasks.workunit.client.0.smithi037.stdout:0/814: dread - d9/da/dd/d54/fe2 zero size 2024-03-29T08:07:40.192 INFO:tasks.workunit.client.0.smithi037.stdout:2/783: mkdir d0/d23/d44/dca/d2e/d5f/da5/d110 0 2024-03-29T08:07:40.192 INFO:tasks.workunit.client.0.smithi037.stdout:6/840: truncate d0/d5/d1f/f4c 1859177 0 2024-03-29T08:07:40.193 INFO:tasks.workunit.client.0.smithi037.stdout:1/833: unlink d2/d6/dc/d5d/d62/ff8 0 2024-03-29T08:07:40.196 INFO:tasks.workunit.client.0.smithi037.stdout:0/815: rename d9/ff to d9/da/dd/d20/d55/d5f/d9f/d108/f11c 0 2024-03-29T08:07:40.198 INFO:tasks.workunit.client.0.smithi037.stdout:5/993: dwrite d8/d22/d4f/d70/d79/f7b [0,4194304] 0 2024-03-29T08:07:40.200 INFO:tasks.workunit.client.0.smithi037.stdout:8/803: dread d5/fb [0,4194304] 0 2024-03-29T08:07:40.201 INFO:tasks.workunit.client.0.smithi037.stdout:0/816: write d9/da/dd/d20/d38/d8a/fe9 [980987,56182] 0 2024-03-29T08:07:40.204 INFO:tasks.workunit.client.0.smithi037.stdout:5/994: readlink d8/d22/d37/d8d/db3/dcb/deb/l8b 0 2024-03-29T08:07:40.205 INFO:tasks.workunit.client.0.smithi037.stdout:2/784: symlink d0/d23/l111 0 2024-03-29T08:07:40.209 INFO:tasks.workunit.client.0.smithi037.stdout:6/841: creat d0/d19/d42/d60/f123 x:0 0 0 2024-03-29T08:07:40.209 INFO:tasks.workunit.client.0.smithi037.stdout:0/817: dread - d9/da/dd/d54/ffe zero size 2024-03-29T08:07:40.212 INFO:tasks.workunit.client.0.smithi037.stdout:8/804: creat d5/d10/d2d/d9c/df8/dc8/f105 x:0 0 0 2024-03-29T08:07:40.212 INFO:tasks.workunit.client.0.smithi037.stdout:6/842: chown d0/cfd 8303895 1 2024-03-29T08:07:40.214 INFO:tasks.workunit.client.0.smithi037.stdout:5/995: read - d8/d4d/f6c zero size 2024-03-29T08:07:40.216 INFO:tasks.workunit.client.0.smithi037.stdout:0/818: mknod d9/da/dd/d54/c11d 0 2024-03-29T08:07:40.219 INFO:tasks.workunit.client.0.smithi037.stdout:5/996: truncate d8/d4d/f10d 405652 0 2024-03-29T08:07:40.220 INFO:tasks.workunit.client.0.smithi037.stdout:0/819: dread - d9/da/dd/d20/d38/d8a/f8e zero size 2024-03-29T08:07:40.226 INFO:tasks.workunit.client.0.smithi037.stdout:5/997: mkdir d8/d4d/ddf/d14e 0 2024-03-29T08:07:40.233 INFO:tasks.workunit.client.0.smithi037.stdout:6/843: dwrite d0/d5/d3f/f96 [0,4194304] 0 2024-03-29T08:07:40.234 INFO:tasks.workunit.client.0.smithi037.stdout:0/820: creat d9/da/dd/d13/d83/f11e x:0 0 0 2024-03-29T08:07:40.236 INFO:tasks.workunit.client.0.smithi037.stdout:6/844: chown d0/d19/d34/d4e/df2/c89 6092678 1 2024-03-29T08:07:40.236 INFO:tasks.workunit.client.0.smithi037.stdout:5/998: truncate d8/d18/d24/fd8 3416120 0 2024-03-29T08:07:40.246 INFO:tasks.workunit.client.0.smithi037.stdout:6/845: dread d0/d19/f9b [0,4194304] 0 2024-03-29T08:07:40.247 INFO:tasks.workunit.client.0.smithi037.stdout:0/821: link d9/da/dd/d54/f56 d9/da/dd/d20/d38/d8a/dfd/f11f 0 2024-03-29T08:07:40.249 INFO:tasks.workunit.client.0.smithi037.stdout:5/999: mkdir d8/d22/d37/dbc/d85/d124/d14f 0 2024-03-29T08:07:40.253 INFO:tasks.workunit.client.0.smithi037.stdout:0/822: creat d9/da/dd/d20/d38/d53/f120 x:0 0 0 2024-03-29T08:07:40.257 INFO:tasks.workunit.client.0.smithi037.stdout:0/823: write d9/da/dd/d20/d38/d53/fff [425319,51700] 0 2024-03-29T08:07:40.257 INFO:tasks.workunit.client.0.smithi037.stdout:8/805: dread d5/d10/d2d/d34/d50/d83/f28 [0,4194304] 0 2024-03-29T08:07:40.264 INFO:tasks.workunit.client.0.smithi037.stdout:8/806: mknod d5/d10/d2d/d9c/dfb/c106 0 2024-03-29T08:07:40.267 INFO:tasks.workunit.client.0.smithi037.stdout:0/824: fsync d9/da/dd/d20/d55/dac/fb2 0 2024-03-29T08:07:40.271 INFO:tasks.workunit.client.0.smithi037.stdout:8/807: chown d5/cd9 2718 1 2024-03-29T08:07:40.277 INFO:tasks.workunit.client.0.smithi037.stdout:6/846: dwrite d0/d19/d34/d5a/dbf/fea [0,4194304] 0 2024-03-29T08:07:40.282 INFO:tasks.workunit.client.0.smithi037.stdout:8/808: rename d5/d10/d19/d3d/c42 to d5/d10/dfa/c107 0 2024-03-29T08:07:40.288 INFO:tasks.workunit.client.0.smithi037.stdout:6/847: link d0/d6a/d7b/d77/lb9 d0/d5/d1e/d44/dbb/ddb/l124 0 2024-03-29T08:07:40.296 INFO:tasks.workunit.client.0.smithi037.stdout:6/848: creat d0/d19/d42/d60/f125 x:0 0 0 2024-03-29T08:07:40.301 INFO:tasks.workunit.client.0.smithi037.stdout:0/825: dwrite d9/da/dd/d20/d38/f84 [0,4194304] 0 2024-03-29T08:07:40.301 INFO:tasks.workunit.client.0.smithi037.stdout:6/849: write d0/d19/d42/d60/f7d [136063,20542] 0 2024-03-29T08:07:40.305 INFO:tasks.workunit.client.0.smithi037.stdout:6/850: stat d0/d19/d42/lba 0 2024-03-29T08:07:40.308 INFO:tasks.workunit.client.0.smithi037.stdout:8/809: dwrite d5/d10/d19/d3d/f5d [0,4194304] 0 2024-03-29T08:07:40.314 INFO:tasks.workunit.client.0.smithi037.stdout:8/810: stat d5/d10/d2d/d34/d50/l5c 0 2024-03-29T08:07:40.316 INFO:tasks.workunit.client.0.smithi037.stdout:6/851: link d0/d5/d3f/c3 d0/d5/d1e/d44/d71/c126 0 2024-03-29T08:07:40.318 INFO:tasks.workunit.client.0.smithi037.stdout:6/852: fsync d0/d19/d34/f66 0 2024-03-29T08:07:40.344 INFO:tasks.workunit.client.0.smithi037.stdout:0/826: dread d9/da/dd/d13/f61 [0,4194304] 0 2024-03-29T08:07:40.348 INFO:tasks.workunit.client.0.smithi037.stdout:0/827: mknod d9/da/d1b/d22/c121 0 2024-03-29T08:07:40.351 INFO:tasks.workunit.client.0.smithi037.stdout:6/853: dread d0/d5/d1a/f2f [0,4194304] 0 2024-03-29T08:07:40.358 INFO:tasks.workunit.client.0.smithi037.stdout:6/854: dread d0/d5/f73 [0,4194304] 0 2024-03-29T08:07:40.361 INFO:tasks.workunit.client.0.smithi037.stdout:6/855: write d0/d5/d1a/fa6 [4620333,129764] 0 2024-03-29T08:07:40.367 INFO:tasks.workunit.client.0.smithi037.stdout:6/856: creat d0/d5/d1e/d84/f127 x:0 0 0 2024-03-29T08:07:40.373 INFO:tasks.workunit.client.0.smithi037.stdout:6/857: write d0/d19/d42/d60/d62/f9a [1258258,68902] 0 2024-03-29T08:07:40.375 INFO:tasks.workunit.client.0.smithi037.stdout:6/858: dread - d0/d19/f99 zero size 2024-03-29T08:07:40.378 INFO:tasks.workunit.client.0.smithi037.stdout:6/859: write d0/d19/d34/d5a/dbf/fea [606296,114062] 0 2024-03-29T08:07:40.381 INFO:tasks.workunit.client.0.smithi037.stdout:6/860: creat d0/d5/d1e/d84/f128 x:0 0 0 2024-03-29T08:07:40.385 INFO:tasks.workunit.client.0.smithi037.stdout:6/861: rmdir d0/d19/d34/d4e/d69 39 2024-03-29T08:07:40.406 INFO:tasks.workunit.client.0.smithi037.stdout:6/862: dread d0/d5/d1f/f30 [0,4194304] 0 2024-03-29T08:07:40.413 INFO:tasks.workunit.client.0.smithi037.stdout:6/863: truncate d0/d20/d95/ff5 188575 0 2024-03-29T08:07:40.417 INFO:tasks.workunit.client.0.smithi037.stdout:6/864: truncate d0/d5/d1a/f6c 1084504 0 2024-03-29T08:07:40.420 INFO:tasks.workunit.client.0.smithi037.stdout:6/865: creat d0/d20/ddd/f129 x:0 0 0 2024-03-29T08:07:40.423 INFO:tasks.workunit.client.0.smithi037.stdout:6/866: fdatasync d0/d5/d1e/f12 0 2024-03-29T08:07:40.428 INFO:tasks.workunit.client.0.smithi037.stdout:6/867: unlink d0/d20/d43/lb0 0 2024-03-29T08:07:40.432 INFO:tasks.workunit.client.0.smithi037.stdout:6/868: truncate d0/d5/d3f/f92 9068693 0 2024-03-29T08:07:40.438 INFO:tasks.workunit.client.0.smithi037.stdout:6/869: write d0/d20/fac [265063,50059] 0 2024-03-29T08:07:40.442 INFO:tasks.workunit.client.0.smithi037.stdout:6/870: mknod d0/d19/d42/c12a 0 2024-03-29T08:07:40.464 INFO:tasks.workunit.client.0.smithi037.stdout:6/871: dwrite d0/d19/d42/d60/d62/f9a [0,4194304] 0 2024-03-29T08:07:40.485 INFO:tasks.workunit.client.0.smithi037.stdout:6/872: symlink d0/d5/l12b 0 2024-03-29T08:07:40.487 INFO:tasks.workunit.client.0.smithi037.stdout:6/873: chown d0/d5/d1e/f7f 938328 1 2024-03-29T08:07:40.490 INFO:tasks.workunit.client.0.smithi037.stdout:6/874: chown d0/d5/d1f/da8/c9c 48 1 2024-03-29T08:07:40.492 INFO:tasks.workunit.client.0.smithi037.stdout:6/875: dread - d0/d19/d34/d4e/df2/d105/fd8 zero size 2024-03-29T08:07:40.514 INFO:tasks.workunit.client.0.smithi037.stdout:6/876: write d0/d19/d42/d60/f9d [9185977,27268] 0 2024-03-29T08:07:40.517 INFO:tasks.workunit.client.0.smithi037.stdout:6/877: fdatasync d0/d5/d1e/d44/d71/f8c 0 2024-03-29T08:07:40.522 INFO:tasks.workunit.client.0.smithi037.stdout:6/878: mkdir d0/d19/d34/d5a/dbf/dde/d12c 0 2024-03-29T08:07:40.524 INFO:tasks.workunit.client.0.smithi037.stdout:6/879: truncate d0/d19/d34/d5a/fd7 299095 0 2024-03-29T08:07:40.543 INFO:tasks.workunit.client.0.smithi037.stdout:6/880: dwrite d0/d19/d34/d4e/d69/d79/feb [0,4194304] 0 2024-03-29T08:07:40.545 INFO:tasks.workunit.client.0.smithi037.stdout:6/881: fsync d0/d19/d42/d60/f125 0 2024-03-29T08:07:40.546 INFO:tasks.workunit.client.0.smithi037.stdout:6/882: chown d0/d20/ddd/f129 818709 1 2024-03-29T08:07:40.548 INFO:tasks.workunit.client.0.smithi037.stdout:6/883: stat d0/d5/d1e/d84/le5 0 2024-03-29T08:07:40.552 INFO:tasks.workunit.client.0.smithi037.stdout:6/884: dread d0/d19/d34/d4e/df2/f65 [0,4194304] 0 2024-03-29T08:07:40.574 INFO:tasks.workunit.client.0.smithi037.stdout:6/885: dread d0/d6a/f70 [0,4194304] 0 2024-03-29T08:07:40.576 INFO:tasks.workunit.client.0.smithi037.stdout:6/886: write d0/d19/d42/d60/f9d [9259729,99019] 0 2024-03-29T08:07:40.578 INFO:tasks.workunit.client.0.smithi037.stdout:6/887: fsync d0/d19/d34/d4e/d94/f119 0 2024-03-29T08:07:40.580 INFO:tasks.workunit.client.0.smithi037.stdout:6/888: dread - d0/d19/d42/d60/d62/f11e zero size 2024-03-29T08:07:40.586 INFO:tasks.workunit.client.0.smithi037.stdout:6/889: rename d0/d19/df8 to d0/d19/d34/d4e/df2/d12d 0 2024-03-29T08:07:40.586 INFO:tasks.workunit.client.0.smithi037.stdout:6/890: readlink d0/d19/d34/d5a/l11b 0 2024-03-29T08:07:40.588 INFO:tasks.workunit.client.0.smithi037.stdout:6/891: read - d0/d5/d1e/fc2 zero size 2024-03-29T08:07:40.591 INFO:tasks.workunit.client.0.smithi037.stdout:6/892: rename d0/d5/d1e/d44/fad to d0/d5/d1f/f12e 0 2024-03-29T08:07:40.594 INFO:tasks.workunit.client.0.smithi037.stdout:6/893: creat d0/d19/d42/d60/d62/f12f x:0 0 0 2024-03-29T08:07:40.596 INFO:tasks.workunit.client.0.smithi037.stdout:6/894: chown d0/d5/d3f/l53 0 1 2024-03-29T08:07:40.607 INFO:tasks.workunit.client.0.smithi037.stdout:6/895: truncate d0/d5/d1a/f5d 3946879 0 2024-03-29T08:07:40.609 INFO:tasks.workunit.client.0.smithi037.stdout:6/896: mkdir d0/d5/d1f/da8/dd0/d130 0 2024-03-29T08:07:40.616 INFO:tasks.workunit.client.0.smithi037.stdout:6/897: write d0/d19/d34/d4e/df2/d105/fd8 [626299,83056] 0 2024-03-29T08:07:40.619 INFO:tasks.workunit.client.0.smithi037.stdout:6/898: truncate d0/d19/d42/d60/f123 488602 0 2024-03-29T08:07:40.621 INFO:tasks.workunit.client.0.smithi037.stdout:6/899: unlink d0/d20/c45 0 2024-03-29T08:07:40.635 INFO:tasks.workunit.client.0.smithi037.stdout:6/900: dwrite d0/d19/d34/d4e/df2/d105/f117 [0,4194304] 0 2024-03-29T08:07:40.639 INFO:tasks.workunit.client.0.smithi037.stdout:6/901: dread - d0/d19/d34/d4e/d94/fc1 zero size 2024-03-29T08:07:40.642 INFO:tasks.workunit.client.0.smithi037.stdout:6/902: mkdir d0/d19/d34/d4e/df2/d131 0 2024-03-29T08:07:40.671 INFO:tasks.workunit.client.0.smithi037.stdout:6/903: dread d0/d5/d1e/f67 [0,4194304] 0 2024-03-29T08:07:40.674 INFO:tasks.workunit.client.0.smithi037.stdout:6/904: unlink d0/d6a/d7b/d77/cf7 0 2024-03-29T08:07:40.676 INFO:tasks.workunit.client.0.smithi037.stdout:6/905: mkdir d0/d20/d43/d132 0 2024-03-29T08:07:40.680 INFO:tasks.workunit.client.0.smithi037.stdout:6/906: rmdir d0/d6a/d7b 39 2024-03-29T08:07:40.686 INFO:tasks.workunit.client.0.smithi037.stdout:6/907: rename d0/d19/d34/d4e/d69/d79/dcd/lfe to d0/d20/d95/l133 0 2024-03-29T08:07:40.690 INFO:tasks.workunit.client.0.smithi037.stdout:6/908: rename d0/d5/d1e/d44/d71/f8c to d0/d5/d1f/da8/f134 0 2024-03-29T08:07:40.697 INFO:tasks.workunit.client.0.smithi037.stdout:6/909: write d0/d6a/d7b/d77/f8e [603703,128974] 0 2024-03-29T08:07:40.699 INFO:tasks.workunit.client.0.smithi037.stdout:6/910: fsync d0/d5/d1e/d44/dbb/ddb/ffb 0 2024-03-29T08:07:40.703 INFO:tasks.workunit.client.0.smithi037.stdout:6/911: creat d0/d19/d34/d5a/dbf/dde/d12c/f135 x:0 0 0 2024-03-29T08:07:40.704 INFO:tasks.workunit.client.0.smithi037.stdout:6/912: fdatasync d0/d6a/d7b/fa1 0 2024-03-29T08:07:40.706 INFO:tasks.workunit.client.0.smithi037.stdout:6/913: write d0/d5/d1e/d44/f101 [1787867,3695] 0 2024-03-29T08:07:40.710 INFO:tasks.workunit.client.0.smithi037.stdout:6/914: dread - d0/d5/d1f/da8/f78 zero size 2024-03-29T08:07:40.728 INFO:tasks.workunit.client.0.smithi037.stdout:6/915: dwrite d0/d19/d34/d4e/f54 [0,4194304] 0 2024-03-29T08:07:40.733 INFO:tasks.workunit.client.0.smithi037.stdout:6/916: write d0/d19/f99 [913526,59839] 0 2024-03-29T08:07:40.737 INFO:tasks.workunit.client.0.smithi037.stdout:6/917: truncate d0/d5/d3f/f6f 4042177 0 2024-03-29T08:07:40.740 INFO:tasks.workunit.client.0.smithi037.stdout:6/918: read d0/d19/d34/d5a/fd7 [97668,105774] 0 2024-03-29T08:07:40.745 INFO:tasks.workunit.client.0.smithi037.stdout:6/919: link d0/d20/f8d d0/d5/d1f/f136 0 2024-03-29T08:07:40.747 INFO:tasks.workunit.client.0.smithi037.stdout:6/920: unlink d0/d19/f99 0 2024-03-29T08:07:40.750 INFO:tasks.workunit.client.0.smithi037.stdout:4/830: sync 2024-03-29T08:07:40.750 INFO:tasks.workunit.client.0.smithi037.stdout:3/780: sync 2024-03-29T08:07:40.750 INFO:tasks.workunit.client.0.smithi037.stdout:7/998: sync 2024-03-29T08:07:40.752 INFO:tasks.workunit.client.0.smithi037.stdout:6/921: mknod d0/d19/d34/d5a/dbf/dde/c137 0 2024-03-29T08:07:40.754 INFO:tasks.workunit.client.0.smithi037.stdout:3/781: chown dd/d46/lc2 28240 1 2024-03-29T08:07:40.755 INFO:tasks.workunit.client.0.smithi037.stdout:6/922: chown d0/d5/d1e/fb5 374305881 1 2024-03-29T08:07:40.757 INFO:tasks.workunit.client.0.smithi037.stdout:3/782: symlink dd/d46/d2d/d3a/l101 0 2024-03-29T08:07:40.758 INFO:tasks.workunit.client.0.smithi037.stdout:6/923: read - d0/d5/d1e/d84/f127 zero size 2024-03-29T08:07:40.763 INFO:tasks.workunit.client.0.smithi037.stdout:6/924: rename d0/d20/lb7 to d0/d20/l138 0 2024-03-29T08:07:40.774 INFO:tasks.workunit.client.0.smithi037.stdout:4/831: dwrite d3/d80/f10b [0,4194304] 0 2024-03-29T08:07:40.777 INFO:tasks.workunit.client.0.smithi037.stdout:6/925: creat d0/d19/f139 x:0 0 0 2024-03-29T08:07:40.778 INFO:tasks.workunit.client.0.smithi037.stdout:7/999: dread dd/d21/d35/d42/ffe [0,4194304] 0 2024-03-29T08:07:40.783 INFO:tasks.workunit.client.0.smithi037.stdout:3/783: rename dd/d5e/d7a/lb4 to dd/d46/d2d/d3a/d6c/l102 0 2024-03-29T08:07:40.786 INFO:tasks.workunit.client.0.smithi037.stdout:4/832: creat d3/d4/f11d x:0 0 0 2024-03-29T08:07:40.786 INFO:tasks.workunit.client.0.smithi037.stdout:6/926: write d0/d5/d1f/da8/fc5 [3382878,79396] 0 2024-03-29T08:07:40.790 INFO:tasks.workunit.client.0.smithi037.stdout:3/784: write dd/d46/d2d/d3a/d6c/db1/dc0/dda/d33/d97/ff1 [871039,50662] 0 2024-03-29T08:07:40.791 INFO:tasks.workunit.client.0.smithi037.stdout:6/927: chown d0/d5/d1f/da8/da3/d113 37 1 2024-03-29T08:07:40.794 INFO:tasks.workunit.client.0.smithi037.stdout:3/785: readlink dd/d46/d2d/d3a/l79 0 2024-03-29T08:07:40.797 INFO:tasks.workunit.client.0.smithi037.stdout:3/786: write dd/d46/f71 [1130728,57496] 0 2024-03-29T08:07:40.803 INFO:tasks.workunit.client.0.smithi037.stdout:3/787: creat dd/d5e/d7a/dd4/f103 x:0 0 0 2024-03-29T08:07:40.807 INFO:tasks.workunit.client.0.smithi037.stdout:6/928: truncate d0/d19/d34/d4e/df2/d105/fd8 663740 0 2024-03-29T08:07:40.813 INFO:tasks.workunit.client.0.smithi037.stdout:4/833: dread d3/d4/d27/f2e [0,4194304] 0 2024-03-29T08:07:40.815 INFO:tasks.workunit.client.0.smithi037.stdout:3/788: write dd/d46/d2d/d3a/d6c/db1/dc0/dda/d33/d84/fb8 [455054,120080] 0 2024-03-29T08:07:40.821 INFO:tasks.workunit.client.0.smithi037.stdout:3/789: creat dd/d46/d2d/d3a/d6c/db1/dc0/dda/d33/dfb/f104 x:0 0 0 2024-03-29T08:07:40.823 INFO:tasks.workunit.client.0.smithi037.stdout:4/834: truncate d3/d57/dec/ded/d1d/db/d45/d1e/d59/d73/fd4 342254 0 2024-03-29T08:07:40.825 INFO:tasks.workunit.client.0.smithi037.stdout:3/790: write dd/d46/d2d/d3a/d6c/db1/dc0/dda/f41 [2381133,35208] 0 2024-03-29T08:07:40.829 INFO:tasks.workunit.client.0.smithi037.stdout:3/791: write dd/d46/d2d/d3a/fb6 [1483398,58012] 0 2024-03-29T08:07:40.835 INFO:tasks.workunit.client.0.smithi037.stdout:3/792: rename dd/d46 to dd/d46/d2d/d3a/d6c/db1/dc0/dda/d33/d97/dd6/d105 22 2024-03-29T08:07:40.838 INFO:tasks.workunit.client.0.smithi037.stdout:3/793: chown dd/d15/d18/d4b/da2/cc4 73921 1 2024-03-29T08:07:40.846 INFO:tasks.workunit.client.0.smithi037.stdout:4/835: dwrite d3/d57/dec/ded/d1d/db/d45/f5a [0,4194304] 0 2024-03-29T08:07:40.849 INFO:tasks.workunit.client.0.smithi037.stdout:4/836: write d3/d57/d85/f10d [755750,36131] 0 2024-03-29T08:07:40.864 INFO:tasks.workunit.client.0.smithi037.stdout:3/794: dread dd/d46/f38 [0,4194304] 0 2024-03-29T08:07:40.873 INFO:tasks.workunit.client.0.smithi037.stdout:4/837: dwrite d3/d80/feb [0,4194304] 0 2024-03-29T08:07:40.877 INFO:tasks.workunit.client.0.smithi037.stdout:4/838: symlink d3/d57/dec/ded/d1d/d4f/l11e 0 2024-03-29T08:07:40.882 INFO:tasks.workunit.client.0.smithi037.stdout:4/839: dread - d3/d80/fb3 zero size 2024-03-29T08:07:40.885 INFO:tasks.workunit.client.0.smithi037.stdout:3/795: dwrite dd/d15/d18/d4b/ff9 [0,4194304] 0 2024-03-29T08:07:40.889 INFO:tasks.workunit.client.0.smithi037.stdout:3/796: dread - dd/ffd zero size 2024-03-29T08:07:40.890 INFO:tasks.workunit.client.0.smithi037.stdout:4/840: mknod d3/d57/dec/ded/d1d/db/d45/d1e/d59/db5/c11f 0 2024-03-29T08:07:40.891 INFO:tasks.workunit.client.0.smithi037.stdout:4/841: readlink d3/d4/d27/l28 0 2024-03-29T08:07:40.907 INFO:tasks.workunit.client.0.smithi037.stdout:3/797: dwrite dd/d5e/f6d [4194304,4194304] 0 2024-03-29T08:07:40.912 INFO:tasks.workunit.client.0.smithi037.stdout:3/798: mkdir dd/d46/d2d/d3a/d6c/db1/dc0/dda/d33/d106 0 2024-03-29T08:07:40.913 INFO:tasks.workunit.client.0.smithi037.stdout:3/799: chown dd/d46/f38 25 1 2024-03-29T08:07:40.920 INFO:tasks.workunit.client.0.smithi037.stdout:3/800: read dd/d15/d18/d4b/fad [741500,1560] 0 2024-03-29T08:07:40.926 INFO:tasks.workunit.client.0.smithi037.stdout:3/801: truncate dd/d5e/f45 3679434 0 2024-03-29T08:07:40.926 INFO:tasks.workunit.client.0.smithi037.stdout:3/802: stat dd/d15/d18/d4b/da2/f94 0 2024-03-29T08:07:40.931 INFO:tasks.workunit.client.0.smithi037.stdout:3/803: symlink dd/d46/d2d/d3a/d6c/db1/dc0/dda/d33/d106/l107 0 2024-03-29T08:07:40.935 INFO:tasks.workunit.client.0.smithi037.stdout:4/842: dread d3/d4/d27/fbc [0,4194304] 0 2024-03-29T08:07:40.936 INFO:tasks.workunit.client.0.smithi037.stdout:3/804: creat dd/d46/d2d/d3a/d6c/db1/dc0/dda/ddb/f108 x:0 0 0 2024-03-29T08:07:40.939 INFO:tasks.workunit.client.0.smithi037.stdout:3/805: creat dd/d15/d18/d4b/f109 x:0 0 0 2024-03-29T08:07:40.944 INFO:tasks.workunit.client.0.smithi037.stdout:4/843: link d3/d57/dec/ded/d1d/db/fa3 d3/d57/dec/ded/d1d/d4f/f120 0 2024-03-29T08:07:40.945 INFO:tasks.workunit.client.0.smithi037.stdout:3/806: creat dd/d46/d2d/d3a/d6c/db1/dc0/dda/d33/d97/dd6/df5/f10a x:0 0 0 2024-03-29T08:07:40.947 INFO:tasks.workunit.client.0.smithi037.stdout:4/844: chown d3/d57/dec/ded/d1d/l17 28657 1 2024-03-29T08:07:40.953 INFO:tasks.workunit.client.0.smithi037.stdout:4/845: link d3/d57/d85/f10d d3/d57/d6b/f121 0 2024-03-29T08:07:40.954 INFO:tasks.workunit.client.0.smithi037.stdout:4/846: dread - d3/d4/f11d zero size 2024-03-29T08:07:40.959 INFO:tasks.workunit.client.0.smithi037.stdout:3/807: dwrite dd/f3f [4194304,4194304] 0 2024-03-29T08:07:40.971 INFO:tasks.workunit.client.0.smithi037.stdout:3/808: rename dd/fe to dd/d15/d18/d30/dcf/f10b 0 2024-03-29T08:07:40.977 INFO:tasks.workunit.client.0.smithi037.stdout:3/809: chown dd/d46/d2d/d3a/d6c/db1/dc0/dda/ddb/c98 669 1 2024-03-29T08:07:40.977 INFO:tasks.workunit.client.0.smithi037.stdout:4/847: dwrite d3/d57/d85/f10d [0,4194304] 0 2024-03-29T08:07:40.980 INFO:tasks.workunit.client.0.smithi037.stdout:3/810: stat dd/d46/d2d/d3a/d6c/db1/dc0/dda/ddb/cd1 0 2024-03-29T08:07:40.993 INFO:tasks.workunit.client.0.smithi037.stdout:3/811: rename dd/d46/d2d/d3a/d6c/db1/dc0/dda/d33/d75/fab to dd/d46/d2d/d3a/d6c/db1/dc0/dda/d33/d75/d8f/f10c 0 2024-03-29T08:07:40.994 INFO:tasks.workunit.client.0.smithi037.stdout:3/812: chown dd/d15/ce2 853547167 1 2024-03-29T08:07:40.997 INFO:tasks.workunit.client.0.smithi037.stdout:4/848: dwrite d3/d57/dec/ded/d1d/db/d45/fce [4194304,4194304] 0 2024-03-29T08:07:41.005 INFO:tasks.workunit.client.0.smithi037.stdout:4/849: creat d3/d57/d6a/f122 x:0 0 0 2024-03-29T08:07:41.013 INFO:tasks.workunit.client.0.smithi037.stdout:4/850: rmdir d3/d57/dec/ded/d1d/db/d45/d93/ddb 0 2024-03-29T08:07:41.019 INFO:tasks.workunit.client.0.smithi037.stdout:4/851: dread - d3/d57/dec/ded/d1d/db/d45/d93/d96/da0/dc9/f11b zero size 2024-03-29T08:07:41.019 INFO:tasks.workunit.client.0.smithi037.stdout:3/813: dwrite dd/d46/d2d/d3a/d6c/db1/dc0/dda/d33/d84/fd8 [0,4194304] 0 2024-03-29T08:07:41.021 INFO:tasks.workunit.client.0.smithi037.stdout:3/814: fdatasync dd/d15/fe6 0 2024-03-29T08:07:41.022 INFO:tasks.workunit.client.0.smithi037.stdout:4/852: creat d3/d57/dec/f123 x:0 0 0 2024-03-29T08:07:41.029 INFO:tasks.workunit.client.0.smithi037.stdout:3/815: rename dd/d46/d2d/d3a/d6c/db1/dc0/dda/ddb/cd1 to dd/c10d 0 2024-03-29T08:07:41.032 INFO:tasks.workunit.client.0.smithi037.stdout:3/816: rmdir dd/d46/d3e 39 2024-03-29T08:07:41.034 INFO:tasks.workunit.client.0.smithi037.stdout:3/817: dread - dd/d15/f49 zero size 2024-03-29T08:07:41.040 INFO:tasks.workunit.client.0.smithi037.stdout:3/818: rmdir dd/d46/d2d/d3a/d6c/db1/dc0/dda/d33/d84 39 2024-03-29T08:07:41.045 INFO:tasks.workunit.client.0.smithi037.stdout:4/853: dwrite d3/d57/dec/ded/d1d/db/d45/d1e/d59/d60/d6f/fa4 [0,4194304] 0 2024-03-29T08:07:41.052 INFO:tasks.workunit.client.0.smithi037.stdout:4/854: creat d3/d57/dec/ded/d1d/dc2/f124 x:0 0 0 2024-03-29T08:07:41.055 INFO:tasks.workunit.client.0.smithi037.stdout:4/855: truncate d3/d4/f11d 1035832 0 2024-03-29T08:07:41.059 INFO:tasks.workunit.client.0.smithi037.stdout:4/856: mkdir d3/d80/df6/d125 0 2024-03-29T08:07:41.067 INFO:tasks.workunit.client.0.smithi037.stdout:4/857: write d3/d57/dec/ded/d1d/db/d45/d1e/f4a [2672091,27834] 0 2024-03-29T08:07:41.069 INFO:tasks.workunit.client.0.smithi037.stdout:4/858: read d3/d80/f10b [2761412,73367] 0 2024-03-29T08:07:41.072 INFO:tasks.workunit.client.0.smithi037.stdout:8/811: sync 2024-03-29T08:07:41.072 INFO:tasks.workunit.client.0.smithi037.stdout:6/929: sync 2024-03-29T08:07:41.072 INFO:tasks.workunit.client.0.smithi037.stdout:2/785: sync 2024-03-29T08:07:41.072 INFO:tasks.workunit.client.0.smithi037.stdout:1/834: sync 2024-03-29T08:07:41.072 INFO:tasks.workunit.client.0.smithi037.stdout:0/828: sync 2024-03-29T08:07:41.074 INFO:tasks.workunit.client.0.smithi037.stdout:4/859: mkdir d3/d4/d27/d126 0 2024-03-29T08:07:41.078 INFO:tasks.workunit.client.0.smithi037.stdout:3/819: dread dd/d5e/fbb [0,4194304] 0 2024-03-29T08:07:41.081 INFO:tasks.workunit.client.0.smithi037.stdout:0/829: mkdir d9/da/dd/d20/d55/d5f/dab/dbb/d122 0 2024-03-29T08:07:41.083 INFO:tasks.workunit.client.0.smithi037.stdout:2/786: write d0/d23/d70/f79 [304408,32381] 0 2024-03-29T08:07:41.084 INFO:tasks.workunit.client.0.smithi037.stdout:6/930: write d0/d20/d95/f10e [272189,46376] 0 2024-03-29T08:07:41.084 INFO:tasks.workunit.client.0.smithi037.stdout:8/812: write d5/f32 [3923943,27578] 0 2024-03-29T08:07:41.086 INFO:tasks.workunit.client.0.smithi037.stdout:0/830: read - d9/da/d1b/d22/d2c/f86 zero size 2024-03-29T08:07:41.092 INFO:tasks.workunit.client.0.smithi037.stdout:6/931: mknod d0/d19/d34/d4e/df2/c13a 0 2024-03-29T08:07:41.095 INFO:tasks.workunit.client.0.smithi037.stdout:0/831: dread - d9/da/dd/d20/ff9 zero size 2024-03-29T08:07:41.096 INFO:tasks.workunit.client.0.smithi037.stdout:1/835: getdents d2/d6/dc/d18/d22/d4f/d86 0 2024-03-29T08:07:41.107 INFO:tasks.workunit.client.0.smithi037.stdout:6/932: write d0/d19/d34/d5a/dbf/fea [2266401,49551] 0 2024-03-29T08:07:41.109 INFO:tasks.workunit.client.0.smithi037.stdout:4/860: dwrite d3/d57/dec/ded/d1d/dad/dfb/f110 [4194304,4194304] 0 2024-03-29T08:07:41.110 INFO:tasks.workunit.client.0.smithi037.stdout:8/813: dread d5/d10/d45/d4b/fa4 [0,4194304] 0 2024-03-29T08:07:41.117 INFO:tasks.workunit.client.0.smithi037.stdout:0/832: getdents d9/da/dd/d20/d55/d5f/dab/dbb 0 2024-03-29T08:07:41.120 INFO:tasks.workunit.client.0.smithi037.stdout:1/836: truncate d2/d6/dc/d18/d22/d4f/d86/d10d/f106 420596 0 2024-03-29T08:07:41.123 INFO:tasks.workunit.client.0.smithi037.stdout:0/833: dread - d9/da/d1b/d22/d2c/ff2 zero size 2024-03-29T08:07:41.124 INFO:tasks.workunit.client.0.smithi037.stdout:4/861: write d3/d57/dec/ded/d1d/dc2/f124 [358281,22893] 0 2024-03-29T08:07:41.130 INFO:tasks.workunit.client.0.smithi037.stdout:1/837: write d2/d6/dc/d7e/f115 [893121,25174] 0 2024-03-29T08:07:41.131 INFO:tasks.workunit.client.0.smithi037.stdout:2/787: dread d0/d23/d44/dca/d2e/d5f/d6f/d5e/f60 [0,4194304] 0 2024-03-29T08:07:41.141 INFO:tasks.workunit.client.0.smithi037.stdout:0/834: rmdir d9/da/dd/d20/d55/d5f/d9f 39 2024-03-29T08:07:41.141 INFO:tasks.workunit.client.0.smithi037.stdout:3/820: dread dd/d46/d3e/d47/f54 [0,4194304] 0 2024-03-29T08:07:41.141 INFO:tasks.workunit.client.0.smithi037.stdout:8/814: link d5/d10/d2d/lb2 d5/d10/d2d/d9c/dfb/l108 0 2024-03-29T08:07:41.144 INFO:tasks.workunit.client.0.smithi037.stdout:1/838: rename d2/d6/dc/d18/d22/d4f to d2/d6/dc/d58/d67/d11b/d11e 0 2024-03-29T08:07:41.145 INFO:tasks.workunit.client.0.smithi037.stdout:4/862: creat d3/d4/f127 x:0 0 0 2024-03-29T08:07:41.150 INFO:tasks.workunit.client.0.smithi037.stdout:8/815: stat d5/f16 0 2024-03-29T08:07:41.154 INFO:tasks.workunit.client.0.smithi037.stdout:6/933: dwrite d0/d6a/dd6/f11c [4194304,4194304] 0 2024-03-29T08:07:41.155 INFO:tasks.workunit.client.0.smithi037.stdout:0/835: symlink d9/da/dd/d20/d38/l123 0 2024-03-29T08:07:41.157 INFO:tasks.workunit.client.0.smithi037.stdout:4/863: fsync d3/d4/f2f 0 2024-03-29T08:07:41.164 INFO:tasks.workunit.client.0.smithi037.stdout:6/934: read d0/d19/d42/d60/f7d [886864,58456] 0 2024-03-29T08:07:41.165 INFO:tasks.workunit.client.0.smithi037.stdout:8/816: rename d5/d78/l93 to d5/d10/d2d/d9c/da1/dbc/l109 0 2024-03-29T08:07:41.167 INFO:tasks.workunit.client.0.smithi037.stdout:0/836: creat d9/d74/d7d/d7e/f124 x:0 0 0 2024-03-29T08:07:41.169 INFO:tasks.workunit.client.0.smithi037.stdout:2/788: getdents d0/d23/d44/dca/d2e/d5f 0 2024-03-29T08:07:41.173 INFO:tasks.workunit.client.0.smithi037.stdout:6/935: dread - d0/d5/fda zero size 2024-03-29T08:07:41.178 INFO:tasks.workunit.client.0.smithi037.stdout:8/817: read d5/f15 [1228048,47918] 0 2024-03-29T08:07:41.185 INFO:tasks.workunit.client.0.smithi037.stdout:0/837: rename d9/da/d1b/d22/d2c to d9/da/dd/d54/d125 0 2024-03-29T08:07:41.190 INFO:tasks.workunit.client.0.smithi037.stdout:0/838: fdatasync d9/da/dd/f99 0 2024-03-29T08:07:41.205 INFO:tasks.workunit.client.0.smithi037.stdout:8/818: link d5/d10/d19/d3d/f7e d5/d10/d19/d3d/f10a 0 2024-03-29T08:07:41.207 INFO:tasks.workunit.client.0.smithi037.stdout:1/839: write d2/d6/dc/d18/dbd/f100 [742437,38037] 0 2024-03-29T08:07:41.216 INFO:tasks.workunit.client.0.smithi037.stdout:1/840: fsync d2/d6/dc/f4a 0 2024-03-29T08:07:41.226 INFO:tasks.workunit.client.0.smithi037.stdout:8/819: write d5/d10/d45/d4b/f6c [8595950,101555] 0 2024-03-29T08:07:41.227 INFO:tasks.workunit.client.0.smithi037.stdout:2/789: dread d0/d23/d44/dca/f3a [0,4194304] 0 2024-03-29T08:07:41.239 INFO:tasks.workunit.client.0.smithi037.stdout:2/790: dread - d0/d23/d44/dca/d2e/fe7 zero size 2024-03-29T08:07:41.243 INFO:tasks.workunit.client.0.smithi037.stdout:8/820: chown d5/d10/d2d/d9c/de0/lfd 228787957 1 2024-03-29T08:07:41.243 INFO:tasks.workunit.client.0.smithi037.stdout:4/864: dwrite d3/d57/dec/ded/d1d/db/d45/d1e/d59/d73/f83 [0,4194304] 0 2024-03-29T08:07:41.245 INFO:tasks.workunit.client.0.smithi037.stdout:3/821: dwrite dd/d46/d2d/d3a/d6c/db1/dc0/dda/d33/f96 [0,4194304] 0 2024-03-29T08:07:41.257 INFO:tasks.workunit.client.0.smithi037.stdout:6/936: dwrite d0/d19/d42/d60/d62/fc6 [0,4194304] 0 2024-03-29T08:07:41.260 INFO:tasks.workunit.client.0.smithi037.stdout:8/821: chown d5/d10/d2d/d34/d50/fc0 32512 1 2024-03-29T08:07:41.262 INFO:tasks.workunit.client.0.smithi037.stdout:3/822: mkdir dd/d46/d2d/d3a/d6c/d10e 0 2024-03-29T08:07:41.266 INFO:tasks.workunit.client.0.smithi037.stdout:0/839: dwrite d9/da/d1b/f3d [0,4194304] 0 2024-03-29T08:07:41.269 INFO:tasks.workunit.client.0.smithi037.stdout:8/822: chown d5/f1c 31770438 1 2024-03-29T08:07:41.269 INFO:tasks.workunit.client.0.smithi037.stdout:4/865: mknod d3/d57/dec/ded/d1d/db/d45/d1e/d59/d60/d6f/db9/c128 0 2024-03-29T08:07:41.271 INFO:tasks.workunit.client.0.smithi037.stdout:0/840: stat d9/d74/f97 0 2024-03-29T08:07:41.272 INFO:tasks.workunit.client.0.smithi037.stdout:2/791: truncate d0/d23/d44/dca/d2e/d38/d68/fac 523804 0 2024-03-29T08:07:41.276 INFO:tasks.workunit.client.0.smithi037.stdout:1/841: dwrite d2/d6/dc/d18/d1d/fff [0,4194304] 0 2024-03-29T08:07:41.280 INFO:tasks.workunit.client.0.smithi037.stdout:8/823: mkdir d5/d10b 0 2024-03-29T08:07:41.282 INFO:tasks.workunit.client.0.smithi037.stdout:6/937: rename d0/d19/d34/d5a/dbf/dde/c137 to d0/d19/d42/d60/c13b 0 2024-03-29T08:07:41.283 INFO:tasks.workunit.client.0.smithi037.stdout:4/866: chown d3/d57/dec/ded/d1d/c31 4820 1 2024-03-29T08:07:41.283 INFO:tasks.workunit.client.0.smithi037.stdout:3/823: write dd/d46/d2d/d3a/d6c/db1/dc0/dda/d33/d84/fb8 [463392,72835] 0 2024-03-29T08:07:41.285 INFO:tasks.workunit.client.0.smithi037.stdout:2/792: dread - d0/d23/daf/dbf/d10f/d55/d92/ff2 zero size 2024-03-29T08:07:41.287 INFO:tasks.workunit.client.0.smithi037.stdout:6/938: chown d0/d19/d42/d60/f7d 9652 1 2024-03-29T08:07:41.288 INFO:tasks.workunit.client.0.smithi037.stdout:1/842: creat d2/d6/dc/d58/d67/d11b/f11f x:0 0 0 2024-03-29T08:07:41.289 INFO:tasks.workunit.client.0.smithi037.stdout:8/824: getdents d5/d10/d2d/d34/dc5 0 2024-03-29T08:07:41.296 INFO:tasks.workunit.client.0.smithi037.stdout:6/939: creat d0/d5/d1f/f13c x:0 0 0 2024-03-29T08:07:41.298 INFO:tasks.workunit.client.0.smithi037.stdout:1/843: creat d2/d40/de8/f120 x:0 0 0 2024-03-29T08:07:41.298 INFO:tasks.workunit.client.0.smithi037.stdout:2/793: truncate d0/d23/d44/dca/d2e/fe7 59376 0 2024-03-29T08:07:41.299 INFO:tasks.workunit.client.0.smithi037.stdout:1/844: chown d2/d6/d13/c116 1387217927 1 2024-03-29T08:07:41.305 INFO:tasks.workunit.client.0.smithi037.stdout:4/867: dread d3/d57/dec/ded/d1d/db/d45/d1e/f108 [0,4194304] 0 2024-03-29T08:07:41.305 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:07:41 smithi148 ceph-mon[44992]: pgmap v52: 65 pgs: 65 active+clean; 3.6 GiB data, 12 GiB used, 524 GiB / 536 GiB avail; 92 MiB/s rd, 186 MiB/s wr, 493 op/s 2024-03-29T08:07:41.308 INFO:tasks.workunit.client.0.smithi037.stdout:6/940: mknod d0/d19/d34/d4e/d69/d79/c13d 0 2024-03-29T08:07:41.310 INFO:tasks.workunit.client.0.smithi037.stdout:1/845: symlink d2/d6/dc/d18/d1d/d5e/l121 0 2024-03-29T08:07:41.316 INFO:tasks.workunit.client.0.smithi037.stdout:3/824: write dd/d5e/d7a/f4f [4325764,21638] 0 2024-03-29T08:07:41.318 INFO:tasks.workunit.client.0.smithi037.stdout:1/846: fsync d2/d6/d13/f93 0 2024-03-29T08:07:41.319 INFO:tasks.workunit.client.0.smithi037.stdout:4/868: dread d3/d57/dec/ded/d1d/db/d72/ff0 [0,4194304] 0 2024-03-29T08:07:41.320 INFO:tasks.workunit.client.0.smithi037.stdout:6/941: creat d0/d19/d42/d60/d62/f13e x:0 0 0 2024-03-29T08:07:41.323 INFO:tasks.workunit.client.0.smithi037.stdout:2/794: link d0/d23/daf/dbf/d10f/l64 d0/d23/d44/dca/d7d/l112 0 2024-03-29T08:07:41.324 INFO:tasks.workunit.client.0.smithi037.stdout:8/825: dwrite d5/d10/d2d/d9c/da1/fa3 [0,4194304] 0 2024-03-29T08:07:41.324 INFO:tasks.workunit.client.0.smithi037.stdout:0/841: dwrite d9/da/fc6 [0,4194304] 0 2024-03-29T08:07:41.340 INFO:tasks.workunit.client.0.smithi037.stdout:2/795: write d0/d23/d44/dca/d2e/d5f/d6f/fa9 [4378363,71985] 0 2024-03-29T08:07:41.343 INFO:tasks.workunit.client.0.smithi037.stdout:1/847: symlink d2/d6/dc/d58/d67/df6/l122 0 2024-03-29T08:07:41.345 INFO:tasks.workunit.client.0.smithi037.stdout:4/869: dread - d3/d57/dec/ded/d1d/db/d45/d93/ffc zero size 2024-03-29T08:07:41.346 INFO:tasks.workunit.client.0.smithi037.stdout:8/826: rmdir d5/d10/d2d/d34/d50/d83/d46/d4e 39 2024-03-29T08:07:41.350 INFO:tasks.workunit.client.0.smithi037.stdout:6/942: getdents d0/d19/d34/d5a 0 2024-03-29T08:07:41.353 INFO:tasks.workunit.client.0.smithi037.stdout:8/827: stat d5/d10/d2d/d9c/df8/d4c/l56 0 2024-03-29T08:07:41.358 INFO:tasks.workunit.client.0.smithi037.stdout:0/842: write d9/da/d1b/d22/fce [914751,91525] 0 2024-03-29T08:07:41.362 INFO:tasks.workunit.client.0.smithi037.stdout:4/870: rmdir d3/d57/dec/ded/d1d/db/d45/d1e/d59/d73 39 2024-03-29T08:07:41.362 INFO:tasks.workunit.client.0.smithi037.stdout:1/848: link d2/d6/dc/d18/d22/lf3 d2/l123 0 2024-03-29T08:07:41.363 INFO:tasks.workunit.client.0.smithi037.stdout:3/825: dwrite dd/d46/d2d/d3a/f53 [0,4194304] 0 2024-03-29T08:07:41.365 INFO:tasks.workunit.client.0.smithi037.stdout:4/871: readlink d3/d54/l7c 0 2024-03-29T08:07:41.365 INFO:tasks.workunit.client.0.smithi037.stdout:6/943: mknod d0/d19/d34/d5a/dbf/dde/d110/c13f 0 2024-03-29T08:07:41.369 INFO:tasks.workunit.client.0.smithi037.stdout:4/872: write d3/d57/d85/f10d [3986572,128307] 0 2024-03-29T08:07:41.376 INFO:tasks.workunit.client.0.smithi037.stdout:1/849: creat d2/d6/dc/d58/d67/d11b/d11e/d86/d10d/f124 x:0 0 0 2024-03-29T08:07:41.378 INFO:tasks.workunit.client.0.smithi037.stdout:0/843: creat d9/da/dd/d13/f126 x:0 0 0 2024-03-29T08:07:41.378 INFO:tasks.workunit.client.0.smithi037.stdout:3/826: mknod dd/d46/d2d/c10f 0 2024-03-29T08:07:41.380 INFO:tasks.workunit.client.0.smithi037.stdout:6/944: symlink d0/d19/d34/d5a/dbf/dde/l140 0 2024-03-29T08:07:41.386 INFO:tasks.workunit.client.0.smithi037.stdout:4/873: rename d3/d57/dec/ded/d1d/db/d45/d1e/d59/d79/fa6 to d3/d54/d88/f129 0 2024-03-29T08:07:41.387 INFO:tasks.workunit.client.0.smithi037.stdout:2/796: dread d0/d23/d44/dca/d2e/f36 [0,4194304] 0 2024-03-29T08:07:41.387 INFO:tasks.workunit.client.0.smithi037.stdout:1/850: stat d2/d6/dc/d18/d1d/dfc/d109/d80/db0/f111 0 2024-03-29T08:07:41.389 INFO:tasks.workunit.client.0.smithi037.stdout:0/844: fsync d9/da/dd/d13/f96 0 2024-03-29T08:07:41.389 INFO:tasks.workunit.client.0.smithi037.stdout:6/945: chown d0/d5/d1f/da8/da3/d113/l121 1336447 1 2024-03-29T08:07:41.391 INFO:tasks.workunit.client.0.smithi037.stdout:8/828: dwrite d5/f48 [0,4194304] 0 2024-03-29T08:07:41.398 INFO:tasks.workunit.client.0.smithi037.stdout:3/827: creat dd/d46/d2d/d3a/d6c/f110 x:0 0 0 2024-03-29T08:07:41.400 INFO:tasks.workunit.client.0.smithi037.stdout:6/946: symlink d0/d19/d34/d4e/df2/d105/l141 0 2024-03-29T08:07:41.404 INFO:tasks.workunit.client.0.smithi037.stdout:1/851: rename d2/d6/dc/d58/fa2 to d2/d6/dc/d7e/dd2/d113/f125 0 2024-03-29T08:07:41.405 INFO:tasks.workunit.client.0.smithi037.stdout:6/947: read d0/d19/d34/d4e/d69/d79/feb [571723,90754] 0 2024-03-29T08:07:41.409 INFO:tasks.workunit.client.0.smithi037.stdout:1/852: stat d2/d6/dc/d18/d1d/d5e/d72/l76 0 2024-03-29T08:07:41.416 INFO:tasks.workunit.client.0.smithi037.stdout:1/853: dread - d2/d6/dc/d18/d1d/d21/d99/f10f zero size 2024-03-29T08:07:41.418 INFO:tasks.workunit.client.0.smithi037.stdout:4/874: symlink d3/d57/dec/ded/d1d/db/d45/d93/d96/da0/dc9/dd8/l12a 0 2024-03-29T08:07:41.422 INFO:tasks.workunit.client.0.smithi037.stdout:6/948: truncate d0/d19/d34/d4e/d94/fc1 381785 0 2024-03-29T08:07:41.425 INFO:tasks.workunit.client.0.smithi037.stdout:1/854: symlink d2/d6/dc/d18/d1d/dfc/l126 0 2024-03-29T08:07:41.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:07:41 smithi037 ceph-mon[35162]: pgmap v52: 65 pgs: 65 active+clean; 3.6 GiB data, 12 GiB used, 524 GiB / 536 GiB avail; 92 MiB/s rd, 186 MiB/s wr, 493 op/s 2024-03-29T08:07:41.427 INFO:tasks.workunit.client.0.smithi037.stdout:6/949: fdatasync d0/d6a/d7b/f9e 0 2024-03-29T08:07:41.427 INFO:tasks.workunit.client.0.smithi037.stdout:1/855: readlink d2/l3c 0 2024-03-29T08:07:41.429 INFO:tasks.workunit.client.0.smithi037.stdout:0/845: dwrite d9/da/dd/d20/d55/dac/fb2 [0,4194304] 0 2024-03-29T08:07:41.431 INFO:tasks.workunit.client.0.smithi037.stdout:4/875: mkdir d3/d57/dec/ded/d1d/db/d45/d1e/d59/d79/d12b 0 2024-03-29T08:07:41.436 INFO:tasks.workunit.client.0.smithi037.stdout:6/950: mknod d0/d19/d34/d5a/dbf/c142 0 2024-03-29T08:07:41.438 INFO:tasks.workunit.client.0.smithi037.stdout:8/829: dwrite d5/f1f [4194304,4194304] 0 2024-03-29T08:07:41.443 INFO:tasks.workunit.client.0.smithi037.stdout:1/856: creat d2/d6/dc/d18/d1d/d21/d99/f127 x:0 0 0 2024-03-29T08:07:41.444 INFO:tasks.workunit.client.0.smithi037.stdout:6/951: dread d0/d19/f9b [0,4194304] 0 2024-03-29T08:07:41.445 INFO:tasks.workunit.client.0.smithi037.stdout:8/830: dread - d5/d10/d2d/df5/f103 zero size 2024-03-29T08:07:41.449 INFO:tasks.workunit.client.0.smithi037.stdout:6/952: stat d0/d6a/d7b 0 2024-03-29T08:07:41.449 INFO:tasks.workunit.client.0.smithi037.stdout:2/797: dread d0/d23/d44/dca/fce [4194304,4194304] 0 2024-03-29T08:07:41.449 INFO:tasks.workunit.client.0.smithi037.stdout:8/831: chown d5/d10/d2d/d34/d50/d83/f5f 113785 1 2024-03-29T08:07:41.449 INFO:tasks.workunit.client.0.smithi037.stdout:1/857: chown d2/d6/dc/d58/d67/d11b/d11e/d86 111 1 2024-03-29T08:07:41.450 INFO:tasks.workunit.client.0.smithi037.stdout:4/876: write d3/d80/dcd/d8f/f11c [1237463,107403] 0 2024-03-29T08:07:41.454 INFO:tasks.workunit.client.0.smithi037.stdout:6/953: chown d0/d5/d1e/d44/d71/f75 0 1 2024-03-29T08:07:41.469 INFO:tasks.workunit.client.0.smithi037.stdout:4/877: mkdir d3/d54/d12c 0 2024-03-29T08:07:41.472 INFO:tasks.workunit.client.0.smithi037.stdout:4/878: readlink d3/d57/dec/ded/d1d/l17 0 2024-03-29T08:07:41.488 INFO:tasks.workunit.client.0.smithi037.stdout:8/832: dwrite d5/d10/d2d/d9c/df8/f77 [0,4194304] 0 2024-03-29T08:07:41.494 INFO:tasks.workunit.client.0.smithi037.stdout:1/858: dread d2/d40/f43 [0,4194304] 0 2024-03-29T08:07:41.504 INFO:tasks.workunit.client.0.smithi037.stdout:2/798: rmdir d0/d23/d44/dca/d2e/d5f/da5/dda 0 2024-03-29T08:07:41.505 INFO:tasks.workunit.client.0.smithi037.stdout:1/859: chown d2/d6/dc/d18/d1d/d5e/fb2 368 1 2024-03-29T08:07:41.507 INFO:tasks.workunit.client.0.smithi037.stdout:4/879: unlink d3/d57/dec/ded/d1d/db/d45/d1e/d59/d60/db7/c116 0 2024-03-29T08:07:41.508 INFO:tasks.workunit.client.0.smithi037.stdout:6/954: dread d0/d5/d1e/f7f [0,4194304] 0 2024-03-29T08:07:41.508 INFO:tasks.workunit.client.0.smithi037.stdout:8/833: dread d5/d10/d45/d4b/fa4 [0,4194304] 0 2024-03-29T08:07:41.509 INFO:tasks.workunit.client.0.smithi037.stdout:2/799: mkdir d0/d23/d113 0 2024-03-29T08:07:41.515 INFO:tasks.workunit.client.0.smithi037.stdout:2/800: mknod d0/d23/daf/dbf/d10f/d4e/c114 0 2024-03-29T08:07:41.517 INFO:tasks.workunit.client.0.smithi037.stdout:1/860: creat d2/d6/dc/d5d/f128 x:0 0 0 2024-03-29T08:07:41.519 INFO:tasks.workunit.client.0.smithi037.stdout:6/955: truncate d0/d5/d1e/d44/dbb/ddb/ffb 588607 0 2024-03-29T08:07:41.521 INFO:tasks.workunit.client.0.smithi037.stdout:8/834: dread d5/d10/d45/d4b/fa4 [0,4194304] 0 2024-03-29T08:07:41.522 INFO:tasks.workunit.client.0.smithi037.stdout:4/880: unlink d3/d57/dec/ded/d1d/db/d72/d98/cd1 0 2024-03-29T08:07:41.526 INFO:tasks.workunit.client.0.smithi037.stdout:2/801: truncate d0/d23/d44/dca/d2e/d9c/df3/f72 426482 0 2024-03-29T08:07:41.530 INFO:tasks.workunit.client.0.smithi037.stdout:6/956: mkdir d0/d19/d34/d4e/df2/d131/d143 0 2024-03-29T08:07:41.532 INFO:tasks.workunit.client.0.smithi037.stdout:2/802: unlink d0/d23/d44/ca8 0 2024-03-29T08:07:41.538 INFO:tasks.workunit.client.0.smithi037.stdout:1/861: link d2/d6/dc/d58/d67/d11b/f11f d2/d40/de8/daf/f129 0 2024-03-29T08:07:41.538 INFO:tasks.workunit.client.0.smithi037.stdout:6/957: rmdir d0/d5 39 2024-03-29T08:07:41.540 INFO:tasks.workunit.client.0.smithi037.stdout:4/881: creat d3/d57/dec/ded/d1d/f12d x:0 0 0 2024-03-29T08:07:41.543 INFO:tasks.workunit.client.0.smithi037.stdout:8/835: creat d5/d10/d2d/d9c/df8/d4c/d67/f10c x:0 0 0 2024-03-29T08:07:41.548 INFO:tasks.workunit.client.0.smithi037.stdout:2/803: rename d0/l6e to d0/d23/d44/dca/d7d/dfb/l115 0 2024-03-29T08:07:41.555 INFO:tasks.workunit.client.0.smithi037.stdout:2/804: mknod d0/d23/daf/dbf/d10f/d55/c116 0 2024-03-29T08:07:41.556 INFO:tasks.workunit.client.0.smithi037.stdout:1/862: dwrite d2/d6/dc/d18/f63 [4194304,4194304] 0 2024-03-29T08:07:41.565 INFO:tasks.workunit.client.0.smithi037.stdout:2/805: mknod d0/d23/daf/dbf/d10f/d4e/c117 0 2024-03-29T08:07:41.566 INFO:tasks.workunit.client.0.smithi037.stdout:1/863: mknod d2/d6/dc/d18/dbd/c12a 0 2024-03-29T08:07:41.568 INFO:tasks.workunit.client.0.smithi037.stdout:6/958: link d0/d5/d1f/da8/f97 d0/d6a/f144 0 2024-03-29T08:07:41.571 INFO:tasks.workunit.client.0.smithi037.stdout:1/864: mknod d2/d6/dc/d7e/dd2/d113/c12b 0 2024-03-29T08:07:41.572 INFO:tasks.workunit.client.0.smithi037.stdout:6/959: mknod d0/d5/d3f/c145 0 2024-03-29T08:07:41.573 INFO:tasks.workunit.client.0.smithi037.stdout:2/806: mknod d0/d23/d44/dca/d2e/d5f/d6f/d5e/c118 0 2024-03-29T08:07:41.577 INFO:tasks.workunit.client.0.smithi037.stdout:4/882: write d3/d57/dec/ded/d1d/db/f12 [1373584,38051] 0 2024-03-29T08:07:41.577 INFO:tasks.workunit.client.0.smithi037.stdout:1/865: fdatasync d2/d6/dc/d18/d1d/dfc/d109/fd0 0 2024-03-29T08:07:41.577 INFO:tasks.workunit.client.0.smithi037.stdout:8/836: write d5/d78/dac/fbe [474055,31733] 0 2024-03-29T08:07:41.578 INFO:tasks.workunit.client.0.smithi037.stdout:2/807: write d0/d23/d44/f105 [134541,56811] 0 2024-03-29T08:07:41.578 INFO:tasks.workunit.client.0.smithi037.stdout:6/960: symlink d0/d5/d1f/da8/da3/l146 0 2024-03-29T08:07:41.583 INFO:tasks.workunit.client.0.smithi037.stdout:8/837: write d5/d10/d2d/f104 [988517,15679] 0 2024-03-29T08:07:41.588 INFO:tasks.workunit.client.0.smithi037.stdout:6/961: readlink d0/d19/d42/d60/lb8 0 2024-03-29T08:07:41.589 INFO:tasks.workunit.client.0.smithi037.stdout:4/883: fsync d3/d57/dec/ded/d1d/db/d45/fae 0 2024-03-29T08:07:41.593 INFO:tasks.workunit.client.0.smithi037.stdout:6/962: write d0/d5/d1a/f2f [5443789,25456] 0 2024-03-29T08:07:41.594 INFO:tasks.workunit.client.0.smithi037.stdout:8/838: mknod d5/d10/d2d/d34/dc5/c10d 0 2024-03-29T08:07:41.595 INFO:tasks.workunit.client.0.smithi037.stdout:1/866: creat d2/d6/dc/d58/f12c x:0 0 0 2024-03-29T08:07:41.598 INFO:tasks.workunit.client.0.smithi037.stdout:2/808: fsync d0/d23/d44/dca/d2e/d5f/d6f/d5e/dc2/fc7 0 2024-03-29T08:07:41.598 INFO:tasks.workunit.client.0.smithi037.stdout:4/884: unlink d3/d4/f3e 0 2024-03-29T08:07:41.599 INFO:tasks.workunit.client.0.smithi037.stdout:1/867: fsync d2/d6/dc/d18/d1d/d5e/fe6 0 2024-03-29T08:07:41.601 INFO:tasks.workunit.client.0.smithi037.stdout:6/963: fsync d0/d19/d34/d4e/d69/d79/feb 0 2024-03-29T08:07:41.603 INFO:tasks.workunit.client.0.smithi037.stdout:1/868: symlink d2/d6/dc/d7e/l12d 0 2024-03-29T08:07:41.604 INFO:tasks.workunit.client.0.smithi037.stdout:8/839: mknod d5/d10/d2d/d34/d50/d83/d46/d4e/c10e 0 2024-03-29T08:07:41.607 INFO:tasks.workunit.client.0.smithi037.stdout:4/885: truncate d3/d57/dec/ded/f63 2922766 0 2024-03-29T08:07:41.609 INFO:tasks.workunit.client.0.smithi037.stdout:2/809: truncate d0/d23/d44/dca/d2e/d5f/d6f/d5e/fc1 756286 0 2024-03-29T08:07:41.614 INFO:tasks.workunit.client.0.smithi037.stdout:4/886: creat d3/d80/df6/d125/f12e x:0 0 0 2024-03-29T08:07:41.616 INFO:tasks.workunit.client.0.smithi037.stdout:1/869: truncate d2/d6/dc/d18/d1d/d5e/fe6 1056546 0 2024-03-29T08:07:41.621 INFO:tasks.workunit.client.0.smithi037.stdout:8/840: getdents d5/d10/d2d/d9c/df8/d4c/d67/d85 0 2024-03-29T08:07:41.622 INFO:tasks.workunit.client.0.smithi037.stdout:1/870: mknod d2/d6/dc/d58/d67/df6/c12e 0 2024-03-29T08:07:41.629 INFO:tasks.workunit.client.0.smithi037.stdout:8/841: creat d5/d10/d2d/d9c/df8/d102/f10f x:0 0 0 2024-03-29T08:07:41.634 INFO:tasks.workunit.client.0.smithi037.stdout:1/871: rename d2/d6/dc/d58/d67/d11b/d11e/d86/d10d/f124 to d2/d6/dc/d58/d67/d11b/d11e/d7c/ddc/f12f 0 2024-03-29T08:07:41.635 INFO:tasks.workunit.client.0.smithi037.stdout:8/842: rmdir d5/d10/d2d/d34 39 2024-03-29T08:07:41.636 INFO:tasks.workunit.client.0.smithi037.stdout:4/887: dwrite d3/d57/d6b/f121 [0,4194304] 0 2024-03-29T08:07:41.637 INFO:tasks.workunit.client.0.smithi037.stdout:1/872: stat d2/d6/dc/d18/d22/lea 0 2024-03-29T08:07:41.640 INFO:tasks.workunit.client.0.smithi037.stdout:8/843: truncate d5/f5b 2985111 0 2024-03-29T08:07:41.641 INFO:tasks.workunit.client.0.smithi037.stdout:1/873: fdatasync d2/d6/dc/d18/d1d/dfc/d109/d80/db0/f111 0 2024-03-29T08:07:41.648 INFO:tasks.workunit.client.0.smithi037.stdout:1/874: truncate d2/d6/dc/fd 4318672 0 2024-03-29T08:07:41.657 INFO:tasks.workunit.client.0.smithi037.stdout:1/875: creat d2/d6/dc/d18/d1d/dfc/d109/da1/f130 x:0 0 0 2024-03-29T08:07:41.665 INFO:tasks.workunit.client.0.smithi037.stdout:1/876: read d2/d6/dc/d18/d22/da0/fcd [856799,14500] 0 2024-03-29T08:07:41.668 INFO:tasks.workunit.client.0.smithi037.stdout:1/877: write d2/d6/d13/f77 [1933658,108581] 0 2024-03-29T08:07:41.670 INFO:tasks.workunit.client.0.smithi037.stdout:4/888: dwrite d3/d57/dec/ded/d1d/d4f/fdd [0,4194304] 0 2024-03-29T08:07:41.676 INFO:tasks.workunit.client.0.smithi037.stdout:4/889: symlink d3/d4/d27/d126/l12f 0 2024-03-29T08:07:41.682 INFO:tasks.workunit.client.0.smithi037.stdout:1/878: link d2/d6/dc/d18/d1d/dfc/d109/l4c d2/d6/dc/d5d/l131 0 2024-03-29T08:07:41.684 INFO:tasks.workunit.client.0.smithi037.stdout:4/890: mknod d3/d57/dec/ded/d1d/db/d45/d1e/d59/d79/d95/c130 0 2024-03-29T08:07:41.684 INFO:tasks.workunit.client.0.smithi037.stdout:1/879: chown d2/d40/de8/l104 8 1 2024-03-29T08:07:41.687 INFO:tasks.workunit.client.0.smithi037.stdout:1/880: rmdir d2/d6/dc/d58/d67/d11b/d11e 39 2024-03-29T08:07:41.704 INFO:tasks.workunit.client.0.smithi037.stdout:1/881: dwrite d2/d6/fe1 [0,4194304] 0 2024-03-29T08:07:41.710 INFO:tasks.workunit.client.0.smithi037.stdout:1/882: creat d2/d6/dc/d58/f132 x:0 0 0 2024-03-29T08:07:41.710 INFO:tasks.workunit.client.0.smithi037.stdout:4/891: dread d3/d57/dec/ded/d1d/db/d45/d1e/d59/d7d/f9f [0,4194304] 0 2024-03-29T08:07:41.715 INFO:tasks.workunit.client.0.smithi037.stdout:1/883: chown d2/d6/dc/d58/d67/d11b/d11e/ff7 7373440 1 2024-03-29T08:07:41.720 INFO:tasks.workunit.client.0.smithi037.stdout:1/884: truncate d2/d6/dc/d18/d1d/f3f 4304476 0 2024-03-29T08:07:41.725 INFO:tasks.workunit.client.0.smithi037.stdout:4/892: dread d3/f3a [0,4194304] 0 2024-03-29T08:07:41.727 INFO:tasks.workunit.client.0.smithi037.stdout:1/885: creat d2/d6/dc/d58/f133 x:0 0 0 2024-03-29T08:07:41.728 INFO:tasks.workunit.client.0.smithi037.stdout:4/893: dread - d3/d54/d88/fd9 zero size 2024-03-29T08:07:41.732 INFO:tasks.workunit.client.0.smithi037.stdout:4/894: write d3/d57/dec/ded/d1d/d4f/fdd [539730,103714] 0 2024-03-29T08:07:41.735 INFO:tasks.workunit.client.0.smithi037.stdout:1/886: truncate d2/d6/dc/d58/d67/d11b/d11e/f69 1724926 0 2024-03-29T08:07:41.738 INFO:tasks.workunit.client.0.smithi037.stdout:1/887: rename d2 to d2/d6/dc/d18/d22/da0/d134 22 2024-03-29T08:07:41.740 INFO:tasks.workunit.client.0.smithi037.stdout:4/895: write d3/d57/dec/ded/d1d/db/d45/d1e/d59/fe9 [77110,100583] 0 2024-03-29T08:07:41.740 INFO:tasks.workunit.client.0.smithi037.stdout:1/888: chown d2/d6/dc/d7e/dd2/d113/c12b 79628448 1 2024-03-29T08:07:41.745 INFO:tasks.workunit.client.0.smithi037.stdout:1/889: unlink d2/d6/dc/d18/d1d/d21/d99/f98 0 2024-03-29T08:07:41.746 INFO:tasks.workunit.client.0.smithi037.stdout:4/896: creat d3/d57/dec/ded/d1d/db/d45/d93/d96/da0/dc9/dd8/f131 x:0 0 0 2024-03-29T08:07:41.750 INFO:tasks.workunit.client.0.smithi037.stdout:1/890: mknod d2/d6/dc/d18/d1d/d21/c135 0 2024-03-29T08:07:41.752 INFO:tasks.workunit.client.0.smithi037.stdout:1/891: chown d2/d6/dc/f1e 91769 1 2024-03-29T08:07:41.756 INFO:tasks.workunit.client.0.smithi037.stdout:1/892: creat d2/d6/dc/d7e/dd2/d113/f136 x:0 0 0 2024-03-29T08:07:41.763 INFO:tasks.workunit.client.0.smithi037.stdout:4/897: dwrite d3/d4/f11d [0,4194304] 0 2024-03-29T08:07:41.769 INFO:tasks.workunit.client.0.smithi037.stdout:4/898: creat d3/d4/f132 x:0 0 0 2024-03-29T08:07:41.776 INFO:tasks.workunit.client.0.smithi037.stdout:4/899: dread d3/d80/dcd/f90 [0,4194304] 0 2024-03-29T08:07:41.784 INFO:tasks.workunit.client.0.smithi037.stdout:1/893: dread d2/d6/dc/d5d/fd5 [4194304,4194304] 0 2024-03-29T08:07:41.785 INFO:tasks.workunit.client.0.smithi037.stdout:4/900: write d3/d57/dec/ded/f1f [3201932,96844] 0 2024-03-29T08:07:41.788 INFO:tasks.workunit.client.0.smithi037.stdout:1/894: write d2/d6/dc/d18/d1d/d21/d99/f127 [166529,114248] 0 2024-03-29T08:07:41.797 INFO:tasks.workunit.client.0.smithi037.stdout:1/895: write d2/d6/dc/d58/d67/d11b/f11f [405530,99409] 0 2024-03-29T08:07:41.801 INFO:tasks.workunit.client.0.smithi037.stdout:1/896: symlink d2/d40/de8/d87/l137 0 2024-03-29T08:07:41.806 INFO:tasks.workunit.client.0.smithi037.stdout:4/901: dwrite d3/d4/d27/f7f [0,4194304] 0 2024-03-29T08:07:41.809 INFO:tasks.workunit.client.0.smithi037.stdout:4/902: chown d3/d57/dec/ded/d1d/db/d45/d1e/d5f 5317834 1 2024-03-29T08:07:41.813 INFO:tasks.workunit.client.0.smithi037.stdout:4/903: creat d3/d80/dcd/d8f/f133 x:0 0 0 2024-03-29T08:07:41.814 INFO:tasks.workunit.client.0.smithi037.stdout:4/904: chown d3/c7 1808851 1 2024-03-29T08:07:41.821 INFO:tasks.workunit.client.0.smithi037.stdout:1/897: dwrite d2/d6/dc/d18/d1d/dfc/d109/d80/f107 [0,4194304] 0 2024-03-29T08:07:41.824 INFO:tasks.workunit.client.0.smithi037.stdout:1/898: dread - d2/d6/dc/d58/fc0 zero size 2024-03-29T08:07:41.830 INFO:tasks.workunit.client.0.smithi037.stdout:1/899: mkdir d2/d6/dc/d18/d1d/d21/d99/d138 0 2024-03-29T08:07:41.833 INFO:tasks.workunit.client.0.smithi037.stdout:1/900: creat d2/d6/dc/d18/d1d/d5e/f139 x:0 0 0 2024-03-29T08:07:41.835 INFO:tasks.workunit.client.0.smithi037.stdout:1/901: chown d2/d6/d13/c34 201 1 2024-03-29T08:07:41.840 INFO:tasks.workunit.client.0.smithi037.stdout:1/902: mknod d2/d6/dc/d18/d1d/dfc/d109/d80/c13a 0 2024-03-29T08:07:41.844 INFO:tasks.workunit.client.0.smithi037.stdout:1/903: symlink d2/d6/dc/d18/d1d/dfc/d109/d80/l13b 0 2024-03-29T08:07:41.854 INFO:tasks.workunit.client.0.smithi037.stdout:1/904: truncate d2/d40/fbc 1854046 0 2024-03-29T08:07:41.857 INFO:tasks.workunit.client.0.smithi037.stdout:1/905: chown d2/d6/dc/c9a 24 1 2024-03-29T08:07:41.858 INFO:tasks.workunit.client.0.smithi037.stdout:1/906: fsync d2/d6/fe1 0 2024-03-29T08:07:41.860 INFO:tasks.workunit.client.0.smithi037.stdout:4/905: dread d3/d57/dec/ded/d1d/f3f [0,4194304] 0 2024-03-29T08:07:41.861 INFO:tasks.workunit.client.0.smithi037.stdout:1/907: symlink d2/d6/dc/d5d/d62/l13c 0 2024-03-29T08:07:41.863 INFO:tasks.workunit.client.0.smithi037.stdout:4/906: readlink d3/d57/dec/ded/d1d/d4f/lcc 0 2024-03-29T08:07:41.868 INFO:tasks.workunit.client.0.smithi037.stdout:1/908: getdents d2/d6/dc/d7e 0 2024-03-29T08:07:41.874 INFO:tasks.workunit.client.0.smithi037.stdout:1/909: creat d2/d6/dc/d58/d67/d11b/d11e/f13d x:0 0 0 2024-03-29T08:07:41.881 INFO:tasks.workunit.client.0.smithi037.stdout:1/910: write d2/d6/dc/d58/d67/d11b/d11e/ff1 [1367568,20138] 0 2024-03-29T08:07:41.882 INFO:tasks.workunit.client.0.smithi037.stdout:4/907: dwrite d3/d4/d27/f7f [4194304,4194304] 0 2024-03-29T08:07:41.883 INFO:tasks.workunit.client.0.smithi037.stdout:4/908: stat d3/d4/f46 0 2024-03-29T08:07:41.888 INFO:tasks.workunit.client.0.smithi037.stdout:1/911: creat d2/d6/dc/d18/d1d/dfc/d109/d80/dfb/f13e x:0 0 0 2024-03-29T08:07:41.902 INFO:tasks.workunit.client.0.smithi037.stdout:1/912: write d2/d6/dc/d58/d67/d11b/d11e/ff7 [1373783,31041] 0 2024-03-29T08:07:41.903 INFO:tasks.workunit.client.0.smithi037.stdout:4/909: dwrite d3/d80/df6/fe0 [0,4194304] 0 2024-03-29T08:07:41.906 INFO:tasks.workunit.client.0.smithi037.stdout:1/913: write d2/d6/dc/d5d/d62/f97 [774975,44296] 0 2024-03-29T08:07:41.912 INFO:tasks.workunit.client.0.smithi037.stdout:4/910: rename d3/d4/c48 to d3/d57/dec/ded/d1d/db/d45/d1e/d59/d79/c134 0 2024-03-29T08:07:41.913 INFO:tasks.workunit.client.0.smithi037.stdout:1/914: symlink d2/d6/dc/d58/d67/d11b/d11e/d86/d10d/l13f 0 2024-03-29T08:07:41.918 INFO:tasks.workunit.client.0.smithi037.stdout:4/911: fsync d3/d57/dec/ded/d1d/db/d45/d1e/f108 0 2024-03-29T08:07:41.921 INFO:tasks.workunit.client.0.smithi037.stdout:1/915: dread d2/d6/dc/d18/dbd/f100 [0,4194304] 0 2024-03-29T08:07:41.925 INFO:tasks.workunit.client.0.smithi037.stdout:1/916: symlink d2/d6/dc/d5d/d62/l140 0 2024-03-29T08:07:41.926 INFO:tasks.workunit.client.0.smithi037.stdout:1/917: dread - d2/d6/dc/d58/f132 zero size 2024-03-29T08:07:41.931 INFO:tasks.workunit.client.0.smithi037.stdout:4/912: write d3/d57/dec/ded/d1d/db/d45/d1e/d59/d7d/d97/fb0 [1849864,35620] 0 2024-03-29T08:07:41.945 INFO:tasks.workunit.client.0.smithi037.stdout:1/918: dread d2/d6/dc/d18/d1d/dfc/d109/da1/f96 [0,4194304] 0 2024-03-29T08:07:41.949 INFO:tasks.workunit.client.0.smithi037.stdout:1/919: write d2/d6/dc/d18/d1d/dfc/d109/d80/db0/f111 [1132222,79809] 0 2024-03-29T08:07:41.954 INFO:tasks.workunit.client.0.smithi037.stdout:1/920: creat d2/d6/dc/d58/d67/d11b/d11e/d86/f141 x:0 0 0 2024-03-29T08:07:41.955 INFO:tasks.workunit.client.0.smithi037.stdout:4/913: dread d3/d54/d88/f8d [0,4194304] 0 2024-03-29T08:07:41.973 INFO:tasks.workunit.client.0.smithi037.stdout:1/921: dwrite d2/d6/dc/f15 [0,4194304] 0 2024-03-29T08:07:41.976 INFO:tasks.workunit.client.0.smithi037.stdout:4/914: dwrite d3/d57/dec/ded/f1f [4194304,4194304] 0 2024-03-29T08:07:41.985 INFO:tasks.workunit.client.0.smithi037.stdout:1/922: rename d2/d6/dc/d58/d67/d11b/d11e/d86/d10d/le0 to d2/d6/dc/d18/d1d/dfc/l142 0 2024-03-29T08:07:41.990 INFO:tasks.workunit.client.0.smithi037.stdout:1/923: symlink d2/d6/dc/d18/d1d/d5e/l143 0 2024-03-29T08:07:41.995 INFO:tasks.workunit.client.0.smithi037.stdout:1/924: symlink d2/d6/dc/d58/d67/df6/l144 0 2024-03-29T08:07:42.001 INFO:tasks.workunit.client.0.smithi037.stdout:4/915: dwrite d3/d57/dec/ded/d1d/db/d45/d1e/d59/d79/d95/ffd [0,4194304] 0 2024-03-29T08:07:42.001 INFO:tasks.workunit.client.0.smithi037.stdout:1/925: mkdir d2/d6/dc/d18/d1d/d21/d99/d145 0 2024-03-29T08:07:42.032 INFO:tasks.workunit.client.0.smithi037.stdout:4/916: dwrite d3/d57/dec/ded/f1f [0,4194304] 0 2024-03-29T08:07:42.054 INFO:tasks.workunit.client.0.smithi037.stdout:1/926: dread d2/d6/dc/d58/fed [0,4194304] 0 2024-03-29T08:07:42.056 INFO:tasks.workunit.client.0.smithi037.stdout:1/927: fsync d2/d6/dc/d58/f133 0 2024-03-29T08:07:42.067 INFO:tasks.workunit.client.0.smithi037.stdout:1/928: dread d2/d6/dc/d58/d67/d11b/d11e/f69 [0,4194304] 0 2024-03-29T08:07:42.074 INFO:tasks.workunit.client.0.smithi037.stdout:1/929: dread d2/d6/dc/d5d/ff2 [0,4194304] 0 2024-03-29T08:07:42.083 INFO:tasks.workunit.client.0.smithi037.stdout:1/930: rename d2/d6/dc/d18/d1d/d5e/f139 to d2/d6/dc/d18/d1d/f146 0 2024-03-29T08:07:42.104 INFO:tasks.workunit.client.0.smithi037.stdout:1/931: dwrite d2/d6/f41 [0,4194304] 0 2024-03-29T08:07:42.108 INFO:tasks.workunit.client.0.smithi037.stdout:1/932: write d2/d6/dc/d7e/dd2/d113/f136 [216150,128356] 0 2024-03-29T08:07:42.116 INFO:tasks.workunit.client.0.smithi037.stdout:1/933: link d2/d6/dc/d18/d1d/dfc/d109/da1/cc1 d2/d6/dc/c147 0 2024-03-29T08:07:42.150 INFO:tasks.workunit.client.0.smithi037.stdout:1/934: dread d2/d6/dc/d7e/dd2/d113/f125 [0,4194304] 0 2024-03-29T08:07:42.154 INFO:tasks.workunit.client.0.smithi037.stdout:1/935: unlink d2/d6/dc/d58/l114 0 2024-03-29T08:07:42.293 INFO:tasks.workunit.client.0.smithi037.stdout:3/828: sync 2024-03-29T08:07:42.297 INFO:tasks.workunit.client.0.smithi037.stdout:3/829: chown dd/d46/d2d/d3a/d6c/db1/dc0/dda/d33/d75/d8f/f10c 464119902 1 2024-03-29T08:07:42.300 INFO:tasks.workunit.client.0.smithi037.stdout:3/830: mknod dd/d46/d3e/c111 0 2024-03-29T08:07:42.314 INFO:tasks.workunit.client.0.smithi037.stdout:3/831: dwrite dd/d15/d18/d4b/f67 [0,4194304] 0 2024-03-29T08:07:42.319 INFO:tasks.workunit.client.0.smithi037.stdout:3/832: truncate dd/d46/d2d/d3a/d6c/db1/dc0/dda/d33/f90 5021049 0 2024-03-29T08:07:42.322 INFO:tasks.workunit.client.0.smithi037.stdout:3/833: stat dd/d46/d2d/d3a/d6c/db1/dc0/dda/f89 0 2024-03-29T08:07:42.324 INFO:tasks.workunit.client.0.smithi037.stdout:3/834: read dd/d15/d18/d4b/f67 [3600841,112637] 0 2024-03-29T08:07:42.354 INFO:tasks.workunit.client.0.smithi037.stdout:3/835: dread dd/d15/d18/f2f [0,4194304] 0 2024-03-29T08:07:42.361 INFO:tasks.workunit.client.0.smithi037.stdout:3/836: link dd/d46/c2e dd/d15/d18/d30/dcf/c112 0 2024-03-29T08:07:42.365 INFO:tasks.workunit.client.0.smithi037.stdout:3/837: symlink dd/d46/d2d/d3a/d6c/db1/dc0/dda/d33/d84/l113 0 2024-03-29T08:07:42.369 INFO:tasks.workunit.client.0.smithi037.stdout:3/838: write dd/d46/d2d/d3a/d6c/db1/dc0/dda/d33/d97/fba [719795,112563] 0 2024-03-29T08:07:42.384 INFO:tasks.workunit.client.0.smithi037.stdout:3/839: dwrite dd/d46/d2d/d3a/d6c/db1/dc0/dda/d33/d97/fba [0,4194304] 0 2024-03-29T08:07:42.387 INFO:tasks.workunit.client.0.smithi037.stdout:3/840: write dd/d46/d2d/d3a/d6c/db1/dc0/dda/ddb/f5a [4490076,9512] 0 2024-03-29T08:07:42.392 INFO:tasks.workunit.client.0.smithi037.stdout:3/841: symlink dd/d46/d2d/d3a/d6c/db1/dc0/dda/l114 0 2024-03-29T08:07:42.408 INFO:tasks.workunit.client.0.smithi037.stdout:3/842: rename dd/d46 to dd/d15/d18/d30/d115 0 2024-03-29T08:07:42.411 INFO:tasks.workunit.client.0.smithi037.stdout:3/843: creat dd/d15/d18/d30/d115/d3e/f116 x:0 0 0 2024-03-29T08:07:42.416 INFO:tasks.workunit.client.0.smithi037.stdout:3/844: mkdir dd/d15/d18/d30/d115/d2d/d3a/d6c/db1/dc0/dda/d33/d75/d8f/d117 0 2024-03-29T08:07:42.418 INFO:tasks.workunit.client.0.smithi037.stdout:3/845: chown dd/d15/d18/d30/d115/d3e/lea 11 1 2024-03-29T08:07:42.421 INFO:tasks.workunit.client.0.smithi037.stdout:3/846: getdents dd/d5e 0 2024-03-29T08:07:42.422 INFO:tasks.workunit.client.0.smithi037.stdout:3/847: fdatasync dd/d5e/d7a/fee 0 2024-03-29T08:07:42.425 INFO:tasks.workunit.client.0.smithi037.stdout:3/848: symlink dd/d15/l118 0 2024-03-29T08:07:42.428 INFO:tasks.workunit.client.0.smithi037.stdout:3/849: creat dd/d15/f119 x:0 0 0 2024-03-29T08:07:42.433 INFO:tasks.workunit.client.0.smithi037.stdout:3/850: truncate dd/d15/d18/d30/d115/d2d/fc9 542304 0 2024-03-29T08:07:42.438 INFO:tasks.workunit.client.0.smithi037.stdout:3/851: link dd/f69 dd/d15/d18/d30/d115/d2d/d3a/d6c/db1/f11a 0 2024-03-29T08:07:42.443 INFO:tasks.workunit.client.0.smithi037.stdout:3/852: read dd/d15/d18/d30/d115/d2d/d3a/d6c/db1/dc0/dda/d33/d84/fd0 [2005815,34200] 0 2024-03-29T08:07:42.445 INFO:tasks.workunit.client.0.smithi037.stdout:3/853: write dd/d15/d18/d30/dcf/fec [257920,117767] 0 2024-03-29T08:07:42.459 INFO:tasks.workunit.client.0.smithi037.stdout:3/854: truncate dd/d15/d18/d30/d115/d2d/d3a/d6c/db1/dc0/dda/ddb/f6a 2665372 0 2024-03-29T08:07:42.463 INFO:tasks.workunit.client.0.smithi037.stdout:3/855: chown dd/d15/d18/d30/d115/d3e/f7e 4774 1 2024-03-29T08:07:42.465 INFO:tasks.workunit.client.0.smithi037.stdout:3/856: write dd/ff [9382535,48729] 0 2024-03-29T08:07:42.498 INFO:tasks.workunit.client.0.smithi037.stdout:3/857: getdents dd/d15/d18/d30/d115/d2d/d3a/d6c/db1/dc0/dda 0 2024-03-29T08:07:42.500 INFO:tasks.workunit.client.0.smithi037.stdout:3/858: creat dd/d15/d18/d4b/f11b x:0 0 0 2024-03-29T08:07:42.501 INFO:tasks.workunit.client.0.smithi037.stdout:3/859: stat dd/f24 0 2024-03-29T08:07:42.507 INFO:tasks.workunit.client.0.smithi037.stdout:3/860: rename dd/d15/d18/d30/d115/c73 to dd/d15/d18/d30/d115/d2d/d3a/d6c/db1/c11c 0 2024-03-29T08:07:42.508 INFO:tasks.workunit.client.0.smithi037.stdout:3/861: chown dd/d15/d18/d8e/cf7 31646 1 2024-03-29T08:07:42.510 INFO:tasks.workunit.client.0.smithi037.stdout:3/862: dread - dd/d5e/d7a/dd4/f103 zero size 2024-03-29T08:07:42.515 INFO:tasks.workunit.client.0.smithi037.stdout:3/863: rename dd/d5e/f65 to dd/d15/d18/d30/d115/d2d/d3a/d6c/db1/dc0/f11d 0 2024-03-29T08:07:42.519 INFO:tasks.workunit.client.0.smithi037.stdout:3/864: dread - dd/d15/d18/d30/d115/d2d/d3a/d6c/db1/dc0/dda/d33/d84/ffa zero size 2024-03-29T08:07:42.523 INFO:tasks.workunit.client.0.smithi037.stdout:3/865: symlink dd/d15/d18/d30/d115/d2d/d3a/d6c/db1/dc0/dda/l11e 0 2024-03-29T08:07:42.811 INFO:tasks.workunit.client.0.smithi037.stdout:6/964: sync 2024-03-29T08:07:42.811 INFO:tasks.workunit.client.0.smithi037.stdout:0/846: sync 2024-03-29T08:07:42.813 INFO:tasks.workunit.client.0.smithi037.stdout:6/965: dread - d0/d19/d34/f40 zero size 2024-03-29T08:07:42.819 INFO:tasks.workunit.client.0.smithi037.stdout:6/966: dread d0/d19/d34/d4e/d94/f116 [0,4194304] 0 2024-03-29T08:07:42.827 INFO:tasks.workunit.client.0.smithi037.stdout:6/967: creat d0/d5/d1f/da8/dd0/d130/f147 x:0 0 0 2024-03-29T08:07:42.829 INFO:tasks.workunit.client.0.smithi037.stdout:6/968: chown d0/d20/fac 62141253 1 2024-03-29T08:07:42.832 INFO:tasks.workunit.client.0.smithi037.stdout:0/847: dwrite d9/da/dd/d13/f6f [0,4194304] 0 2024-03-29T08:07:42.838 INFO:tasks.workunit.client.0.smithi037.stdout:0/848: fdatasync d9/da/d1b/d22/faa 0 2024-03-29T08:07:42.842 INFO:tasks.workunit.client.0.smithi037.stdout:0/849: mkdir d9/d74/d7d/d7e/d115/d127 0 2024-03-29T08:07:42.851 INFO:tasks.workunit.client.0.smithi037.stdout:6/969: dwrite d0/d6a/d7b/d77/f8e [0,4194304] 0 2024-03-29T08:07:42.853 INFO:tasks.workunit.client.0.smithi037.stdout:6/970: chown d0/d20/ccb 404457355 1 2024-03-29T08:07:42.860 INFO:tasks.workunit.client.0.smithi037.stdout:6/971: rename d0/d5/d1e/d84/f112 to d0/d19/d34/d4e/d69/dee/f148 0 2024-03-29T08:07:42.863 INFO:tasks.workunit.client.0.smithi037.stdout:6/972: creat d0/d19/d34/d4e/df2/d105/f149 x:0 0 0 2024-03-29T08:07:42.869 INFO:tasks.workunit.client.0.smithi037.stdout:6/973: unlink d0/d5/d1f/da8/c9c 0 2024-03-29T08:07:42.870 INFO:tasks.workunit.client.0.smithi037.stdout:0/850: write d9/da/f8d [1599496,25529] 0 2024-03-29T08:07:42.874 INFO:tasks.workunit.client.0.smithi037.stdout:0/851: truncate d9/d74/d7d/d7e/f124 524060 0 2024-03-29T08:07:42.878 INFO:tasks.workunit.client.0.smithi037.stdout:0/852: mknod d9/da/dd/d20/d38/d8a/dfd/c128 0 2024-03-29T08:07:42.883 INFO:tasks.workunit.client.0.smithi037.stdout:6/974: truncate d0/d5/d1a/fa6 2402446 0 2024-03-29T08:07:42.885 INFO:tasks.workunit.client.0.smithi037.stdout:6/975: creat d0/d20/f14a x:0 0 0 2024-03-29T08:07:42.889 INFO:tasks.workunit.client.0.smithi037.stdout:8/844: sync 2024-03-29T08:07:42.889 INFO:tasks.workunit.client.0.smithi037.stdout:3/866: sync 2024-03-29T08:07:42.889 INFO:tasks.workunit.client.0.smithi037.stdout:4/917: sync 2024-03-29T08:07:42.889 INFO:tasks.workunit.client.0.smithi037.stdout:1/936: sync 2024-03-29T08:07:42.889 INFO:tasks.workunit.client.0.smithi037.stdout:2/810: sync 2024-03-29T08:07:42.890 INFO:tasks.workunit.client.0.smithi037.stdout:6/976: unlink d0/d19/d34/d5a/fd7 0 2024-03-29T08:07:42.895 INFO:tasks.workunit.client.0.smithi037.stdout:0/853: truncate d9/da/dd/d20/d55/d5f/d9f/fd3 2360920 0 2024-03-29T08:07:42.896 INFO:tasks.workunit.client.0.smithi037.stdout:3/867: fdatasync dd/d15/f44 0 2024-03-29T08:07:42.900 INFO:tasks.workunit.client.0.smithi037.stdout:1/937: chown d2/d6/dc/d18/d1d/dfc/d109/da1/f96 48 1 2024-03-29T08:07:42.903 INFO:tasks.workunit.client.0.smithi037.stdout:8/845: truncate d5/d10/d2d/d9c/df8/d4c/d67/f10c 416209 0 2024-03-29T08:07:42.904 INFO:tasks.workunit.client.0.smithi037.stdout:6/977: unlink d0/d19/d34/d5a/c93 0 2024-03-29T08:07:42.906 INFO:tasks.workunit.client.0.smithi037.stdout:2/811: mkdir d0/d23/daf/dbf/d10f/d119 0 2024-03-29T08:07:42.909 INFO:tasks.workunit.client.0.smithi037.stdout:0/854: creat d9/da/d1b/d22/dee/f129 x:0 0 0 2024-03-29T08:07:42.910 INFO:tasks.workunit.client.0.smithi037.stdout:3/868: symlink dd/d15/d18/d30/d115/d2d/d3a/d6c/db1/dc0/dda/l11f 0 2024-03-29T08:07:42.918 INFO:tasks.workunit.client.0.smithi037.stdout:1/938: rename d2/f3a to d2/d40/de8/d87/dbb/f148 0 2024-03-29T08:07:42.918 INFO:tasks.workunit.client.0.smithi037.stdout:2/812: unlink d0/d23/d44/dca/f75 0 2024-03-29T08:07:42.919 INFO:tasks.workunit.client.0.smithi037.stdout:8/846: creat d5/d10/d45/d4b/f110 x:0 0 0 2024-03-29T08:07:42.920 INFO:tasks.workunit.client.0.smithi037.stdout:0/855: creat d9/d74/d7d/d7e/d115/f12a x:0 0 0 2024-03-29T08:07:42.921 INFO:tasks.workunit.client.0.smithi037.stdout:3/869: read - dd/d15/d18/d30/d115/d3e/fe3 zero size 2024-03-29T08:07:42.923 INFO:tasks.workunit.client.0.smithi037.stdout:4/918: dwrite d3/d57/dec/ded/d1d/db/f16 [0,4194304] 0 2024-03-29T08:07:42.928 INFO:tasks.workunit.client.0.smithi037.stdout:0/856: read - d9/d74/fe7 zero size 2024-03-29T08:07:42.929 INFO:tasks.workunit.client.0.smithi037.stdout:2/813: mknod d0/d23/d44/dca/d2e/da4/c11a 0 2024-03-29T08:07:42.933 INFO:tasks.workunit.client.0.smithi037.stdout:0/857: chown d9/da/dd/d20/l68 101 1 2024-03-29T08:07:42.934 INFO:tasks.workunit.client.0.smithi037.stdout:4/919: rmdir d3/d4/d27/d126 39 2024-03-29T08:07:42.935 INFO:tasks.workunit.client.0.smithi037.stdout:8/847: creat d5/d10/d2d/d34/d50/d83/d46/f111 x:0 0 0 2024-03-29T08:07:42.937 INFO:tasks.workunit.client.0.smithi037.stdout:3/870: mkdir dd/d15/d18/d30/d115/d2d/d3a/d6c/db1/dc0/dda/ddb/d120 0 2024-03-29T08:07:42.948 INFO:tasks.workunit.client.0.smithi037.stdout:0/858: unlink d9/da/dd/d20/c73 0 2024-03-29T08:07:42.957 INFO:tasks.workunit.client.0.smithi037.stdout:1/939: dwrite d2/d6/dc/d18/dbd/f100 [0,4194304] 0 2024-03-29T08:07:42.962 INFO:tasks.workunit.client.0.smithi037.stdout:2/814: getdents d0/d23/daf/dbf/d10f/d55/d92 0 2024-03-29T08:07:42.965 INFO:tasks.workunit.client.0.smithi037.stdout:3/871: mkdir dd/d15/d18/d30/d115/d2d/d3a/d85/d121 0 2024-03-29T08:07:42.965 INFO:tasks.workunit.client.0.smithi037.stdout:8/848: creat d5/d10/d2d/d9c/df8/dc8/de6/f112 x:0 0 0 2024-03-29T08:07:42.971 INFO:tasks.workunit.client.0.smithi037.stdout:3/872: dread - dd/d5e/d7a/fee zero size 2024-03-29T08:07:42.974 INFO:tasks.workunit.client.0.smithi037.stdout:1/940: symlink d2/d6/dc/l149 0 2024-03-29T08:07:42.983 INFO:tasks.workunit.client.0.smithi037.stdout:1/941: chown d2/d6/dc/d18/d1d/d21/f60 349494 1 2024-03-29T08:07:42.998 INFO:tasks.workunit.client.0.smithi037.stdout:0/859: rename d9/da/dd/d20/l64 to d9/da/dd/d54/d92/l12b 0 2024-03-29T08:07:43.005 INFO:tasks.workunit.client.0.smithi037.stdout:8/849: link d5/d10/d2d/d9c/de0/fff d5/d10/d2d/df5/f113 0 2024-03-29T08:07:43.011 INFO:tasks.workunit.client.0.smithi037.stdout:1/942: creat d2/d6/dc/d18/d1d/dfc/f14a x:0 0 0 2024-03-29T08:07:43.022 INFO:tasks.workunit.client.0.smithi037.stdout:0/860: mkdir d9/da/dd/d20/d55/d5f/d9f/d108/d12c 0 2024-03-29T08:07:43.027 INFO:tasks.workunit.client.0.smithi037.stdout:2/815: rename d0/d23/daf/dbf/d10f/d4e/f4f to d0/f11b 0 2024-03-29T08:07:43.028 INFO:tasks.workunit.client.0.smithi037.stdout:4/920: dwrite d3/d57/dec/ded/d1d/db/d45/d1e/d59/dc1/fd3 [4194304,4194304] 0 2024-03-29T08:07:43.030 INFO:tasks.workunit.client.0.smithi037.stdout:8/850: mknod d5/d10/d2d/d9c/df8/d4c/c114 0 2024-03-29T08:07:43.035 INFO:tasks.workunit.client.0.smithi037.stdout:8/851: read d5/d10/d2d/d34/d50/d87/fd7 [4933136,3458] 0 2024-03-29T08:07:43.035 INFO:tasks.workunit.client.0.smithi037.stdout:4/921: creat d3/d57/dec/ded/d1d/db/d45/d1e/f135 x:0 0 0 2024-03-29T08:07:43.040 INFO:tasks.workunit.client.0.smithi037.stdout:2/816: link d0/d23/d44/dca/c2f d0/d23/daf/c11c 0 2024-03-29T08:07:43.042 INFO:tasks.workunit.client.0.smithi037.stdout:4/922: readlink d3/d4/d27/ld5 0 2024-03-29T08:07:43.044 INFO:tasks.workunit.client.0.smithi037.stdout:8/852: unlink d5/d10/d19/d3d/f7e 0 2024-03-29T08:07:43.050 INFO:tasks.workunit.client.0.smithi037.stdout:3/873: dwrite dd/d15/d18/d30/d115/f37 [4194304,4194304] 0 2024-03-29T08:07:43.051 INFO:tasks.workunit.client.0.smithi037.stdout:2/817: symlink d0/d23/d44/dca/d2e/d5f/da5/d110/l11d 0 2024-03-29T08:07:43.057 INFO:tasks.workunit.client.0.smithi037.stdout:3/874: readlink dd/d15/d18/d30/d115/d2d/d3a/d6c/l87 0 2024-03-29T08:07:43.057 INFO:tasks.workunit.client.0.smithi037.stdout:8/853: creat d5/d10/d2d/d9c/df8/dc8/de6/f115 x:0 0 0 2024-03-29T08:07:43.057 INFO:tasks.workunit.client.0.smithi037.stdout:4/923: mknod d3/d57/dec/ded/d1d/db/d45/d1e/d59/d60/c136 0 2024-03-29T08:07:43.061 INFO:tasks.workunit.client.0.smithi037.stdout:2/818: link d0/d23/daf/dbf/d10f/f47 d0/d53/f11e 0 2024-03-29T08:07:43.063 INFO:tasks.workunit.client.0.smithi037.stdout:4/924: chown d3/d57/dec/ded/d1d/db/d45/d93/d96/da0/ca5 61702 1 2024-03-29T08:07:43.069 INFO:tasks.workunit.client.0.smithi037.stdout:3/875: creat dd/d15/d18/d30/d115/d2d/d3a/d6c/db1/dc0/dda/d33/d84/f122 x:0 0 0 2024-03-29T08:07:43.076 INFO:tasks.workunit.client.0.smithi037.stdout:8/854: rmdir d5/d10/d2d/d9c/df8/dc8/de6 39 2024-03-29T08:07:43.077 INFO:tasks.workunit.client.0.smithi037.stdout:2/819: write d0/d23/d44/dca/d2e/da4/fec [4142361,128574] 0 2024-03-29T08:07:43.077 INFO:tasks.workunit.client.0.smithi037.stdout:3/876: readlink dd/d15/d18/d8e/le7 0 2024-03-29T08:07:43.086 INFO:tasks.workunit.client.0.smithi037.stdout:8/855: dread d5/d10/d2d/d9c/df8/d4c/d67/f10c [0,4194304] 0 2024-03-29T08:07:43.088 INFO:tasks.workunit.client.0.smithi037.stdout:3/877: unlink dd/d15/d18/d4b/da2/cc4 0 2024-03-29T08:07:43.090 INFO:tasks.workunit.client.0.smithi037.stdout:8/856: mknod d5/d10/d45/d4b/c116 0 2024-03-29T08:07:43.095 INFO:tasks.workunit.client.0.smithi037.stdout:8/857: chown d5/d10/d2d/d34/d50/d83/d46/d4e/lce 119 1 2024-03-29T08:07:43.098 INFO:tasks.workunit.client.0.smithi037.stdout:3/878: creat dd/f123 x:0 0 0 2024-03-29T08:07:43.099 INFO:tasks.workunit.client.0.smithi037.stdout:8/858: creat d5/d10b/f117 x:0 0 0 2024-03-29T08:07:43.100 INFO:tasks.workunit.client.0.smithi037.stdout:3/879: read dd/d15/f50 [1051233,32575] 0 2024-03-29T08:07:43.104 INFO:tasks.workunit.client.0.smithi037.stdout:3/880: mknod dd/d5e/d7a/c124 0 2024-03-29T08:07:43.105 INFO:tasks.workunit.client.0.smithi037.stdout:4/925: dread d3/d57/dec/ded/d1d/db/d45/f5a [0,4194304] 0 2024-03-29T08:07:43.110 INFO:tasks.workunit.client.0.smithi037.stdout:3/881: fsync dd/d5e/f45 0 2024-03-29T08:07:43.111 INFO:tasks.workunit.client.0.smithi037.stdout:4/926: mkdir d3/d57/dec/ded/d1d/d4f/d137 0 2024-03-29T08:07:43.113 INFO:tasks.workunit.client.0.smithi037.stdout:2/820: dwrite d0/d23/d44/dca/d2e/d38/f6b [0,4194304] 0 2024-03-29T08:07:43.114 INFO:tasks.workunit.client.0.smithi037.stdout:4/927: dread - d3/d57/d6a/f122 zero size 2024-03-29T08:07:43.122 INFO:tasks.workunit.client.0.smithi037.stdout:3/882: creat dd/d15/d18/d30/d115/d2d/d3a/d6c/db1/dc0/dda/d33/d106/f125 x:0 0 0 2024-03-29T08:07:43.127 INFO:tasks.workunit.client.0.smithi037.stdout:2/821: rename d0/d23/d44/dca/d2e/d5f/da5/d110 to d0/d23/daf/dbf/d10f/d55/d92/d11f 0 2024-03-29T08:07:43.129 INFO:tasks.workunit.client.0.smithi037.stdout:6/978: sync 2024-03-29T08:07:43.130 INFO:tasks.workunit.client.0.smithi037.stdout:3/883: mknod dd/d15/d18/d30/d115/d2d/d3a/d85/c126 0 2024-03-29T08:07:43.133 INFO:tasks.workunit.client.0.smithi037.stdout:4/928: write d3/d57/dec/ded/d1d/db/d45/fae [628322,1077] 0 2024-03-29T08:07:43.140 INFO:tasks.workunit.client.0.smithi037.stdout:3/884: read dd/d15/d18/d30/d115/d2d/d3a/fb6 [364789,83648] 0 2024-03-29T08:07:43.143 INFO:tasks.workunit.client.0.smithi037.stdout:2/822: mknod d0/d23/d44/dca/c120 0 2024-03-29T08:07:43.150 INFO:tasks.workunit.client.0.smithi037.stdout:8/859: dread d5/d10/d2d/d9c/df8/d4c/d67/d85/f8e [0,4194304] 0 2024-03-29T08:07:43.151 INFO:tasks.workunit.client.0.smithi037.stdout:3/885: creat dd/d15/d18/f127 x:0 0 0 2024-03-29T08:07:43.152 INFO:tasks.workunit.client.0.smithi037.stdout:4/929: symlink d3/d4/d27/d126/l138 0 2024-03-29T08:07:43.154 INFO:tasks.workunit.client.0.smithi037.stdout:2/823: mknod d0/d23/d44/d106/c121 0 2024-03-29T08:07:43.170 INFO:tasks.workunit.client.0.smithi037.stdout:6/979: dwrite d0/d19/d42/d60/f8b [0,4194304] 0 2024-03-29T08:07:43.176 INFO:tasks.workunit.client.0.smithi037.stdout:4/930: rename d3/d57/dec/ded/d1d/d11a to d3/d57/dec/ded/d1d/db/d72/d139 0 2024-03-29T08:07:43.185 INFO:tasks.workunit.client.0.smithi037.stdout:2/824: dwrite d0/d23/d44/dca/d2e/fd2 [0,4194304] 0 2024-03-29T08:07:43.187 INFO:tasks.workunit.client.0.smithi037.stdout:2/825: fsync d0/d23/d44/f67 0 2024-03-29T08:07:43.189 INFO:tasks.workunit.client.0.smithi037.stdout:2/826: chown d0/d23/d113 80837995 1 2024-03-29T08:07:43.193 INFO:tasks.workunit.client.0.smithi037.stdout:4/931: rename d3/d4/d27/l91 to d3/d57/dec/ded/d1d/db/d45/d1e/d59/d7d/d97/l13a 0 2024-03-29T08:07:43.212 INFO:tasks.workunit.client.0.smithi037.stdout:8/860: dwrite d5/f3a [0,4194304] 0 2024-03-29T08:07:43.216 INFO:tasks.workunit.client.0.smithi037.stdout:3/886: dwrite dd/d15/d18/faf [0,4194304] 0 2024-03-29T08:07:43.218 INFO:tasks.workunit.client.0.smithi037.stdout:8/861: write d5/d10/d19/d8d/fa8 [3872830,38267] 0 2024-03-29T08:07:43.229 INFO:tasks.workunit.client.0.smithi037.stdout:8/862: symlink d5/d10/d2d/d34/d50/d83/d46/d8a/d96/l118 0 2024-03-29T08:07:43.232 INFO:tasks.workunit.client.0.smithi037.stdout:3/887: creat dd/f128 x:0 0 0 2024-03-29T08:07:43.234 INFO:tasks.workunit.client.0.smithi037.stdout:3/888: chown dd/d15/d18/d4b 130549802 1 2024-03-29T08:07:43.241 INFO:tasks.workunit.client.0.smithi037.stdout:6/980: dread d0/d5/d1f/da8/f134 [0,4194304] 0 2024-03-29T08:07:43.242 INFO:tasks.workunit.client.0.smithi037.stdout:2/827: dwrite d0/d23/d44/fc5 [0,4194304] 0 2024-03-29T08:07:43.248 INFO:tasks.workunit.client.0.smithi037.stdout:8/863: rename d5/d10/d2d/d34/d50/d87/fd7 to d5/d10/d19/d3d/f119 0 2024-03-29T08:07:43.248 INFO:tasks.workunit.client.0.smithi037.stdout:3/889: chown dd/d15/d18/d30/d115/d2d/d3a/d6c/db1/dc0/dda/d33/d106/f125 22 1 2024-03-29T08:07:43.252 INFO:tasks.workunit.client.0.smithi037.stdout:8/864: fdatasync d5/d10/d2d/d9c/df8/f6b 0 2024-03-29T08:07:43.254 INFO:tasks.workunit.client.0.smithi037.stdout:3/890: stat dd/d15/d18/d4b/da2/led 0 2024-03-29T08:07:43.255 INFO:tasks.workunit.client.0.smithi037.stdout:1/943: sync 2024-03-29T08:07:43.256 INFO:tasks.workunit.client.0.smithi037.stdout:4/932: dwrite d3/d57/dec/ded/d1d/db/d45/d1e/d59/d7d/fc7 [0,4194304] 0 2024-03-29T08:07:43.262 INFO:tasks.workunit.client.0.smithi037.stdout:4/933: stat d3/d57/dec/ded/d1d/db/d45/fae 0 2024-03-29T08:07:43.266 INFO:tasks.workunit.client.0.smithi037.stdout:4/934: readlink d3/d54/d88/l10a 0 2024-03-29T08:07:43.268 INFO:tasks.workunit.client.0.smithi037.stdout:8/865: mkdir d5/d10/d2d/da7/d11a 0 2024-03-29T08:07:43.272 INFO:tasks.workunit.client.0.smithi037.stdout:1/944: mknod d2/d40/de8/c14b 0 2024-03-29T08:07:43.276 INFO:tasks.workunit.client.0.smithi037.stdout:8/866: stat d5/d10/d2d/d9c/df8/d4c/f8f 0 2024-03-29T08:07:43.283 INFO:tasks.workunit.client.0.smithi037.stdout:1/945: stat d2/d6/dc/d18/d1d/dfc/d109/f66 0 2024-03-29T08:07:43.285 INFO:tasks.workunit.client.0.smithi037.stdout:6/981: write d0/d19/f9b [1247504,35057] 0 2024-03-29T08:07:43.291 INFO:tasks.workunit.client.0.smithi037.stdout:2/828: write d0/d23/d70/f99 [792639,54765] 0 2024-03-29T08:07:43.295 INFO:tasks.workunit.client.0.smithi037.stdout:3/891: rename dd/d15/d18/d30/d115/d2d/d3a/fbf to dd/d15/d18/d30/d115/d2d/d3a/d6c/f129 0 2024-03-29T08:07:43.304 INFO:tasks.workunit.client.0.smithi037.stdout:0/861: sync 2024-03-29T08:07:43.305 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:07:43 smithi148 ceph-mon[44992]: pgmap v53: 65 pgs: 65 active+clean; 3.6 GiB data, 12 GiB used, 524 GiB / 536 GiB avail; 70 MiB/s rd, 141 MiB/s wr, 384 op/s 2024-03-29T08:07:43.310 INFO:tasks.workunit.client.0.smithi037.stdout:8/867: write d5/d10/f1d [678703,49930] 0 2024-03-29T08:07:43.311 INFO:tasks.workunit.client.0.smithi037.stdout:4/935: truncate d3/d80/df6/fe0 2001851 0 2024-03-29T08:07:43.314 INFO:tasks.workunit.client.0.smithi037.stdout:0/862: truncate d9/da/f11 4694471 0 2024-03-29T08:07:43.319 INFO:tasks.workunit.client.0.smithi037.stdout:6/982: write d0/d5/d1e/fc2 [1022108,23272] 0 2024-03-29T08:07:43.321 INFO:tasks.workunit.client.0.smithi037.stdout:0/863: write d9/da/dd/d20/d72/ff1 [968877,86927] 0 2024-03-29T08:07:43.321 INFO:tasks.workunit.client.0.smithi037.stdout:8/868: fdatasync d5/d10/d2d/d34/d50/f54 0 2024-03-29T08:07:43.322 INFO:tasks.workunit.client.0.smithi037.stdout:1/946: dwrite d2/d6/dc/d18/d1d/dfc/d109/d80/f107 [0,4194304] 0 2024-03-29T08:07:43.322 INFO:tasks.workunit.client.0.smithi037.stdout:4/936: rmdir d3/d57/dec/ded/d1d/dc2 39 2024-03-29T08:07:43.330 INFO:tasks.workunit.client.0.smithi037.stdout:4/937: dread - d3/d57/dec/ded/d1d/db/d45/d1e/d5f/f114 zero size 2024-03-29T08:07:43.340 INFO:tasks.workunit.client.0.smithi037.stdout:3/892: mknod dd/d15/d18/d30/d115/d2d/d3a/d85/d121/c12a 0 2024-03-29T08:07:43.340 INFO:tasks.workunit.client.0.smithi037.stdout:8/869: creat d5/d10/d2d/da7/f11b x:0 0 0 2024-03-29T08:07:43.341 INFO:tasks.workunit.client.0.smithi037.stdout:1/947: mknod d2/d6/dc/d18/d1d/d21/c14c 0 2024-03-29T08:07:43.356 INFO:tasks.workunit.client.0.smithi037.stdout:3/893: rename dd/d15/d18/d30/f100 to dd/d15/d18/d30/d115/d2d/f12b 0 2024-03-29T08:07:43.359 INFO:tasks.workunit.client.0.smithi037.stdout:8/870: link d5/d10/d2d/d34/d50/f54 d5/f11c 0 2024-03-29T08:07:43.363 INFO:tasks.workunit.client.0.smithi037.stdout:4/938: chown d3/d80/df6/fe0 207754785 1 2024-03-29T08:07:43.366 INFO:tasks.workunit.client.0.smithi037.stdout:0/864: dread f5 [0,4194304] 0 2024-03-29T08:07:43.378 INFO:tasks.workunit.client.0.smithi037.stdout:6/983: dwrite d0/d19/d34/d4e/d94/fed [0,4194304] 0 2024-03-29T08:07:43.381 INFO:tasks.workunit.client.0.smithi037.stdout:3/894: creat dd/d15/d18/d30/d115/d2d/d3a/d6c/db1/dc0/dda/d33/d75/f12c x:0 0 0 2024-03-29T08:07:43.389 INFO:tasks.workunit.client.0.smithi037.stdout:6/984: dread - d0/d5/d1e/fb5 zero size 2024-03-29T08:07:43.390 INFO:tasks.workunit.client.0.smithi037.stdout:4/939: link d3/d80/dcd/f90 d3/d57/dec/ded/d1d/db/d45/d93/d96/da0/f13b 0 2024-03-29T08:07:43.390 INFO:tasks.workunit.client.0.smithi037.stdout:0/865: truncate d9/da/dd/d20/d55/d5f/d9f/fd3 1792964 0 2024-03-29T08:07:43.394 INFO:tasks.workunit.client.0.smithi037.stdout:0/866: symlink d9/da/dd/d20/l12d 0 2024-03-29T08:07:43.395 INFO:tasks.workunit.client.0.smithi037.stdout:4/940: chown d3/d57/dec/ded/d1d/db/d45/d1e/f135 0 1 2024-03-29T08:07:43.398 INFO:tasks.workunit.client.0.smithi037.stdout:0/867: chown d9/da/dd/d54/d125/f42 1211984 1 2024-03-29T08:07:43.402 INFO:tasks.workunit.client.0.smithi037.stdout:4/941: mknod d3/d57/dec/ded/d1d/dc2/c13c 0 2024-03-29T08:07:43.406 INFO:tasks.workunit.client.0.smithi037.stdout:1/948: dread d2/d6/dc/d18/d1d/d21/f20 [0,4194304] 0 2024-03-29T08:07:43.414 INFO:tasks.workunit.client.0.smithi037.stdout:4/942: mknod d3/d57/dec/ded/d1d/db/d45/d93/d96/da0/db2/c13d 0 2024-03-29T08:07:43.415 INFO:tasks.workunit.client.0.smithi037.stdout:6/985: dwrite d0/d19/d34/d4e/df2/d105/f149 [0,4194304] 0 2024-03-29T08:07:43.415 INFO:tasks.workunit.client.0.smithi037.stdout:1/949: symlink d2/d6/dc/d58/d67/d11b/d11e/d86/l14d 0 2024-03-29T08:07:43.420 INFO:tasks.workunit.client.0.smithi037.stdout:4/943: chown d3/d57/dec/ded/d1d/db/f16 1 1 2024-03-29T08:07:43.421 INFO:tasks.workunit.client.0.smithi037.stdout:8/871: dread d5/d10/d19/d3d/f5d [0,4194304] 0 2024-03-29T08:07:43.422 INFO:tasks.workunit.client.0.smithi037.stdout:6/986: fdatasync d0/d19/d34/d4e/df2/f65 0 2024-03-29T08:07:43.426 INFO:tasks.workunit.client.0.smithi037.stdout:1/950: write d2/d6/dc/d7e/dd2/d113/f125 [3690660,55734] 0 2024-03-29T08:07:43.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:07:43 smithi037 ceph-mon[35162]: pgmap v53: 65 pgs: 65 active+clean; 3.6 GiB data, 12 GiB used, 524 GiB / 536 GiB avail; 70 MiB/s rd, 141 MiB/s wr, 384 op/s 2024-03-29T08:07:43.429 INFO:tasks.workunit.client.0.smithi037.stdout:8/872: chown d5/d10/d2d/d34/d50/d83/d46/fbb 777825729 1 2024-03-29T08:07:43.435 INFO:tasks.workunit.client.0.smithi037.stdout:4/944: creat d3/d57/dec/ded/d1d/db/d45/d1e/d59/d73/f13e x:0 0 0 2024-03-29T08:07:43.436 INFO:tasks.workunit.client.0.smithi037.stdout:1/951: stat d2/d6/dc/d58/d67/df6 0 2024-03-29T08:07:43.440 INFO:tasks.workunit.client.0.smithi037.stdout:8/873: creat d5/d78/dac/f11d x:0 0 0 2024-03-29T08:07:43.442 INFO:tasks.workunit.client.0.smithi037.stdout:0/868: dread d9/da/dd/d20/d38/d53/f62 [0,4194304] 0 2024-03-29T08:07:43.443 INFO:tasks.workunit.client.0.smithi037.stdout:4/945: dread - d3/f104 zero size 2024-03-29T08:07:43.448 INFO:tasks.workunit.client.0.smithi037.stdout:1/952: creat d2/d6/dc/d7e/dd2/f14e x:0 0 0 2024-03-29T08:07:43.448 INFO:tasks.workunit.client.0.smithi037.stdout:6/987: dwrite d0/d19/d42/d60/d62/fc6 [0,4194304] 0 2024-03-29T08:07:43.451 INFO:tasks.workunit.client.0.smithi037.stdout:1/953: write d2/d6/dc/d18/d1d/f118 [385220,52787] 0 2024-03-29T08:07:43.453 INFO:tasks.workunit.client.0.smithi037.stdout:0/869: symlink d9/da/d1b/d22/dee/df4/l12e 0 2024-03-29T08:07:43.464 INFO:tasks.workunit.client.0.smithi037.stdout:6/988: chown d0/d5/f14 184 1 2024-03-29T08:07:43.470 INFO:tasks.workunit.client.0.smithi037.stdout:0/870: mkdir d9/da/dd/d20/d55/d5f/d9f/d108/d12c/d12f 0 2024-03-29T08:07:43.483 INFO:tasks.workunit.client.0.smithi037.stdout:0/871: write d9/da/f11b [965714,71479] 0 2024-03-29T08:07:43.497 INFO:tasks.workunit.client.0.smithi037.stdout:8/874: dwrite d5/d10/d2d/d9c/df8/dc8/fe3 [0,4194304] 0 2024-03-29T08:07:43.500 INFO:tasks.workunit.client.0.smithi037.stdout:1/954: dwrite d2/d6/fe3 [0,4194304] 0 2024-03-29T08:07:43.501 INFO:tasks.workunit.client.0.smithi037.stdout:6/989: dwrite d0/d19/d42/d60/f7d [0,4194304] 0 2024-03-29T08:07:43.509 INFO:tasks.workunit.client.0.smithi037.stdout:1/955: chown d2/d6/dc/d58/d67/d11b/d11e/d7c/f105 94 1 2024-03-29T08:07:43.509 INFO:tasks.workunit.client.0.smithi037.stdout:0/872: link d9/da/f8d d9/da/dd/d20/d38/d53/f130 0 2024-03-29T08:07:43.512 INFO:tasks.workunit.client.0.smithi037.stdout:6/990: unlink d0/d19/d34/cd5 0 2024-03-29T08:07:43.512 INFO:tasks.workunit.client.0.smithi037.stdout:1/956: fsync d2/d6/dc/d7e/f115 0 2024-03-29T08:07:43.513 INFO:tasks.workunit.client.0.smithi037.stdout:8/875: mkdir d5/d10/d2d/d9c/df8/d4c/d67/d70/d11e 0 2024-03-29T08:07:43.529 INFO:tasks.workunit.client.0.smithi037.stdout:8/876: write d5/d10/d2d/d9c/df8/dc8/de6/f112 [19028,53242] 0 2024-03-29T08:07:43.529 INFO:tasks.workunit.client.0.smithi037.stdout:0/873: mkdir d9/da/d131 0 2024-03-29T08:07:43.532 INFO:tasks.workunit.client.0.smithi037.stdout:1/957: creat d2/d6/dc/d18/dbd/f14f x:0 0 0 2024-03-29T08:07:43.534 INFO:tasks.workunit.client.0.smithi037.stdout:0/874: read - d9/d74/fe7 zero size 2024-03-29T08:07:43.539 INFO:tasks.workunit.client.0.smithi037.stdout:1/958: read d2/d6/d13/f93 [33755,56965] 0 2024-03-29T08:07:43.546 INFO:tasks.workunit.client.0.smithi037.stdout:6/991: dwrite d0/d19/d42/d60/f123 [0,4194304] 0 2024-03-29T08:07:43.547 INFO:tasks.workunit.client.0.smithi037.stdout:0/875: creat d9/da/dd/d20/d38/d8a/dfd/f132 x:0 0 0 2024-03-29T08:07:43.548 INFO:tasks.workunit.client.0.smithi037.stdout:1/959: creat d2/d40/de8/daf/f150 x:0 0 0 2024-03-29T08:07:43.553 INFO:tasks.workunit.client.0.smithi037.stdout:0/876: chown d9/da/dd/d54/d125 1412807604 1 2024-03-29T08:07:43.566 INFO:tasks.workunit.client.0.smithi037.stdout:1/960: fsync d2/d6/dc/d58/f59 0 2024-03-29T08:07:43.572 INFO:tasks.workunit.client.0.smithi037.stdout:0/877: rename d9/da/dd/d20/d38/c80 to d9/da/d1b/d22/dee/d110/c133 0 2024-03-29T08:07:43.577 INFO:tasks.workunit.client.0.smithi037.stdout:1/961: symlink d2/d6/dc/d18/d1d/d5e/l151 0 2024-03-29T08:07:43.581 INFO:tasks.workunit.client.0.smithi037.stdout:1/962: chown d2/d6/dc/d18/d1d/fa9 31815 1 2024-03-29T08:07:43.583 INFO:tasks.workunit.client.0.smithi037.stdout:8/877: dread d5/d10/d2d/d34/d50/ddc/ff9 [0,4194304] 0 2024-03-29T08:07:43.586 INFO:tasks.workunit.client.0.smithi037.stdout:0/878: rename d9/da/d1b to d9/da/dd/d20/d72/d134 0 2024-03-29T08:07:43.590 INFO:tasks.workunit.client.0.smithi037.stdout:1/963: mknod d2/d6/dc/d18/d1d/dfc/d109/c152 0 2024-03-29T08:07:43.590 INFO:tasks.workunit.client.0.smithi037.stdout:6/992: dwrite d0/d5/f2a [0,4194304] 0 2024-03-29T08:07:43.592 INFO:tasks.workunit.client.0.smithi037.stdout:1/964: chown d2/le 396710 1 2024-03-29T08:07:43.592 INFO:tasks.workunit.client.0.smithi037.stdout:8/878: symlink d5/d10/d2d/d9c/dfb/l11f 0 2024-03-29T08:07:43.596 INFO:tasks.workunit.client.0.smithi037.stdout:6/993: creat d0/d6a/dd6/f14b x:0 0 0 2024-03-29T08:07:43.597 INFO:tasks.workunit.client.0.smithi037.stdout:8/879: rmdir d5/d10/d19/d8d 39 2024-03-29T08:07:43.608 INFO:tasks.workunit.client.0.smithi037.stdout:1/965: truncate d2/d6/dc/d18/d1d/dfc/d109/fc4 4591277 0 2024-03-29T08:07:43.612 INFO:tasks.workunit.client.0.smithi037.stdout:6/994: truncate d0/d20/ddd/fe8 3415783 0 2024-03-29T08:07:43.613 INFO:tasks.workunit.client.0.smithi037.stdout:2/829: sync 2024-03-29T08:07:43.615 INFO:tasks.workunit.client.0.smithi037.stdout:6/995: readlink d0/d6a/l83 0 2024-03-29T08:07:43.619 INFO:tasks.workunit.client.0.smithi037.stdout:2/830: chown d0/d23/d44/dca/d2e/d5f/d6f/d5e/fb1 184 1 2024-03-29T08:07:43.620 INFO:tasks.workunit.client.0.smithi037.stdout:1/966: creat d2/d6/dc/d18/d1d/dfc/d109/d80/dfb/f153 x:0 0 0 2024-03-29T08:07:43.622 INFO:tasks.workunit.client.0.smithi037.stdout:8/880: write d5/d10/d2d/d34/d50/ddc/fdf [77445,84455] 0 2024-03-29T08:07:43.624 INFO:tasks.workunit.client.0.smithi037.stdout:2/831: rmdir d0/d23/d44/d62 39 2024-03-29T08:07:43.628 INFO:tasks.workunit.client.0.smithi037.stdout:2/832: mknod d0/d23/d44/df6/c122 0 2024-03-29T08:07:43.630 INFO:tasks.workunit.client.0.smithi037.stdout:0/879: dread d9/da/dd/d20/f9e [0,4194304] 0 2024-03-29T08:07:43.630 INFO:tasks.workunit.client.0.smithi037.stdout:8/881: mkdir d5/d10/d2d/d34/d50/d120 0 2024-03-29T08:07:43.632 INFO:tasks.workunit.client.0.smithi037.stdout:2/833: fsync d0/d23/daf/dbf/fde 0 2024-03-29T08:07:43.633 INFO:tasks.workunit.client.0.smithi037.stdout:0/880: chown d9/d74/fa9 3651 1 2024-03-29T08:07:43.640 INFO:tasks.workunit.client.0.smithi037.stdout:8/882: fdatasync d5/d10/d2d/d9c/df8/d4c/d67/fd4 0 2024-03-29T08:07:43.641 INFO:tasks.workunit.client.0.smithi037.stdout:0/881: mknod d9/da/dd/d20/d72/df7/c135 0 2024-03-29T08:07:43.644 INFO:tasks.workunit.client.0.smithi037.stdout:2/834: mknod d0/d23/d44/c123 0 2024-03-29T08:07:43.647 INFO:tasks.workunit.client.0.smithi037.stdout:8/883: stat d5/d10/d2d/d34/d50/d83/f5f 0 2024-03-29T08:07:43.649 INFO:tasks.workunit.client.0.smithi037.stdout:1/967: dwrite d2/d6/dc/d18/d1d/dfc/d109/d80/dfb/f153 [0,4194304] 0 2024-03-29T08:07:43.649 INFO:tasks.workunit.client.0.smithi037.stdout:0/882: symlink d9/d74/l136 0 2024-03-29T08:07:43.650 INFO:tasks.workunit.client.0.smithi037.stdout:2/835: creat d0/d53/d89/f124 x:0 0 0 2024-03-29T08:07:43.650 INFO:tasks.workunit.client.0.smithi037.stdout:2/836: chown d0 5422 1 2024-03-29T08:07:43.653 INFO:tasks.workunit.client.0.smithi037.stdout:8/884: mknod d5/d10/dfa/c121 0 2024-03-29T08:07:43.659 INFO:tasks.workunit.client.0.smithi037.stdout:2/837: dread d0/d23/d70/f79 [0,4194304] 0 2024-03-29T08:07:43.659 INFO:tasks.workunit.client.0.smithi037.stdout:1/968: truncate d2/d6/dc/d18/f8d 4336880 0 2024-03-29T08:07:43.664 INFO:tasks.workunit.client.0.smithi037.stdout:1/969: dread - d2/d6/dc/d58/d67/d11b/d11e/d86/f141 zero size 2024-03-29T08:07:43.665 INFO:tasks.workunit.client.0.smithi037.stdout:6/996: dread d0/d6a/d7b/fa1 [0,4194304] 0 2024-03-29T08:07:43.670 INFO:tasks.workunit.client.0.smithi037.stdout:1/970: fdatasync d2/d6/dc/d18/d1d/dfc/d109/d80/f107 0 2024-03-29T08:07:43.674 INFO:tasks.workunit.client.0.smithi037.stdout:1/971: chown d2/d6/dc/d18/d1d/dfc/d109/d80/db0 83262 1 2024-03-29T08:07:43.702 INFO:tasks.workunit.client.0.smithi037.stdout:0/883: truncate d9/da/dd/d13/f36 681720 0 2024-03-29T08:07:43.732 INFO:tasks.workunit.client.0.smithi037.stdout:6/997: creat d0/d19/d34/d4e/df2/d131/f14c x:0 0 0 2024-03-29T08:07:43.737 INFO:tasks.workunit.client.0.smithi037.stdout:8/885: write d5/d10/d45/f7f [48279,28752] 0 2024-03-29T08:07:43.738 INFO:tasks.workunit.client.0.smithi037.stdout:1/972: rmdir d2/d6/dc/d58/d67/d11b/d11e/d86/d10d 39 2024-03-29T08:07:43.744 INFO:tasks.workunit.client.0.smithi037.stdout:2/838: truncate d0/f11b 4857571 0 2024-03-29T08:07:43.754 INFO:tasks.workunit.client.0.smithi037.stdout:3/895: sync 2024-03-29T08:07:43.757 INFO:tasks.workunit.client.0.smithi037.stdout:6/998: dwrite d0/d6a/dd6/f11c [8388608,4194304] 0 2024-03-29T08:07:43.765 INFO:tasks.workunit.client.0.smithi037.stdout:3/896: mknod dd/d15/d18/d30/d115/d2d/d3a/d6c/db1/dc0/dda/d33/d83/c12d 0 2024-03-29T08:07:43.766 INFO:tasks.workunit.client.0.smithi037.stdout:8/886: getdents d5/d10/d2d/d9c/df8/d4c/d67/d70 0 2024-03-29T08:07:43.769 INFO:tasks.workunit.client.0.smithi037.stdout:3/897: chown dd/d15/d18/fb2 13694536 1 2024-03-29T08:07:43.771 INFO:tasks.workunit.client.0.smithi037.stdout:8/887: fdatasync d5/d10/d2d/d9c/df8/f77 0 2024-03-29T08:07:43.781 INFO:tasks.workunit.client.0.smithi037.stdout:8/888: symlink d5/d10/d2d/d9c/df8/d102/l122 0 2024-03-29T08:07:43.806 INFO:tasks.workunit.client.0.smithi037.stdout:3/898: dread dd/d15/d18/d30/d115/d2d/d3a/d6c/db1/dc0/f11d [0,4194304] 0 2024-03-29T08:07:43.808 INFO:tasks.workunit.client.0.smithi037.stdout:3/899: write dd/d15/d18/faf [1563634,75270] 0 2024-03-29T08:07:43.812 INFO:tasks.workunit.client.0.smithi037.stdout:4/946: sync 2024-03-29T08:07:43.814 INFO:tasks.workunit.client.0.smithi037.stdout:3/900: dread - dd/d15/d18/d30/d115/d2d/d3a/d6c/db1/dc0/dda/ddb/f108 zero size 2024-03-29T08:07:43.818 INFO:tasks.workunit.client.0.smithi037.stdout:4/947: dread - d3/d57/dec/ded/d1d/db/d45/d1e/d59/d7d/fda zero size 2024-03-29T08:07:43.822 INFO:tasks.workunit.client.0.smithi037.stdout:3/901: creat dd/d15/d18/d30/d115/d2d/d3a/d6c/d10e/f12e x:0 0 0 2024-03-29T08:07:43.823 INFO:tasks.workunit.client.0.smithi037.stdout:4/948: truncate d3/d57/dec/ded/d1d/f12d 445699 0 2024-03-29T08:07:43.833 INFO:tasks.workunit.client.0.smithi037.stdout:1/973: write d2/d6/dc/d18/d1d/dfc/d109/d80/dfb/f102 [2862418,35220] 0 2024-03-29T08:07:43.844 INFO:tasks.workunit.client.0.smithi037.stdout:6/999: truncate d0/d5/fa7 3449713 0 2024-03-29T08:07:43.847 INFO:tasks.workunit.client.0.smithi037.stdout:2/839: dwrite d0/d23/d44/dca/d2e/d5f/fe1 [0,4194304] 0 2024-03-29T08:07:43.855 INFO:tasks.workunit.client.0.smithi037.stdout:0/884: link d9/da/dd/d20/d55/dac/ffa d9/da/dd/d13/f137 0 2024-03-29T08:07:43.858 INFO:tasks.workunit.client.0.smithi037.stdout:2/840: mkdir d0/d23/daf/dbf/d10f/d55/d92/d10a/d125 0 2024-03-29T08:07:43.863 INFO:tasks.workunit.client.0.smithi037.stdout:8/889: rename d5/ca9 to d5/d10/c123 0 2024-03-29T08:07:43.869 INFO:tasks.workunit.client.0.smithi037.stdout:0/885: dread d9/d74/fa9 [0,4194304] 0 2024-03-29T08:07:43.878 INFO:tasks.workunit.client.0.smithi037.stdout:3/902: rename dd/d15/d18/d30/d115/d2d/d3a/d6c/db1/dc0/dda/d33/d84/fd0 to dd/d15/d18/d30/d115/d2d/d3a/d6c/db1/dc0/dda/d33/d75/d8f/f12f 0 2024-03-29T08:07:43.882 INFO:tasks.workunit.client.0.smithi037.stdout:4/949: dwrite d3/d57/dec/ded/d1d/db/d45/d1e/d59/d73/f10f [0,4194304] 0 2024-03-29T08:07:43.888 INFO:tasks.workunit.client.0.smithi037.stdout:8/890: creat d5/d10/d2d/d9c/df8/d4c/d67/d70/d11e/f124 x:0 0 0 2024-03-29T08:07:43.890 INFO:tasks.workunit.client.0.smithi037.stdout:2/841: dwrite d0/d23/d70/f99 [0,4194304] 0 2024-03-29T08:07:43.891 INFO:tasks.workunit.client.0.smithi037.stdout:1/974: dwrite d2/d6/dc/fd [0,4194304] 0 2024-03-29T08:07:43.899 INFO:tasks.workunit.client.0.smithi037.stdout:3/903: write dd/d15/d18/d30/d115/d2d/d3a/d6c/db1/dc0/dda/d33/d97/dd6/df5/f10a [476000,7125] 0 2024-03-29T08:07:43.900 INFO:tasks.workunit.client.0.smithi037.stdout:1/975: chown d2/d6/dc/c9b 23 1 2024-03-29T08:07:43.903 INFO:tasks.workunit.client.0.smithi037.stdout:8/891: creat d5/d10/d2d/d9c/da1/dbc/f125 x:0 0 0 2024-03-29T08:07:43.914 INFO:tasks.workunit.client.0.smithi037.stdout:1/976: chown d2/d6/dc/d58/d67/d11b/d11e/d86/cd9 42729 1 2024-03-29T08:07:43.918 INFO:tasks.workunit.client.0.smithi037.stdout:4/950: truncate d3/d57/dec/ded/d1d/db/d72/d98/fbf 690698 0 2024-03-29T08:07:43.921 INFO:tasks.workunit.client.0.smithi037.stdout:0/886: dwrite d9/da/dd/f76 [0,4194304] 0 2024-03-29T08:07:43.922 INFO:tasks.workunit.client.0.smithi037.stdout:8/892: mknod d5/d10/d2d/d34/d50/d87/c126 0 2024-03-29T08:07:43.926 INFO:tasks.workunit.client.0.smithi037.stdout:0/887: stat d9/da/dd/d20/d38/d53 0 2024-03-29T08:07:43.930 INFO:tasks.workunit.client.0.smithi037.stdout:8/893: mkdir d5/d10/d2d/d9c/df8/d102/d127 0 2024-03-29T08:07:43.931 INFO:tasks.workunit.client.0.smithi037.stdout:0/888: dread - d9/da/dd/d20/d38/d53/f120 zero size 2024-03-29T08:07:43.933 INFO:tasks.workunit.client.0.smithi037.stdout:2/842: dwrite d0/d23/d44/dca/d2e/d5f/f8d [0,4194304] 0 2024-03-29T08:07:43.940 INFO:tasks.workunit.client.0.smithi037.stdout:2/843: stat d0/d23/d44/dca/d2e/d38/l73 0 2024-03-29T08:07:43.945 INFO:tasks.workunit.client.0.smithi037.stdout:1/977: link d2/d6/dc/d5d/l131 d2/d6/dc/d7e/l154 0 2024-03-29T08:07:43.955 INFO:tasks.workunit.client.0.smithi037.stdout:4/951: dwrite d3/d4/f132 [0,4194304] 0 2024-03-29T08:07:43.965 INFO:tasks.workunit.client.0.smithi037.stdout:4/952: chown d3/d57/dec/ded/d1d/dad/dfb/f110 383157994 1 2024-03-29T08:07:43.965 INFO:tasks.workunit.client.0.smithi037.stdout:8/894: dwrite d5/ffe [0,4194304] 0 2024-03-29T08:07:43.974 INFO:tasks.workunit.client.0.smithi037.stdout:0/889: dwrite d9/da/dd/d20/f9e [4194304,4194304] 0 2024-03-29T08:07:43.975 INFO:tasks.workunit.client.0.smithi037.stdout:4/953: rmdir d3/d4/d27 39 2024-03-29T08:07:43.980 INFO:tasks.workunit.client.0.smithi037.stdout:1/978: dwrite d2/f94 [0,4194304] 0 2024-03-29T08:07:43.988 INFO:tasks.workunit.client.0.smithi037.stdout:1/979: mknod d2/d6/dc/c155 0 2024-03-29T08:07:43.992 INFO:tasks.workunit.client.0.smithi037.stdout:8/895: dwrite d5/d10b/f117 [0,4194304] 0 2024-03-29T08:07:43.992 INFO:tasks.workunit.client.0.smithi037.stdout:0/890: read d9/da/dd/d54/d125/f57 [2998926,69927] 0 2024-03-29T08:07:43.994 INFO:tasks.workunit.client.0.smithi037.stdout:2/844: dread d0/d23/d44/dca/d7d/fdc [0,4194304] 0 2024-03-29T08:07:43.997 INFO:tasks.workunit.client.0.smithi037.stdout:1/980: write d2/d6/dc/d7e/dd2/d113/f125 [1456110,121947] 0 2024-03-29T08:07:43.998 INFO:tasks.workunit.client.0.smithi037.stdout:0/891: chown d9/d74/d7d/d7e/cb7 131360595 1 2024-03-29T08:07:44.004 INFO:tasks.workunit.client.0.smithi037.stdout:0/892: chown d9/da/dd/d13/f100 7 1 2024-03-29T08:07:44.004 INFO:tasks.workunit.client.0.smithi037.stdout:8/896: getdents d5/d10/d2d/d34/d50/d87/dbd 0 2024-03-29T08:07:44.007 INFO:tasks.workunit.client.0.smithi037.stdout:1/981: dread - d2/d6/dc/d7e/f108 zero size 2024-03-29T08:07:44.014 INFO:tasks.workunit.client.0.smithi037.stdout:2/845: symlink d0/d23/d44/dca/d2e/d9c/df3/l126 0 2024-03-29T08:07:44.018 INFO:tasks.workunit.client.0.smithi037.stdout:8/897: symlink d5/d10/d2d/d34/d50/d83/d46/d4e/l128 0 2024-03-29T08:07:44.020 INFO:tasks.workunit.client.0.smithi037.stdout:0/893: mknod d9/da/dd/d54/d92/dd4/c138 0 2024-03-29T08:07:44.022 INFO:tasks.workunit.client.0.smithi037.stdout:1/982: symlink d2/d6/dc/d18/d1d/d21/d99/d138/l156 0 2024-03-29T08:07:44.024 INFO:tasks.workunit.client.0.smithi037.stdout:2/846: read - d0/d23/d44/d62/fb5 zero size 2024-03-29T08:07:44.024 INFO:tasks.workunit.client.0.smithi037.stdout:4/954: dread d3/d57/dec/ded/d1d/f3f [0,4194304] 0 2024-03-29T08:07:44.026 INFO:tasks.workunit.client.0.smithi037.stdout:8/898: write d5/d10/d2d/d9c/df8/dc8/f105 [244420,84891] 0 2024-03-29T08:07:44.030 INFO:tasks.workunit.client.0.smithi037.stdout:2/847: write d0/d23/d44/dca/d2e/d38/f6b [1279179,67] 0 2024-03-29T08:07:44.035 INFO:tasks.workunit.client.0.smithi037.stdout:4/955: creat d3/d54/d88/f13f x:0 0 0 2024-03-29T08:07:44.036 INFO:tasks.workunit.client.0.smithi037.stdout:1/983: rename d2/d6/dc/d58/d67/d11b/f11f to d2/d40/de8/daf/f157 0 2024-03-29T08:07:44.038 INFO:tasks.workunit.client.0.smithi037.stdout:4/956: readlink d3/d80/dcd/ldc 0 2024-03-29T08:07:44.038 INFO:tasks.workunit.client.0.smithi037.stdout:8/899: chown d5/d10/d2d/d34/d50/d83/d46/d4e/l128 3896 1 2024-03-29T08:07:44.042 INFO:tasks.workunit.client.0.smithi037.stdout:1/984: rmdir d2/d6/dc/d18/d1d/dfc/d109 39 2024-03-29T08:07:44.043 INFO:tasks.workunit.client.0.smithi037.stdout:4/957: readlink d3/d57/dec/ded/d1d/db/d45/d1e/d59/d79/laf 0 2024-03-29T08:07:44.044 INFO:tasks.workunit.client.0.smithi037.stdout:8/900: creat d5/d10/d2d/f129 x:0 0 0 2024-03-29T08:07:44.051 INFO:tasks.workunit.client.0.smithi037.stdout:4/958: mkdir d3/d57/dec/ded/d1d/db/d45/d93/d96/da0/dc9/dd8/d140 0 2024-03-29T08:07:44.053 INFO:tasks.workunit.client.0.smithi037.stdout:1/985: truncate d2/d6/dc/d18/d1d/dfc/d109/da1/f96 1236471 0 2024-03-29T08:07:44.054 INFO:tasks.workunit.client.0.smithi037.stdout:0/894: dread d9/da/dd/d20/d72/d134/fb0 [0,4194304] 0 2024-03-29T08:07:44.057 INFO:tasks.workunit.client.0.smithi037.stdout:8/901: link d5/d10/d2d/d9c/df8/d4c/f6e d5/d10/d2d/d34/d50/d87/dbd/f12a 0 2024-03-29T08:07:44.064 INFO:tasks.workunit.client.0.smithi037.stdout:4/959: link d3/c6 d3/d57/dec/ded/d1d/db/d45/d93/d96/c141 0 2024-03-29T08:07:44.066 INFO:tasks.workunit.client.0.smithi037.stdout:1/986: getdents d2/d6/dc/d58/d67/d11b/d11e/d86/d10d 0 2024-03-29T08:07:44.073 INFO:tasks.workunit.client.0.smithi037.stdout:8/902: write d5/d10/d2d/f40 [646587,125330] 0 2024-03-29T08:07:44.080 INFO:tasks.workunit.client.0.smithi037.stdout:8/903: creat d5/d10/d2d/d9c/df8/d4c/d67/d85/f12b x:0 0 0 2024-03-29T08:07:44.085 INFO:tasks.workunit.client.0.smithi037.stdout:0/895: dwrite d9/da/dd/d20/d72/d134/fb0 [0,4194304] 0 2024-03-29T08:07:44.086 INFO:tasks.workunit.client.0.smithi037.stdout:4/960: dwrite d3/d80/feb [0,4194304] 0 2024-03-29T08:07:44.088 INFO:tasks.workunit.client.0.smithi037.stdout:8/904: mkdir d5/d10/d2d/d9c/df8/d12c 0 2024-03-29T08:07:44.093 INFO:tasks.workunit.client.0.smithi037.stdout:8/905: readlink d5/d10/d45/d4b/l52 0 2024-03-29T08:07:44.093 INFO:tasks.workunit.client.0.smithi037.stdout:0/896: chown d9/da/dd/d20/d72/d134/d22/l52 13082173 1 2024-03-29T08:07:44.093 INFO:tasks.workunit.client.0.smithi037.stdout:1/987: dwrite d2/d40/de8/d87/fce [0,4194304] 0 2024-03-29T08:07:44.096 INFO:tasks.workunit.client.0.smithi037.stdout:4/961: mknod d3/d57/dec/ded/d89/c142 0 2024-03-29T08:07:44.099 INFO:tasks.workunit.client.0.smithi037.stdout:1/988: truncate d2/d6/dc/d5d/f128 880099 0 2024-03-29T08:07:44.103 INFO:tasks.workunit.client.0.smithi037.stdout:4/962: read d3/d57/dec/ded/d1d/db/d45/fce [1887333,87725] 0 2024-03-29T08:07:44.107 INFO:tasks.workunit.client.0.smithi037.stdout:0/897: creat d9/da/dd/d20/f139 x:0 0 0 2024-03-29T08:07:44.115 INFO:tasks.workunit.client.0.smithi037.stdout:8/906: write d5/d10/d2d/d34/d50/fc0 [635731,96706] 0 2024-03-29T08:07:44.121 INFO:tasks.workunit.client.0.smithi037.stdout:0/898: fdatasync d9/da/dd/d20/d38/d8a/f9a 0 2024-03-29T08:07:44.123 INFO:tasks.workunit.client.0.smithi037.stdout:0/899: chown d9/da/dd/d20/l68 14773827 1 2024-03-29T08:07:44.125 INFO:tasks.workunit.client.0.smithi037.stdout:1/989: dread d2/d6/dc/d18/d1d/dfc/d109/d80/f8b [0,4194304] 0 2024-03-29T08:07:44.126 INFO:tasks.workunit.client.0.smithi037.stdout:0/900: chown d9/da/dd/d54/db5/d106 185499 1 2024-03-29T08:07:44.129 INFO:tasks.workunit.client.0.smithi037.stdout:1/990: write d2/f94 [2166937,1335] 0 2024-03-29T08:07:44.139 INFO:tasks.workunit.client.0.smithi037.stdout:1/991: rename d2/d6/d13/c10b to d2/d40/de8/daf/c158 0 2024-03-29T08:07:44.141 INFO:tasks.workunit.client.0.smithi037.stdout:4/963: dwrite d3/d57/dec/ded/d1d/db/d45/d1e/d59/d7d/f9f [4194304,4194304] 0 2024-03-29T08:07:44.141 INFO:tasks.workunit.client.0.smithi037.stdout:8/907: dwrite d5/fb [0,4194304] 0 2024-03-29T08:07:44.149 INFO:tasks.workunit.client.0.smithi037.stdout:4/964: fdatasync d3/d57/dec/ded/d1d/db/d45/d1e/f4a 0 2024-03-29T08:07:44.149 INFO:tasks.workunit.client.0.smithi037.stdout:8/908: chown d5/d10/l36 36 1 2024-03-29T08:07:44.154 INFO:tasks.workunit.client.0.smithi037.stdout:4/965: dread - d3/d57/fee zero size 2024-03-29T08:07:44.157 INFO:tasks.workunit.client.0.smithi037.stdout:8/909: creat d5/d10/d2d/d9c/df8/dc8/de6/f12d x:0 0 0 2024-03-29T08:07:44.161 INFO:tasks.workunit.client.0.smithi037.stdout:0/901: dread d9/da/dd/d54/f56 [0,4194304] 0 2024-03-29T08:07:44.174 INFO:tasks.workunit.client.0.smithi037.stdout:8/910: truncate d5/d10/d2d/d9c/df8/d4c/f8f 4690279 0 2024-03-29T08:07:44.175 INFO:tasks.workunit.client.0.smithi037.stdout:4/966: write d3/d57/dec/f113 [89283,79701] 0 2024-03-29T08:07:44.180 INFO:tasks.workunit.client.0.smithi037.stdout:8/911: dread - d5/f11c zero size 2024-03-29T08:07:44.183 INFO:tasks.workunit.client.0.smithi037.stdout:4/967: creat d3/d54/d12c/f143 x:0 0 0 2024-03-29T08:07:44.190 INFO:tasks.workunit.client.0.smithi037.stdout:4/968: write d3/d80/dcd/d8f/f11c [1009462,18360] 0 2024-03-29T08:07:44.190 INFO:tasks.workunit.client.0.smithi037.stdout:0/902: dwrite d9/da/dd/d54/d125/f107 [0,4194304] 0 2024-03-29T08:07:44.199 INFO:tasks.workunit.client.0.smithi037.stdout:0/903: write d9/da/dd/d20/d55/fe3 [1314534,107845] 0 2024-03-29T08:07:44.208 INFO:tasks.workunit.client.0.smithi037.stdout:4/969: getdents d3/d57/dec/ded/d1d/db/d45/d93 0 2024-03-29T08:07:44.224 INFO:tasks.workunit.client.0.smithi037.stdout:0/904: link c4 d9/da/dd/d20/d72/d134/d22/dee/c13a 0 2024-03-29T08:07:44.225 INFO:tasks.workunit.client.0.smithi037.stdout:8/912: dread d5/d10/d2d/d9c/df8/f6b [0,4194304] 0 2024-03-29T08:07:44.229 INFO:tasks.workunit.client.0.smithi037.stdout:0/905: write d9/da/dd/d20/d72/fcd [1504081,52007] 0 2024-03-29T08:07:44.234 INFO:tasks.workunit.client.0.smithi037.stdout:8/913: unlink d5/d10/d2d/d9c/df8/d4c/d67/l7d 0 2024-03-29T08:07:44.235 INFO:tasks.workunit.client.0.smithi037.stdout:0/906: mknod d9/d74/de6/c13b 0 2024-03-29T08:07:44.237 INFO:tasks.workunit.client.0.smithi037.stdout:4/970: dread d3/d57/dec/ded/f49 [0,4194304] 0 2024-03-29T08:07:44.246 INFO:tasks.workunit.client.0.smithi037.stdout:8/914: rename d5/d10/d2d/d34/d50/d83/d46/d8a/d96/fef to d5/d78/dac/f12e 0 2024-03-29T08:07:44.257 INFO:tasks.workunit.client.0.smithi037.stdout:0/907: write d9/da/f88 [790273,30961] 0 2024-03-29T08:07:44.260 INFO:tasks.workunit.client.0.smithi037.stdout:0/908: stat f5 0 2024-03-29T08:07:44.265 INFO:tasks.workunit.client.0.smithi037.stdout:0/909: dread d9/d74/fa9 [0,4194304] 0 2024-03-29T08:07:44.268 INFO:tasks.workunit.client.0.smithi037.stdout:8/915: getdents d5/d10/d2d 0 2024-03-29T08:07:44.276 INFO:tasks.workunit.client.0.smithi037.stdout:4/971: dwrite d3/d57/dec/ded/d1d/db/d45/d1e/d59/d7d/fda [0,4194304] 0 2024-03-29T08:07:44.280 INFO:tasks.workunit.client.0.smithi037.stdout:8/916: mkdir d5/d10/d2d/d9c/dfb/d12f 0 2024-03-29T08:07:44.282 INFO:tasks.workunit.client.0.smithi037.stdout:0/910: symlink d9/da/dd/d20/d55/d5f/dab/dbb/l13c 0 2024-03-29T08:07:44.283 INFO:tasks.workunit.client.0.smithi037.stdout:8/917: stat d5/d78/fab 0 2024-03-29T08:07:44.295 INFO:tasks.workunit.client.0.smithi037.stdout:4/972: rename d3/d80/dcd/d8f to d3/d57/dec/d144 0 2024-03-29T08:07:44.300 INFO:tasks.workunit.client.0.smithi037.stdout:0/911: dread d9/da/dd/d54/f56 [0,4194304] 0 2024-03-29T08:07:44.308 INFO:tasks.workunit.client.0.smithi037.stdout:0/912: creat d9/da/dd/d54/f13d x:0 0 0 2024-03-29T08:07:44.311 INFO:tasks.workunit.client.0.smithi037.stdout:4/973: symlink d3/d57/dec/ded/d1d/db/d45/d1e/d59/l145 0 2024-03-29T08:07:44.315 INFO:tasks.workunit.client.0.smithi037.stdout:4/974: write d3/d4/f46 [8993279,130764] 0 2024-03-29T08:07:44.317 INFO:tasks.workunit.client.0.smithi037.stdout:8/918: link d5/d10/d2d/d34/d50/d83/d46/f98 d5/d10/f130 0 2024-03-29T08:07:44.327 INFO:tasks.workunit.client.0.smithi037.stdout:8/919: link d5/ffe d5/d10/d2d/df5/f131 0 2024-03-29T08:07:44.331 INFO:tasks.workunit.client.0.smithi037.stdout:0/913: dwrite d9/da/dd/d20/f4e [4194304,4194304] 0 2024-03-29T08:07:44.332 INFO:tasks.workunit.client.0.smithi037.stdout:8/920: chown d5/d10/d2d/d34/d50/d83/d46/ce5 0 1 2024-03-29T08:07:44.340 INFO:tasks.workunit.client.0.smithi037.stdout:8/921: symlink d5/d10/d2d/d34/l132 0 2024-03-29T08:07:44.342 INFO:tasks.workunit.client.0.smithi037.stdout:4/975: write d3/d57/dec/ded/d1d/d4f/f87 [1403869,16695] 0 2024-03-29T08:07:44.346 INFO:tasks.workunit.client.0.smithi037.stdout:8/922: chown d5/d10/d2d/d34/d50/ddc/ceb 2556 1 2024-03-29T08:07:44.348 INFO:tasks.workunit.client.0.smithi037.stdout:0/914: write d9/da/dd/d20/d38/f69 [771075,116296] 0 2024-03-29T08:07:44.354 INFO:tasks.workunit.client.0.smithi037.stdout:0/915: dread - d9/da/dd/d20/d38/d53/f6a zero size 2024-03-29T08:07:44.359 INFO:tasks.workunit.client.0.smithi037.stdout:0/916: mknod d9/d74/d7d/d7e/c13e 0 2024-03-29T08:07:44.363 INFO:tasks.workunit.client.0.smithi037.stdout:4/976: dwrite d3/d57/dec/ded/d1d/db/f16 [0,4194304] 0 2024-03-29T08:07:44.363 INFO:tasks.workunit.client.0.smithi037.stdout:0/917: readlink d9/da/dd/d54/db5/df6/lbe 0 2024-03-29T08:07:44.365 INFO:tasks.workunit.client.0.smithi037.stdout:4/977: fdatasync d3/d4/f11d 0 2024-03-29T08:07:44.366 INFO:tasks.workunit.client.0.smithi037.stdout:8/923: dread d5/d10/d2d/d9c/df8/dc8/fe3 [0,4194304] 0 2024-03-29T08:07:44.369 INFO:tasks.workunit.client.0.smithi037.stdout:0/918: symlink d9/da/dd/d20/d72/d134/d22/l13f 0 2024-03-29T08:07:44.369 INFO:tasks.workunit.client.0.smithi037.stdout:8/924: readlink d5/d10/d2d/d34/d50/d83/lc3 0 2024-03-29T08:07:44.374 INFO:tasks.workunit.client.0.smithi037.stdout:4/978: rename d3/d57/dec/ded/d1d/db/d72/lac to d3/d4/d27/d126/l146 0 2024-03-29T08:07:44.376 INFO:tasks.workunit.client.0.smithi037.stdout:0/919: rmdir d9/da/dd/d20/d38/d8a 39 2024-03-29T08:07:44.386 INFO:tasks.workunit.client.0.smithi037.stdout:4/979: rename d3/d57/dec/ded/d1d/db/d45/d1e/d59/db5/c11f to d3/d57/dec/ded/d1d/db/d72/d139/c147 0 2024-03-29T08:07:44.390 INFO:tasks.workunit.client.0.smithi037.stdout:4/980: chown d3/d57/dec/d144/l119 91 1 2024-03-29T08:07:44.393 INFO:tasks.workunit.client.0.smithi037.stdout:8/925: dread d5/d10/d2d/d34/d50/d83/f28 [0,4194304] 0 2024-03-29T08:07:44.397 INFO:tasks.workunit.client.0.smithi037.stdout:0/920: dwrite d9/da/dd/d20/f9e [0,4194304] 0 2024-03-29T08:07:44.400 INFO:tasks.workunit.client.0.smithi037.stdout:0/921: dread d9/d74/fa9 [0,4194304] 0 2024-03-29T08:07:44.400 INFO:tasks.workunit.client.0.smithi037.stdout:4/981: write d3/d57/dec/ded/d1d/db/d45/d93/fb8 [136521,69242] 0 2024-03-29T08:07:44.406 INFO:tasks.workunit.client.0.smithi037.stdout:8/926: truncate d5/d10/d2d/d9c/df8/d4c/d67/d85/f12b 71496 0 2024-03-29T08:07:44.411 INFO:tasks.workunit.client.0.smithi037.stdout:4/982: write d3/d57/f9e [539580,69938] 0 2024-03-29T08:07:44.418 INFO:tasks.workunit.client.0.smithi037.stdout:4/983: symlink d3/d57/dec/ded/d1d/db/d72/l148 0 2024-03-29T08:07:44.422 INFO:tasks.workunit.client.0.smithi037.stdout:0/922: dwrite d9/da/f88 [0,4194304] 0 2024-03-29T08:07:44.430 INFO:tasks.workunit.client.0.smithi037.stdout:0/923: dread - d9/da/dd/d20/d55/d5f/d9f/fec zero size 2024-03-29T08:07:44.431 INFO:tasks.workunit.client.0.smithi037.stdout:8/927: dwrite d5/d10/d2d/da7/f11b [0,4194304] 0 2024-03-29T08:07:44.437 INFO:tasks.workunit.client.0.smithi037.stdout:8/928: rmdir d5/d10/d2d/d9c/df8/d102 39 2024-03-29T08:07:44.442 INFO:tasks.workunit.client.0.smithi037.stdout:0/924: creat d9/da/dd/d20/d72/d134/d22/dee/df4/f140 x:0 0 0 2024-03-29T08:07:44.442 INFO:tasks.workunit.client.0.smithi037.stdout:8/929: dread - d5/ff0 zero size 2024-03-29T08:07:44.443 INFO:tasks.workunit.client.0.smithi037.stdout:4/984: dwrite d3/d57/dec/ded/d1d/db/d45/d1e/d59/d60/d6f/fa4 [0,4194304] 0 2024-03-29T08:07:44.447 INFO:tasks.workunit.client.0.smithi037.stdout:8/930: chown d5/d10/d45/d4b/f100 53 1 2024-03-29T08:07:44.454 INFO:tasks.workunit.client.0.smithi037.stdout:4/985: creat d3/d57/dec/ded/d1d/db/d45/d93/f149 x:0 0 0 2024-03-29T08:07:44.457 INFO:tasks.workunit.client.0.smithi037.stdout:0/925: getdents d9/da/dd/d20/d72/d134/d22 0 2024-03-29T08:07:44.463 INFO:tasks.workunit.client.0.smithi037.stdout:4/986: unlink d3/d57/dec/ded/d1d/dc2/f124 0 2024-03-29T08:07:44.467 INFO:tasks.workunit.client.0.smithi037.stdout:4/987: mkdir d3/d57/dec/d14a 0 2024-03-29T08:07:44.470 INFO:tasks.workunit.client.0.smithi037.stdout:8/931: dwrite d5/d10/d2d/d34/d50/d83/d46/f111 [0,4194304] 0 2024-03-29T08:07:44.475 INFO:tasks.workunit.client.0.smithi037.stdout:8/932: stat d5/d10/d2d/d34/d50 0 2024-03-29T08:07:44.481 INFO:tasks.workunit.client.0.smithi037.stdout:8/933: chown d5/d78/fec 74026 1 2024-03-29T08:07:44.483 INFO:tasks.workunit.client.0.smithi037.stdout:0/926: dwrite d9/da/dd/d54/ffe [0,4194304] 0 2024-03-29T08:07:44.508 INFO:tasks.workunit.client.0.smithi037.stdout:8/934: mknod d5/d10/d2d/d9c/df8/d4c/c133 0 2024-03-29T08:07:44.508 INFO:tasks.workunit.client.0.smithi037.stdout:0/927: creat d9/d74/f141 x:0 0 0 2024-03-29T08:07:44.511 INFO:tasks.workunit.client.0.smithi037.stdout:4/988: dread d3/d57/d85/f10d [0,4194304] 0 2024-03-29T08:07:44.513 INFO:tasks.workunit.client.0.smithi037.stdout:0/928: mkdir d9/da/dd/d20/d72/d134/d22/d142 0 2024-03-29T08:07:44.514 INFO:tasks.workunit.client.0.smithi037.stdout:8/935: fdatasync d5/d10/d2d/d34/d50/f5e 0 2024-03-29T08:07:44.519 INFO:tasks.workunit.client.0.smithi037.stdout:8/936: creat d5/d10/d2d/d9c/dfb/f134 x:0 0 0 2024-03-29T08:07:44.523 INFO:tasks.workunit.client.0.smithi037.stdout:8/937: mknod d5/d10/d2d/d9c/df8/d4c/d67/d70/d11e/c135 0 2024-03-29T08:07:44.531 INFO:tasks.workunit.client.0.smithi037.stdout:8/938: rename d5/d10/d45/f7f to d5/d10/d2d/d34/d50/f136 0 2024-03-29T08:07:44.535 INFO:tasks.workunit.client.0.smithi037.stdout:8/939: write d5/d10/d19/f2f [2640583,85945] 0 2024-03-29T08:07:44.538 INFO:tasks.workunit.client.0.smithi037.stdout:0/929: mkdir d9/da/dd/d20/d72/d134/d22/dee/df4/d143 0 2024-03-29T08:07:44.539 INFO:tasks.workunit.client.0.smithi037.stdout:3/904: sync 2024-03-29T08:07:44.542 INFO:tasks.workunit.client.0.smithi037.stdout:4/989: dread d3/d57/f8e [0,4194304] 0 2024-03-29T08:07:44.544 INFO:tasks.workunit.client.0.smithi037.stdout:0/930: chown d9/da/dd/d54/d92/l12b 1172 1 2024-03-29T08:07:44.547 INFO:tasks.workunit.client.0.smithi037.stdout:0/931: chown d9/da/dd/d54/db5/dcb/fe5 13194137 1 2024-03-29T08:07:44.549 INFO:tasks.workunit.client.0.smithi037.stdout:8/940: write d5/f1e [1951914,7479] 0 2024-03-29T08:07:44.551 INFO:tasks.workunit.client.0.smithi037.stdout:3/905: getdents dd/d15/d18/d30/d115/d2d/d3a/d6c/db1/dc0/dda/d33/d83 0 2024-03-29T08:07:44.553 INFO:tasks.workunit.client.0.smithi037.stdout:4/990: getdents d3/d80/dcd 0 2024-03-29T08:07:44.553 INFO:tasks.workunit.client.0.smithi037.stdout:0/932: creat d9/da/dd/d20/d72/df7/f144 x:0 0 0 2024-03-29T08:07:44.556 INFO:tasks.workunit.client.0.smithi037.stdout:8/941: mknod d5/d10/d2d/d9c/dfb/c137 0 2024-03-29T08:07:44.557 INFO:tasks.workunit.client.0.smithi037.stdout:0/933: mkdir d9/da/dd/d54/db5/dcb/d145 0 2024-03-29T08:07:44.560 INFO:tasks.workunit.client.0.smithi037.stdout:0/934: write d9/da/dd/d20/ff9 [1001751,124811] 0 2024-03-29T08:07:44.561 INFO:tasks.workunit.client.0.smithi037.stdout:3/906: rename dd/d15/d18/d30/d115/d2d/d3a/l101 to dd/d15/d18/d30/d115/d2d/d3a/d6c/d10e/l130 0 2024-03-29T08:07:44.564 INFO:tasks.workunit.client.0.smithi037.stdout:0/935: readlink d9/da/dd/d13/l67 0 2024-03-29T08:07:44.568 INFO:tasks.workunit.client.0.smithi037.stdout:0/936: write d9/da/dd/d20/d72/fcd [384594,75869] 0 2024-03-29T08:07:44.572 INFO:tasks.workunit.client.0.smithi037.stdout:0/937: chown d9/da/dd/d20/d72/d134/l30 5916 1 2024-03-29T08:07:44.576 INFO:tasks.workunit.client.0.smithi037.stdout:0/938: mkdir d9/da/dd/d13/d83/d146 0 2024-03-29T08:07:44.577 INFO:tasks.workunit.client.0.smithi037.stdout:3/907: dread dd/d15/d18/d30/d115/d2d/d3a/f81 [0,4194304] 0 2024-03-29T08:07:44.577 INFO:tasks.workunit.client.0.smithi037.stdout:8/942: dwrite d5/d10/d2d/d34/d50/f68 [0,4194304] 0 2024-03-29T08:07:44.581 INFO:tasks.workunit.client.0.smithi037.stdout:3/908: symlink dd/d5e/d7a/dd4/l131 0 2024-03-29T08:07:44.581 INFO:tasks.workunit.client.0.smithi037.stdout:8/943: rmdir d5/d10/d2d/d9c 39 2024-03-29T08:07:44.583 INFO:tasks.workunit.client.0.smithi037.stdout:3/909: write f5 [2788711,49521] 0 2024-03-29T08:07:44.587 INFO:tasks.workunit.client.0.smithi037.stdout:3/910: fdatasync dd/d15/d18/d30/d115/f37 0 2024-03-29T08:07:44.587 INFO:tasks.workunit.client.0.smithi037.stdout:1/992: sync 2024-03-29T08:07:44.587 INFO:tasks.workunit.client.0.smithi037.stdout:2/848: sync 2024-03-29T08:07:44.591 INFO:tasks.workunit.client.0.smithi037.stdout:8/944: creat d5/d10/d2d/d34/d50/d83/d46/d4e/f138 x:0 0 0 2024-03-29T08:07:44.593 INFO:tasks.workunit.client.0.smithi037.stdout:2/849: chown d0/f11b 235696914 1 2024-03-29T08:07:44.596 INFO:tasks.workunit.client.0.smithi037.stdout:1/993: mkdir d2/d6/dc/d58/d67/d11b/d11e/d7c/d159 0 2024-03-29T08:07:44.597 INFO:tasks.workunit.client.0.smithi037.stdout:3/911: mkdir dd/d5e/d7a/d132 0 2024-03-29T08:07:44.602 INFO:tasks.workunit.client.0.smithi037.stdout:0/939: dwrite d9/d74/f97 [0,4194304] 0 2024-03-29T08:07:44.605 INFO:tasks.workunit.client.0.smithi037.stdout:2/850: dread d0/d23/f65 [0,4194304] 0 2024-03-29T08:07:44.610 INFO:tasks.workunit.client.0.smithi037.stdout:2/851: dread - d0/d23/daf/dbf/d10f/d55/d92/d10a/f10d zero size 2024-03-29T08:07:44.611 INFO:tasks.workunit.client.0.smithi037.stdout:3/912: mknod dd/d15/d18/d30/d115/d2d/d3a/d6c/d10e/c133 0 2024-03-29T08:07:44.616 INFO:tasks.workunit.client.0.smithi037.stdout:8/945: write d5/d10/d45/fc2 [5140768,24133] 0 2024-03-29T08:07:44.617 INFO:tasks.workunit.client.0.smithi037.stdout:1/994: write d2/d40/de8/daf/fdf [189393,34222] 0 2024-03-29T08:07:44.618 INFO:tasks.workunit.client.0.smithi037.stdout:3/913: stat dd/d15/d18/d30/d115/d3e/d47/cf6 0 2024-03-29T08:07:44.619 INFO:tasks.workunit.client.0.smithi037.stdout:2/852: mkdir d0/d23/d113/d127 0 2024-03-29T08:07:44.620 INFO:tasks.workunit.client.0.smithi037.stdout:0/940: rmdir d9/d74/d7d/d7e/d115/d127 0 2024-03-29T08:07:44.622 INFO:tasks.workunit.client.0.smithi037.stdout:1/995: read - d2/d6/dc/d7e/dd2/fde zero size 2024-03-29T08:07:44.626 INFO:tasks.workunit.client.0.smithi037.stdout:2/853: write d0/d23/d44/dca/d2e/fe7 [150079,119834] 0 2024-03-29T08:07:44.626 INFO:tasks.workunit.client.0.smithi037.stdout:3/914: chown dd/d15/d18/d30/d115/d2d/d3a/d6c/db1/dc0/dda/d33/d106/l107 1 1 2024-03-29T08:07:44.628 INFO:tasks.workunit.client.0.smithi037.stdout:1/996: readlink d2/d40/de8/d87/l137 0 2024-03-29T08:07:44.632 INFO:tasks.workunit.client.0.smithi037.stdout:8/946: rename d5/d10/d2d/d34/d50/d83/d46/d4e/fb6 to d5/d10/d2d/da7/d11a/f139 0 2024-03-29T08:07:44.636 INFO:tasks.workunit.client.0.smithi037.stdout:3/915: symlink dd/d15/d18/d30/d115/d2d/d3a/l134 0 2024-03-29T08:07:44.638 INFO:tasks.workunit.client.0.smithi037.stdout:8/947: rmdir d5/d10/d2d/d34/d50/d83/d46/d8a/d96 39 2024-03-29T08:07:44.642 INFO:tasks.workunit.client.0.smithi037.stdout:3/916: creat dd/d15/d18/d4b/da2/f135 x:0 0 0 2024-03-29T08:07:44.644 INFO:tasks.workunit.client.0.smithi037.stdout:2/854: write d0/d23/daf/dbf/d10f/f39 [4203783,111762] 0 2024-03-29T08:07:44.651 INFO:tasks.workunit.client.0.smithi037.stdout:1/997: dwrite d2/d6/f41 [0,4194304] 0 2024-03-29T08:07:44.654 INFO:tasks.workunit.client.0.smithi037.stdout:2/855: mknod d0/d23/daf/dbf/c128 0 2024-03-29T08:07:44.658 INFO:tasks.workunit.client.0.smithi037.stdout:3/917: dread - dd/d15/d18/d30/d115/d2d/d3a/d6c/db1/dc0/dda/ddb/f108 zero size 2024-03-29T08:07:44.661 INFO:tasks.workunit.client.0.smithi037.stdout:0/941: dread d9/da/fc6 [0,4194304] 0 2024-03-29T08:07:44.663 INFO:tasks.workunit.client.0.smithi037.stdout:1/998: symlink d2/d6/dc/d7e/dd2/d113/l15a 0 2024-03-29T08:07:44.667 INFO:tasks.workunit.client.0.smithi037.stdout:8/948: dwrite d5/f80 [0,4194304] 0 2024-03-29T08:07:44.671 INFO:tasks.workunit.client.0.smithi037.stdout:3/918: read dd/d15/d18/d30/d115/d2d/d3a/d6c/db1/fbe [7719358,69345] 0 2024-03-29T08:07:44.675 INFO:tasks.workunit.client.0.smithi037.stdout:0/942: chown d9/da/dd/d20/d72/d134/d22/l60 80101 1 2024-03-29T08:07:44.675 INFO:tasks.workunit.client.0.smithi037.stdout:1/999: creat d2/d6/dc/d58/d67/d11b/d11e/f15b x:0 0 0 2024-03-29T08:07:44.679 INFO:tasks.workunit.client.0.smithi037.stdout:0/943: creat d9/da/dd/d13/d83/f147 x:0 0 0 2024-03-29T08:07:44.687 INFO:tasks.workunit.client.0.smithi037.stdout:0/944: mknod d9/da/dd/d20/d38/d8a/c148 0 2024-03-29T08:07:44.689 INFO:tasks.workunit.client.0.smithi037.stdout:2/856: dwrite d0/d23/daf/dbf/d10f/d55/f5b [0,4194304] 0 2024-03-29T08:07:44.696 INFO:tasks.workunit.client.0.smithi037.stdout:0/945: dread - d9/da/dd/d20/d72/d134/d22/f103 zero size 2024-03-29T08:07:44.703 INFO:tasks.workunit.client.0.smithi037.stdout:3/919: dwrite dd/d15/fe6 [0,4194304] 0 2024-03-29T08:07:44.708 INFO:tasks.workunit.client.0.smithi037.stdout:8/949: dwrite d5/d10/d2d/d9c/df8/dc8/de6/f115 [0,4194304] 0 2024-03-29T08:07:44.713 INFO:tasks.workunit.client.0.smithi037.stdout:0/946: rename d9/da/dd/d20/d72/d134/f41 to d9/da/dd/d59/f149 0 2024-03-29T08:07:44.717 INFO:tasks.workunit.client.0.smithi037.stdout:8/950: getdents d5/d10/d2d/d9c/dfb/d12f 0 2024-03-29T08:07:44.718 INFO:tasks.workunit.client.0.smithi037.stdout:2/857: dread d0/d23/d44/dca/d2e/d5f/f8d [0,4194304] 0 2024-03-29T08:07:44.724 INFO:tasks.workunit.client.0.smithi037.stdout:0/947: rename d9/da/dd/d54/f56 to d9/da/dd/d20/d55/f14a 0 2024-03-29T08:07:44.727 INFO:tasks.workunit.client.0.smithi037.stdout:2/858: mkdir d0/d23/d44/dca/d2e/d9c/df3/d78/d129 0 2024-03-29T08:07:44.727 INFO:tasks.workunit.client.0.smithi037.stdout:3/920: write dd/d5e/f80 [545869,91077] 0 2024-03-29T08:07:44.728 INFO:tasks.workunit.client.0.smithi037.stdout:0/948: dread - d9/da/dd/d20/d72/d134/d22/f103 zero size 2024-03-29T08:07:44.737 INFO:tasks.workunit.client.0.smithi037.stdout:3/921: rmdir dd/d15/d18/d30/d115/d2d 39 2024-03-29T08:07:44.737 INFO:tasks.workunit.client.0.smithi037.stdout:0/949: symlink d9/da/dd/d20/d38/d53/l14b 0 2024-03-29T08:07:44.739 INFO:tasks.workunit.client.0.smithi037.stdout:2/859: write d0/d23/d44/dca/d2e/da4/fe3 [3932567,98130] 0 2024-03-29T08:07:44.742 INFO:tasks.workunit.client.0.smithi037.stdout:2/860: fdatasync d0/d23/d70/f99 0 2024-03-29T08:07:44.743 INFO:tasks.workunit.client.0.smithi037.stdout:0/950: rmdir d9/da/dd/d20/d38 39 2024-03-29T08:07:44.745 INFO:tasks.workunit.client.0.smithi037.stdout:8/951: dwrite d5/d10/d2d/d9c/df8/d102/f10f [0,4194304] 0 2024-03-29T08:07:44.751 INFO:tasks.workunit.client.0.smithi037.stdout:8/952: truncate d5/d78/fec 1138371 0 2024-03-29T08:07:44.756 INFO:tasks.workunit.client.0.smithi037.stdout:8/953: mknod d5/d10/d2d/d34/d50/ddc/c13a 0 2024-03-29T08:07:44.756 INFO:tasks.workunit.client.0.smithi037.stdout:3/922: write dd/d15/d18/d30/d115/d2d/d3a/d6c/db1/dc0/dda/ddb/f6a [1976944,118083] 0 2024-03-29T08:07:44.763 INFO:tasks.workunit.client.0.smithi037.stdout:3/923: readlink dd/d15/d18/d4b/da2/lb3 0 2024-03-29T08:07:44.766 INFO:tasks.workunit.client.0.smithi037.stdout:8/954: write d5/d10/d2d/d34/d50/d83/d46/d4e/f138 [896127,13188] 0 2024-03-29T08:07:44.769 INFO:tasks.workunit.client.0.smithi037.stdout:0/951: dwrite d9/da/dd/d20/d72/fcd [0,4194304] 0 2024-03-29T08:07:44.771 INFO:tasks.workunit.client.0.smithi037.stdout:8/955: readlink d5/d10/d2d/d9c/df8/d102/l122 0 2024-03-29T08:07:44.771 INFO:tasks.workunit.client.0.smithi037.stdout:2/861: dwrite d0/d23/d44/dca/d2e/f91 [0,4194304] 0 2024-03-29T08:07:44.779 INFO:tasks.workunit.client.0.smithi037.stdout:3/924: creat dd/d15/d18/d30/d115/d2d/d3a/d6c/f136 x:0 0 0 2024-03-29T08:07:44.779 INFO:tasks.workunit.client.0.smithi037.stdout:2/862: write d0/d23/daf/dbf/d10f/d55/f5b [142545,3196] 0 2024-03-29T08:07:44.782 INFO:tasks.workunit.client.0.smithi037.stdout:8/956: creat d5/d10/d2d/d9c/df8/d4c/f13b x:0 0 0 2024-03-29T08:07:44.784 INFO:tasks.workunit.client.0.smithi037.stdout:0/952: mkdir d9/da/dd/d20/d55/d5f/d9f/d14c 0 2024-03-29T08:07:44.794 INFO:tasks.workunit.client.0.smithi037.stdout:0/953: mknod d9/d74/c14d 0 2024-03-29T08:07:44.799 INFO:tasks.workunit.client.0.smithi037.stdout:4/991: sync 2024-03-29T08:07:44.800 INFO:tasks.workunit.client.0.smithi037.stdout:2/863: mkdir d0/d23/d44/d12a 0 2024-03-29T08:07:44.803 INFO:tasks.workunit.client.0.smithi037.stdout:8/957: creat d5/d10/d2d/d34/d50/d120/f13c x:0 0 0 2024-03-29T08:07:44.804 INFO:tasks.workunit.client.0.smithi037.stdout:2/864: rmdir d0/d23/daf/dbf/d10f 39 2024-03-29T08:07:44.811 INFO:tasks.workunit.client.0.smithi037.stdout:4/992: link d3/d4/d27/f7f d3/d57/dec/ded/d1d/db/d72/d98/f14b 0 2024-03-29T08:07:44.811 INFO:tasks.workunit.client.0.smithi037.stdout:8/958: symlink d5/d10/d2d/d9c/df8/d102/d127/l13d 0 2024-03-29T08:07:44.819 INFO:tasks.workunit.client.0.smithi037.stdout:3/925: dwrite dd/d15/d18/d30/d115/d2d/d3a/f81 [0,4194304] 0 2024-03-29T08:07:44.823 INFO:tasks.workunit.client.0.smithi037.stdout:4/993: write d3/d57/dec/ded/d1d/db/d45/d93/d96/da0/dc9/f11b [536401,104057] 0 2024-03-29T08:07:44.825 INFO:tasks.workunit.client.0.smithi037.stdout:3/926: stat dd/d15/d18/d30/d115/d2d/d3a/d6c/db1/dc0/dda/d33/d97/dd6/df5/f10a 0 2024-03-29T08:07:44.830 INFO:tasks.workunit.client.0.smithi037.stdout:2/865: truncate d0/d23/d44/dca/d7d/fdc 3867631 0 2024-03-29T08:07:44.838 INFO:tasks.workunit.client.0.smithi037.stdout:0/954: dread d9/da/dd/d20/d38/d8a/f9a [0,4194304] 0 2024-03-29T08:07:44.843 INFO:tasks.workunit.client.0.smithi037.stdout:4/994: getdents d3/d54 0 2024-03-29T08:07:44.849 INFO:tasks.workunit.client.0.smithi037.stdout:3/927: write dd/d15/d18/d30/d115/d3e/fb7 [783815,36631] 0 2024-03-29T08:07:44.853 INFO:tasks.workunit.client.0.smithi037.stdout:2/866: creat d0/d23/daf/dbf/d10f/d55/d92/d10a/d125/f12b x:0 0 0 2024-03-29T08:07:44.853 INFO:tasks.workunit.client.0.smithi037.stdout:8/959: dwrite d5/d10/d2d/d34/d50/d83/d46/d4e/fd8 [0,4194304] 0 2024-03-29T08:07:44.860 INFO:tasks.workunit.client.0.smithi037.stdout:8/960: chown d5/d10/d2d/d34/d50/d83/f2e 1899478111 1 2024-03-29T08:07:44.860 INFO:tasks.workunit.client.0.smithi037.stdout:0/955: link d9/da/dd/d59/l91 d9/da/dd/d20/d38/d8a/dfd/l14e 0 2024-03-29T08:07:44.865 INFO:tasks.workunit.client.0.smithi037.stdout:0/956: fsync d9/da/dd/f99 0 2024-03-29T08:07:44.870 INFO:tasks.workunit.client.0.smithi037.stdout:4/995: dread d3/d57/dec/ded/d1d/f3f [0,4194304] 0 2024-03-29T08:07:44.870 INFO:tasks.workunit.client.0.smithi037.stdout:3/928: rename dd/d15/d18/d30/d115/d2d/d3a/d6c/db1/dc0/dda/d33/d84/ffa to dd/f137 0 2024-03-29T08:07:44.873 INFO:tasks.workunit.client.0.smithi037.stdout:8/961: symlink d5/d10/d2d/d34/d50/l13e 0 2024-03-29T08:07:44.878 INFO:tasks.workunit.client.0.smithi037.stdout:0/957: symlink d9/da/dd/d20/d55/d5f/d9f/l14f 0 2024-03-29T08:07:44.878 INFO:tasks.workunit.client.0.smithi037.stdout:2/867: write d0/d23/d44/dca/d2e/d5f/d6f/d5e/fc0 [47038,42580] 0 2024-03-29T08:07:44.880 INFO:tasks.workunit.client.0.smithi037.stdout:0/958: chown d9/d74/de6/f11a 305 1 2024-03-29T08:07:44.881 INFO:tasks.workunit.client.0.smithi037.stdout:4/996: creat d3/d57/dec/ded/d89/f14c x:0 0 0 2024-03-29T08:07:44.885 INFO:tasks.workunit.client.0.smithi037.stdout:3/929: mknod dd/d15/d18/d30/d115/c138 0 2024-03-29T08:07:44.890 INFO:tasks.workunit.client.0.smithi037.stdout:2/868: mknod d0/d23/daf/dbf/d10f/c12c 0 2024-03-29T08:07:44.892 INFO:tasks.workunit.client.0.smithi037.stdout:0/959: mkdir d9/da/dd/d20/d38/d53/d150 0 2024-03-29T08:07:44.898 INFO:tasks.workunit.client.0.smithi037.stdout:0/960: chown d9/da/dd/d20/d38/d8a/cc7 5 1 2024-03-29T08:07:44.899 INFO:tasks.workunit.client.0.smithi037.stdout:2/869: symlink d0/d53/d89/l12d 0 2024-03-29T08:07:44.901 INFO:tasks.workunit.client.0.smithi037.stdout:4/997: getdents d3/d54 0 2024-03-29T08:07:44.910 INFO:tasks.workunit.client.0.smithi037.stdout:2/870: creat d0/d23/d44/dca/d2e/d38/d68/f12e x:0 0 0 2024-03-29T08:07:44.911 INFO:tasks.workunit.client.0.smithi037.stdout:4/998: unlink d3/d57/dec/ded/d1d/dad/dfb/fd7 0 2024-03-29T08:07:44.911 INFO:tasks.workunit.client.0.smithi037.stdout:0/961: chown d9/da/dd/d20/d38/d53/c5a 1462 1 2024-03-29T08:07:44.912 INFO:tasks.workunit.client.0.smithi037.stdout:8/962: dwrite d5/d10/d19/d3d/f10a [0,4194304] 0 2024-03-29T08:07:44.914 INFO:tasks.workunit.client.0.smithi037.stdout:4/999: fdatasync d3/d4/f11d 0 2024-03-29T08:07:44.920 INFO:tasks.workunit.client.0.smithi037.stdout:8/963: stat d5/d10/d45/fc2 0 2024-03-29T08:07:44.920 INFO:tasks.workunit.client.0.smithi037.stdout:3/930: dwrite dd/d15/d18/d30/d115/d2d/d3a/d6c/db1/dc0/dda/d33/d106/f125 [0,4194304] 0 2024-03-29T08:07:44.921 INFO:tasks.workunit.client.0.smithi037.stdout:2/871: creat d0/d23/daf/dbf/d10f/d55/d92/d10a/d125/f12f x:0 0 0 2024-03-29T08:07:44.927 INFO:tasks.workunit.client.0.smithi037.stdout:0/962: creat d9/da/dd/d20/d72/d134/d22/dee/d110/f151 x:0 0 0 2024-03-29T08:07:44.930 INFO:tasks.workunit.client.0.smithi037.stdout:2/872: mkdir d0/d23/d44/dca/d2e/d5f/d6f/d5e/dc2/d130 0 2024-03-29T08:07:44.934 INFO:tasks.workunit.client.0.smithi037.stdout:0/963: creat d9/da/dd/d13/d83/f152 x:0 0 0 2024-03-29T08:07:44.938 INFO:tasks.workunit.client.0.smithi037.stdout:2/873: chown d0/d23/d44/dca/d2e/d5f/d6f/d5e/dc2/feb 122874 1 2024-03-29T08:07:44.946 INFO:tasks.workunit.client.0.smithi037.stdout:2/874: read d0/d23/d44/dca/d2e/d5f/d6f/d5e/dc2/feb [3228619,36597] 0 2024-03-29T08:07:44.949 INFO:tasks.workunit.client.0.smithi037.stdout:3/931: dwrite dd/d15/d18/d30/d115/d3e/fb7 [0,4194304] 0 2024-03-29T08:07:44.953 INFO:tasks.workunit.client.0.smithi037.stdout:8/964: dread d5/f1e [0,4194304] 0 2024-03-29T08:07:44.957 INFO:tasks.workunit.client.0.smithi037.stdout:8/965: dread - d5/d10/d2d/d9c/df8/d4c/f13b zero size 2024-03-29T08:07:44.960 INFO:tasks.workunit.client.0.smithi037.stdout:2/875: mkdir d0/d23/daf/dbf/d10f/d55/d92/d11f/d131 0 2024-03-29T08:07:44.963 INFO:tasks.workunit.client.0.smithi037.stdout:2/876: read - d0/d23/daf/dbf/d10f/d55/d92/d10a/f10d zero size 2024-03-29T08:07:44.965 INFO:tasks.workunit.client.0.smithi037.stdout:3/932: rename dd/d15/d18/d30/d115/d2d/d3a/d6c/db1/dc0/dda/ddb/l66 to dd/d15/d18/d30/d115/d2d/d3a/d6c/db1/dc0/l139 0 2024-03-29T08:07:44.965 INFO:tasks.workunit.client.0.smithi037.stdout:8/966: mkdir d5/d10/d2d/d9c/dfb/d12f/d13f 0 2024-03-29T08:07:44.968 INFO:tasks.workunit.client.0.smithi037.stdout:8/967: readlink d5/d10/d2d/d9c/df8/lb8 0 2024-03-29T08:07:44.985 INFO:tasks.workunit.client.0.smithi037.stdout:2/877: write d0/d23/daf/dbf/d10f/d55/d92/fcf [2962292,87076] 0 2024-03-29T08:07:44.987 INFO:tasks.workunit.client.0.smithi037.stdout:0/964: dread d9/f10d [0,4194304] 0 2024-03-29T08:07:44.988 INFO:tasks.workunit.client.0.smithi037.stdout:8/968: dwrite d5/d10/d2d/d9c/df8/d102/f10f [0,4194304] 0 2024-03-29T08:07:44.992 INFO:tasks.workunit.client.0.smithi037.stdout:0/965: chown d9/da/dd/d54/db5/df6/lbe 1872310 1 2024-03-29T08:07:44.996 INFO:tasks.workunit.client.0.smithi037.stdout:2/878: write d0/d23/d44/dca/d7d/fdc [282927,33203] 0 2024-03-29T08:07:44.999 INFO:tasks.workunit.client.0.smithi037.stdout:0/966: truncate d9/da/dd/d20/d38/d8a/dfd/f132 505855 0 2024-03-29T08:07:45.000 INFO:tasks.workunit.client.0.smithi037.stdout:2/879: readlink d0/d23/d44/dca/d2e/l5d 0 2024-03-29T08:07:45.008 INFO:tasks.workunit.client.0.smithi037.stdout:2/880: write d0/d23/d44/dca/d2e/da4/fb4 [692542,88764] 0 2024-03-29T08:07:45.012 INFO:tasks.workunit.client.0.smithi037.stdout:8/969: dwrite d5/d10/d2d/d9c/df8/d4c/d67/d85/f12b [0,4194304] 0 2024-03-29T08:07:45.013 INFO:tasks.workunit.client.0.smithi037.stdout:2/881: chown d0/d23/d44/dca/d2e/d38/ffe 450634998 1 2024-03-29T08:07:45.017 INFO:tasks.workunit.client.0.smithi037.stdout:8/970: chown d5/d10/d2d/d34/d50/d83/f2e 97095 1 2024-03-29T08:07:45.029 INFO:tasks.workunit.client.0.smithi037.stdout:0/967: dread d9/da/dd/d13/f21 [0,4194304] 0 2024-03-29T08:07:45.035 INFO:tasks.workunit.client.0.smithi037.stdout:2/882: dwrite d0/d23/d44/dca/f3a [4194304,4194304] 0 2024-03-29T08:07:45.039 INFO:tasks.workunit.client.0.smithi037.stdout:0/968: link d9/da/dd/d59/fb4 d9/d74/d7d/d7e/d115/f153 0 2024-03-29T08:07:45.039 INFO:tasks.workunit.client.0.smithi037.stdout:2/883: mknod d0/d23/c132 0 2024-03-29T08:07:45.043 INFO:tasks.workunit.client.0.smithi037.stdout:2/884: symlink d0/d23/daf/l133 0 2024-03-29T08:07:45.051 INFO:tasks.workunit.client.0.smithi037.stdout:2/885: fdatasync d0/f7 0 2024-03-29T08:07:45.060 INFO:tasks.workunit.client.0.smithi037.stdout:0/969: dwrite d9/da/dd/d13/d7b/fa6 [0,4194304] 0 2024-03-29T08:07:45.072 INFO:tasks.workunit.client.0.smithi037.stdout:2/886: dwrite d0/d23/d44/dca/d2e/d38/ffe [0,4194304] 0 2024-03-29T08:07:45.080 INFO:tasks.workunit.client.0.smithi037.stdout:0/970: truncate d9/da/dd/d20/d38/d53/f8f 350169 0 2024-03-29T08:07:45.081 INFO:tasks.workunit.client.0.smithi037.stdout:2/887: mknod d0/d23/d44/dca/d2e/d9c/d107/c134 0 2024-03-29T08:07:45.085 INFO:tasks.workunit.client.0.smithi037.stdout:2/888: mknod d0/d23/d44/d12a/c135 0 2024-03-29T08:07:45.089 INFO:tasks.workunit.client.0.smithi037.stdout:2/889: write d0/d23/d44/dca/fce [5830234,90619] 0 2024-03-29T08:07:45.099 INFO:tasks.workunit.client.0.smithi037.stdout:0/971: dwrite d9/da/dd/f5b [0,4194304] 0 2024-03-29T08:07:45.102 INFO:tasks.workunit.client.0.smithi037.stdout:0/972: dread - d9/da/dd/d20/d72/d134/d22/f103 zero size 2024-03-29T08:07:45.103 INFO:tasks.workunit.client.0.smithi037.stdout:2/890: dread d0/d23/d44/dca/d2e/d9c/df3/f72 [0,4194304] 0 2024-03-29T08:07:45.107 INFO:tasks.workunit.client.0.smithi037.stdout:0/973: getdents d9 0 2024-03-29T08:07:45.108 INFO:tasks.workunit.client.0.smithi037.stdout:0/974: read - d9/d74/f141 zero size 2024-03-29T08:07:45.111 INFO:tasks.workunit.client.0.smithi037.stdout:2/891: getdents d0/d23/daf/dbf 0 2024-03-29T08:07:45.116 INFO:tasks.workunit.client.0.smithi037.stdout:2/892: mknod d0/d23/d44/dca/d2e/d5f/d6f/d5e/dc2/d130/c136 0 2024-03-29T08:07:45.118 INFO:tasks.workunit.client.0.smithi037.stdout:0/975: write d9/d74/d7d/d7e/fef [468850,123480] 0 2024-03-29T08:07:45.122 INFO:tasks.workunit.client.0.smithi037.stdout:0/976: dread - d9/da/dd/d20/d55/d5f/d9f/fec zero size 2024-03-29T08:07:45.126 INFO:tasks.workunit.client.0.smithi037.stdout:0/977: dread d9/da/dd/d13/f96 [0,4194304] 0 2024-03-29T08:07:45.130 INFO:tasks.workunit.client.0.smithi037.stdout:0/978: rmdir d9/da/dd/d54/db5/dcb 39 2024-03-29T08:07:45.133 INFO:tasks.workunit.client.0.smithi037.stdout:0/979: mknod d9/d74/d7d/d7e/c154 0 2024-03-29T08:07:45.139 INFO:tasks.workunit.client.0.smithi037.stdout:0/980: write d9/da/dd/d13/d83/f10e [835575,40355] 0 2024-03-29T08:07:45.142 INFO:tasks.workunit.client.0.smithi037.stdout:2/893: dread d0/fa [0,4194304] 0 2024-03-29T08:07:45.142 INFO:tasks.workunit.client.0.smithi037.stdout:0/981: unlink d9/f10d 0 2024-03-29T08:07:45.147 INFO:tasks.workunit.client.0.smithi037.stdout:2/894: creat d0/d23/d44/dca/d2e/d5f/d6f/d5e/dc2/d130/f137 x:0 0 0 2024-03-29T08:07:45.150 INFO:tasks.workunit.client.0.smithi037.stdout:0/982: chown d9/d74/l136 867908 1 2024-03-29T08:07:45.151 INFO:tasks.workunit.client.0.smithi037.stdout:2/895: mkdir d0/d23/daf/dbf/d10f/d4e/d138 0 2024-03-29T08:07:45.153 INFO:tasks.workunit.client.0.smithi037.stdout:2/896: chown d0/d23/d44/dca/c4d 1773469 1 2024-03-29T08:07:45.168 INFO:tasks.workunit.client.0.smithi037.stdout:2/897: write d0/d23/daf/dbf/fde [191251,1958] 0 2024-03-29T08:07:45.170 INFO:tasks.workunit.client.0.smithi037.stdout:2/898: stat d0/d23/d44/df6 0 2024-03-29T08:07:45.171 INFO:tasks.workunit.client.0.smithi037.stdout:0/983: link d9/da/dd/d13/d7b/cf5 d9/da/dd/d20/d55/d5f/c155 0 2024-03-29T08:07:45.177 INFO:tasks.workunit.client.0.smithi037.stdout:2/899: creat d0/d23/d44/dca/d2e/da4/f139 x:0 0 0 2024-03-29T08:07:45.179 INFO:tasks.workunit.client.0.smithi037.stdout:0/984: mkdir d9/da/dd/d20/d55/d5f/d9f/d108/d12c/d156 0 2024-03-29T08:07:45.184 INFO:tasks.workunit.client.0.smithi037.stdout:0/985: readlink d9/le1 0 2024-03-29T08:07:45.188 INFO:tasks.workunit.client.0.smithi037.stdout:0/986: write d9/da/dd/d54/ffe [4664598,19600] 0 2024-03-29T08:07:45.191 INFO:tasks.workunit.client.0.smithi037.stdout:2/900: unlink d0/d23/d44/dca/d2e/d38/f6b 0 2024-03-29T08:07:45.193 INFO:tasks.workunit.client.0.smithi037.stdout:2/901: read d0/d23/f65 [98448,34648] 0 2024-03-29T08:07:45.204 INFO:tasks.workunit.client.0.smithi037.stdout:2/902: rename d0/d23/d44/dca/d7d/dfb/l115 to d0/d23/d44/dca/d2e/d38/d68/l13a 0 2024-03-29T08:07:45.215 INFO:tasks.workunit.client.0.smithi037.stdout:2/903: mkdir d0/d23/d44/dca/d2e/d9c/df3/d13b 0 2024-03-29T08:07:45.216 INFO:tasks.workunit.client.0.smithi037.stdout:0/987: write d9/da/f25 [5002993,88083] 0 2024-03-29T08:07:45.222 INFO:tasks.workunit.client.0.smithi037.stdout:0/988: mknod d9/da/dd/d20/d72/df7/c157 0 2024-03-29T08:07:45.223 INFO:tasks.workunit.client.0.smithi037.stdout:2/904: mkdir d0/d23/d44/dca/d2e/d9c/df3/d78/d129/d13c 0 2024-03-29T08:07:45.225 INFO:tasks.workunit.client.0.smithi037.stdout:0/989: dread - d9/da/dd/d20/d38/d53/f6a zero size 2024-03-29T08:07:45.226 INFO:tasks.workunit.client.0.smithi037.stdout:2/905: write d0/d23/d44/dca/d2e/da4/fe3 [4585766,121013] 0 2024-03-29T08:07:45.231 INFO:tasks.workunit.client.0.smithi037.stdout:0/990: creat d9/da/dd/d54/db5/df6/f158 x:0 0 0 2024-03-29T08:07:45.233 INFO:tasks.workunit.client.0.smithi037.stdout:2/906: creat d0/d23/daf/dbf/d10f/d55/d92/d10a/f13d x:0 0 0 2024-03-29T08:07:45.240 INFO:tasks.workunit.client.0.smithi037.stdout:0/991: rename d9/da/dd/d54/d92/dd4/fe8 to d9/da/dd/d20/d72/d134/d22/d142/f159 0 2024-03-29T08:07:45.245 INFO:tasks.workunit.client.0.smithi037.stdout:0/992: unlink d9/da/dd/d20/d38/d53/f105 0 2024-03-29T08:07:45.249 INFO:tasks.workunit.client.0.smithi037.stdout:0/993: mkdir d9/da/dd/d20/d55/d15a 0 2024-03-29T08:07:45.256 INFO:tasks.workunit.client.0.smithi037.stdout:2/907: dwrite d0/d23/d44/dca/d2e/da4/f103 [0,4194304] 0 2024-03-29T08:07:45.258 INFO:tasks.workunit.client.0.smithi037.stdout:2/908: readlink d0/d23/d44/l96 0 2024-03-29T08:07:45.263 INFO:tasks.workunit.client.0.smithi037.stdout:0/994: dread d9/da/dd/d20/d55/f14a [0,4194304] 0 2024-03-29T08:07:45.269 INFO:tasks.workunit.client.0.smithi037.stdout:0/995: mknod d9/da/c15b 0 2024-03-29T08:07:45.270 INFO:tasks.workunit.client.0.smithi037.stdout:2/909: getdents d0/d23/daf/dbf/d10f/d55/d92/d10a/d125 0 2024-03-29T08:07:45.275 INFO:tasks.workunit.client.0.smithi037.stdout:2/910: symlink d0/d53/d89/l13e 0 2024-03-29T08:07:45.276 INFO:tasks.workunit.client.0.smithi037.stdout:0/996: truncate d9/da/dd/d20/d55/fb6 718136 0 2024-03-29T08:07:45.293 INFO:tasks.workunit.client.0.smithi037.stdout:2/911: dwrite d0/d23/d44/dca/d7d/fdc [0,4194304] 0 2024-03-29T08:07:45.298 INFO:tasks.workunit.client.0.smithi037.stdout:2/912: creat d0/d53/d89/f13f x:0 0 0 2024-03-29T08:07:45.302 INFO:tasks.workunit.client.0.smithi037.stdout:2/913: write d0/d23/d44/dca/d2e/fd2 [2462330,104215] 0 2024-03-29T08:07:45.303 INFO:tasks.workunit.client.0.smithi037.stdout:0/997: dread d9/da/dd/d13/f95 [0,4194304] 0 2024-03-29T08:07:45.310 INFO:tasks.workunit.client.0.smithi037.stdout:2/914: creat d0/d23/d44/dca/d2e/d5f/d6f/d5e/dc2/f140 x:0 0 0 2024-03-29T08:07:45.314 INFO:tasks.workunit.client.0.smithi037.stdout:2/915: mknod d0/d23/d70/c141 0 2024-03-29T08:07:45.317 INFO:tasks.workunit.client.0.smithi037.stdout:2/916: truncate d0/d23/d44/df6/ffa 957463 0 2024-03-29T08:07:45.321 INFO:tasks.workunit.client.0.smithi037.stdout:2/917: symlink d0/d23/d44/dca/d2e/d5f/da5/l142 0 2024-03-29T08:07:45.333 INFO:tasks.workunit.client.0.smithi037.stdout:0/998: dread d9/da/dd/d20/d72/f85 [0,4194304] 0 2024-03-29T08:07:45.342 INFO:tasks.workunit.client.0.smithi037.stdout:2/918: dwrite d0/f8a [0,4194304] 0 2024-03-29T08:07:45.346 INFO:tasks.workunit.client.0.smithi037.stdout:2/919: write d0/d23/d44/dca/f3a [7451094,31237] 0 2024-03-29T08:07:45.351 INFO:tasks.workunit.client.0.smithi037.stdout:2/920: write d0/d23/f30 [334678,78227] 0 2024-03-29T08:07:45.358 INFO:tasks.workunit.client.0.smithi037.stdout:2/921: mkdir d0/d23/d44/dca/d2e/da4/d143 0 2024-03-29T08:07:45.359 INFO:tasks.workunit.client.0.smithi037.stdout:0/999: dwrite d9/d74/fe7 [0,4194304] 0 2024-03-29T08:07:45.361 INFO:tasks.workunit.client.0.smithi037.stdout:2/922: write d0/d23/daf/dbf/d10f/f3c [2463156,60652] 0 2024-03-29T08:07:45.368 INFO:tasks.workunit.client.0.smithi037.stdout:2/923: truncate d0/d23/d44/dca/d2e/d5f/d6f/fd7 779098 0 2024-03-29T08:07:45.392 INFO:tasks.workunit.client.0.smithi037.stdout:2/924: dread d0/d23/d44/f67 [0,4194304] 0 2024-03-29T08:07:45.398 INFO:tasks.workunit.client.0.smithi037.stdout:2/925: rmdir d0/d23/d44/dca/d2e/d9c/df3/d13b 0 2024-03-29T08:07:45.404 INFO:tasks.workunit.client.0.smithi037.stdout:2/926: dread d0/d23/d44/dca/d2e/d38/f94 [0,4194304] 0 2024-03-29T08:07:45.407 INFO:tasks.workunit.client.0.smithi037.stdout:2/927: creat d0/d23/d44/df6/f144 x:0 0 0 2024-03-29T08:07:45.426 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:07:45 smithi037 ceph-mon[35162]: pgmap v54: 65 pgs: 65 active+clean; 3.6 GiB data, 12 GiB used, 524 GiB / 536 GiB avail; 87 MiB/s rd, 178 MiB/s wr, 506 op/s 2024-03-29T08:07:45.464 INFO:tasks.workunit.client.0.smithi037.stdout:3/933: sync 2024-03-29T08:07:45.487 INFO:tasks.workunit.client.0.smithi037.stdout:3/934: dread dd/d15/d18/d30/d115/d2d/d3a/d6c/db1/dc0/dda/ddb/f5d [0,4194304] 0 2024-03-29T08:07:45.506 INFO:tasks.workunit.client.0.smithi037.stdout:3/935: dwrite dd/d15/d18/d4b/f109 [0,4194304] 0 2024-03-29T08:07:45.525 INFO:tasks.workunit.client.0.smithi037.stdout:3/936: getdents dd/d15/d18/d4b/da2 0 2024-03-29T08:07:45.544 INFO:tasks.workunit.client.0.smithi037.stdout:3/937: dwrite dd/d15/d18/d30/d115/d2d/d3a/d6c/d10e/f12e [0,4194304] 0 2024-03-29T08:07:45.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:07:45 smithi148 ceph-mon[44992]: pgmap v54: 65 pgs: 65 active+clean; 3.6 GiB data, 12 GiB used, 524 GiB / 536 GiB avail; 87 MiB/s rd, 178 MiB/s wr, 506 op/s 2024-03-29T08:07:45.577 INFO:tasks.workunit.client.0.smithi037.stdout:3/938: dread dd/ff [0,4194304] 0 2024-03-29T08:07:45.581 INFO:tasks.workunit.client.0.smithi037.stdout:3/939: chown dd/d15/d18/d4b/feb 94156496 1 2024-03-29T08:07:45.585 INFO:tasks.workunit.client.0.smithi037.stdout:3/940: readlink dd/d15/d18/d30/d115/d2d/d3a/d6c/db1/dc0/dda/d33/d75/lce 0 2024-03-29T08:07:45.587 INFO:tasks.workunit.client.0.smithi037.stdout:3/941: dread - dd/d15/d18/d30/d115/d2d/d3a/d6c/f136 zero size 2024-03-29T08:07:45.609 INFO:tasks.workunit.client.0.smithi037.stdout:3/942: dread dd/d15/d18/d30/d115/d2d/d3a/d6c/db1/dc0/dda/d33/f96 [0,4194304] 0 2024-03-29T08:07:45.613 INFO:tasks.workunit.client.0.smithi037.stdout:3/943: mkdir dd/d15/d18/d30/d115/d2d/d3a/d6c/db1/dc0/dda/d33/d97/dd6/d13a 0 2024-03-29T08:07:45.618 INFO:tasks.workunit.client.0.smithi037.stdout:3/944: rename dd/d15/d18/d30/d115/d3e/d47/cf6 to dd/c13b 0 2024-03-29T08:07:45.621 INFO:tasks.workunit.client.0.smithi037.stdout:3/945: chown dd/d15/d18/d30/d115/d3e/d47/lf2 7 1 2024-03-29T08:07:45.639 INFO:tasks.workunit.client.0.smithi037.stdout:3/946: dwrite dd/d15/d18/d4b/feb [0,4194304] 0 2024-03-29T08:07:45.641 INFO:tasks.workunit.client.0.smithi037.stdout:8/971: sync 2024-03-29T08:07:45.644 INFO:tasks.workunit.client.0.smithi037.stdout:3/947: symlink dd/d15/d18/d30/d115/d3e/d47/l13c 0 2024-03-29T08:07:45.645 INFO:tasks.workunit.client.0.smithi037.stdout:8/972: readlink d5/d10/d2d/d9c/df8/d4c/d67/l8c 0 2024-03-29T08:07:45.647 INFO:tasks.workunit.client.0.smithi037.stdout:3/948: chown dd/d15/d18/d30/dcf/fec 2715 1 2024-03-29T08:07:45.648 INFO:tasks.workunit.client.0.smithi037.stdout:8/973: creat d5/d78/dac/f140 x:0 0 0 2024-03-29T08:07:45.652 INFO:tasks.workunit.client.0.smithi037.stdout:8/974: mknod d5/d10/d45/d4b/c141 0 2024-03-29T08:07:45.657 INFO:tasks.workunit.client.0.smithi037.stdout:8/975: getdents d5/d10/d2d/da7 0 2024-03-29T08:07:45.661 INFO:tasks.workunit.client.0.smithi037.stdout:8/976: dread - d5/d10/d2d/d9c/df8/d4c/d67/faa zero size 2024-03-29T08:07:45.667 INFO:tasks.workunit.client.0.smithi037.stdout:8/977: write d5/d10/d2d/d9c/df8/d4c/d67/f10c [1049420,109081] 0 2024-03-29T08:07:45.669 INFO:tasks.workunit.client.0.smithi037.stdout:8/978: chown d5/d10/d45/d4b/c141 93 1 2024-03-29T08:07:45.673 INFO:tasks.workunit.client.0.smithi037.stdout:3/949: dread dd/d5e/fbb [0,4194304] 0 2024-03-29T08:07:45.674 INFO:tasks.workunit.client.0.smithi037.stdout:8/979: unlink d5/c3c 0 2024-03-29T08:07:45.675 INFO:tasks.workunit.client.0.smithi037.stdout:3/950: chown dd/c13b 16570 1 2024-03-29T08:07:45.676 INFO:tasks.workunit.client.0.smithi037.stdout:8/980: chown d5/d10/d2d/d34/l132 60553836 1 2024-03-29T08:07:45.678 INFO:tasks.workunit.client.0.smithi037.stdout:8/981: dread - d5/d10/d2d/d9c/df8/dc8/de6/f12d zero size 2024-03-29T08:07:45.682 INFO:tasks.workunit.client.0.smithi037.stdout:8/982: mkdir d5/d10/d45/d142 0 2024-03-29T08:07:45.686 INFO:tasks.workunit.client.0.smithi037.stdout:8/983: rename d5/d10/l3b to d5/d10/dfa/l143 0 2024-03-29T08:07:45.692 INFO:tasks.workunit.client.0.smithi037.stdout:3/951: dwrite dd/d15/d18/d30/d115/d2d/d3a/d6c/db1/dc0/dda/d33/d106/f125 [0,4194304] 0 2024-03-29T08:07:45.693 INFO:tasks.workunit.client.0.smithi037.stdout:2/928: sync 2024-03-29T08:07:45.700 INFO:tasks.workunit.client.0.smithi037.stdout:3/952: link dd/d5e/faa dd/d5e/d7a/f13d 0 2024-03-29T08:07:45.708 INFO:tasks.workunit.client.0.smithi037.stdout:8/984: dwrite d5/d10/d19/d8d/f90 [0,4194304] 0 2024-03-29T08:07:45.711 INFO:tasks.workunit.client.0.smithi037.stdout:8/985: chown d5/d10/f23 25 1 2024-03-29T08:07:45.717 INFO:tasks.workunit.client.0.smithi037.stdout:2/929: dwrite d0/d23/d70/f79 [0,4194304] 0 2024-03-29T08:07:45.720 INFO:tasks.workunit.client.0.smithi037.stdout:3/953: dwrite dd/d5e/f80 [0,4194304] 0 2024-03-29T08:07:45.722 INFO:tasks.workunit.client.0.smithi037.stdout:8/986: rename d5/d10/dfa/l143 to d5/d10/d2d/d34/d50/d83/d46/d4e/l144 0 2024-03-29T08:07:45.729 INFO:tasks.workunit.client.0.smithi037.stdout:2/930: readlink d0/d23/daf/dbf/d10f/d55/l108 0 2024-03-29T08:07:45.737 INFO:tasks.workunit.client.0.smithi037.stdout:3/954: rename dd/d15/d18/d30/d115/d2d/d3a/d6c/db1/dc0/dda/fa4 to dd/d15/d18/d30/d115/d3e/f13e 0 2024-03-29T08:07:45.741 INFO:tasks.workunit.client.0.smithi037.stdout:3/955: chown dd/d15/d18/d30/dcf/c112 7 1 2024-03-29T08:07:45.749 INFO:tasks.workunit.client.0.smithi037.stdout:2/931: dwrite d0/d23/d44/f105 [0,4194304] 0 2024-03-29T08:07:45.752 INFO:tasks.workunit.client.0.smithi037.stdout:8/987: write d5/d10/d2d/d9c/df8/d4c/d67/d70/fcc [558887,109085] 0 2024-03-29T08:07:45.753 INFO:tasks.workunit.client.0.smithi037.stdout:3/956: write dd/d5e/f45 [4716212,24616] 0 2024-03-29T08:07:45.762 INFO:tasks.workunit.client.0.smithi037.stdout:3/957: creat dd/d15/d18/d30/d115/d2d/d3a/d6c/db1/dc0/dda/d33/d97/f13f x:0 0 0 2024-03-29T08:07:45.766 INFO:tasks.workunit.client.0.smithi037.stdout:2/932: write d0/d23/d44/dca/d2e/d5f/d6f/fb7 [1177293,2294] 0 2024-03-29T08:07:45.769 INFO:tasks.workunit.client.0.smithi037.stdout:3/958: creat dd/d15/d18/d30/d115/d2d/d3a/d6c/f140 x:0 0 0 2024-03-29T08:07:45.770 INFO:tasks.workunit.client.0.smithi037.stdout:3/959: stat dd/d5e/ce5 0 2024-03-29T08:07:45.771 INFO:tasks.workunit.client.0.smithi037.stdout:2/933: symlink d0/d23/d44/l145 0 2024-03-29T08:07:45.776 INFO:tasks.workunit.client.0.smithi037.stdout:2/934: getdents d0/d23/daf 0 2024-03-29T08:07:45.777 INFO:tasks.workunit.client.0.smithi037.stdout:8/988: dread d5/d10/d19/d3d/f5d [0,4194304] 0 2024-03-29T08:07:45.777 INFO:tasks.workunit.client.0.smithi037.stdout:3/960: symlink dd/d15/d18/d30/d115/d2d/d3a/d6c/db1/dc0/dda/d33/d75/l141 0 2024-03-29T08:07:45.783 INFO:tasks.workunit.client.0.smithi037.stdout:8/989: creat d5/d10/d2d/df5/f145 x:0 0 0 2024-03-29T08:07:45.792 INFO:tasks.workunit.client.0.smithi037.stdout:8/990: truncate d5/d10/d2d/d9c/df8/d4c/d67/d85/ff3 545168 0 2024-03-29T08:07:45.800 INFO:tasks.workunit.client.0.smithi037.stdout:2/935: dwrite d0/d23/d44/dca/d2e/d5f/d6f/d5e/f74 [0,4194304] 0 2024-03-29T08:07:45.802 INFO:tasks.workunit.client.0.smithi037.stdout:8/991: write d5/d10/d2d/d34/d50/f54 [843988,75391] 0 2024-03-29T08:07:45.807 INFO:tasks.workunit.client.0.smithi037.stdout:2/936: creat d0/d23/daf/dbf/d10f/d55/f146 x:0 0 0 2024-03-29T08:07:45.808 INFO:tasks.workunit.client.0.smithi037.stdout:8/992: symlink d5/d10/d2d/l146 0 2024-03-29T08:07:45.813 INFO:tasks.workunit.client.0.smithi037.stdout:3/961: dread dd/d15/d18/d30/d115/f3c [0,4194304] 0 2024-03-29T08:07:45.813 INFO:tasks.workunit.client.0.smithi037.stdout:8/993: mknod d5/d10/d2d/d34/d50/d83/c147 0 2024-03-29T08:07:45.815 INFO:tasks.workunit.client.0.smithi037.stdout:8/994: chown d5/f80 102 1 2024-03-29T08:07:45.820 INFO:tasks.workunit.client.0.smithi037.stdout:2/937: dread d0/d23/d44/dca/d2e/da4/fb4 [0,4194304] 0 2024-03-29T08:07:45.826 INFO:tasks.workunit.client.0.smithi037.stdout:2/938: write d0/d23/d44/dca/d2e/d5f/fe1 [1298638,11315] 0 2024-03-29T08:07:45.830 INFO:tasks.workunit.client.0.smithi037.stdout:2/939: chown d0/d53/d89/lb9 58910 1 2024-03-29T08:07:45.834 INFO:tasks.workunit.client.0.smithi037.stdout:3/962: dwrite dd/f123 [0,4194304] 0 2024-03-29T08:07:45.840 INFO:tasks.workunit.client.0.smithi037.stdout:2/940: getdents d0/d23/d70 0 2024-03-29T08:07:45.840 INFO:tasks.workunit.client.0.smithi037.stdout:8/995: dwrite d5/d10/d45/d4b/ffc [0,4194304] 0 2024-03-29T08:07:45.852 INFO:tasks.workunit.client.0.smithi037.stdout:2/941: mknod d0/d23/daf/dbf/d10f/c147 0 2024-03-29T08:07:45.854 INFO:tasks.workunit.client.0.smithi037.stdout:8/996: truncate d5/d10/d2d/fc6 1563988 0 2024-03-29T08:07:45.859 INFO:tasks.workunit.client.0.smithi037.stdout:8/997: chown d5/d10/d2d/d9c/da1/dbc/f125 178 1 2024-03-29T08:07:45.862 INFO:tasks.workunit.client.0.smithi037.stdout:2/942: creat d0/d23/daf/dbf/d10f/d4e/f148 x:0 0 0 2024-03-29T08:07:45.862 INFO:tasks.workunit.client.0.smithi037.stdout:3/963: dwrite dd/d15/d18/d30/d115/d2d/fcb [0,4194304] 0 2024-03-29T08:07:45.868 INFO:tasks.workunit.client.0.smithi037.stdout:2/943: creat d0/d53/d89/f149 x:0 0 0 2024-03-29T08:07:45.870 INFO:tasks.workunit.client.0.smithi037.stdout:3/964: mknod dd/d15/d18/d30/d115/d2d/d3a/d6c/db1/dc0/dda/c142 0 2024-03-29T08:07:45.872 INFO:tasks.workunit.client.0.smithi037.stdout:8/998: write d5/d10/d2d/f3e [1855626,102095] 0 2024-03-29T08:07:45.883 INFO:tasks.workunit.client.0.smithi037.stdout:8/999: dread - d5/d10/d2d/d34/d50/d83/d46/d4e/f66 zero size 2024-03-29T08:07:45.886 INFO:tasks.workunit.client.0.smithi037.stdout:2/944: link d0/d23/d44/dca/d2e/d5f/ld5 d0/d23/d44/dca/d2e/d38/l14a 0 2024-03-29T08:07:45.889 INFO:tasks.workunit.client.0.smithi037.stdout:2/945: mkdir d0/d14b 0 2024-03-29T08:07:45.892 INFO:tasks.workunit.client.0.smithi037.stdout:2/946: creat d0/d23/d113/d127/f14c x:0 0 0 2024-03-29T08:07:45.896 INFO:tasks.workunit.client.0.smithi037.stdout:2/947: creat d0/d23/d44/dca/d2e/da4/d143/f14d x:0 0 0 2024-03-29T08:07:45.918 INFO:tasks.workunit.client.0.smithi037.stdout:2/948: dread d0/d23/d44/dca/d2e/da4/fec [0,4194304] 0 2024-03-29T08:07:45.931 INFO:tasks.workunit.client.0.smithi037.stdout:2/949: dwrite d0/d23/daf/dbf/d10f/f3c [4194304,4194304] 0 2024-03-29T08:07:45.935 INFO:tasks.workunit.client.0.smithi037.stdout:2/950: mkdir d0/d23/d44/d12a/d14e 0 2024-03-29T08:07:45.938 INFO:tasks.workunit.client.0.smithi037.stdout:2/951: mknod d0/d53/c14f 0 2024-03-29T08:07:45.945 INFO:tasks.workunit.client.0.smithi037.stdout:2/952: dread d0/d23/d44/dca/d2e/d38/d68/fac [0,4194304] 0 2024-03-29T08:07:45.963 INFO:tasks.workunit.client.0.smithi037.stdout:2/953: dwrite d0/d23/d44/dca/d2e/d9c/df3/fff [0,4194304] 0 2024-03-29T08:07:45.966 INFO:tasks.workunit.client.0.smithi037.stdout:2/954: chown d0/d23/daf/dbf/d10f/d4e/c117 0 1 2024-03-29T08:07:45.970 INFO:tasks.workunit.client.0.smithi037.stdout:2/955: rmdir d0/d23/daf/dbf/d10f/d55 39 2024-03-29T08:07:45.975 INFO:tasks.workunit.client.0.smithi037.stdout:2/956: mkdir d0/d23/d44/dca/d7d/dfb/d150 0 2024-03-29T08:07:45.980 INFO:tasks.workunit.client.0.smithi037.stdout:2/957: getdents d0/d23/d44/dca/d2e/d5f/da5 0 2024-03-29T08:07:45.983 INFO:tasks.workunit.client.0.smithi037.stdout:2/958: rmdir d0/d23/d44/d106 39 2024-03-29T08:07:45.989 INFO:tasks.workunit.client.0.smithi037.stdout:2/959: dread d0/d23/d44/dca/d2e/d5f/d6f/d5e/fc1 [0,4194304] 0 2024-03-29T08:07:45.991 INFO:tasks.workunit.client.0.smithi037.stdout:2/960: chown d0/d23/d44/dca/d2e/d5f/d6f/d5e/l87 1553964 1 2024-03-29T08:07:45.995 INFO:tasks.workunit.client.0.smithi037.stdout:2/961: creat d0/d23/d44/d12a/d14e/f151 x:0 0 0 2024-03-29T08:07:45.997 INFO:tasks.workunit.client.0.smithi037.stdout:2/962: chown d0/d23/c132 170 1 2024-03-29T08:07:46.000 INFO:tasks.workunit.client.0.smithi037.stdout:2/963: mkdir d0/d23/d44/dca/d2e/d5f/d152 0 2024-03-29T08:07:46.004 INFO:tasks.workunit.client.0.smithi037.stdout:2/964: unlink d0/d23/d44/d12a/c135 0 2024-03-29T08:07:46.007 INFO:tasks.workunit.client.0.smithi037.stdout:2/965: read d0/d23/daf/dbf/f101 [156947,42376] 0 2024-03-29T08:07:46.016 INFO:tasks.workunit.client.0.smithi037.stdout:2/966: truncate d0/d53/f11e 2321169 0 2024-03-29T08:07:46.020 INFO:tasks.workunit.client.0.smithi037.stdout:2/967: truncate d0/d23/d44/d62/fb5 617251 0 2024-03-29T08:07:46.024 INFO:tasks.workunit.client.0.smithi037.stdout:2/968: dread d0/d23/daf/dbf/f101 [0,4194304] 0 2024-03-29T08:07:46.046 INFO:tasks.workunit.client.0.smithi037.stdout:2/969: dread d0/f14 [0,4194304] 0 2024-03-29T08:07:46.066 INFO:tasks.workunit.client.0.smithi037.stdout:2/970: dwrite d0/f8e [0,4194304] 0 2024-03-29T08:07:46.086 INFO:tasks.workunit.client.0.smithi037.stdout:2/971: dwrite d0/d23/d44/dca/d2e/d5f/d6f/fb7 [0,4194304] 0 2024-03-29T08:07:46.092 INFO:tasks.workunit.client.0.smithi037.stdout:2/972: dread - d0/d23/daf/dbf/d10f/d55/d92/d10a/f13d zero size 2024-03-29T08:07:46.101 INFO:tasks.workunit.client.0.smithi037.stdout:2/973: write d0/d23/d44/dca/f3d [239514,46964] 0 2024-03-29T08:07:46.106 INFO:tasks.workunit.client.0.smithi037.stdout:2/974: rename d0/d23/d44/f67 to d0/d23/d44/dca/d2e/d38/f153 0 2024-03-29T08:07:46.113 INFO:tasks.workunit.client.0.smithi037.stdout:2/975: link d0/d23/d44/dca/d2e/f36 d0/d23/d44/dca/d7d/dfb/d150/f154 0 2024-03-29T08:07:46.115 INFO:tasks.workunit.client.0.smithi037.stdout:2/976: symlink d0/l155 0 2024-03-29T08:07:46.117 INFO:tasks.workunit.client.0.smithi037.stdout:2/977: stat d0/d23/d44/dca/d2e/d5f/d6f/d5e/fb1 0 2024-03-29T08:07:46.121 INFO:tasks.workunit.client.0.smithi037.stdout:2/978: creat d0/d23/daf/dbf/d10f/d55/d92/d11f/d131/f156 x:0 0 0 2024-03-29T08:07:46.123 INFO:tasks.workunit.client.0.smithi037.stdout:2/979: read d0/d23/d44/dca/d2e/d5f/d6f/fa9 [3722014,106829] 0 2024-03-29T08:07:46.125 INFO:tasks.workunit.client.0.smithi037.stdout:2/980: readlink d0/d23/d44/dca/d2e/d9c/la6 0 2024-03-29T08:07:46.129 INFO:tasks.workunit.client.0.smithi037.stdout:2/981: creat d0/d23/d44/d62/f157 x:0 0 0 2024-03-29T08:07:46.133 INFO:tasks.workunit.client.0.smithi037.stdout:2/982: mknod d0/d23/d44/dca/d2e/d9c/df3/d78/d129/c158 0 2024-03-29T08:07:46.149 INFO:tasks.workunit.client.0.smithi037.stdout:2/983: dwrite d0/d23/f30 [4194304,4194304] 0 2024-03-29T08:07:46.156 INFO:tasks.workunit.client.0.smithi037.stdout:2/984: write d0/d23/d44/dca/d2e/d9c/df3/f72 [749153,68758] 0 2024-03-29T08:07:46.178 INFO:tasks.workunit.client.0.smithi037.stdout:2/985: dwrite d0/d23/d44/dca/d2e/d9c/df3/fff [0,4194304] 0 2024-03-29T08:07:46.182 INFO:tasks.workunit.client.0.smithi037.stdout:2/986: rename d0/d23/daf/dbf/d10f/d4e/lb2 to d0/d23/d44/d12a/d14e/l159 0 2024-03-29T08:07:46.185 INFO:tasks.workunit.client.0.smithi037.stdout:2/987: symlink d0/d23/l15a 0 2024-03-29T08:07:46.193 INFO:tasks.workunit.client.0.smithi037.stdout:2/988: dread - d0/d23/d44/dca/d7d/fcc zero size 2024-03-29T08:07:46.426 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:07:46 smithi037 ceph-mon[35162]: pgmap v55: 65 pgs: 65 active+clean; 3.6 GiB data, 12 GiB used, 524 GiB / 536 GiB avail; 37 MiB/s rd, 79 MiB/s wr, 262 op/s 2024-03-29T08:07:46.437 INFO:tasks.workunit.client.0.smithi037.stdout:3/965: sync 2024-03-29T08:07:46.441 INFO:tasks.workunit.client.0.smithi037.stdout:3/966: creat dd/d15/d18/d30/d115/d2d/d3a/d6c/db1/dc0/dda/d33/d75/f143 x:0 0 0 2024-03-29T08:07:46.448 INFO:tasks.workunit.client.0.smithi037.stdout:3/967: rename dd/d15/d18/d30/d115/f37 to dd/d15/d18/d30/d115/d2d/d3a/d6c/db1/dc0/dda/d33/d97/f144 0 2024-03-29T08:07:46.450 INFO:tasks.workunit.client.0.smithi037.stdout:3/968: mknod dd/d15/d18/d30/d115/d3e/d47/c145 0 2024-03-29T08:07:46.461 INFO:tasks.workunit.client.0.smithi037.stdout:3/969: truncate dd/d15/d18/d30/d115/f71 3257906 0 2024-03-29T08:07:46.476 INFO:tasks.workunit.client.0.smithi037.stdout:3/970: dread dd/f3f [0,4194304] 0 2024-03-29T08:07:46.480 INFO:tasks.workunit.client.0.smithi037.stdout:3/971: symlink dd/d15/d18/d30/d115/d2d/d3a/d6c/db1/dc0/dda/d33/d97/l146 0 2024-03-29T08:07:46.483 INFO:tasks.workunit.client.0.smithi037.stdout:3/972: fsync dd/d5e/f5c 0 2024-03-29T08:07:46.485 INFO:tasks.workunit.client.0.smithi037.stdout:3/973: chown dd/f69 388645 1 2024-03-29T08:07:46.490 INFO:tasks.workunit.client.0.smithi037.stdout:3/974: chown dd/d15/d18/d30/d115/d2d/d3a/d6c/db1/dc0/dda/ddb/f6a 40 1 2024-03-29T08:07:46.495 INFO:tasks.workunit.client.0.smithi037.stdout:3/975: creat dd/d15/d18/d30/d115/d2d/d3a/d85/f147 x:0 0 0 2024-03-29T08:07:46.511 INFO:tasks.workunit.client.0.smithi037.stdout:2/989: sync 2024-03-29T08:07:46.512 INFO:tasks.workunit.client.0.smithi037.stdout:3/976: dread dd/d15/d18/d30/d115/f3c [0,4194304] 0 2024-03-29T08:07:46.528 INFO:tasks.workunit.client.0.smithi037.stdout:2/990: dwrite d0/d23/d44/fc5 [0,4194304] 0 2024-03-29T08:07:46.532 INFO:tasks.workunit.client.0.smithi037.stdout:2/991: dread - d0/d23/daf/dbf/d10f/d55/d92/d11f/d131/f156 zero size 2024-03-29T08:07:46.536 INFO:tasks.workunit.client.0.smithi037.stdout:3/977: link dd/d5e/d7a/dd4/cef dd/d15/d18/d30/d115/d2d/d3a/d6c/db1/dc0/dda/d33/d75/c148 0 2024-03-29T08:07:46.537 INFO:tasks.workunit.client.0.smithi037.stdout:2/992: write d0/d23/d44/dca/d2e/fd2 [1241408,27929] 0 2024-03-29T08:07:46.545 INFO:tasks.workunit.client.0.smithi037.stdout:3/978: truncate dd/d15/d18/d4b/f11b 195254 0 2024-03-29T08:07:46.550 INFO:tasks.workunit.client.0.smithi037.stdout:3/979: write dd/d15/d18/d4b/ff9 [4214788,30812] 0 2024-03-29T08:07:46.555 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:07:46 smithi148 ceph-mon[44992]: pgmap v55: 65 pgs: 65 active+clean; 3.6 GiB data, 12 GiB used, 524 GiB / 536 GiB avail; 37 MiB/s rd, 79 MiB/s wr, 262 op/s 2024-03-29T08:07:46.559 INFO:tasks.workunit.client.0.smithi037.stdout:2/993: dwrite d0/d23/daf/dbf/fde [0,4194304] 0 2024-03-29T08:07:46.569 INFO:tasks.workunit.client.0.smithi037.stdout:2/994: link d0/d23/daf/dbf/d10f/d55/d92/d10a/d125/f12b d0/d23/d44/dca/d2e/d9c/df3/f15b 0 2024-03-29T08:07:46.584 INFO:tasks.workunit.client.0.smithi037.stdout:2/995: dwrite d0/d23/d44/dca/d7d/dfb/d150/f154 [0,4194304] 0 2024-03-29T08:07:46.587 INFO:tasks.workunit.client.0.smithi037.stdout:2/996: dread - d0/d23/d44/dca/d2e/d38/f93 zero size 2024-03-29T08:07:46.589 INFO:tasks.workunit.client.0.smithi037.stdout:2/997: stat d0/d23/d44/dca/ff1 0 2024-03-29T08:07:46.597 INFO:tasks.workunit.client.0.smithi037.stdout:2/998: link d0/d23/d44/dca/d2e/d5f/d6f/fd7 d0/d23/d44/dca/d2e/d5f/da5/f15c 0 2024-03-29T08:07:46.602 INFO:tasks.workunit.client.0.smithi037.stdout:2/999: creat d0/d23/d44/dca/d2e/d9c/d107/f15d x:0 0 0 2024-03-29T08:07:46.854 INFO:tasks.workunit.client.0.smithi037.stdout:3/980: sync 2024-03-29T08:07:46.869 INFO:tasks.workunit.client.0.smithi037.stdout:3/981: dwrite dd/d15/d18/d30/d115/d2d/d3a/d6c/d10e/f12e [0,4194304] 0 2024-03-29T08:07:46.876 INFO:tasks.workunit.client.0.smithi037.stdout:3/982: creat dd/d15/d18/d30/d115/d2d/d3a/d6c/db1/dc0/dda/ddb/df0/f149 x:0 0 0 2024-03-29T08:07:46.876 INFO:tasks.workunit.client.0.smithi037.stdout:3/983: chown dd/c13b 141738 1 2024-03-29T08:07:46.879 INFO:tasks.workunit.client.0.smithi037.stdout:3/984: fsync dd/d15/d18/d30/f9d 0 2024-03-29T08:07:46.885 INFO:tasks.workunit.client.0.smithi037.stdout:3/985: link dd/d15/d18/d4b/da2/f135 dd/d15/d18/d30/d115/f14a 0 2024-03-29T08:07:46.886 INFO:tasks.workunit.client.0.smithi037.stdout:3/986: readlink dd/l17 0 2024-03-29T08:07:46.890 INFO:tasks.workunit.client.0.smithi037.stdout:3/987: chown dd/d15/d18/d30/d115/d2d/d3a/d6c/db1/dc0/dda/d33/d84/l86 20725043 1 2024-03-29T08:07:46.898 INFO:tasks.workunit.client.0.smithi037.stdout:3/988: write dd/d15/d18/d30/d115/d2d/d3a/d6c/db1/dc0/dda/d33/d75/f8c [694857,84450] 0 2024-03-29T08:07:46.901 INFO:tasks.workunit.client.0.smithi037.stdout:3/989: readlink dd/d15/d18/d30/d115/d2d/d3a/d6c/db1/dc0/dda/d33/d75/l141 0 2024-03-29T08:07:46.904 INFO:tasks.workunit.client.0.smithi037.stdout:3/990: fsync dd/d15/d18/d30/d115/d2d/d3a/d6c/db1/dc0/dda/ddb/f5a 0 2024-03-29T08:07:46.926 INFO:tasks.workunit.client.0.smithi037.stdout:3/991: dread dd/f69 [0,4194304] 0 2024-03-29T08:07:46.933 INFO:tasks.workunit.client.0.smithi037.stdout:3/992: write dd/d15/d18/d30/d115/f71 [3564033,127354] 0 2024-03-29T08:07:46.942 INFO:tasks.workunit.client.0.smithi037.stdout:3/993: rename dd/d15/d18/d30/d115/d2d/d3a/d6c/cdf to dd/d15/d18/d30/d115/d2d/d3a/d6c/db1/dc0/dda/d33/d97/dd6/c14b 0 2024-03-29T08:07:46.944 INFO:tasks.workunit.client.0.smithi037.stdout:3/994: truncate dd/f24 2103666 0 2024-03-29T08:07:46.947 INFO:tasks.workunit.client.0.smithi037.stdout:3/995: creat dd/d15/d18/d30/dcf/f14c x:0 0 0 2024-03-29T08:07:46.952 INFO:tasks.workunit.client.0.smithi037.stdout:3/996: creat dd/d15/d18/d30/d115/d2d/d3a/d6c/db1/dc0/dda/d33/d83/f14d x:0 0 0 2024-03-29T08:07:46.968 INFO:tasks.workunit.client.0.smithi037.stdout:3/997: dwrite dd/d15/d18/d30/d115/d3e/f7c [0,4194304] 0 2024-03-29T08:07:46.991 INFO:tasks.workunit.client.0.smithi037.stdout:3/998: dwrite dd/d15/d18/d30/d115/d2d/d3a/d6c/db1/dc0/dda/f41 [4194304,4194304] 0 2024-03-29T08:07:46.994 INFO:tasks.workunit.client.0.smithi037.stdout:3/999: mkdir dd/d5e/d14e 0 2024-03-29T08:07:47.003 INFO:tasks.workunit.client.0.smithi037.stderr:+ rm -rf -- ./tmp.Rc0GmqBkLJ 2024-03-29T08:07:47.364 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:07:47 smithi037 ceph-mon[35162]: from='mgr.14602 172.21.15.148:0/667342922' entity='mgr.smithi148.onhgvm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:07:47.555 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:07:47 smithi148 ceph-mon[44992]: from='mgr.14602 172.21.15.148:0/667342922' entity='mgr.smithi148.onhgvm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:07:48.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:07:48 smithi037 ceph-mon[35162]: pgmap v56: 65 pgs: 65 active+clean; 3.4 GiB data, 12 GiB used, 525 GiB / 536 GiB avail; 46 MiB/s rd, 101 MiB/s wr, 346 op/s 2024-03-29T08:07:48.555 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:07:48 smithi148 ceph-mon[44992]: pgmap v56: 65 pgs: 65 active+clean; 3.4 GiB data, 12 GiB used, 525 GiB / 536 GiB avail; 46 MiB/s rd, 101 MiB/s wr, 346 op/s 2024-03-29T08:07:50.678 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:07:50 smithi037 ceph-mon[35162]: from='mgr.14602 ' entity='mgr.smithi148.onhgvm' 2024-03-29T08:07:50.678 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:07:50 smithi037 ceph-mon[35162]: from='mgr.14602 ' entity='mgr.smithi148.onhgvm' 2024-03-29T08:07:50.678 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:07:50 smithi037 ceph-mon[35162]: from='mgr.14602 172.21.15.148:0/667342922' entity='mgr.smithi148.onhgvm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T08:07:50.678 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:07:50 smithi037 ceph-mon[35162]: pgmap v57: 65 pgs: 65 active+clean; 3.3 GiB data, 12 GiB used, 525 GiB / 536 GiB avail; 40 MiB/s rd, 91 MiB/s wr, 315 op/s 2024-03-29T08:07:50.678 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:07:50 smithi037 ceph-mon[35162]: from='mgr.14602 ' entity='mgr.smithi148.onhgvm' 2024-03-29T08:07:50.678 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:07:50 smithi037 ceph-mon[35162]: from='mgr.14602 ' entity='mgr.smithi148.onhgvm' 2024-03-29T08:07:50.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:07:50 smithi148 ceph-mon[44992]: from='mgr.14602 ' entity='mgr.smithi148.onhgvm' 2024-03-29T08:07:50.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:07:50 smithi148 ceph-mon[44992]: from='mgr.14602 ' entity='mgr.smithi148.onhgvm' 2024-03-29T08:07:50.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:07:50 smithi148 ceph-mon[44992]: from='mgr.14602 172.21.15.148:0/667342922' entity='mgr.smithi148.onhgvm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T08:07:50.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:07:50 smithi148 ceph-mon[44992]: pgmap v57: 65 pgs: 65 active+clean; 3.3 GiB data, 12 GiB used, 525 GiB / 536 GiB avail; 40 MiB/s rd, 91 MiB/s wr, 315 op/s 2024-03-29T08:07:50.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:07:50 smithi148 ceph-mon[44992]: from='mgr.14602 ' entity='mgr.smithi148.onhgvm' 2024-03-29T08:07:50.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:07:50 smithi148 ceph-mon[44992]: from='mgr.14602 ' entity='mgr.smithi148.onhgvm' 2024-03-29T08:07:53.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:07:52 smithi148 ceph-mon[44992]: pgmap v58: 65 pgs: 65 active+clean; 3.3 GiB data, 12 GiB used, 525 GiB / 536 GiB avail; 32 MiB/s rd, 70 MiB/s wr, 229 op/s 2024-03-29T08:07:53.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:07:52 smithi037 ceph-mon[35162]: pgmap v58: 65 pgs: 65 active+clean; 3.3 GiB data, 12 GiB used, 525 GiB / 536 GiB avail; 32 MiB/s rd, 70 MiB/s wr, 229 op/s 2024-03-29T08:07:54.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:07:54 smithi148 ceph-mon[44992]: from='mgr.14602 ' entity='mgr.smithi148.onhgvm' 2024-03-29T08:07:54.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:07:54 smithi148 ceph-mon[44992]: from='mgr.14602 ' entity='mgr.smithi148.onhgvm' 2024-03-29T08:07:54.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:07:54 smithi148 ceph-mon[44992]: pgmap v59: 65 pgs: 65 active+clean; 2.4 GiB data, 8.9 GiB used, 527 GiB / 536 GiB avail; 32 MiB/s rd, 70 MiB/s wr, 301 op/s 2024-03-29T08:07:54.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:07:54 smithi037 ceph-mon[35162]: from='mgr.14602 ' entity='mgr.smithi148.onhgvm' 2024-03-29T08:07:54.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:07:54 smithi037 ceph-mon[35162]: from='mgr.14602 ' entity='mgr.smithi148.onhgvm' 2024-03-29T08:07:54.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:07:54 smithi037 ceph-mon[35162]: pgmap v59: 65 pgs: 65 active+clean; 2.4 GiB data, 8.9 GiB used, 527 GiB / 536 GiB avail; 32 MiB/s rd, 70 MiB/s wr, 301 op/s 2024-03-29T08:07:56.371 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-03-29T08:07:56.371 DEBUG:teuthology.orchestra.run.smithi148:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.1/client.1/tmp 2024-03-29T08:07:57.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:07:57 smithi148 ceph-mon[44992]: pgmap v60: 65 pgs: 65 active+clean; 2.4 GiB data, 8.9 GiB used, 527 GiB / 536 GiB avail; 15 MiB/s rd, 34 MiB/s wr, 178 op/s 2024-03-29T08:07:57.378 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:07:57 smithi037 ceph-mon[35162]: pgmap v60: 65 pgs: 65 active+clean; 2.4 GiB data, 8.9 GiB used, 527 GiB / 536 GiB avail; 15 MiB/s rd, 34 MiB/s wr, 178 op/s 2024-03-29T08:07:57.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:57.724+0000 7f969bc00700 1 -- 172.21.15.37:0/2409304609 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f96941002b0 msgr2=0x7f9694100720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:07:57.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:57.724+0000 7f969bc00700 1 --2- 172.21.15.37:0/2409304609 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f96941002b0 0x7f9694100720 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f9680004940 tx=0x7f9680009e30 comp rx=0 tx=0).stop 2024-03-29T08:07:57.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:57.724+0000 7f969bc00700 1 -- 172.21.15.37:0/2409304609 shutdown_connections 2024-03-29T08:07:57.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:57.724+0000 7f969bc00700 1 --2- 172.21.15.37:0/2409304609 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f96941002b0 0x7f9694100720 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:07:57.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:57.724+0000 7f969bc00700 1 --2- 172.21.15.37:0/2409304609 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f96940ff000 0x7f96940ff410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:07:57.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:57.724+0000 7f969bc00700 1 -- 172.21.15.37:0/2409304609 >> 172.21.15.37:0/2409304609 conn(0x7f96940fa950 msgr2=0x7f96940fcd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:07:57.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:57.725+0000 7f969bc00700 1 -- 172.21.15.37:0/2409304609 shutdown_connections 2024-03-29T08:07:57.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:57.725+0000 7f969bc00700 1 -- 172.21.15.37:0/2409304609 wait complete. 2024-03-29T08:07:57.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:57.726+0000 7f969bc00700 1 Processor -- start 2024-03-29T08:07:57.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:57.726+0000 7f969bc00700 1 -- start start 2024-03-29T08:07:57.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:57.727+0000 7f969bc00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f96940ff000 0x7f969410fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:07:57.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:57.727+0000 7f969bc00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9694110290 0x7f9694110fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:07:57.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:57.727+0000 7f969bc00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9694110750 con 0x7f9694110290 2024-03-29T08:07:57.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:57.727+0000 7f969bc00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9694110890 con 0x7f96940ff000 2024-03-29T08:07:57.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:57.727+0000 7f9693e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9694110290 0x7f9694110fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:07:57.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:57.727+0000 7f9693e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9694110290 0x7f9694110fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50712/0 (socket says 172.21.15.37:50712) 2024-03-29T08:07:57.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:57.727+0000 7f9693e00700 1 -- 172.21.15.37:0/1634013708 learned_addr learned my addr 172.21.15.37:0/1634013708 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:07:57.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:57.727+0000 7f9699200700 1 --2- 172.21.15.37:0/1634013708 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f96940ff000 0x7f969410fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:07:57.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:57.728+0000 7f9699200700 1 -- 172.21.15.37:0/1634013708 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9694110290 msgr2=0x7f9694110fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:07:57.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:57.728+0000 7f9699200700 1 --2- 172.21.15.37:0/1634013708 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9694110290 0x7f9694110fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:07:57.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:57.728+0000 7f9699200700 1 -- 172.21.15.37:0/1634013708 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9680005040 con 0x7f96940ff000 2024-03-29T08:07:57.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:57.728+0000 7f9699200700 1 --2- 172.21.15.37:0/1634013708 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f96940ff000 0x7f969410fd50 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f9688002820 tx=0x7f9688004570 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:07:57.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:57.728+0000 7f9690c00700 1 -- 172.21.15.37:0/1634013708 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9688007cc0 con 0x7f96940ff000 2024-03-29T08:07:57.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:57.729+0000 7f9690c00700 1 -- 172.21.15.37:0/1634013708 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f968800f040 con 0x7f96940ff000 2024-03-29T08:07:57.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:57.729+0000 7f969bc00700 1 -- 172.21.15.37:0/1634013708 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9694111680 con 0x7f96940ff000 2024-03-29T08:07:57.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:57.729+0000 7f9690c00700 1 -- 172.21.15.37:0/1634013708 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f96880137d0 con 0x7f96940ff000 2024-03-29T08:07:57.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:57.729+0000 7f969bc00700 1 -- 172.21.15.37:0/1634013708 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9694111c40 con 0x7f96940ff000 2024-03-29T08:07:57.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:57.730+0000 7f969bc00700 1 -- 172.21.15.37:0/1634013708 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96740052e0 con 0x7f96940ff000 2024-03-29T08:07:57.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:57.732+0000 7f9690c00700 1 -- 172.21.15.37:0/1634013708 <== mon.1 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 26) v1 ==== 88664+0+0 (secure 0 0 0) 0x7f9688009070 con 0x7f96940ff000 2024-03-29T08:07:57.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:57.734+0000 7f9690c00700 1 --2- 172.21.15.37:0/1634013708 >> [v2:172.21.15.148:6828/1134897184,v1:172.21.15.148:6829/1134897184] conn(0x7f967c06adb0 0x7f967c06d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:07:57.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:57.734+0000 7f9690c00700 1 -- 172.21.15.37:0/1634013708 <== mon.1 v2:172.21.15.148:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f9688089270 con 0x7f96940ff000 2024-03-29T08:07:57.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:57.734+0000 7f9693e00700 1 --2- 172.21.15.37:0/1634013708 >> [v2:172.21.15.148:6828/1134897184,v1:172.21.15.148:6829/1134897184] conn(0x7f967c06adb0 0x7f967c06d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:07:57.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:57.735+0000 7f9693e00700 1 --2- 172.21.15.37:0/1634013708 >> [v2:172.21.15.148:6828/1134897184,v1:172.21.15.148:6829/1134897184] conn(0x7f967c06adb0 0x7f967c06d260 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f9680009fd0 tx=0x7f9680009e30 comp rx=0 tx=0).ready entity=mgr.14602 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:07:57.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:57.738+0000 7f9690c00700 1 -- 172.21.15.37:0/1634013708 <== mon.1 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181579 (secure 0 0 0) 0x7f96880572f0 con 0x7f96940ff000 2024-03-29T08:07:58.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:58.106+0000 7f969bc00700 1 -- 172.21.15.37:0/1634013708 --> [v2:172.21.15.148:6828/1134897184,v1:172.21.15.148:6829/1134897184] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f9674000bf0 con 0x7f967c06adb0 2024-03-29T08:07:58.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:58.108+0000 7f9690c00700 1 -- 172.21.15.37:0/1634013708 <== mgr.14602 v2:172.21.15.148:6828/1134897184 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+360 (secure 0 0 0) 0x7f9674000bf0 con 0x7f967c06adb0 2024-03-29T08:07:58.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:58.112+0000 7f969bc00700 1 -- 172.21.15.37:0/1634013708 >> [v2:172.21.15.148:6828/1134897184,v1:172.21.15.148:6829/1134897184] conn(0x7f967c06adb0 msgr2=0x7f967c06d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:07:58.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:58.112+0000 7f969bc00700 1 --2- 172.21.15.37:0/1634013708 >> [v2:172.21.15.148:6828/1134897184,v1:172.21.15.148:6829/1134897184] conn(0x7f967c06adb0 0x7f967c06d260 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f9680009fd0 tx=0x7f9680009e30 comp rx=0 tx=0).stop 2024-03-29T08:07:58.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:58.112+0000 7f969bc00700 1 -- 172.21.15.37:0/1634013708 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f96940ff000 msgr2=0x7f969410fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:07:58.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:58.112+0000 7f969bc00700 1 --2- 172.21.15.37:0/1634013708 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f96940ff000 0x7f969410fd50 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f9688002820 tx=0x7f9688004570 comp rx=0 tx=0).stop 2024-03-29T08:07:58.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:58.112+0000 7f969bc00700 1 -- 172.21.15.37:0/1634013708 shutdown_connections 2024-03-29T08:07:58.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:58.112+0000 7f969bc00700 1 --2- 172.21.15.37:0/1634013708 >> [v2:172.21.15.148:6828/1134897184,v1:172.21.15.148:6829/1134897184] conn(0x7f967c06adb0 0x7f967c06d260 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:07:58.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:58.112+0000 7f969bc00700 1 --2- 172.21.15.37:0/1634013708 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f96940ff000 0x7f969410fd50 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:07:58.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:58.112+0000 7f969bc00700 1 --2- 172.21.15.37:0/1634013708 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9694110290 0x7f9694110fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:07:58.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:58.112+0000 7f969bc00700 1 -- 172.21.15.37:0/1634013708 >> 172.21.15.37:0/1634013708 conn(0x7f96940fa950 msgr2=0x7f96940fd210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:07:58.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:58.112+0000 7f969bc00700 1 -- 172.21.15.37:0/1634013708 shutdown_connections 2024-03-29T08:07:58.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:58.112+0000 7f969bc00700 1 -- 172.21.15.37:0/1634013708 wait complete. 2024-03-29T08:07:58.134 INFO:teuthology.orchestra.run.smithi037.stdout:true 2024-03-29T08:07:58.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:58.307+0000 7f19f3400700 1 -- 172.21.15.37:0/595882029 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f19e8013aa0 con 0x7f19f40662e0 2024-03-29T08:07:58.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:58.308+0000 7f19fbc00700 1 -- 172.21.15.37:0/595882029 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f19f40662e0 msgr2=0x7f19f40fd7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:07:58.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:58.308+0000 7f19fbc00700 1 --2- 172.21.15.37:0/595882029 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f19f40662e0 0x7f19f40fd7b0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f19e8004880 tx=0x7f19e8009e30 comp rx=0 tx=0).stop 2024-03-29T08:07:58.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:58.308+0000 7f19fbc00700 1 -- 172.21.15.37:0/595882029 shutdown_connections 2024-03-29T08:07:58.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:58.308+0000 7f19fbc00700 1 --2- 172.21.15.37:0/595882029 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f19f40fdcf0 0x7f19f4100170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:07:58.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:58.308+0000 7f19fbc00700 1 --2- 172.21.15.37:0/595882029 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f19f40662e0 0x7f19f40fd7b0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:07:58.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:58.308+0000 7f19fbc00700 1 -- 172.21.15.37:0/595882029 >> 172.21.15.37:0/595882029 conn(0x7f19f40f7430 msgr2=0x7f19f40f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:07:58.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:58.308+0000 7f19fbc00700 1 -- 172.21.15.37:0/595882029 shutdown_connections 2024-03-29T08:07:58.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:58.308+0000 7f19fbc00700 1 -- 172.21.15.37:0/595882029 wait complete. 2024-03-29T08:07:58.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:58.309+0000 7f19fbc00700 1 Processor -- start 2024-03-29T08:07:58.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:58.309+0000 7f19fbc00700 1 -- start start 2024-03-29T08:07:58.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:58.310+0000 7f19fbc00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f19f40662e0 0x7f19f4113f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:07:58.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:58.310+0000 7f19fbc00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f19f40fdcf0 0x7f19f4114460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:07:58.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:58.310+0000 7f19fbc00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19f4114a60 con 0x7f19f40fdcf0 2024-03-29T08:07:58.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:58.310+0000 7f19fbc00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19f41151a0 con 0x7f19f40662e0 2024-03-29T08:07:58.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:58.311+0000 7f19f9200700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f19f40662e0 0x7f19f4113f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:07:58.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:58.311+0000 7f19f9200700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f19f40662e0 0x7f19f4113f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.37:57036/0 (socket says 172.21.15.37:57036) 2024-03-29T08:07:58.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:58.311+0000 7f19f9200700 1 -- 172.21.15.37:0/1626009789 learned_addr learned my addr 172.21.15.37:0/1626009789 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:07:58.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:58.311+0000 7f19f3e00700 1 --2- 172.21.15.37:0/1626009789 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f19f40fdcf0 0x7f19f4114460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:07:58.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:58.311+0000 7f19f9200700 1 -- 172.21.15.37:0/1626009789 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f19f40fdcf0 msgr2=0x7f19f4114460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:07:58.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:58.312+0000 7f19f9200700 1 --2- 172.21.15.37:0/1626009789 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f19f40fdcf0 0x7f19f4114460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:07:58.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:58.312+0000 7f19f9200700 1 -- 172.21.15.37:0/1626009789 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19e8005040 con 0x7f19f40662e0 2024-03-29T08:07:58.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:58.312+0000 7f19f3e00700 1 --2- 172.21.15.37:0/1626009789 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f19f40fdcf0 0x7f19f4114460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:07:58.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:58.312+0000 7f19f9200700 1 --2- 172.21.15.37:0/1626009789 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f19f40662e0 0x7f19f4113f20 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f19e8014ef0 tx=0x7f19e8009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:07:58.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:58.312+0000 7f19f0c00700 1 -- 172.21.15.37:0/1626009789 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f19e8020070 con 0x7f19f40662e0 2024-03-29T08:07:58.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:58.312+0000 7f19f0c00700 1 -- 172.21.15.37:0/1626009789 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f19e8009370 con 0x7f19f40662e0 2024-03-29T08:07:58.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:58.312+0000 7f19f0c00700 1 -- 172.21.15.37:0/1626009789 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f19e800e040 con 0x7f19f40662e0 2024-03-29T08:07:58.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:58.313+0000 7f19fbc00700 1 -- 172.21.15.37:0/1626009789 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19f4115420 con 0x7f19f40662e0 2024-03-29T08:07:58.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:58.313+0000 7f19fbc00700 1 -- 172.21.15.37:0/1626009789 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19f4115870 con 0x7f19f40662e0 2024-03-29T08:07:58.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:58.315+0000 7f19fbc00700 1 -- 172.21.15.37:0/1626009789 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19f405f5f0 con 0x7f19f40662e0 2024-03-29T08:07:58.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:58.316+0000 7f19f0c00700 1 -- 172.21.15.37:0/1626009789 <== mon.1 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 26) v1 ==== 88664+0+0 (secure 0 0 0) 0x7f19e801c430 con 0x7f19f40662e0 2024-03-29T08:07:58.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:58.317+0000 7f19f0c00700 1 --2- 172.21.15.37:0/1626009789 >> [v2:172.21.15.148:6828/1134897184,v1:172.21.15.148:6829/1134897184] conn(0x7f19dc06adf0 0x7f19dc06d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:07:58.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:58.317+0000 7f19f0c00700 1 -- 172.21.15.37:0/1626009789 <== mon.1 v2:172.21.15.148:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f19e808a850 con 0x7f19f40662e0 2024-03-29T08:07:58.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:58.317+0000 7f19f3e00700 1 --2- 172.21.15.37:0/1626009789 >> [v2:172.21.15.148:6828/1134897184,v1:172.21.15.148:6829/1134897184] conn(0x7f19dc06adf0 0x7f19dc06d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:07:58.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:58.318+0000 7f19f3e00700 1 --2- 172.21.15.37:0/1626009789 >> [v2:172.21.15.148:6828/1134897184,v1:172.21.15.148:6829/1134897184] conn(0x7f19dc06adf0 0x7f19dc06d2a0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f19e00043d0 tx=0x7f19e000b040 comp rx=0 tx=0).ready entity=mgr.14602 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:07:58.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:58.322+0000 7f19f0c00700 1 -- 172.21.15.37:0/1626009789 <== mon.1 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181579 (secure 0 0 0) 0x7f19e80588d0 con 0x7f19f40662e0 2024-03-29T08:07:58.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:58.721+0000 7f19fbc00700 1 -- 172.21.15.37:0/1626009789 --> [v2:172.21.15.148:6828/1134897184,v1:172.21.15.148:6829/1134897184] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f19f4115cb0 con 0x7f19dc06adf0 2024-03-29T08:07:58.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:58.722+0000 7f19f0c00700 1 -- 172.21.15.37:0/1626009789 <== mgr.14602 v2:172.21.15.148:6828/1134897184 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+360 (secure 0 0 0) 0x7f19f4115cb0 con 0x7f19dc06adf0 2024-03-29T08:07:58.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:58.725+0000 7f19fbc00700 1 -- 172.21.15.37:0/1626009789 >> [v2:172.21.15.148:6828/1134897184,v1:172.21.15.148:6829/1134897184] conn(0x7f19dc06adf0 msgr2=0x7f19dc06d2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:07:58.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:58.725+0000 7f19fbc00700 1 --2- 172.21.15.37:0/1626009789 >> [v2:172.21.15.148:6828/1134897184,v1:172.21.15.148:6829/1134897184] conn(0x7f19dc06adf0 0x7f19dc06d2a0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f19e00043d0 tx=0x7f19e000b040 comp rx=0 tx=0).stop 2024-03-29T08:07:58.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:58.726+0000 7f19fbc00700 1 -- 172.21.15.37:0/1626009789 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f19f40662e0 msgr2=0x7f19f4113f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:07:58.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:58.726+0000 7f19fbc00700 1 --2- 172.21.15.37:0/1626009789 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f19f40662e0 0x7f19f4113f20 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f19e8014ef0 tx=0x7f19e8009e30 comp rx=0 tx=0).stop 2024-03-29T08:07:58.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:58.726+0000 7f19fbc00700 1 -- 172.21.15.37:0/1626009789 shutdown_connections 2024-03-29T08:07:58.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:58.726+0000 7f19fbc00700 1 --2- 172.21.15.37:0/1626009789 >> [v2:172.21.15.148:6828/1134897184,v1:172.21.15.148:6829/1134897184] conn(0x7f19dc06adf0 0x7f19dc06d2a0 secure :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f19e00043d0 tx=0x7f19e000b040 comp rx=0 tx=0).stop 2024-03-29T08:07:58.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:58.726+0000 7f19fbc00700 1 --2- 172.21.15.37:0/1626009789 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f19f40662e0 0x7f19f4113f20 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:07:58.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:58.726+0000 7f19fbc00700 1 --2- 172.21.15.37:0/1626009789 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f19f40fdcf0 0x7f19f4114460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:07:58.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:58.726+0000 7f19fbc00700 1 -- 172.21.15.37:0/1626009789 >> 172.21.15.37:0/1626009789 conn(0x7f19f40f7430 msgr2=0x7f19f40f7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:07:58.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:58.726+0000 7f19fbc00700 1 -- 172.21.15.37:0/1626009789 shutdown_connections 2024-03-29T08:07:58.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:58.726+0000 7f19fbc00700 1 -- 172.21.15.37:0/1626009789 wait complete. 2024-03-29T08:07:58.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:58.958+0000 7f1c86c00700 1 -- 172.21.15.37:0/638921341 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1c800662e0 msgr2=0x7f1c800ff9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:07:58.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:58.958+0000 7f1c86c00700 1 --2- 172.21.15.37:0/638921341 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1c800662e0 0x7f1c800ff9c0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f1c70004940 tx=0x7f1c70009e30 comp rx=0 tx=0).stop 2024-03-29T08:07:58.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:58.958+0000 7f1c86c00700 1 -- 172.21.15.37:0/638921341 shutdown_connections 2024-03-29T08:07:58.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:58.958+0000 7f1c86c00700 1 --2- 172.21.15.37:0/638921341 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1c800fff00 0x7f1c80102320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:07:58.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:58.958+0000 7f1c86c00700 1 --2- 172.21.15.37:0/638921341 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1c800662e0 0x7f1c800ff9c0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:07:58.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:58.958+0000 7f1c86c00700 1 -- 172.21.15.37:0/638921341 >> 172.21.15.37:0/638921341 conn(0x7f1c800f7430 msgr2=0x7f1c800f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:07:58.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:58.959+0000 7f1c86c00700 1 -- 172.21.15.37:0/638921341 shutdown_connections 2024-03-29T08:07:58.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:58.959+0000 7f1c86c00700 1 -- 172.21.15.37:0/638921341 wait complete. 2024-03-29T08:07:58.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:58.961+0000 7f1c86c00700 1 Processor -- start 2024-03-29T08:07:58.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:58.961+0000 7f1c86c00700 1 -- start start 2024-03-29T08:07:58.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:58.961+0000 7f1c86c00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1c800fff00 0x7f1c8010b830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:07:58.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:58.961+0000 7f1c86c00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1c8010bd70 0x7f1c8010cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:07:58.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:58.962+0000 7f1c86c00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c8010c230 con 0x7f1c8010bd70 2024-03-29T08:07:58.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:58.962+0000 7f1c86c00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c8010c3a0 con 0x7f1c800fff00 2024-03-29T08:07:58.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:58.962+0000 7f1c7fe00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1c800fff00 0x7f1c8010b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:07:58.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:58.962+0000 7f1c7fe00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1c800fff00 0x7f1c8010b830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.37:57054/0 (socket says 172.21.15.37:57054) 2024-03-29T08:07:58.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:58.962+0000 7f1c7fe00700 1 -- 172.21.15.37:0/311628187 learned_addr learned my addr 172.21.15.37:0/311628187 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:07:58.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:58.963+0000 7f1c7fe00700 1 -- 172.21.15.37:0/311628187 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1c8010bd70 msgr2=0x7f1c8010cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:07:58.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:58.963+0000 7f1c7fe00700 1 --2- 172.21.15.37:0/311628187 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1c8010bd70 0x7f1c8010cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:07:58.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:58.963+0000 7f1c7fe00700 1 -- 172.21.15.37:0/311628187 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c70005040 con 0x7f1c800fff00 2024-03-29T08:07:58.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:58.963+0000 7f1c7fe00700 1 --2- 172.21.15.37:0/311628187 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1c800fff00 0x7f1c8010b830 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f1c70000c00 tx=0x7f1c70009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:07:58.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:58.964+0000 7f1c77e00700 1 -- 172.21.15.37:0/311628187 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1c70020070 con 0x7f1c800fff00 2024-03-29T08:07:58.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:58.964+0000 7f1c77e00700 1 -- 172.21.15.37:0/311628187 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f1c7001c400 con 0x7f1c800fff00 2024-03-29T08:07:58.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:58.964+0000 7f1c77e00700 1 -- 172.21.15.37:0/311628187 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1c7000e070 con 0x7f1c800fff00 2024-03-29T08:07:58.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:58.964+0000 7f1c86c00700 1 -- 172.21.15.37:0/311628187 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c8010d210 con 0x7f1c800fff00 2024-03-29T08:07:58.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:58.964+0000 7f1c86c00700 1 -- 172.21.15.37:0/311628187 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c8010d6a0 con 0x7f1c800fff00 2024-03-29T08:07:58.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:58.966+0000 7f1c86c00700 1 -- 172.21.15.37:0/311628187 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c8005f5f0 con 0x7f1c800fff00 2024-03-29T08:07:58.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:58.968+0000 7f1c77e00700 1 -- 172.21.15.37:0/311628187 <== mon.1 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 26) v1 ==== 88664+0+0 (secure 0 0 0) 0x7f1c70041d50 con 0x7f1c800fff00 2024-03-29T08:07:58.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:58.969+0000 7f1c77e00700 1 --2- 172.21.15.37:0/311628187 >> [v2:172.21.15.148:6828/1134897184,v1:172.21.15.148:6829/1134897184] conn(0x7f1c6406ae80 0x7f1c6406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:07:58.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:58.969+0000 7f1c77e00700 1 -- 172.21.15.37:0/311628187 <== mon.1 v2:172.21.15.148:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f1c7008aa30 con 0x7f1c800fff00 2024-03-29T08:07:58.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:58.970+0000 7f1c7f400700 1 --2- 172.21.15.37:0/311628187 >> [v2:172.21.15.148:6828/1134897184,v1:172.21.15.148:6829/1134897184] conn(0x7f1c6406ae80 0x7f1c6406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:07:58.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:58.971+0000 7f1c7f400700 1 --2- 172.21.15.37:0/311628187 >> [v2:172.21.15.148:6828/1134897184,v1:172.21.15.148:6829/1134897184] conn(0x7f1c6406ae80 0x7f1c6406d330 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f1c8010c850 tx=0x7f1c6c00b040 comp rx=0 tx=0).ready entity=mgr.14602 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:07:58.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:58.971+0000 7f1c77e00700 1 -- 172.21.15.37:0/311628187 <== mon.1 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181579 (secure 0 0 0) 0x7f1c700550a0 con 0x7f1c800fff00 2024-03-29T08:07:59.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:07:59 smithi148 ceph-mon[44992]: pgmap v61: 65 pgs: 65 active+clean; 1.9 GiB data, 7.3 GiB used, 529 GiB / 536 GiB avail; 15 MiB/s rd, 34 MiB/s wr, 220 op/s 2024-03-29T08:07:59.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:07:59 smithi148 ceph-mon[44992]: from='mgr.14602 ' entity='mgr.smithi148.onhgvm' 2024-03-29T08:07:59.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:07:59 smithi148 ceph-mon[44992]: from='mgr.14602 ' entity='mgr.smithi148.onhgvm' 2024-03-29T08:07:59.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:59.338+0000 7f1c86c00700 1 -- 172.21.15.37:0/311628187 --> [v2:172.21.15.148:6828/1134897184,v1:172.21.15.148:6829/1134897184] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f1c800f8f70 con 0x7f1c6406ae80 2024-03-29T08:07:59.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:59.349+0000 7f1c77e00700 1 -- 172.21.15.37:0/311628187 <== mgr.14602 v2:172.21.15.148:6828/1134897184 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f1c800f8f70 con 0x7f1c6406ae80 2024-03-29T08:07:59.349 INFO:teuthology.orchestra.run.smithi037.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-29T08:07:59.349 INFO:teuthology.orchestra.run.smithi037.stdout:alertmanager.smithi037 smithi037 *:9093,9094 running (85s) 1s ago 10m 14.8M - 0.25.0 c8568f914cd2 8a8d40a6e26e 2024-03-29T08:07:59.349 INFO:teuthology.orchestra.run.smithi037.stdout:crash.smithi037 smithi037 running (11m) 1s ago 11m 6656k - 17.2.7-789-g4213e17c cf22eb8683be e0343935b25e 2024-03-29T08:07:59.349 INFO:teuthology.orchestra.run.smithi037.stdout:crash.smithi148 smithi148 running (9m) 5s ago 9m 6656k - 17.2.7-789-g4213e17c cf22eb8683be ff5164c552c1 2024-03-29T08:07:59.349 INFO:teuthology.orchestra.run.smithi037.stdout:grafana.smithi037 smithi037 *:3000 running (65s) 1s ago 10m 74.0M - 9.4.7 954c08fa6188 a857385eee0c 2024-03-29T08:07:59.349 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi037.iifsmh smithi037 running (4m) 1s ago 4m 138M - 17.2.7-789-g4213e17c cf22eb8683be 074e1acf331f 2024-03-29T08:07:59.349 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi037.mmwgxe smithi037 running (4m) 1s ago 4m 146M - 17.2.7-789-g4213e17c cf22eb8683be 9a5b1e72de5c 2024-03-29T08:07:59.349 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi148.qukpjh smithi148 running (4m) 5s ago 4m 12.2M - 17.2.7-789-g4213e17c cf22eb8683be 8e469782a273 2024-03-29T08:07:59.350 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi148.sqteih smithi148 running (4m) 5s ago 4m 12.2M - 17.2.7-789-g4213e17c cf22eb8683be 8e68f858bb22 2024-03-29T08:07:59.350 INFO:teuthology.orchestra.run.smithi037.stdout:mgr.smithi037.wjikii smithi037 *:8443,9283,8765 running (10s) 1s ago 12m 78.0M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff c5a1417b9417 2024-03-29T08:07:59.350 INFO:teuthology.orchestra.run.smithi037.stdout:mgr.smithi148.onhgvm smithi148 *:8443,9283 running (2m) 5s ago 9m 497M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 0c02ffa63851 2024-03-29T08:07:59.350 INFO:teuthology.orchestra.run.smithi037.stdout:mon.smithi037 smithi037 running (12m) 1s ago 12m 49.8M 2048M 17.2.7-789-g4213e17c cf22eb8683be bb2101505730 2024-03-29T08:07:59.350 INFO:teuthology.orchestra.run.smithi037.stdout:mon.smithi148 smithi148 running (9m) 5s ago 9m 40.0M 2048M 17.2.7-789-g4213e17c cf22eb8683be 18839ed2952a 2024-03-29T08:07:59.350 INFO:teuthology.orchestra.run.smithi037.stdout:node-exporter.smithi037 smithi037 *:9100 running (92s) 1s ago 10m 7583k - 1.5.0 0da6a335fe13 cdaa34126e04 2024-03-29T08:07:59.350 INFO:teuthology.orchestra.run.smithi037.stdout:node-exporter.smithi148 smithi148 *:9100 running (38s) 5s ago 9m 7541k - 1.5.0 0da6a335fe13 0d793b5bd838 2024-03-29T08:07:59.350 INFO:teuthology.orchestra.run.smithi037.stdout:osd.0 smithi037 running (8m) 1s ago 8m 189M 1619M 17.2.7-789-g4213e17c cf22eb8683be 559fccd0ebe1 2024-03-29T08:07:59.351 INFO:teuthology.orchestra.run.smithi037.stdout:osd.1 smithi037 running (8m) 1s ago 8m 207M 1619M 17.2.7-789-g4213e17c cf22eb8683be 5582cff11b7a 2024-03-29T08:07:59.351 INFO:teuthology.orchestra.run.smithi037.stdout:osd.2 smithi037 running (7m) 1s ago 7m 178M 1619M 17.2.7-789-g4213e17c cf22eb8683be f809bfab13e5 2024-03-29T08:07:59.351 INFO:teuthology.orchestra.run.smithi037.stdout:osd.3 smithi148 running (7m) 5s ago 7m 241M 2642M 17.2.7-789-g4213e17c cf22eb8683be a0a8224bb4b0 2024-03-29T08:07:59.351 INFO:teuthology.orchestra.run.smithi037.stdout:osd.4 smithi148 running (6m) 5s ago 6m 206M 2642M 17.2.7-789-g4213e17c cf22eb8683be a00b69047ed7 2024-03-29T08:07:59.351 INFO:teuthology.orchestra.run.smithi037.stdout:osd.5 smithi148 running (6m) 5s ago 6m 231M 2642M 17.2.7-789-g4213e17c cf22eb8683be 044a3f7db7f8 2024-03-29T08:07:59.351 INFO:teuthology.orchestra.run.smithi037.stdout:prometheus.smithi037 smithi037 *:9095 running (47s) 1s ago 10m 41.5M - 2.43.0 a07b618ecd1d 51e61f03c4ea 2024-03-29T08:07:59.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:59.352+0000 7f1c86c00700 1 -- 172.21.15.37:0/311628187 >> [v2:172.21.15.148:6828/1134897184,v1:172.21.15.148:6829/1134897184] conn(0x7f1c6406ae80 msgr2=0x7f1c6406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:07:59.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:59.352+0000 7f1c86c00700 1 --2- 172.21.15.37:0/311628187 >> [v2:172.21.15.148:6828/1134897184,v1:172.21.15.148:6829/1134897184] conn(0x7f1c6406ae80 0x7f1c6406d330 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f1c8010c850 tx=0x7f1c6c00b040 comp rx=0 tx=0).stop 2024-03-29T08:07:59.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:59.352+0000 7f1c86c00700 1 -- 172.21.15.37:0/311628187 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1c800fff00 msgr2=0x7f1c8010b830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:07:59.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:59.352+0000 7f1c86c00700 1 --2- 172.21.15.37:0/311628187 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1c800fff00 0x7f1c8010b830 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f1c70000c00 tx=0x7f1c70009e30 comp rx=0 tx=0).stop 2024-03-29T08:07:59.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:59.353+0000 7f1c86c00700 1 -- 172.21.15.37:0/311628187 shutdown_connections 2024-03-29T08:07:59.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:59.353+0000 7f1c86c00700 1 --2- 172.21.15.37:0/311628187 >> [v2:172.21.15.148:6828/1134897184,v1:172.21.15.148:6829/1134897184] conn(0x7f1c6406ae80 0x7f1c6406d330 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:07:59.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:59.353+0000 7f1c86c00700 1 --2- 172.21.15.37:0/311628187 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1c800fff00 0x7f1c8010b830 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:07:59.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:59.353+0000 7f1c86c00700 1 --2- 172.21.15.37:0/311628187 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1c8010bd70 0x7f1c8010cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:07:59.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:59.353+0000 7f1c86c00700 1 -- 172.21.15.37:0/311628187 >> 172.21.15.37:0/311628187 conn(0x7f1c800f7430 msgr2=0x7f1c800f7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:07:59.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:59.353+0000 7f1c86c00700 1 -- 172.21.15.37:0/311628187 shutdown_connections 2024-03-29T08:07:59.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:59.354+0000 7f1c86c00700 1 -- 172.21.15.37:0/311628187 wait complete. 2024-03-29T08:07:59.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:07:59 smithi037 ceph-mon[35162]: pgmap v61: 65 pgs: 65 active+clean; 1.9 GiB data, 7.3 GiB used, 529 GiB / 536 GiB avail; 15 MiB/s rd, 34 MiB/s wr, 220 op/s 2024-03-29T08:07:59.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:07:59 smithi037 ceph-mon[35162]: from='mgr.14602 ' entity='mgr.smithi148.onhgvm' 2024-03-29T08:07:59.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:07:59 smithi037 ceph-mon[35162]: from='mgr.14602 ' entity='mgr.smithi148.onhgvm' 2024-03-29T08:07:59.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:59.554+0000 7f7690200700 1 -- 172.21.15.37:0/461275638 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f76880fcf80 msgr2=0x7f76880fd3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:07:59.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:59.554+0000 7f7690200700 1 --2- 172.21.15.37:0/461275638 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f76880fcf80 0x7f76880fd3f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f7674004940 tx=0x7f7674009e30 comp rx=0 tx=0).stop 2024-03-29T08:07:59.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:59.555+0000 7f7690200700 1 -- 172.21.15.37:0/461275638 shutdown_connections 2024-03-29T08:07:59.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:59.555+0000 7f7690200700 1 --2- 172.21.15.37:0/461275638 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f76880fcf80 0x7f76880fd3f0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:07:59.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:59.555+0000 7f7690200700 1 --2- 172.21.15.37:0/461275638 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f76880fbcd0 0x7f76880fc0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:07:59.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:59.555+0000 7f7690200700 1 -- 172.21.15.37:0/461275638 >> 172.21.15.37:0/461275638 conn(0x7f76880f7430 msgr2=0x7f76880f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:07:59.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:59.556+0000 7f7690200700 1 -- 172.21.15.37:0/461275638 shutdown_connections 2024-03-29T08:07:59.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:59.556+0000 7f7690200700 1 -- 172.21.15.37:0/461275638 wait complete. 2024-03-29T08:07:59.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:59.557+0000 7f7690200700 1 Processor -- start 2024-03-29T08:07:59.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:59.558+0000 7f7690200700 1 -- start start 2024-03-29T08:07:59.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:59.559+0000 7f7690200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f76880fbcd0 0x7f7688113f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:07:59.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:59.559+0000 7f7690200700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f76880fcf80 0x7f7688114490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:07:59.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:59.559+0000 7f7690200700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7688114b20 con 0x7f76880fbcd0 2024-03-29T08:07:59.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:59.559+0000 7f7690200700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7688115310 con 0x7f76880fcf80 2024-03-29T08:07:59.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:59.560+0000 7f768ce00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f76880fcf80 0x7f7688114490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:07:59.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:59.560+0000 7f768ce00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f76880fcf80 0x7f7688114490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.37:57072/0 (socket says 172.21.15.37:57072) 2024-03-29T08:07:59.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:59.560+0000 7f768ce00700 1 -- 172.21.15.37:0/32477379 learned_addr learned my addr 172.21.15.37:0/32477379 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:07:59.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:59.560+0000 7f768d800700 1 --2- 172.21.15.37:0/32477379 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f76880fbcd0 0x7f7688113f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:07:59.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:59.560+0000 7f768ce00700 1 -- 172.21.15.37:0/32477379 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f76880fbcd0 msgr2=0x7f7688113f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:07:59.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:59.561+0000 7f768ce00700 1 --2- 172.21.15.37:0/32477379 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f76880fbcd0 0x7f7688113f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:07:59.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:59.561+0000 7f768ce00700 1 -- 172.21.15.37:0/32477379 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7674005040 con 0x7f76880fcf80 2024-03-29T08:07:59.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:59.561+0000 7f768ce00700 1 --2- 172.21.15.37:0/32477379 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f76880fcf80 0x7f7688114490 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f767400b3b0 tx=0x7f7674009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:07:59.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:59.561+0000 7f7685600700 1 -- 172.21.15.37:0/32477379 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f767400e040 con 0x7f76880fcf80 2024-03-29T08:07:59.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:59.562+0000 7f7685600700 1 -- 172.21.15.37:0/32477379 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f767400bc50 con 0x7f76880fcf80 2024-03-29T08:07:59.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:59.562+0000 7f7690200700 1 -- 172.21.15.37:0/32477379 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7688115590 con 0x7f76880fcf80 2024-03-29T08:07:59.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:59.562+0000 7f7685600700 1 -- 172.21.15.37:0/32477379 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f767401c630 con 0x7f76880fcf80 2024-03-29T08:07:59.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:59.562+0000 7f7690200700 1 -- 172.21.15.37:0/32477379 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7688115b50 con 0x7f76880fcf80 2024-03-29T08:07:59.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:59.564+0000 7f7690200700 1 -- 172.21.15.37:0/32477379 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f768805f5f0 con 0x7f76880fcf80 2024-03-29T08:07:59.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:59.566+0000 7f7685600700 1 -- 172.21.15.37:0/32477379 <== mon.1 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 26) v1 ==== 88664+0+0 (secure 0 0 0) 0x7f7674026440 con 0x7f76880fcf80 2024-03-29T08:07:59.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:59.568+0000 7f7685600700 1 --2- 172.21.15.37:0/32477379 >> [v2:172.21.15.148:6828/1134897184,v1:172.21.15.148:6829/1134897184] conn(0x7f767006ae80 0x7f767006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:07:59.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:59.568+0000 7f7685600700 1 -- 172.21.15.37:0/32477379 <== mon.1 v2:172.21.15.148:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f767408a250 con 0x7f76880fcf80 2024-03-29T08:07:59.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:59.569+0000 7f768d800700 1 --2- 172.21.15.37:0/32477379 >> [v2:172.21.15.148:6828/1134897184,v1:172.21.15.148:6829/1134897184] conn(0x7f767006ae80 0x7f767006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:07:59.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:59.571+0000 7f768d800700 1 --2- 172.21.15.37:0/32477379 >> [v2:172.21.15.148:6828/1134897184,v1:172.21.15.148:6829/1134897184] conn(0x7f767006ae80 0x7f767006d330 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f767c006fd0 tx=0x7f767c008040 comp rx=0 tx=0).ready entity=mgr.14602 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:07:59.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:07:59.573+0000 7f7685600700 1 -- 172.21.15.37:0/32477379 <== mon.1 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181579 (secure 0 0 0) 0x7f7674059400 con 0x7f76880fcf80 2024-03-29T08:08:00.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:00.088+0000 7f7690200700 1 -- 172.21.15.37:0/32477379 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f7688115e30 con 0x7f76880fcf80 2024-03-29T08:08:00.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:00.089+0000 7f7685600700 1 -- 172.21.15.37:0/32477379 <== mon.1 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+850 (secure 0 0 0) 0x7f7674058f90 con 0x7f76880fcf80 2024-03-29T08:08:00.089 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-29T08:08:00.089 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": { 2024-03-29T08:08:00.089 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 17.2.7-789-g4213e17c (4213e17c4de67016ed14f55d22503f4923b3a901) quincy (stable)": 2 2024-03-29T08:08:00.089 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:08:00.089 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": { 2024-03-29T08:08:00.089 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 17.2.7-789-g4213e17c (4213e17c4de67016ed14f55d22503f4923b3a901) quincy (stable)": 1, 2024-03-29T08:08:00.089 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 1 2024-03-29T08:08:00.089 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:08:00.089 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": { 2024-03-29T08:08:00.089 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 17.2.7-789-g4213e17c (4213e17c4de67016ed14f55d22503f4923b3a901) quincy (stable)": 6 2024-03-29T08:08:00.089 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:08:00.089 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": { 2024-03-29T08:08:00.090 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 17.2.7-789-g4213e17c (4213e17c4de67016ed14f55d22503f4923b3a901) quincy (stable)": 4 2024-03-29T08:08:00.090 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:08:00.090 INFO:teuthology.orchestra.run.smithi037.stdout: "overall": { 2024-03-29T08:08:00.090 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 17.2.7-789-g4213e17c (4213e17c4de67016ed14f55d22503f4923b3a901) quincy (stable)": 13, 2024-03-29T08:08:00.090 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 1 2024-03-29T08:08:00.090 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-29T08:08:00.090 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-29T08:08:00.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:00.092+0000 7f7690200700 1 -- 172.21.15.37:0/32477379 >> [v2:172.21.15.148:6828/1134897184,v1:172.21.15.148:6829/1134897184] conn(0x7f767006ae80 msgr2=0x7f767006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:08:00.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:00.092+0000 7f7690200700 1 --2- 172.21.15.37:0/32477379 >> [v2:172.21.15.148:6828/1134897184,v1:172.21.15.148:6829/1134897184] conn(0x7f767006ae80 0x7f767006d330 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f767c006fd0 tx=0x7f767c008040 comp rx=0 tx=0).stop 2024-03-29T08:08:00.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:00.092+0000 7f7690200700 1 -- 172.21.15.37:0/32477379 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f76880fcf80 msgr2=0x7f7688114490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:08:00.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:00.092+0000 7f7690200700 1 --2- 172.21.15.37:0/32477379 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f76880fcf80 0x7f7688114490 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f767400b3b0 tx=0x7f7674009e30 comp rx=0 tx=0).stop 2024-03-29T08:08:00.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:00.092+0000 7f7690200700 1 -- 172.21.15.37:0/32477379 shutdown_connections 2024-03-29T08:08:00.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:00.092+0000 7f7690200700 1 --2- 172.21.15.37:0/32477379 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f76880fbcd0 0x7f7688113f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:08:00.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:00.092+0000 7f7690200700 1 --2- 172.21.15.37:0/32477379 >> [v2:172.21.15.148:6828/1134897184,v1:172.21.15.148:6829/1134897184] conn(0x7f767006ae80 0x7f767006d330 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:08:00.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:00.092+0000 7f7690200700 1 --2- 172.21.15.37:0/32477379 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f76880fcf80 0x7f7688114490 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:08:00.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:00.092+0000 7f7690200700 1 -- 172.21.15.37:0/32477379 >> 172.21.15.37:0/32477379 conn(0x7f76880f7430 msgr2=0x7f76880f8f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:08:00.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:00.092+0000 7f7690200700 1 -- 172.21.15.37:0/32477379 shutdown_connections 2024-03-29T08:08:00.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:00.092+0000 7f7690200700 1 -- 172.21.15.37:0/32477379 wait complete. 2024-03-29T08:08:00.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:08:00 smithi037 ceph-mon[35162]: from='client.24511 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:08:00.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:08:00 smithi037 ceph-mon[35162]: from='client.24515 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:08:00.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:08:00 smithi037 ceph-mon[35162]: from='mgr.14602 ' entity='mgr.smithi148.onhgvm' 2024-03-29T08:08:00.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:08:00 smithi037 ceph-mon[35162]: from='mgr.14602 ' entity='mgr.smithi148.onhgvm' 2024-03-29T08:08:00.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:00.290+0000 7f260d600700 1 -- 172.21.15.37:0/1305862116 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f26080fcc10 msgr2=0x7f26080fd020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:08:00.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:00.290+0000 7f260d600700 1 --2- 172.21.15.37:0/1305862116 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f26080fcc10 0x7f26080fd020 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f25f8004940 tx=0x7f25f8009e30 comp rx=0 tx=0).stop 2024-03-29T08:08:00.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:00.290+0000 7f260d600700 1 -- 172.21.15.37:0/1305862116 shutdown_connections 2024-03-29T08:08:00.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:00.290+0000 7f260d600700 1 --2- 172.21.15.37:0/1305862116 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f26080fdec0 0x7f26080fe330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:08:00.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:00.290+0000 7f260d600700 1 --2- 172.21.15.37:0/1305862116 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f26080fcc10 0x7f26080fd020 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:08:00.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:00.290+0000 7f260d600700 1 -- 172.21.15.37:0/1305862116 >> 172.21.15.37:0/1305862116 conn(0x7f26080f83d0 msgr2=0x7f26080fa7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:08:00.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:00.290+0000 7f260d600700 1 -- 172.21.15.37:0/1305862116 shutdown_connections 2024-03-29T08:08:00.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:00.290+0000 7f260d600700 1 -- 172.21.15.37:0/1305862116 wait complete. 2024-03-29T08:08:00.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:00.292+0000 7f260d600700 1 Processor -- start 2024-03-29T08:08:00.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:00.294+0000 7f260d600700 1 -- start start 2024-03-29T08:08:00.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:00.294+0000 7f260d600700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f26080fcc10 0x7f260810b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:08:00.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:00.294+0000 7f260d600700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f26080fdec0 0x7f260810ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:08:00.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:00.294+0000 7f260d600700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f260810c070 con 0x7f26080fdec0 2024-03-29T08:08:00.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:00.294+0000 7f260d600700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f260810c8c0 con 0x7f26080fcc10 2024-03-29T08:08:00.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:00.294+0000 7f2607400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f26080fdec0 0x7f260810ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:08:00.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:00.295+0000 7f2607e00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f26080fcc10 0x7f260810b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:08:00.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:00.295+0000 7f2607400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f26080fdec0 0x7f260810ba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55800/0 (socket says 172.21.15.37:55800) 2024-03-29T08:08:00.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:00.295+0000 7f2607400700 1 -- 172.21.15.37:0/3044341831 learned_addr learned my addr 172.21.15.37:0/3044341831 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:08:00.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:00.295+0000 7f2607e00700 1 -- 172.21.15.37:0/3044341831 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f26080fdec0 msgr2=0x7f260810ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:08:00.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:00.296+0000 7f2607e00700 1 --2- 172.21.15.37:0/3044341831 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f26080fdec0 0x7f260810ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:08:00.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:00.296+0000 7f2607e00700 1 -- 172.21.15.37:0/3044341831 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25f8005040 con 0x7f26080fcc10 2024-03-29T08:08:00.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:00.296+0000 7f2607400700 1 --2- 172.21.15.37:0/3044341831 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f26080fdec0 0x7f260810ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T08:08:00.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:00.296+0000 7f2607e00700 1 --2- 172.21.15.37:0/3044341831 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f26080fcc10 0x7f260810b4e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f25f8000c00 tx=0x7f25f8009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:08:00.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:00.297+0000 7f25ffe00700 1 -- 172.21.15.37:0/3044341831 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f25f8020070 con 0x7f26080fcc10 2024-03-29T08:08:00.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:00.297+0000 7f25ffe00700 1 -- 172.21.15.37:0/3044341831 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f25f801c400 con 0x7f26080fcc10 2024-03-29T08:08:00.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:00.297+0000 7f260d600700 1 -- 172.21.15.37:0/3044341831 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f260810cac0 con 0x7f26080fcc10 2024-03-29T08:08:00.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:00.297+0000 7f260d600700 1 -- 172.21.15.37:0/3044341831 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f260810cff0 con 0x7f26080fcc10 2024-03-29T08:08:00.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:00.297+0000 7f25ffe00700 1 -- 172.21.15.37:0/3044341831 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f25f800e070 con 0x7f26080fcc10 2024-03-29T08:08:00.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:00.299+0000 7f260d600700 1 -- 172.21.15.37:0/3044341831 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25e80052b0 con 0x7f26080fcc10 2024-03-29T08:08:00.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:00.301+0000 7f25ffe00700 1 -- 172.21.15.37:0/3044341831 <== mon.1 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 26) v1 ==== 88664+0+0 (secure 0 0 0) 0x7f25f8041d50 con 0x7f26080fcc10 2024-03-29T08:08:00.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:00.302+0000 7f25ffe00700 1 --2- 172.21.15.37:0/3044341831 >> [v2:172.21.15.148:6828/1134897184,v1:172.21.15.148:6829/1134897184] conn(0x7f25ec06aec0 0x7f25ec06d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:08:00.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:00.303+0000 7f25ffe00700 1 -- 172.21.15.37:0/3044341831 <== mon.1 v2:172.21.15.148:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f25f808a850 con 0x7f26080fcc10 2024-03-29T08:08:00.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:00.303+0000 7f2607400700 1 --2- 172.21.15.37:0/3044341831 >> [v2:172.21.15.148:6828/1134897184,v1:172.21.15.148:6829/1134897184] conn(0x7f25ec06aec0 0x7f25ec06d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:08:00.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:00.304+0000 7f2607400700 1 --2- 172.21.15.37:0/3044341831 >> [v2:172.21.15.148:6828/1134897184,v1:172.21.15.148:6829/1134897184] conn(0x7f25ec06aec0 0x7f25ec06d370 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f25f40041d0 tx=0x7f25f400b040 comp rx=0 tx=0).ready entity=mgr.14602 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:08:00.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:00.305+0000 7f25ffe00700 1 -- 172.21.15.37:0/3044341831 <== mon.1 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181579 (secure 0 0 0) 0x7f25f8054ec0 con 0x7f26080fcc10 2024-03-29T08:08:00.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:08:00 smithi148 ceph-mon[44992]: from='client.24511 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:08:00.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:08:00 smithi148 ceph-mon[44992]: from='client.24515 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:08:00.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:08:00 smithi148 ceph-mon[44992]: from='mgr.14602 ' entity='mgr.smithi148.onhgvm' 2024-03-29T08:08:00.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:08:00 smithi148 ceph-mon[44992]: from='mgr.14602 ' entity='mgr.smithi148.onhgvm' 2024-03-29T08:08:00.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:00.643+0000 7f260d600700 1 -- 172.21.15.37:0/3044341831 --> [v2:172.21.15.148:6828/1134897184,v1:172.21.15.148:6829/1134897184] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f25e8000bc0 con 0x7f25ec06aec0 2024-03-29T08:08:00.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:00.645+0000 7f25ffe00700 1 -- 172.21.15.37:0/3044341831 <== mgr.14602 v2:172.21.15.148:6828/1134897184 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+360 (secure 0 0 0) 0x7f25e8000bc0 con 0x7f25ec06aec0 2024-03-29T08:08:00.645 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-29T08:08:00.645 INFO:teuthology.orchestra.run.smithi037.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59", 2024-03-29T08:08:00.645 INFO:teuthology.orchestra.run.smithi037.stdout: "in_progress": true, 2024-03-29T08:08:00.645 INFO:teuthology.orchestra.run.smithi037.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-03-29T08:08:00.645 INFO:teuthology.orchestra.run.smithi037.stdout: "services_complete": [ 2024-03-29T08:08:00.645 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr" 2024-03-29T08:08:00.645 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-29T08:08:00.645 INFO:teuthology.orchestra.run.smithi037.stdout: "progress": "2/2 daemons upgraded", 2024-03-29T08:08:00.645 INFO:teuthology.orchestra.run.smithi037.stdout: "message": "Currently upgrading mgr daemons", 2024-03-29T08:08:00.645 INFO:teuthology.orchestra.run.smithi037.stdout: "is_paused": false 2024-03-29T08:08:00.645 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-29T08:08:00.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:00.647+0000 7f260d600700 1 -- 172.21.15.37:0/3044341831 >> [v2:172.21.15.148:6828/1134897184,v1:172.21.15.148:6829/1134897184] conn(0x7f25ec06aec0 msgr2=0x7f25ec06d370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:08:00.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:00.647+0000 7f260d600700 1 --2- 172.21.15.37:0/3044341831 >> [v2:172.21.15.148:6828/1134897184,v1:172.21.15.148:6829/1134897184] conn(0x7f25ec06aec0 0x7f25ec06d370 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f25f40041d0 tx=0x7f25f400b040 comp rx=0 tx=0).stop 2024-03-29T08:08:00.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:00.647+0000 7f260d600700 1 -- 172.21.15.37:0/3044341831 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f26080fcc10 msgr2=0x7f260810b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:08:00.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:00.647+0000 7f260d600700 1 --2- 172.21.15.37:0/3044341831 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f26080fcc10 0x7f260810b4e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f25f8000c00 tx=0x7f25f8009e30 comp rx=0 tx=0).stop 2024-03-29T08:08:00.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:00.647+0000 7f260d600700 1 -- 172.21.15.37:0/3044341831 shutdown_connections 2024-03-29T08:08:00.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:00.647+0000 7f260d600700 1 --2- 172.21.15.37:0/3044341831 >> [v2:172.21.15.148:6828/1134897184,v1:172.21.15.148:6829/1134897184] conn(0x7f25ec06aec0 0x7f25ec06d370 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:08:00.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:00.647+0000 7f260d600700 1 --2- 172.21.15.37:0/3044341831 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f26080fcc10 0x7f260810b4e0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:08:00.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:00.647+0000 7f260d600700 1 --2- 172.21.15.37:0/3044341831 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f26080fdec0 0x7f260810ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:08:00.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:00.647+0000 7f260d600700 1 -- 172.21.15.37:0/3044341831 >> 172.21.15.37:0/3044341831 conn(0x7f26080f83d0 msgr2=0x7f26080f9a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:08:00.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:00.648+0000 7f260d600700 1 -- 172.21.15.37:0/3044341831 shutdown_connections 2024-03-29T08:08:00.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:00.648+0000 7f260d600700 1 -- 172.21.15.37:0/3044341831 wait complete. 2024-03-29T08:08:01.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:08:01 smithi037 ceph-mon[35162]: from='client.24519 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:08:01.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:08:01 smithi037 ceph-mon[35162]: pgmap v62: 65 pgs: 65 active+clean; 1.5 GiB data, 6.5 GiB used, 530 GiB / 536 GiB avail; 6.6 MiB/s rd, 12 MiB/s wr, 162 op/s 2024-03-29T08:08:01.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:08:01 smithi037 ceph-mon[35162]: from='client.? 172.21.15.37:0/32477379' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:08:01.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:08:01 smithi148 ceph-mon[44992]: from='client.24519 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:08:01.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:08:01 smithi148 ceph-mon[44992]: pgmap v62: 65 pgs: 65 active+clean; 1.5 GiB data, 6.5 GiB used, 530 GiB / 536 GiB avail; 6.6 MiB/s rd, 12 MiB/s wr, 162 op/s 2024-03-29T08:08:01.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:08:01 smithi148 ceph-mon[44992]: from='client.? 172.21.15.37:0/32477379' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:08:02.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:08:02 smithi037 ceph-mon[35162]: from='client.24527 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:08:02.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:08:02 smithi037 ceph-mon[35162]: from='mgr.14602 172.21.15.148:0/667342922' entity='mgr.smithi148.onhgvm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:08:02.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:08:02 smithi148 ceph-mon[44992]: from='client.24527 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:08:02.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:08:02 smithi148 ceph-mon[44992]: from='mgr.14602 172.21.15.148:0/667342922' entity='mgr.smithi148.onhgvm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:08:03.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:08:03 smithi148 ceph-mon[44992]: pgmap v63: 65 pgs: 65 active+clean; 1.5 GiB data, 6.5 GiB used, 530 GiB / 536 GiB avail; 20 KiB/s rd, 1013 KiB/s wr, 139 op/s 2024-03-29T08:08:03.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:08:03 smithi148 ceph-mon[44992]: Standby manager daemon smithi037.wjikii restarted 2024-03-29T08:08:03.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:08:03 smithi148 ceph-mon[44992]: Standby manager daemon smithi037.wjikii started 2024-03-29T08:08:03.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:08:03 smithi148 ceph-mon[44992]: from='mgr.? 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi037.wjikii/crt"}]: dispatch 2024-03-29T08:08:03.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:08:03 smithi148 ceph-mon[44992]: from='mgr.? 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-29T08:08:03.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:08:03 smithi148 ceph-mon[44992]: from='mgr.? 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi037.wjikii/key"}]: dispatch 2024-03-29T08:08:03.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:08:03 smithi148 ceph-mon[44992]: from='mgr.? 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-29T08:08:03.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:08:03 smithi037 ceph-mon[35162]: pgmap v63: 65 pgs: 65 active+clean; 1.5 GiB data, 6.5 GiB used, 530 GiB / 536 GiB avail; 20 KiB/s rd, 1013 KiB/s wr, 139 op/s 2024-03-29T08:08:03.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:08:03 smithi037 ceph-mon[35162]: Standby manager daemon smithi037.wjikii restarted 2024-03-29T08:08:03.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:08:03 smithi037 ceph-mon[35162]: Standby manager daemon smithi037.wjikii started 2024-03-29T08:08:03.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:08:03 smithi037 ceph-mon[35162]: from='mgr.? 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi037.wjikii/crt"}]: dispatch 2024-03-29T08:08:03.428 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:08:03 smithi037 ceph-mon[35162]: from='mgr.? 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-29T08:08:03.428 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:08:03 smithi037 ceph-mon[35162]: from='mgr.? 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi037.wjikii/key"}]: dispatch 2024-03-29T08:08:03.428 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:08:03 smithi037 ceph-mon[35162]: from='mgr.? 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-29T08:08:04.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:08:04 smithi037 ceph-mon[35162]: mgrmap e27: smithi148.onhgvm(active, since 2m), standbys: smithi037.wjikii 2024-03-29T08:08:04.479 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:08:04 smithi148 ceph-mon[44992]: mgrmap e27: smithi148.onhgvm(active, since 2m), standbys: smithi037.wjikii 2024-03-29T08:08:05.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:08:05 smithi037 ceph-mon[35162]: pgmap v64: 65 pgs: 65 active+clean; 1.1 GiB data, 5.3 GiB used, 531 GiB / 536 GiB avail; 29 KiB/s rd, 1.4 MiB/s wr, 212 op/s 2024-03-29T08:08:05.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:08:05 smithi148 ceph-mon[44992]: pgmap v64: 65 pgs: 65 active+clean; 1.1 GiB data, 5.3 GiB used, 531 GiB / 536 GiB avail; 29 KiB/s rd, 1.4 MiB/s wr, 212 op/s 2024-03-29T08:08:06.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:08:06 smithi037 ceph-mon[35162]: pgmap v65: 65 pgs: 65 active+clean; 1.1 GiB data, 5.3 GiB used, 531 GiB / 536 GiB avail; 19 KiB/s rd, 905 KiB/s wr, 140 op/s 2024-03-29T08:08:06.462 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:08:06 smithi148 ceph-mon[44992]: pgmap v65: 65 pgs: 65 active+clean; 1.1 GiB data, 5.3 GiB used, 531 GiB / 536 GiB avail; 19 KiB/s rd, 905 KiB/s wr, 140 op/s 2024-03-29T08:08:07.309 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:08:07 smithi148 ceph-mon[44992]: from='mgr.14602 ' entity='mgr.smithi148.onhgvm' 2024-03-29T08:08:07.309 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:08:07 smithi148 ceph-mon[44992]: from='mgr.14602 ' entity='mgr.smithi148.onhgvm' 2024-03-29T08:08:07.309 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:08:07 smithi148 ceph-mon[44992]: from='mgr.14602 172.21.15.148:0/667342922' entity='mgr.smithi148.onhgvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T08:08:07.309 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:08:07 smithi148 ceph-mon[44992]: from='mgr.14602 172.21.15.148:0/667342922' entity='mgr.smithi148.onhgvm' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T08:08:07.310 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:08:07 smithi148 ceph-mon[44992]: from='mgr.14602 ' entity='mgr.smithi148.onhgvm' 2024-03-29T08:08:07.310 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:08:07 smithi148 ceph-mon[44992]: from='mgr.14602 172.21.15.148:0/667342922' entity='mgr.smithi148.onhgvm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T08:08:07.310 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:08:07 smithi148 ceph-mon[44992]: from='mgr.14602 172.21.15.148:0/667342922' entity='mgr.smithi148.onhgvm' cmd=[{"prefix": "mgr fail", "who": "smithi148.onhgvm"}]: dispatch 2024-03-29T08:08:07.310 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:08:07 smithi148 ceph-mon[44992]: from='mgr.14602 ' entity='mgr.smithi148.onhgvm' cmd=[{"prefix": "mgr fail", "who": "smithi148.onhgvm"}]: dispatch 2024-03-29T08:08:07.310 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:08:07 smithi148 ceph-mon[44992]: osdmap e49: 6 total, 6 up, 6 in 2024-03-29T08:08:07.310 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:08:07 smithi148 ceph-mon[44992]: from='mgr.14602 ' entity='mgr.smithi148.onhgvm' cmd='[{"prefix": "mgr fail", "who": "smithi148.onhgvm"}]': finished 2024-03-29T08:08:07.310 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:08:07 smithi148 ceph-mon[44992]: mgrmap e28: smithi037.wjikii(active, starting, since 0.00840368s) 2024-03-29T08:08:07.310 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:08:07 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "mon metadata", "id": "smithi037"}]: dispatch 2024-03-29T08:08:07.310 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:08:07 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "mon metadata", "id": "smithi148"}]: dispatch 2024-03-29T08:08:07.310 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:08:07 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi037.mmwgxe"}]: dispatch 2024-03-29T08:08:07.310 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:08:07 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi037.iifsmh"}]: dispatch 2024-03-29T08:08:07.310 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:08:07 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi148.qukpjh"}]: dispatch 2024-03-29T08:08:07.310 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:08:07 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi148.sqteih"}]: dispatch 2024-03-29T08:08:07.311 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:08:07 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "mgr metadata", "who": "smithi037.wjikii", "id": "smithi037.wjikii"}]: dispatch 2024-03-29T08:08:07.311 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:08:07 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-29T08:08:07.311 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:08:07 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-29T08:08:07.311 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:08:07 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-29T08:08:07.311 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:08:07 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-29T08:08:07.311 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:08:07 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-29T08:08:07.311 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:08:07 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-29T08:08:07.311 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:08:07 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-29T08:08:07.311 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:08:07 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-29T08:08:07.311 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:08:07 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-29T08:08:07.311 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:08:07 smithi148 ceph-mon[44992]: Manager daemon smithi037.wjikii is now available 2024-03-29T08:08:07.311 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:08:07 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T08:08:07.311 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:08:07 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:08:07.311 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:08:07 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi037.wjikii/mirror_snapshot_schedule"}]: dispatch 2024-03-29T08:08:07.312 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:08:07 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi037.wjikii/trash_purge_schedule"}]: dispatch 2024-03-29T08:08:07.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:08:07 smithi037 ceph-mon[35162]: from='mgr.14602 ' entity='mgr.smithi148.onhgvm' 2024-03-29T08:08:07.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:08:07 smithi037 ceph-mon[35162]: from='mgr.14602 ' entity='mgr.smithi148.onhgvm' 2024-03-29T08:08:07.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:08:07 smithi037 ceph-mon[35162]: from='mgr.14602 172.21.15.148:0/667342922' entity='mgr.smithi148.onhgvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T08:08:07.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:08:07 smithi037 ceph-mon[35162]: from='mgr.14602 172.21.15.148:0/667342922' entity='mgr.smithi148.onhgvm' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T08:08:07.428 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:08:07 smithi037 ceph-mon[35162]: from='mgr.14602 ' entity='mgr.smithi148.onhgvm' 2024-03-29T08:08:07.428 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:08:07 smithi037 ceph-mon[35162]: from='mgr.14602 172.21.15.148:0/667342922' entity='mgr.smithi148.onhgvm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T08:08:07.428 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:08:07 smithi037 ceph-mon[35162]: from='mgr.14602 172.21.15.148:0/667342922' entity='mgr.smithi148.onhgvm' cmd=[{"prefix": "mgr fail", "who": "smithi148.onhgvm"}]: dispatch 2024-03-29T08:08:07.428 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:08:07 smithi037 ceph-mon[35162]: from='mgr.14602 ' entity='mgr.smithi148.onhgvm' cmd=[{"prefix": "mgr fail", "who": "smithi148.onhgvm"}]: dispatch 2024-03-29T08:08:07.428 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:08:07 smithi037 ceph-mon[35162]: osdmap e49: 6 total, 6 up, 6 in 2024-03-29T08:08:07.428 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:08:07 smithi037 ceph-mon[35162]: from='mgr.14602 ' entity='mgr.smithi148.onhgvm' cmd='[{"prefix": "mgr fail", "who": "smithi148.onhgvm"}]': finished 2024-03-29T08:08:07.428 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:08:07 smithi037 ceph-mon[35162]: mgrmap e28: smithi037.wjikii(active, starting, since 0.00840368s) 2024-03-29T08:08:07.428 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:08:07 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "mon metadata", "id": "smithi037"}]: dispatch 2024-03-29T08:08:07.428 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:08:07 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "mon metadata", "id": "smithi148"}]: dispatch 2024-03-29T08:08:07.428 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:08:07 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi037.mmwgxe"}]: dispatch 2024-03-29T08:08:07.428 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:08:07 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi037.iifsmh"}]: dispatch 2024-03-29T08:08:07.428 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:08:07 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi148.qukpjh"}]: dispatch 2024-03-29T08:08:07.428 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:08:07 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi148.sqteih"}]: dispatch 2024-03-29T08:08:07.428 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:08:07 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "mgr metadata", "who": "smithi037.wjikii", "id": "smithi037.wjikii"}]: dispatch 2024-03-29T08:08:07.429 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:08:07 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-29T08:08:07.429 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:08:07 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-29T08:08:07.429 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:08:07 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-29T08:08:07.429 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:08:07 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-29T08:08:07.429 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:08:07 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-29T08:08:07.429 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:08:07 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-29T08:08:07.429 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:08:07 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-29T08:08:07.429 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:08:07 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-29T08:08:07.429 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:08:07 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-29T08:08:07.429 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:08:07 smithi037 ceph-mon[35162]: Manager daemon smithi037.wjikii is now available 2024-03-29T08:08:07.429 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:08:07 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T08:08:07.429 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:08:07 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:08:07.429 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:08:07 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi037.wjikii/mirror_snapshot_schedule"}]: dispatch 2024-03-29T08:08:07.429 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:08:07 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi037.wjikii/trash_purge_schedule"}]: dispatch 2024-03-29T08:08:08.382 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:08:08 smithi148 ceph-mon[44992]: [29/Mar/2024:08:08:07] ENGINE Bus STARTING 2024-03-29T08:08:08.382 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:08:08 smithi148 ceph-mon[44992]: mgrmap e29: smithi037.wjikii(active, since 1.02022s) 2024-03-29T08:08:08.382 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:08:08 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' 2024-03-29T08:08:08.382 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:08:08 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' 2024-03-29T08:08:08.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:08:08 smithi037 ceph-mon[35162]: [29/Mar/2024:08:08:07] ENGINE Bus STARTING 2024-03-29T08:08:08.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:08:08 smithi037 ceph-mon[35162]: mgrmap e29: smithi037.wjikii(active, since 1.02022s) 2024-03-29T08:08:08.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:08:08 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' 2024-03-29T08:08:08.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:08:08 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' 2024-03-29T08:08:09.036 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-03-29T08:08:09.037 DEBUG:teuthology.orchestra.run.smithi037:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-03-29T08:08:09.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:08:09 smithi037 ceph-mon[35162]: [29/Mar/2024:08:08:07] ENGINE Serving on http://172.21.15.37:8765 2024-03-29T08:08:09.428 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:08:09 smithi037 ceph-mon[35162]: [29/Mar/2024:08:08:07] ENGINE Serving on https://172.21.15.37:7150 2024-03-29T08:08:09.428 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:08:09 smithi037 ceph-mon[35162]: [29/Mar/2024:08:08:07] ENGINE Bus STARTED 2024-03-29T08:08:09.428 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:08:09 smithi037 ceph-mon[35162]: pgmap v3: 65 pgs: 65 active+clean; 669 MiB data, 4.0 GiB used, 532 GiB / 536 GiB avail 2024-03-29T08:08:09.453 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:08:09 smithi148 ceph-mon[44992]: [29/Mar/2024:08:08:07] ENGINE Serving on http://172.21.15.37:8765 2024-03-29T08:08:09.453 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:08:09 smithi148 ceph-mon[44992]: [29/Mar/2024:08:08:07] ENGINE Serving on https://172.21.15.37:7150 2024-03-29T08:08:09.453 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:08:09 smithi148 ceph-mon[44992]: [29/Mar/2024:08:08:07] ENGINE Bus STARTED 2024-03-29T08:08:09.453 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:08:09 smithi148 ceph-mon[44992]: pgmap v3: 65 pgs: 65 active+clean; 669 MiB data, 4.0 GiB used, 532 GiB / 536 GiB avail 2024-03-29T08:08:10.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:08:10 smithi037 ceph-mon[35162]: pgmap v4: 65 pgs: 65 active+clean; 669 MiB data, 4.0 GiB used, 532 GiB / 536 GiB avail 2024-03-29T08:08:10.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:08:10 smithi037 ceph-mon[35162]: mgrmap e30: smithi037.wjikii(active, since 3s) 2024-03-29T08:08:10.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:08:10 smithi148 ceph-mon[44992]: pgmap v4: 65 pgs: 65 active+clean; 669 MiB data, 4.0 GiB used, 532 GiB / 536 GiB avail 2024-03-29T08:08:10.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:08:10 smithi148 ceph-mon[44992]: mgrmap e30: smithi037.wjikii(active, since 3s) 2024-03-29T08:08:12.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:08:12 smithi148 ceph-mon[44992]: pgmap v5: 65 pgs: 65 active+clean; 669 MiB data, 4.0 GiB used, 532 GiB / 536 GiB avail 2024-03-29T08:08:12.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:08:12 smithi148 ceph-mon[44992]: mgrmap e31: smithi037.wjikii(active, since 4s) 2024-03-29T08:08:12.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:08:12 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' 2024-03-29T08:08:12.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:08:12 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' 2024-03-29T08:08:12.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:08:12 smithi037 ceph-mon[35162]: pgmap v5: 65 pgs: 65 active+clean; 669 MiB data, 4.0 GiB used, 532 GiB / 536 GiB avail 2024-03-29T08:08:12.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:08:12 smithi037 ceph-mon[35162]: mgrmap e31: smithi037.wjikii(active, since 4s) 2024-03-29T08:08:12.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:08:12 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' 2024-03-29T08:08:12.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:08:12 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' 2024-03-29T08:08:14.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:08:14 smithi148 ceph-mon[44992]: pgmap v6: 65 pgs: 65 active+clean; 669 MiB data, 4.0 GiB used, 532 GiB / 536 GiB avail 2024-03-29T08:08:14.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:08:14 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' 2024-03-29T08:08:14.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:08:14 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' 2024-03-29T08:08:14.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:08:14 smithi037 ceph-mon[35162]: pgmap v6: 65 pgs: 65 active+clean; 669 MiB data, 4.0 GiB used, 532 GiB / 536 GiB avail 2024-03-29T08:08:14.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:08:14 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' 2024-03-29T08:08:14.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:08:14 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' 2024-03-29T08:08:16.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:08:16 smithi037 ceph-mon[35162]: pgmap v7: 65 pgs: 65 active+clean; 286 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 52 KiB/s rd, 1000 KiB/s wr, 235 op/s 2024-03-29T08:08:16.456 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:08:16 smithi148 ceph-mon[44992]: pgmap v7: 65 pgs: 65 active+clean; 286 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 52 KiB/s rd, 1000 KiB/s wr, 235 op/s 2024-03-29T08:08:18.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:08:18 smithi037 ceph-mon[35162]: pgmap v8: 65 pgs: 65 active+clean; 286 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 41 KiB/s rd, 777 KiB/s wr, 182 op/s 2024-03-29T08:08:18.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:08:18 smithi037 ceph-mon[35162]: Standby manager daemon smithi148.onhgvm started 2024-03-29T08:08:18.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:08:18 smithi037 ceph-mon[35162]: from='mgr.? 172.21.15.148:0/723774537' entity='mgr.smithi148.onhgvm' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi148.onhgvm/crt"}]: dispatch 2024-03-29T08:08:18.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:08:18 smithi037 ceph-mon[35162]: from='mgr.? 172.21.15.148:0/723774537' entity='mgr.smithi148.onhgvm' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-29T08:08:18.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:08:18 smithi037 ceph-mon[35162]: from='mgr.? 172.21.15.148:0/723774537' entity='mgr.smithi148.onhgvm' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi148.onhgvm/key"}]: dispatch 2024-03-29T08:08:18.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:08:18 smithi037 ceph-mon[35162]: from='mgr.? 172.21.15.148:0/723774537' entity='mgr.smithi148.onhgvm' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-29T08:08:18.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:08:18 smithi148 ceph-mon[44992]: pgmap v8: 65 pgs: 65 active+clean; 286 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 41 KiB/s rd, 777 KiB/s wr, 182 op/s 2024-03-29T08:08:18.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:08:18 smithi148 ceph-mon[44992]: Standby manager daemon smithi148.onhgvm started 2024-03-29T08:08:18.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:08:18 smithi148 ceph-mon[44992]: from='mgr.? 172.21.15.148:0/723774537' entity='mgr.smithi148.onhgvm' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi148.onhgvm/crt"}]: dispatch 2024-03-29T08:08:18.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:08:18 smithi148 ceph-mon[44992]: from='mgr.? 172.21.15.148:0/723774537' entity='mgr.smithi148.onhgvm' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-29T08:08:18.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:08:18 smithi148 ceph-mon[44992]: from='mgr.? 172.21.15.148:0/723774537' entity='mgr.smithi148.onhgvm' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi148.onhgvm/key"}]: dispatch 2024-03-29T08:08:18.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:08:18 smithi148 ceph-mon[44992]: from='mgr.? 172.21.15.148:0/723774537' entity='mgr.smithi148.onhgvm' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-29T08:08:19.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:08:19 smithi037 ceph-mon[35162]: mgrmap e32: smithi037.wjikii(active, since 11s), standbys: smithi148.onhgvm 2024-03-29T08:08:19.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:08:19 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "mgr metadata", "who": "smithi148.onhgvm", "id": "smithi148.onhgvm"}]: dispatch 2024-03-29T08:08:19.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:08:19 smithi148 ceph-mon[44992]: mgrmap e32: smithi037.wjikii(active, since 11s), standbys: smithi148.onhgvm 2024-03-29T08:08:19.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:08:19 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "mgr metadata", "who": "smithi148.onhgvm", "id": "smithi148.onhgvm"}]: dispatch 2024-03-29T08:08:20.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:08:20 smithi037 ceph-mon[35162]: pgmap v9: 65 pgs: 65 active+clean; 289 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 52 KiB/s rd, 1.3 MiB/s wr, 332 op/s 2024-03-29T08:08:20.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:08:20 smithi148 ceph-mon[44992]: pgmap v9: 65 pgs: 65 active+clean; 289 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 52 KiB/s rd, 1.3 MiB/s wr, 332 op/s 2024-03-29T08:08:22.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:08:22 smithi037 ceph-mon[35162]: pgmap v10: 65 pgs: 65 active+clean; 289 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 47 KiB/s rd, 1.2 MiB/s wr, 303 op/s 2024-03-29T08:08:22.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:08:22 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:08:22.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:08:22 smithi148 ceph-mon[44992]: pgmap v10: 65 pgs: 65 active+clean; 289 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 47 KiB/s rd, 1.2 MiB/s wr, 303 op/s 2024-03-29T08:08:22.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:08:22 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:08:24.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:08:24 smithi037 ceph-mon[35162]: pgmap v11: 65 pgs: 65 active+clean; 289 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 47 KiB/s rd, 1.2 MiB/s wr, 303 op/s 2024-03-29T08:08:24.482 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:08:24 smithi148 ceph-mon[44992]: pgmap v11: 65 pgs: 65 active+clean; 289 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 47 KiB/s rd, 1.2 MiB/s wr, 303 op/s 2024-03-29T08:08:26.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:08:26 smithi037 ceph-mon[35162]: pgmap v12: 65 pgs: 65 active+clean; 289 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 65 KiB/s rd, 1.8 MiB/s wr, 487 op/s 2024-03-29T08:08:26.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:08:26 smithi148 ceph-mon[44992]: pgmap v12: 65 pgs: 65 active+clean; 289 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 65 KiB/s rd, 1.8 MiB/s wr, 487 op/s 2024-03-29T08:08:28.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:08:28 smithi037 ceph-mon[35162]: pgmap v13: 65 pgs: 65 active+clean; 289 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 34 KiB/s rd, 1.2 MiB/s wr, 350 op/s 2024-03-29T08:08:28.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:08:28 smithi148 ceph-mon[44992]: pgmap v13: 65 pgs: 65 active+clean; 289 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 34 KiB/s rd, 1.2 MiB/s wr, 350 op/s 2024-03-29T08:08:30.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:08:30 smithi148 ceph-mon[44992]: pgmap v14: 65 pgs: 65 active+clean; 285 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 54 KiB/s rd, 2.0 MiB/s wr, 530 op/s 2024-03-29T08:08:30.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:08:30 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' 2024-03-29T08:08:30.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:08:30 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' 2024-03-29T08:08:30.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:08:30 smithi037 ceph-mon[35162]: pgmap v14: 65 pgs: 65 active+clean; 285 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 54 KiB/s rd, 2.0 MiB/s wr, 530 op/s 2024-03-29T08:08:30.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:08:30 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' 2024-03-29T08:08:30.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:08:30 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' 2024-03-29T08:08:30.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:30.865+0000 7ff41c400700 1 -- 172.21.15.37:0/2255049701 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff4140ff070 msgr2=0x7ff4140ff480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:08:30.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:30.865+0000 7ff41c400700 1 --2- 172.21.15.37:0/2255049701 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff4140ff070 0x7ff4140ff480 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7ff408004940 tx=0x7ff408009e30 comp rx=0 tx=0).stop 2024-03-29T08:08:30.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:30.866+0000 7ff41c400700 1 -- 172.21.15.37:0/2255049701 shutdown_connections 2024-03-29T08:08:30.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:30.866+0000 7ff41c400700 1 --2- 172.21.15.37:0/2255049701 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff414100320 0x7ff414100790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:08:30.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:30.866+0000 7ff41c400700 1 --2- 172.21.15.37:0/2255049701 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff4140ff070 0x7ff4140ff480 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:08:30.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:30.866+0000 7ff41c400700 1 -- 172.21.15.37:0/2255049701 >> 172.21.15.37:0/2255049701 conn(0x7ff4140fab10 msgr2=0x7ff4140fcef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:08:30.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:30.867+0000 7ff41c400700 1 -- 172.21.15.37:0/2255049701 shutdown_connections 2024-03-29T08:08:30.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:30.867+0000 7ff41c400700 1 -- 172.21.15.37:0/2255049701 wait complete. 2024-03-29T08:08:30.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:30.868+0000 7ff41c400700 1 Processor -- start 2024-03-29T08:08:30.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:30.869+0000 7ff41c400700 1 -- start start 2024-03-29T08:08:30.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:30.871+0000 7ff41c400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff4140ff070 0x7ff41410b660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:08:30.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:30.872+0000 7ff41c400700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff414100320 0x7ff41410bba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:08:30.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:30.872+0000 7ff41c400700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff41410c1f0 con 0x7ff4140ff070 2024-03-29T08:08:30.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:30.872+0000 7ff41c400700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff41410ca40 con 0x7ff414100320 2024-03-29T08:08:30.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:30.872+0000 7ff419a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff4140ff070 0x7ff41410b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:08:30.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:30.872+0000 7ff419a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff4140ff070 0x7ff41410b660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52254/0 (socket says 172.21.15.37:52254) 2024-03-29T08:08:30.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:30.872+0000 7ff419a00700 1 -- 172.21.15.37:0/3389287119 learned_addr learned my addr 172.21.15.37:0/3389287119 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:08:30.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:30.873+0000 7ff419000700 1 --2- 172.21.15.37:0/3389287119 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff414100320 0x7ff41410bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:08:30.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:30.873+0000 7ff419a00700 1 -- 172.21.15.37:0/3389287119 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff414100320 msgr2=0x7ff41410bba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:08:30.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:30.873+0000 7ff419a00700 1 --2- 172.21.15.37:0/3389287119 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff414100320 0x7ff41410bba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:08:30.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:30.873+0000 7ff419a00700 1 -- 172.21.15.37:0/3389287119 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff408005040 con 0x7ff4140ff070 2024-03-29T08:08:30.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:30.873+0000 7ff419000700 1 --2- 172.21.15.37:0/3389287119 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff414100320 0x7ff41410bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:08:30.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:30.873+0000 7ff419a00700 1 --2- 172.21.15.37:0/3389287119 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff4140ff070 0x7ff41410b660 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7ff4080043d0 tx=0x7ff408009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:08:30.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:30.873+0000 7ff411600700 1 -- 172.21.15.37:0/3389287119 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff408020070 con 0x7ff4140ff070 2024-03-29T08:08:30.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:30.873+0000 7ff41c400700 1 -- 172.21.15.37:0/3389287119 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff41410ccc0 con 0x7ff4140ff070 2024-03-29T08:08:30.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:30.874+0000 7ff41c400700 1 -- 172.21.15.37:0/3389287119 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff41410d220 con 0x7ff4140ff070 2024-03-29T08:08:30.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:30.874+0000 7ff411600700 1 -- 172.21.15.37:0/3389287119 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ff40801c400 con 0x7ff4140ff070 2024-03-29T08:08:30.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:30.874+0000 7ff411600700 1 -- 172.21.15.37:0/3389287119 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff40800e070 con 0x7ff4140ff070 2024-03-29T08:08:30.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:30.877+0000 7ff411600700 1 -- 172.21.15.37:0/3389287119 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 32) v1 ==== 90805+0+0 (secure 0 0 0) 0x7ff408012760 con 0x7ff4140ff070 2024-03-29T08:08:30.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:30.877+0000 7ff41c400700 1 -- 172.21.15.37:0/3389287119 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3f40052e0 con 0x7ff4140ff070 2024-03-29T08:08:30.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:30.880+0000 7ff411600700 1 --2- 172.21.15.37:0/3389287119 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7ff3fc06cab0 0x7ff3fc06ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:08:30.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:30.880+0000 7ff411600700 1 -- 172.21.15.37:0/3389287119 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6066+0+0 (secure 0 0 0) 0x7ff40808dfd0 con 0x7ff4140ff070 2024-03-29T08:08:30.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:30.881+0000 7ff419000700 1 --2- 172.21.15.37:0/3389287119 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7ff3fc06cab0 0x7ff3fc06ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:08:30.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:30.883+0000 7ff419000700 1 --2- 172.21.15.37:0/3389287119 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7ff3fc06cab0 0x7ff3fc06ef60 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7ff400004600 tx=0x7ff40000b040 comp rx=0 tx=0).ready entity=mgr.14686 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:08:30.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:30.886+0000 7ff411600700 1 -- 172.21.15.37:0/3389287119 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181579 (secure 0 0 0) 0x7ff408058270 con 0x7ff4140ff070 2024-03-29T08:08:31.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:31.286+0000 7ff41c400700 1 -- 172.21.15.37:0/3389287119 --> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff3f4000bf0 con 0x7ff3fc06cab0 2024-03-29T08:08:31.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:31.288+0000 7ff411600700 1 -- 172.21.15.37:0/3389287119 <== mgr.14686 v2:172.21.15.37:6800/4188329516 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+329 (secure 0 0 0) 0x7ff3f4000bf0 con 0x7ff3fc06cab0 2024-03-29T08:08:31.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:31.292+0000 7ff41c400700 1 -- 172.21.15.37:0/3389287119 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7ff3fc06cab0 msgr2=0x7ff3fc06ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:08:31.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:31.292+0000 7ff41c400700 1 --2- 172.21.15.37:0/3389287119 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7ff3fc06cab0 0x7ff3fc06ef60 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7ff400004600 tx=0x7ff40000b040 comp rx=0 tx=0).stop 2024-03-29T08:08:31.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:31.292+0000 7ff41c400700 1 -- 172.21.15.37:0/3389287119 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff4140ff070 msgr2=0x7ff41410b660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:08:31.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:31.292+0000 7ff41c400700 1 --2- 172.21.15.37:0/3389287119 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff4140ff070 0x7ff41410b660 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7ff4080043d0 tx=0x7ff408009e30 comp rx=0 tx=0).stop 2024-03-29T08:08:31.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:31.292+0000 7ff41c400700 1 -- 172.21.15.37:0/3389287119 shutdown_connections 2024-03-29T08:08:31.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:31.292+0000 7ff41c400700 1 --2- 172.21.15.37:0/3389287119 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7ff3fc06cab0 0x7ff3fc06ef60 secure :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7ff400004600 tx=0x7ff40000b040 comp rx=0 tx=0).stop 2024-03-29T08:08:31.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:31.292+0000 7ff41c400700 1 --2- 172.21.15.37:0/3389287119 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff4140ff070 0x7ff41410b660 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:08:31.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:31.292+0000 7ff41c400700 1 --2- 172.21.15.37:0/3389287119 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff414100320 0x7ff41410bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:08:31.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:31.292+0000 7ff41c400700 1 -- 172.21.15.37:0/3389287119 >> 172.21.15.37:0/3389287119 conn(0x7ff4140fab10 msgr2=0x7ff4140fd150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:08:31.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:31.294+0000 7ff41c400700 1 -- 172.21.15.37:0/3389287119 shutdown_connections 2024-03-29T08:08:31.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:31.294+0000 7ff41c400700 1 -- 172.21.15.37:0/3389287119 wait complete. 2024-03-29T08:08:31.313 INFO:teuthology.orchestra.run.smithi037.stdout:true 2024-03-29T08:08:31.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:31.492+0000 7f7864200700 1 -- 172.21.15.37:0/1603824181 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f785c06e9a0 msgr2=0x7f785c06ee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:08:31.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:31.492+0000 7f7864200700 1 --2- 172.21.15.37:0/1603824181 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f785c06e9a0 0x7f785c06ee10 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f7848004940 tx=0x7f7848009e30 comp rx=0 tx=0).stop 2024-03-29T08:08:31.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:31.493+0000 7f7864200700 1 -- 172.21.15.37:0/1603824181 shutdown_connections 2024-03-29T08:08:31.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:31.493+0000 7f7864200700 1 --2- 172.21.15.37:0/1603824181 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f785c06e9a0 0x7f785c06ee10 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:08:31.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:31.493+0000 7f7864200700 1 --2- 172.21.15.37:0/1603824181 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f785c06f9d0 0x7f785c06e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:08:31.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:31.493+0000 7f7864200700 1 -- 172.21.15.37:0/1603824181 >> 172.21.15.37:0/1603824181 conn(0x7f785c069de0 msgr2=0x7f785c06c210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:08:31.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:31.493+0000 7f7864200700 1 -- 172.21.15.37:0/1603824181 shutdown_connections 2024-03-29T08:08:31.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:31.493+0000 7f7864200700 1 -- 172.21.15.37:0/1603824181 wait complete. 2024-03-29T08:08:31.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:31.496+0000 7f7864200700 1 Processor -- start 2024-03-29T08:08:31.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:31.497+0000 7f7864200700 1 -- start start 2024-03-29T08:08:31.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:31.497+0000 7f7864200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f785c06e9a0 0x7f785c11c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:08:31.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:31.497+0000 7f7864200700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f785c06f9d0 0x7f785c11cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:08:31.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:31.497+0000 7f7864200700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f785c11d190 con 0x7f785c06e9a0 2024-03-29T08:08:31.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:31.497+0000 7f7864200700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f785c11d300 con 0x7f785c06f9d0 2024-03-29T08:08:31.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:31.498+0000 7f7860e00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f785c06f9d0 0x7f785c11cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:08:31.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:31.498+0000 7f7860e00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f785c06f9d0 0x7f785c11cb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.37:51284/0 (socket says 172.21.15.37:51284) 2024-03-29T08:08:31.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:31.498+0000 7f7860e00700 1 -- 172.21.15.37:0/3687297606 learned_addr learned my addr 172.21.15.37:0/3687297606 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:08:31.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:31.498+0000 7f7861800700 1 --2- 172.21.15.37:0/3687297606 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f785c06e9a0 0x7f785c11c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:08:31.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:31.499+0000 7f7860e00700 1 -- 172.21.15.37:0/3687297606 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f785c06e9a0 msgr2=0x7f785c11c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:08:31.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:31.499+0000 7f7860e00700 1 --2- 172.21.15.37:0/3687297606 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f785c06e9a0 0x7f785c11c650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:08:31.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:31.499+0000 7f7860e00700 1 -- 172.21.15.37:0/3687297606 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7848005040 con 0x7f785c06f9d0 2024-03-29T08:08:31.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:31.499+0000 7f7861800700 1 --2- 172.21.15.37:0/3687297606 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f785c06e9a0 0x7f785c11c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:08:31.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:31.500+0000 7f7860e00700 1 --2- 172.21.15.37:0/3687297606 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f785c06f9d0 0x7f785c11cb90 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f784800b3b0 tx=0x7f7848009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:08:31.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:31.501+0000 7f7859600700 1 -- 172.21.15.37:0/3687297606 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f784800e040 con 0x7f785c06f9d0 2024-03-29T08:08:31.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:31.502+0000 7f7859600700 1 -- 172.21.15.37:0/3687297606 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f7848004290 con 0x7f785c06f9d0 2024-03-29T08:08:31.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:31.503+0000 7f7864200700 1 -- 172.21.15.37:0/3687297606 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f785c10b720 con 0x7f785c06f9d0 2024-03-29T08:08:31.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:31.503+0000 7f7859600700 1 -- 172.21.15.37:0/3687297606 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f784801c3f0 con 0x7f785c06f9d0 2024-03-29T08:08:31.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:31.503+0000 7f7864200700 1 -- 172.21.15.37:0/3687297606 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f785c10bce0 con 0x7f785c06f9d0 2024-03-29T08:08:31.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:31.508+0000 7f7864200700 1 -- 172.21.15.37:0/3687297606 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f785c04c710 con 0x7f785c06f9d0 2024-03-29T08:08:31.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:31.509+0000 7f7859600700 1 -- 172.21.15.37:0/3687297606 <== mon.1 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 32) v1 ==== 90805+0+0 (secure 0 0 0) 0x7f784801c550 con 0x7f785c06f9d0 2024-03-29T08:08:31.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:31.511+0000 7f7859600700 1 --2- 172.21.15.37:0/3687297606 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7f784406cb50 0x7f784406f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:08:31.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:31.511+0000 7f7859600700 1 -- 172.21.15.37:0/3687297606 <== mon.1 v2:172.21.15.148:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f784808d090 con 0x7f785c06f9d0 2024-03-29T08:08:31.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:31.512+0000 7f7861800700 1 --2- 172.21.15.37:0/3687297606 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7f784406cb50 0x7f784406f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:08:31.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:31.513+0000 7f7861800700 1 --2- 172.21.15.37:0/3687297606 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7f784406cb50 0x7f784406f000 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f7854001250 tx=0x7f785400b040 comp rx=0 tx=0).ready entity=mgr.14686 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:08:31.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:31.520+0000 7f7859600700 1 -- 172.21.15.37:0/3687297606 <== mon.1 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181579 (secure 0 0 0) 0x7f784805aeb0 con 0x7f785c06f9d0 2024-03-29T08:08:31.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:31.927+0000 7f7864200700 1 -- 172.21.15.37:0/3687297606 --> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f785c10b8b0 con 0x7f784406cb50 2024-03-29T08:08:31.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:31.929+0000 7f7859600700 1 -- 172.21.15.37:0/3687297606 <== mgr.14686 v2:172.21.15.37:6800/4188329516 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+329 (secure 0 0 0) 0x7f785c10b8b0 con 0x7f784406cb50 2024-03-29T08:08:31.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:31.933+0000 7f7864200700 1 -- 172.21.15.37:0/3687297606 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7f784406cb50 msgr2=0x7f784406f000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:08:31.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:31.933+0000 7f7864200700 1 --2- 172.21.15.37:0/3687297606 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7f784406cb50 0x7f784406f000 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f7854001250 tx=0x7f785400b040 comp rx=0 tx=0).stop 2024-03-29T08:08:31.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:31.933+0000 7f7864200700 1 -- 172.21.15.37:0/3687297606 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f785c06f9d0 msgr2=0x7f785c11cb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:08:31.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:31.933+0000 7f7864200700 1 --2- 172.21.15.37:0/3687297606 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f785c06f9d0 0x7f785c11cb90 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f784800b3b0 tx=0x7f7848009e30 comp rx=0 tx=0).stop 2024-03-29T08:08:31.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:31.933+0000 7f7864200700 1 -- 172.21.15.37:0/3687297606 shutdown_connections 2024-03-29T08:08:31.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:31.933+0000 7f7864200700 1 --2- 172.21.15.37:0/3687297606 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7f784406cb50 0x7f784406f000 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:08:31.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:31.933+0000 7f7864200700 1 --2- 172.21.15.37:0/3687297606 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f785c06e9a0 0x7f785c11c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:08:31.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:31.933+0000 7f7864200700 1 --2- 172.21.15.37:0/3687297606 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f785c06f9d0 0x7f785c11cb90 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:08:31.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:31.933+0000 7f7864200700 1 -- 172.21.15.37:0/3687297606 >> 172.21.15.37:0/3687297606 conn(0x7f785c069de0 msgr2=0x7f785c06b5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:08:31.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:31.934+0000 7f7864200700 1 -- 172.21.15.37:0/3687297606 shutdown_connections 2024-03-29T08:08:31.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:31.934+0000 7f7864200700 1 -- 172.21.15.37:0/3687297606 wait complete. 2024-03-29T08:08:32.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:32.162+0000 7f1edd000700 1 -- 172.21.15.37:0/1275603891 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ed80662e0 msgr2=0x7f1ed80fd7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:08:32.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:32.162+0000 7f1edd000700 1 --2- 172.21.15.37:0/1275603891 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ed80662e0 0x7f1ed80fd7b0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f1ec8004c20 tx=0x7f1ec8009ca0 comp rx=0 tx=0).stop 2024-03-29T08:08:32.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:32.163+0000 7f1edd000700 1 -- 172.21.15.37:0/1275603891 shutdown_connections 2024-03-29T08:08:32.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:32.163+0000 7f1edd000700 1 --2- 172.21.15.37:0/1275603891 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1ed80fdcf0 0x7f1ed8100170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:08:32.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:32.163+0000 7f1edd000700 1 --2- 172.21.15.37:0/1275603891 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ed80662e0 0x7f1ed80fd7b0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:08:32.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:32.163+0000 7f1edd000700 1 -- 172.21.15.37:0/1275603891 >> 172.21.15.37:0/1275603891 conn(0x7f1ed80f7430 msgr2=0x7f1ed80f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:08:32.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:32.163+0000 7f1edd000700 1 -- 172.21.15.37:0/1275603891 shutdown_connections 2024-03-29T08:08:32.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:32.163+0000 7f1edd000700 1 -- 172.21.15.37:0/1275603891 wait complete. 2024-03-29T08:08:32.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:32.165+0000 7f1edd000700 1 Processor -- start 2024-03-29T08:08:32.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:32.165+0000 7f1edd000700 1 -- start start 2024-03-29T08:08:32.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:32.168+0000 7f1edd000700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1ed80fdcf0 0x7f1ed8114170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:08:32.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:32.168+0000 7f1edd000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ed81146b0 0x7f1ed8115500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:08:32.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:32.168+0000 7f1edd000700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ed8114b70 con 0x7f1ed81146b0 2024-03-29T08:08:32.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:32.168+0000 7f1edd000700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ed8114cb0 con 0x7f1ed80fdcf0 2024-03-29T08:08:32.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:32.168+0000 7f1ed5e00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1ed80fdcf0 0x7f1ed8114170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:08:32.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:32.168+0000 7f1ed5e00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1ed80fdcf0 0x7f1ed8114170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.37:51294/0 (socket says 172.21.15.37:51294) 2024-03-29T08:08:32.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:32.168+0000 7f1ed5e00700 1 -- 172.21.15.37:0/3261299915 learned_addr learned my addr 172.21.15.37:0/3261299915 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:08:32.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:32.169+0000 7f1ed5400700 1 --2- 172.21.15.37:0/3261299915 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ed81146b0 0x7f1ed8115500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:08:32.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:32.169+0000 7f1ed5e00700 1 -- 172.21.15.37:0/3261299915 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ed81146b0 msgr2=0x7f1ed8115500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:08:32.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:32.169+0000 7f1ed5e00700 1 --2- 172.21.15.37:0/3261299915 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ed81146b0 0x7f1ed8115500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:08:32.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:32.169+0000 7f1ed5e00700 1 -- 172.21.15.37:0/3261299915 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ec80048f0 con 0x7f1ed80fdcf0 2024-03-29T08:08:32.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:32.169+0000 7f1ed5e00700 1 --2- 172.21.15.37:0/3261299915 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1ed80fdcf0 0x7f1ed8114170 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f1ec8015040 tx=0x7f1ec8009ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:08:32.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:32.172+0000 7f1ece000700 1 -- 172.21.15.37:0/3261299915 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1ec800e040 con 0x7f1ed80fdcf0 2024-03-29T08:08:32.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:32.172+0000 7f1ece000700 1 -- 172.21.15.37:0/3261299915 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f1ec8007e80 con 0x7f1ed80fdcf0 2024-03-29T08:08:32.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:32.172+0000 7f1ece000700 1 -- 172.21.15.37:0/3261299915 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1ec80126e0 con 0x7f1ed80fdcf0 2024-03-29T08:08:32.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:32.172+0000 7f1edd000700 1 -- 172.21.15.37:0/3261299915 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ed8115b00 con 0x7f1ed80fdcf0 2024-03-29T08:08:32.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:32.173+0000 7f1edd000700 1 -- 172.21.15.37:0/3261299915 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ed8116040 con 0x7f1ed80fdcf0 2024-03-29T08:08:32.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:32.175+0000 7f1edd000700 1 -- 172.21.15.37:0/3261299915 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ed805f5f0 con 0x7f1ed80fdcf0 2024-03-29T08:08:32.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:32.175+0000 7f1ece000700 1 -- 172.21.15.37:0/3261299915 <== mon.1 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 32) v1 ==== 90805+0+0 (secure 0 0 0) 0x7f1ec801c440 con 0x7f1ed80fdcf0 2024-03-29T08:08:32.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:32.178+0000 7f1ece000700 1 --2- 172.21.15.37:0/3261299915 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7f1ebc06cad0 0x7f1ebc06ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:08:32.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:32.179+0000 7f1ece000700 1 -- 172.21.15.37:0/3261299915 <== mon.1 v2:172.21.15.148:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f1ec808da20 con 0x7f1ed80fdcf0 2024-03-29T08:08:32.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:32.181+0000 7f1ed5400700 1 --2- 172.21.15.37:0/3261299915 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7f1ebc06cad0 0x7f1ebc06ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:08:32.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:32.181+0000 7f1ece000700 1 -- 172.21.15.37:0/3261299915 <== mon.1 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181579 (secure 0 0 0) 0x7f1ec805b810 con 0x7f1ed80fdcf0 2024-03-29T08:08:32.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:32.184+0000 7f1ed5400700 1 --2- 172.21.15.37:0/3261299915 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7f1ebc06cad0 0x7f1ebc06ef80 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f1ec40046c0 tx=0x7f1ec400b040 comp rx=0 tx=0).ready entity=mgr.14686 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:08:32.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:08:32 smithi037 ceph-mon[35162]: pgmap v15: 65 pgs: 65 active+clean; 285 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 37 KiB/s rd, 1.4 MiB/s wr, 363 op/s 2024-03-29T08:08:32.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:08:32 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' 2024-03-29T08:08:32.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:08:32 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' 2024-03-29T08:08:32.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:32.546+0000 7f1edd000700 1 -- 172.21.15.37:0/3261299915 --> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f1ed8116330 con 0x7f1ebc06cad0 2024-03-29T08:08:32.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:08:32 smithi148 ceph-mon[44992]: pgmap v15: 65 pgs: 65 active+clean; 285 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 37 KiB/s rd, 1.4 MiB/s wr, 363 op/s 2024-03-29T08:08:32.557 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:08:32 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' 2024-03-29T08:08:32.557 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:08:32 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' 2024-03-29T08:08:32.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:32.571+0000 7f1ece000700 1 -- 172.21.15.37:0/3261299915 <== mgr.14686 v2:172.21.15.37:6800/4188329516 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7f1ed8116330 con 0x7f1ebc06cad0 2024-03-29T08:08:32.572 INFO:teuthology.orchestra.run.smithi037.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-29T08:08:32.572 INFO:teuthology.orchestra.run.smithi037.stdout:alertmanager.smithi037 smithi037 *:9093,9094 running (119s) 3s ago 11m 14.9M - 0.25.0 c8568f914cd2 8a8d40a6e26e 2024-03-29T08:08:32.572 INFO:teuthology.orchestra.run.smithi037.stdout:crash.smithi037 smithi037 running (11m) 3s ago 11m 6656k - 17.2.7-789-g4213e17c cf22eb8683be e0343935b25e 2024-03-29T08:08:32.572 INFO:teuthology.orchestra.run.smithi037.stdout:crash.smithi148 smithi148 running (10m) 20s ago 10m 6656k - 17.2.7-789-g4213e17c cf22eb8683be ff5164c552c1 2024-03-29T08:08:32.572 INFO:teuthology.orchestra.run.smithi037.stdout:grafana.smithi037 smithi037 *:3000 running (98s) 3s ago 11m 74.0M - 9.4.7 954c08fa6188 a857385eee0c 2024-03-29T08:08:32.572 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi037.iifsmh smithi037 running (4m) 3s ago 4m 142M - 17.2.7-789-g4213e17c cf22eb8683be 074e1acf331f 2024-03-29T08:08:32.572 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi037.mmwgxe smithi037 running (4m) 3s ago 4m 148M - 17.2.7-789-g4213e17c cf22eb8683be 9a5b1e72de5c 2024-03-29T08:08:32.572 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi148.qukpjh smithi148 running (4m) 20s ago 4m 12.4M - 17.2.7-789-g4213e17c cf22eb8683be 8e469782a273 2024-03-29T08:08:32.573 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi148.sqteih smithi148 running (4m) 20s ago 4m 12.3M - 17.2.7-789-g4213e17c cf22eb8683be 8e68f858bb22 2024-03-29T08:08:32.573 INFO:teuthology.orchestra.run.smithi037.stdout:mgr.smithi037.wjikii smithi037 *:8443,9283,8765 running (43s) 3s ago 12m 473M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff c5a1417b9417 2024-03-29T08:08:32.573 INFO:teuthology.orchestra.run.smithi037.stdout:mgr.smithi148.onhgvm smithi148 *:8443,9283 running (2m) 20s ago 10m 99.9M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 0c02ffa63851 2024-03-29T08:08:32.573 INFO:teuthology.orchestra.run.smithi037.stdout:mon.smithi037 smithi037 running (13m) 3s ago 13m 52.3M 2048M 17.2.7-789-g4213e17c cf22eb8683be bb2101505730 2024-03-29T08:08:32.573 INFO:teuthology.orchestra.run.smithi037.stdout:mon.smithi148 smithi148 running (10m) 20s ago 10m 42.1M 2048M 17.2.7-789-g4213e17c cf22eb8683be 18839ed2952a 2024-03-29T08:08:32.573 INFO:teuthology.orchestra.run.smithi037.stdout:node-exporter.smithi037 smithi037 *:9100 running (2m) 3s ago 11m 7675k - 1.5.0 0da6a335fe13 cdaa34126e04 2024-03-29T08:08:32.573 INFO:teuthology.orchestra.run.smithi037.stdout:node-exporter.smithi148 smithi148 *:9100 running (71s) 20s ago 10m 7277k - 1.5.0 0da6a335fe13 0d793b5bd838 2024-03-29T08:08:32.573 INFO:teuthology.orchestra.run.smithi037.stdout:osd.0 smithi037 running (9m) 3s ago 9m 189M 1619M 17.2.7-789-g4213e17c cf22eb8683be 559fccd0ebe1 2024-03-29T08:08:32.573 INFO:teuthology.orchestra.run.smithi037.stdout:osd.1 smithi037 running (8m) 3s ago 8m 208M 1619M 17.2.7-789-g4213e17c cf22eb8683be 5582cff11b7a 2024-03-29T08:08:32.574 INFO:teuthology.orchestra.run.smithi037.stdout:osd.2 smithi037 running (8m) 3s ago 8m 172M 1619M 17.2.7-789-g4213e17c cf22eb8683be f809bfab13e5 2024-03-29T08:08:32.574 INFO:teuthology.orchestra.run.smithi037.stdout:osd.3 smithi148 running (7m) 20s ago 7m 231M 2642M 17.2.7-789-g4213e17c cf22eb8683be a0a8224bb4b0 2024-03-29T08:08:32.574 INFO:teuthology.orchestra.run.smithi037.stdout:osd.4 smithi148 running (7m) 20s ago 7m 200M 2642M 17.2.7-789-g4213e17c cf22eb8683be a00b69047ed7 2024-03-29T08:08:32.574 INFO:teuthology.orchestra.run.smithi037.stdout:osd.5 smithi148 running (6m) 20s ago 6m 213M 2642M 17.2.7-789-g4213e17c cf22eb8683be 044a3f7db7f8 2024-03-29T08:08:32.574 INFO:teuthology.orchestra.run.smithi037.stdout:prometheus.smithi037 smithi037 *:9095 running (80s) 3s ago 10m 42.2M - 2.43.0 a07b618ecd1d 51e61f03c4ea 2024-03-29T08:08:32.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:32.575+0000 7f1edd000700 1 -- 172.21.15.37:0/3261299915 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7f1ebc06cad0 msgr2=0x7f1ebc06ef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:08:32.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:32.575+0000 7f1edd000700 1 --2- 172.21.15.37:0/3261299915 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7f1ebc06cad0 0x7f1ebc06ef80 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f1ec40046c0 tx=0x7f1ec400b040 comp rx=0 tx=0).stop 2024-03-29T08:08:32.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:32.575+0000 7f1edd000700 1 -- 172.21.15.37:0/3261299915 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1ed80fdcf0 msgr2=0x7f1ed8114170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:08:32.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:32.575+0000 7f1edd000700 1 --2- 172.21.15.37:0/3261299915 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1ed80fdcf0 0x7f1ed8114170 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f1ec8015040 tx=0x7f1ec8009ca0 comp rx=0 tx=0).stop 2024-03-29T08:08:32.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:32.576+0000 7f1edd000700 1 -- 172.21.15.37:0/3261299915 shutdown_connections 2024-03-29T08:08:32.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:32.576+0000 7f1edd000700 1 --2- 172.21.15.37:0/3261299915 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7f1ebc06cad0 0x7f1ebc06ef80 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:08:32.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:32.576+0000 7f1edd000700 1 --2- 172.21.15.37:0/3261299915 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1ed80fdcf0 0x7f1ed8114170 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:08:32.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:32.576+0000 7f1edd000700 1 --2- 172.21.15.37:0/3261299915 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ed81146b0 0x7f1ed8115500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:08:32.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:32.576+0000 7f1edd000700 1 -- 172.21.15.37:0/3261299915 >> 172.21.15.37:0/3261299915 conn(0x7f1ed80f7430 msgr2=0x7f1ed80f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:08:32.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:32.577+0000 7f1edd000700 1 -- 172.21.15.37:0/3261299915 shutdown_connections 2024-03-29T08:08:32.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:32.577+0000 7f1edd000700 1 -- 172.21.15.37:0/3261299915 wait complete. 2024-03-29T08:08:32.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:32.783+0000 7f671f200700 1 -- 172.21.15.37:0/4235419936 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f67200ffdc0 msgr2=0x7f67201021a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:08:32.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:32.783+0000 7f671f200700 1 --2- 172.21.15.37:0/4235419936 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f67200ffdc0 0x7f67201021a0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f670c004940 tx=0x7f670c009e30 comp rx=0 tx=0).stop 2024-03-29T08:08:32.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:32.783+0000 7f671f200700 1 -- 172.21.15.37:0/4235419936 shutdown_connections 2024-03-29T08:08:32.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:32.783+0000 7f671f200700 1 --2- 172.21.15.37:0/4235419936 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f67200ffdc0 0x7f67201021a0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:08:32.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:32.783+0000 7f671f200700 1 --2- 172.21.15.37:0/4235419936 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6720066260 0x7f67200ff880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:08:32.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:32.783+0000 7f671f200700 1 -- 172.21.15.37:0/4235419936 >> 172.21.15.37:0/4235419936 conn(0x7f67200f7270 msgr2=0x7f67200f96a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:08:32.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:32.783+0000 7f671f200700 1 -- 172.21.15.37:0/4235419936 shutdown_connections 2024-03-29T08:08:32.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:32.783+0000 7f671f200700 1 -- 172.21.15.37:0/4235419936 wait complete. 2024-03-29T08:08:32.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:32.785+0000 7f671f200700 1 Processor -- start 2024-03-29T08:08:32.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:32.785+0000 7f671f200700 1 -- start start 2024-03-29T08:08:32.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:32.785+0000 7f671f200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6720066260 0x7f672010f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:08:32.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:32.785+0000 7f671f200700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f67200ffdc0 0x7f672010fe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:08:32.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:32.785+0000 7f671f200700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6720110430 con 0x7f6720066260 2024-03-29T08:08:32.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:32.785+0000 7f671f200700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6720110c80 con 0x7f67200ffdc0 2024-03-29T08:08:32.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:32.786+0000 7f671de00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6720066260 0x7f672010f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:08:32.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:32.786+0000 7f671d400700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f67200ffdc0 0x7f672010fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:08:32.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:32.786+0000 7f671d400700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f67200ffdc0 0x7f672010fe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.37:51320/0 (socket says 172.21.15.37:51320) 2024-03-29T08:08:32.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:32.786+0000 7f671de00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6720066260 0x7f672010f8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52302/0 (socket says 172.21.15.37:52302) 2024-03-29T08:08:32.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:32.786+0000 7f671d400700 1 -- 172.21.15.37:0/2754924871 learned_addr learned my addr 172.21.15.37:0/2754924871 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:08:32.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:32.786+0000 7f671d400700 1 -- 172.21.15.37:0/2754924871 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6720066260 msgr2=0x7f672010f8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:08:32.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:32.786+0000 7f671d400700 1 --2- 172.21.15.37:0/2754924871 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6720066260 0x7f672010f8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:08:32.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:32.786+0000 7f671d400700 1 -- 172.21.15.37:0/2754924871 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f670c005040 con 0x7f67200ffdc0 2024-03-29T08:08:32.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:32.786+0000 7f671de00700 1 --2- 172.21.15.37:0/2754924871 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6720066260 0x7f672010f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T08:08:32.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:32.787+0000 7f671d400700 1 --2- 172.21.15.37:0/2754924871 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f67200ffdc0 0x7f672010fe30 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f670c0137b0 tx=0x7f670c009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:08:32.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:32.787+0000 7f6716000700 1 -- 172.21.15.37:0/2754924871 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f670c021070 con 0x7f67200ffdc0 2024-03-29T08:08:32.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:32.787+0000 7f6716000700 1 -- 172.21.15.37:0/2754924871 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f670c01c400 con 0x7f67200ffdc0 2024-03-29T08:08:32.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:32.787+0000 7f6716000700 1 -- 172.21.15.37:0/2754924871 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f670c00e070 con 0x7f67200ffdc0 2024-03-29T08:08:32.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:32.788+0000 7f671f200700 1 -- 172.21.15.37:0/2754924871 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6720110f00 con 0x7f67200ffdc0 2024-03-29T08:08:32.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:32.788+0000 7f671f200700 1 -- 172.21.15.37:0/2754924871 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6720111440 con 0x7f67200ffdc0 2024-03-29T08:08:32.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:32.790+0000 7f671f200700 1 -- 172.21.15.37:0/2754924871 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f672005f5f0 con 0x7f67200ffdc0 2024-03-29T08:08:32.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:32.790+0000 7f6716000700 1 -- 172.21.15.37:0/2754924871 <== mon.1 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 32) v1 ==== 90805+0+0 (secure 0 0 0) 0x7f670c01c970 con 0x7f67200ffdc0 2024-03-29T08:08:32.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:32.791+0000 7f6716000700 1 --2- 172.21.15.37:0/2754924871 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7f670406ca90 0x7f670406ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:08:32.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:32.792+0000 7f6716000700 1 -- 172.21.15.37:0/2754924871 <== mon.1 v2:172.21.15.148:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f670c08c7e0 con 0x7f67200ffdc0 2024-03-29T08:08:32.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:32.792+0000 7f671de00700 1 --2- 172.21.15.37:0/2754924871 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7f670406ca90 0x7f670406ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:08:32.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:32.793+0000 7f671de00700 1 --2- 172.21.15.37:0/2754924871 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7f670406ca90 0x7f670406ef40 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f6710009b80 tx=0x7f671000b040 comp rx=0 tx=0).ready entity=mgr.14686 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:08:32.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:32.796+0000 7f6716000700 1 -- 172.21.15.37:0/2754924871 <== mon.1 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181579 (secure 0 0 0) 0x7f670c05a700 con 0x7f67200ffdc0 2024-03-29T08:08:33.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:33.236+0000 7f671f200700 1 -- 172.21.15.37:0/2754924871 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f67201055b0 con 0x7f67200ffdc0 2024-03-29T08:08:33.236 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-29T08:08:33.237 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": { 2024-03-29T08:08:33.237 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 17.2.7-789-g4213e17c (4213e17c4de67016ed14f55d22503f4923b3a901) quincy (stable)": 2 2024-03-29T08:08:33.237 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:08:33.237 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": { 2024-03-29T08:08:33.237 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 2 2024-03-29T08:08:33.237 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:08:33.237 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": { 2024-03-29T08:08:33.237 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 17.2.7-789-g4213e17c (4213e17c4de67016ed14f55d22503f4923b3a901) quincy (stable)": 6 2024-03-29T08:08:33.237 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:08:33.237 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": { 2024-03-29T08:08:33.237 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 17.2.7-789-g4213e17c (4213e17c4de67016ed14f55d22503f4923b3a901) quincy (stable)": 4 2024-03-29T08:08:33.237 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:08:33.237 INFO:teuthology.orchestra.run.smithi037.stdout: "overall": { 2024-03-29T08:08:33.237 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 17.2.7-789-g4213e17c (4213e17c4de67016ed14f55d22503f4923b3a901) quincy (stable)": 12, 2024-03-29T08:08:33.237 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 2 2024-03-29T08:08:33.238 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-29T08:08:33.238 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-29T08:08:33.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:33.237+0000 7f6716000700 1 -- 172.21.15.37:0/2754924871 <== mon.1 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+743 (secure 0 0 0) 0x7f670c05a290 con 0x7f67200ffdc0 2024-03-29T08:08:33.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:33.240+0000 7f671f200700 1 -- 172.21.15.37:0/2754924871 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7f670406ca90 msgr2=0x7f670406ef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:08:33.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:33.240+0000 7f671f200700 1 --2- 172.21.15.37:0/2754924871 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7f670406ca90 0x7f670406ef40 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f6710009b80 tx=0x7f671000b040 comp rx=0 tx=0).stop 2024-03-29T08:08:33.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:33.240+0000 7f671f200700 1 -- 172.21.15.37:0/2754924871 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f67200ffdc0 msgr2=0x7f672010fe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:08:33.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:33.240+0000 7f671f200700 1 --2- 172.21.15.37:0/2754924871 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f67200ffdc0 0x7f672010fe30 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f670c0137b0 tx=0x7f670c009e30 comp rx=0 tx=0).stop 2024-03-29T08:08:33.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:33.240+0000 7f671f200700 1 -- 172.21.15.37:0/2754924871 shutdown_connections 2024-03-29T08:08:33.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:33.240+0000 7f671f200700 1 --2- 172.21.15.37:0/2754924871 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7f670406ca90 0x7f670406ef40 secure :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f6710009b80 tx=0x7f671000b040 comp rx=0 tx=0).stop 2024-03-29T08:08:33.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:33.240+0000 7f671f200700 1 --2- 172.21.15.37:0/2754924871 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6720066260 0x7f672010f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:08:33.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:33.240+0000 7f671f200700 1 --2- 172.21.15.37:0/2754924871 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f67200ffdc0 0x7f672010fe30 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:08:33.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:33.240+0000 7f671f200700 1 -- 172.21.15.37:0/2754924871 >> 172.21.15.37:0/2754924871 conn(0x7f67200f7270 msgr2=0x7f67200fb8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:08:33.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:33.240+0000 7f671f200700 1 -- 172.21.15.37:0/2754924871 shutdown_connections 2024-03-29T08:08:33.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:33.241+0000 7f671f200700 1 -- 172.21.15.37:0/2754924871 wait complete. 2024-03-29T08:08:33.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:08:33 smithi037 ceph-mon[35162]: from='client.14726 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:08:33.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:08:33 smithi037 ceph-mon[35162]: from='client.24555 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:08:33.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:33.436+0000 7fea07200700 1 -- 172.21.15.37:0/1931881568 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fea08073170 msgr2=0x7fea08073b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:08:33.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:33.436+0000 7fea07200700 1 --2- 172.21.15.37:0/1931881568 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fea08073170 0x7fea08073b80 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fe9f4004940 tx=0x7fe9f4009e30 comp rx=0 tx=0).stop 2024-03-29T08:08:33.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:33.436+0000 7fea07200700 1 -- 172.21.15.37:0/1931881568 shutdown_connections 2024-03-29T08:08:33.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:33.436+0000 7fea07200700 1 --2- 172.21.15.37:0/1931881568 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fea08073170 0x7fea08073b80 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:08:33.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:33.436+0000 7fea07200700 1 --2- 172.21.15.37:0/1931881568 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fea08072820 0x7fea08072c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:08:33.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:33.436+0000 7fea07200700 1 -- 172.21.15.37:0/1931881568 >> 172.21.15.37:0/1931881568 conn(0x7fea080fa790 msgr2=0x7fea080fcb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:08:33.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:33.436+0000 7fea07200700 1 -- 172.21.15.37:0/1931881568 shutdown_connections 2024-03-29T08:08:33.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:33.436+0000 7fea07200700 1 -- 172.21.15.37:0/1931881568 wait complete. 2024-03-29T08:08:33.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:33.438+0000 7fea07200700 1 Processor -- start 2024-03-29T08:08:33.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:33.439+0000 7fea07200700 1 -- start start 2024-03-29T08:08:33.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:33.439+0000 7fea07200700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fea08072820 0x7fea0806e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:08:33.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:33.439+0000 7fea07200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fea0806e940 0x7fea0806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:08:33.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:33.439+0000 7fea07200700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea0806ff30 con 0x7fea0806e940 2024-03-29T08:08:33.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:33.439+0000 7fea07200700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea080700a0 con 0x7fea08072820 2024-03-29T08:08:33.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:33.439+0000 7fea05e00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fea08072820 0x7fea0806e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:08:33.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:33.439+0000 7fea05e00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fea08072820 0x7fea0806e400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.37:51326/0 (socket says 172.21.15.37:51326) 2024-03-29T08:08:33.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:33.439+0000 7fea05e00700 1 -- 172.21.15.37:0/257859853 learned_addr learned my addr 172.21.15.37:0/257859853 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:08:33.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:33.440+0000 7fea05e00700 1 -- 172.21.15.37:0/257859853 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fea0806e940 msgr2=0x7fea0806edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:08:33.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:33.440+0000 7fea05e00700 1 --2- 172.21.15.37:0/257859853 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fea0806e940 0x7fea0806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:08:33.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:33.440+0000 7fea05e00700 1 -- 172.21.15.37:0/257859853 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9f4005040 con 0x7fea08072820 2024-03-29T08:08:33.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:33.440+0000 7fea05e00700 1 --2- 172.21.15.37:0/257859853 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fea08072820 0x7fea0806e400 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fe9f800eb70 tx=0x7fe9f800b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:08:33.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:33.441+0000 7fe9fe000700 1 -- 172.21.15.37:0/257859853 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe9f8007cc0 con 0x7fea08072820 2024-03-29T08:08:33.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:33.441+0000 7fea07200700 1 -- 172.21.15.37:0/257859853 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea081220b0 con 0x7fea08072820 2024-03-29T08:08:33.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:33.441+0000 7fe9fe000700 1 -- 172.21.15.37:0/257859853 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fe9f800f040 con 0x7fea08072820 2024-03-29T08:08:33.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:33.441+0000 7fe9fe000700 1 -- 172.21.15.37:0/257859853 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe9f800cad0 con 0x7fea08072820 2024-03-29T08:08:33.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:33.441+0000 7fea07200700 1 -- 172.21.15.37:0/257859853 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea08122620 con 0x7fea08072820 2024-03-29T08:08:33.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:33.442+0000 7fea07200700 1 -- 172.21.15.37:0/257859853 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9e80052e0 con 0x7fea08072820 2024-03-29T08:08:33.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:33.444+0000 7fe9fe000700 1 -- 172.21.15.37:0/257859853 <== mon.1 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 32) v1 ==== 90805+0+0 (secure 0 0 0) 0x7fe9f800cc30 con 0x7fea08072820 2024-03-29T08:08:33.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:33.445+0000 7fe9fe000700 1 --2- 172.21.15.37:0/257859853 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7fe9ec06caa0 0x7fe9ec06ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:08:33.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:33.445+0000 7fe9fe000700 1 -- 172.21.15.37:0/257859853 <== mon.1 v2:172.21.15.148:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6066+0+0 (secure 0 0 0) 0x7fe9f808b7c0 con 0x7fea08072820 2024-03-29T08:08:33.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:33.446+0000 7fea05400700 1 --2- 172.21.15.37:0/257859853 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7fe9ec06caa0 0x7fe9ec06ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:08:33.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:33.447+0000 7fea05400700 1 --2- 172.21.15.37:0/257859853 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7fe9ec06caa0 0x7fe9ec06ef50 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fe9f4009fd0 tx=0x7fe9f4009e30 comp rx=0 tx=0).ready entity=mgr.14686 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:08:33.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:33.449+0000 7fe9fe000700 1 -- 172.21.15.37:0/257859853 <== mon.1 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181579 (secure 0 0 0) 0x7fe9f8055b10 con 0x7fea08072820 2024-03-29T08:08:33.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:08:33 smithi148 ceph-mon[44992]: from='client.14726 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:08:33.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:08:33 smithi148 ceph-mon[44992]: from='client.24555 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:08:33.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:33.844+0000 7fea07200700 1 -- 172.21.15.37:0/257859853 --> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe9e8000bf0 con 0x7fe9ec06caa0 2024-03-29T08:08:33.846 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-29T08:08:33.846 INFO:teuthology.orchestra.run.smithi037.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59", 2024-03-29T08:08:33.846 INFO:teuthology.orchestra.run.smithi037.stdout: "in_progress": true, 2024-03-29T08:08:33.846 INFO:teuthology.orchestra.run.smithi037.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-03-29T08:08:33.846 INFO:teuthology.orchestra.run.smithi037.stdout: "services_complete": [ 2024-03-29T08:08:33.846 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr" 2024-03-29T08:08:33.846 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-29T08:08:33.846 INFO:teuthology.orchestra.run.smithi037.stdout: "progress": "2/2 daemons upgraded", 2024-03-29T08:08:33.847 INFO:teuthology.orchestra.run.smithi037.stdout: "message": "", 2024-03-29T08:08:33.847 INFO:teuthology.orchestra.run.smithi037.stdout: "is_paused": false 2024-03-29T08:08:33.847 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-29T08:08:33.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:33.846+0000 7fe9fe000700 1 -- 172.21.15.37:0/257859853 <== mgr.14686 v2:172.21.15.37:6800/4188329516 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+329 (secure 0 0 0) 0x7fe9e8000bf0 con 0x7fe9ec06caa0 2024-03-29T08:08:33.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:33.850+0000 7fea07200700 1 -- 172.21.15.37:0/257859853 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7fe9ec06caa0 msgr2=0x7fe9ec06ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:08:33.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:33.850+0000 7fea07200700 1 --2- 172.21.15.37:0/257859853 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7fe9ec06caa0 0x7fe9ec06ef50 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fe9f4009fd0 tx=0x7fe9f4009e30 comp rx=0 tx=0).stop 2024-03-29T08:08:33.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:33.850+0000 7fea07200700 1 -- 172.21.15.37:0/257859853 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fea08072820 msgr2=0x7fea0806e400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:08:33.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:33.850+0000 7fea07200700 1 --2- 172.21.15.37:0/257859853 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fea08072820 0x7fea0806e400 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fe9f800eb70 tx=0x7fe9f800b040 comp rx=0 tx=0).stop 2024-03-29T08:08:33.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:33.854+0000 7fea07200700 1 -- 172.21.15.37:0/257859853 shutdown_connections 2024-03-29T08:08:33.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:33.854+0000 7fea07200700 1 --2- 172.21.15.37:0/257859853 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7fe9ec06caa0 0x7fe9ec06ef50 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:08:33.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:33.854+0000 7fea07200700 1 --2- 172.21.15.37:0/257859853 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fea08072820 0x7fea0806e400 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:08:33.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:33.854+0000 7fea07200700 1 --2- 172.21.15.37:0/257859853 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fea0806e940 0x7fea0806edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:08:33.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:33.854+0000 7fea07200700 1 -- 172.21.15.37:0/257859853 >> 172.21.15.37:0/257859853 conn(0x7fea080fa790 msgr2=0x7fea080fcdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:08:33.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:33.854+0000 7fea07200700 1 -- 172.21.15.37:0/257859853 shutdown_connections 2024-03-29T08:08:33.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:08:33.854+0000 7fea07200700 1 -- 172.21.15.37:0/257859853 wait complete. 2024-03-29T08:08:34.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:08:34 smithi037 ceph-mon[35162]: pgmap v16: 65 pgs: 65 active+clean; 285 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 37 KiB/s rd, 1.4 MiB/s wr, 363 op/s 2024-03-29T08:08:34.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:08:34 smithi037 ceph-mon[35162]: from='client.24559 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:08:34.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:08:34 smithi037 ceph-mon[35162]: from='client.? 172.21.15.37:0/2754924871' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:08:34.478 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:08:34 smithi148 ceph-mon[44992]: pgmap v16: 65 pgs: 65 active+clean; 285 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 37 KiB/s rd, 1.4 MiB/s wr, 363 op/s 2024-03-29T08:08:34.479 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:08:34 smithi148 ceph-mon[44992]: from='client.24559 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:08:34.479 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:08:34 smithi148 ceph-mon[44992]: from='client.? 172.21.15.37:0/2754924871' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:08:35.410 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:08:35 smithi037 ceph-mon[35162]: from='client.24567 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:08:35.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:08:35 smithi148 ceph-mon[44992]: from='client.24567 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:08:36.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:08:36 smithi037 ceph-mon[35162]: pgmap v17: 65 pgs: 65 active+clean; 284 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 55 KiB/s rd, 2.0 MiB/s wr, 559 op/s 2024-03-29T08:08:36.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:08:36 smithi148 ceph-mon[44992]: pgmap v17: 65 pgs: 65 active+clean; 284 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 55 KiB/s rd, 2.0 MiB/s wr, 559 op/s 2024-03-29T08:08:37.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:08:37 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:08:37.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:08:37 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:08:38.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:08:38 smithi037 ceph-mon[35162]: pgmap v18: 65 pgs: 65 active+clean; 284 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 37 KiB/s rd, 1.3 MiB/s wr, 376 op/s 2024-03-29T08:08:38.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:08:38 smithi148 ceph-mon[44992]: pgmap v18: 65 pgs: 65 active+clean; 284 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 37 KiB/s rd, 1.3 MiB/s wr, 376 op/s 2024-03-29T08:08:40.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:08:40 smithi037 ceph-mon[35162]: pgmap v19: 65 pgs: 65 active+clean; 276 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 53 KiB/s rd, 1.9 MiB/s wr, 528 op/s 2024-03-29T08:08:40.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:08:40 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' 2024-03-29T08:08:40.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:08:40 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' 2024-03-29T08:08:40.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:08:40 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T08:08:40.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:08:40 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T08:08:40.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:08:40 smithi148 ceph-mon[44992]: pgmap v19: 65 pgs: 65 active+clean; 276 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 53 KiB/s rd, 1.9 MiB/s wr, 528 op/s 2024-03-29T08:08:40.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:08:40 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' 2024-03-29T08:08:40.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:08:40 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' 2024-03-29T08:08:40.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:08:40 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T08:08:40.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:08:40 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T08:08:41.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:08:41 smithi037 ceph-mon[35162]: Updating smithi037:/etc/ceph/ceph.conf 2024-03-29T08:08:41.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:08:41 smithi037 ceph-mon[35162]: Updating smithi148:/etc/ceph/ceph.conf 2024-03-29T08:08:41.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:08:41 smithi148 ceph-mon[44992]: Updating smithi037:/etc/ceph/ceph.conf 2024-03-29T08:08:41.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:08:41 smithi148 ceph-mon[44992]: Updating smithi148:/etc/ceph/ceph.conf 2024-03-29T08:08:42.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:08:42 smithi037 ceph-mon[35162]: Updating smithi148:/var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/config/ceph.conf 2024-03-29T08:08:42.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:08:42 smithi037 ceph-mon[35162]: pgmap v20: 65 pgs: 65 active+clean; 276 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 33 KiB/s rd, 1.2 MiB/s wr, 347 op/s 2024-03-29T08:08:42.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:08:42 smithi037 ceph-mon[35162]: Updating smithi037:/var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/config/ceph.conf 2024-03-29T08:08:42.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:08:42 smithi148 ceph-mon[44992]: Updating smithi148:/var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/config/ceph.conf 2024-03-29T08:08:42.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:08:42 smithi148 ceph-mon[44992]: pgmap v20: 65 pgs: 65 active+clean; 276 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 33 KiB/s rd, 1.2 MiB/s wr, 347 op/s 2024-03-29T08:08:42.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:08:42 smithi148 ceph-mon[44992]: Updating smithi037:/var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/config/ceph.conf 2024-03-29T08:08:43.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:08:43 smithi037 ceph-mon[35162]: Updating smithi148:/etc/ceph/ceph.client.admin.keyring 2024-03-29T08:08:43.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:08:43 smithi037 ceph-mon[35162]: Updating smithi037:/etc/ceph/ceph.client.admin.keyring 2024-03-29T08:08:43.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:08:43 smithi037 ceph-mon[35162]: Updating smithi148:/var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/config/ceph.client.admin.keyring 2024-03-29T08:08:43.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:08:43 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' 2024-03-29T08:08:43.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:08:43 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' 2024-03-29T08:08:43.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:08:43 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' 2024-03-29T08:08:43.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:08:43 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' 2024-03-29T08:08:43.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:08:43 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' 2024-03-29T08:08:43.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:08:43 smithi148 ceph-mon[44992]: Updating smithi148:/etc/ceph/ceph.client.admin.keyring 2024-03-29T08:08:43.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:08:43 smithi148 ceph-mon[44992]: Updating smithi037:/etc/ceph/ceph.client.admin.keyring 2024-03-29T08:08:43.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:08:43 smithi148 ceph-mon[44992]: Updating smithi148:/var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/config/ceph.client.admin.keyring 2024-03-29T08:08:43.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:08:43 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' 2024-03-29T08:08:43.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:08:43 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' 2024-03-29T08:08:43.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:08:43 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' 2024-03-29T08:08:43.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:08:43 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' 2024-03-29T08:08:43.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:08:43 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' 2024-03-29T08:08:44.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:08:44 smithi037 ceph-mon[35162]: Updating smithi037:/var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/config/ceph.client.admin.keyring 2024-03-29T08:08:44.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:08:44 smithi037 ceph-mon[35162]: pgmap v21: 65 pgs: 65 active+clean; 275 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 33 KiB/s rd, 1.2 MiB/s wr, 362 op/s 2024-03-29T08:08:44.477 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:08:44 smithi148 ceph-mon[44992]: Updating smithi037:/var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/config/ceph.client.admin.keyring 2024-03-29T08:08:44.477 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:08:44 smithi148 ceph-mon[44992]: pgmap v21: 65 pgs: 65 active+clean; 275 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 33 KiB/s rd, 1.2 MiB/s wr, 362 op/s 2024-03-29T08:08:45.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:08:45 smithi037 ceph-mon[35162]: Reconfiguring prometheus.smithi037 (dependencies changed)... 2024-03-29T08:08:45.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:08:45 smithi037 ceph-mon[35162]: Reconfiguring daemon prometheus.smithi037 on smithi037 2024-03-29T08:08:45.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:08:45 smithi148 ceph-mon[44992]: Reconfiguring prometheus.smithi037 (dependencies changed)... 2024-03-29T08:08:45.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:08:45 smithi148 ceph-mon[44992]: Reconfiguring daemon prometheus.smithi037 on smithi037 2024-03-29T08:08:46.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:08:46 smithi037 ceph-mon[35162]: pgmap v22: 65 pgs: 65 active+clean; 275 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 51 KiB/s rd, 1.8 MiB/s wr, 519 op/s 2024-03-29T08:08:46.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:08:46 smithi148 ceph-mon[44992]: pgmap v22: 65 pgs: 65 active+clean; 275 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 51 KiB/s rd, 1.8 MiB/s wr, 519 op/s 2024-03-29T08:08:48.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:08:48 smithi037 ceph-mon[35162]: pgmap v23: 65 pgs: 65 active+clean; 275 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 33 KiB/s rd, 1.2 MiB/s wr, 323 op/s 2024-03-29T08:08:48.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:08:48 smithi148 ceph-mon[44992]: pgmap v23: 65 pgs: 65 active+clean; 275 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 33 KiB/s rd, 1.2 MiB/s wr, 323 op/s 2024-03-29T08:08:50.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:08:50 smithi037 ceph-mon[35162]: pgmap v24: 65 pgs: 65 active+clean; 275 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 49 KiB/s rd, 1.8 MiB/s wr, 486 op/s 2024-03-29T08:08:50.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:08:50 smithi148 ceph-mon[44992]: pgmap v24: 65 pgs: 65 active+clean; 275 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 49 KiB/s rd, 1.8 MiB/s wr, 486 op/s 2024-03-29T08:08:52.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:08:52 smithi037 ceph-mon[35162]: pgmap v25: 65 pgs: 65 active+clean; 275 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 33 KiB/s rd, 1.2 MiB/s wr, 334 op/s 2024-03-29T08:08:52.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:08:52 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:08:52.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:08:52 smithi148 ceph-mon[44992]: pgmap v25: 65 pgs: 65 active+clean; 275 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 33 KiB/s rd, 1.2 MiB/s wr, 334 op/s 2024-03-29T08:08:52.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:08:52 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:08:54.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:08:54 smithi037 ceph-mon[35162]: pgmap v26: 65 pgs: 65 active+clean; 274 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 33 KiB/s rd, 1.2 MiB/s wr, 364 op/s 2024-03-29T08:08:54.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:08:54 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' 2024-03-29T08:08:54.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:08:54 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' 2024-03-29T08:08:54.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:08:54 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-29T08:08:54.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:08:54 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T08:08:54.508 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:08:54 smithi148 ceph-mon[44992]: pgmap v26: 65 pgs: 65 active+clean; 274 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 33 KiB/s rd, 1.2 MiB/s wr, 364 op/s 2024-03-29T08:08:54.509 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:08:54 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' 2024-03-29T08:08:54.509 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:08:54 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' 2024-03-29T08:08:54.509 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:08:54 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-29T08:08:54.509 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:08:54 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T08:08:55.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:08:55 smithi148 ceph-mon[44992]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-29T08:08:55.426 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:08:55 smithi037 ceph-mon[35162]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-29T08:08:56.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:08:56 smithi037 ceph-mon[35162]: pgmap v27: 65 pgs: 65 active+clean; 275 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 49 KiB/s rd, 1.7 MiB/s wr, 477 op/s 2024-03-29T08:08:56.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:08:56 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi148.onhgvm", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-29T08:08:56.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:08:56 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-29T08:08:56.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:08:56 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T08:08:56.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:08:56 smithi148 ceph-mon[44992]: pgmap v27: 65 pgs: 65 active+clean; 275 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 49 KiB/s rd, 1.7 MiB/s wr, 477 op/s 2024-03-29T08:08:56.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:08:56 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi148.onhgvm", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-29T08:08:56.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:08:56 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-29T08:08:56.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:08:56 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T08:08:57.429 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:08:57 smithi037 ceph-mon[35162]: Upgrade: Updating mgr.smithi148.onhgvm 2024-03-29T08:08:57.429 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:08:57 smithi037 ceph-mon[35162]: Deploying daemon mgr.smithi148.onhgvm on smithi148 2024-03-29T08:08:57.511 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:08:57 smithi148 ceph-mon[44992]: Upgrade: Updating mgr.smithi148.onhgvm 2024-03-29T08:08:57.511 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:08:57 smithi148 ceph-mon[44992]: Deploying daemon mgr.smithi148.onhgvm on smithi148 2024-03-29T08:08:58.293 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:08:58 smithi148 ceph-mon[44992]: pgmap v28: 65 pgs: 65 active+clean; 275 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 31 KiB/s rd, 1.1 MiB/s wr, 320 op/s 2024-03-29T08:08:58.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:08:58 smithi037 ceph-mon[35162]: pgmap v28: 65 pgs: 65 active+clean; 275 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 31 KiB/s rd, 1.1 MiB/s wr, 320 op/s 2024-03-29T08:09:00.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:09:00 smithi148 ceph-mon[44992]: pgmap v29: 65 pgs: 65 active+clean; 273 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 46 KiB/s rd, 1.6 MiB/s wr, 487 op/s 2024-03-29T08:09:00.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:09:00 smithi037 ceph-mon[35162]: pgmap v29: 65 pgs: 65 active+clean; 273 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 46 KiB/s rd, 1.6 MiB/s wr, 487 op/s 2024-03-29T08:09:02.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:09:02 smithi148 ceph-mon[44992]: pgmap v30: 65 pgs: 65 active+clean; 273 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 30 KiB/s rd, 1.1 MiB/s wr, 324 op/s 2024-03-29T08:09:02.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:09:02 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' 2024-03-29T08:09:02.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:09:02 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' 2024-03-29T08:09:02.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:09:02 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T08:09:02.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:09:02 smithi037 ceph-mon[35162]: pgmap v30: 65 pgs: 65 active+clean; 273 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 30 KiB/s rd, 1.1 MiB/s wr, 324 op/s 2024-03-29T08:09:02.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:09:02 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' 2024-03-29T08:09:02.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:09:02 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' 2024-03-29T08:09:02.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:09:02 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T08:09:04.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:04.061+0000 7fcc6b000700 1 -- 172.21.15.37:0/2601148042 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcc64073170 msgr2=0x7fcc64073b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:09:04.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:04.061+0000 7fcc6b000700 1 --2- 172.21.15.37:0/2601148042 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcc64073170 0x7fcc64073b80 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fcc4c004940 tx=0x7fcc4c009e30 comp rx=0 tx=0).stop 2024-03-29T08:09:04.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:04.061+0000 7fcc6b000700 1 -- 172.21.15.37:0/2601148042 shutdown_connections 2024-03-29T08:09:04.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:04.061+0000 7fcc6b000700 1 --2- 172.21.15.37:0/2601148042 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcc64073170 0x7fcc64073b80 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:09:04.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:04.061+0000 7fcc6b000700 1 --2- 172.21.15.37:0/2601148042 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcc64072820 0x7fcc64072c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:09:04.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:04.061+0000 7fcc6b000700 1 -- 172.21.15.37:0/2601148042 >> 172.21.15.37:0/2601148042 conn(0x7fcc640fa950 msgr2=0x7fcc640fcd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:09:04.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:04.061+0000 7fcc6b000700 1 -- 172.21.15.37:0/2601148042 shutdown_connections 2024-03-29T08:09:04.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:04.061+0000 7fcc6b000700 1 -- 172.21.15.37:0/2601148042 wait complete. 2024-03-29T08:09:04.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:04.063+0000 7fcc6b000700 1 Processor -- start 2024-03-29T08:09:04.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:04.064+0000 7fcc6b000700 1 -- start start 2024-03-29T08:09:04.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:04.064+0000 7fcc6b000700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcc64072820 0x7fcc64109650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:09:04.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:04.064+0000 7fcc6b000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcc64109b90 0x7fcc6410a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:09:04.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:04.064+0000 7fcc6b000700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc6410b180 con 0x7fcc64109b90 2024-03-29T08:09:04.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:04.064+0000 7fcc6b000700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc641221c0 con 0x7fcc64072820 2024-03-29T08:09:04.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:04.065+0000 7fcc63e00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcc64072820 0x7fcc64109650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:09:04.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:04.065+0000 7fcc63400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcc64109b90 0x7fcc6410a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:09:04.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:04.065+0000 7fcc63e00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcc64072820 0x7fcc64109650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.37:38748/0 (socket says 172.21.15.37:38748) 2024-03-29T08:09:04.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:04.065+0000 7fcc63400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcc64109b90 0x7fcc6410a000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43762/0 (socket says 172.21.15.37:43762) 2024-03-29T08:09:04.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:04.065+0000 7fcc63e00700 1 -- 172.21.15.37:0/2331252114 learned_addr learned my addr 172.21.15.37:0/2331252114 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:09:04.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:04.065+0000 7fcc63e00700 1 -- 172.21.15.37:0/2331252114 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcc64109b90 msgr2=0x7fcc6410a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:09:04.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:04.065+0000 7fcc63e00700 1 --2- 172.21.15.37:0/2331252114 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcc64109b90 0x7fcc6410a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:09:04.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:04.065+0000 7fcc63e00700 1 -- 172.21.15.37:0/2331252114 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc4c005040 con 0x7fcc64072820 2024-03-29T08:09:04.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:04.065+0000 7fcc63e00700 1 --2- 172.21.15.37:0/2331252114 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcc64072820 0x7fcc64109650 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fcc5400eb40 tx=0x7fcc5400b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:09:04.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:04.066+0000 7fcc5be00700 1 -- 172.21.15.37:0/2331252114 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcc54007cc0 con 0x7fcc64072820 2024-03-29T08:09:04.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:04.066+0000 7fcc5be00700 1 -- 172.21.15.37:0/2331252114 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fcc5400f040 con 0x7fcc64072820 2024-03-29T08:09:04.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:04.066+0000 7fcc6b000700 1 -- 172.21.15.37:0/2331252114 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc64122360 con 0x7fcc64072820 2024-03-29T08:09:04.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:04.066+0000 7fcc5be00700 1 -- 172.21.15.37:0/2331252114 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcc5400c9f0 con 0x7fcc64072820 2024-03-29T08:09:04.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:04.066+0000 7fcc6b000700 1 -- 172.21.15.37:0/2331252114 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc641228f0 con 0x7fcc64072820 2024-03-29T08:09:04.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:04.068+0000 7fcc6b000700 1 -- 172.21.15.37:0/2331252114 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc64103480 con 0x7fcc64072820 2024-03-29T08:09:04.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:04.069+0000 7fcc5be00700 1 -- 172.21.15.37:0/2331252114 <== mon.1 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 32) v1 ==== 90805+0+0 (secure 0 0 0) 0x7fcc5400cb50 con 0x7fcc64072820 2024-03-29T08:09:04.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:04.070+0000 7fcc5be00700 1 --2- 172.21.15.37:0/2331252114 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7fcc4806c9c0 0x7fcc4806ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:09:04.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:04.070+0000 7fcc5be00700 1 -- 172.21.15.37:0/2331252114 <== mon.1 v2:172.21.15.148:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6066+0+0 (secure 0 0 0) 0x7fcc5408b850 con 0x7fcc64072820 2024-03-29T08:09:04.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:04.070+0000 7fcc63400700 1 --2- 172.21.15.37:0/2331252114 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7fcc4806c9c0 0x7fcc4806ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:09:04.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:04.072+0000 7fcc63400700 1 --2- 172.21.15.37:0/2331252114 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7fcc4806c9c0 0x7fcc4806ee70 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fcc6410a900 tx=0x7fcc4c009e30 comp rx=0 tx=0).ready entity=mgr.14686 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:09:04.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:04.075+0000 7fcc5be00700 1 -- 172.21.15.37:0/2331252114 <== mon.1 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181579 (secure 0 0 0) 0x7fcc54055af0 con 0x7fcc64072820 2024-03-29T08:09:04.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:04.312+0000 7fcc6b000700 1 -- 172.21.15.37:0/2331252114 --> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fcc64073170 con 0x7fcc4806c9c0 2024-03-29T08:09:04.374 INFO:teuthology.orchestra.run.smithi037.stdout:true 2024-03-29T08:09:04.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:04.314+0000 7fcc5be00700 1 -- 172.21.15.37:0/2331252114 <== mgr.14686 v2:172.21.15.37:6800/4188329516 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+321 (secure 0 0 0) 0x7fcc64073170 con 0x7fcc4806c9c0 2024-03-29T08:09:04.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:04.316+0000 7fcc6b000700 1 -- 172.21.15.37:0/2331252114 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7fcc4806c9c0 msgr2=0x7fcc4806ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:09:04.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:04.316+0000 7fcc6b000700 1 --2- 172.21.15.37:0/2331252114 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7fcc4806c9c0 0x7fcc4806ee70 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fcc6410a900 tx=0x7fcc4c009e30 comp rx=0 tx=0).stop 2024-03-29T08:09:04.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:04.316+0000 7fcc6b000700 1 -- 172.21.15.37:0/2331252114 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcc64072820 msgr2=0x7fcc64109650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:09:04.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:04.316+0000 7fcc6b000700 1 --2- 172.21.15.37:0/2331252114 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcc64072820 0x7fcc64109650 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fcc5400eb40 tx=0x7fcc5400b040 comp rx=0 tx=0).stop 2024-03-29T08:09:04.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:04.316+0000 7fcc6b000700 1 -- 172.21.15.37:0/2331252114 shutdown_connections 2024-03-29T08:09:04.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:04.316+0000 7fcc6b000700 1 --2- 172.21.15.37:0/2331252114 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7fcc4806c9c0 0x7fcc4806ee70 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:09:04.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:04.316+0000 7fcc6b000700 1 --2- 172.21.15.37:0/2331252114 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcc64072820 0x7fcc64109650 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:09:04.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:04.316+0000 7fcc6b000700 1 --2- 172.21.15.37:0/2331252114 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcc64109b90 0x7fcc6410a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:09:04.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:04.316+0000 7fcc6b000700 1 -- 172.21.15.37:0/2331252114 >> 172.21.15.37:0/2331252114 conn(0x7fcc640fa950 msgr2=0x7fcc640fd020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:09:04.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:04.316+0000 7fcc6b000700 1 -- 172.21.15.37:0/2331252114 shutdown_connections 2024-03-29T08:09:04.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:04.316+0000 7fcc6b000700 1 -- 172.21.15.37:0/2331252114 wait complete. 2024-03-29T08:09:04.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:09:04 smithi037 ceph-mon[35162]: pgmap v31: 65 pgs: 65 active+clean; 273 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 30 KiB/s rd, 1.3 MiB/s wr, 366 op/s 2024-03-29T08:09:04.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:04.512+0000 7f33d8400700 1 -- 172.21.15.37:0/3755396568 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f33d00fbcd0 msgr2=0x7f33d00fc0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:09:04.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:04.512+0000 7f33d8400700 1 --2- 172.21.15.37:0/3755396568 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f33d00fbcd0 0x7f33d00fc0e0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f33c4004940 tx=0x7f33c4009e30 comp rx=0 tx=0).stop 2024-03-29T08:09:04.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:04.512+0000 7f33d8400700 1 -- 172.21.15.37:0/3755396568 shutdown_connections 2024-03-29T08:09:04.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:04.512+0000 7f33d8400700 1 --2- 172.21.15.37:0/3755396568 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f33d00fcf80 0x7f33d00fd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:09:04.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:04.512+0000 7f33d8400700 1 --2- 172.21.15.37:0/3755396568 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f33d00fbcd0 0x7f33d00fc0e0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:09:04.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:04.512+0000 7f33d8400700 1 -- 172.21.15.37:0/3755396568 >> 172.21.15.37:0/3755396568 conn(0x7f33d00f7430 msgr2=0x7f33d00f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:09:04.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:04.512+0000 7f33d8400700 1 -- 172.21.15.37:0/3755396568 shutdown_connections 2024-03-29T08:09:04.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:04.512+0000 7f33d8400700 1 -- 172.21.15.37:0/3755396568 wait complete. 2024-03-29T08:09:04.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:04.513+0000 7f33d8400700 1 Processor -- start 2024-03-29T08:09:04.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:04.514+0000 7f33d8400700 1 -- start start 2024-03-29T08:09:04.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:04.516+0000 7f33d8400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f33d00fbcd0 0x7f33d010f950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:09:04.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:04.516+0000 7f33d8400700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f33d00fcf80 0x7f33d010fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:09:04.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:04.516+0000 7f33d8400700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33d0110490 con 0x7f33d00fbcd0 2024-03-29T08:09:04.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:04.516+0000 7f33d8400700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33d0110c40 con 0x7f33d00fcf80 2024-03-29T08:09:04.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:04.517+0000 7f33d5000700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f33d00fcf80 0x7f33d010fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:09:04.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:04.517+0000 7f33d5000700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f33d00fcf80 0x7f33d010fe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.37:38766/0 (socket says 172.21.15.37:38766) 2024-03-29T08:09:04.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:04.517+0000 7f33d5000700 1 -- 172.21.15.37:0/3530628164 learned_addr learned my addr 172.21.15.37:0/3530628164 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:09:04.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:04.517+0000 7f33d5a00700 1 --2- 172.21.15.37:0/3530628164 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f33d00fbcd0 0x7f33d010f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:09:04.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:04.518+0000 7f33d5000700 1 -- 172.21.15.37:0/3530628164 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f33d00fbcd0 msgr2=0x7f33d010f950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:09:04.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:04.518+0000 7f33d5000700 1 --2- 172.21.15.37:0/3530628164 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f33d00fbcd0 0x7f33d010f950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:09:04.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:04.518+0000 7f33d5000700 1 -- 172.21.15.37:0/3530628164 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33c4005040 con 0x7f33d00fcf80 2024-03-29T08:09:04.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:04.518+0000 7f33d5000700 1 --2- 172.21.15.37:0/3530628164 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f33d00fcf80 0x7f33d010fe90 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f33c000eb40 tx=0x7f33c000b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:09:04.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:04.519+0000 7f33cd600700 1 -- 172.21.15.37:0/3530628164 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f33c0007cc0 con 0x7f33d00fcf80 2024-03-29T08:09:04.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:04.519+0000 7f33cd600700 1 -- 172.21.15.37:0/3530628164 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f33c000f040 con 0x7f33d00fcf80 2024-03-29T08:09:04.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:04.519+0000 7f33d8400700 1 -- 172.21.15.37:0/3530628164 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33d0110ec0 con 0x7f33d00fcf80 2024-03-29T08:09:04.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:04.519+0000 7f33cd600700 1 -- 172.21.15.37:0/3530628164 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f33c0009570 con 0x7f33d00fcf80 2024-03-29T08:09:04.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:04.519+0000 7f33d8400700 1 -- 172.21.15.37:0/3530628164 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33d0111400 con 0x7f33d00fcf80 2024-03-29T08:09:04.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:04.521+0000 7f33d8400700 1 -- 172.21.15.37:0/3530628164 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33b40052e0 con 0x7f33d00fcf80 2024-03-29T08:09:04.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:04.522+0000 7f33cd600700 1 -- 172.21.15.37:0/3530628164 <== mon.1 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 32) v1 ==== 90805+0+0 (secure 0 0 0) 0x7f33c00096d0 con 0x7f33d00fcf80 2024-03-29T08:09:04.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:04.523+0000 7f33cd600700 1 --2- 172.21.15.37:0/3530628164 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7f33b806c9c0 0x7f33b806ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:09:04.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:04.523+0000 7f33cd600700 1 -- 172.21.15.37:0/3530628164 <== mon.1 v2:172.21.15.148:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f33c008b890 con 0x7f33d00fcf80 2024-03-29T08:09:04.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:04.524+0000 7f33d5a00700 1 --2- 172.21.15.37:0/3530628164 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7f33b806c9c0 0x7f33b806ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:09:04.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:04.525+0000 7f33d5a00700 1 --2- 172.21.15.37:0/3530628164 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7f33b806c9c0 0x7f33b806ee70 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f33c400b0d0 tx=0x7f33c4009e30 comp rx=0 tx=0).ready entity=mgr.14686 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:09:04.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:04.533+0000 7f33cd600700 1 -- 172.21.15.37:0/3530628164 <== mon.1 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181579 (secure 0 0 0) 0x7f33c0059710 con 0x7f33d00fcf80 2024-03-29T08:09:04.547 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:09:04 smithi148 ceph-mon[44992]: pgmap v31: 65 pgs: 65 active+clean; 273 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 30 KiB/s rd, 1.3 MiB/s wr, 366 op/s 2024-03-29T08:09:04.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:04.923+0000 7f33d8400700 1 -- 172.21.15.37:0/3530628164 --> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f33b4000bf0 con 0x7f33b806c9c0 2024-03-29T08:09:04.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:04.925+0000 7f33cd600700 1 -- 172.21.15.37:0/3530628164 <== mgr.14686 v2:172.21.15.37:6800/4188329516 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+321 (secure 0 0 0) 0x7f33b4000bf0 con 0x7f33b806c9c0 2024-03-29T08:09:04.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:04.928+0000 7f33d8400700 1 -- 172.21.15.37:0/3530628164 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7f33b806c9c0 msgr2=0x7f33b806ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:09:04.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:04.928+0000 7f33d8400700 1 --2- 172.21.15.37:0/3530628164 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7f33b806c9c0 0x7f33b806ee70 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f33c400b0d0 tx=0x7f33c4009e30 comp rx=0 tx=0).stop 2024-03-29T08:09:04.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:04.928+0000 7f33d8400700 1 -- 172.21.15.37:0/3530628164 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f33d00fcf80 msgr2=0x7f33d010fe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:09:04.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:04.928+0000 7f33d8400700 1 --2- 172.21.15.37:0/3530628164 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f33d00fcf80 0x7f33d010fe90 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f33c000eb40 tx=0x7f33c000b040 comp rx=0 tx=0).stop 2024-03-29T08:09:04.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:04.928+0000 7f33d8400700 1 -- 172.21.15.37:0/3530628164 shutdown_connections 2024-03-29T08:09:04.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:04.928+0000 7f33d8400700 1 --2- 172.21.15.37:0/3530628164 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7f33b806c9c0 0x7f33b806ee70 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:09:04.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:04.928+0000 7f33d8400700 1 --2- 172.21.15.37:0/3530628164 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f33d00fbcd0 0x7f33d010f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:09:04.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:04.928+0000 7f33d8400700 1 --2- 172.21.15.37:0/3530628164 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f33d00fcf80 0x7f33d010fe90 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:09:04.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:04.928+0000 7f33d8400700 1 -- 172.21.15.37:0/3530628164 >> 172.21.15.37:0/3530628164 conn(0x7f33d00f7430 msgr2=0x7f33d00f8f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:09:04.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:04.929+0000 7f33d8400700 1 -- 172.21.15.37:0/3530628164 shutdown_connections 2024-03-29T08:09:04.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:04.929+0000 7f33d8400700 1 -- 172.21.15.37:0/3530628164 wait complete. 2024-03-29T08:09:05.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:05.138+0000 7feb1f200700 1 -- 172.21.15.37:0/1699917945 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feb2006fc00 msgr2=0x7feb2006e2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:09:05.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:05.138+0000 7feb1f200700 1 --2- 172.21.15.37:0/1699917945 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feb2006fc00 0x7feb2006e2a0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7feb10004940 tx=0x7feb10009e30 comp rx=0 tx=0).stop 2024-03-29T08:09:05.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:05.138+0000 7feb1f200700 1 -- 172.21.15.37:0/1699917945 shutdown_connections 2024-03-29T08:09:05.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:05.138+0000 7feb1f200700 1 --2- 172.21.15.37:0/1699917945 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feb2006e7e0 0x7feb2006ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:09:05.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:05.138+0000 7feb1f200700 1 --2- 172.21.15.37:0/1699917945 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feb2006fc00 0x7feb2006e2a0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:09:05.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:05.138+0000 7feb1f200700 1 -- 172.21.15.37:0/1699917945 >> 172.21.15.37:0/1699917945 conn(0x7feb20069c10 msgr2=0x7feb2006c040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:09:05.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:05.138+0000 7feb1f200700 1 -- 172.21.15.37:0/1699917945 shutdown_connections 2024-03-29T08:09:05.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:05.138+0000 7feb1f200700 1 -- 172.21.15.37:0/1699917945 wait complete. 2024-03-29T08:09:05.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:05.139+0000 7feb1f200700 1 Processor -- start 2024-03-29T08:09:05.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:05.139+0000 7feb1f200700 1 -- start start 2024-03-29T08:09:05.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:05.139+0000 7feb1f200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feb2006e7e0 0x7feb200704d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:09:05.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:05.139+0000 7feb1f200700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feb2006fc00 0x7feb20070a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:09:05.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:05.139+0000 7feb1f200700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb20071060 con 0x7feb2006e7e0 2024-03-29T08:09:05.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:05.139+0000 7feb1f200700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb20071470 con 0x7feb2006fc00 2024-03-29T08:09:05.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:05.140+0000 7feb1de00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feb2006e7e0 0x7feb200704d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:09:05.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:05.140+0000 7feb1d400700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feb2006fc00 0x7feb20070a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:09:05.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:05.140+0000 7feb1d400700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feb2006fc00 0x7feb20070a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.37:38784/0 (socket says 172.21.15.37:38784) 2024-03-29T08:09:05.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:05.140+0000 7feb1de00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feb2006e7e0 0x7feb200704d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43798/0 (socket says 172.21.15.37:43798) 2024-03-29T08:09:05.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:05.140+0000 7feb1d400700 1 -- 172.21.15.37:0/4253715331 learned_addr learned my addr 172.21.15.37:0/4253715331 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:09:05.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:05.140+0000 7feb1d400700 1 -- 172.21.15.37:0/4253715331 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feb2006e7e0 msgr2=0x7feb200704d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:09:05.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:05.140+0000 7feb1d400700 1 --2- 172.21.15.37:0/4253715331 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feb2006e7e0 0x7feb200704d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:09:05.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:05.140+0000 7feb1d400700 1 -- 172.21.15.37:0/4253715331 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb10005040 con 0x7feb2006fc00 2024-03-29T08:09:05.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:05.140+0000 7feb1d400700 1 --2- 172.21.15.37:0/4253715331 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feb2006fc00 0x7feb20070a10 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7feb0c00eb40 tx=0x7feb0c00b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:09:05.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:05.141+0000 7feb16000700 1 -- 172.21.15.37:0/4253715331 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feb0c007cc0 con 0x7feb2006fc00 2024-03-29T08:09:05.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:05.141+0000 7feb1f200700 1 -- 172.21.15.37:0/4253715331 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb200716f0 con 0x7feb2006fc00 2024-03-29T08:09:05.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:05.141+0000 7feb1f200700 1 -- 172.21.15.37:0/4253715331 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb20071cb0 con 0x7feb2006fc00 2024-03-29T08:09:05.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:05.141+0000 7feb16000700 1 -- 172.21.15.37:0/4253715331 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7feb0c00f040 con 0x7feb2006fc00 2024-03-29T08:09:05.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:05.141+0000 7feb16000700 1 -- 172.21.15.37:0/4253715331 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feb0c009570 con 0x7feb2006fc00 2024-03-29T08:09:05.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:05.142+0000 7feb1f200700 1 -- 172.21.15.37:0/4253715331 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb000052e0 con 0x7feb2006fc00 2024-03-29T08:09:05.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:05.144+0000 7feb16000700 1 -- 172.21.15.37:0/4253715331 <== mon.1 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 32) v1 ==== 90805+0+0 (secure 0 0 0) 0x7feb0c0096d0 con 0x7feb2006fc00 2024-03-29T08:09:05.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:05.144+0000 7feb16000700 1 --2- 172.21.15.37:0/4253715331 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7feb0406c9d0 0x7feb0406ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:09:05.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:05.145+0000 7feb16000700 1 -- 172.21.15.37:0/4253715331 <== mon.1 v2:172.21.15.148:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6066+0+0 (secure 0 0 0) 0x7feb0c08b910 con 0x7feb2006fc00 2024-03-29T08:09:05.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:05.145+0000 7feb1de00700 1 --2- 172.21.15.37:0/4253715331 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7feb0406c9d0 0x7feb0406ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:09:05.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:05.146+0000 7feb1de00700 1 --2- 172.21.15.37:0/4253715331 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7feb0406c9d0 0x7feb0406ee80 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7feb100137d0 tx=0x7feb10009e30 comp rx=0 tx=0).ready entity=mgr.14686 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:09:05.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:05.152+0000 7feb16000700 1 -- 172.21.15.37:0/4253715331 <== mon.1 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181579 (secure 0 0 0) 0x7feb0c055ce0 con 0x7feb2006fc00 2024-03-29T08:09:05.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:05.546+0000 7feb1f200700 1 -- 172.21.15.37:0/4253715331 --> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7feb00000bf0 con 0x7feb0406c9d0 2024-03-29T08:09:05.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:05.562+0000 7feb16000700 1 -- 172.21.15.37:0/4253715331 <== mgr.14686 v2:172.21.15.37:6800/4188329516 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7feb00000bf0 con 0x7feb0406c9d0 2024-03-29T08:09:05.562 INFO:teuthology.orchestra.run.smithi037.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-29T08:09:05.562 INFO:teuthology.orchestra.run.smithi037.stdout:alertmanager.smithi037 smithi037 *:9093,9094 running (2m) 36s ago 11m 14.9M - 0.25.0 c8568f914cd2 8a8d40a6e26e 2024-03-29T08:09:05.563 INFO:teuthology.orchestra.run.smithi037.stdout:crash.smithi037 smithi037 running (12m) 36s ago 12m 6656k - 17.2.7-789-g4213e17c cf22eb8683be e0343935b25e 2024-03-29T08:09:05.563 INFO:teuthology.orchestra.run.smithi037.stdout:crash.smithi148 smithi148 running (10m) 53s ago 10m 6656k - 17.2.7-789-g4213e17c cf22eb8683be ff5164c552c1 2024-03-29T08:09:05.563 INFO:teuthology.orchestra.run.smithi037.stdout:grafana.smithi037 smithi037 *:3000 running (2m) 36s ago 11m 74.0M - 9.4.7 954c08fa6188 a857385eee0c 2024-03-29T08:09:05.563 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi037.iifsmh smithi037 running (5m) 36s ago 5m 142M - 17.2.7-789-g4213e17c cf22eb8683be 074e1acf331f 2024-03-29T08:09:05.563 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi037.mmwgxe smithi037 running (5m) 36s ago 5m 148M - 17.2.7-789-g4213e17c cf22eb8683be 9a5b1e72de5c 2024-03-29T08:09:05.563 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi148.qukpjh smithi148 running (5m) 53s ago 5m 12.4M - 17.2.7-789-g4213e17c cf22eb8683be 8e469782a273 2024-03-29T08:09:05.563 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi148.sqteih smithi148 running (5m) 53s ago 5m 12.3M - 17.2.7-789-g4213e17c cf22eb8683be 8e68f858bb22 2024-03-29T08:09:05.563 INFO:teuthology.orchestra.run.smithi037.stdout:mgr.smithi037.wjikii smithi037 *:8443,9283,8765 running (76s) 36s ago 13m 473M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff c5a1417b9417 2024-03-29T08:09:05.563 INFO:teuthology.orchestra.run.smithi037.stdout:mgr.smithi148.onhgvm smithi148 *:8443,9283,8765 starting - - - - 2024-03-29T08:09:05.563 INFO:teuthology.orchestra.run.smithi037.stdout:mon.smithi037 smithi037 running (13m) 36s ago 13m 52.3M 2048M 17.2.7-789-g4213e17c cf22eb8683be bb2101505730 2024-03-29T08:09:05.563 INFO:teuthology.orchestra.run.smithi037.stdout:mon.smithi148 smithi148 running (10m) 53s ago 10m 42.1M 2048M 17.2.7-789-g4213e17c cf22eb8683be 18839ed2952a 2024-03-29T08:09:05.563 INFO:teuthology.orchestra.run.smithi037.stdout:node-exporter.smithi037 smithi037 *:9100 running (2m) 36s ago 12m 7675k - 1.5.0 0da6a335fe13 cdaa34126e04 2024-03-29T08:09:05.563 INFO:teuthology.orchestra.run.smithi037.stdout:node-exporter.smithi148 smithi148 *:9100 running (104s) 53s ago 10m 7277k - 1.5.0 0da6a335fe13 0d793b5bd838 2024-03-29T08:09:05.563 INFO:teuthology.orchestra.run.smithi037.stdout:osd.0 smithi037 running (9m) 36s ago 9m 189M 1619M 17.2.7-789-g4213e17c cf22eb8683be 559fccd0ebe1 2024-03-29T08:09:05.563 INFO:teuthology.orchestra.run.smithi037.stdout:osd.1 smithi037 running (9m) 36s ago 9m 208M 1619M 17.2.7-789-g4213e17c cf22eb8683be 5582cff11b7a 2024-03-29T08:09:05.564 INFO:teuthology.orchestra.run.smithi037.stdout:osd.2 smithi037 running (8m) 36s ago 8m 172M 1619M 17.2.7-789-g4213e17c cf22eb8683be f809bfab13e5 2024-03-29T08:09:05.564 INFO:teuthology.orchestra.run.smithi037.stdout:osd.3 smithi148 running (8m) 53s ago 8m 231M 2642M 17.2.7-789-g4213e17c cf22eb8683be a0a8224bb4b0 2024-03-29T08:09:05.564 INFO:teuthology.orchestra.run.smithi037.stdout:osd.4 smithi148 running (7m) 53s ago 7m 200M 2642M 17.2.7-789-g4213e17c cf22eb8683be a00b69047ed7 2024-03-29T08:09:05.564 INFO:teuthology.orchestra.run.smithi037.stdout:osd.5 smithi148 running (7m) 53s ago 7m 213M 2642M 17.2.7-789-g4213e17c cf22eb8683be 044a3f7db7f8 2024-03-29T08:09:05.564 INFO:teuthology.orchestra.run.smithi037.stdout:prometheus.smithi037 smithi037 *:9095 starting - - - - 2024-03-29T08:09:05.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:05.567+0000 7feb1f200700 1 -- 172.21.15.37:0/4253715331 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7feb0406c9d0 msgr2=0x7feb0406ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:09:05.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:05.567+0000 7feb1f200700 1 --2- 172.21.15.37:0/4253715331 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7feb0406c9d0 0x7feb0406ee80 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7feb100137d0 tx=0x7feb10009e30 comp rx=0 tx=0).stop 2024-03-29T08:09:05.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:05.567+0000 7feb1f200700 1 -- 172.21.15.37:0/4253715331 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feb2006fc00 msgr2=0x7feb20070a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:09:05.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:05.567+0000 7feb1f200700 1 --2- 172.21.15.37:0/4253715331 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feb2006fc00 0x7feb20070a10 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7feb0c00eb40 tx=0x7feb0c00b040 comp rx=0 tx=0).stop 2024-03-29T08:09:05.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:05.567+0000 7feb1f200700 1 -- 172.21.15.37:0/4253715331 shutdown_connections 2024-03-29T08:09:05.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:05.567+0000 7feb1f200700 1 --2- 172.21.15.37:0/4253715331 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7feb0406c9d0 0x7feb0406ee80 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:09:05.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:05.568+0000 7feb1f200700 1 --2- 172.21.15.37:0/4253715331 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feb2006e7e0 0x7feb200704d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:09:05.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:05.568+0000 7feb1f200700 1 --2- 172.21.15.37:0/4253715331 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feb2006fc00 0x7feb20070a10 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:09:05.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:05.568+0000 7feb1f200700 1 -- 172.21.15.37:0/4253715331 >> 172.21.15.37:0/4253715331 conn(0x7feb20069c10 msgr2=0x7feb2006b350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:09:05.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:05.568+0000 7feb1f200700 1 -- 172.21.15.37:0/4253715331 shutdown_connections 2024-03-29T08:09:05.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:05.568+0000 7feb1f200700 1 -- 172.21.15.37:0/4253715331 wait complete. 2024-03-29T08:09:05.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:05.789+0000 7fa455e00700 1 -- 172.21.15.37:0/1052943834 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa450070890 msgr2=0x7fa450070d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:09:05.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:05.789+0000 7fa455e00700 1 --2- 172.21.15.37:0/1052943834 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa450070890 0x7fa450070d00 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fa43c0048b0 tx=0x7fa43c009e30 comp rx=0 tx=0).stop 2024-03-29T08:09:05.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:05.789+0000 7fa455e00700 1 -- 172.21.15.37:0/1052943834 shutdown_connections 2024-03-29T08:09:05.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:05.789+0000 7fa455e00700 1 --2- 172.21.15.37:0/1052943834 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa450070890 0x7fa450070d00 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:09:05.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:05.789+0000 7fa455e00700 1 --2- 172.21.15.37:0/1052943834 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa450071e40 0x7fa450070350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:09:05.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:05.789+0000 7fa455e00700 1 -- 172.21.15.37:0/1052943834 >> 172.21.15.37:0/1052943834 conn(0x7fa4500f8540 msgr2=0x7fa4500fa970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:09:05.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:05.789+0000 7fa455e00700 1 -- 172.21.15.37:0/1052943834 shutdown_connections 2024-03-29T08:09:05.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:05.790+0000 7fa455e00700 1 -- 172.21.15.37:0/1052943834 wait complete. 2024-03-29T08:09:05.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:05.791+0000 7fa455e00700 1 Processor -- start 2024-03-29T08:09:05.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:05.791+0000 7fa455e00700 1 -- start start 2024-03-29T08:09:05.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:05.791+0000 7fa455e00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa450070890 0x7fa45010fb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:09:05.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:05.791+0000 7fa455e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa450071e40 0x7fa450110050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:09:05.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:05.792+0000 7fa455e00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa450110650 con 0x7fa450071e40 2024-03-29T08:09:05.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:05.792+0000 7fa455e00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa450110ea0 con 0x7fa450070890 2024-03-29T08:09:05.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:05.792+0000 7fa44ea00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa450070890 0x7fa45010fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:09:05.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:05.792+0000 7fa44e000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa450071e40 0x7fa450110050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:09:05.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:05.792+0000 7fa44ea00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa450070890 0x7fa45010fb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.37:38810/0 (socket says 172.21.15.37:38810) 2024-03-29T08:09:05.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:05.792+0000 7fa44ea00700 1 -- 172.21.15.37:0/2993064512 learned_addr learned my addr 172.21.15.37:0/2993064512 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:09:05.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:05.792+0000 7fa44ea00700 1 -- 172.21.15.37:0/2993064512 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa450071e40 msgr2=0x7fa450110050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:09:05.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:05.792+0000 7fa44ea00700 1 --2- 172.21.15.37:0/2993064512 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa450071e40 0x7fa450110050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:09:05.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:05.792+0000 7fa44ea00700 1 -- 172.21.15.37:0/2993064512 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa43c005040 con 0x7fa450070890 2024-03-29T08:09:05.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:05.793+0000 7fa44ea00700 1 --2- 172.21.15.37:0/2993064512 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa450070890 0x7fa45010fb10 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fa440002a80 tx=0x7fa440004630 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:09:05.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:05.793+0000 7fa446a00700 1 -- 172.21.15.37:0/2993064512 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa440007cc0 con 0x7fa450070890 2024-03-29T08:09:05.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:05.793+0000 7fa446a00700 1 -- 172.21.15.37:0/2993064512 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fa44000f040 con 0x7fa450070890 2024-03-29T08:09:05.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:05.793+0000 7fa446a00700 1 -- 172.21.15.37:0/2993064512 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa44000d9e0 con 0x7fa450070890 2024-03-29T08:09:05.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:05.794+0000 7fa455e00700 1 -- 172.21.15.37:0/2993064512 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa450111180 con 0x7fa450070890 2024-03-29T08:09:05.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:05.794+0000 7fa455e00700 1 -- 172.21.15.37:0/2993064512 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa450111640 con 0x7fa450070890 2024-03-29T08:09:05.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:05.796+0000 7fa455e00700 1 -- 172.21.15.37:0/2993064512 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa45005f5f0 con 0x7fa450070890 2024-03-29T08:09:05.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:05.798+0000 7fa446a00700 1 -- 172.21.15.37:0/2993064512 <== mon.1 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 32) v1 ==== 90805+0+0 (secure 0 0 0) 0x7fa44000db40 con 0x7fa450070890 2024-03-29T08:09:05.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:05.798+0000 7fa446a00700 1 --2- 172.21.15.37:0/2993064512 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7fa43406ca10 0x7fa43406eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:09:05.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:05.799+0000 7fa446a00700 1 -- 172.21.15.37:0/2993064512 <== mon.1 v2:172.21.15.148:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6066+0+0 (secure 0 0 0) 0x7fa44008b550 con 0x7fa450070890 2024-03-29T08:09:05.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:05.799+0000 7fa44e000700 1 --2- 172.21.15.37:0/2993064512 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7fa43406ca10 0x7fa43406eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:09:05.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:05.800+0000 7fa44e000700 1 --2- 172.21.15.37:0/2993064512 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7fa43406ca10 0x7fa43406eec0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fa43c00b090 tx=0x7fa43c009e30 comp rx=0 tx=0).ready entity=mgr.14686 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:09:05.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:05.803+0000 7fa446a00700 1 -- 172.21.15.37:0/2993064512 <== mon.1 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181579 (secure 0 0 0) 0x7fa440059200 con 0x7fa450070890 2024-03-29T08:09:06.232 INFO:tasks.workunit:Stopping ['suites/fsstress.sh'] on client.1... 2024-03-29T08:09:06.232 DEBUG:teuthology.orchestra.run.smithi148:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.1 /home/ubuntu/cephtest/clone.client.1 2024-03-29T08:09:06.263 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:09:06 smithi148 ceph-mon[44992]: from='client.24575 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:09:06.263 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:09:06 smithi148 ceph-mon[44992]: pgmap v32: 65 pgs: 65 active+clean; 271 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 45 KiB/s rd, 1.7 MiB/s wr, 458 op/s 2024-03-29T08:09:06.263 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:09:06 smithi148 ceph-mon[44992]: from='client.24579 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:09:06.263 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:09:06 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' 2024-03-29T08:09:06.263 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:09:06 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' 2024-03-29T08:09:06.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:06.289+0000 7fa455e00700 1 -- 172.21.15.37:0/2993064512 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fa4501079a0 con 0x7fa450070890 2024-03-29T08:09:06.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:06.290+0000 7fa446a00700 1 -- 172.21.15.37:0/2993064512 <== mon.1 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+743 (secure 0 0 0) 0x7fa440058d90 con 0x7fa450070890 2024-03-29T08:09:06.291 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-29T08:09:06.291 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": { 2024-03-29T08:09:06.291 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 17.2.7-789-g4213e17c (4213e17c4de67016ed14f55d22503f4923b3a901) quincy (stable)": 2 2024-03-29T08:09:06.291 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:09:06.291 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": { 2024-03-29T08:09:06.291 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 2 2024-03-29T08:09:06.291 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:09:06.291 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": { 2024-03-29T08:09:06.291 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 17.2.7-789-g4213e17c (4213e17c4de67016ed14f55d22503f4923b3a901) quincy (stable)": 6 2024-03-29T08:09:06.292 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:09:06.292 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": { 2024-03-29T08:09:06.292 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 17.2.7-789-g4213e17c (4213e17c4de67016ed14f55d22503f4923b3a901) quincy (stable)": 4 2024-03-29T08:09:06.292 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:09:06.292 INFO:teuthology.orchestra.run.smithi037.stdout: "overall": { 2024-03-29T08:09:06.292 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 17.2.7-789-g4213e17c (4213e17c4de67016ed14f55d22503f4923b3a901) quincy (stable)": 12, 2024-03-29T08:09:06.292 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 2 2024-03-29T08:09:06.292 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-29T08:09:06.292 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-29T08:09:06.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:06.293+0000 7fa455e00700 1 -- 172.21.15.37:0/2993064512 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7fa43406ca10 msgr2=0x7fa43406eec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:09:06.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:06.293+0000 7fa455e00700 1 --2- 172.21.15.37:0/2993064512 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7fa43406ca10 0x7fa43406eec0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fa43c00b090 tx=0x7fa43c009e30 comp rx=0 tx=0).stop 2024-03-29T08:09:06.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:06.293+0000 7fa455e00700 1 -- 172.21.15.37:0/2993064512 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa450070890 msgr2=0x7fa45010fb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:09:06.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:06.293+0000 7fa455e00700 1 --2- 172.21.15.37:0/2993064512 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa450070890 0x7fa45010fb10 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fa440002a80 tx=0x7fa440004630 comp rx=0 tx=0).stop 2024-03-29T08:09:06.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:06.294+0000 7fa455e00700 1 -- 172.21.15.37:0/2993064512 shutdown_connections 2024-03-29T08:09:06.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:06.294+0000 7fa455e00700 1 --2- 172.21.15.37:0/2993064512 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7fa43406ca10 0x7fa43406eec0 secure :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fa43c00b090 tx=0x7fa43c009e30 comp rx=0 tx=0).stop 2024-03-29T08:09:06.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:06.294+0000 7fa455e00700 1 --2- 172.21.15.37:0/2993064512 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa450070890 0x7fa45010fb10 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:09:06.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:06.294+0000 7fa455e00700 1 --2- 172.21.15.37:0/2993064512 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa450071e40 0x7fa450110050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:09:06.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:06.294+0000 7fa455e00700 1 -- 172.21.15.37:0/2993064512 >> 172.21.15.37:0/2993064512 conn(0x7fa4500f8540 msgr2=0x7fa4500fa020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:09:06.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:06.295+0000 7fa455e00700 1 -- 172.21.15.37:0/2993064512 shutdown_connections 2024-03-29T08:09:06.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:06.295+0000 7fa455e00700 1 -- 172.21.15.37:0/2993064512 wait complete. 2024-03-29T08:09:06.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:09:06 smithi037 ceph-mon[35162]: from='client.24575 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:09:06.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:09:06 smithi037 ceph-mon[35162]: pgmap v32: 65 pgs: 65 active+clean; 271 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 45 KiB/s rd, 1.7 MiB/s wr, 458 op/s 2024-03-29T08:09:06.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:09:06 smithi037 ceph-mon[35162]: from='client.24579 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:09:06.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:09:06 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' 2024-03-29T08:09:06.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:09:06 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' 2024-03-29T08:09:06.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:06.518+0000 7f71f1000700 1 -- 172.21.15.37:0/1475896467 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f71e4007090 msgr2=0x7f71e4007500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:09:06.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:06.518+0000 7f71f1000700 1 --2- 172.21.15.37:0/1475896467 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f71e4007090 0x7f71e4007500 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f71d8004940 tx=0x7f71d8009e30 comp rx=0 tx=0).stop 2024-03-29T08:09:06.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:06.518+0000 7f71f1000700 1 -- 172.21.15.37:0/1475896467 shutdown_connections 2024-03-29T08:09:06.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:06.518+0000 7f71f1000700 1 --2- 172.21.15.37:0/1475896467 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f71e4007090 0x7f71e4007500 secure :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f71d8004940 tx=0x7f71d8009e30 comp rx=0 tx=0).stop 2024-03-29T08:09:06.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:06.518+0000 7f71f1000700 1 --2- 172.21.15.37:0/1475896467 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f71e4008640 0x7f71e4006b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:09:06.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:06.518+0000 7f71f1000700 1 -- 172.21.15.37:0/1475896467 >> 172.21.15.37:0/1475896467 conn(0x7f71e4090350 msgr2=0x7f71e4092760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:09:06.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:06.519+0000 7f71f1000700 1 -- 172.21.15.37:0/1475896467 shutdown_connections 2024-03-29T08:09:06.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:06.519+0000 7f71f1000700 1 -- 172.21.15.37:0/1475896467 wait complete. 2024-03-29T08:09:06.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:06.520+0000 7f71f1000700 1 Processor -- start 2024-03-29T08:09:06.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:06.520+0000 7f71f1000700 1 -- start start 2024-03-29T08:09:06.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:06.521+0000 7f71f1000700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f71e4008640 0x7f71e40abdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:09:06.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:06.521+0000 7f71f1000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f71e40ac2f0 0x7f71e40b1290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:09:06.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:06.521+0000 7f71f1000700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71e40ac7b0 con 0x7f71e40ac2f0 2024-03-29T08:09:06.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:06.521+0000 7f71f1000700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71e40ac920 con 0x7f71e4008640 2024-03-29T08:09:06.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:06.521+0000 7f71eaa00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f71e40ac2f0 0x7f71e40b1290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:09:06.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:06.521+0000 7f71eaa00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f71e40ac2f0 0x7f71e40b1290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43846/0 (socket says 172.21.15.37:43846) 2024-03-29T08:09:06.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:06.521+0000 7f71eaa00700 1 -- 172.21.15.37:0/1971374977 learned_addr learned my addr 172.21.15.37:0/1971374977 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:09:06.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:06.521+0000 7f71eb400700 1 --2- 172.21.15.37:0/1971374977 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f71e4008640 0x7f71e40abdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:09:06.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:06.522+0000 7f71eaa00700 1 -- 172.21.15.37:0/1971374977 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f71e4008640 msgr2=0x7f71e40abdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:09:06.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:06.522+0000 7f71eaa00700 1 --2- 172.21.15.37:0/1971374977 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f71e4008640 0x7f71e40abdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:09:06.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:06.522+0000 7f71eaa00700 1 -- 172.21.15.37:0/1971374977 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71d8005040 con 0x7f71e40ac2f0 2024-03-29T08:09:06.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:06.522+0000 7f71eaa00700 1 --2- 172.21.15.37:0/1971374977 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f71e40ac2f0 0x7f71e40b1290 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f71d8004890 tx=0x7f71d8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:09:06.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:06.524+0000 7f71df400700 1 -- 172.21.15.37:0/1971374977 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f71d8020070 con 0x7f71e40ac2f0 2024-03-29T08:09:06.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:06.524+0000 7f71f1000700 1 -- 172.21.15.37:0/1971374977 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71e40b17d0 con 0x7f71e40ac2f0 2024-03-29T08:09:06.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:06.525+0000 7f71f1000700 1 -- 172.21.15.37:0/1971374977 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71e40b1d00 con 0x7f71e40ac2f0 2024-03-29T08:09:06.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:06.525+0000 7f71df400700 1 -- 172.21.15.37:0/1971374977 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f71d8013d20 con 0x7f71e40ac2f0 2024-03-29T08:09:06.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:06.525+0000 7f71df400700 1 -- 172.21.15.37:0/1971374977 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f71d800e070 con 0x7f71e40ac2f0 2024-03-29T08:09:06.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:06.528+0000 7f71f1000700 1 -- 172.21.15.37:0/1971374977 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71cc0052e0 con 0x7f71e40ac2f0 2024-03-29T08:09:06.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:06.528+0000 7f71df400700 1 -- 172.21.15.37:0/1971374977 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 32) v1 ==== 90805+0+0 (secure 0 0 0) 0x7f71d801c400 con 0x7f71e40ac2f0 2024-03-29T08:09:06.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:06.530+0000 7f71df400700 1 --2- 172.21.15.37:0/1971374977 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7f71d406ca10 0x7f71d406eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:09:06.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:06.530+0000 7f71df400700 1 -- 172.21.15.37:0/1971374977 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f71d808cc10 con 0x7f71e40ac2f0 2024-03-29T08:09:06.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:06.531+0000 7f71eb400700 1 --2- 172.21.15.37:0/1971374977 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7f71d406ca10 0x7f71d406eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:09:06.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:06.534+0000 7f71eb400700 1 --2- 172.21.15.37:0/1971374977 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7f71d406ca10 0x7f71d406eec0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f71e40ad200 tx=0x7f71e000b040 comp rx=0 tx=0).ready entity=mgr.14686 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:09:06.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:06.538+0000 7f71df400700 1 -- 172.21.15.37:0/1971374977 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181579 (secure 0 0 0) 0x7f71d8056ff0 con 0x7f71e40ac2f0 2024-03-29T08:09:06.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:06.901+0000 7f71f1000700 1 -- 172.21.15.37:0/1971374977 --> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f71cc000bf0 con 0x7f71d406ca10 2024-03-29T08:09:06.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:06.906+0000 7f71df400700 1 -- 172.21.15.37:0/1971374977 <== mgr.14686 v2:172.21.15.37:6800/4188329516 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+360 (secure 0 0 0) 0x7f71cc000bf0 con 0x7f71d406ca10 2024-03-29T08:09:06.907 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-29T08:09:06.907 INFO:teuthology.orchestra.run.smithi037.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59", 2024-03-29T08:09:06.907 INFO:teuthology.orchestra.run.smithi037.stdout: "in_progress": true, 2024-03-29T08:09:06.907 INFO:teuthology.orchestra.run.smithi037.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-03-29T08:09:06.908 INFO:teuthology.orchestra.run.smithi037.stdout: "services_complete": [ 2024-03-29T08:09:06.908 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr" 2024-03-29T08:09:06.908 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-29T08:09:06.908 INFO:teuthology.orchestra.run.smithi037.stdout: "progress": "2/2 daemons upgraded", 2024-03-29T08:09:06.908 INFO:teuthology.orchestra.run.smithi037.stdout: "message": "Currently upgrading mgr daemons", 2024-03-29T08:09:06.908 INFO:teuthology.orchestra.run.smithi037.stdout: "is_paused": false 2024-03-29T08:09:06.908 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-29T08:09:06.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:06.910+0000 7f71f1000700 1 -- 172.21.15.37:0/1971374977 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7f71d406ca10 msgr2=0x7f71d406eec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:09:06.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:06.910+0000 7f71f1000700 1 --2- 172.21.15.37:0/1971374977 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7f71d406ca10 0x7f71d406eec0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f71e40ad200 tx=0x7f71e000b040 comp rx=0 tx=0).stop 2024-03-29T08:09:06.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:06.910+0000 7f71f1000700 1 -- 172.21.15.37:0/1971374977 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f71e40ac2f0 msgr2=0x7f71e40b1290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:09:06.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:06.910+0000 7f71f1000700 1 --2- 172.21.15.37:0/1971374977 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f71e40ac2f0 0x7f71e40b1290 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f71d8004890 tx=0x7f71d8009e30 comp rx=0 tx=0).stop 2024-03-29T08:09:06.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:06.910+0000 7f71f1000700 1 -- 172.21.15.37:0/1971374977 shutdown_connections 2024-03-29T08:09:06.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:06.910+0000 7f71f1000700 1 --2- 172.21.15.37:0/1971374977 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7f71d406ca10 0x7f71d406eec0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:09:06.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:06.910+0000 7f71f1000700 1 --2- 172.21.15.37:0/1971374977 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f71e4008640 0x7f71e40abdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:09:06.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:06.910+0000 7f71f1000700 1 --2- 172.21.15.37:0/1971374977 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f71e40ac2f0 0x7f71e40b1290 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:09:06.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:06.910+0000 7f71f1000700 1 -- 172.21.15.37:0/1971374977 >> 172.21.15.37:0/1971374977 conn(0x7f71e4090350 msgr2=0x7f71e40965a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:09:06.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:06.911+0000 7f71f1000700 1 -- 172.21.15.37:0/1971374977 shutdown_connections 2024-03-29T08:09:06.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:06.911+0000 7f71f1000700 1 -- 172.21.15.37:0/1971374977 wait complete. 2024-03-29T08:09:07.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:09:07 smithi037 ceph-mon[35162]: from='client.24583 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:09:07.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:09:07 smithi037 ceph-mon[35162]: from='client.? 172.21.15.37:0/2993064512' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:09:07.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:09:07 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:09:07.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:09:07 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' 2024-03-29T08:09:07.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:09:07 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' 2024-03-29T08:09:07.271 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:09:07 smithi148 ceph-mon[44992]: from='client.24583 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:09:07.271 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:09:07 smithi148 ceph-mon[44992]: from='client.? 172.21.15.37:0/2993064512' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:09:07.272 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:09:07 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:09:07.272 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:09:07 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' 2024-03-29T08:09:07.272 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:09:07 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' 2024-03-29T08:09:07.353 DEBUG:teuthology.parallel:result is None 2024-03-29T08:09:08.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:09:08 smithi148 ceph-mon[44992]: pgmap v33: 65 pgs: 65 active+clean; 271 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 30 KiB/s rd, 1.1 MiB/s wr, 330 op/s 2024-03-29T08:09:08.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:09:08 smithi148 ceph-mon[44992]: from='client.14758 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:09:08.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:09:08 smithi037 ceph-mon[35162]: pgmap v33: 65 pgs: 65 active+clean; 271 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 30 KiB/s rd, 1.1 MiB/s wr, 330 op/s 2024-03-29T08:09:08.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:09:08 smithi037 ceph-mon[35162]: from='client.14758 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:09:10.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:09:10 smithi148 ceph-mon[44992]: pgmap v34: 65 pgs: 65 active+clean; 264 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 44 KiB/s rd, 1.5 MiB/s wr, 471 op/s 2024-03-29T08:09:10.426 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:09:10 smithi037 ceph-mon[35162]: pgmap v34: 65 pgs: 65 active+clean; 264 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 44 KiB/s rd, 1.5 MiB/s wr, 471 op/s 2024-03-29T08:09:11.188 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:09:11 smithi148 ceph-mon[44992]: Standby manager daemon smithi148.onhgvm restarted 2024-03-29T08:09:11.188 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:09:11 smithi148 ceph-mon[44992]: Standby manager daemon smithi148.onhgvm started 2024-03-29T08:09:11.188 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:09:11 smithi148 ceph-mon[44992]: from='mgr.? 172.21.15.148:0/2608329063' entity='mgr.smithi148.onhgvm' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi148.onhgvm/crt"}]: dispatch 2024-03-29T08:09:11.188 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:09:11 smithi148 ceph-mon[44992]: from='mgr.? 172.21.15.148:0/2608329063' entity='mgr.smithi148.onhgvm' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-29T08:09:11.188 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:09:11 smithi148 ceph-mon[44992]: from='mgr.? 172.21.15.148:0/2608329063' entity='mgr.smithi148.onhgvm' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi148.onhgvm/key"}]: dispatch 2024-03-29T08:09:11.188 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:09:11 smithi148 ceph-mon[44992]: from='mgr.? 172.21.15.148:0/2608329063' entity='mgr.smithi148.onhgvm' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-29T08:09:11.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:09:11 smithi037 ceph-mon[35162]: Standby manager daemon smithi148.onhgvm restarted 2024-03-29T08:09:11.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:09:11 smithi037 ceph-mon[35162]: Standby manager daemon smithi148.onhgvm started 2024-03-29T08:09:11.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:09:11 smithi037 ceph-mon[35162]: from='mgr.? 172.21.15.148:0/2608329063' entity='mgr.smithi148.onhgvm' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi148.onhgvm/crt"}]: dispatch 2024-03-29T08:09:11.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:09:11 smithi037 ceph-mon[35162]: from='mgr.? 172.21.15.148:0/2608329063' entity='mgr.smithi148.onhgvm' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-29T08:09:11.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:09:11 smithi037 ceph-mon[35162]: from='mgr.? 172.21.15.148:0/2608329063' entity='mgr.smithi148.onhgvm' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi148.onhgvm/key"}]: dispatch 2024-03-29T08:09:11.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:09:11 smithi037 ceph-mon[35162]: from='mgr.? 172.21.15.148:0/2608329063' entity='mgr.smithi148.onhgvm' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-29T08:09:12.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:09:12 smithi148 ceph-mon[44992]: pgmap v35: 65 pgs: 65 active+clean; 264 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 29 KiB/s rd, 951 KiB/s wr, 304 op/s 2024-03-29T08:09:12.307 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:09:12 smithi148 ceph-mon[44992]: mgrmap e33: smithi037.wjikii(active, since 64s), standbys: smithi148.onhgvm 2024-03-29T08:09:12.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:09:12 smithi037 ceph-mon[35162]: pgmap v35: 65 pgs: 65 active+clean; 264 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 29 KiB/s rd, 951 KiB/s wr, 304 op/s 2024-03-29T08:09:12.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:09:12 smithi037 ceph-mon[35162]: mgrmap e33: smithi037.wjikii(active, since 64s), standbys: smithi148.onhgvm 2024-03-29T08:09:13.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:09:13 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' 2024-03-29T08:09:13.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:09:13 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' 2024-03-29T08:09:13.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:09:13 smithi037 ceph-mon[35162]: pgmap v36: 65 pgs: 65 active+clean; 263 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 29 KiB/s rd, 962 KiB/s wr, 322 op/s 2024-03-29T08:09:13.591 INFO:tasks.workunit:Stopping ['suites/fsstress.sh'] on client.0... 2024-03-29T08:09:13.592 DEBUG:teuthology.orchestra.run.smithi037:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-03-29T08:09:13.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:09:13 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' 2024-03-29T08:09:13.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:09:13 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' 2024-03-29T08:09:13.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:09:13 smithi148 ceph-mon[44992]: pgmap v36: 65 pgs: 65 active+clean; 263 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 29 KiB/s rd, 962 KiB/s wr, 322 op/s 2024-03-29T08:09:15.090 DEBUG:teuthology.parallel:result is None 2024-03-29T08:09:15.091 DEBUG:teuthology.orchestra.run.smithi037:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-03-29T08:09:15.177 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-03-29T08:09:15.177 DEBUG:teuthology.orchestra.run.smithi148:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.1/client.1 2024-03-29T08:09:15.298 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.1/client.1 2024-03-29T08:09:15.298 DEBUG:teuthology.parallel:result is None 2024-03-29T08:09:15.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:09:15 smithi148 ceph-mon[44992]: pgmap v37: 65 pgs: 65 active+clean; 263 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 37 KiB/s rd, 1.1 MiB/s wr, 354 op/s 2024-03-29T08:09:15.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:09:15 smithi037 ceph-mon[35162]: pgmap v37: 65 pgs: 65 active+clean; 263 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 37 KiB/s rd, 1.1 MiB/s wr, 354 op/s 2024-03-29T08:09:17.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:09:16 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' 2024-03-29T08:09:17.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:09:16 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' 2024-03-29T08:09:17.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:09:16 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T08:09:17.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:09:16 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T08:09:17.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:09:16 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' 2024-03-29T08:09:17.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:09:16 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T08:09:17.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:09:16 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:09:17.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:09:16 smithi148 ceph-mon[44992]: Upgrade: Setting container_image for all mgr 2024-03-29T08:09:17.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:09:16 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' 2024-03-29T08:09:17.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:09:16 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi037.wjikii"}]: dispatch 2024-03-29T08:09:17.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:09:16 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi037.wjikii"}]': finished 2024-03-29T08:09:17.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:09:16 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi148.onhgvm"}]: dispatch 2024-03-29T08:09:17.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:09:16 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi148.onhgvm"}]': finished 2024-03-29T08:09:17.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:09:16 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:09:17.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:09:16 smithi148 ceph-mon[44992]: Upgrade: Setting container_image for all rgw 2024-03-29T08:09:17.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:09:16 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' 2024-03-29T08:09:17.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:09:16 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:09:17.058 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:09:16 smithi148 ceph-mon[44992]: Upgrade: Setting container_image for all rbd-mirror 2024-03-29T08:09:17.058 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:09:16 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' 2024-03-29T08:09:17.058 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:09:16 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:09:17.058 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:09:16 smithi148 ceph-mon[44992]: Upgrade: Setting container_image for all cephfs-mirror 2024-03-29T08:09:17.058 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:09:16 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' 2024-03-29T08:09:17.058 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:09:16 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:09:17.058 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:09:16 smithi148 ceph-mon[44992]: Upgrade: Setting container_image for all ceph-exporter 2024-03-29T08:09:17.058 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:09:16 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' 2024-03-29T08:09:17.058 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:09:16 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:09:17.058 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:09:16 smithi148 ceph-mon[44992]: Upgrade: Setting container_image for all iscsi 2024-03-29T08:09:17.058 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:09:16 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' 2024-03-29T08:09:17.058 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:09:16 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:09:17.058 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:09:16 smithi148 ceph-mon[44992]: Upgrade: Setting container_image for all nfs 2024-03-29T08:09:17.058 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:09:16 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' 2024-03-29T08:09:17.059 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:09:16 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:09:17.059 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:09:16 smithi148 ceph-mon[44992]: Upgrade: Setting container_image for all nvmeof 2024-03-29T08:09:17.059 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:09:16 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' 2024-03-29T08:09:17.104 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:09:16 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' 2024-03-29T08:09:17.104 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:09:16 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' 2024-03-29T08:09:17.104 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:09:16 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T08:09:17.104 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:09:16 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T08:09:17.104 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:09:16 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' 2024-03-29T08:09:17.105 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:09:16 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T08:09:17.105 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:09:16 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:09:17.105 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:09:16 smithi037 ceph-mon[35162]: Upgrade: Setting container_image for all mgr 2024-03-29T08:09:17.105 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:09:16 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' 2024-03-29T08:09:17.105 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:09:16 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi037.wjikii"}]: dispatch 2024-03-29T08:09:17.105 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:09:16 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi037.wjikii"}]': finished 2024-03-29T08:09:17.105 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:09:16 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi148.onhgvm"}]: dispatch 2024-03-29T08:09:17.105 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:09:16 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi148.onhgvm"}]': finished 2024-03-29T08:09:17.105 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:09:16 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:09:17.105 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:09:16 smithi037 ceph-mon[35162]: Upgrade: Setting container_image for all rgw 2024-03-29T08:09:17.105 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:09:16 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' 2024-03-29T08:09:17.105 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:09:16 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:09:17.105 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:09:16 smithi037 ceph-mon[35162]: Upgrade: Setting container_image for all rbd-mirror 2024-03-29T08:09:17.105 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:09:16 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' 2024-03-29T08:09:17.105 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:09:16 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:09:17.106 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:09:16 smithi037 ceph-mon[35162]: Upgrade: Setting container_image for all cephfs-mirror 2024-03-29T08:09:17.106 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:09:16 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' 2024-03-29T08:09:17.106 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:09:16 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:09:17.106 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:09:16 smithi037 ceph-mon[35162]: Upgrade: Setting container_image for all ceph-exporter 2024-03-29T08:09:17.106 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:09:16 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' 2024-03-29T08:09:17.106 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:09:16 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:09:17.106 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:09:16 smithi037 ceph-mon[35162]: Upgrade: Setting container_image for all iscsi 2024-03-29T08:09:17.106 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:09:16 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' 2024-03-29T08:09:17.106 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:09:16 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:09:17.106 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:09:16 smithi037 ceph-mon[35162]: Upgrade: Setting container_image for all nfs 2024-03-29T08:09:17.106 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:09:16 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' 2024-03-29T08:09:17.106 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:09:16 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:09:17.106 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:09:16 smithi037 ceph-mon[35162]: Upgrade: Setting container_image for all nvmeof 2024-03-29T08:09:17.106 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:09:16 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' 2024-03-29T08:09:18.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:09:18 smithi037 ceph-mon[35162]: pgmap v38: 65 pgs: 65 active+clean; 263 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 22 KiB/s rd, 752 KiB/s wr, 232 op/s 2024-03-29T08:09:18.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:09:18 smithi148 ceph-mon[44992]: pgmap v38: 65 pgs: 65 active+clean; 263 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 22 KiB/s rd, 752 KiB/s wr, 232 op/s 2024-03-29T08:09:19.338 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:09:19 smithi037 ceph-mon[35162]: Upgrade: Updating node-exporter.smithi037 (1/2) 2024-03-29T08:09:19.338 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:09:19 smithi037 ceph-mon[35162]: Deploying daemon node-exporter.smithi037 on smithi037 2024-03-29T08:09:19.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:09:19 smithi148 ceph-mon[44992]: Upgrade: Updating node-exporter.smithi037 (1/2) 2024-03-29T08:09:19.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:09:19 smithi148 ceph-mon[44992]: Deploying daemon node-exporter.smithi037 on smithi037 2024-03-29T08:09:20.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:09:20 smithi037 ceph-mon[35162]: pgmap v39: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 29 KiB/s rd, 877 KiB/s wr, 296 op/s 2024-03-29T08:09:20.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:09:20 smithi148 ceph-mon[44992]: pgmap v39: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 29 KiB/s rd, 877 KiB/s wr, 296 op/s 2024-03-29T08:09:22.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:09:22 smithi037 ceph-mon[35162]: pgmap v40: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 15 KiB/s rd, 467 KiB/s wr, 156 op/s 2024-03-29T08:09:22.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:09:22 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:09:22.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:09:22 smithi148 ceph-mon[44992]: pgmap v40: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 15 KiB/s rd, 467 KiB/s wr, 156 op/s 2024-03-29T08:09:22.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:09:22 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:09:23.707 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:09:23 smithi148 ceph-mon[44992]: pgmap v41: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 15 KiB/s rd, 472 KiB/s wr, 156 op/s 2024-03-29T08:09:23.707 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:09:23 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' 2024-03-29T08:09:23.708 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:09:23 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' 2024-03-29T08:09:23.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:09:23 smithi037 ceph-mon[35162]: pgmap v41: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 15 KiB/s rd, 472 KiB/s wr, 156 op/s 2024-03-29T08:09:23.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:09:23 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' 2024-03-29T08:09:23.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:09:23 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' 2024-03-29T08:09:25.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:09:25 smithi037 ceph-mon[35162]: Upgrade: Updating node-exporter.smithi148 (2/2) 2024-03-29T08:09:25.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:09:25 smithi037 ceph-mon[35162]: Deploying daemon node-exporter.smithi148 on smithi148 2024-03-29T08:09:25.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:09:25 smithi037 ceph-mon[35162]: pgmap v42: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 15 KiB/s rd, 461 KiB/s wr, 138 op/s 2024-03-29T08:09:25.931 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:09:25 smithi148 ceph-mon[44992]: Upgrade: Updating node-exporter.smithi148 (2/2) 2024-03-29T08:09:25.931 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:09:25 smithi148 ceph-mon[44992]: Deploying daemon node-exporter.smithi148 on smithi148 2024-03-29T08:09:25.931 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:09:25 smithi148 ceph-mon[44992]: pgmap v42: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 15 KiB/s rd, 461 KiB/s wr, 138 op/s 2024-03-29T08:09:27.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:09:27 smithi037 ceph-mon[35162]: pgmap v43: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 7.2 KiB/s rd, 130 KiB/s wr, 64 op/s 2024-03-29T08:09:27.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:09:27 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' 2024-03-29T08:09:27.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:09:27 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' 2024-03-29T08:09:27.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:09:27 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T08:09:28.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:09:27 smithi148 ceph-mon[44992]: pgmap v43: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 7.2 KiB/s rd, 130 KiB/s wr, 64 op/s 2024-03-29T08:09:28.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:09:27 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' 2024-03-29T08:09:28.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:09:27 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' 2024-03-29T08:09:28.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:09:27 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T08:09:29.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:09:29 smithi148 ceph-mon[44992]: pgmap v44: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 7.2 KiB/s rd, 130 KiB/s wr, 64 op/s 2024-03-29T08:09:29.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:09:29 smithi037 ceph-mon[35162]: pgmap v44: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 7.2 KiB/s rd, 130 KiB/s wr, 64 op/s 2024-03-29T08:09:32.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:09:32 smithi037 ceph-mon[35162]: pgmap v45: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 4.8 KiB/s wr, 0 op/s 2024-03-29T08:09:32.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:09:32 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' 2024-03-29T08:09:32.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:09:32 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' 2024-03-29T08:09:32.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:09:32 smithi148 ceph-mon[44992]: pgmap v45: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 4.8 KiB/s wr, 0 op/s 2024-03-29T08:09:32.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:09:32 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' 2024-03-29T08:09:32.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:09:32 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' 2024-03-29T08:09:33.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:09:33 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' 2024-03-29T08:09:33.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:09:33 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' 2024-03-29T08:09:33.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:09:33 smithi037 ceph-mon[35162]: pgmap v46: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 4.8 KiB/s wr, 0 op/s 2024-03-29T08:09:33.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:09:33 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' 2024-03-29T08:09:33.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:09:33 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' 2024-03-29T08:09:33.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:09:33 smithi148 ceph-mon[44992]: pgmap v46: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 4.8 KiB/s wr, 0 op/s 2024-03-29T08:09:34.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:09:34 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' 2024-03-29T08:09:34.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:09:34 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' 2024-03-29T08:09:35.178 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:09:34 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' 2024-03-29T08:09:35.179 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:09:34 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' 2024-03-29T08:09:35.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:09:35 smithi037 ceph-mon[35162]: pgmap v47: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:09:35.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:09:35 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' 2024-03-29T08:09:35.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:09:35 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' 2024-03-29T08:09:36.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:09:35 smithi148 ceph-mon[44992]: pgmap v47: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:09:36.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:09:35 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' 2024-03-29T08:09:36.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:09:35 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' 2024-03-29T08:09:37.058 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:09:36 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:09:37.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:37.067+0000 7fd1d7600700 1 -- 172.21.15.37:0/2571292751 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd1d00709a0 msgr2=0x7fd1d0070e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:09:37.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:37.067+0000 7fd1d7600700 1 --2- 172.21.15.37:0/2571292751 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd1d00709a0 0x7fd1d0070e10 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fd1bc004940 tx=0x7fd1bc009e30 comp rx=0 tx=0).stop 2024-03-29T08:09:37.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:37.067+0000 7fd1d7600700 1 -- 172.21.15.37:0/2571292751 shutdown_connections 2024-03-29T08:09:37.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:37.067+0000 7fd1d7600700 1 --2- 172.21.15.37:0/2571292751 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd1d00709a0 0x7fd1d0070e10 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:09:37.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:37.067+0000 7fd1d7600700 1 --2- 172.21.15.37:0/2571292751 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd1d0071f50 0x7fd1d0070460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:09:37.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:37.067+0000 7fd1d7600700 1 -- 172.21.15.37:0/2571292751 >> 172.21.15.37:0/2571292751 conn(0x7fd1d00f8560 msgr2=0x7fd1d00fa970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:09:37.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:37.067+0000 7fd1d7600700 1 -- 172.21.15.37:0/2571292751 shutdown_connections 2024-03-29T08:09:37.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:37.067+0000 7fd1d7600700 1 -- 172.21.15.37:0/2571292751 wait complete. 2024-03-29T08:09:37.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:37.068+0000 7fd1d7600700 1 Processor -- start 2024-03-29T08:09:37.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:37.068+0000 7fd1d7600700 1 -- start start 2024-03-29T08:09:37.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:37.068+0000 7fd1d7600700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd1d00709a0 0x7fd1d010faa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:09:37.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:37.068+0000 7fd1d7600700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd1d0071f50 0x7fd1d010ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:09:37.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:37.068+0000 7fd1d7600700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1d01105e0 con 0x7fd1d0071f50 2024-03-29T08:09:37.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:37.068+0000 7fd1d7600700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1d0110e30 con 0x7fd1d00709a0 2024-03-29T08:09:37.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:37.069+0000 7fd1cfe00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd1d0071f50 0x7fd1d010ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:09:37.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:37.069+0000 7fd1cfe00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd1d0071f50 0x7fd1d010ffe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45432/0 (socket says 172.21.15.37:45432) 2024-03-29T08:09:37.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:37.069+0000 7fd1cfe00700 1 -- 172.21.15.37:0/93364898 learned_addr learned my addr 172.21.15.37:0/93364898 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:09:37.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:37.069+0000 7fd1d4c00700 1 --2- 172.21.15.37:0/93364898 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd1d00709a0 0x7fd1d010faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:09:37.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:37.069+0000 7fd1cfe00700 1 -- 172.21.15.37:0/93364898 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd1d00709a0 msgr2=0x7fd1d010faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:09:37.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:37.069+0000 7fd1cfe00700 1 --2- 172.21.15.37:0/93364898 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd1d00709a0 0x7fd1d010faa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:09:37.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:37.069+0000 7fd1cfe00700 1 -- 172.21.15.37:0/93364898 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1bc005040 con 0x7fd1d0071f50 2024-03-29T08:09:37.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:37.070+0000 7fd1cfe00700 1 --2- 172.21.15.37:0/93364898 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd1d0071f50 0x7fd1d010ffe0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fd1bc00b0d0 tx=0x7fd1bc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:09:37.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:37.070+0000 7fd1ccc00700 1 -- 172.21.15.37:0/93364898 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd1bc00e070 con 0x7fd1d0071f50 2024-03-29T08:09:37.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:37.070+0000 7fd1ccc00700 1 -- 172.21.15.37:0/93364898 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd1bc00b410 con 0x7fd1d0071f50 2024-03-29T08:09:37.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:37.070+0000 7fd1d7600700 1 -- 172.21.15.37:0/93364898 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1d01110b0 con 0x7fd1d0071f50 2024-03-29T08:09:37.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:37.070+0000 7fd1ccc00700 1 -- 172.21.15.37:0/93364898 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd1bc012600 con 0x7fd1d0071f50 2024-03-29T08:09:37.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:37.070+0000 7fd1d7600700 1 -- 172.21.15.37:0/93364898 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1d01115f0 con 0x7fd1d0071f50 2024-03-29T08:09:37.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:37.071+0000 7fd1d7600700 1 -- 172.21.15.37:0/93364898 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1d005f5f0 con 0x7fd1d0071f50 2024-03-29T08:09:37.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:37.072+0000 7fd1ccc00700 1 -- 172.21.15.37:0/93364898 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 33) v1 ==== 90805+0+0 (secure 0 0 0) 0x7fd1bc00b9f0 con 0x7fd1d0071f50 2024-03-29T08:09:37.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:37.073+0000 7fd1ccc00700 1 --2- 172.21.15.37:0/93364898 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7fd1b806c9c0 0x7fd1b806ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:09:37.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:37.073+0000 7fd1ccc00700 1 -- 172.21.15.37:0/93364898 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6066+0+0 (secure 0 0 0) 0x7fd1bc08caa0 con 0x7fd1d0071f50 2024-03-29T08:09:37.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:37.073+0000 7fd1d4c00700 1 --2- 172.21.15.37:0/93364898 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7fd1b806c9c0 0x7fd1b806ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:09:37.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:37.074+0000 7fd1d4c00700 1 --2- 172.21.15.37:0/93364898 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7fd1b806c9c0 0x7fd1b806ee70 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fd1c4004600 tx=0x7fd1c400b040 comp rx=0 tx=0).ready entity=mgr.14686 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:09:37.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:37.078+0000 7fd1ccc00700 1 -- 172.21.15.37:0/93364898 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181579 (secure 0 0 0) 0x7fd1bc05a880 con 0x7fd1d0071f50 2024-03-29T08:09:37.149 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:09:36 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:09:37.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:37.328+0000 7fd1d7600700 1 -- 172.21.15.37:0/93364898 --> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd1d0111920 con 0x7fd1b806c9c0 2024-03-29T08:09:37.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:37.330+0000 7fd1ccc00700 1 -- 172.21.15.37:0/93364898 <== mgr.14686 v2:172.21.15.37:6800/4188329516 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+370 (secure 0 0 0) 0x7fd1d0111920 con 0x7fd1b806c9c0 2024-03-29T08:09:37.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:37.332+0000 7fd1d7600700 1 -- 172.21.15.37:0/93364898 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7fd1b806c9c0 msgr2=0x7fd1b806ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:09:37.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:37.332+0000 7fd1d7600700 1 --2- 172.21.15.37:0/93364898 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7fd1b806c9c0 0x7fd1b806ee70 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fd1c4004600 tx=0x7fd1c400b040 comp rx=0 tx=0).stop 2024-03-29T08:09:37.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:37.332+0000 7fd1d7600700 1 -- 172.21.15.37:0/93364898 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd1d0071f50 msgr2=0x7fd1d010ffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:09:37.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:37.332+0000 7fd1d7600700 1 --2- 172.21.15.37:0/93364898 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd1d0071f50 0x7fd1d010ffe0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fd1bc00b0d0 tx=0x7fd1bc009e30 comp rx=0 tx=0).stop 2024-03-29T08:09:37.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:37.332+0000 7fd1d7600700 1 -- 172.21.15.37:0/93364898 shutdown_connections 2024-03-29T08:09:37.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:37.332+0000 7fd1d7600700 1 --2- 172.21.15.37:0/93364898 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7fd1b806c9c0 0x7fd1b806ee70 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:09:37.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:37.332+0000 7fd1d7600700 1 --2- 172.21.15.37:0/93364898 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd1d00709a0 0x7fd1d010faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:09:37.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:37.332+0000 7fd1d7600700 1 --2- 172.21.15.37:0/93364898 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd1d0071f50 0x7fd1d010ffe0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:09:37.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:37.332+0000 7fd1d7600700 1 -- 172.21.15.37:0/93364898 >> 172.21.15.37:0/93364898 conn(0x7fd1d00f8560 msgr2=0x7fd1d00f9c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:09:37.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:37.332+0000 7fd1d7600700 1 -- 172.21.15.37:0/93364898 shutdown_connections 2024-03-29T08:09:37.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:37.332+0000 7fd1d7600700 1 -- 172.21.15.37:0/93364898 wait complete. 2024-03-29T08:09:37.348 INFO:teuthology.orchestra.run.smithi037.stdout:true 2024-03-29T08:09:37.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:37.474+0000 7fed15e00700 1 -- 172.21.15.37:0/2692911657 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed100fcee0 msgr2=0x7fed100fd310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:09:37.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:37.474+0000 7fed15e00700 1 --2- 172.21.15.37:0/2692911657 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed100fcee0 0x7fed100fd310 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fed00004940 tx=0x7fed00009e30 comp rx=0 tx=0).stop 2024-03-29T08:09:37.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:37.474+0000 7fed15e00700 1 -- 172.21.15.37:0/2692911657 shutdown_connections 2024-03-29T08:09:37.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:37.474+0000 7fed15e00700 1 --2- 172.21.15.37:0/2692911657 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fed100fe0d0 0x7fed100fe540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:09:37.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:37.474+0000 7fed15e00700 1 --2- 172.21.15.37:0/2692911657 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed100fcee0 0x7fed100fd310 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:09:37.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:37.474+0000 7fed15e00700 1 -- 172.21.15.37:0/2692911657 >> 172.21.15.37:0/2692911657 conn(0x7fed100f8560 msgr2=0x7fed100fa970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:09:37.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:37.474+0000 7fed15e00700 1 -- 172.21.15.37:0/2692911657 shutdown_connections 2024-03-29T08:09:37.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:37.475+0000 7fed15e00700 1 -- 172.21.15.37:0/2692911657 wait complete. 2024-03-29T08:09:37.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:37.475+0000 7fed15e00700 1 Processor -- start 2024-03-29T08:09:37.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:37.476+0000 7fed15e00700 1 -- start start 2024-03-29T08:09:37.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:37.476+0000 7fed15e00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fed100fcee0 0x7fed10071dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:09:37.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:37.476+0000 7fed15e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed100fe0d0 0x7fed10070310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:09:37.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:37.476+0000 7fed15e00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed100708a0 con 0x7fed100fe0d0 2024-03-29T08:09:37.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:37.476+0000 7fed15e00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed100709e0 con 0x7fed100fcee0 2024-03-29T08:09:37.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:37.476+0000 7fed0e000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed100fe0d0 0x7fed10070310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:09:37.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:37.476+0000 7fed0ea00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fed100fcee0 0x7fed10071dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:09:37.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:37.476+0000 7fed0e000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed100fe0d0 0x7fed10070310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45442/0 (socket says 172.21.15.37:45442) 2024-03-29T08:09:37.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:37.476+0000 7fed0e000700 1 -- 172.21.15.37:0/2020800487 learned_addr learned my addr 172.21.15.37:0/2020800487 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:09:37.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:37.477+0000 7fed0e000700 1 -- 172.21.15.37:0/2020800487 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fed100fcee0 msgr2=0x7fed10071dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:09:37.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:37.477+0000 7fed0e000700 1 --2- 172.21.15.37:0/2020800487 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fed100fcee0 0x7fed10071dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:09:37.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:37.477+0000 7fed0e000700 1 -- 172.21.15.37:0/2020800487 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed00005040 con 0x7fed100fe0d0 2024-03-29T08:09:37.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:37.477+0000 7fed0e000700 1 --2- 172.21.15.37:0/2020800487 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed100fe0d0 0x7fed10070310 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fecfc002a50 tx=0x7fecfc0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:09:37.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:37.477+0000 7fed06a00700 1 -- 172.21.15.37:0/2020800487 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fecfc007cc0 con 0x7fed100fe0d0 2024-03-29T08:09:37.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:37.477+0000 7fed06a00700 1 -- 172.21.15.37:0/2020800487 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fecfc00f040 con 0x7fed100fe0d0 2024-03-29T08:09:37.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:37.477+0000 7fed15e00700 1 -- 172.21.15.37:0/2020800487 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed10070c90 con 0x7fed100fe0d0 2024-03-29T08:09:37.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:37.477+0000 7fed06a00700 1 -- 172.21.15.37:0/2020800487 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fecfc00d9c0 con 0x7fed100fe0d0 2024-03-29T08:09:37.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:37.477+0000 7fed15e00700 1 -- 172.21.15.37:0/2020800487 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed10115a00 con 0x7fed100fe0d0 2024-03-29T08:09:37.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:37.479+0000 7fed15e00700 1 -- 172.21.15.37:0/2020800487 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed1005f5f0 con 0x7fed100fe0d0 2024-03-29T08:09:37.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:37.479+0000 7fed06a00700 1 -- 172.21.15.37:0/2020800487 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 33) v1 ==== 90805+0+0 (secure 0 0 0) 0x7fecfc00db20 con 0x7fed100fe0d0 2024-03-29T08:09:37.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:37.480+0000 7fed06a00700 1 --2- 172.21.15.37:0/2020800487 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7fecf406c9c0 0x7fecf406ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:09:37.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:37.480+0000 7fed06a00700 1 -- 172.21.15.37:0/2020800487 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6066+0+0 (secure 0 0 0) 0x7fecfc08ba20 con 0x7fed100fe0d0 2024-03-29T08:09:37.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:37.481+0000 7fed0ea00700 1 --2- 172.21.15.37:0/2020800487 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7fecf406c9c0 0x7fecf406ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:09:37.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:37.481+0000 7fed0ea00700 1 --2- 172.21.15.37:0/2020800487 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7fecf406c9c0 0x7fecf406ee70 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fed10071b60 tx=0x7fed00009e30 comp rx=0 tx=0).ready entity=mgr.14686 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:09:37.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:37.485+0000 7fed06a00700 1 -- 172.21.15.37:0/2020800487 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181579 (secure 0 0 0) 0x7fecfc055cc0 con 0x7fed100fe0d0 2024-03-29T08:09:37.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:37.730+0000 7fed15e00700 1 -- 172.21.15.37:0/2020800487 --> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fed10102580 con 0x7fecf406c9c0 2024-03-29T08:09:37.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:37.731+0000 7fed06a00700 1 -- 172.21.15.37:0/2020800487 <== mgr.14686 v2:172.21.15.37:6800/4188329516 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+370 (secure 0 0 0) 0x7fed10102580 con 0x7fecf406c9c0 2024-03-29T08:09:37.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:37.733+0000 7fed15e00700 1 -- 172.21.15.37:0/2020800487 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7fecf406c9c0 msgr2=0x7fecf406ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:09:37.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:37.733+0000 7fed15e00700 1 --2- 172.21.15.37:0/2020800487 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7fecf406c9c0 0x7fecf406ee70 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fed10071b60 tx=0x7fed00009e30 comp rx=0 tx=0).stop 2024-03-29T08:09:37.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:37.733+0000 7fed15e00700 1 -- 172.21.15.37:0/2020800487 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed100fe0d0 msgr2=0x7fed10070310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:09:37.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:37.733+0000 7fed15e00700 1 --2- 172.21.15.37:0/2020800487 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed100fe0d0 0x7fed10070310 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fecfc002a50 tx=0x7fecfc0046b0 comp rx=0 tx=0).stop 2024-03-29T08:09:37.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:37.733+0000 7fed15e00700 1 -- 172.21.15.37:0/2020800487 shutdown_connections 2024-03-29T08:09:37.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:37.733+0000 7fed15e00700 1 --2- 172.21.15.37:0/2020800487 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7fecf406c9c0 0x7fecf406ee70 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:09:37.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:37.733+0000 7fed15e00700 1 --2- 172.21.15.37:0/2020800487 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fed100fcee0 0x7fed10071dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:09:37.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:37.733+0000 7fed15e00700 1 --2- 172.21.15.37:0/2020800487 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed100fe0d0 0x7fed10070310 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:09:37.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:37.733+0000 7fed15e00700 1 -- 172.21.15.37:0/2020800487 >> 172.21.15.37:0/2020800487 conn(0x7fed100f8560 msgr2=0x7fed100f9da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:09:37.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:37.734+0000 7fed15e00700 1 -- 172.21.15.37:0/2020800487 shutdown_connections 2024-03-29T08:09:37.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:37.734+0000 7fed15e00700 1 -- 172.21.15.37:0/2020800487 wait complete. 2024-03-29T08:09:37.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:37.871+0000 7f25fbe00700 1 -- 172.21.15.37:0/4121986467 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f25fc0ffde0 msgr2=0x7f25fc1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:09:37.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:37.871+0000 7f25fbe00700 1 --2- 172.21.15.37:0/4121986467 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f25fc0ffde0 0x7f25fc1021c0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f25e8004940 tx=0x7f25e8009e30 comp rx=0 tx=0).stop 2024-03-29T08:09:37.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:37.871+0000 7f25fbe00700 1 -- 172.21.15.37:0/4121986467 shutdown_connections 2024-03-29T08:09:37.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:37.871+0000 7f25fbe00700 1 --2- 172.21.15.37:0/4121986467 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f25fc0ffde0 0x7f25fc1021c0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:09:37.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:37.871+0000 7f25fbe00700 1 --2- 172.21.15.37:0/4121986467 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f25fc066240 0x7f25fc0ff8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:09:37.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:37.871+0000 7f25fbe00700 1 -- 172.21.15.37:0/4121986467 >> 172.21.15.37:0/4121986467 conn(0x7f25fc0f7270 msgr2=0x7f25fc0f96a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:09:37.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:37.871+0000 7f25fbe00700 1 -- 172.21.15.37:0/4121986467 shutdown_connections 2024-03-29T08:09:37.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:37.871+0000 7f25fbe00700 1 -- 172.21.15.37:0/4121986467 wait complete. 2024-03-29T08:09:37.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:37.872+0000 7f25fbe00700 1 Processor -- start 2024-03-29T08:09:37.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:37.872+0000 7f25fbe00700 1 -- start start 2024-03-29T08:09:37.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:37.872+0000 7f25fbe00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f25fc066240 0x7f25fc10fa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:09:37.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:37.872+0000 7f25fbe00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f25fc0ffde0 0x7f25fc10ff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:09:37.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:37.872+0000 7f25fbe00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25fc110550 con 0x7f25fc066240 2024-03-29T08:09:37.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:37.872+0000 7f25fbe00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25fc110d50 con 0x7f25fc0ffde0 2024-03-29T08:09:37.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:37.873+0000 7f25faa00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f25fc066240 0x7f25fc10fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:09:37.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:37.873+0000 7f25faa00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f25fc066240 0x7f25fc10fa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45466/0 (socket says 172.21.15.37:45466) 2024-03-29T08:09:37.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:37.873+0000 7f25faa00700 1 -- 172.21.15.37:0/840560629 learned_addr learned my addr 172.21.15.37:0/840560629 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:09:37.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:37.873+0000 7f25fa000700 1 --2- 172.21.15.37:0/840560629 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f25fc0ffde0 0x7f25fc10ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:09:37.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:37.873+0000 7f25faa00700 1 -- 172.21.15.37:0/840560629 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f25fc0ffde0 msgr2=0x7f25fc10ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:09:37.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:37.873+0000 7f25faa00700 1 --2- 172.21.15.37:0/840560629 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f25fc0ffde0 0x7f25fc10ff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:09:37.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:37.873+0000 7f25faa00700 1 -- 172.21.15.37:0/840560629 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25e8005040 con 0x7f25fc066240 2024-03-29T08:09:37.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:37.873+0000 7f25faa00700 1 --2- 172.21.15.37:0/840560629 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f25fc066240 0x7f25fc10fa10 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f25ec002a80 tx=0x7f25ec004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:09:37.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:37.874+0000 7f25f2a00700 1 -- 172.21.15.37:0/840560629 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f25ec007cc0 con 0x7f25fc066240 2024-03-29T08:09:37.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:37.874+0000 7f25f2a00700 1 -- 172.21.15.37:0/840560629 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f25ec00f040 con 0x7f25fc066240 2024-03-29T08:09:37.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:37.874+0000 7f25fbe00700 1 -- 172.21.15.37:0/840560629 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25fc111030 con 0x7f25fc066240 2024-03-29T08:09:37.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:37.874+0000 7f25f2a00700 1 -- 172.21.15.37:0/840560629 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f25ec00d9c0 con 0x7f25fc066240 2024-03-29T08:09:37.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:37.874+0000 7f25fbe00700 1 -- 172.21.15.37:0/840560629 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25fc111570 con 0x7f25fc066240 2024-03-29T08:09:37.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:37.875+0000 7f25fbe00700 1 -- 172.21.15.37:0/840560629 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25fc0f8df0 con 0x7f25fc066240 2024-03-29T08:09:37.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:37.876+0000 7f25f2a00700 1 -- 172.21.15.37:0/840560629 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 33) v1 ==== 90805+0+0 (secure 0 0 0) 0x7f25ec00db20 con 0x7f25fc066240 2024-03-29T08:09:37.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:37.877+0000 7f25f2a00700 1 --2- 172.21.15.37:0/840560629 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7f25e006c9c0 0x7f25e006ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:09:37.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:37.877+0000 7f25f2a00700 1 -- 172.21.15.37:0/840560629 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f25ec08b970 con 0x7f25fc066240 2024-03-29T08:09:37.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:37.877+0000 7f25fa000700 1 --2- 172.21.15.37:0/840560629 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7f25e006c9c0 0x7f25e006ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:09:37.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:37.878+0000 7f25fa000700 1 --2- 172.21.15.37:0/840560629 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7f25e006c9c0 0x7f25e006ee70 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f25e8004f90 tx=0x7f25e8009e30 comp rx=0 tx=0).ready entity=mgr.14686 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:09:37.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:37.882+0000 7f25f2a00700 1 -- 172.21.15.37:0/840560629 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181579 (secure 0 0 0) 0x7f25ec059740 con 0x7f25fc066240 2024-03-29T08:09:38.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:38.114+0000 7f25fbe00700 1 -- 172.21.15.37:0/840560629 --> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f25fc1117e0 con 0x7f25e006c9c0 2024-03-29T08:09:38.241 INFO:teuthology.orchestra.run.smithi037.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-29T08:09:38.241 INFO:teuthology.orchestra.run.smithi037.stdout:alertmanager.smithi037 smithi037 *:9093,9094 running (3m) 4s ago 12m 15.1M - 0.25.0 c8568f914cd2 8a8d40a6e26e 2024-03-29T08:09:38.241 INFO:teuthology.orchestra.run.smithi037.stdout:crash.smithi037 smithi037 running (12m) 4s ago 12m 6656k - 17.2.7-789-g4213e17c cf22eb8683be e0343935b25e 2024-03-29T08:09:38.241 INFO:teuthology.orchestra.run.smithi037.stdout:crash.smithi148 smithi148 running (11m) 6s ago 11m 6656k - 17.2.7-789-g4213e17c cf22eb8683be ff5164c552c1 2024-03-29T08:09:38.241 INFO:teuthology.orchestra.run.smithi037.stdout:grafana.smithi037 smithi037 *:3000 running (2m) 4s ago 12m 74.7M - 9.4.7 954c08fa6188 a857385eee0c 2024-03-29T08:09:38.241 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi037.iifsmh smithi037 running (5m) 4s ago 5m 110M - 17.2.7-789-g4213e17c cf22eb8683be 074e1acf331f 2024-03-29T08:09:38.241 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi037.mmwgxe smithi037 running (6m) 4s ago 6m 115M - 17.2.7-789-g4213e17c cf22eb8683be 9a5b1e72de5c 2024-03-29T08:09:38.241 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi148.qukpjh smithi148 running (5m) 6s ago 5m 12.9M - 17.2.7-789-g4213e17c cf22eb8683be 8e469782a273 2024-03-29T08:09:38.241 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi148.sqteih smithi148 running (6m) 6s ago 6m 12.8M - 17.2.7-789-g4213e17c cf22eb8683be 8e68f858bb22 2024-03-29T08:09:38.241 INFO:teuthology.orchestra.run.smithi037.stdout:mgr.smithi037.wjikii smithi037 *:8443,9283,8765 running (109s) 4s ago 14m 494M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff c5a1417b9417 2024-03-29T08:09:38.241 INFO:teuthology.orchestra.run.smithi037.stdout:mgr.smithi148.onhgvm smithi148 *:8443,9283,8765 running (37s) 6s ago 11m 446M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff fcfdbce062bd 2024-03-29T08:09:38.241 INFO:teuthology.orchestra.run.smithi037.stdout:mon.smithi037 smithi037 running (14m) 4s ago 14m 53.4M 2048M 17.2.7-789-g4213e17c cf22eb8683be bb2101505730 2024-03-29T08:09:38.242 INFO:teuthology.orchestra.run.smithi037.stdout:mon.smithi148 smithi148 running (11m) 6s ago 11m 42.0M 2048M 17.2.7-789-g4213e17c cf22eb8683be 18839ed2952a 2024-03-29T08:09:38.242 INFO:teuthology.orchestra.run.smithi037.stdout:node-exporter.smithi037 smithi037 *:9100 running (17s) 4s ago 12m 6593k - 1.5.0 0da6a335fe13 26b8ae9cb878 2024-03-29T08:09:38.242 INFO:teuthology.orchestra.run.smithi037.stdout:node-exporter.smithi148 smithi148 *:9100 running (11s) 6s ago 11m 2693k - 1.5.0 0da6a335fe13 98414e4e64ed 2024-03-29T08:09:38.242 INFO:teuthology.orchestra.run.smithi037.stdout:osd.0 smithi037 running (10m) 4s ago 10m 216M 1619M 17.2.7-789-g4213e17c cf22eb8683be 559fccd0ebe1 2024-03-29T08:09:38.242 INFO:teuthology.orchestra.run.smithi037.stdout:osd.1 smithi037 running (9m) 4s ago 9m 227M 1619M 17.2.7-789-g4213e17c cf22eb8683be 5582cff11b7a 2024-03-29T08:09:38.242 INFO:teuthology.orchestra.run.smithi037.stdout:osd.2 smithi037 running (9m) 4s ago 9m 195M 1619M 17.2.7-789-g4213e17c cf22eb8683be f809bfab13e5 2024-03-29T08:09:38.242 INFO:teuthology.orchestra.run.smithi037.stdout:osd.3 smithi148 running (8m) 6s ago 8m 259M 2642M 17.2.7-789-g4213e17c cf22eb8683be a0a8224bb4b0 2024-03-29T08:09:38.242 INFO:teuthology.orchestra.run.smithi037.stdout:osd.4 smithi148 running (8m) 6s ago 8m 230M 2642M 17.2.7-789-g4213e17c cf22eb8683be a00b69047ed7 2024-03-29T08:09:38.242 INFO:teuthology.orchestra.run.smithi037.stdout:osd.5 smithi148 running (7m) 6s ago 7m 231M 2642M 17.2.7-789-g4213e17c cf22eb8683be 044a3f7db7f8 2024-03-29T08:09:38.242 INFO:teuthology.orchestra.run.smithi037.stdout:prometheus.smithi037 smithi037 *:9095 running (45s) 4s ago 11m 43.7M - 2.43.0 a07b618ecd1d 2e4b5e74f4a4 2024-03-29T08:09:38.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:38.124+0000 7f25f2a00700 1 -- 172.21.15.37:0/840560629 <== mgr.14686 v2:172.21.15.37:6800/4188329516 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7f25fc1117e0 con 0x7f25e006c9c0 2024-03-29T08:09:38.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:38.126+0000 7f25fbe00700 1 -- 172.21.15.37:0/840560629 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7f25e006c9c0 msgr2=0x7f25e006ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:09:38.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:38.126+0000 7f25fbe00700 1 --2- 172.21.15.37:0/840560629 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7f25e006c9c0 0x7f25e006ee70 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f25e8004f90 tx=0x7f25e8009e30 comp rx=0 tx=0).stop 2024-03-29T08:09:38.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:38.127+0000 7f25fbe00700 1 -- 172.21.15.37:0/840560629 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f25fc066240 msgr2=0x7f25fc10fa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:09:38.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:38.127+0000 7f25fbe00700 1 --2- 172.21.15.37:0/840560629 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f25fc066240 0x7f25fc10fa10 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f25ec002a80 tx=0x7f25ec004600 comp rx=0 tx=0).stop 2024-03-29T08:09:38.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:38.127+0000 7f25fbe00700 1 -- 172.21.15.37:0/840560629 shutdown_connections 2024-03-29T08:09:38.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:38.127+0000 7f25fbe00700 1 --2- 172.21.15.37:0/840560629 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7f25e006c9c0 0x7f25e006ee70 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:09:38.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:38.127+0000 7f25fbe00700 1 --2- 172.21.15.37:0/840560629 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f25fc066240 0x7f25fc10fa10 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:09:38.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:38.127+0000 7f25fbe00700 1 --2- 172.21.15.37:0/840560629 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f25fc0ffde0 0x7f25fc10ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:09:38.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:38.127+0000 7f25fbe00700 1 -- 172.21.15.37:0/840560629 >> 172.21.15.37:0/840560629 conn(0x7f25fc0f7270 msgr2=0x7f25fc0fba80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:09:38.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:38.127+0000 7f25fbe00700 1 -- 172.21.15.37:0/840560629 shutdown_connections 2024-03-29T08:09:38.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:38.127+0000 7f25fbe00700 1 -- 172.21.15.37:0/840560629 wait complete. 2024-03-29T08:09:38.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:38.264+0000 7f8af3200700 1 -- 172.21.15.37:0/1273156 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8af4100080 msgr2=0x7f8af41004f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:09:38.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:38.264+0000 7f8af3200700 1 --2- 172.21.15.37:0/1273156 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8af4100080 0x7f8af41004f0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f8ae0004940 tx=0x7f8ae0009e30 comp rx=0 tx=0).stop 2024-03-29T08:09:38.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:38.264+0000 7f8af3200700 1 -- 172.21.15.37:0/1273156 shutdown_connections 2024-03-29T08:09:38.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:38.264+0000 7f8af3200700 1 --2- 172.21.15.37:0/1273156 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8af4100080 0x7f8af41004f0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:09:38.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:38.264+0000 7f8af3200700 1 --2- 172.21.15.37:0/1273156 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8af40fee80 0x7f8af40ff290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:09:38.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:38.264+0000 7f8af3200700 1 -- 172.21.15.37:0/1273156 >> 172.21.15.37:0/1273156 conn(0x7f8af40fa790 msgr2=0x7f8af40fcb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:09:38.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:38.264+0000 7f8af3200700 1 -- 172.21.15.37:0/1273156 shutdown_connections 2024-03-29T08:09:38.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:38.264+0000 7f8af3200700 1 -- 172.21.15.37:0/1273156 wait complete. 2024-03-29T08:09:38.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:38.265+0000 7f8af3200700 1 Processor -- start 2024-03-29T08:09:38.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:38.265+0000 7f8af3200700 1 -- start start 2024-03-29T08:09:38.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:38.265+0000 7f8af3200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8af40fee80 0x7f8af410f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:09:38.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:38.265+0000 7f8af3200700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8af4100080 0x7f8af410fee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:09:38.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:38.266+0000 7f8af3200700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8af41104e0 con 0x7f8af40fee80 2024-03-29T08:09:38.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:38.266+0000 7f8af3200700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8af4110ce0 con 0x7f8af4100080 2024-03-29T08:09:38.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:38.266+0000 7f8af1e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8af40fee80 0x7f8af410f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:09:38.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:38.266+0000 7f8af1e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8af40fee80 0x7f8af410f9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45476/0 (socket says 172.21.15.37:45476) 2024-03-29T08:09:38.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:38.266+0000 7f8af1e00700 1 -- 172.21.15.37:0/1345860978 learned_addr learned my addr 172.21.15.37:0/1345860978 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:09:38.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:38.266+0000 7f8af1400700 1 --2- 172.21.15.37:0/1345860978 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8af4100080 0x7f8af410fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:09:38.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:38.266+0000 7f8af1e00700 1 -- 172.21.15.37:0/1345860978 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8af4100080 msgr2=0x7f8af410fee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:09:38.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:38.266+0000 7f8af1e00700 1 --2- 172.21.15.37:0/1345860978 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8af4100080 0x7f8af410fee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:09:38.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:38.266+0000 7f8af1e00700 1 -- 172.21.15.37:0/1345860978 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ae0005040 con 0x7f8af40fee80 2024-03-29T08:09:38.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:38.266+0000 7f8af1400700 1 --2- 172.21.15.37:0/1345860978 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8af4100080 0x7f8af410fee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:09:38.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:38.266+0000 7f8af1e00700 1 --2- 172.21.15.37:0/1345860978 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8af40fee80 0x7f8af410f9a0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f8ae4002a80 tx=0x7f8ae4004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:09:38.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:38.267+0000 7f8aea000700 1 -- 172.21.15.37:0/1345860978 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8ae4007cc0 con 0x7f8af40fee80 2024-03-29T08:09:38.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:38.267+0000 7f8aea000700 1 -- 172.21.15.37:0/1345860978 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f8ae400f040 con 0x7f8af40fee80 2024-03-29T08:09:38.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:38.267+0000 7f8af3200700 1 -- 172.21.15.37:0/1345860978 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8af4110fc0 con 0x7f8af40fee80 2024-03-29T08:09:38.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:38.267+0000 7f8aea000700 1 -- 172.21.15.37:0/1345860978 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8ae400da20 con 0x7f8af40fee80 2024-03-29T08:09:38.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:38.267+0000 7f8af3200700 1 -- 172.21.15.37:0/1345860978 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8af4111500 con 0x7f8af40fee80 2024-03-29T08:09:38.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:38.268+0000 7f8af3200700 1 -- 172.21.15.37:0/1345860978 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8af40fe290 con 0x7f8af40fee80 2024-03-29T08:09:38.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:38.269+0000 7f8aea000700 1 -- 172.21.15.37:0/1345860978 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 33) v1 ==== 90805+0+0 (secure 0 0 0) 0x7f8ae400db80 con 0x7f8af40fee80 2024-03-29T08:09:38.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:38.270+0000 7f8aea000700 1 --2- 172.21.15.37:0/1345860978 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7f8ad806c9d0 0x7f8ad806ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:09:38.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:38.270+0000 7f8aea000700 1 -- 172.21.15.37:0/1345860978 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f8ae408bbb0 con 0x7f8af40fee80 2024-03-29T08:09:38.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:38.270+0000 7f8af1400700 1 --2- 172.21.15.37:0/1345860978 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7f8ad806c9d0 0x7f8ad806ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:09:38.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:38.271+0000 7f8af1400700 1 --2- 172.21.15.37:0/1345860978 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7f8ad806c9d0 0x7f8ad806ee80 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f8ae0004f90 tx=0x7f8ae0009e30 comp rx=0 tx=0).ready entity=mgr.14686 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:09:38.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:38.274+0000 7f8aea000700 1 -- 172.21.15.37:0/1345860978 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181579 (secure 0 0 0) 0x7f8ae4059860 con 0x7f8af40fee80 2024-03-29T08:09:38.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:09:38 smithi037 ceph-mon[35162]: pgmap v48: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:09:38.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:09:38 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' 2024-03-29T08:09:38.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:09:38 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' 2024-03-29T08:09:38.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:09:38 smithi148 ceph-mon[44992]: pgmap v48: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:09:38.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:09:38 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' 2024-03-29T08:09:38.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:09:38 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' 2024-03-29T08:09:38.584 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-29T08:09:38.585 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": { 2024-03-29T08:09:38.585 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 17.2.7-789-g4213e17c (4213e17c4de67016ed14f55d22503f4923b3a901) quincy (stable)": 2 2024-03-29T08:09:38.585 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:09:38.585 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": { 2024-03-29T08:09:38.585 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 2 2024-03-29T08:09:38.585 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:09:38.585 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": { 2024-03-29T08:09:38.585 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 17.2.7-789-g4213e17c (4213e17c4de67016ed14f55d22503f4923b3a901) quincy (stable)": 6 2024-03-29T08:09:38.585 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:09:38.585 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": { 2024-03-29T08:09:38.585 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 17.2.7-789-g4213e17c (4213e17c4de67016ed14f55d22503f4923b3a901) quincy (stable)": 4 2024-03-29T08:09:38.585 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:09:38.585 INFO:teuthology.orchestra.run.smithi037.stdout: "overall": { 2024-03-29T08:09:38.585 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 17.2.7-789-g4213e17c (4213e17c4de67016ed14f55d22503f4923b3a901) quincy (stable)": 12, 2024-03-29T08:09:38.585 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 2 2024-03-29T08:09:38.586 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-29T08:09:38.586 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-29T08:09:38.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:38.582+0000 7f8af3200700 1 -- 172.21.15.37:0/1345860978 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f8af404bed0 con 0x7f8af40fee80 2024-03-29T08:09:38.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:38.583+0000 7f8aea000700 1 -- 172.21.15.37:0/1345860978 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+743 (secure 0 0 0) 0x7f8ae40593f0 con 0x7f8af40fee80 2024-03-29T08:09:38.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:38.585+0000 7f8af3200700 1 -- 172.21.15.37:0/1345860978 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7f8ad806c9d0 msgr2=0x7f8ad806ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:09:38.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:38.585+0000 7f8af3200700 1 --2- 172.21.15.37:0/1345860978 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7f8ad806c9d0 0x7f8ad806ee80 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f8ae0004f90 tx=0x7f8ae0009e30 comp rx=0 tx=0).stop 2024-03-29T08:09:38.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:38.585+0000 7f8af3200700 1 -- 172.21.15.37:0/1345860978 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8af40fee80 msgr2=0x7f8af410f9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:09:38.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:38.585+0000 7f8af3200700 1 --2- 172.21.15.37:0/1345860978 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8af40fee80 0x7f8af410f9a0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f8ae4002a80 tx=0x7f8ae4004600 comp rx=0 tx=0).stop 2024-03-29T08:09:38.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:38.586+0000 7f8af3200700 1 -- 172.21.15.37:0/1345860978 shutdown_connections 2024-03-29T08:09:38.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:38.586+0000 7f8af3200700 1 --2- 172.21.15.37:0/1345860978 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7f8ad806c9d0 0x7f8ad806ee80 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:09:38.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:38.586+0000 7f8af3200700 1 --2- 172.21.15.37:0/1345860978 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8af40fee80 0x7f8af410f9a0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:09:38.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:38.586+0000 7f8af3200700 1 --2- 172.21.15.37:0/1345860978 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8af4100080 0x7f8af410fee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:09:38.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:38.586+0000 7f8af3200700 1 -- 172.21.15.37:0/1345860978 >> 172.21.15.37:0/1345860978 conn(0x7f8af40fa790 msgr2=0x7f8af40fce90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:09:38.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:38.586+0000 7f8af3200700 1 -- 172.21.15.37:0/1345860978 shutdown_connections 2024-03-29T08:09:38.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:38.586+0000 7f8af3200700 1 -- 172.21.15.37:0/1345860978 wait complete. 2024-03-29T08:09:38.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:38.723+0000 7fe3d8800700 1 -- 172.21.15.37:0/520354038 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe3d0100290 msgr2=0x7fe3d0100700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:09:38.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:38.723+0000 7fe3d8800700 1 --2- 172.21.15.37:0/520354038 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe3d0100290 0x7fe3d0100700 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fe3c0004940 tx=0x7fe3c0009e30 comp rx=0 tx=0).stop 2024-03-29T08:09:38.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:38.723+0000 7fe3d8800700 1 -- 172.21.15.37:0/520354038 shutdown_connections 2024-03-29T08:09:38.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:38.723+0000 7fe3d8800700 1 --2- 172.21.15.37:0/520354038 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe3d0100290 0x7fe3d0100700 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:09:38.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:38.723+0000 7fe3d8800700 1 --2- 172.21.15.37:0/520354038 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe3d00fefe0 0x7fe3d00ff3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:09:38.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:38.723+0000 7fe3d8800700 1 -- 172.21.15.37:0/520354038 >> 172.21.15.37:0/520354038 conn(0x7fe3d00fa790 msgr2=0x7fe3d00fcb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:09:38.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:38.723+0000 7fe3d8800700 1 -- 172.21.15.37:0/520354038 shutdown_connections 2024-03-29T08:09:38.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:38.723+0000 7fe3d8800700 1 -- 172.21.15.37:0/520354038 wait complete. 2024-03-29T08:09:38.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:38.724+0000 7fe3d8800700 1 Processor -- start 2024-03-29T08:09:38.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:38.724+0000 7fe3d8800700 1 -- start start 2024-03-29T08:09:38.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:38.725+0000 7fe3d8800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe3d00fefe0 0x7fe3d010f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:09:38.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:38.725+0000 7fe3d8800700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe3d0100290 0x7fe3d010fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:09:38.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:38.725+0000 7fe3d8800700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3d01104f0 con 0x7fe3d00fefe0 2024-03-29T08:09:38.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:38.725+0000 7fe3d8800700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3d0110660 con 0x7fe3d0100290 2024-03-29T08:09:38.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:38.725+0000 7fe3d5e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe3d00fefe0 0x7fe3d010f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:09:38.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:38.725+0000 7fe3d5e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe3d00fefe0 0x7fe3d010f9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45490/0 (socket says 172.21.15.37:45490) 2024-03-29T08:09:38.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:38.725+0000 7fe3d5e00700 1 -- 172.21.15.37:0/10403769 learned_addr learned my addr 172.21.15.37:0/10403769 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:09:38.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:38.725+0000 7fe3d5400700 1 --2- 172.21.15.37:0/10403769 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe3d0100290 0x7fe3d010fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:09:38.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:38.725+0000 7fe3d5e00700 1 -- 172.21.15.37:0/10403769 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe3d0100290 msgr2=0x7fe3d010fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:09:38.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:38.725+0000 7fe3d5e00700 1 --2- 172.21.15.37:0/10403769 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe3d0100290 0x7fe3d010fef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:09:38.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:38.725+0000 7fe3d5e00700 1 -- 172.21.15.37:0/10403769 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3c0005040 con 0x7fe3d00fefe0 2024-03-29T08:09:38.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:38.726+0000 7fe3d5e00700 1 --2- 172.21.15.37:0/10403769 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe3d00fefe0 0x7fe3d010f9b0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fe3c400eb70 tx=0x7fe3c400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:09:38.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:38.726+0000 7fe3ca000700 1 -- 172.21.15.37:0/10403769 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe3c4007cc0 con 0x7fe3d00fefe0 2024-03-29T08:09:38.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:38.726+0000 7fe3ca000700 1 -- 172.21.15.37:0/10403769 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fe3c400f040 con 0x7fe3d00fefe0 2024-03-29T08:09:38.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:38.726+0000 7fe3d8800700 1 -- 172.21.15.37:0/10403769 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3d0115060 con 0x7fe3d00fefe0 2024-03-29T08:09:38.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:38.726+0000 7fe3ca000700 1 -- 172.21.15.37:0/10403769 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe3c40095f0 con 0x7fe3d00fefe0 2024-03-29T08:09:38.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:38.726+0000 7fe3d8800700 1 -- 172.21.15.37:0/10403769 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3d0115620 con 0x7fe3d00fefe0 2024-03-29T08:09:38.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:38.727+0000 7fe3d8800700 1 -- 172.21.15.37:0/10403769 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3d004c710 con 0x7fe3d00fefe0 2024-03-29T08:09:38.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:38.728+0000 7fe3ca000700 1 -- 172.21.15.37:0/10403769 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 33) v1 ==== 90805+0+0 (secure 0 0 0) 0x7fe3c4009750 con 0x7fe3d00fefe0 2024-03-29T08:09:38.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:38.729+0000 7fe3ca000700 1 --2- 172.21.15.37:0/10403769 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7fe3b806c9d0 0x7fe3b806ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:09:38.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:38.729+0000 7fe3ca000700 1 -- 172.21.15.37:0/10403769 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6066+0+0 (secure 0 0 0) 0x7fe3c408ba40 con 0x7fe3d00fefe0 2024-03-29T08:09:38.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:38.729+0000 7fe3d5400700 1 --2- 172.21.15.37:0/10403769 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7fe3b806c9d0 0x7fe3b806ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:09:38.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:38.730+0000 7fe3d5400700 1 --2- 172.21.15.37:0/10403769 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7fe3b806c9d0 0x7fe3b806ee80 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fe3c000b0d0 tx=0x7fe3c0009e30 comp rx=0 tx=0).ready entity=mgr.14686 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:09:38.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:38.733+0000 7fe3ca000700 1 -- 172.21.15.37:0/10403769 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181579 (secure 0 0 0) 0x7fe3c40596f0 con 0x7fe3d00fefe0 2024-03-29T08:09:38.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:38.966+0000 7fe3d8800700 1 -- 172.21.15.37:0/10403769 --> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe3d01045c0 con 0x7fe3b806c9d0 2024-03-29T08:09:38.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:38.968+0000 7fe3ca000700 1 -- 172.21.15.37:0/10403769 <== mgr.14686 v2:172.21.15.37:6800/4188329516 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+370 (secure 0 0 0) 0x7fe3d01045c0 con 0x7fe3b806c9d0 2024-03-29T08:09:38.969 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-29T08:09:38.969 INFO:teuthology.orchestra.run.smithi037.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59", 2024-03-29T08:09:38.969 INFO:teuthology.orchestra.run.smithi037.stdout: "in_progress": true, 2024-03-29T08:09:38.969 INFO:teuthology.orchestra.run.smithi037.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-03-29T08:09:38.969 INFO:teuthology.orchestra.run.smithi037.stdout: "services_complete": [ 2024-03-29T08:09:38.969 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr" 2024-03-29T08:09:38.969 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-29T08:09:38.969 INFO:teuthology.orchestra.run.smithi037.stdout: "progress": "2/2 daemons upgraded", 2024-03-29T08:09:38.969 INFO:teuthology.orchestra.run.smithi037.stdout: "message": "Currently upgrading node-exporter daemons", 2024-03-29T08:09:38.969 INFO:teuthology.orchestra.run.smithi037.stdout: "is_paused": false 2024-03-29T08:09:38.970 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-29T08:09:38.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:38.970+0000 7fe3d8800700 1 -- 172.21.15.37:0/10403769 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7fe3b806c9d0 msgr2=0x7fe3b806ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:09:38.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:38.970+0000 7fe3d8800700 1 --2- 172.21.15.37:0/10403769 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7fe3b806c9d0 0x7fe3b806ee80 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fe3c000b0d0 tx=0x7fe3c0009e30 comp rx=0 tx=0).stop 2024-03-29T08:09:38.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:38.970+0000 7fe3d8800700 1 -- 172.21.15.37:0/10403769 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe3d00fefe0 msgr2=0x7fe3d010f9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:09:38.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:38.970+0000 7fe3d8800700 1 --2- 172.21.15.37:0/10403769 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe3d00fefe0 0x7fe3d010f9b0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fe3c400eb70 tx=0x7fe3c400b040 comp rx=0 tx=0).stop 2024-03-29T08:09:38.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:38.970+0000 7fe3d8800700 1 -- 172.21.15.37:0/10403769 shutdown_connections 2024-03-29T08:09:38.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:38.970+0000 7fe3d8800700 1 --2- 172.21.15.37:0/10403769 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7fe3b806c9d0 0x7fe3b806ee80 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:09:38.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:38.970+0000 7fe3d8800700 1 --2- 172.21.15.37:0/10403769 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe3d00fefe0 0x7fe3d010f9b0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:09:38.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:38.970+0000 7fe3d8800700 1 --2- 172.21.15.37:0/10403769 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe3d0100290 0x7fe3d010fef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:09:38.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:38.970+0000 7fe3d8800700 1 -- 172.21.15.37:0/10403769 >> 172.21.15.37:0/10403769 conn(0x7fe3d00fa790 msgr2=0x7fe3d00fbe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:09:38.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:38.971+0000 7fe3d8800700 1 -- 172.21.15.37:0/10403769 shutdown_connections 2024-03-29T08:09:38.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:09:38.971+0000 7fe3d8800700 1 -- 172.21.15.37:0/10403769 wait complete. 2024-03-29T08:09:39.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:09:39 smithi037 ceph-mon[35162]: from='client.14762 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:09:39.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:09:39 smithi037 ceph-mon[35162]: from='client.14766 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:09:39.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:09:39 smithi037 ceph-mon[35162]: from='client.14770 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:09:39.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:09:39 smithi037 ceph-mon[35162]: from='client.? 172.21.15.37:0/1345860978' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:09:39.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:09:39 smithi148 ceph-mon[44992]: from='client.14762 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:09:39.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:09:39 smithi148 ceph-mon[44992]: from='client.14766 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:09:39.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:09:39 smithi148 ceph-mon[44992]: from='client.14770 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:09:39.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:09:39 smithi148 ceph-mon[44992]: from='client.? 172.21.15.37:0/1345860978' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:09:40.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:09:40 smithi037 ceph-mon[35162]: pgmap v49: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-03-29T08:09:40.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:09:40 smithi037 ceph-mon[35162]: from='client.14778 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:09:40.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:09:40 smithi037 ceph-mon[35162]: mgrmap e34: smithi037.wjikii(active, since 92s), standbys: smithi148.onhgvm 2024-03-29T08:09:40.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:09:40 smithi148 ceph-mon[44992]: pgmap v49: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-03-29T08:09:40.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:09:40 smithi148 ceph-mon[44992]: from='client.14778 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:09:40.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:09:40 smithi148 ceph-mon[44992]: mgrmap e34: smithi037.wjikii(active, since 92s), standbys: smithi148.onhgvm 2024-03-29T08:09:42.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:09:42 smithi037 ceph-mon[35162]: pgmap v50: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-03-29T08:09:42.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:09:42 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' 2024-03-29T08:09:42.428 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:09:42 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' 2024-03-29T08:09:42.428 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:09:42 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T08:09:42.428 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:09:42 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T08:09:42.428 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:09:42 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' 2024-03-29T08:09:42.428 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:09:42 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T08:09:42.428 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:09:42 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:09:42.428 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:09:42 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:09:42.428 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:09:42 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:09:42.428 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:09:42 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:09:42.428 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:09:42 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:09:42.428 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:09:42 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:09:42.428 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:09:42 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:09:42.428 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:09:42 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:09:42.428 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:09:42 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:09:42.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:09:42 smithi148 ceph-mon[44992]: pgmap v50: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-03-29T08:09:42.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:09:42 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' 2024-03-29T08:09:42.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:09:42 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' 2024-03-29T08:09:42.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:09:42 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T08:09:42.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:09:42 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T08:09:42.557 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:09:42 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' 2024-03-29T08:09:42.557 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:09:42 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T08:09:42.557 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:09:42 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:09:42.557 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:09:42 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:09:42.557 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:09:42 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:09:42.557 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:09:42 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:09:42.557 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:09:42 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:09:42.557 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:09:42 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:09:42.557 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:09:42 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:09:42.557 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:09:42 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:09:42.557 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:09:42 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:09:44.500 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:09:44 smithi037 ceph-mon[35162]: pgmap v51: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-03-29T08:09:44.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:09:44 smithi148 ceph-mon[44992]: pgmap v51: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-03-29T08:09:45.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:09:45 smithi148 ceph-mon[44992]: Upgrade: Updating prometheus.smithi037 2024-03-29T08:09:45.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:09:45 smithi148 ceph-mon[44992]: Deploying daemon prometheus.smithi037 on smithi037 2024-03-29T08:09:45.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:09:45 smithi037 ceph-mon[35162]: Upgrade: Updating prometheus.smithi037 2024-03-29T08:09:45.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:09:45 smithi037 ceph-mon[35162]: Deploying daemon prometheus.smithi037 on smithi037 2024-03-29T08:09:46.441 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:09:46 smithi037 ceph-mon[35162]: pgmap v52: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-03-29T08:09:46.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:09:46 smithi148 ceph-mon[44992]: pgmap v52: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-03-29T08:09:48.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:09:48 smithi037 ceph-mon[35162]: pgmap v53: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-03-29T08:09:48.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:09:48 smithi148 ceph-mon[44992]: pgmap v53: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-03-29T08:09:50.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:09:50 smithi148 ceph-mon[44992]: pgmap v54: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-03-29T08:09:50.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:09:50 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' 2024-03-29T08:09:50.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:09:50 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' 2024-03-29T08:09:50.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:09:50 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T08:09:50.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:09:50 smithi037 ceph-mon[35162]: pgmap v54: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-03-29T08:09:50.678 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:09:50 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' 2024-03-29T08:09:50.678 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:09:50 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' 2024-03-29T08:09:50.678 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:09:50 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T08:09:52.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:09:52 smithi148 ceph-mon[44992]: pgmap v55: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:09:52.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:09:52 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:09:52.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:09:52 smithi037 ceph-mon[35162]: pgmap v55: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:09:52.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:09:52 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:09:54.430 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:09:54 smithi037 ceph-mon[35162]: pgmap v56: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:09:54.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:09:54 smithi148 ceph-mon[44992]: pgmap v56: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:09:56.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:09:56 smithi148 ceph-mon[44992]: pgmap v57: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:09:56.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:09:56 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' 2024-03-29T08:09:56.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:09:56 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' 2024-03-29T08:09:56.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:09:56 smithi037 ceph-mon[35162]: pgmap v57: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:09:56.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:09:56 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' 2024-03-29T08:09:56.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:09:56 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' 2024-03-29T08:09:58.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:09:57 smithi148 ceph-mon[44992]: pgmap v58: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:09:58.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:09:57 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' 2024-03-29T08:09:58.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:09:57 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' 2024-03-29T08:09:58.334 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:09:57 smithi037 ceph-mon[35162]: pgmap v58: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:09:58.334 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:09:57 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' 2024-03-29T08:09:58.334 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:09:57 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' 2024-03-29T08:10:00.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:09:59 smithi148 ceph-mon[44992]: pgmap v59: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:10:00.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:09:59 smithi037 ceph-mon[35162]: pgmap v59: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:10:01.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:10:00 smithi037 ceph-mon[35162]: overall HEALTH_OK 2024-03-29T08:10:01.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:10:00 smithi148 ceph-mon[44992]: overall HEALTH_OK 2024-03-29T08:10:02.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:10:01 smithi148 ceph-mon[44992]: pgmap v60: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:10:02.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:10:01 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' 2024-03-29T08:10:02.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:10:01 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' 2024-03-29T08:10:02.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:10:01 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T08:10:02.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:10:01 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T08:10:02.307 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:10:01 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' 2024-03-29T08:10:02.307 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:10:01 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-29T08:10:02.307 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:10:01 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T08:10:02.307 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:10:01 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:10:02.307 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:10:01 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:10:02.307 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:10:01 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:10:02.307 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:10:01 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:10:02.307 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:10:01 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:10:02.307 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:10:01 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:10:02.307 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:10:01 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:10:02.307 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:10:01 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:10:02.307 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:10:01 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:10:02.307 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:10:01 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:10:02.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:10:01 smithi037 ceph-mon[35162]: pgmap v60: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:10:02.428 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:10:01 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' 2024-03-29T08:10:02.428 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:10:01 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' 2024-03-29T08:10:02.428 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:10:01 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T08:10:02.428 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:10:01 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T08:10:02.428 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:10:01 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' 2024-03-29T08:10:02.428 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:10:01 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-29T08:10:02.428 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:10:01 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T08:10:02.428 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:10:01 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:10:02.429 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:10:01 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:10:02.429 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:10:01 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:10:02.429 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:10:01 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:10:02.429 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:10:01 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:10:02.429 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:10:01 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:10:02.429 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:10:01 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:10:02.429 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:10:01 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:10:02.429 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:10:01 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:10:02.429 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:10:01 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:10:03.232 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:10:02 smithi037 ceph-mon[35162]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-29T08:10:03.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:10:02 smithi148 ceph-mon[44992]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-29T08:10:04.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:10:03 smithi148 ceph-mon[44992]: pgmap v61: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:10:04.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:10:03 smithi037 ceph-mon[35162]: pgmap v61: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:10:05.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:10:04 smithi148 ceph-mon[44992]: Upgrade: Updating alertmanager.smithi037 2024-03-29T08:10:05.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:10:04 smithi148 ceph-mon[44992]: Deploying daemon alertmanager.smithi037 on smithi037 2024-03-29T08:10:05.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:10:04 smithi037 ceph-mon[35162]: Upgrade: Updating alertmanager.smithi037 2024-03-29T08:10:05.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:10:04 smithi037 ceph-mon[35162]: Deploying daemon alertmanager.smithi037 on smithi037 2024-03-29T08:10:06.221 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:10:05 smithi037 ceph-mon[35162]: pgmap v62: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:10:06.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:10:05 smithi148 ceph-mon[44992]: pgmap v62: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:10:07.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:10:06 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:10:07.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:10:06 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:10:08.114 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:10:07 smithi037 ceph-mon[35162]: pgmap v63: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:10:08.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:10:07 smithi148 ceph-mon[44992]: pgmap v63: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:10:09.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:09.115+0000 7f17fe000700 1 -- 172.21.15.37:0/590483319 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f17f80709a0 msgr2=0x7f17f8070e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:10:09.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:09.115+0000 7f17fe000700 1 --2- 172.21.15.37:0/590483319 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f17f80709a0 0x7f17f8070e10 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f17e4004940 tx=0x7f17e4009e30 comp rx=0 tx=0).stop 2024-03-29T08:10:09.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:09.116+0000 7f17fe000700 1 -- 172.21.15.37:0/590483319 shutdown_connections 2024-03-29T08:10:09.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:09.116+0000 7f17fe000700 1 --2- 172.21.15.37:0/590483319 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f17f80709a0 0x7f17f8070e10 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:10:09.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:09.116+0000 7f17fe000700 1 --2- 172.21.15.37:0/590483319 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f17f8071f50 0x7f17f8070460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:10:09.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:09.116+0000 7f17fe000700 1 -- 172.21.15.37:0/590483319 >> 172.21.15.37:0/590483319 conn(0x7f17f80f8560 msgr2=0x7f17f80fa970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:10:09.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:09.116+0000 7f17fe000700 1 -- 172.21.15.37:0/590483319 shutdown_connections 2024-03-29T08:10:09.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:09.116+0000 7f17fe000700 1 -- 172.21.15.37:0/590483319 wait complete. 2024-03-29T08:10:09.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:09.117+0000 7f17fe000700 1 Processor -- start 2024-03-29T08:10:09.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:09.117+0000 7f17fe000700 1 -- start start 2024-03-29T08:10:09.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:09.117+0000 7f17fe000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f17f80709a0 0x7f17f810fb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:10:09.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:09.117+0000 7f17fe000700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f17f8071f50 0x7f17f8110050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:10:09.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:09.117+0000 7f17fe000700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17f8110650 con 0x7f17f80709a0 2024-03-29T08:10:09.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:09.117+0000 7f17fe000700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17f8110ea0 con 0x7f17f8071f50 2024-03-29T08:10:09.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:09.117+0000 7f17f7400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f17f80709a0 0x7f17f810fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:10:09.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:09.118+0000 7f17f7400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f17f80709a0 0x7f17f810fb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58864/0 (socket says 172.21.15.37:58864) 2024-03-29T08:10:09.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:09.118+0000 7f17f7400700 1 -- 172.21.15.37:0/869720636 learned_addr learned my addr 172.21.15.37:0/869720636 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:10:09.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:09.118+0000 7f17f6a00700 1 --2- 172.21.15.37:0/869720636 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f17f8071f50 0x7f17f8110050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:10:09.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:09.118+0000 7f17f7400700 1 -- 172.21.15.37:0/869720636 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f17f8071f50 msgr2=0x7f17f8110050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:10:09.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:09.118+0000 7f17f7400700 1 --2- 172.21.15.37:0/869720636 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f17f8071f50 0x7f17f8110050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:10:09.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:09.118+0000 7f17f7400700 1 -- 172.21.15.37:0/869720636 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17e4005040 con 0x7f17f80709a0 2024-03-29T08:10:09.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:09.118+0000 7f17f7400700 1 --2- 172.21.15.37:0/869720636 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f17f80709a0 0x7f17f810fb10 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f17e8002a80 tx=0x7f17e8004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:10:09.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:09.118+0000 7f17ef400700 1 -- 172.21.15.37:0/869720636 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f17e8007cc0 con 0x7f17f80709a0 2024-03-29T08:10:09.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:09.119+0000 7f17fe000700 1 -- 172.21.15.37:0/869720636 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17f8111180 con 0x7f17f80709a0 2024-03-29T08:10:09.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:09.119+0000 7f17ef400700 1 -- 172.21.15.37:0/869720636 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f17e800f040 con 0x7f17f80709a0 2024-03-29T08:10:09.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:09.119+0000 7f17ef400700 1 -- 172.21.15.37:0/869720636 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f17e800d9c0 con 0x7f17f80709a0 2024-03-29T08:10:09.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:09.119+0000 7f17fe000700 1 -- 172.21.15.37:0/869720636 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17f81116c0 con 0x7f17f80709a0 2024-03-29T08:10:09.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:09.120+0000 7f17fe000700 1 -- 172.21.15.37:0/869720636 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17f805f5f0 con 0x7f17f80709a0 2024-03-29T08:10:09.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:09.121+0000 7f17ef400700 1 -- 172.21.15.37:0/869720636 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 34) v1 ==== 90703+0+0 (secure 0 0 0) 0x7f17e800db20 con 0x7f17f80709a0 2024-03-29T08:10:09.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:09.122+0000 7f17ef400700 1 --2- 172.21.15.37:0/869720636 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7f17dc06c8b0 0x7f17dc06ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:10:09.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:09.122+0000 7f17ef400700 1 -- 172.21.15.37:0/869720636 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f17e808b890 con 0x7f17f80709a0 2024-03-29T08:10:09.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:09.122+0000 7f17f6a00700 1 --2- 172.21.15.37:0/869720636 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7f17dc06c8b0 0x7f17dc06ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:10:09.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:09.123+0000 7f17f6a00700 1 --2- 172.21.15.37:0/869720636 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7f17dc06c8b0 0x7f17dc06ed60 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f17e400b3b0 tx=0x7f17e4009e30 comp rx=0 tx=0).ready entity=mgr.14686 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:10:09.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:09.126+0000 7f17ef400700 1 -- 172.21.15.37:0/869720636 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181579 (secure 0 0 0) 0x7f17e8059800 con 0x7f17f80709a0 2024-03-29T08:10:09.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:09.378+0000 7f17fe000700 1 -- 172.21.15.37:0/869720636 --> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f17f8111a40 con 0x7f17dc06c8b0 2024-03-29T08:10:09.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:09.379+0000 7f17ef400700 1 -- 172.21.15.37:0/869720636 <== mgr.14686 v2:172.21.15.37:6800/4188329516 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+369 (secure 0 0 0) 0x7f17f8111a40 con 0x7f17dc06c8b0 2024-03-29T08:10:09.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:09.382+0000 7f17fe000700 1 -- 172.21.15.37:0/869720636 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7f17dc06c8b0 msgr2=0x7f17dc06ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:10:09.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:09.382+0000 7f17fe000700 1 --2- 172.21.15.37:0/869720636 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7f17dc06c8b0 0x7f17dc06ed60 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f17e400b3b0 tx=0x7f17e4009e30 comp rx=0 tx=0).stop 2024-03-29T08:10:09.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:09.382+0000 7f17fe000700 1 -- 172.21.15.37:0/869720636 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f17f80709a0 msgr2=0x7f17f810fb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:10:09.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:09.382+0000 7f17fe000700 1 --2- 172.21.15.37:0/869720636 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f17f80709a0 0x7f17f810fb10 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f17e8002a80 tx=0x7f17e8004600 comp rx=0 tx=0).stop 2024-03-29T08:10:09.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:09.382+0000 7f17fe000700 1 -- 172.21.15.37:0/869720636 shutdown_connections 2024-03-29T08:10:09.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:09.382+0000 7f17fe000700 1 --2- 172.21.15.37:0/869720636 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7f17dc06c8b0 0x7f17dc06ed60 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:10:09.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:09.382+0000 7f17fe000700 1 --2- 172.21.15.37:0/869720636 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f17f80709a0 0x7f17f810fb10 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:10:09.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:09.382+0000 7f17fe000700 1 --2- 172.21.15.37:0/869720636 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f17f8071f50 0x7f17f8110050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:10:09.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:09.382+0000 7f17fe000700 1 -- 172.21.15.37:0/869720636 >> 172.21.15.37:0/869720636 conn(0x7f17f80f8560 msgr2=0x7f17f80f9d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:10:09.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:09.382+0000 7f17fe000700 1 -- 172.21.15.37:0/869720636 shutdown_connections 2024-03-29T08:10:09.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:09.382+0000 7f17fe000700 1 -- 172.21.15.37:0/869720636 wait complete. 2024-03-29T08:10:09.397 INFO:teuthology.orchestra.run.smithi037.stdout:true 2024-03-29T08:10:09.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:09.522+0000 7f6a7ec00700 1 -- 172.21.15.37:0/58319009 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a780662e0 msgr2=0x7f6a780ff9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:10:09.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:09.522+0000 7f6a7ec00700 1 --2- 172.21.15.37:0/58319009 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a780662e0 0x7f6a780ff9c0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f6a68004940 tx=0x7f6a68009e30 comp rx=0 tx=0).stop 2024-03-29T08:10:09.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:09.522+0000 7f6a7ec00700 1 -- 172.21.15.37:0/58319009 shutdown_connections 2024-03-29T08:10:09.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:09.522+0000 7f6a7ec00700 1 --2- 172.21.15.37:0/58319009 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6a780fff00 0x7f6a78102320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:10:09.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:09.522+0000 7f6a7ec00700 1 --2- 172.21.15.37:0/58319009 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a780662e0 0x7f6a780ff9c0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:10:09.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:09.522+0000 7f6a7ec00700 1 -- 172.21.15.37:0/58319009 >> 172.21.15.37:0/58319009 conn(0x7f6a780f7430 msgr2=0x7f6a780f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:10:09.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:09.522+0000 7f6a7ec00700 1 -- 172.21.15.37:0/58319009 shutdown_connections 2024-03-29T08:10:09.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:09.522+0000 7f6a7ec00700 1 -- 172.21.15.37:0/58319009 wait complete. 2024-03-29T08:10:09.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:09.523+0000 7f6a7ec00700 1 Processor -- start 2024-03-29T08:10:09.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:09.523+0000 7f6a7ec00700 1 -- start start 2024-03-29T08:10:09.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:09.523+0000 7f6a7ec00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a780662e0 0x7f6a7810d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:10:09.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:09.523+0000 7f6a7ec00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6a780fff00 0x7f6a7810deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:10:09.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:09.524+0000 7f6a7ec00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a7810e4b0 con 0x7f6a780662e0 2024-03-29T08:10:09.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:09.524+0000 7f6a7ec00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a7810ed00 con 0x7f6a780fff00 2024-03-29T08:10:09.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:09.524+0000 7f6a77e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a780662e0 0x7f6a7810d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:10:09.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:09.524+0000 7f6a77e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a780662e0 0x7f6a7810d970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58886/0 (socket says 172.21.15.37:58886) 2024-03-29T08:10:09.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:09.524+0000 7f6a77e00700 1 -- 172.21.15.37:0/15173116 learned_addr learned my addr 172.21.15.37:0/15173116 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:10:09.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:09.524+0000 7f6a77400700 1 --2- 172.21.15.37:0/15173116 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6a780fff00 0x7f6a7810deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:10:09.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:09.524+0000 7f6a77e00700 1 -- 172.21.15.37:0/15173116 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6a780fff00 msgr2=0x7f6a7810deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:10:09.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:09.524+0000 7f6a77e00700 1 --2- 172.21.15.37:0/15173116 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6a780fff00 0x7f6a7810deb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:10:09.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:09.524+0000 7f6a77e00700 1 -- 172.21.15.37:0/15173116 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a68005040 con 0x7f6a780662e0 2024-03-29T08:10:09.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:09.524+0000 7f6a77e00700 1 --2- 172.21.15.37:0/15173116 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a780662e0 0x7f6a7810d970 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f6a6800b900 tx=0x7f6a68009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:10:09.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:09.525+0000 7f6a6fe00700 1 -- 172.21.15.37:0/15173116 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6a6800e040 con 0x7f6a780662e0 2024-03-29T08:10:09.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:09.525+0000 7f6a6fe00700 1 -- 172.21.15.37:0/15173116 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f6a68004290 con 0x7f6a780662e0 2024-03-29T08:10:09.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:09.525+0000 7f6a6fe00700 1 -- 172.21.15.37:0/15173116 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6a6801c3f0 con 0x7f6a780662e0 2024-03-29T08:10:09.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:09.525+0000 7f6a7ec00700 1 -- 172.21.15.37:0/15173116 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a7810ef80 con 0x7f6a780662e0 2024-03-29T08:10:09.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:09.525+0000 7f6a7ec00700 1 -- 172.21.15.37:0/15173116 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a7810f4c0 con 0x7f6a780662e0 2024-03-29T08:10:09.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:09.526+0000 7f6a7ec00700 1 -- 172.21.15.37:0/15173116 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a7805f5f0 con 0x7f6a780662e0 2024-03-29T08:10:09.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:09.527+0000 7f6a6fe00700 1 -- 172.21.15.37:0/15173116 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 34) v1 ==== 90703+0+0 (secure 0 0 0) 0x7f6a68013420 con 0x7f6a780662e0 2024-03-29T08:10:09.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:09.528+0000 7f6a6fe00700 1 --2- 172.21.15.37:0/15173116 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7f6a5c06c8b0 0x7f6a5c06ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:10:09.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:09.528+0000 7f6a6fe00700 1 -- 172.21.15.37:0/15173116 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f6a6808cdd0 con 0x7f6a780662e0 2024-03-29T08:10:09.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:09.528+0000 7f6a77400700 1 --2- 172.21.15.37:0/15173116 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7f6a5c06c8b0 0x7f6a5c06ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:10:09.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:09.529+0000 7f6a77400700 1 --2- 172.21.15.37:0/15173116 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7f6a5c06c8b0 0x7f6a5c06ed60 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f6a64004990 tx=0x7f6a64009ee0 comp rx=0 tx=0).ready entity=mgr.14686 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:10:09.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:09.532+0000 7f6a6fe00700 1 -- 172.21.15.37:0/15173116 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181579 (secure 0 0 0) 0x7f6a6805ada0 con 0x7f6a780662e0 2024-03-29T08:10:09.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:09.769+0000 7f6a7ec00700 1 -- 172.21.15.37:0/15173116 --> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6a7810f890 con 0x7f6a5c06c8b0 2024-03-29T08:10:09.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:09.770+0000 7f6a6fe00700 1 -- 172.21.15.37:0/15173116 <== mgr.14686 v2:172.21.15.37:6800/4188329516 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+369 (secure 0 0 0) 0x7f6a7810f890 con 0x7f6a5c06c8b0 2024-03-29T08:10:09.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:09.772+0000 7f6a7ec00700 1 -- 172.21.15.37:0/15173116 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7f6a5c06c8b0 msgr2=0x7f6a5c06ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:10:09.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:09.772+0000 7f6a7ec00700 1 --2- 172.21.15.37:0/15173116 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7f6a5c06c8b0 0x7f6a5c06ed60 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f6a64004990 tx=0x7f6a64009ee0 comp rx=0 tx=0).stop 2024-03-29T08:10:09.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:09.772+0000 7f6a7ec00700 1 -- 172.21.15.37:0/15173116 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a780662e0 msgr2=0x7f6a7810d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:10:09.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:09.772+0000 7f6a7ec00700 1 --2- 172.21.15.37:0/15173116 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a780662e0 0x7f6a7810d970 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f6a6800b900 tx=0x7f6a68009e30 comp rx=0 tx=0).stop 2024-03-29T08:10:09.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:09.773+0000 7f6a7ec00700 1 -- 172.21.15.37:0/15173116 shutdown_connections 2024-03-29T08:10:09.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:09.773+0000 7f6a7ec00700 1 --2- 172.21.15.37:0/15173116 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7f6a5c06c8b0 0x7f6a5c06ed60 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:10:09.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:09.773+0000 7f6a7ec00700 1 --2- 172.21.15.37:0/15173116 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a780662e0 0x7f6a7810d970 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:10:09.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:09.773+0000 7f6a7ec00700 1 --2- 172.21.15.37:0/15173116 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6a780fff00 0x7f6a7810deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:10:09.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:09.773+0000 7f6a7ec00700 1 -- 172.21.15.37:0/15173116 >> 172.21.15.37:0/15173116 conn(0x7f6a780f7430 msgr2=0x7f6a780f7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:10:09.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:09.773+0000 7f6a7ec00700 1 -- 172.21.15.37:0/15173116 shutdown_connections 2024-03-29T08:10:09.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:09.773+0000 7f6a7ec00700 1 -- 172.21.15.37:0/15173116 wait complete. 2024-03-29T08:10:09.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:09.911+0000 7fab07e00700 1 -- 172.21.15.37:0/4239498055 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fab08071ea0 msgr2=0x7fab080703b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:10:09.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:09.911+0000 7fab07e00700 1 --2- 172.21.15.37:0/4239498055 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fab08071ea0 0x7fab080703b0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7faaf8004940 tx=0x7faaf8009e30 comp rx=0 tx=0).stop 2024-03-29T08:10:09.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:09.912+0000 7fab07e00700 1 -- 172.21.15.37:0/4239498055 shutdown_connections 2024-03-29T08:10:09.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:09.912+0000 7fab07e00700 1 --2- 172.21.15.37:0/4239498055 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fab080708f0 0x7fab08070d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:10:09.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:09.912+0000 7fab07e00700 1 --2- 172.21.15.37:0/4239498055 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fab08071ea0 0x7fab080703b0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:10:09.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:09.912+0000 7fab07e00700 1 -- 172.21.15.37:0/4239498055 >> 172.21.15.37:0/4239498055 conn(0x7fab080f83d0 msgr2=0x7fab080fa7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:10:09.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:09.912+0000 7fab07e00700 1 -- 172.21.15.37:0/4239498055 shutdown_connections 2024-03-29T08:10:09.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:09.912+0000 7fab07e00700 1 -- 172.21.15.37:0/4239498055 wait complete. 2024-03-29T08:10:09.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:09.912+0000 7fab07e00700 1 Processor -- start 2024-03-29T08:10:09.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:09.913+0000 7fab07e00700 1 -- start start 2024-03-29T08:10:09.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:09.913+0000 7fab07e00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fab080708f0 0x7fab0810f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:10:09.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:09.913+0000 7fab07e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fab08071ea0 0x7fab0810fee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:10:09.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:09.913+0000 7fab07e00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab081104e0 con 0x7fab08071ea0 2024-03-29T08:10:09.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:09.913+0000 7fab07e00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab08110d30 con 0x7fab080708f0 2024-03-29T08:10:09.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:09.913+0000 7fab06000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fab08071ea0 0x7fab0810fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:10:09.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:09.913+0000 7fab06000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fab08071ea0 0x7fab0810fee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58906/0 (socket says 172.21.15.37:58906) 2024-03-29T08:10:09.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:09.913+0000 7fab06000700 1 -- 172.21.15.37:0/986579480 learned_addr learned my addr 172.21.15.37:0/986579480 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:10:09.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:09.914+0000 7fab06a00700 1 --2- 172.21.15.37:0/986579480 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fab080708f0 0x7fab0810f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:10:09.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:09.914+0000 7fab06000700 1 -- 172.21.15.37:0/986579480 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fab080708f0 msgr2=0x7fab0810f9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:10:09.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:09.914+0000 7fab06000700 1 --2- 172.21.15.37:0/986579480 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fab080708f0 0x7fab0810f9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:10:09.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:09.914+0000 7fab06000700 1 -- 172.21.15.37:0/986579480 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faaf8005040 con 0x7fab08071ea0 2024-03-29T08:10:09.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:09.914+0000 7fab06000700 1 --2- 172.21.15.37:0/986579480 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fab08071ea0 0x7fab0810fee0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7faaf4002a50 tx=0x7faaf40046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:10:09.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:09.914+0000 7faafea00700 1 -- 172.21.15.37:0/986579480 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faaf4007cc0 con 0x7fab08071ea0 2024-03-29T08:10:09.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:09.914+0000 7faafea00700 1 -- 172.21.15.37:0/986579480 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7faaf400f040 con 0x7fab08071ea0 2024-03-29T08:10:09.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:09.914+0000 7faafea00700 1 -- 172.21.15.37:0/986579480 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faaf4009540 con 0x7fab08071ea0 2024-03-29T08:10:09.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:09.914+0000 7fab07e00700 1 -- 172.21.15.37:0/986579480 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab08111010 con 0x7fab08071ea0 2024-03-29T08:10:09.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:09.914+0000 7fab07e00700 1 -- 172.21.15.37:0/986579480 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab08111550 con 0x7fab08071ea0 2024-03-29T08:10:09.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:09.916+0000 7fab07e00700 1 -- 172.21.15.37:0/986579480 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab0805f5f0 con 0x7fab08071ea0 2024-03-29T08:10:09.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:09.916+0000 7faafea00700 1 -- 172.21.15.37:0/986579480 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 34) v1 ==== 90703+0+0 (secure 0 0 0) 0x7faaf40096a0 con 0x7fab08071ea0 2024-03-29T08:10:09.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:09.917+0000 7faafea00700 1 --2- 172.21.15.37:0/986579480 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7faaec06c8b0 0x7faaec06ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:10:09.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:09.917+0000 7faafea00700 1 -- 172.21.15.37:0/986579480 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6066+0+0 (secure 0 0 0) 0x7faaf408b850 con 0x7fab08071ea0 2024-03-29T08:10:09.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:09.918+0000 7fab06a00700 1 --2- 172.21.15.37:0/986579480 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7faaec06c8b0 0x7faaec06ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:10:09.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:09.918+0000 7fab06a00700 1 --2- 172.21.15.37:0/986579480 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7faaec06c8b0 0x7faaec06ed60 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7faaf8005010 tx=0x7faaf8009e30 comp rx=0 tx=0).ready entity=mgr.14686 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:10:09.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:09.922+0000 7faafea00700 1 -- 172.21.15.37:0/986579480 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181579 (secure 0 0 0) 0x7faaf40597a0 con 0x7fab08071ea0 2024-03-29T08:10:10.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:10.162+0000 7fab07e00700 1 -- 172.21.15.37:0/986579480 --> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fab08111930 con 0x7faaec06c8b0 2024-03-29T08:10:10.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:10.176+0000 7faafea00700 1 -- 172.21.15.37:0/986579480 <== mgr.14686 v2:172.21.15.37:6800/4188329516 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7fab08111930 con 0x7faaec06c8b0 2024-03-29T08:10:10.176 INFO:teuthology.orchestra.run.smithi037.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-29T08:10:10.176 INFO:teuthology.orchestra.run.smithi037.stdout:alertmanager.smithi037 smithi037 *:9093,9094 running (3m) 14s ago 12m 15.1M - 0.25.0 c8568f914cd2 8a8d40a6e26e 2024-03-29T08:10:10.176 INFO:teuthology.orchestra.run.smithi037.stdout:crash.smithi037 smithi037 running (13m) 14s ago 13m 6656k - 17.2.7-789-g4213e17c cf22eb8683be e0343935b25e 2024-03-29T08:10:10.176 INFO:teuthology.orchestra.run.smithi037.stdout:crash.smithi148 smithi148 running (12m) 38s ago 12m 6656k - 17.2.7-789-g4213e17c cf22eb8683be ff5164c552c1 2024-03-29T08:10:10.176 INFO:teuthology.orchestra.run.smithi037.stdout:grafana.smithi037 smithi037 *:3000 running (3m) 14s ago 12m 74.7M - 9.4.7 954c08fa6188 a857385eee0c 2024-03-29T08:10:10.176 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi037.iifsmh smithi037 running (6m) 14s ago 6m 110M - 17.2.7-789-g4213e17c cf22eb8683be 074e1acf331f 2024-03-29T08:10:10.176 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi037.mmwgxe smithi037 running (6m) 14s ago 6m 115M - 17.2.7-789-g4213e17c cf22eb8683be 9a5b1e72de5c 2024-03-29T08:10:10.176 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi148.qukpjh smithi148 running (6m) 38s ago 6m 12.9M - 17.2.7-789-g4213e17c cf22eb8683be 8e469782a273 2024-03-29T08:10:10.176 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi148.sqteih smithi148 running (6m) 38s ago 6m 12.8M - 17.2.7-789-g4213e17c cf22eb8683be 8e68f858bb22 2024-03-29T08:10:10.176 INFO:teuthology.orchestra.run.smithi037.stdout:mgr.smithi037.wjikii smithi037 *:8443,9283,8765 running (2m) 14s ago 14m 497M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff c5a1417b9417 2024-03-29T08:10:10.176 INFO:teuthology.orchestra.run.smithi037.stdout:mgr.smithi148.onhgvm smithi148 *:8443,9283,8765 running (69s) 38s ago 11m 446M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff fcfdbce062bd 2024-03-29T08:10:10.176 INFO:teuthology.orchestra.run.smithi037.stdout:mon.smithi037 smithi037 running (14m) 14s ago 14m 54.0M 2048M 17.2.7-789-g4213e17c cf22eb8683be bb2101505730 2024-03-29T08:10:10.176 INFO:teuthology.orchestra.run.smithi037.stdout:mon.smithi148 smithi148 running (11m) 38s ago 11m 42.0M 2048M 17.2.7-789-g4213e17c cf22eb8683be 18839ed2952a 2024-03-29T08:10:10.177 INFO:teuthology.orchestra.run.smithi037.stdout:node-exporter.smithi037 smithi037 *:9100 running (49s) 14s ago 13m 7042k - 1.5.0 0da6a335fe13 26b8ae9cb878 2024-03-29T08:10:10.177 INFO:teuthology.orchestra.run.smithi037.stdout:node-exporter.smithi148 smithi148 *:9100 running (43s) 38s ago 11m 2693k - 1.5.0 0da6a335fe13 98414e4e64ed 2024-03-29T08:10:10.177 INFO:teuthology.orchestra.run.smithi037.stdout:osd.0 smithi037 running (10m) 14s ago 10m 216M 1619M 17.2.7-789-g4213e17c cf22eb8683be 559fccd0ebe1 2024-03-29T08:10:10.177 INFO:teuthology.orchestra.run.smithi037.stdout:osd.1 smithi037 running (10m) 14s ago 10m 228M 1619M 17.2.7-789-g4213e17c cf22eb8683be 5582cff11b7a 2024-03-29T08:10:10.177 INFO:teuthology.orchestra.run.smithi037.stdout:osd.2 smithi037 running (9m) 14s ago 9m 196M 1619M 17.2.7-789-g4213e17c cf22eb8683be f809bfab13e5 2024-03-29T08:10:10.177 INFO:teuthology.orchestra.run.smithi037.stdout:osd.3 smithi148 running (9m) 38s ago 9m 259M 2642M 17.2.7-789-g4213e17c cf22eb8683be a0a8224bb4b0 2024-03-29T08:10:10.177 INFO:teuthology.orchestra.run.smithi037.stdout:osd.4 smithi148 running (8m) 38s ago 8m 230M 2642M 17.2.7-789-g4213e17c cf22eb8683be a00b69047ed7 2024-03-29T08:10:10.177 INFO:teuthology.orchestra.run.smithi037.stdout:osd.5 smithi148 running (8m) 38s ago 8m 231M 2642M 17.2.7-789-g4213e17c cf22eb8683be 044a3f7db7f8 2024-03-29T08:10:10.177 INFO:teuthology.orchestra.run.smithi037.stdout:prometheus.smithi037 smithi037 *:9095 running (21s) 14s ago 12m 34.3M - 2.43.0 a07b618ecd1d 30c0223ec589 2024-03-29T08:10:10.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:10.178+0000 7fab07e00700 1 -- 172.21.15.37:0/986579480 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7faaec06c8b0 msgr2=0x7faaec06ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:10:10.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:10.178+0000 7fab07e00700 1 --2- 172.21.15.37:0/986579480 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7faaec06c8b0 0x7faaec06ed60 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7faaf8005010 tx=0x7faaf8009e30 comp rx=0 tx=0).stop 2024-03-29T08:10:10.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:10.178+0000 7fab07e00700 1 -- 172.21.15.37:0/986579480 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fab08071ea0 msgr2=0x7fab0810fee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:10:10.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:10.178+0000 7fab07e00700 1 --2- 172.21.15.37:0/986579480 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fab08071ea0 0x7fab0810fee0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7faaf4002a50 tx=0x7faaf40046b0 comp rx=0 tx=0).stop 2024-03-29T08:10:10.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:10.178+0000 7fab07e00700 1 -- 172.21.15.37:0/986579480 shutdown_connections 2024-03-29T08:10:10.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:10.178+0000 7fab07e00700 1 --2- 172.21.15.37:0/986579480 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7faaec06c8b0 0x7faaec06ed60 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:10:10.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:10.178+0000 7fab07e00700 1 --2- 172.21.15.37:0/986579480 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fab080708f0 0x7fab0810f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:10:10.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:10.178+0000 7fab07e00700 1 --2- 172.21.15.37:0/986579480 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fab08071ea0 0x7fab0810fee0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:10:10.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:10.178+0000 7fab07e00700 1 -- 172.21.15.37:0/986579480 >> 172.21.15.37:0/986579480 conn(0x7fab080f83d0 msgr2=0x7fab080f9bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:10:10.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:10.178+0000 7fab07e00700 1 -- 172.21.15.37:0/986579480 shutdown_connections 2024-03-29T08:10:10.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:10.178+0000 7fab07e00700 1 -- 172.21.15.37:0/986579480 wait complete. 2024-03-29T08:10:10.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:10:09 smithi148 ceph-mon[44992]: pgmap v64: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:10:10.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:10.322+0000 7fb872800700 1 -- 172.21.15.37:0/2439432492 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb86c0fbcd0 msgr2=0x7fb86c0fc0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:10:10.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:10.322+0000 7fb872800700 1 --2- 172.21.15.37:0/2439432492 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb86c0fbcd0 0x7fb86c0fc0e0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fb85c004940 tx=0x7fb85c009e30 comp rx=0 tx=0).stop 2024-03-29T08:10:10.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:10.322+0000 7fb872800700 1 -- 172.21.15.37:0/2439432492 shutdown_connections 2024-03-29T08:10:10.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:10.322+0000 7fb872800700 1 --2- 172.21.15.37:0/2439432492 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb86c0fcf80 0x7fb86c0fd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:10:10.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:10.322+0000 7fb872800700 1 --2- 172.21.15.37:0/2439432492 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb86c0fbcd0 0x7fb86c0fc0e0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:10:10.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:10.322+0000 7fb872800700 1 -- 172.21.15.37:0/2439432492 >> 172.21.15.37:0/2439432492 conn(0x7fb86c0f7430 msgr2=0x7fb86c0f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:10:10.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:10.322+0000 7fb872800700 1 -- 172.21.15.37:0/2439432492 shutdown_connections 2024-03-29T08:10:10.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:10.322+0000 7fb872800700 1 -- 172.21.15.37:0/2439432492 wait complete. 2024-03-29T08:10:10.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:10.323+0000 7fb872800700 1 Processor -- start 2024-03-29T08:10:10.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:10.323+0000 7fb872800700 1 -- start start 2024-03-29T08:10:10.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:10.323+0000 7fb872800700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb86c0fbcd0 0x7fb86c10fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:10:10.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:10.323+0000 7fb872800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb86c0fcf80 0x7fb86c1100e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:10:10.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:10.323+0000 7fb872800700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb86c1106e0 con 0x7fb86c0fcf80 2024-03-29T08:10:10.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:10.323+0000 7fb872800700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb86c110ee0 con 0x7fb86c0fbcd0 2024-03-29T08:10:10.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:10.324+0000 7fb86aa00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb86c0fcf80 0x7fb86c1100e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:10:10.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:10.324+0000 7fb86aa00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb86c0fcf80 0x7fb86c1100e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52222/0 (socket says 172.21.15.37:52222) 2024-03-29T08:10:10.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:10.324+0000 7fb86aa00700 1 -- 172.21.15.37:0/1721112724 learned_addr learned my addr 172.21.15.37:0/1721112724 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:10:10.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:10.324+0000 7fb86b400700 1 --2- 172.21.15.37:0/1721112724 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb86c0fbcd0 0x7fb86c10fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:10:10.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:10.324+0000 7fb86aa00700 1 -- 172.21.15.37:0/1721112724 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb86c0fbcd0 msgr2=0x7fb86c10fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:10:10.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:10.324+0000 7fb86aa00700 1 --2- 172.21.15.37:0/1721112724 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb86c0fbcd0 0x7fb86c10fba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:10:10.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:10.324+0000 7fb86aa00700 1 -- 172.21.15.37:0/1721112724 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb85c005040 con 0x7fb86c0fcf80 2024-03-29T08:10:10.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:10.324+0000 7fb86aa00700 1 --2- 172.21.15.37:0/1721112724 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb86c0fcf80 0x7fb86c1100e0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fb858002a50 tx=0x7fb8580046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:10:10.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:10.325+0000 7fb863400700 1 -- 172.21.15.37:0/1721112724 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb858007cc0 con 0x7fb86c0fcf80 2024-03-29T08:10:10.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:10.325+0000 7fb863400700 1 -- 172.21.15.37:0/1721112724 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fb85800f040 con 0x7fb86c0fcf80 2024-03-29T08:10:10.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:10.325+0000 7fb872800700 1 -- 172.21.15.37:0/1721112724 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb86c1111c0 con 0x7fb86c0fcf80 2024-03-29T08:10:10.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:10.325+0000 7fb863400700 1 -- 172.21.15.37:0/1721112724 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb85800d9c0 con 0x7fb86c0fcf80 2024-03-29T08:10:10.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:10.325+0000 7fb872800700 1 -- 172.21.15.37:0/1721112724 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb86c111700 con 0x7fb86c0fcf80 2024-03-29T08:10:10.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:10.326+0000 7fb872800700 1 -- 172.21.15.37:0/1721112724 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb86c101910 con 0x7fb86c0fcf80 2024-03-29T08:10:10.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:10.327+0000 7fb863400700 1 -- 172.21.15.37:0/1721112724 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 34) v1 ==== 90703+0+0 (secure 0 0 0) 0x7fb85800db20 con 0x7fb86c0fcf80 2024-03-29T08:10:10.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:10.329+0000 7fb863400700 1 --2- 172.21.15.37:0/1721112724 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7fb85006c8a0 0x7fb85006ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:10:10.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:10.329+0000 7fb863400700 1 -- 172.21.15.37:0/1721112724 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6066+0+0 (secure 0 0 0) 0x7fb85808b7b0 con 0x7fb86c0fcf80 2024-03-29T08:10:10.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:10.329+0000 7fb86b400700 1 --2- 172.21.15.37:0/1721112724 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7fb85006c8a0 0x7fb85006ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:10:10.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:10.330+0000 7fb86b400700 1 --2- 172.21.15.37:0/1721112724 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7fb85006c8a0 0x7fb85006ed50 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fb85c00b3b0 tx=0x7fb85c009e30 comp rx=0 tx=0).ready entity=mgr.14686 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:10:10.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:10.333+0000 7fb863400700 1 -- 172.21.15.37:0/1721112724 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181579 (secure 0 0 0) 0x7fb858059720 con 0x7fb86c0fcf80 2024-03-29T08:10:10.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:10:09 smithi037 ceph-mon[35162]: pgmap v64: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:10:10.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:10.640+0000 7fb872800700 1 -- 172.21.15.37:0/1721112724 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fb86c05f5f0 con 0x7fb86c0fcf80 2024-03-29T08:10:10.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:10.641+0000 7fb863400700 1 -- 172.21.15.37:0/1721112724 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+743 (secure 0 0 0) 0x7fb8580592b0 con 0x7fb86c0fcf80 2024-03-29T08:10:10.641 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-29T08:10:10.641 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": { 2024-03-29T08:10:10.641 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 17.2.7-789-g4213e17c (4213e17c4de67016ed14f55d22503f4923b3a901) quincy (stable)": 2 2024-03-29T08:10:10.641 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:10:10.641 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": { 2024-03-29T08:10:10.641 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 2 2024-03-29T08:10:10.642 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:10:10.642 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": { 2024-03-29T08:10:10.642 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 17.2.7-789-g4213e17c (4213e17c4de67016ed14f55d22503f4923b3a901) quincy (stable)": 6 2024-03-29T08:10:10.642 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:10:10.642 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": { 2024-03-29T08:10:10.642 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 17.2.7-789-g4213e17c (4213e17c4de67016ed14f55d22503f4923b3a901) quincy (stable)": 4 2024-03-29T08:10:10.642 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:10:10.642 INFO:teuthology.orchestra.run.smithi037.stdout: "overall": { 2024-03-29T08:10:10.642 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 17.2.7-789-g4213e17c (4213e17c4de67016ed14f55d22503f4923b3a901) quincy (stable)": 12, 2024-03-29T08:10:10.642 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 2 2024-03-29T08:10:10.642 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-29T08:10:10.642 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-29T08:10:10.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:10.643+0000 7fb872800700 1 -- 172.21.15.37:0/1721112724 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7fb85006c8a0 msgr2=0x7fb85006ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:10:10.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:10.643+0000 7fb872800700 1 --2- 172.21.15.37:0/1721112724 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7fb85006c8a0 0x7fb85006ed50 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fb85c00b3b0 tx=0x7fb85c009e30 comp rx=0 tx=0).stop 2024-03-29T08:10:10.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:10.643+0000 7fb872800700 1 -- 172.21.15.37:0/1721112724 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb86c0fcf80 msgr2=0x7fb86c1100e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:10:10.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:10.643+0000 7fb872800700 1 --2- 172.21.15.37:0/1721112724 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb86c0fcf80 0x7fb86c1100e0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fb858002a50 tx=0x7fb8580046b0 comp rx=0 tx=0).stop 2024-03-29T08:10:10.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:10.643+0000 7fb872800700 1 -- 172.21.15.37:0/1721112724 shutdown_connections 2024-03-29T08:10:10.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:10.643+0000 7fb872800700 1 --2- 172.21.15.37:0/1721112724 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7fb85006c8a0 0x7fb85006ed50 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:10:10.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:10.643+0000 7fb872800700 1 --2- 172.21.15.37:0/1721112724 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb86c0fbcd0 0x7fb86c10fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:10:10.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:10.643+0000 7fb872800700 1 --2- 172.21.15.37:0/1721112724 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb86c0fcf80 0x7fb86c1100e0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:10:10.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:10.643+0000 7fb872800700 1 -- 172.21.15.37:0/1721112724 >> 172.21.15.37:0/1721112724 conn(0x7fb86c0f7430 msgr2=0x7fb86c065c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:10:10.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:10.644+0000 7fb872800700 1 -- 172.21.15.37:0/1721112724 shutdown_connections 2024-03-29T08:10:10.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:10.644+0000 7fb872800700 1 -- 172.21.15.37:0/1721112724 wait complete. 2024-03-29T08:10:10.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:10.804+0000 7f60a5e00700 1 -- 172.21.15.37:0/1450045061 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f60a01002a0 msgr2=0x7f60a0100710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:10:10.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:10.804+0000 7f60a5e00700 1 --2- 172.21.15.37:0/1450045061 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f60a01002a0 0x7f60a0100710 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f608c004940 tx=0x7f608c009e30 comp rx=0 tx=0).stop 2024-03-29T08:10:10.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:10.804+0000 7f60a5e00700 1 -- 172.21.15.37:0/1450045061 shutdown_connections 2024-03-29T08:10:10.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:10.804+0000 7f60a5e00700 1 --2- 172.21.15.37:0/1450045061 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f60a01002a0 0x7f60a0100710 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:10:10.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:10.804+0000 7f60a5e00700 1 --2- 172.21.15.37:0/1450045061 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f60a00feff0 0x7f60a00ff400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:10:10.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:10.804+0000 7f60a5e00700 1 -- 172.21.15.37:0/1450045061 >> 172.21.15.37:0/1450045061 conn(0x7f60a00fa950 msgr2=0x7f60a00fcd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:10:10.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:10.804+0000 7f60a5e00700 1 -- 172.21.15.37:0/1450045061 shutdown_connections 2024-03-29T08:10:10.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:10.804+0000 7f60a5e00700 1 -- 172.21.15.37:0/1450045061 wait complete. 2024-03-29T08:10:10.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:10.805+0000 7f60a5e00700 1 Processor -- start 2024-03-29T08:10:10.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:10.805+0000 7f60a5e00700 1 -- start start 2024-03-29T08:10:10.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:10.806+0000 7f60a5e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f60a00feff0 0x7f60a006e450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:10:10.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:10.806+0000 7f60a5e00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f60a01002a0 0x7f60a006e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:10:10.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:10.806+0000 7f60a5e00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60a0070090 con 0x7f60a00feff0 2024-03-29T08:10:10.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:10.806+0000 7f60a5e00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60a006eed0 con 0x7f60a01002a0 2024-03-29T08:10:10.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:10.806+0000 7f609ea00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f60a00feff0 0x7f60a006e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:10:10.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:10.806+0000 7f609ea00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f60a00feff0 0x7f60a006e450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52248/0 (socket says 172.21.15.37:52248) 2024-03-29T08:10:10.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:10.806+0000 7f609ea00700 1 -- 172.21.15.37:0/2309676597 learned_addr learned my addr 172.21.15.37:0/2309676597 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:10:10.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:10.806+0000 7f609e000700 1 --2- 172.21.15.37:0/2309676597 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f60a01002a0 0x7f60a006e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:10:10.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:10.807+0000 7f609ea00700 1 -- 172.21.15.37:0/2309676597 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f60a01002a0 msgr2=0x7f60a006e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:10:10.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:10.807+0000 7f609ea00700 1 --2- 172.21.15.37:0/2309676597 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f60a01002a0 0x7f60a006e990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:10:10.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:10.807+0000 7f609ea00700 1 -- 172.21.15.37:0/2309676597 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f608c005040 con 0x7f60a00feff0 2024-03-29T08:10:10.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:10.807+0000 7f609ea00700 1 --2- 172.21.15.37:0/2309676597 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f60a00feff0 0x7f60a006e450 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f609000eb40 tx=0x7f609000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:10:10.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:10.807+0000 7f6096a00700 1 -- 172.21.15.37:0/2309676597 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6090007cc0 con 0x7f60a00feff0 2024-03-29T08:10:10.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:10.807+0000 7f6096a00700 1 -- 172.21.15.37:0/2309676597 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f609000f040 con 0x7f60a00feff0 2024-03-29T08:10:10.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:10.807+0000 7f6096a00700 1 -- 172.21.15.37:0/2309676597 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6090009570 con 0x7f60a00feff0 2024-03-29T08:10:10.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:10.807+0000 7f60a5e00700 1 -- 172.21.15.37:0/2309676597 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60a006f150 con 0x7f60a00feff0 2024-03-29T08:10:10.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:10.807+0000 7f60a5e00700 1 -- 172.21.15.37:0/2309676597 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60a011e010 con 0x7f60a00feff0 2024-03-29T08:10:10.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:10.809+0000 7f60a5e00700 1 -- 172.21.15.37:0/2309676597 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60a005f5f0 con 0x7f60a00feff0 2024-03-29T08:10:10.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:10.810+0000 7f6096a00700 1 -- 172.21.15.37:0/2309676597 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 34) v1 ==== 90703+0+0 (secure 0 0 0) 0x7f60900096d0 con 0x7f60a00feff0 2024-03-29T08:10:10.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:10.810+0000 7f6096a00700 1 --2- 172.21.15.37:0/2309676597 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7f608406c8b0 0x7f608406ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:10:10.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:10.811+0000 7f6096a00700 1 -- 172.21.15.37:0/2309676597 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f609008b670 con 0x7f60a00feff0 2024-03-29T08:10:10.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:10.811+0000 7f609e000700 1 --2- 172.21.15.37:0/2309676597 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7f608406c8b0 0x7f608406ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:10:10.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:10.812+0000 7f609e000700 1 --2- 172.21.15.37:0/2309676597 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7f608406c8b0 0x7f608406ed60 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f608c00b0d0 tx=0x7f608c009e30 comp rx=0 tx=0).ready entity=mgr.14686 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:10:10.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:10.815+0000 7f6096a00700 1 -- 172.21.15.37:0/2309676597 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181579 (secure 0 0 0) 0x7f6090055c80 con 0x7f60a00feff0 2024-03-29T08:10:11.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:11.060+0000 7f60a5e00700 1 -- 172.21.15.37:0/2309676597 --> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f60a0104610 con 0x7f608406c8b0 2024-03-29T08:10:11.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:11.061+0000 7f6096a00700 1 -- 172.21.15.37:0/2309676597 <== mgr.14686 v2:172.21.15.37:6800/4188329516 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+369 (secure 0 0 0) 0x7f60a0104610 con 0x7f608406c8b0 2024-03-29T08:10:11.061 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-29T08:10:11.061 INFO:teuthology.orchestra.run.smithi037.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59", 2024-03-29T08:10:11.061 INFO:teuthology.orchestra.run.smithi037.stdout: "in_progress": true, 2024-03-29T08:10:11.062 INFO:teuthology.orchestra.run.smithi037.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-03-29T08:10:11.062 INFO:teuthology.orchestra.run.smithi037.stdout: "services_complete": [ 2024-03-29T08:10:11.062 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr" 2024-03-29T08:10:11.062 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-29T08:10:11.062 INFO:teuthology.orchestra.run.smithi037.stdout: "progress": "2/2 daemons upgraded", 2024-03-29T08:10:11.062 INFO:teuthology.orchestra.run.smithi037.stdout: "message": "Currently upgrading alertmanager daemons", 2024-03-29T08:10:11.062 INFO:teuthology.orchestra.run.smithi037.stdout: "is_paused": false 2024-03-29T08:10:11.062 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-29T08:10:11.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:11.063+0000 7f60a5e00700 1 -- 172.21.15.37:0/2309676597 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7f608406c8b0 msgr2=0x7f608406ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:10:11.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:11.063+0000 7f60a5e00700 1 --2- 172.21.15.37:0/2309676597 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7f608406c8b0 0x7f608406ed60 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f608c00b0d0 tx=0x7f608c009e30 comp rx=0 tx=0).stop 2024-03-29T08:10:11.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:11.063+0000 7f60a5e00700 1 -- 172.21.15.37:0/2309676597 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f60a00feff0 msgr2=0x7f60a006e450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:10:11.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:11.063+0000 7f60a5e00700 1 --2- 172.21.15.37:0/2309676597 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f60a00feff0 0x7f60a006e450 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f609000eb40 tx=0x7f609000b040 comp rx=0 tx=0).stop 2024-03-29T08:10:11.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:11.064+0000 7f60a5e00700 1 -- 172.21.15.37:0/2309676597 shutdown_connections 2024-03-29T08:10:11.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:11.064+0000 7f60a5e00700 1 --2- 172.21.15.37:0/2309676597 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7f608406c8b0 0x7f608406ed60 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:10:11.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:11.064+0000 7f60a5e00700 1 --2- 172.21.15.37:0/2309676597 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f60a00feff0 0x7f60a006e450 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:10:11.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:11.064+0000 7f60a5e00700 1 --2- 172.21.15.37:0/2309676597 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f60a01002a0 0x7f60a006e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:10:11.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:11.064+0000 7f60a5e00700 1 -- 172.21.15.37:0/2309676597 >> 172.21.15.37:0/2309676597 conn(0x7f60a00fa950 msgr2=0x7f60a00fd0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:10:11.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:11.064+0000 7f60a5e00700 1 -- 172.21.15.37:0/2309676597 shutdown_connections 2024-03-29T08:10:11.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:11.064+0000 7f60a5e00700 1 -- 172.21.15.37:0/2309676597 wait complete. 2024-03-29T08:10:11.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:10:11 smithi148 ceph-mon[44992]: from='client.14782 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:10:11.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:10:11 smithi148 ceph-mon[44992]: from='client.14786 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:10:11.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:10:11 smithi148 ceph-mon[44992]: from='client.14790 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:10:11.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:10:11 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' 2024-03-29T08:10:11.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:10:11 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' 2024-03-29T08:10:11.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:10:11 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T08:10:11.557 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:10:11 smithi148 ceph-mon[44992]: from='client.? 172.21.15.37:0/1721112724' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:10:11.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:10:11 smithi037 ceph-mon[35162]: from='client.14782 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:10:11.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:10:11 smithi037 ceph-mon[35162]: from='client.14786 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:10:11.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:10:11 smithi037 ceph-mon[35162]: from='client.14790 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:10:11.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:10:11 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' 2024-03-29T08:10:11.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:10:11 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' 2024-03-29T08:10:11.678 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:10:11 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T08:10:11.678 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:10:11 smithi037 ceph-mon[35162]: from='client.? 172.21.15.37:0/1721112724' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:10:12.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:10:12 smithi037 ceph-mon[35162]: pgmap v65: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:10:12.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:10:12 smithi037 ceph-mon[35162]: from='client.14798 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:10:12.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:10:12 smithi148 ceph-mon[44992]: pgmap v65: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:10:12.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:10:12 smithi148 ceph-mon[44992]: from='client.14798 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:10:13.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:10:13 smithi148 ceph-mon[44992]: pgmap v66: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:10:13.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:10:13 smithi037 ceph-mon[35162]: pgmap v66: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:10:15.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:10:15 smithi037 ceph-mon[35162]: pgmap v67: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:10:16.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:10:15 smithi148 ceph-mon[44992]: pgmap v67: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:10:17.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:10:17 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' 2024-03-29T08:10:17.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:10:17 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' 2024-03-29T08:10:17.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:10:17 smithi148 ceph-mon[44992]: pgmap v68: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:10:17.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:10:17 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' 2024-03-29T08:10:17.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:10:17 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' 2024-03-29T08:10:17.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:10:17 smithi037 ceph-mon[35162]: pgmap v68: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:10:18.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:10:18 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' 2024-03-29T08:10:18.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:10:18 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' 2024-03-29T08:10:18.812 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:10:18 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' 2024-03-29T08:10:18.812 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:10:18 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' 2024-03-29T08:10:19.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:10:19 smithi148 ceph-mon[44992]: pgmap v69: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:10:19.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:10:19 smithi037 ceph-mon[35162]: pgmap v69: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:10:21.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:10:21 smithi037 ceph-mon[35162]: pgmap v70: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:10:22.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:10:21 smithi148 ceph-mon[44992]: pgmap v70: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:10:22.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:10:22 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:10:22.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:10:22 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' 2024-03-29T08:10:22.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:10:22 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' 2024-03-29T08:10:22.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:10:22 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T08:10:22.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:10:22 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T08:10:22.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:10:22 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' 2024-03-29T08:10:22.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:10:22 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-29T08:10:22.928 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:10:22 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T08:10:22.928 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:10:22 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:10:22.928 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:10:22 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:10:22.928 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:10:22 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:10:22.928 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:10:22 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:10:22.928 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:10:22 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:10:22.928 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:10:22 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:10:22.928 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:10:22 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:10:22.928 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:10:22 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:10:22.928 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:10:22 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:10:22.928 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:10:22 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:10:22.928 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:10:22 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:10:23.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:10:22 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:10:23.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:10:22 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' 2024-03-29T08:10:23.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:10:22 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' 2024-03-29T08:10:23.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:10:22 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T08:10:23.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:10:22 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T08:10:23.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:10:22 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' 2024-03-29T08:10:23.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:10:22 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-29T08:10:23.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:10:22 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T08:10:23.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:10:22 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:10:23.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:10:22 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:10:23.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:10:22 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:10:23.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:10:22 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:10:23.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:10:22 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:10:23.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:10:22 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:10:23.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:10:22 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:10:23.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:10:22 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:10:23.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:10:22 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:10:23.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:10:22 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:10:23.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:10:22 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:10:24.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:10:24 smithi148 ceph-mon[44992]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-29T08:10:24.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:10:24 smithi148 ceph-mon[44992]: pgmap v71: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:10:24.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:10:24 smithi037 ceph-mon[35162]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-29T08:10:24.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:10:24 smithi037 ceph-mon[35162]: pgmap v71: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:10:25.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:10:25 smithi148 ceph-mon[44992]: Upgrade: Updating grafana.smithi037 2024-03-29T08:10:25.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:10:25 smithi148 ceph-mon[44992]: Deploying daemon grafana.smithi037 on smithi037 2024-03-29T08:10:25.407 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:10:25 smithi037 ceph-mon[35162]: Upgrade: Updating grafana.smithi037 2024-03-29T08:10:25.408 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:10:25 smithi037 ceph-mon[35162]: Deploying daemon grafana.smithi037 on smithi037 2024-03-29T08:10:26.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:10:26 smithi148 ceph-mon[44992]: pgmap v72: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:10:26.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:10:26 smithi037 ceph-mon[35162]: pgmap v72: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:10:28.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:10:28 smithi037 ceph-mon[35162]: pgmap v73: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:10:28.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:10:28 smithi148 ceph-mon[44992]: pgmap v73: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:10:30.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:10:30 smithi148 ceph-mon[44992]: pgmap v74: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:10:30.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:10:30 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' 2024-03-29T08:10:30.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:10:30 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' 2024-03-29T08:10:30.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:10:30 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T08:10:30.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:10:30 smithi037 ceph-mon[35162]: pgmap v74: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:10:30.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:10:30 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' 2024-03-29T08:10:30.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:10:30 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' 2024-03-29T08:10:30.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:10:30 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T08:10:32.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:10:32 smithi148 ceph-mon[44992]: pgmap v75: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:10:32.367 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:10:32 smithi037 ceph-mon[35162]: pgmap v75: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:10:34.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:10:34 smithi148 ceph-mon[44992]: pgmap v76: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:10:34.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:10:34 smithi037 ceph-mon[35162]: pgmap v76: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:10:36.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:10:36 smithi037 ceph-mon[35162]: pgmap v77: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:10:36.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:10:36 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' 2024-03-29T08:10:36.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:10:36 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' 2024-03-29T08:10:36.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:10:36 smithi148 ceph-mon[44992]: pgmap v77: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:10:36.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:10:36 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' 2024-03-29T08:10:36.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:10:36 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' 2024-03-29T08:10:37.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:10:37 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:10:37.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:10:37 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' 2024-03-29T08:10:37.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:10:37 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' 2024-03-29T08:10:37.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:10:37 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:10:37.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:10:37 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' 2024-03-29T08:10:37.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:10:37 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' 2024-03-29T08:10:38.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:10:38 smithi037 ceph-mon[35162]: pgmap v78: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:10:38.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:10:38 smithi148 ceph-mon[44992]: pgmap v78: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:10:40.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:10:40 smithi037 ceph-mon[35162]: pgmap v79: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:10:40.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:10:40 smithi148 ceph-mon[44992]: pgmap v79: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:10:41.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:41.213+0000 7fa0a0800700 1 -- 172.21.15.37:0/3595755727 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa098071f50 msgr2=0x7fa098070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:10:41.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:41.213+0000 7fa0a0800700 1 --2- 172.21.15.37:0/3595755727 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa098071f50 0x7fa098070460 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fa08c004940 tx=0x7fa08c009e30 comp rx=0 tx=0).stop 2024-03-29T08:10:41.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:41.213+0000 7fa0a0800700 1 -- 172.21.15.37:0/3595755727 shutdown_connections 2024-03-29T08:10:41.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:41.213+0000 7fa0a0800700 1 --2- 172.21.15.37:0/3595755727 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa0980709a0 0x7fa098070e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:10:41.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:41.213+0000 7fa0a0800700 1 --2- 172.21.15.37:0/3595755727 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa098071f50 0x7fa098070460 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:10:41.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:41.213+0000 7fa0a0800700 1 -- 172.21.15.37:0/3595755727 >> 172.21.15.37:0/3595755727 conn(0x7fa0980f8560 msgr2=0x7fa0980fa970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:10:41.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:41.213+0000 7fa0a0800700 1 -- 172.21.15.37:0/3595755727 shutdown_connections 2024-03-29T08:10:41.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:41.213+0000 7fa0a0800700 1 -- 172.21.15.37:0/3595755727 wait complete. 2024-03-29T08:10:41.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:41.214+0000 7fa0a0800700 1 Processor -- start 2024-03-29T08:10:41.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:41.214+0000 7fa0a0800700 1 -- start start 2024-03-29T08:10:41.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:41.215+0000 7fa0a0800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa0980709a0 0x7fa09810fad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:10:41.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:41.215+0000 7fa0a0800700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa098071f50 0x7fa098110010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:10:41.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:41.215+0000 7fa0a0800700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa098110610 con 0x7fa0980709a0 2024-03-29T08:10:41.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:41.215+0000 7fa0a0800700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa098110e60 con 0x7fa098071f50 2024-03-29T08:10:41.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:41.215+0000 7fa09de00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa0980709a0 0x7fa09810fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:10:41.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:41.215+0000 7fa09de00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa0980709a0 0x7fa09810fad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51500/0 (socket says 172.21.15.37:51500) 2024-03-29T08:10:41.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:41.215+0000 7fa09de00700 1 -- 172.21.15.37:0/3413700918 learned_addr learned my addr 172.21.15.37:0/3413700918 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:10:41.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:41.215+0000 7fa09d400700 1 --2- 172.21.15.37:0/3413700918 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa098071f50 0x7fa098110010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:10:41.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:41.215+0000 7fa09de00700 1 -- 172.21.15.37:0/3413700918 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa098071f50 msgr2=0x7fa098110010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:10:41.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:41.215+0000 7fa09de00700 1 --2- 172.21.15.37:0/3413700918 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa098071f50 0x7fa098110010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:10:41.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:41.215+0000 7fa09de00700 1 -- 172.21.15.37:0/3413700918 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa08c005040 con 0x7fa0980709a0 2024-03-29T08:10:41.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:41.216+0000 7fa09de00700 1 --2- 172.21.15.37:0/3413700918 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa0980709a0 0x7fa09810fad0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fa08c000c00 tx=0x7fa08c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:10:41.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:41.216+0000 7fa092000700 1 -- 172.21.15.37:0/3413700918 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa08c00e070 con 0x7fa0980709a0 2024-03-29T08:10:41.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:41.216+0000 7fa092000700 1 -- 172.21.15.37:0/3413700918 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fa08c01c400 con 0x7fa0980709a0 2024-03-29T08:10:41.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:41.216+0000 7fa0a0800700 1 -- 172.21.15.37:0/3413700918 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0981110e0 con 0x7fa0980709a0 2024-03-29T08:10:41.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:41.216+0000 7fa092000700 1 -- 172.21.15.37:0/3413700918 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa08c00b650 con 0x7fa0980709a0 2024-03-29T08:10:41.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:41.216+0000 7fa0a0800700 1 -- 172.21.15.37:0/3413700918 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0981115c0 con 0x7fa0980709a0 2024-03-29T08:10:41.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:41.217+0000 7fa0a0800700 1 -- 172.21.15.37:0/3413700918 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa09805f5f0 con 0x7fa0980709a0 2024-03-29T08:10:41.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:41.218+0000 7fa092000700 1 -- 172.21.15.37:0/3413700918 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 34) v1 ==== 90703+0+0 (secure 0 0 0) 0x7fa08c01c570 con 0x7fa0980709a0 2024-03-29T08:10:41.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:41.219+0000 7fa092000700 1 --2- 172.21.15.37:0/3413700918 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7fa08006c8b0 0x7fa08006ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:10:41.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:41.220+0000 7fa092000700 1 -- 172.21.15.37:0/3413700918 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6066+0+0 (secure 0 0 0) 0x7fa08c08d0f0 con 0x7fa0980709a0 2024-03-29T08:10:41.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:41.220+0000 7fa09d400700 1 --2- 172.21.15.37:0/3413700918 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7fa08006c8b0 0x7fa08006ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:10:41.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:41.221+0000 7fa09d400700 1 --2- 172.21.15.37:0/3413700918 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7fa08006c8b0 0x7fa08006ed60 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fa088004600 tx=0x7fa08800b040 comp rx=0 tx=0).ready entity=mgr.14686 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:10:41.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:41.224+0000 7fa092000700 1 -- 172.21.15.37:0/3413700918 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181579 (secure 0 0 0) 0x7fa08c05b0c0 con 0x7fa0980709a0 2024-03-29T08:10:41.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:41.469+0000 7fa0a0800700 1 -- 172.21.15.37:0/3413700918 --> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa0981119b0 con 0x7fa08006c8b0 2024-03-29T08:10:41.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:41.471+0000 7fa092000700 1 -- 172.21.15.37:0/3413700918 <== mgr.14686 v2:172.21.15.37:6800/4188329516 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+364 (secure 0 0 0) 0x7fa0981119b0 con 0x7fa08006c8b0 2024-03-29T08:10:41.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:41.473+0000 7fa0a0800700 1 -- 172.21.15.37:0/3413700918 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7fa08006c8b0 msgr2=0x7fa08006ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:10:41.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:41.473+0000 7fa0a0800700 1 --2- 172.21.15.37:0/3413700918 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7fa08006c8b0 0x7fa08006ed60 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fa088004600 tx=0x7fa08800b040 comp rx=0 tx=0).stop 2024-03-29T08:10:41.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:41.473+0000 7fa0a0800700 1 -- 172.21.15.37:0/3413700918 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa0980709a0 msgr2=0x7fa09810fad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:10:41.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:41.473+0000 7fa0a0800700 1 --2- 172.21.15.37:0/3413700918 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa0980709a0 0x7fa09810fad0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fa08c000c00 tx=0x7fa08c009e30 comp rx=0 tx=0).stop 2024-03-29T08:10:41.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:41.473+0000 7fa0a0800700 1 -- 172.21.15.37:0/3413700918 shutdown_connections 2024-03-29T08:10:41.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:41.473+0000 7fa0a0800700 1 --2- 172.21.15.37:0/3413700918 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7fa08006c8b0 0x7fa08006ed60 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:10:41.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:41.473+0000 7fa0a0800700 1 --2- 172.21.15.37:0/3413700918 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa0980709a0 0x7fa09810fad0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:10:41.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:41.473+0000 7fa0a0800700 1 --2- 172.21.15.37:0/3413700918 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa098071f50 0x7fa098110010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:10:41.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:41.473+0000 7fa0a0800700 1 -- 172.21.15.37:0/3413700918 >> 172.21.15.37:0/3413700918 conn(0x7fa0980f8560 msgr2=0x7fa0980f9cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:10:41.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:41.473+0000 7fa0a0800700 1 -- 172.21.15.37:0/3413700918 shutdown_connections 2024-03-29T08:10:41.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:41.473+0000 7fa0a0800700 1 -- 172.21.15.37:0/3413700918 wait complete. 2024-03-29T08:10:41.488 INFO:teuthology.orchestra.run.smithi037.stdout:true 2024-03-29T08:10:41.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:41.629+0000 7f4a3ba00700 1 -- 172.21.15.37:0/2383744034 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4a340662e0 msgr2=0x7f4a340ff9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:10:41.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:41.629+0000 7f4a3ba00700 1 --2- 172.21.15.37:0/2383744034 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4a340662e0 0x7f4a340ff9c0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f4a28004940 tx=0x7f4a28009e30 comp rx=0 tx=0).stop 2024-03-29T08:10:41.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:41.630+0000 7f4a3ba00700 1 -- 172.21.15.37:0/2383744034 shutdown_connections 2024-03-29T08:10:41.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:41.630+0000 7f4a3ba00700 1 --2- 172.21.15.37:0/2383744034 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4a340fff00 0x7f4a34102320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:10:41.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:41.630+0000 7f4a3ba00700 1 --2- 172.21.15.37:0/2383744034 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4a340662e0 0x7f4a340ff9c0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:10:41.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:41.630+0000 7f4a3ba00700 1 -- 172.21.15.37:0/2383744034 >> 172.21.15.37:0/2383744034 conn(0x7f4a340f7430 msgr2=0x7f4a340f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:10:41.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:41.630+0000 7f4a3ba00700 1 -- 172.21.15.37:0/2383744034 shutdown_connections 2024-03-29T08:10:41.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:41.630+0000 7f4a3ba00700 1 -- 172.21.15.37:0/2383744034 wait complete. 2024-03-29T08:10:41.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:41.631+0000 7f4a3ba00700 1 Processor -- start 2024-03-29T08:10:41.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:41.631+0000 7f4a3ba00700 1 -- start start 2024-03-29T08:10:41.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:41.631+0000 7f4a3ba00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4a340662e0 0x7f4a3410fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:10:41.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:41.631+0000 7f4a3ba00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4a340fff00 0x7f4a341100b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:10:41.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:41.631+0000 7f4a3ba00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a341106b0 con 0x7f4a340662e0 2024-03-29T08:10:41.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:41.631+0000 7f4a3ba00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a34110f00 con 0x7f4a340fff00 2024-03-29T08:10:41.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:41.632+0000 7f4a39000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4a340662e0 0x7f4a3410fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:10:41.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:41.632+0000 7f4a39000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4a340662e0 0x7f4a3410fb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51512/0 (socket says 172.21.15.37:51512) 2024-03-29T08:10:41.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:41.632+0000 7f4a39000700 1 -- 172.21.15.37:0/1233175896 learned_addr learned my addr 172.21.15.37:0/1233175896 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:10:41.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:41.632+0000 7f4a33e00700 1 --2- 172.21.15.37:0/1233175896 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4a340fff00 0x7f4a341100b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:10:41.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:41.632+0000 7f4a39000700 1 -- 172.21.15.37:0/1233175896 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4a340fff00 msgr2=0x7f4a341100b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:10:41.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:41.632+0000 7f4a39000700 1 --2- 172.21.15.37:0/1233175896 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4a340fff00 0x7f4a341100b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:10:41.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:41.632+0000 7f4a39000700 1 -- 172.21.15.37:0/1233175896 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a28005040 con 0x7f4a340662e0 2024-03-29T08:10:41.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:41.632+0000 7f4a39000700 1 --2- 172.21.15.37:0/1233175896 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4a340662e0 0x7f4a3410fb70 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f4a2800b900 tx=0x7f4a28009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:10:41.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:41.635+0000 7f4a30c00700 1 -- 172.21.15.37:0/1233175896 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4a2800e040 con 0x7f4a340662e0 2024-03-29T08:10:41.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:41.635+0000 7f4a3ba00700 1 -- 172.21.15.37:0/1233175896 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a34111180 con 0x7f4a340662e0 2024-03-29T08:10:41.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:41.635+0000 7f4a30c00700 1 -- 172.21.15.37:0/1233175896 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f4a28004290 con 0x7f4a340662e0 2024-03-29T08:10:41.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:41.635+0000 7f4a30c00700 1 -- 172.21.15.37:0/1233175896 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4a2801c3f0 con 0x7f4a340662e0 2024-03-29T08:10:41.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:41.636+0000 7f4a3ba00700 1 -- 172.21.15.37:0/1233175896 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a341116c0 con 0x7f4a340662e0 2024-03-29T08:10:41.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:41.637+0000 7f4a3ba00700 1 -- 172.21.15.37:0/1233175896 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a3405f5f0 con 0x7f4a340662e0 2024-03-29T08:10:41.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:41.638+0000 7f4a30c00700 1 -- 172.21.15.37:0/1233175896 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 34) v1 ==== 90703+0+0 (secure 0 0 0) 0x7f4a28013420 con 0x7f4a340662e0 2024-03-29T08:10:41.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:41.639+0000 7f4a30c00700 1 --2- 172.21.15.37:0/1233175896 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7f4a1c06c8b0 0x7f4a1c06ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:10:41.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:41.639+0000 7f4a30c00700 1 -- 172.21.15.37:0/1233175896 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f4a2808cd70 con 0x7f4a340662e0 2024-03-29T08:10:41.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:41.639+0000 7f4a33e00700 1 --2- 172.21.15.37:0/1233175896 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7f4a1c06c8b0 0x7f4a1c06ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:10:41.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:41.640+0000 7f4a33e00700 1 --2- 172.21.15.37:0/1233175896 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7f4a1c06c8b0 0x7f4a1c06ed60 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f4a20004f40 tx=0x7f4a2000b040 comp rx=0 tx=0).ready entity=mgr.14686 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:10:41.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:41.644+0000 7f4a30c00700 1 -- 172.21.15.37:0/1233175896 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181579 (secure 0 0 0) 0x7f4a2805ad60 con 0x7f4a340662e0 2024-03-29T08:10:41.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:41.874+0000 7f4a3ba00700 1 -- 172.21.15.37:0/1233175896 --> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4a34111ab0 con 0x7f4a1c06c8b0 2024-03-29T08:10:41.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:41.875+0000 7f4a30c00700 1 -- 172.21.15.37:0/1233175896 <== mgr.14686 v2:172.21.15.37:6800/4188329516 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+175 (secure 0 0 0) 0x7f4a34111ab0 con 0x7f4a1c06c8b0 2024-03-29T08:10:41.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:41.877+0000 7f4a3ba00700 1 -- 172.21.15.37:0/1233175896 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7f4a1c06c8b0 msgr2=0x7f4a1c06ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:10:41.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:41.877+0000 7f4a3ba00700 1 --2- 172.21.15.37:0/1233175896 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7f4a1c06c8b0 0x7f4a1c06ed60 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f4a20004f40 tx=0x7f4a2000b040 comp rx=0 tx=0).stop 2024-03-29T08:10:41.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:41.877+0000 7f4a3ba00700 1 -- 172.21.15.37:0/1233175896 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4a340662e0 msgr2=0x7f4a3410fb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:10:41.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:41.877+0000 7f4a3ba00700 1 --2- 172.21.15.37:0/1233175896 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4a340662e0 0x7f4a3410fb70 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f4a2800b900 tx=0x7f4a28009e30 comp rx=0 tx=0).stop 2024-03-29T08:10:41.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:41.877+0000 7f4a3ba00700 1 -- 172.21.15.37:0/1233175896 shutdown_connections 2024-03-29T08:10:41.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:41.877+0000 7f4a3ba00700 1 --2- 172.21.15.37:0/1233175896 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7f4a1c06c8b0 0x7f4a1c06ed60 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:10:41.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:41.877+0000 7f4a3ba00700 1 --2- 172.21.15.37:0/1233175896 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4a340662e0 0x7f4a3410fb70 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:10:41.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:41.877+0000 7f4a3ba00700 1 --2- 172.21.15.37:0/1233175896 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4a340fff00 0x7f4a341100b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:10:41.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:41.877+0000 7f4a3ba00700 1 -- 172.21.15.37:0/1233175896 >> 172.21.15.37:0/1233175896 conn(0x7f4a340f7430 msgr2=0x7f4a340fbaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:10:41.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:41.878+0000 7f4a3ba00700 1 -- 172.21.15.37:0/1233175896 shutdown_connections 2024-03-29T08:10:41.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:41.878+0000 7f4a3ba00700 1 -- 172.21.15.37:0/1233175896 wait complete. 2024-03-29T08:10:42.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:42.013+0000 7f5417000700 1 -- 172.21.15.37:0/2912268935 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f54100fcc10 msgr2=0x7f54100fd020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:10:42.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:42.013+0000 7f5417000700 1 --2- 172.21.15.37:0/2912268935 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f54100fcc10 0x7f54100fd020 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f5404004940 tx=0x7f5404009e30 comp rx=0 tx=0).stop 2024-03-29T08:10:42.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:42.013+0000 7f5417000700 1 -- 172.21.15.37:0/2912268935 shutdown_connections 2024-03-29T08:10:42.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:42.013+0000 7f5417000700 1 --2- 172.21.15.37:0/2912268935 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54100fdec0 0x7f54100fe330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:10:42.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:42.013+0000 7f5417000700 1 --2- 172.21.15.37:0/2912268935 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f54100fcc10 0x7f54100fd020 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:10:42.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:42.013+0000 7f5417000700 1 -- 172.21.15.37:0/2912268935 >> 172.21.15.37:0/2912268935 conn(0x7f54100f83d0 msgr2=0x7f54100fa7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:10:42.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:42.013+0000 7f5417000700 1 -- 172.21.15.37:0/2912268935 shutdown_connections 2024-03-29T08:10:42.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:42.013+0000 7f5417000700 1 -- 172.21.15.37:0/2912268935 wait complete. 2024-03-29T08:10:42.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:42.014+0000 7f5417000700 1 Processor -- start 2024-03-29T08:10:42.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:42.014+0000 7f5417000700 1 -- start start 2024-03-29T08:10:42.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:42.015+0000 7f5417000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f54100fcc10 0x7f541010f900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:10:42.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:42.015+0000 7f5417000700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54100fdec0 0x7f541010fe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:10:42.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:42.015+0000 7f5417000700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5410110440 con 0x7f54100fcc10 2024-03-29T08:10:42.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:42.015+0000 7f5417000700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5410110c90 con 0x7f54100fdec0 2024-03-29T08:10:42.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:42.015+0000 7f5415c00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f54100fcc10 0x7f541010f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:10:42.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:42.015+0000 7f5415c00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f54100fcc10 0x7f541010f900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51540/0 (socket says 172.21.15.37:51540) 2024-03-29T08:10:42.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:42.015+0000 7f5415c00700 1 -- 172.21.15.37:0/3564874447 learned_addr learned my addr 172.21.15.37:0/3564874447 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:10:42.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:42.015+0000 7f5415200700 1 --2- 172.21.15.37:0/3564874447 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54100fdec0 0x7f541010fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:10:42.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:42.015+0000 7f5415c00700 1 -- 172.21.15.37:0/3564874447 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54100fdec0 msgr2=0x7f541010fe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:10:42.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:42.015+0000 7f5415c00700 1 --2- 172.21.15.37:0/3564874447 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54100fdec0 0x7f541010fe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:10:42.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:42.015+0000 7f5415c00700 1 -- 172.21.15.37:0/3564874447 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5404005040 con 0x7f54100fcc10 2024-03-29T08:10:42.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:42.016+0000 7f5415c00700 1 --2- 172.21.15.37:0/3564874447 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f54100fcc10 0x7f541010f900 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f54040137b0 tx=0x7f5404009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:10:42.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:42.016+0000 7f540d600700 1 -- 172.21.15.37:0/3564874447 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f540400e070 con 0x7f54100fcc10 2024-03-29T08:10:42.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:42.016+0000 7f540d600700 1 -- 172.21.15.37:0/3564874447 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f540401c400 con 0x7f54100fcc10 2024-03-29T08:10:42.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:42.016+0000 7f5417000700 1 -- 172.21.15.37:0/3564874447 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5410110f10 con 0x7f54100fcc10 2024-03-29T08:10:42.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:42.016+0000 7f540d600700 1 -- 172.21.15.37:0/3564874447 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f540400b650 con 0x7f54100fcc10 2024-03-29T08:10:42.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:42.016+0000 7f5417000700 1 -- 172.21.15.37:0/3564874447 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54101113f0 con 0x7f54100fcc10 2024-03-29T08:10:42.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:42.017+0000 7f5417000700 1 -- 172.21.15.37:0/3564874447 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5410101850 con 0x7f54100fcc10 2024-03-29T08:10:42.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:42.018+0000 7f540d600700 1 -- 172.21.15.37:0/3564874447 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 34) v1 ==== 90703+0+0 (secure 0 0 0) 0x7f540401c9e0 con 0x7f54100fcc10 2024-03-29T08:10:42.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:42.019+0000 7f540d600700 1 --2- 172.21.15.37:0/3564874447 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7f53f806c8a0 0x7f53f806ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:10:42.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:42.019+0000 7f540d600700 1 -- 172.21.15.37:0/3564874447 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f540408d140 con 0x7f54100fcc10 2024-03-29T08:10:42.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:42.019+0000 7f5415200700 1 --2- 172.21.15.37:0/3564874447 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7f53f806c8a0 0x7f53f806ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:10:42.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:42.020+0000 7f5415200700 1 --2- 172.21.15.37:0/3564874447 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7f53f806c8a0 0x7f53f806ed50 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f53fc009b80 tx=0x7f53fc00b040 comp rx=0 tx=0).ready entity=mgr.14686 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:10:42.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:42.023+0000 7f540d600700 1 -- 172.21.15.37:0/3564874447 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181579 (secure 0 0 0) 0x7f540405b090 con 0x7f54100fcc10 2024-03-29T08:10:42.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:42.248+0000 7f5417000700 1 -- 172.21.15.37:0/3564874447 --> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f5410111610 con 0x7f53f806c8a0 2024-03-29T08:10:42.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:42.259+0000 7f540d600700 1 -- 172.21.15.37:0/3564874447 <== mgr.14686 v2:172.21.15.37:6800/4188329516 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7f5410111610 con 0x7f53f806c8a0 2024-03-29T08:10:42.259 INFO:teuthology.orchestra.run.smithi037.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-29T08:10:42.259 INFO:teuthology.orchestra.run.smithi037.stdout:alertmanager.smithi037 smithi037 *:9093,9094 running (32s) 6s ago 13m 13.7M - 0.25.0 c8568f914cd2 2f9f9746ed0f 2024-03-29T08:10:42.259 INFO:teuthology.orchestra.run.smithi037.stdout:crash.smithi037 smithi037 running (13m) 6s ago 13m 6656k - 17.2.7-789-g4213e17c cf22eb8683be e0343935b25e 2024-03-29T08:10:42.259 INFO:teuthology.orchestra.run.smithi037.stdout:crash.smithi148 smithi148 running (12m) 71s ago 12m 6656k - 17.2.7-789-g4213e17c cf22eb8683be ff5164c552c1 2024-03-29T08:10:42.259 INFO:teuthology.orchestra.run.smithi037.stdout:grafana.smithi037 smithi037 *:3000 running (13s) 6s ago 13m 45.0M - 9.4.7 954c08fa6188 6b976fe49952 2024-03-29T08:10:42.259 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi037.iifsmh smithi037 running (7m) 6s ago 7m 110M - 17.2.7-789-g4213e17c cf22eb8683be 074e1acf331f 2024-03-29T08:10:42.259 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi037.mmwgxe smithi037 running (7m) 6s ago 7m 115M - 17.2.7-789-g4213e17c cf22eb8683be 9a5b1e72de5c 2024-03-29T08:10:42.259 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi148.qukpjh smithi148 running (6m) 71s ago 6m 12.9M - 17.2.7-789-g4213e17c cf22eb8683be 8e469782a273 2024-03-29T08:10:42.259 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi148.sqteih smithi148 running (7m) 71s ago 7m 12.8M - 17.2.7-789-g4213e17c cf22eb8683be 8e68f858bb22 2024-03-29T08:10:42.259 INFO:teuthology.orchestra.run.smithi037.stdout:mgr.smithi037.wjikii smithi037 *:8443,9283,8765 running (2m) 6s ago 15m 498M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff c5a1417b9417 2024-03-29T08:10:42.259 INFO:teuthology.orchestra.run.smithi037.stdout:mgr.smithi148.onhgvm smithi148 *:8443,9283,8765 running (101s) 71s ago 12m 446M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff fcfdbce062bd 2024-03-29T08:10:42.260 INFO:teuthology.orchestra.run.smithi037.stdout:mon.smithi037 smithi037 running (15m) 6s ago 15m 54.2M 2048M 17.2.7-789-g4213e17c cf22eb8683be bb2101505730 2024-03-29T08:10:42.260 INFO:teuthology.orchestra.run.smithi037.stdout:mon.smithi148 smithi148 running (12m) 71s ago 12m 42.0M 2048M 17.2.7-789-g4213e17c cf22eb8683be 18839ed2952a 2024-03-29T08:10:42.260 INFO:teuthology.orchestra.run.smithi037.stdout:node-exporter.smithi037 smithi037 *:9100 running (81s) 6s ago 13m 7197k - 1.5.0 0da6a335fe13 26b8ae9cb878 2024-03-29T08:10:42.260 INFO:teuthology.orchestra.run.smithi037.stdout:node-exporter.smithi148 smithi148 *:9100 running (75s) 71s ago 12m 2693k - 1.5.0 0da6a335fe13 98414e4e64ed 2024-03-29T08:10:42.260 INFO:teuthology.orchestra.run.smithi037.stdout:osd.0 smithi037 running (11m) 6s ago 11m 217M 1619M 17.2.7-789-g4213e17c cf22eb8683be 559fccd0ebe1 2024-03-29T08:10:42.260 INFO:teuthology.orchestra.run.smithi037.stdout:osd.1 smithi037 running (10m) 6s ago 10m 228M 1619M 17.2.7-789-g4213e17c cf22eb8683be 5582cff11b7a 2024-03-29T08:10:42.260 INFO:teuthology.orchestra.run.smithi037.stdout:osd.2 smithi037 running (10m) 6s ago 10m 196M 1619M 17.2.7-789-g4213e17c cf22eb8683be f809bfab13e5 2024-03-29T08:10:42.260 INFO:teuthology.orchestra.run.smithi037.stdout:osd.3 smithi148 running (9m) 71s ago 9m 259M 2642M 17.2.7-789-g4213e17c cf22eb8683be a0a8224bb4b0 2024-03-29T08:10:42.260 INFO:teuthology.orchestra.run.smithi037.stdout:osd.4 smithi148 running (9m) 71s ago 9m 230M 2642M 17.2.7-789-g4213e17c cf22eb8683be a00b69047ed7 2024-03-29T08:10:42.260 INFO:teuthology.orchestra.run.smithi037.stdout:osd.5 smithi148 running (8m) 71s ago 8m 231M 2642M 17.2.7-789-g4213e17c cf22eb8683be 044a3f7db7f8 2024-03-29T08:10:42.260 INFO:teuthology.orchestra.run.smithi037.stdout:prometheus.smithi037 smithi037 *:9095 running (53s) 6s ago 12m 44.3M - 2.43.0 a07b618ecd1d 30c0223ec589 2024-03-29T08:10:42.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:42.262+0000 7f5417000700 1 -- 172.21.15.37:0/3564874447 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7f53f806c8a0 msgr2=0x7f53f806ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:10:42.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:42.262+0000 7f5417000700 1 --2- 172.21.15.37:0/3564874447 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7f53f806c8a0 0x7f53f806ed50 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f53fc009b80 tx=0x7f53fc00b040 comp rx=0 tx=0).stop 2024-03-29T08:10:42.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:42.262+0000 7f5417000700 1 -- 172.21.15.37:0/3564874447 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f54100fcc10 msgr2=0x7f541010f900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:10:42.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:42.262+0000 7f5417000700 1 --2- 172.21.15.37:0/3564874447 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f54100fcc10 0x7f541010f900 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f54040137b0 tx=0x7f5404009e30 comp rx=0 tx=0).stop 2024-03-29T08:10:42.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:42.262+0000 7f5417000700 1 -- 172.21.15.37:0/3564874447 shutdown_connections 2024-03-29T08:10:42.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:42.262+0000 7f5417000700 1 --2- 172.21.15.37:0/3564874447 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7f53f806c8a0 0x7f53f806ed50 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:10:42.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:42.262+0000 7f5417000700 1 --2- 172.21.15.37:0/3564874447 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f54100fcc10 0x7f541010f900 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:10:42.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:42.262+0000 7f5417000700 1 --2- 172.21.15.37:0/3564874447 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54100fdec0 0x7f541010fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:10:42.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:42.262+0000 7f5417000700 1 -- 172.21.15.37:0/3564874447 >> 172.21.15.37:0/3564874447 conn(0x7f54100f83d0 msgr2=0x7f54100f9d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:10:42.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:42.262+0000 7f5417000700 1 -- 172.21.15.37:0/3564874447 shutdown_connections 2024-03-29T08:10:42.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:42.262+0000 7f5417000700 1 -- 172.21.15.37:0/3564874447 wait complete. 2024-03-29T08:10:42.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:42.404+0000 7f21c8400700 1 -- 172.21.15.37:0/310481839 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f21c00fe0d0 msgr2=0x7f21c00fe540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:10:42.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:42.404+0000 7f21c8400700 1 --2- 172.21.15.37:0/310481839 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f21c00fe0d0 0x7f21c00fe540 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f21ac004940 tx=0x7f21ac009e30 comp rx=0 tx=0).stop 2024-03-29T08:10:42.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:42.404+0000 7f21c8400700 1 -- 172.21.15.37:0/310481839 shutdown_connections 2024-03-29T08:10:42.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:42.404+0000 7f21c8400700 1 --2- 172.21.15.37:0/310481839 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f21c00fe0d0 0x7f21c00fe540 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:10:42.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:42.404+0000 7f21c8400700 1 --2- 172.21.15.37:0/310481839 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f21c00fcee0 0x7f21c00fd310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:10:42.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:42.404+0000 7f21c8400700 1 -- 172.21.15.37:0/310481839 >> 172.21.15.37:0/310481839 conn(0x7f21c00f8560 msgr2=0x7f21c00fa970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:10:42.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:42.404+0000 7f21c8400700 1 -- 172.21.15.37:0/310481839 shutdown_connections 2024-03-29T08:10:42.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:42.404+0000 7f21c8400700 1 -- 172.21.15.37:0/310481839 wait complete. 2024-03-29T08:10:42.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:42.405+0000 7f21c8400700 1 Processor -- start 2024-03-29T08:10:42.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:42.405+0000 7f21c8400700 1 -- start start 2024-03-29T08:10:42.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:42.405+0000 7f21c8400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f21c00fcee0 0x7f21c0071e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:10:42.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:42.405+0000 7f21c8400700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f21c00fe0d0 0x7f21c0070310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:10:42.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:42.405+0000 7f21c8400700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21c00708c0 con 0x7f21c00fcee0 2024-03-29T08:10:42.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:42.405+0000 7f21c8400700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21c0070a30 con 0x7f21c00fe0d0 2024-03-29T08:10:42.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:42.406+0000 7f21c5a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f21c00fcee0 0x7f21c0071e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:10:42.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:42.406+0000 7f21c5a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f21c00fcee0 0x7f21c0071e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51562/0 (socket says 172.21.15.37:51562) 2024-03-29T08:10:42.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:42.406+0000 7f21c5a00700 1 -- 172.21.15.37:0/987132749 learned_addr learned my addr 172.21.15.37:0/987132749 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:10:42.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:42.406+0000 7f21c5000700 1 --2- 172.21.15.37:0/987132749 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f21c00fe0d0 0x7f21c0070310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:10:42.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:42.406+0000 7f21c5a00700 1 -- 172.21.15.37:0/987132749 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f21c00fe0d0 msgr2=0x7f21c0070310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:10:42.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:42.406+0000 7f21c5a00700 1 --2- 172.21.15.37:0/987132749 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f21c00fe0d0 0x7f21c0070310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:10:42.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:42.406+0000 7f21c5a00700 1 -- 172.21.15.37:0/987132749 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21ac005040 con 0x7f21c00fcee0 2024-03-29T08:10:42.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:42.407+0000 7f21c5a00700 1 --2- 172.21.15.37:0/987132749 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f21c00fcee0 0x7f21c0071e40 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f21b4002a80 tx=0x7f21b4004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:10:42.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:42.407+0000 7f21bd600700 1 -- 172.21.15.37:0/987132749 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f21b4007cc0 con 0x7f21c00fcee0 2024-03-29T08:10:42.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:42.407+0000 7f21bd600700 1 -- 172.21.15.37:0/987132749 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f21b400f040 con 0x7f21c00fcee0 2024-03-29T08:10:42.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:42.407+0000 7f21c8400700 1 -- 172.21.15.37:0/987132749 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21c0070d10 con 0x7f21c00fcee0 2024-03-29T08:10:42.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:42.407+0000 7f21bd600700 1 -- 172.21.15.37:0/987132749 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f21b400d9c0 con 0x7f21c00fcee0 2024-03-29T08:10:42.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:42.407+0000 7f21c8400700 1 -- 172.21.15.37:0/987132749 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21c01159a0 con 0x7f21c00fcee0 2024-03-29T08:10:42.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:42.408+0000 7f21c8400700 1 -- 172.21.15.37:0/987132749 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21c005f5f0 con 0x7f21c00fcee0 2024-03-29T08:10:42.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:42.409+0000 7f21bd600700 1 -- 172.21.15.37:0/987132749 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 34) v1 ==== 90703+0+0 (secure 0 0 0) 0x7f21b400db20 con 0x7f21c00fcee0 2024-03-29T08:10:42.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:42.410+0000 7f21bd600700 1 --2- 172.21.15.37:0/987132749 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7f21a806c8b0 0x7f21a806ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:10:42.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:42.410+0000 7f21bd600700 1 -- 172.21.15.37:0/987132749 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f21b408b780 con 0x7f21c00fcee0 2024-03-29T08:10:42.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:42.411+0000 7f21c5000700 1 --2- 172.21.15.37:0/987132749 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7f21a806c8b0 0x7f21a806ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:10:42.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:42.411+0000 7f21c5000700 1 --2- 172.21.15.37:0/987132749 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7f21a806c8b0 0x7f21a806ed60 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f21ac009fd0 tx=0x7f21ac009e30 comp rx=0 tx=0).ready entity=mgr.14686 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:10:42.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:42.415+0000 7f21bd600700 1 -- 172.21.15.37:0/987132749 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181579 (secure 0 0 0) 0x7f21b4055c40 con 0x7f21c00fcee0 2024-03-29T08:10:42.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:10:42 smithi037 ceph-mon[35162]: pgmap v80: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:10:42.428 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:10:42 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' 2024-03-29T08:10:42.428 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:10:42 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' 2024-03-29T08:10:42.428 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:10:42 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T08:10:42.428 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:10:42 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T08:10:42.428 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:10:42 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' 2024-03-29T08:10:42.428 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:10:42 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-29T08:10:42.428 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:10:42 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T08:10:42.428 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:10:42 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:10:42.428 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:10:42 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:10:42.428 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:10:42 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:10:42.428 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:10:42 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:10:42.429 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:10:42 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:10:42.429 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:10:42 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:10:42.429 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:10:42 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:10:42.429 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:10:42 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:10:42.429 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:10:42 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:10:42.429 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:10:42 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:10:42.429 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:10:42 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:10:42.429 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:10:42 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:10:42.429 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:10:42 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:10:42.429 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:10:42 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:10:42.429 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:10:42 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' 2024-03-29T08:10:42.429 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:10:42 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-03-29T08:10:42.429 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:10:42 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-03-29T08:10:42.430 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:10:42 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-29T08:10:42.430 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:10:42 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-03-29T08:10:42.430 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:10:42 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-03-29T08:10:42.430 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:10:42 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-03-29T08:10:42.430 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:10:42 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-03-29T08:10:42.430 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:10:42 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-03-29T08:10:42.430 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:10:42 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-03-29T08:10:42.430 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:10:42 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-03-29T08:10:42.430 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:10:42 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-03-29T08:10:42.430 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:10:42 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-29T08:10:42.430 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:10:42 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2024-03-29T08:10:42.430 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:10:42 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]': finished 2024-03-29T08:10:42.430 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:10:42 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-03-29T08:10:42.430 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:10:42 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-03-29T08:10:42.431 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:10:42 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-03-29T08:10:42.431 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:10:42 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-03-29T08:10:42.431 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:10:42 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2024-03-29T08:10:42.431 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:10:42 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2024-03-29T08:10:42.431 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:10:42 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-29T08:10:42.431 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:10:42 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-29T08:10:42.431 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:10:42 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-29T08:10:42.431 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:10:42 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-29T08:10:42.431 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:10:42 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-29T08:10:42.431 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:10:42 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-29T08:10:42.431 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:10:42 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config rm", "name": "mon_mds_skip_sanity", "who": "mon"}]: dispatch 2024-03-29T08:10:42.431 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:10:42 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd='[{"prefix": "config rm", "name": "mon_mds_skip_sanity", "who": "mon"}]': finished 2024-03-29T08:10:42.431 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:10:42 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-03-29T08:10:42.431 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:10:42 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-03-29T08:10:42.432 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:10:42 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T08:10:42.432 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:10:42 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' 2024-03-29T08:10:42.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:10:42 smithi148 ceph-mon[44992]: pgmap v80: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:10:42.557 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:10:42 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' 2024-03-29T08:10:42.557 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:10:42 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' 2024-03-29T08:10:42.557 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:10:42 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T08:10:42.557 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:10:42 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T08:10:42.557 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:10:42 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' 2024-03-29T08:10:42.557 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:10:42 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-29T08:10:42.557 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:10:42 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T08:10:42.557 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:10:42 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:10:42.558 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:10:42 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:10:42.558 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:10:42 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:10:42.558 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:10:42 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:10:42.558 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:10:42 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:10:42.558 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:10:42 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:10:42.558 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:10:42 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:10:42.558 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:10:42 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:10:42.558 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:10:42 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:10:42.558 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:10:42 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:10:42.558 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:10:42 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:10:42.559 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:10:42 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:10:42.559 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:10:42 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:10:42.559 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:10:42 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:10:42.559 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:10:42 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' 2024-03-29T08:10:42.559 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:10:42 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-03-29T08:10:42.559 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:10:42 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-03-29T08:10:42.559 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:10:42 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-29T08:10:42.559 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:10:42 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-03-29T08:10:42.559 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:10:42 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-03-29T08:10:42.559 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:10:42 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-03-29T08:10:42.560 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:10:42 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-03-29T08:10:42.560 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:10:42 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-03-29T08:10:42.560 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:10:42 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-03-29T08:10:42.560 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:10:42 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-03-29T08:10:42.560 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:10:42 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-03-29T08:10:42.560 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:10:42 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-29T08:10:42.560 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:10:42 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2024-03-29T08:10:42.560 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:10:42 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]': finished 2024-03-29T08:10:42.560 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:10:42 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-03-29T08:10:42.560 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:10:42 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-03-29T08:10:42.561 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:10:42 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-03-29T08:10:42.561 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:10:42 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-03-29T08:10:42.561 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:10:42 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2024-03-29T08:10:42.561 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:10:42 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2024-03-29T08:10:42.561 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:10:42 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-29T08:10:42.561 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:10:42 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-29T08:10:42.561 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:10:42 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-29T08:10:42.561 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:10:42 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-29T08:10:42.561 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:10:42 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-29T08:10:42.561 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:10:42 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-29T08:10:42.562 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:10:42 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config rm", "name": "mon_mds_skip_sanity", "who": "mon"}]: dispatch 2024-03-29T08:10:42.562 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:10:42 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd='[{"prefix": "config rm", "name": "mon_mds_skip_sanity", "who": "mon"}]': finished 2024-03-29T08:10:42.562 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:10:42 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-03-29T08:10:42.562 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:10:42 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-03-29T08:10:42.562 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:10:42 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T08:10:42.562 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:10:42 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' 2024-03-29T08:10:42.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:42.730+0000 7f21c8400700 1 -- 172.21.15.37:0/987132749 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f21c004bed0 con 0x7f21c00fcee0 2024-03-29T08:10:42.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:42.730+0000 7f21bd600700 1 -- 172.21.15.37:0/987132749 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+743 (secure 0 0 0) 0x7f21b4059260 con 0x7f21c00fcee0 2024-03-29T08:10:42.731 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-29T08:10:42.731 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": { 2024-03-29T08:10:42.731 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 17.2.7-789-g4213e17c (4213e17c4de67016ed14f55d22503f4923b3a901) quincy (stable)": 2 2024-03-29T08:10:42.731 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:10:42.731 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": { 2024-03-29T08:10:42.731 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 2 2024-03-29T08:10:42.731 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:10:42.731 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": { 2024-03-29T08:10:42.731 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 17.2.7-789-g4213e17c (4213e17c4de67016ed14f55d22503f4923b3a901) quincy (stable)": 6 2024-03-29T08:10:42.732 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:10:42.732 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": { 2024-03-29T08:10:42.732 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 17.2.7-789-g4213e17c (4213e17c4de67016ed14f55d22503f4923b3a901) quincy (stable)": 4 2024-03-29T08:10:42.732 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:10:42.732 INFO:teuthology.orchestra.run.smithi037.stdout: "overall": { 2024-03-29T08:10:42.732 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 17.2.7-789-g4213e17c (4213e17c4de67016ed14f55d22503f4923b3a901) quincy (stable)": 12, 2024-03-29T08:10:42.732 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 2 2024-03-29T08:10:42.732 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-29T08:10:42.732 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-29T08:10:42.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:42.733+0000 7f21c8400700 1 -- 172.21.15.37:0/987132749 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7f21a806c8b0 msgr2=0x7f21a806ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:10:42.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:42.733+0000 7f21c8400700 1 --2- 172.21.15.37:0/987132749 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7f21a806c8b0 0x7f21a806ed60 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f21ac009fd0 tx=0x7f21ac009e30 comp rx=0 tx=0).stop 2024-03-29T08:10:42.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:42.733+0000 7f21c8400700 1 -- 172.21.15.37:0/987132749 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f21c00fcee0 msgr2=0x7f21c0071e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:10:42.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:42.733+0000 7f21c8400700 1 --2- 172.21.15.37:0/987132749 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f21c00fcee0 0x7f21c0071e40 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f21b4002a80 tx=0x7f21b4004600 comp rx=0 tx=0).stop 2024-03-29T08:10:42.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:42.733+0000 7f21c8400700 1 -- 172.21.15.37:0/987132749 shutdown_connections 2024-03-29T08:10:42.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:42.733+0000 7f21c8400700 1 --2- 172.21.15.37:0/987132749 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7f21a806c8b0 0x7f21a806ed60 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:10:42.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:42.733+0000 7f21c8400700 1 --2- 172.21.15.37:0/987132749 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f21c00fcee0 0x7f21c0071e40 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:10:42.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:42.733+0000 7f21c8400700 1 --2- 172.21.15.37:0/987132749 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f21c00fe0d0 0x7f21c0070310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:10:42.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:42.733+0000 7f21c8400700 1 -- 172.21.15.37:0/987132749 >> 172.21.15.37:0/987132749 conn(0x7f21c00f8560 msgr2=0x7f21c00f9d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:10:42.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:42.733+0000 7f21c8400700 1 -- 172.21.15.37:0/987132749 shutdown_connections 2024-03-29T08:10:42.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:42.733+0000 7f21c8400700 1 -- 172.21.15.37:0/987132749 wait complete. 2024-03-29T08:10:42.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:42.873+0000 7f7c9f200700 1 -- 172.21.15.37:0/693041922 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7c980709a0 msgr2=0x7f7c98070e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:10:42.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:42.873+0000 7f7c9f200700 1 --2- 172.21.15.37:0/693041922 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7c980709a0 0x7f7c98070e10 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f7c84004940 tx=0x7f7c84009e30 comp rx=0 tx=0).stop 2024-03-29T08:10:42.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:42.873+0000 7f7c9f200700 1 -- 172.21.15.37:0/693041922 shutdown_connections 2024-03-29T08:10:42.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:42.873+0000 7f7c9f200700 1 --2- 172.21.15.37:0/693041922 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7c980709a0 0x7f7c98070e10 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:10:42.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:42.873+0000 7f7c9f200700 1 --2- 172.21.15.37:0/693041922 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7c98071f50 0x7f7c98070460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:10:42.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:42.873+0000 7f7c9f200700 1 -- 172.21.15.37:0/693041922 >> 172.21.15.37:0/693041922 conn(0x7f7c980f8560 msgr2=0x7f7c980fa970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:10:42.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:42.874+0000 7f7c9f200700 1 -- 172.21.15.37:0/693041922 shutdown_connections 2024-03-29T08:10:42.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:42.874+0000 7f7c9f200700 1 -- 172.21.15.37:0/693041922 wait complete. 2024-03-29T08:10:42.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:42.874+0000 7f7c9f200700 1 Processor -- start 2024-03-29T08:10:42.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:42.875+0000 7f7c9f200700 1 -- start start 2024-03-29T08:10:42.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:42.875+0000 7f7c9f200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7c980709a0 0x7f7c9810faa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:10:42.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:42.875+0000 7f7c9f200700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7c98071f50 0x7f7c9810ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:10:42.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:42.875+0000 7f7c9f200700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c981105e0 con 0x7f7c980709a0 2024-03-29T08:10:42.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:42.875+0000 7f7c9f200700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c98110e30 con 0x7f7c98071f50 2024-03-29T08:10:42.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:42.875+0000 7f7c97e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7c980709a0 0x7f7c9810faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:10:42.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:42.875+0000 7f7c97e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7c980709a0 0x7f7c9810faa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51576/0 (socket says 172.21.15.37:51576) 2024-03-29T08:10:42.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:42.875+0000 7f7c97e00700 1 -- 172.21.15.37:0/550323713 learned_addr learned my addr 172.21.15.37:0/550323713 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:10:42.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:42.875+0000 7f7c97400700 1 --2- 172.21.15.37:0/550323713 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7c98071f50 0x7f7c9810ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:10:42.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:42.876+0000 7f7c97e00700 1 -- 172.21.15.37:0/550323713 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7c98071f50 msgr2=0x7f7c9810ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:10:42.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:42.876+0000 7f7c97e00700 1 --2- 172.21.15.37:0/550323713 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7c98071f50 0x7f7c9810ffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:10:42.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:42.876+0000 7f7c97e00700 1 -- 172.21.15.37:0/550323713 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c84005040 con 0x7f7c980709a0 2024-03-29T08:10:42.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:42.876+0000 7f7c97e00700 1 --2- 172.21.15.37:0/550323713 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7c980709a0 0x7f7c9810faa0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f7c8800eb70 tx=0x7f7c8800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:10:42.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:42.876+0000 7f7c8fe00700 1 -- 172.21.15.37:0/550323713 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7c88007cc0 con 0x7f7c980709a0 2024-03-29T08:10:42.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:42.876+0000 7f7c8fe00700 1 -- 172.21.15.37:0/550323713 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f7c8800f040 con 0x7f7c980709a0 2024-03-29T08:10:42.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:42.876+0000 7f7c9f200700 1 -- 172.21.15.37:0/550323713 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c981110b0 con 0x7f7c980709a0 2024-03-29T08:10:42.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:42.876+0000 7f7c8fe00700 1 -- 172.21.15.37:0/550323713 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7c8800c9d0 con 0x7f7c980709a0 2024-03-29T08:10:42.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:42.876+0000 7f7c9f200700 1 -- 172.21.15.37:0/550323713 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c981115f0 con 0x7f7c980709a0 2024-03-29T08:10:42.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:42.878+0000 7f7c9f200700 1 -- 172.21.15.37:0/550323713 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c9805f5f0 con 0x7f7c980709a0 2024-03-29T08:10:42.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:42.878+0000 7f7c8fe00700 1 -- 172.21.15.37:0/550323713 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 34) v1 ==== 90703+0+0 (secure 0 0 0) 0x7f7c8800cb30 con 0x7f7c980709a0 2024-03-29T08:10:42.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:42.880+0000 7f7c8fe00700 1 --2- 172.21.15.37:0/550323713 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7f7c7c06c8b0 0x7f7c7c06ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:10:42.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:42.880+0000 7f7c8fe00700 1 -- 172.21.15.37:0/550323713 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f7c8808b4e0 con 0x7f7c980709a0 2024-03-29T08:10:42.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:42.883+0000 7f7c97400700 1 --2- 172.21.15.37:0/550323713 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7f7c7c06c8b0 0x7f7c7c06ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:10:42.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:42.885+0000 7f7c97400700 1 --2- 172.21.15.37:0/550323713 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7f7c7c06c8b0 0x7f7c7c06ed60 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f7c84009fd0 tx=0x7f7c84009e30 comp rx=0 tx=0).ready entity=mgr.14686 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:10:42.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:42.887+0000 7f7c8fe00700 1 -- 172.21.15.37:0/550323713 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181579 (secure 0 0 0) 0x7f7c88059580 con 0x7f7c980709a0 2024-03-29T08:10:43.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:43.121+0000 7f7c9f200700 1 -- 172.21.15.37:0/550323713 --> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7c981119d0 con 0x7f7c7c06c8b0 2024-03-29T08:10:43.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:43.122+0000 7f7c8fe00700 1 -- 172.21.15.37:0/550323713 <== mgr.14686 v2:172.21.15.37:6800/4188329516 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+175 (secure 0 0 0) 0x7f7c981119d0 con 0x7f7c7c06c8b0 2024-03-29T08:10:43.122 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-29T08:10:43.122 INFO:teuthology.orchestra.run.smithi037.stdout: "target_image": null, 2024-03-29T08:10:43.122 INFO:teuthology.orchestra.run.smithi037.stdout: "in_progress": false, 2024-03-29T08:10:43.122 INFO:teuthology.orchestra.run.smithi037.stdout: "which": "", 2024-03-29T08:10:43.122 INFO:teuthology.orchestra.run.smithi037.stdout: "services_complete": [], 2024-03-29T08:10:43.123 INFO:teuthology.orchestra.run.smithi037.stdout: "progress": null, 2024-03-29T08:10:43.123 INFO:teuthology.orchestra.run.smithi037.stdout: "message": "", 2024-03-29T08:10:43.123 INFO:teuthology.orchestra.run.smithi037.stdout: "is_paused": false 2024-03-29T08:10:43.123 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-29T08:10:43.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:43.125+0000 7f7c9f200700 1 -- 172.21.15.37:0/550323713 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7f7c7c06c8b0 msgr2=0x7f7c7c06ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:10:43.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:43.125+0000 7f7c9f200700 1 --2- 172.21.15.37:0/550323713 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7f7c7c06c8b0 0x7f7c7c06ed60 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f7c84009fd0 tx=0x7f7c84009e30 comp rx=0 tx=0).stop 2024-03-29T08:10:43.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:43.125+0000 7f7c9f200700 1 -- 172.21.15.37:0/550323713 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7c980709a0 msgr2=0x7f7c9810faa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:10:43.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:43.125+0000 7f7c9f200700 1 --2- 172.21.15.37:0/550323713 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7c980709a0 0x7f7c9810faa0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f7c8800eb70 tx=0x7f7c8800b040 comp rx=0 tx=0).stop 2024-03-29T08:10:43.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:43.125+0000 7f7c9f200700 1 -- 172.21.15.37:0/550323713 shutdown_connections 2024-03-29T08:10:43.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:43.125+0000 7f7c9f200700 1 --2- 172.21.15.37:0/550323713 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7f7c7c06c8b0 0x7f7c7c06ed60 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:10:43.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:43.125+0000 7f7c9f200700 1 --2- 172.21.15.37:0/550323713 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7c980709a0 0x7f7c9810faa0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:10:43.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:43.125+0000 7f7c9f200700 1 --2- 172.21.15.37:0/550323713 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7c98071f50 0x7f7c9810ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:10:43.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:43.125+0000 7f7c9f200700 1 -- 172.21.15.37:0/550323713 >> 172.21.15.37:0/550323713 conn(0x7f7c980f8560 msgr2=0x7f7c980f9c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:10:43.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:43.125+0000 7f7c9f200700 1 -- 172.21.15.37:0/550323713 shutdown_connections 2024-03-29T08:10:43.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:10:43.125+0000 7f7c9f200700 1 -- 172.21.15.37:0/550323713 wait complete. 2024-03-29T08:10:43.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:10:43 smithi037 ceph-mon[35162]: from='client.14802 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:10:43.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:10:43 smithi037 ceph-mon[35162]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-29T08:10:43.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:10:43 smithi037 ceph-mon[35162]: Upgrade: Finalizing container_image settings 2024-03-29T08:10:43.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:10:43 smithi037 ceph-mon[35162]: Upgrade: Complete! 2024-03-29T08:10:43.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:10:43 smithi037 ceph-mon[35162]: from='client.14806 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:10:43.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:10:43 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T08:10:43.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:10:43 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T08:10:43.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:10:43 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' 2024-03-29T08:10:43.428 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:10:43 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T08:10:43.428 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:10:43 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T08:10:43.428 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:10:43 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T08:10:43.428 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:10:43 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' 2024-03-29T08:10:43.428 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:10:43 smithi037 ceph-mon[35162]: from='client.? 172.21.15.37:0/987132749' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:10:43.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:10:43 smithi148 ceph-mon[44992]: from='client.14802 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:10:43.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:10:43 smithi148 ceph-mon[44992]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-29T08:10:43.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:10:43 smithi148 ceph-mon[44992]: Upgrade: Finalizing container_image settings 2024-03-29T08:10:43.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:10:43 smithi148 ceph-mon[44992]: Upgrade: Complete! 2024-03-29T08:10:43.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:10:43 smithi148 ceph-mon[44992]: from='client.14806 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:10:43.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:10:43 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T08:10:43.557 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:10:43 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T08:10:43.557 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:10:43 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' 2024-03-29T08:10:43.557 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:10:43 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T08:10:43.557 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:10:43 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T08:10:43.557 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:10:43 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T08:10:43.557 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:10:43 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' 2024-03-29T08:10:43.557 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:10:43 smithi148 ceph-mon[44992]: from='client.? 172.21.15.37:0/987132749' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:10:44.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:10:44 smithi037 ceph-mon[35162]: from='client.14810 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:10:44.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:10:44 smithi037 ceph-mon[35162]: pgmap v81: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:10:44.428 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:10:44 smithi037 ceph-mon[35162]: from='client.14818 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:10:44.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:10:44 smithi148 ceph-mon[44992]: from='client.14810 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:10:44.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:10:44 smithi148 ceph-mon[44992]: pgmap v81: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:10:44.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:10:44 smithi148 ceph-mon[44992]: from='client.14818 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:10:46.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:10:46 smithi037 ceph-mon[35162]: pgmap v82: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:10:46.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:10:46 smithi148 ceph-mon[44992]: pgmap v82: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:10:48.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:10:48 smithi037 ceph-mon[35162]: pgmap v83: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:10:48.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:10:48 smithi148 ceph-mon[44992]: pgmap v83: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:10:50.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:10:50 smithi037 ceph-mon[35162]: pgmap v84: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:10:50.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:10:50 smithi148 ceph-mon[44992]: pgmap v84: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:10:52.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:10:52 smithi037 ceph-mon[35162]: pgmap v85: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:10:52.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:10:52 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:10:52.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:10:52 smithi148 ceph-mon[44992]: pgmap v85: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:10:52.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:10:52 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:10:54.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:10:54 smithi037 ceph-mon[35162]: pgmap v86: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:10:54.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:10:54 smithi148 ceph-mon[44992]: pgmap v86: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:10:56.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:10:56 smithi037 ceph-mon[35162]: pgmap v87: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:10:56.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:10:56 smithi148 ceph-mon[44992]: pgmap v87: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:10:58.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:10:58 smithi037 ceph-mon[35162]: pgmap v88: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:10:58.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:10:58 smithi148 ceph-mon[44992]: pgmap v88: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:11:00.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:00 smithi037 ceph-mon[35162]: pgmap v89: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:11:00.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:11:00 smithi148 ceph-mon[44992]: pgmap v89: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:11:02.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:02 smithi037 ceph-mon[35162]: pgmap v90: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:11:02.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:11:02 smithi148 ceph-mon[44992]: pgmap v90: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:11:04.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:04 smithi037 ceph-mon[35162]: pgmap v91: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:11:04.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:11:04 smithi148 ceph-mon[44992]: pgmap v91: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:11:05.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:05 smithi037 ceph-mon[35162]: fsmap cephfs:2 {0=cephfs.smithi037.mmwgxe=up:active,1=cephfs.smithi037.iifsmh=up:active} 2 up:standby 2024-03-29T08:11:05.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:11:05 smithi148 ceph-mon[44992]: fsmap cephfs:2 {0=cephfs.smithi037.mmwgxe=up:active,1=cephfs.smithi037.iifsmh=up:active} 2 up:standby 2024-03-29T08:11:06.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:06 smithi037 ceph-mon[35162]: pgmap v92: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:11:06.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:11:06 smithi148 ceph-mon[44992]: pgmap v92: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:11:07.428 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:07 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:11:07.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:11:07 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:11:08.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:08 smithi037 ceph-mon[35162]: pgmap v93: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:11:08.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:11:08 smithi148 ceph-mon[44992]: pgmap v93: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:11:10.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:10 smithi037 ceph-mon[35162]: pgmap v94: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:11:10.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:11:10 smithi148 ceph-mon[44992]: pgmap v94: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:11:12.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:12 smithi037 ceph-mon[35162]: pgmap v95: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:11:12.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:11:12 smithi148 ceph-mon[44992]: pgmap v95: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:11:13.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:13.281+0000 7fe5f5600700 1 -- 172.21.15.37:0/2680541855 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe5f00fcc10 msgr2=0x7fe5f00fd020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:11:13.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:13.281+0000 7fe5f5600700 1 --2- 172.21.15.37:0/2680541855 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe5f00fcc10 0x7fe5f00fd020 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fe5e0004940 tx=0x7fe5e0009e30 comp rx=0 tx=0).stop 2024-03-29T08:11:13.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:13.281+0000 7fe5f5600700 1 -- 172.21.15.37:0/2680541855 shutdown_connections 2024-03-29T08:11:13.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:13.281+0000 7fe5f5600700 1 --2- 172.21.15.37:0/2680541855 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe5f00fdec0 0x7fe5f00fe330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:11:13.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:13.281+0000 7fe5f5600700 1 --2- 172.21.15.37:0/2680541855 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe5f00fcc10 0x7fe5f00fd020 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:11:13.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:13.281+0000 7fe5f5600700 1 -- 172.21.15.37:0/2680541855 >> 172.21.15.37:0/2680541855 conn(0x7fe5f00f83d0 msgr2=0x7fe5f00fa7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:11:13.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:13.282+0000 7fe5f5600700 1 -- 172.21.15.37:0/2680541855 shutdown_connections 2024-03-29T08:11:13.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:13.282+0000 7fe5f5600700 1 -- 172.21.15.37:0/2680541855 wait complete. 2024-03-29T08:11:13.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:13.282+0000 7fe5f5600700 1 Processor -- start 2024-03-29T08:11:13.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:13.283+0000 7fe5f5600700 1 -- start start 2024-03-29T08:11:13.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:13.283+0000 7fe5f5600700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe5f00fcc10 0x7fe5f010f980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:11:13.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:13.283+0000 7fe5f5600700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe5f00fdec0 0x7fe5f010fec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:11:13.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:13.283+0000 7fe5f5600700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5f01104c0 con 0x7fe5f00fdec0 2024-03-29T08:11:13.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:13.283+0000 7fe5f5600700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5f0110d10 con 0x7fe5f00fcc10 2024-03-29T08:11:13.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:13.283+0000 7fe5ef400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe5f00fdec0 0x7fe5f010fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:11:13.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:13.283+0000 7fe5ef400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe5f00fdec0 0x7fe5f010fec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57976/0 (socket says 172.21.15.37:57976) 2024-03-29T08:11:13.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:13.283+0000 7fe5ef400700 1 -- 172.21.15.37:0/1487966721 learned_addr learned my addr 172.21.15.37:0/1487966721 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:11:13.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:13.284+0000 7fe5efe00700 1 --2- 172.21.15.37:0/1487966721 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe5f00fcc10 0x7fe5f010f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:11:13.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:13.284+0000 7fe5ef400700 1 -- 172.21.15.37:0/1487966721 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe5f00fcc10 msgr2=0x7fe5f010f980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:11:13.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:13.284+0000 7fe5ef400700 1 --2- 172.21.15.37:0/1487966721 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe5f00fcc10 0x7fe5f010f980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:11:13.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:13.284+0000 7fe5ef400700 1 -- 172.21.15.37:0/1487966721 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5e0005040 con 0x7fe5f00fdec0 2024-03-29T08:11:13.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:13.284+0000 7fe5efe00700 1 --2- 172.21.15.37:0/1487966721 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe5f00fcc10 0x7fe5f010f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:11:13.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:13.284+0000 7fe5ef400700 1 --2- 172.21.15.37:0/1487966721 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe5f00fdec0 0x7fe5f010fec0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fe5dc002a50 tx=0x7fe5dc0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:11:13.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:13.284+0000 7fe5e7e00700 1 -- 172.21.15.37:0/1487966721 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe5dc007cc0 con 0x7fe5f00fdec0 2024-03-29T08:11:13.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:13.284+0000 7fe5e7e00700 1 -- 172.21.15.37:0/1487966721 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fe5dc00f040 con 0x7fe5f00fdec0 2024-03-29T08:11:13.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:13.284+0000 7fe5f5600700 1 -- 172.21.15.37:0/1487966721 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5f0110ff0 con 0x7fe5f00fdec0 2024-03-29T08:11:13.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:13.284+0000 7fe5e7e00700 1 -- 172.21.15.37:0/1487966721 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe5dc009540 con 0x7fe5f00fdec0 2024-03-29T08:11:13.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:13.285+0000 7fe5f5600700 1 -- 172.21.15.37:0/1487966721 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5f0111530 con 0x7fe5f00fdec0 2024-03-29T08:11:13.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:13.286+0000 7fe5f5600700 1 -- 172.21.15.37:0/1487966721 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5f0101850 con 0x7fe5f00fdec0 2024-03-29T08:11:13.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:13.287+0000 7fe5e7e00700 1 -- 172.21.15.37:0/1487966721 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 34) v1 ==== 90703+0+0 (secure 0 0 0) 0x7fe5dc0096a0 con 0x7fe5f00fdec0 2024-03-29T08:11:13.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:13.288+0000 7fe5e7e00700 1 --2- 172.21.15.37:0/1487966721 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7fe5d406c8b0 0x7fe5d406ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:11:13.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:13.288+0000 7fe5e7e00700 1 -- 172.21.15.37:0/1487966721 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6066+0+0 (secure 0 0 0) 0x7fe5dc08b850 con 0x7fe5f00fdec0 2024-03-29T08:11:13.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:13.288+0000 7fe5efe00700 1 --2- 172.21.15.37:0/1487966721 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7fe5d406c8b0 0x7fe5d406ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:11:13.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:13.289+0000 7fe5efe00700 1 --2- 172.21.15.37:0/1487966721 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7fe5d406c8b0 0x7fe5d406ed60 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fe5e0004f90 tx=0x7fe5e0009e30 comp rx=0 tx=0).ready entity=mgr.14686 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:11:13.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:13.293+0000 7fe5e7e00700 1 -- 172.21.15.37:0/1487966721 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181579 (secure 0 0 0) 0x7fe5dc0597a0 con 0x7fe5f00fdec0 2024-03-29T08:11:13.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:13.525+0000 7fe5f5600700 1 -- 172.21.15.37:0/1487966721 --> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe5f01117a0 con 0x7fe5d406c8b0 2024-03-29T08:11:13.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:13.526+0000 7fe5e7e00700 1 -- 172.21.15.37:0/1487966721 <== mgr.14686 v2:172.21.15.37:6800/4188329516 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+175 (secure 0 0 0) 0x7fe5f01117a0 con 0x7fe5d406c8b0 2024-03-29T08:11:13.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:13.528+0000 7fe5f5600700 1 -- 172.21.15.37:0/1487966721 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7fe5d406c8b0 msgr2=0x7fe5d406ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:11:13.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:13.528+0000 7fe5f5600700 1 --2- 172.21.15.37:0/1487966721 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7fe5d406c8b0 0x7fe5d406ed60 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fe5e0004f90 tx=0x7fe5e0009e30 comp rx=0 tx=0).stop 2024-03-29T08:11:13.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:13.528+0000 7fe5f5600700 1 -- 172.21.15.37:0/1487966721 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe5f00fdec0 msgr2=0x7fe5f010fec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:11:13.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:13.528+0000 7fe5f5600700 1 --2- 172.21.15.37:0/1487966721 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe5f00fdec0 0x7fe5f010fec0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fe5dc002a50 tx=0x7fe5dc0046b0 comp rx=0 tx=0).stop 2024-03-29T08:11:13.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:13.528+0000 7fe5f5600700 1 -- 172.21.15.37:0/1487966721 shutdown_connections 2024-03-29T08:11:13.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:13.528+0000 7fe5f5600700 1 --2- 172.21.15.37:0/1487966721 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7fe5d406c8b0 0x7fe5d406ed60 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:11:13.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:13.528+0000 7fe5f5600700 1 --2- 172.21.15.37:0/1487966721 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe5f00fcc10 0x7fe5f010f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:11:13.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:13.528+0000 7fe5f5600700 1 --2- 172.21.15.37:0/1487966721 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe5f00fdec0 0x7fe5f010fec0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:11:13.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:13.528+0000 7fe5f5600700 1 -- 172.21.15.37:0/1487966721 >> 172.21.15.37:0/1487966721 conn(0x7fe5f00f83d0 msgr2=0x7fe5f00f9ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:11:13.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:13.529+0000 7fe5f5600700 1 -- 172.21.15.37:0/1487966721 shutdown_connections 2024-03-29T08:11:13.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:13.529+0000 7fe5f5600700 1 -- 172.21.15.37:0/1487966721 wait complete. 2024-03-29T08:11:14.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:14 smithi037 ceph-mon[35162]: pgmap v96: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:11:14.329 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 99b492c2-eda1-11ee-b647-cb9ed24678a4 -e sha1=d0e392d9aee635f2b65768c460c7a151ead7a5e2 -- bash -c 'ceph versions | jq -e '"'"'.mgr | length == 1'"'"'' 2024-03-29T08:11:14.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:11:14 smithi148 ceph-mon[44992]: pgmap v96: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:11:14.769 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/mon.smithi037/config 2024-03-29T08:11:15.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:15 smithi037 ceph-mon[35162]: from='client.14822 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:11:15.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:11:15 smithi148 ceph-mon[44992]: from='client.14822 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:11:16.383 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:16 smithi037 ceph-mon[35162]: pgmap v97: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:11:16.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:11:16 smithi148 ceph-mon[44992]: pgmap v97: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:11:16.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:16.698+0000 7ff705e00700 1 -- 172.21.15.37:0/646010865 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff700102e00 msgr2=0x7ff7001031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:11:16.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:16.698+0000 7ff705e00700 1 --2- 172.21.15.37:0/646010865 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff700102e00 0x7ff7001031d0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7ff6f0004940 tx=0x7ff6f0009e30 comp rx=0 tx=0).stop 2024-03-29T08:11:16.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:16.698+0000 7ff705e00700 1 -- 172.21.15.37:0/646010865 shutdown_connections 2024-03-29T08:11:16.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:16.698+0000 7ff705e00700 1 --2- 172.21.15.37:0/646010865 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff7001037a0 0x7ff7000fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:11:16.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:16.698+0000 7ff705e00700 1 --2- 172.21.15.37:0/646010865 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff700102e00 0x7ff7001031d0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:11:16.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:16.698+0000 7ff705e00700 1 -- 172.21.15.37:0/646010865 >> 172.21.15.37:0/646010865 conn(0x7ff7000f89b0 msgr2=0x7ff7000fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:11:16.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:16.698+0000 7ff705e00700 1 -- 172.21.15.37:0/646010865 shutdown_connections 2024-03-29T08:11:16.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:16.698+0000 7ff705e00700 1 -- 172.21.15.37:0/646010865 wait complete. 2024-03-29T08:11:16.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:16.699+0000 7ff705e00700 1 Processor -- start 2024-03-29T08:11:16.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:16.699+0000 7ff705e00700 1 -- start start 2024-03-29T08:11:16.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:16.700+0000 7ff705e00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff700102e00 0x7ff7000702c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:11:16.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:16.700+0000 7ff705e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff7001037a0 0x7ff700070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:11:16.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:16.700+0000 7ff705e00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff700070d90 con 0x7ff7001037a0 2024-03-29T08:11:16.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:16.700+0000 7ff705e00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff700070ed0 con 0x7ff700102e00 2024-03-29T08:11:16.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:16.700+0000 7ff6fe000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff7001037a0 0x7ff700070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:11:16.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:16.700+0000 7ff6fe000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff7001037a0 0x7ff700070800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58008/0 (socket says 172.21.15.37:58008) 2024-03-29T08:11:16.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:16.700+0000 7ff6fe000700 1 -- 172.21.15.37:0/25319925 learned_addr learned my addr 172.21.15.37:0/25319925 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:11:16.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:16.700+0000 7ff6fea00700 1 --2- 172.21.15.37:0/25319925 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff700102e00 0x7ff7000702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:11:16.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:16.700+0000 7ff6fe000700 1 -- 172.21.15.37:0/25319925 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff700102e00 msgr2=0x7ff7000702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:11:16.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:16.700+0000 7ff6fe000700 1 --2- 172.21.15.37:0/25319925 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff700102e00 0x7ff7000702c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:11:16.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:16.700+0000 7ff6fe000700 1 -- 172.21.15.37:0/25319925 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6f0005040 con 0x7ff7001037a0 2024-03-29T08:11:16.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:16.700+0000 7ff6fea00700 1 --2- 172.21.15.37:0/25319925 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff700102e00 0x7ff7000702c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:11:16.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:16.701+0000 7ff6fe000700 1 --2- 172.21.15.37:0/25319925 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff7001037a0 0x7ff700070800 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7ff6ec002a50 tx=0x7ff6ec004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:11:16.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:16.701+0000 7ff6f6a00700 1 -- 172.21.15.37:0/25319925 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff6ec007cc0 con 0x7ff7001037a0 2024-03-29T08:11:16.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:16.701+0000 7ff6f6a00700 1 -- 172.21.15.37:0/25319925 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ff6ec00f040 con 0x7ff7001037a0 2024-03-29T08:11:16.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:16.701+0000 7ff6f6a00700 1 -- 172.21.15.37:0/25319925 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff6ec00d9a0 con 0x7ff7001037a0 2024-03-29T08:11:16.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:16.701+0000 7ff705e00700 1 -- 172.21.15.37:0/25319925 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff700119e20 con 0x7ff7001037a0 2024-03-29T08:11:16.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:16.701+0000 7ff705e00700 1 -- 172.21.15.37:0/25319925 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff70011a240 con 0x7ff7001037a0 2024-03-29T08:11:16.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:16.702+0000 7ff705e00700 1 -- 172.21.15.37:0/25319925 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7000fe550 con 0x7ff7001037a0 2024-03-29T08:11:16.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:16.703+0000 7ff6f6a00700 1 -- 172.21.15.37:0/25319925 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 34) v1 ==== 90703+0+0 (secure 0 0 0) 0x7ff6ec00dc60 con 0x7ff7001037a0 2024-03-29T08:11:16.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:16.704+0000 7ff6f6a00700 1 --2- 172.21.15.37:0/25319925 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7ff6e406c8a0 0x7ff6e406ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:11:16.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:16.704+0000 7ff6f6a00700 1 -- 172.21.15.37:0/25319925 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6066+0+0 (secure 0 0 0) 0x7ff6ec08b960 con 0x7ff7001037a0 2024-03-29T08:11:16.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:16.704+0000 7ff6fea00700 1 --2- 172.21.15.37:0/25319925 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7ff6e406c8a0 0x7ff6e406ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:11:16.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:16.705+0000 7ff6fea00700 1 --2- 172.21.15.37:0/25319925 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7ff6e406c8a0 0x7ff6e406ed50 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7ff6f0004560 tx=0x7ff6f0009e30 comp rx=0 tx=0).ready entity=mgr.14686 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:11:16.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:16.709+0000 7ff6f6a00700 1 -- 172.21.15.37:0/25319925 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181579 (secure 0 0 0) 0x7ff6ec055da0 con 0x7ff7001037a0 2024-03-29T08:11:17.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:17.011+0000 7ff705e00700 1 -- 172.21.15.37:0/25319925 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7ff700109e50 con 0x7ff7001037a0 2024-03-29T08:11:17.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:17.011+0000 7ff6f6a00700 1 -- 172.21.15.37:0/25319925 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+743 (secure 0 0 0) 0x7ff6ec0593c0 con 0x7ff7001037a0 2024-03-29T08:11:17.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:17.013+0000 7ff705e00700 1 -- 172.21.15.37:0/25319925 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7ff6e406c8a0 msgr2=0x7ff6e406ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:11:17.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:17.013+0000 7ff705e00700 1 --2- 172.21.15.37:0/25319925 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7ff6e406c8a0 0x7ff6e406ed50 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7ff6f0004560 tx=0x7ff6f0009e30 comp rx=0 tx=0).stop 2024-03-29T08:11:17.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:17.013+0000 7ff705e00700 1 -- 172.21.15.37:0/25319925 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff7001037a0 msgr2=0x7ff700070800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:11:17.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:17.013+0000 7ff705e00700 1 --2- 172.21.15.37:0/25319925 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff7001037a0 0x7ff700070800 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7ff6ec002a50 tx=0x7ff6ec004600 comp rx=0 tx=0).stop 2024-03-29T08:11:17.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:17.014+0000 7ff705e00700 1 -- 172.21.15.37:0/25319925 shutdown_connections 2024-03-29T08:11:17.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:17.014+0000 7ff705e00700 1 --2- 172.21.15.37:0/25319925 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7ff6e406c8a0 0x7ff6e406ed50 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:11:17.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:17.014+0000 7ff705e00700 1 --2- 172.21.15.37:0/25319925 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff700102e00 0x7ff7000702c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:11:17.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:17.014+0000 7ff705e00700 1 --2- 172.21.15.37:0/25319925 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff7001037a0 0x7ff700070800 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:11:17.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:17.014+0000 7ff705e00700 1 -- 172.21.15.37:0/25319925 >> 172.21.15.37:0/25319925 conn(0x7ff7000f89b0 msgr2=0x7ff7000f9a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:11:17.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:17.014+0000 7ff705e00700 1 -- 172.21.15.37:0/25319925 shutdown_connections 2024-03-29T08:11:17.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:17.014+0000 7ff705e00700 1 -- 172.21.15.37:0/25319925 wait complete. 2024-03-29T08:11:17.029 INFO:teuthology.orchestra.run.smithi037.stdout:true 2024-03-29T08:11:17.358 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:17 smithi037 ceph-mon[35162]: from='client.? 172.21.15.37:0/25319925' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:11:17.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:11:17 smithi148 ceph-mon[44992]: from='client.? 172.21.15.37:0/25319925' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:11:17.708 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 99b492c2-eda1-11ee-b647-cb9ed24678a4 -e sha1=d0e392d9aee635f2b65768c460c7a151ead7a5e2 -- bash -c 'ceph versions | jq -e '"'"'.mgr | keys'"'"' | grep $sha1' 2024-03-29T08:11:18.147 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/mon.smithi037/config 2024-03-29T08:11:18.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:18 smithi037 ceph-mon[35162]: pgmap v98: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:11:18.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:11:18 smithi148 ceph-mon[44992]: pgmap v98: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:11:20.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:20.003+0000 7fc84da00700 1 -- 172.21.15.37:0/932449806 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc8481037a0 msgr2=0x7fc8480fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:11:20.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:20.003+0000 7fc84da00700 1 --2- 172.21.15.37:0/932449806 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc8481037a0 0x7fc8480fce10 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fc834004940 tx=0x7fc834009e30 comp rx=0 tx=0).stop 2024-03-29T08:11:20.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:20.003+0000 7fc84da00700 1 -- 172.21.15.37:0/932449806 shutdown_connections 2024-03-29T08:11:20.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:20.003+0000 7fc84da00700 1 --2- 172.21.15.37:0/932449806 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc8481037a0 0x7fc8480fce10 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:11:20.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:20.003+0000 7fc84da00700 1 --2- 172.21.15.37:0/932449806 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc848102e00 0x7fc8481031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:11:20.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:20.003+0000 7fc84da00700 1 -- 172.21.15.37:0/932449806 >> 172.21.15.37:0/932449806 conn(0x7fc8480f89b0 msgr2=0x7fc8480fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:11:20.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:20.003+0000 7fc84da00700 1 -- 172.21.15.37:0/932449806 shutdown_connections 2024-03-29T08:11:20.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:20.003+0000 7fc84da00700 1 -- 172.21.15.37:0/932449806 wait complete. 2024-03-29T08:11:20.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:20.004+0000 7fc84da00700 1 Processor -- start 2024-03-29T08:11:20.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:20.004+0000 7fc84da00700 1 -- start start 2024-03-29T08:11:20.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:20.005+0000 7fc84da00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc848102e00 0x7fc848115d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:11:20.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:20.005+0000 7fc84da00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc8481037a0 0x7fc848116240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:11:20.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:20.005+0000 7fc84da00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc848116890 con 0x7fc848102e00 2024-03-29T08:11:20.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:20.005+0000 7fc84da00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc84810fc60 con 0x7fc8481037a0 2024-03-29T08:11:20.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:20.005+0000 7fc846a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc848102e00 0x7fc848115d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:11:20.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:20.005+0000 7fc846a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc848102e00 0x7fc848115d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39894/0 (socket says 172.21.15.37:39894) 2024-03-29T08:11:20.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:20.005+0000 7fc846a00700 1 -- 172.21.15.37:0/1726767924 learned_addr learned my addr 172.21.15.37:0/1726767924 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:11:20.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:20.005+0000 7fc846000700 1 --2- 172.21.15.37:0/1726767924 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc8481037a0 0x7fc848116240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:11:20.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:20.005+0000 7fc846a00700 1 -- 172.21.15.37:0/1726767924 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc8481037a0 msgr2=0x7fc848116240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:11:20.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:20.005+0000 7fc846a00700 1 --2- 172.21.15.37:0/1726767924 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc8481037a0 0x7fc848116240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:11:20.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:20.005+0000 7fc846a00700 1 -- 172.21.15.37:0/1726767924 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc834005040 con 0x7fc848102e00 2024-03-29T08:11:20.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:20.006+0000 7fc846a00700 1 --2- 172.21.15.37:0/1726767924 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc848102e00 0x7fc848115d00 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fc838002a50 tx=0x7fc8380046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:11:20.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:20.006+0000 7fc83ea00700 1 -- 172.21.15.37:0/1726767924 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc838007cc0 con 0x7fc848102e00 2024-03-29T08:11:20.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:20.006+0000 7fc84da00700 1 -- 172.21.15.37:0/1726767924 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc84810ff40 con 0x7fc848102e00 2024-03-29T08:11:20.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:20.006+0000 7fc83ea00700 1 -- 172.21.15.37:0/1726767924 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fc83800f040 con 0x7fc848102e00 2024-03-29T08:11:20.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:20.006+0000 7fc83ea00700 1 -- 172.21.15.37:0/1726767924 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc83800d9c0 con 0x7fc848102e00 2024-03-29T08:11:20.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:20.006+0000 7fc84da00700 1 -- 172.21.15.37:0/1726767924 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc848110500 con 0x7fc848102e00 2024-03-29T08:11:20.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:20.007+0000 7fc84da00700 1 -- 172.21.15.37:0/1726767924 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8480fe550 con 0x7fc848102e00 2024-03-29T08:11:20.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:20.008+0000 7fc83ea00700 1 -- 172.21.15.37:0/1726767924 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 34) v1 ==== 90703+0+0 (secure 0 0 0) 0x7fc83800db20 con 0x7fc848102e00 2024-03-29T08:11:20.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:20.009+0000 7fc83ea00700 1 --2- 172.21.15.37:0/1726767924 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7fc82c06c8a0 0x7fc82c06ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:11:20.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:20.009+0000 7fc83ea00700 1 -- 172.21.15.37:0/1726767924 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6066+0+0 (secure 0 0 0) 0x7fc83808b6f0 con 0x7fc848102e00 2024-03-29T08:11:20.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:20.010+0000 7fc846000700 1 --2- 172.21.15.37:0/1726767924 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7fc82c06c8a0 0x7fc82c06ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:11:20.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:20.010+0000 7fc846000700 1 --2- 172.21.15.37:0/1726767924 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7fc82c06c8a0 0x7fc82c06ed50 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fc8481115a0 tx=0x7fc834009e30 comp rx=0 tx=0).ready entity=mgr.14686 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:11:20.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:20.014+0000 7fc83ea00700 1 -- 172.21.15.37:0/1726767924 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181579 (secure 0 0 0) 0x7fc8380596e0 con 0x7fc848102e00 2024-03-29T08:11:20.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:20.309+0000 7fc84da00700 1 -- 172.21.15.37:0/1726767924 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fc84804bed0 con 0x7fc848102e00 2024-03-29T08:11:20.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:20.310+0000 7fc83ea00700 1 -- 172.21.15.37:0/1726767924 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+743 (secure 0 0 0) 0x7fc838059270 con 0x7fc848102e00 2024-03-29T08:11:20.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:20.312+0000 7fc84da00700 1 -- 172.21.15.37:0/1726767924 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7fc82c06c8a0 msgr2=0x7fc82c06ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:11:20.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:20.312+0000 7fc84da00700 1 --2- 172.21.15.37:0/1726767924 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7fc82c06c8a0 0x7fc82c06ed50 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fc8481115a0 tx=0x7fc834009e30 comp rx=0 tx=0).stop 2024-03-29T08:11:20.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:20.312+0000 7fc84da00700 1 -- 172.21.15.37:0/1726767924 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc848102e00 msgr2=0x7fc848115d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:11:20.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:20.312+0000 7fc84da00700 1 --2- 172.21.15.37:0/1726767924 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc848102e00 0x7fc848115d00 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fc838002a50 tx=0x7fc8380046b0 comp rx=0 tx=0).stop 2024-03-29T08:11:20.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:20.312+0000 7fc84da00700 1 -- 172.21.15.37:0/1726767924 shutdown_connections 2024-03-29T08:11:20.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:20.312+0000 7fc84da00700 1 --2- 172.21.15.37:0/1726767924 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7fc82c06c8a0 0x7fc82c06ed50 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:11:20.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:20.312+0000 7fc84da00700 1 --2- 172.21.15.37:0/1726767924 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc848102e00 0x7fc848115d00 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:11:20.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:20.312+0000 7fc84da00700 1 --2- 172.21.15.37:0/1726767924 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc8481037a0 0x7fc848116240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:11:20.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:20.312+0000 7fc84da00700 1 -- 172.21.15.37:0/1726767924 >> 172.21.15.37:0/1726767924 conn(0x7fc8480f89b0 msgr2=0x7fc8480f9ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:11:20.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:20.312+0000 7fc84da00700 1 -- 172.21.15.37:0/1726767924 shutdown_connections 2024-03-29T08:11:20.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:20.312+0000 7fc84da00700 1 -- 172.21.15.37:0/1726767924 wait complete. 2024-03-29T08:11:20.326 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)" 2024-03-29T08:11:20.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:20 smithi037 ceph-mon[35162]: pgmap v99: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:11:20.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:11:20 smithi148 ceph-mon[44992]: pgmap v99: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:11:21.024 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 99b492c2-eda1-11ee-b647-cb9ed24678a4 -e sha1=d0e392d9aee635f2b65768c460c7a151ead7a5e2 -- bash -c 'ceph versions | jq -e '"'"'.overall | length == 2'"'"'' 2024-03-29T08:11:21.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:21 smithi037 ceph-mon[35162]: from='client.? 172.21.15.37:0/1726767924' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:11:21.467 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/mon.smithi037/config 2024-03-29T08:11:21.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:11:21 smithi148 ceph-mon[44992]: from='client.? 172.21.15.37:0/1726767924' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:11:22.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:22 smithi037 ceph-mon[35162]: pgmap v100: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:11:22.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:22 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:11:22.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:11:22 smithi148 ceph-mon[44992]: pgmap v100: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:11:22.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:11:22 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:11:23.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:23.315+0000 7fb1d3a00700 1 -- 172.21.15.37:0/1764862383 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb1cc0708a0 msgr2=0x7fb1cc070d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:11:23.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:23.315+0000 7fb1d3a00700 1 --2- 172.21.15.37:0/1764862383 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb1cc0708a0 0x7fb1cc070d10 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fb1b8004970 tx=0x7fb1b8009e30 comp rx=0 tx=0).stop 2024-03-29T08:11:23.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:23.315+0000 7fb1d3a00700 1 -- 172.21.15.37:0/1764862383 shutdown_connections 2024-03-29T08:11:23.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:23.315+0000 7fb1d3a00700 1 --2- 172.21.15.37:0/1764862383 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb1cc0708a0 0x7fb1cc070d10 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:11:23.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:23.315+0000 7fb1d3a00700 1 --2- 172.21.15.37:0/1764862383 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb1cc071f90 0x7fb1cc070360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:11:23.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:23.315+0000 7fb1d3a00700 1 -- 172.21.15.37:0/1764862383 >> 172.21.15.37:0/1764862383 conn(0x7fb1cc0f89b0 msgr2=0x7fb1cc0fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:11:23.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:23.316+0000 7fb1d3a00700 1 -- 172.21.15.37:0/1764862383 shutdown_connections 2024-03-29T08:11:23.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:23.316+0000 7fb1d3a00700 1 -- 172.21.15.37:0/1764862383 wait complete. 2024-03-29T08:11:23.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:23.316+0000 7fb1d3a00700 1 Processor -- start 2024-03-29T08:11:23.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:23.317+0000 7fb1d3a00700 1 -- start start 2024-03-29T08:11:23.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:23.317+0000 7fb1d3a00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb1cc0708a0 0x7fb1cc11a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:11:23.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:23.317+0000 7fb1d3a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb1cc071f90 0x7fb1cc11a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:11:23.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:23.317+0000 7fb1d3a00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1cc11ac60 con 0x7fb1cc071f90 2024-03-29T08:11:23.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:23.317+0000 7fb1d3a00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1cc114050 con 0x7fb1cc0708a0 2024-03-29T08:11:23.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:23.317+0000 7fb1cbe00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb1cc071f90 0x7fb1cc11a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:11:23.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:23.317+0000 7fb1cbe00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb1cc071f90 0x7fb1cc11a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39914/0 (socket says 172.21.15.37:39914) 2024-03-29T08:11:23.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:23.317+0000 7fb1cbe00700 1 -- 172.21.15.37:0/2678710714 learned_addr learned my addr 172.21.15.37:0/2678710714 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:11:23.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:23.318+0000 7fb1d1000700 1 --2- 172.21.15.37:0/2678710714 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb1cc0708a0 0x7fb1cc11a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:11:23.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:23.318+0000 7fb1cbe00700 1 -- 172.21.15.37:0/2678710714 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb1cc0708a0 msgr2=0x7fb1cc11a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:11:23.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:23.318+0000 7fb1cbe00700 1 --2- 172.21.15.37:0/2678710714 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb1cc0708a0 0x7fb1cc11a040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:11:23.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:23.318+0000 7fb1cbe00700 1 -- 172.21.15.37:0/2678710714 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1b8005040 con 0x7fb1cc071f90 2024-03-29T08:11:23.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:23.318+0000 7fb1cbe00700 1 --2- 172.21.15.37:0/2678710714 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb1cc071f90 0x7fb1cc11a580 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fb1b8004f50 tx=0x7fb1b8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:11:23.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:23.318+0000 7fb1c8c00700 1 -- 172.21.15.37:0/2678710714 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb1b800e040 con 0x7fb1cc071f90 2024-03-29T08:11:23.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:23.318+0000 7fb1c8c00700 1 -- 172.21.15.37:0/2678710714 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fb1b8013420 con 0x7fb1cc071f90 2024-03-29T08:11:23.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:23.318+0000 7fb1d3a00700 1 -- 172.21.15.37:0/2678710714 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1cc1142d0 con 0x7fb1cc071f90 2024-03-29T08:11:23.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:23.318+0000 7fb1c8c00700 1 -- 172.21.15.37:0/2678710714 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb1b801c610 con 0x7fb1cc071f90 2024-03-29T08:11:23.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:23.318+0000 7fb1d3a00700 1 -- 172.21.15.37:0/2678710714 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1cc114810 con 0x7fb1cc071f90 2024-03-29T08:11:23.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:23.319+0000 7fb1d3a00700 1 -- 172.21.15.37:0/2678710714 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1cc071aa0 con 0x7fb1cc071f90 2024-03-29T08:11:23.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:23.320+0000 7fb1c8c00700 1 -- 172.21.15.37:0/2678710714 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 34) v1 ==== 90703+0+0 (secure 0 0 0) 0x7fb1b8013590 con 0x7fb1cc071f90 2024-03-29T08:11:23.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:23.322+0000 7fb1c8c00700 1 --2- 172.21.15.37:0/2678710714 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7fb1b406c8b0 0x7fb1b406ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:11:23.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:23.322+0000 7fb1c8c00700 1 -- 172.21.15.37:0/2678710714 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6066+0+0 (secure 0 0 0) 0x7fb1b808ccd0 con 0x7fb1cc071f90 2024-03-29T08:11:23.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:23.322+0000 7fb1d1000700 1 --2- 172.21.15.37:0/2678710714 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7fb1b406c8b0 0x7fb1b406ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:11:23.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:23.323+0000 7fb1d1000700 1 --2- 172.21.15.37:0/2678710714 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7fb1b406c8b0 0x7fb1b406ed60 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fb1c0006fd0 tx=0x7fb1c0008040 comp rx=0 tx=0).ready entity=mgr.14686 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:11:23.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:23.327+0000 7fb1c8c00700 1 -- 172.21.15.37:0/2678710714 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181579 (secure 0 0 0) 0x7fb1b8057190 con 0x7fb1cc071f90 2024-03-29T08:11:23.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:23.624+0000 7fb1d3a00700 1 -- 172.21.15.37:0/2678710714 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fb1cc04bed0 con 0x7fb1cc071f90 2024-03-29T08:11:23.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:23.625+0000 7fb1c8c00700 1 -- 172.21.15.37:0/2678710714 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+743 (secure 0 0 0) 0x7fb1b805a7b0 con 0x7fb1cc071f90 2024-03-29T08:11:23.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:23.627+0000 7fb1d3a00700 1 -- 172.21.15.37:0/2678710714 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7fb1b406c8b0 msgr2=0x7fb1b406ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:11:23.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:23.627+0000 7fb1d3a00700 1 --2- 172.21.15.37:0/2678710714 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7fb1b406c8b0 0x7fb1b406ed60 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fb1c0006fd0 tx=0x7fb1c0008040 comp rx=0 tx=0).stop 2024-03-29T08:11:23.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:23.627+0000 7fb1d3a00700 1 -- 172.21.15.37:0/2678710714 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb1cc071f90 msgr2=0x7fb1cc11a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:11:23.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:23.627+0000 7fb1d3a00700 1 --2- 172.21.15.37:0/2678710714 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb1cc071f90 0x7fb1cc11a580 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fb1b8004f50 tx=0x7fb1b8009e30 comp rx=0 tx=0).stop 2024-03-29T08:11:23.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:23.627+0000 7fb1d3a00700 1 -- 172.21.15.37:0/2678710714 shutdown_connections 2024-03-29T08:11:23.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:23.627+0000 7fb1d3a00700 1 --2- 172.21.15.37:0/2678710714 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7fb1b406c8b0 0x7fb1b406ed60 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:11:23.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:23.627+0000 7fb1d3a00700 1 --2- 172.21.15.37:0/2678710714 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb1cc0708a0 0x7fb1cc11a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:11:23.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:23.627+0000 7fb1d3a00700 1 --2- 172.21.15.37:0/2678710714 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb1cc071f90 0x7fb1cc11a580 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:11:23.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:23.627+0000 7fb1d3a00700 1 -- 172.21.15.37:0/2678710714 >> 172.21.15.37:0/2678710714 conn(0x7fb1cc0f89b0 msgr2=0x7fb1cc0f9b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:11:23.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:23.627+0000 7fb1d3a00700 1 -- 172.21.15.37:0/2678710714 shutdown_connections 2024-03-29T08:11:23.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:23.628+0000 7fb1d3a00700 1 -- 172.21.15.37:0/2678710714 wait complete. 2024-03-29T08:11:23.642 INFO:teuthology.orchestra.run.smithi037.stdout:true 2024-03-29T08:11:24.269 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 99b492c2-eda1-11ee-b647-cb9ed24678a4 -e sha1=d0e392d9aee635f2b65768c460c7a151ead7a5e2 -- bash -c 'ceph orch upgrade check quay.ceph.io/ceph-ci/ceph:$sha1 | jq -e '"'"'.up_to_date | length == 2'"'"'' 2024-03-29T08:11:24.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:24 smithi037 ceph-mon[35162]: pgmap v101: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:11:24.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:24 smithi037 ceph-mon[35162]: from='client.? 172.21.15.37:0/2678710714' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:11:24.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:11:24 smithi148 ceph-mon[44992]: pgmap v101: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:11:24.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:11:24 smithi148 ceph-mon[44992]: from='client.? 172.21.15.37:0/2678710714' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:11:24.705 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/mon.smithi037/config 2024-03-29T08:11:26.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:26.515+0000 7f286b200700 1 -- 172.21.15.37:0/1469489938 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f286c1035e0 msgr2=0x7f286c0fcc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:11:26.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:26.515+0000 7f286b200700 1 --2- 172.21.15.37:0/1469489938 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f286c1035e0 0x7f286c0fcc50 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f2858004940 tx=0x7f2858009e30 comp rx=0 tx=0).stop 2024-03-29T08:11:26.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:26.515+0000 7f286b200700 1 -- 172.21.15.37:0/1469489938 shutdown_connections 2024-03-29T08:11:26.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:26.515+0000 7f286b200700 1 --2- 172.21.15.37:0/1469489938 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f286c1035e0 0x7f286c0fcc50 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:11:26.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:26.515+0000 7f286b200700 1 --2- 172.21.15.37:0/1469489938 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f286c102c40 0x7f286c103010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:11:26.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:26.515+0000 7f286b200700 1 -- 172.21.15.37:0/1469489938 >> 172.21.15.37:0/1469489938 conn(0x7f286c0f8830 msgr2=0x7f286c0fac40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:11:26.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:26.515+0000 7f286b200700 1 -- 172.21.15.37:0/1469489938 shutdown_connections 2024-03-29T08:11:26.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:26.515+0000 7f286b200700 1 -- 172.21.15.37:0/1469489938 wait complete. 2024-03-29T08:11:26.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:26.516+0000 7f286b200700 1 Processor -- start 2024-03-29T08:11:26.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:26.516+0000 7f286b200700 1 -- start start 2024-03-29T08:11:26.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:26.516+0000 7f286b200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f286c102c40 0x7f286c1138e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:11:26.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:26.516+0000 7f286b200700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f286c1035e0 0x7f286c113e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:11:26.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:26.516+0000 7f286b200700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f286c114470 con 0x7f286c102c40 2024-03-29T08:11:26.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:26.516+0000 7f286b200700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f286c10d860 con 0x7f286c1035e0 2024-03-29T08:11:26.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:26.516+0000 7f2869e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f286c102c40 0x7f286c1138e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:11:26.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:26.516+0000 7f2869e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f286c102c40 0x7f286c1138e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39932/0 (socket says 172.21.15.37:39932) 2024-03-29T08:11:26.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:26.516+0000 7f2869e00700 1 -- 172.21.15.37:0/2763430906 learned_addr learned my addr 172.21.15.37:0/2763430906 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:11:26.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:26.516+0000 7f2869400700 1 --2- 172.21.15.37:0/2763430906 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f286c1035e0 0x7f286c113e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:11:26.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:26.517+0000 7f2869e00700 1 -- 172.21.15.37:0/2763430906 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f286c1035e0 msgr2=0x7f286c113e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:11:26.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:26.517+0000 7f2869e00700 1 --2- 172.21.15.37:0/2763430906 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f286c1035e0 0x7f286c113e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:11:26.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:26.517+0000 7f2869e00700 1 -- 172.21.15.37:0/2763430906 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2858005040 con 0x7f286c102c40 2024-03-29T08:11:26.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:26.517+0000 7f2869e00700 1 --2- 172.21.15.37:0/2763430906 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f286c102c40 0x7f286c1138e0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f285c002a50 tx=0x7f285c0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:11:26.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:26.517+0000 7f2862000700 1 -- 172.21.15.37:0/2763430906 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f285c007cc0 con 0x7f286c102c40 2024-03-29T08:11:26.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:26.517+0000 7f2862000700 1 -- 172.21.15.37:0/2763430906 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f285c00f040 con 0x7f286c102c40 2024-03-29T08:11:26.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:26.517+0000 7f286b200700 1 -- 172.21.15.37:0/2763430906 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f286c10db40 con 0x7f286c102c40 2024-03-29T08:11:26.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:26.517+0000 7f2862000700 1 -- 172.21.15.37:0/2763430906 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f285c009540 con 0x7f286c102c40 2024-03-29T08:11:26.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:26.517+0000 7f286b200700 1 -- 172.21.15.37:0/2763430906 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f286c10e100 con 0x7f286c102c40 2024-03-29T08:11:26.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:26.518+0000 7f286b200700 1 -- 172.21.15.37:0/2763430906 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f286c0fe390 con 0x7f286c102c40 2024-03-29T08:11:26.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:26.520+0000 7f2862000700 1 -- 172.21.15.37:0/2763430906 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 34) v1 ==== 90703+0+0 (secure 0 0 0) 0x7f285c0096a0 con 0x7f286c102c40 2024-03-29T08:11:26.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:26.520+0000 7f2862000700 1 --2- 172.21.15.37:0/2763430906 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7f285006c8a0 0x7f285006ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:11:26.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:26.521+0000 7f2862000700 1 -- 172.21.15.37:0/2763430906 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f285c08b950 con 0x7f286c102c40 2024-03-29T08:11:26.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:26.521+0000 7f2869400700 1 --2- 172.21.15.37:0/2763430906 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7f285006c8a0 0x7f285006ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:11:26.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:26.521+0000 7f2869400700 1 --2- 172.21.15.37:0/2763430906 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7f285006c8a0 0x7f285006ed50 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f286c10ea20 tx=0x7f2858009e30 comp rx=0 tx=0).ready entity=mgr.14686 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:11:26.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:26.525+0000 7f2862000700 1 -- 172.21.15.37:0/2763430906 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181579 (secure 0 0 0) 0x7f285c059820 con 0x7f286c102c40 2024-03-29T08:11:26.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:11:26 smithi148 ceph-mon[44992]: pgmap v102: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:11:26.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:26 smithi037 ceph-mon[35162]: pgmap v102: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:11:26.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:26.756+0000 7f286b200700 1 -- 172.21.15.37:0/2763430906 --> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] -- mgr_command(tid 0: {"prefix": "orch upgrade check", "image": "quay.ceph.io/ceph-ci/ceph:d0e392d9aee635f2b65768c460c7a151ead7a5e2", "target": ["mon-mgr", ""]}) v1 -- 0x7f286c04bed0 con 0x7f285006c8a0 2024-03-29T08:11:28.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:28 smithi037 ceph-mon[35162]: pgmap v103: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:11:28.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:28 smithi037 ceph-mon[35162]: from='client.14838 -' entity='client.admin' cmd=[{"prefix": "orch upgrade check", "image": "quay.ceph.io/ceph-ci/ceph:d0e392d9aee635f2b65768c460c7a151ead7a5e2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:11:28.428 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:28 smithi037 ceph-mon[35162]: fsmap cephfs:2 {0=cephfs.smithi037.mmwgxe=up:active,1=cephfs.smithi037.iifsmh=up:active} 2 up:standby 2024-03-29T08:11:28.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:28.476+0000 7f2862000700 1 -- 172.21.15.37:0/2763430906 <== mgr.14686 v2:172.21.15.37:6800/4188329516 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+5094 (secure 0 0 0) 0x7f286c04bed0 con 0x7f285006c8a0 2024-03-29T08:11:28.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:28.479+0000 7f286b200700 1 -- 172.21.15.37:0/2763430906 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7f285006c8a0 msgr2=0x7f285006ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:11:28.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:28.479+0000 7f286b200700 1 --2- 172.21.15.37:0/2763430906 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7f285006c8a0 0x7f285006ed50 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f286c10ea20 tx=0x7f2858009e30 comp rx=0 tx=0).stop 2024-03-29T08:11:28.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:28.479+0000 7f286b200700 1 -- 172.21.15.37:0/2763430906 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f286c102c40 msgr2=0x7f286c1138e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:11:28.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:28.479+0000 7f286b200700 1 --2- 172.21.15.37:0/2763430906 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f286c102c40 0x7f286c1138e0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f285c002a50 tx=0x7f285c0046b0 comp rx=0 tx=0).stop 2024-03-29T08:11:28.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:28.479+0000 7f286b200700 1 -- 172.21.15.37:0/2763430906 shutdown_connections 2024-03-29T08:11:28.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:28.479+0000 7f286b200700 1 --2- 172.21.15.37:0/2763430906 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7f285006c8a0 0x7f285006ed50 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:11:28.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:28.479+0000 7f286b200700 1 --2- 172.21.15.37:0/2763430906 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f286c102c40 0x7f286c1138e0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:11:28.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:28.479+0000 7f286b200700 1 --2- 172.21.15.37:0/2763430906 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f286c1035e0 0x7f286c113e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:11:28.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:28.479+0000 7f286b200700 1 -- 172.21.15.37:0/2763430906 >> 172.21.15.37:0/2763430906 conn(0x7f286c0f8830 msgr2=0x7f286c0f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:11:28.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:28.479+0000 7f286b200700 1 -- 172.21.15.37:0/2763430906 shutdown_connections 2024-03-29T08:11:28.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:28.479+0000 7f286b200700 1 -- 172.21.15.37:0/2763430906 wait complete. 2024-03-29T08:11:28.494 INFO:teuthology.orchestra.run.smithi037.stdout:true 2024-03-29T08:11:28.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:11:28 smithi148 ceph-mon[44992]: pgmap v103: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:11:28.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:11:28 smithi148 ceph-mon[44992]: from='client.14838 -' entity='client.admin' cmd=[{"prefix": "orch upgrade check", "image": "quay.ceph.io/ceph-ci/ceph:d0e392d9aee635f2b65768c460c7a151ead7a5e2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:11:28.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:11:28 smithi148 ceph-mon[44992]: fsmap cephfs:2 {0=cephfs.smithi037.mmwgxe=up:active,1=cephfs.smithi037.iifsmh=up:active} 2 up:standby 2024-03-29T08:11:29.157 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 99b492c2-eda1-11ee-b647-cb9ed24678a4 -e sha1=d0e392d9aee635f2b65768c460c7a151ead7a5e2 -- bash -c 'ceph orch ps' 2024-03-29T08:11:29.597 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/mon.smithi037/config 2024-03-29T08:11:30.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:11:30 smithi148 ceph-mon[44992]: pgmap v104: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:11:30.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:30 smithi037 ceph-mon[35162]: pgmap v104: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:11:31.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:31.433+0000 7f4f33600700 1 -- 172.21.15.37:0/2875453025 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4f2c102e00 msgr2=0x7f4f2c1031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:11:31.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:31.433+0000 7f4f33600700 1 --2- 172.21.15.37:0/2875453025 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4f2c102e00 0x7f4f2c1031d0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f4f20004940 tx=0x7f4f20009e30 comp rx=0 tx=0).stop 2024-03-29T08:11:31.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:31.433+0000 7f4f33600700 1 -- 172.21.15.37:0/2875453025 shutdown_connections 2024-03-29T08:11:31.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:31.433+0000 7f4f33600700 1 --2- 172.21.15.37:0/2875453025 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4f2c1037a0 0x7f4f2c0fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:11:31.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:31.433+0000 7f4f33600700 1 --2- 172.21.15.37:0/2875453025 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4f2c102e00 0x7f4f2c1031d0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:11:31.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:31.433+0000 7f4f33600700 1 -- 172.21.15.37:0/2875453025 >> 172.21.15.37:0/2875453025 conn(0x7f4f2c0f89b0 msgr2=0x7f4f2c0fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:11:31.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:31.433+0000 7f4f33600700 1 -- 172.21.15.37:0/2875453025 shutdown_connections 2024-03-29T08:11:31.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:31.433+0000 7f4f33600700 1 -- 172.21.15.37:0/2875453025 wait complete. 2024-03-29T08:11:31.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:31.434+0000 7f4f33600700 1 Processor -- start 2024-03-29T08:11:31.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:31.434+0000 7f4f33600700 1 -- start start 2024-03-29T08:11:31.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:31.435+0000 7f4f33600700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4f2c102e00 0x7f4f2c115c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:11:31.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:31.435+0000 7f4f33600700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4f2c1037a0 0x7f4f2c116180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:11:31.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:31.435+0000 7f4f33600700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f2c1167d0 con 0x7f4f2c102e00 2024-03-29T08:11:31.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:31.435+0000 7f4f33600700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f2c10fba0 con 0x7f4f2c1037a0 2024-03-29T08:11:31.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:31.435+0000 7f4f30c00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4f2c102e00 0x7f4f2c115c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:11:31.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:31.435+0000 7f4f30c00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4f2c102e00 0x7f4f2c115c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58578/0 (socket says 172.21.15.37:58578) 2024-03-29T08:11:31.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:31.435+0000 7f4f30c00700 1 -- 172.21.15.37:0/1934017963 learned_addr learned my addr 172.21.15.37:0/1934017963 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:11:31.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:31.435+0000 7f4f2be00700 1 --2- 172.21.15.37:0/1934017963 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4f2c1037a0 0x7f4f2c116180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:11:31.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:31.435+0000 7f4f30c00700 1 -- 172.21.15.37:0/1934017963 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4f2c1037a0 msgr2=0x7f4f2c116180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:11:31.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:31.435+0000 7f4f30c00700 1 --2- 172.21.15.37:0/1934017963 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4f2c1037a0 0x7f4f2c116180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:11:31.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:31.435+0000 7f4f30c00700 1 -- 172.21.15.37:0/1934017963 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f20005040 con 0x7f4f2c102e00 2024-03-29T08:11:31.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:31.436+0000 7f4f30c00700 1 --2- 172.21.15.37:0/1934017963 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4f2c102e00 0x7f4f2c115c40 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f4f20009390 tx=0x7f4f20009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:11:31.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:31.436+0000 7f4f28c00700 1 -- 172.21.15.37:0/1934017963 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4f2000e040 con 0x7f4f2c102e00 2024-03-29T08:11:31.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:31.436+0000 7f4f28c00700 1 -- 172.21.15.37:0/1934017963 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f4f20013420 con 0x7f4f2c102e00 2024-03-29T08:11:31.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:31.436+0000 7f4f33600700 1 -- 172.21.15.37:0/1934017963 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f2c10fe20 con 0x7f4f2c102e00 2024-03-29T08:11:31.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:31.436+0000 7f4f28c00700 1 -- 172.21.15.37:0/1934017963 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4f2001c630 con 0x7f4f2c102e00 2024-03-29T08:11:31.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:31.436+0000 7f4f33600700 1 -- 172.21.15.37:0/1934017963 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f2c1103e0 con 0x7f4f2c102e00 2024-03-29T08:11:31.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:31.437+0000 7f4f33600700 1 -- 172.21.15.37:0/1934017963 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f2c0fe550 con 0x7f4f2c102e00 2024-03-29T08:11:31.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:31.438+0000 7f4f28c00700 1 -- 172.21.15.37:0/1934017963 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 34) v1 ==== 90703+0+0 (secure 0 0 0) 0x7f4f200139e0 con 0x7f4f2c102e00 2024-03-29T08:11:31.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:31.439+0000 7f4f28c00700 1 --2- 172.21.15.37:0/1934017963 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7f4f1406c8b0 0x7f4f1406ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:11:31.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:31.439+0000 7f4f28c00700 1 -- 172.21.15.37:0/1934017963 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f4f2008cc00 con 0x7f4f2c102e00 2024-03-29T08:11:31.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:31.439+0000 7f4f2be00700 1 --2- 172.21.15.37:0/1934017963 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7f4f1406c8b0 0x7f4f1406ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:11:31.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:31.440+0000 7f4f2be00700 1 --2- 172.21.15.37:0/1934017963 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7f4f1406c8b0 0x7f4f1406ed60 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f4f18009fd0 tx=0x7f4f18009f60 comp rx=0 tx=0).ready entity=mgr.14686 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:11:31.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:31.444+0000 7f4f28c00700 1 -- 172.21.15.37:0/1934017963 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181579 (secure 0 0 0) 0x7f4f20092020 con 0x7f4f2c102e00 2024-03-29T08:11:31.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:31.668+0000 7f4f33600700 1 -- 172.21.15.37:0/1934017963 --> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f4f2c04bf40 con 0x7f4f1406c8b0 2024-03-29T08:11:31.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:31.678+0000 7f4f28c00700 1 -- 172.21.15.37:0/1934017963 <== mgr.14686 v2:172.21.15.37:6800/4188329516 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7f4f2c04bf40 con 0x7f4f1406c8b0 2024-03-29T08:11:31.679 INFO:teuthology.orchestra.run.smithi037.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-29T08:11:31.679 INFO:teuthology.orchestra.run.smithi037.stdout:alertmanager.smithi037 smithi037 *:9093,9094 running (82s) 55s ago 14m 13.7M - 0.25.0 c8568f914cd2 2f9f9746ed0f 2024-03-29T08:11:31.679 INFO:teuthology.orchestra.run.smithi037.stdout:crash.smithi037 smithi037 running (14m) 55s ago 14m 6656k - 17.2.7-789-g4213e17c cf22eb8683be e0343935b25e 2024-03-29T08:11:31.679 INFO:teuthology.orchestra.run.smithi037.stdout:crash.smithi148 smithi148 running (13m) 2m ago 13m 6656k - 17.2.7-789-g4213e17c cf22eb8683be ff5164c552c1 2024-03-29T08:11:31.679 INFO:teuthology.orchestra.run.smithi037.stdout:grafana.smithi037 smithi037 *:3000 running (62s) 55s ago 13m 45.0M - 9.4.7 954c08fa6188 6b976fe49952 2024-03-29T08:11:31.679 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi037.iifsmh smithi037 running (7m) 55s ago 7m 110M - 17.2.7-789-g4213e17c cf22eb8683be 074e1acf331f 2024-03-29T08:11:31.679 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi037.mmwgxe smithi037 running (7m) 55s ago 7m 115M - 17.2.7-789-g4213e17c cf22eb8683be 9a5b1e72de5c 2024-03-29T08:11:31.679 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi148.qukpjh smithi148 running (7m) 2m ago 7m 12.9M - 17.2.7-789-g4213e17c cf22eb8683be 8e469782a273 2024-03-29T08:11:31.679 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi148.sqteih smithi148 running (7m) 2m ago 7m 12.8M - 17.2.7-789-g4213e17c cf22eb8683be 8e68f858bb22 2024-03-29T08:11:31.679 INFO:teuthology.orchestra.run.smithi037.stdout:mgr.smithi037.wjikii smithi037 *:8443,9283,8765 running (3m) 55s ago 15m 498M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff c5a1417b9417 2024-03-29T08:11:31.680 INFO:teuthology.orchestra.run.smithi037.stdout:mgr.smithi148.onhgvm smithi148 *:8443,9283,8765 running (2m) 2m ago 13m 446M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff fcfdbce062bd 2024-03-29T08:11:31.680 INFO:teuthology.orchestra.run.smithi037.stdout:mon.smithi037 smithi037 running (16m) 55s ago 16m 54.2M 2048M 17.2.7-789-g4213e17c cf22eb8683be bb2101505730 2024-03-29T08:11:31.680 INFO:teuthology.orchestra.run.smithi037.stdout:mon.smithi148 smithi148 running (13m) 2m ago 13m 42.0M 2048M 17.2.7-789-g4213e17c cf22eb8683be 18839ed2952a 2024-03-29T08:11:31.680 INFO:teuthology.orchestra.run.smithi037.stdout:node-exporter.smithi037 smithi037 *:9100 running (2m) 55s ago 14m 7197k - 1.5.0 0da6a335fe13 26b8ae9cb878 2024-03-29T08:11:31.680 INFO:teuthology.orchestra.run.smithi037.stdout:node-exporter.smithi148 smithi148 *:9100 running (2m) 2m ago 13m 2693k - 1.5.0 0da6a335fe13 98414e4e64ed 2024-03-29T08:11:31.680 INFO:teuthology.orchestra.run.smithi037.stdout:osd.0 smithi037 running (12m) 55s ago 12m 217M 1619M 17.2.7-789-g4213e17c cf22eb8683be 559fccd0ebe1 2024-03-29T08:11:31.680 INFO:teuthology.orchestra.run.smithi037.stdout:osd.1 smithi037 running (11m) 55s ago 11m 228M 1619M 17.2.7-789-g4213e17c cf22eb8683be 5582cff11b7a 2024-03-29T08:11:31.680 INFO:teuthology.orchestra.run.smithi037.stdout:osd.2 smithi037 running (11m) 55s ago 11m 196M 1619M 17.2.7-789-g4213e17c cf22eb8683be f809bfab13e5 2024-03-29T08:11:31.680 INFO:teuthology.orchestra.run.smithi037.stdout:osd.3 smithi148 running (10m) 2m ago 10m 259M 2642M 17.2.7-789-g4213e17c cf22eb8683be a0a8224bb4b0 2024-03-29T08:11:31.680 INFO:teuthology.orchestra.run.smithi037.stdout:osd.4 smithi148 running (10m) 2m ago 10m 230M 2642M 17.2.7-789-g4213e17c cf22eb8683be a00b69047ed7 2024-03-29T08:11:31.680 INFO:teuthology.orchestra.run.smithi037.stdout:osd.5 smithi148 running (9m) 2m ago 9m 231M 2642M 17.2.7-789-g4213e17c cf22eb8683be 044a3f7db7f8 2024-03-29T08:11:31.680 INFO:teuthology.orchestra.run.smithi037.stdout:prometheus.smithi037 smithi037 *:9095 running (102s) 55s ago 13m 44.3M - 2.43.0 a07b618ecd1d 30c0223ec589 2024-03-29T08:11:31.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:31.681+0000 7f4f33600700 1 -- 172.21.15.37:0/1934017963 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7f4f1406c8b0 msgr2=0x7f4f1406ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:11:31.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:31.681+0000 7f4f33600700 1 --2- 172.21.15.37:0/1934017963 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7f4f1406c8b0 0x7f4f1406ed60 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f4f18009fd0 tx=0x7f4f18009f60 comp rx=0 tx=0).stop 2024-03-29T08:11:31.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:31.681+0000 7f4f33600700 1 -- 172.21.15.37:0/1934017963 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4f2c102e00 msgr2=0x7f4f2c115c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:11:31.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:31.681+0000 7f4f33600700 1 --2- 172.21.15.37:0/1934017963 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4f2c102e00 0x7f4f2c115c40 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f4f20009390 tx=0x7f4f20009e30 comp rx=0 tx=0).stop 2024-03-29T08:11:31.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:31.681+0000 7f4f33600700 1 -- 172.21.15.37:0/1934017963 shutdown_connections 2024-03-29T08:11:31.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:31.681+0000 7f4f33600700 1 --2- 172.21.15.37:0/1934017963 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7f4f1406c8b0 0x7f4f1406ed60 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:11:31.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:31.681+0000 7f4f33600700 1 --2- 172.21.15.37:0/1934017963 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4f2c102e00 0x7f4f2c115c40 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:11:31.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:31.681+0000 7f4f33600700 1 --2- 172.21.15.37:0/1934017963 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4f2c1037a0 0x7f4f2c116180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:11:31.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:31.681+0000 7f4f33600700 1 -- 172.21.15.37:0/1934017963 >> 172.21.15.37:0/1934017963 conn(0x7f4f2c0f89b0 msgr2=0x7f4f2c0f99f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:11:31.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:31.681+0000 7f4f33600700 1 -- 172.21.15.37:0/1934017963 shutdown_connections 2024-03-29T08:11:31.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:31.681+0000 7f4f33600700 1 -- 172.21.15.37:0/1934017963 wait complete. 2024-03-29T08:11:32.404 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:32 smithi037 ceph-mon[35162]: pgmap v105: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:11:32.429 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-03-29T08:11:32.430 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi037.front.sepia.ceph.com 2024-03-29T08:11:32.430 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 99b492c2-eda1-11ee-b647-cb9ed24678a4 -e sha1=d0e392d9aee635f2b65768c460c7a151ead7a5e2 -- bash -c 'ceph config set mgr mgr/orchestrator/fail_fs true' 2024-03-29T08:11:32.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:11:32 smithi148 ceph-mon[44992]: pgmap v105: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:11:32.869 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/mon.smithi037/config 2024-03-29T08:11:33.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:33 smithi037 ceph-mon[35162]: from='client.14842 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:11:33.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:11:33 smithi148 ceph-mon[44992]: from='client.14842 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:11:34.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:34 smithi037 ceph-mon[35162]: pgmap v106: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:11:34.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:11:34 smithi148 ceph-mon[44992]: pgmap v106: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:11:34.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:34.875+0000 7fbe62800700 1 -- 172.21.15.37:0/1589697178 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbe5c070c10 msgr2=0x7fbe5c109190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:11:34.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:34.875+0000 7fbe62800700 1 --2- 172.21.15.37:0/1589697178 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbe5c070c10 0x7fbe5c109190 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fbe48004970 tx=0x7fbe48009e30 comp rx=0 tx=0).stop 2024-03-29T08:11:34.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:34.875+0000 7fbe62800700 1 -- 172.21.15.37:0/1589697178 shutdown_connections 2024-03-29T08:11:34.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:34.875+0000 7fbe62800700 1 --2- 172.21.15.37:0/1589697178 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbe5c070c10 0x7fbe5c109190 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:11:34.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:34.875+0000 7fbe62800700 1 --2- 172.21.15.37:0/1589697178 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbe5c070300 0x7fbe5c0706d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:11:34.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:34.875+0000 7fbe62800700 1 -- 172.21.15.37:0/1589697178 >> 172.21.15.37:0/1589697178 conn(0x7fbe5c0f8970 msgr2=0x7fbe5c0fad80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:11:34.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:34.875+0000 7fbe62800700 1 -- 172.21.15.37:0/1589697178 shutdown_connections 2024-03-29T08:11:34.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:34.875+0000 7fbe62800700 1 -- 172.21.15.37:0/1589697178 wait complete. 2024-03-29T08:11:34.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:34.876+0000 7fbe62800700 1 Processor -- start 2024-03-29T08:11:34.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:34.876+0000 7fbe62800700 1 -- start start 2024-03-29T08:11:34.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:34.877+0000 7fbe62800700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbe5c070300 0x7fbe5c115cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:11:34.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:34.877+0000 7fbe62800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbe5c070c10 0x7fbe5c116200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:11:34.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:34.877+0000 7fbe62800700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe5c116850 con 0x7fbe5c070c10 2024-03-29T08:11:34.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:34.877+0000 7fbe62800700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe5c10fc20 con 0x7fbe5c070300 2024-03-29T08:11:34.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:34.877+0000 7fbe5aa00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbe5c070c10 0x7fbe5c116200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:11:34.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:34.877+0000 7fbe5aa00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbe5c070c10 0x7fbe5c116200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58596/0 (socket says 172.21.15.37:58596) 2024-03-29T08:11:34.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:34.877+0000 7fbe5aa00700 1 -- 172.21.15.37:0/4261771124 learned_addr learned my addr 172.21.15.37:0/4261771124 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:11:34.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:34.877+0000 7fbe5b400700 1 --2- 172.21.15.37:0/4261771124 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbe5c070300 0x7fbe5c115cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:11:34.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:34.877+0000 7fbe5aa00700 1 -- 172.21.15.37:0/4261771124 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbe5c070300 msgr2=0x7fbe5c115cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:11:34.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:34.877+0000 7fbe5aa00700 1 --2- 172.21.15.37:0/4261771124 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbe5c070300 0x7fbe5c115cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:11:34.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:34.877+0000 7fbe5aa00700 1 -- 172.21.15.37:0/4261771124 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe48005040 con 0x7fbe5c070c10 2024-03-29T08:11:34.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:34.877+0000 7fbe5b400700 1 --2- 172.21.15.37:0/4261771124 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbe5c070300 0x7fbe5c115cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:11:34.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:34.878+0000 7fbe5aa00700 1 --2- 172.21.15.37:0/4261771124 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbe5c070c10 0x7fbe5c116200 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fbe48009360 tx=0x7fbe48009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:11:34.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:34.878+0000 7fbe53400700 1 -- 172.21.15.37:0/4261771124 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbe4800e040 con 0x7fbe5c070c10 2024-03-29T08:11:34.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:34.878+0000 7fbe53400700 1 -- 172.21.15.37:0/4261771124 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fbe48004290 con 0x7fbe5c070c10 2024-03-29T08:11:34.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:34.878+0000 7fbe62800700 1 -- 172.21.15.37:0/4261771124 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe5c10fea0 con 0x7fbe5c070c10 2024-03-29T08:11:34.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:34.878+0000 7fbe53400700 1 -- 172.21.15.37:0/4261771124 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbe4801c3f0 con 0x7fbe5c070c10 2024-03-29T08:11:34.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:34.878+0000 7fbe62800700 1 -- 172.21.15.37:0/4261771124 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe5c1103e0 con 0x7fbe5c070c10 2024-03-29T08:11:34.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:34.879+0000 7fbe62800700 1 -- 172.21.15.37:0/4261771124 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe5c071a40 con 0x7fbe5c070c10 2024-03-29T08:11:34.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:34.880+0000 7fbe53400700 1 -- 172.21.15.37:0/4261771124 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 34) v1 ==== 90703+0+0 (secure 0 0 0) 0x7fbe48013420 con 0x7fbe5c070c10 2024-03-29T08:11:34.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:34.881+0000 7fbe53400700 1 --2- 172.21.15.37:0/4261771124 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7fbe4006c8b0 0x7fbe4006ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:11:34.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:34.881+0000 7fbe53400700 1 -- 172.21.15.37:0/4261771124 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6066+0+0 (secure 0 0 0) 0x7fbe4808ce90 con 0x7fbe5c070c10 2024-03-29T08:11:34.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:34.881+0000 7fbe5b400700 1 --2- 172.21.15.37:0/4261771124 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7fbe4006c8b0 0x7fbe4006ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:11:34.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:34.882+0000 7fbe5b400700 1 --2- 172.21.15.37:0/4261771124 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7fbe4006c8b0 0x7fbe4006ed60 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fbe4c009fd0 tx=0x7fbe4c00b040 comp rx=0 tx=0).ready entity=mgr.14686 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:11:34.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:34.886+0000 7fbe53400700 1 -- 172.21.15.37:0/4261771124 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181579 (secure 0 0 0) 0x7fbe4805ae60 con 0x7fbe5c070c10 2024-03-29T08:11:35.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:35.098+0000 7fbe62800700 1 -- 172.21.15.37:0/4261771124 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command([{prefix=config set, name=mgr/orchestrator/fail_fs}] v 0) v1 -- 0x7fbe5c05f5f0 con 0x7fbe5c070c10 2024-03-29T08:11:35.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:35.103+0000 7fbe53400700 1 -- 172.21.15.37:0/4261771124 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/orchestrator/fail_fs}]=0 v47)=0 v47) v1 ==== 125+0+0 (secure 0 0 0) 0x7fbe4805a9f0 con 0x7fbe5c070c10 2024-03-29T08:11:35.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:35.105+0000 7fbe62800700 1 -- 172.21.15.37:0/4261771124 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7fbe4006c8b0 msgr2=0x7fbe4006ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:11:35.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:35.105+0000 7fbe62800700 1 --2- 172.21.15.37:0/4261771124 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7fbe4006c8b0 0x7fbe4006ed60 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fbe4c009fd0 tx=0x7fbe4c00b040 comp rx=0 tx=0).stop 2024-03-29T08:11:35.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:35.105+0000 7fbe62800700 1 -- 172.21.15.37:0/4261771124 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbe5c070c10 msgr2=0x7fbe5c116200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:11:35.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:35.105+0000 7fbe62800700 1 --2- 172.21.15.37:0/4261771124 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbe5c070c10 0x7fbe5c116200 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fbe48009360 tx=0x7fbe48009e30 comp rx=0 tx=0).stop 2024-03-29T08:11:35.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:35.106+0000 7fbe62800700 1 -- 172.21.15.37:0/4261771124 shutdown_connections 2024-03-29T08:11:35.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:35.106+0000 7fbe62800700 1 --2- 172.21.15.37:0/4261771124 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7fbe4006c8b0 0x7fbe4006ed60 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:11:35.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:35.106+0000 7fbe62800700 1 --2- 172.21.15.37:0/4261771124 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbe5c070300 0x7fbe5c115cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:11:35.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:35.106+0000 7fbe62800700 1 --2- 172.21.15.37:0/4261771124 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbe5c070c10 0x7fbe5c116200 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:11:35.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:35.106+0000 7fbe62800700 1 -- 172.21.15.37:0/4261771124 >> 172.21.15.37:0/4261771124 conn(0x7fbe5c0f8970 msgr2=0x7fbe5c0f9de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:11:35.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:35.106+0000 7fbe62800700 1 -- 172.21.15.37:0/4261771124 shutdown_connections 2024-03-29T08:11:35.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:35.106+0000 7fbe62800700 1 -- 172.21.15.37:0/4261771124 wait complete. 2024-03-29T08:11:35.894 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-03-29T08:11:35.895 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi037.front.sepia.ceph.com 2024-03-29T08:11:35.895 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 99b492c2-eda1-11ee-b647-cb9ed24678a4 -e sha1=d0e392d9aee635f2b65768c460c7a151ead7a5e2 -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force' 2024-03-29T08:11:36.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:36 smithi037 ceph-mon[35162]: pgmap v107: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:11:36.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:36 smithi037 ceph-mon[35162]: from='client.? 172.21.15.37:0/4261771124' entity='client.admin' 2024-03-29T08:11:36.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:36 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T08:11:36.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:36 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T08:11:36.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:36 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T08:11:36.178 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:36 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' 2024-03-29T08:11:36.335 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/mon.smithi037/config 2024-03-29T08:11:36.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:11:36 smithi148 ceph-mon[44992]: pgmap v107: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:11:36.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:11:36 smithi148 ceph-mon[44992]: from='client.? 172.21.15.37:0/4261771124' entity='client.admin' 2024-03-29T08:11:36.557 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:11:36 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T08:11:36.557 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:11:36 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T08:11:36.557 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:11:36 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T08:11:36.557 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:11:36 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' 2024-03-29T08:11:37.365 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:37 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:11:37.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:11:37 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:11:38.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:38.425+0000 7f90e6200700 1 -- 172.21.15.37:0/857980515 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f90e0102e00 msgr2=0x7f90e01031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:11:38.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:38.425+0000 7f90e6200700 1 --2- 172.21.15.37:0/857980515 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f90e0102e00 0x7f90e01031d0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f90d0004940 tx=0x7f90d0009e30 comp rx=0 tx=0).stop 2024-03-29T08:11:38.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:38.425+0000 7f90e6200700 1 -- 172.21.15.37:0/857980515 shutdown_connections 2024-03-29T08:11:38.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:38.425+0000 7f90e6200700 1 --2- 172.21.15.37:0/857980515 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f90e01037a0 0x7f90e00fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:11:38.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:38.425+0000 7f90e6200700 1 --2- 172.21.15.37:0/857980515 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f90e0102e00 0x7f90e01031d0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:11:38.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:38.425+0000 7f90e6200700 1 -- 172.21.15.37:0/857980515 >> 172.21.15.37:0/857980515 conn(0x7f90e00f89b0 msgr2=0x7f90e00fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:11:38.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:38.425+0000 7f90e6200700 1 -- 172.21.15.37:0/857980515 shutdown_connections 2024-03-29T08:11:38.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:38.425+0000 7f90e6200700 1 -- 172.21.15.37:0/857980515 wait complete. 2024-03-29T08:11:38.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:38.426+0000 7f90e6200700 1 Processor -- start 2024-03-29T08:11:38.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:38.426+0000 7f90e6200700 1 -- start start 2024-03-29T08:11:38.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:38.427+0000 7f90e6200700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f90e0102e00 0x7f90e0115cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:11:38.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:38.427+0000 7f90e6200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f90e01037a0 0x7f90e0116230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:11:38.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:38.427+0000 7f90e6200700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90e0116880 con 0x7f90e01037a0 2024-03-29T08:11:38.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:38.427+0000 7f90e6200700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90e010fc50 con 0x7f90e0102e00 2024-03-29T08:11:38.428 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:38 smithi037 ceph-mon[35162]: pgmap v108: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:11:38.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:38.427+0000 7f90dea00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f90e01037a0 0x7f90e0116230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:11:38.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:38.427+0000 7f90dea00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f90e01037a0 0x7f90e0116230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58600/0 (socket says 172.21.15.37:58600) 2024-03-29T08:11:38.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:38.427+0000 7f90dea00700 1 -- 172.21.15.37:0/1922343684 learned_addr learned my addr 172.21.15.37:0/1922343684 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:11:38.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:38.427+0000 7f90df400700 1 --2- 172.21.15.37:0/1922343684 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f90e0102e00 0x7f90e0115cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:11:38.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:38.427+0000 7f90dea00700 1 -- 172.21.15.37:0/1922343684 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f90e0102e00 msgr2=0x7f90e0115cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:11:38.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:38.427+0000 7f90dea00700 1 --2- 172.21.15.37:0/1922343684 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f90e0102e00 0x7f90e0115cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:11:38.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:38.427+0000 7f90dea00700 1 -- 172.21.15.37:0/1922343684 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90d0005040 con 0x7f90e01037a0 2024-03-29T08:11:38.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:38.428+0000 7f90dea00700 1 --2- 172.21.15.37:0/1922343684 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f90e01037a0 0x7f90e0116230 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f90cc002a50 tx=0x7f90cc004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:11:38.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:38.428+0000 7f90d7400700 1 -- 172.21.15.37:0/1922343684 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f90cc007cc0 con 0x7f90e01037a0 2024-03-29T08:11:38.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:38.428+0000 7f90d7400700 1 -- 172.21.15.37:0/1922343684 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f90cc00f040 con 0x7f90e01037a0 2024-03-29T08:11:38.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:38.428+0000 7f90d7400700 1 -- 172.21.15.37:0/1922343684 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f90cc00d920 con 0x7f90e01037a0 2024-03-29T08:11:38.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:38.428+0000 7f90e6200700 1 -- 172.21.15.37:0/1922343684 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90e010ff30 con 0x7f90e01037a0 2024-03-29T08:11:38.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:38.428+0000 7f90e6200700 1 -- 172.21.15.37:0/1922343684 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90e01104f0 con 0x7f90e01037a0 2024-03-29T08:11:38.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:38.429+0000 7f90e6200700 1 -- 172.21.15.37:0/1922343684 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90e00fe550 con 0x7f90e01037a0 2024-03-29T08:11:38.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:38.430+0000 7f90d7400700 1 -- 172.21.15.37:0/1922343684 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 34) v1 ==== 90703+0+0 (secure 0 0 0) 0x7f90cc009680 con 0x7f90e01037a0 2024-03-29T08:11:38.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:38.431+0000 7f90d7400700 1 --2- 172.21.15.37:0/1922343684 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7f90c406c8a0 0x7f90c406ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:11:38.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:38.431+0000 7f90d7400700 1 -- 172.21.15.37:0/1922343684 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f90cc08b780 con 0x7f90e01037a0 2024-03-29T08:11:38.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:38.431+0000 7f90df400700 1 --2- 172.21.15.37:0/1922343684 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7f90c406c8a0 0x7f90c406ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:11:38.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:38.432+0000 7f90df400700 1 --2- 172.21.15.37:0/1922343684 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7f90c406c8a0 0x7f90c406ed50 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f90d0004f90 tx=0x7f90d0009e30 comp rx=0 tx=0).ready entity=mgr.14686 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:11:38.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:38.436+0000 7f90d7400700 1 -- 172.21.15.37:0/1922343684 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181579 (secure 0 0 0) 0x7f90cc0596d0 con 0x7f90e01037a0 2024-03-29T08:11:38.557 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:11:38 smithi148 ceph-mon[44992]: pgmap v108: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:11:38.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:38.649+0000 7f90e6200700 1 -- 172.21.15.37:0/1922343684 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim}] v 0) v1 -- 0x7f90e005f5f0 con 0x7f90e01037a0 2024-03-29T08:11:38.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:38.649+0000 7f90d7400700 1 -- 172.21.15.37:0/1922343684 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim}]=0 v47)=0 v47) v1 ==== 155+0+0 (secure 0 0 0) 0x7f90cc059260 con 0x7f90e01037a0 2024-03-29T08:11:38.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:38.652+0000 7f90e6200700 1 -- 172.21.15.37:0/1922343684 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7f90c406c8a0 msgr2=0x7f90c406ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:11:38.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:38.652+0000 7f90e6200700 1 --2- 172.21.15.37:0/1922343684 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7f90c406c8a0 0x7f90c406ed50 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f90d0004f90 tx=0x7f90d0009e30 comp rx=0 tx=0).stop 2024-03-29T08:11:38.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:38.652+0000 7f90e6200700 1 -- 172.21.15.37:0/1922343684 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f90e01037a0 msgr2=0x7f90e0116230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:11:38.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:38.652+0000 7f90e6200700 1 --2- 172.21.15.37:0/1922343684 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f90e01037a0 0x7f90e0116230 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f90cc002a50 tx=0x7f90cc004600 comp rx=0 tx=0).stop 2024-03-29T08:11:38.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:38.652+0000 7f90e6200700 1 -- 172.21.15.37:0/1922343684 shutdown_connections 2024-03-29T08:11:38.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:38.652+0000 7f90e6200700 1 --2- 172.21.15.37:0/1922343684 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7f90c406c8a0 0x7f90c406ed50 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:11:38.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:38.652+0000 7f90e6200700 1 --2- 172.21.15.37:0/1922343684 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f90e0102e00 0x7f90e0115cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:11:38.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:38.652+0000 7f90e6200700 1 --2- 172.21.15.37:0/1922343684 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f90e01037a0 0x7f90e0116230 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:11:38.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:38.652+0000 7f90e6200700 1 -- 172.21.15.37:0/1922343684 >> 172.21.15.37:0/1922343684 conn(0x7f90e00f89b0 msgr2=0x7f90e00f9980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:11:38.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:38.652+0000 7f90e6200700 1 -- 172.21.15.37:0/1922343684 shutdown_connections 2024-03-29T08:11:38.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:38.652+0000 7f90e6200700 1 -- 172.21.15.37:0/1922343684 wait complete. 2024-03-29T08:11:39.432 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 99b492c2-eda1-11ee-b647-cb9ed24678a4 -e sha1=d0e392d9aee635f2b65768c460c7a151ead7a5e2 -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force' 2024-03-29T08:11:39.868 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/mon.smithi037/config 2024-03-29T08:11:40.388 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:40 smithi037 ceph-mon[35162]: pgmap v109: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:11:40.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:11:40 smithi148 ceph-mon[44992]: pgmap v109: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:11:41.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:41.686+0000 7ff6d8a00700 1 -- 172.21.15.37:0/2567769676 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff6d4071f90 msgr2=0x7ff6d4070360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:11:41.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:41.686+0000 7ff6d8a00700 1 --2- 172.21.15.37:0/2567769676 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff6d4071f90 0x7ff6d4070360 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7ff6c4004970 tx=0x7ff6c4009e30 comp rx=0 tx=0).stop 2024-03-29T08:11:41.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:41.687+0000 7ff6d8a00700 1 -- 172.21.15.37:0/2567769676 shutdown_connections 2024-03-29T08:11:41.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:41.687+0000 7ff6d8a00700 1 --2- 172.21.15.37:0/2567769676 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff6d40708a0 0x7ff6d4070d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:11:41.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:41.687+0000 7ff6d8a00700 1 --2- 172.21.15.37:0/2567769676 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff6d4071f90 0x7ff6d4070360 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:11:41.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:41.687+0000 7ff6d8a00700 1 -- 172.21.15.37:0/2567769676 >> 172.21.15.37:0/2567769676 conn(0x7ff6d40f89b0 msgr2=0x7ff6d40fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:11:41.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:41.687+0000 7ff6d8a00700 1 -- 172.21.15.37:0/2567769676 shutdown_connections 2024-03-29T08:11:41.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:41.687+0000 7ff6d8a00700 1 -- 172.21.15.37:0/2567769676 wait complete. 2024-03-29T08:11:41.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:41.687+0000 7ff6d8a00700 1 Processor -- start 2024-03-29T08:11:41.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:41.688+0000 7ff6d8a00700 1 -- start start 2024-03-29T08:11:41.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:41.688+0000 7ff6d8a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff6d40708a0 0x7ff6d411a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:11:41.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:41.688+0000 7ff6d8a00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff6d4071f90 0x7ff6d411a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:11:41.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:41.688+0000 7ff6d8a00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6d411ad40 con 0x7ff6d40708a0 2024-03-29T08:11:41.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:41.688+0000 7ff6d8a00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6d4114080 con 0x7ff6d4071f90 2024-03-29T08:11:41.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:41.688+0000 7ff6d1e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff6d40708a0 0x7ff6d411a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:11:41.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:41.688+0000 7ff6d1e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff6d40708a0 0x7ff6d411a120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49648/0 (socket says 172.21.15.37:49648) 2024-03-29T08:11:41.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:41.688+0000 7ff6d1e00700 1 -- 172.21.15.37:0/4284142102 learned_addr learned my addr 172.21.15.37:0/4284142102 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:11:41.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:41.688+0000 7ff6d1400700 1 --2- 172.21.15.37:0/4284142102 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff6d4071f90 0x7ff6d411a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:11:41.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:41.689+0000 7ff6d1e00700 1 -- 172.21.15.37:0/4284142102 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff6d4071f90 msgr2=0x7ff6d411a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:11:41.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:41.689+0000 7ff6d1e00700 1 --2- 172.21.15.37:0/4284142102 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff6d4071f90 0x7ff6d411a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:11:41.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:41.689+0000 7ff6d1e00700 1 -- 172.21.15.37:0/4284142102 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6c4005040 con 0x7ff6d40708a0 2024-03-29T08:11:41.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:41.689+0000 7ff6d1e00700 1 --2- 172.21.15.37:0/4284142102 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff6d40708a0 0x7ff6d411a120 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7ff6c4009380 tx=0x7ff6c4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:11:41.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:41.689+0000 7ff6ca000700 1 -- 172.21.15.37:0/4284142102 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff6c400e040 con 0x7ff6d40708a0 2024-03-29T08:11:41.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:41.689+0000 7ff6d8a00700 1 -- 172.21.15.37:0/4284142102 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6d41142a0 con 0x7ff6d40708a0 2024-03-29T08:11:41.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:41.689+0000 7ff6ca000700 1 -- 172.21.15.37:0/4284142102 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ff6c4013420 con 0x7ff6d40708a0 2024-03-29T08:11:41.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:41.689+0000 7ff6ca000700 1 -- 172.21.15.37:0/4284142102 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff6c401c630 con 0x7ff6d40708a0 2024-03-29T08:11:41.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:41.690+0000 7ff6d8a00700 1 -- 172.21.15.37:0/4284142102 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6d41147e0 con 0x7ff6d40708a0 2024-03-29T08:11:41.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:41.690+0000 7ff6d8a00700 1 -- 172.21.15.37:0/4284142102 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6d4071aa0 con 0x7ff6d40708a0 2024-03-29T08:11:41.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:41.692+0000 7ff6ca000700 1 -- 172.21.15.37:0/4284142102 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 34) v1 ==== 90703+0+0 (secure 0 0 0) 0x7ff6c4013990 con 0x7ff6d40708a0 2024-03-29T08:11:41.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:41.693+0000 7ff6ca000700 1 --2- 172.21.15.37:0/4284142102 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7ff6b806c8a0 0x7ff6b806ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:11:41.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:41.693+0000 7ff6ca000700 1 -- 172.21.15.37:0/4284142102 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6066+0+0 (secure 0 0 0) 0x7ff6c408cc90 con 0x7ff6d40708a0 2024-03-29T08:11:41.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:41.693+0000 7ff6d1400700 1 --2- 172.21.15.37:0/4284142102 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7ff6b806c8a0 0x7ff6b806ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:11:41.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:41.694+0000 7ff6d1400700 1 --2- 172.21.15.37:0/4284142102 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7ff6b806c8a0 0x7ff6b806ed50 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ff6d41159c0 tx=0x7ff6c00091a0 comp rx=0 tx=0).ready entity=mgr.14686 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:11:41.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:41.698+0000 7ff6ca000700 1 -- 172.21.15.37:0/4284142102 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181579 (secure 0 0 0) 0x7ff6c405ac60 con 0x7ff6d40708a0 2024-03-29T08:11:41.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:41.909+0000 7ff6d8a00700 1 -- 172.21.15.37:0/4284142102 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim_allowed}] v 0) v1 -- 0x7ff6d405f5f0 con 0x7ff6d40708a0 2024-03-29T08:11:41.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:41.910+0000 7ff6ca000700 1 -- 172.21.15.37:0/4284142102 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim_allowed}]=0 v47)=0 v47) v1 ==== 163+0+0 (secure 0 0 0) 0x7ff6c405a7f0 con 0x7ff6d40708a0 2024-03-29T08:11:41.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:41.912+0000 7ff6d8a00700 1 -- 172.21.15.37:0/4284142102 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7ff6b806c8a0 msgr2=0x7ff6b806ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:11:41.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:41.912+0000 7ff6d8a00700 1 --2- 172.21.15.37:0/4284142102 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7ff6b806c8a0 0x7ff6b806ed50 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ff6d41159c0 tx=0x7ff6c00091a0 comp rx=0 tx=0).stop 2024-03-29T08:11:41.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:41.912+0000 7ff6d8a00700 1 -- 172.21.15.37:0/4284142102 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff6d40708a0 msgr2=0x7ff6d411a120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:11:41.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:41.912+0000 7ff6d8a00700 1 --2- 172.21.15.37:0/4284142102 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff6d40708a0 0x7ff6d411a120 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7ff6c4009380 tx=0x7ff6c4009e30 comp rx=0 tx=0).stop 2024-03-29T08:11:41.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:41.912+0000 7ff6d8a00700 1 -- 172.21.15.37:0/4284142102 shutdown_connections 2024-03-29T08:11:41.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:41.912+0000 7ff6d8a00700 1 --2- 172.21.15.37:0/4284142102 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7ff6b806c8a0 0x7ff6b806ed50 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:11:41.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:41.912+0000 7ff6d8a00700 1 --2- 172.21.15.37:0/4284142102 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff6d40708a0 0x7ff6d411a120 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:11:41.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:41.912+0000 7ff6d8a00700 1 --2- 172.21.15.37:0/4284142102 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff6d4071f90 0x7ff6d411a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:11:41.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:41.912+0000 7ff6d8a00700 1 -- 172.21.15.37:0/4284142102 >> 172.21.15.37:0/4284142102 conn(0x7ff6d40f89b0 msgr2=0x7ff6d40f9b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:11:41.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:41.912+0000 7ff6d8a00700 1 -- 172.21.15.37:0/4284142102 shutdown_connections 2024-03-29T08:11:41.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:41.912+0000 7ff6d8a00700 1 -- 172.21.15.37:0/4284142102 wait complete. 2024-03-29T08:11:42.217 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:42 smithi037 ceph-mon[35162]: pgmap v110: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:11:42.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:11:42 smithi148 ceph-mon[44992]: pgmap v110: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:11:42.639 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 99b492c2-eda1-11ee-b647-cb9ed24678a4 -e sha1=d0e392d9aee635f2b65768c460c7a151ead7a5e2 -- bash -c 'ceph config set global log_to_journald false --force' 2024-03-29T08:11:43.081 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/mon.smithi037/config 2024-03-29T08:11:44.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:44 smithi037 ceph-mon[35162]: pgmap v111: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:11:44.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:11:44 smithi148 ceph-mon[44992]: pgmap v111: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:11:44.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:44.924+0000 7f2b20e00700 1 -- 172.21.15.37:0/853633354 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2b1c070c70 msgr2=0x7f2b1c109110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:11:44.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:44.924+0000 7f2b20e00700 1 --2- 172.21.15.37:0/853633354 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2b1c070c70 0x7f2b1c109110 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f2b08004970 tx=0x7f2b08009e30 comp rx=0 tx=0).stop 2024-03-29T08:11:44.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:44.924+0000 7f2b20e00700 1 -- 172.21.15.37:0/853633354 shutdown_connections 2024-03-29T08:11:44.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:44.924+0000 7f2b20e00700 1 --2- 172.21.15.37:0/853633354 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2b1c070c70 0x7f2b1c109110 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:11:44.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:44.924+0000 7f2b20e00700 1 --2- 172.21.15.37:0/853633354 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2b1c070360 0x7f2b1c070730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:11:44.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:44.924+0000 7f2b20e00700 1 -- 172.21.15.37:0/853633354 >> 172.21.15.37:0/853633354 conn(0x7f2b1c0f89b0 msgr2=0x7f2b1c0fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:11:44.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:44.924+0000 7f2b20e00700 1 -- 172.21.15.37:0/853633354 shutdown_connections 2024-03-29T08:11:44.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:44.924+0000 7f2b20e00700 1 -- 172.21.15.37:0/853633354 wait complete. 2024-03-29T08:11:44.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:44.925+0000 7f2b20e00700 1 Processor -- start 2024-03-29T08:11:44.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:44.925+0000 7f2b20e00700 1 -- start start 2024-03-29T08:11:44.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:44.925+0000 7f2b20e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2b1c070360 0x7f2b1c115d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:11:44.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:44.925+0000 7f2b20e00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2b1c070c70 0x7f2b1c116240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:11:44.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:44.925+0000 7f2b20e00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b1c116890 con 0x7f2b1c070360 2024-03-29T08:11:44.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:44.925+0000 7f2b20e00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b1c10fc60 con 0x7f2b1c070c70 2024-03-29T08:11:44.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:44.926+0000 7f2b19e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2b1c070360 0x7f2b1c115d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:11:44.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:44.926+0000 7f2b19e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2b1c070360 0x7f2b1c115d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49666/0 (socket says 172.21.15.37:49666) 2024-03-29T08:11:44.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:44.926+0000 7f2b19e00700 1 -- 172.21.15.37:0/4191812758 learned_addr learned my addr 172.21.15.37:0/4191812758 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:11:44.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:44.926+0000 7f2b19400700 1 --2- 172.21.15.37:0/4191812758 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2b1c070c70 0x7f2b1c116240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:11:44.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:44.926+0000 7f2b19e00700 1 -- 172.21.15.37:0/4191812758 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2b1c070c70 msgr2=0x7f2b1c116240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:11:44.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:44.926+0000 7f2b19e00700 1 --2- 172.21.15.37:0/4191812758 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2b1c070c70 0x7f2b1c116240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:11:44.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:44.926+0000 7f2b19e00700 1 -- 172.21.15.37:0/4191812758 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b08005040 con 0x7f2b1c070360 2024-03-29T08:11:44.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:44.926+0000 7f2b19e00700 1 --2- 172.21.15.37:0/4191812758 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2b1c070360 0x7f2b1c115d00 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f2b0c002a50 tx=0x7f2b0c0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:11:44.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:44.926+0000 7f2b12000700 1 -- 172.21.15.37:0/4191812758 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2b0c007cc0 con 0x7f2b1c070360 2024-03-29T08:11:44.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:44.926+0000 7f2b12000700 1 -- 172.21.15.37:0/4191812758 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f2b0c00f040 con 0x7f2b1c070360 2024-03-29T08:11:44.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:44.926+0000 7f2b12000700 1 -- 172.21.15.37:0/4191812758 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2b0c00d9c0 con 0x7f2b1c070360 2024-03-29T08:11:44.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:44.927+0000 7f2b20e00700 1 -- 172.21.15.37:0/4191812758 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b1c10ff40 con 0x7f2b1c070360 2024-03-29T08:11:44.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:44.927+0000 7f2b20e00700 1 -- 172.21.15.37:0/4191812758 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b1c110450 con 0x7f2b1c070360 2024-03-29T08:11:44.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:44.928+0000 7f2b20e00700 1 -- 172.21.15.37:0/4191812758 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b1c071aa0 con 0x7f2b1c070360 2024-03-29T08:11:44.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:44.929+0000 7f2b12000700 1 -- 172.21.15.37:0/4191812758 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 34) v1 ==== 90703+0+0 (secure 0 0 0) 0x7f2b0c00db20 con 0x7f2b1c070360 2024-03-29T08:11:44.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:44.930+0000 7f2b12000700 1 --2- 172.21.15.37:0/4191812758 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7f2b0006c8b0 0x7f2b0006ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:11:44.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:44.930+0000 7f2b12000700 1 -- 172.21.15.37:0/4191812758 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f2b0c08b790 con 0x7f2b1c070360 2024-03-29T08:11:44.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:44.930+0000 7f2b19400700 1 --2- 172.21.15.37:0/4191812758 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7f2b0006c8b0 0x7f2b0006ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:11:44.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:44.931+0000 7f2b19400700 1 --2- 172.21.15.37:0/4191812758 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7f2b0006c8b0 0x7f2b0006ed60 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f2b08009fd0 tx=0x7f2b08009e30 comp rx=0 tx=0).ready entity=mgr.14686 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:11:44.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:44.934+0000 7f2b12000700 1 -- 172.21.15.37:0/4191812758 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181579 (secure 0 0 0) 0x7f2b0c0596e0 con 0x7f2b1c070360 2024-03-29T08:11:45.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:45.147+0000 7f2b20e00700 1 -- 172.21.15.37:0/4191812758 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command([{prefix=config set, name=log_to_journald}] v 0) v1 -- 0x7f2b1c05f5f0 con 0x7f2b1c070360 2024-03-29T08:11:45.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:45.148+0000 7f2b12000700 1 -- 172.21.15.37:0/4191812758 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{prefix=config set, name=log_to_journald}]=0 v47)=0 v47) v1 ==== 135+0+0 (secure 0 0 0) 0x7f2b1c05f5f0 con 0x7f2b1c070360 2024-03-29T08:11:45.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:45.150+0000 7f2b20e00700 1 -- 172.21.15.37:0/4191812758 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7f2b0006c8b0 msgr2=0x7f2b0006ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:11:45.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:45.150+0000 7f2b20e00700 1 --2- 172.21.15.37:0/4191812758 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7f2b0006c8b0 0x7f2b0006ed60 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f2b08009fd0 tx=0x7f2b08009e30 comp rx=0 tx=0).stop 2024-03-29T08:11:45.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:45.150+0000 7f2b20e00700 1 -- 172.21.15.37:0/4191812758 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2b1c070360 msgr2=0x7f2b1c115d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:11:45.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:45.150+0000 7f2b20e00700 1 --2- 172.21.15.37:0/4191812758 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2b1c070360 0x7f2b1c115d00 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f2b0c002a50 tx=0x7f2b0c0046b0 comp rx=0 tx=0).stop 2024-03-29T08:11:45.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:45.150+0000 7f2b20e00700 1 -- 172.21.15.37:0/4191812758 shutdown_connections 2024-03-29T08:11:45.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:45.150+0000 7f2b20e00700 1 --2- 172.21.15.37:0/4191812758 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7f2b0006c8b0 0x7f2b0006ed60 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:11:45.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:45.150+0000 7f2b20e00700 1 --2- 172.21.15.37:0/4191812758 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2b1c070360 0x7f2b1c115d00 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:11:45.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:45.150+0000 7f2b20e00700 1 --2- 172.21.15.37:0/4191812758 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2b1c070c70 0x7f2b1c116240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:11:45.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:45.150+0000 7f2b20e00700 1 -- 172.21.15.37:0/4191812758 >> 172.21.15.37:0/4191812758 conn(0x7f2b1c0f89b0 msgr2=0x7f2b1c105a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:11:45.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:45.150+0000 7f2b20e00700 1 -- 172.21.15.37:0/4191812758 shutdown_connections 2024-03-29T08:11:45.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:45.151+0000 7f2b20e00700 1 -- 172.21.15.37:0/4191812758 wait complete. 2024-03-29T08:11:45.907 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 99b492c2-eda1-11ee-b647-cb9ed24678a4 -e sha1=d0e392d9aee635f2b65768c460c7a151ead7a5e2 -- bash -c 'ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1' 2024-03-29T08:11:46.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:46 smithi037 ceph-mon[35162]: pgmap v112: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:11:46.348 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/mon.smithi037/config 2024-03-29T08:11:46.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:11:46 smithi148 ceph-mon[44992]: pgmap v112: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:11:48.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:48.422+0000 7f602f400700 1 -- 172.21.15.37:0/579295032 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6028071f90 msgr2=0x7f6028070360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:11:48.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:48.422+0000 7f602f400700 1 --2- 172.21.15.37:0/579295032 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6028071f90 0x7f6028070360 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f601c004970 tx=0x7f601c009e30 comp rx=0 tx=0).stop 2024-03-29T08:11:48.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:48.422+0000 7f602f400700 1 -- 172.21.15.37:0/579295032 shutdown_connections 2024-03-29T08:11:48.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:48.422+0000 7f602f400700 1 --2- 172.21.15.37:0/579295032 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f60280708a0 0x7f6028070d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:11:48.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:48.422+0000 7f602f400700 1 --2- 172.21.15.37:0/579295032 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6028071f90 0x7f6028070360 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:11:48.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:48.422+0000 7f602f400700 1 -- 172.21.15.37:0/579295032 >> 172.21.15.37:0/579295032 conn(0x7f60280f89b0 msgr2=0x7f60280fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:11:48.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:48.422+0000 7f602f400700 1 -- 172.21.15.37:0/579295032 shutdown_connections 2024-03-29T08:11:48.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:48.422+0000 7f602f400700 1 -- 172.21.15.37:0/579295032 wait complete. 2024-03-29T08:11:48.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:48.423+0000 7f602f400700 1 Processor -- start 2024-03-29T08:11:48.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:48.423+0000 7f602f400700 1 -- start start 2024-03-29T08:11:48.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:48.424+0000 7f602f400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f60280708a0 0x7f602811a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:11:48.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:48.424+0000 7f602f400700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6028071f90 0x7f602811a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:11:48.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:48.424+0000 7f602f400700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f602811ac50 con 0x7f60280708a0 2024-03-29T08:11:48.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:48.424+0000 7f602f400700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6028114020 con 0x7f6028071f90 2024-03-29T08:11:48.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:48.424+0000 7f602ca00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f60280708a0 0x7f602811a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:11:48.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:48.424+0000 7f602ca00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f60280708a0 0x7f602811a0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49694/0 (socket says 172.21.15.37:49694) 2024-03-29T08:11:48.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:48.424+0000 7f602ca00700 1 -- 172.21.15.37:0/2219480437 learned_addr learned my addr 172.21.15.37:0/2219480437 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:11:48.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:48.424+0000 7f6027e00700 1 --2- 172.21.15.37:0/2219480437 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6028071f90 0x7f602811a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:11:48.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:48.424+0000 7f602ca00700 1 -- 172.21.15.37:0/2219480437 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6028071f90 msgr2=0x7f602811a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:11:48.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:48.424+0000 7f602ca00700 1 --2- 172.21.15.37:0/2219480437 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6028071f90 0x7f602811a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:11:48.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:48.424+0000 7f602ca00700 1 -- 172.21.15.37:0/2219480437 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f601c005040 con 0x7f60280708a0 2024-03-29T08:11:48.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:48.424+0000 7f6027e00700 1 --2- 172.21.15.37:0/2219480437 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6028071f90 0x7f602811a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:11:48.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:48.425+0000 7f602ca00700 1 --2- 172.21.15.37:0/2219480437 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f60280708a0 0x7f602811a0c0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f601c0043d0 tx=0x7f601c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:11:48.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:48.425+0000 7f6024c00700 1 -- 172.21.15.37:0/2219480437 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f601c00e040 con 0x7f60280708a0 2024-03-29T08:11:48.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:48.425+0000 7f6024c00700 1 -- 172.21.15.37:0/2219480437 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f601c00bbe0 con 0x7f60280708a0 2024-03-29T08:11:48.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:48.425+0000 7f6024c00700 1 -- 172.21.15.37:0/2219480437 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f601c01c3f0 con 0x7f60280708a0 2024-03-29T08:11:48.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:48.425+0000 7f602f400700 1 -- 172.21.15.37:0/2219480437 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60281142a0 con 0x7f60280708a0 2024-03-29T08:11:48.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:48.425+0000 7f602f400700 1 -- 172.21.15.37:0/2219480437 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60281147e0 con 0x7f60280708a0 2024-03-29T08:11:48.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:48.426+0000 7f602f400700 1 -- 172.21.15.37:0/2219480437 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6028071aa0 con 0x7f60280708a0 2024-03-29T08:11:48.429 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:48 smithi037 ceph-mon[35162]: pgmap v113: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:11:48.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:48.427+0000 7f6024c00700 1 -- 172.21.15.37:0/2219480437 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 34) v1 ==== 90703+0+0 (secure 0 0 0) 0x7f601c00bd50 con 0x7f60280708a0 2024-03-29T08:11:48.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:48.428+0000 7f6024c00700 1 --2- 172.21.15.37:0/2219480437 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7f601006c8a0 0x7f601006ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:11:48.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:48.428+0000 7f6024c00700 1 -- 172.21.15.37:0/2219480437 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f601c08ce10 con 0x7f60280708a0 2024-03-29T08:11:48.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:48.429+0000 7f6027e00700 1 --2- 172.21.15.37:0/2219480437 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7f601006c8a0 0x7f601006ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:11:48.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:48.429+0000 7f6027e00700 1 --2- 172.21.15.37:0/2219480437 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7f601006c8a0 0x7f601006ed50 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f6014009fd0 tx=0x7f6014009ee0 comp rx=0 tx=0).ready entity=mgr.14686 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:11:48.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:48.433+0000 7f6024c00700 1 -- 172.21.15.37:0/2219480437 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181579 (secure 0 0 0) 0x7f601c05ade0 con 0x7f60280708a0 2024-03-29T08:11:48.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:11:48 smithi148 ceph-mon[44992]: pgmap v113: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:11:48.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:48.668+0000 7f602f400700 1 -- 172.21.15.37:0/2219480437 --> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] -- mgr_command(tid 0: {"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:d0e392d9aee635f2b65768c460c7a151ead7a5e2", "target": ["mon-mgr", ""]}) v1 -- 0x7f602804bed0 con 0x7f601006c8a0 2024-03-29T08:11:48.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:48.674+0000 7f6024c00700 1 -- 172.21.15.37:0/2219480437 <== mgr.14686 v2:172.21.15.37:6800/4188329516 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+89 (secure 0 0 0) 0x7f602804bed0 con 0x7f601006c8a0 2024-03-29T08:11:48.674 INFO:teuthology.orchestra.run.smithi037.stdout:Initiating upgrade to quay.ceph.io/ceph-ci/ceph:d0e392d9aee635f2b65768c460c7a151ead7a5e2 2024-03-29T08:11:48.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:48.676+0000 7f602f400700 1 -- 172.21.15.37:0/2219480437 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7f601006c8a0 msgr2=0x7f601006ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:11:48.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:48.676+0000 7f602f400700 1 --2- 172.21.15.37:0/2219480437 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7f601006c8a0 0x7f601006ed50 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f6014009fd0 tx=0x7f6014009ee0 comp rx=0 tx=0).stop 2024-03-29T08:11:48.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:48.676+0000 7f602f400700 1 -- 172.21.15.37:0/2219480437 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f60280708a0 msgr2=0x7f602811a0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:11:48.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:48.676+0000 7f602f400700 1 --2- 172.21.15.37:0/2219480437 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f60280708a0 0x7f602811a0c0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f601c0043d0 tx=0x7f601c009e30 comp rx=0 tx=0).stop 2024-03-29T08:11:48.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:48.677+0000 7f602f400700 1 -- 172.21.15.37:0/2219480437 shutdown_connections 2024-03-29T08:11:48.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:48.677+0000 7f602f400700 1 --2- 172.21.15.37:0/2219480437 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7f601006c8a0 0x7f601006ed50 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:11:48.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:48.677+0000 7f602f400700 1 --2- 172.21.15.37:0/2219480437 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f60280708a0 0x7f602811a0c0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:11:48.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:48.677+0000 7f602f400700 1 --2- 172.21.15.37:0/2219480437 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6028071f90 0x7f602811a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:11:48.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:48.677+0000 7f602f400700 1 -- 172.21.15.37:0/2219480437 >> 172.21.15.37:0/2219480437 conn(0x7f60280f89b0 msgr2=0x7f60280f9b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:11:48.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:48.677+0000 7f602f400700 1 -- 172.21.15.37:0/2219480437 shutdown_connections 2024-03-29T08:11:48.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:48.677+0000 7f602f400700 1 -- 172.21.15.37:0/2219480437 wait complete. 2024-03-29T08:11:49.441 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-03-29T08:11:49.442 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi037.front.sepia.ceph.com 2024-03-29T08:11:49.442 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 99b492c2-eda1-11ee-b647-cb9ed24678a4 -e sha1=d0e392d9aee635f2b65768c460c7a151ead7a5e2 -- bash -c 'while ceph orch upgrade status | jq '"'"'.in_progress'"'"' | grep true && ! ceph orch upgrade status | jq '"'"'.message'"'"' | grep Error ; do ceph orch ps ; ceph versions ; ceph fs dump; ceph orch upgrade status ; ceph health detail ; sleep 30 ; done' 2024-03-29T08:11:49.907 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/mon.smithi037/config 2024-03-29T08:11:49.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:49 smithi037 ceph-mon[35162]: pgmap v114: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:11:49.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:49 smithi037 ceph-mon[35162]: from='client.14862 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:d0e392d9aee635f2b65768c460c7a151ead7a5e2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:11:49.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:49 smithi037 ceph-mon[35162]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:d0e392d9aee635f2b65768c460c7a151ead7a5e2 2024-03-29T08:11:49.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:49 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' 2024-03-29T08:11:49.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:49 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T08:11:49.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:49 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T08:11:49.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:49 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T08:11:49.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:49 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' 2024-03-29T08:11:50.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:11:49 smithi148 ceph-mon[44992]: pgmap v114: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:11:50.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:11:49 smithi148 ceph-mon[44992]: from='client.14862 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:d0e392d9aee635f2b65768c460c7a151ead7a5e2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:11:50.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:11:49 smithi148 ceph-mon[44992]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:d0e392d9aee635f2b65768c460c7a151ead7a5e2 2024-03-29T08:11:50.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:11:49 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' 2024-03-29T08:11:50.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:11:49 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T08:11:50.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:11:49 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T08:11:50.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:11:49 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T08:11:50.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:11:49 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' 2024-03-29T08:11:50.877 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:50 smithi037 ceph-mon[35162]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:d0e392d9aee635f2b65768c460c7a151ead7a5e2 2024-03-29T08:11:51.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:11:50 smithi148 ceph-mon[44992]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:d0e392d9aee635f2b65768c460c7a151ead7a5e2 2024-03-29T08:11:51.917 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:51 smithi037 ceph-mon[35162]: pgmap v115: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:11:51.917 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:51 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:11:52.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:11:51 smithi148 ceph-mon[44992]: pgmap v115: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:11:52.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:11:51 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:11:52.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:52.885+0000 7f0a1c400700 1 -- 172.21.15.37:0/354073529 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0a14070c10 msgr2=0x7f0a14109180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:11:52.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:52.885+0000 7f0a1c400700 1 --2- 172.21.15.37:0/354073529 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0a14070c10 0x7f0a14109180 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f0a00004970 tx=0x7f0a00009e30 comp rx=0 tx=0).stop 2024-03-29T08:11:52.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:52.885+0000 7f0a1c400700 1 -- 172.21.15.37:0/354073529 shutdown_connections 2024-03-29T08:11:52.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:52.885+0000 7f0a1c400700 1 --2- 172.21.15.37:0/354073529 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0a14070c10 0x7f0a14109180 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:11:52.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:52.886+0000 7f0a1c400700 1 --2- 172.21.15.37:0/354073529 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0a14070300 0x7f0a140706d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:11:52.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:52.886+0000 7f0a1c400700 1 -- 172.21.15.37:0/354073529 >> 172.21.15.37:0/354073529 conn(0x7f0a140f89a0 msgr2=0x7f0a140fadb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:11:52.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:52.886+0000 7f0a1c400700 1 -- 172.21.15.37:0/354073529 shutdown_connections 2024-03-29T08:11:52.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:52.886+0000 7f0a1c400700 1 -- 172.21.15.37:0/354073529 wait complete. 2024-03-29T08:11:52.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:52.887+0000 7f0a1c400700 1 Processor -- start 2024-03-29T08:11:52.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:52.887+0000 7f0a1c400700 1 -- start start 2024-03-29T08:11:52.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:52.887+0000 7f0a1c400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0a14070300 0x7f0a14115c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:11:52.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:52.887+0000 7f0a1c400700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0a14070c10 0x7f0a141161b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:11:52.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:52.887+0000 7f0a1c400700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a14116800 con 0x7f0a14070300 2024-03-29T08:11:52.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:52.887+0000 7f0a1c400700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a1410fbd0 con 0x7f0a14070c10 2024-03-29T08:11:52.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:52.887+0000 7f0a19a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0a14070300 0x7f0a14115c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:11:52.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:52.888+0000 7f0a19a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0a14070300 0x7f0a14115c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54404/0 (socket says 172.21.15.37:54404) 2024-03-29T08:11:52.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:52.888+0000 7f0a19a00700 1 -- 172.21.15.37:0/3065245716 learned_addr learned my addr 172.21.15.37:0/3065245716 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:11:52.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:52.888+0000 7f0a19000700 1 --2- 172.21.15.37:0/3065245716 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0a14070c10 0x7f0a141161b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:11:52.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:52.888+0000 7f0a19a00700 1 -- 172.21.15.37:0/3065245716 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0a14070c10 msgr2=0x7f0a141161b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:11:52.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:52.888+0000 7f0a19a00700 1 --2- 172.21.15.37:0/3065245716 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0a14070c10 0x7f0a141161b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:11:52.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:52.888+0000 7f0a19a00700 1 -- 172.21.15.37:0/3065245716 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a00005040 con 0x7f0a14070300 2024-03-29T08:11:52.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:52.888+0000 7f0a19a00700 1 --2- 172.21.15.37:0/3065245716 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0a14070300 0x7f0a14115c70 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f0a08002a80 tx=0x7f0a080046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:11:52.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:52.889+0000 7f0a11600700 1 -- 172.21.15.37:0/3065245716 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0a08007cc0 con 0x7f0a14070300 2024-03-29T08:11:52.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:52.889+0000 7f0a11600700 1 -- 172.21.15.37:0/3065245716 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f0a0800f040 con 0x7f0a14070300 2024-03-29T08:11:52.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:52.889+0000 7f0a11600700 1 -- 172.21.15.37:0/3065245716 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0a08009640 con 0x7f0a14070300 2024-03-29T08:11:52.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:52.889+0000 7f0a1c400700 1 -- 172.21.15.37:0/3065245716 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a1410feb0 con 0x7f0a14070300 2024-03-29T08:11:52.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:52.889+0000 7f0a1c400700 1 -- 172.21.15.37:0/3065245716 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a141103f0 con 0x7f0a14070300 2024-03-29T08:11:52.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:52.890+0000 7f0a1c400700 1 -- 172.21.15.37:0/3065245716 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a14071a40 con 0x7f0a14070300 2024-03-29T08:11:52.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:52.891+0000 7f0a11600700 1 -- 172.21.15.37:0/3065245716 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 34) v1 ==== 90703+0+0 (secure 0 0 0) 0x7f0a080097a0 con 0x7f0a14070300 2024-03-29T08:11:52.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:52.892+0000 7f0a11600700 1 --2- 172.21.15.37:0/3065245716 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7f09fc06c8b0 0x7f09fc06ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:11:52.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:52.892+0000 7f0a11600700 1 -- 172.21.15.37:0/3065245716 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f0a0808b1b0 con 0x7f0a14070300 2024-03-29T08:11:52.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:52.892+0000 7f0a19000700 1 --2- 172.21.15.37:0/3065245716 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7f09fc06c8b0 0x7f09fc06ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:11:52.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:52.893+0000 7f0a19000700 1 --2- 172.21.15.37:0/3065245716 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7f09fc06c8b0 0x7f09fc06ed60 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f0a14111510 tx=0x7f0a00009e30 comp rx=0 tx=0).ready entity=mgr.14686 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:11:52.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:52.896+0000 7f0a11600700 1 -- 172.21.15.37:0/3065245716 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181579 (secure 0 0 0) 0x7f0a080591a0 con 0x7f0a14070300 2024-03-29T08:11:53.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:53.133+0000 7f0a1c400700 1 -- 172.21.15.37:0/3065245716 --> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0a1404bf40 con 0x7f09fc06c8b0 2024-03-29T08:11:53.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:53.135+0000 7f0a11600700 1 -- 172.21.15.37:0/3065245716 <== mgr.14686 v2:172.21.15.37:6800/4188329516 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+368 (secure 0 0 0) 0x7f0a1404bf40 con 0x7f09fc06c8b0 2024-03-29T08:11:53.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:53.137+0000 7f0a1c400700 1 -- 172.21.15.37:0/3065245716 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7f09fc06c8b0 msgr2=0x7f09fc06ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:11:53.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:53.137+0000 7f0a1c400700 1 --2- 172.21.15.37:0/3065245716 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7f09fc06c8b0 0x7f09fc06ed60 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f0a14111510 tx=0x7f0a00009e30 comp rx=0 tx=0).stop 2024-03-29T08:11:53.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:53.137+0000 7f0a1c400700 1 -- 172.21.15.37:0/3065245716 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0a14070300 msgr2=0x7f0a14115c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:11:53.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:53.137+0000 7f0a1c400700 1 --2- 172.21.15.37:0/3065245716 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0a14070300 0x7f0a14115c70 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f0a08002a80 tx=0x7f0a080046b0 comp rx=0 tx=0).stop 2024-03-29T08:11:53.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:53.137+0000 7f0a1c400700 1 -- 172.21.15.37:0/3065245716 shutdown_connections 2024-03-29T08:11:53.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:53.137+0000 7f0a1c400700 1 --2- 172.21.15.37:0/3065245716 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7f09fc06c8b0 0x7f09fc06ed60 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:11:53.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:53.137+0000 7f0a1c400700 1 --2- 172.21.15.37:0/3065245716 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0a14070300 0x7f0a14115c70 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:11:53.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:53.137+0000 7f0a1c400700 1 --2- 172.21.15.37:0/3065245716 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0a14070c10 0x7f0a141161b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:11:53.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:53.137+0000 7f0a1c400700 1 -- 172.21.15.37:0/3065245716 >> 172.21.15.37:0/3065245716 conn(0x7f0a140f89a0 msgr2=0x7f0a140f9f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:11:53.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:53.137+0000 7f0a1c400700 1 -- 172.21.15.37:0/3065245716 shutdown_connections 2024-03-29T08:11:53.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:53.137+0000 7f0a1c400700 1 -- 172.21.15.37:0/3065245716 wait complete. 2024-03-29T08:11:53.152 INFO:teuthology.orchestra.run.smithi037.stdout:true 2024-03-29T08:11:53.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:53.281+0000 7f554ee00700 1 -- 172.21.15.37:0/1909045714 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5548102c40 msgr2=0x7f5548103010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:11:53.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:53.281+0000 7f554ee00700 1 --2- 172.21.15.37:0/1909045714 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5548102c40 0x7f5548103010 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f553c004940 tx=0x7f553c009e30 comp rx=0 tx=0).stop 2024-03-29T08:11:53.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:53.281+0000 7f554ee00700 1 -- 172.21.15.37:0/1909045714 shutdown_connections 2024-03-29T08:11:53.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:53.281+0000 7f554ee00700 1 --2- 172.21.15.37:0/1909045714 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f55481035e0 0x7f55480fcc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:11:53.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:53.281+0000 7f554ee00700 1 --2- 172.21.15.37:0/1909045714 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5548102c40 0x7f5548103010 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:11:53.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:53.281+0000 7f554ee00700 1 -- 172.21.15.37:0/1909045714 >> 172.21.15.37:0/1909045714 conn(0x7f55480f8830 msgr2=0x7f55480fac40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:11:53.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:53.281+0000 7f554ee00700 1 -- 172.21.15.37:0/1909045714 shutdown_connections 2024-03-29T08:11:53.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:53.281+0000 7f554ee00700 1 -- 172.21.15.37:0/1909045714 wait complete. 2024-03-29T08:11:53.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:53.282+0000 7f554ee00700 1 Processor -- start 2024-03-29T08:11:53.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:53.283+0000 7f554ee00700 1 -- start start 2024-03-29T08:11:53.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:53.283+0000 7f554ee00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5548102c40 0x7f55480702c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:11:53.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:53.283+0000 7f554ee00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f55481035e0 0x7f5548070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:11:53.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:53.283+0000 7f554ee00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5548070d90 con 0x7f55481035e0 2024-03-29T08:11:53.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:53.283+0000 7f554ee00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5548070ed0 con 0x7f5548102c40 2024-03-29T08:11:53.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:53.283+0000 7f554d000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f55481035e0 0x7f5548070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:11:53.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:53.283+0000 7f554d000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f55481035e0 0x7f5548070800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54424/0 (socket says 172.21.15.37:54424) 2024-03-29T08:11:53.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:53.283+0000 7f554d000700 1 -- 172.21.15.37:0/645977291 learned_addr learned my addr 172.21.15.37:0/645977291 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:11:53.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:53.283+0000 7f554da00700 1 --2- 172.21.15.37:0/645977291 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5548102c40 0x7f55480702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:11:53.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:53.284+0000 7f554d000700 1 -- 172.21.15.37:0/645977291 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5548102c40 msgr2=0x7f55480702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:11:53.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:53.284+0000 7f554d000700 1 --2- 172.21.15.37:0/645977291 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5548102c40 0x7f55480702c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:11:53.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:53.284+0000 7f554d000700 1 -- 172.21.15.37:0/645977291 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f553c005040 con 0x7f55481035e0 2024-03-29T08:11:53.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:53.284+0000 7f554d000700 1 --2- 172.21.15.37:0/645977291 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f55481035e0 0x7f5548070800 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f553400eb40 tx=0x7f553400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:11:53.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:53.284+0000 7f5545600700 1 -- 172.21.15.37:0/645977291 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5534007cc0 con 0x7f55481035e0 2024-03-29T08:11:53.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:53.284+0000 7f5545600700 1 -- 172.21.15.37:0/645977291 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f553400f040 con 0x7f55481035e0 2024-03-29T08:11:53.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:53.284+0000 7f554ee00700 1 -- 172.21.15.37:0/645977291 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5548071120 con 0x7f55481035e0 2024-03-29T08:11:53.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:53.284+0000 7f5545600700 1 -- 172.21.15.37:0/645977291 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f553400c8f0 con 0x7f55481035e0 2024-03-29T08:11:53.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:53.285+0000 7f554ee00700 1 -- 172.21.15.37:0/645977291 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5548119fa0 con 0x7f55481035e0 2024-03-29T08:11:53.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:53.286+0000 7f554ee00700 1 -- 172.21.15.37:0/645977291 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55480fe390 con 0x7f55481035e0 2024-03-29T08:11:53.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:53.286+0000 7f5545600700 1 -- 172.21.15.37:0/645977291 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 34) v1 ==== 90703+0+0 (secure 0 0 0) 0x7f553400cb60 con 0x7f55481035e0 2024-03-29T08:11:53.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:53.288+0000 7f5545600700 1 --2- 172.21.15.37:0/645977291 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7f553006c8b0 0x7f553006ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:11:53.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:53.288+0000 7f5545600700 1 -- 172.21.15.37:0/645977291 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f553408b520 con 0x7f55481035e0 2024-03-29T08:11:53.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:53.288+0000 7f554da00700 1 --2- 172.21.15.37:0/645977291 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7f553006c8b0 0x7f553006ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:11:53.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:53.289+0000 7f554da00700 1 --2- 172.21.15.37:0/645977291 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7f553006c8b0 0x7f553006ed60 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f553c0045a0 tx=0x7f553c009e30 comp rx=0 tx=0).ready entity=mgr.14686 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:11:53.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:53.292+0000 7f5545600700 1 -- 172.21.15.37:0/645977291 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181579 (secure 0 0 0) 0x7f5534055ab0 con 0x7f55481035e0 2024-03-29T08:11:53.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:53.535+0000 7f554ee00700 1 -- 172.21.15.37:0/645977291 --> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5548109c60 con 0x7f553006c8b0 2024-03-29T08:11:53.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:53.537+0000 7f5545600700 1 -- 172.21.15.37:0/645977291 <== mgr.14686 v2:172.21.15.37:6800/4188329516 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+368 (secure 0 0 0) 0x7f5548109c60 con 0x7f553006c8b0 2024-03-29T08:11:53.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:53.539+0000 7f554ee00700 1 -- 172.21.15.37:0/645977291 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7f553006c8b0 msgr2=0x7f553006ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:11:53.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:53.539+0000 7f554ee00700 1 --2- 172.21.15.37:0/645977291 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7f553006c8b0 0x7f553006ed60 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f553c0045a0 tx=0x7f553c009e30 comp rx=0 tx=0).stop 2024-03-29T08:11:53.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:53.539+0000 7f554ee00700 1 -- 172.21.15.37:0/645977291 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f55481035e0 msgr2=0x7f5548070800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:11:53.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:53.539+0000 7f554ee00700 1 --2- 172.21.15.37:0/645977291 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f55481035e0 0x7f5548070800 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f553400eb40 tx=0x7f553400b040 comp rx=0 tx=0).stop 2024-03-29T08:11:53.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:53.539+0000 7f554ee00700 1 -- 172.21.15.37:0/645977291 shutdown_connections 2024-03-29T08:11:53.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:53.539+0000 7f554ee00700 1 --2- 172.21.15.37:0/645977291 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7f553006c8b0 0x7f553006ed60 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:11:53.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:53.539+0000 7f554ee00700 1 --2- 172.21.15.37:0/645977291 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5548102c40 0x7f55480702c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:11:53.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:53.539+0000 7f554ee00700 1 --2- 172.21.15.37:0/645977291 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f55481035e0 0x7f5548070800 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:11:53.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:53.539+0000 7f554ee00700 1 -- 172.21.15.37:0/645977291 >> 172.21.15.37:0/645977291 conn(0x7f55480f8830 msgr2=0x7f55480f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:11:53.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:53.540+0000 7f554ee00700 1 -- 172.21.15.37:0/645977291 shutdown_connections 2024-03-29T08:11:53.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:53.540+0000 7f554ee00700 1 -- 172.21.15.37:0/645977291 wait complete. 2024-03-29T08:11:53.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:53.679+0000 7f69ba800700 1 -- 172.21.15.37:0/2809918421 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f69b4102e00 msgr2=0x7f69b41031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:11:53.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:53.679+0000 7f69ba800700 1 --2- 172.21.15.37:0/2809918421 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f69b4102e00 0x7f69b41031d0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f69a4004940 tx=0x7f69a4009e30 comp rx=0 tx=0).stop 2024-03-29T08:11:53.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:53.679+0000 7f69ba800700 1 -- 172.21.15.37:0/2809918421 shutdown_connections 2024-03-29T08:11:53.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:53.679+0000 7f69ba800700 1 --2- 172.21.15.37:0/2809918421 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f69b41037a0 0x7f69b40fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:11:53.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:53.679+0000 7f69ba800700 1 --2- 172.21.15.37:0/2809918421 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f69b4102e00 0x7f69b41031d0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:11:53.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:53.679+0000 7f69ba800700 1 -- 172.21.15.37:0/2809918421 >> 172.21.15.37:0/2809918421 conn(0x7f69b40f89b0 msgr2=0x7f69b40fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:11:53.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:53.679+0000 7f69ba800700 1 -- 172.21.15.37:0/2809918421 shutdown_connections 2024-03-29T08:11:53.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:53.679+0000 7f69ba800700 1 -- 172.21.15.37:0/2809918421 wait complete. 2024-03-29T08:11:53.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:53.681+0000 7f69ba800700 1 Processor -- start 2024-03-29T08:11:53.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:53.681+0000 7f69ba800700 1 -- start start 2024-03-29T08:11:53.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:53.681+0000 7f69ba800700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f69b4102e00 0x7f69b40702c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:11:53.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:53.681+0000 7f69ba800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f69b41037a0 0x7f69b4070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:11:53.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:53.681+0000 7f69ba800700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69b4072210 con 0x7f69b41037a0 2024-03-29T08:11:53.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:53.681+0000 7f69ba800700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69b4070d40 con 0x7f69b4102e00 2024-03-29T08:11:53.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:53.681+0000 7f69b2a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f69b41037a0 0x7f69b4070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:11:53.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:53.681+0000 7f69b2a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f69b41037a0 0x7f69b4070800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54440/0 (socket says 172.21.15.37:54440) 2024-03-29T08:11:53.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:53.681+0000 7f69b2a00700 1 -- 172.21.15.37:0/3285232253 learned_addr learned my addr 172.21.15.37:0/3285232253 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:11:53.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:53.681+0000 7f69b3400700 1 --2- 172.21.15.37:0/3285232253 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f69b4102e00 0x7f69b40702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:11:53.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:53.682+0000 7f69b2a00700 1 -- 172.21.15.37:0/3285232253 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f69b4102e00 msgr2=0x7f69b40702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:11:53.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:53.682+0000 7f69b2a00700 1 --2- 172.21.15.37:0/3285232253 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f69b4102e00 0x7f69b40702c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:11:53.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:53.682+0000 7f69b2a00700 1 -- 172.21.15.37:0/3285232253 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69a4005040 con 0x7f69b41037a0 2024-03-29T08:11:53.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:53.682+0000 7f69b3400700 1 --2- 172.21.15.37:0/3285232253 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f69b4102e00 0x7f69b40702c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T08:11:53.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:53.682+0000 7f69b2a00700 1 --2- 172.21.15.37:0/3285232253 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f69b41037a0 0x7f69b4070800 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f69a0002a50 tx=0x7f69a0004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:11:53.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:53.682+0000 7f69ab400700 1 -- 172.21.15.37:0/3285232253 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f69a0007cc0 con 0x7f69b41037a0 2024-03-29T08:11:53.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:53.682+0000 7f69ab400700 1 -- 172.21.15.37:0/3285232253 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f69a000f040 con 0x7f69b41037a0 2024-03-29T08:11:53.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:53.682+0000 7f69ab400700 1 -- 172.21.15.37:0/3285232253 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f69a000d920 con 0x7f69b41037a0 2024-03-29T08:11:53.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:53.682+0000 7f69ba800700 1 -- 172.21.15.37:0/3285232253 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69b4071020 con 0x7f69b41037a0 2024-03-29T08:11:53.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:53.683+0000 7f69ba800700 1 -- 172.21.15.37:0/3285232253 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69b411a060 con 0x7f69b41037a0 2024-03-29T08:11:53.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:53.684+0000 7f69ba800700 1 -- 172.21.15.37:0/3285232253 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69b40fe550 con 0x7f69b41037a0 2024-03-29T08:11:53.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:53.685+0000 7f69ab400700 1 -- 172.21.15.37:0/3285232253 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 34) v1 ==== 90703+0+0 (secure 0 0 0) 0x7f69a0009680 con 0x7f69b41037a0 2024-03-29T08:11:53.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:53.686+0000 7f69ab400700 1 --2- 172.21.15.37:0/3285232253 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7f699806c8a0 0x7f699806ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:11:53.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:53.686+0000 7f69ab400700 1 -- 172.21.15.37:0/3285232253 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f69a008b730 con 0x7f69b41037a0 2024-03-29T08:11:53.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:53.686+0000 7f69b3400700 1 --2- 172.21.15.37:0/3285232253 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7f699806c8a0 0x7f699806ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:11:53.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:53.687+0000 7f69b3400700 1 --2- 172.21.15.37:0/3285232253 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7f699806c8a0 0x7f699806ed50 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f69a4004f90 tx=0x7f69a4009e30 comp rx=0 tx=0).ready entity=mgr.14686 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:11:53.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:53.690+0000 7f69ab400700 1 -- 172.21.15.37:0/3285232253 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181579 (secure 0 0 0) 0x7f69a0055c90 con 0x7f69b41037a0 2024-03-29T08:11:53.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:53.916+0000 7f69ba800700 1 -- 172.21.15.37:0/3285232253 --> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f69b40fa180 con 0x7f699806c8a0 2024-03-29T08:11:53.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:53.927+0000 7f69ab400700 1 -- 172.21.15.37:0/3285232253 <== mgr.14686 v2:172.21.15.37:6800/4188329516 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7f69b40fa180 con 0x7f699806c8a0 2024-03-29T08:11:53.927 INFO:teuthology.orchestra.run.smithi037.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-29T08:11:53.928 INFO:teuthology.orchestra.run.smithi037.stdout:alertmanager.smithi037 smithi037 *:9093,9094 running (104s) 78s ago 14m 13.7M - 0.25.0 c8568f914cd2 2f9f9746ed0f 2024-03-29T08:11:53.928 INFO:teuthology.orchestra.run.smithi037.stdout:crash.smithi037 smithi037 running (14m) 78s ago 14m 6656k - 17.2.7-789-g4213e17c cf22eb8683be e0343935b25e 2024-03-29T08:11:53.928 INFO:teuthology.orchestra.run.smithi037.stdout:crash.smithi148 smithi148 running (13m) 2m ago 13m 6656k - 17.2.7-789-g4213e17c cf22eb8683be ff5164c552c1 2024-03-29T08:11:53.928 INFO:teuthology.orchestra.run.smithi037.stdout:grafana.smithi037 smithi037 *:3000 running (84s) 78s ago 14m 45.0M - 9.4.7 954c08fa6188 6b976fe49952 2024-03-29T08:11:53.928 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi037.iifsmh smithi037 running (8m) 78s ago 8m 110M - 17.2.7-789-g4213e17c cf22eb8683be 074e1acf331f 2024-03-29T08:11:53.928 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi037.mmwgxe smithi037 running (8m) 78s ago 8m 115M - 17.2.7-789-g4213e17c cf22eb8683be 9a5b1e72de5c 2024-03-29T08:11:53.928 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi148.qukpjh smithi148 running (8m) 2m ago 8m 12.9M - 17.2.7-789-g4213e17c cf22eb8683be 8e469782a273 2024-03-29T08:11:53.928 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi148.sqteih smithi148 running (8m) 2m ago 8m 12.8M - 17.2.7-789-g4213e17c cf22eb8683be 8e68f858bb22 2024-03-29T08:11:53.928 INFO:teuthology.orchestra.run.smithi037.stdout:mgr.smithi037.wjikii smithi037 *:8443,9283,8765 running (4m) 78s ago 16m 498M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff c5a1417b9417 2024-03-29T08:11:53.929 INFO:teuthology.orchestra.run.smithi037.stdout:mgr.smithi148.onhgvm smithi148 *:8443,9283,8765 running (2m) 2m ago 13m 446M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff fcfdbce062bd 2024-03-29T08:11:53.929 INFO:teuthology.orchestra.run.smithi037.stdout:mon.smithi037 smithi037 running (16m) 78s ago 16m 54.2M 2048M 17.2.7-789-g4213e17c cf22eb8683be bb2101505730 2024-03-29T08:11:53.929 INFO:teuthology.orchestra.run.smithi037.stdout:mon.smithi148 smithi148 running (13m) 2m ago 13m 42.0M 2048M 17.2.7-789-g4213e17c cf22eb8683be 18839ed2952a 2024-03-29T08:11:53.929 INFO:teuthology.orchestra.run.smithi037.stdout:node-exporter.smithi037 smithi037 *:9100 running (2m) 78s ago 14m 7197k - 1.5.0 0da6a335fe13 26b8ae9cb878 2024-03-29T08:11:53.929 INFO:teuthology.orchestra.run.smithi037.stdout:node-exporter.smithi148 smithi148 *:9100 running (2m) 2m ago 13m 2693k - 1.5.0 0da6a335fe13 98414e4e64ed 2024-03-29T08:11:53.929 INFO:teuthology.orchestra.run.smithi037.stdout:osd.0 smithi037 running (12m) 78s ago 12m 217M 1619M 17.2.7-789-g4213e17c cf22eb8683be 559fccd0ebe1 2024-03-29T08:11:53.929 INFO:teuthology.orchestra.run.smithi037.stdout:osd.1 smithi037 running (12m) 78s ago 12m 228M 1619M 17.2.7-789-g4213e17c cf22eb8683be 5582cff11b7a 2024-03-29T08:11:53.929 INFO:teuthology.orchestra.run.smithi037.stdout:osd.2 smithi037 running (11m) 78s ago 11m 196M 1619M 17.2.7-789-g4213e17c cf22eb8683be f809bfab13e5 2024-03-29T08:11:53.929 INFO:teuthology.orchestra.run.smithi037.stdout:osd.3 smithi148 running (10m) 2m ago 10m 259M 2642M 17.2.7-789-g4213e17c cf22eb8683be a0a8224bb4b0 2024-03-29T08:11:53.930 INFO:teuthology.orchestra.run.smithi037.stdout:osd.4 smithi148 running (10m) 2m ago 10m 230M 2642M 17.2.7-789-g4213e17c cf22eb8683be a00b69047ed7 2024-03-29T08:11:53.930 INFO:teuthology.orchestra.run.smithi037.stdout:osd.5 smithi148 running (9m) 2m ago 9m 231M 2642M 17.2.7-789-g4213e17c cf22eb8683be 044a3f7db7f8 2024-03-29T08:11:53.930 INFO:teuthology.orchestra.run.smithi037.stdout:prometheus.smithi037 smithi037 *:9095 running (2m) 78s ago 14m 44.3M - 2.43.0 a07b618ecd1d 30c0223ec589 2024-03-29T08:11:53.930 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:53 smithi037 ceph-mon[35162]: pgmap v116: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:11:53.931 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:53 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' 2024-03-29T08:11:53.931 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:53 smithi037 ceph-mon[35162]: Upgrade: Target is version 18.2.2-1015-gd0e392d9 (reef) 2024-03-29T08:11:53.931 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:53 smithi037 ceph-mon[35162]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59, digests ['quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59'] 2024-03-29T08:11:53.931 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:53 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T08:11:53.931 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:53 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' 2024-03-29T08:11:53.931 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:53 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:11:53.931 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:53 smithi037 ceph-mon[35162]: Upgrade: Setting container_image for all mgr 2024-03-29T08:11:53.931 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:53 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' 2024-03-29T08:11:53.931 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:53 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-03-29T08:11:53.931 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:53 smithi037 ceph-mon[35162]: from='client.14866 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:11:53.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:53.929+0000 7f69ba800700 1 -- 172.21.15.37:0/3285232253 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7f699806c8a0 msgr2=0x7f699806ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:11:53.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:53.929+0000 7f69ba800700 1 --2- 172.21.15.37:0/3285232253 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7f699806c8a0 0x7f699806ed50 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f69a4004f90 tx=0x7f69a4009e30 comp rx=0 tx=0).stop 2024-03-29T08:11:53.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:53.929+0000 7f69ba800700 1 -- 172.21.15.37:0/3285232253 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f69b41037a0 msgr2=0x7f69b4070800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:11:53.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:53.929+0000 7f69ba800700 1 --2- 172.21.15.37:0/3285232253 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f69b41037a0 0x7f69b4070800 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f69a0002a50 tx=0x7f69a0004600 comp rx=0 tx=0).stop 2024-03-29T08:11:53.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:53.929+0000 7f69ba800700 1 -- 172.21.15.37:0/3285232253 shutdown_connections 2024-03-29T08:11:53.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:53.929+0000 7f69ba800700 1 --2- 172.21.15.37:0/3285232253 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7f699806c8a0 0x7f699806ed50 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:11:53.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:53.929+0000 7f69ba800700 1 --2- 172.21.15.37:0/3285232253 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f69b4102e00 0x7f69b40702c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:11:53.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:53.929+0000 7f69ba800700 1 --2- 172.21.15.37:0/3285232253 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f69b41037a0 0x7f69b4070800 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:11:53.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:53.929+0000 7f69ba800700 1 -- 172.21.15.37:0/3285232253 >> 172.21.15.37:0/3285232253 conn(0x7f69b40f89b0 msgr2=0x7f69b40f9a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:11:53.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:53.930+0000 7f69ba800700 1 -- 172.21.15.37:0/3285232253 shutdown_connections 2024-03-29T08:11:53.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:53.930+0000 7f69ba800700 1 -- 172.21.15.37:0/3285232253 wait complete. 2024-03-29T08:11:54.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:11:53 smithi148 ceph-mon[44992]: pgmap v116: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:11:54.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:11:53 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' 2024-03-29T08:11:54.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:11:53 smithi148 ceph-mon[44992]: Upgrade: Target is version 18.2.2-1015-gd0e392d9 (reef) 2024-03-29T08:11:54.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:11:53 smithi148 ceph-mon[44992]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59, digests ['quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59'] 2024-03-29T08:11:54.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:11:53 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T08:11:54.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:11:53 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' 2024-03-29T08:11:54.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:11:53 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:11:54.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:11:53 smithi148 ceph-mon[44992]: Upgrade: Setting container_image for all mgr 2024-03-29T08:11:54.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:11:53 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' 2024-03-29T08:11:54.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:11:53 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-03-29T08:11:54.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:11:53 smithi148 ceph-mon[44992]: from='client.14866 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:11:54.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:54.071+0000 7fd807600700 1 -- 172.21.15.37:0/937429544 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd800102e00 msgr2=0x7fd8001031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:11:54.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:54.071+0000 7fd807600700 1 --2- 172.21.15.37:0/937429544 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd800102e00 0x7fd8001031d0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fd7f4004940 tx=0x7fd7f4009e30 comp rx=0 tx=0).stop 2024-03-29T08:11:54.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:54.071+0000 7fd807600700 1 -- 172.21.15.37:0/937429544 shutdown_connections 2024-03-29T08:11:54.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:54.071+0000 7fd807600700 1 --2- 172.21.15.37:0/937429544 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd8001037a0 0x7fd8000fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:11:54.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:54.071+0000 7fd807600700 1 --2- 172.21.15.37:0/937429544 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd800102e00 0x7fd8001031d0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:11:54.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:54.071+0000 7fd807600700 1 -- 172.21.15.37:0/937429544 >> 172.21.15.37:0/937429544 conn(0x7fd8000f89b0 msgr2=0x7fd8000fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:11:54.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:54.071+0000 7fd807600700 1 -- 172.21.15.37:0/937429544 shutdown_connections 2024-03-29T08:11:54.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:54.071+0000 7fd807600700 1 -- 172.21.15.37:0/937429544 wait complete. 2024-03-29T08:11:54.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:54.072+0000 7fd807600700 1 Processor -- start 2024-03-29T08:11:54.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:54.072+0000 7fd807600700 1 -- start start 2024-03-29T08:11:54.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:54.073+0000 7fd807600700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd800102e00 0x7fd800115c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:11:54.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:54.073+0000 7fd807600700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd8001037a0 0x7fd800116180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:11:54.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:54.073+0000 7fd807600700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8001167d0 con 0x7fd8001037a0 2024-03-29T08:11:54.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:54.073+0000 7fd807600700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd80010fba0 con 0x7fd800102e00 2024-03-29T08:11:54.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:54.073+0000 7fd7ffe00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd8001037a0 0x7fd800116180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:11:54.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:54.073+0000 7fd7ffe00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd8001037a0 0x7fd800116180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54462/0 (socket says 172.21.15.37:54462) 2024-03-29T08:11:54.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:54.073+0000 7fd7ffe00700 1 -- 172.21.15.37:0/2400999327 learned_addr learned my addr 172.21.15.37:0/2400999327 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:11:54.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:54.073+0000 7fd804c00700 1 --2- 172.21.15.37:0/2400999327 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd800102e00 0x7fd800115c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:11:54.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:54.073+0000 7fd7ffe00700 1 -- 172.21.15.37:0/2400999327 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd800102e00 msgr2=0x7fd800115c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:11:54.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:54.073+0000 7fd7ffe00700 1 --2- 172.21.15.37:0/2400999327 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd800102e00 0x7fd800115c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:11:54.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:54.073+0000 7fd7ffe00700 1 -- 172.21.15.37:0/2400999327 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7f4005040 con 0x7fd8001037a0 2024-03-29T08:11:54.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:54.073+0000 7fd7ffe00700 1 --2- 172.21.15.37:0/2400999327 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd8001037a0 0x7fd800116180 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fd7ec00eb40 tx=0x7fd7ec00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:11:54.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:54.074+0000 7fd7fcc00700 1 -- 172.21.15.37:0/2400999327 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd7ec007cc0 con 0x7fd8001037a0 2024-03-29T08:11:54.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:54.074+0000 7fd7fcc00700 1 -- 172.21.15.37:0/2400999327 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd7ec00f040 con 0x7fd8001037a0 2024-03-29T08:11:54.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:54.074+0000 7fd7fcc00700 1 -- 172.21.15.37:0/2400999327 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd7ec00c8f0 con 0x7fd8001037a0 2024-03-29T08:11:54.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:54.074+0000 7fd807600700 1 -- 172.21.15.37:0/2400999327 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd80010fe20 con 0x7fd8001037a0 2024-03-29T08:11:54.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:54.074+0000 7fd807600700 1 -- 172.21.15.37:0/2400999327 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8001103e0 con 0x7fd8001037a0 2024-03-29T08:11:54.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:54.075+0000 7fd807600700 1 -- 172.21.15.37:0/2400999327 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8000fe550 con 0x7fd8001037a0 2024-03-29T08:11:54.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:54.076+0000 7fd7fcc00700 1 -- 172.21.15.37:0/2400999327 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 34) v1 ==== 90703+0+0 (secure 0 0 0) 0x7fd7ec0090d0 con 0x7fd8001037a0 2024-03-29T08:11:54.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:54.077+0000 7fd7fcc00700 1 --2- 172.21.15.37:0/2400999327 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7fd7e806c8b0 0x7fd7e806ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:11:54.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:54.077+0000 7fd7fcc00700 1 -- 172.21.15.37:0/2400999327 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6066+0+0 (secure 0 0 0) 0x7fd7ec08b4c0 con 0x7fd8001037a0 2024-03-29T08:11:54.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:54.078+0000 7fd804c00700 1 --2- 172.21.15.37:0/2400999327 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7fd7e806c8b0 0x7fd7e806ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:11:54.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:54.078+0000 7fd804c00700 1 --2- 172.21.15.37:0/2400999327 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7fd7e806c8b0 0x7fd7e806ed60 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd7f40045a0 tx=0x7fd7f4009e30 comp rx=0 tx=0).ready entity=mgr.14686 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:11:54.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:54.081+0000 7fd7fcc00700 1 -- 172.21.15.37:0/2400999327 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181579 (secure 0 0 0) 0x7fd7ec0594c0 con 0x7fd8001037a0 2024-03-29T08:11:54.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:54.413+0000 7fd807600700 1 -- 172.21.15.37:0/2400999327 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fd80004bed0 con 0x7fd8001037a0 2024-03-29T08:11:54.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:54.414+0000 7fd7fcc00700 1 -- 172.21.15.37:0/2400999327 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+743 (secure 0 0 0) 0x7fd7ec059050 con 0x7fd8001037a0 2024-03-29T08:11:54.414 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-29T08:11:54.414 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": { 2024-03-29T08:11:54.415 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 17.2.7-789-g4213e17c (4213e17c4de67016ed14f55d22503f4923b3a901) quincy (stable)": 2 2024-03-29T08:11:54.415 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:11:54.415 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": { 2024-03-29T08:11:54.415 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 2 2024-03-29T08:11:54.415 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:11:54.415 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": { 2024-03-29T08:11:54.415 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 17.2.7-789-g4213e17c (4213e17c4de67016ed14f55d22503f4923b3a901) quincy (stable)": 6 2024-03-29T08:11:54.415 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:11:54.415 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": { 2024-03-29T08:11:54.415 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 17.2.7-789-g4213e17c (4213e17c4de67016ed14f55d22503f4923b3a901) quincy (stable)": 4 2024-03-29T08:11:54.415 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:11:54.415 INFO:teuthology.orchestra.run.smithi037.stdout: "overall": { 2024-03-29T08:11:54.416 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 17.2.7-789-g4213e17c (4213e17c4de67016ed14f55d22503f4923b3a901) quincy (stable)": 12, 2024-03-29T08:11:54.416 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 2 2024-03-29T08:11:54.416 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-29T08:11:54.416 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-29T08:11:54.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:54.416+0000 7fd807600700 1 -- 172.21.15.37:0/2400999327 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7fd7e806c8b0 msgr2=0x7fd7e806ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:11:54.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:54.416+0000 7fd807600700 1 --2- 172.21.15.37:0/2400999327 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7fd7e806c8b0 0x7fd7e806ed60 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd7f40045a0 tx=0x7fd7f4009e30 comp rx=0 tx=0).stop 2024-03-29T08:11:54.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:54.416+0000 7fd807600700 1 -- 172.21.15.37:0/2400999327 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd8001037a0 msgr2=0x7fd800116180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:11:54.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:54.416+0000 7fd807600700 1 --2- 172.21.15.37:0/2400999327 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd8001037a0 0x7fd800116180 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fd7ec00eb40 tx=0x7fd7ec00b040 comp rx=0 tx=0).stop 2024-03-29T08:11:54.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:54.416+0000 7fd807600700 1 -- 172.21.15.37:0/2400999327 shutdown_connections 2024-03-29T08:11:54.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:54.416+0000 7fd807600700 1 --2- 172.21.15.37:0/2400999327 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7fd7e806c8b0 0x7fd7e806ed60 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:11:54.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:54.416+0000 7fd807600700 1 --2- 172.21.15.37:0/2400999327 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd800102e00 0x7fd800115c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:11:54.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:54.416+0000 7fd807600700 1 --2- 172.21.15.37:0/2400999327 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd8001037a0 0x7fd800116180 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:11:54.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:54.416+0000 7fd807600700 1 -- 172.21.15.37:0/2400999327 >> 172.21.15.37:0/2400999327 conn(0x7fd8000f89b0 msgr2=0x7fd8000f99f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:11:54.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:54.416+0000 7fd807600700 1 -- 172.21.15.37:0/2400999327 shutdown_connections 2024-03-29T08:11:54.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:54.417+0000 7fd807600700 1 -- 172.21.15.37:0/2400999327 wait complete. 2024-03-29T08:11:54.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:54.561+0000 7fb89b200700 1 -- 172.21.15.37:0/2808151594 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb894070360 msgr2=0x7fb894070730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:11:54.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:54.561+0000 7fb89b200700 1 --2- 172.21.15.37:0/2808151594 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb894070360 0x7fb894070730 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fb884004970 tx=0x7fb884009e30 comp rx=0 tx=0).stop 2024-03-29T08:11:54.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:54.561+0000 7fb89b200700 1 -- 172.21.15.37:0/2808151594 shutdown_connections 2024-03-29T08:11:54.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:54.561+0000 7fb89b200700 1 --2- 172.21.15.37:0/2808151594 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb894070c70 0x7fb894109110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:11:54.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:54.561+0000 7fb89b200700 1 --2- 172.21.15.37:0/2808151594 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb894070360 0x7fb894070730 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:11:54.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:54.561+0000 7fb89b200700 1 -- 172.21.15.37:0/2808151594 >> 172.21.15.37:0/2808151594 conn(0x7fb8940f89b0 msgr2=0x7fb8940fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:11:54.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:54.561+0000 7fb89b200700 1 -- 172.21.15.37:0/2808151594 shutdown_connections 2024-03-29T08:11:54.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:54.561+0000 7fb89b200700 1 -- 172.21.15.37:0/2808151594 wait complete. 2024-03-29T08:11:54.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:54.562+0000 7fb89b200700 1 Processor -- start 2024-03-29T08:11:54.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:54.562+0000 7fb89b200700 1 -- start start 2024-03-29T08:11:54.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:54.563+0000 7fb89b200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb894070360 0x7fb894115c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:11:54.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:54.563+0000 7fb89b200700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb894070c70 0x7fb8941161c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:11:54.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:54.563+0000 7fb89b200700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb894116810 con 0x7fb894070360 2024-03-29T08:11:54.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:54.563+0000 7fb89b200700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb89410fbe0 con 0x7fb894070c70 2024-03-29T08:11:54.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:54.563+0000 7fb893e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb894070360 0x7fb894115c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:11:54.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:54.563+0000 7fb893e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb894070360 0x7fb894115c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54480/0 (socket says 172.21.15.37:54480) 2024-03-29T08:11:54.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:54.563+0000 7fb893e00700 1 -- 172.21.15.37:0/3581205756 learned_addr learned my addr 172.21.15.37:0/3581205756 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:11:54.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:54.563+0000 7fb893400700 1 --2- 172.21.15.37:0/3581205756 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb894070c70 0x7fb8941161c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:11:54.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:54.563+0000 7fb893e00700 1 -- 172.21.15.37:0/3581205756 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb894070c70 msgr2=0x7fb8941161c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:11:54.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:54.563+0000 7fb893e00700 1 --2- 172.21.15.37:0/3581205756 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb894070c70 0x7fb8941161c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:11:54.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:54.563+0000 7fb893e00700 1 -- 172.21.15.37:0/3581205756 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb884005040 con 0x7fb894070360 2024-03-29T08:11:54.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:54.564+0000 7fb893e00700 1 --2- 172.21.15.37:0/3581205756 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb894070360 0x7fb894115c80 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fb884004f50 tx=0x7fb884009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:11:54.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:54.564+0000 7fb88be00700 1 -- 172.21.15.37:0/3581205756 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb88400e040 con 0x7fb894070360 2024-03-29T08:11:54.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:54.564+0000 7fb88be00700 1 -- 172.21.15.37:0/3581205756 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fb884013420 con 0x7fb894070360 2024-03-29T08:11:54.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:54.564+0000 7fb89b200700 1 -- 172.21.15.37:0/3581205756 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb89410fe60 con 0x7fb894070360 2024-03-29T08:11:54.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:54.564+0000 7fb88be00700 1 -- 172.21.15.37:0/3581205756 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb88401c630 con 0x7fb894070360 2024-03-29T08:11:54.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:54.564+0000 7fb89b200700 1 -- 172.21.15.37:0/3581205756 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8941103a0 con 0x7fb894070360 2024-03-29T08:11:54.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:54.566+0000 7fb89b200700 1 -- 172.21.15.37:0/3581205756 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb894103b90 con 0x7fb894070360 2024-03-29T08:11:54.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:54.567+0000 7fb88be00700 1 -- 172.21.15.37:0/3581205756 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 34) v1 ==== 90703+0+0 (secure 0 0 0) 0x7fb884013590 con 0x7fb894070360 2024-03-29T08:11:54.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:54.568+0000 7fb88be00700 1 --2- 172.21.15.37:0/3581205756 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7fb87806c8b0 0x7fb87806ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:11:54.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:54.568+0000 7fb88be00700 1 -- 172.21.15.37:0/3581205756 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6066+0+0 (secure 0 0 0) 0x7fb88408cb40 con 0x7fb894070360 2024-03-29T08:11:54.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:54.568+0000 7fb893400700 1 --2- 172.21.15.37:0/3581205756 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7fb87806c8b0 0x7fb87806ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:11:54.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:54.569+0000 7fb893400700 1 --2- 172.21.15.37:0/3581205756 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7fb87806c8b0 0x7fb87806ed60 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fb894111520 tx=0x7fb88000b040 comp rx=0 tx=0).ready entity=mgr.14686 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:11:54.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:54.573+0000 7fb88be00700 1 -- 172.21.15.37:0/3581205756 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181579 (secure 0 0 0) 0x7fb88405abd0 con 0x7fb894070360 2024-03-29T08:11:54.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:54.837+0000 7fb89b200700 1 -- 172.21.15.37:0/3581205756 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fb89410bfb0 con 0x7fb894070360 2024-03-29T08:11:54.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:54.837+0000 7fb88be00700 1 -- 172.21.15.37:0/3581205756 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 20 v20) v1 ==== 76+0+1820 (secure 0 0 0) 0x7fb88405a760 con 0x7fb894070360 2024-03-29T08:11:54.837 INFO:teuthology.orchestra.run.smithi037.stdout:e20 2024-03-29T08:11:54.837 INFO:teuthology.orchestra.run.smithi037.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-29T08:11:54.837 INFO:teuthology.orchestra.run.smithi037.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T08:11:54.838 INFO:teuthology.orchestra.run.smithi037.stdout:legacy client fscid: 1 2024-03-29T08:11:54.838 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:11:54.838 INFO:teuthology.orchestra.run.smithi037.stdout:Filesystem 'cephfs' (1) 2024-03-29T08:11:54.838 INFO:teuthology.orchestra.run.smithi037.stdout:fs_name cephfs 2024-03-29T08:11:54.838 INFO:teuthology.orchestra.run.smithi037.stdout:epoch 20 2024-03-29T08:11:54.838 INFO:teuthology.orchestra.run.smithi037.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-03-29T08:11:54.838 INFO:teuthology.orchestra.run.smithi037.stdout:created 2024-03-29T08:03:29.241269+0000 2024-03-29T08:11:54.838 INFO:teuthology.orchestra.run.smithi037.stdout:modified 2024-03-29T08:11:27.181622+0000 2024-03-29T08:11:54.838 INFO:teuthology.orchestra.run.smithi037.stdout:tableserver 0 2024-03-29T08:11:54.838 INFO:teuthology.orchestra.run.smithi037.stdout:root 0 2024-03-29T08:11:54.838 INFO:teuthology.orchestra.run.smithi037.stdout:session_timeout 60 2024-03-29T08:11:54.838 INFO:teuthology.orchestra.run.smithi037.stdout:session_autoclose 300 2024-03-29T08:11:54.838 INFO:teuthology.orchestra.run.smithi037.stdout:max_file_size 1099511627776 2024-03-29T08:11:54.838 INFO:teuthology.orchestra.run.smithi037.stdout:required_client_features {} 2024-03-29T08:11:54.839 INFO:teuthology.orchestra.run.smithi037.stdout:last_failure 0 2024-03-29T08:11:54.839 INFO:teuthology.orchestra.run.smithi037.stdout:last_failure_osd_epoch 47 2024-03-29T08:11:54.839 INFO:teuthology.orchestra.run.smithi037.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T08:11:54.839 INFO:teuthology.orchestra.run.smithi037.stdout:max_mds 2 2024-03-29T08:11:54.839 INFO:teuthology.orchestra.run.smithi037.stdout:in 0,1 2024-03-29T08:11:54.839 INFO:teuthology.orchestra.run.smithi037.stdout:up {0=14470,1=14482} 2024-03-29T08:11:54.839 INFO:teuthology.orchestra.run.smithi037.stdout:failed 2024-03-29T08:11:54.839 INFO:teuthology.orchestra.run.smithi037.stdout:damaged 2024-03-29T08:11:54.839 INFO:teuthology.orchestra.run.smithi037.stdout:stopped 2024-03-29T08:11:54.839 INFO:teuthology.orchestra.run.smithi037.stdout:data_pools [3] 2024-03-29T08:11:54.839 INFO:teuthology.orchestra.run.smithi037.stdout:metadata_pool 2 2024-03-29T08:11:54.839 INFO:teuthology.orchestra.run.smithi037.stdout:inline_data disabled 2024-03-29T08:11:54.839 INFO:teuthology.orchestra.run.smithi037.stdout:balancer 2024-03-29T08:11:54.839 INFO:teuthology.orchestra.run.smithi037.stdout:standby_count_wanted 1 2024-03-29T08:11:54.839 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi037.mmwgxe{0:14470} state up:active seq 5 join_fscid=1 addr [v2:172.21.15.37:6826/1559618909,v1:172.21.15.37:6827/1559618909] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:11:54.840 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi037.iifsmh{1:14482} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.37:6828/2341530495,v1:172.21.15.37:6829/2341530495] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:11:54.840 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:11:54.840 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:11:54.840 INFO:teuthology.orchestra.run.smithi037.stdout:Standby daemons: 2024-03-29T08:11:54.840 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:11:54.840 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi148.qukpjh{-1:24319} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.148:6826/2641327912,v1:172.21.15.148:6827/2641327912] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:11:54.840 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi148.sqteih{-1:24327} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.148:6824/3252351726,v1:172.21.15.148:6825/3252351726] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:11:54.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:54.840+0000 7fb89b200700 1 -- 172.21.15.37:0/3581205756 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7fb87806c8b0 msgr2=0x7fb87806ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:11:54.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:54.840+0000 7fb89b200700 1 --2- 172.21.15.37:0/3581205756 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7fb87806c8b0 0x7fb87806ed60 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fb894111520 tx=0x7fb88000b040 comp rx=0 tx=0).stop 2024-03-29T08:11:54.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:54.840+0000 7fb89b200700 1 -- 172.21.15.37:0/3581205756 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb894070360 msgr2=0x7fb894115c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:11:54.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:54.840+0000 7fb89b200700 1 --2- 172.21.15.37:0/3581205756 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb894070360 0x7fb894115c80 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fb884004f50 tx=0x7fb884009e30 comp rx=0 tx=0).stop 2024-03-29T08:11:54.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:54.840+0000 7fb89b200700 1 -- 172.21.15.37:0/3581205756 shutdown_connections 2024-03-29T08:11:54.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:54.840+0000 7fb89b200700 1 --2- 172.21.15.37:0/3581205756 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7fb87806c8b0 0x7fb87806ed60 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:11:54.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:54.840+0000 7fb89b200700 1 --2- 172.21.15.37:0/3581205756 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb894070360 0x7fb894115c80 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:11:54.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:54.840+0000 7fb89b200700 1 --2- 172.21.15.37:0/3581205756 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb894070c70 0x7fb8941161c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:11:54.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:54.840+0000 7fb89b200700 1 -- 172.21.15.37:0/3581205756 >> 172.21.15.37:0/3581205756 conn(0x7fb8940f89b0 msgr2=0x7fb8941015f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:11:54.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:54.840+0000 7fb89b200700 1 -- 172.21.15.37:0/3581205756 shutdown_connections 2024-03-29T08:11:54.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:54.840+0000 7fb89b200700 1 -- 172.21.15.37:0/3581205756 wait complete. 2024-03-29T08:11:54.842 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 20 2024-03-29T08:11:54.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:54 smithi037 ceph-mon[35162]: from='client.14870 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:11:54.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:54 smithi037 ceph-mon[35162]: from='client.14874 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:11:54.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:54 smithi037 ceph-mon[35162]: from='client.? 172.21.15.37:0/2400999327' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:11:54.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:54 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' 2024-03-29T08:11:54.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:54 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-29T08:11:54.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:54 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-29T08:11:54.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:54 smithi037 ceph-mon[35162]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T08:11:54.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:54.991+0000 7fc1b6a00700 1 -- 172.21.15.37:0/240649961 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc1b0070c70 msgr2=0x7fc1b0109110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:11:54.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:54.991+0000 7fc1b6a00700 1 --2- 172.21.15.37:0/240649961 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc1b0070c70 0x7fc1b0109110 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fc19c004970 tx=0x7fc19c009e30 comp rx=0 tx=0).stop 2024-03-29T08:11:54.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:54.991+0000 7fc1b6a00700 1 -- 172.21.15.37:0/240649961 shutdown_connections 2024-03-29T08:11:54.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:54.991+0000 7fc1b6a00700 1 --2- 172.21.15.37:0/240649961 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc1b0070c70 0x7fc1b0109110 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:11:54.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:54.991+0000 7fc1b6a00700 1 --2- 172.21.15.37:0/240649961 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc1b0070360 0x7fc1b0070730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:11:54.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:54.991+0000 7fc1b6a00700 1 -- 172.21.15.37:0/240649961 >> 172.21.15.37:0/240649961 conn(0x7fc1b00f89b0 msgr2=0x7fc1b00fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:11:54.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:54.991+0000 7fc1b6a00700 1 -- 172.21.15.37:0/240649961 shutdown_connections 2024-03-29T08:11:54.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:54.991+0000 7fc1b6a00700 1 -- 172.21.15.37:0/240649961 wait complete. 2024-03-29T08:11:54.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:54.992+0000 7fc1b6a00700 1 Processor -- start 2024-03-29T08:11:54.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:54.992+0000 7fc1b6a00700 1 -- start start 2024-03-29T08:11:54.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:54.992+0000 7fc1b6a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc1b0070360 0x7fc1b0115cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:11:54.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:54.992+0000 7fc1b6a00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc1b0070c70 0x7fc1b01161f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:11:54.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:54.992+0000 7fc1b6a00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1b0116840 con 0x7fc1b0070360 2024-03-29T08:11:54.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:54.992+0000 7fc1b6a00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1b010fc10 con 0x7fc1b0070c70 2024-03-29T08:11:54.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:54.993+0000 7fc1afe00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc1b0070360 0x7fc1b0115cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:11:54.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:54.993+0000 7fc1afe00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc1b0070360 0x7fc1b0115cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54494/0 (socket says 172.21.15.37:54494) 2024-03-29T08:11:54.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:54.993+0000 7fc1afe00700 1 -- 172.21.15.37:0/1062484703 learned_addr learned my addr 172.21.15.37:0/1062484703 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:11:54.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:54.993+0000 7fc1af400700 1 --2- 172.21.15.37:0/1062484703 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc1b0070c70 0x7fc1b01161f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:11:54.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:54.993+0000 7fc1afe00700 1 -- 172.21.15.37:0/1062484703 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc1b0070c70 msgr2=0x7fc1b01161f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:11:54.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:54.993+0000 7fc1afe00700 1 --2- 172.21.15.37:0/1062484703 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc1b0070c70 0x7fc1b01161f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:11:54.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:54.993+0000 7fc1afe00700 1 -- 172.21.15.37:0/1062484703 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc19c005040 con 0x7fc1b0070360 2024-03-29T08:11:54.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:54.993+0000 7fc1afe00700 1 --2- 172.21.15.37:0/1062484703 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc1b0070360 0x7fc1b0115cb0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fc1a0002a50 tx=0x7fc1a00046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:11:54.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:54.994+0000 7fc1a7e00700 1 -- 172.21.15.37:0/1062484703 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc1a0007cc0 con 0x7fc1b0070360 2024-03-29T08:11:54.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:54.994+0000 7fc1a7e00700 1 -- 172.21.15.37:0/1062484703 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fc1a000f040 con 0x7fc1b0070360 2024-03-29T08:11:54.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:54.994+0000 7fc1b6a00700 1 -- 172.21.15.37:0/1062484703 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1b010fef0 con 0x7fc1b0070360 2024-03-29T08:11:54.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:54.994+0000 7fc1a7e00700 1 -- 172.21.15.37:0/1062484703 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc1a000d9c0 con 0x7fc1b0070360 2024-03-29T08:11:54.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:54.994+0000 7fc1b6a00700 1 -- 172.21.15.37:0/1062484703 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1b0110400 con 0x7fc1b0070360 2024-03-29T08:11:54.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:54.995+0000 7fc1b6a00700 1 -- 172.21.15.37:0/1062484703 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1b0071aa0 con 0x7fc1b0070360 2024-03-29T08:11:54.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:54.996+0000 7fc1a7e00700 1 -- 172.21.15.37:0/1062484703 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 34) v1 ==== 90703+0+0 (secure 0 0 0) 0x7fc1a000db60 con 0x7fc1b0070360 2024-03-29T08:11:54.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:54.997+0000 7fc1a7e00700 1 --2- 172.21.15.37:0/1062484703 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7fc19406c8b0 0x7fc19406ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:11:54.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:54.997+0000 7fc1a7e00700 1 -- 172.21.15.37:0/1062484703 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6066+0+0 (secure 0 0 0) 0x7fc1a008b130 con 0x7fc1b0070360 2024-03-29T08:11:54.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:54.997+0000 7fc1af400700 1 --2- 172.21.15.37:0/1062484703 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7fc19406c8b0 0x7fc19406ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:11:54.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:54.998+0000 7fc1af400700 1 --2- 172.21.15.37:0/1062484703 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7fc19406c8b0 0x7fc19406ed60 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fc19c00b370 tx=0x7fc19c009e30 comp rx=0 tx=0).ready entity=mgr.14686 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:11:55.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:55.001+0000 7fc1a7e00700 1 -- 172.21.15.37:0/1062484703 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181579 (secure 0 0 0) 0x7fc1a00590a0 con 0x7fc1b0070360 2024-03-29T08:11:55.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:11:54 smithi148 ceph-mon[44992]: from='client.14870 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:11:55.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:11:54 smithi148 ceph-mon[44992]: from='client.14874 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:11:55.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:11:54 smithi148 ceph-mon[44992]: from='client.? 172.21.15.37:0/2400999327' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:11:55.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:11:54 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' 2024-03-29T08:11:55.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:11:54 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-29T08:11:55.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:11:54 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-29T08:11:55.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:11:54 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T08:11:55.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:55.242+0000 7fc1b6a00700 1 -- 172.21.15.37:0/1062484703 --> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc1b004bf40 con 0x7fc19406c8b0 2024-03-29T08:11:55.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:55.244+0000 7fc1a7e00700 1 -- 172.21.15.37:0/1062484703 <== mgr.14686 v2:172.21.15.37:6800/4188329516 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+368 (secure 0 0 0) 0x7fc1b004bf40 con 0x7fc19406c8b0 2024-03-29T08:11:55.244 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-29T08:11:55.244 INFO:teuthology.orchestra.run.smithi037.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59", 2024-03-29T08:11:55.244 INFO:teuthology.orchestra.run.smithi037.stdout: "in_progress": true, 2024-03-29T08:11:55.244 INFO:teuthology.orchestra.run.smithi037.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-29T08:11:55.244 INFO:teuthology.orchestra.run.smithi037.stdout: "services_complete": [ 2024-03-29T08:11:55.244 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr" 2024-03-29T08:11:55.244 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-29T08:11:55.244 INFO:teuthology.orchestra.run.smithi037.stdout: "progress": "2/21 daemons upgraded", 2024-03-29T08:11:55.244 INFO:teuthology.orchestra.run.smithi037.stdout: "message": "Currently upgrading mon daemons", 2024-03-29T08:11:55.244 INFO:teuthology.orchestra.run.smithi037.stdout: "is_paused": false 2024-03-29T08:11:55.244 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-29T08:11:55.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:55.246+0000 7fc1b6a00700 1 -- 172.21.15.37:0/1062484703 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7fc19406c8b0 msgr2=0x7fc19406ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:11:55.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:55.246+0000 7fc1b6a00700 1 --2- 172.21.15.37:0/1062484703 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7fc19406c8b0 0x7fc19406ed60 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fc19c00b370 tx=0x7fc19c009e30 comp rx=0 tx=0).stop 2024-03-29T08:11:55.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:55.246+0000 7fc1b6a00700 1 -- 172.21.15.37:0/1062484703 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc1b0070360 msgr2=0x7fc1b0115cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:11:55.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:55.246+0000 7fc1b6a00700 1 --2- 172.21.15.37:0/1062484703 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc1b0070360 0x7fc1b0115cb0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fc1a0002a50 tx=0x7fc1a00046b0 comp rx=0 tx=0).stop 2024-03-29T08:11:55.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:55.246+0000 7fc1b6a00700 1 -- 172.21.15.37:0/1062484703 shutdown_connections 2024-03-29T08:11:55.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:55.246+0000 7fc1b6a00700 1 --2- 172.21.15.37:0/1062484703 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7fc19406c8b0 0x7fc19406ed60 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:11:55.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:55.246+0000 7fc1b6a00700 1 --2- 172.21.15.37:0/1062484703 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc1b0070360 0x7fc1b0115cb0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:11:55.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:55.246+0000 7fc1b6a00700 1 --2- 172.21.15.37:0/1062484703 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc1b0070c70 0x7fc1b01161f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:11:55.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:55.246+0000 7fc1b6a00700 1 -- 172.21.15.37:0/1062484703 >> 172.21.15.37:0/1062484703 conn(0x7fc1b00f89b0 msgr2=0x7fc1b00f9eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:11:55.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:55.246+0000 7fc1b6a00700 1 -- 172.21.15.37:0/1062484703 shutdown_connections 2024-03-29T08:11:55.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:55.246+0000 7fc1b6a00700 1 -- 172.21.15.37:0/1062484703 wait complete. 2024-03-29T08:11:55.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:55.385+0000 7f97b4e00700 1 -- 172.21.15.37:0/3776936480 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f97b01037a0 msgr2=0x7f97b00fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:11:55.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:55.385+0000 7f97b4e00700 1 --2- 172.21.15.37:0/3776936480 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f97b01037a0 0x7f97b00fce10 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f979c004940 tx=0x7f979c009e30 comp rx=0 tx=0).stop 2024-03-29T08:11:55.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:55.385+0000 7f97b4e00700 1 -- 172.21.15.37:0/3776936480 shutdown_connections 2024-03-29T08:11:55.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:55.385+0000 7f97b4e00700 1 --2- 172.21.15.37:0/3776936480 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f97b01037a0 0x7f97b00fce10 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:11:55.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:55.385+0000 7f97b4e00700 1 --2- 172.21.15.37:0/3776936480 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f97b0102e00 0x7f97b01031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:11:55.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:55.385+0000 7f97b4e00700 1 -- 172.21.15.37:0/3776936480 >> 172.21.15.37:0/3776936480 conn(0x7f97b00f89b0 msgr2=0x7f97b00fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:11:55.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:55.385+0000 7f97b4e00700 1 -- 172.21.15.37:0/3776936480 shutdown_connections 2024-03-29T08:11:55.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:55.385+0000 7f97b4e00700 1 -- 172.21.15.37:0/3776936480 wait complete. 2024-03-29T08:11:55.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:55.386+0000 7f97b4e00700 1 Processor -- start 2024-03-29T08:11:55.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:55.386+0000 7f97b4e00700 1 -- start start 2024-03-29T08:11:55.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:55.387+0000 7f97b4e00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f97b0102e00 0x7f97b0115cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:11:55.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:55.387+0000 7f97b4e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f97b01037a0 0x7f97b01161f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:11:55.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:55.387+0000 7f97b4e00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97b01168d0 con 0x7f97b01037a0 2024-03-29T08:11:55.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:55.387+0000 7f97b4e00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97b010fc10 con 0x7f97b0102e00 2024-03-29T08:11:55.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:55.387+0000 7f97ad400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f97b01037a0 0x7f97b01161f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:11:55.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:55.387+0000 7f97ad400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f97b01037a0 0x7f97b01161f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54506/0 (socket says 172.21.15.37:54506) 2024-03-29T08:11:55.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:55.387+0000 7f97ad400700 1 -- 172.21.15.37:0/1318036794 learned_addr learned my addr 172.21.15.37:0/1318036794 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:11:55.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:55.387+0000 7f97ade00700 1 --2- 172.21.15.37:0/1318036794 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f97b0102e00 0x7f97b0115cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:11:55.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:55.388+0000 7f97ad400700 1 -- 172.21.15.37:0/1318036794 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f97b0102e00 msgr2=0x7f97b0115cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:11:55.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:55.388+0000 7f97ad400700 1 --2- 172.21.15.37:0/1318036794 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f97b0102e00 0x7f97b0115cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:11:55.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:55.388+0000 7f97ad400700 1 -- 172.21.15.37:0/1318036794 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f979c005040 con 0x7f97b01037a0 2024-03-29T08:11:55.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:55.388+0000 7f97ad400700 1 --2- 172.21.15.37:0/1318036794 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f97b01037a0 0x7f97b01161f0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f979c009390 tx=0x7f979c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:11:55.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:55.388+0000 7f97a6000700 1 -- 172.21.15.37:0/1318036794 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f979c00e040 con 0x7f97b01037a0 2024-03-29T08:11:55.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:55.388+0000 7f97a6000700 1 -- 172.21.15.37:0/1318036794 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f979c013420 con 0x7f97b01037a0 2024-03-29T08:11:55.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:55.388+0000 7f97b4e00700 1 -- 172.21.15.37:0/1318036794 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97b010fe30 con 0x7f97b01037a0 2024-03-29T08:11:55.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:55.388+0000 7f97a6000700 1 -- 172.21.15.37:0/1318036794 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f979c01c630 con 0x7f97b01037a0 2024-03-29T08:11:55.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:55.388+0000 7f97b4e00700 1 -- 172.21.15.37:0/1318036794 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97b01103f0 con 0x7f97b01037a0 2024-03-29T08:11:55.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:55.390+0000 7f97b4e00700 1 -- 172.21.15.37:0/1318036794 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97b00fe550 con 0x7f97b01037a0 2024-03-29T08:11:55.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:55.390+0000 7f97a6000700 1 -- 172.21.15.37:0/1318036794 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 34) v1 ==== 90703+0+0 (secure 0 0 0) 0x7f979c013a00 con 0x7f97b01037a0 2024-03-29T08:11:55.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:55.391+0000 7f97a6000700 1 --2- 172.21.15.37:0/1318036794 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7f979406c8b0 0x7f979406ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:11:55.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:55.391+0000 7f97a6000700 1 -- 172.21.15.37:0/1318036794 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f979c08cc20 con 0x7f97b01037a0 2024-03-29T08:11:55.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:55.392+0000 7f97ade00700 1 --2- 172.21.15.37:0/1318036794 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7f979406c8b0 0x7f979406ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:11:55.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:55.392+0000 7f97ade00700 1 --2- 172.21.15.37:0/1318036794 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7f979406c8b0 0x7f979406ed60 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f97a0009bf0 tx=0x7f97a00091a0 comp rx=0 tx=0).ready entity=mgr.14686 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:11:55.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:55.397+0000 7f97a6000700 1 -- 172.21.15.37:0/1318036794 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181579 (secure 0 0 0) 0x7f979c05ab70 con 0x7f97b01037a0 2024-03-29T08:11:55.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:55.706+0000 7f97b4e00700 1 -- 172.21.15.37:0/1318036794 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f97b005f5f0 con 0x7f97b01037a0 2024-03-29T08:11:55.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:55.706+0000 7f97a6000700 1 -- 172.21.15.37:0/1318036794 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f979c05a700 con 0x7f97b01037a0 2024-03-29T08:11:55.707 INFO:teuthology.orchestra.run.smithi037.stdout:HEALTH_OK 2024-03-29T08:11:55.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:55.709+0000 7f97b4e00700 1 -- 172.21.15.37:0/1318036794 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7f979406c8b0 msgr2=0x7f979406ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:11:55.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:55.709+0000 7f97b4e00700 1 --2- 172.21.15.37:0/1318036794 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7f979406c8b0 0x7f979406ed60 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f97a0009bf0 tx=0x7f97a00091a0 comp rx=0 tx=0).stop 2024-03-29T08:11:55.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:55.709+0000 7f97b4e00700 1 -- 172.21.15.37:0/1318036794 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f97b01037a0 msgr2=0x7f97b01161f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:11:55.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:55.709+0000 7f97b4e00700 1 --2- 172.21.15.37:0/1318036794 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f97b01037a0 0x7f97b01161f0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f979c009390 tx=0x7f979c009e30 comp rx=0 tx=0).stop 2024-03-29T08:11:55.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:55.709+0000 7f97b4e00700 1 -- 172.21.15.37:0/1318036794 shutdown_connections 2024-03-29T08:11:55.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:55.709+0000 7f97b4e00700 1 --2- 172.21.15.37:0/1318036794 >> [v2:172.21.15.37:6800/4188329516,v1:172.21.15.37:6801/4188329516] conn(0x7f979406c8b0 0x7f979406ed60 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:11:55.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:55.709+0000 7f97b4e00700 1 --2- 172.21.15.37:0/1318036794 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f97b0102e00 0x7f97b0115cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:11:55.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:55.709+0000 7f97b4e00700 1 --2- 172.21.15.37:0/1318036794 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f97b01037a0 0x7f97b01161f0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:11:55.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:55.709+0000 7f97b4e00700 1 -- 172.21.15.37:0/1318036794 >> 172.21.15.37:0/1318036794 conn(0x7f97b00f89b0 msgr2=0x7f97b00f9910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:11:55.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:55.709+0000 7f97b4e00700 1 -- 172.21.15.37:0/1318036794 shutdown_connections 2024-03-29T08:11:55.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:11:55.709+0000 7f97b4e00700 1 -- 172.21.15.37:0/1318036794 wait complete. 2024-03-29T08:11:55.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:55 smithi037 ceph-mon[35162]: Upgrade: Updating mon.smithi037 2024-03-29T08:11:55.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:55 smithi037 ceph-mon[35162]: Deploying daemon mon.smithi037 on smithi037 2024-03-29T08:11:55.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:55 smithi037 ceph-mon[35162]: pgmap v117: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:11:55.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:55 smithi037 ceph-mon[35162]: from='client.? 172.21.15.37:0/3581205756' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T08:11:56.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:11:55 smithi148 ceph-mon[44992]: Upgrade: Updating mon.smithi037 2024-03-29T08:11:56.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:11:55 smithi148 ceph-mon[44992]: Deploying daemon mon.smithi037 on smithi037 2024-03-29T08:11:56.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:11:55 smithi148 ceph-mon[44992]: pgmap v117: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:11:56.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:11:55 smithi148 ceph-mon[44992]: from='client.? 172.21.15.37:0/3581205756' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T08:11:56.881 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:56 smithi037 systemd[1]: Stopping Ceph mon.smithi037 for 99b492c2-eda1-11ee-b647-cb9ed24678a4... 2024-03-29T08:11:56.881 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:56 smithi037 ceph-mon[35162]: from='client.14886 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:11:56.881 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:56 smithi037 ceph-mon[35162]: from='client.? 172.21.15.37:0/1318036794' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T08:11:57.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:11:56 smithi148 ceph-mon[44992]: from='client.14886 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:11:57.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:11:56 smithi148 ceph-mon[44992]: from='client.? 172.21.15.37:0/1318036794' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T08:11:57.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:56 smithi037 ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4-mon-smithi037[35139]: 2024-03-29T08:11:56.879+0000 7f3e96400700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi037 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-29T08:11:57.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:56 smithi037 ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4-mon-smithi037[35139]: 2024-03-29T08:11:56.879+0000 7f3e96400700 -1 mon.smithi037@0(leader) e2 *** Got Signal Terminated *** 2024-03-29T08:11:57.616 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:57 smithi037 bash[142115]: ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4-mon-smithi037 2024-03-29T08:11:58.296 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:57 smithi037 systemd[1]: ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4@mon.smithi037.service: Succeeded. 2024-03-29T08:11:58.296 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:57 smithi037 systemd[1]: Stopped Ceph mon.smithi037 for 99b492c2-eda1-11ee-b647-cb9ed24678a4. 2024-03-29T08:11:58.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 systemd[1]: Starting Ceph mon.smithi037 for 99b492c2-eda1-11ee-b647-cb9ed24678a4... 2024-03-29T08:11:59.428 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: set uid:gid to 167:167 (ceph:ceph) 2024-03-29T08:11:59.428 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable), process ceph-mon, pid 7 2024-03-29T08:11:59.429 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: pidfile_write: ignore empty --pid-file 2024-03-29T08:11:59.429 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: load: jerasure load: lrc 2024-03-29T08:11:59.429 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: RocksDB version: 7.9.2 2024-03-29T08:11:59.429 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: Git sha 0 2024-03-29T08:11:59.429 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: Compile date 2024-03-26 15:09:52 2024-03-29T08:11:59.429 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: DB SUMMARY 2024-03-29T08:11:59.429 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: DB Session ID: YTUZZOHRB1HX2J0LEPI9 2024-03-29T08:11:59.429 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: CURRENT file: CURRENT 2024-03-29T08:11:59.429 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: IDENTITY file: IDENTITY 2024-03-29T08:11:59.429 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: MANIFEST file: MANIFEST-000015 size: 1147 Bytes 2024-03-29T08:11:59.429 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi037/store.db dir, Total Num: 1, files: 000033.sst 2024-03-29T08:11:59.429 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi037/store.db: 000031.log size: 3653699 ; 2024-03-29T08:11:59.430 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: Options.error_if_exists: 0 2024-03-29T08:11:59.430 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: Options.create_if_missing: 0 2024-03-29T08:11:59.430 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: Options.paranoid_checks: 1 2024-03-29T08:11:59.430 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: Options.flush_verify_memtable_count: 1 2024-03-29T08:11:59.430 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-03-29T08:11:59.430 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-03-29T08:11:59.430 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: Options.env: 0x55e9d681a720 2024-03-29T08:11:59.430 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: Options.fs: PosixFileSystem 2024-03-29T08:11:59.430 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: Options.info_log: 0x55e9d81eb3c0 2024-03-29T08:11:59.430 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: Options.max_file_opening_threads: 16 2024-03-29T08:11:59.430 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: Options.statistics: (nil) 2024-03-29T08:11:59.430 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: Options.use_fsync: 0 2024-03-29T08:11:59.430 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: Options.max_log_file_size: 0 2024-03-29T08:11:59.430 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-03-29T08:11:59.430 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: Options.log_file_time_to_roll: 0 2024-03-29T08:11:59.431 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: Options.keep_log_file_num: 1000 2024-03-29T08:11:59.431 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: Options.recycle_log_file_num: 0 2024-03-29T08:11:59.431 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: Options.allow_fallocate: 1 2024-03-29T08:11:59.431 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: Options.allow_mmap_reads: 0 2024-03-29T08:11:59.431 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: Options.allow_mmap_writes: 0 2024-03-29T08:11:59.431 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: Options.use_direct_reads: 0 2024-03-29T08:11:59.431 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-03-29T08:11:59.431 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: Options.create_missing_column_families: 0 2024-03-29T08:11:59.431 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: Options.db_log_dir: 2024-03-29T08:11:59.431 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: Options.wal_dir: 2024-03-29T08:11:59.431 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: Options.table_cache_numshardbits: 6 2024-03-29T08:11:59.431 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: Options.WAL_ttl_seconds: 0 2024-03-29T08:11:59.431 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: Options.WAL_size_limit_MB: 0 2024-03-29T08:11:59.431 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-03-29T08:11:59.431 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-03-29T08:11:59.432 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: Options.is_fd_close_on_exec: 1 2024-03-29T08:11:59.432 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: Options.advise_random_on_open: 1 2024-03-29T08:11:59.432 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: Options.db_write_buffer_size: 0 2024-03-29T08:11:59.432 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: Options.write_buffer_manager: 0x55e9d7462140 2024-03-29T08:11:59.432 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-03-29T08:11:59.432 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-03-29T08:11:59.432 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: Options.use_adaptive_mutex: 0 2024-03-29T08:11:59.432 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: Options.rate_limiter: (nil) 2024-03-29T08:11:59.432 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-03-29T08:11:59.432 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: Options.wal_recovery_mode: 2 2024-03-29T08:11:59.432 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: Options.enable_thread_tracking: 0 2024-03-29T08:11:59.432 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: Options.enable_pipelined_write: 0 2024-03-29T08:11:59.432 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: Options.unordered_write: 0 2024-03-29T08:11:59.432 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-03-29T08:11:59.432 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-03-29T08:11:59.433 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-03-29T08:11:59.433 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-03-29T08:11:59.433 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: Options.row_cache: None 2024-03-29T08:11:59.433 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: Options.wal_filter: None 2024-03-29T08:11:59.433 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-03-29T08:11:59.433 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: Options.allow_ingest_behind: 0 2024-03-29T08:11:59.433 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: Options.two_write_queues: 0 2024-03-29T08:11:59.433 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: Options.manual_wal_flush: 0 2024-03-29T08:11:59.433 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: Options.wal_compression: 0 2024-03-29T08:11:59.433 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: Options.atomic_flush: 0 2024-03-29T08:11:59.433 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-03-29T08:11:59.433 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: Options.persist_stats_to_disk: 0 2024-03-29T08:11:59.433 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: Options.write_dbid_to_manifest: 0 2024-03-29T08:11:59.433 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: Options.log_readahead_size: 0 2024-03-29T08:11:59.433 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-03-29T08:11:59.434 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: Options.best_efforts_recovery: 0 2024-03-29T08:11:59.434 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-03-29T08:11:59.434 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-03-29T08:11:59.434 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: Options.allow_data_in_errors: 0 2024-03-29T08:11:59.434 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: Options.db_host_id: __hostname__ 2024-03-29T08:11:59.434 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: Options.enforce_single_del_contracts: true 2024-03-29T08:11:59.434 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: Options.max_background_jobs: 2 2024-03-29T08:11:59.434 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: Options.max_background_compactions: -1 2024-03-29T08:11:59.434 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: Options.max_subcompactions: 1 2024-03-29T08:11:59.434 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-03-29T08:11:59.434 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-03-29T08:11:59.434 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: Options.delayed_write_rate : 16777216 2024-03-29T08:11:59.434 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: Options.max_total_wal_size: 0 2024-03-29T08:11:59.434 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-03-29T08:11:59.434 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: Options.stats_dump_period_sec: 600 2024-03-29T08:11:59.435 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: Options.stats_persist_period_sec: 600 2024-03-29T08:11:59.435 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-03-29T08:11:59.435 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: Options.max_open_files: -1 2024-03-29T08:11:59.435 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: Options.bytes_per_sync: 0 2024-03-29T08:11:59.435 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: Options.wal_bytes_per_sync: 0 2024-03-29T08:11:59.435 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: Options.strict_bytes_per_sync: 0 2024-03-29T08:11:59.435 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: Options.compaction_readahead_size: 0 2024-03-29T08:11:59.435 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: Options.max_background_flushes: -1 2024-03-29T08:11:59.435 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: Compression algorithms supported: 2024-03-29T08:11:59.435 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-03-29T08:11:59.436 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: kZSTD supported: 0 2024-03-29T08:11:59.436 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: kXpressCompression supported: 0 2024-03-29T08:11:59.436 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: kLZ4HCCompression supported: 1 2024-03-29T08:11:59.436 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: kZlibCompression supported: 1 2024-03-29T08:11:59.436 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: kSnappyCompression supported: 1 2024-03-29T08:11:59.436 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: kLZ4Compression supported: 1 2024-03-29T08:11:59.436 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: kBZip2Compression supported: 0 2024-03-29T08:11:59.436 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: Fast CRC32 supported: Supported on x86 2024-03-29T08:11:59.436 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: DMutex implementation: pthread_mutex_t 2024-03-29T08:11:59.436 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi037/store.db/MANIFEST-000015 2024-03-29T08:11:59.436 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-03-29T08:11:59.436 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-03-29T08:11:59.437 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: Options.merge_operator: 2024-03-29T08:11:59.437 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: Options.compaction_filter: None 2024-03-29T08:11:59.437 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: Options.compaction_filter_factory: None 2024-03-29T08:11:59.437 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: Options.sst_partitioner_factory: None 2024-03-29T08:11:59.437 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: Options.memtable_factory: SkipListFactory 2024-03-29T08:11:59.437 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: Options.table_factory: BlockBasedTable 2024-03-29T08:11:59.437 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55e9d81eb4e0) 2024-03-29T08:11:59.437 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: cache_index_and_filter_blocks: 1 2024-03-29T08:11:59.437 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-03-29T08:11:59.437 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-03-29T08:11:59.437 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: pin_top_level_index_and_filter: 1 2024-03-29T08:11:59.437 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: index_type: 0 2024-03-29T08:11:59.437 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: data_block_index_type: 0 2024-03-29T08:11:59.437 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: index_shortening: 1 2024-03-29T08:11:59.437 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: data_block_hash_table_util_ratio: 0.750000 2024-03-29T08:11:59.438 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: checksum: 4 2024-03-29T08:11:59.438 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: no_block_cache: 0 2024-03-29T08:11:59.438 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: block_cache: 0x55e9d7503350 2024-03-29T08:11:59.438 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: block_cache_name: BinnedLRUCache 2024-03-29T08:11:59.438 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: block_cache_options: 2024-03-29T08:11:59.438 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: capacity : 536870912 2024-03-29T08:11:59.438 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: num_shard_bits : 4 2024-03-29T08:11:59.438 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: strict_capacity_limit : 0 2024-03-29T08:11:59.438 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: high_pri_pool_ratio: 0.000 2024-03-29T08:11:59.438 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: block_cache_compressed: (nil) 2024-03-29T08:11:59.438 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: persistent_cache: (nil) 2024-03-29T08:11:59.438 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: block_size: 4096 2024-03-29T08:11:59.438 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: block_size_deviation: 10 2024-03-29T08:11:59.438 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: block_restart_interval: 16 2024-03-29T08:11:59.439 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: index_block_restart_interval: 1 2024-03-29T08:11:59.439 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: metadata_block_size: 4096 2024-03-29T08:11:59.439 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: partition_filters: 0 2024-03-29T08:11:59.439 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: use_delta_encoding: 1 2024-03-29T08:11:59.439 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: filter_policy: bloomfilter 2024-03-29T08:11:59.439 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: whole_key_filtering: 1 2024-03-29T08:11:59.439 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: verify_compression: 0 2024-03-29T08:11:59.439 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: read_amp_bytes_per_bit: 0 2024-03-29T08:11:59.439 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: format_version: 5 2024-03-29T08:11:59.439 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: enable_index_compression: 1 2024-03-29T08:11:59.439 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: block_align: 0 2024-03-29T08:11:59.439 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: max_auto_readahead_size: 262144 2024-03-29T08:11:59.439 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: prepopulate_block_cache: 0 2024-03-29T08:11:59.439 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: initial_auto_readahead_size: 8192 2024-03-29T08:11:59.439 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: num_file_reads_for_auto_readahead: 2 2024-03-29T08:11:59.439 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: Options.write_buffer_size: 33554432 2024-03-29T08:11:59.440 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: Options.max_write_buffer_number: 2 2024-03-29T08:11:59.440 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: Options.compression: NoCompression 2024-03-29T08:11:59.440 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: Options.bottommost_compression: Disabled 2024-03-29T08:11:59.440 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: Options.prefix_extractor: nullptr 2024-03-29T08:11:59.440 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-03-29T08:11:59.440 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: Options.num_levels: 7 2024-03-29T08:11:59.440 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-03-29T08:11:59.440 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-03-29T08:11:59.440 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-03-29T08:11:59.440 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-03-29T08:11:59.440 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-03-29T08:11:59.440 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-03-29T08:11:59.440 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-03-29T08:11:59.440 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-03-29T08:11:59.440 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-03-29T08:11:59.440 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-03-29T08:11:59.441 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-03-29T08:11:59.441 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-03-29T08:11:59.441 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: Options.compression_opts.window_bits: -14 2024-03-29T08:11:59.441 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: Options.compression_opts.level: 32767 2024-03-29T08:11:59.441 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: Options.compression_opts.strategy: 0 2024-03-29T08:11:59.441 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-03-29T08:11:59.441 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-03-29T08:11:59.441 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-03-29T08:11:59.441 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-03-29T08:11:59.441 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: Options.compression_opts.enabled: false 2024-03-29T08:11:59.441 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-03-29T08:11:59.441 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-03-29T08:11:59.441 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-03-29T08:11:59.441 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-03-29T08:11:59.442 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: Options.target_file_size_base: 67108864 2024-03-29T08:11:59.442 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: Options.target_file_size_multiplier: 1 2024-03-29T08:11:59.442 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-03-29T08:11:59.442 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-03-29T08:11:59.442 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-03-29T08:11:59.442 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-03-29T08:11:59.442 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-03-29T08:11:59.442 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-03-29T08:11:59.442 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-03-29T08:11:59.442 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-03-29T08:11:59.442 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-03-29T08:11:59.442 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-03-29T08:11:59.442 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-03-29T08:11:59.442 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-03-29T08:11:59.442 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-03-29T08:11:59.442 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: Options.arena_block_size: 1048576 2024-03-29T08:11:59.443 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-03-29T08:11:59.443 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-03-29T08:11:59.443 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: Options.disable_auto_compactions: 0 2024-03-29T08:11:59.443 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-03-29T08:11:59.443 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-03-29T08:11:59.443 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-03-29T08:11:59.443 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-03-29T08:11:59.443 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-03-29T08:11:59.443 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-03-29T08:11:59.443 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-03-29T08:11:59.443 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-03-29T08:11:59.443 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-03-29T08:11:59.443 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-03-29T08:11:59.443 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: Options.table_properties_collectors: 2024-03-29T08:11:59.443 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: Options.inplace_update_support: 0 2024-03-29T08:11:59.444 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: Options.inplace_update_num_locks: 10000 2024-03-29T08:11:59.444 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-03-29T08:11:59.444 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-03-29T08:11:59.444 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: Options.memtable_huge_page_size: 0 2024-03-29T08:11:59.444 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: Options.bloom_locality: 0 2024-03-29T08:11:59.444 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: Options.max_successive_merges: 0 2024-03-29T08:11:59.444 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: Options.optimize_filters_for_hits: 0 2024-03-29T08:11:59.444 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: Options.paranoid_file_checks: 0 2024-03-29T08:11:59.444 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: Options.force_consistency_checks: 1 2024-03-29T08:11:59.444 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: Options.report_bg_io_stats: 0 2024-03-29T08:11:59.444 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: Options.ttl: 2592000 2024-03-29T08:11:59.444 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: Options.periodic_compaction_seconds: 0 2024-03-29T08:11:59.444 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-03-29T08:11:59.444 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-03-29T08:11:59.444 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: Options.enable_blob_files: false 2024-03-29T08:11:59.444 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: Options.min_blob_size: 0 2024-03-29T08:11:59.445 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: Options.blob_file_size: 268435456 2024-03-29T08:11:59.445 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: Options.blob_compression_type: NoCompression 2024-03-29T08:11:59.445 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: Options.enable_blob_garbage_collection: false 2024-03-29T08:11:59.445 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-03-29T08:11:59.445 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-03-29T08:11:59.445 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-03-29T08:11:59.445 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: Options.blob_file_starting_level: 0 2024-03-29T08:11:59.445 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-03-29T08:11:59.445 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: [table/block_based/block_based_table_reader.cc:721] At least one SST file opened without unique ID to verify: 33.sst 2024-03-29T08:11:59.445 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-03-29T08:11:59.446 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi037/store.db/MANIFEST-000015 succeeded,manifest_file_number is 15, next_file_number is 35, last_sequence is 11850, log_number is 31,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-03-29T08:11:59.446 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 31 2024-03-29T08:11:59.446 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 165e6a94-8278-4f08-bdc9-418bb7516d4f 2024-03-29T08:11:59.446 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711699918994728, "job": 1, "event": "recovery_started", "wal_files": [31]} 2024-03-29T08:11:59.446 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:58 smithi037 ceph-mon[142325]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #31 mode 2 2024-03-29T08:11:59.446 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:59 smithi037 ceph-mon[142325]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711699919011031, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 36, "file_size": 3215334, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 11851, "largest_seqno": 14969, "table_properties": {"data_size": 3202347, "index_size": 8190, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 3781, "raw_key_size": 32965, "raw_average_key_size": 22, "raw_value_size": 3172851, "raw_average_value_size": 2133, "num_data_blocks": 372, "num_entries": 1487, "num_filter_entries": 1487, "num_deletions": 260, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1711699919, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "165e6a94-8278-4f08-bdc9-418bb7516d4f", "db_session_id": "YTUZZOHRB1HX2J0LEPI9", "orig_file_number": 36, "seqno_to_time_mapping": "N/A"}} 2024-03-29T08:11:59.446 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:59 smithi037 ceph-mon[142325]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711699919011225, "job": 1, "event": "recovery_finished"} 2024-03-29T08:11:59.447 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:59 smithi037 ceph-mon[142325]: rocksdb: [db/version_set.cc:5047] Creating manifest 38 2024-03-29T08:11:59.447 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:59 smithi037 ceph-mon[142325]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-03-29T08:11:59.447 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:59 smithi037 ceph-mon[142325]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-smithi037/store.db/000031.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-03-29T08:11:59.447 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:59 smithi037 ceph-mon[142325]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55e9d75a0000 2024-03-29T08:11:59.447 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:59 smithi037 ceph-mon[142325]: rocksdb: DB pointer 0x55e9d758e000 2024-03-29T08:11:59.447 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:59 smithi037 ceph-mon[142325]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-03-29T08:11:59.447 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:59 smithi037 ceph-mon[142325]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-03-29T08:11:59.447 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: ** DB Stats ** 2024-03-29T08:11:59.447 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-29T08:11:59.448 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-03-29T08:11:59.448 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-29T08:11:59.448 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-29T08:11:59.448 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-03-29T08:11:59.448 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-29T08:11:59.448 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-29T08:11:59.448 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: 2024-03-29T08:11:59.448 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: ** Compaction Stats [default] ** 2024-03-29T08:11:59.448 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-29T08:11:59.448 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-03-29T08:11:59.448 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: L0 1/0 3.07 MB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 476.1 0.01 0.00 1 0.006 0 0 0.0 0.0 2024-03-29T08:11:59.448 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: L6 1/0 6.43 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 2024-03-29T08:11:59.448 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: Sum 2/0 9.50 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 476.1 0.01 0.00 1 0.006 0 0 0.0 0.0 2024-03-29T08:11:59.448 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 476.1 0.01 0.00 1 0.006 0 0 0.0 0.0 2024-03-29T08:11:59.449 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: 2024-03-29T08:11:59.449 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: ** Compaction Stats [default] ** 2024-03-29T08:11:59.449 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-29T08:11:59.449 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-29T08:11:59.449 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 476.1 0.01 0.00 1 0.006 0 0 0.0 0.0 2024-03-29T08:11:59.449 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: 2024-03-29T08:11:59.449 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-03-29T08:11:59.449 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: 2024-03-29T08:11:59.449 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-29T08:11:59.449 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: Flush(GB): cumulative 0.003, interval 0.003 2024-03-29T08:11:59.449 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-03-29T08:11:59.450 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-03-29T08:11:59.450 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-03-29T08:11:59.450 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: AddFile(Keys): cumulative 0, interval 0 2024-03-29T08:11:59.450 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: Cumulative compaction: 0.00 GB write, 120.96 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-29T08:11:59.450 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: Interval compaction: 0.00 GB write, 120.96 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-29T08:11:59.450 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-03-29T08:11:59.450 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: Block cache BinnedLRUCache@0x55e9d7503350#7 capacity: 512.00 MB usage: 12.11 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.2e-05 secs_since: 0 2024-03-29T08:11:59.450 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,4.03 KB,0.0007689%) IndexBlock(1,8.08 KB,0.00154078%) Misc(1,0.00 KB,0%) 2024-03-29T08:11:59.450 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: 2024-03-29T08:11:59.450 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: ** File Read Latency Histogram By Level [default] ** 2024-03-29T08:11:59.450 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:59 smithi037 ceph-mon[142325]: starting mon.smithi037 rank 0 at public addrs [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] at bind addrs [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi037 fsid 99b492c2-eda1-11ee-b647-cb9ed24678a4 2024-03-29T08:11:59.451 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:59 smithi037 ceph-mon[142325]: mon.smithi037@-1(???) e2 preinit fsid 99b492c2-eda1-11ee-b647-cb9ed24678a4 2024-03-29T08:11:59.451 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:59 smithi037 ceph-mon[142325]: mon.smithi037@-1(???).mds e20 new map 2024-03-29T08:11:59.451 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:59 smithi037 ceph-mon[142325]: mon.smithi037@-1(???).mds e20 print_map 2024-03-29T08:11:59.451 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: e20 2024-03-29T08:11:59.451 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-03-29T08:11:59.451 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T08:11:59.451 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: legacy client fscid: 1 2024-03-29T08:11:59.451 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: 2024-03-29T08:11:59.451 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: Filesystem 'cephfs' (1) 2024-03-29T08:11:59.451 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: fs_name cephfs 2024-03-29T08:11:59.451 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: epoch 20 2024-03-29T08:11:59.452 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: flags 12 joinable allow_snaps allow_multimds_snaps 2024-03-29T08:11:59.452 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: created 2024-03-29T08:03:29.241269+0000 2024-03-29T08:11:59.452 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: modified 2024-03-29T08:11:27.181622+0000 2024-03-29T08:11:59.452 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: tableserver 0 2024-03-29T08:11:59.452 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: root 0 2024-03-29T08:11:59.452 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: session_timeout 60 2024-03-29T08:11:59.452 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: session_autoclose 300 2024-03-29T08:11:59.452 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: max_file_size 1099511627776 2024-03-29T08:11:59.452 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: max_xattr_size 65536 2024-03-29T08:11:59.452 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: required_client_features {} 2024-03-29T08:11:59.452 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: last_failure 0 2024-03-29T08:11:59.452 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: last_failure_osd_epoch 47 2024-03-29T08:11:59.453 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T08:11:59.453 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: max_mds 2 2024-03-29T08:11:59.453 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: in 0,1 2024-03-29T08:11:59.453 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: up {0=14470,1=14482} 2024-03-29T08:11:59.453 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: failed 2024-03-29T08:11:59.453 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: damaged 2024-03-29T08:11:59.453 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: stopped 2024-03-29T08:11:59.453 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: data_pools [3] 2024-03-29T08:11:59.453 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: metadata_pool 2 2024-03-29T08:11:59.453 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: inline_data disabled 2024-03-29T08:11:59.453 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: balancer 2024-03-29T08:11:59.454 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: bal_rank_mask -1 2024-03-29T08:11:59.454 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: standby_count_wanted 1 2024-03-29T08:11:59.454 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: [mds.cephfs.smithi037.mmwgxe{0:14470} state up:active seq 5 join_fscid=1 addr [v2:172.21.15.37:6826/1559618909,v1:172.21.15.37:6827/1559618909] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:11:59.454 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: [mds.cephfs.smithi037.iifsmh{1:14482} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.37:6828/2341530495,v1:172.21.15.37:6829/2341530495] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:11:59.454 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: 2024-03-29T08:11:59.454 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: 2024-03-29T08:11:59.454 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: Standby daemons: 2024-03-29T08:11:59.454 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: 2024-03-29T08:11:59.454 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: [mds.cephfs.smithi148.qukpjh{-1:24319} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.148:6826/2641327912,v1:172.21.15.148:6827/2641327912] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:11:59.454 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout: [mds.cephfs.smithi148.sqteih{-1:24327} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.148:6824/3252351726,v1:172.21.15.148:6825/3252351726] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:11:59.455 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:59 smithi037 ceph-mon[142325]: mon.smithi037@-1(???).osd e49 crush map has features 3314933000852226048, adjusting msgr requires 2024-03-29T08:11:59.455 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:59 smithi037 ceph-mon[142325]: mon.smithi037@-1(???).osd e49 crush map has features 288514051259236352, adjusting msgr requires 2024-03-29T08:11:59.455 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:59 smithi037 ceph-mon[142325]: mon.smithi037@-1(???).osd e49 crush map has features 288514051259236352, adjusting msgr requires 2024-03-29T08:11:59.455 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:59 smithi037 ceph-mon[142325]: mon.smithi037@-1(???).osd e49 crush map has features 288514051259236352, adjusting msgr requires 2024-03-29T08:11:59.455 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:59 smithi037 ceph-mon[142325]: mon.smithi037@-1(???).paxosservice(auth 1..22) refresh upgraded, format 0 -> 3 2024-03-29T08:11:59.455 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:59 smithi037 bash[142287]: eea2060783af0e284b0ebb8a3c8dcee0edc0cb11984c647d845eade030d19ae7 2024-03-29T08:11:59.455 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:11:59 smithi037 systemd[1]: Started Ceph mon.smithi037 for 99b492c2-eda1-11ee-b647-cb9ed24678a4. 2024-03-29T08:12:00.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:00 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "mon metadata", "id": "smithi037"}]: dispatch 2024-03-29T08:12:00.807 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:00 smithi148 ceph-mon[44992]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "mon metadata", "id": "smithi148"}]: dispatch 2024-03-29T08:12:00.807 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:00 smithi148 ceph-mon[44992]: pgmap v119: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:12:00.807 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:00 smithi148 ceph-mon[44992]: mon.smithi037 calling monitor election 2024-03-29T08:12:00.807 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:00 smithi148 ceph-mon[44992]: mon.smithi037 is new leader, mons smithi037,smithi148 in quorum (ranks 0,1) 2024-03-29T08:12:00.807 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:00 smithi148 ceph-mon[44992]: monmap e2: 2 mons at {smithi037=[v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0],smithi148=[v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-29T08:12:00.807 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:00 smithi148 ceph-mon[44992]: fsmap cephfs:2 {0=cephfs.smithi037.mmwgxe=up:active,1=cephfs.smithi037.iifsmh=up:active} 2 up:standby 2024-03-29T08:12:00.807 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:00 smithi148 ceph-mon[44992]: osdmap e49: 6 total, 6 up, 6 in 2024-03-29T08:12:00.807 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:00 smithi148 ceph-mon[44992]: mgrmap e34: smithi037.wjikii(active, since 3m), standbys: smithi148.onhgvm 2024-03-29T08:12:00.807 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:00 smithi148 ceph-mon[44992]: mgrmap e35: smithi037.wjikii(active, since 3m), standbys: smithi148.onhgvm 2024-03-29T08:12:00.807 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:00 smithi148 ceph-mon[44992]: overall HEALTH_OK 2024-03-29T08:12:00.807 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:00 smithi148 ceph-mon[44992]: from='mgr.14686 ' entity='' 2024-03-29T08:12:00.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:00 smithi037 ceph-mon[142325]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "mon metadata", "id": "smithi037"}]: dispatch 2024-03-29T08:12:00.928 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:00 smithi037 ceph-mon[142325]: from='mgr.14686 172.21.15.37:0/2331070643' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "mon metadata", "id": "smithi148"}]: dispatch 2024-03-29T08:12:00.928 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:00 smithi037 ceph-mon[142325]: pgmap v119: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:12:00.928 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:00 smithi037 ceph-mon[142325]: mon.smithi037 calling monitor election 2024-03-29T08:12:00.928 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:00 smithi037 ceph-mon[142325]: mon.smithi037 is new leader, mons smithi037,smithi148 in quorum (ranks 0,1) 2024-03-29T08:12:00.928 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:00 smithi037 ceph-mon[142325]: monmap e2: 2 mons at {smithi037=[v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0],smithi148=[v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-29T08:12:00.928 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:00 smithi037 ceph-mon[142325]: fsmap cephfs:2 {0=cephfs.smithi037.mmwgxe=up:active,1=cephfs.smithi037.iifsmh=up:active} 2 up:standby 2024-03-29T08:12:00.928 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:00 smithi037 ceph-mon[142325]: osdmap e49: 6 total, 6 up, 6 in 2024-03-29T08:12:00.928 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:00 smithi037 ceph-mon[142325]: mgrmap e34: smithi037.wjikii(active, since 3m), standbys: smithi148.onhgvm 2024-03-29T08:12:00.928 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:00 smithi037 ceph-mon[142325]: mgrmap e35: smithi037.wjikii(active, since 3m), standbys: smithi148.onhgvm 2024-03-29T08:12:00.928 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:00 smithi037 ceph-mon[142325]: overall HEALTH_OK 2024-03-29T08:12:00.928 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:00 smithi037 ceph-mon[142325]: from='mgr.14686 ' entity='' 2024-03-29T08:12:10.307 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:10 smithi148 ceph-mon[44992]: Standby manager daemon smithi148.onhgvm restarted 2024-03-29T08:12:10.307 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:10 smithi148 ceph-mon[44992]: Standby manager daemon smithi148.onhgvm started 2024-03-29T08:12:10.307 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:10 smithi148 ceph-mon[44992]: from='mgr.? 172.21.15.148:0/1635681419' entity='mgr.smithi148.onhgvm' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi148.onhgvm/crt"}]: dispatch 2024-03-29T08:12:10.307 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:10 smithi148 ceph-mon[44992]: from='mgr.? 172.21.15.148:0/1635681419' entity='mgr.smithi148.onhgvm' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-29T08:12:10.307 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:10 smithi148 ceph-mon[44992]: from='mgr.? 172.21.15.148:0/1635681419' entity='mgr.smithi148.onhgvm' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi148.onhgvm/key"}]: dispatch 2024-03-29T08:12:10.307 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:10 smithi148 ceph-mon[44992]: from='mgr.? 172.21.15.148:0/1635681419' entity='mgr.smithi148.onhgvm' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-29T08:12:10.307 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:10 smithi148 ceph-mon[44992]: Active manager daemon smithi037.wjikii restarted 2024-03-29T08:12:10.307 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:10 smithi148 ceph-mon[44992]: Activating manager daemon smithi037.wjikii 2024-03-29T08:12:10.307 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:10 smithi148 ceph-mon[44992]: osdmap e50: 6 total, 6 up, 6 in 2024-03-29T08:12:10.308 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:10 smithi148 ceph-mon[44992]: mgrmap e36: smithi037.wjikii(active, starting, since 0.0074783s), standbys: smithi148.onhgvm 2024-03-29T08:12:10.308 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:10 smithi148 ceph-mon[44992]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "mon metadata", "id": "smithi037"}]: dispatch 2024-03-29T08:12:10.308 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:10 smithi148 ceph-mon[44992]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "mon metadata", "id": "smithi148"}]: dispatch 2024-03-29T08:12:10.308 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:10 smithi148 ceph-mon[44992]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi037.mmwgxe"}]: dispatch 2024-03-29T08:12:10.308 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:10 smithi148 ceph-mon[44992]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi037.iifsmh"}]: dispatch 2024-03-29T08:12:10.308 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:10 smithi148 ceph-mon[44992]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi148.qukpjh"}]: dispatch 2024-03-29T08:12:10.308 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:10 smithi148 ceph-mon[44992]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi148.sqteih"}]: dispatch 2024-03-29T08:12:10.308 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:10 smithi148 ceph-mon[44992]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "mgr metadata", "who": "smithi037.wjikii", "id": "smithi037.wjikii"}]: dispatch 2024-03-29T08:12:10.308 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:10 smithi148 ceph-mon[44992]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "mgr metadata", "who": "smithi148.onhgvm", "id": "smithi148.onhgvm"}]: dispatch 2024-03-29T08:12:10.308 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:10 smithi148 ceph-mon[44992]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-29T08:12:10.308 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:10 smithi148 ceph-mon[44992]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-29T08:12:10.308 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:10 smithi148 ceph-mon[44992]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-29T08:12:10.308 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:10 smithi148 ceph-mon[44992]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-29T08:12:10.308 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:10 smithi148 ceph-mon[44992]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-29T08:12:10.308 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:10 smithi148 ceph-mon[44992]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-29T08:12:10.309 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:10 smithi148 ceph-mon[44992]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-29T08:12:10.309 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:10 smithi148 ceph-mon[44992]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-29T08:12:10.309 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:10 smithi148 ceph-mon[44992]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-29T08:12:10.309 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:10 smithi148 ceph-mon[44992]: Manager daemon smithi037.wjikii is now available 2024-03-29T08:12:10.309 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:10 smithi148 ceph-mon[44992]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T08:12:10.309 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:10 smithi148 ceph-mon[44992]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:12:10.309 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:10 smithi148 ceph-mon[44992]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi037.wjikii/mirror_snapshot_schedule"}]: dispatch 2024-03-29T08:12:10.309 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:10 smithi148 ceph-mon[44992]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi037.wjikii/trash_purge_schedule"}]: dispatch 2024-03-29T08:12:10.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:10 smithi037 ceph-mon[142325]: Standby manager daemon smithi148.onhgvm restarted 2024-03-29T08:12:10.428 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:10 smithi037 ceph-mon[142325]: Standby manager daemon smithi148.onhgvm started 2024-03-29T08:12:10.428 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:10 smithi037 ceph-mon[142325]: from='mgr.? 172.21.15.148:0/1635681419' entity='mgr.smithi148.onhgvm' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi148.onhgvm/crt"}]: dispatch 2024-03-29T08:12:10.428 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:10 smithi037 ceph-mon[142325]: from='mgr.? 172.21.15.148:0/1635681419' entity='mgr.smithi148.onhgvm' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-29T08:12:10.428 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:10 smithi037 ceph-mon[142325]: from='mgr.? 172.21.15.148:0/1635681419' entity='mgr.smithi148.onhgvm' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi148.onhgvm/key"}]: dispatch 2024-03-29T08:12:10.428 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:10 smithi037 ceph-mon[142325]: from='mgr.? 172.21.15.148:0/1635681419' entity='mgr.smithi148.onhgvm' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-29T08:12:10.428 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:10 smithi037 ceph-mon[142325]: Active manager daemon smithi037.wjikii restarted 2024-03-29T08:12:10.428 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:10 smithi037 ceph-mon[142325]: Activating manager daemon smithi037.wjikii 2024-03-29T08:12:10.428 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:10 smithi037 ceph-mon[142325]: osdmap e50: 6 total, 6 up, 6 in 2024-03-29T08:12:10.428 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:10 smithi037 ceph-mon[142325]: mgrmap e36: smithi037.wjikii(active, starting, since 0.0074783s), standbys: smithi148.onhgvm 2024-03-29T08:12:10.428 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:10 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "mon metadata", "id": "smithi037"}]: dispatch 2024-03-29T08:12:10.428 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:10 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "mon metadata", "id": "smithi148"}]: dispatch 2024-03-29T08:12:10.428 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:10 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi037.mmwgxe"}]: dispatch 2024-03-29T08:12:10.429 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:10 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi037.iifsmh"}]: dispatch 2024-03-29T08:12:10.429 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:10 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi148.qukpjh"}]: dispatch 2024-03-29T08:12:10.429 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:10 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi148.sqteih"}]: dispatch 2024-03-29T08:12:10.429 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:10 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "mgr metadata", "who": "smithi037.wjikii", "id": "smithi037.wjikii"}]: dispatch 2024-03-29T08:12:10.429 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:10 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "mgr metadata", "who": "smithi148.onhgvm", "id": "smithi148.onhgvm"}]: dispatch 2024-03-29T08:12:10.429 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:10 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-29T08:12:10.429 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:10 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-29T08:12:10.429 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:10 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-29T08:12:10.429 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:10 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-29T08:12:10.429 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:10 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-29T08:12:10.429 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:10 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-29T08:12:10.429 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:10 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-29T08:12:10.429 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:10 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-29T08:12:10.429 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:10 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-29T08:12:10.429 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:10 smithi037 ceph-mon[142325]: Manager daemon smithi037.wjikii is now available 2024-03-29T08:12:10.430 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:10 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T08:12:10.430 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:10 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:12:10.430 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:10 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi037.wjikii/mirror_snapshot_schedule"}]: dispatch 2024-03-29T08:12:10.430 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:10 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi037.wjikii/trash_purge_schedule"}]: dispatch 2024-03-29T08:12:11.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:11 smithi148 ceph-mon[44992]: mgrmap e37: smithi037.wjikii(active, since 1.01358s), standbys: smithi148.onhgvm 2024-03-29T08:12:11.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:11 smithi148 ceph-mon[44992]: pgmap v3: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:12:11.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:11 smithi037 ceph-mon[142325]: mgrmap e37: smithi037.wjikii(active, since 1.01358s), standbys: smithi148.onhgvm 2024-03-29T08:12:11.928 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:11 smithi037 ceph-mon[142325]: pgmap v3: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:12:12.784 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:12 smithi148 ceph-mon[44992]: pgmap v4: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:12:12.785 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:12 smithi148 ceph-mon[44992]: mgrmap e38: smithi037.wjikii(active, since 2s), standbys: smithi148.onhgvm 2024-03-29T08:12:12.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:12 smithi037 ceph-mon[142325]: pgmap v4: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:12:12.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:12 smithi037 ceph-mon[142325]: mgrmap e38: smithi037.wjikii(active, since 2s), standbys: smithi148.onhgvm 2024-03-29T08:12:13.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:13 smithi148 ceph-mon[44992]: [29/Mar/2024:08:12:11] ENGINE Bus STARTING 2024-03-29T08:12:13.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:13 smithi148 ceph-mon[44992]: [29/Mar/2024:08:12:11] ENGINE Serving on http://172.21.15.37:8765 2024-03-29T08:12:13.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:13 smithi148 ceph-mon[44992]: [29/Mar/2024:08:12:11] ENGINE Serving on https://172.21.15.37:7150 2024-03-29T08:12:13.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:13 smithi148 ceph-mon[44992]: [29/Mar/2024:08:12:11] ENGINE Bus STARTED 2024-03-29T08:12:13.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:13 smithi037 ceph-mon[142325]: [29/Mar/2024:08:12:11] ENGINE Bus STARTING 2024-03-29T08:12:13.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:13 smithi037 ceph-mon[142325]: [29/Mar/2024:08:12:11] ENGINE Serving on http://172.21.15.37:8765 2024-03-29T08:12:13.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:13 smithi037 ceph-mon[142325]: [29/Mar/2024:08:12:11] ENGINE Serving on https://172.21.15.37:7150 2024-03-29T08:12:13.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:13 smithi037 ceph-mon[142325]: [29/Mar/2024:08:12:11] ENGINE Bus STARTED 2024-03-29T08:12:14.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:14 smithi037 ceph-mon[142325]: pgmap v5: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:12:14.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:14 smithi037 ceph-mon[142325]: mgrmap e39: smithi037.wjikii(active, since 4s), standbys: smithi148.onhgvm 2024-03-29T08:12:14.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:14 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:12:14.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:14 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:12:14.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:14 smithi148 ceph-mon[44992]: pgmap v5: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:12:14.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:14 smithi148 ceph-mon[44992]: mgrmap e39: smithi037.wjikii(active, since 4s), standbys: smithi148.onhgvm 2024-03-29T08:12:14.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:14 smithi148 ceph-mon[44992]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:12:14.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:14 smithi148 ceph-mon[44992]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:12:16.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:15 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:12:16.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:15 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:12:16.178 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:15 smithi037 ceph-mon[142325]: pgmap v6: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:12:16.178 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:15 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:12:16.178 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:15 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:12:16.284 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:15 smithi148 ceph-mon[44992]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:12:16.285 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:15 smithi148 ceph-mon[44992]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:12:16.285 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:15 smithi148 ceph-mon[44992]: pgmap v6: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:12:16.285 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:15 smithi148 ceph-mon[44992]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:12:16.285 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:15 smithi148 ceph-mon[44992]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:12:18.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:17 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:12:18.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:17 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:12:18.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:17 smithi037 ceph-mon[142325]: pgmap v7: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 29 KiB/s rd, 0 B/s wr, 16 op/s 2024-03-29T08:12:18.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:17 smithi148 ceph-mon[44992]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:12:18.307 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:17 smithi148 ceph-mon[44992]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:12:18.307 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:17 smithi148 ceph-mon[44992]: pgmap v7: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 29 KiB/s rd, 0 B/s wr, 16 op/s 2024-03-29T08:12:20.807 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:20 smithi148 ceph-mon[44992]: pgmap v8: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 23 KiB/s rd, 0 B/s wr, 12 op/s 2024-03-29T08:12:20.807 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:20 smithi148 ceph-mon[44992]: Detected new or changed devices on smithi148 2024-03-29T08:12:20.807 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:20 smithi148 ceph-mon[44992]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:12:20.807 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:20 smithi148 ceph-mon[44992]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:12:20.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:20 smithi037 ceph-mon[142325]: pgmap v8: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 23 KiB/s rd, 0 B/s wr, 12 op/s 2024-03-29T08:12:20.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:20 smithi037 ceph-mon[142325]: Detected new or changed devices on smithi148 2024-03-29T08:12:20.928 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:20 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:12:20.928 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:20 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:12:22.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:22 smithi148 ceph-mon[44992]: Detected new or changed devices on smithi037 2024-03-29T08:12:22.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:22 smithi148 ceph-mon[44992]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:12:22.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:22 smithi148 ceph-mon[44992]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:12:22.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:22 smithi148 ceph-mon[44992]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T08:12:22.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:22 smithi148 ceph-mon[44992]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T08:12:22.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:22 smithi148 ceph-mon[44992]: Updating smithi037:/etc/ceph/ceph.conf 2024-03-29T08:12:22.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:22 smithi148 ceph-mon[44992]: Updating smithi148:/etc/ceph/ceph.conf 2024-03-29T08:12:22.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:22 smithi148 ceph-mon[44992]: pgmap v9: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 19 KiB/s rd, 0 B/s wr, 10 op/s 2024-03-29T08:12:22.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:22 smithi037 ceph-mon[142325]: Detected new or changed devices on smithi037 2024-03-29T08:12:22.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:22 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:12:22.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:22 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:12:22.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:22 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T08:12:22.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:22 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T08:12:22.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:22 smithi037 ceph-mon[142325]: Updating smithi037:/etc/ceph/ceph.conf 2024-03-29T08:12:22.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:22 smithi037 ceph-mon[142325]: Updating smithi148:/etc/ceph/ceph.conf 2024-03-29T08:12:22.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:22 smithi037 ceph-mon[142325]: pgmap v9: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 19 KiB/s rd, 0 B/s wr, 10 op/s 2024-03-29T08:12:23.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:23 smithi037 ceph-mon[142325]: Updating smithi037:/var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/config/ceph.conf 2024-03-29T08:12:23.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:23 smithi037 ceph-mon[142325]: Updating smithi148:/var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/config/ceph.conf 2024-03-29T08:12:23.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:23 smithi037 ceph-mon[142325]: Updating smithi037:/etc/ceph/ceph.client.admin.keyring 2024-03-29T08:12:23.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:23 smithi037 ceph-mon[142325]: Updating smithi148:/etc/ceph/ceph.client.admin.keyring 2024-03-29T08:12:23.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:23 smithi148 ceph-mon[44992]: Updating smithi037:/var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/config/ceph.conf 2024-03-29T08:12:23.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:23 smithi148 ceph-mon[44992]: Updating smithi148:/var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/config/ceph.conf 2024-03-29T08:12:23.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:23 smithi148 ceph-mon[44992]: Updating smithi037:/etc/ceph/ceph.client.admin.keyring 2024-03-29T08:12:23.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:23 smithi148 ceph-mon[44992]: Updating smithi148:/etc/ceph/ceph.client.admin.keyring 2024-03-29T08:12:24.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:24 smithi148 ceph-mon[44992]: Updating smithi037:/var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/config/ceph.client.admin.keyring 2024-03-29T08:12:24.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:24 smithi148 ceph-mon[44992]: Updating smithi148:/var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/config/ceph.client.admin.keyring 2024-03-29T08:12:24.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:24 smithi148 ceph-mon[44992]: pgmap v10: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 17 KiB/s rd, 0 B/s wr, 9 op/s 2024-03-29T08:12:24.807 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:24 smithi148 ceph-mon[44992]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:12:24.807 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:24 smithi148 ceph-mon[44992]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:12:24.807 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:24 smithi148 ceph-mon[44992]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:12:24.807 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:24 smithi148 ceph-mon[44992]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:12:24.807 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:24 smithi148 ceph-mon[44992]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:12:24.807 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:24 smithi148 ceph-mon[44992]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T08:12:24.807 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:24 smithi148 ceph-mon[44992]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:12:24.807 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:24 smithi148 ceph-mon[44992]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-03-29T08:12:24.807 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:24 smithi148 ceph-mon[44992]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:12:24.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:24 smithi037 ceph-mon[142325]: Updating smithi037:/var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/config/ceph.client.admin.keyring 2024-03-29T08:12:24.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:24 smithi037 ceph-mon[142325]: Updating smithi148:/var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/config/ceph.client.admin.keyring 2024-03-29T08:12:24.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:24 smithi037 ceph-mon[142325]: pgmap v10: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 17 KiB/s rd, 0 B/s wr, 9 op/s 2024-03-29T08:12:24.928 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:24 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:12:24.928 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:24 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:12:24.928 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:24 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:12:24.928 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:24 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:12:24.928 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:24 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:12:24.928 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:24 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T08:12:24.928 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:24 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:12:24.928 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:24 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-03-29T08:12:24.928 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:24 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:12:25.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:25.864+0000 7f74faa00700 1 -- 172.21.15.37:0/464714706 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f74f4102e00 msgr2=0x7f74f41031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:12:25.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:25.864+0000 7f74faa00700 1 --2- 172.21.15.37:0/464714706 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f74f4102e00 0x7f74f41031d0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f74e4004880 tx=0x7f74e4009e30 comp rx=0 tx=0).stop 2024-03-29T08:12:25.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:25.864+0000 7f74faa00700 1 -- 172.21.15.37:0/464714706 shutdown_connections 2024-03-29T08:12:25.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:25.864+0000 7f74faa00700 1 --2- 172.21.15.37:0/464714706 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f74f41037a0 0x7f74f40fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:12:25.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:25.864+0000 7f74faa00700 1 --2- 172.21.15.37:0/464714706 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f74f4102e00 0x7f74f41031d0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:12:25.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:25.864+0000 7f74faa00700 1 -- 172.21.15.37:0/464714706 >> 172.21.15.37:0/464714706 conn(0x7f74f40f89b0 msgr2=0x7f74f40fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:12:25.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:25.865+0000 7f74faa00700 1 -- 172.21.15.37:0/464714706 shutdown_connections 2024-03-29T08:12:25.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:25.865+0000 7f74faa00700 1 -- 172.21.15.37:0/464714706 wait complete. 2024-03-29T08:12:25.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:25.865+0000 7f74faa00700 1 Processor -- start 2024-03-29T08:12:25.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:25.866+0000 7f74faa00700 1 -- start start 2024-03-29T08:12:25.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:25.866+0000 7f74faa00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f74f4102e00 0x7f74f4113a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:12:25.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:25.866+0000 7f74faa00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f74f41037a0 0x7f74f4113f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:12:25.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:25.866+0000 7f74faa00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74f41145d0 con 0x7f74f41037a0 2024-03-29T08:12:25.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:25.866+0000 7f74faa00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74f410d9c0 con 0x7f74f4102e00 2024-03-29T08:12:25.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:25.866+0000 7f74f3400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f74f41037a0 0x7f74f4113f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:12:25.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:25.866+0000 7f74f3400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f74f41037a0 0x7f74f4113f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52940/0 (socket says 172.21.15.37:52940) 2024-03-29T08:12:25.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:25.866+0000 7f74f3400700 1 -- 172.21.15.37:0/3483794449 learned_addr learned my addr 172.21.15.37:0/3483794449 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:12:25.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:25.866+0000 7f74f3e00700 1 --2- 172.21.15.37:0/3483794449 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f74f4102e00 0x7f74f4113a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:12:25.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:25.867+0000 7f74f3400700 1 -- 172.21.15.37:0/3483794449 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f74f4102e00 msgr2=0x7f74f4113a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:12:25.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:25.867+0000 7f74f3400700 1 --2- 172.21.15.37:0/3483794449 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f74f4102e00 0x7f74f4113a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:12:25.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:25.867+0000 7f74f3400700 1 -- 172.21.15.37:0/3483794449 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74e0005040 con 0x7f74f41037a0 2024-03-29T08:12:25.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:25.867+0000 7f74f3400700 1 --2- 172.21.15.37:0/3483794449 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f74f41037a0 0x7f74f4113f80 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f74e00096e0 tx=0x7f74e0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:12:25.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:25.867+0000 7f74ebe00700 1 -- 172.21.15.37:0/3483794449 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f74e0017070 con 0x7f74f41037a0 2024-03-29T08:12:25.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:25.867+0000 7f74ebe00700 1 -- 172.21.15.37:0/3483794449 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f74e000f040 con 0x7f74f41037a0 2024-03-29T08:12:25.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:25.867+0000 7f74faa00700 1 -- 172.21.15.37:0/3483794449 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74e4005040 con 0x7f74f41037a0 2024-03-29T08:12:25.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:25.867+0000 7f74ebe00700 1 -- 172.21.15.37:0/3483794449 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f74e0013ba0 con 0x7f74f41037a0 2024-03-29T08:12:25.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:25.867+0000 7f74faa00700 1 -- 172.21.15.37:0/3483794449 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74f410e070 con 0x7f74f41037a0 2024-03-29T08:12:25.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:25.868+0000 7f74faa00700 1 -- 172.21.15.37:0/3483794449 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74f40fe550 con 0x7f74f41037a0 2024-03-29T08:12:25.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:25.870+0000 7f74ebe00700 1 -- 172.21.15.37:0/3483794449 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 39) v1 ==== 90944+0+0 (secure 0 0 0) 0x7f74e000e840 con 0x7f74f41037a0 2024-03-29T08:12:25.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:25.871+0000 7f74ebe00700 1 --2- 172.21.15.37:0/3483794449 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f74d806cd40 0x7f74d806f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:12:25.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:25.871+0000 7f74ebe00700 1 -- 172.21.15.37:0/3483794449 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f74e008bf80 con 0x7f74f41037a0 2024-03-29T08:12:25.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:25.871+0000 7f74f3e00700 1 --2- 172.21.15.37:0/3483794449 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f74d806cd40 0x7f74d806f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:12:25.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:25.872+0000 7f74f3e00700 1 --2- 172.21.15.37:0/3483794449 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f74d806cd40 0x7f74d806f1f0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f74e4007f50 tx=0x7f74e4009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:12:25.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:25.877+0000 7f74ebe00700 1 -- 172.21.15.37:0/3483794449 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f74e0059830 con 0x7f74f41037a0 2024-03-29T08:12:26.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:26.111+0000 7f74faa00700 1 -- 172.21.15.37:0/3483794449 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f74f4109e60 con 0x7f74d806cd40 2024-03-29T08:12:26.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:26.113+0000 7f74ebe00700 1 -- 172.21.15.37:0/3483794449 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+368 (secure 0 0 0) 0x7f74e40096f0 con 0x7f74d806cd40 2024-03-29T08:12:26.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:26.115+0000 7f74faa00700 1 -- 172.21.15.37:0/3483794449 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f74d806cd40 msgr2=0x7f74d806f1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:12:26.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:26.115+0000 7f74faa00700 1 --2- 172.21.15.37:0/3483794449 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f74d806cd40 0x7f74d806f1f0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f74e4007f50 tx=0x7f74e4009e30 comp rx=0 tx=0).stop 2024-03-29T08:12:26.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:26.115+0000 7f74faa00700 1 -- 172.21.15.37:0/3483794449 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f74f41037a0 msgr2=0x7f74f4113f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:12:26.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:26.115+0000 7f74faa00700 1 --2- 172.21.15.37:0/3483794449 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f74f41037a0 0x7f74f4113f80 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f74e00096e0 tx=0x7f74e0009e30 comp rx=0 tx=0).stop 2024-03-29T08:12:26.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:26.115+0000 7f74faa00700 1 -- 172.21.15.37:0/3483794449 shutdown_connections 2024-03-29T08:12:26.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:26.115+0000 7f74faa00700 1 --2- 172.21.15.37:0/3483794449 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f74d806cd40 0x7f74d806f1f0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:12:26.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:26.115+0000 7f74faa00700 1 --2- 172.21.15.37:0/3483794449 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f74f4102e00 0x7f74f4113a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:12:26.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:26.115+0000 7f74faa00700 1 --2- 172.21.15.37:0/3483794449 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f74f41037a0 0x7f74f4113f80 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:12:26.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:26.115+0000 7f74faa00700 1 -- 172.21.15.37:0/3483794449 >> 172.21.15.37:0/3483794449 conn(0x7f74f40f89b0 msgr2=0x7f74f40f99f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:12:26.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:26.116+0000 7f74faa00700 1 -- 172.21.15.37:0/3483794449 shutdown_connections 2024-03-29T08:12:26.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:26.116+0000 7f74faa00700 1 -- 172.21.15.37:0/3483794449 wait complete. 2024-03-29T08:12:26.130 INFO:teuthology.orchestra.run.smithi037.stdout:true 2024-03-29T08:12:26.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:26.254+0000 7f477f200700 1 -- 172.21.15.37:0/2087961716 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4780102c40 msgr2=0x7f4780103010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:12:26.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:26.254+0000 7f477f200700 1 --2- 172.21.15.37:0/2087961716 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4780102c40 0x7f4780103010 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f4770004940 tx=0x7f4770009e30 comp rx=0 tx=0).stop 2024-03-29T08:12:26.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:26.254+0000 7f477f200700 1 -- 172.21.15.37:0/2087961716 shutdown_connections 2024-03-29T08:12:26.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:26.254+0000 7f477f200700 1 --2- 172.21.15.37:0/2087961716 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f47801035e0 0x7f47800fcc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:12:26.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:26.254+0000 7f477f200700 1 --2- 172.21.15.37:0/2087961716 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4780102c40 0x7f4780103010 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:12:26.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:26.254+0000 7f477f200700 1 -- 172.21.15.37:0/2087961716 >> 172.21.15.37:0/2087961716 conn(0x7f47800f8830 msgr2=0x7f47800fac40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:12:26.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:26.254+0000 7f477f200700 1 -- 172.21.15.37:0/2087961716 shutdown_connections 2024-03-29T08:12:26.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:26.254+0000 7f477f200700 1 -- 172.21.15.37:0/2087961716 wait complete. 2024-03-29T08:12:26.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:26.255+0000 7f477f200700 1 Processor -- start 2024-03-29T08:12:26.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:26.255+0000 7f477f200700 1 -- start start 2024-03-29T08:12:26.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:26.256+0000 7f477f200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4780102c40 0x7f4780115a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:12:26.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:26.256+0000 7f477f200700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f47801035e0 0x7f4780115fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:12:26.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:26.256+0000 7f477f200700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4780116610 con 0x7f4780102c40 2024-03-29T08:12:26.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:26.256+0000 7f477f200700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f478010fa00 con 0x7f47801035e0 2024-03-29T08:12:26.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:26.256+0000 7f477de00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4780102c40 0x7f4780115a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:12:26.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:26.256+0000 7f477de00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4780102c40 0x7f4780115a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52948/0 (socket says 172.21.15.37:52948) 2024-03-29T08:12:26.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:26.256+0000 7f477de00700 1 -- 172.21.15.37:0/1894635710 learned_addr learned my addr 172.21.15.37:0/1894635710 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:12:26.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:26.256+0000 7f477d400700 1 --2- 172.21.15.37:0/1894635710 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f47801035e0 0x7f4780115fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:12:26.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:26.256+0000 7f477de00700 1 -- 172.21.15.37:0/1894635710 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f47801035e0 msgr2=0x7f4780115fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:12:26.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:26.256+0000 7f477de00700 1 --2- 172.21.15.37:0/1894635710 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f47801035e0 0x7f4780115fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:12:26.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:26.256+0000 7f477de00700 1 -- 172.21.15.37:0/1894635710 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4770005040 con 0x7f4780102c40 2024-03-29T08:12:26.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:26.257+0000 7f477de00700 1 --2- 172.21.15.37:0/1894635710 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4780102c40 0x7f4780115a80 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f4770008d80 tx=0x7f4770009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:12:26.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:26.257+0000 7f4776000700 1 -- 172.21.15.37:0/1894635710 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f477000e030 con 0x7f4780102c40 2024-03-29T08:12:26.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:26.257+0000 7f477f200700 1 -- 172.21.15.37:0/1894635710 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f478010fc80 con 0x7f4780102c40 2024-03-29T08:12:26.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:26.257+0000 7f4776000700 1 -- 172.21.15.37:0/1894635710 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f4770013de0 con 0x7f4780102c40 2024-03-29T08:12:26.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:26.257+0000 7f477f200700 1 -- 172.21.15.37:0/1894635710 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4780110240 con 0x7f4780102c40 2024-03-29T08:12:26.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:26.257+0000 7f4776000700 1 -- 172.21.15.37:0/1894635710 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f477001cca0 con 0x7f4780102c40 2024-03-29T08:12:26.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:26.258+0000 7f477f200700 1 -- 172.21.15.37:0/1894635710 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47800fe390 con 0x7f4780102c40 2024-03-29T08:12:26.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:26.259+0000 7f4776000700 1 -- 172.21.15.37:0/1894635710 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 39) v1 ==== 90944+0+0 (secure 0 0 0) 0x7f4770013930 con 0x7f4780102c40 2024-03-29T08:12:26.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:26.260+0000 7f4776000700 1 --2- 172.21.15.37:0/1894635710 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f476406cd40 0x7f476406f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:12:26.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:26.260+0000 7f4776000700 1 -- 172.21.15.37:0/1894635710 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f477008dad0 con 0x7f4780102c40 2024-03-29T08:12:26.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:26.260+0000 7f477d400700 1 --2- 172.21.15.37:0/1894635710 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f476406cd40 0x7f476406f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:12:26.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:26.261+0000 7f477d400700 1 --2- 172.21.15.37:0/1894635710 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f476406cd40 0x7f476406f1f0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f4780110bc0 tx=0x7f476c00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:12:26.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:26.264+0000 7f4776000700 1 -- 172.21.15.37:0/1894635710 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f477005b380 con 0x7f4780102c40 2024-03-29T08:12:26.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:26.500+0000 7f477f200700 1 -- 172.21.15.37:0/1894635710 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4780109c40 con 0x7f476406cd40 2024-03-29T08:12:26.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:26.501+0000 7f4776000700 1 -- 172.21.15.37:0/1894635710 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+368 (secure 0 0 0) 0x7f4780109c40 con 0x7f476406cd40 2024-03-29T08:12:26.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:26.503+0000 7f477f200700 1 -- 172.21.15.37:0/1894635710 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f476406cd40 msgr2=0x7f476406f1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:12:26.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:26.503+0000 7f477f200700 1 --2- 172.21.15.37:0/1894635710 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f476406cd40 0x7f476406f1f0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f4780110bc0 tx=0x7f476c00b040 comp rx=0 tx=0).stop 2024-03-29T08:12:26.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:26.503+0000 7f477f200700 1 -- 172.21.15.37:0/1894635710 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4780102c40 msgr2=0x7f4780115a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:12:26.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:26.503+0000 7f477f200700 1 --2- 172.21.15.37:0/1894635710 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4780102c40 0x7f4780115a80 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f4770008d80 tx=0x7f4770009e30 comp rx=0 tx=0).stop 2024-03-29T08:12:26.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:26.503+0000 7f477f200700 1 -- 172.21.15.37:0/1894635710 shutdown_connections 2024-03-29T08:12:26.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:26.504+0000 7f477f200700 1 --2- 172.21.15.37:0/1894635710 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f476406cd40 0x7f476406f1f0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:12:26.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:26.504+0000 7f477f200700 1 --2- 172.21.15.37:0/1894635710 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4780102c40 0x7f4780115a80 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:12:26.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:26.504+0000 7f477f200700 1 --2- 172.21.15.37:0/1894635710 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f47801035e0 0x7f4780115fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:12:26.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:26.504+0000 7f477f200700 1 -- 172.21.15.37:0/1894635710 >> 172.21.15.37:0/1894635710 conn(0x7f47800f8830 msgr2=0x7f47800f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:12:26.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:26.504+0000 7f477f200700 1 -- 172.21.15.37:0/1894635710 shutdown_connections 2024-03-29T08:12:26.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:26.504+0000 7f477f200700 1 -- 172.21.15.37:0/1894635710 wait complete. 2024-03-29T08:12:26.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:26.640+0000 7f4580800700 1 -- 172.21.15.37:0/2925248702 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f45781037a0 msgr2=0x7f45780fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:12:26.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:26.640+0000 7f4580800700 1 --2- 172.21.15.37:0/2925248702 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f45781037a0 0x7f45780fce10 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f4568004940 tx=0x7f4568009e30 comp rx=0 tx=0).stop 2024-03-29T08:12:26.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:26.640+0000 7f4580800700 1 -- 172.21.15.37:0/2925248702 shutdown_connections 2024-03-29T08:12:26.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:26.640+0000 7f4580800700 1 --2- 172.21.15.37:0/2925248702 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f45781037a0 0x7f45780fce10 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:12:26.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:26.640+0000 7f4580800700 1 --2- 172.21.15.37:0/2925248702 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4578102e00 0x7f45781031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:12:26.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:26.640+0000 7f4580800700 1 -- 172.21.15.37:0/2925248702 >> 172.21.15.37:0/2925248702 conn(0x7f45780f89b0 msgr2=0x7f45780fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:12:26.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:26.640+0000 7f4580800700 1 -- 172.21.15.37:0/2925248702 shutdown_connections 2024-03-29T08:12:26.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:26.640+0000 7f4580800700 1 -- 172.21.15.37:0/2925248702 wait complete. 2024-03-29T08:12:26.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:26.641+0000 7f4580800700 1 Processor -- start 2024-03-29T08:12:26.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:26.641+0000 7f4580800700 1 -- start start 2024-03-29T08:12:26.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:26.641+0000 7f4580800700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4578102e00 0x7f4578113b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:12:26.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:26.641+0000 7f4580800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f45781037a0 0x7f4578114040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:12:26.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:26.641+0000 7f4580800700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4578114690 con 0x7f45781037a0 2024-03-29T08:12:26.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:26.641+0000 7f4580800700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f457810da80 con 0x7f4578102e00 2024-03-29T08:12:26.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:26.641+0000 7f457d400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f45781037a0 0x7f4578114040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:12:26.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:26.642+0000 7f457d400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f45781037a0 0x7f4578114040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52978/0 (socket says 172.21.15.37:52978) 2024-03-29T08:12:26.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:26.642+0000 7f457d400700 1 -- 172.21.15.37:0/1998169462 learned_addr learned my addr 172.21.15.37:0/1998169462 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:12:26.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:26.642+0000 7f457de00700 1 --2- 172.21.15.37:0/1998169462 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4578102e00 0x7f4578113b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:12:26.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:26.642+0000 7f457d400700 1 -- 172.21.15.37:0/1998169462 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4578102e00 msgr2=0x7f4578113b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:12:26.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:26.642+0000 7f457d400700 1 --2- 172.21.15.37:0/1998169462 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4578102e00 0x7f4578113b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:12:26.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:26.642+0000 7f457d400700 1 -- 172.21.15.37:0/1998169462 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4568005040 con 0x7f45781037a0 2024-03-29T08:12:26.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:26.642+0000 7f457d400700 1 --2- 172.21.15.37:0/1998169462 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f45781037a0 0x7f4578114040 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f4568009390 tx=0x7f4568009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:12:26.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:26.642+0000 7f4572000700 1 -- 172.21.15.37:0/1998169462 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f456800e040 con 0x7f45781037a0 2024-03-29T08:12:26.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:26.643+0000 7f4572000700 1 -- 172.21.15.37:0/1998169462 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f4568013420 con 0x7f45781037a0 2024-03-29T08:12:26.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:26.643+0000 7f4580800700 1 -- 172.21.15.37:0/1998169462 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f457810dd00 con 0x7f45781037a0 2024-03-29T08:12:26.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:26.643+0000 7f4572000700 1 -- 172.21.15.37:0/1998169462 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f456801c630 con 0x7f45781037a0 2024-03-29T08:12:26.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:26.643+0000 7f4580800700 1 -- 172.21.15.37:0/1998169462 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f457810e2c0 con 0x7f45781037a0 2024-03-29T08:12:26.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:26.644+0000 7f4580800700 1 -- 172.21.15.37:0/1998169462 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45780fe550 con 0x7f45781037a0 2024-03-29T08:12:26.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:26.645+0000 7f4572000700 1 -- 172.21.15.37:0/1998169462 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 39) v1 ==== 90944+0+0 (secure 0 0 0) 0x7f4568013a00 con 0x7f45781037a0 2024-03-29T08:12:26.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:26.646+0000 7f4572000700 1 --2- 172.21.15.37:0/1998169462 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f456006cd40 0x7f456006f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:12:26.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:26.646+0000 7f4572000700 1 -- 172.21.15.37:0/1998169462 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f456808d4b0 con 0x7f45781037a0 2024-03-29T08:12:26.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:26.646+0000 7f457de00700 1 --2- 172.21.15.37:0/1998169462 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f456006cd40 0x7f456006f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:12:26.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:26.646+0000 7f457de00700 1 --2- 172.21.15.37:0/1998169462 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f456006cd40 0x7f456006f1f0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f456c004990 tx=0x7f456c009ee0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:12:26.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:26.651+0000 7f4572000700 1 -- 172.21.15.37:0/1998169462 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f456805ae00 con 0x7f45781037a0 2024-03-29T08:12:26.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:26 smithi148 ceph-mon[44992]: pgmap v11: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 17 KiB/s rd, 0 B/s wr, 9 op/s 2024-03-29T08:12:26.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:26 smithi148 ceph-mon[44992]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:12:26.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:26 smithi148 ceph-mon[44992]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-29T08:12:26.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:26 smithi148 ceph-mon[44992]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-29T08:12:26.807 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:26 smithi148 ceph-mon[44992]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T08:12:26.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:26.873+0000 7f4580800700 1 -- 172.21.15.37:0/1998169462 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f4578109da0 con 0x7f456006cd40 2024-03-29T08:12:26.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:26.883+0000 7f4572000700 1 -- 172.21.15.37:0/1998169462 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7f4578109da0 con 0x7f456006cd40 2024-03-29T08:12:26.883 INFO:teuthology.orchestra.run.smithi037.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-29T08:12:26.883 INFO:teuthology.orchestra.run.smithi037.stdout:alertmanager.smithi037 smithi037 *:9093,9094 running (2m) 11s ago 15m 14.6M - 0.25.0 c8568f914cd2 2f9f9746ed0f 2024-03-29T08:12:26.883 INFO:teuthology.orchestra.run.smithi037.stdout:crash.smithi037 smithi037 running (15m) 11s ago 15m 6656k - 17.2.7-789-g4213e17c cf22eb8683be e0343935b25e 2024-03-29T08:12:26.883 INFO:teuthology.orchestra.run.smithi037.stdout:crash.smithi148 smithi148 running (14m) 13s ago 14m 6656k - 17.2.7-789-g4213e17c cf22eb8683be ff5164c552c1 2024-03-29T08:12:26.883 INFO:teuthology.orchestra.run.smithi037.stdout:grafana.smithi037 smithi037 *:3000 running (117s) 11s ago 14m 73.4M - 9.4.7 954c08fa6188 6b976fe49952 2024-03-29T08:12:26.884 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi037.iifsmh smithi037 running (8m) 11s ago 8m 110M - 17.2.7-789-g4213e17c cf22eb8683be 074e1acf331f 2024-03-29T08:12:26.884 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi037.mmwgxe smithi037 running (8m) 11s ago 8m 115M - 17.2.7-789-g4213e17c cf22eb8683be 9a5b1e72de5c 2024-03-29T08:12:26.884 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi148.qukpjh smithi148 running (8m) 13s ago 8m 13.7M - 17.2.7-789-g4213e17c cf22eb8683be 8e469782a273 2024-03-29T08:12:26.884 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi148.sqteih smithi148 running (8m) 13s ago 8m 13.7M - 17.2.7-789-g4213e17c cf22eb8683be 8e68f858bb22 2024-03-29T08:12:26.884 INFO:teuthology.orchestra.run.smithi037.stdout:mgr.smithi037.wjikii smithi037 *:8443,9283,8765 running (4m) 11s ago 16m 468M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff c5a1417b9417 2024-03-29T08:12:26.884 INFO:teuthology.orchestra.run.smithi037.stdout:mgr.smithi148.onhgvm smithi148 *:8443,9283,8765 running (3m) 13s ago 14m 445M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff fcfdbce062bd 2024-03-29T08:12:26.884 INFO:teuthology.orchestra.run.smithi037.stdout:mon.smithi037 smithi037 running (28s) 11s ago 17m 27.5M 2048M 18.2.2-1015-gd0e392d9 1cc2f45c17ff eea2060783af 2024-03-29T08:12:26.884 INFO:teuthology.orchestra.run.smithi037.stdout:mon.smithi148 smithi148 running (13m) 13s ago 13m 47.2M 2048M 17.2.7-789-g4213e17c cf22eb8683be 18839ed2952a 2024-03-29T08:12:26.884 INFO:teuthology.orchestra.run.smithi037.stdout:node-exporter.smithi037 smithi037 *:9100 running (3m) 11s ago 15m 7503k - 1.5.0 0da6a335fe13 26b8ae9cb878 2024-03-29T08:12:26.884 INFO:teuthology.orchestra.run.smithi037.stdout:node-exporter.smithi148 smithi148 *:9100 running (3m) 13s ago 14m 7243k - 1.5.0 0da6a335fe13 98414e4e64ed 2024-03-29T08:12:26.884 INFO:teuthology.orchestra.run.smithi037.stdout:osd.0 smithi037 running (13m) 11s ago 13m 217M 1619M 17.2.7-789-g4213e17c cf22eb8683be 559fccd0ebe1 2024-03-29T08:12:26.884 INFO:teuthology.orchestra.run.smithi037.stdout:osd.1 smithi037 running (12m) 11s ago 12m 229M 1619M 17.2.7-789-g4213e17c cf22eb8683be 5582cff11b7a 2024-03-29T08:12:26.884 INFO:teuthology.orchestra.run.smithi037.stdout:osd.2 smithi037 running (12m) 11s ago 12m 198M 1619M 17.2.7-789-g4213e17c cf22eb8683be f809bfab13e5 2024-03-29T08:12:26.884 INFO:teuthology.orchestra.run.smithi037.stdout:osd.3 smithi148 running (11m) 13s ago 11m 262M 2642M 17.2.7-789-g4213e17c cf22eb8683be a0a8224bb4b0 2024-03-29T08:12:26.884 INFO:teuthology.orchestra.run.smithi037.stdout:osd.4 smithi148 running (11m) 13s ago 11m 233M 2642M 17.2.7-789-g4213e17c cf22eb8683be a00b69047ed7 2024-03-29T08:12:26.885 INFO:teuthology.orchestra.run.smithi037.stdout:osd.5 smithi148 running (10m) 13s ago 10m 233M 2642M 17.2.7-789-g4213e17c cf22eb8683be 044a3f7db7f8 2024-03-29T08:12:26.885 INFO:teuthology.orchestra.run.smithi037.stdout:prometheus.smithi037 smithi037 *:9095 running (2m) 11s ago 14m 49.4M - 2.43.0 a07b618ecd1d 30c0223ec589 2024-03-29T08:12:26.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:26.885+0000 7f4580800700 1 -- 172.21.15.37:0/1998169462 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f456006cd40 msgr2=0x7f456006f1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:12:26.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:26.885+0000 7f4580800700 1 --2- 172.21.15.37:0/1998169462 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f456006cd40 0x7f456006f1f0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f456c004990 tx=0x7f456c009ee0 comp rx=0 tx=0).stop 2024-03-29T08:12:26.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:26.885+0000 7f4580800700 1 -- 172.21.15.37:0/1998169462 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f45781037a0 msgr2=0x7f4578114040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:12:26.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:26.885+0000 7f4580800700 1 --2- 172.21.15.37:0/1998169462 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f45781037a0 0x7f4578114040 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f4568009390 tx=0x7f4568009e30 comp rx=0 tx=0).stop 2024-03-29T08:12:26.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:26.886+0000 7f4580800700 1 -- 172.21.15.37:0/1998169462 shutdown_connections 2024-03-29T08:12:26.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:26.886+0000 7f4580800700 1 --2- 172.21.15.37:0/1998169462 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f456006cd40 0x7f456006f1f0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:12:26.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:26.886+0000 7f4580800700 1 --2- 172.21.15.37:0/1998169462 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4578102e00 0x7f4578113b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:12:26.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:26.886+0000 7f4580800700 1 --2- 172.21.15.37:0/1998169462 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f45781037a0 0x7f4578114040 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:12:26.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:26.886+0000 7f4580800700 1 -- 172.21.15.37:0/1998169462 >> 172.21.15.37:0/1998169462 conn(0x7f45780f89b0 msgr2=0x7f45780f9ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:12:26.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:26.886+0000 7f4580800700 1 -- 172.21.15.37:0/1998169462 shutdown_connections 2024-03-29T08:12:26.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:26.886+0000 7f4580800700 1 -- 172.21.15.37:0/1998169462 wait complete. 2024-03-29T08:12:26.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:26 smithi037 ceph-mon[142325]: pgmap v11: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 17 KiB/s rd, 0 B/s wr, 9 op/s 2024-03-29T08:12:26.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:26 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:12:26.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:26 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-29T08:12:26.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:26 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-29T08:12:26.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:26 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T08:12:27.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:27.020+0000 7f5acf800700 1 -- 172.21.15.37:0/3751460001 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ac80fe830 msgr2=0x7f5ac8106f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:12:27.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:27.020+0000 7f5acf800700 1 --2- 172.21.15.37:0/3751460001 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ac80fe830 0x7f5ac8106f00 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f5ab4004940 tx=0x7f5ab4009e30 comp rx=0 tx=0).stop 2024-03-29T08:12:27.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:27.020+0000 7f5acf800700 1 -- 172.21.15.37:0/3751460001 shutdown_connections 2024-03-29T08:12:27.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:27.020+0000 7f5acf800700 1 --2- 172.21.15.37:0/3751460001 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ac80fe830 0x7f5ac8106f00 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:12:27.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:27.021+0000 7f5acf800700 1 --2- 172.21.15.37:0/3751460001 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5ac80fde90 0x7f5ac80fe260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:12:27.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:27.021+0000 7f5acf800700 1 -- 172.21.15.37:0/3751460001 >> 172.21.15.37:0/3751460001 conn(0x7f5ac8072bc0 msgr2=0x7f5ac8072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:12:27.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:27.021+0000 7f5acf800700 1 -- 172.21.15.37:0/3751460001 shutdown_connections 2024-03-29T08:12:27.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:27.021+0000 7f5acf800700 1 -- 172.21.15.37:0/3751460001 wait complete. 2024-03-29T08:12:27.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:27.021+0000 7f5acf800700 1 Processor -- start 2024-03-29T08:12:27.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:27.022+0000 7f5acf800700 1 -- start start 2024-03-29T08:12:27.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:27.022+0000 7f5acf800700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5ac80fde90 0x7f5ac80fef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:12:27.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:27.022+0000 7f5acf800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ac80fe830 0x7f5ac80ff4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:12:27.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:27.022+0000 7f5acf800700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ac8103100 con 0x7f5ac80fe830 2024-03-29T08:12:27.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:27.022+0000 7f5acf800700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ac80ff9f0 con 0x7f5ac80fde90 2024-03-29T08:12:27.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:27.022+0000 7f5ac7e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ac80fe830 0x7f5ac80ff4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:12:27.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:27.022+0000 7f5ac7e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ac80fe830 0x7f5ac80ff4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52998/0 (socket says 172.21.15.37:52998) 2024-03-29T08:12:27.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:27.022+0000 7f5ac7e00700 1 -- 172.21.15.37:0/1482023148 learned_addr learned my addr 172.21.15.37:0/1482023148 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:12:27.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:27.023+0000 7f5acce00700 1 --2- 172.21.15.37:0/1482023148 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5ac80fde90 0x7f5ac80fef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:12:27.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:27.023+0000 7f5ac7e00700 1 -- 172.21.15.37:0/1482023148 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5ac80fde90 msgr2=0x7f5ac80fef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:12:27.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:27.023+0000 7f5ac7e00700 1 --2- 172.21.15.37:0/1482023148 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5ac80fde90 0x7f5ac80fef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:12:27.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:27.023+0000 7f5ac7e00700 1 -- 172.21.15.37:0/1482023148 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ab4005040 con 0x7f5ac80fe830 2024-03-29T08:12:27.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:27.023+0000 7f5acce00700 1 --2- 172.21.15.37:0/1482023148 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5ac80fde90 0x7f5ac80fef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:12:27.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:27.023+0000 7f5ac7e00700 1 --2- 172.21.15.37:0/1482023148 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ac80fe830 0x7f5ac80ff4b0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f5ab4000c00 tx=0x7f5ab4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:12:27.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:27.023+0000 7f5ac4c00700 1 -- 172.21.15.37:0/1482023148 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5ab400e040 con 0x7f5ac80fe830 2024-03-29T08:12:27.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:27.023+0000 7f5acf800700 1 -- 172.21.15.37:0/1482023148 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ac80ffc70 con 0x7f5ac80fe830 2024-03-29T08:12:27.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:27.023+0000 7f5ac4c00700 1 -- 172.21.15.37:0/1482023148 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f5ab4013420 con 0x7f5ac80fe830 2024-03-29T08:12:27.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:27.023+0000 7f5ac4c00700 1 -- 172.21.15.37:0/1482023148 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5ab401c630 con 0x7f5ac80fe830 2024-03-29T08:12:27.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:27.023+0000 7f5acf800700 1 -- 172.21.15.37:0/1482023148 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ac811e600 con 0x7f5ac80fe830 2024-03-29T08:12:27.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:27.025+0000 7f5acf800700 1 -- 172.21.15.37:0/1482023148 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ac8106a20 con 0x7f5ac80fe830 2024-03-29T08:12:27.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:27.026+0000 7f5ac4c00700 1 -- 172.21.15.37:0/1482023148 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 39) v1 ==== 90944+0+0 (secure 0 0 0) 0x7f5ab4013970 con 0x7f5ac80fe830 2024-03-29T08:12:27.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:27.028+0000 7f5ac4c00700 1 --2- 172.21.15.37:0/1482023148 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f5ab006cd40 0x7f5ab006f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:12:27.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:27.028+0000 7f5ac4c00700 1 -- 172.21.15.37:0/1482023148 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f5ab408d4c0 con 0x7f5ac80fe830 2024-03-29T08:12:27.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:27.028+0000 7f5acce00700 1 --2- 172.21.15.37:0/1482023148 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f5ab006cd40 0x7f5ab006f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:12:27.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:27.029+0000 7f5acce00700 1 --2- 172.21.15.37:0/1482023148 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f5ab006cd40 0x7f5ab006f1f0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f5abc009fd0 tx=0x7f5abc00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:12:27.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:27.032+0000 7f5ac4c00700 1 -- 172.21.15.37:0/1482023148 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f5ab40572e0 con 0x7f5ac80fe830 2024-03-29T08:12:27.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:27.328+0000 7f5acf800700 1 -- 172.21.15.37:0/1482023148 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f5ac8109dd0 con 0x7f5ac80fe830 2024-03-29T08:12:27.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:27.328+0000 7f5ac4c00700 1 -- 172.21.15.37:0/1482023148 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+849 (secure 0 0 0) 0x7f5ab405a900 con 0x7f5ac80fe830 2024-03-29T08:12:27.329 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-29T08:12:27.329 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": { 2024-03-29T08:12:27.329 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 17.2.7-789-g4213e17c (4213e17c4de67016ed14f55d22503f4923b3a901) quincy (stable)": 1, 2024-03-29T08:12:27.329 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 1 2024-03-29T08:12:27.329 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:12:27.329 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": { 2024-03-29T08:12:27.329 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 2 2024-03-29T08:12:27.329 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:12:27.329 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": { 2024-03-29T08:12:27.329 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 17.2.7-789-g4213e17c (4213e17c4de67016ed14f55d22503f4923b3a901) quincy (stable)": 6 2024-03-29T08:12:27.329 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:12:27.329 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": { 2024-03-29T08:12:27.329 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 17.2.7-789-g4213e17c (4213e17c4de67016ed14f55d22503f4923b3a901) quincy (stable)": 4 2024-03-29T08:12:27.329 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:12:27.330 INFO:teuthology.orchestra.run.smithi037.stdout: "overall": { 2024-03-29T08:12:27.330 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 17.2.7-789-g4213e17c (4213e17c4de67016ed14f55d22503f4923b3a901) quincy (stable)": 11, 2024-03-29T08:12:27.330 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 3 2024-03-29T08:12:27.330 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-29T08:12:27.330 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-29T08:12:27.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:27.330+0000 7f5acf800700 1 -- 172.21.15.37:0/1482023148 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f5ab006cd40 msgr2=0x7f5ab006f1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:12:27.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:27.330+0000 7f5acf800700 1 --2- 172.21.15.37:0/1482023148 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f5ab006cd40 0x7f5ab006f1f0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f5abc009fd0 tx=0x7f5abc00b040 comp rx=0 tx=0).stop 2024-03-29T08:12:27.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:27.330+0000 7f5acf800700 1 -- 172.21.15.37:0/1482023148 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ac80fe830 msgr2=0x7f5ac80ff4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:12:27.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:27.330+0000 7f5acf800700 1 --2- 172.21.15.37:0/1482023148 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ac80fe830 0x7f5ac80ff4b0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f5ab4000c00 tx=0x7f5ab4009e30 comp rx=0 tx=0).stop 2024-03-29T08:12:27.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:27.331+0000 7f5acf800700 1 -- 172.21.15.37:0/1482023148 shutdown_connections 2024-03-29T08:12:27.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:27.331+0000 7f5acf800700 1 --2- 172.21.15.37:0/1482023148 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f5ab006cd40 0x7f5ab006f1f0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:12:27.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:27.331+0000 7f5acf800700 1 --2- 172.21.15.37:0/1482023148 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5ac80fde90 0x7f5ac80fef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:12:27.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:27.331+0000 7f5acf800700 1 --2- 172.21.15.37:0/1482023148 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ac80fe830 0x7f5ac80ff4b0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:12:27.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:27.331+0000 7f5acf800700 1 -- 172.21.15.37:0/1482023148 >> 172.21.15.37:0/1482023148 conn(0x7f5ac8072bc0 msgr2=0x7f5ac80fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:12:27.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:27.331+0000 7f5acf800700 1 -- 172.21.15.37:0/1482023148 shutdown_connections 2024-03-29T08:12:27.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:27.331+0000 7f5acf800700 1 -- 172.21.15.37:0/1482023148 wait complete. 2024-03-29T08:12:27.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:27.466+0000 7f7a87e00700 1 -- 172.21.15.37:0/649732786 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7a800fe830 msgr2=0x7f7a80106f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:12:27.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:27.466+0000 7f7a87e00700 1 --2- 172.21.15.37:0/649732786 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7a800fe830 0x7f7a80106f00 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f7a6c004940 tx=0x7f7a6c009e30 comp rx=0 tx=0).stop 2024-03-29T08:12:27.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:27.466+0000 7f7a87e00700 1 -- 172.21.15.37:0/649732786 shutdown_connections 2024-03-29T08:12:27.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:27.466+0000 7f7a87e00700 1 --2- 172.21.15.37:0/649732786 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7a800fe830 0x7f7a80106f00 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:12:27.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:27.466+0000 7f7a87e00700 1 --2- 172.21.15.37:0/649732786 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7a800fde90 0x7f7a800fe260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:12:27.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:27.466+0000 7f7a87e00700 1 -- 172.21.15.37:0/649732786 >> 172.21.15.37:0/649732786 conn(0x7f7a80072bc0 msgr2=0x7f7a80072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:12:27.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:27.467+0000 7f7a87e00700 1 -- 172.21.15.37:0/649732786 shutdown_connections 2024-03-29T08:12:27.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:27.467+0000 7f7a87e00700 1 -- 172.21.15.37:0/649732786 wait complete. 2024-03-29T08:12:27.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:27.467+0000 7f7a87e00700 1 Processor -- start 2024-03-29T08:12:27.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:27.468+0000 7f7a87e00700 1 -- start start 2024-03-29T08:12:27.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:27.468+0000 7f7a87e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7a800fde90 0x7f7a80115d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:12:27.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:27.468+0000 7f7a87e00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7a800fe830 0x7f7a80116280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:12:27.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:27.468+0000 7f7a87e00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a80116960 con 0x7f7a800fde90 2024-03-29T08:12:27.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:27.468+0000 7f7a87e00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a8010fca0 con 0x7f7a800fe830 2024-03-29T08:12:27.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:27.468+0000 7f7a85400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7a800fde90 0x7f7a80115d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:12:27.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:27.468+0000 7f7a85400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7a800fde90 0x7f7a80115d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53010/0 (socket says 172.21.15.37:53010) 2024-03-29T08:12:27.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:27.468+0000 7f7a85400700 1 -- 172.21.15.37:0/2900394113 learned_addr learned my addr 172.21.15.37:0/2900394113 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:12:27.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:27.468+0000 7f7a84a00700 1 --2- 172.21.15.37:0/2900394113 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7a800fe830 0x7f7a80116280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:12:27.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:27.469+0000 7f7a85400700 1 -- 172.21.15.37:0/2900394113 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7a800fe830 msgr2=0x7f7a80116280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:12:27.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:27.469+0000 7f7a85400700 1 --2- 172.21.15.37:0/2900394113 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7a800fe830 0x7f7a80116280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:12:27.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:27.469+0000 7f7a85400700 1 -- 172.21.15.37:0/2900394113 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a6c005040 con 0x7f7a800fde90 2024-03-29T08:12:27.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:27.469+0000 7f7a85400700 1 --2- 172.21.15.37:0/2900394113 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7a800fde90 0x7f7a80115d40 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f7a7400b790 tx=0x7f7a74009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:12:27.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:27.469+0000 7f7a7d600700 1 -- 172.21.15.37:0/2900394113 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7a7400f6f0 con 0x7f7a800fde90 2024-03-29T08:12:27.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:27.469+0000 7f7a7d600700 1 -- 172.21.15.37:0/2900394113 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f7a7400fd00 con 0x7f7a800fde90 2024-03-29T08:12:27.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:27.469+0000 7f7a87e00700 1 -- 172.21.15.37:0/2900394113 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a8010ff20 con 0x7f7a800fde90 2024-03-29T08:12:27.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:27.469+0000 7f7a7d600700 1 -- 172.21.15.37:0/2900394113 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7a740103a0 con 0x7f7a800fde90 2024-03-29T08:12:27.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:27.469+0000 7f7a87e00700 1 -- 172.21.15.37:0/2900394113 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a80110460 con 0x7f7a800fde90 2024-03-29T08:12:27.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:27.471+0000 7f7a87e00700 1 -- 172.21.15.37:0/2900394113 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a80106a20 con 0x7f7a800fde90 2024-03-29T08:12:27.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:27.472+0000 7f7a7d600700 1 -- 172.21.15.37:0/2900394113 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 39) v1 ==== 90944+0+0 (secure 0 0 0) 0x7f7a7400f850 con 0x7f7a800fde90 2024-03-29T08:12:27.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:27.472+0000 7f7a7d600700 1 --2- 172.21.15.37:0/2900394113 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f7a6806cd40 0x7f7a6806f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:12:27.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:27.473+0000 7f7a7d600700 1 -- 172.21.15.37:0/2900394113 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f7a7408bf20 con 0x7f7a800fde90 2024-03-29T08:12:27.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:27.473+0000 7f7a84a00700 1 --2- 172.21.15.37:0/2900394113 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f7a6806cd40 0x7f7a6806f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:12:27.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:27.473+0000 7f7a84a00700 1 --2- 172.21.15.37:0/2900394113 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f7a6806cd40 0x7f7a6806f1f0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f7a801115e0 tx=0x7f7a6c009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:12:27.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:27.477+0000 7f7a7d600700 1 -- 172.21.15.37:0/2900394113 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f7a74059960 con 0x7f7a800fde90 2024-03-29T08:12:27.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:27.736+0000 7f7a87e00700 1 -- 172.21.15.37:0/2900394113 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f7a8004bed0 con 0x7f7a800fde90 2024-03-29T08:12:27.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:27.736+0000 7f7a7d600700 1 -- 172.21.15.37:0/2900394113 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 20 v20) v1 ==== 76+0+1858 (secure 0 0 0) 0x7f7a740594f0 con 0x7f7a800fde90 2024-03-29T08:12:27.736 INFO:teuthology.orchestra.run.smithi037.stdout:e20 2024-03-29T08:12:27.737 INFO:teuthology.orchestra.run.smithi037.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-29T08:12:27.737 INFO:teuthology.orchestra.run.smithi037.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T08:12:27.737 INFO:teuthology.orchestra.run.smithi037.stdout:legacy client fscid: 1 2024-03-29T08:12:27.737 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:12:27.737 INFO:teuthology.orchestra.run.smithi037.stdout:Filesystem 'cephfs' (1) 2024-03-29T08:12:27.737 INFO:teuthology.orchestra.run.smithi037.stdout:fs_name cephfs 2024-03-29T08:12:27.737 INFO:teuthology.orchestra.run.smithi037.stdout:epoch 20 2024-03-29T08:12:27.737 INFO:teuthology.orchestra.run.smithi037.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-03-29T08:12:27.737 INFO:teuthology.orchestra.run.smithi037.stdout:created 2024-03-29T08:03:29.241269+0000 2024-03-29T08:12:27.737 INFO:teuthology.orchestra.run.smithi037.stdout:modified 2024-03-29T08:11:27.181622+0000 2024-03-29T08:12:27.737 INFO:teuthology.orchestra.run.smithi037.stdout:tableserver 0 2024-03-29T08:12:27.737 INFO:teuthology.orchestra.run.smithi037.stdout:root 0 2024-03-29T08:12:27.737 INFO:teuthology.orchestra.run.smithi037.stdout:session_timeout 60 2024-03-29T08:12:27.737 INFO:teuthology.orchestra.run.smithi037.stdout:session_autoclose 300 2024-03-29T08:12:27.738 INFO:teuthology.orchestra.run.smithi037.stdout:max_file_size 1099511627776 2024-03-29T08:12:27.738 INFO:teuthology.orchestra.run.smithi037.stdout:max_xattr_size 65536 2024-03-29T08:12:27.738 INFO:teuthology.orchestra.run.smithi037.stdout:required_client_features {} 2024-03-29T08:12:27.738 INFO:teuthology.orchestra.run.smithi037.stdout:last_failure 0 2024-03-29T08:12:27.738 INFO:teuthology.orchestra.run.smithi037.stdout:last_failure_osd_epoch 47 2024-03-29T08:12:27.738 INFO:teuthology.orchestra.run.smithi037.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T08:12:27.738 INFO:teuthology.orchestra.run.smithi037.stdout:max_mds 2 2024-03-29T08:12:27.738 INFO:teuthology.orchestra.run.smithi037.stdout:in 0,1 2024-03-29T08:12:27.738 INFO:teuthology.orchestra.run.smithi037.stdout:up {0=14470,1=14482} 2024-03-29T08:12:27.738 INFO:teuthology.orchestra.run.smithi037.stdout:failed 2024-03-29T08:12:27.738 INFO:teuthology.orchestra.run.smithi037.stdout:damaged 2024-03-29T08:12:27.738 INFO:teuthology.orchestra.run.smithi037.stdout:stopped 2024-03-29T08:12:27.738 INFO:teuthology.orchestra.run.smithi037.stdout:data_pools [3] 2024-03-29T08:12:27.738 INFO:teuthology.orchestra.run.smithi037.stdout:metadata_pool 2 2024-03-29T08:12:27.738 INFO:teuthology.orchestra.run.smithi037.stdout:inline_data disabled 2024-03-29T08:12:27.738 INFO:teuthology.orchestra.run.smithi037.stdout:balancer 2024-03-29T08:12:27.739 INFO:teuthology.orchestra.run.smithi037.stdout:bal_rank_mask -1 2024-03-29T08:12:27.739 INFO:teuthology.orchestra.run.smithi037.stdout:standby_count_wanted 1 2024-03-29T08:12:27.739 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi037.mmwgxe{0:14470} state up:active seq 5 join_fscid=1 addr [v2:172.21.15.37:6826/1559618909,v1:172.21.15.37:6827/1559618909] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:12:27.739 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi037.iifsmh{1:14482} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.37:6828/2341530495,v1:172.21.15.37:6829/2341530495] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:12:27.739 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:12:27.739 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:12:27.739 INFO:teuthology.orchestra.run.smithi037.stdout:Standby daemons: 2024-03-29T08:12:27.739 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:12:27.739 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi148.qukpjh{-1:24319} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.148:6826/2641327912,v1:172.21.15.148:6827/2641327912] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:12:27.739 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi148.sqteih{-1:24327} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.148:6824/3252351726,v1:172.21.15.148:6825/3252351726] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:12:27.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:27.738+0000 7f7a87e00700 1 -- 172.21.15.37:0/2900394113 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f7a6806cd40 msgr2=0x7f7a6806f1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:12:27.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:27.738+0000 7f7a87e00700 1 --2- 172.21.15.37:0/2900394113 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f7a6806cd40 0x7f7a6806f1f0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f7a801115e0 tx=0x7f7a6c009e30 comp rx=0 tx=0).stop 2024-03-29T08:12:27.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:27.738+0000 7f7a87e00700 1 -- 172.21.15.37:0/2900394113 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7a800fde90 msgr2=0x7f7a80115d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:12:27.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:27.738+0000 7f7a87e00700 1 --2- 172.21.15.37:0/2900394113 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7a800fde90 0x7f7a80115d40 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f7a7400b790 tx=0x7f7a74009250 comp rx=0 tx=0).stop 2024-03-29T08:12:27.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:27.739+0000 7f7a87e00700 1 -- 172.21.15.37:0/2900394113 shutdown_connections 2024-03-29T08:12:27.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:27.739+0000 7f7a87e00700 1 --2- 172.21.15.37:0/2900394113 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f7a6806cd40 0x7f7a6806f1f0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:12:27.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:27.739+0000 7f7a87e00700 1 --2- 172.21.15.37:0/2900394113 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7a800fde90 0x7f7a80115d40 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:12:27.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:27.739+0000 7f7a87e00700 1 --2- 172.21.15.37:0/2900394113 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7a800fe830 0x7f7a80116280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:12:27.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:27.739+0000 7f7a87e00700 1 -- 172.21.15.37:0/2900394113 >> 172.21.15.37:0/2900394113 conn(0x7f7a80072bc0 msgr2=0x7f7a800fb740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:12:27.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:27.739+0000 7f7a87e00700 1 -- 172.21.15.37:0/2900394113 shutdown_connections 2024-03-29T08:12:27.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:27.739+0000 7f7a87e00700 1 -- 172.21.15.37:0/2900394113 wait complete. 2024-03-29T08:12:27.741 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 20 2024-03-29T08:12:27.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:27.876+0000 7f2d19c00700 1 -- 172.21.15.37:0/3142282098 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2d141037a0 msgr2=0x7f2d140fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:12:27.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:27.876+0000 7f2d19c00700 1 --2- 172.21.15.37:0/3142282098 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2d141037a0 0x7f2d140fce10 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f2d00004940 tx=0x7f2d00009e30 comp rx=0 tx=0).stop 2024-03-29T08:12:27.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:27.877+0000 7f2d19c00700 1 -- 172.21.15.37:0/3142282098 shutdown_connections 2024-03-29T08:12:27.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:27.877+0000 7f2d19c00700 1 --2- 172.21.15.37:0/3142282098 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2d141037a0 0x7f2d140fce10 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:12:27.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:27.877+0000 7f2d19c00700 1 --2- 172.21.15.37:0/3142282098 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2d14102e00 0x7f2d141031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:12:27.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:27.877+0000 7f2d19c00700 1 -- 172.21.15.37:0/3142282098 >> 172.21.15.37:0/3142282098 conn(0x7f2d140f89b0 msgr2=0x7f2d140fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:12:27.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:27.877+0000 7f2d19c00700 1 -- 172.21.15.37:0/3142282098 shutdown_connections 2024-03-29T08:12:27.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:27.877+0000 7f2d19c00700 1 -- 172.21.15.37:0/3142282098 wait complete. 2024-03-29T08:12:27.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:27.877+0000 7f2d19c00700 1 Processor -- start 2024-03-29T08:12:27.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:27.878+0000 7f2d19c00700 1 -- start start 2024-03-29T08:12:27.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:27.878+0000 7f2d19c00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2d14102e00 0x7f2d14115c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:12:27.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:27.878+0000 7f2d19c00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2d141037a0 0x7f2d141161a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:12:27.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:27.878+0000 7f2d19c00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d141167f0 con 0x7f2d141037a0 2024-03-29T08:12:27.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:27.878+0000 7f2d19c00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d1410fbc0 con 0x7f2d14102e00 2024-03-29T08:12:27.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:27.878+0000 7f2d12000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2d141037a0 0x7f2d141161a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:12:27.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:27.878+0000 7f2d12000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2d141037a0 0x7f2d141161a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53026/0 (socket says 172.21.15.37:53026) 2024-03-29T08:12:27.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:27.878+0000 7f2d12000700 1 -- 172.21.15.37:0/2985230427 learned_addr learned my addr 172.21.15.37:0/2985230427 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:12:27.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:27.878+0000 7f2d12a00700 1 --2- 172.21.15.37:0/2985230427 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2d14102e00 0x7f2d14115c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:12:27.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:27.879+0000 7f2d12000700 1 -- 172.21.15.37:0/2985230427 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2d14102e00 msgr2=0x7f2d14115c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:12:27.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:27.879+0000 7f2d12000700 1 --2- 172.21.15.37:0/2985230427 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2d14102e00 0x7f2d14115c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:12:27.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:27.879+0000 7f2d12000700 1 -- 172.21.15.37:0/2985230427 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d00005040 con 0x7f2d141037a0 2024-03-29T08:12:27.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:27.879+0000 7f2d12000700 1 --2- 172.21.15.37:0/2985230427 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2d141037a0 0x7f2d141161a0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f2d00009fd0 tx=0x7f2d00009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:12:27.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:27.879+0000 7f2d0aa00700 1 -- 172.21.15.37:0/2985230427 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2d0000e030 con 0x7f2d141037a0 2024-03-29T08:12:27.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:27.879+0000 7f2d0aa00700 1 -- 172.21.15.37:0/2985230427 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f2d000093b0 con 0x7f2d141037a0 2024-03-29T08:12:27.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:27.879+0000 7f2d19c00700 1 -- 172.21.15.37:0/2985230427 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d1410fe40 con 0x7f2d141037a0 2024-03-29T08:12:27.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:27.879+0000 7f2d0aa00700 1 -- 172.21.15.37:0/2985230427 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2d0001cc80 con 0x7f2d141037a0 2024-03-29T08:12:27.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:27.879+0000 7f2d19c00700 1 -- 172.21.15.37:0/2985230427 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d14110400 con 0x7f2d141037a0 2024-03-29T08:12:27.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:27.881+0000 7f2d19c00700 1 -- 172.21.15.37:0/2985230427 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d140fe550 con 0x7f2d141037a0 2024-03-29T08:12:27.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:27.882+0000 7f2d0aa00700 1 -- 172.21.15.37:0/2985230427 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 39) v1 ==== 90944+0+0 (secure 0 0 0) 0x7f2d0001c400 con 0x7f2d141037a0 2024-03-29T08:12:27.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:27.882+0000 7f2d0aa00700 1 --2- 172.21.15.37:0/2985230427 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f2cf806cd40 0x7f2cf806f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:12:27.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:27.883+0000 7f2d0aa00700 1 -- 172.21.15.37:0/2985230427 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f2d0008d9f0 con 0x7f2d141037a0 2024-03-29T08:12:27.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:27.883+0000 7f2d12a00700 1 --2- 172.21.15.37:0/2985230427 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f2cf806cd40 0x7f2cf806f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:12:27.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:27.883+0000 7f2d12a00700 1 --2- 172.21.15.37:0/2985230427 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f2cf806cd40 0x7f2cf806f1f0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f2d04009b80 tx=0x7f2d0400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:12:27.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:27.887+0000 7f2d0aa00700 1 -- 172.21.15.37:0/2985230427 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f2d0008fe60 con 0x7f2d141037a0 2024-03-29T08:12:27.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:27 smithi037 ceph-mon[142325]: Upgrade: Updating mon.smithi148 2024-03-29T08:12:27.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:27 smithi037 ceph-mon[142325]: Deploying daemon mon.smithi148 on smithi148 2024-03-29T08:12:27.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:27 smithi037 ceph-mon[142325]: from='client.34132 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:12:27.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:27 smithi037 ceph-mon[142325]: from='client.34136 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:12:27.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:27 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/1482023148' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:12:27.981 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:27 smithi148 ceph-mon[44992]: Upgrade: Updating mon.smithi148 2024-03-29T08:12:27.981 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:27 smithi148 ceph-mon[44992]: Deploying daemon mon.smithi148 on smithi148 2024-03-29T08:12:27.981 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:27 smithi148 ceph-mon[44992]: from='client.34132 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:12:27.981 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:27 smithi148 ceph-mon[44992]: from='client.34136 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:12:27.981 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:27 smithi148 ceph-mon[44992]: from='client.? 172.21.15.37:0/1482023148' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:12:27.981 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:27 smithi148 systemd[1]: Stopping Ceph mon.smithi148 for 99b492c2-eda1-11ee-b647-cb9ed24678a4... 2024-03-29T08:12:28.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:28.126+0000 7f2d19c00700 1 -- 172.21.15.37:0/2985230427 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f2d14109da0 con 0x7f2cf806cd40 2024-03-29T08:12:28.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:28.127+0000 7f2d0aa00700 1 -- 172.21.15.37:0/2985230427 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+368 (secure 0 0 0) 0x7f2d14109da0 con 0x7f2cf806cd40 2024-03-29T08:12:28.127 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-29T08:12:28.127 INFO:teuthology.orchestra.run.smithi037.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59", 2024-03-29T08:12:28.127 INFO:teuthology.orchestra.run.smithi037.stdout: "in_progress": true, 2024-03-29T08:12:28.127 INFO:teuthology.orchestra.run.smithi037.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-29T08:12:28.127 INFO:teuthology.orchestra.run.smithi037.stdout: "services_complete": [ 2024-03-29T08:12:28.127 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr" 2024-03-29T08:12:28.127 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-29T08:12:28.127 INFO:teuthology.orchestra.run.smithi037.stdout: "progress": "3/21 daemons upgraded", 2024-03-29T08:12:28.128 INFO:teuthology.orchestra.run.smithi037.stdout: "message": "Currently upgrading mon daemons", 2024-03-29T08:12:28.128 INFO:teuthology.orchestra.run.smithi037.stdout: "is_paused": false 2024-03-29T08:12:28.128 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-29T08:12:28.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:28.129+0000 7f2d19c00700 1 -- 172.21.15.37:0/2985230427 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f2cf806cd40 msgr2=0x7f2cf806f1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:12:28.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:28.129+0000 7f2d19c00700 1 --2- 172.21.15.37:0/2985230427 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f2cf806cd40 0x7f2cf806f1f0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f2d04009b80 tx=0x7f2d0400b040 comp rx=0 tx=0).stop 2024-03-29T08:12:28.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:28.129+0000 7f2d19c00700 1 -- 172.21.15.37:0/2985230427 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2d141037a0 msgr2=0x7f2d141161a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:12:28.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:28.129+0000 7f2d19c00700 1 --2- 172.21.15.37:0/2985230427 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2d141037a0 0x7f2d141161a0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f2d00009fd0 tx=0x7f2d00009e30 comp rx=0 tx=0).stop 2024-03-29T08:12:28.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:28.129+0000 7f2d19c00700 1 -- 172.21.15.37:0/2985230427 shutdown_connections 2024-03-29T08:12:28.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:28.129+0000 7f2d19c00700 1 --2- 172.21.15.37:0/2985230427 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f2cf806cd40 0x7f2cf806f1f0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:12:28.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:28.129+0000 7f2d19c00700 1 --2- 172.21.15.37:0/2985230427 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2d14102e00 0x7f2d14115c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:12:28.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:28.129+0000 7f2d19c00700 1 --2- 172.21.15.37:0/2985230427 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2d141037a0 0x7f2d141161a0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:12:28.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:28.129+0000 7f2d19c00700 1 -- 172.21.15.37:0/2985230427 >> 172.21.15.37:0/2985230427 conn(0x7f2d140f89b0 msgr2=0x7f2d140f99c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:12:28.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:28.129+0000 7f2d19c00700 1 -- 172.21.15.37:0/2985230427 shutdown_connections 2024-03-29T08:12:28.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:28.130+0000 7f2d19c00700 1 -- 172.21.15.37:0/2985230427 wait complete. 2024-03-29T08:12:28.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:28.265+0000 7f4cebc00700 1 -- 172.21.15.37:0/23184311 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4ce40fbc80 msgr2=0x7f4ce40fc050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:12:28.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:28.265+0000 7f4cebc00700 1 --2- 172.21.15.37:0/23184311 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4ce40fbc80 0x7f4ce40fc050 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f4cd8004940 tx=0x7f4cd8009e30 comp rx=0 tx=0).stop 2024-03-29T08:12:28.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:28.265+0000 7f4cebc00700 1 -- 172.21.15.37:0/23184311 shutdown_connections 2024-03-29T08:12:28.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:28.265+0000 7f4cebc00700 1 --2- 172.21.15.37:0/23184311 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4ce40fc590 0x7f4ce41092f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:12:28.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:28.265+0000 7f4cebc00700 1 --2- 172.21.15.37:0/23184311 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4ce40fbc80 0x7f4ce40fc050 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:12:28.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:28.265+0000 7f4cebc00700 1 -- 172.21.15.37:0/23184311 >> 172.21.15.37:0/23184311 conn(0x7f4ce4072bc0 msgr2=0x7f4ce4072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:12:28.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:28.265+0000 7f4cebc00700 1 -- 172.21.15.37:0/23184311 shutdown_connections 2024-03-29T08:12:28.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:28.265+0000 7f4cebc00700 1 -- 172.21.15.37:0/23184311 wait complete. 2024-03-29T08:12:28.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:28.266+0000 7f4cebc00700 1 Processor -- start 2024-03-29T08:12:28.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:28.266+0000 7f4cebc00700 1 -- start start 2024-03-29T08:12:28.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:28.266+0000 7f4cebc00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4ce40fbc80 0x7f4ce4115c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:12:28.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:28.266+0000 7f4cebc00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4ce40fc590 0x7f4ce41161c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:12:28.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:28.266+0000 7f4cebc00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ce41168a0 con 0x7f4ce40fbc80 2024-03-29T08:12:28.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:28.266+0000 7f4cebc00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ce410fc30 con 0x7f4ce40fc590 2024-03-29T08:12:28.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:28.267+0000 7f4ce3e00700 1 -- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4ce40fc590 msgr2=0x7f4ce41161c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.148:3300/0 2024-03-29T08:12:28.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:28.267+0000 7f4ce9200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4ce40fbc80 0x7f4ce4115c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:12:28.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:28.267+0000 7f4ce3e00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4ce40fc590 0x7f4ce41161c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-29T08:12:28.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:28.267+0000 7f4ce9200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4ce40fbc80 0x7f4ce4115c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53116/0 (socket says 172.21.15.37:53116) 2024-03-29T08:12:28.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:28.267+0000 7f4ce9200700 1 -- 172.21.15.37:0/10733652 learned_addr learned my addr 172.21.15.37:0/10733652 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:12:28.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:28.267+0000 7f4ce9200700 1 -- 172.21.15.37:0/10733652 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4ce40fc590 msgr2=0x7f4ce41161c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-29T08:12:28.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:28.267+0000 7f4ce9200700 1 --2- 172.21.15.37:0/10733652 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4ce40fc590 0x7f4ce41161c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:12:28.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:28.267+0000 7f4ce9200700 1 -- 172.21.15.37:0/10733652 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cd8005040 con 0x7f4ce40fbc80 2024-03-29T08:12:28.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:28.267+0000 7f4ce9200700 1 --2- 172.21.15.37:0/10733652 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4ce40fbc80 0x7f4ce4115c80 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f4cd8004090 tx=0x7f4cd8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:12:28.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:28.268+0000 7f4ce0c00700 1 -- 172.21.15.37:0/10733652 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4cd800e070 con 0x7f4ce40fbc80 2024-03-29T08:12:28.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:28.268+0000 7f4ce0c00700 1 -- 172.21.15.37:0/10733652 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f4cd8004290 con 0x7f4ce40fbc80 2024-03-29T08:12:28.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:28.268+0000 7f4cebc00700 1 -- 172.21.15.37:0/10733652 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ce410fe50 con 0x7f4ce40fbc80 2024-03-29T08:12:28.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:28.268+0000 7f4ce0c00700 1 -- 172.21.15.37:0/10733652 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4cd8013c80 con 0x7f4ce40fbc80 2024-03-29T08:12:28.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:28.268+0000 7f4cebc00700 1 -- 172.21.15.37:0/10733652 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ce41103b0 con 0x7f4ce40fbc80 2024-03-29T08:12:28.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:28.269+0000 7f4cebc00700 1 -- 172.21.15.37:0/10733652 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ce4065e30 con 0x7f4ce40fbc80 2024-03-29T08:12:28.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:28.270+0000 7f4ce0c00700 1 -- 172.21.15.37:0/10733652 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 39) v1 ==== 90944+0+0 (secure 0 0 0) 0x7f4cd80134b0 con 0x7f4ce40fbc80 2024-03-29T08:12:28.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:28.270+0000 7f4ce0c00700 1 --2- 172.21.15.37:0/10733652 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f4ccc06cce0 0x7f4ccc06f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:12:28.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:28.271+0000 7f4ce0c00700 1 -- 172.21.15.37:0/10733652 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f4cd808d960 con 0x7f4ce40fbc80 2024-03-29T08:12:28.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:28.271+0000 7f4ce3e00700 1 --2- 172.21.15.37:0/10733652 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f4ccc06cce0 0x7f4ccc06f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:12:28.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:28.271+0000 7f4ce3e00700 1 --2- 172.21.15.37:0/10733652 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f4ccc06cce0 0x7f4ccc06f190 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f4cd0006fd0 tx=0x7f4cd0006df0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:12:28.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:28.276+0000 7f4ce0c00700 1 -- 172.21.15.37:0/10733652 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f4cd805b2b0 con 0x7f4ce40fbc80 2024-03-29T08:12:28.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:27 smithi148 ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4-mon-smithi148[44940]: 2024-03-29T08:12:27.978+0000 7f4ec5200700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi148 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-29T08:12:28.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:27 smithi148 ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4-mon-smithi148[44940]: 2024-03-29T08:12:27.978+0000 7f4ec5200700 -1 mon.smithi148@1(peon) e2 *** Got Signal Terminated *** 2024-03-29T08:12:28.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:28.572+0000 7f4cebc00700 1 -- 172.21.15.37:0/10733652 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f4ce410c120 con 0x7f4ce40fbc80 2024-03-29T08:12:28.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:28.572+0000 7f4ce0c00700 1 -- 172.21.15.37:0/10733652 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f4cd805ae40 con 0x7f4ce40fbc80 2024-03-29T08:12:28.572 INFO:teuthology.orchestra.run.smithi037.stdout:HEALTH_OK 2024-03-29T08:12:28.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:28.574+0000 7f4cebc00700 1 -- 172.21.15.37:0/10733652 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f4ccc06cce0 msgr2=0x7f4ccc06f190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:12:28.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:28.574+0000 7f4cebc00700 1 --2- 172.21.15.37:0/10733652 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f4ccc06cce0 0x7f4ccc06f190 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f4cd0006fd0 tx=0x7f4cd0006df0 comp rx=0 tx=0).stop 2024-03-29T08:12:28.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:28.574+0000 7f4cebc00700 1 -- 172.21.15.37:0/10733652 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4ce40fbc80 msgr2=0x7f4ce4115c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:12:28.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:28.574+0000 7f4cebc00700 1 --2- 172.21.15.37:0/10733652 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4ce40fbc80 0x7f4ce4115c80 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f4cd8004090 tx=0x7f4cd8009e30 comp rx=0 tx=0).stop 2024-03-29T08:12:28.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:28.575+0000 7f4cebc00700 1 -- 172.21.15.37:0/10733652 shutdown_connections 2024-03-29T08:12:28.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:28.575+0000 7f4cebc00700 1 --2- 172.21.15.37:0/10733652 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f4ccc06cce0 0x7f4ccc06f190 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:12:28.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:28.575+0000 7f4cebc00700 1 --2- 172.21.15.37:0/10733652 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4ce40fbc80 0x7f4ce4115c80 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:12:28.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:28.575+0000 7f4cebc00700 1 --2- 172.21.15.37:0/10733652 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4ce40fc590 0x7f4ce41161c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:12:28.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:28.575+0000 7f4cebc00700 1 -- 172.21.15.37:0/10733652 >> 172.21.15.37:0/10733652 conn(0x7f4ce4072bc0 msgr2=0x7f4ce4105b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:12:28.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:28.575+0000 7f4cebc00700 1 -- 172.21.15.37:0/10733652 shutdown_connections 2024-03-29T08:12:28.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:28.575+0000 7f4cebc00700 1 -- 172.21.15.37:0/10733652 wait complete. 2024-03-29T08:12:29.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:28 smithi148 bash[106500]: ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4-mon-smithi148 2024-03-29T08:12:29.354 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:29 smithi148 systemd[1]: ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4@mon.smithi148.service: Succeeded. 2024-03-29T08:12:29.354 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:29 smithi148 systemd[1]: Stopped Ceph mon.smithi148 for 99b492c2-eda1-11ee-b647-cb9ed24678a4. 2024-03-29T08:12:30.557 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:29 smithi148 systemd[1]: Starting Ceph mon.smithi148 for 99b492c2-eda1-11ee-b647-cb9ed24678a4... 2024-03-29T08:12:30.557 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: set uid:gid to 167:167 (ceph:ceph) 2024-03-29T08:12:30.557 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable), process ceph-mon, pid 7 2024-03-29T08:12:30.557 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: pidfile_write: ignore empty --pid-file 2024-03-29T08:12:30.557 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: load: jerasure load: lrc 2024-03-29T08:12:30.557 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: RocksDB version: 7.9.2 2024-03-29T08:12:30.557 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: Git sha 0 2024-03-29T08:12:30.558 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: Compile date 2024-03-26 15:09:52 2024-03-29T08:12:30.558 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: DB SUMMARY 2024-03-29T08:12:30.558 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: DB Session ID: QXT6GM8XXFHMRPG33ZB2 2024-03-29T08:12:30.558 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: CURRENT file: CURRENT 2024-03-29T08:12:30.558 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: IDENTITY file: IDENTITY 2024-03-29T08:12:30.558 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: MANIFEST file: MANIFEST-000009 size: 1072 Bytes 2024-03-29T08:12:30.558 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi148/store.db dir, Total Num: 1, files: 000027.sst 2024-03-29T08:12:30.558 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi148/store.db: 000025.log size: 8130831 ; 2024-03-29T08:12:30.558 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: Options.error_if_exists: 0 2024-03-29T08:12:30.558 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: Options.create_if_missing: 0 2024-03-29T08:12:30.558 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: Options.paranoid_checks: 1 2024-03-29T08:12:30.559 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: Options.flush_verify_memtable_count: 1 2024-03-29T08:12:30.559 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-03-29T08:12:30.559 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-03-29T08:12:30.559 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: Options.env: 0x55efb5adc720 2024-03-29T08:12:30.559 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: Options.fs: PosixFileSystem 2024-03-29T08:12:30.559 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: Options.info_log: 0x55efb81313c0 2024-03-29T08:12:30.559 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: Options.max_file_opening_threads: 16 2024-03-29T08:12:30.559 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: Options.statistics: (nil) 2024-03-29T08:12:30.559 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: Options.use_fsync: 0 2024-03-29T08:12:30.559 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: Options.max_log_file_size: 0 2024-03-29T08:12:30.559 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-03-29T08:12:30.559 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: Options.log_file_time_to_roll: 0 2024-03-29T08:12:30.559 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: Options.keep_log_file_num: 1000 2024-03-29T08:12:30.559 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: Options.recycle_log_file_num: 0 2024-03-29T08:12:30.560 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: Options.allow_fallocate: 1 2024-03-29T08:12:30.560 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: Options.allow_mmap_reads: 0 2024-03-29T08:12:30.560 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: Options.allow_mmap_writes: 0 2024-03-29T08:12:30.560 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: Options.use_direct_reads: 0 2024-03-29T08:12:30.560 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-03-29T08:12:30.560 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: Options.create_missing_column_families: 0 2024-03-29T08:12:30.560 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: Options.db_log_dir: 2024-03-29T08:12:30.560 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: Options.wal_dir: 2024-03-29T08:12:30.560 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: Options.table_cache_numshardbits: 6 2024-03-29T08:12:30.560 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: Options.WAL_ttl_seconds: 0 2024-03-29T08:12:30.560 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: Options.WAL_size_limit_MB: 0 2024-03-29T08:12:30.560 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-03-29T08:12:30.560 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-03-29T08:12:30.560 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: Options.is_fd_close_on_exec: 1 2024-03-29T08:12:30.561 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: Options.advise_random_on_open: 1 2024-03-29T08:12:30.561 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: Options.db_write_buffer_size: 0 2024-03-29T08:12:30.561 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: Options.write_buffer_manager: 0x55efb73a8140 2024-03-29T08:12:30.561 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-03-29T08:12:30.561 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-03-29T08:12:30.561 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: Options.use_adaptive_mutex: 0 2024-03-29T08:12:30.561 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: Options.rate_limiter: (nil) 2024-03-29T08:12:30.561 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-03-29T08:12:30.561 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: Options.wal_recovery_mode: 2 2024-03-29T08:12:30.562 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: Options.enable_thread_tracking: 0 2024-03-29T08:12:30.562 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: Options.enable_pipelined_write: 0 2024-03-29T08:12:30.562 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: Options.unordered_write: 0 2024-03-29T08:12:30.562 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-03-29T08:12:30.562 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-03-29T08:12:30.562 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-03-29T08:12:30.562 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-03-29T08:12:30.562 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: Options.row_cache: None 2024-03-29T08:12:30.562 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: Options.wal_filter: None 2024-03-29T08:12:30.562 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-03-29T08:12:30.563 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: Options.allow_ingest_behind: 0 2024-03-29T08:12:30.563 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: Options.two_write_queues: 0 2024-03-29T08:12:30.563 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: Options.manual_wal_flush: 0 2024-03-29T08:12:30.563 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: Options.wal_compression: 0 2024-03-29T08:12:30.563 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: Options.atomic_flush: 0 2024-03-29T08:12:30.563 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-03-29T08:12:30.563 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: Options.persist_stats_to_disk: 0 2024-03-29T08:12:30.563 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: Options.write_dbid_to_manifest: 0 2024-03-29T08:12:30.563 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: Options.log_readahead_size: 0 2024-03-29T08:12:30.563 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-03-29T08:12:30.563 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: Options.best_efforts_recovery: 0 2024-03-29T08:12:30.563 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-03-29T08:12:30.563 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-03-29T08:12:30.563 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: Options.allow_data_in_errors: 0 2024-03-29T08:12:30.563 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: Options.db_host_id: __hostname__ 2024-03-29T08:12:30.564 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: Options.enforce_single_del_contracts: true 2024-03-29T08:12:30.564 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: Options.max_background_jobs: 2 2024-03-29T08:12:30.564 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: Options.max_background_compactions: -1 2024-03-29T08:12:30.564 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: Options.max_subcompactions: 1 2024-03-29T08:12:30.564 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-03-29T08:12:30.564 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-03-29T08:12:30.564 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: Options.delayed_write_rate : 16777216 2024-03-29T08:12:30.564 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: Options.max_total_wal_size: 0 2024-03-29T08:12:30.564 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-03-29T08:12:30.564 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: Options.stats_dump_period_sec: 600 2024-03-29T08:12:30.564 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: Options.stats_persist_period_sec: 600 2024-03-29T08:12:30.564 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-03-29T08:12:30.564 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: Options.max_open_files: -1 2024-03-29T08:12:30.564 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: Options.bytes_per_sync: 0 2024-03-29T08:12:30.564 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: Options.wal_bytes_per_sync: 0 2024-03-29T08:12:30.565 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: Options.strict_bytes_per_sync: 0 2024-03-29T08:12:30.565 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: Options.compaction_readahead_size: 0 2024-03-29T08:12:30.565 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: Options.max_background_flushes: -1 2024-03-29T08:12:30.565 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: Compression algorithms supported: 2024-03-29T08:12:30.565 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-03-29T08:12:30.565 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: kZSTD supported: 0 2024-03-29T08:12:30.565 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: kXpressCompression supported: 0 2024-03-29T08:12:30.565 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: kLZ4HCCompression supported: 1 2024-03-29T08:12:30.565 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: kZlibCompression supported: 1 2024-03-29T08:12:30.565 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: kSnappyCompression supported: 1 2024-03-29T08:12:30.565 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: kLZ4Compression supported: 1 2024-03-29T08:12:30.565 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: kBZip2Compression supported: 0 2024-03-29T08:12:30.565 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: Fast CRC32 supported: Supported on x86 2024-03-29T08:12:30.565 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: DMutex implementation: pthread_mutex_t 2024-03-29T08:12:30.566 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi148/store.db/MANIFEST-000009 2024-03-29T08:12:30.566 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-03-29T08:12:30.566 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-03-29T08:12:30.566 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: Options.merge_operator: 2024-03-29T08:12:30.566 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: Options.compaction_filter: None 2024-03-29T08:12:30.566 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: Options.compaction_filter_factory: None 2024-03-29T08:12:30.566 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: Options.sst_partitioner_factory: None 2024-03-29T08:12:30.566 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: Options.memtable_factory: SkipListFactory 2024-03-29T08:12:30.566 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: Options.table_factory: BlockBasedTable 2024-03-29T08:12:30.566 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55efb81314e0) 2024-03-29T08:12:30.566 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: cache_index_and_filter_blocks: 1 2024-03-29T08:12:30.566 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-03-29T08:12:30.567 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-03-29T08:12:30.567 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: pin_top_level_index_and_filter: 1 2024-03-29T08:12:30.567 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: index_type: 0 2024-03-29T08:12:30.567 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: data_block_index_type: 0 2024-03-29T08:12:30.567 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: index_shortening: 1 2024-03-29T08:12:30.567 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: data_block_hash_table_util_ratio: 0.750000 2024-03-29T08:12:30.567 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: checksum: 4 2024-03-29T08:12:30.567 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: no_block_cache: 0 2024-03-29T08:12:30.567 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: block_cache: 0x55efb74491f0 2024-03-29T08:12:30.567 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: block_cache_name: BinnedLRUCache 2024-03-29T08:12:30.567 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: block_cache_options: 2024-03-29T08:12:30.567 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: capacity : 536870912 2024-03-29T08:12:30.567 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: num_shard_bits : 4 2024-03-29T08:12:30.567 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: strict_capacity_limit : 0 2024-03-29T08:12:30.567 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: high_pri_pool_ratio: 0.000 2024-03-29T08:12:30.567 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: block_cache_compressed: (nil) 2024-03-29T08:12:30.568 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: persistent_cache: (nil) 2024-03-29T08:12:30.568 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: block_size: 4096 2024-03-29T08:12:30.568 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: block_size_deviation: 10 2024-03-29T08:12:30.568 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: block_restart_interval: 16 2024-03-29T08:12:30.568 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: index_block_restart_interval: 1 2024-03-29T08:12:30.568 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: metadata_block_size: 4096 2024-03-29T08:12:30.568 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: partition_filters: 0 2024-03-29T08:12:30.568 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: use_delta_encoding: 1 2024-03-29T08:12:30.568 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: filter_policy: bloomfilter 2024-03-29T08:12:30.568 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: whole_key_filtering: 1 2024-03-29T08:12:30.568 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: verify_compression: 0 2024-03-29T08:12:30.568 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: read_amp_bytes_per_bit: 0 2024-03-29T08:12:30.568 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: format_version: 5 2024-03-29T08:12:30.568 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: enable_index_compression: 1 2024-03-29T08:12:30.568 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: block_align: 0 2024-03-29T08:12:30.569 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: max_auto_readahead_size: 262144 2024-03-29T08:12:30.569 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: prepopulate_block_cache: 0 2024-03-29T08:12:30.569 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: initial_auto_readahead_size: 8192 2024-03-29T08:12:30.569 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: num_file_reads_for_auto_readahead: 2 2024-03-29T08:12:30.569 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: Options.write_buffer_size: 33554432 2024-03-29T08:12:30.569 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: Options.max_write_buffer_number: 2 2024-03-29T08:12:30.569 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: Options.compression: NoCompression 2024-03-29T08:12:30.569 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: Options.bottommost_compression: Disabled 2024-03-29T08:12:30.569 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: Options.prefix_extractor: nullptr 2024-03-29T08:12:30.569 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-03-29T08:12:30.569 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: Options.num_levels: 7 2024-03-29T08:12:30.569 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-03-29T08:12:30.569 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-03-29T08:12:30.569 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-03-29T08:12:30.569 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-03-29T08:12:30.570 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-03-29T08:12:30.570 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-03-29T08:12:30.570 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-03-29T08:12:30.570 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-03-29T08:12:30.570 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-03-29T08:12:30.570 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-03-29T08:12:30.570 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-03-29T08:12:30.570 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-03-29T08:12:30.570 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: Options.compression_opts.window_bits: -14 2024-03-29T08:12:30.570 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: Options.compression_opts.level: 32767 2024-03-29T08:12:30.570 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: Options.compression_opts.strategy: 0 2024-03-29T08:12:30.570 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-03-29T08:12:30.570 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-03-29T08:12:30.570 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-03-29T08:12:30.571 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-03-29T08:12:30.571 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: Options.compression_opts.enabled: false 2024-03-29T08:12:30.571 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-03-29T08:12:30.571 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-03-29T08:12:30.571 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-03-29T08:12:30.571 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-03-29T08:12:30.571 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: Options.target_file_size_base: 67108864 2024-03-29T08:12:30.571 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: Options.target_file_size_multiplier: 1 2024-03-29T08:12:30.571 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-03-29T08:12:30.571 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-03-29T08:12:30.571 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-03-29T08:12:30.571 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-03-29T08:12:30.571 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-03-29T08:12:30.571 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-03-29T08:12:30.571 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-03-29T08:12:30.572 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-03-29T08:12:30.572 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-03-29T08:12:30.572 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-03-29T08:12:30.572 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-03-29T08:12:30.572 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-03-29T08:12:30.572 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-03-29T08:12:30.572 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: Options.arena_block_size: 1048576 2024-03-29T08:12:30.572 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-03-29T08:12:30.572 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-03-29T08:12:30.572 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: Options.disable_auto_compactions: 0 2024-03-29T08:12:30.573 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-03-29T08:12:30.573 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-03-29T08:12:30.573 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-03-29T08:12:30.573 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-03-29T08:12:30.573 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-03-29T08:12:30.573 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-03-29T08:12:30.573 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-03-29T08:12:30.573 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-03-29T08:12:30.574 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-03-29T08:12:30.574 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-03-29T08:12:30.574 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: Options.table_properties_collectors: 2024-03-29T08:12:30.574 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: Options.inplace_update_support: 0 2024-03-29T08:12:30.574 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: Options.inplace_update_num_locks: 10000 2024-03-29T08:12:30.574 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-03-29T08:12:30.574 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-03-29T08:12:30.574 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: Options.memtable_huge_page_size: 0 2024-03-29T08:12:30.574 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: Options.bloom_locality: 0 2024-03-29T08:12:30.574 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: Options.max_successive_merges: 0 2024-03-29T08:12:30.574 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: Options.optimize_filters_for_hits: 0 2024-03-29T08:12:30.575 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: Options.paranoid_file_checks: 0 2024-03-29T08:12:30.575 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: Options.force_consistency_checks: 1 2024-03-29T08:12:30.575 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: Options.report_bg_io_stats: 0 2024-03-29T08:12:30.575 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: Options.ttl: 2592000 2024-03-29T08:12:30.575 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: Options.periodic_compaction_seconds: 0 2024-03-29T08:12:30.575 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-03-29T08:12:30.575 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-03-29T08:12:30.575 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: Options.enable_blob_files: false 2024-03-29T08:12:30.575 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: Options.min_blob_size: 0 2024-03-29T08:12:30.575 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: Options.blob_file_size: 268435456 2024-03-29T08:12:30.575 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: Options.blob_compression_type: NoCompression 2024-03-29T08:12:30.575 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: Options.enable_blob_garbage_collection: false 2024-03-29T08:12:30.575 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-03-29T08:12:30.575 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-03-29T08:12:30.575 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-03-29T08:12:30.575 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: Options.blob_file_starting_level: 0 2024-03-29T08:12:30.576 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-03-29T08:12:30.576 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: [table/block_based/block_based_table_reader.cc:721] At least one SST file opened without unique ID to verify: 27.sst 2024-03-29T08:12:30.576 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-03-29T08:12:30.576 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi148/store.db/MANIFEST-000009 succeeded,manifest_file_number is 9, next_file_number is 29, last_sequence is 12008, log_number is 25,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-03-29T08:12:30.576 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 25 2024-03-29T08:12:30.576 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: d599463c-0d6d-4cac-b505-82d64089439a 2024-03-29T08:12:30.576 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711699950110323, "job": 1, "event": "recovery_started", "wal_files": [25]} 2024-03-29T08:12:30.576 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #25 mode 2 2024-03-29T08:12:30.576 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711699950139451, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 30, "file_size": 4868383, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 12013, "largest_seqno": 16032, "table_properties": {"data_size": 4853282, "index_size": 9792, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 4293, "raw_key_size": 38110, "raw_average_key_size": 22, "raw_value_size": 4819050, "raw_average_value_size": 2818, "num_data_blocks": 443, "num_entries": 1710, "num_filter_entries": 1710, "num_deletions": 260, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1711699950, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d599463c-0d6d-4cac-b505-82d64089439a", "db_session_id": "QXT6GM8XXFHMRPG33ZB2", "orig_file_number": 30, "seqno_to_time_mapping": "N/A"}} 2024-03-29T08:12:30.576 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711699950139616, "job": 1, "event": "recovery_finished"} 2024-03-29T08:12:30.576 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: [db/version_set.cc:5047] Creating manifest 32 2024-03-29T08:12:30.576 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-03-29T08:12:30.576 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-smithi148/store.db/000025.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-03-29T08:12:30.576 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55efb74e6000 2024-03-29T08:12:30.577 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: DB pointer 0x55efb74d4000 2024-03-29T08:12:30.577 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-03-29T08:12:30.577 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-03-29T08:12:30.577 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: ** DB Stats ** 2024-03-29T08:12:30.577 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-29T08:12:30.577 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-03-29T08:12:30.577 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-29T08:12:30.577 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-29T08:12:30.577 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-03-29T08:12:30.577 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-29T08:12:30.577 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-29T08:12:30.578 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: 2024-03-29T08:12:30.578 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: ** Compaction Stats [default] ** 2024-03-29T08:12:30.578 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-29T08:12:30.578 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-03-29T08:12:30.578 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: L0 1/0 4.64 MB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 507.7 0.01 0.00 1 0.009 0 0 0.0 0.0 2024-03-29T08:12:30.578 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: L6 1/0 6.43 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 2024-03-29T08:12:30.578 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: Sum 2/0 11.08 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 507.7 0.01 0.00 1 0.009 0 0 0.0 0.0 2024-03-29T08:12:30.578 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 507.7 0.01 0.00 1 0.009 0 0 0.0 0.0 2024-03-29T08:12:30.578 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: 2024-03-29T08:12:30.578 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: ** Compaction Stats [default] ** 2024-03-29T08:12:30.578 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-29T08:12:30.578 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-29T08:12:30.578 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 507.7 0.01 0.00 1 0.009 0 0 0.0 0.0 2024-03-29T08:12:30.578 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: 2024-03-29T08:12:30.579 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-03-29T08:12:30.579 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: 2024-03-29T08:12:30.579 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-29T08:12:30.579 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: Flush(GB): cumulative 0.005, interval 0.005 2024-03-29T08:12:30.579 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-03-29T08:12:30.579 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-03-29T08:12:30.579 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-03-29T08:12:30.579 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: AddFile(Keys): cumulative 0, interval 0 2024-03-29T08:12:30.579 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: Cumulative compaction: 0.00 GB write, 133.38 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-29T08:12:30.579 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: Interval compaction: 0.00 GB write, 133.38 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-29T08:12:30.579 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-03-29T08:12:30.579 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: Block cache BinnedLRUCache@0x55efb74491f0#7 capacity: 512.00 MB usage: 14.61 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.1e-05 secs_since: 0 2024-03-29T08:12:30.579 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,4.53 KB,0.000864267%) IndexBlock(1,10.08 KB,0.00192225%) Misc(1,0.00 KB,0%) 2024-03-29T08:12:30.580 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: 2024-03-29T08:12:30.580 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: ** File Read Latency Histogram By Level [default] ** 2024-03-29T08:12:30.580 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: starting mon.smithi148 rank 1 at public addrs [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] at bind addrs [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi148 fsid 99b492c2-eda1-11ee-b647-cb9ed24678a4 2024-03-29T08:12:30.580 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: mon.smithi148@-1(???) e2 preinit fsid 99b492c2-eda1-11ee-b647-cb9ed24678a4 2024-03-29T08:12:30.580 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: mon.smithi148@-1(???).mds e20 new map 2024-03-29T08:12:30.580 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: mon.smithi148@-1(???).mds e20 print_map 2024-03-29T08:12:30.580 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: e20 2024-03-29T08:12:30.580 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-03-29T08:12:30.580 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T08:12:30.580 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: legacy client fscid: 1 2024-03-29T08:12:30.580 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: 2024-03-29T08:12:30.580 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: Filesystem 'cephfs' (1) 2024-03-29T08:12:30.580 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: fs_name cephfs 2024-03-29T08:12:30.580 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: epoch 20 2024-03-29T08:12:30.580 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: flags 12 joinable allow_snaps allow_multimds_snaps 2024-03-29T08:12:30.581 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: created 2024-03-29T08:03:29.241269+0000 2024-03-29T08:12:30.581 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: modified 2024-03-29T08:11:27.181622+0000 2024-03-29T08:12:30.581 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: tableserver 0 2024-03-29T08:12:30.581 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: root 0 2024-03-29T08:12:30.581 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: session_timeout 60 2024-03-29T08:12:30.581 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: session_autoclose 300 2024-03-29T08:12:30.581 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: max_file_size 1099511627776 2024-03-29T08:12:30.581 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: max_xattr_size 65536 2024-03-29T08:12:30.581 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: required_client_features {} 2024-03-29T08:12:30.582 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: last_failure 0 2024-03-29T08:12:30.582 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: last_failure_osd_epoch 47 2024-03-29T08:12:30.582 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T08:12:30.582 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: max_mds 2 2024-03-29T08:12:30.582 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: in 0,1 2024-03-29T08:12:30.582 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: up {0=14470,1=14482} 2024-03-29T08:12:30.582 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: failed 2024-03-29T08:12:30.582 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: damaged 2024-03-29T08:12:30.582 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: stopped 2024-03-29T08:12:30.582 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: data_pools [3] 2024-03-29T08:12:30.582 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: metadata_pool 2 2024-03-29T08:12:30.582 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: inline_data disabled 2024-03-29T08:12:30.583 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: balancer 2024-03-29T08:12:30.583 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: bal_rank_mask -1 2024-03-29T08:12:30.583 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: standby_count_wanted 1 2024-03-29T08:12:30.583 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: [mds.cephfs.smithi037.mmwgxe{0:14470} state up:active seq 5 join_fscid=1 addr [v2:172.21.15.37:6826/1559618909,v1:172.21.15.37:6827/1559618909] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:12:30.583 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: [mds.cephfs.smithi037.iifsmh{1:14482} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.37:6828/2341530495,v1:172.21.15.37:6829/2341530495] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:12:30.583 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: 2024-03-29T08:12:30.583 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: 2024-03-29T08:12:30.583 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: Standby daemons: 2024-03-29T08:12:30.583 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: 2024-03-29T08:12:30.583 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: [mds.cephfs.smithi148.qukpjh{-1:24319} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.148:6826/2641327912,v1:172.21.15.148:6827/2641327912] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:12:30.583 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout: [mds.cephfs.smithi148.sqteih{-1:24327} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.148:6824/3252351726,v1:172.21.15.148:6825/3252351726] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:12:30.583 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: mon.smithi148@-1(???).osd e50 crush map has features 3314933000852226048, adjusting msgr requires 2024-03-29T08:12:30.583 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: mon.smithi148@-1(???).osd e50 crush map has features 288514051259236352, adjusting msgr requires 2024-03-29T08:12:30.583 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: mon.smithi148@-1(???).osd e50 crush map has features 288514051259236352, adjusting msgr requires 2024-03-29T08:12:30.584 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: mon.smithi148@-1(???).osd e50 crush map has features 288514051259236352, adjusting msgr requires 2024-03-29T08:12:30.584 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 ceph-mon[106692]: mon.smithi148@-1(???).paxosservice(auth 1..23) refresh upgraded, format 0 -> 3 2024-03-29T08:12:30.584 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 bash[106654]: ad4b2c5748beb3504c5d83be4686343f22f3860150b930d30a5cfb5de3949bee 2024-03-29T08:12:30.584 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:30 smithi148 systemd[1]: Started Ceph mon.smithi148 for 99b492c2-eda1-11ee-b647-cb9ed24678a4. 2024-03-29T08:12:31.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:31 smithi148 ceph-mon[106692]: from='client.34140 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:12:31.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:31 smithi148 ceph-mon[106692]: pgmap v12: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 17 KiB/s rd, 0 B/s wr, 9 op/s 2024-03-29T08:12:31.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:31 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/2900394113' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T08:12:31.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:31 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/10733652' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T08:12:31.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:31 smithi037 ceph-mon[142325]: from='client.34140 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:12:31.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:31 smithi037 ceph-mon[142325]: pgmap v12: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 17 KiB/s rd, 0 B/s wr, 9 op/s 2024-03-29T08:12:31.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:31 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/2900394113' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T08:12:31.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:31 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/10733652' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T08:12:32.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:32 smithi148 ceph-mon[106692]: overall HEALTH_OK 2024-03-29T08:12:32.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:32 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "mon metadata", "id": "smithi037"}]: dispatch 2024-03-29T08:12:32.807 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:32 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "mon metadata", "id": "smithi148"}]: dispatch 2024-03-29T08:12:32.807 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:32 smithi148 ceph-mon[106692]: mon.smithi037 calling monitor election 2024-03-29T08:12:32.807 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:32 smithi148 ceph-mon[106692]: mon.smithi148 calling monitor election 2024-03-29T08:12:32.807 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:32 smithi148 ceph-mon[106692]: mon.smithi037 is new leader, mons smithi037,smithi148 in quorum (ranks 0,1) 2024-03-29T08:12:32.807 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:32 smithi148 ceph-mon[106692]: monmap e3: 2 mons at {smithi037=[v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0],smithi148=[v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-29T08:12:32.807 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:32 smithi148 ceph-mon[106692]: fsmap cephfs:2 {0=cephfs.smithi037.mmwgxe=up:active,1=cephfs.smithi037.iifsmh=up:active} 2 up:standby 2024-03-29T08:12:32.807 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:32 smithi148 ceph-mon[106692]: osdmap e50: 6 total, 6 up, 6 in 2024-03-29T08:12:32.807 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:32 smithi148 ceph-mon[106692]: mgrmap e39: smithi037.wjikii(active, since 21s), standbys: smithi148.onhgvm 2024-03-29T08:12:32.807 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:32 smithi148 ceph-mon[106692]: overall HEALTH_OK 2024-03-29T08:12:32.807 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:32 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:12:32.807 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:32 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:12:32.807 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:32 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T08:12:32.807 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:32 smithi148 ceph-mon[106692]: pgmap v14: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:12:32.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:32 smithi037 ceph-mon[142325]: overall HEALTH_OK 2024-03-29T08:12:32.928 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:32 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "mon metadata", "id": "smithi037"}]: dispatch 2024-03-29T08:12:32.928 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:32 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "mon metadata", "id": "smithi148"}]: dispatch 2024-03-29T08:12:32.928 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:32 smithi037 ceph-mon[142325]: mon.smithi037 calling monitor election 2024-03-29T08:12:32.928 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:32 smithi037 ceph-mon[142325]: mon.smithi148 calling monitor election 2024-03-29T08:12:32.928 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:32 smithi037 ceph-mon[142325]: mon.smithi037 is new leader, mons smithi037,smithi148 in quorum (ranks 0,1) 2024-03-29T08:12:32.928 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:32 smithi037 ceph-mon[142325]: monmap e3: 2 mons at {smithi037=[v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0],smithi148=[v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-29T08:12:32.928 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:32 smithi037 ceph-mon[142325]: fsmap cephfs:2 {0=cephfs.smithi037.mmwgxe=up:active,1=cephfs.smithi037.iifsmh=up:active} 2 up:standby 2024-03-29T08:12:32.928 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:32 smithi037 ceph-mon[142325]: osdmap e50: 6 total, 6 up, 6 in 2024-03-29T08:12:32.928 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:32 smithi037 ceph-mon[142325]: mgrmap e39: smithi037.wjikii(active, since 21s), standbys: smithi148.onhgvm 2024-03-29T08:12:32.928 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:32 smithi037 ceph-mon[142325]: overall HEALTH_OK 2024-03-29T08:12:32.929 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:32 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:12:32.929 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:32 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:12:32.929 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:32 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T08:12:32.929 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:32 smithi037 ceph-mon[142325]: pgmap v14: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:12:34.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:34 smithi148 ceph-mon[106692]: pgmap v15: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:12:34.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:34 smithi037 ceph-mon[142325]: pgmap v15: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:12:36.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:36 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:12:36.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:36 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:12:36.807 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:36 smithi148 ceph-mon[106692]: pgmap v16: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:12:36.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:36 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:12:36.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:36 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:12:36.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:36 smithi037 ceph-mon[142325]: pgmap v16: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:12:37.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:37 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:12:37.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:37 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:12:37.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:37 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:12:37.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:37 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:12:38.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:38 smithi148 ceph-mon[106692]: pgmap v17: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:12:38.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:38 smithi037 ceph-mon[142325]: pgmap v17: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:12:40.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:40 smithi148 ceph-mon[106692]: pgmap v18: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:12:40.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:40 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:12:40.429 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:40 smithi037 ceph-mon[142325]: pgmap v18: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:12:40.430 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:40 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:12:42.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:42 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:12:42.428 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:42 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:12:42.428 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:42 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T08:12:42.428 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:42 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T08:12:42.428 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:42 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:12:42.428 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:42 smithi037 ceph-mon[142325]: Reconfiguring mon.smithi037 (monmap changed)... 2024-03-29T08:12:42.428 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:42 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-29T08:12:42.428 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:42 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-29T08:12:42.428 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:42 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T08:12:42.428 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:42 smithi037 ceph-mon[142325]: Reconfiguring daemon mon.smithi037 on smithi037 2024-03-29T08:12:42.428 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:42 smithi037 ceph-mon[142325]: pgmap v19: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:12:42.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:42 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:12:42.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:42 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:12:42.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:42 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T08:12:42.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:42 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T08:12:42.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:42 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:12:42.557 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:42 smithi148 ceph-mon[106692]: Reconfiguring mon.smithi037 (monmap changed)... 2024-03-29T08:12:42.557 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:42 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-29T08:12:42.557 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:42 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-29T08:12:42.557 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:42 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T08:12:42.557 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:42 smithi148 ceph-mon[106692]: Reconfiguring daemon mon.smithi037 on smithi037 2024-03-29T08:12:42.557 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:42 smithi148 ceph-mon[106692]: pgmap v19: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:12:44.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:44 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:12:44.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:44 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:12:44.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:44 smithi148 ceph-mon[106692]: Reconfiguring mgr.smithi037.wjikii (monmap changed)... 2024-03-29T08:12:44.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:44 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi037.wjikii", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-29T08:12:44.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:44 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-29T08:12:44.307 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:44 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T08:12:44.307 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:44 smithi148 ceph-mon[106692]: Reconfiguring daemon mgr.smithi037.wjikii on smithi037 2024-03-29T08:12:44.307 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:44 smithi148 ceph-mon[106692]: pgmap v20: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:12:44.313 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:44 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:12:44.313 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:44 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:12:44.313 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:44 smithi037 ceph-mon[142325]: Reconfiguring mgr.smithi037.wjikii (monmap changed)... 2024-03-29T08:12:44.313 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:44 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi037.wjikii", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-29T08:12:44.313 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:44 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-29T08:12:44.314 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:44 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T08:12:44.314 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:44 smithi037 ceph-mon[142325]: Reconfiguring daemon mgr.smithi037.wjikii on smithi037 2024-03-29T08:12:44.314 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:44 smithi037 ceph-mon[142325]: pgmap v20: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:12:46.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:45 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:12:46.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:45 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:12:46.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:45 smithi148 ceph-mon[106692]: Reconfiguring crash.smithi037 (monmap changed)... 2024-03-29T08:12:46.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:45 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi037", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-29T08:12:46.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:45 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T08:12:46.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:45 smithi148 ceph-mon[106692]: Reconfiguring daemon crash.smithi037 on smithi037 2024-03-29T08:12:46.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:45 smithi148 ceph-mon[106692]: pgmap v21: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:12:46.057 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:45 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:12:46.057 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:45 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:12:46.057 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:45 smithi037 ceph-mon[142325]: Reconfiguring crash.smithi037 (monmap changed)... 2024-03-29T08:12:46.057 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:45 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi037", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-29T08:12:46.058 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:45 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T08:12:46.058 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:45 smithi037 ceph-mon[142325]: Reconfiguring daemon crash.smithi037 on smithi037 2024-03-29T08:12:46.058 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:45 smithi037 ceph-mon[142325]: pgmap v21: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:12:47.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:47 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:12:47.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:47 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:12:47.678 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:47 smithi037 ceph-mon[142325]: Reconfiguring osd.0 (monmap changed)... 2024-03-29T08:12:47.678 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:47 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-29T08:12:47.678 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:47 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T08:12:47.678 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:47 smithi037 ceph-mon[142325]: Reconfiguring daemon osd.0 on smithi037 2024-03-29T08:12:47.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:47 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:12:47.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:47 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:12:47.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:47 smithi148 ceph-mon[106692]: Reconfiguring osd.0 (monmap changed)... 2024-03-29T08:12:47.807 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:47 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-29T08:12:47.807 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:47 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T08:12:47.807 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:47 smithi148 ceph-mon[106692]: Reconfiguring daemon osd.0 on smithi037 2024-03-29T08:12:48.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:48 smithi037 ceph-mon[142325]: pgmap v22: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:12:48.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:48 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:12:48.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:48 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:12:48.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:48 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-29T08:12:48.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:48 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T08:12:48.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:48 smithi148 ceph-mon[106692]: pgmap v22: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:12:48.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:48 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:12:48.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:48 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:12:48.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:48 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-29T08:12:48.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:48 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T08:12:49.663 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:49 smithi037 ceph-mon[142325]: Reconfiguring osd.1 (monmap changed)... 2024-03-29T08:12:49.664 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:49 smithi037 ceph-mon[142325]: Reconfiguring daemon osd.1 on smithi037 2024-03-29T08:12:49.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:49 smithi148 ceph-mon[106692]: Reconfiguring osd.1 (monmap changed)... 2024-03-29T08:12:49.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:49 smithi148 ceph-mon[106692]: Reconfiguring daemon osd.1 on smithi037 2024-03-29T08:12:50.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:50 smithi037 ceph-mon[142325]: pgmap v23: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:12:50.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:50 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:12:50.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:50 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:12:50.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:50 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-29T08:12:50.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:50 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T08:12:50.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:50 smithi148 ceph-mon[106692]: pgmap v23: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:12:50.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:50 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:12:50.807 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:50 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:12:50.807 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:50 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-29T08:12:50.807 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:50 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T08:12:51.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:51 smithi148 ceph-mon[106692]: Reconfiguring osd.2 (monmap changed)... 2024-03-29T08:12:51.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:51 smithi148 ceph-mon[106692]: Reconfiguring daemon osd.2 on smithi037 2024-03-29T08:12:51.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:51 smithi037 ceph-mon[142325]: Reconfiguring osd.2 (monmap changed)... 2024-03-29T08:12:51.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:51 smithi037 ceph-mon[142325]: Reconfiguring daemon osd.2 on smithi037 2024-03-29T08:12:52.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:52 smithi037 ceph-mon[142325]: pgmap v24: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:12:52.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:52 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:12:52.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:52 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:12:52.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:52 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi037.mmwgxe", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-29T08:12:52.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:52 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T08:12:52.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:52 smithi148 ceph-mon[106692]: pgmap v24: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:12:52.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:52 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:12:52.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:52 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:12:52.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:52 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi037.mmwgxe", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-29T08:12:52.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:52 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T08:12:53.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:53 smithi148 ceph-mon[106692]: Reconfiguring mds.cephfs.smithi037.mmwgxe (monmap changed)... 2024-03-29T08:12:53.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:53 smithi148 ceph-mon[106692]: Reconfiguring daemon mds.cephfs.smithi037.mmwgxe on smithi037 2024-03-29T08:12:53.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:53 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:12:53.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:53 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:12:53.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:53 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi037.iifsmh", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-29T08:12:53.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:53 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T08:12:53.854 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:53 smithi037 ceph-mon[142325]: Reconfiguring mds.cephfs.smithi037.mmwgxe (monmap changed)... 2024-03-29T08:12:53.855 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:53 smithi037 ceph-mon[142325]: Reconfiguring daemon mds.cephfs.smithi037.mmwgxe on smithi037 2024-03-29T08:12:53.855 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:53 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:12:53.855 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:53 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:12:53.855 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:53 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi037.iifsmh", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-29T08:12:53.855 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:53 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T08:12:54.758 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:54 smithi037 ceph-mon[142325]: Reconfiguring mds.cephfs.smithi037.iifsmh (monmap changed)... 2024-03-29T08:12:54.758 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:54 smithi037 ceph-mon[142325]: Reconfiguring daemon mds.cephfs.smithi037.iifsmh on smithi037 2024-03-29T08:12:54.758 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:54 smithi037 ceph-mon[142325]: pgmap v25: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:12:54.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:54 smithi148 ceph-mon[106692]: Reconfiguring mds.cephfs.smithi037.iifsmh (monmap changed)... 2024-03-29T08:12:54.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:54 smithi148 ceph-mon[106692]: Reconfiguring daemon mds.cephfs.smithi037.iifsmh on smithi037 2024-03-29T08:12:54.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:54 smithi148 ceph-mon[106692]: pgmap v25: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:12:55.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:55 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:12:55.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:55 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:12:55.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:55 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:12:55.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:55 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi148", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-29T08:12:55.807 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:55 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T08:12:55.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:55 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:12:55.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:55 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:12:55.928 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:55 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:12:55.928 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:55 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi148", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-29T08:12:55.928 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:55 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T08:12:56.709 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:56 smithi148 ceph-mon[106692]: Reconfiguring crash.smithi148 (monmap changed)... 2024-03-29T08:12:56.710 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:56 smithi148 ceph-mon[106692]: Reconfiguring daemon crash.smithi148 on smithi148 2024-03-29T08:12:56.710 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:56 smithi148 ceph-mon[106692]: pgmap v26: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:12:56.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:56 smithi037 ceph-mon[142325]: Reconfiguring crash.smithi148 (monmap changed)... 2024-03-29T08:12:56.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:56 smithi037 ceph-mon[142325]: Reconfiguring daemon crash.smithi148 on smithi148 2024-03-29T08:12:56.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:56 smithi037 ceph-mon[142325]: pgmap v26: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:12:58.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:57 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:12:58.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:57 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:12:58.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:57 smithi148 ceph-mon[106692]: Reconfiguring mgr.smithi148.onhgvm (monmap changed)... 2024-03-29T08:12:58.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:57 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi148.onhgvm", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-29T08:12:58.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:57 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-29T08:12:58.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:57 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T08:12:58.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:57 smithi148 ceph-mon[106692]: Reconfiguring daemon mgr.smithi148.onhgvm on smithi148 2024-03-29T08:12:58.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:57 smithi148 ceph-mon[106692]: pgmap v27: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:12:58.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:57 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:12:58.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:57 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:12:58.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:57 smithi037 ceph-mon[142325]: Reconfiguring mgr.smithi148.onhgvm (monmap changed)... 2024-03-29T08:12:58.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:57 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi148.onhgvm", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-29T08:12:58.178 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:57 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-29T08:12:58.178 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:57 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T08:12:58.178 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:57 smithi037 ceph-mon[142325]: Reconfiguring daemon mgr.smithi148.onhgvm on smithi148 2024-03-29T08:12:58.178 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:57 smithi037 ceph-mon[142325]: pgmap v27: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:12:58.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:58.729+0000 7fb2ce800700 1 -- 172.21.15.37:0/695653309 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb2c81037a0 msgr2=0x7fb2c80fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:12:58.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:58.729+0000 7fb2ce800700 1 --2- 172.21.15.37:0/695653309 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb2c81037a0 0x7fb2c80fce10 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fb2b4004940 tx=0x7fb2b4009e30 comp rx=0 tx=0).stop 2024-03-29T08:12:58.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:58.729+0000 7fb2ce800700 1 -- 172.21.15.37:0/695653309 shutdown_connections 2024-03-29T08:12:58.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:58.729+0000 7fb2ce800700 1 --2- 172.21.15.37:0/695653309 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb2c81037a0 0x7fb2c80fce10 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:12:58.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:58.729+0000 7fb2ce800700 1 --2- 172.21.15.37:0/695653309 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb2c8102e00 0x7fb2c81031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:12:58.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:58.729+0000 7fb2ce800700 1 -- 172.21.15.37:0/695653309 >> 172.21.15.37:0/695653309 conn(0x7fb2c80f89b0 msgr2=0x7fb2c80fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:12:58.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:58.729+0000 7fb2ce800700 1 -- 172.21.15.37:0/695653309 shutdown_connections 2024-03-29T08:12:58.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:58.729+0000 7fb2ce800700 1 -- 172.21.15.37:0/695653309 wait complete. 2024-03-29T08:12:58.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:58.730+0000 7fb2ce800700 1 Processor -- start 2024-03-29T08:12:58.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:58.730+0000 7fb2ce800700 1 -- start start 2024-03-29T08:12:58.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:58.731+0000 7fb2ce800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb2c8102e00 0x7fb2c8115cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:12:58.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:58.731+0000 7fb2ce800700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb2c81037a0 0x7fb2c8116200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:12:58.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:58.731+0000 7fb2ce800700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2c81168e0 con 0x7fb2c8102e00 2024-03-29T08:12:58.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:58.731+0000 7fb2ce800700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2c810fc20 con 0x7fb2c81037a0 2024-03-29T08:12:58.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:58.731+0000 7fb2c7400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb2c8102e00 0x7fb2c8115cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:12:58.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:58.731+0000 7fb2c7400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb2c8102e00 0x7fb2c8115cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41446/0 (socket says 172.21.15.37:41446) 2024-03-29T08:12:58.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:58.731+0000 7fb2c7400700 1 -- 172.21.15.37:0/320735446 learned_addr learned my addr 172.21.15.37:0/320735446 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:12:58.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:58.731+0000 7fb2c6a00700 1 --2- 172.21.15.37:0/320735446 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb2c81037a0 0x7fb2c8116200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:12:58.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:58.732+0000 7fb2c7400700 1 -- 172.21.15.37:0/320735446 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb2c81037a0 msgr2=0x7fb2c8116200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:12:58.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:58.732+0000 7fb2c7400700 1 --2- 172.21.15.37:0/320735446 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb2c81037a0 0x7fb2c8116200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:12:58.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:58.732+0000 7fb2c7400700 1 -- 172.21.15.37:0/320735446 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2b4005040 con 0x7fb2c8102e00 2024-03-29T08:12:58.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:58.732+0000 7fb2c6a00700 1 --2- 172.21.15.37:0/320735446 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb2c81037a0 0x7fb2c8116200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T08:12:58.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:58.732+0000 7fb2c7400700 1 --2- 172.21.15.37:0/320735446 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb2c8102e00 0x7fb2c8115cc0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fb2b800b790 tx=0x7fb2b8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:12:58.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:58.732+0000 7fb2bf400700 1 -- 172.21.15.37:0/320735446 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb2b800f6f0 con 0x7fb2c8102e00 2024-03-29T08:12:58.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:58.732+0000 7fb2bf400700 1 -- 172.21.15.37:0/320735446 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fb2b800fd00 con 0x7fb2c8102e00 2024-03-29T08:12:58.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:58.732+0000 7fb2ce800700 1 -- 172.21.15.37:0/320735446 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2c810fea0 con 0x7fb2c8102e00 2024-03-29T08:12:58.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:58.732+0000 7fb2bf400700 1 -- 172.21.15.37:0/320735446 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb2b80103a0 con 0x7fb2c8102e00 2024-03-29T08:12:58.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:58.732+0000 7fb2ce800700 1 -- 172.21.15.37:0/320735446 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2c8110460 con 0x7fb2c8102e00 2024-03-29T08:12:58.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:58.733+0000 7fb2ce800700 1 -- 172.21.15.37:0/320735446 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2c80fe550 con 0x7fb2c8102e00 2024-03-29T08:12:58.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:58.734+0000 7fb2bf400700 1 -- 172.21.15.37:0/320735446 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 39) v1 ==== 90944+0+0 (secure 0 0 0) 0x7fb2b800f850 con 0x7fb2c8102e00 2024-03-29T08:12:58.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:58.735+0000 7fb2bf400700 1 --2- 172.21.15.37:0/320735446 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fb2ac06cd40 0x7fb2ac06f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:12:58.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:58.735+0000 7fb2bf400700 1 -- 172.21.15.37:0/320735446 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fb2b808bf20 con 0x7fb2c8102e00 2024-03-29T08:12:58.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:58.736+0000 7fb2c6a00700 1 --2- 172.21.15.37:0/320735446 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fb2ac06cd40 0x7fb2ac06f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:12:58.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:58.736+0000 7fb2c6a00700 1 --2- 172.21.15.37:0/320735446 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fb2ac06cd40 0x7fb2ac06f1f0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fb2c8111560 tx=0x7fb2b4009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:12:58.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:58.741+0000 7fb2bf400700 1 -- 172.21.15.37:0/320735446 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fb2b8059960 con 0x7fb2c8102e00 2024-03-29T08:12:58.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:58.975+0000 7fb2ce800700 1 -- 172.21.15.37:0/320735446 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb2c8111310 con 0x7fb2ac06cd40 2024-03-29T08:12:58.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:58.977+0000 7fb2bf400700 1 -- 172.21.15.37:0/320735446 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+383 (secure 0 0 0) 0x7fb2c8111310 con 0x7fb2ac06cd40 2024-03-29T08:12:58.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:58.979+0000 7fb2ce800700 1 -- 172.21.15.37:0/320735446 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fb2ac06cd40 msgr2=0x7fb2ac06f1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:12:58.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:58.979+0000 7fb2ce800700 1 --2- 172.21.15.37:0/320735446 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fb2ac06cd40 0x7fb2ac06f1f0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fb2c8111560 tx=0x7fb2b4009e30 comp rx=0 tx=0).stop 2024-03-29T08:12:58.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:58.979+0000 7fb2ce800700 1 -- 172.21.15.37:0/320735446 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb2c8102e00 msgr2=0x7fb2c8115cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:12:58.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:58.979+0000 7fb2ce800700 1 --2- 172.21.15.37:0/320735446 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb2c8102e00 0x7fb2c8115cc0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fb2b800b790 tx=0x7fb2b8009250 comp rx=0 tx=0).stop 2024-03-29T08:12:58.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:58.979+0000 7fb2ce800700 1 -- 172.21.15.37:0/320735446 shutdown_connections 2024-03-29T08:12:58.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:58.979+0000 7fb2ce800700 1 --2- 172.21.15.37:0/320735446 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fb2ac06cd40 0x7fb2ac06f1f0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:12:58.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:58.979+0000 7fb2ce800700 1 --2- 172.21.15.37:0/320735446 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb2c8102e00 0x7fb2c8115cc0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:12:58.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:58.979+0000 7fb2ce800700 1 --2- 172.21.15.37:0/320735446 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb2c81037a0 0x7fb2c8116200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:12:58.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:58.979+0000 7fb2ce800700 1 -- 172.21.15.37:0/320735446 >> 172.21.15.37:0/320735446 conn(0x7fb2c80f89b0 msgr2=0x7fb2c80f98d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:12:58.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:58.979+0000 7fb2ce800700 1 -- 172.21.15.37:0/320735446 shutdown_connections 2024-03-29T08:12:58.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:58.979+0000 7fb2ce800700 1 -- 172.21.15.37:0/320735446 wait complete. 2024-03-29T08:12:58.993 INFO:teuthology.orchestra.run.smithi037.stdout:true 2024-03-29T08:12:59.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:59.118+0000 7f6902000700 1 -- 172.21.15.37:0/968057970 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f68fc0fde90 msgr2=0x7f68fc0fe260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:12:59.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:59.118+0000 7f6902000700 1 --2- 172.21.15.37:0/968057970 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f68fc0fde90 0x7f68fc0fe260 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f68ec004940 tx=0x7f68ec009e30 comp rx=0 tx=0).stop 2024-03-29T08:12:59.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:59.118+0000 7f6902000700 1 -- 172.21.15.37:0/968057970 shutdown_connections 2024-03-29T08:12:59.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:59.118+0000 7f6902000700 1 --2- 172.21.15.37:0/968057970 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f68fc0fe830 0x7f68fc106f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:12:59.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:59.118+0000 7f6902000700 1 --2- 172.21.15.37:0/968057970 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f68fc0fde90 0x7f68fc0fe260 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:12:59.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:59.118+0000 7f6902000700 1 -- 172.21.15.37:0/968057970 >> 172.21.15.37:0/968057970 conn(0x7f68fc072bc0 msgr2=0x7f68fc072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:12:59.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:59.118+0000 7f6902000700 1 -- 172.21.15.37:0/968057970 shutdown_connections 2024-03-29T08:12:59.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:59.118+0000 7f6902000700 1 -- 172.21.15.37:0/968057970 wait complete. 2024-03-29T08:12:59.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:59.119+0000 7f6902000700 1 Processor -- start 2024-03-29T08:12:59.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:59.119+0000 7f6902000700 1 -- start start 2024-03-29T08:12:59.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:59.119+0000 7f6902000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f68fc0fde90 0x7f68fc115bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:12:59.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:59.119+0000 7f6902000700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f68fc0fe830 0x7f68fc1160f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:12:59.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:59.119+0000 7f6902000700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68fc116740 con 0x7f68fc0fde90 2024-03-29T08:12:59.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:59.119+0000 7f6902000700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68fc10fb50 con 0x7f68fc0fe830 2024-03-29T08:12:59.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:59.120+0000 7f68fb400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f68fc0fde90 0x7f68fc115bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:12:59.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:59.120+0000 7f68fb400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f68fc0fde90 0x7f68fc115bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41462/0 (socket says 172.21.15.37:41462) 2024-03-29T08:12:59.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:59.120+0000 7f68fb400700 1 -- 172.21.15.37:0/1700136779 learned_addr learned my addr 172.21.15.37:0/1700136779 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:12:59.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:59.120+0000 7f68faa00700 1 --2- 172.21.15.37:0/1700136779 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f68fc0fe830 0x7f68fc1160f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:12:59.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:59.120+0000 7f68fb400700 1 -- 172.21.15.37:0/1700136779 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f68fc0fe830 msgr2=0x7f68fc1160f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:12:59.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:59.120+0000 7f68fb400700 1 --2- 172.21.15.37:0/1700136779 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f68fc0fe830 0x7f68fc1160f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:12:59.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:59.120+0000 7f68fb400700 1 -- 172.21.15.37:0/1700136779 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68ec005040 con 0x7f68fc0fde90 2024-03-29T08:12:59.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:59.120+0000 7f68fb400700 1 --2- 172.21.15.37:0/1700136779 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f68fc0fde90 0x7f68fc115bb0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f68ec000c00 tx=0x7f68ec009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:12:59.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:59.121+0000 7f68f3400700 1 -- 172.21.15.37:0/1700136779 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f68ec021070 con 0x7f68fc0fde90 2024-03-29T08:12:59.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:59.121+0000 7f68f3400700 1 -- 172.21.15.37:0/1700136779 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f68ec013e00 con 0x7f68fc0fde90 2024-03-29T08:12:59.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:59.121+0000 7f68f3400700 1 -- 172.21.15.37:0/1700136779 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f68ec00e070 con 0x7f68fc0fde90 2024-03-29T08:12:59.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:59.121+0000 7f6902000700 1 -- 172.21.15.37:0/1700136779 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68fc10fdd0 con 0x7f68fc0fde90 2024-03-29T08:12:59.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:59.121+0000 7f6902000700 1 -- 172.21.15.37:0/1700136779 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68fc1102b0 con 0x7f68fc0fde90 2024-03-29T08:12:59.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:59.122+0000 7f6902000700 1 -- 172.21.15.37:0/1700136779 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68fc106a20 con 0x7f68fc0fde90 2024-03-29T08:12:59.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:59.123+0000 7f68f3400700 1 -- 172.21.15.37:0/1700136779 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 39) v1 ==== 90944+0+0 (secure 0 0 0) 0x7f68ec013950 con 0x7f68fc0fde90 2024-03-29T08:12:59.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:59.124+0000 7f68f3400700 1 --2- 172.21.15.37:0/1700136779 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f68e006cd40 0x7f68e006f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:12:59.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:59.124+0000 7f68f3400700 1 -- 172.21.15.37:0/1700136779 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f68ec08d260 con 0x7f68fc0fde90 2024-03-29T08:12:59.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:59.124+0000 7f68faa00700 1 --2- 172.21.15.37:0/1700136779 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f68e006cd40 0x7f68e006f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:12:59.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:59.124+0000 7f68faa00700 1 --2- 172.21.15.37:0/1700136779 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f68e006cd40 0x7f68e006f1f0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f68fc111450 tx=0x7f68e800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:12:59.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:59.129+0000 7f68f3400700 1 -- 172.21.15.37:0/1700136779 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f68ec057120 con 0x7f68fc0fde90 2024-03-29T08:12:59.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:59.363+0000 7f6902000700 1 -- 172.21.15.37:0/1700136779 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f68fc109d60 con 0x7f68e006cd40 2024-03-29T08:12:59.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:59.364+0000 7f68f3400700 1 -- 172.21.15.37:0/1700136779 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+383 (secure 0 0 0) 0x7f68fc109d60 con 0x7f68e006cd40 2024-03-29T08:12:59.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:59.366+0000 7f6902000700 1 -- 172.21.15.37:0/1700136779 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f68e006cd40 msgr2=0x7f68e006f1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:12:59.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:59.366+0000 7f6902000700 1 --2- 172.21.15.37:0/1700136779 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f68e006cd40 0x7f68e006f1f0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f68fc111450 tx=0x7f68e800b040 comp rx=0 tx=0).stop 2024-03-29T08:12:59.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:59.366+0000 7f6902000700 1 -- 172.21.15.37:0/1700136779 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f68fc0fde90 msgr2=0x7f68fc115bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:12:59.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:59.366+0000 7f6902000700 1 --2- 172.21.15.37:0/1700136779 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f68fc0fde90 0x7f68fc115bb0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f68ec000c00 tx=0x7f68ec009e30 comp rx=0 tx=0).stop 2024-03-29T08:12:59.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:59.366+0000 7f6902000700 1 -- 172.21.15.37:0/1700136779 shutdown_connections 2024-03-29T08:12:59.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:59.366+0000 7f6902000700 1 --2- 172.21.15.37:0/1700136779 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f68e006cd40 0x7f68e006f1f0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:12:59.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:59.366+0000 7f6902000700 1 --2- 172.21.15.37:0/1700136779 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f68fc0fde90 0x7f68fc115bb0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:12:59.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:59.366+0000 7f6902000700 1 --2- 172.21.15.37:0/1700136779 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f68fc0fe830 0x7f68fc1160f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:12:59.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:59.366+0000 7f6902000700 1 -- 172.21.15.37:0/1700136779 >> 172.21.15.37:0/1700136779 conn(0x7f68fc072bc0 msgr2=0x7f68fc0fb7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:12:59.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:59.367+0000 7f6902000700 1 -- 172.21.15.37:0/1700136779 shutdown_connections 2024-03-29T08:12:59.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:59.367+0000 7f6902000700 1 -- 172.21.15.37:0/1700136779 wait complete. 2024-03-29T08:12:59.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:59.504+0000 7f1f81000700 1 -- 172.21.15.37:0/3679148983 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1f7c0fe850 msgr2=0x7f1f7c106f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:12:59.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:59.504+0000 7f1f81000700 1 --2- 172.21.15.37:0/3679148983 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1f7c0fe850 0x7f1f7c106f20 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f1f68004970 tx=0x7f1f68009e30 comp rx=0 tx=0).stop 2024-03-29T08:12:59.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:59.504+0000 7f1f81000700 1 -- 172.21.15.37:0/3679148983 shutdown_connections 2024-03-29T08:12:59.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:59.504+0000 7f1f81000700 1 --2- 172.21.15.37:0/3679148983 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1f7c0fe850 0x7f1f7c106f20 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:12:59.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:59.504+0000 7f1f81000700 1 --2- 172.21.15.37:0/3679148983 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1f7c0fdeb0 0x7f1f7c0fe280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:12:59.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:59.504+0000 7f1f81000700 1 -- 172.21.15.37:0/3679148983 >> 172.21.15.37:0/3679148983 conn(0x7f1f7c072bc0 msgr2=0x7f1f7c072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:12:59.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:59.504+0000 7f1f81000700 1 -- 172.21.15.37:0/3679148983 shutdown_connections 2024-03-29T08:12:59.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:59.504+0000 7f1f81000700 1 -- 172.21.15.37:0/3679148983 wait complete. 2024-03-29T08:12:59.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:59.505+0000 7f1f81000700 1 Processor -- start 2024-03-29T08:12:59.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:59.505+0000 7f1f81000700 1 -- start start 2024-03-29T08:12:59.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:59.505+0000 7f1f81000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1f7c0fdeb0 0x7f1f7c115c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:12:59.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:59.505+0000 7f1f81000700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1f7c0fe850 0x7f1f7c1161a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:12:59.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:59.505+0000 7f1f81000700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f7c1167f0 con 0x7f1f7c0fdeb0 2024-03-29T08:12:59.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:59.505+0000 7f1f81000700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f7c10fc00 con 0x7f1f7c0fe850 2024-03-29T08:12:59.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:59.506+0000 7f1f79e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1f7c0fdeb0 0x7f1f7c115c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:12:59.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:59.506+0000 7f1f79e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1f7c0fdeb0 0x7f1f7c115c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41468/0 (socket says 172.21.15.37:41468) 2024-03-29T08:12:59.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:59.506+0000 7f1f79e00700 1 -- 172.21.15.37:0/1317607226 learned_addr learned my addr 172.21.15.37:0/1317607226 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:12:59.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:59.506+0000 7f1f79e00700 1 -- 172.21.15.37:0/1317607226 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1f7c0fe850 msgr2=0x7f1f7c1161a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:12:59.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:59.506+0000 7f1f79e00700 1 --2- 172.21.15.37:0/1317607226 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1f7c0fe850 0x7f1f7c1161a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:12:59.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:59.506+0000 7f1f79e00700 1 -- 172.21.15.37:0/1317607226 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f68005040 con 0x7f1f7c0fdeb0 2024-03-29T08:12:59.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:59.506+0000 7f1f79e00700 1 --2- 172.21.15.37:0/1317607226 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1f7c0fdeb0 0x7f1f7c115c60 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f1f6c00eb40 tx=0x7f1f6c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:12:59.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:59.507+0000 7f1f72000700 1 -- 172.21.15.37:0/1317607226 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1f6c007cc0 con 0x7f1f7c0fdeb0 2024-03-29T08:12:59.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:59.507+0000 7f1f72000700 1 -- 172.21.15.37:0/1317607226 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f1f6c00f040 con 0x7f1f7c0fdeb0 2024-03-29T08:12:59.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:59.507+0000 7f1f72000700 1 -- 172.21.15.37:0/1317607226 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1f6c009570 con 0x7f1f7c0fdeb0 2024-03-29T08:12:59.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:59.507+0000 7f1f81000700 1 -- 172.21.15.37:0/1317607226 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f7c10fe80 con 0x7f1f7c0fdeb0 2024-03-29T08:12:59.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:59.507+0000 7f1f81000700 1 -- 172.21.15.37:0/1317607226 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f7c1103c0 con 0x7f1f7c0fdeb0 2024-03-29T08:12:59.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:59.508+0000 7f1f81000700 1 -- 172.21.15.37:0/1317607226 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f7c1069f0 con 0x7f1f7c0fdeb0 2024-03-29T08:12:59.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:59.509+0000 7f1f72000700 1 -- 172.21.15.37:0/1317607226 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 39) v1 ==== 90944+0+0 (secure 0 0 0) 0x7f1f6c0096d0 con 0x7f1f7c0fdeb0 2024-03-29T08:12:59.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:59.510+0000 7f1f72000700 1 --2- 172.21.15.37:0/1317607226 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f1f6006cd40 0x7f1f6006f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:12:59.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:59.510+0000 7f1f72000700 1 -- 172.21.15.37:0/1317607226 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f1f6c08be80 con 0x7f1f7c0fdeb0 2024-03-29T08:12:59.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:59.510+0000 7f1f79400700 1 --2- 172.21.15.37:0/1317607226 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f1f6006cd40 0x7f1f6006f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:12:59.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:59.511+0000 7f1f79400700 1 --2- 172.21.15.37:0/1317607226 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f1f6006cd40 0x7f1f6006f1f0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f1f6800b760 tx=0x7f1f68009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:12:59.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:59.514+0000 7f1f72000700 1 -- 172.21.15.37:0/1317607226 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f1f6c055d50 con 0x7f1f7c0fdeb0 2024-03-29T08:12:59.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:59.737+0000 7f1f81000700 1 -- 172.21.15.37:0/1317607226 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f1f7c109e20 con 0x7f1f6006cd40 2024-03-29T08:12:59.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:59.748+0000 7f1f72000700 1 -- 172.21.15.37:0/1317607226 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f1f7c109e20 con 0x7f1f6006cd40 2024-03-29T08:12:59.748 INFO:teuthology.orchestra.run.smithi037.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-29T08:12:59.748 INFO:teuthology.orchestra.run.smithi037.stdout:alertmanager.smithi037 smithi037 *:9093,9094 running (2m) 44s ago 15m 14.6M - 0.25.0 c8568f914cd2 2f9f9746ed0f 2024-03-29T08:12:59.748 INFO:teuthology.orchestra.run.smithi037.stdout:crash.smithi037 smithi037 running (16m) 44s ago 16m 6656k - 17.2.7-789-g4213e17c cf22eb8683be e0343935b25e 2024-03-29T08:12:59.748 INFO:teuthology.orchestra.run.smithi037.stdout:crash.smithi148 smithi148 running (14m) 24s ago 14m 6656k - 17.2.7-789-g4213e17c cf22eb8683be ff5164c552c1 2024-03-29T08:12:59.748 INFO:teuthology.orchestra.run.smithi037.stdout:grafana.smithi037 smithi037 *:3000 running (2m) 44s ago 15m 73.4M - 9.4.7 954c08fa6188 6b976fe49952 2024-03-29T08:12:59.748 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi037.iifsmh smithi037 running (9m) 44s ago 9m 110M - 17.2.7-789-g4213e17c cf22eb8683be 074e1acf331f 2024-03-29T08:12:59.748 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi037.mmwgxe smithi037 running (9m) 44s ago 9m 115M - 17.2.7-789-g4213e17c cf22eb8683be 9a5b1e72de5c 2024-03-29T08:12:59.748 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi148.qukpjh smithi148 running (9m) 24s ago 9m 13.9M - 17.2.7-789-g4213e17c cf22eb8683be 8e469782a273 2024-03-29T08:12:59.748 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi148.sqteih smithi148 running (9m) 24s ago 9m 13.8M - 17.2.7-789-g4213e17c cf22eb8683be 8e68f858bb22 2024-03-29T08:12:59.748 INFO:teuthology.orchestra.run.smithi037.stdout:mgr.smithi037.wjikii smithi037 *:8443,9283,8765 running (5m) 44s ago 17m 468M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff c5a1417b9417 2024-03-29T08:12:59.748 INFO:teuthology.orchestra.run.smithi037.stdout:mgr.smithi148.onhgvm smithi148 *:8443,9283,8765 running (3m) 24s ago 14m 447M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff fcfdbce062bd 2024-03-29T08:12:59.748 INFO:teuthology.orchestra.run.smithi037.stdout:mon.smithi037 smithi037 running (61s) 44s ago 17m 27.5M 2048M 18.2.2-1015-gd0e392d9 1cc2f45c17ff eea2060783af 2024-03-29T08:12:59.748 INFO:teuthology.orchestra.run.smithi037.stdout:mon.smithi148 smithi148 running (30s) 24s ago 14m 23.8M 2048M 18.2.2-1015-gd0e392d9 1cc2f45c17ff ad4b2c5748be 2024-03-29T08:12:59.749 INFO:teuthology.orchestra.run.smithi037.stdout:node-exporter.smithi037 smithi037 *:9100 running (3m) 44s ago 15m 7503k - 1.5.0 0da6a335fe13 26b8ae9cb878 2024-03-29T08:12:59.749 INFO:teuthology.orchestra.run.smithi037.stdout:node-exporter.smithi148 smithi148 *:9100 running (3m) 24s ago 14m 7721k - 1.5.0 0da6a335fe13 98414e4e64ed 2024-03-29T08:12:59.749 INFO:teuthology.orchestra.run.smithi037.stdout:osd.0 smithi037 running (13m) 44s ago 13m 217M 1619M 17.2.7-789-g4213e17c cf22eb8683be 559fccd0ebe1 2024-03-29T08:12:59.749 INFO:teuthology.orchestra.run.smithi037.stdout:osd.1 smithi037 running (13m) 44s ago 13m 229M 1619M 17.2.7-789-g4213e17c cf22eb8683be 5582cff11b7a 2024-03-29T08:12:59.749 INFO:teuthology.orchestra.run.smithi037.stdout:osd.2 smithi037 running (12m) 44s ago 12m 198M 1619M 17.2.7-789-g4213e17c cf22eb8683be f809bfab13e5 2024-03-29T08:12:59.749 INFO:teuthology.orchestra.run.smithi037.stdout:osd.3 smithi148 running (12m) 24s ago 12m 261M 2642M 17.2.7-789-g4213e17c cf22eb8683be a0a8224bb4b0 2024-03-29T08:12:59.749 INFO:teuthology.orchestra.run.smithi037.stdout:osd.4 smithi148 running (11m) 24s ago 11m 233M 2642M 17.2.7-789-g4213e17c cf22eb8683be a00b69047ed7 2024-03-29T08:12:59.749 INFO:teuthology.orchestra.run.smithi037.stdout:osd.5 smithi148 running (11m) 24s ago 11m 233M 2642M 17.2.7-789-g4213e17c cf22eb8683be 044a3f7db7f8 2024-03-29T08:12:59.749 INFO:teuthology.orchestra.run.smithi037.stdout:prometheus.smithi037 smithi037 *:9095 running (3m) 44s ago 15m 49.4M - 2.43.0 a07b618ecd1d 30c0223ec589 2024-03-29T08:12:59.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:59.750+0000 7f1f81000700 1 -- 172.21.15.37:0/1317607226 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f1f6006cd40 msgr2=0x7f1f6006f1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:12:59.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:59.750+0000 7f1f81000700 1 --2- 172.21.15.37:0/1317607226 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f1f6006cd40 0x7f1f6006f1f0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f1f6800b760 tx=0x7f1f68009e30 comp rx=0 tx=0).stop 2024-03-29T08:12:59.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:59.750+0000 7f1f81000700 1 -- 172.21.15.37:0/1317607226 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1f7c0fdeb0 msgr2=0x7f1f7c115c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:12:59.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:59.750+0000 7f1f81000700 1 --2- 172.21.15.37:0/1317607226 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1f7c0fdeb0 0x7f1f7c115c60 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f1f6c00eb40 tx=0x7f1f6c00b040 comp rx=0 tx=0).stop 2024-03-29T08:12:59.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:59.750+0000 7f1f81000700 1 -- 172.21.15.37:0/1317607226 shutdown_connections 2024-03-29T08:12:59.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:59.750+0000 7f1f81000700 1 --2- 172.21.15.37:0/1317607226 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f1f6006cd40 0x7f1f6006f1f0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:12:59.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:59.750+0000 7f1f81000700 1 --2- 172.21.15.37:0/1317607226 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1f7c0fdeb0 0x7f1f7c115c60 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:12:59.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:59.750+0000 7f1f81000700 1 --2- 172.21.15.37:0/1317607226 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1f7c0fe850 0x7f1f7c1161a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:12:59.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:59.750+0000 7f1f81000700 1 -- 172.21.15.37:0/1317607226 >> 172.21.15.37:0/1317607226 conn(0x7f1f7c072bc0 msgr2=0x7f1f7c0fd4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:12:59.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:59.750+0000 7f1f81000700 1 -- 172.21.15.37:0/1317607226 shutdown_connections 2024-03-29T08:12:59.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:59.750+0000 7f1f81000700 1 -- 172.21.15.37:0/1317607226 wait complete. 2024-03-29T08:12:59.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:59.886+0000 7faa58000700 1 -- 172.21.15.37:0/2682864001 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faa50070360 msgr2=0x7faa50070730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:12:59.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:59.886+0000 7faa58000700 1 --2- 172.21.15.37:0/2682864001 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faa50070360 0x7faa50070730 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7faa44004970 tx=0x7faa44009e30 comp rx=0 tx=0).stop 2024-03-29T08:12:59.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:59.886+0000 7faa58000700 1 -- 172.21.15.37:0/2682864001 shutdown_connections 2024-03-29T08:12:59.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:59.886+0000 7faa58000700 1 --2- 172.21.15.37:0/2682864001 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faa50070c70 0x7faa50109110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:12:59.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:59.886+0000 7faa58000700 1 --2- 172.21.15.37:0/2682864001 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faa50070360 0x7faa50070730 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:12:59.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:59.886+0000 7faa58000700 1 -- 172.21.15.37:0/2682864001 >> 172.21.15.37:0/2682864001 conn(0x7faa500f89b0 msgr2=0x7faa500fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:12:59.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:59.886+0000 7faa58000700 1 -- 172.21.15.37:0/2682864001 shutdown_connections 2024-03-29T08:12:59.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:59.886+0000 7faa58000700 1 -- 172.21.15.37:0/2682864001 wait complete. 2024-03-29T08:12:59.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:59.887+0000 7faa58000700 1 Processor -- start 2024-03-29T08:12:59.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:59.887+0000 7faa58000700 1 -- start start 2024-03-29T08:12:59.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:59.887+0000 7faa58000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faa50070360 0x7faa50115c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:12:59.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:59.887+0000 7faa58000700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faa50070c70 0x7faa501161c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:12:59.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:59.887+0000 7faa58000700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa50116810 con 0x7faa50070360 2024-03-29T08:12:59.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:59.887+0000 7faa58000700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa5010fbe0 con 0x7faa50070c70 2024-03-29T08:12:59.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:59.888+0000 7faa55600700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faa50070360 0x7faa50115c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:12:59.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:59.888+0000 7faa55600700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faa50070360 0x7faa50115c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41482/0 (socket says 172.21.15.37:41482) 2024-03-29T08:12:59.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:59.888+0000 7faa55600700 1 -- 172.21.15.37:0/373276996 learned_addr learned my addr 172.21.15.37:0/373276996 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:12:59.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:59.888+0000 7faa54c00700 1 --2- 172.21.15.37:0/373276996 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faa50070c70 0x7faa501161c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:12:59.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:59.888+0000 7faa55600700 1 -- 172.21.15.37:0/373276996 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faa50070c70 msgr2=0x7faa501161c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:12:59.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:59.888+0000 7faa55600700 1 --2- 172.21.15.37:0/373276996 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faa50070c70 0x7faa501161c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:12:59.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:59.888+0000 7faa55600700 1 -- 172.21.15.37:0/373276996 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa44005040 con 0x7faa50070360 2024-03-29T08:12:59.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:59.888+0000 7faa54c00700 1 --2- 172.21.15.37:0/373276996 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faa50070c70 0x7faa501161c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:12:59.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:59.888+0000 7faa55600700 1 --2- 172.21.15.37:0/373276996 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faa50070360 0x7faa50115c80 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7faa44005010 tx=0x7faa44009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:12:59.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:59.889+0000 7faa4d600700 1 -- 172.21.15.37:0/373276996 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faa4400e040 con 0x7faa50070360 2024-03-29T08:12:59.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:59.889+0000 7faa4d600700 1 -- 172.21.15.37:0/373276996 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7faa44004290 con 0x7faa50070360 2024-03-29T08:12:59.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:59.889+0000 7faa58000700 1 -- 172.21.15.37:0/373276996 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa5010fe60 con 0x7faa50070360 2024-03-29T08:12:59.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:59.889+0000 7faa4d600700 1 -- 172.21.15.37:0/373276996 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faa44013e40 con 0x7faa50070360 2024-03-29T08:12:59.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:59.889+0000 7faa58000700 1 -- 172.21.15.37:0/373276996 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa501103a0 con 0x7faa50070360 2024-03-29T08:12:59.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:59.890+0000 7faa58000700 1 -- 172.21.15.37:0/373276996 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa50103b90 con 0x7faa50070360 2024-03-29T08:12:59.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:59.891+0000 7faa4d600700 1 -- 172.21.15.37:0/373276996 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 39) v1 ==== 90944+0+0 (secure 0 0 0) 0x7faa44013620 con 0x7faa50070360 2024-03-29T08:12:59.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:59.891+0000 7faa4d600700 1 --2- 172.21.15.37:0/373276996 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7faa3806cd40 0x7faa3806f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:12:59.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:59.892+0000 7faa4d600700 1 -- 172.21.15.37:0/373276996 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6324+0+0 (secure 0 0 0) 0x7faa4408d3e0 con 0x7faa50070360 2024-03-29T08:12:59.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:59.892+0000 7faa54c00700 1 --2- 172.21.15.37:0/373276996 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7faa3806cd40 0x7faa3806f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:12:59.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:59.892+0000 7faa54c00700 1 --2- 172.21.15.37:0/373276996 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7faa3806cd40 0x7faa3806f1f0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7faa3c004600 tx=0x7faa3c00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:12:59.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:12:59.896+0000 7faa4d600700 1 -- 172.21.15.37:0/373276996 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7faa4405add0 con 0x7faa50070360 2024-03-29T08:13:00.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:59 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:13:00.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:59 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:13:00.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:59 smithi148 ceph-mon[106692]: Reconfiguring mon.smithi148 (monmap changed)... 2024-03-29T08:13:00.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:59 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-29T08:13:00.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:59 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-29T08:13:00.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:59 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T08:13:00.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:59 smithi148 ceph-mon[106692]: Reconfiguring daemon mon.smithi148 on smithi148 2024-03-29T08:13:00.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:59 smithi148 ceph-mon[106692]: from='client.34160 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:13:00.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:59 smithi148 ceph-mon[106692]: from='client.34164 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:13:00.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:12:59 smithi148 ceph-mon[106692]: pgmap v28: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:13:00.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:59 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:13:00.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:59 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:13:00.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:59 smithi037 ceph-mon[142325]: Reconfiguring mon.smithi148 (monmap changed)... 2024-03-29T08:13:00.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:59 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-29T08:13:00.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:59 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-29T08:13:00.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:59 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T08:13:00.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:59 smithi037 ceph-mon[142325]: Reconfiguring daemon mon.smithi148 on smithi148 2024-03-29T08:13:00.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:59 smithi037 ceph-mon[142325]: from='client.34160 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:13:00.178 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:59 smithi037 ceph-mon[142325]: from='client.34164 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:13:00.178 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:12:59 smithi037 ceph-mon[142325]: pgmap v28: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:13:00.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:00.199+0000 7faa58000700 1 -- 172.21.15.37:0/373276996 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7faa5010bfb0 con 0x7faa50070360 2024-03-29T08:13:00.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:00.199+0000 7faa4d600700 1 -- 172.21.15.37:0/373276996 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+742 (secure 0 0 0) 0x7faa4405a960 con 0x7faa50070360 2024-03-29T08:13:00.200 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-29T08:13:00.200 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": { 2024-03-29T08:13:00.200 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 2 2024-03-29T08:13:00.200 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:13:00.200 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": { 2024-03-29T08:13:00.200 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 2 2024-03-29T08:13:00.200 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:13:00.200 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": { 2024-03-29T08:13:00.200 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 17.2.7-789-g4213e17c (4213e17c4de67016ed14f55d22503f4923b3a901) quincy (stable)": 6 2024-03-29T08:13:00.200 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:13:00.200 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": { 2024-03-29T08:13:00.200 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 17.2.7-789-g4213e17c (4213e17c4de67016ed14f55d22503f4923b3a901) quincy (stable)": 4 2024-03-29T08:13:00.200 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:13:00.201 INFO:teuthology.orchestra.run.smithi037.stdout: "overall": { 2024-03-29T08:13:00.201 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 17.2.7-789-g4213e17c (4213e17c4de67016ed14f55d22503f4923b3a901) quincy (stable)": 10, 2024-03-29T08:13:00.201 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 4 2024-03-29T08:13:00.201 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-29T08:13:00.201 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-29T08:13:00.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:00.202+0000 7faa58000700 1 -- 172.21.15.37:0/373276996 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7faa3806cd40 msgr2=0x7faa3806f1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:13:00.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:00.202+0000 7faa58000700 1 --2- 172.21.15.37:0/373276996 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7faa3806cd40 0x7faa3806f1f0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7faa3c004600 tx=0x7faa3c00b040 comp rx=0 tx=0).stop 2024-03-29T08:13:00.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:00.202+0000 7faa58000700 1 -- 172.21.15.37:0/373276996 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faa50070360 msgr2=0x7faa50115c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:13:00.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:00.202+0000 7faa58000700 1 --2- 172.21.15.37:0/373276996 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faa50070360 0x7faa50115c80 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7faa44005010 tx=0x7faa44009e30 comp rx=0 tx=0).stop 2024-03-29T08:13:00.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:00.202+0000 7faa58000700 1 -- 172.21.15.37:0/373276996 shutdown_connections 2024-03-29T08:13:00.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:00.202+0000 7faa58000700 1 --2- 172.21.15.37:0/373276996 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7faa3806cd40 0x7faa3806f1f0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:13:00.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:00.202+0000 7faa58000700 1 --2- 172.21.15.37:0/373276996 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faa50070360 0x7faa50115c80 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:13:00.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:00.202+0000 7faa58000700 1 --2- 172.21.15.37:0/373276996 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faa50070c70 0x7faa501161c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:13:00.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:00.202+0000 7faa58000700 1 -- 172.21.15.37:0/373276996 >> 172.21.15.37:0/373276996 conn(0x7faa500f89b0 msgr2=0x7faa501015f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:13:00.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:00.202+0000 7faa58000700 1 -- 172.21.15.37:0/373276996 shutdown_connections 2024-03-29T08:13:00.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:00.202+0000 7faa58000700 1 -- 172.21.15.37:0/373276996 wait complete. 2024-03-29T08:13:00.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:00.339+0000 7fd688a00700 1 -- 172.21.15.37:0/1198298349 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd684102e00 msgr2=0x7fd6841031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:13:00.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:00.339+0000 7fd688a00700 1 --2- 172.21.15.37:0/1198298349 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd684102e00 0x7fd6841031d0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fd674004940 tx=0x7fd674009e30 comp rx=0 tx=0).stop 2024-03-29T08:13:00.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:00.339+0000 7fd688a00700 1 -- 172.21.15.37:0/1198298349 shutdown_connections 2024-03-29T08:13:00.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:00.339+0000 7fd688a00700 1 --2- 172.21.15.37:0/1198298349 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd6841037a0 0x7fd6840fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:13:00.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:00.339+0000 7fd688a00700 1 --2- 172.21.15.37:0/1198298349 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd684102e00 0x7fd6841031d0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:13:00.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:00.339+0000 7fd688a00700 1 -- 172.21.15.37:0/1198298349 >> 172.21.15.37:0/1198298349 conn(0x7fd6840f89b0 msgr2=0x7fd6840fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:13:00.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:00.339+0000 7fd688a00700 1 -- 172.21.15.37:0/1198298349 shutdown_connections 2024-03-29T08:13:00.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:00.339+0000 7fd688a00700 1 -- 172.21.15.37:0/1198298349 wait complete. 2024-03-29T08:13:00.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:00.340+0000 7fd688a00700 1 Processor -- start 2024-03-29T08:13:00.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:00.340+0000 7fd688a00700 1 -- start start 2024-03-29T08:13:00.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:00.341+0000 7fd688a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd684102e00 0x7fd684115cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:13:00.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:00.341+0000 7fd688a00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd6841037a0 0x7fd684116230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:13:00.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:00.341+0000 7fd688a00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd684116880 con 0x7fd684102e00 2024-03-29T08:13:00.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:00.341+0000 7fd688a00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd68410fc50 con 0x7fd6841037a0 2024-03-29T08:13:00.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:00.341+0000 7fd681e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd684102e00 0x7fd684115cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:13:00.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:00.341+0000 7fd681e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd684102e00 0x7fd684115cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39978/0 (socket says 172.21.15.37:39978) 2024-03-29T08:13:00.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:00.341+0000 7fd681e00700 1 -- 172.21.15.37:0/1750425531 learned_addr learned my addr 172.21.15.37:0/1750425531 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:13:00.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:00.341+0000 7fd681400700 1 --2- 172.21.15.37:0/1750425531 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd6841037a0 0x7fd684116230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:13:00.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:00.341+0000 7fd681e00700 1 -- 172.21.15.37:0/1750425531 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd6841037a0 msgr2=0x7fd684116230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:13:00.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:00.341+0000 7fd681e00700 1 --2- 172.21.15.37:0/1750425531 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd6841037a0 0x7fd684116230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:13:00.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:00.342+0000 7fd681e00700 1 -- 172.21.15.37:0/1750425531 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd674005040 con 0x7fd684102e00 2024-03-29T08:13:00.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:00.342+0000 7fd681400700 1 --2- 172.21.15.37:0/1750425531 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd6841037a0 0x7fd684116230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:13:00.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:00.342+0000 7fd681e00700 1 --2- 172.21.15.37:0/1750425531 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd684102e00 0x7fd684115cf0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fd674009390 tx=0x7fd674009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:13:00.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:00.342+0000 7fd67a000700 1 -- 172.21.15.37:0/1750425531 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd67400e040 con 0x7fd684102e00 2024-03-29T08:13:00.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:00.342+0000 7fd67a000700 1 -- 172.21.15.37:0/1750425531 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd674013420 con 0x7fd684102e00 2024-03-29T08:13:00.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:00.342+0000 7fd67a000700 1 -- 172.21.15.37:0/1750425531 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd67401c630 con 0x7fd684102e00 2024-03-29T08:13:00.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:00.342+0000 7fd688a00700 1 -- 172.21.15.37:0/1750425531 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd68410fed0 con 0x7fd684102e00 2024-03-29T08:13:00.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:00.342+0000 7fd688a00700 1 -- 172.21.15.37:0/1750425531 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd684110490 con 0x7fd684102e00 2024-03-29T08:13:00.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:00.343+0000 7fd688a00700 1 -- 172.21.15.37:0/1750425531 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6840fe550 con 0x7fd684102e00 2024-03-29T08:13:00.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:00.344+0000 7fd67a000700 1 -- 172.21.15.37:0/1750425531 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 39) v1 ==== 90944+0+0 (secure 0 0 0) 0x7fd6740139e0 con 0x7fd684102e00 2024-03-29T08:13:00.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:00.345+0000 7fd67a000700 1 --2- 172.21.15.37:0/1750425531 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fd66806cd40 0x7fd66806f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:13:00.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:00.345+0000 7fd67a000700 1 -- 172.21.15.37:0/1750425531 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fd67408d390 con 0x7fd684102e00 2024-03-29T08:13:00.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:00.345+0000 7fd681400700 1 --2- 172.21.15.37:0/1750425531 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fd66806cd40 0x7fd66806f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:13:00.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:00.346+0000 7fd681400700 1 --2- 172.21.15.37:0/1750425531 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fd66806cd40 0x7fd66806f1f0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fd670009fd0 tx=0x7fd670009ee0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:13:00.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:00.350+0000 7fd67a000700 1 -- 172.21.15.37:0/1750425531 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fd67405ad80 con 0x7fd684102e00 2024-03-29T08:13:00.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:00.606+0000 7fd688a00700 1 -- 172.21.15.37:0/1750425531 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fd68404bed0 con 0x7fd684102e00 2024-03-29T08:13:00.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:00.606+0000 7fd67a000700 1 -- 172.21.15.37:0/1750425531 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 20 v20) v1 ==== 76+0+1858 (secure 0 0 0) 0x7fd67405a910 con 0x7fd684102e00 2024-03-29T08:13:00.606 INFO:teuthology.orchestra.run.smithi037.stdout:e20 2024-03-29T08:13:00.606 INFO:teuthology.orchestra.run.smithi037.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-29T08:13:00.607 INFO:teuthology.orchestra.run.smithi037.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T08:13:00.607 INFO:teuthology.orchestra.run.smithi037.stdout:legacy client fscid: 1 2024-03-29T08:13:00.607 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:13:00.607 INFO:teuthology.orchestra.run.smithi037.stdout:Filesystem 'cephfs' (1) 2024-03-29T08:13:00.607 INFO:teuthology.orchestra.run.smithi037.stdout:fs_name cephfs 2024-03-29T08:13:00.607 INFO:teuthology.orchestra.run.smithi037.stdout:epoch 20 2024-03-29T08:13:00.607 INFO:teuthology.orchestra.run.smithi037.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-03-29T08:13:00.607 INFO:teuthology.orchestra.run.smithi037.stdout:created 2024-03-29T08:03:29.241269+0000 2024-03-29T08:13:00.607 INFO:teuthology.orchestra.run.smithi037.stdout:modified 2024-03-29T08:11:27.181622+0000 2024-03-29T08:13:00.607 INFO:teuthology.orchestra.run.smithi037.stdout:tableserver 0 2024-03-29T08:13:00.607 INFO:teuthology.orchestra.run.smithi037.stdout:root 0 2024-03-29T08:13:00.607 INFO:teuthology.orchestra.run.smithi037.stdout:session_timeout 60 2024-03-29T08:13:00.607 INFO:teuthology.orchestra.run.smithi037.stdout:session_autoclose 300 2024-03-29T08:13:00.607 INFO:teuthology.orchestra.run.smithi037.stdout:max_file_size 1099511627776 2024-03-29T08:13:00.607 INFO:teuthology.orchestra.run.smithi037.stdout:max_xattr_size 65536 2024-03-29T08:13:00.607 INFO:teuthology.orchestra.run.smithi037.stdout:required_client_features {} 2024-03-29T08:13:00.608 INFO:teuthology.orchestra.run.smithi037.stdout:last_failure 0 2024-03-29T08:13:00.608 INFO:teuthology.orchestra.run.smithi037.stdout:last_failure_osd_epoch 47 2024-03-29T08:13:00.608 INFO:teuthology.orchestra.run.smithi037.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T08:13:00.608 INFO:teuthology.orchestra.run.smithi037.stdout:max_mds 2 2024-03-29T08:13:00.608 INFO:teuthology.orchestra.run.smithi037.stdout:in 0,1 2024-03-29T08:13:00.608 INFO:teuthology.orchestra.run.smithi037.stdout:up {0=14470,1=14482} 2024-03-29T08:13:00.608 INFO:teuthology.orchestra.run.smithi037.stdout:failed 2024-03-29T08:13:00.608 INFO:teuthology.orchestra.run.smithi037.stdout:damaged 2024-03-29T08:13:00.608 INFO:teuthology.orchestra.run.smithi037.stdout:stopped 2024-03-29T08:13:00.608 INFO:teuthology.orchestra.run.smithi037.stdout:data_pools [3] 2024-03-29T08:13:00.608 INFO:teuthology.orchestra.run.smithi037.stdout:metadata_pool 2 2024-03-29T08:13:00.608 INFO:teuthology.orchestra.run.smithi037.stdout:inline_data disabled 2024-03-29T08:13:00.608 INFO:teuthology.orchestra.run.smithi037.stdout:balancer 2024-03-29T08:13:00.608 INFO:teuthology.orchestra.run.smithi037.stdout:bal_rank_mask -1 2024-03-29T08:13:00.609 INFO:teuthology.orchestra.run.smithi037.stdout:standby_count_wanted 1 2024-03-29T08:13:00.609 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi037.mmwgxe{0:14470} state up:active seq 5 join_fscid=1 addr [v2:172.21.15.37:6826/1559618909,v1:172.21.15.37:6827/1559618909] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:13:00.609 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi037.iifsmh{1:14482} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.37:6828/2341530495,v1:172.21.15.37:6829/2341530495] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:13:00.609 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:13:00.609 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:13:00.609 INFO:teuthology.orchestra.run.smithi037.stdout:Standby daemons: 2024-03-29T08:13:00.609 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:13:00.609 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi148.qukpjh{-1:24319} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.148:6826/2641327912,v1:172.21.15.148:6827/2641327912] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:13:00.609 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi148.sqteih{-1:24327} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.148:6824/3252351726,v1:172.21.15.148:6825/3252351726] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:13:00.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:00.608+0000 7fd688a00700 1 -- 172.21.15.37:0/1750425531 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fd66806cd40 msgr2=0x7fd66806f1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:13:00.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:00.608+0000 7fd688a00700 1 --2- 172.21.15.37:0/1750425531 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fd66806cd40 0x7fd66806f1f0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fd670009fd0 tx=0x7fd670009ee0 comp rx=0 tx=0).stop 2024-03-29T08:13:00.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:00.608+0000 7fd688a00700 1 -- 172.21.15.37:0/1750425531 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd684102e00 msgr2=0x7fd684115cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:13:00.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:00.608+0000 7fd688a00700 1 --2- 172.21.15.37:0/1750425531 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd684102e00 0x7fd684115cf0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fd674009390 tx=0x7fd674009e30 comp rx=0 tx=0).stop 2024-03-29T08:13:00.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:00.608+0000 7fd688a00700 1 -- 172.21.15.37:0/1750425531 shutdown_connections 2024-03-29T08:13:00.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:00.608+0000 7fd688a00700 1 --2- 172.21.15.37:0/1750425531 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fd66806cd40 0x7fd66806f1f0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:13:00.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:00.608+0000 7fd688a00700 1 --2- 172.21.15.37:0/1750425531 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd684102e00 0x7fd684115cf0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:13:00.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:00.608+0000 7fd688a00700 1 --2- 172.21.15.37:0/1750425531 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd6841037a0 0x7fd684116230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:13:00.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:00.608+0000 7fd688a00700 1 -- 172.21.15.37:0/1750425531 >> 172.21.15.37:0/1750425531 conn(0x7fd6840f89b0 msgr2=0x7fd6840706d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:13:00.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:00.609+0000 7fd688a00700 1 -- 172.21.15.37:0/1750425531 shutdown_connections 2024-03-29T08:13:00.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:00.609+0000 7fd688a00700 1 -- 172.21.15.37:0/1750425531 wait complete. 2024-03-29T08:13:00.611 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 20 2024-03-29T08:13:00.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:00.747+0000 7f3580400700 1 -- 172.21.15.37:0/2335841150 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f35780fe830 msgr2=0x7f3578106f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:13:00.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:00.747+0000 7f3580400700 1 --2- 172.21.15.37:0/2335841150 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f35780fe830 0x7f3578106f00 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f3564004940 tx=0x7f3564009e30 comp rx=0 tx=0).stop 2024-03-29T08:13:00.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:00.748+0000 7f3580400700 1 -- 172.21.15.37:0/2335841150 shutdown_connections 2024-03-29T08:13:00.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:00.748+0000 7f3580400700 1 --2- 172.21.15.37:0/2335841150 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f35780fe830 0x7f3578106f00 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:13:00.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:00.748+0000 7f3580400700 1 --2- 172.21.15.37:0/2335841150 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f35780fde90 0x7f35780fe260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:13:00.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:00.748+0000 7f3580400700 1 -- 172.21.15.37:0/2335841150 >> 172.21.15.37:0/2335841150 conn(0x7f3578072bc0 msgr2=0x7f3578072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:13:00.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:00.748+0000 7f3580400700 1 -- 172.21.15.37:0/2335841150 shutdown_connections 2024-03-29T08:13:00.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:00.748+0000 7f3580400700 1 -- 172.21.15.37:0/2335841150 wait complete. 2024-03-29T08:13:00.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:00.749+0000 7f3580400700 1 Processor -- start 2024-03-29T08:13:00.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:00.749+0000 7f3580400700 1 -- start start 2024-03-29T08:13:00.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:00.749+0000 7f3580400700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f35780fde90 0x7f35780fbd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:13:00.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:00.749+0000 7f3580400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f35780fe830 0x7f35780fc290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:13:00.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:00.749+0000 7f3580400700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35780fc820 con 0x7f35780fe830 2024-03-29T08:13:00.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:00.749+0000 7f3580400700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35780fc960 con 0x7f35780fde90 2024-03-29T08:13:00.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:00.749+0000 7f357d000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f35780fe830 0x7f35780fc290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:13:00.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:00.749+0000 7f357d000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f35780fe830 0x7f35780fc290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39990/0 (socket says 172.21.15.37:39990) 2024-03-29T08:13:00.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:00.749+0000 7f357d000700 1 -- 172.21.15.37:0/351526815 learned_addr learned my addr 172.21.15.37:0/351526815 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:13:00.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:00.749+0000 7f357da00700 1 --2- 172.21.15.37:0/351526815 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f35780fde90 0x7f35780fbd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:13:00.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:00.750+0000 7f357d000700 1 -- 172.21.15.37:0/351526815 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f35780fde90 msgr2=0x7f35780fbd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:13:00.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:00.750+0000 7f357d000700 1 --2- 172.21.15.37:0/351526815 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f35780fde90 0x7f35780fbd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:13:00.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:00.750+0000 7f357d000700 1 -- 172.21.15.37:0/351526815 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3564005040 con 0x7f35780fe830 2024-03-29T08:13:00.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:00.750+0000 7f357d000700 1 --2- 172.21.15.37:0/351526815 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f35780fe830 0x7f35780fc290 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f3564000c00 tx=0x7f3564009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:13:00.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:00.750+0000 7f3575600700 1 -- 172.21.15.37:0/351526815 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3564021070 con 0x7f35780fe830 2024-03-29T08:13:00.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:00.750+0000 7f3575600700 1 -- 172.21.15.37:0/351526815 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f3564013e00 con 0x7f35780fe830 2024-03-29T08:13:00.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:00.750+0000 7f3575600700 1 -- 172.21.15.37:0/351526815 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f356400e070 con 0x7f35780fe830 2024-03-29T08:13:00.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:00.750+0000 7f3580400700 1 -- 172.21.15.37:0/351526815 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35780fcbb0 con 0x7f35780fe830 2024-03-29T08:13:00.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:00.751+0000 7f3580400700 1 -- 172.21.15.37:0/351526815 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f357811a150 con 0x7f35780fe830 2024-03-29T08:13:00.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:00.752+0000 7f3580400700 1 -- 172.21.15.37:0/351526815 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3578065e30 con 0x7f35780fe830 2024-03-29T08:13:00.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:00.753+0000 7f3575600700 1 -- 172.21.15.37:0/351526815 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 39) v1 ==== 90944+0+0 (secure 0 0 0) 0x7f3564013950 con 0x7f35780fe830 2024-03-29T08:13:00.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:00.755+0000 7f3575600700 1 --2- 172.21.15.37:0/351526815 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f356006cd40 0x7f356006f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:13:00.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:00.755+0000 7f3575600700 1 -- 172.21.15.37:0/351526815 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f356408d170 con 0x7f35780fe830 2024-03-29T08:13:00.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:00.755+0000 7f357da00700 1 --2- 172.21.15.37:0/351526815 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f356006cd40 0x7f356006f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:13:00.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:00.756+0000 7f357da00700 1 --2- 172.21.15.37:0/351526815 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f356006cd40 0x7f356006f1f0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f356c009b80 tx=0x7f356c00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:13:00.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:00.759+0000 7f3575600700 1 -- 172.21.15.37:0/351526815 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f3564057010 con 0x7f35780fe830 2024-03-29T08:13:00.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:00.994+0000 7f3580400700 1 -- 172.21.15.37:0/351526815 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f357804bed0 con 0x7f356006cd40 2024-03-29T08:13:00.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:00.995+0000 7f3575600700 1 -- 172.21.15.37:0/351526815 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+383 (secure 0 0 0) 0x7f357804bed0 con 0x7f356006cd40 2024-03-29T08:13:00.995 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-29T08:13:00.996 INFO:teuthology.orchestra.run.smithi037.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59", 2024-03-29T08:13:00.996 INFO:teuthology.orchestra.run.smithi037.stdout: "in_progress": true, 2024-03-29T08:13:00.996 INFO:teuthology.orchestra.run.smithi037.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-29T08:13:00.996 INFO:teuthology.orchestra.run.smithi037.stdout: "services_complete": [ 2024-03-29T08:13:00.996 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr", 2024-03-29T08:13:00.996 INFO:teuthology.orchestra.run.smithi037.stdout: "mon" 2024-03-29T08:13:00.996 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-29T08:13:00.996 INFO:teuthology.orchestra.run.smithi037.stdout: "progress": "4/21 daemons upgraded", 2024-03-29T08:13:00.996 INFO:teuthology.orchestra.run.smithi037.stdout: "message": "Currently upgrading mon daemons", 2024-03-29T08:13:00.996 INFO:teuthology.orchestra.run.smithi037.stdout: "is_paused": false 2024-03-29T08:13:00.996 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-29T08:13:00.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:00.998+0000 7f3580400700 1 -- 172.21.15.37:0/351526815 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f356006cd40 msgr2=0x7f356006f1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:13:00.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:00.998+0000 7f3580400700 1 --2- 172.21.15.37:0/351526815 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f356006cd40 0x7f356006f1f0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f356c009b80 tx=0x7f356c00b040 comp rx=0 tx=0).stop 2024-03-29T08:13:00.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:00.998+0000 7f3580400700 1 -- 172.21.15.37:0/351526815 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f35780fe830 msgr2=0x7f35780fc290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:13:00.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:00.998+0000 7f3580400700 1 --2- 172.21.15.37:0/351526815 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f35780fe830 0x7f35780fc290 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f3564000c00 tx=0x7f3564009e30 comp rx=0 tx=0).stop 2024-03-29T08:13:00.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:00.998+0000 7f3580400700 1 -- 172.21.15.37:0/351526815 shutdown_connections 2024-03-29T08:13:00.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:00.998+0000 7f3580400700 1 --2- 172.21.15.37:0/351526815 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f356006cd40 0x7f356006f1f0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:13:00.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:00.998+0000 7f3580400700 1 --2- 172.21.15.37:0/351526815 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f35780fde90 0x7f35780fbd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:13:00.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:00.998+0000 7f3580400700 1 --2- 172.21.15.37:0/351526815 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f35780fe830 0x7f35780fc290 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:13:00.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:00.998+0000 7f3580400700 1 -- 172.21.15.37:0/351526815 >> 172.21.15.37:0/351526815 conn(0x7f3578072bc0 msgr2=0x7f35780ff3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:13:00.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:00.998+0000 7f3580400700 1 -- 172.21.15.37:0/351526815 shutdown_connections 2024-03-29T08:13:00.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:00.998+0000 7f3580400700 1 -- 172.21.15.37:0/351526815 wait complete. 2024-03-29T08:13:01.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:01.135+0000 7ff063000700 1 -- 172.21.15.37:0/3253407338 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff05c1037c0 msgr2=0x7ff05c0fce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:13:01.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:01.135+0000 7ff063000700 1 --2- 172.21.15.37:0/3253407338 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff05c1037c0 0x7ff05c0fce30 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ff048004940 tx=0x7ff048009e30 comp rx=0 tx=0).stop 2024-03-29T08:13:01.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:01.135+0000 7ff063000700 1 -- 172.21.15.37:0/3253407338 shutdown_connections 2024-03-29T08:13:01.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:01.135+0000 7ff063000700 1 --2- 172.21.15.37:0/3253407338 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff05c1037c0 0x7ff05c0fce30 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:13:01.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:01.135+0000 7ff063000700 1 --2- 172.21.15.37:0/3253407338 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff05c102e20 0x7ff05c1031f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:13:01.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:01.135+0000 7ff063000700 1 -- 172.21.15.37:0/3253407338 >> 172.21.15.37:0/3253407338 conn(0x7ff05c0f89d0 msgr2=0x7ff05c0fade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:13:01.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:01.135+0000 7ff063000700 1 -- 172.21.15.37:0/3253407338 shutdown_connections 2024-03-29T08:13:01.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:01.135+0000 7ff063000700 1 -- 172.21.15.37:0/3253407338 wait complete. 2024-03-29T08:13:01.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:01.136+0000 7ff063000700 1 Processor -- start 2024-03-29T08:13:01.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:01.136+0000 7ff063000700 1 -- start start 2024-03-29T08:13:01.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:01.136+0000 7ff063000700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff05c102e20 0x7ff05c0702c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:13:01.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:01.136+0000 7ff063000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff05c1037c0 0x7ff05c070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:13:01.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:01.136+0000 7ff063000700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff05c070d90 con 0x7ff05c1037c0 2024-03-29T08:13:01.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:01.136+0000 7ff063000700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff05c070ed0 con 0x7ff05c102e20 2024-03-29T08:13:01.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:01.137+0000 7ff05b400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff05c1037c0 0x7ff05c070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:13:01.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:01.137+0000 7ff05be00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff05c102e20 0x7ff05c0702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:13:01.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:01.137+0000 7ff05b400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff05c1037c0 0x7ff05c070800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40016/0 (socket says 172.21.15.37:40016) 2024-03-29T08:13:01.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:01.137+0000 7ff05b400700 1 -- 172.21.15.37:0/1131929045 learned_addr learned my addr 172.21.15.37:0/1131929045 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:13:01.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:01.137+0000 7ff05be00700 1 -- 172.21.15.37:0/1131929045 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff05c1037c0 msgr2=0x7ff05c070800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:13:01.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:01.137+0000 7ff05be00700 1 --2- 172.21.15.37:0/1131929045 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff05c1037c0 0x7ff05c070800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:13:01.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:01.137+0000 7ff05be00700 1 -- 172.21.15.37:0/1131929045 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff048005040 con 0x7ff05c102e20 2024-03-29T08:13:01.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:01.137+0000 7ff05b400700 1 --2- 172.21.15.37:0/1131929045 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff05c1037c0 0x7ff05c070800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T08:13:01.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:01.138+0000 7ff05be00700 1 --2- 172.21.15.37:0/1131929045 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff05c102e20 0x7ff05c0702c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7ff04c009fd0 tx=0x7ff04c00b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:13:01.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:01.138+0000 7ff053e00700 1 -- 172.21.15.37:0/1131929045 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff04c004e90 con 0x7ff05c102e20 2024-03-29T08:13:01.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:01.138+0000 7ff063000700 1 -- 172.21.15.37:0/1131929045 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff05c071120 con 0x7ff05c102e20 2024-03-29T08:13:01.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:01.138+0000 7ff053e00700 1 -- 172.21.15.37:0/1131929045 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ff04c00f040 con 0x7ff05c102e20 2024-03-29T08:13:01.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:01.138+0000 7ff053e00700 1 -- 172.21.15.37:0/1131929045 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff04c00c9b0 con 0x7ff05c102e20 2024-03-29T08:13:01.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:01.138+0000 7ff063000700 1 -- 172.21.15.37:0/1131929045 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff05c11a170 con 0x7ff05c102e20 2024-03-29T08:13:01.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:01.139+0000 7ff063000700 1 -- 172.21.15.37:0/1131929045 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff05c0fe570 con 0x7ff05c102e20 2024-03-29T08:13:01.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:01.140+0000 7ff053e00700 1 -- 172.21.15.37:0/1131929045 <== mon.1 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 39) v1 ==== 90944+0+0 (secure 0 0 0) 0x7ff04c007cc0 con 0x7ff05c102e20 2024-03-29T08:13:01.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:01.141+0000 7ff053e00700 1 --2- 172.21.15.37:0/1131929045 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7ff04006cd40 0x7ff04006f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:13:01.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:01.141+0000 7ff053e00700 1 -- 172.21.15.37:0/1131929045 <== mon.1 v2:172.21.15.148:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6324+0+0 (secure 0 0 0) 0x7ff04c08bcf0 con 0x7ff05c102e20 2024-03-29T08:13:01.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:01.141+0000 7ff05b400700 1 --2- 172.21.15.37:0/1131929045 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7ff04006cd40 0x7ff04006f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:13:01.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:01.142+0000 7ff05b400700 1 --2- 172.21.15.37:0/1131929045 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7ff04006cd40 0x7ff04006f1f0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7ff05c071c00 tx=0x7ff048009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:13:01.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:01.146+0000 7ff053e00700 1 -- 172.21.15.37:0/1131929045 <== mon.1 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7ff04c055b40 con 0x7ff05c102e20 2024-03-29T08:13:01.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:13:01 smithi148 ceph-mon[106692]: from='client.34168 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:13:01.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:13:01 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/373276996' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:13:01.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:13:01 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/1750425531' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T08:13:01.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:13:01 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:13:01.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:13:01 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:13:01.307 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:13:01 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-29T08:13:01.307 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:13:01 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T08:13:01.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:13:01 smithi037 ceph-mon[142325]: from='client.34168 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:13:01.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:13:01 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/373276996' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:13:01.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:13:01 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/1750425531' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T08:13:01.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:13:01 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:13:01.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:13:01 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:13:01.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:13:01 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-29T08:13:01.428 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:13:01 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T08:13:01.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:01.441+0000 7ff063000700 1 -- 172.21.15.37:0/1131929045 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7ff05c04bf60 con 0x7ff05c102e20 2024-03-29T08:13:01.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:01.442+0000 7ff053e00700 1 -- 172.21.15.37:0/1131929045 <== mon.1 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7ff04c059160 con 0x7ff05c102e20 2024-03-29T08:13:01.443 INFO:teuthology.orchestra.run.smithi037.stdout:HEALTH_OK 2024-03-29T08:13:01.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:01.445+0000 7ff063000700 1 -- 172.21.15.37:0/1131929045 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7ff04006cd40 msgr2=0x7ff04006f1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:13:01.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:01.445+0000 7ff063000700 1 --2- 172.21.15.37:0/1131929045 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7ff04006cd40 0x7ff04006f1f0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7ff05c071c00 tx=0x7ff048009e30 comp rx=0 tx=0).stop 2024-03-29T08:13:01.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:01.445+0000 7ff063000700 1 -- 172.21.15.37:0/1131929045 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff05c102e20 msgr2=0x7ff05c0702c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:13:01.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:01.445+0000 7ff063000700 1 --2- 172.21.15.37:0/1131929045 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff05c102e20 0x7ff05c0702c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7ff04c009fd0 tx=0x7ff04c00b040 comp rx=0 tx=0).stop 2024-03-29T08:13:01.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:01.445+0000 7ff063000700 1 -- 172.21.15.37:0/1131929045 shutdown_connections 2024-03-29T08:13:01.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:01.445+0000 7ff063000700 1 --2- 172.21.15.37:0/1131929045 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7ff04006cd40 0x7ff04006f1f0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:13:01.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:01.445+0000 7ff063000700 1 --2- 172.21.15.37:0/1131929045 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff05c102e20 0x7ff05c0702c0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:13:01.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:01.445+0000 7ff063000700 1 --2- 172.21.15.37:0/1131929045 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff05c1037c0 0x7ff05c070800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:13:01.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:01.445+0000 7ff063000700 1 -- 172.21.15.37:0/1131929045 >> 172.21.15.37:0/1131929045 conn(0x7ff05c0f89d0 msgr2=0x7ff05c0f9a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:13:01.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:01.445+0000 7ff063000700 1 -- 172.21.15.37:0/1131929045 shutdown_connections 2024-03-29T08:13:01.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:01.445+0000 7ff063000700 1 -- 172.21.15.37:0/1131929045 wait complete. 2024-03-29T08:13:02.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:13:02 smithi148 ceph-mon[106692]: Reconfiguring osd.3 (monmap changed)... 2024-03-29T08:13:02.307 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:13:02 smithi148 ceph-mon[106692]: Reconfiguring daemon osd.3 on smithi148 2024-03-29T08:13:02.307 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:13:02 smithi148 ceph-mon[106692]: from='client.34180 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:13:02.307 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:13:02 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/1131929045' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T08:13:02.307 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:13:02 smithi148 ceph-mon[106692]: pgmap v29: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:13:02.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:13:02 smithi037 ceph-mon[142325]: Reconfiguring osd.3 (monmap changed)... 2024-03-29T08:13:02.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:13:02 smithi037 ceph-mon[142325]: Reconfiguring daemon osd.3 on smithi148 2024-03-29T08:13:02.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:13:02 smithi037 ceph-mon[142325]: from='client.34180 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:13:02.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:13:02 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/1131929045' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T08:13:02.428 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:13:02 smithi037 ceph-mon[142325]: pgmap v29: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:13:03.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:13:03 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:13:03.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:13:03 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:13:03.807 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:13:03 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-29T08:13:03.807 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:13:03 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T08:13:03.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:13:03 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:13:03.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:13:03 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:13:03.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:13:03 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-29T08:13:03.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:13:03 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T08:13:04.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:13:04 smithi148 ceph-mon[106692]: Reconfiguring osd.4 (monmap changed)... 2024-03-29T08:13:04.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:13:04 smithi148 ceph-mon[106692]: Reconfiguring daemon osd.4 on smithi148 2024-03-29T08:13:04.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:13:04 smithi148 ceph-mon[106692]: pgmap v30: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:13:04.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:13:04 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:13:04.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:13:04 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:13:04.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:13:04 smithi037 ceph-mon[142325]: Reconfiguring osd.4 (monmap changed)... 2024-03-29T08:13:04.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:13:04 smithi037 ceph-mon[142325]: Reconfiguring daemon osd.4 on smithi148 2024-03-29T08:13:04.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:13:04 smithi037 ceph-mon[142325]: pgmap v30: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:13:04.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:13:04 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:13:04.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:13:04 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:13:05.906 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:13:05 smithi148 ceph-mon[106692]: Reconfiguring osd.5 (monmap changed)... 2024-03-29T08:13:05.906 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:13:05 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-29T08:13:05.906 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:13:05 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T08:13:05.906 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:13:05 smithi148 ceph-mon[106692]: Reconfiguring daemon osd.5 on smithi148 2024-03-29T08:13:05.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:13:05 smithi037 ceph-mon[142325]: Reconfiguring osd.5 (monmap changed)... 2024-03-29T08:13:05.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:13:05 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-29T08:13:05.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:13:05 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T08:13:05.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:13:05 smithi037 ceph-mon[142325]: Reconfiguring daemon osd.5 on smithi148 2024-03-29T08:13:06.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:13:06 smithi037 ceph-mon[142325]: pgmap v31: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:13:06.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:13:06 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:13:06.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:13:06 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:13:06.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:13:06 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi148.sqteih", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-29T08:13:06.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:13:06 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T08:13:07.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:13:06 smithi148 ceph-mon[106692]: pgmap v31: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:13:07.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:13:06 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:13:07.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:13:06 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:13:07.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:13:06 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi148.sqteih", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-29T08:13:07.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:13:06 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T08:13:07.840 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:13:07 smithi148 ceph-mon[106692]: Reconfiguring mds.cephfs.smithi148.sqteih (monmap changed)... 2024-03-29T08:13:07.840 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:13:07 smithi148 ceph-mon[106692]: Reconfiguring daemon mds.cephfs.smithi148.sqteih on smithi148 2024-03-29T08:13:07.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:13:07 smithi037 ceph-mon[142325]: Reconfiguring mds.cephfs.smithi148.sqteih (monmap changed)... 2024-03-29T08:13:07.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:13:07 smithi037 ceph-mon[142325]: Reconfiguring daemon mds.cephfs.smithi148.sqteih on smithi148 2024-03-29T08:13:08.888 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:13:08 smithi148 ceph-mon[106692]: pgmap v32: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:13:08.888 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:13:08 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:13:08.888 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:13:08 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:13:08.888 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:13:08 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi148.qukpjh", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-29T08:13:08.888 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:13:08 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T08:13:08.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:13:08 smithi037 ceph-mon[142325]: pgmap v32: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:13:08.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:13:08 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:13:08.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:13:08 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:13:08.928 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:13:08 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi148.qukpjh", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-29T08:13:08.928 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:13:08 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T08:13:09.870 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:13:09 smithi148 ceph-mon[106692]: Reconfiguring mds.cephfs.smithi148.qukpjh (monmap changed)... 2024-03-29T08:13:09.870 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:13:09 smithi148 ceph-mon[106692]: Reconfiguring daemon mds.cephfs.smithi148.qukpjh on smithi148 2024-03-29T08:13:09.870 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:13:09 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:13:09.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:13:09 smithi037 ceph-mon[142325]: Reconfiguring mds.cephfs.smithi148.qukpjh (monmap changed)... 2024-03-29T08:13:09.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:13:09 smithi037 ceph-mon[142325]: Reconfiguring daemon mds.cephfs.smithi148.qukpjh on smithi148 2024-03-29T08:13:09.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:13:09 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:13:10.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:13:10 smithi037 ceph-mon[142325]: pgmap v33: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:13:10.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:13:10 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:13:10.928 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:13:10 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:13:10.928 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:13:10 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T08:13:10.928 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:13:10 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:13:10.928 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:13:10 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:13:10.928 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:13:10 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:13:10.928 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:13:10 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi037"}]: dispatch 2024-03-29T08:13:10.928 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:13:10 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi037"}]': finished 2024-03-29T08:13:10.928 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:13:10 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi148"}]: dispatch 2024-03-29T08:13:10.928 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:13:10 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi148"}]': finished 2024-03-29T08:13:11.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:13:10 smithi148 ceph-mon[106692]: pgmap v33: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:13:11.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:13:10 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:13:11.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:13:10 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:13:11.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:13:10 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T08:13:11.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:13:10 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:13:11.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:13:10 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:13:11.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:13:10 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:13:11.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:13:10 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi037"}]: dispatch 2024-03-29T08:13:11.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:13:10 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi037"}]': finished 2024-03-29T08:13:11.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:13:10 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi148"}]: dispatch 2024-03-29T08:13:11.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:13:10 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi148"}]': finished 2024-03-29T08:13:11.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:13:11 smithi037 ceph-mon[142325]: Upgrade: Setting container_image for all mon 2024-03-29T08:13:11.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:13:11 smithi037 ceph-mon[142325]: pgmap v34: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:13:12.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:13:11 smithi148 ceph-mon[106692]: Upgrade: Setting container_image for all mon 2024-03-29T08:13:12.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:13:11 smithi148 ceph-mon[106692]: pgmap v34: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:13:13.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:13:12 smithi148 ceph-mon[106692]: Upgrade: Updating crash.smithi037 (1/2) 2024-03-29T08:13:13.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:13:12 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:13:13.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:13:12 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi037", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-29T08:13:13.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:13:12 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T08:13:13.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:13:12 smithi148 ceph-mon[106692]: Deploying daemon crash.smithi037 on smithi037 2024-03-29T08:13:13.150 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:13:12 smithi037 ceph-mon[142325]: Upgrade: Updating crash.smithi037 (1/2) 2024-03-29T08:13:13.151 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:13:12 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:13:13.151 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:13:12 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi037", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-29T08:13:13.151 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:13:12 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T08:13:13.151 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:13:12 smithi037 ceph-mon[142325]: Deploying daemon crash.smithi037 on smithi037 2024-03-29T08:13:13.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:13:13 smithi037 ceph-mon[142325]: pgmap v35: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:13:14.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:13:13 smithi148 ceph-mon[106692]: pgmap v35: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:13:16.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:13:16 smithi148 ceph-mon[106692]: pgmap v36: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:13:16.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:13:16 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:13:16.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:13:16 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:13:16.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:13:16 smithi037 ceph-mon[142325]: pgmap v36: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:13:16.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:13:16 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:13:16.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:13:16 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:13:18.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:13:18 smithi148 ceph-mon[106692]: pgmap v37: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:13:18.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:13:18 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:13:18.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:13:18 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi148", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-29T08:13:18.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:13:18 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T08:13:18.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:13:18 smithi037 ceph-mon[142325]: pgmap v37: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:13:18.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:13:18 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:13:18.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:13:18 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi148", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-29T08:13:18.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:13:18 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T08:13:19.782 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:13:19 smithi148 ceph-mon[106692]: Upgrade: Updating crash.smithi148 (2/2) 2024-03-29T08:13:19.783 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:13:19 smithi148 ceph-mon[106692]: Deploying daemon crash.smithi148 on smithi148 2024-03-29T08:13:19.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:13:19 smithi037 ceph-mon[142325]: Upgrade: Updating crash.smithi148 (2/2) 2024-03-29T08:13:19.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:13:19 smithi037 ceph-mon[142325]: Deploying daemon crash.smithi148 on smithi148 2024-03-29T08:13:20.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:13:20 smithi148 ceph-mon[106692]: pgmap v38: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:13:20.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:13:20 smithi037 ceph-mon[142325]: pgmap v38: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:13:22.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:13:22 smithi037 ceph-mon[142325]: pgmap v39: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:13:22.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:13:22 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:13:22.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:13:22 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:13:22.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:13:22 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T08:13:23.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:13:22 smithi148 ceph-mon[106692]: pgmap v39: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:13:23.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:13:22 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:13:23.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:13:22 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:13:23.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:13:22 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T08:13:24.893 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:13:24 smithi037 ceph-mon[142325]: pgmap v40: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:13:24.893 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:13:24 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:13:25.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:13:24 smithi148 ceph-mon[106692]: pgmap v40: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:13:25.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:13:24 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:13:26.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:13:26 smithi037 ceph-mon[142325]: pgmap v41: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:13:26.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:13:26 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:13:26.928 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:13:26 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:13:27.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:13:26 smithi148 ceph-mon[106692]: pgmap v41: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:13:27.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:13:26 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:13:27.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:13:26 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:13:28.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:13:28 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:13:28.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:13:28 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:13:28.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:13:28 smithi148 ceph-mon[106692]: pgmap v42: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:13:28.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:13:28 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:13:28.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:13:28 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:13:28.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:13:28 smithi037 ceph-mon[142325]: pgmap v42: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:13:29.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:13:29 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:13:29.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:13:29 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:13:29.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:13:29 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:13:29.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:13:29 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:13:30.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:13:30 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:13:30.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:13:30 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:13:30.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:13:30 smithi037 ceph-mon[142325]: pgmap v43: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:13:30.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:13:30 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:13:30.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:13:30 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:13:30.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:13:30 smithi148 ceph-mon[106692]: pgmap v43: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:13:31.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:31.593+0000 7f2a21200700 1 -- 172.21.15.37:0/22350601 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2a1c071f90 msgr2=0x7f2a1c070360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:13:31.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:31.593+0000 7f2a21200700 1 --2- 172.21.15.37:0/22350601 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2a1c071f90 0x7f2a1c070360 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f2a0c004970 tx=0x7f2a0c009e30 comp rx=0 tx=0).stop 2024-03-29T08:13:31.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:31.594+0000 7f2a21200700 1 -- 172.21.15.37:0/22350601 shutdown_connections 2024-03-29T08:13:31.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:31.594+0000 7f2a21200700 1 --2- 172.21.15.37:0/22350601 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2a1c0708a0 0x7f2a1c070d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:13:31.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:31.594+0000 7f2a21200700 1 --2- 172.21.15.37:0/22350601 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2a1c071f90 0x7f2a1c070360 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:13:31.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:31.594+0000 7f2a21200700 1 -- 172.21.15.37:0/22350601 >> 172.21.15.37:0/22350601 conn(0x7f2a1c0f89b0 msgr2=0x7f2a1c0fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:13:31.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:31.594+0000 7f2a21200700 1 -- 172.21.15.37:0/22350601 shutdown_connections 2024-03-29T08:13:31.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:31.594+0000 7f2a21200700 1 -- 172.21.15.37:0/22350601 wait complete. 2024-03-29T08:13:31.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:31.594+0000 7f2a21200700 1 Processor -- start 2024-03-29T08:13:31.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:31.595+0000 7f2a21200700 1 -- start start 2024-03-29T08:13:31.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:31.595+0000 7f2a21200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2a1c0708a0 0x7f2a1c11a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:13:31.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:31.595+0000 7f2a21200700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2a1c071f90 0x7f2a1c11a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:13:31.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:31.595+0000 7f2a21200700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a1c11aca0 con 0x7f2a1c0708a0 2024-03-29T08:13:31.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:31.595+0000 7f2a21200700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a1c114090 con 0x7f2a1c071f90 2024-03-29T08:13:31.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:31.595+0000 7f2a19e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2a1c0708a0 0x7f2a1c11a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:13:31.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:31.595+0000 7f2a19e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2a1c0708a0 0x7f2a1c11a080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44018/0 (socket says 172.21.15.37:44018) 2024-03-29T08:13:31.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:31.595+0000 7f2a19e00700 1 -- 172.21.15.37:0/331499605 learned_addr learned my addr 172.21.15.37:0/331499605 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:13:31.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:31.595+0000 7f2a19400700 1 --2- 172.21.15.37:0/331499605 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2a1c071f90 0x7f2a1c11a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:13:31.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:31.596+0000 7f2a19e00700 1 -- 172.21.15.37:0/331499605 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2a1c071f90 msgr2=0x7f2a1c11a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:13:31.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:31.596+0000 7f2a19e00700 1 --2- 172.21.15.37:0/331499605 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2a1c071f90 0x7f2a1c11a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:13:31.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:31.596+0000 7f2a19e00700 1 -- 172.21.15.37:0/331499605 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a0c005040 con 0x7f2a1c0708a0 2024-03-29T08:13:31.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:31.596+0000 7f2a19e00700 1 --2- 172.21.15.37:0/331499605 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2a1c0708a0 0x7f2a1c11a080 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f2a0c004f50 tx=0x7f2a0c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:13:31.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:31.596+0000 7f2a12000700 1 -- 172.21.15.37:0/331499605 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2a0c00e040 con 0x7f2a1c0708a0 2024-03-29T08:13:31.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:31.596+0000 7f2a12000700 1 -- 172.21.15.37:0/331499605 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f2a0c013420 con 0x7f2a1c0708a0 2024-03-29T08:13:31.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:31.596+0000 7f2a21200700 1 -- 172.21.15.37:0/331499605 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a1c114310 con 0x7f2a1c0708a0 2024-03-29T08:13:31.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:31.596+0000 7f2a12000700 1 -- 172.21.15.37:0/331499605 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2a0c01c610 con 0x7f2a1c0708a0 2024-03-29T08:13:31.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:31.596+0000 7f2a21200700 1 -- 172.21.15.37:0/331499605 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a1c114850 con 0x7f2a1c0708a0 2024-03-29T08:13:31.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:31.597+0000 7f2a21200700 1 -- 172.21.15.37:0/331499605 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a1c071aa0 con 0x7f2a1c0708a0 2024-03-29T08:13:31.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:31.598+0000 7f2a12000700 1 -- 172.21.15.37:0/331499605 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 39) v1 ==== 90944+0+0 (secure 0 0 0) 0x7f2a0c013590 con 0x7f2a1c0708a0 2024-03-29T08:13:31.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:31.599+0000 7f2a12000700 1 --2- 172.21.15.37:0/331499605 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f2a0006cd40 0x7f2a0006f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:13:31.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:31.600+0000 7f2a12000700 1 -- 172.21.15.37:0/331499605 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f2a0c08d450 con 0x7f2a1c0708a0 2024-03-29T08:13:31.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:31.600+0000 7f2a19400700 1 --2- 172.21.15.37:0/331499605 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f2a0006cd40 0x7f2a0006f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:13:31.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:31.601+0000 7f2a19400700 1 --2- 172.21.15.37:0/331499605 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f2a0006cd40 0x7f2a0006f1f0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f2a08006fd0 tx=0x7f2a08008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:13:31.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:31.605+0000 7f2a12000700 1 -- 172.21.15.37:0/331499605 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f2a0c057270 con 0x7f2a1c0708a0 2024-03-29T08:13:31.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:31.850+0000 7f2a21200700 1 -- 172.21.15.37:0/331499605 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f2a1c110460 con 0x7f2a0006cd40 2024-03-29T08:13:31.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:31.852+0000 7f2a12000700 1 -- 172.21.15.37:0/331499605 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+402 (secure 0 0 0) 0x7f2a1c110460 con 0x7f2a0006cd40 2024-03-29T08:13:31.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:31.855+0000 7f2a21200700 1 -- 172.21.15.37:0/331499605 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f2a0006cd40 msgr2=0x7f2a0006f1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:13:31.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:31.855+0000 7f2a21200700 1 --2- 172.21.15.37:0/331499605 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f2a0006cd40 0x7f2a0006f1f0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f2a08006fd0 tx=0x7f2a08008040 comp rx=0 tx=0).stop 2024-03-29T08:13:31.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:31.855+0000 7f2a21200700 1 -- 172.21.15.37:0/331499605 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2a1c0708a0 msgr2=0x7f2a1c11a080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:13:31.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:31.855+0000 7f2a21200700 1 --2- 172.21.15.37:0/331499605 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2a1c0708a0 0x7f2a1c11a080 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f2a0c004f50 tx=0x7f2a0c009e30 comp rx=0 tx=0).stop 2024-03-29T08:13:31.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:31.855+0000 7f2a21200700 1 -- 172.21.15.37:0/331499605 shutdown_connections 2024-03-29T08:13:31.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:31.855+0000 7f2a21200700 1 --2- 172.21.15.37:0/331499605 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f2a0006cd40 0x7f2a0006f1f0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:13:31.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:31.855+0000 7f2a21200700 1 --2- 172.21.15.37:0/331499605 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2a1c0708a0 0x7f2a1c11a080 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:13:31.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:31.855+0000 7f2a21200700 1 --2- 172.21.15.37:0/331499605 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2a1c071f90 0x7f2a1c11a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:13:31.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:31.855+0000 7f2a21200700 1 -- 172.21.15.37:0/331499605 >> 172.21.15.37:0/331499605 conn(0x7f2a1c0f89b0 msgr2=0x7f2a1c0f9b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:13:31.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:31.855+0000 7f2a21200700 1 -- 172.21.15.37:0/331499605 shutdown_connections 2024-03-29T08:13:31.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:31.855+0000 7f2a21200700 1 -- 172.21.15.37:0/331499605 wait complete. 2024-03-29T08:13:31.871 INFO:teuthology.orchestra.run.smithi037.stdout:true 2024-03-29T08:13:31.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:31.997+0000 7fdebd200700 1 -- 172.21.15.37:0/2423419935 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdeb80fe830 msgr2=0x7fdeb8106f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:13:31.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:31.997+0000 7fdebd200700 1 --2- 172.21.15.37:0/2423419935 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdeb80fe830 0x7fdeb8106f00 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fdea4004940 tx=0x7fdea4009e30 comp rx=0 tx=0).stop 2024-03-29T08:13:31.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:31.997+0000 7fdebd200700 1 -- 172.21.15.37:0/2423419935 shutdown_connections 2024-03-29T08:13:31.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:31.997+0000 7fdebd200700 1 --2- 172.21.15.37:0/2423419935 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdeb80fe830 0x7fdeb8106f00 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:13:31.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:31.997+0000 7fdebd200700 1 --2- 172.21.15.37:0/2423419935 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdeb80fde90 0x7fdeb80fe260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:13:31.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:31.997+0000 7fdebd200700 1 -- 172.21.15.37:0/2423419935 >> 172.21.15.37:0/2423419935 conn(0x7fdeb8072bc0 msgr2=0x7fdeb8072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:13:31.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:31.997+0000 7fdebd200700 1 -- 172.21.15.37:0/2423419935 shutdown_connections 2024-03-29T08:13:31.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:31.997+0000 7fdebd200700 1 -- 172.21.15.37:0/2423419935 wait complete. 2024-03-29T08:13:32.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:31.998+0000 7fdebd200700 1 Processor -- start 2024-03-29T08:13:32.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:31.998+0000 7fdebd200700 1 -- start start 2024-03-29T08:13:32.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:31.999+0000 7fdebd200700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdeb80fde90 0x7fdeb8115c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:13:32.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:31.999+0000 7fdebd200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdeb80fe830 0x7fdeb81161b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:13:32.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:31.999+0000 7fdebd200700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdeb8116800 con 0x7fdeb80fe830 2024-03-29T08:13:32.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:31.999+0000 7fdebd200700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdeb810fbd0 con 0x7fdeb80fde90 2024-03-29T08:13:32.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:31.999+0000 7fdeb5400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdeb80fe830 0x7fdeb81161b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:13:32.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:31.999+0000 7fdeb5400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdeb80fe830 0x7fdeb81161b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44036/0 (socket says 172.21.15.37:44036) 2024-03-29T08:13:32.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:31.999+0000 7fdeb5400700 1 -- 172.21.15.37:0/3243353633 learned_addr learned my addr 172.21.15.37:0/3243353633 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:13:32.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:31.999+0000 7fdeb5400700 1 -- 172.21.15.37:0/3243353633 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdeb80fde90 msgr2=0x7fdeb8115c70 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-03-29T08:13:32.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:31.999+0000 7fdeb5400700 1 --2- 172.21.15.37:0/3243353633 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdeb80fde90 0x7fdeb8115c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:13:32.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:31.999+0000 7fdeb5400700 1 -- 172.21.15.37:0/3243353633 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdea4005040 con 0x7fdeb80fe830 2024-03-29T08:13:32.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:32.000+0000 7fdeb5400700 1 --2- 172.21.15.37:0/3243353633 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdeb80fe830 0x7fdeb81161b0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fdea4000c00 tx=0x7fdea4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:13:32.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:32.000+0000 7fdeae000700 1 -- 172.21.15.37:0/3243353633 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdea4021070 con 0x7fdeb80fe830 2024-03-29T08:13:32.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:32.000+0000 7fdeae000700 1 -- 172.21.15.37:0/3243353633 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fdea4013e00 con 0x7fdeb80fe830 2024-03-29T08:13:32.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:32.000+0000 7fdebd200700 1 -- 172.21.15.37:0/3243353633 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdeb810fe50 con 0x7fdeb80fe830 2024-03-29T08:13:32.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:32.000+0000 7fdeae000700 1 -- 172.21.15.37:0/3243353633 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdea400e070 con 0x7fdeb80fe830 2024-03-29T08:13:32.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:32.000+0000 7fdebd200700 1 -- 172.21.15.37:0/3243353633 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdeb8110390 con 0x7fdeb80fe830 2024-03-29T08:13:32.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:32.001+0000 7fdebd200700 1 -- 172.21.15.37:0/3243353633 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdeb8106a20 con 0x7fdeb80fe830 2024-03-29T08:13:32.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:32.002+0000 7fdeae000700 1 -- 172.21.15.37:0/3243353633 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 39) v1 ==== 90944+0+0 (secure 0 0 0) 0x7fdea4013950 con 0x7fdeb80fe830 2024-03-29T08:13:32.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:32.003+0000 7fdeae000700 1 --2- 172.21.15.37:0/3243353633 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fde9c06cd40 0x7fde9c06f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:13:32.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:32.003+0000 7fdeae000700 1 -- 172.21.15.37:0/3243353633 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fdea408d250 con 0x7fdeb80fe830 2024-03-29T08:13:32.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:32.003+0000 7fdeb5e00700 1 --2- 172.21.15.37:0/3243353633 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fde9c06cd40 0x7fde9c06f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:13:32.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:32.004+0000 7fdeb5e00700 1 --2- 172.21.15.37:0/3243353633 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fde9c06cd40 0x7fde9c06f1f0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fdea8009fd0 tx=0x7fdea800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:13:32.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:32.007+0000 7fdeae000700 1 -- 172.21.15.37:0/3243353633 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fdea405aba0 con 0x7fdeb80fe830 2024-03-29T08:13:32.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:32.242+0000 7fdebd200700 1 -- 172.21.15.37:0/3243353633 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fdeb81112c0 con 0x7fde9c06cd40 2024-03-29T08:13:32.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:32.243+0000 7fdeae000700 1 -- 172.21.15.37:0/3243353633 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+402 (secure 0 0 0) 0x7fdeb81112c0 con 0x7fde9c06cd40 2024-03-29T08:13:32.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:32.245+0000 7fdebd200700 1 -- 172.21.15.37:0/3243353633 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fde9c06cd40 msgr2=0x7fde9c06f1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:13:32.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:32.245+0000 7fdebd200700 1 --2- 172.21.15.37:0/3243353633 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fde9c06cd40 0x7fde9c06f1f0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fdea8009fd0 tx=0x7fdea800b040 comp rx=0 tx=0).stop 2024-03-29T08:13:32.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:32.245+0000 7fdebd200700 1 -- 172.21.15.37:0/3243353633 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdeb80fe830 msgr2=0x7fdeb81161b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:13:32.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:32.245+0000 7fdebd200700 1 --2- 172.21.15.37:0/3243353633 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdeb80fe830 0x7fdeb81161b0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fdea4000c00 tx=0x7fdea4009e30 comp rx=0 tx=0).stop 2024-03-29T08:13:32.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:32.246+0000 7fdebd200700 1 -- 172.21.15.37:0/3243353633 shutdown_connections 2024-03-29T08:13:32.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:32.246+0000 7fdebd200700 1 --2- 172.21.15.37:0/3243353633 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fde9c06cd40 0x7fde9c06f1f0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:13:32.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:32.246+0000 7fdebd200700 1 --2- 172.21.15.37:0/3243353633 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdeb80fde90 0x7fdeb8115c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:13:32.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:32.246+0000 7fdebd200700 1 --2- 172.21.15.37:0/3243353633 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdeb80fe830 0x7fdeb81161b0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:13:32.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:32.246+0000 7fdebd200700 1 -- 172.21.15.37:0/3243353633 >> 172.21.15.37:0/3243353633 conn(0x7fdeb8072bc0 msgr2=0x7fdeb80fb6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:13:32.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:32.246+0000 7fdebd200700 1 -- 172.21.15.37:0/3243353633 shutdown_connections 2024-03-29T08:13:32.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:32.246+0000 7fdebd200700 1 -- 172.21.15.37:0/3243353633 wait complete. 2024-03-29T08:13:32.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:32.385+0000 7f0851800700 1 -- 172.21.15.37:0/1877110320 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f084c1037a0 msgr2=0x7f084c0fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:13:32.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:32.385+0000 7f0851800700 1 --2- 172.21.15.37:0/1877110320 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f084c1037a0 0x7f084c0fce10 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f0838004940 tx=0x7f0838009e30 comp rx=0 tx=0).stop 2024-03-29T08:13:32.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:32.385+0000 7f0851800700 1 -- 172.21.15.37:0/1877110320 shutdown_connections 2024-03-29T08:13:32.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:32.385+0000 7f0851800700 1 --2- 172.21.15.37:0/1877110320 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f084c1037a0 0x7f084c0fce10 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:13:32.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:32.385+0000 7f0851800700 1 --2- 172.21.15.37:0/1877110320 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f084c102e00 0x7f084c1031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:13:32.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:32.385+0000 7f0851800700 1 -- 172.21.15.37:0/1877110320 >> 172.21.15.37:0/1877110320 conn(0x7f084c0f89b0 msgr2=0x7f084c0fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:13:32.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:32.385+0000 7f0851800700 1 -- 172.21.15.37:0/1877110320 shutdown_connections 2024-03-29T08:13:32.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:32.385+0000 7f0851800700 1 -- 172.21.15.37:0/1877110320 wait complete. 2024-03-29T08:13:32.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:32.386+0000 7f0851800700 1 Processor -- start 2024-03-29T08:13:32.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:32.386+0000 7f0851800700 1 -- start start 2024-03-29T08:13:32.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:32.386+0000 7f0851800700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f084c102e00 0x7f084c0702c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:13:32.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:32.386+0000 7f0851800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f084c1037a0 0x7f084c070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:13:32.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:32.386+0000 7f0851800700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f084c070d90 con 0x7f084c1037a0 2024-03-29T08:13:32.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:32.386+0000 7f0851800700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f084c070ed0 con 0x7f084c102e00 2024-03-29T08:13:32.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:32.387+0000 7f084a000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f084c1037a0 0x7f084c070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:13:32.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:32.387+0000 7f084a000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f084c1037a0 0x7f084c070800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44058/0 (socket says 172.21.15.37:44058) 2024-03-29T08:13:32.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:32.387+0000 7f084a000700 1 -- 172.21.15.37:0/1348643962 learned_addr learned my addr 172.21.15.37:0/1348643962 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:13:32.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:32.387+0000 7f084aa00700 1 --2- 172.21.15.37:0/1348643962 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f084c102e00 0x7f084c0702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:13:32.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:32.387+0000 7f084a000700 1 -- 172.21.15.37:0/1348643962 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f084c102e00 msgr2=0x7f084c0702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:13:32.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:32.387+0000 7f084a000700 1 --2- 172.21.15.37:0/1348643962 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f084c102e00 0x7f084c0702c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:13:32.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:32.387+0000 7f084a000700 1 -- 172.21.15.37:0/1348643962 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0838005040 con 0x7f084c1037a0 2024-03-29T08:13:32.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:32.387+0000 7f084a000700 1 --2- 172.21.15.37:0/1348643962 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f084c1037a0 0x7f084c070800 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f0838000c00 tx=0x7f0838009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:13:32.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:32.388+0000 7f0842a00700 1 -- 172.21.15.37:0/1348643962 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f083800e070 con 0x7f084c1037a0 2024-03-29T08:13:32.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:32.388+0000 7f0851800700 1 -- 172.21.15.37:0/1348643962 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f084c071120 con 0x7f084c1037a0 2024-03-29T08:13:32.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:32.388+0000 7f0842a00700 1 -- 172.21.15.37:0/1348643962 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f08380093b0 con 0x7f084c1037a0 2024-03-29T08:13:32.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:32.388+0000 7f0842a00700 1 -- 172.21.15.37:0/1348643962 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f083801ccb0 con 0x7f084c1037a0 2024-03-29T08:13:32.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:32.388+0000 7f0851800700 1 -- 172.21.15.37:0/1348643962 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f084c11a140 con 0x7f084c1037a0 2024-03-29T08:13:32.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:32.389+0000 7f0851800700 1 -- 172.21.15.37:0/1348643962 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f084c0fe550 con 0x7f084c1037a0 2024-03-29T08:13:32.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:32.390+0000 7f0842a00700 1 -- 172.21.15.37:0/1348643962 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 39) v1 ==== 90944+0+0 (secure 0 0 0) 0x7f083801c490 con 0x7f084c1037a0 2024-03-29T08:13:32.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:32.391+0000 7f0842a00700 1 --2- 172.21.15.37:0/1348643962 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f083006cd40 0x7f083006f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:13:32.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:32.391+0000 7f0842a00700 1 -- 172.21.15.37:0/1348643962 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f083808d5a0 con 0x7f084c1037a0 2024-03-29T08:13:32.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:32.391+0000 7f084aa00700 1 --2- 172.21.15.37:0/1348643962 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f083006cd40 0x7f083006f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:13:32.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:32.392+0000 7f084aa00700 1 --2- 172.21.15.37:0/1348643962 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f083006cd40 0x7f083006f1f0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f083c009b80 tx=0x7f083c00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:13:32.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:32.396+0000 7f0842a00700 1 -- 172.21.15.37:0/1348643962 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f0838057500 con 0x7f084c1037a0 2024-03-29T08:13:32.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:32.629+0000 7f0851800700 1 -- 172.21.15.37:0/1348643962 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f084c109e40 con 0x7f083006cd40 2024-03-29T08:13:32.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:32.641+0000 7f0842a00700 1 -- 172.21.15.37:0/1348643962 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f083c00f040 con 0x7f083006cd40 2024-03-29T08:13:32.643 INFO:teuthology.orchestra.run.smithi037.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-29T08:13:32.643 INFO:teuthology.orchestra.run.smithi037.stdout:alertmanager.smithi037 smithi037 *:9093,9094 running (3m) 4s ago 16m 14.8M - 0.25.0 c8568f914cd2 2f9f9746ed0f 2024-03-29T08:13:32.643 INFO:teuthology.orchestra.run.smithi037.stdout:crash.smithi037 smithi037 running (17s) 4s ago 16m 6647k - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 178d416a3186 2024-03-29T08:13:32.643 INFO:teuthology.orchestra.run.smithi037.stdout:crash.smithi148 smithi148 running (11s) 6s ago 15m 6656k - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 23666989d58f 2024-03-29T08:13:32.643 INFO:teuthology.orchestra.run.smithi037.stdout:grafana.smithi037 smithi037 *:3000 running (3m) 4s ago 16m 73.9M - 9.4.7 954c08fa6188 6b976fe49952 2024-03-29T08:13:32.643 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi037.iifsmh smithi037 running (9m) 4s ago 9m 110M - 17.2.7-789-g4213e17c cf22eb8683be 074e1acf331f 2024-03-29T08:13:32.643 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi037.mmwgxe smithi037 running (9m) 4s ago 9m 114M - 17.2.7-789-g4213e17c cf22eb8683be 9a5b1e72de5c 2024-03-29T08:13:32.643 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi148.qukpjh smithi148 running (9m) 6s ago 9m 14.1M - 17.2.7-789-g4213e17c cf22eb8683be 8e469782a273 2024-03-29T08:13:32.643 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi148.sqteih smithi148 running (9m) 6s ago 9m 14.0M - 17.2.7-789-g4213e17c cf22eb8683be 8e68f858bb22 2024-03-29T08:13:32.643 INFO:teuthology.orchestra.run.smithi037.stdout:mgr.smithi037.wjikii smithi037 *:8443,9283,8765 running (5m) 4s ago 17m 493M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff c5a1417b9417 2024-03-29T08:13:32.643 INFO:teuthology.orchestra.run.smithi037.stdout:mgr.smithi148.onhgvm smithi148 *:8443,9283,8765 running (4m) 6s ago 15m 447M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff fcfdbce062bd 2024-03-29T08:13:32.644 INFO:teuthology.orchestra.run.smithi037.stdout:mon.smithi037 smithi037 running (94s) 4s ago 18m 40.6M 2048M 18.2.2-1015-gd0e392d9 1cc2f45c17ff eea2060783af 2024-03-29T08:13:32.644 INFO:teuthology.orchestra.run.smithi037.stdout:mon.smithi148 smithi148 running (63s) 6s ago 15m 35.5M 2048M 18.2.2-1015-gd0e392d9 1cc2f45c17ff ad4b2c5748be 2024-03-29T08:13:32.644 INFO:teuthology.orchestra.run.smithi037.stdout:node-exporter.smithi037 smithi037 *:9100 running (4m) 4s ago 16m 7717k - 1.5.0 0da6a335fe13 26b8ae9cb878 2024-03-29T08:13:32.644 INFO:teuthology.orchestra.run.smithi037.stdout:node-exporter.smithi148 smithi148 *:9100 running (4m) 6s ago 15m 7600k - 1.5.0 0da6a335fe13 98414e4e64ed 2024-03-29T08:13:32.644 INFO:teuthology.orchestra.run.smithi037.stdout:osd.0 smithi037 running (14m) 4s ago 14m 218M 1619M 17.2.7-789-g4213e17c cf22eb8683be 559fccd0ebe1 2024-03-29T08:13:32.644 INFO:teuthology.orchestra.run.smithi037.stdout:osd.1 smithi037 running (13m) 4s ago 13m 229M 1619M 17.2.7-789-g4213e17c cf22eb8683be 5582cff11b7a 2024-03-29T08:13:32.644 INFO:teuthology.orchestra.run.smithi037.stdout:osd.2 smithi037 running (13m) 4s ago 13m 198M 1619M 17.2.7-789-g4213e17c cf22eb8683be f809bfab13e5 2024-03-29T08:13:32.644 INFO:teuthology.orchestra.run.smithi037.stdout:osd.3 smithi148 running (12m) 6s ago 12m 261M 2642M 17.2.7-789-g4213e17c cf22eb8683be a0a8224bb4b0 2024-03-29T08:13:32.644 INFO:teuthology.orchestra.run.smithi037.stdout:osd.4 smithi148 running (12m) 6s ago 12m 233M 2642M 17.2.7-789-g4213e17c cf22eb8683be a00b69047ed7 2024-03-29T08:13:32.644 INFO:teuthology.orchestra.run.smithi037.stdout:osd.5 smithi148 running (11m) 6s ago 11m 234M 2642M 17.2.7-789-g4213e17c cf22eb8683be 044a3f7db7f8 2024-03-29T08:13:32.644 INFO:teuthology.orchestra.run.smithi037.stdout:prometheus.smithi037 smithi037 *:9095 running (3m) 4s ago 15m 51.5M - 2.43.0 a07b618ecd1d 30c0223ec589 2024-03-29T08:13:32.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:32.644+0000 7f0851800700 1 -- 172.21.15.37:0/1348643962 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f083006cd40 msgr2=0x7f083006f1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:13:32.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:32.644+0000 7f0851800700 1 --2- 172.21.15.37:0/1348643962 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f083006cd40 0x7f083006f1f0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f083c009b80 tx=0x7f083c00b040 comp rx=0 tx=0).stop 2024-03-29T08:13:32.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:32.644+0000 7f0851800700 1 -- 172.21.15.37:0/1348643962 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f084c1037a0 msgr2=0x7f084c070800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:13:32.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:32.644+0000 7f0851800700 1 --2- 172.21.15.37:0/1348643962 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f084c1037a0 0x7f084c070800 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f0838000c00 tx=0x7f0838009e30 comp rx=0 tx=0).stop 2024-03-29T08:13:32.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:32.644+0000 7f0851800700 1 -- 172.21.15.37:0/1348643962 shutdown_connections 2024-03-29T08:13:32.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:32.644+0000 7f0851800700 1 --2- 172.21.15.37:0/1348643962 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f083006cd40 0x7f083006f1f0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:13:32.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:32.644+0000 7f0851800700 1 --2- 172.21.15.37:0/1348643962 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f084c102e00 0x7f084c0702c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:13:32.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:32.644+0000 7f0851800700 1 --2- 172.21.15.37:0/1348643962 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f084c1037a0 0x7f084c070800 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:13:32.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:32.644+0000 7f0851800700 1 -- 172.21.15.37:0/1348643962 >> 172.21.15.37:0/1348643962 conn(0x7f084c0f89b0 msgr2=0x7f084c0f9a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:13:32.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:32.644+0000 7f0851800700 1 -- 172.21.15.37:0/1348643962 shutdown_connections 2024-03-29T08:13:32.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:32.644+0000 7f0851800700 1 -- 172.21.15.37:0/1348643962 wait complete. 2024-03-29T08:13:32.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:32.809+0000 7fe0db400700 1 -- 172.21.15.37:0/652244485 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe0d40fe830 msgr2=0x7fe0d4106f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:13:32.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:32.809+0000 7fe0db400700 1 --2- 172.21.15.37:0/652244485 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe0d40fe830 0x7fe0d4106f00 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fe0c0004940 tx=0x7fe0c0009e30 comp rx=0 tx=0).stop 2024-03-29T08:13:32.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:32.809+0000 7fe0db400700 1 -- 172.21.15.37:0/652244485 shutdown_connections 2024-03-29T08:13:32.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:32.809+0000 7fe0db400700 1 --2- 172.21.15.37:0/652244485 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe0d40fe830 0x7fe0d4106f00 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:13:32.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:32.809+0000 7fe0db400700 1 --2- 172.21.15.37:0/652244485 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe0d40fde90 0x7fe0d40fe260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:13:32.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:32.809+0000 7fe0db400700 1 -- 172.21.15.37:0/652244485 >> 172.21.15.37:0/652244485 conn(0x7fe0d4072bc0 msgr2=0x7fe0d4072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:13:32.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:32.809+0000 7fe0db400700 1 -- 172.21.15.37:0/652244485 shutdown_connections 2024-03-29T08:13:32.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:32.809+0000 7fe0db400700 1 -- 172.21.15.37:0/652244485 wait complete. 2024-03-29T08:13:32.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:32.810+0000 7fe0db400700 1 Processor -- start 2024-03-29T08:13:32.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:32.810+0000 7fe0db400700 1 -- start start 2024-03-29T08:13:32.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:32.810+0000 7fe0db400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe0d40fde90 0x7fe0d4113aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:13:32.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:32.810+0000 7fe0db400700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe0d40fe830 0x7fe0d4113fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:13:32.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:32.811+0000 7fe0db400700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0d4114630 con 0x7fe0d40fde90 2024-03-29T08:13:32.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:32.811+0000 7fe0db400700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0d410da20 con 0x7fe0d40fe830 2024-03-29T08:13:32.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:32.811+0000 7fe0d8a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe0d40fde90 0x7fe0d4113aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:13:32.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:32.811+0000 7fe0d8a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe0d40fde90 0x7fe0d4113aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44074/0 (socket says 172.21.15.37:44074) 2024-03-29T08:13:32.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:32.811+0000 7fe0d8a00700 1 -- 172.21.15.37:0/3039740188 learned_addr learned my addr 172.21.15.37:0/3039740188 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:13:32.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:32.811+0000 7fe0d3e00700 1 --2- 172.21.15.37:0/3039740188 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe0d40fe830 0x7fe0d4113fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:13:32.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:32.811+0000 7fe0d8a00700 1 -- 172.21.15.37:0/3039740188 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe0d40fe830 msgr2=0x7fe0d4113fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:13:32.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:32.811+0000 7fe0d8a00700 1 --2- 172.21.15.37:0/3039740188 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe0d40fe830 0x7fe0d4113fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:13:32.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:32.811+0000 7fe0d8a00700 1 -- 172.21.15.37:0/3039740188 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0c0005040 con 0x7fe0d40fde90 2024-03-29T08:13:32.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:32.812+0000 7fe0d8a00700 1 --2- 172.21.15.37:0/3039740188 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe0d40fde90 0x7fe0d4113aa0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fe0c800eb70 tx=0x7fe0c800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:13:32.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:32.812+0000 7fe0d0c00700 1 -- 172.21.15.37:0/3039740188 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe0c8007cc0 con 0x7fe0d40fde90 2024-03-29T08:13:32.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:32.812+0000 7fe0d0c00700 1 -- 172.21.15.37:0/3039740188 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fe0c800f040 con 0x7fe0d40fde90 2024-03-29T08:13:32.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:32.812+0000 7fe0d0c00700 1 -- 172.21.15.37:0/3039740188 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe0c80095f0 con 0x7fe0d40fde90 2024-03-29T08:13:32.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:32.812+0000 7fe0db400700 1 -- 172.21.15.37:0/3039740188 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0d410dca0 con 0x7fe0d40fde90 2024-03-29T08:13:32.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:32.812+0000 7fe0db400700 1 -- 172.21.15.37:0/3039740188 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0d410e1e0 con 0x7fe0d40fde90 2024-03-29T08:13:32.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:32.813+0000 7fe0db400700 1 -- 172.21.15.37:0/3039740188 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0d4101980 con 0x7fe0d40fde90 2024-03-29T08:13:32.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:32.815+0000 7fe0d0c00700 1 -- 172.21.15.37:0/3039740188 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 39) v1 ==== 90944+0+0 (secure 0 0 0) 0x7fe0c8009750 con 0x7fe0d40fde90 2024-03-29T08:13:32.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:32.815+0000 7fe0d0c00700 1 --2- 172.21.15.37:0/3039740188 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fe0bc06cd40 0x7fe0bc06f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:13:32.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:32.816+0000 7fe0d0c00700 1 -- 172.21.15.37:0/3039740188 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fe0c808bde0 con 0x7fe0d40fde90 2024-03-29T08:13:32.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:32.816+0000 7fe0d3e00700 1 --2- 172.21.15.37:0/3039740188 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fe0bc06cd40 0x7fe0bc06f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:13:32.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:32.817+0000 7fe0d3e00700 1 --2- 172.21.15.37:0/3039740188 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fe0bc06cd40 0x7fe0bc06f1f0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fe0c000b0d0 tx=0x7fe0c0009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:13:32.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:32.820+0000 7fe0d0c00700 1 -- 172.21.15.37:0/3039740188 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fe0c8059730 con 0x7fe0d40fde90 2024-03-29T08:13:32.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:13:32 smithi037 ceph-mon[142325]: pgmap v44: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:13:32.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:13:32 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:13:32.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:13:32 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:13:33.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:13:32 smithi148 ceph-mon[106692]: pgmap v44: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:13:33.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:13:32 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:13:33.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:13:32 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:13:33.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:33.232+0000 7fe0db400700 1 -- 172.21.15.37:0/3039740188 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fe0d410f150 con 0x7fe0d40fde90 2024-03-29T08:13:33.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:33.232+0000 7fe0d0c00700 1 -- 172.21.15.37:0/3039740188 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+742 (secure 0 0 0) 0x7fe0c80592c0 con 0x7fe0d40fde90 2024-03-29T08:13:33.233 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-29T08:13:33.233 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": { 2024-03-29T08:13:33.234 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 2 2024-03-29T08:13:33.234 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:13:33.234 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": { 2024-03-29T08:13:33.234 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 2 2024-03-29T08:13:33.234 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:13:33.234 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": { 2024-03-29T08:13:33.234 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 17.2.7-789-g4213e17c (4213e17c4de67016ed14f55d22503f4923b3a901) quincy (stable)": 6 2024-03-29T08:13:33.234 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:13:33.234 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": { 2024-03-29T08:13:33.234 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 17.2.7-789-g4213e17c (4213e17c4de67016ed14f55d22503f4923b3a901) quincy (stable)": 4 2024-03-29T08:13:33.234 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:13:33.234 INFO:teuthology.orchestra.run.smithi037.stdout: "overall": { 2024-03-29T08:13:33.234 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 17.2.7-789-g4213e17c (4213e17c4de67016ed14f55d22503f4923b3a901) quincy (stable)": 10, 2024-03-29T08:13:33.234 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 4 2024-03-29T08:13:33.235 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-29T08:13:33.235 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-29T08:13:33.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:33.235+0000 7fe0db400700 1 -- 172.21.15.37:0/3039740188 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fe0bc06cd40 msgr2=0x7fe0bc06f1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:13:33.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:33.235+0000 7fe0db400700 1 --2- 172.21.15.37:0/3039740188 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fe0bc06cd40 0x7fe0bc06f1f0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fe0c000b0d0 tx=0x7fe0c0009e30 comp rx=0 tx=0).stop 2024-03-29T08:13:33.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:33.235+0000 7fe0db400700 1 -- 172.21.15.37:0/3039740188 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe0d40fde90 msgr2=0x7fe0d4113aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:13:33.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:33.235+0000 7fe0db400700 1 --2- 172.21.15.37:0/3039740188 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe0d40fde90 0x7fe0d4113aa0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fe0c800eb70 tx=0x7fe0c800b040 comp rx=0 tx=0).stop 2024-03-29T08:13:33.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:33.235+0000 7fe0db400700 1 -- 172.21.15.37:0/3039740188 shutdown_connections 2024-03-29T08:13:33.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:33.235+0000 7fe0db400700 1 --2- 172.21.15.37:0/3039740188 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fe0bc06cd40 0x7fe0bc06f1f0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:13:33.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:33.235+0000 7fe0db400700 1 --2- 172.21.15.37:0/3039740188 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe0d40fde90 0x7fe0d4113aa0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:13:33.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:33.235+0000 7fe0db400700 1 --2- 172.21.15.37:0/3039740188 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe0d40fe830 0x7fe0d4113fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:13:33.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:33.235+0000 7fe0db400700 1 -- 172.21.15.37:0/3039740188 >> 172.21.15.37:0/3039740188 conn(0x7fe0d4072bc0 msgr2=0x7fe0d40ff3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:13:33.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:33.235+0000 7fe0db400700 1 -- 172.21.15.37:0/3039740188 shutdown_connections 2024-03-29T08:13:33.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:33.235+0000 7fe0db400700 1 -- 172.21.15.37:0/3039740188 wait complete. 2024-03-29T08:13:33.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:33.386+0000 7ff948c00700 1 -- 172.21.15.37:0/2591334518 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff944071f90 msgr2=0x7ff944070360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:13:33.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:33.386+0000 7ff948c00700 1 --2- 172.21.15.37:0/2591334518 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff944071f90 0x7ff944070360 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ff934004970 tx=0x7ff934009e30 comp rx=0 tx=0).stop 2024-03-29T08:13:33.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:33.386+0000 7ff948c00700 1 -- 172.21.15.37:0/2591334518 shutdown_connections 2024-03-29T08:13:33.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:33.386+0000 7ff948c00700 1 --2- 172.21.15.37:0/2591334518 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff9440708a0 0x7ff944070d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:13:33.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:33.386+0000 7ff948c00700 1 --2- 172.21.15.37:0/2591334518 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff944071f90 0x7ff944070360 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:13:33.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:33.386+0000 7ff948c00700 1 -- 172.21.15.37:0/2591334518 >> 172.21.15.37:0/2591334518 conn(0x7ff9440f89b0 msgr2=0x7ff9440fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:13:33.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:33.387+0000 7ff948c00700 1 -- 172.21.15.37:0/2591334518 shutdown_connections 2024-03-29T08:13:33.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:33.387+0000 7ff948c00700 1 -- 172.21.15.37:0/2591334518 wait complete. 2024-03-29T08:13:33.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:33.387+0000 7ff948c00700 1 Processor -- start 2024-03-29T08:13:33.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:33.388+0000 7ff948c00700 1 -- start start 2024-03-29T08:13:33.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:33.388+0000 7ff948c00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff9440708a0 0x7ff94411a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:13:33.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:33.388+0000 7ff948c00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff944071f90 0x7ff94411a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:13:33.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:33.388+0000 7ff948c00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff94411ac50 con 0x7ff9440708a0 2024-03-29T08:13:33.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:33.388+0000 7ff948c00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff944114020 con 0x7ff944071f90 2024-03-29T08:13:33.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:33.388+0000 7ff941e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff9440708a0 0x7ff94411a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:13:33.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:33.388+0000 7ff941e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff9440708a0 0x7ff94411a0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44096/0 (socket says 172.21.15.37:44096) 2024-03-29T08:13:33.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:33.388+0000 7ff941e00700 1 -- 172.21.15.37:0/738001124 learned_addr learned my addr 172.21.15.37:0/738001124 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:13:33.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:33.388+0000 7ff941400700 1 --2- 172.21.15.37:0/738001124 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff944071f90 0x7ff94411a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:13:33.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:33.389+0000 7ff941e00700 1 -- 172.21.15.37:0/738001124 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff944071f90 msgr2=0x7ff94411a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:13:33.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:33.389+0000 7ff941e00700 1 --2- 172.21.15.37:0/738001124 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff944071f90 0x7ff94411a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:13:33.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:33.389+0000 7ff941e00700 1 -- 172.21.15.37:0/738001124 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff934005040 con 0x7ff9440708a0 2024-03-29T08:13:33.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:33.389+0000 7ff941e00700 1 --2- 172.21.15.37:0/738001124 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff9440708a0 0x7ff94411a0c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff934004f50 tx=0x7ff934009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:13:33.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:33.389+0000 7ff93a000700 1 -- 172.21.15.37:0/738001124 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff93400e040 con 0x7ff9440708a0 2024-03-29T08:13:33.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:33.389+0000 7ff93a000700 1 -- 172.21.15.37:0/738001124 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ff934013420 con 0x7ff9440708a0 2024-03-29T08:13:33.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:33.389+0000 7ff948c00700 1 -- 172.21.15.37:0/738001124 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9441142a0 con 0x7ff9440708a0 2024-03-29T08:13:33.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:33.389+0000 7ff93a000700 1 -- 172.21.15.37:0/738001124 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff93401c630 con 0x7ff9440708a0 2024-03-29T08:13:33.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:33.389+0000 7ff948c00700 1 -- 172.21.15.37:0/738001124 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9441147e0 con 0x7ff9440708a0 2024-03-29T08:13:33.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:33.390+0000 7ff948c00700 1 -- 172.21.15.37:0/738001124 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff944071aa0 con 0x7ff9440708a0 2024-03-29T08:13:33.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:33.391+0000 7ff93a000700 1 -- 172.21.15.37:0/738001124 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 39) v1 ==== 90944+0+0 (secure 0 0 0) 0x7ff934013590 con 0x7ff9440708a0 2024-03-29T08:13:33.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:33.392+0000 7ff93a000700 1 --2- 172.21.15.37:0/738001124 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7ff92806cd40 0x7ff92806f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:13:33.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:33.392+0000 7ff93a000700 1 -- 172.21.15.37:0/738001124 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6324+0+0 (secure 0 0 0) 0x7ff93408d490 con 0x7ff9440708a0 2024-03-29T08:13:33.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:33.392+0000 7ff941400700 1 --2- 172.21.15.37:0/738001124 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7ff92806cd40 0x7ff92806f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:13:33.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:33.393+0000 7ff941400700 1 --2- 172.21.15.37:0/738001124 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7ff92806cd40 0x7ff92806f1f0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7ff944115960 tx=0x7ff93000b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:13:33.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:33.398+0000 7ff93a000700 1 -- 172.21.15.37:0/738001124 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7ff93405ae70 con 0x7ff9440708a0 2024-03-29T08:13:33.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:33.662+0000 7ff948c00700 1 -- 172.21.15.37:0/738001124 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7ff94404bf00 con 0x7ff9440708a0 2024-03-29T08:13:33.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:33.663+0000 7ff93a000700 1 -- 172.21.15.37:0/738001124 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 20 v20) v1 ==== 76+0+1858 (secure 0 0 0) 0x7ff93405aa00 con 0x7ff9440708a0 2024-03-29T08:13:33.664 INFO:teuthology.orchestra.run.smithi037.stdout:e20 2024-03-29T08:13:33.664 INFO:teuthology.orchestra.run.smithi037.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-29T08:13:33.665 INFO:teuthology.orchestra.run.smithi037.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T08:13:33.665 INFO:teuthology.orchestra.run.smithi037.stdout:legacy client fscid: 1 2024-03-29T08:13:33.665 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:13:33.665 INFO:teuthology.orchestra.run.smithi037.stdout:Filesystem 'cephfs' (1) 2024-03-29T08:13:33.665 INFO:teuthology.orchestra.run.smithi037.stdout:fs_name cephfs 2024-03-29T08:13:33.665 INFO:teuthology.orchestra.run.smithi037.stdout:epoch 20 2024-03-29T08:13:33.665 INFO:teuthology.orchestra.run.smithi037.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-03-29T08:13:33.665 INFO:teuthology.orchestra.run.smithi037.stdout:created 2024-03-29T08:03:29.241269+0000 2024-03-29T08:13:33.665 INFO:teuthology.orchestra.run.smithi037.stdout:modified 2024-03-29T08:11:27.181622+0000 2024-03-29T08:13:33.665 INFO:teuthology.orchestra.run.smithi037.stdout:tableserver 0 2024-03-29T08:13:33.665 INFO:teuthology.orchestra.run.smithi037.stdout:root 0 2024-03-29T08:13:33.665 INFO:teuthology.orchestra.run.smithi037.stdout:session_timeout 60 2024-03-29T08:13:33.665 INFO:teuthology.orchestra.run.smithi037.stdout:session_autoclose 300 2024-03-29T08:13:33.666 INFO:teuthology.orchestra.run.smithi037.stdout:max_file_size 1099511627776 2024-03-29T08:13:33.666 INFO:teuthology.orchestra.run.smithi037.stdout:max_xattr_size 65536 2024-03-29T08:13:33.666 INFO:teuthology.orchestra.run.smithi037.stdout:required_client_features {} 2024-03-29T08:13:33.666 INFO:teuthology.orchestra.run.smithi037.stdout:last_failure 0 2024-03-29T08:13:33.666 INFO:teuthology.orchestra.run.smithi037.stdout:last_failure_osd_epoch 47 2024-03-29T08:13:33.666 INFO:teuthology.orchestra.run.smithi037.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T08:13:33.666 INFO:teuthology.orchestra.run.smithi037.stdout:max_mds 2 2024-03-29T08:13:33.666 INFO:teuthology.orchestra.run.smithi037.stdout:in 0,1 2024-03-29T08:13:33.666 INFO:teuthology.orchestra.run.smithi037.stdout:up {0=14470,1=14482} 2024-03-29T08:13:33.666 INFO:teuthology.orchestra.run.smithi037.stdout:failed 2024-03-29T08:13:33.666 INFO:teuthology.orchestra.run.smithi037.stdout:damaged 2024-03-29T08:13:33.666 INFO:teuthology.orchestra.run.smithi037.stdout:stopped 2024-03-29T08:13:33.667 INFO:teuthology.orchestra.run.smithi037.stdout:data_pools [3] 2024-03-29T08:13:33.667 INFO:teuthology.orchestra.run.smithi037.stdout:metadata_pool 2 2024-03-29T08:13:33.667 INFO:teuthology.orchestra.run.smithi037.stdout:inline_data disabled 2024-03-29T08:13:33.667 INFO:teuthology.orchestra.run.smithi037.stdout:balancer 2024-03-29T08:13:33.667 INFO:teuthology.orchestra.run.smithi037.stdout:bal_rank_mask -1 2024-03-29T08:13:33.667 INFO:teuthology.orchestra.run.smithi037.stdout:standby_count_wanted 1 2024-03-29T08:13:33.667 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi037.mmwgxe{0:14470} state up:active seq 5 join_fscid=1 addr [v2:172.21.15.37:6826/1559618909,v1:172.21.15.37:6827/1559618909] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:13:33.667 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi037.iifsmh{1:14482} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.37:6828/2341530495,v1:172.21.15.37:6829/2341530495] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:13:33.667 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:13:33.667 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:13:33.667 INFO:teuthology.orchestra.run.smithi037.stdout:Standby daemons: 2024-03-29T08:13:33.667 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:13:33.667 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi148.qukpjh{-1:24319} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.148:6826/2641327912,v1:172.21.15.148:6827/2641327912] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:13:33.672 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi148.sqteih{-1:24327} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.148:6824/3252351726,v1:172.21.15.148:6825/3252351726] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:13:33.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:33.665+0000 7ff948c00700 1 -- 172.21.15.37:0/738001124 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7ff92806cd40 msgr2=0x7ff92806f1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:13:33.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:33.665+0000 7ff948c00700 1 --2- 172.21.15.37:0/738001124 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7ff92806cd40 0x7ff92806f1f0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7ff944115960 tx=0x7ff93000b040 comp rx=0 tx=0).stop 2024-03-29T08:13:33.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:33.665+0000 7ff948c00700 1 -- 172.21.15.37:0/738001124 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff9440708a0 msgr2=0x7ff94411a0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:13:33.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:33.665+0000 7ff948c00700 1 --2- 172.21.15.37:0/738001124 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff9440708a0 0x7ff94411a0c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff934004f50 tx=0x7ff934009e30 comp rx=0 tx=0).stop 2024-03-29T08:13:33.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:33.665+0000 7ff948c00700 1 -- 172.21.15.37:0/738001124 shutdown_connections 2024-03-29T08:13:33.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:33.665+0000 7ff948c00700 1 --2- 172.21.15.37:0/738001124 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7ff92806cd40 0x7ff92806f1f0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:13:33.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:33.665+0000 7ff948c00700 1 --2- 172.21.15.37:0/738001124 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff9440708a0 0x7ff94411a0c0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:13:33.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:33.665+0000 7ff948c00700 1 --2- 172.21.15.37:0/738001124 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff944071f90 0x7ff94411a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:13:33.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:33.665+0000 7ff948c00700 1 -- 172.21.15.37:0/738001124 >> 172.21.15.37:0/738001124 conn(0x7ff9440f89b0 msgr2=0x7ff9440f9b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:13:33.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:33.666+0000 7ff948c00700 1 -- 172.21.15.37:0/738001124 shutdown_connections 2024-03-29T08:13:33.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:33.666+0000 7ff948c00700 1 -- 172.21.15.37:0/738001124 wait complete. 2024-03-29T08:13:33.673 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 20 2024-03-29T08:13:33.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:33.804+0000 7f8a48400700 1 -- 172.21.15.37:0/2872557281 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8a400fc550 msgr2=0x7f8a401092b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:13:33.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:33.804+0000 7f8a48400700 1 --2- 172.21.15.37:0/2872557281 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8a400fc550 0x7f8a401092b0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f8a2c004940 tx=0x7f8a2c009e30 comp rx=0 tx=0).stop 2024-03-29T08:13:33.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:33.804+0000 7f8a48400700 1 -- 172.21.15.37:0/2872557281 shutdown_connections 2024-03-29T08:13:33.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:33.804+0000 7f8a48400700 1 --2- 172.21.15.37:0/2872557281 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8a400fc550 0x7f8a401092b0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:13:33.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:33.804+0000 7f8a48400700 1 --2- 172.21.15.37:0/2872557281 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8a400fbc40 0x7f8a400fc010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:13:33.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:33.804+0000 7f8a48400700 1 -- 172.21.15.37:0/2872557281 >> 172.21.15.37:0/2872557281 conn(0x7f8a400f78f0 msgr2=0x7f8a400f9d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:13:33.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:33.804+0000 7f8a48400700 1 -- 172.21.15.37:0/2872557281 shutdown_connections 2024-03-29T08:13:33.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:33.804+0000 7f8a48400700 1 -- 172.21.15.37:0/2872557281 wait complete. 2024-03-29T08:13:33.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:33.805+0000 7f8a48400700 1 Processor -- start 2024-03-29T08:13:33.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:33.805+0000 7f8a48400700 1 -- start start 2024-03-29T08:13:33.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:33.806+0000 7f8a48400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8a400fbc40 0x7f8a40115990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:13:33.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:33.806+0000 7f8a48400700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8a400fc550 0x7f8a40115ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:13:33.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:33.806+0000 7f8a48400700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a40116520 con 0x7f8a400fbc40 2024-03-29T08:13:33.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:33.806+0000 7f8a48400700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a4010fb40 con 0x7f8a400fc550 2024-03-29T08:13:33.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:33.806+0000 7f8a45a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8a400fbc40 0x7f8a40115990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:13:33.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:33.806+0000 7f8a45a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8a400fbc40 0x7f8a40115990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44106/0 (socket says 172.21.15.37:44106) 2024-03-29T08:13:33.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:33.806+0000 7f8a45a00700 1 -- 172.21.15.37:0/2840010530 learned_addr learned my addr 172.21.15.37:0/2840010530 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:13:33.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:33.806+0000 7f8a45000700 1 --2- 172.21.15.37:0/2840010530 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8a400fc550 0x7f8a40115ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:13:33.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:33.806+0000 7f8a45a00700 1 -- 172.21.15.37:0/2840010530 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8a400fc550 msgr2=0x7f8a40115ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:13:33.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:33.806+0000 7f8a45a00700 1 --2- 172.21.15.37:0/2840010530 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8a400fc550 0x7f8a40115ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:13:33.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:33.806+0000 7f8a45a00700 1 -- 172.21.15.37:0/2840010530 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a2c005040 con 0x7f8a400fbc40 2024-03-29T08:13:33.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:33.807+0000 7f8a45a00700 1 --2- 172.21.15.37:0/2840010530 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8a400fbc40 0x7f8a40115990 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f8a34002a50 tx=0x7f8a340046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:13:33.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:33.807+0000 7f8a3d600700 1 -- 172.21.15.37:0/2840010530 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8a34007cc0 con 0x7f8a400fbc40 2024-03-29T08:13:33.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:33.807+0000 7f8a3d600700 1 -- 172.21.15.37:0/2840010530 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f8a3400f040 con 0x7f8a400fbc40 2024-03-29T08:13:33.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:33.807+0000 7f8a48400700 1 -- 172.21.15.37:0/2840010530 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a4010fe20 con 0x7f8a400fbc40 2024-03-29T08:13:33.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:33.807+0000 7f8a3d600700 1 -- 172.21.15.37:0/2840010530 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8a34009540 con 0x7f8a400fbc40 2024-03-29T08:13:33.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:33.807+0000 7f8a48400700 1 -- 172.21.15.37:0/2840010530 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a401103e0 con 0x7f8a400fbc40 2024-03-29T08:13:33.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:33.808+0000 7f8a48400700 1 -- 172.21.15.37:0/2840010530 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a40066a90 con 0x7f8a400fbc40 2024-03-29T08:13:33.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:33.809+0000 7f8a3d600700 1 -- 172.21.15.37:0/2840010530 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 39) v1 ==== 90944+0+0 (secure 0 0 0) 0x7f8a340096a0 con 0x7f8a400fbc40 2024-03-29T08:13:33.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:33.811+0000 7f8a3d600700 1 --2- 172.21.15.37:0/2840010530 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f8a2806cd50 0x7f8a2806f200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:13:33.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:33.811+0000 7f8a3d600700 1 -- 172.21.15.37:0/2840010530 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f8a3408c010 con 0x7f8a400fbc40 2024-03-29T08:13:33.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:33.811+0000 7f8a45000700 1 --2- 172.21.15.37:0/2840010530 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f8a2806cd50 0x7f8a2806f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:13:33.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:33.811+0000 7f8a45000700 1 --2- 172.21.15.37:0/2840010530 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f8a2806cd50 0x7f8a2806f200 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f8a40111230 tx=0x7f8a2c009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:13:33.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:33.815+0000 7f8a3d600700 1 -- 172.21.15.37:0/2840010530 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f8a34055e50 con 0x7f8a400fbc40 2024-03-29T08:13:33.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:13:33 smithi037 ceph-mon[142325]: from='client.34188 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:13:33.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:13:33 smithi037 ceph-mon[142325]: from='client.34192 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:13:33.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:13:33 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/3039740188' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:13:34.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:13:33 smithi148 ceph-mon[106692]: from='client.34188 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:13:34.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:13:33 smithi148 ceph-mon[106692]: from='client.34192 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:13:34.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:13:33 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/3039740188' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:13:34.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:34.065+0000 7f8a48400700 1 -- 172.21.15.37:0/2840010530 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f8a40111050 con 0x7f8a2806cd50 2024-03-29T08:13:34.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:34.067+0000 7f8a3d600700 1 -- 172.21.15.37:0/2840010530 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+402 (secure 0 0 0) 0x7f8a40111050 con 0x7f8a2806cd50 2024-03-29T08:13:34.069 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-29T08:13:34.069 INFO:teuthology.orchestra.run.smithi037.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59", 2024-03-29T08:13:34.069 INFO:teuthology.orchestra.run.smithi037.stdout: "in_progress": true, 2024-03-29T08:13:34.069 INFO:teuthology.orchestra.run.smithi037.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-29T08:13:34.069 INFO:teuthology.orchestra.run.smithi037.stdout: "services_complete": [ 2024-03-29T08:13:34.070 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr", 2024-03-29T08:13:34.070 INFO:teuthology.orchestra.run.smithi037.stdout: "mon", 2024-03-29T08:13:34.070 INFO:teuthology.orchestra.run.smithi037.stdout: "crash" 2024-03-29T08:13:34.070 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-29T08:13:34.070 INFO:teuthology.orchestra.run.smithi037.stdout: "progress": "6/21 daemons upgraded", 2024-03-29T08:13:34.070 INFO:teuthology.orchestra.run.smithi037.stdout: "message": "Currently upgrading crash daemons", 2024-03-29T08:13:34.071 INFO:teuthology.orchestra.run.smithi037.stdout: "is_paused": false 2024-03-29T08:13:34.071 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-29T08:13:34.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:34.069+0000 7f8a48400700 1 -- 172.21.15.37:0/2840010530 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f8a2806cd50 msgr2=0x7f8a2806f200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:13:34.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:34.069+0000 7f8a48400700 1 --2- 172.21.15.37:0/2840010530 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f8a2806cd50 0x7f8a2806f200 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f8a40111230 tx=0x7f8a2c009e30 comp rx=0 tx=0).stop 2024-03-29T08:13:34.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:34.069+0000 7f8a48400700 1 -- 172.21.15.37:0/2840010530 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8a400fbc40 msgr2=0x7f8a40115990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:13:34.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:34.069+0000 7f8a48400700 1 --2- 172.21.15.37:0/2840010530 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8a400fbc40 0x7f8a40115990 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f8a34002a50 tx=0x7f8a340046b0 comp rx=0 tx=0).stop 2024-03-29T08:13:34.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:34.069+0000 7f8a48400700 1 -- 172.21.15.37:0/2840010530 shutdown_connections 2024-03-29T08:13:34.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:34.069+0000 7f8a48400700 1 --2- 172.21.15.37:0/2840010530 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f8a2806cd50 0x7f8a2806f200 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:13:34.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:34.069+0000 7f8a48400700 1 --2- 172.21.15.37:0/2840010530 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8a400fbc40 0x7f8a40115990 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:13:34.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:34.069+0000 7f8a48400700 1 --2- 172.21.15.37:0/2840010530 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8a400fc550 0x7f8a40115ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:13:34.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:34.069+0000 7f8a48400700 1 -- 172.21.15.37:0/2840010530 >> 172.21.15.37:0/2840010530 conn(0x7f8a400f78f0 msgr2=0x7f8a40105b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:13:34.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:34.070+0000 7f8a48400700 1 -- 172.21.15.37:0/2840010530 shutdown_connections 2024-03-29T08:13:34.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:34.070+0000 7f8a48400700 1 -- 172.21.15.37:0/2840010530 wait complete. 2024-03-29T08:13:34.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:34.209+0000 7fc671400700 1 -- 172.21.15.37:0/4080636151 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc66c070bf0 msgr2=0x7fc66c108fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:13:34.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:34.209+0000 7fc671400700 1 --2- 172.21.15.37:0/4080636151 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc66c070bf0 0x7fc66c108fc0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fc658004940 tx=0x7fc658009e30 comp rx=0 tx=0).stop 2024-03-29T08:13:34.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:34.209+0000 7fc671400700 1 -- 172.21.15.37:0/4080636151 shutdown_connections 2024-03-29T08:13:34.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:34.209+0000 7fc671400700 1 --2- 172.21.15.37:0/4080636151 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc66c070bf0 0x7fc66c108fc0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:13:34.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:34.209+0000 7fc671400700 1 --2- 172.21.15.37:0/4080636151 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc66c0702e0 0x7fc66c0706b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:13:34.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:34.209+0000 7fc671400700 1 -- 172.21.15.37:0/4080636151 >> 172.21.15.37:0/4080636151 conn(0x7fc66c0f8760 msgr2=0x7fc66c0fab70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:13:34.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:34.209+0000 7fc671400700 1 -- 172.21.15.37:0/4080636151 shutdown_connections 2024-03-29T08:13:34.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:34.209+0000 7fc671400700 1 -- 172.21.15.37:0/4080636151 wait complete. 2024-03-29T08:13:34.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:34.210+0000 7fc671400700 1 Processor -- start 2024-03-29T08:13:34.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:34.210+0000 7fc671400700 1 -- start start 2024-03-29T08:13:34.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:34.211+0000 7fc671400700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc66c0702e0 0x7fc66c115ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:13:34.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:34.211+0000 7fc671400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc66c070bf0 0x7fc66c115ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:13:34.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:34.211+0000 7fc671400700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc66c116640 con 0x7fc66c070bf0 2024-03-29T08:13:34.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:34.211+0000 7fc671400700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc66c10fa30 con 0x7fc66c0702e0 2024-03-29T08:13:34.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:34.211+0000 7fc66b400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc66c070bf0 0x7fc66c115ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:13:34.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:34.211+0000 7fc66b400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc66c070bf0 0x7fc66c115ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44126/0 (socket says 172.21.15.37:44126) 2024-03-29T08:13:34.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:34.211+0000 7fc66b400700 1 -- 172.21.15.37:0/2942566484 learned_addr learned my addr 172.21.15.37:0/2942566484 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:13:34.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:34.211+0000 7fc66be00700 1 --2- 172.21.15.37:0/2942566484 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc66c0702e0 0x7fc66c115ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:13:34.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:34.211+0000 7fc66b400700 1 -- 172.21.15.37:0/2942566484 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc66c0702e0 msgr2=0x7fc66c115ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:13:34.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:34.211+0000 7fc66b400700 1 --2- 172.21.15.37:0/2942566484 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc66c0702e0 0x7fc66c115ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:13:34.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:34.211+0000 7fc66b400700 1 -- 172.21.15.37:0/2942566484 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc658005040 con 0x7fc66c070bf0 2024-03-29T08:13:34.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:34.212+0000 7fc66b400700 1 --2- 172.21.15.37:0/2942566484 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc66c070bf0 0x7fc66c115ff0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fc658008d80 tx=0x7fc658009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:13:34.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:34.212+0000 7fc663e00700 1 -- 172.21.15.37:0/2942566484 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc65800e030 con 0x7fc66c070bf0 2024-03-29T08:13:34.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:34.212+0000 7fc663e00700 1 -- 172.21.15.37:0/2942566484 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fc658013de0 con 0x7fc66c070bf0 2024-03-29T08:13:34.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:34.212+0000 7fc671400700 1 -- 172.21.15.37:0/2942566484 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc66c10fcb0 con 0x7fc66c070bf0 2024-03-29T08:13:34.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:34.212+0000 7fc663e00700 1 -- 172.21.15.37:0/2942566484 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc65801cd00 con 0x7fc66c070bf0 2024-03-29T08:13:34.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:34.212+0000 7fc671400700 1 -- 172.21.15.37:0/2942566484 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc66c1101f0 con 0x7fc66c070bf0 2024-03-29T08:13:34.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:34.213+0000 7fc671400700 1 -- 172.21.15.37:0/2942566484 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc66c071a20 con 0x7fc66c070bf0 2024-03-29T08:13:34.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:34.214+0000 7fc663e00700 1 -- 172.21.15.37:0/2942566484 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 39) v1 ==== 90944+0+0 (secure 0 0 0) 0x7fc658013930 con 0x7fc66c070bf0 2024-03-29T08:13:34.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:34.215+0000 7fc663e00700 1 --2- 172.21.15.37:0/2942566484 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fc65006cd50 0x7fc65006f200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:13:34.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:34.215+0000 7fc663e00700 1 -- 172.21.15.37:0/2942566484 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fc65808da20 con 0x7fc66c070bf0 2024-03-29T08:13:34.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:34.215+0000 7fc66be00700 1 --2- 172.21.15.37:0/2942566484 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fc65006cd50 0x7fc65006f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:13:34.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:34.216+0000 7fc66be00700 1 --2- 172.21.15.37:0/2942566484 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fc65006cd50 0x7fc65006f200 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fc65c009b80 tx=0x7fc65c00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:13:34.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:34.221+0000 7fc663e00700 1 -- 172.21.15.37:0/2942566484 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fc65805b350 con 0x7fc66c070bf0 2024-03-29T08:13:34.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:34.529+0000 7fc671400700 1 -- 172.21.15.37:0/2942566484 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fc66c0fc3c0 con 0x7fc66c070bf0 2024-03-29T08:13:34.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:34.529+0000 7fc663e00700 1 -- 172.21.15.37:0/2942566484 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7fc65801ce60 con 0x7fc66c070bf0 2024-03-29T08:13:34.530 INFO:teuthology.orchestra.run.smithi037.stdout:HEALTH_OK 2024-03-29T08:13:34.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:34.532+0000 7fc671400700 1 -- 172.21.15.37:0/2942566484 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fc65006cd50 msgr2=0x7fc65006f200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:13:34.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:34.532+0000 7fc671400700 1 --2- 172.21.15.37:0/2942566484 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fc65006cd50 0x7fc65006f200 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fc65c009b80 tx=0x7fc65c00b040 comp rx=0 tx=0).stop 2024-03-29T08:13:34.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:34.532+0000 7fc671400700 1 -- 172.21.15.37:0/2942566484 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc66c070bf0 msgr2=0x7fc66c115ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:13:34.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:34.532+0000 7fc671400700 1 --2- 172.21.15.37:0/2942566484 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc66c070bf0 0x7fc66c115ff0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fc658008d80 tx=0x7fc658009e30 comp rx=0 tx=0).stop 2024-03-29T08:13:34.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:34.532+0000 7fc671400700 1 -- 172.21.15.37:0/2942566484 shutdown_connections 2024-03-29T08:13:34.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:34.532+0000 7fc671400700 1 --2- 172.21.15.37:0/2942566484 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fc65006cd50 0x7fc65006f200 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:13:34.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:34.532+0000 7fc671400700 1 --2- 172.21.15.37:0/2942566484 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc66c0702e0 0x7fc66c115ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:13:34.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:34.532+0000 7fc671400700 1 --2- 172.21.15.37:0/2942566484 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc66c070bf0 0x7fc66c115ff0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:13:34.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:34.532+0000 7fc671400700 1 -- 172.21.15.37:0/2942566484 >> 172.21.15.37:0/2942566484 conn(0x7fc66c0f8760 msgr2=0x7fc66c0f9c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:13:34.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:34.532+0000 7fc671400700 1 -- 172.21.15.37:0/2942566484 shutdown_connections 2024-03-29T08:13:34.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:13:34.532+0000 7fc671400700 1 -- 172.21.15.37:0/2942566484 wait complete. 2024-03-29T08:13:34.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:13:34 smithi037 ceph-mon[142325]: from='client.34196 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:13:34.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:13:34 smithi037 ceph-mon[142325]: pgmap v45: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:13:34.928 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:13:34 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/738001124' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T08:13:34.928 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:13:34 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:13:34.928 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:13:34 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:13:34.928 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:13:34 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T08:13:34.928 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:13:34 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T08:13:34.928 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:13:34 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:13:34.928 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:13:34 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T08:13:34.928 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:13:34 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:13:34.928 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:13:34 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:13:34.928 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:13:34 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:13:34.928 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:13:34 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:13:34.928 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:13:34 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi037"}]: dispatch 2024-03-29T08:13:34.928 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:13:34 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi037"}]': finished 2024-03-29T08:13:34.928 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:13:34 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi148"}]: dispatch 2024-03-29T08:13:34.929 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:13:34 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi148"}]': finished 2024-03-29T08:13:34.929 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:13:34 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-03-29T08:13:34.929 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:13:34 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/2942566484' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T08:13:35.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:13:34 smithi148 ceph-mon[106692]: from='client.34196 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:13:35.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:13:34 smithi148 ceph-mon[106692]: pgmap v45: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:13:35.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:13:34 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/738001124' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T08:13:35.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:13:34 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:13:35.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:13:34 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:13:35.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:13:34 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T08:13:35.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:13:34 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T08:13:35.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:13:34 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:13:35.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:13:34 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T08:13:35.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:13:34 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:13:35.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:13:34 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:13:35.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:13:34 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:13:35.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:13:34 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:13:35.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:13:34 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi037"}]: dispatch 2024-03-29T08:13:35.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:13:34 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi037"}]': finished 2024-03-29T08:13:35.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:13:34 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi148"}]: dispatch 2024-03-29T08:13:35.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:13:34 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi148"}]': finished 2024-03-29T08:13:35.058 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:13:34 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-03-29T08:13:35.058 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:13:34 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/2942566484' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T08:13:35.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:13:35 smithi037 ceph-mon[142325]: from='client.34208 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:13:35.928 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:13:35 smithi037 ceph-mon[142325]: Upgrade: Setting container_image for all crash 2024-03-29T08:13:35.928 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:13:35 smithi037 ceph-mon[142325]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-03-29T08:13:35.928 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:13:35 smithi037 ceph-mon[142325]: Upgrade: osd.0 is safe to restart 2024-03-29T08:13:36.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:13:35 smithi148 ceph-mon[106692]: from='client.34208 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:13:36.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:13:35 smithi148 ceph-mon[106692]: Upgrade: Setting container_image for all crash 2024-03-29T08:13:36.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:13:35 smithi148 ceph-mon[106692]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-03-29T08:13:36.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:13:35 smithi148 ceph-mon[106692]: Upgrade: osd.0 is safe to restart 2024-03-29T08:13:36.870 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:13:36 smithi037 ceph-mon[142325]: pgmap v46: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:13:36.870 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:13:36 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:13:36.870 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:13:36 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-29T08:13:36.871 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:13:36 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T08:13:37.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:13:36 smithi148 ceph-mon[106692]: pgmap v46: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:13:37.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:13:36 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:13:37.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:13:36 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-29T08:13:37.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:13:36 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T08:13:37.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:13:37 smithi037 ceph-mon[142325]: Upgrade: Updating osd.0 2024-03-29T08:13:37.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:13:37 smithi037 ceph-mon[142325]: Deploying daemon osd.0 on smithi037 2024-03-29T08:13:38.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:13:37 smithi148 ceph-mon[106692]: Upgrade: Updating osd.0 2024-03-29T08:13:38.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:13:37 smithi148 ceph-mon[106692]: Deploying daemon osd.0 on smithi037 2024-03-29T08:13:38.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:13:38 smithi037 ceph-mon[142325]: pgmap v47: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:13:39.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:13:38 smithi148 ceph-mon[106692]: pgmap v47: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:13:40.172 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:13:40 smithi037 ceph-mon[142325]: pgmap v48: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:13:40.172 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:13:40 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:13:40.484 INFO:journalctl@ceph.osd.0.smithi037.stdout:Mar 29 08:13:40 smithi037 systemd[1]: Stopping Ceph osd.0 for 99b492c2-eda1-11ee-b647-cb9ed24678a4... 2024-03-29T08:13:40.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:13:40 smithi148 ceph-mon[106692]: pgmap v48: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:13:40.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:13:40 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:13:40.927 INFO:journalctl@ceph.osd.0.smithi037.stdout:Mar 29 08:13:40 smithi037 ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4-osd-0[58581]: 2024-03-29T08:13:40.483+0000 7fcc6bc00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-29T08:13:40.927 INFO:journalctl@ceph.osd.0.smithi037.stdout:Mar 29 08:13:40 smithi037 ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4-osd-0[58581]: 2024-03-29T08:13:40.483+0000 7fcc6bc00700 -1 osd.0 50 *** Got signal Terminated *** 2024-03-29T08:13:40.927 INFO:journalctl@ceph.osd.0.smithi037.stdout:Mar 29 08:13:40 smithi037 ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4-osd-0[58581]: 2024-03-29T08:13:40.483+0000 7fcc6bc00700 -1 osd.0 50 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-29T08:13:41.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:13:41 smithi037 ceph-mon[142325]: osd.0 marked itself down and dead 2024-03-29T08:13:41.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:13:41 smithi148 ceph-mon[106692]: osd.0 marked itself down and dead 2024-03-29T08:13:42.427 INFO:journalctl@ceph.osd.0.smithi037.stdout:Mar 29 08:13:42 smithi037 bash[156681]: ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4-osd-0 2024-03-29T08:13:42.428 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:13:42 smithi037 ceph-mon[142325]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-29T08:13:42.428 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:13:42 smithi037 ceph-mon[142325]: osdmap e51: 6 total, 5 up, 6 in 2024-03-29T08:13:42.428 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:13:42 smithi037 ceph-mon[142325]: pgmap v50: 65 pgs: 9 stale+active+clean, 56 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-03-29T08:13:42.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:13:42 smithi148 ceph-mon[106692]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-29T08:13:42.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:13:42 smithi148 ceph-mon[106692]: osdmap e51: 6 total, 5 up, 6 in 2024-03-29T08:13:42.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:13:42 smithi148 ceph-mon[106692]: pgmap v50: 65 pgs: 9 stale+active+clean, 56 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-03-29T08:13:43.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:13:43 smithi037 ceph-mon[142325]: mgrmap e40: smithi037.wjikii(active, since 92s), standbys: smithi148.onhgvm 2024-03-29T08:13:43.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:13:43 smithi037 ceph-mon[142325]: osdmap e52: 6 total, 5 up, 6 in 2024-03-29T08:13:43.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:13:43 smithi148 ceph-mon[106692]: mgrmap e40: smithi037.wjikii(active, since 92s), standbys: smithi148.onhgvm 2024-03-29T08:13:43.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:13:43 smithi148 ceph-mon[106692]: osdmap e52: 6 total, 5 up, 6 in 2024-03-29T08:13:44.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:13:44 smithi037 ceph-mon[142325]: pgmap v52: 65 pgs: 9 stale+active+clean, 56 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 639 B/s wr, 0 op/s 2024-03-29T08:13:44.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:13:44 smithi148 ceph-mon[106692]: pgmap v52: 65 pgs: 9 stale+active+clean, 56 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 639 B/s wr, 0 op/s 2024-03-29T08:13:44.575 INFO:journalctl@ceph.osd.0.smithi037.stdout:Mar 29 08:13:44 smithi037 systemd[1]: ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4@osd.0.service: Succeeded. 2024-03-29T08:13:44.575 INFO:journalctl@ceph.osd.0.smithi037.stdout:Mar 29 08:13:44 smithi037 systemd[1]: Stopped Ceph osd.0 for 99b492c2-eda1-11ee-b647-cb9ed24678a4. 2024-03-29T08:13:44.927 INFO:journalctl@ceph.osd.0.smithi037.stdout:Mar 29 08:13:44 smithi037 systemd[1]: Starting Ceph osd.0 for 99b492c2-eda1-11ee-b647-cb9ed24678a4... 2024-03-29T08:13:45.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:13:45 smithi037 ceph-mon[142325]: pgmap v53: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 639 B/s wr, 0 op/s; 64/324 objects degraded (19.753%) 2024-03-29T08:13:45.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:13:45 smithi037 ceph-mon[142325]: Health check failed: Degraded data redundancy: 64/324 objects degraded (19.753%), 19 pgs degraded (PG_DEGRADED) 2024-03-29T08:13:46.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:13:45 smithi148 ceph-mon[106692]: pgmap v53: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 639 B/s wr, 0 op/s; 64/324 objects degraded (19.753%) 2024-03-29T08:13:46.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:13:45 smithi148 ceph-mon[106692]: Health check failed: Degraded data redundancy: 64/324 objects degraded (19.753%), 19 pgs degraded (PG_DEGRADED) 2024-03-29T08:13:46.341 INFO:journalctl@ceph.osd.0.smithi037.stdout:Mar 29 08:13:45 smithi037 bash[156975]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-03-29T08:13:46.341 INFO:journalctl@ceph.osd.0.smithi037.stdout:Mar 29 08:13:45 smithi037 bash[156975]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-0 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-03-29T08:13:46.341 INFO:journalctl@ceph.osd.0.smithi037.stdout:Mar 29 08:13:45 smithi037 bash[156975]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-03-29T08:13:46.341 INFO:journalctl@ceph.osd.0.smithi037.stdout:Mar 29 08:13:45 smithi037 bash[156975]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-03-29T08:13:46.341 INFO:journalctl@ceph.osd.0.smithi037.stdout:Mar 29 08:13:45 smithi037 bash[156975]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-0/block 2024-03-29T08:13:46.341 INFO:journalctl@ceph.osd.0.smithi037.stdout:Mar 29 08:13:45 smithi037 bash[156975]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-03-29T08:13:46.341 INFO:journalctl@ceph.osd.0.smithi037.stdout:Mar 29 08:13:45 smithi037 bash[156975]: --> ceph-volume raw activate successful for osd ID: 0 2024-03-29T08:13:47.677 INFO:journalctl@ceph.osd.0.smithi037.stdout:Mar 29 08:13:47 smithi037 ceph-osd[157292]: -- 172.21.15.37:0/1521696061 <== mon.0 v2:172.21.15.37:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55a333c871e0 con 0x55a333c57000 2024-03-29T08:13:47.677 INFO:journalctl@ceph.osd.0.smithi037.stdout:Mar 29 08:13:47 smithi037 bash[157255]: c0c584f6245ca3399cf27dbf9d8d23e4b60aed4f7953f2e27e5a285d387b400c 2024-03-29T08:13:47.677 INFO:journalctl@ceph.osd.0.smithi037.stdout:Mar 29 08:13:47 smithi037 systemd[1]: Started Ceph osd.0 for 99b492c2-eda1-11ee-b647-cb9ed24678a4. 2024-03-29T08:13:48.177 INFO:journalctl@ceph.osd.0.smithi037.stdout:Mar 29 08:13:47 smithi037 ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4-osd-0[157269]: 2024-03-29T08:13:47.777+0000 7feba35a7640 -1 Falling back to public interface 2024-03-29T08:13:48.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:13:48 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:13:48.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:13:48 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:13:48.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:13:48 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T08:13:48.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:13:48 smithi037 ceph-mon[142325]: pgmap v54: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 639 B/s wr, 0 op/s; 64/324 objects degraded (19.753%) 2024-03-29T08:13:48.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:13:48 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:13:48.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:13:48 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:13:48.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:13:48 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T08:13:48.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:13:48 smithi148 ceph-mon[106692]: pgmap v54: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 639 B/s wr, 0 op/s; 64/324 objects degraded (19.753%) 2024-03-29T08:13:50.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:13:50 smithi037 ceph-mon[142325]: pgmap v55: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 64/324 objects degraded (19.753%) 2024-03-29T08:13:50.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:13:50 smithi148 ceph-mon[106692]: pgmap v55: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 64/324 objects degraded (19.753%) 2024-03-29T08:13:52.838 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:13:52 smithi037 ceph-mon[142325]: pgmap v56: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 64/324 objects degraded (19.753%) 2024-03-29T08:13:53.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:13:52 smithi148 ceph-mon[106692]: pgmap v56: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 64/324 objects degraded (19.753%) 2024-03-29T08:13:53.150 INFO:journalctl@ceph.osd.0.smithi037.stdout:Mar 29 08:13:52 smithi037 ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4-osd-0[157269]: 2024-03-29T08:13:52.836+0000 7feba35a7640 -1 osd.0 50 log_to_monitors true 2024-03-29T08:13:53.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:13:53 smithi037 ceph-mon[142325]: from='osd.0 [v2:172.21.15.37:6802/2026381241,v1:172.21.15.37:6803/2026381241]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-29T08:13:54.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:13:53 smithi148 ceph-mon[106692]: from='osd.0 [v2:172.21.15.37:6802/2026381241,v1:172.21.15.37:6803/2026381241]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-29T08:13:54.927 INFO:journalctl@ceph.osd.0.smithi037.stdout:Mar 29 08:13:54 smithi037 ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4-osd-0[157269]: 2024-03-29T08:13:54.673+0000 7feb8da00700 -1 osd.0 50 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-29T08:13:54.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:13:54 smithi037 ceph-mon[142325]: pgmap v57: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 64/324 objects degraded (19.753%) 2024-03-29T08:13:54.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:13:54 smithi037 ceph-mon[142325]: from='osd.0 [v2:172.21.15.37:6802/2026381241,v1:172.21.15.37:6803/2026381241]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-29T08:13:54.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:13:54 smithi037 ceph-mon[142325]: osdmap e53: 6 total, 5 up, 6 in 2024-03-29T08:13:54.928 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:13:54 smithi037 ceph-mon[142325]: from='osd.0 [v2:172.21.15.37:6802/2026381241,v1:172.21.15.37:6803/2026381241]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]: dispatch 2024-03-29T08:13:54.928 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:13:54 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:13:54.928 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:13:54 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:13:54.928 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:13:54 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:13:54.928 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:13:54 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:13:55.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:13:54 smithi148 ceph-mon[106692]: pgmap v57: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 64/324 objects degraded (19.753%) 2024-03-29T08:13:55.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:13:54 smithi148 ceph-mon[106692]: from='osd.0 [v2:172.21.15.37:6802/2026381241,v1:172.21.15.37:6803/2026381241]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-29T08:13:55.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:13:54 smithi148 ceph-mon[106692]: osdmap e53: 6 total, 5 up, 6 in 2024-03-29T08:13:55.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:13:54 smithi148 ceph-mon[106692]: from='osd.0 [v2:172.21.15.37:6802/2026381241,v1:172.21.15.37:6803/2026381241]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]: dispatch 2024-03-29T08:13:55.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:13:54 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:13:55.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:13:54 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:13:55.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:13:54 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:13:55.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:13:54 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:13:55.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:13:55 smithi037 ceph-mon[142325]: from='osd.0 [v2:172.21.15.37:6802/2026381241,v1:172.21.15.37:6803/2026381241]' entity='osd.0' 2024-03-29T08:13:55.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:13:55 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:13:55.928 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:13:55 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:13:55.928 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:13:55 smithi037 ceph-mon[142325]: pgmap v59: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 259 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 64/324 objects degraded (19.753%) 2024-03-29T08:13:56.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:13:55 smithi148 ceph-mon[106692]: from='osd.0 [v2:172.21.15.37:6802/2026381241,v1:172.21.15.37:6803/2026381241]' entity='osd.0' 2024-03-29T08:13:56.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:13:55 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:13:56.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:13:55 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:13:56.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:13:55 smithi148 ceph-mon[106692]: pgmap v59: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 259 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 64/324 objects degraded (19.753%) 2024-03-29T08:13:56.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:13:56 smithi037 ceph-mon[142325]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-29T08:13:56.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:13:56 smithi037 ceph-mon[142325]: osd.0 [v2:172.21.15.37:6802/2026381241,v1:172.21.15.37:6803/2026381241] boot 2024-03-29T08:13:56.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:13:56 smithi037 ceph-mon[142325]: osdmap e54: 6 total, 6 up, 6 in 2024-03-29T08:13:56.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:13:56 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-29T08:13:56.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:13:56 smithi037 ceph-mon[142325]: osdmap e55: 6 total, 6 up, 6 in 2024-03-29T08:13:57.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:13:56 smithi148 ceph-mon[106692]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-29T08:13:57.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:13:56 smithi148 ceph-mon[106692]: osd.0 [v2:172.21.15.37:6802/2026381241,v1:172.21.15.37:6803/2026381241] boot 2024-03-29T08:13:57.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:13:56 smithi148 ceph-mon[106692]: osdmap e54: 6 total, 6 up, 6 in 2024-03-29T08:13:57.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:13:56 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-29T08:13:57.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:13:56 smithi148 ceph-mon[106692]: osdmap e55: 6 total, 6 up, 6 in 2024-03-29T08:13:58.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:13:57 smithi148 ceph-mon[106692]: pgmap v62: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 259 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 64/324 objects degraded (19.753%) 2024-03-29T08:13:58.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:13:57 smithi037 ceph-mon[142325]: pgmap v62: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 259 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 64/324 objects degraded (19.753%) 2024-03-29T08:14:00.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:14:00 smithi037 ceph-mon[142325]: pgmap v63: 65 pgs: 16 active+undersized+degraded, 9 active+undersized, 40 active+clean; 259 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 53/324 objects degraded (16.358%) 2024-03-29T08:14:00.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:14:00 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:14:00.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:14:00 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:14:00.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:14:00 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T08:14:00.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:14:00 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T08:14:00.428 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:14:00 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:14:00.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:14:00 smithi148 ceph-mon[106692]: pgmap v63: 65 pgs: 16 active+undersized+degraded, 9 active+undersized, 40 active+clean; 259 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 53/324 objects degraded (16.358%) 2024-03-29T08:14:00.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:14:00 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:14:00.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:14:00 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:14:00.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:14:00 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T08:14:00.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:14:00 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T08:14:00.557 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:14:00 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:14:01.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:14:01 smithi037 ceph-mon[142325]: Health check update: Degraded data redundancy: 53/324 objects degraded (16.358%), 16 pgs degraded (PG_DEGRADED) 2024-03-29T08:14:01.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:14:01 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T08:14:01.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:14:01 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:14:01.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:14:01 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:14:01.428 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:14:01 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:14:01.428 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:14:01 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-03-29T08:14:01.428 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:14:01 smithi037 ceph-mon[142325]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-03-29T08:14:01.428 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:14:01 smithi037 ceph-mon[142325]: Upgrade: unsafe to stop osd(s) at this time (13 PGs are or would become offline) 2024-03-29T08:14:01.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:14:01 smithi148 ceph-mon[106692]: Health check update: Degraded data redundancy: 53/324 objects degraded (16.358%), 16 pgs degraded (PG_DEGRADED) 2024-03-29T08:14:01.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:14:01 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T08:14:01.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:14:01 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:14:01.557 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:14:01 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:14:01.557 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:14:01 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:14:01.557 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:14:01 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-03-29T08:14:01.557 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:14:01 smithi148 ceph-mon[106692]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-03-29T08:14:01.557 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:14:01 smithi148 ceph-mon[106692]: Upgrade: unsafe to stop osd(s) at this time (13 PGs are or would become offline) 2024-03-29T08:14:02.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:14:02 smithi037 ceph-mon[142325]: pgmap v64: 65 pgs: 65 active+clean; 259 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:14:02.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:14:02 smithi148 ceph-mon[106692]: pgmap v64: 65 pgs: 65 active+clean; 259 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:14:03.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:14:03 smithi037 ceph-mon[142325]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 53/324 objects degraded (16.358%), 16 pgs degraded) 2024-03-29T08:14:03.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:14:03 smithi037 ceph-mon[142325]: Cluster is now healthy 2024-03-29T08:14:03.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:14:03 smithi148 ceph-mon[106692]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 53/324 objects degraded (16.358%), 16 pgs degraded) 2024-03-29T08:14:03.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:14:03 smithi148 ceph-mon[106692]: Cluster is now healthy 2024-03-29T08:14:04.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:14:04 smithi037 ceph-mon[142325]: pgmap v65: 65 pgs: 65 active+clean; 259 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:14:04.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:14:04 smithi148 ceph-mon[106692]: pgmap v65: 65 pgs: 65 active+clean; 259 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:14:04.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:04.684+0000 7f8bee000700 1 -- 172.21.15.37:0/295095302 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8be8102e00 msgr2=0x7f8be81031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:14:04.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:04.684+0000 7f8bee000700 1 --2- 172.21.15.37:0/295095302 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8be8102e00 0x7f8be81031d0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f8bd8004940 tx=0x7f8bd8009e30 comp rx=0 tx=0).stop 2024-03-29T08:14:04.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:04.684+0000 7f8bee000700 1 -- 172.21.15.37:0/295095302 shutdown_connections 2024-03-29T08:14:04.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:04.684+0000 7f8bee000700 1 --2- 172.21.15.37:0/295095302 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8be81037a0 0x7f8be80fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:14:04.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:04.684+0000 7f8bee000700 1 --2- 172.21.15.37:0/295095302 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8be8102e00 0x7f8be81031d0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:14:04.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:04.684+0000 7f8bee000700 1 -- 172.21.15.37:0/295095302 >> 172.21.15.37:0/295095302 conn(0x7f8be80f89b0 msgr2=0x7f8be80fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:14:04.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:04.684+0000 7f8bee000700 1 -- 172.21.15.37:0/295095302 shutdown_connections 2024-03-29T08:14:04.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:04.684+0000 7f8bee000700 1 -- 172.21.15.37:0/295095302 wait complete. 2024-03-29T08:14:04.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:04.685+0000 7f8bee000700 1 Processor -- start 2024-03-29T08:14:04.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:04.685+0000 7f8bee000700 1 -- start start 2024-03-29T08:14:04.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:04.686+0000 7f8bee000700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8be8102e00 0x7f8be810e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:14:04.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:04.686+0000 7f8bee000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8be81037a0 0x7f8be810b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:14:04.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:04.686+0000 7f8bee000700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8be810ef60 con 0x7f8be81037a0 2024-03-29T08:14:04.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:04.686+0000 7f8bee000700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8be810f0a0 con 0x7f8be8102e00 2024-03-29T08:14:04.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:04.686+0000 7f8be6a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8be81037a0 0x7f8be810b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:14:04.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:04.686+0000 7f8be6a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8be81037a0 0x7f8be810b900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53400/0 (socket says 172.21.15.37:53400) 2024-03-29T08:14:04.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:04.686+0000 7f8be6a00700 1 -- 172.21.15.37:0/4129785958 learned_addr learned my addr 172.21.15.37:0/4129785958 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:14:04.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:04.686+0000 7f8be7400700 1 --2- 172.21.15.37:0/4129785958 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8be8102e00 0x7f8be810e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:14:04.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:04.687+0000 7f8be6a00700 1 -- 172.21.15.37:0/4129785958 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8be8102e00 msgr2=0x7f8be810e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:14:04.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:04.687+0000 7f8be6a00700 1 --2- 172.21.15.37:0/4129785958 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8be8102e00 0x7f8be810e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:14:04.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:04.687+0000 7f8be6a00700 1 -- 172.21.15.37:0/4129785958 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bd8005040 con 0x7f8be81037a0 2024-03-29T08:14:04.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:04.687+0000 7f8be6a00700 1 --2- 172.21.15.37:0/4129785958 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8be81037a0 0x7f8be810b900 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f8bd4002a50 tx=0x7f8bd4004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:14:04.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:04.687+0000 7f8bdf400700 1 -- 172.21.15.37:0/4129785958 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8bd4007cc0 con 0x7f8be81037a0 2024-03-29T08:14:04.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:04.687+0000 7f8bdf400700 1 -- 172.21.15.37:0/4129785958 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f8bd400f040 con 0x7f8be81037a0 2024-03-29T08:14:04.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:04.688+0000 7f8bdf400700 1 -- 172.21.15.37:0/4129785958 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8bd400d920 con 0x7f8be81037a0 2024-03-29T08:14:04.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:04.688+0000 7f8bee000700 1 -- 172.21.15.37:0/4129785958 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8be810bf00 con 0x7f8be81037a0 2024-03-29T08:14:04.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:04.688+0000 7f8bee000700 1 -- 172.21.15.37:0/4129785958 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8be810c4c0 con 0x7f8be81037a0 2024-03-29T08:14:04.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:04.689+0000 7f8bee000700 1 -- 172.21.15.37:0/4129785958 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8be80fe550 con 0x7f8be81037a0 2024-03-29T08:14:04.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:04.690+0000 7f8bdf400700 1 -- 172.21.15.37:0/4129785958 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f8bd4009680 con 0x7f8be81037a0 2024-03-29T08:14:04.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:04.691+0000 7f8bdf400700 1 --2- 172.21.15.37:0/4129785958 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f8bcc06cc30 0x7f8bcc06f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:14:04.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:04.691+0000 7f8bdf400700 1 -- 172.21.15.37:0/4129785958 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f8bd408bbf0 con 0x7f8be81037a0 2024-03-29T08:14:04.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:04.691+0000 7f8be7400700 1 --2- 172.21.15.37:0/4129785958 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f8bcc06cc30 0x7f8bcc06f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:14:04.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:04.692+0000 7f8be7400700 1 --2- 172.21.15.37:0/4129785958 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f8bcc06cc30 0x7f8bcc06f0e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f8bd800b840 tx=0x7f8bd8009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:14:04.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:04.697+0000 7f8bdf400700 1 -- 172.21.15.37:0/4129785958 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f8bd4059760 con 0x7f8be81037a0 2024-03-29T08:14:04.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:04.930+0000 7f8bee000700 1 -- 172.21.15.37:0/4129785958 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f8be8109dc0 con 0x7f8bcc06cc30 2024-03-29T08:14:04.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:04.932+0000 7f8bdf400700 1 -- 172.21.15.37:0/4129785958 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f8be8109dc0 con 0x7f8bcc06cc30 2024-03-29T08:14:04.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:04.934+0000 7f8bee000700 1 -- 172.21.15.37:0/4129785958 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f8bcc06cc30 msgr2=0x7f8bcc06f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:14:04.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:04.934+0000 7f8bee000700 1 --2- 172.21.15.37:0/4129785958 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f8bcc06cc30 0x7f8bcc06f0e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f8bd800b840 tx=0x7f8bd8009e30 comp rx=0 tx=0).stop 2024-03-29T08:14:04.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:04.934+0000 7f8bee000700 1 -- 172.21.15.37:0/4129785958 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8be81037a0 msgr2=0x7f8be810b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:14:04.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:04.934+0000 7f8bee000700 1 --2- 172.21.15.37:0/4129785958 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8be81037a0 0x7f8be810b900 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f8bd4002a50 tx=0x7f8bd4004600 comp rx=0 tx=0).stop 2024-03-29T08:14:04.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:04.934+0000 7f8bee000700 1 -- 172.21.15.37:0/4129785958 shutdown_connections 2024-03-29T08:14:04.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:04.934+0000 7f8bee000700 1 --2- 172.21.15.37:0/4129785958 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f8bcc06cc30 0x7f8bcc06f0e0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:14:04.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:04.934+0000 7f8bee000700 1 --2- 172.21.15.37:0/4129785958 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8be8102e00 0x7f8be810e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:14:04.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:04.934+0000 7f8bee000700 1 --2- 172.21.15.37:0/4129785958 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8be81037a0 0x7f8be810b900 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:14:04.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:04.934+0000 7f8bee000700 1 -- 172.21.15.37:0/4129785958 >> 172.21.15.37:0/4129785958 conn(0x7f8be80f89b0 msgr2=0x7f8be80f9980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:14:04.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:04.934+0000 7f8bee000700 1 -- 172.21.15.37:0/4129785958 shutdown_connections 2024-03-29T08:14:04.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:04.934+0000 7f8bee000700 1 -- 172.21.15.37:0/4129785958 wait complete. 2024-03-29T08:14:04.949 INFO:teuthology.orchestra.run.smithi037.stdout:true 2024-03-29T08:14:05.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:05.073+0000 7f532e800700 1 -- 172.21.15.37:0/3257028852 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5328070c70 msgr2=0x7f5328109130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:14:05.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:05.073+0000 7f532e800700 1 --2- 172.21.15.37:0/3257028852 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5328070c70 0x7f5328109130 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f5314004970 tx=0x7f5314009e30 comp rx=0 tx=0).stop 2024-03-29T08:14:05.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:05.073+0000 7f532e800700 1 -- 172.21.15.37:0/3257028852 shutdown_connections 2024-03-29T08:14:05.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:05.073+0000 7f532e800700 1 --2- 172.21.15.37:0/3257028852 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5328070c70 0x7f5328109130 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:14:05.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:05.073+0000 7f532e800700 1 --2- 172.21.15.37:0/3257028852 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5328070360 0x7f5328070730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:14:05.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:05.073+0000 7f532e800700 1 -- 172.21.15.37:0/3257028852 >> 172.21.15.37:0/3257028852 conn(0x7f53280f8970 msgr2=0x7f53280fad80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:14:05.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:05.073+0000 7f532e800700 1 -- 172.21.15.37:0/3257028852 shutdown_connections 2024-03-29T08:14:05.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:05.073+0000 7f532e800700 1 -- 172.21.15.37:0/3257028852 wait complete. 2024-03-29T08:14:05.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:05.074+0000 7f532e800700 1 Processor -- start 2024-03-29T08:14:05.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:05.074+0000 7f532e800700 1 -- start start 2024-03-29T08:14:05.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:05.074+0000 7f532e800700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5328070360 0x7f5328115c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:14:05.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:05.074+0000 7f532e800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5328070c70 0x7f5328116170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:14:05.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:05.074+0000 7f532e800700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53281167c0 con 0x7f5328070c70 2024-03-29T08:14:05.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:05.074+0000 7f532e800700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f532810fbd0 con 0x7f5328070360 2024-03-29T08:14:05.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:05.075+0000 7f5326a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5328070c70 0x7f5328116170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:14:05.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:05.075+0000 7f5326a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5328070c70 0x7f5328116170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53416/0 (socket says 172.21.15.37:53416) 2024-03-29T08:14:05.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:05.075+0000 7f5326a00700 1 -- 172.21.15.37:0/1819269179 learned_addr learned my addr 172.21.15.37:0/1819269179 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:14:05.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:05.075+0000 7f5327400700 1 --2- 172.21.15.37:0/1819269179 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5328070360 0x7f5328115c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:14:05.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:05.075+0000 7f5326a00700 1 -- 172.21.15.37:0/1819269179 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5328070360 msgr2=0x7f5328115c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:14:05.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:05.075+0000 7f5326a00700 1 --2- 172.21.15.37:0/1819269179 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5328070360 0x7f5328115c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:14:05.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:05.075+0000 7f5326a00700 1 -- 172.21.15.37:0/1819269179 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5314005040 con 0x7f5328070c70 2024-03-29T08:14:05.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:05.075+0000 7f5327400700 1 --2- 172.21.15.37:0/1819269179 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5328070360 0x7f5328115c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:14:05.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:05.075+0000 7f5326a00700 1 --2- 172.21.15.37:0/1819269179 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5328070c70 0x7f5328116170 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f5314004f50 tx=0x7f5314009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:14:05.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:05.076+0000 7f531f400700 1 -- 172.21.15.37:0/1819269179 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f531400e040 con 0x7f5328070c70 2024-03-29T08:14:05.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:05.076+0000 7f531f400700 1 -- 172.21.15.37:0/1819269179 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f5314013420 con 0x7f5328070c70 2024-03-29T08:14:05.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:05.076+0000 7f531f400700 1 -- 172.21.15.37:0/1819269179 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f531401c610 con 0x7f5328070c70 2024-03-29T08:14:05.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:05.076+0000 7f532e800700 1 -- 172.21.15.37:0/1819269179 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f532810fe50 con 0x7f5328070c70 2024-03-29T08:14:05.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:05.076+0000 7f532e800700 1 -- 172.21.15.37:0/1819269179 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5328110390 con 0x7f5328070c70 2024-03-29T08:14:05.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:05.077+0000 7f532e800700 1 -- 172.21.15.37:0/1819269179 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5328071aa0 con 0x7f5328070c70 2024-03-29T08:14:05.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:05.078+0000 7f531f400700 1 -- 172.21.15.37:0/1819269179 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f5314013590 con 0x7f5328070c70 2024-03-29T08:14:05.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:05.079+0000 7f531f400700 1 --2- 172.21.15.37:0/1819269179 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f530c06cc30 0x7f530c06f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:14:05.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:05.079+0000 7f531f400700 1 -- 172.21.15.37:0/1819269179 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f531408d1f0 con 0x7f5328070c70 2024-03-29T08:14:05.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:05.079+0000 7f5327400700 1 --2- 172.21.15.37:0/1819269179 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f530c06cc30 0x7f530c06f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:14:05.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:05.080+0000 7f5327400700 1 --2- 172.21.15.37:0/1819269179 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f530c06cc30 0x7f530c06f0e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f5318007fd0 tx=0x7f5318009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:14:05.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:05.083+0000 7f531f400700 1 -- 172.21.15.37:0/1819269179 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f5314057250 con 0x7f5328070c70 2024-03-29T08:14:05.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:05.314+0000 7f532e800700 1 -- 172.21.15.37:0/1819269179 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f532810bfd0 con 0x7f530c06cc30 2024-03-29T08:14:05.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:05.315+0000 7f531f400700 1 -- 172.21.15.37:0/1819269179 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f532810bfd0 con 0x7f530c06cc30 2024-03-29T08:14:05.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:05.318+0000 7f532e800700 1 -- 172.21.15.37:0/1819269179 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f530c06cc30 msgr2=0x7f530c06f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:14:05.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:05.318+0000 7f532e800700 1 --2- 172.21.15.37:0/1819269179 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f530c06cc30 0x7f530c06f0e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f5318007fd0 tx=0x7f5318009040 comp rx=0 tx=0).stop 2024-03-29T08:14:05.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:05.318+0000 7f532e800700 1 -- 172.21.15.37:0/1819269179 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5328070c70 msgr2=0x7f5328116170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:14:05.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:05.318+0000 7f532e800700 1 --2- 172.21.15.37:0/1819269179 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5328070c70 0x7f5328116170 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f5314004f50 tx=0x7f5314009e30 comp rx=0 tx=0).stop 2024-03-29T08:14:05.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:05.318+0000 7f532e800700 1 -- 172.21.15.37:0/1819269179 shutdown_connections 2024-03-29T08:14:05.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:05.318+0000 7f532e800700 1 --2- 172.21.15.37:0/1819269179 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f530c06cc30 0x7f530c06f0e0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:14:05.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:05.318+0000 7f532e800700 1 --2- 172.21.15.37:0/1819269179 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5328070360 0x7f5328115c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:14:05.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:05.318+0000 7f532e800700 1 --2- 172.21.15.37:0/1819269179 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5328070c70 0x7f5328116170 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:14:05.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:05.318+0000 7f532e800700 1 -- 172.21.15.37:0/1819269179 >> 172.21.15.37:0/1819269179 conn(0x7f53280f8970 msgr2=0x7f53280f9e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:14:05.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:05.318+0000 7f532e800700 1 -- 172.21.15.37:0/1819269179 shutdown_connections 2024-03-29T08:14:05.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:05.318+0000 7f532e800700 1 -- 172.21.15.37:0/1819269179 wait complete. 2024-03-29T08:14:05.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:05.455+0000 7f44f0e00700 1 -- 172.21.15.37:0/3117073151 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f44ec071f90 msgr2=0x7f44ec070360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:14:05.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:05.455+0000 7f44f0e00700 1 --2- 172.21.15.37:0/3117073151 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f44ec071f90 0x7f44ec070360 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f44dc004970 tx=0x7f44dc009e30 comp rx=0 tx=0).stop 2024-03-29T08:14:05.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:05.455+0000 7f44f0e00700 1 -- 172.21.15.37:0/3117073151 shutdown_connections 2024-03-29T08:14:05.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:05.455+0000 7f44f0e00700 1 --2- 172.21.15.37:0/3117073151 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f44ec0708a0 0x7f44ec070d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:14:05.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:05.455+0000 7f44f0e00700 1 --2- 172.21.15.37:0/3117073151 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f44ec071f90 0x7f44ec070360 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:14:05.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:05.455+0000 7f44f0e00700 1 -- 172.21.15.37:0/3117073151 >> 172.21.15.37:0/3117073151 conn(0x7f44ec0f89b0 msgr2=0x7f44ec0fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:14:05.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:05.455+0000 7f44f0e00700 1 -- 172.21.15.37:0/3117073151 shutdown_connections 2024-03-29T08:14:05.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:05.455+0000 7f44f0e00700 1 -- 172.21.15.37:0/3117073151 wait complete. 2024-03-29T08:14:05.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:05.456+0000 7f44f0e00700 1 Processor -- start 2024-03-29T08:14:05.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:05.456+0000 7f44f0e00700 1 -- start start 2024-03-29T08:14:05.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:05.456+0000 7f44f0e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f44ec0708a0 0x7f44ec11a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:14:05.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:05.456+0000 7f44f0e00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f44ec071f90 0x7f44ec11a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:14:05.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:05.456+0000 7f44f0e00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44ec11ac50 con 0x7f44ec0708a0 2024-03-29T08:14:05.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:05.456+0000 7f44f0e00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44ec114020 con 0x7f44ec071f90 2024-03-29T08:14:05.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:05.457+0000 7f44e9e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f44ec0708a0 0x7f44ec11a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:14:05.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:05.457+0000 7f44e9e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f44ec0708a0 0x7f44ec11a0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53442/0 (socket says 172.21.15.37:53442) 2024-03-29T08:14:05.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:05.457+0000 7f44e9e00700 1 -- 172.21.15.37:0/119468415 learned_addr learned my addr 172.21.15.37:0/119468415 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:14:05.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:05.457+0000 7f44e9400700 1 --2- 172.21.15.37:0/119468415 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f44ec071f90 0x7f44ec11a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:14:05.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:05.457+0000 7f44e9e00700 1 -- 172.21.15.37:0/119468415 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f44ec071f90 msgr2=0x7f44ec11a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:14:05.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:05.457+0000 7f44e9e00700 1 --2- 172.21.15.37:0/119468415 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f44ec071f90 0x7f44ec11a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:14:05.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:05.457+0000 7f44e9e00700 1 -- 172.21.15.37:0/119468415 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44dc005040 con 0x7f44ec0708a0 2024-03-29T08:14:05.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:05.457+0000 7f44e9e00700 1 --2- 172.21.15.37:0/119468415 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f44ec0708a0 0x7f44ec11a0c0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f44dc005010 tx=0x7f44dc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:14:05.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:05.458+0000 7f44e2000700 1 -- 172.21.15.37:0/119468415 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f44dc00e040 con 0x7f44ec0708a0 2024-03-29T08:14:05.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:05.458+0000 7f44e2000700 1 -- 172.21.15.37:0/119468415 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f44dc009930 con 0x7f44ec0708a0 2024-03-29T08:14:05.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:05.458+0000 7f44f0e00700 1 -- 172.21.15.37:0/119468415 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44ec1142a0 con 0x7f44ec0708a0 2024-03-29T08:14:05.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:05.458+0000 7f44e2000700 1 -- 172.21.15.37:0/119468415 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f44dc01c3f0 con 0x7f44ec0708a0 2024-03-29T08:14:05.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:05.458+0000 7f44f0e00700 1 -- 172.21.15.37:0/119468415 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44ec1147e0 con 0x7f44ec0708a0 2024-03-29T08:14:05.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:05.459+0000 7f44f0e00700 1 -- 172.21.15.37:0/119468415 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44ec071aa0 con 0x7f44ec0708a0 2024-03-29T08:14:05.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:05.460+0000 7f44e2000700 1 -- 172.21.15.37:0/119468415 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f44dc013420 con 0x7f44ec0708a0 2024-03-29T08:14:05.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:05.461+0000 7f44e2000700 1 --2- 172.21.15.37:0/119468415 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f44d006cc30 0x7f44d006f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:14:05.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:05.461+0000 7f44e2000700 1 -- 172.21.15.37:0/119468415 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f44dc08d1f0 con 0x7f44ec0708a0 2024-03-29T08:14:05.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:05.461+0000 7f44e9400700 1 --2- 172.21.15.37:0/119468415 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f44d006cc30 0x7f44d006f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:14:05.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:05.462+0000 7f44e9400700 1 --2- 172.21.15.37:0/119468415 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f44d006cc30 0x7f44d006f0e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f44d8004600 tx=0x7f44d800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:14:05.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:05.465+0000 7f44e2000700 1 -- 172.21.15.37:0/119468415 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f44dc05ad40 con 0x7f44ec0708a0 2024-03-29T08:14:05.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:05.692+0000 7f44f0e00700 1 -- 172.21.15.37:0/119468415 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f44ec04bed0 con 0x7f44d006cc30 2024-03-29T08:14:05.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:05.702+0000 7f44e2000700 1 -- 172.21.15.37:0/119468415 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f44ec04bed0 con 0x7f44d006cc30 2024-03-29T08:14:05.702 INFO:teuthology.orchestra.run.smithi037.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-29T08:14:05.702 INFO:teuthology.orchestra.run.smithi037.stdout:alertmanager.smithi037 smithi037 *:9093,9094 running (3m) 12s ago 16m 14.9M - 0.25.0 c8568f914cd2 2f9f9746ed0f 2024-03-29T08:14:05.702 INFO:teuthology.orchestra.run.smithi037.stdout:crash.smithi037 smithi037 running (50s) 12s ago 17m 6647k - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 178d416a3186 2024-03-29T08:14:05.702 INFO:teuthology.orchestra.run.smithi037.stdout:crash.smithi148 smithi148 running (44s) 39s ago 15m 6656k - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 23666989d58f 2024-03-29T08:14:05.702 INFO:teuthology.orchestra.run.smithi037.stdout:grafana.smithi037 smithi037 *:3000 running (3m) 12s ago 16m 74.6M - 9.4.7 954c08fa6188 6b976fe49952 2024-03-29T08:14:05.703 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi037.iifsmh smithi037 running (10m) 12s ago 10m 110M - 17.2.7-789-g4213e17c cf22eb8683be 074e1acf331f 2024-03-29T08:14:05.703 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi037.mmwgxe smithi037 running (10m) 12s ago 10m 114M - 17.2.7-789-g4213e17c cf22eb8683be 9a5b1e72de5c 2024-03-29T08:14:05.703 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi148.qukpjh smithi148 running (10m) 39s ago 10m 14.1M - 17.2.7-789-g4213e17c cf22eb8683be 8e469782a273 2024-03-29T08:14:05.703 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi148.sqteih smithi148 running (10m) 39s ago 10m 14.0M - 17.2.7-789-g4213e17c cf22eb8683be 8e68f858bb22 2024-03-29T08:14:05.703 INFO:teuthology.orchestra.run.smithi037.stdout:mgr.smithi037.wjikii smithi037 *:8443,9283,8765 running (6m) 12s ago 18m 495M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff c5a1417b9417 2024-03-29T08:14:05.703 INFO:teuthology.orchestra.run.smithi037.stdout:mgr.smithi148.onhgvm smithi148 *:8443,9283,8765 running (5m) 39s ago 15m 447M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff fcfdbce062bd 2024-03-29T08:14:05.703 INFO:teuthology.orchestra.run.smithi037.stdout:mon.smithi037 smithi037 running (2m) 12s ago 18m 44.8M 2048M 18.2.2-1015-gd0e392d9 1cc2f45c17ff eea2060783af 2024-03-29T08:14:05.703 INFO:teuthology.orchestra.run.smithi037.stdout:mon.smithi148 smithi148 running (96s) 39s ago 15m 35.5M 2048M 18.2.2-1015-gd0e392d9 1cc2f45c17ff ad4b2c5748be 2024-03-29T08:14:05.703 INFO:teuthology.orchestra.run.smithi037.stdout:node-exporter.smithi037 smithi037 *:9100 running (4m) 12s ago 17m 7516k - 1.5.0 0da6a335fe13 26b8ae9cb878 2024-03-29T08:14:05.703 INFO:teuthology.orchestra.run.smithi037.stdout:node-exporter.smithi148 smithi148 *:9100 running (4m) 39s ago 15m 7600k - 1.5.0 0da6a335fe13 98414e4e64ed 2024-03-29T08:14:05.703 INFO:teuthology.orchestra.run.smithi037.stdout:osd.0 smithi037 running (18s) 12s ago 14m 10.2M 1619M 18.2.2-1015-gd0e392d9 1cc2f45c17ff c0c584f6245c 2024-03-29T08:14:05.703 INFO:teuthology.orchestra.run.smithi037.stdout:osd.1 smithi037 running (14m) 12s ago 14m 229M 1619M 17.2.7-789-g4213e17c cf22eb8683be 5582cff11b7a 2024-03-29T08:14:05.703 INFO:teuthology.orchestra.run.smithi037.stdout:osd.2 smithi037 running (13m) 12s ago 13m 198M 1619M 17.2.7-789-g4213e17c cf22eb8683be f809bfab13e5 2024-03-29T08:14:05.704 INFO:teuthology.orchestra.run.smithi037.stdout:osd.3 smithi148 running (13m) 39s ago 13m 261M 2642M 17.2.7-789-g4213e17c cf22eb8683be a0a8224bb4b0 2024-03-29T08:14:05.704 INFO:teuthology.orchestra.run.smithi037.stdout:osd.4 smithi148 running (12m) 39s ago 12m 233M 2642M 17.2.7-789-g4213e17c cf22eb8683be a00b69047ed7 2024-03-29T08:14:05.704 INFO:teuthology.orchestra.run.smithi037.stdout:osd.5 smithi148 running (12m) 39s ago 12m 234M 2642M 17.2.7-789-g4213e17c cf22eb8683be 044a3f7db7f8 2024-03-29T08:14:05.704 INFO:teuthology.orchestra.run.smithi037.stdout:prometheus.smithi037 smithi037 *:9095 running (4m) 12s ago 16m 51.6M - 2.43.0 a07b618ecd1d 30c0223ec589 2024-03-29T08:14:05.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:05.704+0000 7f44f0e00700 1 -- 172.21.15.37:0/119468415 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f44d006cc30 msgr2=0x7f44d006f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:14:05.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:05.704+0000 7f44f0e00700 1 --2- 172.21.15.37:0/119468415 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f44d006cc30 0x7f44d006f0e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f44d8004600 tx=0x7f44d800b040 comp rx=0 tx=0).stop 2024-03-29T08:14:05.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:05.704+0000 7f44f0e00700 1 -- 172.21.15.37:0/119468415 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f44ec0708a0 msgr2=0x7f44ec11a0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:14:05.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:05.704+0000 7f44f0e00700 1 --2- 172.21.15.37:0/119468415 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f44ec0708a0 0x7f44ec11a0c0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f44dc005010 tx=0x7f44dc009e30 comp rx=0 tx=0).stop 2024-03-29T08:14:05.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:05.704+0000 7f44f0e00700 1 -- 172.21.15.37:0/119468415 shutdown_connections 2024-03-29T08:14:05.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:05.704+0000 7f44f0e00700 1 --2- 172.21.15.37:0/119468415 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f44d006cc30 0x7f44d006f0e0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:14:05.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:05.704+0000 7f44f0e00700 1 --2- 172.21.15.37:0/119468415 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f44ec0708a0 0x7f44ec11a0c0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:14:05.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:05.704+0000 7f44f0e00700 1 --2- 172.21.15.37:0/119468415 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f44ec071f90 0x7f44ec11a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:14:05.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:05.704+0000 7f44f0e00700 1 -- 172.21.15.37:0/119468415 >> 172.21.15.37:0/119468415 conn(0x7f44ec0f89b0 msgr2=0x7f44ec0f9b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:14:05.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:05.705+0000 7f44f0e00700 1 -- 172.21.15.37:0/119468415 shutdown_connections 2024-03-29T08:14:05.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:05.705+0000 7f44f0e00700 1 -- 172.21.15.37:0/119468415 wait complete. 2024-03-29T08:14:05.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:05.840+0000 7ff332e00700 1 -- 172.21.15.37:0/3386136892 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff32c102710 msgr2=0x7ff32c0fbd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:14:05.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:05.840+0000 7ff332e00700 1 --2- 172.21.15.37:0/3386136892 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff32c102710 0x7ff32c0fbd80 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7ff318004940 tx=0x7ff318009e30 comp rx=0 tx=0).stop 2024-03-29T08:14:05.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:05.840+0000 7ff332e00700 1 -- 172.21.15.37:0/3386136892 shutdown_connections 2024-03-29T08:14:05.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:05.840+0000 7ff332e00700 1 --2- 172.21.15.37:0/3386136892 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff32c102710 0x7ff32c0fbd80 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:14:05.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:05.840+0000 7ff332e00700 1 --2- 172.21.15.37:0/3386136892 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff32c101d70 0x7ff32c102140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:14:05.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:05.840+0000 7ff332e00700 1 -- 172.21.15.37:0/3386136892 >> 172.21.15.37:0/3386136892 conn(0x7ff32c072bc0 msgr2=0x7ff32c072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:14:05.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:05.840+0000 7ff332e00700 1 -- 172.21.15.37:0/3386136892 shutdown_connections 2024-03-29T08:14:05.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:05.841+0000 7ff332e00700 1 -- 172.21.15.37:0/3386136892 wait complete. 2024-03-29T08:14:05.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:05.841+0000 7ff332e00700 1 Processor -- start 2024-03-29T08:14:05.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:05.841+0000 7ff332e00700 1 -- start start 2024-03-29T08:14:05.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:05.842+0000 7ff332e00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff32c101d70 0x7ff32c102bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:14:05.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:05.842+0000 7ff332e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff32c102710 0x7ff32c103110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:14:05.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:05.842+0000 7ff332e00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff32c1036a0 con 0x7ff32c102710 2024-03-29T08:14:05.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:05.842+0000 7ff332e00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff32c1037e0 con 0x7ff32c101d70 2024-03-29T08:14:05.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:05.842+0000 7ff32b400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff32c102710 0x7ff32c103110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:14:05.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:05.842+0000 7ff32b400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff32c102710 0x7ff32c103110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53460/0 (socket says 172.21.15.37:53460) 2024-03-29T08:14:05.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:05.842+0000 7ff32b400700 1 -- 172.21.15.37:0/2119375618 learned_addr learned my addr 172.21.15.37:0/2119375618 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:14:05.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:05.842+0000 7ff32be00700 1 --2- 172.21.15.37:0/2119375618 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff32c101d70 0x7ff32c102bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:14:05.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:05.842+0000 7ff32b400700 1 -- 172.21.15.37:0/2119375618 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff32c101d70 msgr2=0x7ff32c102bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:14:05.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:05.842+0000 7ff32b400700 1 --2- 172.21.15.37:0/2119375618 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff32c101d70 0x7ff32c102bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:14:05.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:05.842+0000 7ff32b400700 1 -- 172.21.15.37:0/2119375618 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff318005040 con 0x7ff32c102710 2024-03-29T08:14:05.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:05.842+0000 7ff32be00700 1 --2- 172.21.15.37:0/2119375618 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff32c101d70 0x7ff32c102bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:14:05.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:05.843+0000 7ff32b400700 1 --2- 172.21.15.37:0/2119375618 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff32c102710 0x7ff32c103110 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ff318009390 tx=0x7ff318009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:14:05.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:05.843+0000 7ff323e00700 1 -- 172.21.15.37:0/2119375618 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff31800e040 con 0x7ff32c102710 2024-03-29T08:14:05.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:05.843+0000 7ff323e00700 1 -- 172.21.15.37:0/2119375618 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ff318013420 con 0x7ff32c102710 2024-03-29T08:14:05.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:05.843+0000 7ff323e00700 1 -- 172.21.15.37:0/2119375618 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff31801c630 con 0x7ff32c102710 2024-03-29T08:14:05.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:05.843+0000 7ff332e00700 1 -- 172.21.15.37:0/2119375618 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff32c103a30 con 0x7ff32c102710 2024-03-29T08:14:05.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:05.843+0000 7ff332e00700 1 -- 172.21.15.37:0/2119375618 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff32c11a1c0 con 0x7ff32c102710 2024-03-29T08:14:05.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:05.844+0000 7ff332e00700 1 -- 172.21.15.37:0/2119375618 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff32c0fd4c0 con 0x7ff32c102710 2024-03-29T08:14:05.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:05.845+0000 7ff323e00700 1 -- 172.21.15.37:0/2119375618 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7ff318013a00 con 0x7ff32c102710 2024-03-29T08:14:05.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:05.846+0000 7ff323e00700 1 --2- 172.21.15.37:0/2119375618 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7ff31006cc30 0x7ff31006f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:14:05.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:05.846+0000 7ff323e00700 1 -- 172.21.15.37:0/2119375618 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6324+0+0 (secure 0 0 0) 0x7ff31808d2e0 con 0x7ff32c102710 2024-03-29T08:14:05.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:05.846+0000 7ff32be00700 1 --2- 172.21.15.37:0/2119375618 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7ff31006cc30 0x7ff31006f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:14:05.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:05.847+0000 7ff32be00700 1 --2- 172.21.15.37:0/2119375618 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7ff31006cc30 0x7ff31006f0e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7ff31c009fd0 tx=0x7ff31c009ee0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:14:05.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:05.851+0000 7ff323e00700 1 -- 172.21.15.37:0/2119375618 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7ff318057320 con 0x7ff32c102710 2024-03-29T08:14:06.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:06.154+0000 7ff332e00700 1 -- 172.21.15.37:0/2119375618 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7ff32c108d40 con 0x7ff32c102710 2024-03-29T08:14:06.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:06.155+0000 7ff323e00700 1 -- 172.21.15.37:0/2119375618 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+847 (secure 0 0 0) 0x7ff31805a940 con 0x7ff32c102710 2024-03-29T08:14:06.155 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-29T08:14:06.155 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": { 2024-03-29T08:14:06.155 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 2 2024-03-29T08:14:06.155 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:14:06.155 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": { 2024-03-29T08:14:06.155 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 2 2024-03-29T08:14:06.156 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:14:06.156 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": { 2024-03-29T08:14:06.156 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 17.2.7-789-g4213e17c (4213e17c4de67016ed14f55d22503f4923b3a901) quincy (stable)": 5, 2024-03-29T08:14:06.156 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 1 2024-03-29T08:14:06.156 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:14:06.156 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": { 2024-03-29T08:14:06.156 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 17.2.7-789-g4213e17c (4213e17c4de67016ed14f55d22503f4923b3a901) quincy (stable)": 4 2024-03-29T08:14:06.156 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:14:06.156 INFO:teuthology.orchestra.run.smithi037.stdout: "overall": { 2024-03-29T08:14:06.156 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 17.2.7-789-g4213e17c (4213e17c4de67016ed14f55d22503f4923b3a901) quincy (stable)": 9, 2024-03-29T08:14:06.156 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 5 2024-03-29T08:14:06.156 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-29T08:14:06.156 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-29T08:14:06.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:06.157+0000 7ff332e00700 1 -- 172.21.15.37:0/2119375618 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7ff31006cc30 msgr2=0x7ff31006f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:14:06.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:06.157+0000 7ff332e00700 1 --2- 172.21.15.37:0/2119375618 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7ff31006cc30 0x7ff31006f0e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7ff31c009fd0 tx=0x7ff31c009ee0 comp rx=0 tx=0).stop 2024-03-29T08:14:06.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:06.157+0000 7ff332e00700 1 -- 172.21.15.37:0/2119375618 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff32c102710 msgr2=0x7ff32c103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:14:06.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:06.157+0000 7ff332e00700 1 --2- 172.21.15.37:0/2119375618 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff32c102710 0x7ff32c103110 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ff318009390 tx=0x7ff318009e30 comp rx=0 tx=0).stop 2024-03-29T08:14:06.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:06.157+0000 7ff332e00700 1 -- 172.21.15.37:0/2119375618 shutdown_connections 2024-03-29T08:14:06.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:06.157+0000 7ff332e00700 1 --2- 172.21.15.37:0/2119375618 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7ff31006cc30 0x7ff31006f0e0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:14:06.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:06.157+0000 7ff332e00700 1 --2- 172.21.15.37:0/2119375618 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff32c101d70 0x7ff32c102bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:14:06.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:06.157+0000 7ff332e00700 1 --2- 172.21.15.37:0/2119375618 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff32c102710 0x7ff32c103110 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:14:06.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:06.157+0000 7ff332e00700 1 -- 172.21.15.37:0/2119375618 >> 172.21.15.37:0/2119375618 conn(0x7ff32c072bc0 msgr2=0x7ff32c065a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:14:06.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:06.157+0000 7ff332e00700 1 -- 172.21.15.37:0/2119375618 shutdown_connections 2024-03-29T08:14:06.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:06.158+0000 7ff332e00700 1 -- 172.21.15.37:0/2119375618 wait complete. 2024-03-29T08:14:06.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:06.293+0000 7f413f200700 1 -- 172.21.15.37:0/938365772 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f41401035e0 msgr2=0x7f41400fcc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:14:06.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:06.293+0000 7f413f200700 1 --2- 172.21.15.37:0/938365772 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f41401035e0 0x7f41400fcc50 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f412c004940 tx=0x7f412c009e30 comp rx=0 tx=0).stop 2024-03-29T08:14:06.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:06.293+0000 7f413f200700 1 -- 172.21.15.37:0/938365772 shutdown_connections 2024-03-29T08:14:06.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:06.293+0000 7f413f200700 1 --2- 172.21.15.37:0/938365772 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f41401035e0 0x7f41400fcc50 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:14:06.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:06.293+0000 7f413f200700 1 --2- 172.21.15.37:0/938365772 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4140102c40 0x7f4140103010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:14:06.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:06.293+0000 7f413f200700 1 -- 172.21.15.37:0/938365772 >> 172.21.15.37:0/938365772 conn(0x7f41400f8830 msgr2=0x7f41400fac40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:14:06.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:06.293+0000 7f413f200700 1 -- 172.21.15.37:0/938365772 shutdown_connections 2024-03-29T08:14:06.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:06.293+0000 7f413f200700 1 -- 172.21.15.37:0/938365772 wait complete. 2024-03-29T08:14:06.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:06.294+0000 7f413f200700 1 Processor -- start 2024-03-29T08:14:06.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:06.295+0000 7f413f200700 1 -- start start 2024-03-29T08:14:06.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:06.295+0000 7f413f200700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4140102c40 0x7f4140115a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:14:06.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:06.295+0000 7f413f200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f41401035e0 0x7f4140115f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:14:06.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:06.295+0000 7f413f200700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41401165d0 con 0x7f41401035e0 2024-03-29T08:14:06.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:06.295+0000 7f413f200700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f414010f9c0 con 0x7f4140102c40 2024-03-29T08:14:06.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:06.296+0000 7f413d400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f41401035e0 0x7f4140115f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:14:06.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:06.296+0000 7f413d400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f41401035e0 0x7f4140115f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53476/0 (socket says 172.21.15.37:53476) 2024-03-29T08:14:06.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:06.296+0000 7f413d400700 1 -- 172.21.15.37:0/4246950529 learned_addr learned my addr 172.21.15.37:0/4246950529 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:14:06.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:06.296+0000 7f413de00700 1 --2- 172.21.15.37:0/4246950529 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4140102c40 0x7f4140115a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:14:06.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:06.296+0000 7f413d400700 1 -- 172.21.15.37:0/4246950529 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4140102c40 msgr2=0x7f4140115a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:14:06.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:06.296+0000 7f413d400700 1 --2- 172.21.15.37:0/4246950529 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4140102c40 0x7f4140115a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:14:06.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:06.296+0000 7f413d400700 1 -- 172.21.15.37:0/4246950529 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f412c005040 con 0x7f41401035e0 2024-03-29T08:14:06.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:06.296+0000 7f413d400700 1 --2- 172.21.15.37:0/4246950529 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f41401035e0 0x7f4140115f80 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f412c009fd0 tx=0x7f412c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:14:06.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:06.296+0000 7f4136000700 1 -- 172.21.15.37:0/4246950529 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f412c00e030 con 0x7f41401035e0 2024-03-29T08:14:06.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:06.297+0000 7f4136000700 1 -- 172.21.15.37:0/4246950529 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f412c013e00 con 0x7f41401035e0 2024-03-29T08:14:06.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:06.297+0000 7f4136000700 1 -- 172.21.15.37:0/4246950529 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f412c01ccd0 con 0x7f41401035e0 2024-03-29T08:14:06.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:06.297+0000 7f413f200700 1 -- 172.21.15.37:0/4246950529 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f414010fc40 con 0x7f41401035e0 2024-03-29T08:14:06.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:06.297+0000 7f413f200700 1 -- 172.21.15.37:0/4246950529 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4140110200 con 0x7f41401035e0 2024-03-29T08:14:06.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:06.298+0000 7f413f200700 1 -- 172.21.15.37:0/4246950529 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41400fe390 con 0x7f41401035e0 2024-03-29T08:14:06.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:06.299+0000 7f4136000700 1 -- 172.21.15.37:0/4246950529 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f412c01c500 con 0x7f41401035e0 2024-03-29T08:14:06.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:06.299+0000 7f4136000700 1 --2- 172.21.15.37:0/4246950529 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f412406cc30 0x7f412406f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:14:06.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:06.300+0000 7f4136000700 1 -- 172.21.15.37:0/4246950529 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f412c08d820 con 0x7f41401035e0 2024-03-29T08:14:06.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:06.300+0000 7f413de00700 1 --2- 172.21.15.37:0/4246950529 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f412406cc30 0x7f412406f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:14:06.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:06.300+0000 7f413de00700 1 --2- 172.21.15.37:0/4246950529 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f412406cc30 0x7f412406f0e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f4130009fd0 tx=0x7f413000b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:14:06.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:06.304+0000 7f4136000700 1 -- 172.21.15.37:0/4246950529 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f412c05b370 con 0x7f41401035e0 2024-03-29T08:14:06.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:06.558+0000 7f413f200700 1 -- 172.21.15.37:0/4246950529 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f4140109bf0 con 0x7f41401035e0 2024-03-29T08:14:06.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:06.558+0000 7f4136000700 1 -- 172.21.15.37:0/4246950529 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 20 v20) v1 ==== 76+0+1858 (secure 0 0 0) 0x7f412c05af00 con 0x7f41401035e0 2024-03-29T08:14:06.560 INFO:teuthology.orchestra.run.smithi037.stdout:e20 2024-03-29T08:14:06.560 INFO:teuthology.orchestra.run.smithi037.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-29T08:14:06.560 INFO:teuthology.orchestra.run.smithi037.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T08:14:06.560 INFO:teuthology.orchestra.run.smithi037.stdout:legacy client fscid: 1 2024-03-29T08:14:06.560 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:14:06.560 INFO:teuthology.orchestra.run.smithi037.stdout:Filesystem 'cephfs' (1) 2024-03-29T08:14:06.560 INFO:teuthology.orchestra.run.smithi037.stdout:fs_name cephfs 2024-03-29T08:14:06.560 INFO:teuthology.orchestra.run.smithi037.stdout:epoch 20 2024-03-29T08:14:06.560 INFO:teuthology.orchestra.run.smithi037.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-03-29T08:14:06.560 INFO:teuthology.orchestra.run.smithi037.stdout:created 2024-03-29T08:03:29.241269+0000 2024-03-29T08:14:06.560 INFO:teuthology.orchestra.run.smithi037.stdout:modified 2024-03-29T08:11:27.181622+0000 2024-03-29T08:14:06.560 INFO:teuthology.orchestra.run.smithi037.stdout:tableserver 0 2024-03-29T08:14:06.561 INFO:teuthology.orchestra.run.smithi037.stdout:root 0 2024-03-29T08:14:06.561 INFO:teuthology.orchestra.run.smithi037.stdout:session_timeout 60 2024-03-29T08:14:06.561 INFO:teuthology.orchestra.run.smithi037.stdout:session_autoclose 300 2024-03-29T08:14:06.561 INFO:teuthology.orchestra.run.smithi037.stdout:max_file_size 1099511627776 2024-03-29T08:14:06.561 INFO:teuthology.orchestra.run.smithi037.stdout:max_xattr_size 65536 2024-03-29T08:14:06.561 INFO:teuthology.orchestra.run.smithi037.stdout:required_client_features {} 2024-03-29T08:14:06.561 INFO:teuthology.orchestra.run.smithi037.stdout:last_failure 0 2024-03-29T08:14:06.561 INFO:teuthology.orchestra.run.smithi037.stdout:last_failure_osd_epoch 47 2024-03-29T08:14:06.561 INFO:teuthology.orchestra.run.smithi037.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T08:14:06.561 INFO:teuthology.orchestra.run.smithi037.stdout:max_mds 2 2024-03-29T08:14:06.561 INFO:teuthology.orchestra.run.smithi037.stdout:in 0,1 2024-03-29T08:14:06.561 INFO:teuthology.orchestra.run.smithi037.stdout:up {0=14470,1=14482} 2024-03-29T08:14:06.561 INFO:teuthology.orchestra.run.smithi037.stdout:failed 2024-03-29T08:14:06.562 INFO:teuthology.orchestra.run.smithi037.stdout:damaged 2024-03-29T08:14:06.562 INFO:teuthology.orchestra.run.smithi037.stdout:stopped 2024-03-29T08:14:06.562 INFO:teuthology.orchestra.run.smithi037.stdout:data_pools [3] 2024-03-29T08:14:06.562 INFO:teuthology.orchestra.run.smithi037.stdout:metadata_pool 2 2024-03-29T08:14:06.562 INFO:teuthology.orchestra.run.smithi037.stdout:inline_data disabled 2024-03-29T08:14:06.562 INFO:teuthology.orchestra.run.smithi037.stdout:balancer 2024-03-29T08:14:06.562 INFO:teuthology.orchestra.run.smithi037.stdout:bal_rank_mask -1 2024-03-29T08:14:06.562 INFO:teuthology.orchestra.run.smithi037.stdout:standby_count_wanted 1 2024-03-29T08:14:06.562 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi037.mmwgxe{0:14470} state up:active seq 5 join_fscid=1 addr [v2:172.21.15.37:6826/1559618909,v1:172.21.15.37:6827/1559618909] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:14:06.562 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi037.iifsmh{1:14482} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.37:6828/2341530495,v1:172.21.15.37:6829/2341530495] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:14:06.562 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:14:06.562 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:14:06.562 INFO:teuthology.orchestra.run.smithi037.stdout:Standby daemons: 2024-03-29T08:14:06.562 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:14:06.563 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi148.qukpjh{-1:24319} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.148:6826/2641327912,v1:172.21.15.148:6827/2641327912] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:14:06.563 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi148.sqteih{-1:24327} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.148:6824/3252351726,v1:172.21.15.148:6825/3252351726] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:14:06.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:06.561+0000 7f413f200700 1 -- 172.21.15.37:0/4246950529 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f412406cc30 msgr2=0x7f412406f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:14:06.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:06.561+0000 7f413f200700 1 --2- 172.21.15.37:0/4246950529 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f412406cc30 0x7f412406f0e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f4130009fd0 tx=0x7f413000b040 comp rx=0 tx=0).stop 2024-03-29T08:14:06.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:06.561+0000 7f413f200700 1 -- 172.21.15.37:0/4246950529 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f41401035e0 msgr2=0x7f4140115f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:14:06.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:06.561+0000 7f413f200700 1 --2- 172.21.15.37:0/4246950529 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f41401035e0 0x7f4140115f80 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f412c009fd0 tx=0x7f412c009e30 comp rx=0 tx=0).stop 2024-03-29T08:14:06.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:06.561+0000 7f413f200700 1 -- 172.21.15.37:0/4246950529 shutdown_connections 2024-03-29T08:14:06.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:06.561+0000 7f413f200700 1 --2- 172.21.15.37:0/4246950529 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f412406cc30 0x7f412406f0e0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:14:06.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:06.561+0000 7f413f200700 1 --2- 172.21.15.37:0/4246950529 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4140102c40 0x7f4140115a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:14:06.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:06.561+0000 7f413f200700 1 --2- 172.21.15.37:0/4246950529 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f41401035e0 0x7f4140115f80 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:14:06.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:06.561+0000 7f413f200700 1 -- 172.21.15.37:0/4246950529 >> 172.21.15.37:0/4246950529 conn(0x7f41400f8830 msgr2=0x7f41400f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:14:06.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:06.561+0000 7f413f200700 1 -- 172.21.15.37:0/4246950529 shutdown_connections 2024-03-29T08:14:06.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:06.561+0000 7f413f200700 1 -- 172.21.15.37:0/4246950529 wait complete. 2024-03-29T08:14:06.564 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 20 2024-03-29T08:14:06.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:06.698+0000 7f197d800700 1 -- 172.21.15.37:0/1012990372 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f19781037a0 msgr2=0x7f19780fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:14:06.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:06.698+0000 7f197d800700 1 --2- 172.21.15.37:0/1012990372 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f19781037a0 0x7f19780fce10 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f1964004940 tx=0x7f1964009e30 comp rx=0 tx=0).stop 2024-03-29T08:14:06.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:06.698+0000 7f197d800700 1 -- 172.21.15.37:0/1012990372 shutdown_connections 2024-03-29T08:14:06.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:06.698+0000 7f197d800700 1 --2- 172.21.15.37:0/1012990372 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f19781037a0 0x7f19780fce10 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:14:06.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:06.698+0000 7f197d800700 1 --2- 172.21.15.37:0/1012990372 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1978102e00 0x7f19781031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:14:06.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:06.698+0000 7f197d800700 1 -- 172.21.15.37:0/1012990372 >> 172.21.15.37:0/1012990372 conn(0x7f19780f89b0 msgr2=0x7f19780fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:14:06.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:06.698+0000 7f197d800700 1 -- 172.21.15.37:0/1012990372 shutdown_connections 2024-03-29T08:14:06.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:06.698+0000 7f197d800700 1 -- 172.21.15.37:0/1012990372 wait complete. 2024-03-29T08:14:06.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:06.699+0000 7f197d800700 1 Processor -- start 2024-03-29T08:14:06.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:06.699+0000 7f197d800700 1 -- start start 2024-03-29T08:14:06.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:06.699+0000 7f197d800700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1978102e00 0x7f1978115c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:14:06.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:06.700+0000 7f197d800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f19781037a0 0x7f19781161c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:14:06.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:06.700+0000 7f197d800700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1978116810 con 0x7f19781037a0 2024-03-29T08:14:06.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:06.700+0000 7f197d800700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f197810fbe0 con 0x7f1978102e00 2024-03-29T08:14:06.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:06.700+0000 7f1976000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f19781037a0 0x7f19781161c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:14:06.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:06.700+0000 7f1976000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f19781037a0 0x7f19781161c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53502/0 (socket says 172.21.15.37:53502) 2024-03-29T08:14:06.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:06.700+0000 7f1976000700 1 -- 172.21.15.37:0/1758707924 learned_addr learned my addr 172.21.15.37:0/1758707924 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:14:06.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:06.700+0000 7f1976a00700 1 --2- 172.21.15.37:0/1758707924 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1978102e00 0x7f1978115c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:14:06.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:06.700+0000 7f1976000700 1 -- 172.21.15.37:0/1758707924 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1978102e00 msgr2=0x7f1978115c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:14:06.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:06.700+0000 7f1976000700 1 --2- 172.21.15.37:0/1758707924 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1978102e00 0x7f1978115c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:14:06.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:06.700+0000 7f1976000700 1 -- 172.21.15.37:0/1758707924 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1964005040 con 0x7f19781037a0 2024-03-29T08:14:06.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:06.701+0000 7f1976000700 1 --2- 172.21.15.37:0/1758707924 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f19781037a0 0x7f19781161c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f1964009390 tx=0x7f1964009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:14:06.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:06.701+0000 7f196ea00700 1 -- 172.21.15.37:0/1758707924 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f196400e040 con 0x7f19781037a0 2024-03-29T08:14:06.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:06.701+0000 7f196ea00700 1 -- 172.21.15.37:0/1758707924 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f1964013420 con 0x7f19781037a0 2024-03-29T08:14:06.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:06.701+0000 7f196ea00700 1 -- 172.21.15.37:0/1758707924 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f196401c630 con 0x7f19781037a0 2024-03-29T08:14:06.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:06.701+0000 7f197d800700 1 -- 172.21.15.37:0/1758707924 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f197810fe60 con 0x7f19781037a0 2024-03-29T08:14:06.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:06.701+0000 7f197d800700 1 -- 172.21.15.37:0/1758707924 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1978110420 con 0x7f19781037a0 2024-03-29T08:14:06.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:06.702+0000 7f197d800700 1 -- 172.21.15.37:0/1758707924 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19780fe550 con 0x7f19781037a0 2024-03-29T08:14:06.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:06.703+0000 7f196ea00700 1 -- 172.21.15.37:0/1758707924 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f1964013a00 con 0x7f19781037a0 2024-03-29T08:14:06.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:06.704+0000 7f196ea00700 1 --2- 172.21.15.37:0/1758707924 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f195c06cc30 0x7f195c06f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:14:06.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:06.705+0000 7f196ea00700 1 -- 172.21.15.37:0/1758707924 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f196408d0a0 con 0x7f19781037a0 2024-03-29T08:14:06.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:06.705+0000 7f1976a00700 1 --2- 172.21.15.37:0/1758707924 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f195c06cc30 0x7f195c06f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:14:06.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:06.705+0000 7f1976a00700 1 --2- 172.21.15.37:0/1758707924 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f195c06cc30 0x7f195c06f0e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f1968009fd0 tx=0x7f196800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:14:06.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:06.709+0000 7f196ea00700 1 -- 172.21.15.37:0/1758707924 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f196405abf0 con 0x7f19781037a0 2024-03-29T08:14:06.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:14:06 smithi037 ceph-mon[142325]: from='client.34220 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:14:06.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:14:06 smithi037 ceph-mon[142325]: from='client.34224 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:14:06.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:14:06 smithi037 ceph-mon[142325]: pgmap v66: 65 pgs: 65 active+clean; 259 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:14:06.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:14:06 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/2119375618' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:14:06.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:14:06 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/4246950529' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T08:14:06.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:06.943+0000 7f197d800700 1 -- 172.21.15.37:0/1758707924 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f1978109e10 con 0x7f195c06cc30 2024-03-29T08:14:06.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:06.944+0000 7f196ea00700 1 -- 172.21.15.37:0/1758707924 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f1978109e10 con 0x7f195c06cc30 2024-03-29T08:14:06.945 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-29T08:14:06.945 INFO:teuthology.orchestra.run.smithi037.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59", 2024-03-29T08:14:06.945 INFO:teuthology.orchestra.run.smithi037.stdout: "in_progress": true, 2024-03-29T08:14:06.945 INFO:teuthology.orchestra.run.smithi037.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-29T08:14:06.945 INFO:teuthology.orchestra.run.smithi037.stdout: "services_complete": [ 2024-03-29T08:14:06.945 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr", 2024-03-29T08:14:06.945 INFO:teuthology.orchestra.run.smithi037.stdout: "mon", 2024-03-29T08:14:06.945 INFO:teuthology.orchestra.run.smithi037.stdout: "crash" 2024-03-29T08:14:06.945 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-29T08:14:06.945 INFO:teuthology.orchestra.run.smithi037.stdout: "progress": "7/21 daemons upgraded", 2024-03-29T08:14:06.945 INFO:teuthology.orchestra.run.smithi037.stdout: "message": "Currently upgrading osd daemons", 2024-03-29T08:14:06.945 INFO:teuthology.orchestra.run.smithi037.stdout: "is_paused": false 2024-03-29T08:14:06.945 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-29T08:14:06.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:06.947+0000 7f197d800700 1 -- 172.21.15.37:0/1758707924 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f195c06cc30 msgr2=0x7f195c06f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:14:06.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:06.947+0000 7f197d800700 1 --2- 172.21.15.37:0/1758707924 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f195c06cc30 0x7f195c06f0e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f1968009fd0 tx=0x7f196800b040 comp rx=0 tx=0).stop 2024-03-29T08:14:06.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:06.947+0000 7f197d800700 1 -- 172.21.15.37:0/1758707924 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f19781037a0 msgr2=0x7f19781161c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:14:06.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:06.947+0000 7f197d800700 1 --2- 172.21.15.37:0/1758707924 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f19781037a0 0x7f19781161c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f1964009390 tx=0x7f1964009e30 comp rx=0 tx=0).stop 2024-03-29T08:14:06.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:06.947+0000 7f197d800700 1 -- 172.21.15.37:0/1758707924 shutdown_connections 2024-03-29T08:14:06.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:06.947+0000 7f197d800700 1 --2- 172.21.15.37:0/1758707924 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f195c06cc30 0x7f195c06f0e0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:14:06.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:06.947+0000 7f197d800700 1 --2- 172.21.15.37:0/1758707924 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1978102e00 0x7f1978115c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:14:06.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:06.947+0000 7f197d800700 1 --2- 172.21.15.37:0/1758707924 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f19781037a0 0x7f19781161c0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:14:06.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:06.947+0000 7f197d800700 1 -- 172.21.15.37:0/1758707924 >> 172.21.15.37:0/1758707924 conn(0x7f19780f89b0 msgr2=0x7f19780f9a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:14:06.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:06.947+0000 7f197d800700 1 -- 172.21.15.37:0/1758707924 shutdown_connections 2024-03-29T08:14:06.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:06.947+0000 7f197d800700 1 -- 172.21.15.37:0/1758707924 wait complete. 2024-03-29T08:14:07.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:14:06 smithi148 ceph-mon[106692]: from='client.34220 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:14:07.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:14:06 smithi148 ceph-mon[106692]: from='client.34224 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:14:07.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:14:06 smithi148 ceph-mon[106692]: pgmap v66: 65 pgs: 65 active+clean; 259 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:14:07.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:14:06 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/2119375618' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:14:07.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:14:06 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/4246950529' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T08:14:07.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:07.085+0000 7f8bb3000700 1 -- 172.21.15.37:0/3097072663 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8bac1037a0 msgr2=0x7f8bac0fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:14:07.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:07.085+0000 7f8bb3000700 1 --2- 172.21.15.37:0/3097072663 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8bac1037a0 0x7f8bac0fce10 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f8b98004940 tx=0x7f8b98009e30 comp rx=0 tx=0).stop 2024-03-29T08:14:07.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:07.085+0000 7f8bb3000700 1 -- 172.21.15.37:0/3097072663 shutdown_connections 2024-03-29T08:14:07.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:07.085+0000 7f8bb3000700 1 --2- 172.21.15.37:0/3097072663 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8bac1037a0 0x7f8bac0fce10 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:14:07.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:07.085+0000 7f8bb3000700 1 --2- 172.21.15.37:0/3097072663 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8bac102e00 0x7f8bac1031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:14:07.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:07.085+0000 7f8bb3000700 1 -- 172.21.15.37:0/3097072663 >> 172.21.15.37:0/3097072663 conn(0x7f8bac0f89b0 msgr2=0x7f8bac0fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:14:07.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:07.085+0000 7f8bb3000700 1 -- 172.21.15.37:0/3097072663 shutdown_connections 2024-03-29T08:14:07.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:07.085+0000 7f8bb3000700 1 -- 172.21.15.37:0/3097072663 wait complete. 2024-03-29T08:14:07.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:07.086+0000 7f8bb3000700 1 Processor -- start 2024-03-29T08:14:07.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:07.086+0000 7f8bb3000700 1 -- start start 2024-03-29T08:14:07.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:07.086+0000 7f8bb3000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8bac102e00 0x7f8bac115d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:14:07.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:07.086+0000 7f8bb3000700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8bac1037a0 0x7f8bac116260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:14:07.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:07.086+0000 7f8bb3000700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bac1168b0 con 0x7f8bac102e00 2024-03-29T08:14:07.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:07.086+0000 7f8bb3000700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bac10fc80 con 0x7f8bac1037a0 2024-03-29T08:14:07.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:07.087+0000 7f8babe00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8bac102e00 0x7f8bac115d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:14:07.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:07.087+0000 7f8babe00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8bac102e00 0x7f8bac115d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53510/0 (socket says 172.21.15.37:53510) 2024-03-29T08:14:07.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:07.087+0000 7f8babe00700 1 -- 172.21.15.37:0/687797564 learned_addr learned my addr 172.21.15.37:0/687797564 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:14:07.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:07.087+0000 7f8bab400700 1 --2- 172.21.15.37:0/687797564 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8bac1037a0 0x7f8bac116260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:14:07.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:07.087+0000 7f8babe00700 1 -- 172.21.15.37:0/687797564 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8bac1037a0 msgr2=0x7f8bac116260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:14:07.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:07.087+0000 7f8babe00700 1 --2- 172.21.15.37:0/687797564 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8bac1037a0 0x7f8bac116260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:14:07.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:07.087+0000 7f8babe00700 1 -- 172.21.15.37:0/687797564 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b98005040 con 0x7f8bac102e00 2024-03-29T08:14:07.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:07.088+0000 7f8babe00700 1 --2- 172.21.15.37:0/687797564 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8bac102e00 0x7f8bac115d20 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f8b9c002a50 tx=0x7f8b9c0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:14:07.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:07.088+0000 7f8ba3e00700 1 -- 172.21.15.37:0/687797564 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8b9c007cc0 con 0x7f8bac102e00 2024-03-29T08:14:07.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:07.088+0000 7f8ba3e00700 1 -- 172.21.15.37:0/687797564 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f8b9c00f040 con 0x7f8bac102e00 2024-03-29T08:14:07.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:07.088+0000 7f8ba3e00700 1 -- 172.21.15.37:0/687797564 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8b9c00d9c0 con 0x7f8bac102e00 2024-03-29T08:14:07.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:07.088+0000 7f8bb3000700 1 -- 172.21.15.37:0/687797564 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bac10ff60 con 0x7f8bac102e00 2024-03-29T08:14:07.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:07.088+0000 7f8bb3000700 1 -- 172.21.15.37:0/687797564 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bac110520 con 0x7f8bac102e00 2024-03-29T08:14:07.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:07.089+0000 7f8bb3000700 1 -- 172.21.15.37:0/687797564 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bac0fe550 con 0x7f8bac102e00 2024-03-29T08:14:07.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:07.090+0000 7f8ba3e00700 1 -- 172.21.15.37:0/687797564 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f8b9c00db20 con 0x7f8bac102e00 2024-03-29T08:14:07.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:07.091+0000 7f8ba3e00700 1 --2- 172.21.15.37:0/687797564 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f8b9006cc30 0x7f8b9006f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:14:07.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:07.091+0000 7f8ba3e00700 1 -- 172.21.15.37:0/687797564 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f8b9c08bca0 con 0x7f8bac102e00 2024-03-29T08:14:07.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:07.091+0000 7f8bab400700 1 --2- 172.21.15.37:0/687797564 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f8b9006cc30 0x7f8b9006f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:14:07.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:07.092+0000 7f8bab400700 1 --2- 172.21.15.37:0/687797564 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f8b9006cc30 0x7f8b9006f0e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f8bac1115c0 tx=0x7f8b98009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:14:07.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:07.095+0000 7f8ba3e00700 1 -- 172.21.15.37:0/687797564 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f8b9c059770 con 0x7f8bac102e00 2024-03-29T08:14:07.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:07.389+0000 7f8bb3000700 1 -- 172.21.15.37:0/687797564 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f8bac04bed0 con 0x7f8bac102e00 2024-03-29T08:14:07.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:07.389+0000 7f8ba3e00700 1 -- 172.21.15.37:0/687797564 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f8b9c059300 con 0x7f8bac102e00 2024-03-29T08:14:07.390 INFO:teuthology.orchestra.run.smithi037.stdout:HEALTH_OK 2024-03-29T08:14:07.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:07.392+0000 7f8bb3000700 1 -- 172.21.15.37:0/687797564 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f8b9006cc30 msgr2=0x7f8b9006f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:14:07.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:07.392+0000 7f8bb3000700 1 --2- 172.21.15.37:0/687797564 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f8b9006cc30 0x7f8b9006f0e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f8bac1115c0 tx=0x7f8b98009e30 comp rx=0 tx=0).stop 2024-03-29T08:14:07.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:07.392+0000 7f8bb3000700 1 -- 172.21.15.37:0/687797564 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8bac102e00 msgr2=0x7f8bac115d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:14:07.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:07.392+0000 7f8bb3000700 1 --2- 172.21.15.37:0/687797564 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8bac102e00 0x7f8bac115d20 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f8b9c002a50 tx=0x7f8b9c0046b0 comp rx=0 tx=0).stop 2024-03-29T08:14:07.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:07.392+0000 7f8bb3000700 1 -- 172.21.15.37:0/687797564 shutdown_connections 2024-03-29T08:14:07.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:07.392+0000 7f8bb3000700 1 --2- 172.21.15.37:0/687797564 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f8b9006cc30 0x7f8b9006f0e0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:14:07.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:07.392+0000 7f8bb3000700 1 --2- 172.21.15.37:0/687797564 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8bac102e00 0x7f8bac115d20 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:14:07.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:07.392+0000 7f8bb3000700 1 --2- 172.21.15.37:0/687797564 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8bac1037a0 0x7f8bac116260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:14:07.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:07.392+0000 7f8bb3000700 1 -- 172.21.15.37:0/687797564 >> 172.21.15.37:0/687797564 conn(0x7f8bac0f89b0 msgr2=0x7f8bac0f99f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:14:07.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:07.392+0000 7f8bb3000700 1 -- 172.21.15.37:0/687797564 shutdown_connections 2024-03-29T08:14:07.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:07.392+0000 7f8bb3000700 1 -- 172.21.15.37:0/687797564 wait complete. 2024-03-29T08:14:07.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:14:07 smithi037 ceph-mon[142325]: from='client.34228 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:14:07.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:14:07 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/687797564' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T08:14:08.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:14:07 smithi148 ceph-mon[106692]: from='client.34228 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:14:08.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:14:07 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/687797564' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T08:14:08.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:14:08 smithi037 ceph-mon[142325]: from='client.34240 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:14:08.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:14:08 smithi037 ceph-mon[142325]: pgmap v67: 65 pgs: 65 active+clean; 259 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:14:09.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:14:08 smithi148 ceph-mon[106692]: from='client.34240 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:14:09.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:14:08 smithi148 ceph-mon[106692]: pgmap v67: 65 pgs: 65 active+clean; 259 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:14:10.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:14:10 smithi037 ceph-mon[142325]: pgmap v68: 65 pgs: 65 active+clean; 259 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:14:10.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:14:10 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:14:10.928 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:14:10 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:14:10.928 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:14:10 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:14:11.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:14:10 smithi148 ceph-mon[106692]: pgmap v68: 65 pgs: 65 active+clean; 259 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:14:11.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:14:10 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:14:11.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:14:10 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:14:11.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:14:10 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:14:12.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:14:12 smithi037 ceph-mon[142325]: pgmap v69: 65 pgs: 65 active+clean; 259 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:14:13.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:14:12 smithi148 ceph-mon[106692]: pgmap v69: 65 pgs: 65 active+clean; 259 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:14:14.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:14:14 smithi037 ceph-mon[142325]: pgmap v70: 65 pgs: 65 active+clean; 259 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:14:15.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:14:14 smithi148 ceph-mon[106692]: pgmap v70: 65 pgs: 65 active+clean; 259 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:14:15.870 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:14:15 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-03-29T08:14:16.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:14:15 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-03-29T08:14:16.803 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:14:16 smithi037 ceph-mon[142325]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-03-29T08:14:16.803 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:14:16 smithi037 ceph-mon[142325]: Upgrade: osd.1 is safe to restart 2024-03-29T08:14:16.803 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:14:16 smithi037 ceph-mon[142325]: pgmap v71: 65 pgs: 65 active+clean; 259 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:14:17.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:14:16 smithi148 ceph-mon[106692]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-03-29T08:14:17.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:14:16 smithi148 ceph-mon[106692]: Upgrade: osd.1 is safe to restart 2024-03-29T08:14:17.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:14:16 smithi148 ceph-mon[106692]: pgmap v71: 65 pgs: 65 active+clean; 259 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:14:18.178 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:14:17 smithi037 ceph-mon[142325]: Upgrade: Updating osd.1 2024-03-29T08:14:18.178 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:14:17 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:14:18.178 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:14:17 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-29T08:14:18.178 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:14:17 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T08:14:18.178 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:14:17 smithi037 ceph-mon[142325]: Deploying daemon osd.1 on smithi037 2024-03-29T08:14:18.178 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:14:17 smithi037 ceph-mon[142325]: pgmap v72: 65 pgs: 65 active+clean; 259 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:14:18.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:14:17 smithi148 ceph-mon[106692]: Upgrade: Updating osd.1 2024-03-29T08:14:18.307 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:14:17 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:14:18.307 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:14:17 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-29T08:14:18.307 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:14:17 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T08:14:18.307 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:14:17 smithi148 ceph-mon[106692]: Deploying daemon osd.1 on smithi037 2024-03-29T08:14:18.307 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:14:17 smithi148 ceph-mon[106692]: pgmap v72: 65 pgs: 65 active+clean; 259 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:14:20.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:14:20 smithi037 ceph-mon[142325]: pgmap v73: 65 pgs: 65 active+clean; 259 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:14:20.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:14:20 smithi148 ceph-mon[106692]: pgmap v73: 65 pgs: 65 active+clean; 259 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:14:21.069 INFO:journalctl@ceph.osd.1.smithi037.stdout:Mar 29 08:14:20 smithi037 systemd[1]: Stopping Ceph osd.1 for 99b492c2-eda1-11ee-b647-cb9ed24678a4... 2024-03-29T08:14:21.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:14:21 smithi037 ceph-mon[142325]: osd.1 marked itself down and dead 2024-03-29T08:14:21.428 INFO:journalctl@ceph.osd.1.smithi037.stdout:Mar 29 08:14:21 smithi037 ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4-osd-1[67815]: 2024-03-29T08:14:21.068+0000 7fca3b800700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-29T08:14:21.428 INFO:journalctl@ceph.osd.1.smithi037.stdout:Mar 29 08:14:21 smithi037 ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4-osd-1[67815]: 2024-03-29T08:14:21.068+0000 7fca3b800700 -1 osd.1 55 *** Got signal Terminated *** 2024-03-29T08:14:21.428 INFO:journalctl@ceph.osd.1.smithi037.stdout:Mar 29 08:14:21 smithi037 ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4-osd-1[67815]: 2024-03-29T08:14:21.068+0000 7fca3b800700 -1 osd.1 55 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-29T08:14:21.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:14:21 smithi148 ceph-mon[106692]: osd.1 marked itself down and dead 2024-03-29T08:14:22.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:14:22 smithi037 ceph-mon[142325]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-29T08:14:22.178 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:14:22 smithi037 ceph-mon[142325]: osdmap e56: 6 total, 5 up, 6 in 2024-03-29T08:14:22.178 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:14:22 smithi037 ceph-mon[142325]: pgmap v75: 65 pgs: 12 stale+active+clean, 53 active+clean; 259 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:14:22.178 INFO:journalctl@ceph.osd.1.smithi037.stdout:Mar 29 08:14:22 smithi037 bash[164040]: ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4-osd-1 2024-03-29T08:14:22.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:14:22 smithi148 ceph-mon[106692]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-29T08:14:22.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:14:22 smithi148 ceph-mon[106692]: osdmap e56: 6 total, 5 up, 6 in 2024-03-29T08:14:22.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:14:22 smithi148 ceph-mon[106692]: pgmap v75: 65 pgs: 12 stale+active+clean, 53 active+clean; 259 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:14:23.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:14:23 smithi037 ceph-mon[142325]: osdmap e57: 6 total, 5 up, 6 in 2024-03-29T08:14:23.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:14:23 smithi148 ceph-mon[106692]: osdmap e57: 6 total, 5 up, 6 in 2024-03-29T08:14:23.965 INFO:journalctl@ceph.osd.1.smithi037.stdout:Mar 29 08:14:23 smithi037 systemd[1]: ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4@osd.1.service: Succeeded. 2024-03-29T08:14:23.965 INFO:journalctl@ceph.osd.1.smithi037.stdout:Mar 29 08:14:23 smithi037 systemd[1]: Stopped Ceph osd.1 for 99b492c2-eda1-11ee-b647-cb9ed24678a4. 2024-03-29T08:14:24.257 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:14:24 smithi037 ceph-mon[142325]: pgmap v77: 65 pgs: 12 stale+active+clean, 53 active+clean; 259 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:14:24.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:14:24 smithi148 ceph-mon[106692]: pgmap v77: 65 pgs: 12 stale+active+clean, 53 active+clean; 259 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:14:24.677 INFO:journalctl@ceph.osd.1.smithi037.stdout:Mar 29 08:14:24 smithi037 systemd[1]: Starting Ceph osd.1 for 99b492c2-eda1-11ee-b647-cb9ed24678a4... 2024-03-29T08:14:25.928 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:14:25 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:14:25.928 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:14:25 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:14:25.929 INFO:journalctl@ceph.osd.1.smithi037.stdout:Mar 29 08:14:25 smithi037 bash[164307]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-03-29T08:14:25.929 INFO:journalctl@ceph.osd.1.smithi037.stdout:Mar 29 08:14:25 smithi037 bash[164307]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-1 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-03-29T08:14:25.929 INFO:journalctl@ceph.osd.1.smithi037.stdout:Mar 29 08:14:25 smithi037 bash[164307]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-03-29T08:14:25.930 INFO:journalctl@ceph.osd.1.smithi037.stdout:Mar 29 08:14:25 smithi037 bash[164307]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-03-29T08:14:25.930 INFO:journalctl@ceph.osd.1.smithi037.stdout:Mar 29 08:14:25 smithi037 bash[164307]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-1/block 2024-03-29T08:14:25.930 INFO:journalctl@ceph.osd.1.smithi037.stdout:Mar 29 08:14:25 smithi037 bash[164307]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-03-29T08:14:25.930 INFO:journalctl@ceph.osd.1.smithi037.stdout:Mar 29 08:14:25 smithi037 bash[164307]: --> ceph-volume raw activate successful for osd ID: 1 2024-03-29T08:14:26.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:14:25 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:14:26.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:14:25 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:14:26.900 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:14:26 smithi037 ceph-mon[142325]: pgmap v78: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 259 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 51/324 objects degraded (15.741%) 2024-03-29T08:14:26.900 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:14:26 smithi037 ceph-mon[142325]: Health check failed: Degraded data redundancy: 51/324 objects degraded (15.741%), 16 pgs degraded (PG_DEGRADED) 2024-03-29T08:14:27.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:14:26 smithi148 ceph-mon[106692]: pgmap v78: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 259 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 51/324 objects degraded (15.741%) 2024-03-29T08:14:27.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:14:26 smithi148 ceph-mon[106692]: Health check failed: Degraded data redundancy: 51/324 objects degraded (15.741%), 16 pgs degraded (PG_DEGRADED) 2024-03-29T08:14:27.177 INFO:journalctl@ceph.osd.1.smithi037.stdout:Mar 29 08:14:26 smithi037 ceph-osd[164599]: -- 172.21.15.37:0/2761904024 <== mon.0 v2:172.21.15.37:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55671f4071e0 con 0x55671f3d7000 2024-03-29T08:14:27.177 INFO:journalctl@ceph.osd.1.smithi037.stdout:Mar 29 08:14:26 smithi037 bash[164562]: a4a2ff4cc83d362d15b3ae4e6c288c6d8ac8543b1e693d9b8d832c30fc792e83 2024-03-29T08:14:27.177 INFO:journalctl@ceph.osd.1.smithi037.stdout:Mar 29 08:14:26 smithi037 systemd[1]: Started Ceph osd.1 for 99b492c2-eda1-11ee-b647-cb9ed24678a4. 2024-03-29T08:14:27.927 INFO:journalctl@ceph.osd.1.smithi037.stdout:Mar 29 08:14:27 smithi037 ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4-osd-1[164576]: 2024-03-29T08:14:27.432+0000 7f1ad9633640 -1 Falling back to public interface 2024-03-29T08:14:28.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:14:28 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:14:28.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:14:28 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:14:28.307 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:14:28 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T08:14:28.307 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:14:28 smithi148 ceph-mon[106692]: pgmap v79: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 259 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 51/324 objects degraded (15.741%) 2024-03-29T08:14:28.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:14:28 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:14:28.428 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:14:28 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:14:28.428 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:14:28 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T08:14:28.428 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:14:28 smithi037 ceph-mon[142325]: pgmap v79: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 259 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 51/324 objects degraded (15.741%) 2024-03-29T08:14:30.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:14:30 smithi037 ceph-mon[142325]: pgmap v80: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 259 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 51/324 objects degraded (15.741%) 2024-03-29T08:14:30.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:14:30 smithi148 ceph-mon[106692]: pgmap v80: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 259 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 51/324 objects degraded (15.741%) 2024-03-29T08:14:32.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:14:32 smithi037 ceph-mon[142325]: pgmap v81: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 259 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 51/324 objects degraded (15.741%) 2024-03-29T08:14:33.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:14:32 smithi148 ceph-mon[106692]: pgmap v81: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 259 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 51/324 objects degraded (15.741%) 2024-03-29T08:14:33.427 INFO:journalctl@ceph.osd.1.smithi037.stdout:Mar 29 08:14:33 smithi037 ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4-osd-1[164576]: 2024-03-29T08:14:33.202+0000 7f1ad9633640 -1 osd.1 55 log_to_monitors true 2024-03-29T08:14:34.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:14:34 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:14:34.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:14:34 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:14:34.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:14:34 smithi037 ceph-mon[142325]: from='osd.1 [v2:172.21.15.37:6810/2942536992,v1:172.21.15.37:6811/2942536992]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-29T08:14:34.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:14:34 smithi037 ceph-mon[142325]: pgmap v82: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 259 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 51/324 objects degraded (15.741%) 2024-03-29T08:14:34.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:14:34 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:14:34.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:14:34 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:14:34.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:14:34 smithi148 ceph-mon[106692]: from='osd.1 [v2:172.21.15.37:6810/2942536992,v1:172.21.15.37:6811/2942536992]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-29T08:14:34.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:14:34 smithi148 ceph-mon[106692]: pgmap v82: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 259 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 51/324 objects degraded (15.741%) 2024-03-29T08:14:35.427 INFO:journalctl@ceph.osd.1.smithi037.stdout:Mar 29 08:14:35 smithi037 ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4-osd-1[164576]: 2024-03-29T08:14:35.216+0000 7f1ac3a00700 -1 osd.1 55 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-29T08:14:35.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:14:35 smithi037 ceph-mon[142325]: from='osd.1 [v2:172.21.15.37:6810/2942536992,v1:172.21.15.37:6811/2942536992]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-29T08:14:35.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:14:35 smithi037 ceph-mon[142325]: osdmap e58: 6 total, 5 up, 6 in 2024-03-29T08:14:35.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:14:35 smithi037 ceph-mon[142325]: from='osd.1 [v2:172.21.15.37:6810/2942536992,v1:172.21.15.37:6811/2942536992]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]: dispatch 2024-03-29T08:14:35.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:14:35 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:14:35.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:14:35 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:14:35.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:14:35 smithi148 ceph-mon[106692]: from='osd.1 [v2:172.21.15.37:6810/2942536992,v1:172.21.15.37:6811/2942536992]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-29T08:14:35.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:14:35 smithi148 ceph-mon[106692]: osdmap e58: 6 total, 5 up, 6 in 2024-03-29T08:14:35.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:14:35 smithi148 ceph-mon[106692]: from='osd.1 [v2:172.21.15.37:6810/2942536992,v1:172.21.15.37:6811/2942536992]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]: dispatch 2024-03-29T08:14:35.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:14:35 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:14:35.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:14:35 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:14:36.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:14:36 smithi148 ceph-mon[106692]: from='osd.1 [v2:172.21.15.37:6810/2942536992,v1:172.21.15.37:6811/2942536992]' entity='osd.1' 2024-03-29T08:14:36.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:14:36 smithi148 ceph-mon[106692]: pgmap v84: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 259 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 51/324 objects degraded (15.741%) 2024-03-29T08:14:36.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:14:36 smithi037 ceph-mon[142325]: from='osd.1 [v2:172.21.15.37:6810/2942536992,v1:172.21.15.37:6811/2942536992]' entity='osd.1' 2024-03-29T08:14:36.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:14:36 smithi037 ceph-mon[142325]: pgmap v84: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 259 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 51/324 objects degraded (15.741%) 2024-03-29T08:14:37.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:37.538+0000 7f44e5a00700 1 -- 172.21.15.37:0/3406437011 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f44e0070c70 msgr2=0x7f44e0109110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:14:37.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:37.538+0000 7f44e5a00700 1 --2- 172.21.15.37:0/3406437011 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f44e0070c70 0x7f44e0109110 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f44cc004970 tx=0x7f44cc009e30 comp rx=0 tx=0).stop 2024-03-29T08:14:37.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:37.538+0000 7f44e5a00700 1 -- 172.21.15.37:0/3406437011 shutdown_connections 2024-03-29T08:14:37.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:37.538+0000 7f44e5a00700 1 --2- 172.21.15.37:0/3406437011 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f44e0070c70 0x7f44e0109110 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:14:37.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:37.538+0000 7f44e5a00700 1 --2- 172.21.15.37:0/3406437011 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f44e0070360 0x7f44e0070730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:14:37.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:37.538+0000 7f44e5a00700 1 -- 172.21.15.37:0/3406437011 >> 172.21.15.37:0/3406437011 conn(0x7f44e00f89b0 msgr2=0x7f44e00fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:14:37.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:37.539+0000 7f44e5a00700 1 -- 172.21.15.37:0/3406437011 shutdown_connections 2024-03-29T08:14:37.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:37.539+0000 7f44e5a00700 1 -- 172.21.15.37:0/3406437011 wait complete. 2024-03-29T08:14:37.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:37.539+0000 7f44e5a00700 1 Processor -- start 2024-03-29T08:14:37.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:37.540+0000 7f44e5a00700 1 -- start start 2024-03-29T08:14:37.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:37.540+0000 7f44e5a00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f44e0070360 0x7f44e0115ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:14:37.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:37.540+0000 7f44e5a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f44e0070c70 0x7f44e01161e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:14:37.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:37.540+0000 7f44e5a00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44e0116830 con 0x7f44e0070c70 2024-03-29T08:14:37.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:37.540+0000 7f44e5a00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44e010fc00 con 0x7f44e0070360 2024-03-29T08:14:37.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:37.540+0000 7f44de000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f44e0070c70 0x7f44e01161e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:14:37.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:37.540+0000 7f44de000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f44e0070c70 0x7f44e01161e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39622/0 (socket says 172.21.15.37:39622) 2024-03-29T08:14:37.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:37.540+0000 7f44de000700 1 -- 172.21.15.37:0/2553139818 learned_addr learned my addr 172.21.15.37:0/2553139818 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:14:37.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:37.540+0000 7f44dea00700 1 --2- 172.21.15.37:0/2553139818 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f44e0070360 0x7f44e0115ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:14:37.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:37.541+0000 7f44de000700 1 -- 172.21.15.37:0/2553139818 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f44e0070360 msgr2=0x7f44e0115ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:14:37.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:37.541+0000 7f44de000700 1 --2- 172.21.15.37:0/2553139818 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f44e0070360 0x7f44e0115ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:14:37.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:37.541+0000 7f44de000700 1 -- 172.21.15.37:0/2553139818 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44cc005040 con 0x7f44e0070c70 2024-03-29T08:14:37.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:37.541+0000 7f44de000700 1 --2- 172.21.15.37:0/2553139818 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f44e0070c70 0x7f44e01161e0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f44cc004f50 tx=0x7f44cc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:14:37.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:37.541+0000 7f44d6a00700 1 -- 172.21.15.37:0/2553139818 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f44cc00e040 con 0x7f44e0070c70 2024-03-29T08:14:37.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:37.541+0000 7f44d6a00700 1 -- 172.21.15.37:0/2553139818 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f44cc013420 con 0x7f44e0070c70 2024-03-29T08:14:37.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:37.541+0000 7f44e5a00700 1 -- 172.21.15.37:0/2553139818 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44e010fe80 con 0x7f44e0070c70 2024-03-29T08:14:37.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:37.542+0000 7f44d6a00700 1 -- 172.21.15.37:0/2553139818 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f44cc01c630 con 0x7f44e0070c70 2024-03-29T08:14:37.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:37.542+0000 7f44e5a00700 1 -- 172.21.15.37:0/2553139818 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44e01103c0 con 0x7f44e0070c70 2024-03-29T08:14:37.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:37.543+0000 7f44e5a00700 1 -- 172.21.15.37:0/2553139818 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44e0103bd0 con 0x7f44e0070c70 2024-03-29T08:14:37.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:37.544+0000 7f44d6a00700 1 -- 172.21.15.37:0/2553139818 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f44cc013590 con 0x7f44e0070c70 2024-03-29T08:14:37.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:37.545+0000 7f44d6a00700 1 --2- 172.21.15.37:0/2553139818 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f44c406cc30 0x7f44c406f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:14:37.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:37.546+0000 7f44d6a00700 1 -- 172.21.15.37:0/2553139818 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f44cc08d140 con 0x7f44e0070c70 2024-03-29T08:14:37.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:37.546+0000 7f44dea00700 1 --2- 172.21.15.37:0/2553139818 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f44c406cc30 0x7f44c406f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:14:37.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:37.546+0000 7f44dea00700 1 --2- 172.21.15.37:0/2553139818 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f44c406cc30 0x7f44c406f0e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f44d0004f40 tx=0x7f44d000b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:14:37.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:37.550+0000 7f44d6a00700 1 -- 172.21.15.37:0/2553139818 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f44cc05ac90 con 0x7f44e0070c70 2024-03-29T08:14:37.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:14:37 smithi148 ceph-mon[106692]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-29T08:14:37.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:14:37 smithi148 ceph-mon[106692]: osd.1 [v2:172.21.15.37:6810/2942536992,v1:172.21.15.37:6811/2942536992] boot 2024-03-29T08:14:37.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:14:37 smithi148 ceph-mon[106692]: osdmap e59: 6 total, 6 up, 6 in 2024-03-29T08:14:37.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:14:37 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-29T08:14:37.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:14:37 smithi148 ceph-mon[106692]: osdmap e60: 6 total, 6 up, 6 in 2024-03-29T08:14:37.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:14:37 smithi037 ceph-mon[142325]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-29T08:14:37.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:14:37 smithi037 ceph-mon[142325]: osd.1 [v2:172.21.15.37:6810/2942536992,v1:172.21.15.37:6811/2942536992] boot 2024-03-29T08:14:37.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:14:37 smithi037 ceph-mon[142325]: osdmap e59: 6 total, 6 up, 6 in 2024-03-29T08:14:37.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:14:37 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-29T08:14:37.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:14:37 smithi037 ceph-mon[142325]: osdmap e60: 6 total, 6 up, 6 in 2024-03-29T08:14:37.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:37.786+0000 7f44e5a00700 1 -- 172.21.15.37:0/2553139818 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f44e0111390 con 0x7f44c406cc30 2024-03-29T08:14:37.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:37.788+0000 7f44d6a00700 1 -- 172.21.15.37:0/2553139818 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f44e0111390 con 0x7f44c406cc30 2024-03-29T08:14:37.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:37.790+0000 7f44e5a00700 1 -- 172.21.15.37:0/2553139818 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f44c406cc30 msgr2=0x7f44c406f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:14:37.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:37.790+0000 7f44e5a00700 1 --2- 172.21.15.37:0/2553139818 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f44c406cc30 0x7f44c406f0e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f44d0004f40 tx=0x7f44d000b040 comp rx=0 tx=0).stop 2024-03-29T08:14:37.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:37.790+0000 7f44e5a00700 1 -- 172.21.15.37:0/2553139818 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f44e0070c70 msgr2=0x7f44e01161e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:14:37.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:37.790+0000 7f44e5a00700 1 --2- 172.21.15.37:0/2553139818 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f44e0070c70 0x7f44e01161e0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f44cc004f50 tx=0x7f44cc009e30 comp rx=0 tx=0).stop 2024-03-29T08:14:37.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:37.790+0000 7f44e5a00700 1 -- 172.21.15.37:0/2553139818 shutdown_connections 2024-03-29T08:14:37.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:37.790+0000 7f44e5a00700 1 --2- 172.21.15.37:0/2553139818 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f44c406cc30 0x7f44c406f0e0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:14:37.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:37.790+0000 7f44e5a00700 1 --2- 172.21.15.37:0/2553139818 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f44e0070360 0x7f44e0115ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:14:37.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:37.790+0000 7f44e5a00700 1 --2- 172.21.15.37:0/2553139818 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f44e0070c70 0x7f44e01161e0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:14:37.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:37.790+0000 7f44e5a00700 1 -- 172.21.15.37:0/2553139818 >> 172.21.15.37:0/2553139818 conn(0x7f44e00f89b0 msgr2=0x7f44e01015f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:14:37.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:37.790+0000 7f44e5a00700 1 -- 172.21.15.37:0/2553139818 shutdown_connections 2024-03-29T08:14:37.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:37.791+0000 7f44e5a00700 1 -- 172.21.15.37:0/2553139818 wait complete. 2024-03-29T08:14:37.806 INFO:teuthology.orchestra.run.smithi037.stdout:true 2024-03-29T08:14:37.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:37.931+0000 7f2c5bc00700 1 -- 172.21.15.37:0/841299286 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2c54102e00 msgr2=0x7f2c541031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:14:37.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:37.931+0000 7f2c5bc00700 1 --2- 172.21.15.37:0/841299286 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2c54102e00 0x7f2c541031d0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f2c48004940 tx=0x7f2c48009e30 comp rx=0 tx=0).stop 2024-03-29T08:14:37.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:37.931+0000 7f2c5bc00700 1 -- 172.21.15.37:0/841299286 shutdown_connections 2024-03-29T08:14:37.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:37.931+0000 7f2c5bc00700 1 --2- 172.21.15.37:0/841299286 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2c541037a0 0x7f2c540fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:14:37.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:37.931+0000 7f2c5bc00700 1 --2- 172.21.15.37:0/841299286 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2c54102e00 0x7f2c541031d0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:14:37.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:37.931+0000 7f2c5bc00700 1 -- 172.21.15.37:0/841299286 >> 172.21.15.37:0/841299286 conn(0x7f2c540f89b0 msgr2=0x7f2c540fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:14:37.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:37.932+0000 7f2c5bc00700 1 -- 172.21.15.37:0/841299286 shutdown_connections 2024-03-29T08:14:37.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:37.932+0000 7f2c5bc00700 1 -- 172.21.15.37:0/841299286 wait complete. 2024-03-29T08:14:37.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:37.933+0000 7f2c5bc00700 1 Processor -- start 2024-03-29T08:14:37.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:37.933+0000 7f2c5bc00700 1 -- start start 2024-03-29T08:14:37.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:37.934+0000 7f2c5bc00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2c54102e00 0x7f2c54113820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:14:37.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:37.934+0000 7f2c5bc00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2c541037a0 0x7f2c54113d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:14:37.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:37.934+0000 7f2c5bc00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c541143b0 con 0x7f2c54102e00 2024-03-29T08:14:37.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:37.934+0000 7f2c5bc00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c5410d9b0 con 0x7f2c541037a0 2024-03-29T08:14:37.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:37.934+0000 7f2c59200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2c54102e00 0x7f2c54113820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:14:37.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:37.934+0000 7f2c59200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2c54102e00 0x7f2c54113820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39634/0 (socket says 172.21.15.37:39634) 2024-03-29T08:14:37.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:37.934+0000 7f2c59200700 1 -- 172.21.15.37:0/3327033634 learned_addr learned my addr 172.21.15.37:0/3327033634 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:14:37.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:37.934+0000 7f2c53e00700 1 --2- 172.21.15.37:0/3327033634 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2c541037a0 0x7f2c54113d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:14:37.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:37.934+0000 7f2c59200700 1 -- 172.21.15.37:0/3327033634 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2c541037a0 msgr2=0x7f2c54113d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:14:37.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:37.934+0000 7f2c59200700 1 --2- 172.21.15.37:0/3327033634 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2c541037a0 0x7f2c54113d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:14:37.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:37.934+0000 7f2c59200700 1 -- 172.21.15.37:0/3327033634 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c48005040 con 0x7f2c54102e00 2024-03-29T08:14:37.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:37.935+0000 7f2c59200700 1 --2- 172.21.15.37:0/3327033634 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2c54102e00 0x7f2c54113820 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f2c48007f50 tx=0x7f2c48009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:14:37.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:37.935+0000 7f2c50c00700 1 -- 172.21.15.37:0/3327033634 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2c48021070 con 0x7f2c54102e00 2024-03-29T08:14:37.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:37.935+0000 7f2c50c00700 1 -- 172.21.15.37:0/3327033634 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f2c480093b0 con 0x7f2c54102e00 2024-03-29T08:14:37.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:37.935+0000 7f2c50c00700 1 -- 172.21.15.37:0/3327033634 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2c4800e070 con 0x7f2c54102e00 2024-03-29T08:14:37.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:37.935+0000 7f2c5bc00700 1 -- 172.21.15.37:0/3327033634 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c5410dc30 con 0x7f2c54102e00 2024-03-29T08:14:37.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:37.935+0000 7f2c5bc00700 1 -- 172.21.15.37:0/3327033634 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c5410e190 con 0x7f2c54102e00 2024-03-29T08:14:37.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:37.936+0000 7f2c5bc00700 1 -- 172.21.15.37:0/3327033634 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c540fe550 con 0x7f2c54102e00 2024-03-29T08:14:37.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:37.937+0000 7f2c50c00700 1 -- 172.21.15.37:0/3327033634 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f2c4801c400 con 0x7f2c54102e00 2024-03-29T08:14:37.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:37.938+0000 7f2c50c00700 1 --2- 172.21.15.37:0/3327033634 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f2c3c06cc20 0x7f2c3c06f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:14:37.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:37.938+0000 7f2c50c00700 1 -- 172.21.15.37:0/3327033634 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f2c4808d0a0 con 0x7f2c54102e00 2024-03-29T08:14:37.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:37.938+0000 7f2c53e00700 1 --2- 172.21.15.37:0/3327033634 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f2c3c06cc20 0x7f2c3c06f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:14:37.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:37.939+0000 7f2c53e00700 1 --2- 172.21.15.37:0/3327033634 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f2c3c06cc20 0x7f2c3c06f0d0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f2c5410eb70 tx=0x7f2c4000b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:14:37.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:37.943+0000 7f2c50c00700 1 -- 172.21.15.37:0/3327033634 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f2c4805abf0 con 0x7f2c54102e00 2024-03-29T08:14:38.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:38.183+0000 7f2c5bc00700 1 -- 172.21.15.37:0/3327033634 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f2c54109e30 con 0x7f2c3c06cc20 2024-03-29T08:14:38.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:38.184+0000 7f2c50c00700 1 -- 172.21.15.37:0/3327033634 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f2c54109e30 con 0x7f2c3c06cc20 2024-03-29T08:14:38.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:38.187+0000 7f2c5bc00700 1 -- 172.21.15.37:0/3327033634 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f2c3c06cc20 msgr2=0x7f2c3c06f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:14:38.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:38.187+0000 7f2c5bc00700 1 --2- 172.21.15.37:0/3327033634 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f2c3c06cc20 0x7f2c3c06f0d0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f2c5410eb70 tx=0x7f2c4000b040 comp rx=0 tx=0).stop 2024-03-29T08:14:38.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:38.187+0000 7f2c5bc00700 1 -- 172.21.15.37:0/3327033634 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2c54102e00 msgr2=0x7f2c54113820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:14:38.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:38.187+0000 7f2c5bc00700 1 --2- 172.21.15.37:0/3327033634 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2c54102e00 0x7f2c54113820 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f2c48007f50 tx=0x7f2c48009e30 comp rx=0 tx=0).stop 2024-03-29T08:14:38.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:38.187+0000 7f2c5bc00700 1 -- 172.21.15.37:0/3327033634 shutdown_connections 2024-03-29T08:14:38.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:38.187+0000 7f2c5bc00700 1 --2- 172.21.15.37:0/3327033634 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f2c3c06cc20 0x7f2c3c06f0d0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:14:38.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:38.187+0000 7f2c5bc00700 1 --2- 172.21.15.37:0/3327033634 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2c54102e00 0x7f2c54113820 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:14:38.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:38.187+0000 7f2c5bc00700 1 --2- 172.21.15.37:0/3327033634 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2c541037a0 0x7f2c54113d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:14:38.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:38.187+0000 7f2c5bc00700 1 -- 172.21.15.37:0/3327033634 >> 172.21.15.37:0/3327033634 conn(0x7f2c540f89b0 msgr2=0x7f2c540f9980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:14:38.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:38.187+0000 7f2c5bc00700 1 -- 172.21.15.37:0/3327033634 shutdown_connections 2024-03-29T08:14:38.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:38.187+0000 7f2c5bc00700 1 -- 172.21.15.37:0/3327033634 wait complete. 2024-03-29T08:14:38.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:38.352+0000 7f528d800700 1 -- 172.21.15.37:0/53469527 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f52881059b0 msgr2=0x7f52880ff070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:14:38.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:38.352+0000 7f528d800700 1 --2- 172.21.15.37:0/53469527 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f52881059b0 0x7f52880ff070 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f5274004940 tx=0x7f5274009e30 comp rx=0 tx=0).stop 2024-03-29T08:14:38.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:38.352+0000 7f528d800700 1 -- 172.21.15.37:0/53469527 shutdown_connections 2024-03-29T08:14:38.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:38.352+0000 7f528d800700 1 --2- 172.21.15.37:0/53469527 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f52881059b0 0x7f52880ff070 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:14:38.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:38.352+0000 7f528d800700 1 --2- 172.21.15.37:0/53469527 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5288105010 0x7f52881053e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:14:38.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:38.352+0000 7f528d800700 1 -- 172.21.15.37:0/53469527 >> 172.21.15.37:0/53469527 conn(0x7f52880fabd0 msgr2=0x7f52880fcfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:14:38.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:38.352+0000 7f528d800700 1 -- 172.21.15.37:0/53469527 shutdown_connections 2024-03-29T08:14:38.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:38.352+0000 7f528d800700 1 -- 172.21.15.37:0/53469527 wait complete. 2024-03-29T08:14:38.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:38.353+0000 7f528d800700 1 Processor -- start 2024-03-29T08:14:38.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:38.354+0000 7f528d800700 1 -- start start 2024-03-29T08:14:38.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:38.354+0000 7f528d800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5288105010 0x7f52880724d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:14:38.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:38.354+0000 7f528d800700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f52881059b0 0x7f5288072a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:14:38.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:38.354+0000 7f528d800700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5288076660 con 0x7f5288105010 2024-03-29T08:14:38.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:38.354+0000 7f528d800700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5288072f50 con 0x7f52881059b0 2024-03-29T08:14:38.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:38.355+0000 7f5286a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5288105010 0x7f52880724d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:14:38.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:38.355+0000 7f5286a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5288105010 0x7f52880724d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39658/0 (socket says 172.21.15.37:39658) 2024-03-29T08:14:38.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:38.355+0000 7f5286a00700 1 -- 172.21.15.37:0/3576076508 learned_addr learned my addr 172.21.15.37:0/3576076508 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:14:38.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:38.355+0000 7f5286000700 1 --2- 172.21.15.37:0/3576076508 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f52881059b0 0x7f5288072a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:14:38.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:38.355+0000 7f5286a00700 1 -- 172.21.15.37:0/3576076508 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f52881059b0 msgr2=0x7f5288072a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:14:38.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:38.355+0000 7f5286a00700 1 --2- 172.21.15.37:0/3576076508 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f52881059b0 0x7f5288072a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:14:38.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:38.355+0000 7f5286a00700 1 -- 172.21.15.37:0/3576076508 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5274005040 con 0x7f5288105010 2024-03-29T08:14:38.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:38.355+0000 7f5286a00700 1 --2- 172.21.15.37:0/3576076508 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5288105010 0x7f52880724d0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f5278002a50 tx=0x7f52780046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:14:38.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:38.356+0000 7f527ea00700 1 -- 172.21.15.37:0/3576076508 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5278007cc0 con 0x7f5288105010 2024-03-29T08:14:38.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:38.356+0000 7f527ea00700 1 -- 172.21.15.37:0/3576076508 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f527800f040 con 0x7f5288105010 2024-03-29T08:14:38.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:38.356+0000 7f527ea00700 1 -- 172.21.15.37:0/3576076508 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f527800d9c0 con 0x7f5288105010 2024-03-29T08:14:38.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:38.356+0000 7f528d800700 1 -- 172.21.15.37:0/3576076508 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5288073230 con 0x7f5288105010 2024-03-29T08:14:38.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:38.356+0000 7f528d800700 1 -- 172.21.15.37:0/3576076508 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f528811e610 con 0x7f5288105010 2024-03-29T08:14:38.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:38.357+0000 7f528d800700 1 -- 172.21.15.37:0/3576076508 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5288102320 con 0x7f5288105010 2024-03-29T08:14:38.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:38.358+0000 7f527ea00700 1 -- 172.21.15.37:0/3576076508 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f527800db20 con 0x7f5288105010 2024-03-29T08:14:38.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:38.359+0000 7f527ea00700 1 --2- 172.21.15.37:0/3576076508 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f526c06cc20 0x7f526c06f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:14:38.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:38.359+0000 7f527ea00700 1 -- 172.21.15.37:0/3576076508 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f527808bd60 con 0x7f5288105010 2024-03-29T08:14:38.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:38.360+0000 7f5286000700 1 --2- 172.21.15.37:0/3576076508 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f526c06cc20 0x7f526c06f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:14:38.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:38.360+0000 7f5286000700 1 --2- 172.21.15.37:0/3576076508 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f526c06cc20 0x7f526c06f0d0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f5288073e10 tx=0x7f5274009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:14:38.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:38.364+0000 7f527ea00700 1 -- 172.21.15.37:0/3576076508 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f5278090070 con 0x7f5288105010 2024-03-29T08:14:38.557 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:14:38 smithi148 ceph-mon[106692]: pgmap v87: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 259 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 51/324 objects degraded (15.741%) 2024-03-29T08:14:38.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:38.651+0000 7f528d800700 1 -- 172.21.15.37:0/3576076508 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f528804bed0 con 0x7f526c06cc20 2024-03-29T08:14:38.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:38.664+0000 7f527ea00700 1 -- 172.21.15.37:0/3576076508 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f528804bed0 con 0x7f526c06cc20 2024-03-29T08:14:38.665 INFO:teuthology.orchestra.run.smithi037.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-29T08:14:38.665 INFO:teuthology.orchestra.run.smithi037.stdout:alertmanager.smithi037 smithi037 *:9093,9094 running (4m) 5s ago 17m 13.8M - 0.25.0 c8568f914cd2 2f9f9746ed0f 2024-03-29T08:14:38.665 INFO:teuthology.orchestra.run.smithi037.stdout:crash.smithi037 smithi037 running (83s) 5s ago 17m 6647k - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 178d416a3186 2024-03-29T08:14:38.665 INFO:teuthology.orchestra.run.smithi037.stdout:crash.smithi148 smithi148 running (77s) 72s ago 16m 6656k - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 23666989d58f 2024-03-29T08:14:38.666 INFO:teuthology.orchestra.run.smithi037.stdout:grafana.smithi037 smithi037 *:3000 running (4m) 5s ago 17m 74.7M - 9.4.7 954c08fa6188 6b976fe49952 2024-03-29T08:14:38.666 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi037.iifsmh smithi037 running (10m) 5s ago 10m 110M - 17.2.7-789-g4213e17c cf22eb8683be 074e1acf331f 2024-03-29T08:14:38.666 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi037.mmwgxe smithi037 running (11m) 5s ago 11m 114M - 17.2.7-789-g4213e17c cf22eb8683be 9a5b1e72de5c 2024-03-29T08:14:38.667 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi148.qukpjh smithi148 running (10m) 72s ago 10m 14.1M - 17.2.7-789-g4213e17c cf22eb8683be 8e469782a273 2024-03-29T08:14:38.667 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi148.sqteih smithi148 running (11m) 72s ago 11m 14.0M - 17.2.7-789-g4213e17c cf22eb8683be 8e68f858bb22 2024-03-29T08:14:38.667 INFO:teuthology.orchestra.run.smithi037.stdout:mgr.smithi037.wjikii smithi037 *:8443,9283,8765 running (6m) 5s ago 19m 497M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff c5a1417b9417 2024-03-29T08:14:38.667 INFO:teuthology.orchestra.run.smithi037.stdout:mgr.smithi148.onhgvm smithi148 *:8443,9283,8765 running (5m) 72s ago 16m 447M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff fcfdbce062bd 2024-03-29T08:14:38.667 INFO:teuthology.orchestra.run.smithi037.stdout:mon.smithi037 smithi037 running (2m) 5s ago 19m 45.0M 2048M 18.2.2-1015-gd0e392d9 1cc2f45c17ff eea2060783af 2024-03-29T08:14:38.667 INFO:teuthology.orchestra.run.smithi037.stdout:mon.smithi148 smithi148 running (2m) 72s ago 16m 35.5M 2048M 18.2.2-1015-gd0e392d9 1cc2f45c17ff ad4b2c5748be 2024-03-29T08:14:38.667 INFO:teuthology.orchestra.run.smithi037.stdout:node-exporter.smithi037 smithi037 *:9100 running (5m) 5s ago 17m 7465k - 1.5.0 0da6a335fe13 26b8ae9cb878 2024-03-29T08:14:38.667 INFO:teuthology.orchestra.run.smithi037.stdout:node-exporter.smithi148 smithi148 *:9100 running (5m) 72s ago 16m 7600k - 1.5.0 0da6a335fe13 98414e4e64ed 2024-03-29T08:14:38.667 INFO:teuthology.orchestra.run.smithi037.stdout:osd.0 smithi037 running (51s) 5s ago 15m 97.6M 1619M 18.2.2-1015-gd0e392d9 1cc2f45c17ff c0c584f6245c 2024-03-29T08:14:38.667 INFO:teuthology.orchestra.run.smithi037.stdout:osd.1 smithi037 running (12s) 5s ago 14m 11.0M 1619M 18.2.2-1015-gd0e392d9 1cc2f45c17ff a4a2ff4cc83d 2024-03-29T08:14:38.667 INFO:teuthology.orchestra.run.smithi037.stdout:osd.2 smithi037 running (14m) 5s ago 14m 197M 1619M 17.2.7-789-g4213e17c cf22eb8683be f809bfab13e5 2024-03-29T08:14:38.667 INFO:teuthology.orchestra.run.smithi037.stdout:osd.3 smithi148 running (13m) 72s ago 13m 261M 2642M 17.2.7-789-g4213e17c cf22eb8683be a0a8224bb4b0 2024-03-29T08:14:38.668 INFO:teuthology.orchestra.run.smithi037.stdout:osd.4 smithi148 running (13m) 72s ago 13m 233M 2642M 17.2.7-789-g4213e17c cf22eb8683be a00b69047ed7 2024-03-29T08:14:38.668 INFO:teuthology.orchestra.run.smithi037.stdout:osd.5 smithi148 running (12m) 72s ago 12m 234M 2642M 17.2.7-789-g4213e17c cf22eb8683be 044a3f7db7f8 2024-03-29T08:14:38.668 INFO:teuthology.orchestra.run.smithi037.stdout:prometheus.smithi037 smithi037 *:9095 running (4m) 5s ago 16m 52.0M - 2.43.0 a07b618ecd1d 30c0223ec589 2024-03-29T08:14:38.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:38.666+0000 7f528d800700 1 -- 172.21.15.37:0/3576076508 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f526c06cc20 msgr2=0x7f526c06f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:14:38.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:38.666+0000 7f528d800700 1 --2- 172.21.15.37:0/3576076508 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f526c06cc20 0x7f526c06f0d0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f5288073e10 tx=0x7f5274009e30 comp rx=0 tx=0).stop 2024-03-29T08:14:38.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:38.666+0000 7f528d800700 1 -- 172.21.15.37:0/3576076508 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5288105010 msgr2=0x7f52880724d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:14:38.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:38.666+0000 7f528d800700 1 --2- 172.21.15.37:0/3576076508 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5288105010 0x7f52880724d0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f5278002a50 tx=0x7f52780046b0 comp rx=0 tx=0).stop 2024-03-29T08:14:38.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:38.667+0000 7f528d800700 1 -- 172.21.15.37:0/3576076508 shutdown_connections 2024-03-29T08:14:38.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:38.667+0000 7f528d800700 1 --2- 172.21.15.37:0/3576076508 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f526c06cc20 0x7f526c06f0d0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:14:38.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:38.667+0000 7f528d800700 1 --2- 172.21.15.37:0/3576076508 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5288105010 0x7f52880724d0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:14:38.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:38.667+0000 7f528d800700 1 --2- 172.21.15.37:0/3576076508 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f52881059b0 0x7f5288072a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:14:38.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:38.667+0000 7f528d800700 1 -- 172.21.15.37:0/3576076508 >> 172.21.15.37:0/3576076508 conn(0x7f52880fabd0 msgr2=0x7f52881095b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:14:38.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:38.667+0000 7f528d800700 1 -- 172.21.15.37:0/3576076508 shutdown_connections 2024-03-29T08:14:38.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:38.667+0000 7f528d800700 1 -- 172.21.15.37:0/3576076508 wait complete. 2024-03-29T08:14:38.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:14:38 smithi037 ceph-mon[142325]: pgmap v87: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 259 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 51/324 objects degraded (15.741%) 2024-03-29T08:14:38.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:38.847+0000 7f9436e00700 1 -- 172.21.15.37:0/1302241277 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f94301059a0 msgr2=0x7f94300ff010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:14:38.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:38.847+0000 7f9436e00700 1 --2- 172.21.15.37:0/1302241277 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f94301059a0 0x7f94300ff010 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f941c004940 tx=0x7f941c009e30 comp rx=0 tx=0).stop 2024-03-29T08:14:38.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:38.847+0000 7f9436e00700 1 -- 172.21.15.37:0/1302241277 shutdown_connections 2024-03-29T08:14:38.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:38.847+0000 7f9436e00700 1 --2- 172.21.15.37:0/1302241277 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f94301059a0 0x7f94300ff010 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:14:38.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:38.847+0000 7f9436e00700 1 --2- 172.21.15.37:0/1302241277 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9430105000 0x7f94301053d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:14:38.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:38.847+0000 7f9436e00700 1 -- 172.21.15.37:0/1302241277 >> 172.21.15.37:0/1302241277 conn(0x7f94300fabd0 msgr2=0x7f94300fcfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:14:38.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:38.848+0000 7f9436e00700 1 -- 172.21.15.37:0/1302241277 shutdown_connections 2024-03-29T08:14:38.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:38.848+0000 7f9436e00700 1 -- 172.21.15.37:0/1302241277 wait complete. 2024-03-29T08:14:38.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:38.848+0000 7f9436e00700 1 Processor -- start 2024-03-29T08:14:38.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:38.849+0000 7f9436e00700 1 -- start start 2024-03-29T08:14:38.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:38.849+0000 7f9436e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9430105000 0x7f9430115cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:14:38.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:38.849+0000 7f9436e00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f94301059a0 0x7f9430116210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:14:38.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:38.849+0000 7f9436e00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9430116860 con 0x7f9430105000 2024-03-29T08:14:38.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:38.849+0000 7f9436e00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f943010fc30 con 0x7f94301059a0 2024-03-29T08:14:38.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:38.849+0000 7f942fe00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9430105000 0x7f9430115cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:14:38.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:38.849+0000 7f942fe00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9430105000 0x7f9430115cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39664/0 (socket says 172.21.15.37:39664) 2024-03-29T08:14:38.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:38.849+0000 7f942fe00700 1 -- 172.21.15.37:0/749851211 learned_addr learned my addr 172.21.15.37:0/749851211 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:14:38.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:38.850+0000 7f942f400700 1 --2- 172.21.15.37:0/749851211 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f94301059a0 0x7f9430116210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:14:38.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:38.850+0000 7f942fe00700 1 -- 172.21.15.37:0/749851211 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f94301059a0 msgr2=0x7f9430116210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:14:38.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:38.850+0000 7f942fe00700 1 --2- 172.21.15.37:0/749851211 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f94301059a0 0x7f9430116210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:14:38.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:38.850+0000 7f942fe00700 1 -- 172.21.15.37:0/749851211 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f941c005040 con 0x7f9430105000 2024-03-29T08:14:38.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:38.850+0000 7f942fe00700 1 --2- 172.21.15.37:0/749851211 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9430105000 0x7f9430115cd0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f9420002a50 tx=0x7f94200046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:14:38.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:38.850+0000 7f9427e00700 1 -- 172.21.15.37:0/749851211 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9420007cc0 con 0x7f9430105000 2024-03-29T08:14:38.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:38.850+0000 7f9427e00700 1 -- 172.21.15.37:0/749851211 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f942000f040 con 0x7f9430105000 2024-03-29T08:14:38.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:38.851+0000 7f9436e00700 1 -- 172.21.15.37:0/749851211 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f943010ff10 con 0x7f9430105000 2024-03-29T08:14:38.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:38.851+0000 7f9427e00700 1 -- 172.21.15.37:0/749851211 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9420009540 con 0x7f9430105000 2024-03-29T08:14:38.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:38.851+0000 7f9436e00700 1 -- 172.21.15.37:0/749851211 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94301104d0 con 0x7f9430105000 2024-03-29T08:14:38.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:38.852+0000 7f9436e00700 1 -- 172.21.15.37:0/749851211 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9430102310 con 0x7f9430105000 2024-03-29T08:14:38.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:38.853+0000 7f9427e00700 1 -- 172.21.15.37:0/749851211 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f94200096a0 con 0x7f9430105000 2024-03-29T08:14:38.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:38.854+0000 7f9427e00700 1 --2- 172.21.15.37:0/749851211 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f941406cc30 0x7f941406f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:14:38.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:38.854+0000 7f9427e00700 1 -- 172.21.15.37:0/749851211 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f942008bcc0 con 0x7f9430105000 2024-03-29T08:14:38.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:38.854+0000 7f942f400700 1 --2- 172.21.15.37:0/749851211 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f941406cc30 0x7f941406f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:14:38.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:38.855+0000 7f942f400700 1 --2- 172.21.15.37:0/749851211 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f941406cc30 0x7f941406f0e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f9430111570 tx=0x7f941c009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:14:38.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:38.859+0000 7f9427e00700 1 -- 172.21.15.37:0/749851211 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f9420059830 con 0x7f9430105000 2024-03-29T08:14:39.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:39.183+0000 7f9436e00700 1 -- 172.21.15.37:0/749851211 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f943004bed0 con 0x7f9430105000 2024-03-29T08:14:39.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:39.183+0000 7f9427e00700 1 -- 172.21.15.37:0/749851211 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+847 (secure 0 0 0) 0x7f94200593c0 con 0x7f9430105000 2024-03-29T08:14:39.185 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-29T08:14:39.185 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": { 2024-03-29T08:14:39.185 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 2 2024-03-29T08:14:39.185 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:14:39.185 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": { 2024-03-29T08:14:39.185 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 2 2024-03-29T08:14:39.185 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:14:39.185 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": { 2024-03-29T08:14:39.186 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 17.2.7-789-g4213e17c (4213e17c4de67016ed14f55d22503f4923b3a901) quincy (stable)": 4, 2024-03-29T08:14:39.186 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 2 2024-03-29T08:14:39.186 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:14:39.186 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": { 2024-03-29T08:14:39.186 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 17.2.7-789-g4213e17c (4213e17c4de67016ed14f55d22503f4923b3a901) quincy (stable)": 4 2024-03-29T08:14:39.186 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:14:39.186 INFO:teuthology.orchestra.run.smithi037.stdout: "overall": { 2024-03-29T08:14:39.186 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 17.2.7-789-g4213e17c (4213e17c4de67016ed14f55d22503f4923b3a901) quincy (stable)": 8, 2024-03-29T08:14:39.186 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 6 2024-03-29T08:14:39.186 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-29T08:14:39.186 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-29T08:14:39.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:39.186+0000 7f9436e00700 1 -- 172.21.15.37:0/749851211 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f941406cc30 msgr2=0x7f941406f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:14:39.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:39.186+0000 7f9436e00700 1 --2- 172.21.15.37:0/749851211 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f941406cc30 0x7f941406f0e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f9430111570 tx=0x7f941c009e30 comp rx=0 tx=0).stop 2024-03-29T08:14:39.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:39.186+0000 7f9436e00700 1 -- 172.21.15.37:0/749851211 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9430105000 msgr2=0x7f9430115cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:14:39.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:39.186+0000 7f9436e00700 1 --2- 172.21.15.37:0/749851211 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9430105000 0x7f9430115cd0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f9420002a50 tx=0x7f94200046b0 comp rx=0 tx=0).stop 2024-03-29T08:14:39.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:39.186+0000 7f9436e00700 1 -- 172.21.15.37:0/749851211 shutdown_connections 2024-03-29T08:14:39.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:39.186+0000 7f9436e00700 1 --2- 172.21.15.37:0/749851211 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f941406cc30 0x7f941406f0e0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:14:39.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:39.186+0000 7f9436e00700 1 --2- 172.21.15.37:0/749851211 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9430105000 0x7f9430115cd0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:14:39.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:39.186+0000 7f9436e00700 1 --2- 172.21.15.37:0/749851211 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f94301059a0 0x7f9430116210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:14:39.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:39.186+0000 7f9436e00700 1 -- 172.21.15.37:0/749851211 >> 172.21.15.37:0/749851211 conn(0x7f94300fabd0 msgr2=0x7f94300fcf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:14:39.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:39.186+0000 7f9436e00700 1 -- 172.21.15.37:0/749851211 shutdown_connections 2024-03-29T08:14:39.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:39.186+0000 7f9436e00700 1 -- 172.21.15.37:0/749851211 wait complete. 2024-03-29T08:14:39.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:39.326+0000 7f72f5000700 1 -- 172.21.15.37:0/2985750713 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f72f0102e00 msgr2=0x7f72f01031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:14:39.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:39.326+0000 7f72f5000700 1 --2- 172.21.15.37:0/2985750713 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f72f0102e00 0x7f72f01031d0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f72e0004940 tx=0x7f72e0009e30 comp rx=0 tx=0).stop 2024-03-29T08:14:39.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:39.326+0000 7f72f5000700 1 -- 172.21.15.37:0/2985750713 shutdown_connections 2024-03-29T08:14:39.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:39.326+0000 7f72f5000700 1 --2- 172.21.15.37:0/2985750713 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f72f01037a0 0x7f72f00fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:14:39.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:39.326+0000 7f72f5000700 1 --2- 172.21.15.37:0/2985750713 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f72f0102e00 0x7f72f01031d0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:14:39.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:39.326+0000 7f72f5000700 1 -- 172.21.15.37:0/2985750713 >> 172.21.15.37:0/2985750713 conn(0x7f72f00f89b0 msgr2=0x7f72f00fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:14:39.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:39.326+0000 7f72f5000700 1 -- 172.21.15.37:0/2985750713 shutdown_connections 2024-03-29T08:14:39.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:39.326+0000 7f72f5000700 1 -- 172.21.15.37:0/2985750713 wait complete. 2024-03-29T08:14:39.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:39.327+0000 7f72f5000700 1 Processor -- start 2024-03-29T08:14:39.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:39.327+0000 7f72f5000700 1 -- start start 2024-03-29T08:14:39.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:39.327+0000 7f72f5000700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f72f0102e00 0x7f72f00702c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:14:39.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:39.327+0000 7f72f5000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f72f01037a0 0x7f72f0070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:14:39.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:39.327+0000 7f72f5000700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72f0070d90 con 0x7f72f01037a0 2024-03-29T08:14:39.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:39.327+0000 7f72f5000700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72f0070ed0 con 0x7f72f0102e00 2024-03-29T08:14:39.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:39.328+0000 7f72ed400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f72f01037a0 0x7f72f0070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:14:39.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:39.328+0000 7f72ed400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f72f01037a0 0x7f72f0070800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39674/0 (socket says 172.21.15.37:39674) 2024-03-29T08:14:39.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:39.328+0000 7f72ed400700 1 -- 172.21.15.37:0/432026316 learned_addr learned my addr 172.21.15.37:0/432026316 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:14:39.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:39.328+0000 7f72ede00700 1 --2- 172.21.15.37:0/432026316 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f72f0102e00 0x7f72f00702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:14:39.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:39.328+0000 7f72ed400700 1 -- 172.21.15.37:0/432026316 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f72f0102e00 msgr2=0x7f72f00702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:14:39.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:39.328+0000 7f72ed400700 1 --2- 172.21.15.37:0/432026316 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f72f0102e00 0x7f72f00702c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:14:39.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:39.328+0000 7f72ed400700 1 -- 172.21.15.37:0/432026316 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72e0005040 con 0x7f72f01037a0 2024-03-29T08:14:39.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:39.328+0000 7f72ed400700 1 --2- 172.21.15.37:0/432026316 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f72f01037a0 0x7f72f0070800 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f72dc002a50 tx=0x7f72dc004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:14:39.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:39.329+0000 7f72e6000700 1 -- 172.21.15.37:0/432026316 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f72dc007cc0 con 0x7f72f01037a0 2024-03-29T08:14:39.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:39.329+0000 7f72e6000700 1 -- 172.21.15.37:0/432026316 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f72dc00f040 con 0x7f72f01037a0 2024-03-29T08:14:39.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:39.329+0000 7f72e6000700 1 -- 172.21.15.37:0/432026316 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f72dc00d9a0 con 0x7f72f01037a0 2024-03-29T08:14:39.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:39.329+0000 7f72f5000700 1 -- 172.21.15.37:0/432026316 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72f0119e20 con 0x7f72f01037a0 2024-03-29T08:14:39.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:39.329+0000 7f72f5000700 1 -- 172.21.15.37:0/432026316 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72f011a2a0 con 0x7f72f01037a0 2024-03-29T08:14:39.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:39.330+0000 7f72f5000700 1 -- 172.21.15.37:0/432026316 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72f00fe550 con 0x7f72f01037a0 2024-03-29T08:14:39.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:39.331+0000 7f72e6000700 1 -- 172.21.15.37:0/432026316 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f72dc00dc10 con 0x7f72f01037a0 2024-03-29T08:14:39.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:39.331+0000 7f72e6000700 1 --2- 172.21.15.37:0/432026316 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f72d406cc30 0x7f72d406f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:14:39.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:39.332+0000 7f72e6000700 1 -- 172.21.15.37:0/432026316 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f72dc08bd60 con 0x7f72f01037a0 2024-03-29T08:14:39.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:39.332+0000 7f72ede00700 1 --2- 172.21.15.37:0/432026316 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f72d406cc30 0x7f72d406f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:14:39.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:39.332+0000 7f72ede00700 1 --2- 172.21.15.37:0/432026316 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f72d406cc30 0x7f72d406f0e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f72e000b020 tx=0x7f72e0009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:14:39.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:39.337+0000 7f72e6000700 1 -- 172.21.15.37:0/432026316 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f72dc055de0 con 0x7f72f01037a0 2024-03-29T08:14:39.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:14:39 smithi148 ceph-mon[106692]: from='client.34252 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:14:39.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:14:39 smithi148 ceph-mon[106692]: from='client.34256 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:14:39.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:14:39 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/749851211' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:14:39.594 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:14:39 smithi037 ceph-mon[142325]: from='client.34252 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:14:39.594 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:14:39 smithi037 ceph-mon[142325]: from='client.34256 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:14:39.594 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:14:39 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/749851211' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:14:39.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:39.603+0000 7f72f5000700 1 -- 172.21.15.37:0/432026316 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f72f004bed0 con 0x7f72f01037a0 2024-03-29T08:14:39.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:39.603+0000 7f72e6000700 1 -- 172.21.15.37:0/432026316 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 20 v20) v1 ==== 76+0+1858 (secure 0 0 0) 0x7f72dc059400 con 0x7f72f01037a0 2024-03-29T08:14:39.605 INFO:teuthology.orchestra.run.smithi037.stdout:e20 2024-03-29T08:14:39.605 INFO:teuthology.orchestra.run.smithi037.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-29T08:14:39.605 INFO:teuthology.orchestra.run.smithi037.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T08:14:39.605 INFO:teuthology.orchestra.run.smithi037.stdout:legacy client fscid: 1 2024-03-29T08:14:39.605 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:14:39.605 INFO:teuthology.orchestra.run.smithi037.stdout:Filesystem 'cephfs' (1) 2024-03-29T08:14:39.605 INFO:teuthology.orchestra.run.smithi037.stdout:fs_name cephfs 2024-03-29T08:14:39.605 INFO:teuthology.orchestra.run.smithi037.stdout:epoch 20 2024-03-29T08:14:39.605 INFO:teuthology.orchestra.run.smithi037.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-03-29T08:14:39.605 INFO:teuthology.orchestra.run.smithi037.stdout:created 2024-03-29T08:03:29.241269+0000 2024-03-29T08:14:39.605 INFO:teuthology.orchestra.run.smithi037.stdout:modified 2024-03-29T08:11:27.181622+0000 2024-03-29T08:14:39.605 INFO:teuthology.orchestra.run.smithi037.stdout:tableserver 0 2024-03-29T08:14:39.606 INFO:teuthology.orchestra.run.smithi037.stdout:root 0 2024-03-29T08:14:39.606 INFO:teuthology.orchestra.run.smithi037.stdout:session_timeout 60 2024-03-29T08:14:39.606 INFO:teuthology.orchestra.run.smithi037.stdout:session_autoclose 300 2024-03-29T08:14:39.606 INFO:teuthology.orchestra.run.smithi037.stdout:max_file_size 1099511627776 2024-03-29T08:14:39.606 INFO:teuthology.orchestra.run.smithi037.stdout:max_xattr_size 65536 2024-03-29T08:14:39.606 INFO:teuthology.orchestra.run.smithi037.stdout:required_client_features {} 2024-03-29T08:14:39.606 INFO:teuthology.orchestra.run.smithi037.stdout:last_failure 0 2024-03-29T08:14:39.606 INFO:teuthology.orchestra.run.smithi037.stdout:last_failure_osd_epoch 47 2024-03-29T08:14:39.606 INFO:teuthology.orchestra.run.smithi037.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T08:14:39.606 INFO:teuthology.orchestra.run.smithi037.stdout:max_mds 2 2024-03-29T08:14:39.606 INFO:teuthology.orchestra.run.smithi037.stdout:in 0,1 2024-03-29T08:14:39.606 INFO:teuthology.orchestra.run.smithi037.stdout:up {0=14470,1=14482} 2024-03-29T08:14:39.606 INFO:teuthology.orchestra.run.smithi037.stdout:failed 2024-03-29T08:14:39.606 INFO:teuthology.orchestra.run.smithi037.stdout:damaged 2024-03-29T08:14:39.606 INFO:teuthology.orchestra.run.smithi037.stdout:stopped 2024-03-29T08:14:39.607 INFO:teuthology.orchestra.run.smithi037.stdout:data_pools [3] 2024-03-29T08:14:39.607 INFO:teuthology.orchestra.run.smithi037.stdout:metadata_pool 2 2024-03-29T08:14:39.607 INFO:teuthology.orchestra.run.smithi037.stdout:inline_data disabled 2024-03-29T08:14:39.607 INFO:teuthology.orchestra.run.smithi037.stdout:balancer 2024-03-29T08:14:39.607 INFO:teuthology.orchestra.run.smithi037.stdout:bal_rank_mask -1 2024-03-29T08:14:39.607 INFO:teuthology.orchestra.run.smithi037.stdout:standby_count_wanted 1 2024-03-29T08:14:39.607 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi037.mmwgxe{0:14470} state up:active seq 5 join_fscid=1 addr [v2:172.21.15.37:6826/1559618909,v1:172.21.15.37:6827/1559618909] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:14:39.607 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi037.iifsmh{1:14482} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.37:6828/2341530495,v1:172.21.15.37:6829/2341530495] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:14:39.607 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:14:39.607 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:14:39.607 INFO:teuthology.orchestra.run.smithi037.stdout:Standby daemons: 2024-03-29T08:14:39.607 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:14:39.607 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi148.qukpjh{-1:24319} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.148:6826/2641327912,v1:172.21.15.148:6827/2641327912] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:14:39.607 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi148.sqteih{-1:24327} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.148:6824/3252351726,v1:172.21.15.148:6825/3252351726] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:14:39.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:39.606+0000 7f72f5000700 1 -- 172.21.15.37:0/432026316 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f72d406cc30 msgr2=0x7f72d406f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:14:39.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:39.606+0000 7f72f5000700 1 --2- 172.21.15.37:0/432026316 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f72d406cc30 0x7f72d406f0e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f72e000b020 tx=0x7f72e0009e30 comp rx=0 tx=0).stop 2024-03-29T08:14:39.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:39.606+0000 7f72f5000700 1 -- 172.21.15.37:0/432026316 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f72f01037a0 msgr2=0x7f72f0070800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:14:39.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:39.606+0000 7f72f5000700 1 --2- 172.21.15.37:0/432026316 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f72f01037a0 0x7f72f0070800 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f72dc002a50 tx=0x7f72dc004600 comp rx=0 tx=0).stop 2024-03-29T08:14:39.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:39.606+0000 7f72f5000700 1 -- 172.21.15.37:0/432026316 shutdown_connections 2024-03-29T08:14:39.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:39.606+0000 7f72f5000700 1 --2- 172.21.15.37:0/432026316 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f72d406cc30 0x7f72d406f0e0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:14:39.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:39.606+0000 7f72f5000700 1 --2- 172.21.15.37:0/432026316 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f72f0102e00 0x7f72f00702c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:14:39.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:39.606+0000 7f72f5000700 1 --2- 172.21.15.37:0/432026316 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f72f01037a0 0x7f72f0070800 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:14:39.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:39.606+0000 7f72f5000700 1 -- 172.21.15.37:0/432026316 >> 172.21.15.37:0/432026316 conn(0x7f72f00f89b0 msgr2=0x7f72f00f9a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:14:39.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:39.606+0000 7f72f5000700 1 -- 172.21.15.37:0/432026316 shutdown_connections 2024-03-29T08:14:39.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:39.606+0000 7f72f5000700 1 -- 172.21.15.37:0/432026316 wait complete. 2024-03-29T08:14:39.609 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 20 2024-03-29T08:14:39.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:39.752+0000 7fd335600700 1 -- 172.21.15.37:0/1601270715 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd3300708a0 msgr2=0x7fd330070d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:14:39.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:39.752+0000 7fd335600700 1 --2- 172.21.15.37:0/1601270715 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd3300708a0 0x7fd330070d10 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fd31c004970 tx=0x7fd31c009e30 comp rx=0 tx=0).stop 2024-03-29T08:14:39.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:39.752+0000 7fd335600700 1 -- 172.21.15.37:0/1601270715 shutdown_connections 2024-03-29T08:14:39.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:39.752+0000 7fd335600700 1 --2- 172.21.15.37:0/1601270715 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd3300708a0 0x7fd330070d10 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:14:39.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:39.752+0000 7fd335600700 1 --2- 172.21.15.37:0/1601270715 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd330071f90 0x7fd330070360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:14:39.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:39.752+0000 7fd335600700 1 -- 172.21.15.37:0/1601270715 >> 172.21.15.37:0/1601270715 conn(0x7fd3300f89d0 msgr2=0x7fd3300fade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:14:39.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:39.752+0000 7fd335600700 1 -- 172.21.15.37:0/1601270715 shutdown_connections 2024-03-29T08:14:39.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:39.752+0000 7fd335600700 1 -- 172.21.15.37:0/1601270715 wait complete. 2024-03-29T08:14:39.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:39.753+0000 7fd335600700 1 Processor -- start 2024-03-29T08:14:39.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:39.753+0000 7fd335600700 1 -- start start 2024-03-29T08:14:39.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:39.754+0000 7fd335600700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd3300708a0 0x7fd33011a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:14:39.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:39.754+0000 7fd335600700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd330071f90 0x7fd33011a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:14:39.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:39.754+0000 7fd335600700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd33011ac30 con 0x7fd3300708a0 2024-03-29T08:14:39.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:39.754+0000 7fd335600700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd330114000 con 0x7fd330071f90 2024-03-29T08:14:39.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:39.754+0000 7fd32ea00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd3300708a0 0x7fd33011a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:14:39.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:39.754+0000 7fd32ea00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd3300708a0 0x7fd33011a0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39692/0 (socket says 172.21.15.37:39692) 2024-03-29T08:14:39.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:39.754+0000 7fd32ea00700 1 -- 172.21.15.37:0/2392349375 learned_addr learned my addr 172.21.15.37:0/2392349375 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:14:39.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:39.754+0000 7fd32e000700 1 --2- 172.21.15.37:0/2392349375 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd330071f90 0x7fd33011a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:14:39.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:39.754+0000 7fd32ea00700 1 -- 172.21.15.37:0/2392349375 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd330071f90 msgr2=0x7fd33011a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:14:39.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:39.755+0000 7fd32ea00700 1 --2- 172.21.15.37:0/2392349375 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd330071f90 0x7fd33011a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:14:39.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:39.755+0000 7fd32ea00700 1 -- 172.21.15.37:0/2392349375 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd31c005040 con 0x7fd3300708a0 2024-03-29T08:14:39.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:39.755+0000 7fd32ea00700 1 --2- 172.21.15.37:0/2392349375 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd3300708a0 0x7fd33011a0a0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fd32000eb40 tx=0x7fd32000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:14:39.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:39.755+0000 7fd326a00700 1 -- 172.21.15.37:0/2392349375 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd320007cc0 con 0x7fd3300708a0 2024-03-29T08:14:39.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:39.755+0000 7fd326a00700 1 -- 172.21.15.37:0/2392349375 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd32000f040 con 0x7fd3300708a0 2024-03-29T08:14:39.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:39.755+0000 7fd326a00700 1 -- 172.21.15.37:0/2392349375 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd32000c990 con 0x7fd3300708a0 2024-03-29T08:14:39.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:39.755+0000 7fd335600700 1 -- 172.21.15.37:0/2392349375 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd330114280 con 0x7fd3300708a0 2024-03-29T08:14:39.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:39.755+0000 7fd335600700 1 -- 172.21.15.37:0/2392349375 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd330114790 con 0x7fd3300708a0 2024-03-29T08:14:39.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:39.756+0000 7fd335600700 1 -- 172.21.15.37:0/2392349375 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd330071aa0 con 0x7fd3300708a0 2024-03-29T08:14:39.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:39.757+0000 7fd326a00700 1 -- 172.21.15.37:0/2392349375 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fd32000caf0 con 0x7fd3300708a0 2024-03-29T08:14:39.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:39.758+0000 7fd326a00700 1 --2- 172.21.15.37:0/2392349375 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fd31406cc30 0x7fd31406f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:14:39.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:39.758+0000 7fd326a00700 1 -- 172.21.15.37:0/2392349375 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fd32008bb80 con 0x7fd3300708a0 2024-03-29T08:14:39.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:39.758+0000 7fd32e000700 1 --2- 172.21.15.37:0/2392349375 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fd31406cc30 0x7fd31406f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:14:39.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:39.759+0000 7fd32e000700 1 --2- 172.21.15.37:0/2392349375 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fd31406cc30 0x7fd31406f0e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fd31c009fd0 tx=0x7fd31c009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:14:39.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:39.763+0000 7fd326a00700 1 -- 172.21.15.37:0/2392349375 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fd320059700 con 0x7fd3300708a0 2024-03-29T08:14:40.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:40.006+0000 7fd335600700 1 -- 172.21.15.37:0/2392349375 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd33004bed0 con 0x7fd31406cc30 2024-03-29T08:14:40.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:40.007+0000 7fd326a00700 1 -- 172.21.15.37:0/2392349375 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7fd33004bed0 con 0x7fd31406cc30 2024-03-29T08:14:40.009 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-29T08:14:40.009 INFO:teuthology.orchestra.run.smithi037.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59", 2024-03-29T08:14:40.009 INFO:teuthology.orchestra.run.smithi037.stdout: "in_progress": true, 2024-03-29T08:14:40.009 INFO:teuthology.orchestra.run.smithi037.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-29T08:14:40.009 INFO:teuthology.orchestra.run.smithi037.stdout: "services_complete": [ 2024-03-29T08:14:40.009 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr", 2024-03-29T08:14:40.009 INFO:teuthology.orchestra.run.smithi037.stdout: "mon", 2024-03-29T08:14:40.009 INFO:teuthology.orchestra.run.smithi037.stdout: "crash" 2024-03-29T08:14:40.010 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-29T08:14:40.010 INFO:teuthology.orchestra.run.smithi037.stdout: "progress": "8/21 daemons upgraded", 2024-03-29T08:14:40.010 INFO:teuthology.orchestra.run.smithi037.stdout: "message": "Currently upgrading osd daemons", 2024-03-29T08:14:40.010 INFO:teuthology.orchestra.run.smithi037.stdout: "is_paused": false 2024-03-29T08:14:40.010 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-29T08:14:40.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:40.010+0000 7fd335600700 1 -- 172.21.15.37:0/2392349375 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fd31406cc30 msgr2=0x7fd31406f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:14:40.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:40.010+0000 7fd335600700 1 --2- 172.21.15.37:0/2392349375 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fd31406cc30 0x7fd31406f0e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fd31c009fd0 tx=0x7fd31c009e30 comp rx=0 tx=0).stop 2024-03-29T08:14:40.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:40.010+0000 7fd335600700 1 -- 172.21.15.37:0/2392349375 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd3300708a0 msgr2=0x7fd33011a0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:14:40.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:40.010+0000 7fd335600700 1 --2- 172.21.15.37:0/2392349375 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd3300708a0 0x7fd33011a0a0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fd32000eb40 tx=0x7fd32000b040 comp rx=0 tx=0).stop 2024-03-29T08:14:40.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:40.010+0000 7fd335600700 1 -- 172.21.15.37:0/2392349375 shutdown_connections 2024-03-29T08:14:40.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:40.010+0000 7fd335600700 1 --2- 172.21.15.37:0/2392349375 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fd31406cc30 0x7fd31406f0e0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:14:40.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:40.010+0000 7fd335600700 1 --2- 172.21.15.37:0/2392349375 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd3300708a0 0x7fd33011a0a0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:14:40.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:40.010+0000 7fd335600700 1 --2- 172.21.15.37:0/2392349375 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd330071f90 0x7fd33011a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:14:40.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:40.010+0000 7fd335600700 1 -- 172.21.15.37:0/2392349375 >> 172.21.15.37:0/2392349375 conn(0x7fd3300f89d0 msgr2=0x7fd3300f9c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:14:40.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:40.010+0000 7fd335600700 1 -- 172.21.15.37:0/2392349375 shutdown_connections 2024-03-29T08:14:40.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:40.010+0000 7fd335600700 1 -- 172.21.15.37:0/2392349375 wait complete. 2024-03-29T08:14:40.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:40.147+0000 7fb57a800700 1 -- 172.21.15.37:0/2919923530 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb574102e00 msgr2=0x7fb5741031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:14:40.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:40.147+0000 7fb57a800700 1 --2- 172.21.15.37:0/2919923530 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb574102e00 0x7fb5741031d0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fb564004940 tx=0x7fb564009e30 comp rx=0 tx=0).stop 2024-03-29T08:14:40.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:40.147+0000 7fb57a800700 1 -- 172.21.15.37:0/2919923530 shutdown_connections 2024-03-29T08:14:40.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:40.147+0000 7fb57a800700 1 --2- 172.21.15.37:0/2919923530 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb5741037a0 0x7fb5740fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:14:40.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:40.147+0000 7fb57a800700 1 --2- 172.21.15.37:0/2919923530 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb574102e00 0x7fb5741031d0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:14:40.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:40.147+0000 7fb57a800700 1 -- 172.21.15.37:0/2919923530 >> 172.21.15.37:0/2919923530 conn(0x7fb5740f89b0 msgr2=0x7fb5740fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:14:40.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:40.148+0000 7fb57a800700 1 -- 172.21.15.37:0/2919923530 shutdown_connections 2024-03-29T08:14:40.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:40.148+0000 7fb57a800700 1 -- 172.21.15.37:0/2919923530 wait complete. 2024-03-29T08:14:40.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:40.148+0000 7fb57a800700 1 Processor -- start 2024-03-29T08:14:40.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:40.149+0000 7fb57a800700 1 -- start start 2024-03-29T08:14:40.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:40.149+0000 7fb57a800700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb574102e00 0x7fb5740702c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:14:40.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:40.149+0000 7fb57a800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb5741037a0 0x7fb574070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:14:40.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:40.149+0000 7fb57a800700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb574070d90 con 0x7fb5741037a0 2024-03-29T08:14:40.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:40.149+0000 7fb57a800700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb574070ed0 con 0x7fb574102e00 2024-03-29T08:14:40.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:40.149+0000 7fb572a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb5741037a0 0x7fb574070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:14:40.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:40.149+0000 7fb572a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb5741037a0 0x7fb574070800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54210/0 (socket says 172.21.15.37:54210) 2024-03-29T08:14:40.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:40.149+0000 7fb572a00700 1 -- 172.21.15.37:0/1444074535 learned_addr learned my addr 172.21.15.37:0/1444074535 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:14:40.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:40.149+0000 7fb573400700 1 --2- 172.21.15.37:0/1444074535 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb574102e00 0x7fb5740702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:14:40.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:40.150+0000 7fb572a00700 1 -- 172.21.15.37:0/1444074535 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb574102e00 msgr2=0x7fb5740702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:14:40.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:40.150+0000 7fb572a00700 1 --2- 172.21.15.37:0/1444074535 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb574102e00 0x7fb5740702c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:14:40.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:40.150+0000 7fb572a00700 1 -- 172.21.15.37:0/1444074535 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb564005040 con 0x7fb5741037a0 2024-03-29T08:14:40.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:40.150+0000 7fb572a00700 1 --2- 172.21.15.37:0/1444074535 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb5741037a0 0x7fb574070800 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fb560002a50 tx=0x7fb5600046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:14:40.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:40.150+0000 7fb56b400700 1 -- 172.21.15.37:0/1444074535 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb560007cc0 con 0x7fb5741037a0 2024-03-29T08:14:40.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:40.150+0000 7fb56b400700 1 -- 172.21.15.37:0/1444074535 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fb56000f040 con 0x7fb5741037a0 2024-03-29T08:14:40.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:40.150+0000 7fb56b400700 1 -- 172.21.15.37:0/1444074535 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb56000da30 con 0x7fb5741037a0 2024-03-29T08:14:40.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:40.150+0000 7fb57a800700 1 -- 172.21.15.37:0/1444074535 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb574119d40 con 0x7fb5741037a0 2024-03-29T08:14:40.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:40.150+0000 7fb57a800700 1 -- 172.21.15.37:0/1444074535 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb57411a1c0 con 0x7fb5741037a0 2024-03-29T08:14:40.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:40.152+0000 7fb57a800700 1 -- 172.21.15.37:0/1444074535 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5740fe550 con 0x7fb5741037a0 2024-03-29T08:14:40.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:40.153+0000 7fb56b400700 1 -- 172.21.15.37:0/1444074535 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fb56000db90 con 0x7fb5741037a0 2024-03-29T08:14:40.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:40.153+0000 7fb56b400700 1 --2- 172.21.15.37:0/1444074535 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fb55806cc30 0x7fb55806f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:14:40.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:40.154+0000 7fb56b400700 1 -- 172.21.15.37:0/1444074535 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fb56008bb60 con 0x7fb5741037a0 2024-03-29T08:14:40.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:40.154+0000 7fb573400700 1 --2- 172.21.15.37:0/1444074535 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fb55806cc30 0x7fb55806f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:14:40.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:40.154+0000 7fb573400700 1 --2- 172.21.15.37:0/1444074535 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fb55806cc30 0x7fb55806f0e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fb564009fd0 tx=0x7fb564009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:14:40.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:40.158+0000 7fb56b400700 1 -- 172.21.15.37:0/1444074535 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fb560055c40 con 0x7fb5741037a0 2024-03-29T08:14:40.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:40.458+0000 7fb57a800700 1 -- 172.21.15.37:0/1444074535 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fb57405f5f0 con 0x7fb5741037a0 2024-03-29T08:14:40.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:40.458+0000 7fb56b400700 1 -- 172.21.15.37:0/1444074535 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+680 (secure 0 0 0) 0x7fb560059260 con 0x7fb5741037a0 2024-03-29T08:14:40.460 INFO:teuthology.orchestra.run.smithi037.stdout:HEALTH_WARN Degraded data redundancy: 29/324 objects degraded (8.951%), 9 pgs degraded 2024-03-29T08:14:40.460 INFO:teuthology.orchestra.run.smithi037.stdout:[WRN] PG_DEGRADED: Degraded data redundancy: 29/324 objects degraded (8.951%), 9 pgs degraded 2024-03-29T08:14:40.460 INFO:teuthology.orchestra.run.smithi037.stdout: pg 1.0 is active+undersized+degraded, acting [3,0] 2024-03-29T08:14:40.460 INFO:teuthology.orchestra.run.smithi037.stdout: pg 2.0 is active+undersized+degraded, acting [3,0] 2024-03-29T08:14:40.460 INFO:teuthology.orchestra.run.smithi037.stdout: pg 2.1 is active+undersized+degraded, acting [2,0] 2024-03-29T08:14:40.460 INFO:teuthology.orchestra.run.smithi037.stdout: pg 2.2 is active+undersized+degraded, acting [5,0] 2024-03-29T08:14:40.460 INFO:teuthology.orchestra.run.smithi037.stdout: pg 2.3 is active+undersized+degraded, acting [5,2] 2024-03-29T08:14:40.460 INFO:teuthology.orchestra.run.smithi037.stdout: pg 2.10 is active+undersized+degraded, acting [2,0] 2024-03-29T08:14:40.460 INFO:teuthology.orchestra.run.smithi037.stdout: pg 2.11 is active+undersized+degraded, acting [4,3] 2024-03-29T08:14:40.460 INFO:teuthology.orchestra.run.smithi037.stdout: pg 2.12 is active+undersized+degraded, acting [3,0] 2024-03-29T08:14:40.460 INFO:teuthology.orchestra.run.smithi037.stdout: pg 2.17 is active+undersized+degraded, acting [5,2] 2024-03-29T08:14:40.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:40.461+0000 7fb57a800700 1 -- 172.21.15.37:0/1444074535 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fb55806cc30 msgr2=0x7fb55806f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:14:40.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:40.461+0000 7fb57a800700 1 --2- 172.21.15.37:0/1444074535 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fb55806cc30 0x7fb55806f0e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fb564009fd0 tx=0x7fb564009e30 comp rx=0 tx=0).stop 2024-03-29T08:14:40.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:40.461+0000 7fb57a800700 1 -- 172.21.15.37:0/1444074535 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb5741037a0 msgr2=0x7fb574070800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:14:40.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:40.461+0000 7fb57a800700 1 --2- 172.21.15.37:0/1444074535 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb5741037a0 0x7fb574070800 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fb560002a50 tx=0x7fb5600046b0 comp rx=0 tx=0).stop 2024-03-29T08:14:40.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:40.461+0000 7fb57a800700 1 -- 172.21.15.37:0/1444074535 shutdown_connections 2024-03-29T08:14:40.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:40.461+0000 7fb57a800700 1 --2- 172.21.15.37:0/1444074535 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fb55806cc30 0x7fb55806f0e0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:14:40.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:40.461+0000 7fb57a800700 1 --2- 172.21.15.37:0/1444074535 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb574102e00 0x7fb5740702c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:14:40.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:40.461+0000 7fb57a800700 1 --2- 172.21.15.37:0/1444074535 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb5741037a0 0x7fb574070800 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:14:40.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:40.461+0000 7fb57a800700 1 -- 172.21.15.37:0/1444074535 >> 172.21.15.37:0/1444074535 conn(0x7fb5740f89b0 msgr2=0x7fb5740f99f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:14:40.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:40.461+0000 7fb57a800700 1 -- 172.21.15.37:0/1444074535 shutdown_connections 2024-03-29T08:14:40.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:14:40.461+0000 7fb57a800700 1 -- 172.21.15.37:0/1444074535 wait complete. 2024-03-29T08:14:40.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:14:40 smithi037 ceph-mon[142325]: from='client.34260 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:14:40.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:14:40 smithi037 ceph-mon[142325]: pgmap v88: 65 pgs: 9 active+undersized+degraded, 11 active+undersized, 45 active+clean; 259 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 29/324 objects degraded (8.951%) 2024-03-29T08:14:40.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:14:40 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:14:40.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:14:40 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:14:40.928 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:14:40 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/432026316' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T08:14:40.928 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:14:40 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:14:40.928 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:14:40 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:14:40.928 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:14:40 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T08:14:40.928 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:14:40 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T08:14:40.928 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:14:40 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:14:40.928 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:14:40 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T08:14:40.928 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:14:40 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:14:40.928 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:14:40 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:14:40.928 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:14:40 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:14:40.928 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:14:40 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-03-29T08:14:40.928 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:14:40 smithi037 ceph-mon[142325]: Health check update: Degraded data redundancy: 29/324 objects degraded (8.951%), 9 pgs degraded (PG_DEGRADED) 2024-03-29T08:14:40.928 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:14:40 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/1444074535' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T08:14:41.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:14:40 smithi148 ceph-mon[106692]: from='client.34260 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:14:41.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:14:40 smithi148 ceph-mon[106692]: pgmap v88: 65 pgs: 9 active+undersized+degraded, 11 active+undersized, 45 active+clean; 259 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 29/324 objects degraded (8.951%) 2024-03-29T08:14:41.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:14:40 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:14:41.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:14:40 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:14:41.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:14:40 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/432026316' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T08:14:41.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:14:40 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:14:41.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:14:40 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:14:41.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:14:40 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T08:14:41.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:14:40 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T08:14:41.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:14:40 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:14:41.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:14:40 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T08:14:41.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:14:40 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:14:41.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:14:40 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:14:41.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:14:40 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:14:41.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:14:40 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-03-29T08:14:41.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:14:40 smithi148 ceph-mon[106692]: Health check update: Degraded data redundancy: 29/324 objects degraded (8.951%), 9 pgs degraded (PG_DEGRADED) 2024-03-29T08:14:41.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:14:40 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/1444074535' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T08:14:41.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:14:41 smithi037 ceph-mon[142325]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-03-29T08:14:41.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:14:41 smithi037 ceph-mon[142325]: Upgrade: unsafe to stop osd(s) at this time (9 PGs are or would become offline) 2024-03-29T08:14:41.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:14:41 smithi037 ceph-mon[142325]: from='client.34272 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:14:42.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:14:41 smithi148 ceph-mon[106692]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-03-29T08:14:42.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:14:41 smithi148 ceph-mon[106692]: Upgrade: unsafe to stop osd(s) at this time (9 PGs are or would become offline) 2024-03-29T08:14:42.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:14:41 smithi148 ceph-mon[106692]: from='client.34272 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:14:42.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:14:42 smithi037 ceph-mon[142325]: pgmap v89: 65 pgs: 65 active+clean; 259 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:14:42.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:14:42 smithi037 ceph-mon[142325]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 29/324 objects degraded (8.951%), 9 pgs degraded) 2024-03-29T08:14:42.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:14:42 smithi037 ceph-mon[142325]: Cluster is now healthy 2024-03-29T08:14:43.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:14:42 smithi148 ceph-mon[106692]: pgmap v89: 65 pgs: 65 active+clean; 259 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:14:43.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:14:42 smithi148 ceph-mon[106692]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 29/324 objects degraded (8.951%), 9 pgs degraded) 2024-03-29T08:14:43.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:14:42 smithi148 ceph-mon[106692]: Cluster is now healthy 2024-03-29T08:14:44.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:14:44 smithi037 ceph-mon[142325]: pgmap v90: 65 pgs: 65 active+clean; 259 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:14:45.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:14:44 smithi148 ceph-mon[106692]: pgmap v90: 65 pgs: 65 active+clean; 259 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:14:46.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:14:46 smithi037 ceph-mon[142325]: pgmap v91: 65 pgs: 65 active+clean; 259 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:14:47.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:14:46 smithi148 ceph-mon[106692]: pgmap v91: 65 pgs: 65 active+clean; 259 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:14:48.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:14:48 smithi037 ceph-mon[142325]: pgmap v92: 65 pgs: 65 active+clean; 259 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:14:49.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:14:48 smithi148 ceph-mon[106692]: pgmap v92: 65 pgs: 65 active+clean; 259 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:14:50.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:14:50 smithi037 ceph-mon[142325]: pgmap v93: 65 pgs: 65 active+clean; 259 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:14:50.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:14:50 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:14:51.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:14:50 smithi148 ceph-mon[106692]: pgmap v93: 65 pgs: 65 active+clean; 259 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:14:51.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:14:50 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:14:52.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:14:52 smithi037 ceph-mon[142325]: pgmap v94: 65 pgs: 65 active+clean; 259 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:14:53.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:14:52 smithi148 ceph-mon[106692]: pgmap v94: 65 pgs: 65 active+clean; 259 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:14:53.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:14:53 smithi037 ceph-mon[142325]: pgmap v95: 65 pgs: 65 active+clean; 259 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:14:54.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:14:53 smithi148 ceph-mon[106692]: pgmap v95: 65 pgs: 65 active+clean; 259 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:14:55.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:14:55 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:14:55.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:14:55 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:14:55.928 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:14:55 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-03-29T08:14:56.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:14:55 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:14:56.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:14:55 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:14:56.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:14:55 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-03-29T08:14:56.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:14:56 smithi037 ceph-mon[142325]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-03-29T08:14:56.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:14:56 smithi037 ceph-mon[142325]: Upgrade: osd.2 is safe to restart 2024-03-29T08:14:56.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:14:56 smithi037 ceph-mon[142325]: pgmap v96: 65 pgs: 65 active+clean; 259 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:14:57.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:14:56 smithi148 ceph-mon[106692]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-03-29T08:14:57.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:14:56 smithi148 ceph-mon[106692]: Upgrade: osd.2 is safe to restart 2024-03-29T08:14:57.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:14:56 smithi148 ceph-mon[106692]: pgmap v96: 65 pgs: 65 active+clean; 259 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:14:57.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:14:57 smithi037 ceph-mon[142325]: Upgrade: Updating osd.2 2024-03-29T08:14:57.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:14:57 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:14:57.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:14:57 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-29T08:14:57.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:14:57 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T08:14:57.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:14:57 smithi037 ceph-mon[142325]: Deploying daemon osd.2 on smithi037 2024-03-29T08:14:57.928 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:14:57 smithi037 ceph-mon[142325]: pgmap v97: 65 pgs: 65 active+clean; 259 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:14:58.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:14:57 smithi148 ceph-mon[106692]: Upgrade: Updating osd.2 2024-03-29T08:14:58.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:14:57 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:14:58.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:14:57 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-29T08:14:58.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:14:57 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T08:14:58.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:14:57 smithi148 ceph-mon[106692]: Deploying daemon osd.2 on smithi037 2024-03-29T08:14:58.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:14:57 smithi148 ceph-mon[106692]: pgmap v97: 65 pgs: 65 active+clean; 259 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:15:00.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:15:00 smithi037 ceph-mon[142325]: pgmap v98: 65 pgs: 65 active+clean; 259 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:15:00.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:15:00 smithi148 ceph-mon[106692]: pgmap v98: 65 pgs: 65 active+clean; 259 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:15:01.123 INFO:journalctl@ceph.osd.2.smithi037.stdout:Mar 29 08:15:00 smithi037 systemd[1]: Stopping Ceph osd.2 for 99b492c2-eda1-11ee-b647-cb9ed24678a4... 2024-03-29T08:15:01.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:15:01 smithi037 ceph-mon[142325]: osd.2 marked itself down and dead 2024-03-29T08:15:01.427 INFO:journalctl@ceph.osd.2.smithi037.stdout:Mar 29 08:15:01 smithi037 ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4-osd-2[77503]: 2024-03-29T08:15:01.122+0000 7ffa75800700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-29T08:15:01.427 INFO:journalctl@ceph.osd.2.smithi037.stdout:Mar 29 08:15:01 smithi037 ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4-osd-2[77503]: 2024-03-29T08:15:01.122+0000 7ffa75800700 -1 osd.2 60 *** Got signal Terminated *** 2024-03-29T08:15:01.427 INFO:journalctl@ceph.osd.2.smithi037.stdout:Mar 29 08:15:01 smithi037 ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4-osd-2[77503]: 2024-03-29T08:15:01.122+0000 7ffa75800700 -1 osd.2 60 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-29T08:15:01.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:15:01 smithi148 ceph-mon[106692]: osd.2 marked itself down and dead 2024-03-29T08:15:02.177 INFO:journalctl@ceph.osd.2.smithi037.stdout:Mar 29 08:15:02 smithi037 bash[170606]: ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4-osd-2 2024-03-29T08:15:02.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:15:02 smithi148 ceph-mon[106692]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-29T08:15:02.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:15:02 smithi148 ceph-mon[106692]: osdmap e61: 6 total, 5 up, 6 in 2024-03-29T08:15:02.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:15:02 smithi148 ceph-mon[106692]: pgmap v100: 65 pgs: 9 stale+active+clean, 56 active+clean; 259 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:15:02.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:15:02 smithi148 ceph-mon[106692]: osdmap e62: 6 total, 5 up, 6 in 2024-03-29T08:15:02.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:15:02 smithi037 ceph-mon[142325]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-29T08:15:02.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:15:02 smithi037 ceph-mon[142325]: osdmap e61: 6 total, 5 up, 6 in 2024-03-29T08:15:02.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:15:02 smithi037 ceph-mon[142325]: pgmap v100: 65 pgs: 9 stale+active+clean, 56 active+clean; 259 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:15:02.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:15:02 smithi037 ceph-mon[142325]: osdmap e62: 6 total, 5 up, 6 in 2024-03-29T08:15:04.539 INFO:journalctl@ceph.osd.2.smithi037.stdout:Mar 29 08:15:04 smithi037 systemd[1]: ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4@osd.2.service: Succeeded. 2024-03-29T08:15:04.539 INFO:journalctl@ceph.osd.2.smithi037.stdout:Mar 29 08:15:04 smithi037 systemd[1]: Stopped Ceph osd.2 for 99b492c2-eda1-11ee-b647-cb9ed24678a4. 2024-03-29T08:15:04.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:15:04 smithi037 ceph-mon[142325]: pgmap v102: 65 pgs: 9 stale+active+clean, 56 active+clean; 259 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:15:04.927 INFO:journalctl@ceph.osd.2.smithi037.stdout:Mar 29 08:15:04 smithi037 systemd[1]: Starting Ceph osd.2 for 99b492c2-eda1-11ee-b647-cb9ed24678a4... 2024-03-29T08:15:05.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:15:04 smithi148 ceph-mon[106692]: pgmap v102: 65 pgs: 9 stale+active+clean, 56 active+clean; 259 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail 2024-03-29T08:15:05.907 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:15:05 smithi037 ceph-mon[142325]: pgmap v103: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 259 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 39/324 objects degraded (12.037%) 2024-03-29T08:15:05.907 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:15:05 smithi037 ceph-mon[142325]: Health check failed: Degraded data redundancy: 39/324 objects degraded (12.037%), 12 pgs degraded (PG_DEGRADED) 2024-03-29T08:15:05.907 INFO:journalctl@ceph.osd.2.smithi037.stdout:Mar 29 08:15:05 smithi037 bash[170876]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-03-29T08:15:05.908 INFO:journalctl@ceph.osd.2.smithi037.stdout:Mar 29 08:15:05 smithi037 bash[170876]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-2 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-03-29T08:15:05.908 INFO:journalctl@ceph.osd.2.smithi037.stdout:Mar 29 08:15:05 smithi037 bash[170876]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-03-29T08:15:05.908 INFO:journalctl@ceph.osd.2.smithi037.stdout:Mar 29 08:15:05 smithi037 bash[170876]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-03-29T08:15:05.908 INFO:journalctl@ceph.osd.2.smithi037.stdout:Mar 29 08:15:05 smithi037 bash[170876]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-2/block 2024-03-29T08:15:05.908 INFO:journalctl@ceph.osd.2.smithi037.stdout:Mar 29 08:15:05 smithi037 bash[170876]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-03-29T08:15:05.908 INFO:journalctl@ceph.osd.2.smithi037.stdout:Mar 29 08:15:05 smithi037 bash[170876]: --> ceph-volume raw activate successful for osd ID: 2 2024-03-29T08:15:06.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:15:05 smithi148 ceph-mon[106692]: pgmap v103: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 259 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 39/324 objects degraded (12.037%) 2024-03-29T08:15:06.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:15:05 smithi148 ceph-mon[106692]: Health check failed: Degraded data redundancy: 39/324 objects degraded (12.037%), 12 pgs degraded (PG_DEGRADED) 2024-03-29T08:15:07.927 INFO:journalctl@ceph.osd.2.smithi037.stdout:Mar 29 08:15:07 smithi037 ceph-osd[171170]: -- 172.21.15.37:0/2075320973 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x55f226c8b600 con 0x55f226c13000 2024-03-29T08:15:07.927 INFO:journalctl@ceph.osd.2.smithi037.stdout:Mar 29 08:15:07 smithi037 ceph-osd[171170]: -- 172.21.15.37:0/2075320973 <== mon.0 v2:172.21.15.37:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55f226c431e0 con 0x55f226c13000 2024-03-29T08:15:07.927 INFO:journalctl@ceph.osd.2.smithi037.stdout:Mar 29 08:15:07 smithi037 bash[171132]: 8b1bc71aa79e34b70d0ccffa7e78096d217e67abce55ad333185bd32f7e0d3da 2024-03-29T08:15:07.927 INFO:journalctl@ceph.osd.2.smithi037.stdout:Mar 29 08:15:07 smithi037 systemd[1]: Started Ceph osd.2 for 99b492c2-eda1-11ee-b647-cb9ed24678a4. 2024-03-29T08:15:08.427 INFO:journalctl@ceph.osd.2.smithi037.stdout:Mar 29 08:15:07 smithi037 ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4-osd-2[171147]: 2024-03-29T08:15:07.966+0000 7f322cb4d640 -1 Falling back to public interface 2024-03-29T08:15:08.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:15:08 smithi037 ceph-mon[142325]: pgmap v104: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 259 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 39/324 objects degraded (12.037%) 2024-03-29T08:15:08.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:15:08 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:15:08.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:15:08 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:15:08.928 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:15:08 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T08:15:09.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:15:08 smithi148 ceph-mon[106692]: pgmap v104: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 259 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 39/324 objects degraded (12.037%) 2024-03-29T08:15:09.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:15:08 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:15:09.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:15:08 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:15:09.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:15:08 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T08:15:10.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:10.610+0000 7fa61de00700 1 -- 172.21.15.37:0/2866893913 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa6180fe830 msgr2=0x7fa618106f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:15:10.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:10.610+0000 7fa61de00700 1 --2- 172.21.15.37:0/2866893913 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa6180fe830 0x7fa618106f00 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fa604004940 tx=0x7fa604009e30 comp rx=0 tx=0).stop 2024-03-29T08:15:10.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:10.610+0000 7fa61de00700 1 -- 172.21.15.37:0/2866893913 shutdown_connections 2024-03-29T08:15:10.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:10.610+0000 7fa61de00700 1 --2- 172.21.15.37:0/2866893913 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa6180fe830 0x7fa618106f00 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:15:10.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:10.610+0000 7fa61de00700 1 --2- 172.21.15.37:0/2866893913 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa6180fde90 0x7fa6180fe260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:15:10.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:10.610+0000 7fa61de00700 1 -- 172.21.15.37:0/2866893913 >> 172.21.15.37:0/2866893913 conn(0x7fa618072bc0 msgr2=0x7fa618072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:15:10.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:10.610+0000 7fa61de00700 1 -- 172.21.15.37:0/2866893913 shutdown_connections 2024-03-29T08:15:10.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:10.610+0000 7fa61de00700 1 -- 172.21.15.37:0/2866893913 wait complete. 2024-03-29T08:15:10.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:10.611+0000 7fa61de00700 1 Processor -- start 2024-03-29T08:15:10.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:10.611+0000 7fa61de00700 1 -- start start 2024-03-29T08:15:10.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:10.611+0000 7fa61de00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa6180fde90 0x7fa618115d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:15:10.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:10.611+0000 7fa61de00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa6180fe830 0x7fa618116270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:15:10.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:10.611+0000 7fa61de00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6181168c0 con 0x7fa6180fde90 2024-03-29T08:15:10.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:10.611+0000 7fa61de00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa61810fc90 con 0x7fa6180fe830 2024-03-29T08:15:10.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:10.612+0000 7fa616a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa6180fde90 0x7fa618115d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:15:10.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:10.612+0000 7fa616a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa6180fde90 0x7fa618115d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50782/0 (socket says 172.21.15.37:50782) 2024-03-29T08:15:10.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:10.612+0000 7fa616a00700 1 -- 172.21.15.37:0/1640850800 learned_addr learned my addr 172.21.15.37:0/1640850800 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:15:10.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:10.612+0000 7fa616000700 1 --2- 172.21.15.37:0/1640850800 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa6180fe830 0x7fa618116270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:15:10.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:10.612+0000 7fa616000700 1 -- 172.21.15.37:0/1640850800 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa6180fde90 msgr2=0x7fa618115d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:15:10.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:10.612+0000 7fa616000700 1 --2- 172.21.15.37:0/1640850800 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa6180fde90 0x7fa618115d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:15:10.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:10.612+0000 7fa616000700 1 -- 172.21.15.37:0/1640850800 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa604005040 con 0x7fa6180fe830 2024-03-29T08:15:10.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:10.613+0000 7fa616000700 1 --2- 172.21.15.37:0/1640850800 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa6180fe830 0x7fa618116270 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fa604000c00 tx=0x7fa604009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:15:10.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:10.613+0000 7fa60ea00700 1 -- 172.21.15.37:0/1640850800 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa60400e040 con 0x7fa6180fe830 2024-03-29T08:15:10.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:10.613+0000 7fa60ea00700 1 -- 172.21.15.37:0/1640850800 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fa60400bed0 con 0x7fa6180fe830 2024-03-29T08:15:10.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:10.613+0000 7fa61de00700 1 -- 172.21.15.37:0/1640850800 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa61810ff10 con 0x7fa6180fe830 2024-03-29T08:15:10.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:10.613+0000 7fa60ea00700 1 -- 172.21.15.37:0/1640850800 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa604013420 con 0x7fa6180fe830 2024-03-29T08:15:10.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:10.613+0000 7fa61de00700 1 -- 172.21.15.37:0/1640850800 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa618110450 con 0x7fa6180fe830 2024-03-29T08:15:10.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:10.614+0000 7fa61de00700 1 -- 172.21.15.37:0/1640850800 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa618106a20 con 0x7fa6180fe830 2024-03-29T08:15:10.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:10.615+0000 7fa60ea00700 1 -- 172.21.15.37:0/1640850800 <== mon.1 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fa60401c6c0 con 0x7fa6180fe830 2024-03-29T08:15:10.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:10.616+0000 7fa60ea00700 1 --2- 172.21.15.37:0/1640850800 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fa5fc06cc30 0x7fa5fc06f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:15:10.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:10.616+0000 7fa60ea00700 1 -- 172.21.15.37:0/1640850800 <== mon.1 v2:172.21.15.148:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fa60408cce0 con 0x7fa6180fe830 2024-03-29T08:15:10.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:10.617+0000 7fa616a00700 1 --2- 172.21.15.37:0/1640850800 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fa5fc06cc30 0x7fa5fc06f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:15:10.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:10.617+0000 7fa616a00700 1 --2- 172.21.15.37:0/1640850800 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fa5fc06cc30 0x7fa5fc06f0e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fa6080041d0 tx=0x7fa60800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:15:10.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:10.622+0000 7fa60ea00700 1 -- 172.21.15.37:0/1640850800 <== mon.1 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fa60405a7b0 con 0x7fa6180fe830 2024-03-29T08:15:10.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:15:10 smithi037 ceph-mon[142325]: pgmap v105: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 259 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 39/324 objects degraded (12.037%) 2024-03-29T08:15:10.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:15:10 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:15:10.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:15:10 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:15:10.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:10.877+0000 7fa61de00700 1 -- 172.21.15.37:0/1640850800 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa618109dd0 con 0x7fa5fc06cc30 2024-03-29T08:15:10.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:10.878+0000 7fa60ea00700 1 -- 172.21.15.37:0/1640850800 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7fa618109dd0 con 0x7fa5fc06cc30 2024-03-29T08:15:10.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:10.880+0000 7fa61de00700 1 -- 172.21.15.37:0/1640850800 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fa5fc06cc30 msgr2=0x7fa5fc06f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:15:10.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:10.880+0000 7fa61de00700 1 --2- 172.21.15.37:0/1640850800 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fa5fc06cc30 0x7fa5fc06f0e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fa6080041d0 tx=0x7fa60800b040 comp rx=0 tx=0).stop 2024-03-29T08:15:10.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:10.880+0000 7fa61de00700 1 -- 172.21.15.37:0/1640850800 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa6180fe830 msgr2=0x7fa618116270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:15:10.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:10.880+0000 7fa61de00700 1 --2- 172.21.15.37:0/1640850800 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa6180fe830 0x7fa618116270 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fa604000c00 tx=0x7fa604009e30 comp rx=0 tx=0).stop 2024-03-29T08:15:10.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:10.881+0000 7fa61de00700 1 -- 172.21.15.37:0/1640850800 shutdown_connections 2024-03-29T08:15:10.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:10.881+0000 7fa61de00700 1 --2- 172.21.15.37:0/1640850800 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fa5fc06cc30 0x7fa5fc06f0e0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:15:10.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:10.881+0000 7fa61de00700 1 --2- 172.21.15.37:0/1640850800 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa6180fde90 0x7fa618115d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:15:10.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:10.881+0000 7fa61de00700 1 --2- 172.21.15.37:0/1640850800 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa6180fe830 0x7fa618116270 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:15:10.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:10.881+0000 7fa61de00700 1 -- 172.21.15.37:0/1640850800 >> 172.21.15.37:0/1640850800 conn(0x7fa618072bc0 msgr2=0x7fa6180fb840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:15:10.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:10.881+0000 7fa61de00700 1 -- 172.21.15.37:0/1640850800 shutdown_connections 2024-03-29T08:15:10.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:10.881+0000 7fa61de00700 1 -- 172.21.15.37:0/1640850800 wait complete. 2024-03-29T08:15:10.898 INFO:teuthology.orchestra.run.smithi037.stdout:true 2024-03-29T08:15:11.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:11.031+0000 7f7973400700 1 -- 172.21.15.37:0/3754712391 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f796c102e00 msgr2=0x7f796c1031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:15:11.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:11.031+0000 7f7973400700 1 --2- 172.21.15.37:0/3754712391 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f796c102e00 0x7f796c1031d0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f7960004940 tx=0x7f7960009e30 comp rx=0 tx=0).stop 2024-03-29T08:15:11.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:11.031+0000 7f7973400700 1 -- 172.21.15.37:0/3754712391 shutdown_connections 2024-03-29T08:15:11.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:11.031+0000 7f7973400700 1 --2- 172.21.15.37:0/3754712391 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f796c1037a0 0x7f796c0fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:15:11.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:11.031+0000 7f7973400700 1 --2- 172.21.15.37:0/3754712391 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f796c102e00 0x7f796c1031d0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:15:11.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:11.031+0000 7f7973400700 1 -- 172.21.15.37:0/3754712391 >> 172.21.15.37:0/3754712391 conn(0x7f796c0f89b0 msgr2=0x7f796c0fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:15:11.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:11.031+0000 7f7973400700 1 -- 172.21.15.37:0/3754712391 shutdown_connections 2024-03-29T08:15:11.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:11.032+0000 7f7973400700 1 -- 172.21.15.37:0/3754712391 wait complete. 2024-03-29T08:15:11.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:11.033+0000 7f7973400700 1 Processor -- start 2024-03-29T08:15:11.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:11.033+0000 7f7973400700 1 -- start start 2024-03-29T08:15:11.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:11.033+0000 7f7973400700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f796c102e00 0x7f796c115c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:15:11.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:11.033+0000 7f7973400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f796c1037a0 0x7f796c1161a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:15:11.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:11.033+0000 7f7973400700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f796c1167f0 con 0x7f796c1037a0 2024-03-29T08:15:11.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:11.033+0000 7f7973400700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f796c10fc70 con 0x7f796c102e00 2024-03-29T08:15:11.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:11.034+0000 7f796be00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f796c1037a0 0x7f796c1161a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:15:11.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:11.034+0000 7f796be00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f796c1037a0 0x7f796c1161a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50804/0 (socket says 172.21.15.37:50804) 2024-03-29T08:15:11.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:11.034+0000 7f796be00700 1 -- 172.21.15.37:0/4093954240 learned_addr learned my addr 172.21.15.37:0/4093954240 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:15:11.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:11.034+0000 7f7970a00700 1 --2- 172.21.15.37:0/4093954240 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f796c102e00 0x7f796c115c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:15:11.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:11.034+0000 7f796be00700 1 -- 172.21.15.37:0/4093954240 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f796c102e00 msgr2=0x7f796c115c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:15:11.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:11.034+0000 7f796be00700 1 --2- 172.21.15.37:0/4093954240 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f796c102e00 0x7f796c115c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:15:11.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:11.034+0000 7f796be00700 1 -- 172.21.15.37:0/4093954240 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7960005040 con 0x7f796c1037a0 2024-03-29T08:15:11.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:11.035+0000 7f796be00700 1 --2- 172.21.15.37:0/4093954240 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f796c1037a0 0x7f796c1161a0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f7958002a50 tx=0x7f7958004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:15:11.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:11.035+0000 7f7968c00700 1 -- 172.21.15.37:0/4093954240 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7958007cc0 con 0x7f796c1037a0 2024-03-29T08:15:11.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:11.035+0000 7f7968c00700 1 -- 172.21.15.37:0/4093954240 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f795800f040 con 0x7f796c1037a0 2024-03-29T08:15:11.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:11.035+0000 7f7973400700 1 -- 172.21.15.37:0/4093954240 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f796c10ff50 con 0x7f796c1037a0 2024-03-29T08:15:11.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:11.035+0000 7f7968c00700 1 -- 172.21.15.37:0/4093954240 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f795800d920 con 0x7f796c1037a0 2024-03-29T08:15:11.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:11.035+0000 7f7973400700 1 -- 172.21.15.37:0/4093954240 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f796c110510 con 0x7f796c1037a0 2024-03-29T08:15:11.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:11.036+0000 7f7973400700 1 -- 172.21.15.37:0/4093954240 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f796c0fe550 con 0x7f796c1037a0 2024-03-29T08:15:11.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:11.038+0000 7f7968c00700 1 -- 172.21.15.37:0/4093954240 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f7958009680 con 0x7f796c1037a0 2024-03-29T08:15:11.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:11.039+0000 7f7968c00700 1 --2- 172.21.15.37:0/4093954240 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f795406cc30 0x7f795406f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:15:11.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:11.039+0000 7f7968c00700 1 -- 172.21.15.37:0/4093954240 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f795808bd00 con 0x7f796c1037a0 2024-03-29T08:15:11.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:11.039+0000 7f7970a00700 1 --2- 172.21.15.37:0/4093954240 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f795406cc30 0x7f795406f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:15:11.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:11.040+0000 7f7970a00700 1 --2- 172.21.15.37:0/4093954240 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f795406cc30 0x7f795406f0e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f7960009fd0 tx=0x7f7960009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:15:11.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:11.048+0000 7f7968c00700 1 -- 172.21.15.37:0/4093954240 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f7958055d40 con 0x7f796c1037a0 2024-03-29T08:15:11.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:15:10 smithi148 ceph-mon[106692]: pgmap v105: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 259 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 39/324 objects degraded (12.037%) 2024-03-29T08:15:11.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:15:10 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:15:11.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:15:10 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:15:11.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:11.295+0000 7f7973400700 1 -- 172.21.15.37:0/4093954240 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f796c109e30 con 0x7f795406cc30 2024-03-29T08:15:11.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:11.296+0000 7f7968c00700 1 -- 172.21.15.37:0/4093954240 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f796c109e30 con 0x7f795406cc30 2024-03-29T08:15:11.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:11.300+0000 7f7973400700 1 -- 172.21.15.37:0/4093954240 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f795406cc30 msgr2=0x7f795406f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:15:11.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:11.300+0000 7f7973400700 1 --2- 172.21.15.37:0/4093954240 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f795406cc30 0x7f795406f0e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f7960009fd0 tx=0x7f7960009e30 comp rx=0 tx=0).stop 2024-03-29T08:15:11.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:11.300+0000 7f7973400700 1 -- 172.21.15.37:0/4093954240 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f796c1037a0 msgr2=0x7f796c1161a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:15:11.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:11.300+0000 7f7973400700 1 --2- 172.21.15.37:0/4093954240 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f796c1037a0 0x7f796c1161a0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f7958002a50 tx=0x7f7958004600 comp rx=0 tx=0).stop 2024-03-29T08:15:11.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:11.300+0000 7f7973400700 1 -- 172.21.15.37:0/4093954240 shutdown_connections 2024-03-29T08:15:11.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:11.300+0000 7f7973400700 1 --2- 172.21.15.37:0/4093954240 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f795406cc30 0x7f795406f0e0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:15:11.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:11.300+0000 7f7973400700 1 --2- 172.21.15.37:0/4093954240 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f796c102e00 0x7f796c115c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:15:11.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:11.300+0000 7f7973400700 1 --2- 172.21.15.37:0/4093954240 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f796c1037a0 0x7f796c1161a0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:15:11.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:11.300+0000 7f7973400700 1 -- 172.21.15.37:0/4093954240 >> 172.21.15.37:0/4093954240 conn(0x7f796c0f89b0 msgr2=0x7f796c0704f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:15:11.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:11.301+0000 7f7973400700 1 -- 172.21.15.37:0/4093954240 shutdown_connections 2024-03-29T08:15:11.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:11.301+0000 7f7973400700 1 -- 172.21.15.37:0/4093954240 wait complete. 2024-03-29T08:15:11.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:11.453+0000 7fd222c00700 1 -- 172.21.15.37:0/3295879803 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd21c102e00 msgr2=0x7fd21c1031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:15:11.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:11.453+0000 7fd222c00700 1 --2- 172.21.15.37:0/3295879803 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd21c102e00 0x7fd21c1031d0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fd20c004940 tx=0x7fd20c009e30 comp rx=0 tx=0).stop 2024-03-29T08:15:11.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:11.453+0000 7fd222c00700 1 -- 172.21.15.37:0/3295879803 shutdown_connections 2024-03-29T08:15:11.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:11.453+0000 7fd222c00700 1 --2- 172.21.15.37:0/3295879803 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd21c1037a0 0x7fd21c0fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:15:11.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:11.453+0000 7fd222c00700 1 --2- 172.21.15.37:0/3295879803 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd21c102e00 0x7fd21c1031d0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:15:11.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:11.453+0000 7fd222c00700 1 -- 172.21.15.37:0/3295879803 >> 172.21.15.37:0/3295879803 conn(0x7fd21c0f89b0 msgr2=0x7fd21c0fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:15:11.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:11.453+0000 7fd222c00700 1 -- 172.21.15.37:0/3295879803 shutdown_connections 2024-03-29T08:15:11.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:11.453+0000 7fd222c00700 1 -- 172.21.15.37:0/3295879803 wait complete. 2024-03-29T08:15:11.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:11.454+0000 7fd222c00700 1 Processor -- start 2024-03-29T08:15:11.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:11.455+0000 7fd222c00700 1 -- start start 2024-03-29T08:15:11.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:11.455+0000 7fd222c00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd21c102e00 0x7fd21c115c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:15:11.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:11.455+0000 7fd222c00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd21c1037a0 0x7fd21c1161a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:15:11.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:11.455+0000 7fd222c00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd21c1167f0 con 0x7fd21c1037a0 2024-03-29T08:15:11.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:11.455+0000 7fd222c00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd21c10fbc0 con 0x7fd21c102e00 2024-03-29T08:15:11.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:11.456+0000 7fd21b400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd21c1037a0 0x7fd21c1161a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:15:11.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:11.456+0000 7fd21b400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd21c1037a0 0x7fd21c1161a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50828/0 (socket says 172.21.15.37:50828) 2024-03-29T08:15:11.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:11.456+0000 7fd21b400700 1 -- 172.21.15.37:0/322678965 learned_addr learned my addr 172.21.15.37:0/322678965 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:15:11.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:11.456+0000 7fd21be00700 1 --2- 172.21.15.37:0/322678965 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd21c102e00 0x7fd21c115c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:15:11.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:11.456+0000 7fd21be00700 1 -- 172.21.15.37:0/322678965 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd21c1037a0 msgr2=0x7fd21c1161a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:15:11.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:11.456+0000 7fd21be00700 1 --2- 172.21.15.37:0/322678965 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd21c1037a0 0x7fd21c1161a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:15:11.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:11.456+0000 7fd21be00700 1 -- 172.21.15.37:0/322678965 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd20c005040 con 0x7fd21c102e00 2024-03-29T08:15:11.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:11.457+0000 7fd21be00700 1 --2- 172.21.15.37:0/322678965 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd21c102e00 0x7fd21c115c60 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fd20c009fd0 tx=0x7fd20c009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:15:11.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:11.457+0000 7fd213e00700 1 -- 172.21.15.37:0/322678965 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd20c00e070 con 0x7fd21c102e00 2024-03-29T08:15:11.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:11.457+0000 7fd213e00700 1 -- 172.21.15.37:0/322678965 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd20c013e00 con 0x7fd21c102e00 2024-03-29T08:15:11.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:11.457+0000 7fd222c00700 1 -- 172.21.15.37:0/322678965 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd21c10fe40 con 0x7fd21c102e00 2024-03-29T08:15:11.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:11.457+0000 7fd213e00700 1 -- 172.21.15.37:0/322678965 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd20c01cd00 con 0x7fd21c102e00 2024-03-29T08:15:11.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:11.457+0000 7fd222c00700 1 -- 172.21.15.37:0/322678965 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd21c1103b0 con 0x7fd21c102e00 2024-03-29T08:15:11.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:11.458+0000 7fd222c00700 1 -- 172.21.15.37:0/322678965 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd21c0fe550 con 0x7fd21c102e00 2024-03-29T08:15:11.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:11.460+0000 7fd213e00700 1 -- 172.21.15.37:0/322678965 <== mon.1 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fd20c013950 con 0x7fd21c102e00 2024-03-29T08:15:11.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:11.461+0000 7fd213e00700 1 --2- 172.21.15.37:0/322678965 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fd20006cc30 0x7fd20006f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:15:11.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:11.461+0000 7fd213e00700 1 -- 172.21.15.37:0/322678965 <== mon.1 v2:172.21.15.148:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fd20c08d0d0 con 0x7fd21c102e00 2024-03-29T08:15:11.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:11.461+0000 7fd21b400700 1 --2- 172.21.15.37:0/322678965 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fd20006cc30 0x7fd20006f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:15:11.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:11.462+0000 7fd21b400700 1 --2- 172.21.15.37:0/322678965 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fd20006cc30 0x7fd20006f0e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fd21c111500 tx=0x7fd20800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:15:11.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:11.465+0000 7fd213e00700 1 -- 172.21.15.37:0/322678965 <== mon.1 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fd20c05ad00 con 0x7fd21c102e00 2024-03-29T08:15:11.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:11.700+0000 7fd222c00700 1 -- 172.21.15.37:0/322678965 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fd21c04bf70 con 0x7fd20006cc30 2024-03-29T08:15:11.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:11.718+0000 7fd213e00700 1 -- 172.21.15.37:0/322678965 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7fd20800f040 con 0x7fd20006cc30 2024-03-29T08:15:11.718 INFO:teuthology.orchestra.run.smithi037.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-29T08:15:11.718 INFO:teuthology.orchestra.run.smithi037.stdout:alertmanager.smithi037 smithi037 *:9093,9094 running (5m) 38s ago 17m 13.8M - 0.25.0 c8568f914cd2 2f9f9746ed0f 2024-03-29T08:15:11.718 INFO:teuthology.orchestra.run.smithi037.stdout:crash.smithi037 smithi037 running (116s) 38s ago 18m 6647k - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 178d416a3186 2024-03-29T08:15:11.718 INFO:teuthology.orchestra.run.smithi037.stdout:crash.smithi148 smithi148 running (110s) 105s ago 17m 6656k - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 23666989d58f 2024-03-29T08:15:11.718 INFO:teuthology.orchestra.run.smithi037.stdout:grafana.smithi037 smithi037 *:3000 running (4m) 38s ago 17m 74.7M - 9.4.7 954c08fa6188 6b976fe49952 2024-03-29T08:15:11.718 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi037.iifsmh smithi037 running (11m) 38s ago 11m 110M - 17.2.7-789-g4213e17c cf22eb8683be 074e1acf331f 2024-03-29T08:15:11.718 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi037.mmwgxe smithi037 running (11m) 38s ago 11m 114M - 17.2.7-789-g4213e17c cf22eb8683be 9a5b1e72de5c 2024-03-29T08:15:11.718 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi148.qukpjh smithi148 running (11m) 105s ago 11m 14.1M - 17.2.7-789-g4213e17c cf22eb8683be 8e469782a273 2024-03-29T08:15:11.719 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi148.sqteih smithi148 running (11m) 105s ago 11m 14.0M - 17.2.7-789-g4213e17c cf22eb8683be 8e68f858bb22 2024-03-29T08:15:11.719 INFO:teuthology.orchestra.run.smithi037.stdout:mgr.smithi037.wjikii smithi037 *:8443,9283,8765 running (7m) 38s ago 19m 497M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff c5a1417b9417 2024-03-29T08:15:11.719 INFO:teuthology.orchestra.run.smithi037.stdout:mgr.smithi148.onhgvm smithi148 *:8443,9283,8765 running (6m) 105s ago 16m 447M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff fcfdbce062bd 2024-03-29T08:15:11.719 INFO:teuthology.orchestra.run.smithi037.stdout:mon.smithi037 smithi037 running (3m) 38s ago 19m 45.0M 2048M 18.2.2-1015-gd0e392d9 1cc2f45c17ff eea2060783af 2024-03-29T08:15:11.719 INFO:teuthology.orchestra.run.smithi037.stdout:mon.smithi148 smithi148 running (2m) 105s ago 16m 35.5M 2048M 18.2.2-1015-gd0e392d9 1cc2f45c17ff ad4b2c5748be 2024-03-29T08:15:11.719 INFO:teuthology.orchestra.run.smithi037.stdout:node-exporter.smithi037 smithi037 *:9100 running (5m) 38s ago 18m 7465k - 1.5.0 0da6a335fe13 26b8ae9cb878 2024-03-29T08:15:11.719 INFO:teuthology.orchestra.run.smithi037.stdout:node-exporter.smithi148 smithi148 *:9100 running (5m) 105s ago 16m 7600k - 1.5.0 0da6a335fe13 98414e4e64ed 2024-03-29T08:15:11.719 INFO:teuthology.orchestra.run.smithi037.stdout:osd.0 smithi037 running (84s) 38s ago 15m 97.6M 1619M 18.2.2-1015-gd0e392d9 1cc2f45c17ff c0c584f6245c 2024-03-29T08:15:11.719 INFO:teuthology.orchestra.run.smithi037.stdout:osd.1 smithi037 running (45s) 38s ago 15m 11.0M 1619M 18.2.2-1015-gd0e392d9 1cc2f45c17ff a4a2ff4cc83d 2024-03-29T08:15:11.719 INFO:teuthology.orchestra.run.smithi037.stdout:osd.2 smithi037 starting - - - 1619M 2024-03-29T08:15:11.719 INFO:teuthology.orchestra.run.smithi037.stdout:osd.3 smithi148 running (14m) 105s ago 14m 261M 2642M 17.2.7-789-g4213e17c cf22eb8683be a0a8224bb4b0 2024-03-29T08:15:11.719 INFO:teuthology.orchestra.run.smithi037.stdout:osd.4 smithi148 running (13m) 105s ago 13m 233M 2642M 17.2.7-789-g4213e17c cf22eb8683be a00b69047ed7 2024-03-29T08:15:11.719 INFO:teuthology.orchestra.run.smithi037.stdout:osd.5 smithi148 running (13m) 105s ago 13m 234M 2642M 17.2.7-789-g4213e17c cf22eb8683be 044a3f7db7f8 2024-03-29T08:15:11.719 INFO:teuthology.orchestra.run.smithi037.stdout:prometheus.smithi037 smithi037 *:9095 running (5m) 38s ago 17m 52.0M - 2.43.0 a07b618ecd1d 30c0223ec589 2024-03-29T08:15:11.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:11.720+0000 7fd222c00700 1 -- 172.21.15.37:0/322678965 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fd20006cc30 msgr2=0x7fd20006f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:15:11.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:11.720+0000 7fd222c00700 1 --2- 172.21.15.37:0/322678965 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fd20006cc30 0x7fd20006f0e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fd21c111500 tx=0x7fd20800b040 comp rx=0 tx=0).stop 2024-03-29T08:15:11.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:11.720+0000 7fd222c00700 1 -- 172.21.15.37:0/322678965 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd21c102e00 msgr2=0x7fd21c115c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:15:11.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:11.720+0000 7fd222c00700 1 --2- 172.21.15.37:0/322678965 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd21c102e00 0x7fd21c115c60 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fd20c009fd0 tx=0x7fd20c009e30 comp rx=0 tx=0).stop 2024-03-29T08:15:11.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:11.720+0000 7fd222c00700 1 -- 172.21.15.37:0/322678965 shutdown_connections 2024-03-29T08:15:11.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:11.720+0000 7fd222c00700 1 --2- 172.21.15.37:0/322678965 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fd20006cc30 0x7fd20006f0e0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:15:11.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:11.720+0000 7fd222c00700 1 --2- 172.21.15.37:0/322678965 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd21c102e00 0x7fd21c115c60 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:15:11.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:11.720+0000 7fd222c00700 1 --2- 172.21.15.37:0/322678965 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd21c1037a0 0x7fd21c1161a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:15:11.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:11.720+0000 7fd222c00700 1 -- 172.21.15.37:0/322678965 >> 172.21.15.37:0/322678965 conn(0x7fd21c0f89b0 msgr2=0x7fd21c0f99f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:15:11.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:11.720+0000 7fd222c00700 1 -- 172.21.15.37:0/322678965 shutdown_connections 2024-03-29T08:15:11.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:11.720+0000 7fd222c00700 1 -- 172.21.15.37:0/322678965 wait complete. 2024-03-29T08:15:11.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:11.863+0000 7ff64fe00700 1 -- 172.21.15.37:0/4253459898 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff6480fc620 msgr2=0x7ff648100cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:15:11.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:11.863+0000 7ff64fe00700 1 --2- 172.21.15.37:0/4253459898 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff6480fc620 0x7ff648100cd0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7ff634004940 tx=0x7ff634009e30 comp rx=0 tx=0).stop 2024-03-29T08:15:11.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:11.863+0000 7ff64fe00700 1 -- 172.21.15.37:0/4253459898 shutdown_connections 2024-03-29T08:15:11.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:11.863+0000 7ff64fe00700 1 --2- 172.21.15.37:0/4253459898 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff6480fc620 0x7ff648100cd0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:15:11.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:11.863+0000 7ff64fe00700 1 --2- 172.21.15.37:0/4253459898 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff6480fbc80 0x7ff6480fc050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:15:11.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:11.863+0000 7ff64fe00700 1 -- 172.21.15.37:0/4253459898 >> 172.21.15.37:0/4253459898 conn(0x7ff648072bc0 msgr2=0x7ff648072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:15:11.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:11.863+0000 7ff64fe00700 1 -- 172.21.15.37:0/4253459898 shutdown_connections 2024-03-29T08:15:11.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:11.864+0000 7ff64fe00700 1 -- 172.21.15.37:0/4253459898 wait complete. 2024-03-29T08:15:11.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:11.864+0000 7ff64fe00700 1 Processor -- start 2024-03-29T08:15:11.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:11.864+0000 7ff64fe00700 1 -- start start 2024-03-29T08:15:11.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:11.865+0000 7ff64fe00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff6480fbc80 0x7ff64811a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:15:11.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:11.865+0000 7ff64fe00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff6480fc620 0x7ff64811a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:15:11.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:11.865+0000 7ff64fe00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff64811ac50 con 0x7ff6480fbc80 2024-03-29T08:15:11.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:11.865+0000 7ff64fe00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff648114070 con 0x7ff6480fc620 2024-03-29T08:15:11.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:11.865+0000 7ff64d400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff6480fbc80 0x7ff64811a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:15:11.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:11.865+0000 7ff64d400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff6480fbc80 0x7ff64811a0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50844/0 (socket says 172.21.15.37:50844) 2024-03-29T08:15:11.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:11.865+0000 7ff64d400700 1 -- 172.21.15.37:0/1394837626 learned_addr learned my addr 172.21.15.37:0/1394837626 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:15:11.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:11.865+0000 7ff64ca00700 1 --2- 172.21.15.37:0/1394837626 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff6480fc620 0x7ff64811a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:15:11.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:11.866+0000 7ff64d400700 1 -- 172.21.15.37:0/1394837626 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff6480fc620 msgr2=0x7ff64811a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:15:11.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:11.866+0000 7ff64d400700 1 --2- 172.21.15.37:0/1394837626 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff6480fc620 0x7ff64811a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:15:11.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:11.866+0000 7ff64d400700 1 -- 172.21.15.37:0/1394837626 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff634005040 con 0x7ff6480fbc80 2024-03-29T08:15:11.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:11.866+0000 7ff64d400700 1 --2- 172.21.15.37:0/1394837626 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff6480fbc80 0x7ff64811a0c0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ff63c002a50 tx=0x7ff63c0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:15:11.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:11.866+0000 7ff645600700 1 -- 172.21.15.37:0/1394837626 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff63c007cc0 con 0x7ff6480fbc80 2024-03-29T08:15:11.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:11.866+0000 7ff64fe00700 1 -- 172.21.15.37:0/1394837626 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff648114350 con 0x7ff6480fbc80 2024-03-29T08:15:11.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:11.866+0000 7ff645600700 1 -- 172.21.15.37:0/1394837626 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ff63c00f040 con 0x7ff6480fbc80 2024-03-29T08:15:11.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:11.866+0000 7ff645600700 1 -- 172.21.15.37:0/1394837626 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff63c00d9c0 con 0x7ff6480fbc80 2024-03-29T08:15:11.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:11.867+0000 7ff64fe00700 1 -- 172.21.15.37:0/1394837626 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff648114890 con 0x7ff6480fbc80 2024-03-29T08:15:11.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:11.868+0000 7ff64fe00700 1 -- 172.21.15.37:0/1394837626 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff648065e30 con 0x7ff6480fbc80 2024-03-29T08:15:11.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:11.869+0000 7ff645600700 1 -- 172.21.15.37:0/1394837626 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7ff63c00db20 con 0x7ff6480fbc80 2024-03-29T08:15:11.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:11.870+0000 7ff645600700 1 --2- 172.21.15.37:0/1394837626 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7ff63006cc30 0x7ff63006f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:15:11.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:11.870+0000 7ff645600700 1 -- 172.21.15.37:0/1394837626 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6324+0+0 (secure 0 0 0) 0x7ff63c08bcc0 con 0x7ff6480fbc80 2024-03-29T08:15:11.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:11.870+0000 7ff64ca00700 1 --2- 172.21.15.37:0/1394837626 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7ff63006cc30 0x7ff63006f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:15:11.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:11.871+0000 7ff64ca00700 1 --2- 172.21.15.37:0/1394837626 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7ff63006cc30 0x7ff63006f0e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ff648115960 tx=0x7ff634009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:15:11.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:11.875+0000 7ff645600700 1 -- 172.21.15.37:0/1394837626 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7ff63c0597b0 con 0x7ff6480fbc80 2024-03-29T08:15:12.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:12.195+0000 7ff64fe00700 1 -- 172.21.15.37:0/1394837626 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7ff64804bed0 con 0x7ff6480fbc80 2024-03-29T08:15:12.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:12.196+0000 7ff645600700 1 -- 172.21.15.37:0/1394837626 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+847 (secure 0 0 0) 0x7ff63c059340 con 0x7ff6480fbc80 2024-03-29T08:15:12.196 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-29T08:15:12.196 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": { 2024-03-29T08:15:12.196 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 2 2024-03-29T08:15:12.196 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:15:12.196 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": { 2024-03-29T08:15:12.196 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 2 2024-03-29T08:15:12.196 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:15:12.197 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": { 2024-03-29T08:15:12.197 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 17.2.7-789-g4213e17c (4213e17c4de67016ed14f55d22503f4923b3a901) quincy (stable)": 3, 2024-03-29T08:15:12.197 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 2 2024-03-29T08:15:12.197 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:15:12.197 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": { 2024-03-29T08:15:12.197 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 17.2.7-789-g4213e17c (4213e17c4de67016ed14f55d22503f4923b3a901) quincy (stable)": 4 2024-03-29T08:15:12.197 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:15:12.197 INFO:teuthology.orchestra.run.smithi037.stdout: "overall": { 2024-03-29T08:15:12.197 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 17.2.7-789-g4213e17c (4213e17c4de67016ed14f55d22503f4923b3a901) quincy (stable)": 7, 2024-03-29T08:15:12.197 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 6 2024-03-29T08:15:12.197 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-29T08:15:12.197 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-29T08:15:12.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:12.198+0000 7ff64fe00700 1 -- 172.21.15.37:0/1394837626 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7ff63006cc30 msgr2=0x7ff63006f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:15:12.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:12.198+0000 7ff64fe00700 1 --2- 172.21.15.37:0/1394837626 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7ff63006cc30 0x7ff63006f0e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ff648115960 tx=0x7ff634009e30 comp rx=0 tx=0).stop 2024-03-29T08:15:12.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:12.198+0000 7ff64fe00700 1 -- 172.21.15.37:0/1394837626 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff6480fbc80 msgr2=0x7ff64811a0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:15:12.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:12.198+0000 7ff64fe00700 1 --2- 172.21.15.37:0/1394837626 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff6480fbc80 0x7ff64811a0c0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ff63c002a50 tx=0x7ff63c0046b0 comp rx=0 tx=0).stop 2024-03-29T08:15:12.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:12.198+0000 7ff64fe00700 1 -- 172.21.15.37:0/1394837626 shutdown_connections 2024-03-29T08:15:12.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:12.198+0000 7ff64fe00700 1 --2- 172.21.15.37:0/1394837626 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7ff63006cc30 0x7ff63006f0e0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:15:12.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:12.198+0000 7ff64fe00700 1 --2- 172.21.15.37:0/1394837626 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff6480fbc80 0x7ff64811a0c0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:15:12.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:12.198+0000 7ff64fe00700 1 --2- 172.21.15.37:0/1394837626 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff6480fc620 0x7ff64811a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:15:12.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:12.198+0000 7ff64fe00700 1 -- 172.21.15.37:0/1394837626 >> 172.21.15.37:0/1394837626 conn(0x7ff648072bc0 msgr2=0x7ff64810aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:15:12.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:12.199+0000 7ff64fe00700 1 -- 172.21.15.37:0/1394837626 shutdown_connections 2024-03-29T08:15:12.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:12.199+0000 7ff64fe00700 1 -- 172.21.15.37:0/1394837626 wait complete. 2024-03-29T08:15:12.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:12.347+0000 7fabf5600700 1 -- 172.21.15.37:0/1911205846 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fabf01037a0 msgr2=0x7fabf00fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:15:12.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:12.347+0000 7fabf5600700 1 --2- 172.21.15.37:0/1911205846 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fabf01037a0 0x7fabf00fce10 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fabdc004940 tx=0x7fabdc009e30 comp rx=0 tx=0).stop 2024-03-29T08:15:12.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:12.348+0000 7fabf5600700 1 -- 172.21.15.37:0/1911205846 shutdown_connections 2024-03-29T08:15:12.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:12.348+0000 7fabf5600700 1 --2- 172.21.15.37:0/1911205846 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fabf01037a0 0x7fabf00fce10 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:15:12.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:12.348+0000 7fabf5600700 1 --2- 172.21.15.37:0/1911205846 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fabf0102e00 0x7fabf01031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:15:12.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:12.348+0000 7fabf5600700 1 -- 172.21.15.37:0/1911205846 >> 172.21.15.37:0/1911205846 conn(0x7fabf00f89b0 msgr2=0x7fabf00fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:15:12.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:12.348+0000 7fabf5600700 1 -- 172.21.15.37:0/1911205846 shutdown_connections 2024-03-29T08:15:12.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:12.348+0000 7fabf5600700 1 -- 172.21.15.37:0/1911205846 wait complete. 2024-03-29T08:15:12.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:12.349+0000 7fabf5600700 1 Processor -- start 2024-03-29T08:15:12.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:12.349+0000 7fabf5600700 1 -- start start 2024-03-29T08:15:12.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:12.349+0000 7fabf5600700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fabf0102e00 0x7fabf0115cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:15:12.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:12.349+0000 7fabf5600700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fabf01037a0 0x7fabf01161f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:15:12.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:12.349+0000 7fabf5600700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabf0116840 con 0x7fabf0102e00 2024-03-29T08:15:12.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:12.349+0000 7fabf5600700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabf010fc10 con 0x7fabf01037a0 2024-03-29T08:15:12.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:12.349+0000 7fabeea00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fabf0102e00 0x7fabf0115cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:15:12.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:12.349+0000 7fabeea00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fabf0102e00 0x7fabf0115cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50874/0 (socket says 172.21.15.37:50874) 2024-03-29T08:15:12.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:12.350+0000 7fabeea00700 1 -- 172.21.15.37:0/1580982865 learned_addr learned my addr 172.21.15.37:0/1580982865 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:15:12.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:12.350+0000 7fabee000700 1 --2- 172.21.15.37:0/1580982865 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fabf01037a0 0x7fabf01161f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:15:12.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:12.350+0000 7fabeea00700 1 -- 172.21.15.37:0/1580982865 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fabf01037a0 msgr2=0x7fabf01161f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:15:12.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:12.350+0000 7fabeea00700 1 --2- 172.21.15.37:0/1580982865 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fabf01037a0 0x7fabf01161f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:15:12.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:12.350+0000 7fabeea00700 1 -- 172.21.15.37:0/1580982865 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabdc005040 con 0x7fabf0102e00 2024-03-29T08:15:12.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:12.350+0000 7fabeea00700 1 --2- 172.21.15.37:0/1580982865 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fabf0102e00 0x7fabf0115cb0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fabe000eb40 tx=0x7fabe000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:15:12.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:12.351+0000 7fabe6a00700 1 -- 172.21.15.37:0/1580982865 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fabe0007cc0 con 0x7fabf0102e00 2024-03-29T08:15:12.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:12.351+0000 7fabe6a00700 1 -- 172.21.15.37:0/1580982865 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fabe000f040 con 0x7fabf0102e00 2024-03-29T08:15:12.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:12.351+0000 7fabf5600700 1 -- 172.21.15.37:0/1580982865 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabf010fe90 con 0x7fabf0102e00 2024-03-29T08:15:12.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:12.351+0000 7fabe6a00700 1 -- 172.21.15.37:0/1580982865 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fabe0009570 con 0x7fabf0102e00 2024-03-29T08:15:12.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:12.351+0000 7fabf5600700 1 -- 172.21.15.37:0/1580982865 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabf0110450 con 0x7fabf0102e00 2024-03-29T08:15:12.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:12.352+0000 7fabf5600700 1 -- 172.21.15.37:0/1580982865 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabf00fe550 con 0x7fabf0102e00 2024-03-29T08:15:12.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:12.353+0000 7fabe6a00700 1 -- 172.21.15.37:0/1580982865 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fabe00096d0 con 0x7fabf0102e00 2024-03-29T08:15:12.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:12.354+0000 7fabe6a00700 1 --2- 172.21.15.37:0/1580982865 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fabd406cc30 0x7fabd406f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:15:12.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:12.354+0000 7fabe6a00700 1 -- 172.21.15.37:0/1580982865 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fabe008bb90 con 0x7fabf0102e00 2024-03-29T08:15:12.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:12.357+0000 7fabee000700 1 --2- 172.21.15.37:0/1580982865 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fabd406cc30 0x7fabd406f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:15:12.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:12.357+0000 7fabee000700 1 --2- 172.21.15.37:0/1580982865 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fabd406cc30 0x7fabd406f0e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fabf0111550 tx=0x7fabdc009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:15:12.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:12.358+0000 7fabe6a00700 1 -- 172.21.15.37:0/1580982865 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fabe00597b0 con 0x7fabf0102e00 2024-03-29T08:15:12.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:12.627+0000 7fabf5600700 1 -- 172.21.15.37:0/1580982865 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fabf004bed0 con 0x7fabf0102e00 2024-03-29T08:15:12.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:12.628+0000 7fabe6a00700 1 -- 172.21.15.37:0/1580982865 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 20 v20) v1 ==== 76+0+1858 (secure 0 0 0) 0x7fabe0019760 con 0x7fabf0102e00 2024-03-29T08:15:12.628 INFO:teuthology.orchestra.run.smithi037.stdout:e20 2024-03-29T08:15:12.629 INFO:teuthology.orchestra.run.smithi037.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-29T08:15:12.629 INFO:teuthology.orchestra.run.smithi037.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T08:15:12.629 INFO:teuthology.orchestra.run.smithi037.stdout:legacy client fscid: 1 2024-03-29T08:15:12.629 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:15:12.629 INFO:teuthology.orchestra.run.smithi037.stdout:Filesystem 'cephfs' (1) 2024-03-29T08:15:12.629 INFO:teuthology.orchestra.run.smithi037.stdout:fs_name cephfs 2024-03-29T08:15:12.629 INFO:teuthology.orchestra.run.smithi037.stdout:epoch 20 2024-03-29T08:15:12.629 INFO:teuthology.orchestra.run.smithi037.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-03-29T08:15:12.629 INFO:teuthology.orchestra.run.smithi037.stdout:created 2024-03-29T08:03:29.241269+0000 2024-03-29T08:15:12.629 INFO:teuthology.orchestra.run.smithi037.stdout:modified 2024-03-29T08:11:27.181622+0000 2024-03-29T08:15:12.629 INFO:teuthology.orchestra.run.smithi037.stdout:tableserver 0 2024-03-29T08:15:12.629 INFO:teuthology.orchestra.run.smithi037.stdout:root 0 2024-03-29T08:15:12.630 INFO:teuthology.orchestra.run.smithi037.stdout:session_timeout 60 2024-03-29T08:15:12.630 INFO:teuthology.orchestra.run.smithi037.stdout:session_autoclose 300 2024-03-29T08:15:12.630 INFO:teuthology.orchestra.run.smithi037.stdout:max_file_size 1099511627776 2024-03-29T08:15:12.630 INFO:teuthology.orchestra.run.smithi037.stdout:max_xattr_size 65536 2024-03-29T08:15:12.630 INFO:teuthology.orchestra.run.smithi037.stdout:required_client_features {} 2024-03-29T08:15:12.630 INFO:teuthology.orchestra.run.smithi037.stdout:last_failure 0 2024-03-29T08:15:12.630 INFO:teuthology.orchestra.run.smithi037.stdout:last_failure_osd_epoch 47 2024-03-29T08:15:12.630 INFO:teuthology.orchestra.run.smithi037.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T08:15:12.630 INFO:teuthology.orchestra.run.smithi037.stdout:max_mds 2 2024-03-29T08:15:12.630 INFO:teuthology.orchestra.run.smithi037.stdout:in 0,1 2024-03-29T08:15:12.630 INFO:teuthology.orchestra.run.smithi037.stdout:up {0=14470,1=14482} 2024-03-29T08:15:12.630 INFO:teuthology.orchestra.run.smithi037.stdout:failed 2024-03-29T08:15:12.631 INFO:teuthology.orchestra.run.smithi037.stdout:damaged 2024-03-29T08:15:12.631 INFO:teuthology.orchestra.run.smithi037.stdout:stopped 2024-03-29T08:15:12.631 INFO:teuthology.orchestra.run.smithi037.stdout:data_pools [3] 2024-03-29T08:15:12.631 INFO:teuthology.orchestra.run.smithi037.stdout:metadata_pool 2 2024-03-29T08:15:12.631 INFO:teuthology.orchestra.run.smithi037.stdout:inline_data disabled 2024-03-29T08:15:12.631 INFO:teuthology.orchestra.run.smithi037.stdout:balancer 2024-03-29T08:15:12.631 INFO:teuthology.orchestra.run.smithi037.stdout:bal_rank_mask -1 2024-03-29T08:15:12.631 INFO:teuthology.orchestra.run.smithi037.stdout:standby_count_wanted 1 2024-03-29T08:15:12.631 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi037.mmwgxe{0:14470} state up:active seq 5 join_fscid=1 addr [v2:172.21.15.37:6826/1559618909,v1:172.21.15.37:6827/1559618909] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:15:12.631 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi037.iifsmh{1:14482} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.37:6828/2341530495,v1:172.21.15.37:6829/2341530495] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:15:12.631 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:15:12.631 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:15:12.631 INFO:teuthology.orchestra.run.smithi037.stdout:Standby daemons: 2024-03-29T08:15:12.631 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:15:12.631 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi148.qukpjh{-1:24319} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.148:6826/2641327912,v1:172.21.15.148:6827/2641327912] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:15:12.632 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi148.sqteih{-1:24327} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.148:6824/3252351726,v1:172.21.15.148:6825/3252351726] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:15:12.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:12.630+0000 7fabf5600700 1 -- 172.21.15.37:0/1580982865 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fabd406cc30 msgr2=0x7fabd406f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:15:12.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:12.630+0000 7fabf5600700 1 --2- 172.21.15.37:0/1580982865 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fabd406cc30 0x7fabd406f0e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fabf0111550 tx=0x7fabdc009e30 comp rx=0 tx=0).stop 2024-03-29T08:15:12.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:12.630+0000 7fabf5600700 1 -- 172.21.15.37:0/1580982865 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fabf0102e00 msgr2=0x7fabf0115cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:15:12.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:12.630+0000 7fabf5600700 1 --2- 172.21.15.37:0/1580982865 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fabf0102e00 0x7fabf0115cb0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fabe000eb40 tx=0x7fabe000b040 comp rx=0 tx=0).stop 2024-03-29T08:15:12.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:12.631+0000 7fabf5600700 1 -- 172.21.15.37:0/1580982865 shutdown_connections 2024-03-29T08:15:12.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:12.631+0000 7fabf5600700 1 --2- 172.21.15.37:0/1580982865 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fabd406cc30 0x7fabd406f0e0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:15:12.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:12.631+0000 7fabf5600700 1 --2- 172.21.15.37:0/1580982865 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fabf0102e00 0x7fabf0115cb0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:15:12.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:12.631+0000 7fabf5600700 1 --2- 172.21.15.37:0/1580982865 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fabf01037a0 0x7fabf01161f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:15:12.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:12.631+0000 7fabf5600700 1 -- 172.21.15.37:0/1580982865 >> 172.21.15.37:0/1580982865 conn(0x7fabf00f89b0 msgr2=0x7fabf0070480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:15:12.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:12.631+0000 7fabf5600700 1 -- 172.21.15.37:0/1580982865 shutdown_connections 2024-03-29T08:15:12.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:12.631+0000 7fabf5600700 1 -- 172.21.15.37:0/1580982865 wait complete. 2024-03-29T08:15:12.633 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 20 2024-03-29T08:15:12.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:12.785+0000 7fb57ba00700 1 -- 172.21.15.37:0/4227677121 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb5740708a0 msgr2=0x7fb574070d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:15:12.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:12.785+0000 7fb57ba00700 1 --2- 172.21.15.37:0/4227677121 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb5740708a0 0x7fb574070d10 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fb560004970 tx=0x7fb560009e30 comp rx=0 tx=0).stop 2024-03-29T08:15:12.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:12.785+0000 7fb57ba00700 1 -- 172.21.15.37:0/4227677121 shutdown_connections 2024-03-29T08:15:12.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:12.785+0000 7fb57ba00700 1 --2- 172.21.15.37:0/4227677121 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb5740708a0 0x7fb574070d10 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:15:12.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:12.785+0000 7fb57ba00700 1 --2- 172.21.15.37:0/4227677121 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb574071f90 0x7fb574070360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:15:12.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:12.785+0000 7fb57ba00700 1 -- 172.21.15.37:0/4227677121 >> 172.21.15.37:0/4227677121 conn(0x7fb5740f89b0 msgr2=0x7fb5740fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:15:12.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:12.786+0000 7fb57ba00700 1 -- 172.21.15.37:0/4227677121 shutdown_connections 2024-03-29T08:15:12.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:12.786+0000 7fb57ba00700 1 -- 172.21.15.37:0/4227677121 wait complete. 2024-03-29T08:15:12.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:12.787+0000 7fb57ba00700 1 Processor -- start 2024-03-29T08:15:12.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:12.787+0000 7fb57ba00700 1 -- start start 2024-03-29T08:15:12.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:12.788+0000 7fb57ba00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb5740708a0 0x7fb57411a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:15:12.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:12.788+0000 7fb57ba00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb574071f90 0x7fb57411a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:15:12.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:12.788+0000 7fb57ba00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb57411ac40 con 0x7fb5740708a0 2024-03-29T08:15:12.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:12.788+0000 7fb57ba00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb574114010 con 0x7fb574071f90 2024-03-29T08:15:12.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:12.789+0000 7fb579000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb5740708a0 0x7fb57411a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:15:12.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:12.789+0000 7fb579000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb5740708a0 0x7fb57411a0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50894/0 (socket says 172.21.15.37:50894) 2024-03-29T08:15:12.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:12.789+0000 7fb579000700 1 -- 172.21.15.37:0/1799842411 learned_addr learned my addr 172.21.15.37:0/1799842411 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:15:12.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:12.789+0000 7fb573e00700 1 --2- 172.21.15.37:0/1799842411 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb574071f90 0x7fb57411a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:15:12.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:12.789+0000 7fb579000700 1 -- 172.21.15.37:0/1799842411 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb574071f90 msgr2=0x7fb57411a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:15:12.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:12.789+0000 7fb579000700 1 --2- 172.21.15.37:0/1799842411 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb574071f90 0x7fb57411a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:15:12.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:12.789+0000 7fb579000700 1 -- 172.21.15.37:0/1799842411 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb560005040 con 0x7fb5740708a0 2024-03-29T08:15:12.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:12.789+0000 7fb579000700 1 --2- 172.21.15.37:0/1799842411 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb5740708a0 0x7fb57411a0b0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fb56800eb40 tx=0x7fb56800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:15:12.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:12.790+0000 7fb570c00700 1 -- 172.21.15.37:0/1799842411 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb568007cc0 con 0x7fb5740708a0 2024-03-29T08:15:12.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:12.790+0000 7fb570c00700 1 -- 172.21.15.37:0/1799842411 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fb56800f040 con 0x7fb5740708a0 2024-03-29T08:15:12.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:12.790+0000 7fb57ba00700 1 -- 172.21.15.37:0/1799842411 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb574114290 con 0x7fb5740708a0 2024-03-29T08:15:12.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:12.790+0000 7fb570c00700 1 -- 172.21.15.37:0/1799842411 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb56800c990 con 0x7fb5740708a0 2024-03-29T08:15:12.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:12.790+0000 7fb57ba00700 1 -- 172.21.15.37:0/1799842411 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5741147a0 con 0x7fb5740708a0 2024-03-29T08:15:12.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:12.791+0000 7fb57ba00700 1 -- 172.21.15.37:0/1799842411 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb574071aa0 con 0x7fb5740708a0 2024-03-29T08:15:12.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:12.792+0000 7fb570c00700 1 -- 172.21.15.37:0/1799842411 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fb5680090d0 con 0x7fb5740708a0 2024-03-29T08:15:12.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:12.793+0000 7fb570c00700 1 --2- 172.21.15.37:0/1799842411 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fb55c06ccf0 0x7fb55c06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:15:12.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:12.793+0000 7fb570c00700 1 -- 172.21.15.37:0/1799842411 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fb56808bb70 con 0x7fb5740708a0 2024-03-29T08:15:12.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:12.793+0000 7fb573e00700 1 --2- 172.21.15.37:0/1799842411 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fb55c06ccf0 0x7fb55c06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:15:12.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:12.794+0000 7fb573e00700 1 --2- 172.21.15.37:0/1799842411 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fb55c06ccf0 0x7fb55c06f1a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fb560009fd0 tx=0x7fb560009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:15:12.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:12.798+0000 7fb570c00700 1 -- 172.21.15.37:0/1799842411 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fb5680596f0 con 0x7fb5740708a0 2024-03-29T08:15:12.927 INFO:journalctl@ceph.osd.2.smithi037.stdout:Mar 29 08:15:12 smithi037 ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4-osd-2[171147]: 2024-03-29T08:15:12.774+0000 7f322cb4d640 -1 osd.2 60 log_to_monitors true 2024-03-29T08:15:12.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:15:12 smithi037 ceph-mon[142325]: from='client.44211 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:15:12.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:15:12 smithi037 ceph-mon[142325]: from='client.34286 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:15:12.928 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:15:12 smithi037 ceph-mon[142325]: pgmap v106: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 259 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 39/324 objects degraded (12.037%) 2024-03-29T08:15:12.928 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:15:12 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/1394837626' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:15:13.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:15:12 smithi148 ceph-mon[106692]: from='client.44211 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:15:13.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:15:12 smithi148 ceph-mon[106692]: from='client.34286 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:15:13.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:15:12 smithi148 ceph-mon[106692]: pgmap v106: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 259 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 39/324 objects degraded (12.037%) 2024-03-29T08:15:13.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:15:12 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/1394837626' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:15:13.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:13.097+0000 7fb57ba00700 1 -- 172.21.15.37:0/1799842411 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb57404bed0 con 0x7fb55c06ccf0 2024-03-29T08:15:13.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:13.098+0000 7fb570c00700 1 -- 172.21.15.37:0/1799842411 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7fb57404bed0 con 0x7fb55c06ccf0 2024-03-29T08:15:13.099 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-29T08:15:13.099 INFO:teuthology.orchestra.run.smithi037.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59", 2024-03-29T08:15:13.099 INFO:teuthology.orchestra.run.smithi037.stdout: "in_progress": true, 2024-03-29T08:15:13.099 INFO:teuthology.orchestra.run.smithi037.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-29T08:15:13.099 INFO:teuthology.orchestra.run.smithi037.stdout: "services_complete": [ 2024-03-29T08:15:13.099 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr", 2024-03-29T08:15:13.099 INFO:teuthology.orchestra.run.smithi037.stdout: "mon", 2024-03-29T08:15:13.099 INFO:teuthology.orchestra.run.smithi037.stdout: "crash" 2024-03-29T08:15:13.099 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-29T08:15:13.099 INFO:teuthology.orchestra.run.smithi037.stdout: "progress": "8/21 daemons upgraded", 2024-03-29T08:15:13.100 INFO:teuthology.orchestra.run.smithi037.stdout: "message": "Currently upgrading osd daemons", 2024-03-29T08:15:13.100 INFO:teuthology.orchestra.run.smithi037.stdout: "is_paused": false 2024-03-29T08:15:13.100 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-29T08:15:13.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:13.101+0000 7fb57ba00700 1 -- 172.21.15.37:0/1799842411 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fb55c06ccf0 msgr2=0x7fb55c06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:15:13.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:13.101+0000 7fb57ba00700 1 --2- 172.21.15.37:0/1799842411 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fb55c06ccf0 0x7fb55c06f1a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fb560009fd0 tx=0x7fb560009e30 comp rx=0 tx=0).stop 2024-03-29T08:15:13.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:13.101+0000 7fb57ba00700 1 -- 172.21.15.37:0/1799842411 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb5740708a0 msgr2=0x7fb57411a0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:15:13.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:13.101+0000 7fb57ba00700 1 --2- 172.21.15.37:0/1799842411 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb5740708a0 0x7fb57411a0b0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fb56800eb40 tx=0x7fb56800b040 comp rx=0 tx=0).stop 2024-03-29T08:15:13.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:13.101+0000 7fb57ba00700 1 -- 172.21.15.37:0/1799842411 shutdown_connections 2024-03-29T08:15:13.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:13.101+0000 7fb57ba00700 1 --2- 172.21.15.37:0/1799842411 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fb55c06ccf0 0x7fb55c06f1a0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:15:13.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:13.101+0000 7fb57ba00700 1 --2- 172.21.15.37:0/1799842411 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb5740708a0 0x7fb57411a0b0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:15:13.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:13.101+0000 7fb57ba00700 1 --2- 172.21.15.37:0/1799842411 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb574071f90 0x7fb57411a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:15:13.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:13.101+0000 7fb57ba00700 1 -- 172.21.15.37:0/1799842411 >> 172.21.15.37:0/1799842411 conn(0x7fb5740f89b0 msgr2=0x7fb5740f9b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:15:13.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:13.101+0000 7fb57ba00700 1 -- 172.21.15.37:0/1799842411 shutdown_connections 2024-03-29T08:15:13.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:13.101+0000 7fb57ba00700 1 -- 172.21.15.37:0/1799842411 wait complete. 2024-03-29T08:15:13.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:13.258+0000 7f38fee00700 1 -- 172.21.15.37:0/3415978069 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f38f81037a0 msgr2=0x7f38f80fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:15:13.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:13.258+0000 7f38fee00700 1 --2- 172.21.15.37:0/3415978069 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f38f81037a0 0x7f38f80fce10 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f38e4004940 tx=0x7f38e4009e30 comp rx=0 tx=0).stop 2024-03-29T08:15:13.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:13.259+0000 7f38fee00700 1 -- 172.21.15.37:0/3415978069 shutdown_connections 2024-03-29T08:15:13.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:13.259+0000 7f38fee00700 1 --2- 172.21.15.37:0/3415978069 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f38f81037a0 0x7f38f80fce10 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:15:13.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:13.259+0000 7f38fee00700 1 --2- 172.21.15.37:0/3415978069 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f38f8102e00 0x7f38f81031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:15:13.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:13.259+0000 7f38fee00700 1 -- 172.21.15.37:0/3415978069 >> 172.21.15.37:0/3415978069 conn(0x7f38f80f89b0 msgr2=0x7f38f80fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:15:13.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:13.259+0000 7f38fee00700 1 -- 172.21.15.37:0/3415978069 shutdown_connections 2024-03-29T08:15:13.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:13.259+0000 7f38fee00700 1 -- 172.21.15.37:0/3415978069 wait complete. 2024-03-29T08:15:13.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:13.259+0000 7f38fee00700 1 Processor -- start 2024-03-29T08:15:13.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:13.260+0000 7f38fee00700 1 -- start start 2024-03-29T08:15:13.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:13.260+0000 7f38fee00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f38f8102e00 0x7f38f8115cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:15:13.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:13.260+0000 7f38fee00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f38f81037a0 0x7f38f8116230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:15:13.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:13.260+0000 7f38fee00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38f8116880 con 0x7f38f8102e00 2024-03-29T08:15:13.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:13.260+0000 7f38fee00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38f810fc50 con 0x7f38f81037a0 2024-03-29T08:15:13.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:13.260+0000 7f38f7e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f38f8102e00 0x7f38f8115cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:15:13.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:13.260+0000 7f38f7e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f38f8102e00 0x7f38f8115cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50920/0 (socket says 172.21.15.37:50920) 2024-03-29T08:15:13.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:13.260+0000 7f38f7e00700 1 -- 172.21.15.37:0/1941927613 learned_addr learned my addr 172.21.15.37:0/1941927613 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:15:13.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:13.260+0000 7f38f7400700 1 --2- 172.21.15.37:0/1941927613 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f38f81037a0 0x7f38f8116230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:15:13.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:13.261+0000 7f38f7e00700 1 -- 172.21.15.37:0/1941927613 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f38f81037a0 msgr2=0x7f38f8116230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:15:13.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:13.261+0000 7f38f7e00700 1 --2- 172.21.15.37:0/1941927613 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f38f81037a0 0x7f38f8116230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:15:13.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:13.261+0000 7f38f7e00700 1 -- 172.21.15.37:0/1941927613 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38e4005040 con 0x7f38f8102e00 2024-03-29T08:15:13.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:13.261+0000 7f38f7e00700 1 --2- 172.21.15.37:0/1941927613 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f38f8102e00 0x7f38f8115cf0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f38e8002a50 tx=0x7f38e80046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:15:13.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:13.261+0000 7f38efe00700 1 -- 172.21.15.37:0/1941927613 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f38e8007cc0 con 0x7f38f8102e00 2024-03-29T08:15:13.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:13.261+0000 7f38efe00700 1 -- 172.21.15.37:0/1941927613 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f38e800f040 con 0x7f38f8102e00 2024-03-29T08:15:13.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:13.261+0000 7f38efe00700 1 -- 172.21.15.37:0/1941927613 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f38e800d9c0 con 0x7f38f8102e00 2024-03-29T08:15:13.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:13.261+0000 7f38fee00700 1 -- 172.21.15.37:0/1941927613 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38f810ff30 con 0x7f38f8102e00 2024-03-29T08:15:13.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:13.261+0000 7f38fee00700 1 -- 172.21.15.37:0/1941927613 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38f81104f0 con 0x7f38f8102e00 2024-03-29T08:15:13.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:13.263+0000 7f38fee00700 1 -- 172.21.15.37:0/1941927613 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38f80fe550 con 0x7f38f8102e00 2024-03-29T08:15:13.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:13.264+0000 7f38efe00700 1 -- 172.21.15.37:0/1941927613 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f38e800db20 con 0x7f38f8102e00 2024-03-29T08:15:13.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:13.264+0000 7f38efe00700 1 --2- 172.21.15.37:0/1941927613 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f38dc06cc30 0x7f38dc06f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:15:13.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:13.265+0000 7f38efe00700 1 -- 172.21.15.37:0/1941927613 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f38e808bca0 con 0x7f38f8102e00 2024-03-29T08:15:13.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:13.265+0000 7f38f7400700 1 --2- 172.21.15.37:0/1941927613 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f38dc06cc30 0x7f38dc06f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:15:13.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:13.265+0000 7f38f7400700 1 --2- 172.21.15.37:0/1941927613 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f38dc06cc30 0x7f38dc06f0e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f38f8111590 tx=0x7f38e4009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:15:13.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:13.269+0000 7f38efe00700 1 -- 172.21.15.37:0/1941927613 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f38e8059770 con 0x7f38f8102e00 2024-03-29T08:15:13.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:13.580+0000 7f38fee00700 1 -- 172.21.15.37:0/1941927613 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f38f804bed0 con 0x7f38f8102e00 2024-03-29T08:15:13.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:13.580+0000 7f38efe00700 1 -- 172.21.15.37:0/1941927613 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+941 (secure 0 0 0) 0x7f38e8059300 con 0x7f38f8102e00 2024-03-29T08:15:13.581 INFO:teuthology.orchestra.run.smithi037.stdout:HEALTH_WARN 1 osds down; Degraded data redundancy: 39/324 objects degraded (12.037%), 12 pgs degraded 2024-03-29T08:15:13.581 INFO:teuthology.orchestra.run.smithi037.stdout:[WRN] OSD_DOWN: 1 osds down 2024-03-29T08:15:13.581 INFO:teuthology.orchestra.run.smithi037.stdout: osd.2 (root=default,host=smithi037) is down 2024-03-29T08:15:13.581 INFO:teuthology.orchestra.run.smithi037.stdout:[WRN] PG_DEGRADED: Degraded data redundancy: 39/324 objects degraded (12.037%), 12 pgs degraded 2024-03-29T08:15:13.581 INFO:teuthology.orchestra.run.smithi037.stdout: pg 2.1 is active+undersized+degraded, acting [1,0] 2024-03-29T08:15:13.581 INFO:teuthology.orchestra.run.smithi037.stdout: pg 2.3 is active+undersized+degraded, acting [5,1] 2024-03-29T08:15:13.581 INFO:teuthology.orchestra.run.smithi037.stdout: pg 2.7 is active+undersized+degraded, acting [3,4] 2024-03-29T08:15:13.581 INFO:teuthology.orchestra.run.smithi037.stdout: pg 2.d is active+undersized+degraded, acting [1,3] 2024-03-29T08:15:13.581 INFO:teuthology.orchestra.run.smithi037.stdout: pg 2.e is active+undersized+degraded, acting [0,3] 2024-03-29T08:15:13.581 INFO:teuthology.orchestra.run.smithi037.stdout: pg 2.10 is active+undersized+degraded, acting [1,0] 2024-03-29T08:15:13.581 INFO:teuthology.orchestra.run.smithi037.stdout: pg 2.13 is active+undersized+degraded, acting [0,4] 2024-03-29T08:15:13.581 INFO:teuthology.orchestra.run.smithi037.stdout: pg 2.16 is active+undersized+degraded, acting [5,3] 2024-03-29T08:15:13.581 INFO:teuthology.orchestra.run.smithi037.stdout: pg 2.17 is active+undersized+degraded, acting [5,1] 2024-03-29T08:15:13.581 INFO:teuthology.orchestra.run.smithi037.stdout: pg 2.19 is active+undersized+degraded, acting [0,4] 2024-03-29T08:15:13.582 INFO:teuthology.orchestra.run.smithi037.stdout: pg 2.1c is active+undersized+degraded, acting [4,5] 2024-03-29T08:15:13.582 INFO:teuthology.orchestra.run.smithi037.stdout: pg 2.1e is active+undersized+degraded, acting [0,5] 2024-03-29T08:15:13.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:13.583+0000 7f38fee00700 1 -- 172.21.15.37:0/1941927613 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f38dc06cc30 msgr2=0x7f38dc06f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:15:13.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:13.583+0000 7f38fee00700 1 --2- 172.21.15.37:0/1941927613 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f38dc06cc30 0x7f38dc06f0e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f38f8111590 tx=0x7f38e4009e30 comp rx=0 tx=0).stop 2024-03-29T08:15:13.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:13.583+0000 7f38fee00700 1 -- 172.21.15.37:0/1941927613 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f38f8102e00 msgr2=0x7f38f8115cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:15:13.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:13.583+0000 7f38fee00700 1 --2- 172.21.15.37:0/1941927613 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f38f8102e00 0x7f38f8115cf0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f38e8002a50 tx=0x7f38e80046b0 comp rx=0 tx=0).stop 2024-03-29T08:15:13.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:13.583+0000 7f38fee00700 1 -- 172.21.15.37:0/1941927613 shutdown_connections 2024-03-29T08:15:13.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:13.583+0000 7f38fee00700 1 --2- 172.21.15.37:0/1941927613 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f38dc06cc30 0x7f38dc06f0e0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:15:13.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:13.583+0000 7f38fee00700 1 --2- 172.21.15.37:0/1941927613 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f38f8102e00 0x7f38f8115cf0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:15:13.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:13.583+0000 7f38fee00700 1 --2- 172.21.15.37:0/1941927613 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f38f81037a0 0x7f38f8116230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:15:13.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:13.583+0000 7f38fee00700 1 -- 172.21.15.37:0/1941927613 >> 172.21.15.37:0/1941927613 conn(0x7f38f80f89b0 msgr2=0x7f38f80f9a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:15:13.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:13.583+0000 7f38fee00700 1 -- 172.21.15.37:0/1941927613 shutdown_connections 2024-03-29T08:15:13.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:13.583+0000 7f38fee00700 1 -- 172.21.15.37:0/1941927613 wait complete. 2024-03-29T08:15:13.927 INFO:journalctl@ceph.osd.2.smithi037.stdout:Mar 29 08:15:13 smithi037 ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4-osd-2[171147]: 2024-03-29T08:15:13.612+0000 7f321be00700 -1 osd.2 60 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-29T08:15:13.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:15:13 smithi037 ceph-mon[142325]: from='client.44219 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:15:13.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:15:13 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/1580982865' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T08:15:13.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:15:13 smithi037 ceph-mon[142325]: from='osd.2 [v2:172.21.15.37:6818/4140655393,v1:172.21.15.37:6819/4140655393]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-29T08:15:13.928 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:15:13 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/1941927613' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T08:15:14.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:15:13 smithi148 ceph-mon[106692]: from='client.44219 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:15:14.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:15:13 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/1580982865' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T08:15:14.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:15:13 smithi148 ceph-mon[106692]: from='osd.2 [v2:172.21.15.37:6818/4140655393,v1:172.21.15.37:6819/4140655393]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-29T08:15:14.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:15:13 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/1941927613' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T08:15:14.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:15:14 smithi037 ceph-mon[142325]: from='client.34304 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:15:14.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:15:14 smithi037 ceph-mon[142325]: pgmap v107: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 259 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 39/324 objects degraded (12.037%) 2024-03-29T08:15:14.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:15:14 smithi037 ceph-mon[142325]: from='osd.2 [v2:172.21.15.37:6818/4140655393,v1:172.21.15.37:6819/4140655393]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-29T08:15:14.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:15:14 smithi037 ceph-mon[142325]: osdmap e63: 6 total, 5 up, 6 in 2024-03-29T08:15:14.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:15:14 smithi037 ceph-mon[142325]: from='osd.2 [v2:172.21.15.37:6818/4140655393,v1:172.21.15.37:6819/4140655393]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]: dispatch 2024-03-29T08:15:14.928 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:15:14 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:15:14.928 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:15:14 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:15:15.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:15:14 smithi148 ceph-mon[106692]: from='client.34304 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:15:15.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:15:14 smithi148 ceph-mon[106692]: pgmap v107: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 259 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 39/324 objects degraded (12.037%) 2024-03-29T08:15:15.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:15:14 smithi148 ceph-mon[106692]: from='osd.2 [v2:172.21.15.37:6818/4140655393,v1:172.21.15.37:6819/4140655393]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-29T08:15:15.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:15:14 smithi148 ceph-mon[106692]: osdmap e63: 6 total, 5 up, 6 in 2024-03-29T08:15:15.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:15:14 smithi148 ceph-mon[106692]: from='osd.2 [v2:172.21.15.37:6818/4140655393,v1:172.21.15.37:6819/4140655393]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]: dispatch 2024-03-29T08:15:15.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:15:14 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:15:15.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:15:14 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:15:15.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:15:15 smithi037 ceph-mon[142325]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-29T08:15:15.928 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:15:15 smithi037 ceph-mon[142325]: osd.2 [v2:172.21.15.37:6818/4140655393,v1:172.21.15.37:6819/4140655393] boot 2024-03-29T08:15:15.928 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:15:15 smithi037 ceph-mon[142325]: osdmap e64: 6 total, 6 up, 6 in 2024-03-29T08:15:15.928 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:15:15 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-29T08:15:15.928 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:15:15 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:15:15.928 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:15:15 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:15:15.928 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:15:15 smithi037 ceph-mon[142325]: osdmap e65: 6 total, 6 up, 6 in 2024-03-29T08:15:16.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:15:15 smithi148 ceph-mon[106692]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-29T08:15:16.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:15:15 smithi148 ceph-mon[106692]: osd.2 [v2:172.21.15.37:6818/4140655393,v1:172.21.15.37:6819/4140655393] boot 2024-03-29T08:15:16.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:15:15 smithi148 ceph-mon[106692]: osdmap e64: 6 total, 6 up, 6 in 2024-03-29T08:15:16.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:15:15 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-29T08:15:16.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:15:15 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:15:16.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:15:15 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:15:16.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:15:15 smithi148 ceph-mon[106692]: osdmap e65: 6 total, 6 up, 6 in 2024-03-29T08:15:16.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:15:16 smithi037 ceph-mon[142325]: pgmap v110: 65 pgs: 10 peering, 7 active+undersized+degraded, 10 active+undersized, 38 active+clean; 259 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 25/324 objects degraded (7.716%) 2024-03-29T08:15:16.928 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:15:16 smithi037 ceph-mon[142325]: Health check update: Degraded data redundancy: 25/324 objects degraded (7.716%), 7 pgs degraded (PG_DEGRADED) 2024-03-29T08:15:17.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:15:16 smithi148 ceph-mon[106692]: pgmap v110: 65 pgs: 10 peering, 7 active+undersized+degraded, 10 active+undersized, 38 active+clean; 259 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 25/324 objects degraded (7.716%) 2024-03-29T08:15:17.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:15:16 smithi148 ceph-mon[106692]: Health check update: Degraded data redundancy: 25/324 objects degraded (7.716%), 7 pgs degraded (PG_DEGRADED) 2024-03-29T08:15:18.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:15:18 smithi037 ceph-mon[142325]: pgmap v112: 65 pgs: 10 peering, 7 active+undersized+degraded, 10 active+undersized, 38 active+clean; 259 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 25/324 objects degraded (7.716%) 2024-03-29T08:15:19.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:15:18 smithi148 ceph-mon[106692]: pgmap v112: 65 pgs: 10 peering, 7 active+undersized+degraded, 10 active+undersized, 38 active+clean; 259 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 25/324 objects degraded (7.716%) 2024-03-29T08:15:20.270 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:15:20 smithi037 ceph-mon[142325]: pgmap v113: 65 pgs: 10 peering, 55 active+clean; 259 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:15:20.270 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:15:20 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:15:20.271 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:15:20 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:15:20.271 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:15:20 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T08:15:20.271 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:15:20 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T08:15:20.271 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:15:20 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:15:20.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:15:20 smithi148 ceph-mon[106692]: pgmap v113: 65 pgs: 10 peering, 55 active+clean; 259 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:15:20.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:15:20 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:15:20.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:15:20 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:15:20.557 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:15:20 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T08:15:20.557 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:15:20 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T08:15:20.557 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:15:20 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:15:21.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:15:21 smithi037 ceph-mon[142325]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 25/324 objects degraded (7.716%), 7 pgs degraded) 2024-03-29T08:15:21.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:15:21 smithi037 ceph-mon[142325]: Cluster is now healthy 2024-03-29T08:15:21.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:15:21 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T08:15:21.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:15:21 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:15:21.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:15:21 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:15:21.428 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:15:21 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:15:21.428 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:15:21 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-03-29T08:15:21.428 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:15:21 smithi037 ceph-mon[142325]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-03-29T08:15:21.428 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:15:21 smithi037 ceph-mon[142325]: Upgrade: unsafe to stop osd(s) at this time (5 PGs are or would become offline) 2024-03-29T08:15:21.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:15:21 smithi148 ceph-mon[106692]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 25/324 objects degraded (7.716%), 7 pgs degraded) 2024-03-29T08:15:21.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:15:21 smithi148 ceph-mon[106692]: Cluster is now healthy 2024-03-29T08:15:21.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:15:21 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T08:15:21.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:15:21 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:15:21.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:15:21 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:15:21.557 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:15:21 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:15:21.557 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:15:21 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-03-29T08:15:21.557 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:15:21 smithi148 ceph-mon[106692]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-03-29T08:15:21.557 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:15:21 smithi148 ceph-mon[106692]: Upgrade: unsafe to stop osd(s) at this time (5 PGs are or would become offline) 2024-03-29T08:15:22.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:15:22 smithi037 ceph-mon[142325]: pgmap v114: 65 pgs: 65 active+clean; 259 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:15:22.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:15:22 smithi148 ceph-mon[106692]: pgmap v114: 65 pgs: 65 active+clean; 259 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:15:24.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:15:24 smithi037 ceph-mon[142325]: pgmap v115: 65 pgs: 65 active+clean; 259 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:15:24.928 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:15:24 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:15:24.928 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:15:24 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:15:25.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:15:24 smithi148 ceph-mon[106692]: pgmap v115: 65 pgs: 65 active+clean; 259 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:15:25.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:15:24 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:15:25.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:15:24 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:15:26.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:15:26 smithi037 ceph-mon[142325]: pgmap v116: 65 pgs: 65 active+clean; 259 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:15:27.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:15:26 smithi148 ceph-mon[106692]: pgmap v116: 65 pgs: 65 active+clean; 259 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:15:28.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:15:28 smithi037 ceph-mon[142325]: pgmap v117: 65 pgs: 65 active+clean; 259 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:15:29.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:15:28 smithi148 ceph-mon[106692]: pgmap v117: 65 pgs: 65 active+clean; 259 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:15:30.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:15:30 smithi037 ceph-mon[142325]: pgmap v118: 65 pgs: 65 active+clean; 259 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:15:30.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:15:30 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:15:30.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:15:30 smithi148 ceph-mon[106692]: pgmap v118: 65 pgs: 65 active+clean; 259 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:15:30.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:15:30 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:15:32.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:15:32 smithi037 ceph-mon[142325]: pgmap v119: 65 pgs: 65 active+clean; 259 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:15:33.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:15:32 smithi148 ceph-mon[106692]: pgmap v119: 65 pgs: 65 active+clean; 259 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:15:34.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:15:34 smithi037 ceph-mon[142325]: pgmap v120: 65 pgs: 65 active+clean; 259 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:15:35.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:15:34 smithi148 ceph-mon[106692]: pgmap v120: 65 pgs: 65 active+clean; 259 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:15:35.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:15:35 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-03-29T08:15:35.991 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:15:35 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-03-29T08:15:36.715 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:15:36 smithi148 ceph-mon[106692]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-03-29T08:15:36.715 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:15:36 smithi148 ceph-mon[106692]: Upgrade: osd.3 is safe to restart 2024-03-29T08:15:36.715 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:15:36 smithi148 ceph-mon[106692]: pgmap v121: 65 pgs: 65 active+clean; 259 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:15:36.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:15:36 smithi037 ceph-mon[142325]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-03-29T08:15:36.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:15:36 smithi037 ceph-mon[142325]: Upgrade: osd.3 is safe to restart 2024-03-29T08:15:36.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:15:36 smithi037 ceph-mon[142325]: pgmap v121: 65 pgs: 65 active+clean; 259 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:15:38.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:15:38 smithi148 ceph-mon[106692]: Upgrade: Updating osd.3 2024-03-29T08:15:38.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:15:38 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:15:38.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:15:38 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-29T08:15:38.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:15:38 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T08:15:38.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:15:38 smithi148 ceph-mon[106692]: Deploying daemon osd.3 on smithi148 2024-03-29T08:15:38.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:15:38 smithi148 ceph-mon[106692]: pgmap v122: 65 pgs: 65 active+clean; 259 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:15:38.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:15:38 smithi037 ceph-mon[142325]: Upgrade: Updating osd.3 2024-03-29T08:15:38.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:15:38 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:15:38.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:15:38 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-29T08:15:38.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:15:38 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T08:15:38.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:15:38 smithi037 ceph-mon[142325]: Deploying daemon osd.3 on smithi148 2024-03-29T08:15:38.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:15:38 smithi037 ceph-mon[142325]: pgmap v122: 65 pgs: 65 active+clean; 259 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:15:40.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:15:40 smithi148 ceph-mon[106692]: pgmap v123: 65 pgs: 65 active+clean; 259 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:15:40.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:15:40 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:15:40.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:15:40 smithi037 ceph-mon[142325]: pgmap v123: 65 pgs: 65 active+clean; 259 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:15:40.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:15:40 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:15:41.306 INFO:journalctl@ceph.osd.3.smithi148.stdout:Mar 29 08:15:41 smithi148 systemd[1]: Stopping Ceph osd.3 for 99b492c2-eda1-11ee-b647-cb9ed24678a4... 2024-03-29T08:15:41.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:15:41 smithi037 ceph-mon[142325]: pgmap v124: 65 pgs: 65 active+clean; 259 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:15:41.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:15:41 smithi037 ceph-mon[142325]: osd.3 marked itself down and dead 2024-03-29T08:15:42.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:15:41 smithi148 ceph-mon[106692]: pgmap v124: 65 pgs: 65 active+clean; 259 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:15:42.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:15:41 smithi148 ceph-mon[106692]: osd.3 marked itself down and dead 2024-03-29T08:15:42.057 INFO:journalctl@ceph.osd.3.smithi148.stdout:Mar 29 08:15:41 smithi148 ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4-osd-3[51296]: 2024-03-29T08:15:41.597+0000 7efe7c400700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-29T08:15:42.057 INFO:journalctl@ceph.osd.3.smithi148.stdout:Mar 29 08:15:41 smithi148 ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4-osd-3[51296]: 2024-03-29T08:15:41.597+0000 7efe7c400700 -1 osd.3 65 *** Got signal Terminated *** 2024-03-29T08:15:42.057 INFO:journalctl@ceph.osd.3.smithi148.stdout:Mar 29 08:15:41 smithi148 ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4-osd-3[51296]: 2024-03-29T08:15:41.597+0000 7efe7c400700 -1 osd.3 65 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-29T08:15:42.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:15:42 smithi148 ceph-mon[106692]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-29T08:15:42.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:15:42 smithi148 ceph-mon[106692]: osdmap e66: 6 total, 5 up, 6 in 2024-03-29T08:15:42.806 INFO:journalctl@ceph.osd.3.smithi148.stdout:Mar 29 08:15:42 smithi148 bash[118149]: ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4-osd-3 2024-03-29T08:15:42.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:15:42 smithi037 ceph-mon[142325]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-29T08:15:42.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:15:42 smithi037 ceph-mon[142325]: osdmap e66: 6 total, 5 up, 6 in 2024-03-29T08:15:43.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:43.743+0000 7facdd600700 1 -- 172.21.15.37:0/1836611440 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7facd80fe830 msgr2=0x7facd8106f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:15:43.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:43.743+0000 7facdd600700 1 --2- 172.21.15.37:0/1836611440 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7facd80fe830 0x7facd8106f00 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7facc4004940 tx=0x7facc4009e30 comp rx=0 tx=0).stop 2024-03-29T08:15:43.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:43.743+0000 7facdd600700 1 -- 172.21.15.37:0/1836611440 shutdown_connections 2024-03-29T08:15:43.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:43.743+0000 7facdd600700 1 --2- 172.21.15.37:0/1836611440 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7facd80fe830 0x7facd8106f00 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:15:43.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:43.743+0000 7facdd600700 1 --2- 172.21.15.37:0/1836611440 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7facd80fde90 0x7facd80fe260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:15:43.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:43.743+0000 7facdd600700 1 -- 172.21.15.37:0/1836611440 >> 172.21.15.37:0/1836611440 conn(0x7facd8072bc0 msgr2=0x7facd8072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:15:43.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:43.744+0000 7facdd600700 1 -- 172.21.15.37:0/1836611440 shutdown_connections 2024-03-29T08:15:43.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:43.744+0000 7facdd600700 1 -- 172.21.15.37:0/1836611440 wait complete. 2024-03-29T08:15:43.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:43.744+0000 7facdd600700 1 Processor -- start 2024-03-29T08:15:43.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:43.745+0000 7facdd600700 1 -- start start 2024-03-29T08:15:43.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:43.745+0000 7facdd600700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7facd80fde90 0x7facd8115d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:15:43.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:43.745+0000 7facdd600700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7facd80fe830 0x7facd8116270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:15:43.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:43.745+0000 7facdd600700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facd81168c0 con 0x7facd80fe830 2024-03-29T08:15:43.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:43.745+0000 7facdd600700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facd810fc90 con 0x7facd80fde90 2024-03-29T08:15:43.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:43.745+0000 7facd6000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7facd80fe830 0x7facd8116270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:15:43.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:43.745+0000 7facd6000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7facd80fe830 0x7facd8116270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34070/0 (socket says 172.21.15.37:34070) 2024-03-29T08:15:43.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:43.745+0000 7facd6000700 1 -- 172.21.15.37:0/1243493289 learned_addr learned my addr 172.21.15.37:0/1243493289 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:15:43.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:43.745+0000 7facd6a00700 1 --2- 172.21.15.37:0/1243493289 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7facd80fde90 0x7facd8115d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:15:43.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:43.746+0000 7facd6000700 1 -- 172.21.15.37:0/1243493289 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7facd80fde90 msgr2=0x7facd8115d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:15:43.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:43.746+0000 7facd6000700 1 --2- 172.21.15.37:0/1243493289 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7facd80fde90 0x7facd8115d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:15:43.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:43.746+0000 7facd6000700 1 -- 172.21.15.37:0/1243493289 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facc4005040 con 0x7facd80fe830 2024-03-29T08:15:43.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:43.746+0000 7facd6a00700 1 --2- 172.21.15.37:0/1243493289 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7facd80fde90 0x7facd8115d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:15:43.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:43.746+0000 7facd6000700 1 --2- 172.21.15.37:0/1243493289 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7facd80fe830 0x7facd8116270 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7facc4000c00 tx=0x7facc4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:15:43.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:43.746+0000 7faccea00700 1 -- 172.21.15.37:0/1243493289 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7facc400e040 con 0x7facd80fe830 2024-03-29T08:15:43.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:43.746+0000 7faccea00700 1 -- 172.21.15.37:0/1243493289 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7facc4013420 con 0x7facd80fe830 2024-03-29T08:15:43.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:43.746+0000 7facdd600700 1 -- 172.21.15.37:0/1243493289 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7facd810ff10 con 0x7facd80fe830 2024-03-29T08:15:43.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:43.746+0000 7faccea00700 1 -- 172.21.15.37:0/1243493289 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7facc401c630 con 0x7facd80fe830 2024-03-29T08:15:43.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:43.746+0000 7facdd600700 1 -- 172.21.15.37:0/1243493289 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7facd8110450 con 0x7facd80fe830 2024-03-29T08:15:43.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:43.747+0000 7facdd600700 1 -- 172.21.15.37:0/1243493289 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7facd8106a20 con 0x7facd80fe830 2024-03-29T08:15:43.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:43.748+0000 7faccea00700 1 -- 172.21.15.37:0/1243493289 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7facc4013970 con 0x7facd80fe830 2024-03-29T08:15:43.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:43.749+0000 7faccea00700 1 --2- 172.21.15.37:0/1243493289 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7facbc06cc30 0x7facbc06f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:15:43.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:43.749+0000 7faccea00700 1 -- 172.21.15.37:0/1243493289 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6324+0+0 (secure 0 0 0) 0x7facc408d1c0 con 0x7facd80fe830 2024-03-29T08:15:43.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:43.749+0000 7facd6a00700 1 --2- 172.21.15.37:0/1243493289 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7facbc06cc30 0x7facbc06f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:15:43.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:43.750+0000 7facd6a00700 1 --2- 172.21.15.37:0/1243493289 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7facbc06cc30 0x7facbc06f0e0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7facc8009fd0 tx=0x7facc8009ee0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:15:43.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:43.754+0000 7faccea00700 1 -- 172.21.15.37:0/1243493289 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7facc405ad10 con 0x7facd80fe830 2024-03-29T08:15:43.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:15:43 smithi037 ceph-mon[142325]: osdmap e67: 6 total, 5 up, 6 in 2024-03-29T08:15:43.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:15:43 smithi037 ceph-mon[142325]: pgmap v127: 65 pgs: 16 stale+active+clean, 49 active+clean; 259 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:15:43.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:43.987+0000 7facdd600700 1 -- 172.21.15.37:0/1243493289 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7facd81113f0 con 0x7facbc06cc30 2024-03-29T08:15:43.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:43.989+0000 7faccea00700 1 -- 172.21.15.37:0/1243493289 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7facd81113f0 con 0x7facbc06cc30 2024-03-29T08:15:43.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:43.991+0000 7facdd600700 1 -- 172.21.15.37:0/1243493289 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7facbc06cc30 msgr2=0x7facbc06f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:15:43.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:43.991+0000 7facdd600700 1 --2- 172.21.15.37:0/1243493289 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7facbc06cc30 0x7facbc06f0e0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7facc8009fd0 tx=0x7facc8009ee0 comp rx=0 tx=0).stop 2024-03-29T08:15:43.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:43.991+0000 7facdd600700 1 -- 172.21.15.37:0/1243493289 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7facd80fe830 msgr2=0x7facd8116270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:15:43.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:43.991+0000 7facdd600700 1 --2- 172.21.15.37:0/1243493289 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7facd80fe830 0x7facd8116270 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7facc4000c00 tx=0x7facc4009e30 comp rx=0 tx=0).stop 2024-03-29T08:15:43.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:43.991+0000 7facdd600700 1 -- 172.21.15.37:0/1243493289 shutdown_connections 2024-03-29T08:15:43.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:43.991+0000 7facdd600700 1 --2- 172.21.15.37:0/1243493289 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7facbc06cc30 0x7facbc06f0e0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:15:43.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:43.991+0000 7facdd600700 1 --2- 172.21.15.37:0/1243493289 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7facd80fde90 0x7facd8115d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:15:43.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:43.991+0000 7facdd600700 1 --2- 172.21.15.37:0/1243493289 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7facd80fe830 0x7facd8116270 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:15:43.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:43.991+0000 7facdd600700 1 -- 172.21.15.37:0/1243493289 >> 172.21.15.37:0/1243493289 conn(0x7facd8072bc0 msgr2=0x7facd80fb840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:15:43.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:43.991+0000 7facdd600700 1 -- 172.21.15.37:0/1243493289 shutdown_connections 2024-03-29T08:15:43.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:43.991+0000 7facdd600700 1 -- 172.21.15.37:0/1243493289 wait complete. 2024-03-29T08:15:44.006 INFO:teuthology.orchestra.run.smithi037.stdout:true 2024-03-29T08:15:44.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:15:43 smithi148 ceph-mon[106692]: osdmap e67: 6 total, 5 up, 6 in 2024-03-29T08:15:44.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:15:43 smithi148 ceph-mon[106692]: pgmap v127: 65 pgs: 16 stale+active+clean, 49 active+clean; 259 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:15:44.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:44.130+0000 7fe2b1600700 1 -- 172.21.15.37:0/534246929 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe2ac1035e0 msgr2=0x7fe2ac0fcc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:15:44.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:44.130+0000 7fe2b1600700 1 --2- 172.21.15.37:0/534246929 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe2ac1035e0 0x7fe2ac0fcc50 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fe298004940 tx=0x7fe298009e30 comp rx=0 tx=0).stop 2024-03-29T08:15:44.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:44.131+0000 7fe2b1600700 1 -- 172.21.15.37:0/534246929 shutdown_connections 2024-03-29T08:15:44.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:44.131+0000 7fe2b1600700 1 --2- 172.21.15.37:0/534246929 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe2ac1035e0 0x7fe2ac0fcc50 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:15:44.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:44.131+0000 7fe2b1600700 1 --2- 172.21.15.37:0/534246929 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe2ac102c40 0x7fe2ac103010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:15:44.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:44.131+0000 7fe2b1600700 1 -- 172.21.15.37:0/534246929 >> 172.21.15.37:0/534246929 conn(0x7fe2ac0f8830 msgr2=0x7fe2ac0fac40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:15:44.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:44.131+0000 7fe2b1600700 1 -- 172.21.15.37:0/534246929 shutdown_connections 2024-03-29T08:15:44.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:44.131+0000 7fe2b1600700 1 -- 172.21.15.37:0/534246929 wait complete. 2024-03-29T08:15:44.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:44.131+0000 7fe2b1600700 1 Processor -- start 2024-03-29T08:15:44.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:44.132+0000 7fe2b1600700 1 -- start start 2024-03-29T08:15:44.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:44.132+0000 7fe2b1600700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe2ac102c40 0x7fe2ac115a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:15:44.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:44.132+0000 7fe2b1600700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe2ac1035e0 0x7fe2ac115fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:15:44.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:44.132+0000 7fe2b1600700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2ac116610 con 0x7fe2ac102c40 2024-03-29T08:15:44.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:44.132+0000 7fe2b1600700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2ac10fa00 con 0x7fe2ac1035e0 2024-03-29T08:15:44.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:44.132+0000 7fe2abe00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe2ac102c40 0x7fe2ac115a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:15:44.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:44.132+0000 7fe2abe00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe2ac102c40 0x7fe2ac115a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34084/0 (socket says 172.21.15.37:34084) 2024-03-29T08:15:44.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:44.132+0000 7fe2abe00700 1 -- 172.21.15.37:0/350354911 learned_addr learned my addr 172.21.15.37:0/350354911 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:15:44.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:44.132+0000 7fe2ab400700 1 --2- 172.21.15.37:0/350354911 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe2ac1035e0 0x7fe2ac115fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:15:44.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:44.133+0000 7fe2abe00700 1 -- 172.21.15.37:0/350354911 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe2ac1035e0 msgr2=0x7fe2ac115fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:15:44.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:44.133+0000 7fe2abe00700 1 --2- 172.21.15.37:0/350354911 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe2ac1035e0 0x7fe2ac115fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:15:44.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:44.133+0000 7fe2abe00700 1 -- 172.21.15.37:0/350354911 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe298005040 con 0x7fe2ac102c40 2024-03-29T08:15:44.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:44.133+0000 7fe2abe00700 1 --2- 172.21.15.37:0/350354911 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe2ac102c40 0x7fe2ac115a80 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fe29c00eb40 tx=0x7fe29c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:15:44.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:44.133+0000 7fe2a3e00700 1 -- 172.21.15.37:0/350354911 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe29c007cc0 con 0x7fe2ac102c40 2024-03-29T08:15:44.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:44.133+0000 7fe2a3e00700 1 -- 172.21.15.37:0/350354911 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fe29c00f040 con 0x7fe2ac102c40 2024-03-29T08:15:44.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:44.133+0000 7fe2b1600700 1 -- 172.21.15.37:0/350354911 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2ac10fc80 con 0x7fe2ac102c40 2024-03-29T08:15:44.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:44.133+0000 7fe2a3e00700 1 -- 172.21.15.37:0/350354911 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe29c00c990 con 0x7fe2ac102c40 2024-03-29T08:15:44.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:44.133+0000 7fe2b1600700 1 -- 172.21.15.37:0/350354911 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2ac110210 con 0x7fe2ac102c40 2024-03-29T08:15:44.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:44.134+0000 7fe2b1600700 1 -- 172.21.15.37:0/350354911 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2ac0fe390 con 0x7fe2ac102c40 2024-03-29T08:15:44.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:44.135+0000 7fe2a3e00700 1 -- 172.21.15.37:0/350354911 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fe29c00cc00 con 0x7fe2ac102c40 2024-03-29T08:15:44.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:44.136+0000 7fe2a3e00700 1 --2- 172.21.15.37:0/350354911 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fe29006cc20 0x7fe29006f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:15:44.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:44.136+0000 7fe2a3e00700 1 -- 172.21.15.37:0/350354911 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fe29c08bc50 con 0x7fe2ac102c40 2024-03-29T08:15:44.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:44.137+0000 7fe2ab400700 1 --2- 172.21.15.37:0/350354911 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fe29006cc20 0x7fe29006f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:15:44.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:44.137+0000 7fe2ab400700 1 --2- 172.21.15.37:0/350354911 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fe29006cc20 0x7fe29006f0d0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fe2ac110ce0 tx=0x7fe298009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:15:44.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:44.141+0000 7fe2a3e00700 1 -- 172.21.15.37:0/350354911 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fe29c059750 con 0x7fe2ac102c40 2024-03-29T08:15:44.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:44.372+0000 7fe2b1600700 1 -- 172.21.15.37:0/350354911 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe2ac109bd0 con 0x7fe29006cc20 2024-03-29T08:15:44.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:44.374+0000 7fe2a3e00700 1 -- 172.21.15.37:0/350354911 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7fe2ac109bd0 con 0x7fe29006cc20 2024-03-29T08:15:44.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:44.376+0000 7fe2b1600700 1 -- 172.21.15.37:0/350354911 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fe29006cc20 msgr2=0x7fe29006f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:15:44.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:44.376+0000 7fe2b1600700 1 --2- 172.21.15.37:0/350354911 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fe29006cc20 0x7fe29006f0d0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fe2ac110ce0 tx=0x7fe298009e30 comp rx=0 tx=0).stop 2024-03-29T08:15:44.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:44.376+0000 7fe2b1600700 1 -- 172.21.15.37:0/350354911 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe2ac102c40 msgr2=0x7fe2ac115a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:15:44.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:44.376+0000 7fe2b1600700 1 --2- 172.21.15.37:0/350354911 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe2ac102c40 0x7fe2ac115a80 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fe29c00eb40 tx=0x7fe29c00b040 comp rx=0 tx=0).stop 2024-03-29T08:15:44.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:44.376+0000 7fe2b1600700 1 -- 172.21.15.37:0/350354911 shutdown_connections 2024-03-29T08:15:44.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:44.376+0000 7fe2b1600700 1 --2- 172.21.15.37:0/350354911 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fe29006cc20 0x7fe29006f0d0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:15:44.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:44.376+0000 7fe2b1600700 1 --2- 172.21.15.37:0/350354911 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe2ac102c40 0x7fe2ac115a80 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:15:44.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:44.376+0000 7fe2b1600700 1 --2- 172.21.15.37:0/350354911 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe2ac1035e0 0x7fe2ac115fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:15:44.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:44.376+0000 7fe2b1600700 1 -- 172.21.15.37:0/350354911 >> 172.21.15.37:0/350354911 conn(0x7fe2ac0f8830 msgr2=0x7fe2ac0f9810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:15:44.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:44.376+0000 7fe2b1600700 1 -- 172.21.15.37:0/350354911 shutdown_connections 2024-03-29T08:15:44.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:44.376+0000 7fe2b1600700 1 -- 172.21.15.37:0/350354911 wait complete. 2024-03-29T08:15:44.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:44.512+0000 7f7ec9200700 1 -- 172.21.15.37:0/2530013877 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ec41037a0 msgr2=0x7f7ec40fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:15:44.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:44.512+0000 7f7ec9200700 1 --2- 172.21.15.37:0/2530013877 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ec41037a0 0x7f7ec40fce10 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f7eb0004940 tx=0x7f7eb0009e30 comp rx=0 tx=0).stop 2024-03-29T08:15:44.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:44.512+0000 7f7ec9200700 1 -- 172.21.15.37:0/2530013877 shutdown_connections 2024-03-29T08:15:44.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:44.512+0000 7f7ec9200700 1 --2- 172.21.15.37:0/2530013877 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ec41037a0 0x7f7ec40fce10 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:15:44.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:44.512+0000 7f7ec9200700 1 --2- 172.21.15.37:0/2530013877 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7ec4102e00 0x7f7ec41031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:15:44.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:44.512+0000 7f7ec9200700 1 -- 172.21.15.37:0/2530013877 >> 172.21.15.37:0/2530013877 conn(0x7f7ec40f89b0 msgr2=0x7f7ec40fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:15:44.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:44.513+0000 7f7ec9200700 1 -- 172.21.15.37:0/2530013877 shutdown_connections 2024-03-29T08:15:44.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:44.513+0000 7f7ec9200700 1 -- 172.21.15.37:0/2530013877 wait complete. 2024-03-29T08:15:44.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:44.513+0000 7f7ec9200700 1 Processor -- start 2024-03-29T08:15:44.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:44.514+0000 7f7ec9200700 1 -- start start 2024-03-29T08:15:44.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:44.514+0000 7f7ec9200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ec4102e00 0x7f7ec4115c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:15:44.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:44.514+0000 7f7ec9200700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7ec41037a0 0x7f7ec41161a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:15:44.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:44.514+0000 7f7ec9200700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ec41167f0 con 0x7f7ec4102e00 2024-03-29T08:15:44.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:44.514+0000 7f7ec9200700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ec410fbc0 con 0x7f7ec41037a0 2024-03-29T08:15:44.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:44.514+0000 7f7ec1e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ec4102e00 0x7f7ec4115c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:15:44.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:44.514+0000 7f7ec1e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ec4102e00 0x7f7ec4115c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34098/0 (socket says 172.21.15.37:34098) 2024-03-29T08:15:44.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:44.514+0000 7f7ec1e00700 1 -- 172.21.15.37:0/380704885 learned_addr learned my addr 172.21.15.37:0/380704885 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:15:44.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:44.514+0000 7f7ec1400700 1 --2- 172.21.15.37:0/380704885 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7ec41037a0 0x7f7ec41161a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:15:44.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:44.515+0000 7f7ec1e00700 1 -- 172.21.15.37:0/380704885 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7ec41037a0 msgr2=0x7f7ec41161a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:15:44.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:44.515+0000 7f7ec1e00700 1 --2- 172.21.15.37:0/380704885 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7ec41037a0 0x7f7ec41161a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:15:44.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:44.515+0000 7f7ec1e00700 1 -- 172.21.15.37:0/380704885 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7eb0005040 con 0x7f7ec4102e00 2024-03-29T08:15:44.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:44.515+0000 7f7ec1e00700 1 --2- 172.21.15.37:0/380704885 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ec4102e00 0x7f7ec4115c60 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f7eb400eb40 tx=0x7f7eb400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:15:44.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:44.515+0000 7f7eba000700 1 -- 172.21.15.37:0/380704885 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7eb4007cc0 con 0x7f7ec4102e00 2024-03-29T08:15:44.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:44.515+0000 7f7eba000700 1 -- 172.21.15.37:0/380704885 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f7eb400f040 con 0x7f7ec4102e00 2024-03-29T08:15:44.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:44.515+0000 7f7ec9200700 1 -- 172.21.15.37:0/380704885 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ec410fe40 con 0x7f7ec4102e00 2024-03-29T08:15:44.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:44.515+0000 7f7eba000700 1 -- 172.21.15.37:0/380704885 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7eb400c990 con 0x7f7ec4102e00 2024-03-29T08:15:44.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:44.515+0000 7f7ec9200700 1 -- 172.21.15.37:0/380704885 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ec4110400 con 0x7f7ec4102e00 2024-03-29T08:15:44.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:44.516+0000 7f7ec9200700 1 -- 172.21.15.37:0/380704885 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ec40fe550 con 0x7f7ec4102e00 2024-03-29T08:15:44.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:44.517+0000 7f7eba000700 1 -- 172.21.15.37:0/380704885 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f7eb400cc00 con 0x7f7ec4102e00 2024-03-29T08:15:44.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:44.518+0000 7f7eba000700 1 --2- 172.21.15.37:0/380704885 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f7ea806cc30 0x7f7ea806f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:15:44.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:44.518+0000 7f7eba000700 1 -- 172.21.15.37:0/380704885 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f7eb408bbb0 con 0x7f7ec4102e00 2024-03-29T08:15:44.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:44.518+0000 7f7ec1400700 1 --2- 172.21.15.37:0/380704885 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f7ea806cc30 0x7f7ea806f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:15:44.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:44.519+0000 7f7ec1400700 1 --2- 172.21.15.37:0/380704885 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f7ea806cc30 0x7f7ea806f0e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f7ec4111500 tx=0x7f7eb0009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:15:44.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:44.523+0000 7f7eba000700 1 -- 172.21.15.37:0/380704885 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f7eb4092080 con 0x7f7ec4102e00 2024-03-29T08:15:44.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:44.746+0000 7f7ec9200700 1 -- 172.21.15.37:0/380704885 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f7ec404bf40 con 0x7f7ea806cc30 2024-03-29T08:15:44.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:44.756+0000 7f7eba000700 1 -- 172.21.15.37:0/380704885 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7f7ec404bf40 con 0x7f7ea806cc30 2024-03-29T08:15:44.756 INFO:teuthology.orchestra.run.smithi037.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-29T08:15:44.756 INFO:teuthology.orchestra.run.smithi037.stdout:alertmanager.smithi037 smithi037 *:9093,9094 running (5m) 30s ago 18m 14.1M - 0.25.0 c8568f914cd2 2f9f9746ed0f 2024-03-29T08:15:44.757 INFO:teuthology.orchestra.run.smithi037.stdout:crash.smithi037 smithi037 running (2m) 30s ago 18m 6647k - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 178d416a3186 2024-03-29T08:15:44.757 INFO:teuthology.orchestra.run.smithi037.stdout:crash.smithi148 smithi148 running (2m) 2m ago 17m 6656k - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 23666989d58f 2024-03-29T08:15:44.757 INFO:teuthology.orchestra.run.smithi037.stdout:grafana.smithi037 smithi037 *:3000 running (5m) 30s ago 18m 73.4M - 9.4.7 954c08fa6188 6b976fe49952 2024-03-29T08:15:44.757 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi037.iifsmh smithi037 running (12m) 30s ago 12m 110M - 17.2.7-789-g4213e17c cf22eb8683be 074e1acf331f 2024-03-29T08:15:44.757 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi037.mmwgxe smithi037 running (12m) 30s ago 12m 114M - 17.2.7-789-g4213e17c cf22eb8683be 9a5b1e72de5c 2024-03-29T08:15:44.757 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi148.qukpjh smithi148 running (11m) 2m ago 11m 14.1M - 17.2.7-789-g4213e17c cf22eb8683be 8e469782a273 2024-03-29T08:15:44.757 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi148.sqteih smithi148 running (12m) 2m ago 12m 14.0M - 17.2.7-789-g4213e17c cf22eb8683be 8e68f858bb22 2024-03-29T08:15:44.757 INFO:teuthology.orchestra.run.smithi037.stdout:mgr.smithi037.wjikii smithi037 *:8443,9283,8765 running (7m) 30s ago 20m 501M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff c5a1417b9417 2024-03-29T08:15:44.757 INFO:teuthology.orchestra.run.smithi037.stdout:mgr.smithi148.onhgvm smithi148 *:8443,9283,8765 running (6m) 2m ago 17m 447M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff fcfdbce062bd 2024-03-29T08:15:44.757 INFO:teuthology.orchestra.run.smithi037.stdout:mon.smithi037 smithi037 running (3m) 30s ago 20m 47.5M 2048M 18.2.2-1015-gd0e392d9 1cc2f45c17ff eea2060783af 2024-03-29T08:15:44.757 INFO:teuthology.orchestra.run.smithi037.stdout:mon.smithi148 smithi148 running (3m) 2m ago 17m 35.5M 2048M 18.2.2-1015-gd0e392d9 1cc2f45c17ff ad4b2c5748be 2024-03-29T08:15:44.757 INFO:teuthology.orchestra.run.smithi037.stdout:node-exporter.smithi037 smithi037 *:9100 running (6m) 30s ago 18m 7847k - 1.5.0 0da6a335fe13 26b8ae9cb878 2024-03-29T08:15:44.757 INFO:teuthology.orchestra.run.smithi037.stdout:node-exporter.smithi148 smithi148 *:9100 running (6m) 2m ago 17m 7600k - 1.5.0 0da6a335fe13 98414e4e64ed 2024-03-29T08:15:44.757 INFO:teuthology.orchestra.run.smithi037.stdout:osd.0 smithi037 running (117s) 30s ago 16m 99.0M 1619M 18.2.2-1015-gd0e392d9 1cc2f45c17ff c0c584f6245c 2024-03-29T08:15:44.757 INFO:teuthology.orchestra.run.smithi037.stdout:osd.1 smithi037 running (78s) 30s ago 15m 97.2M 1619M 18.2.2-1015-gd0e392d9 1cc2f45c17ff a4a2ff4cc83d 2024-03-29T08:15:44.757 INFO:teuthology.orchestra.run.smithi037.stdout:osd.2 smithi037 running (37s) 30s ago 15m 11.0M 1619M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 8b1bc71aa79e 2024-03-29T08:15:44.757 INFO:teuthology.orchestra.run.smithi037.stdout:osd.3 smithi148 running (14m) 2m ago 14m 261M 2642M 17.2.7-789-g4213e17c cf22eb8683be a0a8224bb4b0 2024-03-29T08:15:44.758 INFO:teuthology.orchestra.run.smithi037.stdout:osd.4 smithi148 running (14m) 2m ago 14m 233M 2642M 17.2.7-789-g4213e17c cf22eb8683be a00b69047ed7 2024-03-29T08:15:44.758 INFO:teuthology.orchestra.run.smithi037.stdout:osd.5 smithi148 running (13m) 2m ago 13m 234M 2642M 17.2.7-789-g4213e17c cf22eb8683be 044a3f7db7f8 2024-03-29T08:15:44.758 INFO:teuthology.orchestra.run.smithi037.stdout:prometheus.smithi037 smithi037 *:9095 running (5m) 30s ago 17m 52.1M - 2.43.0 a07b618ecd1d 30c0223ec589 2024-03-29T08:15:44.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:44.758+0000 7f7ec9200700 1 -- 172.21.15.37:0/380704885 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f7ea806cc30 msgr2=0x7f7ea806f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:15:44.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:44.758+0000 7f7ec9200700 1 --2- 172.21.15.37:0/380704885 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f7ea806cc30 0x7f7ea806f0e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f7ec4111500 tx=0x7f7eb0009e30 comp rx=0 tx=0).stop 2024-03-29T08:15:44.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:44.758+0000 7f7ec9200700 1 -- 172.21.15.37:0/380704885 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ec4102e00 msgr2=0x7f7ec4115c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:15:44.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:44.758+0000 7f7ec9200700 1 --2- 172.21.15.37:0/380704885 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ec4102e00 0x7f7ec4115c60 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f7eb400eb40 tx=0x7f7eb400b040 comp rx=0 tx=0).stop 2024-03-29T08:15:44.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:44.759+0000 7f7ec9200700 1 -- 172.21.15.37:0/380704885 shutdown_connections 2024-03-29T08:15:44.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:44.759+0000 7f7ec9200700 1 --2- 172.21.15.37:0/380704885 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f7ea806cc30 0x7f7ea806f0e0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:15:44.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:44.759+0000 7f7ec9200700 1 --2- 172.21.15.37:0/380704885 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ec4102e00 0x7f7ec4115c60 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:15:44.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:44.759+0000 7f7ec9200700 1 --2- 172.21.15.37:0/380704885 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7ec41037a0 0x7f7ec41161a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:15:44.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:44.759+0000 7f7ec9200700 1 -- 172.21.15.37:0/380704885 >> 172.21.15.37:0/380704885 conn(0x7f7ec40f89b0 msgr2=0x7f7ec40f99c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:15:44.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:44.759+0000 7f7ec9200700 1 -- 172.21.15.37:0/380704885 shutdown_connections 2024-03-29T08:15:44.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:44.759+0000 7f7ec9200700 1 -- 172.21.15.37:0/380704885 wait complete. 2024-03-29T08:15:44.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:44.894+0000 7f92d4a00700 1 -- 172.21.15.37:0/2862662961 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f92d0102e00 msgr2=0x7f92d01031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:15:44.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:44.894+0000 7f92d4a00700 1 --2- 172.21.15.37:0/2862662961 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f92d0102e00 0x7f92d01031d0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f92c0004940 tx=0x7f92c0009e30 comp rx=0 tx=0).stop 2024-03-29T08:15:44.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:44.894+0000 7f92d4a00700 1 -- 172.21.15.37:0/2862662961 shutdown_connections 2024-03-29T08:15:44.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:44.894+0000 7f92d4a00700 1 --2- 172.21.15.37:0/2862662961 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f92d01037a0 0x7f92d00fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:15:44.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:44.894+0000 7f92d4a00700 1 --2- 172.21.15.37:0/2862662961 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f92d0102e00 0x7f92d01031d0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:15:44.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:44.894+0000 7f92d4a00700 1 -- 172.21.15.37:0/2862662961 >> 172.21.15.37:0/2862662961 conn(0x7f92d00f89b0 msgr2=0x7f92d00fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:15:44.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:44.894+0000 7f92d4a00700 1 -- 172.21.15.37:0/2862662961 shutdown_connections 2024-03-29T08:15:44.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:44.894+0000 7f92d4a00700 1 -- 172.21.15.37:0/2862662961 wait complete. 2024-03-29T08:15:44.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:44.895+0000 7f92d4a00700 1 Processor -- start 2024-03-29T08:15:44.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:44.895+0000 7f92d4a00700 1 -- start start 2024-03-29T08:15:44.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:44.896+0000 7f92d4a00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f92d0102e00 0x7f92d0113ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:15:44.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:44.896+0000 7f92d4a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f92d01037a0 0x7f92d0114000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:15:44.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:44.896+0000 7f92d4a00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92d0114650 con 0x7f92d01037a0 2024-03-29T08:15:44.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:44.896+0000 7f92d4a00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92d010da80 con 0x7f92d0102e00 2024-03-29T08:15:44.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:44.896+0000 7f92cde00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f92d0102e00 0x7f92d0113ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:15:44.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:44.896+0000 7f92cd400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f92d01037a0 0x7f92d0114000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:15:44.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:44.896+0000 7f92cde00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f92d0102e00 0x7f92d0113ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.37:58956/0 (socket says 172.21.15.37:58956) 2024-03-29T08:15:44.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:44.896+0000 7f92cde00700 1 -- 172.21.15.37:0/99724735 learned_addr learned my addr 172.21.15.37:0/99724735 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:15:44.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:44.896+0000 7f92cde00700 1 -- 172.21.15.37:0/99724735 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f92d01037a0 msgr2=0x7f92d0114000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:15:44.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:44.896+0000 7f92cde00700 1 --2- 172.21.15.37:0/99724735 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f92d01037a0 0x7f92d0114000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:15:44.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:44.896+0000 7f92cde00700 1 -- 172.21.15.37:0/99724735 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92c0005040 con 0x7f92d0102e00 2024-03-29T08:15:44.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:44.896+0000 7f92cd400700 1 --2- 172.21.15.37:0/99724735 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f92d01037a0 0x7f92d0114000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T08:15:44.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:44.897+0000 7f92cde00700 1 --2- 172.21.15.37:0/99724735 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f92d0102e00 0x7f92d0113ac0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f92c0009390 tx=0x7f92c0009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:15:44.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:44.897+0000 7f92c6000700 1 -- 172.21.15.37:0/99724735 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f92c000e040 con 0x7f92d0102e00 2024-03-29T08:15:44.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:44.897+0000 7f92c6000700 1 -- 172.21.15.37:0/99724735 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f92c0013420 con 0x7f92d0102e00 2024-03-29T08:15:44.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:44.897+0000 7f92d4a00700 1 -- 172.21.15.37:0/99724735 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92d010dd00 con 0x7f92d0102e00 2024-03-29T08:15:44.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:44.897+0000 7f92c6000700 1 -- 172.21.15.37:0/99724735 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f92c001c630 con 0x7f92d0102e00 2024-03-29T08:15:44.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:44.897+0000 7f92d4a00700 1 -- 172.21.15.37:0/99724735 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92d010e2c0 con 0x7f92d0102e00 2024-03-29T08:15:44.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:44.898+0000 7f92d4a00700 1 -- 172.21.15.37:0/99724735 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92d00fe550 con 0x7f92d0102e00 2024-03-29T08:15:44.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:44.900+0000 7f92c6000700 1 -- 172.21.15.37:0/99724735 <== mon.1 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f92c00139e0 con 0x7f92d0102e00 2024-03-29T08:15:44.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:44.901+0000 7f92c6000700 1 --2- 172.21.15.37:0/99724735 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f92b406cc30 0x7f92b406f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:15:44.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:44.901+0000 7f92c6000700 1 -- 172.21.15.37:0/99724735 <== mon.1 v2:172.21.15.148:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f92c008d4e0 con 0x7f92d0102e00 2024-03-29T08:15:44.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:44.901+0000 7f92cd400700 1 --2- 172.21.15.37:0/99724735 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f92b406cc30 0x7f92b406f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:15:44.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:44.901+0000 7f92cd400700 1 --2- 172.21.15.37:0/99724735 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f92b406cc30 0x7f92b406f0e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f92d0107170 tx=0x7f92bc00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:15:44.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:44.905+0000 7f92c6000700 1 -- 172.21.15.37:0/99724735 <== mon.1 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f92c005b030 con 0x7f92d0102e00 2024-03-29T08:15:45.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:45.205+0000 7f92d4a00700 1 -- 172.21.15.37:0/99724735 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f92d005f5f0 con 0x7f92d0102e00 2024-03-29T08:15:45.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:45.206+0000 7f92c6000700 1 -- 172.21.15.37:0/99724735 <== mon.1 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+847 (secure 0 0 0) 0x7f92c005abc0 con 0x7f92d0102e00 2024-03-29T08:15:45.206 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-29T08:15:45.206 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": { 2024-03-29T08:15:45.206 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 2 2024-03-29T08:15:45.206 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:15:45.206 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": { 2024-03-29T08:15:45.206 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 2 2024-03-29T08:15:45.206 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:15:45.206 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": { 2024-03-29T08:15:45.206 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 17.2.7-789-g4213e17c (4213e17c4de67016ed14f55d22503f4923b3a901) quincy (stable)": 2, 2024-03-29T08:15:45.206 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 3 2024-03-29T08:15:45.206 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:15:45.207 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": { 2024-03-29T08:15:45.207 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 17.2.7-789-g4213e17c (4213e17c4de67016ed14f55d22503f4923b3a901) quincy (stable)": 4 2024-03-29T08:15:45.207 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:15:45.207 INFO:teuthology.orchestra.run.smithi037.stdout: "overall": { 2024-03-29T08:15:45.207 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 17.2.7-789-g4213e17c (4213e17c4de67016ed14f55d22503f4923b3a901) quincy (stable)": 6, 2024-03-29T08:15:45.207 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 7 2024-03-29T08:15:45.207 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-29T08:15:45.207 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-29T08:15:45.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:45.208+0000 7f92d4a00700 1 -- 172.21.15.37:0/99724735 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f92b406cc30 msgr2=0x7f92b406f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:15:45.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:45.208+0000 7f92d4a00700 1 --2- 172.21.15.37:0/99724735 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f92b406cc30 0x7f92b406f0e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f92d0107170 tx=0x7f92bc00b040 comp rx=0 tx=0).stop 2024-03-29T08:15:45.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:45.208+0000 7f92d4a00700 1 -- 172.21.15.37:0/99724735 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f92d0102e00 msgr2=0x7f92d0113ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:15:45.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:45.208+0000 7f92d4a00700 1 --2- 172.21.15.37:0/99724735 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f92d0102e00 0x7f92d0113ac0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f92c0009390 tx=0x7f92c0009e30 comp rx=0 tx=0).stop 2024-03-29T08:15:45.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:45.208+0000 7f92d4a00700 1 -- 172.21.15.37:0/99724735 shutdown_connections 2024-03-29T08:15:45.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:45.208+0000 7f92d4a00700 1 --2- 172.21.15.37:0/99724735 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f92b406cc30 0x7f92b406f0e0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:15:45.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:45.208+0000 7f92d4a00700 1 --2- 172.21.15.37:0/99724735 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f92d0102e00 0x7f92d0113ac0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:15:45.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:45.208+0000 7f92d4a00700 1 --2- 172.21.15.37:0/99724735 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f92d01037a0 0x7f92d0114000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:15:45.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:45.208+0000 7f92d4a00700 1 -- 172.21.15.37:0/99724735 >> 172.21.15.37:0/99724735 conn(0x7f92d00f89b0 msgr2=0x7f92d00f9a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:15:45.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:45.208+0000 7f92d4a00700 1 -- 172.21.15.37:0/99724735 shutdown_connections 2024-03-29T08:15:45.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:45.208+0000 7f92d4a00700 1 -- 172.21.15.37:0/99724735 wait complete. 2024-03-29T08:15:45.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:15:45 smithi148 ceph-mon[106692]: from='client.34312 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:15:45.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:15:45 smithi148 ceph-mon[106692]: from='client.34316 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:15:45.307 INFO:journalctl@ceph.osd.3.smithi148.stdout:Mar 29 08:15:45 smithi148 systemd[1]: ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4@osd.3.service: Succeeded. 2024-03-29T08:15:45.307 INFO:journalctl@ceph.osd.3.smithi148.stdout:Mar 29 08:15:45 smithi148 systemd[1]: Stopped Ceph osd.3 for 99b492c2-eda1-11ee-b647-cb9ed24678a4. 2024-03-29T08:15:45.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:45.344+0000 7fa002e00700 1 -- 172.21.15.37:0/1216100159 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9ffc1037a0 msgr2=0x7f9ffc0fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:15:45.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:45.344+0000 7fa002e00700 1 --2- 172.21.15.37:0/1216100159 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9ffc1037a0 0x7f9ffc0fce10 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f9fe8004940 tx=0x7f9fe8009e30 comp rx=0 tx=0).stop 2024-03-29T08:15:45.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:45.344+0000 7fa002e00700 1 -- 172.21.15.37:0/1216100159 shutdown_connections 2024-03-29T08:15:45.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:45.344+0000 7fa002e00700 1 --2- 172.21.15.37:0/1216100159 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9ffc1037a0 0x7f9ffc0fce10 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:15:45.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:45.344+0000 7fa002e00700 1 --2- 172.21.15.37:0/1216100159 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9ffc102e00 0x7f9ffc1031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:15:45.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:45.344+0000 7fa002e00700 1 -- 172.21.15.37:0/1216100159 >> 172.21.15.37:0/1216100159 conn(0x7f9ffc0f89b0 msgr2=0x7f9ffc0fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:15:45.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:45.344+0000 7fa002e00700 1 -- 172.21.15.37:0/1216100159 shutdown_connections 2024-03-29T08:15:45.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:45.344+0000 7fa002e00700 1 -- 172.21.15.37:0/1216100159 wait complete. 2024-03-29T08:15:45.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:45.346+0000 7fa002e00700 1 Processor -- start 2024-03-29T08:15:45.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:45.346+0000 7fa002e00700 1 -- start start 2024-03-29T08:15:45.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:45.346+0000 7fa002e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9ffc102e00 0x7f9ffc0702c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:15:45.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:45.346+0000 7fa002e00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9ffc1037a0 0x7f9ffc070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:15:45.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:45.346+0000 7fa002e00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ffc070d90 con 0x7f9ffc102e00 2024-03-29T08:15:45.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:45.346+0000 7fa002e00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ffc070ed0 con 0x7f9ffc1037a0 2024-03-29T08:15:45.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:45.347+0000 7f9ffbe00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9ffc102e00 0x7f9ffc0702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:15:45.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:45.347+0000 7f9ffbe00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9ffc102e00 0x7f9ffc0702c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34140/0 (socket says 172.21.15.37:34140) 2024-03-29T08:15:45.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:45.347+0000 7f9ffbe00700 1 -- 172.21.15.37:0/2471684099 learned_addr learned my addr 172.21.15.37:0/2471684099 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:15:45.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:45.347+0000 7f9ffb400700 1 --2- 172.21.15.37:0/2471684099 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9ffc1037a0 0x7f9ffc070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:15:45.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:45.347+0000 7f9ffbe00700 1 -- 172.21.15.37:0/2471684099 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9ffc1037a0 msgr2=0x7f9ffc070800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:15:45.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:45.347+0000 7f9ffbe00700 1 --2- 172.21.15.37:0/2471684099 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9ffc1037a0 0x7f9ffc070800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:15:45.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:45.347+0000 7f9ffbe00700 1 -- 172.21.15.37:0/2471684099 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fe8005040 con 0x7f9ffc102e00 2024-03-29T08:15:45.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:45.347+0000 7f9ffbe00700 1 --2- 172.21.15.37:0/2471684099 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9ffc102e00 0x7f9ffc0702c0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f9fec00eb40 tx=0x7f9fec00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:15:45.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:45.348+0000 7f9ff3e00700 1 -- 172.21.15.37:0/2471684099 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9fec007cc0 con 0x7f9ffc102e00 2024-03-29T08:15:45.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:45.348+0000 7f9ff3e00700 1 -- 172.21.15.37:0/2471684099 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f9fec00f040 con 0x7f9ffc102e00 2024-03-29T08:15:45.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:45.348+0000 7fa002e00700 1 -- 172.21.15.37:0/2471684099 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ffc071120 con 0x7f9ffc102e00 2024-03-29T08:15:45.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:45.348+0000 7f9ff3e00700 1 -- 172.21.15.37:0/2471684099 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9fec00c990 con 0x7f9ffc102e00 2024-03-29T08:15:45.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:45.348+0000 7fa002e00700 1 -- 172.21.15.37:0/2471684099 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ffc11a1a0 con 0x7f9ffc102e00 2024-03-29T08:15:45.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:45.349+0000 7fa002e00700 1 -- 172.21.15.37:0/2471684099 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ffc0fe550 con 0x7f9ffc102e00 2024-03-29T08:15:45.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:45.350+0000 7f9ff3e00700 1 -- 172.21.15.37:0/2471684099 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f9fec00cc00 con 0x7f9ffc102e00 2024-03-29T08:15:45.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:45.351+0000 7f9ff3e00700 1 --2- 172.21.15.37:0/2471684099 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f9fe006cc30 0x7f9fe006f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:15:45.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:45.351+0000 7f9ff3e00700 1 -- 172.21.15.37:0/2471684099 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f9fec08bb90 con 0x7f9ffc102e00 2024-03-29T08:15:45.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:45.351+0000 7f9ffb400700 1 --2- 172.21.15.37:0/2471684099 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f9fe006cc30 0x7f9fe006f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:15:45.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:45.352+0000 7f9ffb400700 1 --2- 172.21.15.37:0/2471684099 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f9fe006cc30 0x7f9fe006f0e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f9ffc071c00 tx=0x7f9fe8009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:15:45.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:45.355+0000 7f9ff3e00700 1 -- 172.21.15.37:0/2471684099 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f9fec055ca0 con 0x7f9ffc102e00 2024-03-29T08:15:45.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:15:45 smithi037 ceph-mon[142325]: from='client.34312 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:15:45.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:15:45 smithi037 ceph-mon[142325]: from='client.34316 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:15:45.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:45.615+0000 7fa002e00700 1 -- 172.21.15.37:0/2471684099 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f9ffc04bf60 con 0x7f9ffc102e00 2024-03-29T08:15:45.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:45.615+0000 7f9ff3e00700 1 -- 172.21.15.37:0/2471684099 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 20 v20) v1 ==== 76+0+1858 (secure 0 0 0) 0x7f9fec0592c0 con 0x7f9ffc102e00 2024-03-29T08:15:45.615 INFO:teuthology.orchestra.run.smithi037.stdout:e20 2024-03-29T08:15:45.616 INFO:teuthology.orchestra.run.smithi037.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-29T08:15:45.616 INFO:teuthology.orchestra.run.smithi037.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T08:15:45.616 INFO:teuthology.orchestra.run.smithi037.stdout:legacy client fscid: 1 2024-03-29T08:15:45.616 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:15:45.616 INFO:teuthology.orchestra.run.smithi037.stdout:Filesystem 'cephfs' (1) 2024-03-29T08:15:45.616 INFO:teuthology.orchestra.run.smithi037.stdout:fs_name cephfs 2024-03-29T08:15:45.616 INFO:teuthology.orchestra.run.smithi037.stdout:epoch 20 2024-03-29T08:15:45.616 INFO:teuthology.orchestra.run.smithi037.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-03-29T08:15:45.616 INFO:teuthology.orchestra.run.smithi037.stdout:created 2024-03-29T08:03:29.241269+0000 2024-03-29T08:15:45.616 INFO:teuthology.orchestra.run.smithi037.stdout:modified 2024-03-29T08:11:27.181622+0000 2024-03-29T08:15:45.616 INFO:teuthology.orchestra.run.smithi037.stdout:tableserver 0 2024-03-29T08:15:45.616 INFO:teuthology.orchestra.run.smithi037.stdout:root 0 2024-03-29T08:15:45.616 INFO:teuthology.orchestra.run.smithi037.stdout:session_timeout 60 2024-03-29T08:15:45.616 INFO:teuthology.orchestra.run.smithi037.stdout:session_autoclose 300 2024-03-29T08:15:45.616 INFO:teuthology.orchestra.run.smithi037.stdout:max_file_size 1099511627776 2024-03-29T08:15:45.616 INFO:teuthology.orchestra.run.smithi037.stdout:max_xattr_size 65536 2024-03-29T08:15:45.617 INFO:teuthology.orchestra.run.smithi037.stdout:required_client_features {} 2024-03-29T08:15:45.617 INFO:teuthology.orchestra.run.smithi037.stdout:last_failure 0 2024-03-29T08:15:45.617 INFO:teuthology.orchestra.run.smithi037.stdout:last_failure_osd_epoch 47 2024-03-29T08:15:45.617 INFO:teuthology.orchestra.run.smithi037.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T08:15:45.617 INFO:teuthology.orchestra.run.smithi037.stdout:max_mds 2 2024-03-29T08:15:45.617 INFO:teuthology.orchestra.run.smithi037.stdout:in 0,1 2024-03-29T08:15:45.617 INFO:teuthology.orchestra.run.smithi037.stdout:up {0=14470,1=14482} 2024-03-29T08:15:45.617 INFO:teuthology.orchestra.run.smithi037.stdout:failed 2024-03-29T08:15:45.617 INFO:teuthology.orchestra.run.smithi037.stdout:damaged 2024-03-29T08:15:45.617 INFO:teuthology.orchestra.run.smithi037.stdout:stopped 2024-03-29T08:15:45.617 INFO:teuthology.orchestra.run.smithi037.stdout:data_pools [3] 2024-03-29T08:15:45.617 INFO:teuthology.orchestra.run.smithi037.stdout:metadata_pool 2 2024-03-29T08:15:45.617 INFO:teuthology.orchestra.run.smithi037.stdout:inline_data disabled 2024-03-29T08:15:45.617 INFO:teuthology.orchestra.run.smithi037.stdout:balancer 2024-03-29T08:15:45.617 INFO:teuthology.orchestra.run.smithi037.stdout:bal_rank_mask -1 2024-03-29T08:15:45.617 INFO:teuthology.orchestra.run.smithi037.stdout:standby_count_wanted 1 2024-03-29T08:15:45.618 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi037.mmwgxe{0:14470} state up:active seq 5 join_fscid=1 addr [v2:172.21.15.37:6826/1559618909,v1:172.21.15.37:6827/1559618909] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:15:45.618 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi037.iifsmh{1:14482} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.37:6828/2341530495,v1:172.21.15.37:6829/2341530495] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:15:45.618 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:15:45.618 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:15:45.618 INFO:teuthology.orchestra.run.smithi037.stdout:Standby daemons: 2024-03-29T08:15:45.618 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:15:45.618 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi148.qukpjh{-1:24319} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.148:6826/2641327912,v1:172.21.15.148:6827/2641327912] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:15:45.618 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi148.sqteih{-1:24327} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.148:6824/3252351726,v1:172.21.15.148:6825/3252351726] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:15:45.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:45.618+0000 7fa002e00700 1 -- 172.21.15.37:0/2471684099 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f9fe006cc30 msgr2=0x7f9fe006f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:15:45.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:45.618+0000 7fa002e00700 1 --2- 172.21.15.37:0/2471684099 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f9fe006cc30 0x7f9fe006f0e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f9ffc071c00 tx=0x7f9fe8009e30 comp rx=0 tx=0).stop 2024-03-29T08:15:45.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:45.618+0000 7fa002e00700 1 -- 172.21.15.37:0/2471684099 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9ffc102e00 msgr2=0x7f9ffc0702c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:15:45.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:45.618+0000 7fa002e00700 1 --2- 172.21.15.37:0/2471684099 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9ffc102e00 0x7f9ffc0702c0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f9fec00eb40 tx=0x7f9fec00b040 comp rx=0 tx=0).stop 2024-03-29T08:15:45.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:45.618+0000 7fa002e00700 1 -- 172.21.15.37:0/2471684099 shutdown_connections 2024-03-29T08:15:45.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:45.618+0000 7fa002e00700 1 --2- 172.21.15.37:0/2471684099 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f9fe006cc30 0x7f9fe006f0e0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:15:45.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:45.618+0000 7fa002e00700 1 --2- 172.21.15.37:0/2471684099 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9ffc102e00 0x7f9ffc0702c0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:15:45.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:45.618+0000 7fa002e00700 1 --2- 172.21.15.37:0/2471684099 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9ffc1037a0 0x7f9ffc070800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:15:45.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:45.618+0000 7fa002e00700 1 -- 172.21.15.37:0/2471684099 >> 172.21.15.37:0/2471684099 conn(0x7f9ffc0f89b0 msgr2=0x7f9ffc0f9a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:15:45.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:45.618+0000 7fa002e00700 1 -- 172.21.15.37:0/2471684099 shutdown_connections 2024-03-29T08:15:45.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:45.618+0000 7fa002e00700 1 -- 172.21.15.37:0/2471684099 wait complete. 2024-03-29T08:15:45.620 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 20 2024-03-29T08:15:45.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:45.755+0000 7fa4bf600700 1 -- 172.21.15.37:0/117769476 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa4b81037a0 msgr2=0x7fa4b80fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:15:45.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:45.755+0000 7fa4bf600700 1 --2- 172.21.15.37:0/117769476 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa4b81037a0 0x7fa4b80fce10 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fa4a4004940 tx=0x7fa4a4009e30 comp rx=0 tx=0).stop 2024-03-29T08:15:45.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:45.755+0000 7fa4bf600700 1 -- 172.21.15.37:0/117769476 shutdown_connections 2024-03-29T08:15:45.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:45.755+0000 7fa4bf600700 1 --2- 172.21.15.37:0/117769476 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa4b81037a0 0x7fa4b80fce10 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:15:45.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:45.755+0000 7fa4bf600700 1 --2- 172.21.15.37:0/117769476 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa4b8102e00 0x7fa4b81031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:15:45.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:45.755+0000 7fa4bf600700 1 -- 172.21.15.37:0/117769476 >> 172.21.15.37:0/117769476 conn(0x7fa4b80f89b0 msgr2=0x7fa4b80fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:15:45.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:45.755+0000 7fa4bf600700 1 -- 172.21.15.37:0/117769476 shutdown_connections 2024-03-29T08:15:45.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:45.755+0000 7fa4bf600700 1 -- 172.21.15.37:0/117769476 wait complete. 2024-03-29T08:15:45.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:45.756+0000 7fa4bf600700 1 Processor -- start 2024-03-29T08:15:45.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:45.756+0000 7fa4bf600700 1 -- start start 2024-03-29T08:15:45.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:45.756+0000 7fa4bf600700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa4b8102e00 0x7fa4b8115c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:15:45.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:45.756+0000 7fa4bf600700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa4b81037a0 0x7fa4b81161c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:15:45.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:45.756+0000 7fa4bf600700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4b8116810 con 0x7fa4b81037a0 2024-03-29T08:15:45.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:45.756+0000 7fa4bf600700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4b810fbe0 con 0x7fa4b8102e00 2024-03-29T08:15:45.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:45.757+0000 7fa4b7e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa4b81037a0 0x7fa4b81161c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:15:45.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:45.757+0000 7fa4b7e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa4b81037a0 0x7fa4b81161c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34152/0 (socket says 172.21.15.37:34152) 2024-03-29T08:15:45.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:45.757+0000 7fa4b7e00700 1 -- 172.21.15.37:0/2318624404 learned_addr learned my addr 172.21.15.37:0/2318624404 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:15:45.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:45.757+0000 7fa4bcc00700 1 --2- 172.21.15.37:0/2318624404 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa4b8102e00 0x7fa4b8115c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:15:45.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:45.757+0000 7fa4b7e00700 1 -- 172.21.15.37:0/2318624404 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa4b8102e00 msgr2=0x7fa4b8115c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:15:45.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:45.757+0000 7fa4b7e00700 1 --2- 172.21.15.37:0/2318624404 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa4b8102e00 0x7fa4b8115c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:15:45.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:45.757+0000 7fa4b7e00700 1 -- 172.21.15.37:0/2318624404 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4a4005040 con 0x7fa4b81037a0 2024-03-29T08:15:45.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:45.757+0000 7fa4b7e00700 1 --2- 172.21.15.37:0/2318624404 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa4b81037a0 0x7fa4b81161c0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fa4a4009fd0 tx=0x7fa4a4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:15:45.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:45.758+0000 7fa4b4c00700 1 -- 172.21.15.37:0/2318624404 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa4a400e030 con 0x7fa4b81037a0 2024-03-29T08:15:45.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:45.758+0000 7fa4b4c00700 1 -- 172.21.15.37:0/2318624404 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fa4a40093b0 con 0x7fa4b81037a0 2024-03-29T08:15:45.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:45.758+0000 7fa4b4c00700 1 -- 172.21.15.37:0/2318624404 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa4a401cc80 con 0x7fa4b81037a0 2024-03-29T08:15:45.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:45.758+0000 7fa4bf600700 1 -- 172.21.15.37:0/2318624404 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4b810fe60 con 0x7fa4b81037a0 2024-03-29T08:15:45.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:45.758+0000 7fa4bf600700 1 -- 172.21.15.37:0/2318624404 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4b8110420 con 0x7fa4b81037a0 2024-03-29T08:15:45.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:45.759+0000 7fa4bf600700 1 -- 172.21.15.37:0/2318624404 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4b80fe550 con 0x7fa4b81037a0 2024-03-29T08:15:45.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:45.760+0000 7fa4b4c00700 1 -- 172.21.15.37:0/2318624404 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fa4a401c400 con 0x7fa4b81037a0 2024-03-29T08:15:45.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:45.761+0000 7fa4b4c00700 1 --2- 172.21.15.37:0/2318624404 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fa4a006cc30 0x7fa4a006f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:15:45.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:45.761+0000 7fa4b4c00700 1 -- 172.21.15.37:0/2318624404 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fa4a408d830 con 0x7fa4b81037a0 2024-03-29T08:15:45.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:45.761+0000 7fa4bcc00700 1 --2- 172.21.15.37:0/2318624404 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fa4a006cc30 0x7fa4a006f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:15:45.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:45.762+0000 7fa4bcc00700 1 --2- 172.21.15.37:0/2318624404 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fa4a006cc30 0x7fa4a006f0e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fa4ac009b80 tx=0x7fa4ac00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:15:45.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:45.765+0000 7fa4b4c00700 1 -- 172.21.15.37:0/2318624404 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fa4a405b300 con 0x7fa4b81037a0 2024-03-29T08:15:45.806 INFO:journalctl@ceph.osd.3.smithi148.stdout:Mar 29 08:15:45 smithi148 systemd[1]: Starting Ceph osd.3 for 99b492c2-eda1-11ee-b647-cb9ed24678a4... 2024-03-29T08:15:46.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:46.005+0000 7fa4bf600700 1 -- 172.21.15.37:0/2318624404 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa4b8109e10 con 0x7fa4a006cc30 2024-03-29T08:15:46.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:46.006+0000 7fa4b4c00700 1 -- 172.21.15.37:0/2318624404 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7fa4b8109e10 con 0x7fa4a006cc30 2024-03-29T08:15:46.007 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-29T08:15:46.007 INFO:teuthology.orchestra.run.smithi037.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59", 2024-03-29T08:15:46.007 INFO:teuthology.orchestra.run.smithi037.stdout: "in_progress": true, 2024-03-29T08:15:46.007 INFO:teuthology.orchestra.run.smithi037.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-29T08:15:46.007 INFO:teuthology.orchestra.run.smithi037.stdout: "services_complete": [ 2024-03-29T08:15:46.007 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr", 2024-03-29T08:15:46.007 INFO:teuthology.orchestra.run.smithi037.stdout: "mon", 2024-03-29T08:15:46.007 INFO:teuthology.orchestra.run.smithi037.stdout: "crash" 2024-03-29T08:15:46.007 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-29T08:15:46.007 INFO:teuthology.orchestra.run.smithi037.stdout: "progress": "9/21 daemons upgraded", 2024-03-29T08:15:46.007 INFO:teuthology.orchestra.run.smithi037.stdout: "message": "Currently upgrading osd daemons", 2024-03-29T08:15:46.007 INFO:teuthology.orchestra.run.smithi037.stdout: "is_paused": false 2024-03-29T08:15:46.007 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-29T08:15:46.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:46.009+0000 7fa4bf600700 1 -- 172.21.15.37:0/2318624404 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fa4a006cc30 msgr2=0x7fa4a006f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:15:46.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:46.009+0000 7fa4bf600700 1 --2- 172.21.15.37:0/2318624404 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fa4a006cc30 0x7fa4a006f0e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fa4ac009b80 tx=0x7fa4ac00b040 comp rx=0 tx=0).stop 2024-03-29T08:15:46.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:46.009+0000 7fa4bf600700 1 -- 172.21.15.37:0/2318624404 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa4b81037a0 msgr2=0x7fa4b81161c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:15:46.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:46.009+0000 7fa4bf600700 1 --2- 172.21.15.37:0/2318624404 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa4b81037a0 0x7fa4b81161c0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fa4a4009fd0 tx=0x7fa4a4009e30 comp rx=0 tx=0).stop 2024-03-29T08:15:46.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:46.009+0000 7fa4bf600700 1 -- 172.21.15.37:0/2318624404 shutdown_connections 2024-03-29T08:15:46.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:46.009+0000 7fa4bf600700 1 --2- 172.21.15.37:0/2318624404 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fa4a006cc30 0x7fa4a006f0e0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:15:46.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:46.009+0000 7fa4bf600700 1 --2- 172.21.15.37:0/2318624404 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa4b8102e00 0x7fa4b8115c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:15:46.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:46.009+0000 7fa4bf600700 1 --2- 172.21.15.37:0/2318624404 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa4b81037a0 0x7fa4b81161c0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:15:46.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:46.009+0000 7fa4bf600700 1 -- 172.21.15.37:0/2318624404 >> 172.21.15.37:0/2318624404 conn(0x7fa4b80f89b0 msgr2=0x7fa4b80f9a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:15:46.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:46.009+0000 7fa4bf600700 1 -- 172.21.15.37:0/2318624404 shutdown_connections 2024-03-29T08:15:46.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:46.009+0000 7fa4bf600700 1 -- 172.21.15.37:0/2318624404 wait complete. 2024-03-29T08:15:46.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:46.147+0000 7f34ee000700 1 -- 172.21.15.37:0/839855783 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f34e80fe830 msgr2=0x7f34e8106f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:15:46.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:46.147+0000 7f34ee000700 1 --2- 172.21.15.37:0/839855783 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f34e80fe830 0x7f34e8106f00 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f34d4004940 tx=0x7f34d4009e30 comp rx=0 tx=0).stop 2024-03-29T08:15:46.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:46.147+0000 7f34ee000700 1 -- 172.21.15.37:0/839855783 shutdown_connections 2024-03-29T08:15:46.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:46.147+0000 7f34ee000700 1 --2- 172.21.15.37:0/839855783 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f34e80fe830 0x7f34e8106f00 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:15:46.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:46.147+0000 7f34ee000700 1 --2- 172.21.15.37:0/839855783 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f34e80fde90 0x7f34e80fe260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:15:46.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:46.147+0000 7f34ee000700 1 -- 172.21.15.37:0/839855783 >> 172.21.15.37:0/839855783 conn(0x7f34e8072bc0 msgr2=0x7f34e8072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:15:46.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:46.147+0000 7f34ee000700 1 -- 172.21.15.37:0/839855783 shutdown_connections 2024-03-29T08:15:46.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:46.147+0000 7f34ee000700 1 -- 172.21.15.37:0/839855783 wait complete. 2024-03-29T08:15:46.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:46.148+0000 7f34ee000700 1 Processor -- start 2024-03-29T08:15:46.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:46.148+0000 7f34ee000700 1 -- start start 2024-03-29T08:15:46.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:46.149+0000 7f34ee000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f34e80fde90 0x7f34e8115c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:15:46.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:46.149+0000 7f34ee000700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f34e80fe830 0x7f34e81161d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:15:46.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:46.149+0000 7f34ee000700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34e8116820 con 0x7f34e80fde90 2024-03-29T08:15:46.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:46.149+0000 7f34ee000700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34e810fbf0 con 0x7f34e80fe830 2024-03-29T08:15:46.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:46.149+0000 7f34e7400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f34e80fde90 0x7f34e8115c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:15:46.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:46.149+0000 7f34e7400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f34e80fde90 0x7f34e8115c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34160/0 (socket says 172.21.15.37:34160) 2024-03-29T08:15:46.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:46.149+0000 7f34e7400700 1 -- 172.21.15.37:0/2002748408 learned_addr learned my addr 172.21.15.37:0/2002748408 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:15:46.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:46.149+0000 7f34e6a00700 1 --2- 172.21.15.37:0/2002748408 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f34e80fe830 0x7f34e81161d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:15:46.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:46.149+0000 7f34e7400700 1 -- 172.21.15.37:0/2002748408 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f34e80fe830 msgr2=0x7f34e81161d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:15:46.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:46.149+0000 7f34e7400700 1 --2- 172.21.15.37:0/2002748408 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f34e80fe830 0x7f34e81161d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:15:46.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:46.149+0000 7f34e7400700 1 -- 172.21.15.37:0/2002748408 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34d4005040 con 0x7f34e80fde90 2024-03-29T08:15:46.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:46.150+0000 7f34e7400700 1 --2- 172.21.15.37:0/2002748408 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f34e80fde90 0x7f34e8115c90 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f34d800eb70 tx=0x7f34d800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:15:46.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:46.150+0000 7f34df400700 1 -- 172.21.15.37:0/2002748408 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f34d8007cc0 con 0x7f34e80fde90 2024-03-29T08:15:46.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:46.150+0000 7f34df400700 1 -- 172.21.15.37:0/2002748408 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f34d800f040 con 0x7f34e80fde90 2024-03-29T08:15:46.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:46.150+0000 7f34df400700 1 -- 172.21.15.37:0/2002748408 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f34d800c9f0 con 0x7f34e80fde90 2024-03-29T08:15:46.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:46.150+0000 7f34ee000700 1 -- 172.21.15.37:0/2002748408 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34e810fe70 con 0x7f34e80fde90 2024-03-29T08:15:46.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:46.150+0000 7f34ee000700 1 -- 172.21.15.37:0/2002748408 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34e8110360 con 0x7f34e80fde90 2024-03-29T08:15:46.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:46.151+0000 7f34ee000700 1 -- 172.21.15.37:0/2002748408 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34e8106a20 con 0x7f34e80fde90 2024-03-29T08:15:46.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:46.152+0000 7f34df400700 1 -- 172.21.15.37:0/2002748408 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f34d800cb50 con 0x7f34e80fde90 2024-03-29T08:15:46.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:46.153+0000 7f34df400700 1 --2- 172.21.15.37:0/2002748408 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f34cc06cc30 0x7f34cc06f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:15:46.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:46.153+0000 7f34df400700 1 -- 172.21.15.37:0/2002748408 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f34d808bc20 con 0x7f34e80fde90 2024-03-29T08:15:46.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:46.153+0000 7f34e6a00700 1 --2- 172.21.15.37:0/2002748408 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f34cc06cc30 0x7f34cc06f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:15:46.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:46.154+0000 7f34e6a00700 1 --2- 172.21.15.37:0/2002748408 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f34cc06cc30 0x7f34cc06f0e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f34e8111530 tx=0x7f34d4009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:15:46.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:46.158+0000 7f34df400700 1 -- 172.21.15.37:0/2002748408 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f34d8059670 con 0x7f34e80fde90 2024-03-29T08:15:46.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:15:46 smithi148 ceph-mon[106692]: from='client.34320 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:15:46.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:15:46 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/99724735' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:15:46.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:15:46 smithi148 ceph-mon[106692]: pgmap v128: 65 pgs: 19 active+undersized+degraded, 19 active+undersized, 27 active+clean; 259 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 66/324 objects degraded (20.370%) 2024-03-29T08:15:46.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:15:46 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/2471684099' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T08:15:46.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:15:46 smithi037 ceph-mon[142325]: from='client.34320 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:15:46.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:15:46 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/99724735' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:15:46.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:15:46 smithi037 ceph-mon[142325]: pgmap v128: 65 pgs: 19 active+undersized+degraded, 19 active+undersized, 27 active+clean; 259 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 66/324 objects degraded (20.370%) 2024-03-29T08:15:46.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:15:46 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/2471684099' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T08:15:46.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:46.454+0000 7f34ee000700 1 -- 172.21.15.37:0/2002748408 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f34e8065e30 con 0x7f34e80fde90 2024-03-29T08:15:46.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:46.454+0000 7f34df400700 1 -- 172.21.15.37:0/2002748408 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+1328 (secure 0 0 0) 0x7f34d8059200 con 0x7f34e80fde90 2024-03-29T08:15:46.454 INFO:teuthology.orchestra.run.smithi037.stdout:HEALTH_WARN 1 osds down; Degraded data redundancy: 66/324 objects degraded (20.370%), 19 pgs degraded 2024-03-29T08:15:46.454 INFO:teuthology.orchestra.run.smithi037.stdout:[WRN] OSD_DOWN: 1 osds down 2024-03-29T08:15:46.454 INFO:teuthology.orchestra.run.smithi037.stdout: osd.3 (root=default,host=smithi148) is down 2024-03-29T08:15:46.455 INFO:teuthology.orchestra.run.smithi037.stdout:[WRN] PG_DEGRADED: Degraded data redundancy: 66/324 objects degraded (20.370%), 19 pgs degraded 2024-03-29T08:15:46.455 INFO:teuthology.orchestra.run.smithi037.stdout: pg 1.0 is active+undersized+degraded, acting [0,1] 2024-03-29T08:15:46.455 INFO:teuthology.orchestra.run.smithi037.stdout: pg 2.0 is active+undersized+degraded, acting [1,0] 2024-03-29T08:15:46.455 INFO:teuthology.orchestra.run.smithi037.stdout: pg 2.5 is active+undersized+degraded, acting [0,4] 2024-03-29T08:15:46.455 INFO:teuthology.orchestra.run.smithi037.stdout: pg 2.6 is active+undersized+degraded, acting [1,4] 2024-03-29T08:15:46.455 INFO:teuthology.orchestra.run.smithi037.stdout: pg 2.7 is active+undersized+degraded, acting [4,2] 2024-03-29T08:15:46.455 INFO:teuthology.orchestra.run.smithi037.stdout: pg 2.8 is active+undersized+degraded, acting [5,0] 2024-03-29T08:15:46.455 INFO:teuthology.orchestra.run.smithi037.stdout: pg 2.a is active+undersized+degraded, acting [1,4] 2024-03-29T08:15:46.455 INFO:teuthology.orchestra.run.smithi037.stdout: pg 2.b is active+undersized+degraded, acting [4,5] 2024-03-29T08:15:46.455 INFO:teuthology.orchestra.run.smithi037.stdout: pg 2.d is active+undersized+degraded, acting [1,2] 2024-03-29T08:15:46.455 INFO:teuthology.orchestra.run.smithi037.stdout: pg 2.e is active+undersized+degraded, acting [2,0] 2024-03-29T08:15:46.455 INFO:teuthology.orchestra.run.smithi037.stdout: pg 2.11 is active+undersized+degraded, acting [4,1] 2024-03-29T08:15:46.456 INFO:teuthology.orchestra.run.smithi037.stdout: pg 2.12 is active+undersized+degraded, acting [1,0] 2024-03-29T08:15:46.456 INFO:teuthology.orchestra.run.smithi037.stdout: pg 2.14 is active+undersized+degraded, acting [4,5] 2024-03-29T08:15:46.456 INFO:teuthology.orchestra.run.smithi037.stdout: pg 2.15 is active+undersized+degraded, acting [1,0] 2024-03-29T08:15:46.456 INFO:teuthology.orchestra.run.smithi037.stdout: pg 2.16 is active+undersized+degraded, acting [5,2] 2024-03-29T08:15:46.456 INFO:teuthology.orchestra.run.smithi037.stdout: pg 2.18 is active+undersized+degraded, acting [5,4] 2024-03-29T08:15:46.456 INFO:teuthology.orchestra.run.smithi037.stdout: pg 2.1a is active+undersized+degraded, acting [4,5] 2024-03-29T08:15:46.456 INFO:teuthology.orchestra.run.smithi037.stdout: pg 2.1d is active+undersized+degraded, acting [5,0] 2024-03-29T08:15:46.456 INFO:teuthology.orchestra.run.smithi037.stdout: pg 2.1f is active+undersized+degraded, acting [0,4] 2024-03-29T08:15:46.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:46.456+0000 7f34ee000700 1 -- 172.21.15.37:0/2002748408 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f34cc06cc30 msgr2=0x7f34cc06f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:15:46.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:46.456+0000 7f34ee000700 1 --2- 172.21.15.37:0/2002748408 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f34cc06cc30 0x7f34cc06f0e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f34e8111530 tx=0x7f34d4009e30 comp rx=0 tx=0).stop 2024-03-29T08:15:46.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:46.456+0000 7f34ee000700 1 -- 172.21.15.37:0/2002748408 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f34e80fde90 msgr2=0x7f34e8115c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:15:46.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:46.456+0000 7f34ee000700 1 --2- 172.21.15.37:0/2002748408 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f34e80fde90 0x7f34e8115c90 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f34d800eb70 tx=0x7f34d800b040 comp rx=0 tx=0).stop 2024-03-29T08:15:46.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:46.457+0000 7f34ee000700 1 -- 172.21.15.37:0/2002748408 shutdown_connections 2024-03-29T08:15:46.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:46.457+0000 7f34ee000700 1 --2- 172.21.15.37:0/2002748408 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f34cc06cc30 0x7f34cc06f0e0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:15:46.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:46.457+0000 7f34ee000700 1 --2- 172.21.15.37:0/2002748408 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f34e80fde90 0x7f34e8115c90 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:15:46.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:46.457+0000 7f34ee000700 1 --2- 172.21.15.37:0/2002748408 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f34e80fe830 0x7f34e81161d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:15:46.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:46.457+0000 7f34ee000700 1 -- 172.21.15.37:0/2002748408 >> 172.21.15.37:0/2002748408 conn(0x7f34e8072bc0 msgr2=0x7f34e80fb6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:15:46.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:46.457+0000 7f34ee000700 1 -- 172.21.15.37:0/2002748408 shutdown_connections 2024-03-29T08:15:46.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:15:46.457+0000 7f34ee000700 1 -- 172.21.15.37:0/2002748408 wait complete. 2024-03-29T08:15:47.056 INFO:journalctl@ceph.osd.3.smithi148.stdout:Mar 29 08:15:46 smithi148 bash[118422]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-03-29T08:15:47.056 INFO:journalctl@ceph.osd.3.smithi148.stdout:Mar 29 08:15:46 smithi148 bash[118422]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-3 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-03-29T08:15:47.056 INFO:journalctl@ceph.osd.3.smithi148.stdout:Mar 29 08:15:46 smithi148 bash[118422]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-03-29T08:15:47.057 INFO:journalctl@ceph.osd.3.smithi148.stdout:Mar 29 08:15:46 smithi148 bash[118422]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-03-29T08:15:47.057 INFO:journalctl@ceph.osd.3.smithi148.stdout:Mar 29 08:15:46 smithi148 bash[118422]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3/block 2024-03-29T08:15:47.057 INFO:journalctl@ceph.osd.3.smithi148.stdout:Mar 29 08:15:46 smithi148 bash[118422]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-03-29T08:15:47.057 INFO:journalctl@ceph.osd.3.smithi148.stdout:Mar 29 08:15:46 smithi148 bash[118422]: --> ceph-volume raw activate successful for osd ID: 3 2024-03-29T08:15:47.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:15:47 smithi037 ceph-mon[142325]: from='client.34332 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:15:47.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:15:47 smithi037 ceph-mon[142325]: Health check failed: Degraded data redundancy: 66/324 objects degraded (20.370%), 19 pgs degraded (PG_DEGRADED) 2024-03-29T08:15:47.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:15:47 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/2002748408' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T08:15:47.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:15:47 smithi148 ceph-mon[106692]: from='client.34332 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:15:47.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:15:47 smithi148 ceph-mon[106692]: Health check failed: Degraded data redundancy: 66/324 objects degraded (20.370%), 19 pgs degraded (PG_DEGRADED) 2024-03-29T08:15:47.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:15:47 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/2002748408' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T08:15:48.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:15:48 smithi037 ceph-mon[142325]: pgmap v129: 65 pgs: 19 active+undersized+degraded, 19 active+undersized, 27 active+clean; 259 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 66/324 objects degraded (20.370%) 2024-03-29T08:15:48.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:15:48 smithi148 ceph-mon[106692]: pgmap v129: 65 pgs: 19 active+undersized+degraded, 19 active+undersized, 27 active+clean; 259 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 66/324 objects degraded (20.370%) 2024-03-29T08:15:48.556 INFO:journalctl@ceph.osd.3.smithi148.stdout:Mar 29 08:15:48 smithi148 ceph-osd[118731]: -- 172.21.15.148:0/3178423159 <== mon.1 v2:172.21.15.148:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55cd14e811e0 con 0x55cd14e51000 2024-03-29T08:15:48.556 INFO:journalctl@ceph.osd.3.smithi148.stdout:Mar 29 08:15:48 smithi148 bash[118693]: 1c656ab8ff866550011fabbed9450690b7d31fc71a7dbeedd38288ffc7dd45bb 2024-03-29T08:15:48.556 INFO:journalctl@ceph.osd.3.smithi148.stdout:Mar 29 08:15:48 smithi148 systemd[1]: Started Ceph osd.3 for 99b492c2-eda1-11ee-b647-cb9ed24678a4. 2024-03-29T08:15:49.306 INFO:journalctl@ceph.osd.3.smithi148.stdout:Mar 29 08:15:48 smithi148 ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4-osd-3[118708]: 2024-03-29T08:15:48.807+0000 7f9ad1aa0640 -1 Falling back to public interface 2024-03-29T08:15:49.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:15:49 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:15:49.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:15:49 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:15:49.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:15:49 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T08:15:49.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:15:49 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:15:49.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:15:49 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:15:49.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:15:49 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T08:15:50.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:15:50 smithi037 ceph-mon[142325]: pgmap v130: 65 pgs: 19 active+undersized+degraded, 19 active+undersized, 27 active+clean; 259 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 66/324 objects degraded (20.370%) 2024-03-29T08:15:50.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:15:50 smithi148 ceph-mon[106692]: pgmap v130: 65 pgs: 19 active+undersized+degraded, 19 active+undersized, 27 active+clean; 259 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 66/324 objects degraded (20.370%) 2024-03-29T08:15:52.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:15:52 smithi037 ceph-mon[142325]: pgmap v131: 65 pgs: 19 active+undersized+degraded, 19 active+undersized, 27 active+clean; 259 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 66/324 objects degraded (20.370%) 2024-03-29T08:15:52.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:15:52 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:15:52.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:15:52 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:15:53.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:15:52 smithi148 ceph-mon[106692]: pgmap v131: 65 pgs: 19 active+undersized+degraded, 19 active+undersized, 27 active+clean; 259 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 66/324 objects degraded (20.370%) 2024-03-29T08:15:53.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:15:52 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:15:53.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:15:52 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:15:54.806 INFO:journalctl@ceph.osd.3.smithi148.stdout:Mar 29 08:15:54 smithi148 ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4-osd-3[118708]: 2024-03-29T08:15:54.577+0000 7f9ad1aa0640 -1 osd.3 65 log_to_monitors true 2024-03-29T08:15:54.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:15:54 smithi148 ceph-mon[106692]: pgmap v132: 65 pgs: 19 active+undersized+degraded, 19 active+undersized, 27 active+clean; 259 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 66/324 objects degraded (20.370%) 2024-03-29T08:15:54.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:15:54 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:15:54.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:15:54 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:15:54.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:15:54 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:15:54.807 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:15:54 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:15:54.807 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:15:54 smithi148 ceph-mon[106692]: from='osd.3 [v2:172.21.15.148:6800/227480876,v1:172.21.15.148:6801/227480876]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-29T08:15:54.807 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:15:54 smithi148 ceph-mon[106692]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-29T08:15:54.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:15:54 smithi037 ceph-mon[142325]: pgmap v132: 65 pgs: 19 active+undersized+degraded, 19 active+undersized, 27 active+clean; 259 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 66/324 objects degraded (20.370%) 2024-03-29T08:15:54.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:15:54 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:15:54.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:15:54 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:15:54.928 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:15:54 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:15:54.928 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:15:54 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:15:54.928 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:15:54 smithi037 ceph-mon[142325]: from='osd.3 [v2:172.21.15.148:6800/227480876,v1:172.21.15.148:6801/227480876]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-29T08:15:54.928 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:15:54 smithi037 ceph-mon[142325]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-29T08:15:56.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:15:56 smithi037 ceph-mon[142325]: pgmap v133: 65 pgs: 19 active+undersized+degraded, 19 active+undersized, 27 active+clean; 259 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 66/324 objects degraded (20.370%) 2024-03-29T08:15:56.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:15:56 smithi037 ceph-mon[142325]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-29T08:15:56.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:15:56 smithi037 ceph-mon[142325]: osdmap e68: 6 total, 5 up, 6 in 2024-03-29T08:15:56.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:15:56 smithi037 ceph-mon[142325]: from='osd.3 [v2:172.21.15.148:6800/227480876,v1:172.21.15.148:6801/227480876]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-03-29T08:15:56.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:15:56 smithi037 ceph-mon[142325]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-03-29T08:15:57.056 INFO:journalctl@ceph.osd.3.smithi148.stdout:Mar 29 08:15:56 smithi148 ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4-osd-3[118708]: 2024-03-29T08:15:56.621+0000 7f9abbe00700 -1 osd.3 65 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-29T08:15:57.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:15:56 smithi148 ceph-mon[106692]: pgmap v133: 65 pgs: 19 active+undersized+degraded, 19 active+undersized, 27 active+clean; 259 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 66/324 objects degraded (20.370%) 2024-03-29T08:15:57.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:15:56 smithi148 ceph-mon[106692]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-29T08:15:57.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:15:56 smithi148 ceph-mon[106692]: osdmap e68: 6 total, 5 up, 6 in 2024-03-29T08:15:57.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:15:56 smithi148 ceph-mon[106692]: from='osd.3 [v2:172.21.15.148:6800/227480876,v1:172.21.15.148:6801/227480876]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-03-29T08:15:57.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:15:56 smithi148 ceph-mon[106692]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-03-29T08:15:57.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:15:57 smithi037 ceph-mon[142325]: from='osd.3 ' entity='osd.3' 2024-03-29T08:15:58.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:15:57 smithi148 ceph-mon[106692]: from='osd.3 ' entity='osd.3' 2024-03-29T08:15:58.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:15:58 smithi037 ceph-mon[142325]: pgmap v135: 65 pgs: 19 active+undersized+degraded, 19 active+undersized, 27 active+clean; 259 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 66/324 objects degraded (20.370%) 2024-03-29T08:15:58.928 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:15:58 smithi037 ceph-mon[142325]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-29T08:15:58.928 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:15:58 smithi037 ceph-mon[142325]: osd.3 [v2:172.21.15.148:6800/227480876,v1:172.21.15.148:6801/227480876] boot 2024-03-29T08:15:58.928 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:15:58 smithi037 ceph-mon[142325]: osdmap e69: 6 total, 6 up, 6 in 2024-03-29T08:15:58.928 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:15:58 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-29T08:15:58.928 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:15:58 smithi037 ceph-mon[142325]: osdmap e70: 6 total, 6 up, 6 in 2024-03-29T08:15:59.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:15:58 smithi148 ceph-mon[106692]: pgmap v135: 65 pgs: 19 active+undersized+degraded, 19 active+undersized, 27 active+clean; 259 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 66/324 objects degraded (20.370%) 2024-03-29T08:15:59.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:15:58 smithi148 ceph-mon[106692]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-29T08:15:59.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:15:58 smithi148 ceph-mon[106692]: osd.3 [v2:172.21.15.148:6800/227480876,v1:172.21.15.148:6801/227480876] boot 2024-03-29T08:15:59.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:15:58 smithi148 ceph-mon[106692]: osdmap e69: 6 total, 6 up, 6 in 2024-03-29T08:15:59.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:15:58 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-29T08:15:59.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:15:58 smithi148 ceph-mon[106692]: osdmap e70: 6 total, 6 up, 6 in 2024-03-29T08:16:00.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:15:59 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:16:00.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:15:59 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:16:00.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:15:59 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T08:16:00.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:15:59 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T08:16:00.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:15:59 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:16:00.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:15:59 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T08:16:00.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:15:59 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:16:00.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:15:59 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:16:00.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:15:59 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:16:00.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:15:59 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-03-29T08:16:00.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:15:59 smithi148 ceph-mon[106692]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-03-29T08:16:00.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:15:59 smithi148 ceph-mon[106692]: Upgrade: unsafe to stop osd(s) at this time (18 PGs are or would become offline) 2024-03-29T08:16:00.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:15:59 smithi148 ceph-mon[106692]: pgmap v138: 65 pgs: 7 peering, 13 active+undersized+degraded, 11 active+undersized, 34 active+clean; 259 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 49/324 objects degraded (15.123%) 2024-03-29T08:16:00.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:15:59 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:16:00.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:15:59 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:16:00.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:15:59 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T08:16:00.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:15:59 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T08:16:00.178 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:15:59 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:16:00.178 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:15:59 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T08:16:00.178 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:15:59 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:16:00.178 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:15:59 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:16:00.178 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:15:59 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:16:00.178 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:15:59 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-03-29T08:16:00.178 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:15:59 smithi037 ceph-mon[142325]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-03-29T08:16:00.178 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:15:59 smithi037 ceph-mon[142325]: Upgrade: unsafe to stop osd(s) at this time (18 PGs are or would become offline) 2024-03-29T08:16:00.178 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:15:59 smithi037 ceph-mon[142325]: pgmap v138: 65 pgs: 7 peering, 13 active+undersized+degraded, 11 active+undersized, 34 active+clean; 259 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 49/324 objects degraded (15.123%) 2024-03-29T08:16:01.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:16:00 smithi148 ceph-mon[106692]: Health check update: Degraded data redundancy: 49/324 objects degraded (15.123%), 13 pgs degraded (PG_DEGRADED) 2024-03-29T08:16:01.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:16:00 smithi037 ceph-mon[142325]: Health check update: Degraded data redundancy: 49/324 objects degraded (15.123%), 13 pgs degraded (PG_DEGRADED) 2024-03-29T08:16:02.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:16:01 smithi148 ceph-mon[106692]: pgmap v139: 65 pgs: 7 peering, 58 active+clean; 259 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:16:02.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:16:01 smithi037 ceph-mon[142325]: pgmap v139: 65 pgs: 7 peering, 58 active+clean; 259 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:16:03.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:16:02 smithi148 ceph-mon[106692]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 49/324 objects degraded (15.123%), 13 pgs degraded) 2024-03-29T08:16:03.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:16:02 smithi148 ceph-mon[106692]: Cluster is now healthy 2024-03-29T08:16:03.149 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:16:02 smithi037 ceph-mon[142325]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 49/324 objects degraded (15.123%), 13 pgs degraded) 2024-03-29T08:16:03.149 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:16:02 smithi037 ceph-mon[142325]: Cluster is now healthy 2024-03-29T08:16:04.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:16:03 smithi148 ceph-mon[106692]: pgmap v140: 65 pgs: 7 peering, 58 active+clean; 259 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:16:04.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:16:03 smithi037 ceph-mon[142325]: pgmap v140: 65 pgs: 7 peering, 58 active+clean; 259 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:16:06.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:16:06 smithi037 ceph-mon[142325]: pgmap v141: 65 pgs: 65 active+clean; 259 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:16:07.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:16:06 smithi148 ceph-mon[106692]: pgmap v141: 65 pgs: 65 active+clean; 259 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:16:08.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:16:08 smithi037 ceph-mon[142325]: pgmap v142: 65 pgs: 65 active+clean; 259 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:16:09.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:16:08 smithi148 ceph-mon[106692]: pgmap v142: 65 pgs: 65 active+clean; 259 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:16:10.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:16:10 smithi037 ceph-mon[142325]: pgmap v143: 65 pgs: 65 active+clean; 259 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:16:10.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:16:10 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:16:10.928 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:16:10 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:16:11.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:16:10 smithi148 ceph-mon[106692]: pgmap v143: 65 pgs: 65 active+clean; 259 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:16:11.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:16:10 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:16:11.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:16:10 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:16:12.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:16:12 smithi037 ceph-mon[142325]: pgmap v144: 65 pgs: 65 active+clean; 259 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:16:13.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:16:12 smithi148 ceph-mon[106692]: pgmap v144: 65 pgs: 65 active+clean; 259 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:16:14.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:16:14 smithi037 ceph-mon[142325]: pgmap v145: 65 pgs: 65 active+clean; 259 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:16:14.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:16:14 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-03-29T08:16:15.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:16:14 smithi148 ceph-mon[106692]: pgmap v145: 65 pgs: 65 active+clean; 259 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:16:15.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:16:14 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-03-29T08:16:15.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:16:15 smithi148 ceph-mon[106692]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-03-29T08:16:15.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:16:15 smithi148 ceph-mon[106692]: Upgrade: osd.4 is safe to restart 2024-03-29T08:16:15.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:16:15 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:16:15.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:16:15 smithi037 ceph-mon[142325]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-03-29T08:16:15.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:16:15 smithi037 ceph-mon[142325]: Upgrade: osd.4 is safe to restart 2024-03-29T08:16:15.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:16:15 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:16:16.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:16.610+0000 7fb575c00700 1 -- 172.21.15.37:0/1365967440 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb5701037a0 msgr2=0x7fb5700fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:16:16.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:16.610+0000 7fb575c00700 1 --2- 172.21.15.37:0/1365967440 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb5701037a0 0x7fb5700fce10 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fb55c004940 tx=0x7fb55c009e30 comp rx=0 tx=0).stop 2024-03-29T08:16:16.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:16.610+0000 7fb575c00700 1 -- 172.21.15.37:0/1365967440 shutdown_connections 2024-03-29T08:16:16.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:16.610+0000 7fb575c00700 1 --2- 172.21.15.37:0/1365967440 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb5701037a0 0x7fb5700fce10 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:16:16.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:16.610+0000 7fb575c00700 1 --2- 172.21.15.37:0/1365967440 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb570102e00 0x7fb5701031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:16:16.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:16.610+0000 7fb575c00700 1 -- 172.21.15.37:0/1365967440 >> 172.21.15.37:0/1365967440 conn(0x7fb5700f89b0 msgr2=0x7fb5700fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:16:16.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:16.610+0000 7fb575c00700 1 -- 172.21.15.37:0/1365967440 shutdown_connections 2024-03-29T08:16:16.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:16.610+0000 7fb575c00700 1 -- 172.21.15.37:0/1365967440 wait complete. 2024-03-29T08:16:16.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:16.611+0000 7fb575c00700 1 Processor -- start 2024-03-29T08:16:16.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:16.611+0000 7fb575c00700 1 -- start start 2024-03-29T08:16:16.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:16.611+0000 7fb575c00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb570102e00 0x7fb570115c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:16:16.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:16.611+0000 7fb575c00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb5701037a0 0x7fb5701161c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:16:16.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:16.611+0000 7fb575c00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb570116810 con 0x7fb570102e00 2024-03-29T08:16:16.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:16.611+0000 7fb575c00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb57010fbe0 con 0x7fb5701037a0 2024-03-29T08:16:16.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:16.612+0000 7fb56ea00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb570102e00 0x7fb570115c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:16:16.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:16.612+0000 7fb56ea00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb570102e00 0x7fb570115c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34578/0 (socket says 172.21.15.37:34578) 2024-03-29T08:16:16.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:16.612+0000 7fb56ea00700 1 -- 172.21.15.37:0/2882586108 learned_addr learned my addr 172.21.15.37:0/2882586108 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:16:16.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:16.612+0000 7fb56e000700 1 --2- 172.21.15.37:0/2882586108 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb5701037a0 0x7fb5701161c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:16:16.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:16.612+0000 7fb56ea00700 1 -- 172.21.15.37:0/2882586108 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb5701037a0 msgr2=0x7fb5701161c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:16:16.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:16.612+0000 7fb56ea00700 1 --2- 172.21.15.37:0/2882586108 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb5701037a0 0x7fb5701161c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:16:16.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:16.612+0000 7fb56ea00700 1 -- 172.21.15.37:0/2882586108 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb55c005040 con 0x7fb570102e00 2024-03-29T08:16:16.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:16.612+0000 7fb56ea00700 1 --2- 172.21.15.37:0/2882586108 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb570102e00 0x7fb570115c80 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fb560002a80 tx=0x7fb5600046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:16:16.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:16.613+0000 7fb566a00700 1 -- 172.21.15.37:0/2882586108 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb560007cc0 con 0x7fb570102e00 2024-03-29T08:16:16.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:16.613+0000 7fb566a00700 1 -- 172.21.15.37:0/2882586108 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fb56000f040 con 0x7fb570102e00 2024-03-29T08:16:16.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:16.613+0000 7fb566a00700 1 -- 172.21.15.37:0/2882586108 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb56000da60 con 0x7fb570102e00 2024-03-29T08:16:16.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:16.613+0000 7fb575c00700 1 -- 172.21.15.37:0/2882586108 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb57010fec0 con 0x7fb570102e00 2024-03-29T08:16:16.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:16.613+0000 7fb575c00700 1 -- 172.21.15.37:0/2882586108 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb570110430 con 0x7fb570102e00 2024-03-29T08:16:16.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:16.614+0000 7fb575c00700 1 -- 172.21.15.37:0/2882586108 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5700fe550 con 0x7fb570102e00 2024-03-29T08:16:16.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:16.615+0000 7fb566a00700 1 -- 172.21.15.37:0/2882586108 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fb56000dbc0 con 0x7fb570102e00 2024-03-29T08:16:16.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:16.616+0000 7fb566a00700 1 --2- 172.21.15.37:0/2882586108 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fb55406cc30 0x7fb55406f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:16:16.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:16.616+0000 7fb566a00700 1 -- 172.21.15.37:0/2882586108 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fb56008bca0 con 0x7fb570102e00 2024-03-29T08:16:16.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:16.616+0000 7fb56e000700 1 --2- 172.21.15.37:0/2882586108 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fb55406cc30 0x7fb55406f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:16:16.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:16.617+0000 7fb56e000700 1 --2- 172.21.15.37:0/2882586108 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fb55406cc30 0x7fb55406f0e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fb570111520 tx=0x7fb55c009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:16:16.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:16.621+0000 7fb566a00700 1 -- 172.21.15.37:0/2882586108 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fb5600596f0 con 0x7fb570102e00 2024-03-29T08:16:16.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:16.851+0000 7fb575c00700 1 -- 172.21.15.37:0/2882586108 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb57004bf40 con 0x7fb55406cc30 2024-03-29T08:16:16.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:16.852+0000 7fb566a00700 1 -- 172.21.15.37:0/2882586108 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7fb57004bf40 con 0x7fb55406cc30 2024-03-29T08:16:16.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:16.854+0000 7fb575c00700 1 -- 172.21.15.37:0/2882586108 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fb55406cc30 msgr2=0x7fb55406f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:16:16.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:16.854+0000 7fb575c00700 1 --2- 172.21.15.37:0/2882586108 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fb55406cc30 0x7fb55406f0e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fb570111520 tx=0x7fb55c009e30 comp rx=0 tx=0).stop 2024-03-29T08:16:16.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:16.854+0000 7fb575c00700 1 -- 172.21.15.37:0/2882586108 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb570102e00 msgr2=0x7fb570115c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:16:16.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:16.854+0000 7fb575c00700 1 --2- 172.21.15.37:0/2882586108 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb570102e00 0x7fb570115c80 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fb560002a80 tx=0x7fb5600046b0 comp rx=0 tx=0).stop 2024-03-29T08:16:16.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:16.854+0000 7fb575c00700 1 -- 172.21.15.37:0/2882586108 shutdown_connections 2024-03-29T08:16:16.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:16.854+0000 7fb575c00700 1 --2- 172.21.15.37:0/2882586108 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fb55406cc30 0x7fb55406f0e0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:16:16.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:16.854+0000 7fb575c00700 1 --2- 172.21.15.37:0/2882586108 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb570102e00 0x7fb570115c80 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:16:16.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:16.854+0000 7fb575c00700 1 --2- 172.21.15.37:0/2882586108 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb5701037a0 0x7fb5701161c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:16:16.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:16.854+0000 7fb575c00700 1 -- 172.21.15.37:0/2882586108 >> 172.21.15.37:0/2882586108 conn(0x7fb5700f89b0 msgr2=0x7fb5700f9a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:16:16.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:16.855+0000 7fb575c00700 1 -- 172.21.15.37:0/2882586108 shutdown_connections 2024-03-29T08:16:16.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:16.855+0000 7fb575c00700 1 -- 172.21.15.37:0/2882586108 wait complete. 2024-03-29T08:16:16.869 INFO:teuthology.orchestra.run.smithi037.stdout:true 2024-03-29T08:16:16.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:16:16 smithi037 ceph-mon[142325]: pgmap v146: 65 pgs: 65 active+clean; 259 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:16:16.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:16:16 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:16:16.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:16:16 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-29T08:16:16.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:16:16 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T08:16:16.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:16.993+0000 7f52b1000700 1 -- 172.21.15.37:0/1881174414 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f52ac1037a0 msgr2=0x7f52ac0fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:16:16.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:16.993+0000 7f52b1000700 1 --2- 172.21.15.37:0/1881174414 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f52ac1037a0 0x7f52ac0fce10 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f5298004940 tx=0x7f5298009e30 comp rx=0 tx=0).stop 2024-03-29T08:16:16.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:16.993+0000 7f52b1000700 1 -- 172.21.15.37:0/1881174414 shutdown_connections 2024-03-29T08:16:16.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:16.993+0000 7f52b1000700 1 --2- 172.21.15.37:0/1881174414 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f52ac1037a0 0x7f52ac0fce10 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:16:16.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:16.993+0000 7f52b1000700 1 --2- 172.21.15.37:0/1881174414 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f52ac102e00 0x7f52ac1031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:16:16.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:16.993+0000 7f52b1000700 1 -- 172.21.15.37:0/1881174414 >> 172.21.15.37:0/1881174414 conn(0x7f52ac0f89b0 msgr2=0x7f52ac0fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:16:16.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:16.993+0000 7f52b1000700 1 -- 172.21.15.37:0/1881174414 shutdown_connections 2024-03-29T08:16:16.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:16.993+0000 7f52b1000700 1 -- 172.21.15.37:0/1881174414 wait complete. 2024-03-29T08:16:16.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:16.994+0000 7f52b1000700 1 Processor -- start 2024-03-29T08:16:16.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:16.994+0000 7f52b1000700 1 -- start start 2024-03-29T08:16:16.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:16.995+0000 7f52b1000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f52ac102e00 0x7f52ac0702c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:16:16.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:16.995+0000 7f52b1000700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f52ac1037a0 0x7f52ac070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:16:16.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:16.995+0000 7f52b1000700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52ac070d90 con 0x7f52ac102e00 2024-03-29T08:16:16.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:16.995+0000 7f52b1000700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52ac070ed0 con 0x7f52ac1037a0 2024-03-29T08:16:16.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:16.995+0000 7f52a9e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f52ac102e00 0x7f52ac0702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:16:16.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:16.995+0000 7f52a9e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f52ac102e00 0x7f52ac0702c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34588/0 (socket says 172.21.15.37:34588) 2024-03-29T08:16:16.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:16.995+0000 7f52a9e00700 1 -- 172.21.15.37:0/1373250517 learned_addr learned my addr 172.21.15.37:0/1373250517 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:16:16.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:16.995+0000 7f52a9400700 1 --2- 172.21.15.37:0/1373250517 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f52ac1037a0 0x7f52ac070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:16:16.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:16.995+0000 7f52a9e00700 1 -- 172.21.15.37:0/1373250517 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f52ac1037a0 msgr2=0x7f52ac070800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:16:16.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:16.995+0000 7f52a9e00700 1 --2- 172.21.15.37:0/1373250517 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f52ac1037a0 0x7f52ac070800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:16:16.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:16.995+0000 7f52a9e00700 1 -- 172.21.15.37:0/1373250517 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5298005040 con 0x7f52ac102e00 2024-03-29T08:16:16.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:16.996+0000 7f52a9e00700 1 --2- 172.21.15.37:0/1373250517 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f52ac102e00 0x7f52ac0702c0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f529c002a50 tx=0x7f529c0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:16:16.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:16.996+0000 7f52a2000700 1 -- 172.21.15.37:0/1373250517 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f529c007cc0 con 0x7f52ac102e00 2024-03-29T08:16:16.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:16.996+0000 7f52a2000700 1 -- 172.21.15.37:0/1373250517 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f529c00f040 con 0x7f52ac102e00 2024-03-29T08:16:16.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:16.996+0000 7f52a2000700 1 -- 172.21.15.37:0/1373250517 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f529c00d9c0 con 0x7f52ac102e00 2024-03-29T08:16:16.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:16.996+0000 7f52b1000700 1 -- 172.21.15.37:0/1373250517 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52ac119e00 con 0x7f52ac102e00 2024-03-29T08:16:16.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:16.996+0000 7f52b1000700 1 -- 172.21.15.37:0/1373250517 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52ac11a280 con 0x7f52ac102e00 2024-03-29T08:16:16.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:16.997+0000 7f52b1000700 1 -- 172.21.15.37:0/1373250517 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52ac0fe550 con 0x7f52ac102e00 2024-03-29T08:16:16.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:16.998+0000 7f52a2000700 1 -- 172.21.15.37:0/1373250517 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f529c00db20 con 0x7f52ac102e00 2024-03-29T08:16:16.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:16.999+0000 7f52a2000700 1 --2- 172.21.15.37:0/1373250517 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f529006cc30 0x7f529006f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:16:16.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:16.999+0000 7f52a2000700 1 -- 172.21.15.37:0/1373250517 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f529c08bbe0 con 0x7f52ac102e00 2024-03-29T08:16:16.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:16.999+0000 7f52a9400700 1 --2- 172.21.15.37:0/1373250517 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f529006cc30 0x7f529006f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:16:17.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:17.000+0000 7f52a9400700 1 --2- 172.21.15.37:0/1373250517 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f529006cc30 0x7f529006f0e0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f52ac071c00 tx=0x7f5298009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:16:17.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:17.004+0000 7f52a2000700 1 -- 172.21.15.37:0/1373250517 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f529c055cc0 con 0x7f52ac102e00 2024-03-29T08:16:17.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:16:16 smithi148 ceph-mon[106692]: pgmap v146: 65 pgs: 65 active+clean; 259 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:16:17.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:16:16 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:16:17.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:16:16 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-29T08:16:17.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:16:16 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T08:16:17.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:17.236+0000 7f52b1000700 1 -- 172.21.15.37:0/1373250517 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f52ac0719b0 con 0x7f529006cc30 2024-03-29T08:16:17.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:17.237+0000 7f52a2000700 1 -- 172.21.15.37:0/1373250517 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f52ac0719b0 con 0x7f529006cc30 2024-03-29T08:16:17.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:17.240+0000 7f52b1000700 1 -- 172.21.15.37:0/1373250517 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f529006cc30 msgr2=0x7f529006f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:16:17.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:17.240+0000 7f52b1000700 1 --2- 172.21.15.37:0/1373250517 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f529006cc30 0x7f529006f0e0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f52ac071c00 tx=0x7f5298009e30 comp rx=0 tx=0).stop 2024-03-29T08:16:17.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:17.240+0000 7f52b1000700 1 -- 172.21.15.37:0/1373250517 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f52ac102e00 msgr2=0x7f52ac0702c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:16:17.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:17.240+0000 7f52b1000700 1 --2- 172.21.15.37:0/1373250517 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f52ac102e00 0x7f52ac0702c0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f529c002a50 tx=0x7f529c0046b0 comp rx=0 tx=0).stop 2024-03-29T08:16:17.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:17.240+0000 7f52b1000700 1 -- 172.21.15.37:0/1373250517 shutdown_connections 2024-03-29T08:16:17.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:17.240+0000 7f52b1000700 1 --2- 172.21.15.37:0/1373250517 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f529006cc30 0x7f529006f0e0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:16:17.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:17.240+0000 7f52b1000700 1 --2- 172.21.15.37:0/1373250517 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f52ac102e00 0x7f52ac0702c0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:16:17.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:17.240+0000 7f52b1000700 1 --2- 172.21.15.37:0/1373250517 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f52ac1037a0 0x7f52ac070800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:16:17.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:17.240+0000 7f52b1000700 1 -- 172.21.15.37:0/1373250517 >> 172.21.15.37:0/1373250517 conn(0x7f52ac0f89b0 msgr2=0x7f52ac0f9ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:16:17.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:17.240+0000 7f52b1000700 1 -- 172.21.15.37:0/1373250517 shutdown_connections 2024-03-29T08:16:17.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:17.240+0000 7f52b1000700 1 -- 172.21.15.37:0/1373250517 wait complete. 2024-03-29T08:16:17.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:17.376+0000 7f18df800700 1 -- 172.21.15.37:0/3412104291 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f18d8102e00 msgr2=0x7f18d81031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:16:17.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:17.376+0000 7f18df800700 1 --2- 172.21.15.37:0/3412104291 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f18d8102e00 0x7f18d81031d0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f18cc004940 tx=0x7f18cc009e30 comp rx=0 tx=0).stop 2024-03-29T08:16:17.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:17.376+0000 7f18df800700 1 -- 172.21.15.37:0/3412104291 shutdown_connections 2024-03-29T08:16:17.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:17.376+0000 7f18df800700 1 --2- 172.21.15.37:0/3412104291 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f18d81037a0 0x7f18d80fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:16:17.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:17.377+0000 7f18df800700 1 --2- 172.21.15.37:0/3412104291 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f18d8102e00 0x7f18d81031d0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:16:17.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:17.377+0000 7f18df800700 1 -- 172.21.15.37:0/3412104291 >> 172.21.15.37:0/3412104291 conn(0x7f18d80f89b0 msgr2=0x7f18d80fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:16:17.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:17.377+0000 7f18df800700 1 -- 172.21.15.37:0/3412104291 shutdown_connections 2024-03-29T08:16:17.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:17.377+0000 7f18df800700 1 -- 172.21.15.37:0/3412104291 wait complete. 2024-03-29T08:16:17.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:17.377+0000 7f18df800700 1 Processor -- start 2024-03-29T08:16:17.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:17.378+0000 7f18df800700 1 -- start start 2024-03-29T08:16:17.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:17.378+0000 7f18df800700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f18d8102e00 0x7f18d8115c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:16:17.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:17.378+0000 7f18df800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f18d81037a0 0x7f18d81161c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:16:17.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:17.378+0000 7f18df800700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18d8116810 con 0x7f18d81037a0 2024-03-29T08:16:17.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:17.378+0000 7f18df800700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18d810fbe0 con 0x7f18d8102e00 2024-03-29T08:16:17.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:17.378+0000 7f18d7e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f18d81037a0 0x7f18d81161c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:16:17.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:17.378+0000 7f18d7e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f18d81037a0 0x7f18d81161c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34596/0 (socket says 172.21.15.37:34596) 2024-03-29T08:16:17.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:17.378+0000 7f18d7e00700 1 -- 172.21.15.37:0/2403609380 learned_addr learned my addr 172.21.15.37:0/2403609380 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:16:17.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:17.378+0000 7f18dce00700 1 --2- 172.21.15.37:0/2403609380 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f18d8102e00 0x7f18d8115c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:16:17.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:17.379+0000 7f18d7e00700 1 -- 172.21.15.37:0/2403609380 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f18d8102e00 msgr2=0x7f18d8115c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:16:17.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:17.379+0000 7f18d7e00700 1 --2- 172.21.15.37:0/2403609380 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f18d8102e00 0x7f18d8115c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:16:17.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:17.379+0000 7f18d7e00700 1 -- 172.21.15.37:0/2403609380 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18cc005040 con 0x7f18d81037a0 2024-03-29T08:16:17.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:17.379+0000 7f18dce00700 1 --2- 172.21.15.37:0/2403609380 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f18d8102e00 0x7f18d8115c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:16:17.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:17.379+0000 7f18d7e00700 1 --2- 172.21.15.37:0/2403609380 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f18d81037a0 0x7f18d81161c0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f18c400eb40 tx=0x7f18c400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:16:17.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:17.379+0000 7f18d4c00700 1 -- 172.21.15.37:0/2403609380 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f18c4007cc0 con 0x7f18d81037a0 2024-03-29T08:16:17.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:17.379+0000 7f18d4c00700 1 -- 172.21.15.37:0/2403609380 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f18c400f040 con 0x7f18d81037a0 2024-03-29T08:16:17.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:17.379+0000 7f18d4c00700 1 -- 172.21.15.37:0/2403609380 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f18c4009570 con 0x7f18d81037a0 2024-03-29T08:16:17.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:17.379+0000 7f18df800700 1 -- 172.21.15.37:0/2403609380 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18d810fe60 con 0x7f18d81037a0 2024-03-29T08:16:17.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:17.379+0000 7f18df800700 1 -- 172.21.15.37:0/2403609380 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18d8110420 con 0x7f18d81037a0 2024-03-29T08:16:17.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:17.380+0000 7f18df800700 1 -- 172.21.15.37:0/2403609380 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18d80fe550 con 0x7f18d81037a0 2024-03-29T08:16:17.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:17.381+0000 7f18d4c00700 1 -- 172.21.15.37:0/2403609380 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f18c40096d0 con 0x7f18d81037a0 2024-03-29T08:16:17.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:17.383+0000 7f18d4c00700 1 --2- 172.21.15.37:0/2403609380 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f18c006cc30 0x7f18c006f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:16:17.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:17.384+0000 7f18d4c00700 1 -- 172.21.15.37:0/2403609380 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f18c408bb90 con 0x7f18d81037a0 2024-03-29T08:16:17.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:17.384+0000 7f18dce00700 1 --2- 172.21.15.37:0/2403609380 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f18c006cc30 0x7f18c006f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:16:17.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:17.384+0000 7f18dce00700 1 --2- 172.21.15.37:0/2403609380 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f18c006cc30 0x7f18c006f0e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f18cc00b0d0 tx=0x7f18cc009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:16:17.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:17.388+0000 7f18d4c00700 1 -- 172.21.15.37:0/2403609380 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f18c4059690 con 0x7f18d81037a0 2024-03-29T08:16:17.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:17.610+0000 7f18df800700 1 -- 172.21.15.37:0/2403609380 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f18d8109e30 con 0x7f18c006cc30 2024-03-29T08:16:17.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:17.620+0000 7f18d4c00700 1 -- 172.21.15.37:0/2403609380 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7f18d8109e30 con 0x7f18c006cc30 2024-03-29T08:16:17.621 INFO:teuthology.orchestra.run.smithi037.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-29T08:16:17.621 INFO:teuthology.orchestra.run.smithi037.stdout:alertmanager.smithi037 smithi037 *:9093,9094 running (6m) 63s ago 19m 14.1M - 0.25.0 c8568f914cd2 2f9f9746ed0f 2024-03-29T08:16:17.621 INFO:teuthology.orchestra.run.smithi037.stdout:crash.smithi037 smithi037 running (3m) 63s ago 19m 6647k - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 178d416a3186 2024-03-29T08:16:17.621 INFO:teuthology.orchestra.run.smithi037.stdout:crash.smithi148 smithi148 running (2m) 25s ago 18m 6656k - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 23666989d58f 2024-03-29T08:16:17.621 INFO:teuthology.orchestra.run.smithi037.stdout:grafana.smithi037 smithi037 *:3000 running (5m) 63s ago 18m 73.4M - 9.4.7 954c08fa6188 6b976fe49952 2024-03-29T08:16:17.621 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi037.iifsmh smithi037 running (12m) 63s ago 12m 110M - 17.2.7-789-g4213e17c cf22eb8683be 074e1acf331f 2024-03-29T08:16:17.621 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi037.mmwgxe smithi037 running (12m) 63s ago 12m 114M - 17.2.7-789-g4213e17c cf22eb8683be 9a5b1e72de5c 2024-03-29T08:16:17.621 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi148.qukpjh smithi148 running (12m) 25s ago 12m 14.7M - 17.2.7-789-g4213e17c cf22eb8683be 8e469782a273 2024-03-29T08:16:17.621 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi148.sqteih smithi148 running (12m) 25s ago 12m 14.7M - 17.2.7-789-g4213e17c cf22eb8683be 8e68f858bb22 2024-03-29T08:16:17.621 INFO:teuthology.orchestra.run.smithi037.stdout:mgr.smithi037.wjikii smithi037 *:8443,9283,8765 running (8m) 63s ago 20m 501M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff c5a1417b9417 2024-03-29T08:16:17.621 INFO:teuthology.orchestra.run.smithi037.stdout:mgr.smithi148.onhgvm smithi148 *:8443,9283,8765 running (7m) 25s ago 17m 449M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff fcfdbce062bd 2024-03-29T08:16:17.621 INFO:teuthology.orchestra.run.smithi037.stdout:mon.smithi037 smithi037 running (4m) 63s ago 20m 47.5M 2048M 18.2.2-1015-gd0e392d9 1cc2f45c17ff eea2060783af 2024-03-29T08:16:17.621 INFO:teuthology.orchestra.run.smithi037.stdout:mon.smithi148 smithi148 running (3m) 25s ago 17m 39.5M 2048M 18.2.2-1015-gd0e392d9 1cc2f45c17ff ad4b2c5748be 2024-03-29T08:16:17.621 INFO:teuthology.orchestra.run.smithi037.stdout:node-exporter.smithi037 smithi037 *:9100 running (6m) 63s ago 19m 7847k - 1.5.0 0da6a335fe13 26b8ae9cb878 2024-03-29T08:16:17.622 INFO:teuthology.orchestra.run.smithi037.stdout:node-exporter.smithi148 smithi148 *:9100 running (6m) 25s ago 18m 7470k - 1.5.0 0da6a335fe13 98414e4e64ed 2024-03-29T08:16:17.622 INFO:teuthology.orchestra.run.smithi037.stdout:osd.0 smithi037 running (2m) 63s ago 16m 99.0M 1619M 18.2.2-1015-gd0e392d9 1cc2f45c17ff c0c584f6245c 2024-03-29T08:16:17.622 INFO:teuthology.orchestra.run.smithi037.stdout:osd.1 smithi037 running (111s) 63s ago 16m 97.2M 1619M 18.2.2-1015-gd0e392d9 1cc2f45c17ff a4a2ff4cc83d 2024-03-29T08:16:17.622 INFO:teuthology.orchestra.run.smithi037.stdout:osd.2 smithi037 running (70s) 63s ago 15m 11.0M 1619M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 8b1bc71aa79e 2024-03-29T08:16:17.622 INFO:teuthology.orchestra.run.smithi037.stdout:osd.3 smithi148 running (29s) 25s ago 15m 10.9M 2642M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 1c656ab8ff86 2024-03-29T08:16:17.622 INFO:teuthology.orchestra.run.smithi037.stdout:osd.4 smithi148 running (14m) 25s ago 14m 233M 2642M 17.2.7-789-g4213e17c cf22eb8683be a00b69047ed7 2024-03-29T08:16:17.622 INFO:teuthology.orchestra.run.smithi037.stdout:osd.5 smithi148 running (14m) 25s ago 14m 233M 2642M 17.2.7-789-g4213e17c cf22eb8683be 044a3f7db7f8 2024-03-29T08:16:17.622 INFO:teuthology.orchestra.run.smithi037.stdout:prometheus.smithi037 smithi037 *:9095 running (6m) 63s ago 18m 52.1M - 2.43.0 a07b618ecd1d 30c0223ec589 2024-03-29T08:16:17.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:17.623+0000 7f18df800700 1 -- 172.21.15.37:0/2403609380 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f18c006cc30 msgr2=0x7f18c006f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:16:17.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:17.623+0000 7f18df800700 1 --2- 172.21.15.37:0/2403609380 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f18c006cc30 0x7f18c006f0e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f18cc00b0d0 tx=0x7f18cc009e30 comp rx=0 tx=0).stop 2024-03-29T08:16:17.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:17.623+0000 7f18df800700 1 -- 172.21.15.37:0/2403609380 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f18d81037a0 msgr2=0x7f18d81161c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:16:17.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:17.623+0000 7f18df800700 1 --2- 172.21.15.37:0/2403609380 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f18d81037a0 0x7f18d81161c0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f18c400eb40 tx=0x7f18c400b040 comp rx=0 tx=0).stop 2024-03-29T08:16:17.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:17.623+0000 7f18df800700 1 -- 172.21.15.37:0/2403609380 shutdown_connections 2024-03-29T08:16:17.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:17.623+0000 7f18df800700 1 --2- 172.21.15.37:0/2403609380 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f18c006cc30 0x7f18c006f0e0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:16:17.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:17.623+0000 7f18df800700 1 --2- 172.21.15.37:0/2403609380 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f18d8102e00 0x7f18d8115c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:16:17.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:17.623+0000 7f18df800700 1 --2- 172.21.15.37:0/2403609380 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f18d81037a0 0x7f18d81161c0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:16:17.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:17.623+0000 7f18df800700 1 -- 172.21.15.37:0/2403609380 >> 172.21.15.37:0/2403609380 conn(0x7f18d80f89b0 msgr2=0x7f18d80f9940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:16:17.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:17.623+0000 7f18df800700 1 -- 172.21.15.37:0/2403609380 shutdown_connections 2024-03-29T08:16:17.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:17.623+0000 7f18df800700 1 -- 172.21.15.37:0/2403609380 wait complete. 2024-03-29T08:16:17.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:17.759+0000 7f702b800700 1 -- 172.21.15.37:0/466153088 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7024071f90 msgr2=0x7f7024070360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:16:17.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:17.759+0000 7f702b800700 1 --2- 172.21.15.37:0/466153088 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7024071f90 0x7f7024070360 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f7018004970 tx=0x7f7018009e30 comp rx=0 tx=0).stop 2024-03-29T08:16:17.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:17.759+0000 7f702b800700 1 -- 172.21.15.37:0/466153088 shutdown_connections 2024-03-29T08:16:17.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:17.759+0000 7f702b800700 1 --2- 172.21.15.37:0/466153088 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f70240708a0 0x7f7024070d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:16:17.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:17.759+0000 7f702b800700 1 --2- 172.21.15.37:0/466153088 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7024071f90 0x7f7024070360 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:16:17.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:17.759+0000 7f702b800700 1 -- 172.21.15.37:0/466153088 >> 172.21.15.37:0/466153088 conn(0x7f70240f89b0 msgr2=0x7f70240fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:16:17.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:17.759+0000 7f702b800700 1 -- 172.21.15.37:0/466153088 shutdown_connections 2024-03-29T08:16:17.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:17.759+0000 7f702b800700 1 -- 172.21.15.37:0/466153088 wait complete. 2024-03-29T08:16:17.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:17.760+0000 7f702b800700 1 Processor -- start 2024-03-29T08:16:17.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:17.760+0000 7f702b800700 1 -- start start 2024-03-29T08:16:17.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:17.760+0000 7f702b800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f70240708a0 0x7f702411a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:16:17.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:17.760+0000 7f702b800700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7024071f90 0x7f702411a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:16:17.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:17.760+0000 7f702b800700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f702411aba0 con 0x7f70240708a0 2024-03-29T08:16:17.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:17.760+0000 7f702b800700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7024114020 con 0x7f7024071f90 2024-03-29T08:16:17.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:17.761+0000 7f7028e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f70240708a0 0x7f702411a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:16:17.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:17.761+0000 7f7028e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f70240708a0 0x7f702411a010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34602/0 (socket says 172.21.15.37:34602) 2024-03-29T08:16:17.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:17.761+0000 7f7028e00700 1 -- 172.21.15.37:0/3065705770 learned_addr learned my addr 172.21.15.37:0/3065705770 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:16:17.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:17.761+0000 7f7023e00700 1 --2- 172.21.15.37:0/3065705770 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7024071f90 0x7f702411a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:16:17.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:17.761+0000 7f7028e00700 1 -- 172.21.15.37:0/3065705770 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7024071f90 msgr2=0x7f702411a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:16:17.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:17.761+0000 7f7028e00700 1 --2- 172.21.15.37:0/3065705770 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7024071f90 0x7f702411a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:16:17.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:17.761+0000 7f7028e00700 1 -- 172.21.15.37:0/3065705770 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7018005040 con 0x7f70240708a0 2024-03-29T08:16:17.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:17.761+0000 7f7028e00700 1 --2- 172.21.15.37:0/3065705770 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f70240708a0 0x7f702411a010 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f7018004f50 tx=0x7f7018009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:16:17.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:17.762+0000 7f7020c00700 1 -- 172.21.15.37:0/3065705770 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f701800e040 con 0x7f70240708a0 2024-03-29T08:16:17.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:17.762+0000 7f702b800700 1 -- 172.21.15.37:0/3065705770 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70241142a0 con 0x7f70240708a0 2024-03-29T08:16:17.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:17.762+0000 7f7020c00700 1 -- 172.21.15.37:0/3065705770 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f7018013420 con 0x7f70240708a0 2024-03-29T08:16:17.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:17.762+0000 7f7020c00700 1 -- 172.21.15.37:0/3065705770 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f701801c630 con 0x7f70240708a0 2024-03-29T08:16:17.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:17.762+0000 7f702b800700 1 -- 172.21.15.37:0/3065705770 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70241147e0 con 0x7f70240708a0 2024-03-29T08:16:17.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:17.763+0000 7f702b800700 1 -- 172.21.15.37:0/3065705770 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7024071aa0 con 0x7f70240708a0 2024-03-29T08:16:17.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:17.764+0000 7f7020c00700 1 -- 172.21.15.37:0/3065705770 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f7018013590 con 0x7f70240708a0 2024-03-29T08:16:17.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:17.765+0000 7f7020c00700 1 --2- 172.21.15.37:0/3065705770 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f700c06cc30 0x7f700c06f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:16:17.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:17.765+0000 7f7020c00700 1 -- 172.21.15.37:0/3065705770 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f701808d250 con 0x7f70240708a0 2024-03-29T08:16:17.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:17.765+0000 7f7023e00700 1 --2- 172.21.15.37:0/3065705770 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f700c06cc30 0x7f700c06f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:16:17.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:17.766+0000 7f7023e00700 1 --2- 172.21.15.37:0/3065705770 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f700c06cc30 0x7f700c06f0e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f702410cfc0 tx=0x7f701000b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:16:17.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:17.769+0000 7f7020c00700 1 -- 172.21.15.37:0/3065705770 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f7018057290 con 0x7f70240708a0 2024-03-29T08:16:17.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:16:17 smithi037 ceph-mon[142325]: Upgrade: Updating osd.4 2024-03-29T08:16:17.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:16:17 smithi037 ceph-mon[142325]: Deploying daemon osd.4 on smithi148 2024-03-29T08:16:18.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:16:17 smithi148 ceph-mon[106692]: Upgrade: Updating osd.4 2024-03-29T08:16:18.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:16:17 smithi148 ceph-mon[106692]: Deploying daemon osd.4 on smithi148 2024-03-29T08:16:18.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:18.067+0000 7f702b800700 1 -- 172.21.15.37:0/3065705770 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f70241103e0 con 0x7f70240708a0 2024-03-29T08:16:18.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:18.067+0000 7f7020c00700 1 -- 172.21.15.37:0/3065705770 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+847 (secure 0 0 0) 0x7f701805a8b0 con 0x7f70240708a0 2024-03-29T08:16:18.067 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-29T08:16:18.068 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": { 2024-03-29T08:16:18.068 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 2 2024-03-29T08:16:18.068 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:16:18.068 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": { 2024-03-29T08:16:18.068 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 2 2024-03-29T08:16:18.068 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:16:18.068 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": { 2024-03-29T08:16:18.068 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 17.2.7-789-g4213e17c (4213e17c4de67016ed14f55d22503f4923b3a901) quincy (stable)": 2, 2024-03-29T08:16:18.068 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 4 2024-03-29T08:16:18.068 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:16:18.068 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": { 2024-03-29T08:16:18.068 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 17.2.7-789-g4213e17c (4213e17c4de67016ed14f55d22503f4923b3a901) quincy (stable)": 4 2024-03-29T08:16:18.068 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:16:18.068 INFO:teuthology.orchestra.run.smithi037.stdout: "overall": { 2024-03-29T08:16:18.069 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 17.2.7-789-g4213e17c (4213e17c4de67016ed14f55d22503f4923b3a901) quincy (stable)": 6, 2024-03-29T08:16:18.069 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 8 2024-03-29T08:16:18.069 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-29T08:16:18.069 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-29T08:16:18.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:18.070+0000 7f702b800700 1 -- 172.21.15.37:0/3065705770 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f700c06cc30 msgr2=0x7f700c06f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:16:18.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:18.070+0000 7f702b800700 1 --2- 172.21.15.37:0/3065705770 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f700c06cc30 0x7f700c06f0e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f702410cfc0 tx=0x7f701000b040 comp rx=0 tx=0).stop 2024-03-29T08:16:18.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:18.070+0000 7f702b800700 1 -- 172.21.15.37:0/3065705770 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f70240708a0 msgr2=0x7f702411a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:16:18.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:18.070+0000 7f702b800700 1 --2- 172.21.15.37:0/3065705770 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f70240708a0 0x7f702411a010 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f7018004f50 tx=0x7f7018009e30 comp rx=0 tx=0).stop 2024-03-29T08:16:18.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:18.070+0000 7f702b800700 1 -- 172.21.15.37:0/3065705770 shutdown_connections 2024-03-29T08:16:18.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:18.070+0000 7f702b800700 1 --2- 172.21.15.37:0/3065705770 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f700c06cc30 0x7f700c06f0e0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:16:18.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:18.070+0000 7f702b800700 1 --2- 172.21.15.37:0/3065705770 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f70240708a0 0x7f702411a010 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:16:18.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:18.070+0000 7f702b800700 1 --2- 172.21.15.37:0/3065705770 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7024071f90 0x7f702411a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:16:18.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:18.070+0000 7f702b800700 1 -- 172.21.15.37:0/3065705770 >> 172.21.15.37:0/3065705770 conn(0x7f70240f89b0 msgr2=0x7f70240f9b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:16:18.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:18.070+0000 7f702b800700 1 -- 172.21.15.37:0/3065705770 shutdown_connections 2024-03-29T08:16:18.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:18.070+0000 7f702b800700 1 -- 172.21.15.37:0/3065705770 wait complete. 2024-03-29T08:16:18.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:18.206+0000 7f7f5b600700 1 -- 172.21.15.37:0/2510433034 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7f540fe830 msgr2=0x7f7f54106f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:16:18.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:18.206+0000 7f7f5b600700 1 --2- 172.21.15.37:0/2510433034 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7f540fe830 0x7f7f54106f00 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f7f40004940 tx=0x7f7f40009e30 comp rx=0 tx=0).stop 2024-03-29T08:16:18.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:18.206+0000 7f7f5b600700 1 -- 172.21.15.37:0/2510433034 shutdown_connections 2024-03-29T08:16:18.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:18.206+0000 7f7f5b600700 1 --2- 172.21.15.37:0/2510433034 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7f540fe830 0x7f7f54106f00 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:16:18.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:18.206+0000 7f7f5b600700 1 --2- 172.21.15.37:0/2510433034 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7f540fde90 0x7f7f540fe260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:16:18.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:18.206+0000 7f7f5b600700 1 -- 172.21.15.37:0/2510433034 >> 172.21.15.37:0/2510433034 conn(0x7f7f54072bc0 msgr2=0x7f7f54072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:16:18.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:18.207+0000 7f7f5b600700 1 -- 172.21.15.37:0/2510433034 shutdown_connections 2024-03-29T08:16:18.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:18.207+0000 7f7f5b600700 1 -- 172.21.15.37:0/2510433034 wait complete. 2024-03-29T08:16:18.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:18.207+0000 7f7f5b600700 1 Processor -- start 2024-03-29T08:16:18.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:18.207+0000 7f7f5b600700 1 -- start start 2024-03-29T08:16:18.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:18.208+0000 7f7f5b600700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7f540fde90 0x7f7f54115c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:16:18.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:18.208+0000 7f7f5b600700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7f540fe830 0x7f7f541161b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:16:18.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:18.208+0000 7f7f5b600700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f54116800 con 0x7f7f540fe830 2024-03-29T08:16:18.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:18.208+0000 7f7f5b600700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f5410fbd0 con 0x7f7f540fde90 2024-03-29T08:16:18.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:18.208+0000 7f7f53e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7f540fe830 0x7f7f541161b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:16:18.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:18.208+0000 7f7f53e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7f540fe830 0x7f7f541161b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34626/0 (socket says 172.21.15.37:34626) 2024-03-29T08:16:18.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:18.208+0000 7f7f53e00700 1 -- 172.21.15.37:0/4232674971 learned_addr learned my addr 172.21.15.37:0/4232674971 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:16:18.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:18.208+0000 7f7f58c00700 1 --2- 172.21.15.37:0/4232674971 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7f540fde90 0x7f7f54115c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:16:18.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:18.208+0000 7f7f53e00700 1 -- 172.21.15.37:0/4232674971 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7f540fde90 msgr2=0x7f7f54115c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:16:18.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:18.208+0000 7f7f53e00700 1 --2- 172.21.15.37:0/4232674971 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7f540fde90 0x7f7f54115c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:16:18.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:18.208+0000 7f7f53e00700 1 -- 172.21.15.37:0/4232674971 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f40005040 con 0x7f7f540fe830 2024-03-29T08:16:18.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:18.208+0000 7f7f58c00700 1 --2- 172.21.15.37:0/4232674971 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7f540fde90 0x7f7f54115c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:16:18.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:18.209+0000 7f7f53e00700 1 --2- 172.21.15.37:0/4232674971 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7f540fe830 0x7f7f541161b0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f7f40000c00 tx=0x7f7f40009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:16:18.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:18.209+0000 7f7f50c00700 1 -- 172.21.15.37:0/4232674971 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7f4000e040 con 0x7f7f540fe830 2024-03-29T08:16:18.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:18.209+0000 7f7f50c00700 1 -- 172.21.15.37:0/4232674971 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f7f40013420 con 0x7f7f540fe830 2024-03-29T08:16:18.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:18.209+0000 7f7f50c00700 1 -- 172.21.15.37:0/4232674971 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7f4001c630 con 0x7f7f540fe830 2024-03-29T08:16:18.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:18.209+0000 7f7f5b600700 1 -- 172.21.15.37:0/4232674971 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f5410fe50 con 0x7f7f540fe830 2024-03-29T08:16:18.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:18.209+0000 7f7f5b600700 1 -- 172.21.15.37:0/4232674971 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f54110340 con 0x7f7f540fe830 2024-03-29T08:16:18.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:18.210+0000 7f7f5b600700 1 -- 172.21.15.37:0/4232674971 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f54106a20 con 0x7f7f540fe830 2024-03-29T08:16:18.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:18.211+0000 7f7f50c00700 1 -- 172.21.15.37:0/4232674971 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f7f40013970 con 0x7f7f540fe830 2024-03-29T08:16:18.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:18.212+0000 7f7f50c00700 1 --2- 172.21.15.37:0/4232674971 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f7f3c06cc30 0x7f7f3c06f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:16:18.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:18.212+0000 7f7f50c00700 1 -- 172.21.15.37:0/4232674971 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f7f4008d260 con 0x7f7f540fe830 2024-03-29T08:16:18.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:18.212+0000 7f7f58c00700 1 --2- 172.21.15.37:0/4232674971 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f7f3c06cc30 0x7f7f3c06f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:16:18.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:18.213+0000 7f7f58c00700 1 --2- 172.21.15.37:0/4232674971 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f7f3c06cc30 0x7f7f3c06f0e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f7f48007fd0 tx=0x7f7f48009f60 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:16:18.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:18.217+0000 7f7f50c00700 1 -- 172.21.15.37:0/4232674971 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f7f4005ad30 con 0x7f7f540fe830 2024-03-29T08:16:18.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:18.476+0000 7f7f5b600700 1 -- 172.21.15.37:0/4232674971 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f7f541112c0 con 0x7f7f540fe830 2024-03-29T08:16:18.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:18.477+0000 7f7f50c00700 1 -- 172.21.15.37:0/4232674971 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 20 v20) v1 ==== 76+0+1858 (secure 0 0 0) 0x7f7f4005a8c0 con 0x7f7f540fe830 2024-03-29T08:16:18.477 INFO:teuthology.orchestra.run.smithi037.stdout:e20 2024-03-29T08:16:18.477 INFO:teuthology.orchestra.run.smithi037.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-29T08:16:18.477 INFO:teuthology.orchestra.run.smithi037.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T08:16:18.477 INFO:teuthology.orchestra.run.smithi037.stdout:legacy client fscid: 1 2024-03-29T08:16:18.478 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:16:18.478 INFO:teuthology.orchestra.run.smithi037.stdout:Filesystem 'cephfs' (1) 2024-03-29T08:16:18.478 INFO:teuthology.orchestra.run.smithi037.stdout:fs_name cephfs 2024-03-29T08:16:18.478 INFO:teuthology.orchestra.run.smithi037.stdout:epoch 20 2024-03-29T08:16:18.478 INFO:teuthology.orchestra.run.smithi037.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-03-29T08:16:18.478 INFO:teuthology.orchestra.run.smithi037.stdout:created 2024-03-29T08:03:29.241269+0000 2024-03-29T08:16:18.478 INFO:teuthology.orchestra.run.smithi037.stdout:modified 2024-03-29T08:11:27.181622+0000 2024-03-29T08:16:18.478 INFO:teuthology.orchestra.run.smithi037.stdout:tableserver 0 2024-03-29T08:16:18.478 INFO:teuthology.orchestra.run.smithi037.stdout:root 0 2024-03-29T08:16:18.478 INFO:teuthology.orchestra.run.smithi037.stdout:session_timeout 60 2024-03-29T08:16:18.478 INFO:teuthology.orchestra.run.smithi037.stdout:session_autoclose 300 2024-03-29T08:16:18.478 INFO:teuthology.orchestra.run.smithi037.stdout:max_file_size 1099511627776 2024-03-29T08:16:18.478 INFO:teuthology.orchestra.run.smithi037.stdout:max_xattr_size 65536 2024-03-29T08:16:18.478 INFO:teuthology.orchestra.run.smithi037.stdout:required_client_features {} 2024-03-29T08:16:18.479 INFO:teuthology.orchestra.run.smithi037.stdout:last_failure 0 2024-03-29T08:16:18.479 INFO:teuthology.orchestra.run.smithi037.stdout:last_failure_osd_epoch 47 2024-03-29T08:16:18.479 INFO:teuthology.orchestra.run.smithi037.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T08:16:18.479 INFO:teuthology.orchestra.run.smithi037.stdout:max_mds 2 2024-03-29T08:16:18.479 INFO:teuthology.orchestra.run.smithi037.stdout:in 0,1 2024-03-29T08:16:18.479 INFO:teuthology.orchestra.run.smithi037.stdout:up {0=14470,1=14482} 2024-03-29T08:16:18.479 INFO:teuthology.orchestra.run.smithi037.stdout:failed 2024-03-29T08:16:18.479 INFO:teuthology.orchestra.run.smithi037.stdout:damaged 2024-03-29T08:16:18.479 INFO:teuthology.orchestra.run.smithi037.stdout:stopped 2024-03-29T08:16:18.479 INFO:teuthology.orchestra.run.smithi037.stdout:data_pools [3] 2024-03-29T08:16:18.479 INFO:teuthology.orchestra.run.smithi037.stdout:metadata_pool 2 2024-03-29T08:16:18.479 INFO:teuthology.orchestra.run.smithi037.stdout:inline_data disabled 2024-03-29T08:16:18.479 INFO:teuthology.orchestra.run.smithi037.stdout:balancer 2024-03-29T08:16:18.480 INFO:teuthology.orchestra.run.smithi037.stdout:bal_rank_mask -1 2024-03-29T08:16:18.480 INFO:teuthology.orchestra.run.smithi037.stdout:standby_count_wanted 1 2024-03-29T08:16:18.480 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi037.mmwgxe{0:14470} state up:active seq 5 join_fscid=1 addr [v2:172.21.15.37:6826/1559618909,v1:172.21.15.37:6827/1559618909] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:16:18.480 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi037.iifsmh{1:14482} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.37:6828/2341530495,v1:172.21.15.37:6829/2341530495] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:16:18.480 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:16:18.480 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:16:18.480 INFO:teuthology.orchestra.run.smithi037.stdout:Standby daemons: 2024-03-29T08:16:18.480 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:16:18.480 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi148.qukpjh{-1:24319} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.148:6826/2641327912,v1:172.21.15.148:6827/2641327912] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:16:18.480 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi148.sqteih{-1:24327} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.148:6824/3252351726,v1:172.21.15.148:6825/3252351726] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:16:18.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:18.479+0000 7f7f5b600700 1 -- 172.21.15.37:0/4232674971 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f7f3c06cc30 msgr2=0x7f7f3c06f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:16:18.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:18.479+0000 7f7f5b600700 1 --2- 172.21.15.37:0/4232674971 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f7f3c06cc30 0x7f7f3c06f0e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f7f48007fd0 tx=0x7f7f48009f60 comp rx=0 tx=0).stop 2024-03-29T08:16:18.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:18.479+0000 7f7f5b600700 1 -- 172.21.15.37:0/4232674971 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7f540fe830 msgr2=0x7f7f541161b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:16:18.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:18.479+0000 7f7f5b600700 1 --2- 172.21.15.37:0/4232674971 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7f540fe830 0x7f7f541161b0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f7f40000c00 tx=0x7f7f40009e30 comp rx=0 tx=0).stop 2024-03-29T08:16:18.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:18.479+0000 7f7f5b600700 1 -- 172.21.15.37:0/4232674971 shutdown_connections 2024-03-29T08:16:18.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:18.479+0000 7f7f5b600700 1 --2- 172.21.15.37:0/4232674971 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f7f3c06cc30 0x7f7f3c06f0e0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:16:18.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:18.479+0000 7f7f5b600700 1 --2- 172.21.15.37:0/4232674971 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7f540fde90 0x7f7f54115c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:16:18.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:18.479+0000 7f7f5b600700 1 --2- 172.21.15.37:0/4232674971 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7f540fe830 0x7f7f541161b0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:16:18.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:18.479+0000 7f7f5b600700 1 -- 172.21.15.37:0/4232674971 >> 172.21.15.37:0/4232674971 conn(0x7f7f54072bc0 msgr2=0x7f7f540fb6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:16:18.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:18.479+0000 7f7f5b600700 1 -- 172.21.15.37:0/4232674971 shutdown_connections 2024-03-29T08:16:18.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:18.479+0000 7f7f5b600700 1 -- 172.21.15.37:0/4232674971 wait complete. 2024-03-29T08:16:18.482 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 20 2024-03-29T08:16:18.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:18.618+0000 7fdae4200700 1 -- 172.21.15.37:0/2339430781 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdadc102e00 msgr2=0x7fdadc1031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:16:18.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:18.618+0000 7fdae4200700 1 --2- 172.21.15.37:0/2339430781 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdadc102e00 0x7fdadc1031d0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fdad0004940 tx=0x7fdad0009e30 comp rx=0 tx=0).stop 2024-03-29T08:16:18.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:18.618+0000 7fdae4200700 1 -- 172.21.15.37:0/2339430781 shutdown_connections 2024-03-29T08:16:18.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:18.618+0000 7fdae4200700 1 --2- 172.21.15.37:0/2339430781 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdadc1037a0 0x7fdadc0fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:16:18.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:18.618+0000 7fdae4200700 1 --2- 172.21.15.37:0/2339430781 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdadc102e00 0x7fdadc1031d0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:16:18.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:18.618+0000 7fdae4200700 1 -- 172.21.15.37:0/2339430781 >> 172.21.15.37:0/2339430781 conn(0x7fdadc0f89b0 msgr2=0x7fdadc0fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:16:18.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:18.618+0000 7fdae4200700 1 -- 172.21.15.37:0/2339430781 shutdown_connections 2024-03-29T08:16:18.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:18.618+0000 7fdae4200700 1 -- 172.21.15.37:0/2339430781 wait complete. 2024-03-29T08:16:18.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:18.619+0000 7fdae4200700 1 Processor -- start 2024-03-29T08:16:18.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:18.619+0000 7fdae4200700 1 -- start start 2024-03-29T08:16:18.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:18.620+0000 7fdae4200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdadc102e00 0x7fdadc115cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:16:18.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:18.620+0000 7fdae4200700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdadc1037a0 0x7fdadc116210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:16:18.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:18.620+0000 7fdae4200700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdadc116860 con 0x7fdadc102e00 2024-03-29T08:16:18.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:18.620+0000 7fdae4200700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdadc10fc30 con 0x7fdadc1037a0 2024-03-29T08:16:18.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:18.620+0000 7fdae1800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdadc102e00 0x7fdadc115cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:16:18.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:18.620+0000 7fdae1800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdadc102e00 0x7fdadc115cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34644/0 (socket says 172.21.15.37:34644) 2024-03-29T08:16:18.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:18.620+0000 7fdae1800700 1 -- 172.21.15.37:0/3166314311 learned_addr learned my addr 172.21.15.37:0/3166314311 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:16:18.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:18.620+0000 7fdae0e00700 1 --2- 172.21.15.37:0/3166314311 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdadc1037a0 0x7fdadc116210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:16:18.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:18.620+0000 7fdae1800700 1 -- 172.21.15.37:0/3166314311 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdadc1037a0 msgr2=0x7fdadc116210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:16:18.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:18.620+0000 7fdae1800700 1 --2- 172.21.15.37:0/3166314311 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdadc1037a0 0x7fdadc116210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:16:18.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:18.620+0000 7fdae1800700 1 -- 172.21.15.37:0/3166314311 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdad0005040 con 0x7fdadc102e00 2024-03-29T08:16:18.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:18.621+0000 7fdae1800700 1 --2- 172.21.15.37:0/3166314311 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdadc102e00 0x7fdadc115cd0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fdad0009390 tx=0x7fdad0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:16:18.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:18.621+0000 7fdad9600700 1 -- 172.21.15.37:0/3166314311 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdad000e040 con 0x7fdadc102e00 2024-03-29T08:16:18.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:18.621+0000 7fdad9600700 1 -- 172.21.15.37:0/3166314311 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fdad0013420 con 0x7fdadc102e00 2024-03-29T08:16:18.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:18.621+0000 7fdae4200700 1 -- 172.21.15.37:0/3166314311 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdadc10feb0 con 0x7fdadc102e00 2024-03-29T08:16:18.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:18.621+0000 7fdad9600700 1 -- 172.21.15.37:0/3166314311 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdad001c630 con 0x7fdadc102e00 2024-03-29T08:16:18.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:18.621+0000 7fdae4200700 1 -- 172.21.15.37:0/3166314311 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdadc110470 con 0x7fdadc102e00 2024-03-29T08:16:18.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:18.622+0000 7fdae4200700 1 -- 172.21.15.37:0/3166314311 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdadc0fe550 con 0x7fdadc102e00 2024-03-29T08:16:18.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:18.623+0000 7fdad9600700 1 -- 172.21.15.37:0/3166314311 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fdad00139e0 con 0x7fdadc102e00 2024-03-29T08:16:18.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:18.624+0000 7fdad9600700 1 --2- 172.21.15.37:0/3166314311 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fdac406cc30 0x7fdac406f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:16:18.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:18.624+0000 7fdad9600700 1 -- 172.21.15.37:0/3166314311 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fdad008d370 con 0x7fdadc102e00 2024-03-29T08:16:18.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:18.624+0000 7fdae0e00700 1 --2- 172.21.15.37:0/3166314311 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fdac406cc30 0x7fdac406f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:16:18.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:18.625+0000 7fdae0e00700 1 --2- 172.21.15.37:0/3166314311 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fdac406cc30 0x7fdac406f0e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fdadc111570 tx=0x7fdac800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:16:18.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:18.628+0000 7fdad9600700 1 -- 172.21.15.37:0/3166314311 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fdad005ae40 con 0x7fdadc102e00 2024-03-29T08:16:18.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:16:18 smithi148 ceph-mon[106692]: from='client.34342 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:16:18.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:16:18 smithi148 ceph-mon[106692]: from='client.34346 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:16:18.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:16:18 smithi148 ceph-mon[106692]: pgmap v147: 65 pgs: 65 active+clean; 259 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:16:18.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:16:18 smithi148 ceph-mon[106692]: from='client.34350 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:16:18.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:16:18 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/3065705770' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:16:18.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:16:18 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/4232674971' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T08:16:18.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:18.859+0000 7fdae4200700 1 -- 172.21.15.37:0/3166314311 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fdadc04bed0 con 0x7fdac406cc30 2024-03-29T08:16:18.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:18.860+0000 7fdad9600700 1 -- 172.21.15.37:0/3166314311 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7fdadc04bed0 con 0x7fdac406cc30 2024-03-29T08:16:18.860 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-29T08:16:18.860 INFO:teuthology.orchestra.run.smithi037.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59", 2024-03-29T08:16:18.861 INFO:teuthology.orchestra.run.smithi037.stdout: "in_progress": true, 2024-03-29T08:16:18.861 INFO:teuthology.orchestra.run.smithi037.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-29T08:16:18.861 INFO:teuthology.orchestra.run.smithi037.stdout: "services_complete": [ 2024-03-29T08:16:18.861 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr", 2024-03-29T08:16:18.861 INFO:teuthology.orchestra.run.smithi037.stdout: "mon", 2024-03-29T08:16:18.861 INFO:teuthology.orchestra.run.smithi037.stdout: "crash" 2024-03-29T08:16:18.861 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-29T08:16:18.861 INFO:teuthology.orchestra.run.smithi037.stdout: "progress": "10/21 daemons upgraded", 2024-03-29T08:16:18.861 INFO:teuthology.orchestra.run.smithi037.stdout: "message": "Currently upgrading osd daemons", 2024-03-29T08:16:18.862 INFO:teuthology.orchestra.run.smithi037.stdout: "is_paused": false 2024-03-29T08:16:18.862 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-29T08:16:18.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:18.862+0000 7fdae4200700 1 -- 172.21.15.37:0/3166314311 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fdac406cc30 msgr2=0x7fdac406f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:16:18.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:18.862+0000 7fdae4200700 1 --2- 172.21.15.37:0/3166314311 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fdac406cc30 0x7fdac406f0e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fdadc111570 tx=0x7fdac800b040 comp rx=0 tx=0).stop 2024-03-29T08:16:18.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:18.862+0000 7fdae4200700 1 -- 172.21.15.37:0/3166314311 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdadc102e00 msgr2=0x7fdadc115cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:16:18.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:18.862+0000 7fdae4200700 1 --2- 172.21.15.37:0/3166314311 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdadc102e00 0x7fdadc115cd0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fdad0009390 tx=0x7fdad0009e30 comp rx=0 tx=0).stop 2024-03-29T08:16:18.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:18.862+0000 7fdae4200700 1 -- 172.21.15.37:0/3166314311 shutdown_connections 2024-03-29T08:16:18.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:18.862+0000 7fdae4200700 1 --2- 172.21.15.37:0/3166314311 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fdac406cc30 0x7fdac406f0e0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:16:18.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:18.862+0000 7fdae4200700 1 --2- 172.21.15.37:0/3166314311 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdadc102e00 0x7fdadc115cd0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:16:18.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:18.862+0000 7fdae4200700 1 --2- 172.21.15.37:0/3166314311 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdadc1037a0 0x7fdadc116210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:16:18.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:18.862+0000 7fdae4200700 1 -- 172.21.15.37:0/3166314311 >> 172.21.15.37:0/3166314311 conn(0x7fdadc0f89b0 msgr2=0x7fdadc0f99e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:16:18.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:18.863+0000 7fdae4200700 1 -- 172.21.15.37:0/3166314311 shutdown_connections 2024-03-29T08:16:18.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:18.863+0000 7fdae4200700 1 -- 172.21.15.37:0/3166314311 wait complete. 2024-03-29T08:16:18.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:16:18 smithi037 ceph-mon[142325]: from='client.34342 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:16:18.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:16:18 smithi037 ceph-mon[142325]: from='client.34346 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:16:18.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:16:18 smithi037 ceph-mon[142325]: pgmap v147: 65 pgs: 65 active+clean; 259 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:16:18.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:16:18 smithi037 ceph-mon[142325]: from='client.34350 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:16:18.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:16:18 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/3065705770' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:16:18.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:16:18 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/4232674971' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T08:16:19.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:19.002+0000 7f120aa00700 1 -- 172.21.15.37:0/349399705 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f12041037a0 msgr2=0x7f12040fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:16:19.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:19.002+0000 7f120aa00700 1 --2- 172.21.15.37:0/349399705 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f12041037a0 0x7f12040fce10 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f11f0004940 tx=0x7f11f0009e30 comp rx=0 tx=0).stop 2024-03-29T08:16:19.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:19.002+0000 7f120aa00700 1 -- 172.21.15.37:0/349399705 shutdown_connections 2024-03-29T08:16:19.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:19.002+0000 7f120aa00700 1 --2- 172.21.15.37:0/349399705 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f12041037a0 0x7f12040fce10 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:16:19.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:19.002+0000 7f120aa00700 1 --2- 172.21.15.37:0/349399705 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1204102e00 0x7f12041031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:16:19.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:19.002+0000 7f120aa00700 1 -- 172.21.15.37:0/349399705 >> 172.21.15.37:0/349399705 conn(0x7f12040f89b0 msgr2=0x7f12040fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:16:19.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:19.002+0000 7f120aa00700 1 -- 172.21.15.37:0/349399705 shutdown_connections 2024-03-29T08:16:19.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:19.002+0000 7f120aa00700 1 -- 172.21.15.37:0/349399705 wait complete. 2024-03-29T08:16:19.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:19.003+0000 7f120aa00700 1 Processor -- start 2024-03-29T08:16:19.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:19.003+0000 7f120aa00700 1 -- start start 2024-03-29T08:16:19.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:19.003+0000 7f120aa00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1204102e00 0x7f12040702c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:16:19.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:19.003+0000 7f120aa00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f12041037a0 0x7f1204070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:16:19.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:19.003+0000 7f120aa00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1204070d90 con 0x7f12041037a0 2024-03-29T08:16:19.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:19.003+0000 7f120aa00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1204070ed0 con 0x7f1204102e00 2024-03-29T08:16:19.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:19.004+0000 7f1203400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f12041037a0 0x7f1204070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:16:19.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:19.004+0000 7f1203400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f12041037a0 0x7f1204070800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34662/0 (socket says 172.21.15.37:34662) 2024-03-29T08:16:19.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:19.004+0000 7f1203400700 1 -- 172.21.15.37:0/2491433044 learned_addr learned my addr 172.21.15.37:0/2491433044 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:16:19.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:19.004+0000 7f1203e00700 1 --2- 172.21.15.37:0/2491433044 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1204102e00 0x7f12040702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:16:19.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:19.004+0000 7f1203400700 1 -- 172.21.15.37:0/2491433044 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1204102e00 msgr2=0x7f12040702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:16:19.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:19.004+0000 7f1203400700 1 --2- 172.21.15.37:0/2491433044 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1204102e00 0x7f12040702c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:16:19.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:19.004+0000 7f1203400700 1 -- 172.21.15.37:0/2491433044 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11f0005040 con 0x7f12041037a0 2024-03-29T08:16:19.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:19.004+0000 7f1203400700 1 --2- 172.21.15.37:0/2491433044 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f12041037a0 0x7f1204070800 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f11f0009390 tx=0x7f11f0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:16:19.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:19.005+0000 7f11fbe00700 1 -- 172.21.15.37:0/2491433044 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f11f000e040 con 0x7f12041037a0 2024-03-29T08:16:19.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:19.005+0000 7f11fbe00700 1 -- 172.21.15.37:0/2491433044 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f11f0013420 con 0x7f12041037a0 2024-03-29T08:16:19.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:19.005+0000 7f11fbe00700 1 -- 172.21.15.37:0/2491433044 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f11f001c630 con 0x7f12041037a0 2024-03-29T08:16:19.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:19.005+0000 7f120aa00700 1 -- 172.21.15.37:0/2491433044 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1204071120 con 0x7f12041037a0 2024-03-29T08:16:19.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:19.005+0000 7f120aa00700 1 -- 172.21.15.37:0/2491433044 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f120411a230 con 0x7f12041037a0 2024-03-29T08:16:19.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:19.006+0000 7f120aa00700 1 -- 172.21.15.37:0/2491433044 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12040fe550 con 0x7f12041037a0 2024-03-29T08:16:19.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:19.007+0000 7f11fbe00700 1 -- 172.21.15.37:0/2491433044 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f11f00139e0 con 0x7f12041037a0 2024-03-29T08:16:19.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:19.008+0000 7f11fbe00700 1 --2- 172.21.15.37:0/2491433044 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f11e806cc30 0x7f11e806f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:16:19.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:19.008+0000 7f11fbe00700 1 -- 172.21.15.37:0/2491433044 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f11f008d230 con 0x7f12041037a0 2024-03-29T08:16:19.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:19.008+0000 7f1203e00700 1 --2- 172.21.15.37:0/2491433044 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f11e806cc30 0x7f11e806f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:16:19.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:19.009+0000 7f1203e00700 1 --2- 172.21.15.37:0/2491433044 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f11e806cc30 0x7f11e806f0e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f11f4004990 tx=0x7f11f4009ee0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:16:19.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:19.012+0000 7f11fbe00700 1 -- 172.21.15.37:0/2491433044 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f11f00572f0 con 0x7f12041037a0 2024-03-29T08:16:19.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:19.312+0000 7f120aa00700 1 -- 172.21.15.37:0/2491433044 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f120405f5f0 con 0x7f12041037a0 2024-03-29T08:16:19.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:19.313+0000 7f11fbe00700 1 -- 172.21.15.37:0/2491433044 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f11f005a910 con 0x7f12041037a0 2024-03-29T08:16:19.313 INFO:teuthology.orchestra.run.smithi037.stdout:HEALTH_OK 2024-03-29T08:16:19.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:19.315+0000 7f120aa00700 1 -- 172.21.15.37:0/2491433044 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f11e806cc30 msgr2=0x7f11e806f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:16:19.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:19.315+0000 7f120aa00700 1 --2- 172.21.15.37:0/2491433044 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f11e806cc30 0x7f11e806f0e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f11f4004990 tx=0x7f11f4009ee0 comp rx=0 tx=0).stop 2024-03-29T08:16:19.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:19.315+0000 7f120aa00700 1 -- 172.21.15.37:0/2491433044 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f12041037a0 msgr2=0x7f1204070800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:16:19.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:19.315+0000 7f120aa00700 1 --2- 172.21.15.37:0/2491433044 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f12041037a0 0x7f1204070800 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f11f0009390 tx=0x7f11f0009e30 comp rx=0 tx=0).stop 2024-03-29T08:16:19.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:19.315+0000 7f120aa00700 1 -- 172.21.15.37:0/2491433044 shutdown_connections 2024-03-29T08:16:19.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:19.315+0000 7f120aa00700 1 --2- 172.21.15.37:0/2491433044 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f11e806cc30 0x7f11e806f0e0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:16:19.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:19.315+0000 7f120aa00700 1 --2- 172.21.15.37:0/2491433044 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1204102e00 0x7f12040702c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:16:19.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:19.315+0000 7f120aa00700 1 --2- 172.21.15.37:0/2491433044 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f12041037a0 0x7f1204070800 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:16:19.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:19.315+0000 7f120aa00700 1 -- 172.21.15.37:0/2491433044 >> 172.21.15.37:0/2491433044 conn(0x7f12040f89b0 msgr2=0x7f12040f9ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:16:19.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:19.315+0000 7f120aa00700 1 -- 172.21.15.37:0/2491433044 shutdown_connections 2024-03-29T08:16:19.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:19.315+0000 7f120aa00700 1 -- 172.21.15.37:0/2491433044 wait complete. 2024-03-29T08:16:19.806 INFO:journalctl@ceph.osd.4.smithi148.stdout:Mar 29 08:16:19 smithi148 systemd[1]: Stopping Ceph osd.4 for 99b492c2-eda1-11ee-b647-cb9ed24678a4... 2024-03-29T08:16:20.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:16:20 smithi037 ceph-mon[142325]: from='client.34362 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:16:20.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:16:20 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/2491433044' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T08:16:20.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:16:20 smithi037 ceph-mon[142325]: pgmap v148: 65 pgs: 65 active+clean; 259 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:16:20.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:16:20 smithi037 ceph-mon[142325]: osd.4 marked itself down and dead 2024-03-29T08:16:20.556 INFO:journalctl@ceph.osd.4.smithi148.stdout:Mar 29 08:16:20 smithi148 ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4-osd-4[59397]: 2024-03-29T08:16:20.087+0000 7f287dc00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-29T08:16:20.556 INFO:journalctl@ceph.osd.4.smithi148.stdout:Mar 29 08:16:20 smithi148 ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4-osd-4[59397]: 2024-03-29T08:16:20.087+0000 7f287dc00700 -1 osd.4 70 *** Got signal Terminated *** 2024-03-29T08:16:20.556 INFO:journalctl@ceph.osd.4.smithi148.stdout:Mar 29 08:16:20 smithi148 ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4-osd-4[59397]: 2024-03-29T08:16:20.087+0000 7f287dc00700 -1 osd.4 70 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-29T08:16:20.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:16:20 smithi148 ceph-mon[106692]: from='client.34362 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:16:20.557 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:16:20 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/2491433044' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T08:16:20.557 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:16:20 smithi148 ceph-mon[106692]: pgmap v148: 65 pgs: 65 active+clean; 259 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:16:20.557 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:16:20 smithi148 ceph-mon[106692]: osd.4 marked itself down and dead 2024-03-29T08:16:21.306 INFO:journalctl@ceph.osd.4.smithi148.stdout:Mar 29 08:16:21 smithi148 bash[124370]: ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4-osd-4 2024-03-29T08:16:21.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:16:21 smithi148 ceph-mon[106692]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-29T08:16:21.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:16:21 smithi148 ceph-mon[106692]: osdmap e71: 6 total, 5 up, 6 in 2024-03-29T08:16:21.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:16:21 smithi037 ceph-mon[142325]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-29T08:16:21.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:16:21 smithi037 ceph-mon[142325]: osdmap e71: 6 total, 5 up, 6 in 2024-03-29T08:16:22.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:16:22 smithi148 ceph-mon[106692]: osdmap e72: 6 total, 5 up, 6 in 2024-03-29T08:16:22.307 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:16:22 smithi148 ceph-mon[106692]: pgmap v151: 65 pgs: 16 peering, 4 stale+active+clean, 45 active+clean; 259 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:16:22.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:16:22 smithi037 ceph-mon[142325]: osdmap e72: 6 total, 5 up, 6 in 2024-03-29T08:16:22.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:16:22 smithi037 ceph-mon[142325]: pgmap v151: 65 pgs: 16 peering, 4 stale+active+clean, 45 active+clean; 259 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:16:23.422 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:16:23 smithi148 ceph-mon[106692]: Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY) 2024-03-29T08:16:23.423 INFO:journalctl@ceph.osd.4.smithi148.stdout:Mar 29 08:16:23 smithi148 systemd[1]: ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4@osd.4.service: Succeeded. 2024-03-29T08:16:23.423 INFO:journalctl@ceph.osd.4.smithi148.stdout:Mar 29 08:16:23 smithi148 systemd[1]: Stopped Ceph osd.4 for 99b492c2-eda1-11ee-b647-cb9ed24678a4. 2024-03-29T08:16:23.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:16:23 smithi037 ceph-mon[142325]: Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY) 2024-03-29T08:16:23.806 INFO:journalctl@ceph.osd.4.smithi148.stdout:Mar 29 08:16:23 smithi148 systemd[1]: Starting Ceph osd.4 for 99b492c2-eda1-11ee-b647-cb9ed24678a4... 2024-03-29T08:16:24.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:16:24 smithi148 ceph-mon[106692]: pgmap v152: 65 pgs: 16 peering, 4 stale+active+clean, 45 active+clean; 259 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:16:24.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:16:24 smithi037 ceph-mon[142325]: pgmap v152: 65 pgs: 16 peering, 4 stale+active+clean, 45 active+clean; 259 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:16:25.056 INFO:journalctl@ceph.osd.4.smithi148.stdout:Mar 29 08:16:24 smithi148 bash[124643]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-03-29T08:16:25.056 INFO:journalctl@ceph.osd.4.smithi148.stdout:Mar 29 08:16:24 smithi148 bash[124643]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-4 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-03-29T08:16:25.056 INFO:journalctl@ceph.osd.4.smithi148.stdout:Mar 29 08:16:24 smithi148 bash[124643]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-03-29T08:16:25.056 INFO:journalctl@ceph.osd.4.smithi148.stdout:Mar 29 08:16:24 smithi148 bash[124643]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-03-29T08:16:25.057 INFO:journalctl@ceph.osd.4.smithi148.stdout:Mar 29 08:16:24 smithi148 bash[124643]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-4/block 2024-03-29T08:16:25.057 INFO:journalctl@ceph.osd.4.smithi148.stdout:Mar 29 08:16:24 smithi148 bash[124643]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-03-29T08:16:25.057 INFO:journalctl@ceph.osd.4.smithi148.stdout:Mar 29 08:16:24 smithi148 bash[124643]: --> ceph-volume raw activate successful for osd ID: 4 2024-03-29T08:16:25.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:16:25 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:16:25.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:16:25 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:16:26.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:16:25 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:16:26.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:16:25 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:16:26.556 INFO:journalctl@ceph.osd.4.smithi148.stdout:Mar 29 08:16:26 smithi148 ceph-osd[124952]: -- 172.21.15.148:0/3119694504 <== mon.1 v2:172.21.15.148:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5562e9ea71e0 con 0x5562e9e76c00 2024-03-29T08:16:26.556 INFO:journalctl@ceph.osd.4.smithi148.stdout:Mar 29 08:16:26 smithi148 bash[124914]: 966a9de11d224b11c25ea0e67532b0ddadccb5627924748ad3057c029480c32e 2024-03-29T08:16:26.556 INFO:journalctl@ceph.osd.4.smithi148.stdout:Mar 29 08:16:26 smithi148 systemd[1]: Started Ceph osd.4 for 99b492c2-eda1-11ee-b647-cb9ed24678a4. 2024-03-29T08:16:26.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:16:26 smithi037 ceph-mon[142325]: pgmap v153: 65 pgs: 10 active+undersized+degraded, 11 peering, 13 active+undersized, 31 active+clean; 259 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 29/324 objects degraded (8.951%) 2024-03-29T08:16:26.928 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:16:26 smithi037 ceph-mon[142325]: Health check failed: Degraded data redundancy: 29/324 objects degraded (8.951%), 10 pgs degraded (PG_DEGRADED) 2024-03-29T08:16:26.928 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:16:26 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:16:26.929 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:16:26 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:16:26.929 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:16:26 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T08:16:27.056 INFO:journalctl@ceph.osd.4.smithi148.stdout:Mar 29 08:16:26 smithi148 ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4-osd-4[124929]: 2024-03-29T08:16:26.764+0000 7f281afd2640 -1 Falling back to public interface 2024-03-29T08:16:27.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:16:26 smithi148 ceph-mon[106692]: pgmap v153: 65 pgs: 10 active+undersized+degraded, 11 peering, 13 active+undersized, 31 active+clean; 259 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 29/324 objects degraded (8.951%) 2024-03-29T08:16:27.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:16:26 smithi148 ceph-mon[106692]: Health check failed: Degraded data redundancy: 29/324 objects degraded (8.951%), 10 pgs degraded (PG_DEGRADED) 2024-03-29T08:16:27.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:16:26 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:16:27.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:16:26 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:16:27.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:16:26 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T08:16:27.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:16:27 smithi148 ceph-mon[106692]: pgmap v154: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 259 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 53/324 objects degraded (16.358%) 2024-03-29T08:16:27.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:16:27 smithi148 ceph-mon[106692]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-03-29T08:16:27.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:16:27 smithi037 ceph-mon[142325]: pgmap v154: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 259 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 53/324 objects degraded (16.358%) 2024-03-29T08:16:27.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:16:27 smithi037 ceph-mon[142325]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-03-29T08:16:30.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:16:30 smithi037 ceph-mon[142325]: pgmap v155: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 259 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 53/324 objects degraded (16.358%) 2024-03-29T08:16:30.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:16:30 smithi148 ceph-mon[106692]: pgmap v155: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 259 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 53/324 objects degraded (16.358%) 2024-03-29T08:16:31.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:16:31 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:16:31.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:16:31 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:16:31.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:16:31 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:16:31.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:16:31 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:16:32.441 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:16:32 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:16:32.442 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:16:32 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:16:32.442 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:16:32 smithi148 ceph-mon[106692]: pgmap v156: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 259 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 53/324 objects degraded (16.358%) 2024-03-29T08:16:32.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:16:32 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:16:32.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:16:32 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:16:32.678 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:16:32 smithi037 ceph-mon[142325]: pgmap v156: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 259 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 53/324 objects degraded (16.358%) 2024-03-29T08:16:32.806 INFO:journalctl@ceph.osd.4.smithi148.stdout:Mar 29 08:16:32 smithi148 ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4-osd-4[124929]: 2024-03-29T08:16:32.484+0000 7f281afd2640 -1 osd.4 70 log_to_monitors true 2024-03-29T08:16:33.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:16:33 smithi037 ceph-mon[142325]: from='osd.4 [v2:172.21.15.148:6808/572347061,v1:172.21.15.148:6809/572347061]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-29T08:16:33.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:16:33 smithi037 ceph-mon[142325]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-29T08:16:33.558 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:16:33 smithi148 ceph-mon[106692]: from='osd.4 [v2:172.21.15.148:6808/572347061,v1:172.21.15.148:6809/572347061]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-29T08:16:33.558 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:16:33 smithi148 ceph-mon[106692]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-29T08:16:34.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:16:34 smithi037 ceph-mon[142325]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-29T08:16:34.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:16:34 smithi037 ceph-mon[142325]: osdmap e73: 6 total, 5 up, 6 in 2024-03-29T08:16:34.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:16:34 smithi037 ceph-mon[142325]: from='osd.4 [v2:172.21.15.148:6808/572347061,v1:172.21.15.148:6809/572347061]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-03-29T08:16:34.678 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:16:34 smithi037 ceph-mon[142325]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-03-29T08:16:34.678 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:16:34 smithi037 ceph-mon[142325]: pgmap v158: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 259 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 53/324 objects degraded (16.358%) 2024-03-29T08:16:34.678 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:16:34 smithi037 ceph-mon[142325]: Health check update: Degraded data redundancy: 53/324 objects degraded (16.358%), 16 pgs degraded (PG_DEGRADED) 2024-03-29T08:16:34.678 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:16:34 smithi037 ceph-mon[142325]: from='osd.4 ' entity='osd.4' 2024-03-29T08:16:34.806 INFO:journalctl@ceph.osd.4.smithi148.stdout:Mar 29 08:16:34 smithi148 ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4-osd-4[124929]: 2024-03-29T08:16:34.323+0000 7f2805400700 -1 osd.4 70 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-29T08:16:34.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:16:34 smithi148 ceph-mon[106692]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-29T08:16:34.807 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:16:34 smithi148 ceph-mon[106692]: osdmap e73: 6 total, 5 up, 6 in 2024-03-29T08:16:34.807 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:16:34 smithi148 ceph-mon[106692]: from='osd.4 [v2:172.21.15.148:6808/572347061,v1:172.21.15.148:6809/572347061]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-03-29T08:16:34.807 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:16:34 smithi148 ceph-mon[106692]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-03-29T08:16:34.807 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:16:34 smithi148 ceph-mon[106692]: pgmap v158: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 259 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 53/324 objects degraded (16.358%) 2024-03-29T08:16:34.807 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:16:34 smithi148 ceph-mon[106692]: Health check update: Degraded data redundancy: 53/324 objects degraded (16.358%), 16 pgs degraded (PG_DEGRADED) 2024-03-29T08:16:34.807 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:16:34 smithi148 ceph-mon[106692]: from='osd.4 ' entity='osd.4' 2024-03-29T08:16:35.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:16:35 smithi037 ceph-mon[142325]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-29T08:16:35.678 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:16:35 smithi037 ceph-mon[142325]: osd.4 [v2:172.21.15.148:6808/572347061,v1:172.21.15.148:6809/572347061] boot 2024-03-29T08:16:35.678 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:16:35 smithi037 ceph-mon[142325]: osdmap e74: 6 total, 6 up, 6 in 2024-03-29T08:16:35.678 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:16:35 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-29T08:16:35.717 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:16:35 smithi148 ceph-mon[106692]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-29T08:16:35.717 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:16:35 smithi148 ceph-mon[106692]: osd.4 [v2:172.21.15.148:6808/572347061,v1:172.21.15.148:6809/572347061] boot 2024-03-29T08:16:35.717 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:16:35 smithi148 ceph-mon[106692]: osdmap e74: 6 total, 6 up, 6 in 2024-03-29T08:16:35.717 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:16:35 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-29T08:16:36.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:16:36 smithi037 ceph-mon[142325]: pgmap v160: 65 pgs: 4 peering, 15 active+undersized+degraded, 15 active+undersized, 31 active+clean; 259 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 50/324 objects degraded (15.432%) 2024-03-29T08:16:36.678 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:16:36 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:16:36.678 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:16:36 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:16:36.678 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:16:36 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T08:16:36.678 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:16:36 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T08:16:36.678 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:16:36 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:16:36.678 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:16:36 smithi037 ceph-mon[142325]: osdmap e75: 6 total, 6 up, 6 in 2024-03-29T08:16:36.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:16:36 smithi148 ceph-mon[106692]: pgmap v160: 65 pgs: 4 peering, 15 active+undersized+degraded, 15 active+undersized, 31 active+clean; 259 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 50/324 objects degraded (15.432%) 2024-03-29T08:16:36.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:16:36 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:16:36.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:16:36 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:16:36.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:16:36 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T08:16:36.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:16:36 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T08:16:36.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:16:36 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:16:36.807 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:16:36 smithi148 ceph-mon[106692]: osdmap e75: 6 total, 6 up, 6 in 2024-03-29T08:16:37.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:16:37 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T08:16:37.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:16:37 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:16:37.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:16:37 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:16:37.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:16:37 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:16:37.678 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:16:37 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-03-29T08:16:37.678 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:16:37 smithi037 ceph-mon[142325]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-03-29T08:16:37.678 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:16:37 smithi037 ceph-mon[142325]: Upgrade: unsafe to stop osd(s) at this time (11 PGs are or would become offline) 2024-03-29T08:16:37.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:16:37 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T08:16:37.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:16:37 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:16:37.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:16:37 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:16:37.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:16:37 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:16:37.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:16:37 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-03-29T08:16:37.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:16:37 smithi148 ceph-mon[106692]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-03-29T08:16:37.807 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:16:37 smithi148 ceph-mon[106692]: Upgrade: unsafe to stop osd(s) at this time (11 PGs are or would become offline) 2024-03-29T08:16:38.678 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:16:38 smithi037 ceph-mon[142325]: pgmap v162: 65 pgs: 14 peering, 10 active+undersized+degraded, 10 active+undersized, 31 active+clean; 259 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 28/324 objects degraded (8.642%) 2024-03-29T08:16:38.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:16:38 smithi148 ceph-mon[106692]: pgmap v162: 65 pgs: 14 peering, 10 active+undersized+degraded, 10 active+undersized, 31 active+clean; 259 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 28/324 objects degraded (8.642%) 2024-03-29T08:16:39.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:16:39 smithi037 ceph-mon[142325]: Health check update: Degraded data redundancy: 28/324 objects degraded (8.642%), 10 pgs degraded (PG_DEGRADED) 2024-03-29T08:16:39.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:16:39 smithi148 ceph-mon[106692]: Health check update: Degraded data redundancy: 28/324 objects degraded (8.642%), 10 pgs degraded (PG_DEGRADED) 2024-03-29T08:16:40.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:16:40 smithi037 ceph-mon[142325]: pgmap v163: 65 pgs: 14 peering, 51 active+clean; 259 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:16:40.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:16:40 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:16:40.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:16:40 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:16:40.928 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:16:40 smithi037 ceph-mon[142325]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 28/324 objects degraded (8.642%), 10 pgs degraded) 2024-03-29T08:16:40.928 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:16:40 smithi037 ceph-mon[142325]: Cluster is now healthy 2024-03-29T08:16:41.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:16:40 smithi148 ceph-mon[106692]: pgmap v163: 65 pgs: 14 peering, 51 active+clean; 259 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:16:41.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:16:40 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:16:41.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:16:40 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:16:41.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:16:40 smithi148 ceph-mon[106692]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 28/324 objects degraded (8.642%), 10 pgs degraded) 2024-03-29T08:16:41.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:16:40 smithi148 ceph-mon[106692]: Cluster is now healthy 2024-03-29T08:16:42.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:16:42 smithi037 ceph-mon[142325]: pgmap v164: 65 pgs: 65 active+clean; 259 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:16:43.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:16:42 smithi148 ceph-mon[106692]: pgmap v164: 65 pgs: 65 active+clean; 259 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:16:44.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:16:44 smithi037 ceph-mon[142325]: pgmap v165: 65 pgs: 65 active+clean; 259 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:16:45.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:16:44 smithi148 ceph-mon[106692]: pgmap v165: 65 pgs: 65 active+clean; 259 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:16:46.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:16:46 smithi037 ceph-mon[142325]: pgmap v166: 65 pgs: 65 active+clean; 259 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:16:47.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:16:46 smithi148 ceph-mon[106692]: pgmap v166: 65 pgs: 65 active+clean; 259 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:16:48.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:16:48 smithi037 ceph-mon[142325]: pgmap v167: 65 pgs: 65 active+clean; 259 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:16:49.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:16:48 smithi148 ceph-mon[106692]: pgmap v167: 65 pgs: 65 active+clean; 259 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:16:49.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:49.469+0000 7f5a9f200700 1 -- 172.21.15.37:0/1933838859 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5a981037a0 msgr2=0x7f5a980fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:16:49.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:49.469+0000 7f5a9f200700 1 --2- 172.21.15.37:0/1933838859 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5a981037a0 0x7f5a980fce10 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f5a84004940 tx=0x7f5a84009e30 comp rx=0 tx=0).stop 2024-03-29T08:16:49.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:49.470+0000 7f5a9f200700 1 -- 172.21.15.37:0/1933838859 shutdown_connections 2024-03-29T08:16:49.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:49.470+0000 7f5a9f200700 1 --2- 172.21.15.37:0/1933838859 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5a981037a0 0x7f5a980fce10 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:16:49.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:49.470+0000 7f5a9f200700 1 --2- 172.21.15.37:0/1933838859 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5a98102e00 0x7f5a981031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:16:49.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:49.470+0000 7f5a9f200700 1 -- 172.21.15.37:0/1933838859 >> 172.21.15.37:0/1933838859 conn(0x7f5a980f89b0 msgr2=0x7f5a980fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:16:49.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:49.470+0000 7f5a9f200700 1 -- 172.21.15.37:0/1933838859 shutdown_connections 2024-03-29T08:16:49.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:49.470+0000 7f5a9f200700 1 -- 172.21.15.37:0/1933838859 wait complete. 2024-03-29T08:16:49.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:49.470+0000 7f5a9f200700 1 Processor -- start 2024-03-29T08:16:49.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:49.471+0000 7f5a9f200700 1 -- start start 2024-03-29T08:16:49.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:49.471+0000 7f5a9f200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5a98102e00 0x7f5a98115da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:16:49.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:49.471+0000 7f5a9f200700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5a981037a0 0x7f5a981162e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:16:49.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:49.471+0000 7f5a9f200700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a98116930 con 0x7f5a98102e00 2024-03-29T08:16:49.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:49.471+0000 7f5a9f200700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a9810fd00 con 0x7f5a981037a0 2024-03-29T08:16:49.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:49.471+0000 7f5a97e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5a98102e00 0x7f5a98115da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:16:49.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:49.471+0000 7f5a97e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5a98102e00 0x7f5a98115da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50790/0 (socket says 172.21.15.37:50790) 2024-03-29T08:16:49.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:49.471+0000 7f5a97e00700 1 -- 172.21.15.37:0/953905729 learned_addr learned my addr 172.21.15.37:0/953905729 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:16:49.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:49.472+0000 7f5a97400700 1 --2- 172.21.15.37:0/953905729 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5a981037a0 0x7f5a981162e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:16:49.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:49.472+0000 7f5a97e00700 1 -- 172.21.15.37:0/953905729 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5a981037a0 msgr2=0x7f5a981162e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:16:49.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:49.472+0000 7f5a97e00700 1 --2- 172.21.15.37:0/953905729 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5a981037a0 0x7f5a981162e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:16:49.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:49.472+0000 7f5a97e00700 1 -- 172.21.15.37:0/953905729 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a84005040 con 0x7f5a98102e00 2024-03-29T08:16:49.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:49.472+0000 7f5a97e00700 1 --2- 172.21.15.37:0/953905729 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5a98102e00 0x7f5a98115da0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f5a88002a50 tx=0x7f5a880046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:16:49.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:49.472+0000 7f5a8fe00700 1 -- 172.21.15.37:0/953905729 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5a88007cc0 con 0x7f5a98102e00 2024-03-29T08:16:49.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:49.472+0000 7f5a8fe00700 1 -- 172.21.15.37:0/953905729 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f5a8800f040 con 0x7f5a98102e00 2024-03-29T08:16:49.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:49.472+0000 7f5a9f200700 1 -- 172.21.15.37:0/953905729 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a9810ffe0 con 0x7f5a98102e00 2024-03-29T08:16:49.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:49.472+0000 7f5a8fe00700 1 -- 172.21.15.37:0/953905729 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5a8800d9c0 con 0x7f5a98102e00 2024-03-29T08:16:49.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:49.473+0000 7f5a9f200700 1 -- 172.21.15.37:0/953905729 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a98110520 con 0x7f5a98102e00 2024-03-29T08:16:49.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:49.474+0000 7f5a9f200700 1 -- 172.21.15.37:0/953905729 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a980fe550 con 0x7f5a98102e00 2024-03-29T08:16:49.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:49.475+0000 7f5a8fe00700 1 -- 172.21.15.37:0/953905729 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f5a8800db20 con 0x7f5a98102e00 2024-03-29T08:16:49.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:49.475+0000 7f5a8fe00700 1 --2- 172.21.15.37:0/953905729 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f5a7c06cc30 0x7f5a7c06f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:16:49.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:49.476+0000 7f5a8fe00700 1 -- 172.21.15.37:0/953905729 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f5a8808bca0 con 0x7f5a98102e00 2024-03-29T08:16:49.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:49.476+0000 7f5a97400700 1 --2- 172.21.15.37:0/953905729 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f5a7c06cc30 0x7f5a7c06f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:16:49.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:49.476+0000 7f5a97400700 1 --2- 172.21.15.37:0/953905729 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f5a7c06cc30 0x7f5a7c06f0e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f5a98111640 tx=0x7f5a84009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:16:49.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:49.481+0000 7f5a8fe00700 1 -- 172.21.15.37:0/953905729 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f5a88059770 con 0x7f5a98102e00 2024-03-29T08:16:49.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:49.716+0000 7f5a9f200700 1 -- 172.21.15.37:0/953905729 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5a9804bf40 con 0x7f5a7c06cc30 2024-03-29T08:16:49.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:49.717+0000 7f5a8fe00700 1 -- 172.21.15.37:0/953905729 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f5a9804bf40 con 0x7f5a7c06cc30 2024-03-29T08:16:49.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:49.719+0000 7f5a9f200700 1 -- 172.21.15.37:0/953905729 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f5a7c06cc30 msgr2=0x7f5a7c06f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:16:49.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:49.719+0000 7f5a9f200700 1 --2- 172.21.15.37:0/953905729 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f5a7c06cc30 0x7f5a7c06f0e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f5a98111640 tx=0x7f5a84009e30 comp rx=0 tx=0).stop 2024-03-29T08:16:49.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:49.719+0000 7f5a9f200700 1 -- 172.21.15.37:0/953905729 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5a98102e00 msgr2=0x7f5a98115da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:16:49.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:49.719+0000 7f5a9f200700 1 --2- 172.21.15.37:0/953905729 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5a98102e00 0x7f5a98115da0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f5a88002a50 tx=0x7f5a880046b0 comp rx=0 tx=0).stop 2024-03-29T08:16:49.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:49.719+0000 7f5a9f200700 1 -- 172.21.15.37:0/953905729 shutdown_connections 2024-03-29T08:16:49.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:49.719+0000 7f5a9f200700 1 --2- 172.21.15.37:0/953905729 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f5a7c06cc30 0x7f5a7c06f0e0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:16:49.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:49.719+0000 7f5a9f200700 1 --2- 172.21.15.37:0/953905729 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5a98102e00 0x7f5a98115da0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:16:49.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:49.719+0000 7f5a9f200700 1 --2- 172.21.15.37:0/953905729 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5a981037a0 0x7f5a981162e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:16:49.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:49.719+0000 7f5a9f200700 1 -- 172.21.15.37:0/953905729 >> 172.21.15.37:0/953905729 conn(0x7f5a980f89b0 msgr2=0x7f5a980f9a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:16:49.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:49.719+0000 7f5a9f200700 1 -- 172.21.15.37:0/953905729 shutdown_connections 2024-03-29T08:16:49.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:49.719+0000 7f5a9f200700 1 -- 172.21.15.37:0/953905729 wait complete. 2024-03-29T08:16:49.734 INFO:teuthology.orchestra.run.smithi037.stdout:true 2024-03-29T08:16:49.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:49.857+0000 7f5c0d000700 1 -- 172.21.15.37:0/4250449509 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5c08102ca0 msgr2=0x7f5c08103070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:16:49.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:49.857+0000 7f5c0d000700 1 --2- 172.21.15.37:0/4250449509 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5c08102ca0 0x7f5c08103070 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f5bf8004940 tx=0x7f5bf8009e30 comp rx=0 tx=0).stop 2024-03-29T08:16:49.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:49.857+0000 7f5c0d000700 1 -- 172.21.15.37:0/4250449509 shutdown_connections 2024-03-29T08:16:49.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:49.857+0000 7f5c0d000700 1 --2- 172.21.15.37:0/4250449509 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5c08065630 0x7f5c08065a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:16:49.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:49.857+0000 7f5c0d000700 1 --2- 172.21.15.37:0/4250449509 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5c08102ca0 0x7f5c08103070 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:16:49.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:49.857+0000 7f5c0d000700 1 -- 172.21.15.37:0/4250449509 >> 172.21.15.37:0/4250449509 conn(0x7f5c08071b70 msgr2=0x7f5c08071f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:16:49.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:49.857+0000 7f5c0d000700 1 -- 172.21.15.37:0/4250449509 shutdown_connections 2024-03-29T08:16:49.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:49.857+0000 7f5c0d000700 1 -- 172.21.15.37:0/4250449509 wait complete. 2024-03-29T08:16:49.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:49.858+0000 7f5c0d000700 1 Processor -- start 2024-03-29T08:16:49.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:49.858+0000 7f5c0d000700 1 -- start start 2024-03-29T08:16:49.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:49.858+0000 7f5c0d000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5c08065630 0x7f5c08115a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:16:49.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:49.859+0000 7f5c0d000700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5c08102ca0 0x7f5c08115fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:16:49.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:49.859+0000 7f5c0d000700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c081166a0 con 0x7f5c08065630 2024-03-29T08:16:49.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:49.859+0000 7f5c0d000700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c0810fa00 con 0x7f5c08102ca0 2024-03-29T08:16:49.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:49.859+0000 7f5c07400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5c08065630 0x7f5c08115a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:16:49.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:49.859+0000 7f5c07400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5c08065630 0x7f5c08115a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50802/0 (socket says 172.21.15.37:50802) 2024-03-29T08:16:49.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:49.859+0000 7f5c07400700 1 -- 172.21.15.37:0/3027967490 learned_addr learned my addr 172.21.15.37:0/3027967490 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:16:49.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:49.859+0000 7f5c06a00700 1 --2- 172.21.15.37:0/3027967490 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5c08102ca0 0x7f5c08115fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:16:49.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:49.859+0000 7f5c07400700 1 -- 172.21.15.37:0/3027967490 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5c08102ca0 msgr2=0x7f5c08115fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:16:49.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:49.859+0000 7f5c07400700 1 --2- 172.21.15.37:0/3027967490 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5c08102ca0 0x7f5c08115fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:16:49.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:49.859+0000 7f5c07400700 1 -- 172.21.15.37:0/3027967490 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bf8005040 con 0x7f5c08065630 2024-03-29T08:16:49.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:49.859+0000 7f5c06a00700 1 --2- 172.21.15.37:0/3027967490 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5c08102ca0 0x7f5c08115fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:16:49.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:49.860+0000 7f5c07400700 1 --2- 172.21.15.37:0/3027967490 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5c08065630 0x7f5c08115a80 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f5bf8009180 tx=0x7f5bf8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:16:49.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:49.860+0000 7f5bff400700 1 -- 172.21.15.37:0/3027967490 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5bf800e040 con 0x7f5c08065630 2024-03-29T08:16:49.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:49.860+0000 7f5bff400700 1 -- 172.21.15.37:0/3027967490 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f5bf8004290 con 0x7f5c08065630 2024-03-29T08:16:49.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:49.860+0000 7f5bff400700 1 -- 172.21.15.37:0/3027967490 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5bf801c3f0 con 0x7f5c08065630 2024-03-29T08:16:49.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:49.860+0000 7f5c0d000700 1 -- 172.21.15.37:0/3027967490 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c0810fc80 con 0x7f5c08065630 2024-03-29T08:16:49.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:49.860+0000 7f5c0d000700 1 -- 172.21.15.37:0/3027967490 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c08110240 con 0x7f5c08065630 2024-03-29T08:16:49.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:49.861+0000 7f5c0d000700 1 -- 172.21.15.37:0/3027967490 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c08106130 con 0x7f5c08065630 2024-03-29T08:16:49.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:49.862+0000 7f5bff400700 1 -- 172.21.15.37:0/3027967490 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f5bf8013790 con 0x7f5c08065630 2024-03-29T08:16:49.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:49.863+0000 7f5bff400700 1 --2- 172.21.15.37:0/3027967490 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f5bec06cc20 0x7f5bec06f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:16:49.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:49.863+0000 7f5bff400700 1 -- 172.21.15.37:0/3027967490 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f5bf808d230 con 0x7f5c08065630 2024-03-29T08:16:49.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:49.863+0000 7f5c06a00700 1 --2- 172.21.15.37:0/3027967490 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f5bec06cc20 0x7f5bec06f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:16:49.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:49.864+0000 7f5c06a00700 1 --2- 172.21.15.37:0/3027967490 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f5bec06cc20 0x7f5bec06f0d0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f5c08110bc0 tx=0x7f5bf4008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:16:49.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:49.868+0000 7f5bff400700 1 -- 172.21.15.37:0/3027967490 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f5bf805ad80 con 0x7f5c08065630 2024-03-29T08:16:50.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:50.101+0000 7f5c0d000700 1 -- 172.21.15.37:0/3027967490 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5c08109c20 con 0x7f5bec06cc20 2024-03-29T08:16:50.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:50.102+0000 7f5bff400700 1 -- 172.21.15.37:0/3027967490 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f5c08109c20 con 0x7f5bec06cc20 2024-03-29T08:16:50.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:50.104+0000 7f5c0d000700 1 -- 172.21.15.37:0/3027967490 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f5bec06cc20 msgr2=0x7f5bec06f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:16:50.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:50.104+0000 7f5c0d000700 1 --2- 172.21.15.37:0/3027967490 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f5bec06cc20 0x7f5bec06f0d0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f5c08110bc0 tx=0x7f5bf4008040 comp rx=0 tx=0).stop 2024-03-29T08:16:50.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:50.104+0000 7f5c0d000700 1 -- 172.21.15.37:0/3027967490 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5c08065630 msgr2=0x7f5c08115a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:16:50.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:50.104+0000 7f5c0d000700 1 --2- 172.21.15.37:0/3027967490 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5c08065630 0x7f5c08115a80 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f5bf8009180 tx=0x7f5bf8009e30 comp rx=0 tx=0).stop 2024-03-29T08:16:50.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:50.104+0000 7f5c0d000700 1 -- 172.21.15.37:0/3027967490 shutdown_connections 2024-03-29T08:16:50.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:50.104+0000 7f5c0d000700 1 --2- 172.21.15.37:0/3027967490 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f5bec06cc20 0x7f5bec06f0d0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:16:50.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:50.104+0000 7f5c0d000700 1 --2- 172.21.15.37:0/3027967490 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5c08065630 0x7f5c08115a80 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:16:50.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:50.104+0000 7f5c0d000700 1 --2- 172.21.15.37:0/3027967490 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5c08102ca0 0x7f5c08115fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:16:50.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:50.104+0000 7f5c0d000700 1 -- 172.21.15.37:0/3027967490 >> 172.21.15.37:0/3027967490 conn(0x7f5c08071b70 msgr2=0x7f5c08072f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:16:50.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:50.105+0000 7f5c0d000700 1 -- 172.21.15.37:0/3027967490 shutdown_connections 2024-03-29T08:16:50.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:50.105+0000 7f5c0d000700 1 -- 172.21.15.37:0/3027967490 wait complete. 2024-03-29T08:16:50.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:50.243+0000 7f130d600700 1 -- 172.21.15.37:0/1812092961 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f13081037a0 msgr2=0x7f13080fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:16:50.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:50.243+0000 7f130d600700 1 --2- 172.21.15.37:0/1812092961 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f13081037a0 0x7f13080fce10 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f12f4004940 tx=0x7f12f4009e30 comp rx=0 tx=0).stop 2024-03-29T08:16:50.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:50.243+0000 7f130d600700 1 -- 172.21.15.37:0/1812092961 shutdown_connections 2024-03-29T08:16:50.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:50.243+0000 7f130d600700 1 --2- 172.21.15.37:0/1812092961 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f13081037a0 0x7f13080fce10 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:16:50.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:50.243+0000 7f130d600700 1 --2- 172.21.15.37:0/1812092961 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1308102e00 0x7f13081031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:16:50.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:50.243+0000 7f130d600700 1 -- 172.21.15.37:0/1812092961 >> 172.21.15.37:0/1812092961 conn(0x7f13080f89b0 msgr2=0x7f13080fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:16:50.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:50.243+0000 7f130d600700 1 -- 172.21.15.37:0/1812092961 shutdown_connections 2024-03-29T08:16:50.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:50.243+0000 7f130d600700 1 -- 172.21.15.37:0/1812092961 wait complete. 2024-03-29T08:16:50.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:50.244+0000 7f130d600700 1 Processor -- start 2024-03-29T08:16:50.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:50.244+0000 7f130d600700 1 -- start start 2024-03-29T08:16:50.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:50.244+0000 7f130d600700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1308102e00 0x7f13080702c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:16:50.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:50.245+0000 7f130d600700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f13081037a0 0x7f1308070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:16:50.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:50.245+0000 7f130d600700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1308070d90 con 0x7f1308102e00 2024-03-29T08:16:50.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:50.245+0000 7f130d600700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1308070ed0 con 0x7f13081037a0 2024-03-29T08:16:50.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:50.245+0000 7f1306a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1308102e00 0x7f13080702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:16:50.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:50.245+0000 7f1306a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1308102e00 0x7f13080702c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52580/0 (socket says 172.21.15.37:52580) 2024-03-29T08:16:50.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:50.245+0000 7f1306a00700 1 -- 172.21.15.37:0/2908808539 learned_addr learned my addr 172.21.15.37:0/2908808539 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:16:50.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:50.245+0000 7f1306000700 1 --2- 172.21.15.37:0/2908808539 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f13081037a0 0x7f1308070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:16:50.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:50.245+0000 7f1306a00700 1 -- 172.21.15.37:0/2908808539 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f13081037a0 msgr2=0x7f1308070800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:16:50.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:50.245+0000 7f1306a00700 1 --2- 172.21.15.37:0/2908808539 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f13081037a0 0x7f1308070800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:16:50.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:50.245+0000 7f1306a00700 1 -- 172.21.15.37:0/2908808539 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12f4005040 con 0x7f1308102e00 2024-03-29T08:16:50.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:50.246+0000 7f1306a00700 1 --2- 172.21.15.37:0/2908808539 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1308102e00 0x7f13080702c0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f12f8002a50 tx=0x7f12f80046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:16:50.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:50.246+0000 7f12fea00700 1 -- 172.21.15.37:0/2908808539 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f12f8007cc0 con 0x7f1308102e00 2024-03-29T08:16:50.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:50.246+0000 7f130d600700 1 -- 172.21.15.37:0/2908808539 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1308119e00 con 0x7f1308102e00 2024-03-29T08:16:50.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:50.246+0000 7f12fea00700 1 -- 172.21.15.37:0/2908808539 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f12f800f040 con 0x7f1308102e00 2024-03-29T08:16:50.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:50.246+0000 7f12fea00700 1 -- 172.21.15.37:0/2908808539 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f12f800d9c0 con 0x7f1308102e00 2024-03-29T08:16:50.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:50.246+0000 7f130d600700 1 -- 172.21.15.37:0/2908808539 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f130811a280 con 0x7f1308102e00 2024-03-29T08:16:50.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:50.247+0000 7f130d600700 1 -- 172.21.15.37:0/2908808539 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13080fe550 con 0x7f1308102e00 2024-03-29T08:16:50.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:50.248+0000 7f12fea00700 1 -- 172.21.15.37:0/2908808539 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f12f800db20 con 0x7f1308102e00 2024-03-29T08:16:50.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:50.249+0000 7f12fea00700 1 --2- 172.21.15.37:0/2908808539 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f12ec06cc30 0x7f12ec06f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:16:50.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:50.249+0000 7f12fea00700 1 -- 172.21.15.37:0/2908808539 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f12f808bda0 con 0x7f1308102e00 2024-03-29T08:16:50.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:50.249+0000 7f1306000700 1 --2- 172.21.15.37:0/2908808539 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f12ec06cc30 0x7f12ec06f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:16:50.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:50.250+0000 7f1306000700 1 --2- 172.21.15.37:0/2908808539 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f12ec06cc30 0x7f12ec06f0e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f1308071c00 tx=0x7f12f4009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:16:50.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:50.254+0000 7f12fea00700 1 -- 172.21.15.37:0/2908808539 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f12f8055d60 con 0x7f1308102e00 2024-03-29T08:16:50.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:16:50 smithi037 ceph-mon[142325]: pgmap v168: 65 pgs: 65 active+clean; 259 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:16:50.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:16:50 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:16:50.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:50.480+0000 7f130d600700 1 -- 172.21.15.37:0/2908808539 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f13080719b0 con 0x7f12ec06cc30 2024-03-29T08:16:50.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:50.491+0000 7f12fea00700 1 -- 172.21.15.37:0/2908808539 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7f13080719b0 con 0x7f12ec06cc30 2024-03-29T08:16:50.491 INFO:teuthology.orchestra.run.smithi037.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-29T08:16:50.491 INFO:teuthology.orchestra.run.smithi037.stdout:alertmanager.smithi037 smithi037 *:9093,9094 running (6m) 96s ago 19m 14.1M - 0.25.0 c8568f914cd2 2f9f9746ed0f 2024-03-29T08:16:50.491 INFO:teuthology.orchestra.run.smithi037.stdout:crash.smithi037 smithi037 running (3m) 96s ago 19m 6647k - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 178d416a3186 2024-03-29T08:16:50.492 INFO:teuthology.orchestra.run.smithi037.stdout:crash.smithi148 smithi148 running (3m) 20s ago 18m 6656k - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 23666989d58f 2024-03-29T08:16:50.492 INFO:teuthology.orchestra.run.smithi037.stdout:grafana.smithi037 smithi037 *:3000 running (6m) 96s ago 19m 73.4M - 9.4.7 954c08fa6188 6b976fe49952 2024-03-29T08:16:50.492 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi037.iifsmh smithi037 running (13m) 96s ago 13m 110M - 17.2.7-789-g4213e17c cf22eb8683be 074e1acf331f 2024-03-29T08:16:50.492 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi037.mmwgxe smithi037 running (13m) 96s ago 13m 114M - 17.2.7-789-g4213e17c cf22eb8683be 9a5b1e72de5c 2024-03-29T08:16:50.492 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi148.qukpjh smithi148 running (13m) 20s ago 13m 14.9M - 17.2.7-789-g4213e17c cf22eb8683be 8e469782a273 2024-03-29T08:16:50.492 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi148.sqteih smithi148 running (13m) 20s ago 13m 14.8M - 17.2.7-789-g4213e17c cf22eb8683be 8e68f858bb22 2024-03-29T08:16:50.492 INFO:teuthology.orchestra.run.smithi037.stdout:mgr.smithi037.wjikii smithi037 *:8443,9283,8765 running (9m) 96s ago 21m 501M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff c5a1417b9417 2024-03-29T08:16:50.492 INFO:teuthology.orchestra.run.smithi037.stdout:mgr.smithi148.onhgvm smithi148 *:8443,9283,8765 running (7m) 20s ago 18m 449M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff fcfdbce062bd 2024-03-29T08:16:50.492 INFO:teuthology.orchestra.run.smithi037.stdout:mon.smithi037 smithi037 running (4m) 96s ago 21m 47.5M 2048M 18.2.2-1015-gd0e392d9 1cc2f45c17ff eea2060783af 2024-03-29T08:16:50.493 INFO:teuthology.orchestra.run.smithi037.stdout:mon.smithi148 smithi148 running (4m) 20s ago 18m 39.0M 2048M 18.2.2-1015-gd0e392d9 1cc2f45c17ff ad4b2c5748be 2024-03-29T08:16:50.493 INFO:teuthology.orchestra.run.smithi037.stdout:node-exporter.smithi037 smithi037 *:9100 running (7m) 96s ago 19m 7847k - 1.5.0 0da6a335fe13 26b8ae9cb878 2024-03-29T08:16:50.493 INFO:teuthology.orchestra.run.smithi037.stdout:node-exporter.smithi148 smithi148 *:9100 running (7m) 20s ago 18m 7914k - 1.5.0 0da6a335fe13 98414e4e64ed 2024-03-29T08:16:50.493 INFO:teuthology.orchestra.run.smithi037.stdout:osd.0 smithi037 running (3m) 96s ago 17m 99.0M 1619M 18.2.2-1015-gd0e392d9 1cc2f45c17ff c0c584f6245c 2024-03-29T08:16:50.493 INFO:teuthology.orchestra.run.smithi037.stdout:osd.1 smithi037 running (2m) 96s ago 16m 97.2M 1619M 18.2.2-1015-gd0e392d9 1cc2f45c17ff a4a2ff4cc83d 2024-03-29T08:16:50.493 INFO:teuthology.orchestra.run.smithi037.stdout:osd.2 smithi037 running (103s) 96s ago 16m 11.0M 1619M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 8b1bc71aa79e 2024-03-29T08:16:50.493 INFO:teuthology.orchestra.run.smithi037.stdout:osd.3 smithi148 running (62s) 20s ago 15m 103M 2642M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 1c656ab8ff86 2024-03-29T08:16:50.493 INFO:teuthology.orchestra.run.smithi037.stdout:osd.4 smithi148 running (24s) 20s ago 15m 10.8M 2642M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 966a9de11d22 2024-03-29T08:16:50.493 INFO:teuthology.orchestra.run.smithi037.stdout:osd.5 smithi148 running (14m) 20s ago 14m 232M 2642M 17.2.7-789-g4213e17c cf22eb8683be 044a3f7db7f8 2024-03-29T08:16:50.493 INFO:teuthology.orchestra.run.smithi037.stdout:prometheus.smithi037 smithi037 *:9095 running (7m) 96s ago 19m 52.1M - 2.43.0 a07b618ecd1d 30c0223ec589 2024-03-29T08:16:50.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:50.493+0000 7f130d600700 1 -- 172.21.15.37:0/2908808539 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f12ec06cc30 msgr2=0x7f12ec06f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:16:50.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:50.493+0000 7f130d600700 1 --2- 172.21.15.37:0/2908808539 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f12ec06cc30 0x7f12ec06f0e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f1308071c00 tx=0x7f12f4009e30 comp rx=0 tx=0).stop 2024-03-29T08:16:50.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:50.493+0000 7f130d600700 1 -- 172.21.15.37:0/2908808539 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1308102e00 msgr2=0x7f13080702c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:16:50.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:50.493+0000 7f130d600700 1 --2- 172.21.15.37:0/2908808539 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1308102e00 0x7f13080702c0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f12f8002a50 tx=0x7f12f80046b0 comp rx=0 tx=0).stop 2024-03-29T08:16:50.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:50.493+0000 7f130d600700 1 -- 172.21.15.37:0/2908808539 shutdown_connections 2024-03-29T08:16:50.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:50.493+0000 7f130d600700 1 --2- 172.21.15.37:0/2908808539 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f12ec06cc30 0x7f12ec06f0e0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:16:50.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:50.493+0000 7f130d600700 1 --2- 172.21.15.37:0/2908808539 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1308102e00 0x7f13080702c0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:16:50.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:50.493+0000 7f130d600700 1 --2- 172.21.15.37:0/2908808539 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f13081037a0 0x7f1308070800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:16:50.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:50.493+0000 7f130d600700 1 -- 172.21.15.37:0/2908808539 >> 172.21.15.37:0/2908808539 conn(0x7f13080f89b0 msgr2=0x7f13080f9ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:16:50.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:50.493+0000 7f130d600700 1 -- 172.21.15.37:0/2908808539 shutdown_connections 2024-03-29T08:16:50.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:50.493+0000 7f130d600700 1 -- 172.21.15.37:0/2908808539 wait complete. 2024-03-29T08:16:50.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:16:50 smithi148 ceph-mon[106692]: pgmap v168: 65 pgs: 65 active+clean; 259 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:16:50.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:16:50 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:16:50.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:50.629+0000 7fd880200700 1 -- 172.21.15.37:0/3181953045 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd878102e00 msgr2=0x7fd8781031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:16:50.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:50.629+0000 7fd880200700 1 --2- 172.21.15.37:0/3181953045 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd878102e00 0x7fd8781031d0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fd86c004940 tx=0x7fd86c009e30 comp rx=0 tx=0).stop 2024-03-29T08:16:50.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:50.630+0000 7fd880200700 1 -- 172.21.15.37:0/3181953045 shutdown_connections 2024-03-29T08:16:50.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:50.630+0000 7fd880200700 1 --2- 172.21.15.37:0/3181953045 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd8781037a0 0x7fd8780fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:16:50.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:50.630+0000 7fd880200700 1 --2- 172.21.15.37:0/3181953045 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd878102e00 0x7fd8781031d0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:16:50.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:50.630+0000 7fd880200700 1 -- 172.21.15.37:0/3181953045 >> 172.21.15.37:0/3181953045 conn(0x7fd8780f89b0 msgr2=0x7fd8780fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:16:50.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:50.630+0000 7fd880200700 1 -- 172.21.15.37:0/3181953045 shutdown_connections 2024-03-29T08:16:50.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:50.630+0000 7fd880200700 1 -- 172.21.15.37:0/3181953045 wait complete. 2024-03-29T08:16:50.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:50.630+0000 7fd880200700 1 Processor -- start 2024-03-29T08:16:50.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:50.631+0000 7fd880200700 1 -- start start 2024-03-29T08:16:50.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:50.631+0000 7fd880200700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd878102e00 0x7fd878113b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:16:50.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:50.631+0000 7fd880200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd8781037a0 0x7fd878114060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:16:50.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:50.631+0000 7fd880200700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8781146b0 con 0x7fd8781037a0 2024-03-29T08:16:50.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:50.631+0000 7fd880200700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd87810daa0 con 0x7fd878102e00 2024-03-29T08:16:50.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:50.631+0000 7fd87ce00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd8781037a0 0x7fd878114060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:16:50.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:50.631+0000 7fd87ce00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd8781037a0 0x7fd878114060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52602/0 (socket says 172.21.15.37:52602) 2024-03-29T08:16:50.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:50.631+0000 7fd87ce00700 1 -- 172.21.15.37:0/3097986172 learned_addr learned my addr 172.21.15.37:0/3097986172 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:16:50.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:50.631+0000 7fd87d800700 1 --2- 172.21.15.37:0/3097986172 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd878102e00 0x7fd878113b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:16:50.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:50.632+0000 7fd87ce00700 1 -- 172.21.15.37:0/3097986172 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd878102e00 msgr2=0x7fd878113b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:16:50.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:50.632+0000 7fd87ce00700 1 --2- 172.21.15.37:0/3097986172 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd878102e00 0x7fd878113b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:16:50.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:50.632+0000 7fd87ce00700 1 -- 172.21.15.37:0/3097986172 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd86c005040 con 0x7fd8781037a0 2024-03-29T08:16:50.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:50.632+0000 7fd87ce00700 1 --2- 172.21.15.37:0/3097986172 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd8781037a0 0x7fd878114060 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fd864002a50 tx=0x7fd864004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:16:50.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:50.632+0000 7fd875600700 1 -- 172.21.15.37:0/3097986172 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd864007cc0 con 0x7fd8781037a0 2024-03-29T08:16:50.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:50.632+0000 7fd880200700 1 -- 172.21.15.37:0/3097986172 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd87810dd80 con 0x7fd8781037a0 2024-03-29T08:16:50.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:50.632+0000 7fd875600700 1 -- 172.21.15.37:0/3097986172 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd86400f040 con 0x7fd8781037a0 2024-03-29T08:16:50.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:50.632+0000 7fd875600700 1 -- 172.21.15.37:0/3097986172 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd86400d920 con 0x7fd8781037a0 2024-03-29T08:16:50.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:50.632+0000 7fd880200700 1 -- 172.21.15.37:0/3097986172 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd87810e340 con 0x7fd8781037a0 2024-03-29T08:16:50.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:50.633+0000 7fd880200700 1 -- 172.21.15.37:0/3097986172 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8780fe550 con 0x7fd8781037a0 2024-03-29T08:16:50.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:50.634+0000 7fd875600700 1 -- 172.21.15.37:0/3097986172 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fd864009680 con 0x7fd8781037a0 2024-03-29T08:16:50.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:50.636+0000 7fd875600700 1 --2- 172.21.15.37:0/3097986172 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fd86006cc30 0x7fd86006f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:16:50.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:50.636+0000 7fd875600700 1 -- 172.21.15.37:0/3097986172 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fd86408bdb0 con 0x7fd8781037a0 2024-03-29T08:16:50.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:50.636+0000 7fd87d800700 1 --2- 172.21.15.37:0/3097986172 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fd86006cc30 0x7fd86006f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:16:50.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:50.636+0000 7fd87d800700 1 --2- 172.21.15.37:0/3097986172 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fd86006cc30 0x7fd86006f0e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fd86c00b840 tx=0x7fd86c009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:16:50.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:50.640+0000 7fd875600700 1 -- 172.21.15.37:0/3097986172 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fd864059800 con 0x7fd8781037a0 2024-03-29T08:16:50.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:50.939+0000 7fd880200700 1 -- 172.21.15.37:0/3097986172 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fd878109dc0 con 0x7fd8781037a0 2024-03-29T08:16:50.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:50.940+0000 7fd875600700 1 -- 172.21.15.37:0/3097986172 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+847 (secure 0 0 0) 0x7fd864059390 con 0x7fd8781037a0 2024-03-29T08:16:50.940 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-29T08:16:50.940 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": { 2024-03-29T08:16:50.940 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 2 2024-03-29T08:16:50.940 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:16:50.940 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": { 2024-03-29T08:16:50.941 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 2 2024-03-29T08:16:50.941 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:16:50.941 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": { 2024-03-29T08:16:50.941 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 17.2.7-789-g4213e17c (4213e17c4de67016ed14f55d22503f4923b3a901) quincy (stable)": 1, 2024-03-29T08:16:50.941 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 5 2024-03-29T08:16:50.941 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:16:50.941 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": { 2024-03-29T08:16:50.941 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 17.2.7-789-g4213e17c (4213e17c4de67016ed14f55d22503f4923b3a901) quincy (stable)": 4 2024-03-29T08:16:50.941 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:16:50.941 INFO:teuthology.orchestra.run.smithi037.stdout: "overall": { 2024-03-29T08:16:50.941 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 17.2.7-789-g4213e17c (4213e17c4de67016ed14f55d22503f4923b3a901) quincy (stable)": 5, 2024-03-29T08:16:50.941 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 9 2024-03-29T08:16:50.941 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-29T08:16:50.941 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-29T08:16:50.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:50.942+0000 7fd880200700 1 -- 172.21.15.37:0/3097986172 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fd86006cc30 msgr2=0x7fd86006f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:16:50.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:50.942+0000 7fd880200700 1 --2- 172.21.15.37:0/3097986172 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fd86006cc30 0x7fd86006f0e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fd86c00b840 tx=0x7fd86c009e30 comp rx=0 tx=0).stop 2024-03-29T08:16:50.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:50.942+0000 7fd880200700 1 -- 172.21.15.37:0/3097986172 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd8781037a0 msgr2=0x7fd878114060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:16:50.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:50.942+0000 7fd880200700 1 --2- 172.21.15.37:0/3097986172 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd8781037a0 0x7fd878114060 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fd864002a50 tx=0x7fd864004600 comp rx=0 tx=0).stop 2024-03-29T08:16:50.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:50.942+0000 7fd880200700 1 -- 172.21.15.37:0/3097986172 shutdown_connections 2024-03-29T08:16:50.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:50.942+0000 7fd880200700 1 --2- 172.21.15.37:0/3097986172 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fd86006cc30 0x7fd86006f0e0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:16:50.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:50.942+0000 7fd880200700 1 --2- 172.21.15.37:0/3097986172 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd878102e00 0x7fd878113b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:16:50.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:50.942+0000 7fd880200700 1 --2- 172.21.15.37:0/3097986172 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd8781037a0 0x7fd878114060 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:16:50.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:50.942+0000 7fd880200700 1 -- 172.21.15.37:0/3097986172 >> 172.21.15.37:0/3097986172 conn(0x7fd8780f89b0 msgr2=0x7fd8780f9a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:16:50.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:50.942+0000 7fd880200700 1 -- 172.21.15.37:0/3097986172 shutdown_connections 2024-03-29T08:16:50.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:50.942+0000 7fd880200700 1 -- 172.21.15.37:0/3097986172 wait complete. 2024-03-29T08:16:51.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:51.078+0000 7f8c1b000700 1 -- 172.21.15.37:0/2600495424 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8c14102e00 msgr2=0x7f8c141031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:16:51.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:51.078+0000 7f8c1b000700 1 --2- 172.21.15.37:0/2600495424 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8c14102e00 0x7f8c141031d0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f8c04004940 tx=0x7f8c04009e30 comp rx=0 tx=0).stop 2024-03-29T08:16:51.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:51.079+0000 7f8c1b000700 1 -- 172.21.15.37:0/2600495424 shutdown_connections 2024-03-29T08:16:51.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:51.079+0000 7f8c1b000700 1 --2- 172.21.15.37:0/2600495424 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8c141037a0 0x7f8c140fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:16:51.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:51.079+0000 7f8c1b000700 1 --2- 172.21.15.37:0/2600495424 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8c14102e00 0x7f8c141031d0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:16:51.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:51.079+0000 7f8c1b000700 1 -- 172.21.15.37:0/2600495424 >> 172.21.15.37:0/2600495424 conn(0x7f8c140f89b0 msgr2=0x7f8c140fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:16:51.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:51.079+0000 7f8c1b000700 1 -- 172.21.15.37:0/2600495424 shutdown_connections 2024-03-29T08:16:51.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:51.079+0000 7f8c1b000700 1 -- 172.21.15.37:0/2600495424 wait complete. 2024-03-29T08:16:51.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:51.080+0000 7f8c1b000700 1 Processor -- start 2024-03-29T08:16:51.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:51.080+0000 7f8c1b000700 1 -- start start 2024-03-29T08:16:51.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:51.081+0000 7f8c1b000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8c14102e00 0x7f8c14115d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:16:51.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:51.081+0000 7f8c1b000700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8c141037a0 0x7f8c14116250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:16:51.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:51.081+0000 7f8c1b000700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c141168a0 con 0x7f8c14102e00 2024-03-29T08:16:51.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:51.081+0000 7f8c1b000700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c1410fc70 con 0x7f8c141037a0 2024-03-29T08:16:51.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:51.081+0000 7f8c13e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8c14102e00 0x7f8c14115d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:16:51.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:51.081+0000 7f8c13e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8c14102e00 0x7f8c14115d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52618/0 (socket says 172.21.15.37:52618) 2024-03-29T08:16:51.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:51.081+0000 7f8c13e00700 1 -- 172.21.15.37:0/4200390190 learned_addr learned my addr 172.21.15.37:0/4200390190 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:16:51.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:51.081+0000 7f8c13400700 1 --2- 172.21.15.37:0/4200390190 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8c141037a0 0x7f8c14116250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:16:51.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:51.081+0000 7f8c13e00700 1 -- 172.21.15.37:0/4200390190 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8c141037a0 msgr2=0x7f8c14116250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:16:51.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:51.081+0000 7f8c13e00700 1 --2- 172.21.15.37:0/4200390190 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8c141037a0 0x7f8c14116250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:16:51.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:51.081+0000 7f8c13e00700 1 -- 172.21.15.37:0/4200390190 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c04005040 con 0x7f8c14102e00 2024-03-29T08:16:51.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:51.081+0000 7f8c13400700 1 --2- 172.21.15.37:0/4200390190 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8c141037a0 0x7f8c14116250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:16:51.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:51.082+0000 7f8c13e00700 1 --2- 172.21.15.37:0/4200390190 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8c14102e00 0x7f8c14115d10 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f8c04009390 tx=0x7f8c04009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:16:51.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:51.082+0000 7f8c0be00700 1 -- 172.21.15.37:0/4200390190 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8c0400e040 con 0x7f8c14102e00 2024-03-29T08:16:51.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:51.082+0000 7f8c0be00700 1 -- 172.21.15.37:0/4200390190 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f8c04013420 con 0x7f8c14102e00 2024-03-29T08:16:51.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:51.082+0000 7f8c1b000700 1 -- 172.21.15.37:0/4200390190 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c1410fef0 con 0x7f8c14102e00 2024-03-29T08:16:51.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:51.082+0000 7f8c0be00700 1 -- 172.21.15.37:0/4200390190 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8c0401c630 con 0x7f8c14102e00 2024-03-29T08:16:51.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:51.082+0000 7f8c1b000700 1 -- 172.21.15.37:0/4200390190 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c141104b0 con 0x7f8c14102e00 2024-03-29T08:16:51.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:51.083+0000 7f8c1b000700 1 -- 172.21.15.37:0/4200390190 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c140fe550 con 0x7f8c14102e00 2024-03-29T08:16:51.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:51.084+0000 7f8c0be00700 1 -- 172.21.15.37:0/4200390190 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f8c040139e0 con 0x7f8c14102e00 2024-03-29T08:16:51.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:51.085+0000 7f8c0be00700 1 --2- 172.21.15.37:0/4200390190 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f8bf806cc30 0x7f8bf806f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:16:51.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:51.085+0000 7f8c0be00700 1 -- 172.21.15.37:0/4200390190 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f8c0408d110 con 0x7f8c14102e00 2024-03-29T08:16:51.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:51.085+0000 7f8c13400700 1 --2- 172.21.15.37:0/4200390190 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f8bf806cc30 0x7f8bf806f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:16:51.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:51.086+0000 7f8c13400700 1 --2- 172.21.15.37:0/4200390190 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f8bf806cc30 0x7f8bf806f0e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f8c00007fd0 tx=0x7f8c00009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:16:51.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:51.089+0000 7f8c0be00700 1 -- 172.21.15.37:0/4200390190 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f8c0405add0 con 0x7f8c14102e00 2024-03-29T08:16:51.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:51.348+0000 7f8c1b000700 1 -- 172.21.15.37:0/4200390190 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f8c1404bf60 con 0x7f8c14102e00 2024-03-29T08:16:51.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:51.348+0000 7f8c0be00700 1 -- 172.21.15.37:0/4200390190 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 20 v20) v1 ==== 76+0+1858 (secure 0 0 0) 0x7f8c0405a960 con 0x7f8c14102e00 2024-03-29T08:16:51.348 INFO:teuthology.orchestra.run.smithi037.stdout:e20 2024-03-29T08:16:51.349 INFO:teuthology.orchestra.run.smithi037.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-29T08:16:51.349 INFO:teuthology.orchestra.run.smithi037.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T08:16:51.349 INFO:teuthology.orchestra.run.smithi037.stdout:legacy client fscid: 1 2024-03-29T08:16:51.349 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:16:51.349 INFO:teuthology.orchestra.run.smithi037.stdout:Filesystem 'cephfs' (1) 2024-03-29T08:16:51.349 INFO:teuthology.orchestra.run.smithi037.stdout:fs_name cephfs 2024-03-29T08:16:51.349 INFO:teuthology.orchestra.run.smithi037.stdout:epoch 20 2024-03-29T08:16:51.349 INFO:teuthology.orchestra.run.smithi037.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-03-29T08:16:51.349 INFO:teuthology.orchestra.run.smithi037.stdout:created 2024-03-29T08:03:29.241269+0000 2024-03-29T08:16:51.349 INFO:teuthology.orchestra.run.smithi037.stdout:modified 2024-03-29T08:11:27.181622+0000 2024-03-29T08:16:51.350 INFO:teuthology.orchestra.run.smithi037.stdout:tableserver 0 2024-03-29T08:16:51.350 INFO:teuthology.orchestra.run.smithi037.stdout:root 0 2024-03-29T08:16:51.350 INFO:teuthology.orchestra.run.smithi037.stdout:session_timeout 60 2024-03-29T08:16:51.350 INFO:teuthology.orchestra.run.smithi037.stdout:session_autoclose 300 2024-03-29T08:16:51.350 INFO:teuthology.orchestra.run.smithi037.stdout:max_file_size 1099511627776 2024-03-29T08:16:51.350 INFO:teuthology.orchestra.run.smithi037.stdout:max_xattr_size 65536 2024-03-29T08:16:51.350 INFO:teuthology.orchestra.run.smithi037.stdout:required_client_features {} 2024-03-29T08:16:51.350 INFO:teuthology.orchestra.run.smithi037.stdout:last_failure 0 2024-03-29T08:16:51.350 INFO:teuthology.orchestra.run.smithi037.stdout:last_failure_osd_epoch 47 2024-03-29T08:16:51.350 INFO:teuthology.orchestra.run.smithi037.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T08:16:51.351 INFO:teuthology.orchestra.run.smithi037.stdout:max_mds 2 2024-03-29T08:16:51.351 INFO:teuthology.orchestra.run.smithi037.stdout:in 0,1 2024-03-29T08:16:51.351 INFO:teuthology.orchestra.run.smithi037.stdout:up {0=14470,1=14482} 2024-03-29T08:16:51.351 INFO:teuthology.orchestra.run.smithi037.stdout:failed 2024-03-29T08:16:51.351 INFO:teuthology.orchestra.run.smithi037.stdout:damaged 2024-03-29T08:16:51.351 INFO:teuthology.orchestra.run.smithi037.stdout:stopped 2024-03-29T08:16:51.351 INFO:teuthology.orchestra.run.smithi037.stdout:data_pools [3] 2024-03-29T08:16:51.351 INFO:teuthology.orchestra.run.smithi037.stdout:metadata_pool 2 2024-03-29T08:16:51.351 INFO:teuthology.orchestra.run.smithi037.stdout:inline_data disabled 2024-03-29T08:16:51.351 INFO:teuthology.orchestra.run.smithi037.stdout:balancer 2024-03-29T08:16:51.351 INFO:teuthology.orchestra.run.smithi037.stdout:bal_rank_mask -1 2024-03-29T08:16:51.351 INFO:teuthology.orchestra.run.smithi037.stdout:standby_count_wanted 1 2024-03-29T08:16:51.351 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi037.mmwgxe{0:14470} state up:active seq 5 join_fscid=1 addr [v2:172.21.15.37:6826/1559618909,v1:172.21.15.37:6827/1559618909] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:16:51.351 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi037.iifsmh{1:14482} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.37:6828/2341530495,v1:172.21.15.37:6829/2341530495] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:16:51.351 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:16:51.352 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:16:51.352 INFO:teuthology.orchestra.run.smithi037.stdout:Standby daemons: 2024-03-29T08:16:51.352 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:16:51.352 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi148.qukpjh{-1:24319} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.148:6826/2641327912,v1:172.21.15.148:6827/2641327912] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:16:51.352 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi148.sqteih{-1:24327} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.148:6824/3252351726,v1:172.21.15.148:6825/3252351726] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:16:51.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:51.351+0000 7f8c1b000700 1 -- 172.21.15.37:0/4200390190 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f8bf806cc30 msgr2=0x7f8bf806f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:16:51.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:51.351+0000 7f8c1b000700 1 --2- 172.21.15.37:0/4200390190 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f8bf806cc30 0x7f8bf806f0e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f8c00007fd0 tx=0x7f8c00009040 comp rx=0 tx=0).stop 2024-03-29T08:16:51.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:51.351+0000 7f8c1b000700 1 -- 172.21.15.37:0/4200390190 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8c14102e00 msgr2=0x7f8c14115d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:16:51.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:51.351+0000 7f8c1b000700 1 --2- 172.21.15.37:0/4200390190 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8c14102e00 0x7f8c14115d10 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f8c04009390 tx=0x7f8c04009e30 comp rx=0 tx=0).stop 2024-03-29T08:16:51.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:51.351+0000 7f8c1b000700 1 -- 172.21.15.37:0/4200390190 shutdown_connections 2024-03-29T08:16:51.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:51.351+0000 7f8c1b000700 1 --2- 172.21.15.37:0/4200390190 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f8bf806cc30 0x7f8bf806f0e0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:16:51.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:51.351+0000 7f8c1b000700 1 --2- 172.21.15.37:0/4200390190 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8c14102e00 0x7f8c14115d10 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:16:51.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:51.351+0000 7f8c1b000700 1 --2- 172.21.15.37:0/4200390190 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8c141037a0 0x7f8c14116250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:16:51.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:51.351+0000 7f8c1b000700 1 -- 172.21.15.37:0/4200390190 >> 172.21.15.37:0/4200390190 conn(0x7f8c140f89b0 msgr2=0x7f8c140704f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:16:51.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:51.351+0000 7f8c1b000700 1 -- 172.21.15.37:0/4200390190 shutdown_connections 2024-03-29T08:16:51.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:51.351+0000 7f8c1b000700 1 -- 172.21.15.37:0/4200390190 wait complete. 2024-03-29T08:16:51.353 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 20 2024-03-29T08:16:51.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:16:51 smithi037 ceph-mon[142325]: from='client.34372 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:16:51.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:16:51 smithi037 ceph-mon[142325]: from='client.34376 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:16:51.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:16:51 smithi037 ceph-mon[142325]: from='client.34380 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:16:51.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:16:51 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/3097986172' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:16:51.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:51.492+0000 7f66b8a00700 1 -- 172.21.15.37:0/2001017816 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f66b4065630 msgr2=0x7f66b4065a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:16:51.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:51.492+0000 7f66b8a00700 1 --2- 172.21.15.37:0/2001017816 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f66b4065630 0x7f66b4065a80 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f66a0004940 tx=0x7f66a0009e30 comp rx=0 tx=0).stop 2024-03-29T08:16:51.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:51.492+0000 7f66b8a00700 1 -- 172.21.15.37:0/2001017816 shutdown_connections 2024-03-29T08:16:51.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:51.492+0000 7f66b8a00700 1 --2- 172.21.15.37:0/2001017816 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f66b4065630 0x7f66b4065a80 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:16:51.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:51.492+0000 7f66b8a00700 1 --2- 172.21.15.37:0/2001017816 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f66b4102e40 0x7f66b4103210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:16:51.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:51.492+0000 7f66b8a00700 1 -- 172.21.15.37:0/2001017816 >> 172.21.15.37:0/2001017816 conn(0x7f66b4071d10 msgr2=0x7f66b4072110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:16:51.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:51.492+0000 7f66b8a00700 1 -- 172.21.15.37:0/2001017816 shutdown_connections 2024-03-29T08:16:51.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:51.493+0000 7f66b8a00700 1 -- 172.21.15.37:0/2001017816 wait complete. 2024-03-29T08:16:51.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:51.494+0000 7f66b8a00700 1 Processor -- start 2024-03-29T08:16:51.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:51.494+0000 7f66b8a00700 1 -- start start 2024-03-29T08:16:51.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:51.494+0000 7f66b8a00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f66b4065630 0x7f66b4115c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:16:51.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:51.494+0000 7f66b8a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f66b4102e40 0x7f66b4116140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:16:51.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:51.494+0000 7f66b8a00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66b4116790 con 0x7f66b4102e40 2024-03-29T08:16:51.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:51.494+0000 7f66b8a00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66b410fbd0 con 0x7f66b4065630 2024-03-29T08:16:51.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:51.495+0000 7f66b1e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f66b4102e40 0x7f66b4116140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:16:51.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:51.495+0000 7f66b1e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f66b4102e40 0x7f66b4116140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52638/0 (socket says 172.21.15.37:52638) 2024-03-29T08:16:51.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:51.495+0000 7f66b1e00700 1 -- 172.21.15.37:0/44299352 learned_addr learned my addr 172.21.15.37:0/44299352 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:16:51.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:51.495+0000 7f66b2800700 1 --2- 172.21.15.37:0/44299352 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f66b4065630 0x7f66b4115c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:16:51.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:51.495+0000 7f66b1e00700 1 -- 172.21.15.37:0/44299352 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f66b4065630 msgr2=0x7f66b4115c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:16:51.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:51.495+0000 7f66b1e00700 1 --2- 172.21.15.37:0/44299352 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f66b4065630 0x7f66b4115c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:16:51.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:51.495+0000 7f66b1e00700 1 -- 172.21.15.37:0/44299352 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66a0005040 con 0x7f66b4102e40 2024-03-29T08:16:51.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:51.496+0000 7f66b1e00700 1 --2- 172.21.15.37:0/44299352 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f66b4102e40 0x7f66b4116140 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f66a0007f50 tx=0x7f66a0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:16:51.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:51.496+0000 7f66aaa00700 1 -- 172.21.15.37:0/44299352 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f66a0021070 con 0x7f66b4102e40 2024-03-29T08:16:51.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:51.496+0000 7f66b8a00700 1 -- 172.21.15.37:0/44299352 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66b410fe50 con 0x7f66b4102e40 2024-03-29T08:16:51.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:51.496+0000 7f66aaa00700 1 -- 172.21.15.37:0/44299352 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f66a00093b0 con 0x7f66b4102e40 2024-03-29T08:16:51.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:51.496+0000 7f66aaa00700 1 -- 172.21.15.37:0/44299352 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f66a000e070 con 0x7f66b4102e40 2024-03-29T08:16:51.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:51.496+0000 7f66b8a00700 1 -- 172.21.15.37:0/44299352 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66b41103b0 con 0x7f66b4102e40 2024-03-29T08:16:51.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:51.497+0000 7f66b8a00700 1 -- 172.21.15.37:0/44299352 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66b4101800 con 0x7f66b4102e40 2024-03-29T08:16:51.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:51.498+0000 7f66aaa00700 1 -- 172.21.15.37:0/44299352 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f66a001c400 con 0x7f66b4102e40 2024-03-29T08:16:51.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:51.499+0000 7f66aaa00700 1 --2- 172.21.15.37:0/44299352 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f669806cc30 0x7f669806f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:16:51.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:51.499+0000 7f66aaa00700 1 -- 172.21.15.37:0/44299352 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f66a008d0b0 con 0x7f66b4102e40 2024-03-29T08:16:51.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:51.499+0000 7f66b2800700 1 --2- 172.21.15.37:0/44299352 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f669806cc30 0x7f669806f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:16:51.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:51.500+0000 7f66b2800700 1 --2- 172.21.15.37:0/44299352 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f669806cc30 0x7f669806f0e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f66a4009b80 tx=0x7f66a400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:16:51.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:51.503+0000 7f66aaa00700 1 -- 172.21.15.37:0/44299352 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f66a005ac00 con 0x7f66b4102e40 2024-03-29T08:16:51.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:16:51 smithi148 ceph-mon[106692]: from='client.34372 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:16:51.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:16:51 smithi148 ceph-mon[106692]: from='client.34376 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:16:51.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:16:51 smithi148 ceph-mon[106692]: from='client.34380 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:16:51.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:16:51 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/3097986172' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:16:51.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:51.739+0000 7f66b8a00700 1 -- 172.21.15.37:0/44299352 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f66b41111e0 con 0x7f669806cc30 2024-03-29T08:16:51.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:51.740+0000 7f66aaa00700 1 -- 172.21.15.37:0/44299352 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f66b41111e0 con 0x7f669806cc30 2024-03-29T08:16:51.741 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-29T08:16:51.741 INFO:teuthology.orchestra.run.smithi037.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59", 2024-03-29T08:16:51.741 INFO:teuthology.orchestra.run.smithi037.stdout: "in_progress": true, 2024-03-29T08:16:51.741 INFO:teuthology.orchestra.run.smithi037.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-29T08:16:51.741 INFO:teuthology.orchestra.run.smithi037.stdout: "services_complete": [ 2024-03-29T08:16:51.741 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr", 2024-03-29T08:16:51.741 INFO:teuthology.orchestra.run.smithi037.stdout: "mon", 2024-03-29T08:16:51.741 INFO:teuthology.orchestra.run.smithi037.stdout: "crash" 2024-03-29T08:16:51.741 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-29T08:16:51.741 INFO:teuthology.orchestra.run.smithi037.stdout: "progress": "11/21 daemons upgraded", 2024-03-29T08:16:51.741 INFO:teuthology.orchestra.run.smithi037.stdout: "message": "Currently upgrading osd daemons", 2024-03-29T08:16:51.741 INFO:teuthology.orchestra.run.smithi037.stdout: "is_paused": false 2024-03-29T08:16:51.741 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-29T08:16:51.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:51.743+0000 7f66b8a00700 1 -- 172.21.15.37:0/44299352 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f669806cc30 msgr2=0x7f669806f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:16:51.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:51.743+0000 7f66b8a00700 1 --2- 172.21.15.37:0/44299352 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f669806cc30 0x7f669806f0e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f66a4009b80 tx=0x7f66a400b040 comp rx=0 tx=0).stop 2024-03-29T08:16:51.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:51.743+0000 7f66b8a00700 1 -- 172.21.15.37:0/44299352 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f66b4102e40 msgr2=0x7f66b4116140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:16:51.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:51.743+0000 7f66b8a00700 1 --2- 172.21.15.37:0/44299352 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f66b4102e40 0x7f66b4116140 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f66a0007f50 tx=0x7f66a0009e30 comp rx=0 tx=0).stop 2024-03-29T08:16:51.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:51.743+0000 7f66b8a00700 1 -- 172.21.15.37:0/44299352 shutdown_connections 2024-03-29T08:16:51.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:51.743+0000 7f66b8a00700 1 --2- 172.21.15.37:0/44299352 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f669806cc30 0x7f669806f0e0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:16:51.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:51.743+0000 7f66b8a00700 1 --2- 172.21.15.37:0/44299352 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f66b4065630 0x7f66b4115c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:16:51.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:51.743+0000 7f66b8a00700 1 --2- 172.21.15.37:0/44299352 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f66b4102e40 0x7f66b4116140 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:16:51.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:51.743+0000 7f66b8a00700 1 -- 172.21.15.37:0/44299352 >> 172.21.15.37:0/44299352 conn(0x7f66b4071d10 msgr2=0x7f66b40fe980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:16:51.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:51.743+0000 7f66b8a00700 1 -- 172.21.15.37:0/44299352 shutdown_connections 2024-03-29T08:16:51.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:51.743+0000 7f66b8a00700 1 -- 172.21.15.37:0/44299352 wait complete. 2024-03-29T08:16:51.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:51.880+0000 7f2b54c00700 1 -- 172.21.15.37:0/3717847676 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2b501037a0 msgr2=0x7f2b500fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:16:51.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:51.880+0000 7f2b54c00700 1 --2- 172.21.15.37:0/3717847676 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2b501037a0 0x7f2b500fce10 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f2b3c004940 tx=0x7f2b3c009e30 comp rx=0 tx=0).stop 2024-03-29T08:16:51.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:51.880+0000 7f2b54c00700 1 -- 172.21.15.37:0/3717847676 shutdown_connections 2024-03-29T08:16:51.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:51.880+0000 7f2b54c00700 1 --2- 172.21.15.37:0/3717847676 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2b501037a0 0x7f2b500fce10 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:16:51.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:51.880+0000 7f2b54c00700 1 --2- 172.21.15.37:0/3717847676 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2b50102e00 0x7f2b501031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:16:51.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:51.880+0000 7f2b54c00700 1 -- 172.21.15.37:0/3717847676 >> 172.21.15.37:0/3717847676 conn(0x7f2b500f89b0 msgr2=0x7f2b500fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:16:51.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:51.881+0000 7f2b54c00700 1 -- 172.21.15.37:0/3717847676 shutdown_connections 2024-03-29T08:16:51.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:51.881+0000 7f2b54c00700 1 -- 172.21.15.37:0/3717847676 wait complete. 2024-03-29T08:16:51.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:51.881+0000 7f2b54c00700 1 Processor -- start 2024-03-29T08:16:51.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:51.882+0000 7f2b54c00700 1 -- start start 2024-03-29T08:16:51.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:51.882+0000 7f2b54c00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2b50102e00 0x7f2b50113a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:16:51.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:51.882+0000 7f2b54c00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2b501037a0 0x7f2b50113fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:16:51.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:51.882+0000 7f2b54c00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b50114600 con 0x7f2b501037a0 2024-03-29T08:16:51.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:51.882+0000 7f2b54c00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b5010d9f0 con 0x7f2b50102e00 2024-03-29T08:16:51.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:51.882+0000 7f2b4d400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2b501037a0 0x7f2b50113fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:16:51.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:51.882+0000 7f2b4de00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2b50102e00 0x7f2b50113a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:16:51.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:51.882+0000 7f2b4d400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2b501037a0 0x7f2b50113fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52642/0 (socket says 172.21.15.37:52642) 2024-03-29T08:16:51.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:51.882+0000 7f2b4d400700 1 -- 172.21.15.37:0/3928219044 learned_addr learned my addr 172.21.15.37:0/3928219044 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:16:51.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:51.883+0000 7f2b4d400700 1 -- 172.21.15.37:0/3928219044 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2b50102e00 msgr2=0x7f2b50113a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:16:51.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:51.883+0000 7f2b4d400700 1 --2- 172.21.15.37:0/3928219044 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2b50102e00 0x7f2b50113a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:16:51.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:51.883+0000 7f2b4d400700 1 -- 172.21.15.37:0/3928219044 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b3c005040 con 0x7f2b501037a0 2024-03-29T08:16:51.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:51.883+0000 7f2b4d400700 1 --2- 172.21.15.37:0/3928219044 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2b501037a0 0x7f2b50113fb0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f2b3c008d50 tx=0x7f2b3c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:16:51.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:51.883+0000 7f2b46000700 1 -- 172.21.15.37:0/3928219044 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2b3c00e030 con 0x7f2b501037a0 2024-03-29T08:16:51.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:51.883+0000 7f2b46000700 1 -- 172.21.15.37:0/3928219044 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f2b3c0093b0 con 0x7f2b501037a0 2024-03-29T08:16:51.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:51.883+0000 7f2b54c00700 1 -- 172.21.15.37:0/3928219044 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b5010dc70 con 0x7f2b501037a0 2024-03-29T08:16:51.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:51.883+0000 7f2b46000700 1 -- 172.21.15.37:0/3928219044 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2b3c01cbb0 con 0x7f2b501037a0 2024-03-29T08:16:51.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:51.883+0000 7f2b54c00700 1 -- 172.21.15.37:0/3928219044 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b5010e230 con 0x7f2b501037a0 2024-03-29T08:16:51.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:51.884+0000 7f2b54c00700 1 -- 172.21.15.37:0/3928219044 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b500fe550 con 0x7f2b501037a0 2024-03-29T08:16:51.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:51.885+0000 7f2b46000700 1 -- 172.21.15.37:0/3928219044 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f2b3c013930 con 0x7f2b501037a0 2024-03-29T08:16:51.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:51.887+0000 7f2b46000700 1 --2- 172.21.15.37:0/3928219044 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f2b3406cc30 0x7f2b3406f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:16:51.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:51.887+0000 7f2b46000700 1 -- 172.21.15.37:0/3928219044 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f2b3c08d7d0 con 0x7f2b501037a0 2024-03-29T08:16:51.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:51.887+0000 7f2b4de00700 1 --2- 172.21.15.37:0/3928219044 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f2b3406cc30 0x7f2b3406f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:16:51.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:51.888+0000 7f2b4de00700 1 --2- 172.21.15.37:0/3928219044 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f2b3406cc30 0x7f2b3406f0e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f2b40009b80 tx=0x7f2b4000b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:16:51.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:51.891+0000 7f2b46000700 1 -- 172.21.15.37:0/3928219044 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f2b3c05b2a0 con 0x7f2b501037a0 2024-03-29T08:16:52.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:52.191+0000 7f2b54c00700 1 -- 172.21.15.37:0/3928219044 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f2b5005f5f0 con 0x7f2b501037a0 2024-03-29T08:16:52.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:52.192+0000 7f2b46000700 1 -- 172.21.15.37:0/3928219044 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f2b3c05ae30 con 0x7f2b501037a0 2024-03-29T08:16:52.192 INFO:teuthology.orchestra.run.smithi037.stdout:HEALTH_OK 2024-03-29T08:16:52.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:52.194+0000 7f2b54c00700 1 -- 172.21.15.37:0/3928219044 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f2b3406cc30 msgr2=0x7f2b3406f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:16:52.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:52.195+0000 7f2b54c00700 1 --2- 172.21.15.37:0/3928219044 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f2b3406cc30 0x7f2b3406f0e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f2b40009b80 tx=0x7f2b4000b040 comp rx=0 tx=0).stop 2024-03-29T08:16:52.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:52.195+0000 7f2b54c00700 1 -- 172.21.15.37:0/3928219044 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2b501037a0 msgr2=0x7f2b50113fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:16:52.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:52.195+0000 7f2b54c00700 1 --2- 172.21.15.37:0/3928219044 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2b501037a0 0x7f2b50113fb0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f2b3c008d50 tx=0x7f2b3c009e30 comp rx=0 tx=0).stop 2024-03-29T08:16:52.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:52.195+0000 7f2b54c00700 1 -- 172.21.15.37:0/3928219044 shutdown_connections 2024-03-29T08:16:52.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:52.195+0000 7f2b54c00700 1 --2- 172.21.15.37:0/3928219044 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f2b3406cc30 0x7f2b3406f0e0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:16:52.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:52.195+0000 7f2b54c00700 1 --2- 172.21.15.37:0/3928219044 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2b50102e00 0x7f2b50113a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:16:52.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:52.195+0000 7f2b54c00700 1 --2- 172.21.15.37:0/3928219044 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2b501037a0 0x7f2b50113fb0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:16:52.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:52.195+0000 7f2b54c00700 1 -- 172.21.15.37:0/3928219044 >> 172.21.15.37:0/3928219044 conn(0x7f2b500f89b0 msgr2=0x7f2b500f9a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:16:52.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:52.195+0000 7f2b54c00700 1 -- 172.21.15.37:0/3928219044 shutdown_connections 2024-03-29T08:16:52.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:16:52.195+0000 7f2b54c00700 1 -- 172.21.15.37:0/3928219044 wait complete. 2024-03-29T08:16:52.422 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:16:52 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/4200390190' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T08:16:52.422 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:16:52 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-03-29T08:16:52.422 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:16:52 smithi148 ceph-mon[106692]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-03-29T08:16:52.422 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:16:52 smithi148 ceph-mon[106692]: Upgrade: osd.5 is safe to restart 2024-03-29T08:16:52.422 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:16:52 smithi148 ceph-mon[106692]: pgmap v169: 65 pgs: 65 active+clean; 259 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:16:52.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:16:52 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/4200390190' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T08:16:52.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:16:52 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-03-29T08:16:52.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:16:52 smithi037 ceph-mon[142325]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-03-29T08:16:52.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:16:52 smithi037 ceph-mon[142325]: Upgrade: osd.5 is safe to restart 2024-03-29T08:16:52.428 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:16:52 smithi037 ceph-mon[142325]: pgmap v169: 65 pgs: 65 active+clean; 259 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:16:53.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:16:53 smithi037 ceph-mon[142325]: from='client.34392 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:16:53.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:16:53 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/3928219044' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T08:16:53.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:16:53 smithi148 ceph-mon[106692]: from='client.34392 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:16:53.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:16:53 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/3928219044' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T08:16:54.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:16:54 smithi148 ceph-mon[106692]: Upgrade: Updating osd.5 2024-03-29T08:16:54.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:16:54 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:16:54.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:16:54 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-29T08:16:54.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:16:54 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T08:16:54.557 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:16:54 smithi148 ceph-mon[106692]: Deploying daemon osd.5 on smithi148 2024-03-29T08:16:54.557 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:16:54 smithi148 ceph-mon[106692]: pgmap v170: 65 pgs: 65 active+clean; 259 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:16:54.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:16:54 smithi037 ceph-mon[142325]: Upgrade: Updating osd.5 2024-03-29T08:16:54.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:16:54 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:16:54.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:16:54 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-29T08:16:54.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:16:54 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T08:16:54.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:16:54 smithi037 ceph-mon[142325]: Deploying daemon osd.5 on smithi148 2024-03-29T08:16:54.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:16:54 smithi037 ceph-mon[142325]: pgmap v170: 65 pgs: 65 active+clean; 259 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:16:55.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:16:55 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:16:55.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:16:55 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:16:55.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:16:55 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:16:55.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:16:55 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:16:56.746 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:16:56 smithi148 ceph-mon[106692]: pgmap v171: 65 pgs: 65 active+clean; 259 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:16:56.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:16:56 smithi037 ceph-mon[142325]: pgmap v171: 65 pgs: 65 active+clean; 259 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:16:57.556 INFO:journalctl@ceph.osd.5.smithi148.stdout:Mar 29 08:16:57 smithi148 systemd[1]: Stopping Ceph osd.5 for 99b492c2-eda1-11ee-b647-cb9ed24678a4... 2024-03-29T08:16:58.056 INFO:journalctl@ceph.osd.5.smithi148.stdout:Mar 29 08:16:57 smithi148 ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4-osd-5[68071]: 2024-03-29T08:16:57.630+0000 7fa898c00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-29T08:16:58.056 INFO:journalctl@ceph.osd.5.smithi148.stdout:Mar 29 08:16:57 smithi148 ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4-osd-5[68071]: 2024-03-29T08:16:57.630+0000 7fa898c00700 -1 osd.5 75 *** Got signal Terminated *** 2024-03-29T08:16:58.056 INFO:journalctl@ceph.osd.5.smithi148.stdout:Mar 29 08:16:57 smithi148 ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4-osd-5[68071]: 2024-03-29T08:16:57.630+0000 7fa898c00700 -1 osd.5 75 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-29T08:16:58.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:16:58 smithi037 ceph-mon[142325]: pgmap v172: 65 pgs: 65 active+clean; 259 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:16:58.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:16:58 smithi037 ceph-mon[142325]: osd.5 marked itself down and dead 2024-03-29T08:16:59.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:16:58 smithi148 ceph-mon[106692]: pgmap v172: 65 pgs: 65 active+clean; 259 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:16:59.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:16:58 smithi148 ceph-mon[106692]: osd.5 marked itself down and dead 2024-03-29T08:16:59.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:16:59 smithi148 ceph-mon[106692]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-29T08:16:59.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:16:59 smithi148 ceph-mon[106692]: osdmap e76: 6 total, 5 up, 6 in 2024-03-29T08:16:59.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:16:59 smithi148 ceph-mon[106692]: osdmap e77: 6 total, 5 up, 6 in 2024-03-29T08:16:59.807 INFO:journalctl@ceph.osd.5.smithi148.stdout:Mar 29 08:16:59 smithi148 bash[130589]: ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4-osd-5 2024-03-29T08:16:59.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:16:59 smithi037 ceph-mon[142325]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-29T08:16:59.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:16:59 smithi037 ceph-mon[142325]: osdmap e76: 6 total, 5 up, 6 in 2024-03-29T08:16:59.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:16:59 smithi037 ceph-mon[142325]: osdmap e77: 6 total, 5 up, 6 in 2024-03-29T08:17:00.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:17:00 smithi148 ceph-mon[106692]: pgmap v174: 65 pgs: 11 peering, 7 stale+active+clean, 47 active+clean; 259 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:17:00.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:17:00 smithi148 ceph-mon[106692]: Health check failed: Reduced data availability: 6 pgs peering (PG_AVAILABILITY) 2024-03-29T08:17:00.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:17:00 smithi037 ceph-mon[142325]: pgmap v174: 65 pgs: 11 peering, 7 stale+active+clean, 47 active+clean; 259 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-29T08:17:00.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:17:00 smithi037 ceph-mon[142325]: Health check failed: Reduced data availability: 6 pgs peering (PG_AVAILABILITY) 2024-03-29T08:17:01.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:17:01 smithi037 ceph-mon[142325]: pgmap v176: 65 pgs: 6 active+undersized+degraded, 11 peering, 4 stale+active+clean, 4 active+undersized, 40 active+clean; 259 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 30/324 objects degraded (9.259%) 2024-03-29T08:17:01.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:17:01 smithi037 ceph-mon[142325]: Health check failed: Degraded data redundancy: 30/324 objects degraded (9.259%), 6 pgs degraded (PG_DEGRADED) 2024-03-29T08:17:01.942 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:17:01 smithi148 ceph-mon[106692]: pgmap v176: 65 pgs: 6 active+undersized+degraded, 11 peering, 4 stale+active+clean, 4 active+undersized, 40 active+clean; 259 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 30/324 objects degraded (9.259%) 2024-03-29T08:17:01.942 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:17:01 smithi148 ceph-mon[106692]: Health check failed: Degraded data redundancy: 30/324 objects degraded (9.259%), 6 pgs degraded (PG_DEGRADED) 2024-03-29T08:17:01.942 INFO:journalctl@ceph.osd.5.smithi148.stdout:Mar 29 08:17:01 smithi148 systemd[1]: ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4@osd.5.service: Succeeded. 2024-03-29T08:17:01.942 INFO:journalctl@ceph.osd.5.smithi148.stdout:Mar 29 08:17:01 smithi148 systemd[1]: Stopped Ceph osd.5 for 99b492c2-eda1-11ee-b647-cb9ed24678a4. 2024-03-29T08:17:02.306 INFO:journalctl@ceph.osd.5.smithi148.stdout:Mar 29 08:17:01 smithi148 systemd[1]: Starting Ceph osd.5 for 99b492c2-eda1-11ee-b647-cb9ed24678a4... 2024-03-29T08:17:03.681 INFO:journalctl@ceph.osd.5.smithi148.stdout:Mar 29 08:17:03 smithi148 bash[130857]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-03-29T08:17:03.681 INFO:journalctl@ceph.osd.5.smithi148.stdout:Mar 29 08:17:03 smithi148 bash[130857]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-5 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-03-29T08:17:03.681 INFO:journalctl@ceph.osd.5.smithi148.stdout:Mar 29 08:17:03 smithi148 bash[130857]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-03-29T08:17:03.681 INFO:journalctl@ceph.osd.5.smithi148.stdout:Mar 29 08:17:03 smithi148 bash[130857]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-03-29T08:17:03.681 INFO:journalctl@ceph.osd.5.smithi148.stdout:Mar 29 08:17:03 smithi148 bash[130857]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5/block 2024-03-29T08:17:03.682 INFO:journalctl@ceph.osd.5.smithi148.stdout:Mar 29 08:17:03 smithi148 bash[130857]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-03-29T08:17:03.682 INFO:journalctl@ceph.osd.5.smithi148.stdout:Mar 29 08:17:03 smithi148 bash[130857]: --> ceph-volume raw activate successful for osd ID: 5 2024-03-29T08:17:04.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:17:03 smithi148 ceph-mon[106692]: pgmap v177: 65 pgs: 9 active+undersized+degraded, 11 peering, 8 active+undersized, 37 active+clean; 259 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 37/324 objects degraded (11.420%) 2024-03-29T08:17:04.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:17:03 smithi037 ceph-mon[142325]: pgmap v177: 65 pgs: 9 active+undersized+degraded, 11 peering, 8 active+undersized, 37 active+clean; 259 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 37/324 objects degraded (11.420%) 2024-03-29T08:17:05.056 INFO:journalctl@ceph.osd.5.smithi148.stdout:Mar 29 08:17:04 smithi148 ceph-osd[131146]: -- 172.21.15.148:0/3499970226 <== mon.1 v2:172.21.15.148:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55b9b4dd11e0 con 0x55b9b4da1000 2024-03-29T08:17:05.056 INFO:journalctl@ceph.osd.5.smithi148.stdout:Mar 29 08:17:04 smithi148 bash[131109]: 474c4f9212cef592d9961f4cff2079ce174000f6cf1cfbb79801c656acdd6877 2024-03-29T08:17:05.056 INFO:journalctl@ceph.osd.5.smithi148.stdout:Mar 29 08:17:04 smithi148 systemd[1]: Started Ceph osd.5 for 99b492c2-eda1-11ee-b647-cb9ed24678a4. 2024-03-29T08:17:05.556 INFO:journalctl@ceph.osd.5.smithi148.stdout:Mar 29 08:17:05 smithi148 ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4-osd-5[131123]: 2024-03-29T08:17:05.267+0000 7f681505d640 -1 Falling back to public interface 2024-03-29T08:17:06.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:17:05 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:17:06.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:17:05 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:17:06.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:17:05 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T08:17:06.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:17:05 smithi037 ceph-mon[142325]: pgmap v178: 65 pgs: 14 active+undersized+degraded, 14 active+undersized, 37 active+clean; 259 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 51/324 objects degraded (15.741%) 2024-03-29T08:17:06.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:17:05 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:17:06.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:17:05 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:17:06.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:17:05 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T08:17:06.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:17:05 smithi148 ceph-mon[106692]: pgmap v178: 65 pgs: 14 active+undersized+degraded, 14 active+undersized, 37 active+clean; 259 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 51/324 objects degraded (15.741%) 2024-03-29T08:17:07.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:17:06 smithi148 ceph-mon[106692]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 6 pgs peering) 2024-03-29T08:17:07.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:17:06 smithi037 ceph-mon[142325]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 6 pgs peering) 2024-03-29T08:17:08.175 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:17:07 smithi148 ceph-mon[106692]: pgmap v179: 65 pgs: 14 active+undersized+degraded, 14 active+undersized, 37 active+clean; 259 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 51/324 objects degraded (15.741%) 2024-03-29T08:17:08.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:17:07 smithi037 ceph-mon[142325]: pgmap v179: 65 pgs: 14 active+undersized+degraded, 14 active+undersized, 37 active+clean; 259 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 51/324 objects degraded (15.741%) 2024-03-29T08:17:10.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:17:09 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:17:10.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:17:09 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:17:10.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:17:09 smithi148 ceph-mon[106692]: Health check update: Degraded data redundancy: 51/324 objects degraded (15.741%), 14 pgs degraded (PG_DEGRADED) 2024-03-29T08:17:10.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:17:09 smithi148 ceph-mon[106692]: pgmap v180: 65 pgs: 14 active+undersized+degraded, 14 active+undersized, 37 active+clean; 259 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 51/324 objects degraded (15.741%) 2024-03-29T08:17:10.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:17:09 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:17:10.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:17:09 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:17:10.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:17:09 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:17:10.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:17:09 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:17:10.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:17:09 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:17:10.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:17:09 smithi037 ceph-mon[142325]: Health check update: Degraded data redundancy: 51/324 objects degraded (15.741%), 14 pgs degraded (PG_DEGRADED) 2024-03-29T08:17:10.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:17:09 smithi037 ceph-mon[142325]: pgmap v180: 65 pgs: 14 active+undersized+degraded, 14 active+undersized, 37 active+clean; 259 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 51/324 objects degraded (15.741%) 2024-03-29T08:17:10.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:17:09 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:17:10.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:17:09 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:17:10.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:17:09 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:17:10.802 INFO:journalctl@ceph.osd.5.smithi148.stdout:Mar 29 08:17:10 smithi148 ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4-osd-5[131123]: 2024-03-29T08:17:10.400+0000 7f681505d640 -1 osd.5 75 log_to_monitors true 2024-03-29T08:17:11.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:17:11 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:17:11.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:17:11 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:17:11.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:17:11 smithi148 ceph-mon[106692]: from='osd.5 [v2:172.21.15.148:6816/2577126633,v1:172.21.15.148:6817/2577126633]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-29T08:17:11.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:17:11 smithi148 ceph-mon[106692]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-29T08:17:11.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:17:11 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:17:11.428 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:17:11 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:17:11.428 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:17:11 smithi037 ceph-mon[142325]: from='osd.5 [v2:172.21.15.148:6816/2577126633,v1:172.21.15.148:6817/2577126633]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-29T08:17:11.428 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:17:11 smithi037 ceph-mon[142325]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-29T08:17:11.556 INFO:journalctl@ceph.osd.5.smithi148.stdout:Mar 29 08:17:11 smithi148 ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4-osd-5[131123]: 2024-03-29T08:17:11.140+0000 7f6804400700 -1 osd.5 75 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-29T08:17:12.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:17:12 smithi148 ceph-mon[106692]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-29T08:17:12.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:17:12 smithi148 ceph-mon[106692]: osdmap e78: 6 total, 5 up, 6 in 2024-03-29T08:17:12.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:17:12 smithi148 ceph-mon[106692]: from='osd.5 [v2:172.21.15.148:6816/2577126633,v1:172.21.15.148:6817/2577126633]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-03-29T08:17:12.307 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:17:12 smithi148 ceph-mon[106692]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-03-29T08:17:12.307 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:17:12 smithi148 ceph-mon[106692]: pgmap v182: 65 pgs: 14 active+undersized+degraded, 14 active+undersized, 37 active+clean; 259 MiB data, 987 MiB used, 535 GiB / 536 GiB avail; 51/324 objects degraded (15.741%) 2024-03-29T08:17:12.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:17:12 smithi037 ceph-mon[142325]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-29T08:17:12.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:17:12 smithi037 ceph-mon[142325]: osdmap e78: 6 total, 5 up, 6 in 2024-03-29T08:17:12.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:17:12 smithi037 ceph-mon[142325]: from='osd.5 [v2:172.21.15.148:6816/2577126633,v1:172.21.15.148:6817/2577126633]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-03-29T08:17:12.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:17:12 smithi037 ceph-mon[142325]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-03-29T08:17:12.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:17:12 smithi037 ceph-mon[142325]: pgmap v182: 65 pgs: 14 active+undersized+degraded, 14 active+undersized, 37 active+clean; 259 MiB data, 987 MiB used, 535 GiB / 536 GiB avail; 51/324 objects degraded (15.741%) 2024-03-29T08:17:13.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:17:13 smithi037 ceph-mon[142325]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-29T08:17:13.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:17:13 smithi037 ceph-mon[142325]: osd.5 [v2:172.21.15.148:6816/2577126633,v1:172.21.15.148:6817/2577126633] boot 2024-03-29T08:17:13.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:17:13 smithi037 ceph-mon[142325]: osdmap e79: 6 total, 6 up, 6 in 2024-03-29T08:17:13.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:17:13 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-29T08:17:13.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:17:13 smithi148 ceph-mon[106692]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-29T08:17:13.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:17:13 smithi148 ceph-mon[106692]: osd.5 [v2:172.21.15.148:6816/2577126633,v1:172.21.15.148:6817/2577126633] boot 2024-03-29T08:17:13.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:17:13 smithi148 ceph-mon[106692]: osdmap e79: 6 total, 6 up, 6 in 2024-03-29T08:17:13.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:17:13 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-29T08:17:14.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:17:14 smithi037 ceph-mon[142325]: osdmap e80: 6 total, 6 up, 6 in 2024-03-29T08:17:14.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:17:14 smithi037 ceph-mon[142325]: pgmap v185: 65 pgs: 12 active+undersized+degraded, 13 active+undersized, 40 active+clean; 259 MiB data, 987 MiB used, 535 GiB / 536 GiB avail; 47/324 objects degraded (14.506%) 2024-03-29T08:17:14.499 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:17:14 smithi148 ceph-mon[106692]: osdmap e80: 6 total, 6 up, 6 in 2024-03-29T08:17:14.499 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:17:14 smithi148 ceph-mon[106692]: pgmap v185: 65 pgs: 12 active+undersized+degraded, 13 active+undersized, 40 active+clean; 259 MiB data, 987 MiB used, 535 GiB / 536 GiB avail; 47/324 objects degraded (14.506%) 2024-03-29T08:17:15.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:17:15 smithi037 ceph-mon[142325]: Health check update: Degraded data redundancy: 47/324 objects degraded (14.506%), 12 pgs degraded (PG_DEGRADED) 2024-03-29T08:17:15.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:17:15 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:17:15.428 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:17:15 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:17:15.428 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:17:15 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T08:17:15.428 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:17:15 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T08:17:15.428 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:17:15 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:17:15.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:17:15 smithi148 ceph-mon[106692]: Health check update: Degraded data redundancy: 47/324 objects degraded (14.506%), 12 pgs degraded (PG_DEGRADED) 2024-03-29T08:17:15.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:17:15 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:17:15.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:17:15 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:17:15.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:17:15 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T08:17:15.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:17:15 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T08:17:15.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:17:15 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:17:16.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:17:16 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T08:17:16.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:17:16 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:17:16.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:17:16 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:17:16.428 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:17:16 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:17:16.428 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:17:16 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:17:16.428 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:17:16 smithi037 ceph-mon[142325]: Upgrade: Setting container_image for all osd 2024-03-29T08:17:16.428 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:17:16 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:17:16.428 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:17:16 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]: dispatch 2024-03-29T08:17:16.428 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:17:16 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]': finished 2024-03-29T08:17:16.428 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:17:16 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]: dispatch 2024-03-29T08:17:16.428 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:17:16 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]': finished 2024-03-29T08:17:16.428 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:17:16 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]: dispatch 2024-03-29T08:17:16.428 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:17:16 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]': finished 2024-03-29T08:17:16.428 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:17:16 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]: dispatch 2024-03-29T08:17:16.428 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:17:16 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]': finished 2024-03-29T08:17:16.428 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:17:16 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]: dispatch 2024-03-29T08:17:16.429 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:17:16 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]': finished 2024-03-29T08:17:16.429 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:17:16 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]: dispatch 2024-03-29T08:17:16.429 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:17:16 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]': finished 2024-03-29T08:17:16.429 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:17:16 smithi037 ceph-mon[142325]: Upgrade: Setting require_osd_release to 18 reef 2024-03-29T08:17:16.429 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:17:16 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd require-osd-release", "release": "reef"}]: dispatch 2024-03-29T08:17:16.429 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:17:16 smithi037 ceph-mon[142325]: pgmap v186: 65 pgs: 10 active+undersized+degraded, 10 active+undersized, 45 active+clean; 259 MiB data, 987 MiB used, 535 GiB / 536 GiB avail; 41/324 objects degraded (12.654%) 2024-03-29T08:17:16.429 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:17:16 smithi037 ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4-mon-smithi037[142302]: 2024-03-29T08:17:16.235+0000 7f2c5d200700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-29T08:17:16.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:17:16 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T08:17:16.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:17:16 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:17:16.557 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:17:16 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:17:16.557 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:17:16 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:17:16.557 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:17:16 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:17:16.557 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:17:16 smithi148 ceph-mon[106692]: Upgrade: Setting container_image for all osd 2024-03-29T08:17:16.557 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:17:16 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:17:16.557 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:17:16 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]: dispatch 2024-03-29T08:17:16.557 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:17:16 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]': finished 2024-03-29T08:17:16.557 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:17:16 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]: dispatch 2024-03-29T08:17:16.557 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:17:16 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]': finished 2024-03-29T08:17:16.557 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:17:16 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]: dispatch 2024-03-29T08:17:16.557 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:17:16 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]': finished 2024-03-29T08:17:16.557 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:17:16 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]: dispatch 2024-03-29T08:17:16.557 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:17:16 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]': finished 2024-03-29T08:17:16.558 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:17:16 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]: dispatch 2024-03-29T08:17:16.558 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:17:16 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]': finished 2024-03-29T08:17:16.558 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:17:16 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]: dispatch 2024-03-29T08:17:16.558 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:17:16 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]': finished 2024-03-29T08:17:16.558 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:17:16 smithi148 ceph-mon[106692]: Upgrade: Setting require_osd_release to 18 reef 2024-03-29T08:17:16.558 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:17:16 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd require-osd-release", "release": "reef"}]: dispatch 2024-03-29T08:17:16.558 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:17:16 smithi148 ceph-mon[106692]: pgmap v186: 65 pgs: 10 active+undersized+degraded, 10 active+undersized, 45 active+clean; 259 MiB data, 987 MiB used, 535 GiB / 536 GiB avail; 41/324 objects degraded (12.654%) 2024-03-29T08:17:17.379 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:17:17 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd='[{"prefix": "osd require-osd-release", "release": "reef"}]': finished 2024-03-29T08:17:17.379 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:17:17 smithi037 ceph-mon[142325]: osdmap e81: 6 total, 6 up, 6 in 2024-03-29T08:17:17.379 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:17:17 smithi037 ceph-mon[142325]: Upgrade: failing fs cephfs for rapid multi-rank mds upgrade 2024-03-29T08:17:17.379 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:17:17 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-03-29T08:17:17.379 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:17:17 smithi037 ceph-mon[142325]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-29T08:17:17.379 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:17:17 smithi037 ceph-mon[142325]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-29T08:17:17.380 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:17:17 smithi037 ceph-mon[142325]: osdmap e82: 6 total, 6 up, 6 in 2024-03-29T08:17:17.380 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:17:17 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-03-29T08:17:17.380 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:17:17 smithi037 ceph-mon[142325]: fsmap cephfs:0/2 2 up:standby, 2 failed 2024-03-29T08:17:17.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:17:17 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd='[{"prefix": "osd require-osd-release", "release": "reef"}]': finished 2024-03-29T08:17:17.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:17:17 smithi148 ceph-mon[106692]: osdmap e81: 6 total, 6 up, 6 in 2024-03-29T08:17:17.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:17:17 smithi148 ceph-mon[106692]: Upgrade: failing fs cephfs for rapid multi-rank mds upgrade 2024-03-29T08:17:17.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:17:17 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-03-29T08:17:17.557 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:17:17 smithi148 ceph-mon[106692]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-29T08:17:17.557 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:17:17 smithi148 ceph-mon[106692]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-29T08:17:17.557 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:17:17 smithi148 ceph-mon[106692]: osdmap e82: 6 total, 6 up, 6 in 2024-03-29T08:17:17.557 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:17:17 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-03-29T08:17:17.557 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:17:17 smithi148 ceph-mon[106692]: fsmap cephfs:0/2 2 up:standby, 2 failed 2024-03-29T08:17:18.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:17:18 smithi148 ceph-mon[106692]: mds.? [v2:172.21.15.37:6826/3877453059,v1:172.21.15.37:6827/3877453059] up:boot 2024-03-29T08:17:18.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:17:18 smithi148 ceph-mon[106692]: mds.? [v2:172.21.15.37:6828/2521422502,v1:172.21.15.37:6829/2521422502] up:boot 2024-03-29T08:17:18.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:17:18 smithi148 ceph-mon[106692]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-03-29T08:17:18.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:17:18 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi037.iifsmh"}]: dispatch 2024-03-29T08:17:18.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:17:18 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi037.mmwgxe"}]: dispatch 2024-03-29T08:17:18.557 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:17:18 smithi148 ceph-mon[106692]: pgmap v189: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:17:18.557 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:17:18 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:17:18.557 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:17:18 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi037.mmwgxe", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-29T08:17:18.557 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:17:18 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T08:17:18.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:17:18 smithi037 ceph-mon[142325]: mds.? [v2:172.21.15.37:6826/3877453059,v1:172.21.15.37:6827/3877453059] up:boot 2024-03-29T08:17:18.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:17:18 smithi037 ceph-mon[142325]: mds.? [v2:172.21.15.37:6828/2521422502,v1:172.21.15.37:6829/2521422502] up:boot 2024-03-29T08:17:18.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:17:18 smithi037 ceph-mon[142325]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-03-29T08:17:18.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:17:18 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi037.iifsmh"}]: dispatch 2024-03-29T08:17:18.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:17:18 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi037.mmwgxe"}]: dispatch 2024-03-29T08:17:18.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:17:18 smithi037 ceph-mon[142325]: pgmap v189: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:17:18.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:17:18 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:17:18.678 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:17:18 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi037.mmwgxe", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-29T08:17:18.678 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:17:18 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T08:17:19.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:17:19 smithi037 ceph-mon[142325]: Upgrade: Updating mds.cephfs.smithi037.mmwgxe 2024-03-29T08:17:19.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:17:19 smithi037 ceph-mon[142325]: Deploying daemon mds.cephfs.smithi037.mmwgxe on smithi037 2024-03-29T08:17:19.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:17:19 smithi037 ceph-mon[142325]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 41/324 objects degraded (12.654%), 10 pgs degraded) 2024-03-29T08:17:19.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:17:19 smithi148 ceph-mon[106692]: Upgrade: Updating mds.cephfs.smithi037.mmwgxe 2024-03-29T08:17:19.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:17:19 smithi148 ceph-mon[106692]: Deploying daemon mds.cephfs.smithi037.mmwgxe on smithi037 2024-03-29T08:17:19.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:17:19 smithi148 ceph-mon[106692]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 41/324 objects degraded (12.654%), 10 pgs degraded) 2024-03-29T08:17:20.428 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:17:20 smithi037 ceph-mon[142325]: pgmap v190: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:17:20.428 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:17:20 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:17:20.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:17:20 smithi148 ceph-mon[106692]: pgmap v190: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:17:20.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:17:20 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:17:22.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:17:21 smithi037 ceph-mon[142325]: osdmap e83: 6 total, 6 up, 6 in 2024-03-29T08:17:22.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:17:21 smithi037 ceph-mon[142325]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-03-29T08:17:22.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:17:21 smithi037 ceph-mon[142325]: pgmap v192: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:17:22.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:17:21 smithi148 ceph-mon[106692]: osdmap e83: 6 total, 6 up, 6 in 2024-03-29T08:17:22.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:17:21 smithi148 ceph-mon[106692]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-03-29T08:17:22.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:17:21 smithi148 ceph-mon[106692]: pgmap v192: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:17:22.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:22.350+0000 7fc1d1000700 1 -- 172.21.15.37:0/3724330556 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc1cc1037a0 msgr2=0x7fc1cc0fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:17:22.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:22.350+0000 7fc1d1000700 1 --2- 172.21.15.37:0/3724330556 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc1cc1037a0 0x7fc1cc0fce10 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fc1b8004940 tx=0x7fc1b8009e30 comp rx=0 tx=0).stop 2024-03-29T08:17:22.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:22.350+0000 7fc1d1000700 1 -- 172.21.15.37:0/3724330556 shutdown_connections 2024-03-29T08:17:22.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:22.350+0000 7fc1d1000700 1 --2- 172.21.15.37:0/3724330556 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc1cc1037a0 0x7fc1cc0fce10 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:17:22.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:22.350+0000 7fc1d1000700 1 --2- 172.21.15.37:0/3724330556 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc1cc102e00 0x7fc1cc1031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:17:22.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:22.350+0000 7fc1d1000700 1 -- 172.21.15.37:0/3724330556 >> 172.21.15.37:0/3724330556 conn(0x7fc1cc0f89b0 msgr2=0x7fc1cc0fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:17:22.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:22.350+0000 7fc1d1000700 1 -- 172.21.15.37:0/3724330556 shutdown_connections 2024-03-29T08:17:22.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:22.350+0000 7fc1d1000700 1 -- 172.21.15.37:0/3724330556 wait complete. 2024-03-29T08:17:22.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:22.351+0000 7fc1d1000700 1 Processor -- start 2024-03-29T08:17:22.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:22.351+0000 7fc1d1000700 1 -- start start 2024-03-29T08:17:22.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:22.351+0000 7fc1d1000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc1cc102e00 0x7fc1cc113b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:17:22.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:22.351+0000 7fc1d1000700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc1cc1037a0 0x7fc1cc114040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:17:22.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:22.351+0000 7fc1d1000700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1cc114690 con 0x7fc1cc102e00 2024-03-29T08:17:22.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:22.351+0000 7fc1d1000700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1cc10da80 con 0x7fc1cc1037a0 2024-03-29T08:17:22.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:22.352+0000 7fc1c9e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc1cc102e00 0x7fc1cc113b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:17:22.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:22.352+0000 7fc1c9e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc1cc102e00 0x7fc1cc113b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34106/0 (socket says 172.21.15.37:34106) 2024-03-29T08:17:22.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:22.352+0000 7fc1c9e00700 1 -- 172.21.15.37:0/2991675378 learned_addr learned my addr 172.21.15.37:0/2991675378 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:17:22.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:22.352+0000 7fc1c9400700 1 --2- 172.21.15.37:0/2991675378 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc1cc1037a0 0x7fc1cc114040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:17:22.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:22.352+0000 7fc1c9e00700 1 -- 172.21.15.37:0/2991675378 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc1cc1037a0 msgr2=0x7fc1cc114040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:17:22.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:22.352+0000 7fc1c9e00700 1 --2- 172.21.15.37:0/2991675378 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc1cc1037a0 0x7fc1cc114040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:17:22.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:22.352+0000 7fc1c9e00700 1 -- 172.21.15.37:0/2991675378 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1b8005040 con 0x7fc1cc102e00 2024-03-29T08:17:22.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:22.352+0000 7fc1c9e00700 1 --2- 172.21.15.37:0/2991675378 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc1cc102e00 0x7fc1cc113b00 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fc1bc002a50 tx=0x7fc1bc0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:17:22.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:22.353+0000 7fc1c2000700 1 -- 172.21.15.37:0/2991675378 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc1bc007cc0 con 0x7fc1cc102e00 2024-03-29T08:17:22.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:22.353+0000 7fc1c2000700 1 -- 172.21.15.37:0/2991675378 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fc1bc00f040 con 0x7fc1cc102e00 2024-03-29T08:17:22.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:22.353+0000 7fc1d1000700 1 -- 172.21.15.37:0/2991675378 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1cc10dd60 con 0x7fc1cc102e00 2024-03-29T08:17:22.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:22.353+0000 7fc1c2000700 1 -- 172.21.15.37:0/2991675378 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc1bc009540 con 0x7fc1cc102e00 2024-03-29T08:17:22.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:22.353+0000 7fc1d1000700 1 -- 172.21.15.37:0/2991675378 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1cc10e2f0 con 0x7fc1cc102e00 2024-03-29T08:17:22.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:22.354+0000 7fc1d1000700 1 -- 172.21.15.37:0/2991675378 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1cc0fe550 con 0x7fc1cc102e00 2024-03-29T08:17:22.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:22.355+0000 7fc1c2000700 1 -- 172.21.15.37:0/2991675378 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fc1bc0096a0 con 0x7fc1cc102e00 2024-03-29T08:17:22.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:22.356+0000 7fc1c2000700 1 --2- 172.21.15.37:0/2991675378 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fc1b006cc30 0x7fc1b006f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:17:22.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:22.356+0000 7fc1c2000700 1 -- 172.21.15.37:0/2991675378 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fc1bc08bee0 con 0x7fc1cc102e00 2024-03-29T08:17:22.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:22.356+0000 7fc1c9400700 1 --2- 172.21.15.37:0/2991675378 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fc1b006cc30 0x7fc1b006f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:17:22.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:22.357+0000 7fc1c9400700 1 --2- 172.21.15.37:0/2991675378 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fc1b006cc30 0x7fc1b006f0e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fc1cc10ed60 tx=0x7fc1b8009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:17:22.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:22.361+0000 7fc1c2000700 1 -- 172.21.15.37:0/2991675378 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fc1bc0598b0 con 0x7fc1cc102e00 2024-03-29T08:17:22.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:22.593+0000 7fc1d1000700 1 -- 172.21.15.37:0/2991675378 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc1cc109df0 con 0x7fc1b006cc30 2024-03-29T08:17:22.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:22.594+0000 7fc1c2000700 1 -- 172.21.15.37:0/2991675378 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fc1cc109df0 con 0x7fc1b006cc30 2024-03-29T08:17:22.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:22.596+0000 7fc1d1000700 1 -- 172.21.15.37:0/2991675378 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fc1b006cc30 msgr2=0x7fc1b006f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:17:22.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:22.596+0000 7fc1d1000700 1 --2- 172.21.15.37:0/2991675378 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fc1b006cc30 0x7fc1b006f0e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fc1cc10ed60 tx=0x7fc1b8009e30 comp rx=0 tx=0).stop 2024-03-29T08:17:22.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:22.596+0000 7fc1d1000700 1 -- 172.21.15.37:0/2991675378 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc1cc102e00 msgr2=0x7fc1cc113b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:17:22.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:22.596+0000 7fc1d1000700 1 --2- 172.21.15.37:0/2991675378 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc1cc102e00 0x7fc1cc113b00 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fc1bc002a50 tx=0x7fc1bc0046b0 comp rx=0 tx=0).stop 2024-03-29T08:17:22.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:22.597+0000 7fc1d1000700 1 -- 172.21.15.37:0/2991675378 shutdown_connections 2024-03-29T08:17:22.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:22.597+0000 7fc1d1000700 1 --2- 172.21.15.37:0/2991675378 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fc1b006cc30 0x7fc1b006f0e0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:17:22.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:22.597+0000 7fc1d1000700 1 --2- 172.21.15.37:0/2991675378 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc1cc102e00 0x7fc1cc113b00 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:17:22.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:22.597+0000 7fc1d1000700 1 --2- 172.21.15.37:0/2991675378 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc1cc1037a0 0x7fc1cc114040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:17:22.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:22.597+0000 7fc1d1000700 1 -- 172.21.15.37:0/2991675378 >> 172.21.15.37:0/2991675378 conn(0x7fc1cc0f89b0 msgr2=0x7fc1cc0f9ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:17:22.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:22.597+0000 7fc1d1000700 1 -- 172.21.15.37:0/2991675378 shutdown_connections 2024-03-29T08:17:22.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:22.597+0000 7fc1d1000700 1 -- 172.21.15.37:0/2991675378 wait complete. 2024-03-29T08:17:22.611 INFO:teuthology.orchestra.run.smithi037.stdout:true 2024-03-29T08:17:22.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:22.734+0000 7f0a88800700 1 -- 172.21.15.37:0/3869637532 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0a801037a0 msgr2=0x7f0a800fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:17:22.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:22.734+0000 7f0a88800700 1 --2- 172.21.15.37:0/3869637532 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0a801037a0 0x7f0a800fce10 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f0a70004940 tx=0x7f0a70009e30 comp rx=0 tx=0).stop 2024-03-29T08:17:22.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:22.734+0000 7f0a88800700 1 -- 172.21.15.37:0/3869637532 shutdown_connections 2024-03-29T08:17:22.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:22.734+0000 7f0a88800700 1 --2- 172.21.15.37:0/3869637532 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0a801037a0 0x7f0a800fce10 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:17:22.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:22.734+0000 7f0a88800700 1 --2- 172.21.15.37:0/3869637532 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0a80102e00 0x7f0a801031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:17:22.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:22.734+0000 7f0a88800700 1 -- 172.21.15.37:0/3869637532 >> 172.21.15.37:0/3869637532 conn(0x7f0a800f89b0 msgr2=0x7f0a800fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:17:22.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:22.735+0000 7f0a88800700 1 -- 172.21.15.37:0/3869637532 shutdown_connections 2024-03-29T08:17:22.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:22.735+0000 7f0a88800700 1 -- 172.21.15.37:0/3869637532 wait complete. 2024-03-29T08:17:22.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:22.735+0000 7f0a88800700 1 Processor -- start 2024-03-29T08:17:22.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:22.736+0000 7f0a88800700 1 -- start start 2024-03-29T08:17:22.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:22.736+0000 7f0a88800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0a80102e00 0x7f0a80115da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:17:22.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:22.736+0000 7f0a88800700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0a801037a0 0x7f0a801162e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:17:22.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:22.736+0000 7f0a88800700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a80116930 con 0x7f0a80102e00 2024-03-29T08:17:22.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:22.736+0000 7f0a88800700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a8010fd00 con 0x7f0a801037a0 2024-03-29T08:17:22.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:22.736+0000 7f0a85e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0a80102e00 0x7f0a80115da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:17:22.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:22.736+0000 7f0a85e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0a80102e00 0x7f0a80115da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34122/0 (socket says 172.21.15.37:34122) 2024-03-29T08:17:22.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:22.736+0000 7f0a85e00700 1 -- 172.21.15.37:0/1741122561 learned_addr learned my addr 172.21.15.37:0/1741122561 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:17:22.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:22.736+0000 7f0a85400700 1 --2- 172.21.15.37:0/1741122561 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0a801037a0 0x7f0a801162e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:17:22.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:22.737+0000 7f0a85e00700 1 -- 172.21.15.37:0/1741122561 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0a801037a0 msgr2=0x7f0a801162e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:17:22.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:22.737+0000 7f0a85e00700 1 --2- 172.21.15.37:0/1741122561 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0a801037a0 0x7f0a801162e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:17:22.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:22.737+0000 7f0a85e00700 1 -- 172.21.15.37:0/1741122561 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a70005040 con 0x7f0a80102e00 2024-03-29T08:17:22.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:22.737+0000 7f0a85400700 1 --2- 172.21.15.37:0/1741122561 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0a801037a0 0x7f0a801162e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:17:22.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:22.737+0000 7f0a85e00700 1 --2- 172.21.15.37:0/1741122561 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0a80102e00 0x7f0a80115da0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f0a7400c8c0 tx=0x7f0a740044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:17:22.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:22.737+0000 7f0a7a000700 1 -- 172.21.15.37:0/1741122561 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0a7400a980 con 0x7f0a80102e00 2024-03-29T08:17:22.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:22.737+0000 7f0a7a000700 1 -- 172.21.15.37:0/1741122561 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f0a7400ce60 con 0x7f0a80102e00 2024-03-29T08:17:22.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:22.737+0000 7f0a88800700 1 -- 172.21.15.37:0/1741122561 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a8010ffe0 con 0x7f0a80102e00 2024-03-29T08:17:22.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:22.737+0000 7f0a7a000700 1 -- 172.21.15.37:0/1741122561 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0a740113a0 con 0x7f0a80102e00 2024-03-29T08:17:22.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:22.737+0000 7f0a88800700 1 -- 172.21.15.37:0/1741122561 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a80110520 con 0x7f0a80102e00 2024-03-29T08:17:22.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:22.738+0000 7f0a88800700 1 -- 172.21.15.37:0/1741122561 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a800fe550 con 0x7f0a80102e00 2024-03-29T08:17:22.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:22.739+0000 7f0a7a000700 1 -- 172.21.15.37:0/1741122561 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f0a74012400 con 0x7f0a80102e00 2024-03-29T08:17:22.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:22.741+0000 7f0a7a000700 1 --2- 172.21.15.37:0/1741122561 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f0a6806cc20 0x7f0a6806f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:17:22.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:22.741+0000 7f0a7a000700 1 -- 172.21.15.37:0/1741122561 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f0a7408c150 con 0x7f0a80102e00 2024-03-29T08:17:22.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:22.741+0000 7f0a85400700 1 --2- 172.21.15.37:0/1741122561 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f0a6806cc20 0x7f0a6806f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:17:22.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:22.742+0000 7f0a85400700 1 --2- 172.21.15.37:0/1741122561 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f0a6806cc20 0x7f0a6806f0d0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f0a70009fd0 tx=0x7f0a70009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:17:22.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:22.745+0000 7f0a7a000700 1 -- 172.21.15.37:0/1741122561 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f0a74059c20 con 0x7f0a80102e00 2024-03-29T08:17:22.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:22.980+0000 7f0a88800700 1 -- 172.21.15.37:0/1741122561 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0a8004bed0 con 0x7f0a6806cc20 2024-03-29T08:17:22.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:22.981+0000 7f0a7a000700 1 -- 172.21.15.37:0/1741122561 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f0a8004bed0 con 0x7f0a6806cc20 2024-03-29T08:17:22.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:22.984+0000 7f0a88800700 1 -- 172.21.15.37:0/1741122561 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f0a6806cc20 msgr2=0x7f0a6806f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:17:22.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:22.984+0000 7f0a88800700 1 --2- 172.21.15.37:0/1741122561 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f0a6806cc20 0x7f0a6806f0d0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f0a70009fd0 tx=0x7f0a70009e30 comp rx=0 tx=0).stop 2024-03-29T08:17:22.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:22.984+0000 7f0a88800700 1 -- 172.21.15.37:0/1741122561 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0a80102e00 msgr2=0x7f0a80115da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:17:22.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:22.984+0000 7f0a88800700 1 --2- 172.21.15.37:0/1741122561 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0a80102e00 0x7f0a80115da0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f0a7400c8c0 tx=0x7f0a740044c0 comp rx=0 tx=0).stop 2024-03-29T08:17:22.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:22.984+0000 7f0a88800700 1 -- 172.21.15.37:0/1741122561 shutdown_connections 2024-03-29T08:17:22.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:22.984+0000 7f0a88800700 1 --2- 172.21.15.37:0/1741122561 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f0a6806cc20 0x7f0a6806f0d0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:17:22.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:22.984+0000 7f0a88800700 1 --2- 172.21.15.37:0/1741122561 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0a80102e00 0x7f0a80115da0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:17:22.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:22.984+0000 7f0a88800700 1 --2- 172.21.15.37:0/1741122561 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0a801037a0 0x7f0a801162e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:17:22.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:22.984+0000 7f0a88800700 1 -- 172.21.15.37:0/1741122561 >> 172.21.15.37:0/1741122561 conn(0x7f0a800f89b0 msgr2=0x7f0a800f9a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:17:22.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:22.984+0000 7f0a88800700 1 -- 172.21.15.37:0/1741122561 shutdown_connections 2024-03-29T08:17:22.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:22.984+0000 7f0a88800700 1 -- 172.21.15.37:0/1741122561 wait complete. 2024-03-29T08:17:23.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:23.121+0000 7f76ecc00700 1 -- 172.21.15.37:0/3744199225 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f76e81059a0 msgr2=0x7f76e80ff010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:17:23.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:23.121+0000 7f76ecc00700 1 --2- 172.21.15.37:0/3744199225 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f76e81059a0 0x7f76e80ff010 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f76d4004940 tx=0x7f76d4009e30 comp rx=0 tx=0).stop 2024-03-29T08:17:23.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:23.121+0000 7f76ecc00700 1 -- 172.21.15.37:0/3744199225 shutdown_connections 2024-03-29T08:17:23.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:23.121+0000 7f76ecc00700 1 --2- 172.21.15.37:0/3744199225 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f76e81059a0 0x7f76e80ff010 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:17:23.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:23.121+0000 7f76ecc00700 1 --2- 172.21.15.37:0/3744199225 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f76e8105000 0x7f76e81053d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:17:23.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:23.121+0000 7f76ecc00700 1 -- 172.21.15.37:0/3744199225 >> 172.21.15.37:0/3744199225 conn(0x7f76e80fabd0 msgr2=0x7f76e80fcfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:17:23.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:23.121+0000 7f76ecc00700 1 -- 172.21.15.37:0/3744199225 shutdown_connections 2024-03-29T08:17:23.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:23.121+0000 7f76ecc00700 1 -- 172.21.15.37:0/3744199225 wait complete. 2024-03-29T08:17:23.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:23.122+0000 7f76ecc00700 1 Processor -- start 2024-03-29T08:17:23.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:23.122+0000 7f76ecc00700 1 -- start start 2024-03-29T08:17:23.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:23.123+0000 7f76ecc00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f76e8105000 0x7f76e8115cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:17:23.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:23.123+0000 7f76ecc00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f76e81059a0 0x7f76e8116210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:17:23.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:23.123+0000 7f76ecc00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76e8116860 con 0x7f76e81059a0 2024-03-29T08:17:23.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:23.123+0000 7f76ecc00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76e810fc30 con 0x7f76e8105000 2024-03-29T08:17:23.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:23.123+0000 7f76e5400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f76e81059a0 0x7f76e8116210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:17:23.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:23.123+0000 7f76e5400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f76e81059a0 0x7f76e8116210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34146/0 (socket says 172.21.15.37:34146) 2024-03-29T08:17:23.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:23.123+0000 7f76e5400700 1 -- 172.21.15.37:0/3653416964 learned_addr learned my addr 172.21.15.37:0/3653416964 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:17:23.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:23.123+0000 7f76e5e00700 1 --2- 172.21.15.37:0/3653416964 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f76e8105000 0x7f76e8115cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:17:23.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:23.123+0000 7f76e5400700 1 -- 172.21.15.37:0/3653416964 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f76e8105000 msgr2=0x7f76e8115cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:17:23.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:23.123+0000 7f76e5400700 1 --2- 172.21.15.37:0/3653416964 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f76e8105000 0x7f76e8115cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:17:23.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:23.123+0000 7f76e5400700 1 -- 172.21.15.37:0/3653416964 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76d4005040 con 0x7f76e81059a0 2024-03-29T08:17:23.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:23.124+0000 7f76e5400700 1 --2- 172.21.15.37:0/3653416964 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f76e81059a0 0x7f76e8116210 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f76d4009390 tx=0x7f76d4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:17:23.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:23.124+0000 7f76de000700 1 -- 172.21.15.37:0/3653416964 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f76d400e040 con 0x7f76e81059a0 2024-03-29T08:17:23.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:23.124+0000 7f76ecc00700 1 -- 172.21.15.37:0/3653416964 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76e810feb0 con 0x7f76e81059a0 2024-03-29T08:17:23.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:23.124+0000 7f76de000700 1 -- 172.21.15.37:0/3653416964 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f76d4004290 con 0x7f76e81059a0 2024-03-29T08:17:23.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:23.124+0000 7f76de000700 1 -- 172.21.15.37:0/3653416964 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f76d4013e70 con 0x7f76e81059a0 2024-03-29T08:17:23.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:23.124+0000 7f76ecc00700 1 -- 172.21.15.37:0/3653416964 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76e8110470 con 0x7f76e81059a0 2024-03-29T08:17:23.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:23.125+0000 7f76ecc00700 1 -- 172.21.15.37:0/3653416964 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76e8102310 con 0x7f76e81059a0 2024-03-29T08:17:23.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:23.126+0000 7f76de000700 1 -- 172.21.15.37:0/3653416964 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f76d4013690 con 0x7f76e81059a0 2024-03-29T08:17:23.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:23.128+0000 7f76de000700 1 --2- 172.21.15.37:0/3653416964 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f76cc06cc30 0x7f76cc06f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:17:23.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:23.128+0000 7f76de000700 1 -- 172.21.15.37:0/3653416964 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f76d408d1a0 con 0x7f76e81059a0 2024-03-29T08:17:23.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:23.128+0000 7f76e5e00700 1 --2- 172.21.15.37:0/3653416964 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f76cc06cc30 0x7f76cc06f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:17:23.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:23.128+0000 7f76e5e00700 1 --2- 172.21.15.37:0/3653416964 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f76cc06cc30 0x7f76cc06f0e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f76d8004f40 tx=0x7f76d800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:17:23.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:23.132+0000 7f76de000700 1 -- 172.21.15.37:0/3653416964 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f76d405ad20 con 0x7f76e81059a0 2024-03-29T08:17:23.149 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:17:22 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi037.mmwgxe"}]: dispatch 2024-03-29T08:17:23.149 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:17:22 smithi037 ceph-mon[142325]: from='client.34412 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:17:23.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:17:22 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi037.mmwgxe"}]: dispatch 2024-03-29T08:17:23.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:17:22 smithi148 ceph-mon[106692]: from='client.34412 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:17:23.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:23.358+0000 7f76ecc00700 1 -- 172.21.15.37:0/3653416964 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f76e8111390 con 0x7f76cc06cc30 2024-03-29T08:17:23.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:23.368+0000 7f76de000700 1 -- 172.21.15.37:0/3653416964 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f76e8111390 con 0x7f76cc06cc30 2024-03-29T08:17:23.369 INFO:teuthology.orchestra.run.smithi037.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-29T08:17:23.369 INFO:teuthology.orchestra.run.smithi037.stdout:alertmanager.smithi037 smithi037 *:9093,9094 running (7m) 2m ago 20m 14.1M - 0.25.0 c8568f914cd2 2f9f9746ed0f 2024-03-29T08:17:23.369 INFO:teuthology.orchestra.run.smithi037.stdout:crash.smithi037 smithi037 running (4m) 2m ago 20m 6647k - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 178d416a3186 2024-03-29T08:17:23.370 INFO:teuthology.orchestra.run.smithi037.stdout:crash.smithi148 smithi148 running (4m) 14s ago 19m 6656k - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 23666989d58f 2024-03-29T08:17:23.370 INFO:teuthology.orchestra.run.smithi037.stdout:grafana.smithi037 smithi037 *:3000 running (6m) 2m ago 19m 73.4M - 9.4.7 954c08fa6188 6b976fe49952 2024-03-29T08:17:23.370 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi037.iifsmh smithi037 running (13m) 2m ago 13m 110M - 17.2.7-789-g4213e17c cf22eb8683be 074e1acf331f 2024-03-29T08:17:23.370 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi037.mmwgxe smithi037 running (13m) 2m ago 13m 114M - 17.2.7-789-g4213e17c cf22eb8683be 9a5b1e72de5c 2024-03-29T08:17:23.370 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi148.qukpjh smithi148 running (13m) 14s ago 13m 15.0M - 17.2.7-789-g4213e17c cf22eb8683be 8e469782a273 2024-03-29T08:17:23.370 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi148.sqteih smithi148 running (13m) 14s ago 13m 15.0M - 17.2.7-789-g4213e17c cf22eb8683be 8e68f858bb22 2024-03-29T08:17:23.370 INFO:teuthology.orchestra.run.smithi037.stdout:mgr.smithi037.wjikii smithi037 *:8443,9283,8765 running (9m) 2m ago 21m 501M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff c5a1417b9417 2024-03-29T08:17:23.370 INFO:teuthology.orchestra.run.smithi037.stdout:mgr.smithi148.onhgvm smithi148 *:8443,9283,8765 running (8m) 14s ago 19m 450M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff fcfdbce062bd 2024-03-29T08:17:23.370 INFO:teuthology.orchestra.run.smithi037.stdout:mon.smithi037 smithi037 running (5m) 2m ago 21m 47.5M 2048M 18.2.2-1015-gd0e392d9 1cc2f45c17ff eea2060783af 2024-03-29T08:17:23.370 INFO:teuthology.orchestra.run.smithi037.stdout:mon.smithi148 smithi148 running (4m) 14s ago 18m 40.3M 2048M 18.2.2-1015-gd0e392d9 1cc2f45c17ff ad4b2c5748be 2024-03-29T08:17:23.370 INFO:teuthology.orchestra.run.smithi037.stdout:node-exporter.smithi037 smithi037 *:9100 running (8m) 2m ago 20m 7847k - 1.5.0 0da6a335fe13 26b8ae9cb878 2024-03-29T08:17:23.370 INFO:teuthology.orchestra.run.smithi037.stdout:node-exporter.smithi148 smithi148 *:9100 running (7m) 14s ago 19m 7528k - 1.5.0 0da6a335fe13 98414e4e64ed 2024-03-29T08:17:23.370 INFO:teuthology.orchestra.run.smithi037.stdout:osd.0 smithi037 running (3m) 2m ago 18m 99.0M 1619M 18.2.2-1015-gd0e392d9 1cc2f45c17ff c0c584f6245c 2024-03-29T08:17:23.370 INFO:teuthology.orchestra.run.smithi037.stdout:osd.1 smithi037 running (2m) 2m ago 17m 97.2M 1619M 18.2.2-1015-gd0e392d9 1cc2f45c17ff a4a2ff4cc83d 2024-03-29T08:17:23.370 INFO:teuthology.orchestra.run.smithi037.stdout:osd.2 smithi037 running (2m) 2m ago 17m 11.0M 1619M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 8b1bc71aa79e 2024-03-29T08:17:23.371 INFO:teuthology.orchestra.run.smithi037.stdout:osd.3 smithi148 running (95s) 14s ago 16m 104M 2642M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 1c656ab8ff86 2024-03-29T08:17:23.371 INFO:teuthology.orchestra.run.smithi037.stdout:osd.4 smithi148 running (57s) 14s ago 15m 96.4M 2642M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 966a9de11d22 2024-03-29T08:17:23.371 INFO:teuthology.orchestra.run.smithi037.stdout:osd.5 smithi148 running (19s) 14s ago 15m 11.0M 2642M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 474c4f9212ce 2024-03-29T08:17:23.371 INFO:teuthology.orchestra.run.smithi037.stdout:prometheus.smithi037 smithi037 *:9095 running (7m) 2m ago 19m 52.1M - 2.43.0 a07b618ecd1d 30c0223ec589 2024-03-29T08:17:23.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:23.370+0000 7f76ecc00700 1 -- 172.21.15.37:0/3653416964 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f76cc06cc30 msgr2=0x7f76cc06f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:17:23.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:23.370+0000 7f76ecc00700 1 --2- 172.21.15.37:0/3653416964 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f76cc06cc30 0x7f76cc06f0e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f76d8004f40 tx=0x7f76d800b040 comp rx=0 tx=0).stop 2024-03-29T08:17:23.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:23.370+0000 7f76ecc00700 1 -- 172.21.15.37:0/3653416964 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f76e81059a0 msgr2=0x7f76e8116210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:17:23.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:23.370+0000 7f76ecc00700 1 --2- 172.21.15.37:0/3653416964 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f76e81059a0 0x7f76e8116210 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f76d4009390 tx=0x7f76d4009e30 comp rx=0 tx=0).stop 2024-03-29T08:17:23.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:23.370+0000 7f76ecc00700 1 -- 172.21.15.37:0/3653416964 shutdown_connections 2024-03-29T08:17:23.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:23.370+0000 7f76ecc00700 1 --2- 172.21.15.37:0/3653416964 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f76cc06cc30 0x7f76cc06f0e0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:17:23.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:23.370+0000 7f76ecc00700 1 --2- 172.21.15.37:0/3653416964 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f76e8105000 0x7f76e8115cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:17:23.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:23.370+0000 7f76ecc00700 1 --2- 172.21.15.37:0/3653416964 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f76e81059a0 0x7f76e8116210 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:17:23.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:23.370+0000 7f76ecc00700 1 -- 172.21.15.37:0/3653416964 >> 172.21.15.37:0/3653416964 conn(0x7f76e80fabd0 msgr2=0x7f76e80fcf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:17:23.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:23.371+0000 7f76ecc00700 1 -- 172.21.15.37:0/3653416964 shutdown_connections 2024-03-29T08:17:23.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:23.371+0000 7f76ecc00700 1 -- 172.21.15.37:0/3653416964 wait complete. 2024-03-29T08:17:23.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:23.507+0000 7f4292e00700 1 -- 172.21.15.37:0/4056312830 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f428c105800 msgr2=0x7f428c0fee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:17:23.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:23.507+0000 7f4292e00700 1 --2- 172.21.15.37:0/4056312830 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f428c105800 0x7f428c0fee70 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f4278004940 tx=0x7f4278009e30 comp rx=0 tx=0).stop 2024-03-29T08:17:23.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:23.507+0000 7f4292e00700 1 -- 172.21.15.37:0/4056312830 shutdown_connections 2024-03-29T08:17:23.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:23.507+0000 7f4292e00700 1 --2- 172.21.15.37:0/4056312830 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f428c105800 0x7f428c0fee70 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:17:23.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:23.507+0000 7f4292e00700 1 --2- 172.21.15.37:0/4056312830 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f428c104e60 0x7f428c105230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:17:23.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:23.507+0000 7f4292e00700 1 -- 172.21.15.37:0/4056312830 >> 172.21.15.37:0/4056312830 conn(0x7f428c0fab10 msgr2=0x7f428c0fcef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:17:23.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:23.507+0000 7f4292e00700 1 -- 172.21.15.37:0/4056312830 shutdown_connections 2024-03-29T08:17:23.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:23.508+0000 7f4292e00700 1 -- 172.21.15.37:0/4056312830 wait complete. 2024-03-29T08:17:23.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:23.508+0000 7f4292e00700 1 Processor -- start 2024-03-29T08:17:23.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:23.508+0000 7f4292e00700 1 -- start start 2024-03-29T08:17:23.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:23.509+0000 7f4292e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f428c104e60 0x7f428c115ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:17:23.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:23.509+0000 7f4292e00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f428c105800 0x7f428c116010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:17:23.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:23.509+0000 7f4292e00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f428c116660 con 0x7f428c104e60 2024-03-29T08:17:23.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:23.509+0000 7f4292e00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f428c10fa50 con 0x7f428c105800 2024-03-29T08:17:23.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:23.509+0000 7f4291a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f428c104e60 0x7f428c115ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:17:23.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:23.509+0000 7f4291a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f428c104e60 0x7f428c115ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34162/0 (socket says 172.21.15.37:34162) 2024-03-29T08:17:23.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:23.509+0000 7f4291a00700 1 -- 172.21.15.37:0/3750176225 learned_addr learned my addr 172.21.15.37:0/3750176225 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:17:23.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:23.509+0000 7f4291000700 1 --2- 172.21.15.37:0/3750176225 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f428c105800 0x7f428c116010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:17:23.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:23.509+0000 7f4291a00700 1 -- 172.21.15.37:0/3750176225 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f428c105800 msgr2=0x7f428c116010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:17:23.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:23.509+0000 7f4291a00700 1 --2- 172.21.15.37:0/3750176225 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f428c105800 0x7f428c116010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:17:23.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:23.510+0000 7f4291a00700 1 -- 172.21.15.37:0/3750176225 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4278005040 con 0x7f428c104e60 2024-03-29T08:17:23.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:23.510+0000 7f4291000700 1 --2- 172.21.15.37:0/3750176225 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f428c105800 0x7f428c116010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:17:23.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:23.510+0000 7f4291a00700 1 --2- 172.21.15.37:0/3750176225 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f428c104e60 0x7f428c115ad0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f428000eb40 tx=0x7f428000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:17:23.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:23.510+0000 7f4289600700 1 -- 172.21.15.37:0/3750176225 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4280007cc0 con 0x7f428c104e60 2024-03-29T08:17:23.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:23.510+0000 7f4289600700 1 -- 172.21.15.37:0/3750176225 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f428000f040 con 0x7f428c104e60 2024-03-29T08:17:23.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:23.510+0000 7f4292e00700 1 -- 172.21.15.37:0/3750176225 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f428c10fcd0 con 0x7f428c104e60 2024-03-29T08:17:23.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:23.510+0000 7f4289600700 1 -- 172.21.15.37:0/3750176225 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4280009570 con 0x7f428c104e60 2024-03-29T08:17:23.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:23.510+0000 7f4292e00700 1 -- 172.21.15.37:0/3750176225 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f428c110290 con 0x7f428c104e60 2024-03-29T08:17:23.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:23.511+0000 7f4292e00700 1 -- 172.21.15.37:0/3750176225 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f428c1005b0 con 0x7f428c104e60 2024-03-29T08:17:23.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:23.512+0000 7f4289600700 1 -- 172.21.15.37:0/3750176225 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f42800096d0 con 0x7f428c104e60 2024-03-29T08:17:23.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:23.513+0000 7f4289600700 1 --2- 172.21.15.37:0/3750176225 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f427406cc20 0x7f427406f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:17:23.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:23.513+0000 7f4289600700 1 -- 172.21.15.37:0/3750176225 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f428008bda0 con 0x7f428c104e60 2024-03-29T08:17:23.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:23.513+0000 7f4291000700 1 --2- 172.21.15.37:0/3750176225 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f427406cc20 0x7f427406f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:17:23.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:23.514+0000 7f4291000700 1 --2- 172.21.15.37:0/3750176225 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f427406cc20 0x7f427406f0d0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f427800b0d0 tx=0x7f4278009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:17:23.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:23.517+0000 7f4289600700 1 -- 172.21.15.37:0/3750176225 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f4280059820 con 0x7f428c104e60 2024-03-29T08:17:23.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:23.818+0000 7f4292e00700 1 -- 172.21.15.37:0/3750176225 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f428c10be00 con 0x7f428c104e60 2024-03-29T08:17:23.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:23.818+0000 7f4289600700 1 -- 172.21.15.37:0/3750176225 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+741 (secure 0 0 0) 0x7f42800593b0 con 0x7f428c104e60 2024-03-29T08:17:23.820 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-29T08:17:23.820 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": { 2024-03-29T08:17:23.820 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 2 2024-03-29T08:17:23.820 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:17:23.820 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": { 2024-03-29T08:17:23.820 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 2 2024-03-29T08:17:23.820 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:17:23.820 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": { 2024-03-29T08:17:23.820 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 6 2024-03-29T08:17:23.820 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:17:23.820 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": { 2024-03-29T08:17:23.820 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 17.2.7-789-g4213e17c (4213e17c4de67016ed14f55d22503f4923b3a901) quincy (stable)": 3 2024-03-29T08:17:23.821 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:17:23.821 INFO:teuthology.orchestra.run.smithi037.stdout: "overall": { 2024-03-29T08:17:23.821 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 17.2.7-789-g4213e17c (4213e17c4de67016ed14f55d22503f4923b3a901) quincy (stable)": 3, 2024-03-29T08:17:23.821 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 10 2024-03-29T08:17:23.821 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-29T08:17:23.821 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-29T08:17:23.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:23.821+0000 7f4292e00700 1 -- 172.21.15.37:0/3750176225 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f427406cc20 msgr2=0x7f427406f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:17:23.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:23.821+0000 7f4292e00700 1 --2- 172.21.15.37:0/3750176225 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f427406cc20 0x7f427406f0d0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f427800b0d0 tx=0x7f4278009e30 comp rx=0 tx=0).stop 2024-03-29T08:17:23.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:23.821+0000 7f4292e00700 1 -- 172.21.15.37:0/3750176225 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f428c104e60 msgr2=0x7f428c115ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:17:23.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:23.821+0000 7f4292e00700 1 --2- 172.21.15.37:0/3750176225 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f428c104e60 0x7f428c115ad0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f428000eb40 tx=0x7f428000b040 comp rx=0 tx=0).stop 2024-03-29T08:17:23.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:23.821+0000 7f4292e00700 1 -- 172.21.15.37:0/3750176225 shutdown_connections 2024-03-29T08:17:23.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:23.821+0000 7f4292e00700 1 --2- 172.21.15.37:0/3750176225 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f427406cc20 0x7f427406f0d0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:17:23.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:23.821+0000 7f4292e00700 1 --2- 172.21.15.37:0/3750176225 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f428c104e60 0x7f428c115ad0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:17:23.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:23.821+0000 7f4292e00700 1 --2- 172.21.15.37:0/3750176225 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f428c105800 0x7f428c116010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:17:23.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:23.821+0000 7f4292e00700 1 -- 172.21.15.37:0/3750176225 >> 172.21.15.37:0/3750176225 conn(0x7f428c0fab10 msgr2=0x7f428c0fcae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:17:23.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:23.821+0000 7f4292e00700 1 -- 172.21.15.37:0/3750176225 shutdown_connections 2024-03-29T08:17:23.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:23.821+0000 7f4292e00700 1 -- 172.21.15.37:0/3750176225 wait complete. 2024-03-29T08:17:23.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:23.957+0000 7f581b800700 1 -- 172.21.15.37:0/4211980356 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5814102e00 msgr2=0x7f58141031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:17:23.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:23.957+0000 7f581b800700 1 --2- 172.21.15.37:0/4211980356 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5814102e00 0x7f58141031d0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f5808004940 tx=0x7f5808009e30 comp rx=0 tx=0).stop 2024-03-29T08:17:23.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:23.957+0000 7f581b800700 1 -- 172.21.15.37:0/4211980356 shutdown_connections 2024-03-29T08:17:23.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:23.957+0000 7f581b800700 1 --2- 172.21.15.37:0/4211980356 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f58141037a0 0x7f58140fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:17:23.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:23.957+0000 7f581b800700 1 --2- 172.21.15.37:0/4211980356 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5814102e00 0x7f58141031d0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:17:23.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:23.957+0000 7f581b800700 1 -- 172.21.15.37:0/4211980356 >> 172.21.15.37:0/4211980356 conn(0x7f58140f89b0 msgr2=0x7f58140fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:17:23.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:23.957+0000 7f581b800700 1 -- 172.21.15.37:0/4211980356 shutdown_connections 2024-03-29T08:17:23.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:23.957+0000 7f581b800700 1 -- 172.21.15.37:0/4211980356 wait complete. 2024-03-29T08:17:23.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:23.958+0000 7f581b800700 1 Processor -- start 2024-03-29T08:17:23.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:23.958+0000 7f581b800700 1 -- start start 2024-03-29T08:17:23.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:23.958+0000 7f581b800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5814102e00 0x7f5814115cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:17:23.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:23.958+0000 7f581b800700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f58141037a0 0x7f5814116230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:17:23.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:23.958+0000 7f581b800700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5814116880 con 0x7f5814102e00 2024-03-29T08:17:23.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:23.958+0000 7f581b800700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f581410fc50 con 0x7f58141037a0 2024-03-29T08:17:23.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:23.959+0000 7f5818e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5814102e00 0x7f5814115cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:17:23.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:23.959+0000 7f5818e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5814102e00 0x7f5814115cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34182/0 (socket says 172.21.15.37:34182) 2024-03-29T08:17:23.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:23.959+0000 7f5818e00700 1 -- 172.21.15.37:0/3570990883 learned_addr learned my addr 172.21.15.37:0/3570990883 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:17:23.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:23.959+0000 7f5813e00700 1 --2- 172.21.15.37:0/3570990883 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f58141037a0 0x7f5814116230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:17:23.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:23.959+0000 7f5818e00700 1 -- 172.21.15.37:0/3570990883 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f58141037a0 msgr2=0x7f5814116230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:17:23.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:23.959+0000 7f5818e00700 1 --2- 172.21.15.37:0/3570990883 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f58141037a0 0x7f5814116230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:17:23.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:23.959+0000 7f5818e00700 1 -- 172.21.15.37:0/3570990883 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5808005040 con 0x7f5814102e00 2024-03-29T08:17:23.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:23.959+0000 7f5818e00700 1 --2- 172.21.15.37:0/3570990883 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5814102e00 0x7f5814115cf0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f5808009360 tx=0x7f5808009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:17:23.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:23.960+0000 7f5810c00700 1 -- 172.21.15.37:0/3570990883 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f580800e040 con 0x7f5814102e00 2024-03-29T08:17:23.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:23.960+0000 7f5810c00700 1 -- 172.21.15.37:0/3570990883 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f5808013420 con 0x7f5814102e00 2024-03-29T08:17:23.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:23.960+0000 7f581b800700 1 -- 172.21.15.37:0/3570990883 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f581410fed0 con 0x7f5814102e00 2024-03-29T08:17:23.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:23.960+0000 7f5810c00700 1 -- 172.21.15.37:0/3570990883 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f580801c5d0 con 0x7f5814102e00 2024-03-29T08:17:23.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:23.960+0000 7f581b800700 1 -- 172.21.15.37:0/3570990883 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5814110490 con 0x7f5814102e00 2024-03-29T08:17:23.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:23.961+0000 7f581b800700 1 -- 172.21.15.37:0/3570990883 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58140fe550 con 0x7f5814102e00 2024-03-29T08:17:23.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:23.962+0000 7f5810c00700 1 -- 172.21.15.37:0/3570990883 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f5808013590 con 0x7f5814102e00 2024-03-29T08:17:23.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:23.963+0000 7f5810c00700 1 --2- 172.21.15.37:0/3570990883 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f57fc06cc30 0x7f57fc06f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:17:23.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:23.963+0000 7f5810c00700 1 -- 172.21.15.37:0/3570990883 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f580808d250 con 0x7f5814102e00 2024-03-29T08:17:23.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:23.963+0000 7f5813e00700 1 --2- 172.21.15.37:0/3570990883 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f57fc06cc30 0x7f57fc06f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:17:23.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:23.964+0000 7f5813e00700 1 --2- 172.21.15.37:0/3570990883 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f57fc06cc30 0x7f57fc06f0e0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f5814111590 tx=0x7f580000b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:17:23.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:23.967+0000 7f5810c00700 1 -- 172.21.15.37:0/3570990883 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f580805ac20 con 0x7f5814102e00 2024-03-29T08:17:24.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:17:23 smithi037 ceph-mon[142325]: from='client.34416 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:17:24.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:17:23 smithi037 ceph-mon[142325]: from='client.34420 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:17:24.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:17:23 smithi037 ceph-mon[142325]: pgmap v193: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:17:24.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:17:23 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/3750176225' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:17:24.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:24.226+0000 7f581b800700 1 -- 172.21.15.37:0/3570990883 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f581404bed0 con 0x7f5814102e00 2024-03-29T08:17:24.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:24.226+0000 7f5810c00700 1 -- 172.21.15.37:0/3570990883 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 23 v23) v1 ==== 76+0+1665 (secure 0 0 0) 0x7f580805a7b0 con 0x7f5814102e00 2024-03-29T08:17:24.228 INFO:teuthology.orchestra.run.smithi037.stdout:e23 2024-03-29T08:17:24.228 INFO:teuthology.orchestra.run.smithi037.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-29T08:17:24.228 INFO:teuthology.orchestra.run.smithi037.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T08:17:24.228 INFO:teuthology.orchestra.run.smithi037.stdout:legacy client fscid: 1 2024-03-29T08:17:24.228 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:17:24.228 INFO:teuthology.orchestra.run.smithi037.stdout:Filesystem 'cephfs' (1) 2024-03-29T08:17:24.228 INFO:teuthology.orchestra.run.smithi037.stdout:fs_name cephfs 2024-03-29T08:17:24.228 INFO:teuthology.orchestra.run.smithi037.stdout:epoch 21 2024-03-29T08:17:24.228 INFO:teuthology.orchestra.run.smithi037.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-03-29T08:17:24.228 INFO:teuthology.orchestra.run.smithi037.stdout:created 2024-03-29T08:03:29.241269+0000 2024-03-29T08:17:24.228 INFO:teuthology.orchestra.run.smithi037.stdout:modified 2024-03-29T08:17:16.237238+0000 2024-03-29T08:17:24.228 INFO:teuthology.orchestra.run.smithi037.stdout:tableserver 0 2024-03-29T08:17:24.228 INFO:teuthology.orchestra.run.smithi037.stdout:root 0 2024-03-29T08:17:24.228 INFO:teuthology.orchestra.run.smithi037.stdout:session_timeout 60 2024-03-29T08:17:24.229 INFO:teuthology.orchestra.run.smithi037.stdout:session_autoclose 300 2024-03-29T08:17:24.229 INFO:teuthology.orchestra.run.smithi037.stdout:max_file_size 1099511627776 2024-03-29T08:17:24.229 INFO:teuthology.orchestra.run.smithi037.stdout:max_xattr_size 65536 2024-03-29T08:17:24.229 INFO:teuthology.orchestra.run.smithi037.stdout:required_client_features {} 2024-03-29T08:17:24.229 INFO:teuthology.orchestra.run.smithi037.stdout:last_failure 0 2024-03-29T08:17:24.229 INFO:teuthology.orchestra.run.smithi037.stdout:last_failure_osd_epoch 82 2024-03-29T08:17:24.229 INFO:teuthology.orchestra.run.smithi037.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T08:17:24.229 INFO:teuthology.orchestra.run.smithi037.stdout:max_mds 2 2024-03-29T08:17:24.229 INFO:teuthology.orchestra.run.smithi037.stdout:in 0,1 2024-03-29T08:17:24.229 INFO:teuthology.orchestra.run.smithi037.stdout:up {} 2024-03-29T08:17:24.229 INFO:teuthology.orchestra.run.smithi037.stdout:failed 0,1 2024-03-29T08:17:24.229 INFO:teuthology.orchestra.run.smithi037.stdout:damaged 2024-03-29T08:17:24.229 INFO:teuthology.orchestra.run.smithi037.stdout:stopped 2024-03-29T08:17:24.229 INFO:teuthology.orchestra.run.smithi037.stdout:data_pools [3] 2024-03-29T08:17:24.230 INFO:teuthology.orchestra.run.smithi037.stdout:metadata_pool 2 2024-03-29T08:17:24.230 INFO:teuthology.orchestra.run.smithi037.stdout:inline_data disabled 2024-03-29T08:17:24.230 INFO:teuthology.orchestra.run.smithi037.stdout:balancer 2024-03-29T08:17:24.230 INFO:teuthology.orchestra.run.smithi037.stdout:bal_rank_mask -1 2024-03-29T08:17:24.230 INFO:teuthology.orchestra.run.smithi037.stdout:standby_count_wanted 1 2024-03-29T08:17:24.230 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:17:24.230 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:17:24.230 INFO:teuthology.orchestra.run.smithi037.stdout:Standby daemons: 2024-03-29T08:17:24.230 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:17:24.230 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi148.qukpjh{-1:24319} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.148:6826/2641327912,v1:172.21.15.148:6827/2641327912] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:17:24.230 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi148.sqteih{-1:24327} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.148:6824/3252351726,v1:172.21.15.148:6825/3252351726] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:17:24.230 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi037.iifsmh{-1:34406} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.37:6826/3877453059,v1:172.21.15.37:6827/3877453059] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:17:24.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:24.229+0000 7f581b800700 1 -- 172.21.15.37:0/3570990883 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f57fc06cc30 msgr2=0x7f57fc06f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:17:24.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:24.229+0000 7f581b800700 1 --2- 172.21.15.37:0/3570990883 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f57fc06cc30 0x7f57fc06f0e0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f5814111590 tx=0x7f580000b040 comp rx=0 tx=0).stop 2024-03-29T08:17:24.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:24.229+0000 7f581b800700 1 -- 172.21.15.37:0/3570990883 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5814102e00 msgr2=0x7f5814115cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:17:24.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:24.229+0000 7f581b800700 1 --2- 172.21.15.37:0/3570990883 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5814102e00 0x7f5814115cf0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f5808009360 tx=0x7f5808009e30 comp rx=0 tx=0).stop 2024-03-29T08:17:24.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:24.229+0000 7f581b800700 1 -- 172.21.15.37:0/3570990883 shutdown_connections 2024-03-29T08:17:24.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:24.229+0000 7f581b800700 1 --2- 172.21.15.37:0/3570990883 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f57fc06cc30 0x7f57fc06f0e0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:17:24.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:24.229+0000 7f581b800700 1 --2- 172.21.15.37:0/3570990883 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5814102e00 0x7f5814115cf0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:17:24.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:24.229+0000 7f581b800700 1 --2- 172.21.15.37:0/3570990883 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f58141037a0 0x7f5814116230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:17:24.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:24.229+0000 7f581b800700 1 -- 172.21.15.37:0/3570990883 >> 172.21.15.37:0/3570990883 conn(0x7f58140f89b0 msgr2=0x7f58140706d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:17:24.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:24.229+0000 7f581b800700 1 -- 172.21.15.37:0/3570990883 shutdown_connections 2024-03-29T08:17:24.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:24.229+0000 7f581b800700 1 -- 172.21.15.37:0/3570990883 wait complete. 2024-03-29T08:17:24.232 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 23 2024-03-29T08:17:24.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:17:23 smithi148 ceph-mon[106692]: from='client.34416 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:17:24.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:17:23 smithi148 ceph-mon[106692]: from='client.34420 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:17:24.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:17:23 smithi148 ceph-mon[106692]: pgmap v193: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:17:24.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:17:23 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/3750176225' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:17:24.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:24.371+0000 7fd2b5a00700 1 -- 172.21.15.37:0/1789751960 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd2b0102e00 msgr2=0x7fd2b01031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:17:24.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:24.371+0000 7fd2b5a00700 1 --2- 172.21.15.37:0/1789751960 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd2b0102e00 0x7fd2b01031d0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fd2a0004940 tx=0x7fd2a0009e30 comp rx=0 tx=0).stop 2024-03-29T08:17:24.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:24.371+0000 7fd2b5a00700 1 -- 172.21.15.37:0/1789751960 shutdown_connections 2024-03-29T08:17:24.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:24.371+0000 7fd2b5a00700 1 --2- 172.21.15.37:0/1789751960 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd2b01037a0 0x7fd2b00fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:17:24.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:24.371+0000 7fd2b5a00700 1 --2- 172.21.15.37:0/1789751960 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd2b0102e00 0x7fd2b01031d0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:17:24.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:24.371+0000 7fd2b5a00700 1 -- 172.21.15.37:0/1789751960 >> 172.21.15.37:0/1789751960 conn(0x7fd2b00f89b0 msgr2=0x7fd2b00fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:17:24.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:24.371+0000 7fd2b5a00700 1 -- 172.21.15.37:0/1789751960 shutdown_connections 2024-03-29T08:17:24.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:24.371+0000 7fd2b5a00700 1 -- 172.21.15.37:0/1789751960 wait complete. 2024-03-29T08:17:24.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:24.372+0000 7fd2b5a00700 1 Processor -- start 2024-03-29T08:17:24.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:24.373+0000 7fd2b5a00700 1 -- start start 2024-03-29T08:17:24.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:24.373+0000 7fd2b5a00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd2b0102e00 0x7fd2b00702c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:17:24.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:24.373+0000 7fd2b5a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd2b01037a0 0x7fd2b0070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:17:24.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:24.373+0000 7fd2b5a00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2b0070d90 con 0x7fd2b01037a0 2024-03-29T08:17:24.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:24.373+0000 7fd2b5a00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2b0070ed0 con 0x7fd2b0102e00 2024-03-29T08:17:24.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:24.374+0000 7fd2ae000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd2b01037a0 0x7fd2b0070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:17:24.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:24.374+0000 7fd2ae000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd2b01037a0 0x7fd2b0070800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34202/0 (socket says 172.21.15.37:34202) 2024-03-29T08:17:24.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:24.374+0000 7fd2ae000700 1 -- 172.21.15.37:0/109333534 learned_addr learned my addr 172.21.15.37:0/109333534 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:17:24.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:24.374+0000 7fd2aea00700 1 --2- 172.21.15.37:0/109333534 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd2b0102e00 0x7fd2b00702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:17:24.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:24.374+0000 7fd2ae000700 1 -- 172.21.15.37:0/109333534 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd2b0102e00 msgr2=0x7fd2b00702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:17:24.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:24.374+0000 7fd2ae000700 1 --2- 172.21.15.37:0/109333534 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd2b0102e00 0x7fd2b00702c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:17:24.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:24.374+0000 7fd2ae000700 1 -- 172.21.15.37:0/109333534 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2a0005040 con 0x7fd2b01037a0 2024-03-29T08:17:24.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:24.374+0000 7fd2ae000700 1 --2- 172.21.15.37:0/109333534 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd2b01037a0 0x7fd2b0070800 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fd29c002a50 tx=0x7fd29c004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:17:24.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:24.375+0000 7fd2a6a00700 1 -- 172.21.15.37:0/109333534 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd29c007cc0 con 0x7fd2b01037a0 2024-03-29T08:17:24.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:24.375+0000 7fd2a6a00700 1 -- 172.21.15.37:0/109333534 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd29c00f040 con 0x7fd2b01037a0 2024-03-29T08:17:24.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:24.375+0000 7fd2b5a00700 1 -- 172.21.15.37:0/109333534 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2b0119e20 con 0x7fd2b01037a0 2024-03-29T08:17:24.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:24.375+0000 7fd2a6a00700 1 -- 172.21.15.37:0/109333534 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd29c00d920 con 0x7fd2b01037a0 2024-03-29T08:17:24.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:24.375+0000 7fd2b5a00700 1 -- 172.21.15.37:0/109333534 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2b011a2a0 con 0x7fd2b01037a0 2024-03-29T08:17:24.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:24.376+0000 7fd2b5a00700 1 -- 172.21.15.37:0/109333534 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2b00fe550 con 0x7fd2b01037a0 2024-03-29T08:17:24.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:24.377+0000 7fd2a6a00700 1 -- 172.21.15.37:0/109333534 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fd29c009680 con 0x7fd2b01037a0 2024-03-29T08:17:24.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:24.379+0000 7fd2a6a00700 1 --2- 172.21.15.37:0/109333534 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fd29406cc30 0x7fd29406f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:17:24.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:24.379+0000 7fd2a6a00700 1 -- 172.21.15.37:0/109333534 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fd29c08ba80 con 0x7fd2b01037a0 2024-03-29T08:17:24.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:24.379+0000 7fd2aea00700 1 --2- 172.21.15.37:0/109333534 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fd29406cc30 0x7fd29406f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:17:24.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:24.380+0000 7fd2aea00700 1 --2- 172.21.15.37:0/109333534 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fd29406cc30 0x7fd29406f0e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fd2a0009fd0 tx=0x7fd2a0009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:17:24.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:24.387+0000 7fd2a6a00700 1 -- 172.21.15.37:0/109333534 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fd29c055a40 con 0x7fd2b01037a0 2024-03-29T08:17:24.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:24.622+0000 7fd2b5a00700 1 -- 172.21.15.37:0/109333534 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd2b004bed0 con 0x7fd29406cc30 2024-03-29T08:17:24.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:24.624+0000 7fd2a6a00700 1 -- 172.21.15.37:0/109333534 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fd2b004bed0 con 0x7fd29406cc30 2024-03-29T08:17:24.625 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-29T08:17:24.625 INFO:teuthology.orchestra.run.smithi037.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59", 2024-03-29T08:17:24.625 INFO:teuthology.orchestra.run.smithi037.stdout: "in_progress": true, 2024-03-29T08:17:24.625 INFO:teuthology.orchestra.run.smithi037.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-29T08:17:24.625 INFO:teuthology.orchestra.run.smithi037.stdout: "services_complete": [ 2024-03-29T08:17:24.625 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr", 2024-03-29T08:17:24.625 INFO:teuthology.orchestra.run.smithi037.stdout: "mon", 2024-03-29T08:17:24.625 INFO:teuthology.orchestra.run.smithi037.stdout: "crash", 2024-03-29T08:17:24.626 INFO:teuthology.orchestra.run.smithi037.stdout: "osd" 2024-03-29T08:17:24.626 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-29T08:17:24.626 INFO:teuthology.orchestra.run.smithi037.stdout: "progress": "12/21 daemons upgraded", 2024-03-29T08:17:24.626 INFO:teuthology.orchestra.run.smithi037.stdout: "message": "Currently upgrading mds daemons", 2024-03-29T08:17:24.626 INFO:teuthology.orchestra.run.smithi037.stdout: "is_paused": false 2024-03-29T08:17:24.626 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-29T08:17:24.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:24.626+0000 7fd2b5a00700 1 -- 172.21.15.37:0/109333534 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fd29406cc30 msgr2=0x7fd29406f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:17:24.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:24.626+0000 7fd2b5a00700 1 --2- 172.21.15.37:0/109333534 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fd29406cc30 0x7fd29406f0e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fd2a0009fd0 tx=0x7fd2a0009e30 comp rx=0 tx=0).stop 2024-03-29T08:17:24.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:24.626+0000 7fd2b5a00700 1 -- 172.21.15.37:0/109333534 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd2b01037a0 msgr2=0x7fd2b0070800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:17:24.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:24.626+0000 7fd2b5a00700 1 --2- 172.21.15.37:0/109333534 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd2b01037a0 0x7fd2b0070800 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fd29c002a50 tx=0x7fd29c004600 comp rx=0 tx=0).stop 2024-03-29T08:17:24.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:24.626+0000 7fd2b5a00700 1 -- 172.21.15.37:0/109333534 shutdown_connections 2024-03-29T08:17:24.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:24.626+0000 7fd2b5a00700 1 --2- 172.21.15.37:0/109333534 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fd29406cc30 0x7fd29406f0e0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:17:24.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:24.626+0000 7fd2b5a00700 1 --2- 172.21.15.37:0/109333534 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd2b0102e00 0x7fd2b00702c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:17:24.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:24.626+0000 7fd2b5a00700 1 --2- 172.21.15.37:0/109333534 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd2b01037a0 0x7fd2b0070800 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:17:24.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:24.626+0000 7fd2b5a00700 1 -- 172.21.15.37:0/109333534 >> 172.21.15.37:0/109333534 conn(0x7fd2b00f89b0 msgr2=0x7fd2b00f9a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:17:24.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:24.627+0000 7fd2b5a00700 1 -- 172.21.15.37:0/109333534 shutdown_connections 2024-03-29T08:17:24.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:24.627+0000 7fd2b5a00700 1 -- 172.21.15.37:0/109333534 wait complete. 2024-03-29T08:17:24.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:24.765+0000 7f7efda00700 1 -- 172.21.15.37:0/1141013179 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ef8070360 msgr2=0x7f7ef8070730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:17:24.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:24.765+0000 7f7efda00700 1 --2- 172.21.15.37:0/1141013179 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ef8070360 0x7f7ef8070730 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f7ee8004970 tx=0x7f7ee8009e30 comp rx=0 tx=0).stop 2024-03-29T08:17:24.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:24.765+0000 7f7efda00700 1 -- 172.21.15.37:0/1141013179 shutdown_connections 2024-03-29T08:17:24.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:24.765+0000 7f7efda00700 1 --2- 172.21.15.37:0/1141013179 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7ef8070c70 0x7f7ef8109110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:17:24.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:24.765+0000 7f7efda00700 1 --2- 172.21.15.37:0/1141013179 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ef8070360 0x7f7ef8070730 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:17:24.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:24.765+0000 7f7efda00700 1 -- 172.21.15.37:0/1141013179 >> 172.21.15.37:0/1141013179 conn(0x7f7ef80f89b0 msgr2=0x7f7ef80fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:17:24.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:24.765+0000 7f7efda00700 1 -- 172.21.15.37:0/1141013179 shutdown_connections 2024-03-29T08:17:24.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:24.765+0000 7f7efda00700 1 -- 172.21.15.37:0/1141013179 wait complete. 2024-03-29T08:17:24.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:24.766+0000 7f7efda00700 1 Processor -- start 2024-03-29T08:17:24.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:24.766+0000 7f7efda00700 1 -- start start 2024-03-29T08:17:24.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:24.766+0000 7f7efda00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7ef8070360 0x7f7ef8115c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:17:24.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:24.766+0000 7f7efda00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ef8070c70 0x7f7ef81161c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:17:24.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:24.766+0000 7f7efda00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ef8116810 con 0x7f7ef8070c70 2024-03-29T08:17:24.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:24.766+0000 7f7efda00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ef810fbe0 con 0x7f7ef8070360 2024-03-29T08:17:24.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:24.766+0000 7f7ef6000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ef8070c70 0x7f7ef81161c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:17:24.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:24.767+0000 7f7ef6000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ef8070c70 0x7f7ef81161c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34222/0 (socket says 172.21.15.37:34222) 2024-03-29T08:17:24.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:24.767+0000 7f7ef6000700 1 -- 172.21.15.37:0/2163882973 learned_addr learned my addr 172.21.15.37:0/2163882973 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:17:24.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:24.767+0000 7f7ef6a00700 1 --2- 172.21.15.37:0/2163882973 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7ef8070360 0x7f7ef8115c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:17:24.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:24.767+0000 7f7ef6000700 1 -- 172.21.15.37:0/2163882973 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7ef8070360 msgr2=0x7f7ef8115c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:17:24.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:24.767+0000 7f7ef6000700 1 --2- 172.21.15.37:0/2163882973 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7ef8070360 0x7f7ef8115c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:17:24.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:24.767+0000 7f7ef6000700 1 -- 172.21.15.37:0/2163882973 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ee8005040 con 0x7f7ef8070c70 2024-03-29T08:17:24.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:24.767+0000 7f7ef6000700 1 --2- 172.21.15.37:0/2163882973 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ef8070c70 0x7f7ef81161c0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f7ee4002a80 tx=0x7f7ee40046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:17:24.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:24.767+0000 7f7eeea00700 1 -- 172.21.15.37:0/2163882973 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7ee4007cc0 con 0x7f7ef8070c70 2024-03-29T08:17:24.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:24.767+0000 7f7eeea00700 1 -- 172.21.15.37:0/2163882973 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f7ee400f040 con 0x7f7ef8070c70 2024-03-29T08:17:24.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:24.767+0000 7f7eeea00700 1 -- 172.21.15.37:0/2163882973 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7ee4009640 con 0x7f7ef8070c70 2024-03-29T08:17:24.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:24.767+0000 7f7efda00700 1 -- 172.21.15.37:0/2163882973 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ef810fec0 con 0x7f7ef8070c70 2024-03-29T08:17:24.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:24.768+0000 7f7efda00700 1 -- 172.21.15.37:0/2163882973 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ef8110400 con 0x7f7ef8070c70 2024-03-29T08:17:24.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:24.769+0000 7f7efda00700 1 -- 172.21.15.37:0/2163882973 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ef8103b90 con 0x7f7ef8070c70 2024-03-29T08:17:24.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:24.771+0000 7f7eeea00700 1 -- 172.21.15.37:0/2163882973 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f7ee40097a0 con 0x7f7ef8070c70 2024-03-29T08:17:24.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:24.771+0000 7f7eeea00700 1 --2- 172.21.15.37:0/2163882973 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f7edc06cc30 0x7f7edc06f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:17:24.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:24.771+0000 7f7eeea00700 1 -- 172.21.15.37:0/2163882973 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f7ee408b7e0 con 0x7f7ef8070c70 2024-03-29T08:17:24.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:24.771+0000 7f7ef6a00700 1 --2- 172.21.15.37:0/2163882973 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f7edc06cc30 0x7f7edc06f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:17:24.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:24.771+0000 7f7ef6a00700 1 --2- 172.21.15.37:0/2163882973 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f7edc06cc30 0x7f7edc06f0e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f7ee8004ee0 tx=0x7f7ee8009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:17:24.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:24.775+0000 7f7eeea00700 1 -- 172.21.15.37:0/2163882973 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f7ee4059230 con 0x7f7ef8070c70 2024-03-29T08:17:25.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:25.087+0000 7f7efda00700 1 -- 172.21.15.37:0/2163882973 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f7ef81112d0 con 0x7f7ef8070c70 2024-03-29T08:17:25.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:25.087+0000 7f7eeea00700 1 -- 172.21.15.37:0/2163882973 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7f7ee4058dc0 con 0x7f7ef8070c70 2024-03-29T08:17:25.088 INFO:teuthology.orchestra.run.smithi037.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-03-29T08:17:25.088 INFO:teuthology.orchestra.run.smithi037.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-03-29T08:17:25.088 INFO:teuthology.orchestra.run.smithi037.stdout: fs cephfs is degraded 2024-03-29T08:17:25.088 INFO:teuthology.orchestra.run.smithi037.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-03-29T08:17:25.088 INFO:teuthology.orchestra.run.smithi037.stdout: fs cephfs is offline because no MDS is active for it. 2024-03-29T08:17:25.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:25.090+0000 7f7efda00700 1 -- 172.21.15.37:0/2163882973 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f7edc06cc30 msgr2=0x7f7edc06f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:17:25.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:25.091+0000 7f7efda00700 1 --2- 172.21.15.37:0/2163882973 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f7edc06cc30 0x7f7edc06f0e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f7ee8004ee0 tx=0x7f7ee8009e30 comp rx=0 tx=0).stop 2024-03-29T08:17:25.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:25.091+0000 7f7efda00700 1 -- 172.21.15.37:0/2163882973 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ef8070c70 msgr2=0x7f7ef81161c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:17:25.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:25.091+0000 7f7efda00700 1 --2- 172.21.15.37:0/2163882973 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ef8070c70 0x7f7ef81161c0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f7ee4002a80 tx=0x7f7ee40046b0 comp rx=0 tx=0).stop 2024-03-29T08:17:25.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:25.091+0000 7f7efda00700 1 -- 172.21.15.37:0/2163882973 shutdown_connections 2024-03-29T08:17:25.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:25.091+0000 7f7efda00700 1 --2- 172.21.15.37:0/2163882973 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f7edc06cc30 0x7f7edc06f0e0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:17:25.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:25.091+0000 7f7efda00700 1 --2- 172.21.15.37:0/2163882973 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7ef8070360 0x7f7ef8115c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:17:25.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:25.091+0000 7f7efda00700 1 --2- 172.21.15.37:0/2163882973 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ef8070c70 0x7f7ef81161c0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:17:25.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:25.091+0000 7f7efda00700 1 -- 172.21.15.37:0/2163882973 >> 172.21.15.37:0/2163882973 conn(0x7f7ef80f89b0 msgr2=0x7f7ef81015f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:17:25.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:25.091+0000 7f7efda00700 1 -- 172.21.15.37:0/2163882973 shutdown_connections 2024-03-29T08:17:25.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:25.091+0000 7f7efda00700 1 -- 172.21.15.37:0/2163882973 wait complete. 2024-03-29T08:17:25.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:17:25 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/3570990883' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T08:17:25.428 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:17:25 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:17:25.428 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:17:25 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:17:25.428 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:17:25 smithi037 ceph-mon[142325]: from='client.34432 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:17:25.428 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:17:25 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/2163882973' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T08:17:25.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:17:25 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/3570990883' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T08:17:25.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:17:25 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:17:25.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:17:25 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:17:25.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:17:25 smithi148 ceph-mon[106692]: from='client.34432 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:17:25.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:17:25 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/2163882973' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T08:17:26.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:17:26 smithi037 ceph-mon[142325]: pgmap v194: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:17:26.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:17:26 smithi148 ceph-mon[106692]: pgmap v194: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:17:27.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:17:27 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:17:27.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:17:27 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:17:27.678 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:17:27 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T08:17:27.678 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:17:27 smithi037 ceph-mon[142325]: mds.? [v2:172.21.15.37:6828/1248142005,v1:172.21.15.37:6829/1248142005] up:boot 2024-03-29T08:17:27.678 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:17:27 smithi037 ceph-mon[142325]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-03-29T08:17:27.678 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:17:27 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi037.mmwgxe"}]: dispatch 2024-03-29T08:17:27.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:17:27 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:17:27.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:17:27 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:17:27.807 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:17:27 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T08:17:27.807 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:17:27 smithi148 ceph-mon[106692]: mds.? [v2:172.21.15.37:6828/1248142005,v1:172.21.15.37:6829/1248142005] up:boot 2024-03-29T08:17:27.807 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:17:27 smithi148 ceph-mon[106692]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-03-29T08:17:27.807 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:17:27 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi037.mmwgxe"}]: dispatch 2024-03-29T08:17:28.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:17:28 smithi037 ceph-mon[142325]: pgmap v195: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:17:28.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:17:28 smithi148 ceph-mon[106692]: pgmap v195: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:17:30.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:17:30 smithi037 ceph-mon[142325]: pgmap v196: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:17:30.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:17:30 smithi148 ceph-mon[106692]: pgmap v196: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:17:32.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:17:31 smithi148 ceph-mon[106692]: pgmap v197: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:17:32.105 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:17:31 smithi037 ceph-mon[142325]: pgmap v197: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:17:33.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:17:33 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:17:33.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:17:33 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:17:33.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:17:33 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:17:33.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:17:33 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:17:34.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:17:34 smithi037 ceph-mon[142325]: pgmap v198: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:17:34.678 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:17:34 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:17:34.678 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:17:34 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:17:35.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:17:34 smithi148 ceph-mon[106692]: pgmap v198: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:17:35.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:17:34 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:17:35.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:17:34 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:17:36.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:17:36 smithi037 ceph-mon[142325]: pgmap v199: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:17:37.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:17:36 smithi148 ceph-mon[106692]: pgmap v199: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:17:37.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:17:37 smithi037 ceph-mon[142325]: pgmap v200: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:17:38.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:17:37 smithi148 ceph-mon[106692]: pgmap v200: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:17:39.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:17:39 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:17:39.178 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:17:39 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:17:39.178 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:17:39 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T08:17:39.178 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:17:39 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T08:17:39.178 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:17:39 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:17:39.178 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:17:39 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T08:17:39.178 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:17:39 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:17:39.178 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:17:39 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:17:39.178 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:17:39 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:17:39.178 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:17:39 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:17:39.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:17:39 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:17:39.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:17:39 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:17:39.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:17:39 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T08:17:39.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:17:39 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T08:17:39.307 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:17:39 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:17:39.307 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:17:39 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T08:17:39.307 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:17:39 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:17:39.307 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:17:39 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:17:39.307 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:17:39 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:17:39.307 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:17:39 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:17:40.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:17:40 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:17:40.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:17:40 smithi037 ceph-mon[142325]: pgmap v201: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:17:40.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:17:40 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:17:40.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:17:40 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi037.iifsmh", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-29T08:17:40.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:17:40 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T08:17:40.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:17:40 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:17:40.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:17:40 smithi148 ceph-mon[106692]: pgmap v201: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:17:40.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:17:40 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:17:40.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:17:40 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi037.iifsmh", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-29T08:17:40.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:17:40 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T08:17:41.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:17:41 smithi037 ceph-mon[142325]: Upgrade: Updating mds.cephfs.smithi037.iifsmh 2024-03-29T08:17:41.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:17:41 smithi037 ceph-mon[142325]: Deploying daemon mds.cephfs.smithi037.iifsmh on smithi037 2024-03-29T08:17:41.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:17:41 smithi148 ceph-mon[106692]: Upgrade: Updating mds.cephfs.smithi037.iifsmh 2024-03-29T08:17:41.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:17:41 smithi148 ceph-mon[106692]: Deploying daemon mds.cephfs.smithi037.iifsmh on smithi037 2024-03-29T08:17:42.264 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:17:42 smithi037 ceph-mon[142325]: pgmap v202: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:17:42.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:17:42 smithi148 ceph-mon[106692]: pgmap v202: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:17:43.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:17:43 smithi037 ceph-mon[142325]: osdmap e84: 6 total, 6 up, 6 in 2024-03-29T08:17:43.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:17:43 smithi037 ceph-mon[142325]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-03-29T08:17:43.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:17:43 smithi148 ceph-mon[106692]: osdmap e84: 6 total, 6 up, 6 in 2024-03-29T08:17:43.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:17:43 smithi148 ceph-mon[106692]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-03-29T08:17:44.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:17:44 smithi148 ceph-mon[106692]: pgmap v204: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:17:44.674 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:17:44 smithi037 ceph-mon[142325]: pgmap v204: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:17:46.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:17:45 smithi148 ceph-mon[106692]: pgmap v205: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:17:46.171 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:17:45 smithi037 ceph-mon[142325]: pgmap v205: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:17:47.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:17:47 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:17:47.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:17:47 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:17:47.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:17:47 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T08:17:47.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:17:47 smithi148 ceph-mon[106692]: mds.? [v2:172.21.15.37:6826/2060882959,v1:172.21.15.37:6827/2060882959] up:boot 2024-03-29T08:17:47.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:17:47 smithi148 ceph-mon[106692]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-03-29T08:17:47.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:17:47 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi037.iifsmh"}]: dispatch 2024-03-29T08:17:47.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:17:47 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:17:47.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:17:47 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:17:47.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:17:47 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T08:17:47.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:17:47 smithi037 ceph-mon[142325]: mds.? [v2:172.21.15.37:6826/2060882959,v1:172.21.15.37:6827/2060882959] up:boot 2024-03-29T08:17:47.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:17:47 smithi037 ceph-mon[142325]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-03-29T08:17:47.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:17:47 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi037.iifsmh"}]: dispatch 2024-03-29T08:17:48.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:17:48 smithi037 ceph-mon[142325]: pgmap v206: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:17:48.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:17:48 smithi148 ceph-mon[106692]: pgmap v206: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:17:50.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:17:50 smithi037 ceph-mon[142325]: pgmap v207: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:17:50.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:17:50 smithi148 ceph-mon[106692]: pgmap v207: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:17:52.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:17:51 smithi148 ceph-mon[106692]: pgmap v208: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:17:52.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:17:51 smithi037 ceph-mon[142325]: pgmap v208: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:17:53.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:17:52 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:17:53.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:17:52 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:17:53.151 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:17:52 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:17:53.151 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:17:52 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:17:54.258 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:17:53 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:17:54.258 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:17:53 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:17:54.259 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:17:53 smithi037 ceph-mon[142325]: pgmap v209: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:17:54.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:17:53 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:17:54.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:17:53 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:17:54.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:17:53 smithi148 ceph-mon[106692]: pgmap v209: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:17:55.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:55.240+0000 7fcb23a00700 1 -- 172.21.15.37:0/3070757291 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcb1c070360 msgr2=0x7fcb1c070730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:17:55.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:55.240+0000 7fcb23a00700 1 --2- 172.21.15.37:0/3070757291 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcb1c070360 0x7fcb1c070730 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fcb10004970 tx=0x7fcb10009e30 comp rx=0 tx=0).stop 2024-03-29T08:17:55.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:55.240+0000 7fcb23a00700 1 -- 172.21.15.37:0/3070757291 shutdown_connections 2024-03-29T08:17:55.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:55.240+0000 7fcb23a00700 1 --2- 172.21.15.37:0/3070757291 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcb1c070c70 0x7fcb1c109110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:17:55.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:55.240+0000 7fcb23a00700 1 --2- 172.21.15.37:0/3070757291 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcb1c070360 0x7fcb1c070730 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:17:55.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:55.240+0000 7fcb23a00700 1 -- 172.21.15.37:0/3070757291 >> 172.21.15.37:0/3070757291 conn(0x7fcb1c0f89b0 msgr2=0x7fcb1c0fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:17:55.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:55.240+0000 7fcb23a00700 1 -- 172.21.15.37:0/3070757291 shutdown_connections 2024-03-29T08:17:55.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:55.241+0000 7fcb23a00700 1 -- 172.21.15.37:0/3070757291 wait complete. 2024-03-29T08:17:55.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:55.241+0000 7fcb23a00700 1 Processor -- start 2024-03-29T08:17:55.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:55.242+0000 7fcb23a00700 1 -- start start 2024-03-29T08:17:55.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:55.242+0000 7fcb23a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcb1c070360 0x7fcb1c115a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:17:55.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:55.242+0000 7fcb23a00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcb1c070c70 0x7fcb1c115fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:17:55.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:55.242+0000 7fcb23a00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb1c116600 con 0x7fcb1c070360 2024-03-29T08:17:55.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:55.242+0000 7fcb23a00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb1c10fbe0 con 0x7fcb1c070c70 2024-03-29T08:17:55.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:55.242+0000 7fcb21000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcb1c070360 0x7fcb1c115a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:17:55.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:55.242+0000 7fcb21000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcb1c070360 0x7fcb1c115a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52442/0 (socket says 172.21.15.37:52442) 2024-03-29T08:17:55.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:55.242+0000 7fcb21000700 1 -- 172.21.15.37:0/2585484046 learned_addr learned my addr 172.21.15.37:0/2585484046 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:17:55.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:55.243+0000 7fcb1be00700 1 --2- 172.21.15.37:0/2585484046 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcb1c070c70 0x7fcb1c115fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:17:55.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:55.243+0000 7fcb21000700 1 -- 172.21.15.37:0/2585484046 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcb1c070c70 msgr2=0x7fcb1c115fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:17:55.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:55.243+0000 7fcb21000700 1 --2- 172.21.15.37:0/2585484046 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcb1c070c70 0x7fcb1c115fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:17:55.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:55.243+0000 7fcb21000700 1 -- 172.21.15.37:0/2585484046 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb10005040 con 0x7fcb1c070360 2024-03-29T08:17:55.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:55.243+0000 7fcb21000700 1 --2- 172.21.15.37:0/2585484046 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcb1c070360 0x7fcb1c115a70 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fcb10009360 tx=0x7fcb10009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:17:55.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:55.243+0000 7fcb18c00700 1 -- 172.21.15.37:0/2585484046 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcb1000e040 con 0x7fcb1c070360 2024-03-29T08:17:55.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:55.243+0000 7fcb18c00700 1 -- 172.21.15.37:0/2585484046 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fcb10004290 con 0x7fcb1c070360 2024-03-29T08:17:55.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:55.243+0000 7fcb23a00700 1 -- 172.21.15.37:0/2585484046 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb1c10fe60 con 0x7fcb1c070360 2024-03-29T08:17:55.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:55.243+0000 7fcb18c00700 1 -- 172.21.15.37:0/2585484046 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcb1001c3f0 con 0x7fcb1c070360 2024-03-29T08:17:55.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:55.244+0000 7fcb23a00700 1 -- 172.21.15.37:0/2585484046 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb1c1103a0 con 0x7fcb1c070360 2024-03-29T08:17:55.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:55.245+0000 7fcb23a00700 1 -- 172.21.15.37:0/2585484046 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb1c071aa0 con 0x7fcb1c070360 2024-03-29T08:17:55.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:55.245+0000 7fcb18c00700 1 -- 172.21.15.37:0/2585484046 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fcb10013420 con 0x7fcb1c070360 2024-03-29T08:17:55.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:55.246+0000 7fcb18c00700 1 --2- 172.21.15.37:0/2585484046 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fcb0406cc20 0x7fcb0406f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:17:55.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:55.246+0000 7fcb18c00700 1 -- 172.21.15.37:0/2585484046 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fcb1008d4d0 con 0x7fcb1c070360 2024-03-29T08:17:55.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:55.247+0000 7fcb1be00700 1 --2- 172.21.15.37:0/2585484046 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fcb0406cc20 0x7fcb0406f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:17:55.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:55.247+0000 7fcb1be00700 1 --2- 172.21.15.37:0/2585484046 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fcb0406cc20 0x7fcb0406f0d0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fcb1c111310 tx=0x7fcb08009ee0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:17:55.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:55.251+0000 7fcb18c00700 1 -- 172.21.15.37:0/2585484046 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fcb1005af20 con 0x7fcb1c070360 2024-03-29T08:17:55.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:17:55 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:17:55.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:55.485+0000 7fcb23a00700 1 -- 172.21.15.37:0/2585484046 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fcb1c111130 con 0x7fcb0406cc20 2024-03-29T08:17:55.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:55.487+0000 7fcb18c00700 1 -- 172.21.15.37:0/2585484046 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fcb1c111130 con 0x7fcb0406cc20 2024-03-29T08:17:55.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:55.489+0000 7fcb23a00700 1 -- 172.21.15.37:0/2585484046 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fcb0406cc20 msgr2=0x7fcb0406f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:17:55.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:55.489+0000 7fcb23a00700 1 --2- 172.21.15.37:0/2585484046 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fcb0406cc20 0x7fcb0406f0d0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fcb1c111310 tx=0x7fcb08009ee0 comp rx=0 tx=0).stop 2024-03-29T08:17:55.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:55.489+0000 7fcb23a00700 1 -- 172.21.15.37:0/2585484046 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcb1c070360 msgr2=0x7fcb1c115a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:17:55.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:55.489+0000 7fcb23a00700 1 --2- 172.21.15.37:0/2585484046 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcb1c070360 0x7fcb1c115a70 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fcb10009360 tx=0x7fcb10009e30 comp rx=0 tx=0).stop 2024-03-29T08:17:55.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:55.489+0000 7fcb23a00700 1 -- 172.21.15.37:0/2585484046 shutdown_connections 2024-03-29T08:17:55.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:55.489+0000 7fcb23a00700 1 --2- 172.21.15.37:0/2585484046 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fcb0406cc20 0x7fcb0406f0d0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:17:55.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:55.489+0000 7fcb23a00700 1 --2- 172.21.15.37:0/2585484046 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcb1c070360 0x7fcb1c115a70 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:17:55.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:55.489+0000 7fcb23a00700 1 --2- 172.21.15.37:0/2585484046 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcb1c070c70 0x7fcb1c115fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:17:55.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:55.489+0000 7fcb23a00700 1 -- 172.21.15.37:0/2585484046 >> 172.21.15.37:0/2585484046 conn(0x7fcb1c0f89b0 msgr2=0x7fcb1c1059c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:17:55.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:55.490+0000 7fcb23a00700 1 -- 172.21.15.37:0/2585484046 shutdown_connections 2024-03-29T08:17:55.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:55.490+0000 7fcb23a00700 1 -- 172.21.15.37:0/2585484046 wait complete. 2024-03-29T08:17:55.504 INFO:teuthology.orchestra.run.smithi037.stdout:true 2024-03-29T08:17:55.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:17:55 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:17:55.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:55.631+0000 7f8007800700 1 -- 172.21.15.37:0/3667582551 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8000071f90 msgr2=0x7f8000070360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:17:55.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:55.631+0000 7f8007800700 1 --2- 172.21.15.37:0/3667582551 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8000071f90 0x7f8000070360 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f7ff4009d10 tx=0x7f7ff4004d10 comp rx=0 tx=0).stop 2024-03-29T08:17:55.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:55.631+0000 7f8007800700 1 -- 172.21.15.37:0/3667582551 shutdown_connections 2024-03-29T08:17:55.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:55.631+0000 7f8007800700 1 --2- 172.21.15.37:0/3667582551 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f80000708a0 0x7f8000070d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:17:55.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:55.631+0000 7f8007800700 1 --2- 172.21.15.37:0/3667582551 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8000071f90 0x7f8000070360 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:17:55.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:55.631+0000 7f8007800700 1 -- 172.21.15.37:0/3667582551 >> 172.21.15.37:0/3667582551 conn(0x7f80000f89b0 msgr2=0x7f80000fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:17:55.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:55.631+0000 7f8007800700 1 -- 172.21.15.37:0/3667582551 shutdown_connections 2024-03-29T08:17:55.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:55.631+0000 7f8007800700 1 -- 172.21.15.37:0/3667582551 wait complete. 2024-03-29T08:17:55.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:55.632+0000 7f8007800700 1 Processor -- start 2024-03-29T08:17:55.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:55.632+0000 7f8007800700 1 -- start start 2024-03-29T08:17:55.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:55.632+0000 7f8007800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f80000708a0 0x7f800011a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:17:55.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:55.632+0000 7f8007800700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8000071f90 0x7f800011a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:17:55.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:55.632+0000 7f8007800700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f800011ae10 con 0x7f80000708a0 2024-03-29T08:17:55.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:55.632+0000 7f8007800700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80001141e0 con 0x7f8000071f90 2024-03-29T08:17:55.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:55.633+0000 7f8004e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f80000708a0 0x7f800011a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:17:55.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:55.633+0000 7f8004e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f80000708a0 0x7f800011a280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52472/0 (socket says 172.21.15.37:52472) 2024-03-29T08:17:55.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:55.633+0000 7f8004e00700 1 -- 172.21.15.37:0/3846480488 learned_addr learned my addr 172.21.15.37:0/3846480488 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:17:55.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:55.633+0000 7f7fffe00700 1 --2- 172.21.15.37:0/3846480488 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8000071f90 0x7f800011a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:17:55.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:55.633+0000 7f8004e00700 1 -- 172.21.15.37:0/3846480488 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8000071f90 msgr2=0x7f800011a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:17:55.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:55.633+0000 7f8004e00700 1 --2- 172.21.15.37:0/3846480488 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8000071f90 0x7f800011a7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:17:55.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:55.633+0000 7f8004e00700 1 -- 172.21.15.37:0/3846480488 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ff4009940 con 0x7f80000708a0 2024-03-29T08:17:55.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:55.633+0000 7f8004e00700 1 --2- 172.21.15.37:0/3846480488 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f80000708a0 0x7f800011a280 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f7ff4009740 tx=0x7f7ff4004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:17:55.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:55.634+0000 7f7ffcc00700 1 -- 172.21.15.37:0/3846480488 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7ff400e050 con 0x7f80000708a0 2024-03-29T08:17:55.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:55.634+0000 7f7ffcc00700 1 -- 172.21.15.37:0/3846480488 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f7ff400bdd0 con 0x7f80000708a0 2024-03-29T08:17:55.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:55.634+0000 7f8007800700 1 -- 172.21.15.37:0/3846480488 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8000114460 con 0x7f80000708a0 2024-03-29T08:17:55.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:55.634+0000 7f7ffcc00700 1 -- 172.21.15.37:0/3846480488 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7ff4012770 con 0x7f80000708a0 2024-03-29T08:17:55.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:55.634+0000 7f8007800700 1 -- 172.21.15.37:0/3846480488 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80001149a0 con 0x7f80000708a0 2024-03-29T08:17:55.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:55.635+0000 7f8007800700 1 -- 172.21.15.37:0/3846480488 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8000071aa0 con 0x7f80000708a0 2024-03-29T08:17:55.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:55.636+0000 7f7ffcc00700 1 -- 172.21.15.37:0/3846480488 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f7ff4019040 con 0x7f80000708a0 2024-03-29T08:17:55.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:55.637+0000 7f7ffcc00700 1 --2- 172.21.15.37:0/3846480488 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f7fe806cc30 0x7f7fe806f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:17:55.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:55.638+0000 7f7ffcc00700 1 -- 172.21.15.37:0/3846480488 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f7ff408da90 con 0x7f80000708a0 2024-03-29T08:17:55.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:55.638+0000 7f7fffe00700 1 --2- 172.21.15.37:0/3846480488 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f7fe806cc30 0x7f7fe806f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:17:55.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:55.638+0000 7f7fffe00700 1 --2- 172.21.15.37:0/3846480488 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f7fe806cc30 0x7f7fe806f0e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f8000115b20 tx=0x7f7fec00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:17:55.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:55.642+0000 7f7ffcc00700 1 -- 172.21.15.37:0/3846480488 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f7ff405b4e0 con 0x7f80000708a0 2024-03-29T08:17:55.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:55.873+0000 7f8007800700 1 -- 172.21.15.37:0/3846480488 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f800004bed0 con 0x7f7fe806cc30 2024-03-29T08:17:55.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:55.874+0000 7f7ffcc00700 1 -- 172.21.15.37:0/3846480488 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f800004bed0 con 0x7f7fe806cc30 2024-03-29T08:17:55.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:55.876+0000 7f8007800700 1 -- 172.21.15.37:0/3846480488 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f7fe806cc30 msgr2=0x7f7fe806f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:17:55.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:55.876+0000 7f8007800700 1 --2- 172.21.15.37:0/3846480488 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f7fe806cc30 0x7f7fe806f0e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f8000115b20 tx=0x7f7fec00b040 comp rx=0 tx=0).stop 2024-03-29T08:17:55.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:55.876+0000 7f8007800700 1 -- 172.21.15.37:0/3846480488 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f80000708a0 msgr2=0x7f800011a280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:17:55.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:55.876+0000 7f8007800700 1 --2- 172.21.15.37:0/3846480488 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f80000708a0 0x7f800011a280 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f7ff4009740 tx=0x7f7ff4004d10 comp rx=0 tx=0).stop 2024-03-29T08:17:55.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:55.877+0000 7f8007800700 1 -- 172.21.15.37:0/3846480488 shutdown_connections 2024-03-29T08:17:55.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:55.877+0000 7f8007800700 1 --2- 172.21.15.37:0/3846480488 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f7fe806cc30 0x7f7fe806f0e0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:17:55.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:55.877+0000 7f8007800700 1 --2- 172.21.15.37:0/3846480488 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f80000708a0 0x7f800011a280 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:17:55.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:55.877+0000 7f8007800700 1 --2- 172.21.15.37:0/3846480488 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8000071f90 0x7f800011a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:17:55.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:55.877+0000 7f8007800700 1 -- 172.21.15.37:0/3846480488 >> 172.21.15.37:0/3846480488 conn(0x7f80000f89b0 msgr2=0x7f80000f9ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:17:55.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:55.877+0000 7f8007800700 1 -- 172.21.15.37:0/3846480488 shutdown_connections 2024-03-29T08:17:55.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:55.877+0000 7f8007800700 1 -- 172.21.15.37:0/3846480488 wait complete. 2024-03-29T08:17:56.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:56.017+0000 7f94b2400700 1 -- 172.21.15.37:0/3166434751 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f94ac101ae0 msgr2=0x7f94ac065630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:17:56.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:56.017+0000 7f94b2400700 1 --2- 172.21.15.37:0/3166434751 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f94ac101ae0 0x7f94ac065630 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f94a0004940 tx=0x7f94a0009e30 comp rx=0 tx=0).stop 2024-03-29T08:17:56.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:56.017+0000 7f94b2400700 1 -- 172.21.15.37:0/3166434751 shutdown_connections 2024-03-29T08:17:56.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:56.017+0000 7f94b2400700 1 --2- 172.21.15.37:0/3166434751 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f94ac065b70 0x7f94ac065fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:17:56.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:56.017+0000 7f94b2400700 1 --2- 172.21.15.37:0/3166434751 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f94ac101ae0 0x7f94ac065630 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:17:56.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:56.017+0000 7f94b2400700 1 -- 172.21.15.37:0/3166434751 >> 172.21.15.37:0/3166434751 conn(0x7f94ac071b70 msgr2=0x7f94ac071f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:17:56.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:56.018+0000 7f94b2400700 1 -- 172.21.15.37:0/3166434751 shutdown_connections 2024-03-29T08:17:56.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:56.018+0000 7f94b2400700 1 -- 172.21.15.37:0/3166434751 wait complete. 2024-03-29T08:17:56.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:56.018+0000 7f94b2400700 1 Processor -- start 2024-03-29T08:17:56.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:56.019+0000 7f94b2400700 1 -- start start 2024-03-29T08:17:56.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:56.019+0000 7f94b2400700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f94ac065b70 0x7f94ac119f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:17:56.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:56.019+0000 7f94b2400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f94ac101ae0 0x7f94ac11a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:17:56.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:56.019+0000 7f94b2400700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94ac11aac0 con 0x7f94ac101ae0 2024-03-29T08:17:56.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:56.019+0000 7f94b2400700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94ac113e90 con 0x7f94ac065b70 2024-03-29T08:17:56.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:56.019+0000 7f94abe00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f94ac101ae0 0x7f94ac11a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:17:56.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:56.019+0000 7f94abe00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f94ac101ae0 0x7f94ac11a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52490/0 (socket says 172.21.15.37:52490) 2024-03-29T08:17:56.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:56.019+0000 7f94abe00700 1 -- 172.21.15.37:0/1418263282 learned_addr learned my addr 172.21.15.37:0/1418263282 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:17:56.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:56.019+0000 7f94b1000700 1 --2- 172.21.15.37:0/1418263282 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f94ac065b70 0x7f94ac119f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:17:56.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:56.020+0000 7f94abe00700 1 -- 172.21.15.37:0/1418263282 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f94ac065b70 msgr2=0x7f94ac119f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:17:56.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:56.020+0000 7f94abe00700 1 --2- 172.21.15.37:0/1418263282 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f94ac065b70 0x7f94ac119f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:17:56.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:56.020+0000 7f94abe00700 1 -- 172.21.15.37:0/1418263282 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94a0005040 con 0x7f94ac101ae0 2024-03-29T08:17:56.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:56.020+0000 7f94abe00700 1 --2- 172.21.15.37:0/1418263282 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f94ac101ae0 0x7f94ac11a470 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f9498002a50 tx=0x7f94980046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:17:56.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:56.020+0000 7f94a8c00700 1 -- 172.21.15.37:0/1418263282 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9498007cc0 con 0x7f94ac101ae0 2024-03-29T08:17:56.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:56.021+0000 7f94a8c00700 1 -- 172.21.15.37:0/1418263282 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f949800f040 con 0x7f94ac101ae0 2024-03-29T08:17:56.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:56.021+0000 7f94b2400700 1 -- 172.21.15.37:0/1418263282 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94ac114170 con 0x7f94ac101ae0 2024-03-29T08:17:56.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:56.021+0000 7f94a8c00700 1 -- 172.21.15.37:0/1418263282 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f949800d9c0 con 0x7f94ac101ae0 2024-03-29T08:17:56.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:56.021+0000 7f94b2400700 1 -- 172.21.15.37:0/1418263282 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94ac1146b0 con 0x7f94ac101ae0 2024-03-29T08:17:56.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:56.022+0000 7f94b2400700 1 -- 172.21.15.37:0/1418263282 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94ac0fc170 con 0x7f94ac101ae0 2024-03-29T08:17:56.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:56.023+0000 7f94a8c00700 1 -- 172.21.15.37:0/1418263282 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f949800db20 con 0x7f94ac101ae0 2024-03-29T08:17:56.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:56.024+0000 7f94a8c00700 1 --2- 172.21.15.37:0/1418263282 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f949406cc30 0x7f949406f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:17:56.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:56.024+0000 7f94a8c00700 1 -- 172.21.15.37:0/1418263282 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f949808bde0 con 0x7f94ac101ae0 2024-03-29T08:17:56.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:56.024+0000 7f94b1000700 1 --2- 172.21.15.37:0/1418263282 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f949406cc30 0x7f949406f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:17:56.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:56.025+0000 7f94b1000700 1 --2- 172.21.15.37:0/1418263282 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f949406cc30 0x7f949406f0e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f94a0009fd0 tx=0x7f94a0009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:17:56.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:56.028+0000 7f94a8c00700 1 -- 172.21.15.37:0/1418263282 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f94980597b0 con 0x7f94ac101ae0 2024-03-29T08:17:56.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:56.330+0000 7f94b2400700 1 -- 172.21.15.37:0/1418263282 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f94ac110240 con 0x7f949406cc30 2024-03-29T08:17:56.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:56.343+0000 7f94a8c00700 1 -- 172.21.15.37:0/1418263282 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f94ac110240 con 0x7f949406cc30 2024-03-29T08:17:56.343 INFO:teuthology.orchestra.run.smithi037.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-29T08:17:56.344 INFO:teuthology.orchestra.run.smithi037.stdout:alertmanager.smithi037 smithi037 *:9093,9094 running (7m) 4s ago 20m 14.9M - 0.25.0 c8568f914cd2 2f9f9746ed0f 2024-03-29T08:17:56.344 INFO:teuthology.orchestra.run.smithi037.stdout:crash.smithi037 smithi037 running (4m) 4s ago 21m 6647k - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 178d416a3186 2024-03-29T08:17:56.344 INFO:teuthology.orchestra.run.smithi037.stdout:crash.smithi148 smithi148 running (4m) 47s ago 19m 6656k - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 23666989d58f 2024-03-29T08:17:56.344 INFO:teuthology.orchestra.run.smithi037.stdout:grafana.smithi037 smithi037 *:3000 running (7m) 4s ago 20m 71.9M - 9.4.7 954c08fa6188 6b976fe49952 2024-03-29T08:17:56.344 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi037.iifsmh smithi037 running (10s) 4s ago 14m 10.8M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 2d97f562cc5b 2024-03-29T08:17:56.344 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi037.mmwgxe smithi037 running (30s) 4s ago 14m 10.9M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff ec37037f46e8 2024-03-29T08:17:56.344 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi148.qukpjh smithi148 running (14m) 47s ago 14m 15.0M - 17.2.7-789-g4213e17c cf22eb8683be 8e469782a273 2024-03-29T08:17:56.344 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi148.sqteih smithi148 running (14m) 47s ago 14m 15.0M - 17.2.7-789-g4213e17c cf22eb8683be 8e68f858bb22 2024-03-29T08:17:56.344 INFO:teuthology.orchestra.run.smithi037.stdout:mgr.smithi037.wjikii smithi037 *:8443,9283,8765 running (10m) 4s ago 22m 506M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff c5a1417b9417 2024-03-29T08:17:56.344 INFO:teuthology.orchestra.run.smithi037.stdout:mgr.smithi148.onhgvm smithi148 *:8443,9283,8765 running (8m) 47s ago 19m 450M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff fcfdbce062bd 2024-03-29T08:17:56.344 INFO:teuthology.orchestra.run.smithi037.stdout:mon.smithi037 smithi037 running (5m) 4s ago 22m 54.8M 2048M 18.2.2-1015-gd0e392d9 1cc2f45c17ff eea2060783af 2024-03-29T08:17:56.344 INFO:teuthology.orchestra.run.smithi037.stdout:mon.smithi148 smithi148 running (5m) 47s ago 19m 40.3M 2048M 18.2.2-1015-gd0e392d9 1cc2f45c17ff ad4b2c5748be 2024-03-29T08:17:56.344 INFO:teuthology.orchestra.run.smithi037.stdout:node-exporter.smithi037 smithi037 *:9100 running (8m) 4s ago 20m 7625k - 1.5.0 0da6a335fe13 26b8ae9cb878 2024-03-29T08:17:56.344 INFO:teuthology.orchestra.run.smithi037.stdout:node-exporter.smithi148 smithi148 *:9100 running (8m) 47s ago 19m 7528k - 1.5.0 0da6a335fe13 98414e4e64ed 2024-03-29T08:17:56.344 INFO:teuthology.orchestra.run.smithi037.stdout:osd.0 smithi037 running (4m) 4s ago 18m 104M 1619M 18.2.2-1015-gd0e392d9 1cc2f45c17ff c0c584f6245c 2024-03-29T08:17:56.345 INFO:teuthology.orchestra.run.smithi037.stdout:osd.1 smithi037 running (3m) 4s ago 18m 100M 1619M 18.2.2-1015-gd0e392d9 1cc2f45c17ff a4a2ff4cc83d 2024-03-29T08:17:56.345 INFO:teuthology.orchestra.run.smithi037.stdout:osd.2 smithi037 running (2m) 4s ago 17m 82.3M 1619M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 8b1bc71aa79e 2024-03-29T08:17:56.345 INFO:teuthology.orchestra.run.smithi037.stdout:osd.3 smithi148 running (2m) 47s ago 17m 104M 2642M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 1c656ab8ff86 2024-03-29T08:17:56.345 INFO:teuthology.orchestra.run.smithi037.stdout:osd.4 smithi148 running (90s) 47s ago 16m 96.4M 2642M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 966a9de11d22 2024-03-29T08:17:56.345 INFO:teuthology.orchestra.run.smithi037.stdout:osd.5 smithi148 running (52s) 47s ago 15m 11.0M 2642M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 474c4f9212ce 2024-03-29T08:17:56.345 INFO:teuthology.orchestra.run.smithi037.stdout:prometheus.smithi037 smithi037 *:9095 running (8m) 4s ago 20m 54.4M - 2.43.0 a07b618ecd1d 30c0223ec589 2024-03-29T08:17:56.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:56.345+0000 7f94b2400700 1 -- 172.21.15.37:0/1418263282 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f949406cc30 msgr2=0x7f949406f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:17:56.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:56.345+0000 7f94b2400700 1 --2- 172.21.15.37:0/1418263282 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f949406cc30 0x7f949406f0e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f94a0009fd0 tx=0x7f94a0009e30 comp rx=0 tx=0).stop 2024-03-29T08:17:56.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:56.345+0000 7f94b2400700 1 -- 172.21.15.37:0/1418263282 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f94ac101ae0 msgr2=0x7f94ac11a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:17:56.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:56.345+0000 7f94b2400700 1 --2- 172.21.15.37:0/1418263282 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f94ac101ae0 0x7f94ac11a470 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f9498002a50 tx=0x7f94980046b0 comp rx=0 tx=0).stop 2024-03-29T08:17:56.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:56.346+0000 7f94b2400700 1 -- 172.21.15.37:0/1418263282 shutdown_connections 2024-03-29T08:17:56.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:56.346+0000 7f94b2400700 1 --2- 172.21.15.37:0/1418263282 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f949406cc30 0x7f949406f0e0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:17:56.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:56.346+0000 7f94b2400700 1 --2- 172.21.15.37:0/1418263282 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f94ac065b70 0x7f94ac119f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:17:56.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:56.346+0000 7f94b2400700 1 --2- 172.21.15.37:0/1418263282 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f94ac101ae0 0x7f94ac11a470 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:17:56.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:56.346+0000 7f94b2400700 1 -- 172.21.15.37:0/1418263282 >> 172.21.15.37:0/1418263282 conn(0x7f94ac071b70 msgr2=0x7f94ac0f9b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:17:56.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:56.346+0000 7f94b2400700 1 -- 172.21.15.37:0/1418263282 shutdown_connections 2024-03-29T08:17:56.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:56.346+0000 7f94b2400700 1 -- 172.21.15.37:0/1418263282 wait complete. 2024-03-29T08:17:56.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:17:56 smithi037 ceph-mon[142325]: from='client.34448 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:17:56.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:17:56 smithi037 ceph-mon[142325]: pgmap v210: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:17:56.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:56.494+0000 7fe4abe00700 1 -- 172.21.15.37:0/391983298 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe4a4105a30 msgr2=0x7fe4a40ff0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:17:56.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:56.494+0000 7fe4abe00700 1 --2- 172.21.15.37:0/391983298 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe4a4105a30 0x7fe4a40ff0a0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fe490004940 tx=0x7fe490009e30 comp rx=0 tx=0).stop 2024-03-29T08:17:56.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:56.494+0000 7fe4abe00700 1 -- 172.21.15.37:0/391983298 shutdown_connections 2024-03-29T08:17:56.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:56.494+0000 7fe4abe00700 1 --2- 172.21.15.37:0/391983298 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe4a4105a30 0x7fe4a40ff0a0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:17:56.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:56.494+0000 7fe4abe00700 1 --2- 172.21.15.37:0/391983298 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe4a4105090 0x7fe4a4105460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:17:56.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:56.494+0000 7fe4abe00700 1 -- 172.21.15.37:0/391983298 >> 172.21.15.37:0/391983298 conn(0x7fe4a40fac00 msgr2=0x7fe4a40fd010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:17:56.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:56.494+0000 7fe4abe00700 1 -- 172.21.15.37:0/391983298 shutdown_connections 2024-03-29T08:17:56.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:56.494+0000 7fe4abe00700 1 -- 172.21.15.37:0/391983298 wait complete. 2024-03-29T08:17:56.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:56.495+0000 7fe4abe00700 1 Processor -- start 2024-03-29T08:17:56.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:56.495+0000 7fe4abe00700 1 -- start start 2024-03-29T08:17:56.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:56.495+0000 7fe4abe00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe4a4105090 0x7fe4a4115a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:17:56.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:56.495+0000 7fe4abe00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe4a4105a30 0x7fe4a4115f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:17:56.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:56.495+0000 7fe4abe00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4a41165a0 con 0x7fe4a4105a30 2024-03-29T08:17:56.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:56.495+0000 7fe4abe00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4a410fba0 con 0x7fe4a4105090 2024-03-29T08:17:56.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:56.496+0000 7fe4a8a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe4a4105a30 0x7fe4a4115f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:17:56.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:56.496+0000 7fe4a9400700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe4a4105090 0x7fe4a4115a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:17:56.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:56.496+0000 7fe4a8a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe4a4105a30 0x7fe4a4115f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52512/0 (socket says 172.21.15.37:52512) 2024-03-29T08:17:56.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:56.496+0000 7fe4a9400700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe4a4105090 0x7fe4a4115a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.37:50358/0 (socket says 172.21.15.37:50358) 2024-03-29T08:17:56.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:56.496+0000 7fe4a8a00700 1 -- 172.21.15.37:0/2915193027 learned_addr learned my addr 172.21.15.37:0/2915193027 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:17:56.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:56.496+0000 7fe4a8a00700 1 -- 172.21.15.37:0/2915193027 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe4a4105090 msgr2=0x7fe4a4115a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:17:56.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:56.496+0000 7fe4a8a00700 1 --2- 172.21.15.37:0/2915193027 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe4a4105090 0x7fe4a4115a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:17:56.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:56.496+0000 7fe4a8a00700 1 -- 172.21.15.37:0/2915193027 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe490005040 con 0x7fe4a4105a30 2024-03-29T08:17:56.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:56.497+0000 7fe4a8a00700 1 --2- 172.21.15.37:0/2915193027 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe4a4105a30 0x7fe4a4115f50 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fe490009fd0 tx=0x7fe490009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:17:56.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:56.497+0000 7fe4a1600700 1 -- 172.21.15.37:0/2915193027 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe49000e030 con 0x7fe4a4105a30 2024-03-29T08:17:56.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:56.497+0000 7fe4abe00700 1 -- 172.21.15.37:0/2915193027 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4a410fe20 con 0x7fe4a4105a30 2024-03-29T08:17:56.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:56.497+0000 7fe4a1600700 1 -- 172.21.15.37:0/2915193027 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fe4900093b0 con 0x7fe4a4105a30 2024-03-29T08:17:56.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:56.497+0000 7fe4a1600700 1 -- 172.21.15.37:0/2915193027 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe49001cc80 con 0x7fe4a4105a30 2024-03-29T08:17:56.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:56.497+0000 7fe4abe00700 1 -- 172.21.15.37:0/2915193027 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4a41103e0 con 0x7fe4a4105a30 2024-03-29T08:17:56.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:56.498+0000 7fe4abe00700 1 -- 172.21.15.37:0/2915193027 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4a41023a0 con 0x7fe4a4105a30 2024-03-29T08:17:56.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:56.500+0000 7fe4a1600700 1 -- 172.21.15.37:0/2915193027 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fe49001c400 con 0x7fe4a4105a30 2024-03-29T08:17:56.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:56.502+0000 7fe4a1600700 1 --2- 172.21.15.37:0/2915193027 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fe48c06cc20 0x7fe48c06f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:17:56.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:56.502+0000 7fe4a1600700 1 -- 172.21.15.37:0/2915193027 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fe49008d780 con 0x7fe4a4105a30 2024-03-29T08:17:56.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:56.502+0000 7fe4a9400700 1 --2- 172.21.15.37:0/2915193027 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fe48c06cc20 0x7fe48c06f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:17:56.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:56.503+0000 7fe4a9400700 1 --2- 172.21.15.37:0/2915193027 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fe48c06cc20 0x7fe48c06f0d0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fe498009b80 tx=0x7fe49800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:17:56.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:56.506+0000 7fe4a1600700 1 -- 172.21.15.37:0/2915193027 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fe49005b290 con 0x7fe4a4105a30 2024-03-29T08:17:56.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:17:56 smithi148 ceph-mon[106692]: from='client.34448 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:17:56.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:17:56 smithi148 ceph-mon[106692]: pgmap v210: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:17:56.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:56.811+0000 7fe4abe00700 1 -- 172.21.15.37:0/2915193027 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fe4a41007e0 con 0x7fe4a4105a30 2024-03-29T08:17:56.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:56.812+0000 7fe4a1600700 1 -- 172.21.15.37:0/2915193027 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+847 (secure 0 0 0) 0x7fe49005ae20 con 0x7fe4a4105a30 2024-03-29T08:17:56.812 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-29T08:17:56.812 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": { 2024-03-29T08:17:56.812 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 2 2024-03-29T08:17:56.812 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:17:56.812 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": { 2024-03-29T08:17:56.812 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 2 2024-03-29T08:17:56.812 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:17:56.813 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": { 2024-03-29T08:17:56.813 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 6 2024-03-29T08:17:56.813 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:17:56.813 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": { 2024-03-29T08:17:56.813 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 17.2.7-789-g4213e17c (4213e17c4de67016ed14f55d22503f4923b3a901) quincy (stable)": 2, 2024-03-29T08:17:56.813 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 2 2024-03-29T08:17:56.813 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:17:56.813 INFO:teuthology.orchestra.run.smithi037.stdout: "overall": { 2024-03-29T08:17:56.813 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 17.2.7-789-g4213e17c (4213e17c4de67016ed14f55d22503f4923b3a901) quincy (stable)": 2, 2024-03-29T08:17:56.813 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 12 2024-03-29T08:17:56.813 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-29T08:17:56.813 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-29T08:17:56.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:56.815+0000 7fe4abe00700 1 -- 172.21.15.37:0/2915193027 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fe48c06cc20 msgr2=0x7fe48c06f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:17:56.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:56.815+0000 7fe4abe00700 1 --2- 172.21.15.37:0/2915193027 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fe48c06cc20 0x7fe48c06f0d0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fe498009b80 tx=0x7fe49800b040 comp rx=0 tx=0).stop 2024-03-29T08:17:56.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:56.815+0000 7fe4abe00700 1 -- 172.21.15.37:0/2915193027 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe4a4105a30 msgr2=0x7fe4a4115f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:17:56.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:56.815+0000 7fe4abe00700 1 --2- 172.21.15.37:0/2915193027 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe4a4105a30 0x7fe4a4115f50 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fe490009fd0 tx=0x7fe490009e30 comp rx=0 tx=0).stop 2024-03-29T08:17:56.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:56.815+0000 7fe4abe00700 1 -- 172.21.15.37:0/2915193027 shutdown_connections 2024-03-29T08:17:56.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:56.815+0000 7fe4abe00700 1 --2- 172.21.15.37:0/2915193027 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fe48c06cc20 0x7fe48c06f0d0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:17:56.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:56.815+0000 7fe4abe00700 1 --2- 172.21.15.37:0/2915193027 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe4a4105090 0x7fe4a4115a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:17:56.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:56.815+0000 7fe4abe00700 1 --2- 172.21.15.37:0/2915193027 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe4a4105a30 0x7fe4a4115f50 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:17:56.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:56.815+0000 7fe4abe00700 1 -- 172.21.15.37:0/2915193027 >> 172.21.15.37:0/2915193027 conn(0x7fe4a40fac00 msgr2=0x7fe4a40fc980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:17:56.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:56.815+0000 7fe4abe00700 1 -- 172.21.15.37:0/2915193027 shutdown_connections 2024-03-29T08:17:56.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:56.815+0000 7fe4abe00700 1 -- 172.21.15.37:0/2915193027 wait complete. 2024-03-29T08:17:56.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:56.956+0000 7fc557800700 1 -- 172.21.15.37:0/2434797974 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc550070bf0 msgr2=0x7fc5501091b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:17:56.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:56.956+0000 7fc557800700 1 --2- 172.21.15.37:0/2434797974 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc550070bf0 0x7fc5501091b0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fc53c004940 tx=0x7fc53c009e30 comp rx=0 tx=0).stop 2024-03-29T08:17:56.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:56.956+0000 7fc557800700 1 -- 172.21.15.37:0/2434797974 shutdown_connections 2024-03-29T08:17:56.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:56.956+0000 7fc557800700 1 --2- 172.21.15.37:0/2434797974 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc550070bf0 0x7fc5501091b0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:17:56.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:56.956+0000 7fc557800700 1 --2- 172.21.15.37:0/2434797974 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc5500702e0 0x7fc5500706b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:17:56.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:56.956+0000 7fc557800700 1 -- 172.21.15.37:0/2434797974 >> 172.21.15.37:0/2434797974 conn(0x7fc5500f8970 msgr2=0x7fc5500fad80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:17:56.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:56.956+0000 7fc557800700 1 -- 172.21.15.37:0/2434797974 shutdown_connections 2024-03-29T08:17:56.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:56.956+0000 7fc557800700 1 -- 172.21.15.37:0/2434797974 wait complete. 2024-03-29T08:17:56.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:56.957+0000 7fc557800700 1 Processor -- start 2024-03-29T08:17:56.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:56.957+0000 7fc557800700 1 -- start start 2024-03-29T08:17:56.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:56.958+0000 7fc557800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc5500702e0 0x7fc550115d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:17:56.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:56.958+0000 7fc557800700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc550070bf0 0x7fc550116270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:17:56.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:56.958+0000 7fc557800700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5501168c0 con 0x7fc5500702e0 2024-03-29T08:17:56.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:56.958+0000 7fc557800700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc55010fc90 con 0x7fc550070bf0 2024-03-29T08:17:56.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:56.958+0000 7fc554e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc5500702e0 0x7fc550115d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:17:56.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:56.958+0000 7fc554e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc5500702e0 0x7fc550115d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52532/0 (socket says 172.21.15.37:52532) 2024-03-29T08:17:56.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:56.958+0000 7fc554e00700 1 -- 172.21.15.37:0/2866561852 learned_addr learned my addr 172.21.15.37:0/2866561852 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:17:56.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:56.958+0000 7fc54fe00700 1 --2- 172.21.15.37:0/2866561852 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc550070bf0 0x7fc550116270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:17:56.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:56.958+0000 7fc554e00700 1 -- 172.21.15.37:0/2866561852 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc550070bf0 msgr2=0x7fc550116270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:17:56.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:56.958+0000 7fc554e00700 1 --2- 172.21.15.37:0/2866561852 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc550070bf0 0x7fc550116270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:17:56.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:56.958+0000 7fc554e00700 1 -- 172.21.15.37:0/2866561852 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc53c005040 con 0x7fc5500702e0 2024-03-29T08:17:56.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:56.959+0000 7fc554e00700 1 --2- 172.21.15.37:0/2866561852 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc5500702e0 0x7fc550115d30 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fc54400c8c0 tx=0x7fc5440044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:17:56.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:56.959+0000 7fc54cc00700 1 -- 172.21.15.37:0/2866561852 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc54400a980 con 0x7fc5500702e0 2024-03-29T08:17:56.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:56.959+0000 7fc54cc00700 1 -- 172.21.15.37:0/2866561852 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fc54400ce60 con 0x7fc5500702e0 2024-03-29T08:17:56.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:56.959+0000 7fc557800700 1 -- 172.21.15.37:0/2866561852 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc55010ff70 con 0x7fc5500702e0 2024-03-29T08:17:56.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:56.959+0000 7fc54cc00700 1 -- 172.21.15.37:0/2866561852 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc5440113a0 con 0x7fc5500702e0 2024-03-29T08:17:56.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:56.959+0000 7fc557800700 1 -- 172.21.15.37:0/2866561852 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5501104b0 con 0x7fc5500702e0 2024-03-29T08:17:56.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:56.960+0000 7fc557800700 1 -- 172.21.15.37:0/2866561852 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc550071a20 con 0x7fc5500702e0 2024-03-29T08:17:56.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:56.961+0000 7fc54cc00700 1 -- 172.21.15.37:0/2866561852 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fc544012400 con 0x7fc5500702e0 2024-03-29T08:17:56.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:56.962+0000 7fc54cc00700 1 --2- 172.21.15.37:0/2866561852 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fc53806cc30 0x7fc53806f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:17:56.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:56.962+0000 7fc54cc00700 1 -- 172.21.15.37:0/2866561852 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fc54408c1d0 con 0x7fc5500702e0 2024-03-29T08:17:56.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:56.962+0000 7fc54fe00700 1 --2- 172.21.15.37:0/2866561852 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fc53806cc30 0x7fc53806f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:17:56.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:56.963+0000 7fc54fe00700 1 --2- 172.21.15.37:0/2866561852 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fc53806cc30 0x7fc53806f0e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fc53c00b020 tx=0x7fc53c009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:17:56.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:56.966+0000 7fc54cc00700 1 -- 172.21.15.37:0/2866561852 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fc544059c20 con 0x7fc5500702e0 2024-03-29T08:17:57.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:17:57 smithi037 ceph-mon[142325]: from='client.34452 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:17:57.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:17:57 smithi037 ceph-mon[142325]: from='client.34456 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:17:57.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:17:57 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/2915193027' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:17:57.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:57.224+0000 7fc557800700 1 -- 172.21.15.37:0/2866561852 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fc55005f5f0 con 0x7fc5500702e0 2024-03-29T08:17:57.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:57.225+0000 7fc54cc00700 1 -- 172.21.15.37:0/2866561852 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 26 v26) v1 ==== 76+0+1841 (secure 0 0 0) 0x7fc5440597b0 con 0x7fc5500702e0 2024-03-29T08:17:57.225 INFO:teuthology.orchestra.run.smithi037.stdout:e26 2024-03-29T08:17:57.225 INFO:teuthology.orchestra.run.smithi037.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-29T08:17:57.225 INFO:teuthology.orchestra.run.smithi037.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T08:17:57.225 INFO:teuthology.orchestra.run.smithi037.stdout:legacy client fscid: 1 2024-03-29T08:17:57.225 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:17:57.225 INFO:teuthology.orchestra.run.smithi037.stdout:Filesystem 'cephfs' (1) 2024-03-29T08:17:57.226 INFO:teuthology.orchestra.run.smithi037.stdout:fs_name cephfs 2024-03-29T08:17:57.226 INFO:teuthology.orchestra.run.smithi037.stdout:epoch 21 2024-03-29T08:17:57.226 INFO:teuthology.orchestra.run.smithi037.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-03-29T08:17:57.226 INFO:teuthology.orchestra.run.smithi037.stdout:created 2024-03-29T08:03:29.241269+0000 2024-03-29T08:17:57.226 INFO:teuthology.orchestra.run.smithi037.stdout:modified 2024-03-29T08:17:16.237238+0000 2024-03-29T08:17:57.226 INFO:teuthology.orchestra.run.smithi037.stdout:tableserver 0 2024-03-29T08:17:57.226 INFO:teuthology.orchestra.run.smithi037.stdout:root 0 2024-03-29T08:17:57.226 INFO:teuthology.orchestra.run.smithi037.stdout:session_timeout 60 2024-03-29T08:17:57.226 INFO:teuthology.orchestra.run.smithi037.stdout:session_autoclose 300 2024-03-29T08:17:57.226 INFO:teuthology.orchestra.run.smithi037.stdout:max_file_size 1099511627776 2024-03-29T08:17:57.226 INFO:teuthology.orchestra.run.smithi037.stdout:max_xattr_size 65536 2024-03-29T08:17:57.226 INFO:teuthology.orchestra.run.smithi037.stdout:required_client_features {} 2024-03-29T08:17:57.226 INFO:teuthology.orchestra.run.smithi037.stdout:last_failure 0 2024-03-29T08:17:57.226 INFO:teuthology.orchestra.run.smithi037.stdout:last_failure_osd_epoch 82 2024-03-29T08:17:57.226 INFO:teuthology.orchestra.run.smithi037.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T08:17:57.227 INFO:teuthology.orchestra.run.smithi037.stdout:max_mds 2 2024-03-29T08:17:57.227 INFO:teuthology.orchestra.run.smithi037.stdout:in 0,1 2024-03-29T08:17:57.227 INFO:teuthology.orchestra.run.smithi037.stdout:up {} 2024-03-29T08:17:57.227 INFO:teuthology.orchestra.run.smithi037.stdout:failed 0,1 2024-03-29T08:17:57.227 INFO:teuthology.orchestra.run.smithi037.stdout:damaged 2024-03-29T08:17:57.227 INFO:teuthology.orchestra.run.smithi037.stdout:stopped 2024-03-29T08:17:57.227 INFO:teuthology.orchestra.run.smithi037.stdout:data_pools [3] 2024-03-29T08:17:57.227 INFO:teuthology.orchestra.run.smithi037.stdout:metadata_pool 2 2024-03-29T08:17:57.227 INFO:teuthology.orchestra.run.smithi037.stdout:inline_data disabled 2024-03-29T08:17:57.227 INFO:teuthology.orchestra.run.smithi037.stdout:balancer 2024-03-29T08:17:57.227 INFO:teuthology.orchestra.run.smithi037.stdout:bal_rank_mask -1 2024-03-29T08:17:57.227 INFO:teuthology.orchestra.run.smithi037.stdout:standby_count_wanted 1 2024-03-29T08:17:57.227 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:17:57.227 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:17:57.227 INFO:teuthology.orchestra.run.smithi037.stdout:Standby daemons: 2024-03-29T08:17:57.228 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:17:57.228 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi148.qukpjh{-1:24319} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.148:6826/2641327912,v1:172.21.15.148:6827/2641327912] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:17:57.228 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi148.sqteih{-1:24327} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.148:6824/3252351726,v1:172.21.15.148:6825/3252351726] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:17:57.228 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi037.mmwgxe{-1:34440} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.37:6828/1248142005,v1:172.21.15.37:6829/1248142005] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:17:57.228 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi037.iifsmh{-1:34444} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.37:6826/2060882959,v1:172.21.15.37:6827/2060882959] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:17:57.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:57.227+0000 7fc557800700 1 -- 172.21.15.37:0/2866561852 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fc53806cc30 msgr2=0x7fc53806f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:17:57.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:57.227+0000 7fc557800700 1 --2- 172.21.15.37:0/2866561852 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fc53806cc30 0x7fc53806f0e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fc53c00b020 tx=0x7fc53c009e30 comp rx=0 tx=0).stop 2024-03-29T08:17:57.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:57.227+0000 7fc557800700 1 -- 172.21.15.37:0/2866561852 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc5500702e0 msgr2=0x7fc550115d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:17:57.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:57.227+0000 7fc557800700 1 --2- 172.21.15.37:0/2866561852 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc5500702e0 0x7fc550115d30 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fc54400c8c0 tx=0x7fc5440044c0 comp rx=0 tx=0).stop 2024-03-29T08:17:57.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:57.227+0000 7fc557800700 1 -- 172.21.15.37:0/2866561852 shutdown_connections 2024-03-29T08:17:57.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:57.227+0000 7fc557800700 1 --2- 172.21.15.37:0/2866561852 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fc53806cc30 0x7fc53806f0e0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:17:57.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:57.227+0000 7fc557800700 1 --2- 172.21.15.37:0/2866561852 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc5500702e0 0x7fc550115d30 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:17:57.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:57.227+0000 7fc557800700 1 --2- 172.21.15.37:0/2866561852 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc550070bf0 0x7fc550116270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:17:57.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:57.227+0000 7fc557800700 1 -- 172.21.15.37:0/2866561852 >> 172.21.15.37:0/2866561852 conn(0x7fc5500f8970 msgr2=0x7fc550105b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:17:57.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:57.227+0000 7fc557800700 1 -- 172.21.15.37:0/2866561852 shutdown_connections 2024-03-29T08:17:57.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:57.227+0000 7fc557800700 1 -- 172.21.15.37:0/2866561852 wait complete. 2024-03-29T08:17:57.229 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 26 2024-03-29T08:17:57.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:57.378+0000 7fc63b000700 1 -- 172.21.15.37:0/1959550659 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc634070c10 msgr2=0x7fc634109170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:17:57.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:57.378+0000 7fc63b000700 1 --2- 172.21.15.37:0/1959550659 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc634070c10 0x7fc634109170 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fc620004970 tx=0x7fc620009e30 comp rx=0 tx=0).stop 2024-03-29T08:17:57.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:57.378+0000 7fc63b000700 1 -- 172.21.15.37:0/1959550659 shutdown_connections 2024-03-29T08:17:57.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:57.378+0000 7fc63b000700 1 --2- 172.21.15.37:0/1959550659 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc634070c10 0x7fc634109170 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:17:57.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:57.378+0000 7fc63b000700 1 --2- 172.21.15.37:0/1959550659 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc634070300 0x7fc6340706d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:17:57.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:57.378+0000 7fc63b000700 1 -- 172.21.15.37:0/1959550659 >> 172.21.15.37:0/1959550659 conn(0x7fc6340f89b0 msgr2=0x7fc6340fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:17:57.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:57.378+0000 7fc63b000700 1 -- 172.21.15.37:0/1959550659 shutdown_connections 2024-03-29T08:17:57.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:57.378+0000 7fc63b000700 1 -- 172.21.15.37:0/1959550659 wait complete. 2024-03-29T08:17:57.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:57.379+0000 7fc63b000700 1 Processor -- start 2024-03-29T08:17:57.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:57.379+0000 7fc63b000700 1 -- start start 2024-03-29T08:17:57.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:57.380+0000 7fc63b000700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc634070300 0x7fc634115a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:17:57.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:57.380+0000 7fc63b000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc634070c10 0x7fc634115fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:17:57.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:57.380+0000 7fc63b000700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc634116610 con 0x7fc634070c10 2024-03-29T08:17:57.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:57.380+0000 7fc63b000700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc63410fbf0 con 0x7fc634070300 2024-03-29T08:17:57.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:57.380+0000 7fc633400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc634070c10 0x7fc634115fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:17:57.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:57.380+0000 7fc633400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc634070c10 0x7fc634115fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52546/0 (socket says 172.21.15.37:52546) 2024-03-29T08:17:57.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:57.380+0000 7fc633400700 1 -- 172.21.15.37:0/1684419011 learned_addr learned my addr 172.21.15.37:0/1684419011 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:17:57.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:57.380+0000 7fc633e00700 1 --2- 172.21.15.37:0/1684419011 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc634070300 0x7fc634115a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:17:57.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:57.380+0000 7fc633400700 1 -- 172.21.15.37:0/1684419011 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc634070300 msgr2=0x7fc634115a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:17:57.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:57.380+0000 7fc633400700 1 --2- 172.21.15.37:0/1684419011 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc634070300 0x7fc634115a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:17:57.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:57.380+0000 7fc633400700 1 -- 172.21.15.37:0/1684419011 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc620005040 con 0x7fc634070c10 2024-03-29T08:17:57.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:57.381+0000 7fc633400700 1 --2- 172.21.15.37:0/1684419011 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc634070c10 0x7fc634115fc0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fc620009360 tx=0x7fc620009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:17:57.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:57.381+0000 7fc62be00700 1 -- 172.21.15.37:0/1684419011 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc62000e040 con 0x7fc634070c10 2024-03-29T08:17:57.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:57.381+0000 7fc62be00700 1 -- 172.21.15.37:0/1684419011 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fc620004290 con 0x7fc634070c10 2024-03-29T08:17:57.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:57.381+0000 7fc63b000700 1 -- 172.21.15.37:0/1684419011 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc63410fe70 con 0x7fc634070c10 2024-03-29T08:17:57.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:57.381+0000 7fc62be00700 1 -- 172.21.15.37:0/1684419011 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc620013d90 con 0x7fc634070c10 2024-03-29T08:17:57.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:57.381+0000 7fc63b000700 1 -- 172.21.15.37:0/1684419011 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6341103b0 con 0x7fc634070c10 2024-03-29T08:17:57.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:57.383+0000 7fc63b000700 1 -- 172.21.15.37:0/1684419011 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc634071a40 con 0x7fc634070c10 2024-03-29T08:17:57.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:57.383+0000 7fc62be00700 1 -- 172.21.15.37:0/1684419011 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fc62000ba40 con 0x7fc634070c10 2024-03-29T08:17:57.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:57.384+0000 7fc62be00700 1 --2- 172.21.15.37:0/1684419011 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fc61806cc20 0x7fc61806f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:17:57.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:57.384+0000 7fc62be00700 1 -- 172.21.15.37:0/1684419011 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fc62008d270 con 0x7fc634070c10 2024-03-29T08:17:57.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:57.385+0000 7fc633e00700 1 --2- 172.21.15.37:0/1684419011 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fc61806cc20 0x7fc61806f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:17:57.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:57.385+0000 7fc633e00700 1 --2- 172.21.15.37:0/1684419011 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fc61806cc20 0x7fc61806f0d0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fc624004f40 tx=0x7fc62400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:17:57.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:57.389+0000 7fc62be00700 1 -- 172.21.15.37:0/1684419011 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fc62005ad40 con 0x7fc634070c10 2024-03-29T08:17:57.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:17:57 smithi148 ceph-mon[106692]: from='client.34452 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:17:57.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:17:57 smithi148 ceph-mon[106692]: from='client.34456 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:17:57.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:17:57 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/2915193027' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:17:57.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:57.625+0000 7fc63b000700 1 -- 172.21.15.37:0/1684419011 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc63410bfc0 con 0x7fc61806cc20 2024-03-29T08:17:57.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:57.626+0000 7fc62be00700 1 -- 172.21.15.37:0/1684419011 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fc63410bfc0 con 0x7fc61806cc20 2024-03-29T08:17:57.626 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-29T08:17:57.626 INFO:teuthology.orchestra.run.smithi037.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59", 2024-03-29T08:17:57.626 INFO:teuthology.orchestra.run.smithi037.stdout: "in_progress": true, 2024-03-29T08:17:57.626 INFO:teuthology.orchestra.run.smithi037.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-29T08:17:57.627 INFO:teuthology.orchestra.run.smithi037.stdout: "services_complete": [ 2024-03-29T08:17:57.627 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr", 2024-03-29T08:17:57.627 INFO:teuthology.orchestra.run.smithi037.stdout: "mon", 2024-03-29T08:17:57.627 INFO:teuthology.orchestra.run.smithi037.stdout: "crash", 2024-03-29T08:17:57.627 INFO:teuthology.orchestra.run.smithi037.stdout: "osd" 2024-03-29T08:17:57.627 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-29T08:17:57.627 INFO:teuthology.orchestra.run.smithi037.stdout: "progress": "14/21 daemons upgraded", 2024-03-29T08:17:57.627 INFO:teuthology.orchestra.run.smithi037.stdout: "message": "Currently upgrading mds daemons", 2024-03-29T08:17:57.627 INFO:teuthology.orchestra.run.smithi037.stdout: "is_paused": false 2024-03-29T08:17:57.627 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-29T08:17:57.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:57.628+0000 7fc63b000700 1 -- 172.21.15.37:0/1684419011 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fc61806cc20 msgr2=0x7fc61806f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:17:57.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:57.629+0000 7fc63b000700 1 --2- 172.21.15.37:0/1684419011 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fc61806cc20 0x7fc61806f0d0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fc624004f40 tx=0x7fc62400b040 comp rx=0 tx=0).stop 2024-03-29T08:17:57.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:57.629+0000 7fc63b000700 1 -- 172.21.15.37:0/1684419011 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc634070c10 msgr2=0x7fc634115fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:17:57.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:57.629+0000 7fc63b000700 1 --2- 172.21.15.37:0/1684419011 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc634070c10 0x7fc634115fc0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fc620009360 tx=0x7fc620009e30 comp rx=0 tx=0).stop 2024-03-29T08:17:57.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:57.629+0000 7fc63b000700 1 -- 172.21.15.37:0/1684419011 shutdown_connections 2024-03-29T08:17:57.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:57.629+0000 7fc63b000700 1 --2- 172.21.15.37:0/1684419011 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fc61806cc20 0x7fc61806f0d0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:17:57.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:57.629+0000 7fc63b000700 1 --2- 172.21.15.37:0/1684419011 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc634070300 0x7fc634115a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:17:57.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:57.629+0000 7fc63b000700 1 --2- 172.21.15.37:0/1684419011 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc634070c10 0x7fc634115fc0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:17:57.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:57.629+0000 7fc63b000700 1 -- 172.21.15.37:0/1684419011 >> 172.21.15.37:0/1684419011 conn(0x7fc6340f89b0 msgr2=0x7fc634105a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:17:57.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:57.629+0000 7fc63b000700 1 -- 172.21.15.37:0/1684419011 shutdown_connections 2024-03-29T08:17:57.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:57.629+0000 7fc63b000700 1 -- 172.21.15.37:0/1684419011 wait complete. 2024-03-29T08:17:57.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:57.767+0000 7ff54b200700 1 -- 172.21.15.37:0/3396025326 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff54c070c10 msgr2=0x7ff54c108fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:17:57.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:57.767+0000 7ff54b200700 1 --2- 172.21.15.37:0/3396025326 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff54c070c10 0x7ff54c108fa0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7ff538004970 tx=0x7ff538009e30 comp rx=0 tx=0).stop 2024-03-29T08:17:57.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:57.767+0000 7ff54b200700 1 -- 172.21.15.37:0/3396025326 shutdown_connections 2024-03-29T08:17:57.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:57.767+0000 7ff54b200700 1 --2- 172.21.15.37:0/3396025326 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff54c070c10 0x7ff54c108fa0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:17:57.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:57.767+0000 7ff54b200700 1 --2- 172.21.15.37:0/3396025326 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff54c070300 0x7ff54c0706d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:17:57.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:57.767+0000 7ff54b200700 1 -- 172.21.15.37:0/3396025326 >> 172.21.15.37:0/3396025326 conn(0x7ff54c0f87a0 msgr2=0x7ff54c0fabb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:17:57.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:57.767+0000 7ff54b200700 1 -- 172.21.15.37:0/3396025326 shutdown_connections 2024-03-29T08:17:57.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:57.767+0000 7ff54b200700 1 -- 172.21.15.37:0/3396025326 wait complete. 2024-03-29T08:17:57.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:57.768+0000 7ff54b200700 1 Processor -- start 2024-03-29T08:17:57.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:57.768+0000 7ff54b200700 1 -- start start 2024-03-29T08:17:57.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:57.769+0000 7ff54b200700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff54c070300 0x7ff54c115ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:17:57.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:57.769+0000 7ff54b200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff54c070c10 0x7ff54c115ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:17:57.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:57.769+0000 7ff54b200700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff54c116640 con 0x7ff54c070c10 2024-03-29T08:17:57.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:57.769+0000 7ff54b200700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff54c10fa30 con 0x7ff54c070300 2024-03-29T08:17:57.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:57.769+0000 7ff549400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff54c070c10 0x7ff54c115ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:17:57.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:57.769+0000 7ff549400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff54c070c10 0x7ff54c115ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52566/0 (socket says 172.21.15.37:52566) 2024-03-29T08:17:57.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:57.769+0000 7ff549400700 1 -- 172.21.15.37:0/790073818 learned_addr learned my addr 172.21.15.37:0/790073818 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:17:57.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:57.769+0000 7ff549e00700 1 --2- 172.21.15.37:0/790073818 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff54c070300 0x7ff54c115ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:17:57.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:57.769+0000 7ff549400700 1 -- 172.21.15.37:0/790073818 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff54c070300 msgr2=0x7ff54c115ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:17:57.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:57.769+0000 7ff549400700 1 --2- 172.21.15.37:0/790073818 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff54c070300 0x7ff54c115ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:17:57.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:57.769+0000 7ff549400700 1 -- 172.21.15.37:0/790073818 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff538005040 con 0x7ff54c070c10 2024-03-29T08:17:57.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:57.770+0000 7ff549400700 1 --2- 172.21.15.37:0/790073818 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff54c070c10 0x7ff54c115ff0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7ff538005010 tx=0x7ff538009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:17:57.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:57.770+0000 7ff542000700 1 -- 172.21.15.37:0/790073818 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff53800e040 con 0x7ff54c070c10 2024-03-29T08:17:57.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:57.770+0000 7ff542000700 1 -- 172.21.15.37:0/790073818 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ff538004290 con 0x7ff54c070c10 2024-03-29T08:17:57.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:57.770+0000 7ff542000700 1 -- 172.21.15.37:0/790073818 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff538013d00 con 0x7ff54c070c10 2024-03-29T08:17:57.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:57.770+0000 7ff54b200700 1 -- 172.21.15.37:0/790073818 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff54c10fcb0 con 0x7ff54c070c10 2024-03-29T08:17:57.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:57.770+0000 7ff54b200700 1 -- 172.21.15.37:0/790073818 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff54c1101f0 con 0x7ff54c070c10 2024-03-29T08:17:57.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:57.771+0000 7ff54b200700 1 -- 172.21.15.37:0/790073818 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff54c071a40 con 0x7ff54c070c10 2024-03-29T08:17:57.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:57.772+0000 7ff542000700 1 -- 172.21.15.37:0/790073818 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7ff5380134b0 con 0x7ff54c070c10 2024-03-29T08:17:57.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:57.773+0000 7ff542000700 1 --2- 172.21.15.37:0/790073818 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7ff53006cc30 0x7ff53006f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:17:57.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:57.773+0000 7ff542000700 1 -- 172.21.15.37:0/790073818 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6496+0+0 (secure 0 0 0) 0x7ff53808d150 con 0x7ff54c070c10 2024-03-29T08:17:57.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:57.773+0000 7ff549e00700 1 --2- 172.21.15.37:0/790073818 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7ff53006cc30 0x7ff53006f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:17:57.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:57.774+0000 7ff549e00700 1 --2- 172.21.15.37:0/790073818 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7ff53006cc30 0x7ff53006f0e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7ff53c009b80 tx=0x7ff53c00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:17:57.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:57.777+0000 7ff542000700 1 -- 172.21.15.37:0/790073818 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7ff53805ac20 con 0x7ff54c070c10 2024-03-29T08:17:58.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:58.073+0000 7ff54b200700 1 -- 172.21.15.37:0/790073818 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7ff54c04bed0 con 0x7ff54c070c10 2024-03-29T08:17:58.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:58.073+0000 7ff542000700 1 -- 172.21.15.37:0/790073818 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7ff538013e60 con 0x7ff54c070c10 2024-03-29T08:17:58.074 INFO:teuthology.orchestra.run.smithi037.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-03-29T08:17:58.074 INFO:teuthology.orchestra.run.smithi037.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-03-29T08:17:58.074 INFO:teuthology.orchestra.run.smithi037.stdout: fs cephfs is degraded 2024-03-29T08:17:58.074 INFO:teuthology.orchestra.run.smithi037.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-03-29T08:17:58.074 INFO:teuthology.orchestra.run.smithi037.stdout: fs cephfs is offline because no MDS is active for it. 2024-03-29T08:17:58.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:58.076+0000 7ff54b200700 1 -- 172.21.15.37:0/790073818 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7ff53006cc30 msgr2=0x7ff53006f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:17:58.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:58.076+0000 7ff54b200700 1 --2- 172.21.15.37:0/790073818 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7ff53006cc30 0x7ff53006f0e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7ff53c009b80 tx=0x7ff53c00b040 comp rx=0 tx=0).stop 2024-03-29T08:17:58.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:58.076+0000 7ff54b200700 1 -- 172.21.15.37:0/790073818 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff54c070c10 msgr2=0x7ff54c115ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:17:58.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:58.076+0000 7ff54b200700 1 --2- 172.21.15.37:0/790073818 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff54c070c10 0x7ff54c115ff0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7ff538005010 tx=0x7ff538009e30 comp rx=0 tx=0).stop 2024-03-29T08:17:58.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:58.076+0000 7ff54b200700 1 -- 172.21.15.37:0/790073818 shutdown_connections 2024-03-29T08:17:58.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:58.076+0000 7ff54b200700 1 --2- 172.21.15.37:0/790073818 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7ff53006cc30 0x7ff53006f0e0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:17:58.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:58.076+0000 7ff54b200700 1 --2- 172.21.15.37:0/790073818 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff54c070300 0x7ff54c115ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:17:58.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:58.076+0000 7ff54b200700 1 --2- 172.21.15.37:0/790073818 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff54c070c10 0x7ff54c115ff0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:17:58.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:58.076+0000 7ff54b200700 1 -- 172.21.15.37:0/790073818 >> 172.21.15.37:0/790073818 conn(0x7ff54c0f87a0 msgr2=0x7ff54c0f9ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:17:58.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:58.076+0000 7ff54b200700 1 -- 172.21.15.37:0/790073818 shutdown_connections 2024-03-29T08:17:58.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:17:58.076+0000 7ff54b200700 1 -- 172.21.15.37:0/790073818 wait complete. 2024-03-29T08:17:58.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:17:58 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/2866561852' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T08:17:58.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:17:58 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:17:58.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:17:58 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:17:58.428 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:17:58 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T08:17:58.428 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:17:58 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T08:17:58.428 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:17:58 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:17:58.428 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:17:58 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T08:17:58.428 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:17:58 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:17:58.428 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:17:58 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:17:58.428 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:17:58 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:17:58.428 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:17:58 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:17:58.428 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:17:58 smithi037 ceph-mon[142325]: pgmap v211: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:17:58.428 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:17:58 smithi037 ceph-mon[142325]: from='client.34468 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:17:58.428 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:17:58 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/790073818' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T08:17:58.557 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:17:58 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/2866561852' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T08:17:58.557 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:17:58 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:17:58.557 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:17:58 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:17:58.557 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:17:58 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T08:17:58.557 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:17:58 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T08:17:58.557 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:17:58 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:17:58.558 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:17:58 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T08:17:58.558 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:17:58 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:17:58.558 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:17:58 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:17:58.558 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:17:58 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:17:58.558 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:17:58 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:17:58.558 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:17:58 smithi148 ceph-mon[106692]: pgmap v211: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:17:58.558 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:17:58 smithi148 ceph-mon[106692]: from='client.34468 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:17:58.558 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:17:58 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/790073818' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T08:18:00.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:18:00 smithi148 ceph-mon[106692]: Upgrade: Updating mds.cephfs.smithi148.sqteih 2024-03-29T08:18:00.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:18:00 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:18:00.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:18:00 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi148.sqteih", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-29T08:18:00.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:18:00 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T08:18:00.807 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:18:00 smithi148 ceph-mon[106692]: Deploying daemon mds.cephfs.smithi148.sqteih on smithi148 2024-03-29T08:18:00.807 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:18:00 smithi148 ceph-mon[106692]: pgmap v212: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:18:00.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:18:00 smithi037 ceph-mon[142325]: Upgrade: Updating mds.cephfs.smithi148.sqteih 2024-03-29T08:18:00.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:18:00 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:18:00.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:18:00 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi148.sqteih", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-29T08:18:00.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:18:00 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T08:18:00.928 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:18:00 smithi037 ceph-mon[142325]: Deploying daemon mds.cephfs.smithi148.sqteih on smithi148 2024-03-29T08:18:00.928 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:18:00 smithi037 ceph-mon[142325]: pgmap v212: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:18:01.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:18:01 smithi148 ceph-mon[106692]: pgmap v213: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:18:02.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:18:01 smithi037 ceph-mon[142325]: pgmap v213: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:18:03.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:18:03 smithi037 ceph-mon[142325]: osdmap e85: 6 total, 6 up, 6 in 2024-03-29T08:18:03.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:18:03 smithi037 ceph-mon[142325]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-03-29T08:18:03.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:18:03 smithi148 ceph-mon[106692]: osdmap e85: 6 total, 6 up, 6 in 2024-03-29T08:18:03.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:18:03 smithi148 ceph-mon[106692]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-03-29T08:18:04.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:18:04 smithi148 ceph-mon[106692]: pgmap v215: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:18:04.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:18:04 smithi037 ceph-mon[142325]: pgmap v215: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:18:05.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:18:05 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi148.sqteih"}]: dispatch 2024-03-29T08:18:06.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:18:05 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi148.sqteih"}]: dispatch 2024-03-29T08:18:06.839 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:18:06 smithi148 ceph-mon[106692]: pgmap v216: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:18:06.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:18:06 smithi037 ceph-mon[142325]: pgmap v216: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:18:07.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:18:07 smithi148 ceph-mon[106692]: pgmap v217: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:18:08.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:18:07 smithi037 ceph-mon[142325]: pgmap v217: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:18:09.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:18:09 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:18:09.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:18:09 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:18:09.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:18:09 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T08:18:09.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:18:09 smithi037 ceph-mon[142325]: mds.? [v2:172.21.15.148:6824/4078652242,v1:172.21.15.148:6825/4078652242] up:boot 2024-03-29T08:18:09.678 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:18:09 smithi037 ceph-mon[142325]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-03-29T08:18:09.678 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:18:09 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi148.sqteih"}]: dispatch 2024-03-29T08:18:09.678 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:18:09 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:18:09.678 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:18:09 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:18:09.678 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:18:09 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:18:09.678 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:18:09 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:18:09.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:18:09 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:18:09.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:18:09 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:18:09.807 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:18:09 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T08:18:09.807 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:18:09 smithi148 ceph-mon[106692]: mds.? [v2:172.21.15.148:6824/4078652242,v1:172.21.15.148:6825/4078652242] up:boot 2024-03-29T08:18:09.807 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:18:09 smithi148 ceph-mon[106692]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-03-29T08:18:09.807 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:18:09 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi148.sqteih"}]: dispatch 2024-03-29T08:18:09.807 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:18:09 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:18:09.807 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:18:09 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:18:09.807 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:18:09 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:18:09.807 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:18:09 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:18:10.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:18:10 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:18:10.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:18:10 smithi037 ceph-mon[142325]: pgmap v218: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:18:10.743 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:18:10 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:18:10.743 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:18:10 smithi148 ceph-mon[106692]: pgmap v218: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:18:12.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:18:11 smithi148 ceph-mon[106692]: pgmap v219: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:18:12.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:18:11 smithi037 ceph-mon[142325]: pgmap v219: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:18:14.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:18:13 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:18:14.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:18:13 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:18:14.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:18:13 smithi148 ceph-mon[106692]: pgmap v220: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:18:14.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:18:13 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:18:14.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:18:13 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:18:14.178 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:18:13 smithi037 ceph-mon[142325]: pgmap v220: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:18:15.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:18:14 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:18:15.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:18:14 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:18:15.261 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:18:14 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:18:15.261 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:18:14 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:18:16.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:18:15 smithi148 ceph-mon[106692]: pgmap v221: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:18:16.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:18:15 smithi037 ceph-mon[142325]: pgmap v221: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:18:18.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:18:17 smithi148 ceph-mon[106692]: pgmap v222: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:18:18.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:18:17 smithi037 ceph-mon[142325]: pgmap v222: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:18:20.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:18:20 smithi037 ceph-mon[142325]: pgmap v223: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:18:20.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:18:20 smithi148 ceph-mon[106692]: pgmap v223: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:18:22.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:18:21 smithi148 ceph-mon[106692]: pgmap v224: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:18:22.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:18:21 smithi037 ceph-mon[142325]: pgmap v224: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:18:24.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:18:23 smithi148 ceph-mon[106692]: pgmap v225: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:18:24.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:18:23 smithi037 ceph-mon[142325]: pgmap v225: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:18:25.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:18:25 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:18:25.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:18:25 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:18:26.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:18:26 smithi037 ceph-mon[142325]: pgmap v226: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:18:26.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:18:26 smithi148 ceph-mon[106692]: pgmap v226: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:18:28.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:18:27 smithi148 ceph-mon[106692]: pgmap v227: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:18:28.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:18:27 smithi037 ceph-mon[142325]: pgmap v227: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:18:28.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:28.229+0000 7f59e4e00700 1 -- 172.21.15.37:0/2452308552 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f59e01035d0 msgr2=0x7f59e00fcc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:28.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:28.229+0000 7f59e4e00700 1 --2- 172.21.15.37:0/2452308552 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f59e01035d0 0x7f59e00fcc40 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f59cc004940 tx=0x7f59cc009e30 comp rx=0 tx=0).stop 2024-03-29T08:18:28.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:28.230+0000 7f59e4e00700 1 -- 172.21.15.37:0/2452308552 shutdown_connections 2024-03-29T08:18:28.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:28.230+0000 7f59e4e00700 1 --2- 172.21.15.37:0/2452308552 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f59e01035d0 0x7f59e00fcc40 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:28.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:28.230+0000 7f59e4e00700 1 --2- 172.21.15.37:0/2452308552 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f59e0102c30 0x7f59e0103000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:28.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:28.230+0000 7f59e4e00700 1 -- 172.21.15.37:0/2452308552 >> 172.21.15.37:0/2452308552 conn(0x7f59e00f87a0 msgr2=0x7f59e00fabb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:28.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:28.230+0000 7f59e4e00700 1 -- 172.21.15.37:0/2452308552 shutdown_connections 2024-03-29T08:18:28.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:28.230+0000 7f59e4e00700 1 -- 172.21.15.37:0/2452308552 wait complete. 2024-03-29T08:18:28.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:28.230+0000 7f59e4e00700 1 Processor -- start 2024-03-29T08:18:28.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:28.231+0000 7f59e4e00700 1 -- start start 2024-03-29T08:18:28.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:28.231+0000 7f59e4e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f59e0102c30 0x7f59e0115a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:28.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:28.231+0000 7f59e4e00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f59e01035d0 0x7f59e0115fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:28.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:28.231+0000 7f59e4e00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59e0116600 con 0x7f59e0102c30 2024-03-29T08:18:28.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:28.231+0000 7f59e4e00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59e010f9f0 con 0x7f59e01035d0 2024-03-29T08:18:28.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:28.231+0000 7f59df400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f59e0102c30 0x7f59e0115a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:28.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:28.231+0000 7f59df400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f59e0102c30 0x7f59e0115a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55884/0 (socket says 172.21.15.37:55884) 2024-03-29T08:18:28.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:28.231+0000 7f59df400700 1 -- 172.21.15.37:0/1722080600 learned_addr learned my addr 172.21.15.37:0/1722080600 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:18:28.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:28.232+0000 7f59dea00700 1 --2- 172.21.15.37:0/1722080600 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f59e01035d0 0x7f59e0115fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:28.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:28.232+0000 7f59df400700 1 -- 172.21.15.37:0/1722080600 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f59e01035d0 msgr2=0x7f59e0115fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:28.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:28.232+0000 7f59df400700 1 --2- 172.21.15.37:0/1722080600 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f59e01035d0 0x7f59e0115fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:28.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:28.232+0000 7f59df400700 1 -- 172.21.15.37:0/1722080600 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59cc005040 con 0x7f59e0102c30 2024-03-29T08:18:28.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:28.232+0000 7f59df400700 1 --2- 172.21.15.37:0/1722080600 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f59e0102c30 0x7f59e0115a70 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f59d0002a80 tx=0x7f59d00046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:28.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:28.232+0000 7f59d7400700 1 -- 172.21.15.37:0/1722080600 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f59d0007cc0 con 0x7f59e0102c30 2024-03-29T08:18:28.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:28.232+0000 7f59d7400700 1 -- 172.21.15.37:0/1722080600 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f59d000f040 con 0x7f59e0102c30 2024-03-29T08:18:28.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:28.232+0000 7f59d7400700 1 -- 172.21.15.37:0/1722080600 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f59d0009640 con 0x7f59e0102c30 2024-03-29T08:18:28.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:28.232+0000 7f59e4e00700 1 -- 172.21.15.37:0/1722080600 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59e010fcd0 con 0x7f59e0102c30 2024-03-29T08:18:28.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:28.233+0000 7f59e4e00700 1 -- 172.21.15.37:0/1722080600 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59e0110290 con 0x7f59e0102c30 2024-03-29T08:18:28.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:28.234+0000 7f59e4e00700 1 -- 172.21.15.37:0/1722080600 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59e00fe380 con 0x7f59e0102c30 2024-03-29T08:18:28.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:28.235+0000 7f59d7400700 1 -- 172.21.15.37:0/1722080600 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f59d00097a0 con 0x7f59e0102c30 2024-03-29T08:18:28.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:28.235+0000 7f59d7400700 1 --2- 172.21.15.37:0/1722080600 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f59c406cc30 0x7f59c406f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:28.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:28.236+0000 7f59d7400700 1 -- 172.21.15.37:0/1722080600 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f59d008b7e0 con 0x7f59e0102c30 2024-03-29T08:18:28.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:28.236+0000 7f59dea00700 1 --2- 172.21.15.37:0/1722080600 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f59c406cc30 0x7f59c406f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:28.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:28.236+0000 7f59dea00700 1 --2- 172.21.15.37:0/1722080600 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f59c406cc30 0x7f59c406f0e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f59e0110bb0 tx=0x7f59cc009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:28.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:28.241+0000 7f59d7400700 1 -- 172.21.15.37:0/1722080600 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f59d0059230 con 0x7f59e0102c30 2024-03-29T08:18:28.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:28.473+0000 7f59e4e00700 1 -- 172.21.15.37:0/1722080600 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f59e0109bc0 con 0x7f59c406cc30 2024-03-29T08:18:28.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:28.475+0000 7f59d7400700 1 -- 172.21.15.37:0/1722080600 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f59e0109bc0 con 0x7f59c406cc30 2024-03-29T08:18:28.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:28.477+0000 7f59e4e00700 1 -- 172.21.15.37:0/1722080600 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f59c406cc30 msgr2=0x7f59c406f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:28.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:28.477+0000 7f59e4e00700 1 --2- 172.21.15.37:0/1722080600 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f59c406cc30 0x7f59c406f0e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f59e0110bb0 tx=0x7f59cc009e30 comp rx=0 tx=0).stop 2024-03-29T08:18:28.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:28.477+0000 7f59e4e00700 1 -- 172.21.15.37:0/1722080600 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f59e0102c30 msgr2=0x7f59e0115a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:28.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:28.477+0000 7f59e4e00700 1 --2- 172.21.15.37:0/1722080600 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f59e0102c30 0x7f59e0115a70 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f59d0002a80 tx=0x7f59d00046b0 comp rx=0 tx=0).stop 2024-03-29T08:18:28.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:28.477+0000 7f59e4e00700 1 -- 172.21.15.37:0/1722080600 shutdown_connections 2024-03-29T08:18:28.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:28.477+0000 7f59e4e00700 1 --2- 172.21.15.37:0/1722080600 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f59c406cc30 0x7f59c406f0e0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:28.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:28.477+0000 7f59e4e00700 1 --2- 172.21.15.37:0/1722080600 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f59e0102c30 0x7f59e0115a70 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:28.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:28.477+0000 7f59e4e00700 1 --2- 172.21.15.37:0/1722080600 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f59e01035d0 0x7f59e0115fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:28.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:28.477+0000 7f59e4e00700 1 -- 172.21.15.37:0/1722080600 >> 172.21.15.37:0/1722080600 conn(0x7f59e00f87a0 msgr2=0x7f59e0070480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:28.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:28.477+0000 7f59e4e00700 1 -- 172.21.15.37:0/1722080600 shutdown_connections 2024-03-29T08:18:28.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:28.477+0000 7f59e4e00700 1 -- 172.21.15.37:0/1722080600 wait complete. 2024-03-29T08:18:28.493 INFO:teuthology.orchestra.run.smithi037.stdout:true 2024-03-29T08:18:28.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:28.615+0000 7fcd25a00700 1 -- 172.21.15.37:0/2799918197 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcd201037a0 msgr2=0x7fcd200fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:28.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:28.615+0000 7fcd25a00700 1 --2- 172.21.15.37:0/2799918197 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcd201037a0 0x7fcd200fce10 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fcd0c004940 tx=0x7fcd0c009e30 comp rx=0 tx=0).stop 2024-03-29T08:18:28.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:28.615+0000 7fcd25a00700 1 -- 172.21.15.37:0/2799918197 shutdown_connections 2024-03-29T08:18:28.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:28.615+0000 7fcd25a00700 1 --2- 172.21.15.37:0/2799918197 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcd201037a0 0x7fcd200fce10 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:28.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:28.615+0000 7fcd25a00700 1 --2- 172.21.15.37:0/2799918197 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcd20102e00 0x7fcd201031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:28.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:28.615+0000 7fcd25a00700 1 -- 172.21.15.37:0/2799918197 >> 172.21.15.37:0/2799918197 conn(0x7fcd200f89b0 msgr2=0x7fcd200fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:28.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:28.616+0000 7fcd25a00700 1 -- 172.21.15.37:0/2799918197 shutdown_connections 2024-03-29T08:18:28.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:28.616+0000 7fcd25a00700 1 -- 172.21.15.37:0/2799918197 wait complete. 2024-03-29T08:18:28.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:28.616+0000 7fcd25a00700 1 Processor -- start 2024-03-29T08:18:28.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:28.617+0000 7fcd25a00700 1 -- start start 2024-03-29T08:18:28.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:28.617+0000 7fcd25a00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcd20102e00 0x7fcd200702c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:28.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:28.617+0000 7fcd25a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcd201037a0 0x7fcd20070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:28.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:28.617+0000 7fcd25a00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd20070e20 con 0x7fcd201037a0 2024-03-29T08:18:28.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:28.617+0000 7fcd25a00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd20070f60 con 0x7fcd20102e00 2024-03-29T08:18:28.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:28.617+0000 7fcd1e000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcd201037a0 0x7fcd20070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:28.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:28.617+0000 7fcd1e000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcd201037a0 0x7fcd20070800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55896/0 (socket says 172.21.15.37:55896) 2024-03-29T08:18:28.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:28.617+0000 7fcd1e000700 1 -- 172.21.15.37:0/2901672030 learned_addr learned my addr 172.21.15.37:0/2901672030 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:18:28.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:28.617+0000 7fcd1ea00700 1 --2- 172.21.15.37:0/2901672030 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcd20102e00 0x7fcd200702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:28.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:28.618+0000 7fcd1e000700 1 -- 172.21.15.37:0/2901672030 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcd20102e00 msgr2=0x7fcd200702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:28.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:28.618+0000 7fcd1e000700 1 --2- 172.21.15.37:0/2901672030 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcd20102e00 0x7fcd200702c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:28.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:28.618+0000 7fcd1e000700 1 -- 172.21.15.37:0/2901672030 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd0c005040 con 0x7fcd201037a0 2024-03-29T08:18:28.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:28.618+0000 7fcd1e000700 1 --2- 172.21.15.37:0/2901672030 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcd201037a0 0x7fcd20070800 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fcd0c009390 tx=0x7fcd0c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:28.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:28.618+0000 7fcd16a00700 1 -- 172.21.15.37:0/2901672030 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcd0c00e040 con 0x7fcd201037a0 2024-03-29T08:18:28.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:28.618+0000 7fcd16a00700 1 -- 172.21.15.37:0/2901672030 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fcd0c013420 con 0x7fcd201037a0 2024-03-29T08:18:28.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:28.618+0000 7fcd25a00700 1 -- 172.21.15.37:0/2901672030 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd20119de0 con 0x7fcd201037a0 2024-03-29T08:18:28.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:28.618+0000 7fcd16a00700 1 -- 172.21.15.37:0/2901672030 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcd0c01c630 con 0x7fcd201037a0 2024-03-29T08:18:28.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:28.618+0000 7fcd25a00700 1 -- 172.21.15.37:0/2901672030 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd2011a2c0 con 0x7fcd201037a0 2024-03-29T08:18:28.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:28.619+0000 7fcd25a00700 1 -- 172.21.15.37:0/2901672030 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd200fe550 con 0x7fcd201037a0 2024-03-29T08:18:28.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:28.620+0000 7fcd16a00700 1 -- 172.21.15.37:0/2901672030 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fcd0c013a00 con 0x7fcd201037a0 2024-03-29T08:18:28.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:28.621+0000 7fcd16a00700 1 --2- 172.21.15.37:0/2901672030 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fcd0406cc30 0x7fcd0406f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:28.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:28.621+0000 7fcd16a00700 1 -- 172.21.15.37:0/2901672030 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fcd0c08d2a0 con 0x7fcd201037a0 2024-03-29T08:18:28.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:28.621+0000 7fcd1ea00700 1 --2- 172.21.15.37:0/2901672030 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fcd0406cc30 0x7fcd0406f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:28.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:28.622+0000 7fcd1ea00700 1 --2- 172.21.15.37:0/2901672030 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fcd0406cc30 0x7fcd0406f0e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fcd10006fd0 tx=0x7fcd10008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:28.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:28.626+0000 7fcd16a00700 1 -- 172.21.15.37:0/2901672030 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fcd0c0572e0 con 0x7fcd201037a0 2024-03-29T08:18:28.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:28.860+0000 7fcd25a00700 1 -- 172.21.15.37:0/2901672030 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fcd20109e80 con 0x7fcd0406cc30 2024-03-29T08:18:28.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:28.861+0000 7fcd16a00700 1 -- 172.21.15.37:0/2901672030 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fcd20109e80 con 0x7fcd0406cc30 2024-03-29T08:18:28.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:28.863+0000 7fcd25a00700 1 -- 172.21.15.37:0/2901672030 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fcd0406cc30 msgr2=0x7fcd0406f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:28.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:28.863+0000 7fcd25a00700 1 --2- 172.21.15.37:0/2901672030 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fcd0406cc30 0x7fcd0406f0e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fcd10006fd0 tx=0x7fcd10008040 comp rx=0 tx=0).stop 2024-03-29T08:18:28.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:28.863+0000 7fcd25a00700 1 -- 172.21.15.37:0/2901672030 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcd201037a0 msgr2=0x7fcd20070800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:28.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:28.863+0000 7fcd25a00700 1 --2- 172.21.15.37:0/2901672030 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcd201037a0 0x7fcd20070800 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fcd0c009390 tx=0x7fcd0c009e30 comp rx=0 tx=0).stop 2024-03-29T08:18:28.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:28.863+0000 7fcd25a00700 1 -- 172.21.15.37:0/2901672030 shutdown_connections 2024-03-29T08:18:28.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:28.863+0000 7fcd25a00700 1 --2- 172.21.15.37:0/2901672030 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fcd0406cc30 0x7fcd0406f0e0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:28.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:28.863+0000 7fcd25a00700 1 --2- 172.21.15.37:0/2901672030 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcd20102e00 0x7fcd200702c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:28.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:28.863+0000 7fcd25a00700 1 --2- 172.21.15.37:0/2901672030 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcd201037a0 0x7fcd20070800 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:28.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:28.863+0000 7fcd25a00700 1 -- 172.21.15.37:0/2901672030 >> 172.21.15.37:0/2901672030 conn(0x7fcd200f89b0 msgr2=0x7fcd200f99c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:28.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:28.863+0000 7fcd25a00700 1 -- 172.21.15.37:0/2901672030 shutdown_connections 2024-03-29T08:18:28.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:28.863+0000 7fcd25a00700 1 -- 172.21.15.37:0/2901672030 wait complete. 2024-03-29T08:18:29.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:29.000+0000 7fd51a800700 1 -- 172.21.15.37:0/1461849201 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd5140fde90 msgr2=0x7fd5140fe260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:29.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:29.000+0000 7fd51a800700 1 --2- 172.21.15.37:0/1461849201 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd5140fde90 0x7fd5140fe260 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fd504004940 tx=0x7fd504009e30 comp rx=0 tx=0).stop 2024-03-29T08:18:29.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:29.000+0000 7fd51a800700 1 -- 172.21.15.37:0/1461849201 shutdown_connections 2024-03-29T08:18:29.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:29.000+0000 7fd51a800700 1 --2- 172.21.15.37:0/1461849201 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd5140fe830 0x7fd514106f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:29.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:29.000+0000 7fd51a800700 1 --2- 172.21.15.37:0/1461849201 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd5140fde90 0x7fd5140fe260 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:29.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:29.000+0000 7fd51a800700 1 -- 172.21.15.37:0/1461849201 >> 172.21.15.37:0/1461849201 conn(0x7fd514072bc0 msgr2=0x7fd514072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:29.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:29.000+0000 7fd51a800700 1 -- 172.21.15.37:0/1461849201 shutdown_connections 2024-03-29T08:18:29.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:29.000+0000 7fd51a800700 1 -- 172.21.15.37:0/1461849201 wait complete. 2024-03-29T08:18:29.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:29.001+0000 7fd51a800700 1 Processor -- start 2024-03-29T08:18:29.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:29.001+0000 7fd51a800700 1 -- start start 2024-03-29T08:18:29.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:29.002+0000 7fd51a800700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd5140fde90 0x7fd514113af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:29.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:29.002+0000 7fd51a800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd5140fe830 0x7fd514114030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:29.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:29.002+0000 7fd51a800700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd514114710 con 0x7fd5140fe830 2024-03-29T08:18:29.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:29.002+0000 7fd51a800700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd51410da70 con 0x7fd5140fde90 2024-03-29T08:18:29.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:29.002+0000 7fd512a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd5140fe830 0x7fd514114030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:29.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:29.002+0000 7fd512a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd5140fe830 0x7fd514114030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55908/0 (socket says 172.21.15.37:55908) 2024-03-29T08:18:29.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:29.002+0000 7fd512a00700 1 -- 172.21.15.37:0/2695824115 learned_addr learned my addr 172.21.15.37:0/2695824115 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:18:29.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:29.002+0000 7fd513400700 1 --2- 172.21.15.37:0/2695824115 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd5140fde90 0x7fd514113af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:29.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:29.002+0000 7fd512a00700 1 -- 172.21.15.37:0/2695824115 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd5140fde90 msgr2=0x7fd514113af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:29.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:29.002+0000 7fd512a00700 1 --2- 172.21.15.37:0/2695824115 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd5140fde90 0x7fd514113af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:29.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:29.002+0000 7fd512a00700 1 -- 172.21.15.37:0/2695824115 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd504005040 con 0x7fd5140fe830 2024-03-29T08:18:29.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:29.003+0000 7fd512a00700 1 --2- 172.21.15.37:0/2695824115 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd5140fe830 0x7fd514114030 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fd50000b790 tx=0x7fd500009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:29.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:29.003+0000 7fd50b400700 1 -- 172.21.15.37:0/2695824115 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd50000f770 con 0x7fd5140fe830 2024-03-29T08:18:29.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:29.003+0000 7fd50b400700 1 -- 172.21.15.37:0/2695824115 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd50000fd80 con 0x7fd5140fe830 2024-03-29T08:18:29.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:29.003+0000 7fd51a800700 1 -- 172.21.15.37:0/2695824115 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd51410dd50 con 0x7fd5140fe830 2024-03-29T08:18:29.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:29.003+0000 7fd50b400700 1 -- 172.21.15.37:0/2695824115 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd50000d310 con 0x7fd5140fe830 2024-03-29T08:18:29.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:29.003+0000 7fd51a800700 1 -- 172.21.15.37:0/2695824115 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd51410e290 con 0x7fd5140fe830 2024-03-29T08:18:29.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:29.004+0000 7fd51a800700 1 -- 172.21.15.37:0/2695824115 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd514065e30 con 0x7fd5140fe830 2024-03-29T08:18:29.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:29.005+0000 7fd50b400700 1 -- 172.21.15.37:0/2695824115 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fd500010420 con 0x7fd5140fe830 2024-03-29T08:18:29.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:29.006+0000 7fd50b400700 1 --2- 172.21.15.37:0/2695824115 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fd4f806cc30 0x7fd4f806f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:29.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:29.006+0000 7fd50b400700 1 -- 172.21.15.37:0/2695824115 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fd50008c040 con 0x7fd5140fe830 2024-03-29T08:18:29.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:29.006+0000 7fd513400700 1 --2- 172.21.15.37:0/2695824115 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fd4f806cc30 0x7fd4f806f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:29.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:29.007+0000 7fd513400700 1 --2- 172.21.15.37:0/2695824115 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fd4f806cc30 0x7fd4f806f0e0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fd50400b3b0 tx=0x7fd504009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:29.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:29.011+0000 7fd50b400700 1 -- 172.21.15.37:0/2695824115 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fd500059b10 con 0x7fd5140fe830 2024-03-29T08:18:29.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:18:28 smithi148 ceph-mon[106692]: from='client.34476 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:18:29.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:18:28 smithi037 ceph-mon[142325]: from='client.34476 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:18:29.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:29.232+0000 7fd51a800700 1 -- 172.21.15.37:0/2695824115 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fd514109e40 con 0x7fd4f806cc30 2024-03-29T08:18:29.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:29.242+0000 7fd50b400700 1 -- 172.21.15.37:0/2695824115 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7fd514109e40 con 0x7fd4f806cc30 2024-03-29T08:18:29.243 INFO:teuthology.orchestra.run.smithi037.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-29T08:18:29.243 INFO:teuthology.orchestra.run.smithi037.stdout:alertmanager.smithi037 smithi037 *:9093,9094 running (8m) 37s ago 21m 14.9M - 0.25.0 c8568f914cd2 2f9f9746ed0f 2024-03-29T08:18:29.243 INFO:teuthology.orchestra.run.smithi037.stdout:crash.smithi037 smithi037 running (5m) 37s ago 21m 6647k - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 178d416a3186 2024-03-29T08:18:29.244 INFO:teuthology.orchestra.run.smithi037.stdout:crash.smithi148 smithi148 running (5m) 16s ago 20m 6656k - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 23666989d58f 2024-03-29T08:18:29.244 INFO:teuthology.orchestra.run.smithi037.stdout:grafana.smithi037 smithi037 *:3000 running (8m) 37s ago 20m 71.9M - 9.4.7 954c08fa6188 6b976fe49952 2024-03-29T08:18:29.244 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi037.iifsmh smithi037 running (43s) 37s ago 14m 10.8M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 2d97f562cc5b 2024-03-29T08:18:29.244 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi037.mmwgxe smithi037 running (63s) 37s ago 14m 10.9M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff ec37037f46e8 2024-03-29T08:18:29.244 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi148.qukpjh smithi148 running (14m) 16s ago 14m 15.3M - 17.2.7-789-g4213e17c cf22eb8683be 8e469782a273 2024-03-29T08:18:29.244 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi148.sqteih smithi148 running (21s) 16s ago 14m 10.7M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 44c15c77037a 2024-03-29T08:18:29.244 INFO:teuthology.orchestra.run.smithi037.stdout:mgr.smithi037.wjikii smithi037 *:8443,9283,8765 running (10m) 37s ago 22m 506M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff c5a1417b9417 2024-03-29T08:18:29.244 INFO:teuthology.orchestra.run.smithi037.stdout:mgr.smithi148.onhgvm smithi148 *:8443,9283,8765 running (9m) 16s ago 20m 450M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff fcfdbce062bd 2024-03-29T08:18:29.244 INFO:teuthology.orchestra.run.smithi037.stdout:mon.smithi037 smithi037 running (6m) 37s ago 23m 54.8M 2048M 18.2.2-1015-gd0e392d9 1cc2f45c17ff eea2060783af 2024-03-29T08:18:29.244 INFO:teuthology.orchestra.run.smithi037.stdout:mon.smithi148 smithi148 running (5m) 16s ago 20m 43.7M 2048M 18.2.2-1015-gd0e392d9 1cc2f45c17ff ad4b2c5748be 2024-03-29T08:18:29.244 INFO:teuthology.orchestra.run.smithi037.stdout:node-exporter.smithi037 smithi037 *:9100 running (9m) 37s ago 21m 7625k - 1.5.0 0da6a335fe13 26b8ae9cb878 2024-03-29T08:18:29.244 INFO:teuthology.orchestra.run.smithi037.stdout:node-exporter.smithi148 smithi148 *:9100 running (9m) 16s ago 20m 8166k - 1.5.0 0da6a335fe13 98414e4e64ed 2024-03-29T08:18:29.244 INFO:teuthology.orchestra.run.smithi037.stdout:osd.0 smithi037 running (4m) 37s ago 19m 104M 1619M 18.2.2-1015-gd0e392d9 1cc2f45c17ff c0c584f6245c 2024-03-29T08:18:29.244 INFO:teuthology.orchestra.run.smithi037.stdout:osd.1 smithi037 running (4m) 37s ago 18m 100M 1619M 18.2.2-1015-gd0e392d9 1cc2f45c17ff a4a2ff4cc83d 2024-03-29T08:18:29.245 INFO:teuthology.orchestra.run.smithi037.stdout:osd.2 smithi037 running (3m) 37s ago 18m 82.3M 1619M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 8b1bc71aa79e 2024-03-29T08:18:29.245 INFO:teuthology.orchestra.run.smithi037.stdout:osd.3 smithi148 running (2m) 16s ago 17m 105M 2642M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 1c656ab8ff86 2024-03-29T08:18:29.245 INFO:teuthology.orchestra.run.smithi037.stdout:osd.4 smithi148 running (2m) 16s ago 17m 97.3M 2642M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 966a9de11d22 2024-03-29T08:18:29.245 INFO:teuthology.orchestra.run.smithi037.stdout:osd.5 smithi148 running (84s) 16s ago 16m 85.8M 2642M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 474c4f9212ce 2024-03-29T08:18:29.245 INFO:teuthology.orchestra.run.smithi037.stdout:prometheus.smithi037 smithi037 *:9095 running (8m) 37s ago 20m 54.4M - 2.43.0 a07b618ecd1d 30c0223ec589 2024-03-29T08:18:29.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:29.245+0000 7fd51a800700 1 -- 172.21.15.37:0/2695824115 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fd4f806cc30 msgr2=0x7fd4f806f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:29.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:29.245+0000 7fd51a800700 1 --2- 172.21.15.37:0/2695824115 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fd4f806cc30 0x7fd4f806f0e0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fd50400b3b0 tx=0x7fd504009e30 comp rx=0 tx=0).stop 2024-03-29T08:18:29.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:29.245+0000 7fd51a800700 1 -- 172.21.15.37:0/2695824115 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd5140fe830 msgr2=0x7fd514114030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:29.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:29.245+0000 7fd51a800700 1 --2- 172.21.15.37:0/2695824115 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd5140fe830 0x7fd514114030 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fd50000b790 tx=0x7fd500009250 comp rx=0 tx=0).stop 2024-03-29T08:18:29.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:29.245+0000 7fd51a800700 1 -- 172.21.15.37:0/2695824115 shutdown_connections 2024-03-29T08:18:29.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:29.245+0000 7fd51a800700 1 --2- 172.21.15.37:0/2695824115 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fd4f806cc30 0x7fd4f806f0e0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:29.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:29.245+0000 7fd51a800700 1 --2- 172.21.15.37:0/2695824115 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd5140fde90 0x7fd514113af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:29.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:29.245+0000 7fd51a800700 1 --2- 172.21.15.37:0/2695824115 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd5140fe830 0x7fd514114030 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:29.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:29.245+0000 7fd51a800700 1 -- 172.21.15.37:0/2695824115 >> 172.21.15.37:0/2695824115 conn(0x7fd514072bc0 msgr2=0x7fd5141037c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:29.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:29.245+0000 7fd51a800700 1 -- 172.21.15.37:0/2695824115 shutdown_connections 2024-03-29T08:18:29.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:29.245+0000 7fd51a800700 1 -- 172.21.15.37:0/2695824115 wait complete. 2024-03-29T08:18:29.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:29.382+0000 7fb482800700 1 -- 172.21.15.37:0/3801305795 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb47c102ca0 msgr2=0x7fb47c103070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:29.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:29.382+0000 7fb482800700 1 --2- 172.21.15.37:0/3801305795 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb47c102ca0 0x7fb47c103070 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fb470004940 tx=0x7fb470009e30 comp rx=0 tx=0).stop 2024-03-29T08:18:29.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:29.382+0000 7fb482800700 1 -- 172.21.15.37:0/3801305795 shutdown_connections 2024-03-29T08:18:29.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:29.382+0000 7fb482800700 1 --2- 172.21.15.37:0/3801305795 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb47c065630 0x7fb47c065a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:29.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:29.382+0000 7fb482800700 1 --2- 172.21.15.37:0/3801305795 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb47c102ca0 0x7fb47c103070 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:29.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:29.382+0000 7fb482800700 1 -- 172.21.15.37:0/3801305795 >> 172.21.15.37:0/3801305795 conn(0x7fb47c071b70 msgr2=0x7fb47c071f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:29.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:29.382+0000 7fb482800700 1 -- 172.21.15.37:0/3801305795 shutdown_connections 2024-03-29T08:18:29.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:29.382+0000 7fb482800700 1 -- 172.21.15.37:0/3801305795 wait complete. 2024-03-29T08:18:29.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:29.383+0000 7fb482800700 1 Processor -- start 2024-03-29T08:18:29.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:29.383+0000 7fb482800700 1 -- start start 2024-03-29T08:18:29.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:29.383+0000 7fb482800700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb47c065630 0x7fb47c115a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:29.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:29.383+0000 7fb482800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb47c102ca0 0x7fb47c115f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:29.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:29.383+0000 7fb482800700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb47c1165c0 con 0x7fb47c102ca0 2024-03-29T08:18:29.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:29.383+0000 7fb482800700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb47c10f9b0 con 0x7fb47c065630 2024-03-29T08:18:29.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:29.384+0000 7fb480a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb47c102ca0 0x7fb47c115f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:29.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:29.384+0000 7fb480a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb47c102ca0 0x7fb47c115f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55928/0 (socket says 172.21.15.37:55928) 2024-03-29T08:18:29.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:29.384+0000 7fb480a00700 1 -- 172.21.15.37:0/1462787226 learned_addr learned my addr 172.21.15.37:0/1462787226 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:18:29.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:29.384+0000 7fb481400700 1 --2- 172.21.15.37:0/1462787226 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb47c065630 0x7fb47c115a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:29.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:29.384+0000 7fb480a00700 1 -- 172.21.15.37:0/1462787226 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb47c065630 msgr2=0x7fb47c115a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:29.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:29.384+0000 7fb480a00700 1 --2- 172.21.15.37:0/1462787226 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb47c065630 0x7fb47c115a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:29.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:29.384+0000 7fb480a00700 1 -- 172.21.15.37:0/1462787226 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb470005040 con 0x7fb47c102ca0 2024-03-29T08:18:29.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:29.384+0000 7fb480a00700 1 --2- 172.21.15.37:0/1462787226 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb47c102ca0 0x7fb47c115f70 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fb46800eb40 tx=0x7fb46800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:29.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:29.385+0000 7fb479600700 1 -- 172.21.15.37:0/1462787226 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb468007cc0 con 0x7fb47c102ca0 2024-03-29T08:18:29.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:29.385+0000 7fb479600700 1 -- 172.21.15.37:0/1462787226 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fb46800f040 con 0x7fb47c102ca0 2024-03-29T08:18:29.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:29.385+0000 7fb479600700 1 -- 172.21.15.37:0/1462787226 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb46800c990 con 0x7fb47c102ca0 2024-03-29T08:18:29.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:29.385+0000 7fb482800700 1 -- 172.21.15.37:0/1462787226 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb47c10fc30 con 0x7fb47c102ca0 2024-03-29T08:18:29.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:29.385+0000 7fb482800700 1 -- 172.21.15.37:0/1462787226 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb47c1101f0 con 0x7fb47c102ca0 2024-03-29T08:18:29.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:29.386+0000 7fb482800700 1 -- 172.21.15.37:0/1462787226 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb47c0fe380 con 0x7fb47c102ca0 2024-03-29T08:18:29.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:29.387+0000 7fb479600700 1 -- 172.21.15.37:0/1462787226 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fb46800caf0 con 0x7fb47c102ca0 2024-03-29T08:18:29.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:29.388+0000 7fb479600700 1 --2- 172.21.15.37:0/1462787226 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fb46406cc30 0x7fb46406f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:29.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:29.388+0000 7fb479600700 1 -- 172.21.15.37:0/1462787226 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fb46808bc00 con 0x7fb47c102ca0 2024-03-29T08:18:29.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:29.388+0000 7fb481400700 1 --2- 172.21.15.37:0/1462787226 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fb46406cc30 0x7fb46406f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:29.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:29.389+0000 7fb481400700 1 --2- 172.21.15.37:0/1462787226 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fb46406cc30 0x7fb46406f0e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fb4700045a0 tx=0x7fb470009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:29.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:29.393+0000 7fb479600700 1 -- 172.21.15.37:0/1462787226 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fb468059700 con 0x7fb47c102ca0 2024-03-29T08:18:29.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:29.693+0000 7fb482800700 1 -- 172.21.15.37:0/1462787226 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fb47c109ba0 con 0x7fb47c102ca0 2024-03-29T08:18:29.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:29.693+0000 7fb479600700 1 -- 172.21.15.37:0/1462787226 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+847 (secure 0 0 0) 0x7fb468059290 con 0x7fb47c102ca0 2024-03-29T08:18:29.695 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-29T08:18:29.695 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": { 2024-03-29T08:18:29.695 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 2 2024-03-29T08:18:29.695 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:18:29.695 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": { 2024-03-29T08:18:29.695 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 2 2024-03-29T08:18:29.695 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:18:29.695 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": { 2024-03-29T08:18:29.695 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 6 2024-03-29T08:18:29.695 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:18:29.695 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": { 2024-03-29T08:18:29.695 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 17.2.7-789-g4213e17c (4213e17c4de67016ed14f55d22503f4923b3a901) quincy (stable)": 1, 2024-03-29T08:18:29.695 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 3 2024-03-29T08:18:29.696 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:18:29.696 INFO:teuthology.orchestra.run.smithi037.stdout: "overall": { 2024-03-29T08:18:29.696 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 17.2.7-789-g4213e17c (4213e17c4de67016ed14f55d22503f4923b3a901) quincy (stable)": 1, 2024-03-29T08:18:29.696 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 13 2024-03-29T08:18:29.696 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-29T08:18:29.696 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-29T08:18:29.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:29.696+0000 7fb482800700 1 -- 172.21.15.37:0/1462787226 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fb46406cc30 msgr2=0x7fb46406f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:29.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:29.696+0000 7fb482800700 1 --2- 172.21.15.37:0/1462787226 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fb46406cc30 0x7fb46406f0e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fb4700045a0 tx=0x7fb470009e30 comp rx=0 tx=0).stop 2024-03-29T08:18:29.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:29.696+0000 7fb482800700 1 -- 172.21.15.37:0/1462787226 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb47c102ca0 msgr2=0x7fb47c115f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:29.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:29.696+0000 7fb482800700 1 --2- 172.21.15.37:0/1462787226 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb47c102ca0 0x7fb47c115f70 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fb46800eb40 tx=0x7fb46800b040 comp rx=0 tx=0).stop 2024-03-29T08:18:29.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:29.696+0000 7fb482800700 1 -- 172.21.15.37:0/1462787226 shutdown_connections 2024-03-29T08:18:29.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:29.696+0000 7fb482800700 1 --2- 172.21.15.37:0/1462787226 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fb46406cc30 0x7fb46406f0e0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:29.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:29.696+0000 7fb482800700 1 --2- 172.21.15.37:0/1462787226 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb47c065630 0x7fb47c115a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:29.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:29.696+0000 7fb482800700 1 --2- 172.21.15.37:0/1462787226 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb47c102ca0 0x7fb47c115f70 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:29.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:29.696+0000 7fb482800700 1 -- 172.21.15.37:0/1462787226 >> 172.21.15.37:0/1462787226 conn(0x7fb47c071b70 msgr2=0x7fb47c0f9fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:29.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:29.696+0000 7fb482800700 1 -- 172.21.15.37:0/1462787226 shutdown_connections 2024-03-29T08:18:29.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:29.696+0000 7fb482800700 1 -- 172.21.15.37:0/1462787226 wait complete. 2024-03-29T08:18:29.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:29.834+0000 7f5b07a00700 1 -- 172.21.15.37:0/2017129320 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5b000fe830 msgr2=0x7f5b00106f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:29.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:29.834+0000 7f5b07a00700 1 --2- 172.21.15.37:0/2017129320 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5b000fe830 0x7f5b00106f00 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f5aec004940 tx=0x7f5aec009e30 comp rx=0 tx=0).stop 2024-03-29T08:18:29.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:29.834+0000 7f5b07a00700 1 -- 172.21.15.37:0/2017129320 shutdown_connections 2024-03-29T08:18:29.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:29.834+0000 7f5b07a00700 1 --2- 172.21.15.37:0/2017129320 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5b000fe830 0x7f5b00106f00 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:29.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:29.834+0000 7f5b07a00700 1 --2- 172.21.15.37:0/2017129320 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5b000fde90 0x7f5b000fe260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:29.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:29.834+0000 7f5b07a00700 1 -- 172.21.15.37:0/2017129320 >> 172.21.15.37:0/2017129320 conn(0x7f5b00072bc0 msgr2=0x7f5b00072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:29.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:29.834+0000 7f5b07a00700 1 -- 172.21.15.37:0/2017129320 shutdown_connections 2024-03-29T08:18:29.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:29.834+0000 7f5b07a00700 1 -- 172.21.15.37:0/2017129320 wait complete. 2024-03-29T08:18:29.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:29.835+0000 7f5b07a00700 1 Processor -- start 2024-03-29T08:18:29.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:29.835+0000 7f5b07a00700 1 -- start start 2024-03-29T08:18:29.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:29.836+0000 7f5b07a00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5b000fde90 0x7f5b00115c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:29.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:29.836+0000 7f5b07a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5b000fe830 0x7f5b001161b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:29.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:29.836+0000 7f5b07a00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b00116800 con 0x7f5b000fe830 2024-03-29T08:18:29.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:29.836+0000 7f5b07a00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b0010fbd0 con 0x7f5b000fde90 2024-03-29T08:18:29.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:29.836+0000 7f5affe00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5b000fe830 0x7f5b001161b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:29.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:29.836+0000 7f5affe00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5b000fe830 0x7f5b001161b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55956/0 (socket says 172.21.15.37:55956) 2024-03-29T08:18:29.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:29.836+0000 7f5affe00700 1 -- 172.21.15.37:0/1267266180 learned_addr learned my addr 172.21.15.37:0/1267266180 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:18:29.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:29.836+0000 7f5affe00700 1 -- 172.21.15.37:0/1267266180 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5b000fde90 msgr2=0x7f5b00115c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:29.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:29.836+0000 7f5b05000700 1 --2- 172.21.15.37:0/1267266180 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5b000fde90 0x7f5b00115c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:29.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:29.836+0000 7f5affe00700 1 --2- 172.21.15.37:0/1267266180 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5b000fde90 0x7f5b00115c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:29.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:29.836+0000 7f5affe00700 1 -- 172.21.15.37:0/1267266180 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5aec005040 con 0x7f5b000fe830 2024-03-29T08:18:29.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:29.836+0000 7f5b05000700 1 --2- 172.21.15.37:0/1267266180 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5b000fde90 0x7f5b00115c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:18:29.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:29.837+0000 7f5affe00700 1 --2- 172.21.15.37:0/1267266180 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5b000fe830 0x7f5b001161b0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f5aec000c00 tx=0x7f5aec009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:29.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:29.837+0000 7f5afcc00700 1 -- 172.21.15.37:0/1267266180 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5aec021070 con 0x7f5b000fe830 2024-03-29T08:18:29.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:29.837+0000 7f5afcc00700 1 -- 172.21.15.37:0/1267266180 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f5aec013e00 con 0x7f5b000fe830 2024-03-29T08:18:29.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:29.837+0000 7f5b07a00700 1 -- 172.21.15.37:0/1267266180 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b0010fe50 con 0x7f5b000fe830 2024-03-29T08:18:29.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:29.837+0000 7f5afcc00700 1 -- 172.21.15.37:0/1267266180 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5aec00e070 con 0x7f5b000fe830 2024-03-29T08:18:29.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:29.837+0000 7f5b07a00700 1 -- 172.21.15.37:0/1267266180 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b00110390 con 0x7f5b000fe830 2024-03-29T08:18:29.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:29.838+0000 7f5b07a00700 1 -- 172.21.15.37:0/1267266180 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b00106a20 con 0x7f5b000fe830 2024-03-29T08:18:29.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:29.839+0000 7f5afcc00700 1 -- 172.21.15.37:0/1267266180 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f5aec013950 con 0x7f5b000fe830 2024-03-29T08:18:29.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:29.840+0000 7f5afcc00700 1 --2- 172.21.15.37:0/1267266180 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f5ae806cc30 0x7f5ae806f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:29.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:29.840+0000 7f5afcc00700 1 -- 172.21.15.37:0/1267266180 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f5aec08d050 con 0x7f5b000fe830 2024-03-29T08:18:29.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:29.841+0000 7f5b05000700 1 --2- 172.21.15.37:0/1267266180 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f5ae806cc30 0x7f5ae806f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:29.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:29.841+0000 7f5b05000700 1 --2- 172.21.15.37:0/1267266180 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f5ae806cc30 0x7f5ae806f0e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f5af4004600 tx=0x7f5af400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:29.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:29.845+0000 7f5afcc00700 1 -- 172.21.15.37:0/1267266180 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f5aec05ab20 con 0x7f5b000fe830 2024-03-29T08:18:30.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:30.106+0000 7f5b07a00700 1 -- 172.21.15.37:0/1267266180 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f5b001112c0 con 0x7f5b000fe830 2024-03-29T08:18:30.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:30.107+0000 7f5afcc00700 1 -- 172.21.15.37:0/1267266180 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 28 v28) v1 ==== 76+0+1841 (secure 0 0 0) 0x7f5aec05a6b0 con 0x7f5b000fe830 2024-03-29T08:18:30.109 INFO:teuthology.orchestra.run.smithi037.stdout:e28 2024-03-29T08:18:30.109 INFO:teuthology.orchestra.run.smithi037.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-29T08:18:30.109 INFO:teuthology.orchestra.run.smithi037.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T08:18:30.109 INFO:teuthology.orchestra.run.smithi037.stdout:legacy client fscid: 1 2024-03-29T08:18:30.109 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:18:30.109 INFO:teuthology.orchestra.run.smithi037.stdout:Filesystem 'cephfs' (1) 2024-03-29T08:18:30.110 INFO:teuthology.orchestra.run.smithi037.stdout:fs_name cephfs 2024-03-29T08:18:30.110 INFO:teuthology.orchestra.run.smithi037.stdout:epoch 21 2024-03-29T08:18:30.110 INFO:teuthology.orchestra.run.smithi037.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-03-29T08:18:30.110 INFO:teuthology.orchestra.run.smithi037.stdout:created 2024-03-29T08:03:29.241269+0000 2024-03-29T08:18:30.110 INFO:teuthology.orchestra.run.smithi037.stdout:modified 2024-03-29T08:17:16.237238+0000 2024-03-29T08:18:30.110 INFO:teuthology.orchestra.run.smithi037.stdout:tableserver 0 2024-03-29T08:18:30.110 INFO:teuthology.orchestra.run.smithi037.stdout:root 0 2024-03-29T08:18:30.110 INFO:teuthology.orchestra.run.smithi037.stdout:session_timeout 60 2024-03-29T08:18:30.110 INFO:teuthology.orchestra.run.smithi037.stdout:session_autoclose 300 2024-03-29T08:18:30.110 INFO:teuthology.orchestra.run.smithi037.stdout:max_file_size 1099511627776 2024-03-29T08:18:30.110 INFO:teuthology.orchestra.run.smithi037.stdout:max_xattr_size 65536 2024-03-29T08:18:30.110 INFO:teuthology.orchestra.run.smithi037.stdout:required_client_features {} 2024-03-29T08:18:30.110 INFO:teuthology.orchestra.run.smithi037.stdout:last_failure 0 2024-03-29T08:18:30.110 INFO:teuthology.orchestra.run.smithi037.stdout:last_failure_osd_epoch 82 2024-03-29T08:18:30.111 INFO:teuthology.orchestra.run.smithi037.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T08:18:30.111 INFO:teuthology.orchestra.run.smithi037.stdout:max_mds 2 2024-03-29T08:18:30.111 INFO:teuthology.orchestra.run.smithi037.stdout:in 0,1 2024-03-29T08:18:30.111 INFO:teuthology.orchestra.run.smithi037.stdout:up {} 2024-03-29T08:18:30.111 INFO:teuthology.orchestra.run.smithi037.stdout:failed 0,1 2024-03-29T08:18:30.111 INFO:teuthology.orchestra.run.smithi037.stdout:damaged 2024-03-29T08:18:30.111 INFO:teuthology.orchestra.run.smithi037.stdout:stopped 2024-03-29T08:18:30.111 INFO:teuthology.orchestra.run.smithi037.stdout:data_pools [3] 2024-03-29T08:18:30.111 INFO:teuthology.orchestra.run.smithi037.stdout:metadata_pool 2 2024-03-29T08:18:30.111 INFO:teuthology.orchestra.run.smithi037.stdout:inline_data disabled 2024-03-29T08:18:30.111 INFO:teuthology.orchestra.run.smithi037.stdout:balancer 2024-03-29T08:18:30.111 INFO:teuthology.orchestra.run.smithi037.stdout:bal_rank_mask -1 2024-03-29T08:18:30.111 INFO:teuthology.orchestra.run.smithi037.stdout:standby_count_wanted 1 2024-03-29T08:18:30.112 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:18:30.112 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:18:30.112 INFO:teuthology.orchestra.run.smithi037.stdout:Standby daemons: 2024-03-29T08:18:30.112 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:18:30.112 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi148.qukpjh{-1:24319} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.148:6826/2641327912,v1:172.21.15.148:6827/2641327912] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:18:30.112 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi037.mmwgxe{-1:34440} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.37:6828/1248142005,v1:172.21.15.37:6829/1248142005] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:18:30.112 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi037.iifsmh{-1:34444} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.37:6826/2060882959,v1:172.21.15.37:6827/2060882959] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:18:30.112 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi148.sqteih{-1:44395} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.148:6824/4078652242,v1:172.21.15.148:6825/4078652242] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:18:30.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:30.109+0000 7f5b07a00700 1 -- 172.21.15.37:0/1267266180 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f5ae806cc30 msgr2=0x7f5ae806f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:30.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:30.109+0000 7f5b07a00700 1 --2- 172.21.15.37:0/1267266180 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f5ae806cc30 0x7f5ae806f0e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f5af4004600 tx=0x7f5af400b040 comp rx=0 tx=0).stop 2024-03-29T08:18:30.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:30.109+0000 7f5b07a00700 1 -- 172.21.15.37:0/1267266180 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5b000fe830 msgr2=0x7f5b001161b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:30.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:30.109+0000 7f5b07a00700 1 --2- 172.21.15.37:0/1267266180 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5b000fe830 0x7f5b001161b0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f5aec000c00 tx=0x7f5aec009e30 comp rx=0 tx=0).stop 2024-03-29T08:18:30.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:30.109+0000 7f5b07a00700 1 -- 172.21.15.37:0/1267266180 shutdown_connections 2024-03-29T08:18:30.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:30.109+0000 7f5b07a00700 1 --2- 172.21.15.37:0/1267266180 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f5ae806cc30 0x7f5ae806f0e0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:30.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:30.109+0000 7f5b07a00700 1 --2- 172.21.15.37:0/1267266180 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5b000fde90 0x7f5b00115c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:30.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:30.109+0000 7f5b07a00700 1 --2- 172.21.15.37:0/1267266180 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5b000fe830 0x7f5b001161b0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:30.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:30.109+0000 7f5b07a00700 1 -- 172.21.15.37:0/1267266180 >> 172.21.15.37:0/1267266180 conn(0x7f5b00072bc0 msgr2=0x7f5b000fb6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:30.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:30.109+0000 7f5b07a00700 1 -- 172.21.15.37:0/1267266180 shutdown_connections 2024-03-29T08:18:30.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:30.109+0000 7f5b07a00700 1 -- 172.21.15.37:0/1267266180 wait complete. 2024-03-29T08:18:30.114 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 28 2024-03-29T08:18:30.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:30.249+0000 7f7bbc000700 1 -- 172.21.15.37:0/3575914707 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7bb41037c0 msgr2=0x7f7bb40fce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:30.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:30.249+0000 7f7bbc000700 1 --2- 172.21.15.37:0/3575914707 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7bb41037c0 0x7f7bb40fce30 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f7ba0004940 tx=0x7f7ba0009e30 comp rx=0 tx=0).stop 2024-03-29T08:18:30.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:30.249+0000 7f7bbc000700 1 -- 172.21.15.37:0/3575914707 shutdown_connections 2024-03-29T08:18:30.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:30.249+0000 7f7bbc000700 1 --2- 172.21.15.37:0/3575914707 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7bb41037c0 0x7f7bb40fce30 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:30.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:30.249+0000 7f7bbc000700 1 --2- 172.21.15.37:0/3575914707 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7bb4102e20 0x7f7bb41031f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:30.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:30.249+0000 7f7bbc000700 1 -- 172.21.15.37:0/3575914707 >> 172.21.15.37:0/3575914707 conn(0x7f7bb40f8970 msgr2=0x7f7bb40fad80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:30.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:30.249+0000 7f7bbc000700 1 -- 172.21.15.37:0/3575914707 shutdown_connections 2024-03-29T08:18:30.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:30.249+0000 7f7bbc000700 1 -- 172.21.15.37:0/3575914707 wait complete. 2024-03-29T08:18:30.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:30.251+0000 7f7bbc000700 1 Processor -- start 2024-03-29T08:18:30.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:30.251+0000 7f7bbc000700 1 -- start start 2024-03-29T08:18:30.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:30.251+0000 7f7bbc000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7bb4102e20 0x7f7bb4115cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:30.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:30.251+0000 7f7bbc000700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7bb41037c0 0x7f7bb4116200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:30.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:30.251+0000 7f7bbc000700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bb4116850 con 0x7f7bb4102e20 2024-03-29T08:18:30.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:30.251+0000 7f7bbc000700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bb410fc20 con 0x7f7bb41037c0 2024-03-29T08:18:30.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:30.252+0000 7f7bb9600700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7bb4102e20 0x7f7bb4115cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:30.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:30.252+0000 7f7bb9600700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7bb4102e20 0x7f7bb4115cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34674/0 (socket says 172.21.15.37:34674) 2024-03-29T08:18:30.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:30.252+0000 7f7bb9600700 1 -- 172.21.15.37:0/3846274010 learned_addr learned my addr 172.21.15.37:0/3846274010 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:18:30.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:30.252+0000 7f7bb8c00700 1 --2- 172.21.15.37:0/3846274010 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7bb41037c0 0x7f7bb4116200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:30.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:30.252+0000 7f7bb9600700 1 -- 172.21.15.37:0/3846274010 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7bb41037c0 msgr2=0x7f7bb4116200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:30.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:30.252+0000 7f7bb9600700 1 --2- 172.21.15.37:0/3846274010 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7bb41037c0 0x7f7bb4116200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:30.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:30.252+0000 7f7bb9600700 1 -- 172.21.15.37:0/3846274010 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ba0005040 con 0x7f7bb4102e20 2024-03-29T08:18:30.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:30.252+0000 7f7bb9600700 1 --2- 172.21.15.37:0/3846274010 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7bb4102e20 0x7f7bb4115cc0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f7ba8002a50 tx=0x7f7ba80046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:30.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:30.253+0000 7f7bb1600700 1 -- 172.21.15.37:0/3846274010 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7ba8007cc0 con 0x7f7bb4102e20 2024-03-29T08:18:30.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:30.253+0000 7f7bb1600700 1 -- 172.21.15.37:0/3846274010 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f7ba800f040 con 0x7f7bb4102e20 2024-03-29T08:18:30.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:30.253+0000 7f7bb1600700 1 -- 172.21.15.37:0/3846274010 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7ba800d9c0 con 0x7f7bb4102e20 2024-03-29T08:18:30.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:30.253+0000 7f7bbc000700 1 -- 172.21.15.37:0/3846274010 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bb410ff00 con 0x7f7bb4102e20 2024-03-29T08:18:30.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:30.253+0000 7f7bbc000700 1 -- 172.21.15.37:0/3846274010 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bb41104c0 con 0x7f7bb4102e20 2024-03-29T08:18:30.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:30.254+0000 7f7bbc000700 1 -- 172.21.15.37:0/3846274010 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7bb40fe570 con 0x7f7bb4102e20 2024-03-29T08:18:30.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:30.255+0000 7f7bb1600700 1 -- 172.21.15.37:0/3846274010 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f7ba800db20 con 0x7f7bb4102e20 2024-03-29T08:18:30.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:30.256+0000 7f7bb1600700 1 --2- 172.21.15.37:0/3846274010 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f7b9c06cc30 0x7f7b9c06f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:30.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:30.256+0000 7f7bb1600700 1 -- 172.21.15.37:0/3846274010 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f7ba808bd60 con 0x7f7bb4102e20 2024-03-29T08:18:30.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:30.256+0000 7f7bb8c00700 1 --2- 172.21.15.37:0/3846274010 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f7b9c06cc30 0x7f7b9c06f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:30.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:30.257+0000 7f7bb8c00700 1 --2- 172.21.15.37:0/3846274010 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f7b9c06cc30 0x7f7b9c06f0e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f7bb4111560 tx=0x7f7ba0009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:30.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:30.260+0000 7f7bb1600700 1 -- 172.21.15.37:0/3846274010 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f7ba80597b0 con 0x7f7bb4102e20 2024-03-29T08:18:30.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:18:30 smithi037 ceph-mon[142325]: from='client.34480 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:18:30.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:18:30 smithi037 ceph-mon[142325]: from='client.34484 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:18:30.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:18:30 smithi037 ceph-mon[142325]: pgmap v228: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:18:30.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:18:30 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/1462787226' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:18:30.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:18:30 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/1267266180' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T08:18:30.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:30.493+0000 7f7bbc000700 1 -- 172.21.15.37:0/3846274010 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7bb404bf40 con 0x7f7b9c06cc30 2024-03-29T08:18:30.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:30.494+0000 7f7bb1600700 1 -- 172.21.15.37:0/3846274010 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f7bb404bf40 con 0x7f7b9c06cc30 2024-03-29T08:18:30.495 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-29T08:18:30.495 INFO:teuthology.orchestra.run.smithi037.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59", 2024-03-29T08:18:30.495 INFO:teuthology.orchestra.run.smithi037.stdout: "in_progress": true, 2024-03-29T08:18:30.495 INFO:teuthology.orchestra.run.smithi037.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-29T08:18:30.495 INFO:teuthology.orchestra.run.smithi037.stdout: "services_complete": [ 2024-03-29T08:18:30.496 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr", 2024-03-29T08:18:30.496 INFO:teuthology.orchestra.run.smithi037.stdout: "mon", 2024-03-29T08:18:30.496 INFO:teuthology.orchestra.run.smithi037.stdout: "crash", 2024-03-29T08:18:30.496 INFO:teuthology.orchestra.run.smithi037.stdout: "osd" 2024-03-29T08:18:30.496 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-29T08:18:30.496 INFO:teuthology.orchestra.run.smithi037.stdout: "progress": "15/21 daemons upgraded", 2024-03-29T08:18:30.496 INFO:teuthology.orchestra.run.smithi037.stdout: "message": "Currently upgrading mds daemons", 2024-03-29T08:18:30.496 INFO:teuthology.orchestra.run.smithi037.stdout: "is_paused": false 2024-03-29T08:18:30.496 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-29T08:18:30.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:30.497+0000 7f7bbc000700 1 -- 172.21.15.37:0/3846274010 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f7b9c06cc30 msgr2=0x7f7b9c06f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:30.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:30.497+0000 7f7bbc000700 1 --2- 172.21.15.37:0/3846274010 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f7b9c06cc30 0x7f7b9c06f0e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f7bb4111560 tx=0x7f7ba0009e30 comp rx=0 tx=0).stop 2024-03-29T08:18:30.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:30.497+0000 7f7bbc000700 1 -- 172.21.15.37:0/3846274010 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7bb4102e20 msgr2=0x7f7bb4115cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:30.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:30.497+0000 7f7bbc000700 1 --2- 172.21.15.37:0/3846274010 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7bb4102e20 0x7f7bb4115cc0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f7ba8002a50 tx=0x7f7ba80046b0 comp rx=0 tx=0).stop 2024-03-29T08:18:30.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:30.497+0000 7f7bbc000700 1 -- 172.21.15.37:0/3846274010 shutdown_connections 2024-03-29T08:18:30.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:30.497+0000 7f7bbc000700 1 --2- 172.21.15.37:0/3846274010 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f7b9c06cc30 0x7f7b9c06f0e0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:30.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:30.497+0000 7f7bbc000700 1 --2- 172.21.15.37:0/3846274010 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7bb4102e20 0x7f7bb4115cc0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:30.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:30.497+0000 7f7bbc000700 1 --2- 172.21.15.37:0/3846274010 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7bb41037c0 0x7f7bb4116200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:30.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:30.497+0000 7f7bbc000700 1 -- 172.21.15.37:0/3846274010 >> 172.21.15.37:0/3846274010 conn(0x7f7bb40f8970 msgr2=0x7f7bb40f99a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:30.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:30.497+0000 7f7bbc000700 1 -- 172.21.15.37:0/3846274010 shutdown_connections 2024-03-29T08:18:30.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:30.497+0000 7f7bbc000700 1 -- 172.21.15.37:0/3846274010 wait complete. 2024-03-29T08:18:30.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:18:30 smithi148 ceph-mon[106692]: from='client.34480 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:18:30.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:18:30 smithi148 ceph-mon[106692]: from='client.34484 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:18:30.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:18:30 smithi148 ceph-mon[106692]: pgmap v228: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:18:30.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:18:30 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/1462787226' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:18:30.557 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:18:30 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/1267266180' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T08:18:30.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:30.635+0000 7f7dd5a00700 1 -- 172.21.15.37:0/3574921317 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7dd00fde90 msgr2=0x7f7dd00fe260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:30.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:30.635+0000 7f7dd5a00700 1 --2- 172.21.15.37:0/3574921317 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7dd00fde90 0x7f7dd00fe260 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f7dc0004940 tx=0x7f7dc0009e30 comp rx=0 tx=0).stop 2024-03-29T08:18:30.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:30.635+0000 7f7dd5a00700 1 -- 172.21.15.37:0/3574921317 shutdown_connections 2024-03-29T08:18:30.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:30.635+0000 7f7dd5a00700 1 --2- 172.21.15.37:0/3574921317 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7dd00fe830 0x7f7dd0106f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:30.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:30.635+0000 7f7dd5a00700 1 --2- 172.21.15.37:0/3574921317 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7dd00fde90 0x7f7dd00fe260 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:30.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:30.635+0000 7f7dd5a00700 1 -- 172.21.15.37:0/3574921317 >> 172.21.15.37:0/3574921317 conn(0x7f7dd0072bc0 msgr2=0x7f7dd0072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:30.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:30.635+0000 7f7dd5a00700 1 -- 172.21.15.37:0/3574921317 shutdown_connections 2024-03-29T08:18:30.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:30.635+0000 7f7dd5a00700 1 -- 172.21.15.37:0/3574921317 wait complete. 2024-03-29T08:18:30.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:30.636+0000 7f7dd5a00700 1 Processor -- start 2024-03-29T08:18:30.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:30.636+0000 7f7dd5a00700 1 -- start start 2024-03-29T08:18:30.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:30.636+0000 7f7dd5a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7dd00fde90 0x7f7dd0115d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:30.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:30.636+0000 7f7dd5a00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7dd00fe830 0x7f7dd0116260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:30.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:30.636+0000 7f7dd5a00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dd01168b0 con 0x7f7dd00fde90 2024-03-29T08:18:30.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:30.637+0000 7f7dd5a00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dd010fc80 con 0x7f7dd00fe830 2024-03-29T08:18:30.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:30.637+0000 7f7dcea00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7dd00fde90 0x7f7dd0115d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:30.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:30.637+0000 7f7dcea00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7dd00fde90 0x7f7dd0115d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34704/0 (socket says 172.21.15.37:34704) 2024-03-29T08:18:30.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:30.637+0000 7f7dcea00700 1 -- 172.21.15.37:0/2203185050 learned_addr learned my addr 172.21.15.37:0/2203185050 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:18:30.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:30.637+0000 7f7dce000700 1 --2- 172.21.15.37:0/2203185050 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7dd00fe830 0x7f7dd0116260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:30.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:30.637+0000 7f7dcea00700 1 -- 172.21.15.37:0/2203185050 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7dd00fe830 msgr2=0x7f7dd0116260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:30.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:30.637+0000 7f7dcea00700 1 --2- 172.21.15.37:0/2203185050 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7dd00fe830 0x7f7dd0116260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:30.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:30.637+0000 7f7dcea00700 1 -- 172.21.15.37:0/2203185050 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7dc0005040 con 0x7f7dd00fde90 2024-03-29T08:18:30.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:30.637+0000 7f7dcea00700 1 --2- 172.21.15.37:0/2203185050 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7dd00fde90 0x7f7dd0115d20 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f7dc0000c00 tx=0x7f7dc0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:30.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:30.638+0000 7f7dc6a00700 1 -- 172.21.15.37:0/2203185050 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7dc000e040 con 0x7f7dd00fde90 2024-03-29T08:18:30.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:30.638+0000 7f7dc6a00700 1 -- 172.21.15.37:0/2203185050 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f7dc000bed0 con 0x7f7dd00fde90 2024-03-29T08:18:30.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:30.638+0000 7f7dd5a00700 1 -- 172.21.15.37:0/2203185050 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7dd010ff00 con 0x7f7dd00fde90 2024-03-29T08:18:30.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:30.638+0000 7f7dc6a00700 1 -- 172.21.15.37:0/2203185050 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7dc001c3f0 con 0x7f7dd00fde90 2024-03-29T08:18:30.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:30.638+0000 7f7dd5a00700 1 -- 172.21.15.37:0/2203185050 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7dd0110440 con 0x7f7dd00fde90 2024-03-29T08:18:30.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:30.639+0000 7f7dd5a00700 1 -- 172.21.15.37:0/2203185050 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7dd0106a20 con 0x7f7dd00fde90 2024-03-29T08:18:30.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:30.640+0000 7f7dc6a00700 1 -- 172.21.15.37:0/2203185050 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f7dc0013420 con 0x7f7dd00fde90 2024-03-29T08:18:30.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:30.641+0000 7f7dc6a00700 1 --2- 172.21.15.37:0/2203185050 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f7db406cc30 0x7f7db406f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:30.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:30.641+0000 7f7dc6a00700 1 -- 172.21.15.37:0/2203185050 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f7dc008d500 con 0x7f7dd00fde90 2024-03-29T08:18:30.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:30.641+0000 7f7dce000700 1 --2- 172.21.15.37:0/2203185050 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f7db406cc30 0x7f7db406f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:30.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:30.642+0000 7f7dce000700 1 --2- 172.21.15.37:0/2203185050 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f7db406cc30 0x7f7db406f0e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f7dd01115c0 tx=0x7f7dbc00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:30.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:30.646+0000 7f7dc6a00700 1 -- 172.21.15.37:0/2203185050 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f7dc005af50 con 0x7f7dd00fde90 2024-03-29T08:18:30.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:30.950+0000 7f7dd5a00700 1 -- 172.21.15.37:0/2203185050 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f7dd004bed0 con 0x7f7dd00fde90 2024-03-29T08:18:30.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:30.950+0000 7f7dc6a00700 1 -- 172.21.15.37:0/2203185050 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7f7dc005aae0 con 0x7f7dd00fde90 2024-03-29T08:18:30.952 INFO:teuthology.orchestra.run.smithi037.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-03-29T08:18:30.952 INFO:teuthology.orchestra.run.smithi037.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-03-29T08:18:30.952 INFO:teuthology.orchestra.run.smithi037.stdout: fs cephfs is degraded 2024-03-29T08:18:30.952 INFO:teuthology.orchestra.run.smithi037.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-03-29T08:18:30.952 INFO:teuthology.orchestra.run.smithi037.stdout: fs cephfs is offline because no MDS is active for it. 2024-03-29T08:18:30.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:30.953+0000 7f7dd5a00700 1 -- 172.21.15.37:0/2203185050 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f7db406cc30 msgr2=0x7f7db406f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:30.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:30.953+0000 7f7dd5a00700 1 --2- 172.21.15.37:0/2203185050 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f7db406cc30 0x7f7db406f0e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f7dd01115c0 tx=0x7f7dbc00b040 comp rx=0 tx=0).stop 2024-03-29T08:18:30.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:30.953+0000 7f7dd5a00700 1 -- 172.21.15.37:0/2203185050 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7dd00fde90 msgr2=0x7f7dd0115d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:30.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:30.953+0000 7f7dd5a00700 1 --2- 172.21.15.37:0/2203185050 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7dd00fde90 0x7f7dd0115d20 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f7dc0000c00 tx=0x7f7dc0009e30 comp rx=0 tx=0).stop 2024-03-29T08:18:30.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:30.953+0000 7f7dd5a00700 1 -- 172.21.15.37:0/2203185050 shutdown_connections 2024-03-29T08:18:30.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:30.953+0000 7f7dd5a00700 1 --2- 172.21.15.37:0/2203185050 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f7db406cc30 0x7f7db406f0e0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:30.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:30.953+0000 7f7dd5a00700 1 --2- 172.21.15.37:0/2203185050 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7dd00fde90 0x7f7dd0115d20 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:30.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:30.953+0000 7f7dd5a00700 1 --2- 172.21.15.37:0/2203185050 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7dd00fe830 0x7f7dd0116260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:30.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:30.953+0000 7f7dd5a00700 1 -- 172.21.15.37:0/2203185050 >> 172.21.15.37:0/2203185050 conn(0x7f7dd0072bc0 msgr2=0x7f7dd00fb8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:30.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:30.953+0000 7f7dd5a00700 1 -- 172.21.15.37:0/2203185050 shutdown_connections 2024-03-29T08:18:30.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:18:30.953+0000 7f7dd5a00700 1 -- 172.21.15.37:0/2203185050 wait complete. 2024-03-29T08:18:31.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:18:31 smithi037 ceph-mon[142325]: from='client.34496 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:18:31.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:18:31 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/2203185050' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T08:18:31.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:18:31 smithi148 ceph-mon[106692]: from='client.34496 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:18:31.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:18:31 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/2203185050' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T08:18:32.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:18:32 smithi037 ceph-mon[142325]: pgmap v229: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:18:32.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:18:32 smithi148 ceph-mon[106692]: pgmap v229: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:18:34.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:18:33 smithi148 ceph-mon[106692]: pgmap v230: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:18:34.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:18:33 smithi037 ceph-mon[142325]: pgmap v230: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:18:36.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:18:35 smithi148 ceph-mon[106692]: pgmap v231: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:18:36.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:18:35 smithi037 ceph-mon[142325]: pgmap v231: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:18:38.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:18:37 smithi148 ceph-mon[106692]: pgmap v232: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:18:38.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:18:37 smithi037 ceph-mon[142325]: pgmap v232: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:18:40.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:18:40 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:18:40.428 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:18:40 smithi037 ceph-mon[142325]: pgmap v233: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:18:40.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:18:40 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:18:40.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:18:40 smithi148 ceph-mon[106692]: pgmap v233: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:18:42.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:18:41 smithi148 ceph-mon[106692]: pgmap v234: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:18:42.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:18:41 smithi037 ceph-mon[142325]: pgmap v234: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:18:44.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:18:43 smithi148 ceph-mon[106692]: pgmap v235: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:18:44.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:18:43 smithi037 ceph-mon[142325]: pgmap v235: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:18:46.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:18:45 smithi148 ceph-mon[106692]: pgmap v236: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:18:46.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:18:45 smithi037 ceph-mon[142325]: pgmap v236: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:18:48.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:18:47 smithi148 ceph-mon[106692]: pgmap v237: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:18:48.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:18:47 smithi037 ceph-mon[142325]: pgmap v237: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:18:50.431 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:18:50 smithi037 ceph-mon[142325]: pgmap v238: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:18:50.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:18:50 smithi148 ceph-mon[106692]: pgmap v238: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:18:52.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:18:51 smithi148 ceph-mon[106692]: pgmap v239: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:18:52.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:18:51 smithi037 ceph-mon[142325]: pgmap v239: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:18:54.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:18:53 smithi148 ceph-mon[106692]: pgmap v240: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:18:54.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:18:53 smithi037 ceph-mon[142325]: pgmap v240: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:18:55.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:18:55 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:18:55.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:18:55 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:18:56.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:18:56 smithi037 ceph-mon[142325]: pgmap v241: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:18:56.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:18:56 smithi148 ceph-mon[106692]: pgmap v241: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:18:58.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:18:57 smithi148 ceph-mon[106692]: pgmap v242: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:18:58.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:18:57 smithi037 ceph-mon[142325]: pgmap v242: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:19:00.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:19:00 smithi037 ceph-mon[142325]: pgmap v243: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:19:00.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:19:00 smithi148 ceph-mon[106692]: pgmap v243: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:19:01.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:01.109+0000 7f9644400700 1 -- 172.21.15.37:0/653312199 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f963c070300 msgr2=0x7f963c0706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:01.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:01.109+0000 7f9644400700 1 --2- 172.21.15.37:0/653312199 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f963c070300 0x7f963c0706d0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f9630004970 tx=0x7f9630009e30 comp rx=0 tx=0).stop 2024-03-29T08:19:01.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:01.109+0000 7f9644400700 1 -- 172.21.15.37:0/653312199 shutdown_connections 2024-03-29T08:19:01.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:01.109+0000 7f9644400700 1 --2- 172.21.15.37:0/653312199 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f963c070c10 0x7f963c109180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:01.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:01.109+0000 7f9644400700 1 --2- 172.21.15.37:0/653312199 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f963c070300 0x7f963c0706d0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:01.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:01.109+0000 7f9644400700 1 -- 172.21.15.37:0/653312199 >> 172.21.15.37:0/653312199 conn(0x7f963c0f89a0 msgr2=0x7f963c0fadb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:19:01.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:01.109+0000 7f9644400700 1 -- 172.21.15.37:0/653312199 shutdown_connections 2024-03-29T08:19:01.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:01.109+0000 7f9644400700 1 -- 172.21.15.37:0/653312199 wait complete. 2024-03-29T08:19:01.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:01.110+0000 7f9644400700 1 Processor -- start 2024-03-29T08:19:01.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:01.110+0000 7f9644400700 1 -- start start 2024-03-29T08:19:01.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:01.110+0000 7f9644400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f963c070300 0x7f963c115c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:01.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:01.110+0000 7f9644400700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f963c070c10 0x7f963c1161d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:01.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:01.110+0000 7f9644400700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f963c116820 con 0x7f963c070300 2024-03-29T08:19:01.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:01.110+0000 7f9644400700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f963c10fbf0 con 0x7f963c070c10 2024-03-29T08:19:01.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:01.111+0000 7f9641a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f963c070300 0x7f963c115c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:01.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:01.111+0000 7f9641a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f963c070300 0x7f963c115c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42202/0 (socket says 172.21.15.37:42202) 2024-03-29T08:19:01.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:01.111+0000 7f9641a00700 1 -- 172.21.15.37:0/2379691112 learned_addr learned my addr 172.21.15.37:0/2379691112 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:19:01.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:01.111+0000 7f9641000700 1 --2- 172.21.15.37:0/2379691112 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f963c070c10 0x7f963c1161d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:01.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:01.111+0000 7f9641a00700 1 -- 172.21.15.37:0/2379691112 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f963c070c10 msgr2=0x7f963c1161d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:01.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:01.111+0000 7f9641a00700 1 --2- 172.21.15.37:0/2379691112 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f963c070c10 0x7f963c1161d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:01.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:01.111+0000 7f9641a00700 1 -- 172.21.15.37:0/2379691112 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9630005040 con 0x7f963c070300 2024-03-29T08:19:01.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:01.111+0000 7f9641a00700 1 --2- 172.21.15.37:0/2379691112 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f963c070300 0x7f963c115c90 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f9630005010 tx=0x7f9630009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:01.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:01.112+0000 7f9639600700 1 -- 172.21.15.37:0/2379691112 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f963000e040 con 0x7f963c070300 2024-03-29T08:19:01.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:01.112+0000 7f9639600700 1 -- 172.21.15.37:0/2379691112 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f9630009930 con 0x7f963c070300 2024-03-29T08:19:01.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:01.112+0000 7f9644400700 1 -- 172.21.15.37:0/2379691112 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f963c10fe70 con 0x7f963c070300 2024-03-29T08:19:01.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:01.112+0000 7f9639600700 1 -- 172.21.15.37:0/2379691112 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f963001c3f0 con 0x7f963c070300 2024-03-29T08:19:01.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:01.112+0000 7f9644400700 1 -- 172.21.15.37:0/2379691112 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f963c1103b0 con 0x7f963c070300 2024-03-29T08:19:01.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:01.113+0000 7f9644400700 1 -- 172.21.15.37:0/2379691112 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f963c071a40 con 0x7f963c070300 2024-03-29T08:19:01.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:01.114+0000 7f9639600700 1 -- 172.21.15.37:0/2379691112 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f96300136b0 con 0x7f963c070300 2024-03-29T08:19:01.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:01.115+0000 7f9639600700 1 --2- 172.21.15.37:0/2379691112 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f962406cc30 0x7f962406f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:01.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:01.115+0000 7f9639600700 1 -- 172.21.15.37:0/2379691112 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f963008d310 con 0x7f963c070300 2024-03-29T08:19:01.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:01.115+0000 7f9641000700 1 --2- 172.21.15.37:0/2379691112 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f962406cc30 0x7f962406f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:01.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:01.116+0000 7f9641000700 1 --2- 172.21.15.37:0/2379691112 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f962406cc30 0x7f962406f0e0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f9628004600 tx=0x7f962800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:01.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:01.120+0000 7f9639600700 1 -- 172.21.15.37:0/2379691112 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f963005ad60 con 0x7f963c070300 2024-03-29T08:19:01.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:01.352+0000 7f9644400700 1 -- 172.21.15.37:0/2379691112 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f963c04bf40 con 0x7f962406cc30 2024-03-29T08:19:01.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:01.353+0000 7f9639600700 1 -- 172.21.15.37:0/2379691112 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f963c04bf40 con 0x7f962406cc30 2024-03-29T08:19:01.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:01.356+0000 7f9644400700 1 -- 172.21.15.37:0/2379691112 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f962406cc30 msgr2=0x7f962406f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:01.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:01.356+0000 7f9644400700 1 --2- 172.21.15.37:0/2379691112 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f962406cc30 0x7f962406f0e0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f9628004600 tx=0x7f962800b040 comp rx=0 tx=0).stop 2024-03-29T08:19:01.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:01.356+0000 7f9644400700 1 -- 172.21.15.37:0/2379691112 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f963c070300 msgr2=0x7f963c115c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:01.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:01.356+0000 7f9644400700 1 --2- 172.21.15.37:0/2379691112 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f963c070300 0x7f963c115c90 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f9630005010 tx=0x7f9630009e30 comp rx=0 tx=0).stop 2024-03-29T08:19:01.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:01.356+0000 7f9644400700 1 -- 172.21.15.37:0/2379691112 shutdown_connections 2024-03-29T08:19:01.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:01.356+0000 7f9644400700 1 --2- 172.21.15.37:0/2379691112 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f962406cc30 0x7f962406f0e0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:01.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:01.356+0000 7f9644400700 1 --2- 172.21.15.37:0/2379691112 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f963c070300 0x7f963c115c90 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:01.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:01.356+0000 7f9644400700 1 --2- 172.21.15.37:0/2379691112 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f963c070c10 0x7f963c1161d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:01.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:01.356+0000 7f9644400700 1 -- 172.21.15.37:0/2379691112 >> 172.21.15.37:0/2379691112 conn(0x7f963c0f89a0 msgr2=0x7f963c105a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:19:01.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:01.356+0000 7f9644400700 1 -- 172.21.15.37:0/2379691112 shutdown_connections 2024-03-29T08:19:01.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:01.356+0000 7f9644400700 1 -- 172.21.15.37:0/2379691112 wait complete. 2024-03-29T08:19:01.371 INFO:teuthology.orchestra.run.smithi037.stdout:true 2024-03-29T08:19:01.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:01.495+0000 7f4d16a00700 1 -- 172.21.15.37:0/1350723453 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4d10102e00 msgr2=0x7f4d101031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:01.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:01.495+0000 7f4d16a00700 1 --2- 172.21.15.37:0/1350723453 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4d10102e00 0x7f4d101031d0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f4d00004940 tx=0x7f4d00009e30 comp rx=0 tx=0).stop 2024-03-29T08:19:01.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:01.495+0000 7f4d16a00700 1 -- 172.21.15.37:0/1350723453 shutdown_connections 2024-03-29T08:19:01.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:01.495+0000 7f4d16a00700 1 --2- 172.21.15.37:0/1350723453 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4d101037a0 0x7f4d100fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:01.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:01.495+0000 7f4d16a00700 1 --2- 172.21.15.37:0/1350723453 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4d10102e00 0x7f4d101031d0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:01.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:01.495+0000 7f4d16a00700 1 -- 172.21.15.37:0/1350723453 >> 172.21.15.37:0/1350723453 conn(0x7f4d100f89b0 msgr2=0x7f4d100fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:19:01.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:01.495+0000 7f4d16a00700 1 -- 172.21.15.37:0/1350723453 shutdown_connections 2024-03-29T08:19:01.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:01.495+0000 7f4d16a00700 1 -- 172.21.15.37:0/1350723453 wait complete. 2024-03-29T08:19:01.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:01.496+0000 7f4d16a00700 1 Processor -- start 2024-03-29T08:19:01.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:01.496+0000 7f4d16a00700 1 -- start start 2024-03-29T08:19:01.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:01.496+0000 7f4d16a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4d10102e00 0x7f4d100702c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:01.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:01.496+0000 7f4d16a00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4d101037a0 0x7f4d10070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:01.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:01.496+0000 7f4d16a00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d10070d90 con 0x7f4d10102e00 2024-03-29T08:19:01.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:01.496+0000 7f4d16a00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d10070ed0 con 0x7f4d101037a0 2024-03-29T08:19:01.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:01.497+0000 7f4d0fe00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4d10102e00 0x7f4d100702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:01.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:01.497+0000 7f4d0fe00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4d10102e00 0x7f4d100702c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42210/0 (socket says 172.21.15.37:42210) 2024-03-29T08:19:01.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:01.497+0000 7f4d0fe00700 1 -- 172.21.15.37:0/1360724795 learned_addr learned my addr 172.21.15.37:0/1360724795 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:19:01.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:01.497+0000 7f4d0f400700 1 --2- 172.21.15.37:0/1360724795 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4d101037a0 0x7f4d10070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:01.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:01.497+0000 7f4d0fe00700 1 -- 172.21.15.37:0/1360724795 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4d101037a0 msgr2=0x7f4d10070800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:01.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:01.497+0000 7f4d0fe00700 1 --2- 172.21.15.37:0/1360724795 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4d101037a0 0x7f4d10070800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:01.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:01.497+0000 7f4d0fe00700 1 -- 172.21.15.37:0/1360724795 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d00005040 con 0x7f4d10102e00 2024-03-29T08:19:01.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:01.497+0000 7f4d0fe00700 1 --2- 172.21.15.37:0/1360724795 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4d10102e00 0x7f4d100702c0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f4d00008d80 tx=0x7f4d00009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:01.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:01.498+0000 7f4d07e00700 1 -- 172.21.15.37:0/1360724795 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4d0000e030 con 0x7f4d10102e00 2024-03-29T08:19:01.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:01.498+0000 7f4d07e00700 1 -- 172.21.15.37:0/1360724795 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f4d00013de0 con 0x7f4d10102e00 2024-03-29T08:19:01.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:01.498+0000 7f4d07e00700 1 -- 172.21.15.37:0/1360724795 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4d0001cd00 con 0x7f4d10102e00 2024-03-29T08:19:01.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:01.498+0000 7f4d16a00700 1 -- 172.21.15.37:0/1360724795 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d10071120 con 0x7f4d10102e00 2024-03-29T08:19:01.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:01.498+0000 7f4d16a00700 1 -- 172.21.15.37:0/1360724795 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d1011a170 con 0x7f4d10102e00 2024-03-29T08:19:01.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:01.499+0000 7f4d16a00700 1 -- 172.21.15.37:0/1360724795 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d100fe550 con 0x7f4d10102e00 2024-03-29T08:19:01.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:01.500+0000 7f4d07e00700 1 -- 172.21.15.37:0/1360724795 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f4d00013930 con 0x7f4d10102e00 2024-03-29T08:19:01.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:01.501+0000 7f4d07e00700 1 --2- 172.21.15.37:0/1360724795 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f4cf406cc30 0x7f4cf406f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:01.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:01.501+0000 7f4d07e00700 1 -- 172.21.15.37:0/1360724795 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f4d0008d820 con 0x7f4d10102e00 2024-03-29T08:19:01.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:01.502+0000 7f4d0f400700 1 --2- 172.21.15.37:0/1360724795 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f4cf406cc30 0x7f4cf406f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:01.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:01.502+0000 7f4d0f400700 1 --2- 172.21.15.37:0/1360724795 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f4cf406cc30 0x7f4cf406f0e0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f4d10071c00 tx=0x7f4cfc00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:01.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:01.506+0000 7f4d07e00700 1 -- 172.21.15.37:0/1360724795 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f4d00057860 con 0x7f4d10102e00 2024-03-29T08:19:01.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:01.743+0000 7f4d16a00700 1 -- 172.21.15.37:0/1360724795 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4d100719b0 con 0x7f4cf406cc30 2024-03-29T08:19:01.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:01.745+0000 7f4d07e00700 1 -- 172.21.15.37:0/1360724795 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f4d100719b0 con 0x7f4cf406cc30 2024-03-29T08:19:01.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:01.747+0000 7f4d16a00700 1 -- 172.21.15.37:0/1360724795 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f4cf406cc30 msgr2=0x7f4cf406f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:01.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:01.747+0000 7f4d16a00700 1 --2- 172.21.15.37:0/1360724795 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f4cf406cc30 0x7f4cf406f0e0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f4d10071c00 tx=0x7f4cfc00b040 comp rx=0 tx=0).stop 2024-03-29T08:19:01.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:01.747+0000 7f4d16a00700 1 -- 172.21.15.37:0/1360724795 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4d10102e00 msgr2=0x7f4d100702c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:01.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:01.747+0000 7f4d16a00700 1 --2- 172.21.15.37:0/1360724795 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4d10102e00 0x7f4d100702c0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f4d00008d80 tx=0x7f4d00009e30 comp rx=0 tx=0).stop 2024-03-29T08:19:01.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:01.747+0000 7f4d16a00700 1 -- 172.21.15.37:0/1360724795 shutdown_connections 2024-03-29T08:19:01.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:01.747+0000 7f4d16a00700 1 --2- 172.21.15.37:0/1360724795 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f4cf406cc30 0x7f4cf406f0e0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:01.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:01.747+0000 7f4d16a00700 1 --2- 172.21.15.37:0/1360724795 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4d10102e00 0x7f4d100702c0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:01.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:01.747+0000 7f4d16a00700 1 --2- 172.21.15.37:0/1360724795 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4d101037a0 0x7f4d10070800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:01.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:01.747+0000 7f4d16a00700 1 -- 172.21.15.37:0/1360724795 >> 172.21.15.37:0/1360724795 conn(0x7f4d100f89b0 msgr2=0x7f4d100f99f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:19:01.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:01.747+0000 7f4d16a00700 1 -- 172.21.15.37:0/1360724795 shutdown_connections 2024-03-29T08:19:01.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:01.747+0000 7f4d16a00700 1 -- 172.21.15.37:0/1360724795 wait complete. 2024-03-29T08:19:01.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:01.884+0000 7f5cfdc00700 1 -- 172.21.15.37:0/3285309175 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5cf8070c10 msgr2=0x7f5cf8109180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:01.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:01.884+0000 7f5cfdc00700 1 --2- 172.21.15.37:0/3285309175 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5cf8070c10 0x7f5cf8109180 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f5ce4004970 tx=0x7f5ce4009e30 comp rx=0 tx=0).stop 2024-03-29T08:19:01.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:01.884+0000 7f5cfdc00700 1 -- 172.21.15.37:0/3285309175 shutdown_connections 2024-03-29T08:19:01.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:01.884+0000 7f5cfdc00700 1 --2- 172.21.15.37:0/3285309175 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5cf8070c10 0x7f5cf8109180 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:01.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:01.884+0000 7f5cfdc00700 1 --2- 172.21.15.37:0/3285309175 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5cf8070300 0x7f5cf80706d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:01.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:01.884+0000 7f5cfdc00700 1 -- 172.21.15.37:0/3285309175 >> 172.21.15.37:0/3285309175 conn(0x7f5cf80f89a0 msgr2=0x7f5cf80fadb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:19:01.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:01.885+0000 7f5cfdc00700 1 -- 172.21.15.37:0/3285309175 shutdown_connections 2024-03-29T08:19:01.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:01.885+0000 7f5cfdc00700 1 -- 172.21.15.37:0/3285309175 wait complete. 2024-03-29T08:19:01.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:01.885+0000 7f5cfdc00700 1 Processor -- start 2024-03-29T08:19:01.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:01.885+0000 7f5cfdc00700 1 -- start start 2024-03-29T08:19:01.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:01.886+0000 7f5cfdc00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5cf8070300 0x7f5cf8115ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:01.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:01.886+0000 7f5cfdc00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5cf8070c10 0x7f5cf81161e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:01.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:01.886+0000 7f5cfdc00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cf8116830 con 0x7f5cf8070300 2024-03-29T08:19:01.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:01.886+0000 7f5cfdc00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cf810fc00 con 0x7f5cf8070c10 2024-03-29T08:19:01.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:01.886+0000 7f5cf6a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5cf8070300 0x7f5cf8115ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:01.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:01.886+0000 7f5cf6a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5cf8070300 0x7f5cf8115ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42238/0 (socket says 172.21.15.37:42238) 2024-03-29T08:19:01.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:01.886+0000 7f5cf6a00700 1 -- 172.21.15.37:0/2268741525 learned_addr learned my addr 172.21.15.37:0/2268741525 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:19:01.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:01.886+0000 7f5cf6000700 1 --2- 172.21.15.37:0/2268741525 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5cf8070c10 0x7f5cf81161e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:01.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:01.887+0000 7f5cf6a00700 1 -- 172.21.15.37:0/2268741525 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5cf8070c10 msgr2=0x7f5cf81161e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:01.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:01.887+0000 7f5cf6a00700 1 --2- 172.21.15.37:0/2268741525 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5cf8070c10 0x7f5cf81161e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:01.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:01.887+0000 7f5cf6a00700 1 -- 172.21.15.37:0/2268741525 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ce4005040 con 0x7f5cf8070300 2024-03-29T08:19:01.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:01.887+0000 7f5cf6a00700 1 --2- 172.21.15.37:0/2268741525 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5cf8070300 0x7f5cf8115ca0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f5ce800c8c0 tx=0x7f5ce80044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:01.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:01.887+0000 7f5ceea00700 1 -- 172.21.15.37:0/2268741525 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5ce800a760 con 0x7f5cf8070300 2024-03-29T08:19:01.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:01.887+0000 7f5ceea00700 1 -- 172.21.15.37:0/2268741525 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f5ce800ad70 con 0x7f5cf8070300 2024-03-29T08:19:01.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:01.887+0000 7f5ceea00700 1 -- 172.21.15.37:0/2268741525 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5ce80113a0 con 0x7f5cf8070300 2024-03-29T08:19:01.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:01.887+0000 7f5cfdc00700 1 -- 172.21.15.37:0/2268741525 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5cf810fee0 con 0x7f5cf8070300 2024-03-29T08:19:01.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:01.887+0000 7f5cfdc00700 1 -- 172.21.15.37:0/2268741525 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cf81103a0 con 0x7f5cf8070300 2024-03-29T08:19:01.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:01.889+0000 7f5cfdc00700 1 -- 172.21.15.37:0/2268741525 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5cf8071a40 con 0x7f5cf8070300 2024-03-29T08:19:01.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:01.889+0000 7f5ceea00700 1 -- 172.21.15.37:0/2268741525 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f5ce8012400 con 0x7f5cf8070300 2024-03-29T08:19:01.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:01.891+0000 7f5ceea00700 1 --2- 172.21.15.37:0/2268741525 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f5cdc06cc30 0x7f5cdc06f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:01.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:01.891+0000 7f5ceea00700 1 -- 172.21.15.37:0/2268741525 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f5ce808c100 con 0x7f5cf8070300 2024-03-29T08:19:01.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:01.891+0000 7f5cf6000700 1 --2- 172.21.15.37:0/2268741525 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f5cdc06cc30 0x7f5cdc06f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:01.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:01.892+0000 7f5cf6000700 1 --2- 172.21.15.37:0/2268741525 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f5cdc06cc30 0x7f5cdc06f0e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f5ce4009fd0 tx=0x7f5ce4009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:01.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:01.895+0000 7f5ceea00700 1 -- 172.21.15.37:0/2268741525 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f5ce8059b50 con 0x7f5cf8070300 2024-03-29T08:19:02.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:19:01 smithi148 ceph-mon[106692]: from='client.34504 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:19:02.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:19:01 smithi148 ceph-mon[106692]: pgmap v244: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:19:02.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:02.114+0000 7f5cfdc00700 1 -- 172.21.15.37:0/2268741525 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f5cf804bf40 con 0x7f5cdc06cc30 2024-03-29T08:19:02.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:02.125+0000 7f5ceea00700 1 -- 172.21.15.37:0/2268741525 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7f5cf804bf40 con 0x7f5cdc06cc30 2024-03-29T08:19:02.125 INFO:teuthology.orchestra.run.smithi037.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-29T08:19:02.125 INFO:teuthology.orchestra.run.smithi037.stdout:alertmanager.smithi037 smithi037 *:9093,9094 running (8m) 70s ago 21m 14.9M - 0.25.0 c8568f914cd2 2f9f9746ed0f 2024-03-29T08:19:02.125 INFO:teuthology.orchestra.run.smithi037.stdout:crash.smithi037 smithi037 running (5m) 70s ago 22m 6647k - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 178d416a3186 2024-03-29T08:19:02.125 INFO:teuthology.orchestra.run.smithi037.stdout:crash.smithi148 smithi148 running (5m) 49s ago 20m 6656k - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 23666989d58f 2024-03-29T08:19:02.126 INFO:teuthology.orchestra.run.smithi037.stdout:grafana.smithi037 smithi037 *:3000 running (8m) 70s ago 21m 71.9M - 9.4.7 954c08fa6188 6b976fe49952 2024-03-29T08:19:02.126 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi037.iifsmh smithi037 running (76s) 70s ago 15m 10.8M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 2d97f562cc5b 2024-03-29T08:19:02.126 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi037.mmwgxe smithi037 running (96s) 70s ago 15m 10.9M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff ec37037f46e8 2024-03-29T08:19:02.126 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi148.qukpjh smithi148 running (15m) 49s ago 15m 15.3M - 17.2.7-789-g4213e17c cf22eb8683be 8e469782a273 2024-03-29T08:19:02.126 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi148.sqteih smithi148 running (54s) 49s ago 15m 10.7M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 44c15c77037a 2024-03-29T08:19:02.126 INFO:teuthology.orchestra.run.smithi037.stdout:mgr.smithi037.wjikii smithi037 *:8443,9283,8765 running (11m) 70s ago 23m 506M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff c5a1417b9417 2024-03-29T08:19:02.126 INFO:teuthology.orchestra.run.smithi037.stdout:mgr.smithi148.onhgvm smithi148 *:8443,9283,8765 running (10m) 49s ago 20m 450M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff fcfdbce062bd 2024-03-29T08:19:02.126 INFO:teuthology.orchestra.run.smithi037.stdout:mon.smithi037 smithi037 running (7m) 70s ago 23m 54.8M 2048M 18.2.2-1015-gd0e392d9 1cc2f45c17ff eea2060783af 2024-03-29T08:19:02.126 INFO:teuthology.orchestra.run.smithi037.stdout:mon.smithi148 smithi148 running (6m) 49s ago 20m 43.7M 2048M 18.2.2-1015-gd0e392d9 1cc2f45c17ff ad4b2c5748be 2024-03-29T08:19:02.126 INFO:teuthology.orchestra.run.smithi037.stdout:node-exporter.smithi037 smithi037 *:9100 running (9m) 70s ago 22m 7625k - 1.5.0 0da6a335fe13 26b8ae9cb878 2024-03-29T08:19:02.126 INFO:teuthology.orchestra.run.smithi037.stdout:node-exporter.smithi148 smithi148 *:9100 running (9m) 49s ago 20m 8166k - 1.5.0 0da6a335fe13 98414e4e64ed 2024-03-29T08:19:02.126 INFO:teuthology.orchestra.run.smithi037.stdout:osd.0 smithi037 running (5m) 70s ago 19m 104M 1619M 18.2.2-1015-gd0e392d9 1cc2f45c17ff c0c584f6245c 2024-03-29T08:19:02.127 INFO:teuthology.orchestra.run.smithi037.stdout:osd.1 smithi037 running (4m) 70s ago 19m 100M 1619M 18.2.2-1015-gd0e392d9 1cc2f45c17ff a4a2ff4cc83d 2024-03-29T08:19:02.127 INFO:teuthology.orchestra.run.smithi037.stdout:osd.2 smithi037 running (3m) 70s ago 18m 82.3M 1619M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 8b1bc71aa79e 2024-03-29T08:19:02.127 INFO:teuthology.orchestra.run.smithi037.stdout:osd.3 smithi148 running (3m) 49s ago 18m 105M 2642M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 1c656ab8ff86 2024-03-29T08:19:02.127 INFO:teuthology.orchestra.run.smithi037.stdout:osd.4 smithi148 running (2m) 49s ago 17m 97.3M 2642M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 966a9de11d22 2024-03-29T08:19:02.127 INFO:teuthology.orchestra.run.smithi037.stdout:osd.5 smithi148 running (117s) 49s ago 17m 85.8M 2642M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 474c4f9212ce 2024-03-29T08:19:02.127 INFO:teuthology.orchestra.run.smithi037.stdout:prometheus.smithi037 smithi037 *:9095 running (9m) 70s ago 21m 54.4M - 2.43.0 a07b618ecd1d 30c0223ec589 2024-03-29T08:19:02.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:02.127+0000 7f5cfdc00700 1 -- 172.21.15.37:0/2268741525 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f5cdc06cc30 msgr2=0x7f5cdc06f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:02.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:02.127+0000 7f5cfdc00700 1 --2- 172.21.15.37:0/2268741525 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f5cdc06cc30 0x7f5cdc06f0e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f5ce4009fd0 tx=0x7f5ce4009e30 comp rx=0 tx=0).stop 2024-03-29T08:19:02.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:02.127+0000 7f5cfdc00700 1 -- 172.21.15.37:0/2268741525 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5cf8070300 msgr2=0x7f5cf8115ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:02.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:02.127+0000 7f5cfdc00700 1 --2- 172.21.15.37:0/2268741525 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5cf8070300 0x7f5cf8115ca0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f5ce800c8c0 tx=0x7f5ce80044c0 comp rx=0 tx=0).stop 2024-03-29T08:19:02.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:02.127+0000 7f5cfdc00700 1 -- 172.21.15.37:0/2268741525 shutdown_connections 2024-03-29T08:19:02.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:02.127+0000 7f5cfdc00700 1 --2- 172.21.15.37:0/2268741525 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f5cdc06cc30 0x7f5cdc06f0e0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:02.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:02.127+0000 7f5cfdc00700 1 --2- 172.21.15.37:0/2268741525 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5cf8070300 0x7f5cf8115ca0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:02.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:02.127+0000 7f5cfdc00700 1 --2- 172.21.15.37:0/2268741525 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5cf8070c10 0x7f5cf81161e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:02.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:02.127+0000 7f5cfdc00700 1 -- 172.21.15.37:0/2268741525 >> 172.21.15.37:0/2268741525 conn(0x7f5cf80f89a0 msgr2=0x7f5cf80f9f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:19:02.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:02.128+0000 7f5cfdc00700 1 -- 172.21.15.37:0/2268741525 shutdown_connections 2024-03-29T08:19:02.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:02.128+0000 7f5cfdc00700 1 -- 172.21.15.37:0/2268741525 wait complete. 2024-03-29T08:19:02.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:19:01 smithi037 ceph-mon[142325]: from='client.34504 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:19:02.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:19:01 smithi037 ceph-mon[142325]: pgmap v244: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:19:02.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:02.264+0000 7fcd3ee00700 1 -- 172.21.15.37:0/2028013537 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcd38102c40 msgr2=0x7fcd38103010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:02.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:02.264+0000 7fcd3ee00700 1 --2- 172.21.15.37:0/2028013537 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcd38102c40 0x7fcd38103010 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fcd2c004940 tx=0x7fcd2c009e30 comp rx=0 tx=0).stop 2024-03-29T08:19:02.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:02.264+0000 7fcd3ee00700 1 -- 172.21.15.37:0/2028013537 shutdown_connections 2024-03-29T08:19:02.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:02.264+0000 7fcd3ee00700 1 --2- 172.21.15.37:0/2028013537 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcd38065620 0x7fcd38065a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:02.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:02.264+0000 7fcd3ee00700 1 --2- 172.21.15.37:0/2028013537 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcd38102c40 0x7fcd38103010 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:02.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:02.264+0000 7fcd3ee00700 1 -- 172.21.15.37:0/2028013537 >> 172.21.15.37:0/2028013537 conn(0x7fcd38071b70 msgr2=0x7fcd38071f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:19:02.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:02.264+0000 7fcd3ee00700 1 -- 172.21.15.37:0/2028013537 shutdown_connections 2024-03-29T08:19:02.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:02.264+0000 7fcd3ee00700 1 -- 172.21.15.37:0/2028013537 wait complete. 2024-03-29T08:19:02.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:02.265+0000 7fcd3ee00700 1 Processor -- start 2024-03-29T08:19:02.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:02.265+0000 7fcd3ee00700 1 -- start start 2024-03-29T08:19:02.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:02.266+0000 7fcd3ee00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcd38065620 0x7fcd38115b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:02.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:02.266+0000 7fcd3ee00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcd38102c40 0x7fcd381160c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:02.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:02.266+0000 7fcd3ee00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd38116710 con 0x7fcd38065620 2024-03-29T08:19:02.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:02.266+0000 7fcd3ee00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd3810fb00 con 0x7fcd38102c40 2024-03-29T08:19:02.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:02.266+0000 7fcd3da00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcd38065620 0x7fcd38115b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:02.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:02.266+0000 7fcd3da00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcd38065620 0x7fcd38115b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42242/0 (socket says 172.21.15.37:42242) 2024-03-29T08:19:02.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:02.266+0000 7fcd3da00700 1 -- 172.21.15.37:0/1023141857 learned_addr learned my addr 172.21.15.37:0/1023141857 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:19:02.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:02.266+0000 7fcd3d000700 1 --2- 172.21.15.37:0/1023141857 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcd38102c40 0x7fcd381160c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:02.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:02.266+0000 7fcd3da00700 1 -- 172.21.15.37:0/1023141857 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcd38102c40 msgr2=0x7fcd381160c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:02.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:02.266+0000 7fcd3da00700 1 --2- 172.21.15.37:0/1023141857 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcd38102c40 0x7fcd381160c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:02.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:02.266+0000 7fcd3da00700 1 -- 172.21.15.37:0/1023141857 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd2c005040 con 0x7fcd38065620 2024-03-29T08:19:02.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:02.267+0000 7fcd3da00700 1 --2- 172.21.15.37:0/1023141857 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcd38065620 0x7fcd38115b80 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fcd2c009390 tx=0x7fcd2c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:02.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:02.267+0000 7fcd35600700 1 -- 172.21.15.37:0/1023141857 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcd2c00e040 con 0x7fcd38065620 2024-03-29T08:19:02.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:02.267+0000 7fcd35600700 1 -- 172.21.15.37:0/1023141857 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fcd2c013420 con 0x7fcd38065620 2024-03-29T08:19:02.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:02.267+0000 7fcd3ee00700 1 -- 172.21.15.37:0/1023141857 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd3810fd80 con 0x7fcd38065620 2024-03-29T08:19:02.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:02.267+0000 7fcd35600700 1 -- 172.21.15.37:0/1023141857 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcd2c01c630 con 0x7fcd38065620 2024-03-29T08:19:02.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:02.267+0000 7fcd3ee00700 1 -- 172.21.15.37:0/1023141857 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd381102c0 con 0x7fcd38065620 2024-03-29T08:19:02.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:02.268+0000 7fcd3ee00700 1 -- 172.21.15.37:0/1023141857 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd38106810 con 0x7fcd38065620 2024-03-29T08:19:02.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:02.269+0000 7fcd35600700 1 -- 172.21.15.37:0/1023141857 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fcd2c0139e0 con 0x7fcd38065620 2024-03-29T08:19:02.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:02.270+0000 7fcd35600700 1 --2- 172.21.15.37:0/1023141857 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fcd2006cc30 0x7fcd2006f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:02.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:02.270+0000 7fcd35600700 1 -- 172.21.15.37:0/1023141857 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fcd2c08d3d0 con 0x7fcd38065620 2024-03-29T08:19:02.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:02.270+0000 7fcd3d000700 1 --2- 172.21.15.37:0/1023141857 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fcd2006cc30 0x7fcd2006f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:02.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:02.271+0000 7fcd3d000700 1 --2- 172.21.15.37:0/1023141857 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fcd2006cc30 0x7fcd2006f0e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fcd38110cc0 tx=0x7fcd2400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:02.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:02.275+0000 7fcd35600700 1 -- 172.21.15.37:0/1023141857 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fcd2c05ae20 con 0x7fcd38065620 2024-03-29T08:19:02.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:02.573+0000 7fcd3ee00700 1 -- 172.21.15.37:0/1023141857 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fcd381111d0 con 0x7fcd38065620 2024-03-29T08:19:02.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:02.574+0000 7fcd35600700 1 -- 172.21.15.37:0/1023141857 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+847 (secure 0 0 0) 0x7fcd2c05a9b0 con 0x7fcd38065620 2024-03-29T08:19:02.574 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-29T08:19:02.574 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": { 2024-03-29T08:19:02.574 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 2 2024-03-29T08:19:02.574 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:19:02.574 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": { 2024-03-29T08:19:02.574 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 2 2024-03-29T08:19:02.574 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:19:02.575 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": { 2024-03-29T08:19:02.575 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 6 2024-03-29T08:19:02.575 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:19:02.575 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": { 2024-03-29T08:19:02.575 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 17.2.7-789-g4213e17c (4213e17c4de67016ed14f55d22503f4923b3a901) quincy (stable)": 1, 2024-03-29T08:19:02.575 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 3 2024-03-29T08:19:02.575 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:19:02.575 INFO:teuthology.orchestra.run.smithi037.stdout: "overall": { 2024-03-29T08:19:02.575 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 17.2.7-789-g4213e17c (4213e17c4de67016ed14f55d22503f4923b3a901) quincy (stable)": 1, 2024-03-29T08:19:02.575 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 13 2024-03-29T08:19:02.575 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-29T08:19:02.575 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-29T08:19:02.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:02.576+0000 7fcd3ee00700 1 -- 172.21.15.37:0/1023141857 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fcd2006cc30 msgr2=0x7fcd2006f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:02.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:02.576+0000 7fcd3ee00700 1 --2- 172.21.15.37:0/1023141857 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fcd2006cc30 0x7fcd2006f0e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fcd38110cc0 tx=0x7fcd2400b040 comp rx=0 tx=0).stop 2024-03-29T08:19:02.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:02.576+0000 7fcd3ee00700 1 -- 172.21.15.37:0/1023141857 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcd38065620 msgr2=0x7fcd38115b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:02.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:02.576+0000 7fcd3ee00700 1 --2- 172.21.15.37:0/1023141857 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcd38065620 0x7fcd38115b80 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fcd2c009390 tx=0x7fcd2c009e30 comp rx=0 tx=0).stop 2024-03-29T08:19:02.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:02.576+0000 7fcd3ee00700 1 -- 172.21.15.37:0/1023141857 shutdown_connections 2024-03-29T08:19:02.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:02.576+0000 7fcd3ee00700 1 --2- 172.21.15.37:0/1023141857 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fcd2006cc30 0x7fcd2006f0e0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:02.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:02.576+0000 7fcd3ee00700 1 --2- 172.21.15.37:0/1023141857 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcd38065620 0x7fcd38115b80 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:02.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:02.576+0000 7fcd3ee00700 1 --2- 172.21.15.37:0/1023141857 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcd38102c40 0x7fcd381160c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:02.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:02.577+0000 7fcd3ee00700 1 -- 172.21.15.37:0/1023141857 >> 172.21.15.37:0/1023141857 conn(0x7fcd38071b70 msgr2=0x7fcd38103990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:19:02.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:02.577+0000 7fcd3ee00700 1 -- 172.21.15.37:0/1023141857 shutdown_connections 2024-03-29T08:19:02.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:02.577+0000 7fcd3ee00700 1 -- 172.21.15.37:0/1023141857 wait complete. 2024-03-29T08:19:02.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:02.716+0000 7f0c89600700 1 -- 172.21.15.37:0/705459617 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0c84102e00 msgr2=0x7f0c841031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:02.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:02.716+0000 7f0c89600700 1 --2- 172.21.15.37:0/705459617 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0c84102e00 0x7f0c841031d0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f0c74004940 tx=0x7f0c74009e30 comp rx=0 tx=0).stop 2024-03-29T08:19:02.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:02.716+0000 7f0c89600700 1 -- 172.21.15.37:0/705459617 shutdown_connections 2024-03-29T08:19:02.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:02.716+0000 7f0c89600700 1 --2- 172.21.15.37:0/705459617 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0c841037a0 0x7f0c840fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:02.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:02.716+0000 7f0c89600700 1 --2- 172.21.15.37:0/705459617 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0c84102e00 0x7f0c841031d0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:02.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:02.716+0000 7f0c89600700 1 -- 172.21.15.37:0/705459617 >> 172.21.15.37:0/705459617 conn(0x7f0c840f89b0 msgr2=0x7f0c840fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:19:02.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:02.716+0000 7f0c89600700 1 -- 172.21.15.37:0/705459617 shutdown_connections 2024-03-29T08:19:02.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:02.716+0000 7f0c89600700 1 -- 172.21.15.37:0/705459617 wait complete. 2024-03-29T08:19:02.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:02.717+0000 7f0c89600700 1 Processor -- start 2024-03-29T08:19:02.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:02.717+0000 7f0c89600700 1 -- start start 2024-03-29T08:19:02.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:02.718+0000 7f0c89600700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0c84102e00 0x7f0c84113a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:02.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:02.718+0000 7f0c89600700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0c841037a0 0x7f0c84113f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:02.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:02.718+0000 7f0c89600700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c841145d0 con 0x7f0c841037a0 2024-03-29T08:19:02.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:02.718+0000 7f0c89600700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c8410d9c0 con 0x7f0c84102e00 2024-03-29T08:19:02.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:02.718+0000 7f0c82000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0c841037a0 0x7f0c84113f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:02.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:02.718+0000 7f0c82000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0c841037a0 0x7f0c84113f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42248/0 (socket says 172.21.15.37:42248) 2024-03-29T08:19:02.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:02.718+0000 7f0c82000700 1 -- 172.21.15.37:0/1476736667 learned_addr learned my addr 172.21.15.37:0/1476736667 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:19:02.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:02.718+0000 7f0c82a00700 1 --2- 172.21.15.37:0/1476736667 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0c84102e00 0x7f0c84113a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:02.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:02.718+0000 7f0c82000700 1 -- 172.21.15.37:0/1476736667 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0c84102e00 msgr2=0x7f0c84113a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:02.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:02.718+0000 7f0c82000700 1 --2- 172.21.15.37:0/1476736667 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0c84102e00 0x7f0c84113a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:02.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:02.718+0000 7f0c82000700 1 -- 172.21.15.37:0/1476736667 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c74005040 con 0x7f0c841037a0 2024-03-29T08:19:02.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:02.719+0000 7f0c82000700 1 --2- 172.21.15.37:0/1476736667 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0c841037a0 0x7f0c84113f80 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f0c7000eb40 tx=0x7f0c7000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:02.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:02.719+0000 7f0c7aa00700 1 -- 172.21.15.37:0/1476736667 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0c70007cc0 con 0x7f0c841037a0 2024-03-29T08:19:02.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:02.719+0000 7f0c7aa00700 1 -- 172.21.15.37:0/1476736667 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f0c7000f040 con 0x7f0c841037a0 2024-03-29T08:19:02.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:02.719+0000 7f0c89600700 1 -- 172.21.15.37:0/1476736667 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c8410dc40 con 0x7f0c841037a0 2024-03-29T08:19:02.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:02.719+0000 7f0c7aa00700 1 -- 172.21.15.37:0/1476736667 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0c7000c8f0 con 0x7f0c841037a0 2024-03-29T08:19:02.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:02.719+0000 7f0c89600700 1 -- 172.21.15.37:0/1476736667 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c8410e200 con 0x7f0c841037a0 2024-03-29T08:19:02.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:02.720+0000 7f0c89600700 1 -- 172.21.15.37:0/1476736667 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c840fe550 con 0x7f0c841037a0 2024-03-29T08:19:02.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:02.721+0000 7f0c7aa00700 1 -- 172.21.15.37:0/1476736667 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f0c700090d0 con 0x7f0c841037a0 2024-03-29T08:19:02.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:02.722+0000 7f0c7aa00700 1 --2- 172.21.15.37:0/1476736667 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f0c6806cc30 0x7f0c6806f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:02.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:02.722+0000 7f0c7aa00700 1 -- 172.21.15.37:0/1476736667 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f0c7008b9d0 con 0x7f0c841037a0 2024-03-29T08:19:02.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:02.722+0000 7f0c82a00700 1 --2- 172.21.15.37:0/1476736667 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f0c6806cc30 0x7f0c6806f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:02.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:02.723+0000 7f0c82a00700 1 --2- 172.21.15.37:0/1476736667 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f0c6806cc30 0x7f0c6806f0e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f0c7400b0d0 tx=0x7f0c74009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:02.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:02.727+0000 7f0c7aa00700 1 -- 172.21.15.37:0/1476736667 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f0c70059550 con 0x7f0c841037a0 2024-03-29T08:19:02.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:02.986+0000 7f0c89600700 1 -- 172.21.15.37:0/1476736667 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f0c84109dc0 con 0x7f0c841037a0 2024-03-29T08:19:02.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:02.987+0000 7f0c7aa00700 1 -- 172.21.15.37:0/1476736667 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 28 v28) v1 ==== 76+0+1841 (secure 0 0 0) 0x7f0c700590e0 con 0x7f0c841037a0 2024-03-29T08:19:02.987 INFO:teuthology.orchestra.run.smithi037.stdout:e28 2024-03-29T08:19:02.987 INFO:teuthology.orchestra.run.smithi037.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-29T08:19:02.987 INFO:teuthology.orchestra.run.smithi037.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T08:19:02.987 INFO:teuthology.orchestra.run.smithi037.stdout:legacy client fscid: 1 2024-03-29T08:19:02.987 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:19:02.987 INFO:teuthology.orchestra.run.smithi037.stdout:Filesystem 'cephfs' (1) 2024-03-29T08:19:02.987 INFO:teuthology.orchestra.run.smithi037.stdout:fs_name cephfs 2024-03-29T08:19:02.988 INFO:teuthology.orchestra.run.smithi037.stdout:epoch 21 2024-03-29T08:19:02.988 INFO:teuthology.orchestra.run.smithi037.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-03-29T08:19:02.988 INFO:teuthology.orchestra.run.smithi037.stdout:created 2024-03-29T08:03:29.241269+0000 2024-03-29T08:19:02.988 INFO:teuthology.orchestra.run.smithi037.stdout:modified 2024-03-29T08:17:16.237238+0000 2024-03-29T08:19:02.988 INFO:teuthology.orchestra.run.smithi037.stdout:tableserver 0 2024-03-29T08:19:02.988 INFO:teuthology.orchestra.run.smithi037.stdout:root 0 2024-03-29T08:19:02.988 INFO:teuthology.orchestra.run.smithi037.stdout:session_timeout 60 2024-03-29T08:19:02.988 INFO:teuthology.orchestra.run.smithi037.stdout:session_autoclose 300 2024-03-29T08:19:02.988 INFO:teuthology.orchestra.run.smithi037.stdout:max_file_size 1099511627776 2024-03-29T08:19:02.988 INFO:teuthology.orchestra.run.smithi037.stdout:max_xattr_size 65536 2024-03-29T08:19:02.988 INFO:teuthology.orchestra.run.smithi037.stdout:required_client_features {} 2024-03-29T08:19:02.988 INFO:teuthology.orchestra.run.smithi037.stdout:last_failure 0 2024-03-29T08:19:02.988 INFO:teuthology.orchestra.run.smithi037.stdout:last_failure_osd_epoch 82 2024-03-29T08:19:02.988 INFO:teuthology.orchestra.run.smithi037.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T08:19:02.988 INFO:teuthology.orchestra.run.smithi037.stdout:max_mds 2 2024-03-29T08:19:02.989 INFO:teuthology.orchestra.run.smithi037.stdout:in 0,1 2024-03-29T08:19:02.989 INFO:teuthology.orchestra.run.smithi037.stdout:up {} 2024-03-29T08:19:02.989 INFO:teuthology.orchestra.run.smithi037.stdout:failed 0,1 2024-03-29T08:19:02.989 INFO:teuthology.orchestra.run.smithi037.stdout:damaged 2024-03-29T08:19:02.989 INFO:teuthology.orchestra.run.smithi037.stdout:stopped 2024-03-29T08:19:02.989 INFO:teuthology.orchestra.run.smithi037.stdout:data_pools [3] 2024-03-29T08:19:02.989 INFO:teuthology.orchestra.run.smithi037.stdout:metadata_pool 2 2024-03-29T08:19:02.989 INFO:teuthology.orchestra.run.smithi037.stdout:inline_data disabled 2024-03-29T08:19:02.989 INFO:teuthology.orchestra.run.smithi037.stdout:balancer 2024-03-29T08:19:02.989 INFO:teuthology.orchestra.run.smithi037.stdout:bal_rank_mask -1 2024-03-29T08:19:02.989 INFO:teuthology.orchestra.run.smithi037.stdout:standby_count_wanted 1 2024-03-29T08:19:02.989 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:19:02.989 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:19:02.989 INFO:teuthology.orchestra.run.smithi037.stdout:Standby daemons: 2024-03-29T08:19:02.989 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:19:02.989 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi148.qukpjh{-1:24319} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.148:6826/2641327912,v1:172.21.15.148:6827/2641327912] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:19:02.990 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi037.mmwgxe{-1:34440} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.37:6828/1248142005,v1:172.21.15.37:6829/1248142005] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:19:02.990 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi037.iifsmh{-1:34444} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.37:6826/2060882959,v1:172.21.15.37:6827/2060882959] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:19:02.990 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi148.sqteih{-1:44395} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.148:6824/4078652242,v1:172.21.15.148:6825/4078652242] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:19:02.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:02.989+0000 7f0c89600700 1 -- 172.21.15.37:0/1476736667 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f0c6806cc30 msgr2=0x7f0c6806f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:02.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:02.989+0000 7f0c89600700 1 --2- 172.21.15.37:0/1476736667 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f0c6806cc30 0x7f0c6806f0e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f0c7400b0d0 tx=0x7f0c74009e30 comp rx=0 tx=0).stop 2024-03-29T08:19:02.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:02.989+0000 7f0c89600700 1 -- 172.21.15.37:0/1476736667 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0c841037a0 msgr2=0x7f0c84113f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:02.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:02.989+0000 7f0c89600700 1 --2- 172.21.15.37:0/1476736667 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0c841037a0 0x7f0c84113f80 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f0c7000eb40 tx=0x7f0c7000b040 comp rx=0 tx=0).stop 2024-03-29T08:19:02.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:02.989+0000 7f0c89600700 1 -- 172.21.15.37:0/1476736667 shutdown_connections 2024-03-29T08:19:02.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:02.989+0000 7f0c89600700 1 --2- 172.21.15.37:0/1476736667 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f0c6806cc30 0x7f0c6806f0e0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:02.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:02.989+0000 7f0c89600700 1 --2- 172.21.15.37:0/1476736667 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0c84102e00 0x7f0c84113a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:02.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:02.989+0000 7f0c89600700 1 --2- 172.21.15.37:0/1476736667 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0c841037a0 0x7f0c84113f80 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:02.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:02.989+0000 7f0c89600700 1 -- 172.21.15.37:0/1476736667 >> 172.21.15.37:0/1476736667 conn(0x7f0c840f89b0 msgr2=0x7f0c840f99f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:19:02.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:02.989+0000 7f0c89600700 1 -- 172.21.15.37:0/1476736667 shutdown_connections 2024-03-29T08:19:02.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:02.989+0000 7f0c89600700 1 -- 172.21.15.37:0/1476736667 wait complete. 2024-03-29T08:19:02.992 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 28 2024-03-29T08:19:03.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:19:02 smithi148 ceph-mon[106692]: from='client.34508 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:19:03.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:19:02 smithi148 ceph-mon[106692]: from='client.34512 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:19:03.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:19:02 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/1023141857' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:19:03.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:03.126+0000 7f5e3c800700 1 -- 172.21.15.37:0/3498050334 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5e34102e00 msgr2=0x7f5e341031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:03.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:03.126+0000 7f5e3c800700 1 --2- 172.21.15.37:0/3498050334 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5e34102e00 0x7f5e341031d0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f5e28004940 tx=0x7f5e28009e30 comp rx=0 tx=0).stop 2024-03-29T08:19:03.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:03.127+0000 7f5e3c800700 1 -- 172.21.15.37:0/3498050334 shutdown_connections 2024-03-29T08:19:03.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:03.127+0000 7f5e3c800700 1 --2- 172.21.15.37:0/3498050334 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5e341037a0 0x7f5e340fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:03.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:03.127+0000 7f5e3c800700 1 --2- 172.21.15.37:0/3498050334 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5e34102e00 0x7f5e341031d0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:03.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:03.127+0000 7f5e3c800700 1 -- 172.21.15.37:0/3498050334 >> 172.21.15.37:0/3498050334 conn(0x7f5e340f89b0 msgr2=0x7f5e340fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:19:03.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:03.127+0000 7f5e3c800700 1 -- 172.21.15.37:0/3498050334 shutdown_connections 2024-03-29T08:19:03.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:03.127+0000 7f5e3c800700 1 -- 172.21.15.37:0/3498050334 wait complete. 2024-03-29T08:19:03.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:03.127+0000 7f5e3c800700 1 Processor -- start 2024-03-29T08:19:03.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:03.128+0000 7f5e3c800700 1 -- start start 2024-03-29T08:19:03.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:03.128+0000 7f5e3c800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5e34102e00 0x7f5e34115cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:03.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:03.128+0000 7f5e3c800700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5e341037a0 0x7f5e34116210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:03.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:03.128+0000 7f5e3c800700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e34116860 con 0x7f5e34102e00 2024-03-29T08:19:03.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:03.128+0000 7f5e3c800700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e3410fc30 con 0x7f5e341037a0 2024-03-29T08:19:03.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:03.128+0000 7f5e39e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5e34102e00 0x7f5e34115cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:03.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:03.128+0000 7f5e39e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5e34102e00 0x7f5e34115cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42276/0 (socket says 172.21.15.37:42276) 2024-03-29T08:19:03.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:03.128+0000 7f5e39e00700 1 -- 172.21.15.37:0/1098991817 learned_addr learned my addr 172.21.15.37:0/1098991817 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:19:03.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:03.129+0000 7f5e39400700 1 --2- 172.21.15.37:0/1098991817 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5e341037a0 0x7f5e34116210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:03.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:03.129+0000 7f5e39e00700 1 -- 172.21.15.37:0/1098991817 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5e341037a0 msgr2=0x7f5e34116210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:03.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:03.129+0000 7f5e39e00700 1 --2- 172.21.15.37:0/1098991817 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5e341037a0 0x7f5e34116210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:03.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:03.129+0000 7f5e39e00700 1 -- 172.21.15.37:0/1098991817 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e28005040 con 0x7f5e34102e00 2024-03-29T08:19:03.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:03.129+0000 7f5e39400700 1 --2- 172.21.15.37:0/1098991817 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5e341037a0 0x7f5e34116210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:19:03.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:03.129+0000 7f5e39e00700 1 --2- 172.21.15.37:0/1098991817 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5e34102e00 0x7f5e34115cd0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f5e2800b3b0 tx=0x7f5e28009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:03.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:03.129+0000 7f5e2e000700 1 -- 172.21.15.37:0/1098991817 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5e2800e040 con 0x7f5e34102e00 2024-03-29T08:19:03.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:03.129+0000 7f5e2e000700 1 -- 172.21.15.37:0/1098991817 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f5e28013420 con 0x7f5e34102e00 2024-03-29T08:19:03.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:03.129+0000 7f5e3c800700 1 -- 172.21.15.37:0/1098991817 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e3410feb0 con 0x7f5e34102e00 2024-03-29T08:19:03.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:03.129+0000 7f5e2e000700 1 -- 172.21.15.37:0/1098991817 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5e2801c630 con 0x7f5e34102e00 2024-03-29T08:19:03.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:03.129+0000 7f5e3c800700 1 -- 172.21.15.37:0/1098991817 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e34110470 con 0x7f5e34102e00 2024-03-29T08:19:03.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:03.131+0000 7f5e3c800700 1 -- 172.21.15.37:0/1098991817 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e340fe550 con 0x7f5e34102e00 2024-03-29T08:19:03.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:03.131+0000 7f5e2e000700 1 -- 172.21.15.37:0/1098991817 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f5e28013970 con 0x7f5e34102e00 2024-03-29T08:19:03.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:03.132+0000 7f5e2e000700 1 --2- 172.21.15.37:0/1098991817 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f5e1c06cc20 0x7f5e1c06f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:03.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:03.132+0000 7f5e2e000700 1 -- 172.21.15.37:0/1098991817 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f5e2808d320 con 0x7f5e34102e00 2024-03-29T08:19:03.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:03.132+0000 7f5e39400700 1 --2- 172.21.15.37:0/1098991817 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f5e1c06cc20 0x7f5e1c06f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:03.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:03.133+0000 7f5e39400700 1 --2- 172.21.15.37:0/1098991817 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f5e1c06cc20 0x7f5e1c06f0d0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f5e24007fd0 tx=0x7f5e24009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:03.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:03.137+0000 7f5e2e000700 1 -- 172.21.15.37:0/1098991817 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f5e2805ad70 con 0x7f5e34102e00 2024-03-29T08:19:03.149 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:19:02 smithi037 ceph-mon[142325]: from='client.34508 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:19:03.149 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:19:02 smithi037 ceph-mon[142325]: from='client.34512 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:19:03.149 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:19:02 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/1023141857' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:19:03.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:03.370+0000 7f5e3c800700 1 -- 172.21.15.37:0/1098991817 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5e3404bed0 con 0x7f5e1c06cc20 2024-03-29T08:19:03.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:03.371+0000 7f5e2e000700 1 -- 172.21.15.37:0/1098991817 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f5e3404bed0 con 0x7f5e1c06cc20 2024-03-29T08:19:03.371 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-29T08:19:03.371 INFO:teuthology.orchestra.run.smithi037.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59", 2024-03-29T08:19:03.372 INFO:teuthology.orchestra.run.smithi037.stdout: "in_progress": true, 2024-03-29T08:19:03.372 INFO:teuthology.orchestra.run.smithi037.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-29T08:19:03.372 INFO:teuthology.orchestra.run.smithi037.stdout: "services_complete": [ 2024-03-29T08:19:03.372 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr", 2024-03-29T08:19:03.372 INFO:teuthology.orchestra.run.smithi037.stdout: "mon", 2024-03-29T08:19:03.372 INFO:teuthology.orchestra.run.smithi037.stdout: "crash", 2024-03-29T08:19:03.372 INFO:teuthology.orchestra.run.smithi037.stdout: "osd" 2024-03-29T08:19:03.372 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-29T08:19:03.372 INFO:teuthology.orchestra.run.smithi037.stdout: "progress": "15/21 daemons upgraded", 2024-03-29T08:19:03.372 INFO:teuthology.orchestra.run.smithi037.stdout: "message": "Currently upgrading mds daemons", 2024-03-29T08:19:03.372 INFO:teuthology.orchestra.run.smithi037.stdout: "is_paused": false 2024-03-29T08:19:03.372 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-29T08:19:03.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:03.374+0000 7f5e3c800700 1 -- 172.21.15.37:0/1098991817 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f5e1c06cc20 msgr2=0x7f5e1c06f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:03.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:03.374+0000 7f5e3c800700 1 --2- 172.21.15.37:0/1098991817 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f5e1c06cc20 0x7f5e1c06f0d0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f5e24007fd0 tx=0x7f5e24009040 comp rx=0 tx=0).stop 2024-03-29T08:19:03.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:03.374+0000 7f5e3c800700 1 -- 172.21.15.37:0/1098991817 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5e34102e00 msgr2=0x7f5e34115cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:03.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:03.374+0000 7f5e3c800700 1 --2- 172.21.15.37:0/1098991817 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5e34102e00 0x7f5e34115cd0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f5e2800b3b0 tx=0x7f5e28009e30 comp rx=0 tx=0).stop 2024-03-29T08:19:03.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:03.374+0000 7f5e3c800700 1 -- 172.21.15.37:0/1098991817 shutdown_connections 2024-03-29T08:19:03.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:03.374+0000 7f5e3c800700 1 --2- 172.21.15.37:0/1098991817 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f5e1c06cc20 0x7f5e1c06f0d0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:03.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:03.374+0000 7f5e3c800700 1 --2- 172.21.15.37:0/1098991817 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5e34102e00 0x7f5e34115cd0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:03.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:03.374+0000 7f5e3c800700 1 --2- 172.21.15.37:0/1098991817 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5e341037a0 0x7f5e34116210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:03.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:03.374+0000 7f5e3c800700 1 -- 172.21.15.37:0/1098991817 >> 172.21.15.37:0/1098991817 conn(0x7f5e340f89b0 msgr2=0x7f5e340f9980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:19:03.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:03.374+0000 7f5e3c800700 1 -- 172.21.15.37:0/1098991817 shutdown_connections 2024-03-29T08:19:03.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:03.374+0000 7f5e3c800700 1 -- 172.21.15.37:0/1098991817 wait complete. 2024-03-29T08:19:03.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:03.511+0000 7f0791800700 1 -- 172.21.15.37:0/3576941105 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f078c1037a0 msgr2=0x7f078c0fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:03.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:03.511+0000 7f0791800700 1 --2- 172.21.15.37:0/3576941105 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f078c1037a0 0x7f078c0fce10 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f0778004940 tx=0x7f0778009e30 comp rx=0 tx=0).stop 2024-03-29T08:19:03.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:03.511+0000 7f0791800700 1 -- 172.21.15.37:0/3576941105 shutdown_connections 2024-03-29T08:19:03.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:03.511+0000 7f0791800700 1 --2- 172.21.15.37:0/3576941105 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f078c1037a0 0x7f078c0fce10 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:03.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:03.511+0000 7f0791800700 1 --2- 172.21.15.37:0/3576941105 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f078c102e00 0x7f078c1031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:03.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:03.511+0000 7f0791800700 1 -- 172.21.15.37:0/3576941105 >> 172.21.15.37:0/3576941105 conn(0x7f078c0f89b0 msgr2=0x7f078c0fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:19:03.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:03.511+0000 7f0791800700 1 -- 172.21.15.37:0/3576941105 shutdown_connections 2024-03-29T08:19:03.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:03.511+0000 7f0791800700 1 -- 172.21.15.37:0/3576941105 wait complete. 2024-03-29T08:19:03.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:03.512+0000 7f0791800700 1 Processor -- start 2024-03-29T08:19:03.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:03.512+0000 7f0791800700 1 -- start start 2024-03-29T08:19:03.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:03.512+0000 7f0791800700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f078c102e00 0x7f078c113a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:03.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:03.513+0000 7f0791800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f078c1037a0 0x7f078c113fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:03.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:03.513+0000 7f0791800700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f078c114600 con 0x7f078c1037a0 2024-03-29T08:19:03.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:03.513+0000 7f0791800700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f078c10d9f0 con 0x7f078c102e00 2024-03-29T08:19:03.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:03.513+0000 7f078a000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f078c1037a0 0x7f078c113fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:03.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:03.513+0000 7f078a000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f078c1037a0 0x7f078c113fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42290/0 (socket says 172.21.15.37:42290) 2024-03-29T08:19:03.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:03.513+0000 7f078a000700 1 -- 172.21.15.37:0/800165542 learned_addr learned my addr 172.21.15.37:0/800165542 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:19:03.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:03.513+0000 7f078aa00700 1 --2- 172.21.15.37:0/800165542 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f078c102e00 0x7f078c113a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:03.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:03.513+0000 7f078a000700 1 -- 172.21.15.37:0/800165542 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f078c102e00 msgr2=0x7f078c113a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:03.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:03.513+0000 7f078a000700 1 --2- 172.21.15.37:0/800165542 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f078c102e00 0x7f078c113a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:03.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:03.513+0000 7f078a000700 1 -- 172.21.15.37:0/800165542 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0778005040 con 0x7f078c1037a0 2024-03-29T08:19:03.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:03.513+0000 7f078a000700 1 --2- 172.21.15.37:0/800165542 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f078c1037a0 0x7f078c113fb0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f0778009fd0 tx=0x7f0778009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:03.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:03.514+0000 7f0782a00700 1 -- 172.21.15.37:0/800165542 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f077800e030 con 0x7f078c1037a0 2024-03-29T08:19:03.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:03.514+0000 7f0782a00700 1 -- 172.21.15.37:0/800165542 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f0778013e00 con 0x7f078c1037a0 2024-03-29T08:19:03.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:03.514+0000 7f0791800700 1 -- 172.21.15.37:0/800165542 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f078c10dc70 con 0x7f078c1037a0 2024-03-29T08:19:03.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:03.514+0000 7f0782a00700 1 -- 172.21.15.37:0/800165542 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f077801ccd0 con 0x7f078c1037a0 2024-03-29T08:19:03.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:03.514+0000 7f0791800700 1 -- 172.21.15.37:0/800165542 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f078c10e230 con 0x7f078c1037a0 2024-03-29T08:19:03.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:03.515+0000 7f0791800700 1 -- 172.21.15.37:0/800165542 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f078c0fe550 con 0x7f078c1037a0 2024-03-29T08:19:03.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:03.516+0000 7f0782a00700 1 -- 172.21.15.37:0/800165542 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f077801c500 con 0x7f078c1037a0 2024-03-29T08:19:03.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:03.517+0000 7f0782a00700 1 --2- 172.21.15.37:0/800165542 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f077006cc30 0x7f077006f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:03.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:03.517+0000 7f0782a00700 1 -- 172.21.15.37:0/800165542 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f077808d900 con 0x7f078c1037a0 2024-03-29T08:19:03.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:03.517+0000 7f078aa00700 1 --2- 172.21.15.37:0/800165542 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f077006cc30 0x7f077006f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:03.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:03.518+0000 7f078aa00700 1 --2- 172.21.15.37:0/800165542 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f077006cc30 0x7f077006f0e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f077c009fd0 tx=0x7f077c00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:03.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:03.522+0000 7f0782a00700 1 -- 172.21.15.37:0/800165542 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f077805b3d0 con 0x7f078c1037a0 2024-03-29T08:19:03.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:03.827+0000 7f0791800700 1 -- 172.21.15.37:0/800165542 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f078c109da0 con 0x7f078c1037a0 2024-03-29T08:19:03.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:03.827+0000 7f0782a00700 1 -- 172.21.15.37:0/800165542 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7f0778004490 con 0x7f078c1037a0 2024-03-29T08:19:03.828 INFO:teuthology.orchestra.run.smithi037.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-03-29T08:19:03.828 INFO:teuthology.orchestra.run.smithi037.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-03-29T08:19:03.828 INFO:teuthology.orchestra.run.smithi037.stdout: fs cephfs is degraded 2024-03-29T08:19:03.828 INFO:teuthology.orchestra.run.smithi037.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-03-29T08:19:03.828 INFO:teuthology.orchestra.run.smithi037.stdout: fs cephfs is offline because no MDS is active for it. 2024-03-29T08:19:03.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:03.830+0000 7f0791800700 1 -- 172.21.15.37:0/800165542 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f077006cc30 msgr2=0x7f077006f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:03.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:03.830+0000 7f0791800700 1 --2- 172.21.15.37:0/800165542 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f077006cc30 0x7f077006f0e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f077c009fd0 tx=0x7f077c00b040 comp rx=0 tx=0).stop 2024-03-29T08:19:03.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:03.830+0000 7f0791800700 1 -- 172.21.15.37:0/800165542 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f078c1037a0 msgr2=0x7f078c113fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:03.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:03.830+0000 7f0791800700 1 --2- 172.21.15.37:0/800165542 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f078c1037a0 0x7f078c113fb0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f0778009fd0 tx=0x7f0778009e30 comp rx=0 tx=0).stop 2024-03-29T08:19:03.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:03.830+0000 7f0791800700 1 -- 172.21.15.37:0/800165542 shutdown_connections 2024-03-29T08:19:03.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:03.830+0000 7f0791800700 1 --2- 172.21.15.37:0/800165542 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f077006cc30 0x7f077006f0e0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:03.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:03.830+0000 7f0791800700 1 --2- 172.21.15.37:0/800165542 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f078c102e00 0x7f078c113a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:03.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:03.830+0000 7f0791800700 1 --2- 172.21.15.37:0/800165542 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f078c1037a0 0x7f078c113fb0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:03.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:03.830+0000 7f0791800700 1 -- 172.21.15.37:0/800165542 >> 172.21.15.37:0/800165542 conn(0x7f078c0f89b0 msgr2=0x7f078c0f9a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:19:03.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:03.830+0000 7f0791800700 1 -- 172.21.15.37:0/800165542 shutdown_connections 2024-03-29T08:19:03.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:03.830+0000 7f0791800700 1 -- 172.21.15.37:0/800165542 wait complete. 2024-03-29T08:19:04.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:19:03 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/1476736667' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T08:19:04.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:19:03 smithi148 ceph-mon[106692]: from='client.34524 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:19:04.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:19:03 smithi148 ceph-mon[106692]: pgmap v245: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:19:04.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:19:03 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/1476736667' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T08:19:04.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:19:03 smithi037 ceph-mon[142325]: from='client.34524 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:19:04.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:19:03 smithi037 ceph-mon[142325]: pgmap v245: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:19:05.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:19:04 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/800165542' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T08:19:05.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:19:04 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/800165542' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T08:19:06.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:19:05 smithi148 ceph-mon[106692]: pgmap v246: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:19:06.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:19:05 smithi037 ceph-mon[142325]: pgmap v246: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:19:08.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:19:07 smithi148 ceph-mon[106692]: pgmap v247: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:19:08.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:19:07 smithi037 ceph-mon[142325]: pgmap v247: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:19:10.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:19:10 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:19:10.428 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:19:10 smithi037 ceph-mon[142325]: pgmap v248: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:19:10.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:19:10 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:19:10.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:19:10 smithi148 ceph-mon[106692]: pgmap v248: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:19:12.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:19:11 smithi148 ceph-mon[106692]: pgmap v249: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:19:12.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:19:11 smithi037 ceph-mon[142325]: pgmap v249: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:19:14.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:19:13 smithi148 ceph-mon[106692]: pgmap v250: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:19:14.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:19:13 smithi037 ceph-mon[142325]: pgmap v250: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:19:16.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:19:15 smithi148 ceph-mon[106692]: pgmap v251: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:19:16.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:19:15 smithi037 ceph-mon[142325]: pgmap v251: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:19:18.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:19:17 smithi148 ceph-mon[106692]: pgmap v252: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:19:18.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:19:17 smithi037 ceph-mon[142325]: pgmap v252: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:19:20.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:19:20 smithi037 ceph-mon[142325]: pgmap v253: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:19:20.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:19:20 smithi148 ceph-mon[106692]: pgmap v253: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:19:22.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:19:21 smithi148 ceph-mon[106692]: pgmap v254: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:19:22.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:19:21 smithi037 ceph-mon[142325]: pgmap v254: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:19:24.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:19:23 smithi148 ceph-mon[106692]: pgmap v255: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:19:24.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:19:23 smithi037 ceph-mon[142325]: pgmap v255: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:19:25.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:19:25 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:19:25.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:19:25 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:19:26.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:19:26 smithi037 ceph-mon[142325]: pgmap v256: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:19:26.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:19:26 smithi148 ceph-mon[106692]: pgmap v256: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:19:28.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:19:27 smithi148 ceph-mon[106692]: pgmap v257: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:19:28.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:19:27 smithi037 ceph-mon[142325]: pgmap v257: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:19:30.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:19:30 smithi037 ceph-mon[142325]: pgmap v258: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:19:30.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:19:30 smithi148 ceph-mon[106692]: pgmap v258: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:19:32.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:19:31 smithi148 ceph-mon[106692]: pgmap v259: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:19:32.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:19:31 smithi037 ceph-mon[142325]: pgmap v259: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:19:33.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:33.979+0000 7f7acce00700 1 -- 172.21.15.37:0/3894531382 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ac8102e00 msgr2=0x7f7ac81031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:33.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:33.979+0000 7f7acce00700 1 --2- 172.21.15.37:0/3894531382 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ac8102e00 0x7f7ac81031d0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f7ab8004940 tx=0x7f7ab8009e30 comp rx=0 tx=0).stop 2024-03-29T08:19:33.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:33.979+0000 7f7acce00700 1 -- 172.21.15.37:0/3894531382 shutdown_connections 2024-03-29T08:19:33.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:33.979+0000 7f7acce00700 1 --2- 172.21.15.37:0/3894531382 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7ac81037a0 0x7f7ac80fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:33.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:33.979+0000 7f7acce00700 1 --2- 172.21.15.37:0/3894531382 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ac8102e00 0x7f7ac81031d0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:33.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:33.979+0000 7f7acce00700 1 -- 172.21.15.37:0/3894531382 >> 172.21.15.37:0/3894531382 conn(0x7f7ac80f89b0 msgr2=0x7f7ac80fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:19:33.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:33.979+0000 7f7acce00700 1 -- 172.21.15.37:0/3894531382 shutdown_connections 2024-03-29T08:19:33.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:33.979+0000 7f7acce00700 1 -- 172.21.15.37:0/3894531382 wait complete. 2024-03-29T08:19:33.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:33.980+0000 7f7acce00700 1 Processor -- start 2024-03-29T08:19:33.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:33.980+0000 7f7acce00700 1 -- start start 2024-03-29T08:19:33.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:33.980+0000 7f7acce00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ac8102e00 0x7f7ac80702c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:33.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:33.980+0000 7f7acce00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7ac81037a0 0x7f7ac8070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:33.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:33.980+0000 7f7acce00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ac8070d90 con 0x7f7ac8102e00 2024-03-29T08:19:33.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:33.980+0000 7f7acce00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ac8070ed0 con 0x7f7ac81037a0 2024-03-29T08:19:33.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:33.981+0000 7f7ac5e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ac8102e00 0x7f7ac80702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:33.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:33.981+0000 7f7ac5e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ac8102e00 0x7f7ac80702c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40272/0 (socket says 172.21.15.37:40272) 2024-03-29T08:19:33.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:33.981+0000 7f7ac5e00700 1 -- 172.21.15.37:0/2168809568 learned_addr learned my addr 172.21.15.37:0/2168809568 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:19:33.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:33.981+0000 7f7ac5400700 1 --2- 172.21.15.37:0/2168809568 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7ac81037a0 0x7f7ac8070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:33.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:33.981+0000 7f7ac5e00700 1 -- 172.21.15.37:0/2168809568 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7ac81037a0 msgr2=0x7f7ac8070800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:33.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:33.981+0000 7f7ac5e00700 1 --2- 172.21.15.37:0/2168809568 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7ac81037a0 0x7f7ac8070800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:33.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:33.981+0000 7f7ac5e00700 1 -- 172.21.15.37:0/2168809568 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ab8005040 con 0x7f7ac8102e00 2024-03-29T08:19:33.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:33.981+0000 7f7ac5e00700 1 --2- 172.21.15.37:0/2168809568 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ac8102e00 0x7f7ac80702c0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f7ab8009360 tx=0x7f7ab8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:33.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:33.982+0000 7f7abe000700 1 -- 172.21.15.37:0/2168809568 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7ab800e040 con 0x7f7ac8102e00 2024-03-29T08:19:33.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:33.982+0000 7f7abe000700 1 -- 172.21.15.37:0/2168809568 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f7ab8013420 con 0x7f7ac8102e00 2024-03-29T08:19:33.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:33.982+0000 7f7abe000700 1 -- 172.21.15.37:0/2168809568 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7ab801c5f0 con 0x7f7ac8102e00 2024-03-29T08:19:33.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:33.982+0000 7f7acce00700 1 -- 172.21.15.37:0/2168809568 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ac8071120 con 0x7f7ac8102e00 2024-03-29T08:19:33.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:33.982+0000 7f7acce00700 1 -- 172.21.15.37:0/2168809568 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ac811a250 con 0x7f7ac8102e00 2024-03-29T08:19:33.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:33.983+0000 7f7acce00700 1 -- 172.21.15.37:0/2168809568 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ac80fe550 con 0x7f7ac8102e00 2024-03-29T08:19:33.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:33.984+0000 7f7abe000700 1 -- 172.21.15.37:0/2168809568 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f7ab8013590 con 0x7f7ac8102e00 2024-03-29T08:19:33.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:33.985+0000 7f7abe000700 1 --2- 172.21.15.37:0/2168809568 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f7aac06cc30 0x7f7aac06f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:33.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:33.985+0000 7f7abe000700 1 -- 172.21.15.37:0/2168809568 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f7ab808d270 con 0x7f7ac8102e00 2024-03-29T08:19:33.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:33.985+0000 7f7ac5400700 1 --2- 172.21.15.37:0/2168809568 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f7aac06cc30 0x7f7aac06f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:33.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:33.986+0000 7f7ac5400700 1 --2- 172.21.15.37:0/2168809568 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f7aac06cc30 0x7f7aac06f0e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f7ac8071c00 tx=0x7f7ab4009ee0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:33.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:33.990+0000 7f7abe000700 1 -- 172.21.15.37:0/2168809568 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f7ab80572b0 con 0x7f7ac8102e00 2024-03-29T08:19:34.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:19:33 smithi148 ceph-mon[106692]: pgmap v260: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:19:34.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:19:33 smithi037 ceph-mon[142325]: pgmap v260: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:19:34.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:34.224+0000 7f7acce00700 1 -- 172.21.15.37:0/2168809568 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7ac80719b0 con 0x7f7aac06cc30 2024-03-29T08:19:34.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:34.225+0000 7f7abe000700 1 -- 172.21.15.37:0/2168809568 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f7ac80719b0 con 0x7f7aac06cc30 2024-03-29T08:19:34.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:34.227+0000 7f7acce00700 1 -- 172.21.15.37:0/2168809568 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f7aac06cc30 msgr2=0x7f7aac06f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:34.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:34.227+0000 7f7acce00700 1 --2- 172.21.15.37:0/2168809568 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f7aac06cc30 0x7f7aac06f0e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f7ac8071c00 tx=0x7f7ab4009ee0 comp rx=0 tx=0).stop 2024-03-29T08:19:34.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:34.227+0000 7f7acce00700 1 -- 172.21.15.37:0/2168809568 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ac8102e00 msgr2=0x7f7ac80702c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:34.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:34.227+0000 7f7acce00700 1 --2- 172.21.15.37:0/2168809568 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ac8102e00 0x7f7ac80702c0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f7ab8009360 tx=0x7f7ab8009e30 comp rx=0 tx=0).stop 2024-03-29T08:19:34.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:34.227+0000 7f7acce00700 1 -- 172.21.15.37:0/2168809568 shutdown_connections 2024-03-29T08:19:34.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:34.227+0000 7f7acce00700 1 --2- 172.21.15.37:0/2168809568 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f7aac06cc30 0x7f7aac06f0e0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:34.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:34.227+0000 7f7acce00700 1 --2- 172.21.15.37:0/2168809568 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ac8102e00 0x7f7ac80702c0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:34.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:34.227+0000 7f7acce00700 1 --2- 172.21.15.37:0/2168809568 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7ac81037a0 0x7f7ac8070800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:34.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:34.227+0000 7f7acce00700 1 -- 172.21.15.37:0/2168809568 >> 172.21.15.37:0/2168809568 conn(0x7f7ac80f89b0 msgr2=0x7f7ac80f9a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:19:34.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:34.228+0000 7f7acce00700 1 -- 172.21.15.37:0/2168809568 shutdown_connections 2024-03-29T08:19:34.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:34.228+0000 7f7acce00700 1 -- 172.21.15.37:0/2168809568 wait complete. 2024-03-29T08:19:34.243 INFO:teuthology.orchestra.run.smithi037.stdout:true 2024-03-29T08:19:34.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:34.367+0000 7fc3df200700 1 -- 172.21.15.37:0/2755382493 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc3d8102e00 msgr2=0x7fc3d81031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:34.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:34.367+0000 7fc3df200700 1 --2- 172.21.15.37:0/2755382493 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc3d8102e00 0x7fc3d81031d0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fc3c8004940 tx=0x7fc3c8009e30 comp rx=0 tx=0).stop 2024-03-29T08:19:34.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:34.367+0000 7fc3df200700 1 -- 172.21.15.37:0/2755382493 shutdown_connections 2024-03-29T08:19:34.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:34.367+0000 7fc3df200700 1 --2- 172.21.15.37:0/2755382493 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc3d81037a0 0x7fc3d80fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:34.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:34.367+0000 7fc3df200700 1 --2- 172.21.15.37:0/2755382493 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc3d8102e00 0x7fc3d81031d0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:34.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:34.367+0000 7fc3df200700 1 -- 172.21.15.37:0/2755382493 >> 172.21.15.37:0/2755382493 conn(0x7fc3d80f89b0 msgr2=0x7fc3d80fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:19:34.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:34.367+0000 7fc3df200700 1 -- 172.21.15.37:0/2755382493 shutdown_connections 2024-03-29T08:19:34.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:34.367+0000 7fc3df200700 1 -- 172.21.15.37:0/2755382493 wait complete. 2024-03-29T08:19:34.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:34.368+0000 7fc3df200700 1 Processor -- start 2024-03-29T08:19:34.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:34.368+0000 7fc3df200700 1 -- start start 2024-03-29T08:19:34.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:34.369+0000 7fc3df200700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc3d8102e00 0x7fc3d8115c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:34.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:34.369+0000 7fc3df200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc3d81037a0 0x7fc3d8116190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:34.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:34.369+0000 7fc3df200700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3d81167e0 con 0x7fc3d81037a0 2024-03-29T08:19:34.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:34.369+0000 7fc3df200700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3d810fbb0 con 0x7fc3d8102e00 2024-03-29T08:19:34.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:34.369+0000 7fc3d7400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc3d81037a0 0x7fc3d8116190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:34.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:34.369+0000 7fc3d7400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc3d81037a0 0x7fc3d8116190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40302/0 (socket says 172.21.15.37:40302) 2024-03-29T08:19:34.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:34.369+0000 7fc3d7400700 1 -- 172.21.15.37:0/272736701 learned_addr learned my addr 172.21.15.37:0/272736701 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:19:34.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:34.369+0000 7fc3d7e00700 1 --2- 172.21.15.37:0/272736701 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc3d8102e00 0x7fc3d8115c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:34.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:34.369+0000 7fc3d7400700 1 -- 172.21.15.37:0/272736701 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc3d8102e00 msgr2=0x7fc3d8115c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:34.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:34.369+0000 7fc3d7400700 1 --2- 172.21.15.37:0/272736701 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc3d8102e00 0x7fc3d8115c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:34.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:34.370+0000 7fc3d7400700 1 -- 172.21.15.37:0/272736701 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3c8005040 con 0x7fc3d81037a0 2024-03-29T08:19:34.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:34.370+0000 7fc3d7400700 1 --2- 172.21.15.37:0/272736701 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc3d81037a0 0x7fc3d8116190 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fc3c400eb40 tx=0x7fc3c400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:34.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:34.370+0000 7fc3cfe00700 1 -- 172.21.15.37:0/272736701 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc3c4007cc0 con 0x7fc3d81037a0 2024-03-29T08:19:34.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:34.370+0000 7fc3cfe00700 1 -- 172.21.15.37:0/272736701 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fc3c400f040 con 0x7fc3d81037a0 2024-03-29T08:19:34.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:34.370+0000 7fc3cfe00700 1 -- 172.21.15.37:0/272736701 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc3c400c8f0 con 0x7fc3d81037a0 2024-03-29T08:19:34.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:34.370+0000 7fc3df200700 1 -- 172.21.15.37:0/272736701 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3d810fe30 con 0x7fc3d81037a0 2024-03-29T08:19:34.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:34.370+0000 7fc3df200700 1 -- 172.21.15.37:0/272736701 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3d81103f0 con 0x7fc3d81037a0 2024-03-29T08:19:34.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:34.371+0000 7fc3df200700 1 -- 172.21.15.37:0/272736701 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3d80fe550 con 0x7fc3d81037a0 2024-03-29T08:19:34.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:34.373+0000 7fc3cfe00700 1 -- 172.21.15.37:0/272736701 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fc3c40090d0 con 0x7fc3d81037a0 2024-03-29T08:19:34.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:34.374+0000 7fc3cfe00700 1 --2- 172.21.15.37:0/272736701 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fc3bc06cc30 0x7fc3bc06f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:34.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:34.374+0000 7fc3cfe00700 1 -- 172.21.15.37:0/272736701 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fc3c408ba50 con 0x7fc3d81037a0 2024-03-29T08:19:34.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:34.374+0000 7fc3d7e00700 1 --2- 172.21.15.37:0/272736701 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fc3bc06cc30 0x7fc3bc06f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:34.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:34.375+0000 7fc3d7e00700 1 --2- 172.21.15.37:0/272736701 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fc3bc06cc30 0x7fc3bc06f0e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fc3c800b0d0 tx=0x7fc3c8009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:34.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:34.379+0000 7fc3cfe00700 1 -- 172.21.15.37:0/272736701 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fc3c4059550 con 0x7fc3d81037a0 2024-03-29T08:19:34.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:34.611+0000 7fc3df200700 1 -- 172.21.15.37:0/272736701 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc3d8109e60 con 0x7fc3bc06cc30 2024-03-29T08:19:34.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:34.612+0000 7fc3cfe00700 1 -- 172.21.15.37:0/272736701 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fc3c80096f0 con 0x7fc3bc06cc30 2024-03-29T08:19:34.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:34.614+0000 7fc3df200700 1 -- 172.21.15.37:0/272736701 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fc3bc06cc30 msgr2=0x7fc3bc06f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:34.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:34.614+0000 7fc3df200700 1 --2- 172.21.15.37:0/272736701 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fc3bc06cc30 0x7fc3bc06f0e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fc3c800b0d0 tx=0x7fc3c8009e30 comp rx=0 tx=0).stop 2024-03-29T08:19:34.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:34.614+0000 7fc3df200700 1 -- 172.21.15.37:0/272736701 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc3d81037a0 msgr2=0x7fc3d8116190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:34.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:34.614+0000 7fc3df200700 1 --2- 172.21.15.37:0/272736701 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc3d81037a0 0x7fc3d8116190 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fc3c400eb40 tx=0x7fc3c400b040 comp rx=0 tx=0).stop 2024-03-29T08:19:34.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:34.614+0000 7fc3df200700 1 -- 172.21.15.37:0/272736701 shutdown_connections 2024-03-29T08:19:34.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:34.614+0000 7fc3df200700 1 --2- 172.21.15.37:0/272736701 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fc3bc06cc30 0x7fc3bc06f0e0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:34.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:34.614+0000 7fc3df200700 1 --2- 172.21.15.37:0/272736701 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc3d8102e00 0x7fc3d8115c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:34.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:34.614+0000 7fc3df200700 1 --2- 172.21.15.37:0/272736701 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc3d81037a0 0x7fc3d8116190 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:34.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:34.614+0000 7fc3df200700 1 -- 172.21.15.37:0/272736701 >> 172.21.15.37:0/272736701 conn(0x7fc3d80f89b0 msgr2=0x7fc3d80f9900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:19:34.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:34.614+0000 7fc3df200700 1 -- 172.21.15.37:0/272736701 shutdown_connections 2024-03-29T08:19:34.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:34.614+0000 7fc3df200700 1 -- 172.21.15.37:0/272736701 wait complete. 2024-03-29T08:19:34.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:34.753+0000 7f1021a00700 1 -- 172.21.15.37:0/2219146241 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f101c105000 msgr2=0x7f101c1053d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:34.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:34.753+0000 7f1021a00700 1 --2- 172.21.15.37:0/2219146241 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f101c105000 0x7f101c1053d0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f100c004940 tx=0x7f100c009e30 comp rx=0 tx=0).stop 2024-03-29T08:19:34.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:34.753+0000 7f1021a00700 1 -- 172.21.15.37:0/2219146241 shutdown_connections 2024-03-29T08:19:34.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:34.753+0000 7f1021a00700 1 --2- 172.21.15.37:0/2219146241 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f101c1059a0 0x7f101c0ff010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:34.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:34.753+0000 7f1021a00700 1 --2- 172.21.15.37:0/2219146241 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f101c105000 0x7f101c1053d0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:34.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:34.753+0000 7f1021a00700 1 -- 172.21.15.37:0/2219146241 >> 172.21.15.37:0/2219146241 conn(0x7f101c0fabd0 msgr2=0x7f101c0fcfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:19:34.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:34.753+0000 7f1021a00700 1 -- 172.21.15.37:0/2219146241 shutdown_connections 2024-03-29T08:19:34.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:34.753+0000 7f1021a00700 1 -- 172.21.15.37:0/2219146241 wait complete. 2024-03-29T08:19:34.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:34.754+0000 7f1021a00700 1 Processor -- start 2024-03-29T08:19:34.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:34.754+0000 7f1021a00700 1 -- start start 2024-03-29T08:19:34.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:34.754+0000 7f1021a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f101c105000 0x7f101c115c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:34.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:34.754+0000 7f1021a00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f101c1059a0 0x7f101c1161c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:34.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:34.754+0000 7f1021a00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f101c116810 con 0x7f101c105000 2024-03-29T08:19:34.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:34.754+0000 7f1021a00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f101c10fc30 con 0x7f101c1059a0 2024-03-29T08:19:34.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:34.754+0000 7f101aa00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f101c105000 0x7f101c115c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:34.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:34.754+0000 7f101aa00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f101c105000 0x7f101c115c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40322/0 (socket says 172.21.15.37:40322) 2024-03-29T08:19:34.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:34.754+0000 7f101aa00700 1 -- 172.21.15.37:0/1970297385 learned_addr learned my addr 172.21.15.37:0/1970297385 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:19:34.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:34.755+0000 7f101aa00700 1 -- 172.21.15.37:0/1970297385 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f101c1059a0 msgr2=0x7f101c1161c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:34.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:34.755+0000 7f101aa00700 1 --2- 172.21.15.37:0/1970297385 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f101c1059a0 0x7f101c1161c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:34.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:34.755+0000 7f101aa00700 1 -- 172.21.15.37:0/1970297385 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f100c005040 con 0x7f101c105000 2024-03-29T08:19:34.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:34.755+0000 7f101aa00700 1 --2- 172.21.15.37:0/1970297385 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f101c105000 0x7f101c115c80 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f100c008d80 tx=0x7f100c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:34.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:34.755+0000 7f1012a00700 1 -- 172.21.15.37:0/1970297385 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f100c00e030 con 0x7f101c105000 2024-03-29T08:19:34.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:34.755+0000 7f1012a00700 1 -- 172.21.15.37:0/1970297385 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f100c013de0 con 0x7f101c105000 2024-03-29T08:19:34.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:34.755+0000 7f1021a00700 1 -- 172.21.15.37:0/1970297385 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f101c10feb0 con 0x7f101c105000 2024-03-29T08:19:34.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:34.755+0000 7f1012a00700 1 -- 172.21.15.37:0/1970297385 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f100c01cd60 con 0x7f101c105000 2024-03-29T08:19:34.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:34.756+0000 7f1021a00700 1 -- 172.21.15.37:0/1970297385 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f101c110470 con 0x7f101c105000 2024-03-29T08:19:34.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:34.756+0000 7f1021a00700 1 -- 172.21.15.37:0/1970297385 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f101c102310 con 0x7f101c105000 2024-03-29T08:19:34.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:34.758+0000 7f1012a00700 1 -- 172.21.15.37:0/1970297385 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f100c01c590 con 0x7f101c105000 2024-03-29T08:19:34.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:34.758+0000 7f1012a00700 1 --2- 172.21.15.37:0/1970297385 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f100006cc30 0x7f100006f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:34.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:34.759+0000 7f1012a00700 1 -- 172.21.15.37:0/1970297385 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f100c08d9b0 con 0x7f101c105000 2024-03-29T08:19:34.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:34.759+0000 7f101a000700 1 --2- 172.21.15.37:0/1970297385 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f100006cc30 0x7f100006f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:34.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:34.759+0000 7f101a000700 1 --2- 172.21.15.37:0/1970297385 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f100006cc30 0x7f100006f0e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f101c111520 tx=0x7f100800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:34.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:34.763+0000 7f1012a00700 1 -- 172.21.15.37:0/1970297385 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f100c05b400 con 0x7f101c105000 2024-03-29T08:19:34.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:34.985+0000 7f1021a00700 1 -- 172.21.15.37:0/1970297385 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f101c111340 con 0x7f100006cc30 2024-03-29T08:19:34.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:34.995+0000 7f1012a00700 1 -- 172.21.15.37:0/1970297385 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7f101c111340 con 0x7f100006cc30 2024-03-29T08:19:34.995 INFO:teuthology.orchestra.run.smithi037.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-29T08:19:34.995 INFO:teuthology.orchestra.run.smithi037.stdout:alertmanager.smithi037 smithi037 *:9093,9094 running (9m) 103s ago 22m 14.9M - 0.25.0 c8568f914cd2 2f9f9746ed0f 2024-03-29T08:19:34.995 INFO:teuthology.orchestra.run.smithi037.stdout:crash.smithi037 smithi037 running (6m) 103s ago 22m 6647k - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 178d416a3186 2024-03-29T08:19:34.995 INFO:teuthology.orchestra.run.smithi037.stdout:crash.smithi148 smithi148 running (6m) 82s ago 21m 6656k - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 23666989d58f 2024-03-29T08:19:34.995 INFO:teuthology.orchestra.run.smithi037.stdout:grafana.smithi037 smithi037 *:3000 running (9m) 103s ago 22m 71.9M - 9.4.7 954c08fa6188 6b976fe49952 2024-03-29T08:19:34.995 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi037.iifsmh smithi037 running (109s) 103s ago 15m 10.8M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 2d97f562cc5b 2024-03-29T08:19:34.995 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi037.mmwgxe smithi037 running (2m) 103s ago 16m 10.9M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff ec37037f46e8 2024-03-29T08:19:34.996 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi148.qukpjh smithi148 running (15m) 82s ago 15m 15.3M - 17.2.7-789-g4213e17c cf22eb8683be 8e469782a273 2024-03-29T08:19:34.996 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi148.sqteih smithi148 running (87s) 82s ago 15m 10.7M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 44c15c77037a 2024-03-29T08:19:34.996 INFO:teuthology.orchestra.run.smithi037.stdout:mgr.smithi037.wjikii smithi037 *:8443,9283,8765 running (11m) 103s ago 24m 506M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff c5a1417b9417 2024-03-29T08:19:34.996 INFO:teuthology.orchestra.run.smithi037.stdout:mgr.smithi148.onhgvm smithi148 *:8443,9283,8765 running (10m) 82s ago 21m 450M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff fcfdbce062bd 2024-03-29T08:19:34.996 INFO:teuthology.orchestra.run.smithi037.stdout:mon.smithi037 smithi037 running (7m) 103s ago 24m 54.8M 2048M 18.2.2-1015-gd0e392d9 1cc2f45c17ff eea2060783af 2024-03-29T08:19:34.996 INFO:teuthology.orchestra.run.smithi037.stdout:mon.smithi148 smithi148 running (7m) 82s ago 21m 43.7M 2048M 18.2.2-1015-gd0e392d9 1cc2f45c17ff ad4b2c5748be 2024-03-29T08:19:34.996 INFO:teuthology.orchestra.run.smithi037.stdout:node-exporter.smithi037 smithi037 *:9100 running (10m) 103s ago 22m 7625k - 1.5.0 0da6a335fe13 26b8ae9cb878 2024-03-29T08:19:34.996 INFO:teuthology.orchestra.run.smithi037.stdout:node-exporter.smithi148 smithi148 *:9100 running (10m) 82s ago 21m 8166k - 1.5.0 0da6a335fe13 98414e4e64ed 2024-03-29T08:19:34.996 INFO:teuthology.orchestra.run.smithi037.stdout:osd.0 smithi037 running (5m) 103s ago 20m 104M 1619M 18.2.2-1015-gd0e392d9 1cc2f45c17ff c0c584f6245c 2024-03-29T08:19:34.996 INFO:teuthology.orchestra.run.smithi037.stdout:osd.1 smithi037 running (5m) 103s ago 19m 100M 1619M 18.2.2-1015-gd0e392d9 1cc2f45c17ff a4a2ff4cc83d 2024-03-29T08:19:34.996 INFO:teuthology.orchestra.run.smithi037.stdout:osd.2 smithi037 running (4m) 103s ago 19m 82.3M 1619M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 8b1bc71aa79e 2024-03-29T08:19:34.996 INFO:teuthology.orchestra.run.smithi037.stdout:osd.3 smithi148 running (3m) 82s ago 18m 105M 2642M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 1c656ab8ff86 2024-03-29T08:19:34.996 INFO:teuthology.orchestra.run.smithi037.stdout:osd.4 smithi148 running (3m) 82s ago 18m 97.3M 2642M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 966a9de11d22 2024-03-29T08:19:34.996 INFO:teuthology.orchestra.run.smithi037.stdout:osd.5 smithi148 running (2m) 82s ago 17m 85.8M 2642M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 474c4f9212ce 2024-03-29T08:19:34.996 INFO:teuthology.orchestra.run.smithi037.stdout:prometheus.smithi037 smithi037 *:9095 running (9m) 103s ago 21m 54.4M - 2.43.0 a07b618ecd1d 30c0223ec589 2024-03-29T08:19:34.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:34.997+0000 7f1021a00700 1 -- 172.21.15.37:0/1970297385 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f100006cc30 msgr2=0x7f100006f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:34.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:34.997+0000 7f1021a00700 1 --2- 172.21.15.37:0/1970297385 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f100006cc30 0x7f100006f0e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f101c111520 tx=0x7f100800b040 comp rx=0 tx=0).stop 2024-03-29T08:19:34.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:34.997+0000 7f1021a00700 1 -- 172.21.15.37:0/1970297385 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f101c105000 msgr2=0x7f101c115c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:34.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:34.997+0000 7f1021a00700 1 --2- 172.21.15.37:0/1970297385 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f101c105000 0x7f101c115c80 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f100c008d80 tx=0x7f100c009e30 comp rx=0 tx=0).stop 2024-03-29T08:19:34.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:34.997+0000 7f1021a00700 1 -- 172.21.15.37:0/1970297385 shutdown_connections 2024-03-29T08:19:34.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:34.997+0000 7f1021a00700 1 --2- 172.21.15.37:0/1970297385 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f100006cc30 0x7f100006f0e0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:34.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:34.997+0000 7f1021a00700 1 --2- 172.21.15.37:0/1970297385 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f101c105000 0x7f101c115c80 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:34.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:34.997+0000 7f1021a00700 1 --2- 172.21.15.37:0/1970297385 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f101c1059a0 0x7f101c1161c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:34.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:34.997+0000 7f1021a00700 1 -- 172.21.15.37:0/1970297385 >> 172.21.15.37:0/1970297385 conn(0x7f101c0fabd0 msgr2=0x7f101c0fcf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:19:34.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:34.997+0000 7f1021a00700 1 -- 172.21.15.37:0/1970297385 shutdown_connections 2024-03-29T08:19:34.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:34.997+0000 7f1021a00700 1 -- 172.21.15.37:0/1970297385 wait complete. 2024-03-29T08:19:35.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:35.133+0000 7fc1c4800700 1 -- 172.21.15.37:0/142352188 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc1bc0fde90 msgr2=0x7fc1bc0fe260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:35.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:35.133+0000 7fc1c4800700 1 --2- 172.21.15.37:0/142352188 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc1bc0fde90 0x7fc1bc0fe260 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fc1b0004940 tx=0x7fc1b0009e30 comp rx=0 tx=0).stop 2024-03-29T08:19:35.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:35.133+0000 7fc1c4800700 1 -- 172.21.15.37:0/142352188 shutdown_connections 2024-03-29T08:19:35.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:35.133+0000 7fc1c4800700 1 --2- 172.21.15.37:0/142352188 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc1bc0fe830 0x7fc1bc106f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:35.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:35.133+0000 7fc1c4800700 1 --2- 172.21.15.37:0/142352188 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc1bc0fde90 0x7fc1bc0fe260 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:35.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:35.133+0000 7fc1c4800700 1 -- 172.21.15.37:0/142352188 >> 172.21.15.37:0/142352188 conn(0x7fc1bc072bc0 msgr2=0x7fc1bc072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:19:35.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:35.134+0000 7fc1c4800700 1 -- 172.21.15.37:0/142352188 shutdown_connections 2024-03-29T08:19:35.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:35.134+0000 7fc1c4800700 1 -- 172.21.15.37:0/142352188 wait complete. 2024-03-29T08:19:35.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:35.134+0000 7fc1c4800700 1 Processor -- start 2024-03-29T08:19:35.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:35.135+0000 7fc1c4800700 1 -- start start 2024-03-29T08:19:35.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:35.135+0000 7fc1c4800700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc1bc0fde90 0x7fc1bc115d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:35.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:35.135+0000 7fc1c4800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc1bc0fe830 0x7fc1bc116260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:35.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:35.135+0000 7fc1c4800700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1bc1168b0 con 0x7fc1bc0fe830 2024-03-29T08:19:35.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:35.135+0000 7fc1c4800700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1bc10fc80 con 0x7fc1bc0fde90 2024-03-29T08:19:35.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:35.135+0000 7fc1c1400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc1bc0fe830 0x7fc1bc116260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:35.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:35.135+0000 7fc1c1400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc1bc0fe830 0x7fc1bc116260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40344/0 (socket says 172.21.15.37:40344) 2024-03-29T08:19:35.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:35.135+0000 7fc1c1400700 1 -- 172.21.15.37:0/645175888 learned_addr learned my addr 172.21.15.37:0/645175888 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:19:35.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:35.135+0000 7fc1c1e00700 1 --2- 172.21.15.37:0/645175888 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc1bc0fde90 0x7fc1bc115d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:35.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:35.136+0000 7fc1c1400700 1 -- 172.21.15.37:0/645175888 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc1bc0fde90 msgr2=0x7fc1bc115d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:35.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:35.136+0000 7fc1c1400700 1 --2- 172.21.15.37:0/645175888 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc1bc0fde90 0x7fc1bc115d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:35.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:35.136+0000 7fc1c1400700 1 -- 172.21.15.37:0/645175888 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1b0005040 con 0x7fc1bc0fe830 2024-03-29T08:19:35.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:35.136+0000 7fc1c1400700 1 --2- 172.21.15.37:0/645175888 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc1bc0fe830 0x7fc1bc116260 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fc1ac002a50 tx=0x7fc1ac0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:35.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:35.136+0000 7fc1b6000700 1 -- 172.21.15.37:0/645175888 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc1ac007cc0 con 0x7fc1bc0fe830 2024-03-29T08:19:35.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:35.136+0000 7fc1b6000700 1 -- 172.21.15.37:0/645175888 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fc1ac00f040 con 0x7fc1bc0fe830 2024-03-29T08:19:35.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:35.136+0000 7fc1b6000700 1 -- 172.21.15.37:0/645175888 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc1ac009540 con 0x7fc1bc0fe830 2024-03-29T08:19:35.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:35.136+0000 7fc1c4800700 1 -- 172.21.15.37:0/645175888 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1bc10ff60 con 0x7fc1bc0fe830 2024-03-29T08:19:35.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:35.136+0000 7fc1c4800700 1 -- 172.21.15.37:0/645175888 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1bc1104a0 con 0x7fc1bc0fe830 2024-03-29T08:19:35.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:35.137+0000 7fc1c4800700 1 -- 172.21.15.37:0/645175888 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1bc106a20 con 0x7fc1bc0fe830 2024-03-29T08:19:35.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:35.138+0000 7fc1b6000700 1 -- 172.21.15.37:0/645175888 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fc1ac0096a0 con 0x7fc1bc0fe830 2024-03-29T08:19:35.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:35.139+0000 7fc1b6000700 1 --2- 172.21.15.37:0/645175888 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fc1a406cc30 0x7fc1a406f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:35.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:35.139+0000 7fc1b6000700 1 -- 172.21.15.37:0/645175888 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fc1ac08bde0 con 0x7fc1bc0fe830 2024-03-29T08:19:35.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:35.139+0000 7fc1c1e00700 1 --2- 172.21.15.37:0/645175888 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fc1a406cc30 0x7fc1a406f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:35.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:35.140+0000 7fc1c1e00700 1 --2- 172.21.15.37:0/645175888 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fc1a406cc30 0x7fc1a406f0e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fc1b000b3b0 tx=0x7fc1b0009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:35.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:35.144+0000 7fc1b6000700 1 -- 172.21.15.37:0/645175888 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fc1ac059830 con 0x7fc1bc0fe830 2024-03-29T08:19:35.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:19:35 smithi037 ceph-mon[142325]: from='client.34532 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:19:35.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:19:35 smithi037 ceph-mon[142325]: from='client.34536 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:19:35.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:35.450+0000 7fc1c4800700 1 -- 172.21.15.37:0/645175888 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fc1bc111370 con 0x7fc1bc0fe830 2024-03-29T08:19:35.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:35.450+0000 7fc1b6000700 1 -- 172.21.15.37:0/645175888 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+847 (secure 0 0 0) 0x7fc1ac0593c0 con 0x7fc1bc0fe830 2024-03-29T08:19:35.451 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-29T08:19:35.451 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": { 2024-03-29T08:19:35.451 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 2 2024-03-29T08:19:35.451 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:19:35.451 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": { 2024-03-29T08:19:35.451 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 2 2024-03-29T08:19:35.451 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:19:35.451 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": { 2024-03-29T08:19:35.451 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 6 2024-03-29T08:19:35.451 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:19:35.451 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": { 2024-03-29T08:19:35.451 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 17.2.7-789-g4213e17c (4213e17c4de67016ed14f55d22503f4923b3a901) quincy (stable)": 1, 2024-03-29T08:19:35.451 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 3 2024-03-29T08:19:35.453 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:19:35.453 INFO:teuthology.orchestra.run.smithi037.stdout: "overall": { 2024-03-29T08:19:35.453 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 17.2.7-789-g4213e17c (4213e17c4de67016ed14f55d22503f4923b3a901) quincy (stable)": 1, 2024-03-29T08:19:35.453 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 13 2024-03-29T08:19:35.453 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-29T08:19:35.453 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-29T08:19:35.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:35.453+0000 7fc1c4800700 1 -- 172.21.15.37:0/645175888 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fc1a406cc30 msgr2=0x7fc1a406f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:35.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:35.453+0000 7fc1c4800700 1 --2- 172.21.15.37:0/645175888 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fc1a406cc30 0x7fc1a406f0e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fc1b000b3b0 tx=0x7fc1b0009e30 comp rx=0 tx=0).stop 2024-03-29T08:19:35.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:35.453+0000 7fc1c4800700 1 -- 172.21.15.37:0/645175888 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc1bc0fe830 msgr2=0x7fc1bc116260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:35.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:35.453+0000 7fc1c4800700 1 --2- 172.21.15.37:0/645175888 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc1bc0fe830 0x7fc1bc116260 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fc1ac002a50 tx=0x7fc1ac0046b0 comp rx=0 tx=0).stop 2024-03-29T08:19:35.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:35.453+0000 7fc1c4800700 1 -- 172.21.15.37:0/645175888 shutdown_connections 2024-03-29T08:19:35.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:35.453+0000 7fc1c4800700 1 --2- 172.21.15.37:0/645175888 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fc1a406cc30 0x7fc1a406f0e0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:35.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:35.453+0000 7fc1c4800700 1 --2- 172.21.15.37:0/645175888 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc1bc0fde90 0x7fc1bc115d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:35.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:35.453+0000 7fc1c4800700 1 --2- 172.21.15.37:0/645175888 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc1bc0fe830 0x7fc1bc116260 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:35.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:35.453+0000 7fc1c4800700 1 -- 172.21.15.37:0/645175888 >> 172.21.15.37:0/645175888 conn(0x7fc1bc072bc0 msgr2=0x7fc1bc0fb8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:19:35.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:35.453+0000 7fc1c4800700 1 -- 172.21.15.37:0/645175888 shutdown_connections 2024-03-29T08:19:35.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:35.453+0000 7fc1c4800700 1 -- 172.21.15.37:0/645175888 wait complete. 2024-03-29T08:19:35.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:19:35 smithi148 ceph-mon[106692]: from='client.34532 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:19:35.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:19:35 smithi148 ceph-mon[106692]: from='client.34536 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:19:35.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:35.590+0000 7fb8e8600700 1 -- 172.21.15.37:0/2556620184 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb8e00fde90 msgr2=0x7fb8e00fe260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:35.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:35.590+0000 7fb8e8600700 1 --2- 172.21.15.37:0/2556620184 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb8e00fde90 0x7fb8e00fe260 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fb8d4004940 tx=0x7fb8d4009e30 comp rx=0 tx=0).stop 2024-03-29T08:19:35.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:35.590+0000 7fb8e8600700 1 -- 172.21.15.37:0/2556620184 shutdown_connections 2024-03-29T08:19:35.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:35.590+0000 7fb8e8600700 1 --2- 172.21.15.37:0/2556620184 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb8e00fe830 0x7fb8e0106f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:35.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:35.590+0000 7fb8e8600700 1 --2- 172.21.15.37:0/2556620184 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb8e00fde90 0x7fb8e00fe260 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:35.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:35.590+0000 7fb8e8600700 1 -- 172.21.15.37:0/2556620184 >> 172.21.15.37:0/2556620184 conn(0x7fb8e0072bc0 msgr2=0x7fb8e0072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:19:35.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:35.590+0000 7fb8e8600700 1 -- 172.21.15.37:0/2556620184 shutdown_connections 2024-03-29T08:19:35.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:35.590+0000 7fb8e8600700 1 -- 172.21.15.37:0/2556620184 wait complete. 2024-03-29T08:19:35.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:35.591+0000 7fb8e8600700 1 Processor -- start 2024-03-29T08:19:35.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:35.591+0000 7fb8e8600700 1 -- start start 2024-03-29T08:19:35.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:35.592+0000 7fb8e8600700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb8e00fde90 0x7fb8e0113af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:35.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:35.592+0000 7fb8e8600700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb8e00fe830 0x7fb8e0114030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:35.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:35.592+0000 7fb8e8600700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8e0114680 con 0x7fb8e00fe830 2024-03-29T08:19:35.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:35.592+0000 7fb8e8600700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8e010da70 con 0x7fb8e00fde90 2024-03-29T08:19:35.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:35.592+0000 7fb8e5200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb8e00fe830 0x7fb8e0114030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:35.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:35.592+0000 7fb8e5200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb8e00fe830 0x7fb8e0114030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40366/0 (socket says 172.21.15.37:40366) 2024-03-29T08:19:35.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:35.592+0000 7fb8e5200700 1 -- 172.21.15.37:0/3328570855 learned_addr learned my addr 172.21.15.37:0/3328570855 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:19:35.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:35.592+0000 7fb8e5c00700 1 --2- 172.21.15.37:0/3328570855 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb8e00fde90 0x7fb8e0113af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:35.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:35.592+0000 7fb8e5200700 1 -- 172.21.15.37:0/3328570855 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb8e00fde90 msgr2=0x7fb8e0113af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:35.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:35.592+0000 7fb8e5200700 1 --2- 172.21.15.37:0/3328570855 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb8e00fde90 0x7fb8e0113af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:35.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:35.592+0000 7fb8e5200700 1 -- 172.21.15.37:0/3328570855 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8d4005040 con 0x7fb8e00fe830 2024-03-29T08:19:35.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:35.593+0000 7fb8e5200700 1 --2- 172.21.15.37:0/3328570855 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb8e00fe830 0x7fb8e0114030 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fb8cc002a50 tx=0x7fb8cc0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:35.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:35.593+0000 7fb8dd600700 1 -- 172.21.15.37:0/3328570855 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb8cc007cc0 con 0x7fb8e00fe830 2024-03-29T08:19:35.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:35.593+0000 7fb8dd600700 1 -- 172.21.15.37:0/3328570855 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fb8cc00f040 con 0x7fb8e00fe830 2024-03-29T08:19:35.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:35.593+0000 7fb8dd600700 1 -- 172.21.15.37:0/3328570855 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb8cc00d9c0 con 0x7fb8e00fe830 2024-03-29T08:19:35.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:35.593+0000 7fb8e8600700 1 -- 172.21.15.37:0/3328570855 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8e010dd50 con 0x7fb8e00fe830 2024-03-29T08:19:35.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:35.593+0000 7fb8e8600700 1 -- 172.21.15.37:0/3328570855 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8e010e290 con 0x7fb8e00fe830 2024-03-29T08:19:35.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:35.594+0000 7fb8e8600700 1 -- 172.21.15.37:0/3328570855 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8e01019b0 con 0x7fb8e00fe830 2024-03-29T08:19:35.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:35.595+0000 7fb8dd600700 1 -- 172.21.15.37:0/3328570855 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fb8cc00db20 con 0x7fb8e00fe830 2024-03-29T08:19:35.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:35.597+0000 7fb8dd600700 1 --2- 172.21.15.37:0/3328570855 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fb8c806cc30 0x7fb8c806f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:35.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:35.597+0000 7fb8dd600700 1 -- 172.21.15.37:0/3328570855 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fb8cc08bca0 con 0x7fb8e00fe830 2024-03-29T08:19:35.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:35.598+0000 7fb8e5c00700 1 --2- 172.21.15.37:0/3328570855 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fb8c806cc30 0x7fb8c806f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:35.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:35.598+0000 7fb8e5c00700 1 --2- 172.21.15.37:0/3328570855 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fb8c806cc30 0x7fb8c806f0e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fb8d400b3b0 tx=0x7fb8d4009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:35.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:35.601+0000 7fb8dd600700 1 -- 172.21.15.37:0/3328570855 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fb8cc059770 con 0x7fb8e00fe830 2024-03-29T08:19:35.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:35.860+0000 7fb8e8600700 1 -- 172.21.15.37:0/3328570855 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fb8e010f140 con 0x7fb8e00fe830 2024-03-29T08:19:35.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:35.860+0000 7fb8dd600700 1 -- 172.21.15.37:0/3328570855 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 28 v28) v1 ==== 76+0+1841 (secure 0 0 0) 0x7fb8cc059300 con 0x7fb8e00fe830 2024-03-29T08:19:35.861 INFO:teuthology.orchestra.run.smithi037.stdout:e28 2024-03-29T08:19:35.861 INFO:teuthology.orchestra.run.smithi037.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-29T08:19:35.861 INFO:teuthology.orchestra.run.smithi037.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T08:19:35.861 INFO:teuthology.orchestra.run.smithi037.stdout:legacy client fscid: 1 2024-03-29T08:19:35.861 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:19:35.861 INFO:teuthology.orchestra.run.smithi037.stdout:Filesystem 'cephfs' (1) 2024-03-29T08:19:35.861 INFO:teuthology.orchestra.run.smithi037.stdout:fs_name cephfs 2024-03-29T08:19:35.861 INFO:teuthology.orchestra.run.smithi037.stdout:epoch 21 2024-03-29T08:19:35.861 INFO:teuthology.orchestra.run.smithi037.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-03-29T08:19:35.861 INFO:teuthology.orchestra.run.smithi037.stdout:created 2024-03-29T08:03:29.241269+0000 2024-03-29T08:19:35.861 INFO:teuthology.orchestra.run.smithi037.stdout:modified 2024-03-29T08:17:16.237238+0000 2024-03-29T08:19:35.861 INFO:teuthology.orchestra.run.smithi037.stdout:tableserver 0 2024-03-29T08:19:35.862 INFO:teuthology.orchestra.run.smithi037.stdout:root 0 2024-03-29T08:19:35.862 INFO:teuthology.orchestra.run.smithi037.stdout:session_timeout 60 2024-03-29T08:19:35.862 INFO:teuthology.orchestra.run.smithi037.stdout:session_autoclose 300 2024-03-29T08:19:35.862 INFO:teuthology.orchestra.run.smithi037.stdout:max_file_size 1099511627776 2024-03-29T08:19:35.862 INFO:teuthology.orchestra.run.smithi037.stdout:max_xattr_size 65536 2024-03-29T08:19:35.862 INFO:teuthology.orchestra.run.smithi037.stdout:required_client_features {} 2024-03-29T08:19:35.862 INFO:teuthology.orchestra.run.smithi037.stdout:last_failure 0 2024-03-29T08:19:35.862 INFO:teuthology.orchestra.run.smithi037.stdout:last_failure_osd_epoch 82 2024-03-29T08:19:35.862 INFO:teuthology.orchestra.run.smithi037.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T08:19:35.862 INFO:teuthology.orchestra.run.smithi037.stdout:max_mds 2 2024-03-29T08:19:35.862 INFO:teuthology.orchestra.run.smithi037.stdout:in 0,1 2024-03-29T08:19:35.862 INFO:teuthology.orchestra.run.smithi037.stdout:up {} 2024-03-29T08:19:35.862 INFO:teuthology.orchestra.run.smithi037.stdout:failed 0,1 2024-03-29T08:19:35.862 INFO:teuthology.orchestra.run.smithi037.stdout:damaged 2024-03-29T08:19:35.863 INFO:teuthology.orchestra.run.smithi037.stdout:stopped 2024-03-29T08:19:35.863 INFO:teuthology.orchestra.run.smithi037.stdout:data_pools [3] 2024-03-29T08:19:35.863 INFO:teuthology.orchestra.run.smithi037.stdout:metadata_pool 2 2024-03-29T08:19:35.863 INFO:teuthology.orchestra.run.smithi037.stdout:inline_data disabled 2024-03-29T08:19:35.863 INFO:teuthology.orchestra.run.smithi037.stdout:balancer 2024-03-29T08:19:35.863 INFO:teuthology.orchestra.run.smithi037.stdout:bal_rank_mask -1 2024-03-29T08:19:35.863 INFO:teuthology.orchestra.run.smithi037.stdout:standby_count_wanted 1 2024-03-29T08:19:35.863 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:19:35.863 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:19:35.863 INFO:teuthology.orchestra.run.smithi037.stdout:Standby daemons: 2024-03-29T08:19:35.863 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:19:35.863 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi148.qukpjh{-1:24319} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.148:6826/2641327912,v1:172.21.15.148:6827/2641327912] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:19:35.863 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi037.mmwgxe{-1:34440} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.37:6828/1248142005,v1:172.21.15.37:6829/1248142005] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:19:35.863 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi037.iifsmh{-1:34444} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.37:6826/2060882959,v1:172.21.15.37:6827/2060882959] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:19:35.863 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi148.sqteih{-1:44395} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.148:6824/4078652242,v1:172.21.15.148:6825/4078652242] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:19:35.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:35.862+0000 7fb8e8600700 1 -- 172.21.15.37:0/3328570855 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fb8c806cc30 msgr2=0x7fb8c806f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:35.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:35.862+0000 7fb8e8600700 1 --2- 172.21.15.37:0/3328570855 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fb8c806cc30 0x7fb8c806f0e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fb8d400b3b0 tx=0x7fb8d4009e30 comp rx=0 tx=0).stop 2024-03-29T08:19:35.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:35.862+0000 7fb8e8600700 1 -- 172.21.15.37:0/3328570855 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb8e00fe830 msgr2=0x7fb8e0114030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:35.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:35.862+0000 7fb8e8600700 1 --2- 172.21.15.37:0/3328570855 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb8e00fe830 0x7fb8e0114030 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fb8cc002a50 tx=0x7fb8cc0046b0 comp rx=0 tx=0).stop 2024-03-29T08:19:35.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:35.863+0000 7fb8e8600700 1 -- 172.21.15.37:0/3328570855 shutdown_connections 2024-03-29T08:19:35.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:35.863+0000 7fb8e8600700 1 --2- 172.21.15.37:0/3328570855 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fb8c806cc30 0x7fb8c806f0e0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:35.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:35.863+0000 7fb8e8600700 1 --2- 172.21.15.37:0/3328570855 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb8e00fde90 0x7fb8e0113af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:35.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:35.863+0000 7fb8e8600700 1 --2- 172.21.15.37:0/3328570855 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb8e00fe830 0x7fb8e0114030 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:35.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:35.863+0000 7fb8e8600700 1 -- 172.21.15.37:0/3328570855 >> 172.21.15.37:0/3328570855 conn(0x7fb8e0072bc0 msgr2=0x7fb8e00ff3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:19:35.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:35.863+0000 7fb8e8600700 1 -- 172.21.15.37:0/3328570855 shutdown_connections 2024-03-29T08:19:35.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:35.863+0000 7fb8e8600700 1 -- 172.21.15.37:0/3328570855 wait complete. 2024-03-29T08:19:35.865 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 28 2024-03-29T08:19:36.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:36.000+0000 7fe5d0e00700 1 -- 172.21.15.37:0/314382021 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe5cc102e00 msgr2=0x7fe5cc1031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:36.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:36.000+0000 7fe5d0e00700 1 --2- 172.21.15.37:0/314382021 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe5cc102e00 0x7fe5cc1031d0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fe5bc004940 tx=0x7fe5bc009e30 comp rx=0 tx=0).stop 2024-03-29T08:19:36.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:36.000+0000 7fe5d0e00700 1 -- 172.21.15.37:0/314382021 shutdown_connections 2024-03-29T08:19:36.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:36.000+0000 7fe5d0e00700 1 --2- 172.21.15.37:0/314382021 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe5cc1037a0 0x7fe5cc0fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:36.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:36.000+0000 7fe5d0e00700 1 --2- 172.21.15.37:0/314382021 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe5cc102e00 0x7fe5cc1031d0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:36.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:36.000+0000 7fe5d0e00700 1 -- 172.21.15.37:0/314382021 >> 172.21.15.37:0/314382021 conn(0x7fe5cc0f89b0 msgr2=0x7fe5cc0fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:19:36.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:36.000+0000 7fe5d0e00700 1 -- 172.21.15.37:0/314382021 shutdown_connections 2024-03-29T08:19:36.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:36.000+0000 7fe5d0e00700 1 -- 172.21.15.37:0/314382021 wait complete. 2024-03-29T08:19:36.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:36.001+0000 7fe5d0e00700 1 Processor -- start 2024-03-29T08:19:36.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:36.001+0000 7fe5d0e00700 1 -- start start 2024-03-29T08:19:36.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:36.002+0000 7fe5d0e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe5cc102e00 0x7fe5cc115cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:36.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:36.002+0000 7fe5d0e00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe5cc1037a0 0x7fe5cc116210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:36.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:36.002+0000 7fe5d0e00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5cc116860 con 0x7fe5cc102e00 2024-03-29T08:19:36.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:36.002+0000 7fe5d0e00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5cc10fc30 con 0x7fe5cc1037a0 2024-03-29T08:19:36.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:36.002+0000 7fe5c9e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe5cc102e00 0x7fe5cc115cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:36.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:36.002+0000 7fe5c9e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe5cc102e00 0x7fe5cc115cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40380/0 (socket says 172.21.15.37:40380) 2024-03-29T08:19:36.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:36.002+0000 7fe5c9e00700 1 -- 172.21.15.37:0/6903606 learned_addr learned my addr 172.21.15.37:0/6903606 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:19:36.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:36.002+0000 7fe5c9400700 1 --2- 172.21.15.37:0/6903606 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe5cc1037a0 0x7fe5cc116210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:36.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:36.002+0000 7fe5c9e00700 1 -- 172.21.15.37:0/6903606 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe5cc1037a0 msgr2=0x7fe5cc116210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:36.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:36.002+0000 7fe5c9e00700 1 --2- 172.21.15.37:0/6903606 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe5cc1037a0 0x7fe5cc116210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:36.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:36.002+0000 7fe5c9e00700 1 -- 172.21.15.37:0/6903606 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5bc005040 con 0x7fe5cc102e00 2024-03-29T08:19:36.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:36.003+0000 7fe5c9e00700 1 --2- 172.21.15.37:0/6903606 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe5cc102e00 0x7fe5cc115cd0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fe5bc009390 tx=0x7fe5bc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:36.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:36.003+0000 7fe5c2000700 1 -- 172.21.15.37:0/6903606 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe5bc00e040 con 0x7fe5cc102e00 2024-03-29T08:19:36.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:36.003+0000 7fe5c2000700 1 -- 172.21.15.37:0/6903606 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fe5bc013420 con 0x7fe5cc102e00 2024-03-29T08:19:36.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:36.003+0000 7fe5d0e00700 1 -- 172.21.15.37:0/6903606 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5cc10feb0 con 0x7fe5cc102e00 2024-03-29T08:19:36.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:36.003+0000 7fe5c2000700 1 -- 172.21.15.37:0/6903606 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe5bc01c630 con 0x7fe5cc102e00 2024-03-29T08:19:36.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:36.003+0000 7fe5d0e00700 1 -- 172.21.15.37:0/6903606 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5cc110470 con 0x7fe5cc102e00 2024-03-29T08:19:36.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:36.004+0000 7fe5d0e00700 1 -- 172.21.15.37:0/6903606 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5cc0fe550 con 0x7fe5cc102e00 2024-03-29T08:19:36.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:36.005+0000 7fe5c2000700 1 -- 172.21.15.37:0/6903606 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fe5bc0139e0 con 0x7fe5cc102e00 2024-03-29T08:19:36.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:36.006+0000 7fe5c2000700 1 --2- 172.21.15.37:0/6903606 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fe5b006cc30 0x7fe5b006f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:36.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:36.006+0000 7fe5c2000700 1 -- 172.21.15.37:0/6903606 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fe5bc08d340 con 0x7fe5cc102e00 2024-03-29T08:19:36.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:36.006+0000 7fe5c9400700 1 --2- 172.21.15.37:0/6903606 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fe5b006cc30 0x7fe5b006f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:36.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:36.007+0000 7fe5c9400700 1 --2- 172.21.15.37:0/6903606 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fe5b006cc30 0x7fe5b006f0e0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fe5cc111570 tx=0x7fe5b800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:36.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:36.011+0000 7fe5c2000700 1 -- 172.21.15.37:0/6903606 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fe5bc05ae10 con 0x7fe5cc102e00 2024-03-29T08:19:36.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:36.245+0000 7fe5d0e00700 1 -- 172.21.15.37:0/6903606 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe5cc04bf40 con 0x7fe5b006cc30 2024-03-29T08:19:36.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:36.246+0000 7fe5c2000700 1 -- 172.21.15.37:0/6903606 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fe5cc04bf40 con 0x7fe5b006cc30 2024-03-29T08:19:36.246 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-29T08:19:36.246 INFO:teuthology.orchestra.run.smithi037.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59", 2024-03-29T08:19:36.246 INFO:teuthology.orchestra.run.smithi037.stdout: "in_progress": true, 2024-03-29T08:19:36.246 INFO:teuthology.orchestra.run.smithi037.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-29T08:19:36.247 INFO:teuthology.orchestra.run.smithi037.stdout: "services_complete": [ 2024-03-29T08:19:36.247 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr", 2024-03-29T08:19:36.247 INFO:teuthology.orchestra.run.smithi037.stdout: "mon", 2024-03-29T08:19:36.247 INFO:teuthology.orchestra.run.smithi037.stdout: "crash", 2024-03-29T08:19:36.247 INFO:teuthology.orchestra.run.smithi037.stdout: "osd" 2024-03-29T08:19:36.247 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-29T08:19:36.247 INFO:teuthology.orchestra.run.smithi037.stdout: "progress": "15/21 daemons upgraded", 2024-03-29T08:19:36.247 INFO:teuthology.orchestra.run.smithi037.stdout: "message": "Currently upgrading mds daemons", 2024-03-29T08:19:36.247 INFO:teuthology.orchestra.run.smithi037.stdout: "is_paused": false 2024-03-29T08:19:36.247 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-29T08:19:36.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:36.248+0000 7fe5d0e00700 1 -- 172.21.15.37:0/6903606 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fe5b006cc30 msgr2=0x7fe5b006f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:36.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:36.248+0000 7fe5d0e00700 1 --2- 172.21.15.37:0/6903606 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fe5b006cc30 0x7fe5b006f0e0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fe5cc111570 tx=0x7fe5b800b040 comp rx=0 tx=0).stop 2024-03-29T08:19:36.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:36.248+0000 7fe5d0e00700 1 -- 172.21.15.37:0/6903606 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe5cc102e00 msgr2=0x7fe5cc115cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:36.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:36.248+0000 7fe5d0e00700 1 --2- 172.21.15.37:0/6903606 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe5cc102e00 0x7fe5cc115cd0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fe5bc009390 tx=0x7fe5bc009e30 comp rx=0 tx=0).stop 2024-03-29T08:19:36.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:36.249+0000 7fe5d0e00700 1 -- 172.21.15.37:0/6903606 shutdown_connections 2024-03-29T08:19:36.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:36.249+0000 7fe5d0e00700 1 --2- 172.21.15.37:0/6903606 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fe5b006cc30 0x7fe5b006f0e0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:36.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:36.249+0000 7fe5d0e00700 1 --2- 172.21.15.37:0/6903606 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe5cc102e00 0x7fe5cc115cd0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:36.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:36.249+0000 7fe5d0e00700 1 --2- 172.21.15.37:0/6903606 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe5cc1037a0 0x7fe5cc116210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:36.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:36.249+0000 7fe5d0e00700 1 -- 172.21.15.37:0/6903606 >> 172.21.15.37:0/6903606 conn(0x7fe5cc0f89b0 msgr2=0x7fe5cc0f9980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:19:36.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:36.249+0000 7fe5d0e00700 1 -- 172.21.15.37:0/6903606 shutdown_connections 2024-03-29T08:19:36.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:36.249+0000 7fe5d0e00700 1 -- 172.21.15.37:0/6903606 wait complete. 2024-03-29T08:19:36.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:36.385+0000 7fa97d200700 1 -- 172.21.15.37:0/796197070 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa978071f90 msgr2=0x7fa978070360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:36.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:36.385+0000 7fa97d200700 1 --2- 172.21.15.37:0/796197070 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa978071f90 0x7fa978070360 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fa968004970 tx=0x7fa968009e30 comp rx=0 tx=0).stop 2024-03-29T08:19:36.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:36.386+0000 7fa97d200700 1 -- 172.21.15.37:0/796197070 shutdown_connections 2024-03-29T08:19:36.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:36.386+0000 7fa97d200700 1 --2- 172.21.15.37:0/796197070 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa9780708a0 0x7fa978070d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:36.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:36.386+0000 7fa97d200700 1 --2- 172.21.15.37:0/796197070 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa978071f90 0x7fa978070360 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:36.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:36.386+0000 7fa97d200700 1 -- 172.21.15.37:0/796197070 >> 172.21.15.37:0/796197070 conn(0x7fa9780f89b0 msgr2=0x7fa9780fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:19:36.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:36.386+0000 7fa97d200700 1 -- 172.21.15.37:0/796197070 shutdown_connections 2024-03-29T08:19:36.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:36.386+0000 7fa97d200700 1 -- 172.21.15.37:0/796197070 wait complete. 2024-03-29T08:19:36.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:36.387+0000 7fa97d200700 1 Processor -- start 2024-03-29T08:19:36.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:36.387+0000 7fa97d200700 1 -- start start 2024-03-29T08:19:36.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:36.387+0000 7fa97d200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa9780708a0 0x7fa97811a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:36.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:36.387+0000 7fa97d200700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa978071f90 0x7fa97811a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:36.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:36.387+0000 7fa97d200700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa97811aba0 con 0x7fa9780708a0 2024-03-29T08:19:36.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:36.387+0000 7fa97d200700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa978114020 con 0x7fa978071f90 2024-03-29T08:19:36.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:36.387+0000 7fa975e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa9780708a0 0x7fa97811a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:36.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:36.387+0000 7fa975e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa9780708a0 0x7fa97811a010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40392/0 (socket says 172.21.15.37:40392) 2024-03-29T08:19:36.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:36.387+0000 7fa975e00700 1 -- 172.21.15.37:0/3881884924 learned_addr learned my addr 172.21.15.37:0/3881884924 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:19:36.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:36.388+0000 7fa975400700 1 --2- 172.21.15.37:0/3881884924 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa978071f90 0x7fa97811a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:36.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:36.388+0000 7fa975e00700 1 -- 172.21.15.37:0/3881884924 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa978071f90 msgr2=0x7fa97811a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:36.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:36.388+0000 7fa975e00700 1 --2- 172.21.15.37:0/3881884924 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa978071f90 0x7fa97811a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:36.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:36.388+0000 7fa975e00700 1 -- 172.21.15.37:0/3881884924 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa968005040 con 0x7fa9780708a0 2024-03-29T08:19:36.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:36.388+0000 7fa975e00700 1 --2- 172.21.15.37:0/3881884924 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa9780708a0 0x7fa97811a010 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fa968005010 tx=0x7fa968009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:36.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:36.388+0000 7fa96e000700 1 -- 172.21.15.37:0/3881884924 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa96800e040 con 0x7fa9780708a0 2024-03-29T08:19:36.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:36.388+0000 7fa96e000700 1 -- 172.21.15.37:0/3881884924 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fa968009930 con 0x7fa9780708a0 2024-03-29T08:19:36.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:36.388+0000 7fa96e000700 1 -- 172.21.15.37:0/3881884924 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa96801c3f0 con 0x7fa9780708a0 2024-03-29T08:19:36.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:36.388+0000 7fa97d200700 1 -- 172.21.15.37:0/3881884924 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9781142a0 con 0x7fa9780708a0 2024-03-29T08:19:36.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:36.389+0000 7fa97d200700 1 -- 172.21.15.37:0/3881884924 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9781147e0 con 0x7fa9780708a0 2024-03-29T08:19:36.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:36.390+0000 7fa97d200700 1 -- 172.21.15.37:0/3881884924 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa978071aa0 con 0x7fa9780708a0 2024-03-29T08:19:36.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:36.390+0000 7fa96e000700 1 -- 172.21.15.37:0/3881884924 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fa968013420 con 0x7fa9780708a0 2024-03-29T08:19:36.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:36.391+0000 7fa96e000700 1 --2- 172.21.15.37:0/3881884924 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fa95c06cc30 0x7fa95c06f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:36.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:36.392+0000 7fa96e000700 1 -- 172.21.15.37:0/3881884924 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fa96808d1b0 con 0x7fa9780708a0 2024-03-29T08:19:36.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:36.392+0000 7fa975400700 1 --2- 172.21.15.37:0/3881884924 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fa95c06cc30 0x7fa95c06f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:36.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:36.392+0000 7fa975400700 1 --2- 172.21.15.37:0/3881884924 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fa95c06cc30 0x7fa95c06f0e0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fa964004600 tx=0x7fa96400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:36.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:36.396+0000 7fa96e000700 1 -- 172.21.15.37:0/3881884924 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fa9680572b0 con 0x7fa9780708a0 2024-03-29T08:19:36.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:19:36 smithi037 ceph-mon[142325]: from='client.34540 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:19:36.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:19:36 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/645175888' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:19:36.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:19:36 smithi037 ceph-mon[142325]: pgmap v261: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:19:36.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:19:36 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/3328570855' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T08:19:36.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:19:36 smithi148 ceph-mon[106692]: from='client.34540 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:19:36.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:19:36 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/645175888' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:19:36.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:19:36 smithi148 ceph-mon[106692]: pgmap v261: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:19:36.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:19:36 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/3328570855' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T08:19:36.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:36.696+0000 7fa97d200700 1 -- 172.21.15.37:0/3881884924 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fa97805f5f0 con 0x7fa9780708a0 2024-03-29T08:19:36.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:36.697+0000 7fa96e000700 1 -- 172.21.15.37:0/3881884924 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7fa96805a8d0 con 0x7fa9780708a0 2024-03-29T08:19:36.697 INFO:teuthology.orchestra.run.smithi037.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-03-29T08:19:36.697 INFO:teuthology.orchestra.run.smithi037.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-03-29T08:19:36.697 INFO:teuthology.orchestra.run.smithi037.stdout: fs cephfs is degraded 2024-03-29T08:19:36.697 INFO:teuthology.orchestra.run.smithi037.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-03-29T08:19:36.697 INFO:teuthology.orchestra.run.smithi037.stdout: fs cephfs is offline because no MDS is active for it. 2024-03-29T08:19:36.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:36.699+0000 7fa97d200700 1 -- 172.21.15.37:0/3881884924 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fa95c06cc30 msgr2=0x7fa95c06f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:36.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:36.699+0000 7fa97d200700 1 --2- 172.21.15.37:0/3881884924 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fa95c06cc30 0x7fa95c06f0e0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fa964004600 tx=0x7fa96400b040 comp rx=0 tx=0).stop 2024-03-29T08:19:36.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:36.699+0000 7fa97d200700 1 -- 172.21.15.37:0/3881884924 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa9780708a0 msgr2=0x7fa97811a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:36.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:36.699+0000 7fa97d200700 1 --2- 172.21.15.37:0/3881884924 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa9780708a0 0x7fa97811a010 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fa968005010 tx=0x7fa968009e30 comp rx=0 tx=0).stop 2024-03-29T08:19:36.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:36.699+0000 7fa97d200700 1 -- 172.21.15.37:0/3881884924 shutdown_connections 2024-03-29T08:19:36.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:36.699+0000 7fa97d200700 1 --2- 172.21.15.37:0/3881884924 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fa95c06cc30 0x7fa95c06f0e0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:36.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:36.699+0000 7fa97d200700 1 --2- 172.21.15.37:0/3881884924 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa9780708a0 0x7fa97811a010 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:36.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:36.699+0000 7fa97d200700 1 --2- 172.21.15.37:0/3881884924 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa978071f90 0x7fa97811a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:36.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:36.699+0000 7fa97d200700 1 -- 172.21.15.37:0/3881884924 >> 172.21.15.37:0/3881884924 conn(0x7fa9780f89b0 msgr2=0x7fa9780f9ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:19:36.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:36.699+0000 7fa97d200700 1 -- 172.21.15.37:0/3881884924 shutdown_connections 2024-03-29T08:19:36.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:19:36.699+0000 7fa97d200700 1 -- 172.21.15.37:0/3881884924 wait complete. 2024-03-29T08:19:37.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:19:37 smithi037 ceph-mon[142325]: from='client.34552 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:19:37.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:19:37 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/3881884924' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T08:19:37.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:19:37 smithi148 ceph-mon[106692]: from='client.34552 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:19:37.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:19:37 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/3881884924' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T08:19:38.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:19:38 smithi037 ceph-mon[142325]: pgmap v262: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:19:38.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:19:38 smithi148 ceph-mon[106692]: pgmap v262: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:19:40.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:19:40 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:19:40.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:19:40 smithi037 ceph-mon[142325]: pgmap v263: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:19:40.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:19:40 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:19:40.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:19:40 smithi148 ceph-mon[106692]: pgmap v263: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:19:42.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:19:41 smithi148 ceph-mon[106692]: pgmap v264: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:19:42.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:19:41 smithi037 ceph-mon[142325]: pgmap v264: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:19:44.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:19:43 smithi148 ceph-mon[106692]: pgmap v265: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:19:44.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:19:43 smithi037 ceph-mon[142325]: pgmap v265: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:19:46.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:19:45 smithi148 ceph-mon[106692]: pgmap v266: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:19:46.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:19:45 smithi037 ceph-mon[142325]: pgmap v266: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:19:48.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:19:47 smithi148 ceph-mon[106692]: pgmap v267: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:19:48.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:19:47 smithi037 ceph-mon[142325]: pgmap v267: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:19:50.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:19:50 smithi037 ceph-mon[142325]: pgmap v268: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:19:50.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:19:50 smithi148 ceph-mon[106692]: pgmap v268: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:19:52.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:19:51 smithi148 ceph-mon[106692]: pgmap v269: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:19:52.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:19:51 smithi037 ceph-mon[142325]: pgmap v269: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:19:54.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:19:53 smithi148 ceph-mon[106692]: pgmap v270: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:19:54.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:19:53 smithi037 ceph-mon[142325]: pgmap v270: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:19:55.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:19:55 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:19:55.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:19:55 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:19:56.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:19:56 smithi037 ceph-mon[142325]: pgmap v271: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:19:56.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:19:56 smithi148 ceph-mon[106692]: pgmap v271: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:19:58.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:19:57 smithi148 ceph-mon[106692]: pgmap v272: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:19:58.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:19:57 smithi037 ceph-mon[142325]: pgmap v272: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:20:00.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:20:00 smithi037 ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4-mon-smithi037[142302]: 2024-03-29T08:19:59.999+0000 7f2c60400700 -1 log_channel(cluster) log [ERR] : Health detail: HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-03-29T08:20:00.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:20:00 smithi037 ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4-mon-smithi037[142302]: 2024-03-29T08:19:59.999+0000 7f2c60400700 -1 log_channel(cluster) log [ERR] : [WRN] FS_DEGRADED: 1 filesystem is degraded 2024-03-29T08:20:00.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:20:00 smithi037 ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4-mon-smithi037[142302]: 2024-03-29T08:19:59.999+0000 7f2c60400700 -1 log_channel(cluster) log [ERR] : fs cephfs is degraded 2024-03-29T08:20:00.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:20:00 smithi037 ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4-mon-smithi037[142302]: 2024-03-29T08:19:59.999+0000 7f2c60400700 -1 log_channel(cluster) log [ERR] : [ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-03-29T08:20:00.428 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:20:00 smithi037 ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4-mon-smithi037[142302]: 2024-03-29T08:19:59.999+0000 7f2c60400700 -1 log_channel(cluster) log [ERR] : fs cephfs is offline because no MDS is active for it. 2024-03-29T08:20:00.428 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:20:00 smithi037 ceph-mon[142325]: pgmap v273: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:20:00.428 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:20:00 smithi037 ceph-mon[142325]: Health detail: HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-03-29T08:20:00.428 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:20:00 smithi037 ceph-mon[142325]: [WRN] FS_DEGRADED: 1 filesystem is degraded 2024-03-29T08:20:00.428 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:20:00 smithi037 ceph-mon[142325]: fs cephfs is degraded 2024-03-29T08:20:00.428 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:20:00 smithi037 ceph-mon[142325]: [ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-03-29T08:20:00.428 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:20:00 smithi037 ceph-mon[142325]: fs cephfs is offline because no MDS is active for it. 2024-03-29T08:20:00.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:20:00 smithi148 ceph-mon[106692]: pgmap v273: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:20:00.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:20:00 smithi148 ceph-mon[106692]: Health detail: HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-03-29T08:20:00.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:20:00 smithi148 ceph-mon[106692]: [WRN] FS_DEGRADED: 1 filesystem is degraded 2024-03-29T08:20:00.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:20:00 smithi148 ceph-mon[106692]: fs cephfs is degraded 2024-03-29T08:20:00.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:20:00 smithi148 ceph-mon[106692]: [ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-03-29T08:20:00.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:20:00 smithi148 ceph-mon[106692]: fs cephfs is offline because no MDS is active for it. 2024-03-29T08:20:02.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:20:01 smithi148 ceph-mon[106692]: pgmap v274: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:20:02.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:20:01 smithi037 ceph-mon[142325]: pgmap v274: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:20:04.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:20:03 smithi148 ceph-mon[106692]: pgmap v275: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:20:04.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:20:03 smithi037 ceph-mon[142325]: pgmap v275: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:20:06.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:20:05 smithi148 ceph-mon[106692]: pgmap v276: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:20:06.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:20:05 smithi037 ceph-mon[142325]: pgmap v276: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:20:06.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:06.853+0000 7f4c98200700 1 -- 172.21.15.37:0/3198460521 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4c900fde90 msgr2=0x7f4c900fe260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:06.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:06.853+0000 7f4c98200700 1 --2- 172.21.15.37:0/3198460521 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4c900fde90 0x7f4c900fe260 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f4c84004940 tx=0x7f4c84009e30 comp rx=0 tx=0).stop 2024-03-29T08:20:06.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:06.853+0000 7f4c98200700 1 -- 172.21.15.37:0/3198460521 shutdown_connections 2024-03-29T08:20:06.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:06.853+0000 7f4c98200700 1 --2- 172.21.15.37:0/3198460521 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4c900fe830 0x7f4c90106f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:06.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:06.853+0000 7f4c98200700 1 --2- 172.21.15.37:0/3198460521 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4c900fde90 0x7f4c900fe260 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:06.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:06.853+0000 7f4c98200700 1 -- 172.21.15.37:0/3198460521 >> 172.21.15.37:0/3198460521 conn(0x7f4c90072bc0 msgr2=0x7f4c90072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:20:06.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:06.853+0000 7f4c98200700 1 -- 172.21.15.37:0/3198460521 shutdown_connections 2024-03-29T08:20:06.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:06.853+0000 7f4c98200700 1 -- 172.21.15.37:0/3198460521 wait complete. 2024-03-29T08:20:06.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:06.854+0000 7f4c98200700 1 Processor -- start 2024-03-29T08:20:06.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:06.854+0000 7f4c98200700 1 -- start start 2024-03-29T08:20:06.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:06.855+0000 7f4c98200700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4c900fde90 0x7f4c900fbd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:06.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:06.855+0000 7f4c98200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4c900fe830 0x7f4c900fc290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:06.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:06.855+0000 7f4c98200700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c900fc820 con 0x7f4c900fe830 2024-03-29T08:20:06.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:06.855+0000 7f4c98200700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c900fc960 con 0x7f4c900fde90 2024-03-29T08:20:06.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:06.855+0000 7f4c94e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4c900fe830 0x7f4c900fc290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:06.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:06.855+0000 7f4c94e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4c900fe830 0x7f4c900fc290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37440/0 (socket says 172.21.15.37:37440) 2024-03-29T08:20:06.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:06.855+0000 7f4c94e00700 1 -- 172.21.15.37:0/400806987 learned_addr learned my addr 172.21.15.37:0/400806987 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:20:06.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:06.855+0000 7f4c95800700 1 --2- 172.21.15.37:0/400806987 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4c900fde90 0x7f4c900fbd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:06.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:06.856+0000 7f4c94e00700 1 -- 172.21.15.37:0/400806987 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4c900fde90 msgr2=0x7f4c900fbd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:06.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:06.856+0000 7f4c94e00700 1 --2- 172.21.15.37:0/400806987 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4c900fde90 0x7f4c900fbd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:06.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:06.856+0000 7f4c94e00700 1 -- 172.21.15.37:0/400806987 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c84005040 con 0x7f4c900fe830 2024-03-29T08:20:06.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:06.856+0000 7f4c95800700 1 --2- 172.21.15.37:0/400806987 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4c900fde90 0x7f4c900fbd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:20:06.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:06.856+0000 7f4c94e00700 1 --2- 172.21.15.37:0/400806987 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4c900fe830 0x7f4c900fc290 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f4c7c00eb40 tx=0x7f4c7c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:20:06.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:06.856+0000 7f4c8d600700 1 -- 172.21.15.37:0/400806987 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4c7c007cc0 con 0x7f4c900fe830 2024-03-29T08:20:06.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:06.856+0000 7f4c98200700 1 -- 172.21.15.37:0/400806987 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c900fcbb0 con 0x7f4c900fe830 2024-03-29T08:20:06.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:06.856+0000 7f4c8d600700 1 -- 172.21.15.37:0/400806987 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f4c7c00f040 con 0x7f4c900fe830 2024-03-29T08:20:06.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:06.856+0000 7f4c8d600700 1 -- 172.21.15.37:0/400806987 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4c7c009570 con 0x7f4c900fe830 2024-03-29T08:20:06.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:06.856+0000 7f4c98200700 1 -- 172.21.15.37:0/400806987 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c9011a120 con 0x7f4c900fe830 2024-03-29T08:20:06.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:06.858+0000 7f4c98200700 1 -- 172.21.15.37:0/400806987 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c90065e30 con 0x7f4c900fe830 2024-03-29T08:20:06.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:06.858+0000 7f4c8d600700 1 -- 172.21.15.37:0/400806987 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f4c7c0096d0 con 0x7f4c900fe830 2024-03-29T08:20:06.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:06.860+0000 7f4c8d600700 1 --2- 172.21.15.37:0/400806987 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f4c7806cc30 0x7f4c7806f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:06.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:06.860+0000 7f4c8d600700 1 -- 172.21.15.37:0/400806987 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f4c7c08bd80 con 0x7f4c900fe830 2024-03-29T08:20:06.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:06.860+0000 7f4c95800700 1 --2- 172.21.15.37:0/400806987 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f4c7806cc30 0x7f4c7806f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:06.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:06.861+0000 7f4c95800700 1 --2- 172.21.15.37:0/400806987 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f4c7806cc30 0x7f4c7806f0e0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f4c8400b0d0 tx=0x7f4c84009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:20:06.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:06.864+0000 7f4c8d600700 1 -- 172.21.15.37:0/400806987 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f4c7c055d70 con 0x7f4c900fe830 2024-03-29T08:20:07.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:07.096+0000 7f4c98200700 1 -- 172.21.15.37:0/400806987 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4c9004bed0 con 0x7f4c7806cc30 2024-03-29T08:20:07.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:07.097+0000 7f4c8d600700 1 -- 172.21.15.37:0/400806987 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f4c9004bed0 con 0x7f4c7806cc30 2024-03-29T08:20:07.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:07.100+0000 7f4c98200700 1 -- 172.21.15.37:0/400806987 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f4c7806cc30 msgr2=0x7f4c7806f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:07.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:07.100+0000 7f4c98200700 1 --2- 172.21.15.37:0/400806987 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f4c7806cc30 0x7f4c7806f0e0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f4c8400b0d0 tx=0x7f4c84009e30 comp rx=0 tx=0).stop 2024-03-29T08:20:07.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:07.100+0000 7f4c98200700 1 -- 172.21.15.37:0/400806987 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4c900fe830 msgr2=0x7f4c900fc290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:07.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:07.100+0000 7f4c98200700 1 --2- 172.21.15.37:0/400806987 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4c900fe830 0x7f4c900fc290 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f4c7c00eb40 tx=0x7f4c7c00b040 comp rx=0 tx=0).stop 2024-03-29T08:20:07.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:07.100+0000 7f4c98200700 1 -- 172.21.15.37:0/400806987 shutdown_connections 2024-03-29T08:20:07.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:07.100+0000 7f4c98200700 1 --2- 172.21.15.37:0/400806987 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f4c7806cc30 0x7f4c7806f0e0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:07.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:07.100+0000 7f4c98200700 1 --2- 172.21.15.37:0/400806987 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4c900fde90 0x7f4c900fbd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:07.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:07.100+0000 7f4c98200700 1 --2- 172.21.15.37:0/400806987 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4c900fe830 0x7f4c900fc290 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:07.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:07.100+0000 7f4c98200700 1 -- 172.21.15.37:0/400806987 >> 172.21.15.37:0/400806987 conn(0x7f4c90072bc0 msgr2=0x7f4c901037c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:20:07.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:07.100+0000 7f4c98200700 1 -- 172.21.15.37:0/400806987 shutdown_connections 2024-03-29T08:20:07.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:07.100+0000 7f4c98200700 1 -- 172.21.15.37:0/400806987 wait complete. 2024-03-29T08:20:07.115 INFO:teuthology.orchestra.run.smithi037.stdout:true 2024-03-29T08:20:07.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:07.239+0000 7fe207200700 1 -- 172.21.15.37:0/3738621619 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe208102c40 msgr2=0x7fe208103010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:07.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:07.239+0000 7fe207200700 1 --2- 172.21.15.37:0/3738621619 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe208102c40 0x7fe208103010 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fe1f8004940 tx=0x7fe1f8009e30 comp rx=0 tx=0).stop 2024-03-29T08:20:07.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:07.239+0000 7fe207200700 1 -- 172.21.15.37:0/3738621619 shutdown_connections 2024-03-29T08:20:07.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:07.239+0000 7fe207200700 1 --2- 172.21.15.37:0/3738621619 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe2081035e0 0x7fe2080fcc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:07.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:07.239+0000 7fe207200700 1 --2- 172.21.15.37:0/3738621619 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe208102c40 0x7fe208103010 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:07.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:07.239+0000 7fe207200700 1 -- 172.21.15.37:0/3738621619 >> 172.21.15.37:0/3738621619 conn(0x7fe2080f8830 msgr2=0x7fe2080fac40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:20:07.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:07.239+0000 7fe207200700 1 -- 172.21.15.37:0/3738621619 shutdown_connections 2024-03-29T08:20:07.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:07.239+0000 7fe207200700 1 -- 172.21.15.37:0/3738621619 wait complete. 2024-03-29T08:20:07.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:07.240+0000 7fe207200700 1 Processor -- start 2024-03-29T08:20:07.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:07.240+0000 7fe207200700 1 -- start start 2024-03-29T08:20:07.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:07.240+0000 7fe207200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe208102c40 0x7fe208115b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:07.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:07.240+0000 7fe207200700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe2081035e0 0x7fe208116070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:07.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:07.240+0000 7fe207200700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2081166c0 con 0x7fe208102c40 2024-03-29T08:20:07.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:07.240+0000 7fe207200700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe20810fab0 con 0x7fe2081035e0 2024-03-29T08:20:07.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:07.241+0000 7fe205e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe208102c40 0x7fe208115b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:07.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:07.241+0000 7fe205e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe208102c40 0x7fe208115b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37452/0 (socket says 172.21.15.37:37452) 2024-03-29T08:20:07.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:07.241+0000 7fe205e00700 1 -- 172.21.15.37:0/1504472293 learned_addr learned my addr 172.21.15.37:0/1504472293 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:20:07.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:07.241+0000 7fe205400700 1 --2- 172.21.15.37:0/1504472293 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe2081035e0 0x7fe208116070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:07.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:07.241+0000 7fe205e00700 1 -- 172.21.15.37:0/1504472293 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe2081035e0 msgr2=0x7fe208116070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:07.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:07.241+0000 7fe205e00700 1 --2- 172.21.15.37:0/1504472293 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe2081035e0 0x7fe208116070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:07.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:07.241+0000 7fe205e00700 1 -- 172.21.15.37:0/1504472293 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1f8005040 con 0x7fe208102c40 2024-03-29T08:20:07.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:07.241+0000 7fe205e00700 1 --2- 172.21.15.37:0/1504472293 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe208102c40 0x7fe208115b30 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fe1f8009390 tx=0x7fe1f8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:20:07.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:07.242+0000 7fe1fe000700 1 -- 172.21.15.37:0/1504472293 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe1f800e040 con 0x7fe208102c40 2024-03-29T08:20:07.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:07.242+0000 7fe207200700 1 -- 172.21.15.37:0/1504472293 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe20810fd30 con 0x7fe208102c40 2024-03-29T08:20:07.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:07.242+0000 7fe1fe000700 1 -- 172.21.15.37:0/1504472293 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fe1f8013420 con 0x7fe208102c40 2024-03-29T08:20:07.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:07.242+0000 7fe1fe000700 1 -- 172.21.15.37:0/1504472293 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe1f801c630 con 0x7fe208102c40 2024-03-29T08:20:07.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:07.242+0000 7fe207200700 1 -- 172.21.15.37:0/1504472293 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2081102f0 con 0x7fe208102c40 2024-03-29T08:20:07.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:07.243+0000 7fe207200700 1 -- 172.21.15.37:0/1504472293 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2080fe390 con 0x7fe208102c40 2024-03-29T08:20:07.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:07.244+0000 7fe1fe000700 1 -- 172.21.15.37:0/1504472293 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fe1f80139e0 con 0x7fe208102c40 2024-03-29T08:20:07.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:07.245+0000 7fe1fe000700 1 --2- 172.21.15.37:0/1504472293 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fe1ec06cc30 0x7fe1ec06f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:07.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:07.245+0000 7fe1fe000700 1 -- 172.21.15.37:0/1504472293 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fe1f808d3d0 con 0x7fe208102c40 2024-03-29T08:20:07.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:07.245+0000 7fe205400700 1 --2- 172.21.15.37:0/1504472293 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fe1ec06cc30 0x7fe1ec06f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:07.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:07.246+0000 7fe205400700 1 --2- 172.21.15.37:0/1504472293 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fe1ec06cc30 0x7fe1ec06f0e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fe208110c70 tx=0x7fe1f400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:20:07.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:07.249+0000 7fe1fe000700 1 -- 172.21.15.37:0/1504472293 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fe1f805ae20 con 0x7fe208102c40 2024-03-29T08:20:07.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:07.481+0000 7fe207200700 1 -- 172.21.15.37:0/1504472293 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe208109bf0 con 0x7fe1ec06cc30 2024-03-29T08:20:07.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:07.482+0000 7fe1fe000700 1 -- 172.21.15.37:0/1504472293 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fe208109bf0 con 0x7fe1ec06cc30 2024-03-29T08:20:07.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:07.484+0000 7fe207200700 1 -- 172.21.15.37:0/1504472293 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fe1ec06cc30 msgr2=0x7fe1ec06f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:07.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:07.484+0000 7fe207200700 1 --2- 172.21.15.37:0/1504472293 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fe1ec06cc30 0x7fe1ec06f0e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fe208110c70 tx=0x7fe1f400b040 comp rx=0 tx=0).stop 2024-03-29T08:20:07.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:07.484+0000 7fe207200700 1 -- 172.21.15.37:0/1504472293 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe208102c40 msgr2=0x7fe208115b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:07.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:07.484+0000 7fe207200700 1 --2- 172.21.15.37:0/1504472293 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe208102c40 0x7fe208115b30 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fe1f8009390 tx=0x7fe1f8009e30 comp rx=0 tx=0).stop 2024-03-29T08:20:07.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:07.484+0000 7fe207200700 1 -- 172.21.15.37:0/1504472293 shutdown_connections 2024-03-29T08:20:07.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:07.484+0000 7fe207200700 1 --2- 172.21.15.37:0/1504472293 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fe1ec06cc30 0x7fe1ec06f0e0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:07.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:07.484+0000 7fe207200700 1 --2- 172.21.15.37:0/1504472293 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe208102c40 0x7fe208115b30 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:07.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:07.484+0000 7fe207200700 1 --2- 172.21.15.37:0/1504472293 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe2081035e0 0x7fe208116070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:07.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:07.484+0000 7fe207200700 1 -- 172.21.15.37:0/1504472293 >> 172.21.15.37:0/1504472293 conn(0x7fe2080f8830 msgr2=0x7fe2080f9890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:20:07.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:07.485+0000 7fe207200700 1 -- 172.21.15.37:0/1504472293 shutdown_connections 2024-03-29T08:20:07.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:07.485+0000 7fe207200700 1 -- 172.21.15.37:0/1504472293 wait complete. 2024-03-29T08:20:07.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:07.621+0000 7f6cef600700 1 -- 172.21.15.37:0/473532107 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6ce8105990 msgr2=0x7f6ce80ff000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:07.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:07.621+0000 7f6cef600700 1 --2- 172.21.15.37:0/473532107 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6ce8105990 0x7f6ce80ff000 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f6cd4004940 tx=0x7f6cd4009e30 comp rx=0 tx=0).stop 2024-03-29T08:20:07.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:07.622+0000 7f6cef600700 1 -- 172.21.15.37:0/473532107 shutdown_connections 2024-03-29T08:20:07.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:07.622+0000 7f6cef600700 1 --2- 172.21.15.37:0/473532107 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6ce8105990 0x7f6ce80ff000 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:07.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:07.622+0000 7f6cef600700 1 --2- 172.21.15.37:0/473532107 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6ce8104ff0 0x7f6ce81053c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:07.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:07.622+0000 7f6cef600700 1 -- 172.21.15.37:0/473532107 >> 172.21.15.37:0/473532107 conn(0x7f6ce80faba0 msgr2=0x7f6ce80fcfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:20:07.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:07.622+0000 7f6cef600700 1 -- 172.21.15.37:0/473532107 shutdown_connections 2024-03-29T08:20:07.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:07.622+0000 7f6cef600700 1 -- 172.21.15.37:0/473532107 wait complete. 2024-03-29T08:20:07.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:07.622+0000 7f6cef600700 1 Processor -- start 2024-03-29T08:20:07.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:07.623+0000 7f6cef600700 1 -- start start 2024-03-29T08:20:07.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:07.623+0000 7f6cef600700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6ce8104ff0 0x7f6ce8115b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:07.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:07.623+0000 7f6cef600700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6ce8105990 0x7f6ce8116080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:07.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:07.623+0000 7f6cef600700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ce81166d0 con 0x7f6ce8105990 2024-03-29T08:20:07.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:07.623+0000 7f6cef600700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ce810fcf0 con 0x7f6ce8104ff0 2024-03-29T08:20:07.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:07.623+0000 7f6ce7e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6ce8105990 0x7f6ce8116080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:07.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:07.623+0000 7f6ce7e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6ce8105990 0x7f6ce8116080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37476/0 (socket says 172.21.15.37:37476) 2024-03-29T08:20:07.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:07.623+0000 7f6ce7e00700 1 -- 172.21.15.37:0/2030119622 learned_addr learned my addr 172.21.15.37:0/2030119622 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:20:07.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:07.623+0000 7f6cecc00700 1 --2- 172.21.15.37:0/2030119622 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6ce8104ff0 0x7f6ce8115b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:07.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:07.624+0000 7f6ce7e00700 1 -- 172.21.15.37:0/2030119622 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6ce8104ff0 msgr2=0x7f6ce8115b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:07.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:07.624+0000 7f6ce7e00700 1 --2- 172.21.15.37:0/2030119622 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6ce8104ff0 0x7f6ce8115b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:07.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:07.624+0000 7f6ce7e00700 1 -- 172.21.15.37:0/2030119622 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cd4005040 con 0x7f6ce8105990 2024-03-29T08:20:07.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:07.624+0000 7f6ce7e00700 1 --2- 172.21.15.37:0/2030119622 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6ce8105990 0x7f6ce8116080 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f6cd4009390 tx=0x7f6cd4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:20:07.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:07.624+0000 7f6ce4c00700 1 -- 172.21.15.37:0/2030119622 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6cd400e040 con 0x7f6ce8105990 2024-03-29T08:20:07.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:07.624+0000 7f6cef600700 1 -- 172.21.15.37:0/2030119622 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ce810ff70 con 0x7f6ce8105990 2024-03-29T08:20:07.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:07.624+0000 7f6ce4c00700 1 -- 172.21.15.37:0/2030119622 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f6cd4013420 con 0x7f6ce8105990 2024-03-29T08:20:07.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:07.624+0000 7f6ce4c00700 1 -- 172.21.15.37:0/2030119622 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6cd401c630 con 0x7f6ce8105990 2024-03-29T08:20:07.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:07.624+0000 7f6cef600700 1 -- 172.21.15.37:0/2030119622 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ce81104b0 con 0x7f6ce8105990 2024-03-29T08:20:07.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:07.625+0000 7f6cef600700 1 -- 172.21.15.37:0/2030119622 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ce8102300 con 0x7f6ce8105990 2024-03-29T08:20:07.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:07.626+0000 7f6ce4c00700 1 -- 172.21.15.37:0/2030119622 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f6cd4013a00 con 0x7f6ce8105990 2024-03-29T08:20:07.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:07.627+0000 7f6ce4c00700 1 --2- 172.21.15.37:0/2030119622 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f6cd006cc20 0x7f6cd006f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:07.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:07.627+0000 7f6ce4c00700 1 -- 172.21.15.37:0/2030119622 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f6cd408d1b0 con 0x7f6ce8105990 2024-03-29T08:20:07.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:07.627+0000 7f6cecc00700 1 --2- 172.21.15.37:0/2030119622 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f6cd006cc20 0x7f6cd006f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:07.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:07.628+0000 7f6cecc00700 1 --2- 172.21.15.37:0/2030119622 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f6cd006cc20 0x7f6cd006f0d0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f6cdc004f40 tx=0x7f6cdc00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:20:07.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:07.632+0000 7f6ce4c00700 1 -- 172.21.15.37:0/2030119622 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f6cd4057170 con 0x7f6ce8105990 2024-03-29T08:20:07.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:07.856+0000 7f6cef600700 1 -- 172.21.15.37:0/2030119622 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f6ce81111d0 con 0x7f6cd006cc20 2024-03-29T08:20:07.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:07.866+0000 7f6ce4c00700 1 -- 172.21.15.37:0/2030119622 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f6ce81111d0 con 0x7f6cd006cc20 2024-03-29T08:20:07.866 INFO:teuthology.orchestra.run.smithi037.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-29T08:20:07.866 INFO:teuthology.orchestra.run.smithi037.stdout:alertmanager.smithi037 smithi037 *:9093,9094 running (9m) 2m ago 22m 14.9M - 0.25.0 c8568f914cd2 2f9f9746ed0f 2024-03-29T08:20:07.866 INFO:teuthology.orchestra.run.smithi037.stdout:crash.smithi037 smithi037 running (6m) 2m ago 23m 6647k - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 178d416a3186 2024-03-29T08:20:07.866 INFO:teuthology.orchestra.run.smithi037.stdout:crash.smithi148 smithi148 running (6m) 115s ago 21m 6656k - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 23666989d58f 2024-03-29T08:20:07.867 INFO:teuthology.orchestra.run.smithi037.stdout:grafana.smithi037 smithi037 *:3000 running (9m) 2m ago 22m 71.9M - 9.4.7 954c08fa6188 6b976fe49952 2024-03-29T08:20:07.867 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi037.iifsmh smithi037 running (2m) 2m ago 16m 10.8M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 2d97f562cc5b 2024-03-29T08:20:07.867 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi037.mmwgxe smithi037 running (2m) 2m ago 16m 10.9M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff ec37037f46e8 2024-03-29T08:20:07.867 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi148.qukpjh smithi148 running (16m) 115s ago 16m 15.3M - 17.2.7-789-g4213e17c cf22eb8683be 8e469782a273 2024-03-29T08:20:07.867 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi148.sqteih smithi148 running (2m) 115s ago 16m 10.7M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 44c15c77037a 2024-03-29T08:20:07.867 INFO:teuthology.orchestra.run.smithi037.stdout:mgr.smithi037.wjikii smithi037 *:8443,9283,8765 running (12m) 2m ago 24m 506M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff c5a1417b9417 2024-03-29T08:20:07.867 INFO:teuthology.orchestra.run.smithi037.stdout:mgr.smithi148.onhgvm smithi148 *:8443,9283,8765 running (11m) 115s ago 21m 450M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff fcfdbce062bd 2024-03-29T08:20:07.867 INFO:teuthology.orchestra.run.smithi037.stdout:mon.smithi037 smithi037 running (8m) 2m ago 24m 54.8M 2048M 18.2.2-1015-gd0e392d9 1cc2f45c17ff eea2060783af 2024-03-29T08:20:07.867 INFO:teuthology.orchestra.run.smithi037.stdout:mon.smithi148 smithi148 running (7m) 115s ago 21m 43.7M 2048M 18.2.2-1015-gd0e392d9 1cc2f45c17ff ad4b2c5748be 2024-03-29T08:20:07.867 INFO:teuthology.orchestra.run.smithi037.stdout:node-exporter.smithi037 smithi037 *:9100 running (10m) 2m ago 23m 7625k - 1.5.0 0da6a335fe13 26b8ae9cb878 2024-03-29T08:20:07.867 INFO:teuthology.orchestra.run.smithi037.stdout:node-exporter.smithi148 smithi148 *:9100 running (10m) 115s ago 21m 8166k - 1.5.0 0da6a335fe13 98414e4e64ed 2024-03-29T08:20:07.867 INFO:teuthology.orchestra.run.smithi037.stdout:osd.0 smithi037 running (6m) 2m ago 20m 104M 1619M 18.2.2-1015-gd0e392d9 1cc2f45c17ff c0c584f6245c 2024-03-29T08:20:07.867 INFO:teuthology.orchestra.run.smithi037.stdout:osd.1 smithi037 running (5m) 2m ago 20m 100M 1619M 18.2.2-1015-gd0e392d9 1cc2f45c17ff a4a2ff4cc83d 2024-03-29T08:20:07.867 INFO:teuthology.orchestra.run.smithi037.stdout:osd.2 smithi037 running (5m) 2m ago 19m 82.3M 1619M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 8b1bc71aa79e 2024-03-29T08:20:07.868 INFO:teuthology.orchestra.run.smithi037.stdout:osd.3 smithi148 running (4m) 115s ago 19m 105M 2642M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 1c656ab8ff86 2024-03-29T08:20:07.868 INFO:teuthology.orchestra.run.smithi037.stdout:osd.4 smithi148 running (3m) 115s ago 18m 97.3M 2642M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 966a9de11d22 2024-03-29T08:20:07.868 INFO:teuthology.orchestra.run.smithi037.stdout:osd.5 smithi148 running (3m) 115s ago 18m 85.8M 2642M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 474c4f9212ce 2024-03-29T08:20:07.868 INFO:teuthology.orchestra.run.smithi037.stdout:prometheus.smithi037 smithi037 *:9095 running (10m) 2m ago 22m 54.4M - 2.43.0 a07b618ecd1d 30c0223ec589 2024-03-29T08:20:07.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:07.869+0000 7f6cef600700 1 -- 172.21.15.37:0/2030119622 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f6cd006cc20 msgr2=0x7f6cd006f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:07.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:07.869+0000 7f6cef600700 1 --2- 172.21.15.37:0/2030119622 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f6cd006cc20 0x7f6cd006f0d0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f6cdc004f40 tx=0x7f6cdc00b040 comp rx=0 tx=0).stop 2024-03-29T08:20:07.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:07.869+0000 7f6cef600700 1 -- 172.21.15.37:0/2030119622 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6ce8105990 msgr2=0x7f6ce8116080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:07.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:07.869+0000 7f6cef600700 1 --2- 172.21.15.37:0/2030119622 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6ce8105990 0x7f6ce8116080 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f6cd4009390 tx=0x7f6cd4009e30 comp rx=0 tx=0).stop 2024-03-29T08:20:07.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:07.869+0000 7f6cef600700 1 -- 172.21.15.37:0/2030119622 shutdown_connections 2024-03-29T08:20:07.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:07.869+0000 7f6cef600700 1 --2- 172.21.15.37:0/2030119622 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f6cd006cc20 0x7f6cd006f0d0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:07.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:07.869+0000 7f6cef600700 1 --2- 172.21.15.37:0/2030119622 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6ce8104ff0 0x7f6ce8115b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:07.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:07.869+0000 7f6cef600700 1 --2- 172.21.15.37:0/2030119622 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6ce8105990 0x7f6ce8116080 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:07.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:07.869+0000 7f6cef600700 1 -- 172.21.15.37:0/2030119622 >> 172.21.15.37:0/2030119622 conn(0x7f6ce80faba0 msgr2=0x7f6ce80fcad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:20:07.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:07.869+0000 7f6cef600700 1 -- 172.21.15.37:0/2030119622 shutdown_connections 2024-03-29T08:20:07.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:07.869+0000 7f6cef600700 1 -- 172.21.15.37:0/2030119622 wait complete. 2024-03-29T08:20:08.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:08.004+0000 7f325ce00700 1 -- 172.21.15.37:0/1558395996 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f32580fe830 msgr2=0x7f3258106f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:08.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:08.004+0000 7f325ce00700 1 --2- 172.21.15.37:0/1558395996 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f32580fe830 0x7f3258106f00 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f3244004940 tx=0x7f3244009e30 comp rx=0 tx=0).stop 2024-03-29T08:20:08.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:08.005+0000 7f325ce00700 1 -- 172.21.15.37:0/1558395996 shutdown_connections 2024-03-29T08:20:08.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:08.005+0000 7f325ce00700 1 --2- 172.21.15.37:0/1558395996 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f32580fe830 0x7f3258106f00 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:08.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:08.005+0000 7f325ce00700 1 --2- 172.21.15.37:0/1558395996 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f32580fde90 0x7f32580fe260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:08.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:08.005+0000 7f325ce00700 1 -- 172.21.15.37:0/1558395996 >> 172.21.15.37:0/1558395996 conn(0x7f3258072bc0 msgr2=0x7f3258072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:20:08.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:08.005+0000 7f325ce00700 1 -- 172.21.15.37:0/1558395996 shutdown_connections 2024-03-29T08:20:08.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:08.005+0000 7f325ce00700 1 -- 172.21.15.37:0/1558395996 wait complete. 2024-03-29T08:20:08.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:08.005+0000 7f325ce00700 1 Processor -- start 2024-03-29T08:20:08.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:08.006+0000 7f325ce00700 1 -- start start 2024-03-29T08:20:08.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:08.006+0000 7f325ce00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f32580fde90 0x7f3258113ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:08.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:08.006+0000 7f325ce00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f32580fe830 0x7f3258114020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:08.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:08.006+0000 7f325ce00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3258114670 con 0x7f32580fe830 2024-03-29T08:20:08.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:08.006+0000 7f325ce00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f325810da60 con 0x7f32580fde90 2024-03-29T08:20:08.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:08.006+0000 7f3255400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f32580fe830 0x7f3258114020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:08.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:08.006+0000 7f3255400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f32580fe830 0x7f3258114020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37498/0 (socket says 172.21.15.37:37498) 2024-03-29T08:20:08.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:08.006+0000 7f3255400700 1 -- 172.21.15.37:0/456768795 learned_addr learned my addr 172.21.15.37:0/456768795 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:20:08.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:08.007+0000 7f3255e00700 1 --2- 172.21.15.37:0/456768795 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f32580fde90 0x7f3258113ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:08.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:08.007+0000 7f3255400700 1 -- 172.21.15.37:0/456768795 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f32580fde90 msgr2=0x7f3258113ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:08.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:08.007+0000 7f3255400700 1 --2- 172.21.15.37:0/456768795 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f32580fde90 0x7f3258113ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:08.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:08.007+0000 7f3255400700 1 -- 172.21.15.37:0/456768795 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3244005040 con 0x7f32580fe830 2024-03-29T08:20:08.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:08.007+0000 7f3255e00700 1 --2- 172.21.15.37:0/456768795 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f32580fde90 0x7f3258113ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:20:08.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:08.007+0000 7f3255400700 1 --2- 172.21.15.37:0/456768795 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f32580fe830 0x7f3258114020 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f3244000c00 tx=0x7f3244009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:20:08.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:08.007+0000 7f324e000700 1 -- 172.21.15.37:0/456768795 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f324400e040 con 0x7f32580fe830 2024-03-29T08:20:08.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:08.007+0000 7f324e000700 1 -- 172.21.15.37:0/456768795 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f3244013420 con 0x7f32580fe830 2024-03-29T08:20:08.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:08.007+0000 7f324e000700 1 -- 172.21.15.37:0/456768795 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f324401c630 con 0x7f32580fe830 2024-03-29T08:20:08.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:08.007+0000 7f325ce00700 1 -- 172.21.15.37:0/456768795 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f325810dce0 con 0x7f32580fe830 2024-03-29T08:20:08.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:08.008+0000 7f325ce00700 1 -- 172.21.15.37:0/456768795 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f325810e220 con 0x7f32580fe830 2024-03-29T08:20:08.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:08.009+0000 7f325ce00700 1 -- 172.21.15.37:0/456768795 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3258065e30 con 0x7f32580fe830 2024-03-29T08:20:08.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:08.010+0000 7f324e000700 1 -- 172.21.15.37:0/456768795 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f3244013970 con 0x7f32580fe830 2024-03-29T08:20:08.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:08.011+0000 7f324e000700 1 --2- 172.21.15.37:0/456768795 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f323c06cc30 0x7f323c06f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:08.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:08.011+0000 7f324e000700 1 -- 172.21.15.37:0/456768795 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f324408d240 con 0x7f32580fe830 2024-03-29T08:20:08.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:08.011+0000 7f3255e00700 1 --2- 172.21.15.37:0/456768795 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f323c06cc30 0x7f323c06f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:08.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:08.012+0000 7f3255e00700 1 --2- 172.21.15.37:0/456768795 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f323c06cc30 0x7f323c06f0e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f3248009fd0 tx=0x7f3248009ee0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:20:08.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:08.015+0000 7f324e000700 1 -- 172.21.15.37:0/456768795 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f324405ad10 con 0x7f32580fe830 2024-03-29T08:20:08.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:20:07 smithi148 ceph-mon[106692]: from='client.34560 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:20:08.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:20:07 smithi148 ceph-mon[106692]: from='client.34564 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:20:08.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:20:07 smithi148 ceph-mon[106692]: pgmap v277: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:20:08.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:20:07 smithi037 ceph-mon[142325]: from='client.34560 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:20:08.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:20:07 smithi037 ceph-mon[142325]: from='client.34564 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:20:08.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:20:07 smithi037 ceph-mon[142325]: pgmap v277: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:20:08.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:08.320+0000 7f325ce00700 1 -- 172.21.15.37:0/456768795 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f3258109dd0 con 0x7f32580fe830 2024-03-29T08:20:08.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:08.320+0000 7f324e000700 1 -- 172.21.15.37:0/456768795 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+847 (secure 0 0 0) 0x7f324405a8a0 con 0x7f32580fe830 2024-03-29T08:20:08.320 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-29T08:20:08.320 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": { 2024-03-29T08:20:08.320 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 2 2024-03-29T08:20:08.321 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:20:08.321 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": { 2024-03-29T08:20:08.321 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 2 2024-03-29T08:20:08.321 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:20:08.321 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": { 2024-03-29T08:20:08.321 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 6 2024-03-29T08:20:08.321 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:20:08.321 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": { 2024-03-29T08:20:08.321 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 17.2.7-789-g4213e17c (4213e17c4de67016ed14f55d22503f4923b3a901) quincy (stable)": 1, 2024-03-29T08:20:08.321 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 3 2024-03-29T08:20:08.321 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:20:08.321 INFO:teuthology.orchestra.run.smithi037.stdout: "overall": { 2024-03-29T08:20:08.321 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 17.2.7-789-g4213e17c (4213e17c4de67016ed14f55d22503f4923b3a901) quincy (stable)": 1, 2024-03-29T08:20:08.321 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 13 2024-03-29T08:20:08.321 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-29T08:20:08.321 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-29T08:20:08.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:08.323+0000 7f325ce00700 1 -- 172.21.15.37:0/456768795 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f323c06cc30 msgr2=0x7f323c06f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:08.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:08.323+0000 7f325ce00700 1 --2- 172.21.15.37:0/456768795 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f323c06cc30 0x7f323c06f0e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f3248009fd0 tx=0x7f3248009ee0 comp rx=0 tx=0).stop 2024-03-29T08:20:08.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:08.323+0000 7f325ce00700 1 -- 172.21.15.37:0/456768795 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f32580fe830 msgr2=0x7f3258114020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:08.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:08.323+0000 7f325ce00700 1 --2- 172.21.15.37:0/456768795 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f32580fe830 0x7f3258114020 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f3244000c00 tx=0x7f3244009e30 comp rx=0 tx=0).stop 2024-03-29T08:20:08.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:08.323+0000 7f325ce00700 1 -- 172.21.15.37:0/456768795 shutdown_connections 2024-03-29T08:20:08.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:08.323+0000 7f325ce00700 1 --2- 172.21.15.37:0/456768795 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f323c06cc30 0x7f323c06f0e0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:08.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:08.323+0000 7f325ce00700 1 --2- 172.21.15.37:0/456768795 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f32580fde90 0x7f3258113ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:08.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:08.323+0000 7f325ce00700 1 --2- 172.21.15.37:0/456768795 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f32580fe830 0x7f3258114020 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:08.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:08.323+0000 7f325ce00700 1 -- 172.21.15.37:0/456768795 >> 172.21.15.37:0/456768795 conn(0x7f3258072bc0 msgr2=0x7f3258103860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:20:08.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:08.323+0000 7f325ce00700 1 -- 172.21.15.37:0/456768795 shutdown_connections 2024-03-29T08:20:08.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:08.323+0000 7f325ce00700 1 -- 172.21.15.37:0/456768795 wait complete. 2024-03-29T08:20:08.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:08.460+0000 7f17d7200700 1 -- 172.21.15.37:0/2675317894 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f17d01037a0 msgr2=0x7f17d00fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:08.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:08.460+0000 7f17d7200700 1 --2- 172.21.15.37:0/2675317894 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f17d01037a0 0x7f17d00fce10 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f17bc004940 tx=0x7f17bc009e30 comp rx=0 tx=0).stop 2024-03-29T08:20:08.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:08.460+0000 7f17d7200700 1 -- 172.21.15.37:0/2675317894 shutdown_connections 2024-03-29T08:20:08.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:08.460+0000 7f17d7200700 1 --2- 172.21.15.37:0/2675317894 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f17d01037a0 0x7f17d00fce10 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:08.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:08.460+0000 7f17d7200700 1 --2- 172.21.15.37:0/2675317894 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f17d0102e00 0x7f17d01031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:08.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:08.460+0000 7f17d7200700 1 -- 172.21.15.37:0/2675317894 >> 172.21.15.37:0/2675317894 conn(0x7f17d00f89b0 msgr2=0x7f17d00fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:20:08.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:08.460+0000 7f17d7200700 1 -- 172.21.15.37:0/2675317894 shutdown_connections 2024-03-29T08:20:08.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:08.460+0000 7f17d7200700 1 -- 172.21.15.37:0/2675317894 wait complete. 2024-03-29T08:20:08.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:08.461+0000 7f17d7200700 1 Processor -- start 2024-03-29T08:20:08.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:08.461+0000 7f17d7200700 1 -- start start 2024-03-29T08:20:08.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:08.462+0000 7f17d7200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f17d0102e00 0x7f17d0115cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:08.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:08.462+0000 7f17d7200700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f17d01037a0 0x7f17d0116230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:08.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:08.462+0000 7f17d7200700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17d0116880 con 0x7f17d0102e00 2024-03-29T08:20:08.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:08.462+0000 7f17d7200700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17d010fc50 con 0x7f17d01037a0 2024-03-29T08:20:08.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:08.462+0000 7f17cfe00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f17d0102e00 0x7f17d0115cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:08.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:08.462+0000 7f17cfe00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f17d0102e00 0x7f17d0115cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37520/0 (socket says 172.21.15.37:37520) 2024-03-29T08:20:08.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:08.462+0000 7f17cfe00700 1 -- 172.21.15.37:0/4205522908 learned_addr learned my addr 172.21.15.37:0/4205522908 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:20:08.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:08.462+0000 7f17cf400700 1 --2- 172.21.15.37:0/4205522908 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f17d01037a0 0x7f17d0116230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:08.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:08.462+0000 7f17cfe00700 1 -- 172.21.15.37:0/4205522908 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f17d01037a0 msgr2=0x7f17d0116230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:08.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:08.462+0000 7f17cfe00700 1 --2- 172.21.15.37:0/4205522908 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f17d01037a0 0x7f17d0116230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:08.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:08.462+0000 7f17cfe00700 1 -- 172.21.15.37:0/4205522908 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17bc005040 con 0x7f17d0102e00 2024-03-29T08:20:08.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:08.463+0000 7f17cfe00700 1 --2- 172.21.15.37:0/4205522908 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f17d0102e00 0x7f17d0115cf0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f17c0002a50 tx=0x7f17c00046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:20:08.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:08.463+0000 7f17c7e00700 1 -- 172.21.15.37:0/4205522908 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f17c0007cc0 con 0x7f17d0102e00 2024-03-29T08:20:08.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:08.463+0000 7f17c7e00700 1 -- 172.21.15.37:0/4205522908 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f17c000f040 con 0x7f17d0102e00 2024-03-29T08:20:08.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:08.463+0000 7f17d7200700 1 -- 172.21.15.37:0/4205522908 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17d010ff30 con 0x7f17d0102e00 2024-03-29T08:20:08.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:08.463+0000 7f17c7e00700 1 -- 172.21.15.37:0/4205522908 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f17c000d9c0 con 0x7f17d0102e00 2024-03-29T08:20:08.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:08.463+0000 7f17d7200700 1 -- 172.21.15.37:0/4205522908 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17d01104f0 con 0x7f17d0102e00 2024-03-29T08:20:08.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:08.464+0000 7f17d7200700 1 -- 172.21.15.37:0/4205522908 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17d00fe550 con 0x7f17d0102e00 2024-03-29T08:20:08.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:08.465+0000 7f17c7e00700 1 -- 172.21.15.37:0/4205522908 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f17c000db20 con 0x7f17d0102e00 2024-03-29T08:20:08.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:08.466+0000 7f17c7e00700 1 --2- 172.21.15.37:0/4205522908 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f17b406cc30 0x7f17b406f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:08.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:08.466+0000 7f17c7e00700 1 -- 172.21.15.37:0/4205522908 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f17c008bde0 con 0x7f17d0102e00 2024-03-29T08:20:08.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:08.466+0000 7f17cf400700 1 --2- 172.21.15.37:0/4205522908 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f17b406cc30 0x7f17b406f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:08.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:08.467+0000 7f17cf400700 1 --2- 172.21.15.37:0/4205522908 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f17b406cc30 0x7f17b406f0e0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f17d0111590 tx=0x7f17bc009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:20:08.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:08.471+0000 7f17c7e00700 1 -- 172.21.15.37:0/4205522908 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f17c00597b0 con 0x7f17d0102e00 2024-03-29T08:20:08.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:08.729+0000 7f17d7200700 1 -- 172.21.15.37:0/4205522908 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f17d004bed0 con 0x7f17d0102e00 2024-03-29T08:20:08.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:08.730+0000 7f17c7e00700 1 -- 172.21.15.37:0/4205522908 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 28 v28) v1 ==== 76+0+1841 (secure 0 0 0) 0x7f17c0059340 con 0x7f17d0102e00 2024-03-29T08:20:08.730 INFO:teuthology.orchestra.run.smithi037.stdout:e28 2024-03-29T08:20:08.730 INFO:teuthology.orchestra.run.smithi037.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-29T08:20:08.731 INFO:teuthology.orchestra.run.smithi037.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T08:20:08.731 INFO:teuthology.orchestra.run.smithi037.stdout:legacy client fscid: 1 2024-03-29T08:20:08.731 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:20:08.731 INFO:teuthology.orchestra.run.smithi037.stdout:Filesystem 'cephfs' (1) 2024-03-29T08:20:08.731 INFO:teuthology.orchestra.run.smithi037.stdout:fs_name cephfs 2024-03-29T08:20:08.731 INFO:teuthology.orchestra.run.smithi037.stdout:epoch 21 2024-03-29T08:20:08.731 INFO:teuthology.orchestra.run.smithi037.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-03-29T08:20:08.731 INFO:teuthology.orchestra.run.smithi037.stdout:created 2024-03-29T08:03:29.241269+0000 2024-03-29T08:20:08.731 INFO:teuthology.orchestra.run.smithi037.stdout:modified 2024-03-29T08:17:16.237238+0000 2024-03-29T08:20:08.731 INFO:teuthology.orchestra.run.smithi037.stdout:tableserver 0 2024-03-29T08:20:08.731 INFO:teuthology.orchestra.run.smithi037.stdout:root 0 2024-03-29T08:20:08.731 INFO:teuthology.orchestra.run.smithi037.stdout:session_timeout 60 2024-03-29T08:20:08.731 INFO:teuthology.orchestra.run.smithi037.stdout:session_autoclose 300 2024-03-29T08:20:08.731 INFO:teuthology.orchestra.run.smithi037.stdout:max_file_size 1099511627776 2024-03-29T08:20:08.732 INFO:teuthology.orchestra.run.smithi037.stdout:max_xattr_size 65536 2024-03-29T08:20:08.732 INFO:teuthology.orchestra.run.smithi037.stdout:required_client_features {} 2024-03-29T08:20:08.732 INFO:teuthology.orchestra.run.smithi037.stdout:last_failure 0 2024-03-29T08:20:08.732 INFO:teuthology.orchestra.run.smithi037.stdout:last_failure_osd_epoch 82 2024-03-29T08:20:08.732 INFO:teuthology.orchestra.run.smithi037.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T08:20:08.732 INFO:teuthology.orchestra.run.smithi037.stdout:max_mds 2 2024-03-29T08:20:08.732 INFO:teuthology.orchestra.run.smithi037.stdout:in 0,1 2024-03-29T08:20:08.732 INFO:teuthology.orchestra.run.smithi037.stdout:up {} 2024-03-29T08:20:08.732 INFO:teuthology.orchestra.run.smithi037.stdout:failed 0,1 2024-03-29T08:20:08.732 INFO:teuthology.orchestra.run.smithi037.stdout:damaged 2024-03-29T08:20:08.732 INFO:teuthology.orchestra.run.smithi037.stdout:stopped 2024-03-29T08:20:08.732 INFO:teuthology.orchestra.run.smithi037.stdout:data_pools [3] 2024-03-29T08:20:08.732 INFO:teuthology.orchestra.run.smithi037.stdout:metadata_pool 2 2024-03-29T08:20:08.732 INFO:teuthology.orchestra.run.smithi037.stdout:inline_data disabled 2024-03-29T08:20:08.732 INFO:teuthology.orchestra.run.smithi037.stdout:balancer 2024-03-29T08:20:08.733 INFO:teuthology.orchestra.run.smithi037.stdout:bal_rank_mask -1 2024-03-29T08:20:08.733 INFO:teuthology.orchestra.run.smithi037.stdout:standby_count_wanted 1 2024-03-29T08:20:08.733 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:20:08.733 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:20:08.733 INFO:teuthology.orchestra.run.smithi037.stdout:Standby daemons: 2024-03-29T08:20:08.733 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:20:08.733 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi148.qukpjh{-1:24319} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.148:6826/2641327912,v1:172.21.15.148:6827/2641327912] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:20:08.733 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi037.mmwgxe{-1:34440} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.37:6828/1248142005,v1:172.21.15.37:6829/1248142005] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:20:08.733 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi037.iifsmh{-1:34444} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.37:6826/2060882959,v1:172.21.15.37:6827/2060882959] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:20:08.733 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi148.sqteih{-1:44395} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.148:6824/4078652242,v1:172.21.15.148:6825/4078652242] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:20:08.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:08.732+0000 7f17d7200700 1 -- 172.21.15.37:0/4205522908 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f17b406cc30 msgr2=0x7f17b406f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:08.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:08.732+0000 7f17d7200700 1 --2- 172.21.15.37:0/4205522908 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f17b406cc30 0x7f17b406f0e0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f17d0111590 tx=0x7f17bc009e30 comp rx=0 tx=0).stop 2024-03-29T08:20:08.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:08.732+0000 7f17d7200700 1 -- 172.21.15.37:0/4205522908 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f17d0102e00 msgr2=0x7f17d0115cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:08.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:08.732+0000 7f17d7200700 1 --2- 172.21.15.37:0/4205522908 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f17d0102e00 0x7f17d0115cf0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f17c0002a50 tx=0x7f17c00046b0 comp rx=0 tx=0).stop 2024-03-29T08:20:08.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:08.732+0000 7f17d7200700 1 -- 172.21.15.37:0/4205522908 shutdown_connections 2024-03-29T08:20:08.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:08.732+0000 7f17d7200700 1 --2- 172.21.15.37:0/4205522908 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f17b406cc30 0x7f17b406f0e0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:08.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:08.732+0000 7f17d7200700 1 --2- 172.21.15.37:0/4205522908 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f17d0102e00 0x7f17d0115cf0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:08.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:08.732+0000 7f17d7200700 1 --2- 172.21.15.37:0/4205522908 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f17d01037a0 0x7f17d0116230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:08.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:08.732+0000 7f17d7200700 1 -- 172.21.15.37:0/4205522908 >> 172.21.15.37:0/4205522908 conn(0x7f17d00f89b0 msgr2=0x7f17d00f9a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:20:08.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:08.733+0000 7f17d7200700 1 -- 172.21.15.37:0/4205522908 shutdown_connections 2024-03-29T08:20:08.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:08.733+0000 7f17d7200700 1 -- 172.21.15.37:0/4205522908 wait complete. 2024-03-29T08:20:08.735 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 28 2024-03-29T08:20:08.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:08.870+0000 7febf9c00700 1 -- 172.21.15.37:0/2074360477 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7febf4102e00 msgr2=0x7febf41031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:08.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:08.870+0000 7febf9c00700 1 --2- 172.21.15.37:0/2074360477 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7febf4102e00 0x7febf41031d0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7febe4004940 tx=0x7febe4009e30 comp rx=0 tx=0).stop 2024-03-29T08:20:08.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:08.870+0000 7febf9c00700 1 -- 172.21.15.37:0/2074360477 shutdown_connections 2024-03-29T08:20:08.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:08.870+0000 7febf9c00700 1 --2- 172.21.15.37:0/2074360477 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7febf41037a0 0x7febf40fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:08.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:08.870+0000 7febf9c00700 1 --2- 172.21.15.37:0/2074360477 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7febf4102e00 0x7febf41031d0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:08.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:08.870+0000 7febf9c00700 1 -- 172.21.15.37:0/2074360477 >> 172.21.15.37:0/2074360477 conn(0x7febf40f89b0 msgr2=0x7febf40fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:20:08.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:08.870+0000 7febf9c00700 1 -- 172.21.15.37:0/2074360477 shutdown_connections 2024-03-29T08:20:08.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:08.870+0000 7febf9c00700 1 -- 172.21.15.37:0/2074360477 wait complete. 2024-03-29T08:20:08.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:08.871+0000 7febf9c00700 1 Processor -- start 2024-03-29T08:20:08.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:08.871+0000 7febf9c00700 1 -- start start 2024-03-29T08:20:08.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:08.871+0000 7febf9c00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7febf4102e00 0x7febf40702c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:08.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:08.872+0000 7febf9c00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7febf41037a0 0x7febf4070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:08.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:08.872+0000 7febf9c00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febf40720b0 con 0x7febf41037a0 2024-03-29T08:20:08.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:08.872+0000 7febf9c00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febf4072220 con 0x7febf4102e00 2024-03-29T08:20:08.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:08.872+0000 7febf2000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7febf41037a0 0x7febf4070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:08.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:08.872+0000 7febf2000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7febf41037a0 0x7febf4070800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37534/0 (socket says 172.21.15.37:37534) 2024-03-29T08:20:08.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:08.872+0000 7febf2000700 1 -- 172.21.15.37:0/2658902548 learned_addr learned my addr 172.21.15.37:0/2658902548 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:20:08.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:08.872+0000 7febf2a00700 1 --2- 172.21.15.37:0/2658902548 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7febf4102e00 0x7febf40702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:08.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:08.872+0000 7febf2000700 1 -- 172.21.15.37:0/2658902548 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7febf4102e00 msgr2=0x7febf40702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:08.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:08.872+0000 7febf2000700 1 --2- 172.21.15.37:0/2658902548 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7febf4102e00 0x7febf40702c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:08.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:08.872+0000 7febf2000700 1 -- 172.21.15.37:0/2658902548 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febe4005040 con 0x7febf41037a0 2024-03-29T08:20:08.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:08.872+0000 7febf2a00700 1 --2- 172.21.15.37:0/2658902548 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7febf4102e00 0x7febf40702c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:20:08.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:08.872+0000 7febf2000700 1 --2- 172.21.15.37:0/2658902548 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7febf41037a0 0x7febf4070800 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7febe0002a50 tx=0x7febe00046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:20:08.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:08.873+0000 7febeaa00700 1 -- 172.21.15.37:0/2658902548 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7febe0007cc0 con 0x7febf41037a0 2024-03-29T08:20:08.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:08.873+0000 7febeaa00700 1 -- 172.21.15.37:0/2658902548 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7febe000f040 con 0x7febf41037a0 2024-03-29T08:20:08.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:08.873+0000 7febeaa00700 1 -- 172.21.15.37:0/2658902548 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7febe00095d0 con 0x7febf41037a0 2024-03-29T08:20:08.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:08.873+0000 7febf9c00700 1 -- 172.21.15.37:0/2658902548 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febf4070e00 con 0x7febf41037a0 2024-03-29T08:20:08.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:08.873+0000 7febf9c00700 1 -- 172.21.15.37:0/2658902548 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febf411e6d0 con 0x7febf41037a0 2024-03-29T08:20:08.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:08.874+0000 7febf9c00700 1 -- 172.21.15.37:0/2658902548 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febf40fe550 con 0x7febf41037a0 2024-03-29T08:20:08.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:08.875+0000 7febeaa00700 1 -- 172.21.15.37:0/2658902548 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7febe0009770 con 0x7febf41037a0 2024-03-29T08:20:08.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:08.876+0000 7febeaa00700 1 --2- 172.21.15.37:0/2658902548 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7febd806cc30 0x7febd806f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:08.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:08.876+0000 7febeaa00700 1 -- 172.21.15.37:0/2658902548 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7febe008bcf0 con 0x7febf41037a0 2024-03-29T08:20:08.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:08.876+0000 7febf2a00700 1 --2- 172.21.15.37:0/2658902548 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7febd806cc30 0x7febd806f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:08.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:08.877+0000 7febf2a00700 1 --2- 172.21.15.37:0/2658902548 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7febd806cc30 0x7febd806f0e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7febe4000c00 tx=0x7febe4009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:20:08.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:08.880+0000 7febeaa00700 1 -- 172.21.15.37:0/2658902548 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7febe0055cb0 con 0x7febf41037a0 2024-03-29T08:20:09.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:20:08 smithi148 ceph-mon[106692]: from='client.34568 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:20:09.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:20:08 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/456768795' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:20:09.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:09.112+0000 7febf9c00700 1 -- 172.21.15.37:0/2658902548 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7febf404bed0 con 0x7febd806cc30 2024-03-29T08:20:09.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:09.114+0000 7febeaa00700 1 -- 172.21.15.37:0/2658902548 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7febf404bed0 con 0x7febd806cc30 2024-03-29T08:20:09.114 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-29T08:20:09.114 INFO:teuthology.orchestra.run.smithi037.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59", 2024-03-29T08:20:09.114 INFO:teuthology.orchestra.run.smithi037.stdout: "in_progress": true, 2024-03-29T08:20:09.114 INFO:teuthology.orchestra.run.smithi037.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-29T08:20:09.114 INFO:teuthology.orchestra.run.smithi037.stdout: "services_complete": [ 2024-03-29T08:20:09.114 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr", 2024-03-29T08:20:09.114 INFO:teuthology.orchestra.run.smithi037.stdout: "mon", 2024-03-29T08:20:09.114 INFO:teuthology.orchestra.run.smithi037.stdout: "crash", 2024-03-29T08:20:09.114 INFO:teuthology.orchestra.run.smithi037.stdout: "osd" 2024-03-29T08:20:09.115 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-29T08:20:09.115 INFO:teuthology.orchestra.run.smithi037.stdout: "progress": "15/21 daemons upgraded", 2024-03-29T08:20:09.115 INFO:teuthology.orchestra.run.smithi037.stdout: "message": "Currently upgrading mds daemons", 2024-03-29T08:20:09.115 INFO:teuthology.orchestra.run.smithi037.stdout: "is_paused": false 2024-03-29T08:20:09.115 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-29T08:20:09.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:09.116+0000 7febf9c00700 1 -- 172.21.15.37:0/2658902548 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7febd806cc30 msgr2=0x7febd806f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:09.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:09.116+0000 7febf9c00700 1 --2- 172.21.15.37:0/2658902548 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7febd806cc30 0x7febd806f0e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7febe4000c00 tx=0x7febe4009e30 comp rx=0 tx=0).stop 2024-03-29T08:20:09.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:09.116+0000 7febf9c00700 1 -- 172.21.15.37:0/2658902548 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7febf41037a0 msgr2=0x7febf4070800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:09.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:09.116+0000 7febf9c00700 1 --2- 172.21.15.37:0/2658902548 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7febf41037a0 0x7febf4070800 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7febe0002a50 tx=0x7febe00046b0 comp rx=0 tx=0).stop 2024-03-29T08:20:09.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:09.116+0000 7febf9c00700 1 -- 172.21.15.37:0/2658902548 shutdown_connections 2024-03-29T08:20:09.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:09.116+0000 7febf9c00700 1 --2- 172.21.15.37:0/2658902548 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7febd806cc30 0x7febd806f0e0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:09.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:09.116+0000 7febf9c00700 1 --2- 172.21.15.37:0/2658902548 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7febf4102e00 0x7febf40702c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:09.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:09.116+0000 7febf9c00700 1 --2- 172.21.15.37:0/2658902548 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7febf41037a0 0x7febf4070800 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:09.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:09.116+0000 7febf9c00700 1 -- 172.21.15.37:0/2658902548 >> 172.21.15.37:0/2658902548 conn(0x7febf40f89b0 msgr2=0x7febf40f99f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:20:09.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:09.116+0000 7febf9c00700 1 -- 172.21.15.37:0/2658902548 shutdown_connections 2024-03-29T08:20:09.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:09.116+0000 7febf9c00700 1 -- 172.21.15.37:0/2658902548 wait complete. 2024-03-29T08:20:09.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:20:08 smithi037 ceph-mon[142325]: from='client.34568 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:20:09.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:20:08 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/456768795' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:20:09.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:09.256+0000 7fb756c00700 1 -- 172.21.15.37:0/2338099384 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb750102e20 msgr2=0x7fb7501031f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:09.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:09.256+0000 7fb756c00700 1 --2- 172.21.15.37:0/2338099384 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb750102e20 0x7fb7501031f0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fb740004940 tx=0x7fb740009e30 comp rx=0 tx=0).stop 2024-03-29T08:20:09.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:09.256+0000 7fb756c00700 1 -- 172.21.15.37:0/2338099384 shutdown_connections 2024-03-29T08:20:09.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:09.256+0000 7fb756c00700 1 --2- 172.21.15.37:0/2338099384 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb7501037c0 0x7fb7500fce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:09.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:09.256+0000 7fb756c00700 1 --2- 172.21.15.37:0/2338099384 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb750102e20 0x7fb7501031f0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:09.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:09.256+0000 7fb756c00700 1 -- 172.21.15.37:0/2338099384 >> 172.21.15.37:0/2338099384 conn(0x7fb7500f89d0 msgr2=0x7fb7500fade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:20:09.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:09.256+0000 7fb756c00700 1 -- 172.21.15.37:0/2338099384 shutdown_connections 2024-03-29T08:20:09.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:09.256+0000 7fb756c00700 1 -- 172.21.15.37:0/2338099384 wait complete. 2024-03-29T08:20:09.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:09.257+0000 7fb756c00700 1 Processor -- start 2024-03-29T08:20:09.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:09.257+0000 7fb756c00700 1 -- start start 2024-03-29T08:20:09.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:09.258+0000 7fb756c00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb750102e20 0x7fb750115cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:09.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:09.258+0000 7fb756c00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb7501037c0 0x7fb750116230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:09.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:09.258+0000 7fb756c00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb750116880 con 0x7fb750102e20 2024-03-29T08:20:09.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:09.258+0000 7fb756c00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb75010fc50 con 0x7fb7501037c0 2024-03-29T08:20:09.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:09.258+0000 7fb74fe00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb750102e20 0x7fb750115cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:09.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:09.258+0000 7fb74fe00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb750102e20 0x7fb750115cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37546/0 (socket says 172.21.15.37:37546) 2024-03-29T08:20:09.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:09.258+0000 7fb74fe00700 1 -- 172.21.15.37:0/2275624376 learned_addr learned my addr 172.21.15.37:0/2275624376 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:20:09.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:09.258+0000 7fb747e00700 1 --2- 172.21.15.37:0/2275624376 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb7501037c0 0x7fb750116230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:09.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:09.258+0000 7fb74fe00700 1 -- 172.21.15.37:0/2275624376 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb7501037c0 msgr2=0x7fb750116230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:09.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:09.258+0000 7fb74fe00700 1 --2- 172.21.15.37:0/2275624376 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb7501037c0 0x7fb750116230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:09.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:09.258+0000 7fb74fe00700 1 -- 172.21.15.37:0/2275624376 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb740005040 con 0x7fb750102e20 2024-03-29T08:20:09.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:09.259+0000 7fb74fe00700 1 --2- 172.21.15.37:0/2275624376 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb750102e20 0x7fb750115cf0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fb740009360 tx=0x7fb740009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:20:09.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:09.259+0000 7fb74cc00700 1 -- 172.21.15.37:0/2275624376 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb74000e040 con 0x7fb750102e20 2024-03-29T08:20:09.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:09.259+0000 7fb74cc00700 1 -- 172.21.15.37:0/2275624376 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fb740013420 con 0x7fb750102e20 2024-03-29T08:20:09.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:09.259+0000 7fb74cc00700 1 -- 172.21.15.37:0/2275624376 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb74001c5f0 con 0x7fb750102e20 2024-03-29T08:20:09.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:09.259+0000 7fb756c00700 1 -- 172.21.15.37:0/2275624376 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb75010fed0 con 0x7fb750102e20 2024-03-29T08:20:09.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:09.259+0000 7fb756c00700 1 -- 172.21.15.37:0/2275624376 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb750110490 con 0x7fb750102e20 2024-03-29T08:20:09.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:09.260+0000 7fb756c00700 1 -- 172.21.15.37:0/2275624376 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7500fe570 con 0x7fb750102e20 2024-03-29T08:20:09.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:09.261+0000 7fb74cc00700 1 -- 172.21.15.37:0/2275624376 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fb740013590 con 0x7fb750102e20 2024-03-29T08:20:09.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:09.262+0000 7fb74cc00700 1 --2- 172.21.15.37:0/2275624376 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fb73406cc30 0x7fb73406f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:09.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:09.262+0000 7fb74cc00700 1 -- 172.21.15.37:0/2275624376 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fb74008d250 con 0x7fb750102e20 2024-03-29T08:20:09.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:09.262+0000 7fb747e00700 1 --2- 172.21.15.37:0/2275624376 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fb73406cc30 0x7fb73406f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:09.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:09.263+0000 7fb747e00700 1 --2- 172.21.15.37:0/2275624376 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fb73406cc30 0x7fb73406f0e0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fb738007fd0 tx=0x7fb738009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:20:09.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:09.267+0000 7fb74cc00700 1 -- 172.21.15.37:0/2275624376 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fb74005add0 con 0x7fb750102e20 2024-03-29T08:20:09.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:09.562+0000 7fb756c00700 1 -- 172.21.15.37:0/2275624376 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fb75005f5f0 con 0x7fb750102e20 2024-03-29T08:20:09.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:09.562+0000 7fb74cc00700 1 -- 172.21.15.37:0/2275624376 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7fb74005a960 con 0x7fb750102e20 2024-03-29T08:20:09.563 INFO:teuthology.orchestra.run.smithi037.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-03-29T08:20:09.563 INFO:teuthology.orchestra.run.smithi037.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-03-29T08:20:09.563 INFO:teuthology.orchestra.run.smithi037.stdout: fs cephfs is degraded 2024-03-29T08:20:09.563 INFO:teuthology.orchestra.run.smithi037.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-03-29T08:20:09.563 INFO:teuthology.orchestra.run.smithi037.stdout: fs cephfs is offline because no MDS is active for it. 2024-03-29T08:20:09.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:09.565+0000 7fb756c00700 1 -- 172.21.15.37:0/2275624376 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fb73406cc30 msgr2=0x7fb73406f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:09.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:09.565+0000 7fb756c00700 1 --2- 172.21.15.37:0/2275624376 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fb73406cc30 0x7fb73406f0e0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fb738007fd0 tx=0x7fb738009040 comp rx=0 tx=0).stop 2024-03-29T08:20:09.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:09.565+0000 7fb756c00700 1 -- 172.21.15.37:0/2275624376 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb750102e20 msgr2=0x7fb750115cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:09.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:09.565+0000 7fb756c00700 1 --2- 172.21.15.37:0/2275624376 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb750102e20 0x7fb750115cf0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fb740009360 tx=0x7fb740009e30 comp rx=0 tx=0).stop 2024-03-29T08:20:09.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:09.565+0000 7fb756c00700 1 -- 172.21.15.37:0/2275624376 shutdown_connections 2024-03-29T08:20:09.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:09.565+0000 7fb756c00700 1 --2- 172.21.15.37:0/2275624376 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fb73406cc30 0x7fb73406f0e0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:09.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:09.565+0000 7fb756c00700 1 --2- 172.21.15.37:0/2275624376 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb750102e20 0x7fb750115cf0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:09.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:09.565+0000 7fb756c00700 1 --2- 172.21.15.37:0/2275624376 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb7501037c0 0x7fb750116230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:09.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:09.565+0000 7fb756c00700 1 -- 172.21.15.37:0/2275624376 >> 172.21.15.37:0/2275624376 conn(0x7fb7500f89d0 msgr2=0x7fb7500f9ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:20:09.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:09.565+0000 7fb756c00700 1 -- 172.21.15.37:0/2275624376 shutdown_connections 2024-03-29T08:20:09.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:09.565+0000 7fb756c00700 1 -- 172.21.15.37:0/2275624376 wait complete. 2024-03-29T08:20:10.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:20:09 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/4205522908' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T08:20:10.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:20:09 smithi148 ceph-mon[106692]: from='client.34580 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:20:10.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:20:09 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/2275624376' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T08:20:10.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:20:09 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:20:10.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:20:09 smithi148 ceph-mon[106692]: pgmap v278: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:20:10.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:20:09 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/4205522908' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T08:20:10.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:20:09 smithi037 ceph-mon[142325]: from='client.34580 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:20:10.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:20:09 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/2275624376' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T08:20:10.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:20:09 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:20:10.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:20:09 smithi037 ceph-mon[142325]: pgmap v278: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:20:12.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:20:11 smithi148 ceph-mon[106692]: pgmap v279: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:20:12.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:20:11 smithi037 ceph-mon[142325]: pgmap v279: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:20:14.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:20:13 smithi148 ceph-mon[106692]: pgmap v280: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:20:14.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:20:13 smithi037 ceph-mon[142325]: pgmap v280: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:20:16.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:20:15 smithi148 ceph-mon[106692]: pgmap v281: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:20:16.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:20:15 smithi037 ceph-mon[142325]: pgmap v281: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:20:18.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:20:17 smithi148 ceph-mon[106692]: pgmap v282: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:20:18.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:20:17 smithi037 ceph-mon[142325]: pgmap v282: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:20:20.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:20:20 smithi037 ceph-mon[142325]: pgmap v283: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:20:20.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:20:20 smithi148 ceph-mon[106692]: pgmap v283: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:20:22.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:20:21 smithi148 ceph-mon[106692]: pgmap v284: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:20:22.178 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:20:21 smithi037 ceph-mon[142325]: pgmap v284: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:20:24.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:20:23 smithi148 ceph-mon[106692]: pgmap v285: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:20:24.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:20:23 smithi037 ceph-mon[142325]: pgmap v285: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:20:25.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:20:25 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:20:25.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:20:25 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:20:26.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:20:26 smithi037 ceph-mon[142325]: pgmap v286: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:20:26.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:20:26 smithi148 ceph-mon[106692]: pgmap v286: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:20:28.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:20:27 smithi148 ceph-mon[106692]: pgmap v287: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:20:28.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:20:27 smithi037 ceph-mon[142325]: pgmap v287: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:20:30.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:20:30 smithi037 ceph-mon[142325]: pgmap v288: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:20:30.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:20:30 smithi148 ceph-mon[106692]: pgmap v288: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:20:32.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:20:31 smithi148 ceph-mon[106692]: pgmap v289: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:20:32.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:20:31 smithi037 ceph-mon[142325]: pgmap v289: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:20:34.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:20:33 smithi148 ceph-mon[106692]: pgmap v290: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:20:34.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:20:33 smithi037 ceph-mon[142325]: pgmap v290: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:20:36.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:20:35 smithi148 ceph-mon[106692]: pgmap v291: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:20:36.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:20:35 smithi037 ceph-mon[142325]: pgmap v291: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:20:38.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:20:37 smithi148 ceph-mon[106692]: pgmap v292: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:20:38.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:20:37 smithi037 ceph-mon[142325]: pgmap v292: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:20:39.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:39.721+0000 7ff15a000700 1 -- 172.21.15.37:0/3300353345 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff154102e00 msgr2=0x7ff1541031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:39.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:39.721+0000 7ff15a000700 1 --2- 172.21.15.37:0/3300353345 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff154102e00 0x7ff1541031d0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7ff144004940 tx=0x7ff144009e30 comp rx=0 tx=0).stop 2024-03-29T08:20:39.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:39.721+0000 7ff15a000700 1 -- 172.21.15.37:0/3300353345 shutdown_connections 2024-03-29T08:20:39.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:39.721+0000 7ff15a000700 1 --2- 172.21.15.37:0/3300353345 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff1541037a0 0x7ff1540fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:39.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:39.721+0000 7ff15a000700 1 --2- 172.21.15.37:0/3300353345 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff154102e00 0x7ff1541031d0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:39.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:39.721+0000 7ff15a000700 1 -- 172.21.15.37:0/3300353345 >> 172.21.15.37:0/3300353345 conn(0x7ff1540f89b0 msgr2=0x7ff1540fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:20:39.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:39.721+0000 7ff15a000700 1 -- 172.21.15.37:0/3300353345 shutdown_connections 2024-03-29T08:20:39.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:39.721+0000 7ff15a000700 1 -- 172.21.15.37:0/3300353345 wait complete. 2024-03-29T08:20:39.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:39.722+0000 7ff15a000700 1 Processor -- start 2024-03-29T08:20:39.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:39.722+0000 7ff15a000700 1 -- start start 2024-03-29T08:20:39.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:39.722+0000 7ff15a000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff154102e00 0x7ff1540702c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:39.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:39.722+0000 7ff15a000700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff1541037a0 0x7ff154070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:39.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:39.722+0000 7ff15a000700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff154070e20 con 0x7ff154102e00 2024-03-29T08:20:39.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:39.722+0000 7ff15a000700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff154070f60 con 0x7ff1541037a0 2024-03-29T08:20:39.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:39.723+0000 7ff153400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff154102e00 0x7ff1540702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:39.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:39.723+0000 7ff153400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff154102e00 0x7ff1540702c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54380/0 (socket says 172.21.15.37:54380) 2024-03-29T08:20:39.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:39.723+0000 7ff153400700 1 -- 172.21.15.37:0/2206982138 learned_addr learned my addr 172.21.15.37:0/2206982138 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:20:39.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:39.723+0000 7ff152a00700 1 --2- 172.21.15.37:0/2206982138 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff1541037a0 0x7ff154070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:39.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:39.723+0000 7ff153400700 1 -- 172.21.15.37:0/2206982138 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff1541037a0 msgr2=0x7ff154070800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:39.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:39.723+0000 7ff153400700 1 --2- 172.21.15.37:0/2206982138 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff1541037a0 0x7ff154070800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:39.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:39.723+0000 7ff153400700 1 -- 172.21.15.37:0/2206982138 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff144005040 con 0x7ff154102e00 2024-03-29T08:20:39.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:39.723+0000 7ff152a00700 1 --2- 172.21.15.37:0/2206982138 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff1541037a0 0x7ff154070800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:20:39.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:39.724+0000 7ff153400700 1 --2- 172.21.15.37:0/2206982138 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff154102e00 0x7ff1540702c0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7ff144009390 tx=0x7ff144009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:20:39.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:39.724+0000 7ff14b400700 1 -- 172.21.15.37:0/2206982138 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff14400e040 con 0x7ff154102e00 2024-03-29T08:20:39.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:39.724+0000 7ff14b400700 1 -- 172.21.15.37:0/2206982138 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ff144013420 con 0x7ff154102e00 2024-03-29T08:20:39.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:39.724+0000 7ff14b400700 1 -- 172.21.15.37:0/2206982138 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff14401c630 con 0x7ff154102e00 2024-03-29T08:20:39.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:39.724+0000 7ff15a000700 1 -- 172.21.15.37:0/2206982138 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff154119da0 con 0x7ff154102e00 2024-03-29T08:20:39.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:39.724+0000 7ff15a000700 1 -- 172.21.15.37:0/2206982138 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff15411a220 con 0x7ff154102e00 2024-03-29T08:20:39.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:39.725+0000 7ff15a000700 1 -- 172.21.15.37:0/2206982138 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1540fe550 con 0x7ff154102e00 2024-03-29T08:20:39.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:39.726+0000 7ff14b400700 1 -- 172.21.15.37:0/2206982138 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7ff1440139e0 con 0x7ff154102e00 2024-03-29T08:20:39.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:39.727+0000 7ff14b400700 1 --2- 172.21.15.37:0/2206982138 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7ff13806cc30 0x7ff13806f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:39.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:39.727+0000 7ff14b400700 1 -- 172.21.15.37:0/2206982138 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7ff14408d320 con 0x7ff154102e00 2024-03-29T08:20:39.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:39.727+0000 7ff152a00700 1 --2- 172.21.15.37:0/2206982138 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7ff13806cc30 0x7ff13806f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:39.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:39.728+0000 7ff152a00700 1 --2- 172.21.15.37:0/2206982138 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7ff13806cc30 0x7ff13806f0e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7ff140006fd0 tx=0x7ff140008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:20:39.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:39.732+0000 7ff14b400700 1 -- 172.21.15.37:0/2206982138 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7ff144057390 con 0x7ff154102e00 2024-03-29T08:20:39.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:39.967+0000 7ff15a000700 1 -- 172.21.15.37:0/2206982138 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff1540719b0 con 0x7ff13806cc30 2024-03-29T08:20:39.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:39.969+0000 7ff14b400700 1 -- 172.21.15.37:0/2206982138 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7ff1540719b0 con 0x7ff13806cc30 2024-03-29T08:20:39.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:39.971+0000 7ff15a000700 1 -- 172.21.15.37:0/2206982138 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7ff13806cc30 msgr2=0x7ff13806f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:39.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:39.971+0000 7ff15a000700 1 --2- 172.21.15.37:0/2206982138 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7ff13806cc30 0x7ff13806f0e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7ff140006fd0 tx=0x7ff140008040 comp rx=0 tx=0).stop 2024-03-29T08:20:39.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:39.971+0000 7ff15a000700 1 -- 172.21.15.37:0/2206982138 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff154102e00 msgr2=0x7ff1540702c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:39.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:39.971+0000 7ff15a000700 1 --2- 172.21.15.37:0/2206982138 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff154102e00 0x7ff1540702c0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7ff144009390 tx=0x7ff144009e30 comp rx=0 tx=0).stop 2024-03-29T08:20:39.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:39.971+0000 7ff15a000700 1 -- 172.21.15.37:0/2206982138 shutdown_connections 2024-03-29T08:20:39.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:39.971+0000 7ff15a000700 1 --2- 172.21.15.37:0/2206982138 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7ff13806cc30 0x7ff13806f0e0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:39.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:39.971+0000 7ff15a000700 1 --2- 172.21.15.37:0/2206982138 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff154102e00 0x7ff1540702c0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:39.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:39.971+0000 7ff15a000700 1 --2- 172.21.15.37:0/2206982138 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff1541037a0 0x7ff154070800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:39.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:39.971+0000 7ff15a000700 1 -- 172.21.15.37:0/2206982138 >> 172.21.15.37:0/2206982138 conn(0x7ff1540f89b0 msgr2=0x7ff1540f9a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:20:39.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:39.971+0000 7ff15a000700 1 -- 172.21.15.37:0/2206982138 shutdown_connections 2024-03-29T08:20:39.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:39.971+0000 7ff15a000700 1 -- 172.21.15.37:0/2206982138 wait complete. 2024-03-29T08:20:39.986 INFO:teuthology.orchestra.run.smithi037.stdout:true 2024-03-29T08:20:40.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:40.109+0000 7f4f96c00700 1 -- 172.21.15.37:0/3355449077 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4f901035d0 msgr2=0x7f4f900fcc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:40.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:40.109+0000 7f4f96c00700 1 --2- 172.21.15.37:0/3355449077 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4f901035d0 0x7f4f900fcc40 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f4f7c004940 tx=0x7f4f7c009e30 comp rx=0 tx=0).stop 2024-03-29T08:20:40.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:40.109+0000 7f4f96c00700 1 -- 172.21.15.37:0/3355449077 shutdown_connections 2024-03-29T08:20:40.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:40.109+0000 7f4f96c00700 1 --2- 172.21.15.37:0/3355449077 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4f901035d0 0x7f4f900fcc40 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:40.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:40.109+0000 7f4f96c00700 1 --2- 172.21.15.37:0/3355449077 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4f90102c30 0x7f4f90103000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:40.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:40.109+0000 7f4f96c00700 1 -- 172.21.15.37:0/3355449077 >> 172.21.15.37:0/3355449077 conn(0x7f4f900f8760 msgr2=0x7f4f900fab70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:20:40.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:40.110+0000 7f4f96c00700 1 -- 172.21.15.37:0/3355449077 shutdown_connections 2024-03-29T08:20:40.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:40.110+0000 7f4f96c00700 1 -- 172.21.15.37:0/3355449077 wait complete. 2024-03-29T08:20:40.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:40.110+0000 7f4f96c00700 1 Processor -- start 2024-03-29T08:20:40.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:40.111+0000 7f4f96c00700 1 -- start start 2024-03-29T08:20:40.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:40.111+0000 7f4f96c00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4f90102c30 0x7f4f900702c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:40.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:40.111+0000 7f4f96c00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4f901035d0 0x7f4f90070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:40.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:40.111+0000 7f4f96c00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f90070d90 con 0x7f4f90102c30 2024-03-29T08:20:40.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:40.111+0000 7f4f96c00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f90070ed0 con 0x7f4f901035d0 2024-03-29T08:20:40.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:40.111+0000 7f4f95800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4f90102c30 0x7f4f900702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:40.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:40.111+0000 7f4f95800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4f90102c30 0x7f4f900702c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50100/0 (socket says 172.21.15.37:50100) 2024-03-29T08:20:40.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:40.111+0000 7f4f95800700 1 -- 172.21.15.37:0/1291780192 learned_addr learned my addr 172.21.15.37:0/1291780192 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:20:40.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:40.111+0000 7f4f94e00700 1 --2- 172.21.15.37:0/1291780192 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4f901035d0 0x7f4f90070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:40.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:40.112+0000 7f4f95800700 1 -- 172.21.15.37:0/1291780192 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4f901035d0 msgr2=0x7f4f90070800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:40.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:40.112+0000 7f4f95800700 1 --2- 172.21.15.37:0/1291780192 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4f901035d0 0x7f4f90070800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:40.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:40.112+0000 7f4f95800700 1 -- 172.21.15.37:0/1291780192 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f7c005040 con 0x7f4f90102c30 2024-03-29T08:20:40.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:40.112+0000 7f4f95800700 1 --2- 172.21.15.37:0/1291780192 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4f90102c30 0x7f4f900702c0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f4f8400eb40 tx=0x7f4f8400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:20:40.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:40.112+0000 7f4f8d600700 1 -- 172.21.15.37:0/1291780192 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4f84007cc0 con 0x7f4f90102c30 2024-03-29T08:20:40.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:40.112+0000 7f4f8d600700 1 -- 172.21.15.37:0/1291780192 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f4f8400f040 con 0x7f4f90102c30 2024-03-29T08:20:40.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:40.112+0000 7f4f96c00700 1 -- 172.21.15.37:0/1291780192 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f90071120 con 0x7f4f90102c30 2024-03-29T08:20:40.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:40.112+0000 7f4f8d600700 1 -- 172.21.15.37:0/1291780192 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4f8400c990 con 0x7f4f90102c30 2024-03-29T08:20:40.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:40.112+0000 7f4f96c00700 1 -- 172.21.15.37:0/1291780192 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f90119fb0 con 0x7f4f90102c30 2024-03-29T08:20:40.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:40.114+0000 7f4f96c00700 1 -- 172.21.15.37:0/1291780192 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f900fe380 con 0x7f4f90102c30 2024-03-29T08:20:40.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:40.114+0000 7f4f8d600700 1 -- 172.21.15.37:0/1291780192 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f4f8400cc00 con 0x7f4f90102c30 2024-03-29T08:20:40.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:40.115+0000 7f4f8d600700 1 --2- 172.21.15.37:0/1291780192 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f4f7806cc20 0x7f4f7806f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:40.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:40.115+0000 7f4f8d600700 1 -- 172.21.15.37:0/1291780192 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f4f8408bc50 con 0x7f4f90102c30 2024-03-29T08:20:40.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:40.116+0000 7f4f94e00700 1 --2- 172.21.15.37:0/1291780192 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f4f7806cc20 0x7f4f7806f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:40.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:40.116+0000 7f4f94e00700 1 --2- 172.21.15.37:0/1291780192 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f4f7806cc20 0x7f4f7806f0d0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f4f90071c00 tx=0x7f4f7c009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:20:40.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:40.120+0000 7f4f8d600700 1 -- 172.21.15.37:0/1291780192 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f4f84055cc0 con 0x7f4f90102c30 2024-03-29T08:20:40.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:40.353+0000 7f4f96c00700 1 -- 172.21.15.37:0/1291780192 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4f900719b0 con 0x7f4f7806cc20 2024-03-29T08:20:40.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:40.354+0000 7f4f8d600700 1 -- 172.21.15.37:0/1291780192 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f4f900719b0 con 0x7f4f7806cc20 2024-03-29T08:20:40.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:40.356+0000 7f4f96c00700 1 -- 172.21.15.37:0/1291780192 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f4f7806cc20 msgr2=0x7f4f7806f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:40.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:40.356+0000 7f4f96c00700 1 --2- 172.21.15.37:0/1291780192 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f4f7806cc20 0x7f4f7806f0d0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f4f90071c00 tx=0x7f4f7c009e30 comp rx=0 tx=0).stop 2024-03-29T08:20:40.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:40.356+0000 7f4f96c00700 1 -- 172.21.15.37:0/1291780192 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4f90102c30 msgr2=0x7f4f900702c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:40.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:40.356+0000 7f4f96c00700 1 --2- 172.21.15.37:0/1291780192 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4f90102c30 0x7f4f900702c0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f4f8400eb40 tx=0x7f4f8400b040 comp rx=0 tx=0).stop 2024-03-29T08:20:40.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:40.357+0000 7f4f96c00700 1 -- 172.21.15.37:0/1291780192 shutdown_connections 2024-03-29T08:20:40.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:40.357+0000 7f4f96c00700 1 --2- 172.21.15.37:0/1291780192 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f4f7806cc20 0x7f4f7806f0d0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:40.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:40.357+0000 7f4f96c00700 1 --2- 172.21.15.37:0/1291780192 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4f90102c30 0x7f4f900702c0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:40.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:40.357+0000 7f4f96c00700 1 --2- 172.21.15.37:0/1291780192 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4f901035d0 0x7f4f90070800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:40.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:40.357+0000 7f4f96c00700 1 -- 172.21.15.37:0/1291780192 >> 172.21.15.37:0/1291780192 conn(0x7f4f900f8760 msgr2=0x7f4f900f9790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:20:40.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:40.357+0000 7f4f96c00700 1 -- 172.21.15.37:0/1291780192 shutdown_connections 2024-03-29T08:20:40.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:40.357+0000 7f4f96c00700 1 -- 172.21.15.37:0/1291780192 wait complete. 2024-03-29T08:20:40.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:20:40 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:20:40.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:20:40 smithi037 ceph-mon[142325]: pgmap v293: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:20:40.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:40.493+0000 7f64b6000700 1 -- 172.21.15.37:0/935328686 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f64b0070c70 msgr2=0x7f64b0109110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:40.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:40.493+0000 7f64b6000700 1 --2- 172.21.15.37:0/935328686 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f64b0070c70 0x7f64b0109110 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f649c004970 tx=0x7f649c009e30 comp rx=0 tx=0).stop 2024-03-29T08:20:40.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:40.493+0000 7f64b6000700 1 -- 172.21.15.37:0/935328686 shutdown_connections 2024-03-29T08:20:40.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:40.493+0000 7f64b6000700 1 --2- 172.21.15.37:0/935328686 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f64b0070c70 0x7f64b0109110 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:40.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:40.493+0000 7f64b6000700 1 --2- 172.21.15.37:0/935328686 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f64b0070360 0x7f64b0070730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:40.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:40.493+0000 7f64b6000700 1 -- 172.21.15.37:0/935328686 >> 172.21.15.37:0/935328686 conn(0x7f64b00f89b0 msgr2=0x7f64b00fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:20:40.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:40.494+0000 7f64b6000700 1 -- 172.21.15.37:0/935328686 shutdown_connections 2024-03-29T08:20:40.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:40.494+0000 7f64b6000700 1 -- 172.21.15.37:0/935328686 wait complete. 2024-03-29T08:20:40.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:40.494+0000 7f64b6000700 1 Processor -- start 2024-03-29T08:20:40.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:40.494+0000 7f64b6000700 1 -- start start 2024-03-29T08:20:40.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:40.495+0000 7f64b6000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f64b0070360 0x7f64b0115ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:40.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:40.495+0000 7f64b6000700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f64b0070c70 0x7f64b01161e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:40.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:40.495+0000 7f64b6000700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64b0116830 con 0x7f64b0070360 2024-03-29T08:20:40.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:40.495+0000 7f64b6000700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64b010fc00 con 0x7f64b0070c70 2024-03-29T08:20:40.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:40.495+0000 7f64af400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f64b0070360 0x7f64b0115ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:40.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:40.495+0000 7f64af400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f64b0070360 0x7f64b0115ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50112/0 (socket says 172.21.15.37:50112) 2024-03-29T08:20:40.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:40.495+0000 7f64af400700 1 -- 172.21.15.37:0/1261075080 learned_addr learned my addr 172.21.15.37:0/1261075080 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:20:40.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:40.495+0000 7f64aea00700 1 --2- 172.21.15.37:0/1261075080 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f64b0070c70 0x7f64b01161e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:40.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:40.496+0000 7f64af400700 1 -- 172.21.15.37:0/1261075080 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f64b0070c70 msgr2=0x7f64b01161e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:40.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:40.496+0000 7f64af400700 1 --2- 172.21.15.37:0/1261075080 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f64b0070c70 0x7f64b01161e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:40.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:40.496+0000 7f64af400700 1 -- 172.21.15.37:0/1261075080 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f649c005040 con 0x7f64b0070360 2024-03-29T08:20:40.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:40.496+0000 7f64af400700 1 --2- 172.21.15.37:0/1261075080 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f64b0070360 0x7f64b0115ca0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f64a000c8c0 tx=0x7f64a00044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:20:40.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:40.496+0000 7f64a7400700 1 -- 172.21.15.37:0/1261075080 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f64a000a760 con 0x7f64b0070360 2024-03-29T08:20:40.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:40.496+0000 7f64a7400700 1 -- 172.21.15.37:0/1261075080 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f64a000ad70 con 0x7f64b0070360 2024-03-29T08:20:40.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:40.496+0000 7f64b6000700 1 -- 172.21.15.37:0/1261075080 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64b010fee0 con 0x7f64b0070360 2024-03-29T08:20:40.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:40.496+0000 7f64a7400700 1 -- 172.21.15.37:0/1261075080 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f64a00113a0 con 0x7f64b0070360 2024-03-29T08:20:40.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:40.496+0000 7f64b6000700 1 -- 172.21.15.37:0/1261075080 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64b01103f0 con 0x7f64b0070360 2024-03-29T08:20:40.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:40.497+0000 7f64b6000700 1 -- 172.21.15.37:0/1261075080 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64b0103bd0 con 0x7f64b0070360 2024-03-29T08:20:40.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:40.498+0000 7f64a7400700 1 -- 172.21.15.37:0/1261075080 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f64a0012400 con 0x7f64b0070360 2024-03-29T08:20:40.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:40.499+0000 7f64a7400700 1 --2- 172.21.15.37:0/1261075080 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f649406cc30 0x7f649406f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:40.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:40.499+0000 7f64a7400700 1 -- 172.21.15.37:0/1261075080 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f64a008bfe0 con 0x7f64b0070360 2024-03-29T08:20:40.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:40.500+0000 7f64aea00700 1 --2- 172.21.15.37:0/1261075080 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f649406cc30 0x7f649406f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:40.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:40.500+0000 7f64aea00700 1 --2- 172.21.15.37:0/1261075080 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f649406cc30 0x7f649406f0e0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f649c009fd0 tx=0x7f649c009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:20:40.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:40.504+0000 7f64a7400700 1 -- 172.21.15.37:0/1261075080 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f64a0059af0 con 0x7f64b0070360 2024-03-29T08:20:40.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:20:40 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:20:40.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:20:40 smithi148 ceph-mon[106692]: pgmap v293: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:20:40.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:40.725+0000 7f64b6000700 1 -- 172.21.15.37:0/1261075080 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f64b010bfb0 con 0x7f649406cc30 2024-03-29T08:20:40.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:40.736+0000 7f64a7400700 1 -- 172.21.15.37:0/1261075080 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f64b010bfb0 con 0x7f649406cc30 2024-03-29T08:20:40.736 INFO:teuthology.orchestra.run.smithi037.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-29T08:20:40.736 INFO:teuthology.orchestra.run.smithi037.stdout:alertmanager.smithi037 smithi037 *:9093,9094 running (10m) 2m ago 23m 14.9M - 0.25.0 c8568f914cd2 2f9f9746ed0f 2024-03-29T08:20:40.736 INFO:teuthology.orchestra.run.smithi037.stdout:crash.smithi037 smithi037 running (7m) 2m ago 23m 6647k - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 178d416a3186 2024-03-29T08:20:40.736 INFO:teuthology.orchestra.run.smithi037.stdout:crash.smithi148 smithi148 running (7m) 2m ago 22m 6656k - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 23666989d58f 2024-03-29T08:20:40.736 INFO:teuthology.orchestra.run.smithi037.stdout:grafana.smithi037 smithi037 *:3000 running (10m) 2m ago 23m 71.9M - 9.4.7 954c08fa6188 6b976fe49952 2024-03-29T08:20:40.736 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi037.iifsmh smithi037 running (2m) 2m ago 16m 10.8M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 2d97f562cc5b 2024-03-29T08:20:40.736 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi037.mmwgxe smithi037 running (3m) 2m ago 17m 10.9M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff ec37037f46e8 2024-03-29T08:20:40.736 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi148.qukpjh smithi148 running (16m) 2m ago 16m 15.3M - 17.2.7-789-g4213e17c cf22eb8683be 8e469782a273 2024-03-29T08:20:40.736 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi148.sqteih smithi148 running (2m) 2m ago 17m 10.7M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 44c15c77037a 2024-03-29T08:20:40.736 INFO:teuthology.orchestra.run.smithi037.stdout:mgr.smithi037.wjikii smithi037 *:8443,9283,8765 running (12m) 2m ago 25m 506M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff c5a1417b9417 2024-03-29T08:20:40.737 INFO:teuthology.orchestra.run.smithi037.stdout:mgr.smithi148.onhgvm smithi148 *:8443,9283,8765 running (11m) 2m ago 22m 450M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff fcfdbce062bd 2024-03-29T08:20:40.737 INFO:teuthology.orchestra.run.smithi037.stdout:mon.smithi037 smithi037 running (8m) 2m ago 25m 54.8M 2048M 18.2.2-1015-gd0e392d9 1cc2f45c17ff eea2060783af 2024-03-29T08:20:40.737 INFO:teuthology.orchestra.run.smithi037.stdout:mon.smithi148 smithi148 running (8m) 2m ago 22m 43.7M 2048M 18.2.2-1015-gd0e392d9 1cc2f45c17ff ad4b2c5748be 2024-03-29T08:20:40.737 INFO:teuthology.orchestra.run.smithi037.stdout:node-exporter.smithi037 smithi037 *:9100 running (11m) 2m ago 23m 7625k - 1.5.0 0da6a335fe13 26b8ae9cb878 2024-03-29T08:20:40.737 INFO:teuthology.orchestra.run.smithi037.stdout:node-exporter.smithi148 smithi148 *:9100 running (11m) 2m ago 22m 8166k - 1.5.0 0da6a335fe13 98414e4e64ed 2024-03-29T08:20:40.737 INFO:teuthology.orchestra.run.smithi037.stdout:osd.0 smithi037 running (6m) 2m ago 21m 104M 1619M 18.2.2-1015-gd0e392d9 1cc2f45c17ff c0c584f6245c 2024-03-29T08:20:40.737 INFO:teuthology.orchestra.run.smithi037.stdout:osd.1 smithi037 running (6m) 2m ago 20m 100M 1619M 18.2.2-1015-gd0e392d9 1cc2f45c17ff a4a2ff4cc83d 2024-03-29T08:20:40.737 INFO:teuthology.orchestra.run.smithi037.stdout:osd.2 smithi037 running (5m) 2m ago 20m 82.3M 1619M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 8b1bc71aa79e 2024-03-29T08:20:40.737 INFO:teuthology.orchestra.run.smithi037.stdout:osd.3 smithi148 running (4m) 2m ago 19m 105M 2642M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 1c656ab8ff86 2024-03-29T08:20:40.737 INFO:teuthology.orchestra.run.smithi037.stdout:osd.4 smithi148 running (4m) 2m ago 19m 97.3M 2642M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 966a9de11d22 2024-03-29T08:20:40.737 INFO:teuthology.orchestra.run.smithi037.stdout:osd.5 smithi148 running (3m) 2m ago 18m 85.8M 2642M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 474c4f9212ce 2024-03-29T08:20:40.737 INFO:teuthology.orchestra.run.smithi037.stdout:prometheus.smithi037 smithi037 *:9095 running (10m) 2m ago 22m 54.4M - 2.43.0 a07b618ecd1d 30c0223ec589 2024-03-29T08:20:40.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:40.738+0000 7f64b6000700 1 -- 172.21.15.37:0/1261075080 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f649406cc30 msgr2=0x7f649406f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:40.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:40.738+0000 7f64b6000700 1 --2- 172.21.15.37:0/1261075080 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f649406cc30 0x7f649406f0e0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f649c009fd0 tx=0x7f649c009e30 comp rx=0 tx=0).stop 2024-03-29T08:20:40.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:40.738+0000 7f64b6000700 1 -- 172.21.15.37:0/1261075080 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f64b0070360 msgr2=0x7f64b0115ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:40.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:40.738+0000 7f64b6000700 1 --2- 172.21.15.37:0/1261075080 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f64b0070360 0x7f64b0115ca0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f64a000c8c0 tx=0x7f64a00044c0 comp rx=0 tx=0).stop 2024-03-29T08:20:40.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:40.738+0000 7f64b6000700 1 -- 172.21.15.37:0/1261075080 shutdown_connections 2024-03-29T08:20:40.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:40.738+0000 7f64b6000700 1 --2- 172.21.15.37:0/1261075080 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f649406cc30 0x7f649406f0e0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:40.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:40.738+0000 7f64b6000700 1 --2- 172.21.15.37:0/1261075080 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f64b0070360 0x7f64b0115ca0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:40.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:40.738+0000 7f64b6000700 1 --2- 172.21.15.37:0/1261075080 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f64b0070c70 0x7f64b01161e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:40.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:40.738+0000 7f64b6000700 1 -- 172.21.15.37:0/1261075080 >> 172.21.15.37:0/1261075080 conn(0x7f64b00f89b0 msgr2=0x7f64b01015f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:20:40.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:40.739+0000 7f64b6000700 1 -- 172.21.15.37:0/1261075080 shutdown_connections 2024-03-29T08:20:40.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:40.739+0000 7f64b6000700 1 -- 172.21.15.37:0/1261075080 wait complete. 2024-03-29T08:20:40.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:40.875+0000 7f9390600700 1 -- 172.21.15.37:0/3623699061 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f93881037c0 msgr2=0x7f93880fce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:40.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:40.875+0000 7f9390600700 1 --2- 172.21.15.37:0/3623699061 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f93881037c0 0x7f93880fce30 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f9374004940 tx=0x7f9374009e30 comp rx=0 tx=0).stop 2024-03-29T08:20:40.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:40.876+0000 7f9390600700 1 -- 172.21.15.37:0/3623699061 shutdown_connections 2024-03-29T08:20:40.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:40.876+0000 7f9390600700 1 --2- 172.21.15.37:0/3623699061 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f93881037c0 0x7f93880fce30 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:40.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:40.876+0000 7f9390600700 1 --2- 172.21.15.37:0/3623699061 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9388102e20 0x7f93881031f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:40.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:40.876+0000 7f9390600700 1 -- 172.21.15.37:0/3623699061 >> 172.21.15.37:0/3623699061 conn(0x7f93880f8970 msgr2=0x7f93880fad80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:20:40.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:40.876+0000 7f9390600700 1 -- 172.21.15.37:0/3623699061 shutdown_connections 2024-03-29T08:20:40.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:40.876+0000 7f9390600700 1 -- 172.21.15.37:0/3623699061 wait complete. 2024-03-29T08:20:40.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:40.876+0000 7f9390600700 1 Processor -- start 2024-03-29T08:20:40.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:40.877+0000 7f9390600700 1 -- start start 2024-03-29T08:20:40.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:40.877+0000 7f9390600700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9388102e20 0x7f93880702c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:40.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:40.877+0000 7f9390600700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f93881037c0 0x7f9388070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:40.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:40.877+0000 7f9390600700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9388070e20 con 0x7f9388102e20 2024-03-29T08:20:40.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:40.877+0000 7f9390600700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9388070f60 con 0x7f93881037c0 2024-03-29T08:20:40.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:40.877+0000 7f938dc00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9388102e20 0x7f93880702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:40.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:40.877+0000 7f938dc00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9388102e20 0x7f93880702c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50132/0 (socket says 172.21.15.37:50132) 2024-03-29T08:20:40.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:40.877+0000 7f938dc00700 1 -- 172.21.15.37:0/2605081963 learned_addr learned my addr 172.21.15.37:0/2605081963 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:20:40.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:40.877+0000 7f938d200700 1 --2- 172.21.15.37:0/2605081963 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f93881037c0 0x7f9388070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:40.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:40.878+0000 7f938dc00700 1 -- 172.21.15.37:0/2605081963 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f93881037c0 msgr2=0x7f9388070800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:40.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:40.878+0000 7f938dc00700 1 --2- 172.21.15.37:0/2605081963 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f93881037c0 0x7f9388070800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:40.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:40.878+0000 7f938dc00700 1 -- 172.21.15.37:0/2605081963 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9374005040 con 0x7f9388102e20 2024-03-29T08:20:40.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:40.878+0000 7f938d200700 1 --2- 172.21.15.37:0/2605081963 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f93881037c0 0x7f9388070800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:20:40.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:40.878+0000 7f938dc00700 1 --2- 172.21.15.37:0/2605081963 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9388102e20 0x7f93880702c0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f937c00b790 tx=0x7f937c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:20:40.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:40.878+0000 7f9385600700 1 -- 172.21.15.37:0/2605081963 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f937c00f770 con 0x7f9388102e20 2024-03-29T08:20:40.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:40.878+0000 7f9385600700 1 -- 172.21.15.37:0/2605081963 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f937c00fd80 con 0x7f9388102e20 2024-03-29T08:20:40.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:40.878+0000 7f9390600700 1 -- 172.21.15.37:0/2605081963 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9388119db0 con 0x7f9388102e20 2024-03-29T08:20:40.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:40.878+0000 7f9385600700 1 -- 172.21.15.37:0/2605081963 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f937c00d310 con 0x7f9388102e20 2024-03-29T08:20:40.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:40.878+0000 7f9390600700 1 -- 172.21.15.37:0/2605081963 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f938811a2c0 con 0x7f9388102e20 2024-03-29T08:20:40.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:40.879+0000 7f9390600700 1 -- 172.21.15.37:0/2605081963 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93880fe570 con 0x7f9388102e20 2024-03-29T08:20:40.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:40.880+0000 7f9385600700 1 -- 172.21.15.37:0/2605081963 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f937c010420 con 0x7f9388102e20 2024-03-29T08:20:40.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:40.882+0000 7f9385600700 1 --2- 172.21.15.37:0/2605081963 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f937006cc30 0x7f937006f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:40.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:40.882+0000 7f9385600700 1 -- 172.21.15.37:0/2605081963 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f937c08bfe0 con 0x7f9388102e20 2024-03-29T08:20:40.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:40.882+0000 7f938d200700 1 --2- 172.21.15.37:0/2605081963 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f937006cc30 0x7f937006f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:40.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:40.883+0000 7f938d200700 1 --2- 172.21.15.37:0/2605081963 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f937006cc30 0x7f937006f0e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f9388071c00 tx=0x7f9374009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:20:40.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:40.886+0000 7f9385600700 1 -- 172.21.15.37:0/2605081963 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f937c0bf9f0 con 0x7f9388102e20 2024-03-29T08:20:41.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:41.188+0000 7f9390600700 1 -- 172.21.15.37:0/2605081963 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f93880719b0 con 0x7f9388102e20 2024-03-29T08:20:41.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:41.188+0000 7f9385600700 1 -- 172.21.15.37:0/2605081963 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+847 (secure 0 0 0) 0x7f937c056020 con 0x7f9388102e20 2024-03-29T08:20:41.189 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-29T08:20:41.189 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": { 2024-03-29T08:20:41.189 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 2 2024-03-29T08:20:41.189 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:20:41.189 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": { 2024-03-29T08:20:41.189 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 2 2024-03-29T08:20:41.189 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:20:41.189 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": { 2024-03-29T08:20:41.189 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 6 2024-03-29T08:20:41.189 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:20:41.189 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": { 2024-03-29T08:20:41.189 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 17.2.7-789-g4213e17c (4213e17c4de67016ed14f55d22503f4923b3a901) quincy (stable)": 1, 2024-03-29T08:20:41.189 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 3 2024-03-29T08:20:41.189 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:20:41.190 INFO:teuthology.orchestra.run.smithi037.stdout: "overall": { 2024-03-29T08:20:41.190 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 17.2.7-789-g4213e17c (4213e17c4de67016ed14f55d22503f4923b3a901) quincy (stable)": 1, 2024-03-29T08:20:41.190 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 13 2024-03-29T08:20:41.190 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-29T08:20:41.190 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-29T08:20:41.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:41.191+0000 7f9390600700 1 -- 172.21.15.37:0/2605081963 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f937006cc30 msgr2=0x7f937006f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:41.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:41.191+0000 7f9390600700 1 --2- 172.21.15.37:0/2605081963 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f937006cc30 0x7f937006f0e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f9388071c00 tx=0x7f9374009e30 comp rx=0 tx=0).stop 2024-03-29T08:20:41.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:41.191+0000 7f9390600700 1 -- 172.21.15.37:0/2605081963 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9388102e20 msgr2=0x7f93880702c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:41.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:41.191+0000 7f9390600700 1 --2- 172.21.15.37:0/2605081963 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9388102e20 0x7f93880702c0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f937c00b790 tx=0x7f937c009250 comp rx=0 tx=0).stop 2024-03-29T08:20:41.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:41.191+0000 7f9390600700 1 -- 172.21.15.37:0/2605081963 shutdown_connections 2024-03-29T08:20:41.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:41.191+0000 7f9390600700 1 --2- 172.21.15.37:0/2605081963 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f937006cc30 0x7f937006f0e0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:41.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:41.191+0000 7f9390600700 1 --2- 172.21.15.37:0/2605081963 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9388102e20 0x7f93880702c0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:41.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:41.191+0000 7f9390600700 1 --2- 172.21.15.37:0/2605081963 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f93881037c0 0x7f9388070800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:41.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:41.191+0000 7f9390600700 1 -- 172.21.15.37:0/2605081963 >> 172.21.15.37:0/2605081963 conn(0x7f93880f8970 msgr2=0x7f93880f9990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:20:41.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:41.191+0000 7f9390600700 1 -- 172.21.15.37:0/2605081963 shutdown_connections 2024-03-29T08:20:41.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:41.191+0000 7f9390600700 1 -- 172.21.15.37:0/2605081963 wait complete. 2024-03-29T08:20:41.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:41.329+0000 7f4617400700 1 -- 172.21.15.37:0/3337735573 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4610102e00 msgr2=0x7f46101031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:41.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:41.329+0000 7f4617400700 1 --2- 172.21.15.37:0/3337735573 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4610102e00 0x7f46101031d0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f4604004940 tx=0x7f4604009e30 comp rx=0 tx=0).stop 2024-03-29T08:20:41.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:41.329+0000 7f4617400700 1 -- 172.21.15.37:0/3337735573 shutdown_connections 2024-03-29T08:20:41.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:41.329+0000 7f4617400700 1 --2- 172.21.15.37:0/3337735573 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f46101037a0 0x7f46100fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:41.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:41.329+0000 7f4617400700 1 --2- 172.21.15.37:0/3337735573 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4610102e00 0x7f46101031d0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:41.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:41.329+0000 7f4617400700 1 -- 172.21.15.37:0/3337735573 >> 172.21.15.37:0/3337735573 conn(0x7f46100f89b0 msgr2=0x7f46100fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:20:41.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:41.329+0000 7f4617400700 1 -- 172.21.15.37:0/3337735573 shutdown_connections 2024-03-29T08:20:41.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:41.329+0000 7f4617400700 1 -- 172.21.15.37:0/3337735573 wait complete. 2024-03-29T08:20:41.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:41.330+0000 7f4617400700 1 Processor -- start 2024-03-29T08:20:41.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:41.331+0000 7f4617400700 1 -- start start 2024-03-29T08:20:41.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:41.331+0000 7f4617400700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4610102e00 0x7f4610115cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:41.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:41.331+0000 7f4617400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f46101037a0 0x7f4610116210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:41.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:41.331+0000 7f4617400700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4610116860 con 0x7f46101037a0 2024-03-29T08:20:41.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:41.331+0000 7f4617400700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f461010fc30 con 0x7f4610102e00 2024-03-29T08:20:41.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:41.331+0000 7f460fe00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f46101037a0 0x7f4610116210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:41.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:41.332+0000 7f460fe00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f46101037a0 0x7f4610116210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50140/0 (socket says 172.21.15.37:50140) 2024-03-29T08:20:41.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:41.332+0000 7f460fe00700 1 -- 172.21.15.37:0/962126602 learned_addr learned my addr 172.21.15.37:0/962126602 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:20:41.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:41.332+0000 7f4614a00700 1 --2- 172.21.15.37:0/962126602 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4610102e00 0x7f4610115cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:41.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:41.332+0000 7f460fe00700 1 -- 172.21.15.37:0/962126602 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4610102e00 msgr2=0x7f4610115cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:41.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:41.332+0000 7f460fe00700 1 --2- 172.21.15.37:0/962126602 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4610102e00 0x7f4610115cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:41.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:41.332+0000 7f460fe00700 1 -- 172.21.15.37:0/962126602 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4604005040 con 0x7f46101037a0 2024-03-29T08:20:41.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:41.332+0000 7f460fe00700 1 --2- 172.21.15.37:0/962126602 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f46101037a0 0x7f4610116210 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f45fc00c800 tx=0x7f45fc004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:20:41.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:41.332+0000 7f460cc00700 1 -- 172.21.15.37:0/962126602 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f45fc0106f0 con 0x7f46101037a0 2024-03-29T08:20:41.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:41.332+0000 7f460cc00700 1 -- 172.21.15.37:0/962126602 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f45fc010d00 con 0x7f46101037a0 2024-03-29T08:20:41.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:41.332+0000 7f460cc00700 1 -- 172.21.15.37:0/962126602 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f45fc0113a0 con 0x7f46101037a0 2024-03-29T08:20:41.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:41.333+0000 7f4617400700 1 -- 172.21.15.37:0/962126602 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f461010ff10 con 0x7f46101037a0 2024-03-29T08:20:41.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:41.333+0000 7f4617400700 1 -- 172.21.15.37:0/962126602 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4610110480 con 0x7f46101037a0 2024-03-29T08:20:41.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:41.334+0000 7f4617400700 1 -- 172.21.15.37:0/962126602 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46100fe550 con 0x7f46101037a0 2024-03-29T08:20:41.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:41.335+0000 7f460cc00700 1 -- 172.21.15.37:0/962126602 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f45fc012400 con 0x7f46101037a0 2024-03-29T08:20:41.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:41.336+0000 7f460cc00700 1 --2- 172.21.15.37:0/962126602 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f45f806cc30 0x7f45f806f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:41.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:41.336+0000 7f460cc00700 1 -- 172.21.15.37:0/962126602 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f45fc08c050 con 0x7f46101037a0 2024-03-29T08:20:41.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:41.336+0000 7f4614a00700 1 --2- 172.21.15.37:0/962126602 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f45f806cc30 0x7f45f806f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:41.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:41.337+0000 7f4614a00700 1 --2- 172.21.15.37:0/962126602 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f45f806cc30 0x7f45f806f0e0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f4604009fd0 tx=0x7f4604009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:20:41.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:41.340+0000 7f460cc00700 1 -- 172.21.15.37:0/962126602 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f45fc059aa0 con 0x7f46101037a0 2024-03-29T08:20:41.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:20:41 smithi037 ceph-mon[142325]: from='client.34588 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:20:41.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:20:41 smithi037 ceph-mon[142325]: from='client.34592 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:20:41.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:20:41 smithi148 ceph-mon[106692]: from='client.34588 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:20:41.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:20:41 smithi148 ceph-mon[106692]: from='client.34592 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:20:41.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:41.600+0000 7f4617400700 1 -- 172.21.15.37:0/962126602 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f461004bed0 con 0x7f46101037a0 2024-03-29T08:20:41.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:41.600+0000 7f460cc00700 1 -- 172.21.15.37:0/962126602 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 28 v28) v1 ==== 76+0+1841 (secure 0 0 0) 0x7f45fc059630 con 0x7f46101037a0 2024-03-29T08:20:41.600 INFO:teuthology.orchestra.run.smithi037.stdout:e28 2024-03-29T08:20:41.601 INFO:teuthology.orchestra.run.smithi037.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-29T08:20:41.601 INFO:teuthology.orchestra.run.smithi037.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T08:20:41.601 INFO:teuthology.orchestra.run.smithi037.stdout:legacy client fscid: 1 2024-03-29T08:20:41.601 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:20:41.601 INFO:teuthology.orchestra.run.smithi037.stdout:Filesystem 'cephfs' (1) 2024-03-29T08:20:41.601 INFO:teuthology.orchestra.run.smithi037.stdout:fs_name cephfs 2024-03-29T08:20:41.601 INFO:teuthology.orchestra.run.smithi037.stdout:epoch 21 2024-03-29T08:20:41.601 INFO:teuthology.orchestra.run.smithi037.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-03-29T08:20:41.601 INFO:teuthology.orchestra.run.smithi037.stdout:created 2024-03-29T08:03:29.241269+0000 2024-03-29T08:20:41.601 INFO:teuthology.orchestra.run.smithi037.stdout:modified 2024-03-29T08:17:16.237238+0000 2024-03-29T08:20:41.601 INFO:teuthology.orchestra.run.smithi037.stdout:tableserver 0 2024-03-29T08:20:41.601 INFO:teuthology.orchestra.run.smithi037.stdout:root 0 2024-03-29T08:20:41.601 INFO:teuthology.orchestra.run.smithi037.stdout:session_timeout 60 2024-03-29T08:20:41.601 INFO:teuthology.orchestra.run.smithi037.stdout:session_autoclose 300 2024-03-29T08:20:41.601 INFO:teuthology.orchestra.run.smithi037.stdout:max_file_size 1099511627776 2024-03-29T08:20:41.602 INFO:teuthology.orchestra.run.smithi037.stdout:max_xattr_size 65536 2024-03-29T08:20:41.602 INFO:teuthology.orchestra.run.smithi037.stdout:required_client_features {} 2024-03-29T08:20:41.602 INFO:teuthology.orchestra.run.smithi037.stdout:last_failure 0 2024-03-29T08:20:41.602 INFO:teuthology.orchestra.run.smithi037.stdout:last_failure_osd_epoch 82 2024-03-29T08:20:41.602 INFO:teuthology.orchestra.run.smithi037.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T08:20:41.602 INFO:teuthology.orchestra.run.smithi037.stdout:max_mds 2 2024-03-29T08:20:41.602 INFO:teuthology.orchestra.run.smithi037.stdout:in 0,1 2024-03-29T08:20:41.602 INFO:teuthology.orchestra.run.smithi037.stdout:up {} 2024-03-29T08:20:41.602 INFO:teuthology.orchestra.run.smithi037.stdout:failed 0,1 2024-03-29T08:20:41.602 INFO:teuthology.orchestra.run.smithi037.stdout:damaged 2024-03-29T08:20:41.602 INFO:teuthology.orchestra.run.smithi037.stdout:stopped 2024-03-29T08:20:41.602 INFO:teuthology.orchestra.run.smithi037.stdout:data_pools [3] 2024-03-29T08:20:41.602 INFO:teuthology.orchestra.run.smithi037.stdout:metadata_pool 2 2024-03-29T08:20:41.602 INFO:teuthology.orchestra.run.smithi037.stdout:inline_data disabled 2024-03-29T08:20:41.603 INFO:teuthology.orchestra.run.smithi037.stdout:balancer 2024-03-29T08:20:41.603 INFO:teuthology.orchestra.run.smithi037.stdout:bal_rank_mask -1 2024-03-29T08:20:41.603 INFO:teuthology.orchestra.run.smithi037.stdout:standby_count_wanted 1 2024-03-29T08:20:41.603 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:20:41.603 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:20:41.603 INFO:teuthology.orchestra.run.smithi037.stdout:Standby daemons: 2024-03-29T08:20:41.603 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:20:41.603 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi148.qukpjh{-1:24319} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.148:6826/2641327912,v1:172.21.15.148:6827/2641327912] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:20:41.603 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi037.mmwgxe{-1:34440} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.37:6828/1248142005,v1:172.21.15.37:6829/1248142005] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:20:41.603 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi037.iifsmh{-1:34444} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.37:6826/2060882959,v1:172.21.15.37:6827/2060882959] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:20:41.603 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi148.sqteih{-1:44395} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.148:6824/4078652242,v1:172.21.15.148:6825/4078652242] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:20:41.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:41.602+0000 7f4617400700 1 -- 172.21.15.37:0/962126602 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f45f806cc30 msgr2=0x7f45f806f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:41.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:41.602+0000 7f4617400700 1 --2- 172.21.15.37:0/962126602 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f45f806cc30 0x7f45f806f0e0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f4604009fd0 tx=0x7f4604009e30 comp rx=0 tx=0).stop 2024-03-29T08:20:41.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:41.602+0000 7f4617400700 1 -- 172.21.15.37:0/962126602 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f46101037a0 msgr2=0x7f4610116210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:41.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:41.602+0000 7f4617400700 1 --2- 172.21.15.37:0/962126602 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f46101037a0 0x7f4610116210 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f45fc00c800 tx=0x7f45fc004470 comp rx=0 tx=0).stop 2024-03-29T08:20:41.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:41.603+0000 7f4617400700 1 -- 172.21.15.37:0/962126602 shutdown_connections 2024-03-29T08:20:41.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:41.603+0000 7f4617400700 1 --2- 172.21.15.37:0/962126602 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f45f806cc30 0x7f45f806f0e0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:41.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:41.603+0000 7f4617400700 1 --2- 172.21.15.37:0/962126602 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4610102e00 0x7f4610115cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:41.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:41.603+0000 7f4617400700 1 --2- 172.21.15.37:0/962126602 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f46101037a0 0x7f4610116210 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:41.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:41.603+0000 7f4617400700 1 -- 172.21.15.37:0/962126602 >> 172.21.15.37:0/962126602 conn(0x7f46100f89b0 msgr2=0x7f46100f99e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:20:41.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:41.603+0000 7f4617400700 1 -- 172.21.15.37:0/962126602 shutdown_connections 2024-03-29T08:20:41.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:41.603+0000 7f4617400700 1 -- 172.21.15.37:0/962126602 wait complete. 2024-03-29T08:20:41.605 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 28 2024-03-29T08:20:41.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:41.742+0000 7fce17200700 1 -- 172.21.15.37:0/4167351014 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fce101037a0 msgr2=0x7fce100fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:41.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:41.742+0000 7fce17200700 1 --2- 172.21.15.37:0/4167351014 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fce101037a0 0x7fce100fce10 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fcdfc004940 tx=0x7fcdfc009e30 comp rx=0 tx=0).stop 2024-03-29T08:20:41.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:41.742+0000 7fce17200700 1 -- 172.21.15.37:0/4167351014 shutdown_connections 2024-03-29T08:20:41.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:41.742+0000 7fce17200700 1 --2- 172.21.15.37:0/4167351014 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fce101037a0 0x7fce100fce10 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:41.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:41.742+0000 7fce17200700 1 --2- 172.21.15.37:0/4167351014 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fce10102e00 0x7fce101031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:41.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:41.742+0000 7fce17200700 1 -- 172.21.15.37:0/4167351014 >> 172.21.15.37:0/4167351014 conn(0x7fce100f89b0 msgr2=0x7fce100fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:20:41.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:41.742+0000 7fce17200700 1 -- 172.21.15.37:0/4167351014 shutdown_connections 2024-03-29T08:20:41.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:41.742+0000 7fce17200700 1 -- 172.21.15.37:0/4167351014 wait complete. 2024-03-29T08:20:41.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:41.743+0000 7fce17200700 1 Processor -- start 2024-03-29T08:20:41.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:41.743+0000 7fce17200700 1 -- start start 2024-03-29T08:20:41.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:41.744+0000 7fce17200700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fce10102e00 0x7fce10113b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:41.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:41.744+0000 7fce17200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fce101037a0 0x7fce10114040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:41.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:41.744+0000 7fce17200700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce10114690 con 0x7fce101037a0 2024-03-29T08:20:41.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:41.744+0000 7fce17200700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce1010da80 con 0x7fce10102e00 2024-03-29T08:20:41.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:41.744+0000 7fce0f400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fce101037a0 0x7fce10114040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:41.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:41.744+0000 7fce0f400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fce101037a0 0x7fce10114040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50160/0 (socket says 172.21.15.37:50160) 2024-03-29T08:20:41.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:41.744+0000 7fce0f400700 1 -- 172.21.15.37:0/690651969 learned_addr learned my addr 172.21.15.37:0/690651969 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:20:41.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:41.744+0000 7fce0fe00700 1 --2- 172.21.15.37:0/690651969 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fce10102e00 0x7fce10113b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:41.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:41.744+0000 7fce0f400700 1 -- 172.21.15.37:0/690651969 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fce10102e00 msgr2=0x7fce10113b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:41.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:41.744+0000 7fce0f400700 1 --2- 172.21.15.37:0/690651969 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fce10102e00 0x7fce10113b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:41.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:41.744+0000 7fce0f400700 1 -- 172.21.15.37:0/690651969 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdfc005040 con 0x7fce101037a0 2024-03-29T08:20:41.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:41.745+0000 7fce0f400700 1 --2- 172.21.15.37:0/690651969 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fce101037a0 0x7fce10114040 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fcdfc009390 tx=0x7fcdfc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:20:41.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:41.745+0000 7fce07e00700 1 -- 172.21.15.37:0/690651969 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcdfc00e040 con 0x7fce101037a0 2024-03-29T08:20:41.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:41.745+0000 7fce07e00700 1 -- 172.21.15.37:0/690651969 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fcdfc013420 con 0x7fce101037a0 2024-03-29T08:20:41.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:41.745+0000 7fce07e00700 1 -- 172.21.15.37:0/690651969 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcdfc01c630 con 0x7fce101037a0 2024-03-29T08:20:41.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:41.745+0000 7fce17200700 1 -- 172.21.15.37:0/690651969 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce1010dd00 con 0x7fce101037a0 2024-03-29T08:20:41.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:41.745+0000 7fce17200700 1 -- 172.21.15.37:0/690651969 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce1010e2c0 con 0x7fce101037a0 2024-03-29T08:20:41.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:41.746+0000 7fce17200700 1 -- 172.21.15.37:0/690651969 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce100fe550 con 0x7fce101037a0 2024-03-29T08:20:41.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:41.747+0000 7fce07e00700 1 -- 172.21.15.37:0/690651969 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fcdfc0139e0 con 0x7fce101037a0 2024-03-29T08:20:41.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:41.748+0000 7fce07e00700 1 --2- 172.21.15.37:0/690651969 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fcdf406cc30 0x7fcdf406f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:41.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:41.748+0000 7fce07e00700 1 -- 172.21.15.37:0/690651969 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fcdfc08d1e0 con 0x7fce101037a0 2024-03-29T08:20:41.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:41.748+0000 7fce0fe00700 1 --2- 172.21.15.37:0/690651969 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fcdf406cc30 0x7fcdf406f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:41.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:41.749+0000 7fce0fe00700 1 --2- 172.21.15.37:0/690651969 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fcdf406cc30 0x7fcdf406f0e0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fce00004990 tx=0x7fce00009ee0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:20:41.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:41.753+0000 7fce07e00700 1 -- 172.21.15.37:0/690651969 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fcdfc05ad70 con 0x7fce101037a0 2024-03-29T08:20:41.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:41.985+0000 7fce17200700 1 -- 172.21.15.37:0/690651969 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fce10109da0 con 0x7fcdf406cc30 2024-03-29T08:20:41.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:41.986+0000 7fce07e00700 1 -- 172.21.15.37:0/690651969 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fce10109da0 con 0x7fcdf406cc30 2024-03-29T08:20:41.986 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-29T08:20:41.986 INFO:teuthology.orchestra.run.smithi037.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59", 2024-03-29T08:20:41.986 INFO:teuthology.orchestra.run.smithi037.stdout: "in_progress": true, 2024-03-29T08:20:41.987 INFO:teuthology.orchestra.run.smithi037.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-29T08:20:41.987 INFO:teuthology.orchestra.run.smithi037.stdout: "services_complete": [ 2024-03-29T08:20:41.987 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr", 2024-03-29T08:20:41.987 INFO:teuthology.orchestra.run.smithi037.stdout: "mon", 2024-03-29T08:20:41.987 INFO:teuthology.orchestra.run.smithi037.stdout: "crash", 2024-03-29T08:20:41.987 INFO:teuthology.orchestra.run.smithi037.stdout: "osd" 2024-03-29T08:20:41.987 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-29T08:20:41.987 INFO:teuthology.orchestra.run.smithi037.stdout: "progress": "15/21 daemons upgraded", 2024-03-29T08:20:41.987 INFO:teuthology.orchestra.run.smithi037.stdout: "message": "Currently upgrading mds daemons", 2024-03-29T08:20:41.987 INFO:teuthology.orchestra.run.smithi037.stdout: "is_paused": false 2024-03-29T08:20:41.987 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-29T08:20:41.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:41.988+0000 7fce17200700 1 -- 172.21.15.37:0/690651969 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fcdf406cc30 msgr2=0x7fcdf406f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:41.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:41.988+0000 7fce17200700 1 --2- 172.21.15.37:0/690651969 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fcdf406cc30 0x7fcdf406f0e0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fce00004990 tx=0x7fce00009ee0 comp rx=0 tx=0).stop 2024-03-29T08:20:41.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:41.988+0000 7fce17200700 1 -- 172.21.15.37:0/690651969 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fce101037a0 msgr2=0x7fce10114040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:41.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:41.988+0000 7fce17200700 1 --2- 172.21.15.37:0/690651969 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fce101037a0 0x7fce10114040 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fcdfc009390 tx=0x7fcdfc009e30 comp rx=0 tx=0).stop 2024-03-29T08:20:41.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:41.989+0000 7fce17200700 1 -- 172.21.15.37:0/690651969 shutdown_connections 2024-03-29T08:20:41.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:41.989+0000 7fce17200700 1 --2- 172.21.15.37:0/690651969 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fcdf406cc30 0x7fcdf406f0e0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:41.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:41.989+0000 7fce17200700 1 --2- 172.21.15.37:0/690651969 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fce10102e00 0x7fce10113b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:41.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:41.989+0000 7fce17200700 1 --2- 172.21.15.37:0/690651969 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fce101037a0 0x7fce10114040 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:41.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:41.989+0000 7fce17200700 1 -- 172.21.15.37:0/690651969 >> 172.21.15.37:0/690651969 conn(0x7fce100f89b0 msgr2=0x7fce100f9ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:20:41.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:41.989+0000 7fce17200700 1 -- 172.21.15.37:0/690651969 shutdown_connections 2024-03-29T08:20:41.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:41.989+0000 7fce17200700 1 -- 172.21.15.37:0/690651969 wait complete. 2024-03-29T08:20:42.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:42.125+0000 7f2e70600700 1 -- 172.21.15.37:0/2356981112 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2e681037a0 msgr2=0x7f2e680fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:42.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:42.125+0000 7f2e70600700 1 --2- 172.21.15.37:0/2356981112 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2e681037a0 0x7f2e680fce10 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f2e54004940 tx=0x7f2e54009e30 comp rx=0 tx=0).stop 2024-03-29T08:20:42.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:42.125+0000 7f2e70600700 1 -- 172.21.15.37:0/2356981112 shutdown_connections 2024-03-29T08:20:42.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:42.125+0000 7f2e70600700 1 --2- 172.21.15.37:0/2356981112 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2e681037a0 0x7f2e680fce10 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:42.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:42.125+0000 7f2e70600700 1 --2- 172.21.15.37:0/2356981112 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2e68102e00 0x7f2e681031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:42.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:42.125+0000 7f2e70600700 1 -- 172.21.15.37:0/2356981112 >> 172.21.15.37:0/2356981112 conn(0x7f2e680f89b0 msgr2=0x7f2e680fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:20:42.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:42.126+0000 7f2e70600700 1 -- 172.21.15.37:0/2356981112 shutdown_connections 2024-03-29T08:20:42.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:42.126+0000 7f2e70600700 1 -- 172.21.15.37:0/2356981112 wait complete. 2024-03-29T08:20:42.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:42.126+0000 7f2e70600700 1 Processor -- start 2024-03-29T08:20:42.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:42.127+0000 7f2e70600700 1 -- start start 2024-03-29T08:20:42.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:42.127+0000 7f2e70600700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2e68102e00 0x7f2e68115d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:42.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:42.127+0000 7f2e70600700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2e681037a0 0x7f2e68116260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:42.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:42.127+0000 7f2e70600700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e681168b0 con 0x7f2e68102e00 2024-03-29T08:20:42.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:42.127+0000 7f2e70600700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e6810fc80 con 0x7f2e681037a0 2024-03-29T08:20:42.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:42.127+0000 7f2e6dc00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2e68102e00 0x7f2e68115d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:42.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:42.127+0000 7f2e6dc00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2e68102e00 0x7f2e68115d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50188/0 (socket says 172.21.15.37:50188) 2024-03-29T08:20:42.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:42.127+0000 7f2e6dc00700 1 -- 172.21.15.37:0/1744751018 learned_addr learned my addr 172.21.15.37:0/1744751018 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:20:42.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:42.128+0000 7f2e6d200700 1 --2- 172.21.15.37:0/1744751018 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2e681037a0 0x7f2e68116260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:42.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:42.128+0000 7f2e6dc00700 1 -- 172.21.15.37:0/1744751018 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2e681037a0 msgr2=0x7f2e68116260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:42.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:42.128+0000 7f2e6dc00700 1 --2- 172.21.15.37:0/1744751018 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2e681037a0 0x7f2e68116260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:42.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:42.128+0000 7f2e6dc00700 1 -- 172.21.15.37:0/1744751018 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e54005040 con 0x7f2e68102e00 2024-03-29T08:20:42.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:42.128+0000 7f2e6d200700 1 --2- 172.21.15.37:0/1744751018 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2e681037a0 0x7f2e68116260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:20:42.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:42.128+0000 7f2e6dc00700 1 --2- 172.21.15.37:0/1744751018 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2e68102e00 0x7f2e68115d20 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f2e5c002a50 tx=0x7f2e5c0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:20:42.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:42.128+0000 7f2e65600700 1 -- 172.21.15.37:0/1744751018 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2e5c007cc0 con 0x7f2e68102e00 2024-03-29T08:20:42.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:42.128+0000 7f2e65600700 1 -- 172.21.15.37:0/1744751018 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f2e5c00f040 con 0x7f2e68102e00 2024-03-29T08:20:42.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:42.128+0000 7f2e65600700 1 -- 172.21.15.37:0/1744751018 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2e5c009540 con 0x7f2e68102e00 2024-03-29T08:20:42.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:42.128+0000 7f2e70600700 1 -- 172.21.15.37:0/1744751018 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e6810ff60 con 0x7f2e68102e00 2024-03-29T08:20:42.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:42.128+0000 7f2e70600700 1 -- 172.21.15.37:0/1744751018 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e68110520 con 0x7f2e68102e00 2024-03-29T08:20:42.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:42.129+0000 7f2e70600700 1 -- 172.21.15.37:0/1744751018 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e680fe550 con 0x7f2e68102e00 2024-03-29T08:20:42.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:42.130+0000 7f2e65600700 1 -- 172.21.15.37:0/1744751018 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f2e5c0096a0 con 0x7f2e68102e00 2024-03-29T08:20:42.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:42.131+0000 7f2e65600700 1 --2- 172.21.15.37:0/1744751018 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f2e5006cc20 0x7f2e5006f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:42.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:42.131+0000 7f2e65600700 1 -- 172.21.15.37:0/1744751018 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f2e5c08bde0 con 0x7f2e68102e00 2024-03-29T08:20:42.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:42.131+0000 7f2e6d200700 1 --2- 172.21.15.37:0/1744751018 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f2e5006cc20 0x7f2e5006f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:42.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:42.132+0000 7f2e6d200700 1 --2- 172.21.15.37:0/1744751018 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f2e5006cc20 0x7f2e5006f0d0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f2e54000c00 tx=0x7f2e54009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:20:42.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:42.136+0000 7f2e65600700 1 -- 172.21.15.37:0/1744751018 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f2e5c059830 con 0x7f2e68102e00 2024-03-29T08:20:42.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:42.435+0000 7f2e70600700 1 -- 172.21.15.37:0/1744751018 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f2e6804bed0 con 0x7f2e68102e00 2024-03-29T08:20:42.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:42.436+0000 7f2e65600700 1 -- 172.21.15.37:0/1744751018 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7f2e5c0593c0 con 0x7f2e68102e00 2024-03-29T08:20:42.437 INFO:teuthology.orchestra.run.smithi037.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-03-29T08:20:42.437 INFO:teuthology.orchestra.run.smithi037.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-03-29T08:20:42.437 INFO:teuthology.orchestra.run.smithi037.stdout: fs cephfs is degraded 2024-03-29T08:20:42.437 INFO:teuthology.orchestra.run.smithi037.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-03-29T08:20:42.437 INFO:teuthology.orchestra.run.smithi037.stdout: fs cephfs is offline because no MDS is active for it. 2024-03-29T08:20:42.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:42.438+0000 7f2e70600700 1 -- 172.21.15.37:0/1744751018 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f2e5006cc20 msgr2=0x7f2e5006f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:42.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:42.438+0000 7f2e70600700 1 --2- 172.21.15.37:0/1744751018 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f2e5006cc20 0x7f2e5006f0d0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f2e54000c00 tx=0x7f2e54009e30 comp rx=0 tx=0).stop 2024-03-29T08:20:42.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:42.438+0000 7f2e70600700 1 -- 172.21.15.37:0/1744751018 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2e68102e00 msgr2=0x7f2e68115d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:42.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:42.438+0000 7f2e70600700 1 --2- 172.21.15.37:0/1744751018 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2e68102e00 0x7f2e68115d20 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f2e5c002a50 tx=0x7f2e5c0046b0 comp rx=0 tx=0).stop 2024-03-29T08:20:42.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:42.438+0000 7f2e70600700 1 -- 172.21.15.37:0/1744751018 shutdown_connections 2024-03-29T08:20:42.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:42.438+0000 7f2e70600700 1 --2- 172.21.15.37:0/1744751018 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f2e5006cc20 0x7f2e5006f0d0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:42.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:42.438+0000 7f2e70600700 1 --2- 172.21.15.37:0/1744751018 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2e68102e00 0x7f2e68115d20 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:42.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:42.439+0000 7f2e70600700 1 --2- 172.21.15.37:0/1744751018 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2e681037a0 0x7f2e68116260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:42.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:42.439+0000 7f2e70600700 1 -- 172.21.15.37:0/1744751018 >> 172.21.15.37:0/1744751018 conn(0x7f2e680f89b0 msgr2=0x7f2e680f99f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:20:42.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:42.439+0000 7f2e70600700 1 -- 172.21.15.37:0/1744751018 shutdown_connections 2024-03-29T08:20:42.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:20:42.439+0000 7f2e70600700 1 -- 172.21.15.37:0/1744751018 wait complete. 2024-03-29T08:20:42.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:20:42 smithi148 ceph-mon[106692]: from='client.34596 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:20:42.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:20:42 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/2605081963' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:20:42.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:20:42 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/962126602' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T08:20:42.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:20:42 smithi148 ceph-mon[106692]: pgmap v294: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:20:42.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:20:42 smithi037 ceph-mon[142325]: from='client.34596 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:20:42.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:20:42 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/2605081963' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:20:42.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:20:42 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/962126602' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T08:20:42.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:20:42 smithi037 ceph-mon[142325]: pgmap v294: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:20:43.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:20:43 smithi037 ceph-mon[142325]: from='client.34608 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:20:43.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:20:43 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/1744751018' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T08:20:43.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:20:43 smithi148 ceph-mon[106692]: from='client.34608 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:20:43.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:20:43 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/1744751018' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T08:20:44.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:20:44 smithi148 ceph-mon[106692]: pgmap v295: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:20:44.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:20:44 smithi037 ceph-mon[142325]: pgmap v295: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:20:46.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:20:45 smithi148 ceph-mon[106692]: pgmap v296: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:20:46.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:20:45 smithi037 ceph-mon[142325]: pgmap v296: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:20:48.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:20:47 smithi148 ceph-mon[106692]: pgmap v297: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:20:48.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:20:47 smithi037 ceph-mon[142325]: pgmap v297: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:20:50.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:20:50 smithi148 ceph-mon[106692]: pgmap v298: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:20:50.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:20:50 smithi037 ceph-mon[142325]: pgmap v298: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:20:52.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:20:51 smithi148 ceph-mon[106692]: pgmap v299: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:20:52.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:20:51 smithi037 ceph-mon[142325]: pgmap v299: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:20:54.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:20:53 smithi148 ceph-mon[106692]: pgmap v300: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:20:54.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:20:53 smithi037 ceph-mon[142325]: pgmap v300: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:20:55.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:20:55 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:20:55.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:20:55 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:20:56.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:20:56 smithi148 ceph-mon[106692]: pgmap v301: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:20:56.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:20:56 smithi037 ceph-mon[142325]: pgmap v301: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:20:58.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:20:57 smithi148 ceph-mon[106692]: pgmap v302: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:20:58.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:20:57 smithi037 ceph-mon[142325]: pgmap v302: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:21:00.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:21:00 smithi148 ceph-mon[106692]: pgmap v303: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:21:00.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:21:00 smithi037 ceph-mon[142325]: pgmap v303: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:21:02.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:21:01 smithi148 ceph-mon[106692]: pgmap v304: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:21:02.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:21:01 smithi037 ceph-mon[142325]: pgmap v304: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:21:04.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:21:03 smithi148 ceph-mon[106692]: pgmap v305: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:21:04.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:21:03 smithi037 ceph-mon[142325]: pgmap v305: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:21:06.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:21:05 smithi148 ceph-mon[106692]: pgmap v306: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:21:06.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:21:05 smithi037 ceph-mon[142325]: pgmap v306: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:21:08.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:21:07 smithi148 ceph-mon[106692]: pgmap v307: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:21:08.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:21:07 smithi037 ceph-mon[142325]: pgmap v307: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:21:10.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:21:10 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:21:10.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:21:10 smithi148 ceph-mon[106692]: pgmap v308: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:21:10.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:21:10 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:21:10.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:21:10 smithi037 ceph-mon[142325]: pgmap v308: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:21:12.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:21:11 smithi148 ceph-mon[106692]: pgmap v309: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:21:12.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:21:11 smithi037 ceph-mon[142325]: pgmap v309: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:21:12.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:12.591+0000 7fc919000700 1 -- 172.21.15.37:0/166050798 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc914065630 msgr2=0x7fc914065a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:12.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:12.591+0000 7fc919000700 1 --2- 172.21.15.37:0/166050798 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc914065630 0x7fc914065a80 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fc900004940 tx=0x7fc900009e30 comp rx=0 tx=0).stop 2024-03-29T08:21:12.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:12.591+0000 7fc919000700 1 -- 172.21.15.37:0/166050798 shutdown_connections 2024-03-29T08:21:12.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:12.591+0000 7fc919000700 1 --2- 172.21.15.37:0/166050798 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc914065630 0x7fc914065a80 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:12.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:12.591+0000 7fc919000700 1 --2- 172.21.15.37:0/166050798 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc914102ca0 0x7fc914103070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:12.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:12.591+0000 7fc919000700 1 -- 172.21.15.37:0/166050798 >> 172.21.15.37:0/166050798 conn(0x7fc914071b70 msgr2=0x7fc914071f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:21:12.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:12.591+0000 7fc919000700 1 -- 172.21.15.37:0/166050798 shutdown_connections 2024-03-29T08:21:12.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:12.591+0000 7fc919000700 1 -- 172.21.15.37:0/166050798 wait complete. 2024-03-29T08:21:12.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:12.592+0000 7fc919000700 1 Processor -- start 2024-03-29T08:21:12.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:12.592+0000 7fc919000700 1 -- start start 2024-03-29T08:21:12.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:12.593+0000 7fc919000700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc914065630 0x7fc914115ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:12.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:12.593+0000 7fc919000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc914102ca0 0x7fc914116010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:12.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:12.593+0000 7fc919000700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9141166f0 con 0x7fc914102ca0 2024-03-29T08:21:12.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:12.593+0000 7fc919000700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc91410fa50 con 0x7fc914065630 2024-03-29T08:21:12.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:12.593+0000 7fc912a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc914102ca0 0x7fc914116010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:12.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:12.593+0000 7fc912a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc914102ca0 0x7fc914116010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57848/0 (socket says 172.21.15.37:57848) 2024-03-29T08:21:12.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:12.593+0000 7fc912a00700 1 -- 172.21.15.37:0/1836123609 learned_addr learned my addr 172.21.15.37:0/1836123609 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:21:12.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:12.593+0000 7fc913400700 1 --2- 172.21.15.37:0/1836123609 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc914065630 0x7fc914115ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:12.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:12.594+0000 7fc912a00700 1 -- 172.21.15.37:0/1836123609 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc914065630 msgr2=0x7fc914115ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:12.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:12.594+0000 7fc912a00700 1 --2- 172.21.15.37:0/1836123609 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc914065630 0x7fc914115ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:12.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:12.594+0000 7fc912a00700 1 -- 172.21.15.37:0/1836123609 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc900005040 con 0x7fc914102ca0 2024-03-29T08:21:12.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:12.594+0000 7fc913400700 1 --2- 172.21.15.37:0/1836123609 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc914065630 0x7fc914115ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:21:12.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:12.594+0000 7fc912a00700 1 --2- 172.21.15.37:0/1836123609 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc914102ca0 0x7fc914116010 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fc900009390 tx=0x7fc900009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:21:12.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:12.594+0000 7fc90b400700 1 -- 172.21.15.37:0/1836123609 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc90000e040 con 0x7fc914102ca0 2024-03-29T08:21:12.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:12.594+0000 7fc90b400700 1 -- 172.21.15.37:0/1836123609 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fc900013420 con 0x7fc914102ca0 2024-03-29T08:21:12.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:12.594+0000 7fc919000700 1 -- 172.21.15.37:0/1836123609 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc91410fcd0 con 0x7fc914102ca0 2024-03-29T08:21:12.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:12.594+0000 7fc90b400700 1 -- 172.21.15.37:0/1836123609 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc90001c630 con 0x7fc914102ca0 2024-03-29T08:21:12.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:12.594+0000 7fc919000700 1 -- 172.21.15.37:0/1836123609 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc914110290 con 0x7fc914102ca0 2024-03-29T08:21:12.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:12.596+0000 7fc919000700 1 -- 172.21.15.37:0/1836123609 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc914106840 con 0x7fc914102ca0 2024-03-29T08:21:12.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:12.596+0000 7fc90b400700 1 -- 172.21.15.37:0/1836123609 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fc9000139e0 con 0x7fc914102ca0 2024-03-29T08:21:12.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:12.597+0000 7fc90b400700 1 --2- 172.21.15.37:0/1836123609 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fc8f806cc30 0x7fc8f806f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:12.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:12.597+0000 7fc90b400700 1 -- 172.21.15.37:0/1836123609 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fc90008d2e0 con 0x7fc914102ca0 2024-03-29T08:21:12.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:12.598+0000 7fc913400700 1 --2- 172.21.15.37:0/1836123609 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fc8f806cc30 0x7fc8f806f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:12.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:12.598+0000 7fc913400700 1 --2- 172.21.15.37:0/1836123609 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fc8f806cc30 0x7fc8f806f0e0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fc904009b10 tx=0x7fc904008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:21:12.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:12.602+0000 7fc90b400700 1 -- 172.21.15.37:0/1836123609 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fc90005ae60 con 0x7fc914102ca0 2024-03-29T08:21:12.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:12.834+0000 7fc919000700 1 -- 172.21.15.37:0/1836123609 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc914111190 con 0x7fc8f806cc30 2024-03-29T08:21:12.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:12.836+0000 7fc90b400700 1 -- 172.21.15.37:0/1836123609 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fc914111190 con 0x7fc8f806cc30 2024-03-29T08:21:12.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:12.838+0000 7fc919000700 1 -- 172.21.15.37:0/1836123609 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fc8f806cc30 msgr2=0x7fc8f806f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:12.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:12.838+0000 7fc919000700 1 --2- 172.21.15.37:0/1836123609 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fc8f806cc30 0x7fc8f806f0e0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fc904009b10 tx=0x7fc904008040 comp rx=0 tx=0).stop 2024-03-29T08:21:12.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:12.838+0000 7fc919000700 1 -- 172.21.15.37:0/1836123609 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc914102ca0 msgr2=0x7fc914116010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:12.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:12.838+0000 7fc919000700 1 --2- 172.21.15.37:0/1836123609 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc914102ca0 0x7fc914116010 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fc900009390 tx=0x7fc900009e30 comp rx=0 tx=0).stop 2024-03-29T08:21:12.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:12.838+0000 7fc919000700 1 -- 172.21.15.37:0/1836123609 shutdown_connections 2024-03-29T08:21:12.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:12.838+0000 7fc919000700 1 --2- 172.21.15.37:0/1836123609 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fc8f806cc30 0x7fc8f806f0e0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:12.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:12.838+0000 7fc919000700 1 --2- 172.21.15.37:0/1836123609 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc914065630 0x7fc914115ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:12.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:12.838+0000 7fc919000700 1 --2- 172.21.15.37:0/1836123609 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc914102ca0 0x7fc914116010 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:12.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:12.838+0000 7fc919000700 1 -- 172.21.15.37:0/1836123609 >> 172.21.15.37:0/1836123609 conn(0x7fc914071b70 msgr2=0x7fc9141039c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:21:12.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:12.838+0000 7fc919000700 1 -- 172.21.15.37:0/1836123609 shutdown_connections 2024-03-29T08:21:12.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:12.838+0000 7fc919000700 1 -- 172.21.15.37:0/1836123609 wait complete. 2024-03-29T08:21:12.853 INFO:teuthology.orchestra.run.smithi037.stdout:true 2024-03-29T08:21:12.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:12.977+0000 7fd6c7400700 1 -- 172.21.15.37:0/26985693 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd6c01037a0 msgr2=0x7fd6c00fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:12.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:12.977+0000 7fd6c7400700 1 --2- 172.21.15.37:0/26985693 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd6c01037a0 0x7fd6c00fce10 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fd6ac004940 tx=0x7fd6ac009e30 comp rx=0 tx=0).stop 2024-03-29T08:21:12.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:12.977+0000 7fd6c7400700 1 -- 172.21.15.37:0/26985693 shutdown_connections 2024-03-29T08:21:12.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:12.977+0000 7fd6c7400700 1 --2- 172.21.15.37:0/26985693 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd6c01037a0 0x7fd6c00fce10 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:12.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:12.977+0000 7fd6c7400700 1 --2- 172.21.15.37:0/26985693 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd6c0102e00 0x7fd6c01031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:12.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:12.977+0000 7fd6c7400700 1 -- 172.21.15.37:0/26985693 >> 172.21.15.37:0/26985693 conn(0x7fd6c00f89b0 msgr2=0x7fd6c00fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:21:12.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:12.977+0000 7fd6c7400700 1 -- 172.21.15.37:0/26985693 shutdown_connections 2024-03-29T08:21:12.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:12.977+0000 7fd6c7400700 1 -- 172.21.15.37:0/26985693 wait complete. 2024-03-29T08:21:12.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:12.978+0000 7fd6c7400700 1 Processor -- start 2024-03-29T08:21:12.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:12.978+0000 7fd6c7400700 1 -- start start 2024-03-29T08:21:12.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:12.978+0000 7fd6c7400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd6c0102e00 0x7fd6c0115c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:12.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:12.978+0000 7fd6c7400700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd6c01037a0 0x7fd6c01161a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:12.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:12.978+0000 7fd6c7400700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6c01167f0 con 0x7fd6c0102e00 2024-03-29T08:21:12.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:12.978+0000 7fd6c7400700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6c010fbc0 con 0x7fd6c01037a0 2024-03-29T08:21:12.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:12.979+0000 7fd6c4a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd6c0102e00 0x7fd6c0115c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:12.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:12.979+0000 7fd6c4a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd6c0102e00 0x7fd6c0115c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57866/0 (socket says 172.21.15.37:57866) 2024-03-29T08:21:12.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:12.979+0000 7fd6c4a00700 1 -- 172.21.15.37:0/3825936551 learned_addr learned my addr 172.21.15.37:0/3825936551 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:21:12.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:12.979+0000 7fd6bfe00700 1 --2- 172.21.15.37:0/3825936551 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd6c01037a0 0x7fd6c01161a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:12.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:12.979+0000 7fd6c4a00700 1 -- 172.21.15.37:0/3825936551 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd6c01037a0 msgr2=0x7fd6c01161a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:12.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:12.979+0000 7fd6c4a00700 1 --2- 172.21.15.37:0/3825936551 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd6c01037a0 0x7fd6c01161a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:12.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:12.979+0000 7fd6c4a00700 1 -- 172.21.15.37:0/3825936551 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6ac005040 con 0x7fd6c0102e00 2024-03-29T08:21:12.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:12.979+0000 7fd6bfe00700 1 --2- 172.21.15.37:0/3825936551 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd6c01037a0 0x7fd6c01161a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:21:12.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:12.979+0000 7fd6c4a00700 1 --2- 172.21.15.37:0/3825936551 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd6c0102e00 0x7fd6c0115c60 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fd6b400eb40 tx=0x7fd6b400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:21:12.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:12.980+0000 7fd6bcc00700 1 -- 172.21.15.37:0/3825936551 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd6b4007cc0 con 0x7fd6c0102e00 2024-03-29T08:21:12.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:12.980+0000 7fd6bcc00700 1 -- 172.21.15.37:0/3825936551 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd6b400f040 con 0x7fd6c0102e00 2024-03-29T08:21:12.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:12.980+0000 7fd6c7400700 1 -- 172.21.15.37:0/3825936551 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6c010fe40 con 0x7fd6c0102e00 2024-03-29T08:21:12.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:12.980+0000 7fd6bcc00700 1 -- 172.21.15.37:0/3825936551 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd6b4009570 con 0x7fd6c0102e00 2024-03-29T08:21:12.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:12.980+0000 7fd6c7400700 1 -- 172.21.15.37:0/3825936551 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6c0110400 con 0x7fd6c0102e00 2024-03-29T08:21:12.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:12.981+0000 7fd6c7400700 1 -- 172.21.15.37:0/3825936551 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6c00fe550 con 0x7fd6c0102e00 2024-03-29T08:21:12.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:12.982+0000 7fd6bcc00700 1 -- 172.21.15.37:0/3825936551 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fd6b40096d0 con 0x7fd6c0102e00 2024-03-29T08:21:12.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:12.983+0000 7fd6bcc00700 1 --2- 172.21.15.37:0/3825936551 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fd6a806cc30 0x7fd6a806f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:12.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:12.983+0000 7fd6bcc00700 1 -- 172.21.15.37:0/3825936551 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fd6b408bda0 con 0x7fd6c0102e00 2024-03-29T08:21:12.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:12.983+0000 7fd6bfe00700 1 --2- 172.21.15.37:0/3825936551 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fd6a806cc30 0x7fd6a806f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:12.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:12.984+0000 7fd6bfe00700 1 --2- 172.21.15.37:0/3825936551 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fd6a806cc30 0x7fd6a806f0e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fd6ac00b0d0 tx=0x7fd6ac009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:21:12.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:12.987+0000 7fd6bcc00700 1 -- 172.21.15.37:0/3825936551 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fd6b4059820 con 0x7fd6c0102e00 2024-03-29T08:21:13.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:13.220+0000 7fd6c7400700 1 -- 172.21.15.37:0/3825936551 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd6c004bf40 con 0x7fd6a806cc30 2024-03-29T08:21:13.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:13.221+0000 7fd6bcc00700 1 -- 172.21.15.37:0/3825936551 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fd6c004bf40 con 0x7fd6a806cc30 2024-03-29T08:21:13.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:13.223+0000 7fd6c7400700 1 -- 172.21.15.37:0/3825936551 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fd6a806cc30 msgr2=0x7fd6a806f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:13.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:13.223+0000 7fd6c7400700 1 --2- 172.21.15.37:0/3825936551 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fd6a806cc30 0x7fd6a806f0e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fd6ac00b0d0 tx=0x7fd6ac009e30 comp rx=0 tx=0).stop 2024-03-29T08:21:13.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:13.223+0000 7fd6c7400700 1 -- 172.21.15.37:0/3825936551 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd6c0102e00 msgr2=0x7fd6c0115c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:13.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:13.223+0000 7fd6c7400700 1 --2- 172.21.15.37:0/3825936551 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd6c0102e00 0x7fd6c0115c60 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fd6b400eb40 tx=0x7fd6b400b040 comp rx=0 tx=0).stop 2024-03-29T08:21:13.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:13.223+0000 7fd6c7400700 1 -- 172.21.15.37:0/3825936551 shutdown_connections 2024-03-29T08:21:13.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:13.223+0000 7fd6c7400700 1 --2- 172.21.15.37:0/3825936551 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fd6a806cc30 0x7fd6a806f0e0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:13.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:13.223+0000 7fd6c7400700 1 --2- 172.21.15.37:0/3825936551 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd6c0102e00 0x7fd6c0115c60 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:13.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:13.223+0000 7fd6c7400700 1 --2- 172.21.15.37:0/3825936551 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd6c01037a0 0x7fd6c01161a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:13.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:13.223+0000 7fd6c7400700 1 -- 172.21.15.37:0/3825936551 >> 172.21.15.37:0/3825936551 conn(0x7fd6c00f89b0 msgr2=0x7fd6c00f99c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:21:13.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:13.224+0000 7fd6c7400700 1 -- 172.21.15.37:0/3825936551 shutdown_connections 2024-03-29T08:21:13.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:13.224+0000 7fd6c7400700 1 -- 172.21.15.37:0/3825936551 wait complete. 2024-03-29T08:21:13.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:13.362+0000 7fd7db800700 1 -- 172.21.15.37:0/4058165213 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd7d4070360 msgr2=0x7fd7d4070730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:13.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:13.362+0000 7fd7db800700 1 --2- 172.21.15.37:0/4058165213 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd7d4070360 0x7fd7d4070730 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fd7c8004970 tx=0x7fd7c8009e30 comp rx=0 tx=0).stop 2024-03-29T08:21:13.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:13.362+0000 7fd7db800700 1 -- 172.21.15.37:0/4058165213 shutdown_connections 2024-03-29T08:21:13.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:13.362+0000 7fd7db800700 1 --2- 172.21.15.37:0/4058165213 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd7d4070c70 0x7fd7d4109110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:13.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:13.362+0000 7fd7db800700 1 --2- 172.21.15.37:0/4058165213 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd7d4070360 0x7fd7d4070730 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:13.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:13.362+0000 7fd7db800700 1 -- 172.21.15.37:0/4058165213 >> 172.21.15.37:0/4058165213 conn(0x7fd7d40f89b0 msgr2=0x7fd7d40fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:21:13.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:13.362+0000 7fd7db800700 1 -- 172.21.15.37:0/4058165213 shutdown_connections 2024-03-29T08:21:13.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:13.362+0000 7fd7db800700 1 -- 172.21.15.37:0/4058165213 wait complete. 2024-03-29T08:21:13.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:13.363+0000 7fd7db800700 1 Processor -- start 2024-03-29T08:21:13.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:13.363+0000 7fd7db800700 1 -- start start 2024-03-29T08:21:13.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:13.363+0000 7fd7db800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd7d4070360 0x7fd7d4115c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:13.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:13.363+0000 7fd7db800700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd7d4070c70 0x7fd7d41161c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:13.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:13.363+0000 7fd7db800700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7d4116810 con 0x7fd7d4070360 2024-03-29T08:21:13.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:13.363+0000 7fd7db800700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7d410fbe0 con 0x7fd7d4070c70 2024-03-29T08:21:13.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:13.364+0000 7fd7d8e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd7d4070360 0x7fd7d4115c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:13.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:13.364+0000 7fd7d8e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd7d4070360 0x7fd7d4115c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57874/0 (socket says 172.21.15.37:57874) 2024-03-29T08:21:13.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:13.364+0000 7fd7d8e00700 1 -- 172.21.15.37:0/2624819402 learned_addr learned my addr 172.21.15.37:0/2624819402 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:21:13.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:13.364+0000 7fd7d3e00700 1 --2- 172.21.15.37:0/2624819402 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd7d4070c70 0x7fd7d41161c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:13.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:13.364+0000 7fd7d8e00700 1 -- 172.21.15.37:0/2624819402 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd7d4070c70 msgr2=0x7fd7d41161c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:13.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:13.364+0000 7fd7d8e00700 1 --2- 172.21.15.37:0/2624819402 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd7d4070c70 0x7fd7d41161c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:13.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:13.364+0000 7fd7d8e00700 1 -- 172.21.15.37:0/2624819402 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7c8005040 con 0x7fd7d4070360 2024-03-29T08:21:13.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:13.364+0000 7fd7d8e00700 1 --2- 172.21.15.37:0/2624819402 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd7d4070360 0x7fd7d4115c80 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fd7c8009360 tx=0x7fd7c8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:21:13.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:13.365+0000 7fd7d0c00700 1 -- 172.21.15.37:0/2624819402 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd7c800e040 con 0x7fd7d4070360 2024-03-29T08:21:13.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:13.365+0000 7fd7d0c00700 1 -- 172.21.15.37:0/2624819402 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd7c8004290 con 0x7fd7d4070360 2024-03-29T08:21:13.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:13.365+0000 7fd7d0c00700 1 -- 172.21.15.37:0/2624819402 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd7c8013d90 con 0x7fd7d4070360 2024-03-29T08:21:13.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:13.365+0000 7fd7db800700 1 -- 172.21.15.37:0/2624819402 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7d410fe60 con 0x7fd7d4070360 2024-03-29T08:21:13.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:13.365+0000 7fd7db800700 1 -- 172.21.15.37:0/2624819402 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7d41103a0 con 0x7fd7d4070360 2024-03-29T08:21:13.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:13.366+0000 7fd7db800700 1 -- 172.21.15.37:0/2624819402 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7d4103b90 con 0x7fd7d4070360 2024-03-29T08:21:13.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:13.367+0000 7fd7d0c00700 1 -- 172.21.15.37:0/2624819402 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fd7c800ba40 con 0x7fd7d4070360 2024-03-29T08:21:13.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:13.368+0000 7fd7d0c00700 1 --2- 172.21.15.37:0/2624819402 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fd7bc06cc30 0x7fd7bc06f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:13.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:13.368+0000 7fd7d0c00700 1 -- 172.21.15.37:0/2624819402 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fd7c808d310 con 0x7fd7d4070360 2024-03-29T08:21:13.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:13.368+0000 7fd7d3e00700 1 --2- 172.21.15.37:0/2624819402 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fd7bc06cc30 0x7fd7bc06f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:13.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:13.369+0000 7fd7d3e00700 1 --2- 172.21.15.37:0/2624819402 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fd7bc06cc30 0x7fd7bc06f0e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fd7d4111520 tx=0x7fd7c0009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:21:13.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:13.373+0000 7fd7d0c00700 1 -- 172.21.15.37:0/2624819402 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fd7c805ad60 con 0x7fd7d4070360 2024-03-29T08:21:13.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:13.597+0000 7fd7db800700 1 -- 172.21.15.37:0/2624819402 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fd7d410bfb0 con 0x7fd7bc06cc30 2024-03-29T08:21:13.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:13.607+0000 7fd7d0c00700 1 -- 172.21.15.37:0/2624819402 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7fd7d410bfb0 con 0x7fd7bc06cc30 2024-03-29T08:21:13.608 INFO:teuthology.orchestra.run.smithi037.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-29T08:21:13.609 INFO:teuthology.orchestra.run.smithi037.stdout:alertmanager.smithi037 smithi037 *:9093,9094 running (11m) 3m ago 24m 14.9M - 0.25.0 c8568f914cd2 2f9f9746ed0f 2024-03-29T08:21:13.609 INFO:teuthology.orchestra.run.smithi037.stdout:crash.smithi037 smithi037 running (7m) 3m ago 24m 6647k - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 178d416a3186 2024-03-29T08:21:13.609 INFO:teuthology.orchestra.run.smithi037.stdout:crash.smithi148 smithi148 running (7m) 3m ago 23m 6656k - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 23666989d58f 2024-03-29T08:21:13.609 INFO:teuthology.orchestra.run.smithi037.stdout:grafana.smithi037 smithi037 *:3000 running (10m) 3m ago 23m 71.9M - 9.4.7 954c08fa6188 6b976fe49952 2024-03-29T08:21:13.609 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi037.iifsmh smithi037 running (3m) 3m ago 17m 10.8M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 2d97f562cc5b 2024-03-29T08:21:13.609 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi037.mmwgxe smithi037 running (3m) 3m ago 17m 10.9M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff ec37037f46e8 2024-03-29T08:21:13.609 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi148.qukpjh smithi148 running (17m) 3m ago 17m 15.3M - 17.2.7-789-g4213e17c cf22eb8683be 8e469782a273 2024-03-29T08:21:13.609 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi148.sqteih smithi148 running (3m) 3m ago 17m 10.7M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 44c15c77037a 2024-03-29T08:21:13.609 INFO:teuthology.orchestra.run.smithi037.stdout:mgr.smithi037.wjikii smithi037 *:8443,9283,8765 running (13m) 3m ago 25m 506M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff c5a1417b9417 2024-03-29T08:21:13.609 INFO:teuthology.orchestra.run.smithi037.stdout:mgr.smithi148.onhgvm smithi148 *:8443,9283,8765 running (12m) 3m ago 22m 450M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff fcfdbce062bd 2024-03-29T08:21:13.609 INFO:teuthology.orchestra.run.smithi037.stdout:mon.smithi037 smithi037 running (9m) 3m ago 25m 54.8M 2048M 18.2.2-1015-gd0e392d9 1cc2f45c17ff eea2060783af 2024-03-29T08:21:13.609 INFO:teuthology.orchestra.run.smithi037.stdout:mon.smithi148 smithi148 running (8m) 3m ago 22m 43.7M 2048M 18.2.2-1015-gd0e392d9 1cc2f45c17ff ad4b2c5748be 2024-03-29T08:21:13.609 INFO:teuthology.orchestra.run.smithi037.stdout:node-exporter.smithi037 smithi037 *:9100 running (11m) 3m ago 24m 7625k - 1.5.0 0da6a335fe13 26b8ae9cb878 2024-03-29T08:21:13.610 INFO:teuthology.orchestra.run.smithi037.stdout:node-exporter.smithi148 smithi148 *:9100 running (11m) 3m ago 22m 8166k - 1.5.0 0da6a335fe13 98414e4e64ed 2024-03-29T08:21:13.610 INFO:teuthology.orchestra.run.smithi037.stdout:osd.0 smithi037 running (7m) 3m ago 21m 104M 1619M 18.2.2-1015-gd0e392d9 1cc2f45c17ff c0c584f6245c 2024-03-29T08:21:13.610 INFO:teuthology.orchestra.run.smithi037.stdout:osd.1 smithi037 running (6m) 3m ago 21m 100M 1619M 18.2.2-1015-gd0e392d9 1cc2f45c17ff a4a2ff4cc83d 2024-03-29T08:21:13.610 INFO:teuthology.orchestra.run.smithi037.stdout:osd.2 smithi037 running (6m) 3m ago 20m 82.3M 1619M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 8b1bc71aa79e 2024-03-29T08:21:13.610 INFO:teuthology.orchestra.run.smithi037.stdout:osd.3 smithi148 running (5m) 3m ago 20m 105M 2642M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 1c656ab8ff86 2024-03-29T08:21:13.610 INFO:teuthology.orchestra.run.smithi037.stdout:osd.4 smithi148 running (4m) 3m ago 19m 97.3M 2642M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 966a9de11d22 2024-03-29T08:21:13.610 INFO:teuthology.orchestra.run.smithi037.stdout:osd.5 smithi148 running (4m) 3m ago 19m 85.8M 2642M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 474c4f9212ce 2024-03-29T08:21:13.610 INFO:teuthology.orchestra.run.smithi037.stdout:prometheus.smithi037 smithi037 *:9095 running (11m) 3m ago 23m 54.4M - 2.43.0 a07b618ecd1d 30c0223ec589 2024-03-29T08:21:13.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:13.610+0000 7fd7db800700 1 -- 172.21.15.37:0/2624819402 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fd7bc06cc30 msgr2=0x7fd7bc06f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:13.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:13.610+0000 7fd7db800700 1 --2- 172.21.15.37:0/2624819402 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fd7bc06cc30 0x7fd7bc06f0e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fd7d4111520 tx=0x7fd7c0009040 comp rx=0 tx=0).stop 2024-03-29T08:21:13.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:13.610+0000 7fd7db800700 1 -- 172.21.15.37:0/2624819402 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd7d4070360 msgr2=0x7fd7d4115c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:13.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:13.610+0000 7fd7db800700 1 --2- 172.21.15.37:0/2624819402 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd7d4070360 0x7fd7d4115c80 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fd7c8009360 tx=0x7fd7c8009e30 comp rx=0 tx=0).stop 2024-03-29T08:21:13.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:13.610+0000 7fd7db800700 1 -- 172.21.15.37:0/2624819402 shutdown_connections 2024-03-29T08:21:13.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:13.610+0000 7fd7db800700 1 --2- 172.21.15.37:0/2624819402 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fd7bc06cc30 0x7fd7bc06f0e0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:13.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:13.610+0000 7fd7db800700 1 --2- 172.21.15.37:0/2624819402 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd7d4070360 0x7fd7d4115c80 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:13.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:13.610+0000 7fd7db800700 1 --2- 172.21.15.37:0/2624819402 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd7d4070c70 0x7fd7d41161c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:13.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:13.610+0000 7fd7db800700 1 -- 172.21.15.37:0/2624819402 >> 172.21.15.37:0/2624819402 conn(0x7fd7d40f89b0 msgr2=0x7fd7d41015f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:21:13.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:13.610+0000 7fd7db800700 1 -- 172.21.15.37:0/2624819402 shutdown_connections 2024-03-29T08:21:13.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:13.610+0000 7fd7db800700 1 -- 172.21.15.37:0/2624819402 wait complete. 2024-03-29T08:21:13.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:13.748+0000 7f122d000700 1 -- 172.21.15.37:0/2260224219 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1228102e20 msgr2=0x7f12281031f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:13.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:13.748+0000 7f122d000700 1 --2- 172.21.15.37:0/2260224219 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1228102e20 0x7f12281031f0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f1218004940 tx=0x7f1218009e30 comp rx=0 tx=0).stop 2024-03-29T08:21:13.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:13.748+0000 7f122d000700 1 -- 172.21.15.37:0/2260224219 shutdown_connections 2024-03-29T08:21:13.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:13.748+0000 7f122d000700 1 --2- 172.21.15.37:0/2260224219 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f12281037c0 0x7f12280fce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:13.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:13.748+0000 7f122d000700 1 --2- 172.21.15.37:0/2260224219 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1228102e20 0x7f12281031f0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:13.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:13.748+0000 7f122d000700 1 -- 172.21.15.37:0/2260224219 >> 172.21.15.37:0/2260224219 conn(0x7f12280f89d0 msgr2=0x7f12280fade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:21:13.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:13.748+0000 7f122d000700 1 -- 172.21.15.37:0/2260224219 shutdown_connections 2024-03-29T08:21:13.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:13.748+0000 7f122d000700 1 -- 172.21.15.37:0/2260224219 wait complete. 2024-03-29T08:21:13.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:13.749+0000 7f122d000700 1 Processor -- start 2024-03-29T08:21:13.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:13.749+0000 7f122d000700 1 -- start start 2024-03-29T08:21:13.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:13.749+0000 7f122d000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1228102e20 0x7f1228115cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:13.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:13.749+0000 7f122d000700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f12281037c0 0x7f1228116230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:13.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:13.749+0000 7f122d000700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1228116880 con 0x7f1228102e20 2024-03-29T08:21:13.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:13.749+0000 7f122d000700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f122810fc50 con 0x7f12281037c0 2024-03-29T08:21:13.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:13.749+0000 7f1225e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1228102e20 0x7f1228115cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:13.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:13.750+0000 7f1225e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1228102e20 0x7f1228115cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57886/0 (socket says 172.21.15.37:57886) 2024-03-29T08:21:13.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:13.750+0000 7f1225e00700 1 -- 172.21.15.37:0/4072557804 learned_addr learned my addr 172.21.15.37:0/4072557804 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:21:13.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:13.750+0000 7f1225e00700 1 -- 172.21.15.37:0/4072557804 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f12281037c0 msgr2=0x7f1228116230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:13.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:13.750+0000 7f1225400700 1 --2- 172.21.15.37:0/4072557804 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f12281037c0 0x7f1228116230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:13.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:13.750+0000 7f1225e00700 1 --2- 172.21.15.37:0/4072557804 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f12281037c0 0x7f1228116230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:13.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:13.750+0000 7f1225e00700 1 -- 172.21.15.37:0/4072557804 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1218005040 con 0x7f1228102e20 2024-03-29T08:21:13.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:13.750+0000 7f1225400700 1 --2- 172.21.15.37:0/4072557804 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f12281037c0 0x7f1228116230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:21:13.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:13.750+0000 7f1225e00700 1 --2- 172.21.15.37:0/4072557804 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1228102e20 0x7f1228115cf0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f1218009360 tx=0x7f1218009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:21:13.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:13.750+0000 7f121e000700 1 -- 172.21.15.37:0/4072557804 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f121800e040 con 0x7f1228102e20 2024-03-29T08:21:13.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:13.751+0000 7f122d000700 1 -- 172.21.15.37:0/4072557804 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f122810fed0 con 0x7f1228102e20 2024-03-29T08:21:13.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:13.751+0000 7f121e000700 1 -- 172.21.15.37:0/4072557804 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f1218013420 con 0x7f1228102e20 2024-03-29T08:21:13.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:13.751+0000 7f121e000700 1 -- 172.21.15.37:0/4072557804 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f121801c5f0 con 0x7f1228102e20 2024-03-29T08:21:13.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:13.751+0000 7f122d000700 1 -- 172.21.15.37:0/4072557804 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1228110490 con 0x7f1228102e20 2024-03-29T08:21:13.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:13.752+0000 7f122d000700 1 -- 172.21.15.37:0/4072557804 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12280fe570 con 0x7f1228102e20 2024-03-29T08:21:13.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:13.753+0000 7f121e000700 1 -- 172.21.15.37:0/4072557804 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f1218013590 con 0x7f1228102e20 2024-03-29T08:21:13.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:13.753+0000 7f121e000700 1 --2- 172.21.15.37:0/4072557804 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f120c06cc30 0x7f120c06f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:13.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:13.754+0000 7f121e000700 1 -- 172.21.15.37:0/4072557804 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f121808d320 con 0x7f1228102e20 2024-03-29T08:21:13.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:13.754+0000 7f1225400700 1 --2- 172.21.15.37:0/4072557804 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f120c06cc30 0x7f120c06f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:13.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:13.754+0000 7f1225400700 1 --2- 172.21.15.37:0/4072557804 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f120c06cc30 0x7f120c06f0e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f1214007fd0 tx=0x7f1214009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:21:13.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:13.758+0000 7f121e000700 1 -- 172.21.15.37:0/4072557804 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f121805ae20 con 0x7f1228102e20 2024-03-29T08:21:14.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:21:13 smithi148 ceph-mon[106692]: from='client.34616 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:21:14.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:21:13 smithi148 ceph-mon[106692]: from='client.34620 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:21:14.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:21:13 smithi148 ceph-mon[106692]: from='client.34624 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:21:14.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:21:13 smithi148 ceph-mon[106692]: pgmap v310: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:21:14.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:14.062+0000 7f122d000700 1 -- 172.21.15.37:0/4072557804 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f1228109e10 con 0x7f1228102e20 2024-03-29T08:21:14.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:14.063+0000 7f121e000700 1 -- 172.21.15.37:0/4072557804 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+847 (secure 0 0 0) 0x7f121805a9b0 con 0x7f1228102e20 2024-03-29T08:21:14.064 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-29T08:21:14.064 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": { 2024-03-29T08:21:14.064 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 2 2024-03-29T08:21:14.064 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:21:14.064 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": { 2024-03-29T08:21:14.064 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 2 2024-03-29T08:21:14.064 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:21:14.064 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": { 2024-03-29T08:21:14.064 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 6 2024-03-29T08:21:14.064 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:21:14.065 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": { 2024-03-29T08:21:14.065 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 17.2.7-789-g4213e17c (4213e17c4de67016ed14f55d22503f4923b3a901) quincy (stable)": 1, 2024-03-29T08:21:14.065 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 3 2024-03-29T08:21:14.065 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:21:14.065 INFO:teuthology.orchestra.run.smithi037.stdout: "overall": { 2024-03-29T08:21:14.065 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 17.2.7-789-g4213e17c (4213e17c4de67016ed14f55d22503f4923b3a901) quincy (stable)": 1, 2024-03-29T08:21:14.065 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 13 2024-03-29T08:21:14.065 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-29T08:21:14.065 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-29T08:21:14.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:14.065+0000 7f122d000700 1 -- 172.21.15.37:0/4072557804 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f120c06cc30 msgr2=0x7f120c06f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:14.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:14.065+0000 7f122d000700 1 --2- 172.21.15.37:0/4072557804 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f120c06cc30 0x7f120c06f0e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f1214007fd0 tx=0x7f1214009040 comp rx=0 tx=0).stop 2024-03-29T08:21:14.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:14.065+0000 7f122d000700 1 -- 172.21.15.37:0/4072557804 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1228102e20 msgr2=0x7f1228115cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:14.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:14.065+0000 7f122d000700 1 --2- 172.21.15.37:0/4072557804 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1228102e20 0x7f1228115cf0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f1218009360 tx=0x7f1218009e30 comp rx=0 tx=0).stop 2024-03-29T08:21:14.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:14.065+0000 7f122d000700 1 -- 172.21.15.37:0/4072557804 shutdown_connections 2024-03-29T08:21:14.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:14.065+0000 7f122d000700 1 --2- 172.21.15.37:0/4072557804 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f120c06cc30 0x7f120c06f0e0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:14.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:14.065+0000 7f122d000700 1 --2- 172.21.15.37:0/4072557804 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1228102e20 0x7f1228115cf0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:14.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:14.065+0000 7f122d000700 1 --2- 172.21.15.37:0/4072557804 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f12281037c0 0x7f1228116230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:14.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:14.065+0000 7f122d000700 1 -- 172.21.15.37:0/4072557804 >> 172.21.15.37:0/4072557804 conn(0x7f12280f89d0 msgr2=0x7f12280f9ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:21:14.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:14.065+0000 7f122d000700 1 -- 172.21.15.37:0/4072557804 shutdown_connections 2024-03-29T08:21:14.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:14.065+0000 7f122d000700 1 -- 172.21.15.37:0/4072557804 wait complete. 2024-03-29T08:21:14.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:21:13 smithi037 ceph-mon[142325]: from='client.34616 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:21:14.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:21:13 smithi037 ceph-mon[142325]: from='client.34620 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:21:14.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:21:13 smithi037 ceph-mon[142325]: from='client.34624 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:21:14.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:21:13 smithi037 ceph-mon[142325]: pgmap v310: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:21:14.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:14.203+0000 7fcca9400700 1 -- 172.21.15.37:0/2095856237 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcca4070c10 msgr2=0x7fcca4109190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:14.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:14.203+0000 7fcca9400700 1 --2- 172.21.15.37:0/2095856237 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcca4070c10 0x7fcca4109190 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fcc8c004970 tx=0x7fcc8c009e30 comp rx=0 tx=0).stop 2024-03-29T08:21:14.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:14.203+0000 7fcca9400700 1 -- 172.21.15.37:0/2095856237 shutdown_connections 2024-03-29T08:21:14.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:14.203+0000 7fcca9400700 1 --2- 172.21.15.37:0/2095856237 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcca4070c10 0x7fcca4109190 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:14.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:14.203+0000 7fcca9400700 1 --2- 172.21.15.37:0/2095856237 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcca4070300 0x7fcca40706d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:14.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:14.203+0000 7fcca9400700 1 -- 172.21.15.37:0/2095856237 >> 172.21.15.37:0/2095856237 conn(0x7fcca40f8970 msgr2=0x7fcca40fad80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:21:14.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:14.203+0000 7fcca9400700 1 -- 172.21.15.37:0/2095856237 shutdown_connections 2024-03-29T08:21:14.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:14.203+0000 7fcca9400700 1 -- 172.21.15.37:0/2095856237 wait complete. 2024-03-29T08:21:14.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:14.204+0000 7fcca9400700 1 Processor -- start 2024-03-29T08:21:14.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:14.204+0000 7fcca9400700 1 -- start start 2024-03-29T08:21:14.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:14.204+0000 7fcca9400700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcca4070300 0x7fcca4115cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:14.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:14.204+0000 7fcca9400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcca4070c10 0x7fcca4116200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:14.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:14.204+0000 7fcca9400700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcca4116850 con 0x7fcca4070c10 2024-03-29T08:21:14.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:14.204+0000 7fcca9400700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcca410fc20 con 0x7fcca4070300 2024-03-29T08:21:14.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:14.205+0000 7fcc9be00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcca4070c10 0x7fcca4116200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:14.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:14.205+0000 7fcc9be00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcca4070c10 0x7fcca4116200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57910/0 (socket says 172.21.15.37:57910) 2024-03-29T08:21:14.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:14.205+0000 7fcc9be00700 1 -- 172.21.15.37:0/2583317990 learned_addr learned my addr 172.21.15.37:0/2583317990 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:21:14.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:14.205+0000 7fcca2800700 1 --2- 172.21.15.37:0/2583317990 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcca4070300 0x7fcca4115cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:14.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:14.205+0000 7fcc9be00700 1 -- 172.21.15.37:0/2583317990 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcca4070300 msgr2=0x7fcca4115cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:14.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:14.205+0000 7fcc9be00700 1 --2- 172.21.15.37:0/2583317990 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcca4070300 0x7fcca4115cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:14.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:14.205+0000 7fcc9be00700 1 -- 172.21.15.37:0/2583317990 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc8c005040 con 0x7fcca4070c10 2024-03-29T08:21:14.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:14.205+0000 7fcc9be00700 1 --2- 172.21.15.37:0/2583317990 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcca4070c10 0x7fcca4116200 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fcc8c009360 tx=0x7fcc8c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:21:14.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:14.206+0000 7fcc9aa00700 1 -- 172.21.15.37:0/2583317990 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcc8c00e040 con 0x7fcca4070c10 2024-03-29T08:21:14.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:14.206+0000 7fcc9aa00700 1 -- 172.21.15.37:0/2583317990 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fcc8c004290 con 0x7fcca4070c10 2024-03-29T08:21:14.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:14.206+0000 7fcca9400700 1 -- 172.21.15.37:0/2583317990 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcca410fea0 con 0x7fcca4070c10 2024-03-29T08:21:14.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:14.206+0000 7fcc9aa00700 1 -- 172.21.15.37:0/2583317990 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcc8c01c3f0 con 0x7fcca4070c10 2024-03-29T08:21:14.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:14.206+0000 7fcca9400700 1 -- 172.21.15.37:0/2583317990 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcca41103e0 con 0x7fcca4070c10 2024-03-29T08:21:14.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:14.207+0000 7fcca9400700 1 -- 172.21.15.37:0/2583317990 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcca4071a40 con 0x7fcca4070c10 2024-03-29T08:21:14.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:14.208+0000 7fcc9aa00700 1 -- 172.21.15.37:0/2583317990 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fcc8c013420 con 0x7fcca4070c10 2024-03-29T08:21:14.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:14.210+0000 7fcc9aa00700 1 --2- 172.21.15.37:0/2583317990 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fcc8806cc30 0x7fcc8806f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:14.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:14.210+0000 7fcc9aa00700 1 -- 172.21.15.37:0/2583317990 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fcc8c08d4d0 con 0x7fcca4070c10 2024-03-29T08:21:14.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:14.210+0000 7fcca2800700 1 --2- 172.21.15.37:0/2583317990 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fcc8806cc30 0x7fcc8806f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:14.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:14.210+0000 7fcca2800700 1 --2- 172.21.15.37:0/2583317990 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fcc8806cc30 0x7fcc8806f0e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fcc94004990 tx=0x7fcc94009ee0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:21:14.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:14.215+0000 7fcc9aa00700 1 -- 172.21.15.37:0/2583317990 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fcc8c05af20 con 0x7fcca4070c10 2024-03-29T08:21:14.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:14.474+0000 7fcca9400700 1 -- 172.21.15.37:0/2583317990 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fcca410bfe0 con 0x7fcca4070c10 2024-03-29T08:21:14.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:14.474+0000 7fcc9aa00700 1 -- 172.21.15.37:0/2583317990 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 28 v28) v1 ==== 76+0+1841 (secure 0 0 0) 0x7fcc8c05aab0 con 0x7fcca4070c10 2024-03-29T08:21:14.476 INFO:teuthology.orchestra.run.smithi037.stdout:e28 2024-03-29T08:21:14.476 INFO:teuthology.orchestra.run.smithi037.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-29T08:21:14.476 INFO:teuthology.orchestra.run.smithi037.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T08:21:14.476 INFO:teuthology.orchestra.run.smithi037.stdout:legacy client fscid: 1 2024-03-29T08:21:14.476 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:21:14.476 INFO:teuthology.orchestra.run.smithi037.stdout:Filesystem 'cephfs' (1) 2024-03-29T08:21:14.476 INFO:teuthology.orchestra.run.smithi037.stdout:fs_name cephfs 2024-03-29T08:21:14.476 INFO:teuthology.orchestra.run.smithi037.stdout:epoch 21 2024-03-29T08:21:14.476 INFO:teuthology.orchestra.run.smithi037.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-03-29T08:21:14.477 INFO:teuthology.orchestra.run.smithi037.stdout:created 2024-03-29T08:03:29.241269+0000 2024-03-29T08:21:14.477 INFO:teuthology.orchestra.run.smithi037.stdout:modified 2024-03-29T08:17:16.237238+0000 2024-03-29T08:21:14.477 INFO:teuthology.orchestra.run.smithi037.stdout:tableserver 0 2024-03-29T08:21:14.477 INFO:teuthology.orchestra.run.smithi037.stdout:root 0 2024-03-29T08:21:14.477 INFO:teuthology.orchestra.run.smithi037.stdout:session_timeout 60 2024-03-29T08:21:14.477 INFO:teuthology.orchestra.run.smithi037.stdout:session_autoclose 300 2024-03-29T08:21:14.477 INFO:teuthology.orchestra.run.smithi037.stdout:max_file_size 1099511627776 2024-03-29T08:21:14.477 INFO:teuthology.orchestra.run.smithi037.stdout:max_xattr_size 65536 2024-03-29T08:21:14.477 INFO:teuthology.orchestra.run.smithi037.stdout:required_client_features {} 2024-03-29T08:21:14.477 INFO:teuthology.orchestra.run.smithi037.stdout:last_failure 0 2024-03-29T08:21:14.477 INFO:teuthology.orchestra.run.smithi037.stdout:last_failure_osd_epoch 82 2024-03-29T08:21:14.477 INFO:teuthology.orchestra.run.smithi037.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T08:21:14.477 INFO:teuthology.orchestra.run.smithi037.stdout:max_mds 2 2024-03-29T08:21:14.478 INFO:teuthology.orchestra.run.smithi037.stdout:in 0,1 2024-03-29T08:21:14.478 INFO:teuthology.orchestra.run.smithi037.stdout:up {} 2024-03-29T08:21:14.478 INFO:teuthology.orchestra.run.smithi037.stdout:failed 0,1 2024-03-29T08:21:14.478 INFO:teuthology.orchestra.run.smithi037.stdout:damaged 2024-03-29T08:21:14.478 INFO:teuthology.orchestra.run.smithi037.stdout:stopped 2024-03-29T08:21:14.478 INFO:teuthology.orchestra.run.smithi037.stdout:data_pools [3] 2024-03-29T08:21:14.478 INFO:teuthology.orchestra.run.smithi037.stdout:metadata_pool 2 2024-03-29T08:21:14.478 INFO:teuthology.orchestra.run.smithi037.stdout:inline_data disabled 2024-03-29T08:21:14.478 INFO:teuthology.orchestra.run.smithi037.stdout:balancer 2024-03-29T08:21:14.478 INFO:teuthology.orchestra.run.smithi037.stdout:bal_rank_mask -1 2024-03-29T08:21:14.478 INFO:teuthology.orchestra.run.smithi037.stdout:standby_count_wanted 1 2024-03-29T08:21:14.481 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:21:14.481 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:21:14.481 INFO:teuthology.orchestra.run.smithi037.stdout:Standby daemons: 2024-03-29T08:21:14.481 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:21:14.481 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi148.qukpjh{-1:24319} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.148:6826/2641327912,v1:172.21.15.148:6827/2641327912] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:21:14.482 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi037.mmwgxe{-1:34440} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.37:6828/1248142005,v1:172.21.15.37:6829/1248142005] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:21:14.482 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi037.iifsmh{-1:34444} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.37:6826/2060882959,v1:172.21.15.37:6827/2060882959] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:21:14.482 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi148.sqteih{-1:44395} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.148:6824/4078652242,v1:172.21.15.148:6825/4078652242] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:21:14.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:14.477+0000 7fcca9400700 1 -- 172.21.15.37:0/2583317990 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fcc8806cc30 msgr2=0x7fcc8806f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:14.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:14.477+0000 7fcca9400700 1 --2- 172.21.15.37:0/2583317990 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fcc8806cc30 0x7fcc8806f0e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fcc94004990 tx=0x7fcc94009ee0 comp rx=0 tx=0).stop 2024-03-29T08:21:14.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:14.477+0000 7fcca9400700 1 -- 172.21.15.37:0/2583317990 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcca4070c10 msgr2=0x7fcca4116200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:14.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:14.477+0000 7fcca9400700 1 --2- 172.21.15.37:0/2583317990 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcca4070c10 0x7fcca4116200 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fcc8c009360 tx=0x7fcc8c009e30 comp rx=0 tx=0).stop 2024-03-29T08:21:14.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:14.477+0000 7fcca9400700 1 -- 172.21.15.37:0/2583317990 shutdown_connections 2024-03-29T08:21:14.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:14.477+0000 7fcca9400700 1 --2- 172.21.15.37:0/2583317990 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fcc8806cc30 0x7fcc8806f0e0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:14.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:14.477+0000 7fcca9400700 1 --2- 172.21.15.37:0/2583317990 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcca4070300 0x7fcca4115cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:14.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:14.477+0000 7fcca9400700 1 --2- 172.21.15.37:0/2583317990 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcca4070c10 0x7fcca4116200 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:14.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:14.477+0000 7fcca9400700 1 -- 172.21.15.37:0/2583317990 >> 172.21.15.37:0/2583317990 conn(0x7fcca40f8970 msgr2=0x7fcca40f9de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:21:14.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:14.477+0000 7fcca9400700 1 -- 172.21.15.37:0/2583317990 shutdown_connections 2024-03-29T08:21:14.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:14.477+0000 7fcca9400700 1 -- 172.21.15.37:0/2583317990 wait complete. 2024-03-29T08:21:14.484 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 28 2024-03-29T08:21:14.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:14.614+0000 7fa933800700 1 -- 172.21.15.37:0/4221125635 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa92c0fe830 msgr2=0x7fa92c106f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:14.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:14.614+0000 7fa933800700 1 --2- 172.21.15.37:0/4221125635 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa92c0fe830 0x7fa92c106f00 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fa918004940 tx=0x7fa918009e30 comp rx=0 tx=0).stop 2024-03-29T08:21:14.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:14.615+0000 7fa933800700 1 -- 172.21.15.37:0/4221125635 shutdown_connections 2024-03-29T08:21:14.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:14.615+0000 7fa933800700 1 --2- 172.21.15.37:0/4221125635 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa92c0fe830 0x7fa92c106f00 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:14.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:14.615+0000 7fa933800700 1 --2- 172.21.15.37:0/4221125635 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa92c0fde90 0x7fa92c0fe260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:14.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:14.615+0000 7fa933800700 1 -- 172.21.15.37:0/4221125635 >> 172.21.15.37:0/4221125635 conn(0x7fa92c072bc0 msgr2=0x7fa92c072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:21:14.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:14.615+0000 7fa933800700 1 -- 172.21.15.37:0/4221125635 shutdown_connections 2024-03-29T08:21:14.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:14.615+0000 7fa933800700 1 -- 172.21.15.37:0/4221125635 wait complete. 2024-03-29T08:21:14.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:14.616+0000 7fa933800700 1 Processor -- start 2024-03-29T08:21:14.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:14.616+0000 7fa933800700 1 -- start start 2024-03-29T08:21:14.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:14.617+0000 7fa933800700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa92c0fde90 0x7fa92c0fbd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:14.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:14.617+0000 7fa933800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa92c0fe830 0x7fa92c0fc290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:14.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:14.617+0000 7fa933800700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa92c0fc820 con 0x7fa92c0fe830 2024-03-29T08:21:14.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:14.617+0000 7fa933800700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa92c0fc960 con 0x7fa92c0fde90 2024-03-29T08:21:14.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:14.617+0000 7fa92be00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa92c0fe830 0x7fa92c0fc290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:14.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:14.617+0000 7fa92be00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa92c0fe830 0x7fa92c0fc290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57938/0 (socket says 172.21.15.37:57938) 2024-03-29T08:21:14.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:14.617+0000 7fa92be00700 1 -- 172.21.15.37:0/4156307960 learned_addr learned my addr 172.21.15.37:0/4156307960 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:21:14.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:14.617+0000 7fa930e00700 1 --2- 172.21.15.37:0/4156307960 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa92c0fde90 0x7fa92c0fbd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:14.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:14.618+0000 7fa92be00700 1 -- 172.21.15.37:0/4156307960 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa92c0fde90 msgr2=0x7fa92c0fbd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:14.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:14.618+0000 7fa92be00700 1 --2- 172.21.15.37:0/4156307960 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa92c0fde90 0x7fa92c0fbd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:14.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:14.618+0000 7fa92be00700 1 -- 172.21.15.37:0/4156307960 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa918005040 con 0x7fa92c0fe830 2024-03-29T08:21:14.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:14.618+0000 7fa92be00700 1 --2- 172.21.15.37:0/4156307960 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa92c0fe830 0x7fa92c0fc290 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fa918000c00 tx=0x7fa918009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:21:14.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:14.618+0000 7fa928c00700 1 -- 172.21.15.37:0/4156307960 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa918021070 con 0x7fa92c0fe830 2024-03-29T08:21:14.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:14.618+0000 7fa928c00700 1 -- 172.21.15.37:0/4156307960 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fa918013e00 con 0x7fa92c0fe830 2024-03-29T08:21:14.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:14.618+0000 7fa933800700 1 -- 172.21.15.37:0/4156307960 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa92c0fcbb0 con 0x7fa92c0fe830 2024-03-29T08:21:14.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:14.618+0000 7fa928c00700 1 -- 172.21.15.37:0/4156307960 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa91800e070 con 0x7fa92c0fe830 2024-03-29T08:21:14.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:14.618+0000 7fa933800700 1 -- 172.21.15.37:0/4156307960 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa92c11a130 con 0x7fa92c0fe830 2024-03-29T08:21:14.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:14.619+0000 7fa933800700 1 -- 172.21.15.37:0/4156307960 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa92c065e30 con 0x7fa92c0fe830 2024-03-29T08:21:14.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:14.621+0000 7fa928c00700 1 -- 172.21.15.37:0/4156307960 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fa918013950 con 0x7fa92c0fe830 2024-03-29T08:21:14.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:14.623+0000 7fa928c00700 1 --2- 172.21.15.37:0/4156307960 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fa91406cc30 0x7fa91406f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:14.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:14.623+0000 7fa928c00700 1 -- 172.21.15.37:0/4156307960 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fa91808cee0 con 0x7fa92c0fe830 2024-03-29T08:21:14.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:14.623+0000 7fa930e00700 1 --2- 172.21.15.37:0/4156307960 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fa91406cc30 0x7fa91406f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:14.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:14.623+0000 7fa930e00700 1 --2- 172.21.15.37:0/4156307960 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fa91406cc30 0x7fa91406f0e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fa920009b80 tx=0x7fa92000b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:21:14.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:14.626+0000 7fa928c00700 1 -- 172.21.15.37:0/4156307960 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fa918056fe0 con 0x7fa92c0fe830 2024-03-29T08:21:14.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:14.858+0000 7fa933800700 1 -- 172.21.15.37:0/4156307960 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa92c04bf40 con 0x7fa91406cc30 2024-03-29T08:21:14.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:14.859+0000 7fa928c00700 1 -- 172.21.15.37:0/4156307960 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fa92c04bf40 con 0x7fa91406cc30 2024-03-29T08:21:14.860 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-29T08:21:14.860 INFO:teuthology.orchestra.run.smithi037.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59", 2024-03-29T08:21:14.860 INFO:teuthology.orchestra.run.smithi037.stdout: "in_progress": true, 2024-03-29T08:21:14.860 INFO:teuthology.orchestra.run.smithi037.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-29T08:21:14.860 INFO:teuthology.orchestra.run.smithi037.stdout: "services_complete": [ 2024-03-29T08:21:14.860 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr", 2024-03-29T08:21:14.861 INFO:teuthology.orchestra.run.smithi037.stdout: "mon", 2024-03-29T08:21:14.861 INFO:teuthology.orchestra.run.smithi037.stdout: "crash", 2024-03-29T08:21:14.861 INFO:teuthology.orchestra.run.smithi037.stdout: "osd" 2024-03-29T08:21:14.861 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-29T08:21:14.861 INFO:teuthology.orchestra.run.smithi037.stdout: "progress": "15/21 daemons upgraded", 2024-03-29T08:21:14.861 INFO:teuthology.orchestra.run.smithi037.stdout: "message": "Currently upgrading mds daemons", 2024-03-29T08:21:14.861 INFO:teuthology.orchestra.run.smithi037.stdout: "is_paused": false 2024-03-29T08:21:14.861 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-29T08:21:14.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:14.861+0000 7fa933800700 1 -- 172.21.15.37:0/4156307960 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fa91406cc30 msgr2=0x7fa91406f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:14.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:14.861+0000 7fa933800700 1 --2- 172.21.15.37:0/4156307960 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fa91406cc30 0x7fa91406f0e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fa920009b80 tx=0x7fa92000b040 comp rx=0 tx=0).stop 2024-03-29T08:21:14.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:14.861+0000 7fa933800700 1 -- 172.21.15.37:0/4156307960 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa92c0fe830 msgr2=0x7fa92c0fc290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:14.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:14.861+0000 7fa933800700 1 --2- 172.21.15.37:0/4156307960 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa92c0fe830 0x7fa92c0fc290 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fa918000c00 tx=0x7fa918009e30 comp rx=0 tx=0).stop 2024-03-29T08:21:14.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:14.861+0000 7fa933800700 1 -- 172.21.15.37:0/4156307960 shutdown_connections 2024-03-29T08:21:14.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:14.861+0000 7fa933800700 1 --2- 172.21.15.37:0/4156307960 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fa91406cc30 0x7fa91406f0e0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:14.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:14.861+0000 7fa933800700 1 --2- 172.21.15.37:0/4156307960 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa92c0fde90 0x7fa92c0fbd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:14.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:14.861+0000 7fa933800700 1 --2- 172.21.15.37:0/4156307960 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa92c0fe830 0x7fa92c0fc290 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:14.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:14.861+0000 7fa933800700 1 -- 172.21.15.37:0/4156307960 >> 172.21.15.37:0/4156307960 conn(0x7fa92c072bc0 msgr2=0x7fa92c0ff3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:21:14.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:14.862+0000 7fa933800700 1 -- 172.21.15.37:0/4156307960 shutdown_connections 2024-03-29T08:21:14.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:14.862+0000 7fa933800700 1 -- 172.21.15.37:0/4156307960 wait complete. 2024-03-29T08:21:15.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:14.998+0000 7fbed0200700 1 -- 172.21.15.37:0/880346224 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbec81037a0 msgr2=0x7fbec80fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:15.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:14.998+0000 7fbed0200700 1 --2- 172.21.15.37:0/880346224 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbec81037a0 0x7fbec80fce10 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fbeb4004940 tx=0x7fbeb4009e30 comp rx=0 tx=0).stop 2024-03-29T08:21:15.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:14.998+0000 7fbed0200700 1 -- 172.21.15.37:0/880346224 shutdown_connections 2024-03-29T08:21:15.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:14.998+0000 7fbed0200700 1 --2- 172.21.15.37:0/880346224 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbec81037a0 0x7fbec80fce10 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:15.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:14.998+0000 7fbed0200700 1 --2- 172.21.15.37:0/880346224 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbec8102e00 0x7fbec81031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:15.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:14.998+0000 7fbed0200700 1 -- 172.21.15.37:0/880346224 >> 172.21.15.37:0/880346224 conn(0x7fbec80f89b0 msgr2=0x7fbec80fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:21:15.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:14.999+0000 7fbed0200700 1 -- 172.21.15.37:0/880346224 shutdown_connections 2024-03-29T08:21:15.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:14.999+0000 7fbed0200700 1 -- 172.21.15.37:0/880346224 wait complete. 2024-03-29T08:21:15.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:14.999+0000 7fbed0200700 1 Processor -- start 2024-03-29T08:21:15.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:15.000+0000 7fbed0200700 1 -- start start 2024-03-29T08:21:15.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:15.000+0000 7fbed0200700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbec8102e00 0x7fbec8115cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:15.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:15.000+0000 7fbed0200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbec81037a0 0x7fbec8116230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:15.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:15.000+0000 7fbed0200700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbec8116910 con 0x7fbec81037a0 2024-03-29T08:21:15.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:15.000+0000 7fbed0200700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbec810fc50 con 0x7fbec8102e00 2024-03-29T08:21:15.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:15.000+0000 7fbecce00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbec81037a0 0x7fbec8116230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:15.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:15.000+0000 7fbecce00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbec81037a0 0x7fbec8116230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57960/0 (socket says 172.21.15.37:57960) 2024-03-29T08:21:15.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:15.000+0000 7fbecce00700 1 -- 172.21.15.37:0/919666301 learned_addr learned my addr 172.21.15.37:0/919666301 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:21:15.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:15.000+0000 7fbecd800700 1 --2- 172.21.15.37:0/919666301 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbec8102e00 0x7fbec8115cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:15.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:15.000+0000 7fbecce00700 1 -- 172.21.15.37:0/919666301 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbec8102e00 msgr2=0x7fbec8115cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:15.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:15.000+0000 7fbecce00700 1 --2- 172.21.15.37:0/919666301 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbec8102e00 0x7fbec8115cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:15.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:15.001+0000 7fbecce00700 1 -- 172.21.15.37:0/919666301 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbeb4005040 con 0x7fbec81037a0 2024-03-29T08:21:15.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:15.001+0000 7fbecce00700 1 --2- 172.21.15.37:0/919666301 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbec81037a0 0x7fbec8116230 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fbeb4009390 tx=0x7fbeb4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:21:15.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:15.001+0000 7fbec5600700 1 -- 172.21.15.37:0/919666301 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbeb400e040 con 0x7fbec81037a0 2024-03-29T08:21:15.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:15.001+0000 7fbec5600700 1 -- 172.21.15.37:0/919666301 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fbeb4013420 con 0x7fbec81037a0 2024-03-29T08:21:15.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:15.001+0000 7fbec5600700 1 -- 172.21.15.37:0/919666301 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbeb401c630 con 0x7fbec81037a0 2024-03-29T08:21:15.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:15.001+0000 7fbed0200700 1 -- 172.21.15.37:0/919666301 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbec810fe70 con 0x7fbec81037a0 2024-03-29T08:21:15.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:15.001+0000 7fbed0200700 1 -- 172.21.15.37:0/919666301 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbec8110430 con 0x7fbec81037a0 2024-03-29T08:21:15.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:15.002+0000 7fbed0200700 1 -- 172.21.15.37:0/919666301 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbec80fe550 con 0x7fbec81037a0 2024-03-29T08:21:15.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:15.003+0000 7fbec5600700 1 -- 172.21.15.37:0/919666301 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fbeb4013a00 con 0x7fbec81037a0 2024-03-29T08:21:15.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:15.004+0000 7fbec5600700 1 --2- 172.21.15.37:0/919666301 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fbeb006cc30 0x7fbeb006f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:15.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:15.004+0000 7fbec5600700 1 -- 172.21.15.37:0/919666301 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fbeb408d2c0 con 0x7fbec81037a0 2024-03-29T08:21:15.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:15.004+0000 7fbecd800700 1 --2- 172.21.15.37:0/919666301 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fbeb006cc30 0x7fbeb006f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:15.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:15.005+0000 7fbecd800700 1 --2- 172.21.15.37:0/919666301 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fbeb006cc30 0x7fbeb006f0e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fbebc006fd0 tx=0x7fbebc008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:21:15.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:15.009+0000 7fbec5600700 1 -- 172.21.15.37:0/919666301 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fbeb405ad90 con 0x7fbec81037a0 2024-03-29T08:21:15.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:21:14 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/4072557804' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:21:15.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:21:14 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/2583317990' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T08:21:15.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:21:14 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/4072557804' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:21:15.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:21:14 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/2583317990' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T08:21:15.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:15.307+0000 7fbed0200700 1 -- 172.21.15.37:0/919666301 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fbec805f5f0 con 0x7fbec81037a0 2024-03-29T08:21:15.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:15.307+0000 7fbec5600700 1 -- 172.21.15.37:0/919666301 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7fbeb405a920 con 0x7fbec81037a0 2024-03-29T08:21:15.307 INFO:teuthology.orchestra.run.smithi037.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-03-29T08:21:15.308 INFO:teuthology.orchestra.run.smithi037.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-03-29T08:21:15.308 INFO:teuthology.orchestra.run.smithi037.stdout: fs cephfs is degraded 2024-03-29T08:21:15.308 INFO:teuthology.orchestra.run.smithi037.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-03-29T08:21:15.308 INFO:teuthology.orchestra.run.smithi037.stdout: fs cephfs is offline because no MDS is active for it. 2024-03-29T08:21:15.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:15.309+0000 7fbed0200700 1 -- 172.21.15.37:0/919666301 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fbeb006cc30 msgr2=0x7fbeb006f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:15.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:15.309+0000 7fbed0200700 1 --2- 172.21.15.37:0/919666301 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fbeb006cc30 0x7fbeb006f0e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fbebc006fd0 tx=0x7fbebc008040 comp rx=0 tx=0).stop 2024-03-29T08:21:15.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:15.309+0000 7fbed0200700 1 -- 172.21.15.37:0/919666301 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbec81037a0 msgr2=0x7fbec8116230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:15.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:15.309+0000 7fbed0200700 1 --2- 172.21.15.37:0/919666301 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbec81037a0 0x7fbec8116230 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fbeb4009390 tx=0x7fbeb4009e30 comp rx=0 tx=0).stop 2024-03-29T08:21:15.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:15.310+0000 7fbed0200700 1 -- 172.21.15.37:0/919666301 shutdown_connections 2024-03-29T08:21:15.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:15.310+0000 7fbed0200700 1 --2- 172.21.15.37:0/919666301 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fbeb006cc30 0x7fbeb006f0e0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:15.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:15.310+0000 7fbed0200700 1 --2- 172.21.15.37:0/919666301 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbec8102e00 0x7fbec8115cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:15.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:15.310+0000 7fbed0200700 1 --2- 172.21.15.37:0/919666301 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbec81037a0 0x7fbec8116230 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:15.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:15.310+0000 7fbed0200700 1 -- 172.21.15.37:0/919666301 >> 172.21.15.37:0/919666301 conn(0x7fbec80f89b0 msgr2=0x7fbec8070680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:21:15.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:15.310+0000 7fbed0200700 1 -- 172.21.15.37:0/919666301 shutdown_connections 2024-03-29T08:21:15.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:15.310+0000 7fbed0200700 1 -- 172.21.15.37:0/919666301 wait complete. 2024-03-29T08:21:16.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:21:15 smithi148 ceph-mon[106692]: from='client.34636 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:21:16.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:21:15 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/919666301' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T08:21:16.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:21:15 smithi148 ceph-mon[106692]: pgmap v311: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:21:16.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:21:15 smithi037 ceph-mon[142325]: from='client.34636 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:21:16.178 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:21:15 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/919666301' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T08:21:16.178 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:21:15 smithi037 ceph-mon[142325]: pgmap v311: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:21:18.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:21:17 smithi148 ceph-mon[106692]: pgmap v312: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:21:18.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:21:17 smithi037 ceph-mon[142325]: pgmap v312: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:21:20.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:21:20 smithi148 ceph-mon[106692]: pgmap v313: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:21:20.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:21:20 smithi037 ceph-mon[142325]: pgmap v313: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:21:22.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:21:21 smithi148 ceph-mon[106692]: pgmap v314: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:21:22.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:21:21 smithi037 ceph-mon[142325]: pgmap v314: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:21:24.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:21:23 smithi148 ceph-mon[106692]: pgmap v315: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:21:24.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:21:23 smithi037 ceph-mon[142325]: pgmap v315: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:21:25.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:21:25 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:21:25.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:21:25 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:21:26.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:21:26 smithi148 ceph-mon[106692]: pgmap v316: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:21:26.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:21:26 smithi037 ceph-mon[142325]: pgmap v316: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:21:28.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:21:27 smithi148 ceph-mon[106692]: pgmap v317: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:21:28.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:21:27 smithi037 ceph-mon[142325]: pgmap v317: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:21:30.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:21:30 smithi148 ceph-mon[106692]: pgmap v318: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:21:30.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:21:30 smithi037 ceph-mon[142325]: pgmap v318: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:21:32.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:21:31 smithi148 ceph-mon[106692]: pgmap v319: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:21:32.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:21:31 smithi037 ceph-mon[142325]: pgmap v319: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:21:34.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:21:33 smithi148 ceph-mon[106692]: pgmap v320: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:21:34.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:21:33 smithi037 ceph-mon[142325]: pgmap v320: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:21:36.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:21:35 smithi148 ceph-mon[106692]: pgmap v321: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:21:36.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:21:35 smithi037 ceph-mon[142325]: pgmap v321: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:21:38.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:21:37 smithi148 ceph-mon[106692]: pgmap v322: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:21:38.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:21:37 smithi037 ceph-mon[142325]: pgmap v322: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:21:40.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:21:40 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:21:40.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:21:40 smithi148 ceph-mon[106692]: pgmap v323: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:21:40.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:21:40 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:21:40.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:21:40 smithi037 ceph-mon[142325]: pgmap v323: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:21:42.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:21:41 smithi148 ceph-mon[106692]: pgmap v324: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:21:42.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:21:41 smithi037 ceph-mon[142325]: pgmap v324: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:21:44.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:21:43 smithi148 ceph-mon[106692]: pgmap v325: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:21:44.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:21:43 smithi037 ceph-mon[142325]: pgmap v325: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:21:45.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:45.463+0000 7fe557c00700 1 -- 172.21.15.37:0/2220813001 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe550071f90 msgr2=0x7fe550070360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:45.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:45.463+0000 7fe557c00700 1 --2- 172.21.15.37:0/2220813001 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe550071f90 0x7fe550070360 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fe544004970 tx=0x7fe544009e30 comp rx=0 tx=0).stop 2024-03-29T08:21:45.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:45.463+0000 7fe557c00700 1 -- 172.21.15.37:0/2220813001 shutdown_connections 2024-03-29T08:21:45.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:45.464+0000 7fe557c00700 1 --2- 172.21.15.37:0/2220813001 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe5500708a0 0x7fe550070d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:45.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:45.464+0000 7fe557c00700 1 --2- 172.21.15.37:0/2220813001 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe550071f90 0x7fe550070360 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:45.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:45.464+0000 7fe557c00700 1 -- 172.21.15.37:0/2220813001 >> 172.21.15.37:0/2220813001 conn(0x7fe5500f8970 msgr2=0x7fe5500fad80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:21:45.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:45.464+0000 7fe557c00700 1 -- 172.21.15.37:0/2220813001 shutdown_connections 2024-03-29T08:21:45.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:45.464+0000 7fe557c00700 1 -- 172.21.15.37:0/2220813001 wait complete. 2024-03-29T08:21:45.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:45.464+0000 7fe557c00700 1 Processor -- start 2024-03-29T08:21:45.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:45.465+0000 7fe557c00700 1 -- start start 2024-03-29T08:21:45.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:45.465+0000 7fe557c00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe5500708a0 0x7fe55011a140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:45.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:45.465+0000 7fe557c00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe550071f90 0x7fe55011a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:45.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:45.465+0000 7fe557c00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe55011acd0 con 0x7fe550071f90 2024-03-29T08:21:45.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:45.465+0000 7fe557c00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5501140a0 con 0x7fe5500708a0 2024-03-29T08:21:45.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:45.465+0000 7fe54fe00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe550071f90 0x7fe55011a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:45.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:45.465+0000 7fe54fe00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe550071f90 0x7fe55011a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52074/0 (socket says 172.21.15.37:52074) 2024-03-29T08:21:45.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:45.465+0000 7fe54fe00700 1 -- 172.21.15.37:0/1336654690 learned_addr learned my addr 172.21.15.37:0/1336654690 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:21:45.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:45.466+0000 7fe555200700 1 --2- 172.21.15.37:0/1336654690 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe5500708a0 0x7fe55011a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:45.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:45.466+0000 7fe54fe00700 1 -- 172.21.15.37:0/1336654690 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe5500708a0 msgr2=0x7fe55011a140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:45.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:45.466+0000 7fe54fe00700 1 --2- 172.21.15.37:0/1336654690 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe5500708a0 0x7fe55011a140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:45.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:45.466+0000 7fe54fe00700 1 -- 172.21.15.37:0/1336654690 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe544005040 con 0x7fe550071f90 2024-03-29T08:21:45.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:45.466+0000 7fe54fe00700 1 --2- 172.21.15.37:0/1336654690 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe550071f90 0x7fe55011a680 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fe53c00c8c0 tx=0x7fe53c0044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:21:45.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:45.466+0000 7fe54cc00700 1 -- 172.21.15.37:0/1336654690 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe53c00a980 con 0x7fe550071f90 2024-03-29T08:21:45.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:45.466+0000 7fe54cc00700 1 -- 172.21.15.37:0/1336654690 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fe53c00ce60 con 0x7fe550071f90 2024-03-29T08:21:45.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:45.466+0000 7fe54cc00700 1 -- 172.21.15.37:0/1336654690 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe53c0113a0 con 0x7fe550071f90 2024-03-29T08:21:45.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:45.466+0000 7fe557c00700 1 -- 172.21.15.37:0/1336654690 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe550114380 con 0x7fe550071f90 2024-03-29T08:21:45.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:45.467+0000 7fe557c00700 1 -- 172.21.15.37:0/1336654690 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5501148c0 con 0x7fe550071f90 2024-03-29T08:21:45.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:45.468+0000 7fe557c00700 1 -- 172.21.15.37:0/1336654690 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe550071aa0 con 0x7fe550071f90 2024-03-29T08:21:45.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:45.468+0000 7fe54cc00700 1 -- 172.21.15.37:0/1336654690 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fe53c012400 con 0x7fe550071f90 2024-03-29T08:21:45.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:45.471+0000 7fe54cc00700 1 --2- 172.21.15.37:0/1336654690 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fe53806cc20 0x7fe53806f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:45.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:45.471+0000 7fe54cc00700 1 -- 172.21.15.37:0/1336654690 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fe53c08c0f0 con 0x7fe550071f90 2024-03-29T08:21:45.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:45.471+0000 7fe555200700 1 --2- 172.21.15.37:0/1336654690 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fe53806cc20 0x7fe53806f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:45.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:45.471+0000 7fe555200700 1 --2- 172.21.15.37:0/1336654690 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fe53806cc20 0x7fe53806f0d0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fe544004f50 tx=0x7fe544009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:21:45.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:45.475+0000 7fe54cc00700 1 -- 172.21.15.37:0/1336654690 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fe53c059bc0 con 0x7fe550071f90 2024-03-29T08:21:45.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:45.708+0000 7fe557c00700 1 -- 172.21.15.37:0/1336654690 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe550110400 con 0x7fe53806cc20 2024-03-29T08:21:45.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:45.710+0000 7fe54cc00700 1 -- 172.21.15.37:0/1336654690 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fe550110400 con 0x7fe53806cc20 2024-03-29T08:21:45.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:45.712+0000 7fe557c00700 1 -- 172.21.15.37:0/1336654690 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fe53806cc20 msgr2=0x7fe53806f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:45.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:45.712+0000 7fe557c00700 1 --2- 172.21.15.37:0/1336654690 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fe53806cc20 0x7fe53806f0d0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fe544004f50 tx=0x7fe544009e30 comp rx=0 tx=0).stop 2024-03-29T08:21:45.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:45.712+0000 7fe557c00700 1 -- 172.21.15.37:0/1336654690 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe550071f90 msgr2=0x7fe55011a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:45.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:45.712+0000 7fe557c00700 1 --2- 172.21.15.37:0/1336654690 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe550071f90 0x7fe55011a680 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fe53c00c8c0 tx=0x7fe53c0044c0 comp rx=0 tx=0).stop 2024-03-29T08:21:45.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:45.712+0000 7fe557c00700 1 -- 172.21.15.37:0/1336654690 shutdown_connections 2024-03-29T08:21:45.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:45.712+0000 7fe557c00700 1 --2- 172.21.15.37:0/1336654690 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fe53806cc20 0x7fe53806f0d0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:45.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:45.712+0000 7fe557c00700 1 --2- 172.21.15.37:0/1336654690 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe5500708a0 0x7fe55011a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:45.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:45.712+0000 7fe557c00700 1 --2- 172.21.15.37:0/1336654690 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe550071f90 0x7fe55011a680 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:45.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:45.712+0000 7fe557c00700 1 -- 172.21.15.37:0/1336654690 >> 172.21.15.37:0/1336654690 conn(0x7fe5500f8970 msgr2=0x7fe5500f9b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:21:45.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:45.712+0000 7fe557c00700 1 -- 172.21.15.37:0/1336654690 shutdown_connections 2024-03-29T08:21:45.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:45.712+0000 7fe557c00700 1 -- 172.21.15.37:0/1336654690 wait complete. 2024-03-29T08:21:45.727 INFO:teuthology.orchestra.run.smithi037.stdout:true 2024-03-29T08:21:45.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:45.850+0000 7f9b28e00700 1 -- 172.21.15.37:0/693341381 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9b241059b0 msgr2=0x7f9b240ff020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:45.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:45.850+0000 7f9b28e00700 1 --2- 172.21.15.37:0/693341381 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9b241059b0 0x7f9b240ff020 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f9b10004940 tx=0x7f9b10009e30 comp rx=0 tx=0).stop 2024-03-29T08:21:45.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:45.850+0000 7f9b28e00700 1 -- 172.21.15.37:0/693341381 shutdown_connections 2024-03-29T08:21:45.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:45.850+0000 7f9b28e00700 1 --2- 172.21.15.37:0/693341381 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9b241059b0 0x7f9b240ff020 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:45.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:45.850+0000 7f9b28e00700 1 --2- 172.21.15.37:0/693341381 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9b24105010 0x7f9b241053e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:45.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:45.850+0000 7f9b28e00700 1 -- 172.21.15.37:0/693341381 >> 172.21.15.37:0/693341381 conn(0x7f9b240faba0 msgr2=0x7f9b240fcfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:21:45.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:45.850+0000 7f9b28e00700 1 -- 172.21.15.37:0/693341381 shutdown_connections 2024-03-29T08:21:45.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:45.850+0000 7f9b28e00700 1 -- 172.21.15.37:0/693341381 wait complete. 2024-03-29T08:21:45.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:45.851+0000 7f9b28e00700 1 Processor -- start 2024-03-29T08:21:45.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:45.851+0000 7f9b28e00700 1 -- start start 2024-03-29T08:21:45.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:45.852+0000 7f9b28e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9b24105010 0x7f9b24115cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:45.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:45.852+0000 7f9b28e00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9b241059b0 0x7f9b241161f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:45.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:45.852+0000 7f9b28e00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b24116840 con 0x7f9b24105010 2024-03-29T08:21:45.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:45.852+0000 7f9b28e00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b2410fc60 con 0x7f9b241059b0 2024-03-29T08:21:45.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:45.852+0000 7f9b21e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9b24105010 0x7f9b24115cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:45.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:45.852+0000 7f9b21e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9b24105010 0x7f9b24115cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52098/0 (socket says 172.21.15.37:52098) 2024-03-29T08:21:45.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:45.852+0000 7f9b21e00700 1 -- 172.21.15.37:0/3921410301 learned_addr learned my addr 172.21.15.37:0/3921410301 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:21:45.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:45.852+0000 7f9b21400700 1 --2- 172.21.15.37:0/3921410301 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9b241059b0 0x7f9b241161f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:45.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:45.852+0000 7f9b21e00700 1 -- 172.21.15.37:0/3921410301 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9b241059b0 msgr2=0x7f9b241161f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:45.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:45.852+0000 7f9b21e00700 1 --2- 172.21.15.37:0/3921410301 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9b241059b0 0x7f9b241161f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:45.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:45.852+0000 7f9b21e00700 1 -- 172.21.15.37:0/3921410301 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b10005040 con 0x7f9b24105010 2024-03-29T08:21:45.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:45.853+0000 7f9b21e00700 1 --2- 172.21.15.37:0/3921410301 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9b24105010 0x7f9b24115cb0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f9b14002a80 tx=0x7f9b140046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:21:45.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:45.853+0000 7f9b1a000700 1 -- 172.21.15.37:0/3921410301 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9b14007cc0 con 0x7f9b24105010 2024-03-29T08:21:45.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:45.853+0000 7f9b1a000700 1 -- 172.21.15.37:0/3921410301 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f9b1400f040 con 0x7f9b24105010 2024-03-29T08:21:45.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:45.853+0000 7f9b28e00700 1 -- 172.21.15.37:0/3921410301 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b2410ff40 con 0x7f9b24105010 2024-03-29T08:21:45.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:45.853+0000 7f9b1a000700 1 -- 172.21.15.37:0/3921410301 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9b1400da60 con 0x7f9b24105010 2024-03-29T08:21:45.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:45.853+0000 7f9b28e00700 1 -- 172.21.15.37:0/3921410301 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b24110500 con 0x7f9b24105010 2024-03-29T08:21:45.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:45.854+0000 7f9b28e00700 1 -- 172.21.15.37:0/3921410301 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b24102320 con 0x7f9b24105010 2024-03-29T08:21:45.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:45.855+0000 7f9b1a000700 1 -- 172.21.15.37:0/3921410301 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f9b1400dbc0 con 0x7f9b24105010 2024-03-29T08:21:45.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:45.856+0000 7f9b1a000700 1 --2- 172.21.15.37:0/3921410301 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f9b0806cc30 0x7f9b0806f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:45.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:45.856+0000 7f9b1a000700 1 -- 172.21.15.37:0/3921410301 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f9b1408bd20 con 0x7f9b24105010 2024-03-29T08:21:45.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:45.856+0000 7f9b21400700 1 --2- 172.21.15.37:0/3921410301 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f9b0806cc30 0x7f9b0806f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:45.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:45.857+0000 7f9b21400700 1 --2- 172.21.15.37:0/3921410301 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f9b0806cc30 0x7f9b0806f0e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f9b24111550 tx=0x7f9b10009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:21:45.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:45.861+0000 7f9b1a000700 1 -- 172.21.15.37:0/3921410301 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f9b140596f0 con 0x7f9b24105010 2024-03-29T08:21:46.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:21:45 smithi148 ceph-mon[106692]: pgmap v326: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:21:46.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:46.091+0000 7f9b28e00700 1 -- 172.21.15.37:0/3921410301 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f9b2410c020 con 0x7f9b0806cc30 2024-03-29T08:21:46.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:46.093+0000 7f9b1a000700 1 -- 172.21.15.37:0/3921410301 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f9b2410c020 con 0x7f9b0806cc30 2024-03-29T08:21:46.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:46.095+0000 7f9b28e00700 1 -- 172.21.15.37:0/3921410301 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f9b0806cc30 msgr2=0x7f9b0806f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:46.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:46.095+0000 7f9b28e00700 1 --2- 172.21.15.37:0/3921410301 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f9b0806cc30 0x7f9b0806f0e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f9b24111550 tx=0x7f9b10009e30 comp rx=0 tx=0).stop 2024-03-29T08:21:46.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:46.095+0000 7f9b28e00700 1 -- 172.21.15.37:0/3921410301 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9b24105010 msgr2=0x7f9b24115cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:46.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:46.095+0000 7f9b28e00700 1 --2- 172.21.15.37:0/3921410301 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9b24105010 0x7f9b24115cb0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f9b14002a80 tx=0x7f9b140046b0 comp rx=0 tx=0).stop 2024-03-29T08:21:46.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:46.095+0000 7f9b28e00700 1 -- 172.21.15.37:0/3921410301 shutdown_connections 2024-03-29T08:21:46.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:46.095+0000 7f9b28e00700 1 --2- 172.21.15.37:0/3921410301 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f9b0806cc30 0x7f9b0806f0e0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:46.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:46.095+0000 7f9b28e00700 1 --2- 172.21.15.37:0/3921410301 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9b24105010 0x7f9b24115cb0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:46.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:46.095+0000 7f9b28e00700 1 --2- 172.21.15.37:0/3921410301 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9b241059b0 0x7f9b241161f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:46.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:46.095+0000 7f9b28e00700 1 -- 172.21.15.37:0/3921410301 >> 172.21.15.37:0/3921410301 conn(0x7f9b240faba0 msgr2=0x7f9b24109600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:21:46.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:46.095+0000 7f9b28e00700 1 -- 172.21.15.37:0/3921410301 shutdown_connections 2024-03-29T08:21:46.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:46.095+0000 7f9b28e00700 1 -- 172.21.15.37:0/3921410301 wait complete. 2024-03-29T08:21:46.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:21:45 smithi037 ceph-mon[142325]: pgmap v326: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:21:46.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:46.233+0000 7fbb9ba00700 1 -- 172.21.15.37:0/550349292 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbb94102e00 msgr2=0x7fbb941031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:46.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:46.233+0000 7fbb9ba00700 1 --2- 172.21.15.37:0/550349292 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbb94102e00 0x7fbb941031d0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fbb88004940 tx=0x7fbb88009e30 comp rx=0 tx=0).stop 2024-03-29T08:21:46.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:46.233+0000 7fbb9ba00700 1 -- 172.21.15.37:0/550349292 shutdown_connections 2024-03-29T08:21:46.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:46.233+0000 7fbb9ba00700 1 --2- 172.21.15.37:0/550349292 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb941037a0 0x7fbb940fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:46.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:46.233+0000 7fbb9ba00700 1 --2- 172.21.15.37:0/550349292 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbb94102e00 0x7fbb941031d0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:46.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:46.233+0000 7fbb9ba00700 1 -- 172.21.15.37:0/550349292 >> 172.21.15.37:0/550349292 conn(0x7fbb940f89b0 msgr2=0x7fbb940fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:21:46.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:46.233+0000 7fbb9ba00700 1 -- 172.21.15.37:0/550349292 shutdown_connections 2024-03-29T08:21:46.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:46.233+0000 7fbb9ba00700 1 -- 172.21.15.37:0/550349292 wait complete. 2024-03-29T08:21:46.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:46.234+0000 7fbb9ba00700 1 Processor -- start 2024-03-29T08:21:46.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:46.234+0000 7fbb9ba00700 1 -- start start 2024-03-29T08:21:46.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:46.234+0000 7fbb9ba00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbb94102e00 0x7fbb94115cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:46.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:46.234+0000 7fbb9ba00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb941037a0 0x7fbb94116230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:46.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:46.234+0000 7fbb9ba00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb94116880 con 0x7fbb94102e00 2024-03-29T08:21:46.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:46.234+0000 7fbb9ba00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb9410fc50 con 0x7fbb941037a0 2024-03-29T08:21:46.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:46.234+0000 7fbb99000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbb94102e00 0x7fbb94115cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:46.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:46.234+0000 7fbb99000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbb94102e00 0x7fbb94115cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52128/0 (socket says 172.21.15.37:52128) 2024-03-29T08:21:46.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:46.234+0000 7fbb99000700 1 -- 172.21.15.37:0/1792402507 learned_addr learned my addr 172.21.15.37:0/1792402507 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:21:46.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:46.235+0000 7fbb99000700 1 -- 172.21.15.37:0/1792402507 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb941037a0 msgr2=0x7fbb94116230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:46.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:46.235+0000 7fbb93e00700 1 --2- 172.21.15.37:0/1792402507 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb941037a0 0x7fbb94116230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:46.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:46.235+0000 7fbb99000700 1 --2- 172.21.15.37:0/1792402507 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb941037a0 0x7fbb94116230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:46.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:46.235+0000 7fbb99000700 1 -- 172.21.15.37:0/1792402507 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb88005040 con 0x7fbb94102e00 2024-03-29T08:21:46.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:46.235+0000 7fbb93e00700 1 --2- 172.21.15.37:0/1792402507 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb941037a0 0x7fbb94116230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:21:46.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:46.235+0000 7fbb99000700 1 --2- 172.21.15.37:0/1792402507 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbb94102e00 0x7fbb94115cf0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fbb88009390 tx=0x7fbb88009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:21:46.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:46.235+0000 7fbb90c00700 1 -- 172.21.15.37:0/1792402507 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbb8800e040 con 0x7fbb94102e00 2024-03-29T08:21:46.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:46.235+0000 7fbb90c00700 1 -- 172.21.15.37:0/1792402507 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fbb88013420 con 0x7fbb94102e00 2024-03-29T08:21:46.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:46.235+0000 7fbb90c00700 1 -- 172.21.15.37:0/1792402507 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbb8801c630 con 0x7fbb94102e00 2024-03-29T08:21:46.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:46.235+0000 7fbb9ba00700 1 -- 172.21.15.37:0/1792402507 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb9410fed0 con 0x7fbb94102e00 2024-03-29T08:21:46.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:46.236+0000 7fbb9ba00700 1 -- 172.21.15.37:0/1792402507 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb94110490 con 0x7fbb94102e00 2024-03-29T08:21:46.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:46.237+0000 7fbb9ba00700 1 -- 172.21.15.37:0/1792402507 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb940fe550 con 0x7fbb94102e00 2024-03-29T08:21:46.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:46.237+0000 7fbb90c00700 1 -- 172.21.15.37:0/1792402507 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fbb880139e0 con 0x7fbb94102e00 2024-03-29T08:21:46.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:46.238+0000 7fbb90c00700 1 --2- 172.21.15.37:0/1792402507 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fbb7c06cc30 0x7fbb7c06f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:46.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:46.238+0000 7fbb90c00700 1 -- 172.21.15.37:0/1792402507 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fbb8808d250 con 0x7fbb94102e00 2024-03-29T08:21:46.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:46.239+0000 7fbb93e00700 1 --2- 172.21.15.37:0/1792402507 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fbb7c06cc30 0x7fbb7c06f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:46.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:46.239+0000 7fbb93e00700 1 --2- 172.21.15.37:0/1792402507 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fbb7c06cc30 0x7fbb7c06f0e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fbb80009fd0 tx=0x7fbb8000b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:21:46.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:46.243+0000 7fbb90c00700 1 -- 172.21.15.37:0/1792402507 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fbb8805add0 con 0x7fbb94102e00 2024-03-29T08:21:46.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:46.466+0000 7fbb9ba00700 1 -- 172.21.15.37:0/1792402507 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fbb9404bf40 con 0x7fbb7c06cc30 2024-03-29T08:21:46.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:46.476+0000 7fbb90c00700 1 -- 172.21.15.37:0/1792402507 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7fbb9404bf40 con 0x7fbb7c06cc30 2024-03-29T08:21:46.476 INFO:teuthology.orchestra.run.smithi037.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-29T08:21:46.476 INFO:teuthology.orchestra.run.smithi037.stdout:alertmanager.smithi037 smithi037 *:9093,9094 running (11m) 3m ago 24m 14.9M - 0.25.0 c8568f914cd2 2f9f9746ed0f 2024-03-29T08:21:46.477 INFO:teuthology.orchestra.run.smithi037.stdout:crash.smithi037 smithi037 running (8m) 3m ago 24m 6647k - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 178d416a3186 2024-03-29T08:21:46.477 INFO:teuthology.orchestra.run.smithi037.stdout:crash.smithi148 smithi148 running (8m) 3m ago 23m 6656k - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 23666989d58f 2024-03-29T08:21:46.477 INFO:teuthology.orchestra.run.smithi037.stdout:grafana.smithi037 smithi037 *:3000 running (11m) 3m ago 24m 71.9M - 9.4.7 954c08fa6188 6b976fe49952 2024-03-29T08:21:46.477 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi037.iifsmh smithi037 running (4m) 3m ago 18m 10.8M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 2d97f562cc5b 2024-03-29T08:21:46.477 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi037.mmwgxe smithi037 running (4m) 3m ago 18m 10.9M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff ec37037f46e8 2024-03-29T08:21:46.477 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi148.qukpjh smithi148 running (18m) 3m ago 18m 15.3M - 17.2.7-789-g4213e17c cf22eb8683be 8e469782a273 2024-03-29T08:21:46.477 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi148.sqteih smithi148 running (3m) 3m ago 18m 10.7M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 44c15c77037a 2024-03-29T08:21:46.477 INFO:teuthology.orchestra.run.smithi037.stdout:mgr.smithi037.wjikii smithi037 *:8443,9283,8765 running (13m) 3m ago 26m 506M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff c5a1417b9417 2024-03-29T08:21:46.477 INFO:teuthology.orchestra.run.smithi037.stdout:mgr.smithi148.onhgvm smithi148 *:8443,9283,8765 running (12m) 3m ago 23m 450M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff fcfdbce062bd 2024-03-29T08:21:46.477 INFO:teuthology.orchestra.run.smithi037.stdout:mon.smithi037 smithi037 running (9m) 3m ago 26m 54.8M 2048M 18.2.2-1015-gd0e392d9 1cc2f45c17ff eea2060783af 2024-03-29T08:21:46.477 INFO:teuthology.orchestra.run.smithi037.stdout:mon.smithi148 smithi148 running (9m) 3m ago 23m 43.7M 2048M 18.2.2-1015-gd0e392d9 1cc2f45c17ff ad4b2c5748be 2024-03-29T08:21:46.477 INFO:teuthology.orchestra.run.smithi037.stdout:node-exporter.smithi037 smithi037 *:9100 running (12m) 3m ago 24m 7625k - 1.5.0 0da6a335fe13 26b8ae9cb878 2024-03-29T08:21:46.477 INFO:teuthology.orchestra.run.smithi037.stdout:node-exporter.smithi148 smithi148 *:9100 running (12m) 3m ago 23m 8166k - 1.5.0 0da6a335fe13 98414e4e64ed 2024-03-29T08:21:46.477 INFO:teuthology.orchestra.run.smithi037.stdout:osd.0 smithi037 running (7m) 3m ago 22m 104M 1619M 18.2.2-1015-gd0e392d9 1cc2f45c17ff c0c584f6245c 2024-03-29T08:21:46.478 INFO:teuthology.orchestra.run.smithi037.stdout:osd.1 smithi037 running (7m) 3m ago 21m 100M 1619M 18.2.2-1015-gd0e392d9 1cc2f45c17ff a4a2ff4cc83d 2024-03-29T08:21:46.478 INFO:teuthology.orchestra.run.smithi037.stdout:osd.2 smithi037 running (6m) 3m ago 21m 82.3M 1619M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 8b1bc71aa79e 2024-03-29T08:21:46.478 INFO:teuthology.orchestra.run.smithi037.stdout:osd.3 smithi148 running (5m) 3m ago 20m 105M 2642M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 1c656ab8ff86 2024-03-29T08:21:46.478 INFO:teuthology.orchestra.run.smithi037.stdout:osd.4 smithi148 running (5m) 3m ago 20m 97.3M 2642M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 966a9de11d22 2024-03-29T08:21:46.478 INFO:teuthology.orchestra.run.smithi037.stdout:osd.5 smithi148 running (4m) 3m ago 19m 85.8M 2642M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 474c4f9212ce 2024-03-29T08:21:46.478 INFO:teuthology.orchestra.run.smithi037.stdout:prometheus.smithi037 smithi037 *:9095 running (11m) 3m ago 24m 54.4M - 2.43.0 a07b618ecd1d 30c0223ec589 2024-03-29T08:21:46.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:46.479+0000 7fbb9ba00700 1 -- 172.21.15.37:0/1792402507 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fbb7c06cc30 msgr2=0x7fbb7c06f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:46.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:46.479+0000 7fbb9ba00700 1 --2- 172.21.15.37:0/1792402507 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fbb7c06cc30 0x7fbb7c06f0e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fbb80009fd0 tx=0x7fbb8000b040 comp rx=0 tx=0).stop 2024-03-29T08:21:46.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:46.479+0000 7fbb9ba00700 1 -- 172.21.15.37:0/1792402507 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbb94102e00 msgr2=0x7fbb94115cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:46.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:46.479+0000 7fbb9ba00700 1 --2- 172.21.15.37:0/1792402507 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbb94102e00 0x7fbb94115cf0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fbb88009390 tx=0x7fbb88009e30 comp rx=0 tx=0).stop 2024-03-29T08:21:46.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:46.479+0000 7fbb9ba00700 1 -- 172.21.15.37:0/1792402507 shutdown_connections 2024-03-29T08:21:46.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:46.479+0000 7fbb9ba00700 1 --2- 172.21.15.37:0/1792402507 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fbb7c06cc30 0x7fbb7c06f0e0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:46.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:46.479+0000 7fbb9ba00700 1 --2- 172.21.15.37:0/1792402507 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbb94102e00 0x7fbb94115cf0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:46.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:46.479+0000 7fbb9ba00700 1 --2- 172.21.15.37:0/1792402507 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb941037a0 0x7fbb94116230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:46.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:46.479+0000 7fbb9ba00700 1 -- 172.21.15.37:0/1792402507 >> 172.21.15.37:0/1792402507 conn(0x7fbb940f89b0 msgr2=0x7fbb940706d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:21:46.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:46.479+0000 7fbb9ba00700 1 -- 172.21.15.37:0/1792402507 shutdown_connections 2024-03-29T08:21:46.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:46.479+0000 7fbb9ba00700 1 -- 172.21.15.37:0/1792402507 wait complete. 2024-03-29T08:21:46.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:46.615+0000 7fc194a00700 1 -- 172.21.15.37:0/3634050342 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc1901035d0 msgr2=0x7fc1900fcc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:46.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:46.615+0000 7fc194a00700 1 --2- 172.21.15.37:0/3634050342 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc1901035d0 0x7fc1900fcc40 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fc17c004940 tx=0x7fc17c009e30 comp rx=0 tx=0).stop 2024-03-29T08:21:46.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:46.615+0000 7fc194a00700 1 -- 172.21.15.37:0/3634050342 shutdown_connections 2024-03-29T08:21:46.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:46.615+0000 7fc194a00700 1 --2- 172.21.15.37:0/3634050342 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc1901035d0 0x7fc1900fcc40 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:46.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:46.615+0000 7fc194a00700 1 --2- 172.21.15.37:0/3634050342 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc190102c30 0x7fc190103000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:46.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:46.615+0000 7fc194a00700 1 -- 172.21.15.37:0/3634050342 >> 172.21.15.37:0/3634050342 conn(0x7fc1900f8760 msgr2=0x7fc1900fab70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:21:46.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:46.615+0000 7fc194a00700 1 -- 172.21.15.37:0/3634050342 shutdown_connections 2024-03-29T08:21:46.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:46.615+0000 7fc194a00700 1 -- 172.21.15.37:0/3634050342 wait complete. 2024-03-29T08:21:46.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:46.616+0000 7fc194a00700 1 Processor -- start 2024-03-29T08:21:46.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:46.616+0000 7fc194a00700 1 -- start start 2024-03-29T08:21:46.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:46.617+0000 7fc194a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc190102c30 0x7fc1901138f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:46.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:46.617+0000 7fc194a00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc1901035d0 0x7fc190113e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:46.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:46.617+0000 7fc194a00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc190114480 con 0x7fc190102c30 2024-03-29T08:21:46.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:46.617+0000 7fc194a00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc19010d870 con 0x7fc1901035d0 2024-03-29T08:21:46.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:46.617+0000 7fc18e800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc190102c30 0x7fc1901138f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:46.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:46.617+0000 7fc18e800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc190102c30 0x7fc1901138f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52132/0 (socket says 172.21.15.37:52132) 2024-03-29T08:21:46.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:46.617+0000 7fc18e800700 1 -- 172.21.15.37:0/1014936915 learned_addr learned my addr 172.21.15.37:0/1014936915 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:21:46.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:46.617+0000 7fc18de00700 1 --2- 172.21.15.37:0/1014936915 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc1901035d0 0x7fc190113e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:46.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:46.617+0000 7fc18e800700 1 -- 172.21.15.37:0/1014936915 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc1901035d0 msgr2=0x7fc190113e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:46.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:46.617+0000 7fc18e800700 1 --2- 172.21.15.37:0/1014936915 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc1901035d0 0x7fc190113e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:46.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:46.617+0000 7fc18e800700 1 -- 172.21.15.37:0/1014936915 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc17c005040 con 0x7fc190102c30 2024-03-29T08:21:46.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:46.617+0000 7fc18e800700 1 --2- 172.21.15.37:0/1014936915 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc190102c30 0x7fc1901138f0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fc18000c8c0 tx=0x7fc1800044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:21:46.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:46.618+0000 7fc186a00700 1 -- 172.21.15.37:0/1014936915 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc18000a980 con 0x7fc190102c30 2024-03-29T08:21:46.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:46.618+0000 7fc186a00700 1 -- 172.21.15.37:0/1014936915 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fc18000ce60 con 0x7fc190102c30 2024-03-29T08:21:46.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:46.618+0000 7fc186a00700 1 -- 172.21.15.37:0/1014936915 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc1800113a0 con 0x7fc190102c30 2024-03-29T08:21:46.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:46.618+0000 7fc194a00700 1 -- 172.21.15.37:0/1014936915 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc19010db50 con 0x7fc190102c30 2024-03-29T08:21:46.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:46.618+0000 7fc194a00700 1 -- 172.21.15.37:0/1014936915 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc19010e110 con 0x7fc190102c30 2024-03-29T08:21:46.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:46.619+0000 7fc194a00700 1 -- 172.21.15.37:0/1014936915 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1900fe380 con 0x7fc190102c30 2024-03-29T08:21:46.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:46.620+0000 7fc186a00700 1 -- 172.21.15.37:0/1014936915 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fc180012400 con 0x7fc190102c30 2024-03-29T08:21:46.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:46.621+0000 7fc186a00700 1 --2- 172.21.15.37:0/1014936915 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fc17406cc20 0x7fc17406f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:46.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:46.621+0000 7fc186a00700 1 -- 172.21.15.37:0/1014936915 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fc18008c0f0 con 0x7fc190102c30 2024-03-29T08:21:46.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:46.621+0000 7fc18de00700 1 --2- 172.21.15.37:0/1014936915 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fc17406cc20 0x7fc17406f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:46.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:46.622+0000 7fc18de00700 1 --2- 172.21.15.37:0/1014936915 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fc17406cc20 0x7fc17406f0d0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fc19010ea30 tx=0x7fc17c009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:21:46.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:46.625+0000 7fc186a00700 1 -- 172.21.15.37:0/1014936915 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fc180059bc0 con 0x7fc190102c30 2024-03-29T08:21:46.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:46.924+0000 7fc194a00700 1 -- 172.21.15.37:0/1014936915 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fc190109bc0 con 0x7fc190102c30 2024-03-29T08:21:46.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:46.925+0000 7fc186a00700 1 -- 172.21.15.37:0/1014936915 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+847 (secure 0 0 0) 0x7fc180059750 con 0x7fc190102c30 2024-03-29T08:21:46.925 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-29T08:21:46.925 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": { 2024-03-29T08:21:46.925 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 2 2024-03-29T08:21:46.925 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:21:46.925 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": { 2024-03-29T08:21:46.925 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 2 2024-03-29T08:21:46.926 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:21:46.926 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": { 2024-03-29T08:21:46.926 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 6 2024-03-29T08:21:46.926 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:21:46.926 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": { 2024-03-29T08:21:46.926 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 17.2.7-789-g4213e17c (4213e17c4de67016ed14f55d22503f4923b3a901) quincy (stable)": 1, 2024-03-29T08:21:46.926 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 3 2024-03-29T08:21:46.926 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:21:46.926 INFO:teuthology.orchestra.run.smithi037.stdout: "overall": { 2024-03-29T08:21:46.926 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 17.2.7-789-g4213e17c (4213e17c4de67016ed14f55d22503f4923b3a901) quincy (stable)": 1, 2024-03-29T08:21:46.926 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 13 2024-03-29T08:21:46.926 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-29T08:21:46.926 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-29T08:21:46.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:46.927+0000 7fc194a00700 1 -- 172.21.15.37:0/1014936915 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fc17406cc20 msgr2=0x7fc17406f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:46.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:46.927+0000 7fc194a00700 1 --2- 172.21.15.37:0/1014936915 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fc17406cc20 0x7fc17406f0d0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fc19010ea30 tx=0x7fc17c009e30 comp rx=0 tx=0).stop 2024-03-29T08:21:46.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:46.927+0000 7fc194a00700 1 -- 172.21.15.37:0/1014936915 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc190102c30 msgr2=0x7fc1901138f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:46.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:46.927+0000 7fc194a00700 1 --2- 172.21.15.37:0/1014936915 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc190102c30 0x7fc1901138f0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fc18000c8c0 tx=0x7fc1800044c0 comp rx=0 tx=0).stop 2024-03-29T08:21:46.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:46.927+0000 7fc194a00700 1 -- 172.21.15.37:0/1014936915 shutdown_connections 2024-03-29T08:21:46.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:46.927+0000 7fc194a00700 1 --2- 172.21.15.37:0/1014936915 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fc17406cc20 0x7fc17406f0d0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:46.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:46.927+0000 7fc194a00700 1 --2- 172.21.15.37:0/1014936915 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc190102c30 0x7fc1901138f0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:46.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:46.927+0000 7fc194a00700 1 --2- 172.21.15.37:0/1014936915 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc1901035d0 0x7fc190113e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:46.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:46.927+0000 7fc194a00700 1 -- 172.21.15.37:0/1014936915 >> 172.21.15.37:0/1014936915 conn(0x7fc1900f8760 msgr2=0x7fc1900f9830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:21:46.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:46.927+0000 7fc194a00700 1 -- 172.21.15.37:0/1014936915 shutdown_connections 2024-03-29T08:21:46.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:46.927+0000 7fc194a00700 1 -- 172.21.15.37:0/1014936915 wait complete. 2024-03-29T08:21:47.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:21:46 smithi148 ceph-mon[106692]: from='client.34644 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:21:47.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:21:46 smithi148 ceph-mon[106692]: from='client.34648 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:21:47.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:21:46 smithi148 ceph-mon[106692]: from='client.34652 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:21:47.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:47.065+0000 7f62cb000700 1 -- 172.21.15.37:0/3101452805 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f62c41037a0 msgr2=0x7f62c40fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:47.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:47.065+0000 7f62cb000700 1 --2- 172.21.15.37:0/3101452805 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f62c41037a0 0x7f62c40fce10 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f62b0004940 tx=0x7f62b0009e30 comp rx=0 tx=0).stop 2024-03-29T08:21:47.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:47.065+0000 7f62cb000700 1 -- 172.21.15.37:0/3101452805 shutdown_connections 2024-03-29T08:21:47.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:47.065+0000 7f62cb000700 1 --2- 172.21.15.37:0/3101452805 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f62c41037a0 0x7f62c40fce10 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:47.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:47.065+0000 7f62cb000700 1 --2- 172.21.15.37:0/3101452805 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f62c4102e00 0x7f62c41031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:47.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:47.065+0000 7f62cb000700 1 -- 172.21.15.37:0/3101452805 >> 172.21.15.37:0/3101452805 conn(0x7f62c40f89b0 msgr2=0x7f62c40fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:21:47.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:47.065+0000 7f62cb000700 1 -- 172.21.15.37:0/3101452805 shutdown_connections 2024-03-29T08:21:47.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:47.065+0000 7f62cb000700 1 -- 172.21.15.37:0/3101452805 wait complete. 2024-03-29T08:21:47.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:47.066+0000 7f62cb000700 1 Processor -- start 2024-03-29T08:21:47.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:47.066+0000 7f62cb000700 1 -- start start 2024-03-29T08:21:47.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:47.066+0000 7f62cb000700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f62c4102e00 0x7f62c410af20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:47.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:47.066+0000 7f62cb000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f62c41037a0 0x7f62c410b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:47.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:47.066+0000 7f62cb000700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62c410f0b0 con 0x7f62c41037a0 2024-03-29T08:21:47.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:47.066+0000 7f62cb000700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62c410b9a0 con 0x7f62c4102e00 2024-03-29T08:21:47.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:47.066+0000 7f62c3400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f62c41037a0 0x7f62c410b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:47.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:47.066+0000 7f62c3400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f62c41037a0 0x7f62c410b460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52148/0 (socket says 172.21.15.37:52148) 2024-03-29T08:21:47.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:47.067+0000 7f62c3400700 1 -- 172.21.15.37:0/1515000988 learned_addr learned my addr 172.21.15.37:0/1515000988 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:21:47.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:47.067+0000 7f62c3e00700 1 --2- 172.21.15.37:0/1515000988 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f62c4102e00 0x7f62c410af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:47.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:47.067+0000 7f62c3400700 1 -- 172.21.15.37:0/1515000988 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f62c4102e00 msgr2=0x7f62c410af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:47.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:47.067+0000 7f62c3400700 1 --2- 172.21.15.37:0/1515000988 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f62c4102e00 0x7f62c410af20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:47.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:47.067+0000 7f62c3400700 1 -- 172.21.15.37:0/1515000988 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62b0005040 con 0x7f62c41037a0 2024-03-29T08:21:47.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:47.067+0000 7f62c3e00700 1 --2- 172.21.15.37:0/1515000988 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f62c4102e00 0x7f62c410af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:21:47.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:47.067+0000 7f62c3400700 1 --2- 172.21.15.37:0/1515000988 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f62c41037a0 0x7f62c410b460 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f62b0009390 tx=0x7f62b0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:21:47.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:47.067+0000 7f62bbe00700 1 -- 172.21.15.37:0/1515000988 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f62b000e040 con 0x7f62c41037a0 2024-03-29T08:21:47.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:47.067+0000 7f62bbe00700 1 -- 172.21.15.37:0/1515000988 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f62b0013420 con 0x7f62c41037a0 2024-03-29T08:21:47.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:47.067+0000 7f62cb000700 1 -- 172.21.15.37:0/1515000988 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62c410bc20 con 0x7f62c41037a0 2024-03-29T08:21:47.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:47.067+0000 7f62bbe00700 1 -- 172.21.15.37:0/1515000988 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f62b001c630 con 0x7f62c41037a0 2024-03-29T08:21:47.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:47.068+0000 7f62cb000700 1 -- 172.21.15.37:0/1515000988 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62c411e690 con 0x7f62c41037a0 2024-03-29T08:21:47.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:47.069+0000 7f62cb000700 1 -- 172.21.15.37:0/1515000988 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62c40fe550 con 0x7f62c41037a0 2024-03-29T08:21:47.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:47.070+0000 7f62bbe00700 1 -- 172.21.15.37:0/1515000988 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f62b0013a00 con 0x7f62c41037a0 2024-03-29T08:21:47.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:47.071+0000 7f62bbe00700 1 --2- 172.21.15.37:0/1515000988 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f62a806cc30 0x7f62a806f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:47.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:47.071+0000 7f62bbe00700 1 -- 172.21.15.37:0/1515000988 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f62b008d370 con 0x7f62c41037a0 2024-03-29T08:21:47.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:47.071+0000 7f62c3e00700 1 --2- 172.21.15.37:0/1515000988 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f62a806cc30 0x7f62a806f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:47.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:47.071+0000 7f62c3e00700 1 --2- 172.21.15.37:0/1515000988 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f62a806cc30 0x7f62a806f0e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f62b4009fd0 tx=0x7f62b4009ee0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:21:47.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:47.075+0000 7f62bbe00700 1 -- 172.21.15.37:0/1515000988 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f62b0057330 con 0x7f62c41037a0 2024-03-29T08:21:47.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:21:46 smithi037 ceph-mon[142325]: from='client.34644 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:21:47.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:21:46 smithi037 ceph-mon[142325]: from='client.34648 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:21:47.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:21:46 smithi037 ceph-mon[142325]: from='client.34652 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:21:47.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:47.330+0000 7f62cb000700 1 -- 172.21.15.37:0/1515000988 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f62c405f5f0 con 0x7f62c41037a0 2024-03-29T08:21:47.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:47.331+0000 7f62bbe00700 1 -- 172.21.15.37:0/1515000988 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 28 v28) v1 ==== 76+0+1841 (secure 0 0 0) 0x7f62b005a950 con 0x7f62c41037a0 2024-03-29T08:21:47.331 INFO:teuthology.orchestra.run.smithi037.stdout:e28 2024-03-29T08:21:47.331 INFO:teuthology.orchestra.run.smithi037.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-29T08:21:47.331 INFO:teuthology.orchestra.run.smithi037.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T08:21:47.331 INFO:teuthology.orchestra.run.smithi037.stdout:legacy client fscid: 1 2024-03-29T08:21:47.331 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:21:47.331 INFO:teuthology.orchestra.run.smithi037.stdout:Filesystem 'cephfs' (1) 2024-03-29T08:21:47.331 INFO:teuthology.orchestra.run.smithi037.stdout:fs_name cephfs 2024-03-29T08:21:47.331 INFO:teuthology.orchestra.run.smithi037.stdout:epoch 21 2024-03-29T08:21:47.331 INFO:teuthology.orchestra.run.smithi037.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-03-29T08:21:47.331 INFO:teuthology.orchestra.run.smithi037.stdout:created 2024-03-29T08:03:29.241269+0000 2024-03-29T08:21:47.331 INFO:teuthology.orchestra.run.smithi037.stdout:modified 2024-03-29T08:17:16.237238+0000 2024-03-29T08:21:47.332 INFO:teuthology.orchestra.run.smithi037.stdout:tableserver 0 2024-03-29T08:21:47.332 INFO:teuthology.orchestra.run.smithi037.stdout:root 0 2024-03-29T08:21:47.332 INFO:teuthology.orchestra.run.smithi037.stdout:session_timeout 60 2024-03-29T08:21:47.332 INFO:teuthology.orchestra.run.smithi037.stdout:session_autoclose 300 2024-03-29T08:21:47.332 INFO:teuthology.orchestra.run.smithi037.stdout:max_file_size 1099511627776 2024-03-29T08:21:47.332 INFO:teuthology.orchestra.run.smithi037.stdout:max_xattr_size 65536 2024-03-29T08:21:47.332 INFO:teuthology.orchestra.run.smithi037.stdout:required_client_features {} 2024-03-29T08:21:47.332 INFO:teuthology.orchestra.run.smithi037.stdout:last_failure 0 2024-03-29T08:21:47.332 INFO:teuthology.orchestra.run.smithi037.stdout:last_failure_osd_epoch 82 2024-03-29T08:21:47.332 INFO:teuthology.orchestra.run.smithi037.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T08:21:47.332 INFO:teuthology.orchestra.run.smithi037.stdout:max_mds 2 2024-03-29T08:21:47.332 INFO:teuthology.orchestra.run.smithi037.stdout:in 0,1 2024-03-29T08:21:47.332 INFO:teuthology.orchestra.run.smithi037.stdout:up {} 2024-03-29T08:21:47.332 INFO:teuthology.orchestra.run.smithi037.stdout:failed 0,1 2024-03-29T08:21:47.332 INFO:teuthology.orchestra.run.smithi037.stdout:damaged 2024-03-29T08:21:47.333 INFO:teuthology.orchestra.run.smithi037.stdout:stopped 2024-03-29T08:21:47.333 INFO:teuthology.orchestra.run.smithi037.stdout:data_pools [3] 2024-03-29T08:21:47.333 INFO:teuthology.orchestra.run.smithi037.stdout:metadata_pool 2 2024-03-29T08:21:47.333 INFO:teuthology.orchestra.run.smithi037.stdout:inline_data disabled 2024-03-29T08:21:47.333 INFO:teuthology.orchestra.run.smithi037.stdout:balancer 2024-03-29T08:21:47.333 INFO:teuthology.orchestra.run.smithi037.stdout:bal_rank_mask -1 2024-03-29T08:21:47.333 INFO:teuthology.orchestra.run.smithi037.stdout:standby_count_wanted 1 2024-03-29T08:21:47.333 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:21:47.333 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:21:47.333 INFO:teuthology.orchestra.run.smithi037.stdout:Standby daemons: 2024-03-29T08:21:47.333 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:21:47.333 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi148.qukpjh{-1:24319} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.148:6826/2641327912,v1:172.21.15.148:6827/2641327912] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:21:47.333 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi037.mmwgxe{-1:34440} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.37:6828/1248142005,v1:172.21.15.37:6829/1248142005] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:21:47.333 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi037.iifsmh{-1:34444} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.37:6826/2060882959,v1:172.21.15.37:6827/2060882959] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:21:47.333 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi148.sqteih{-1:44395} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.148:6824/4078652242,v1:172.21.15.148:6825/4078652242] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:21:47.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:47.333+0000 7f62cb000700 1 -- 172.21.15.37:0/1515000988 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f62a806cc30 msgr2=0x7f62a806f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:47.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:47.333+0000 7f62cb000700 1 --2- 172.21.15.37:0/1515000988 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f62a806cc30 0x7f62a806f0e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f62b4009fd0 tx=0x7f62b4009ee0 comp rx=0 tx=0).stop 2024-03-29T08:21:47.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:47.333+0000 7f62cb000700 1 -- 172.21.15.37:0/1515000988 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f62c41037a0 msgr2=0x7f62c410b460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:47.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:47.333+0000 7f62cb000700 1 --2- 172.21.15.37:0/1515000988 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f62c41037a0 0x7f62c410b460 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f62b0009390 tx=0x7f62b0009e30 comp rx=0 tx=0).stop 2024-03-29T08:21:47.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:47.333+0000 7f62cb000700 1 -- 172.21.15.37:0/1515000988 shutdown_connections 2024-03-29T08:21:47.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:47.333+0000 7f62cb000700 1 --2- 172.21.15.37:0/1515000988 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f62a806cc30 0x7f62a806f0e0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:47.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:47.333+0000 7f62cb000700 1 --2- 172.21.15.37:0/1515000988 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f62c4102e00 0x7f62c410af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:47.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:47.333+0000 7f62cb000700 1 --2- 172.21.15.37:0/1515000988 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f62c41037a0 0x7f62c410b460 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:47.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:47.333+0000 7f62cb000700 1 -- 172.21.15.37:0/1515000988 >> 172.21.15.37:0/1515000988 conn(0x7f62c40f89b0 msgr2=0x7f62c40f99f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:21:47.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:47.333+0000 7f62cb000700 1 -- 172.21.15.37:0/1515000988 shutdown_connections 2024-03-29T08:21:47.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:47.333+0000 7f62cb000700 1 -- 172.21.15.37:0/1515000988 wait complete. 2024-03-29T08:21:47.335 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 28 2024-03-29T08:21:47.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:47.471+0000 7fa6c2800700 1 -- 172.21.15.37:0/4074722657 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa6bc0fdeb0 msgr2=0x7fa6bc0fe280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:47.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:47.471+0000 7fa6c2800700 1 --2- 172.21.15.37:0/4074722657 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa6bc0fdeb0 0x7fa6bc0fe280 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fa6ac004970 tx=0x7fa6ac009e30 comp rx=0 tx=0).stop 2024-03-29T08:21:47.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:47.471+0000 7fa6c2800700 1 -- 172.21.15.37:0/4074722657 shutdown_connections 2024-03-29T08:21:47.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:47.471+0000 7fa6c2800700 1 --2- 172.21.15.37:0/4074722657 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa6bc0fe850 0x7fa6bc106f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:47.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:47.471+0000 7fa6c2800700 1 --2- 172.21.15.37:0/4074722657 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa6bc0fdeb0 0x7fa6bc0fe280 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:47.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:47.471+0000 7fa6c2800700 1 -- 172.21.15.37:0/4074722657 >> 172.21.15.37:0/4074722657 conn(0x7fa6bc072bc0 msgr2=0x7fa6bc072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:21:47.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:47.471+0000 7fa6c2800700 1 -- 172.21.15.37:0/4074722657 shutdown_connections 2024-03-29T08:21:47.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:47.471+0000 7fa6c2800700 1 -- 172.21.15.37:0/4074722657 wait complete. 2024-03-29T08:21:47.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:47.472+0000 7fa6c2800700 1 Processor -- start 2024-03-29T08:21:47.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:47.472+0000 7fa6c2800700 1 -- start start 2024-03-29T08:21:47.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:47.472+0000 7fa6c2800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa6bc0fdeb0 0x7fa6bc0fbd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:47.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:47.472+0000 7fa6c2800700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa6bc0fe850 0x7fa6bc0fc2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:47.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:47.472+0000 7fa6c2800700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6bc0fdb80 con 0x7fa6bc0fdeb0 2024-03-29T08:21:47.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:47.472+0000 7fa6c2800700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6bc0fdcf0 con 0x7fa6bc0fe850 2024-03-29T08:21:47.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:47.473+0000 7fa6bb400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa6bc0fdeb0 0x7fa6bc0fbd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:47.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:47.473+0000 7fa6bb400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa6bc0fdeb0 0x7fa6bc0fbd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52158/0 (socket says 172.21.15.37:52158) 2024-03-29T08:21:47.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:47.473+0000 7fa6bb400700 1 -- 172.21.15.37:0/65762387 learned_addr learned my addr 172.21.15.37:0/65762387 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:21:47.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:47.473+0000 7fa6baa00700 1 --2- 172.21.15.37:0/65762387 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa6bc0fe850 0x7fa6bc0fc2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:47.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:47.473+0000 7fa6bb400700 1 -- 172.21.15.37:0/65762387 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa6bc0fe850 msgr2=0x7fa6bc0fc2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:47.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:47.473+0000 7fa6bb400700 1 --2- 172.21.15.37:0/65762387 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa6bc0fe850 0x7fa6bc0fc2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:47.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:47.473+0000 7fa6bb400700 1 -- 172.21.15.37:0/65762387 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6ac005040 con 0x7fa6bc0fdeb0 2024-03-29T08:21:47.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:47.473+0000 7fa6bb400700 1 --2- 172.21.15.37:0/65762387 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa6bc0fdeb0 0x7fa6bc0fbd70 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fa6ac009360 tx=0x7fa6ac009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:21:47.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:47.474+0000 7fa6b3400700 1 -- 172.21.15.37:0/65762387 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa6ac00e040 con 0x7fa6bc0fdeb0 2024-03-29T08:21:47.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:47.474+0000 7fa6b3400700 1 -- 172.21.15.37:0/65762387 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fa6ac004290 con 0x7fa6bc0fdeb0 2024-03-29T08:21:47.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:47.474+0000 7fa6c2800700 1 -- 172.21.15.37:0/65762387 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6bc0fc8b0 con 0x7fa6bc0fdeb0 2024-03-29T08:21:47.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:47.474+0000 7fa6b3400700 1 -- 172.21.15.37:0/65762387 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa6ac01c3f0 con 0x7fa6bc0fdeb0 2024-03-29T08:21:47.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:47.474+0000 7fa6c2800700 1 -- 172.21.15.37:0/65762387 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6bc11e700 con 0x7fa6bc0fdeb0 2024-03-29T08:21:47.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:47.475+0000 7fa6c2800700 1 -- 172.21.15.37:0/65762387 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6bc065e10 con 0x7fa6bc0fdeb0 2024-03-29T08:21:47.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:47.476+0000 7fa6b3400700 1 -- 172.21.15.37:0/65762387 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fa6ac013420 con 0x7fa6bc0fdeb0 2024-03-29T08:21:47.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:47.477+0000 7fa6b3400700 1 --2- 172.21.15.37:0/65762387 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fa6a006cc30 0x7fa6a006f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:47.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:47.477+0000 7fa6b3400700 1 -- 172.21.15.37:0/65762387 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fa6ac08d3e0 con 0x7fa6bc0fdeb0 2024-03-29T08:21:47.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:47.477+0000 7fa6baa00700 1 --2- 172.21.15.37:0/65762387 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fa6a006cc30 0x7fa6a006f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:47.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:47.478+0000 7fa6baa00700 1 --2- 172.21.15.37:0/65762387 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fa6a006cc30 0x7fa6a006f0e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fa6a8007fd0 tx=0x7fa6a8009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:21:47.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:47.481+0000 7fa6b3400700 1 -- 172.21.15.37:0/65762387 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fa6ac057460 con 0x7fa6bc0fdeb0 2024-03-29T08:21:47.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:47.714+0000 7fa6c2800700 1 -- 172.21.15.37:0/65762387 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa6bc04bed0 con 0x7fa6a006cc30 2024-03-29T08:21:47.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:47.715+0000 7fa6b3400700 1 -- 172.21.15.37:0/65762387 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fa6bc04bed0 con 0x7fa6a006cc30 2024-03-29T08:21:47.715 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-29T08:21:47.715 INFO:teuthology.orchestra.run.smithi037.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59", 2024-03-29T08:21:47.715 INFO:teuthology.orchestra.run.smithi037.stdout: "in_progress": true, 2024-03-29T08:21:47.715 INFO:teuthology.orchestra.run.smithi037.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-29T08:21:47.715 INFO:teuthology.orchestra.run.smithi037.stdout: "services_complete": [ 2024-03-29T08:21:47.715 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr", 2024-03-29T08:21:47.716 INFO:teuthology.orchestra.run.smithi037.stdout: "mon", 2024-03-29T08:21:47.716 INFO:teuthology.orchestra.run.smithi037.stdout: "crash", 2024-03-29T08:21:47.716 INFO:teuthology.orchestra.run.smithi037.stdout: "osd" 2024-03-29T08:21:47.716 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-29T08:21:47.716 INFO:teuthology.orchestra.run.smithi037.stdout: "progress": "15/21 daemons upgraded", 2024-03-29T08:21:47.716 INFO:teuthology.orchestra.run.smithi037.stdout: "message": "Currently upgrading mds daemons", 2024-03-29T08:21:47.716 INFO:teuthology.orchestra.run.smithi037.stdout: "is_paused": false 2024-03-29T08:21:47.716 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-29T08:21:47.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:47.717+0000 7fa6c2800700 1 -- 172.21.15.37:0/65762387 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fa6a006cc30 msgr2=0x7fa6a006f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:47.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:47.717+0000 7fa6c2800700 1 --2- 172.21.15.37:0/65762387 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fa6a006cc30 0x7fa6a006f0e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fa6a8007fd0 tx=0x7fa6a8009040 comp rx=0 tx=0).stop 2024-03-29T08:21:47.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:47.717+0000 7fa6c2800700 1 -- 172.21.15.37:0/65762387 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa6bc0fdeb0 msgr2=0x7fa6bc0fbd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:47.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:47.717+0000 7fa6c2800700 1 --2- 172.21.15.37:0/65762387 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa6bc0fdeb0 0x7fa6bc0fbd70 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fa6ac009360 tx=0x7fa6ac009e30 comp rx=0 tx=0).stop 2024-03-29T08:21:47.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:47.718+0000 7fa6c2800700 1 -- 172.21.15.37:0/65762387 shutdown_connections 2024-03-29T08:21:47.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:47.718+0000 7fa6c2800700 1 --2- 172.21.15.37:0/65762387 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fa6a006cc30 0x7fa6a006f0e0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:47.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:47.718+0000 7fa6c2800700 1 --2- 172.21.15.37:0/65762387 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa6bc0fdeb0 0x7fa6bc0fbd70 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:47.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:47.718+0000 7fa6c2800700 1 --2- 172.21.15.37:0/65762387 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa6bc0fe850 0x7fa6bc0fc2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:47.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:47.718+0000 7fa6c2800700 1 -- 172.21.15.37:0/65762387 >> 172.21.15.37:0/65762387 conn(0x7fa6bc072bc0 msgr2=0x7fa6bc0ff410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:21:47.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:47.718+0000 7fa6c2800700 1 -- 172.21.15.37:0/65762387 shutdown_connections 2024-03-29T08:21:47.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:47.718+0000 7fa6c2800700 1 -- 172.21.15.37:0/65762387 wait complete. 2024-03-29T08:21:47.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:47.854+0000 7fd163200700 1 -- 172.21.15.37:0/1506639959 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd1641035e0 msgr2=0x7fd1640fcc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:47.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:47.854+0000 7fd163200700 1 --2- 172.21.15.37:0/1506639959 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd1641035e0 0x7fd1640fcc50 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fd150004940 tx=0x7fd150009e30 comp rx=0 tx=0).stop 2024-03-29T08:21:47.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:47.854+0000 7fd163200700 1 -- 172.21.15.37:0/1506639959 shutdown_connections 2024-03-29T08:21:47.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:47.854+0000 7fd163200700 1 --2- 172.21.15.37:0/1506639959 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd1641035e0 0x7fd1640fcc50 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:47.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:47.854+0000 7fd163200700 1 --2- 172.21.15.37:0/1506639959 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd164102c40 0x7fd164103010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:47.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:47.854+0000 7fd163200700 1 -- 172.21.15.37:0/1506639959 >> 172.21.15.37:0/1506639959 conn(0x7fd1640f8830 msgr2=0x7fd1640fac40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:21:47.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:47.855+0000 7fd163200700 1 -- 172.21.15.37:0/1506639959 shutdown_connections 2024-03-29T08:21:47.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:47.855+0000 7fd163200700 1 -- 172.21.15.37:0/1506639959 wait complete. 2024-03-29T08:21:47.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:47.855+0000 7fd163200700 1 Processor -- start 2024-03-29T08:21:47.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:47.856+0000 7fd163200700 1 -- start start 2024-03-29T08:21:47.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:47.856+0000 7fd163200700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd164102c40 0x7fd164115af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:47.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:47.856+0000 7fd163200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd1641035e0 0x7fd164116030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:47.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:47.856+0000 7fd163200700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd164116680 con 0x7fd1641035e0 2024-03-29T08:21:47.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:47.856+0000 7fd163200700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd16410fa70 con 0x7fd164102c40 2024-03-29T08:21:47.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:47.856+0000 7fd161400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd1641035e0 0x7fd164116030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:47.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:47.856+0000 7fd161400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd1641035e0 0x7fd164116030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52184/0 (socket says 172.21.15.37:52184) 2024-03-29T08:21:47.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:47.856+0000 7fd161400700 1 -- 172.21.15.37:0/158123819 learned_addr learned my addr 172.21.15.37:0/158123819 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:21:47.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:47.856+0000 7fd161e00700 1 --2- 172.21.15.37:0/158123819 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd164102c40 0x7fd164115af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:47.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:47.857+0000 7fd161400700 1 -- 172.21.15.37:0/158123819 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd164102c40 msgr2=0x7fd164115af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:47.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:47.857+0000 7fd161400700 1 --2- 172.21.15.37:0/158123819 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd164102c40 0x7fd164115af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:47.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:47.857+0000 7fd161400700 1 -- 172.21.15.37:0/158123819 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd150005040 con 0x7fd1641035e0 2024-03-29T08:21:47.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:47.857+0000 7fd161e00700 1 --2- 172.21.15.37:0/158123819 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd164102c40 0x7fd164115af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:21:47.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:47.857+0000 7fd161400700 1 --2- 172.21.15.37:0/158123819 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd1641035e0 0x7fd164116030 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fd150009390 tx=0x7fd150009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:21:47.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:47.857+0000 7fd15a000700 1 -- 172.21.15.37:0/158123819 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd15000e040 con 0x7fd1641035e0 2024-03-29T08:21:47.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:47.857+0000 7fd15a000700 1 -- 172.21.15.37:0/158123819 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd150013420 con 0x7fd1641035e0 2024-03-29T08:21:47.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:47.857+0000 7fd163200700 1 -- 172.21.15.37:0/158123819 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd16410fcf0 con 0x7fd1641035e0 2024-03-29T08:21:47.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:47.857+0000 7fd15a000700 1 -- 172.21.15.37:0/158123819 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd15001c630 con 0x7fd1641035e0 2024-03-29T08:21:47.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:47.858+0000 7fd163200700 1 -- 172.21.15.37:0/158123819 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1641102b0 con 0x7fd1641035e0 2024-03-29T08:21:47.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:47.858+0000 7fd163200700 1 -- 172.21.15.37:0/158123819 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1640fe390 con 0x7fd1641035e0 2024-03-29T08:21:47.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:47.859+0000 7fd15a000700 1 -- 172.21.15.37:0/158123819 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fd150013a00 con 0x7fd1641035e0 2024-03-29T08:21:47.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:47.860+0000 7fd15a000700 1 --2- 172.21.15.37:0/158123819 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fd14806cc30 0x7fd14806f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:47.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:47.860+0000 7fd15a000700 1 -- 172.21.15.37:0/158123819 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fd15008d2c0 con 0x7fd1641035e0 2024-03-29T08:21:47.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:47.861+0000 7fd161e00700 1 --2- 172.21.15.37:0/158123819 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fd14806cc30 0x7fd14806f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:47.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:47.861+0000 7fd161e00700 1 --2- 172.21.15.37:0/158123819 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fd14806cc30 0x7fd14806f0e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fd154007fd0 tx=0x7fd154009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:21:47.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:47.865+0000 7fd15a000700 1 -- 172.21.15.37:0/158123819 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fd15005ad90 con 0x7fd1641035e0 2024-03-29T08:21:48.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:21:47 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/1014936915' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:21:48.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:21:47 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/1515000988' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T08:21:48.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:21:47 smithi148 ceph-mon[106692]: pgmap v327: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:21:48.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:48.162+0000 7fd163200700 1 -- 172.21.15.37:0/158123819 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fd164109bd0 con 0x7fd1641035e0 2024-03-29T08:21:48.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:48.162+0000 7fd15a000700 1 -- 172.21.15.37:0/158123819 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7fd15005a920 con 0x7fd1641035e0 2024-03-29T08:21:48.163 INFO:teuthology.orchestra.run.smithi037.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-03-29T08:21:48.163 INFO:teuthology.orchestra.run.smithi037.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-03-29T08:21:48.163 INFO:teuthology.orchestra.run.smithi037.stdout: fs cephfs is degraded 2024-03-29T08:21:48.163 INFO:teuthology.orchestra.run.smithi037.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-03-29T08:21:48.163 INFO:teuthology.orchestra.run.smithi037.stdout: fs cephfs is offline because no MDS is active for it. 2024-03-29T08:21:48.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:48.165+0000 7fd163200700 1 -- 172.21.15.37:0/158123819 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fd14806cc30 msgr2=0x7fd14806f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:48.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:48.165+0000 7fd163200700 1 --2- 172.21.15.37:0/158123819 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fd14806cc30 0x7fd14806f0e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fd154007fd0 tx=0x7fd154009040 comp rx=0 tx=0).stop 2024-03-29T08:21:48.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:48.165+0000 7fd163200700 1 -- 172.21.15.37:0/158123819 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd1641035e0 msgr2=0x7fd164116030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:48.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:48.165+0000 7fd163200700 1 --2- 172.21.15.37:0/158123819 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd1641035e0 0x7fd164116030 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fd150009390 tx=0x7fd150009e30 comp rx=0 tx=0).stop 2024-03-29T08:21:48.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:48.165+0000 7fd163200700 1 -- 172.21.15.37:0/158123819 shutdown_connections 2024-03-29T08:21:48.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:48.165+0000 7fd163200700 1 --2- 172.21.15.37:0/158123819 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fd14806cc30 0x7fd14806f0e0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:48.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:48.165+0000 7fd163200700 1 --2- 172.21.15.37:0/158123819 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd164102c40 0x7fd164115af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:48.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:48.165+0000 7fd163200700 1 --2- 172.21.15.37:0/158123819 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd1641035e0 0x7fd164116030 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:48.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:48.165+0000 7fd163200700 1 -- 172.21.15.37:0/158123819 >> 172.21.15.37:0/158123819 conn(0x7fd1640f8830 msgr2=0x7fd1640f98b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:21:48.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:48.165+0000 7fd163200700 1 -- 172.21.15.37:0/158123819 shutdown_connections 2024-03-29T08:21:48.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:21:48.165+0000 7fd163200700 1 -- 172.21.15.37:0/158123819 wait complete. 2024-03-29T08:21:48.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:21:47 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/1014936915' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:21:48.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:21:47 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/1515000988' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T08:21:48.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:21:47 smithi037 ceph-mon[142325]: pgmap v327: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:21:48.996 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:21:48 smithi148 ceph-mon[106692]: from='client.34664 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:21:48.996 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:21:48 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/158123819' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T08:21:49.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:21:48 smithi037 ceph-mon[142325]: from='client.34664 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:21:49.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:21:48 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/158123819' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T08:21:50.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:21:50 smithi148 ceph-mon[106692]: pgmap v328: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:21:50.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:21:50 smithi037 ceph-mon[142325]: pgmap v328: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:21:52.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:21:51 smithi148 ceph-mon[106692]: pgmap v329: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:21:52.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:21:51 smithi037 ceph-mon[142325]: pgmap v329: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:21:54.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:21:53 smithi148 ceph-mon[106692]: pgmap v330: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:21:54.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:21:53 smithi037 ceph-mon[142325]: pgmap v330: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:21:55.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:21:55 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:21:55.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:21:55 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:21:56.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:21:56 smithi148 ceph-mon[106692]: pgmap v331: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:21:56.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:21:56 smithi037 ceph-mon[142325]: pgmap v331: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:21:58.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:21:57 smithi148 ceph-mon[106692]: pgmap v332: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:21:58.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:21:57 smithi037 ceph-mon[142325]: pgmap v332: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:22:00.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:22:00 smithi148 ceph-mon[106692]: pgmap v333: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:22:00.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:22:00 smithi037 ceph-mon[142325]: pgmap v333: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:22:02.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:22:01 smithi148 ceph-mon[106692]: pgmap v334: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:22:02.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:22:01 smithi037 ceph-mon[142325]: pgmap v334: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:22:04.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:22:03 smithi148 ceph-mon[106692]: pgmap v335: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:22:04.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:22:03 smithi037 ceph-mon[142325]: pgmap v335: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:22:06.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:22:05 smithi148 ceph-mon[106692]: pgmap v336: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:22:06.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:22:05 smithi037 ceph-mon[142325]: pgmap v336: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:22:08.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:22:07 smithi148 ceph-mon[106692]: pgmap v337: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:22:08.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:22:07 smithi037 ceph-mon[142325]: pgmap v337: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:22:10.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:22:10 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:22:10.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:22:10 smithi148 ceph-mon[106692]: pgmap v338: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:22:10.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:22:10 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:22:10.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:22:10 smithi037 ceph-mon[142325]: pgmap v338: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:22:12.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:22:11 smithi148 ceph-mon[106692]: pgmap v339: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:22:12.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:22:11 smithi037 ceph-mon[142325]: pgmap v339: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:22:14.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:22:13 smithi148 ceph-mon[106692]: pgmap v340: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:22:14.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:22:13 smithi037 ceph-mon[142325]: pgmap v340: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:22:16.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:22:15 smithi148 ceph-mon[106692]: pgmap v341: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:22:16.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:22:15 smithi037 ceph-mon[142325]: pgmap v341: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:22:18.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:22:17 smithi148 ceph-mon[106692]: pgmap v342: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:22:18.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:22:17 smithi037 ceph-mon[142325]: pgmap v342: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:22:18.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:18.317+0000 7f57e1200700 1 -- 172.21.15.37:0/208445238 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f57dc102e00 msgr2=0x7f57dc1031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:18.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:18.317+0000 7f57e1200700 1 --2- 172.21.15.37:0/208445238 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f57dc102e00 0x7f57dc1031d0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f57cc004940 tx=0x7f57cc009e30 comp rx=0 tx=0).stop 2024-03-29T08:22:18.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:18.317+0000 7f57e1200700 1 -- 172.21.15.37:0/208445238 shutdown_connections 2024-03-29T08:22:18.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:18.317+0000 7f57e1200700 1 --2- 172.21.15.37:0/208445238 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f57dc1037a0 0x7f57dc0fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:18.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:18.317+0000 7f57e1200700 1 --2- 172.21.15.37:0/208445238 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f57dc102e00 0x7f57dc1031d0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:18.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:18.317+0000 7f57e1200700 1 -- 172.21.15.37:0/208445238 >> 172.21.15.37:0/208445238 conn(0x7f57dc0f89b0 msgr2=0x7f57dc0fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:22:18.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:18.317+0000 7f57e1200700 1 -- 172.21.15.37:0/208445238 shutdown_connections 2024-03-29T08:22:18.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:18.317+0000 7f57e1200700 1 -- 172.21.15.37:0/208445238 wait complete. 2024-03-29T08:22:18.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:18.318+0000 7f57e1200700 1 Processor -- start 2024-03-29T08:22:18.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:18.318+0000 7f57e1200700 1 -- start start 2024-03-29T08:22:18.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:18.319+0000 7f57e1200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f57dc102e00 0x7f57dc115cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:18.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:18.319+0000 7f57e1200700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f57dc1037a0 0x7f57dc116230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:18.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:18.319+0000 7f57e1200700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57dc116880 con 0x7f57dc102e00 2024-03-29T08:22:18.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:18.319+0000 7f57e1200700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57dc10fc50 con 0x7f57dc1037a0 2024-03-29T08:22:18.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:18.319+0000 7f57d9e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f57dc102e00 0x7f57dc115cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:18.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:18.319+0000 7f57d9e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f57dc102e00 0x7f57dc115cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:33478/0 (socket says 172.21.15.37:33478) 2024-03-29T08:22:18.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:18.319+0000 7f57d9e00700 1 -- 172.21.15.37:0/3046339742 learned_addr learned my addr 172.21.15.37:0/3046339742 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:22:18.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:18.319+0000 7f57d9400700 1 --2- 172.21.15.37:0/3046339742 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f57dc1037a0 0x7f57dc116230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:18.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:18.320+0000 7f57d9e00700 1 -- 172.21.15.37:0/3046339742 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f57dc1037a0 msgr2=0x7f57dc116230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:18.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:18.320+0000 7f57d9e00700 1 --2- 172.21.15.37:0/3046339742 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f57dc1037a0 0x7f57dc116230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:18.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:18.320+0000 7f57d9e00700 1 -- 172.21.15.37:0/3046339742 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57cc005040 con 0x7f57dc102e00 2024-03-29T08:22:18.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:18.320+0000 7f57d9e00700 1 --2- 172.21.15.37:0/3046339742 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f57dc102e00 0x7f57dc115cf0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f57cc009360 tx=0x7f57cc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:18.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:18.320+0000 7f57d2000700 1 -- 172.21.15.37:0/3046339742 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f57cc00e040 con 0x7f57dc102e00 2024-03-29T08:22:18.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:18.320+0000 7f57d2000700 1 -- 172.21.15.37:0/3046339742 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f57cc004290 con 0x7f57dc102e00 2024-03-29T08:22:18.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:18.320+0000 7f57e1200700 1 -- 172.21.15.37:0/3046339742 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57dc10fed0 con 0x7f57dc102e00 2024-03-29T08:22:18.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:18.320+0000 7f57d2000700 1 -- 172.21.15.37:0/3046339742 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f57cc013d60 con 0x7f57dc102e00 2024-03-29T08:22:18.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:18.320+0000 7f57e1200700 1 -- 172.21.15.37:0/3046339742 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57dc110490 con 0x7f57dc102e00 2024-03-29T08:22:18.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:18.321+0000 7f57e1200700 1 -- 172.21.15.37:0/3046339742 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57dc0fe550 con 0x7f57dc102e00 2024-03-29T08:22:18.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:18.323+0000 7f57d2000700 1 -- 172.21.15.37:0/3046339742 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f57cc00ba80 con 0x7f57dc102e00 2024-03-29T08:22:18.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:18.323+0000 7f57d2000700 1 --2- 172.21.15.37:0/3046339742 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f57c006cc30 0x7f57c006f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:18.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:18.324+0000 7f57d2000700 1 -- 172.21.15.37:0/3046339742 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f57cc08dc90 con 0x7f57dc102e00 2024-03-29T08:22:18.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:18.324+0000 7f57d9400700 1 --2- 172.21.15.37:0/3046339742 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f57c006cc30 0x7f57c006f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:18.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:18.324+0000 7f57d9400700 1 --2- 172.21.15.37:0/3046339742 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f57c006cc30 0x7f57c006f0e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f57dc111590 tx=0x7f57c800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:18.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:18.327+0000 7f57d2000700 1 -- 172.21.15.37:0/3046339742 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f57cc05b6e0 con 0x7f57dc102e00 2024-03-29T08:22:18.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:18.558+0000 7f57e1200700 1 -- 172.21.15.37:0/3046339742 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f57dc04bf70 con 0x7f57c006cc30 2024-03-29T08:22:18.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:18.560+0000 7f57d2000700 1 -- 172.21.15.37:0/3046339742 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f57c800f040 con 0x7f57c006cc30 2024-03-29T08:22:18.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:18.562+0000 7f57e1200700 1 -- 172.21.15.37:0/3046339742 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f57c006cc30 msgr2=0x7f57c006f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:18.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:18.562+0000 7f57e1200700 1 --2- 172.21.15.37:0/3046339742 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f57c006cc30 0x7f57c006f0e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f57dc111590 tx=0x7f57c800b040 comp rx=0 tx=0).stop 2024-03-29T08:22:18.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:18.562+0000 7f57e1200700 1 -- 172.21.15.37:0/3046339742 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f57dc102e00 msgr2=0x7f57dc115cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:18.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:18.562+0000 7f57e1200700 1 --2- 172.21.15.37:0/3046339742 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f57dc102e00 0x7f57dc115cf0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f57cc009360 tx=0x7f57cc009e30 comp rx=0 tx=0).stop 2024-03-29T08:22:18.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:18.562+0000 7f57e1200700 1 -- 172.21.15.37:0/3046339742 shutdown_connections 2024-03-29T08:22:18.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:18.562+0000 7f57e1200700 1 --2- 172.21.15.37:0/3046339742 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f57c006cc30 0x7f57c006f0e0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:18.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:18.562+0000 7f57e1200700 1 --2- 172.21.15.37:0/3046339742 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f57dc102e00 0x7f57dc115cf0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:18.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:18.562+0000 7f57e1200700 1 --2- 172.21.15.37:0/3046339742 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f57dc1037a0 0x7f57dc116230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:18.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:18.562+0000 7f57e1200700 1 -- 172.21.15.37:0/3046339742 >> 172.21.15.37:0/3046339742 conn(0x7f57dc0f89b0 msgr2=0x7f57dc0f99e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:22:18.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:18.562+0000 7f57e1200700 1 -- 172.21.15.37:0/3046339742 shutdown_connections 2024-03-29T08:22:18.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:18.562+0000 7f57e1200700 1 -- 172.21.15.37:0/3046339742 wait complete. 2024-03-29T08:22:18.578 INFO:teuthology.orchestra.run.smithi037.stdout:true 2024-03-29T08:22:18.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:18.701+0000 7fe415c00700 1 -- 172.21.15.37:0/3339580681 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe4100fe850 msgr2=0x7fe410106f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:18.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:18.701+0000 7fe415c00700 1 --2- 172.21.15.37:0/3339580681 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe4100fe850 0x7fe410106f20 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fe3fc004970 tx=0x7fe3fc009e30 comp rx=0 tx=0).stop 2024-03-29T08:22:18.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:18.701+0000 7fe415c00700 1 -- 172.21.15.37:0/3339580681 shutdown_connections 2024-03-29T08:22:18.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:18.701+0000 7fe415c00700 1 --2- 172.21.15.37:0/3339580681 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe4100fe850 0x7fe410106f20 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:18.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:18.701+0000 7fe415c00700 1 --2- 172.21.15.37:0/3339580681 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe4100fdeb0 0x7fe4100fe280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:18.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:18.701+0000 7fe415c00700 1 -- 172.21.15.37:0/3339580681 >> 172.21.15.37:0/3339580681 conn(0x7fe410072bc0 msgr2=0x7fe410072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:22:18.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:18.701+0000 7fe415c00700 1 -- 172.21.15.37:0/3339580681 shutdown_connections 2024-03-29T08:22:18.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:18.701+0000 7fe415c00700 1 -- 172.21.15.37:0/3339580681 wait complete. 2024-03-29T08:22:18.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:18.702+0000 7fe415c00700 1 Processor -- start 2024-03-29T08:22:18.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:18.702+0000 7fe415c00700 1 -- start start 2024-03-29T08:22:18.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:18.702+0000 7fe415c00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe4100fdeb0 0x7fe410115c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:18.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:18.702+0000 7fe415c00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe4100fe850 0x7fe4101161a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:18.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:18.702+0000 7fe415c00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4101167f0 con 0x7fe4100fe850 2024-03-29T08:22:18.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:18.702+0000 7fe415c00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe41010fc00 con 0x7fe4100fdeb0 2024-03-29T08:22:18.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:18.703+0000 7fe40e000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe4100fe850 0x7fe4101161a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:18.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:18.703+0000 7fe40e000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe4100fe850 0x7fe4101161a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:33494/0 (socket says 172.21.15.37:33494) 2024-03-29T08:22:18.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:18.703+0000 7fe40e000700 1 -- 172.21.15.37:0/3094292428 learned_addr learned my addr 172.21.15.37:0/3094292428 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:22:18.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:18.703+0000 7fe40e000700 1 -- 172.21.15.37:0/3094292428 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe4100fdeb0 msgr2=0x7fe410115c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:18.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:18.703+0000 7fe40e000700 1 --2- 172.21.15.37:0/3094292428 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe4100fdeb0 0x7fe410115c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:18.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:18.703+0000 7fe40e000700 1 -- 172.21.15.37:0/3094292428 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3fc005040 con 0x7fe4100fe850 2024-03-29T08:22:18.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:18.703+0000 7fe40e000700 1 --2- 172.21.15.37:0/3094292428 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe4100fe850 0x7fe4101161a0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fe3fc004f50 tx=0x7fe3fc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:18.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:18.703+0000 7fe406a00700 1 -- 172.21.15.37:0/3094292428 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe3fc00e040 con 0x7fe4100fe850 2024-03-29T08:22:18.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:18.704+0000 7fe406a00700 1 -- 172.21.15.37:0/3094292428 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fe3fc013420 con 0x7fe4100fe850 2024-03-29T08:22:18.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:18.704+0000 7fe415c00700 1 -- 172.21.15.37:0/3094292428 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe41010fe80 con 0x7fe4100fe850 2024-03-29T08:22:18.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:18.704+0000 7fe406a00700 1 -- 172.21.15.37:0/3094292428 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe3fc01c610 con 0x7fe4100fe850 2024-03-29T08:22:18.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:18.704+0000 7fe415c00700 1 -- 172.21.15.37:0/3094292428 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4101103c0 con 0x7fe4100fe850 2024-03-29T08:22:18.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:18.705+0000 7fe415c00700 1 -- 172.21.15.37:0/3094292428 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4101069f0 con 0x7fe4100fe850 2024-03-29T08:22:18.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:18.706+0000 7fe406a00700 1 -- 172.21.15.37:0/3094292428 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fe3fc013590 con 0x7fe4100fe850 2024-03-29T08:22:18.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:18.707+0000 7fe406a00700 1 --2- 172.21.15.37:0/3094292428 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fe3f406cc30 0x7fe3f406f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:18.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:18.707+0000 7fe406a00700 1 -- 172.21.15.37:0/3094292428 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fe3fc08d1e0 con 0x7fe4100fe850 2024-03-29T08:22:18.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:18.707+0000 7fe40ea00700 1 --2- 172.21.15.37:0/3094292428 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fe3f406cc30 0x7fe3f406f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:18.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:18.708+0000 7fe40ea00700 1 --2- 172.21.15.37:0/3094292428 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fe3f406cc30 0x7fe3f406f0e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fe400004d00 tx=0x7fe400009ce0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:18.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:18.712+0000 7fe406a00700 1 -- 172.21.15.37:0/3094292428 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fe3fc057220 con 0x7fe4100fe850 2024-03-29T08:22:18.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:18.943+0000 7fe415c00700 1 -- 172.21.15.37:0/3094292428 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe4101112f0 con 0x7fe3f406cc30 2024-03-29T08:22:18.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:18.944+0000 7fe406a00700 1 -- 172.21.15.37:0/3094292428 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fe4101112f0 con 0x7fe3f406cc30 2024-03-29T08:22:18.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:18.947+0000 7fe415c00700 1 -- 172.21.15.37:0/3094292428 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fe3f406cc30 msgr2=0x7fe3f406f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:18.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:18.947+0000 7fe415c00700 1 --2- 172.21.15.37:0/3094292428 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fe3f406cc30 0x7fe3f406f0e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fe400004d00 tx=0x7fe400009ce0 comp rx=0 tx=0).stop 2024-03-29T08:22:18.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:18.947+0000 7fe415c00700 1 -- 172.21.15.37:0/3094292428 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe4100fe850 msgr2=0x7fe4101161a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:18.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:18.947+0000 7fe415c00700 1 --2- 172.21.15.37:0/3094292428 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe4100fe850 0x7fe4101161a0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fe3fc004f50 tx=0x7fe3fc009e30 comp rx=0 tx=0).stop 2024-03-29T08:22:18.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:18.947+0000 7fe415c00700 1 -- 172.21.15.37:0/3094292428 shutdown_connections 2024-03-29T08:22:18.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:18.947+0000 7fe415c00700 1 --2- 172.21.15.37:0/3094292428 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fe3f406cc30 0x7fe3f406f0e0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:18.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:18.947+0000 7fe415c00700 1 --2- 172.21.15.37:0/3094292428 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe4100fdeb0 0x7fe410115c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:18.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:18.947+0000 7fe415c00700 1 --2- 172.21.15.37:0/3094292428 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe4100fe850 0x7fe4101161a0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:18.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:18.947+0000 7fe415c00700 1 -- 172.21.15.37:0/3094292428 >> 172.21.15.37:0/3094292428 conn(0x7fe410072bc0 msgr2=0x7fe4100fd4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:22:18.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:18.947+0000 7fe415c00700 1 -- 172.21.15.37:0/3094292428 shutdown_connections 2024-03-29T08:22:18.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:18.947+0000 7fe415c00700 1 -- 172.21.15.37:0/3094292428 wait complete. 2024-03-29T08:22:19.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:22:18 smithi148 ceph-mon[106692]: from='client.34672 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:22:19.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:19.083+0000 7f8f74800700 1 -- 172.21.15.37:0/3603029156 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8f6c1037a0 msgr2=0x7f8f6c0fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:19.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:19.083+0000 7f8f74800700 1 --2- 172.21.15.37:0/3603029156 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8f6c1037a0 0x7f8f6c0fce10 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f8f5c004940 tx=0x7f8f5c009e30 comp rx=0 tx=0).stop 2024-03-29T08:22:19.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:19.083+0000 7f8f74800700 1 -- 172.21.15.37:0/3603029156 shutdown_connections 2024-03-29T08:22:19.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:19.083+0000 7f8f74800700 1 --2- 172.21.15.37:0/3603029156 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8f6c1037a0 0x7f8f6c0fce10 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:19.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:19.083+0000 7f8f74800700 1 --2- 172.21.15.37:0/3603029156 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8f6c102e00 0x7f8f6c1031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:19.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:19.083+0000 7f8f74800700 1 -- 172.21.15.37:0/3603029156 >> 172.21.15.37:0/3603029156 conn(0x7f8f6c0f89b0 msgr2=0x7f8f6c0fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:22:19.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:19.084+0000 7f8f74800700 1 -- 172.21.15.37:0/3603029156 shutdown_connections 2024-03-29T08:22:19.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:19.084+0000 7f8f74800700 1 -- 172.21.15.37:0/3603029156 wait complete. 2024-03-29T08:22:19.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:19.084+0000 7f8f74800700 1 Processor -- start 2024-03-29T08:22:19.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:19.085+0000 7f8f74800700 1 -- start start 2024-03-29T08:22:19.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:19.085+0000 7f8f74800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8f6c102e00 0x7f8f6c113ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:19.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:19.085+0000 7f8f74800700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8f6c1037a0 0x7f8f6c114020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:19.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:19.085+0000 7f8f74800700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f6c114670 con 0x7f8f6c102e00 2024-03-29T08:22:19.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:19.085+0000 7f8f74800700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f6c10da60 con 0x7f8f6c1037a0 2024-03-29T08:22:19.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:19.085+0000 7f8f71e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8f6c102e00 0x7f8f6c113ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:19.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:19.085+0000 7f8f71e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8f6c102e00 0x7f8f6c113ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:33504/0 (socket says 172.21.15.37:33504) 2024-03-29T08:22:19.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:19.085+0000 7f8f71e00700 1 -- 172.21.15.37:0/1688497661 learned_addr learned my addr 172.21.15.37:0/1688497661 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:22:19.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:19.085+0000 7f8f71400700 1 --2- 172.21.15.37:0/1688497661 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8f6c1037a0 0x7f8f6c114020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:19.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:19.086+0000 7f8f71e00700 1 -- 172.21.15.37:0/1688497661 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8f6c1037a0 msgr2=0x7f8f6c114020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:19.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:19.086+0000 7f8f71e00700 1 --2- 172.21.15.37:0/1688497661 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8f6c1037a0 0x7f8f6c114020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:19.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:19.086+0000 7f8f71e00700 1 -- 172.21.15.37:0/1688497661 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f5c005040 con 0x7f8f6c102e00 2024-03-29T08:22:19.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:19.086+0000 7f8f71e00700 1 --2- 172.21.15.37:0/1688497661 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8f6c102e00 0x7f8f6c113ae0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f8f60002a50 tx=0x7f8f600046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:19.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:19.086+0000 7f8f66000700 1 -- 172.21.15.37:0/1688497661 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8f60007cc0 con 0x7f8f6c102e00 2024-03-29T08:22:19.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:19.086+0000 7f8f66000700 1 -- 172.21.15.37:0/1688497661 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f8f6000f040 con 0x7f8f6c102e00 2024-03-29T08:22:19.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:19.086+0000 7f8f74800700 1 -- 172.21.15.37:0/1688497661 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f6c10dd40 con 0x7f8f6c102e00 2024-03-29T08:22:19.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:19.086+0000 7f8f66000700 1 -- 172.21.15.37:0/1688497661 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8f6000d9c0 con 0x7f8f6c102e00 2024-03-29T08:22:19.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:19.086+0000 7f8f74800700 1 -- 172.21.15.37:0/1688497661 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f6c10e2d0 con 0x7f8f6c102e00 2024-03-29T08:22:19.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:19.088+0000 7f8f74800700 1 -- 172.21.15.37:0/1688497661 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f6c0fe550 con 0x7f8f6c102e00 2024-03-29T08:22:19.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:19.088+0000 7f8f66000700 1 -- 172.21.15.37:0/1688497661 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f8f6000db20 con 0x7f8f6c102e00 2024-03-29T08:22:19.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:19.089+0000 7f8f66000700 1 --2- 172.21.15.37:0/1688497661 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f8f5406cc30 0x7f8f5406f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:19.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:19.089+0000 7f8f66000700 1 -- 172.21.15.37:0/1688497661 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f8f6008bd20 con 0x7f8f6c102e00 2024-03-29T08:22:19.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:19.090+0000 7f8f71400700 1 --2- 172.21.15.37:0/1688497661 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f8f5406cc30 0x7f8f5406f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:19.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:19.090+0000 7f8f71400700 1 --2- 172.21.15.37:0/1688497661 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f8f5406cc30 0x7f8f5406f0e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f8f6c10ed40 tx=0x7f8f5c009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:19.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:19.094+0000 7f8f66000700 1 -- 172.21.15.37:0/1688497661 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f8f60059770 con 0x7f8f6c102e00 2024-03-29T08:22:19.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:22:18 smithi037 ceph-mon[142325]: from='client.34672 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:22:19.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:19.318+0000 7f8f74800700 1 -- 172.21.15.37:0/1688497661 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f8f6c109df0 con 0x7f8f5406cc30 2024-03-29T08:22:19.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:19.328+0000 7f8f66000700 1 -- 172.21.15.37:0/1688497661 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f8f6c109df0 con 0x7f8f5406cc30 2024-03-29T08:22:19.329 INFO:teuthology.orchestra.run.smithi037.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-29T08:22:19.330 INFO:teuthology.orchestra.run.smithi037.stdout:alertmanager.smithi037 smithi037 *:9093,9094 running (12m) 4m ago 25m 14.9M - 0.25.0 c8568f914cd2 2f9f9746ed0f 2024-03-29T08:22:19.330 INFO:teuthology.orchestra.run.smithi037.stdout:crash.smithi037 smithi037 running (9m) 4m ago 25m 6647k - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 178d416a3186 2024-03-29T08:22:19.330 INFO:teuthology.orchestra.run.smithi037.stdout:crash.smithi148 smithi148 running (8m) 4m ago 24m 6656k - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 23666989d58f 2024-03-29T08:22:19.330 INFO:teuthology.orchestra.run.smithi037.stdout:grafana.smithi037 smithi037 *:3000 running (11m) 4m ago 24m 71.9M - 9.4.7 954c08fa6188 6b976fe49952 2024-03-29T08:22:19.330 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi037.iifsmh smithi037 running (4m) 4m ago 18m 10.8M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 2d97f562cc5b 2024-03-29T08:22:19.330 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi037.mmwgxe smithi037 running (4m) 4m ago 18m 10.9M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff ec37037f46e8 2024-03-29T08:22:19.330 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi148.qukpjh smithi148 running (18m) 4m ago 18m 15.3M - 17.2.7-789-g4213e17c cf22eb8683be 8e469782a273 2024-03-29T08:22:19.330 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi148.sqteih smithi148 running (4m) 4m ago 18m 10.7M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 44c15c77037a 2024-03-29T08:22:19.330 INFO:teuthology.orchestra.run.smithi037.stdout:mgr.smithi037.wjikii smithi037 *:8443,9283,8765 running (14m) 4m ago 26m 506M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff c5a1417b9417 2024-03-29T08:22:19.330 INFO:teuthology.orchestra.run.smithi037.stdout:mgr.smithi148.onhgvm smithi148 *:8443,9283,8765 running (13m) 4m ago 23m 450M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff fcfdbce062bd 2024-03-29T08:22:19.330 INFO:teuthology.orchestra.run.smithi037.stdout:mon.smithi037 smithi037 running (10m) 4m ago 26m 54.8M 2048M 18.2.2-1015-gd0e392d9 1cc2f45c17ff eea2060783af 2024-03-29T08:22:19.330 INFO:teuthology.orchestra.run.smithi037.stdout:mon.smithi148 smithi148 running (9m) 4m ago 23m 43.7M 2048M 18.2.2-1015-gd0e392d9 1cc2f45c17ff ad4b2c5748be 2024-03-29T08:22:19.330 INFO:teuthology.orchestra.run.smithi037.stdout:node-exporter.smithi037 smithi037 *:9100 running (12m) 4m ago 25m 7625k - 1.5.0 0da6a335fe13 26b8ae9cb878 2024-03-29T08:22:19.331 INFO:teuthology.orchestra.run.smithi037.stdout:node-exporter.smithi148 smithi148 *:9100 running (12m) 4m ago 24m 8166k - 1.5.0 0da6a335fe13 98414e4e64ed 2024-03-29T08:22:19.331 INFO:teuthology.orchestra.run.smithi037.stdout:osd.0 smithi037 running (8m) 4m ago 23m 104M 1619M 18.2.2-1015-gd0e392d9 1cc2f45c17ff c0c584f6245c 2024-03-29T08:22:19.331 INFO:teuthology.orchestra.run.smithi037.stdout:osd.1 smithi037 running (7m) 4m ago 22m 100M 1619M 18.2.2-1015-gd0e392d9 1cc2f45c17ff a4a2ff4cc83d 2024-03-29T08:22:19.331 INFO:teuthology.orchestra.run.smithi037.stdout:osd.2 smithi037 running (7m) 4m ago 21m 82.3M 1619M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 8b1bc71aa79e 2024-03-29T08:22:19.331 INFO:teuthology.orchestra.run.smithi037.stdout:osd.3 smithi148 running (6m) 4m ago 21m 105M 2642M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 1c656ab8ff86 2024-03-29T08:22:19.331 INFO:teuthology.orchestra.run.smithi037.stdout:osd.4 smithi148 running (5m) 4m ago 20m 97.3M 2642M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 966a9de11d22 2024-03-29T08:22:19.331 INFO:teuthology.orchestra.run.smithi037.stdout:osd.5 smithi148 running (5m) 4m ago 20m 85.8M 2642M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 474c4f9212ce 2024-03-29T08:22:19.331 INFO:teuthology.orchestra.run.smithi037.stdout:prometheus.smithi037 smithi037 *:9095 running (12m) 4m ago 24m 54.4M - 2.43.0 a07b618ecd1d 30c0223ec589 2024-03-29T08:22:19.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:19.330+0000 7f8f74800700 1 -- 172.21.15.37:0/1688497661 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f8f5406cc30 msgr2=0x7f8f5406f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:19.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:19.330+0000 7f8f74800700 1 --2- 172.21.15.37:0/1688497661 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f8f5406cc30 0x7f8f5406f0e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f8f6c10ed40 tx=0x7f8f5c009e30 comp rx=0 tx=0).stop 2024-03-29T08:22:19.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:19.330+0000 7f8f74800700 1 -- 172.21.15.37:0/1688497661 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8f6c102e00 msgr2=0x7f8f6c113ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:19.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:19.330+0000 7f8f74800700 1 --2- 172.21.15.37:0/1688497661 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8f6c102e00 0x7f8f6c113ae0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f8f60002a50 tx=0x7f8f600046b0 comp rx=0 tx=0).stop 2024-03-29T08:22:19.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:19.331+0000 7f8f74800700 1 -- 172.21.15.37:0/1688497661 shutdown_connections 2024-03-29T08:22:19.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:19.331+0000 7f8f74800700 1 --2- 172.21.15.37:0/1688497661 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f8f5406cc30 0x7f8f5406f0e0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:19.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:19.331+0000 7f8f74800700 1 --2- 172.21.15.37:0/1688497661 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8f6c102e00 0x7f8f6c113ae0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:19.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:19.331+0000 7f8f74800700 1 --2- 172.21.15.37:0/1688497661 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8f6c1037a0 0x7f8f6c114020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:19.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:19.331+0000 7f8f74800700 1 -- 172.21.15.37:0/1688497661 >> 172.21.15.37:0/1688497661 conn(0x7f8f6c0f89b0 msgr2=0x7f8f6c0f9a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:22:19.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:19.331+0000 7f8f74800700 1 -- 172.21.15.37:0/1688497661 shutdown_connections 2024-03-29T08:22:19.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:19.331+0000 7f8f74800700 1 -- 172.21.15.37:0/1688497661 wait complete. 2024-03-29T08:22:19.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:19.467+0000 7f48d6400700 1 -- 172.21.15.37:0/1125314913 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f48d00fe830 msgr2=0x7f48d0106f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:19.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:19.467+0000 7f48d6400700 1 --2- 172.21.15.37:0/1125314913 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f48d00fe830 0x7f48d0106f00 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f48bc004940 tx=0x7f48bc009e30 comp rx=0 tx=0).stop 2024-03-29T08:22:19.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:19.467+0000 7f48d6400700 1 -- 172.21.15.37:0/1125314913 shutdown_connections 2024-03-29T08:22:19.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:19.467+0000 7f48d6400700 1 --2- 172.21.15.37:0/1125314913 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f48d00fe830 0x7f48d0106f00 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:19.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:19.467+0000 7f48d6400700 1 --2- 172.21.15.37:0/1125314913 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f48d00fde90 0x7f48d00fe260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:19.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:19.467+0000 7f48d6400700 1 -- 172.21.15.37:0/1125314913 >> 172.21.15.37:0/1125314913 conn(0x7f48d0072bc0 msgr2=0x7f48d0072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:22:19.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:19.467+0000 7f48d6400700 1 -- 172.21.15.37:0/1125314913 shutdown_connections 2024-03-29T08:22:19.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:19.467+0000 7f48d6400700 1 -- 172.21.15.37:0/1125314913 wait complete. 2024-03-29T08:22:19.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:19.468+0000 7f48d6400700 1 Processor -- start 2024-03-29T08:22:19.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:19.468+0000 7f48d6400700 1 -- start start 2024-03-29T08:22:19.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:19.468+0000 7f48d6400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f48d00fde90 0x7f48d0115d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:19.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:19.469+0000 7f48d6400700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f48d00fe830 0x7f48d0116270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:19.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:19.469+0000 7f48d6400700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48d01168c0 con 0x7f48d00fde90 2024-03-29T08:22:19.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:19.469+0000 7f48d6400700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48d010fc90 con 0x7f48d00fe830 2024-03-29T08:22:19.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:19.469+0000 7f48cf400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f48d00fde90 0x7f48d0115d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:19.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:19.469+0000 7f48cf400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f48d00fde90 0x7f48d0115d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:33526/0 (socket says 172.21.15.37:33526) 2024-03-29T08:22:19.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:19.469+0000 7f48cf400700 1 -- 172.21.15.37:0/4073841221 learned_addr learned my addr 172.21.15.37:0/4073841221 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:22:19.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:19.469+0000 7f48cea00700 1 --2- 172.21.15.37:0/4073841221 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f48d00fe830 0x7f48d0116270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:19.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:19.469+0000 7f48cf400700 1 -- 172.21.15.37:0/4073841221 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f48d00fe830 msgr2=0x7f48d0116270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:19.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:19.469+0000 7f48cf400700 1 --2- 172.21.15.37:0/4073841221 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f48d00fe830 0x7f48d0116270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:19.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:19.469+0000 7f48cf400700 1 -- 172.21.15.37:0/4073841221 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48bc005040 con 0x7f48d00fde90 2024-03-29T08:22:19.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:19.469+0000 7f48cf400700 1 --2- 172.21.15.37:0/4073841221 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f48d00fde90 0x7f48d0115d30 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f48c0002a50 tx=0x7f48c00046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:19.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:19.470+0000 7f48c7400700 1 -- 172.21.15.37:0/4073841221 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f48c0007cc0 con 0x7f48d00fde90 2024-03-29T08:22:19.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:19.470+0000 7f48c7400700 1 -- 172.21.15.37:0/4073841221 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f48c000f040 con 0x7f48d00fde90 2024-03-29T08:22:19.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:19.470+0000 7f48c7400700 1 -- 172.21.15.37:0/4073841221 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f48c0009540 con 0x7f48d00fde90 2024-03-29T08:22:19.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:19.470+0000 7f48d6400700 1 -- 172.21.15.37:0/4073841221 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48d010ff70 con 0x7f48d00fde90 2024-03-29T08:22:19.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:19.470+0000 7f48d6400700 1 -- 172.21.15.37:0/4073841221 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48d01104b0 con 0x7f48d00fde90 2024-03-29T08:22:19.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:19.471+0000 7f48d6400700 1 -- 172.21.15.37:0/4073841221 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48d0106a20 con 0x7f48d00fde90 2024-03-29T08:22:19.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:19.472+0000 7f48c7400700 1 -- 172.21.15.37:0/4073841221 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f48c00096a0 con 0x7f48d00fde90 2024-03-29T08:22:19.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:19.473+0000 7f48c7400700 1 --2- 172.21.15.37:0/4073841221 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f48b406cc30 0x7f48b406f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:19.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:19.474+0000 7f48c7400700 1 -- 172.21.15.37:0/4073841221 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f48c008bde0 con 0x7f48d00fde90 2024-03-29T08:22:19.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:19.474+0000 7f48cea00700 1 --2- 172.21.15.37:0/4073841221 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f48b406cc30 0x7f48b406f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:19.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:19.474+0000 7f48cea00700 1 --2- 172.21.15.37:0/4073841221 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f48b406cc30 0x7f48b406f0e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f48d01115d0 tx=0x7f48bc009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:19.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:19.479+0000 7f48c7400700 1 -- 172.21.15.37:0/4073841221 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f48c0059830 con 0x7f48d00fde90 2024-03-29T08:22:19.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:19.778+0000 7f48d6400700 1 -- 172.21.15.37:0/4073841221 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f48d004bed0 con 0x7f48d00fde90 2024-03-29T08:22:19.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:19.779+0000 7f48c7400700 1 -- 172.21.15.37:0/4073841221 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+847 (secure 0 0 0) 0x7f48c00593c0 con 0x7f48d00fde90 2024-03-29T08:22:19.780 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-29T08:22:19.780 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": { 2024-03-29T08:22:19.780 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 2 2024-03-29T08:22:19.780 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:22:19.780 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": { 2024-03-29T08:22:19.780 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 2 2024-03-29T08:22:19.780 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:22:19.780 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": { 2024-03-29T08:22:19.781 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 6 2024-03-29T08:22:19.781 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:22:19.781 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": { 2024-03-29T08:22:19.781 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 17.2.7-789-g4213e17c (4213e17c4de67016ed14f55d22503f4923b3a901) quincy (stable)": 1, 2024-03-29T08:22:19.781 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 3 2024-03-29T08:22:19.781 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:22:19.781 INFO:teuthology.orchestra.run.smithi037.stdout: "overall": { 2024-03-29T08:22:19.781 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 17.2.7-789-g4213e17c (4213e17c4de67016ed14f55d22503f4923b3a901) quincy (stable)": 1, 2024-03-29T08:22:19.781 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 13 2024-03-29T08:22:19.781 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-29T08:22:19.781 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-29T08:22:19.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:19.781+0000 7f48d6400700 1 -- 172.21.15.37:0/4073841221 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f48b406cc30 msgr2=0x7f48b406f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:19.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:19.781+0000 7f48d6400700 1 --2- 172.21.15.37:0/4073841221 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f48b406cc30 0x7f48b406f0e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f48d01115d0 tx=0x7f48bc009e30 comp rx=0 tx=0).stop 2024-03-29T08:22:19.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:19.781+0000 7f48d6400700 1 -- 172.21.15.37:0/4073841221 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f48d00fde90 msgr2=0x7f48d0115d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:19.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:19.781+0000 7f48d6400700 1 --2- 172.21.15.37:0/4073841221 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f48d00fde90 0x7f48d0115d30 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f48c0002a50 tx=0x7f48c00046b0 comp rx=0 tx=0).stop 2024-03-29T08:22:19.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:19.781+0000 7f48d6400700 1 -- 172.21.15.37:0/4073841221 shutdown_connections 2024-03-29T08:22:19.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:19.781+0000 7f48d6400700 1 --2- 172.21.15.37:0/4073841221 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f48b406cc30 0x7f48b406f0e0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:19.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:19.781+0000 7f48d6400700 1 --2- 172.21.15.37:0/4073841221 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f48d00fde90 0x7f48d0115d30 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:19.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:19.781+0000 7f48d6400700 1 --2- 172.21.15.37:0/4073841221 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f48d00fe830 0x7f48d0116270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:19.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:19.781+0000 7f48d6400700 1 -- 172.21.15.37:0/4073841221 >> 172.21.15.37:0/4073841221 conn(0x7f48d0072bc0 msgr2=0x7f48d00fb840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:22:19.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:19.781+0000 7f48d6400700 1 -- 172.21.15.37:0/4073841221 shutdown_connections 2024-03-29T08:22:19.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:19.781+0000 7f48d6400700 1 -- 172.21.15.37:0/4073841221 wait complete. 2024-03-29T08:22:19.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:19.918+0000 7f408c000700 1 -- 172.21.15.37:0/2462181588 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4084102e00 msgr2=0x7f40841031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:19.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:19.918+0000 7f408c000700 1 --2- 172.21.15.37:0/2462181588 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4084102e00 0x7f40841031d0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f4078004940 tx=0x7f4078009e30 comp rx=0 tx=0).stop 2024-03-29T08:22:19.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:19.918+0000 7f408c000700 1 -- 172.21.15.37:0/2462181588 shutdown_connections 2024-03-29T08:22:19.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:19.918+0000 7f408c000700 1 --2- 172.21.15.37:0/2462181588 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f40841037a0 0x7f40840fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:19.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:19.918+0000 7f408c000700 1 --2- 172.21.15.37:0/2462181588 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4084102e00 0x7f40841031d0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:19.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:19.918+0000 7f408c000700 1 -- 172.21.15.37:0/2462181588 >> 172.21.15.37:0/2462181588 conn(0x7f40840f89b0 msgr2=0x7f40840fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:22:19.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:19.918+0000 7f408c000700 1 -- 172.21.15.37:0/2462181588 shutdown_connections 2024-03-29T08:22:19.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:19.918+0000 7f408c000700 1 -- 172.21.15.37:0/2462181588 wait complete. 2024-03-29T08:22:19.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:19.919+0000 7f408c000700 1 Processor -- start 2024-03-29T08:22:19.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:19.919+0000 7f408c000700 1 -- start start 2024-03-29T08:22:19.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:19.920+0000 7f408c000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4084102e00 0x7f4084113a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:19.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:19.920+0000 7f408c000700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f40841037a0 0x7f4084113f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:19.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:19.920+0000 7f408c000700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40841145d0 con 0x7f4084102e00 2024-03-29T08:22:19.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:19.920+0000 7f408c000700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f408410d9c0 con 0x7f40841037a0 2024-03-29T08:22:19.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:19.920+0000 7f4089600700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4084102e00 0x7f4084113a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:19.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:19.920+0000 7f4089600700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4084102e00 0x7f4084113a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:33544/0 (socket says 172.21.15.37:33544) 2024-03-29T08:22:19.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:19.920+0000 7f4089600700 1 -- 172.21.15.37:0/3944074033 learned_addr learned my addr 172.21.15.37:0/3944074033 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:22:19.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:19.920+0000 7f4088c00700 1 --2- 172.21.15.37:0/3944074033 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f40841037a0 0x7f4084113f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:19.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:19.921+0000 7f4089600700 1 -- 172.21.15.37:0/3944074033 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f40841037a0 msgr2=0x7f4084113f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:19.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:19.921+0000 7f4089600700 1 --2- 172.21.15.37:0/3944074033 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f40841037a0 0x7f4084113f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:19.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:19.921+0000 7f4089600700 1 -- 172.21.15.37:0/3944074033 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4078005040 con 0x7f4084102e00 2024-03-29T08:22:19.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:19.921+0000 7f4089600700 1 --2- 172.21.15.37:0/3944074033 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4084102e00 0x7f4084113a40 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f4078009390 tx=0x7f4078009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:19.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:19.921+0000 7f4081600700 1 -- 172.21.15.37:0/3944074033 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f407800e040 con 0x7f4084102e00 2024-03-29T08:22:19.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:19.921+0000 7f4081600700 1 -- 172.21.15.37:0/3944074033 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f4078013420 con 0x7f4084102e00 2024-03-29T08:22:19.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:19.921+0000 7f408c000700 1 -- 172.21.15.37:0/3944074033 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f408410dc40 con 0x7f4084102e00 2024-03-29T08:22:19.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:19.921+0000 7f4081600700 1 -- 172.21.15.37:0/3944074033 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f407801c630 con 0x7f4084102e00 2024-03-29T08:22:19.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:19.921+0000 7f408c000700 1 -- 172.21.15.37:0/3944074033 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f408410e200 con 0x7f4084102e00 2024-03-29T08:22:19.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:19.922+0000 7f408c000700 1 -- 172.21.15.37:0/3944074033 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40840fe550 con 0x7f4084102e00 2024-03-29T08:22:19.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:19.923+0000 7f4081600700 1 -- 172.21.15.37:0/3944074033 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f40780139e0 con 0x7f4084102e00 2024-03-29T08:22:19.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:19.924+0000 7f4081600700 1 --2- 172.21.15.37:0/3944074033 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f406c06cc30 0x7f406c06f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:19.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:19.924+0000 7f4081600700 1 -- 172.21.15.37:0/3944074033 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f407808d250 con 0x7f4084102e00 2024-03-29T08:22:19.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:19.924+0000 7f4088c00700 1 --2- 172.21.15.37:0/3944074033 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f406c06cc30 0x7f406c06f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:19.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:19.925+0000 7f4088c00700 1 --2- 172.21.15.37:0/3944074033 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f406c06cc30 0x7f406c06f0e0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f4070009fd0 tx=0x7f4070009f60 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:19.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:19.929+0000 7f4081600700 1 -- 172.21.15.37:0/3944074033 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f407805add0 con 0x7f4084102e00 2024-03-29T08:22:20.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:20.184+0000 7f408c000700 1 -- 172.21.15.37:0/3944074033 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f4084109e50 con 0x7f4084102e00 2024-03-29T08:22:20.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:20.184+0000 7f4081600700 1 -- 172.21.15.37:0/3944074033 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 28 v28) v1 ==== 76+0+1841 (secure 0 0 0) 0x7f407805a960 con 0x7f4084102e00 2024-03-29T08:22:20.186 INFO:teuthology.orchestra.run.smithi037.stdout:e28 2024-03-29T08:22:20.186 INFO:teuthology.orchestra.run.smithi037.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-29T08:22:20.186 INFO:teuthology.orchestra.run.smithi037.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T08:22:20.186 INFO:teuthology.orchestra.run.smithi037.stdout:legacy client fscid: 1 2024-03-29T08:22:20.186 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:22:20.186 INFO:teuthology.orchestra.run.smithi037.stdout:Filesystem 'cephfs' (1) 2024-03-29T08:22:20.186 INFO:teuthology.orchestra.run.smithi037.stdout:fs_name cephfs 2024-03-29T08:22:20.186 INFO:teuthology.orchestra.run.smithi037.stdout:epoch 21 2024-03-29T08:22:20.187 INFO:teuthology.orchestra.run.smithi037.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-03-29T08:22:20.187 INFO:teuthology.orchestra.run.smithi037.stdout:created 2024-03-29T08:03:29.241269+0000 2024-03-29T08:22:20.187 INFO:teuthology.orchestra.run.smithi037.stdout:modified 2024-03-29T08:17:16.237238+0000 2024-03-29T08:22:20.187 INFO:teuthology.orchestra.run.smithi037.stdout:tableserver 0 2024-03-29T08:22:20.187 INFO:teuthology.orchestra.run.smithi037.stdout:root 0 2024-03-29T08:22:20.187 INFO:teuthology.orchestra.run.smithi037.stdout:session_timeout 60 2024-03-29T08:22:20.187 INFO:teuthology.orchestra.run.smithi037.stdout:session_autoclose 300 2024-03-29T08:22:20.187 INFO:teuthology.orchestra.run.smithi037.stdout:max_file_size 1099511627776 2024-03-29T08:22:20.187 INFO:teuthology.orchestra.run.smithi037.stdout:max_xattr_size 65536 2024-03-29T08:22:20.187 INFO:teuthology.orchestra.run.smithi037.stdout:required_client_features {} 2024-03-29T08:22:20.187 INFO:teuthology.orchestra.run.smithi037.stdout:last_failure 0 2024-03-29T08:22:20.188 INFO:teuthology.orchestra.run.smithi037.stdout:last_failure_osd_epoch 82 2024-03-29T08:22:20.188 INFO:teuthology.orchestra.run.smithi037.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T08:22:20.188 INFO:teuthology.orchestra.run.smithi037.stdout:max_mds 2 2024-03-29T08:22:20.188 INFO:teuthology.orchestra.run.smithi037.stdout:in 0,1 2024-03-29T08:22:20.188 INFO:teuthology.orchestra.run.smithi037.stdout:up {} 2024-03-29T08:22:20.188 INFO:teuthology.orchestra.run.smithi037.stdout:failed 0,1 2024-03-29T08:22:20.188 INFO:teuthology.orchestra.run.smithi037.stdout:damaged 2024-03-29T08:22:20.189 INFO:teuthology.orchestra.run.smithi037.stdout:stopped 2024-03-29T08:22:20.189 INFO:teuthology.orchestra.run.smithi037.stdout:data_pools [3] 2024-03-29T08:22:20.189 INFO:teuthology.orchestra.run.smithi037.stdout:metadata_pool 2 2024-03-29T08:22:20.189 INFO:teuthology.orchestra.run.smithi037.stdout:inline_data disabled 2024-03-29T08:22:20.189 INFO:teuthology.orchestra.run.smithi037.stdout:balancer 2024-03-29T08:22:20.189 INFO:teuthology.orchestra.run.smithi037.stdout:bal_rank_mask -1 2024-03-29T08:22:20.189 INFO:teuthology.orchestra.run.smithi037.stdout:standby_count_wanted 1 2024-03-29T08:22:20.189 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:22:20.189 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:22:20.189 INFO:teuthology.orchestra.run.smithi037.stdout:Standby daemons: 2024-03-29T08:22:20.189 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:22:20.189 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi148.qukpjh{-1:24319} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.148:6826/2641327912,v1:172.21.15.148:6827/2641327912] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:22:20.189 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi037.mmwgxe{-1:34440} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.37:6828/1248142005,v1:172.21.15.37:6829/1248142005] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:22:20.189 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi037.iifsmh{-1:34444} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.37:6826/2060882959,v1:172.21.15.37:6827/2060882959] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:22:20.189 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi148.sqteih{-1:44395} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.148:6824/4078652242,v1:172.21.15.148:6825/4078652242] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:22:20.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:20.186+0000 7f408c000700 1 -- 172.21.15.37:0/3944074033 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f406c06cc30 msgr2=0x7f406c06f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:20.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:20.186+0000 7f408c000700 1 --2- 172.21.15.37:0/3944074033 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f406c06cc30 0x7f406c06f0e0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f4070009fd0 tx=0x7f4070009f60 comp rx=0 tx=0).stop 2024-03-29T08:22:20.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:20.187+0000 7f408c000700 1 -- 172.21.15.37:0/3944074033 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4084102e00 msgr2=0x7f4084113a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:20.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:20.187+0000 7f408c000700 1 --2- 172.21.15.37:0/3944074033 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4084102e00 0x7f4084113a40 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f4078009390 tx=0x7f4078009e30 comp rx=0 tx=0).stop 2024-03-29T08:22:20.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:20.187+0000 7f408c000700 1 -- 172.21.15.37:0/3944074033 shutdown_connections 2024-03-29T08:22:20.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:20.187+0000 7f408c000700 1 --2- 172.21.15.37:0/3944074033 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f406c06cc30 0x7f406c06f0e0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:20.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:20.187+0000 7f408c000700 1 --2- 172.21.15.37:0/3944074033 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4084102e00 0x7f4084113a40 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:20.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:20.187+0000 7f408c000700 1 --2- 172.21.15.37:0/3944074033 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f40841037a0 0x7f4084113f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:20.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:20.187+0000 7f408c000700 1 -- 172.21.15.37:0/3944074033 >> 172.21.15.37:0/3944074033 conn(0x7f40840f89b0 msgr2=0x7f40840f99f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:22:20.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:20.187+0000 7f408c000700 1 -- 172.21.15.37:0/3944074033 shutdown_connections 2024-03-29T08:22:20.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:20.187+0000 7f408c000700 1 -- 172.21.15.37:0/3944074033 wait complete. 2024-03-29T08:22:20.191 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 28 2024-03-29T08:22:20.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:20.327+0000 7fd497a00700 1 -- 172.21.15.37:0/47826767 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd490102e00 msgr2=0x7fd4901031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:20.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:20.327+0000 7fd497a00700 1 --2- 172.21.15.37:0/47826767 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd490102e00 0x7fd4901031d0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fd484004940 tx=0x7fd484009e30 comp rx=0 tx=0).stop 2024-03-29T08:22:20.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:20.327+0000 7fd497a00700 1 -- 172.21.15.37:0/47826767 shutdown_connections 2024-03-29T08:22:20.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:20.327+0000 7fd497a00700 1 --2- 172.21.15.37:0/47826767 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd4901037a0 0x7fd4900fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:20.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:20.327+0000 7fd497a00700 1 --2- 172.21.15.37:0/47826767 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd490102e00 0x7fd4901031d0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:20.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:20.327+0000 7fd497a00700 1 -- 172.21.15.37:0/47826767 >> 172.21.15.37:0/47826767 conn(0x7fd4900f89b0 msgr2=0x7fd4900fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:22:20.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:20.327+0000 7fd497a00700 1 -- 172.21.15.37:0/47826767 shutdown_connections 2024-03-29T08:22:20.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:20.327+0000 7fd497a00700 1 -- 172.21.15.37:0/47826767 wait complete. 2024-03-29T08:22:20.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:20.328+0000 7fd497a00700 1 Processor -- start 2024-03-29T08:22:20.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:20.328+0000 7fd497a00700 1 -- start start 2024-03-29T08:22:20.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:20.329+0000 7fd497a00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd490102e00 0x7fd490113b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:20.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:20.329+0000 7fd497a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd4901037a0 0x7fd490114060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:20.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:20.329+0000 7fd497a00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4901146b0 con 0x7fd4901037a0 2024-03-29T08:22:20.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:20.329+0000 7fd497a00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd49010daa0 con 0x7fd490102e00 2024-03-29T08:22:20.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:20.329+0000 7fd48fe00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd4901037a0 0x7fd490114060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:20.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:20.329+0000 7fd48fe00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd4901037a0 0x7fd490114060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43008/0 (socket says 172.21.15.37:43008) 2024-03-29T08:22:20.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:20.329+0000 7fd48fe00700 1 -- 172.21.15.37:0/892784993 learned_addr learned my addr 172.21.15.37:0/892784993 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:22:20.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:20.329+0000 7fd495000700 1 --2- 172.21.15.37:0/892784993 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd490102e00 0x7fd490113b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:20.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:20.329+0000 7fd48fe00700 1 -- 172.21.15.37:0/892784993 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd490102e00 msgr2=0x7fd490113b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:20.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:20.329+0000 7fd48fe00700 1 --2- 172.21.15.37:0/892784993 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd490102e00 0x7fd490113b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:20.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:20.329+0000 7fd48fe00700 1 -- 172.21.15.37:0/892784993 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd484005040 con 0x7fd4901037a0 2024-03-29T08:22:20.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:20.330+0000 7fd48fe00700 1 --2- 172.21.15.37:0/892784993 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd4901037a0 0x7fd490114060 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fd47c002a50 tx=0x7fd47c004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:20.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:20.330+0000 7fd48cc00700 1 -- 172.21.15.37:0/892784993 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd47c007cc0 con 0x7fd4901037a0 2024-03-29T08:22:20.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:20.330+0000 7fd48cc00700 1 -- 172.21.15.37:0/892784993 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd47c00f040 con 0x7fd4901037a0 2024-03-29T08:22:20.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:20.330+0000 7fd48cc00700 1 -- 172.21.15.37:0/892784993 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd47c00d9a0 con 0x7fd4901037a0 2024-03-29T08:22:20.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:20.330+0000 7fd497a00700 1 -- 172.21.15.37:0/892784993 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd49010dd80 con 0x7fd4901037a0 2024-03-29T08:22:20.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:20.330+0000 7fd497a00700 1 -- 172.21.15.37:0/892784993 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd49010e340 con 0x7fd4901037a0 2024-03-29T08:22:20.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:20.331+0000 7fd497a00700 1 -- 172.21.15.37:0/892784993 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4900fe550 con 0x7fd4901037a0 2024-03-29T08:22:20.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:20.332+0000 7fd48cc00700 1 -- 172.21.15.37:0/892784993 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fd47c00dc10 con 0x7fd4901037a0 2024-03-29T08:22:20.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:20.333+0000 7fd48cc00700 1 --2- 172.21.15.37:0/892784993 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fd47806cc30 0x7fd47806f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:20.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:20.333+0000 7fd48cc00700 1 -- 172.21.15.37:0/892784993 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fd47c08be30 con 0x7fd4901037a0 2024-03-29T08:22:20.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:20.334+0000 7fd495000700 1 --2- 172.21.15.37:0/892784993 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fd47806cc30 0x7fd47806f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:20.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:20.334+0000 7fd495000700 1 --2- 172.21.15.37:0/892784993 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fd47806cc30 0x7fd47806f0e0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fd48400b020 tx=0x7fd484009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:20.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:20.337+0000 7fd48cc00700 1 -- 172.21.15.37:0/892784993 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fd47c059880 con 0x7fd4901037a0 2024-03-29T08:22:20.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:22:20 smithi148 ceph-mon[106692]: from='client.34676 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:22:20.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:22:20 smithi148 ceph-mon[106692]: from='client.34680 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:22:20.557 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:22:20 smithi148 ceph-mon[106692]: pgmap v343: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:22:20.557 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:22:20 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/4073841221' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:22:20.557 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:22:20 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/3944074033' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T08:22:20.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:20.573+0000 7fd497a00700 1 -- 172.21.15.37:0/892784993 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd490109dc0 con 0x7fd47806cc30 2024-03-29T08:22:20.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:20.575+0000 7fd48cc00700 1 -- 172.21.15.37:0/892784993 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fd490109dc0 con 0x7fd47806cc30 2024-03-29T08:22:20.576 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-29T08:22:20.576 INFO:teuthology.orchestra.run.smithi037.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59", 2024-03-29T08:22:20.576 INFO:teuthology.orchestra.run.smithi037.stdout: "in_progress": true, 2024-03-29T08:22:20.576 INFO:teuthology.orchestra.run.smithi037.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-29T08:22:20.576 INFO:teuthology.orchestra.run.smithi037.stdout: "services_complete": [ 2024-03-29T08:22:20.576 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr", 2024-03-29T08:22:20.576 INFO:teuthology.orchestra.run.smithi037.stdout: "mon", 2024-03-29T08:22:20.576 INFO:teuthology.orchestra.run.smithi037.stdout: "crash", 2024-03-29T08:22:20.576 INFO:teuthology.orchestra.run.smithi037.stdout: "osd" 2024-03-29T08:22:20.576 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-29T08:22:20.577 INFO:teuthology.orchestra.run.smithi037.stdout: "progress": "15/21 daemons upgraded", 2024-03-29T08:22:20.577 INFO:teuthology.orchestra.run.smithi037.stdout: "message": "Currently upgrading mds daemons", 2024-03-29T08:22:20.577 INFO:teuthology.orchestra.run.smithi037.stdout: "is_paused": false 2024-03-29T08:22:20.577 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-29T08:22:20.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:20.577+0000 7fd497a00700 1 -- 172.21.15.37:0/892784993 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fd47806cc30 msgr2=0x7fd47806f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:20.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:20.577+0000 7fd497a00700 1 --2- 172.21.15.37:0/892784993 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fd47806cc30 0x7fd47806f0e0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fd48400b020 tx=0x7fd484009e30 comp rx=0 tx=0).stop 2024-03-29T08:22:20.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:20.577+0000 7fd497a00700 1 -- 172.21.15.37:0/892784993 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd4901037a0 msgr2=0x7fd490114060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:20.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:20.577+0000 7fd497a00700 1 --2- 172.21.15.37:0/892784993 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd4901037a0 0x7fd490114060 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fd47c002a50 tx=0x7fd47c004600 comp rx=0 tx=0).stop 2024-03-29T08:22:20.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:20.577+0000 7fd497a00700 1 -- 172.21.15.37:0/892784993 shutdown_connections 2024-03-29T08:22:20.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:20.577+0000 7fd497a00700 1 --2- 172.21.15.37:0/892784993 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fd47806cc30 0x7fd47806f0e0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:20.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:20.577+0000 7fd497a00700 1 --2- 172.21.15.37:0/892784993 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd490102e00 0x7fd490113b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:20.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:20.577+0000 7fd497a00700 1 --2- 172.21.15.37:0/892784993 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd4901037a0 0x7fd490114060 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:20.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:20.577+0000 7fd497a00700 1 -- 172.21.15.37:0/892784993 >> 172.21.15.37:0/892784993 conn(0x7fd4900f89b0 msgr2=0x7fd4900f9a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:22:20.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:20.577+0000 7fd497a00700 1 -- 172.21.15.37:0/892784993 shutdown_connections 2024-03-29T08:22:20.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:20.577+0000 7fd497a00700 1 -- 172.21.15.37:0/892784993 wait complete. 2024-03-29T08:22:20.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:22:20 smithi037 ceph-mon[142325]: from='client.34676 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:22:20.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:22:20 smithi037 ceph-mon[142325]: from='client.34680 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:22:20.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:22:20 smithi037 ceph-mon[142325]: pgmap v343: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:22:20.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:22:20 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/4073841221' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:22:20.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:22:20 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/3944074033' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T08:22:20.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:20.714+0000 7f3719600700 1 -- 172.21.15.37:0/1624765892 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f37141037a0 msgr2=0x7f37140fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:20.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:20.714+0000 7f3719600700 1 --2- 172.21.15.37:0/1624765892 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f37141037a0 0x7f37140fce10 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f3700004940 tx=0x7f3700009e30 comp rx=0 tx=0).stop 2024-03-29T08:22:20.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:20.714+0000 7f3719600700 1 -- 172.21.15.37:0/1624765892 shutdown_connections 2024-03-29T08:22:20.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:20.714+0000 7f3719600700 1 --2- 172.21.15.37:0/1624765892 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f37141037a0 0x7f37140fce10 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:20.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:20.714+0000 7f3719600700 1 --2- 172.21.15.37:0/1624765892 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3714102e00 0x7f37141031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:20.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:20.714+0000 7f3719600700 1 -- 172.21.15.37:0/1624765892 >> 172.21.15.37:0/1624765892 conn(0x7f37140f89b0 msgr2=0x7f37140fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:22:20.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:20.714+0000 7f3719600700 1 -- 172.21.15.37:0/1624765892 shutdown_connections 2024-03-29T08:22:20.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:20.714+0000 7f3719600700 1 -- 172.21.15.37:0/1624765892 wait complete. 2024-03-29T08:22:20.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:20.715+0000 7f3719600700 1 Processor -- start 2024-03-29T08:22:20.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:20.715+0000 7f3719600700 1 -- start start 2024-03-29T08:22:20.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:20.715+0000 7f3719600700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3714102e00 0x7f37140702c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:20.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:20.715+0000 7f3719600700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f37141037a0 0x7f3714070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:20.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:20.716+0000 7f3719600700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3714070d90 con 0x7f3714102e00 2024-03-29T08:22:20.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:20.716+0000 7f3719600700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3714070ed0 con 0x7f37141037a0 2024-03-29T08:22:20.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:20.716+0000 7f3712a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3714102e00 0x7f37140702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:20.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:20.716+0000 7f3712a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3714102e00 0x7f37140702c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43014/0 (socket says 172.21.15.37:43014) 2024-03-29T08:22:20.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:20.716+0000 7f3712a00700 1 -- 172.21.15.37:0/57817365 learned_addr learned my addr 172.21.15.37:0/57817365 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:22:20.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:20.716+0000 7f3712000700 1 --2- 172.21.15.37:0/57817365 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f37141037a0 0x7f3714070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:20.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:20.716+0000 7f3712a00700 1 -- 172.21.15.37:0/57817365 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f37141037a0 msgr2=0x7f3714070800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:20.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:20.716+0000 7f3712a00700 1 --2- 172.21.15.37:0/57817365 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f37141037a0 0x7f3714070800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:20.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:20.716+0000 7f3712a00700 1 -- 172.21.15.37:0/57817365 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3700005040 con 0x7f3714102e00 2024-03-29T08:22:20.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:20.716+0000 7f3712a00700 1 --2- 172.21.15.37:0/57817365 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3714102e00 0x7f37140702c0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f3704002a80 tx=0x7f37040046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:20.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:20.717+0000 7f370aa00700 1 -- 172.21.15.37:0/57817365 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3704007cc0 con 0x7f3714102e00 2024-03-29T08:22:20.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:20.717+0000 7f370aa00700 1 -- 172.21.15.37:0/57817365 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f370400f040 con 0x7f3714102e00 2024-03-29T08:22:20.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:20.717+0000 7f3719600700 1 -- 172.21.15.37:0/57817365 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3714119d70 con 0x7f3714102e00 2024-03-29T08:22:20.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:20.717+0000 7f370aa00700 1 -- 172.21.15.37:0/57817365 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3704009640 con 0x7f3714102e00 2024-03-29T08:22:20.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:20.717+0000 7f3719600700 1 -- 172.21.15.37:0/57817365 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f371411a1f0 con 0x7f3714102e00 2024-03-29T08:22:20.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:20.718+0000 7f3719600700 1 -- 172.21.15.37:0/57817365 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37140fe550 con 0x7f3714102e00 2024-03-29T08:22:20.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:20.719+0000 7f370aa00700 1 -- 172.21.15.37:0/57817365 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f37040097a0 con 0x7f3714102e00 2024-03-29T08:22:20.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:20.720+0000 7f370aa00700 1 --2- 172.21.15.37:0/57817365 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f36f806cc30 0x7f36f806f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:20.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:20.720+0000 7f370aa00700 1 -- 172.21.15.37:0/57817365 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f370408b8a0 con 0x7f3714102e00 2024-03-29T08:22:20.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:20.720+0000 7f3712000700 1 --2- 172.21.15.37:0/57817365 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f36f806cc30 0x7f36f806f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:20.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:20.721+0000 7f3712000700 1 --2- 172.21.15.37:0/57817365 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f36f806cc30 0x7f36f806f0e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f370000b3b0 tx=0x7f3700009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:20.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:20.725+0000 7f370aa00700 1 -- 172.21.15.37:0/57817365 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f37040557e0 con 0x7f3714102e00 2024-03-29T08:22:21.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:21.021+0000 7f3719600700 1 -- 172.21.15.37:0/57817365 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f371405f5f0 con 0x7f3714102e00 2024-03-29T08:22:21.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:21.022+0000 7f370aa00700 1 -- 172.21.15.37:0/57817365 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7f3704058e00 con 0x7f3714102e00 2024-03-29T08:22:21.023 INFO:teuthology.orchestra.run.smithi037.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-03-29T08:22:21.023 INFO:teuthology.orchestra.run.smithi037.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-03-29T08:22:21.023 INFO:teuthology.orchestra.run.smithi037.stdout: fs cephfs is degraded 2024-03-29T08:22:21.023 INFO:teuthology.orchestra.run.smithi037.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-03-29T08:22:21.024 INFO:teuthology.orchestra.run.smithi037.stdout: fs cephfs is offline because no MDS is active for it. 2024-03-29T08:22:21.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:21.024+0000 7f3719600700 1 -- 172.21.15.37:0/57817365 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f36f806cc30 msgr2=0x7f36f806f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:21.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:21.024+0000 7f3719600700 1 --2- 172.21.15.37:0/57817365 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f36f806cc30 0x7f36f806f0e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f370000b3b0 tx=0x7f3700009e30 comp rx=0 tx=0).stop 2024-03-29T08:22:21.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:21.024+0000 7f3719600700 1 -- 172.21.15.37:0/57817365 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3714102e00 msgr2=0x7f37140702c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:21.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:21.024+0000 7f3719600700 1 --2- 172.21.15.37:0/57817365 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3714102e00 0x7f37140702c0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f3704002a80 tx=0x7f37040046b0 comp rx=0 tx=0).stop 2024-03-29T08:22:21.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:21.024+0000 7f3719600700 1 -- 172.21.15.37:0/57817365 shutdown_connections 2024-03-29T08:22:21.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:21.024+0000 7f3719600700 1 --2- 172.21.15.37:0/57817365 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f36f806cc30 0x7f36f806f0e0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:21.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:21.024+0000 7f3719600700 1 --2- 172.21.15.37:0/57817365 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3714102e00 0x7f37140702c0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:21.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:21.024+0000 7f3719600700 1 --2- 172.21.15.37:0/57817365 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f37141037a0 0x7f3714070800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:21.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:21.024+0000 7f3719600700 1 -- 172.21.15.37:0/57817365 >> 172.21.15.37:0/57817365 conn(0x7f37140f89b0 msgr2=0x7f37140f9a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:22:21.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:21.025+0000 7f3719600700 1 -- 172.21.15.37:0/57817365 shutdown_connections 2024-03-29T08:22:21.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:21.025+0000 7f3719600700 1 -- 172.21.15.37:0/57817365 wait complete. 2024-03-29T08:22:21.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:22:21 smithi148 ceph-mon[106692]: from='client.34692 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:22:21.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:22:21 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/57817365' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T08:22:21.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:22:21 smithi037 ceph-mon[142325]: from='client.34692 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:22:21.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:22:21 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/57817365' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T08:22:22.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:22:22 smithi148 ceph-mon[106692]: pgmap v344: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:22:22.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:22:22 smithi037 ceph-mon[142325]: pgmap v344: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:22:24.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:22:23 smithi148 ceph-mon[106692]: pgmap v345: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:22:24.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:22:23 smithi037 ceph-mon[142325]: pgmap v345: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:22:25.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:22:25 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:22:25.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:22:25 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:22:26.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:22:26 smithi148 ceph-mon[106692]: pgmap v346: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:22:26.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:22:26 smithi037 ceph-mon[142325]: pgmap v346: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:22:28.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:22:27 smithi148 ceph-mon[106692]: pgmap v347: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:22:28.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:22:27 smithi037 ceph-mon[142325]: pgmap v347: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:22:30.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:22:30 smithi148 ceph-mon[106692]: pgmap v348: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:22:30.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:22:30 smithi037 ceph-mon[142325]: pgmap v348: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:22:32.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:22:31 smithi148 ceph-mon[106692]: pgmap v349: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:22:32.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:22:31 smithi037 ceph-mon[142325]: pgmap v349: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:22:34.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:22:33 smithi148 ceph-mon[106692]: pgmap v350: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:22:34.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:22:33 smithi037 ceph-mon[142325]: pgmap v350: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:22:36.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:22:35 smithi148 ceph-mon[106692]: pgmap v351: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:22:36.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:22:35 smithi037 ceph-mon[142325]: pgmap v351: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:22:38.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:22:37 smithi148 ceph-mon[106692]: pgmap v352: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:22:38.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:22:37 smithi037 ceph-mon[142325]: pgmap v352: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:22:40.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:22:40 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:22:40.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:22:40 smithi148 ceph-mon[106692]: pgmap v353: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:22:40.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:22:40 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:22:40.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:22:40 smithi037 ceph-mon[142325]: pgmap v353: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:22:42.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:22:41 smithi148 ceph-mon[106692]: pgmap v354: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:22:42.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:22:41 smithi037 ceph-mon[142325]: pgmap v354: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:22:44.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:22:43 smithi148 ceph-mon[106692]: pgmap v355: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:22:44.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:22:43 smithi037 ceph-mon[142325]: pgmap v355: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:22:46.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:22:45 smithi148 ceph-mon[106692]: pgmap v356: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:22:46.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:22:45 smithi037 ceph-mon[142325]: pgmap v356: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:22:48.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:22:47 smithi148 ceph-mon[106692]: pgmap v357: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:22:48.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:22:47 smithi037 ceph-mon[142325]: pgmap v357: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:22:50.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:22:50 smithi148 ceph-mon[106692]: pgmap v358: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:22:50.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:22:50 smithi037 ceph-mon[142325]: pgmap v358: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:22:51.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:51.180+0000 7fa5c5a00700 1 -- 172.21.15.37:0/2662565108 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa5c0102e00 msgr2=0x7fa5c01031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:51.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:51.180+0000 7fa5c5a00700 1 --2- 172.21.15.37:0/2662565108 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa5c0102e00 0x7fa5c01031d0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fa5b0004940 tx=0x7fa5b0009e30 comp rx=0 tx=0).stop 2024-03-29T08:22:51.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:51.180+0000 7fa5c5a00700 1 -- 172.21.15.37:0/2662565108 shutdown_connections 2024-03-29T08:22:51.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:51.180+0000 7fa5c5a00700 1 --2- 172.21.15.37:0/2662565108 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa5c01037a0 0x7fa5c00fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:51.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:51.180+0000 7fa5c5a00700 1 --2- 172.21.15.37:0/2662565108 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa5c0102e00 0x7fa5c01031d0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:51.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:51.180+0000 7fa5c5a00700 1 -- 172.21.15.37:0/2662565108 >> 172.21.15.37:0/2662565108 conn(0x7fa5c00f89b0 msgr2=0x7fa5c00fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:22:51.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:51.180+0000 7fa5c5a00700 1 -- 172.21.15.37:0/2662565108 shutdown_connections 2024-03-29T08:22:51.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:51.180+0000 7fa5c5a00700 1 -- 172.21.15.37:0/2662565108 wait complete. 2024-03-29T08:22:51.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:51.181+0000 7fa5c5a00700 1 Processor -- start 2024-03-29T08:22:51.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:51.181+0000 7fa5c5a00700 1 -- start start 2024-03-29T08:22:51.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:51.181+0000 7fa5c5a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa5c0102e00 0x7fa5c0115c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:51.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:51.181+0000 7fa5c5a00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa5c01037a0 0x7fa5c0116190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:51.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:51.181+0000 7fa5c5a00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5c01167e0 con 0x7fa5c0102e00 2024-03-29T08:22:51.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:51.181+0000 7fa5c5a00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5c010fbb0 con 0x7fa5c01037a0 2024-03-29T08:22:51.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:51.182+0000 7fa5bea00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa5c0102e00 0x7fa5c0115c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:51.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:51.182+0000 7fa5bea00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa5c0102e00 0x7fa5c0115c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:48714/0 (socket says 172.21.15.37:48714) 2024-03-29T08:22:51.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:51.182+0000 7fa5bea00700 1 -- 172.21.15.37:0/1540001187 learned_addr learned my addr 172.21.15.37:0/1540001187 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:22:51.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:51.182+0000 7fa5be000700 1 --2- 172.21.15.37:0/1540001187 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa5c01037a0 0x7fa5c0116190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:51.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:51.182+0000 7fa5bea00700 1 -- 172.21.15.37:0/1540001187 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa5c01037a0 msgr2=0x7fa5c0116190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:51.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:51.182+0000 7fa5bea00700 1 --2- 172.21.15.37:0/1540001187 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa5c01037a0 0x7fa5c0116190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:51.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:51.182+0000 7fa5bea00700 1 -- 172.21.15.37:0/1540001187 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5b0005040 con 0x7fa5c0102e00 2024-03-29T08:22:51.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:51.182+0000 7fa5bea00700 1 --2- 172.21.15.37:0/1540001187 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa5c0102e00 0x7fa5c0115c50 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fa5b0007f50 tx=0x7fa5b0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:51.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:51.183+0000 7fa5b6a00700 1 -- 172.21.15.37:0/1540001187 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa5b0021070 con 0x7fa5c0102e00 2024-03-29T08:22:51.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:51.183+0000 7fa5c5a00700 1 -- 172.21.15.37:0/1540001187 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5c010fe30 con 0x7fa5c0102e00 2024-03-29T08:22:51.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:51.183+0000 7fa5b6a00700 1 -- 172.21.15.37:0/1540001187 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fa5b00093b0 con 0x7fa5c0102e00 2024-03-29T08:22:51.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:51.183+0000 7fa5b6a00700 1 -- 172.21.15.37:0/1540001187 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa5b000e070 con 0x7fa5c0102e00 2024-03-29T08:22:51.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:51.183+0000 7fa5c5a00700 1 -- 172.21.15.37:0/1540001187 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5c0110390 con 0x7fa5c0102e00 2024-03-29T08:22:51.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:51.184+0000 7fa5c5a00700 1 -- 172.21.15.37:0/1540001187 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5c00fe550 con 0x7fa5c0102e00 2024-03-29T08:22:51.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:51.185+0000 7fa5b6a00700 1 -- 172.21.15.37:0/1540001187 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fa5b001c400 con 0x7fa5c0102e00 2024-03-29T08:22:51.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:51.186+0000 7fa5b6a00700 1 --2- 172.21.15.37:0/1540001187 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fa5a406cc30 0x7fa5a406f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:51.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:51.186+0000 7fa5b6a00700 1 -- 172.21.15.37:0/1540001187 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fa5b008d120 con 0x7fa5c0102e00 2024-03-29T08:22:51.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:51.187+0000 7fa5be000700 1 --2- 172.21.15.37:0/1540001187 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fa5a406cc30 0x7fa5a406f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:51.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:51.187+0000 7fa5be000700 1 --2- 172.21.15.37:0/1540001187 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fa5a406cc30 0x7fa5a406f0e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fa5c01114f0 tx=0x7fa5ac00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:51.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:51.191+0000 7fa5b6a00700 1 -- 172.21.15.37:0/1540001187 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fa5b005abf0 con 0x7fa5c0102e00 2024-03-29T08:22:51.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:51.424+0000 7fa5c5a00700 1 -- 172.21.15.37:0/1540001187 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa5c004bf70 con 0x7fa5a406cc30 2024-03-29T08:22:51.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:51.425+0000 7fa5b6a00700 1 -- 172.21.15.37:0/1540001187 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fa5ac00f040 con 0x7fa5a406cc30 2024-03-29T08:22:51.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:51.428+0000 7fa5c5a00700 1 -- 172.21.15.37:0/1540001187 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fa5a406cc30 msgr2=0x7fa5a406f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:51.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:51.428+0000 7fa5c5a00700 1 --2- 172.21.15.37:0/1540001187 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fa5a406cc30 0x7fa5a406f0e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fa5c01114f0 tx=0x7fa5ac00b040 comp rx=0 tx=0).stop 2024-03-29T08:22:51.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:51.428+0000 7fa5c5a00700 1 -- 172.21.15.37:0/1540001187 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa5c0102e00 msgr2=0x7fa5c0115c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:51.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:51.428+0000 7fa5c5a00700 1 --2- 172.21.15.37:0/1540001187 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa5c0102e00 0x7fa5c0115c50 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fa5b0007f50 tx=0x7fa5b0009e30 comp rx=0 tx=0).stop 2024-03-29T08:22:51.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:51.428+0000 7fa5c5a00700 1 -- 172.21.15.37:0/1540001187 shutdown_connections 2024-03-29T08:22:51.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:51.428+0000 7fa5c5a00700 1 --2- 172.21.15.37:0/1540001187 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fa5a406cc30 0x7fa5a406f0e0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:51.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:51.428+0000 7fa5c5a00700 1 --2- 172.21.15.37:0/1540001187 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa5c0102e00 0x7fa5c0115c50 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:51.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:51.428+0000 7fa5c5a00700 1 --2- 172.21.15.37:0/1540001187 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa5c01037a0 0x7fa5c0116190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:51.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:51.428+0000 7fa5c5a00700 1 -- 172.21.15.37:0/1540001187 >> 172.21.15.37:0/1540001187 conn(0x7fa5c00f89b0 msgr2=0x7fa5c00f9900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:22:51.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:51.428+0000 7fa5c5a00700 1 -- 172.21.15.37:0/1540001187 shutdown_connections 2024-03-29T08:22:51.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:51.428+0000 7fa5c5a00700 1 -- 172.21.15.37:0/1540001187 wait complete. 2024-03-29T08:22:51.442 INFO:teuthology.orchestra.run.smithi037.stdout:true 2024-03-29T08:22:51.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:51.566+0000 7f11d6600700 1 -- 172.21.15.37:0/3889114924 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f11d0102e00 msgr2=0x7f11d01031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:51.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:51.566+0000 7f11d6600700 1 --2- 172.21.15.37:0/3889114924 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f11d0102e00 0x7f11d01031d0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f11c0004940 tx=0x7f11c0009e30 comp rx=0 tx=0).stop 2024-03-29T08:22:51.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:51.566+0000 7f11d6600700 1 -- 172.21.15.37:0/3889114924 shutdown_connections 2024-03-29T08:22:51.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:51.566+0000 7f11d6600700 1 --2- 172.21.15.37:0/3889114924 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f11d01037a0 0x7f11d00fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:51.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:51.566+0000 7f11d6600700 1 --2- 172.21.15.37:0/3889114924 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f11d0102e00 0x7f11d01031d0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:51.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:51.566+0000 7f11d6600700 1 -- 172.21.15.37:0/3889114924 >> 172.21.15.37:0/3889114924 conn(0x7f11d00f89b0 msgr2=0x7f11d00fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:22:51.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:51.566+0000 7f11d6600700 1 -- 172.21.15.37:0/3889114924 shutdown_connections 2024-03-29T08:22:51.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:51.566+0000 7f11d6600700 1 -- 172.21.15.37:0/3889114924 wait complete. 2024-03-29T08:22:51.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:51.567+0000 7f11d6600700 1 Processor -- start 2024-03-29T08:22:51.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:51.567+0000 7f11d6600700 1 -- start start 2024-03-29T08:22:51.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:51.567+0000 7f11d6600700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f11d0102e00 0x7f11d0113b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:51.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:51.567+0000 7f11d6600700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f11d01037a0 0x7f11d0114060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:51.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:51.567+0000 7f11d6600700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11d01146b0 con 0x7f11d01037a0 2024-03-29T08:22:51.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:51.567+0000 7f11d6600700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11d010daa0 con 0x7f11d0102e00 2024-03-29T08:22:51.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:51.568+0000 7f11cea00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f11d01037a0 0x7f11d0114060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:51.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:51.568+0000 7f11cea00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f11d01037a0 0x7f11d0114060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:48734/0 (socket says 172.21.15.37:48734) 2024-03-29T08:22:51.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:51.568+0000 7f11cea00700 1 -- 172.21.15.37:0/3212892380 learned_addr learned my addr 172.21.15.37:0/3212892380 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:22:51.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:51.568+0000 7f11cf400700 1 --2- 172.21.15.37:0/3212892380 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f11d0102e00 0x7f11d0113b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:51.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:51.568+0000 7f11cea00700 1 -- 172.21.15.37:0/3212892380 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f11d0102e00 msgr2=0x7f11d0113b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:51.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:51.568+0000 7f11cea00700 1 --2- 172.21.15.37:0/3212892380 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f11d0102e00 0x7f11d0113b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:51.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:51.568+0000 7f11cea00700 1 -- 172.21.15.37:0/3212892380 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11c0005040 con 0x7f11d01037a0 2024-03-29T08:22:51.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:51.568+0000 7f11cf400700 1 --2- 172.21.15.37:0/3212892380 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f11d0102e00 0x7f11d0113b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:22:51.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:51.568+0000 7f11cea00700 1 --2- 172.21.15.37:0/3212892380 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f11d01037a0 0x7f11d0114060 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f11bc002a50 tx=0x7f11bc004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:51.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:51.568+0000 7f11c7400700 1 -- 172.21.15.37:0/3212892380 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f11bc007cc0 con 0x7f11d01037a0 2024-03-29T08:22:51.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:51.568+0000 7f11c7400700 1 -- 172.21.15.37:0/3212892380 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f11bc00f040 con 0x7f11d01037a0 2024-03-29T08:22:51.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:51.569+0000 7f11d6600700 1 -- 172.21.15.37:0/3212892380 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11d010dd80 con 0x7f11d01037a0 2024-03-29T08:22:51.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:51.569+0000 7f11c7400700 1 -- 172.21.15.37:0/3212892380 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f11bc00d9a0 con 0x7f11d01037a0 2024-03-29T08:22:51.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:51.569+0000 7f11d6600700 1 -- 172.21.15.37:0/3212892380 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11d010e340 con 0x7f11d01037a0 2024-03-29T08:22:51.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:51.570+0000 7f11d6600700 1 -- 172.21.15.37:0/3212892380 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11d00fe550 con 0x7f11d01037a0 2024-03-29T08:22:51.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:51.571+0000 7f11c7400700 1 -- 172.21.15.37:0/3212892380 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f11bc00dc10 con 0x7f11d01037a0 2024-03-29T08:22:51.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:51.571+0000 7f11c7400700 1 --2- 172.21.15.37:0/3212892380 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f11b406cc30 0x7f11b406f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:51.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:51.572+0000 7f11c7400700 1 -- 172.21.15.37:0/3212892380 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f11bc08be30 con 0x7f11d01037a0 2024-03-29T08:22:51.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:51.572+0000 7f11cf400700 1 --2- 172.21.15.37:0/3212892380 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f11b406cc30 0x7f11b406f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:51.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:51.572+0000 7f11cf400700 1 --2- 172.21.15.37:0/3212892380 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f11b406cc30 0x7f11b406f0e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f11c000b840 tx=0x7f11c0009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:51.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:51.576+0000 7f11c7400700 1 -- 172.21.15.37:0/3212892380 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f11bc059880 con 0x7f11d01037a0 2024-03-29T08:22:51.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:51.809+0000 7f11d6600700 1 -- 172.21.15.37:0/3212892380 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f11d01137f0 con 0x7f11b406cc30 2024-03-29T08:22:51.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:51.810+0000 7f11c7400700 1 -- 172.21.15.37:0/3212892380 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f11d01137f0 con 0x7f11b406cc30 2024-03-29T08:22:51.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:51.812+0000 7f11d6600700 1 -- 172.21.15.37:0/3212892380 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f11b406cc30 msgr2=0x7f11b406f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:51.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:51.812+0000 7f11d6600700 1 --2- 172.21.15.37:0/3212892380 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f11b406cc30 0x7f11b406f0e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f11c000b840 tx=0x7f11c0009e30 comp rx=0 tx=0).stop 2024-03-29T08:22:51.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:51.812+0000 7f11d6600700 1 -- 172.21.15.37:0/3212892380 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f11d01037a0 msgr2=0x7f11d0114060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:51.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:51.812+0000 7f11d6600700 1 --2- 172.21.15.37:0/3212892380 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f11d01037a0 0x7f11d0114060 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f11bc002a50 tx=0x7f11bc004600 comp rx=0 tx=0).stop 2024-03-29T08:22:51.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:51.812+0000 7f11d6600700 1 -- 172.21.15.37:0/3212892380 shutdown_connections 2024-03-29T08:22:51.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:51.812+0000 7f11d6600700 1 --2- 172.21.15.37:0/3212892380 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f11b406cc30 0x7f11b406f0e0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:51.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:51.812+0000 7f11d6600700 1 --2- 172.21.15.37:0/3212892380 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f11d0102e00 0x7f11d0113b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:51.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:51.812+0000 7f11d6600700 1 --2- 172.21.15.37:0/3212892380 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f11d01037a0 0x7f11d0114060 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:51.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:51.812+0000 7f11d6600700 1 -- 172.21.15.37:0/3212892380 >> 172.21.15.37:0/3212892380 conn(0x7f11d00f89b0 msgr2=0x7f11d00f9a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:22:51.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:51.813+0000 7f11d6600700 1 -- 172.21.15.37:0/3212892380 shutdown_connections 2024-03-29T08:22:51.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:51.813+0000 7f11d6600700 1 -- 172.21.15.37:0/3212892380 wait complete. 2024-03-29T08:22:51.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:51.951+0000 7fd338c00700 1 -- 172.21.15.37:0/617634833 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd334102e00 msgr2=0x7fd3341031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:51.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:51.951+0000 7fd338c00700 1 --2- 172.21.15.37:0/617634833 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd334102e00 0x7fd3341031d0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fd324004940 tx=0x7fd324009e30 comp rx=0 tx=0).stop 2024-03-29T08:22:51.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:51.951+0000 7fd338c00700 1 -- 172.21.15.37:0/617634833 shutdown_connections 2024-03-29T08:22:51.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:51.951+0000 7fd338c00700 1 --2- 172.21.15.37:0/617634833 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd3341037a0 0x7fd3340fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:51.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:51.951+0000 7fd338c00700 1 --2- 172.21.15.37:0/617634833 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd334102e00 0x7fd3341031d0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:51.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:51.951+0000 7fd338c00700 1 -- 172.21.15.37:0/617634833 >> 172.21.15.37:0/617634833 conn(0x7fd3340f89b0 msgr2=0x7fd3340fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:22:51.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:51.951+0000 7fd338c00700 1 -- 172.21.15.37:0/617634833 shutdown_connections 2024-03-29T08:22:51.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:51.951+0000 7fd338c00700 1 -- 172.21.15.37:0/617634833 wait complete. 2024-03-29T08:22:51.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:51.952+0000 7fd338c00700 1 Processor -- start 2024-03-29T08:22:51.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:51.952+0000 7fd338c00700 1 -- start start 2024-03-29T08:22:51.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:51.953+0000 7fd338c00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd334102e00 0x7fd3340702c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:51.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:51.953+0000 7fd338c00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd3341037a0 0x7fd334070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:51.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:51.953+0000 7fd338c00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd334070d90 con 0x7fd334102e00 2024-03-29T08:22:51.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:51.953+0000 7fd338c00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd334070ed0 con 0x7fd3341037a0 2024-03-29T08:22:51.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:51.953+0000 7fd331e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd334102e00 0x7fd3340702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:51.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:51.953+0000 7fd331e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd334102e00 0x7fd3340702c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:48750/0 (socket says 172.21.15.37:48750) 2024-03-29T08:22:51.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:51.953+0000 7fd331e00700 1 -- 172.21.15.37:0/1256479802 learned_addr learned my addr 172.21.15.37:0/1256479802 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:22:51.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:51.953+0000 7fd331e00700 1 -- 172.21.15.37:0/1256479802 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd3341037a0 msgr2=0x7fd334070800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:51.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:51.953+0000 7fd331e00700 1 --2- 172.21.15.37:0/1256479802 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd3341037a0 0x7fd334070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:51.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:51.953+0000 7fd331e00700 1 -- 172.21.15.37:0/1256479802 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd324005040 con 0x7fd334102e00 2024-03-29T08:22:51.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:51.953+0000 7fd331e00700 1 --2- 172.21.15.37:0/1256479802 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd334102e00 0x7fd3340702c0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fd324009360 tx=0x7fd324009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:51.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:51.954+0000 7fd32a000700 1 -- 172.21.15.37:0/1256479802 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd32400e040 con 0x7fd334102e00 2024-03-29T08:22:51.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:51.954+0000 7fd32a000700 1 -- 172.21.15.37:0/1256479802 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd324013420 con 0x7fd334102e00 2024-03-29T08:22:51.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:51.954+0000 7fd32a000700 1 -- 172.21.15.37:0/1256479802 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd32401c5f0 con 0x7fd334102e00 2024-03-29T08:22:51.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:51.954+0000 7fd338c00700 1 -- 172.21.15.37:0/1256479802 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd334071120 con 0x7fd334102e00 2024-03-29T08:22:51.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:51.954+0000 7fd338c00700 1 -- 172.21.15.37:0/1256479802 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd33411a250 con 0x7fd334102e00 2024-03-29T08:22:51.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:51.955+0000 7fd338c00700 1 -- 172.21.15.37:0/1256479802 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3340fe550 con 0x7fd334102e00 2024-03-29T08:22:51.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:51.956+0000 7fd32a000700 1 -- 172.21.15.37:0/1256479802 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fd324013590 con 0x7fd334102e00 2024-03-29T08:22:51.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:51.957+0000 7fd32a000700 1 --2- 172.21.15.37:0/1256479802 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fd31806cc30 0x7fd31806f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:51.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:51.957+0000 7fd32a000700 1 -- 172.21.15.37:0/1256479802 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fd32408d220 con 0x7fd334102e00 2024-03-29T08:22:51.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:51.957+0000 7fd331400700 1 --2- 172.21.15.37:0/1256479802 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fd31806cc30 0x7fd31806f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:51.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:51.958+0000 7fd331400700 1 --2- 172.21.15.37:0/1256479802 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fd31806cc30 0x7fd31806f0e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fd334071c00 tx=0x7fd320009ce0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:51.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:51.961+0000 7fd32a000700 1 -- 172.21.15.37:0/1256479802 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fd324057260 con 0x7fd334102e00 2024-03-29T08:22:52.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:22:51 smithi148 ceph-mon[106692]: from='client.34700 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:22:52.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:22:51 smithi148 ceph-mon[106692]: pgmap v359: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:22:52.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:22:51 smithi037 ceph-mon[142325]: from='client.34700 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:22:52.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:22:51 smithi037 ceph-mon[142325]: pgmap v359: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:22:52.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:52.183+0000 7fd338c00700 1 -- 172.21.15.37:0/1256479802 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fd334109e30 con 0x7fd31806cc30 2024-03-29T08:22:52.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:52.194+0000 7fd32a000700 1 -- 172.21.15.37:0/1256479802 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7fd334109e30 con 0x7fd31806cc30 2024-03-29T08:22:52.194 INFO:teuthology.orchestra.run.smithi037.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-29T08:22:52.194 INFO:teuthology.orchestra.run.smithi037.stdout:alertmanager.smithi037 smithi037 *:9093,9094 running (12m) 5m ago 25m 14.9M - 0.25.0 c8568f914cd2 2f9f9746ed0f 2024-03-29T08:22:52.194 INFO:teuthology.orchestra.run.smithi037.stdout:crash.smithi037 smithi037 running (9m) 5m ago 25m 6647k - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 178d416a3186 2024-03-29T08:22:52.194 INFO:teuthology.orchestra.run.smithi037.stdout:crash.smithi148 smithi148 running (9m) 4m ago 24m 6656k - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 23666989d58f 2024-03-29T08:22:52.194 INFO:teuthology.orchestra.run.smithi037.stdout:grafana.smithi037 smithi037 *:3000 running (12m) 5m ago 25m 71.9M - 9.4.7 954c08fa6188 6b976fe49952 2024-03-29T08:22:52.194 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi037.iifsmh smithi037 running (5m) 5m ago 19m 10.8M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 2d97f562cc5b 2024-03-29T08:22:52.194 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi037.mmwgxe smithi037 running (5m) 5m ago 19m 10.9M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff ec37037f46e8 2024-03-29T08:22:52.194 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi148.qukpjh smithi148 running (19m) 4m ago 19m 15.3M - 17.2.7-789-g4213e17c cf22eb8683be 8e469782a273 2024-03-29T08:22:52.194 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi148.sqteih smithi148 running (4m) 4m ago 19m 10.7M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 44c15c77037a 2024-03-29T08:22:52.194 INFO:teuthology.orchestra.run.smithi037.stdout:mgr.smithi037.wjikii smithi037 *:8443,9283,8765 running (15m) 5m ago 27m 506M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff c5a1417b9417 2024-03-29T08:22:52.194 INFO:teuthology.orchestra.run.smithi037.stdout:mgr.smithi148.onhgvm smithi148 *:8443,9283,8765 running (13m) 4m ago 24m 450M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff fcfdbce062bd 2024-03-29T08:22:52.195 INFO:teuthology.orchestra.run.smithi037.stdout:mon.smithi037 smithi037 running (10m) 5m ago 27m 54.8M 2048M 18.2.2-1015-gd0e392d9 1cc2f45c17ff eea2060783af 2024-03-29T08:22:52.195 INFO:teuthology.orchestra.run.smithi037.stdout:mon.smithi148 smithi148 running (10m) 4m ago 24m 43.7M 2048M 18.2.2-1015-gd0e392d9 1cc2f45c17ff ad4b2c5748be 2024-03-29T08:22:52.195 INFO:teuthology.orchestra.run.smithi037.stdout:node-exporter.smithi037 smithi037 *:9100 running (13m) 5m ago 25m 7625k - 1.5.0 0da6a335fe13 26b8ae9cb878 2024-03-29T08:22:52.195 INFO:teuthology.orchestra.run.smithi037.stdout:node-exporter.smithi148 smithi148 *:9100 running (13m) 4m ago 24m 8166k - 1.5.0 0da6a335fe13 98414e4e64ed 2024-03-29T08:22:52.195 INFO:teuthology.orchestra.run.smithi037.stdout:osd.0 smithi037 running (9m) 5m ago 23m 104M 1619M 18.2.2-1015-gd0e392d9 1cc2f45c17ff c0c584f6245c 2024-03-29T08:22:52.195 INFO:teuthology.orchestra.run.smithi037.stdout:osd.1 smithi037 running (8m) 5m ago 23m 100M 1619M 18.2.2-1015-gd0e392d9 1cc2f45c17ff a4a2ff4cc83d 2024-03-29T08:22:52.195 INFO:teuthology.orchestra.run.smithi037.stdout:osd.2 smithi037 running (7m) 5m ago 22m 82.3M 1619M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 8b1bc71aa79e 2024-03-29T08:22:52.195 INFO:teuthology.orchestra.run.smithi037.stdout:osd.3 smithi148 running (7m) 4m ago 21m 105M 2642M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 1c656ab8ff86 2024-03-29T08:22:52.195 INFO:teuthology.orchestra.run.smithi037.stdout:osd.4 smithi148 running (6m) 4m ago 21m 97.3M 2642M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 966a9de11d22 2024-03-29T08:22:52.195 INFO:teuthology.orchestra.run.smithi037.stdout:osd.5 smithi148 running (5m) 4m ago 20m 85.8M 2642M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 474c4f9212ce 2024-03-29T08:22:52.195 INFO:teuthology.orchestra.run.smithi037.stdout:prometheus.smithi037 smithi037 *:9095 running (13m) 5m ago 25m 54.4M - 2.43.0 a07b618ecd1d 30c0223ec589 2024-03-29T08:22:52.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:52.196+0000 7fd338c00700 1 -- 172.21.15.37:0/1256479802 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fd31806cc30 msgr2=0x7fd31806f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:52.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:52.196+0000 7fd338c00700 1 --2- 172.21.15.37:0/1256479802 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fd31806cc30 0x7fd31806f0e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fd334071c00 tx=0x7fd320009ce0 comp rx=0 tx=0).stop 2024-03-29T08:22:52.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:52.196+0000 7fd338c00700 1 -- 172.21.15.37:0/1256479802 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd334102e00 msgr2=0x7fd3340702c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:52.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:52.196+0000 7fd338c00700 1 --2- 172.21.15.37:0/1256479802 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd334102e00 0x7fd3340702c0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fd324009360 tx=0x7fd324009e30 comp rx=0 tx=0).stop 2024-03-29T08:22:52.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:52.196+0000 7fd338c00700 1 -- 172.21.15.37:0/1256479802 shutdown_connections 2024-03-29T08:22:52.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:52.196+0000 7fd338c00700 1 --2- 172.21.15.37:0/1256479802 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fd31806cc30 0x7fd31806f0e0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:52.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:52.196+0000 7fd338c00700 1 --2- 172.21.15.37:0/1256479802 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd334102e00 0x7fd3340702c0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:52.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:52.196+0000 7fd338c00700 1 --2- 172.21.15.37:0/1256479802 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd3341037a0 0x7fd334070800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:52.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:52.196+0000 7fd338c00700 1 -- 172.21.15.37:0/1256479802 >> 172.21.15.37:0/1256479802 conn(0x7fd3340f89b0 msgr2=0x7fd3340f9a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:22:52.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:52.196+0000 7fd338c00700 1 -- 172.21.15.37:0/1256479802 shutdown_connections 2024-03-29T08:22:52.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:52.196+0000 7fd338c00700 1 -- 172.21.15.37:0/1256479802 wait complete. 2024-03-29T08:22:52.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:52.332+0000 7f07b0600700 1 -- 172.21.15.37:0/3508818769 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f07a80fe830 msgr2=0x7f07a8106f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:52.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:52.332+0000 7f07b0600700 1 --2- 172.21.15.37:0/3508818769 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f07a80fe830 0x7f07a8106f00 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f0794004940 tx=0x7f0794009e30 comp rx=0 tx=0).stop 2024-03-29T08:22:52.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:52.332+0000 7f07b0600700 1 -- 172.21.15.37:0/3508818769 shutdown_connections 2024-03-29T08:22:52.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:52.332+0000 7f07b0600700 1 --2- 172.21.15.37:0/3508818769 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f07a80fe830 0x7f07a8106f00 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:52.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:52.332+0000 7f07b0600700 1 --2- 172.21.15.37:0/3508818769 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f07a80fde90 0x7f07a80fe260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:52.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:52.332+0000 7f07b0600700 1 -- 172.21.15.37:0/3508818769 >> 172.21.15.37:0/3508818769 conn(0x7f07a8072bc0 msgr2=0x7f07a8072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:22:52.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:52.332+0000 7f07b0600700 1 -- 172.21.15.37:0/3508818769 shutdown_connections 2024-03-29T08:22:52.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:52.332+0000 7f07b0600700 1 -- 172.21.15.37:0/3508818769 wait complete. 2024-03-29T08:22:52.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:52.333+0000 7f07b0600700 1 Processor -- start 2024-03-29T08:22:52.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:52.333+0000 7f07b0600700 1 -- start start 2024-03-29T08:22:52.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:52.333+0000 7f07b0600700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f07a80fde90 0x7f07a80fbd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:52.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:52.333+0000 7f07b0600700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f07a80fe830 0x7f07a80fc290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:52.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:52.333+0000 7f07b0600700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07a80fc820 con 0x7f07a80fe830 2024-03-29T08:22:52.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:52.333+0000 7f07b0600700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07a80fc960 con 0x7f07a80fde90 2024-03-29T08:22:52.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:52.334+0000 7f07ad200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f07a80fe830 0x7f07a80fc290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:52.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:52.334+0000 7f07ad200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f07a80fe830 0x7f07a80fc290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:48762/0 (socket says 172.21.15.37:48762) 2024-03-29T08:22:52.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:52.334+0000 7f07ad200700 1 -- 172.21.15.37:0/2748046230 learned_addr learned my addr 172.21.15.37:0/2748046230 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:22:52.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:52.334+0000 7f07adc00700 1 --2- 172.21.15.37:0/2748046230 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f07a80fde90 0x7f07a80fbd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:52.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:52.334+0000 7f07ad200700 1 -- 172.21.15.37:0/2748046230 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f07a80fde90 msgr2=0x7f07a80fbd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:52.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:52.334+0000 7f07ad200700 1 --2- 172.21.15.37:0/2748046230 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f07a80fde90 0x7f07a80fbd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:52.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:52.334+0000 7f07ad200700 1 -- 172.21.15.37:0/2748046230 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0794005040 con 0x7f07a80fe830 2024-03-29T08:22:52.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:52.334+0000 7f07adc00700 1 --2- 172.21.15.37:0/2748046230 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f07a80fde90 0x7f07a80fbd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:22:52.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:52.334+0000 7f07ad200700 1 --2- 172.21.15.37:0/2748046230 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f07a80fe830 0x7f07a80fc290 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f0794000c00 tx=0x7f0794009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:52.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:52.335+0000 7f07a5600700 1 -- 172.21.15.37:0/2748046230 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f079400e040 con 0x7f07a80fe830 2024-03-29T08:22:52.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:52.335+0000 7f07a5600700 1 -- 172.21.15.37:0/2748046230 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f0794013420 con 0x7f07a80fe830 2024-03-29T08:22:52.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:52.335+0000 7f07b0600700 1 -- 172.21.15.37:0/2748046230 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07a80fcbb0 con 0x7f07a80fe830 2024-03-29T08:22:52.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:52.335+0000 7f07a5600700 1 -- 172.21.15.37:0/2748046230 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f079401c630 con 0x7f07a80fe830 2024-03-29T08:22:52.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:52.335+0000 7f07b0600700 1 -- 172.21.15.37:0/2748046230 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07a811a190 con 0x7f07a80fe830 2024-03-29T08:22:52.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:52.336+0000 7f07b0600700 1 -- 172.21.15.37:0/2748046230 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07a8065e30 con 0x7f07a80fe830 2024-03-29T08:22:52.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:52.337+0000 7f07a5600700 1 -- 172.21.15.37:0/2748046230 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f0794013970 con 0x7f07a80fe830 2024-03-29T08:22:52.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:52.338+0000 7f07a5600700 1 --2- 172.21.15.37:0/2748046230 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f079006cc30 0x7f079006f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:52.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:52.338+0000 7f07a5600700 1 -- 172.21.15.37:0/2748046230 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f079408d2e0 con 0x7f07a80fe830 2024-03-29T08:22:52.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:52.338+0000 7f07adc00700 1 --2- 172.21.15.37:0/2748046230 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f079006cc30 0x7f079006f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:52.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:52.339+0000 7f07adc00700 1 --2- 172.21.15.37:0/2748046230 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f079006cc30 0x7f079006f0e0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f079c009fd0 tx=0x7f079c00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:52.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:52.343+0000 7f07a5600700 1 -- 172.21.15.37:0/2748046230 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f07940572a0 con 0x7f07a80fe830 2024-03-29T08:22:52.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:52.643+0000 7f07b0600700 1 -- 172.21.15.37:0/2748046230 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f07a805f5f0 con 0x7f07a80fe830 2024-03-29T08:22:52.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:52.643+0000 7f07a5600700 1 -- 172.21.15.37:0/2748046230 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+847 (secure 0 0 0) 0x7f079405a8c0 con 0x7f07a80fe830 2024-03-29T08:22:52.644 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-29T08:22:52.644 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": { 2024-03-29T08:22:52.644 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 2 2024-03-29T08:22:52.644 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:22:52.644 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": { 2024-03-29T08:22:52.644 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 2 2024-03-29T08:22:52.644 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:22:52.644 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": { 2024-03-29T08:22:52.644 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 6 2024-03-29T08:22:52.644 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:22:52.645 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": { 2024-03-29T08:22:52.645 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 17.2.7-789-g4213e17c (4213e17c4de67016ed14f55d22503f4923b3a901) quincy (stable)": 1, 2024-03-29T08:22:52.645 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 3 2024-03-29T08:22:52.645 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:22:52.645 INFO:teuthology.orchestra.run.smithi037.stdout: "overall": { 2024-03-29T08:22:52.645 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 17.2.7-789-g4213e17c (4213e17c4de67016ed14f55d22503f4923b3a901) quincy (stable)": 1, 2024-03-29T08:22:52.645 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 13 2024-03-29T08:22:52.645 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-29T08:22:52.645 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-29T08:22:52.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:52.646+0000 7f07b0600700 1 -- 172.21.15.37:0/2748046230 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f079006cc30 msgr2=0x7f079006f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:52.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:52.646+0000 7f07b0600700 1 --2- 172.21.15.37:0/2748046230 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f079006cc30 0x7f079006f0e0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f079c009fd0 tx=0x7f079c00b040 comp rx=0 tx=0).stop 2024-03-29T08:22:52.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:52.646+0000 7f07b0600700 1 -- 172.21.15.37:0/2748046230 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f07a80fe830 msgr2=0x7f07a80fc290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:52.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:52.646+0000 7f07b0600700 1 --2- 172.21.15.37:0/2748046230 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f07a80fe830 0x7f07a80fc290 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f0794000c00 tx=0x7f0794009e30 comp rx=0 tx=0).stop 2024-03-29T08:22:52.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:52.646+0000 7f07b0600700 1 -- 172.21.15.37:0/2748046230 shutdown_connections 2024-03-29T08:22:52.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:52.646+0000 7f07b0600700 1 --2- 172.21.15.37:0/2748046230 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f079006cc30 0x7f079006f0e0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:52.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:52.646+0000 7f07b0600700 1 --2- 172.21.15.37:0/2748046230 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f07a80fde90 0x7f07a80fbd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:52.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:52.646+0000 7f07b0600700 1 --2- 172.21.15.37:0/2748046230 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f07a80fe830 0x7f07a80fc290 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:52.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:52.646+0000 7f07b0600700 1 -- 172.21.15.37:0/2748046230 >> 172.21.15.37:0/2748046230 conn(0x7f07a8072bc0 msgr2=0x7f07a8103860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:22:52.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:52.646+0000 7f07b0600700 1 -- 172.21.15.37:0/2748046230 shutdown_connections 2024-03-29T08:22:52.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:52.646+0000 7f07b0600700 1 -- 172.21.15.37:0/2748046230 wait complete. 2024-03-29T08:22:52.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:52.782+0000 7f0d41200700 1 -- 172.21.15.37:0/1899412311 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0d3c1037a0 msgr2=0x7f0d3c0fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:52.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:52.782+0000 7f0d41200700 1 --2- 172.21.15.37:0/1899412311 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0d3c1037a0 0x7f0d3c0fce10 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f0d28004940 tx=0x7f0d28009e30 comp rx=0 tx=0).stop 2024-03-29T08:22:52.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:52.782+0000 7f0d41200700 1 -- 172.21.15.37:0/1899412311 shutdown_connections 2024-03-29T08:22:52.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:52.782+0000 7f0d41200700 1 --2- 172.21.15.37:0/1899412311 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0d3c1037a0 0x7f0d3c0fce10 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:52.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:52.783+0000 7f0d41200700 1 --2- 172.21.15.37:0/1899412311 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0d3c102e00 0x7f0d3c1031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:52.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:52.783+0000 7f0d41200700 1 -- 172.21.15.37:0/1899412311 >> 172.21.15.37:0/1899412311 conn(0x7f0d3c0f89b0 msgr2=0x7f0d3c0fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:22:52.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:52.783+0000 7f0d41200700 1 -- 172.21.15.37:0/1899412311 shutdown_connections 2024-03-29T08:22:52.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:52.783+0000 7f0d41200700 1 -- 172.21.15.37:0/1899412311 wait complete. 2024-03-29T08:22:52.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:52.784+0000 7f0d41200700 1 Processor -- start 2024-03-29T08:22:52.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:52.784+0000 7f0d41200700 1 -- start start 2024-03-29T08:22:52.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:52.785+0000 7f0d41200700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0d3c102e00 0x7f0d3c0702c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:52.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:52.785+0000 7f0d41200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0d3c1037a0 0x7f0d3c070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:52.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:52.785+0000 7f0d41200700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d3c070d90 con 0x7f0d3c1037a0 2024-03-29T08:22:52.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:52.785+0000 7f0d41200700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d3c070ed0 con 0x7f0d3c102e00 2024-03-29T08:22:52.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:52.785+0000 7f0d39400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0d3c1037a0 0x7f0d3c070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:52.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:52.785+0000 7f0d39400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0d3c1037a0 0x7f0d3c070800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:48786/0 (socket says 172.21.15.37:48786) 2024-03-29T08:22:52.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:52.785+0000 7f0d39400700 1 -- 172.21.15.37:0/2077801431 learned_addr learned my addr 172.21.15.37:0/2077801431 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:22:52.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:52.785+0000 7f0d39e00700 1 --2- 172.21.15.37:0/2077801431 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0d3c102e00 0x7f0d3c0702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:52.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:52.785+0000 7f0d39400700 1 -- 172.21.15.37:0/2077801431 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0d3c102e00 msgr2=0x7f0d3c0702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:52.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:52.785+0000 7f0d39400700 1 --2- 172.21.15.37:0/2077801431 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0d3c102e00 0x7f0d3c0702c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:52.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:52.785+0000 7f0d39400700 1 -- 172.21.15.37:0/2077801431 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d28005040 con 0x7f0d3c1037a0 2024-03-29T08:22:52.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:52.786+0000 7f0d39400700 1 --2- 172.21.15.37:0/2077801431 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0d3c1037a0 0x7f0d3c070800 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f0d28009390 tx=0x7f0d28009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:52.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:52.786+0000 7f0d32000700 1 -- 172.21.15.37:0/2077801431 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0d2800e040 con 0x7f0d3c1037a0 2024-03-29T08:22:52.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:52.786+0000 7f0d32000700 1 -- 172.21.15.37:0/2077801431 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f0d28013420 con 0x7f0d3c1037a0 2024-03-29T08:22:52.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:52.786+0000 7f0d41200700 1 -- 172.21.15.37:0/2077801431 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d3c071120 con 0x7f0d3c1037a0 2024-03-29T08:22:52.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:52.786+0000 7f0d32000700 1 -- 172.21.15.37:0/2077801431 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0d2801c630 con 0x7f0d3c1037a0 2024-03-29T08:22:52.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:52.786+0000 7f0d41200700 1 -- 172.21.15.37:0/2077801431 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d3c11a1a0 con 0x7f0d3c1037a0 2024-03-29T08:22:52.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:52.787+0000 7f0d41200700 1 -- 172.21.15.37:0/2077801431 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d3c0fe550 con 0x7f0d3c1037a0 2024-03-29T08:22:52.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:52.788+0000 7f0d32000700 1 -- 172.21.15.37:0/2077801431 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f0d28013a00 con 0x7f0d3c1037a0 2024-03-29T08:22:52.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:52.789+0000 7f0d32000700 1 --2- 172.21.15.37:0/2077801431 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f0d2006cc30 0x7f0d2006f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:52.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:52.789+0000 7f0d32000700 1 -- 172.21.15.37:0/2077801431 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f0d2808d0e0 con 0x7f0d3c1037a0 2024-03-29T08:22:52.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:52.789+0000 7f0d39e00700 1 --2- 172.21.15.37:0/2077801431 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f0d2006cc30 0x7f0d2006f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:52.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:52.790+0000 7f0d39e00700 1 --2- 172.21.15.37:0/2077801431 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f0d2006cc30 0x7f0d2006f0e0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f0d2c009fd0 tx=0x7f0d2c00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:52.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:52.793+0000 7f0d32000700 1 -- 172.21.15.37:0/2077801431 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f0d28057120 con 0x7f0d3c1037a0 2024-03-29T08:22:53.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:53.053+0000 7f0d41200700 1 -- 172.21.15.37:0/2077801431 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f0d3c109dd0 con 0x7f0d3c1037a0 2024-03-29T08:22:53.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:53.053+0000 7f0d32000700 1 -- 172.21.15.37:0/2077801431 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 28 v28) v1 ==== 76+0+1841 (secure 0 0 0) 0x7f0d2805a740 con 0x7f0d3c1037a0 2024-03-29T08:22:53.053 INFO:teuthology.orchestra.run.smithi037.stdout:e28 2024-03-29T08:22:53.053 INFO:teuthology.orchestra.run.smithi037.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-29T08:22:53.054 INFO:teuthology.orchestra.run.smithi037.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T08:22:53.054 INFO:teuthology.orchestra.run.smithi037.stdout:legacy client fscid: 1 2024-03-29T08:22:53.054 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:22:53.054 INFO:teuthology.orchestra.run.smithi037.stdout:Filesystem 'cephfs' (1) 2024-03-29T08:22:53.054 INFO:teuthology.orchestra.run.smithi037.stdout:fs_name cephfs 2024-03-29T08:22:53.054 INFO:teuthology.orchestra.run.smithi037.stdout:epoch 21 2024-03-29T08:22:53.054 INFO:teuthology.orchestra.run.smithi037.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-03-29T08:22:53.054 INFO:teuthology.orchestra.run.smithi037.stdout:created 2024-03-29T08:03:29.241269+0000 2024-03-29T08:22:53.054 INFO:teuthology.orchestra.run.smithi037.stdout:modified 2024-03-29T08:17:16.237238+0000 2024-03-29T08:22:53.054 INFO:teuthology.orchestra.run.smithi037.stdout:tableserver 0 2024-03-29T08:22:53.054 INFO:teuthology.orchestra.run.smithi037.stdout:root 0 2024-03-29T08:22:53.054 INFO:teuthology.orchestra.run.smithi037.stdout:session_timeout 60 2024-03-29T08:22:53.054 INFO:teuthology.orchestra.run.smithi037.stdout:session_autoclose 300 2024-03-29T08:22:53.055 INFO:teuthology.orchestra.run.smithi037.stdout:max_file_size 1099511627776 2024-03-29T08:22:53.055 INFO:teuthology.orchestra.run.smithi037.stdout:max_xattr_size 65536 2024-03-29T08:22:53.055 INFO:teuthology.orchestra.run.smithi037.stdout:required_client_features {} 2024-03-29T08:22:53.055 INFO:teuthology.orchestra.run.smithi037.stdout:last_failure 0 2024-03-29T08:22:53.055 INFO:teuthology.orchestra.run.smithi037.stdout:last_failure_osd_epoch 82 2024-03-29T08:22:53.055 INFO:teuthology.orchestra.run.smithi037.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T08:22:53.055 INFO:teuthology.orchestra.run.smithi037.stdout:max_mds 2 2024-03-29T08:22:53.055 INFO:teuthology.orchestra.run.smithi037.stdout:in 0,1 2024-03-29T08:22:53.055 INFO:teuthology.orchestra.run.smithi037.stdout:up {} 2024-03-29T08:22:53.055 INFO:teuthology.orchestra.run.smithi037.stdout:failed 0,1 2024-03-29T08:22:53.055 INFO:teuthology.orchestra.run.smithi037.stdout:damaged 2024-03-29T08:22:53.055 INFO:teuthology.orchestra.run.smithi037.stdout:stopped 2024-03-29T08:22:53.055 INFO:teuthology.orchestra.run.smithi037.stdout:data_pools [3] 2024-03-29T08:22:53.056 INFO:teuthology.orchestra.run.smithi037.stdout:metadata_pool 2 2024-03-29T08:22:53.056 INFO:teuthology.orchestra.run.smithi037.stdout:inline_data disabled 2024-03-29T08:22:53.056 INFO:teuthology.orchestra.run.smithi037.stdout:balancer 2024-03-29T08:22:53.056 INFO:teuthology.orchestra.run.smithi037.stdout:bal_rank_mask -1 2024-03-29T08:22:53.056 INFO:teuthology.orchestra.run.smithi037.stdout:standby_count_wanted 1 2024-03-29T08:22:53.056 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:22:53.056 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:22:53.056 INFO:teuthology.orchestra.run.smithi037.stdout:Standby daemons: 2024-03-29T08:22:53.056 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:22:53.056 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi148.qukpjh{-1:24319} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.148:6826/2641327912,v1:172.21.15.148:6827/2641327912] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:22:53.056 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi037.mmwgxe{-1:34440} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.37:6828/1248142005,v1:172.21.15.37:6829/1248142005] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:22:53.056 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi037.iifsmh{-1:34444} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.37:6826/2060882959,v1:172.21.15.37:6827/2060882959] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:22:53.056 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi148.sqteih{-1:44395} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.148:6824/4078652242,v1:172.21.15.148:6825/4078652242] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:22:53.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:22:52 smithi148 ceph-mon[106692]: from='client.34704 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:22:53.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:22:52 smithi148 ceph-mon[106692]: from='client.34708 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:22:53.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:22:52 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/2748046230' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:22:53.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:53.055+0000 7f0d41200700 1 -- 172.21.15.37:0/2077801431 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f0d2006cc30 msgr2=0x7f0d2006f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:53.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:53.055+0000 7f0d41200700 1 --2- 172.21.15.37:0/2077801431 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f0d2006cc30 0x7f0d2006f0e0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f0d2c009fd0 tx=0x7f0d2c00b040 comp rx=0 tx=0).stop 2024-03-29T08:22:53.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:53.055+0000 7f0d41200700 1 -- 172.21.15.37:0/2077801431 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0d3c1037a0 msgr2=0x7f0d3c070800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:53.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:53.055+0000 7f0d41200700 1 --2- 172.21.15.37:0/2077801431 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0d3c1037a0 0x7f0d3c070800 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f0d28009390 tx=0x7f0d28009e30 comp rx=0 tx=0).stop 2024-03-29T08:22:53.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:53.056+0000 7f0d41200700 1 -- 172.21.15.37:0/2077801431 shutdown_connections 2024-03-29T08:22:53.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:53.056+0000 7f0d41200700 1 --2- 172.21.15.37:0/2077801431 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f0d2006cc30 0x7f0d2006f0e0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:53.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:53.056+0000 7f0d41200700 1 --2- 172.21.15.37:0/2077801431 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0d3c102e00 0x7f0d3c0702c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:53.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:53.056+0000 7f0d41200700 1 --2- 172.21.15.37:0/2077801431 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0d3c1037a0 0x7f0d3c070800 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:53.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:53.056+0000 7f0d41200700 1 -- 172.21.15.37:0/2077801431 >> 172.21.15.37:0/2077801431 conn(0x7f0d3c0f89b0 msgr2=0x7f0d3c0f9a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:22:53.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:53.056+0000 7f0d41200700 1 -- 172.21.15.37:0/2077801431 shutdown_connections 2024-03-29T08:22:53.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:53.056+0000 7f0d41200700 1 -- 172.21.15.37:0/2077801431 wait complete. 2024-03-29T08:22:53.058 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 28 2024-03-29T08:22:53.149 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:22:52 smithi037 ceph-mon[142325]: from='client.34704 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:22:53.149 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:22:52 smithi037 ceph-mon[142325]: from='client.34708 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:22:53.149 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:22:52 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/2748046230' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:22:53.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:53.194+0000 7f6b21200700 1 -- 172.21.15.37:0/3956329233 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6b1c102e00 msgr2=0x7f6b1c1031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:53.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:53.194+0000 7f6b21200700 1 --2- 172.21.15.37:0/3956329233 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6b1c102e00 0x7f6b1c1031d0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f6b0c004940 tx=0x7f6b0c009e30 comp rx=0 tx=0).stop 2024-03-29T08:22:53.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:53.194+0000 7f6b21200700 1 -- 172.21.15.37:0/3956329233 shutdown_connections 2024-03-29T08:22:53.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:53.194+0000 7f6b21200700 1 --2- 172.21.15.37:0/3956329233 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6b1c1037a0 0x7f6b1c0fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:53.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:53.194+0000 7f6b21200700 1 --2- 172.21.15.37:0/3956329233 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6b1c102e00 0x7f6b1c1031d0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:53.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:53.194+0000 7f6b21200700 1 -- 172.21.15.37:0/3956329233 >> 172.21.15.37:0/3956329233 conn(0x7f6b1c0f89b0 msgr2=0x7f6b1c0fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:22:53.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:53.194+0000 7f6b21200700 1 -- 172.21.15.37:0/3956329233 shutdown_connections 2024-03-29T08:22:53.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:53.194+0000 7f6b21200700 1 -- 172.21.15.37:0/3956329233 wait complete. 2024-03-29T08:22:53.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:53.195+0000 7f6b21200700 1 Processor -- start 2024-03-29T08:22:53.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:53.195+0000 7f6b21200700 1 -- start start 2024-03-29T08:22:53.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:53.195+0000 7f6b21200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6b1c102e00 0x7f6b1c115cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:53.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:53.196+0000 7f6b21200700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6b1c1037a0 0x7f6b1c1161f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:53.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:53.196+0000 7f6b21200700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b1c1168d0 con 0x7f6b1c102e00 2024-03-29T08:22:53.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:53.196+0000 7f6b21200700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b1c10fc10 con 0x7f6b1c1037a0 2024-03-29T08:22:53.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:53.196+0000 7f6b19e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6b1c102e00 0x7f6b1c115cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:53.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:53.196+0000 7f6b19e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6b1c102e00 0x7f6b1c115cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:48810/0 (socket says 172.21.15.37:48810) 2024-03-29T08:22:53.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:53.196+0000 7f6b19e00700 1 -- 172.21.15.37:0/3267088058 learned_addr learned my addr 172.21.15.37:0/3267088058 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:22:53.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:53.196+0000 7f6b19400700 1 --2- 172.21.15.37:0/3267088058 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6b1c1037a0 0x7f6b1c1161f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:53.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:53.196+0000 7f6b19e00700 1 -- 172.21.15.37:0/3267088058 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6b1c1037a0 msgr2=0x7f6b1c1161f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:53.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:53.196+0000 7f6b19e00700 1 --2- 172.21.15.37:0/3267088058 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6b1c1037a0 0x7f6b1c1161f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:53.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:53.196+0000 7f6b19e00700 1 -- 172.21.15.37:0/3267088058 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b0c005040 con 0x7f6b1c102e00 2024-03-29T08:22:53.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:53.196+0000 7f6b19400700 1 --2- 172.21.15.37:0/3267088058 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6b1c1037a0 0x7f6b1c1161f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:22:53.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:53.196+0000 7f6b19e00700 1 --2- 172.21.15.37:0/3267088058 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6b1c102e00 0x7f6b1c115cb0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f6b0c009360 tx=0x7f6b0c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:53.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:53.197+0000 7f6b12000700 1 -- 172.21.15.37:0/3267088058 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6b0c00e040 con 0x7f6b1c102e00 2024-03-29T08:22:53.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:53.197+0000 7f6b12000700 1 -- 172.21.15.37:0/3267088058 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f6b0c013420 con 0x7f6b1c102e00 2024-03-29T08:22:53.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:53.197+0000 7f6b12000700 1 -- 172.21.15.37:0/3267088058 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6b0c01c5f0 con 0x7f6b1c102e00 2024-03-29T08:22:53.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:53.197+0000 7f6b21200700 1 -- 172.21.15.37:0/3267088058 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b1c10fe30 con 0x7f6b1c102e00 2024-03-29T08:22:53.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:53.197+0000 7f6b21200700 1 -- 172.21.15.37:0/3267088058 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b1c1103f0 con 0x7f6b1c102e00 2024-03-29T08:22:53.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:53.198+0000 7f6b21200700 1 -- 172.21.15.37:0/3267088058 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b1c0fe550 con 0x7f6b1c102e00 2024-03-29T08:22:53.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:53.199+0000 7f6b12000700 1 -- 172.21.15.37:0/3267088058 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f6b0c013590 con 0x7f6b1c102e00 2024-03-29T08:22:53.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:53.200+0000 7f6b12000700 1 --2- 172.21.15.37:0/3267088058 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f6b0006cc30 0x7f6b0006f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:53.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:53.200+0000 7f6b12000700 1 -- 172.21.15.37:0/3267088058 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f6b0c08d250 con 0x7f6b1c102e00 2024-03-29T08:22:53.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:53.200+0000 7f6b19400700 1 --2- 172.21.15.37:0/3267088058 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f6b0006cc30 0x7f6b0006f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:53.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:53.201+0000 7f6b19400700 1 --2- 172.21.15.37:0/3267088058 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f6b0006cc30 0x7f6b0006f0e0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f6b08006fd0 tx=0x7f6b08008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:53.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:53.204+0000 7f6b12000700 1 -- 172.21.15.37:0/3267088058 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f6b0c05add0 con 0x7f6b1c102e00 2024-03-29T08:22:53.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:53.436+0000 7f6b21200700 1 -- 172.21.15.37:0/3267088058 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6b1c111300 con 0x7f6b0006cc30 2024-03-29T08:22:53.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:53.437+0000 7f6b12000700 1 -- 172.21.15.37:0/3267088058 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f6b1c111300 con 0x7f6b0006cc30 2024-03-29T08:22:53.437 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-29T08:22:53.438 INFO:teuthology.orchestra.run.smithi037.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59", 2024-03-29T08:22:53.438 INFO:teuthology.orchestra.run.smithi037.stdout: "in_progress": true, 2024-03-29T08:22:53.438 INFO:teuthology.orchestra.run.smithi037.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-29T08:22:53.438 INFO:teuthology.orchestra.run.smithi037.stdout: "services_complete": [ 2024-03-29T08:22:53.438 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr", 2024-03-29T08:22:53.438 INFO:teuthology.orchestra.run.smithi037.stdout: "mon", 2024-03-29T08:22:53.438 INFO:teuthology.orchestra.run.smithi037.stdout: "crash", 2024-03-29T08:22:53.438 INFO:teuthology.orchestra.run.smithi037.stdout: "osd" 2024-03-29T08:22:53.438 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-29T08:22:53.438 INFO:teuthology.orchestra.run.smithi037.stdout: "progress": "15/21 daemons upgraded", 2024-03-29T08:22:53.438 INFO:teuthology.orchestra.run.smithi037.stdout: "message": "Currently upgrading mds daemons", 2024-03-29T08:22:53.438 INFO:teuthology.orchestra.run.smithi037.stdout: "is_paused": false 2024-03-29T08:22:53.438 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-29T08:22:53.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:53.440+0000 7f6b21200700 1 -- 172.21.15.37:0/3267088058 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f6b0006cc30 msgr2=0x7f6b0006f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:53.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:53.440+0000 7f6b21200700 1 --2- 172.21.15.37:0/3267088058 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f6b0006cc30 0x7f6b0006f0e0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f6b08006fd0 tx=0x7f6b08008040 comp rx=0 tx=0).stop 2024-03-29T08:22:53.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:53.440+0000 7f6b21200700 1 -- 172.21.15.37:0/3267088058 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6b1c102e00 msgr2=0x7f6b1c115cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:53.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:53.440+0000 7f6b21200700 1 --2- 172.21.15.37:0/3267088058 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6b1c102e00 0x7f6b1c115cb0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f6b0c009360 tx=0x7f6b0c009e30 comp rx=0 tx=0).stop 2024-03-29T08:22:53.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:53.440+0000 7f6b21200700 1 -- 172.21.15.37:0/3267088058 shutdown_connections 2024-03-29T08:22:53.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:53.440+0000 7f6b21200700 1 --2- 172.21.15.37:0/3267088058 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f6b0006cc30 0x7f6b0006f0e0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:53.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:53.440+0000 7f6b21200700 1 --2- 172.21.15.37:0/3267088058 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6b1c102e00 0x7f6b1c115cb0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:53.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:53.440+0000 7f6b21200700 1 --2- 172.21.15.37:0/3267088058 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6b1c1037a0 0x7f6b1c1161f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:53.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:53.440+0000 7f6b21200700 1 -- 172.21.15.37:0/3267088058 >> 172.21.15.37:0/3267088058 conn(0x7f6b1c0f89b0 msgr2=0x7f6b1c0f9a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:22:53.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:53.440+0000 7f6b21200700 1 -- 172.21.15.37:0/3267088058 shutdown_connections 2024-03-29T08:22:53.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:53.440+0000 7f6b21200700 1 -- 172.21.15.37:0/3267088058 wait complete. 2024-03-29T08:22:53.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:53.577+0000 7f1ffe800700 1 -- 172.21.15.37:0/446715726 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ff81037c0 msgr2=0x7f1ff80fce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:53.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:53.577+0000 7f1ffe800700 1 --2- 172.21.15.37:0/446715726 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ff81037c0 0x7f1ff80fce30 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f1fe4004940 tx=0x7f1fe4009e30 comp rx=0 tx=0).stop 2024-03-29T08:22:53.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:53.577+0000 7f1ffe800700 1 -- 172.21.15.37:0/446715726 shutdown_connections 2024-03-29T08:22:53.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:53.577+0000 7f1ffe800700 1 --2- 172.21.15.37:0/446715726 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ff81037c0 0x7f1ff80fce30 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:53.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:53.577+0000 7f1ffe800700 1 --2- 172.21.15.37:0/446715726 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1ff8102e20 0x7f1ff81031f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:53.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:53.577+0000 7f1ffe800700 1 -- 172.21.15.37:0/446715726 >> 172.21.15.37:0/446715726 conn(0x7f1ff80f89d0 msgr2=0x7f1ff80fade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:22:53.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:53.577+0000 7f1ffe800700 1 -- 172.21.15.37:0/446715726 shutdown_connections 2024-03-29T08:22:53.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:53.578+0000 7f1ffe800700 1 -- 172.21.15.37:0/446715726 wait complete. 2024-03-29T08:22:53.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:53.578+0000 7f1ffe800700 1 Processor -- start 2024-03-29T08:22:53.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:53.578+0000 7f1ffe800700 1 -- start start 2024-03-29T08:22:53.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:53.579+0000 7f1ffe800700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1ff8102e20 0x7f1ff8113a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:53.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:53.579+0000 7f1ffe800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ff81037c0 0x7f1ff8113f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:53.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:53.579+0000 7f1ffe800700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ff81145d0 con 0x7f1ff81037c0 2024-03-29T08:22:53.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:53.579+0000 7f1ffe800700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ff810d9c0 con 0x7f1ff8102e20 2024-03-29T08:22:53.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:53.579+0000 7f1ff6a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ff81037c0 0x7f1ff8113f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:53.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:53.579+0000 7f1ff6a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ff81037c0 0x7f1ff8113f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:48840/0 (socket says 172.21.15.37:48840) 2024-03-29T08:22:53.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:53.579+0000 7f1ff6a00700 1 -- 172.21.15.37:0/1208190488 learned_addr learned my addr 172.21.15.37:0/1208190488 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:22:53.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:53.579+0000 7f1ff7400700 1 --2- 172.21.15.37:0/1208190488 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1ff8102e20 0x7f1ff8113a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:53.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:53.579+0000 7f1ff6a00700 1 -- 172.21.15.37:0/1208190488 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1ff8102e20 msgr2=0x7f1ff8113a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:53.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:53.579+0000 7f1ff6a00700 1 --2- 172.21.15.37:0/1208190488 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1ff8102e20 0x7f1ff8113a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:53.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:53.579+0000 7f1ff6a00700 1 -- 172.21.15.37:0/1208190488 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fe4005040 con 0x7f1ff81037c0 2024-03-29T08:22:53.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:53.580+0000 7f1ff7400700 1 --2- 172.21.15.37:0/1208190488 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1ff8102e20 0x7f1ff8113a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:22:53.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:53.580+0000 7f1ff6a00700 1 --2- 172.21.15.37:0/1208190488 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ff81037c0 0x7f1ff8113f80 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f1fe4009fd0 tx=0x7f1fe4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:53.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:53.580+0000 7f1fef400700 1 -- 172.21.15.37:0/1208190488 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1fe400e030 con 0x7f1ff81037c0 2024-03-29T08:22:53.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:53.580+0000 7f1fef400700 1 -- 172.21.15.37:0/1208190488 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f1fe4013e00 con 0x7f1ff81037c0 2024-03-29T08:22:53.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:53.580+0000 7f1ffe800700 1 -- 172.21.15.37:0/1208190488 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ff810dc40 con 0x7f1ff81037c0 2024-03-29T08:22:53.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:53.580+0000 7f1fef400700 1 -- 172.21.15.37:0/1208190488 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1fe401ccd0 con 0x7f1ff81037c0 2024-03-29T08:22:53.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:53.580+0000 7f1ffe800700 1 -- 172.21.15.37:0/1208190488 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ff810e200 con 0x7f1ff81037c0 2024-03-29T08:22:53.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:53.581+0000 7f1ffe800700 1 -- 172.21.15.37:0/1208190488 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ff80fe570 con 0x7f1ff81037c0 2024-03-29T08:22:53.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:53.582+0000 7f1fef400700 1 -- 172.21.15.37:0/1208190488 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f1fe401c500 con 0x7f1ff81037c0 2024-03-29T08:22:53.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:53.583+0000 7f1fef400700 1 --2- 172.21.15.37:0/1208190488 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f1fdc06cc30 0x7f1fdc06f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:53.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:53.583+0000 7f1fef400700 1 -- 172.21.15.37:0/1208190488 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f1fe408d7e0 con 0x7f1ff81037c0 2024-03-29T08:22:53.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:53.583+0000 7f1ff7400700 1 --2- 172.21.15.37:0/1208190488 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f1fdc06cc30 0x7f1fdc06f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:53.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:53.584+0000 7f1ff7400700 1 --2- 172.21.15.37:0/1208190488 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f1fdc06cc30 0x7f1fdc06f0e0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f1fe8009fd0 tx=0x7f1fe800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:53.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:53.588+0000 7f1fef400700 1 -- 172.21.15.37:0/1208190488 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f1fe405b370 con 0x7f1ff81037c0 2024-03-29T08:22:53.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:53.886+0000 7f1ffe800700 1 -- 172.21.15.37:0/1208190488 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f1ff8109dc0 con 0x7f1ff81037c0 2024-03-29T08:22:53.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:53.887+0000 7f1fef400700 1 -- 172.21.15.37:0/1208190488 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7f1fe405af00 con 0x7f1ff81037c0 2024-03-29T08:22:53.887 INFO:teuthology.orchestra.run.smithi037.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-03-29T08:22:53.887 INFO:teuthology.orchestra.run.smithi037.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-03-29T08:22:53.887 INFO:teuthology.orchestra.run.smithi037.stdout: fs cephfs is degraded 2024-03-29T08:22:53.887 INFO:teuthology.orchestra.run.smithi037.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-03-29T08:22:53.888 INFO:teuthology.orchestra.run.smithi037.stdout: fs cephfs is offline because no MDS is active for it. 2024-03-29T08:22:53.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:53.889+0000 7f1ffe800700 1 -- 172.21.15.37:0/1208190488 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f1fdc06cc30 msgr2=0x7f1fdc06f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:53.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:53.889+0000 7f1ffe800700 1 --2- 172.21.15.37:0/1208190488 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f1fdc06cc30 0x7f1fdc06f0e0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f1fe8009fd0 tx=0x7f1fe800b040 comp rx=0 tx=0).stop 2024-03-29T08:22:53.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:53.889+0000 7f1ffe800700 1 -- 172.21.15.37:0/1208190488 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ff81037c0 msgr2=0x7f1ff8113f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:53.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:53.889+0000 7f1ffe800700 1 --2- 172.21.15.37:0/1208190488 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ff81037c0 0x7f1ff8113f80 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f1fe4009fd0 tx=0x7f1fe4009e30 comp rx=0 tx=0).stop 2024-03-29T08:22:53.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:53.890+0000 7f1ffe800700 1 -- 172.21.15.37:0/1208190488 shutdown_connections 2024-03-29T08:22:53.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:53.890+0000 7f1ffe800700 1 --2- 172.21.15.37:0/1208190488 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f1fdc06cc30 0x7f1fdc06f0e0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:53.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:53.890+0000 7f1ffe800700 1 --2- 172.21.15.37:0/1208190488 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1ff8102e20 0x7f1ff8113a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:53.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:53.890+0000 7f1ffe800700 1 --2- 172.21.15.37:0/1208190488 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ff81037c0 0x7f1ff8113f80 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:53.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:53.890+0000 7f1ffe800700 1 -- 172.21.15.37:0/1208190488 >> 172.21.15.37:0/1208190488 conn(0x7f1ff80f89d0 msgr2=0x7f1ff80f9a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:22:53.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:53.890+0000 7f1ffe800700 1 -- 172.21.15.37:0/1208190488 shutdown_connections 2024-03-29T08:22:53.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:22:53.890+0000 7f1ffe800700 1 -- 172.21.15.37:0/1208190488 wait complete. 2024-03-29T08:22:54.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:22:53 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/2077801431' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T08:22:54.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:22:53 smithi148 ceph-mon[106692]: from='client.34720 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:22:54.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:22:53 smithi148 ceph-mon[106692]: pgmap v360: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:22:54.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:22:53 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/2077801431' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T08:22:54.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:22:53 smithi037 ceph-mon[142325]: from='client.34720 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:22:54.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:22:53 smithi037 ceph-mon[142325]: pgmap v360: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:22:55.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:22:54 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/1208190488' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T08:22:55.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:22:54 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:22:55.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:22:54 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/1208190488' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T08:22:55.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:22:54 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:22:56.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:22:55 smithi148 ceph-mon[106692]: pgmap v361: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:22:56.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:22:55 smithi037 ceph-mon[142325]: pgmap v361: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:22:58.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:22:57 smithi148 ceph-mon[106692]: pgmap v362: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:22:58.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:22:57 smithi037 ceph-mon[142325]: pgmap v362: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:23:00.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:23:00 smithi148 ceph-mon[106692]: pgmap v363: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:23:00.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:23:00 smithi037 ceph-mon[142325]: pgmap v363: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:23:02.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:23:01 smithi148 ceph-mon[106692]: pgmap v364: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:23:02.178 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:23:01 smithi037 ceph-mon[142325]: pgmap v364: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:23:04.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:23:03 smithi148 ceph-mon[106692]: pgmap v365: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:23:04.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:23:03 smithi037 ceph-mon[142325]: pgmap v365: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:23:06.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:23:05 smithi148 ceph-mon[106692]: pgmap v366: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:23:06.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:23:05 smithi037 ceph-mon[142325]: pgmap v366: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:23:08.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:23:07 smithi148 ceph-mon[106692]: pgmap v367: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:23:08.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:23:07 smithi037 ceph-mon[142325]: pgmap v367: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:23:10.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:23:10 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:23:10.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:23:10 smithi148 ceph-mon[106692]: pgmap v368: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:23:10.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:23:10 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:23:10.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:23:10 smithi037 ceph-mon[142325]: pgmap v368: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:23:12.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:23:11 smithi148 ceph-mon[106692]: pgmap v369: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:23:12.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:23:11 smithi037 ceph-mon[142325]: pgmap v369: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:23:14.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:23:13 smithi148 ceph-mon[106692]: pgmap v370: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:23:14.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:23:13 smithi037 ceph-mon[142325]: pgmap v370: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:23:16.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:23:15 smithi148 ceph-mon[106692]: pgmap v371: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:23:16.178 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:23:15 smithi037 ceph-mon[142325]: pgmap v371: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:23:18.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:23:17 smithi148 ceph-mon[106692]: pgmap v372: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:23:18.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:23:17 smithi037 ceph-mon[142325]: pgmap v372: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:23:20.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:23:20 smithi148 ceph-mon[106692]: pgmap v373: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:23:20.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:23:20 smithi037 ceph-mon[142325]: pgmap v373: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:23:22.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:23:21 smithi148 ceph-mon[106692]: pgmap v374: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:23:22.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:23:21 smithi037 ceph-mon[142325]: pgmap v374: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:23:24.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:24.044+0000 7fe390800700 1 -- 172.21.15.37:0/651906099 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe388102e00 msgr2=0x7fe3881031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:24.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:24.044+0000 7fe390800700 1 --2- 172.21.15.37:0/651906099 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe388102e00 0x7fe3881031d0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fe37c004940 tx=0x7fe37c009e30 comp rx=0 tx=0).stop 2024-03-29T08:23:24.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:24.044+0000 7fe390800700 1 -- 172.21.15.37:0/651906099 shutdown_connections 2024-03-29T08:23:24.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:24.044+0000 7fe390800700 1 --2- 172.21.15.37:0/651906099 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe3881037a0 0x7fe3880fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:24.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:24.044+0000 7fe390800700 1 --2- 172.21.15.37:0/651906099 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe388102e00 0x7fe3881031d0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:24.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:24.044+0000 7fe390800700 1 -- 172.21.15.37:0/651906099 >> 172.21.15.37:0/651906099 conn(0x7fe3880f89b0 msgr2=0x7fe3880fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:23:24.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:24.044+0000 7fe390800700 1 -- 172.21.15.37:0/651906099 shutdown_connections 2024-03-29T08:23:24.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:24.044+0000 7fe390800700 1 -- 172.21.15.37:0/651906099 wait complete. 2024-03-29T08:23:24.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:24.045+0000 7fe390800700 1 Processor -- start 2024-03-29T08:23:24.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:24.045+0000 7fe390800700 1 -- start start 2024-03-29T08:23:24.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:24.045+0000 7fe390800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe388102e00 0x7fe388115c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:24.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:24.045+0000 7fe390800700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe3881037a0 0x7fe388116190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:24.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:24.045+0000 7fe390800700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3881167e0 con 0x7fe388102e00 2024-03-29T08:23:24.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:24.045+0000 7fe390800700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe38810fbb0 con 0x7fe3881037a0 2024-03-29T08:23:24.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:24.046+0000 7fe38de00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe388102e00 0x7fe388115c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:24.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:24.046+0000 7fe38de00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe388102e00 0x7fe388115c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52298/0 (socket says 172.21.15.37:52298) 2024-03-29T08:23:24.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:24.046+0000 7fe38de00700 1 -- 172.21.15.37:0/190485501 learned_addr learned my addr 172.21.15.37:0/190485501 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:23:24.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:24.046+0000 7fe38d400700 1 --2- 172.21.15.37:0/190485501 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe3881037a0 0x7fe388116190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:24.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:24.046+0000 7fe38de00700 1 -- 172.21.15.37:0/190485501 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe3881037a0 msgr2=0x7fe388116190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:24.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:24.046+0000 7fe38de00700 1 --2- 172.21.15.37:0/190485501 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe3881037a0 0x7fe388116190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:24.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:24.046+0000 7fe38de00700 1 -- 172.21.15.37:0/190485501 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe37c005040 con 0x7fe388102e00 2024-03-29T08:23:24.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:24.047+0000 7fe38de00700 1 --2- 172.21.15.37:0/190485501 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe388102e00 0x7fe388115c50 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fe37c009390 tx=0x7fe37c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:23:24.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:24.047+0000 7fe382000700 1 -- 172.21.15.37:0/190485501 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe37c00e040 con 0x7fe388102e00 2024-03-29T08:23:24.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:24.047+0000 7fe390800700 1 -- 172.21.15.37:0/190485501 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe38810fe30 con 0x7fe388102e00 2024-03-29T08:23:24.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:24.047+0000 7fe382000700 1 -- 172.21.15.37:0/190485501 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fe37c013420 con 0x7fe388102e00 2024-03-29T08:23:24.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:24.047+0000 7fe382000700 1 -- 172.21.15.37:0/190485501 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe37c01c630 con 0x7fe388102e00 2024-03-29T08:23:24.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:24.047+0000 7fe390800700 1 -- 172.21.15.37:0/190485501 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3881103f0 con 0x7fe388102e00 2024-03-29T08:23:24.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:24.048+0000 7fe390800700 1 -- 172.21.15.37:0/190485501 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3880fe550 con 0x7fe388102e00 2024-03-29T08:23:24.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:24.049+0000 7fe382000700 1 -- 172.21.15.37:0/190485501 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fe37c0139e0 con 0x7fe388102e00 2024-03-29T08:23:24.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:24.050+0000 7fe382000700 1 --2- 172.21.15.37:0/190485501 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fe37006cc30 0x7fe37006f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:24.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:24.050+0000 7fe382000700 1 -- 172.21.15.37:0/190485501 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fe37c08d320 con 0x7fe388102e00 2024-03-29T08:23:24.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:24.050+0000 7fe38d400700 1 --2- 172.21.15.37:0/190485501 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fe37006cc30 0x7fe37006f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:24.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:24.051+0000 7fe38d400700 1 --2- 172.21.15.37:0/190485501 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fe37006cc30 0x7fe37006f0e0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fe378009fd0 tx=0x7fe378009f60 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:23:24.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:24.055+0000 7fe382000700 1 -- 172.21.15.37:0/190485501 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fe37c05ae20 con 0x7fe388102e00 2024-03-29T08:23:24.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:23:23 smithi148 ceph-mon[106692]: pgmap v375: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:23:24.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:23:23 smithi037 ceph-mon[142325]: pgmap v375: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:23:24.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:24.289+0000 7fe390800700 1 -- 172.21.15.37:0/190485501 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe38804bf70 con 0x7fe37006cc30 2024-03-29T08:23:24.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:24.290+0000 7fe382000700 1 -- 172.21.15.37:0/190485501 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fe378009420 con 0x7fe37006cc30 2024-03-29T08:23:24.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:24.293+0000 7fe390800700 1 -- 172.21.15.37:0/190485501 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fe37006cc30 msgr2=0x7fe37006f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:24.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:24.293+0000 7fe390800700 1 --2- 172.21.15.37:0/190485501 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fe37006cc30 0x7fe37006f0e0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fe378009fd0 tx=0x7fe378009f60 comp rx=0 tx=0).stop 2024-03-29T08:23:24.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:24.293+0000 7fe390800700 1 -- 172.21.15.37:0/190485501 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe388102e00 msgr2=0x7fe388115c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:24.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:24.293+0000 7fe390800700 1 --2- 172.21.15.37:0/190485501 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe388102e00 0x7fe388115c50 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fe37c009390 tx=0x7fe37c009e30 comp rx=0 tx=0).stop 2024-03-29T08:23:24.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:24.293+0000 7fe390800700 1 -- 172.21.15.37:0/190485501 shutdown_connections 2024-03-29T08:23:24.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:24.293+0000 7fe390800700 1 --2- 172.21.15.37:0/190485501 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fe37006cc30 0x7fe37006f0e0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:24.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:24.293+0000 7fe390800700 1 --2- 172.21.15.37:0/190485501 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe388102e00 0x7fe388115c50 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:24.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:24.293+0000 7fe390800700 1 --2- 172.21.15.37:0/190485501 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe3881037a0 0x7fe388116190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:24.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:24.293+0000 7fe390800700 1 -- 172.21.15.37:0/190485501 >> 172.21.15.37:0/190485501 conn(0x7fe3880f89b0 msgr2=0x7fe3880f9900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:23:24.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:24.293+0000 7fe390800700 1 -- 172.21.15.37:0/190485501 shutdown_connections 2024-03-29T08:23:24.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:24.293+0000 7fe390800700 1 -- 172.21.15.37:0/190485501 wait complete. 2024-03-29T08:23:24.308 INFO:teuthology.orchestra.run.smithi037.stdout:true 2024-03-29T08:23:24.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:24.434+0000 7f2e44200700 1 -- 172.21.15.37:0/1841852094 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2e3c102e00 msgr2=0x7f2e3c1031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:24.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:24.434+0000 7f2e44200700 1 --2- 172.21.15.37:0/1841852094 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2e3c102e00 0x7f2e3c1031d0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f2e30004940 tx=0x7f2e30009e30 comp rx=0 tx=0).stop 2024-03-29T08:23:24.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:24.434+0000 7f2e44200700 1 -- 172.21.15.37:0/1841852094 shutdown_connections 2024-03-29T08:23:24.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:24.434+0000 7f2e44200700 1 --2- 172.21.15.37:0/1841852094 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2e3c1037a0 0x7f2e3c0fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:24.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:24.434+0000 7f2e44200700 1 --2- 172.21.15.37:0/1841852094 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2e3c102e00 0x7f2e3c1031d0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:24.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:24.434+0000 7f2e44200700 1 -- 172.21.15.37:0/1841852094 >> 172.21.15.37:0/1841852094 conn(0x7f2e3c0f89b0 msgr2=0x7f2e3c0fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:23:24.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:24.434+0000 7f2e44200700 1 -- 172.21.15.37:0/1841852094 shutdown_connections 2024-03-29T08:23:24.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:24.434+0000 7f2e44200700 1 -- 172.21.15.37:0/1841852094 wait complete. 2024-03-29T08:23:24.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:24.435+0000 7f2e44200700 1 Processor -- start 2024-03-29T08:23:24.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:24.435+0000 7f2e44200700 1 -- start start 2024-03-29T08:23:24.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:24.435+0000 7f2e44200700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2e3c102e00 0x7f2e3c115cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:24.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:24.435+0000 7f2e44200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2e3c1037a0 0x7f2e3c116230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:24.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:24.435+0000 7f2e44200700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e3c116880 con 0x7f2e3c1037a0 2024-03-29T08:23:24.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:24.435+0000 7f2e44200700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e3c10fc50 con 0x7f2e3c102e00 2024-03-29T08:23:24.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:24.436+0000 7f2e40e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2e3c1037a0 0x7f2e3c116230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:24.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:24.436+0000 7f2e40e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2e3c1037a0 0x7f2e3c116230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52302/0 (socket says 172.21.15.37:52302) 2024-03-29T08:23:24.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:24.436+0000 7f2e40e00700 1 -- 172.21.15.37:0/3507333376 learned_addr learned my addr 172.21.15.37:0/3507333376 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:23:24.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:24.436+0000 7f2e41800700 1 --2- 172.21.15.37:0/3507333376 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2e3c102e00 0x7f2e3c115cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:24.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:24.436+0000 7f2e40e00700 1 -- 172.21.15.37:0/3507333376 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2e3c102e00 msgr2=0x7f2e3c115cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:24.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:24.436+0000 7f2e40e00700 1 --2- 172.21.15.37:0/3507333376 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2e3c102e00 0x7f2e3c115cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:24.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:24.436+0000 7f2e40e00700 1 -- 172.21.15.37:0/3507333376 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e30005040 con 0x7f2e3c1037a0 2024-03-29T08:23:24.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:24.436+0000 7f2e40e00700 1 --2- 172.21.15.37:0/3507333376 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2e3c1037a0 0x7f2e3c116230 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f2e2800c890 tx=0x7f2e280044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:23:24.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:24.436+0000 7f2e39600700 1 -- 172.21.15.37:0/3507333376 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2e2800a7d0 con 0x7f2e3c1037a0 2024-03-29T08:23:24.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:24.436+0000 7f2e44200700 1 -- 172.21.15.37:0/3507333376 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e3c10ff30 con 0x7f2e3c1037a0 2024-03-29T08:23:24.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:24.436+0000 7f2e39600700 1 -- 172.21.15.37:0/3507333376 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f2e2800ade0 con 0x7f2e3c1037a0 2024-03-29T08:23:24.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:24.436+0000 7f2e39600700 1 -- 172.21.15.37:0/3507333376 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2e280113a0 con 0x7f2e3c1037a0 2024-03-29T08:23:24.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:24.437+0000 7f2e44200700 1 -- 172.21.15.37:0/3507333376 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e3c1104f0 con 0x7f2e3c1037a0 2024-03-29T08:23:24.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:24.438+0000 7f2e44200700 1 -- 172.21.15.37:0/3507333376 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e3c0fe550 con 0x7f2e3c1037a0 2024-03-29T08:23:24.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:24.439+0000 7f2e39600700 1 -- 172.21.15.37:0/3507333376 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f2e28012400 con 0x7f2e3c1037a0 2024-03-29T08:23:24.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:24.441+0000 7f2e39600700 1 --2- 172.21.15.37:0/3507333376 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f2e2406cc30 0x7f2e2406f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:24.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:24.441+0000 7f2e39600700 1 -- 172.21.15.37:0/3507333376 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f2e2808c1b0 con 0x7f2e3c1037a0 2024-03-29T08:23:24.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:24.441+0000 7f2e41800700 1 --2- 172.21.15.37:0/3507333376 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f2e2406cc30 0x7f2e2406f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:24.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:24.442+0000 7f2e41800700 1 --2- 172.21.15.37:0/3507333376 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f2e2406cc30 0x7f2e2406f0e0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f2e3000b020 tx=0x7f2e30009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:23:24.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:24.445+0000 7f2e39600700 1 -- 172.21.15.37:0/3507333376 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f2e28059c00 con 0x7f2e3c1037a0 2024-03-29T08:23:24.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:24.689+0000 7f2e44200700 1 -- 172.21.15.37:0/3507333376 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f2e3c109e30 con 0x7f2e2406cc30 2024-03-29T08:23:24.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:24.690+0000 7f2e39600700 1 -- 172.21.15.37:0/3507333376 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f2e3c109e30 con 0x7f2e2406cc30 2024-03-29T08:23:24.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:24.693+0000 7f2e44200700 1 -- 172.21.15.37:0/3507333376 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f2e2406cc30 msgr2=0x7f2e2406f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:24.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:24.693+0000 7f2e44200700 1 --2- 172.21.15.37:0/3507333376 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f2e2406cc30 0x7f2e2406f0e0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f2e3000b020 tx=0x7f2e30009e30 comp rx=0 tx=0).stop 2024-03-29T08:23:24.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:24.693+0000 7f2e44200700 1 -- 172.21.15.37:0/3507333376 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2e3c1037a0 msgr2=0x7f2e3c116230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:24.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:24.693+0000 7f2e44200700 1 --2- 172.21.15.37:0/3507333376 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2e3c1037a0 0x7f2e3c116230 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f2e2800c890 tx=0x7f2e280044c0 comp rx=0 tx=0).stop 2024-03-29T08:23:24.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:24.693+0000 7f2e44200700 1 -- 172.21.15.37:0/3507333376 shutdown_connections 2024-03-29T08:23:24.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:24.693+0000 7f2e44200700 1 --2- 172.21.15.37:0/3507333376 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f2e2406cc30 0x7f2e2406f0e0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:24.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:24.693+0000 7f2e44200700 1 --2- 172.21.15.37:0/3507333376 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2e3c102e00 0x7f2e3c115cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:24.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:24.693+0000 7f2e44200700 1 --2- 172.21.15.37:0/3507333376 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2e3c1037a0 0x7f2e3c116230 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:24.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:24.693+0000 7f2e44200700 1 -- 172.21.15.37:0/3507333376 >> 172.21.15.37:0/3507333376 conn(0x7f2e3c0f89b0 msgr2=0x7f2e3c0706d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:23:24.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:24.693+0000 7f2e44200700 1 -- 172.21.15.37:0/3507333376 shutdown_connections 2024-03-29T08:23:24.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:24.693+0000 7f2e44200700 1 -- 172.21.15.37:0/3507333376 wait complete. 2024-03-29T08:23:24.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:24.830+0000 7fa281a00700 1 -- 172.21.15.37:0/1453040280 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa27c071f90 msgr2=0x7fa27c070360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:24.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:24.830+0000 7fa281a00700 1 --2- 172.21.15.37:0/1453040280 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa27c071f90 0x7fa27c070360 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fa26c004970 tx=0x7fa26c009e30 comp rx=0 tx=0).stop 2024-03-29T08:23:24.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:24.830+0000 7fa281a00700 1 -- 172.21.15.37:0/1453040280 shutdown_connections 2024-03-29T08:23:24.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:24.830+0000 7fa281a00700 1 --2- 172.21.15.37:0/1453040280 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa27c0708a0 0x7fa27c070d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:24.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:24.830+0000 7fa281a00700 1 --2- 172.21.15.37:0/1453040280 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa27c071f90 0x7fa27c070360 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:24.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:24.830+0000 7fa281a00700 1 -- 172.21.15.37:0/1453040280 >> 172.21.15.37:0/1453040280 conn(0x7fa27c0f89b0 msgr2=0x7fa27c0fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:23:24.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:24.830+0000 7fa281a00700 1 -- 172.21.15.37:0/1453040280 shutdown_connections 2024-03-29T08:23:24.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:24.830+0000 7fa281a00700 1 -- 172.21.15.37:0/1453040280 wait complete. 2024-03-29T08:23:24.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:24.831+0000 7fa281a00700 1 Processor -- start 2024-03-29T08:23:24.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:24.831+0000 7fa281a00700 1 -- start start 2024-03-29T08:23:24.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:24.831+0000 7fa281a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa27c0708a0 0x7fa27c11a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:24.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:24.831+0000 7fa281a00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa27c071f90 0x7fa27c11a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:24.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:24.831+0000 7fa281a00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa27c11ac50 con 0x7fa27c0708a0 2024-03-29T08:23:24.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:24.831+0000 7fa281a00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa27c114020 con 0x7fa27c071f90 2024-03-29T08:23:24.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:24.831+0000 7fa27aa00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa27c0708a0 0x7fa27c11a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:24.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:24.831+0000 7fa27aa00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa27c0708a0 0x7fa27c11a0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52332/0 (socket says 172.21.15.37:52332) 2024-03-29T08:23:24.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:24.831+0000 7fa27aa00700 1 -- 172.21.15.37:0/854976304 learned_addr learned my addr 172.21.15.37:0/854976304 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:23:24.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:24.832+0000 7fa27aa00700 1 -- 172.21.15.37:0/854976304 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa27c071f90 msgr2=0x7fa27c11a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:24.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:24.832+0000 7fa27aa00700 1 --2- 172.21.15.37:0/854976304 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa27c071f90 0x7fa27c11a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:24.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:24.832+0000 7fa27aa00700 1 -- 172.21.15.37:0/854976304 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa26c005040 con 0x7fa27c0708a0 2024-03-29T08:23:24.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:24.832+0000 7fa27aa00700 1 --2- 172.21.15.37:0/854976304 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa27c0708a0 0x7fa27c11a0c0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fa26c004f50 tx=0x7fa26c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:23:24.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:24.832+0000 7fa272a00700 1 -- 172.21.15.37:0/854976304 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa26c00e040 con 0x7fa27c0708a0 2024-03-29T08:23:24.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:24.832+0000 7fa272a00700 1 -- 172.21.15.37:0/854976304 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fa26c013420 con 0x7fa27c0708a0 2024-03-29T08:23:24.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:24.832+0000 7fa281a00700 1 -- 172.21.15.37:0/854976304 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa27c1142a0 con 0x7fa27c0708a0 2024-03-29T08:23:24.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:24.832+0000 7fa272a00700 1 -- 172.21.15.37:0/854976304 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa26c01c610 con 0x7fa27c0708a0 2024-03-29T08:23:24.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:24.833+0000 7fa281a00700 1 -- 172.21.15.37:0/854976304 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa27c1147e0 con 0x7fa27c0708a0 2024-03-29T08:23:24.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:24.834+0000 7fa281a00700 1 -- 172.21.15.37:0/854976304 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa27c071aa0 con 0x7fa27c0708a0 2024-03-29T08:23:24.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:24.835+0000 7fa272a00700 1 -- 172.21.15.37:0/854976304 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fa26c013590 con 0x7fa27c0708a0 2024-03-29T08:23:24.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:24.836+0000 7fa272a00700 1 --2- 172.21.15.37:0/854976304 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fa26006cc30 0x7fa26006f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:24.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:24.836+0000 7fa272a00700 1 -- 172.21.15.37:0/854976304 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fa26c08d310 con 0x7fa27c0708a0 2024-03-29T08:23:24.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:24.836+0000 7fa27a000700 1 --2- 172.21.15.37:0/854976304 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fa26006cc30 0x7fa26006f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:24.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:24.836+0000 7fa27a000700 1 --2- 172.21.15.37:0/854976304 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fa26006cc30 0x7fa26006f0e0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fa27c115960 tx=0x7fa268009ce0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:23:24.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:24.840+0000 7fa272a00700 1 -- 172.21.15.37:0/854976304 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fa26c05ae10 con 0x7fa27c0708a0 2024-03-29T08:23:25.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:25.071+0000 7fa281a00700 1 -- 172.21.15.37:0/854976304 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fa27c1103e0 con 0x7fa26006cc30 2024-03-29T08:23:25.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:25.081+0000 7fa272a00700 1 -- 172.21.15.37:0/854976304 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7fa27c1103e0 con 0x7fa26006cc30 2024-03-29T08:23:25.081 INFO:teuthology.orchestra.run.smithi037.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-29T08:23:25.081 INFO:teuthology.orchestra.run.smithi037.stdout:alertmanager.smithi037 smithi037 *:9093,9094 running (13m) 5m ago 26m 14.9M - 0.25.0 c8568f914cd2 2f9f9746ed0f 2024-03-29T08:23:25.081 INFO:teuthology.orchestra.run.smithi037.stdout:crash.smithi037 smithi037 running (10m) 5m ago 26m 6647k - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 178d416a3186 2024-03-29T08:23:25.081 INFO:teuthology.orchestra.run.smithi037.stdout:crash.smithi148 smithi148 running (10m) 5m ago 25m 6656k - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 23666989d58f 2024-03-29T08:23:25.081 INFO:teuthology.orchestra.run.smithi037.stdout:grafana.smithi037 smithi037 *:3000 running (12m) 5m ago 25m 71.9M - 9.4.7 954c08fa6188 6b976fe49952 2024-03-29T08:23:25.081 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi037.iifsmh smithi037 running (5m) 5m ago 19m 10.8M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 2d97f562cc5b 2024-03-29T08:23:25.082 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi037.mmwgxe smithi037 running (5m) 5m ago 19m 10.9M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff ec37037f46e8 2024-03-29T08:23:25.082 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi148.qukpjh smithi148 running (19m) 5m ago 19m 15.3M - 17.2.7-789-g4213e17c cf22eb8683be 8e469782a273 2024-03-29T08:23:25.082 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi148.sqteih smithi148 running (5m) 5m ago 19m 10.7M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 44c15c77037a 2024-03-29T08:23:25.082 INFO:teuthology.orchestra.run.smithi037.stdout:mgr.smithi037.wjikii smithi037 *:8443,9283,8765 running (15m) 5m ago 27m 506M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff c5a1417b9417 2024-03-29T08:23:25.082 INFO:teuthology.orchestra.run.smithi037.stdout:mgr.smithi148.onhgvm smithi148 *:8443,9283,8765 running (14m) 5m ago 25m 450M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff fcfdbce062bd 2024-03-29T08:23:25.082 INFO:teuthology.orchestra.run.smithi037.stdout:mon.smithi037 smithi037 running (11m) 5m ago 28m 54.8M 2048M 18.2.2-1015-gd0e392d9 1cc2f45c17ff eea2060783af 2024-03-29T08:23:25.082 INFO:teuthology.orchestra.run.smithi037.stdout:mon.smithi148 smithi148 running (10m) 5m ago 24m 43.7M 2048M 18.2.2-1015-gd0e392d9 1cc2f45c17ff ad4b2c5748be 2024-03-29T08:23:25.082 INFO:teuthology.orchestra.run.smithi037.stdout:node-exporter.smithi037 smithi037 *:9100 running (14m) 5m ago 26m 7625k - 1.5.0 0da6a335fe13 26b8ae9cb878 2024-03-29T08:23:25.082 INFO:teuthology.orchestra.run.smithi037.stdout:node-exporter.smithi148 smithi148 *:9100 running (13m) 5m ago 25m 8166k - 1.5.0 0da6a335fe13 98414e4e64ed 2024-03-29T08:23:25.082 INFO:teuthology.orchestra.run.smithi037.stdout:osd.0 smithi037 running (9m) 5m ago 24m 104M 1619M 18.2.2-1015-gd0e392d9 1cc2f45c17ff c0c584f6245c 2024-03-29T08:23:25.082 INFO:teuthology.orchestra.run.smithi037.stdout:osd.1 smithi037 running (8m) 5m ago 23m 100M 1619M 18.2.2-1015-gd0e392d9 1cc2f45c17ff a4a2ff4cc83d 2024-03-29T08:23:25.082 INFO:teuthology.orchestra.run.smithi037.stdout:osd.2 smithi037 running (8m) 5m ago 23m 82.3M 1619M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 8b1bc71aa79e 2024-03-29T08:23:25.082 INFO:teuthology.orchestra.run.smithi037.stdout:osd.3 smithi148 running (7m) 5m ago 22m 105M 2642M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 1c656ab8ff86 2024-03-29T08:23:25.082 INFO:teuthology.orchestra.run.smithi037.stdout:osd.4 smithi148 running (6m) 5m ago 21m 97.3M 2642M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 966a9de11d22 2024-03-29T08:23:25.083 INFO:teuthology.orchestra.run.smithi037.stdout:osd.5 smithi148 running (6m) 5m ago 21m 85.8M 2642M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 474c4f9212ce 2024-03-29T08:23:25.083 INFO:teuthology.orchestra.run.smithi037.stdout:prometheus.smithi037 smithi037 *:9095 running (13m) 5m ago 25m 54.4M - 2.43.0 a07b618ecd1d 30c0223ec589 2024-03-29T08:23:25.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:25.083+0000 7fa281a00700 1 -- 172.21.15.37:0/854976304 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fa26006cc30 msgr2=0x7fa26006f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:25.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:25.083+0000 7fa281a00700 1 --2- 172.21.15.37:0/854976304 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fa26006cc30 0x7fa26006f0e0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fa27c115960 tx=0x7fa268009ce0 comp rx=0 tx=0).stop 2024-03-29T08:23:25.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:25.083+0000 7fa281a00700 1 -- 172.21.15.37:0/854976304 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa27c0708a0 msgr2=0x7fa27c11a0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:25.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:25.083+0000 7fa281a00700 1 --2- 172.21.15.37:0/854976304 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa27c0708a0 0x7fa27c11a0c0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fa26c004f50 tx=0x7fa26c009e30 comp rx=0 tx=0).stop 2024-03-29T08:23:25.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:25.083+0000 7fa281a00700 1 -- 172.21.15.37:0/854976304 shutdown_connections 2024-03-29T08:23:25.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:25.083+0000 7fa281a00700 1 --2- 172.21.15.37:0/854976304 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fa26006cc30 0x7fa26006f0e0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:25.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:25.083+0000 7fa281a00700 1 --2- 172.21.15.37:0/854976304 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa27c0708a0 0x7fa27c11a0c0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:25.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:25.083+0000 7fa281a00700 1 --2- 172.21.15.37:0/854976304 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa27c071f90 0x7fa27c11a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:25.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:25.083+0000 7fa281a00700 1 -- 172.21.15.37:0/854976304 >> 172.21.15.37:0/854976304 conn(0x7fa27c0f89b0 msgr2=0x7fa27c0f9b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:23:25.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:25.083+0000 7fa281a00700 1 -- 172.21.15.37:0/854976304 shutdown_connections 2024-03-29T08:23:25.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:25.084+0000 7fa281a00700 1 -- 172.21.15.37:0/854976304 wait complete. 2024-03-29T08:23:25.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:25.222+0000 7f685f200700 1 -- 172.21.15.37:0/313045244 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6858102c40 msgr2=0x7f6858103010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:25.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:25.222+0000 7f685f200700 1 --2- 172.21.15.37:0/313045244 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6858102c40 0x7f6858103010 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f684c004940 tx=0x7f684c009e30 comp rx=0 tx=0).stop 2024-03-29T08:23:25.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:25.222+0000 7f685f200700 1 -- 172.21.15.37:0/313045244 shutdown_connections 2024-03-29T08:23:25.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:25.222+0000 7f685f200700 1 --2- 172.21.15.37:0/313045244 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f68581035e0 0x7f68580fcc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:25.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:25.222+0000 7f685f200700 1 --2- 172.21.15.37:0/313045244 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6858102c40 0x7f6858103010 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:25.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:25.222+0000 7f685f200700 1 -- 172.21.15.37:0/313045244 >> 172.21.15.37:0/313045244 conn(0x7f68580f8830 msgr2=0x7f68580fac40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:23:25.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:25.222+0000 7f685f200700 1 -- 172.21.15.37:0/313045244 shutdown_connections 2024-03-29T08:23:25.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:25.222+0000 7f685f200700 1 -- 172.21.15.37:0/313045244 wait complete. 2024-03-29T08:23:25.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:25.223+0000 7f685f200700 1 Processor -- start 2024-03-29T08:23:25.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:25.223+0000 7f685f200700 1 -- start start 2024-03-29T08:23:25.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:25.223+0000 7f685f200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6858102c40 0x7f6858113870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:25.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:25.223+0000 7f685f200700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f68581035e0 0x7f6858113db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:25.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:25.224+0000 7f685f200700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6858114400 con 0x7f6858102c40 2024-03-29T08:23:25.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:25.224+0000 7f685f200700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f685810d7f0 con 0x7f68581035e0 2024-03-29T08:23:25.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:25.224+0000 7f685de00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6858102c40 0x7f6858113870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:25.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:25.224+0000 7f685de00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6858102c40 0x7f6858113870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52348/0 (socket says 172.21.15.37:52348) 2024-03-29T08:23:25.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:25.224+0000 7f685de00700 1 -- 172.21.15.37:0/1500312551 learned_addr learned my addr 172.21.15.37:0/1500312551 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:23:25.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:25.224+0000 7f685d400700 1 --2- 172.21.15.37:0/1500312551 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f68581035e0 0x7f6858113db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:25.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:25.224+0000 7f685de00700 1 -- 172.21.15.37:0/1500312551 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f68581035e0 msgr2=0x7f6858113db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:25.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:25.224+0000 7f685de00700 1 --2- 172.21.15.37:0/1500312551 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f68581035e0 0x7f6858113db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:25.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:25.224+0000 7f685de00700 1 -- 172.21.15.37:0/1500312551 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f684c005040 con 0x7f6858102c40 2024-03-29T08:23:25.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:25.224+0000 7f685de00700 1 --2- 172.21.15.37:0/1500312551 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6858102c40 0x7f6858113870 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f684c007f50 tx=0x7f684c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:23:25.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:25.225+0000 7f6852000700 1 -- 172.21.15.37:0/1500312551 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f684c021070 con 0x7f6858102c40 2024-03-29T08:23:25.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:25.225+0000 7f6852000700 1 -- 172.21.15.37:0/1500312551 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f684c013e00 con 0x7f6858102c40 2024-03-29T08:23:25.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:25.225+0000 7f685f200700 1 -- 172.21.15.37:0/1500312551 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f685810da70 con 0x7f6858102c40 2024-03-29T08:23:25.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:25.225+0000 7f6852000700 1 -- 172.21.15.37:0/1500312551 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f684c00e070 con 0x7f6858102c40 2024-03-29T08:23:25.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:25.225+0000 7f685f200700 1 -- 172.21.15.37:0/1500312551 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f685810e030 con 0x7f6858102c40 2024-03-29T08:23:25.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:25.226+0000 7f685f200700 1 -- 172.21.15.37:0/1500312551 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68580fe390 con 0x7f6858102c40 2024-03-29T08:23:25.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:25.227+0000 7f6852000700 1 -- 172.21.15.37:0/1500312551 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f684c013950 con 0x7f6858102c40 2024-03-29T08:23:25.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:25.228+0000 7f6852000700 1 --2- 172.21.15.37:0/1500312551 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f684006cc30 0x7f684006f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:25.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:25.228+0000 7f6852000700 1 -- 172.21.15.37:0/1500312551 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f684c08d2d0 con 0x7f6858102c40 2024-03-29T08:23:25.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:25.228+0000 7f685d400700 1 --2- 172.21.15.37:0/1500312551 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f684006cc30 0x7f684006f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:25.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:25.229+0000 7f685d400700 1 --2- 172.21.15.37:0/1500312551 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f684006cc30 0x7f684006f0e0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f6848004600 tx=0x7f684800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:23:25.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:25.232+0000 7f6852000700 1 -- 172.21.15.37:0/1500312551 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f684c05ada0 con 0x7f6858102c40 2024-03-29T08:23:25.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:25.533+0000 7f685f200700 1 -- 172.21.15.37:0/1500312551 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f6858109bf0 con 0x7f6858102c40 2024-03-29T08:23:25.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:25.533+0000 7f6852000700 1 -- 172.21.15.37:0/1500312551 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+847 (secure 0 0 0) 0x7f684c05a930 con 0x7f6858102c40 2024-03-29T08:23:25.533 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-29T08:23:25.534 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": { 2024-03-29T08:23:25.534 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 2 2024-03-29T08:23:25.534 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:23:25.534 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": { 2024-03-29T08:23:25.534 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 2 2024-03-29T08:23:25.534 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:23:25.534 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": { 2024-03-29T08:23:25.534 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 6 2024-03-29T08:23:25.534 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:23:25.534 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": { 2024-03-29T08:23:25.534 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 17.2.7-789-g4213e17c (4213e17c4de67016ed14f55d22503f4923b3a901) quincy (stable)": 1, 2024-03-29T08:23:25.534 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 3 2024-03-29T08:23:25.534 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:23:25.534 INFO:teuthology.orchestra.run.smithi037.stdout: "overall": { 2024-03-29T08:23:25.535 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 17.2.7-789-g4213e17c (4213e17c4de67016ed14f55d22503f4923b3a901) quincy (stable)": 1, 2024-03-29T08:23:25.535 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 13 2024-03-29T08:23:25.535 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-29T08:23:25.535 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-29T08:23:25.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:25.536+0000 7f685f200700 1 -- 172.21.15.37:0/1500312551 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f684006cc30 msgr2=0x7f684006f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:25.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:25.536+0000 7f685f200700 1 --2- 172.21.15.37:0/1500312551 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f684006cc30 0x7f684006f0e0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f6848004600 tx=0x7f684800b040 comp rx=0 tx=0).stop 2024-03-29T08:23:25.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:25.536+0000 7f685f200700 1 -- 172.21.15.37:0/1500312551 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6858102c40 msgr2=0x7f6858113870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:25.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:25.536+0000 7f685f200700 1 --2- 172.21.15.37:0/1500312551 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6858102c40 0x7f6858113870 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f684c007f50 tx=0x7f684c009e30 comp rx=0 tx=0).stop 2024-03-29T08:23:25.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:25.536+0000 7f685f200700 1 -- 172.21.15.37:0/1500312551 shutdown_connections 2024-03-29T08:23:25.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:25.536+0000 7f685f200700 1 --2- 172.21.15.37:0/1500312551 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f684006cc30 0x7f684006f0e0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:25.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:25.536+0000 7f685f200700 1 --2- 172.21.15.37:0/1500312551 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6858102c40 0x7f6858113870 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:25.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:25.536+0000 7f685f200700 1 --2- 172.21.15.37:0/1500312551 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f68581035e0 0x7f6858113db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:25.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:25.536+0000 7f685f200700 1 -- 172.21.15.37:0/1500312551 >> 172.21.15.37:0/1500312551 conn(0x7f68580f8830 msgr2=0x7f68580f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:23:25.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:25.536+0000 7f685f200700 1 -- 172.21.15.37:0/1500312551 shutdown_connections 2024-03-29T08:23:25.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:25.536+0000 7f685f200700 1 -- 172.21.15.37:0/1500312551 wait complete. 2024-03-29T08:23:25.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:23:25 smithi148 ceph-mon[106692]: from='client.34728 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:23:25.557 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:23:25 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:23:25.557 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:23:25 smithi148 ceph-mon[106692]: from='client.34732 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:23:25.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:25.673+0000 7f7097c00700 1 -- 172.21.15.37:0/762697166 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f70900fc590 msgr2=0x7f70901092f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:25.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:25.673+0000 7f7097c00700 1 --2- 172.21.15.37:0/762697166 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f70900fc590 0x7f70901092f0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f707c004940 tx=0x7f707c009e30 comp rx=0 tx=0).stop 2024-03-29T08:23:25.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:25.673+0000 7f7097c00700 1 -- 172.21.15.37:0/762697166 shutdown_connections 2024-03-29T08:23:25.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:25.673+0000 7f7097c00700 1 --2- 172.21.15.37:0/762697166 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f70900fc590 0x7f70901092f0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:25.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:25.673+0000 7f7097c00700 1 --2- 172.21.15.37:0/762697166 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f70900fbc80 0x7f70900fc050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:25.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:25.673+0000 7f7097c00700 1 -- 172.21.15.37:0/762697166 >> 172.21.15.37:0/762697166 conn(0x7f7090072bc0 msgr2=0x7f7090072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:23:25.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:25.674+0000 7f7097c00700 1 -- 172.21.15.37:0/762697166 shutdown_connections 2024-03-29T08:23:25.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:25.674+0000 7f7097c00700 1 -- 172.21.15.37:0/762697166 wait complete. 2024-03-29T08:23:25.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:25.674+0000 7f7097c00700 1 Processor -- start 2024-03-29T08:23:25.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:25.674+0000 7f7097c00700 1 -- start start 2024-03-29T08:23:25.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:25.675+0000 7f7097c00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f70900fbc80 0x7f7090115ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:25.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:25.675+0000 7f7097c00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f70900fc590 0x7f70901160e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:25.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:25.675+0000 7f7097c00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7090116730 con 0x7f70900fc590 2024-03-29T08:23:25.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:25.675+0000 7f7097c00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f709010fb90 con 0x7f70900fbc80 2024-03-29T08:23:25.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:25.675+0000 7f708fe00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f70900fc590 0x7f70901160e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:25.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:25.675+0000 7f708fe00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f70900fc590 0x7f70901160e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52368/0 (socket says 172.21.15.37:52368) 2024-03-29T08:23:25.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:25.675+0000 7f708fe00700 1 -- 172.21.15.37:0/3831622511 learned_addr learned my addr 172.21.15.37:0/3831622511 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:23:25.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:25.675+0000 7f7095200700 1 --2- 172.21.15.37:0/3831622511 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f70900fbc80 0x7f7090115ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:25.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:25.676+0000 7f708fe00700 1 -- 172.21.15.37:0/3831622511 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f70900fbc80 msgr2=0x7f7090115ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:25.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:25.676+0000 7f708fe00700 1 --2- 172.21.15.37:0/3831622511 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f70900fbc80 0x7f7090115ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:25.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:25.676+0000 7f708fe00700 1 -- 172.21.15.37:0/3831622511 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f707c005040 con 0x7f70900fc590 2024-03-29T08:23:25.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:25.676+0000 7f708fe00700 1 --2- 172.21.15.37:0/3831622511 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f70900fc590 0x7f70901160e0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f707c008d80 tx=0x7f707c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:23:25.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:25.676+0000 7f708cc00700 1 -- 172.21.15.37:0/3831622511 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f707c00e030 con 0x7f70900fc590 2024-03-29T08:23:25.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:25.676+0000 7f708cc00700 1 -- 172.21.15.37:0/3831622511 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f707c013de0 con 0x7f70900fc590 2024-03-29T08:23:25.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:25.676+0000 7f7097c00700 1 -- 172.21.15.37:0/3831622511 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f709010fe10 con 0x7f70900fc590 2024-03-29T08:23:25.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:25.676+0000 7f708cc00700 1 -- 172.21.15.37:0/3831622511 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f707c01cd00 con 0x7f70900fc590 2024-03-29T08:23:25.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:25.676+0000 7f7097c00700 1 -- 172.21.15.37:0/3831622511 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70901103d0 con 0x7f70900fc590 2024-03-29T08:23:25.678 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:23:25 smithi037 ceph-mon[142325]: from='client.34728 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:23:25.678 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:23:25 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:23:25.678 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:23:25 smithi037 ceph-mon[142325]: from='client.34732 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:23:25.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:25.677+0000 7f7097c00700 1 -- 172.21.15.37:0/3831622511 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7090065e30 con 0x7f70900fc590 2024-03-29T08:23:25.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:25.678+0000 7f708cc00700 1 -- 172.21.15.37:0/3831622511 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f707c013930 con 0x7f70900fc590 2024-03-29T08:23:25.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:25.679+0000 7f708cc00700 1 --2- 172.21.15.37:0/3831622511 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f707806cc30 0x7f707806f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:25.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:25.679+0000 7f708cc00700 1 -- 172.21.15.37:0/3831622511 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f707c08d940 con 0x7f70900fc590 2024-03-29T08:23:25.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:25.680+0000 7f7095200700 1 --2- 172.21.15.37:0/3831622511 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f707806cc30 0x7f707806f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:25.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:25.680+0000 7f7095200700 1 --2- 172.21.15.37:0/3831622511 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f707806cc30 0x7f707806f0e0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f7084009b80 tx=0x7f708400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:23:25.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:25.684+0000 7f708cc00700 1 -- 172.21.15.37:0/3831622511 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f707c057900 con 0x7f70900fc590 2024-03-29T08:23:25.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:25.944+0000 7f7097c00700 1 -- 172.21.15.37:0/3831622511 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f709010c020 con 0x7f70900fc590 2024-03-29T08:23:25.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:25.944+0000 7f708cc00700 1 -- 172.21.15.37:0/3831622511 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 28 v28) v1 ==== 76+0+1841 (secure 0 0 0) 0x7f707c01ce60 con 0x7f70900fc590 2024-03-29T08:23:25.944 INFO:teuthology.orchestra.run.smithi037.stdout:e28 2024-03-29T08:23:25.945 INFO:teuthology.orchestra.run.smithi037.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-29T08:23:25.945 INFO:teuthology.orchestra.run.smithi037.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T08:23:25.945 INFO:teuthology.orchestra.run.smithi037.stdout:legacy client fscid: 1 2024-03-29T08:23:25.945 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:23:25.945 INFO:teuthology.orchestra.run.smithi037.stdout:Filesystem 'cephfs' (1) 2024-03-29T08:23:25.945 INFO:teuthology.orchestra.run.smithi037.stdout:fs_name cephfs 2024-03-29T08:23:25.945 INFO:teuthology.orchestra.run.smithi037.stdout:epoch 21 2024-03-29T08:23:25.945 INFO:teuthology.orchestra.run.smithi037.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-03-29T08:23:25.945 INFO:teuthology.orchestra.run.smithi037.stdout:created 2024-03-29T08:03:29.241269+0000 2024-03-29T08:23:25.945 INFO:teuthology.orchestra.run.smithi037.stdout:modified 2024-03-29T08:17:16.237238+0000 2024-03-29T08:23:25.945 INFO:teuthology.orchestra.run.smithi037.stdout:tableserver 0 2024-03-29T08:23:25.945 INFO:teuthology.orchestra.run.smithi037.stdout:root 0 2024-03-29T08:23:25.945 INFO:teuthology.orchestra.run.smithi037.stdout:session_timeout 60 2024-03-29T08:23:25.946 INFO:teuthology.orchestra.run.smithi037.stdout:session_autoclose 300 2024-03-29T08:23:25.946 INFO:teuthology.orchestra.run.smithi037.stdout:max_file_size 1099511627776 2024-03-29T08:23:25.946 INFO:teuthology.orchestra.run.smithi037.stdout:max_xattr_size 65536 2024-03-29T08:23:25.946 INFO:teuthology.orchestra.run.smithi037.stdout:required_client_features {} 2024-03-29T08:23:25.946 INFO:teuthology.orchestra.run.smithi037.stdout:last_failure 0 2024-03-29T08:23:25.946 INFO:teuthology.orchestra.run.smithi037.stdout:last_failure_osd_epoch 82 2024-03-29T08:23:25.946 INFO:teuthology.orchestra.run.smithi037.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T08:23:25.946 INFO:teuthology.orchestra.run.smithi037.stdout:max_mds 2 2024-03-29T08:23:25.946 INFO:teuthology.orchestra.run.smithi037.stdout:in 0,1 2024-03-29T08:23:25.946 INFO:teuthology.orchestra.run.smithi037.stdout:up {} 2024-03-29T08:23:25.946 INFO:teuthology.orchestra.run.smithi037.stdout:failed 0,1 2024-03-29T08:23:25.946 INFO:teuthology.orchestra.run.smithi037.stdout:damaged 2024-03-29T08:23:25.946 INFO:teuthology.orchestra.run.smithi037.stdout:stopped 2024-03-29T08:23:25.947 INFO:teuthology.orchestra.run.smithi037.stdout:data_pools [3] 2024-03-29T08:23:25.947 INFO:teuthology.orchestra.run.smithi037.stdout:metadata_pool 2 2024-03-29T08:23:25.947 INFO:teuthology.orchestra.run.smithi037.stdout:inline_data disabled 2024-03-29T08:23:25.947 INFO:teuthology.orchestra.run.smithi037.stdout:balancer 2024-03-29T08:23:25.947 INFO:teuthology.orchestra.run.smithi037.stdout:bal_rank_mask -1 2024-03-29T08:23:25.947 INFO:teuthology.orchestra.run.smithi037.stdout:standby_count_wanted 1 2024-03-29T08:23:25.947 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:23:25.947 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:23:25.947 INFO:teuthology.orchestra.run.smithi037.stdout:Standby daemons: 2024-03-29T08:23:25.947 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:23:25.947 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi148.qukpjh{-1:24319} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.148:6826/2641327912,v1:172.21.15.148:6827/2641327912] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:23:25.947 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi037.mmwgxe{-1:34440} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.37:6828/1248142005,v1:172.21.15.37:6829/1248142005] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:23:25.947 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi037.iifsmh{-1:34444} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.37:6826/2060882959,v1:172.21.15.37:6827/2060882959] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:23:25.947 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi148.sqteih{-1:44395} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.148:6824/4078652242,v1:172.21.15.148:6825/4078652242] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:23:25.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:25.947+0000 7f7097c00700 1 -- 172.21.15.37:0/3831622511 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f707806cc30 msgr2=0x7f707806f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:25.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:25.947+0000 7f7097c00700 1 --2- 172.21.15.37:0/3831622511 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f707806cc30 0x7f707806f0e0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f7084009b80 tx=0x7f708400b040 comp rx=0 tx=0).stop 2024-03-29T08:23:25.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:25.947+0000 7f7097c00700 1 -- 172.21.15.37:0/3831622511 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f70900fc590 msgr2=0x7f70901160e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:25.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:25.947+0000 7f7097c00700 1 --2- 172.21.15.37:0/3831622511 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f70900fc590 0x7f70901160e0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f707c008d80 tx=0x7f707c009e30 comp rx=0 tx=0).stop 2024-03-29T08:23:25.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:25.947+0000 7f7097c00700 1 -- 172.21.15.37:0/3831622511 shutdown_connections 2024-03-29T08:23:25.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:25.947+0000 7f7097c00700 1 --2- 172.21.15.37:0/3831622511 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f707806cc30 0x7f707806f0e0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:25.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:25.947+0000 7f7097c00700 1 --2- 172.21.15.37:0/3831622511 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f70900fbc80 0x7f7090115ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:25.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:25.947+0000 7f7097c00700 1 --2- 172.21.15.37:0/3831622511 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f70900fc590 0x7f70901160e0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:25.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:25.947+0000 7f7097c00700 1 -- 172.21.15.37:0/3831622511 >> 172.21.15.37:0/3831622511 conn(0x7f7090072bc0 msgr2=0x7f7090105b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:23:25.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:25.947+0000 7f7097c00700 1 -- 172.21.15.37:0/3831622511 shutdown_connections 2024-03-29T08:23:25.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:25.947+0000 7f7097c00700 1 -- 172.21.15.37:0/3831622511 wait complete. 2024-03-29T08:23:25.949 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 28 2024-03-29T08:23:26.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:26.085+0000 7f494cc00700 1 -- 172.21.15.37:0/1903056863 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f49480fde90 msgr2=0x7f49480fe260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:26.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:26.085+0000 7f494cc00700 1 --2- 172.21.15.37:0/1903056863 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f49480fde90 0x7f49480fe260 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f4938004940 tx=0x7f4938009e30 comp rx=0 tx=0).stop 2024-03-29T08:23:26.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:26.085+0000 7f494cc00700 1 -- 172.21.15.37:0/1903056863 shutdown_connections 2024-03-29T08:23:26.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:26.085+0000 7f494cc00700 1 --2- 172.21.15.37:0/1903056863 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f49480fe830 0x7f4948106f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:26.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:26.085+0000 7f494cc00700 1 --2- 172.21.15.37:0/1903056863 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f49480fde90 0x7f49480fe260 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:26.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:26.085+0000 7f494cc00700 1 -- 172.21.15.37:0/1903056863 >> 172.21.15.37:0/1903056863 conn(0x7f4948072bc0 msgr2=0x7f4948072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:23:26.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:26.085+0000 7f494cc00700 1 -- 172.21.15.37:0/1903056863 shutdown_connections 2024-03-29T08:23:26.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:26.085+0000 7f494cc00700 1 -- 172.21.15.37:0/1903056863 wait complete. 2024-03-29T08:23:26.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:26.086+0000 7f494cc00700 1 Processor -- start 2024-03-29T08:23:26.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:26.086+0000 7f494cc00700 1 -- start start 2024-03-29T08:23:26.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:26.087+0000 7f494cc00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f49480fde90 0x7f4948113a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:26.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:26.087+0000 7f494cc00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f49480fe830 0x7f4948113fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:26.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:26.087+0000 7f494cc00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4948114600 con 0x7f49480fde90 2024-03-29T08:23:26.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:26.087+0000 7f494cc00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f494810d9f0 con 0x7f49480fe830 2024-03-29T08:23:26.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:26.087+0000 7f4945e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f49480fde90 0x7f4948113a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:26.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:26.087+0000 7f4945e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f49480fde90 0x7f4948113a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52372/0 (socket says 172.21.15.37:52372) 2024-03-29T08:23:26.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:26.087+0000 7f4945e00700 1 -- 172.21.15.37:0/1741449612 learned_addr learned my addr 172.21.15.37:0/1741449612 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:23:26.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:26.087+0000 7f4945400700 1 --2- 172.21.15.37:0/1741449612 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f49480fe830 0x7f4948113fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:26.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:26.087+0000 7f4945e00700 1 -- 172.21.15.37:0/1741449612 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f49480fe830 msgr2=0x7f4948113fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:26.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:26.087+0000 7f4945e00700 1 --2- 172.21.15.37:0/1741449612 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f49480fe830 0x7f4948113fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:26.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:26.087+0000 7f4945e00700 1 -- 172.21.15.37:0/1741449612 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4938005040 con 0x7f49480fde90 2024-03-29T08:23:26.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:26.088+0000 7f4945e00700 1 --2- 172.21.15.37:0/1741449612 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f49480fde90 0x7f4948113a70 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f49380134a0 tx=0x7f4938009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:23:26.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:26.088+0000 7f493e000700 1 -- 172.21.15.37:0/1741449612 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f493800e070 con 0x7f49480fde90 2024-03-29T08:23:26.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:26.088+0000 7f493e000700 1 -- 172.21.15.37:0/1741449612 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f4938009af0 con 0x7f49480fde90 2024-03-29T08:23:26.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:26.088+0000 7f494cc00700 1 -- 172.21.15.37:0/1741449612 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f494810dc70 con 0x7f49480fde90 2024-03-29T08:23:26.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:26.088+0000 7f493e000700 1 -- 172.21.15.37:0/1741449612 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f49380123c0 con 0x7f49480fde90 2024-03-29T08:23:26.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:26.088+0000 7f494cc00700 1 -- 172.21.15.37:0/1741449612 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f494810e1b0 con 0x7f49480fde90 2024-03-29T08:23:26.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:26.089+0000 7f494cc00700 1 -- 172.21.15.37:0/1741449612 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4948065e30 con 0x7f49480fde90 2024-03-29T08:23:26.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:26.090+0000 7f493e000700 1 -- 172.21.15.37:0/1741449612 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f4938013950 con 0x7f49480fde90 2024-03-29T08:23:26.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:26.091+0000 7f493e000700 1 --2- 172.21.15.37:0/1741449612 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f492c06cc20 0x7f492c06f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:26.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:26.091+0000 7f493e000700 1 -- 172.21.15.37:0/1741449612 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f493808d620 con 0x7f49480fde90 2024-03-29T08:23:26.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:26.091+0000 7f4945400700 1 --2- 172.21.15.37:0/1741449612 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f492c06cc20 0x7f492c06f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:26.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:26.092+0000 7f4945400700 1 --2- 172.21.15.37:0/1741449612 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f492c06cc20 0x7f492c06f0d0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f4934004600 tx=0x7f493400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:23:26.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:26.095+0000 7f493e000700 1 -- 172.21.15.37:0/1741449612 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f493805b0f0 con 0x7f49480fde90 2024-03-29T08:23:26.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:26.327+0000 7f494cc00700 1 -- 172.21.15.37:0/1741449612 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4948109e10 con 0x7f492c06cc20 2024-03-29T08:23:26.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:26.329+0000 7f493e000700 1 -- 172.21.15.37:0/1741449612 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f4948109e10 con 0x7f492c06cc20 2024-03-29T08:23:26.329 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-29T08:23:26.329 INFO:teuthology.orchestra.run.smithi037.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59", 2024-03-29T08:23:26.329 INFO:teuthology.orchestra.run.smithi037.stdout: "in_progress": true, 2024-03-29T08:23:26.329 INFO:teuthology.orchestra.run.smithi037.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-29T08:23:26.329 INFO:teuthology.orchestra.run.smithi037.stdout: "services_complete": [ 2024-03-29T08:23:26.329 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr", 2024-03-29T08:23:26.329 INFO:teuthology.orchestra.run.smithi037.stdout: "mon", 2024-03-29T08:23:26.329 INFO:teuthology.orchestra.run.smithi037.stdout: "crash", 2024-03-29T08:23:26.330 INFO:teuthology.orchestra.run.smithi037.stdout: "osd" 2024-03-29T08:23:26.330 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-29T08:23:26.330 INFO:teuthology.orchestra.run.smithi037.stdout: "progress": "15/21 daemons upgraded", 2024-03-29T08:23:26.330 INFO:teuthology.orchestra.run.smithi037.stdout: "message": "Currently upgrading mds daemons", 2024-03-29T08:23:26.330 INFO:teuthology.orchestra.run.smithi037.stdout: "is_paused": false 2024-03-29T08:23:26.330 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-29T08:23:26.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:26.331+0000 7f494cc00700 1 -- 172.21.15.37:0/1741449612 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f492c06cc20 msgr2=0x7f492c06f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:26.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:26.331+0000 7f494cc00700 1 --2- 172.21.15.37:0/1741449612 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f492c06cc20 0x7f492c06f0d0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f4934004600 tx=0x7f493400b040 comp rx=0 tx=0).stop 2024-03-29T08:23:26.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:26.331+0000 7f494cc00700 1 -- 172.21.15.37:0/1741449612 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f49480fde90 msgr2=0x7f4948113a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:26.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:26.331+0000 7f494cc00700 1 --2- 172.21.15.37:0/1741449612 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f49480fde90 0x7f4948113a70 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f49380134a0 tx=0x7f4938009e30 comp rx=0 tx=0).stop 2024-03-29T08:23:26.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:26.331+0000 7f494cc00700 1 -- 172.21.15.37:0/1741449612 shutdown_connections 2024-03-29T08:23:26.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:26.331+0000 7f494cc00700 1 --2- 172.21.15.37:0/1741449612 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f492c06cc20 0x7f492c06f0d0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:26.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:26.331+0000 7f494cc00700 1 --2- 172.21.15.37:0/1741449612 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f49480fde90 0x7f4948113a70 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:26.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:26.331+0000 7f494cc00700 1 --2- 172.21.15.37:0/1741449612 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f49480fe830 0x7f4948113fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:26.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:26.331+0000 7f494cc00700 1 -- 172.21.15.37:0/1741449612 >> 172.21.15.37:0/1741449612 conn(0x7f4948072bc0 msgr2=0x7f49481037c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:23:26.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:26.331+0000 7f494cc00700 1 -- 172.21.15.37:0/1741449612 shutdown_connections 2024-03-29T08:23:26.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:26.331+0000 7f494cc00700 1 -- 172.21.15.37:0/1741449612 wait complete. 2024-03-29T08:23:26.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:26.467+0000 7f62c7e00700 1 -- 172.21.15.37:0/2019573324 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f62c01037a0 msgr2=0x7f62c00fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:26.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:26.467+0000 7f62c7e00700 1 --2- 172.21.15.37:0/2019573324 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f62c01037a0 0x7f62c00fce10 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f62ac004940 tx=0x7f62ac009e30 comp rx=0 tx=0).stop 2024-03-29T08:23:26.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:26.467+0000 7f62c7e00700 1 -- 172.21.15.37:0/2019573324 shutdown_connections 2024-03-29T08:23:26.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:26.467+0000 7f62c7e00700 1 --2- 172.21.15.37:0/2019573324 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f62c01037a0 0x7f62c00fce10 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:26.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:26.467+0000 7f62c7e00700 1 --2- 172.21.15.37:0/2019573324 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f62c0102e00 0x7f62c01031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:26.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:26.467+0000 7f62c7e00700 1 -- 172.21.15.37:0/2019573324 >> 172.21.15.37:0/2019573324 conn(0x7f62c00f89b0 msgr2=0x7f62c00fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:23:26.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:26.467+0000 7f62c7e00700 1 -- 172.21.15.37:0/2019573324 shutdown_connections 2024-03-29T08:23:26.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:26.468+0000 7f62c7e00700 1 -- 172.21.15.37:0/2019573324 wait complete. 2024-03-29T08:23:26.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:26.468+0000 7f62c7e00700 1 Processor -- start 2024-03-29T08:23:26.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:26.468+0000 7f62c7e00700 1 -- start start 2024-03-29T08:23:26.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:26.469+0000 7f62c7e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f62c0102e00 0x7f62c0113850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:26.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:26.469+0000 7f62c7e00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f62c01037a0 0x7f62c0113d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:26.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:26.469+0000 7f62c7e00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62c01143e0 con 0x7f62c0102e00 2024-03-29T08:23:26.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:26.469+0000 7f62c7e00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62c010d9e0 con 0x7f62c01037a0 2024-03-29T08:23:26.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:26.469+0000 7f62c5400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f62c0102e00 0x7f62c0113850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:26.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:26.469+0000 7f62c5400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f62c0102e00 0x7f62c0113850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52384/0 (socket says 172.21.15.37:52384) 2024-03-29T08:23:26.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:26.469+0000 7f62c5400700 1 -- 172.21.15.37:0/1382726712 learned_addr learned my addr 172.21.15.37:0/1382726712 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:23:26.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:26.469+0000 7f62c4a00700 1 --2- 172.21.15.37:0/1382726712 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f62c01037a0 0x7f62c0113d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:26.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:26.470+0000 7f62c5400700 1 -- 172.21.15.37:0/1382726712 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f62c01037a0 msgr2=0x7f62c0113d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:26.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:26.470+0000 7f62c5400700 1 --2- 172.21.15.37:0/1382726712 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f62c01037a0 0x7f62c0113d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:26.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:26.470+0000 7f62c5400700 1 -- 172.21.15.37:0/1382726712 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62ac005040 con 0x7f62c0102e00 2024-03-29T08:23:26.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:26.470+0000 7f62c5400700 1 --2- 172.21.15.37:0/1382726712 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f62c0102e00 0x7f62c0113850 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f62b400eb40 tx=0x7f62b400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:23:26.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:26.470+0000 7f62bd600700 1 -- 172.21.15.37:0/1382726712 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f62b4007cc0 con 0x7f62c0102e00 2024-03-29T08:23:26.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:26.470+0000 7f62bd600700 1 -- 172.21.15.37:0/1382726712 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f62b400f040 con 0x7f62c0102e00 2024-03-29T08:23:26.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:26.470+0000 7f62c7e00700 1 -- 172.21.15.37:0/1382726712 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62c010dc60 con 0x7f62c0102e00 2024-03-29T08:23:26.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:26.470+0000 7f62bd600700 1 -- 172.21.15.37:0/1382726712 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f62b4009570 con 0x7f62c0102e00 2024-03-29T08:23:26.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:26.470+0000 7f62c7e00700 1 -- 172.21.15.37:0/1382726712 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62c010e220 con 0x7f62c0102e00 2024-03-29T08:23:26.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:26.471+0000 7f62c7e00700 1 -- 172.21.15.37:0/1382726712 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62c00fe550 con 0x7f62c0102e00 2024-03-29T08:23:26.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:26.472+0000 7f62bd600700 1 -- 172.21.15.37:0/1382726712 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f62b40096d0 con 0x7f62c0102e00 2024-03-29T08:23:26.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:26.474+0000 7f62bd600700 1 --2- 172.21.15.37:0/1382726712 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f62a806cc20 0x7f62a806f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:26.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:26.474+0000 7f62bd600700 1 -- 172.21.15.37:0/1382726712 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f62b408bda0 con 0x7f62c0102e00 2024-03-29T08:23:26.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:26.474+0000 7f62c4a00700 1 --2- 172.21.15.37:0/1382726712 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f62a806cc20 0x7f62a806f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:26.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:26.475+0000 7f62c4a00700 1 --2- 172.21.15.37:0/1382726712 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f62a806cc20 0x7f62a806f0d0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f62ac00b0d0 tx=0x7f62ac009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:23:26.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:26.479+0000 7f62bd600700 1 -- 172.21.15.37:0/1382726712 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f62b4059820 con 0x7f62c0102e00 2024-03-29T08:23:26.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:23:26 smithi148 ceph-mon[106692]: from='client.34736 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:23:26.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:23:26 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/1500312551' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:23:26.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:23:26 smithi148 ceph-mon[106692]: pgmap v376: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:23:26.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:23:26 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/3831622511' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T08:23:26.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:23:26 smithi037 ceph-mon[142325]: from='client.34736 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:23:26.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:23:26 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/1500312551' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:23:26.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:23:26 smithi037 ceph-mon[142325]: pgmap v376: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:23:26.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:23:26 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/3831622511' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T08:23:26.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:26.779+0000 7f62c7e00700 1 -- 172.21.15.37:0/1382726712 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f62c0109dd0 con 0x7f62c0102e00 2024-03-29T08:23:26.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:26.779+0000 7f62bd600700 1 -- 172.21.15.37:0/1382726712 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7f62b40593b0 con 0x7f62c0102e00 2024-03-29T08:23:26.779 INFO:teuthology.orchestra.run.smithi037.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-03-29T08:23:26.779 INFO:teuthology.orchestra.run.smithi037.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-03-29T08:23:26.779 INFO:teuthology.orchestra.run.smithi037.stdout: fs cephfs is degraded 2024-03-29T08:23:26.780 INFO:teuthology.orchestra.run.smithi037.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-03-29T08:23:26.780 INFO:teuthology.orchestra.run.smithi037.stdout: fs cephfs is offline because no MDS is active for it. 2024-03-29T08:23:26.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:26.781+0000 7f62c7e00700 1 -- 172.21.15.37:0/1382726712 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f62a806cc20 msgr2=0x7f62a806f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:26.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:26.781+0000 7f62c7e00700 1 --2- 172.21.15.37:0/1382726712 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f62a806cc20 0x7f62a806f0d0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f62ac00b0d0 tx=0x7f62ac009e30 comp rx=0 tx=0).stop 2024-03-29T08:23:26.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:26.781+0000 7f62c7e00700 1 -- 172.21.15.37:0/1382726712 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f62c0102e00 msgr2=0x7f62c0113850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:26.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:26.781+0000 7f62c7e00700 1 --2- 172.21.15.37:0/1382726712 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f62c0102e00 0x7f62c0113850 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f62b400eb40 tx=0x7f62b400b040 comp rx=0 tx=0).stop 2024-03-29T08:23:26.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:26.781+0000 7f62c7e00700 1 -- 172.21.15.37:0/1382726712 shutdown_connections 2024-03-29T08:23:26.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:26.781+0000 7f62c7e00700 1 --2- 172.21.15.37:0/1382726712 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f62a806cc20 0x7f62a806f0d0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:26.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:26.781+0000 7f62c7e00700 1 --2- 172.21.15.37:0/1382726712 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f62c0102e00 0x7f62c0113850 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:26.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:26.781+0000 7f62c7e00700 1 --2- 172.21.15.37:0/1382726712 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f62c01037a0 0x7f62c0113d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:26.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:26.781+0000 7f62c7e00700 1 -- 172.21.15.37:0/1382726712 >> 172.21.15.37:0/1382726712 conn(0x7f62c00f89b0 msgr2=0x7f62c00f99a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:23:26.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:26.782+0000 7f62c7e00700 1 -- 172.21.15.37:0/1382726712 shutdown_connections 2024-03-29T08:23:26.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:26.782+0000 7f62c7e00700 1 -- 172.21.15.37:0/1382726712 wait complete. 2024-03-29T08:23:27.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:23:27 smithi148 ceph-mon[106692]: from='client.34748 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:23:27.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:23:27 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/1382726712' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T08:23:27.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:23:27 smithi037 ceph-mon[142325]: from='client.34748 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:23:27.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:23:27 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/1382726712' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T08:23:28.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:23:28 smithi148 ceph-mon[106692]: pgmap v377: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:23:28.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:23:28 smithi037 ceph-mon[142325]: pgmap v377: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:23:30.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:23:30 smithi148 ceph-mon[106692]: pgmap v378: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:23:30.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:23:30 smithi037 ceph-mon[142325]: pgmap v378: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:23:32.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:23:31 smithi148 ceph-mon[106692]: pgmap v379: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:23:32.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:23:31 smithi037 ceph-mon[142325]: pgmap v379: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:23:34.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:23:33 smithi148 ceph-mon[106692]: pgmap v380: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:23:34.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:23:33 smithi037 ceph-mon[142325]: pgmap v380: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:23:36.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:23:35 smithi148 ceph-mon[106692]: pgmap v381: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:23:36.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:23:35 smithi037 ceph-mon[142325]: pgmap v381: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:23:38.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:23:37 smithi148 ceph-mon[106692]: pgmap v382: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:23:38.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:23:37 smithi037 ceph-mon[142325]: pgmap v382: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:23:40.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:23:40 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:23:40.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:23:40 smithi148 ceph-mon[106692]: pgmap v383: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:23:40.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:23:40 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:23:40.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:23:40 smithi037 ceph-mon[142325]: pgmap v383: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:23:42.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:23:41 smithi148 ceph-mon[106692]: pgmap v384: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:23:42.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:23:41 smithi037 ceph-mon[142325]: pgmap v384: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:23:44.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:23:43 smithi148 ceph-mon[106692]: pgmap v385: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:23:44.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:23:43 smithi037 ceph-mon[142325]: pgmap v385: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:23:46.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:23:45 smithi148 ceph-mon[106692]: pgmap v386: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:23:46.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:23:45 smithi037 ceph-mon[142325]: pgmap v386: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:23:48.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:23:47 smithi148 ceph-mon[106692]: pgmap v387: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:23:48.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:23:47 smithi037 ceph-mon[142325]: pgmap v387: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:23:50.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:23:50 smithi148 ceph-mon[106692]: pgmap v388: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:23:50.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:23:50 smithi037 ceph-mon[142325]: pgmap v388: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:23:52.014 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:23:51 smithi148 ceph-mon[106692]: pgmap v389: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:23:52.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:23:51 smithi037 ceph-mon[142325]: pgmap v389: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:23:54.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:23:53 smithi148 ceph-mon[106692]: pgmap v390: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:23:54.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:23:53 smithi037 ceph-mon[142325]: pgmap v390: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:23:55.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:23:55 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:23:55.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:23:55 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:23:56.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:23:56 smithi148 ceph-mon[106692]: pgmap v391: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:23:56.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:23:56 smithi037 ceph-mon[142325]: pgmap v391: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:23:56.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:56.937+0000 7fc8b8a00700 1 -- 172.21.15.37:0/3855869479 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc8b41037a0 msgr2=0x7fc8b40fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:56.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:56.937+0000 7fc8b8a00700 1 --2- 172.21.15.37:0/3855869479 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc8b41037a0 0x7fc8b40fce10 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fc8a0004940 tx=0x7fc8a0009e30 comp rx=0 tx=0).stop 2024-03-29T08:23:56.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:56.937+0000 7fc8b8a00700 1 -- 172.21.15.37:0/3855869479 shutdown_connections 2024-03-29T08:23:56.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:56.937+0000 7fc8b8a00700 1 --2- 172.21.15.37:0/3855869479 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc8b41037a0 0x7fc8b40fce10 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:56.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:56.937+0000 7fc8b8a00700 1 --2- 172.21.15.37:0/3855869479 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc8b4102e00 0x7fc8b41031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:56.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:56.937+0000 7fc8b8a00700 1 -- 172.21.15.37:0/3855869479 >> 172.21.15.37:0/3855869479 conn(0x7fc8b40f89b0 msgr2=0x7fc8b40fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:23:56.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:56.937+0000 7fc8b8a00700 1 -- 172.21.15.37:0/3855869479 shutdown_connections 2024-03-29T08:23:56.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:56.937+0000 7fc8b8a00700 1 -- 172.21.15.37:0/3855869479 wait complete. 2024-03-29T08:23:56.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:56.938+0000 7fc8b8a00700 1 Processor -- start 2024-03-29T08:23:56.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:56.938+0000 7fc8b8a00700 1 -- start start 2024-03-29T08:23:56.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:56.938+0000 7fc8b8a00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc8b4102e00 0x7fc8b40702c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:56.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:56.938+0000 7fc8b8a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc8b41037a0 0x7fc8b4070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:56.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:56.938+0000 7fc8b8a00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8b4070d90 con 0x7fc8b41037a0 2024-03-29T08:23:56.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:56.938+0000 7fc8b8a00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8b4070ed0 con 0x7fc8b4102e00 2024-03-29T08:23:56.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:56.939+0000 7fc8b1400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc8b41037a0 0x7fc8b4070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:56.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:56.939+0000 7fc8b1400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc8b41037a0 0x7fc8b4070800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:33838/0 (socket says 172.21.15.37:33838) 2024-03-29T08:23:56.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:56.939+0000 7fc8b1400700 1 -- 172.21.15.37:0/2001679720 learned_addr learned my addr 172.21.15.37:0/2001679720 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:23:56.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:56.939+0000 7fc8b1e00700 1 --2- 172.21.15.37:0/2001679720 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc8b4102e00 0x7fc8b40702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:56.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:56.939+0000 7fc8b1400700 1 -- 172.21.15.37:0/2001679720 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc8b4102e00 msgr2=0x7fc8b40702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:56.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:56.939+0000 7fc8b1400700 1 --2- 172.21.15.37:0/2001679720 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc8b4102e00 0x7fc8b40702c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:56.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:56.939+0000 7fc8b1400700 1 -- 172.21.15.37:0/2001679720 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8a0005040 con 0x7fc8b41037a0 2024-03-29T08:23:56.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:56.939+0000 7fc8b1e00700 1 --2- 172.21.15.37:0/2001679720 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc8b4102e00 0x7fc8b40702c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:23:56.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:56.940+0000 7fc8b1400700 1 --2- 172.21.15.37:0/2001679720 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc8b41037a0 0x7fc8b4070800 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fc8a0009390 tx=0x7fc8a0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:23:56.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:56.940+0000 7fc8aa000700 1 -- 172.21.15.37:0/2001679720 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc8a000e040 con 0x7fc8b41037a0 2024-03-29T08:23:56.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:56.940+0000 7fc8aa000700 1 -- 172.21.15.37:0/2001679720 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fc8a0013420 con 0x7fc8b41037a0 2024-03-29T08:23:56.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:56.940+0000 7fc8aa000700 1 -- 172.21.15.37:0/2001679720 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc8a001c630 con 0x7fc8b41037a0 2024-03-29T08:23:56.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:56.940+0000 7fc8b8a00700 1 -- 172.21.15.37:0/2001679720 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8b4071120 con 0x7fc8b41037a0 2024-03-29T08:23:56.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:56.940+0000 7fc8b8a00700 1 -- 172.21.15.37:0/2001679720 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8b411a230 con 0x7fc8b41037a0 2024-03-29T08:23:56.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:56.941+0000 7fc8b8a00700 1 -- 172.21.15.37:0/2001679720 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8b40fe550 con 0x7fc8b41037a0 2024-03-29T08:23:56.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:56.942+0000 7fc8aa000700 1 -- 172.21.15.37:0/2001679720 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fc8a0013a00 con 0x7fc8b41037a0 2024-03-29T08:23:56.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:56.943+0000 7fc8aa000700 1 --2- 172.21.15.37:0/2001679720 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fc89806cc30 0x7fc89806f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:56.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:56.943+0000 7fc8aa000700 1 -- 172.21.15.37:0/2001679720 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fc8a008d370 con 0x7fc8b41037a0 2024-03-29T08:23:56.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:56.943+0000 7fc8b1e00700 1 --2- 172.21.15.37:0/2001679720 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fc89806cc30 0x7fc89806f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:56.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:56.944+0000 7fc8b1e00700 1 --2- 172.21.15.37:0/2001679720 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fc89806cc30 0x7fc89806f0e0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fc8a4009fd0 tx=0x7fc8a4009ee0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:23:56.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:56.948+0000 7fc8aa000700 1 -- 172.21.15.37:0/2001679720 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fc8a0057330 con 0x7fc8b41037a0 2024-03-29T08:23:57.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:57.181+0000 7fc8b8a00700 1 -- 172.21.15.37:0/2001679720 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc8b4109e10 con 0x7fc89806cc30 2024-03-29T08:23:57.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:57.182+0000 7fc8aa000700 1 -- 172.21.15.37:0/2001679720 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fc8b4109e10 con 0x7fc89806cc30 2024-03-29T08:23:57.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:57.184+0000 7fc8b8a00700 1 -- 172.21.15.37:0/2001679720 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fc89806cc30 msgr2=0x7fc89806f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:57.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:57.184+0000 7fc8b8a00700 1 --2- 172.21.15.37:0/2001679720 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fc89806cc30 0x7fc89806f0e0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fc8a4009fd0 tx=0x7fc8a4009ee0 comp rx=0 tx=0).stop 2024-03-29T08:23:57.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:57.184+0000 7fc8b8a00700 1 -- 172.21.15.37:0/2001679720 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc8b41037a0 msgr2=0x7fc8b4070800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:57.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:57.184+0000 7fc8b8a00700 1 --2- 172.21.15.37:0/2001679720 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc8b41037a0 0x7fc8b4070800 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fc8a0009390 tx=0x7fc8a0009e30 comp rx=0 tx=0).stop 2024-03-29T08:23:57.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:57.184+0000 7fc8b8a00700 1 -- 172.21.15.37:0/2001679720 shutdown_connections 2024-03-29T08:23:57.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:57.185+0000 7fc8b8a00700 1 --2- 172.21.15.37:0/2001679720 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fc89806cc30 0x7fc89806f0e0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:57.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:57.185+0000 7fc8b8a00700 1 --2- 172.21.15.37:0/2001679720 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc8b4102e00 0x7fc8b40702c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:57.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:57.185+0000 7fc8b8a00700 1 --2- 172.21.15.37:0/2001679720 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc8b41037a0 0x7fc8b4070800 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:57.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:57.185+0000 7fc8b8a00700 1 -- 172.21.15.37:0/2001679720 >> 172.21.15.37:0/2001679720 conn(0x7fc8b40f89b0 msgr2=0x7fc8b40f9ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:23:57.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:57.185+0000 7fc8b8a00700 1 -- 172.21.15.37:0/2001679720 shutdown_connections 2024-03-29T08:23:57.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:57.185+0000 7fc8b8a00700 1 -- 172.21.15.37:0/2001679720 wait complete. 2024-03-29T08:23:57.200 INFO:teuthology.orchestra.run.smithi037.stdout:true 2024-03-29T08:23:57.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:57.323+0000 7fa136800700 1 -- 172.21.15.37:0/3998478287 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa1300fe830 msgr2=0x7fa130106f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:57.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:57.323+0000 7fa136800700 1 --2- 172.21.15.37:0/3998478287 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa1300fe830 0x7fa130106f00 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fa11c004940 tx=0x7fa11c009e30 comp rx=0 tx=0).stop 2024-03-29T08:23:57.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:57.323+0000 7fa136800700 1 -- 172.21.15.37:0/3998478287 shutdown_connections 2024-03-29T08:23:57.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:57.323+0000 7fa136800700 1 --2- 172.21.15.37:0/3998478287 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa1300fe830 0x7fa130106f00 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:57.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:57.323+0000 7fa136800700 1 --2- 172.21.15.37:0/3998478287 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa1300fde90 0x7fa1300fe260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:57.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:57.323+0000 7fa136800700 1 -- 172.21.15.37:0/3998478287 >> 172.21.15.37:0/3998478287 conn(0x7fa130072bc0 msgr2=0x7fa130072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:23:57.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:57.323+0000 7fa136800700 1 -- 172.21.15.37:0/3998478287 shutdown_connections 2024-03-29T08:23:57.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:57.324+0000 7fa136800700 1 -- 172.21.15.37:0/3998478287 wait complete. 2024-03-29T08:23:57.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:57.324+0000 7fa136800700 1 Processor -- start 2024-03-29T08:23:57.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:57.324+0000 7fa136800700 1 -- start start 2024-03-29T08:23:57.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:57.325+0000 7fa136800700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa1300fde90 0x7fa130115ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:57.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:57.325+0000 7fa136800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa1300fe830 0x7fa1301161e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:57.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:57.325+0000 7fa136800700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa130116830 con 0x7fa1300fe830 2024-03-29T08:23:57.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:57.325+0000 7fa136800700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa13010fc50 con 0x7fa1300fde90 2024-03-29T08:23:57.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:57.325+0000 7fa12ea00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa1300fe830 0x7fa1301161e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:57.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:57.325+0000 7fa12ea00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa1300fe830 0x7fa1301161e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:33854/0 (socket says 172.21.15.37:33854) 2024-03-29T08:23:57.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:57.325+0000 7fa12ea00700 1 -- 172.21.15.37:0/153896362 learned_addr learned my addr 172.21.15.37:0/153896362 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:23:57.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:57.325+0000 7fa12f400700 1 --2- 172.21.15.37:0/153896362 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa1300fde90 0x7fa130115ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:57.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:57.325+0000 7fa12ea00700 1 -- 172.21.15.37:0/153896362 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa1300fde90 msgr2=0x7fa130115ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:57.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:57.325+0000 7fa12ea00700 1 --2- 172.21.15.37:0/153896362 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa1300fde90 0x7fa130115ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:57.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:57.325+0000 7fa12ea00700 1 -- 172.21.15.37:0/153896362 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa11c005040 con 0x7fa1300fe830 2024-03-29T08:23:57.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:57.326+0000 7fa12ea00700 1 --2- 172.21.15.37:0/153896362 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa1300fe830 0x7fa1301161e0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fa11c000c00 tx=0x7fa11c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:23:57.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:57.326+0000 7fa127400700 1 -- 172.21.15.37:0/153896362 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa11c021070 con 0x7fa1300fe830 2024-03-29T08:23:57.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:57.326+0000 7fa127400700 1 -- 172.21.15.37:0/153896362 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fa11c013e00 con 0x7fa1300fe830 2024-03-29T08:23:57.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:57.326+0000 7fa136800700 1 -- 172.21.15.37:0/153896362 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa13010fed0 con 0x7fa1300fe830 2024-03-29T08:23:57.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:57.326+0000 7fa127400700 1 -- 172.21.15.37:0/153896362 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa11c00e070 con 0x7fa1300fe830 2024-03-29T08:23:57.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:57.326+0000 7fa136800700 1 -- 172.21.15.37:0/153896362 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa130110410 con 0x7fa1300fe830 2024-03-29T08:23:57.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:57.327+0000 7fa136800700 1 -- 172.21.15.37:0/153896362 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa130106a20 con 0x7fa1300fe830 2024-03-29T08:23:57.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:57.328+0000 7fa127400700 1 -- 172.21.15.37:0/153896362 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fa11c013950 con 0x7fa1300fe830 2024-03-29T08:23:57.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:57.329+0000 7fa127400700 1 --2- 172.21.15.37:0/153896362 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fa11406cc30 0x7fa11406f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:57.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:57.329+0000 7fa127400700 1 -- 172.21.15.37:0/153896362 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fa11c08d120 con 0x7fa1300fe830 2024-03-29T08:23:57.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:57.329+0000 7fa12f400700 1 --2- 172.21.15.37:0/153896362 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fa11406cc30 0x7fa11406f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:57.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:57.330+0000 7fa12f400700 1 --2- 172.21.15.37:0/153896362 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fa11406cc30 0x7fa11406f0e0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fa120004600 tx=0x7fa12000b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:23:57.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:57.334+0000 7fa127400700 1 -- 172.21.15.37:0/153896362 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fa11c0570e0 con 0x7fa1300fe830 2024-03-29T08:23:57.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:57.568+0000 7fa136800700 1 -- 172.21.15.37:0/153896362 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa1301112f0 con 0x7fa11406cc30 2024-03-29T08:23:57.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:57.569+0000 7fa127400700 1 -- 172.21.15.37:0/153896362 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fa1301112f0 con 0x7fa11406cc30 2024-03-29T08:23:57.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:57.572+0000 7fa136800700 1 -- 172.21.15.37:0/153896362 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fa11406cc30 msgr2=0x7fa11406f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:57.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:57.572+0000 7fa136800700 1 --2- 172.21.15.37:0/153896362 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fa11406cc30 0x7fa11406f0e0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fa120004600 tx=0x7fa12000b040 comp rx=0 tx=0).stop 2024-03-29T08:23:57.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:57.572+0000 7fa136800700 1 -- 172.21.15.37:0/153896362 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa1300fe830 msgr2=0x7fa1301161e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:57.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:57.572+0000 7fa136800700 1 --2- 172.21.15.37:0/153896362 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa1300fe830 0x7fa1301161e0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fa11c000c00 tx=0x7fa11c009e30 comp rx=0 tx=0).stop 2024-03-29T08:23:57.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:57.572+0000 7fa136800700 1 -- 172.21.15.37:0/153896362 shutdown_connections 2024-03-29T08:23:57.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:57.572+0000 7fa136800700 1 --2- 172.21.15.37:0/153896362 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fa11406cc30 0x7fa11406f0e0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:57.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:57.572+0000 7fa136800700 1 --2- 172.21.15.37:0/153896362 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa1300fde90 0x7fa130115ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:57.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:57.572+0000 7fa136800700 1 --2- 172.21.15.37:0/153896362 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa1300fe830 0x7fa1301161e0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:57.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:57.572+0000 7fa136800700 1 -- 172.21.15.37:0/153896362 >> 172.21.15.37:0/153896362 conn(0x7fa130072bc0 msgr2=0x7fa1300fd510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:23:57.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:57.572+0000 7fa136800700 1 -- 172.21.15.37:0/153896362 shutdown_connections 2024-03-29T08:23:57.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:57.572+0000 7fa136800700 1 -- 172.21.15.37:0/153896362 wait complete. 2024-03-29T08:23:57.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:57.710+0000 7fc483400700 1 -- 172.21.15.37:0/4041152177 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc47c0fe830 msgr2=0x7fc47c106f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:57.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:57.710+0000 7fc483400700 1 --2- 172.21.15.37:0/4041152177 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc47c0fe830 0x7fc47c106f00 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fc468004940 tx=0x7fc468009e30 comp rx=0 tx=0).stop 2024-03-29T08:23:57.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:57.710+0000 7fc483400700 1 -- 172.21.15.37:0/4041152177 shutdown_connections 2024-03-29T08:23:57.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:57.710+0000 7fc483400700 1 --2- 172.21.15.37:0/4041152177 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc47c0fe830 0x7fc47c106f00 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:57.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:57.710+0000 7fc483400700 1 --2- 172.21.15.37:0/4041152177 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc47c0fde90 0x7fc47c0fe260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:57.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:57.710+0000 7fc483400700 1 -- 172.21.15.37:0/4041152177 >> 172.21.15.37:0/4041152177 conn(0x7fc47c072bc0 msgr2=0x7fc47c072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:23:57.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:57.711+0000 7fc483400700 1 -- 172.21.15.37:0/4041152177 shutdown_connections 2024-03-29T08:23:57.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:57.711+0000 7fc483400700 1 -- 172.21.15.37:0/4041152177 wait complete. 2024-03-29T08:23:57.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:57.711+0000 7fc483400700 1 Processor -- start 2024-03-29T08:23:57.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:57.711+0000 7fc483400700 1 -- start start 2024-03-29T08:23:57.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:57.712+0000 7fc483400700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc47c0fde90 0x7fc47c113b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:57.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:57.712+0000 7fc483400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc47c0fe830 0x7fc47c114040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:57.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:57.712+0000 7fc483400700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc47c114690 con 0x7fc47c0fe830 2024-03-29T08:23:57.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:57.712+0000 7fc483400700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc47c10da80 con 0x7fc47c0fde90 2024-03-29T08:23:57.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:57.712+0000 7fc47be00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc47c0fe830 0x7fc47c114040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:57.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:57.712+0000 7fc47be00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc47c0fe830 0x7fc47c114040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:33878/0 (socket says 172.21.15.37:33878) 2024-03-29T08:23:57.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:57.712+0000 7fc47be00700 1 -- 172.21.15.37:0/3118162783 learned_addr learned my addr 172.21.15.37:0/3118162783 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:23:57.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:57.712+0000 7fc480a00700 1 --2- 172.21.15.37:0/3118162783 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc47c0fde90 0x7fc47c113b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:57.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:57.713+0000 7fc47be00700 1 -- 172.21.15.37:0/3118162783 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc47c0fde90 msgr2=0x7fc47c113b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:57.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:57.713+0000 7fc47be00700 1 --2- 172.21.15.37:0/3118162783 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc47c0fde90 0x7fc47c113b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:57.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:57.713+0000 7fc47be00700 1 -- 172.21.15.37:0/3118162783 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc468005040 con 0x7fc47c0fe830 2024-03-29T08:23:57.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:57.713+0000 7fc47be00700 1 --2- 172.21.15.37:0/3118162783 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc47c0fe830 0x7fc47c114040 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fc468000c00 tx=0x7fc468009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:23:57.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:57.713+0000 7fc478c00700 1 -- 172.21.15.37:0/3118162783 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc46800e040 con 0x7fc47c0fe830 2024-03-29T08:23:57.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:57.713+0000 7fc478c00700 1 -- 172.21.15.37:0/3118162783 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fc46800bed0 con 0x7fc47c0fe830 2024-03-29T08:23:57.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:57.713+0000 7fc483400700 1 -- 172.21.15.37:0/3118162783 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc47c10dd00 con 0x7fc47c0fe830 2024-03-29T08:23:57.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:57.713+0000 7fc478c00700 1 -- 172.21.15.37:0/3118162783 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc46801c3f0 con 0x7fc47c0fe830 2024-03-29T08:23:57.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:57.713+0000 7fc483400700 1 -- 172.21.15.37:0/3118162783 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc47c10e240 con 0x7fc47c0fe830 2024-03-29T08:23:57.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:57.714+0000 7fc483400700 1 -- 172.21.15.37:0/3118162783 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc47c065e30 con 0x7fc47c0fe830 2024-03-29T08:23:57.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:57.715+0000 7fc478c00700 1 -- 172.21.15.37:0/3118162783 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fc468013420 con 0x7fc47c0fe830 2024-03-29T08:23:57.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:57.716+0000 7fc478c00700 1 --2- 172.21.15.37:0/3118162783 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fc46406cc30 0x7fc46406f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:57.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:57.716+0000 7fc478c00700 1 -- 172.21.15.37:0/3118162783 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fc46808d400 con 0x7fc47c0fe830 2024-03-29T08:23:57.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:57.716+0000 7fc480a00700 1 --2- 172.21.15.37:0/3118162783 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fc46406cc30 0x7fc46406f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:57.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:57.717+0000 7fc480a00700 1 --2- 172.21.15.37:0/3118162783 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fc46406cc30 0x7fc46406f0e0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fc470004f60 tx=0x7fc47000b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:23:57.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:57.721+0000 7fc478c00700 1 -- 172.21.15.37:0/3118162783 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fc46805aed0 con 0x7fc47c0fe830 2024-03-29T08:23:57.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:57.957+0000 7fc483400700 1 -- 172.21.15.37:0/3118162783 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fc47c109df0 con 0x7fc46406cc30 2024-03-29T08:23:57.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:57.967+0000 7fc478c00700 1 -- 172.21.15.37:0/3118162783 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7fc47c109df0 con 0x7fc46406cc30 2024-03-29T08:23:57.967 INFO:teuthology.orchestra.run.smithi037.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-29T08:23:57.967 INFO:teuthology.orchestra.run.smithi037.stdout:alertmanager.smithi037 smithi037 *:9093,9094 running (13m) 6m ago 26m 14.9M - 0.25.0 c8568f914cd2 2f9f9746ed0f 2024-03-29T08:23:57.968 INFO:teuthology.orchestra.run.smithi037.stdout:crash.smithi037 smithi037 running (10m) 6m ago 27m 6647k - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 178d416a3186 2024-03-29T08:23:57.968 INFO:teuthology.orchestra.run.smithi037.stdout:crash.smithi148 smithi148 running (10m) 5m ago 25m 6656k - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 23666989d58f 2024-03-29T08:23:57.968 INFO:teuthology.orchestra.run.smithi037.stdout:grafana.smithi037 smithi037 *:3000 running (13m) 6m ago 26m 71.9M - 9.4.7 954c08fa6188 6b976fe49952 2024-03-29T08:23:57.968 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi037.iifsmh smithi037 running (6m) 6m ago 20m 10.8M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 2d97f562cc5b 2024-03-29T08:23:57.968 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi037.mmwgxe smithi037 running (6m) 6m ago 20m 10.9M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff ec37037f46e8 2024-03-29T08:23:57.968 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi148.qukpjh smithi148 running (20m) 5m ago 20m 15.3M - 17.2.7-789-g4213e17c cf22eb8683be 8e469782a273 2024-03-29T08:23:57.968 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi148.sqteih smithi148 running (5m) 5m ago 20m 10.7M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 44c15c77037a 2024-03-29T08:23:57.968 INFO:teuthology.orchestra.run.smithi037.stdout:mgr.smithi037.wjikii smithi037 *:8443,9283,8765 running (16m) 6m ago 28m 506M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff c5a1417b9417 2024-03-29T08:23:57.968 INFO:teuthology.orchestra.run.smithi037.stdout:mgr.smithi148.onhgvm smithi148 *:8443,9283,8765 running (14m) 5m ago 25m 450M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff fcfdbce062bd 2024-03-29T08:23:57.968 INFO:teuthology.orchestra.run.smithi037.stdout:mon.smithi037 smithi037 running (11m) 6m ago 28m 54.8M 2048M 18.2.2-1015-gd0e392d9 1cc2f45c17ff eea2060783af 2024-03-29T08:23:57.968 INFO:teuthology.orchestra.run.smithi037.stdout:mon.smithi148 smithi148 running (11m) 5m ago 25m 43.7M 2048M 18.2.2-1015-gd0e392d9 1cc2f45c17ff ad4b2c5748be 2024-03-29T08:23:57.968 INFO:teuthology.orchestra.run.smithi037.stdout:node-exporter.smithi037 smithi037 *:9100 running (14m) 6m ago 26m 7625k - 1.5.0 0da6a335fe13 26b8ae9cb878 2024-03-29T08:23:57.968 INFO:teuthology.orchestra.run.smithi037.stdout:node-exporter.smithi148 smithi148 *:9100 running (14m) 5m ago 25m 8166k - 1.5.0 0da6a335fe13 98414e4e64ed 2024-03-29T08:23:57.968 INFO:teuthology.orchestra.run.smithi037.stdout:osd.0 smithi037 running (10m) 6m ago 24m 104M 1619M 18.2.2-1015-gd0e392d9 1cc2f45c17ff c0c584f6245c 2024-03-29T08:23:57.968 INFO:teuthology.orchestra.run.smithi037.stdout:osd.1 smithi037 running (9m) 6m ago 24m 100M 1619M 18.2.2-1015-gd0e392d9 1cc2f45c17ff a4a2ff4cc83d 2024-03-29T08:23:57.969 INFO:teuthology.orchestra.run.smithi037.stdout:osd.2 smithi037 running (8m) 6m ago 23m 82.3M 1619M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 8b1bc71aa79e 2024-03-29T08:23:57.969 INFO:teuthology.orchestra.run.smithi037.stdout:osd.3 smithi148 running (8m) 5m ago 23m 105M 2642M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 1c656ab8ff86 2024-03-29T08:23:57.969 INFO:teuthology.orchestra.run.smithi037.stdout:osd.4 smithi148 running (7m) 5m ago 22m 97.3M 2642M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 966a9de11d22 2024-03-29T08:23:57.969 INFO:teuthology.orchestra.run.smithi037.stdout:osd.5 smithi148 running (6m) 5m ago 21m 85.8M 2642M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 474c4f9212ce 2024-03-29T08:23:57.969 INFO:teuthology.orchestra.run.smithi037.stdout:prometheus.smithi037 smithi037 *:9095 running (14m) 6m ago 26m 54.4M - 2.43.0 a07b618ecd1d 30c0223ec589 2024-03-29T08:23:57.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:57.969+0000 7fc483400700 1 -- 172.21.15.37:0/3118162783 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fc46406cc30 msgr2=0x7fc46406f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:57.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:57.969+0000 7fc483400700 1 --2- 172.21.15.37:0/3118162783 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fc46406cc30 0x7fc46406f0e0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fc470004f60 tx=0x7fc47000b040 comp rx=0 tx=0).stop 2024-03-29T08:23:57.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:57.969+0000 7fc483400700 1 -- 172.21.15.37:0/3118162783 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc47c0fe830 msgr2=0x7fc47c114040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:57.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:57.969+0000 7fc483400700 1 --2- 172.21.15.37:0/3118162783 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc47c0fe830 0x7fc47c114040 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fc468000c00 tx=0x7fc468009e30 comp rx=0 tx=0).stop 2024-03-29T08:23:57.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:57.969+0000 7fc483400700 1 -- 172.21.15.37:0/3118162783 shutdown_connections 2024-03-29T08:23:57.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:57.969+0000 7fc483400700 1 --2- 172.21.15.37:0/3118162783 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fc46406cc30 0x7fc46406f0e0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:57.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:57.969+0000 7fc483400700 1 --2- 172.21.15.37:0/3118162783 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc47c0fde90 0x7fc47c113b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:57.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:57.969+0000 7fc483400700 1 --2- 172.21.15.37:0/3118162783 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc47c0fe830 0x7fc47c114040 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:57.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:57.969+0000 7fc483400700 1 -- 172.21.15.37:0/3118162783 >> 172.21.15.37:0/3118162783 conn(0x7fc47c072bc0 msgr2=0x7fc47c103860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:23:57.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:57.970+0000 7fc483400700 1 -- 172.21.15.37:0/3118162783 shutdown_connections 2024-03-29T08:23:57.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:57.970+0000 7fc483400700 1 -- 172.21.15.37:0/3118162783 wait complete. 2024-03-29T08:23:58.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:23:57 smithi148 ceph-mon[106692]: from='client.34756 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:23:58.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:23:57 smithi148 ceph-mon[106692]: from='client.34760 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:23:58.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:23:57 smithi148 ceph-mon[106692]: pgmap v392: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:23:58.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:58.105+0000 7f19f4000700 1 -- 172.21.15.37:0/2250845128 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f19ec0fde90 msgr2=0x7f19ec0fe260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:58.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:58.105+0000 7f19f4000700 1 --2- 172.21.15.37:0/2250845128 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f19ec0fde90 0x7f19ec0fe260 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f19e0004940 tx=0x7f19e0009e30 comp rx=0 tx=0).stop 2024-03-29T08:23:58.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:58.105+0000 7f19f4000700 1 -- 172.21.15.37:0/2250845128 shutdown_connections 2024-03-29T08:23:58.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:58.105+0000 7f19f4000700 1 --2- 172.21.15.37:0/2250845128 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f19ec0fe830 0x7f19ec106f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:58.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:58.105+0000 7f19f4000700 1 --2- 172.21.15.37:0/2250845128 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f19ec0fde90 0x7f19ec0fe260 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:58.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:58.105+0000 7f19f4000700 1 -- 172.21.15.37:0/2250845128 >> 172.21.15.37:0/2250845128 conn(0x7f19ec072bc0 msgr2=0x7f19ec072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:23:58.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:58.105+0000 7f19f4000700 1 -- 172.21.15.37:0/2250845128 shutdown_connections 2024-03-29T08:23:58.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:58.105+0000 7f19f4000700 1 -- 172.21.15.37:0/2250845128 wait complete. 2024-03-29T08:23:58.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:58.106+0000 7f19f4000700 1 Processor -- start 2024-03-29T08:23:58.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:58.106+0000 7f19f4000700 1 -- start start 2024-03-29T08:23:58.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:58.107+0000 7f19f4000700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f19ec0fde90 0x7f19ec113af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:58.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:58.107+0000 7f19f4000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f19ec0fe830 0x7f19ec114030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:58.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:58.107+0000 7f19f4000700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19ec114680 con 0x7f19ec0fe830 2024-03-29T08:23:58.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:58.107+0000 7f19f4000700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19ec10da70 con 0x7f19ec0fde90 2024-03-29T08:23:58.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:58.107+0000 7f19f0c00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f19ec0fe830 0x7f19ec114030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:58.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:58.107+0000 7f19f0c00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f19ec0fe830 0x7f19ec114030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:33898/0 (socket says 172.21.15.37:33898) 2024-03-29T08:23:58.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:58.107+0000 7f19f0c00700 1 -- 172.21.15.37:0/1283137773 learned_addr learned my addr 172.21.15.37:0/1283137773 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:23:58.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:58.107+0000 7f19f1600700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f19ec0fde90 0x7f19ec113af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:58.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:58.107+0000 7f19f0c00700 1 -- 172.21.15.37:0/1283137773 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f19ec0fde90 msgr2=0x7f19ec113af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:58.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:58.107+0000 7f19f0c00700 1 --2- 172.21.15.37:0/1283137773 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f19ec0fde90 0x7f19ec113af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:58.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:58.107+0000 7f19f0c00700 1 -- 172.21.15.37:0/1283137773 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19e0005040 con 0x7f19ec0fe830 2024-03-29T08:23:58.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:58.108+0000 7f19f0c00700 1 --2- 172.21.15.37:0/1283137773 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f19ec0fe830 0x7f19ec114030 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f19d8002a50 tx=0x7f19d80046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:23:58.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:58.108+0000 7f19e9600700 1 -- 172.21.15.37:0/1283137773 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f19d8007cc0 con 0x7f19ec0fe830 2024-03-29T08:23:58.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:58.108+0000 7f19e9600700 1 -- 172.21.15.37:0/1283137773 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f19d800f040 con 0x7f19ec0fe830 2024-03-29T08:23:58.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:58.108+0000 7f19e9600700 1 -- 172.21.15.37:0/1283137773 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f19d800d9c0 con 0x7f19ec0fe830 2024-03-29T08:23:58.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:58.108+0000 7f19f4000700 1 -- 172.21.15.37:0/1283137773 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19ec10dd50 con 0x7f19ec0fe830 2024-03-29T08:23:58.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:58.108+0000 7f19f4000700 1 -- 172.21.15.37:0/1283137773 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19ec10e240 con 0x7f19ec0fe830 2024-03-29T08:23:58.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:58.109+0000 7f19f4000700 1 -- 172.21.15.37:0/1283137773 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19ec1019b0 con 0x7f19ec0fe830 2024-03-29T08:23:58.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:58.110+0000 7f19e9600700 1 -- 172.21.15.37:0/1283137773 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f19d800db20 con 0x7f19ec0fe830 2024-03-29T08:23:58.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:58.111+0000 7f19e9600700 1 --2- 172.21.15.37:0/1283137773 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f19d406cc30 0x7f19d406f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:58.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:58.112+0000 7f19e9600700 1 -- 172.21.15.37:0/1283137773 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f19d808bde0 con 0x7f19ec0fe830 2024-03-29T08:23:58.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:58.112+0000 7f19f1600700 1 --2- 172.21.15.37:0/1283137773 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f19d406cc30 0x7f19d406f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:58.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:58.112+0000 7f19f1600700 1 --2- 172.21.15.37:0/1283137773 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f19d406cc30 0x7f19d406f0e0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f19e000b3b0 tx=0x7f19e0009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:23:58.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:58.116+0000 7f19e9600700 1 -- 172.21.15.37:0/1283137773 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f19d80597b0 con 0x7f19ec0fe830 2024-03-29T08:23:58.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:23:57 smithi037 ceph-mon[142325]: from='client.34756 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:23:58.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:23:57 smithi037 ceph-mon[142325]: from='client.34760 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:23:58.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:23:57 smithi037 ceph-mon[142325]: pgmap v392: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:23:58.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:58.416+0000 7f19f4000700 1 -- 172.21.15.37:0/1283137773 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f19ec10f140 con 0x7f19ec0fe830 2024-03-29T08:23:58.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:58.416+0000 7f19e9600700 1 -- 172.21.15.37:0/1283137773 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+847 (secure 0 0 0) 0x7f19d8059340 con 0x7f19ec0fe830 2024-03-29T08:23:58.416 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-29T08:23:58.417 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": { 2024-03-29T08:23:58.417 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 2 2024-03-29T08:23:58.417 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:23:58.417 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": { 2024-03-29T08:23:58.417 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 2 2024-03-29T08:23:58.417 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:23:58.417 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": { 2024-03-29T08:23:58.417 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 6 2024-03-29T08:23:58.417 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:23:58.417 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": { 2024-03-29T08:23:58.417 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 17.2.7-789-g4213e17c (4213e17c4de67016ed14f55d22503f4923b3a901) quincy (stable)": 1, 2024-03-29T08:23:58.417 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 3 2024-03-29T08:23:58.417 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:23:58.417 INFO:teuthology.orchestra.run.smithi037.stdout: "overall": { 2024-03-29T08:23:58.418 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 17.2.7-789-g4213e17c (4213e17c4de67016ed14f55d22503f4923b3a901) quincy (stable)": 1, 2024-03-29T08:23:58.418 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 13 2024-03-29T08:23:58.418 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-29T08:23:58.418 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-29T08:23:58.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:58.418+0000 7f19f4000700 1 -- 172.21.15.37:0/1283137773 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f19d406cc30 msgr2=0x7f19d406f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:58.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:58.418+0000 7f19f4000700 1 --2- 172.21.15.37:0/1283137773 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f19d406cc30 0x7f19d406f0e0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f19e000b3b0 tx=0x7f19e0009e30 comp rx=0 tx=0).stop 2024-03-29T08:23:58.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:58.419+0000 7f19f4000700 1 -- 172.21.15.37:0/1283137773 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f19ec0fe830 msgr2=0x7f19ec114030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:58.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:58.419+0000 7f19f4000700 1 --2- 172.21.15.37:0/1283137773 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f19ec0fe830 0x7f19ec114030 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f19d8002a50 tx=0x7f19d80046b0 comp rx=0 tx=0).stop 2024-03-29T08:23:58.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:58.419+0000 7f19f4000700 1 -- 172.21.15.37:0/1283137773 shutdown_connections 2024-03-29T08:23:58.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:58.419+0000 7f19f4000700 1 --2- 172.21.15.37:0/1283137773 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f19d406cc30 0x7f19d406f0e0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:58.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:58.419+0000 7f19f4000700 1 --2- 172.21.15.37:0/1283137773 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f19ec0fde90 0x7f19ec113af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:58.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:58.419+0000 7f19f4000700 1 --2- 172.21.15.37:0/1283137773 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f19ec0fe830 0x7f19ec114030 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:58.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:58.419+0000 7f19f4000700 1 -- 172.21.15.37:0/1283137773 >> 172.21.15.37:0/1283137773 conn(0x7f19ec072bc0 msgr2=0x7f19ec0ff3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:23:58.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:58.419+0000 7f19f4000700 1 -- 172.21.15.37:0/1283137773 shutdown_connections 2024-03-29T08:23:58.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:58.419+0000 7f19f4000700 1 -- 172.21.15.37:0/1283137773 wait complete. 2024-03-29T08:23:58.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:58.556+0000 7f2f48e00700 1 -- 172.21.15.37:0/1354568242 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2f44102ca0 msgr2=0x7f2f44103070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:58.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:58.556+0000 7f2f48e00700 1 --2- 172.21.15.37:0/1354568242 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2f44102ca0 0x7f2f44103070 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f2f34004940 tx=0x7f2f34009e30 comp rx=0 tx=0).stop 2024-03-29T08:23:58.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:58.556+0000 7f2f48e00700 1 -- 172.21.15.37:0/1354568242 shutdown_connections 2024-03-29T08:23:58.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:58.556+0000 7f2f48e00700 1 --2- 172.21.15.37:0/1354568242 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2f44065630 0x7f2f44065a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:58.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:58.556+0000 7f2f48e00700 1 --2- 172.21.15.37:0/1354568242 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2f44102ca0 0x7f2f44103070 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:58.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:58.556+0000 7f2f48e00700 1 -- 172.21.15.37:0/1354568242 >> 172.21.15.37:0/1354568242 conn(0x7f2f44071b70 msgr2=0x7f2f44071f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:23:58.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:58.556+0000 7f2f48e00700 1 -- 172.21.15.37:0/1354568242 shutdown_connections 2024-03-29T08:23:58.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:58.556+0000 7f2f48e00700 1 -- 172.21.15.37:0/1354568242 wait complete. 2024-03-29T08:23:58.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:58.557+0000 7f2f48e00700 1 Processor -- start 2024-03-29T08:23:58.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:58.558+0000 7f2f48e00700 1 -- start start 2024-03-29T08:23:58.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:58.558+0000 7f2f48e00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2f44065630 0x7f2f441138a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:58.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:58.558+0000 7f2f48e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2f44102ca0 0x7f2f44113de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:58.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:58.558+0000 7f2f48e00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f44114430 con 0x7f2f44102ca0 2024-03-29T08:23:58.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:58.558+0000 7f2f48e00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f4410d820 con 0x7f2f44065630 2024-03-29T08:23:58.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:58.558+0000 7f2f42a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2f44102ca0 0x7f2f44113de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:58.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:58.558+0000 7f2f42a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2f44102ca0 0x7f2f44113de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:33920/0 (socket says 172.21.15.37:33920) 2024-03-29T08:23:58.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:58.558+0000 7f2f42a00700 1 -- 172.21.15.37:0/2525364359 learned_addr learned my addr 172.21.15.37:0/2525364359 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:23:58.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:58.559+0000 7f2f43400700 1 --2- 172.21.15.37:0/2525364359 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2f44065630 0x7f2f441138a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:58.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:58.559+0000 7f2f42a00700 1 -- 172.21.15.37:0/2525364359 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2f44065630 msgr2=0x7f2f441138a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:58.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:58.559+0000 7f2f42a00700 1 --2- 172.21.15.37:0/2525364359 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2f44065630 0x7f2f441138a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:58.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:58.559+0000 7f2f42a00700 1 -- 172.21.15.37:0/2525364359 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f34005040 con 0x7f2f44102ca0 2024-03-29T08:23:58.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:58.559+0000 7f2f42a00700 1 --2- 172.21.15.37:0/2525364359 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2f44102ca0 0x7f2f44113de0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f2f30002a50 tx=0x7f2f300046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:23:58.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:58.559+0000 7f2f3b400700 1 -- 172.21.15.37:0/2525364359 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2f30007cc0 con 0x7f2f44102ca0 2024-03-29T08:23:58.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:58.559+0000 7f2f3b400700 1 -- 172.21.15.37:0/2525364359 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f2f3000f040 con 0x7f2f44102ca0 2024-03-29T08:23:58.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:58.559+0000 7f2f3b400700 1 -- 172.21.15.37:0/2525364359 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2f30009540 con 0x7f2f44102ca0 2024-03-29T08:23:58.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:58.559+0000 7f2f48e00700 1 -- 172.21.15.37:0/2525364359 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f4410db00 con 0x7f2f44102ca0 2024-03-29T08:23:58.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:58.560+0000 7f2f48e00700 1 -- 172.21.15.37:0/2525364359 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f4410e0c0 con 0x7f2f44102ca0 2024-03-29T08:23:58.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:58.561+0000 7f2f48e00700 1 -- 172.21.15.37:0/2525364359 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f440fe380 con 0x7f2f44102ca0 2024-03-29T08:23:58.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:58.562+0000 7f2f3b400700 1 -- 172.21.15.37:0/2525364359 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f2f300096a0 con 0x7f2f44102ca0 2024-03-29T08:23:58.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:58.563+0000 7f2f3b400700 1 --2- 172.21.15.37:0/2525364359 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f2f2806cc30 0x7f2f2806f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:58.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:58.563+0000 7f2f3b400700 1 -- 172.21.15.37:0/2525364359 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f2f3008bee0 con 0x7f2f44102ca0 2024-03-29T08:23:58.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:58.563+0000 7f2f43400700 1 --2- 172.21.15.37:0/2525364359 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f2f2806cc30 0x7f2f2806f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:58.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:58.564+0000 7f2f43400700 1 --2- 172.21.15.37:0/2525364359 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f2f2806cc30 0x7f2f2806f0e0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f2f3400b020 tx=0x7f2f34009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:23:58.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:58.567+0000 7f2f3b400700 1 -- 172.21.15.37:0/2525364359 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f2f300598b0 con 0x7f2f44102ca0 2024-03-29T08:23:58.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:58.823+0000 7f2f48e00700 1 -- 172.21.15.37:0/2525364359 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f2f4405f5f0 con 0x7f2f44102ca0 2024-03-29T08:23:58.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:58.824+0000 7f2f3b400700 1 -- 172.21.15.37:0/2525364359 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 28 v28) v1 ==== 76+0+1841 (secure 0 0 0) 0x7f2f30059440 con 0x7f2f44102ca0 2024-03-29T08:23:58.824 INFO:teuthology.orchestra.run.smithi037.stdout:e28 2024-03-29T08:23:58.824 INFO:teuthology.orchestra.run.smithi037.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-29T08:23:58.824 INFO:teuthology.orchestra.run.smithi037.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T08:23:58.824 INFO:teuthology.orchestra.run.smithi037.stdout:legacy client fscid: 1 2024-03-29T08:23:58.824 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:23:58.824 INFO:teuthology.orchestra.run.smithi037.stdout:Filesystem 'cephfs' (1) 2024-03-29T08:23:58.824 INFO:teuthology.orchestra.run.smithi037.stdout:fs_name cephfs 2024-03-29T08:23:58.824 INFO:teuthology.orchestra.run.smithi037.stdout:epoch 21 2024-03-29T08:23:58.825 INFO:teuthology.orchestra.run.smithi037.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-03-29T08:23:58.825 INFO:teuthology.orchestra.run.smithi037.stdout:created 2024-03-29T08:03:29.241269+0000 2024-03-29T08:23:58.825 INFO:teuthology.orchestra.run.smithi037.stdout:modified 2024-03-29T08:17:16.237238+0000 2024-03-29T08:23:58.825 INFO:teuthology.orchestra.run.smithi037.stdout:tableserver 0 2024-03-29T08:23:58.825 INFO:teuthology.orchestra.run.smithi037.stdout:root 0 2024-03-29T08:23:58.825 INFO:teuthology.orchestra.run.smithi037.stdout:session_timeout 60 2024-03-29T08:23:58.825 INFO:teuthology.orchestra.run.smithi037.stdout:session_autoclose 300 2024-03-29T08:23:58.825 INFO:teuthology.orchestra.run.smithi037.stdout:max_file_size 1099511627776 2024-03-29T08:23:58.825 INFO:teuthology.orchestra.run.smithi037.stdout:max_xattr_size 65536 2024-03-29T08:23:58.825 INFO:teuthology.orchestra.run.smithi037.stdout:required_client_features {} 2024-03-29T08:23:58.825 INFO:teuthology.orchestra.run.smithi037.stdout:last_failure 0 2024-03-29T08:23:58.825 INFO:teuthology.orchestra.run.smithi037.stdout:last_failure_osd_epoch 82 2024-03-29T08:23:58.825 INFO:teuthology.orchestra.run.smithi037.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T08:23:58.825 INFO:teuthology.orchestra.run.smithi037.stdout:max_mds 2 2024-03-29T08:23:58.826 INFO:teuthology.orchestra.run.smithi037.stdout:in 0,1 2024-03-29T08:23:58.826 INFO:teuthology.orchestra.run.smithi037.stdout:up {} 2024-03-29T08:23:58.826 INFO:teuthology.orchestra.run.smithi037.stdout:failed 0,1 2024-03-29T08:23:58.826 INFO:teuthology.orchestra.run.smithi037.stdout:damaged 2024-03-29T08:23:58.826 INFO:teuthology.orchestra.run.smithi037.stdout:stopped 2024-03-29T08:23:58.826 INFO:teuthology.orchestra.run.smithi037.stdout:data_pools [3] 2024-03-29T08:23:58.826 INFO:teuthology.orchestra.run.smithi037.stdout:metadata_pool 2 2024-03-29T08:23:58.826 INFO:teuthology.orchestra.run.smithi037.stdout:inline_data disabled 2024-03-29T08:23:58.826 INFO:teuthology.orchestra.run.smithi037.stdout:balancer 2024-03-29T08:23:58.826 INFO:teuthology.orchestra.run.smithi037.stdout:bal_rank_mask -1 2024-03-29T08:23:58.826 INFO:teuthology.orchestra.run.smithi037.stdout:standby_count_wanted 1 2024-03-29T08:23:58.826 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:23:58.826 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:23:58.827 INFO:teuthology.orchestra.run.smithi037.stdout:Standby daemons: 2024-03-29T08:23:58.827 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:23:58.827 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi148.qukpjh{-1:24319} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.148:6826/2641327912,v1:172.21.15.148:6827/2641327912] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:23:58.827 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi037.mmwgxe{-1:34440} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.37:6828/1248142005,v1:172.21.15.37:6829/1248142005] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:23:58.827 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi037.iifsmh{-1:34444} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.37:6826/2060882959,v1:172.21.15.37:6827/2060882959] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:23:58.827 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi148.sqteih{-1:44395} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.148:6824/4078652242,v1:172.21.15.148:6825/4078652242] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:23:58.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:58.826+0000 7f2f48e00700 1 -- 172.21.15.37:0/2525364359 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f2f2806cc30 msgr2=0x7f2f2806f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:58.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:58.826+0000 7f2f48e00700 1 --2- 172.21.15.37:0/2525364359 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f2f2806cc30 0x7f2f2806f0e0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f2f3400b020 tx=0x7f2f34009e30 comp rx=0 tx=0).stop 2024-03-29T08:23:58.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:58.826+0000 7f2f48e00700 1 -- 172.21.15.37:0/2525364359 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2f44102ca0 msgr2=0x7f2f44113de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:58.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:58.826+0000 7f2f48e00700 1 --2- 172.21.15.37:0/2525364359 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2f44102ca0 0x7f2f44113de0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f2f30002a50 tx=0x7f2f300046b0 comp rx=0 tx=0).stop 2024-03-29T08:23:58.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:58.826+0000 7f2f48e00700 1 -- 172.21.15.37:0/2525364359 shutdown_connections 2024-03-29T08:23:58.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:58.826+0000 7f2f48e00700 1 --2- 172.21.15.37:0/2525364359 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f2f2806cc30 0x7f2f2806f0e0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:58.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:58.826+0000 7f2f48e00700 1 --2- 172.21.15.37:0/2525364359 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2f44065630 0x7f2f441138a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:58.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:58.826+0000 7f2f48e00700 1 --2- 172.21.15.37:0/2525364359 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2f44102ca0 0x7f2f44113de0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:58.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:58.826+0000 7f2f48e00700 1 -- 172.21.15.37:0/2525364359 >> 172.21.15.37:0/2525364359 conn(0x7f2f44071b70 msgr2=0x7f2f441059d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:23:58.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:58.826+0000 7f2f48e00700 1 -- 172.21.15.37:0/2525364359 shutdown_connections 2024-03-29T08:23:58.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:58.826+0000 7f2f48e00700 1 -- 172.21.15.37:0/2525364359 wait complete. 2024-03-29T08:23:58.829 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 28 2024-03-29T08:23:58.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:58.964+0000 7f2791e00700 1 -- 172.21.15.37:0/2213302330 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f278c102e00 msgr2=0x7f278c1031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:58.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:58.964+0000 7f2791e00700 1 --2- 172.21.15.37:0/2213302330 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f278c102e00 0x7f278c1031d0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f277c004940 tx=0x7f277c009e30 comp rx=0 tx=0).stop 2024-03-29T08:23:58.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:58.964+0000 7f2791e00700 1 -- 172.21.15.37:0/2213302330 shutdown_connections 2024-03-29T08:23:58.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:58.964+0000 7f2791e00700 1 --2- 172.21.15.37:0/2213302330 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f278c1037a0 0x7f278c0fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:58.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:58.964+0000 7f2791e00700 1 --2- 172.21.15.37:0/2213302330 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f278c102e00 0x7f278c1031d0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:58.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:58.964+0000 7f2791e00700 1 -- 172.21.15.37:0/2213302330 >> 172.21.15.37:0/2213302330 conn(0x7f278c0f89b0 msgr2=0x7f278c0fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:23:58.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:58.964+0000 7f2791e00700 1 -- 172.21.15.37:0/2213302330 shutdown_connections 2024-03-29T08:23:58.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:58.964+0000 7f2791e00700 1 -- 172.21.15.37:0/2213302330 wait complete. 2024-03-29T08:23:58.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:58.965+0000 7f2791e00700 1 Processor -- start 2024-03-29T08:23:58.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:58.965+0000 7f2791e00700 1 -- start start 2024-03-29T08:23:58.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:58.966+0000 7f2791e00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f278c102e00 0x7f278c1138b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:58.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:58.966+0000 7f2791e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f278c1037a0 0x7f278c113df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:58.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:58.966+0000 7f2791e00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f278c114440 con 0x7f278c1037a0 2024-03-29T08:23:58.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:58.966+0000 7f2791e00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f278c10da40 con 0x7f278c102e00 2024-03-29T08:23:58.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:58.966+0000 7f278a000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f278c1037a0 0x7f278c113df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:58.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:58.966+0000 7f278a000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f278c1037a0 0x7f278c113df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:33934/0 (socket says 172.21.15.37:33934) 2024-03-29T08:23:58.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:58.966+0000 7f278a000700 1 -- 172.21.15.37:0/2821182770 learned_addr learned my addr 172.21.15.37:0/2821182770 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:23:58.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:58.966+0000 7f278aa00700 1 --2- 172.21.15.37:0/2821182770 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f278c102e00 0x7f278c1138b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:58.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:58.966+0000 7f278a000700 1 -- 172.21.15.37:0/2821182770 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f278c102e00 msgr2=0x7f278c1138b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:58.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:58.966+0000 7f278a000700 1 --2- 172.21.15.37:0/2821182770 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f278c102e00 0x7f278c1138b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:58.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:58.966+0000 7f278a000700 1 -- 172.21.15.37:0/2821182770 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f277c005040 con 0x7f278c1037a0 2024-03-29T08:23:58.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:58.967+0000 7f278a000700 1 --2- 172.21.15.37:0/2821182770 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f278c1037a0 0x7f278c113df0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f2778002a50 tx=0x7f2778004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:23:58.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:58.967+0000 7f2782a00700 1 -- 172.21.15.37:0/2821182770 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2778007cc0 con 0x7f278c1037a0 2024-03-29T08:23:58.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:58.967+0000 7f2782a00700 1 -- 172.21.15.37:0/2821182770 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f277800f040 con 0x7f278c1037a0 2024-03-29T08:23:58.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:58.967+0000 7f2791e00700 1 -- 172.21.15.37:0/2821182770 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f278c10dd20 con 0x7f278c1037a0 2024-03-29T08:23:58.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:58.967+0000 7f2782a00700 1 -- 172.21.15.37:0/2821182770 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f277800d920 con 0x7f278c1037a0 2024-03-29T08:23:58.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:58.967+0000 7f2791e00700 1 -- 172.21.15.37:0/2821182770 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f278c10e2e0 con 0x7f278c1037a0 2024-03-29T08:23:58.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:58.968+0000 7f2791e00700 1 -- 172.21.15.37:0/2821182770 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f278c0fe550 con 0x7f278c1037a0 2024-03-29T08:23:58.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:58.969+0000 7f2782a00700 1 -- 172.21.15.37:0/2821182770 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f2778009680 con 0x7f278c1037a0 2024-03-29T08:23:58.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:58.970+0000 7f2782a00700 1 --2- 172.21.15.37:0/2821182770 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f277006cc20 0x7f277006f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:58.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:58.970+0000 7f2782a00700 1 -- 172.21.15.37:0/2821182770 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f277808bd80 con 0x7f278c1037a0 2024-03-29T08:23:58.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:58.970+0000 7f278aa00700 1 --2- 172.21.15.37:0/2821182770 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f277006cc20 0x7f277006f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:58.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:58.971+0000 7f278aa00700 1 --2- 172.21.15.37:0/2821182770 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f277006cc20 0x7f277006f0d0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f277c009fd0 tx=0x7f277c009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:23:58.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:58.975+0000 7f2782a00700 1 -- 172.21.15.37:0/2821182770 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f27780597d0 con 0x7f278c1037a0 2024-03-29T08:23:59.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:23:58 smithi148 ceph-mon[106692]: from='client.34764 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:23:59.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:23:58 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/1283137773' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:23:59.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:23:58 smithi037 ceph-mon[142325]: from='client.34764 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:23:59.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:23:58 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/1283137773' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:23:59.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:59.210+0000 7f2791e00700 1 -- 172.21.15.37:0/2821182770 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f278c109dc0 con 0x7f277006cc20 2024-03-29T08:23:59.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:59.212+0000 7f2782a00700 1 -- 172.21.15.37:0/2821182770 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f278c109dc0 con 0x7f277006cc20 2024-03-29T08:23:59.212 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-29T08:23:59.212 INFO:teuthology.orchestra.run.smithi037.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59", 2024-03-29T08:23:59.212 INFO:teuthology.orchestra.run.smithi037.stdout: "in_progress": true, 2024-03-29T08:23:59.212 INFO:teuthology.orchestra.run.smithi037.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-29T08:23:59.212 INFO:teuthology.orchestra.run.smithi037.stdout: "services_complete": [ 2024-03-29T08:23:59.212 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr", 2024-03-29T08:23:59.212 INFO:teuthology.orchestra.run.smithi037.stdout: "mon", 2024-03-29T08:23:59.212 INFO:teuthology.orchestra.run.smithi037.stdout: "crash", 2024-03-29T08:23:59.212 INFO:teuthology.orchestra.run.smithi037.stdout: "osd" 2024-03-29T08:23:59.212 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-29T08:23:59.212 INFO:teuthology.orchestra.run.smithi037.stdout: "progress": "15/21 daemons upgraded", 2024-03-29T08:23:59.213 INFO:teuthology.orchestra.run.smithi037.stdout: "message": "Currently upgrading mds daemons", 2024-03-29T08:23:59.213 INFO:teuthology.orchestra.run.smithi037.stdout: "is_paused": false 2024-03-29T08:23:59.213 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-29T08:23:59.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:59.214+0000 7f2791e00700 1 -- 172.21.15.37:0/2821182770 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f277006cc20 msgr2=0x7f277006f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:59.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:59.214+0000 7f2791e00700 1 --2- 172.21.15.37:0/2821182770 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f277006cc20 0x7f277006f0d0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f277c009fd0 tx=0x7f277c009e30 comp rx=0 tx=0).stop 2024-03-29T08:23:59.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:59.214+0000 7f2791e00700 1 -- 172.21.15.37:0/2821182770 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f278c1037a0 msgr2=0x7f278c113df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:59.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:59.214+0000 7f2791e00700 1 --2- 172.21.15.37:0/2821182770 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f278c1037a0 0x7f278c113df0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f2778002a50 tx=0x7f2778004600 comp rx=0 tx=0).stop 2024-03-29T08:23:59.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:59.214+0000 7f2791e00700 1 -- 172.21.15.37:0/2821182770 shutdown_connections 2024-03-29T08:23:59.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:59.214+0000 7f2791e00700 1 --2- 172.21.15.37:0/2821182770 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f277006cc20 0x7f277006f0d0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:59.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:59.214+0000 7f2791e00700 1 --2- 172.21.15.37:0/2821182770 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f278c102e00 0x7f278c1138b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:59.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:59.214+0000 7f2791e00700 1 --2- 172.21.15.37:0/2821182770 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f278c1037a0 0x7f278c113df0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:59.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:59.214+0000 7f2791e00700 1 -- 172.21.15.37:0/2821182770 >> 172.21.15.37:0/2821182770 conn(0x7f278c0f89b0 msgr2=0x7f278c0f9a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:23:59.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:59.214+0000 7f2791e00700 1 -- 172.21.15.37:0/2821182770 shutdown_connections 2024-03-29T08:23:59.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:59.214+0000 7f2791e00700 1 -- 172.21.15.37:0/2821182770 wait complete. 2024-03-29T08:23:59.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:59.351+0000 7fdbc2c00700 1 -- 172.21.15.37:0/42711779 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdbbc0fde30 msgr2=0x7fdbbc0fe200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:59.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:59.351+0000 7fdbc2c00700 1 --2- 172.21.15.37:0/42711779 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdbbc0fde30 0x7fdbbc0fe200 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fdbac004970 tx=0x7fdbac009e30 comp rx=0 tx=0).stop 2024-03-29T08:23:59.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:59.351+0000 7fdbc2c00700 1 -- 172.21.15.37:0/42711779 shutdown_connections 2024-03-29T08:23:59.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:59.351+0000 7fdbc2c00700 1 --2- 172.21.15.37:0/42711779 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdbbc0fe7d0 0x7fdbbc106ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:59.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:59.351+0000 7fdbc2c00700 1 --2- 172.21.15.37:0/42711779 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdbbc0fde30 0x7fdbbc0fe200 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:59.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:59.351+0000 7fdbc2c00700 1 -- 172.21.15.37:0/42711779 >> 172.21.15.37:0/42711779 conn(0x7fdbbc0f7870 msgr2=0x7fdbbc0f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:23:59.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:59.351+0000 7fdbc2c00700 1 -- 172.21.15.37:0/42711779 shutdown_connections 2024-03-29T08:23:59.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:59.351+0000 7fdbc2c00700 1 -- 172.21.15.37:0/42711779 wait complete. 2024-03-29T08:23:59.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:59.352+0000 7fdbc2c00700 1 Processor -- start 2024-03-29T08:23:59.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:59.353+0000 7fdbc2c00700 1 -- start start 2024-03-29T08:23:59.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:59.353+0000 7fdbc2c00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdbbc0fde30 0x7fdbbc115a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:59.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:59.353+0000 7fdbc2c00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdbbc0fe7d0 0x7fdbbc115fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:59.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:59.353+0000 7fdbc2c00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbbc116610 con 0x7fdbbc0fe7d0 2024-03-29T08:23:59.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:59.353+0000 7fdbc2c00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbbc10fbf0 con 0x7fdbbc0fde30 2024-03-29T08:23:59.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:59.353+0000 7fdbbb400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdbbc0fe7d0 0x7fdbbc115fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:59.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:59.354+0000 7fdbbb400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdbbc0fe7d0 0x7fdbbc115fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:33954/0 (socket says 172.21.15.37:33954) 2024-03-29T08:23:59.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:59.354+0000 7fdbbb400700 1 -- 172.21.15.37:0/1524603589 learned_addr learned my addr 172.21.15.37:0/1524603589 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:23:59.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:59.354+0000 7fdbbbe00700 1 --2- 172.21.15.37:0/1524603589 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdbbc0fde30 0x7fdbbc115a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:59.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:59.354+0000 7fdbbb400700 1 -- 172.21.15.37:0/1524603589 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdbbc0fde30 msgr2=0x7fdbbc115a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:59.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:59.354+0000 7fdbbb400700 1 --2- 172.21.15.37:0/1524603589 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdbbc0fde30 0x7fdbbc115a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:59.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:59.354+0000 7fdbbb400700 1 -- 172.21.15.37:0/1524603589 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbac005040 con 0x7fdbbc0fe7d0 2024-03-29T08:23:59.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:59.354+0000 7fdbbb400700 1 --2- 172.21.15.37:0/1524603589 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdbbc0fe7d0 0x7fdbbc115fc0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fdba800c8c0 tx=0x7fdba80044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:23:59.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:59.354+0000 7fdbb3e00700 1 -- 172.21.15.37:0/1524603589 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdba800a760 con 0x7fdbbc0fe7d0 2024-03-29T08:23:59.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:59.355+0000 7fdbb3e00700 1 -- 172.21.15.37:0/1524603589 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fdba800ad70 con 0x7fdbbc0fe7d0 2024-03-29T08:23:59.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:59.355+0000 7fdbc2c00700 1 -- 172.21.15.37:0/1524603589 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbbc10fed0 con 0x7fdbbc0fe7d0 2024-03-29T08:23:59.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:59.355+0000 7fdbb3e00700 1 -- 172.21.15.37:0/1524603589 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdba80113a0 con 0x7fdbbc0fe7d0 2024-03-29T08:23:59.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:59.355+0000 7fdbc2c00700 1 -- 172.21.15.37:0/1524603589 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbbc110410 con 0x7fdbbc0fe7d0 2024-03-29T08:23:59.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:59.356+0000 7fdbc2c00700 1 -- 172.21.15.37:0/1524603589 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdbbc066ab0 con 0x7fdbbc0fe7d0 2024-03-29T08:23:59.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:59.357+0000 7fdbb3e00700 1 -- 172.21.15.37:0/1524603589 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fdba8012400 con 0x7fdbbc0fe7d0 2024-03-29T08:23:59.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:59.358+0000 7fdbb3e00700 1 --2- 172.21.15.37:0/1524603589 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fdba006cc20 0x7fdba006f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:59.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:59.358+0000 7fdbb3e00700 1 -- 172.21.15.37:0/1524603589 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fdba808c100 con 0x7fdbbc0fe7d0 2024-03-29T08:23:59.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:59.358+0000 7fdbbbe00700 1 --2- 172.21.15.37:0/1524603589 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fdba006cc20 0x7fdba006f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:59.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:59.359+0000 7fdbbbe00700 1 --2- 172.21.15.37:0/1524603589 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fdba006cc20 0x7fdba006f0d0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fdbac004f50 tx=0x7fdbac009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:23:59.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:59.363+0000 7fdbb3e00700 1 -- 172.21.15.37:0/1524603589 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fdba8059b50 con 0x7fdbbc0fe7d0 2024-03-29T08:23:59.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:59.665+0000 7fdbc2c00700 1 -- 172.21.15.37:0/1524603589 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fdbbc04c710 con 0x7fdbbc0fe7d0 2024-03-29T08:23:59.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:59.666+0000 7fdbb3e00700 1 -- 172.21.15.37:0/1524603589 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7fdba80596e0 con 0x7fdbbc0fe7d0 2024-03-29T08:23:59.666 INFO:teuthology.orchestra.run.smithi037.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-03-29T08:23:59.666 INFO:teuthology.orchestra.run.smithi037.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-03-29T08:23:59.666 INFO:teuthology.orchestra.run.smithi037.stdout: fs cephfs is degraded 2024-03-29T08:23:59.666 INFO:teuthology.orchestra.run.smithi037.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-03-29T08:23:59.666 INFO:teuthology.orchestra.run.smithi037.stdout: fs cephfs is offline because no MDS is active for it. 2024-03-29T08:23:59.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:59.668+0000 7fdbc2c00700 1 -- 172.21.15.37:0/1524603589 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fdba006cc20 msgr2=0x7fdba006f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:59.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:59.668+0000 7fdbc2c00700 1 --2- 172.21.15.37:0/1524603589 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fdba006cc20 0x7fdba006f0d0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fdbac004f50 tx=0x7fdbac009e30 comp rx=0 tx=0).stop 2024-03-29T08:23:59.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:59.668+0000 7fdbc2c00700 1 -- 172.21.15.37:0/1524603589 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdbbc0fe7d0 msgr2=0x7fdbbc115fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:59.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:59.668+0000 7fdbc2c00700 1 --2- 172.21.15.37:0/1524603589 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdbbc0fe7d0 0x7fdbbc115fc0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fdba800c8c0 tx=0x7fdba80044c0 comp rx=0 tx=0).stop 2024-03-29T08:23:59.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:59.668+0000 7fdbc2c00700 1 -- 172.21.15.37:0/1524603589 shutdown_connections 2024-03-29T08:23:59.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:59.668+0000 7fdbc2c00700 1 --2- 172.21.15.37:0/1524603589 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fdba006cc20 0x7fdba006f0d0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:59.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:59.668+0000 7fdbc2c00700 1 --2- 172.21.15.37:0/1524603589 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdbbc0fde30 0x7fdbbc115a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:59.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:59.668+0000 7fdbc2c00700 1 --2- 172.21.15.37:0/1524603589 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdbbc0fe7d0 0x7fdbbc115fc0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:59.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:59.668+0000 7fdbc2c00700 1 -- 172.21.15.37:0/1524603589 >> 172.21.15.37:0/1524603589 conn(0x7fdbbc0f7870 msgr2=0x7fdbbc0f8d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:23:59.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:59.668+0000 7fdbc2c00700 1 -- 172.21.15.37:0/1524603589 shutdown_connections 2024-03-29T08:23:59.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:23:59.668+0000 7fdbc2c00700 1 -- 172.21.15.37:0/1524603589 wait complete. 2024-03-29T08:24:00.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:23:59 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/2525364359' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T08:24:00.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:23:59 smithi148 ceph-mon[106692]: from='client.34776 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:24:00.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:23:59 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/1524603589' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T08:24:00.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:23:59 smithi148 ceph-mon[106692]: pgmap v393: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:24:00.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:23:59 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/2525364359' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T08:24:00.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:23:59 smithi037 ceph-mon[142325]: from='client.34776 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:24:00.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:23:59 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/1524603589' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T08:24:00.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:23:59 smithi037 ceph-mon[142325]: pgmap v393: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:24:02.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:24:01 smithi148 ceph-mon[106692]: pgmap v394: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:24:02.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:24:01 smithi037 ceph-mon[142325]: pgmap v394: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:24:04.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:24:03 smithi148 ceph-mon[106692]: pgmap v395: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:24:04.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:24:03 smithi037 ceph-mon[142325]: pgmap v395: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:24:06.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:24:05 smithi148 ceph-mon[106692]: pgmap v396: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:24:06.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:24:05 smithi037 ceph-mon[142325]: pgmap v396: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:24:08.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:24:07 smithi148 ceph-mon[106692]: pgmap v397: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:24:08.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:24:07 smithi037 ceph-mon[142325]: pgmap v397: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:24:10.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:24:10 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:24:10.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:24:10 smithi148 ceph-mon[106692]: pgmap v398: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:24:10.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:24:10 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:24:10.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:24:10 smithi037 ceph-mon[142325]: pgmap v398: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:24:12.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:24:11 smithi148 ceph-mon[106692]: pgmap v399: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:24:12.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:24:11 smithi037 ceph-mon[142325]: pgmap v399: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:24:14.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:24:13 smithi148 ceph-mon[106692]: pgmap v400: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:24:14.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:24:13 smithi037 ceph-mon[142325]: pgmap v400: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:24:16.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:24:15 smithi148 ceph-mon[106692]: pgmap v401: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:24:16.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:24:15 smithi037 ceph-mon[142325]: pgmap v401: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:24:18.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:24:17 smithi148 ceph-mon[106692]: pgmap v402: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:24:18.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:24:17 smithi037 ceph-mon[142325]: pgmap v402: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:24:20.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:24:20 smithi148 ceph-mon[106692]: pgmap v403: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:24:20.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:24:20 smithi037 ceph-mon[142325]: pgmap v403: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:24:22.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:24:21 smithi148 ceph-mon[106692]: pgmap v404: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:24:22.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:24:21 smithi037 ceph-mon[142325]: pgmap v404: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:24:24.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:24:23 smithi148 ceph-mon[106692]: pgmap v405: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:24:24.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:24:23 smithi037 ceph-mon[142325]: pgmap v405: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:24:25.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:24:25 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:24:25.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:24:25 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:24:26.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:24:26 smithi148 ceph-mon[106692]: pgmap v406: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:24:26.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:24:26 smithi037 ceph-mon[142325]: pgmap v406: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:24:28.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:24:27 smithi148 ceph-mon[106692]: pgmap v407: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:24:28.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:24:27 smithi037 ceph-mon[142325]: pgmap v407: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:24:29.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:29.823+0000 7f9840200700 1 -- 172.21.15.37:0/3898914611 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f98381037a0 msgr2=0x7f98380fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:29.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:29.823+0000 7f9840200700 1 --2- 172.21.15.37:0/3898914611 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f98381037a0 0x7f98380fce10 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f9824004940 tx=0x7f9824009e30 comp rx=0 tx=0).stop 2024-03-29T08:24:29.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:29.823+0000 7f9840200700 1 -- 172.21.15.37:0/3898914611 shutdown_connections 2024-03-29T08:24:29.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:29.823+0000 7f9840200700 1 --2- 172.21.15.37:0/3898914611 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f98381037a0 0x7f98380fce10 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:29.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:29.823+0000 7f9840200700 1 --2- 172.21.15.37:0/3898914611 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9838102e00 0x7f98381031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:29.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:29.823+0000 7f9840200700 1 -- 172.21.15.37:0/3898914611 >> 172.21.15.37:0/3898914611 conn(0x7f98380f89b0 msgr2=0x7f98380fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:24:29.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:29.823+0000 7f9840200700 1 -- 172.21.15.37:0/3898914611 shutdown_connections 2024-03-29T08:24:29.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:29.823+0000 7f9840200700 1 -- 172.21.15.37:0/3898914611 wait complete. 2024-03-29T08:24:29.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:29.824+0000 7f9840200700 1 Processor -- start 2024-03-29T08:24:29.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:29.824+0000 7f9840200700 1 -- start start 2024-03-29T08:24:29.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:29.825+0000 7f9840200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9838102e00 0x7f9838115cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:29.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:29.825+0000 7f9840200700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f98381037a0 0x7f9838116230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:29.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:29.825+0000 7f9840200700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9838116880 con 0x7f9838102e00 2024-03-29T08:24:29.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:29.825+0000 7f9840200700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f983810fc50 con 0x7f98381037a0 2024-03-29T08:24:29.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:29.825+0000 7f983d800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9838102e00 0x7f9838115cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:29.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:29.825+0000 7f983d800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9838102e00 0x7f9838115cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44894/0 (socket says 172.21.15.37:44894) 2024-03-29T08:24:29.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:29.825+0000 7f983d800700 1 -- 172.21.15.37:0/2887006148 learned_addr learned my addr 172.21.15.37:0/2887006148 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:24:29.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:29.826+0000 7f983d800700 1 -- 172.21.15.37:0/2887006148 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f98381037a0 msgr2=0x7f9838116230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:29.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:29.826+0000 7f983d800700 1 --2- 172.21.15.37:0/2887006148 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f98381037a0 0x7f9838116230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:29.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:29.826+0000 7f983d800700 1 -- 172.21.15.37:0/2887006148 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9824005040 con 0x7f9838102e00 2024-03-29T08:24:29.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:29.826+0000 7f983d800700 1 --2- 172.21.15.37:0/2887006148 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9838102e00 0x7f9838115cf0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f982c002a50 tx=0x7f982c0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:29.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:29.826+0000 7f9835600700 1 -- 172.21.15.37:0/2887006148 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f982c007cc0 con 0x7f9838102e00 2024-03-29T08:24:29.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:29.826+0000 7f9835600700 1 -- 172.21.15.37:0/2887006148 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f982c00f040 con 0x7f9838102e00 2024-03-29T08:24:29.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:29.826+0000 7f9835600700 1 -- 172.21.15.37:0/2887006148 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f982c009540 con 0x7f9838102e00 2024-03-29T08:24:29.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:29.826+0000 7f9840200700 1 -- 172.21.15.37:0/2887006148 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f983810ff30 con 0x7f9838102e00 2024-03-29T08:24:29.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:29.826+0000 7f9840200700 1 -- 172.21.15.37:0/2887006148 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98381104a0 con 0x7f9838102e00 2024-03-29T08:24:29.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:29.827+0000 7f9840200700 1 -- 172.21.15.37:0/2887006148 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98380fe550 con 0x7f9838102e00 2024-03-29T08:24:29.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:29.828+0000 7f9835600700 1 -- 172.21.15.37:0/2887006148 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f982c0096a0 con 0x7f9838102e00 2024-03-29T08:24:29.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:29.829+0000 7f9835600700 1 --2- 172.21.15.37:0/2887006148 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f982006cc30 0x7f982006f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:29.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:29.829+0000 7f9835600700 1 -- 172.21.15.37:0/2887006148 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f982c08bee0 con 0x7f9838102e00 2024-03-29T08:24:29.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:29.830+0000 7f983ce00700 1 --2- 172.21.15.37:0/2887006148 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f982006cc30 0x7f982006f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:29.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:29.830+0000 7f983ce00700 1 --2- 172.21.15.37:0/2887006148 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f982006cc30 0x7f982006f0e0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f9824004f90 tx=0x7f9824009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:29.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:29.834+0000 7f9835600700 1 -- 172.21.15.37:0/2887006148 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f982c0598b0 con 0x7f9838102e00 2024-03-29T08:24:30.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:30.064+0000 7f9840200700 1 -- 172.21.15.37:0/2887006148 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f9838109e10 con 0x7f982006cc30 2024-03-29T08:24:30.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:30.065+0000 7f9835600700 1 -- 172.21.15.37:0/2887006148 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f9838109e10 con 0x7f982006cc30 2024-03-29T08:24:30.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:30.068+0000 7f9840200700 1 -- 172.21.15.37:0/2887006148 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f982006cc30 msgr2=0x7f982006f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:30.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:30.068+0000 7f9840200700 1 --2- 172.21.15.37:0/2887006148 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f982006cc30 0x7f982006f0e0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f9824004f90 tx=0x7f9824009e30 comp rx=0 tx=0).stop 2024-03-29T08:24:30.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:30.068+0000 7f9840200700 1 -- 172.21.15.37:0/2887006148 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9838102e00 msgr2=0x7f9838115cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:30.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:30.068+0000 7f9840200700 1 --2- 172.21.15.37:0/2887006148 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9838102e00 0x7f9838115cf0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f982c002a50 tx=0x7f982c0046b0 comp rx=0 tx=0).stop 2024-03-29T08:24:30.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:30.068+0000 7f9840200700 1 -- 172.21.15.37:0/2887006148 shutdown_connections 2024-03-29T08:24:30.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:30.068+0000 7f9840200700 1 --2- 172.21.15.37:0/2887006148 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f982006cc30 0x7f982006f0e0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:30.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:30.068+0000 7f9840200700 1 --2- 172.21.15.37:0/2887006148 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9838102e00 0x7f9838115cf0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:30.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:30.068+0000 7f9840200700 1 --2- 172.21.15.37:0/2887006148 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f98381037a0 0x7f9838116230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:30.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:30.068+0000 7f9840200700 1 -- 172.21.15.37:0/2887006148 >> 172.21.15.37:0/2887006148 conn(0x7f98380f89b0 msgr2=0x7f98380f9a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:24:30.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:30.068+0000 7f9840200700 1 -- 172.21.15.37:0/2887006148 shutdown_connections 2024-03-29T08:24:30.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:30.068+0000 7f9840200700 1 -- 172.21.15.37:0/2887006148 wait complete. 2024-03-29T08:24:30.083 INFO:teuthology.orchestra.run.smithi037.stdout:true 2024-03-29T08:24:30.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:30.207+0000 7fdd20c00700 1 -- 172.21.15.37:0/380846989 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdd1c070360 msgr2=0x7fdd1c070730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:30.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:30.207+0000 7fdd20c00700 1 --2- 172.21.15.37:0/380846989 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdd1c070360 0x7fdd1c070730 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fdd0c004970 tx=0x7fdd0c009e30 comp rx=0 tx=0).stop 2024-03-29T08:24:30.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:30.207+0000 7fdd20c00700 1 -- 172.21.15.37:0/380846989 shutdown_connections 2024-03-29T08:24:30.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:30.207+0000 7fdd20c00700 1 --2- 172.21.15.37:0/380846989 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdd1c070c70 0x7fdd1c109110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:30.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:30.207+0000 7fdd20c00700 1 --2- 172.21.15.37:0/380846989 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdd1c070360 0x7fdd1c070730 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:30.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:30.207+0000 7fdd20c00700 1 -- 172.21.15.37:0/380846989 >> 172.21.15.37:0/380846989 conn(0x7fdd1c0f89b0 msgr2=0x7fdd1c0fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:24:30.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:30.207+0000 7fdd20c00700 1 -- 172.21.15.37:0/380846989 shutdown_connections 2024-03-29T08:24:30.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:30.207+0000 7fdd20c00700 1 -- 172.21.15.37:0/380846989 wait complete. 2024-03-29T08:24:30.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:30.208+0000 7fdd20c00700 1 Processor -- start 2024-03-29T08:24:30.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:30.208+0000 7fdd20c00700 1 -- start start 2024-03-29T08:24:30.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:30.208+0000 7fdd20c00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdd1c070360 0x7fdd1c115c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:30.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:30.208+0000 7fdd20c00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdd1c070c70 0x7fdd1c1161c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:30.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:30.208+0000 7fdd20c00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd1c116810 con 0x7fdd1c070360 2024-03-29T08:24:30.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:30.208+0000 7fdd20c00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd1c10fbe0 con 0x7fdd1c070c70 2024-03-29T08:24:30.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:30.209+0000 7fdd19e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdd1c070360 0x7fdd1c115c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:30.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:30.209+0000 7fdd19e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdd1c070360 0x7fdd1c115c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40486/0 (socket says 172.21.15.37:40486) 2024-03-29T08:24:30.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:30.209+0000 7fdd19e00700 1 -- 172.21.15.37:0/2784687881 learned_addr learned my addr 172.21.15.37:0/2784687881 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:24:30.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:30.209+0000 7fdd19400700 1 --2- 172.21.15.37:0/2784687881 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdd1c070c70 0x7fdd1c1161c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:30.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:30.209+0000 7fdd19e00700 1 -- 172.21.15.37:0/2784687881 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdd1c070c70 msgr2=0x7fdd1c1161c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:30.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:30.209+0000 7fdd19e00700 1 --2- 172.21.15.37:0/2784687881 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdd1c070c70 0x7fdd1c1161c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:30.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:30.209+0000 7fdd19e00700 1 -- 172.21.15.37:0/2784687881 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd0c005040 con 0x7fdd1c070360 2024-03-29T08:24:30.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:30.209+0000 7fdd19e00700 1 --2- 172.21.15.37:0/2784687881 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdd1c070360 0x7fdd1c115c80 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fdd0c004f50 tx=0x7fdd0c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:30.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:30.210+0000 7fdd12000700 1 -- 172.21.15.37:0/2784687881 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdd0c00e040 con 0x7fdd1c070360 2024-03-29T08:24:30.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:30.210+0000 7fdd20c00700 1 -- 172.21.15.37:0/2784687881 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd1c10fe60 con 0x7fdd1c070360 2024-03-29T08:24:30.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:30.210+0000 7fdd12000700 1 -- 172.21.15.37:0/2784687881 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fdd0c013420 con 0x7fdd1c070360 2024-03-29T08:24:30.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:30.210+0000 7fdd12000700 1 -- 172.21.15.37:0/2784687881 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdd0c01c610 con 0x7fdd1c070360 2024-03-29T08:24:30.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:30.210+0000 7fdd20c00700 1 -- 172.21.15.37:0/2784687881 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd1c1103a0 con 0x7fdd1c070360 2024-03-29T08:24:30.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:30.211+0000 7fdd20c00700 1 -- 172.21.15.37:0/2784687881 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd1c103b90 con 0x7fdd1c070360 2024-03-29T08:24:30.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:30.212+0000 7fdd12000700 1 -- 172.21.15.37:0/2784687881 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fdd0c013590 con 0x7fdd1c070360 2024-03-29T08:24:30.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:30.213+0000 7fdd12000700 1 --2- 172.21.15.37:0/2784687881 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fdd0006cc30 0x7fdd0006f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:30.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:30.213+0000 7fdd12000700 1 -- 172.21.15.37:0/2784687881 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fdd0c08d2b0 con 0x7fdd1c070360 2024-03-29T08:24:30.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:30.213+0000 7fdd19400700 1 --2- 172.21.15.37:0/2784687881 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fdd0006cc30 0x7fdd0006f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:30.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:30.214+0000 7fdd19400700 1 --2- 172.21.15.37:0/2784687881 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fdd0006cc30 0x7fdd0006f0e0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fdd08009fd0 tx=0x7fdd08009f60 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:30.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:30.217+0000 7fdd12000700 1 -- 172.21.15.37:0/2784687881 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fdd0c05ac80 con 0x7fdd1c070360 2024-03-29T08:24:30.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:30.448+0000 7fdd20c00700 1 -- 172.21.15.37:0/2784687881 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fdd1c10c020 con 0x7fdd0006cc30 2024-03-29T08:24:30.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:30.450+0000 7fdd12000700 1 -- 172.21.15.37:0/2784687881 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fdd1c10c020 con 0x7fdd0006cc30 2024-03-29T08:24:30.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:30.452+0000 7fdd20c00700 1 -- 172.21.15.37:0/2784687881 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fdd0006cc30 msgr2=0x7fdd0006f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:30.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:30.452+0000 7fdd20c00700 1 --2- 172.21.15.37:0/2784687881 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fdd0006cc30 0x7fdd0006f0e0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fdd08009fd0 tx=0x7fdd08009f60 comp rx=0 tx=0).stop 2024-03-29T08:24:30.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:30.452+0000 7fdd20c00700 1 -- 172.21.15.37:0/2784687881 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdd1c070360 msgr2=0x7fdd1c115c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:30.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:30.452+0000 7fdd20c00700 1 --2- 172.21.15.37:0/2784687881 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdd1c070360 0x7fdd1c115c80 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fdd0c004f50 tx=0x7fdd0c009e30 comp rx=0 tx=0).stop 2024-03-29T08:24:30.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:30.452+0000 7fdd20c00700 1 -- 172.21.15.37:0/2784687881 shutdown_connections 2024-03-29T08:24:30.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:30.452+0000 7fdd20c00700 1 --2- 172.21.15.37:0/2784687881 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fdd0006cc30 0x7fdd0006f0e0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:30.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:30.452+0000 7fdd20c00700 1 --2- 172.21.15.37:0/2784687881 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdd1c070360 0x7fdd1c115c80 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:30.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:30.452+0000 7fdd20c00700 1 --2- 172.21.15.37:0/2784687881 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdd1c070c70 0x7fdd1c1161c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:30.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:30.452+0000 7fdd20c00700 1 -- 172.21.15.37:0/2784687881 >> 172.21.15.37:0/2784687881 conn(0x7fdd1c0f89b0 msgr2=0x7fdd1c1015f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:24:30.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:30.452+0000 7fdd20c00700 1 -- 172.21.15.37:0/2784687881 shutdown_connections 2024-03-29T08:24:30.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:30.452+0000 7fdd20c00700 1 -- 172.21.15.37:0/2784687881 wait complete. 2024-03-29T08:24:30.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:24:30 smithi148 ceph-mon[106692]: pgmap v408: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:24:30.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:30.589+0000 7ff35a400700 1 -- 172.21.15.37:0/1087703039 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff354070300 msgr2=0x7ff3540706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:30.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:30.589+0000 7ff35a400700 1 --2- 172.21.15.37:0/1087703039 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff354070300 0x7ff3540706d0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7ff348004970 tx=0x7ff348009e30 comp rx=0 tx=0).stop 2024-03-29T08:24:30.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:30.589+0000 7ff35a400700 1 -- 172.21.15.37:0/1087703039 shutdown_connections 2024-03-29T08:24:30.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:30.589+0000 7ff35a400700 1 --2- 172.21.15.37:0/1087703039 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff354070c10 0x7ff354108fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:30.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:30.589+0000 7ff35a400700 1 --2- 172.21.15.37:0/1087703039 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff354070300 0x7ff3540706d0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:30.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:30.589+0000 7ff35a400700 1 -- 172.21.15.37:0/1087703039 >> 172.21.15.37:0/1087703039 conn(0x7ff3540f8830 msgr2=0x7ff3540fac40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:24:30.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:30.589+0000 7ff35a400700 1 -- 172.21.15.37:0/1087703039 shutdown_connections 2024-03-29T08:24:30.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:30.589+0000 7ff35a400700 1 -- 172.21.15.37:0/1087703039 wait complete. 2024-03-29T08:24:30.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:30.590+0000 7ff35a400700 1 Processor -- start 2024-03-29T08:24:30.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:30.590+0000 7ff35a400700 1 -- start start 2024-03-29T08:24:30.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:30.590+0000 7ff35a400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff354070300 0x7ff354115ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:30.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:30.590+0000 7ff35a400700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff354070c10 0x7ff354116000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:30.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:30.590+0000 7ff35a400700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff354116650 con 0x7ff354070300 2024-03-29T08:24:30.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:30.590+0000 7ff35a400700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff35410fa40 con 0x7ff354070c10 2024-03-29T08:24:30.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:30.591+0000 7ff359000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff354070300 0x7ff354115ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:30.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:30.591+0000 7ff359000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff354070300 0x7ff354115ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40508/0 (socket says 172.21.15.37:40508) 2024-03-29T08:24:30.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:30.591+0000 7ff359000700 1 -- 172.21.15.37:0/3083064745 learned_addr learned my addr 172.21.15.37:0/3083064745 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:24:30.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:30.591+0000 7ff353e00700 1 --2- 172.21.15.37:0/3083064745 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff354070c10 0x7ff354116000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:30.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:30.591+0000 7ff359000700 1 -- 172.21.15.37:0/3083064745 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff354070c10 msgr2=0x7ff354116000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:30.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:30.591+0000 7ff359000700 1 --2- 172.21.15.37:0/3083064745 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff354070c10 0x7ff354116000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:30.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:30.591+0000 7ff359000700 1 -- 172.21.15.37:0/3083064745 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff348005040 con 0x7ff354070300 2024-03-29T08:24:30.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:30.591+0000 7ff359000700 1 --2- 172.21.15.37:0/3083064745 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff354070300 0x7ff354115ac0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7ff348004f50 tx=0x7ff348009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:30.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:30.592+0000 7ff350c00700 1 -- 172.21.15.37:0/3083064745 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff34800e040 con 0x7ff354070300 2024-03-29T08:24:30.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:30.592+0000 7ff350c00700 1 -- 172.21.15.37:0/3083064745 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ff348013420 con 0x7ff354070300 2024-03-29T08:24:30.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:30.592+0000 7ff35a400700 1 -- 172.21.15.37:0/3083064745 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff35410fcc0 con 0x7ff354070300 2024-03-29T08:24:30.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:30.592+0000 7ff350c00700 1 -- 172.21.15.37:0/3083064745 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff34801c630 con 0x7ff354070300 2024-03-29T08:24:30.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:30.592+0000 7ff35a400700 1 -- 172.21.15.37:0/3083064745 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff354110200 con 0x7ff354070300 2024-03-29T08:24:30.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:30.593+0000 7ff35a400700 1 -- 172.21.15.37:0/3083064745 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff354071a40 con 0x7ff354070300 2024-03-29T08:24:30.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:30.594+0000 7ff350c00700 1 -- 172.21.15.37:0/3083064745 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7ff348013590 con 0x7ff354070300 2024-03-29T08:24:30.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:30.595+0000 7ff350c00700 1 --2- 172.21.15.37:0/3083064745 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7ff33c06cc30 0x7ff33c06f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:30.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:30.595+0000 7ff350c00700 1 -- 172.21.15.37:0/3083064745 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7ff34808d280 con 0x7ff354070300 2024-03-29T08:24:30.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:30.595+0000 7ff353e00700 1 --2- 172.21.15.37:0/3083064745 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7ff33c06cc30 0x7ff33c06f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:30.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:30.596+0000 7ff353e00700 1 --2- 172.21.15.37:0/3083064745 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7ff33c06cc30 0x7ff33c06f0e0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ff354110c00 tx=0x7ff34000b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:30.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:30.599+0000 7ff350c00700 1 -- 172.21.15.37:0/3083064745 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7ff34805acd0 con 0x7ff354070300 2024-03-29T08:24:30.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:24:30 smithi037 ceph-mon[142325]: pgmap v408: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:24:30.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:30.822+0000 7ff35a400700 1 -- 172.21.15.37:0/3083064745 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7ff35410beb0 con 0x7ff33c06cc30 2024-03-29T08:24:30.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:30.832+0000 7ff350c00700 1 -- 172.21.15.37:0/3083064745 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7ff34000f040 con 0x7ff33c06cc30 2024-03-29T08:24:30.833 INFO:teuthology.orchestra.run.smithi037.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-29T08:24:30.833 INFO:teuthology.orchestra.run.smithi037.stdout:alertmanager.smithi037 smithi037 *:9093,9094 running (14m) 6m ago 27m 14.9M - 0.25.0 c8568f914cd2 2f9f9746ed0f 2024-03-29T08:24:30.833 INFO:teuthology.orchestra.run.smithi037.stdout:crash.smithi037 smithi037 running (11m) 6m ago 27m 6647k - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 178d416a3186 2024-03-29T08:24:30.833 INFO:teuthology.orchestra.run.smithi037.stdout:crash.smithi148 smithi148 running (11m) 6m ago 26m 6656k - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 23666989d58f 2024-03-29T08:24:30.833 INFO:teuthology.orchestra.run.smithi037.stdout:grafana.smithi037 smithi037 *:3000 running (14m) 6m ago 26m 71.9M - 9.4.7 954c08fa6188 6b976fe49952 2024-03-29T08:24:30.833 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi037.iifsmh smithi037 running (6m) 6m ago 20m 10.8M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 2d97f562cc5b 2024-03-29T08:24:30.834 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi037.mmwgxe smithi037 running (7m) 6m ago 20m 10.9M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff ec37037f46e8 2024-03-29T08:24:30.834 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi148.qukpjh smithi148 running (20m) 6m ago 20m 15.3M - 17.2.7-789-g4213e17c cf22eb8683be 8e469782a273 2024-03-29T08:24:30.834 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi148.sqteih smithi148 running (6m) 6m ago 20m 10.7M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 44c15c77037a 2024-03-29T08:24:30.834 INFO:teuthology.orchestra.run.smithi037.stdout:mgr.smithi037.wjikii smithi037 *:8443,9283,8765 running (16m) 6m ago 28m 506M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff c5a1417b9417 2024-03-29T08:24:30.834 INFO:teuthology.orchestra.run.smithi037.stdout:mgr.smithi148.onhgvm smithi148 *:8443,9283,8765 running (15m) 6m ago 26m 450M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff fcfdbce062bd 2024-03-29T08:24:30.834 INFO:teuthology.orchestra.run.smithi037.stdout:mon.smithi037 smithi037 running (12m) 6m ago 29m 54.8M 2048M 18.2.2-1015-gd0e392d9 1cc2f45c17ff eea2060783af 2024-03-29T08:24:30.834 INFO:teuthology.orchestra.run.smithi037.stdout:mon.smithi148 smithi148 running (12m) 6m ago 26m 43.7M 2048M 18.2.2-1015-gd0e392d9 1cc2f45c17ff ad4b2c5748be 2024-03-29T08:24:30.834 INFO:teuthology.orchestra.run.smithi037.stdout:node-exporter.smithi037 smithi037 *:9100 running (15m) 6m ago 27m 7625k - 1.5.0 0da6a335fe13 26b8ae9cb878 2024-03-29T08:24:30.835 INFO:teuthology.orchestra.run.smithi037.stdout:node-exporter.smithi148 smithi148 *:9100 running (15m) 6m ago 26m 8166k - 1.5.0 0da6a335fe13 98414e4e64ed 2024-03-29T08:24:30.835 INFO:teuthology.orchestra.run.smithi037.stdout:osd.0 smithi037 running (10m) 6m ago 25m 104M 1619M 18.2.2-1015-gd0e392d9 1cc2f45c17ff c0c584f6245c 2024-03-29T08:24:30.835 INFO:teuthology.orchestra.run.smithi037.stdout:osd.1 smithi037 running (10m) 6m ago 24m 100M 1619M 18.2.2-1015-gd0e392d9 1cc2f45c17ff a4a2ff4cc83d 2024-03-29T08:24:30.835 INFO:teuthology.orchestra.run.smithi037.stdout:osd.2 smithi037 running (9m) 6m ago 24m 82.3M 1619M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 8b1bc71aa79e 2024-03-29T08:24:30.835 INFO:teuthology.orchestra.run.smithi037.stdout:osd.3 smithi148 running (8m) 6m ago 23m 105M 2642M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 1c656ab8ff86 2024-03-29T08:24:30.835 INFO:teuthology.orchestra.run.smithi037.stdout:osd.4 smithi148 running (8m) 6m ago 23m 97.3M 2642M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 966a9de11d22 2024-03-29T08:24:30.835 INFO:teuthology.orchestra.run.smithi037.stdout:osd.5 smithi148 running (7m) 6m ago 22m 85.8M 2642M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 474c4f9212ce 2024-03-29T08:24:30.835 INFO:teuthology.orchestra.run.smithi037.stdout:prometheus.smithi037 smithi037 *:9095 running (14m) 6m ago 26m 54.4M - 2.43.0 a07b618ecd1d 30c0223ec589 2024-03-29T08:24:30.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:30.835+0000 7ff35a400700 1 -- 172.21.15.37:0/3083064745 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7ff33c06cc30 msgr2=0x7ff33c06f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:30.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:30.835+0000 7ff35a400700 1 --2- 172.21.15.37:0/3083064745 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7ff33c06cc30 0x7ff33c06f0e0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ff354110c00 tx=0x7ff34000b040 comp rx=0 tx=0).stop 2024-03-29T08:24:30.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:30.835+0000 7ff35a400700 1 -- 172.21.15.37:0/3083064745 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff354070300 msgr2=0x7ff354115ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:30.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:30.835+0000 7ff35a400700 1 --2- 172.21.15.37:0/3083064745 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff354070300 0x7ff354115ac0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7ff348004f50 tx=0x7ff348009e30 comp rx=0 tx=0).stop 2024-03-29T08:24:30.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:30.835+0000 7ff35a400700 1 -- 172.21.15.37:0/3083064745 shutdown_connections 2024-03-29T08:24:30.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:30.835+0000 7ff35a400700 1 --2- 172.21.15.37:0/3083064745 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7ff33c06cc30 0x7ff33c06f0e0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:30.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:30.835+0000 7ff35a400700 1 --2- 172.21.15.37:0/3083064745 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff354070300 0x7ff354115ac0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:30.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:30.835+0000 7ff35a400700 1 --2- 172.21.15.37:0/3083064745 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff354070c10 0x7ff354116000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:30.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:30.835+0000 7ff35a400700 1 -- 172.21.15.37:0/3083064745 >> 172.21.15.37:0/3083064745 conn(0x7ff3540f8830 msgr2=0x7ff3540f9e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:24:30.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:30.835+0000 7ff35a400700 1 -- 172.21.15.37:0/3083064745 shutdown_connections 2024-03-29T08:24:30.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:30.835+0000 7ff35a400700 1 -- 172.21.15.37:0/3083064745 wait complete. 2024-03-29T08:24:30.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:30.972+0000 7f0ba2200700 1 -- 172.21.15.37:0/3307733398 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0b9c0fe830 msgr2=0x7f0b9c106f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:30.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:30.972+0000 7f0ba2200700 1 --2- 172.21.15.37:0/3307733398 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0b9c0fe830 0x7f0b9c106f00 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f0b88004940 tx=0x7f0b88009e30 comp rx=0 tx=0).stop 2024-03-29T08:24:30.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:30.972+0000 7f0ba2200700 1 -- 172.21.15.37:0/3307733398 shutdown_connections 2024-03-29T08:24:30.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:30.972+0000 7f0ba2200700 1 --2- 172.21.15.37:0/3307733398 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0b9c0fe830 0x7f0b9c106f00 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:30.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:30.972+0000 7f0ba2200700 1 --2- 172.21.15.37:0/3307733398 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0b9c0fde90 0x7f0b9c0fe260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:30.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:30.972+0000 7f0ba2200700 1 -- 172.21.15.37:0/3307733398 >> 172.21.15.37:0/3307733398 conn(0x7f0b9c072bc0 msgr2=0x7f0b9c072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:24:30.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:30.972+0000 7f0ba2200700 1 -- 172.21.15.37:0/3307733398 shutdown_connections 2024-03-29T08:24:30.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:30.972+0000 7f0ba2200700 1 -- 172.21.15.37:0/3307733398 wait complete. 2024-03-29T08:24:30.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:30.973+0000 7f0ba2200700 1 Processor -- start 2024-03-29T08:24:30.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:30.973+0000 7f0ba2200700 1 -- start start 2024-03-29T08:24:30.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:30.974+0000 7f0ba2200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0b9c0fde90 0x7f0b9c115d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:30.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:30.974+0000 7f0ba2200700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0b9c0fe830 0x7f0b9c116270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:30.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:30.974+0000 7f0ba2200700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b9c1168c0 con 0x7f0b9c0fde90 2024-03-29T08:24:30.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:30.974+0000 7f0ba2200700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b9c10fc90 con 0x7f0b9c0fe830 2024-03-29T08:24:30.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:30.974+0000 7f0b9b400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0b9c0fde90 0x7f0b9c115d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:30.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:30.974+0000 7f0b9b400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0b9c0fde90 0x7f0b9c115d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40522/0 (socket says 172.21.15.37:40522) 2024-03-29T08:24:30.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:30.974+0000 7f0b9b400700 1 -- 172.21.15.37:0/2340082659 learned_addr learned my addr 172.21.15.37:0/2340082659 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:24:30.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:30.974+0000 7f0b9aa00700 1 --2- 172.21.15.37:0/2340082659 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0b9c0fe830 0x7f0b9c116270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:30.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:30.974+0000 7f0b9b400700 1 -- 172.21.15.37:0/2340082659 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0b9c0fe830 msgr2=0x7f0b9c116270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:30.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:30.974+0000 7f0b9b400700 1 --2- 172.21.15.37:0/2340082659 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0b9c0fe830 0x7f0b9c116270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:30.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:30.974+0000 7f0b9b400700 1 -- 172.21.15.37:0/2340082659 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b88005040 con 0x7f0b9c0fde90 2024-03-29T08:24:30.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:30.975+0000 7f0b9b400700 1 --2- 172.21.15.37:0/2340082659 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0b9c0fde90 0x7f0b9c115d30 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f0b8c002a50 tx=0x7f0b8c0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:30.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:30.975+0000 7f0b93400700 1 -- 172.21.15.37:0/2340082659 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0b8c007cc0 con 0x7f0b9c0fde90 2024-03-29T08:24:30.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:30.975+0000 7f0b93400700 1 -- 172.21.15.37:0/2340082659 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f0b8c00f040 con 0x7f0b9c0fde90 2024-03-29T08:24:30.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:30.975+0000 7f0b93400700 1 -- 172.21.15.37:0/2340082659 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0b8c00da70 con 0x7f0b9c0fde90 2024-03-29T08:24:30.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:30.975+0000 7f0ba2200700 1 -- 172.21.15.37:0/2340082659 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b9c10ff70 con 0x7f0b9c0fde90 2024-03-29T08:24:30.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:30.975+0000 7f0ba2200700 1 -- 172.21.15.37:0/2340082659 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b9c1104b0 con 0x7f0b9c0fde90 2024-03-29T08:24:30.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:30.976+0000 7f0ba2200700 1 -- 172.21.15.37:0/2340082659 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b9c106a20 con 0x7f0b9c0fde90 2024-03-29T08:24:30.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:30.977+0000 7f0b93400700 1 -- 172.21.15.37:0/2340082659 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f0b8c00dc10 con 0x7f0b9c0fde90 2024-03-29T08:24:30.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:30.978+0000 7f0b93400700 1 --2- 172.21.15.37:0/2340082659 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f0b8006cc30 0x7f0b8006f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:30.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:30.978+0000 7f0b93400700 1 -- 172.21.15.37:0/2340082659 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f0b8c08b740 con 0x7f0b9c0fde90 2024-03-29T08:24:30.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:30.979+0000 7f0b9aa00700 1 --2- 172.21.15.37:0/2340082659 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f0b8006cc30 0x7f0b8006f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:30.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:30.979+0000 7f0b9aa00700 1 --2- 172.21.15.37:0/2340082659 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f0b8006cc30 0x7f0b8006f0e0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f0b9c1115d0 tx=0x7f0b88009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:30.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:30.983+0000 7f0b93400700 1 -- 172.21.15.37:0/2340082659 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f0b8c059190 con 0x7f0b9c0fde90 2024-03-29T08:24:31.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:31.281+0000 7f0ba2200700 1 -- 172.21.15.37:0/2340082659 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f0b9c109df0 con 0x7f0b9c0fde90 2024-03-29T08:24:31.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:31.282+0000 7f0b93400700 1 -- 172.21.15.37:0/2340082659 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+847 (secure 0 0 0) 0x7f0b8c058d20 con 0x7f0b9c0fde90 2024-03-29T08:24:31.282 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-29T08:24:31.282 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": { 2024-03-29T08:24:31.282 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 2 2024-03-29T08:24:31.283 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:24:31.283 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": { 2024-03-29T08:24:31.283 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 2 2024-03-29T08:24:31.283 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:24:31.283 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": { 2024-03-29T08:24:31.283 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 6 2024-03-29T08:24:31.283 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:24:31.283 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": { 2024-03-29T08:24:31.283 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 17.2.7-789-g4213e17c (4213e17c4de67016ed14f55d22503f4923b3a901) quincy (stable)": 1, 2024-03-29T08:24:31.283 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 3 2024-03-29T08:24:31.283 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:24:31.283 INFO:teuthology.orchestra.run.smithi037.stdout: "overall": { 2024-03-29T08:24:31.283 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 17.2.7-789-g4213e17c (4213e17c4de67016ed14f55d22503f4923b3a901) quincy (stable)": 1, 2024-03-29T08:24:31.283 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 13 2024-03-29T08:24:31.283 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-29T08:24:31.283 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-29T08:24:31.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:31.284+0000 7f0ba2200700 1 -- 172.21.15.37:0/2340082659 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f0b8006cc30 msgr2=0x7f0b8006f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:31.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:31.284+0000 7f0ba2200700 1 --2- 172.21.15.37:0/2340082659 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f0b8006cc30 0x7f0b8006f0e0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f0b9c1115d0 tx=0x7f0b88009e30 comp rx=0 tx=0).stop 2024-03-29T08:24:31.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:31.284+0000 7f0ba2200700 1 -- 172.21.15.37:0/2340082659 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0b9c0fde90 msgr2=0x7f0b9c115d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:31.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:31.284+0000 7f0ba2200700 1 --2- 172.21.15.37:0/2340082659 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0b9c0fde90 0x7f0b9c115d30 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f0b8c002a50 tx=0x7f0b8c0046b0 comp rx=0 tx=0).stop 2024-03-29T08:24:31.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:31.284+0000 7f0ba2200700 1 -- 172.21.15.37:0/2340082659 shutdown_connections 2024-03-29T08:24:31.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:31.284+0000 7f0ba2200700 1 --2- 172.21.15.37:0/2340082659 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f0b8006cc30 0x7f0b8006f0e0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:31.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:31.284+0000 7f0ba2200700 1 --2- 172.21.15.37:0/2340082659 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0b9c0fde90 0x7f0b9c115d30 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:31.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:31.284+0000 7f0ba2200700 1 --2- 172.21.15.37:0/2340082659 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0b9c0fe830 0x7f0b9c116270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:31.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:31.284+0000 7f0ba2200700 1 -- 172.21.15.37:0/2340082659 >> 172.21.15.37:0/2340082659 conn(0x7f0b9c072bc0 msgr2=0x7f0b9c0fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:24:31.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:31.285+0000 7f0ba2200700 1 -- 172.21.15.37:0/2340082659 shutdown_connections 2024-03-29T08:24:31.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:31.285+0000 7f0ba2200700 1 -- 172.21.15.37:0/2340082659 wait complete. 2024-03-29T08:24:31.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:31.421+0000 7f25cfe00700 1 -- 172.21.15.37:0/2455368591 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f25c8070c70 msgr2=0x7f25c8109110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:31.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:31.421+0000 7f25cfe00700 1 --2- 172.21.15.37:0/2455368591 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f25c8070c70 0x7f25c8109110 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f25b4004970 tx=0x7f25b4009e30 comp rx=0 tx=0).stop 2024-03-29T08:24:31.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:31.421+0000 7f25cfe00700 1 -- 172.21.15.37:0/2455368591 shutdown_connections 2024-03-29T08:24:31.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:31.421+0000 7f25cfe00700 1 --2- 172.21.15.37:0/2455368591 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f25c8070c70 0x7f25c8109110 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:31.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:31.421+0000 7f25cfe00700 1 --2- 172.21.15.37:0/2455368591 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f25c8070360 0x7f25c8070730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:31.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:31.421+0000 7f25cfe00700 1 -- 172.21.15.37:0/2455368591 >> 172.21.15.37:0/2455368591 conn(0x7f25c80f89b0 msgr2=0x7f25c80fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:24:31.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:31.421+0000 7f25cfe00700 1 -- 172.21.15.37:0/2455368591 shutdown_connections 2024-03-29T08:24:31.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:31.421+0000 7f25cfe00700 1 -- 172.21.15.37:0/2455368591 wait complete. 2024-03-29T08:24:31.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:31.422+0000 7f25cfe00700 1 Processor -- start 2024-03-29T08:24:31.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:31.422+0000 7f25cfe00700 1 -- start start 2024-03-29T08:24:31.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:31.422+0000 7f25cfe00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f25c8070360 0x7f25c8115ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:31.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:31.422+0000 7f25cfe00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f25c8070c70 0x7f25c81161e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:31.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:31.423+0000 7f25cfe00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25c8116830 con 0x7f25c8070c70 2024-03-29T08:24:31.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:31.423+0000 7f25cfe00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25c810fc00 con 0x7f25c8070360 2024-03-29T08:24:31.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:31.423+0000 7f25cca00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f25c8070c70 0x7f25c81161e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:31.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:31.423+0000 7f25cca00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f25c8070c70 0x7f25c81161e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40548/0 (socket says 172.21.15.37:40548) 2024-03-29T08:24:31.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:31.423+0000 7f25cca00700 1 -- 172.21.15.37:0/2240477854 learned_addr learned my addr 172.21.15.37:0/2240477854 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:24:31.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:31.423+0000 7f25cd400700 1 --2- 172.21.15.37:0/2240477854 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f25c8070360 0x7f25c8115ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:31.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:31.423+0000 7f25cca00700 1 -- 172.21.15.37:0/2240477854 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f25c8070360 msgr2=0x7f25c8115ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:31.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:31.423+0000 7f25cca00700 1 --2- 172.21.15.37:0/2240477854 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f25c8070360 0x7f25c8115ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:31.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:31.423+0000 7f25cca00700 1 -- 172.21.15.37:0/2240477854 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25b4005040 con 0x7f25c8070c70 2024-03-29T08:24:31.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:31.423+0000 7f25cd400700 1 --2- 172.21.15.37:0/2240477854 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f25c8070360 0x7f25c8115ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:24:31.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:31.423+0000 7f25cca00700 1 --2- 172.21.15.37:0/2240477854 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f25c8070c70 0x7f25c81161e0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f25b4005010 tx=0x7f25b4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:31.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:31.424+0000 7f25c5600700 1 -- 172.21.15.37:0/2240477854 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f25b400e040 con 0x7f25c8070c70 2024-03-29T08:24:31.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:31.424+0000 7f25c5600700 1 -- 172.21.15.37:0/2240477854 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f25b4004290 con 0x7f25c8070c70 2024-03-29T08:24:31.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:31.424+0000 7f25cfe00700 1 -- 172.21.15.37:0/2240477854 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25c810fe80 con 0x7f25c8070c70 2024-03-29T08:24:31.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:31.424+0000 7f25c5600700 1 -- 172.21.15.37:0/2240477854 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f25b4013e40 con 0x7f25c8070c70 2024-03-29T08:24:31.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:31.424+0000 7f25cfe00700 1 -- 172.21.15.37:0/2240477854 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25c81103c0 con 0x7f25c8070c70 2024-03-29T08:24:31.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:31.425+0000 7f25cfe00700 1 -- 172.21.15.37:0/2240477854 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25c8103bd0 con 0x7f25c8070c70 2024-03-29T08:24:31.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:31.426+0000 7f25c5600700 1 -- 172.21.15.37:0/2240477854 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f25b4013620 con 0x7f25c8070c70 2024-03-29T08:24:31.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:31.427+0000 7f25c5600700 1 --2- 172.21.15.37:0/2240477854 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f25b006cc30 0x7f25b006f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:31.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:31.427+0000 7f25c5600700 1 -- 172.21.15.37:0/2240477854 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f25b408d290 con 0x7f25c8070c70 2024-03-29T08:24:31.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:31.427+0000 7f25cd400700 1 --2- 172.21.15.37:0/2240477854 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f25b006cc30 0x7f25b006f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:31.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:31.428+0000 7f25cd400700 1 --2- 172.21.15.37:0/2240477854 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f25b006cc30 0x7f25b006f0e0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f25bc004600 tx=0x7f25bc00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:31.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:31.431+0000 7f25c5600700 1 -- 172.21.15.37:0/2240477854 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f25b405ad60 con 0x7f25c8070c70 2024-03-29T08:24:31.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:24:31 smithi148 ceph-mon[106692]: from='client.34784 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:24:31.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:24:31 smithi148 ceph-mon[106692]: from='client.34788 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:24:31.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:24:31 smithi037 ceph-mon[142325]: from='client.34784 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:24:31.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:24:31 smithi037 ceph-mon[142325]: from='client.34788 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:24:31.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:31.689+0000 7f25cfe00700 1 -- 172.21.15.37:0/2240477854 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f25c8071aa0 con 0x7f25c8070c70 2024-03-29T08:24:31.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:31.689+0000 7f25c5600700 1 -- 172.21.15.37:0/2240477854 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 28 v28) v1 ==== 76+0+1841 (secure 0 0 0) 0x7f25b405a8f0 con 0x7f25c8070c70 2024-03-29T08:24:31.690 INFO:teuthology.orchestra.run.smithi037.stdout:e28 2024-03-29T08:24:31.690 INFO:teuthology.orchestra.run.smithi037.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-29T08:24:31.690 INFO:teuthology.orchestra.run.smithi037.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T08:24:31.690 INFO:teuthology.orchestra.run.smithi037.stdout:legacy client fscid: 1 2024-03-29T08:24:31.690 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:24:31.690 INFO:teuthology.orchestra.run.smithi037.stdout:Filesystem 'cephfs' (1) 2024-03-29T08:24:31.690 INFO:teuthology.orchestra.run.smithi037.stdout:fs_name cephfs 2024-03-29T08:24:31.690 INFO:teuthology.orchestra.run.smithi037.stdout:epoch 21 2024-03-29T08:24:31.690 INFO:teuthology.orchestra.run.smithi037.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-03-29T08:24:31.691 INFO:teuthology.orchestra.run.smithi037.stdout:created 2024-03-29T08:03:29.241269+0000 2024-03-29T08:24:31.691 INFO:teuthology.orchestra.run.smithi037.stdout:modified 2024-03-29T08:17:16.237238+0000 2024-03-29T08:24:31.691 INFO:teuthology.orchestra.run.smithi037.stdout:tableserver 0 2024-03-29T08:24:31.691 INFO:teuthology.orchestra.run.smithi037.stdout:root 0 2024-03-29T08:24:31.691 INFO:teuthology.orchestra.run.smithi037.stdout:session_timeout 60 2024-03-29T08:24:31.691 INFO:teuthology.orchestra.run.smithi037.stdout:session_autoclose 300 2024-03-29T08:24:31.691 INFO:teuthology.orchestra.run.smithi037.stdout:max_file_size 1099511627776 2024-03-29T08:24:31.691 INFO:teuthology.orchestra.run.smithi037.stdout:max_xattr_size 65536 2024-03-29T08:24:31.691 INFO:teuthology.orchestra.run.smithi037.stdout:required_client_features {} 2024-03-29T08:24:31.691 INFO:teuthology.orchestra.run.smithi037.stdout:last_failure 0 2024-03-29T08:24:31.691 INFO:teuthology.orchestra.run.smithi037.stdout:last_failure_osd_epoch 82 2024-03-29T08:24:31.691 INFO:teuthology.orchestra.run.smithi037.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T08:24:31.691 INFO:teuthology.orchestra.run.smithi037.stdout:max_mds 2 2024-03-29T08:24:31.691 INFO:teuthology.orchestra.run.smithi037.stdout:in 0,1 2024-03-29T08:24:31.691 INFO:teuthology.orchestra.run.smithi037.stdout:up {} 2024-03-29T08:24:31.691 INFO:teuthology.orchestra.run.smithi037.stdout:failed 0,1 2024-03-29T08:24:31.692 INFO:teuthology.orchestra.run.smithi037.stdout:damaged 2024-03-29T08:24:31.692 INFO:teuthology.orchestra.run.smithi037.stdout:stopped 2024-03-29T08:24:31.692 INFO:teuthology.orchestra.run.smithi037.stdout:data_pools [3] 2024-03-29T08:24:31.692 INFO:teuthology.orchestra.run.smithi037.stdout:metadata_pool 2 2024-03-29T08:24:31.692 INFO:teuthology.orchestra.run.smithi037.stdout:inline_data disabled 2024-03-29T08:24:31.692 INFO:teuthology.orchestra.run.smithi037.stdout:balancer 2024-03-29T08:24:31.692 INFO:teuthology.orchestra.run.smithi037.stdout:bal_rank_mask -1 2024-03-29T08:24:31.692 INFO:teuthology.orchestra.run.smithi037.stdout:standby_count_wanted 1 2024-03-29T08:24:31.692 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:24:31.692 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:24:31.692 INFO:teuthology.orchestra.run.smithi037.stdout:Standby daemons: 2024-03-29T08:24:31.692 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:24:31.692 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi148.qukpjh{-1:24319} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.148:6826/2641327912,v1:172.21.15.148:6827/2641327912] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:24:31.692 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi037.mmwgxe{-1:34440} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.37:6828/1248142005,v1:172.21.15.37:6829/1248142005] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:24:31.692 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi037.iifsmh{-1:34444} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.37:6826/2060882959,v1:172.21.15.37:6827/2060882959] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:24:31.692 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi148.sqteih{-1:44395} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.148:6824/4078652242,v1:172.21.15.148:6825/4078652242] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:24:31.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:31.692+0000 7f25cfe00700 1 -- 172.21.15.37:0/2240477854 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f25b006cc30 msgr2=0x7f25b006f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:31.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:31.692+0000 7f25cfe00700 1 --2- 172.21.15.37:0/2240477854 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f25b006cc30 0x7f25b006f0e0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f25bc004600 tx=0x7f25bc00b040 comp rx=0 tx=0).stop 2024-03-29T08:24:31.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:31.692+0000 7f25cfe00700 1 -- 172.21.15.37:0/2240477854 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f25c8070c70 msgr2=0x7f25c81161e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:31.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:31.692+0000 7f25cfe00700 1 --2- 172.21.15.37:0/2240477854 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f25c8070c70 0x7f25c81161e0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f25b4005010 tx=0x7f25b4009e30 comp rx=0 tx=0).stop 2024-03-29T08:24:31.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:31.692+0000 7f25cfe00700 1 -- 172.21.15.37:0/2240477854 shutdown_connections 2024-03-29T08:24:31.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:31.692+0000 7f25cfe00700 1 --2- 172.21.15.37:0/2240477854 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f25b006cc30 0x7f25b006f0e0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:31.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:31.692+0000 7f25cfe00700 1 --2- 172.21.15.37:0/2240477854 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f25c8070360 0x7f25c8115ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:31.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:31.692+0000 7f25cfe00700 1 --2- 172.21.15.37:0/2240477854 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f25c8070c70 0x7f25c81161e0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:31.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:31.692+0000 7f25cfe00700 1 -- 172.21.15.37:0/2240477854 >> 172.21.15.37:0/2240477854 conn(0x7f25c80f89b0 msgr2=0x7f25c81015f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:24:31.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:31.692+0000 7f25cfe00700 1 -- 172.21.15.37:0/2240477854 shutdown_connections 2024-03-29T08:24:31.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:31.692+0000 7f25cfe00700 1 -- 172.21.15.37:0/2240477854 wait complete. 2024-03-29T08:24:31.694 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 28 2024-03-29T08:24:31.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:31.831+0000 7efc87e00700 1 -- 172.21.15.37:0/3516342767 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efc800fe830 msgr2=0x7efc80106f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:31.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:31.831+0000 7efc87e00700 1 --2- 172.21.15.37:0/3516342767 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efc800fe830 0x7efc80106f00 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7efc6c004940 tx=0x7efc6c009e30 comp rx=0 tx=0).stop 2024-03-29T08:24:31.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:31.831+0000 7efc87e00700 1 -- 172.21.15.37:0/3516342767 shutdown_connections 2024-03-29T08:24:31.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:31.831+0000 7efc87e00700 1 --2- 172.21.15.37:0/3516342767 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efc800fe830 0x7efc80106f00 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:31.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:31.831+0000 7efc87e00700 1 --2- 172.21.15.37:0/3516342767 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efc800fde90 0x7efc800fe260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:31.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:31.831+0000 7efc87e00700 1 -- 172.21.15.37:0/3516342767 >> 172.21.15.37:0/3516342767 conn(0x7efc80072bc0 msgr2=0x7efc80072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:24:31.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:31.831+0000 7efc87e00700 1 -- 172.21.15.37:0/3516342767 shutdown_connections 2024-03-29T08:24:31.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:31.832+0000 7efc87e00700 1 -- 172.21.15.37:0/3516342767 wait complete. 2024-03-29T08:24:31.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:31.832+0000 7efc87e00700 1 Processor -- start 2024-03-29T08:24:31.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:31.832+0000 7efc87e00700 1 -- start start 2024-03-29T08:24:31.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:31.833+0000 7efc87e00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efc800fde90 0x7efc80115d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:31.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:31.833+0000 7efc87e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efc800fe830 0x7efc80116250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:31.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:31.833+0000 7efc87e00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc801168a0 con 0x7efc800fe830 2024-03-29T08:24:31.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:31.833+0000 7efc87e00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc8010fc70 con 0x7efc800fde90 2024-03-29T08:24:31.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:31.833+0000 7efc84a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efc800fe830 0x7efc80116250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:31.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:31.833+0000 7efc84a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efc800fe830 0x7efc80116250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40580/0 (socket says 172.21.15.37:40580) 2024-03-29T08:24:31.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:31.833+0000 7efc84a00700 1 -- 172.21.15.37:0/1950802986 learned_addr learned my addr 172.21.15.37:0/1950802986 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:24:31.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:31.833+0000 7efc85400700 1 --2- 172.21.15.37:0/1950802986 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efc800fde90 0x7efc80115d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:31.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:31.833+0000 7efc84a00700 1 -- 172.21.15.37:0/1950802986 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efc800fde90 msgr2=0x7efc80115d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:31.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:31.833+0000 7efc84a00700 1 --2- 172.21.15.37:0/1950802986 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efc800fde90 0x7efc80115d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:31.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:31.833+0000 7efc84a00700 1 -- 172.21.15.37:0/1950802986 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc6c005040 con 0x7efc800fe830 2024-03-29T08:24:31.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:31.834+0000 7efc84a00700 1 --2- 172.21.15.37:0/1950802986 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efc800fe830 0x7efc80116250 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7efc6c0093b0 tx=0x7efc6c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:31.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:31.834+0000 7efc7d600700 1 -- 172.21.15.37:0/1950802986 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efc6c00e040 con 0x7efc800fe830 2024-03-29T08:24:31.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:31.834+0000 7efc7d600700 1 -- 172.21.15.37:0/1950802986 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7efc6c00bed0 con 0x7efc800fe830 2024-03-29T08:24:31.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:31.834+0000 7efc87e00700 1 -- 172.21.15.37:0/1950802986 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc8010fef0 con 0x7efc800fe830 2024-03-29T08:24:31.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:31.834+0000 7efc7d600700 1 -- 172.21.15.37:0/1950802986 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efc6c01c3f0 con 0x7efc800fe830 2024-03-29T08:24:31.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:31.834+0000 7efc87e00700 1 -- 172.21.15.37:0/1950802986 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc80110430 con 0x7efc800fe830 2024-03-29T08:24:31.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:31.835+0000 7efc87e00700 1 -- 172.21.15.37:0/1950802986 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc80106a20 con 0x7efc800fe830 2024-03-29T08:24:31.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:31.836+0000 7efc7d600700 1 -- 172.21.15.37:0/1950802986 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7efc6c013790 con 0x7efc800fe830 2024-03-29T08:24:31.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:31.837+0000 7efc7d600700 1 --2- 172.21.15.37:0/1950802986 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7efc6806cc30 0x7efc6806f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:31.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:31.837+0000 7efc7d600700 1 -- 172.21.15.37:0/1950802986 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7efc6c08d3d0 con 0x7efc800fe830 2024-03-29T08:24:31.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:31.837+0000 7efc85400700 1 --2- 172.21.15.37:0/1950802986 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7efc6806cc30 0x7efc6806f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:31.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:31.838+0000 7efc85400700 1 --2- 172.21.15.37:0/1950802986 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7efc6806cc30 0x7efc6806f0e0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7efc74007fd0 tx=0x7efc74009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:31.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:31.842+0000 7efc7d600700 1 -- 172.21.15.37:0/1950802986 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7efc6c05ae20 con 0x7efc800fe830 2024-03-29T08:24:32.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:32.074+0000 7efc87e00700 1 -- 172.21.15.37:0/1950802986 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7efc80111360 con 0x7efc6806cc30 2024-03-29T08:24:32.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:32.075+0000 7efc7d600700 1 -- 172.21.15.37:0/1950802986 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7efc80111360 con 0x7efc6806cc30 2024-03-29T08:24:32.075 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-29T08:24:32.075 INFO:teuthology.orchestra.run.smithi037.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59", 2024-03-29T08:24:32.075 INFO:teuthology.orchestra.run.smithi037.stdout: "in_progress": true, 2024-03-29T08:24:32.076 INFO:teuthology.orchestra.run.smithi037.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-29T08:24:32.076 INFO:teuthology.orchestra.run.smithi037.stdout: "services_complete": [ 2024-03-29T08:24:32.076 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr", 2024-03-29T08:24:32.076 INFO:teuthology.orchestra.run.smithi037.stdout: "mon", 2024-03-29T08:24:32.076 INFO:teuthology.orchestra.run.smithi037.stdout: "crash", 2024-03-29T08:24:32.076 INFO:teuthology.orchestra.run.smithi037.stdout: "osd" 2024-03-29T08:24:32.076 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-29T08:24:32.076 INFO:teuthology.orchestra.run.smithi037.stdout: "progress": "15/21 daemons upgraded", 2024-03-29T08:24:32.076 INFO:teuthology.orchestra.run.smithi037.stdout: "message": "Currently upgrading mds daemons", 2024-03-29T08:24:32.076 INFO:teuthology.orchestra.run.smithi037.stdout: "is_paused": false 2024-03-29T08:24:32.076 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-29T08:24:32.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:32.077+0000 7efc87e00700 1 -- 172.21.15.37:0/1950802986 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7efc6806cc30 msgr2=0x7efc6806f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:32.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:32.077+0000 7efc87e00700 1 --2- 172.21.15.37:0/1950802986 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7efc6806cc30 0x7efc6806f0e0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7efc74007fd0 tx=0x7efc74009040 comp rx=0 tx=0).stop 2024-03-29T08:24:32.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:32.077+0000 7efc87e00700 1 -- 172.21.15.37:0/1950802986 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efc800fe830 msgr2=0x7efc80116250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:32.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:32.077+0000 7efc87e00700 1 --2- 172.21.15.37:0/1950802986 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efc800fe830 0x7efc80116250 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7efc6c0093b0 tx=0x7efc6c009e30 comp rx=0 tx=0).stop 2024-03-29T08:24:32.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:32.078+0000 7efc87e00700 1 -- 172.21.15.37:0/1950802986 shutdown_connections 2024-03-29T08:24:32.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:32.078+0000 7efc87e00700 1 --2- 172.21.15.37:0/1950802986 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7efc6806cc30 0x7efc6806f0e0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:32.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:32.078+0000 7efc87e00700 1 --2- 172.21.15.37:0/1950802986 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efc800fde90 0x7efc80115d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:32.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:32.078+0000 7efc87e00700 1 --2- 172.21.15.37:0/1950802986 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efc800fe830 0x7efc80116250 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:32.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:32.078+0000 7efc87e00700 1 -- 172.21.15.37:0/1950802986 >> 172.21.15.37:0/1950802986 conn(0x7efc80072bc0 msgr2=0x7efc800fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:24:32.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:32.078+0000 7efc87e00700 1 -- 172.21.15.37:0/1950802986 shutdown_connections 2024-03-29T08:24:32.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:32.078+0000 7efc87e00700 1 -- 172.21.15.37:0/1950802986 wait complete. 2024-03-29T08:24:32.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:32.214+0000 7fa88ce00700 1 -- 172.21.15.37:0/1480814027 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa888102e00 msgr2=0x7fa8881031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:32.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:32.214+0000 7fa88ce00700 1 --2- 172.21.15.37:0/1480814027 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa888102e00 0x7fa8881031d0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fa878004940 tx=0x7fa878009e30 comp rx=0 tx=0).stop 2024-03-29T08:24:32.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:32.214+0000 7fa88ce00700 1 -- 172.21.15.37:0/1480814027 shutdown_connections 2024-03-29T08:24:32.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:32.214+0000 7fa88ce00700 1 --2- 172.21.15.37:0/1480814027 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8881037a0 0x7fa8880fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:32.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:32.214+0000 7fa88ce00700 1 --2- 172.21.15.37:0/1480814027 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa888102e00 0x7fa8881031d0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:32.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:32.214+0000 7fa88ce00700 1 -- 172.21.15.37:0/1480814027 >> 172.21.15.37:0/1480814027 conn(0x7fa8880f89b0 msgr2=0x7fa8880fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:24:32.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:32.215+0000 7fa88ce00700 1 -- 172.21.15.37:0/1480814027 shutdown_connections 2024-03-29T08:24:32.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:32.215+0000 7fa88ce00700 1 -- 172.21.15.37:0/1480814027 wait complete. 2024-03-29T08:24:32.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:32.215+0000 7fa88ce00700 1 Processor -- start 2024-03-29T08:24:32.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:32.216+0000 7fa88ce00700 1 -- start start 2024-03-29T08:24:32.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:32.216+0000 7fa88ce00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa888102e00 0x7fa8880702c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:32.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:32.216+0000 7fa88ce00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa8881037a0 0x7fa888070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:32.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:32.216+0000 7fa88ce00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa888070d90 con 0x7fa8881037a0 2024-03-29T08:24:32.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:32.216+0000 7fa88ce00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa888070ed0 con 0x7fa888102e00 2024-03-29T08:24:32.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:32.216+0000 7fa885400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa8881037a0 0x7fa888070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:32.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:32.216+0000 7fa885400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa8881037a0 0x7fa888070800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40602/0 (socket says 172.21.15.37:40602) 2024-03-29T08:24:32.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:32.216+0000 7fa885400700 1 -- 172.21.15.37:0/2495034885 learned_addr learned my addr 172.21.15.37:0/2495034885 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:24:32.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:32.216+0000 7fa885e00700 1 --2- 172.21.15.37:0/2495034885 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa888102e00 0x7fa8880702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:32.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:32.217+0000 7fa885400700 1 -- 172.21.15.37:0/2495034885 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa888102e00 msgr2=0x7fa8880702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:32.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:32.217+0000 7fa885400700 1 --2- 172.21.15.37:0/2495034885 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa888102e00 0x7fa8880702c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:32.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:32.217+0000 7fa885400700 1 -- 172.21.15.37:0/2495034885 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa878005040 con 0x7fa8881037a0 2024-03-29T08:24:32.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:32.217+0000 7fa885400700 1 --2- 172.21.15.37:0/2495034885 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa8881037a0 0x7fa888070800 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fa874002a50 tx=0x7fa874004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:32.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:32.217+0000 7fa87e000700 1 -- 172.21.15.37:0/2495034885 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa874007cc0 con 0x7fa8881037a0 2024-03-29T08:24:32.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:32.217+0000 7fa87e000700 1 -- 172.21.15.37:0/2495034885 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fa87400f040 con 0x7fa8881037a0 2024-03-29T08:24:32.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:32.217+0000 7fa88ce00700 1 -- 172.21.15.37:0/2495034885 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa888119e20 con 0x7fa8881037a0 2024-03-29T08:24:32.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:32.217+0000 7fa87e000700 1 -- 172.21.15.37:0/2495034885 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa87400d920 con 0x7fa8881037a0 2024-03-29T08:24:32.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:32.217+0000 7fa88ce00700 1 -- 172.21.15.37:0/2495034885 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa88811a2a0 con 0x7fa8881037a0 2024-03-29T08:24:32.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:32.218+0000 7fa88ce00700 1 -- 172.21.15.37:0/2495034885 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8880fe550 con 0x7fa8881037a0 2024-03-29T08:24:32.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:32.220+0000 7fa87e000700 1 -- 172.21.15.37:0/2495034885 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fa874009680 con 0x7fa8881037a0 2024-03-29T08:24:32.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:32.220+0000 7fa87e000700 1 --2- 172.21.15.37:0/2495034885 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fa86c06cc30 0x7fa86c06f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:32.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:32.220+0000 7fa87e000700 1 -- 172.21.15.37:0/2495034885 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fa87408bcf0 con 0x7fa8881037a0 2024-03-29T08:24:32.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:32.221+0000 7fa885e00700 1 --2- 172.21.15.37:0/2495034885 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fa86c06cc30 0x7fa86c06f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:32.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:32.221+0000 7fa885e00700 1 --2- 172.21.15.37:0/2495034885 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fa86c06cc30 0x7fa86c06f0e0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fa87800b840 tx=0x7fa878009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:32.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:32.225+0000 7fa87e000700 1 -- 172.21.15.37:0/2495034885 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fa874055cb0 con 0x7fa8881037a0 2024-03-29T08:24:32.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:32.520+0000 7fa88ce00700 1 -- 172.21.15.37:0/2495034885 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fa88805f5f0 con 0x7fa8881037a0 2024-03-29T08:24:32.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:32.520+0000 7fa87e000700 1 -- 172.21.15.37:0/2495034885 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7fa88805f5f0 con 0x7fa8881037a0 2024-03-29T08:24:32.520 INFO:teuthology.orchestra.run.smithi037.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-03-29T08:24:32.520 INFO:teuthology.orchestra.run.smithi037.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-03-29T08:24:32.520 INFO:teuthology.orchestra.run.smithi037.stdout: fs cephfs is degraded 2024-03-29T08:24:32.521 INFO:teuthology.orchestra.run.smithi037.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-03-29T08:24:32.521 INFO:teuthology.orchestra.run.smithi037.stdout: fs cephfs is offline because no MDS is active for it. 2024-03-29T08:24:32.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:32.522+0000 7fa88ce00700 1 -- 172.21.15.37:0/2495034885 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fa86c06cc30 msgr2=0x7fa86c06f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:32.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:32.522+0000 7fa88ce00700 1 --2- 172.21.15.37:0/2495034885 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fa86c06cc30 0x7fa86c06f0e0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fa87800b840 tx=0x7fa878009e30 comp rx=0 tx=0).stop 2024-03-29T08:24:32.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:32.522+0000 7fa88ce00700 1 -- 172.21.15.37:0/2495034885 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa8881037a0 msgr2=0x7fa888070800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:32.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:32.522+0000 7fa88ce00700 1 --2- 172.21.15.37:0/2495034885 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa8881037a0 0x7fa888070800 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fa874002a50 tx=0x7fa874004600 comp rx=0 tx=0).stop 2024-03-29T08:24:32.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:32.523+0000 7fa88ce00700 1 -- 172.21.15.37:0/2495034885 shutdown_connections 2024-03-29T08:24:32.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:32.523+0000 7fa88ce00700 1 --2- 172.21.15.37:0/2495034885 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fa86c06cc30 0x7fa86c06f0e0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:32.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:32.523+0000 7fa88ce00700 1 --2- 172.21.15.37:0/2495034885 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa888102e00 0x7fa8880702c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:32.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:32.523+0000 7fa88ce00700 1 --2- 172.21.15.37:0/2495034885 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa8881037a0 0x7fa888070800 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:32.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:32.523+0000 7fa88ce00700 1 -- 172.21.15.37:0/2495034885 >> 172.21.15.37:0/2495034885 conn(0x7fa8880f89b0 msgr2=0x7fa8880f9a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:24:32.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:32.523+0000 7fa88ce00700 1 -- 172.21.15.37:0/2495034885 shutdown_connections 2024-03-29T08:24:32.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:24:32.523+0000 7fa88ce00700 1 -- 172.21.15.37:0/2495034885 wait complete. 2024-03-29T08:24:32.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:24:32 smithi148 ceph-mon[106692]: from='client.34792 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:24:32.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:24:32 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/2340082659' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:24:32.557 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:24:32 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/2240477854' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T08:24:32.557 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:24:32 smithi148 ceph-mon[106692]: pgmap v409: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:24:32.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:24:32 smithi037 ceph-mon[142325]: from='client.34792 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:24:32.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:24:32 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/2340082659' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:24:32.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:24:32 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/2240477854' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T08:24:32.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:24:32 smithi037 ceph-mon[142325]: pgmap v409: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:24:33.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:24:33 smithi037 ceph-mon[142325]: from='client.34804 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:24:33.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:24:33 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/2495034885' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T08:24:33.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:24:33 smithi148 ceph-mon[106692]: from='client.34804 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:24:33.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:24:33 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/2495034885' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T08:24:34.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:24:34 smithi148 ceph-mon[106692]: pgmap v410: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:24:34.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:24:34 smithi037 ceph-mon[142325]: pgmap v410: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:24:36.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:24:35 smithi148 ceph-mon[106692]: pgmap v411: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:24:36.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:24:35 smithi037 ceph-mon[142325]: pgmap v411: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:24:38.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:24:37 smithi148 ceph-mon[106692]: pgmap v412: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:24:38.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:24:37 smithi037 ceph-mon[142325]: pgmap v412: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:24:40.558 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:24:40 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:24:40.558 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:24:40 smithi148 ceph-mon[106692]: pgmap v413: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:24:40.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:24:40 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:24:40.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:24:40 smithi037 ceph-mon[142325]: pgmap v413: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:24:42.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:24:41 smithi148 ceph-mon[106692]: pgmap v414: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:24:42.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:24:41 smithi037 ceph-mon[142325]: pgmap v414: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:24:44.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:24:43 smithi148 ceph-mon[106692]: pgmap v415: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:24:44.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:24:43 smithi037 ceph-mon[142325]: pgmap v415: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:24:46.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:24:45 smithi148 ceph-mon[106692]: pgmap v416: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:24:46.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:24:45 smithi037 ceph-mon[142325]: pgmap v416: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:24:48.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:24:47 smithi148 ceph-mon[106692]: pgmap v417: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:24:48.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:24:47 smithi037 ceph-mon[142325]: pgmap v417: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:24:50.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:24:50 smithi148 ceph-mon[106692]: pgmap v418: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:24:50.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:24:50 smithi037 ceph-mon[142325]: pgmap v418: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:24:52.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:24:51 smithi148 ceph-mon[106692]: pgmap v419: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:24:52.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:24:51 smithi037 ceph-mon[142325]: pgmap v419: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:24:54.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:24:53 smithi148 ceph-mon[106692]: pgmap v420: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:24:54.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:24:53 smithi037 ceph-mon[142325]: pgmap v420: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:24:55.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:24:55 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:24:55.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:24:55 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:24:56.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:24:56 smithi148 ceph-mon[106692]: pgmap v421: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:24:56.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:24:56 smithi037 ceph-mon[142325]: pgmap v421: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:24:58.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:24:57 smithi148 ceph-mon[106692]: pgmap v422: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:24:58.178 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:24:57 smithi037 ceph-mon[142325]: pgmap v422: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:25:00.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:25:00 smithi148 ceph-mon[106692]: pgmap v423: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:25:00.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:25:00 smithi037 ceph-mon[142325]: pgmap v423: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:25:02.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:25:01 smithi148 ceph-mon[106692]: pgmap v424: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:25:02.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:25:01 smithi037 ceph-mon[142325]: pgmap v424: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:25:02.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:02.677+0000 7fba68a00700 1 -- 172.21.15.37:0/1443445486 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fba640fde90 msgr2=0x7fba640fe260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:25:02.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:02.677+0000 7fba68a00700 1 --2- 172.21.15.37:0/1443445486 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fba640fde90 0x7fba640fe260 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fba54004940 tx=0x7fba54009e30 comp rx=0 tx=0).stop 2024-03-29T08:25:02.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:02.677+0000 7fba68a00700 1 -- 172.21.15.37:0/1443445486 shutdown_connections 2024-03-29T08:25:02.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:02.677+0000 7fba68a00700 1 --2- 172.21.15.37:0/1443445486 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fba640fe830 0x7fba64106f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:02.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:02.677+0000 7fba68a00700 1 --2- 172.21.15.37:0/1443445486 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fba640fde90 0x7fba640fe260 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:02.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:02.677+0000 7fba68a00700 1 -- 172.21.15.37:0/1443445486 >> 172.21.15.37:0/1443445486 conn(0x7fba64072bc0 msgr2=0x7fba64072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:25:02.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:02.677+0000 7fba68a00700 1 -- 172.21.15.37:0/1443445486 shutdown_connections 2024-03-29T08:25:02.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:02.677+0000 7fba68a00700 1 -- 172.21.15.37:0/1443445486 wait complete. 2024-03-29T08:25:02.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:02.678+0000 7fba68a00700 1 Processor -- start 2024-03-29T08:25:02.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:02.678+0000 7fba68a00700 1 -- start start 2024-03-29T08:25:02.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:02.679+0000 7fba68a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fba640fde90 0x7fba640fbd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:02.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:02.679+0000 7fba68a00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fba640fe830 0x7fba640fc290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:02.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:02.679+0000 7fba68a00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba640fc820 con 0x7fba640fde90 2024-03-29T08:25:02.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:02.679+0000 7fba68a00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba640fc960 con 0x7fba640fe830 2024-03-29T08:25:02.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:02.679+0000 7fba61e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fba640fde90 0x7fba640fbd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:02.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:02.679+0000 7fba61e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fba640fde90 0x7fba640fbd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43826/0 (socket says 172.21.15.37:43826) 2024-03-29T08:25:02.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:02.679+0000 7fba61e00700 1 -- 172.21.15.37:0/2118201404 learned_addr learned my addr 172.21.15.37:0/2118201404 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:25:02.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:02.679+0000 7fba61400700 1 --2- 172.21.15.37:0/2118201404 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fba640fe830 0x7fba640fc290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:02.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:02.680+0000 7fba61e00700 1 -- 172.21.15.37:0/2118201404 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fba640fe830 msgr2=0x7fba640fc290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:25:02.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:02.680+0000 7fba61e00700 1 --2- 172.21.15.37:0/2118201404 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fba640fe830 0x7fba640fc290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:02.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:02.680+0000 7fba61e00700 1 -- 172.21.15.37:0/2118201404 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba54005040 con 0x7fba640fde90 2024-03-29T08:25:02.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:02.680+0000 7fba61400700 1 --2- 172.21.15.37:0/2118201404 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fba640fe830 0x7fba640fc290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:25:02.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:02.680+0000 7fba61e00700 1 --2- 172.21.15.37:0/2118201404 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fba640fde90 0x7fba640fbd50 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fba54000c00 tx=0x7fba54009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:25:02.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:02.680+0000 7fba5a000700 1 -- 172.21.15.37:0/2118201404 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fba5400e040 con 0x7fba640fde90 2024-03-29T08:25:02.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:02.680+0000 7fba5a000700 1 -- 172.21.15.37:0/2118201404 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fba54013420 con 0x7fba640fde90 2024-03-29T08:25:02.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:02.680+0000 7fba5a000700 1 -- 172.21.15.37:0/2118201404 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fba5401c630 con 0x7fba640fde90 2024-03-29T08:25:02.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:02.680+0000 7fba68a00700 1 -- 172.21.15.37:0/2118201404 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba640fcbb0 con 0x7fba640fde90 2024-03-29T08:25:02.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:02.680+0000 7fba68a00700 1 -- 172.21.15.37:0/2118201404 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba6411a1a0 con 0x7fba640fde90 2024-03-29T08:25:02.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:02.681+0000 7fba68a00700 1 -- 172.21.15.37:0/2118201404 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba64065e30 con 0x7fba640fde90 2024-03-29T08:25:02.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:02.682+0000 7fba5a000700 1 -- 172.21.15.37:0/2118201404 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fba54013970 con 0x7fba640fde90 2024-03-29T08:25:02.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:02.684+0000 7fba5a000700 1 --2- 172.21.15.37:0/2118201404 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fba4806cc30 0x7fba4806f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:02.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:02.684+0000 7fba5a000700 1 -- 172.21.15.37:0/2118201404 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fba5408d230 con 0x7fba640fde90 2024-03-29T08:25:02.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:02.684+0000 7fba61400700 1 --2- 172.21.15.37:0/2118201404 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fba4806cc30 0x7fba4806f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:02.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:02.685+0000 7fba61400700 1 --2- 172.21.15.37:0/2118201404 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fba4806cc30 0x7fba4806f0e0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fba50009fd0 tx=0x7fba50009ee0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:25:02.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:02.689+0000 7fba5a000700 1 -- 172.21.15.37:0/2118201404 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fba54057270 con 0x7fba640fde90 2024-03-29T08:25:02.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:02.921+0000 7fba68a00700 1 -- 172.21.15.37:0/2118201404 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fba640fd490 con 0x7fba4806cc30 2024-03-29T08:25:02.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:02.923+0000 7fba5a000700 1 -- 172.21.15.37:0/2118201404 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fba640fd490 con 0x7fba4806cc30 2024-03-29T08:25:02.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:02.926+0000 7fba68a00700 1 -- 172.21.15.37:0/2118201404 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fba4806cc30 msgr2=0x7fba4806f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:25:02.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:02.926+0000 7fba68a00700 1 --2- 172.21.15.37:0/2118201404 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fba4806cc30 0x7fba4806f0e0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fba50009fd0 tx=0x7fba50009ee0 comp rx=0 tx=0).stop 2024-03-29T08:25:02.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:02.926+0000 7fba68a00700 1 -- 172.21.15.37:0/2118201404 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fba640fde90 msgr2=0x7fba640fbd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:25:02.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:02.926+0000 7fba68a00700 1 --2- 172.21.15.37:0/2118201404 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fba640fde90 0x7fba640fbd50 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fba54000c00 tx=0x7fba54009e30 comp rx=0 tx=0).stop 2024-03-29T08:25:02.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:02.926+0000 7fba68a00700 1 -- 172.21.15.37:0/2118201404 shutdown_connections 2024-03-29T08:25:02.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:02.926+0000 7fba68a00700 1 --2- 172.21.15.37:0/2118201404 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fba4806cc30 0x7fba4806f0e0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:02.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:02.926+0000 7fba68a00700 1 --2- 172.21.15.37:0/2118201404 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fba640fde90 0x7fba640fbd50 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:02.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:02.926+0000 7fba68a00700 1 --2- 172.21.15.37:0/2118201404 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fba640fe830 0x7fba640fc290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:02.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:02.926+0000 7fba68a00700 1 -- 172.21.15.37:0/2118201404 >> 172.21.15.37:0/2118201404 conn(0x7fba64072bc0 msgr2=0x7fba640ff3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:25:02.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:02.926+0000 7fba68a00700 1 -- 172.21.15.37:0/2118201404 shutdown_connections 2024-03-29T08:25:02.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:02.926+0000 7fba68a00700 1 -- 172.21.15.37:0/2118201404 wait complete. 2024-03-29T08:25:02.940 INFO:teuthology.orchestra.run.smithi037.stdout:true 2024-03-29T08:25:03.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:03.064+0000 7fbded000700 1 -- 172.21.15.37:0/615029369 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbde81037a0 msgr2=0x7fbde80fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:25:03.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:03.064+0000 7fbded000700 1 --2- 172.21.15.37:0/615029369 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbde81037a0 0x7fbde80fce10 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fbdd4004940 tx=0x7fbdd4009e30 comp rx=0 tx=0).stop 2024-03-29T08:25:03.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:03.064+0000 7fbded000700 1 -- 172.21.15.37:0/615029369 shutdown_connections 2024-03-29T08:25:03.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:03.064+0000 7fbded000700 1 --2- 172.21.15.37:0/615029369 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbde81037a0 0x7fbde80fce10 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:03.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:03.064+0000 7fbded000700 1 --2- 172.21.15.37:0/615029369 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbde8102e00 0x7fbde81031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:03.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:03.064+0000 7fbded000700 1 -- 172.21.15.37:0/615029369 >> 172.21.15.37:0/615029369 conn(0x7fbde80f89b0 msgr2=0x7fbde80fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:25:03.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:03.064+0000 7fbded000700 1 -- 172.21.15.37:0/615029369 shutdown_connections 2024-03-29T08:25:03.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:03.064+0000 7fbded000700 1 -- 172.21.15.37:0/615029369 wait complete. 2024-03-29T08:25:03.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:03.065+0000 7fbded000700 1 Processor -- start 2024-03-29T08:25:03.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:03.065+0000 7fbded000700 1 -- start start 2024-03-29T08:25:03.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:03.065+0000 7fbded000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbde8102e00 0x7fbde8115d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:03.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:03.065+0000 7fbded000700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbde81037a0 0x7fbde8116260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:03.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:03.065+0000 7fbded000700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbde81168b0 con 0x7fbde8102e00 2024-03-29T08:25:03.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:03.065+0000 7fbded000700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbde810fc80 con 0x7fbde81037a0 2024-03-29T08:25:03.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:03.066+0000 7fbde5e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbde8102e00 0x7fbde8115d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:03.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:03.066+0000 7fbde5e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbde8102e00 0x7fbde8115d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43846/0 (socket says 172.21.15.37:43846) 2024-03-29T08:25:03.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:03.066+0000 7fbde5e00700 1 -- 172.21.15.37:0/4175091483 learned_addr learned my addr 172.21.15.37:0/4175091483 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:25:03.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:03.066+0000 7fbde5400700 1 --2- 172.21.15.37:0/4175091483 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbde81037a0 0x7fbde8116260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:03.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:03.066+0000 7fbde5e00700 1 -- 172.21.15.37:0/4175091483 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbde81037a0 msgr2=0x7fbde8116260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:25:03.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:03.066+0000 7fbde5e00700 1 --2- 172.21.15.37:0/4175091483 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbde81037a0 0x7fbde8116260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:03.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:03.066+0000 7fbde5e00700 1 -- 172.21.15.37:0/4175091483 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdd4005040 con 0x7fbde8102e00 2024-03-29T08:25:03.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:03.066+0000 7fbde5400700 1 --2- 172.21.15.37:0/4175091483 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbde81037a0 0x7fbde8116260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:25:03.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:03.066+0000 7fbde5e00700 1 --2- 172.21.15.37:0/4175091483 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbde8102e00 0x7fbde8115d20 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fbdd8002a50 tx=0x7fbdd80046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:25:03.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:03.067+0000 7fbdde000700 1 -- 172.21.15.37:0/4175091483 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbdd8007cc0 con 0x7fbde8102e00 2024-03-29T08:25:03.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:03.067+0000 7fbdde000700 1 -- 172.21.15.37:0/4175091483 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fbdd800f040 con 0x7fbde8102e00 2024-03-29T08:25:03.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:03.067+0000 7fbded000700 1 -- 172.21.15.37:0/4175091483 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbde810ff60 con 0x7fbde8102e00 2024-03-29T08:25:03.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:03.067+0000 7fbdde000700 1 -- 172.21.15.37:0/4175091483 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbdd8009540 con 0x7fbde8102e00 2024-03-29T08:25:03.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:03.067+0000 7fbded000700 1 -- 172.21.15.37:0/4175091483 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbde8110520 con 0x7fbde8102e00 2024-03-29T08:25:03.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:03.068+0000 7fbded000700 1 -- 172.21.15.37:0/4175091483 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbde80fe550 con 0x7fbde8102e00 2024-03-29T08:25:03.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:03.069+0000 7fbdde000700 1 -- 172.21.15.37:0/4175091483 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fbdd80096a0 con 0x7fbde8102e00 2024-03-29T08:25:03.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:03.070+0000 7fbdde000700 1 --2- 172.21.15.37:0/4175091483 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fbdcc06cc30 0x7fbdcc06f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:03.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:03.070+0000 7fbdde000700 1 -- 172.21.15.37:0/4175091483 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fbdd808bee0 con 0x7fbde8102e00 2024-03-29T08:25:03.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:03.070+0000 7fbde5400700 1 --2- 172.21.15.37:0/4175091483 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fbdcc06cc30 0x7fbdcc06f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:03.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:03.071+0000 7fbde5400700 1 --2- 172.21.15.37:0/4175091483 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fbdcc06cc30 0x7fbdcc06f0e0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fbde81115c0 tx=0x7fbdd4009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:25:03.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:03.074+0000 7fbdde000700 1 -- 172.21.15.37:0/4175091483 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fbdd80598b0 con 0x7fbde8102e00 2024-03-29T08:25:03.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:03.305+0000 7fbded000700 1 -- 172.21.15.37:0/4175091483 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fbde804bf40 con 0x7fbdcc06cc30 2024-03-29T08:25:03.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:03.307+0000 7fbdde000700 1 -- 172.21.15.37:0/4175091483 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fbde804bf40 con 0x7fbdcc06cc30 2024-03-29T08:25:03.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:03.309+0000 7fbded000700 1 -- 172.21.15.37:0/4175091483 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fbdcc06cc30 msgr2=0x7fbdcc06f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:25:03.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:03.309+0000 7fbded000700 1 --2- 172.21.15.37:0/4175091483 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fbdcc06cc30 0x7fbdcc06f0e0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fbde81115c0 tx=0x7fbdd4009e30 comp rx=0 tx=0).stop 2024-03-29T08:25:03.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:03.309+0000 7fbded000700 1 -- 172.21.15.37:0/4175091483 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbde8102e00 msgr2=0x7fbde8115d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:25:03.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:03.309+0000 7fbded000700 1 --2- 172.21.15.37:0/4175091483 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbde8102e00 0x7fbde8115d20 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fbdd8002a50 tx=0x7fbdd80046b0 comp rx=0 tx=0).stop 2024-03-29T08:25:03.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:03.309+0000 7fbded000700 1 -- 172.21.15.37:0/4175091483 shutdown_connections 2024-03-29T08:25:03.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:03.309+0000 7fbded000700 1 --2- 172.21.15.37:0/4175091483 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fbdcc06cc30 0x7fbdcc06f0e0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:03.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:03.309+0000 7fbded000700 1 --2- 172.21.15.37:0/4175091483 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbde8102e00 0x7fbde8115d20 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:03.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:03.309+0000 7fbded000700 1 --2- 172.21.15.37:0/4175091483 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbde81037a0 0x7fbde8116260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:03.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:03.309+0000 7fbded000700 1 -- 172.21.15.37:0/4175091483 >> 172.21.15.37:0/4175091483 conn(0x7fbde80f89b0 msgr2=0x7fbde80f99f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:25:03.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:03.309+0000 7fbded000700 1 -- 172.21.15.37:0/4175091483 shutdown_connections 2024-03-29T08:25:03.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:03.309+0000 7fbded000700 1 -- 172.21.15.37:0/4175091483 wait complete. 2024-03-29T08:25:03.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:03.447+0000 7f79c9e00700 1 -- 172.21.15.37:0/2659027857 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f79c4102e00 msgr2=0x7f79c41031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:25:03.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:03.447+0000 7f79c9e00700 1 --2- 172.21.15.37:0/2659027857 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f79c4102e00 0x7f79c41031d0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f79b4004940 tx=0x7f79b4009e30 comp rx=0 tx=0).stop 2024-03-29T08:25:03.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:03.447+0000 7f79c9e00700 1 -- 172.21.15.37:0/2659027857 shutdown_connections 2024-03-29T08:25:03.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:03.447+0000 7f79c9e00700 1 --2- 172.21.15.37:0/2659027857 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f79c41037a0 0x7f79c40fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:03.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:03.447+0000 7f79c9e00700 1 --2- 172.21.15.37:0/2659027857 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f79c4102e00 0x7f79c41031d0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:03.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:03.447+0000 7f79c9e00700 1 -- 172.21.15.37:0/2659027857 >> 172.21.15.37:0/2659027857 conn(0x7f79c40f89b0 msgr2=0x7f79c40fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:25:03.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:03.447+0000 7f79c9e00700 1 -- 172.21.15.37:0/2659027857 shutdown_connections 2024-03-29T08:25:03.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:03.447+0000 7f79c9e00700 1 -- 172.21.15.37:0/2659027857 wait complete. 2024-03-29T08:25:03.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:03.448+0000 7f79c9e00700 1 Processor -- start 2024-03-29T08:25:03.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:03.448+0000 7f79c9e00700 1 -- start start 2024-03-29T08:25:03.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:03.449+0000 7f79c9e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f79c4102e00 0x7f79c4115c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:03.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:03.449+0000 7f79c9e00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f79c41037a0 0x7f79c4116180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:03.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:03.449+0000 7f79c9e00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79c41167d0 con 0x7f79c4102e00 2024-03-29T08:25:03.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:03.449+0000 7f79c9e00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79c410fba0 con 0x7f79c41037a0 2024-03-29T08:25:03.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:03.449+0000 7f79c2a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f79c4102e00 0x7f79c4115c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:03.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:03.449+0000 7f79c2a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f79c4102e00 0x7f79c4115c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43858/0 (socket says 172.21.15.37:43858) 2024-03-29T08:25:03.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:03.449+0000 7f79c2a00700 1 -- 172.21.15.37:0/814512901 learned_addr learned my addr 172.21.15.37:0/814512901 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:25:03.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:03.449+0000 7f79c2000700 1 --2- 172.21.15.37:0/814512901 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f79c41037a0 0x7f79c4116180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:03.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:03.449+0000 7f79c2a00700 1 -- 172.21.15.37:0/814512901 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f79c41037a0 msgr2=0x7f79c4116180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:25:03.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:03.449+0000 7f79c2a00700 1 --2- 172.21.15.37:0/814512901 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f79c41037a0 0x7f79c4116180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:03.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:03.449+0000 7f79c2a00700 1 -- 172.21.15.37:0/814512901 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79b4005040 con 0x7f79c4102e00 2024-03-29T08:25:03.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:03.449+0000 7f79c2000700 1 --2- 172.21.15.37:0/814512901 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f79c41037a0 0x7f79c4116180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:25:03.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:03.450+0000 7f79c2a00700 1 --2- 172.21.15.37:0/814512901 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f79c4102e00 0x7f79c4115c40 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f79b4008d80 tx=0x7f79b4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:25:03.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:03.450+0000 7f79baa00700 1 -- 172.21.15.37:0/814512901 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f79b400e030 con 0x7f79c4102e00 2024-03-29T08:25:03.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:03.450+0000 7f79baa00700 1 -- 172.21.15.37:0/814512901 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f79b4013de0 con 0x7f79c4102e00 2024-03-29T08:25:03.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:03.450+0000 7f79baa00700 1 -- 172.21.15.37:0/814512901 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f79b401cd60 con 0x7f79c4102e00 2024-03-29T08:25:03.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:03.450+0000 7f79c9e00700 1 -- 172.21.15.37:0/814512901 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79c410fe20 con 0x7f79c4102e00 2024-03-29T08:25:03.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:03.450+0000 7f79c9e00700 1 -- 172.21.15.37:0/814512901 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79c41103e0 con 0x7f79c4102e00 2024-03-29T08:25:03.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:03.451+0000 7f79c9e00700 1 -- 172.21.15.37:0/814512901 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79c40fe550 con 0x7f79c4102e00 2024-03-29T08:25:03.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:03.452+0000 7f79baa00700 1 -- 172.21.15.37:0/814512901 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f79b401c590 con 0x7f79c4102e00 2024-03-29T08:25:03.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:03.453+0000 7f79baa00700 1 --2- 172.21.15.37:0/814512901 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f79a806cc30 0x7f79a806f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:03.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:03.453+0000 7f79baa00700 1 -- 172.21.15.37:0/814512901 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f79b408d8c0 con 0x7f79c4102e00 2024-03-29T08:25:03.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:03.453+0000 7f79c2000700 1 --2- 172.21.15.37:0/814512901 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f79a806cc30 0x7f79a806f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:03.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:03.454+0000 7f79c2000700 1 --2- 172.21.15.37:0/814512901 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f79a806cc30 0x7f79a806f0e0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f79b0004600 tx=0x7f79b000b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:25:03.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:03.457+0000 7f79baa00700 1 -- 172.21.15.37:0/814512901 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f79b405b390 con 0x7f79c4102e00 2024-03-29T08:25:03.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:03.680+0000 7f79c9e00700 1 -- 172.21.15.37:0/814512901 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f79c404bf70 con 0x7f79a806cc30 2024-03-29T08:25:03.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:03.690+0000 7f79baa00700 1 -- 172.21.15.37:0/814512901 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f79b0009420 con 0x7f79a806cc30 2024-03-29T08:25:03.691 INFO:teuthology.orchestra.run.smithi037.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-29T08:25:03.691 INFO:teuthology.orchestra.run.smithi037.stdout:alertmanager.smithi037 smithi037 *:9093,9094 running (14m) 7m ago 27m 14.9M - 0.25.0 c8568f914cd2 2f9f9746ed0f 2024-03-29T08:25:03.691 INFO:teuthology.orchestra.run.smithi037.stdout:crash.smithi037 smithi037 running (11m) 7m ago 28m 6647k - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 178d416a3186 2024-03-29T08:25:03.691 INFO:teuthology.orchestra.run.smithi037.stdout:crash.smithi148 smithi148 running (11m) 6m ago 26m 6656k - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 23666989d58f 2024-03-29T08:25:03.691 INFO:teuthology.orchestra.run.smithi037.stdout:grafana.smithi037 smithi037 *:3000 running (14m) 7m ago 27m 71.9M - 9.4.7 954c08fa6188 6b976fe49952 2024-03-29T08:25:03.691 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi037.iifsmh smithi037 running (7m) 7m ago 21m 10.8M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 2d97f562cc5b 2024-03-29T08:25:03.691 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi037.mmwgxe smithi037 running (7m) 7m ago 21m 10.9M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff ec37037f46e8 2024-03-29T08:25:03.691 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi148.qukpjh smithi148 running (21m) 6m ago 21m 15.3M - 17.2.7-789-g4213e17c cf22eb8683be 8e469782a273 2024-03-29T08:25:03.691 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi148.sqteih smithi148 running (6m) 6m ago 21m 10.7M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 44c15c77037a 2024-03-29T08:25:03.691 INFO:teuthology.orchestra.run.smithi037.stdout:mgr.smithi037.wjikii smithi037 *:8443,9283,8765 running (17m) 7m ago 29m 506M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff c5a1417b9417 2024-03-29T08:25:03.691 INFO:teuthology.orchestra.run.smithi037.stdout:mgr.smithi148.onhgvm smithi148 *:8443,9283,8765 running (16m) 6m ago 26m 450M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff fcfdbce062bd 2024-03-29T08:25:03.691 INFO:teuthology.orchestra.run.smithi037.stdout:mon.smithi037 smithi037 running (13m) 7m ago 29m 54.8M 2048M 18.2.2-1015-gd0e392d9 1cc2f45c17ff eea2060783af 2024-03-29T08:25:03.692 INFO:teuthology.orchestra.run.smithi037.stdout:mon.smithi148 smithi148 running (12m) 6m ago 26m 43.7M 2048M 18.2.2-1015-gd0e392d9 1cc2f45c17ff ad4b2c5748be 2024-03-29T08:25:03.692 INFO:teuthology.orchestra.run.smithi037.stdout:node-exporter.smithi037 smithi037 *:9100 running (15m) 7m ago 28m 7625k - 1.5.0 0da6a335fe13 26b8ae9cb878 2024-03-29T08:25:03.692 INFO:teuthology.orchestra.run.smithi037.stdout:node-exporter.smithi148 smithi148 *:9100 running (15m) 6m ago 26m 8166k - 1.5.0 0da6a335fe13 98414e4e64ed 2024-03-29T08:25:03.692 INFO:teuthology.orchestra.run.smithi037.stdout:osd.0 smithi037 running (11m) 7m ago 25m 104M 1619M 18.2.2-1015-gd0e392d9 1cc2f45c17ff c0c584f6245c 2024-03-29T08:25:03.692 INFO:teuthology.orchestra.run.smithi037.stdout:osd.1 smithi037 running (10m) 7m ago 25m 100M 1619M 18.2.2-1015-gd0e392d9 1cc2f45c17ff a4a2ff4cc83d 2024-03-29T08:25:03.692 INFO:teuthology.orchestra.run.smithi037.stdout:osd.2 smithi037 running (9m) 7m ago 24m 82.3M 1619M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 8b1bc71aa79e 2024-03-29T08:25:03.692 INFO:teuthology.orchestra.run.smithi037.stdout:osd.3 smithi148 running (9m) 6m ago 24m 105M 2642M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 1c656ab8ff86 2024-03-29T08:25:03.692 INFO:teuthology.orchestra.run.smithi037.stdout:osd.4 smithi148 running (8m) 6m ago 23m 97.3M 2642M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 966a9de11d22 2024-03-29T08:25:03.692 INFO:teuthology.orchestra.run.smithi037.stdout:osd.5 smithi148 running (7m) 6m ago 23m 85.8M 2642M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 474c4f9212ce 2024-03-29T08:25:03.692 INFO:teuthology.orchestra.run.smithi037.stdout:prometheus.smithi037 smithi037 *:9095 running (15m) 7m ago 27m 54.4M - 2.43.0 a07b618ecd1d 30c0223ec589 2024-03-29T08:25:03.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:03.693+0000 7f79c9e00700 1 -- 172.21.15.37:0/814512901 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f79a806cc30 msgr2=0x7f79a806f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:25:03.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:03.693+0000 7f79c9e00700 1 --2- 172.21.15.37:0/814512901 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f79a806cc30 0x7f79a806f0e0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f79b0004600 tx=0x7f79b000b040 comp rx=0 tx=0).stop 2024-03-29T08:25:03.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:03.693+0000 7f79c9e00700 1 -- 172.21.15.37:0/814512901 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f79c4102e00 msgr2=0x7f79c4115c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:25:03.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:03.693+0000 7f79c9e00700 1 --2- 172.21.15.37:0/814512901 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f79c4102e00 0x7f79c4115c40 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f79b4008d80 tx=0x7f79b4009e30 comp rx=0 tx=0).stop 2024-03-29T08:25:03.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:03.693+0000 7f79c9e00700 1 -- 172.21.15.37:0/814512901 shutdown_connections 2024-03-29T08:25:03.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:03.693+0000 7f79c9e00700 1 --2- 172.21.15.37:0/814512901 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f79a806cc30 0x7f79a806f0e0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:03.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:03.693+0000 7f79c9e00700 1 --2- 172.21.15.37:0/814512901 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f79c4102e00 0x7f79c4115c40 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:03.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:03.693+0000 7f79c9e00700 1 --2- 172.21.15.37:0/814512901 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f79c41037a0 0x7f79c4116180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:03.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:03.693+0000 7f79c9e00700 1 -- 172.21.15.37:0/814512901 >> 172.21.15.37:0/814512901 conn(0x7f79c40f89b0 msgr2=0x7f79c40f99f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:25:03.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:03.693+0000 7f79c9e00700 1 -- 172.21.15.37:0/814512901 shutdown_connections 2024-03-29T08:25:03.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:03.693+0000 7f79c9e00700 1 -- 172.21.15.37:0/814512901 wait complete. 2024-03-29T08:25:03.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:03.830+0000 7f2680000700 1 -- 172.21.15.37:0/323463001 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2678102e20 msgr2=0x7f26781031f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:25:03.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:03.830+0000 7f2680000700 1 --2- 172.21.15.37:0/323463001 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2678102e20 0x7f26781031f0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f266c004940 tx=0x7f266c009e30 comp rx=0 tx=0).stop 2024-03-29T08:25:03.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:03.831+0000 7f2680000700 1 -- 172.21.15.37:0/323463001 shutdown_connections 2024-03-29T08:25:03.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:03.831+0000 7f2680000700 1 --2- 172.21.15.37:0/323463001 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f26781037c0 0x7f26780fce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:03.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:03.831+0000 7f2680000700 1 --2- 172.21.15.37:0/323463001 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2678102e20 0x7f26781031f0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:03.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:03.831+0000 7f2680000700 1 -- 172.21.15.37:0/323463001 >> 172.21.15.37:0/323463001 conn(0x7f26780f8970 msgr2=0x7f26780fad80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:25:03.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:03.831+0000 7f2680000700 1 -- 172.21.15.37:0/323463001 shutdown_connections 2024-03-29T08:25:03.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:03.831+0000 7f2680000700 1 -- 172.21.15.37:0/323463001 wait complete. 2024-03-29T08:25:03.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:03.831+0000 7f2680000700 1 Processor -- start 2024-03-29T08:25:03.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:03.832+0000 7f2680000700 1 -- start start 2024-03-29T08:25:03.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:03.832+0000 7f2680000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2678102e20 0x7f2678115cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:03.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:03.832+0000 7f2680000700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f26781037c0 0x7f26781161f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:03.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:03.832+0000 7f2680000700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2678116840 con 0x7f2678102e20 2024-03-29T08:25:03.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:03.832+0000 7f2680000700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f267810fc10 con 0x7f26781037c0 2024-03-29T08:25:03.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:03.832+0000 7f267d600700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2678102e20 0x7f2678115cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:03.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:03.832+0000 7f267d600700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2678102e20 0x7f2678115cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43866/0 (socket says 172.21.15.37:43866) 2024-03-29T08:25:03.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:03.832+0000 7f267d600700 1 -- 172.21.15.37:0/1095867024 learned_addr learned my addr 172.21.15.37:0/1095867024 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:25:03.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:03.832+0000 7f267cc00700 1 --2- 172.21.15.37:0/1095867024 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f26781037c0 0x7f26781161f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:03.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:03.833+0000 7f267d600700 1 -- 172.21.15.37:0/1095867024 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f26781037c0 msgr2=0x7f26781161f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:25:03.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:03.833+0000 7f267d600700 1 --2- 172.21.15.37:0/1095867024 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f26781037c0 0x7f26781161f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:03.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:03.833+0000 7f267d600700 1 -- 172.21.15.37:0/1095867024 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f266c005040 con 0x7f2678102e20 2024-03-29T08:25:03.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:03.833+0000 7f267d600700 1 --2- 172.21.15.37:0/1095867024 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2678102e20 0x7f2678115cb0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f266c008d80 tx=0x7f266c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:25:03.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:03.833+0000 7f2675600700 1 -- 172.21.15.37:0/1095867024 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f266c00e030 con 0x7f2678102e20 2024-03-29T08:25:03.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:03.833+0000 7f2675600700 1 -- 172.21.15.37:0/1095867024 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f266c013de0 con 0x7f2678102e20 2024-03-29T08:25:03.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:03.833+0000 7f2675600700 1 -- 172.21.15.37:0/1095867024 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f266c01cd00 con 0x7f2678102e20 2024-03-29T08:25:03.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:03.833+0000 7f2680000700 1 -- 172.21.15.37:0/1095867024 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f267810fe90 con 0x7f2678102e20 2024-03-29T08:25:03.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:03.833+0000 7f2680000700 1 -- 172.21.15.37:0/1095867024 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2678110450 con 0x7f2678102e20 2024-03-29T08:25:03.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:03.834+0000 7f2680000700 1 -- 172.21.15.37:0/1095867024 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26780fe570 con 0x7f2678102e20 2024-03-29T08:25:03.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:03.835+0000 7f2675600700 1 -- 172.21.15.37:0/1095867024 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f266c013930 con 0x7f2678102e20 2024-03-29T08:25:03.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:03.836+0000 7f2675600700 1 --2- 172.21.15.37:0/1095867024 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f266006cc30 0x7f266006f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:03.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:03.836+0000 7f2675600700 1 -- 172.21.15.37:0/1095867024 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f266c08d870 con 0x7f2678102e20 2024-03-29T08:25:03.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:03.837+0000 7f267cc00700 1 --2- 172.21.15.37:0/1095867024 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f266006cc30 0x7f266006f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:03.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:03.837+0000 7f267cc00700 1 --2- 172.21.15.37:0/1095867024 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f266006cc30 0x7f266006f0e0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f2678111550 tx=0x7f266400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:25:03.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:03.841+0000 7f2675600700 1 -- 172.21.15.37:0/1095867024 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f266c05b340 con 0x7f2678102e20 2024-03-29T08:25:04.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:25:03 smithi148 ceph-mon[106692]: from='client.34812 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:25:04.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:25:03 smithi148 ceph-mon[106692]: from='client.34816 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:25:04.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:25:03 smithi148 ceph-mon[106692]: from='client.34820 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:25:04.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:25:03 smithi148 ceph-mon[106692]: pgmap v425: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:25:04.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:04.141+0000 7f2680000700 1 -- 172.21.15.37:0/1095867024 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f267804bed0 con 0x7f2678102e20 2024-03-29T08:25:04.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:04.141+0000 7f2675600700 1 -- 172.21.15.37:0/1095867024 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+847 (secure 0 0 0) 0x7f266c01ce60 con 0x7f2678102e20 2024-03-29T08:25:04.142 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-29T08:25:04.142 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": { 2024-03-29T08:25:04.142 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 2 2024-03-29T08:25:04.142 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:25:04.142 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": { 2024-03-29T08:25:04.142 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 2 2024-03-29T08:25:04.142 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:25:04.142 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": { 2024-03-29T08:25:04.142 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 6 2024-03-29T08:25:04.142 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:25:04.142 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": { 2024-03-29T08:25:04.142 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 17.2.7-789-g4213e17c (4213e17c4de67016ed14f55d22503f4923b3a901) quincy (stable)": 1, 2024-03-29T08:25:04.142 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 3 2024-03-29T08:25:04.143 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:25:04.143 INFO:teuthology.orchestra.run.smithi037.stdout: "overall": { 2024-03-29T08:25:04.143 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 17.2.7-789-g4213e17c (4213e17c4de67016ed14f55d22503f4923b3a901) quincy (stable)": 1, 2024-03-29T08:25:04.143 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 13 2024-03-29T08:25:04.143 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-29T08:25:04.143 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-29T08:25:04.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:04.144+0000 7f2680000700 1 -- 172.21.15.37:0/1095867024 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f266006cc30 msgr2=0x7f266006f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:25:04.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:04.144+0000 7f2680000700 1 --2- 172.21.15.37:0/1095867024 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f266006cc30 0x7f266006f0e0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f2678111550 tx=0x7f266400b040 comp rx=0 tx=0).stop 2024-03-29T08:25:04.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:04.144+0000 7f2680000700 1 -- 172.21.15.37:0/1095867024 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2678102e20 msgr2=0x7f2678115cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:25:04.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:04.144+0000 7f2680000700 1 --2- 172.21.15.37:0/1095867024 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2678102e20 0x7f2678115cb0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f266c008d80 tx=0x7f266c009e30 comp rx=0 tx=0).stop 2024-03-29T08:25:04.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:04.144+0000 7f2680000700 1 -- 172.21.15.37:0/1095867024 shutdown_connections 2024-03-29T08:25:04.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:04.144+0000 7f2680000700 1 --2- 172.21.15.37:0/1095867024 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f266006cc30 0x7f266006f0e0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:04.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:04.144+0000 7f2680000700 1 --2- 172.21.15.37:0/1095867024 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2678102e20 0x7f2678115cb0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:04.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:04.144+0000 7f2680000700 1 --2- 172.21.15.37:0/1095867024 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f26781037c0 0x7f26781161f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:04.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:04.144+0000 7f2680000700 1 -- 172.21.15.37:0/1095867024 >> 172.21.15.37:0/1095867024 conn(0x7f26780f8970 msgr2=0x7f2678070480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:25:04.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:04.144+0000 7f2680000700 1 -- 172.21.15.37:0/1095867024 shutdown_connections 2024-03-29T08:25:04.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:04.144+0000 7f2680000700 1 -- 172.21.15.37:0/1095867024 wait complete. 2024-03-29T08:25:04.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:25:03 smithi037 ceph-mon[142325]: from='client.34812 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:25:04.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:25:03 smithi037 ceph-mon[142325]: from='client.34816 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:25:04.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:25:03 smithi037 ceph-mon[142325]: from='client.34820 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:25:04.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:25:03 smithi037 ceph-mon[142325]: pgmap v425: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:25:04.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:04.283+0000 7fd906800700 1 -- 172.21.15.37:0/1674548745 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd900070c10 msgr2=0x7fd900109180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:25:04.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:04.283+0000 7fd906800700 1 --2- 172.21.15.37:0/1674548745 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd900070c10 0x7fd900109180 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fd8ec004970 tx=0x7fd8ec009e30 comp rx=0 tx=0).stop 2024-03-29T08:25:04.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:04.283+0000 7fd906800700 1 -- 172.21.15.37:0/1674548745 shutdown_connections 2024-03-29T08:25:04.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:04.283+0000 7fd906800700 1 --2- 172.21.15.37:0/1674548745 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd900070c10 0x7fd900109180 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:04.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:04.283+0000 7fd906800700 1 --2- 172.21.15.37:0/1674548745 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd900070300 0x7fd9000706d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:04.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:04.283+0000 7fd906800700 1 -- 172.21.15.37:0/1674548745 >> 172.21.15.37:0/1674548745 conn(0x7fd9000f89a0 msgr2=0x7fd9000fadb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:25:04.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:04.283+0000 7fd906800700 1 -- 172.21.15.37:0/1674548745 shutdown_connections 2024-03-29T08:25:04.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:04.283+0000 7fd906800700 1 -- 172.21.15.37:0/1674548745 wait complete. 2024-03-29T08:25:04.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:04.284+0000 7fd906800700 1 Processor -- start 2024-03-29T08:25:04.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:04.284+0000 7fd906800700 1 -- start start 2024-03-29T08:25:04.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:04.285+0000 7fd906800700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd900070300 0x7fd900115ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:04.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:04.285+0000 7fd906800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd900070c10 0x7fd9001161e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:04.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:04.285+0000 7fd906800700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd900116830 con 0x7fd900070c10 2024-03-29T08:25:04.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:04.285+0000 7fd906800700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd90010fc00 con 0x7fd900070300 2024-03-29T08:25:04.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:04.285+0000 7fd8fea00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd900070c10 0x7fd9001161e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:04.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:04.285+0000 7fd8fea00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd900070c10 0x7fd9001161e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43880/0 (socket says 172.21.15.37:43880) 2024-03-29T08:25:04.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:04.285+0000 7fd8fea00700 1 -- 172.21.15.37:0/898805722 learned_addr learned my addr 172.21.15.37:0/898805722 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:25:04.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:04.285+0000 7fd8fea00700 1 -- 172.21.15.37:0/898805722 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd900070300 msgr2=0x7fd900115ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:25:04.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:04.285+0000 7fd8fea00700 1 --2- 172.21.15.37:0/898805722 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd900070300 0x7fd900115ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:04.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:04.285+0000 7fd8fea00700 1 -- 172.21.15.37:0/898805722 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8ec005040 con 0x7fd900070c10 2024-03-29T08:25:04.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:04.286+0000 7fd8fea00700 1 --2- 172.21.15.37:0/898805722 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd900070c10 0x7fd9001161e0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fd8ec005010 tx=0x7fd8ec009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:25:04.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:04.286+0000 7fd8f7400700 1 -- 172.21.15.37:0/898805722 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd8ec00e040 con 0x7fd900070c10 2024-03-29T08:25:04.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:04.286+0000 7fd8f7400700 1 -- 172.21.15.37:0/898805722 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd8ec004290 con 0x7fd900070c10 2024-03-29T08:25:04.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:04.286+0000 7fd906800700 1 -- 172.21.15.37:0/898805722 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd90010fe80 con 0x7fd900070c10 2024-03-29T08:25:04.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:04.286+0000 7fd8f7400700 1 -- 172.21.15.37:0/898805722 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd8ec013e40 con 0x7fd900070c10 2024-03-29T08:25:04.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:04.286+0000 7fd906800700 1 -- 172.21.15.37:0/898805722 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9001103c0 con 0x7fd900070c10 2024-03-29T08:25:04.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:04.287+0000 7fd906800700 1 -- 172.21.15.37:0/898805722 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd900071a40 con 0x7fd900070c10 2024-03-29T08:25:04.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:04.288+0000 7fd8f7400700 1 -- 172.21.15.37:0/898805722 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fd8ec013620 con 0x7fd900070c10 2024-03-29T08:25:04.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:04.289+0000 7fd8f7400700 1 --2- 172.21.15.37:0/898805722 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fd8e406cbc0 0x7fd8e406f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:04.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:04.289+0000 7fd8f7400700 1 -- 172.21.15.37:0/898805722 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fd8ec08d360 con 0x7fd900070c10 2024-03-29T08:25:04.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:04.289+0000 7fd8ff400700 1 --2- 172.21.15.37:0/898805722 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fd8e406cbc0 0x7fd8e406f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:04.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:04.290+0000 7fd8ff400700 1 --2- 172.21.15.37:0/898805722 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fd8e406cbc0 0x7fd8e406f070 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fd8f0009fd0 tx=0x7fd8f000b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:25:04.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:04.294+0000 7fd8f7400700 1 -- 172.21.15.37:0/898805722 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fd8ec05adb0 con 0x7fd900070c10 2024-03-29T08:25:04.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:04.554+0000 7fd906800700 1 -- 172.21.15.37:0/898805722 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fd90004bed0 con 0x7fd900070c10 2024-03-29T08:25:04.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:04.554+0000 7fd8f7400700 1 -- 172.21.15.37:0/898805722 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 28 v28) v1 ==== 76+0+1841 (secure 0 0 0) 0x7fd8ec05a940 con 0x7fd900070c10 2024-03-29T08:25:04.555 INFO:teuthology.orchestra.run.smithi037.stdout:e28 2024-03-29T08:25:04.555 INFO:teuthology.orchestra.run.smithi037.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-29T08:25:04.555 INFO:teuthology.orchestra.run.smithi037.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T08:25:04.555 INFO:teuthology.orchestra.run.smithi037.stdout:legacy client fscid: 1 2024-03-29T08:25:04.555 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:25:04.555 INFO:teuthology.orchestra.run.smithi037.stdout:Filesystem 'cephfs' (1) 2024-03-29T08:25:04.555 INFO:teuthology.orchestra.run.smithi037.stdout:fs_name cephfs 2024-03-29T08:25:04.555 INFO:teuthology.orchestra.run.smithi037.stdout:epoch 21 2024-03-29T08:25:04.556 INFO:teuthology.orchestra.run.smithi037.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-03-29T08:25:04.556 INFO:teuthology.orchestra.run.smithi037.stdout:created 2024-03-29T08:03:29.241269+0000 2024-03-29T08:25:04.556 INFO:teuthology.orchestra.run.smithi037.stdout:modified 2024-03-29T08:17:16.237238+0000 2024-03-29T08:25:04.556 INFO:teuthology.orchestra.run.smithi037.stdout:tableserver 0 2024-03-29T08:25:04.556 INFO:teuthology.orchestra.run.smithi037.stdout:root 0 2024-03-29T08:25:04.556 INFO:teuthology.orchestra.run.smithi037.stdout:session_timeout 60 2024-03-29T08:25:04.556 INFO:teuthology.orchestra.run.smithi037.stdout:session_autoclose 300 2024-03-29T08:25:04.556 INFO:teuthology.orchestra.run.smithi037.stdout:max_file_size 1099511627776 2024-03-29T08:25:04.556 INFO:teuthology.orchestra.run.smithi037.stdout:max_xattr_size 65536 2024-03-29T08:25:04.556 INFO:teuthology.orchestra.run.smithi037.stdout:required_client_features {} 2024-03-29T08:25:04.556 INFO:teuthology.orchestra.run.smithi037.stdout:last_failure 0 2024-03-29T08:25:04.556 INFO:teuthology.orchestra.run.smithi037.stdout:last_failure_osd_epoch 82 2024-03-29T08:25:04.556 INFO:teuthology.orchestra.run.smithi037.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T08:25:04.556 INFO:teuthology.orchestra.run.smithi037.stdout:max_mds 2 2024-03-29T08:25:04.556 INFO:teuthology.orchestra.run.smithi037.stdout:in 0,1 2024-03-29T08:25:04.557 INFO:teuthology.orchestra.run.smithi037.stdout:up {} 2024-03-29T08:25:04.557 INFO:teuthology.orchestra.run.smithi037.stdout:failed 0,1 2024-03-29T08:25:04.557 INFO:teuthology.orchestra.run.smithi037.stdout:damaged 2024-03-29T08:25:04.557 INFO:teuthology.orchestra.run.smithi037.stdout:stopped 2024-03-29T08:25:04.557 INFO:teuthology.orchestra.run.smithi037.stdout:data_pools [3] 2024-03-29T08:25:04.557 INFO:teuthology.orchestra.run.smithi037.stdout:metadata_pool 2 2024-03-29T08:25:04.557 INFO:teuthology.orchestra.run.smithi037.stdout:inline_data disabled 2024-03-29T08:25:04.557 INFO:teuthology.orchestra.run.smithi037.stdout:balancer 2024-03-29T08:25:04.557 INFO:teuthology.orchestra.run.smithi037.stdout:bal_rank_mask -1 2024-03-29T08:25:04.557 INFO:teuthology.orchestra.run.smithi037.stdout:standby_count_wanted 1 2024-03-29T08:25:04.557 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:25:04.557 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:25:04.557 INFO:teuthology.orchestra.run.smithi037.stdout:Standby daemons: 2024-03-29T08:25:04.557 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:25:04.558 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi148.qukpjh{-1:24319} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.148:6826/2641327912,v1:172.21.15.148:6827/2641327912] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:25:04.558 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi037.mmwgxe{-1:34440} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.37:6828/1248142005,v1:172.21.15.37:6829/1248142005] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:25:04.558 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi037.iifsmh{-1:34444} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.37:6826/2060882959,v1:172.21.15.37:6827/2060882959] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:25:04.558 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi148.sqteih{-1:44395} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.148:6824/4078652242,v1:172.21.15.148:6825/4078652242] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:25:04.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:04.557+0000 7fd906800700 1 -- 172.21.15.37:0/898805722 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fd8e406cbc0 msgr2=0x7fd8e406f070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:25:04.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:04.557+0000 7fd906800700 1 --2- 172.21.15.37:0/898805722 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fd8e406cbc0 0x7fd8e406f070 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fd8f0009fd0 tx=0x7fd8f000b040 comp rx=0 tx=0).stop 2024-03-29T08:25:04.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:04.557+0000 7fd906800700 1 -- 172.21.15.37:0/898805722 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd900070c10 msgr2=0x7fd9001161e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:25:04.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:04.557+0000 7fd906800700 1 --2- 172.21.15.37:0/898805722 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd900070c10 0x7fd9001161e0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fd8ec005010 tx=0x7fd8ec009e30 comp rx=0 tx=0).stop 2024-03-29T08:25:04.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:04.557+0000 7fd906800700 1 -- 172.21.15.37:0/898805722 shutdown_connections 2024-03-29T08:25:04.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:04.557+0000 7fd906800700 1 --2- 172.21.15.37:0/898805722 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fd8e406cbc0 0x7fd8e406f070 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:04.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:04.557+0000 7fd906800700 1 --2- 172.21.15.37:0/898805722 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd900070300 0x7fd900115ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:04.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:04.557+0000 7fd906800700 1 --2- 172.21.15.37:0/898805722 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd900070c10 0x7fd9001161e0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:04.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:04.557+0000 7fd906800700 1 -- 172.21.15.37:0/898805722 >> 172.21.15.37:0/898805722 conn(0x7fd9000f89a0 msgr2=0x7fd9000f9f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:25:04.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:04.558+0000 7fd906800700 1 -- 172.21.15.37:0/898805722 shutdown_connections 2024-03-29T08:25:04.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:04.558+0000 7fd906800700 1 -- 172.21.15.37:0/898805722 wait complete. 2024-03-29T08:25:04.559 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 28 2024-03-29T08:25:04.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:04.696+0000 7f1433200700 1 -- 172.21.15.37:0/3552904934 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f142c1037a0 msgr2=0x7f142c0fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:25:04.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:04.696+0000 7f1433200700 1 --2- 172.21.15.37:0/3552904934 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f142c1037a0 0x7f142c0fce10 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f1418004940 tx=0x7f1418009e30 comp rx=0 tx=0).stop 2024-03-29T08:25:04.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:04.696+0000 7f1433200700 1 -- 172.21.15.37:0/3552904934 shutdown_connections 2024-03-29T08:25:04.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:04.696+0000 7f1433200700 1 --2- 172.21.15.37:0/3552904934 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f142c1037a0 0x7f142c0fce10 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:04.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:04.696+0000 7f1433200700 1 --2- 172.21.15.37:0/3552904934 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f142c102e00 0x7f142c1031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:04.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:04.696+0000 7f1433200700 1 -- 172.21.15.37:0/3552904934 >> 172.21.15.37:0/3552904934 conn(0x7f142c0f89b0 msgr2=0x7f142c0fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:25:04.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:04.696+0000 7f1433200700 1 -- 172.21.15.37:0/3552904934 shutdown_connections 2024-03-29T08:25:04.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:04.696+0000 7f1433200700 1 -- 172.21.15.37:0/3552904934 wait complete. 2024-03-29T08:25:04.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:04.697+0000 7f1433200700 1 Processor -- start 2024-03-29T08:25:04.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:04.697+0000 7f1433200700 1 -- start start 2024-03-29T08:25:04.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:04.697+0000 7f1433200700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f142c102e00 0x7f142c115da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:04.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:04.697+0000 7f1433200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f142c1037a0 0x7f142c1162e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:04.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:04.697+0000 7f1433200700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f142c116930 con 0x7f142c1037a0 2024-03-29T08:25:04.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:04.697+0000 7f1433200700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f142c10fd00 con 0x7f142c102e00 2024-03-29T08:25:04.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:04.698+0000 7f142b400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f142c1037a0 0x7f142c1162e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:04.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:04.698+0000 7f142b400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f142c1037a0 0x7f142c1162e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43892/0 (socket says 172.21.15.37:43892) 2024-03-29T08:25:04.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:04.698+0000 7f142b400700 1 -- 172.21.15.37:0/2349682149 learned_addr learned my addr 172.21.15.37:0/2349682149 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:25:04.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:04.698+0000 7f142be00700 1 --2- 172.21.15.37:0/2349682149 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f142c102e00 0x7f142c115da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:04.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:04.698+0000 7f142b400700 1 -- 172.21.15.37:0/2349682149 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f142c102e00 msgr2=0x7f142c115da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:25:04.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:04.698+0000 7f142b400700 1 --2- 172.21.15.37:0/2349682149 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f142c102e00 0x7f142c115da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:04.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:04.698+0000 7f142b400700 1 -- 172.21.15.37:0/2349682149 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1418005040 con 0x7f142c1037a0 2024-03-29T08:25:04.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:04.698+0000 7f142b400700 1 --2- 172.21.15.37:0/2349682149 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f142c1037a0 0x7f142c1162e0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f1418009390 tx=0x7f1418009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:25:04.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:04.699+0000 7f1423e00700 1 -- 172.21.15.37:0/2349682149 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f141800e040 con 0x7f142c1037a0 2024-03-29T08:25:04.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:04.699+0000 7f1423e00700 1 -- 172.21.15.37:0/2349682149 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f1418013420 con 0x7f142c1037a0 2024-03-29T08:25:04.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:04.699+0000 7f1423e00700 1 -- 172.21.15.37:0/2349682149 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f141801c630 con 0x7f142c1037a0 2024-03-29T08:25:04.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:04.699+0000 7f1433200700 1 -- 172.21.15.37:0/2349682149 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f142c10ff80 con 0x7f142c1037a0 2024-03-29T08:25:04.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:04.699+0000 7f1433200700 1 -- 172.21.15.37:0/2349682149 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f142c1104c0 con 0x7f142c1037a0 2024-03-29T08:25:04.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:04.700+0000 7f1433200700 1 -- 172.21.15.37:0/2349682149 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f142c0fe550 con 0x7f142c1037a0 2024-03-29T08:25:04.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:04.701+0000 7f1423e00700 1 -- 172.21.15.37:0/2349682149 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f1418013a00 con 0x7f142c1037a0 2024-03-29T08:25:04.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:04.702+0000 7f1423e00700 1 --2- 172.21.15.37:0/2349682149 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f141006cc30 0x7f141006f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:04.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:04.702+0000 7f1423e00700 1 -- 172.21.15.37:0/2349682149 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f141808d2c0 con 0x7f142c1037a0 2024-03-29T08:25:04.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:04.702+0000 7f142be00700 1 --2- 172.21.15.37:0/2349682149 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f141006cc30 0x7f141006f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:04.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:04.703+0000 7f142be00700 1 --2- 172.21.15.37:0/2349682149 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f141006cc30 0x7f141006f0e0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f141c004990 tx=0x7f141c009ee0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:25:04.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:04.706+0000 7f1423e00700 1 -- 172.21.15.37:0/2349682149 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f141805ad90 con 0x7f142c1037a0 2024-03-29T08:25:04.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:04.942+0000 7f1433200700 1 -- 172.21.15.37:0/2349682149 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f142c109e10 con 0x7f141006cc30 2024-03-29T08:25:04.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:04.944+0000 7f1423e00700 1 -- 172.21.15.37:0/2349682149 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f142c109e10 con 0x7f141006cc30 2024-03-29T08:25:04.947 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-29T08:25:04.947 INFO:teuthology.orchestra.run.smithi037.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59", 2024-03-29T08:25:04.947 INFO:teuthology.orchestra.run.smithi037.stdout: "in_progress": true, 2024-03-29T08:25:04.947 INFO:teuthology.orchestra.run.smithi037.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-29T08:25:04.947 INFO:teuthology.orchestra.run.smithi037.stdout: "services_complete": [ 2024-03-29T08:25:04.947 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr", 2024-03-29T08:25:04.947 INFO:teuthology.orchestra.run.smithi037.stdout: "mon", 2024-03-29T08:25:04.947 INFO:teuthology.orchestra.run.smithi037.stdout: "crash", 2024-03-29T08:25:04.947 INFO:teuthology.orchestra.run.smithi037.stdout: "osd" 2024-03-29T08:25:04.948 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-29T08:25:04.948 INFO:teuthology.orchestra.run.smithi037.stdout: "progress": "15/21 daemons upgraded", 2024-03-29T08:25:04.948 INFO:teuthology.orchestra.run.smithi037.stdout: "message": "Currently upgrading mds daemons", 2024-03-29T08:25:04.948 INFO:teuthology.orchestra.run.smithi037.stdout: "is_paused": false 2024-03-29T08:25:04.948 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-29T08:25:04.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:04.946+0000 7f1433200700 1 -- 172.21.15.37:0/2349682149 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f141006cc30 msgr2=0x7f141006f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:25:04.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:04.946+0000 7f1433200700 1 --2- 172.21.15.37:0/2349682149 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f141006cc30 0x7f141006f0e0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f141c004990 tx=0x7f141c009ee0 comp rx=0 tx=0).stop 2024-03-29T08:25:04.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:04.946+0000 7f1433200700 1 -- 172.21.15.37:0/2349682149 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f142c1037a0 msgr2=0x7f142c1162e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:25:04.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:04.946+0000 7f1433200700 1 --2- 172.21.15.37:0/2349682149 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f142c1037a0 0x7f142c1162e0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f1418009390 tx=0x7f1418009e30 comp rx=0 tx=0).stop 2024-03-29T08:25:04.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:04.946+0000 7f1433200700 1 -- 172.21.15.37:0/2349682149 shutdown_connections 2024-03-29T08:25:04.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:04.946+0000 7f1433200700 1 --2- 172.21.15.37:0/2349682149 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f141006cc30 0x7f141006f0e0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:04.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:04.946+0000 7f1433200700 1 --2- 172.21.15.37:0/2349682149 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f142c102e00 0x7f142c115da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:04.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:04.946+0000 7f1433200700 1 --2- 172.21.15.37:0/2349682149 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f142c1037a0 0x7f142c1162e0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:04.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:04.946+0000 7f1433200700 1 -- 172.21.15.37:0/2349682149 >> 172.21.15.37:0/2349682149 conn(0x7f142c0f89b0 msgr2=0x7f142c0f9a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:25:04.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:04.946+0000 7f1433200700 1 -- 172.21.15.37:0/2349682149 shutdown_connections 2024-03-29T08:25:04.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:04.946+0000 7f1433200700 1 -- 172.21.15.37:0/2349682149 wait complete. 2024-03-29T08:25:05.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:25:04 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/1095867024' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:25:05.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:25:04 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/898805722' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T08:25:05.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:05.083+0000 7fb3bf000700 1 -- 172.21.15.37:0/864222408 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb3b8102e20 msgr2=0x7fb3b81031f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:25:05.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:05.083+0000 7fb3bf000700 1 --2- 172.21.15.37:0/864222408 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb3b8102e20 0x7fb3b81031f0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fb3a8004940 tx=0x7fb3a8009e30 comp rx=0 tx=0).stop 2024-03-29T08:25:05.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:05.083+0000 7fb3bf000700 1 -- 172.21.15.37:0/864222408 shutdown_connections 2024-03-29T08:25:05.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:05.083+0000 7fb3bf000700 1 --2- 172.21.15.37:0/864222408 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb3b81037c0 0x7fb3b80fce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:05.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:05.083+0000 7fb3bf000700 1 --2- 172.21.15.37:0/864222408 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb3b8102e20 0x7fb3b81031f0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:05.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:05.083+0000 7fb3bf000700 1 -- 172.21.15.37:0/864222408 >> 172.21.15.37:0/864222408 conn(0x7fb3b80f8970 msgr2=0x7fb3b80fad80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:25:05.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:05.083+0000 7fb3bf000700 1 -- 172.21.15.37:0/864222408 shutdown_connections 2024-03-29T08:25:05.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:05.083+0000 7fb3bf000700 1 -- 172.21.15.37:0/864222408 wait complete. 2024-03-29T08:25:05.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:05.084+0000 7fb3bf000700 1 Processor -- start 2024-03-29T08:25:05.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:05.084+0000 7fb3bf000700 1 -- start start 2024-03-29T08:25:05.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:05.084+0000 7fb3bf000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb3b8102e20 0x7fb3b8115cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:05.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:05.084+0000 7fb3bf000700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb3b81037c0 0x7fb3b8116230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:05.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:05.084+0000 7fb3bf000700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3b8116880 con 0x7fb3b8102e20 2024-03-29T08:25:05.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:05.084+0000 7fb3bf000700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3b810fc50 con 0x7fb3b81037c0 2024-03-29T08:25:05.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:05.084+0000 7fb3b7e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb3b8102e20 0x7fb3b8115cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:05.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:05.085+0000 7fb3b7e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb3b8102e20 0x7fb3b8115cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43902/0 (socket says 172.21.15.37:43902) 2024-03-29T08:25:05.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:05.085+0000 7fb3b7e00700 1 -- 172.21.15.37:0/1623004164 learned_addr learned my addr 172.21.15.37:0/1623004164 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:25:05.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:05.085+0000 7fb3b7e00700 1 -- 172.21.15.37:0/1623004164 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb3b81037c0 msgr2=0x7fb3b8116230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:25:05.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:05.085+0000 7fb3b7e00700 1 --2- 172.21.15.37:0/1623004164 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb3b81037c0 0x7fb3b8116230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:05.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:05.085+0000 7fb3b7e00700 1 -- 172.21.15.37:0/1623004164 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3a8005040 con 0x7fb3b8102e20 2024-03-29T08:25:05.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:05.085+0000 7fb3b7e00700 1 --2- 172.21.15.37:0/1623004164 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb3b8102e20 0x7fb3b8115cf0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fb3a8009360 tx=0x7fb3a8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:25:05.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:05.085+0000 7fb3afe00700 1 -- 172.21.15.37:0/1623004164 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb3a800e040 con 0x7fb3b8102e20 2024-03-29T08:25:05.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:05.085+0000 7fb3afe00700 1 -- 172.21.15.37:0/1623004164 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fb3a8013420 con 0x7fb3b8102e20 2024-03-29T08:25:05.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:05.085+0000 7fb3afe00700 1 -- 172.21.15.37:0/1623004164 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb3a801c5f0 con 0x7fb3b8102e20 2024-03-29T08:25:05.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:05.086+0000 7fb3bf000700 1 -- 172.21.15.37:0/1623004164 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3b810fed0 con 0x7fb3b8102e20 2024-03-29T08:25:05.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:05.086+0000 7fb3bf000700 1 -- 172.21.15.37:0/1623004164 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3b8110490 con 0x7fb3b8102e20 2024-03-29T08:25:05.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:05.087+0000 7fb3bf000700 1 -- 172.21.15.37:0/1623004164 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3b80fe570 con 0x7fb3b8102e20 2024-03-29T08:25:05.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:05.088+0000 7fb3afe00700 1 -- 172.21.15.37:0/1623004164 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fb3a8013590 con 0x7fb3b8102e20 2024-03-29T08:25:05.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:05.089+0000 7fb3afe00700 1 --2- 172.21.15.37:0/1623004164 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fb39c06cc30 0x7fb39c06f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:05.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:05.089+0000 7fb3afe00700 1 -- 172.21.15.37:0/1623004164 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fb3a808d250 con 0x7fb3b8102e20 2024-03-29T08:25:05.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:05.089+0000 7fb3b7400700 1 --2- 172.21.15.37:0/1623004164 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fb39c06cc30 0x7fb39c06f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:05.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:05.090+0000 7fb3b7400700 1 --2- 172.21.15.37:0/1623004164 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fb39c06cc30 0x7fb39c06f0e0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fb3b8111590 tx=0x7fb3a4009ce0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:25:05.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:05.094+0000 7fb3afe00700 1 -- 172.21.15.37:0/1623004164 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fb3a805add0 con 0x7fb3b8102e20 2024-03-29T08:25:05.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:25:04 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/1095867024' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:25:05.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:25:04 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/898805722' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T08:25:05.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:05.395+0000 7fb3bf000700 1 -- 172.21.15.37:0/1623004164 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fb3b805f5f0 con 0x7fb3b8102e20 2024-03-29T08:25:05.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:05.395+0000 7fb3afe00700 1 -- 172.21.15.37:0/1623004164 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7fb3b805f5f0 con 0x7fb3b8102e20 2024-03-29T08:25:05.396 INFO:teuthology.orchestra.run.smithi037.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-03-29T08:25:05.396 INFO:teuthology.orchestra.run.smithi037.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-03-29T08:25:05.396 INFO:teuthology.orchestra.run.smithi037.stdout: fs cephfs is degraded 2024-03-29T08:25:05.396 INFO:teuthology.orchestra.run.smithi037.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-03-29T08:25:05.396 INFO:teuthology.orchestra.run.smithi037.stdout: fs cephfs is offline because no MDS is active for it. 2024-03-29T08:25:05.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:05.397+0000 7fb3bf000700 1 -- 172.21.15.37:0/1623004164 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fb39c06cc30 msgr2=0x7fb39c06f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:25:05.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:05.397+0000 7fb3bf000700 1 --2- 172.21.15.37:0/1623004164 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fb39c06cc30 0x7fb39c06f0e0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fb3b8111590 tx=0x7fb3a4009ce0 comp rx=0 tx=0).stop 2024-03-29T08:25:05.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:05.397+0000 7fb3bf000700 1 -- 172.21.15.37:0/1623004164 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb3b8102e20 msgr2=0x7fb3b8115cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:25:05.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:05.397+0000 7fb3bf000700 1 --2- 172.21.15.37:0/1623004164 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb3b8102e20 0x7fb3b8115cf0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fb3a8009360 tx=0x7fb3a8009e30 comp rx=0 tx=0).stop 2024-03-29T08:25:05.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:05.398+0000 7fb3bf000700 1 -- 172.21.15.37:0/1623004164 shutdown_connections 2024-03-29T08:25:05.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:05.398+0000 7fb3bf000700 1 --2- 172.21.15.37:0/1623004164 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fb39c06cc30 0x7fb39c06f0e0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:05.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:05.398+0000 7fb3bf000700 1 --2- 172.21.15.37:0/1623004164 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb3b8102e20 0x7fb3b8115cf0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:05.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:05.398+0000 7fb3bf000700 1 --2- 172.21.15.37:0/1623004164 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb3b81037c0 0x7fb3b8116230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:05.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:05.398+0000 7fb3bf000700 1 -- 172.21.15.37:0/1623004164 >> 172.21.15.37:0/1623004164 conn(0x7fb3b80f8970 msgr2=0x7fb3b80f9a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:25:05.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:05.398+0000 7fb3bf000700 1 -- 172.21.15.37:0/1623004164 shutdown_connections 2024-03-29T08:25:05.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:05.398+0000 7fb3bf000700 1 -- 172.21.15.37:0/1623004164 wait complete. 2024-03-29T08:25:06.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:25:05 smithi148 ceph-mon[106692]: from='client.34832 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:25:06.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:25:05 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/1623004164' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T08:25:06.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:25:05 smithi148 ceph-mon[106692]: pgmap v426: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:25:06.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:25:05 smithi037 ceph-mon[142325]: from='client.34832 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:25:06.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:25:05 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/1623004164' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T08:25:06.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:25:05 smithi037 ceph-mon[142325]: pgmap v426: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:25:08.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:25:07 smithi148 ceph-mon[106692]: pgmap v427: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:25:08.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:25:07 smithi037 ceph-mon[142325]: pgmap v427: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:25:10.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:25:10 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:25:10.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:25:10 smithi148 ceph-mon[106692]: pgmap v428: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:25:10.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:25:10 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:25:10.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:25:10 smithi037 ceph-mon[142325]: pgmap v428: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:25:12.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:25:11 smithi148 ceph-mon[106692]: pgmap v429: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:25:12.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:25:11 smithi037 ceph-mon[142325]: pgmap v429: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:25:14.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:25:13 smithi148 ceph-mon[106692]: pgmap v430: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:25:14.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:25:13 smithi037 ceph-mon[142325]: pgmap v430: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:25:16.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:25:15 smithi148 ceph-mon[106692]: pgmap v431: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:25:16.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:25:15 smithi037 ceph-mon[142325]: pgmap v431: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:25:18.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:25:17 smithi148 ceph-mon[106692]: pgmap v432: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:25:18.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:25:17 smithi037 ceph-mon[142325]: pgmap v432: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:25:20.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:25:20 smithi148 ceph-mon[106692]: pgmap v433: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:25:20.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:25:20 smithi037 ceph-mon[142325]: pgmap v433: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:25:22.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:25:21 smithi148 ceph-mon[106692]: pgmap v434: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:25:22.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:25:21 smithi037 ceph-mon[142325]: pgmap v434: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:25:24.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:25:23 smithi148 ceph-mon[106692]: pgmap v435: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:25:24.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:25:23 smithi037 ceph-mon[142325]: pgmap v435: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:25:25.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:25:25 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:25:25.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:25:25 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:25:26.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:25:26 smithi148 ceph-mon[106692]: pgmap v436: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:25:26.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:25:26 smithi037 ceph-mon[142325]: pgmap v436: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:25:28.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:25:27 smithi148 ceph-mon[106692]: pgmap v437: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:25:28.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:25:27 smithi037 ceph-mon[142325]: pgmap v437: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:25:30.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:25:30 smithi148 ceph-mon[106692]: pgmap v438: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:25:30.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:25:30 smithi037 ceph-mon[142325]: pgmap v438: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:25:32.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:25:31 smithi148 ceph-mon[106692]: pgmap v439: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:25:32.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:25:31 smithi037 ceph-mon[142325]: pgmap v439: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:25:34.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:25:33 smithi148 ceph-mon[106692]: pgmap v440: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:25:34.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:25:33 smithi037 ceph-mon[142325]: pgmap v440: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:25:35.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:35.552+0000 7ff6b8400700 1 -- 172.21.15.37:0/3892800244 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff6b01037c0 msgr2=0x7ff6b00fce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:25:35.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:35.552+0000 7ff6b8400700 1 --2- 172.21.15.37:0/3892800244 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff6b01037c0 0x7ff6b00fce30 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7ff69c004940 tx=0x7ff69c009e30 comp rx=0 tx=0).stop 2024-03-29T08:25:35.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:35.552+0000 7ff6b8400700 1 -- 172.21.15.37:0/3892800244 shutdown_connections 2024-03-29T08:25:35.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:35.552+0000 7ff6b8400700 1 --2- 172.21.15.37:0/3892800244 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff6b01037c0 0x7ff6b00fce30 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:35.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:35.552+0000 7ff6b8400700 1 --2- 172.21.15.37:0/3892800244 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff6b0102e20 0x7ff6b01031f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:35.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:35.552+0000 7ff6b8400700 1 -- 172.21.15.37:0/3892800244 >> 172.21.15.37:0/3892800244 conn(0x7ff6b00f8970 msgr2=0x7ff6b00fad80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:25:35.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:35.552+0000 7ff6b8400700 1 -- 172.21.15.37:0/3892800244 shutdown_connections 2024-03-29T08:25:35.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:35.552+0000 7ff6b8400700 1 -- 172.21.15.37:0/3892800244 wait complete. 2024-03-29T08:25:35.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:35.553+0000 7ff6b8400700 1 Processor -- start 2024-03-29T08:25:35.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:35.553+0000 7ff6b8400700 1 -- start start 2024-03-29T08:25:35.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:35.554+0000 7ff6b8400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff6b0102e20 0x7ff6b0115cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:35.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:35.554+0000 7ff6b8400700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff6b01037c0 0x7ff6b01161f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:35.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:35.554+0000 7ff6b8400700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6b0116840 con 0x7ff6b0102e20 2024-03-29T08:25:35.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:35.554+0000 7ff6b8400700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6b010fc10 con 0x7ff6b01037c0 2024-03-29T08:25:35.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:35.554+0000 7ff6b5a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff6b0102e20 0x7ff6b0115cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:35.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:35.554+0000 7ff6b5a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff6b0102e20 0x7ff6b0115cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37756/0 (socket says 172.21.15.37:37756) 2024-03-29T08:25:35.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:35.554+0000 7ff6b5a00700 1 -- 172.21.15.37:0/2784997285 learned_addr learned my addr 172.21.15.37:0/2784997285 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:25:35.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:35.554+0000 7ff6b5a00700 1 -- 172.21.15.37:0/2784997285 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff6b01037c0 msgr2=0x7ff6b01161f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:25:35.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:35.554+0000 7ff6b5a00700 1 --2- 172.21.15.37:0/2784997285 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff6b01037c0 0x7ff6b01161f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:35.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:35.554+0000 7ff6b5a00700 1 -- 172.21.15.37:0/2784997285 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff69c005040 con 0x7ff6b0102e20 2024-03-29T08:25:35.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:35.554+0000 7ff6b5a00700 1 --2- 172.21.15.37:0/2784997285 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff6b0102e20 0x7ff6b0115cb0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7ff6a4002a50 tx=0x7ff6a40046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:25:35.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:35.555+0000 7ff6ad600700 1 -- 172.21.15.37:0/2784997285 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff6a4007cc0 con 0x7ff6b0102e20 2024-03-29T08:25:35.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:35.555+0000 7ff6ad600700 1 -- 172.21.15.37:0/2784997285 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ff6a400f040 con 0x7ff6b0102e20 2024-03-29T08:25:35.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:35.555+0000 7ff6b8400700 1 -- 172.21.15.37:0/2784997285 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6b010fef0 con 0x7ff6b0102e20 2024-03-29T08:25:35.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:35.555+0000 7ff6ad600700 1 -- 172.21.15.37:0/2784997285 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff6a4009540 con 0x7ff6b0102e20 2024-03-29T08:25:35.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:35.555+0000 7ff6b8400700 1 -- 172.21.15.37:0/2784997285 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6b01104b0 con 0x7ff6b0102e20 2024-03-29T08:25:35.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:35.556+0000 7ff6b8400700 1 -- 172.21.15.37:0/2784997285 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6b00fe570 con 0x7ff6b0102e20 2024-03-29T08:25:35.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:35.557+0000 7ff6ad600700 1 -- 172.21.15.37:0/2784997285 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7ff6a40096a0 con 0x7ff6b0102e20 2024-03-29T08:25:35.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:35.558+0000 7ff6ad600700 1 --2- 172.21.15.37:0/2784997285 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7ff69806cc30 0x7ff69806f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:35.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:35.558+0000 7ff6ad600700 1 -- 172.21.15.37:0/2784997285 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7ff6a408bde0 con 0x7ff6b0102e20 2024-03-29T08:25:35.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:35.558+0000 7ff6b5000700 1 --2- 172.21.15.37:0/2784997285 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7ff69806cc30 0x7ff69806f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:35.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:35.559+0000 7ff6b5000700 1 --2- 172.21.15.37:0/2784997285 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7ff69806cc30 0x7ff69806f0e0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7ff69c004f90 tx=0x7ff69c009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:25:35.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:35.563+0000 7ff6ad600700 1 -- 172.21.15.37:0/2784997285 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7ff6a4059830 con 0x7ff6b0102e20 2024-03-29T08:25:35.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:35.800+0000 7ff6b8400700 1 -- 172.21.15.37:0/2784997285 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff6b0109e30 con 0x7ff69806cc30 2024-03-29T08:25:35.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:35.801+0000 7ff6ad600700 1 -- 172.21.15.37:0/2784997285 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7ff6b0109e30 con 0x7ff69806cc30 2024-03-29T08:25:35.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:35.804+0000 7ff6b8400700 1 -- 172.21.15.37:0/2784997285 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7ff69806cc30 msgr2=0x7ff69806f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:25:35.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:35.804+0000 7ff6b8400700 1 --2- 172.21.15.37:0/2784997285 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7ff69806cc30 0x7ff69806f0e0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7ff69c004f90 tx=0x7ff69c009e30 comp rx=0 tx=0).stop 2024-03-29T08:25:35.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:35.804+0000 7ff6b8400700 1 -- 172.21.15.37:0/2784997285 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff6b0102e20 msgr2=0x7ff6b0115cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:25:35.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:35.804+0000 7ff6b8400700 1 --2- 172.21.15.37:0/2784997285 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff6b0102e20 0x7ff6b0115cb0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7ff6a4002a50 tx=0x7ff6a40046b0 comp rx=0 tx=0).stop 2024-03-29T08:25:35.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:35.804+0000 7ff6b8400700 1 -- 172.21.15.37:0/2784997285 shutdown_connections 2024-03-29T08:25:35.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:35.804+0000 7ff6b8400700 1 --2- 172.21.15.37:0/2784997285 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7ff69806cc30 0x7ff69806f0e0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:35.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:35.804+0000 7ff6b8400700 1 --2- 172.21.15.37:0/2784997285 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff6b0102e20 0x7ff6b0115cb0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:35.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:35.804+0000 7ff6b8400700 1 --2- 172.21.15.37:0/2784997285 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff6b01037c0 0x7ff6b01161f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:35.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:35.804+0000 7ff6b8400700 1 -- 172.21.15.37:0/2784997285 >> 172.21.15.37:0/2784997285 conn(0x7ff6b00f8970 msgr2=0x7ff6b00f9a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:25:35.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:35.804+0000 7ff6b8400700 1 -- 172.21.15.37:0/2784997285 shutdown_connections 2024-03-29T08:25:35.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:35.804+0000 7ff6b8400700 1 -- 172.21.15.37:0/2784997285 wait complete. 2024-03-29T08:25:35.818 INFO:teuthology.orchestra.run.smithi037.stdout:true 2024-03-29T08:25:35.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:35.942+0000 7fa562e00700 1 -- 172.21.15.37:0/4125961602 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa55c102e00 msgr2=0x7fa55c1031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:25:35.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:35.942+0000 7fa562e00700 1 --2- 172.21.15.37:0/4125961602 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa55c102e00 0x7fa55c1031d0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fa54c004940 tx=0x7fa54c009e30 comp rx=0 tx=0).stop 2024-03-29T08:25:35.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:35.942+0000 7fa562e00700 1 -- 172.21.15.37:0/4125961602 shutdown_connections 2024-03-29T08:25:35.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:35.942+0000 7fa562e00700 1 --2- 172.21.15.37:0/4125961602 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa55c1037a0 0x7fa55c0fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:35.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:35.942+0000 7fa562e00700 1 --2- 172.21.15.37:0/4125961602 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa55c102e00 0x7fa55c1031d0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:35.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:35.942+0000 7fa562e00700 1 -- 172.21.15.37:0/4125961602 >> 172.21.15.37:0/4125961602 conn(0x7fa55c0f89b0 msgr2=0x7fa55c0fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:25:35.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:35.942+0000 7fa562e00700 1 -- 172.21.15.37:0/4125961602 shutdown_connections 2024-03-29T08:25:35.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:35.942+0000 7fa562e00700 1 -- 172.21.15.37:0/4125961602 wait complete. 2024-03-29T08:25:35.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:35.943+0000 7fa562e00700 1 Processor -- start 2024-03-29T08:25:35.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:35.943+0000 7fa562e00700 1 -- start start 2024-03-29T08:25:35.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:35.944+0000 7fa562e00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa55c102e00 0x7fa55c115c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:35.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:35.944+0000 7fa562e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa55c1037a0 0x7fa55c1161b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:35.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:35.944+0000 7fa562e00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa55c116800 con 0x7fa55c1037a0 2024-03-29T08:25:35.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:35.944+0000 7fa562e00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa55c10fbd0 con 0x7fa55c102e00 2024-03-29T08:25:35.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:35.944+0000 7fa55b400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa55c1037a0 0x7fa55c1161b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:35.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:35.944+0000 7fa55b400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa55c1037a0 0x7fa55c1161b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37780/0 (socket says 172.21.15.37:37780) 2024-03-29T08:25:35.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:35.944+0000 7fa55b400700 1 -- 172.21.15.37:0/223531910 learned_addr learned my addr 172.21.15.37:0/223531910 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:25:35.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:35.944+0000 7fa55be00700 1 --2- 172.21.15.37:0/223531910 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa55c102e00 0x7fa55c115c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:35.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:35.944+0000 7fa55b400700 1 -- 172.21.15.37:0/223531910 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa55c102e00 msgr2=0x7fa55c115c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:25:35.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:35.944+0000 7fa55b400700 1 --2- 172.21.15.37:0/223531910 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa55c102e00 0x7fa55c115c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:35.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:35.945+0000 7fa55b400700 1 -- 172.21.15.37:0/223531910 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa54c005040 con 0x7fa55c1037a0 2024-03-29T08:25:35.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:35.945+0000 7fa55b400700 1 --2- 172.21.15.37:0/223531910 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa55c1037a0 0x7fa55c1161b0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fa54800eb40 tx=0x7fa54800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:25:35.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:35.945+0000 7fa553e00700 1 -- 172.21.15.37:0/223531910 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa548007cc0 con 0x7fa55c1037a0 2024-03-29T08:25:35.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:35.945+0000 7fa553e00700 1 -- 172.21.15.37:0/223531910 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fa54800f040 con 0x7fa55c1037a0 2024-03-29T08:25:35.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:35.945+0000 7fa562e00700 1 -- 172.21.15.37:0/223531910 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa55c10fe50 con 0x7fa55c1037a0 2024-03-29T08:25:35.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:35.945+0000 7fa553e00700 1 -- 172.21.15.37:0/223531910 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa54800ca30 con 0x7fa55c1037a0 2024-03-29T08:25:35.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:35.945+0000 7fa562e00700 1 -- 172.21.15.37:0/223531910 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa55c110410 con 0x7fa55c1037a0 2024-03-29T08:25:35.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:35.946+0000 7fa562e00700 1 -- 172.21.15.37:0/223531910 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa55c0fe550 con 0x7fa55c1037a0 2024-03-29T08:25:35.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:35.948+0000 7fa553e00700 1 -- 172.21.15.37:0/223531910 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fa54800cb90 con 0x7fa55c1037a0 2024-03-29T08:25:35.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:35.949+0000 7fa553e00700 1 --2- 172.21.15.37:0/223531910 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fa54006cc30 0x7fa54006f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:35.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:35.949+0000 7fa553e00700 1 -- 172.21.15.37:0/223531910 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fa54808bba0 con 0x7fa55c1037a0 2024-03-29T08:25:35.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:35.949+0000 7fa55be00700 1 --2- 172.21.15.37:0/223531910 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fa54006cc30 0x7fa54006f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:35.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:35.950+0000 7fa55be00700 1 --2- 172.21.15.37:0/223531910 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fa54006cc30 0x7fa54006f0e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fa54c0045a0 tx=0x7fa54c009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:25:35.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:35.954+0000 7fa553e00700 1 -- 172.21.15.37:0/223531910 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fa5480595f0 con 0x7fa55c1037a0 2024-03-29T08:25:36.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:25:35 smithi148 ceph-mon[106692]: pgmap v441: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:25:36.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:25:35 smithi037 ceph-mon[142325]: pgmap v441: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:25:36.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:36.185+0000 7fa562e00700 1 -- 172.21.15.37:0/223531910 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa55c109de0 con 0x7fa54006cc30 2024-03-29T08:25:36.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:36.186+0000 7fa553e00700 1 -- 172.21.15.37:0/223531910 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fa55c109de0 con 0x7fa54006cc30 2024-03-29T08:25:36.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:36.188+0000 7fa562e00700 1 -- 172.21.15.37:0/223531910 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fa54006cc30 msgr2=0x7fa54006f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:25:36.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:36.188+0000 7fa562e00700 1 --2- 172.21.15.37:0/223531910 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fa54006cc30 0x7fa54006f0e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fa54c0045a0 tx=0x7fa54c009e30 comp rx=0 tx=0).stop 2024-03-29T08:25:36.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:36.188+0000 7fa562e00700 1 -- 172.21.15.37:0/223531910 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa55c1037a0 msgr2=0x7fa55c1161b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:25:36.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:36.188+0000 7fa562e00700 1 --2- 172.21.15.37:0/223531910 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa55c1037a0 0x7fa55c1161b0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fa54800eb40 tx=0x7fa54800b040 comp rx=0 tx=0).stop 2024-03-29T08:25:36.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:36.188+0000 7fa562e00700 1 -- 172.21.15.37:0/223531910 shutdown_connections 2024-03-29T08:25:36.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:36.189+0000 7fa562e00700 1 --2- 172.21.15.37:0/223531910 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fa54006cc30 0x7fa54006f0e0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:36.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:36.189+0000 7fa562e00700 1 --2- 172.21.15.37:0/223531910 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa55c102e00 0x7fa55c115c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:36.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:36.189+0000 7fa562e00700 1 --2- 172.21.15.37:0/223531910 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa55c1037a0 0x7fa55c1161b0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:36.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:36.189+0000 7fa562e00700 1 -- 172.21.15.37:0/223531910 >> 172.21.15.37:0/223531910 conn(0x7fa55c0f89b0 msgr2=0x7fa55c0f9900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:25:36.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:36.189+0000 7fa562e00700 1 -- 172.21.15.37:0/223531910 shutdown_connections 2024-03-29T08:25:36.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:36.189+0000 7fa562e00700 1 -- 172.21.15.37:0/223531910 wait complete. 2024-03-29T08:25:36.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:36.326+0000 7fb580200700 1 -- 172.21.15.37:0/3987005515 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb578070300 msgr2=0x7fb5780706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:25:36.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:36.326+0000 7fb580200700 1 --2- 172.21.15.37:0/3987005515 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb578070300 0x7fb5780706d0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fb56c004970 tx=0x7fb56c009e30 comp rx=0 tx=0).stop 2024-03-29T08:25:36.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:36.326+0000 7fb580200700 1 -- 172.21.15.37:0/3987005515 shutdown_connections 2024-03-29T08:25:36.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:36.326+0000 7fb580200700 1 --2- 172.21.15.37:0/3987005515 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb578070c10 0x7fb578109170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:36.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:36.326+0000 7fb580200700 1 --2- 172.21.15.37:0/3987005515 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb578070300 0x7fb5780706d0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:36.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:36.326+0000 7fb580200700 1 -- 172.21.15.37:0/3987005515 >> 172.21.15.37:0/3987005515 conn(0x7fb5780f89b0 msgr2=0x7fb5780fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:25:36.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:36.326+0000 7fb580200700 1 -- 172.21.15.37:0/3987005515 shutdown_connections 2024-03-29T08:25:36.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:36.326+0000 7fb580200700 1 -- 172.21.15.37:0/3987005515 wait complete. 2024-03-29T08:25:36.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:36.327+0000 7fb580200700 1 Processor -- start 2024-03-29T08:25:36.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:36.327+0000 7fb580200700 1 -- start start 2024-03-29T08:25:36.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:36.328+0000 7fb580200700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb578070300 0x7fb578115a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:36.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:36.328+0000 7fb580200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb578070c10 0x7fb578115f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:36.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:36.328+0000 7fb580200700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5781165e0 con 0x7fb578070c10 2024-03-29T08:25:36.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:36.328+0000 7fb580200700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb57810fbc0 con 0x7fb578070300 2024-03-29T08:25:36.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:36.328+0000 7fb57ce00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb578070c10 0x7fb578115f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:36.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:36.328+0000 7fb57ce00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb578070c10 0x7fb578115f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37808/0 (socket says 172.21.15.37:37808) 2024-03-29T08:25:36.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:36.328+0000 7fb57ce00700 1 -- 172.21.15.37:0/3720575759 learned_addr learned my addr 172.21.15.37:0/3720575759 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:25:36.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:36.328+0000 7fb57d800700 1 --2- 172.21.15.37:0/3720575759 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb578070300 0x7fb578115a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:36.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:36.328+0000 7fb57ce00700 1 -- 172.21.15.37:0/3720575759 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb578070300 msgr2=0x7fb578115a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:25:36.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:36.328+0000 7fb57ce00700 1 --2- 172.21.15.37:0/3720575759 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb578070300 0x7fb578115a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:36.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:36.328+0000 7fb57ce00700 1 -- 172.21.15.37:0/3720575759 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb56c005040 con 0x7fb578070c10 2024-03-29T08:25:36.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:36.329+0000 7fb57ce00700 1 --2- 172.21.15.37:0/3720575759 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb578070c10 0x7fb578115f90 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fb56400c8c0 tx=0x7fb5640044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:25:36.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:36.329+0000 7fb575600700 1 -- 172.21.15.37:0/3720575759 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb56400a980 con 0x7fb578070c10 2024-03-29T08:25:36.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:36.329+0000 7fb575600700 1 -- 172.21.15.37:0/3720575759 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fb56400ce60 con 0x7fb578070c10 2024-03-29T08:25:36.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:36.329+0000 7fb575600700 1 -- 172.21.15.37:0/3720575759 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb5640113a0 con 0x7fb578070c10 2024-03-29T08:25:36.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:36.329+0000 7fb580200700 1 -- 172.21.15.37:0/3720575759 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb57810fea0 con 0x7fb578070c10 2024-03-29T08:25:36.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:36.329+0000 7fb580200700 1 -- 172.21.15.37:0/3720575759 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5781103e0 con 0x7fb578070c10 2024-03-29T08:25:36.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:36.330+0000 7fb580200700 1 -- 172.21.15.37:0/3720575759 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb578071a40 con 0x7fb578070c10 2024-03-29T08:25:36.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:36.331+0000 7fb575600700 1 -- 172.21.15.37:0/3720575759 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fb564012400 con 0x7fb578070c10 2024-03-29T08:25:36.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:36.332+0000 7fb575600700 1 --2- 172.21.15.37:0/3720575759 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fb56006cc20 0x7fb56006f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:36.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:36.332+0000 7fb575600700 1 -- 172.21.15.37:0/3720575759 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fb56408c0f0 con 0x7fb578070c10 2024-03-29T08:25:36.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:36.333+0000 7fb57d800700 1 --2- 172.21.15.37:0/3720575759 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fb56006cc20 0x7fb56006f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:36.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:36.333+0000 7fb57d800700 1 --2- 172.21.15.37:0/3720575759 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fb56006cc20 0x7fb56006f0d0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fb56c004540 tx=0x7fb56c009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:25:36.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:36.336+0000 7fb575600700 1 -- 172.21.15.37:0/3720575759 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fb564059bc0 con 0x7fb578070c10 2024-03-29T08:25:36.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:36.561+0000 7fb580200700 1 -- 172.21.15.37:0/3720575759 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fb57810bfb0 con 0x7fb56006cc20 2024-03-29T08:25:36.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:36.571+0000 7fb575600700 1 -- 172.21.15.37:0/3720575759 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7fb57810bfb0 con 0x7fb56006cc20 2024-03-29T08:25:36.571 INFO:teuthology.orchestra.run.smithi037.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-29T08:25:36.571 INFO:teuthology.orchestra.run.smithi037.stdout:alertmanager.smithi037 smithi037 *:9093,9094 running (15m) 7m ago 28m 14.9M - 0.25.0 c8568f914cd2 2f9f9746ed0f 2024-03-29T08:25:36.571 INFO:teuthology.orchestra.run.smithi037.stdout:crash.smithi037 smithi037 running (12m) 7m ago 28m 6647k - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 178d416a3186 2024-03-29T08:25:36.571 INFO:teuthology.orchestra.run.smithi037.stdout:crash.smithi148 smithi148 running (12m) 7m ago 27m 6656k - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 23666989d58f 2024-03-29T08:25:36.571 INFO:teuthology.orchestra.run.smithi037.stdout:grafana.smithi037 smithi037 *:3000 running (15m) 7m ago 28m 71.9M - 9.4.7 954c08fa6188 6b976fe49952 2024-03-29T08:25:36.572 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi037.iifsmh smithi037 running (7m) 7m ago 21m 10.8M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 2d97f562cc5b 2024-03-29T08:25:36.572 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi037.mmwgxe smithi037 running (8m) 7m ago 22m 10.9M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff ec37037f46e8 2024-03-29T08:25:36.572 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi148.qukpjh smithi148 running (21m) 7m ago 21m 15.3M - 17.2.7-789-g4213e17c cf22eb8683be 8e469782a273 2024-03-29T08:25:36.572 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi148.sqteih smithi148 running (7m) 7m ago 21m 10.7M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 44c15c77037a 2024-03-29T08:25:36.572 INFO:teuthology.orchestra.run.smithi037.stdout:mgr.smithi037.wjikii smithi037 *:8443,9283,8765 running (17m) 7m ago 30m 506M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff c5a1417b9417 2024-03-29T08:25:36.572 INFO:teuthology.orchestra.run.smithi037.stdout:mgr.smithi148.onhgvm smithi148 *:8443,9283,8765 running (16m) 7m ago 27m 450M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff fcfdbce062bd 2024-03-29T08:25:36.572 INFO:teuthology.orchestra.run.smithi037.stdout:mon.smithi037 smithi037 running (13m) 7m ago 30m 54.8M 2048M 18.2.2-1015-gd0e392d9 1cc2f45c17ff eea2060783af 2024-03-29T08:25:36.572 INFO:teuthology.orchestra.run.smithi037.stdout:mon.smithi148 smithi148 running (13m) 7m ago 27m 43.7M 2048M 18.2.2-1015-gd0e392d9 1cc2f45c17ff ad4b2c5748be 2024-03-29T08:25:36.572 INFO:teuthology.orchestra.run.smithi037.stdout:node-exporter.smithi037 smithi037 *:9100 running (16m) 7m ago 28m 7625k - 1.5.0 0da6a335fe13 26b8ae9cb878 2024-03-29T08:25:36.572 INFO:teuthology.orchestra.run.smithi037.stdout:node-exporter.smithi148 smithi148 *:9100 running (16m) 7m ago 27m 8166k - 1.5.0 0da6a335fe13 98414e4e64ed 2024-03-29T08:25:36.572 INFO:teuthology.orchestra.run.smithi037.stdout:osd.0 smithi037 running (11m) 7m ago 26m 104M 1619M 18.2.2-1015-gd0e392d9 1cc2f45c17ff c0c584f6245c 2024-03-29T08:25:36.572 INFO:teuthology.orchestra.run.smithi037.stdout:osd.1 smithi037 running (11m) 7m ago 25m 100M 1619M 18.2.2-1015-gd0e392d9 1cc2f45c17ff a4a2ff4cc83d 2024-03-29T08:25:36.572 INFO:teuthology.orchestra.run.smithi037.stdout:osd.2 smithi037 running (10m) 7m ago 25m 82.3M 1619M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 8b1bc71aa79e 2024-03-29T08:25:36.572 INFO:teuthology.orchestra.run.smithi037.stdout:osd.3 smithi148 running (9m) 7m ago 24m 105M 2642M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 1c656ab8ff86 2024-03-29T08:25:36.573 INFO:teuthology.orchestra.run.smithi037.stdout:osd.4 smithi148 running (9m) 7m ago 24m 97.3M 2642M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 966a9de11d22 2024-03-29T08:25:36.573 INFO:teuthology.orchestra.run.smithi037.stdout:osd.5 smithi148 running (8m) 7m ago 23m 85.8M 2642M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 474c4f9212ce 2024-03-29T08:25:36.573 INFO:teuthology.orchestra.run.smithi037.stdout:prometheus.smithi037 smithi037 *:9095 running (15m) 7m ago 27m 54.4M - 2.43.0 a07b618ecd1d 30c0223ec589 2024-03-29T08:25:36.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:36.573+0000 7fb580200700 1 -- 172.21.15.37:0/3720575759 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fb56006cc20 msgr2=0x7fb56006f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:25:36.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:36.573+0000 7fb580200700 1 --2- 172.21.15.37:0/3720575759 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fb56006cc20 0x7fb56006f0d0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fb56c004540 tx=0x7fb56c009e30 comp rx=0 tx=0).stop 2024-03-29T08:25:36.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:36.573+0000 7fb580200700 1 -- 172.21.15.37:0/3720575759 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb578070c10 msgr2=0x7fb578115f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:25:36.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:36.573+0000 7fb580200700 1 --2- 172.21.15.37:0/3720575759 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb578070c10 0x7fb578115f90 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fb56400c8c0 tx=0x7fb5640044c0 comp rx=0 tx=0).stop 2024-03-29T08:25:36.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:36.574+0000 7fb580200700 1 -- 172.21.15.37:0/3720575759 shutdown_connections 2024-03-29T08:25:36.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:36.574+0000 7fb580200700 1 --2- 172.21.15.37:0/3720575759 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fb56006cc20 0x7fb56006f0d0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:36.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:36.574+0000 7fb580200700 1 --2- 172.21.15.37:0/3720575759 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb578070300 0x7fb578115a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:36.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:36.574+0000 7fb580200700 1 --2- 172.21.15.37:0/3720575759 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb578070c10 0x7fb578115f90 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:36.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:36.574+0000 7fb580200700 1 -- 172.21.15.37:0/3720575759 >> 172.21.15.37:0/3720575759 conn(0x7fb5780f89b0 msgr2=0x7fb578105a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:25:36.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:36.574+0000 7fb580200700 1 -- 172.21.15.37:0/3720575759 shutdown_connections 2024-03-29T08:25:36.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:36.574+0000 7fb580200700 1 -- 172.21.15.37:0/3720575759 wait complete. 2024-03-29T08:25:36.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:36.709+0000 7fadb1c00700 1 -- 172.21.15.37:0/1096250428 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fadac0fe830 msgr2=0x7fadac106f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:25:36.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:36.709+0000 7fadb1c00700 1 --2- 172.21.15.37:0/1096250428 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fadac0fe830 0x7fadac106f00 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fad98004940 tx=0x7fad98009e30 comp rx=0 tx=0).stop 2024-03-29T08:25:36.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:36.709+0000 7fadb1c00700 1 -- 172.21.15.37:0/1096250428 shutdown_connections 2024-03-29T08:25:36.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:36.709+0000 7fadb1c00700 1 --2- 172.21.15.37:0/1096250428 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fadac0fe830 0x7fadac106f00 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:36.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:36.709+0000 7fadb1c00700 1 --2- 172.21.15.37:0/1096250428 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fadac0fde90 0x7fadac0fe260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:36.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:36.709+0000 7fadb1c00700 1 -- 172.21.15.37:0/1096250428 >> 172.21.15.37:0/1096250428 conn(0x7fadac072bc0 msgr2=0x7fadac072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:25:36.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:36.709+0000 7fadb1c00700 1 -- 172.21.15.37:0/1096250428 shutdown_connections 2024-03-29T08:25:36.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:36.709+0000 7fadb1c00700 1 -- 172.21.15.37:0/1096250428 wait complete. 2024-03-29T08:25:36.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:36.710+0000 7fadb1c00700 1 Processor -- start 2024-03-29T08:25:36.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:36.710+0000 7fadb1c00700 1 -- start start 2024-03-29T08:25:36.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:36.711+0000 7fadb1c00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fadac0fde90 0x7fadac113a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:36.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:36.711+0000 7fadb1c00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fadac0fe830 0x7fadac113fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:36.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:36.711+0000 7fadb1c00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadac114610 con 0x7fadac0fde90 2024-03-29T08:25:36.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:36.711+0000 7fadb1c00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadac10da00 con 0x7fadac0fe830 2024-03-29T08:25:36.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:36.711+0000 7fadaaa00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fadac0fde90 0x7fadac113a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:36.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:36.711+0000 7fadaaa00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fadac0fde90 0x7fadac113a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37828/0 (socket says 172.21.15.37:37828) 2024-03-29T08:25:36.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:36.711+0000 7fadaaa00700 1 -- 172.21.15.37:0/3363272540 learned_addr learned my addr 172.21.15.37:0/3363272540 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:25:36.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:36.711+0000 7fadaa000700 1 --2- 172.21.15.37:0/3363272540 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fadac0fe830 0x7fadac113fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:36.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:36.711+0000 7fadaaa00700 1 -- 172.21.15.37:0/3363272540 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fadac0fe830 msgr2=0x7fadac113fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:25:36.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:36.711+0000 7fadaaa00700 1 --2- 172.21.15.37:0/3363272540 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fadac0fe830 0x7fadac113fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:36.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:36.711+0000 7fadaaa00700 1 -- 172.21.15.37:0/3363272540 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad98005040 con 0x7fadac0fde90 2024-03-29T08:25:36.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:36.711+0000 7fadaa000700 1 --2- 172.21.15.37:0/3363272540 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fadac0fe830 0x7fadac113fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:25:36.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:36.712+0000 7fadaaa00700 1 --2- 172.21.15.37:0/3363272540 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fadac0fde90 0x7fadac113a80 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fad9c00eb40 tx=0x7fad9c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:25:36.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:36.712+0000 7fada2a00700 1 -- 172.21.15.37:0/3363272540 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fad9c007cc0 con 0x7fadac0fde90 2024-03-29T08:25:36.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:36.712+0000 7fada2a00700 1 -- 172.21.15.37:0/3363272540 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fad9c00f040 con 0x7fadac0fde90 2024-03-29T08:25:36.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:36.712+0000 7fadb1c00700 1 -- 172.21.15.37:0/3363272540 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadac10dc80 con 0x7fadac0fde90 2024-03-29T08:25:36.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:36.712+0000 7fada2a00700 1 -- 172.21.15.37:0/3363272540 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fad9c009570 con 0x7fadac0fde90 2024-03-29T08:25:36.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:36.712+0000 7fadb1c00700 1 -- 172.21.15.37:0/3363272540 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadac10e1c0 con 0x7fadac0fde90 2024-03-29T08:25:36.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:36.713+0000 7fadb1c00700 1 -- 172.21.15.37:0/3363272540 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fadac101980 con 0x7fadac0fde90 2024-03-29T08:25:36.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:36.714+0000 7fada2a00700 1 -- 172.21.15.37:0/3363272540 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fad9c0096d0 con 0x7fadac0fde90 2024-03-29T08:25:36.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:36.715+0000 7fada2a00700 1 --2- 172.21.15.37:0/3363272540 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fad9006cc30 0x7fad9006f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:36.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:36.715+0000 7fada2a00700 1 -- 172.21.15.37:0/3363272540 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fad9c08bda0 con 0x7fadac0fde90 2024-03-29T08:25:36.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:36.715+0000 7fadaa000700 1 --2- 172.21.15.37:0/3363272540 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fad9006cc30 0x7fad9006f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:36.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:36.716+0000 7fadaa000700 1 --2- 172.21.15.37:0/3363272540 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fad9006cc30 0x7fad9006f0e0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fad98000c00 tx=0x7fad98009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:25:36.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:36.720+0000 7fada2a00700 1 -- 172.21.15.37:0/3363272540 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fad9c059820 con 0x7fadac0fde90 2024-03-29T08:25:37.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:37.018+0000 7fadb1c00700 1 -- 172.21.15.37:0/3363272540 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fadac10f0d0 con 0x7fadac0fde90 2024-03-29T08:25:37.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:37.019+0000 7fada2a00700 1 -- 172.21.15.37:0/3363272540 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+847 (secure 0 0 0) 0x7fad9c0593b0 con 0x7fadac0fde90 2024-03-29T08:25:37.019 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-29T08:25:37.019 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": { 2024-03-29T08:25:37.019 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 2 2024-03-29T08:25:37.019 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:25:37.019 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": { 2024-03-29T08:25:37.019 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 2 2024-03-29T08:25:37.019 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:25:37.019 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": { 2024-03-29T08:25:37.020 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 6 2024-03-29T08:25:37.020 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:25:37.020 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": { 2024-03-29T08:25:37.020 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 17.2.7-789-g4213e17c (4213e17c4de67016ed14f55d22503f4923b3a901) quincy (stable)": 1, 2024-03-29T08:25:37.020 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 3 2024-03-29T08:25:37.020 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:25:37.020 INFO:teuthology.orchestra.run.smithi037.stdout: "overall": { 2024-03-29T08:25:37.020 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 17.2.7-789-g4213e17c (4213e17c4de67016ed14f55d22503f4923b3a901) quincy (stable)": 1, 2024-03-29T08:25:37.020 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 13 2024-03-29T08:25:37.020 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-29T08:25:37.020 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-29T08:25:37.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:37.021+0000 7fadb1c00700 1 -- 172.21.15.37:0/3363272540 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fad9006cc30 msgr2=0x7fad9006f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:25:37.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:37.021+0000 7fadb1c00700 1 --2- 172.21.15.37:0/3363272540 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fad9006cc30 0x7fad9006f0e0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fad98000c00 tx=0x7fad98009e30 comp rx=0 tx=0).stop 2024-03-29T08:25:37.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:37.021+0000 7fadb1c00700 1 -- 172.21.15.37:0/3363272540 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fadac0fde90 msgr2=0x7fadac113a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:25:37.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:37.021+0000 7fadb1c00700 1 --2- 172.21.15.37:0/3363272540 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fadac0fde90 0x7fadac113a80 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fad9c00eb40 tx=0x7fad9c00b040 comp rx=0 tx=0).stop 2024-03-29T08:25:37.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:37.021+0000 7fadb1c00700 1 -- 172.21.15.37:0/3363272540 shutdown_connections 2024-03-29T08:25:37.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:37.021+0000 7fadb1c00700 1 --2- 172.21.15.37:0/3363272540 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fad9006cc30 0x7fad9006f0e0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:37.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:37.021+0000 7fadb1c00700 1 --2- 172.21.15.37:0/3363272540 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fadac0fde90 0x7fadac113a80 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:37.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:37.021+0000 7fadb1c00700 1 --2- 172.21.15.37:0/3363272540 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fadac0fe830 0x7fadac113fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:37.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:37.021+0000 7fadb1c00700 1 -- 172.21.15.37:0/3363272540 >> 172.21.15.37:0/3363272540 conn(0x7fadac072bc0 msgr2=0x7fadac0ff3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:25:37.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:37.021+0000 7fadb1c00700 1 -- 172.21.15.37:0/3363272540 shutdown_connections 2024-03-29T08:25:37.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:37.022+0000 7fadb1c00700 1 -- 172.21.15.37:0/3363272540 wait complete. 2024-03-29T08:25:37.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:25:36 smithi148 ceph-mon[106692]: from='client.34840 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:25:37.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:25:36 smithi148 ceph-mon[106692]: from='client.34844 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:25:37.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:25:36 smithi148 ceph-mon[106692]: from='client.34848 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:25:37.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:37.159+0000 7f3e58c00700 1 -- 172.21.15.37:0/779451489 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3e54070c70 msgr2=0x7f3e54109110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:25:37.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:37.159+0000 7f3e58c00700 1 --2- 172.21.15.37:0/779451489 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3e54070c70 0x7f3e54109110 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f3e40004970 tx=0x7f3e40009e30 comp rx=0 tx=0).stop 2024-03-29T08:25:37.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:37.159+0000 7f3e58c00700 1 -- 172.21.15.37:0/779451489 shutdown_connections 2024-03-29T08:25:37.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:37.159+0000 7f3e58c00700 1 --2- 172.21.15.37:0/779451489 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3e54070c70 0x7f3e54109110 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:37.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:37.159+0000 7f3e58c00700 1 --2- 172.21.15.37:0/779451489 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3e54070360 0x7f3e54070730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:37.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:37.159+0000 7f3e58c00700 1 -- 172.21.15.37:0/779451489 >> 172.21.15.37:0/779451489 conn(0x7f3e540f89b0 msgr2=0x7f3e540fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:25:37.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:37.160+0000 7f3e58c00700 1 -- 172.21.15.37:0/779451489 shutdown_connections 2024-03-29T08:25:37.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:37.160+0000 7f3e58c00700 1 -- 172.21.15.37:0/779451489 wait complete. 2024-03-29T08:25:37.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:37.160+0000 7f3e58c00700 1 Processor -- start 2024-03-29T08:25:37.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:37.161+0000 7f3e58c00700 1 -- start start 2024-03-29T08:25:37.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:37.161+0000 7f3e58c00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3e54070360 0x7f3e54115ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:37.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:37.161+0000 7f3e58c00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3e54070c70 0x7f3e541161e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:37.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:37.161+0000 7f3e58c00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e54116830 con 0x7f3e54070360 2024-03-29T08:25:37.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:37.161+0000 7f3e58c00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e5410fc00 con 0x7f3e54070c70 2024-03-29T08:25:37.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:37.161+0000 7f3e51e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3e54070360 0x7f3e54115ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:37.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:37.161+0000 7f3e51e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3e54070360 0x7f3e54115ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37850/0 (socket says 172.21.15.37:37850) 2024-03-29T08:25:37.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:37.161+0000 7f3e51e00700 1 -- 172.21.15.37:0/1354830680 learned_addr learned my addr 172.21.15.37:0/1354830680 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:25:37.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:37.161+0000 7f3e51400700 1 --2- 172.21.15.37:0/1354830680 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3e54070c70 0x7f3e541161e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:37.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:37.162+0000 7f3e51e00700 1 -- 172.21.15.37:0/1354830680 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3e54070c70 msgr2=0x7f3e541161e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:25:37.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:37.162+0000 7f3e51e00700 1 --2- 172.21.15.37:0/1354830680 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3e54070c70 0x7f3e541161e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:37.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:37.162+0000 7f3e51e00700 1 -- 172.21.15.37:0/1354830680 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e40005040 con 0x7f3e54070360 2024-03-29T08:25:37.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:37.162+0000 7f3e51e00700 1 --2- 172.21.15.37:0/1354830680 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3e54070360 0x7f3e54115ca0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f3e44002a50 tx=0x7f3e440046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:25:37.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:37.162+0000 7f3e4a000700 1 -- 172.21.15.37:0/1354830680 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3e44007cc0 con 0x7f3e54070360 2024-03-29T08:25:37.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:37.162+0000 7f3e4a000700 1 -- 172.21.15.37:0/1354830680 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f3e4400f040 con 0x7f3e54070360 2024-03-29T08:25:37.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:37.162+0000 7f3e58c00700 1 -- 172.21.15.37:0/1354830680 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e5410fee0 con 0x7f3e54070360 2024-03-29T08:25:37.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:37.163+0000 7f3e4a000700 1 -- 172.21.15.37:0/1354830680 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3e4400d9c0 con 0x7f3e54070360 2024-03-29T08:25:37.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:37.163+0000 7f3e58c00700 1 -- 172.21.15.37:0/1354830680 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e541103f0 con 0x7f3e54070360 2024-03-29T08:25:37.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:37.164+0000 7f3e58c00700 1 -- 172.21.15.37:0/1354830680 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e54103bd0 con 0x7f3e54070360 2024-03-29T08:25:37.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:37.166+0000 7f3e4a000700 1 -- 172.21.15.37:0/1354830680 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f3e4400db20 con 0x7f3e54070360 2024-03-29T08:25:37.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:37.167+0000 7f3e4a000700 1 --2- 172.21.15.37:0/1354830680 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f3e3806cc30 0x7f3e3806f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:37.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:37.167+0000 7f3e4a000700 1 -- 172.21.15.37:0/1354830680 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f3e4408bd20 con 0x7f3e54070360 2024-03-29T08:25:37.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:37.167+0000 7f3e51400700 1 --2- 172.21.15.37:0/1354830680 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f3e3806cc30 0x7f3e3806f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:37.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:37.168+0000 7f3e51400700 1 --2- 172.21.15.37:0/1354830680 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f3e3806cc30 0x7f3e3806f0e0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f3e4000b370 tx=0x7f3e40009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:25:37.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:37.171+0000 7f3e4a000700 1 -- 172.21.15.37:0/1354830680 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f3e44059770 con 0x7f3e54070360 2024-03-29T08:25:37.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:25:36 smithi037 ceph-mon[142325]: from='client.34840 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:25:37.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:25:36 smithi037 ceph-mon[142325]: from='client.34844 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:25:37.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:25:36 smithi037 ceph-mon[142325]: from='client.34848 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:25:37.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:37.429+0000 7f3e58c00700 1 -- 172.21.15.37:0/1354830680 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f3e5410bfe0 con 0x7f3e54070360 2024-03-29T08:25:37.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:37.429+0000 7f3e4a000700 1 -- 172.21.15.37:0/1354830680 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 28 v28) v1 ==== 76+0+1841 (secure 0 0 0) 0x7f3e44059300 con 0x7f3e54070360 2024-03-29T08:25:37.429 INFO:teuthology.orchestra.run.smithi037.stdout:e28 2024-03-29T08:25:37.430 INFO:teuthology.orchestra.run.smithi037.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-29T08:25:37.430 INFO:teuthology.orchestra.run.smithi037.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T08:25:37.430 INFO:teuthology.orchestra.run.smithi037.stdout:legacy client fscid: 1 2024-03-29T08:25:37.430 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:25:37.430 INFO:teuthology.orchestra.run.smithi037.stdout:Filesystem 'cephfs' (1) 2024-03-29T08:25:37.430 INFO:teuthology.orchestra.run.smithi037.stdout:fs_name cephfs 2024-03-29T08:25:37.430 INFO:teuthology.orchestra.run.smithi037.stdout:epoch 21 2024-03-29T08:25:37.430 INFO:teuthology.orchestra.run.smithi037.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-03-29T08:25:37.430 INFO:teuthology.orchestra.run.smithi037.stdout:created 2024-03-29T08:03:29.241269+0000 2024-03-29T08:25:37.430 INFO:teuthology.orchestra.run.smithi037.stdout:modified 2024-03-29T08:17:16.237238+0000 2024-03-29T08:25:37.430 INFO:teuthology.orchestra.run.smithi037.stdout:tableserver 0 2024-03-29T08:25:37.430 INFO:teuthology.orchestra.run.smithi037.stdout:root 0 2024-03-29T08:25:37.430 INFO:teuthology.orchestra.run.smithi037.stdout:session_timeout 60 2024-03-29T08:25:37.431 INFO:teuthology.orchestra.run.smithi037.stdout:session_autoclose 300 2024-03-29T08:25:37.431 INFO:teuthology.orchestra.run.smithi037.stdout:max_file_size 1099511627776 2024-03-29T08:25:37.431 INFO:teuthology.orchestra.run.smithi037.stdout:max_xattr_size 65536 2024-03-29T08:25:37.431 INFO:teuthology.orchestra.run.smithi037.stdout:required_client_features {} 2024-03-29T08:25:37.431 INFO:teuthology.orchestra.run.smithi037.stdout:last_failure 0 2024-03-29T08:25:37.431 INFO:teuthology.orchestra.run.smithi037.stdout:last_failure_osd_epoch 82 2024-03-29T08:25:37.431 INFO:teuthology.orchestra.run.smithi037.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T08:25:37.431 INFO:teuthology.orchestra.run.smithi037.stdout:max_mds 2 2024-03-29T08:25:37.431 INFO:teuthology.orchestra.run.smithi037.stdout:in 0,1 2024-03-29T08:25:37.431 INFO:teuthology.orchestra.run.smithi037.stdout:up {} 2024-03-29T08:25:37.431 INFO:teuthology.orchestra.run.smithi037.stdout:failed 0,1 2024-03-29T08:25:37.431 INFO:teuthology.orchestra.run.smithi037.stdout:damaged 2024-03-29T08:25:37.431 INFO:teuthology.orchestra.run.smithi037.stdout:stopped 2024-03-29T08:25:37.431 INFO:teuthology.orchestra.run.smithi037.stdout:data_pools [3] 2024-03-29T08:25:37.431 INFO:teuthology.orchestra.run.smithi037.stdout:metadata_pool 2 2024-03-29T08:25:37.432 INFO:teuthology.orchestra.run.smithi037.stdout:inline_data disabled 2024-03-29T08:25:37.432 INFO:teuthology.orchestra.run.smithi037.stdout:balancer 2024-03-29T08:25:37.432 INFO:teuthology.orchestra.run.smithi037.stdout:bal_rank_mask -1 2024-03-29T08:25:37.432 INFO:teuthology.orchestra.run.smithi037.stdout:standby_count_wanted 1 2024-03-29T08:25:37.432 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:25:37.432 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:25:37.432 INFO:teuthology.orchestra.run.smithi037.stdout:Standby daemons: 2024-03-29T08:25:37.432 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:25:37.432 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi148.qukpjh{-1:24319} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.148:6826/2641327912,v1:172.21.15.148:6827/2641327912] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:25:37.432 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi037.mmwgxe{-1:34440} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.37:6828/1248142005,v1:172.21.15.37:6829/1248142005] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:25:37.432 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi037.iifsmh{-1:34444} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.37:6826/2060882959,v1:172.21.15.37:6827/2060882959] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:25:37.432 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi148.sqteih{-1:44395} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.148:6824/4078652242,v1:172.21.15.148:6825/4078652242] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:25:37.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:37.431+0000 7f3e58c00700 1 -- 172.21.15.37:0/1354830680 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f3e3806cc30 msgr2=0x7f3e3806f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:25:37.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:37.431+0000 7f3e58c00700 1 --2- 172.21.15.37:0/1354830680 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f3e3806cc30 0x7f3e3806f0e0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f3e4000b370 tx=0x7f3e40009e30 comp rx=0 tx=0).stop 2024-03-29T08:25:37.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:37.431+0000 7f3e58c00700 1 -- 172.21.15.37:0/1354830680 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3e54070360 msgr2=0x7f3e54115ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:25:37.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:37.431+0000 7f3e58c00700 1 --2- 172.21.15.37:0/1354830680 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3e54070360 0x7f3e54115ca0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f3e44002a50 tx=0x7f3e440046b0 comp rx=0 tx=0).stop 2024-03-29T08:25:37.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:37.432+0000 7f3e58c00700 1 -- 172.21.15.37:0/1354830680 shutdown_connections 2024-03-29T08:25:37.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:37.432+0000 7f3e58c00700 1 --2- 172.21.15.37:0/1354830680 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f3e3806cc30 0x7f3e3806f0e0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:37.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:37.432+0000 7f3e58c00700 1 --2- 172.21.15.37:0/1354830680 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3e54070360 0x7f3e54115ca0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:37.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:37.432+0000 7f3e58c00700 1 --2- 172.21.15.37:0/1354830680 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3e54070c70 0x7f3e541161e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:37.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:37.432+0000 7f3e58c00700 1 -- 172.21.15.37:0/1354830680 >> 172.21.15.37:0/1354830680 conn(0x7f3e540f89b0 msgr2=0x7f3e541015f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:25:37.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:37.432+0000 7f3e58c00700 1 -- 172.21.15.37:0/1354830680 shutdown_connections 2024-03-29T08:25:37.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:37.432+0000 7f3e58c00700 1 -- 172.21.15.37:0/1354830680 wait complete. 2024-03-29T08:25:37.434 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 28 2024-03-29T08:25:37.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:37.569+0000 7fc118200700 1 -- 172.21.15.37:0/1405665087 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc110105000 msgr2=0x7fc1101053d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:25:37.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:37.569+0000 7fc118200700 1 --2- 172.21.15.37:0/1405665087 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc110105000 0x7fc1101053d0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fc104004940 tx=0x7fc104009e30 comp rx=0 tx=0).stop 2024-03-29T08:25:37.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:37.569+0000 7fc118200700 1 -- 172.21.15.37:0/1405665087 shutdown_connections 2024-03-29T08:25:37.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:37.569+0000 7fc118200700 1 --2- 172.21.15.37:0/1405665087 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc1101059a0 0x7fc1100ff010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:37.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:37.569+0000 7fc118200700 1 --2- 172.21.15.37:0/1405665087 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc110105000 0x7fc1101053d0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:37.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:37.569+0000 7fc118200700 1 -- 172.21.15.37:0/1405665087 >> 172.21.15.37:0/1405665087 conn(0x7fc1100fabd0 msgr2=0x7fc1100fcfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:25:37.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:37.569+0000 7fc118200700 1 -- 172.21.15.37:0/1405665087 shutdown_connections 2024-03-29T08:25:37.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:37.569+0000 7fc118200700 1 -- 172.21.15.37:0/1405665087 wait complete. 2024-03-29T08:25:37.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:37.570+0000 7fc118200700 1 Processor -- start 2024-03-29T08:25:37.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:37.570+0000 7fc118200700 1 -- start start 2024-03-29T08:25:37.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:37.570+0000 7fc118200700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc110105000 0x7fc110115d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:37.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:37.571+0000 7fc118200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc1101059a0 0x7fc1101162b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:37.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:37.571+0000 7fc118200700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc110116900 con 0x7fc1101059a0 2024-03-29T08:25:37.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:37.571+0000 7fc118200700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc11010fcd0 con 0x7fc110105000 2024-03-29T08:25:37.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:37.571+0000 7fc114e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc1101059a0 0x7fc1101162b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:37.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:37.571+0000 7fc114e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc1101059a0 0x7fc1101162b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37866/0 (socket says 172.21.15.37:37866) 2024-03-29T08:25:37.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:37.571+0000 7fc114e00700 1 -- 172.21.15.37:0/1346181448 learned_addr learned my addr 172.21.15.37:0/1346181448 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:25:37.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:37.571+0000 7fc115800700 1 --2- 172.21.15.37:0/1346181448 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc110105000 0x7fc110115d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:37.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:37.571+0000 7fc114e00700 1 -- 172.21.15.37:0/1346181448 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc110105000 msgr2=0x7fc110115d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:25:37.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:37.571+0000 7fc114e00700 1 --2- 172.21.15.37:0/1346181448 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc110105000 0x7fc110115d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:37.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:37.571+0000 7fc114e00700 1 -- 172.21.15.37:0/1346181448 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc104005040 con 0x7fc1101059a0 2024-03-29T08:25:37.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:37.572+0000 7fc114e00700 1 --2- 172.21.15.37:0/1346181448 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc1101059a0 0x7fc1101162b0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fc0fc002a50 tx=0x7fc0fc004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:25:37.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:37.572+0000 7fc10d600700 1 -- 172.21.15.37:0/1346181448 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc0fc007cc0 con 0x7fc1101059a0 2024-03-29T08:25:37.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:37.572+0000 7fc10d600700 1 -- 172.21.15.37:0/1346181448 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fc0fc00f040 con 0x7fc1101059a0 2024-03-29T08:25:37.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:37.572+0000 7fc118200700 1 -- 172.21.15.37:0/1346181448 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc11010ffb0 con 0x7fc1101059a0 2024-03-29T08:25:37.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:37.572+0000 7fc10d600700 1 -- 172.21.15.37:0/1346181448 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc0fc00d920 con 0x7fc1101059a0 2024-03-29T08:25:37.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:37.572+0000 7fc118200700 1 -- 172.21.15.37:0/1346181448 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1101104f0 con 0x7fc1101059a0 2024-03-29T08:25:37.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:37.573+0000 7fc118200700 1 -- 172.21.15.37:0/1346181448 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc110102310 con 0x7fc1101059a0 2024-03-29T08:25:37.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:37.574+0000 7fc10d600700 1 -- 172.21.15.37:0/1346181448 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fc0fc009680 con 0x7fc1101059a0 2024-03-29T08:25:37.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:37.575+0000 7fc10d600700 1 --2- 172.21.15.37:0/1346181448 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fc0f806cc30 0x7fc0f806f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:37.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:37.575+0000 7fc10d600700 1 -- 172.21.15.37:0/1346181448 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fc0fc08bd80 con 0x7fc1101059a0 2024-03-29T08:25:37.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:37.576+0000 7fc115800700 1 --2- 172.21.15.37:0/1346181448 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fc0f806cc30 0x7fc0f806f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:37.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:37.576+0000 7fc115800700 1 --2- 172.21.15.37:0/1346181448 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fc0f806cc30 0x7fc0f806f0e0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fc10400b840 tx=0x7fc104009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:25:37.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:37.579+0000 7fc10d600700 1 -- 172.21.15.37:0/1346181448 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fc0fc0597d0 con 0x7fc1101059a0 2024-03-29T08:25:37.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:37.812+0000 7fc118200700 1 -- 172.21.15.37:0/1346181448 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc110111430 con 0x7fc0f806cc30 2024-03-29T08:25:37.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:37.813+0000 7fc10d600700 1 -- 172.21.15.37:0/1346181448 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fc110111430 con 0x7fc0f806cc30 2024-03-29T08:25:37.813 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-29T08:25:37.813 INFO:teuthology.orchestra.run.smithi037.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59", 2024-03-29T08:25:37.813 INFO:teuthology.orchestra.run.smithi037.stdout: "in_progress": true, 2024-03-29T08:25:37.813 INFO:teuthology.orchestra.run.smithi037.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-29T08:25:37.813 INFO:teuthology.orchestra.run.smithi037.stdout: "services_complete": [ 2024-03-29T08:25:37.814 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr", 2024-03-29T08:25:37.814 INFO:teuthology.orchestra.run.smithi037.stdout: "mon", 2024-03-29T08:25:37.814 INFO:teuthology.orchestra.run.smithi037.stdout: "crash", 2024-03-29T08:25:37.814 INFO:teuthology.orchestra.run.smithi037.stdout: "osd" 2024-03-29T08:25:37.814 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-29T08:25:37.814 INFO:teuthology.orchestra.run.smithi037.stdout: "progress": "15/21 daemons upgraded", 2024-03-29T08:25:37.814 INFO:teuthology.orchestra.run.smithi037.stdout: "message": "Currently upgrading mds daemons", 2024-03-29T08:25:37.814 INFO:teuthology.orchestra.run.smithi037.stdout: "is_paused": false 2024-03-29T08:25:37.814 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-29T08:25:37.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:37.815+0000 7fc118200700 1 -- 172.21.15.37:0/1346181448 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fc0f806cc30 msgr2=0x7fc0f806f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:25:37.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:37.815+0000 7fc118200700 1 --2- 172.21.15.37:0/1346181448 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fc0f806cc30 0x7fc0f806f0e0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fc10400b840 tx=0x7fc104009e30 comp rx=0 tx=0).stop 2024-03-29T08:25:37.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:37.816+0000 7fc118200700 1 -- 172.21.15.37:0/1346181448 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc1101059a0 msgr2=0x7fc1101162b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:25:37.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:37.816+0000 7fc118200700 1 --2- 172.21.15.37:0/1346181448 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc1101059a0 0x7fc1101162b0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fc0fc002a50 tx=0x7fc0fc004600 comp rx=0 tx=0).stop 2024-03-29T08:25:37.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:37.816+0000 7fc118200700 1 -- 172.21.15.37:0/1346181448 shutdown_connections 2024-03-29T08:25:37.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:37.816+0000 7fc118200700 1 --2- 172.21.15.37:0/1346181448 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fc0f806cc30 0x7fc0f806f0e0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:37.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:37.816+0000 7fc118200700 1 --2- 172.21.15.37:0/1346181448 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc110105000 0x7fc110115d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:37.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:37.816+0000 7fc118200700 1 --2- 172.21.15.37:0/1346181448 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc1101059a0 0x7fc1101162b0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:37.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:37.816+0000 7fc118200700 1 -- 172.21.15.37:0/1346181448 >> 172.21.15.37:0/1346181448 conn(0x7fc1100fabd0 msgr2=0x7fc110108630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:25:37.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:37.816+0000 7fc118200700 1 -- 172.21.15.37:0/1346181448 shutdown_connections 2024-03-29T08:25:37.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:37.816+0000 7fc118200700 1 -- 172.21.15.37:0/1346181448 wait complete. 2024-03-29T08:25:37.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:37.952+0000 7ff798600700 1 -- 172.21.15.37:0/714341439 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff790102e00 msgr2=0x7ff7901031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:25:37.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:37.952+0000 7ff798600700 1 --2- 172.21.15.37:0/714341439 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff790102e00 0x7ff7901031d0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7ff784004940 tx=0x7ff784009e30 comp rx=0 tx=0).stop 2024-03-29T08:25:37.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:37.953+0000 7ff798600700 1 -- 172.21.15.37:0/714341439 shutdown_connections 2024-03-29T08:25:37.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:37.953+0000 7ff798600700 1 --2- 172.21.15.37:0/714341439 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff7901037a0 0x7ff7900fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:37.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:37.953+0000 7ff798600700 1 --2- 172.21.15.37:0/714341439 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff790102e00 0x7ff7901031d0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:37.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:37.953+0000 7ff798600700 1 -- 172.21.15.37:0/714341439 >> 172.21.15.37:0/714341439 conn(0x7ff7900f89b0 msgr2=0x7ff7900fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:25:37.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:37.953+0000 7ff798600700 1 -- 172.21.15.37:0/714341439 shutdown_connections 2024-03-29T08:25:37.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:37.953+0000 7ff798600700 1 -- 172.21.15.37:0/714341439 wait complete. 2024-03-29T08:25:37.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:37.954+0000 7ff798600700 1 Processor -- start 2024-03-29T08:25:37.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:37.954+0000 7ff798600700 1 -- start start 2024-03-29T08:25:37.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:37.954+0000 7ff798600700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff790102e00 0x7ff7900702c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:37.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:37.954+0000 7ff798600700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff7901037a0 0x7ff790070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:37.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:37.954+0000 7ff798600700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff790070d90 con 0x7ff7901037a0 2024-03-29T08:25:37.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:37.954+0000 7ff798600700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff790070ed0 con 0x7ff790102e00 2024-03-29T08:25:37.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:37.954+0000 7ff795200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff7901037a0 0x7ff790070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:37.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:37.954+0000 7ff795200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff7901037a0 0x7ff790070800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37884/0 (socket says 172.21.15.37:37884) 2024-03-29T08:25:37.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:37.955+0000 7ff795200700 1 -- 172.21.15.37:0/1372728984 learned_addr learned my addr 172.21.15.37:0/1372728984 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:25:37.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:37.955+0000 7ff795c00700 1 --2- 172.21.15.37:0/1372728984 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff790102e00 0x7ff7900702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:37.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:37.955+0000 7ff795200700 1 -- 172.21.15.37:0/1372728984 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff790102e00 msgr2=0x7ff7900702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:25:37.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:37.955+0000 7ff795200700 1 --2- 172.21.15.37:0/1372728984 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff790102e00 0x7ff7900702c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:37.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:37.955+0000 7ff795200700 1 -- 172.21.15.37:0/1372728984 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff784005040 con 0x7ff7901037a0 2024-03-29T08:25:37.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:37.955+0000 7ff795c00700 1 --2- 172.21.15.37:0/1372728984 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff790102e00 0x7ff7900702c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:25:37.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:37.955+0000 7ff795200700 1 --2- 172.21.15.37:0/1372728984 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff7901037a0 0x7ff790070800 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7ff77c002a50 tx=0x7ff77c004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:25:37.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:37.955+0000 7ff78d600700 1 -- 172.21.15.37:0/1372728984 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff77c007cc0 con 0x7ff7901037a0 2024-03-29T08:25:37.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:37.955+0000 7ff78d600700 1 -- 172.21.15.37:0/1372728984 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ff77c00f040 con 0x7ff7901037a0 2024-03-29T08:25:37.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:37.955+0000 7ff78d600700 1 -- 172.21.15.37:0/1372728984 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff77c00d990 con 0x7ff7901037a0 2024-03-29T08:25:37.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:37.955+0000 7ff798600700 1 -- 172.21.15.37:0/1372728984 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff790119e20 con 0x7ff7901037a0 2024-03-29T08:25:37.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:37.956+0000 7ff798600700 1 -- 172.21.15.37:0/1372728984 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff79011a2a0 con 0x7ff7901037a0 2024-03-29T08:25:37.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:37.957+0000 7ff798600700 1 -- 172.21.15.37:0/1372728984 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7900fe550 con 0x7ff7901037a0 2024-03-29T08:25:37.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:37.958+0000 7ff78d600700 1 -- 172.21.15.37:0/1372728984 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7ff77c009710 con 0x7ff7901037a0 2024-03-29T08:25:37.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:37.959+0000 7ff78d600700 1 --2- 172.21.15.37:0/1372728984 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7ff77806cc30 0x7ff77806f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:37.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:37.959+0000 7ff78d600700 1 -- 172.21.15.37:0/1372728984 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7ff77c08bdf0 con 0x7ff7901037a0 2024-03-29T08:25:37.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:37.959+0000 7ff795c00700 1 --2- 172.21.15.37:0/1372728984 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7ff77806cc30 0x7ff77806f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:37.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:37.960+0000 7ff795c00700 1 --2- 172.21.15.37:0/1372728984 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7ff77806cc30 0x7ff77806f0e0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7ff784000c00 tx=0x7ff784009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:25:37.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:37.963+0000 7ff78d600700 1 -- 172.21.15.37:0/1372728984 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7ff77c059840 con 0x7ff7901037a0 2024-03-29T08:25:38.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:25:37 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/3363272540' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:25:38.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:25:37 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/1354830680' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T08:25:38.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:25:37 smithi148 ceph-mon[106692]: pgmap v442: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:25:38.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:25:37 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/3363272540' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:25:38.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:25:37 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/1354830680' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T08:25:38.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:25:37 smithi037 ceph-mon[142325]: pgmap v442: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:25:38.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:38.260+0000 7ff798600700 1 -- 172.21.15.37:0/1372728984 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7ff79005f5f0 con 0x7ff7901037a0 2024-03-29T08:25:38.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:38.260+0000 7ff78d600700 1 -- 172.21.15.37:0/1372728984 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7ff77c0593d0 con 0x7ff7901037a0 2024-03-29T08:25:38.260 INFO:teuthology.orchestra.run.smithi037.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-03-29T08:25:38.261 INFO:teuthology.orchestra.run.smithi037.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-03-29T08:25:38.261 INFO:teuthology.orchestra.run.smithi037.stdout: fs cephfs is degraded 2024-03-29T08:25:38.261 INFO:teuthology.orchestra.run.smithi037.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-03-29T08:25:38.261 INFO:teuthology.orchestra.run.smithi037.stdout: fs cephfs is offline because no MDS is active for it. 2024-03-29T08:25:38.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:38.262+0000 7ff798600700 1 -- 172.21.15.37:0/1372728984 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7ff77806cc30 msgr2=0x7ff77806f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:25:38.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:38.262+0000 7ff798600700 1 --2- 172.21.15.37:0/1372728984 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7ff77806cc30 0x7ff77806f0e0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7ff784000c00 tx=0x7ff784009e30 comp rx=0 tx=0).stop 2024-03-29T08:25:38.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:38.262+0000 7ff798600700 1 -- 172.21.15.37:0/1372728984 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff7901037a0 msgr2=0x7ff790070800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:25:38.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:38.262+0000 7ff798600700 1 --2- 172.21.15.37:0/1372728984 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff7901037a0 0x7ff790070800 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7ff77c002a50 tx=0x7ff77c004600 comp rx=0 tx=0).stop 2024-03-29T08:25:38.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:38.263+0000 7ff798600700 1 -- 172.21.15.37:0/1372728984 shutdown_connections 2024-03-29T08:25:38.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:38.263+0000 7ff798600700 1 --2- 172.21.15.37:0/1372728984 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7ff77806cc30 0x7ff77806f0e0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:38.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:38.263+0000 7ff798600700 1 --2- 172.21.15.37:0/1372728984 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff790102e00 0x7ff7900702c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:38.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:38.263+0000 7ff798600700 1 --2- 172.21.15.37:0/1372728984 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff7901037a0 0x7ff790070800 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:38.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:38.263+0000 7ff798600700 1 -- 172.21.15.37:0/1372728984 >> 172.21.15.37:0/1372728984 conn(0x7ff7900f89b0 msgr2=0x7ff7900f9a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:25:38.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:38.263+0000 7ff798600700 1 -- 172.21.15.37:0/1372728984 shutdown_connections 2024-03-29T08:25:38.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:25:38.263+0000 7ff798600700 1 -- 172.21.15.37:0/1372728984 wait complete. 2024-03-29T08:25:39.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:25:38 smithi148 ceph-mon[106692]: from='client.34860 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:25:39.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:25:38 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/1372728984' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T08:25:39.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:25:38 smithi037 ceph-mon[142325]: from='client.34860 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:25:39.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:25:38 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/1372728984' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T08:25:40.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:25:40 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:25:40.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:25:40 smithi148 ceph-mon[106692]: pgmap v443: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:25:40.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:25:40 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:25:40.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:25:40 smithi037 ceph-mon[142325]: pgmap v443: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:25:42.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:25:41 smithi148 ceph-mon[106692]: pgmap v444: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:25:42.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:25:41 smithi037 ceph-mon[142325]: pgmap v444: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:25:44.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:25:43 smithi148 ceph-mon[106692]: pgmap v445: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:25:44.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:25:43 smithi037 ceph-mon[142325]: pgmap v445: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:25:46.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:25:45 smithi148 ceph-mon[106692]: pgmap v446: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:25:46.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:25:45 smithi037 ceph-mon[142325]: pgmap v446: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:25:48.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:25:47 smithi148 ceph-mon[106692]: pgmap v447: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:25:48.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:25:47 smithi037 ceph-mon[142325]: pgmap v447: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:25:50.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:25:50 smithi148 ceph-mon[106692]: pgmap v448: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:25:50.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:25:50 smithi037 ceph-mon[142325]: pgmap v448: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:25:52.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:25:51 smithi148 ceph-mon[106692]: pgmap v449: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:25:52.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:25:51 smithi037 ceph-mon[142325]: pgmap v449: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:25:54.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:25:53 smithi148 ceph-mon[106692]: pgmap v450: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:25:54.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:25:53 smithi037 ceph-mon[142325]: pgmap v450: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:25:55.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:25:55 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:25:55.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:25:55 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:25:56.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:25:56 smithi148 ceph-mon[106692]: pgmap v451: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:25:56.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:25:56 smithi037 ceph-mon[142325]: pgmap v451: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:25:58.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:25:57 smithi148 ceph-mon[106692]: pgmap v452: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:25:58.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:25:57 smithi037 ceph-mon[142325]: pgmap v452: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:26:01.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:26:01 smithi037 ceph-mon[142325]: pgmap v453: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:26:01.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:26:01 smithi148 ceph-mon[106692]: pgmap v453: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:26:03.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:26:03 smithi037 ceph-mon[142325]: pgmap v454: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:26:03.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:26:03 smithi148 ceph-mon[106692]: pgmap v454: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:26:05.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:26:05 smithi148 ceph-mon[106692]: pgmap v455: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:26:05.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:26:05 smithi037 ceph-mon[142325]: pgmap v455: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:26:07.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:26:07 smithi148 ceph-mon[106692]: pgmap v456: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:26:07.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:26:07 smithi037 ceph-mon[142325]: pgmap v456: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:26:08.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:08.415+0000 7f878b400700 1 -- 172.21.15.37:0/2260910134 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8784105a30 msgr2=0x7f87840ff0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:08.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:08.415+0000 7f878b400700 1 --2- 172.21.15.37:0/2260910134 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8784105a30 0x7f87840ff0a0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f8770004940 tx=0x7f8770009e30 comp rx=0 tx=0).stop 2024-03-29T08:26:08.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:08.416+0000 7f878b400700 1 -- 172.21.15.37:0/2260910134 shutdown_connections 2024-03-29T08:26:08.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:08.416+0000 7f878b400700 1 --2- 172.21.15.37:0/2260910134 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8784105a30 0x7f87840ff0a0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:08.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:08.416+0000 7f878b400700 1 --2- 172.21.15.37:0/2260910134 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8784105090 0x7f8784105460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:08.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:08.416+0000 7f878b400700 1 -- 172.21.15.37:0/2260910134 >> 172.21.15.37:0/2260910134 conn(0x7f87840fac00 msgr2=0x7f87840fd010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:26:08.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:08.416+0000 7f878b400700 1 -- 172.21.15.37:0/2260910134 shutdown_connections 2024-03-29T08:26:08.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:08.416+0000 7f878b400700 1 -- 172.21.15.37:0/2260910134 wait complete. 2024-03-29T08:26:08.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:08.417+0000 7f878b400700 1 Processor -- start 2024-03-29T08:26:08.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:08.417+0000 7f878b400700 1 -- start start 2024-03-29T08:26:08.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:08.417+0000 7f878b400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8784105090 0x7f8784115a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:08.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:08.417+0000 7f878b400700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8784105a30 0x7f8784115fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:08.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:08.417+0000 7f878b400700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8784116600 con 0x7f8784105090 2024-03-29T08:26:08.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:08.417+0000 7f878b400700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f878410fbe0 con 0x7f8784105a30 2024-03-29T08:26:08.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:08.417+0000 7f8788a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8784105090 0x7f8784115a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:08.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:08.417+0000 7f8788a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8784105090 0x7f8784115a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53680/0 (socket says 172.21.15.37:53680) 2024-03-29T08:26:08.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:08.417+0000 7f8788a00700 1 -- 172.21.15.37:0/3648644572 learned_addr learned my addr 172.21.15.37:0/3648644572 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:26:08.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:08.418+0000 7f8783e00700 1 --2- 172.21.15.37:0/3648644572 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8784105a30 0x7f8784115fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:08.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:08.418+0000 7f8788a00700 1 -- 172.21.15.37:0/3648644572 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8784105a30 msgr2=0x7f8784115fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:08.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:08.418+0000 7f8788a00700 1 --2- 172.21.15.37:0/3648644572 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8784105a30 0x7f8784115fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:08.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:08.418+0000 7f8788a00700 1 -- 172.21.15.37:0/3648644572 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8770005040 con 0x7f8784105090 2024-03-29T08:26:08.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:08.418+0000 7f8783e00700 1 --2- 172.21.15.37:0/3648644572 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8784105a30 0x7f8784115fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:26:08.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:08.418+0000 7f8788a00700 1 --2- 172.21.15.37:0/3648644572 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8784105090 0x7f8784115a70 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f877800c8c0 tx=0x7f87780044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:26:08.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:08.418+0000 7f8780c00700 1 -- 172.21.15.37:0/3648644572 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f877800a980 con 0x7f8784105090 2024-03-29T08:26:08.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:08.418+0000 7f8780c00700 1 -- 172.21.15.37:0/3648644572 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f877800ce60 con 0x7f8784105090 2024-03-29T08:26:08.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:08.418+0000 7f8780c00700 1 -- 172.21.15.37:0/3648644572 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f87780113a0 con 0x7f8784105090 2024-03-29T08:26:08.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:08.418+0000 7f878b400700 1 -- 172.21.15.37:0/3648644572 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f878410fec0 con 0x7f8784105090 2024-03-29T08:26:08.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:08.419+0000 7f878b400700 1 -- 172.21.15.37:0/3648644572 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8784110430 con 0x7f8784105090 2024-03-29T08:26:08.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:08.420+0000 7f878b400700 1 -- 172.21.15.37:0/3648644572 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87841023a0 con 0x7f8784105090 2024-03-29T08:26:08.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:08.420+0000 7f8780c00700 1 -- 172.21.15.37:0/3648644572 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f8778012400 con 0x7f8784105090 2024-03-29T08:26:08.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:08.421+0000 7f8780c00700 1 --2- 172.21.15.37:0/3648644572 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f876c06cc20 0x7f876c06f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:08.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:08.421+0000 7f8780c00700 1 -- 172.21.15.37:0/3648644572 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f877808c1d0 con 0x7f8784105090 2024-03-29T08:26:08.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:08.422+0000 7f8783e00700 1 --2- 172.21.15.37:0/3648644572 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f876c06cc20 0x7f876c06f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:08.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:08.422+0000 7f8783e00700 1 --2- 172.21.15.37:0/3648644572 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f876c06cc20 0x7f876c06f0d0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f8784111310 tx=0x7f8770009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:26:08.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:08.427+0000 7f8780c00700 1 -- 172.21.15.37:0/3648644572 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f8778059c20 con 0x7f8784105090 2024-03-29T08:26:08.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:08.660+0000 7f878b400700 1 -- 172.21.15.37:0/3648644572 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f878410c030 con 0x7f876c06cc20 2024-03-29T08:26:08.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:08.661+0000 7f8780c00700 1 -- 172.21.15.37:0/3648644572 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f878410c030 con 0x7f876c06cc20 2024-03-29T08:26:08.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:08.664+0000 7f878b400700 1 -- 172.21.15.37:0/3648644572 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f876c06cc20 msgr2=0x7f876c06f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:08.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:08.664+0000 7f878b400700 1 --2- 172.21.15.37:0/3648644572 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f876c06cc20 0x7f876c06f0d0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f8784111310 tx=0x7f8770009e30 comp rx=0 tx=0).stop 2024-03-29T08:26:08.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:08.664+0000 7f878b400700 1 -- 172.21.15.37:0/3648644572 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8784105090 msgr2=0x7f8784115a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:08.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:08.664+0000 7f878b400700 1 --2- 172.21.15.37:0/3648644572 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8784105090 0x7f8784115a70 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f877800c8c0 tx=0x7f87780044c0 comp rx=0 tx=0).stop 2024-03-29T08:26:08.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:08.664+0000 7f878b400700 1 -- 172.21.15.37:0/3648644572 shutdown_connections 2024-03-29T08:26:08.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:08.664+0000 7f878b400700 1 --2- 172.21.15.37:0/3648644572 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f876c06cc20 0x7f876c06f0d0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:08.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:08.664+0000 7f878b400700 1 --2- 172.21.15.37:0/3648644572 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8784105090 0x7f8784115a70 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:08.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:08.664+0000 7f878b400700 1 --2- 172.21.15.37:0/3648644572 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8784105a30 0x7f8784115fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:08.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:08.664+0000 7f878b400700 1 -- 172.21.15.37:0/3648644572 >> 172.21.15.37:0/3648644572 conn(0x7f87840fac00 msgr2=0x7f87840fcaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:26:08.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:08.664+0000 7f878b400700 1 -- 172.21.15.37:0/3648644572 shutdown_connections 2024-03-29T08:26:08.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:08.664+0000 7f878b400700 1 -- 172.21.15.37:0/3648644572 wait complete. 2024-03-29T08:26:08.680 INFO:teuthology.orchestra.run.smithi037.stdout:true 2024-03-29T08:26:08.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:08.804+0000 7fca7b200700 1 -- 172.21.15.37:0/872192054 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fca7c070360 msgr2=0x7fca7c070730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:08.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:08.804+0000 7fca7b200700 1 --2- 172.21.15.37:0/872192054 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fca7c070360 0x7fca7c070730 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fca6c004970 tx=0x7fca6c009e30 comp rx=0 tx=0).stop 2024-03-29T08:26:08.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:08.804+0000 7fca7b200700 1 -- 172.21.15.37:0/872192054 shutdown_connections 2024-03-29T08:26:08.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:08.804+0000 7fca7b200700 1 --2- 172.21.15.37:0/872192054 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fca7c070c70 0x7fca7c108f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:08.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:08.804+0000 7fca7b200700 1 --2- 172.21.15.37:0/872192054 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fca7c070360 0x7fca7c070730 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:08.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:08.804+0000 7fca7b200700 1 -- 172.21.15.37:0/872192054 >> 172.21.15.37:0/872192054 conn(0x7fca7c0f8830 msgr2=0x7fca7c0fac40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:26:08.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:08.804+0000 7fca7b200700 1 -- 172.21.15.37:0/872192054 shutdown_connections 2024-03-29T08:26:08.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:08.804+0000 7fca7b200700 1 -- 172.21.15.37:0/872192054 wait complete. 2024-03-29T08:26:08.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:08.805+0000 7fca7b200700 1 Processor -- start 2024-03-29T08:26:08.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:08.805+0000 7fca7b200700 1 -- start start 2024-03-29T08:26:08.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:08.806+0000 7fca7b200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fca7c070360 0x7fca7c115af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:08.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:08.806+0000 7fca7b200700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fca7c070c70 0x7fca7c116030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:08.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:08.806+0000 7fca7b200700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca7c116680 con 0x7fca7c070360 2024-03-29T08:26:08.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:08.806+0000 7fca7b200700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca7c10fa70 con 0x7fca7c070c70 2024-03-29T08:26:08.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:08.806+0000 7fca79e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fca7c070360 0x7fca7c115af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:08.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:08.806+0000 7fca79e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fca7c070360 0x7fca7c115af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53706/0 (socket says 172.21.15.37:53706) 2024-03-29T08:26:08.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:08.806+0000 7fca79e00700 1 -- 172.21.15.37:0/1384134800 learned_addr learned my addr 172.21.15.37:0/1384134800 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:26:08.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:08.806+0000 7fca79400700 1 --2- 172.21.15.37:0/1384134800 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fca7c070c70 0x7fca7c116030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:08.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:08.806+0000 7fca79e00700 1 -- 172.21.15.37:0/1384134800 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fca7c070c70 msgr2=0x7fca7c116030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:08.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:08.806+0000 7fca79e00700 1 --2- 172.21.15.37:0/1384134800 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fca7c070c70 0x7fca7c116030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:08.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:08.806+0000 7fca79e00700 1 -- 172.21.15.37:0/1384134800 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca6c005040 con 0x7fca7c070360 2024-03-29T08:26:08.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:08.807+0000 7fca79e00700 1 --2- 172.21.15.37:0/1384134800 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fca7c070360 0x7fca7c115af0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fca6c009360 tx=0x7fca6c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:26:08.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:08.807+0000 7fca72000700 1 -- 172.21.15.37:0/1384134800 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fca6c00e040 con 0x7fca7c070360 2024-03-29T08:26:08.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:08.807+0000 7fca72000700 1 -- 172.21.15.37:0/1384134800 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fca6c004290 con 0x7fca7c070360 2024-03-29T08:26:08.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:08.807+0000 7fca72000700 1 -- 172.21.15.37:0/1384134800 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fca6c013d90 con 0x7fca7c070360 2024-03-29T08:26:08.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:08.807+0000 7fca7b200700 1 -- 172.21.15.37:0/1384134800 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca7c10fcf0 con 0x7fca7c070360 2024-03-29T08:26:08.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:08.807+0000 7fca7b200700 1 -- 172.21.15.37:0/1384134800 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca7c110230 con 0x7fca7c070360 2024-03-29T08:26:08.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:08.808+0000 7fca7b200700 1 -- 172.21.15.37:0/1384134800 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca7c071aa0 con 0x7fca7c070360 2024-03-29T08:26:08.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:08.809+0000 7fca72000700 1 -- 172.21.15.37:0/1384134800 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fca6c00ba40 con 0x7fca7c070360 2024-03-29T08:26:08.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:08.810+0000 7fca72000700 1 --2- 172.21.15.37:0/1384134800 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fca6006cc30 0x7fca6006f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:08.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:08.810+0000 7fca72000700 1 -- 172.21.15.37:0/1384134800 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fca6c08d2f0 con 0x7fca7c070360 2024-03-29T08:26:08.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:08.810+0000 7fca79400700 1 --2- 172.21.15.37:0/1384134800 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fca6006cc30 0x7fca6006f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:08.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:08.811+0000 7fca79400700 1 --2- 172.21.15.37:0/1384134800 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fca6006cc30 0x7fca6006f0e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fca7c110c30 tx=0x7fca6800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:26:08.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:08.815+0000 7fca72000700 1 -- 172.21.15.37:0/1384134800 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fca6c05ad40 con 0x7fca7c070360 2024-03-29T08:26:09.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:09.044+0000 7fca7b200700 1 -- 172.21.15.37:0/1384134800 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fca7c10bdf0 con 0x7fca6006cc30 2024-03-29T08:26:09.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:09.046+0000 7fca72000700 1 -- 172.21.15.37:0/1384134800 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fca7c10bdf0 con 0x7fca6006cc30 2024-03-29T08:26:09.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:09.048+0000 7fca7b200700 1 -- 172.21.15.37:0/1384134800 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fca6006cc30 msgr2=0x7fca6006f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:09.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:09.048+0000 7fca7b200700 1 --2- 172.21.15.37:0/1384134800 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fca6006cc30 0x7fca6006f0e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fca7c110c30 tx=0x7fca6800b040 comp rx=0 tx=0).stop 2024-03-29T08:26:09.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:09.048+0000 7fca7b200700 1 -- 172.21.15.37:0/1384134800 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fca7c070360 msgr2=0x7fca7c115af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:09.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:09.048+0000 7fca7b200700 1 --2- 172.21.15.37:0/1384134800 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fca7c070360 0x7fca7c115af0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fca6c009360 tx=0x7fca6c009e30 comp rx=0 tx=0).stop 2024-03-29T08:26:09.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:09.048+0000 7fca7b200700 1 -- 172.21.15.37:0/1384134800 shutdown_connections 2024-03-29T08:26:09.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:09.048+0000 7fca7b200700 1 --2- 172.21.15.37:0/1384134800 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fca6006cc30 0x7fca6006f0e0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:09.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:09.048+0000 7fca7b200700 1 --2- 172.21.15.37:0/1384134800 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fca7c070360 0x7fca7c115af0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:09.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:09.048+0000 7fca7b200700 1 --2- 172.21.15.37:0/1384134800 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fca7c070c70 0x7fca7c116030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:09.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:09.048+0000 7fca7b200700 1 -- 172.21.15.37:0/1384134800 >> 172.21.15.37:0/1384134800 conn(0x7fca7c0f8830 msgr2=0x7fca7c0f9de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:26:09.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:09.048+0000 7fca7b200700 1 -- 172.21.15.37:0/1384134800 shutdown_connections 2024-03-29T08:26:09.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:09.048+0000 7fca7b200700 1 -- 172.21.15.37:0/1384134800 wait complete. 2024-03-29T08:26:09.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:09.187+0000 7f739f000700 1 -- 172.21.15.37:0/719114105 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7398102e00 msgr2=0x7f73981031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:09.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:09.187+0000 7f739f000700 1 --2- 172.21.15.37:0/719114105 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7398102e00 0x7f73981031d0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f7388004940 tx=0x7f7388009e30 comp rx=0 tx=0).stop 2024-03-29T08:26:09.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:09.187+0000 7f739f000700 1 -- 172.21.15.37:0/719114105 shutdown_connections 2024-03-29T08:26:09.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:09.187+0000 7f739f000700 1 --2- 172.21.15.37:0/719114105 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f73981037a0 0x7f73980fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:09.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:09.187+0000 7f739f000700 1 --2- 172.21.15.37:0/719114105 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7398102e00 0x7f73981031d0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:09.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:09.187+0000 7f739f000700 1 -- 172.21.15.37:0/719114105 >> 172.21.15.37:0/719114105 conn(0x7f73980f89b0 msgr2=0x7f73980fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:26:09.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:09.187+0000 7f739f000700 1 -- 172.21.15.37:0/719114105 shutdown_connections 2024-03-29T08:26:09.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:09.187+0000 7f739f000700 1 -- 172.21.15.37:0/719114105 wait complete. 2024-03-29T08:26:09.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:09.188+0000 7f739f000700 1 Processor -- start 2024-03-29T08:26:09.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:09.188+0000 7f739f000700 1 -- start start 2024-03-29T08:26:09.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:09.189+0000 7f739f000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7398102e00 0x7f7398115c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:09.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:09.189+0000 7f739f000700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f73981037a0 0x7f73981161b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:09.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:09.189+0000 7f739f000700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7398116800 con 0x7f7398102e00 2024-03-29T08:26:09.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:09.189+0000 7f739f000700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f739810fbd0 con 0x7f73981037a0 2024-03-29T08:26:09.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:09.189+0000 7f7397e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7398102e00 0x7f7398115c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:09.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:09.189+0000 7f7397e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7398102e00 0x7f7398115c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53718/0 (socket says 172.21.15.37:53718) 2024-03-29T08:26:09.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:09.189+0000 7f7397e00700 1 -- 172.21.15.37:0/3204705548 learned_addr learned my addr 172.21.15.37:0/3204705548 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:26:09.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:09.189+0000 7f7397400700 1 --2- 172.21.15.37:0/3204705548 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f73981037a0 0x7f73981161b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:09.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:09.189+0000 7f7397e00700 1 -- 172.21.15.37:0/3204705548 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f73981037a0 msgr2=0x7f73981161b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:09.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:09.190+0000 7f7397e00700 1 --2- 172.21.15.37:0/3204705548 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f73981037a0 0x7f73981161b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:09.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:09.190+0000 7f7397e00700 1 -- 172.21.15.37:0/3204705548 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7388005040 con 0x7f7398102e00 2024-03-29T08:26:09.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:09.190+0000 7f7397e00700 1 --2- 172.21.15.37:0/3204705548 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7398102e00 0x7f7398115c70 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f7388009390 tx=0x7f7388009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:26:09.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:09.190+0000 7f738fe00700 1 -- 172.21.15.37:0/3204705548 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f738800e040 con 0x7f7398102e00 2024-03-29T08:26:09.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:09.190+0000 7f739f000700 1 -- 172.21.15.37:0/3204705548 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f739810fe50 con 0x7f7398102e00 2024-03-29T08:26:09.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:09.190+0000 7f738fe00700 1 -- 172.21.15.37:0/3204705548 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f7388013420 con 0x7f7398102e00 2024-03-29T08:26:09.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:09.190+0000 7f738fe00700 1 -- 172.21.15.37:0/3204705548 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f738801c630 con 0x7f7398102e00 2024-03-29T08:26:09.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:09.190+0000 7f739f000700 1 -- 172.21.15.37:0/3204705548 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7398110410 con 0x7f7398102e00 2024-03-29T08:26:09.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:09.191+0000 7f739f000700 1 -- 172.21.15.37:0/3204705548 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73980fe550 con 0x7f7398102e00 2024-03-29T08:26:09.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:09.192+0000 7f738fe00700 1 -- 172.21.15.37:0/3204705548 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f73880139e0 con 0x7f7398102e00 2024-03-29T08:26:09.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:09.193+0000 7f738fe00700 1 --2- 172.21.15.37:0/3204705548 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f737c06cc30 0x7f737c06f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:09.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:09.193+0000 7f738fe00700 1 -- 172.21.15.37:0/3204705548 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f738808d3f0 con 0x7f7398102e00 2024-03-29T08:26:09.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:09.193+0000 7f7397400700 1 --2- 172.21.15.37:0/3204705548 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f737c06cc30 0x7f737c06f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:09.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:09.194+0000 7f7397400700 1 --2- 172.21.15.37:0/3204705548 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f737c06cc30 0x7f737c06f0e0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f7398111510 tx=0x7f738400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:26:09.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:09.198+0000 7f738fe00700 1 -- 172.21.15.37:0/3204705548 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f738805ae40 con 0x7f7398102e00 2024-03-29T08:26:09.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:09.421+0000 7f739f000700 1 -- 172.21.15.37:0/3204705548 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f739804bf20 con 0x7f737c06cc30 2024-03-29T08:26:09.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:09.431+0000 7f738fe00700 1 -- 172.21.15.37:0/3204705548 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f739804bf20 con 0x7f737c06cc30 2024-03-29T08:26:09.432 INFO:teuthology.orchestra.run.smithi037.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-29T08:26:09.432 INFO:teuthology.orchestra.run.smithi037.stdout:alertmanager.smithi037 smithi037 *:9093,9094 running (16m) 8m ago 28m 14.9M - 0.25.0 c8568f914cd2 2f9f9746ed0f 2024-03-29T08:26:09.432 INFO:teuthology.orchestra.run.smithi037.stdout:crash.smithi037 smithi037 running (12m) 8m ago 29m 6647k - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 178d416a3186 2024-03-29T08:26:09.433 INFO:teuthology.orchestra.run.smithi037.stdout:crash.smithi148 smithi148 running (12m) 7m ago 27m 6656k - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 23666989d58f 2024-03-29T08:26:09.433 INFO:teuthology.orchestra.run.smithi037.stdout:grafana.smithi037 smithi037 *:3000 running (15m) 8m ago 28m 71.9M - 9.4.7 954c08fa6188 6b976fe49952 2024-03-29T08:26:09.433 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi037.iifsmh smithi037 running (8m) 8m ago 22m 10.8M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 2d97f562cc5b 2024-03-29T08:26:09.433 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi037.mmwgxe smithi037 running (8m) 8m ago 22m 10.9M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff ec37037f46e8 2024-03-29T08:26:09.433 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi148.qukpjh smithi148 running (22m) 7m ago 22m 15.3M - 17.2.7-789-g4213e17c cf22eb8683be 8e469782a273 2024-03-29T08:26:09.433 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi148.sqteih smithi148 running (8m) 7m ago 22m 10.7M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 44c15c77037a 2024-03-29T08:26:09.433 INFO:teuthology.orchestra.run.smithi037.stdout:mgr.smithi037.wjikii smithi037 *:8443,9283,8765 running (18m) 8m ago 30m 506M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff c5a1417b9417 2024-03-29T08:26:09.433 INFO:teuthology.orchestra.run.smithi037.stdout:mgr.smithi148.onhgvm smithi148 *:8443,9283,8765 running (17m) 7m ago 27m 450M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff fcfdbce062bd 2024-03-29T08:26:09.433 INFO:teuthology.orchestra.run.smithi037.stdout:mon.smithi037 smithi037 running (14m) 8m ago 30m 54.8M 2048M 18.2.2-1015-gd0e392d9 1cc2f45c17ff eea2060783af 2024-03-29T08:26:09.433 INFO:teuthology.orchestra.run.smithi037.stdout:mon.smithi148 smithi148 running (13m) 7m ago 27m 43.7M 2048M 18.2.2-1015-gd0e392d9 1cc2f45c17ff ad4b2c5748be 2024-03-29T08:26:09.433 INFO:teuthology.orchestra.run.smithi037.stdout:node-exporter.smithi037 smithi037 *:9100 running (16m) 8m ago 29m 7625k - 1.5.0 0da6a335fe13 26b8ae9cb878 2024-03-29T08:26:09.433 INFO:teuthology.orchestra.run.smithi037.stdout:node-exporter.smithi148 smithi148 *:9100 running (16m) 7m ago 27m 8166k - 1.5.0 0da6a335fe13 98414e4e64ed 2024-03-29T08:26:09.433 INFO:teuthology.orchestra.run.smithi037.stdout:osd.0 smithi037 running (12m) 8m ago 26m 104M 1619M 18.2.2-1015-gd0e392d9 1cc2f45c17ff c0c584f6245c 2024-03-29T08:26:09.434 INFO:teuthology.orchestra.run.smithi037.stdout:osd.1 smithi037 running (11m) 8m ago 26m 100M 1619M 18.2.2-1015-gd0e392d9 1cc2f45c17ff a4a2ff4cc83d 2024-03-29T08:26:09.434 INFO:teuthology.orchestra.run.smithi037.stdout:osd.2 smithi037 running (11m) 8m ago 25m 82.3M 1619M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 8b1bc71aa79e 2024-03-29T08:26:09.434 INFO:teuthology.orchestra.run.smithi037.stdout:osd.3 smithi148 running (10m) 7m ago 25m 105M 2642M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 1c656ab8ff86 2024-03-29T08:26:09.434 INFO:teuthology.orchestra.run.smithi037.stdout:osd.4 smithi148 running (9m) 7m ago 24m 97.3M 2642M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 966a9de11d22 2024-03-29T08:26:09.434 INFO:teuthology.orchestra.run.smithi037.stdout:osd.5 smithi148 running (9m) 7m ago 24m 85.8M 2642M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 474c4f9212ce 2024-03-29T08:26:09.434 INFO:teuthology.orchestra.run.smithi037.stdout:prometheus.smithi037 smithi037 *:9095 running (16m) 8m ago 28m 54.4M - 2.43.0 a07b618ecd1d 30c0223ec589 2024-03-29T08:26:09.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:09.433+0000 7f739f000700 1 -- 172.21.15.37:0/3204705548 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f737c06cc30 msgr2=0x7f737c06f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:09.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:09.433+0000 7f739f000700 1 --2- 172.21.15.37:0/3204705548 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f737c06cc30 0x7f737c06f0e0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f7398111510 tx=0x7f738400b040 comp rx=0 tx=0).stop 2024-03-29T08:26:09.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:09.433+0000 7f739f000700 1 -- 172.21.15.37:0/3204705548 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7398102e00 msgr2=0x7f7398115c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:09.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:09.433+0000 7f739f000700 1 --2- 172.21.15.37:0/3204705548 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7398102e00 0x7f7398115c70 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f7388009390 tx=0x7f7388009e30 comp rx=0 tx=0).stop 2024-03-29T08:26:09.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:09.433+0000 7f739f000700 1 -- 172.21.15.37:0/3204705548 shutdown_connections 2024-03-29T08:26:09.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:09.433+0000 7f739f000700 1 --2- 172.21.15.37:0/3204705548 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f737c06cc30 0x7f737c06f0e0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:09.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:09.433+0000 7f739f000700 1 --2- 172.21.15.37:0/3204705548 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7398102e00 0x7f7398115c70 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:09.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:09.433+0000 7f739f000700 1 --2- 172.21.15.37:0/3204705548 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f73981037a0 0x7f73981161b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:09.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:09.433+0000 7f739f000700 1 -- 172.21.15.37:0/3204705548 >> 172.21.15.37:0/3204705548 conn(0x7f73980f89b0 msgr2=0x7f73980f9960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:26:09.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:09.433+0000 7f739f000700 1 -- 172.21.15.37:0/3204705548 shutdown_connections 2024-03-29T08:26:09.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:09.433+0000 7f739f000700 1 -- 172.21.15.37:0/3204705548 wait complete. 2024-03-29T08:26:09.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:26:09 smithi148 ceph-mon[106692]: pgmap v457: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:26:09.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:26:09 smithi148 ceph-mon[106692]: from='client.34868 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:26:09.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:09.569+0000 7f65dda00700 1 -- 172.21.15.37:0/2158718036 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f65d81037a0 msgr2=0x7f65d80fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:09.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:09.569+0000 7f65dda00700 1 --2- 172.21.15.37:0/2158718036 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f65d81037a0 0x7f65d80fce10 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f65c4004940 tx=0x7f65c4009e30 comp rx=0 tx=0).stop 2024-03-29T08:26:09.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:09.569+0000 7f65dda00700 1 -- 172.21.15.37:0/2158718036 shutdown_connections 2024-03-29T08:26:09.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:09.569+0000 7f65dda00700 1 --2- 172.21.15.37:0/2158718036 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f65d81037a0 0x7f65d80fce10 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:09.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:09.569+0000 7f65dda00700 1 --2- 172.21.15.37:0/2158718036 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f65d8102e00 0x7f65d81031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:09.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:09.569+0000 7f65dda00700 1 -- 172.21.15.37:0/2158718036 >> 172.21.15.37:0/2158718036 conn(0x7f65d80f89b0 msgr2=0x7f65d80fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:26:09.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:09.569+0000 7f65dda00700 1 -- 172.21.15.37:0/2158718036 shutdown_connections 2024-03-29T08:26:09.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:09.570+0000 7f65dda00700 1 -- 172.21.15.37:0/2158718036 wait complete. 2024-03-29T08:26:09.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:09.570+0000 7f65dda00700 1 Processor -- start 2024-03-29T08:26:09.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:09.571+0000 7f65dda00700 1 -- start start 2024-03-29T08:26:09.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:09.571+0000 7f65dda00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f65d8102e00 0x7f65d8115d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:09.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:09.571+0000 7f65dda00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f65d81037a0 0x7f65d8116240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:09.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:09.571+0000 7f65dda00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65d8116890 con 0x7f65d81037a0 2024-03-29T08:26:09.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:09.571+0000 7f65dda00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65d810fc60 con 0x7f65d8102e00 2024-03-29T08:26:09.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:09.571+0000 7f65d6000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f65d81037a0 0x7f65d8116240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:09.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:09.571+0000 7f65d6000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f65d81037a0 0x7f65d8116240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53728/0 (socket says 172.21.15.37:53728) 2024-03-29T08:26:09.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:09.571+0000 7f65d6000700 1 -- 172.21.15.37:0/3406719543 learned_addr learned my addr 172.21.15.37:0/3406719543 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:26:09.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:09.571+0000 7f65d6a00700 1 --2- 172.21.15.37:0/3406719543 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f65d8102e00 0x7f65d8115d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:09.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:09.572+0000 7f65d6000700 1 -- 172.21.15.37:0/3406719543 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f65d8102e00 msgr2=0x7f65d8115d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:09.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:09.572+0000 7f65d6000700 1 --2- 172.21.15.37:0/3406719543 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f65d8102e00 0x7f65d8115d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:09.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:09.572+0000 7f65d6000700 1 -- 172.21.15.37:0/3406719543 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65c4005040 con 0x7f65d81037a0 2024-03-29T08:26:09.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:09.572+0000 7f65d6000700 1 --2- 172.21.15.37:0/3406719543 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f65d81037a0 0x7f65d8116240 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f65c4009380 tx=0x7f65c4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:26:09.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:09.572+0000 7f65cea00700 1 -- 172.21.15.37:0/3406719543 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f65c400e040 con 0x7f65d81037a0 2024-03-29T08:26:09.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:09.572+0000 7f65cea00700 1 -- 172.21.15.37:0/3406719543 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f65c4013420 con 0x7f65d81037a0 2024-03-29T08:26:09.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:09.572+0000 7f65cea00700 1 -- 172.21.15.37:0/3406719543 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f65c401c630 con 0x7f65d81037a0 2024-03-29T08:26:09.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:09.572+0000 7f65dda00700 1 -- 172.21.15.37:0/3406719543 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65d810fee0 con 0x7f65d81037a0 2024-03-29T08:26:09.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:09.572+0000 7f65dda00700 1 -- 172.21.15.37:0/3406719543 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65d81104a0 con 0x7f65d81037a0 2024-03-29T08:26:09.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:09.573+0000 7f65dda00700 1 -- 172.21.15.37:0/3406719543 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65d80fe550 con 0x7f65d81037a0 2024-03-29T08:26:09.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:09.574+0000 7f65cea00700 1 -- 172.21.15.37:0/3406719543 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f65c40139b0 con 0x7f65d81037a0 2024-03-29T08:26:09.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:09.575+0000 7f65cea00700 1 --2- 172.21.15.37:0/3406719543 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f65bc06cc20 0x7f65bc06f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:09.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:09.575+0000 7f65cea00700 1 -- 172.21.15.37:0/3406719543 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f65c408d220 con 0x7f65d81037a0 2024-03-29T08:26:09.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:09.576+0000 7f65d6a00700 1 --2- 172.21.15.37:0/3406719543 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f65bc06cc20 0x7f65bc06f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:09.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:09.576+0000 7f65d6a00700 1 --2- 172.21.15.37:0/3406719543 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f65bc06cc20 0x7f65bc06f0d0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f65c8004f40 tx=0x7f65c800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:26:09.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:09.580+0000 7f65cea00700 1 -- 172.21.15.37:0/3406719543 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f65c405ac70 con 0x7f65d81037a0 2024-03-29T08:26:09.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:26:09 smithi037 ceph-mon[142325]: pgmap v457: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:26:09.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:26:09 smithi037 ceph-mon[142325]: from='client.34868 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:26:09.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:09.886+0000 7f65dda00700 1 -- 172.21.15.37:0/3406719543 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f65d804bed0 con 0x7f65d81037a0 2024-03-29T08:26:09.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:09.887+0000 7f65cea00700 1 -- 172.21.15.37:0/3406719543 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+847 (secure 0 0 0) 0x7f65c405a800 con 0x7f65d81037a0 2024-03-29T08:26:09.888 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-29T08:26:09.888 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": { 2024-03-29T08:26:09.888 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 2 2024-03-29T08:26:09.889 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:26:09.889 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": { 2024-03-29T08:26:09.889 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 2 2024-03-29T08:26:09.889 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:26:09.889 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": { 2024-03-29T08:26:09.889 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 6 2024-03-29T08:26:09.889 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:26:09.889 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": { 2024-03-29T08:26:09.889 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 17.2.7-789-g4213e17c (4213e17c4de67016ed14f55d22503f4923b3a901) quincy (stable)": 1, 2024-03-29T08:26:09.889 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 3 2024-03-29T08:26:09.889 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:26:09.889 INFO:teuthology.orchestra.run.smithi037.stdout: "overall": { 2024-03-29T08:26:09.889 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 17.2.7-789-g4213e17c (4213e17c4de67016ed14f55d22503f4923b3a901) quincy (stable)": 1, 2024-03-29T08:26:09.889 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 13 2024-03-29T08:26:09.889 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-29T08:26:09.890 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-29T08:26:09.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:09.889+0000 7f65dda00700 1 -- 172.21.15.37:0/3406719543 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f65bc06cc20 msgr2=0x7f65bc06f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:09.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:09.889+0000 7f65dda00700 1 --2- 172.21.15.37:0/3406719543 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f65bc06cc20 0x7f65bc06f0d0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f65c8004f40 tx=0x7f65c800b040 comp rx=0 tx=0).stop 2024-03-29T08:26:09.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:09.889+0000 7f65dda00700 1 -- 172.21.15.37:0/3406719543 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f65d81037a0 msgr2=0x7f65d8116240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:09.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:09.889+0000 7f65dda00700 1 --2- 172.21.15.37:0/3406719543 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f65d81037a0 0x7f65d8116240 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f65c4009380 tx=0x7f65c4009e30 comp rx=0 tx=0).stop 2024-03-29T08:26:09.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:09.889+0000 7f65dda00700 1 -- 172.21.15.37:0/3406719543 shutdown_connections 2024-03-29T08:26:09.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:09.889+0000 7f65dda00700 1 --2- 172.21.15.37:0/3406719543 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f65bc06cc20 0x7f65bc06f0d0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:09.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:09.889+0000 7f65dda00700 1 --2- 172.21.15.37:0/3406719543 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f65d8102e00 0x7f65d8115d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:09.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:09.889+0000 7f65dda00700 1 --2- 172.21.15.37:0/3406719543 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f65d81037a0 0x7f65d8116240 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:09.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:09.889+0000 7f65dda00700 1 -- 172.21.15.37:0/3406719543 >> 172.21.15.37:0/3406719543 conn(0x7f65d80f89b0 msgr2=0x7f65d80f9ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:26:09.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:09.890+0000 7f65dda00700 1 -- 172.21.15.37:0/3406719543 shutdown_connections 2024-03-29T08:26:09.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:09.890+0000 7f65dda00700 1 -- 172.21.15.37:0/3406719543 wait complete. 2024-03-29T08:26:10.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:10.027+0000 7ffa43800700 1 -- 172.21.15.37:0/1733544658 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffa3c0708a0 msgr2=0x7ffa3c070d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:10.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:10.027+0000 7ffa43800700 1 --2- 172.21.15.37:0/1733544658 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffa3c0708a0 0x7ffa3c070d10 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7ffa28004970 tx=0x7ffa28009e30 comp rx=0 tx=0).stop 2024-03-29T08:26:10.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:10.027+0000 7ffa43800700 1 -- 172.21.15.37:0/1733544658 shutdown_connections 2024-03-29T08:26:10.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:10.027+0000 7ffa43800700 1 --2- 172.21.15.37:0/1733544658 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffa3c0708a0 0x7ffa3c070d10 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:10.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:10.027+0000 7ffa43800700 1 --2- 172.21.15.37:0/1733544658 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffa3c071f90 0x7ffa3c070360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:10.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:10.027+0000 7ffa43800700 1 -- 172.21.15.37:0/1733544658 >> 172.21.15.37:0/1733544658 conn(0x7ffa3c0f89b0 msgr2=0x7ffa3c0fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:26:10.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:10.027+0000 7ffa43800700 1 -- 172.21.15.37:0/1733544658 shutdown_connections 2024-03-29T08:26:10.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:10.027+0000 7ffa43800700 1 -- 172.21.15.37:0/1733544658 wait complete. 2024-03-29T08:26:10.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:10.028+0000 7ffa43800700 1 Processor -- start 2024-03-29T08:26:10.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:10.029+0000 7ffa43800700 1 -- start start 2024-03-29T08:26:10.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:10.029+0000 7ffa43800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffa3c0708a0 0x7ffa3c11a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:10.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:10.029+0000 7ffa43800700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffa3c071f90 0x7ffa3c11a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:10.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:10.029+0000 7ffa43800700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa3c11ab90 con 0x7ffa3c0708a0 2024-03-29T08:26:10.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:10.029+0000 7ffa43800700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa3c114010 con 0x7ffa3c071f90 2024-03-29T08:26:10.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:10.029+0000 7ffa40e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffa3c0708a0 0x7ffa3c11a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:10.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:10.029+0000 7ffa40e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffa3c0708a0 0x7ffa3c11a000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41232/0 (socket says 172.21.15.37:41232) 2024-03-29T08:26:10.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:10.029+0000 7ffa40e00700 1 -- 172.21.15.37:0/674174048 learned_addr learned my addr 172.21.15.37:0/674174048 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:26:10.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:10.030+0000 7ffa3be00700 1 --2- 172.21.15.37:0/674174048 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffa3c071f90 0x7ffa3c11a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:10.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:10.030+0000 7ffa40e00700 1 -- 172.21.15.37:0/674174048 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffa3c071f90 msgr2=0x7ffa3c11a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:10.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:10.030+0000 7ffa40e00700 1 --2- 172.21.15.37:0/674174048 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffa3c071f90 0x7ffa3c11a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:10.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:10.030+0000 7ffa40e00700 1 -- 172.21.15.37:0/674174048 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa28005040 con 0x7ffa3c0708a0 2024-03-29T08:26:10.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:10.030+0000 7ffa40e00700 1 --2- 172.21.15.37:0/674174048 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffa3c0708a0 0x7ffa3c11a000 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7ffa30002a50 tx=0x7ffa300046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:26:10.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:10.030+0000 7ffa38c00700 1 -- 172.21.15.37:0/674174048 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffa30007cc0 con 0x7ffa3c0708a0 2024-03-29T08:26:10.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:10.030+0000 7ffa38c00700 1 -- 172.21.15.37:0/674174048 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ffa3000f040 con 0x7ffa3c0708a0 2024-03-29T08:26:10.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:10.030+0000 7ffa43800700 1 -- 172.21.15.37:0/674174048 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa3c1142f0 con 0x7ffa3c0708a0 2024-03-29T08:26:10.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:10.030+0000 7ffa38c00700 1 -- 172.21.15.37:0/674174048 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffa3000d9c0 con 0x7ffa3c0708a0 2024-03-29T08:26:10.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:10.031+0000 7ffa43800700 1 -- 172.21.15.37:0/674174048 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa3c114800 con 0x7ffa3c0708a0 2024-03-29T08:26:10.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:10.032+0000 7ffa43800700 1 -- 172.21.15.37:0/674174048 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa3c071aa0 con 0x7ffa3c0708a0 2024-03-29T08:26:10.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:10.033+0000 7ffa38c00700 1 -- 172.21.15.37:0/674174048 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7ffa3000db20 con 0x7ffa3c0708a0 2024-03-29T08:26:10.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:10.033+0000 7ffa38c00700 1 --2- 172.21.15.37:0/674174048 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7ffa2406cc30 0x7ffa2406f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:10.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:10.033+0000 7ffa38c00700 1 -- 172.21.15.37:0/674174048 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7ffa3008bca0 con 0x7ffa3c0708a0 2024-03-29T08:26:10.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:10.034+0000 7ffa3be00700 1 --2- 172.21.15.37:0/674174048 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7ffa2406cc30 0x7ffa2406f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:10.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:10.034+0000 7ffa3be00700 1 --2- 172.21.15.37:0/674174048 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7ffa2406cc30 0x7ffa2406f0e0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7ffa2800b370 tx=0x7ffa28009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:26:10.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:10.038+0000 7ffa38c00700 1 -- 172.21.15.37:0/674174048 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7ffa30055ce0 con 0x7ffa3c0708a0 2024-03-29T08:26:10.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:10.298+0000 7ffa43800700 1 -- 172.21.15.37:0/674174048 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7ffa3c1103e0 con 0x7ffa3c0708a0 2024-03-29T08:26:10.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:10.299+0000 7ffa38c00700 1 -- 172.21.15.37:0/674174048 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 28 v28) v1 ==== 76+0+1841 (secure 0 0 0) 0x7ffa30059300 con 0x7ffa3c0708a0 2024-03-29T08:26:10.300 INFO:teuthology.orchestra.run.smithi037.stdout:e28 2024-03-29T08:26:10.300 INFO:teuthology.orchestra.run.smithi037.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-29T08:26:10.300 INFO:teuthology.orchestra.run.smithi037.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T08:26:10.300 INFO:teuthology.orchestra.run.smithi037.stdout:legacy client fscid: 1 2024-03-29T08:26:10.300 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:26:10.300 INFO:teuthology.orchestra.run.smithi037.stdout:Filesystem 'cephfs' (1) 2024-03-29T08:26:10.300 INFO:teuthology.orchestra.run.smithi037.stdout:fs_name cephfs 2024-03-29T08:26:10.300 INFO:teuthology.orchestra.run.smithi037.stdout:epoch 21 2024-03-29T08:26:10.300 INFO:teuthology.orchestra.run.smithi037.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-03-29T08:26:10.301 INFO:teuthology.orchestra.run.smithi037.stdout:created 2024-03-29T08:03:29.241269+0000 2024-03-29T08:26:10.301 INFO:teuthology.orchestra.run.smithi037.stdout:modified 2024-03-29T08:17:16.237238+0000 2024-03-29T08:26:10.301 INFO:teuthology.orchestra.run.smithi037.stdout:tableserver 0 2024-03-29T08:26:10.301 INFO:teuthology.orchestra.run.smithi037.stdout:root 0 2024-03-29T08:26:10.301 INFO:teuthology.orchestra.run.smithi037.stdout:session_timeout 60 2024-03-29T08:26:10.301 INFO:teuthology.orchestra.run.smithi037.stdout:session_autoclose 300 2024-03-29T08:26:10.301 INFO:teuthology.orchestra.run.smithi037.stdout:max_file_size 1099511627776 2024-03-29T08:26:10.301 INFO:teuthology.orchestra.run.smithi037.stdout:max_xattr_size 65536 2024-03-29T08:26:10.301 INFO:teuthology.orchestra.run.smithi037.stdout:required_client_features {} 2024-03-29T08:26:10.301 INFO:teuthology.orchestra.run.smithi037.stdout:last_failure 0 2024-03-29T08:26:10.301 INFO:teuthology.orchestra.run.smithi037.stdout:last_failure_osd_epoch 82 2024-03-29T08:26:10.302 INFO:teuthology.orchestra.run.smithi037.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T08:26:10.302 INFO:teuthology.orchestra.run.smithi037.stdout:max_mds 2 2024-03-29T08:26:10.302 INFO:teuthology.orchestra.run.smithi037.stdout:in 0,1 2024-03-29T08:26:10.302 INFO:teuthology.orchestra.run.smithi037.stdout:up {} 2024-03-29T08:26:10.302 INFO:teuthology.orchestra.run.smithi037.stdout:failed 0,1 2024-03-29T08:26:10.302 INFO:teuthology.orchestra.run.smithi037.stdout:damaged 2024-03-29T08:26:10.302 INFO:teuthology.orchestra.run.smithi037.stdout:stopped 2024-03-29T08:26:10.302 INFO:teuthology.orchestra.run.smithi037.stdout:data_pools [3] 2024-03-29T08:26:10.302 INFO:teuthology.orchestra.run.smithi037.stdout:metadata_pool 2 2024-03-29T08:26:10.302 INFO:teuthology.orchestra.run.smithi037.stdout:inline_data disabled 2024-03-29T08:26:10.302 INFO:teuthology.orchestra.run.smithi037.stdout:balancer 2024-03-29T08:26:10.302 INFO:teuthology.orchestra.run.smithi037.stdout:bal_rank_mask -1 2024-03-29T08:26:10.302 INFO:teuthology.orchestra.run.smithi037.stdout:standby_count_wanted 1 2024-03-29T08:26:10.303 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:26:10.303 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:26:10.303 INFO:teuthology.orchestra.run.smithi037.stdout:Standby daemons: 2024-03-29T08:26:10.303 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:26:10.303 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi148.qukpjh{-1:24319} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.148:6826/2641327912,v1:172.21.15.148:6827/2641327912] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:26:10.303 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi037.mmwgxe{-1:34440} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.37:6828/1248142005,v1:172.21.15.37:6829/1248142005] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:26:10.303 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi037.iifsmh{-1:34444} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.37:6826/2060882959,v1:172.21.15.37:6827/2060882959] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:26:10.303 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi148.sqteih{-1:44395} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.148:6824/4078652242,v1:172.21.15.148:6825/4078652242] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:26:10.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:10.301+0000 7ffa43800700 1 -- 172.21.15.37:0/674174048 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7ffa2406cc30 msgr2=0x7ffa2406f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:10.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:10.301+0000 7ffa43800700 1 --2- 172.21.15.37:0/674174048 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7ffa2406cc30 0x7ffa2406f0e0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7ffa2800b370 tx=0x7ffa28009e30 comp rx=0 tx=0).stop 2024-03-29T08:26:10.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:10.301+0000 7ffa43800700 1 -- 172.21.15.37:0/674174048 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffa3c0708a0 msgr2=0x7ffa3c11a000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:10.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:10.301+0000 7ffa43800700 1 --2- 172.21.15.37:0/674174048 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffa3c0708a0 0x7ffa3c11a000 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7ffa30002a50 tx=0x7ffa300046b0 comp rx=0 tx=0).stop 2024-03-29T08:26:10.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:10.301+0000 7ffa43800700 1 -- 172.21.15.37:0/674174048 shutdown_connections 2024-03-29T08:26:10.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:10.301+0000 7ffa43800700 1 --2- 172.21.15.37:0/674174048 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7ffa2406cc30 0x7ffa2406f0e0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:10.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:10.301+0000 7ffa43800700 1 --2- 172.21.15.37:0/674174048 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffa3c0708a0 0x7ffa3c11a000 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:10.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:10.301+0000 7ffa43800700 1 --2- 172.21.15.37:0/674174048 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffa3c071f90 0x7ffa3c11a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:10.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:10.301+0000 7ffa43800700 1 -- 172.21.15.37:0/674174048 >> 172.21.15.37:0/674174048 conn(0x7ffa3c0f89b0 msgr2=0x7ffa3c0f9b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:26:10.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:10.301+0000 7ffa43800700 1 -- 172.21.15.37:0/674174048 shutdown_connections 2024-03-29T08:26:10.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:10.301+0000 7ffa43800700 1 -- 172.21.15.37:0/674174048 wait complete. 2024-03-29T08:26:10.305 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 28 2024-03-29T08:26:10.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:10.438+0000 7f2037600700 1 -- 172.21.15.37:0/1003319542 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f20301037a0 msgr2=0x7f20300fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:10.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:10.438+0000 7f2037600700 1 --2- 172.21.15.37:0/1003319542 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f20301037a0 0x7f20300fce10 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f201c004940 tx=0x7f201c009e30 comp rx=0 tx=0).stop 2024-03-29T08:26:10.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:10.438+0000 7f2037600700 1 -- 172.21.15.37:0/1003319542 shutdown_connections 2024-03-29T08:26:10.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:10.438+0000 7f2037600700 1 --2- 172.21.15.37:0/1003319542 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f20301037a0 0x7f20300fce10 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:10.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:10.438+0000 7f2037600700 1 --2- 172.21.15.37:0/1003319542 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2030102e00 0x7f20301031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:10.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:10.438+0000 7f2037600700 1 -- 172.21.15.37:0/1003319542 >> 172.21.15.37:0/1003319542 conn(0x7f20300f89b0 msgr2=0x7f20300fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:26:10.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:10.438+0000 7f2037600700 1 -- 172.21.15.37:0/1003319542 shutdown_connections 2024-03-29T08:26:10.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:10.438+0000 7f2037600700 1 -- 172.21.15.37:0/1003319542 wait complete. 2024-03-29T08:26:10.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:10.439+0000 7f2037600700 1 Processor -- start 2024-03-29T08:26:10.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:10.439+0000 7f2037600700 1 -- start start 2024-03-29T08:26:10.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:10.439+0000 7f2037600700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2030102e00 0x7f2030113a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:10.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:10.439+0000 7f2037600700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f20301037a0 0x7f2030113fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:10.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:10.439+0000 7f2037600700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2030114600 con 0x7f20301037a0 2024-03-29T08:26:10.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:10.439+0000 7f2037600700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f203010d9f0 con 0x7f2030102e00 2024-03-29T08:26:10.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:10.440+0000 7f202fe00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f20301037a0 0x7f2030113fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:10.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:10.440+0000 7f202fe00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f20301037a0 0x7f2030113fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41262/0 (socket says 172.21.15.37:41262) 2024-03-29T08:26:10.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:10.440+0000 7f202fe00700 1 -- 172.21.15.37:0/2617801172 learned_addr learned my addr 172.21.15.37:0/2617801172 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:26:10.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:10.440+0000 7f2034c00700 1 --2- 172.21.15.37:0/2617801172 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2030102e00 0x7f2030113a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:10.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:10.440+0000 7f202fe00700 1 -- 172.21.15.37:0/2617801172 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2030102e00 msgr2=0x7f2030113a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:10.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:10.440+0000 7f202fe00700 1 --2- 172.21.15.37:0/2617801172 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2030102e00 0x7f2030113a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:10.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:10.440+0000 7f202fe00700 1 -- 172.21.15.37:0/2617801172 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f201c005040 con 0x7f20301037a0 2024-03-29T08:26:10.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:10.440+0000 7f2034c00700 1 --2- 172.21.15.37:0/2617801172 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2030102e00 0x7f2030113a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:26:10.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:10.440+0000 7f202fe00700 1 --2- 172.21.15.37:0/2617801172 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f20301037a0 0x7f2030113fb0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f201c009fd0 tx=0x7f201c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:26:10.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:10.441+0000 7f202cc00700 1 -- 172.21.15.37:0/2617801172 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f201c00e030 con 0x7f20301037a0 2024-03-29T08:26:10.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:10.441+0000 7f202cc00700 1 -- 172.21.15.37:0/2617801172 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f201c013e00 con 0x7f20301037a0 2024-03-29T08:26:10.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:10.441+0000 7f2037600700 1 -- 172.21.15.37:0/2617801172 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f203010dc70 con 0x7f20301037a0 2024-03-29T08:26:10.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:10.441+0000 7f202cc00700 1 -- 172.21.15.37:0/2617801172 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f201c01ccd0 con 0x7f20301037a0 2024-03-29T08:26:10.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:10.441+0000 7f2037600700 1 -- 172.21.15.37:0/2617801172 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f203010e230 con 0x7f20301037a0 2024-03-29T08:26:10.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:10.442+0000 7f2037600700 1 -- 172.21.15.37:0/2617801172 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20300fe550 con 0x7f20301037a0 2024-03-29T08:26:10.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:10.443+0000 7f202cc00700 1 -- 172.21.15.37:0/2617801172 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f201c01c500 con 0x7f20301037a0 2024-03-29T08:26:10.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:10.445+0000 7f202cc00700 1 --2- 172.21.15.37:0/2617801172 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f201806cc30 0x7f201806f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:10.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:10.445+0000 7f202cc00700 1 -- 172.21.15.37:0/2617801172 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f201c08d9a0 con 0x7f20301037a0 2024-03-29T08:26:10.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:10.445+0000 7f2034c00700 1 --2- 172.21.15.37:0/2617801172 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f201806cc30 0x7f201806f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:10.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:10.446+0000 7f2034c00700 1 --2- 172.21.15.37:0/2617801172 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f201806cc30 0x7f201806f0e0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f2024009fd0 tx=0x7f202400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:26:10.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:10.449+0000 7f202cc00700 1 -- 172.21.15.37:0/2617801172 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f201c05b3f0 con 0x7f20301037a0 2024-03-29T08:26:10.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:26:10 smithi148 ceph-mon[106692]: from='client.34872 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:26:10.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:26:10 smithi148 ceph-mon[106692]: from='client.34876 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:26:10.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:26:10 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:26:10.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:26:10 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/3406719543' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:26:10.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:26:10 smithi037 ceph-mon[142325]: from='client.34872 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:26:10.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:26:10 smithi037 ceph-mon[142325]: from='client.34876 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:26:10.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:26:10 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:26:10.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:26:10 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/3406719543' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:26:10.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:10.680+0000 7f2037600700 1 -- 172.21.15.37:0/2617801172 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f2030109da0 con 0x7f201806cc30 2024-03-29T08:26:10.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:10.681+0000 7f202cc00700 1 -- 172.21.15.37:0/2617801172 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f2030109da0 con 0x7f201806cc30 2024-03-29T08:26:10.682 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-29T08:26:10.682 INFO:teuthology.orchestra.run.smithi037.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59", 2024-03-29T08:26:10.682 INFO:teuthology.orchestra.run.smithi037.stdout: "in_progress": true, 2024-03-29T08:26:10.683 INFO:teuthology.orchestra.run.smithi037.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-29T08:26:10.683 INFO:teuthology.orchestra.run.smithi037.stdout: "services_complete": [ 2024-03-29T08:26:10.683 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr", 2024-03-29T08:26:10.683 INFO:teuthology.orchestra.run.smithi037.stdout: "mon", 2024-03-29T08:26:10.683 INFO:teuthology.orchestra.run.smithi037.stdout: "crash", 2024-03-29T08:26:10.683 INFO:teuthology.orchestra.run.smithi037.stdout: "osd" 2024-03-29T08:26:10.683 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-29T08:26:10.683 INFO:teuthology.orchestra.run.smithi037.stdout: "progress": "15/21 daemons upgraded", 2024-03-29T08:26:10.684 INFO:teuthology.orchestra.run.smithi037.stdout: "message": "Currently upgrading mds daemons", 2024-03-29T08:26:10.684 INFO:teuthology.orchestra.run.smithi037.stdout: "is_paused": false 2024-03-29T08:26:10.684 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-29T08:26:10.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:10.683+0000 7f2037600700 1 -- 172.21.15.37:0/2617801172 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f201806cc30 msgr2=0x7f201806f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:10.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:10.683+0000 7f2037600700 1 --2- 172.21.15.37:0/2617801172 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f201806cc30 0x7f201806f0e0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f2024009fd0 tx=0x7f202400b040 comp rx=0 tx=0).stop 2024-03-29T08:26:10.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:10.683+0000 7f2037600700 1 -- 172.21.15.37:0/2617801172 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f20301037a0 msgr2=0x7f2030113fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:10.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:10.683+0000 7f2037600700 1 --2- 172.21.15.37:0/2617801172 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f20301037a0 0x7f2030113fb0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f201c009fd0 tx=0x7f201c009e30 comp rx=0 tx=0).stop 2024-03-29T08:26:10.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:10.684+0000 7f2037600700 1 -- 172.21.15.37:0/2617801172 shutdown_connections 2024-03-29T08:26:10.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:10.684+0000 7f2037600700 1 --2- 172.21.15.37:0/2617801172 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f201806cc30 0x7f201806f0e0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:10.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:10.684+0000 7f2037600700 1 --2- 172.21.15.37:0/2617801172 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2030102e00 0x7f2030113a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:10.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:10.684+0000 7f2037600700 1 --2- 172.21.15.37:0/2617801172 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f20301037a0 0x7f2030113fb0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:10.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:10.684+0000 7f2037600700 1 -- 172.21.15.37:0/2617801172 >> 172.21.15.37:0/2617801172 conn(0x7f20300f89b0 msgr2=0x7f20300f9a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:26:10.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:10.684+0000 7f2037600700 1 -- 172.21.15.37:0/2617801172 shutdown_connections 2024-03-29T08:26:10.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:10.684+0000 7f2037600700 1 -- 172.21.15.37:0/2617801172 wait complete. 2024-03-29T08:26:10.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:10.820+0000 7f4637c00700 1 -- 172.21.15.37:0/482274957 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f46300fe830 msgr2=0x7f4630106f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:10.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:10.820+0000 7f4637c00700 1 --2- 172.21.15.37:0/482274957 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f46300fe830 0x7f4630106f00 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f461c004940 tx=0x7f461c009e30 comp rx=0 tx=0).stop 2024-03-29T08:26:10.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:10.821+0000 7f4637c00700 1 -- 172.21.15.37:0/482274957 shutdown_connections 2024-03-29T08:26:10.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:10.821+0000 7f4637c00700 1 --2- 172.21.15.37:0/482274957 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f46300fe830 0x7f4630106f00 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:10.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:10.821+0000 7f4637c00700 1 --2- 172.21.15.37:0/482274957 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f46300fde90 0x7f46300fe260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:10.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:10.821+0000 7f4637c00700 1 -- 172.21.15.37:0/482274957 >> 172.21.15.37:0/482274957 conn(0x7f4630072bc0 msgr2=0x7f4630072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:26:10.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:10.821+0000 7f4637c00700 1 -- 172.21.15.37:0/482274957 shutdown_connections 2024-03-29T08:26:10.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:10.821+0000 7f4637c00700 1 -- 172.21.15.37:0/482274957 wait complete. 2024-03-29T08:26:10.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:10.822+0000 7f4637c00700 1 Processor -- start 2024-03-29T08:26:10.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:10.822+0000 7f4637c00700 1 -- start start 2024-03-29T08:26:10.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:10.822+0000 7f4637c00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f46300fde90 0x7f46300fbd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:10.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:10.822+0000 7f4637c00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f46300fe830 0x7f46300fc290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:10.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:10.822+0000 7f4637c00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46300fc820 con 0x7f46300fde90 2024-03-29T08:26:10.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:10.822+0000 7f4637c00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46300fc960 con 0x7f46300fe830 2024-03-29T08:26:10.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:10.822+0000 7f4635200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f46300fde90 0x7f46300fbd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:10.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:10.822+0000 7f4635200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f46300fde90 0x7f46300fbd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41282/0 (socket says 172.21.15.37:41282) 2024-03-29T08:26:10.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:10.822+0000 7f4635200700 1 -- 172.21.15.37:0/1486286930 learned_addr learned my addr 172.21.15.37:0/1486286930 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:26:10.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:10.823+0000 7f462fe00700 1 --2- 172.21.15.37:0/1486286930 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f46300fe830 0x7f46300fc290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:10.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:10.823+0000 7f4635200700 1 -- 172.21.15.37:0/1486286930 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f46300fe830 msgr2=0x7f46300fc290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:10.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:10.823+0000 7f4635200700 1 --2- 172.21.15.37:0/1486286930 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f46300fe830 0x7f46300fc290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:10.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:10.823+0000 7f4635200700 1 -- 172.21.15.37:0/1486286930 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f461c005040 con 0x7f46300fde90 2024-03-29T08:26:10.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:10.823+0000 7f4635200700 1 --2- 172.21.15.37:0/1486286930 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f46300fde90 0x7f46300fbd50 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f4624002a50 tx=0x7f46240046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:26:10.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:10.823+0000 7f462cc00700 1 -- 172.21.15.37:0/1486286930 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4624007cc0 con 0x7f46300fde90 2024-03-29T08:26:10.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:10.823+0000 7f462cc00700 1 -- 172.21.15.37:0/1486286930 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f462400f040 con 0x7f46300fde90 2024-03-29T08:26:10.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:10.823+0000 7f4637c00700 1 -- 172.21.15.37:0/1486286930 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4630119de0 con 0x7f46300fde90 2024-03-29T08:26:10.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:10.823+0000 7f462cc00700 1 -- 172.21.15.37:0/1486286930 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4624009540 con 0x7f46300fde90 2024-03-29T08:26:10.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:10.823+0000 7f4637c00700 1 -- 172.21.15.37:0/1486286930 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f463011a1e0 con 0x7f46300fde90 2024-03-29T08:26:10.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:10.824+0000 7f4637c00700 1 -- 172.21.15.37:0/1486286930 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4630065e30 con 0x7f46300fde90 2024-03-29T08:26:10.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:10.826+0000 7f462cc00700 1 -- 172.21.15.37:0/1486286930 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f46240096a0 con 0x7f46300fde90 2024-03-29T08:26:10.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:10.826+0000 7f462cc00700 1 --2- 172.21.15.37:0/1486286930 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f461806cc30 0x7f461806f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:10.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:10.827+0000 7f462cc00700 1 -- 172.21.15.37:0/1486286930 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f462408be00 con 0x7f46300fde90 2024-03-29T08:26:10.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:10.827+0000 7f462fe00700 1 --2- 172.21.15.37:0/1486286930 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f461806cc30 0x7f461806f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:10.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:10.827+0000 7f462fe00700 1 --2- 172.21.15.37:0/1486286930 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f461806cc30 0x7f461806f0e0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f46300fd690 tx=0x7f461c009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:26:10.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:10.831+0000 7f462cc00700 1 -- 172.21.15.37:0/1486286930 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f4624055dc0 con 0x7f46300fde90 2024-03-29T08:26:11.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:11.126+0000 7f4637c00700 1 -- 172.21.15.37:0/1486286930 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f463005f5f0 con 0x7f46300fde90 2024-03-29T08:26:11.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:11.127+0000 7f462cc00700 1 -- 172.21.15.37:0/1486286930 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7f46240593e0 con 0x7f46300fde90 2024-03-29T08:26:11.128 INFO:teuthology.orchestra.run.smithi037.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-03-29T08:26:11.128 INFO:teuthology.orchestra.run.smithi037.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-03-29T08:26:11.128 INFO:teuthology.orchestra.run.smithi037.stdout: fs cephfs is degraded 2024-03-29T08:26:11.128 INFO:teuthology.orchestra.run.smithi037.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-03-29T08:26:11.128 INFO:teuthology.orchestra.run.smithi037.stdout: fs cephfs is offline because no MDS is active for it. 2024-03-29T08:26:11.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:11.129+0000 7f4637c00700 1 -- 172.21.15.37:0/1486286930 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f461806cc30 msgr2=0x7f461806f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:11.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:11.129+0000 7f4637c00700 1 --2- 172.21.15.37:0/1486286930 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f461806cc30 0x7f461806f0e0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f46300fd690 tx=0x7f461c009e30 comp rx=0 tx=0).stop 2024-03-29T08:26:11.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:11.129+0000 7f4637c00700 1 -- 172.21.15.37:0/1486286930 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f46300fde90 msgr2=0x7f46300fbd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:11.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:11.129+0000 7f4637c00700 1 --2- 172.21.15.37:0/1486286930 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f46300fde90 0x7f46300fbd50 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f4624002a50 tx=0x7f46240046b0 comp rx=0 tx=0).stop 2024-03-29T08:26:11.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:11.129+0000 7f4637c00700 1 -- 172.21.15.37:0/1486286930 shutdown_connections 2024-03-29T08:26:11.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:11.129+0000 7f4637c00700 1 --2- 172.21.15.37:0/1486286930 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f461806cc30 0x7f461806f0e0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:11.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:11.129+0000 7f4637c00700 1 --2- 172.21.15.37:0/1486286930 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f46300fde90 0x7f46300fbd50 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:11.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:11.129+0000 7f4637c00700 1 --2- 172.21.15.37:0/1486286930 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f46300fe830 0x7f46300fc290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:11.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:11.129+0000 7f4637c00700 1 -- 172.21.15.37:0/1486286930 >> 172.21.15.37:0/1486286930 conn(0x7f4630072bc0 msgr2=0x7f4630103860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:26:11.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:11.129+0000 7f4637c00700 1 -- 172.21.15.37:0/1486286930 shutdown_connections 2024-03-29T08:26:11.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:11.129+0000 7f4637c00700 1 -- 172.21.15.37:0/1486286930 wait complete. 2024-03-29T08:26:11.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:26:11 smithi148 ceph-mon[106692]: pgmap v458: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:26:11.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:26:11 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/674174048' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T08:26:11.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:26:11 smithi148 ceph-mon[106692]: from='client.34888 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:26:11.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:26:11 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/1486286930' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T08:26:11.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:26:11 smithi037 ceph-mon[142325]: pgmap v458: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:26:11.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:26:11 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/674174048' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T08:26:11.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:26:11 smithi037 ceph-mon[142325]: from='client.34888 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:26:11.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:26:11 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/1486286930' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T08:26:13.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:26:13 smithi037 ceph-mon[142325]: pgmap v459: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:26:13.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:26:13 smithi148 ceph-mon[106692]: pgmap v459: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:26:15.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:26:15 smithi148 ceph-mon[106692]: pgmap v460: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:26:15.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:26:15 smithi037 ceph-mon[142325]: pgmap v460: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:26:17.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:26:17 smithi148 ceph-mon[106692]: pgmap v461: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:26:17.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:26:17 smithi037 ceph-mon[142325]: pgmap v461: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:26:19.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:26:19 smithi037 ceph-mon[142325]: pgmap v462: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:26:19.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:26:19 smithi148 ceph-mon[106692]: pgmap v462: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:26:21.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:26:21 smithi037 ceph-mon[142325]: pgmap v463: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:26:21.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:26:21 smithi148 ceph-mon[106692]: pgmap v463: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:26:23.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:26:23 smithi037 ceph-mon[142325]: pgmap v464: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:26:23.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:26:23 smithi148 ceph-mon[106692]: pgmap v464: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:26:25.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:26:25 smithi037 ceph-mon[142325]: pgmap v465: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:26:25.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:26:25 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:26:25.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:26:25 smithi148 ceph-mon[106692]: pgmap v465: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:26:25.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:26:25 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:26:27.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:26:27 smithi037 ceph-mon[142325]: pgmap v466: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:26:27.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:26:27 smithi148 ceph-mon[106692]: pgmap v466: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:26:29.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:26:29 smithi037 ceph-mon[142325]: pgmap v467: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:26:29.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:26:29 smithi148 ceph-mon[106692]: pgmap v467: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:26:31.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:26:31 smithi037 ceph-mon[142325]: pgmap v468: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:26:31.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:26:31 smithi148 ceph-mon[106692]: pgmap v468: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:26:33.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:26:33 smithi037 ceph-mon[142325]: pgmap v469: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:26:33.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:26:33 smithi148 ceph-mon[106692]: pgmap v469: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:26:35.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:26:35 smithi037 ceph-mon[142325]: pgmap v470: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:26:35.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:26:35 smithi148 ceph-mon[106692]: pgmap v470: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:26:37.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:26:37 smithi037 ceph-mon[142325]: pgmap v471: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:26:37.809 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:26:37 smithi148 ceph-mon[106692]: pgmap v471: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:26:39.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:26:39 smithi037 ceph-mon[142325]: pgmap v472: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:26:39.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:26:39 smithi148 ceph-mon[106692]: pgmap v472: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:26:40.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:26:40 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:26:40.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:26:40 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:26:41.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:41.286+0000 7f155b600700 1 -- 172.21.15.37:0/2541117310 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15540708a0 msgr2=0x7f1554070d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:41.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:41.286+0000 7f155b600700 1 --2- 172.21.15.37:0/2541117310 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15540708a0 0x7f1554070d10 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f1540004970 tx=0x7f1540009e30 comp rx=0 tx=0).stop 2024-03-29T08:26:41.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:41.286+0000 7f155b600700 1 -- 172.21.15.37:0/2541117310 shutdown_connections 2024-03-29T08:26:41.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:41.286+0000 7f155b600700 1 --2- 172.21.15.37:0/2541117310 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15540708a0 0x7f1554070d10 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:41.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:41.286+0000 7f155b600700 1 --2- 172.21.15.37:0/2541117310 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1554071f90 0x7f1554070360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:41.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:41.286+0000 7f155b600700 1 -- 172.21.15.37:0/2541117310 >> 172.21.15.37:0/2541117310 conn(0x7f15540f89b0 msgr2=0x7f15540fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:26:41.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:41.286+0000 7f155b600700 1 -- 172.21.15.37:0/2541117310 shutdown_connections 2024-03-29T08:26:41.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:41.286+0000 7f155b600700 1 -- 172.21.15.37:0/2541117310 wait complete. 2024-03-29T08:26:41.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:41.287+0000 7f155b600700 1 Processor -- start 2024-03-29T08:26:41.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:41.287+0000 7f155b600700 1 -- start start 2024-03-29T08:26:41.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:41.287+0000 7f155b600700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15540708a0 0x7f155411a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:41.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:41.287+0000 7f155b600700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1554071f90 0x7f155411a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:41.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:41.287+0000 7f155b600700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f155411ac40 con 0x7f15540708a0 2024-03-29T08:26:41.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:41.287+0000 7f155b600700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1554114010 con 0x7f1554071f90 2024-03-29T08:26:41.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:41.288+0000 7f1558c00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15540708a0 0x7f155411a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:41.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:41.288+0000 7f1558c00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15540708a0 0x7f155411a0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44538/0 (socket says 172.21.15.37:44538) 2024-03-29T08:26:41.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:41.288+0000 7f1558c00700 1 -- 172.21.15.37:0/2581002422 learned_addr learned my addr 172.21.15.37:0/2581002422 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:26:41.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:41.288+0000 7f1553e00700 1 --2- 172.21.15.37:0/2581002422 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1554071f90 0x7f155411a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:41.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:41.288+0000 7f1558c00700 1 -- 172.21.15.37:0/2581002422 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1554071f90 msgr2=0x7f155411a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:41.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:41.288+0000 7f1558c00700 1 --2- 172.21.15.37:0/2581002422 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1554071f90 0x7f155411a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:41.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:41.288+0000 7f1558c00700 1 -- 172.21.15.37:0/2581002422 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1540005040 con 0x7f15540708a0 2024-03-29T08:26:41.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:41.288+0000 7f1553e00700 1 --2- 172.21.15.37:0/2581002422 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1554071f90 0x7f155411a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:26:41.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:41.288+0000 7f1558c00700 1 --2- 172.21.15.37:0/2581002422 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15540708a0 0x7f155411a0b0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f154800c8c0 tx=0x7f15480044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:26:41.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:41.288+0000 7f1550c00700 1 -- 172.21.15.37:0/2581002422 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f154800a980 con 0x7f15540708a0 2024-03-29T08:26:41.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:41.289+0000 7f1550c00700 1 -- 172.21.15.37:0/2581002422 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f154800ce60 con 0x7f15540708a0 2024-03-29T08:26:41.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:41.289+0000 7f1550c00700 1 -- 172.21.15.37:0/2581002422 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f15480113a0 con 0x7f15540708a0 2024-03-29T08:26:41.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:41.289+0000 7f155b600700 1 -- 172.21.15.37:0/2581002422 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15541142f0 con 0x7f15540708a0 2024-03-29T08:26:41.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:41.289+0000 7f155b600700 1 -- 172.21.15.37:0/2581002422 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1554114800 con 0x7f15540708a0 2024-03-29T08:26:41.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:41.290+0000 7f155b600700 1 -- 172.21.15.37:0/2581002422 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1554071aa0 con 0x7f15540708a0 2024-03-29T08:26:41.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:41.291+0000 7f1550c00700 1 -- 172.21.15.37:0/2581002422 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f1548012400 con 0x7f15540708a0 2024-03-29T08:26:41.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:41.292+0000 7f1550c00700 1 --2- 172.21.15.37:0/2581002422 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f153c06cc30 0x7f153c06f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:41.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:41.292+0000 7f1550c00700 1 -- 172.21.15.37:0/2581002422 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f154808c0f0 con 0x7f15540708a0 2024-03-29T08:26:41.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:41.292+0000 7f1553e00700 1 --2- 172.21.15.37:0/2581002422 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f153c06cc30 0x7f153c06f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:41.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:41.293+0000 7f1553e00700 1 --2- 172.21.15.37:0/2581002422 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f153c06cc30 0x7f153c06f0e0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f1540009fd0 tx=0x7f1540009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:26:41.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:41.297+0000 7f1550c00700 1 -- 172.21.15.37:0/2581002422 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f1548059bc0 con 0x7f15540708a0 2024-03-29T08:26:41.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:41.529+0000 7f155b600700 1 -- 172.21.15.37:0/2581002422 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f1554115700 con 0x7f153c06cc30 2024-03-29T08:26:41.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:41.531+0000 7f1550c00700 1 -- 172.21.15.37:0/2581002422 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f1554115700 con 0x7f153c06cc30 2024-03-29T08:26:41.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:41.533+0000 7f155b600700 1 -- 172.21.15.37:0/2581002422 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f153c06cc30 msgr2=0x7f153c06f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:41.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:41.533+0000 7f155b600700 1 --2- 172.21.15.37:0/2581002422 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f153c06cc30 0x7f153c06f0e0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f1540009fd0 tx=0x7f1540009e30 comp rx=0 tx=0).stop 2024-03-29T08:26:41.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:41.533+0000 7f155b600700 1 -- 172.21.15.37:0/2581002422 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15540708a0 msgr2=0x7f155411a0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:41.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:41.533+0000 7f155b600700 1 --2- 172.21.15.37:0/2581002422 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15540708a0 0x7f155411a0b0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f154800c8c0 tx=0x7f15480044c0 comp rx=0 tx=0).stop 2024-03-29T08:26:41.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:41.533+0000 7f155b600700 1 -- 172.21.15.37:0/2581002422 shutdown_connections 2024-03-29T08:26:41.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:41.533+0000 7f155b600700 1 --2- 172.21.15.37:0/2581002422 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f153c06cc30 0x7f153c06f0e0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:41.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:41.533+0000 7f155b600700 1 --2- 172.21.15.37:0/2581002422 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15540708a0 0x7f155411a0b0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:41.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:41.533+0000 7f155b600700 1 --2- 172.21.15.37:0/2581002422 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1554071f90 0x7f155411a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:41.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:41.533+0000 7f155b600700 1 -- 172.21.15.37:0/2581002422 >> 172.21.15.37:0/2581002422 conn(0x7f15540f89b0 msgr2=0x7f15540f9b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:26:41.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:41.534+0000 7f155b600700 1 -- 172.21.15.37:0/2581002422 shutdown_connections 2024-03-29T08:26:41.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:41.534+0000 7f155b600700 1 -- 172.21.15.37:0/2581002422 wait complete. 2024-03-29T08:26:41.549 INFO:teuthology.orchestra.run.smithi037.stdout:true 2024-03-29T08:26:41.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:41.673+0000 7f60c0000700 1 -- 172.21.15.37:0/3976062863 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f60b8102e00 msgr2=0x7f60b81031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:41.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:41.673+0000 7f60c0000700 1 --2- 172.21.15.37:0/3976062863 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f60b8102e00 0x7f60b81031d0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f60ac004940 tx=0x7f60ac009e30 comp rx=0 tx=0).stop 2024-03-29T08:26:41.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:41.673+0000 7f60c0000700 1 -- 172.21.15.37:0/3976062863 shutdown_connections 2024-03-29T08:26:41.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:41.673+0000 7f60c0000700 1 --2- 172.21.15.37:0/3976062863 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f60b81037a0 0x7f60b80fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:41.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:41.673+0000 7f60c0000700 1 --2- 172.21.15.37:0/3976062863 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f60b8102e00 0x7f60b81031d0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:41.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:41.673+0000 7f60c0000700 1 -- 172.21.15.37:0/3976062863 >> 172.21.15.37:0/3976062863 conn(0x7f60b80f89b0 msgr2=0x7f60b80fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:26:41.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:41.673+0000 7f60c0000700 1 -- 172.21.15.37:0/3976062863 shutdown_connections 2024-03-29T08:26:41.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:41.673+0000 7f60c0000700 1 -- 172.21.15.37:0/3976062863 wait complete. 2024-03-29T08:26:41.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:41.674+0000 7f60c0000700 1 Processor -- start 2024-03-29T08:26:41.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:41.674+0000 7f60c0000700 1 -- start start 2024-03-29T08:26:41.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:41.675+0000 7f60c0000700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f60b8102e00 0x7f60b8113a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:41.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:41.675+0000 7f60c0000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f60b81037a0 0x7f60b8113f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:41.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:41.675+0000 7f60c0000700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60b81145d0 con 0x7f60b81037a0 2024-03-29T08:26:41.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:41.675+0000 7f60c0000700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60b810d9c0 con 0x7f60b8102e00 2024-03-29T08:26:41.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:41.675+0000 7f60bcc00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f60b81037a0 0x7f60b8113f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:41.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:41.675+0000 7f60bcc00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f60b81037a0 0x7f60b8113f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44550/0 (socket says 172.21.15.37:44550) 2024-03-29T08:26:41.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:41.675+0000 7f60bcc00700 1 -- 172.21.15.37:0/3478877735 learned_addr learned my addr 172.21.15.37:0/3478877735 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:26:41.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:41.675+0000 7f60bd600700 1 --2- 172.21.15.37:0/3478877735 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f60b8102e00 0x7f60b8113a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:41.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:41.675+0000 7f60bcc00700 1 -- 172.21.15.37:0/3478877735 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f60b8102e00 msgr2=0x7f60b8113a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:41.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:41.675+0000 7f60bcc00700 1 --2- 172.21.15.37:0/3478877735 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f60b8102e00 0x7f60b8113a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:41.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:41.675+0000 7f60bcc00700 1 -- 172.21.15.37:0/3478877735 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60ac005040 con 0x7f60b81037a0 2024-03-29T08:26:41.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:41.676+0000 7f60bcc00700 1 --2- 172.21.15.37:0/3478877735 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f60b81037a0 0x7f60b8113f80 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f60a400eb40 tx=0x7f60a400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:26:41.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:41.676+0000 7f60b5600700 1 -- 172.21.15.37:0/3478877735 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f60a4007cc0 con 0x7f60b81037a0 2024-03-29T08:26:41.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:41.676+0000 7f60b5600700 1 -- 172.21.15.37:0/3478877735 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f60a400f040 con 0x7f60b81037a0 2024-03-29T08:26:41.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:41.676+0000 7f60b5600700 1 -- 172.21.15.37:0/3478877735 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f60a4009570 con 0x7f60b81037a0 2024-03-29T08:26:41.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:41.676+0000 7f60c0000700 1 -- 172.21.15.37:0/3478877735 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60b810dc40 con 0x7f60b81037a0 2024-03-29T08:26:41.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:41.676+0000 7f60c0000700 1 -- 172.21.15.37:0/3478877735 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60b810e200 con 0x7f60b81037a0 2024-03-29T08:26:41.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:26:41 smithi037 ceph-mon[142325]: pgmap v473: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:26:41.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:41.677+0000 7f60c0000700 1 -- 172.21.15.37:0/3478877735 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60b80fe550 con 0x7f60b81037a0 2024-03-29T08:26:41.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:41.678+0000 7f60b5600700 1 -- 172.21.15.37:0/3478877735 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f60a40096d0 con 0x7f60b81037a0 2024-03-29T08:26:41.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:41.679+0000 7f60b5600700 1 --2- 172.21.15.37:0/3478877735 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f60a006cc30 0x7f60a006f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:41.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:41.679+0000 7f60b5600700 1 -- 172.21.15.37:0/3478877735 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f60a408bb00 con 0x7f60b81037a0 2024-03-29T08:26:41.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:41.679+0000 7f60bd600700 1 --2- 172.21.15.37:0/3478877735 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f60a006cc30 0x7f60a006f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:41.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:41.680+0000 7f60bd600700 1 --2- 172.21.15.37:0/3478877735 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f60a006cc30 0x7f60a006f0e0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f60ac00b0d0 tx=0x7f60ac009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:26:41.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:41.684+0000 7f60b5600700 1 -- 172.21.15.37:0/3478877735 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f60a4059600 con 0x7f60b81037a0 2024-03-29T08:26:41.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:26:41 smithi148 ceph-mon[106692]: pgmap v473: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:26:41.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:41.917+0000 7f60c0000700 1 -- 172.21.15.37:0/3478877735 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f60b8109dc0 con 0x7f60a006cc30 2024-03-29T08:26:41.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:41.918+0000 7f60b5600700 1 -- 172.21.15.37:0/3478877735 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f60b8109dc0 con 0x7f60a006cc30 2024-03-29T08:26:41.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:41.920+0000 7f60c0000700 1 -- 172.21.15.37:0/3478877735 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f60a006cc30 msgr2=0x7f60a006f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:41.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:41.920+0000 7f60c0000700 1 --2- 172.21.15.37:0/3478877735 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f60a006cc30 0x7f60a006f0e0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f60ac00b0d0 tx=0x7f60ac009e30 comp rx=0 tx=0).stop 2024-03-29T08:26:41.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:41.920+0000 7f60c0000700 1 -- 172.21.15.37:0/3478877735 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f60b81037a0 msgr2=0x7f60b8113f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:41.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:41.920+0000 7f60c0000700 1 --2- 172.21.15.37:0/3478877735 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f60b81037a0 0x7f60b8113f80 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f60a400eb40 tx=0x7f60a400b040 comp rx=0 tx=0).stop 2024-03-29T08:26:41.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:41.920+0000 7f60c0000700 1 -- 172.21.15.37:0/3478877735 shutdown_connections 2024-03-29T08:26:41.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:41.920+0000 7f60c0000700 1 --2- 172.21.15.37:0/3478877735 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f60a006cc30 0x7f60a006f0e0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:41.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:41.920+0000 7f60c0000700 1 --2- 172.21.15.37:0/3478877735 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f60b8102e00 0x7f60b8113a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:41.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:41.920+0000 7f60c0000700 1 --2- 172.21.15.37:0/3478877735 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f60b81037a0 0x7f60b8113f80 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:41.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:41.920+0000 7f60c0000700 1 -- 172.21.15.37:0/3478877735 >> 172.21.15.37:0/3478877735 conn(0x7f60b80f89b0 msgr2=0x7f60b80f99f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:26:41.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:41.921+0000 7f60c0000700 1 -- 172.21.15.37:0/3478877735 shutdown_connections 2024-03-29T08:26:41.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:41.921+0000 7f60c0000700 1 -- 172.21.15.37:0/3478877735 wait complete. 2024-03-29T08:26:42.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:42.058+0000 7faee4000700 1 -- 172.21.15.37:0/3254423303 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faedc0fe830 msgr2=0x7faedc106f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:42.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:42.058+0000 7faee4000700 1 --2- 172.21.15.37:0/3254423303 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faedc0fe830 0x7faedc106f00 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7faec8004940 tx=0x7faec8009e30 comp rx=0 tx=0).stop 2024-03-29T08:26:42.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:42.058+0000 7faee4000700 1 -- 172.21.15.37:0/3254423303 shutdown_connections 2024-03-29T08:26:42.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:42.058+0000 7faee4000700 1 --2- 172.21.15.37:0/3254423303 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faedc0fe830 0x7faedc106f00 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:42.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:42.058+0000 7faee4000700 1 --2- 172.21.15.37:0/3254423303 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faedc0fde90 0x7faedc0fe260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:42.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:42.058+0000 7faee4000700 1 -- 172.21.15.37:0/3254423303 >> 172.21.15.37:0/3254423303 conn(0x7faedc072bc0 msgr2=0x7faedc072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:26:42.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:42.058+0000 7faee4000700 1 -- 172.21.15.37:0/3254423303 shutdown_connections 2024-03-29T08:26:42.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:42.058+0000 7faee4000700 1 -- 172.21.15.37:0/3254423303 wait complete. 2024-03-29T08:26:42.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:42.059+0000 7faee4000700 1 Processor -- start 2024-03-29T08:26:42.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:42.059+0000 7faee4000700 1 -- start start 2024-03-29T08:26:42.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:42.060+0000 7faee4000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faedc0fde90 0x7faedc113ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:42.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:42.060+0000 7faee4000700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faedc0fe830 0x7faedc114020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:42.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:42.060+0000 7faee4000700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faedc114670 con 0x7faedc0fde90 2024-03-29T08:26:42.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:42.060+0000 7faee4000700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faedc10da60 con 0x7faedc0fe830 2024-03-29T08:26:42.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:42.060+0000 7faee1600700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faedc0fde90 0x7faedc113ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:42.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:42.060+0000 7faee1600700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faedc0fde90 0x7faedc113ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44580/0 (socket says 172.21.15.37:44580) 2024-03-29T08:26:42.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:42.060+0000 7faee1600700 1 -- 172.21.15.37:0/189535646 learned_addr learned my addr 172.21.15.37:0/189535646 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:26:42.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:42.061+0000 7faee0c00700 1 --2- 172.21.15.37:0/189535646 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faedc0fe830 0x7faedc114020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:42.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:42.061+0000 7faee1600700 1 -- 172.21.15.37:0/189535646 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faedc0fe830 msgr2=0x7faedc114020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:42.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:42.061+0000 7faee1600700 1 --2- 172.21.15.37:0/189535646 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faedc0fe830 0x7faedc114020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:42.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:42.061+0000 7faee1600700 1 -- 172.21.15.37:0/189535646 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faec8005040 con 0x7faedc0fde90 2024-03-29T08:26:42.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:42.061+0000 7faee1600700 1 --2- 172.21.15.37:0/189535646 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faedc0fde90 0x7faedc113ae0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7faed0002a50 tx=0x7faed00046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:26:42.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:42.061+0000 7faed9600700 1 -- 172.21.15.37:0/189535646 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faed0007cc0 con 0x7faedc0fde90 2024-03-29T08:26:42.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:42.061+0000 7faed9600700 1 -- 172.21.15.37:0/189535646 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7faed000f040 con 0x7faedc0fde90 2024-03-29T08:26:42.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:42.061+0000 7faee4000700 1 -- 172.21.15.37:0/189535646 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faedc10dd40 con 0x7faedc0fde90 2024-03-29T08:26:42.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:42.061+0000 7faed9600700 1 -- 172.21.15.37:0/189535646 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faed0009540 con 0x7faedc0fde90 2024-03-29T08:26:42.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:42.061+0000 7faee4000700 1 -- 172.21.15.37:0/189535646 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faedc10e280 con 0x7faedc0fde90 2024-03-29T08:26:42.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:42.062+0000 7faee4000700 1 -- 172.21.15.37:0/189535646 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faedc065e30 con 0x7faedc0fde90 2024-03-29T08:26:42.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:42.063+0000 7faed9600700 1 -- 172.21.15.37:0/189535646 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7faed00096a0 con 0x7faedc0fde90 2024-03-29T08:26:42.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:42.065+0000 7faed9600700 1 --2- 172.21.15.37:0/189535646 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7faec406cc30 0x7faec406f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:42.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:42.065+0000 7faed9600700 1 -- 172.21.15.37:0/189535646 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7faed008bee0 con 0x7faedc0fde90 2024-03-29T08:26:42.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:42.065+0000 7faee0c00700 1 --2- 172.21.15.37:0/189535646 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7faec406cc30 0x7faec406f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:42.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:42.066+0000 7faee0c00700 1 --2- 172.21.15.37:0/189535646 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7faec406cc30 0x7faec406f0e0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7faec800b3b0 tx=0x7faec8009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:26:42.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:42.069+0000 7faed9600700 1 -- 172.21.15.37:0/189535646 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7faed00598b0 con 0x7faedc0fde90 2024-03-29T08:26:42.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:42.290+0000 7faee4000700 1 -- 172.21.15.37:0/189535646 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7faedc109dd0 con 0x7faec406cc30 2024-03-29T08:26:42.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:42.300+0000 7faed9600700 1 -- 172.21.15.37:0/189535646 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7faedc109dd0 con 0x7faec406cc30 2024-03-29T08:26:42.300 INFO:teuthology.orchestra.run.smithi037.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-29T08:26:42.300 INFO:teuthology.orchestra.run.smithi037.stdout:alertmanager.smithi037 smithi037 *:9093,9094 running (16m) 8m ago 29m 14.9M - 0.25.0 c8568f914cd2 2f9f9746ed0f 2024-03-29T08:26:42.300 INFO:teuthology.orchestra.run.smithi037.stdout:crash.smithi037 smithi037 running (13m) 8m ago 29m 6647k - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 178d416a3186 2024-03-29T08:26:42.301 INFO:teuthology.orchestra.run.smithi037.stdout:crash.smithi148 smithi148 running (13m) 8m ago 28m 6656k - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 23666989d58f 2024-03-29T08:26:42.301 INFO:teuthology.orchestra.run.smithi037.stdout:grafana.smithi037 smithi037 *:3000 running (16m) 8m ago 29m 71.9M - 9.4.7 954c08fa6188 6b976fe49952 2024-03-29T08:26:42.301 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi037.iifsmh smithi037 running (8m) 8m ago 23m 10.8M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 2d97f562cc5b 2024-03-29T08:26:42.301 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi037.mmwgxe smithi037 running (9m) 8m ago 23m 10.9M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff ec37037f46e8 2024-03-29T08:26:42.301 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi148.qukpjh smithi148 running (22m) 8m ago 22m 15.3M - 17.2.7-789-g4213e17c cf22eb8683be 8e469782a273 2024-03-29T08:26:42.301 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi148.sqteih smithi148 running (8m) 8m ago 23m 10.7M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 44c15c77037a 2024-03-29T08:26:42.301 INFO:teuthology.orchestra.run.smithi037.stdout:mgr.smithi037.wjikii smithi037 *:8443,9283,8765 running (18m) 8m ago 31m 506M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff c5a1417b9417 2024-03-29T08:26:42.301 INFO:teuthology.orchestra.run.smithi037.stdout:mgr.smithi148.onhgvm smithi148 *:8443,9283,8765 running (17m) 8m ago 28m 450M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff fcfdbce062bd 2024-03-29T08:26:42.301 INFO:teuthology.orchestra.run.smithi037.stdout:mon.smithi037 smithi037 running (14m) 8m ago 31m 54.8M 2048M 18.2.2-1015-gd0e392d9 1cc2f45c17ff eea2060783af 2024-03-29T08:26:42.301 INFO:teuthology.orchestra.run.smithi037.stdout:mon.smithi148 smithi148 running (14m) 8m ago 28m 43.7M 2048M 18.2.2-1015-gd0e392d9 1cc2f45c17ff ad4b2c5748be 2024-03-29T08:26:42.301 INFO:teuthology.orchestra.run.smithi037.stdout:node-exporter.smithi037 smithi037 *:9100 running (17m) 8m ago 29m 7625k - 1.5.0 0da6a335fe13 26b8ae9cb878 2024-03-29T08:26:42.301 INFO:teuthology.orchestra.run.smithi037.stdout:node-exporter.smithi148 smithi148 *:9100 running (17m) 8m ago 28m 8166k - 1.5.0 0da6a335fe13 98414e4e64ed 2024-03-29T08:26:42.301 INFO:teuthology.orchestra.run.smithi037.stdout:osd.0 smithi037 running (12m) 8m ago 27m 104M 1619M 18.2.2-1015-gd0e392d9 1cc2f45c17ff c0c584f6245c 2024-03-29T08:26:42.301 INFO:teuthology.orchestra.run.smithi037.stdout:osd.1 smithi037 running (12m) 8m ago 26m 100M 1619M 18.2.2-1015-gd0e392d9 1cc2f45c17ff a4a2ff4cc83d 2024-03-29T08:26:42.301 INFO:teuthology.orchestra.run.smithi037.stdout:osd.2 smithi037 running (11m) 8m ago 26m 82.3M 1619M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 8b1bc71aa79e 2024-03-29T08:26:42.302 INFO:teuthology.orchestra.run.smithi037.stdout:osd.3 smithi148 running (10m) 8m ago 25m 105M 2642M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 1c656ab8ff86 2024-03-29T08:26:42.302 INFO:teuthology.orchestra.run.smithi037.stdout:osd.4 smithi148 running (10m) 8m ago 25m 97.3M 2642M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 966a9de11d22 2024-03-29T08:26:42.302 INFO:teuthology.orchestra.run.smithi037.stdout:osd.5 smithi148 running (9m) 8m ago 24m 85.8M 2642M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 474c4f9212ce 2024-03-29T08:26:42.302 INFO:teuthology.orchestra.run.smithi037.stdout:prometheus.smithi037 smithi037 *:9095 running (16m) 8m ago 28m 54.4M - 2.43.0 a07b618ecd1d 30c0223ec589 2024-03-29T08:26:42.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:42.302+0000 7faee4000700 1 -- 172.21.15.37:0/189535646 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7faec406cc30 msgr2=0x7faec406f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:42.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:42.302+0000 7faee4000700 1 --2- 172.21.15.37:0/189535646 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7faec406cc30 0x7faec406f0e0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7faec800b3b0 tx=0x7faec8009e30 comp rx=0 tx=0).stop 2024-03-29T08:26:42.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:42.302+0000 7faee4000700 1 -- 172.21.15.37:0/189535646 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faedc0fde90 msgr2=0x7faedc113ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:42.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:42.302+0000 7faee4000700 1 --2- 172.21.15.37:0/189535646 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faedc0fde90 0x7faedc113ae0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7faed0002a50 tx=0x7faed00046b0 comp rx=0 tx=0).stop 2024-03-29T08:26:42.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:42.302+0000 7faee4000700 1 -- 172.21.15.37:0/189535646 shutdown_connections 2024-03-29T08:26:42.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:42.302+0000 7faee4000700 1 --2- 172.21.15.37:0/189535646 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7faec406cc30 0x7faec406f0e0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:42.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:42.302+0000 7faee4000700 1 --2- 172.21.15.37:0/189535646 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faedc0fde90 0x7faedc113ae0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:42.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:42.302+0000 7faee4000700 1 --2- 172.21.15.37:0/189535646 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faedc0fe830 0x7faedc114020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:42.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:42.302+0000 7faee4000700 1 -- 172.21.15.37:0/189535646 >> 172.21.15.37:0/189535646 conn(0x7faedc072bc0 msgr2=0x7faedc103860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:26:42.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:42.303+0000 7faee4000700 1 -- 172.21.15.37:0/189535646 shutdown_connections 2024-03-29T08:26:42.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:42.303+0000 7faee4000700 1 -- 172.21.15.37:0/189535646 wait complete. 2024-03-29T08:26:42.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:42.440+0000 7f1157000700 1 -- 172.21.15.37:0/896947884 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1150102e20 msgr2=0x7f11501031f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:42.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:42.440+0000 7f1157000700 1 --2- 172.21.15.37:0/896947884 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1150102e20 0x7f11501031f0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f1140004940 tx=0x7f1140009e30 comp rx=0 tx=0).stop 2024-03-29T08:26:42.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:42.440+0000 7f1157000700 1 -- 172.21.15.37:0/896947884 shutdown_connections 2024-03-29T08:26:42.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:42.440+0000 7f1157000700 1 --2- 172.21.15.37:0/896947884 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f11501037c0 0x7f11500fce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:42.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:42.440+0000 7f1157000700 1 --2- 172.21.15.37:0/896947884 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1150102e20 0x7f11501031f0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:42.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:42.440+0000 7f1157000700 1 -- 172.21.15.37:0/896947884 >> 172.21.15.37:0/896947884 conn(0x7f11500f8970 msgr2=0x7f11500fad80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:26:42.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:42.440+0000 7f1157000700 1 -- 172.21.15.37:0/896947884 shutdown_connections 2024-03-29T08:26:42.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:42.440+0000 7f1157000700 1 -- 172.21.15.37:0/896947884 wait complete. 2024-03-29T08:26:42.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:42.441+0000 7f1157000700 1 Processor -- start 2024-03-29T08:26:42.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:42.441+0000 7f1157000700 1 -- start start 2024-03-29T08:26:42.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:42.441+0000 7f1157000700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1150102e20 0x7f1150115d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:42.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:42.441+0000 7f1157000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f11501037c0 0x7f1150116250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:42.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:42.441+0000 7f1157000700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11501168a0 con 0x7f11501037c0 2024-03-29T08:26:42.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:42.441+0000 7f1157000700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f115010fc70 con 0x7f1150102e20 2024-03-29T08:26:42.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:42.442+0000 7f114f400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f11501037c0 0x7f1150116250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:42.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:42.442+0000 7f114f400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f11501037c0 0x7f1150116250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44610/0 (socket says 172.21.15.37:44610) 2024-03-29T08:26:42.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:42.442+0000 7f114f400700 1 -- 172.21.15.37:0/3517284242 learned_addr learned my addr 172.21.15.37:0/3517284242 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:26:42.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:42.442+0000 7f114fe00700 1 --2- 172.21.15.37:0/3517284242 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1150102e20 0x7f1150115d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:42.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:42.442+0000 7f114f400700 1 -- 172.21.15.37:0/3517284242 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1150102e20 msgr2=0x7f1150115d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:42.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:42.442+0000 7f114f400700 1 --2- 172.21.15.37:0/3517284242 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1150102e20 0x7f1150115d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:42.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:42.442+0000 7f114f400700 1 -- 172.21.15.37:0/3517284242 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1140005040 con 0x7f11501037c0 2024-03-29T08:26:42.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:42.442+0000 7f114fe00700 1 --2- 172.21.15.37:0/3517284242 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1150102e20 0x7f1150115d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:26:42.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:42.442+0000 7f114f400700 1 --2- 172.21.15.37:0/3517284242 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f11501037c0 0x7f1150116250 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f113c002a50 tx=0x7f113c004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:26:42.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:42.443+0000 7f1147e00700 1 -- 172.21.15.37:0/3517284242 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f113c007cc0 con 0x7f11501037c0 2024-03-29T08:26:42.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:42.443+0000 7f1147e00700 1 -- 172.21.15.37:0/3517284242 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f113c00f040 con 0x7f11501037c0 2024-03-29T08:26:42.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:42.443+0000 7f1157000700 1 -- 172.21.15.37:0/3517284242 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f115010ff50 con 0x7f11501037c0 2024-03-29T08:26:42.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:42.443+0000 7f1147e00700 1 -- 172.21.15.37:0/3517284242 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f113c00d990 con 0x7f11501037c0 2024-03-29T08:26:42.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:42.443+0000 7f1157000700 1 -- 172.21.15.37:0/3517284242 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1150110510 con 0x7f11501037c0 2024-03-29T08:26:42.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:42.444+0000 7f1157000700 1 -- 172.21.15.37:0/3517284242 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11500fe570 con 0x7f11501037c0 2024-03-29T08:26:42.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:42.445+0000 7f1147e00700 1 -- 172.21.15.37:0/3517284242 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f113c009710 con 0x7f11501037c0 2024-03-29T08:26:42.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:42.446+0000 7f1147e00700 1 --2- 172.21.15.37:0/3517284242 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f113406cc30 0x7f113406f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:42.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:42.446+0000 7f1147e00700 1 -- 172.21.15.37:0/3517284242 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f113c08be20 con 0x7f11501037c0 2024-03-29T08:26:42.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:42.446+0000 7f114fe00700 1 --2- 172.21.15.37:0/3517284242 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f113406cc30 0x7f113406f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:42.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:42.447+0000 7f114fe00700 1 --2- 172.21.15.37:0/3517284242 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f113406cc30 0x7f113406f0e0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f114000b840 tx=0x7f1140009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:26:42.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:42.451+0000 7f1147e00700 1 -- 172.21.15.37:0/3517284242 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f113c0595e0 con 0x7f11501037c0 2024-03-29T08:26:42.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:26:42 smithi037 ceph-mon[142325]: from='client.34896 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:26:42.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:42.753+0000 7f1157000700 1 -- 172.21.15.37:0/3517284242 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f115004bed0 con 0x7f11501037c0 2024-03-29T08:26:42.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:42.754+0000 7f1147e00700 1 -- 172.21.15.37:0/3517284242 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+847 (secure 0 0 0) 0x7f113c059400 con 0x7f11501037c0 2024-03-29T08:26:42.754 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-29T08:26:42.754 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": { 2024-03-29T08:26:42.755 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 2 2024-03-29T08:26:42.755 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:26:42.755 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": { 2024-03-29T08:26:42.755 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 2 2024-03-29T08:26:42.755 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:26:42.755 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": { 2024-03-29T08:26:42.755 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 6 2024-03-29T08:26:42.755 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:26:42.755 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": { 2024-03-29T08:26:42.755 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 17.2.7-789-g4213e17c (4213e17c4de67016ed14f55d22503f4923b3a901) quincy (stable)": 1, 2024-03-29T08:26:42.755 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 3 2024-03-29T08:26:42.756 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:26:42.756 INFO:teuthology.orchestra.run.smithi037.stdout: "overall": { 2024-03-29T08:26:42.756 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 17.2.7-789-g4213e17c (4213e17c4de67016ed14f55d22503f4923b3a901) quincy (stable)": 1, 2024-03-29T08:26:42.756 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 13 2024-03-29T08:26:42.756 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-29T08:26:42.756 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-29T08:26:42.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:42.756+0000 7f1157000700 1 -- 172.21.15.37:0/3517284242 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f113406cc30 msgr2=0x7f113406f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:42.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:42.756+0000 7f1157000700 1 --2- 172.21.15.37:0/3517284242 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f113406cc30 0x7f113406f0e0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f114000b840 tx=0x7f1140009e30 comp rx=0 tx=0).stop 2024-03-29T08:26:42.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:42.756+0000 7f1157000700 1 -- 172.21.15.37:0/3517284242 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f11501037c0 msgr2=0x7f1150116250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:42.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:42.756+0000 7f1157000700 1 --2- 172.21.15.37:0/3517284242 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f11501037c0 0x7f1150116250 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f113c002a50 tx=0x7f113c004600 comp rx=0 tx=0).stop 2024-03-29T08:26:42.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:42.756+0000 7f1157000700 1 -- 172.21.15.37:0/3517284242 shutdown_connections 2024-03-29T08:26:42.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:42.756+0000 7f1157000700 1 --2- 172.21.15.37:0/3517284242 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f113406cc30 0x7f113406f0e0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:42.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:42.756+0000 7f1157000700 1 --2- 172.21.15.37:0/3517284242 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1150102e20 0x7f1150115d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:42.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:42.756+0000 7f1157000700 1 --2- 172.21.15.37:0/3517284242 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f11501037c0 0x7f1150116250 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:42.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:42.756+0000 7f1157000700 1 -- 172.21.15.37:0/3517284242 >> 172.21.15.37:0/3517284242 conn(0x7f11500f8970 msgr2=0x7f11500f9970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:26:42.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:42.757+0000 7f1157000700 1 -- 172.21.15.37:0/3517284242 shutdown_connections 2024-03-29T08:26:42.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:42.757+0000 7f1157000700 1 -- 172.21.15.37:0/3517284242 wait complete. 2024-03-29T08:26:42.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:26:42 smithi148 ceph-mon[106692]: from='client.34896 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:26:42.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:42.894+0000 7f0b9e800700 1 -- 172.21.15.37:0/3825657610 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0b980708a0 msgr2=0x7f0b98070d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:42.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:42.894+0000 7f0b9e800700 1 --2- 172.21.15.37:0/3825657610 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0b980708a0 0x7f0b98070d10 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f0b84004970 tx=0x7f0b84009e30 comp rx=0 tx=0).stop 2024-03-29T08:26:42.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:42.894+0000 7f0b9e800700 1 -- 172.21.15.37:0/3825657610 shutdown_connections 2024-03-29T08:26:42.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:42.894+0000 7f0b9e800700 1 --2- 172.21.15.37:0/3825657610 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0b980708a0 0x7f0b98070d10 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:42.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:42.894+0000 7f0b9e800700 1 --2- 172.21.15.37:0/3825657610 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0b98071f90 0x7f0b98070360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:42.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:42.894+0000 7f0b9e800700 1 -- 172.21.15.37:0/3825657610 >> 172.21.15.37:0/3825657610 conn(0x7f0b980f8830 msgr2=0x7f0b980fac40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:26:42.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:42.894+0000 7f0b9e800700 1 -- 172.21.15.37:0/3825657610 shutdown_connections 2024-03-29T08:26:42.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:42.894+0000 7f0b9e800700 1 -- 172.21.15.37:0/3825657610 wait complete. 2024-03-29T08:26:42.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:42.895+0000 7f0b9e800700 1 Processor -- start 2024-03-29T08:26:42.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:42.895+0000 7f0b9e800700 1 -- start start 2024-03-29T08:26:42.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:42.896+0000 7f0b9e800700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0b980708a0 0x7f0b98119f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:42.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:42.896+0000 7f0b9e800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0b98071f90 0x7f0b9811a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:42.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:42.896+0000 7f0b9e800700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b9811aab0 con 0x7f0b98071f90 2024-03-29T08:26:42.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:42.896+0000 7f0b9e800700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b98113e80 con 0x7f0b980708a0 2024-03-29T08:26:42.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:42.896+0000 7f0b9ca00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0b98071f90 0x7f0b9811a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:42.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:42.896+0000 7f0b9ca00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0b98071f90 0x7f0b9811a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44618/0 (socket says 172.21.15.37:44618) 2024-03-29T08:26:42.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:42.896+0000 7f0b9ca00700 1 -- 172.21.15.37:0/4272439088 learned_addr learned my addr 172.21.15.37:0/4272439088 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:26:42.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:42.896+0000 7f0b9d400700 1 --2- 172.21.15.37:0/4272439088 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0b980708a0 0x7f0b98119f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:42.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:42.896+0000 7f0b9ca00700 1 -- 172.21.15.37:0/4272439088 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0b980708a0 msgr2=0x7f0b98119f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:42.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:42.896+0000 7f0b9ca00700 1 --2- 172.21.15.37:0/4272439088 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0b980708a0 0x7f0b98119f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:42.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:42.896+0000 7f0b9ca00700 1 -- 172.21.15.37:0/4272439088 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b84005040 con 0x7f0b98071f90 2024-03-29T08:26:42.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:42.897+0000 7f0b9d400700 1 --2- 172.21.15.37:0/4272439088 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0b980708a0 0x7f0b98119f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:26:42.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:42.897+0000 7f0b9ca00700 1 --2- 172.21.15.37:0/4272439088 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0b98071f90 0x7f0b9811a460 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f0b84004f50 tx=0x7f0b84009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:26:42.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:42.897+0000 7f0b95600700 1 -- 172.21.15.37:0/4272439088 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0b8400e040 con 0x7f0b98071f90 2024-03-29T08:26:42.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:42.897+0000 7f0b95600700 1 -- 172.21.15.37:0/4272439088 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f0b84013420 con 0x7f0b98071f90 2024-03-29T08:26:42.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:42.897+0000 7f0b9e800700 1 -- 172.21.15.37:0/4272439088 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b98114100 con 0x7f0b98071f90 2024-03-29T08:26:42.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:42.897+0000 7f0b95600700 1 -- 172.21.15.37:0/4272439088 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0b8401c610 con 0x7f0b98071f90 2024-03-29T08:26:42.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:42.897+0000 7f0b9e800700 1 -- 172.21.15.37:0/4272439088 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b98114640 con 0x7f0b98071f90 2024-03-29T08:26:42.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:42.898+0000 7f0b9e800700 1 -- 172.21.15.37:0/4272439088 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b98071aa0 con 0x7f0b98071f90 2024-03-29T08:26:42.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:42.899+0000 7f0b95600700 1 -- 172.21.15.37:0/4272439088 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f0b84013590 con 0x7f0b98071f90 2024-03-29T08:26:42.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:42.900+0000 7f0b95600700 1 --2- 172.21.15.37:0/4272439088 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f0b8006cc30 0x7f0b8006f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:42.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:42.900+0000 7f0b95600700 1 -- 172.21.15.37:0/4272439088 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f0b8408d2f0 con 0x7f0b98071f90 2024-03-29T08:26:42.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:42.900+0000 7f0b9d400700 1 --2- 172.21.15.37:0/4272439088 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f0b8006cc30 0x7f0b8006f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:42.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:42.901+0000 7f0b9d400700 1 --2- 172.21.15.37:0/4272439088 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f0b8006cc30 0x7f0b8006f0e0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f0b8c007fd0 tx=0x7f0b8c009f60 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:26:42.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:42.905+0000 7f0b95600700 1 -- 172.21.15.37:0/4272439088 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f0b8405ad40 con 0x7f0b98071f90 2024-03-29T08:26:43.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:43.164+0000 7f0b9e800700 1 -- 172.21.15.37:0/4272439088 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f0b9804bed0 con 0x7f0b98071f90 2024-03-29T08:26:43.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:43.165+0000 7f0b95600700 1 -- 172.21.15.37:0/4272439088 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 28 v28) v1 ==== 76+0+1841 (secure 0 0 0) 0x7f0b8405a8d0 con 0x7f0b98071f90 2024-03-29T08:26:43.165 INFO:teuthology.orchestra.run.smithi037.stdout:e28 2024-03-29T08:26:43.165 INFO:teuthology.orchestra.run.smithi037.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-29T08:26:43.165 INFO:teuthology.orchestra.run.smithi037.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T08:26:43.165 INFO:teuthology.orchestra.run.smithi037.stdout:legacy client fscid: 1 2024-03-29T08:26:43.165 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:26:43.165 INFO:teuthology.orchestra.run.smithi037.stdout:Filesystem 'cephfs' (1) 2024-03-29T08:26:43.165 INFO:teuthology.orchestra.run.smithi037.stdout:fs_name cephfs 2024-03-29T08:26:43.165 INFO:teuthology.orchestra.run.smithi037.stdout:epoch 21 2024-03-29T08:26:43.165 INFO:teuthology.orchestra.run.smithi037.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-03-29T08:26:43.165 INFO:teuthology.orchestra.run.smithi037.stdout:created 2024-03-29T08:03:29.241269+0000 2024-03-29T08:26:43.166 INFO:teuthology.orchestra.run.smithi037.stdout:modified 2024-03-29T08:17:16.237238+0000 2024-03-29T08:26:43.166 INFO:teuthology.orchestra.run.smithi037.stdout:tableserver 0 2024-03-29T08:26:43.166 INFO:teuthology.orchestra.run.smithi037.stdout:root 0 2024-03-29T08:26:43.166 INFO:teuthology.orchestra.run.smithi037.stdout:session_timeout 60 2024-03-29T08:26:43.166 INFO:teuthology.orchestra.run.smithi037.stdout:session_autoclose 300 2024-03-29T08:26:43.166 INFO:teuthology.orchestra.run.smithi037.stdout:max_file_size 1099511627776 2024-03-29T08:26:43.166 INFO:teuthology.orchestra.run.smithi037.stdout:max_xattr_size 65536 2024-03-29T08:26:43.166 INFO:teuthology.orchestra.run.smithi037.stdout:required_client_features {} 2024-03-29T08:26:43.166 INFO:teuthology.orchestra.run.smithi037.stdout:last_failure 0 2024-03-29T08:26:43.166 INFO:teuthology.orchestra.run.smithi037.stdout:last_failure_osd_epoch 82 2024-03-29T08:26:43.166 INFO:teuthology.orchestra.run.smithi037.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T08:26:43.166 INFO:teuthology.orchestra.run.smithi037.stdout:max_mds 2 2024-03-29T08:26:43.166 INFO:teuthology.orchestra.run.smithi037.stdout:in 0,1 2024-03-29T08:26:43.166 INFO:teuthology.orchestra.run.smithi037.stdout:up {} 2024-03-29T08:26:43.167 INFO:teuthology.orchestra.run.smithi037.stdout:failed 0,1 2024-03-29T08:26:43.167 INFO:teuthology.orchestra.run.smithi037.stdout:damaged 2024-03-29T08:26:43.167 INFO:teuthology.orchestra.run.smithi037.stdout:stopped 2024-03-29T08:26:43.167 INFO:teuthology.orchestra.run.smithi037.stdout:data_pools [3] 2024-03-29T08:26:43.167 INFO:teuthology.orchestra.run.smithi037.stdout:metadata_pool 2 2024-03-29T08:26:43.167 INFO:teuthology.orchestra.run.smithi037.stdout:inline_data disabled 2024-03-29T08:26:43.167 INFO:teuthology.orchestra.run.smithi037.stdout:balancer 2024-03-29T08:26:43.167 INFO:teuthology.orchestra.run.smithi037.stdout:bal_rank_mask -1 2024-03-29T08:26:43.167 INFO:teuthology.orchestra.run.smithi037.stdout:standby_count_wanted 1 2024-03-29T08:26:43.167 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:26:43.167 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:26:43.167 INFO:teuthology.orchestra.run.smithi037.stdout:Standby daemons: 2024-03-29T08:26:43.167 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:26:43.167 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi148.qukpjh{-1:24319} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.148:6826/2641327912,v1:172.21.15.148:6827/2641327912] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:26:43.167 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi037.mmwgxe{-1:34440} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.37:6828/1248142005,v1:172.21.15.37:6829/1248142005] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:26:43.168 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi037.iifsmh{-1:34444} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.37:6826/2060882959,v1:172.21.15.37:6827/2060882959] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:26:43.168 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi148.sqteih{-1:44395} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.148:6824/4078652242,v1:172.21.15.148:6825/4078652242] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:26:43.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:43.167+0000 7f0b9e800700 1 -- 172.21.15.37:0/4272439088 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f0b8006cc30 msgr2=0x7f0b8006f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:43.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:43.167+0000 7f0b9e800700 1 --2- 172.21.15.37:0/4272439088 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f0b8006cc30 0x7f0b8006f0e0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f0b8c007fd0 tx=0x7f0b8c009f60 comp rx=0 tx=0).stop 2024-03-29T08:26:43.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:43.167+0000 7f0b9e800700 1 -- 172.21.15.37:0/4272439088 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0b98071f90 msgr2=0x7f0b9811a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:43.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:43.167+0000 7f0b9e800700 1 --2- 172.21.15.37:0/4272439088 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0b98071f90 0x7f0b9811a460 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f0b84004f50 tx=0x7f0b84009e30 comp rx=0 tx=0).stop 2024-03-29T08:26:43.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:43.167+0000 7f0b9e800700 1 -- 172.21.15.37:0/4272439088 shutdown_connections 2024-03-29T08:26:43.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:43.167+0000 7f0b9e800700 1 --2- 172.21.15.37:0/4272439088 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f0b8006cc30 0x7f0b8006f0e0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:43.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:43.167+0000 7f0b9e800700 1 --2- 172.21.15.37:0/4272439088 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0b980708a0 0x7f0b98119f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:43.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:43.167+0000 7f0b9e800700 1 --2- 172.21.15.37:0/4272439088 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0b98071f90 0x7f0b9811a460 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:43.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:43.167+0000 7f0b9e800700 1 -- 172.21.15.37:0/4272439088 >> 172.21.15.37:0/4272439088 conn(0x7f0b980f8830 msgr2=0x7f0b980f99a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:26:43.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:43.167+0000 7f0b9e800700 1 -- 172.21.15.37:0/4272439088 shutdown_connections 2024-03-29T08:26:43.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:43.167+0000 7f0b9e800700 1 -- 172.21.15.37:0/4272439088 wait complete. 2024-03-29T08:26:43.169 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 28 2024-03-29T08:26:43.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:43.305+0000 7f04a7000700 1 -- 172.21.15.37:0/1331985203 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f04a00fe830 msgr2=0x7f04a0106f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:43.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:43.305+0000 7f04a7000700 1 --2- 172.21.15.37:0/1331985203 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f04a00fe830 0x7f04a0106f00 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f048c004940 tx=0x7f048c009e30 comp rx=0 tx=0).stop 2024-03-29T08:26:43.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:43.305+0000 7f04a7000700 1 -- 172.21.15.37:0/1331985203 shutdown_connections 2024-03-29T08:26:43.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:43.305+0000 7f04a7000700 1 --2- 172.21.15.37:0/1331985203 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f04a00fe830 0x7f04a0106f00 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:43.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:43.305+0000 7f04a7000700 1 --2- 172.21.15.37:0/1331985203 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f04a00fde90 0x7f04a00fe260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:43.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:43.305+0000 7f04a7000700 1 -- 172.21.15.37:0/1331985203 >> 172.21.15.37:0/1331985203 conn(0x7f04a0072bc0 msgr2=0x7f04a0072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:26:43.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:43.305+0000 7f04a7000700 1 -- 172.21.15.37:0/1331985203 shutdown_connections 2024-03-29T08:26:43.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:43.306+0000 7f04a7000700 1 -- 172.21.15.37:0/1331985203 wait complete. 2024-03-29T08:26:43.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:43.306+0000 7f04a7000700 1 Processor -- start 2024-03-29T08:26:43.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:43.307+0000 7f04a7000700 1 -- start start 2024-03-29T08:26:43.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:43.307+0000 7f04a7000700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f04a00fde90 0x7f04a0115d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:43.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:43.307+0000 7f04a7000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f04a00fe830 0x7f04a0116250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:43.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:43.307+0000 7f04a7000700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04a01168a0 con 0x7f04a00fe830 2024-03-29T08:26:43.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:43.307+0000 7f04a7000700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04a010fc70 con 0x7f04a00fde90 2024-03-29T08:26:43.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:43.307+0000 7f049f400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f04a00fe830 0x7f04a0116250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:43.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:43.307+0000 7f049f400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f04a00fe830 0x7f04a0116250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44638/0 (socket says 172.21.15.37:44638) 2024-03-29T08:26:43.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:43.307+0000 7f049f400700 1 -- 172.21.15.37:0/2539602325 learned_addr learned my addr 172.21.15.37:0/2539602325 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:26:43.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:43.307+0000 7f049fe00700 1 --2- 172.21.15.37:0/2539602325 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f04a00fde90 0x7f04a0115d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:43.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:43.308+0000 7f049f400700 1 -- 172.21.15.37:0/2539602325 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f04a00fde90 msgr2=0x7f04a0115d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:43.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:43.308+0000 7f049f400700 1 --2- 172.21.15.37:0/2539602325 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f04a00fde90 0x7f04a0115d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:43.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:43.308+0000 7f049f400700 1 -- 172.21.15.37:0/2539602325 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f048c005040 con 0x7f04a00fe830 2024-03-29T08:26:43.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:43.308+0000 7f049f400700 1 --2- 172.21.15.37:0/2539602325 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f04a00fe830 0x7f04a0116250 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f048c000c00 tx=0x7f048c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:26:43.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:43.308+0000 7f0497e00700 1 -- 172.21.15.37:0/2539602325 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f048c00e040 con 0x7f04a00fe830 2024-03-29T08:26:43.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:43.308+0000 7f0497e00700 1 -- 172.21.15.37:0/2539602325 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f048c00bed0 con 0x7f04a00fe830 2024-03-29T08:26:43.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:43.308+0000 7f04a7000700 1 -- 172.21.15.37:0/2539602325 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04a010fef0 con 0x7f04a00fe830 2024-03-29T08:26:43.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:43.308+0000 7f0497e00700 1 -- 172.21.15.37:0/2539602325 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f048c01c3f0 con 0x7f04a00fe830 2024-03-29T08:26:43.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:43.308+0000 7f04a7000700 1 -- 172.21.15.37:0/2539602325 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04a0110430 con 0x7f04a00fe830 2024-03-29T08:26:43.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:43.310+0000 7f04a7000700 1 -- 172.21.15.37:0/2539602325 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04a0106a20 con 0x7f04a00fe830 2024-03-29T08:26:43.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:43.310+0000 7f0497e00700 1 -- 172.21.15.37:0/2539602325 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f048c013420 con 0x7f04a00fe830 2024-03-29T08:26:43.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:43.311+0000 7f0497e00700 1 --2- 172.21.15.37:0/2539602325 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f048406cc30 0x7f048406f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:43.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:43.311+0000 7f0497e00700 1 -- 172.21.15.37:0/2539602325 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f048c08d440 con 0x7f04a00fe830 2024-03-29T08:26:43.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:43.312+0000 7f049fe00700 1 --2- 172.21.15.37:0/2539602325 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f048406cc30 0x7f048406f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:43.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:43.312+0000 7f049fe00700 1 --2- 172.21.15.37:0/2539602325 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f048406cc30 0x7f048406f0e0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f0490004f40 tx=0x7f049000b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:26:43.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:43.316+0000 7f0497e00700 1 -- 172.21.15.37:0/2539602325 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f048c05af10 con 0x7f04a00fe830 2024-03-29T08:26:43.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:26:43 smithi037 ceph-mon[142325]: pgmap v474: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:26:43.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:26:43 smithi037 ceph-mon[142325]: from='client.34900 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:26:43.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:26:43 smithi037 ceph-mon[142325]: from='client.34904 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:26:43.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:26:43 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/3517284242' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:26:43.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:26:43 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/4272439088' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T08:26:43.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:43.548+0000 7f04a7000700 1 -- 172.21.15.37:0/2539602325 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f04a01113d0 con 0x7f048406cc30 2024-03-29T08:26:43.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:43.550+0000 7f0497e00700 1 -- 172.21.15.37:0/2539602325 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f04a01113d0 con 0x7f048406cc30 2024-03-29T08:26:43.550 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-29T08:26:43.550 INFO:teuthology.orchestra.run.smithi037.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59", 2024-03-29T08:26:43.550 INFO:teuthology.orchestra.run.smithi037.stdout: "in_progress": true, 2024-03-29T08:26:43.550 INFO:teuthology.orchestra.run.smithi037.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-29T08:26:43.550 INFO:teuthology.orchestra.run.smithi037.stdout: "services_complete": [ 2024-03-29T08:26:43.550 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr", 2024-03-29T08:26:43.550 INFO:teuthology.orchestra.run.smithi037.stdout: "mon", 2024-03-29T08:26:43.550 INFO:teuthology.orchestra.run.smithi037.stdout: "crash", 2024-03-29T08:26:43.550 INFO:teuthology.orchestra.run.smithi037.stdout: "osd" 2024-03-29T08:26:43.550 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-29T08:26:43.550 INFO:teuthology.orchestra.run.smithi037.stdout: "progress": "15/21 daemons upgraded", 2024-03-29T08:26:43.551 INFO:teuthology.orchestra.run.smithi037.stdout: "message": "Currently upgrading mds daemons", 2024-03-29T08:26:43.551 INFO:teuthology.orchestra.run.smithi037.stdout: "is_paused": false 2024-03-29T08:26:43.551 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-29T08:26:43.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:43.552+0000 7f04a7000700 1 -- 172.21.15.37:0/2539602325 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f048406cc30 msgr2=0x7f048406f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:43.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:43.552+0000 7f04a7000700 1 --2- 172.21.15.37:0/2539602325 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f048406cc30 0x7f048406f0e0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f0490004f40 tx=0x7f049000b040 comp rx=0 tx=0).stop 2024-03-29T08:26:43.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:43.552+0000 7f04a7000700 1 -- 172.21.15.37:0/2539602325 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f04a00fe830 msgr2=0x7f04a0116250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:43.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:43.552+0000 7f04a7000700 1 --2- 172.21.15.37:0/2539602325 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f04a00fe830 0x7f04a0116250 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f048c000c00 tx=0x7f048c009e30 comp rx=0 tx=0).stop 2024-03-29T08:26:43.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:43.552+0000 7f04a7000700 1 -- 172.21.15.37:0/2539602325 shutdown_connections 2024-03-29T08:26:43.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:43.552+0000 7f04a7000700 1 --2- 172.21.15.37:0/2539602325 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f048406cc30 0x7f048406f0e0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:43.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:43.552+0000 7f04a7000700 1 --2- 172.21.15.37:0/2539602325 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f04a00fde90 0x7f04a0115d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:43.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:43.552+0000 7f04a7000700 1 --2- 172.21.15.37:0/2539602325 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f04a00fe830 0x7f04a0116250 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:43.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:43.552+0000 7f04a7000700 1 -- 172.21.15.37:0/2539602325 >> 172.21.15.37:0/2539602325 conn(0x7f04a0072bc0 msgr2=0x7f04a00fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:26:43.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:43.552+0000 7f04a7000700 1 -- 172.21.15.37:0/2539602325 shutdown_connections 2024-03-29T08:26:43.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:43.552+0000 7f04a7000700 1 -- 172.21.15.37:0/2539602325 wait complete. 2024-03-29T08:26:43.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:43.690+0000 7f9a5dc00700 1 -- 172.21.15.37:0/3059142649 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9a581037c0 msgr2=0x7f9a580fce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:43.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:43.690+0000 7f9a5dc00700 1 --2- 172.21.15.37:0/3059142649 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9a581037c0 0x7f9a580fce30 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f9a44004940 tx=0x7f9a44009e30 comp rx=0 tx=0).stop 2024-03-29T08:26:43.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:43.690+0000 7f9a5dc00700 1 -- 172.21.15.37:0/3059142649 shutdown_connections 2024-03-29T08:26:43.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:43.690+0000 7f9a5dc00700 1 --2- 172.21.15.37:0/3059142649 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9a581037c0 0x7f9a580fce30 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:43.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:43.690+0000 7f9a5dc00700 1 --2- 172.21.15.37:0/3059142649 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9a58102e20 0x7f9a581031f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:43.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:43.690+0000 7f9a5dc00700 1 -- 172.21.15.37:0/3059142649 >> 172.21.15.37:0/3059142649 conn(0x7f9a580f8970 msgr2=0x7f9a580fad80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:26:43.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:43.690+0000 7f9a5dc00700 1 -- 172.21.15.37:0/3059142649 shutdown_connections 2024-03-29T08:26:43.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:43.690+0000 7f9a5dc00700 1 -- 172.21.15.37:0/3059142649 wait complete. 2024-03-29T08:26:43.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:43.691+0000 7f9a5dc00700 1 Processor -- start 2024-03-29T08:26:43.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:43.691+0000 7f9a5dc00700 1 -- start start 2024-03-29T08:26:43.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:43.691+0000 7f9a5dc00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9a58102e20 0x7f9a58115c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:43.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:43.691+0000 7f9a5dc00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9a581037c0 0x7f9a58116180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:43.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:43.691+0000 7f9a5dc00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a581167d0 con 0x7f9a581037c0 2024-03-29T08:26:43.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:43.691+0000 7f9a5dc00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a5810fba0 con 0x7f9a58102e20 2024-03-29T08:26:43.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:43.692+0000 7f9a56000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9a581037c0 0x7f9a58116180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:43.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:43.692+0000 7f9a56000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9a581037c0 0x7f9a58116180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44644/0 (socket says 172.21.15.37:44644) 2024-03-29T08:26:43.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:43.692+0000 7f9a56000700 1 -- 172.21.15.37:0/1918497176 learned_addr learned my addr 172.21.15.37:0/1918497176 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:26:43.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:43.692+0000 7f9a56a00700 1 --2- 172.21.15.37:0/1918497176 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9a58102e20 0x7f9a58115c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:43.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:43.692+0000 7f9a56000700 1 -- 172.21.15.37:0/1918497176 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9a58102e20 msgr2=0x7f9a58115c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:43.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:43.692+0000 7f9a56000700 1 --2- 172.21.15.37:0/1918497176 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9a58102e20 0x7f9a58115c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:43.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:43.692+0000 7f9a56000700 1 -- 172.21.15.37:0/1918497176 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a44005040 con 0x7f9a581037c0 2024-03-29T08:26:43.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:43.692+0000 7f9a56a00700 1 --2- 172.21.15.37:0/1918497176 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9a58102e20 0x7f9a58115c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:26:43.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:43.692+0000 7f9a56000700 1 --2- 172.21.15.37:0/1918497176 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9a581037c0 0x7f9a58116180 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f9a44009fd0 tx=0x7f9a44009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:26:43.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:43.693+0000 7f9a4ea00700 1 -- 172.21.15.37:0/1918497176 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9a4400e030 con 0x7f9a581037c0 2024-03-29T08:26:43.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:43.693+0000 7f9a4ea00700 1 -- 172.21.15.37:0/1918497176 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f9a44013e00 con 0x7f9a581037c0 2024-03-29T08:26:43.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:43.693+0000 7f9a4ea00700 1 -- 172.21.15.37:0/1918497176 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9a4401ccd0 con 0x7f9a581037c0 2024-03-29T08:26:43.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:43.693+0000 7f9a5dc00700 1 -- 172.21.15.37:0/1918497176 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a5810fe20 con 0x7f9a581037c0 2024-03-29T08:26:43.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:43.693+0000 7f9a5dc00700 1 -- 172.21.15.37:0/1918497176 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a581103e0 con 0x7f9a581037c0 2024-03-29T08:26:43.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:43.694+0000 7f9a5dc00700 1 -- 172.21.15.37:0/1918497176 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a580fe570 con 0x7f9a581037c0 2024-03-29T08:26:43.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:43.695+0000 7f9a4ea00700 1 -- 172.21.15.37:0/1918497176 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f9a4401c500 con 0x7f9a581037c0 2024-03-29T08:26:43.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:43.696+0000 7f9a4ea00700 1 --2- 172.21.15.37:0/1918497176 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f9a3c06cc30 0x7f9a3c06f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:43.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:43.696+0000 7f9a4ea00700 1 -- 172.21.15.37:0/1918497176 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f9a4408d8a0 con 0x7f9a581037c0 2024-03-29T08:26:43.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:43.696+0000 7f9a56a00700 1 --2- 172.21.15.37:0/1918497176 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f9a3c06cc30 0x7f9a3c06f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:43.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:43.697+0000 7f9a56a00700 1 --2- 172.21.15.37:0/1918497176 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f9a3c06cc30 0x7f9a3c06f0e0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f9a48009fd0 tx=0x7f9a4800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:26:43.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:43.700+0000 7f9a4ea00700 1 -- 172.21.15.37:0/1918497176 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f9a4405b370 con 0x7f9a581037c0 2024-03-29T08:26:43.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:26:43 smithi148 ceph-mon[106692]: pgmap v474: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:26:43.807 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:26:43 smithi148 ceph-mon[106692]: from='client.34900 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:26:43.807 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:26:43 smithi148 ceph-mon[106692]: from='client.34904 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:26:43.807 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:26:43 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/3517284242' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:26:43.807 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:26:43 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/4272439088' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T08:26:44.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:44.000+0000 7f9a5dc00700 1 -- 172.21.15.37:0/1918497176 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f9a5805f5f0 con 0x7f9a581037c0 2024-03-29T08:26:44.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:44.000+0000 7f9a4ea00700 1 -- 172.21.15.37:0/1918497176 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7f9a4405af00 con 0x7f9a581037c0 2024-03-29T08:26:44.000 INFO:teuthology.orchestra.run.smithi037.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-03-29T08:26:44.000 INFO:teuthology.orchestra.run.smithi037.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-03-29T08:26:44.000 INFO:teuthology.orchestra.run.smithi037.stdout: fs cephfs is degraded 2024-03-29T08:26:44.000 INFO:teuthology.orchestra.run.smithi037.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-03-29T08:26:44.001 INFO:teuthology.orchestra.run.smithi037.stdout: fs cephfs is offline because no MDS is active for it. 2024-03-29T08:26:44.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:44.002+0000 7f9a5dc00700 1 -- 172.21.15.37:0/1918497176 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f9a3c06cc30 msgr2=0x7f9a3c06f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:44.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:44.002+0000 7f9a5dc00700 1 --2- 172.21.15.37:0/1918497176 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f9a3c06cc30 0x7f9a3c06f0e0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f9a48009fd0 tx=0x7f9a4800b040 comp rx=0 tx=0).stop 2024-03-29T08:26:44.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:44.002+0000 7f9a5dc00700 1 -- 172.21.15.37:0/1918497176 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9a581037c0 msgr2=0x7f9a58116180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:44.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:44.002+0000 7f9a5dc00700 1 --2- 172.21.15.37:0/1918497176 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9a581037c0 0x7f9a58116180 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f9a44009fd0 tx=0x7f9a44009e30 comp rx=0 tx=0).stop 2024-03-29T08:26:44.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:44.002+0000 7f9a5dc00700 1 -- 172.21.15.37:0/1918497176 shutdown_connections 2024-03-29T08:26:44.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:44.002+0000 7f9a5dc00700 1 --2- 172.21.15.37:0/1918497176 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f9a3c06cc30 0x7f9a3c06f0e0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:44.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:44.002+0000 7f9a5dc00700 1 --2- 172.21.15.37:0/1918497176 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9a58102e20 0x7f9a58115c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:44.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:44.002+0000 7f9a5dc00700 1 --2- 172.21.15.37:0/1918497176 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9a581037c0 0x7f9a58116180 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:44.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:44.002+0000 7f9a5dc00700 1 -- 172.21.15.37:0/1918497176 >> 172.21.15.37:0/1918497176 conn(0x7f9a580f8970 msgr2=0x7f9a580f9990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:26:44.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:44.003+0000 7f9a5dc00700 1 -- 172.21.15.37:0/1918497176 shutdown_connections 2024-03-29T08:26:44.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:26:44.003+0000 7f9a5dc00700 1 -- 172.21.15.37:0/1918497176 wait complete. 2024-03-29T08:26:44.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:26:44 smithi037 ceph-mon[142325]: from='client.34916 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:26:44.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:26:44 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/1918497176' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T08:26:44.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:26:44 smithi148 ceph-mon[106692]: from='client.34916 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:26:44.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:26:44 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/1918497176' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T08:26:45.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:26:45 smithi037 ceph-mon[142325]: pgmap v475: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:26:45.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:26:45 smithi148 ceph-mon[106692]: pgmap v475: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:26:47.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:26:47 smithi037 ceph-mon[142325]: pgmap v476: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:26:47.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:26:47 smithi148 ceph-mon[106692]: pgmap v476: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:26:49.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:26:49 smithi037 ceph-mon[142325]: pgmap v477: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:26:49.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:26:49 smithi148 ceph-mon[106692]: pgmap v477: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:26:51.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:26:51 smithi037 ceph-mon[142325]: pgmap v478: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:26:51.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:26:51 smithi148 ceph-mon[106692]: pgmap v478: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:26:53.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:26:53 smithi037 ceph-mon[142325]: pgmap v479: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:26:53.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:26:53 smithi148 ceph-mon[106692]: pgmap v479: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:26:55.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:26:55 smithi037 ceph-mon[142325]: pgmap v480: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:26:55.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:26:55 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:26:55.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:26:55 smithi148 ceph-mon[106692]: pgmap v480: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:26:55.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:26:55 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:26:57.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:26:57 smithi037 ceph-mon[142325]: pgmap v481: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:26:57.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:26:57 smithi148 ceph-mon[106692]: pgmap v481: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:26:59.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:26:59 smithi037 ceph-mon[142325]: pgmap v482: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:26:59.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:26:59 smithi148 ceph-mon[106692]: pgmap v482: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:27:01.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:27:01 smithi037 ceph-mon[142325]: pgmap v483: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:27:01.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:27:01 smithi148 ceph-mon[106692]: pgmap v483: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:27:03.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:27:03 smithi037 ceph-mon[142325]: pgmap v484: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:27:03.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:27:03 smithi148 ceph-mon[106692]: pgmap v484: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:27:05.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:27:05 smithi037 ceph-mon[142325]: pgmap v485: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:27:05.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:27:05 smithi148 ceph-mon[106692]: pgmap v485: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:27:07.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:27:07 smithi037 ceph-mon[142325]: pgmap v486: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:27:07.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:27:07 smithi148 ceph-mon[106692]: pgmap v486: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:27:09.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:27:09 smithi037 ceph-mon[142325]: pgmap v487: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:27:09.807 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:27:09 smithi148 ceph-mon[106692]: pgmap v487: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:27:10.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:27:10 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:27:10.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:27:10 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:27:11.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:27:11 smithi037 ceph-mon[142325]: pgmap v488: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:27:11.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:27:11 smithi148 ceph-mon[106692]: pgmap v488: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:27:13.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:27:13 smithi148 ceph-mon[106692]: pgmap v489: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:27:13.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:27:13 smithi037 ceph-mon[142325]: pgmap v489: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:27:14.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:14.154+0000 7f8806c00700 1 -- 172.21.15.37:0/727783542 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8800070c10 msgr2=0x7f8800109190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:14.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:14.154+0000 7f8806c00700 1 --2- 172.21.15.37:0/727783542 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8800070c10 0x7f8800109190 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f87ec004970 tx=0x7f87ec009e30 comp rx=0 tx=0).stop 2024-03-29T08:27:14.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:14.154+0000 7f8806c00700 1 -- 172.21.15.37:0/727783542 shutdown_connections 2024-03-29T08:27:14.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:14.154+0000 7f8806c00700 1 --2- 172.21.15.37:0/727783542 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8800070c10 0x7f8800109190 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:14.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:14.154+0000 7f8806c00700 1 --2- 172.21.15.37:0/727783542 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8800070300 0x7f88000706d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:14.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:14.154+0000 7f8806c00700 1 -- 172.21.15.37:0/727783542 >> 172.21.15.37:0/727783542 conn(0x7f88000f8970 msgr2=0x7f88000fad80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:27:14.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:14.155+0000 7f8806c00700 1 -- 172.21.15.37:0/727783542 shutdown_connections 2024-03-29T08:27:14.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:14.155+0000 7f8806c00700 1 -- 172.21.15.37:0/727783542 wait complete. 2024-03-29T08:27:14.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:14.155+0000 7f8806c00700 1 Processor -- start 2024-03-29T08:27:14.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:14.156+0000 7f8806c00700 1 -- start start 2024-03-29T08:27:14.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:14.156+0000 7f8806c00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8800070300 0x7f8800115d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:14.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:14.156+0000 7f8806c00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8800070c10 0x7f88001162c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:14.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:14.156+0000 7f8806c00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88001169a0 con 0x7f8800070300 2024-03-29T08:27:14.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:14.156+0000 7f8806c00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f880010fce0 con 0x7f8800070c10 2024-03-29T08:27:14.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:14.156+0000 7f87ffe00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8800070300 0x7f8800115d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:14.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:14.156+0000 7f87ffe00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8800070300 0x7f8800115d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60470/0 (socket says 172.21.15.37:60470) 2024-03-29T08:27:14.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:14.156+0000 7f87ffe00700 1 -- 172.21.15.37:0/2544981315 learned_addr learned my addr 172.21.15.37:0/2544981315 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:27:14.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:14.157+0000 7f87ff400700 1 --2- 172.21.15.37:0/2544981315 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8800070c10 0x7f88001162c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:14.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:14.157+0000 7f87ffe00700 1 -- 172.21.15.37:0/2544981315 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8800070c10 msgr2=0x7f88001162c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:14.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:14.157+0000 7f87ffe00700 1 --2- 172.21.15.37:0/2544981315 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8800070c10 0x7f88001162c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:14.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:14.157+0000 7f87ffe00700 1 -- 172.21.15.37:0/2544981315 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87ec005040 con 0x7f8800070300 2024-03-29T08:27:14.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:14.157+0000 7f87ffe00700 1 --2- 172.21.15.37:0/2544981315 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8800070300 0x7f8800115d80 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f87f000b790 tx=0x7f87f0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:14.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:14.157+0000 7f87f7e00700 1 -- 172.21.15.37:0/2544981315 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f87f000f6f0 con 0x7f8800070300 2024-03-29T08:27:14.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:14.157+0000 7f87f7e00700 1 -- 172.21.15.37:0/2544981315 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f87f000fd00 con 0x7f8800070300 2024-03-29T08:27:14.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:14.157+0000 7f87f7e00700 1 -- 172.21.15.37:0/2544981315 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f87f00103a0 con 0x7f8800070300 2024-03-29T08:27:14.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:14.157+0000 7f8806c00700 1 -- 172.21.15.37:0/2544981315 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f880010ff60 con 0x7f8800070300 2024-03-29T08:27:14.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:14.158+0000 7f8806c00700 1 -- 172.21.15.37:0/2544981315 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8800110470 con 0x7f8800070300 2024-03-29T08:27:14.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:14.159+0000 7f8806c00700 1 -- 172.21.15.37:0/2544981315 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8800071a40 con 0x7f8800070300 2024-03-29T08:27:14.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:14.160+0000 7f87f7e00700 1 -- 172.21.15.37:0/2544981315 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f87f000f850 con 0x7f8800070300 2024-03-29T08:27:14.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:14.161+0000 7f87f7e00700 1 --2- 172.21.15.37:0/2544981315 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f87e406cc30 0x7f87e406f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:14.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:14.161+0000 7f87f7e00700 1 -- 172.21.15.37:0/2544981315 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f87f008be60 con 0x7f8800070300 2024-03-29T08:27:14.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:14.161+0000 7f87ff400700 1 --2- 172.21.15.37:0/2544981315 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f87e406cc30 0x7f87e406f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:14.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:14.161+0000 7f87ff400700 1 --2- 172.21.15.37:0/2544981315 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f87e406cc30 0x7f87e406f0e0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f87ec009fd0 tx=0x7f87ec009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:14.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:14.165+0000 7f87f7e00700 1 -- 172.21.15.37:0/2544981315 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f87f0059930 con 0x7f8800070300 2024-03-29T08:27:14.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:14.397+0000 7f8806c00700 1 -- 172.21.15.37:0/2544981315 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f880004bf20 con 0x7f87e406cc30 2024-03-29T08:27:14.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:14.399+0000 7f87f7e00700 1 -- 172.21.15.37:0/2544981315 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f880004bf20 con 0x7f87e406cc30 2024-03-29T08:27:14.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:14.401+0000 7f8806c00700 1 -- 172.21.15.37:0/2544981315 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f87e406cc30 msgr2=0x7f87e406f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:14.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:14.401+0000 7f8806c00700 1 --2- 172.21.15.37:0/2544981315 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f87e406cc30 0x7f87e406f0e0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f87ec009fd0 tx=0x7f87ec009e30 comp rx=0 tx=0).stop 2024-03-29T08:27:14.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:14.401+0000 7f8806c00700 1 -- 172.21.15.37:0/2544981315 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8800070300 msgr2=0x7f8800115d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:14.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:14.401+0000 7f8806c00700 1 --2- 172.21.15.37:0/2544981315 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8800070300 0x7f8800115d80 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f87f000b790 tx=0x7f87f0009250 comp rx=0 tx=0).stop 2024-03-29T08:27:14.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:14.401+0000 7f8806c00700 1 -- 172.21.15.37:0/2544981315 shutdown_connections 2024-03-29T08:27:14.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:14.401+0000 7f8806c00700 1 --2- 172.21.15.37:0/2544981315 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f87e406cc30 0x7f87e406f0e0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:14.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:14.401+0000 7f8806c00700 1 --2- 172.21.15.37:0/2544981315 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8800070300 0x7f8800115d80 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:14.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:14.401+0000 7f8806c00700 1 --2- 172.21.15.37:0/2544981315 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8800070c10 0x7f88001162c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:14.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:14.401+0000 7f8806c00700 1 -- 172.21.15.37:0/2544981315 >> 172.21.15.37:0/2544981315 conn(0x7f88000f8970 msgr2=0x7f88001053c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:27:14.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:14.402+0000 7f8806c00700 1 -- 172.21.15.37:0/2544981315 shutdown_connections 2024-03-29T08:27:14.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:14.402+0000 7f8806c00700 1 -- 172.21.15.37:0/2544981315 wait complete. 2024-03-29T08:27:14.417 INFO:teuthology.orchestra.run.smithi037.stdout:true 2024-03-29T08:27:14.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:14.542+0000 7f2d9ae00700 1 -- 172.21.15.37:0/2620710456 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2d940fde90 msgr2=0x7f2d940fe260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:14.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:14.542+0000 7f2d9ae00700 1 --2- 172.21.15.37:0/2620710456 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2d940fde90 0x7f2d940fe260 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f2d84004940 tx=0x7f2d84009e30 comp rx=0 tx=0).stop 2024-03-29T08:27:14.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:14.542+0000 7f2d9ae00700 1 -- 172.21.15.37:0/2620710456 shutdown_connections 2024-03-29T08:27:14.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:14.542+0000 7f2d9ae00700 1 --2- 172.21.15.37:0/2620710456 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2d940fe830 0x7f2d94106f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:14.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:14.542+0000 7f2d9ae00700 1 --2- 172.21.15.37:0/2620710456 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2d940fde90 0x7f2d940fe260 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:14.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:14.542+0000 7f2d9ae00700 1 -- 172.21.15.37:0/2620710456 >> 172.21.15.37:0/2620710456 conn(0x7f2d94072bc0 msgr2=0x7f2d94072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:27:14.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:14.542+0000 7f2d9ae00700 1 -- 172.21.15.37:0/2620710456 shutdown_connections 2024-03-29T08:27:14.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:14.542+0000 7f2d9ae00700 1 -- 172.21.15.37:0/2620710456 wait complete. 2024-03-29T08:27:14.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:14.543+0000 7f2d9ae00700 1 Processor -- start 2024-03-29T08:27:14.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:14.543+0000 7f2d9ae00700 1 -- start start 2024-03-29T08:27:14.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:14.543+0000 7f2d9ae00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2d940fde90 0x7f2d94115d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:14.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:14.543+0000 7f2d9ae00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2d940fe830 0x7f2d94116260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:14.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:14.543+0000 7f2d9ae00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d941168b0 con 0x7f2d940fe830 2024-03-29T08:27:14.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:14.543+0000 7f2d9ae00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d9410fc80 con 0x7f2d940fde90 2024-03-29T08:27:14.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:14.544+0000 7f2d93400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2d940fe830 0x7f2d94116260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:14.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:14.544+0000 7f2d93400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2d940fe830 0x7f2d94116260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60486/0 (socket says 172.21.15.37:60486) 2024-03-29T08:27:14.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:14.544+0000 7f2d93400700 1 -- 172.21.15.37:0/2366536748 learned_addr learned my addr 172.21.15.37:0/2366536748 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:27:14.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:14.544+0000 7f2d93400700 1 -- 172.21.15.37:0/2366536748 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2d940fde90 msgr2=0x7f2d94115d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:14.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:14.544+0000 7f2d93400700 1 --2- 172.21.15.37:0/2366536748 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2d940fde90 0x7f2d94115d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:14.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:14.544+0000 7f2d93400700 1 -- 172.21.15.37:0/2366536748 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d84005040 con 0x7f2d940fe830 2024-03-29T08:27:14.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:14.544+0000 7f2d93400700 1 --2- 172.21.15.37:0/2366536748 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2d940fe830 0x7f2d94116260 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f2d8000c8c0 tx=0x7f2d800044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:14.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:14.545+0000 7f2d8be00700 1 -- 172.21.15.37:0/2366536748 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2d8000a980 con 0x7f2d940fe830 2024-03-29T08:27:14.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:14.545+0000 7f2d8be00700 1 -- 172.21.15.37:0/2366536748 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f2d8000ce60 con 0x7f2d940fe830 2024-03-29T08:27:14.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:14.545+0000 7f2d9ae00700 1 -- 172.21.15.37:0/2366536748 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d9410ff60 con 0x7f2d940fe830 2024-03-29T08:27:14.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:14.545+0000 7f2d8be00700 1 -- 172.21.15.37:0/2366536748 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2d800113a0 con 0x7f2d940fe830 2024-03-29T08:27:14.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:14.545+0000 7f2d9ae00700 1 -- 172.21.15.37:0/2366536748 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d941104a0 con 0x7f2d940fe830 2024-03-29T08:27:14.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:14.546+0000 7f2d9ae00700 1 -- 172.21.15.37:0/2366536748 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d94106a20 con 0x7f2d940fe830 2024-03-29T08:27:14.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:14.547+0000 7f2d8be00700 1 -- 172.21.15.37:0/2366536748 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f2d80012400 con 0x7f2d940fe830 2024-03-29T08:27:14.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:14.548+0000 7f2d8be00700 1 --2- 172.21.15.37:0/2366536748 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f2d7806cc30 0x7f2d7806f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:14.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:14.548+0000 7f2d8be00700 1 -- 172.21.15.37:0/2366536748 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f2d8008c190 con 0x7f2d940fe830 2024-03-29T08:27:14.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:14.548+0000 7f2d93e00700 1 --2- 172.21.15.37:0/2366536748 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f2d7806cc30 0x7f2d7806f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:14.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:14.549+0000 7f2d93e00700 1 --2- 172.21.15.37:0/2366536748 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f2d7806cc30 0x7f2d7806f0e0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f2d84004f90 tx=0x7f2d84009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:14.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:14.553+0000 7f2d8be00700 1 -- 172.21.15.37:0/2366536748 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f2d80059c20 con 0x7f2d940fe830 2024-03-29T08:27:14.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:14.785+0000 7f2d9ae00700 1 -- 172.21.15.37:0/2366536748 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f2d94111370 con 0x7f2d7806cc30 2024-03-29T08:27:14.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:14.787+0000 7f2d8be00700 1 -- 172.21.15.37:0/2366536748 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f2d94111370 con 0x7f2d7806cc30 2024-03-29T08:27:14.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:14.789+0000 7f2d9ae00700 1 -- 172.21.15.37:0/2366536748 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f2d7806cc30 msgr2=0x7f2d7806f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:14.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:14.789+0000 7f2d9ae00700 1 --2- 172.21.15.37:0/2366536748 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f2d7806cc30 0x7f2d7806f0e0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f2d84004f90 tx=0x7f2d84009e30 comp rx=0 tx=0).stop 2024-03-29T08:27:14.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:14.789+0000 7f2d9ae00700 1 -- 172.21.15.37:0/2366536748 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2d940fe830 msgr2=0x7f2d94116260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:14.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:14.789+0000 7f2d9ae00700 1 --2- 172.21.15.37:0/2366536748 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2d940fe830 0x7f2d94116260 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f2d8000c8c0 tx=0x7f2d800044c0 comp rx=0 tx=0).stop 2024-03-29T08:27:14.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:14.789+0000 7f2d9ae00700 1 -- 172.21.15.37:0/2366536748 shutdown_connections 2024-03-29T08:27:14.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:14.789+0000 7f2d9ae00700 1 --2- 172.21.15.37:0/2366536748 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f2d7806cc30 0x7f2d7806f0e0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:14.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:14.789+0000 7f2d9ae00700 1 --2- 172.21.15.37:0/2366536748 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2d940fde90 0x7f2d94115d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:14.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:14.789+0000 7f2d9ae00700 1 --2- 172.21.15.37:0/2366536748 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2d940fe830 0x7f2d94116260 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:14.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:14.789+0000 7f2d9ae00700 1 -- 172.21.15.37:0/2366536748 >> 172.21.15.37:0/2366536748 conn(0x7f2d94072bc0 msgr2=0x7f2d940fb8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:27:14.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:14.790+0000 7f2d9ae00700 1 -- 172.21.15.37:0/2366536748 shutdown_connections 2024-03-29T08:27:14.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:14.790+0000 7f2d9ae00700 1 -- 172.21.15.37:0/2366536748 wait complete. 2024-03-29T08:27:14.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:14.926+0000 7f21a8a00700 1 -- 172.21.15.37:0/72766695 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f21a40fde90 msgr2=0x7f21a40fe260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:14.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:14.926+0000 7f21a8a00700 1 --2- 172.21.15.37:0/72766695 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f21a40fde90 0x7f21a40fe260 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f2194004940 tx=0x7f2194009e30 comp rx=0 tx=0).stop 2024-03-29T08:27:14.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:14.926+0000 7f21a8a00700 1 -- 172.21.15.37:0/72766695 shutdown_connections 2024-03-29T08:27:14.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:14.926+0000 7f21a8a00700 1 --2- 172.21.15.37:0/72766695 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f21a40fe830 0x7f21a4106f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:14.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:14.926+0000 7f21a8a00700 1 --2- 172.21.15.37:0/72766695 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f21a40fde90 0x7f21a40fe260 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:14.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:14.926+0000 7f21a8a00700 1 -- 172.21.15.37:0/72766695 >> 172.21.15.37:0/72766695 conn(0x7f21a4072bc0 msgr2=0x7f21a4072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:27:14.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:14.926+0000 7f21a8a00700 1 -- 172.21.15.37:0/72766695 shutdown_connections 2024-03-29T08:27:14.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:14.927+0000 7f21a8a00700 1 -- 172.21.15.37:0/72766695 wait complete. 2024-03-29T08:27:14.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:14.927+0000 7f21a8a00700 1 Processor -- start 2024-03-29T08:27:14.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:14.927+0000 7f21a8a00700 1 -- start start 2024-03-29T08:27:14.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:14.928+0000 7f21a8a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f21a40fde90 0x7f21a4115d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:14.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:14.928+0000 7f21a8a00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f21a40fe830 0x7f21a4116240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:14.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:14.928+0000 7f21a8a00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21a4116920 con 0x7f21a40fde90 2024-03-29T08:27:14.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:14.928+0000 7f21a8a00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21a410fc60 con 0x7f21a40fe830 2024-03-29T08:27:14.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:14.928+0000 7f21a1e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f21a40fde90 0x7f21a4115d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:14.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:14.928+0000 7f21a1e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f21a40fde90 0x7f21a4115d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60500/0 (socket says 172.21.15.37:60500) 2024-03-29T08:27:14.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:14.928+0000 7f21a1e00700 1 -- 172.21.15.37:0/2055579663 learned_addr learned my addr 172.21.15.37:0/2055579663 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:27:14.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:14.928+0000 7f21a1e00700 1 -- 172.21.15.37:0/2055579663 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f21a40fe830 msgr2=0x7f21a4116240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:14.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:14.928+0000 7f21a1e00700 1 --2- 172.21.15.37:0/2055579663 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f21a40fe830 0x7f21a4116240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:14.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:14.928+0000 7f21a1e00700 1 -- 172.21.15.37:0/2055579663 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2194005040 con 0x7f21a40fde90 2024-03-29T08:27:14.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:14.929+0000 7f21a1e00700 1 --2- 172.21.15.37:0/2055579663 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f21a40fde90 0x7f21a4115d00 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f2194000c00 tx=0x7f2194009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:14.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:14.929+0000 7f219a000700 1 -- 172.21.15.37:0/2055579663 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f219400e040 con 0x7f21a40fde90 2024-03-29T08:27:14.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:14.929+0000 7f219a000700 1 -- 172.21.15.37:0/2055579663 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f2194013420 con 0x7f21a40fde90 2024-03-29T08:27:14.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:14.929+0000 7f21a8a00700 1 -- 172.21.15.37:0/2055579663 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21a410fe80 con 0x7f21a40fde90 2024-03-29T08:27:14.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:14.929+0000 7f219a000700 1 -- 172.21.15.37:0/2055579663 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f219401c630 con 0x7f21a40fde90 2024-03-29T08:27:14.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:14.929+0000 7f21a8a00700 1 -- 172.21.15.37:0/2055579663 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21a41103c0 con 0x7f21a40fde90 2024-03-29T08:27:14.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:14.930+0000 7f21a8a00700 1 -- 172.21.15.37:0/2055579663 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21a4106a20 con 0x7f21a40fde90 2024-03-29T08:27:14.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:14.931+0000 7f219a000700 1 -- 172.21.15.37:0/2055579663 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f2194013970 con 0x7f21a40fde90 2024-03-29T08:27:14.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:14.932+0000 7f219a000700 1 --2- 172.21.15.37:0/2055579663 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f218806cc30 0x7f218806f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:14.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:14.932+0000 7f219a000700 1 -- 172.21.15.37:0/2055579663 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f219408d2e0 con 0x7f21a40fde90 2024-03-29T08:27:14.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:14.932+0000 7f21a1400700 1 --2- 172.21.15.37:0/2055579663 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f218806cc30 0x7f218806f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:14.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:14.933+0000 7f21a1400700 1 --2- 172.21.15.37:0/2055579663 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f218806cc30 0x7f218806f0e0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f2190006fd0 tx=0x7f2190006db0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:14.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:14.937+0000 7f219a000700 1 -- 172.21.15.37:0/2055579663 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f219405ad30 con 0x7f21a40fde90 2024-03-29T08:27:15.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:15.168+0000 7f21a8a00700 1 -- 172.21.15.37:0/2055579663 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f21a4109e40 con 0x7f218806cc30 2024-03-29T08:27:15.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:15.178+0000 7f219a000700 1 -- 172.21.15.37:0/2055579663 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f21a4109e40 con 0x7f218806cc30 2024-03-29T08:27:15.179 INFO:teuthology.orchestra.run.smithi037.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-29T08:27:15.179 INFO:teuthology.orchestra.run.smithi037.stdout:alertmanager.smithi037 smithi037 *:9093,9094 running (17m) 9m ago 30m 14.9M - 0.25.0 c8568f914cd2 2f9f9746ed0f 2024-03-29T08:27:15.179 INFO:teuthology.orchestra.run.smithi037.stdout:crash.smithi037 smithi037 running (13m) 9m ago 30m 6647k - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 178d416a3186 2024-03-29T08:27:15.179 INFO:teuthology.orchestra.run.smithi037.stdout:crash.smithi148 smithi148 running (13m) 9m ago 29m 6656k - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 23666989d58f 2024-03-29T08:27:15.179 INFO:teuthology.orchestra.run.smithi037.stdout:grafana.smithi037 smithi037 *:3000 running (16m) 9m ago 29m 71.9M - 9.4.7 954c08fa6188 6b976fe49952 2024-03-29T08:27:15.179 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi037.iifsmh smithi037 running (9m) 9m ago 23m 10.8M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 2d97f562cc5b 2024-03-29T08:27:15.179 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi037.mmwgxe smithi037 running (9m) 9m ago 23m 10.9M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff ec37037f46e8 2024-03-29T08:27:15.180 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi148.qukpjh smithi148 running (23m) 9m ago 23m 15.3M - 17.2.7-789-g4213e17c cf22eb8683be 8e469782a273 2024-03-29T08:27:15.180 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi148.sqteih smithi148 running (9m) 9m ago 23m 10.7M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 44c15c77037a 2024-03-29T08:27:15.180 INFO:teuthology.orchestra.run.smithi037.stdout:mgr.smithi037.wjikii smithi037 *:8443,9283,8765 running (19m) 9m ago 31m 506M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff c5a1417b9417 2024-03-29T08:27:15.180 INFO:teuthology.orchestra.run.smithi037.stdout:mgr.smithi148.onhgvm smithi148 *:8443,9283,8765 running (18m) 9m ago 28m 450M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff fcfdbce062bd 2024-03-29T08:27:15.180 INFO:teuthology.orchestra.run.smithi037.stdout:mon.smithi037 smithi037 running (15m) 9m ago 31m 54.8M 2048M 18.2.2-1015-gd0e392d9 1cc2f45c17ff eea2060783af 2024-03-29T08:27:15.180 INFO:teuthology.orchestra.run.smithi037.stdout:mon.smithi148 smithi148 running (14m) 9m ago 28m 43.7M 2048M 18.2.2-1015-gd0e392d9 1cc2f45c17ff ad4b2c5748be 2024-03-29T08:27:15.180 INFO:teuthology.orchestra.run.smithi037.stdout:node-exporter.smithi037 smithi037 *:9100 running (17m) 9m ago 30m 7625k - 1.5.0 0da6a335fe13 26b8ae9cb878 2024-03-29T08:27:15.180 INFO:teuthology.orchestra.run.smithi037.stdout:node-exporter.smithi148 smithi148 *:9100 running (17m) 9m ago 28m 8166k - 1.5.0 0da6a335fe13 98414e4e64ed 2024-03-29T08:27:15.180 INFO:teuthology.orchestra.run.smithi037.stdout:osd.0 smithi037 running (13m) 9m ago 27m 104M 1619M 18.2.2-1015-gd0e392d9 1cc2f45c17ff c0c584f6245c 2024-03-29T08:27:15.180 INFO:teuthology.orchestra.run.smithi037.stdout:osd.1 smithi037 running (12m) 9m ago 27m 100M 1619M 18.2.2-1015-gd0e392d9 1cc2f45c17ff a4a2ff4cc83d 2024-03-29T08:27:15.180 INFO:teuthology.orchestra.run.smithi037.stdout:osd.2 smithi037 running (12m) 9m ago 26m 82.3M 1619M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 8b1bc71aa79e 2024-03-29T08:27:15.180 INFO:teuthology.orchestra.run.smithi037.stdout:osd.3 smithi148 running (11m) 9m ago 26m 105M 2642M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 1c656ab8ff86 2024-03-29T08:27:15.180 INFO:teuthology.orchestra.run.smithi037.stdout:osd.4 smithi148 running (10m) 9m ago 25m 97.3M 2642M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 966a9de11d22 2024-03-29T08:27:15.180 INFO:teuthology.orchestra.run.smithi037.stdout:osd.5 smithi148 running (10m) 9m ago 25m 85.8M 2642M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 474c4f9212ce 2024-03-29T08:27:15.180 INFO:teuthology.orchestra.run.smithi037.stdout:prometheus.smithi037 smithi037 *:9095 running (17m) 9m ago 29m 54.4M - 2.43.0 a07b618ecd1d 30c0223ec589 2024-03-29T08:27:15.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:15.181+0000 7f21a8a00700 1 -- 172.21.15.37:0/2055579663 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f218806cc30 msgr2=0x7f218806f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:15.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:15.181+0000 7f21a8a00700 1 --2- 172.21.15.37:0/2055579663 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f218806cc30 0x7f218806f0e0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f2190006fd0 tx=0x7f2190006db0 comp rx=0 tx=0).stop 2024-03-29T08:27:15.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:15.181+0000 7f21a8a00700 1 -- 172.21.15.37:0/2055579663 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f21a40fde90 msgr2=0x7f21a4115d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:15.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:15.181+0000 7f21a8a00700 1 --2- 172.21.15.37:0/2055579663 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f21a40fde90 0x7f21a4115d00 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f2194000c00 tx=0x7f2194009e30 comp rx=0 tx=0).stop 2024-03-29T08:27:15.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:15.182+0000 7f21a8a00700 1 -- 172.21.15.37:0/2055579663 shutdown_connections 2024-03-29T08:27:15.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:15.182+0000 7f21a8a00700 1 --2- 172.21.15.37:0/2055579663 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f218806cc30 0x7f218806f0e0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:15.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:15.182+0000 7f21a8a00700 1 --2- 172.21.15.37:0/2055579663 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f21a40fde90 0x7f21a4115d00 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:15.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:15.182+0000 7f21a8a00700 1 --2- 172.21.15.37:0/2055579663 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f21a40fe830 0x7f21a4116240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:15.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:15.182+0000 7f21a8a00700 1 -- 172.21.15.37:0/2055579663 >> 172.21.15.37:0/2055579663 conn(0x7f21a4072bc0 msgr2=0x7f21a40fb7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:27:15.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:15.182+0000 7f21a8a00700 1 -- 172.21.15.37:0/2055579663 shutdown_connections 2024-03-29T08:27:15.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:15.182+0000 7f21a8a00700 1 -- 172.21.15.37:0/2055579663 wait complete. 2024-03-29T08:27:15.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:15.321+0000 7f0c91000700 1 -- 172.21.15.37:0/3635536705 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0c8c102e00 msgr2=0x7f0c8c1031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:15.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:15.321+0000 7f0c91000700 1 --2- 172.21.15.37:0/3635536705 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0c8c102e00 0x7f0c8c1031d0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f0c7c004940 tx=0x7f0c7c009e30 comp rx=0 tx=0).stop 2024-03-29T08:27:15.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:15.322+0000 7f0c91000700 1 -- 172.21.15.37:0/3635536705 shutdown_connections 2024-03-29T08:27:15.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:15.322+0000 7f0c91000700 1 --2- 172.21.15.37:0/3635536705 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0c8c1037a0 0x7f0c8c0fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:15.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:15.322+0000 7f0c91000700 1 --2- 172.21.15.37:0/3635536705 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0c8c102e00 0x7f0c8c1031d0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:15.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:15.322+0000 7f0c91000700 1 -- 172.21.15.37:0/3635536705 >> 172.21.15.37:0/3635536705 conn(0x7f0c8c0f89b0 msgr2=0x7f0c8c0fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:27:15.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:15.322+0000 7f0c91000700 1 -- 172.21.15.37:0/3635536705 shutdown_connections 2024-03-29T08:27:15.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:15.322+0000 7f0c91000700 1 -- 172.21.15.37:0/3635536705 wait complete. 2024-03-29T08:27:15.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:15.322+0000 7f0c91000700 1 Processor -- start 2024-03-29T08:27:15.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:15.323+0000 7f0c91000700 1 -- start start 2024-03-29T08:27:15.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:15.323+0000 7f0c91000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0c8c102e00 0x7f0c8c0702c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:15.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:15.323+0000 7f0c91000700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0c8c1037a0 0x7f0c8c070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:15.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:15.323+0000 7f0c91000700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c8c070d90 con 0x7f0c8c102e00 2024-03-29T08:27:15.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:15.323+0000 7f0c91000700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c8c070ed0 con 0x7f0c8c1037a0 2024-03-29T08:27:15.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:15.323+0000 7f0c89e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0c8c102e00 0x7f0c8c0702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:15.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:15.323+0000 7f0c89e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0c8c102e00 0x7f0c8c0702c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60520/0 (socket says 172.21.15.37:60520) 2024-03-29T08:27:15.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:15.323+0000 7f0c89e00700 1 -- 172.21.15.37:0/3768661918 learned_addr learned my addr 172.21.15.37:0/3768661918 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:27:15.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:15.323+0000 7f0c89400700 1 --2- 172.21.15.37:0/3768661918 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0c8c1037a0 0x7f0c8c070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:15.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:15.324+0000 7f0c89e00700 1 -- 172.21.15.37:0/3768661918 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0c8c1037a0 msgr2=0x7f0c8c070800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:15.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:15.324+0000 7f0c89e00700 1 --2- 172.21.15.37:0/3768661918 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0c8c1037a0 0x7f0c8c070800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:15.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:15.324+0000 7f0c89e00700 1 -- 172.21.15.37:0/3768661918 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c7c005040 con 0x7f0c8c102e00 2024-03-29T08:27:15.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:15.324+0000 7f0c89e00700 1 --2- 172.21.15.37:0/3768661918 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0c8c102e00 0x7f0c8c0702c0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f0c7c007f50 tx=0x7f0c7c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:15.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:15.324+0000 7f0c82000700 1 -- 172.21.15.37:0/3768661918 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0c7c021070 con 0x7f0c8c102e00 2024-03-29T08:27:15.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:15.324+0000 7f0c82000700 1 -- 172.21.15.37:0/3768661918 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f0c7c0093b0 con 0x7f0c8c102e00 2024-03-29T08:27:15.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:15.324+0000 7f0c82000700 1 -- 172.21.15.37:0/3768661918 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0c7c00e070 con 0x7f0c8c102e00 2024-03-29T08:27:15.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:15.324+0000 7f0c91000700 1 -- 172.21.15.37:0/3768661918 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c8c071120 con 0x7f0c8c102e00 2024-03-29T08:27:15.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:15.325+0000 7f0c91000700 1 -- 172.21.15.37:0/3768661918 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c8c11a130 con 0x7f0c8c102e00 2024-03-29T08:27:15.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:15.326+0000 7f0c91000700 1 -- 172.21.15.37:0/3768661918 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c8c0fe550 con 0x7f0c8c102e00 2024-03-29T08:27:15.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:15.326+0000 7f0c82000700 1 -- 172.21.15.37:0/3768661918 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f0c7c01c400 con 0x7f0c8c102e00 2024-03-29T08:27:15.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:15.327+0000 7f0c82000700 1 --2- 172.21.15.37:0/3768661918 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f0c7006cc30 0x7f0c7006f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:15.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:15.327+0000 7f0c82000700 1 -- 172.21.15.37:0/3768661918 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f0c7c08d100 con 0x7f0c8c102e00 2024-03-29T08:27:15.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:15.328+0000 7f0c89400700 1 --2- 172.21.15.37:0/3768661918 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f0c7006cc30 0x7f0c7006f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:15.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:15.328+0000 7f0c89400700 1 --2- 172.21.15.37:0/3768661918 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f0c7006cc30 0x7f0c7006f0e0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f0c8c071c00 tx=0x7f0c7800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:15.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:15.332+0000 7f0c82000700 1 -- 172.21.15.37:0/3768661918 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f0c7c057140 con 0x7f0c8c102e00 2024-03-29T08:27:15.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:15.631+0000 7f0c91000700 1 -- 172.21.15.37:0/3768661918 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f0c8c04bf00 con 0x7f0c8c102e00 2024-03-29T08:27:15.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:15.631+0000 7f0c82000700 1 -- 172.21.15.37:0/3768661918 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+847 (secure 0 0 0) 0x7f0c7c05a760 con 0x7f0c8c102e00 2024-03-29T08:27:15.632 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-29T08:27:15.632 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": { 2024-03-29T08:27:15.633 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 2 2024-03-29T08:27:15.633 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:27:15.633 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": { 2024-03-29T08:27:15.633 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 2 2024-03-29T08:27:15.633 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:27:15.633 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": { 2024-03-29T08:27:15.633 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 6 2024-03-29T08:27:15.633 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:27:15.633 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": { 2024-03-29T08:27:15.633 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 17.2.7-789-g4213e17c (4213e17c4de67016ed14f55d22503f4923b3a901) quincy (stable)": 1, 2024-03-29T08:27:15.633 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 3 2024-03-29T08:27:15.633 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:27:15.633 INFO:teuthology.orchestra.run.smithi037.stdout: "overall": { 2024-03-29T08:27:15.633 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 17.2.7-789-g4213e17c (4213e17c4de67016ed14f55d22503f4923b3a901) quincy (stable)": 1, 2024-03-29T08:27:15.633 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 13 2024-03-29T08:27:15.634 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-29T08:27:15.634 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-29T08:27:15.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:15.634+0000 7f0c91000700 1 -- 172.21.15.37:0/3768661918 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f0c7006cc30 msgr2=0x7f0c7006f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:15.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:15.634+0000 7f0c91000700 1 --2- 172.21.15.37:0/3768661918 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f0c7006cc30 0x7f0c7006f0e0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f0c8c071c00 tx=0x7f0c7800b040 comp rx=0 tx=0).stop 2024-03-29T08:27:15.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:15.634+0000 7f0c91000700 1 -- 172.21.15.37:0/3768661918 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0c8c102e00 msgr2=0x7f0c8c0702c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:15.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:15.634+0000 7f0c91000700 1 --2- 172.21.15.37:0/3768661918 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0c8c102e00 0x7f0c8c0702c0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f0c7c007f50 tx=0x7f0c7c009e30 comp rx=0 tx=0).stop 2024-03-29T08:27:15.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:15.634+0000 7f0c91000700 1 -- 172.21.15.37:0/3768661918 shutdown_connections 2024-03-29T08:27:15.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:15.634+0000 7f0c91000700 1 --2- 172.21.15.37:0/3768661918 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f0c7006cc30 0x7f0c7006f0e0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:15.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:15.634+0000 7f0c91000700 1 --2- 172.21.15.37:0/3768661918 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0c8c102e00 0x7f0c8c0702c0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:15.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:15.634+0000 7f0c91000700 1 --2- 172.21.15.37:0/3768661918 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0c8c1037a0 0x7f0c8c070800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:15.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:15.634+0000 7f0c91000700 1 -- 172.21.15.37:0/3768661918 >> 172.21.15.37:0/3768661918 conn(0x7f0c8c0f89b0 msgr2=0x7f0c8c0f99f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:27:15.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:15.634+0000 7f0c91000700 1 -- 172.21.15.37:0/3768661918 shutdown_connections 2024-03-29T08:27:15.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:15.634+0000 7f0c91000700 1 -- 172.21.15.37:0/3768661918 wait complete. 2024-03-29T08:27:15.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:15.773+0000 7fe6a9600700 1 -- 172.21.15.37:0/3605492392 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe6a41037a0 msgr2=0x7fe6a40fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:15.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:15.773+0000 7fe6a9600700 1 --2- 172.21.15.37:0/3605492392 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe6a41037a0 0x7fe6a40fce10 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fe690004940 tx=0x7fe690009e30 comp rx=0 tx=0).stop 2024-03-29T08:27:15.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:15.773+0000 7fe6a9600700 1 -- 172.21.15.37:0/3605492392 shutdown_connections 2024-03-29T08:27:15.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:15.773+0000 7fe6a9600700 1 --2- 172.21.15.37:0/3605492392 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe6a41037a0 0x7fe6a40fce10 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:15.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:15.773+0000 7fe6a9600700 1 --2- 172.21.15.37:0/3605492392 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe6a4102e00 0x7fe6a41031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:15.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:15.773+0000 7fe6a9600700 1 -- 172.21.15.37:0/3605492392 >> 172.21.15.37:0/3605492392 conn(0x7fe6a40f89b0 msgr2=0x7fe6a40fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:27:15.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:15.773+0000 7fe6a9600700 1 -- 172.21.15.37:0/3605492392 shutdown_connections 2024-03-29T08:27:15.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:15.773+0000 7fe6a9600700 1 -- 172.21.15.37:0/3605492392 wait complete. 2024-03-29T08:27:15.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:15.774+0000 7fe6a9600700 1 Processor -- start 2024-03-29T08:27:15.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:15.774+0000 7fe6a9600700 1 -- start start 2024-03-29T08:27:15.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:15.774+0000 7fe6a9600700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe6a4102e00 0x7fe6a4115cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:15.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:15.774+0000 7fe6a9600700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe6a41037a0 0x7fe6a4116230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:15.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:15.774+0000 7fe6a9600700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6a4116880 con 0x7fe6a4102e00 2024-03-29T08:27:15.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:15.774+0000 7fe6a9600700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6a410fc50 con 0x7fe6a41037a0 2024-03-29T08:27:15.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:15.775+0000 7fe6a2a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe6a4102e00 0x7fe6a4115cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:15.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:15.775+0000 7fe6a2a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe6a4102e00 0x7fe6a4115cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60534/0 (socket says 172.21.15.37:60534) 2024-03-29T08:27:15.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:15.775+0000 7fe6a2a00700 1 -- 172.21.15.37:0/4188509089 learned_addr learned my addr 172.21.15.37:0/4188509089 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:27:15.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:15.775+0000 7fe6a2000700 1 --2- 172.21.15.37:0/4188509089 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe6a41037a0 0x7fe6a4116230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:15.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:15.775+0000 7fe6a2a00700 1 -- 172.21.15.37:0/4188509089 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe6a41037a0 msgr2=0x7fe6a4116230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:15.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:15.775+0000 7fe6a2a00700 1 --2- 172.21.15.37:0/4188509089 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe6a41037a0 0x7fe6a4116230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:15.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:15.775+0000 7fe6a2a00700 1 -- 172.21.15.37:0/4188509089 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe690005040 con 0x7fe6a4102e00 2024-03-29T08:27:15.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:15.775+0000 7fe6a2a00700 1 --2- 172.21.15.37:0/4188509089 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe6a4102e00 0x7fe6a4115cf0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fe69400c8c0 tx=0x7fe6940044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:15.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:15.776+0000 7fe69aa00700 1 -- 172.21.15.37:0/4188509089 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe69400a760 con 0x7fe6a4102e00 2024-03-29T08:27:15.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:15.776+0000 7fe69aa00700 1 -- 172.21.15.37:0/4188509089 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fe69400ad70 con 0x7fe6a4102e00 2024-03-29T08:27:15.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:15.776+0000 7fe6a9600700 1 -- 172.21.15.37:0/4188509089 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6a410ff30 con 0x7fe6a4102e00 2024-03-29T08:27:15.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:15.776+0000 7fe69aa00700 1 -- 172.21.15.37:0/4188509089 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe6940113a0 con 0x7fe6a4102e00 2024-03-29T08:27:15.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:15.776+0000 7fe6a9600700 1 -- 172.21.15.37:0/4188509089 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6a41104f0 con 0x7fe6a4102e00 2024-03-29T08:27:15.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:15.777+0000 7fe6a9600700 1 -- 172.21.15.37:0/4188509089 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6a40fe550 con 0x7fe6a4102e00 2024-03-29T08:27:15.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:15.778+0000 7fe69aa00700 1 -- 172.21.15.37:0/4188509089 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fe694012400 con 0x7fe6a4102e00 2024-03-29T08:27:15.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:15.779+0000 7fe69aa00700 1 --2- 172.21.15.37:0/4188509089 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fe68806cc30 0x7fe68806f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:15.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:15.779+0000 7fe69aa00700 1 -- 172.21.15.37:0/4188509089 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fe69408bfe0 con 0x7fe6a4102e00 2024-03-29T08:27:15.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:15.779+0000 7fe6a2000700 1 --2- 172.21.15.37:0/4188509089 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fe68806cc30 0x7fe68806f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:15.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:15.780+0000 7fe6a2000700 1 --2- 172.21.15.37:0/4188509089 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fe68806cc30 0x7fe68806f0e0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fe6a4111590 tx=0x7fe690009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:15.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:15.783+0000 7fe69aa00700 1 -- 172.21.15.37:0/4188509089 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fe694059af0 con 0x7fe6a4102e00 2024-03-29T08:27:15.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:27:15 smithi148 ceph-mon[106692]: pgmap v490: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:27:15.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:27:15 smithi148 ceph-mon[106692]: from='client.34924 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:27:15.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:27:15 smithi037 ceph-mon[142325]: pgmap v490: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:27:15.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:27:15 smithi037 ceph-mon[142325]: from='client.34924 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:27:16.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:16.043+0000 7fe6a9600700 1 -- 172.21.15.37:0/4188509089 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fe6a404bed0 con 0x7fe6a4102e00 2024-03-29T08:27:16.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:16.044+0000 7fe69aa00700 1 -- 172.21.15.37:0/4188509089 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 28 v28) v1 ==== 76+0+1841 (secure 0 0 0) 0x7fe694059680 con 0x7fe6a4102e00 2024-03-29T08:27:16.045 INFO:teuthology.orchestra.run.smithi037.stdout:e28 2024-03-29T08:27:16.046 INFO:teuthology.orchestra.run.smithi037.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-29T08:27:16.046 INFO:teuthology.orchestra.run.smithi037.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T08:27:16.046 INFO:teuthology.orchestra.run.smithi037.stdout:legacy client fscid: 1 2024-03-29T08:27:16.046 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:27:16.046 INFO:teuthology.orchestra.run.smithi037.stdout:Filesystem 'cephfs' (1) 2024-03-29T08:27:16.047 INFO:teuthology.orchestra.run.smithi037.stdout:fs_name cephfs 2024-03-29T08:27:16.047 INFO:teuthology.orchestra.run.smithi037.stdout:epoch 21 2024-03-29T08:27:16.047 INFO:teuthology.orchestra.run.smithi037.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-03-29T08:27:16.048 INFO:teuthology.orchestra.run.smithi037.stdout:created 2024-03-29T08:03:29.241269+0000 2024-03-29T08:27:16.048 INFO:teuthology.orchestra.run.smithi037.stdout:modified 2024-03-29T08:17:16.237238+0000 2024-03-29T08:27:16.048 INFO:teuthology.orchestra.run.smithi037.stdout:tableserver 0 2024-03-29T08:27:16.048 INFO:teuthology.orchestra.run.smithi037.stdout:root 0 2024-03-29T08:27:16.048 INFO:teuthology.orchestra.run.smithi037.stdout:session_timeout 60 2024-03-29T08:27:16.048 INFO:teuthology.orchestra.run.smithi037.stdout:session_autoclose 300 2024-03-29T08:27:16.049 INFO:teuthology.orchestra.run.smithi037.stdout:max_file_size 1099511627776 2024-03-29T08:27:16.049 INFO:teuthology.orchestra.run.smithi037.stdout:max_xattr_size 65536 2024-03-29T08:27:16.049 INFO:teuthology.orchestra.run.smithi037.stdout:required_client_features {} 2024-03-29T08:27:16.049 INFO:teuthology.orchestra.run.smithi037.stdout:last_failure 0 2024-03-29T08:27:16.049 INFO:teuthology.orchestra.run.smithi037.stdout:last_failure_osd_epoch 82 2024-03-29T08:27:16.050 INFO:teuthology.orchestra.run.smithi037.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T08:27:16.050 INFO:teuthology.orchestra.run.smithi037.stdout:max_mds 2 2024-03-29T08:27:16.050 INFO:teuthology.orchestra.run.smithi037.stdout:in 0,1 2024-03-29T08:27:16.050 INFO:teuthology.orchestra.run.smithi037.stdout:up {} 2024-03-29T08:27:16.050 INFO:teuthology.orchestra.run.smithi037.stdout:failed 0,1 2024-03-29T08:27:16.050 INFO:teuthology.orchestra.run.smithi037.stdout:damaged 2024-03-29T08:27:16.051 INFO:teuthology.orchestra.run.smithi037.stdout:stopped 2024-03-29T08:27:16.051 INFO:teuthology.orchestra.run.smithi037.stdout:data_pools [3] 2024-03-29T08:27:16.051 INFO:teuthology.orchestra.run.smithi037.stdout:metadata_pool 2 2024-03-29T08:27:16.051 INFO:teuthology.orchestra.run.smithi037.stdout:inline_data disabled 2024-03-29T08:27:16.051 INFO:teuthology.orchestra.run.smithi037.stdout:balancer 2024-03-29T08:27:16.051 INFO:teuthology.orchestra.run.smithi037.stdout:bal_rank_mask -1 2024-03-29T08:27:16.051 INFO:teuthology.orchestra.run.smithi037.stdout:standby_count_wanted 1 2024-03-29T08:27:16.052 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:27:16.052 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:27:16.052 INFO:teuthology.orchestra.run.smithi037.stdout:Standby daemons: 2024-03-29T08:27:16.052 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:27:16.052 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi148.qukpjh{-1:24319} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.148:6826/2641327912,v1:172.21.15.148:6827/2641327912] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:27:16.052 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi037.mmwgxe{-1:34440} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.37:6828/1248142005,v1:172.21.15.37:6829/1248142005] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:27:16.053 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi037.iifsmh{-1:34444} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.37:6826/2060882959,v1:172.21.15.37:6827/2060882959] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:27:16.053 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi148.sqteih{-1:44395} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.148:6824/4078652242,v1:172.21.15.148:6825/4078652242] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:27:16.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:16.046+0000 7fe6a9600700 1 -- 172.21.15.37:0/4188509089 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fe68806cc30 msgr2=0x7fe68806f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:16.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:16.046+0000 7fe6a9600700 1 --2- 172.21.15.37:0/4188509089 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fe68806cc30 0x7fe68806f0e0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fe6a4111590 tx=0x7fe690009e30 comp rx=0 tx=0).stop 2024-03-29T08:27:16.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:16.046+0000 7fe6a9600700 1 -- 172.21.15.37:0/4188509089 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe6a4102e00 msgr2=0x7fe6a4115cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:16.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:16.046+0000 7fe6a9600700 1 --2- 172.21.15.37:0/4188509089 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe6a4102e00 0x7fe6a4115cf0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fe69400c8c0 tx=0x7fe6940044c0 comp rx=0 tx=0).stop 2024-03-29T08:27:16.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:16.046+0000 7fe6a9600700 1 -- 172.21.15.37:0/4188509089 shutdown_connections 2024-03-29T08:27:16.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:16.046+0000 7fe6a9600700 1 --2- 172.21.15.37:0/4188509089 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fe68806cc30 0x7fe68806f0e0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:16.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:16.046+0000 7fe6a9600700 1 --2- 172.21.15.37:0/4188509089 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe6a4102e00 0x7fe6a4115cf0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:16.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:16.046+0000 7fe6a9600700 1 --2- 172.21.15.37:0/4188509089 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe6a41037a0 0x7fe6a4116230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:16.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:16.046+0000 7fe6a9600700 1 -- 172.21.15.37:0/4188509089 >> 172.21.15.37:0/4188509089 conn(0x7fe6a40f89b0 msgr2=0x7fe6a40f9a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:27:16.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:16.046+0000 7fe6a9600700 1 -- 172.21.15.37:0/4188509089 shutdown_connections 2024-03-29T08:27:16.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:16.046+0000 7fe6a9600700 1 -- 172.21.15.37:0/4188509089 wait complete. 2024-03-29T08:27:16.056 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 28 2024-03-29T08:27:16.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:16.184+0000 7f5037200700 1 -- 172.21.15.37:0/4060620239 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f50381035e0 msgr2=0x7f50380fcc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:16.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:16.184+0000 7f5037200700 1 --2- 172.21.15.37:0/4060620239 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f50381035e0 0x7f50380fcc50 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f5024004940 tx=0x7f5024009e30 comp rx=0 tx=0).stop 2024-03-29T08:27:16.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:16.185+0000 7f5037200700 1 -- 172.21.15.37:0/4060620239 shutdown_connections 2024-03-29T08:27:16.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:16.185+0000 7f5037200700 1 --2- 172.21.15.37:0/4060620239 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f50381035e0 0x7f50380fcc50 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:16.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:16.185+0000 7f5037200700 1 --2- 172.21.15.37:0/4060620239 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5038102c40 0x7f5038103010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:16.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:16.185+0000 7f5037200700 1 -- 172.21.15.37:0/4060620239 >> 172.21.15.37:0/4060620239 conn(0x7f50380f8830 msgr2=0x7f50380fac40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:27:16.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:16.185+0000 7f5037200700 1 -- 172.21.15.37:0/4060620239 shutdown_connections 2024-03-29T08:27:16.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:16.185+0000 7f5037200700 1 -- 172.21.15.37:0/4060620239 wait complete. 2024-03-29T08:27:16.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:16.186+0000 7f5037200700 1 Processor -- start 2024-03-29T08:27:16.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:16.186+0000 7f5037200700 1 -- start start 2024-03-29T08:27:16.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:16.187+0000 7f5037200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5038102c40 0x7f50381138e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:16.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:16.187+0000 7f5037200700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f50381035e0 0x7f5038113e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:16.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:16.187+0000 7f5037200700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5038114470 con 0x7f5038102c40 2024-03-29T08:27:16.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:16.187+0000 7f5037200700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f503810d860 con 0x7f50381035e0 2024-03-29T08:27:16.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:16.187+0000 7f5035e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5038102c40 0x7f50381138e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:16.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:16.187+0000 7f5035e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5038102c40 0x7f50381138e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60546/0 (socket says 172.21.15.37:60546) 2024-03-29T08:27:16.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:16.187+0000 7f5035e00700 1 -- 172.21.15.37:0/3724707023 learned_addr learned my addr 172.21.15.37:0/3724707023 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:27:16.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:16.187+0000 7f5035400700 1 --2- 172.21.15.37:0/3724707023 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f50381035e0 0x7f5038113e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:16.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:16.187+0000 7f5035e00700 1 -- 172.21.15.37:0/3724707023 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f50381035e0 msgr2=0x7f5038113e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:16.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:16.187+0000 7f5035e00700 1 --2- 172.21.15.37:0/3724707023 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f50381035e0 0x7f5038113e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:16.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:16.187+0000 7f5035e00700 1 -- 172.21.15.37:0/3724707023 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5024005040 con 0x7f5038102c40 2024-03-29T08:27:16.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:16.188+0000 7f5035e00700 1 --2- 172.21.15.37:0/3724707023 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5038102c40 0x7f50381138e0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f5028002a50 tx=0x7f50280046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:16.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:16.188+0000 7f502e000700 1 -- 172.21.15.37:0/3724707023 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5028007cc0 con 0x7f5038102c40 2024-03-29T08:27:16.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:16.188+0000 7f502e000700 1 -- 172.21.15.37:0/3724707023 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f502800f040 con 0x7f5038102c40 2024-03-29T08:27:16.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:16.188+0000 7f5037200700 1 -- 172.21.15.37:0/3724707023 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f503810db40 con 0x7f5038102c40 2024-03-29T08:27:16.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:16.188+0000 7f502e000700 1 -- 172.21.15.37:0/3724707023 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f502800d9c0 con 0x7f5038102c40 2024-03-29T08:27:16.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:16.188+0000 7f5037200700 1 -- 172.21.15.37:0/3724707023 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f503810e0d0 con 0x7f5038102c40 2024-03-29T08:27:16.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:16.189+0000 7f5037200700 1 -- 172.21.15.37:0/3724707023 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50380fe390 con 0x7f5038102c40 2024-03-29T08:27:16.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:16.191+0000 7f502e000700 1 -- 172.21.15.37:0/3724707023 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f502800db20 con 0x7f5038102c40 2024-03-29T08:27:16.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:16.192+0000 7f502e000700 1 --2- 172.21.15.37:0/3724707023 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f501c06cc30 0x7f501c06f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:16.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:16.192+0000 7f502e000700 1 -- 172.21.15.37:0/3724707023 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f502808bde0 con 0x7f5038102c40 2024-03-29T08:27:16.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:16.192+0000 7f5035400700 1 --2- 172.21.15.37:0/3724707023 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f501c06cc30 0x7f501c06f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:16.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:16.193+0000 7f5035400700 1 --2- 172.21.15.37:0/3724707023 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f501c06cc30 0x7f501c06f0e0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f503810eb40 tx=0x7f5024009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:16.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:16.197+0000 7f502e000700 1 -- 172.21.15.37:0/3724707023 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f50280597b0 con 0x7f5038102c40 2024-03-29T08:27:16.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:16.432+0000 7f5037200700 1 -- 172.21.15.37:0/3724707023 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5038109c70 con 0x7f501c06cc30 2024-03-29T08:27:16.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:16.434+0000 7f502e000700 1 -- 172.21.15.37:0/3724707023 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f50240096f0 con 0x7f501c06cc30 2024-03-29T08:27:16.435 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-29T08:27:16.435 INFO:teuthology.orchestra.run.smithi037.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59", 2024-03-29T08:27:16.435 INFO:teuthology.orchestra.run.smithi037.stdout: "in_progress": true, 2024-03-29T08:27:16.435 INFO:teuthology.orchestra.run.smithi037.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-29T08:27:16.435 INFO:teuthology.orchestra.run.smithi037.stdout: "services_complete": [ 2024-03-29T08:27:16.435 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr", 2024-03-29T08:27:16.435 INFO:teuthology.orchestra.run.smithi037.stdout: "mon", 2024-03-29T08:27:16.435 INFO:teuthology.orchestra.run.smithi037.stdout: "crash", 2024-03-29T08:27:16.435 INFO:teuthology.orchestra.run.smithi037.stdout: "osd" 2024-03-29T08:27:16.435 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-29T08:27:16.435 INFO:teuthology.orchestra.run.smithi037.stdout: "progress": "15/21 daemons upgraded", 2024-03-29T08:27:16.435 INFO:teuthology.orchestra.run.smithi037.stdout: "message": "Currently upgrading mds daemons", 2024-03-29T08:27:16.436 INFO:teuthology.orchestra.run.smithi037.stdout: "is_paused": false 2024-03-29T08:27:16.436 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-29T08:27:16.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:16.436+0000 7f5037200700 1 -- 172.21.15.37:0/3724707023 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f501c06cc30 msgr2=0x7f501c06f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:16.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:16.436+0000 7f5037200700 1 --2- 172.21.15.37:0/3724707023 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f501c06cc30 0x7f501c06f0e0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f503810eb40 tx=0x7f5024009e30 comp rx=0 tx=0).stop 2024-03-29T08:27:16.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:16.436+0000 7f5037200700 1 -- 172.21.15.37:0/3724707023 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5038102c40 msgr2=0x7f50381138e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:16.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:16.436+0000 7f5037200700 1 --2- 172.21.15.37:0/3724707023 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5038102c40 0x7f50381138e0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f5028002a50 tx=0x7f50280046b0 comp rx=0 tx=0).stop 2024-03-29T08:27:16.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:16.436+0000 7f5037200700 1 -- 172.21.15.37:0/3724707023 shutdown_connections 2024-03-29T08:27:16.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:16.436+0000 7f5037200700 1 --2- 172.21.15.37:0/3724707023 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f501c06cc30 0x7f501c06f0e0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:16.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:16.436+0000 7f5037200700 1 --2- 172.21.15.37:0/3724707023 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5038102c40 0x7f50381138e0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:16.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:16.436+0000 7f5037200700 1 --2- 172.21.15.37:0/3724707023 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f50381035e0 0x7f5038113e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:16.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:16.437+0000 7f5037200700 1 -- 172.21.15.37:0/3724707023 >> 172.21.15.37:0/3724707023 conn(0x7f50380f8830 msgr2=0x7f50380f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:27:16.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:16.437+0000 7f5037200700 1 -- 172.21.15.37:0/3724707023 shutdown_connections 2024-03-29T08:27:16.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:16.437+0000 7f5037200700 1 -- 172.21.15.37:0/3724707023 wait complete. 2024-03-29T08:27:16.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:16.574+0000 7fcdd3000700 1 -- 172.21.15.37:0/1242011072 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcdcc065630 msgr2=0x7fcdcc065a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:16.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:16.574+0000 7fcdd3000700 1 --2- 172.21.15.37:0/1242011072 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcdcc065630 0x7fcdcc065a80 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fcdb8004940 tx=0x7fcdb8009e30 comp rx=0 tx=0).stop 2024-03-29T08:27:16.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:16.574+0000 7fcdd3000700 1 -- 172.21.15.37:0/1242011072 shutdown_connections 2024-03-29T08:27:16.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:16.574+0000 7fcdd3000700 1 --2- 172.21.15.37:0/1242011072 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcdcc065630 0x7fcdcc065a80 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:16.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:16.574+0000 7fcdd3000700 1 --2- 172.21.15.37:0/1242011072 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcdcc102ca0 0x7fcdcc103070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:16.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:16.574+0000 7fcdd3000700 1 -- 172.21.15.37:0/1242011072 >> 172.21.15.37:0/1242011072 conn(0x7fcdcc071b70 msgr2=0x7fcdcc071f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:27:16.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:16.574+0000 7fcdd3000700 1 -- 172.21.15.37:0/1242011072 shutdown_connections 2024-03-29T08:27:16.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:16.574+0000 7fcdd3000700 1 -- 172.21.15.37:0/1242011072 wait complete. 2024-03-29T08:27:16.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:16.575+0000 7fcdd3000700 1 Processor -- start 2024-03-29T08:27:16.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:16.575+0000 7fcdd3000700 1 -- start start 2024-03-29T08:27:16.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:16.576+0000 7fcdd3000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcdcc065630 0x7fcdcc115a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:16.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:16.576+0000 7fcdd3000700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcdcc102ca0 0x7fcdcc115f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:16.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:16.576+0000 7fcdd3000700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdcc1165e0 con 0x7fcdcc065630 2024-03-29T08:27:16.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:16.576+0000 7fcdd3000700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdcc10f9d0 con 0x7fcdcc102ca0 2024-03-29T08:27:16.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:16.576+0000 7fcdd1c00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcdcc065630 0x7fcdcc115a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:16.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:16.576+0000 7fcdd1c00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcdcc065630 0x7fcdcc115a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60570/0 (socket says 172.21.15.37:60570) 2024-03-29T08:27:16.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:16.576+0000 7fcdd1c00700 1 -- 172.21.15.37:0/142732693 learned_addr learned my addr 172.21.15.37:0/142732693 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:27:16.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:16.576+0000 7fcdd1c00700 1 -- 172.21.15.37:0/142732693 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcdcc102ca0 msgr2=0x7fcdcc115f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:16.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:16.576+0000 7fcdd1200700 1 --2- 172.21.15.37:0/142732693 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcdcc102ca0 0x7fcdcc115f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:16.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:16.576+0000 7fcdd1c00700 1 --2- 172.21.15.37:0/142732693 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcdcc102ca0 0x7fcdcc115f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:16.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:16.576+0000 7fcdd1c00700 1 -- 172.21.15.37:0/142732693 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdb8005040 con 0x7fcdcc065630 2024-03-29T08:27:16.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:16.576+0000 7fcdd1200700 1 --2- 172.21.15.37:0/142732693 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcdcc102ca0 0x7fcdcc115f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:27:16.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:16.577+0000 7fcdd1c00700 1 --2- 172.21.15.37:0/142732693 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcdcc065630 0x7fcdcc115a50 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fcdc000eb40 tx=0x7fcdc000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:16.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:16.577+0000 7fcdc9600700 1 -- 172.21.15.37:0/142732693 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcdc0007cc0 con 0x7fcdcc065630 2024-03-29T08:27:16.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:16.577+0000 7fcdc9600700 1 -- 172.21.15.37:0/142732693 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fcdc000f040 con 0x7fcdcc065630 2024-03-29T08:27:16.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:16.577+0000 7fcdd3000700 1 -- 172.21.15.37:0/142732693 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdcc10fc50 con 0x7fcdcc065630 2024-03-29T08:27:16.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:16.577+0000 7fcdc9600700 1 -- 172.21.15.37:0/142732693 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcdc0009570 con 0x7fcdcc065630 2024-03-29T08:27:16.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:16.577+0000 7fcdd3000700 1 -- 172.21.15.37:0/142732693 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdcc110210 con 0x7fcdcc065630 2024-03-29T08:27:16.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:16.578+0000 7fcdd3000700 1 -- 172.21.15.37:0/142732693 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcdcc106840 con 0x7fcdcc065630 2024-03-29T08:27:16.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:16.580+0000 7fcdc9600700 1 -- 172.21.15.37:0/142732693 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fcdc00096d0 con 0x7fcdcc065630 2024-03-29T08:27:16.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:16.580+0000 7fcdc9600700 1 --2- 172.21.15.37:0/142732693 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fcdb406cc30 0x7fcdb406f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:16.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:16.581+0000 7fcdc9600700 1 -- 172.21.15.37:0/142732693 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fcdc008bfe0 con 0x7fcdcc065630 2024-03-29T08:27:16.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:16.581+0000 7fcdd1200700 1 --2- 172.21.15.37:0/142732693 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fcdb406cc30 0x7fcdb406f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:16.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:16.581+0000 7fcdd1200700 1 --2- 172.21.15.37:0/142732693 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fcdb406cc30 0x7fcdb406f0e0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fcdb800b0d0 tx=0x7fcdb8009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:16.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:16.584+0000 7fcdc9600700 1 -- 172.21.15.37:0/142732693 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fcdc0059ae0 con 0x7fcdcc065630 2024-03-29T08:27:16.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:27:16 smithi148 ceph-mon[106692]: from='client.34928 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:27:16.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:27:16 smithi148 ceph-mon[106692]: from='client.34932 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:27:16.807 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:27:16 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/3768661918' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:27:16.807 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:27:16 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/4188509089' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T08:27:16.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:16.887+0000 7fcdd3000700 1 -- 172.21.15.37:0/142732693 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fcdcc0fe380 con 0x7fcdcc065630 2024-03-29T08:27:16.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:16.887+0000 7fcdc9600700 1 -- 172.21.15.37:0/142732693 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7fcdc0059670 con 0x7fcdcc065630 2024-03-29T08:27:16.888 INFO:teuthology.orchestra.run.smithi037.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-03-29T08:27:16.888 INFO:teuthology.orchestra.run.smithi037.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-03-29T08:27:16.889 INFO:teuthology.orchestra.run.smithi037.stdout: fs cephfs is degraded 2024-03-29T08:27:16.889 INFO:teuthology.orchestra.run.smithi037.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-03-29T08:27:16.889 INFO:teuthology.orchestra.run.smithi037.stdout: fs cephfs is offline because no MDS is active for it. 2024-03-29T08:27:16.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:16.890+0000 7fcdd3000700 1 -- 172.21.15.37:0/142732693 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fcdb406cc30 msgr2=0x7fcdb406f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:16.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:16.890+0000 7fcdd3000700 1 --2- 172.21.15.37:0/142732693 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fcdb406cc30 0x7fcdb406f0e0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fcdb800b0d0 tx=0x7fcdb8009e30 comp rx=0 tx=0).stop 2024-03-29T08:27:16.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:16.890+0000 7fcdd3000700 1 -- 172.21.15.37:0/142732693 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcdcc065630 msgr2=0x7fcdcc115a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:16.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:16.890+0000 7fcdd3000700 1 --2- 172.21.15.37:0/142732693 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcdcc065630 0x7fcdcc115a50 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fcdc000eb40 tx=0x7fcdc000b040 comp rx=0 tx=0).stop 2024-03-29T08:27:16.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:16.890+0000 7fcdd3000700 1 -- 172.21.15.37:0/142732693 shutdown_connections 2024-03-29T08:27:16.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:16.890+0000 7fcdd3000700 1 --2- 172.21.15.37:0/142732693 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fcdb406cc30 0x7fcdb406f0e0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:16.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:16.890+0000 7fcdd3000700 1 --2- 172.21.15.37:0/142732693 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcdcc065630 0x7fcdcc115a50 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:16.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:16.890+0000 7fcdd3000700 1 --2- 172.21.15.37:0/142732693 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcdcc102ca0 0x7fcdcc115f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:16.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:16.890+0000 7fcdd3000700 1 -- 172.21.15.37:0/142732693 >> 172.21.15.37:0/142732693 conn(0x7fcdcc071b70 msgr2=0x7fcdcc1039c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:27:16.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:16.890+0000 7fcdd3000700 1 -- 172.21.15.37:0/142732693 shutdown_connections 2024-03-29T08:27:16.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:16.890+0000 7fcdd3000700 1 -- 172.21.15.37:0/142732693 wait complete. 2024-03-29T08:27:16.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:27:16 smithi037 ceph-mon[142325]: from='client.34928 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:27:16.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:27:16 smithi037 ceph-mon[142325]: from='client.34932 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:27:16.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:27:16 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/3768661918' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:27:16.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:27:16 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/4188509089' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T08:27:17.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:27:17 smithi148 ceph-mon[106692]: pgmap v491: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:27:17.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:27:17 smithi148 ceph-mon[106692]: from='client.34944 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:27:17.807 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:27:17 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/142732693' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T08:27:17.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:27:17 smithi037 ceph-mon[142325]: pgmap v491: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:27:17.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:27:17 smithi037 ceph-mon[142325]: from='client.34944 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:27:17.928 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:27:17 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/142732693' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T08:27:19.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:27:19 smithi148 ceph-mon[106692]: pgmap v492: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:27:19.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:27:19 smithi037 ceph-mon[142325]: pgmap v492: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:27:21.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:27:21 smithi148 ceph-mon[106692]: pgmap v493: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:27:21.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:27:21 smithi037 ceph-mon[142325]: pgmap v493: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:27:23.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:27:23 smithi148 ceph-mon[106692]: pgmap v494: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:27:23.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:27:23 smithi037 ceph-mon[142325]: pgmap v494: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:27:25.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:27:25 smithi148 ceph-mon[106692]: pgmap v495: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:27:25.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:27:25 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:27:25.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:27:25 smithi037 ceph-mon[142325]: pgmap v495: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:27:25.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:27:25 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:27:27.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:27:27 smithi148 ceph-mon[106692]: pgmap v496: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:27:27.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:27:27 smithi037 ceph-mon[142325]: pgmap v496: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:27:29.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:27:29 smithi148 ceph-mon[106692]: pgmap v497: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:27:29.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:27:29 smithi037 ceph-mon[142325]: pgmap v497: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:27:31.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:27:31 smithi148 ceph-mon[106692]: pgmap v498: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:27:31.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:27:31 smithi037 ceph-mon[142325]: pgmap v498: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:27:33.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:27:33 smithi148 ceph-mon[106692]: pgmap v499: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:27:33.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:27:33 smithi037 ceph-mon[142325]: pgmap v499: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:27:35.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:27:35 smithi148 ceph-mon[106692]: pgmap v500: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:27:35.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:27:35 smithi037 ceph-mon[142325]: pgmap v500: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:27:37.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:27:37 smithi148 ceph-mon[106692]: pgmap v501: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:27:37.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:27:37 smithi037 ceph-mon[142325]: pgmap v501: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:27:39.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:27:39 smithi148 ceph-mon[106692]: pgmap v502: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:27:39.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:27:39 smithi037 ceph-mon[142325]: pgmap v502: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:27:40.807 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:27:40 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:27:40.990 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:27:40 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:27:41.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:27:41 smithi148 ceph-mon[106692]: pgmap v503: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:27:41.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:27:41 smithi037 ceph-mon[142325]: pgmap v503: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:27:43.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:27:43 smithi148 ceph-mon[106692]: pgmap v504: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:27:43.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:27:43 smithi037 ceph-mon[142325]: pgmap v504: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:27:45.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:27:45 smithi148 ceph-mon[106692]: pgmap v505: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:27:45.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:27:45 smithi037 ceph-mon[142325]: pgmap v505: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:27:47.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:47.045+0000 7fc87c200700 1 -- 172.21.15.37:0/455920110 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc8741037a0 msgr2=0x7fc8740fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:47.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:47.045+0000 7fc87c200700 1 --2- 172.21.15.37:0/455920110 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc8741037a0 0x7fc8740fce10 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fc860004940 tx=0x7fc860009e30 comp rx=0 tx=0).stop 2024-03-29T08:27:47.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:47.045+0000 7fc87c200700 1 -- 172.21.15.37:0/455920110 shutdown_connections 2024-03-29T08:27:47.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:47.045+0000 7fc87c200700 1 --2- 172.21.15.37:0/455920110 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc8741037a0 0x7fc8740fce10 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:47.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:47.045+0000 7fc87c200700 1 --2- 172.21.15.37:0/455920110 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc874102e00 0x7fc8741031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:47.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:47.046+0000 7fc87c200700 1 -- 172.21.15.37:0/455920110 >> 172.21.15.37:0/455920110 conn(0x7fc8740f89b0 msgr2=0x7fc8740fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:27:47.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:47.046+0000 7fc87c200700 1 -- 172.21.15.37:0/455920110 shutdown_connections 2024-03-29T08:27:47.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:47.046+0000 7fc87c200700 1 -- 172.21.15.37:0/455920110 wait complete. 2024-03-29T08:27:47.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:47.046+0000 7fc87c200700 1 Processor -- start 2024-03-29T08:27:47.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:47.047+0000 7fc87c200700 1 -- start start 2024-03-29T08:27:47.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:47.047+0000 7fc87c200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc874102e00 0x7fc874115cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:47.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:47.047+0000 7fc87c200700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc8741037a0 0x7fc874116230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:47.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:47.047+0000 7fc87c200700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc874116880 con 0x7fc874102e00 2024-03-29T08:27:47.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:47.047+0000 7fc87c200700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc87410fc50 con 0x7fc8741037a0 2024-03-29T08:27:47.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:47.047+0000 7fc879800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc874102e00 0x7fc874115cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:47.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:47.047+0000 7fc879800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc874102e00 0x7fc874115cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52058/0 (socket says 172.21.15.37:52058) 2024-03-29T08:27:47.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:47.047+0000 7fc879800700 1 -- 172.21.15.37:0/1790396596 learned_addr learned my addr 172.21.15.37:0/1790396596 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:27:47.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:47.048+0000 7fc878e00700 1 --2- 172.21.15.37:0/1790396596 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc8741037a0 0x7fc874116230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:47.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:47.048+0000 7fc879800700 1 -- 172.21.15.37:0/1790396596 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc8741037a0 msgr2=0x7fc874116230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:47.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:47.048+0000 7fc879800700 1 --2- 172.21.15.37:0/1790396596 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc8741037a0 0x7fc874116230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:47.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:47.048+0000 7fc879800700 1 -- 172.21.15.37:0/1790396596 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc860005040 con 0x7fc874102e00 2024-03-29T08:27:47.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:47.048+0000 7fc878e00700 1 --2- 172.21.15.37:0/1790396596 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc8741037a0 0x7fc874116230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:27:47.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:47.048+0000 7fc879800700 1 --2- 172.21.15.37:0/1790396596 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc874102e00 0x7fc874115cf0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fc868002a50 tx=0x7fc8680046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:47.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:47.048+0000 7fc871600700 1 -- 172.21.15.37:0/1790396596 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc868007cc0 con 0x7fc874102e00 2024-03-29T08:27:47.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:47.048+0000 7fc871600700 1 -- 172.21.15.37:0/1790396596 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fc86800f040 con 0x7fc874102e00 2024-03-29T08:27:47.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:47.048+0000 7fc87c200700 1 -- 172.21.15.37:0/1790396596 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc87410ff30 con 0x7fc874102e00 2024-03-29T08:27:47.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:47.048+0000 7fc871600700 1 -- 172.21.15.37:0/1790396596 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc868009540 con 0x7fc874102e00 2024-03-29T08:27:47.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:47.049+0000 7fc87c200700 1 -- 172.21.15.37:0/1790396596 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8741104f0 con 0x7fc874102e00 2024-03-29T08:27:47.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:47.050+0000 7fc87c200700 1 -- 172.21.15.37:0/1790396596 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8740fe550 con 0x7fc874102e00 2024-03-29T08:27:47.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:47.051+0000 7fc871600700 1 -- 172.21.15.37:0/1790396596 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fc8680096a0 con 0x7fc874102e00 2024-03-29T08:27:47.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:47.052+0000 7fc871600700 1 --2- 172.21.15.37:0/1790396596 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fc85c06cc30 0x7fc85c06f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:47.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:47.052+0000 7fc871600700 1 -- 172.21.15.37:0/1790396596 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fc86808bee0 con 0x7fc874102e00 2024-03-29T08:27:47.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:47.052+0000 7fc878e00700 1 --2- 172.21.15.37:0/1790396596 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fc85c06cc30 0x7fc85c06f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:47.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:47.053+0000 7fc878e00700 1 --2- 172.21.15.37:0/1790396596 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fc85c06cc30 0x7fc85c06f0e0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fc874111590 tx=0x7fc860009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:47.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:47.057+0000 7fc871600700 1 -- 172.21.15.37:0/1790396596 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fc8680598b0 con 0x7fc874102e00 2024-03-29T08:27:47.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:47.293+0000 7fc87c200700 1 -- 172.21.15.37:0/1790396596 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc87404bf40 con 0x7fc85c06cc30 2024-03-29T08:27:47.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:47.294+0000 7fc871600700 1 -- 172.21.15.37:0/1790396596 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fc87404bf40 con 0x7fc85c06cc30 2024-03-29T08:27:47.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:47.296+0000 7fc87c200700 1 -- 172.21.15.37:0/1790396596 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fc85c06cc30 msgr2=0x7fc85c06f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:47.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:47.296+0000 7fc87c200700 1 --2- 172.21.15.37:0/1790396596 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fc85c06cc30 0x7fc85c06f0e0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fc874111590 tx=0x7fc860009e30 comp rx=0 tx=0).stop 2024-03-29T08:27:47.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:47.296+0000 7fc87c200700 1 -- 172.21.15.37:0/1790396596 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc874102e00 msgr2=0x7fc874115cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:47.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:47.296+0000 7fc87c200700 1 --2- 172.21.15.37:0/1790396596 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc874102e00 0x7fc874115cf0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fc868002a50 tx=0x7fc8680046b0 comp rx=0 tx=0).stop 2024-03-29T08:27:47.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:47.296+0000 7fc87c200700 1 -- 172.21.15.37:0/1790396596 shutdown_connections 2024-03-29T08:27:47.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:47.296+0000 7fc87c200700 1 --2- 172.21.15.37:0/1790396596 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fc85c06cc30 0x7fc85c06f0e0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:47.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:47.296+0000 7fc87c200700 1 --2- 172.21.15.37:0/1790396596 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc874102e00 0x7fc874115cf0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:47.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:47.296+0000 7fc87c200700 1 --2- 172.21.15.37:0/1790396596 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc8741037a0 0x7fc874116230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:47.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:47.296+0000 7fc87c200700 1 -- 172.21.15.37:0/1790396596 >> 172.21.15.37:0/1790396596 conn(0x7fc8740f89b0 msgr2=0x7fc8740f9a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:27:47.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:47.297+0000 7fc87c200700 1 -- 172.21.15.37:0/1790396596 shutdown_connections 2024-03-29T08:27:47.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:47.297+0000 7fc87c200700 1 -- 172.21.15.37:0/1790396596 wait complete. 2024-03-29T08:27:47.311 INFO:teuthology.orchestra.run.smithi037.stdout:true 2024-03-29T08:27:47.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:47.435+0000 7f204f400700 1 -- 172.21.15.37:0/2468901242 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2048102e00 msgr2=0x7f20481031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:47.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:47.435+0000 7f204f400700 1 --2- 172.21.15.37:0/2468901242 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2048102e00 0x7f20481031d0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f203c004940 tx=0x7f203c009e30 comp rx=0 tx=0).stop 2024-03-29T08:27:47.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:47.435+0000 7f204f400700 1 -- 172.21.15.37:0/2468901242 shutdown_connections 2024-03-29T08:27:47.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:47.435+0000 7f204f400700 1 --2- 172.21.15.37:0/2468901242 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f20481037a0 0x7f20480fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:47.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:47.435+0000 7f204f400700 1 --2- 172.21.15.37:0/2468901242 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2048102e00 0x7f20481031d0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:47.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:47.435+0000 7f204f400700 1 -- 172.21.15.37:0/2468901242 >> 172.21.15.37:0/2468901242 conn(0x7f20480f89b0 msgr2=0x7f20480fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:27:47.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:47.435+0000 7f204f400700 1 -- 172.21.15.37:0/2468901242 shutdown_connections 2024-03-29T08:27:47.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:47.435+0000 7f204f400700 1 -- 172.21.15.37:0/2468901242 wait complete. 2024-03-29T08:27:47.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:47.436+0000 7f204f400700 1 Processor -- start 2024-03-29T08:27:47.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:47.436+0000 7f204f400700 1 -- start start 2024-03-29T08:27:47.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:47.436+0000 7f204f400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2048102e00 0x7f2048115c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:47.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:47.436+0000 7f204f400700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f20481037a0 0x7f20481161b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:47.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:47.436+0000 7f204f400700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2048116800 con 0x7f2048102e00 2024-03-29T08:27:47.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:47.436+0000 7f204f400700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f204810fbd0 con 0x7f20481037a0 2024-03-29T08:27:47.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:47.437+0000 7f204ca00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2048102e00 0x7f2048115c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:47.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:47.437+0000 7f204ca00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2048102e00 0x7f2048115c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52080/0 (socket says 172.21.15.37:52080) 2024-03-29T08:27:47.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:47.437+0000 7f204ca00700 1 -- 172.21.15.37:0/2496045456 learned_addr learned my addr 172.21.15.37:0/2496045456 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:27:47.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:47.437+0000 7f204ca00700 1 -- 172.21.15.37:0/2496045456 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f20481037a0 msgr2=0x7f20481161b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:47.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:47.437+0000 7f204ca00700 1 --2- 172.21.15.37:0/2496045456 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f20481037a0 0x7f20481161b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:47.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:47.437+0000 7f204ca00700 1 -- 172.21.15.37:0/2496045456 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f203c005040 con 0x7f2048102e00 2024-03-29T08:27:47.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:47.437+0000 7f204ca00700 1 --2- 172.21.15.37:0/2496045456 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2048102e00 0x7f2048115c70 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f203c008d80 tx=0x7f203c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:47.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:47.438+0000 7f2044c00700 1 -- 172.21.15.37:0/2496045456 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f203c00e030 con 0x7f2048102e00 2024-03-29T08:27:47.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:47.438+0000 7f2044c00700 1 -- 172.21.15.37:0/2496045456 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f203c013de0 con 0x7f2048102e00 2024-03-29T08:27:47.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:47.438+0000 7f2044c00700 1 -- 172.21.15.37:0/2496045456 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f203c01cd60 con 0x7f2048102e00 2024-03-29T08:27:47.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:47.438+0000 7f204f400700 1 -- 172.21.15.37:0/2496045456 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f204810fe50 con 0x7f2048102e00 2024-03-29T08:27:47.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:47.438+0000 7f204f400700 1 -- 172.21.15.37:0/2496045456 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2048110410 con 0x7f2048102e00 2024-03-29T08:27:47.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:47.439+0000 7f204f400700 1 -- 172.21.15.37:0/2496045456 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20480fe550 con 0x7f2048102e00 2024-03-29T08:27:47.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:47.440+0000 7f2044c00700 1 -- 172.21.15.37:0/2496045456 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f203c01c590 con 0x7f2048102e00 2024-03-29T08:27:47.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:47.441+0000 7f2044c00700 1 --2- 172.21.15.37:0/2496045456 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f203006cc30 0x7f203006f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:47.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:47.441+0000 7f2044c00700 1 -- 172.21.15.37:0/2496045456 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f203c08d9c0 con 0x7f2048102e00 2024-03-29T08:27:47.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:47.441+0000 7f2047e00700 1 --2- 172.21.15.37:0/2496045456 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f203006cc30 0x7f203006f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:47.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:47.442+0000 7f2047e00700 1 --2- 172.21.15.37:0/2496045456 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f203006cc30 0x7f203006f0e0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f2048111510 tx=0x7f203400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:47.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:47.446+0000 7f2044c00700 1 -- 172.21.15.37:0/2496045456 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f203c05b4c0 con 0x7f2048102e00 2024-03-29T08:27:47.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:47.683+0000 7f204f400700 1 -- 172.21.15.37:0/2496045456 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f204804bed0 con 0x7f203006cc30 2024-03-29T08:27:47.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:47.685+0000 7f2044c00700 1 -- 172.21.15.37:0/2496045456 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f204804bed0 con 0x7f203006cc30 2024-03-29T08:27:47.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:47.687+0000 7f204f400700 1 -- 172.21.15.37:0/2496045456 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f203006cc30 msgr2=0x7f203006f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:47.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:47.687+0000 7f204f400700 1 --2- 172.21.15.37:0/2496045456 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f203006cc30 0x7f203006f0e0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f2048111510 tx=0x7f203400b040 comp rx=0 tx=0).stop 2024-03-29T08:27:47.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:47.687+0000 7f204f400700 1 -- 172.21.15.37:0/2496045456 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2048102e00 msgr2=0x7f2048115c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:47.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:47.687+0000 7f204f400700 1 --2- 172.21.15.37:0/2496045456 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2048102e00 0x7f2048115c70 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f203c008d80 tx=0x7f203c009e30 comp rx=0 tx=0).stop 2024-03-29T08:27:47.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:47.687+0000 7f204f400700 1 -- 172.21.15.37:0/2496045456 shutdown_connections 2024-03-29T08:27:47.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:47.687+0000 7f204f400700 1 --2- 172.21.15.37:0/2496045456 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f203006cc30 0x7f203006f0e0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:47.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:47.687+0000 7f204f400700 1 --2- 172.21.15.37:0/2496045456 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2048102e00 0x7f2048115c70 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:47.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:47.687+0000 7f204f400700 1 --2- 172.21.15.37:0/2496045456 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f20481037a0 0x7f20481161b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:47.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:47.687+0000 7f204f400700 1 -- 172.21.15.37:0/2496045456 >> 172.21.15.37:0/2496045456 conn(0x7f20480f89b0 msgr2=0x7f20480f9960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:27:47.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:47.687+0000 7f204f400700 1 -- 172.21.15.37:0/2496045456 shutdown_connections 2024-03-29T08:27:47.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:47.687+0000 7f204f400700 1 -- 172.21.15.37:0/2496045456 wait complete. 2024-03-29T08:27:47.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:27:47 smithi148 ceph-mon[106692]: pgmap v506: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:27:47.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:47.825+0000 7fb3f5c00700 1 -- 172.21.15.37:0/3107472795 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb3f00fe850 msgr2=0x7fb3f0106f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:47.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:47.825+0000 7fb3f5c00700 1 --2- 172.21.15.37:0/3107472795 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb3f00fe850 0x7fb3f0106f20 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fb3dc004970 tx=0x7fb3dc009e30 comp rx=0 tx=0).stop 2024-03-29T08:27:47.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:47.825+0000 7fb3f5c00700 1 -- 172.21.15.37:0/3107472795 shutdown_connections 2024-03-29T08:27:47.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:47.825+0000 7fb3f5c00700 1 --2- 172.21.15.37:0/3107472795 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb3f00fe850 0x7fb3f0106f20 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:47.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:47.825+0000 7fb3f5c00700 1 --2- 172.21.15.37:0/3107472795 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb3f00fdeb0 0x7fb3f00fe280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:47.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:47.825+0000 7fb3f5c00700 1 -- 172.21.15.37:0/3107472795 >> 172.21.15.37:0/3107472795 conn(0x7fb3f0072bc0 msgr2=0x7fb3f0072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:27:47.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:47.825+0000 7fb3f5c00700 1 -- 172.21.15.37:0/3107472795 shutdown_connections 2024-03-29T08:27:47.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:47.825+0000 7fb3f5c00700 1 -- 172.21.15.37:0/3107472795 wait complete. 2024-03-29T08:27:47.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:47.826+0000 7fb3f5c00700 1 Processor -- start 2024-03-29T08:27:47.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:47.826+0000 7fb3f5c00700 1 -- start start 2024-03-29T08:27:47.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:47.826+0000 7fb3f5c00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb3f00fdeb0 0x7fb3f00fef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:47.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:47.826+0000 7fb3f5c00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb3f00fe850 0x7fb3f00ff480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:47.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:47.826+0000 7fb3f5c00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3f01030d0 con 0x7fb3f00fdeb0 2024-03-29T08:27:47.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:47.826+0000 7fb3f5c00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3f00ff9c0 con 0x7fb3f00fe850 2024-03-29T08:27:47.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:47.827+0000 7fb3eea00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb3f00fdeb0 0x7fb3f00fef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:47.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:47.827+0000 7fb3eea00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb3f00fdeb0 0x7fb3f00fef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52096/0 (socket says 172.21.15.37:52096) 2024-03-29T08:27:47.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:47.827+0000 7fb3eea00700 1 -- 172.21.15.37:0/2076755843 learned_addr learned my addr 172.21.15.37:0/2076755843 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:27:47.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:47.827+0000 7fb3ee000700 1 --2- 172.21.15.37:0/2076755843 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb3f00fe850 0x7fb3f00ff480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:47.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:47.827+0000 7fb3eea00700 1 -- 172.21.15.37:0/2076755843 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb3f00fe850 msgr2=0x7fb3f00ff480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:47.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:47.827+0000 7fb3eea00700 1 --2- 172.21.15.37:0/2076755843 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb3f00fe850 0x7fb3f00ff480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:47.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:47.827+0000 7fb3eea00700 1 -- 172.21.15.37:0/2076755843 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3dc005040 con 0x7fb3f00fdeb0 2024-03-29T08:27:47.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:47.827+0000 7fb3eea00700 1 --2- 172.21.15.37:0/2076755843 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb3f00fdeb0 0x7fb3f00fef40 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fb3e0002a50 tx=0x7fb3e00046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:47.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:47.828+0000 7fb3e6a00700 1 -- 172.21.15.37:0/2076755843 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb3e0007cc0 con 0x7fb3f00fdeb0 2024-03-29T08:27:47.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:47.828+0000 7fb3e6a00700 1 -- 172.21.15.37:0/2076755843 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fb3e000f040 con 0x7fb3f00fdeb0 2024-03-29T08:27:47.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:47.828+0000 7fb3f5c00700 1 -- 172.21.15.37:0/2076755843 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3f00ffca0 con 0x7fb3f00fdeb0 2024-03-29T08:27:47.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:47.828+0000 7fb3e6a00700 1 -- 172.21.15.37:0/2076755843 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb3e000d9c0 con 0x7fb3f00fdeb0 2024-03-29T08:27:47.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:47.828+0000 7fb3f5c00700 1 -- 172.21.15.37:0/2076755843 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3f011e4f0 con 0x7fb3f00fdeb0 2024-03-29T08:27:47.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:47.829+0000 7fb3f5c00700 1 -- 172.21.15.37:0/2076755843 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3f01069f0 con 0x7fb3f00fdeb0 2024-03-29T08:27:47.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:47.830+0000 7fb3e6a00700 1 -- 172.21.15.37:0/2076755843 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fb3e000db20 con 0x7fb3f00fdeb0 2024-03-29T08:27:47.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:47.831+0000 7fb3e6a00700 1 --2- 172.21.15.37:0/2076755843 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fb3d406cc30 0x7fb3d406f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:47.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:47.831+0000 7fb3e6a00700 1 -- 172.21.15.37:0/2076755843 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fb3e008bd20 con 0x7fb3f00fdeb0 2024-03-29T08:27:47.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:47.831+0000 7fb3ee000700 1 --2- 172.21.15.37:0/2076755843 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fb3d406cc30 0x7fb3d406f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:47.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:47.832+0000 7fb3ee000700 1 --2- 172.21.15.37:0/2076755843 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fb3d406cc30 0x7fb3d406f0e0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fb3dc00b370 tx=0x7fb3dc009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:47.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:47.836+0000 7fb3e6a00700 1 -- 172.21.15.37:0/2076755843 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fb3e0055ce0 con 0x7fb3f00fdeb0 2024-03-29T08:27:47.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:27:47 smithi037 ceph-mon[142325]: pgmap v506: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:27:48.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:48.058+0000 7fb3f5c00700 1 -- 172.21.15.37:0/2076755843 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fb3f004bed0 con 0x7fb3d406cc30 2024-03-29T08:27:48.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:48.068+0000 7fb3e6a00700 1 -- 172.21.15.37:0/2076755843 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7fb3f004bed0 con 0x7fb3d406cc30 2024-03-29T08:27:48.068 INFO:teuthology.orchestra.run.smithi037.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-29T08:27:48.069 INFO:teuthology.orchestra.run.smithi037.stdout:alertmanager.smithi037 smithi037 *:9093,9094 running (17m) 9m ago 30m 14.9M - 0.25.0 c8568f914cd2 2f9f9746ed0f 2024-03-29T08:27:48.069 INFO:teuthology.orchestra.run.smithi037.stdout:crash.smithi037 smithi037 running (14m) 9m ago 30m 6647k - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 178d416a3186 2024-03-29T08:27:48.069 INFO:teuthology.orchestra.run.smithi037.stdout:crash.smithi148 smithi148 running (14m) 9m ago 29m 6656k - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 23666989d58f 2024-03-29T08:27:48.069 INFO:teuthology.orchestra.run.smithi037.stdout:grafana.smithi037 smithi037 *:3000 running (17m) 9m ago 30m 71.9M - 9.4.7 954c08fa6188 6b976fe49952 2024-03-29T08:27:48.069 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi037.iifsmh smithi037 running (10m) 9m ago 24m 10.8M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 2d97f562cc5b 2024-03-29T08:27:48.069 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi037.mmwgxe smithi037 running (10m) 9m ago 24m 10.9M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff ec37037f46e8 2024-03-29T08:27:48.069 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi148.qukpjh smithi148 running (24m) 9m ago 24m 15.3M - 17.2.7-789-g4213e17c cf22eb8683be 8e469782a273 2024-03-29T08:27:48.069 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi148.sqteih smithi148 running (9m) 9m ago 24m 10.7M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 44c15c77037a 2024-03-29T08:27:48.069 INFO:teuthology.orchestra.run.smithi037.stdout:mgr.smithi037.wjikii smithi037 *:8443,9283,8765 running (19m) 9m ago 32m 506M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff c5a1417b9417 2024-03-29T08:27:48.069 INFO:teuthology.orchestra.run.smithi037.stdout:mgr.smithi148.onhgvm smithi148 *:8443,9283,8765 running (18m) 9m ago 29m 450M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff fcfdbce062bd 2024-03-29T08:27:48.069 INFO:teuthology.orchestra.run.smithi037.stdout:mon.smithi037 smithi037 running (15m) 9m ago 32m 54.8M 2048M 18.2.2-1015-gd0e392d9 1cc2f45c17ff eea2060783af 2024-03-29T08:27:48.070 INFO:teuthology.orchestra.run.smithi037.stdout:mon.smithi148 smithi148 running (15m) 9m ago 29m 43.7M 2048M 18.2.2-1015-gd0e392d9 1cc2f45c17ff ad4b2c5748be 2024-03-29T08:27:48.070 INFO:teuthology.orchestra.run.smithi037.stdout:node-exporter.smithi037 smithi037 *:9100 running (18m) 9m ago 30m 7625k - 1.5.0 0da6a335fe13 26b8ae9cb878 2024-03-29T08:27:48.070 INFO:teuthology.orchestra.run.smithi037.stdout:node-exporter.smithi148 smithi148 *:9100 running (18m) 9m ago 29m 8166k - 1.5.0 0da6a335fe13 98414e4e64ed 2024-03-29T08:27:48.070 INFO:teuthology.orchestra.run.smithi037.stdout:osd.0 smithi037 running (14m) 9m ago 28m 104M 1619M 18.2.2-1015-gd0e392d9 1cc2f45c17ff c0c584f6245c 2024-03-29T08:27:48.070 INFO:teuthology.orchestra.run.smithi037.stdout:osd.1 smithi037 running (13m) 9m ago 27m 100M 1619M 18.2.2-1015-gd0e392d9 1cc2f45c17ff a4a2ff4cc83d 2024-03-29T08:27:48.070 INFO:teuthology.orchestra.run.smithi037.stdout:osd.2 smithi037 running (12m) 9m ago 27m 82.3M 1619M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 8b1bc71aa79e 2024-03-29T08:27:48.070 INFO:teuthology.orchestra.run.smithi037.stdout:osd.3 smithi148 running (12m) 9m ago 26m 105M 2642M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 1c656ab8ff86 2024-03-29T08:27:48.070 INFO:teuthology.orchestra.run.smithi037.stdout:osd.4 smithi148 running (11m) 9m ago 26m 97.3M 2642M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 966a9de11d22 2024-03-29T08:27:48.070 INFO:teuthology.orchestra.run.smithi037.stdout:osd.5 smithi148 running (10m) 9m ago 25m 85.8M 2642M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 474c4f9212ce 2024-03-29T08:27:48.070 INFO:teuthology.orchestra.run.smithi037.stdout:prometheus.smithi037 smithi037 *:9095 running (17m) 9m ago 30m 54.4M - 2.43.0 a07b618ecd1d 30c0223ec589 2024-03-29T08:27:48.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:48.070+0000 7fb3f5c00700 1 -- 172.21.15.37:0/2076755843 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fb3d406cc30 msgr2=0x7fb3d406f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:48.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:48.070+0000 7fb3f5c00700 1 --2- 172.21.15.37:0/2076755843 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fb3d406cc30 0x7fb3d406f0e0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fb3dc00b370 tx=0x7fb3dc009e30 comp rx=0 tx=0).stop 2024-03-29T08:27:48.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:48.070+0000 7fb3f5c00700 1 -- 172.21.15.37:0/2076755843 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb3f00fdeb0 msgr2=0x7fb3f00fef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:48.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:48.070+0000 7fb3f5c00700 1 --2- 172.21.15.37:0/2076755843 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb3f00fdeb0 0x7fb3f00fef40 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fb3e0002a50 tx=0x7fb3e00046b0 comp rx=0 tx=0).stop 2024-03-29T08:27:48.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:48.070+0000 7fb3f5c00700 1 -- 172.21.15.37:0/2076755843 shutdown_connections 2024-03-29T08:27:48.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:48.070+0000 7fb3f5c00700 1 --2- 172.21.15.37:0/2076755843 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fb3d406cc30 0x7fb3d406f0e0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:48.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:48.070+0000 7fb3f5c00700 1 --2- 172.21.15.37:0/2076755843 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb3f00fdeb0 0x7fb3f00fef40 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:48.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:48.070+0000 7fb3f5c00700 1 --2- 172.21.15.37:0/2076755843 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb3f00fe850 0x7fb3f00ff480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:48.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:48.070+0000 7fb3f5c00700 1 -- 172.21.15.37:0/2076755843 >> 172.21.15.37:0/2076755843 conn(0x7fb3f0072bc0 msgr2=0x7fb3f00fb640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:27:48.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:48.071+0000 7fb3f5c00700 1 -- 172.21.15.37:0/2076755843 shutdown_connections 2024-03-29T08:27:48.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:48.071+0000 7fb3f5c00700 1 -- 172.21.15.37:0/2076755843 wait complete. 2024-03-29T08:27:48.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:48.206+0000 7f5dd0c00700 1 -- 172.21.15.37:0/4078494485 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5dcc070300 msgr2=0x7f5dcc0706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:48.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:48.206+0000 7f5dd0c00700 1 --2- 172.21.15.37:0/4078494485 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5dcc070300 0x7f5dcc0706d0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f5dbc004970 tx=0x7f5dbc009e30 comp rx=0 tx=0).stop 2024-03-29T08:27:48.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:48.207+0000 7f5dd0c00700 1 -- 172.21.15.37:0/4078494485 shutdown_connections 2024-03-29T08:27:48.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:48.207+0000 7f5dd0c00700 1 --2- 172.21.15.37:0/4078494485 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5dcc070c10 0x7f5dcc109190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:48.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:48.207+0000 7f5dd0c00700 1 --2- 172.21.15.37:0/4078494485 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5dcc070300 0x7f5dcc0706d0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:48.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:48.207+0000 7f5dd0c00700 1 -- 172.21.15.37:0/4078494485 >> 172.21.15.37:0/4078494485 conn(0x7f5dcc0f8970 msgr2=0x7f5dcc0fad80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:27:48.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:48.207+0000 7f5dd0c00700 1 -- 172.21.15.37:0/4078494485 shutdown_connections 2024-03-29T08:27:48.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:48.207+0000 7f5dd0c00700 1 -- 172.21.15.37:0/4078494485 wait complete. 2024-03-29T08:27:48.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:48.208+0000 7f5dd0c00700 1 Processor -- start 2024-03-29T08:27:48.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:48.208+0000 7f5dd0c00700 1 -- start start 2024-03-29T08:27:48.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:48.209+0000 7f5dd0c00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5dcc070300 0x7f5dcc115c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:48.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:48.209+0000 7f5dd0c00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5dcc070c10 0x7f5dcc1161d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:48.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:48.209+0000 7f5dd0c00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dcc116820 con 0x7f5dcc070c10 2024-03-29T08:27:48.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:48.209+0000 7f5dd0c00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dcc10fc30 con 0x7f5dcc070300 2024-03-29T08:27:48.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:48.209+0000 7f5dc9400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5dcc070c10 0x7f5dcc1161d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:48.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:48.209+0000 7f5dc9400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5dcc070c10 0x7f5dcc1161d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52118/0 (socket says 172.21.15.37:52118) 2024-03-29T08:27:48.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:48.209+0000 7f5dc9400700 1 -- 172.21.15.37:0/209234858 learned_addr learned my addr 172.21.15.37:0/209234858 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:27:48.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:48.209+0000 7f5dc9e00700 1 --2- 172.21.15.37:0/209234858 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5dcc070300 0x7f5dcc115c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:48.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:48.209+0000 7f5dc9400700 1 -- 172.21.15.37:0/209234858 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5dcc070300 msgr2=0x7f5dcc115c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:48.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:48.209+0000 7f5dc9400700 1 --2- 172.21.15.37:0/209234858 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5dcc070300 0x7f5dcc115c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:48.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:48.209+0000 7f5dc9400700 1 -- 172.21.15.37:0/209234858 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dbc005040 con 0x7f5dcc070c10 2024-03-29T08:27:48.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:48.210+0000 7f5dc9400700 1 --2- 172.21.15.37:0/209234858 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5dcc070c10 0x7f5dcc1161d0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f5db800c8c0 tx=0x7f5db80044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:48.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:48.210+0000 7f5dc2000700 1 -- 172.21.15.37:0/209234858 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5db800a760 con 0x7f5dcc070c10 2024-03-29T08:27:48.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:48.210+0000 7f5dc2000700 1 -- 172.21.15.37:0/209234858 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f5db800ad70 con 0x7f5dcc070c10 2024-03-29T08:27:48.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:48.210+0000 7f5dd0c00700 1 -- 172.21.15.37:0/209234858 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5dcc10ff10 con 0x7f5dcc070c10 2024-03-29T08:27:48.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:48.210+0000 7f5dc2000700 1 -- 172.21.15.37:0/209234858 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5db80113a0 con 0x7f5dcc070c10 2024-03-29T08:27:48.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:48.210+0000 7f5dd0c00700 1 -- 172.21.15.37:0/209234858 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5dcc110450 con 0x7f5dcc070c10 2024-03-29T08:27:48.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:48.211+0000 7f5dd0c00700 1 -- 172.21.15.37:0/209234858 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5dcc071a40 con 0x7f5dcc070c10 2024-03-29T08:27:48.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:48.212+0000 7f5dc2000700 1 -- 172.21.15.37:0/209234858 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f5db8012400 con 0x7f5dcc070c10 2024-03-29T08:27:48.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:48.213+0000 7f5dc2000700 1 --2- 172.21.15.37:0/209234858 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f5db006cc30 0x7f5db006f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:48.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:48.213+0000 7f5dc2000700 1 -- 172.21.15.37:0/209234858 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f5db808bfe0 con 0x7f5dcc070c10 2024-03-29T08:27:48.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:48.213+0000 7f5dc9e00700 1 --2- 172.21.15.37:0/209234858 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f5db006cc30 0x7f5db006f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:48.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:48.214+0000 7f5dc9e00700 1 --2- 172.21.15.37:0/209234858 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f5db006cc30 0x7f5db006f0e0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f5dbc00b370 tx=0x7f5dbc009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:48.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:48.217+0000 7f5dc2000700 1 -- 172.21.15.37:0/209234858 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f5db8056060 con 0x7f5dcc070c10 2024-03-29T08:27:48.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:48.525+0000 7f5dd0c00700 1 -- 172.21.15.37:0/209234858 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f5dcc10bfd0 con 0x7f5dcc070c10 2024-03-29T08:27:48.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:48.526+0000 7f5dc2000700 1 -- 172.21.15.37:0/209234858 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+847 (secure 0 0 0) 0x7f5db8059680 con 0x7f5dcc070c10 2024-03-29T08:27:48.526 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-29T08:27:48.526 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": { 2024-03-29T08:27:48.526 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 2 2024-03-29T08:27:48.526 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:27:48.526 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": { 2024-03-29T08:27:48.526 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 2 2024-03-29T08:27:48.526 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:27:48.527 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": { 2024-03-29T08:27:48.527 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 6 2024-03-29T08:27:48.527 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:27:48.527 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": { 2024-03-29T08:27:48.527 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 17.2.7-789-g4213e17c (4213e17c4de67016ed14f55d22503f4923b3a901) quincy (stable)": 1, 2024-03-29T08:27:48.527 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 3 2024-03-29T08:27:48.527 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:27:48.527 INFO:teuthology.orchestra.run.smithi037.stdout: "overall": { 2024-03-29T08:27:48.527 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 17.2.7-789-g4213e17c (4213e17c4de67016ed14f55d22503f4923b3a901) quincy (stable)": 1, 2024-03-29T08:27:48.527 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 13 2024-03-29T08:27:48.527 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-29T08:27:48.527 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-29T08:27:48.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:48.528+0000 7f5dd0c00700 1 -- 172.21.15.37:0/209234858 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f5db006cc30 msgr2=0x7f5db006f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:48.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:48.528+0000 7f5dd0c00700 1 --2- 172.21.15.37:0/209234858 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f5db006cc30 0x7f5db006f0e0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f5dbc00b370 tx=0x7f5dbc009e30 comp rx=0 tx=0).stop 2024-03-29T08:27:48.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:48.528+0000 7f5dd0c00700 1 -- 172.21.15.37:0/209234858 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5dcc070c10 msgr2=0x7f5dcc1161d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:48.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:48.528+0000 7f5dd0c00700 1 --2- 172.21.15.37:0/209234858 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5dcc070c10 0x7f5dcc1161d0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f5db800c8c0 tx=0x7f5db80044c0 comp rx=0 tx=0).stop 2024-03-29T08:27:48.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:48.528+0000 7f5dd0c00700 1 -- 172.21.15.37:0/209234858 shutdown_connections 2024-03-29T08:27:48.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:48.528+0000 7f5dd0c00700 1 --2- 172.21.15.37:0/209234858 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f5db006cc30 0x7f5db006f0e0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:48.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:48.528+0000 7f5dd0c00700 1 --2- 172.21.15.37:0/209234858 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5dcc070300 0x7f5dcc115c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:48.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:48.528+0000 7f5dd0c00700 1 --2- 172.21.15.37:0/209234858 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5dcc070c10 0x7f5dcc1161d0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:48.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:48.528+0000 7f5dd0c00700 1 -- 172.21.15.37:0/209234858 >> 172.21.15.37:0/209234858 conn(0x7f5dcc0f8970 msgr2=0x7f5dcc0f9ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:27:48.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:48.528+0000 7f5dd0c00700 1 -- 172.21.15.37:0/209234858 shutdown_connections 2024-03-29T08:27:48.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:48.528+0000 7f5dd0c00700 1 -- 172.21.15.37:0/209234858 wait complete. 2024-03-29T08:27:48.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:48.666+0000 7fd99de00700 1 -- 172.21.15.37:0/2006545387 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd998102e00 msgr2=0x7fd9981031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:48.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:48.666+0000 7fd99de00700 1 --2- 172.21.15.37:0/2006545387 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd998102e00 0x7fd9981031d0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fd988004940 tx=0x7fd988009e30 comp rx=0 tx=0).stop 2024-03-29T08:27:48.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:48.666+0000 7fd99de00700 1 -- 172.21.15.37:0/2006545387 shutdown_connections 2024-03-29T08:27:48.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:48.666+0000 7fd99de00700 1 --2- 172.21.15.37:0/2006545387 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd9981037a0 0x7fd9980fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:48.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:48.666+0000 7fd99de00700 1 --2- 172.21.15.37:0/2006545387 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd998102e00 0x7fd9981031d0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:48.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:48.666+0000 7fd99de00700 1 -- 172.21.15.37:0/2006545387 >> 172.21.15.37:0/2006545387 conn(0x7fd9980f89b0 msgr2=0x7fd9980fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:27:48.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:48.666+0000 7fd99de00700 1 -- 172.21.15.37:0/2006545387 shutdown_connections 2024-03-29T08:27:48.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:48.666+0000 7fd99de00700 1 -- 172.21.15.37:0/2006545387 wait complete. 2024-03-29T08:27:48.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:48.667+0000 7fd99de00700 1 Processor -- start 2024-03-29T08:27:48.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:48.667+0000 7fd99de00700 1 -- start start 2024-03-29T08:27:48.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:48.668+0000 7fd99de00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd998102e00 0x7fd9980702c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:48.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:48.668+0000 7fd99de00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd9981037a0 0x7fd998070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:48.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:48.668+0000 7fd99de00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd998070d90 con 0x7fd9981037a0 2024-03-29T08:27:48.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:48.668+0000 7fd99de00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd998070ed0 con 0x7fd998102e00 2024-03-29T08:27:48.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:48.668+0000 7fd996000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd9981037a0 0x7fd998070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:48.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:48.668+0000 7fd996000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd9981037a0 0x7fd998070800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52126/0 (socket says 172.21.15.37:52126) 2024-03-29T08:27:48.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:48.668+0000 7fd996000700 1 -- 172.21.15.37:0/186915732 learned_addr learned my addr 172.21.15.37:0/186915732 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:27:48.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:48.668+0000 7fd996a00700 1 --2- 172.21.15.37:0/186915732 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd998102e00 0x7fd9980702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:48.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:48.669+0000 7fd996000700 1 -- 172.21.15.37:0/186915732 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd998102e00 msgr2=0x7fd9980702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:48.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:48.669+0000 7fd996000700 1 --2- 172.21.15.37:0/186915732 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd998102e00 0x7fd9980702c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:48.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:48.669+0000 7fd996000700 1 -- 172.21.15.37:0/186915732 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd988005040 con 0x7fd9981037a0 2024-03-29T08:27:48.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:48.669+0000 7fd996000700 1 --2- 172.21.15.37:0/186915732 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd9981037a0 0x7fd998070800 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fd98400c890 tx=0x7fd9840044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:48.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:48.669+0000 7fd98ea00700 1 -- 172.21.15.37:0/186915732 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd98400a700 con 0x7fd9981037a0 2024-03-29T08:27:48.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:48.669+0000 7fd98ea00700 1 -- 172.21.15.37:0/186915732 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd98400ad10 con 0x7fd9981037a0 2024-03-29T08:27:48.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:48.669+0000 7fd99de00700 1 -- 172.21.15.37:0/186915732 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd998119e20 con 0x7fd9981037a0 2024-03-29T08:27:48.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:48.669+0000 7fd98ea00700 1 -- 172.21.15.37:0/186915732 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd984012e70 con 0x7fd9981037a0 2024-03-29T08:27:48.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:48.669+0000 7fd99de00700 1 -- 172.21.15.37:0/186915732 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd99811a2a0 con 0x7fd9981037a0 2024-03-29T08:27:48.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:48.670+0000 7fd99de00700 1 -- 172.21.15.37:0/186915732 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9980fe550 con 0x7fd9981037a0 2024-03-29T08:27:48.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:48.672+0000 7fd98ea00700 1 -- 172.21.15.37:0/186915732 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fd98400a860 con 0x7fd9981037a0 2024-03-29T08:27:48.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:48.672+0000 7fd98ea00700 1 --2- 172.21.15.37:0/186915732 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fd97c06cc30 0x7fd97c06f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:48.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:48.672+0000 7fd98ea00700 1 -- 172.21.15.37:0/186915732 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fd98408c080 con 0x7fd9981037a0 2024-03-29T08:27:48.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:48.673+0000 7fd996a00700 1 --2- 172.21.15.37:0/186915732 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fd97c06cc30 0x7fd97c06f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:48.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:48.673+0000 7fd996a00700 1 --2- 172.21.15.37:0/186915732 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fd97c06cc30 0x7fd97c06f0e0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fd98800b840 tx=0x7fd988009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:48.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:48.677+0000 7fd98ea00700 1 -- 172.21.15.37:0/186915732 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fd9840560c0 con 0x7fd9981037a0 2024-03-29T08:27:48.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:27:48 smithi148 ceph-mon[106692]: from='client.34952 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:27:48.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:27:48 smithi148 ceph-mon[106692]: from='client.34956 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:27:48.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:27:48 smithi037 ceph-mon[142325]: from='client.34952 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:27:48.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:27:48 smithi037 ceph-mon[142325]: from='client.34956 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:27:48.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:48.940+0000 7fd99de00700 1 -- 172.21.15.37:0/186915732 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fd99804bed0 con 0x7fd9981037a0 2024-03-29T08:27:48.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:48.940+0000 7fd98ea00700 1 -- 172.21.15.37:0/186915732 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 28 v28) v1 ==== 76+0+1841 (secure 0 0 0) 0x7fd9840596e0 con 0x7fd9981037a0 2024-03-29T08:27:48.941 INFO:teuthology.orchestra.run.smithi037.stdout:e28 2024-03-29T08:27:48.941 INFO:teuthology.orchestra.run.smithi037.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-29T08:27:48.941 INFO:teuthology.orchestra.run.smithi037.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T08:27:48.942 INFO:teuthology.orchestra.run.smithi037.stdout:legacy client fscid: 1 2024-03-29T08:27:48.942 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:27:48.942 INFO:teuthology.orchestra.run.smithi037.stdout:Filesystem 'cephfs' (1) 2024-03-29T08:27:48.942 INFO:teuthology.orchestra.run.smithi037.stdout:fs_name cephfs 2024-03-29T08:27:48.943 INFO:teuthology.orchestra.run.smithi037.stdout:epoch 21 2024-03-29T08:27:48.943 INFO:teuthology.orchestra.run.smithi037.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-03-29T08:27:48.943 INFO:teuthology.orchestra.run.smithi037.stdout:created 2024-03-29T08:03:29.241269+0000 2024-03-29T08:27:48.943 INFO:teuthology.orchestra.run.smithi037.stdout:modified 2024-03-29T08:17:16.237238+0000 2024-03-29T08:27:48.943 INFO:teuthology.orchestra.run.smithi037.stdout:tableserver 0 2024-03-29T08:27:48.943 INFO:teuthology.orchestra.run.smithi037.stdout:root 0 2024-03-29T08:27:48.944 INFO:teuthology.orchestra.run.smithi037.stdout:session_timeout 60 2024-03-29T08:27:48.944 INFO:teuthology.orchestra.run.smithi037.stdout:session_autoclose 300 2024-03-29T08:27:48.944 INFO:teuthology.orchestra.run.smithi037.stdout:max_file_size 1099511627776 2024-03-29T08:27:48.944 INFO:teuthology.orchestra.run.smithi037.stdout:max_xattr_size 65536 2024-03-29T08:27:48.944 INFO:teuthology.orchestra.run.smithi037.stdout:required_client_features {} 2024-03-29T08:27:48.944 INFO:teuthology.orchestra.run.smithi037.stdout:last_failure 0 2024-03-29T08:27:48.945 INFO:teuthology.orchestra.run.smithi037.stdout:last_failure_osd_epoch 82 2024-03-29T08:27:48.945 INFO:teuthology.orchestra.run.smithi037.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T08:27:48.945 INFO:teuthology.orchestra.run.smithi037.stdout:max_mds 2 2024-03-29T08:27:48.945 INFO:teuthology.orchestra.run.smithi037.stdout:in 0,1 2024-03-29T08:27:48.945 INFO:teuthology.orchestra.run.smithi037.stdout:up {} 2024-03-29T08:27:48.945 INFO:teuthology.orchestra.run.smithi037.stdout:failed 0,1 2024-03-29T08:27:48.946 INFO:teuthology.orchestra.run.smithi037.stdout:damaged 2024-03-29T08:27:48.946 INFO:teuthology.orchestra.run.smithi037.stdout:stopped 2024-03-29T08:27:48.946 INFO:teuthology.orchestra.run.smithi037.stdout:data_pools [3] 2024-03-29T08:27:48.946 INFO:teuthology.orchestra.run.smithi037.stdout:metadata_pool 2 2024-03-29T08:27:48.946 INFO:teuthology.orchestra.run.smithi037.stdout:inline_data disabled 2024-03-29T08:27:48.946 INFO:teuthology.orchestra.run.smithi037.stdout:balancer 2024-03-29T08:27:48.946 INFO:teuthology.orchestra.run.smithi037.stdout:bal_rank_mask -1 2024-03-29T08:27:48.946 INFO:teuthology.orchestra.run.smithi037.stdout:standby_count_wanted 1 2024-03-29T08:27:48.950 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:27:48.950 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:27:48.950 INFO:teuthology.orchestra.run.smithi037.stdout:Standby daemons: 2024-03-29T08:27:48.950 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:27:48.950 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi148.qukpjh{-1:24319} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.148:6826/2641327912,v1:172.21.15.148:6827/2641327912] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:27:48.950 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi037.mmwgxe{-1:34440} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.37:6828/1248142005,v1:172.21.15.37:6829/1248142005] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:27:48.950 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi037.iifsmh{-1:34444} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.37:6826/2060882959,v1:172.21.15.37:6827/2060882959] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:27:48.950 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi148.sqteih{-1:44395} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.148:6824/4078652242,v1:172.21.15.148:6825/4078652242] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:27:48.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:48.942+0000 7fd99de00700 1 -- 172.21.15.37:0/186915732 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fd97c06cc30 msgr2=0x7fd97c06f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:48.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:48.943+0000 7fd99de00700 1 --2- 172.21.15.37:0/186915732 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fd97c06cc30 0x7fd97c06f0e0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fd98800b840 tx=0x7fd988009e30 comp rx=0 tx=0).stop 2024-03-29T08:27:48.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:48.943+0000 7fd99de00700 1 -- 172.21.15.37:0/186915732 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd9981037a0 msgr2=0x7fd998070800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:48.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:48.943+0000 7fd99de00700 1 --2- 172.21.15.37:0/186915732 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd9981037a0 0x7fd998070800 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fd98400c890 tx=0x7fd9840044c0 comp rx=0 tx=0).stop 2024-03-29T08:27:48.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:48.943+0000 7fd99de00700 1 -- 172.21.15.37:0/186915732 shutdown_connections 2024-03-29T08:27:48.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:48.943+0000 7fd99de00700 1 --2- 172.21.15.37:0/186915732 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fd97c06cc30 0x7fd97c06f0e0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:48.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:48.943+0000 7fd99de00700 1 --2- 172.21.15.37:0/186915732 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd998102e00 0x7fd9980702c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:48.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:48.943+0000 7fd99de00700 1 --2- 172.21.15.37:0/186915732 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd9981037a0 0x7fd998070800 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:48.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:48.943+0000 7fd99de00700 1 -- 172.21.15.37:0/186915732 >> 172.21.15.37:0/186915732 conn(0x7fd9980f89b0 msgr2=0x7fd9980f9a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:27:48.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:48.943+0000 7fd99de00700 1 -- 172.21.15.37:0/186915732 shutdown_connections 2024-03-29T08:27:48.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:48.943+0000 7fd99de00700 1 -- 172.21.15.37:0/186915732 wait complete. 2024-03-29T08:27:48.952 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 28 2024-03-29T08:27:49.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:49.080+0000 7f6f8b800700 1 -- 172.21.15.37:0/2052794333 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6f840708a0 msgr2=0x7f6f84070d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:49.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:49.080+0000 7f6f8b800700 1 --2- 172.21.15.37:0/2052794333 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6f840708a0 0x7f6f84070d10 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f6f70004970 tx=0x7f6f70009e30 comp rx=0 tx=0).stop 2024-03-29T08:27:49.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:49.080+0000 7f6f8b800700 1 -- 172.21.15.37:0/2052794333 shutdown_connections 2024-03-29T08:27:49.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:49.080+0000 7f6f8b800700 1 --2- 172.21.15.37:0/2052794333 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6f840708a0 0x7f6f84070d10 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:49.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:49.080+0000 7f6f8b800700 1 --2- 172.21.15.37:0/2052794333 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6f84071f90 0x7f6f84070360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:49.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:49.080+0000 7f6f8b800700 1 -- 172.21.15.37:0/2052794333 >> 172.21.15.37:0/2052794333 conn(0x7f6f840f89b0 msgr2=0x7f6f840fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:27:49.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:49.080+0000 7f6f8b800700 1 -- 172.21.15.37:0/2052794333 shutdown_connections 2024-03-29T08:27:49.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:49.080+0000 7f6f8b800700 1 -- 172.21.15.37:0/2052794333 wait complete. 2024-03-29T08:27:49.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:49.081+0000 7f6f8b800700 1 Processor -- start 2024-03-29T08:27:49.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:49.081+0000 7f6f8b800700 1 -- start start 2024-03-29T08:27:49.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:49.081+0000 7f6f8b800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6f840708a0 0x7f6f8411a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:49.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:49.081+0000 7f6f8b800700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6f84071f90 0x7f6f8411a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:49.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:49.081+0000 7f6f8b800700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f8411ab90 con 0x7f6f840708a0 2024-03-29T08:27:49.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:49.081+0000 7f6f8b800700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f84114010 con 0x7f6f84071f90 2024-03-29T08:27:49.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:49.082+0000 7f6f88e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6f840708a0 0x7f6f8411a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:49.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:49.082+0000 7f6f88e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6f840708a0 0x7f6f8411a000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52132/0 (socket says 172.21.15.37:52132) 2024-03-29T08:27:49.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:49.082+0000 7f6f88e00700 1 -- 172.21.15.37:0/160942706 learned_addr learned my addr 172.21.15.37:0/160942706 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:27:49.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:49.082+0000 7f6f88e00700 1 -- 172.21.15.37:0/160942706 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6f84071f90 msgr2=0x7f6f8411a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:49.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:49.082+0000 7f6f83e00700 1 --2- 172.21.15.37:0/160942706 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6f84071f90 0x7f6f8411a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:49.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:49.082+0000 7f6f88e00700 1 --2- 172.21.15.37:0/160942706 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6f84071f90 0x7f6f8411a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:49.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:49.082+0000 7f6f88e00700 1 -- 172.21.15.37:0/160942706 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f70005040 con 0x7f6f840708a0 2024-03-29T08:27:49.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:49.082+0000 7f6f83e00700 1 --2- 172.21.15.37:0/160942706 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6f84071f90 0x7f6f8411a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:27:49.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:49.082+0000 7f6f88e00700 1 --2- 172.21.15.37:0/160942706 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6f840708a0 0x7f6f8411a000 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f6f78002a50 tx=0x7f6f780046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:49.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:49.083+0000 7f6f80c00700 1 -- 172.21.15.37:0/160942706 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6f78007cc0 con 0x7f6f840708a0 2024-03-29T08:27:49.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:49.083+0000 7f6f80c00700 1 -- 172.21.15.37:0/160942706 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f6f7800f040 con 0x7f6f840708a0 2024-03-29T08:27:49.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:49.083+0000 7f6f80c00700 1 -- 172.21.15.37:0/160942706 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6f78009540 con 0x7f6f840708a0 2024-03-29T08:27:49.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:49.083+0000 7f6f8b800700 1 -- 172.21.15.37:0/160942706 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f841142f0 con 0x7f6f840708a0 2024-03-29T08:27:49.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:49.083+0000 7f6f8b800700 1 -- 172.21.15.37:0/160942706 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f84114830 con 0x7f6f840708a0 2024-03-29T08:27:49.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:49.084+0000 7f6f8b800700 1 -- 172.21.15.37:0/160942706 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f84071aa0 con 0x7f6f840708a0 2024-03-29T08:27:49.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:49.085+0000 7f6f80c00700 1 -- 172.21.15.37:0/160942706 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f6f780096a0 con 0x7f6f840708a0 2024-03-29T08:27:49.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:49.086+0000 7f6f80c00700 1 --2- 172.21.15.37:0/160942706 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f6f6c06cc30 0x7f6f6c06f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:49.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:49.086+0000 7f6f80c00700 1 -- 172.21.15.37:0/160942706 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f6f7808bde0 con 0x7f6f840708a0 2024-03-29T08:27:49.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:49.086+0000 7f6f83e00700 1 --2- 172.21.15.37:0/160942706 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f6f6c06cc30 0x7f6f6c06f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:49.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:49.086+0000 7f6f83e00700 1 --2- 172.21.15.37:0/160942706 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f6f6c06cc30 0x7f6f6c06f0e0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f6f7000b800 tx=0x7f6f70009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:49.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:49.090+0000 7f6f80c00700 1 -- 172.21.15.37:0/160942706 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f6f78055da0 con 0x7f6f840708a0 2024-03-29T08:27:49.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:49.325+0000 7f6f8b800700 1 -- 172.21.15.37:0/160942706 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6f84110450 con 0x7f6f6c06cc30 2024-03-29T08:27:49.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:49.326+0000 7f6f80c00700 1 -- 172.21.15.37:0/160942706 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f6f84110450 con 0x7f6f6c06cc30 2024-03-29T08:27:49.327 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-29T08:27:49.327 INFO:teuthology.orchestra.run.smithi037.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59", 2024-03-29T08:27:49.327 INFO:teuthology.orchestra.run.smithi037.stdout: "in_progress": true, 2024-03-29T08:27:49.327 INFO:teuthology.orchestra.run.smithi037.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-29T08:27:49.327 INFO:teuthology.orchestra.run.smithi037.stdout: "services_complete": [ 2024-03-29T08:27:49.327 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr", 2024-03-29T08:27:49.327 INFO:teuthology.orchestra.run.smithi037.stdout: "mon", 2024-03-29T08:27:49.327 INFO:teuthology.orchestra.run.smithi037.stdout: "crash", 2024-03-29T08:27:49.327 INFO:teuthology.orchestra.run.smithi037.stdout: "osd" 2024-03-29T08:27:49.327 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-29T08:27:49.327 INFO:teuthology.orchestra.run.smithi037.stdout: "progress": "15/21 daemons upgraded", 2024-03-29T08:27:49.327 INFO:teuthology.orchestra.run.smithi037.stdout: "message": "Currently upgrading mds daemons", 2024-03-29T08:27:49.328 INFO:teuthology.orchestra.run.smithi037.stdout: "is_paused": false 2024-03-29T08:27:49.328 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-29T08:27:49.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:49.329+0000 7f6f8b800700 1 -- 172.21.15.37:0/160942706 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f6f6c06cc30 msgr2=0x7f6f6c06f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:49.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:49.329+0000 7f6f8b800700 1 --2- 172.21.15.37:0/160942706 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f6f6c06cc30 0x7f6f6c06f0e0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f6f7000b800 tx=0x7f6f70009e30 comp rx=0 tx=0).stop 2024-03-29T08:27:49.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:49.329+0000 7f6f8b800700 1 -- 172.21.15.37:0/160942706 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6f840708a0 msgr2=0x7f6f8411a000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:49.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:49.329+0000 7f6f8b800700 1 --2- 172.21.15.37:0/160942706 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6f840708a0 0x7f6f8411a000 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f6f78002a50 tx=0x7f6f780046b0 comp rx=0 tx=0).stop 2024-03-29T08:27:49.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:49.329+0000 7f6f8b800700 1 -- 172.21.15.37:0/160942706 shutdown_connections 2024-03-29T08:27:49.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:49.329+0000 7f6f8b800700 1 --2- 172.21.15.37:0/160942706 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f6f6c06cc30 0x7f6f6c06f0e0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:49.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:49.329+0000 7f6f8b800700 1 --2- 172.21.15.37:0/160942706 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6f840708a0 0x7f6f8411a000 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:49.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:49.329+0000 7f6f8b800700 1 --2- 172.21.15.37:0/160942706 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6f84071f90 0x7f6f8411a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:49.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:49.329+0000 7f6f8b800700 1 -- 172.21.15.37:0/160942706 >> 172.21.15.37:0/160942706 conn(0x7f6f840f89b0 msgr2=0x7f6f840f9b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:27:49.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:49.329+0000 7f6f8b800700 1 -- 172.21.15.37:0/160942706 shutdown_connections 2024-03-29T08:27:49.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:49.329+0000 7f6f8b800700 1 -- 172.21.15.37:0/160942706 wait complete. 2024-03-29T08:27:49.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:49.466+0000 7ffac2200700 1 -- 172.21.15.37:0/1706970446 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffabc0fe830 msgr2=0x7ffabc106f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:49.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:49.466+0000 7ffac2200700 1 --2- 172.21.15.37:0/1706970446 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffabc0fe830 0x7ffabc106f00 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7ffaa8004940 tx=0x7ffaa8009e30 comp rx=0 tx=0).stop 2024-03-29T08:27:49.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:49.466+0000 7ffac2200700 1 -- 172.21.15.37:0/1706970446 shutdown_connections 2024-03-29T08:27:49.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:49.466+0000 7ffac2200700 1 --2- 172.21.15.37:0/1706970446 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffabc0fe830 0x7ffabc106f00 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:49.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:49.466+0000 7ffac2200700 1 --2- 172.21.15.37:0/1706970446 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffabc0fde90 0x7ffabc0fe260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:49.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:49.466+0000 7ffac2200700 1 -- 172.21.15.37:0/1706970446 >> 172.21.15.37:0/1706970446 conn(0x7ffabc072bc0 msgr2=0x7ffabc072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:27:49.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:49.466+0000 7ffac2200700 1 -- 172.21.15.37:0/1706970446 shutdown_connections 2024-03-29T08:27:49.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:49.466+0000 7ffac2200700 1 -- 172.21.15.37:0/1706970446 wait complete. 2024-03-29T08:27:49.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:49.467+0000 7ffac2200700 1 Processor -- start 2024-03-29T08:27:49.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:49.467+0000 7ffac2200700 1 -- start start 2024-03-29T08:27:49.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:49.468+0000 7ffac2200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffabc0fde90 0x7ffabc115c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:49.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:49.468+0000 7ffac2200700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffabc0fe830 0x7ffabc1161b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:49.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:49.468+0000 7ffac2200700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffabc116800 con 0x7ffabc0fde90 2024-03-29T08:27:49.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:49.468+0000 7ffac2200700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffabc10fbd0 con 0x7ffabc0fe830 2024-03-29T08:27:49.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:49.468+0000 7ffabb400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffabc0fde90 0x7ffabc115c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:49.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:49.468+0000 7ffabb400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffabc0fde90 0x7ffabc115c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52150/0 (socket says 172.21.15.37:52150) 2024-03-29T08:27:49.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:49.468+0000 7ffabb400700 1 -- 172.21.15.37:0/2826129126 learned_addr learned my addr 172.21.15.37:0/2826129126 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:27:49.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:49.468+0000 7ffabaa00700 1 --2- 172.21.15.37:0/2826129126 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffabc0fe830 0x7ffabc1161b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:49.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:49.469+0000 7ffabb400700 1 -- 172.21.15.37:0/2826129126 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffabc0fe830 msgr2=0x7ffabc1161b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:49.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:49.469+0000 7ffabb400700 1 --2- 172.21.15.37:0/2826129126 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffabc0fe830 0x7ffabc1161b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:49.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:49.469+0000 7ffabb400700 1 -- 172.21.15.37:0/2826129126 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffaa8005040 con 0x7ffabc0fde90 2024-03-29T08:27:49.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:49.469+0000 7ffabb400700 1 --2- 172.21.15.37:0/2826129126 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffabc0fde90 0x7ffabc115c70 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7ffaac00eb40 tx=0x7ffaac00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:49.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:49.469+0000 7ffab3400700 1 -- 172.21.15.37:0/2826129126 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffaac007cc0 con 0x7ffabc0fde90 2024-03-29T08:27:49.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:49.469+0000 7ffab3400700 1 -- 172.21.15.37:0/2826129126 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ffaac00f040 con 0x7ffabc0fde90 2024-03-29T08:27:49.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:49.469+0000 7ffab3400700 1 -- 172.21.15.37:0/2826129126 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffaac00c990 con 0x7ffabc0fde90 2024-03-29T08:27:49.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:49.469+0000 7ffac2200700 1 -- 172.21.15.37:0/2826129126 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffabc10fe50 con 0x7ffabc0fde90 2024-03-29T08:27:49.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:49.469+0000 7ffac2200700 1 -- 172.21.15.37:0/2826129126 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffabc110390 con 0x7ffabc0fde90 2024-03-29T08:27:49.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:49.470+0000 7ffac2200700 1 -- 172.21.15.37:0/2826129126 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffabc106a20 con 0x7ffabc0fde90 2024-03-29T08:27:49.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:49.471+0000 7ffab3400700 1 -- 172.21.15.37:0/2826129126 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7ffaac00caf0 con 0x7ffabc0fde90 2024-03-29T08:27:49.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:49.472+0000 7ffab3400700 1 --2- 172.21.15.37:0/2826129126 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7ffaa006cc30 0x7ffaa006f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:49.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:49.472+0000 7ffab3400700 1 -- 172.21.15.37:0/2826129126 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7ffaac08bc00 con 0x7ffabc0fde90 2024-03-29T08:27:49.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:49.473+0000 7ffabaa00700 1 --2- 172.21.15.37:0/2826129126 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7ffaa006cc30 0x7ffaa006f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:49.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:49.473+0000 7ffabaa00700 1 --2- 172.21.15.37:0/2826129126 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7ffaa006cc30 0x7ffaa006f0e0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7ffabc111510 tx=0x7ffaa8009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:49.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:49.477+0000 7ffab3400700 1 -- 172.21.15.37:0/2826129126 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7ffaac059700 con 0x7ffabc0fde90 2024-03-29T08:27:49.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:49.775+0000 7ffac2200700 1 -- 172.21.15.37:0/2826129126 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7ffabc04bed0 con 0x7ffabc0fde90 2024-03-29T08:27:49.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:49.776+0000 7ffab3400700 1 -- 172.21.15.37:0/2826129126 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7ffaac059290 con 0x7ffabc0fde90 2024-03-29T08:27:49.776 INFO:teuthology.orchestra.run.smithi037.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-03-29T08:27:49.776 INFO:teuthology.orchestra.run.smithi037.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-03-29T08:27:49.776 INFO:teuthology.orchestra.run.smithi037.stdout: fs cephfs is degraded 2024-03-29T08:27:49.777 INFO:teuthology.orchestra.run.smithi037.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-03-29T08:27:49.777 INFO:teuthology.orchestra.run.smithi037.stdout: fs cephfs is offline because no MDS is active for it. 2024-03-29T08:27:49.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:49.778+0000 7ffac2200700 1 -- 172.21.15.37:0/2826129126 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7ffaa006cc30 msgr2=0x7ffaa006f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:49.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:49.778+0000 7ffac2200700 1 --2- 172.21.15.37:0/2826129126 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7ffaa006cc30 0x7ffaa006f0e0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7ffabc111510 tx=0x7ffaa8009e30 comp rx=0 tx=0).stop 2024-03-29T08:27:49.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:49.778+0000 7ffac2200700 1 -- 172.21.15.37:0/2826129126 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffabc0fde90 msgr2=0x7ffabc115c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:49.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:49.778+0000 7ffac2200700 1 --2- 172.21.15.37:0/2826129126 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffabc0fde90 0x7ffabc115c70 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7ffaac00eb40 tx=0x7ffaac00b040 comp rx=0 tx=0).stop 2024-03-29T08:27:49.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:49.778+0000 7ffac2200700 1 -- 172.21.15.37:0/2826129126 shutdown_connections 2024-03-29T08:27:49.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:49.778+0000 7ffac2200700 1 --2- 172.21.15.37:0/2826129126 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7ffaa006cc30 0x7ffaa006f0e0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:49.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:49.778+0000 7ffac2200700 1 --2- 172.21.15.37:0/2826129126 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffabc0fde90 0x7ffabc115c70 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:49.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:49.778+0000 7ffac2200700 1 --2- 172.21.15.37:0/2826129126 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffabc0fe830 0x7ffabc1161b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:49.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:49.778+0000 7ffac2200700 1 -- 172.21.15.37:0/2826129126 >> 172.21.15.37:0/2826129126 conn(0x7ffabc072bc0 msgr2=0x7ffabc0fb6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:27:49.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:49.779+0000 7ffac2200700 1 -- 172.21.15.37:0/2826129126 shutdown_connections 2024-03-29T08:27:49.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:27:49.779+0000 7ffac2200700 1 -- 172.21.15.37:0/2826129126 wait complete. 2024-03-29T08:27:49.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:27:49 smithi148 ceph-mon[106692]: pgmap v507: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:27:49.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:27:49 smithi148 ceph-mon[106692]: from='client.34960 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:27:49.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:27:49 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/209234858' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:27:49.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:27:49 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/186915732' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T08:27:49.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:27:49 smithi037 ceph-mon[142325]: pgmap v507: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:27:49.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:27:49 smithi037 ceph-mon[142325]: from='client.34960 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:27:49.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:27:49 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/209234858' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:27:49.928 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:27:49 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/186915732' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T08:27:50.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:27:50 smithi148 ceph-mon[106692]: from='client.34972 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:27:50.807 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:27:50 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/2826129126' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T08:27:50.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:27:50 smithi037 ceph-mon[142325]: from='client.34972 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:27:50.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:27:50 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/2826129126' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T08:27:51.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:27:51 smithi148 ceph-mon[106692]: pgmap v508: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:27:51.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:27:51 smithi037 ceph-mon[142325]: pgmap v508: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:27:53.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:27:53 smithi148 ceph-mon[106692]: pgmap v509: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:27:53.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:27:53 smithi037 ceph-mon[142325]: pgmap v509: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:27:55.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:27:55 smithi148 ceph-mon[106692]: pgmap v510: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:27:55.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:27:55 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:27:55.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:27:55 smithi037 ceph-mon[142325]: pgmap v510: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:27:55.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:27:55 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:27:57.787 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:27:57 smithi148 ceph-mon[106692]: pgmap v511: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:27:57.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:27:57 smithi037 ceph-mon[142325]: pgmap v511: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:27:59.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:27:59 smithi148 ceph-mon[106692]: pgmap v512: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:27:59.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:27:59 smithi037 ceph-mon[142325]: pgmap v512: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:28:01.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:28:01 smithi148 ceph-mon[106692]: pgmap v513: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:28:01.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:28:01 smithi037 ceph-mon[142325]: pgmap v513: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:28:03.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:28:03 smithi148 ceph-mon[106692]: pgmap v514: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:28:03.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:28:03 smithi037 ceph-mon[142325]: pgmap v514: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:28:05.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:28:05 smithi148 ceph-mon[106692]: pgmap v515: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:28:05.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:28:05 smithi037 ceph-mon[142325]: pgmap v515: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:28:07.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:28:07 smithi148 ceph-mon[106692]: pgmap v516: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:28:07.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:28:07 smithi037 ceph-mon[142325]: pgmap v516: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:28:09.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:28:09 smithi037 ceph-mon[142325]: pgmap v517: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:28:10.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:28:09 smithi148 ceph-mon[106692]: pgmap v517: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:28:10.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:28:10 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:28:11.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:28:10 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:28:11.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:28:11 smithi037 ceph-mon[142325]: pgmap v518: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:28:12.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:28:11 smithi148 ceph-mon[106692]: pgmap v518: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:28:13.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:28:13 smithi037 ceph-mon[142325]: pgmap v519: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:28:14.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:28:13 smithi148 ceph-mon[106692]: pgmap v519: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:28:15.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:28:15 smithi037 ceph-mon[142325]: pgmap v520: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:28:16.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:28:15 smithi148 ceph-mon[106692]: pgmap v520: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:28:17.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:28:17 smithi037 ceph-mon[142325]: pgmap v521: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:28:18.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:28:17 smithi148 ceph-mon[106692]: pgmap v521: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:28:19.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:28:19 smithi037 ceph-mon[142325]: pgmap v522: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:28:19.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:19.932+0000 7fed4ba00700 1 -- 172.21.15.37:0/4180609847 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed44102e00 msgr2=0x7fed441031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:19.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:19.932+0000 7fed4ba00700 1 --2- 172.21.15.37:0/4180609847 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed44102e00 0x7fed441031d0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fed38004940 tx=0x7fed38009e30 comp rx=0 tx=0).stop 2024-03-29T08:28:19.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:19.932+0000 7fed4ba00700 1 -- 172.21.15.37:0/4180609847 shutdown_connections 2024-03-29T08:28:19.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:19.932+0000 7fed4ba00700 1 --2- 172.21.15.37:0/4180609847 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fed441037a0 0x7fed440fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:19.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:19.932+0000 7fed4ba00700 1 --2- 172.21.15.37:0/4180609847 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed44102e00 0x7fed441031d0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:19.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:19.932+0000 7fed4ba00700 1 -- 172.21.15.37:0/4180609847 >> 172.21.15.37:0/4180609847 conn(0x7fed440f89b0 msgr2=0x7fed440fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:28:19.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:19.932+0000 7fed4ba00700 1 -- 172.21.15.37:0/4180609847 shutdown_connections 2024-03-29T08:28:19.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:19.932+0000 7fed4ba00700 1 -- 172.21.15.37:0/4180609847 wait complete. 2024-03-29T08:28:19.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:19.933+0000 7fed4ba00700 1 Processor -- start 2024-03-29T08:28:19.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:19.933+0000 7fed4ba00700 1 -- start start 2024-03-29T08:28:19.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:19.934+0000 7fed4ba00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed44102e00 0x7fed44115cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:19.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:19.934+0000 7fed4ba00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fed441037a0 0x7fed44116210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:19.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:19.934+0000 7fed4ba00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed44116860 con 0x7fed44102e00 2024-03-29T08:28:19.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:19.934+0000 7fed4ba00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed4410fc30 con 0x7fed441037a0 2024-03-29T08:28:19.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:19.934+0000 7fed49000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed44102e00 0x7fed44115cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:19.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:19.934+0000 7fed49000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed44102e00 0x7fed44115cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57738/0 (socket says 172.21.15.37:57738) 2024-03-29T08:28:19.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:19.934+0000 7fed49000700 1 -- 172.21.15.37:0/1617554467 learned_addr learned my addr 172.21.15.37:0/1617554467 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:28:19.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:19.934+0000 7fed43e00700 1 --2- 172.21.15.37:0/1617554467 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fed441037a0 0x7fed44116210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:19.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:19.934+0000 7fed49000700 1 -- 172.21.15.37:0/1617554467 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fed441037a0 msgr2=0x7fed44116210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:19.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:19.934+0000 7fed49000700 1 --2- 172.21.15.37:0/1617554467 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fed441037a0 0x7fed44116210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:19.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:19.934+0000 7fed49000700 1 -- 172.21.15.37:0/1617554467 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed38005040 con 0x7fed44102e00 2024-03-29T08:28:19.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:19.935+0000 7fed49000700 1 --2- 172.21.15.37:0/1617554467 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed44102e00 0x7fed44115cd0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fed38009390 tx=0x7fed38009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:28:19.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:19.935+0000 7fed40c00700 1 -- 172.21.15.37:0/1617554467 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fed3800e040 con 0x7fed44102e00 2024-03-29T08:28:19.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:19.935+0000 7fed40c00700 1 -- 172.21.15.37:0/1617554467 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fed38013420 con 0x7fed44102e00 2024-03-29T08:28:19.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:19.935+0000 7fed40c00700 1 -- 172.21.15.37:0/1617554467 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fed3801c630 con 0x7fed44102e00 2024-03-29T08:28:19.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:19.935+0000 7fed4ba00700 1 -- 172.21.15.37:0/1617554467 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed4410feb0 con 0x7fed44102e00 2024-03-29T08:28:19.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:19.935+0000 7fed4ba00700 1 -- 172.21.15.37:0/1617554467 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed44110470 con 0x7fed44102e00 2024-03-29T08:28:19.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:19.936+0000 7fed4ba00700 1 -- 172.21.15.37:0/1617554467 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed440fe550 con 0x7fed44102e00 2024-03-29T08:28:19.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:19.937+0000 7fed40c00700 1 -- 172.21.15.37:0/1617554467 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fed380139e0 con 0x7fed44102e00 2024-03-29T08:28:19.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:19.938+0000 7fed40c00700 1 --2- 172.21.15.37:0/1617554467 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fed2c06cc30 0x7fed2c06f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:19.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:19.938+0000 7fed40c00700 1 -- 172.21.15.37:0/1617554467 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fed3808d340 con 0x7fed44102e00 2024-03-29T08:28:19.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:19.938+0000 7fed43e00700 1 --2- 172.21.15.37:0/1617554467 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fed2c06cc30 0x7fed2c06f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:19.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:19.939+0000 7fed43e00700 1 --2- 172.21.15.37:0/1617554467 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fed2c06cc30 0x7fed2c06f0e0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fed44111570 tx=0x7fed3000b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:28:19.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:19.943+0000 7fed40c00700 1 -- 172.21.15.37:0/1617554467 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fed3805ae10 con 0x7fed44102e00 2024-03-29T08:28:20.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:28:19 smithi148 ceph-mon[106692]: pgmap v522: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:28:20.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:20.177+0000 7fed4ba00700 1 -- 172.21.15.37:0/1617554467 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fed4404bf70 con 0x7fed2c06cc30 2024-03-29T08:28:20.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:20.178+0000 7fed40c00700 1 -- 172.21.15.37:0/1617554467 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fed3000f040 con 0x7fed2c06cc30 2024-03-29T08:28:20.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:20.180+0000 7fed4ba00700 1 -- 172.21.15.37:0/1617554467 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fed2c06cc30 msgr2=0x7fed2c06f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:20.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:20.180+0000 7fed4ba00700 1 --2- 172.21.15.37:0/1617554467 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fed2c06cc30 0x7fed2c06f0e0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fed44111570 tx=0x7fed3000b040 comp rx=0 tx=0).stop 2024-03-29T08:28:20.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:20.180+0000 7fed4ba00700 1 -- 172.21.15.37:0/1617554467 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed44102e00 msgr2=0x7fed44115cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:20.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:20.180+0000 7fed4ba00700 1 --2- 172.21.15.37:0/1617554467 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed44102e00 0x7fed44115cd0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fed38009390 tx=0x7fed38009e30 comp rx=0 tx=0).stop 2024-03-29T08:28:20.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:20.181+0000 7fed4ba00700 1 -- 172.21.15.37:0/1617554467 shutdown_connections 2024-03-29T08:28:20.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:20.181+0000 7fed4ba00700 1 --2- 172.21.15.37:0/1617554467 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fed2c06cc30 0x7fed2c06f0e0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:20.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:20.181+0000 7fed4ba00700 1 --2- 172.21.15.37:0/1617554467 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed44102e00 0x7fed44115cd0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:20.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:20.181+0000 7fed4ba00700 1 --2- 172.21.15.37:0/1617554467 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fed441037a0 0x7fed44116210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:20.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:20.181+0000 7fed4ba00700 1 -- 172.21.15.37:0/1617554467 >> 172.21.15.37:0/1617554467 conn(0x7fed440f89b0 msgr2=0x7fed440f99e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:28:20.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:20.181+0000 7fed4ba00700 1 -- 172.21.15.37:0/1617554467 shutdown_connections 2024-03-29T08:28:20.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:20.181+0000 7fed4ba00700 1 -- 172.21.15.37:0/1617554467 wait complete. 2024-03-29T08:28:20.195 INFO:teuthology.orchestra.run.smithi037.stdout:true 2024-03-29T08:28:20.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:20.318+0000 7f3407000700 1 -- 172.21.15.37:0/3904563295 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f34000fde90 msgr2=0x7f34000fe260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:20.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:20.318+0000 7f3407000700 1 --2- 172.21.15.37:0/3904563295 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f34000fde90 0x7f34000fe260 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f33f0004940 tx=0x7f33f0009e30 comp rx=0 tx=0).stop 2024-03-29T08:28:20.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:20.318+0000 7f3407000700 1 -- 172.21.15.37:0/3904563295 shutdown_connections 2024-03-29T08:28:20.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:20.318+0000 7f3407000700 1 --2- 172.21.15.37:0/3904563295 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f34000fe830 0x7f3400106f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:20.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:20.318+0000 7f3407000700 1 --2- 172.21.15.37:0/3904563295 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f34000fde90 0x7f34000fe260 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:20.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:20.318+0000 7f3407000700 1 -- 172.21.15.37:0/3904563295 >> 172.21.15.37:0/3904563295 conn(0x7f3400072bc0 msgr2=0x7f3400072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:28:20.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:20.319+0000 7f3407000700 1 -- 172.21.15.37:0/3904563295 shutdown_connections 2024-03-29T08:28:20.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:20.319+0000 7f3407000700 1 -- 172.21.15.37:0/3904563295 wait complete. 2024-03-29T08:28:20.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:20.319+0000 7f3407000700 1 Processor -- start 2024-03-29T08:28:20.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:20.320+0000 7f3407000700 1 -- start start 2024-03-29T08:28:20.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:20.320+0000 7f3407000700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f34000fde90 0x7f3400113af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:20.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:20.320+0000 7f3407000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f34000fe830 0x7f3400114030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:20.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:20.320+0000 7f3407000700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3400114680 con 0x7f34000fe830 2024-03-29T08:28:20.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:20.320+0000 7f3407000700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f340010da70 con 0x7f34000fde90 2024-03-29T08:28:20.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:20.320+0000 7f33ff400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f34000fe830 0x7f3400114030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:20.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:20.320+0000 7f33ff400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f34000fe830 0x7f3400114030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57748/0 (socket says 172.21.15.37:57748) 2024-03-29T08:28:20.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:20.320+0000 7f33ff400700 1 -- 172.21.15.37:0/2657977088 learned_addr learned my addr 172.21.15.37:0/2657977088 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:28:20.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:20.320+0000 7f33ffe00700 1 --2- 172.21.15.37:0/2657977088 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f34000fde90 0x7f3400113af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:20.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:20.321+0000 7f33ff400700 1 -- 172.21.15.37:0/2657977088 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f34000fde90 msgr2=0x7f3400113af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:20.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:20.321+0000 7f33ff400700 1 --2- 172.21.15.37:0/2657977088 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f34000fde90 0x7f3400113af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:20.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:20.321+0000 7f33ff400700 1 -- 172.21.15.37:0/2657977088 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33f0005040 con 0x7f34000fe830 2024-03-29T08:28:20.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:20.321+0000 7f33ff400700 1 --2- 172.21.15.37:0/2657977088 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f34000fe830 0x7f3400114030 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f33ec002a50 tx=0x7f33ec0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:28:20.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:20.321+0000 7f33f7e00700 1 -- 172.21.15.37:0/2657977088 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f33ec007cc0 con 0x7f34000fe830 2024-03-29T08:28:20.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:20.321+0000 7f33f7e00700 1 -- 172.21.15.37:0/2657977088 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f33ec00f040 con 0x7f34000fe830 2024-03-29T08:28:20.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:20.321+0000 7f3407000700 1 -- 172.21.15.37:0/2657977088 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f340010dd50 con 0x7f34000fe830 2024-03-29T08:28:20.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:20.321+0000 7f33f7e00700 1 -- 172.21.15.37:0/2657977088 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f33ec00d9c0 con 0x7f34000fe830 2024-03-29T08:28:20.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:20.321+0000 7f3407000700 1 -- 172.21.15.37:0/2657977088 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f340010e290 con 0x7f34000fe830 2024-03-29T08:28:20.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:20.322+0000 7f3407000700 1 -- 172.21.15.37:0/2657977088 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34001019b0 con 0x7f34000fe830 2024-03-29T08:28:20.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:20.324+0000 7f33f7e00700 1 -- 172.21.15.37:0/2657977088 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f33ec00db20 con 0x7f34000fe830 2024-03-29T08:28:20.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:20.325+0000 7f33f7e00700 1 --2- 172.21.15.37:0/2657977088 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f33e406cc30 0x7f33e406f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:20.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:20.325+0000 7f33f7e00700 1 -- 172.21.15.37:0/2657977088 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f33ec08bd20 con 0x7f34000fe830 2024-03-29T08:28:20.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:20.325+0000 7f33ffe00700 1 --2- 172.21.15.37:0/2657977088 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f33e406cc30 0x7f33e406f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:20.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:20.326+0000 7f33ffe00700 1 --2- 172.21.15.37:0/2657977088 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f33e406cc30 0x7f33e406f0e0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f33f000b3b0 tx=0x7f33f0009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:28:20.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:20.330+0000 7f33f7e00700 1 -- 172.21.15.37:0/2657977088 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f33ec059770 con 0x7f34000fe830 2024-03-29T08:28:20.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:20.564+0000 7f3407000700 1 -- 172.21.15.37:0/2657977088 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f34001137c0 con 0x7f33e406cc30 2024-03-29T08:28:20.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:20.565+0000 7f33f7e00700 1 -- 172.21.15.37:0/2657977088 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f34001137c0 con 0x7f33e406cc30 2024-03-29T08:28:20.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:20.568+0000 7f3407000700 1 -- 172.21.15.37:0/2657977088 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f33e406cc30 msgr2=0x7f33e406f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:20.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:20.568+0000 7f3407000700 1 --2- 172.21.15.37:0/2657977088 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f33e406cc30 0x7f33e406f0e0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f33f000b3b0 tx=0x7f33f0009e30 comp rx=0 tx=0).stop 2024-03-29T08:28:20.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:20.568+0000 7f3407000700 1 -- 172.21.15.37:0/2657977088 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f34000fe830 msgr2=0x7f3400114030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:20.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:20.568+0000 7f3407000700 1 --2- 172.21.15.37:0/2657977088 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f34000fe830 0x7f3400114030 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f33ec002a50 tx=0x7f33ec0046b0 comp rx=0 tx=0).stop 2024-03-29T08:28:20.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:20.568+0000 7f3407000700 1 -- 172.21.15.37:0/2657977088 shutdown_connections 2024-03-29T08:28:20.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:20.568+0000 7f3407000700 1 --2- 172.21.15.37:0/2657977088 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f33e406cc30 0x7f33e406f0e0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:20.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:20.568+0000 7f3407000700 1 --2- 172.21.15.37:0/2657977088 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f34000fde90 0x7f3400113af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:20.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:20.568+0000 7f3407000700 1 --2- 172.21.15.37:0/2657977088 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f34000fe830 0x7f3400114030 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:20.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:20.568+0000 7f3407000700 1 -- 172.21.15.37:0/2657977088 >> 172.21.15.37:0/2657977088 conn(0x7f3400072bc0 msgr2=0x7f34000ff3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:28:20.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:20.568+0000 7f3407000700 1 -- 172.21.15.37:0/2657977088 shutdown_connections 2024-03-29T08:28:20.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:20.568+0000 7f3407000700 1 -- 172.21.15.37:0/2657977088 wait complete. 2024-03-29T08:28:20.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:20.705+0000 7f751ca00700 1 -- 172.21.15.37:0/1199792246 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f75181037a0 msgr2=0x7f75180fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:20.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:20.705+0000 7f751ca00700 1 --2- 172.21.15.37:0/1199792246 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f75181037a0 0x7f75180fce10 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f7504004940 tx=0x7f7504009e30 comp rx=0 tx=0).stop 2024-03-29T08:28:20.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:20.706+0000 7f751ca00700 1 -- 172.21.15.37:0/1199792246 shutdown_connections 2024-03-29T08:28:20.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:20.706+0000 7f751ca00700 1 --2- 172.21.15.37:0/1199792246 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f75181037a0 0x7f75180fce10 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:20.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:20.706+0000 7f751ca00700 1 --2- 172.21.15.37:0/1199792246 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7518102e00 0x7f75181031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:20.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:20.706+0000 7f751ca00700 1 -- 172.21.15.37:0/1199792246 >> 172.21.15.37:0/1199792246 conn(0x7f75180f89b0 msgr2=0x7f75180fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:28:20.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:20.706+0000 7f751ca00700 1 -- 172.21.15.37:0/1199792246 shutdown_connections 2024-03-29T08:28:20.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:20.706+0000 7f751ca00700 1 -- 172.21.15.37:0/1199792246 wait complete. 2024-03-29T08:28:20.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:20.707+0000 7f751ca00700 1 Processor -- start 2024-03-29T08:28:20.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:20.707+0000 7f751ca00700 1 -- start start 2024-03-29T08:28:20.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:20.707+0000 7f751ca00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7518102e00 0x7f7518115d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:20.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:20.707+0000 7f751ca00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f75181037a0 0x7f7518116240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:20.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:20.707+0000 7f751ca00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7518116890 con 0x7f7518102e00 2024-03-29T08:28:20.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:20.707+0000 7f751ca00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f751810fc60 con 0x7f75181037a0 2024-03-29T08:28:20.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:20.707+0000 7f7515e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7518102e00 0x7f7518115d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:20.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:20.707+0000 7f7515e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7518102e00 0x7f7518115d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57772/0 (socket says 172.21.15.37:57772) 2024-03-29T08:28:20.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:20.708+0000 7f7515e00700 1 -- 172.21.15.37:0/631948662 learned_addr learned my addr 172.21.15.37:0/631948662 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:28:20.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:20.708+0000 7f7515400700 1 --2- 172.21.15.37:0/631948662 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f75181037a0 0x7f7518116240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:20.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:20.708+0000 7f7515e00700 1 -- 172.21.15.37:0/631948662 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f75181037a0 msgr2=0x7f7518116240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:20.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:20.708+0000 7f7515e00700 1 --2- 172.21.15.37:0/631948662 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f75181037a0 0x7f7518116240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:20.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:20.708+0000 7f7515e00700 1 -- 172.21.15.37:0/631948662 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7504005040 con 0x7f7518102e00 2024-03-29T08:28:20.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:20.708+0000 7f7515e00700 1 --2- 172.21.15.37:0/631948662 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7518102e00 0x7f7518115d00 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f7508002a50 tx=0x7f75080046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:28:20.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:20.708+0000 7f750e000700 1 -- 172.21.15.37:0/631948662 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7508007cc0 con 0x7f7518102e00 2024-03-29T08:28:20.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:20.709+0000 7f750e000700 1 -- 172.21.15.37:0/631948662 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f750800f040 con 0x7f7518102e00 2024-03-29T08:28:20.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:20.709+0000 7f751ca00700 1 -- 172.21.15.37:0/631948662 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f751810ff40 con 0x7f7518102e00 2024-03-29T08:28:20.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:20.709+0000 7f750e000700 1 -- 172.21.15.37:0/631948662 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7508009540 con 0x7f7518102e00 2024-03-29T08:28:20.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:20.709+0000 7f751ca00700 1 -- 172.21.15.37:0/631948662 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7518110500 con 0x7f7518102e00 2024-03-29T08:28:20.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:20.710+0000 7f751ca00700 1 -- 172.21.15.37:0/631948662 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75180fe550 con 0x7f7518102e00 2024-03-29T08:28:20.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:20.711+0000 7f750e000700 1 -- 172.21.15.37:0/631948662 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f75080096a0 con 0x7f7518102e00 2024-03-29T08:28:20.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:20.712+0000 7f750e000700 1 --2- 172.21.15.37:0/631948662 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f74fc06cc30 0x7f74fc06f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:20.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:20.712+0000 7f750e000700 1 -- 172.21.15.37:0/631948662 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f750808bee0 con 0x7f7518102e00 2024-03-29T08:28:20.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:20.712+0000 7f7515400700 1 --2- 172.21.15.37:0/631948662 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f74fc06cc30 0x7f74fc06f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:20.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:20.713+0000 7f7515400700 1 --2- 172.21.15.37:0/631948662 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f74fc06cc30 0x7f74fc06f0e0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f750400b020 tx=0x7f7504009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:28:20.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:20.716+0000 7f750e000700 1 -- 172.21.15.37:0/631948662 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f75080598b0 con 0x7f7518102e00 2024-03-29T08:28:20.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:20.941+0000 7f751ca00700 1 -- 172.21.15.37:0/631948662 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f751804bf40 con 0x7f74fc06cc30 2024-03-29T08:28:20.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:20.951+0000 7f750e000700 1 -- 172.21.15.37:0/631948662 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f751804bf40 con 0x7f74fc06cc30 2024-03-29T08:28:20.952 INFO:teuthology.orchestra.run.smithi037.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-29T08:28:20.952 INFO:teuthology.orchestra.run.smithi037.stdout:alertmanager.smithi037 smithi037 *:9093,9094 running (18m) 10m ago 31m 14.9M - 0.25.0 c8568f914cd2 2f9f9746ed0f 2024-03-29T08:28:20.952 INFO:teuthology.orchestra.run.smithi037.stdout:crash.smithi037 smithi037 running (15m) 10m ago 31m 6647k - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 178d416a3186 2024-03-29T08:28:20.952 INFO:teuthology.orchestra.run.smithi037.stdout:crash.smithi148 smithi148 running (14m) 10m ago 30m 6656k - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 23666989d58f 2024-03-29T08:28:20.952 INFO:teuthology.orchestra.run.smithi037.stdout:grafana.smithi037 smithi037 *:3000 running (17m) 10m ago 30m 71.9M - 9.4.7 954c08fa6188 6b976fe49952 2024-03-29T08:28:20.952 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi037.iifsmh smithi037 running (10m) 10m ago 24m 10.8M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 2d97f562cc5b 2024-03-29T08:28:20.952 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi037.mmwgxe smithi037 running (10m) 10m ago 24m 10.9M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff ec37037f46e8 2024-03-29T08:28:20.952 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi148.qukpjh smithi148 running (24m) 10m ago 24m 15.3M - 17.2.7-789-g4213e17c cf22eb8683be 8e469782a273 2024-03-29T08:28:20.952 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi148.sqteih smithi148 running (10m) 10m ago 24m 10.7M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 44c15c77037a 2024-03-29T08:28:20.952 INFO:teuthology.orchestra.run.smithi037.stdout:mgr.smithi037.wjikii smithi037 *:8443,9283,8765 running (20m) 10m ago 32m 506M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff c5a1417b9417 2024-03-29T08:28:20.952 INFO:teuthology.orchestra.run.smithi037.stdout:mgr.smithi148.onhgvm smithi148 *:8443,9283,8765 running (19m) 10m ago 29m 450M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff fcfdbce062bd 2024-03-29T08:28:20.953 INFO:teuthology.orchestra.run.smithi037.stdout:mon.smithi037 smithi037 running (16m) 10m ago 32m 54.8M 2048M 18.2.2-1015-gd0e392d9 1cc2f45c17ff eea2060783af 2024-03-29T08:28:20.953 INFO:teuthology.orchestra.run.smithi037.stdout:mon.smithi148 smithi148 running (15m) 10m ago 29m 43.7M 2048M 18.2.2-1015-gd0e392d9 1cc2f45c17ff ad4b2c5748be 2024-03-29T08:28:20.953 INFO:teuthology.orchestra.run.smithi037.stdout:node-exporter.smithi037 smithi037 *:9100 running (19m) 10m ago 31m 7625k - 1.5.0 0da6a335fe13 26b8ae9cb878 2024-03-29T08:28:20.953 INFO:teuthology.orchestra.run.smithi037.stdout:node-exporter.smithi148 smithi148 *:9100 running (18m) 10m ago 30m 8166k - 1.5.0 0da6a335fe13 98414e4e64ed 2024-03-29T08:28:20.953 INFO:teuthology.orchestra.run.smithi037.stdout:osd.0 smithi037 running (14m) 10m ago 29m 104M 1619M 18.2.2-1015-gd0e392d9 1cc2f45c17ff c0c584f6245c 2024-03-29T08:28:20.953 INFO:teuthology.orchestra.run.smithi037.stdout:osd.1 smithi037 running (13m) 10m ago 28m 100M 1619M 18.2.2-1015-gd0e392d9 1cc2f45c17ff a4a2ff4cc83d 2024-03-29T08:28:20.953 INFO:teuthology.orchestra.run.smithi037.stdout:osd.2 smithi037 running (13m) 10m ago 27m 82.3M 1619M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 8b1bc71aa79e 2024-03-29T08:28:20.953 INFO:teuthology.orchestra.run.smithi037.stdout:osd.3 smithi148 running (12m) 10m ago 27m 105M 2642M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 1c656ab8ff86 2024-03-29T08:28:20.953 INFO:teuthology.orchestra.run.smithi037.stdout:osd.4 smithi148 running (11m) 10m ago 26m 97.3M 2642M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 966a9de11d22 2024-03-29T08:28:20.953 INFO:teuthology.orchestra.run.smithi037.stdout:osd.5 smithi148 running (11m) 10m ago 26m 85.8M 2642M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 474c4f9212ce 2024-03-29T08:28:20.953 INFO:teuthology.orchestra.run.smithi037.stdout:prometheus.smithi037 smithi037 *:9095 running (18m) 10m ago 30m 54.4M - 2.43.0 a07b618ecd1d 30c0223ec589 2024-03-29T08:28:20.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:20.954+0000 7f751ca00700 1 -- 172.21.15.37:0/631948662 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f74fc06cc30 msgr2=0x7f74fc06f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:20.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:20.954+0000 7f751ca00700 1 --2- 172.21.15.37:0/631948662 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f74fc06cc30 0x7f74fc06f0e0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f750400b020 tx=0x7f7504009e30 comp rx=0 tx=0).stop 2024-03-29T08:28:20.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:20.954+0000 7f751ca00700 1 -- 172.21.15.37:0/631948662 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7518102e00 msgr2=0x7f7518115d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:20.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:20.954+0000 7f751ca00700 1 --2- 172.21.15.37:0/631948662 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7518102e00 0x7f7518115d00 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f7508002a50 tx=0x7f75080046b0 comp rx=0 tx=0).stop 2024-03-29T08:28:20.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:20.954+0000 7f751ca00700 1 -- 172.21.15.37:0/631948662 shutdown_connections 2024-03-29T08:28:20.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:20.954+0000 7f751ca00700 1 --2- 172.21.15.37:0/631948662 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f74fc06cc30 0x7f74fc06f0e0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:20.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:20.954+0000 7f751ca00700 1 --2- 172.21.15.37:0/631948662 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7518102e00 0x7f7518115d00 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:20.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:20.954+0000 7f751ca00700 1 --2- 172.21.15.37:0/631948662 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f75181037a0 0x7f7518116240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:20.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:20.954+0000 7f751ca00700 1 -- 172.21.15.37:0/631948662 >> 172.21.15.37:0/631948662 conn(0x7f75180f89b0 msgr2=0x7f75180f9ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:28:20.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:20.954+0000 7f751ca00700 1 -- 172.21.15.37:0/631948662 shutdown_connections 2024-03-29T08:28:20.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:20.954+0000 7f751ca00700 1 -- 172.21.15.37:0/631948662 wait complete. 2024-03-29T08:28:21.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:21.091+0000 7ff7fbc00700 1 -- 172.21.15.37:0/3298052651 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff7f41037a0 msgr2=0x7ff7f40fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:21.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:21.091+0000 7ff7fbc00700 1 --2- 172.21.15.37:0/3298052651 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff7f41037a0 0x7ff7f40fce10 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7ff7e0004940 tx=0x7ff7e0009e30 comp rx=0 tx=0).stop 2024-03-29T08:28:21.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:21.091+0000 7ff7fbc00700 1 -- 172.21.15.37:0/3298052651 shutdown_connections 2024-03-29T08:28:21.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:21.091+0000 7ff7fbc00700 1 --2- 172.21.15.37:0/3298052651 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff7f41037a0 0x7ff7f40fce10 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:21.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:21.091+0000 7ff7fbc00700 1 --2- 172.21.15.37:0/3298052651 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff7f4102e00 0x7ff7f41031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:21.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:21.091+0000 7ff7fbc00700 1 -- 172.21.15.37:0/3298052651 >> 172.21.15.37:0/3298052651 conn(0x7ff7f40f89b0 msgr2=0x7ff7f40fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:28:21.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:21.091+0000 7ff7fbc00700 1 -- 172.21.15.37:0/3298052651 shutdown_connections 2024-03-29T08:28:21.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:21.091+0000 7ff7fbc00700 1 -- 172.21.15.37:0/3298052651 wait complete. 2024-03-29T08:28:21.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:21.092+0000 7ff7fbc00700 1 Processor -- start 2024-03-29T08:28:21.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:21.092+0000 7ff7fbc00700 1 -- start start 2024-03-29T08:28:21.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:21.092+0000 7ff7fbc00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff7f4102e00 0x7ff7f4115c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:21.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:21.093+0000 7ff7fbc00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff7f41037a0 0x7ff7f41161c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:21.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:21.093+0000 7ff7fbc00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7f4116810 con 0x7ff7f41037a0 2024-03-29T08:28:21.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:21.093+0000 7ff7fbc00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7f410fbe0 con 0x7ff7f4102e00 2024-03-29T08:28:21.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:21.093+0000 7ff7f3e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff7f41037a0 0x7ff7f41161c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:21.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:21.093+0000 7ff7f3e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff7f41037a0 0x7ff7f41161c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57794/0 (socket says 172.21.15.37:57794) 2024-03-29T08:28:21.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:21.093+0000 7ff7f3e00700 1 -- 172.21.15.37:0/3620230564 learned_addr learned my addr 172.21.15.37:0/3620230564 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:28:21.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:21.093+0000 7ff7f9200700 1 --2- 172.21.15.37:0/3620230564 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff7f4102e00 0x7ff7f4115c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:21.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:21.093+0000 7ff7f3e00700 1 -- 172.21.15.37:0/3620230564 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff7f4102e00 msgr2=0x7ff7f4115c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:21.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:21.093+0000 7ff7f3e00700 1 --2- 172.21.15.37:0/3620230564 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff7f4102e00 0x7ff7f4115c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:21.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:21.093+0000 7ff7f3e00700 1 -- 172.21.15.37:0/3620230564 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7e0005040 con 0x7ff7f41037a0 2024-03-29T08:28:21.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:21.094+0000 7ff7f3e00700 1 --2- 172.21.15.37:0/3620230564 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff7f41037a0 0x7ff7f41161c0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7ff7e0009390 tx=0x7ff7e0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:28:21.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:21.094+0000 7ff7f0c00700 1 -- 172.21.15.37:0/3620230564 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff7e000e040 con 0x7ff7f41037a0 2024-03-29T08:28:21.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:21.094+0000 7ff7fbc00700 1 -- 172.21.15.37:0/3620230564 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7f410fe60 con 0x7ff7f41037a0 2024-03-29T08:28:21.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:21.094+0000 7ff7f0c00700 1 -- 172.21.15.37:0/3620230564 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ff7e0013420 con 0x7ff7f41037a0 2024-03-29T08:28:21.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:21.094+0000 7ff7fbc00700 1 -- 172.21.15.37:0/3620230564 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7f4110420 con 0x7ff7f41037a0 2024-03-29T08:28:21.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:21.094+0000 7ff7f0c00700 1 -- 172.21.15.37:0/3620230564 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff7e00255e0 con 0x7ff7f41037a0 2024-03-29T08:28:21.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:21.095+0000 7ff7fbc00700 1 -- 172.21.15.37:0/3620230564 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7f40fe550 con 0x7ff7f41037a0 2024-03-29T08:28:21.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:21.096+0000 7ff7f0c00700 1 -- 172.21.15.37:0/3620230564 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7ff7e0013a00 con 0x7ff7f41037a0 2024-03-29T08:28:21.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:21.097+0000 7ff7f0c00700 1 --2- 172.21.15.37:0/3620230564 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7ff7dc06cc30 0x7ff7dc06f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:21.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:21.097+0000 7ff7f0c00700 1 -- 172.21.15.37:0/3620230564 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7ff7e008d350 con 0x7ff7f41037a0 2024-03-29T08:28:21.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:21.097+0000 7ff7f9200700 1 --2- 172.21.15.37:0/3620230564 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7ff7dc06cc30 0x7ff7dc06f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:21.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:21.098+0000 7ff7f9200700 1 --2- 172.21.15.37:0/3620230564 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7ff7dc06cc30 0x7ff7dc06f0e0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7ff7e8009fd0 tx=0x7ff7e8009f60 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:28:21.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:21.101+0000 7ff7f0c00700 1 -- 172.21.15.37:0/3620230564 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7ff7e005ada0 con 0x7ff7f41037a0 2024-03-29T08:28:21.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:21.403+0000 7ff7fbc00700 1 -- 172.21.15.37:0/3620230564 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7ff7f4111300 con 0x7ff7f41037a0 2024-03-29T08:28:21.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:21.404+0000 7ff7f0c00700 1 -- 172.21.15.37:0/3620230564 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+847 (secure 0 0 0) 0x7ff7e005a930 con 0x7ff7f41037a0 2024-03-29T08:28:21.404 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-29T08:28:21.405 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": { 2024-03-29T08:28:21.405 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 2 2024-03-29T08:28:21.405 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:28:21.405 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": { 2024-03-29T08:28:21.405 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 2 2024-03-29T08:28:21.405 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:28:21.405 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": { 2024-03-29T08:28:21.405 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 6 2024-03-29T08:28:21.405 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:28:21.405 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": { 2024-03-29T08:28:21.405 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 17.2.7-789-g4213e17c (4213e17c4de67016ed14f55d22503f4923b3a901) quincy (stable)": 1, 2024-03-29T08:28:21.405 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 3 2024-03-29T08:28:21.405 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:28:21.406 INFO:teuthology.orchestra.run.smithi037.stdout: "overall": { 2024-03-29T08:28:21.406 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 17.2.7-789-g4213e17c (4213e17c4de67016ed14f55d22503f4923b3a901) quincy (stable)": 1, 2024-03-29T08:28:21.406 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 13 2024-03-29T08:28:21.406 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-29T08:28:21.406 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-29T08:28:21.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:21.406+0000 7ff7fbc00700 1 -- 172.21.15.37:0/3620230564 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7ff7dc06cc30 msgr2=0x7ff7dc06f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:21.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:21.406+0000 7ff7fbc00700 1 --2- 172.21.15.37:0/3620230564 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7ff7dc06cc30 0x7ff7dc06f0e0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7ff7e8009fd0 tx=0x7ff7e8009f60 comp rx=0 tx=0).stop 2024-03-29T08:28:21.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:21.406+0000 7ff7fbc00700 1 -- 172.21.15.37:0/3620230564 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff7f41037a0 msgr2=0x7ff7f41161c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:21.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:21.406+0000 7ff7fbc00700 1 --2- 172.21.15.37:0/3620230564 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff7f41037a0 0x7ff7f41161c0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7ff7e0009390 tx=0x7ff7e0009e30 comp rx=0 tx=0).stop 2024-03-29T08:28:21.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:21.407+0000 7ff7fbc00700 1 -- 172.21.15.37:0/3620230564 shutdown_connections 2024-03-29T08:28:21.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:21.407+0000 7ff7fbc00700 1 --2- 172.21.15.37:0/3620230564 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7ff7dc06cc30 0x7ff7dc06f0e0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:21.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:21.407+0000 7ff7fbc00700 1 --2- 172.21.15.37:0/3620230564 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff7f4102e00 0x7ff7f4115c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:21.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:21.407+0000 7ff7fbc00700 1 --2- 172.21.15.37:0/3620230564 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff7f41037a0 0x7ff7f41161c0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:21.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:21.407+0000 7ff7fbc00700 1 -- 172.21.15.37:0/3620230564 >> 172.21.15.37:0/3620230564 conn(0x7ff7f40f89b0 msgr2=0x7ff7f40f99a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:28:21.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:21.407+0000 7ff7fbc00700 1 -- 172.21.15.37:0/3620230564 shutdown_connections 2024-03-29T08:28:21.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:21.407+0000 7ff7fbc00700 1 -- 172.21.15.37:0/3620230564 wait complete. 2024-03-29T08:28:21.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:21.545+0000 7f4ee1e00700 1 -- 172.21.15.37:0/1422614228 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4edc105990 msgr2=0x7f4edc0ff000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:21.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:21.545+0000 7f4ee1e00700 1 --2- 172.21.15.37:0/1422614228 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4edc105990 0x7f4edc0ff000 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f4ec8004940 tx=0x7f4ec8009e30 comp rx=0 tx=0).stop 2024-03-29T08:28:21.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:21.545+0000 7f4ee1e00700 1 -- 172.21.15.37:0/1422614228 shutdown_connections 2024-03-29T08:28:21.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:21.545+0000 7f4ee1e00700 1 --2- 172.21.15.37:0/1422614228 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4edc105990 0x7f4edc0ff000 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:21.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:21.545+0000 7f4ee1e00700 1 --2- 172.21.15.37:0/1422614228 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4edc104ff0 0x7f4edc1053c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:21.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:21.545+0000 7f4ee1e00700 1 -- 172.21.15.37:0/1422614228 >> 172.21.15.37:0/1422614228 conn(0x7f4edc0faba0 msgr2=0x7f4edc0fcfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:28:21.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:21.545+0000 7f4ee1e00700 1 -- 172.21.15.37:0/1422614228 shutdown_connections 2024-03-29T08:28:21.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:21.545+0000 7f4ee1e00700 1 -- 172.21.15.37:0/1422614228 wait complete. 2024-03-29T08:28:21.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:21.546+0000 7f4ee1e00700 1 Processor -- start 2024-03-29T08:28:21.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:21.546+0000 7f4ee1e00700 1 -- start start 2024-03-29T08:28:21.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:21.546+0000 7f4ee1e00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4edc104ff0 0x7f4edc115a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:21.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:21.546+0000 7f4ee1e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4edc105990 0x7f4edc115f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:21.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:21.546+0000 7f4ee1e00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4edc1165c0 con 0x7f4edc105990 2024-03-29T08:28:21.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:21.546+0000 7f4ee1e00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4edc10fba0 con 0x7f4edc104ff0 2024-03-29T08:28:21.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:21.547+0000 7f4eda000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4edc105990 0x7f4edc115f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:21.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:21.547+0000 7f4eda000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4edc105990 0x7f4edc115f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57806/0 (socket says 172.21.15.37:57806) 2024-03-29T08:28:21.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:21.547+0000 7f4eda000700 1 -- 172.21.15.37:0/3331016089 learned_addr learned my addr 172.21.15.37:0/3331016089 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:28:21.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:21.547+0000 7f4edaa00700 1 --2- 172.21.15.37:0/3331016089 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4edc104ff0 0x7f4edc115a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:21.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:21.547+0000 7f4eda000700 1 -- 172.21.15.37:0/3331016089 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4edc104ff0 msgr2=0x7f4edc115a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:21.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:21.547+0000 7f4eda000700 1 --2- 172.21.15.37:0/3331016089 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4edc104ff0 0x7f4edc115a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:21.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:21.547+0000 7f4eda000700 1 -- 172.21.15.37:0/3331016089 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ec8005040 con 0x7f4edc105990 2024-03-29T08:28:21.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:21.547+0000 7f4eda000700 1 --2- 172.21.15.37:0/3331016089 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4edc105990 0x7f4edc115f70 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f4ec8009390 tx=0x7f4ec8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:28:21.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:21.548+0000 7f4ed2a00700 1 -- 172.21.15.37:0/3331016089 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4ec800e040 con 0x7f4edc105990 2024-03-29T08:28:21.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:21.548+0000 7f4ed2a00700 1 -- 172.21.15.37:0/3331016089 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f4ec8013420 con 0x7f4edc105990 2024-03-29T08:28:21.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:21.548+0000 7f4ee1e00700 1 -- 172.21.15.37:0/3331016089 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4edc10fe20 con 0x7f4edc105990 2024-03-29T08:28:21.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:21.548+0000 7f4ed2a00700 1 -- 172.21.15.37:0/3331016089 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4ec801c630 con 0x7f4edc105990 2024-03-29T08:28:21.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:21.548+0000 7f4ee1e00700 1 -- 172.21.15.37:0/3331016089 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4edc1103e0 con 0x7f4edc105990 2024-03-29T08:28:21.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:21.549+0000 7f4ee1e00700 1 -- 172.21.15.37:0/3331016089 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4edc102300 con 0x7f4edc105990 2024-03-29T08:28:21.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:21.550+0000 7f4ed2a00700 1 -- 172.21.15.37:0/3331016089 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f4ec8013a00 con 0x7f4edc105990 2024-03-29T08:28:21.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:21.551+0000 7f4ed2a00700 1 --2- 172.21.15.37:0/3331016089 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f4ec006cc20 0x7f4ec006f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:21.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:21.551+0000 7f4ed2a00700 1 -- 172.21.15.37:0/3331016089 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f4ec808d1b0 con 0x7f4edc105990 2024-03-29T08:28:21.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:21.551+0000 7f4edaa00700 1 --2- 172.21.15.37:0/3331016089 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f4ec006cc20 0x7f4ec006f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:21.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:21.551+0000 7f4edaa00700 1 --2- 172.21.15.37:0/3331016089 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f4ec006cc20 0x7f4ec006f0d0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f4ecc009fd0 tx=0x7f4ecc00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:28:21.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:21.555+0000 7f4ed2a00700 1 -- 172.21.15.37:0/3331016089 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f4ec805ac00 con 0x7f4edc105990 2024-03-29T08:28:21.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:21.814+0000 7f4ee1e00700 1 -- 172.21.15.37:0/3331016089 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f4edc111080 con 0x7f4edc105990 2024-03-29T08:28:21.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:21.814+0000 7f4ed2a00700 1 -- 172.21.15.37:0/3331016089 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 28 v28) v1 ==== 76+0+1841 (secure 0 0 0) 0x7f4ec805a790 con 0x7f4edc105990 2024-03-29T08:28:21.815 INFO:teuthology.orchestra.run.smithi037.stdout:e28 2024-03-29T08:28:21.815 INFO:teuthology.orchestra.run.smithi037.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-29T08:28:21.815 INFO:teuthology.orchestra.run.smithi037.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T08:28:21.815 INFO:teuthology.orchestra.run.smithi037.stdout:legacy client fscid: 1 2024-03-29T08:28:21.815 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:28:21.815 INFO:teuthology.orchestra.run.smithi037.stdout:Filesystem 'cephfs' (1) 2024-03-29T08:28:21.815 INFO:teuthology.orchestra.run.smithi037.stdout:fs_name cephfs 2024-03-29T08:28:21.815 INFO:teuthology.orchestra.run.smithi037.stdout:epoch 21 2024-03-29T08:28:21.815 INFO:teuthology.orchestra.run.smithi037.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-03-29T08:28:21.815 INFO:teuthology.orchestra.run.smithi037.stdout:created 2024-03-29T08:03:29.241269+0000 2024-03-29T08:28:21.815 INFO:teuthology.orchestra.run.smithi037.stdout:modified 2024-03-29T08:17:16.237238+0000 2024-03-29T08:28:21.815 INFO:teuthology.orchestra.run.smithi037.stdout:tableserver 0 2024-03-29T08:28:21.816 INFO:teuthology.orchestra.run.smithi037.stdout:root 0 2024-03-29T08:28:21.816 INFO:teuthology.orchestra.run.smithi037.stdout:session_timeout 60 2024-03-29T08:28:21.816 INFO:teuthology.orchestra.run.smithi037.stdout:session_autoclose 300 2024-03-29T08:28:21.816 INFO:teuthology.orchestra.run.smithi037.stdout:max_file_size 1099511627776 2024-03-29T08:28:21.816 INFO:teuthology.orchestra.run.smithi037.stdout:max_xattr_size 65536 2024-03-29T08:28:21.816 INFO:teuthology.orchestra.run.smithi037.stdout:required_client_features {} 2024-03-29T08:28:21.816 INFO:teuthology.orchestra.run.smithi037.stdout:last_failure 0 2024-03-29T08:28:21.816 INFO:teuthology.orchestra.run.smithi037.stdout:last_failure_osd_epoch 82 2024-03-29T08:28:21.816 INFO:teuthology.orchestra.run.smithi037.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T08:28:21.816 INFO:teuthology.orchestra.run.smithi037.stdout:max_mds 2 2024-03-29T08:28:21.816 INFO:teuthology.orchestra.run.smithi037.stdout:in 0,1 2024-03-29T08:28:21.816 INFO:teuthology.orchestra.run.smithi037.stdout:up {} 2024-03-29T08:28:21.816 INFO:teuthology.orchestra.run.smithi037.stdout:failed 0,1 2024-03-29T08:28:21.816 INFO:teuthology.orchestra.run.smithi037.stdout:damaged 2024-03-29T08:28:21.816 INFO:teuthology.orchestra.run.smithi037.stdout:stopped 2024-03-29T08:28:21.816 INFO:teuthology.orchestra.run.smithi037.stdout:data_pools [3] 2024-03-29T08:28:21.817 INFO:teuthology.orchestra.run.smithi037.stdout:metadata_pool 2 2024-03-29T08:28:21.817 INFO:teuthology.orchestra.run.smithi037.stdout:inline_data disabled 2024-03-29T08:28:21.817 INFO:teuthology.orchestra.run.smithi037.stdout:balancer 2024-03-29T08:28:21.817 INFO:teuthology.orchestra.run.smithi037.stdout:bal_rank_mask -1 2024-03-29T08:28:21.817 INFO:teuthology.orchestra.run.smithi037.stdout:standby_count_wanted 1 2024-03-29T08:28:21.817 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:28:21.817 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:28:21.817 INFO:teuthology.orchestra.run.smithi037.stdout:Standby daemons: 2024-03-29T08:28:21.817 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:28:21.817 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi148.qukpjh{-1:24319} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.148:6826/2641327912,v1:172.21.15.148:6827/2641327912] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:28:21.817 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi037.mmwgxe{-1:34440} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.37:6828/1248142005,v1:172.21.15.37:6829/1248142005] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:28:21.817 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi037.iifsmh{-1:34444} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.37:6826/2060882959,v1:172.21.15.37:6827/2060882959] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:28:21.817 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi148.sqteih{-1:44395} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.148:6824/4078652242,v1:172.21.15.148:6825/4078652242] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:28:21.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:21.817+0000 7f4ee1e00700 1 -- 172.21.15.37:0/3331016089 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f4ec006cc20 msgr2=0x7f4ec006f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:21.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:21.817+0000 7f4ee1e00700 1 --2- 172.21.15.37:0/3331016089 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f4ec006cc20 0x7f4ec006f0d0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f4ecc009fd0 tx=0x7f4ecc00b040 comp rx=0 tx=0).stop 2024-03-29T08:28:21.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:21.817+0000 7f4ee1e00700 1 -- 172.21.15.37:0/3331016089 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4edc105990 msgr2=0x7f4edc115f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:21.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:21.817+0000 7f4ee1e00700 1 --2- 172.21.15.37:0/3331016089 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4edc105990 0x7f4edc115f70 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f4ec8009390 tx=0x7f4ec8009e30 comp rx=0 tx=0).stop 2024-03-29T08:28:21.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:21.817+0000 7f4ee1e00700 1 -- 172.21.15.37:0/3331016089 shutdown_connections 2024-03-29T08:28:21.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:21.817+0000 7f4ee1e00700 1 --2- 172.21.15.37:0/3331016089 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f4ec006cc20 0x7f4ec006f0d0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:21.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:21.817+0000 7f4ee1e00700 1 --2- 172.21.15.37:0/3331016089 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4edc104ff0 0x7f4edc115a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:21.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:21.817+0000 7f4ee1e00700 1 --2- 172.21.15.37:0/3331016089 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4edc105990 0x7f4edc115f70 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:21.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:21.817+0000 7f4ee1e00700 1 -- 172.21.15.37:0/3331016089 >> 172.21.15.37:0/3331016089 conn(0x7f4edc0faba0 msgr2=0x7f4edc0fc990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:28:21.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:21.817+0000 7f4ee1e00700 1 -- 172.21.15.37:0/3331016089 shutdown_connections 2024-03-29T08:28:21.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:21.817+0000 7f4ee1e00700 1 -- 172.21.15.37:0/3331016089 wait complete. 2024-03-29T08:28:21.819 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 28 2024-03-29T08:28:21.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:28:21 smithi037 ceph-mon[142325]: pgmap v523: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:28:21.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:28:21 smithi037 ceph-mon[142325]: from='client.34980 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:28:21.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:28:21 smithi037 ceph-mon[142325]: from='client.34984 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:28:21.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:28:21 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/3620230564' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:28:21.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:21.955+0000 7fb4ca000700 1 -- 172.21.15.37:0/2331184946 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb4c41037a0 msgr2=0x7fb4c40fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:21.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:21.955+0000 7fb4ca000700 1 --2- 172.21.15.37:0/2331184946 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb4c41037a0 0x7fb4c40fce10 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fb4b0004940 tx=0x7fb4b0009e30 comp rx=0 tx=0).stop 2024-03-29T08:28:21.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:21.955+0000 7fb4ca000700 1 -- 172.21.15.37:0/2331184946 shutdown_connections 2024-03-29T08:28:21.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:21.955+0000 7fb4ca000700 1 --2- 172.21.15.37:0/2331184946 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb4c41037a0 0x7fb4c40fce10 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:21.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:21.955+0000 7fb4ca000700 1 --2- 172.21.15.37:0/2331184946 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb4c4102e00 0x7fb4c41031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:21.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:21.956+0000 7fb4ca000700 1 -- 172.21.15.37:0/2331184946 >> 172.21.15.37:0/2331184946 conn(0x7fb4c40f89b0 msgr2=0x7fb4c40fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:28:21.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:21.956+0000 7fb4ca000700 1 -- 172.21.15.37:0/2331184946 shutdown_connections 2024-03-29T08:28:21.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:21.956+0000 7fb4ca000700 1 -- 172.21.15.37:0/2331184946 wait complete. 2024-03-29T08:28:21.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:21.956+0000 7fb4ca000700 1 Processor -- start 2024-03-29T08:28:21.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:21.957+0000 7fb4ca000700 1 -- start start 2024-03-29T08:28:21.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:21.957+0000 7fb4ca000700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb4c4102e00 0x7fb4c40702c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:21.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:21.957+0000 7fb4ca000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb4c41037a0 0x7fb4c4070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:21.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:21.957+0000 7fb4ca000700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4c4070d90 con 0x7fb4c41037a0 2024-03-29T08:28:21.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:21.957+0000 7fb4ca000700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4c4070ed0 con 0x7fb4c4102e00 2024-03-29T08:28:21.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:21.957+0000 7fb4c2a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb4c41037a0 0x7fb4c4070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:21.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:21.957+0000 7fb4c2a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb4c41037a0 0x7fb4c4070800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57826/0 (socket says 172.21.15.37:57826) 2024-03-29T08:28:21.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:21.957+0000 7fb4c2a00700 1 -- 172.21.15.37:0/4191297891 learned_addr learned my addr 172.21.15.37:0/4191297891 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:28:21.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:21.957+0000 7fb4c3400700 1 --2- 172.21.15.37:0/4191297891 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb4c4102e00 0x7fb4c40702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:21.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:21.958+0000 7fb4c2a00700 1 -- 172.21.15.37:0/4191297891 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb4c4102e00 msgr2=0x7fb4c40702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:21.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:21.958+0000 7fb4c2a00700 1 --2- 172.21.15.37:0/4191297891 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb4c4102e00 0x7fb4c40702c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:21.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:21.958+0000 7fb4c2a00700 1 -- 172.21.15.37:0/4191297891 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4b0005040 con 0x7fb4c41037a0 2024-03-29T08:28:21.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:21.958+0000 7fb4c2a00700 1 --2- 172.21.15.37:0/4191297891 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb4c41037a0 0x7fb4c4070800 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fb4b0008d50 tx=0x7fb4b0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:28:21.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:21.958+0000 7fb4bb400700 1 -- 172.21.15.37:0/4191297891 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb4b000e030 con 0x7fb4c41037a0 2024-03-29T08:28:21.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:21.958+0000 7fb4bb400700 1 -- 172.21.15.37:0/4191297891 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fb4b00093b0 con 0x7fb4c41037a0 2024-03-29T08:28:21.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:21.958+0000 7fb4bb400700 1 -- 172.21.15.37:0/4191297891 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb4b001cbb0 con 0x7fb4c41037a0 2024-03-29T08:28:21.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:21.958+0000 7fb4ca000700 1 -- 172.21.15.37:0/4191297891 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4c4071120 con 0x7fb4c41037a0 2024-03-29T08:28:21.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:21.958+0000 7fb4ca000700 1 -- 172.21.15.37:0/4191297891 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4c411a150 con 0x7fb4c41037a0 2024-03-29T08:28:21.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:21.959+0000 7fb4ca000700 1 -- 172.21.15.37:0/4191297891 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4c40fe550 con 0x7fb4c41037a0 2024-03-29T08:28:21.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:21.960+0000 7fb4bb400700 1 -- 172.21.15.37:0/4191297891 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fb4b0013930 con 0x7fb4c41037a0 2024-03-29T08:28:21.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:21.961+0000 7fb4bb400700 1 --2- 172.21.15.37:0/4191297891 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fb4a806cc30 0x7fb4a806f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:21.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:21.961+0000 7fb4bb400700 1 -- 172.21.15.37:0/4191297891 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fb4b008d880 con 0x7fb4c41037a0 2024-03-29T08:28:21.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:21.962+0000 7fb4c3400700 1 --2- 172.21.15.37:0/4191297891 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fb4a806cc30 0x7fb4a806f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:21.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:21.962+0000 7fb4c3400700 1 --2- 172.21.15.37:0/4191297891 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fb4a806cc30 0x7fb4a806f0e0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fb4b4009b80 tx=0x7fb4b400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:28:21.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:21.966+0000 7fb4bb400700 1 -- 172.21.15.37:0/4191297891 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fb4b00577c0 con 0x7fb4c41037a0 2024-03-29T08:28:22.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:28:21 smithi148 ceph-mon[106692]: pgmap v523: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:28:22.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:28:21 smithi148 ceph-mon[106692]: from='client.34980 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:28:22.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:28:21 smithi148 ceph-mon[106692]: from='client.34984 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:28:22.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:28:21 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/3620230564' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:28:22.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:22.196+0000 7fb4ca000700 1 -- 172.21.15.37:0/4191297891 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb4c4109e40 con 0x7fb4a806cc30 2024-03-29T08:28:22.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:22.197+0000 7fb4bb400700 1 -- 172.21.15.37:0/4191297891 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fb4b400f040 con 0x7fb4a806cc30 2024-03-29T08:28:22.198 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-29T08:28:22.198 INFO:teuthology.orchestra.run.smithi037.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59", 2024-03-29T08:28:22.198 INFO:teuthology.orchestra.run.smithi037.stdout: "in_progress": true, 2024-03-29T08:28:22.198 INFO:teuthology.orchestra.run.smithi037.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-29T08:28:22.198 INFO:teuthology.orchestra.run.smithi037.stdout: "services_complete": [ 2024-03-29T08:28:22.198 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr", 2024-03-29T08:28:22.198 INFO:teuthology.orchestra.run.smithi037.stdout: "mon", 2024-03-29T08:28:22.198 INFO:teuthology.orchestra.run.smithi037.stdout: "crash", 2024-03-29T08:28:22.198 INFO:teuthology.orchestra.run.smithi037.stdout: "osd" 2024-03-29T08:28:22.198 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-29T08:28:22.198 INFO:teuthology.orchestra.run.smithi037.stdout: "progress": "15/21 daemons upgraded", 2024-03-29T08:28:22.199 INFO:teuthology.orchestra.run.smithi037.stdout: "message": "Currently upgrading mds daemons", 2024-03-29T08:28:22.199 INFO:teuthology.orchestra.run.smithi037.stdout: "is_paused": false 2024-03-29T08:28:22.199 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-29T08:28:22.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:22.200+0000 7fb4ca000700 1 -- 172.21.15.37:0/4191297891 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fb4a806cc30 msgr2=0x7fb4a806f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:22.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:22.200+0000 7fb4ca000700 1 --2- 172.21.15.37:0/4191297891 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fb4a806cc30 0x7fb4a806f0e0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fb4b4009b80 tx=0x7fb4b400b040 comp rx=0 tx=0).stop 2024-03-29T08:28:22.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:22.200+0000 7fb4ca000700 1 -- 172.21.15.37:0/4191297891 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb4c41037a0 msgr2=0x7fb4c4070800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:22.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:22.200+0000 7fb4ca000700 1 --2- 172.21.15.37:0/4191297891 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb4c41037a0 0x7fb4c4070800 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fb4b0008d50 tx=0x7fb4b0009e30 comp rx=0 tx=0).stop 2024-03-29T08:28:22.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:22.200+0000 7fb4ca000700 1 -- 172.21.15.37:0/4191297891 shutdown_connections 2024-03-29T08:28:22.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:22.200+0000 7fb4ca000700 1 --2- 172.21.15.37:0/4191297891 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fb4a806cc30 0x7fb4a806f0e0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:22.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:22.200+0000 7fb4ca000700 1 --2- 172.21.15.37:0/4191297891 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb4c4102e00 0x7fb4c40702c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:22.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:22.200+0000 7fb4ca000700 1 --2- 172.21.15.37:0/4191297891 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb4c41037a0 0x7fb4c4070800 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:22.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:22.200+0000 7fb4ca000700 1 -- 172.21.15.37:0/4191297891 >> 172.21.15.37:0/4191297891 conn(0x7fb4c40f89b0 msgr2=0x7fb4c40f9a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:28:22.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:22.200+0000 7fb4ca000700 1 -- 172.21.15.37:0/4191297891 shutdown_connections 2024-03-29T08:28:22.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:22.200+0000 7fb4ca000700 1 -- 172.21.15.37:0/4191297891 wait complete. 2024-03-29T08:28:22.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:22.337+0000 7ffb93000700 1 -- 172.21.15.37:0/723162437 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffb8c1037a0 msgr2=0x7ffb8c0fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:22.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:22.337+0000 7ffb93000700 1 --2- 172.21.15.37:0/723162437 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffb8c1037a0 0x7ffb8c0fce10 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7ffb78004940 tx=0x7ffb78009e30 comp rx=0 tx=0).stop 2024-03-29T08:28:22.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:22.338+0000 7ffb93000700 1 -- 172.21.15.37:0/723162437 shutdown_connections 2024-03-29T08:28:22.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:22.338+0000 7ffb93000700 1 --2- 172.21.15.37:0/723162437 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffb8c1037a0 0x7ffb8c0fce10 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:22.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:22.338+0000 7ffb93000700 1 --2- 172.21.15.37:0/723162437 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffb8c102e00 0x7ffb8c1031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:22.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:22.338+0000 7ffb93000700 1 -- 172.21.15.37:0/723162437 >> 172.21.15.37:0/723162437 conn(0x7ffb8c0f89b0 msgr2=0x7ffb8c0fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:28:22.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:22.338+0000 7ffb93000700 1 -- 172.21.15.37:0/723162437 shutdown_connections 2024-03-29T08:28:22.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:22.338+0000 7ffb93000700 1 -- 172.21.15.37:0/723162437 wait complete. 2024-03-29T08:28:22.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:22.338+0000 7ffb93000700 1 Processor -- start 2024-03-29T08:28:22.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:22.339+0000 7ffb93000700 1 -- start start 2024-03-29T08:28:22.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:22.339+0000 7ffb93000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffb8c102e00 0x7ffb8c0702c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:22.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:22.339+0000 7ffb93000700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffb8c1037a0 0x7ffb8c070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:22.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:22.339+0000 7ffb93000700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb8c070d90 con 0x7ffb8c102e00 2024-03-29T08:28:22.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:22.339+0000 7ffb93000700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb8c070ed0 con 0x7ffb8c1037a0 2024-03-29T08:28:22.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:22.339+0000 7ffb8be00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffb8c102e00 0x7ffb8c0702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:22.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:22.339+0000 7ffb8be00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffb8c102e00 0x7ffb8c0702c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57840/0 (socket says 172.21.15.37:57840) 2024-03-29T08:28:22.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:22.339+0000 7ffb8be00700 1 -- 172.21.15.37:0/3500947915 learned_addr learned my addr 172.21.15.37:0/3500947915 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:28:22.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:22.339+0000 7ffb8b400700 1 --2- 172.21.15.37:0/3500947915 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffb8c1037a0 0x7ffb8c070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:22.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:22.340+0000 7ffb8be00700 1 -- 172.21.15.37:0/3500947915 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffb8c1037a0 msgr2=0x7ffb8c070800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:22.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:22.340+0000 7ffb8be00700 1 --2- 172.21.15.37:0/3500947915 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffb8c1037a0 0x7ffb8c070800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:22.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:22.340+0000 7ffb8be00700 1 -- 172.21.15.37:0/3500947915 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb78005040 con 0x7ffb8c102e00 2024-03-29T08:28:22.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:22.340+0000 7ffb8be00700 1 --2- 172.21.15.37:0/3500947915 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffb8c102e00 0x7ffb8c0702c0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7ffb7c002a50 tx=0x7ffb7c0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:28:22.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:22.340+0000 7ffb83e00700 1 -- 172.21.15.37:0/3500947915 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffb7c007cc0 con 0x7ffb8c102e00 2024-03-29T08:28:22.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:22.340+0000 7ffb83e00700 1 -- 172.21.15.37:0/3500947915 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ffb7c00f040 con 0x7ffb8c102e00 2024-03-29T08:28:22.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:22.340+0000 7ffb83e00700 1 -- 172.21.15.37:0/3500947915 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffb7c00d9c0 con 0x7ffb8c102e00 2024-03-29T08:28:22.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:22.340+0000 7ffb93000700 1 -- 172.21.15.37:0/3500947915 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb8c119e00 con 0x7ffb8c102e00 2024-03-29T08:28:22.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:22.340+0000 7ffb93000700 1 -- 172.21.15.37:0/3500947915 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb8c11a280 con 0x7ffb8c102e00 2024-03-29T08:28:22.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:22.341+0000 7ffb93000700 1 -- 172.21.15.37:0/3500947915 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb8c0fe550 con 0x7ffb8c102e00 2024-03-29T08:28:22.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:22.343+0000 7ffb83e00700 1 -- 172.21.15.37:0/3500947915 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7ffb7c00db20 con 0x7ffb8c102e00 2024-03-29T08:28:22.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:22.343+0000 7ffb83e00700 1 --2- 172.21.15.37:0/3500947915 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7ffb7006cc30 0x7ffb7006f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:22.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:22.344+0000 7ffb83e00700 1 -- 172.21.15.37:0/3500947915 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7ffb7c08be20 con 0x7ffb8c102e00 2024-03-29T08:28:22.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:22.344+0000 7ffb8b400700 1 --2- 172.21.15.37:0/3500947915 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7ffb7006cc30 0x7ffb7006f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:22.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:22.344+0000 7ffb8b400700 1 --2- 172.21.15.37:0/3500947915 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7ffb7006cc30 0x7ffb7006f0e0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7ffb8c071c00 tx=0x7ffb78009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:28:22.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:22.348+0000 7ffb83e00700 1 -- 172.21.15.37:0/3500947915 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7ffb7c055d60 con 0x7ffb8c102e00 2024-03-29T08:28:22.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:22.650+0000 7ffb93000700 1 -- 172.21.15.37:0/3500947915 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7ffb8c05f5f0 con 0x7ffb8c102e00 2024-03-29T08:28:22.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:22.650+0000 7ffb83e00700 1 -- 172.21.15.37:0/3500947915 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7ffb8c05f5f0 con 0x7ffb8c102e00 2024-03-29T08:28:22.651 INFO:teuthology.orchestra.run.smithi037.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-03-29T08:28:22.651 INFO:teuthology.orchestra.run.smithi037.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-03-29T08:28:22.651 INFO:teuthology.orchestra.run.smithi037.stdout: fs cephfs is degraded 2024-03-29T08:28:22.651 INFO:teuthology.orchestra.run.smithi037.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-03-29T08:28:22.651 INFO:teuthology.orchestra.run.smithi037.stdout: fs cephfs is offline because no MDS is active for it. 2024-03-29T08:28:22.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:22.653+0000 7ffb93000700 1 -- 172.21.15.37:0/3500947915 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7ffb7006cc30 msgr2=0x7ffb7006f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:22.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:22.653+0000 7ffb93000700 1 --2- 172.21.15.37:0/3500947915 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7ffb7006cc30 0x7ffb7006f0e0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7ffb8c071c00 tx=0x7ffb78009e30 comp rx=0 tx=0).stop 2024-03-29T08:28:22.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:22.653+0000 7ffb93000700 1 -- 172.21.15.37:0/3500947915 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffb8c102e00 msgr2=0x7ffb8c0702c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:22.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:22.653+0000 7ffb93000700 1 --2- 172.21.15.37:0/3500947915 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffb8c102e00 0x7ffb8c0702c0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7ffb7c002a50 tx=0x7ffb7c0046b0 comp rx=0 tx=0).stop 2024-03-29T08:28:22.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:22.653+0000 7ffb93000700 1 -- 172.21.15.37:0/3500947915 shutdown_connections 2024-03-29T08:28:22.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:22.653+0000 7ffb93000700 1 --2- 172.21.15.37:0/3500947915 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7ffb7006cc30 0x7ffb7006f0e0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:22.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:22.653+0000 7ffb93000700 1 --2- 172.21.15.37:0/3500947915 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffb8c102e00 0x7ffb8c0702c0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:22.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:22.653+0000 7ffb93000700 1 --2- 172.21.15.37:0/3500947915 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffb8c1037a0 0x7ffb8c070800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:22.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:22.653+0000 7ffb93000700 1 -- 172.21.15.37:0/3500947915 >> 172.21.15.37:0/3500947915 conn(0x7ffb8c0f89b0 msgr2=0x7ffb8c0f9ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:28:22.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:22.653+0000 7ffb93000700 1 -- 172.21.15.37:0/3500947915 shutdown_connections 2024-03-29T08:28:22.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:22.653+0000 7ffb93000700 1 -- 172.21.15.37:0/3500947915 wait complete. 2024-03-29T08:28:22.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:28:22 smithi037 ceph-mon[142325]: from='client.34988 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:28:22.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:28:22 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/3331016089' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T08:28:23.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:28:22 smithi148 ceph-mon[106692]: from='client.34988 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:28:23.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:28:22 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/3331016089' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T08:28:23.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:28:23 smithi037 ceph-mon[142325]: pgmap v524: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:28:23.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:28:23 smithi037 ceph-mon[142325]: from='client.35000 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:28:23.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:28:23 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/3500947915' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T08:28:24.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:28:23 smithi148 ceph-mon[106692]: pgmap v524: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:28:24.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:28:23 smithi148 ceph-mon[106692]: from='client.35000 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:28:24.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:28:23 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/3500947915' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T08:28:25.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:28:25 smithi148 ceph-mon[106692]: pgmap v525: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:28:25.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:28:25 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:28:25.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:28:25 smithi037 ceph-mon[142325]: pgmap v525: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:28:25.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:28:25 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:28:27.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:28:27 smithi037 ceph-mon[142325]: pgmap v526: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:28:27.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:28:27 smithi148 ceph-mon[106692]: pgmap v526: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:28:29.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:28:29 smithi037 ceph-mon[142325]: pgmap v527: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:28:29.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:28:29 smithi148 ceph-mon[106692]: pgmap v527: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:28:31.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:28:31 smithi037 ceph-mon[142325]: pgmap v528: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:28:31.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:28:31 smithi148 ceph-mon[106692]: pgmap v528: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:28:33.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:28:33 smithi037 ceph-mon[142325]: pgmap v529: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:28:33.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:28:33 smithi148 ceph-mon[106692]: pgmap v529: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:28:35.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:28:35 smithi037 ceph-mon[142325]: pgmap v530: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:28:35.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:28:35 smithi148 ceph-mon[106692]: pgmap v530: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:28:37.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:28:37 smithi037 ceph-mon[142325]: pgmap v531: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:28:37.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:28:37 smithi148 ceph-mon[106692]: pgmap v531: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:28:39.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:28:39 smithi037 ceph-mon[142325]: pgmap v532: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:28:39.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:28:39 smithi148 ceph-mon[106692]: pgmap v532: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:28:40.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:28:40 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:28:40.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:28:40 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:28:41.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:28:41 smithi037 ceph-mon[142325]: pgmap v533: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:28:41.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:28:41 smithi148 ceph-mon[106692]: pgmap v533: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:28:43.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:28:43 smithi037 ceph-mon[142325]: pgmap v534: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:28:43.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:28:43 smithi148 ceph-mon[106692]: pgmap v534: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:28:45.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:28:45 smithi037 ceph-mon[142325]: pgmap v535: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:28:45.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:28:45 smithi148 ceph-mon[106692]: pgmap v535: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:28:47.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:28:47 smithi037 ceph-mon[142325]: pgmap v536: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:28:47.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:28:47 smithi148 ceph-mon[106692]: pgmap v536: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:28:49.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:28:49 smithi037 ceph-mon[142325]: pgmap v537: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:28:49.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:28:49 smithi148 ceph-mon[106692]: pgmap v537: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:28:51.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:28:51 smithi037 ceph-mon[142325]: pgmap v538: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:28:51.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:28:51 smithi148 ceph-mon[106692]: pgmap v538: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:28:52.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:52.807+0000 7fa05f000700 1 -- 172.21.15.37:0/3327721262 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa0581037a0 msgr2=0x7fa0580fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:52.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:52.807+0000 7fa05f000700 1 --2- 172.21.15.37:0/3327721262 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa0581037a0 0x7fa0580fce10 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fa044004940 tx=0x7fa044009e30 comp rx=0 tx=0).stop 2024-03-29T08:28:52.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:52.807+0000 7fa05f000700 1 -- 172.21.15.37:0/3327721262 shutdown_connections 2024-03-29T08:28:52.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:52.807+0000 7fa05f000700 1 --2- 172.21.15.37:0/3327721262 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa0581037a0 0x7fa0580fce10 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:52.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:52.807+0000 7fa05f000700 1 --2- 172.21.15.37:0/3327721262 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa058102e00 0x7fa0581031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:52.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:52.807+0000 7fa05f000700 1 -- 172.21.15.37:0/3327721262 >> 172.21.15.37:0/3327721262 conn(0x7fa0580f89b0 msgr2=0x7fa0580fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:28:52.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:52.807+0000 7fa05f000700 1 -- 172.21.15.37:0/3327721262 shutdown_connections 2024-03-29T08:28:52.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:52.808+0000 7fa05f000700 1 -- 172.21.15.37:0/3327721262 wait complete. 2024-03-29T08:28:52.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:52.808+0000 7fa05f000700 1 Processor -- start 2024-03-29T08:28:52.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:52.808+0000 7fa05f000700 1 -- start start 2024-03-29T08:28:52.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:52.809+0000 7fa05f000700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa058102e00 0x7fa0580702c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:52.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:52.809+0000 7fa05f000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa0581037a0 0x7fa058070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:52.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:52.809+0000 7fa05f000700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa058070d90 con 0x7fa0581037a0 2024-03-29T08:28:52.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:52.809+0000 7fa05f000700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa058070ed0 con 0x7fa058102e00 2024-03-29T08:28:52.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:52.809+0000 7fa057400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa0581037a0 0x7fa058070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:52.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:52.809+0000 7fa057400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa0581037a0 0x7fa058070800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:48432/0 (socket says 172.21.15.37:48432) 2024-03-29T08:28:52.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:52.809+0000 7fa057400700 1 -- 172.21.15.37:0/31552450 learned_addr learned my addr 172.21.15.37:0/31552450 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:28:52.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:52.809+0000 7fa057e00700 1 --2- 172.21.15.37:0/31552450 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa058102e00 0x7fa0580702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:52.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:52.810+0000 7fa057400700 1 -- 172.21.15.37:0/31552450 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa058102e00 msgr2=0x7fa0580702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:52.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:52.810+0000 7fa057400700 1 --2- 172.21.15.37:0/31552450 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa058102e00 0x7fa0580702c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:52.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:52.810+0000 7fa057400700 1 -- 172.21.15.37:0/31552450 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa044005040 con 0x7fa0581037a0 2024-03-29T08:28:52.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:52.810+0000 7fa057e00700 1 --2- 172.21.15.37:0/31552450 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa058102e00 0x7fa0580702c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:28:52.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:52.810+0000 7fa057400700 1 --2- 172.21.15.37:0/31552450 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa0581037a0 0x7fa058070800 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fa044009390 tx=0x7fa044009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:28:52.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:52.810+0000 7fa04fe00700 1 -- 172.21.15.37:0/31552450 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa04400e040 con 0x7fa0581037a0 2024-03-29T08:28:52.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:52.810+0000 7fa04fe00700 1 -- 172.21.15.37:0/31552450 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fa044013420 con 0x7fa0581037a0 2024-03-29T08:28:52.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:52.810+0000 7fa05f000700 1 -- 172.21.15.37:0/31552450 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa058071120 con 0x7fa0581037a0 2024-03-29T08:28:52.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:52.810+0000 7fa04fe00700 1 -- 172.21.15.37:0/31552450 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa04401c630 con 0x7fa0581037a0 2024-03-29T08:28:52.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:52.810+0000 7fa05f000700 1 -- 172.21.15.37:0/31552450 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa05811a230 con 0x7fa0581037a0 2024-03-29T08:28:52.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:52.811+0000 7fa05f000700 1 -- 172.21.15.37:0/31552450 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0580fe550 con 0x7fa0581037a0 2024-03-29T08:28:52.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:52.812+0000 7fa04fe00700 1 -- 172.21.15.37:0/31552450 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fa0440139e0 con 0x7fa0581037a0 2024-03-29T08:28:52.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:52.813+0000 7fa04fe00700 1 --2- 172.21.15.37:0/31552450 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fa03c06cc30 0x7fa03c06f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:52.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:52.813+0000 7fa04fe00700 1 -- 172.21.15.37:0/31552450 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fa04408d380 con 0x7fa0581037a0 2024-03-29T08:28:52.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:52.813+0000 7fa057e00700 1 --2- 172.21.15.37:0/31552450 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fa03c06cc30 0x7fa03c06f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:52.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:52.814+0000 7fa057e00700 1 --2- 172.21.15.37:0/31552450 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fa03c06cc30 0x7fa03c06f0e0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fa048007fd0 tx=0x7fa048009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:28:52.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:52.818+0000 7fa04fe00700 1 -- 172.21.15.37:0/31552450 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fa044057340 con 0x7fa0581037a0 2024-03-29T08:28:53.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:53.053+0000 7fa05f000700 1 -- 172.21.15.37:0/31552450 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa058109e10 con 0x7fa03c06cc30 2024-03-29T08:28:53.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:53.055+0000 7fa04fe00700 1 -- 172.21.15.37:0/31552450 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fa058109e10 con 0x7fa03c06cc30 2024-03-29T08:28:53.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:53.057+0000 7fa05f000700 1 -- 172.21.15.37:0/31552450 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fa03c06cc30 msgr2=0x7fa03c06f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:53.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:53.057+0000 7fa05f000700 1 --2- 172.21.15.37:0/31552450 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fa03c06cc30 0x7fa03c06f0e0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fa048007fd0 tx=0x7fa048009040 comp rx=0 tx=0).stop 2024-03-29T08:28:53.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:53.057+0000 7fa05f000700 1 -- 172.21.15.37:0/31552450 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa0581037a0 msgr2=0x7fa058070800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:53.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:53.057+0000 7fa05f000700 1 --2- 172.21.15.37:0/31552450 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa0581037a0 0x7fa058070800 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fa044009390 tx=0x7fa044009e30 comp rx=0 tx=0).stop 2024-03-29T08:28:53.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:53.057+0000 7fa05f000700 1 -- 172.21.15.37:0/31552450 shutdown_connections 2024-03-29T08:28:53.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:53.057+0000 7fa05f000700 1 --2- 172.21.15.37:0/31552450 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fa03c06cc30 0x7fa03c06f0e0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:53.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:53.057+0000 7fa05f000700 1 --2- 172.21.15.37:0/31552450 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa058102e00 0x7fa0580702c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:53.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:53.057+0000 7fa05f000700 1 --2- 172.21.15.37:0/31552450 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa0581037a0 0x7fa058070800 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:53.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:53.057+0000 7fa05f000700 1 -- 172.21.15.37:0/31552450 >> 172.21.15.37:0/31552450 conn(0x7fa0580f89b0 msgr2=0x7fa0580f9ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:28:53.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:53.058+0000 7fa05f000700 1 -- 172.21.15.37:0/31552450 shutdown_connections 2024-03-29T08:28:53.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:53.058+0000 7fa05f000700 1 -- 172.21.15.37:0/31552450 wait complete. 2024-03-29T08:28:53.073 INFO:teuthology.orchestra.run.smithi037.stdout:true 2024-03-29T08:28:53.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:53.197+0000 7f91a3000700 1 -- 172.21.15.37:0/3139072420 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f919c102e00 msgr2=0x7f919c1031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:53.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:53.197+0000 7f91a3000700 1 --2- 172.21.15.37:0/3139072420 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f919c102e00 0x7f919c1031d0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f918c004940 tx=0x7f918c009e30 comp rx=0 tx=0).stop 2024-03-29T08:28:53.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:53.197+0000 7f91a3000700 1 -- 172.21.15.37:0/3139072420 shutdown_connections 2024-03-29T08:28:53.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:53.197+0000 7f91a3000700 1 --2- 172.21.15.37:0/3139072420 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f919c1037a0 0x7f919c0fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:53.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:53.197+0000 7f91a3000700 1 --2- 172.21.15.37:0/3139072420 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f919c102e00 0x7f919c1031d0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:53.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:53.197+0000 7f91a3000700 1 -- 172.21.15.37:0/3139072420 >> 172.21.15.37:0/3139072420 conn(0x7f919c0f89b0 msgr2=0x7f919c0fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:28:53.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:53.197+0000 7f91a3000700 1 -- 172.21.15.37:0/3139072420 shutdown_connections 2024-03-29T08:28:53.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:53.197+0000 7f91a3000700 1 -- 172.21.15.37:0/3139072420 wait complete. 2024-03-29T08:28:53.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:53.198+0000 7f91a3000700 1 Processor -- start 2024-03-29T08:28:53.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:53.198+0000 7f91a3000700 1 -- start start 2024-03-29T08:28:53.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:53.198+0000 7f91a3000700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f919c102e00 0x7f919c115c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:53.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:53.198+0000 7f91a3000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f919c1037a0 0x7f919c116180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:53.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:53.198+0000 7f91a3000700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f919c1167d0 con 0x7f919c1037a0 2024-03-29T08:28:53.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:53.198+0000 7f91a3000700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f919c10fba0 con 0x7f919c102e00 2024-03-29T08:28:53.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:53.199+0000 7f919b400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f919c1037a0 0x7f919c116180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:53.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:53.199+0000 7f919b400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f919c1037a0 0x7f919c116180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:48446/0 (socket says 172.21.15.37:48446) 2024-03-29T08:28:53.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:53.199+0000 7f919b400700 1 -- 172.21.15.37:0/3525804935 learned_addr learned my addr 172.21.15.37:0/3525804935 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:28:53.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:53.199+0000 7f919be00700 1 --2- 172.21.15.37:0/3525804935 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f919c102e00 0x7f919c115c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:53.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:53.199+0000 7f919b400700 1 -- 172.21.15.37:0/3525804935 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f919c102e00 msgr2=0x7f919c115c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:53.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:53.199+0000 7f919b400700 1 --2- 172.21.15.37:0/3525804935 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f919c102e00 0x7f919c115c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:53.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:53.199+0000 7f919b400700 1 -- 172.21.15.37:0/3525804935 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f918c005040 con 0x7f919c1037a0 2024-03-29T08:28:53.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:53.199+0000 7f919b400700 1 --2- 172.21.15.37:0/3525804935 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f919c1037a0 0x7f919c116180 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f918800eb40 tx=0x7f918800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:28:53.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:53.200+0000 7f9193e00700 1 -- 172.21.15.37:0/3525804935 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9188007cc0 con 0x7f919c1037a0 2024-03-29T08:28:53.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:53.200+0000 7f9193e00700 1 -- 172.21.15.37:0/3525804935 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f918800f040 con 0x7f919c1037a0 2024-03-29T08:28:53.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:53.200+0000 7f91a3000700 1 -- 172.21.15.37:0/3525804935 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f919c10fe20 con 0x7f919c1037a0 2024-03-29T08:28:53.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:53.200+0000 7f9193e00700 1 -- 172.21.15.37:0/3525804935 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9188009570 con 0x7f919c1037a0 2024-03-29T08:28:53.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:53.200+0000 7f91a3000700 1 -- 172.21.15.37:0/3525804935 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f919c1103e0 con 0x7f919c1037a0 2024-03-29T08:28:53.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:53.201+0000 7f91a3000700 1 -- 172.21.15.37:0/3525804935 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f919c0fe550 con 0x7f919c1037a0 2024-03-29T08:28:53.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:53.202+0000 7f9193e00700 1 -- 172.21.15.37:0/3525804935 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f91880096d0 con 0x7f919c1037a0 2024-03-29T08:28:53.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:53.203+0000 7f9193e00700 1 --2- 172.21.15.37:0/3525804935 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f918006cc30 0x7f918006f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:53.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:53.203+0000 7f9193e00700 1 -- 172.21.15.37:0/3525804935 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f918808bc10 con 0x7f919c1037a0 2024-03-29T08:28:53.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:53.203+0000 7f919be00700 1 --2- 172.21.15.37:0/3525804935 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f918006cc30 0x7f918006f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:53.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:53.204+0000 7f919be00700 1 --2- 172.21.15.37:0/3525804935 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f918006cc30 0x7f918006f0e0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f918c00b0d0 tx=0x7f918c009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:28:53.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:53.208+0000 7f9193e00700 1 -- 172.21.15.37:0/3525804935 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f9188059690 con 0x7f919c1037a0 2024-03-29T08:28:53.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:28:53 smithi037 ceph-mon[142325]: pgmap v539: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:28:53.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:53.442+0000 7f91a3000700 1 -- 172.21.15.37:0/3525804935 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f919c109e30 con 0x7f918006cc30 2024-03-29T08:28:53.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:53.443+0000 7f9193e00700 1 -- 172.21.15.37:0/3525804935 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f919c109e30 con 0x7f918006cc30 2024-03-29T08:28:53.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:53.445+0000 7f91a3000700 1 -- 172.21.15.37:0/3525804935 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f918006cc30 msgr2=0x7f918006f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:53.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:53.445+0000 7f91a3000700 1 --2- 172.21.15.37:0/3525804935 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f918006cc30 0x7f918006f0e0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f918c00b0d0 tx=0x7f918c009e30 comp rx=0 tx=0).stop 2024-03-29T08:28:53.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:53.445+0000 7f91a3000700 1 -- 172.21.15.37:0/3525804935 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f919c1037a0 msgr2=0x7f919c116180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:53.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:53.445+0000 7f91a3000700 1 --2- 172.21.15.37:0/3525804935 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f919c1037a0 0x7f919c116180 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f918800eb40 tx=0x7f918800b040 comp rx=0 tx=0).stop 2024-03-29T08:28:53.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:53.445+0000 7f91a3000700 1 -- 172.21.15.37:0/3525804935 shutdown_connections 2024-03-29T08:28:53.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:53.445+0000 7f91a3000700 1 --2- 172.21.15.37:0/3525804935 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f918006cc30 0x7f918006f0e0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:53.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:53.445+0000 7f91a3000700 1 --2- 172.21.15.37:0/3525804935 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f919c102e00 0x7f919c115c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:53.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:53.445+0000 7f91a3000700 1 --2- 172.21.15.37:0/3525804935 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f919c1037a0 0x7f919c116180 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:53.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:53.445+0000 7f91a3000700 1 -- 172.21.15.37:0/3525804935 >> 172.21.15.37:0/3525804935 conn(0x7f919c0f89b0 msgr2=0x7f919c0f99f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:28:53.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:53.446+0000 7f91a3000700 1 -- 172.21.15.37:0/3525804935 shutdown_connections 2024-03-29T08:28:53.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:53.446+0000 7f91a3000700 1 -- 172.21.15.37:0/3525804935 wait complete. 2024-03-29T08:28:53.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:53.584+0000 7f4ef3800700 1 -- 172.21.15.37:0/1236826102 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4eec1037a0 msgr2=0x7f4eec0fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:53.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:53.584+0000 7f4ef3800700 1 --2- 172.21.15.37:0/1236826102 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4eec1037a0 0x7f4eec0fce10 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f4ed8004940 tx=0x7f4ed8009e30 comp rx=0 tx=0).stop 2024-03-29T08:28:53.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:53.584+0000 7f4ef3800700 1 -- 172.21.15.37:0/1236826102 shutdown_connections 2024-03-29T08:28:53.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:53.584+0000 7f4ef3800700 1 --2- 172.21.15.37:0/1236826102 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4eec1037a0 0x7f4eec0fce10 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:53.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:53.584+0000 7f4ef3800700 1 --2- 172.21.15.37:0/1236826102 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4eec102e00 0x7f4eec1031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:53.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:53.584+0000 7f4ef3800700 1 -- 172.21.15.37:0/1236826102 >> 172.21.15.37:0/1236826102 conn(0x7f4eec0f89b0 msgr2=0x7f4eec0fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:28:53.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:53.584+0000 7f4ef3800700 1 -- 172.21.15.37:0/1236826102 shutdown_connections 2024-03-29T08:28:53.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:53.584+0000 7f4ef3800700 1 -- 172.21.15.37:0/1236826102 wait complete. 2024-03-29T08:28:53.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:53.585+0000 7f4ef3800700 1 Processor -- start 2024-03-29T08:28:53.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:53.585+0000 7f4ef3800700 1 -- start start 2024-03-29T08:28:53.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:53.585+0000 7f4ef3800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4eec102e00 0x7f4eec115c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:53.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:53.585+0000 7f4ef3800700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4eec1037a0 0x7f4eec1161a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:53.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:53.585+0000 7f4ef3800700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4eec1167f0 con 0x7f4eec102e00 2024-03-29T08:28:53.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:53.585+0000 7f4ef3800700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4eec10fbc0 con 0x7f4eec1037a0 2024-03-29T08:28:53.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:53.586+0000 7f4ef0e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4eec102e00 0x7f4eec115c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:53.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:53.586+0000 7f4ef0e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4eec102e00 0x7f4eec115c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:48464/0 (socket says 172.21.15.37:48464) 2024-03-29T08:28:53.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:53.586+0000 7f4ef0e00700 1 -- 172.21.15.37:0/1599625812 learned_addr learned my addr 172.21.15.37:0/1599625812 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:28:53.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:53.586+0000 7f4eebe00700 1 --2- 172.21.15.37:0/1599625812 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4eec1037a0 0x7f4eec1161a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:53.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:53.586+0000 7f4ef0e00700 1 -- 172.21.15.37:0/1599625812 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4eec1037a0 msgr2=0x7f4eec1161a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:53.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:53.586+0000 7f4ef0e00700 1 --2- 172.21.15.37:0/1599625812 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4eec1037a0 0x7f4eec1161a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:53.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:53.586+0000 7f4ef0e00700 1 -- 172.21.15.37:0/1599625812 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ed8005040 con 0x7f4eec102e00 2024-03-29T08:28:53.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:53.586+0000 7f4ef0e00700 1 --2- 172.21.15.37:0/1599625812 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4eec102e00 0x7f4eec115c60 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f4ee0002a80 tx=0x7f4ee00046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:28:53.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:53.586+0000 7f4ee8c00700 1 -- 172.21.15.37:0/1599625812 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4ee0007cc0 con 0x7f4eec102e00 2024-03-29T08:28:53.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:53.587+0000 7f4ee8c00700 1 -- 172.21.15.37:0/1599625812 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f4ee000f040 con 0x7f4eec102e00 2024-03-29T08:28:53.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:53.587+0000 7f4ef3800700 1 -- 172.21.15.37:0/1599625812 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4eec10fea0 con 0x7f4eec102e00 2024-03-29T08:28:53.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:53.587+0000 7f4ee8c00700 1 -- 172.21.15.37:0/1599625812 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4ee0009640 con 0x7f4eec102e00 2024-03-29T08:28:53.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:53.587+0000 7f4ef3800700 1 -- 172.21.15.37:0/1599625812 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4eec110460 con 0x7f4eec102e00 2024-03-29T08:28:53.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:53.588+0000 7f4ef3800700 1 -- 172.21.15.37:0/1599625812 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4eec0fe550 con 0x7f4eec102e00 2024-03-29T08:28:53.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:53.589+0000 7f4ee8c00700 1 -- 172.21.15.37:0/1599625812 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f4ee00097a0 con 0x7f4eec102e00 2024-03-29T08:28:53.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:53.590+0000 7f4ee8c00700 1 --2- 172.21.15.37:0/1599625812 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f4ed406cc30 0x7f4ed406f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:53.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:53.590+0000 7f4ee8c00700 1 -- 172.21.15.37:0/1599625812 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f4ee008b7e0 con 0x7f4eec102e00 2024-03-29T08:28:53.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:53.590+0000 7f4eebe00700 1 --2- 172.21.15.37:0/1599625812 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f4ed406cc30 0x7f4ed406f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:53.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:53.590+0000 7f4eebe00700 1 --2- 172.21.15.37:0/1599625812 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f4ed406cc30 0x7f4ed406f0e0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f4ed800b020 tx=0x7f4ed8009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:28:53.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:53.594+0000 7f4ee8c00700 1 -- 172.21.15.37:0/1599625812 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f4ee0059230 con 0x7f4eec102e00 2024-03-29T08:28:53.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:28:53 smithi148 ceph-mon[106692]: pgmap v539: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:28:53.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:53.817+0000 7f4ef3800700 1 -- 172.21.15.37:0/1599625812 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f4eec04bf40 con 0x7f4ed406cc30 2024-03-29T08:28:53.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:53.828+0000 7f4ee8c00700 1 -- 172.21.15.37:0/1599625812 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f4eec04bf40 con 0x7f4ed406cc30 2024-03-29T08:28:53.828 INFO:teuthology.orchestra.run.smithi037.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-29T08:28:53.829 INFO:teuthology.orchestra.run.smithi037.stdout:alertmanager.smithi037 smithi037 *:9093,9094 running (18m) 11m ago 31m 14.9M - 0.25.0 c8568f914cd2 2f9f9746ed0f 2024-03-29T08:28:53.829 INFO:teuthology.orchestra.run.smithi037.stdout:crash.smithi037 smithi037 running (15m) 11m ago 31m 6647k - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 178d416a3186 2024-03-29T08:28:53.829 INFO:teuthology.orchestra.run.smithi037.stdout:crash.smithi148 smithi148 running (15m) 10m ago 30m 6656k - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 23666989d58f 2024-03-29T08:28:53.829 INFO:teuthology.orchestra.run.smithi037.stdout:grafana.smithi037 smithi037 *:3000 running (18m) 11m ago 31m 71.9M - 9.4.7 954c08fa6188 6b976fe49952 2024-03-29T08:28:53.829 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi037.iifsmh smithi037 running (11m) 11m ago 25m 10.8M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 2d97f562cc5b 2024-03-29T08:28:53.830 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi037.mmwgxe smithi037 running (11m) 11m ago 25m 10.9M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff ec37037f46e8 2024-03-29T08:28:53.830 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi148.qukpjh smithi148 running (25m) 10m ago 25m 15.3M - 17.2.7-789-g4213e17c cf22eb8683be 8e469782a273 2024-03-29T08:28:53.830 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi148.sqteih smithi148 running (10m) 10m ago 25m 10.7M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 44c15c77037a 2024-03-29T08:28:53.830 INFO:teuthology.orchestra.run.smithi037.stdout:mgr.smithi037.wjikii smithi037 *:8443,9283,8765 running (21m) 11m ago 33m 506M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff c5a1417b9417 2024-03-29T08:28:53.830 INFO:teuthology.orchestra.run.smithi037.stdout:mgr.smithi148.onhgvm smithi148 *:8443,9283,8765 running (19m) 10m ago 30m 450M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff fcfdbce062bd 2024-03-29T08:28:53.830 INFO:teuthology.orchestra.run.smithi037.stdout:mon.smithi037 smithi037 running (16m) 11m ago 33m 54.8M 2048M 18.2.2-1015-gd0e392d9 1cc2f45c17ff eea2060783af 2024-03-29T08:28:53.831 INFO:teuthology.orchestra.run.smithi037.stdout:mon.smithi148 smithi148 running (16m) 10m ago 30m 43.7M 2048M 18.2.2-1015-gd0e392d9 1cc2f45c17ff ad4b2c5748be 2024-03-29T08:28:53.831 INFO:teuthology.orchestra.run.smithi037.stdout:node-exporter.smithi037 smithi037 *:9100 running (19m) 11m ago 31m 7625k - 1.5.0 0da6a335fe13 26b8ae9cb878 2024-03-29T08:28:53.831 INFO:teuthology.orchestra.run.smithi037.stdout:node-exporter.smithi148 smithi148 *:9100 running (19m) 10m ago 30m 8166k - 1.5.0 0da6a335fe13 98414e4e64ed 2024-03-29T08:28:53.831 INFO:teuthology.orchestra.run.smithi037.stdout:osd.0 smithi037 running (15m) 11m ago 29m 104M 1619M 18.2.2-1015-gd0e392d9 1cc2f45c17ff c0c584f6245c 2024-03-29T08:28:53.831 INFO:teuthology.orchestra.run.smithi037.stdout:osd.1 smithi037 running (14m) 11m ago 29m 100M 1619M 18.2.2-1015-gd0e392d9 1cc2f45c17ff a4a2ff4cc83d 2024-03-29T08:28:53.831 INFO:teuthology.orchestra.run.smithi037.stdout:osd.2 smithi037 running (13m) 11m ago 28m 82.3M 1619M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 8b1bc71aa79e 2024-03-29T08:28:53.832 INFO:teuthology.orchestra.run.smithi037.stdout:osd.3 smithi148 running (13m) 10m ago 27m 105M 2642M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 1c656ab8ff86 2024-03-29T08:28:53.832 INFO:teuthology.orchestra.run.smithi037.stdout:osd.4 smithi148 running (12m) 10m ago 27m 97.3M 2642M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 966a9de11d22 2024-03-29T08:28:53.832 INFO:teuthology.orchestra.run.smithi037.stdout:osd.5 smithi148 running (11m) 10m ago 26m 85.8M 2642M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 474c4f9212ce 2024-03-29T08:28:53.832 INFO:teuthology.orchestra.run.smithi037.stdout:prometheus.smithi037 smithi037 *:9095 running (19m) 11m ago 31m 54.4M - 2.43.0 a07b618ecd1d 30c0223ec589 2024-03-29T08:28:53.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:53.830+0000 7f4ef3800700 1 -- 172.21.15.37:0/1599625812 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f4ed406cc30 msgr2=0x7f4ed406f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:53.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:53.830+0000 7f4ef3800700 1 --2- 172.21.15.37:0/1599625812 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f4ed406cc30 0x7f4ed406f0e0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f4ed800b020 tx=0x7f4ed8009e30 comp rx=0 tx=0).stop 2024-03-29T08:28:53.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:53.830+0000 7f4ef3800700 1 -- 172.21.15.37:0/1599625812 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4eec102e00 msgr2=0x7f4eec115c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:53.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:53.830+0000 7f4ef3800700 1 --2- 172.21.15.37:0/1599625812 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4eec102e00 0x7f4eec115c60 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f4ee0002a80 tx=0x7f4ee00046b0 comp rx=0 tx=0).stop 2024-03-29T08:28:53.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:53.830+0000 7f4ef3800700 1 -- 172.21.15.37:0/1599625812 shutdown_connections 2024-03-29T08:28:53.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:53.830+0000 7f4ef3800700 1 --2- 172.21.15.37:0/1599625812 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f4ed406cc30 0x7f4ed406f0e0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:53.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:53.830+0000 7f4ef3800700 1 --2- 172.21.15.37:0/1599625812 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4eec102e00 0x7f4eec115c60 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:53.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:53.830+0000 7f4ef3800700 1 --2- 172.21.15.37:0/1599625812 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4eec1037a0 0x7f4eec1161a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:53.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:53.830+0000 7f4ef3800700 1 -- 172.21.15.37:0/1599625812 >> 172.21.15.37:0/1599625812 conn(0x7f4eec0f89b0 msgr2=0x7f4eec0f99c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:28:53.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:53.831+0000 7f4ef3800700 1 -- 172.21.15.37:0/1599625812 shutdown_connections 2024-03-29T08:28:53.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:53.831+0000 7f4ef3800700 1 -- 172.21.15.37:0/1599625812 wait complete. 2024-03-29T08:28:53.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:53.967+0000 7ff7a5800700 1 -- 172.21.15.37:0/3986409891 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff7a0102e20 msgr2=0x7ff7a01031f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:53.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:53.967+0000 7ff7a5800700 1 --2- 172.21.15.37:0/3986409891 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff7a0102e20 0x7ff7a01031f0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7ff790004940 tx=0x7ff790009e30 comp rx=0 tx=0).stop 2024-03-29T08:28:53.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:53.967+0000 7ff7a5800700 1 -- 172.21.15.37:0/3986409891 shutdown_connections 2024-03-29T08:28:53.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:53.967+0000 7ff7a5800700 1 --2- 172.21.15.37:0/3986409891 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff7a01037c0 0x7ff7a00fce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:53.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:53.967+0000 7ff7a5800700 1 --2- 172.21.15.37:0/3986409891 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff7a0102e20 0x7ff7a01031f0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:53.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:53.967+0000 7ff7a5800700 1 -- 172.21.15.37:0/3986409891 >> 172.21.15.37:0/3986409891 conn(0x7ff7a00f89d0 msgr2=0x7ff7a00fade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:28:53.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:53.967+0000 7ff7a5800700 1 -- 172.21.15.37:0/3986409891 shutdown_connections 2024-03-29T08:28:53.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:53.967+0000 7ff7a5800700 1 -- 172.21.15.37:0/3986409891 wait complete. 2024-03-29T08:28:53.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:53.968+0000 7ff7a5800700 1 Processor -- start 2024-03-29T08:28:53.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:53.968+0000 7ff7a5800700 1 -- start start 2024-03-29T08:28:53.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:53.969+0000 7ff7a5800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff7a0102e20 0x7ff7a0113b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:53.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:53.969+0000 7ff7a5800700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff7a01037c0 0x7ff7a0114040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:53.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:53.969+0000 7ff7a5800700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7a0114690 con 0x7ff7a0102e20 2024-03-29T08:28:53.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:53.969+0000 7ff7a5800700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7a010da80 con 0x7ff7a01037c0 2024-03-29T08:28:53.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:53.969+0000 7ff79ea00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff7a0102e20 0x7ff7a0113b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:53.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:53.969+0000 7ff79ea00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff7a0102e20 0x7ff7a0113b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:48478/0 (socket says 172.21.15.37:48478) 2024-03-29T08:28:53.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:53.969+0000 7ff79ea00700 1 -- 172.21.15.37:0/3820080811 learned_addr learned my addr 172.21.15.37:0/3820080811 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:28:53.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:53.970+0000 7ff79e000700 1 --2- 172.21.15.37:0/3820080811 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff7a01037c0 0x7ff7a0114040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:53.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:53.970+0000 7ff79ea00700 1 -- 172.21.15.37:0/3820080811 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff7a01037c0 msgr2=0x7ff7a0114040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:53.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:53.970+0000 7ff79ea00700 1 --2- 172.21.15.37:0/3820080811 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff7a01037c0 0x7ff7a0114040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:53.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:53.970+0000 7ff79ea00700 1 -- 172.21.15.37:0/3820080811 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff790005040 con 0x7ff7a0102e20 2024-03-29T08:28:53.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:53.970+0000 7ff79ea00700 1 --2- 172.21.15.37:0/3820080811 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff7a0102e20 0x7ff7a0113b00 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7ff790009390 tx=0x7ff790009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:28:53.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:53.970+0000 7ff796a00700 1 -- 172.21.15.37:0/3820080811 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff79000e040 con 0x7ff7a0102e20 2024-03-29T08:28:53.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:53.970+0000 7ff796a00700 1 -- 172.21.15.37:0/3820080811 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ff790013420 con 0x7ff7a0102e20 2024-03-29T08:28:53.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:53.970+0000 7ff796a00700 1 -- 172.21.15.37:0/3820080811 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff79001c630 con 0x7ff7a0102e20 2024-03-29T08:28:53.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:53.970+0000 7ff7a5800700 1 -- 172.21.15.37:0/3820080811 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7a010dd00 con 0x7ff7a0102e20 2024-03-29T08:28:53.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:53.970+0000 7ff7a5800700 1 -- 172.21.15.37:0/3820080811 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7a010e2c0 con 0x7ff7a0102e20 2024-03-29T08:28:53.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:53.972+0000 7ff7a5800700 1 -- 172.21.15.37:0/3820080811 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7a00fe570 con 0x7ff7a0102e20 2024-03-29T08:28:53.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:53.972+0000 7ff796a00700 1 -- 172.21.15.37:0/3820080811 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7ff7900139e0 con 0x7ff7a0102e20 2024-03-29T08:28:53.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:53.973+0000 7ff796a00700 1 --2- 172.21.15.37:0/3820080811 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7ff78406cc30 0x7ff78406f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:53.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:53.973+0000 7ff796a00700 1 -- 172.21.15.37:0/3820080811 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7ff79008d250 con 0x7ff7a0102e20 2024-03-29T08:28:53.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:53.973+0000 7ff79e000700 1 --2- 172.21.15.37:0/3820080811 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7ff78406cc30 0x7ff78406f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:53.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:53.974+0000 7ff79e000700 1 --2- 172.21.15.37:0/3820080811 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7ff78406cc30 0x7ff78406f0e0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7ff7a010ec40 tx=0x7ff78c009ee0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:28:53.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:53.978+0000 7ff796a00700 1 -- 172.21.15.37:0/3820080811 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7ff79005add0 con 0x7ff7a0102e20 2024-03-29T08:28:54.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:54.278+0000 7ff7a5800700 1 -- 172.21.15.37:0/3820080811 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7ff7a004bed0 con 0x7ff7a0102e20 2024-03-29T08:28:54.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:54.279+0000 7ff796a00700 1 -- 172.21.15.37:0/3820080811 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+847 (secure 0 0 0) 0x7ff79005a960 con 0x7ff7a0102e20 2024-03-29T08:28:54.279 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-29T08:28:54.279 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": { 2024-03-29T08:28:54.279 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 2 2024-03-29T08:28:54.280 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:28:54.280 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": { 2024-03-29T08:28:54.280 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 2 2024-03-29T08:28:54.280 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:28:54.280 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": { 2024-03-29T08:28:54.280 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 6 2024-03-29T08:28:54.280 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:28:54.280 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": { 2024-03-29T08:28:54.280 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 17.2.7-789-g4213e17c (4213e17c4de67016ed14f55d22503f4923b3a901) quincy (stable)": 1, 2024-03-29T08:28:54.280 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 3 2024-03-29T08:28:54.280 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:28:54.280 INFO:teuthology.orchestra.run.smithi037.stdout: "overall": { 2024-03-29T08:28:54.280 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 17.2.7-789-g4213e17c (4213e17c4de67016ed14f55d22503f4923b3a901) quincy (stable)": 1, 2024-03-29T08:28:54.280 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 13 2024-03-29T08:28:54.281 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-29T08:28:54.281 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-29T08:28:54.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:54.281+0000 7ff7a5800700 1 -- 172.21.15.37:0/3820080811 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7ff78406cc30 msgr2=0x7ff78406f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:54.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:54.281+0000 7ff7a5800700 1 --2- 172.21.15.37:0/3820080811 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7ff78406cc30 0x7ff78406f0e0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7ff7a010ec40 tx=0x7ff78c009ee0 comp rx=0 tx=0).stop 2024-03-29T08:28:54.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:54.281+0000 7ff7a5800700 1 -- 172.21.15.37:0/3820080811 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff7a0102e20 msgr2=0x7ff7a0113b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:54.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:54.281+0000 7ff7a5800700 1 --2- 172.21.15.37:0/3820080811 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff7a0102e20 0x7ff7a0113b00 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7ff790009390 tx=0x7ff790009e30 comp rx=0 tx=0).stop 2024-03-29T08:28:54.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:54.282+0000 7ff7a5800700 1 -- 172.21.15.37:0/3820080811 shutdown_connections 2024-03-29T08:28:54.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:54.282+0000 7ff7a5800700 1 --2- 172.21.15.37:0/3820080811 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7ff78406cc30 0x7ff78406f0e0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:54.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:54.282+0000 7ff7a5800700 1 --2- 172.21.15.37:0/3820080811 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff7a0102e20 0x7ff7a0113b00 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:54.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:54.282+0000 7ff7a5800700 1 --2- 172.21.15.37:0/3820080811 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff7a01037c0 0x7ff7a0114040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:54.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:54.282+0000 7ff7a5800700 1 -- 172.21.15.37:0/3820080811 >> 172.21.15.37:0/3820080811 conn(0x7ff7a00f89d0 msgr2=0x7ff7a00f9ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:28:54.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:54.282+0000 7ff7a5800700 1 -- 172.21.15.37:0/3820080811 shutdown_connections 2024-03-29T08:28:54.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:54.282+0000 7ff7a5800700 1 -- 172.21.15.37:0/3820080811 wait complete. 2024-03-29T08:28:54.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:54.419+0000 7f1153200700 1 -- 172.21.15.37:0/2557135029 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f114c102e20 msgr2=0x7f114c1031f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:54.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:54.419+0000 7f1153200700 1 --2- 172.21.15.37:0/2557135029 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f114c102e20 0x7f114c1031f0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f113c004940 tx=0x7f113c009e30 comp rx=0 tx=0).stop 2024-03-29T08:28:54.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:54.419+0000 7f1153200700 1 -- 172.21.15.37:0/2557135029 shutdown_connections 2024-03-29T08:28:54.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:54.419+0000 7f1153200700 1 --2- 172.21.15.37:0/2557135029 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f114c1037c0 0x7f114c0fce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:54.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:54.419+0000 7f1153200700 1 --2- 172.21.15.37:0/2557135029 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f114c102e20 0x7f114c1031f0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:54.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:54.419+0000 7f1153200700 1 -- 172.21.15.37:0/2557135029 >> 172.21.15.37:0/2557135029 conn(0x7f114c0f8970 msgr2=0x7f114c0fad80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:28:54.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:54.419+0000 7f1153200700 1 -- 172.21.15.37:0/2557135029 shutdown_connections 2024-03-29T08:28:54.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:54.419+0000 7f1153200700 1 -- 172.21.15.37:0/2557135029 wait complete. 2024-03-29T08:28:54.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:54.420+0000 7f1153200700 1 Processor -- start 2024-03-29T08:28:54.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:54.420+0000 7f1153200700 1 -- start start 2024-03-29T08:28:54.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:54.421+0000 7f1153200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f114c102e20 0x7f114c0702c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:54.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:54.421+0000 7f1153200700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f114c1037c0 0x7f114c070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:54.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:54.421+0000 7f1153200700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f114c070d90 con 0x7f114c102e20 2024-03-29T08:28:54.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:54.421+0000 7f1153200700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f114c070ed0 con 0x7f114c1037c0 2024-03-29T08:28:54.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:54.421+0000 7f114be00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f114c102e20 0x7f114c0702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:54.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:54.421+0000 7f114be00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f114c102e20 0x7f114c0702c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:48502/0 (socket says 172.21.15.37:48502) 2024-03-29T08:28:54.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:54.421+0000 7f114be00700 1 -- 172.21.15.37:0/4235997384 learned_addr learned my addr 172.21.15.37:0/4235997384 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:28:54.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:54.421+0000 7f114b400700 1 --2- 172.21.15.37:0/4235997384 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f114c1037c0 0x7f114c070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:54.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:54.422+0000 7f114be00700 1 -- 172.21.15.37:0/4235997384 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f114c1037c0 msgr2=0x7f114c070800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:54.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:54.422+0000 7f114be00700 1 --2- 172.21.15.37:0/4235997384 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f114c1037c0 0x7f114c070800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:54.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:54.422+0000 7f114be00700 1 -- 172.21.15.37:0/4235997384 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f113c005040 con 0x7f114c102e20 2024-03-29T08:28:54.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:54.422+0000 7f114be00700 1 --2- 172.21.15.37:0/4235997384 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f114c102e20 0x7f114c0702c0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f113c009360 tx=0x7f113c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:28:54.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:54.422+0000 7f1143e00700 1 -- 172.21.15.37:0/4235997384 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f113c00e040 con 0x7f114c102e20 2024-03-29T08:28:54.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:54.422+0000 7f1153200700 1 -- 172.21.15.37:0/4235997384 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f114c071120 con 0x7f114c102e20 2024-03-29T08:28:54.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:54.422+0000 7f1143e00700 1 -- 172.21.15.37:0/4235997384 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f113c013420 con 0x7f114c102e20 2024-03-29T08:28:54.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:54.422+0000 7f1143e00700 1 -- 172.21.15.37:0/4235997384 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f113c01c5d0 con 0x7f114c102e20 2024-03-29T08:28:54.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:54.422+0000 7f1153200700 1 -- 172.21.15.37:0/4235997384 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f114c11a210 con 0x7f114c102e20 2024-03-29T08:28:54.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:54.423+0000 7f1153200700 1 -- 172.21.15.37:0/4235997384 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f114c0fe570 con 0x7f114c102e20 2024-03-29T08:28:54.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:54.424+0000 7f1143e00700 1 -- 172.21.15.37:0/4235997384 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f113c013590 con 0x7f114c102e20 2024-03-29T08:28:54.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:54.425+0000 7f1143e00700 1 --2- 172.21.15.37:0/4235997384 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f113006cc30 0x7f113006f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:54.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:54.425+0000 7f1143e00700 1 -- 172.21.15.37:0/4235997384 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f113c08d250 con 0x7f114c102e20 2024-03-29T08:28:54.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:54.426+0000 7f114b400700 1 --2- 172.21.15.37:0/4235997384 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f113006cc30 0x7f113006f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:54.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:54.426+0000 7f114b400700 1 --2- 172.21.15.37:0/4235997384 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f113006cc30 0x7f113006f0e0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f114c071c00 tx=0x7f1138009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:28:54.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:54.430+0000 7f1143e00700 1 -- 172.21.15.37:0/4235997384 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f113c057190 con 0x7f114c102e20 2024-03-29T08:28:54.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:28:54 smithi037 ceph-mon[142325]: from='client.35008 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:28:54.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:28:54 smithi037 ceph-mon[142325]: from='client.35012 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:28:54.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:28:54 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/3820080811' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:28:54.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:54.688+0000 7f1153200700 1 -- 172.21.15.37:0/4235997384 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f114c0719b0 con 0x7f114c102e20 2024-03-29T08:28:54.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:54.688+0000 7f1143e00700 1 -- 172.21.15.37:0/4235997384 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 28 v28) v1 ==== 76+0+1841 (secure 0 0 0) 0x7f113c05a7b0 con 0x7f114c102e20 2024-03-29T08:28:54.688 INFO:teuthology.orchestra.run.smithi037.stdout:e28 2024-03-29T08:28:54.689 INFO:teuthology.orchestra.run.smithi037.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-29T08:28:54.689 INFO:teuthology.orchestra.run.smithi037.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T08:28:54.689 INFO:teuthology.orchestra.run.smithi037.stdout:legacy client fscid: 1 2024-03-29T08:28:54.689 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:28:54.689 INFO:teuthology.orchestra.run.smithi037.stdout:Filesystem 'cephfs' (1) 2024-03-29T08:28:54.689 INFO:teuthology.orchestra.run.smithi037.stdout:fs_name cephfs 2024-03-29T08:28:54.689 INFO:teuthology.orchestra.run.smithi037.stdout:epoch 21 2024-03-29T08:28:54.689 INFO:teuthology.orchestra.run.smithi037.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-03-29T08:28:54.689 INFO:teuthology.orchestra.run.smithi037.stdout:created 2024-03-29T08:03:29.241269+0000 2024-03-29T08:28:54.689 INFO:teuthology.orchestra.run.smithi037.stdout:modified 2024-03-29T08:17:16.237238+0000 2024-03-29T08:28:54.689 INFO:teuthology.orchestra.run.smithi037.stdout:tableserver 0 2024-03-29T08:28:54.689 INFO:teuthology.orchestra.run.smithi037.stdout:root 0 2024-03-29T08:28:54.689 INFO:teuthology.orchestra.run.smithi037.stdout:session_timeout 60 2024-03-29T08:28:54.690 INFO:teuthology.orchestra.run.smithi037.stdout:session_autoclose 300 2024-03-29T08:28:54.690 INFO:teuthology.orchestra.run.smithi037.stdout:max_file_size 1099511627776 2024-03-29T08:28:54.690 INFO:teuthology.orchestra.run.smithi037.stdout:max_xattr_size 65536 2024-03-29T08:28:54.690 INFO:teuthology.orchestra.run.smithi037.stdout:required_client_features {} 2024-03-29T08:28:54.690 INFO:teuthology.orchestra.run.smithi037.stdout:last_failure 0 2024-03-29T08:28:54.690 INFO:teuthology.orchestra.run.smithi037.stdout:last_failure_osd_epoch 82 2024-03-29T08:28:54.690 INFO:teuthology.orchestra.run.smithi037.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T08:28:54.690 INFO:teuthology.orchestra.run.smithi037.stdout:max_mds 2 2024-03-29T08:28:54.690 INFO:teuthology.orchestra.run.smithi037.stdout:in 0,1 2024-03-29T08:28:54.690 INFO:teuthology.orchestra.run.smithi037.stdout:up {} 2024-03-29T08:28:54.690 INFO:teuthology.orchestra.run.smithi037.stdout:failed 0,1 2024-03-29T08:28:54.690 INFO:teuthology.orchestra.run.smithi037.stdout:damaged 2024-03-29T08:28:54.690 INFO:teuthology.orchestra.run.smithi037.stdout:stopped 2024-03-29T08:28:54.690 INFO:teuthology.orchestra.run.smithi037.stdout:data_pools [3] 2024-03-29T08:28:54.691 INFO:teuthology.orchestra.run.smithi037.stdout:metadata_pool 2 2024-03-29T08:28:54.691 INFO:teuthology.orchestra.run.smithi037.stdout:inline_data disabled 2024-03-29T08:28:54.691 INFO:teuthology.orchestra.run.smithi037.stdout:balancer 2024-03-29T08:28:54.691 INFO:teuthology.orchestra.run.smithi037.stdout:bal_rank_mask -1 2024-03-29T08:28:54.691 INFO:teuthology.orchestra.run.smithi037.stdout:standby_count_wanted 1 2024-03-29T08:28:54.691 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:28:54.691 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:28:54.691 INFO:teuthology.orchestra.run.smithi037.stdout:Standby daemons: 2024-03-29T08:28:54.691 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:28:54.691 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi148.qukpjh{-1:24319} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.148:6826/2641327912,v1:172.21.15.148:6827/2641327912] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:28:54.691 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi037.mmwgxe{-1:34440} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.37:6828/1248142005,v1:172.21.15.37:6829/1248142005] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:28:54.691 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi037.iifsmh{-1:34444} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.37:6826/2060882959,v1:172.21.15.37:6827/2060882959] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:28:54.691 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi148.sqteih{-1:44395} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.148:6824/4078652242,v1:172.21.15.148:6825/4078652242] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:28:54.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:54.690+0000 7f1153200700 1 -- 172.21.15.37:0/4235997384 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f113006cc30 msgr2=0x7f113006f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:54.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:54.690+0000 7f1153200700 1 --2- 172.21.15.37:0/4235997384 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f113006cc30 0x7f113006f0e0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f114c071c00 tx=0x7f1138009040 comp rx=0 tx=0).stop 2024-03-29T08:28:54.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:54.690+0000 7f1153200700 1 -- 172.21.15.37:0/4235997384 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f114c102e20 msgr2=0x7f114c0702c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:54.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:54.690+0000 7f1153200700 1 --2- 172.21.15.37:0/4235997384 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f114c102e20 0x7f114c0702c0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f113c009360 tx=0x7f113c009e30 comp rx=0 tx=0).stop 2024-03-29T08:28:54.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:54.691+0000 7f1153200700 1 -- 172.21.15.37:0/4235997384 shutdown_connections 2024-03-29T08:28:54.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:54.691+0000 7f1153200700 1 --2- 172.21.15.37:0/4235997384 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f113006cc30 0x7f113006f0e0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:54.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:54.691+0000 7f1153200700 1 --2- 172.21.15.37:0/4235997384 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f114c102e20 0x7f114c0702c0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:54.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:54.691+0000 7f1153200700 1 --2- 172.21.15.37:0/4235997384 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f114c1037c0 0x7f114c070800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:54.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:54.691+0000 7f1153200700 1 -- 172.21.15.37:0/4235997384 >> 172.21.15.37:0/4235997384 conn(0x7f114c0f8970 msgr2=0x7f114c0f9a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:28:54.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:54.691+0000 7f1153200700 1 -- 172.21.15.37:0/4235997384 shutdown_connections 2024-03-29T08:28:54.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:54.691+0000 7f1153200700 1 -- 172.21.15.37:0/4235997384 wait complete. 2024-03-29T08:28:54.693 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 28 2024-03-29T08:28:54.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:28:54 smithi148 ceph-mon[106692]: from='client.35008 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:28:54.807 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:28:54 smithi148 ceph-mon[106692]: from='client.35012 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:28:54.807 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:28:54 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/3820080811' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:28:54.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:54.830+0000 7f4bdee00700 1 -- 172.21.15.37:0/3659889879 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4bd8102710 msgr2=0x7f4bd80fbd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:54.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:54.830+0000 7f4bdee00700 1 --2- 172.21.15.37:0/3659889879 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4bd8102710 0x7f4bd80fbd80 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f4bc4004940 tx=0x7f4bc4009e30 comp rx=0 tx=0).stop 2024-03-29T08:28:54.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:54.830+0000 7f4bdee00700 1 -- 172.21.15.37:0/3659889879 shutdown_connections 2024-03-29T08:28:54.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:54.830+0000 7f4bdee00700 1 --2- 172.21.15.37:0/3659889879 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4bd8102710 0x7f4bd80fbd80 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:54.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:54.830+0000 7f4bdee00700 1 --2- 172.21.15.37:0/3659889879 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4bd8101d70 0x7f4bd8102140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:54.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:54.830+0000 7f4bdee00700 1 -- 172.21.15.37:0/3659889879 >> 172.21.15.37:0/3659889879 conn(0x7f4bd8072bc0 msgr2=0x7f4bd8072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:28:54.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:54.830+0000 7f4bdee00700 1 -- 172.21.15.37:0/3659889879 shutdown_connections 2024-03-29T08:28:54.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:54.830+0000 7f4bdee00700 1 -- 172.21.15.37:0/3659889879 wait complete. 2024-03-29T08:28:54.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:54.831+0000 7f4bdee00700 1 Processor -- start 2024-03-29T08:28:54.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:54.831+0000 7f4bdee00700 1 -- start start 2024-03-29T08:28:54.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:54.831+0000 7f4bdee00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4bd8101d70 0x7f4bd8102bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:54.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:54.831+0000 7f4bdee00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4bd8102710 0x7f4bd8103110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:54.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:54.831+0000 7f4bdee00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bd81036a0 con 0x7f4bd8102710 2024-03-29T08:28:54.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:54.831+0000 7f4bdee00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bd81037e0 con 0x7f4bd8101d70 2024-03-29T08:28:54.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:54.832+0000 7f4bd7400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4bd8102710 0x7f4bd8103110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:54.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:54.832+0000 7f4bd7400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4bd8102710 0x7f4bd8103110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:48512/0 (socket says 172.21.15.37:48512) 2024-03-29T08:28:54.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:54.832+0000 7f4bd7400700 1 -- 172.21.15.37:0/4098424505 learned_addr learned my addr 172.21.15.37:0/4098424505 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:28:54.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:54.832+0000 7f4bd7e00700 1 --2- 172.21.15.37:0/4098424505 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4bd8101d70 0x7f4bd8102bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:54.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:54.832+0000 7f4bd7400700 1 -- 172.21.15.37:0/4098424505 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4bd8101d70 msgr2=0x7f4bd8102bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:54.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:54.832+0000 7f4bd7400700 1 --2- 172.21.15.37:0/4098424505 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4bd8101d70 0x7f4bd8102bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:54.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:54.832+0000 7f4bd7400700 1 -- 172.21.15.37:0/4098424505 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bc4005040 con 0x7f4bd8102710 2024-03-29T08:28:54.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:54.832+0000 7f4bd7400700 1 --2- 172.21.15.37:0/4098424505 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4bd8102710 0x7f4bd8103110 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f4bc40093b0 tx=0x7f4bc4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:28:54.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:54.833+0000 7f4bcfe00700 1 -- 172.21.15.37:0/4098424505 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4bc400e040 con 0x7f4bd8102710 2024-03-29T08:28:54.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:54.833+0000 7f4bcfe00700 1 -- 172.21.15.37:0/4098424505 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f4bc4013420 con 0x7f4bd8102710 2024-03-29T08:28:54.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:54.833+0000 7f4bdee00700 1 -- 172.21.15.37:0/4098424505 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bd8103a30 con 0x7f4bd8102710 2024-03-29T08:28:54.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:54.833+0000 7f4bcfe00700 1 -- 172.21.15.37:0/4098424505 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4bc401c630 con 0x7f4bd8102710 2024-03-29T08:28:54.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:54.833+0000 7f4bdee00700 1 -- 172.21.15.37:0/4098424505 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bd811a1f0 con 0x7f4bd8102710 2024-03-29T08:28:54.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:54.834+0000 7f4bdee00700 1 -- 172.21.15.37:0/4098424505 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4bd80fd4c0 con 0x7f4bd8102710 2024-03-29T08:28:54.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:54.835+0000 7f4bcfe00700 1 -- 172.21.15.37:0/4098424505 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f4bc40139e0 con 0x7f4bd8102710 2024-03-29T08:28:54.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:54.836+0000 7f4bcfe00700 1 --2- 172.21.15.37:0/4098424505 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f4bbc06cc20 0x7f4bbc06f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:54.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:54.836+0000 7f4bcfe00700 1 -- 172.21.15.37:0/4098424505 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f4bc408d160 con 0x7f4bd8102710 2024-03-29T08:28:54.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:54.836+0000 7f4bd7e00700 1 --2- 172.21.15.37:0/4098424505 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f4bbc06cc20 0x7f4bbc06f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:54.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:54.837+0000 7f4bd7e00700 1 --2- 172.21.15.37:0/4098424505 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f4bbc06cc20 0x7f4bbc06f0d0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f4bc8004f40 tx=0x7f4bc800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:28:54.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:54.841+0000 7f4bcfe00700 1 -- 172.21.15.37:0/4098424505 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f4bc4057250 con 0x7f4bd8102710 2024-03-29T08:28:55.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:55.072+0000 7f4bdee00700 1 -- 172.21.15.37:0/4098424505 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4bd8108db0 con 0x7f4bbc06cc20 2024-03-29T08:28:55.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:55.074+0000 7f4bcfe00700 1 -- 172.21.15.37:0/4098424505 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f4bc800f040 con 0x7f4bbc06cc20 2024-03-29T08:28:55.074 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-29T08:28:55.074 INFO:teuthology.orchestra.run.smithi037.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59", 2024-03-29T08:28:55.074 INFO:teuthology.orchestra.run.smithi037.stdout: "in_progress": true, 2024-03-29T08:28:55.074 INFO:teuthology.orchestra.run.smithi037.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-29T08:28:55.074 INFO:teuthology.orchestra.run.smithi037.stdout: "services_complete": [ 2024-03-29T08:28:55.074 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr", 2024-03-29T08:28:55.074 INFO:teuthology.orchestra.run.smithi037.stdout: "mon", 2024-03-29T08:28:55.074 INFO:teuthology.orchestra.run.smithi037.stdout: "crash", 2024-03-29T08:28:55.074 INFO:teuthology.orchestra.run.smithi037.stdout: "osd" 2024-03-29T08:28:55.075 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-29T08:28:55.075 INFO:teuthology.orchestra.run.smithi037.stdout: "progress": "15/21 daemons upgraded", 2024-03-29T08:28:55.075 INFO:teuthology.orchestra.run.smithi037.stdout: "message": "Currently upgrading mds daemons", 2024-03-29T08:28:55.075 INFO:teuthology.orchestra.run.smithi037.stdout: "is_paused": false 2024-03-29T08:28:55.075 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-29T08:28:55.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:55.076+0000 7f4bdee00700 1 -- 172.21.15.37:0/4098424505 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f4bbc06cc20 msgr2=0x7f4bbc06f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:55.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:55.076+0000 7f4bdee00700 1 --2- 172.21.15.37:0/4098424505 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f4bbc06cc20 0x7f4bbc06f0d0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f4bc8004f40 tx=0x7f4bc800b040 comp rx=0 tx=0).stop 2024-03-29T08:28:55.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:55.076+0000 7f4bdee00700 1 -- 172.21.15.37:0/4098424505 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4bd8102710 msgr2=0x7f4bd8103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:55.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:55.076+0000 7f4bdee00700 1 --2- 172.21.15.37:0/4098424505 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4bd8102710 0x7f4bd8103110 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f4bc40093b0 tx=0x7f4bc4009e30 comp rx=0 tx=0).stop 2024-03-29T08:28:55.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:55.076+0000 7f4bdee00700 1 -- 172.21.15.37:0/4098424505 shutdown_connections 2024-03-29T08:28:55.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:55.076+0000 7f4bdee00700 1 --2- 172.21.15.37:0/4098424505 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f4bbc06cc20 0x7f4bbc06f0d0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:55.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:55.076+0000 7f4bdee00700 1 --2- 172.21.15.37:0/4098424505 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4bd8101d70 0x7f4bd8102bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:55.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:55.076+0000 7f4bdee00700 1 --2- 172.21.15.37:0/4098424505 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4bd8102710 0x7f4bd8103110 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:55.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:55.077+0000 7f4bdee00700 1 -- 172.21.15.37:0/4098424505 >> 172.21.15.37:0/4098424505 conn(0x7f4bd8072bc0 msgr2=0x7f4bd8065a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:28:55.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:55.077+0000 7f4bdee00700 1 -- 172.21.15.37:0/4098424505 shutdown_connections 2024-03-29T08:28:55.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:55.077+0000 7f4bdee00700 1 -- 172.21.15.37:0/4098424505 wait complete. 2024-03-29T08:28:55.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:55.214+0000 7f829d800700 1 -- 172.21.15.37:0/2257269854 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8298102e00 msgr2=0x7f82981031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:55.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:55.214+0000 7f829d800700 1 --2- 172.21.15.37:0/2257269854 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8298102e00 0x7f82981031d0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f8288004940 tx=0x7f8288009e30 comp rx=0 tx=0).stop 2024-03-29T08:28:55.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:55.214+0000 7f829d800700 1 -- 172.21.15.37:0/2257269854 shutdown_connections 2024-03-29T08:28:55.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:55.214+0000 7f829d800700 1 --2- 172.21.15.37:0/2257269854 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f82981037a0 0x7f82980fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:55.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:55.214+0000 7f829d800700 1 --2- 172.21.15.37:0/2257269854 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8298102e00 0x7f82981031d0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:55.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:55.214+0000 7f829d800700 1 -- 172.21.15.37:0/2257269854 >> 172.21.15.37:0/2257269854 conn(0x7f82980f89b0 msgr2=0x7f82980fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:28:55.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:55.214+0000 7f829d800700 1 -- 172.21.15.37:0/2257269854 shutdown_connections 2024-03-29T08:28:55.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:55.214+0000 7f829d800700 1 -- 172.21.15.37:0/2257269854 wait complete. 2024-03-29T08:28:55.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:55.215+0000 7f829d800700 1 Processor -- start 2024-03-29T08:28:55.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:55.215+0000 7f829d800700 1 -- start start 2024-03-29T08:28:55.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:55.216+0000 7f829d800700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8298102e00 0x7f8298115c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:55.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:55.216+0000 7f829d800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f82981037a0 0x7f8298116180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:55.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:55.216+0000 7f829d800700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82981167d0 con 0x7f82981037a0 2024-03-29T08:28:55.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:55.216+0000 7f829d800700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f829810fba0 con 0x7f8298102e00 2024-03-29T08:28:55.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:55.216+0000 7f8296000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f82981037a0 0x7f8298116180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:55.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:55.216+0000 7f8296000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f82981037a0 0x7f8298116180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:48526/0 (socket says 172.21.15.37:48526) 2024-03-29T08:28:55.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:55.216+0000 7f8296000700 1 -- 172.21.15.37:0/2149946967 learned_addr learned my addr 172.21.15.37:0/2149946967 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:28:55.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:55.216+0000 7f8296a00700 1 --2- 172.21.15.37:0/2149946967 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8298102e00 0x7f8298115c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:55.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:55.216+0000 7f8296000700 1 -- 172.21.15.37:0/2149946967 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8298102e00 msgr2=0x7f8298115c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:55.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:55.216+0000 7f8296000700 1 --2- 172.21.15.37:0/2149946967 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8298102e00 0x7f8298115c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:55.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:55.216+0000 7f8296000700 1 -- 172.21.15.37:0/2149946967 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8288005040 con 0x7f82981037a0 2024-03-29T08:28:55.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:55.217+0000 7f8296000700 1 --2- 172.21.15.37:0/2149946967 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f82981037a0 0x7f8298116180 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f8284002a50 tx=0x7f82840046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:28:55.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:55.217+0000 7f828ea00700 1 -- 172.21.15.37:0/2149946967 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8284007cc0 con 0x7f82981037a0 2024-03-29T08:28:55.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:55.217+0000 7f828ea00700 1 -- 172.21.15.37:0/2149946967 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f828400f040 con 0x7f82981037a0 2024-03-29T08:28:55.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:55.217+0000 7f829d800700 1 -- 172.21.15.37:0/2149946967 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f829810fe80 con 0x7f82981037a0 2024-03-29T08:28:55.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:55.217+0000 7f828ea00700 1 -- 172.21.15.37:0/2149946967 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f828400da30 con 0x7f82981037a0 2024-03-29T08:28:55.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:55.217+0000 7f829d800700 1 -- 172.21.15.37:0/2149946967 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8298110440 con 0x7f82981037a0 2024-03-29T08:28:55.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:55.218+0000 7f829d800700 1 -- 172.21.15.37:0/2149946967 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82980fe550 con 0x7f82981037a0 2024-03-29T08:28:55.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:55.219+0000 7f828ea00700 1 -- 172.21.15.37:0/2149946967 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f828400db90 con 0x7f82981037a0 2024-03-29T08:28:55.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:55.220+0000 7f828ea00700 1 --2- 172.21.15.37:0/2149946967 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f827c06cc20 0x7f827c06f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:55.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:55.220+0000 7f828ea00700 1 -- 172.21.15.37:0/2149946967 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f828408bd00 con 0x7f82981037a0 2024-03-29T08:28:55.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:55.220+0000 7f8296a00700 1 --2- 172.21.15.37:0/2149946967 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f827c06cc20 0x7f827c06f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:55.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:55.221+0000 7f8296a00700 1 --2- 172.21.15.37:0/2149946967 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f827c06cc20 0x7f827c06f0d0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f8288004f90 tx=0x7f8288009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:28:55.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:55.225+0000 7f828ea00700 1 -- 172.21.15.37:0/2149946967 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f82840596d0 con 0x7f82981037a0 2024-03-29T08:28:55.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:55.521+0000 7f829d800700 1 -- 172.21.15.37:0/2149946967 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f829805f5f0 con 0x7f82981037a0 2024-03-29T08:28:55.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:55.521+0000 7f828ea00700 1 -- 172.21.15.37:0/2149946967 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7f8284077e60 con 0x7f82981037a0 2024-03-29T08:28:55.521 INFO:teuthology.orchestra.run.smithi037.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-03-29T08:28:55.521 INFO:teuthology.orchestra.run.smithi037.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-03-29T08:28:55.521 INFO:teuthology.orchestra.run.smithi037.stdout: fs cephfs is degraded 2024-03-29T08:28:55.522 INFO:teuthology.orchestra.run.smithi037.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-03-29T08:28:55.522 INFO:teuthology.orchestra.run.smithi037.stdout: fs cephfs is offline because no MDS is active for it. 2024-03-29T08:28:55.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:55.523+0000 7f829d800700 1 -- 172.21.15.37:0/2149946967 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f827c06cc20 msgr2=0x7f827c06f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:55.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:55.523+0000 7f829d800700 1 --2- 172.21.15.37:0/2149946967 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f827c06cc20 0x7f827c06f0d0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f8288004f90 tx=0x7f8288009e30 comp rx=0 tx=0).stop 2024-03-29T08:28:55.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:55.523+0000 7f829d800700 1 -- 172.21.15.37:0/2149946967 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f82981037a0 msgr2=0x7f8298116180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:55.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:55.523+0000 7f829d800700 1 --2- 172.21.15.37:0/2149946967 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f82981037a0 0x7f8298116180 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f8284002a50 tx=0x7f82840046b0 comp rx=0 tx=0).stop 2024-03-29T08:28:55.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:55.523+0000 7f829d800700 1 -- 172.21.15.37:0/2149946967 shutdown_connections 2024-03-29T08:28:55.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:55.523+0000 7f829d800700 1 --2- 172.21.15.37:0/2149946967 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f827c06cc20 0x7f827c06f0d0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:55.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:55.523+0000 7f829d800700 1 --2- 172.21.15.37:0/2149946967 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8298102e00 0x7f8298115c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:55.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:55.523+0000 7f829d800700 1 --2- 172.21.15.37:0/2149946967 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f82981037a0 0x7f8298116180 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:55.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:55.523+0000 7f829d800700 1 -- 172.21.15.37:0/2149946967 >> 172.21.15.37:0/2149946967 conn(0x7f82980f89b0 msgr2=0x7f82980f99f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:28:55.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:55.524+0000 7f829d800700 1 -- 172.21.15.37:0/2149946967 shutdown_connections 2024-03-29T08:28:55.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:28:55.524+0000 7f829d800700 1 -- 172.21.15.37:0/2149946967 wait complete. 2024-03-29T08:28:55.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:28:55 smithi037 ceph-mon[142325]: pgmap v540: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:28:55.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:28:55 smithi037 ceph-mon[142325]: from='client.35016 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:28:55.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:28:55 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:28:55.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:28:55 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/4235997384' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T08:28:55.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:28:55 smithi148 ceph-mon[106692]: pgmap v540: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:28:55.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:28:55 smithi148 ceph-mon[106692]: from='client.35016 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:28:55.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:28:55 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:28:55.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:28:55 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/4235997384' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T08:28:56.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:28:56 smithi037 ceph-mon[142325]: from='client.35028 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:28:56.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:28:56 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/2149946967' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T08:28:56.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:28:56 smithi148 ceph-mon[106692]: from='client.35028 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:28:56.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:28:56 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/2149946967' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T08:28:57.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:28:57 smithi037 ceph-mon[142325]: pgmap v541: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:28:57.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:28:57 smithi148 ceph-mon[106692]: pgmap v541: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:28:59.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:28:59 smithi037 ceph-mon[142325]: pgmap v542: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:28:59.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:28:59 smithi148 ceph-mon[106692]: pgmap v542: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:29:01.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:29:01 smithi037 ceph-mon[142325]: pgmap v543: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:29:01.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:29:01 smithi148 ceph-mon[106692]: pgmap v543: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:29:03.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:29:03 smithi037 ceph-mon[142325]: pgmap v544: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:29:03.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:29:03 smithi148 ceph-mon[106692]: pgmap v544: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:29:05.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:29:05 smithi037 ceph-mon[142325]: pgmap v545: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:29:05.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:29:05 smithi148 ceph-mon[106692]: pgmap v545: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:29:07.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:29:07 smithi037 ceph-mon[142325]: pgmap v546: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:29:07.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:29:07 smithi148 ceph-mon[106692]: pgmap v546: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:29:09.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:29:09 smithi037 ceph-mon[142325]: pgmap v547: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:29:09.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:29:09 smithi148 ceph-mon[106692]: pgmap v547: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:29:10.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:29:10 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:29:10.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:29:10 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:29:11.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:29:11 smithi037 ceph-mon[142325]: pgmap v548: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:29:11.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:29:11 smithi148 ceph-mon[106692]: pgmap v548: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:29:13.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:29:13 smithi037 ceph-mon[142325]: pgmap v549: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:29:13.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:29:13 smithi148 ceph-mon[106692]: pgmap v549: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:29:15.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:29:15 smithi037 ceph-mon[142325]: pgmap v550: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:29:15.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:29:15 smithi148 ceph-mon[106692]: pgmap v550: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:29:17.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:29:17 smithi037 ceph-mon[142325]: pgmap v551: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:29:17.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:29:17 smithi148 ceph-mon[106692]: pgmap v551: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:29:19.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:29:19 smithi037 ceph-mon[142325]: pgmap v552: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:29:19.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:29:19 smithi148 ceph-mon[106692]: pgmap v552: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:29:21.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:29:21 smithi037 ceph-mon[142325]: pgmap v553: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:29:21.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:29:21 smithi148 ceph-mon[106692]: pgmap v553: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:29:23.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:29:23 smithi148 ceph-mon[106692]: pgmap v554: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:29:23.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:29:23 smithi037 ceph-mon[142325]: pgmap v554: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:29:25.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:25.678+0000 7f1b27200700 1 -- 172.21.15.37:0/4029263290 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1b28102c40 msgr2=0x7f1b28103010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:25.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:25.678+0000 7f1b27200700 1 --2- 172.21.15.37:0/4029263290 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1b28102c40 0x7f1b28103010 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f1b18004940 tx=0x7f1b18009e30 comp rx=0 tx=0).stop 2024-03-29T08:29:25.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:25.678+0000 7f1b27200700 1 -- 172.21.15.37:0/4029263290 shutdown_connections 2024-03-29T08:29:25.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:25.678+0000 7f1b27200700 1 --2- 172.21.15.37:0/4029263290 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1b281035e0 0x7f1b280fcc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:25.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:25.678+0000 7f1b27200700 1 --2- 172.21.15.37:0/4029263290 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1b28102c40 0x7f1b28103010 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:25.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:25.678+0000 7f1b27200700 1 -- 172.21.15.37:0/4029263290 >> 172.21.15.37:0/4029263290 conn(0x7f1b280f87d0 msgr2=0x7f1b280fabe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:29:25.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:25.678+0000 7f1b27200700 1 -- 172.21.15.37:0/4029263290 shutdown_connections 2024-03-29T08:29:25.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:25.678+0000 7f1b27200700 1 -- 172.21.15.37:0/4029263290 wait complete. 2024-03-29T08:29:25.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:25.679+0000 7f1b27200700 1 Processor -- start 2024-03-29T08:29:25.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:25.679+0000 7f1b27200700 1 -- start start 2024-03-29T08:29:25.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:25.680+0000 7f1b27200700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1b28102c40 0x7f1b28115b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:25.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:25.680+0000 7f1b27200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1b281035e0 0x7f1b281160b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:25.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:25.680+0000 7f1b27200700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b28116700 con 0x7f1b281035e0 2024-03-29T08:29:25.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:25.680+0000 7f1b27200700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b2810faf0 con 0x7f1b28102c40 2024-03-29T08:29:25.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:25.680+0000 7f1b25400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1b281035e0 0x7f1b281160b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:25.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:25.680+0000 7f1b25400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1b281035e0 0x7f1b281160b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43788/0 (socket says 172.21.15.37:43788) 2024-03-29T08:29:25.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:25.680+0000 7f1b25400700 1 -- 172.21.15.37:0/1197072117 learned_addr learned my addr 172.21.15.37:0/1197072117 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:29:25.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:25.680+0000 7f1b25e00700 1 --2- 172.21.15.37:0/1197072117 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1b28102c40 0x7f1b28115b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:25.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:25.680+0000 7f1b25400700 1 -- 172.21.15.37:0/1197072117 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1b28102c40 msgr2=0x7f1b28115b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:25.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:25.680+0000 7f1b25400700 1 --2- 172.21.15.37:0/1197072117 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1b28102c40 0x7f1b28115b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:25.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:25.680+0000 7f1b25400700 1 -- 172.21.15.37:0/1197072117 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b18005040 con 0x7f1b281035e0 2024-03-29T08:29:25.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:25.681+0000 7f1b25400700 1 --2- 172.21.15.37:0/1197072117 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1b281035e0 0x7f1b281160b0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f1b14002a50 tx=0x7f1b14004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:25.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:25.681+0000 7f1b1e000700 1 -- 172.21.15.37:0/1197072117 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1b14007cc0 con 0x7f1b281035e0 2024-03-29T08:29:25.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:25.681+0000 7f1b1e000700 1 -- 172.21.15.37:0/1197072117 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f1b1400f040 con 0x7f1b281035e0 2024-03-29T08:29:25.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:25.681+0000 7f1b1e000700 1 -- 172.21.15.37:0/1197072117 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1b1400d920 con 0x7f1b281035e0 2024-03-29T08:29:25.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:25.681+0000 7f1b27200700 1 -- 172.21.15.37:0/1197072117 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b2810fdd0 con 0x7f1b281035e0 2024-03-29T08:29:25.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:25.681+0000 7f1b27200700 1 -- 172.21.15.37:0/1197072117 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b28110310 con 0x7f1b281035e0 2024-03-29T08:29:25.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:25.682+0000 7f1b27200700 1 -- 172.21.15.37:0/1197072117 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b280fe390 con 0x7f1b281035e0 2024-03-29T08:29:25.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:25.683+0000 7f1b1e000700 1 -- 172.21.15.37:0/1197072117 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f1b14009680 con 0x7f1b281035e0 2024-03-29T08:29:25.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:25.684+0000 7f1b1e000700 1 --2- 172.21.15.37:0/1197072117 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f1b0c06cc30 0x7f1b0c06f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:25.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:25.685+0000 7f1b1e000700 1 -- 172.21.15.37:0/1197072117 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f1b1408bd10 con 0x7f1b281035e0 2024-03-29T08:29:25.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:25.685+0000 7f1b25e00700 1 --2- 172.21.15.37:0/1197072117 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f1b0c06cc30 0x7f1b0c06f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:25.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:25.685+0000 7f1b25e00700 1 --2- 172.21.15.37:0/1197072117 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f1b0c06cc30 0x7f1b0c06f0e0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f1b1800b840 tx=0x7f1b18009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:25.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:25.690+0000 7f1b1e000700 1 -- 172.21.15.37:0/1197072117 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f1b14059760 con 0x7f1b281035e0 2024-03-29T08:29:25.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:29:25 smithi148 ceph-mon[106692]: pgmap v555: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:29:25.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:29:25 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:29:25.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:25.919+0000 7f1b27200700 1 -- 172.21.15.37:0/1197072117 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f1b28115840 con 0x7f1b0c06cc30 2024-03-29T08:29:25.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:25.921+0000 7f1b1e000700 1 -- 172.21.15.37:0/1197072117 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f1b28115840 con 0x7f1b0c06cc30 2024-03-29T08:29:25.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:25.923+0000 7f1b27200700 1 -- 172.21.15.37:0/1197072117 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f1b0c06cc30 msgr2=0x7f1b0c06f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:25.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:25.923+0000 7f1b27200700 1 --2- 172.21.15.37:0/1197072117 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f1b0c06cc30 0x7f1b0c06f0e0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f1b1800b840 tx=0x7f1b18009e30 comp rx=0 tx=0).stop 2024-03-29T08:29:25.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:25.923+0000 7f1b27200700 1 -- 172.21.15.37:0/1197072117 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1b281035e0 msgr2=0x7f1b281160b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:25.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:25.923+0000 7f1b27200700 1 --2- 172.21.15.37:0/1197072117 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1b281035e0 0x7f1b281160b0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f1b14002a50 tx=0x7f1b14004600 comp rx=0 tx=0).stop 2024-03-29T08:29:25.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:25.923+0000 7f1b27200700 1 -- 172.21.15.37:0/1197072117 shutdown_connections 2024-03-29T08:29:25.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:25.923+0000 7f1b27200700 1 --2- 172.21.15.37:0/1197072117 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f1b0c06cc30 0x7f1b0c06f0e0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:25.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:25.923+0000 7f1b27200700 1 --2- 172.21.15.37:0/1197072117 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1b28102c40 0x7f1b28115b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:25.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:25.923+0000 7f1b27200700 1 --2- 172.21.15.37:0/1197072117 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1b281035e0 0x7f1b281160b0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:25.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:25.923+0000 7f1b27200700 1 -- 172.21.15.37:0/1197072117 >> 172.21.15.37:0/1197072117 conn(0x7f1b280f87d0 msgr2=0x7f1b280f97e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:29:25.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:25.923+0000 7f1b27200700 1 -- 172.21.15.37:0/1197072117 shutdown_connections 2024-03-29T08:29:25.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:25.923+0000 7f1b27200700 1 -- 172.21.15.37:0/1197072117 wait complete. 2024-03-29T08:29:25.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:29:25 smithi037 ceph-mon[142325]: pgmap v555: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:29:25.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:29:25 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:29:25.938 INFO:teuthology.orchestra.run.smithi037.stdout:true 2024-03-29T08:29:26.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:26.062+0000 7fe216400700 1 -- 172.21.15.37:0/978283695 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe210102c40 msgr2=0x7fe210103010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:26.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:26.062+0000 7fe216400700 1 --2- 172.21.15.37:0/978283695 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe210102c40 0x7fe210103010 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fe204004940 tx=0x7fe204009e30 comp rx=0 tx=0).stop 2024-03-29T08:29:26.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:26.062+0000 7fe216400700 1 -- 172.21.15.37:0/978283695 shutdown_connections 2024-03-29T08:29:26.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:26.062+0000 7fe216400700 1 --2- 172.21.15.37:0/978283695 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe2101035e0 0x7fe2100fcc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:26.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:26.062+0000 7fe216400700 1 --2- 172.21.15.37:0/978283695 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe210102c40 0x7fe210103010 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:26.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:26.062+0000 7fe216400700 1 -- 172.21.15.37:0/978283695 >> 172.21.15.37:0/978283695 conn(0x7fe2100f8830 msgr2=0x7fe2100fac40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:29:26.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:26.062+0000 7fe216400700 1 -- 172.21.15.37:0/978283695 shutdown_connections 2024-03-29T08:29:26.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:26.062+0000 7fe216400700 1 -- 172.21.15.37:0/978283695 wait complete. 2024-03-29T08:29:26.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:26.063+0000 7fe216400700 1 Processor -- start 2024-03-29T08:29:26.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:26.063+0000 7fe216400700 1 -- start start 2024-03-29T08:29:26.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:26.064+0000 7fe216400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe210102c40 0x7fe2101138f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:26.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:26.064+0000 7fe216400700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe2101035e0 0x7fe210113e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:26.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:26.064+0000 7fe216400700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe210114510 con 0x7fe210102c40 2024-03-29T08:29:26.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:26.064+0000 7fe216400700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe21010d870 con 0x7fe2101035e0 2024-03-29T08:29:26.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:26.064+0000 7fe215000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe210102c40 0x7fe2101138f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:26.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:26.064+0000 7fe215000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe210102c40 0x7fe2101138f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43806/0 (socket says 172.21.15.37:43806) 2024-03-29T08:29:26.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:26.064+0000 7fe215000700 1 -- 172.21.15.37:0/2774133669 learned_addr learned my addr 172.21.15.37:0/2774133669 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:29:26.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:26.064+0000 7fe215000700 1 -- 172.21.15.37:0/2774133669 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe2101035e0 msgr2=0x7fe210113e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:26.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:26.064+0000 7fe215000700 1 --2- 172.21.15.37:0/2774133669 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe2101035e0 0x7fe210113e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:26.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:26.064+0000 7fe215000700 1 -- 172.21.15.37:0/2774133669 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe204005040 con 0x7fe210102c40 2024-03-29T08:29:26.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:26.065+0000 7fe215000700 1 --2- 172.21.15.37:0/2774133669 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe210102c40 0x7fe2101138f0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fe204009390 tx=0x7fe204009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:26.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:26.065+0000 7fe20cc00700 1 -- 172.21.15.37:0/2774133669 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe20400e040 con 0x7fe210102c40 2024-03-29T08:29:26.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:26.065+0000 7fe216400700 1 -- 172.21.15.37:0/2774133669 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe21010daf0 con 0x7fe210102c40 2024-03-29T08:29:26.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:26.065+0000 7fe20cc00700 1 -- 172.21.15.37:0/2774133669 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fe204013420 con 0x7fe210102c40 2024-03-29T08:29:26.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:26.065+0000 7fe20cc00700 1 -- 172.21.15.37:0/2774133669 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe20401c630 con 0x7fe210102c40 2024-03-29T08:29:26.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:26.065+0000 7fe216400700 1 -- 172.21.15.37:0/2774133669 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe21010e0b0 con 0x7fe210102c40 2024-03-29T08:29:26.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:26.066+0000 7fe216400700 1 -- 172.21.15.37:0/2774133669 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2100fe390 con 0x7fe210102c40 2024-03-29T08:29:26.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:26.067+0000 7fe20cc00700 1 -- 172.21.15.37:0/2774133669 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fe2040139e0 con 0x7fe210102c40 2024-03-29T08:29:26.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:26.068+0000 7fe20cc00700 1 --2- 172.21.15.37:0/2774133669 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fe1f806cc30 0x7fe1f806f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:26.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:26.068+0000 7fe20cc00700 1 -- 172.21.15.37:0/2774133669 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fe20408d320 con 0x7fe210102c40 2024-03-29T08:29:26.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:26.068+0000 7fe20fe00700 1 --2- 172.21.15.37:0/2774133669 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fe1f806cc30 0x7fe1f806f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:26.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:26.069+0000 7fe20fe00700 1 --2- 172.21.15.37:0/2774133669 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fe1f806cc30 0x7fe1f806f0e0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fe1fc006fd0 tx=0x7fe1fc006db0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:26.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:26.073+0000 7fe20cc00700 1 -- 172.21.15.37:0/2774133669 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fe20405ae20 con 0x7fe210102c40 2024-03-29T08:29:26.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:26.305+0000 7fe216400700 1 -- 172.21.15.37:0/2774133669 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe21004bed0 con 0x7fe1f806cc30 2024-03-29T08:29:26.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:26.306+0000 7fe20cc00700 1 -- 172.21.15.37:0/2774133669 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fe21004bed0 con 0x7fe1f806cc30 2024-03-29T08:29:26.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:26.309+0000 7fe216400700 1 -- 172.21.15.37:0/2774133669 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fe1f806cc30 msgr2=0x7fe1f806f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:26.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:26.309+0000 7fe216400700 1 --2- 172.21.15.37:0/2774133669 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fe1f806cc30 0x7fe1f806f0e0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fe1fc006fd0 tx=0x7fe1fc006db0 comp rx=0 tx=0).stop 2024-03-29T08:29:26.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:26.309+0000 7fe216400700 1 -- 172.21.15.37:0/2774133669 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe210102c40 msgr2=0x7fe2101138f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:26.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:26.309+0000 7fe216400700 1 --2- 172.21.15.37:0/2774133669 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe210102c40 0x7fe2101138f0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fe204009390 tx=0x7fe204009e30 comp rx=0 tx=0).stop 2024-03-29T08:29:26.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:26.309+0000 7fe216400700 1 -- 172.21.15.37:0/2774133669 shutdown_connections 2024-03-29T08:29:26.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:26.309+0000 7fe216400700 1 --2- 172.21.15.37:0/2774133669 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fe1f806cc30 0x7fe1f806f0e0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:26.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:26.309+0000 7fe216400700 1 --2- 172.21.15.37:0/2774133669 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe210102c40 0x7fe2101138f0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:26.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:26.309+0000 7fe216400700 1 --2- 172.21.15.37:0/2774133669 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe2101035e0 0x7fe210113e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:26.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:26.309+0000 7fe216400700 1 -- 172.21.15.37:0/2774133669 >> 172.21.15.37:0/2774133669 conn(0x7fe2100f8830 msgr2=0x7fe2100f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:29:26.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:26.309+0000 7fe216400700 1 -- 172.21.15.37:0/2774133669 shutdown_connections 2024-03-29T08:29:26.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:26.309+0000 7fe216400700 1 -- 172.21.15.37:0/2774133669 wait complete. 2024-03-29T08:29:26.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:26.448+0000 7fa572400700 1 -- 172.21.15.37:0/2625579486 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa56c102e00 msgr2=0x7fa56c1031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:26.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:26.448+0000 7fa572400700 1 --2- 172.21.15.37:0/2625579486 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa56c102e00 0x7fa56c1031d0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fa55c004940 tx=0x7fa55c009e30 comp rx=0 tx=0).stop 2024-03-29T08:29:26.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:26.448+0000 7fa572400700 1 -- 172.21.15.37:0/2625579486 shutdown_connections 2024-03-29T08:29:26.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:26.448+0000 7fa572400700 1 --2- 172.21.15.37:0/2625579486 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa56c1037a0 0x7fa56c0fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:26.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:26.448+0000 7fa572400700 1 --2- 172.21.15.37:0/2625579486 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa56c102e00 0x7fa56c1031d0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:26.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:26.448+0000 7fa572400700 1 -- 172.21.15.37:0/2625579486 >> 172.21.15.37:0/2625579486 conn(0x7fa56c0f89b0 msgr2=0x7fa56c0fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:29:26.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:26.448+0000 7fa572400700 1 -- 172.21.15.37:0/2625579486 shutdown_connections 2024-03-29T08:29:26.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:26.448+0000 7fa572400700 1 -- 172.21.15.37:0/2625579486 wait complete. 2024-03-29T08:29:26.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:26.449+0000 7fa572400700 1 Processor -- start 2024-03-29T08:29:26.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:26.449+0000 7fa572400700 1 -- start start 2024-03-29T08:29:26.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:26.450+0000 7fa572400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa56c102e00 0x7fa56c115cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:26.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:26.450+0000 7fa572400700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa56c1037a0 0x7fa56c116230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:26.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:26.450+0000 7fa572400700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa56c116880 con 0x7fa56c102e00 2024-03-29T08:29:26.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:26.450+0000 7fa572400700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa56c10fc50 con 0x7fa56c1037a0 2024-03-29T08:29:26.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:26.450+0000 7fa56b400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa56c102e00 0x7fa56c115cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:26.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:26.450+0000 7fa56b400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa56c102e00 0x7fa56c115cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43838/0 (socket says 172.21.15.37:43838) 2024-03-29T08:29:26.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:26.450+0000 7fa56b400700 1 -- 172.21.15.37:0/4228559341 learned_addr learned my addr 172.21.15.37:0/4228559341 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:29:26.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:26.450+0000 7fa56aa00700 1 --2- 172.21.15.37:0/4228559341 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa56c1037a0 0x7fa56c116230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:26.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:26.450+0000 7fa56b400700 1 -- 172.21.15.37:0/4228559341 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa56c1037a0 msgr2=0x7fa56c116230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:26.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:26.450+0000 7fa56b400700 1 --2- 172.21.15.37:0/4228559341 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa56c1037a0 0x7fa56c116230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:26.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:26.450+0000 7fa56b400700 1 -- 172.21.15.37:0/4228559341 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa55c005040 con 0x7fa56c102e00 2024-03-29T08:29:26.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:26.450+0000 7fa56aa00700 1 --2- 172.21.15.37:0/4228559341 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa56c1037a0 0x7fa56c116230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:29:26.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:26.451+0000 7fa56b400700 1 --2- 172.21.15.37:0/4228559341 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa56c102e00 0x7fa56c115cf0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fa55c009360 tx=0x7fa55c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:26.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:26.451+0000 7fa563400700 1 -- 172.21.15.37:0/4228559341 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa55c00e040 con 0x7fa56c102e00 2024-03-29T08:29:26.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:26.451+0000 7fa563400700 1 -- 172.21.15.37:0/4228559341 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fa55c013420 con 0x7fa56c102e00 2024-03-29T08:29:26.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:26.451+0000 7fa563400700 1 -- 172.21.15.37:0/4228559341 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa55c01c5f0 con 0x7fa56c102e00 2024-03-29T08:29:26.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:26.451+0000 7fa572400700 1 -- 172.21.15.37:0/4228559341 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa56c10fed0 con 0x7fa56c102e00 2024-03-29T08:29:26.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:26.451+0000 7fa572400700 1 -- 172.21.15.37:0/4228559341 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa56c110490 con 0x7fa56c102e00 2024-03-29T08:29:26.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:26.452+0000 7fa572400700 1 -- 172.21.15.37:0/4228559341 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa56c0fe550 con 0x7fa56c102e00 2024-03-29T08:29:26.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:26.453+0000 7fa563400700 1 -- 172.21.15.37:0/4228559341 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fa55c013590 con 0x7fa56c102e00 2024-03-29T08:29:26.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:26.454+0000 7fa563400700 1 --2- 172.21.15.37:0/4228559341 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fa55006cc30 0x7fa55006f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:26.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:26.454+0000 7fa563400700 1 -- 172.21.15.37:0/4228559341 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fa55c08d240 con 0x7fa56c102e00 2024-03-29T08:29:26.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:26.454+0000 7fa56aa00700 1 --2- 172.21.15.37:0/4228559341 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fa55006cc30 0x7fa55006f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:26.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:26.455+0000 7fa56aa00700 1 --2- 172.21.15.37:0/4228559341 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fa55006cc30 0x7fa55006f0e0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fa558009fd0 tx=0x7fa55800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:26.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:26.459+0000 7fa563400700 1 -- 172.21.15.37:0/4228559341 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fa55c05ad10 con 0x7fa56c102e00 2024-03-29T08:29:26.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:26.681+0000 7fa572400700 1 -- 172.21.15.37:0/4228559341 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fa56c04bf40 con 0x7fa55006cc30 2024-03-29T08:29:26.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:26.691+0000 7fa563400700 1 -- 172.21.15.37:0/4228559341 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7fa56c04bf40 con 0x7fa55006cc30 2024-03-29T08:29:26.691 INFO:teuthology.orchestra.run.smithi037.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-29T08:29:26.691 INFO:teuthology.orchestra.run.smithi037.stdout:alertmanager.smithi037 smithi037 *:9093,9094 running (19m) 11m ago 32m 14.9M - 0.25.0 c8568f914cd2 2f9f9746ed0f 2024-03-29T08:29:26.691 INFO:teuthology.orchestra.run.smithi037.stdout:crash.smithi037 smithi037 running (16m) 11m ago 32m 6647k - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 178d416a3186 2024-03-29T08:29:26.691 INFO:teuthology.orchestra.run.smithi037.stdout:crash.smithi148 smithi148 running (16m) 11m ago 31m 6656k - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 23666989d58f 2024-03-29T08:29:26.691 INFO:teuthology.orchestra.run.smithi037.stdout:grafana.smithi037 smithi037 *:3000 running (18m) 11m ago 31m 71.9M - 9.4.7 954c08fa6188 6b976fe49952 2024-03-29T08:29:26.691 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi037.iifsmh smithi037 running (11m) 11m ago 25m 10.8M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 2d97f562cc5b 2024-03-29T08:29:26.691 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi037.mmwgxe smithi037 running (12m) 11m ago 25m 10.9M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff ec37037f46e8 2024-03-29T08:29:26.691 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi148.qukpjh smithi148 running (25m) 11m ago 25m 15.3M - 17.2.7-789-g4213e17c cf22eb8683be 8e469782a273 2024-03-29T08:29:26.692 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi148.sqteih smithi148 running (11m) 11m ago 25m 10.7M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 44c15c77037a 2024-03-29T08:29:26.692 INFO:teuthology.orchestra.run.smithi037.stdout:mgr.smithi037.wjikii smithi037 *:8443,9283,8765 running (21m) 11m ago 33m 506M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff c5a1417b9417 2024-03-29T08:29:26.692 INFO:teuthology.orchestra.run.smithi037.stdout:mgr.smithi148.onhgvm smithi148 *:8443,9283,8765 running (20m) 11m ago 31m 450M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff fcfdbce062bd 2024-03-29T08:29:26.692 INFO:teuthology.orchestra.run.smithi037.stdout:mon.smithi037 smithi037 running (17m) 11m ago 34m 54.8M 2048M 18.2.2-1015-gd0e392d9 1cc2f45c17ff eea2060783af 2024-03-29T08:29:26.692 INFO:teuthology.orchestra.run.smithi037.stdout:mon.smithi148 smithi148 running (16m) 11m ago 30m 43.7M 2048M 18.2.2-1015-gd0e392d9 1cc2f45c17ff ad4b2c5748be 2024-03-29T08:29:26.692 INFO:teuthology.orchestra.run.smithi037.stdout:node-exporter.smithi037 smithi037 *:9100 running (20m) 11m ago 32m 7625k - 1.5.0 0da6a335fe13 26b8ae9cb878 2024-03-29T08:29:26.692 INFO:teuthology.orchestra.run.smithi037.stdout:node-exporter.smithi148 smithi148 *:9100 running (19m) 11m ago 31m 8166k - 1.5.0 0da6a335fe13 98414e4e64ed 2024-03-29T08:29:26.692 INFO:teuthology.orchestra.run.smithi037.stdout:osd.0 smithi037 running (15m) 11m ago 30m 104M 1619M 18.2.2-1015-gd0e392d9 1cc2f45c17ff c0c584f6245c 2024-03-29T08:29:26.692 INFO:teuthology.orchestra.run.smithi037.stdout:osd.1 smithi037 running (15m) 11m ago 29m 100M 1619M 18.2.2-1015-gd0e392d9 1cc2f45c17ff a4a2ff4cc83d 2024-03-29T08:29:26.692 INFO:teuthology.orchestra.run.smithi037.stdout:osd.2 smithi037 running (14m) 11m ago 29m 82.3M 1619M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 8b1bc71aa79e 2024-03-29T08:29:26.692 INFO:teuthology.orchestra.run.smithi037.stdout:osd.3 smithi148 running (13m) 11m ago 28m 105M 2642M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 1c656ab8ff86 2024-03-29T08:29:26.692 INFO:teuthology.orchestra.run.smithi037.stdout:osd.4 smithi148 running (13m) 11m ago 28m 97.3M 2642M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 966a9de11d22 2024-03-29T08:29:26.692 INFO:teuthology.orchestra.run.smithi037.stdout:osd.5 smithi148 running (12m) 11m ago 27m 85.8M 2642M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 474c4f9212ce 2024-03-29T08:29:26.692 INFO:teuthology.orchestra.run.smithi037.stdout:prometheus.smithi037 smithi037 *:9095 running (19m) 11m ago 31m 54.4M - 2.43.0 a07b618ecd1d 30c0223ec589 2024-03-29T08:29:26.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:26.693+0000 7fa572400700 1 -- 172.21.15.37:0/4228559341 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fa55006cc30 msgr2=0x7fa55006f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:26.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:26.693+0000 7fa572400700 1 --2- 172.21.15.37:0/4228559341 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fa55006cc30 0x7fa55006f0e0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fa558009fd0 tx=0x7fa55800b040 comp rx=0 tx=0).stop 2024-03-29T08:29:26.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:26.693+0000 7fa572400700 1 -- 172.21.15.37:0/4228559341 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa56c102e00 msgr2=0x7fa56c115cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:26.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:26.693+0000 7fa572400700 1 --2- 172.21.15.37:0/4228559341 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa56c102e00 0x7fa56c115cf0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fa55c009360 tx=0x7fa55c009e30 comp rx=0 tx=0).stop 2024-03-29T08:29:26.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:26.694+0000 7fa572400700 1 -- 172.21.15.37:0/4228559341 shutdown_connections 2024-03-29T08:29:26.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:26.694+0000 7fa572400700 1 --2- 172.21.15.37:0/4228559341 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fa55006cc30 0x7fa55006f0e0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:26.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:26.694+0000 7fa572400700 1 --2- 172.21.15.37:0/4228559341 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa56c102e00 0x7fa56c115cf0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:26.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:26.694+0000 7fa572400700 1 --2- 172.21.15.37:0/4228559341 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa56c1037a0 0x7fa56c116230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:26.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:26.694+0000 7fa572400700 1 -- 172.21.15.37:0/4228559341 >> 172.21.15.37:0/4228559341 conn(0x7fa56c0f89b0 msgr2=0x7fa56c0706d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:29:26.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:26.694+0000 7fa572400700 1 -- 172.21.15.37:0/4228559341 shutdown_connections 2024-03-29T08:29:26.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:26.694+0000 7fa572400700 1 -- 172.21.15.37:0/4228559341 wait complete. 2024-03-29T08:29:26.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:26.831+0000 7f473f400700 1 -- 172.21.15.37:0/360652713 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4738102e00 msgr2=0x7f47381031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:26.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:26.831+0000 7f473f400700 1 --2- 172.21.15.37:0/360652713 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4738102e00 0x7f47381031d0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f472c004940 tx=0x7f472c009e30 comp rx=0 tx=0).stop 2024-03-29T08:29:26.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:26.831+0000 7f473f400700 1 -- 172.21.15.37:0/360652713 shutdown_connections 2024-03-29T08:29:26.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:26.831+0000 7f473f400700 1 --2- 172.21.15.37:0/360652713 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f47381037a0 0x7f47380fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:26.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:26.831+0000 7f473f400700 1 --2- 172.21.15.37:0/360652713 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4738102e00 0x7f47381031d0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:26.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:26.831+0000 7f473f400700 1 -- 172.21.15.37:0/360652713 >> 172.21.15.37:0/360652713 conn(0x7f47380f89b0 msgr2=0x7f47380fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:29:26.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:26.831+0000 7f473f400700 1 -- 172.21.15.37:0/360652713 shutdown_connections 2024-03-29T08:29:26.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:26.831+0000 7f473f400700 1 -- 172.21.15.37:0/360652713 wait complete. 2024-03-29T08:29:26.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:26.832+0000 7f473f400700 1 Processor -- start 2024-03-29T08:29:26.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:26.832+0000 7f473f400700 1 -- start start 2024-03-29T08:29:26.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:26.833+0000 7f473f400700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4738102e00 0x7f47380702c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:26.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:26.833+0000 7f473f400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f47381037a0 0x7f4738070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:26.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:26.833+0000 7f473f400700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4738070d90 con 0x7f47381037a0 2024-03-29T08:29:26.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:26.833+0000 7f473f400700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4738070ed0 con 0x7f4738102e00 2024-03-29T08:29:26.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:26.833+0000 7f4737e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f47381037a0 0x7f4738070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:26.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:26.833+0000 7f4737e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f47381037a0 0x7f4738070800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43856/0 (socket says 172.21.15.37:43856) 2024-03-29T08:29:26.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:26.833+0000 7f4737e00700 1 -- 172.21.15.37:0/3974615634 learned_addr learned my addr 172.21.15.37:0/3974615634 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:29:26.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:26.833+0000 7f473ca00700 1 --2- 172.21.15.37:0/3974615634 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4738102e00 0x7f47380702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:26.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:26.833+0000 7f4737e00700 1 -- 172.21.15.37:0/3974615634 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4738102e00 msgr2=0x7f47380702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:26.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:26.833+0000 7f4737e00700 1 --2- 172.21.15.37:0/3974615634 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4738102e00 0x7f47380702c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:26.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:26.833+0000 7f4737e00700 1 -- 172.21.15.37:0/3974615634 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f472c005040 con 0x7f47381037a0 2024-03-29T08:29:26.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:26.834+0000 7f4737e00700 1 --2- 172.21.15.37:0/3974615634 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f47381037a0 0x7f4738070800 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f4724002a50 tx=0x7f47240046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:26.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:26.834+0000 7f4734c00700 1 -- 172.21.15.37:0/3974615634 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4724007cc0 con 0x7f47381037a0 2024-03-29T08:29:26.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:26.834+0000 7f4734c00700 1 -- 172.21.15.37:0/3974615634 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f472400f040 con 0x7f47381037a0 2024-03-29T08:29:26.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:26.834+0000 7f4734c00700 1 -- 172.21.15.37:0/3974615634 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f472400da30 con 0x7f47381037a0 2024-03-29T08:29:26.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:26.834+0000 7f473f400700 1 -- 172.21.15.37:0/3974615634 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4738119d40 con 0x7f47381037a0 2024-03-29T08:29:26.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:26.834+0000 7f473f400700 1 -- 172.21.15.37:0/3974615634 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f473811a160 con 0x7f47381037a0 2024-03-29T08:29:26.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:26.835+0000 7f473f400700 1 -- 172.21.15.37:0/3974615634 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47380fe550 con 0x7f47381037a0 2024-03-29T08:29:26.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:26.836+0000 7f4734c00700 1 -- 172.21.15.37:0/3974615634 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f472400db90 con 0x7f47381037a0 2024-03-29T08:29:26.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:26.837+0000 7f4734c00700 1 --2- 172.21.15.37:0/3974615634 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f472006cc30 0x7f472006f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:26.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:26.837+0000 7f4734c00700 1 -- 172.21.15.37:0/3974615634 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f472408bcf0 con 0x7f47381037a0 2024-03-29T08:29:26.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:26.838+0000 7f473ca00700 1 --2- 172.21.15.37:0/3974615634 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f472006cc30 0x7f472006f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:26.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:26.838+0000 7f473ca00700 1 --2- 172.21.15.37:0/3974615634 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f472006cc30 0x7f472006f0e0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f472c009fd0 tx=0x7f472c009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:26.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:26.841+0000 7f4734c00700 1 -- 172.21.15.37:0/3974615634 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f4724055c30 con 0x7f47381037a0 2024-03-29T08:29:27.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:27.142+0000 7f473f400700 1 -- 172.21.15.37:0/3974615634 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f473804bed0 con 0x7f47381037a0 2024-03-29T08:29:27.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:27.142+0000 7f4734c00700 1 -- 172.21.15.37:0/3974615634 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+847 (secure 0 0 0) 0x7f4724059250 con 0x7f47381037a0 2024-03-29T08:29:27.143 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-29T08:29:27.143 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": { 2024-03-29T08:29:27.143 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 2 2024-03-29T08:29:27.143 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:29:27.143 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": { 2024-03-29T08:29:27.143 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 2 2024-03-29T08:29:27.143 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:29:27.143 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": { 2024-03-29T08:29:27.143 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 6 2024-03-29T08:29:27.143 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:29:27.144 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": { 2024-03-29T08:29:27.144 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 17.2.7-789-g4213e17c (4213e17c4de67016ed14f55d22503f4923b3a901) quincy (stable)": 1, 2024-03-29T08:29:27.144 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 3 2024-03-29T08:29:27.144 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:29:27.144 INFO:teuthology.orchestra.run.smithi037.stdout: "overall": { 2024-03-29T08:29:27.144 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 17.2.7-789-g4213e17c (4213e17c4de67016ed14f55d22503f4923b3a901) quincy (stable)": 1, 2024-03-29T08:29:27.144 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 13 2024-03-29T08:29:27.144 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-29T08:29:27.144 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-29T08:29:27.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:27.144+0000 7f473f400700 1 -- 172.21.15.37:0/3974615634 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f472006cc30 msgr2=0x7f472006f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:27.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:27.144+0000 7f473f400700 1 --2- 172.21.15.37:0/3974615634 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f472006cc30 0x7f472006f0e0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f472c009fd0 tx=0x7f472c009e30 comp rx=0 tx=0).stop 2024-03-29T08:29:27.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:27.144+0000 7f473f400700 1 -- 172.21.15.37:0/3974615634 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f47381037a0 msgr2=0x7f4738070800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:27.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:27.144+0000 7f473f400700 1 --2- 172.21.15.37:0/3974615634 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f47381037a0 0x7f4738070800 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f4724002a50 tx=0x7f47240046b0 comp rx=0 tx=0).stop 2024-03-29T08:29:27.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:27.145+0000 7f473f400700 1 -- 172.21.15.37:0/3974615634 shutdown_connections 2024-03-29T08:29:27.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:27.145+0000 7f473f400700 1 --2- 172.21.15.37:0/3974615634 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f472006cc30 0x7f472006f0e0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:27.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:27.145+0000 7f473f400700 1 --2- 172.21.15.37:0/3974615634 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4738102e00 0x7f47380702c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:27.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:27.145+0000 7f473f400700 1 --2- 172.21.15.37:0/3974615634 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f47381037a0 0x7f4738070800 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:27.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:27.145+0000 7f473f400700 1 -- 172.21.15.37:0/3974615634 >> 172.21.15.37:0/3974615634 conn(0x7f47380f89b0 msgr2=0x7f47380f99f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:29:27.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:27.145+0000 7f473f400700 1 -- 172.21.15.37:0/3974615634 shutdown_connections 2024-03-29T08:29:27.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:27.145+0000 7f473f400700 1 -- 172.21.15.37:0/3974615634 wait complete. 2024-03-29T08:29:27.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:27.281+0000 7ff840600700 1 -- 172.21.15.37:0/3314078643 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff8380708a0 msgr2=0x7ff838070d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:27.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:27.281+0000 7ff840600700 1 --2- 172.21.15.37:0/3314078643 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff8380708a0 0x7ff838070d10 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7ff824004970 tx=0x7ff824009e30 comp rx=0 tx=0).stop 2024-03-29T08:29:27.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:27.281+0000 7ff840600700 1 -- 172.21.15.37:0/3314078643 shutdown_connections 2024-03-29T08:29:27.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:27.281+0000 7ff840600700 1 --2- 172.21.15.37:0/3314078643 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff8380708a0 0x7ff838070d10 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:27.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:27.281+0000 7ff840600700 1 --2- 172.21.15.37:0/3314078643 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff838071f90 0x7ff838070360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:27.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:27.281+0000 7ff840600700 1 -- 172.21.15.37:0/3314078643 >> 172.21.15.37:0/3314078643 conn(0x7ff8380f89b0 msgr2=0x7ff8380fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:29:27.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:27.282+0000 7ff840600700 1 -- 172.21.15.37:0/3314078643 shutdown_connections 2024-03-29T08:29:27.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:27.282+0000 7ff840600700 1 -- 172.21.15.37:0/3314078643 wait complete. 2024-03-29T08:29:27.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:27.282+0000 7ff840600700 1 Processor -- start 2024-03-29T08:29:27.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:27.283+0000 7ff840600700 1 -- start start 2024-03-29T08:29:27.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:27.283+0000 7ff840600700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff8380708a0 0x7ff83811a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:27.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:27.283+0000 7ff840600700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff838071f90 0x7ff83811a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:27.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:27.283+0000 7ff840600700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff83811ab90 con 0x7ff8380708a0 2024-03-29T08:29:27.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:27.283+0000 7ff840600700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff838114010 con 0x7ff838071f90 2024-03-29T08:29:27.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:27.283+0000 7ff83dc00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff8380708a0 0x7ff83811a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:27.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:27.283+0000 7ff83dc00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff8380708a0 0x7ff83811a000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43870/0 (socket says 172.21.15.37:43870) 2024-03-29T08:29:27.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:27.283+0000 7ff83dc00700 1 -- 172.21.15.37:0/1576270918 learned_addr learned my addr 172.21.15.37:0/1576270918 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:29:27.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:27.283+0000 7ff83d200700 1 --2- 172.21.15.37:0/1576270918 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff838071f90 0x7ff83811a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:27.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:27.284+0000 7ff83dc00700 1 -- 172.21.15.37:0/1576270918 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff838071f90 msgr2=0x7ff83811a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:27.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:27.284+0000 7ff83dc00700 1 --2- 172.21.15.37:0/1576270918 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff838071f90 0x7ff83811a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:27.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:27.284+0000 7ff83dc00700 1 -- 172.21.15.37:0/1576270918 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff824005040 con 0x7ff8380708a0 2024-03-29T08:29:27.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:27.284+0000 7ff83dc00700 1 --2- 172.21.15.37:0/1576270918 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff8380708a0 0x7ff83811a000 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7ff82c002a50 tx=0x7ff82c0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:27.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:27.284+0000 7ff835600700 1 -- 172.21.15.37:0/1576270918 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff82c007cc0 con 0x7ff8380708a0 2024-03-29T08:29:27.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:27.284+0000 7ff835600700 1 -- 172.21.15.37:0/1576270918 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ff82c00f040 con 0x7ff8380708a0 2024-03-29T08:29:27.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:27.284+0000 7ff840600700 1 -- 172.21.15.37:0/1576270918 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8381142f0 con 0x7ff8380708a0 2024-03-29T08:29:27.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:27.284+0000 7ff835600700 1 -- 172.21.15.37:0/1576270918 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff82c009540 con 0x7ff8380708a0 2024-03-29T08:29:27.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:27.284+0000 7ff840600700 1 -- 172.21.15.37:0/1576270918 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff838114830 con 0x7ff8380708a0 2024-03-29T08:29:27.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:27.285+0000 7ff840600700 1 -- 172.21.15.37:0/1576270918 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff838071aa0 con 0x7ff8380708a0 2024-03-29T08:29:27.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:27.286+0000 7ff835600700 1 -- 172.21.15.37:0/1576270918 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7ff82c0096a0 con 0x7ff8380708a0 2024-03-29T08:29:27.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:27.287+0000 7ff835600700 1 --2- 172.21.15.37:0/1576270918 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7ff82006cc30 0x7ff82006f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:27.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:27.287+0000 7ff835600700 1 -- 172.21.15.37:0/1576270918 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7ff82c08bee0 con 0x7ff8380708a0 2024-03-29T08:29:27.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:27.288+0000 7ff83d200700 1 --2- 172.21.15.37:0/1576270918 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7ff82006cc30 0x7ff82006f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:27.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:27.288+0000 7ff83d200700 1 --2- 172.21.15.37:0/1576270918 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7ff82006cc30 0x7ff82006f0e0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7ff83810cf40 tx=0x7ff824009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:27.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:27.292+0000 7ff835600700 1 -- 172.21.15.37:0/1576270918 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7ff82c055e20 con 0x7ff8380708a0 2024-03-29T08:29:27.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:27.552+0000 7ff840600700 1 -- 172.21.15.37:0/1576270918 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7ff83804bed0 con 0x7ff8380708a0 2024-03-29T08:29:27.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:27.552+0000 7ff835600700 1 -- 172.21.15.37:0/1576270918 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 28 v28) v1 ==== 76+0+1841 (secure 0 0 0) 0x7ff82c059440 con 0x7ff8380708a0 2024-03-29T08:29:27.553 INFO:teuthology.orchestra.run.smithi037.stdout:e28 2024-03-29T08:29:27.553 INFO:teuthology.orchestra.run.smithi037.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-29T08:29:27.553 INFO:teuthology.orchestra.run.smithi037.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T08:29:27.553 INFO:teuthology.orchestra.run.smithi037.stdout:legacy client fscid: 1 2024-03-29T08:29:27.553 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:29:27.553 INFO:teuthology.orchestra.run.smithi037.stdout:Filesystem 'cephfs' (1) 2024-03-29T08:29:27.553 INFO:teuthology.orchestra.run.smithi037.stdout:fs_name cephfs 2024-03-29T08:29:27.553 INFO:teuthology.orchestra.run.smithi037.stdout:epoch 21 2024-03-29T08:29:27.553 INFO:teuthology.orchestra.run.smithi037.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-03-29T08:29:27.553 INFO:teuthology.orchestra.run.smithi037.stdout:created 2024-03-29T08:03:29.241269+0000 2024-03-29T08:29:27.553 INFO:teuthology.orchestra.run.smithi037.stdout:modified 2024-03-29T08:17:16.237238+0000 2024-03-29T08:29:27.553 INFO:teuthology.orchestra.run.smithi037.stdout:tableserver 0 2024-03-29T08:29:27.553 INFO:teuthology.orchestra.run.smithi037.stdout:root 0 2024-03-29T08:29:27.553 INFO:teuthology.orchestra.run.smithi037.stdout:session_timeout 60 2024-03-29T08:29:27.553 INFO:teuthology.orchestra.run.smithi037.stdout:session_autoclose 300 2024-03-29T08:29:27.554 INFO:teuthology.orchestra.run.smithi037.stdout:max_file_size 1099511627776 2024-03-29T08:29:27.554 INFO:teuthology.orchestra.run.smithi037.stdout:max_xattr_size 65536 2024-03-29T08:29:27.554 INFO:teuthology.orchestra.run.smithi037.stdout:required_client_features {} 2024-03-29T08:29:27.554 INFO:teuthology.orchestra.run.smithi037.stdout:last_failure 0 2024-03-29T08:29:27.554 INFO:teuthology.orchestra.run.smithi037.stdout:last_failure_osd_epoch 82 2024-03-29T08:29:27.554 INFO:teuthology.orchestra.run.smithi037.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T08:29:27.554 INFO:teuthology.orchestra.run.smithi037.stdout:max_mds 2 2024-03-29T08:29:27.554 INFO:teuthology.orchestra.run.smithi037.stdout:in 0,1 2024-03-29T08:29:27.554 INFO:teuthology.orchestra.run.smithi037.stdout:up {} 2024-03-29T08:29:27.554 INFO:teuthology.orchestra.run.smithi037.stdout:failed 0,1 2024-03-29T08:29:27.554 INFO:teuthology.orchestra.run.smithi037.stdout:damaged 2024-03-29T08:29:27.554 INFO:teuthology.orchestra.run.smithi037.stdout:stopped 2024-03-29T08:29:27.554 INFO:teuthology.orchestra.run.smithi037.stdout:data_pools [3] 2024-03-29T08:29:27.554 INFO:teuthology.orchestra.run.smithi037.stdout:metadata_pool 2 2024-03-29T08:29:27.555 INFO:teuthology.orchestra.run.smithi037.stdout:inline_data disabled 2024-03-29T08:29:27.555 INFO:teuthology.orchestra.run.smithi037.stdout:balancer 2024-03-29T08:29:27.555 INFO:teuthology.orchestra.run.smithi037.stdout:bal_rank_mask -1 2024-03-29T08:29:27.555 INFO:teuthology.orchestra.run.smithi037.stdout:standby_count_wanted 1 2024-03-29T08:29:27.555 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:29:27.555 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:29:27.555 INFO:teuthology.orchestra.run.smithi037.stdout:Standby daemons: 2024-03-29T08:29:27.555 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:29:27.555 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi148.qukpjh{-1:24319} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.148:6826/2641327912,v1:172.21.15.148:6827/2641327912] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:29:27.555 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi037.mmwgxe{-1:34440} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.37:6828/1248142005,v1:172.21.15.37:6829/1248142005] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:29:27.555 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi037.iifsmh{-1:34444} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.37:6826/2060882959,v1:172.21.15.37:6827/2060882959] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:29:27.555 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi148.sqteih{-1:44395} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.148:6824/4078652242,v1:172.21.15.148:6825/4078652242] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:29:27.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:27.555+0000 7ff840600700 1 -- 172.21.15.37:0/1576270918 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7ff82006cc30 msgr2=0x7ff82006f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:27.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:27.555+0000 7ff840600700 1 --2- 172.21.15.37:0/1576270918 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7ff82006cc30 0x7ff82006f0e0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7ff83810cf40 tx=0x7ff824009e30 comp rx=0 tx=0).stop 2024-03-29T08:29:27.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:27.555+0000 7ff840600700 1 -- 172.21.15.37:0/1576270918 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff8380708a0 msgr2=0x7ff83811a000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:27.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:27.555+0000 7ff840600700 1 --2- 172.21.15.37:0/1576270918 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff8380708a0 0x7ff83811a000 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7ff82c002a50 tx=0x7ff82c0046b0 comp rx=0 tx=0).stop 2024-03-29T08:29:27.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:27.555+0000 7ff840600700 1 -- 172.21.15.37:0/1576270918 shutdown_connections 2024-03-29T08:29:27.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:27.555+0000 7ff840600700 1 --2- 172.21.15.37:0/1576270918 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7ff82006cc30 0x7ff82006f0e0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:27.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:27.555+0000 7ff840600700 1 --2- 172.21.15.37:0/1576270918 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff8380708a0 0x7ff83811a000 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:27.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:27.555+0000 7ff840600700 1 --2- 172.21.15.37:0/1576270918 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff838071f90 0x7ff83811a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:27.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:27.555+0000 7ff840600700 1 -- 172.21.15.37:0/1576270918 >> 172.21.15.37:0/1576270918 conn(0x7ff8380f89b0 msgr2=0x7ff8380f9b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:29:27.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:27.555+0000 7ff840600700 1 -- 172.21.15.37:0/1576270918 shutdown_connections 2024-03-29T08:29:27.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:27.555+0000 7ff840600700 1 -- 172.21.15.37:0/1576270918 wait complete. 2024-03-29T08:29:27.557 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 28 2024-03-29T08:29:27.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:27.693+0000 7fac10200700 1 -- 172.21.15.37:0/3085642981 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fac081037a0 msgr2=0x7fac080fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:27.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:27.693+0000 7fac10200700 1 --2- 172.21.15.37:0/3085642981 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fac081037a0 0x7fac080fce10 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fabf4004940 tx=0x7fabf4009e30 comp rx=0 tx=0).stop 2024-03-29T08:29:27.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:27.693+0000 7fac10200700 1 -- 172.21.15.37:0/3085642981 shutdown_connections 2024-03-29T08:29:27.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:27.693+0000 7fac10200700 1 --2- 172.21.15.37:0/3085642981 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fac081037a0 0x7fac080fce10 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:27.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:27.693+0000 7fac10200700 1 --2- 172.21.15.37:0/3085642981 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fac08102e00 0x7fac081031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:27.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:27.693+0000 7fac10200700 1 -- 172.21.15.37:0/3085642981 >> 172.21.15.37:0/3085642981 conn(0x7fac080f89b0 msgr2=0x7fac080fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:29:27.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:27.693+0000 7fac10200700 1 -- 172.21.15.37:0/3085642981 shutdown_connections 2024-03-29T08:29:27.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:27.693+0000 7fac10200700 1 -- 172.21.15.37:0/3085642981 wait complete. 2024-03-29T08:29:27.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:27.694+0000 7fac10200700 1 Processor -- start 2024-03-29T08:29:27.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:27.694+0000 7fac10200700 1 -- start start 2024-03-29T08:29:27.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:27.694+0000 7fac10200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fac08102e00 0x7fac08115c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:27.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:27.694+0000 7fac10200700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fac081037a0 0x7fac081161c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:27.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:27.694+0000 7fac10200700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac08116810 con 0x7fac08102e00 2024-03-29T08:29:27.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:27.694+0000 7fac10200700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac0810fbe0 con 0x7fac081037a0 2024-03-29T08:29:27.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:27.694+0000 7fac0d800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fac08102e00 0x7fac08115c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:27.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:27.695+0000 7fac0d800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fac08102e00 0x7fac08115c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43886/0 (socket says 172.21.15.37:43886) 2024-03-29T08:29:27.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:27.695+0000 7fac0d800700 1 -- 172.21.15.37:0/1128909074 learned_addr learned my addr 172.21.15.37:0/1128909074 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:29:27.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:27.695+0000 7fac0ce00700 1 --2- 172.21.15.37:0/1128909074 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fac081037a0 0x7fac081161c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:27.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:27.695+0000 7fac0d800700 1 -- 172.21.15.37:0/1128909074 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fac081037a0 msgr2=0x7fac081161c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:27.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:27.695+0000 7fac0d800700 1 --2- 172.21.15.37:0/1128909074 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fac081037a0 0x7fac081161c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:27.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:27.695+0000 7fac0d800700 1 -- 172.21.15.37:0/1128909074 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabf4005040 con 0x7fac08102e00 2024-03-29T08:29:27.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:27.695+0000 7fac0d800700 1 --2- 172.21.15.37:0/1128909074 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fac08102e00 0x7fac08115c80 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fabfc00eb40 tx=0x7fabfc00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:27.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:27.695+0000 7fac05600700 1 -- 172.21.15.37:0/1128909074 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fabfc007cc0 con 0x7fac08102e00 2024-03-29T08:29:27.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:27.695+0000 7fac05600700 1 -- 172.21.15.37:0/1128909074 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fabfc00f040 con 0x7fac08102e00 2024-03-29T08:29:27.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:27.695+0000 7fac10200700 1 -- 172.21.15.37:0/1128909074 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac0810fe60 con 0x7fac08102e00 2024-03-29T08:29:27.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:27.696+0000 7fac05600700 1 -- 172.21.15.37:0/1128909074 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fabfc00c990 con 0x7fac08102e00 2024-03-29T08:29:27.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:27.696+0000 7fac10200700 1 -- 172.21.15.37:0/1128909074 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac08110420 con 0x7fac08102e00 2024-03-29T08:29:27.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:27.697+0000 7fac10200700 1 -- 172.21.15.37:0/1128909074 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac080fe550 con 0x7fac08102e00 2024-03-29T08:29:27.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:27.698+0000 7fac05600700 1 -- 172.21.15.37:0/1128909074 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fabfc00caf0 con 0x7fac08102e00 2024-03-29T08:29:27.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:27.699+0000 7fac05600700 1 --2- 172.21.15.37:0/1128909074 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fabf006cc30 0x7fabf006f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:27.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:27.699+0000 7fac05600700 1 -- 172.21.15.37:0/1128909074 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fabfc08bc00 con 0x7fac08102e00 2024-03-29T08:29:27.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:27.699+0000 7fac0ce00700 1 --2- 172.21.15.37:0/1128909074 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fabf006cc30 0x7fabf006f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:27.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:27.700+0000 7fac0ce00700 1 --2- 172.21.15.37:0/1128909074 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fabf006cc30 0x7fabf006f0e0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fac08111520 tx=0x7fabf4009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:27.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:27.704+0000 7fac05600700 1 -- 172.21.15.37:0/1128909074 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fabfc059700 con 0x7fac08102e00 2024-03-29T08:29:27.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:29:27 smithi148 ceph-mon[106692]: pgmap v556: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:29:27.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:29:27 smithi148 ceph-mon[106692]: from='client.35036 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:29:27.807 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:29:27 smithi148 ceph-mon[106692]: from='client.35040 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:29:27.807 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:29:27 smithi148 ceph-mon[106692]: from='client.35044 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:29:27.807 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:29:27 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/3974615634' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:29:27.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:29:27 smithi037 ceph-mon[142325]: pgmap v556: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:29:27.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:29:27 smithi037 ceph-mon[142325]: from='client.35036 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:29:27.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:29:27 smithi037 ceph-mon[142325]: from='client.35040 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:29:27.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:29:27 smithi037 ceph-mon[142325]: from='client.35044 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:29:27.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:29:27 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/3974615634' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:29:27.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:27.937+0000 7fac10200700 1 -- 172.21.15.37:0/1128909074 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fac0804bf40 con 0x7fabf006cc30 2024-03-29T08:29:27.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:27.938+0000 7fac05600700 1 -- 172.21.15.37:0/1128909074 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fac0804bf40 con 0x7fabf006cc30 2024-03-29T08:29:27.938 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-29T08:29:27.938 INFO:teuthology.orchestra.run.smithi037.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59", 2024-03-29T08:29:27.938 INFO:teuthology.orchestra.run.smithi037.stdout: "in_progress": true, 2024-03-29T08:29:27.938 INFO:teuthology.orchestra.run.smithi037.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-29T08:29:27.939 INFO:teuthology.orchestra.run.smithi037.stdout: "services_complete": [ 2024-03-29T08:29:27.939 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr", 2024-03-29T08:29:27.939 INFO:teuthology.orchestra.run.smithi037.stdout: "mon", 2024-03-29T08:29:27.939 INFO:teuthology.orchestra.run.smithi037.stdout: "crash", 2024-03-29T08:29:27.939 INFO:teuthology.orchestra.run.smithi037.stdout: "osd" 2024-03-29T08:29:27.939 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-29T08:29:27.939 INFO:teuthology.orchestra.run.smithi037.stdout: "progress": "15/21 daemons upgraded", 2024-03-29T08:29:27.939 INFO:teuthology.orchestra.run.smithi037.stdout: "message": "Currently upgrading mds daemons", 2024-03-29T08:29:27.939 INFO:teuthology.orchestra.run.smithi037.stdout: "is_paused": false 2024-03-29T08:29:27.939 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-29T08:29:27.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:27.941+0000 7fac10200700 1 -- 172.21.15.37:0/1128909074 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fabf006cc30 msgr2=0x7fabf006f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:27.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:27.941+0000 7fac10200700 1 --2- 172.21.15.37:0/1128909074 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fabf006cc30 0x7fabf006f0e0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fac08111520 tx=0x7fabf4009e30 comp rx=0 tx=0).stop 2024-03-29T08:29:27.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:27.941+0000 7fac10200700 1 -- 172.21.15.37:0/1128909074 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fac08102e00 msgr2=0x7fac08115c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:27.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:27.941+0000 7fac10200700 1 --2- 172.21.15.37:0/1128909074 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fac08102e00 0x7fac08115c80 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fabfc00eb40 tx=0x7fabfc00b040 comp rx=0 tx=0).stop 2024-03-29T08:29:27.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:27.941+0000 7fac10200700 1 -- 172.21.15.37:0/1128909074 shutdown_connections 2024-03-29T08:29:27.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:27.941+0000 7fac10200700 1 --2- 172.21.15.37:0/1128909074 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fabf006cc30 0x7fabf006f0e0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:27.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:27.941+0000 7fac10200700 1 --2- 172.21.15.37:0/1128909074 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fac08102e00 0x7fac08115c80 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:27.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:27.941+0000 7fac10200700 1 --2- 172.21.15.37:0/1128909074 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fac081037a0 0x7fac081161c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:27.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:27.941+0000 7fac10200700 1 -- 172.21.15.37:0/1128909074 >> 172.21.15.37:0/1128909074 conn(0x7fac080f89b0 msgr2=0x7fac080f9a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:29:27.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:27.941+0000 7fac10200700 1 -- 172.21.15.37:0/1128909074 shutdown_connections 2024-03-29T08:29:27.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:27.941+0000 7fac10200700 1 -- 172.21.15.37:0/1128909074 wait complete. 2024-03-29T08:29:28.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:28.078+0000 7f54aa800700 1 -- 172.21.15.37:0/73117605 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f54a40fde90 msgr2=0x7f54a40fe260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:28.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:28.078+0000 7f54aa800700 1 --2- 172.21.15.37:0/73117605 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f54a40fde90 0x7f54a40fe260 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f5494004940 tx=0x7f5494009e30 comp rx=0 tx=0).stop 2024-03-29T08:29:28.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:28.078+0000 7f54aa800700 1 -- 172.21.15.37:0/73117605 shutdown_connections 2024-03-29T08:29:28.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:28.078+0000 7f54aa800700 1 --2- 172.21.15.37:0/73117605 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54a40fe830 0x7f54a4106f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:28.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:28.078+0000 7f54aa800700 1 --2- 172.21.15.37:0/73117605 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f54a40fde90 0x7f54a40fe260 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:28.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:28.078+0000 7f54aa800700 1 -- 172.21.15.37:0/73117605 >> 172.21.15.37:0/73117605 conn(0x7f54a4072bc0 msgr2=0x7f54a4072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:29:28.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:28.078+0000 7f54aa800700 1 -- 172.21.15.37:0/73117605 shutdown_connections 2024-03-29T08:29:28.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:28.078+0000 7f54aa800700 1 -- 172.21.15.37:0/73117605 wait complete. 2024-03-29T08:29:28.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:28.079+0000 7f54aa800700 1 Processor -- start 2024-03-29T08:29:28.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:28.079+0000 7f54aa800700 1 -- start start 2024-03-29T08:29:28.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:28.079+0000 7f54aa800700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54a40fde90 0x7f54a40fbd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:28.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:28.079+0000 7f54aa800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f54a40fe830 0x7f54a40fc290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:28.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:28.079+0000 7f54aa800700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54a40fc820 con 0x7f54a40fe830 2024-03-29T08:29:28.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:28.079+0000 7f54aa800700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54a40fc960 con 0x7f54a40fde90 2024-03-29T08:29:28.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:28.079+0000 7f54a2a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f54a40fe830 0x7f54a40fc290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:28.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:28.080+0000 7f54a2a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f54a40fe830 0x7f54a40fc290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43892/0 (socket says 172.21.15.37:43892) 2024-03-29T08:29:28.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:28.080+0000 7f54a2a00700 1 -- 172.21.15.37:0/4236863966 learned_addr learned my addr 172.21.15.37:0/4236863966 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:29:28.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:28.080+0000 7f54a3400700 1 --2- 172.21.15.37:0/4236863966 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54a40fde90 0x7f54a40fbd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:28.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:28.080+0000 7f54a2a00700 1 -- 172.21.15.37:0/4236863966 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54a40fde90 msgr2=0x7f54a40fbd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:28.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:28.080+0000 7f54a2a00700 1 --2- 172.21.15.37:0/4236863966 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54a40fde90 0x7f54a40fbd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:28.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:28.080+0000 7f54a2a00700 1 -- 172.21.15.37:0/4236863966 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5494005040 con 0x7f54a40fe830 2024-03-29T08:29:28.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:28.080+0000 7f54a3400700 1 --2- 172.21.15.37:0/4236863966 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54a40fde90 0x7f54a40fbd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:29:28.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:28.080+0000 7f54a2a00700 1 --2- 172.21.15.37:0/4236863966 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f54a40fe830 0x7f54a40fc290 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f5490002a50 tx=0x7f54900046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:28.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:28.080+0000 7f549b400700 1 -- 172.21.15.37:0/4236863966 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5490007cc0 con 0x7f54a40fe830 2024-03-29T08:29:28.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:28.080+0000 7f549b400700 1 -- 172.21.15.37:0/4236863966 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f549000f040 con 0x7f54a40fe830 2024-03-29T08:29:28.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:28.080+0000 7f54aa800700 1 -- 172.21.15.37:0/4236863966 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54a4119df0 con 0x7f54a40fe830 2024-03-29T08:29:28.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:28.080+0000 7f549b400700 1 -- 172.21.15.37:0/4236863966 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5490009540 con 0x7f54a40fe830 2024-03-29T08:29:28.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:28.081+0000 7f54aa800700 1 -- 172.21.15.37:0/4236863966 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54a411a1f0 con 0x7f54a40fe830 2024-03-29T08:29:28.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:28.082+0000 7f54aa800700 1 -- 172.21.15.37:0/4236863966 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54a4065e30 con 0x7f54a40fe830 2024-03-29T08:29:28.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:28.083+0000 7f549b400700 1 -- 172.21.15.37:0/4236863966 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f54900096a0 con 0x7f54a40fe830 2024-03-29T08:29:28.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:28.084+0000 7f549b400700 1 --2- 172.21.15.37:0/4236863966 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f548806cc30 0x7f548806f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:28.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:28.084+0000 7f549b400700 1 -- 172.21.15.37:0/4236863966 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f549008be00 con 0x7f54a40fe830 2024-03-29T08:29:28.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:28.084+0000 7f54a3400700 1 --2- 172.21.15.37:0/4236863966 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f548806cc30 0x7f548806f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:28.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:28.085+0000 7f54a3400700 1 --2- 172.21.15.37:0/4236863966 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f548806cc30 0x7f548806f0e0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f549400b3b0 tx=0x7f5494009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:28.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:28.089+0000 7f549b400700 1 -- 172.21.15.37:0/4236863966 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f5490055dc0 con 0x7f54a40fe830 2024-03-29T08:29:28.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:28.388+0000 7f54aa800700 1 -- 172.21.15.37:0/4236863966 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f54a405f5f0 con 0x7f54a40fe830 2024-03-29T08:29:28.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:28.388+0000 7f549b400700 1 -- 172.21.15.37:0/4236863966 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7f54900593e0 con 0x7f54a40fe830 2024-03-29T08:29:28.388 INFO:teuthology.orchestra.run.smithi037.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-03-29T08:29:28.388 INFO:teuthology.orchestra.run.smithi037.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-03-29T08:29:28.389 INFO:teuthology.orchestra.run.smithi037.stdout: fs cephfs is degraded 2024-03-29T08:29:28.389 INFO:teuthology.orchestra.run.smithi037.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-03-29T08:29:28.389 INFO:teuthology.orchestra.run.smithi037.stdout: fs cephfs is offline because no MDS is active for it. 2024-03-29T08:29:28.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:28.390+0000 7f54aa800700 1 -- 172.21.15.37:0/4236863966 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f548806cc30 msgr2=0x7f548806f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:28.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:28.390+0000 7f54aa800700 1 --2- 172.21.15.37:0/4236863966 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f548806cc30 0x7f548806f0e0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f549400b3b0 tx=0x7f5494009e30 comp rx=0 tx=0).stop 2024-03-29T08:29:28.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:28.390+0000 7f54aa800700 1 -- 172.21.15.37:0/4236863966 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f54a40fe830 msgr2=0x7f54a40fc290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:28.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:28.390+0000 7f54aa800700 1 --2- 172.21.15.37:0/4236863966 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f54a40fe830 0x7f54a40fc290 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f5490002a50 tx=0x7f54900046b0 comp rx=0 tx=0).stop 2024-03-29T08:29:28.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:28.391+0000 7f54aa800700 1 -- 172.21.15.37:0/4236863966 shutdown_connections 2024-03-29T08:29:28.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:28.391+0000 7f54aa800700 1 --2- 172.21.15.37:0/4236863966 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f548806cc30 0x7f548806f0e0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:28.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:28.391+0000 7f54aa800700 1 --2- 172.21.15.37:0/4236863966 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54a40fde90 0x7f54a40fbd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:28.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:28.391+0000 7f54aa800700 1 --2- 172.21.15.37:0/4236863966 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f54a40fe830 0x7f54a40fc290 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:28.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:28.391+0000 7f54aa800700 1 -- 172.21.15.37:0/4236863966 >> 172.21.15.37:0/4236863966 conn(0x7f54a4072bc0 msgr2=0x7f54a40ff3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:29:28.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:28.391+0000 7f54aa800700 1 -- 172.21.15.37:0/4236863966 shutdown_connections 2024-03-29T08:29:28.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:28.391+0000 7f54aa800700 1 -- 172.21.15.37:0/4236863966 wait complete. 2024-03-29T08:29:28.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:29:28 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/1576270918' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T08:29:28.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:29:28 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/4236863966' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T08:29:28.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:29:28 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/1576270918' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T08:29:28.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:29:28 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/4236863966' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T08:29:29.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:29:29 smithi148 ceph-mon[106692]: pgmap v557: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:29:29.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:29:29 smithi148 ceph-mon[106692]: from='client.35056 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:29:29.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:29:29 smithi037 ceph-mon[142325]: pgmap v557: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:29:29.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:29:29 smithi037 ceph-mon[142325]: from='client.35056 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:29:31.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:29:31 smithi148 ceph-mon[106692]: pgmap v558: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:29:31.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:29:31 smithi037 ceph-mon[142325]: pgmap v558: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:29:33.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:29:33 smithi148 ceph-mon[106692]: pgmap v559: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:29:33.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:29:33 smithi037 ceph-mon[142325]: pgmap v559: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:29:35.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:29:35 smithi148 ceph-mon[106692]: pgmap v560: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:29:35.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:29:35 smithi037 ceph-mon[142325]: pgmap v560: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:29:37.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:29:37 smithi148 ceph-mon[106692]: pgmap v561: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:29:37.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:29:37 smithi037 ceph-mon[142325]: pgmap v561: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:29:39.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:29:39 smithi148 ceph-mon[106692]: pgmap v562: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:29:39.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:29:39 smithi037 ceph-mon[142325]: pgmap v562: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:29:40.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:29:40 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:29:40.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:29:40 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:29:41.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:29:41 smithi148 ceph-mon[106692]: pgmap v563: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:29:41.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:29:41 smithi037 ceph-mon[142325]: pgmap v563: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:29:43.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:29:43 smithi148 ceph-mon[106692]: pgmap v564: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:29:43.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:29:43 smithi037 ceph-mon[142325]: pgmap v564: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:29:45.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:29:45 smithi148 ceph-mon[106692]: pgmap v565: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:29:45.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:29:45 smithi037 ceph-mon[142325]: pgmap v565: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:29:47.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:29:47 smithi148 ceph-mon[106692]: pgmap v566: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:29:47.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:29:47 smithi037 ceph-mon[142325]: pgmap v566: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:29:49.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:29:49 smithi148 ceph-mon[106692]: pgmap v567: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:29:49.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:29:49 smithi037 ceph-mon[142325]: pgmap v567: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:29:51.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:29:51 smithi148 ceph-mon[106692]: pgmap v568: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:29:51.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:29:51 smithi037 ceph-mon[142325]: pgmap v568: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:29:53.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:29:53 smithi148 ceph-mon[106692]: pgmap v569: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:29:53.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:29:53 smithi037 ceph-mon[142325]: pgmap v569: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:29:55.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:29:55 smithi148 ceph-mon[106692]: pgmap v570: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:29:55.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:29:55 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:29:55.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:29:55 smithi037 ceph-mon[142325]: pgmap v570: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:29:55.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:29:55 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:29:57.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:29:57 smithi148 ceph-mon[106692]: pgmap v571: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:29:57.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:29:57 smithi037 ceph-mon[142325]: pgmap v571: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:29:58.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:58.544+0000 7fafac600700 1 -- 172.21.15.37:0/3166241472 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fafa4102e00 msgr2=0x7fafa41031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:58.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:58.544+0000 7fafac600700 1 --2- 172.21.15.37:0/3166241472 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fafa4102e00 0x7fafa41031d0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7faf98004940 tx=0x7faf98009e30 comp rx=0 tx=0).stop 2024-03-29T08:29:58.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:58.544+0000 7fafac600700 1 -- 172.21.15.37:0/3166241472 shutdown_connections 2024-03-29T08:29:58.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:58.544+0000 7fafac600700 1 --2- 172.21.15.37:0/3166241472 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fafa41037a0 0x7fafa40fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:58.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:58.544+0000 7fafac600700 1 --2- 172.21.15.37:0/3166241472 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fafa4102e00 0x7fafa41031d0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:58.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:58.544+0000 7fafac600700 1 -- 172.21.15.37:0/3166241472 >> 172.21.15.37:0/3166241472 conn(0x7fafa40f89b0 msgr2=0x7fafa40fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:29:58.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:58.545+0000 7fafac600700 1 -- 172.21.15.37:0/3166241472 shutdown_connections 2024-03-29T08:29:58.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:58.545+0000 7fafac600700 1 -- 172.21.15.37:0/3166241472 wait complete. 2024-03-29T08:29:58.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:58.546+0000 7fafac600700 1 Processor -- start 2024-03-29T08:29:58.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:58.546+0000 7fafac600700 1 -- start start 2024-03-29T08:29:58.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:58.547+0000 7fafac600700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fafa4102e00 0x7fafa4115cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:58.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:58.547+0000 7fafac600700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fafa41037a0 0x7fafa4116210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:58.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:58.547+0000 7fafac600700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafa4116860 con 0x7fafa4102e00 2024-03-29T08:29:58.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:58.547+0000 7fafac600700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafa410fc30 con 0x7fafa41037a0 2024-03-29T08:29:58.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:58.547+0000 7fafa9c00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fafa4102e00 0x7fafa4115cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:58.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:58.547+0000 7fafa9c00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fafa4102e00 0x7fafa4115cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40868/0 (socket says 172.21.15.37:40868) 2024-03-29T08:29:58.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:58.547+0000 7fafa9c00700 1 -- 172.21.15.37:0/1187544936 learned_addr learned my addr 172.21.15.37:0/1187544936 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:29:58.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:58.547+0000 7fafa9200700 1 --2- 172.21.15.37:0/1187544936 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fafa41037a0 0x7fafa4116210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:58.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:58.547+0000 7fafa9c00700 1 -- 172.21.15.37:0/1187544936 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fafa41037a0 msgr2=0x7fafa4116210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:58.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:58.547+0000 7fafa9c00700 1 --2- 172.21.15.37:0/1187544936 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fafa41037a0 0x7fafa4116210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:58.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:58.547+0000 7fafa9c00700 1 -- 172.21.15.37:0/1187544936 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf98005040 con 0x7fafa4102e00 2024-03-29T08:29:58.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:58.548+0000 7fafa9c00700 1 --2- 172.21.15.37:0/1187544936 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fafa4102e00 0x7fafa4115cd0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7faf98009390 tx=0x7faf98009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:58.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:58.548+0000 7fafa1600700 1 -- 172.21.15.37:0/1187544936 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faf9800e040 con 0x7fafa4102e00 2024-03-29T08:29:58.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:58.548+0000 7fafa1600700 1 -- 172.21.15.37:0/1187544936 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7faf98013420 con 0x7fafa4102e00 2024-03-29T08:29:58.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:58.548+0000 7fafac600700 1 -- 172.21.15.37:0/1187544936 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafa410feb0 con 0x7fafa4102e00 2024-03-29T08:29:58.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:58.548+0000 7fafa1600700 1 -- 172.21.15.37:0/1187544936 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faf9801c630 con 0x7fafa4102e00 2024-03-29T08:29:58.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:58.548+0000 7fafac600700 1 -- 172.21.15.37:0/1187544936 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafa4110470 con 0x7fafa4102e00 2024-03-29T08:29:58.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:58.549+0000 7fafac600700 1 -- 172.21.15.37:0/1187544936 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fafa40fe550 con 0x7fafa4102e00 2024-03-29T08:29:58.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:58.550+0000 7fafa1600700 1 -- 172.21.15.37:0/1187544936 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7faf980139e0 con 0x7fafa4102e00 2024-03-29T08:29:58.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:58.551+0000 7fafa1600700 1 --2- 172.21.15.37:0/1187544936 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7faf8c06cc30 0x7faf8c06f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:58.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:58.551+0000 7fafa1600700 1 -- 172.21.15.37:0/1187544936 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7faf9808d250 con 0x7fafa4102e00 2024-03-29T08:29:58.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:58.551+0000 7fafa9200700 1 --2- 172.21.15.37:0/1187544936 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7faf8c06cc30 0x7faf8c06f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:58.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:58.552+0000 7fafa9200700 1 --2- 172.21.15.37:0/1187544936 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7faf8c06cc30 0x7faf8c06f0e0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fafa4111570 tx=0x7faf90009ee0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:58.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:58.556+0000 7fafa1600700 1 -- 172.21.15.37:0/1187544936 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7faf9805add0 con 0x7fafa4102e00 2024-03-29T08:29:58.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:58.789+0000 7fafac600700 1 -- 172.21.15.37:0/1187544936 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fafa404bf40 con 0x7faf8c06cc30 2024-03-29T08:29:58.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:58.790+0000 7fafa1600700 1 -- 172.21.15.37:0/1187544936 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fafa404bf40 con 0x7faf8c06cc30 2024-03-29T08:29:58.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:58.793+0000 7fafac600700 1 -- 172.21.15.37:0/1187544936 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7faf8c06cc30 msgr2=0x7faf8c06f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:58.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:58.793+0000 7fafac600700 1 --2- 172.21.15.37:0/1187544936 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7faf8c06cc30 0x7faf8c06f0e0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fafa4111570 tx=0x7faf90009ee0 comp rx=0 tx=0).stop 2024-03-29T08:29:58.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:58.793+0000 7fafac600700 1 -- 172.21.15.37:0/1187544936 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fafa4102e00 msgr2=0x7fafa4115cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:58.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:58.793+0000 7fafac600700 1 --2- 172.21.15.37:0/1187544936 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fafa4102e00 0x7fafa4115cd0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7faf98009390 tx=0x7faf98009e30 comp rx=0 tx=0).stop 2024-03-29T08:29:58.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:58.793+0000 7fafac600700 1 -- 172.21.15.37:0/1187544936 shutdown_connections 2024-03-29T08:29:58.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:58.793+0000 7fafac600700 1 --2- 172.21.15.37:0/1187544936 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7faf8c06cc30 0x7faf8c06f0e0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:58.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:58.793+0000 7fafac600700 1 --2- 172.21.15.37:0/1187544936 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fafa4102e00 0x7fafa4115cd0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:58.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:58.793+0000 7fafac600700 1 --2- 172.21.15.37:0/1187544936 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fafa41037a0 0x7fafa4116210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:58.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:58.793+0000 7fafac600700 1 -- 172.21.15.37:0/1187544936 >> 172.21.15.37:0/1187544936 conn(0x7fafa40f89b0 msgr2=0x7fafa40f99e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:29:58.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:58.793+0000 7fafac600700 1 -- 172.21.15.37:0/1187544936 shutdown_connections 2024-03-29T08:29:58.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:58.793+0000 7fafac600700 1 -- 172.21.15.37:0/1187544936 wait complete. 2024-03-29T08:29:58.808 INFO:teuthology.orchestra.run.smithi037.stdout:true 2024-03-29T08:29:58.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:58.932+0000 7f4b3ac00700 1 -- 172.21.15.37:0/2743841660 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4b340fde90 msgr2=0x7f4b340fe260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:58.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:58.932+0000 7f4b3ac00700 1 --2- 172.21.15.37:0/2743841660 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4b340fde90 0x7f4b340fe260 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f4b24004940 tx=0x7f4b24009e30 comp rx=0 tx=0).stop 2024-03-29T08:29:58.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:58.932+0000 7f4b3ac00700 1 -- 172.21.15.37:0/2743841660 shutdown_connections 2024-03-29T08:29:58.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:58.932+0000 7f4b3ac00700 1 --2- 172.21.15.37:0/2743841660 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4b340fe830 0x7f4b34106f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:58.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:58.932+0000 7f4b3ac00700 1 --2- 172.21.15.37:0/2743841660 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4b340fde90 0x7f4b340fe260 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:58.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:58.932+0000 7f4b3ac00700 1 -- 172.21.15.37:0/2743841660 >> 172.21.15.37:0/2743841660 conn(0x7f4b34072bc0 msgr2=0x7f4b34072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:29:58.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:58.932+0000 7f4b3ac00700 1 -- 172.21.15.37:0/2743841660 shutdown_connections 2024-03-29T08:29:58.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:58.932+0000 7f4b3ac00700 1 -- 172.21.15.37:0/2743841660 wait complete. 2024-03-29T08:29:58.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:58.933+0000 7f4b3ac00700 1 Processor -- start 2024-03-29T08:29:58.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:58.933+0000 7f4b3ac00700 1 -- start start 2024-03-29T08:29:58.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:58.934+0000 7f4b3ac00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4b340fde90 0x7f4b34115c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:58.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:58.934+0000 7f4b3ac00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4b340fe830 0x7f4b341161a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:58.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:58.934+0000 7f4b3ac00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b341167f0 con 0x7f4b340fde90 2024-03-29T08:29:58.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:58.934+0000 7f4b3ac00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b3410fbc0 con 0x7f4b340fe830 2024-03-29T08:29:58.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:58.934+0000 7f4b33e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4b340fde90 0x7f4b34115c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:58.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:58.934+0000 7f4b33e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4b340fde90 0x7f4b34115c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40880/0 (socket says 172.21.15.37:40880) 2024-03-29T08:29:58.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:58.934+0000 7f4b33e00700 1 -- 172.21.15.37:0/49011765 learned_addr learned my addr 172.21.15.37:0/49011765 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:29:58.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:58.934+0000 7f4b33400700 1 --2- 172.21.15.37:0/49011765 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4b340fe830 0x7f4b341161a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:58.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:58.934+0000 7f4b33e00700 1 -- 172.21.15.37:0/49011765 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4b340fe830 msgr2=0x7f4b341161a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:58.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:58.934+0000 7f4b33e00700 1 --2- 172.21.15.37:0/49011765 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4b340fe830 0x7f4b341161a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:58.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:58.934+0000 7f4b33e00700 1 -- 172.21.15.37:0/49011765 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b24005040 con 0x7f4b340fde90 2024-03-29T08:29:58.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:58.935+0000 7f4b33e00700 1 --2- 172.21.15.37:0/49011765 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4b340fde90 0x7f4b34115c60 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f4b24000c00 tx=0x7f4b24009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:58.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:58.935+0000 7f4b2be00700 1 -- 172.21.15.37:0/49011765 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4b24021070 con 0x7f4b340fde90 2024-03-29T08:29:58.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:58.935+0000 7f4b2be00700 1 -- 172.21.15.37:0/49011765 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f4b24013e00 con 0x7f4b340fde90 2024-03-29T08:29:58.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:58.935+0000 7f4b3ac00700 1 -- 172.21.15.37:0/49011765 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b3410fe40 con 0x7f4b340fde90 2024-03-29T08:29:58.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:58.935+0000 7f4b2be00700 1 -- 172.21.15.37:0/49011765 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4b2400e070 con 0x7f4b340fde90 2024-03-29T08:29:58.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:58.935+0000 7f4b3ac00700 1 -- 172.21.15.37:0/49011765 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b34110380 con 0x7f4b340fde90 2024-03-29T08:29:58.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:58.936+0000 7f4b3ac00700 1 -- 172.21.15.37:0/49011765 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b34106a20 con 0x7f4b340fde90 2024-03-29T08:29:58.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:58.937+0000 7f4b2be00700 1 -- 172.21.15.37:0/49011765 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f4b24013950 con 0x7f4b340fde90 2024-03-29T08:29:58.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:58.938+0000 7f4b2be00700 1 --2- 172.21.15.37:0/49011765 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f4b1806cc30 0x7f4b1806f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:58.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:58.938+0000 7f4b2be00700 1 -- 172.21.15.37:0/49011765 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f4b2408cff0 con 0x7f4b340fde90 2024-03-29T08:29:58.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:58.938+0000 7f4b33400700 1 --2- 172.21.15.37:0/49011765 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f4b1806cc30 0x7f4b1806f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:58.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:58.939+0000 7f4b33400700 1 --2- 172.21.15.37:0/49011765 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f4b1806cc30 0x7f4b1806f0e0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f4b34111500 tx=0x7f4b2000b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:58.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:58.943+0000 7f4b2be00700 1 -- 172.21.15.37:0/49011765 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f4b240c09f0 con 0x7f4b340fde90 2024-03-29T08:29:59.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:59.179+0000 7f4b3ac00700 1 -- 172.21.15.37:0/49011765 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4b34109dc0 con 0x7f4b1806cc30 2024-03-29T08:29:59.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:59.180+0000 7f4b2be00700 1 -- 172.21.15.37:0/49011765 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f4b34109dc0 con 0x7f4b1806cc30 2024-03-29T08:29:59.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:59.182+0000 7f4b3ac00700 1 -- 172.21.15.37:0/49011765 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f4b1806cc30 msgr2=0x7f4b1806f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:59.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:59.182+0000 7f4b3ac00700 1 --2- 172.21.15.37:0/49011765 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f4b1806cc30 0x7f4b1806f0e0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f4b34111500 tx=0x7f4b2000b040 comp rx=0 tx=0).stop 2024-03-29T08:29:59.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:59.182+0000 7f4b3ac00700 1 -- 172.21.15.37:0/49011765 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4b340fde90 msgr2=0x7f4b34115c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:59.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:59.182+0000 7f4b3ac00700 1 --2- 172.21.15.37:0/49011765 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4b340fde90 0x7f4b34115c60 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f4b24000c00 tx=0x7f4b24009e30 comp rx=0 tx=0).stop 2024-03-29T08:29:59.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:59.183+0000 7f4b3ac00700 1 -- 172.21.15.37:0/49011765 shutdown_connections 2024-03-29T08:29:59.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:59.183+0000 7f4b3ac00700 1 --2- 172.21.15.37:0/49011765 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f4b1806cc30 0x7f4b1806f0e0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:59.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:59.183+0000 7f4b3ac00700 1 --2- 172.21.15.37:0/49011765 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4b340fde90 0x7f4b34115c60 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:59.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:59.183+0000 7f4b3ac00700 1 --2- 172.21.15.37:0/49011765 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4b340fe830 0x7f4b341161a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:59.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:59.183+0000 7f4b3ac00700 1 -- 172.21.15.37:0/49011765 >> 172.21.15.37:0/49011765 conn(0x7f4b34072bc0 msgr2=0x7f4b340fd570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:29:59.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:59.183+0000 7f4b3ac00700 1 -- 172.21.15.37:0/49011765 shutdown_connections 2024-03-29T08:29:59.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:59.183+0000 7f4b3ac00700 1 -- 172.21.15.37:0/49011765 wait complete. 2024-03-29T08:29:59.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:59.319+0000 7fcba6400700 1 -- 172.21.15.37:0/3068279333 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcba01037a0 msgr2=0x7fcba00fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:59.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:59.319+0000 7fcba6400700 1 --2- 172.21.15.37:0/3068279333 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcba01037a0 0x7fcba00fce10 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fcb8c004940 tx=0x7fcb8c009e30 comp rx=0 tx=0).stop 2024-03-29T08:29:59.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:59.320+0000 7fcba6400700 1 -- 172.21.15.37:0/3068279333 shutdown_connections 2024-03-29T08:29:59.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:59.320+0000 7fcba6400700 1 --2- 172.21.15.37:0/3068279333 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcba01037a0 0x7fcba00fce10 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:59.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:59.320+0000 7fcba6400700 1 --2- 172.21.15.37:0/3068279333 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcba0102e00 0x7fcba01031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:59.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:59.320+0000 7fcba6400700 1 -- 172.21.15.37:0/3068279333 >> 172.21.15.37:0/3068279333 conn(0x7fcba00f89b0 msgr2=0x7fcba00fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:29:59.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:59.320+0000 7fcba6400700 1 -- 172.21.15.37:0/3068279333 shutdown_connections 2024-03-29T08:29:59.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:59.320+0000 7fcba6400700 1 -- 172.21.15.37:0/3068279333 wait complete. 2024-03-29T08:29:59.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:59.321+0000 7fcba6400700 1 Processor -- start 2024-03-29T08:29:59.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:59.321+0000 7fcba6400700 1 -- start start 2024-03-29T08:29:59.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:59.321+0000 7fcba6400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcba0102e00 0x7fcba0115cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:59.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:59.321+0000 7fcba6400700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcba01037a0 0x7fcba0116230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:59.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:59.321+0000 7fcba6400700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcba0116910 con 0x7fcba0102e00 2024-03-29T08:29:59.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:59.321+0000 7fcba6400700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcba010fc50 con 0x7fcba01037a0 2024-03-29T08:29:59.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:59.322+0000 7fcb9f400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcba0102e00 0x7fcba0115cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:59.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:59.322+0000 7fcb9f400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcba0102e00 0x7fcba0115cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40896/0 (socket says 172.21.15.37:40896) 2024-03-29T08:29:59.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:59.322+0000 7fcb9f400700 1 -- 172.21.15.37:0/896256892 learned_addr learned my addr 172.21.15.37:0/896256892 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:29:59.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:59.322+0000 7fcb9ea00700 1 --2- 172.21.15.37:0/896256892 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcba01037a0 0x7fcba0116230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:59.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:59.322+0000 7fcb9f400700 1 -- 172.21.15.37:0/896256892 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcba01037a0 msgr2=0x7fcba0116230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:59.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:59.322+0000 7fcb9f400700 1 --2- 172.21.15.37:0/896256892 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcba01037a0 0x7fcba0116230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:59.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:59.322+0000 7fcb9f400700 1 -- 172.21.15.37:0/896256892 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb8c005040 con 0x7fcba0102e00 2024-03-29T08:29:59.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:59.322+0000 7fcb9ea00700 1 --2- 172.21.15.37:0/896256892 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcba01037a0 0x7fcba0116230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:29:59.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:59.322+0000 7fcb9f400700 1 --2- 172.21.15.37:0/896256892 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcba0102e00 0x7fcba0115cf0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fcb9000b790 tx=0x7fcb90009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:59.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:59.324+0000 7fcb97400700 1 -- 172.21.15.37:0/896256892 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcb9000f770 con 0x7fcba0102e00 2024-03-29T08:29:59.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:59.324+0000 7fcb97400700 1 -- 172.21.15.37:0/896256892 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fcb9000fd80 con 0x7fcba0102e00 2024-03-29T08:29:59.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:59.324+0000 7fcba6400700 1 -- 172.21.15.37:0/896256892 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcba010fed0 con 0x7fcba0102e00 2024-03-29T08:29:59.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:59.324+0000 7fcb97400700 1 -- 172.21.15.37:0/896256892 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcb9000d310 con 0x7fcba0102e00 2024-03-29T08:29:59.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:59.324+0000 7fcba6400700 1 -- 172.21.15.37:0/896256892 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcba0110490 con 0x7fcba0102e00 2024-03-29T08:29:59.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:59.325+0000 7fcba6400700 1 -- 172.21.15.37:0/896256892 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcba00fe550 con 0x7fcba0102e00 2024-03-29T08:29:59.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:59.326+0000 7fcb97400700 1 -- 172.21.15.37:0/896256892 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fcb9000f8d0 con 0x7fcba0102e00 2024-03-29T08:29:59.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:59.327+0000 7fcb97400700 1 --2- 172.21.15.37:0/896256892 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fcb8406cc30 0x7fcb8406f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:59.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:59.327+0000 7fcb97400700 1 -- 172.21.15.37:0/896256892 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fcb9008c0a0 con 0x7fcba0102e00 2024-03-29T08:29:59.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:59.327+0000 7fcb9ea00700 1 --2- 172.21.15.37:0/896256892 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fcb8406cc30 0x7fcb8406f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:59.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:59.328+0000 7fcb9ea00700 1 --2- 172.21.15.37:0/896256892 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fcb8406cc30 0x7fcb8406f0e0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fcba0111590 tx=0x7fcb8c009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:59.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:59.332+0000 7fcb97400700 1 -- 172.21.15.37:0/896256892 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fcb90059af0 con 0x7fcba0102e00 2024-03-29T08:29:59.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:59.559+0000 7fcba6400700 1 -- 172.21.15.37:0/896256892 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fcba004bf70 con 0x7fcb8406cc30 2024-03-29T08:29:59.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:59.569+0000 7fcb97400700 1 -- 172.21.15.37:0/896256892 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7fcb8c0096f0 con 0x7fcb8406cc30 2024-03-29T08:29:59.569 INFO:teuthology.orchestra.run.smithi037.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-29T08:29:59.569 INFO:teuthology.orchestra.run.smithi037.stdout:alertmanager.smithi037 smithi037 *:9093,9094 running (19m) 12m ago 32m 14.9M - 0.25.0 c8568f914cd2 2f9f9746ed0f 2024-03-29T08:29:59.569 INFO:teuthology.orchestra.run.smithi037.stdout:crash.smithi037 smithi037 running (16m) 12m ago 33m 6647k - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 178d416a3186 2024-03-29T08:29:59.569 INFO:teuthology.orchestra.run.smithi037.stdout:crash.smithi148 smithi148 running (16m) 11m ago 31m 6656k - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 23666989d58f 2024-03-29T08:29:59.570 INFO:teuthology.orchestra.run.smithi037.stdout:grafana.smithi037 smithi037 *:3000 running (19m) 12m ago 32m 71.9M - 9.4.7 954c08fa6188 6b976fe49952 2024-03-29T08:29:59.570 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi037.iifsmh smithi037 running (12m) 12m ago 26m 10.8M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 2d97f562cc5b 2024-03-29T08:29:59.570 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi037.mmwgxe smithi037 running (12m) 12m ago 26m 10.9M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff ec37037f46e8 2024-03-29T08:29:59.570 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi148.qukpjh smithi148 running (26m) 11m ago 26m 15.3M - 17.2.7-789-g4213e17c cf22eb8683be 8e469782a273 2024-03-29T08:29:59.570 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi148.sqteih smithi148 running (11m) 11m ago 26m 10.7M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 44c15c77037a 2024-03-29T08:29:59.570 INFO:teuthology.orchestra.run.smithi037.stdout:mgr.smithi037.wjikii smithi037 *:8443,9283,8765 running (22m) 12m ago 34m 506M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff c5a1417b9417 2024-03-29T08:29:59.570 INFO:teuthology.orchestra.run.smithi037.stdout:mgr.smithi148.onhgvm smithi148 *:8443,9283,8765 running (20m) 11m ago 31m 450M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff fcfdbce062bd 2024-03-29T08:29:59.570 INFO:teuthology.orchestra.run.smithi037.stdout:mon.smithi037 smithi037 running (18m) 12m ago 34m 54.8M 2048M 18.2.2-1015-gd0e392d9 1cc2f45c17ff eea2060783af 2024-03-29T08:29:59.570 INFO:teuthology.orchestra.run.smithi037.stdout:mon.smithi148 smithi148 running (17m) 11m ago 31m 43.7M 2048M 18.2.2-1015-gd0e392d9 1cc2f45c17ff ad4b2c5748be 2024-03-29T08:29:59.570 INFO:teuthology.orchestra.run.smithi037.stdout:node-exporter.smithi037 smithi037 *:9100 running (20m) 12m ago 32m 7625k - 1.5.0 0da6a335fe13 26b8ae9cb878 2024-03-29T08:29:59.570 INFO:teuthology.orchestra.run.smithi037.stdout:node-exporter.smithi148 smithi148 *:9100 running (20m) 11m ago 31m 8166k - 1.5.0 0da6a335fe13 98414e4e64ed 2024-03-29T08:29:59.570 INFO:teuthology.orchestra.run.smithi037.stdout:osd.0 smithi037 running (16m) 12m ago 30m 104M 1619M 18.2.2-1015-gd0e392d9 1cc2f45c17ff c0c584f6245c 2024-03-29T08:29:59.570 INFO:teuthology.orchestra.run.smithi037.stdout:osd.1 smithi037 running (15m) 12m ago 30m 100M 1619M 18.2.2-1015-gd0e392d9 1cc2f45c17ff a4a2ff4cc83d 2024-03-29T08:29:59.570 INFO:teuthology.orchestra.run.smithi037.stdout:osd.2 smithi037 running (14m) 12m ago 29m 82.3M 1619M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 8b1bc71aa79e 2024-03-29T08:29:59.571 INFO:teuthology.orchestra.run.smithi037.stdout:osd.3 smithi148 running (14m) 11m ago 29m 105M 2642M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 1c656ab8ff86 2024-03-29T08:29:59.571 INFO:teuthology.orchestra.run.smithi037.stdout:osd.4 smithi148 running (13m) 11m ago 28m 97.3M 2642M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 966a9de11d22 2024-03-29T08:29:59.571 INFO:teuthology.orchestra.run.smithi037.stdout:osd.5 smithi148 running (12m) 11m ago 28m 85.8M 2642M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 474c4f9212ce 2024-03-29T08:29:59.571 INFO:teuthology.orchestra.run.smithi037.stdout:prometheus.smithi037 smithi037 *:9095 running (20m) 12m ago 32m 54.4M - 2.43.0 a07b618ecd1d 30c0223ec589 2024-03-29T08:29:59.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:59.571+0000 7fcba6400700 1 -- 172.21.15.37:0/896256892 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fcb8406cc30 msgr2=0x7fcb8406f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:59.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:59.571+0000 7fcba6400700 1 --2- 172.21.15.37:0/896256892 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fcb8406cc30 0x7fcb8406f0e0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fcba0111590 tx=0x7fcb8c009e30 comp rx=0 tx=0).stop 2024-03-29T08:29:59.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:59.571+0000 7fcba6400700 1 -- 172.21.15.37:0/896256892 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcba0102e00 msgr2=0x7fcba0115cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:59.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:59.571+0000 7fcba6400700 1 --2- 172.21.15.37:0/896256892 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcba0102e00 0x7fcba0115cf0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fcb9000b790 tx=0x7fcb90009250 comp rx=0 tx=0).stop 2024-03-29T08:29:59.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:59.572+0000 7fcba6400700 1 -- 172.21.15.37:0/896256892 shutdown_connections 2024-03-29T08:29:59.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:59.572+0000 7fcba6400700 1 --2- 172.21.15.37:0/896256892 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fcb8406cc30 0x7fcb8406f0e0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:59.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:59.572+0000 7fcba6400700 1 --2- 172.21.15.37:0/896256892 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcba0102e00 0x7fcba0115cf0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:59.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:59.572+0000 7fcba6400700 1 --2- 172.21.15.37:0/896256892 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcba01037a0 0x7fcba0116230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:59.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:59.572+0000 7fcba6400700 1 -- 172.21.15.37:0/896256892 >> 172.21.15.37:0/896256892 conn(0x7fcba00f89b0 msgr2=0x7fcba0070680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:29:59.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:59.572+0000 7fcba6400700 1 -- 172.21.15.37:0/896256892 shutdown_connections 2024-03-29T08:29:59.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:59.572+0000 7fcba6400700 1 -- 172.21.15.37:0/896256892 wait complete. 2024-03-29T08:29:59.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:59.708+0000 7f1380000700 1 -- 172.21.15.37:0/3595989624 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f13780fe830 msgr2=0x7f1378106f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:59.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:59.708+0000 7f1380000700 1 --2- 172.21.15.37:0/3595989624 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f13780fe830 0x7f1378106f00 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f1364004940 tx=0x7f1364009e30 comp rx=0 tx=0).stop 2024-03-29T08:29:59.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:59.708+0000 7f1380000700 1 -- 172.21.15.37:0/3595989624 shutdown_connections 2024-03-29T08:29:59.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:59.708+0000 7f1380000700 1 --2- 172.21.15.37:0/3595989624 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f13780fe830 0x7f1378106f00 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:59.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:59.708+0000 7f1380000700 1 --2- 172.21.15.37:0/3595989624 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f13780fde90 0x7f13780fe260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:59.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:59.708+0000 7f1380000700 1 -- 172.21.15.37:0/3595989624 >> 172.21.15.37:0/3595989624 conn(0x7f1378072bc0 msgr2=0x7f1378072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:29:59.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:59.708+0000 7f1380000700 1 -- 172.21.15.37:0/3595989624 shutdown_connections 2024-03-29T08:29:59.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:59.708+0000 7f1380000700 1 -- 172.21.15.37:0/3595989624 wait complete. 2024-03-29T08:29:59.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:59.709+0000 7f1380000700 1 Processor -- start 2024-03-29T08:29:59.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:59.709+0000 7f1380000700 1 -- start start 2024-03-29T08:29:59.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:59.710+0000 7f1380000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f13780fde90 0x7f1378115d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:59.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:59.710+0000 7f1380000700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f13780fe830 0x7f1378116250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:59.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:59.710+0000 7f1380000700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13781168a0 con 0x7f13780fde90 2024-03-29T08:29:59.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:59.710+0000 7f1380000700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f137810fc70 con 0x7f13780fe830 2024-03-29T08:29:59.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:59.710+0000 7f137d600700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f13780fde90 0x7f1378115d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:59.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:59.710+0000 7f137d600700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f13780fde90 0x7f1378115d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40906/0 (socket says 172.21.15.37:40906) 2024-03-29T08:29:59.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:59.710+0000 7f137d600700 1 -- 172.21.15.37:0/25629078 learned_addr learned my addr 172.21.15.37:0/25629078 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:29:59.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:59.710+0000 7f137cc00700 1 --2- 172.21.15.37:0/25629078 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f13780fe830 0x7f1378116250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:59.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:59.711+0000 7f137d600700 1 -- 172.21.15.37:0/25629078 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f13780fe830 msgr2=0x7f1378116250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:59.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:59.711+0000 7f137d600700 1 --2- 172.21.15.37:0/25629078 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f13780fe830 0x7f1378116250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:59.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:59.711+0000 7f137d600700 1 -- 172.21.15.37:0/25629078 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1364005040 con 0x7f13780fde90 2024-03-29T08:29:59.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:59.711+0000 7f137d600700 1 --2- 172.21.15.37:0/25629078 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f13780fde90 0x7f1378115d10 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f136c002a50 tx=0x7f136c0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:59.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:59.711+0000 7f1375600700 1 -- 172.21.15.37:0/25629078 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f136c007cc0 con 0x7f13780fde90 2024-03-29T08:29:59.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:59.711+0000 7f1375600700 1 -- 172.21.15.37:0/25629078 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f136c00f040 con 0x7f13780fde90 2024-03-29T08:29:59.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:59.711+0000 7f1375600700 1 -- 172.21.15.37:0/25629078 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f136c00d9c0 con 0x7f13780fde90 2024-03-29T08:29:59.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:59.711+0000 7f1380000700 1 -- 172.21.15.37:0/25629078 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f137810ff50 con 0x7f13780fde90 2024-03-29T08:29:59.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:59.711+0000 7f1380000700 1 -- 172.21.15.37:0/25629078 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1378110490 con 0x7f13780fde90 2024-03-29T08:29:59.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:59.712+0000 7f1380000700 1 -- 172.21.15.37:0/25629078 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1378106a20 con 0x7f13780fde90 2024-03-29T08:29:59.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:59.713+0000 7f1375600700 1 -- 172.21.15.37:0/25629078 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f136c00db20 con 0x7f13780fde90 2024-03-29T08:29:59.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:59.714+0000 7f1375600700 1 --2- 172.21.15.37:0/25629078 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f136006cc30 0x7f136006f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:59.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:59.714+0000 7f1375600700 1 -- 172.21.15.37:0/25629078 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f136c08bd20 con 0x7f13780fde90 2024-03-29T08:29:59.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:59.714+0000 7f137cc00700 1 --2- 172.21.15.37:0/25629078 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f136006cc30 0x7f136006f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:59.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:59.715+0000 7f137cc00700 1 --2- 172.21.15.37:0/25629078 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f136006cc30 0x7f136006f0e0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f13781115b0 tx=0x7f1364009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:59.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:29:59.719+0000 7f1375600700 1 -- 172.21.15.37:0/25629078 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f136c059770 con 0x7f13780fde90 2024-03-29T08:29:59.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:29:59 smithi148 ceph-mon[106692]: pgmap v572: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:29:59.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:29:59 smithi037 ceph-mon[142325]: pgmap v572: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:30:00.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:00.021+0000 7f1380000700 1 -- 172.21.15.37:0/25629078 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f137804bed0 con 0x7f13780fde90 2024-03-29T08:30:00.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:00.021+0000 7f1375600700 1 -- 172.21.15.37:0/25629078 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+847 (secure 0 0 0) 0x7f136c059300 con 0x7f13780fde90 2024-03-29T08:30:00.021 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-29T08:30:00.022 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": { 2024-03-29T08:30:00.022 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 2 2024-03-29T08:30:00.022 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:30:00.022 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": { 2024-03-29T08:30:00.022 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 2 2024-03-29T08:30:00.022 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:30:00.022 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": { 2024-03-29T08:30:00.022 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 6 2024-03-29T08:30:00.022 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:30:00.022 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": { 2024-03-29T08:30:00.022 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 17.2.7-789-g4213e17c (4213e17c4de67016ed14f55d22503f4923b3a901) quincy (stable)": 1, 2024-03-29T08:30:00.022 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 3 2024-03-29T08:30:00.022 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:30:00.022 INFO:teuthology.orchestra.run.smithi037.stdout: "overall": { 2024-03-29T08:30:00.023 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 17.2.7-789-g4213e17c (4213e17c4de67016ed14f55d22503f4923b3a901) quincy (stable)": 1, 2024-03-29T08:30:00.023 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 13 2024-03-29T08:30:00.023 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-29T08:30:00.023 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-29T08:30:00.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:00.024+0000 7f1380000700 1 -- 172.21.15.37:0/25629078 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f136006cc30 msgr2=0x7f136006f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:00.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:00.024+0000 7f1380000700 1 --2- 172.21.15.37:0/25629078 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f136006cc30 0x7f136006f0e0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f13781115b0 tx=0x7f1364009e30 comp rx=0 tx=0).stop 2024-03-29T08:30:00.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:00.024+0000 7f1380000700 1 -- 172.21.15.37:0/25629078 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f13780fde90 msgr2=0x7f1378115d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:00.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:00.024+0000 7f1380000700 1 --2- 172.21.15.37:0/25629078 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f13780fde90 0x7f1378115d10 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f136c002a50 tx=0x7f136c0046b0 comp rx=0 tx=0).stop 2024-03-29T08:30:00.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:00.024+0000 7f1380000700 1 -- 172.21.15.37:0/25629078 shutdown_connections 2024-03-29T08:30:00.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:00.024+0000 7f1380000700 1 --2- 172.21.15.37:0/25629078 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f136006cc30 0x7f136006f0e0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:00.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:00.024+0000 7f1380000700 1 --2- 172.21.15.37:0/25629078 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f13780fde90 0x7f1378115d10 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:00.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:00.024+0000 7f1380000700 1 --2- 172.21.15.37:0/25629078 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f13780fe830 0x7f1378116250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:00.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:00.024+0000 7f1380000700 1 -- 172.21.15.37:0/25629078 >> 172.21.15.37:0/25629078 conn(0x7f1378072bc0 msgr2=0x7f13780fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:30:00.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:00.024+0000 7f1380000700 1 -- 172.21.15.37:0/25629078 shutdown_connections 2024-03-29T08:30:00.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:00.024+0000 7f1380000700 1 -- 172.21.15.37:0/25629078 wait complete. 2024-03-29T08:30:00.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:00.161+0000 7fc894c00700 1 -- 172.21.15.37:0/470170682 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc890102e00 msgr2=0x7fc8901031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:00.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:00.161+0000 7fc894c00700 1 --2- 172.21.15.37:0/470170682 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc890102e00 0x7fc8901031d0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fc880004940 tx=0x7fc880009e30 comp rx=0 tx=0).stop 2024-03-29T08:30:00.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:00.161+0000 7fc894c00700 1 -- 172.21.15.37:0/470170682 shutdown_connections 2024-03-29T08:30:00.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:00.161+0000 7fc894c00700 1 --2- 172.21.15.37:0/470170682 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc8901037a0 0x7fc8900fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:00.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:00.161+0000 7fc894c00700 1 --2- 172.21.15.37:0/470170682 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc890102e00 0x7fc8901031d0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:00.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:00.161+0000 7fc894c00700 1 -- 172.21.15.37:0/470170682 >> 172.21.15.37:0/470170682 conn(0x7fc8900f89b0 msgr2=0x7fc8900fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:30:00.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:00.161+0000 7fc894c00700 1 -- 172.21.15.37:0/470170682 shutdown_connections 2024-03-29T08:30:00.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:00.161+0000 7fc894c00700 1 -- 172.21.15.37:0/470170682 wait complete. 2024-03-29T08:30:00.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:00.162+0000 7fc894c00700 1 Processor -- start 2024-03-29T08:30:00.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:00.163+0000 7fc894c00700 1 -- start start 2024-03-29T08:30:00.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:00.163+0000 7fc894c00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc890102e00 0x7fc890113b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:00.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:00.163+0000 7fc894c00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc8901037a0 0x7fc890114060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:00.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:00.163+0000 7fc894c00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8901146b0 con 0x7fc8901037a0 2024-03-29T08:30:00.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:00.163+0000 7fc894c00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc89010daa0 con 0x7fc890102e00 2024-03-29T08:30:00.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:00.164+0000 7fc88d400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc8901037a0 0x7fc890114060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:00.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:00.164+0000 7fc88d400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc8901037a0 0x7fc890114060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43172/0 (socket says 172.21.15.37:43172) 2024-03-29T08:30:00.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:00.164+0000 7fc88d400700 1 -- 172.21.15.37:0/4088085575 learned_addr learned my addr 172.21.15.37:0/4088085575 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:30:00.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:00.164+0000 7fc88de00700 1 --2- 172.21.15.37:0/4088085575 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc890102e00 0x7fc890113b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:00.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:00.164+0000 7fc88d400700 1 -- 172.21.15.37:0/4088085575 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc890102e00 msgr2=0x7fc890113b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:00.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:00.164+0000 7fc88d400700 1 --2- 172.21.15.37:0/4088085575 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc890102e00 0x7fc890113b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:00.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:00.164+0000 7fc88d400700 1 -- 172.21.15.37:0/4088085575 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc880005040 con 0x7fc8901037a0 2024-03-29T08:30:00.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:00.164+0000 7fc88d400700 1 --2- 172.21.15.37:0/4088085575 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc8901037a0 0x7fc890114060 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fc87c002a50 tx=0x7fc87c004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:00.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:00.165+0000 7fc886000700 1 -- 172.21.15.37:0/4088085575 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc87c007cc0 con 0x7fc8901037a0 2024-03-29T08:30:00.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:00.165+0000 7fc886000700 1 -- 172.21.15.37:0/4088085575 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fc87c00f040 con 0x7fc8901037a0 2024-03-29T08:30:00.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:00.165+0000 7fc894c00700 1 -- 172.21.15.37:0/4088085575 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc89010dd80 con 0x7fc8901037a0 2024-03-29T08:30:00.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:00.165+0000 7fc886000700 1 -- 172.21.15.37:0/4088085575 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc87c00d920 con 0x7fc8901037a0 2024-03-29T08:30:00.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:00.165+0000 7fc894c00700 1 -- 172.21.15.37:0/4088085575 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc89010e340 con 0x7fc8901037a0 2024-03-29T08:30:00.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:00.166+0000 7fc894c00700 1 -- 172.21.15.37:0/4088085575 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8900fe550 con 0x7fc8901037a0 2024-03-29T08:30:00.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:00.167+0000 7fc886000700 1 -- 172.21.15.37:0/4088085575 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fc87c009680 con 0x7fc8901037a0 2024-03-29T08:30:00.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:00.168+0000 7fc886000700 1 --2- 172.21.15.37:0/4088085575 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fc87406cc30 0x7fc87406f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:00.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:00.168+0000 7fc886000700 1 -- 172.21.15.37:0/4088085575 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fc87c08bd10 con 0x7fc8901037a0 2024-03-29T08:30:00.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:00.168+0000 7fc88de00700 1 --2- 172.21.15.37:0/4088085575 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fc87406cc30 0x7fc87406f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:00.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:00.169+0000 7fc88de00700 1 --2- 172.21.15.37:0/4088085575 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fc87406cc30 0x7fc87406f0e0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fc88000b840 tx=0x7fc880009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:00.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:00.172+0000 7fc886000700 1 -- 172.21.15.37:0/4088085575 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fc87c059760 con 0x7fc8901037a0 2024-03-29T08:30:00.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:30:00 smithi037 ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4-mon-smithi037[142302]: 2024-03-29T08:29:59.999+0000 7f2c60400700 -1 log_channel(cluster) log [ERR] : overall HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-03-29T08:30:00.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:00.430+0000 7fc894c00700 1 -- 172.21.15.37:0/4088085575 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fc890109dc0 con 0x7fc8901037a0 2024-03-29T08:30:00.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:00.431+0000 7fc886000700 1 -- 172.21.15.37:0/4088085575 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 28 v28) v1 ==== 76+0+1841 (secure 0 0 0) 0x7fc87c0592f0 con 0x7fc8901037a0 2024-03-29T08:30:00.431 INFO:teuthology.orchestra.run.smithi037.stdout:e28 2024-03-29T08:30:00.431 INFO:teuthology.orchestra.run.smithi037.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-29T08:30:00.431 INFO:teuthology.orchestra.run.smithi037.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T08:30:00.431 INFO:teuthology.orchestra.run.smithi037.stdout:legacy client fscid: 1 2024-03-29T08:30:00.431 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:30:00.431 INFO:teuthology.orchestra.run.smithi037.stdout:Filesystem 'cephfs' (1) 2024-03-29T08:30:00.431 INFO:teuthology.orchestra.run.smithi037.stdout:fs_name cephfs 2024-03-29T08:30:00.432 INFO:teuthology.orchestra.run.smithi037.stdout:epoch 21 2024-03-29T08:30:00.432 INFO:teuthology.orchestra.run.smithi037.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-03-29T08:30:00.432 INFO:teuthology.orchestra.run.smithi037.stdout:created 2024-03-29T08:03:29.241269+0000 2024-03-29T08:30:00.432 INFO:teuthology.orchestra.run.smithi037.stdout:modified 2024-03-29T08:17:16.237238+0000 2024-03-29T08:30:00.432 INFO:teuthology.orchestra.run.smithi037.stdout:tableserver 0 2024-03-29T08:30:00.432 INFO:teuthology.orchestra.run.smithi037.stdout:root 0 2024-03-29T08:30:00.432 INFO:teuthology.orchestra.run.smithi037.stdout:session_timeout 60 2024-03-29T08:30:00.432 INFO:teuthology.orchestra.run.smithi037.stdout:session_autoclose 300 2024-03-29T08:30:00.432 INFO:teuthology.orchestra.run.smithi037.stdout:max_file_size 1099511627776 2024-03-29T08:30:00.432 INFO:teuthology.orchestra.run.smithi037.stdout:max_xattr_size 65536 2024-03-29T08:30:00.432 INFO:teuthology.orchestra.run.smithi037.stdout:required_client_features {} 2024-03-29T08:30:00.432 INFO:teuthology.orchestra.run.smithi037.stdout:last_failure 0 2024-03-29T08:30:00.432 INFO:teuthology.orchestra.run.smithi037.stdout:last_failure_osd_epoch 82 2024-03-29T08:30:00.433 INFO:teuthology.orchestra.run.smithi037.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T08:30:00.433 INFO:teuthology.orchestra.run.smithi037.stdout:max_mds 2 2024-03-29T08:30:00.433 INFO:teuthology.orchestra.run.smithi037.stdout:in 0,1 2024-03-29T08:30:00.433 INFO:teuthology.orchestra.run.smithi037.stdout:up {} 2024-03-29T08:30:00.433 INFO:teuthology.orchestra.run.smithi037.stdout:failed 0,1 2024-03-29T08:30:00.433 INFO:teuthology.orchestra.run.smithi037.stdout:damaged 2024-03-29T08:30:00.433 INFO:teuthology.orchestra.run.smithi037.stdout:stopped 2024-03-29T08:30:00.433 INFO:teuthology.orchestra.run.smithi037.stdout:data_pools [3] 2024-03-29T08:30:00.433 INFO:teuthology.orchestra.run.smithi037.stdout:metadata_pool 2 2024-03-29T08:30:00.433 INFO:teuthology.orchestra.run.smithi037.stdout:inline_data disabled 2024-03-29T08:30:00.433 INFO:teuthology.orchestra.run.smithi037.stdout:balancer 2024-03-29T08:30:00.433 INFO:teuthology.orchestra.run.smithi037.stdout:bal_rank_mask -1 2024-03-29T08:30:00.434 INFO:teuthology.orchestra.run.smithi037.stdout:standby_count_wanted 1 2024-03-29T08:30:00.434 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:30:00.434 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:30:00.434 INFO:teuthology.orchestra.run.smithi037.stdout:Standby daemons: 2024-03-29T08:30:00.434 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:30:00.434 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi148.qukpjh{-1:24319} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.148:6826/2641327912,v1:172.21.15.148:6827/2641327912] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:30:00.434 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi037.mmwgxe{-1:34440} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.37:6828/1248142005,v1:172.21.15.37:6829/1248142005] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:30:00.434 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi037.iifsmh{-1:34444} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.37:6826/2060882959,v1:172.21.15.37:6827/2060882959] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:30:00.434 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi148.sqteih{-1:44395} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.148:6824/4078652242,v1:172.21.15.148:6825/4078652242] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:30:00.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:00.433+0000 7fc894c00700 1 -- 172.21.15.37:0/4088085575 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fc87406cc30 msgr2=0x7fc87406f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:00.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:00.433+0000 7fc894c00700 1 --2- 172.21.15.37:0/4088085575 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fc87406cc30 0x7fc87406f0e0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fc88000b840 tx=0x7fc880009e30 comp rx=0 tx=0).stop 2024-03-29T08:30:00.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:00.433+0000 7fc894c00700 1 -- 172.21.15.37:0/4088085575 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc8901037a0 msgr2=0x7fc890114060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:00.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:00.433+0000 7fc894c00700 1 --2- 172.21.15.37:0/4088085575 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc8901037a0 0x7fc890114060 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fc87c002a50 tx=0x7fc87c004600 comp rx=0 tx=0).stop 2024-03-29T08:30:00.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:00.433+0000 7fc894c00700 1 -- 172.21.15.37:0/4088085575 shutdown_connections 2024-03-29T08:30:00.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:00.433+0000 7fc894c00700 1 --2- 172.21.15.37:0/4088085575 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fc87406cc30 0x7fc87406f0e0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:00.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:00.433+0000 7fc894c00700 1 --2- 172.21.15.37:0/4088085575 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc890102e00 0x7fc890113b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:00.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:00.433+0000 7fc894c00700 1 --2- 172.21.15.37:0/4088085575 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc8901037a0 0x7fc890114060 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:00.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:00.433+0000 7fc894c00700 1 -- 172.21.15.37:0/4088085575 >> 172.21.15.37:0/4088085575 conn(0x7fc8900f89b0 msgr2=0x7fc8900f9a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:30:00.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:00.434+0000 7fc894c00700 1 -- 172.21.15.37:0/4088085575 shutdown_connections 2024-03-29T08:30:00.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:00.434+0000 7fc894c00700 1 -- 172.21.15.37:0/4088085575 wait complete. 2024-03-29T08:30:00.436 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 28 2024-03-29T08:30:00.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:00.572+0000 7f3941200700 1 -- 172.21.15.37:0/1533559689 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f393c1037a0 msgr2=0x7f393c0fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:00.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:00.572+0000 7f3941200700 1 --2- 172.21.15.37:0/1533559689 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f393c1037a0 0x7f393c0fce10 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f3928004940 tx=0x7f3928009e30 comp rx=0 tx=0).stop 2024-03-29T08:30:00.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:00.572+0000 7f3941200700 1 -- 172.21.15.37:0/1533559689 shutdown_connections 2024-03-29T08:30:00.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:00.572+0000 7f3941200700 1 --2- 172.21.15.37:0/1533559689 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f393c1037a0 0x7f393c0fce10 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:00.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:00.572+0000 7f3941200700 1 --2- 172.21.15.37:0/1533559689 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f393c102e00 0x7f393c1031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:00.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:00.572+0000 7f3941200700 1 -- 172.21.15.37:0/1533559689 >> 172.21.15.37:0/1533559689 conn(0x7f393c0f89b0 msgr2=0x7f393c0fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:30:00.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:00.573+0000 7f3941200700 1 -- 172.21.15.37:0/1533559689 shutdown_connections 2024-03-29T08:30:00.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:00.573+0000 7f3941200700 1 -- 172.21.15.37:0/1533559689 wait complete. 2024-03-29T08:30:00.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:00.573+0000 7f3941200700 1 Processor -- start 2024-03-29T08:30:00.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:00.574+0000 7f3941200700 1 -- start start 2024-03-29T08:30:00.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:00.574+0000 7f3941200700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f393c102e00 0x7f393c115cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:00.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:00.574+0000 7f3941200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f393c1037a0 0x7f393c116200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:00.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:00.574+0000 7f3941200700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f393c1168e0 con 0x7f393c1037a0 2024-03-29T08:30:00.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:00.574+0000 7f3941200700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f393c10fc20 con 0x7f393c102e00 2024-03-29T08:30:00.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:00.574+0000 7f3939400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f393c1037a0 0x7f393c116200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:00.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:00.574+0000 7f3939400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f393c1037a0 0x7f393c116200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43182/0 (socket says 172.21.15.37:43182) 2024-03-29T08:30:00.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:00.574+0000 7f3939400700 1 -- 172.21.15.37:0/812882739 learned_addr learned my addr 172.21.15.37:0/812882739 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:30:00.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:00.574+0000 7f3939e00700 1 --2- 172.21.15.37:0/812882739 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f393c102e00 0x7f393c115cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:00.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:00.575+0000 7f3939400700 1 -- 172.21.15.37:0/812882739 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f393c102e00 msgr2=0x7f393c115cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:00.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:00.575+0000 7f3939400700 1 --2- 172.21.15.37:0/812882739 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f393c102e00 0x7f393c115cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:00.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:00.575+0000 7f3939400700 1 -- 172.21.15.37:0/812882739 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3928005040 con 0x7f393c1037a0 2024-03-29T08:30:00.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:00.575+0000 7f3939400700 1 --2- 172.21.15.37:0/812882739 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f393c1037a0 0x7f393c116200 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f3928009390 tx=0x7f3928009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:00.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:00.575+0000 7f3932000700 1 -- 172.21.15.37:0/812882739 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f392800e040 con 0x7f393c1037a0 2024-03-29T08:30:00.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:00.575+0000 7f3932000700 1 -- 172.21.15.37:0/812882739 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f3928013420 con 0x7f393c1037a0 2024-03-29T08:30:00.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:00.575+0000 7f3932000700 1 -- 172.21.15.37:0/812882739 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f392801c630 con 0x7f393c1037a0 2024-03-29T08:30:00.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:00.575+0000 7f3941200700 1 -- 172.21.15.37:0/812882739 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f393c10fe40 con 0x7f393c1037a0 2024-03-29T08:30:00.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:00.575+0000 7f3941200700 1 -- 172.21.15.37:0/812882739 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f393c110400 con 0x7f393c1037a0 2024-03-29T08:30:00.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:00.576+0000 7f3941200700 1 -- 172.21.15.37:0/812882739 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f393c0fe550 con 0x7f393c1037a0 2024-03-29T08:30:00.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:00.577+0000 7f3932000700 1 -- 172.21.15.37:0/812882739 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f3928013a00 con 0x7f393c1037a0 2024-03-29T08:30:00.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:00.578+0000 7f3932000700 1 --2- 172.21.15.37:0/812882739 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f392006cc30 0x7f392006f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:00.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:00.578+0000 7f3932000700 1 -- 172.21.15.37:0/812882739 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f392808d0e0 con 0x7f393c1037a0 2024-03-29T08:30:00.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:00.579+0000 7f3939e00700 1 --2- 172.21.15.37:0/812882739 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f392006cc30 0x7f392006f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:00.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:00.579+0000 7f3939e00700 1 --2- 172.21.15.37:0/812882739 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f392006cc30 0x7f392006f0e0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f392c009bf0 tx=0x7f392c0091a0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:00.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:00.583+0000 7f3932000700 1 -- 172.21.15.37:0/812882739 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f392805abb0 con 0x7f393c1037a0 2024-03-29T08:30:00.754 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:30:00 smithi148 ceph-mon[106692]: from='client.35064 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:30:00.754 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:30:00 smithi148 ceph-mon[106692]: from='client.35068 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:30:00.754 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:30:00 smithi148 ceph-mon[106692]: from='client.35072 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:30:00.755 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:30:00 smithi148 ceph-mon[106692]: overall HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-03-29T08:30:00.755 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:30:00 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/25629078' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:30:00.755 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:30:00 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/4088085575' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T08:30:00.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:00.814+0000 7f3941200700 1 -- 172.21.15.37:0/812882739 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f393c04bed0 con 0x7f392006cc30 2024-03-29T08:30:00.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:00.816+0000 7f3932000700 1 -- 172.21.15.37:0/812882739 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f393c04bed0 con 0x7f392006cc30 2024-03-29T08:30:00.816 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-29T08:30:00.816 INFO:teuthology.orchestra.run.smithi037.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59", 2024-03-29T08:30:00.816 INFO:teuthology.orchestra.run.smithi037.stdout: "in_progress": true, 2024-03-29T08:30:00.816 INFO:teuthology.orchestra.run.smithi037.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-29T08:30:00.816 INFO:teuthology.orchestra.run.smithi037.stdout: "services_complete": [ 2024-03-29T08:30:00.816 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr", 2024-03-29T08:30:00.816 INFO:teuthology.orchestra.run.smithi037.stdout: "mon", 2024-03-29T08:30:00.817 INFO:teuthology.orchestra.run.smithi037.stdout: "crash", 2024-03-29T08:30:00.817 INFO:teuthology.orchestra.run.smithi037.stdout: "osd" 2024-03-29T08:30:00.817 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-29T08:30:00.817 INFO:teuthology.orchestra.run.smithi037.stdout: "progress": "15/21 daemons upgraded", 2024-03-29T08:30:00.817 INFO:teuthology.orchestra.run.smithi037.stdout: "message": "Currently upgrading mds daemons", 2024-03-29T08:30:00.817 INFO:teuthology.orchestra.run.smithi037.stdout: "is_paused": false 2024-03-29T08:30:00.817 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-29T08:30:00.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:00.818+0000 7f3941200700 1 -- 172.21.15.37:0/812882739 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f392006cc30 msgr2=0x7f392006f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:00.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:00.818+0000 7f3941200700 1 --2- 172.21.15.37:0/812882739 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f392006cc30 0x7f392006f0e0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f392c009bf0 tx=0x7f392c0091a0 comp rx=0 tx=0).stop 2024-03-29T08:30:00.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:00.818+0000 7f3941200700 1 -- 172.21.15.37:0/812882739 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f393c1037a0 msgr2=0x7f393c116200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:00.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:00.818+0000 7f3941200700 1 --2- 172.21.15.37:0/812882739 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f393c1037a0 0x7f393c116200 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f3928009390 tx=0x7f3928009e30 comp rx=0 tx=0).stop 2024-03-29T08:30:00.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:00.818+0000 7f3941200700 1 -- 172.21.15.37:0/812882739 shutdown_connections 2024-03-29T08:30:00.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:00.818+0000 7f3941200700 1 --2- 172.21.15.37:0/812882739 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f392006cc30 0x7f392006f0e0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:00.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:00.818+0000 7f3941200700 1 --2- 172.21.15.37:0/812882739 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f393c102e00 0x7f393c115cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:00.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:00.818+0000 7f3941200700 1 --2- 172.21.15.37:0/812882739 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f393c1037a0 0x7f393c116200 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:00.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:00.818+0000 7f3941200700 1 -- 172.21.15.37:0/812882739 >> 172.21.15.37:0/812882739 conn(0x7f393c0f89b0 msgr2=0x7f393c0f98d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:30:00.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:00.818+0000 7f3941200700 1 -- 172.21.15.37:0/812882739 shutdown_connections 2024-03-29T08:30:00.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:00.818+0000 7f3941200700 1 -- 172.21.15.37:0/812882739 wait complete. 2024-03-29T08:30:00.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:30:00 smithi037 ceph-mon[142325]: from='client.35064 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:30:00.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:30:00 smithi037 ceph-mon[142325]: from='client.35068 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:30:00.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:30:00 smithi037 ceph-mon[142325]: from='client.35072 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:30:00.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:30:00 smithi037 ceph-mon[142325]: overall HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-03-29T08:30:00.928 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:30:00 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/25629078' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:30:00.928 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:30:00 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/4088085575' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T08:30:00.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:00.956+0000 7fe15da00700 1 -- 172.21.15.37:0/3201994832 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe158070c70 msgr2=0x7fe158109110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:00.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:00.956+0000 7fe15da00700 1 --2- 172.21.15.37:0/3201994832 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe158070c70 0x7fe158109110 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fe144004970 tx=0x7fe144009e30 comp rx=0 tx=0).stop 2024-03-29T08:30:00.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:00.956+0000 7fe15da00700 1 -- 172.21.15.37:0/3201994832 shutdown_connections 2024-03-29T08:30:00.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:00.956+0000 7fe15da00700 1 --2- 172.21.15.37:0/3201994832 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe158070c70 0x7fe158109110 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:00.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:00.956+0000 7fe15da00700 1 --2- 172.21.15.37:0/3201994832 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe158070360 0x7fe158070730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:00.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:00.956+0000 7fe15da00700 1 -- 172.21.15.37:0/3201994832 >> 172.21.15.37:0/3201994832 conn(0x7fe1580f89b0 msgr2=0x7fe1580fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:30:00.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:00.956+0000 7fe15da00700 1 -- 172.21.15.37:0/3201994832 shutdown_connections 2024-03-29T08:30:00.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:00.956+0000 7fe15da00700 1 -- 172.21.15.37:0/3201994832 wait complete. 2024-03-29T08:30:00.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:00.957+0000 7fe15da00700 1 Processor -- start 2024-03-29T08:30:00.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:00.957+0000 7fe15da00700 1 -- start start 2024-03-29T08:30:00.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:00.958+0000 7fe15da00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe158070360 0x7fe158115ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:00.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:00.958+0000 7fe15da00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe158070c70 0x7fe1581161e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:00.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:00.958+0000 7fe15da00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe158116830 con 0x7fe158070360 2024-03-29T08:30:00.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:00.958+0000 7fe15da00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe15810fc00 con 0x7fe158070c70 2024-03-29T08:30:00.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:00.958+0000 7fe156a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe158070360 0x7fe158115ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:00.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:00.958+0000 7fe156a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe158070360 0x7fe158115ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43194/0 (socket says 172.21.15.37:43194) 2024-03-29T08:30:00.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:00.958+0000 7fe156a00700 1 -- 172.21.15.37:0/1690388374 learned_addr learned my addr 172.21.15.37:0/1690388374 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:30:00.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:00.958+0000 7fe156000700 1 --2- 172.21.15.37:0/1690388374 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe158070c70 0x7fe1581161e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:00.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:00.959+0000 7fe156a00700 1 -- 172.21.15.37:0/1690388374 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe158070c70 msgr2=0x7fe1581161e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:00.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:00.959+0000 7fe156a00700 1 --2- 172.21.15.37:0/1690388374 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe158070c70 0x7fe1581161e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:00.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:00.959+0000 7fe156a00700 1 -- 172.21.15.37:0/1690388374 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe144005040 con 0x7fe158070360 2024-03-29T08:30:00.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:00.959+0000 7fe156a00700 1 --2- 172.21.15.37:0/1690388374 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe158070360 0x7fe158115ca0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fe148002a50 tx=0x7fe1480046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:00.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:00.959+0000 7fe14ea00700 1 -- 172.21.15.37:0/1690388374 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe148007cc0 con 0x7fe158070360 2024-03-29T08:30:00.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:00.959+0000 7fe14ea00700 1 -- 172.21.15.37:0/1690388374 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fe14800f040 con 0x7fe158070360 2024-03-29T08:30:00.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:00.959+0000 7fe15da00700 1 -- 172.21.15.37:0/1690388374 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe15810fee0 con 0x7fe158070360 2024-03-29T08:30:00.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:00.959+0000 7fe14ea00700 1 -- 172.21.15.37:0/1690388374 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe14800d9c0 con 0x7fe158070360 2024-03-29T08:30:00.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:00.959+0000 7fe15da00700 1 -- 172.21.15.37:0/1690388374 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1581103f0 con 0x7fe158070360 2024-03-29T08:30:00.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:00.960+0000 7fe15da00700 1 -- 172.21.15.37:0/1690388374 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe158103bd0 con 0x7fe158070360 2024-03-29T08:30:00.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:00.962+0000 7fe14ea00700 1 -- 172.21.15.37:0/1690388374 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fe14800db20 con 0x7fe158070360 2024-03-29T08:30:00.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:00.963+0000 7fe14ea00700 1 --2- 172.21.15.37:0/1690388374 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fe13c06cc30 0x7fe13c06f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:00.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:00.963+0000 7fe14ea00700 1 -- 172.21.15.37:0/1690388374 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fe14808bde0 con 0x7fe158070360 2024-03-29T08:30:00.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:00.963+0000 7fe156000700 1 --2- 172.21.15.37:0/1690388374 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fe13c06cc30 0x7fe13c06f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:00.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:00.964+0000 7fe156000700 1 --2- 172.21.15.37:0/1690388374 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fe13c06cc30 0x7fe13c06f0e0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fe14400b370 tx=0x7fe144009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:00.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:00.967+0000 7fe14ea00700 1 -- 172.21.15.37:0/1690388374 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fe1480597b0 con 0x7fe158070360 2024-03-29T08:30:01.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:01.267+0000 7fe15da00700 1 -- 172.21.15.37:0/1690388374 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fe15810bfe0 con 0x7fe158070360 2024-03-29T08:30:01.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:01.267+0000 7fe14ea00700 1 -- 172.21.15.37:0/1690388374 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7fe148059340 con 0x7fe158070360 2024-03-29T08:30:01.268 INFO:teuthology.orchestra.run.smithi037.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-03-29T08:30:01.268 INFO:teuthology.orchestra.run.smithi037.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-03-29T08:30:01.268 INFO:teuthology.orchestra.run.smithi037.stdout: fs cephfs is degraded 2024-03-29T08:30:01.268 INFO:teuthology.orchestra.run.smithi037.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-03-29T08:30:01.268 INFO:teuthology.orchestra.run.smithi037.stdout: fs cephfs is offline because no MDS is active for it. 2024-03-29T08:30:01.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:01.270+0000 7fe15da00700 1 -- 172.21.15.37:0/1690388374 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fe13c06cc30 msgr2=0x7fe13c06f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:01.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:01.270+0000 7fe15da00700 1 --2- 172.21.15.37:0/1690388374 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fe13c06cc30 0x7fe13c06f0e0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fe14400b370 tx=0x7fe144009e30 comp rx=0 tx=0).stop 2024-03-29T08:30:01.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:01.270+0000 7fe15da00700 1 -- 172.21.15.37:0/1690388374 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe158070360 msgr2=0x7fe158115ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:01.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:01.270+0000 7fe15da00700 1 --2- 172.21.15.37:0/1690388374 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe158070360 0x7fe158115ca0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fe148002a50 tx=0x7fe1480046b0 comp rx=0 tx=0).stop 2024-03-29T08:30:01.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:01.270+0000 7fe15da00700 1 -- 172.21.15.37:0/1690388374 shutdown_connections 2024-03-29T08:30:01.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:01.270+0000 7fe15da00700 1 --2- 172.21.15.37:0/1690388374 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fe13c06cc30 0x7fe13c06f0e0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:01.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:01.270+0000 7fe15da00700 1 --2- 172.21.15.37:0/1690388374 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe158070360 0x7fe158115ca0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:01.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:01.270+0000 7fe15da00700 1 --2- 172.21.15.37:0/1690388374 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe158070c70 0x7fe1581161e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:01.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:01.270+0000 7fe15da00700 1 -- 172.21.15.37:0/1690388374 >> 172.21.15.37:0/1690388374 conn(0x7fe1580f89b0 msgr2=0x7fe1581015f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:30:01.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:01.270+0000 7fe15da00700 1 -- 172.21.15.37:0/1690388374 shutdown_connections 2024-03-29T08:30:01.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:01.270+0000 7fe15da00700 1 -- 172.21.15.37:0/1690388374 wait complete. 2024-03-29T08:30:01.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:30:01 smithi148 ceph-mon[106692]: pgmap v573: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:30:01.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:30:01 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/1690388374' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T08:30:01.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:30:01 smithi037 ceph-mon[142325]: pgmap v573: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:30:01.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:30:01 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/1690388374' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T08:30:02.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:30:02 smithi148 ceph-mon[106692]: from='client.35084 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:30:02.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:30:02 smithi037 ceph-mon[142325]: from='client.35084 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:30:03.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:30:03 smithi148 ceph-mon[106692]: pgmap v574: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:30:03.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:30:03 smithi037 ceph-mon[142325]: pgmap v574: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:30:05.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:30:05 smithi148 ceph-mon[106692]: pgmap v575: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:30:05.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:30:05 smithi037 ceph-mon[142325]: pgmap v575: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:30:07.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:30:07 smithi148 ceph-mon[106692]: pgmap v576: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:30:07.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:30:07 smithi037 ceph-mon[142325]: pgmap v576: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:30:09.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:30:09 smithi148 ceph-mon[106692]: pgmap v577: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:30:09.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:30:09 smithi037 ceph-mon[142325]: pgmap v577: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:30:10.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:30:10 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:30:11.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:30:10 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:30:11.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:30:11 smithi037 ceph-mon[142325]: pgmap v578: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:30:12.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:30:11 smithi148 ceph-mon[106692]: pgmap v578: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:30:13.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:30:13 smithi037 ceph-mon[142325]: pgmap v579: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:30:14.059 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:30:13 smithi148 ceph-mon[106692]: pgmap v579: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:30:15.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:30:15 smithi037 ceph-mon[142325]: pgmap v580: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:30:16.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:30:15 smithi148 ceph-mon[106692]: pgmap v580: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:30:17.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:30:17 smithi037 ceph-mon[142325]: pgmap v581: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:30:18.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:30:17 smithi148 ceph-mon[106692]: pgmap v581: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:30:19.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:30:19 smithi037 ceph-mon[142325]: pgmap v582: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:30:20.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:30:19 smithi148 ceph-mon[106692]: pgmap v582: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:30:21.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:30:21 smithi037 ceph-mon[142325]: pgmap v583: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:30:22.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:30:21 smithi148 ceph-mon[106692]: pgmap v583: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:30:23.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:30:23 smithi037 ceph-mon[142325]: pgmap v584: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:30:24.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:30:23 smithi148 ceph-mon[106692]: pgmap v584: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:30:25.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:30:25 smithi037 ceph-mon[142325]: pgmap v585: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:30:25.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:30:25 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:30:26.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:30:25 smithi148 ceph-mon[106692]: pgmap v585: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:30:26.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:30:25 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:30:27.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:30:27 smithi037 ceph-mon[142325]: pgmap v586: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:30:28.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:30:27 smithi148 ceph-mon[106692]: pgmap v586: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:30:29.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:30:29 smithi037 ceph-mon[142325]: pgmap v587: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:30:30.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:30:29 smithi148 ceph-mon[106692]: pgmap v587: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:30:31.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:31.423+0000 7f9d4d000700 1 -- 172.21.15.37:0/599420888 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9d480fde90 msgr2=0x7f9d480fe260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:31.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:31.423+0000 7f9d4d000700 1 --2- 172.21.15.37:0/599420888 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9d480fde90 0x7f9d480fe260 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f9d38004940 tx=0x7f9d38009e30 comp rx=0 tx=0).stop 2024-03-29T08:30:31.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:31.423+0000 7f9d4d000700 1 -- 172.21.15.37:0/599420888 shutdown_connections 2024-03-29T08:30:31.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:31.423+0000 7f9d4d000700 1 --2- 172.21.15.37:0/599420888 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9d480fe830 0x7f9d48106f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:31.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:31.423+0000 7f9d4d000700 1 --2- 172.21.15.37:0/599420888 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9d480fde90 0x7f9d480fe260 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:31.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:31.423+0000 7f9d4d000700 1 -- 172.21.15.37:0/599420888 >> 172.21.15.37:0/599420888 conn(0x7f9d48072bc0 msgr2=0x7f9d48072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:30:31.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:31.423+0000 7f9d4d000700 1 -- 172.21.15.37:0/599420888 shutdown_connections 2024-03-29T08:30:31.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:31.423+0000 7f9d4d000700 1 -- 172.21.15.37:0/599420888 wait complete. 2024-03-29T08:30:31.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:31.424+0000 7f9d4d000700 1 Processor -- start 2024-03-29T08:30:31.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:31.424+0000 7f9d4d000700 1 -- start start 2024-03-29T08:30:31.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:31.425+0000 7f9d4d000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9d480fde90 0x7f9d48115d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:31.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:31.425+0000 7f9d4d000700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9d480fe830 0x7f9d48116260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:31.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:31.425+0000 7f9d4d000700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d481168b0 con 0x7f9d480fde90 2024-03-29T08:30:31.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:31.425+0000 7f9d4d000700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d4810fc80 con 0x7f9d480fe830 2024-03-29T08:30:31.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:31.425+0000 7f9d45e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9d480fde90 0x7f9d48115d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:31.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:31.425+0000 7f9d45e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9d480fde90 0x7f9d48115d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40858/0 (socket says 172.21.15.37:40858) 2024-03-29T08:30:31.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:31.425+0000 7f9d45e00700 1 -- 172.21.15.37:0/3762400285 learned_addr learned my addr 172.21.15.37:0/3762400285 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:30:31.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:31.425+0000 7f9d45400700 1 --2- 172.21.15.37:0/3762400285 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9d480fe830 0x7f9d48116260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:31.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:31.425+0000 7f9d45e00700 1 -- 172.21.15.37:0/3762400285 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9d480fe830 msgr2=0x7f9d48116260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:31.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:31.425+0000 7f9d45e00700 1 --2- 172.21.15.37:0/3762400285 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9d480fe830 0x7f9d48116260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:31.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:31.425+0000 7f9d45e00700 1 -- 172.21.15.37:0/3762400285 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d38005040 con 0x7f9d480fde90 2024-03-29T08:30:31.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:31.425+0000 7f9d45400700 1 --2- 172.21.15.37:0/3762400285 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9d480fe830 0x7f9d48116260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:30:31.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:31.426+0000 7f9d45e00700 1 --2- 172.21.15.37:0/3762400285 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9d480fde90 0x7f9d48115d20 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f9d380093b0 tx=0x7f9d38009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:31.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:31.426+0000 7f9d3e000700 1 -- 172.21.15.37:0/3762400285 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9d3800e040 con 0x7f9d480fde90 2024-03-29T08:30:31.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:31.426+0000 7f9d3e000700 1 -- 172.21.15.37:0/3762400285 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f9d38004290 con 0x7f9d480fde90 2024-03-29T08:30:31.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:31.426+0000 7f9d4d000700 1 -- 172.21.15.37:0/3762400285 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d4810ff00 con 0x7f9d480fde90 2024-03-29T08:30:31.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:31.426+0000 7f9d3e000700 1 -- 172.21.15.37:0/3762400285 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9d3801c3f0 con 0x7f9d480fde90 2024-03-29T08:30:31.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:31.426+0000 7f9d4d000700 1 -- 172.21.15.37:0/3762400285 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d48110440 con 0x7f9d480fde90 2024-03-29T08:30:31.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:31.427+0000 7f9d4d000700 1 -- 172.21.15.37:0/3762400285 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d48106a20 con 0x7f9d480fde90 2024-03-29T08:30:31.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:31.428+0000 7f9d3e000700 1 -- 172.21.15.37:0/3762400285 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f9d38013790 con 0x7f9d480fde90 2024-03-29T08:30:31.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:31.429+0000 7f9d3e000700 1 --2- 172.21.15.37:0/3762400285 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f9d2c06cc30 0x7f9d2c06f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:31.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:31.429+0000 7f9d3e000700 1 -- 172.21.15.37:0/3762400285 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f9d3808d380 con 0x7f9d480fde90 2024-03-29T08:30:31.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:31.429+0000 7f9d45400700 1 --2- 172.21.15.37:0/3762400285 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f9d2c06cc30 0x7f9d2c06f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:31.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:31.430+0000 7f9d45400700 1 --2- 172.21.15.37:0/3762400285 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f9d2c06cc30 0x7f9d2c06f0e0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f9d34009fd0 tx=0x7f9d3400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:31.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:31.434+0000 7f9d3e000700 1 -- 172.21.15.37:0/3762400285 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f9d3805add0 con 0x7f9d480fde90 2024-03-29T08:30:31.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:31.669+0000 7f9d4d000700 1 -- 172.21.15.37:0/3762400285 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f9d48109e60 con 0x7f9d2c06cc30 2024-03-29T08:30:31.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:31.670+0000 7f9d3e000700 1 -- 172.21.15.37:0/3762400285 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f9d3400f040 con 0x7f9d2c06cc30 2024-03-29T08:30:31.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:31.673+0000 7f9d4d000700 1 -- 172.21.15.37:0/3762400285 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f9d2c06cc30 msgr2=0x7f9d2c06f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:31.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:31.673+0000 7f9d4d000700 1 --2- 172.21.15.37:0/3762400285 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f9d2c06cc30 0x7f9d2c06f0e0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f9d34009fd0 tx=0x7f9d3400b040 comp rx=0 tx=0).stop 2024-03-29T08:30:31.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:31.673+0000 7f9d4d000700 1 -- 172.21.15.37:0/3762400285 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9d480fde90 msgr2=0x7f9d48115d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:31.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:31.673+0000 7f9d4d000700 1 --2- 172.21.15.37:0/3762400285 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9d480fde90 0x7f9d48115d20 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f9d380093b0 tx=0x7f9d38009e30 comp rx=0 tx=0).stop 2024-03-29T08:30:31.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:31.673+0000 7f9d4d000700 1 -- 172.21.15.37:0/3762400285 shutdown_connections 2024-03-29T08:30:31.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:31.673+0000 7f9d4d000700 1 --2- 172.21.15.37:0/3762400285 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f9d2c06cc30 0x7f9d2c06f0e0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:31.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:31.673+0000 7f9d4d000700 1 --2- 172.21.15.37:0/3762400285 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9d480fde90 0x7f9d48115d20 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:31.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:31.673+0000 7f9d4d000700 1 --2- 172.21.15.37:0/3762400285 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9d480fe830 0x7f9d48116260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:31.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:31.673+0000 7f9d4d000700 1 -- 172.21.15.37:0/3762400285 >> 172.21.15.37:0/3762400285 conn(0x7f9d48072bc0 msgr2=0x7f9d480fb8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:30:31.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:31.673+0000 7f9d4d000700 1 -- 172.21.15.37:0/3762400285 shutdown_connections 2024-03-29T08:30:31.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:31.673+0000 7f9d4d000700 1 -- 172.21.15.37:0/3762400285 wait complete. 2024-03-29T08:30:31.688 INFO:teuthology.orchestra.run.smithi037.stdout:true 2024-03-29T08:30:31.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:31.811+0000 7f4fbdc00700 1 -- 172.21.15.37:0/166056614 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4fb8102e00 msgr2=0x7f4fb81031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:31.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:31.811+0000 7f4fbdc00700 1 --2- 172.21.15.37:0/166056614 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4fb8102e00 0x7f4fb81031d0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f4fa8004940 tx=0x7f4fa8009e30 comp rx=0 tx=0).stop 2024-03-29T08:30:31.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:31.812+0000 7f4fbdc00700 1 -- 172.21.15.37:0/166056614 shutdown_connections 2024-03-29T08:30:31.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:31.812+0000 7f4fbdc00700 1 --2- 172.21.15.37:0/166056614 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4fb81037a0 0x7f4fb80fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:31.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:31.812+0000 7f4fbdc00700 1 --2- 172.21.15.37:0/166056614 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4fb8102e00 0x7f4fb81031d0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:31.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:31.812+0000 7f4fbdc00700 1 -- 172.21.15.37:0/166056614 >> 172.21.15.37:0/166056614 conn(0x7f4fb80f89b0 msgr2=0x7f4fb80fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:30:31.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:31.812+0000 7f4fbdc00700 1 -- 172.21.15.37:0/166056614 shutdown_connections 2024-03-29T08:30:31.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:31.812+0000 7f4fbdc00700 1 -- 172.21.15.37:0/166056614 wait complete. 2024-03-29T08:30:31.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:31.812+0000 7f4fbdc00700 1 Processor -- start 2024-03-29T08:30:31.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:31.813+0000 7f4fbdc00700 1 -- start start 2024-03-29T08:30:31.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:31.813+0000 7f4fbdc00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4fb8102e00 0x7f4fb80702c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:31.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:31.813+0000 7f4fbdc00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4fb81037a0 0x7f4fb8070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:31.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:31.813+0000 7f4fbdc00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fb8070d40 con 0x7f4fb8102e00 2024-03-29T08:30:31.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:31.813+0000 7f4fbdc00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fb8070eb0 con 0x7f4fb81037a0 2024-03-29T08:30:31.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:31.813+0000 7f4fb6a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4fb8102e00 0x7f4fb80702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:31.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:31.813+0000 7f4fb6a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4fb8102e00 0x7f4fb80702c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40872/0 (socket says 172.21.15.37:40872) 2024-03-29T08:30:31.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:31.813+0000 7f4fb6a00700 1 -- 172.21.15.37:0/3796822056 learned_addr learned my addr 172.21.15.37:0/3796822056 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:30:31.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:31.814+0000 7f4fb6000700 1 --2- 172.21.15.37:0/3796822056 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4fb81037a0 0x7f4fb8070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:31.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:31.814+0000 7f4fb6a00700 1 -- 172.21.15.37:0/3796822056 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4fb81037a0 msgr2=0x7f4fb8070800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:31.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:31.814+0000 7f4fb6a00700 1 --2- 172.21.15.37:0/3796822056 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4fb81037a0 0x7f4fb8070800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:31.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:31.814+0000 7f4fb6a00700 1 -- 172.21.15.37:0/3796822056 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fa8005040 con 0x7f4fb8102e00 2024-03-29T08:30:31.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:31.814+0000 7f4fb6000700 1 --2- 172.21.15.37:0/3796822056 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4fb81037a0 0x7f4fb8070800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:30:31.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:31.814+0000 7f4fb6a00700 1 --2- 172.21.15.37:0/3796822056 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4fb8102e00 0x7f4fb80702c0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f4fa8009390 tx=0x7f4fa8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:31.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:31.814+0000 7f4faea00700 1 -- 172.21.15.37:0/3796822056 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4fa800e040 con 0x7f4fb8102e00 2024-03-29T08:30:31.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:31.814+0000 7f4fbdc00700 1 -- 172.21.15.37:0/3796822056 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fb8071130 con 0x7f4fb8102e00 2024-03-29T08:30:31.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:31.814+0000 7f4faea00700 1 -- 172.21.15.37:0/3796822056 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f4fa8013420 con 0x7f4fb8102e00 2024-03-29T08:30:31.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:31.814+0000 7f4faea00700 1 -- 172.21.15.37:0/3796822056 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4fa801c630 con 0x7f4fb8102e00 2024-03-29T08:30:31.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:31.814+0000 7f4fbdc00700 1 -- 172.21.15.37:0/3796822056 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fb811a1d0 con 0x7f4fb8102e00 2024-03-29T08:30:31.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:31.816+0000 7f4fbdc00700 1 -- 172.21.15.37:0/3796822056 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fb80fe550 con 0x7f4fb8102e00 2024-03-29T08:30:31.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:31.816+0000 7f4faea00700 1 -- 172.21.15.37:0/3796822056 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f4fa80139e0 con 0x7f4fb8102e00 2024-03-29T08:30:31.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:31.817+0000 7f4faea00700 1 --2- 172.21.15.37:0/3796822056 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f4f9c06cc30 0x7f4f9c06f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:31.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:31.817+0000 7f4faea00700 1 -- 172.21.15.37:0/3796822056 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f4fa808d090 con 0x7f4fb8102e00 2024-03-29T08:30:31.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:31.818+0000 7f4fb6000700 1 --2- 172.21.15.37:0/3796822056 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f4f9c06cc30 0x7f4f9c06f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:31.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:31.818+0000 7f4fb6000700 1 --2- 172.21.15.37:0/3796822056 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f4f9c06cc30 0x7f4f9c06f0e0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f4fa4009fd0 tx=0x7f4fa400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:31.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:31.822+0000 7f4faea00700 1 -- 172.21.15.37:0/3796822056 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f4fa80570d0 con 0x7f4fb8102e00 2024-03-29T08:30:31.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:30:31 smithi037 ceph-mon[142325]: pgmap v588: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:30:32.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:32.054+0000 7f4fbdc00700 1 -- 172.21.15.37:0/3796822056 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4fb8071130 con 0x7f4f9c06cc30 2024-03-29T08:30:32.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:32.056+0000 7f4faea00700 1 -- 172.21.15.37:0/3796822056 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f4fb8071130 con 0x7f4f9c06cc30 2024-03-29T08:30:32.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:30:31 smithi148 ceph-mon[106692]: pgmap v588: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:30:32.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:32.058+0000 7f4fbdc00700 1 -- 172.21.15.37:0/3796822056 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f4f9c06cc30 msgr2=0x7f4f9c06f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:32.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:32.058+0000 7f4fbdc00700 1 --2- 172.21.15.37:0/3796822056 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f4f9c06cc30 0x7f4f9c06f0e0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f4fa4009fd0 tx=0x7f4fa400b040 comp rx=0 tx=0).stop 2024-03-29T08:30:32.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:32.058+0000 7f4fbdc00700 1 -- 172.21.15.37:0/3796822056 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4fb8102e00 msgr2=0x7f4fb80702c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:32.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:32.058+0000 7f4fbdc00700 1 --2- 172.21.15.37:0/3796822056 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4fb8102e00 0x7f4fb80702c0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f4fa8009390 tx=0x7f4fa8009e30 comp rx=0 tx=0).stop 2024-03-29T08:30:32.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:32.058+0000 7f4fbdc00700 1 -- 172.21.15.37:0/3796822056 shutdown_connections 2024-03-29T08:30:32.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:32.058+0000 7f4fbdc00700 1 --2- 172.21.15.37:0/3796822056 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f4f9c06cc30 0x7f4f9c06f0e0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:32.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:32.058+0000 7f4fbdc00700 1 --2- 172.21.15.37:0/3796822056 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4fb8102e00 0x7f4fb80702c0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:32.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:32.058+0000 7f4fbdc00700 1 --2- 172.21.15.37:0/3796822056 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4fb81037a0 0x7f4fb8070800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:32.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:32.058+0000 7f4fbdc00700 1 -- 172.21.15.37:0/3796822056 >> 172.21.15.37:0/3796822056 conn(0x7f4fb80f89b0 msgr2=0x7f4fb80f9a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:30:32.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:32.058+0000 7f4fbdc00700 1 -- 172.21.15.37:0/3796822056 shutdown_connections 2024-03-29T08:30:32.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:32.058+0000 7f4fbdc00700 1 -- 172.21.15.37:0/3796822056 wait complete. 2024-03-29T08:30:32.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:32.195+0000 7faa2fe00700 1 -- 172.21.15.37:0/2263361775 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faa30065630 msgr2=0x7faa30065a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:32.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:32.195+0000 7faa2fe00700 1 --2- 172.21.15.37:0/2263361775 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faa30065630 0x7faa30065a80 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7faa1c004940 tx=0x7faa1c009e30 comp rx=0 tx=0).stop 2024-03-29T08:30:32.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:32.195+0000 7faa2fe00700 1 -- 172.21.15.37:0/2263361775 shutdown_connections 2024-03-29T08:30:32.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:32.195+0000 7faa2fe00700 1 --2- 172.21.15.37:0/2263361775 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faa30065630 0x7faa30065a80 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:32.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:32.195+0000 7faa2fe00700 1 --2- 172.21.15.37:0/2263361775 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faa30102ca0 0x7faa30103070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:32.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:32.195+0000 7faa2fe00700 1 -- 172.21.15.37:0/2263361775 >> 172.21.15.37:0/2263361775 conn(0x7faa30071b70 msgr2=0x7faa30071f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:30:32.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:32.195+0000 7faa2fe00700 1 -- 172.21.15.37:0/2263361775 shutdown_connections 2024-03-29T08:30:32.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:32.195+0000 7faa2fe00700 1 -- 172.21.15.37:0/2263361775 wait complete. 2024-03-29T08:30:32.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:32.196+0000 7faa2fe00700 1 Processor -- start 2024-03-29T08:30:32.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:32.197+0000 7faa2fe00700 1 -- start start 2024-03-29T08:30:32.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:32.197+0000 7faa2fe00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faa30065630 0x7faa30115b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:32.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:32.197+0000 7faa2fe00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faa30102ca0 0x7faa30116070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:32.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:32.197+0000 7faa2fe00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa301166c0 con 0x7faa30065630 2024-03-29T08:30:32.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:32.197+0000 7faa2fe00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa3010fab0 con 0x7faa30102ca0 2024-03-29T08:30:32.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:32.197+0000 7faa2ea00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faa30065630 0x7faa30115b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:32.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:32.198+0000 7faa2ea00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faa30065630 0x7faa30115b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40882/0 (socket says 172.21.15.37:40882) 2024-03-29T08:30:32.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:32.198+0000 7faa2ea00700 1 -- 172.21.15.37:0/3872731990 learned_addr learned my addr 172.21.15.37:0/3872731990 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:30:32.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:32.198+0000 7faa2e000700 1 --2- 172.21.15.37:0/3872731990 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faa30102ca0 0x7faa30116070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:32.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:32.198+0000 7faa2ea00700 1 -- 172.21.15.37:0/3872731990 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faa30102ca0 msgr2=0x7faa30116070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:32.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:32.198+0000 7faa2ea00700 1 --2- 172.21.15.37:0/3872731990 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faa30102ca0 0x7faa30116070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:32.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:32.198+0000 7faa2ea00700 1 -- 172.21.15.37:0/3872731990 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa1c005040 con 0x7faa30065630 2024-03-29T08:30:32.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:32.198+0000 7faa2ea00700 1 --2- 172.21.15.37:0/3872731990 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faa30065630 0x7faa30115b30 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7faa20002a50 tx=0x7faa200046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:32.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:32.198+0000 7faa26a00700 1 -- 172.21.15.37:0/3872731990 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faa20007cc0 con 0x7faa30065630 2024-03-29T08:30:32.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:32.198+0000 7faa26a00700 1 -- 172.21.15.37:0/3872731990 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7faa2000f040 con 0x7faa30065630 2024-03-29T08:30:32.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:32.198+0000 7faa26a00700 1 -- 172.21.15.37:0/3872731990 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faa2000d9c0 con 0x7faa30065630 2024-03-29T08:30:32.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:32.199+0000 7faa2fe00700 1 -- 172.21.15.37:0/3872731990 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa3010fd90 con 0x7faa30065630 2024-03-29T08:30:32.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:32.199+0000 7faa2fe00700 1 -- 172.21.15.37:0/3872731990 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa301102d0 con 0x7faa30065630 2024-03-29T08:30:32.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:32.200+0000 7faa2fe00700 1 -- 172.21.15.37:0/3872731990 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa30106840 con 0x7faa30065630 2024-03-29T08:30:32.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:32.201+0000 7faa26a00700 1 -- 172.21.15.37:0/3872731990 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7faa2000db20 con 0x7faa30065630 2024-03-29T08:30:32.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:32.202+0000 7faa26a00700 1 --2- 172.21.15.37:0/3872731990 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7faa1406cc30 0x7faa1406f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:32.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:32.202+0000 7faa26a00700 1 -- 172.21.15.37:0/3872731990 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7faa2008bd20 con 0x7faa30065630 2024-03-29T08:30:32.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:32.202+0000 7faa2e000700 1 --2- 172.21.15.37:0/3872731990 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7faa1406cc30 0x7faa1406f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:32.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:32.203+0000 7faa2e000700 1 --2- 172.21.15.37:0/3872731990 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7faa1406cc30 0x7faa1406f0e0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7faa30110c70 tx=0x7faa1c009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:32.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:32.206+0000 7faa26a00700 1 -- 172.21.15.37:0/3872731990 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7faa20059770 con 0x7faa30065630 2024-03-29T08:30:32.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:32.431+0000 7faa2fe00700 1 -- 172.21.15.37:0/3872731990 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7faa30111180 con 0x7faa1406cc30 2024-03-29T08:30:32.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:32.441+0000 7faa26a00700 1 -- 172.21.15.37:0/3872731990 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7faa30111180 con 0x7faa1406cc30 2024-03-29T08:30:32.441 INFO:teuthology.orchestra.run.smithi037.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-29T08:30:32.442 INFO:teuthology.orchestra.run.smithi037.stdout:alertmanager.smithi037 smithi037 *:9093,9094 running (20m) 12m ago 33m 14.9M - 0.25.0 c8568f914cd2 2f9f9746ed0f 2024-03-29T08:30:32.442 INFO:teuthology.orchestra.run.smithi037.stdout:crash.smithi037 smithi037 running (17m) 12m ago 33m 6647k - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 178d416a3186 2024-03-29T08:30:32.442 INFO:teuthology.orchestra.run.smithi037.stdout:crash.smithi148 smithi148 running (17m) 12m ago 32m 6656k - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 23666989d58f 2024-03-29T08:30:32.442 INFO:teuthology.orchestra.run.smithi037.stdout:grafana.smithi037 smithi037 *:3000 running (20m) 12m ago 33m 71.9M - 9.4.7 954c08fa6188 6b976fe49952 2024-03-29T08:30:32.442 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi037.iifsmh smithi037 running (12m) 12m ago 26m 10.8M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 2d97f562cc5b 2024-03-29T08:30:32.442 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi037.mmwgxe smithi037 running (13m) 12m ago 26m 10.9M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff ec37037f46e8 2024-03-29T08:30:32.442 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi148.qukpjh smithi148 running (26m) 12m ago 26m 15.3M - 17.2.7-789-g4213e17c cf22eb8683be 8e469782a273 2024-03-29T08:30:32.442 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi148.sqteih smithi148 running (12m) 12m ago 26m 10.7M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 44c15c77037a 2024-03-29T08:30:32.442 INFO:teuthology.orchestra.run.smithi037.stdout:mgr.smithi037.wjikii smithi037 *:8443,9283,8765 running (22m) 12m ago 34m 506M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff c5a1417b9417 2024-03-29T08:30:32.442 INFO:teuthology.orchestra.run.smithi037.stdout:mgr.smithi148.onhgvm smithi148 *:8443,9283,8765 running (21m) 12m ago 32m 450M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff fcfdbce062bd 2024-03-29T08:30:32.442 INFO:teuthology.orchestra.run.smithi037.stdout:mon.smithi037 smithi037 running (18m) 12m ago 35m 54.8M 2048M 18.2.2-1015-gd0e392d9 1cc2f45c17ff eea2060783af 2024-03-29T08:30:32.442 INFO:teuthology.orchestra.run.smithi037.stdout:mon.smithi148 smithi148 running (18m) 12m ago 32m 43.7M 2048M 18.2.2-1015-gd0e392d9 1cc2f45c17ff ad4b2c5748be 2024-03-29T08:30:32.442 INFO:teuthology.orchestra.run.smithi037.stdout:node-exporter.smithi037 smithi037 *:9100 running (21m) 12m ago 33m 7625k - 1.5.0 0da6a335fe13 26b8ae9cb878 2024-03-29T08:30:32.442 INFO:teuthology.orchestra.run.smithi037.stdout:node-exporter.smithi148 smithi148 *:9100 running (21m) 12m ago 32m 8166k - 1.5.0 0da6a335fe13 98414e4e64ed 2024-03-29T08:30:32.443 INFO:teuthology.orchestra.run.smithi037.stdout:osd.0 smithi037 running (16m) 12m ago 31m 104M 1619M 18.2.2-1015-gd0e392d9 1cc2f45c17ff c0c584f6245c 2024-03-29T08:30:32.443 INFO:teuthology.orchestra.run.smithi037.stdout:osd.1 smithi037 running (16m) 12m ago 30m 100M 1619M 18.2.2-1015-gd0e392d9 1cc2f45c17ff a4a2ff4cc83d 2024-03-29T08:30:32.443 INFO:teuthology.orchestra.run.smithi037.stdout:osd.2 smithi037 running (15m) 12m ago 30m 82.3M 1619M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 8b1bc71aa79e 2024-03-29T08:30:32.443 INFO:teuthology.orchestra.run.smithi037.stdout:osd.3 smithi148 running (14m) 12m ago 29m 105M 2642M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 1c656ab8ff86 2024-03-29T08:30:32.443 INFO:teuthology.orchestra.run.smithi037.stdout:osd.4 smithi148 running (14m) 12m ago 29m 97.3M 2642M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 966a9de11d22 2024-03-29T08:30:32.443 INFO:teuthology.orchestra.run.smithi037.stdout:osd.5 smithi148 running (13m) 12m ago 28m 85.8M 2642M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 474c4f9212ce 2024-03-29T08:30:32.443 INFO:teuthology.orchestra.run.smithi037.stdout:prometheus.smithi037 smithi037 *:9095 running (20m) 12m ago 32m 54.4M - 2.43.0 a07b618ecd1d 30c0223ec589 2024-03-29T08:30:32.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:32.444+0000 7faa2fe00700 1 -- 172.21.15.37:0/3872731990 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7faa1406cc30 msgr2=0x7faa1406f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:32.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:32.444+0000 7faa2fe00700 1 --2- 172.21.15.37:0/3872731990 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7faa1406cc30 0x7faa1406f0e0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7faa30110c70 tx=0x7faa1c009e30 comp rx=0 tx=0).stop 2024-03-29T08:30:32.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:32.444+0000 7faa2fe00700 1 -- 172.21.15.37:0/3872731990 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faa30065630 msgr2=0x7faa30115b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:32.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:32.444+0000 7faa2fe00700 1 --2- 172.21.15.37:0/3872731990 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faa30065630 0x7faa30115b30 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7faa20002a50 tx=0x7faa200046b0 comp rx=0 tx=0).stop 2024-03-29T08:30:32.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:32.444+0000 7faa2fe00700 1 -- 172.21.15.37:0/3872731990 shutdown_connections 2024-03-29T08:30:32.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:32.444+0000 7faa2fe00700 1 --2- 172.21.15.37:0/3872731990 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7faa1406cc30 0x7faa1406f0e0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:32.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:32.444+0000 7faa2fe00700 1 --2- 172.21.15.37:0/3872731990 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faa30065630 0x7faa30115b30 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:32.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:32.444+0000 7faa2fe00700 1 --2- 172.21.15.37:0/3872731990 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faa30102ca0 0x7faa30116070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:32.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:32.444+0000 7faa2fe00700 1 -- 172.21.15.37:0/3872731990 >> 172.21.15.37:0/3872731990 conn(0x7faa30071b70 msgr2=0x7faa301039c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:30:32.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:32.444+0000 7faa2fe00700 1 -- 172.21.15.37:0/3872731990 shutdown_connections 2024-03-29T08:30:32.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:32.444+0000 7faa2fe00700 1 -- 172.21.15.37:0/3872731990 wait complete. 2024-03-29T08:30:32.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:32.581+0000 7f552c800700 1 -- 172.21.15.37:0/1048282872 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5524102e00 msgr2=0x7f55241031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:32.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:32.581+0000 7f552c800700 1 --2- 172.21.15.37:0/1048282872 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5524102e00 0x7f55241031d0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f5518004940 tx=0x7f5518009e30 comp rx=0 tx=0).stop 2024-03-29T08:30:32.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:32.581+0000 7f552c800700 1 -- 172.21.15.37:0/1048282872 shutdown_connections 2024-03-29T08:30:32.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:32.581+0000 7f552c800700 1 --2- 172.21.15.37:0/1048282872 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f55241037a0 0x7f55240fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:32.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:32.581+0000 7f552c800700 1 --2- 172.21.15.37:0/1048282872 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5524102e00 0x7f55241031d0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:32.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:32.581+0000 7f552c800700 1 -- 172.21.15.37:0/1048282872 >> 172.21.15.37:0/1048282872 conn(0x7f55240f89b0 msgr2=0x7f55240fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:30:32.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:32.581+0000 7f552c800700 1 -- 172.21.15.37:0/1048282872 shutdown_connections 2024-03-29T08:30:32.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:32.581+0000 7f552c800700 1 -- 172.21.15.37:0/1048282872 wait complete. 2024-03-29T08:30:32.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:32.582+0000 7f552c800700 1 Processor -- start 2024-03-29T08:30:32.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:32.582+0000 7f552c800700 1 -- start start 2024-03-29T08:30:32.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:32.582+0000 7f552c800700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5524102e00 0x7f5524115cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:32.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:32.582+0000 7f552c800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f55241037a0 0x7f5524116230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:32.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:32.582+0000 7f552c800700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5524116880 con 0x7f55241037a0 2024-03-29T08:30:32.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:32.582+0000 7f552c800700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f552410fc50 con 0x7f5524102e00 2024-03-29T08:30:32.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:32.583+0000 7f5529400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f55241037a0 0x7f5524116230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:32.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:32.583+0000 7f5529400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f55241037a0 0x7f5524116230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40908/0 (socket says 172.21.15.37:40908) 2024-03-29T08:30:32.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:32.583+0000 7f5529400700 1 -- 172.21.15.37:0/3318573429 learned_addr learned my addr 172.21.15.37:0/3318573429 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:30:32.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:32.583+0000 7f5529e00700 1 --2- 172.21.15.37:0/3318573429 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5524102e00 0x7f5524115cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:32.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:32.583+0000 7f5529400700 1 -- 172.21.15.37:0/3318573429 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5524102e00 msgr2=0x7f5524115cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:32.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:32.583+0000 7f5529400700 1 --2- 172.21.15.37:0/3318573429 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5524102e00 0x7f5524115cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:32.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:32.583+0000 7f5529400700 1 -- 172.21.15.37:0/3318573429 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5518005040 con 0x7f55241037a0 2024-03-29T08:30:32.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:32.583+0000 7f5529400700 1 --2- 172.21.15.37:0/3318573429 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f55241037a0 0x7f5524116230 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f5514002a50 tx=0x7f5514004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:32.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:32.583+0000 7f551e000700 1 -- 172.21.15.37:0/3318573429 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5514007cc0 con 0x7f55241037a0 2024-03-29T08:30:32.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:32.584+0000 7f551e000700 1 -- 172.21.15.37:0/3318573429 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f551400f040 con 0x7f55241037a0 2024-03-29T08:30:32.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:32.584+0000 7f552c800700 1 -- 172.21.15.37:0/3318573429 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f552410ff30 con 0x7f55241037a0 2024-03-29T08:30:32.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:32.584+0000 7f551e000700 1 -- 172.21.15.37:0/3318573429 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f551400d9a0 con 0x7f55241037a0 2024-03-29T08:30:32.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:32.584+0000 7f552c800700 1 -- 172.21.15.37:0/3318573429 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55241104f0 con 0x7f55241037a0 2024-03-29T08:30:32.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:32.585+0000 7f552c800700 1 -- 172.21.15.37:0/3318573429 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55240fe550 con 0x7f55241037a0 2024-03-29T08:30:32.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:32.586+0000 7f551e000700 1 -- 172.21.15.37:0/3318573429 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f551400dc60 con 0x7f55241037a0 2024-03-29T08:30:32.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:32.587+0000 7f551e000700 1 --2- 172.21.15.37:0/3318573429 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f550c06cc30 0x7f550c06f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:32.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:32.587+0000 7f551e000700 1 -- 172.21.15.37:0/3318573429 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f551408bc90 con 0x7f55241037a0 2024-03-29T08:30:32.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:32.587+0000 7f5529e00700 1 --2- 172.21.15.37:0/3318573429 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f550c06cc30 0x7f550c06f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:32.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:32.587+0000 7f5529e00700 1 --2- 172.21.15.37:0/3318573429 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f550c06cc30 0x7f550c06f0e0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f551800b020 tx=0x7f5518009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:32.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:32.591+0000 7f551e000700 1 -- 172.21.15.37:0/3318573429 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f5514059660 con 0x7f55241037a0 2024-03-29T08:30:32.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:32.889+0000 7f552c800700 1 -- 172.21.15.37:0/3318573429 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f552404bed0 con 0x7f55241037a0 2024-03-29T08:30:32.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:32.890+0000 7f551e000700 1 -- 172.21.15.37:0/3318573429 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+847 (secure 0 0 0) 0x7f55140591f0 con 0x7f55241037a0 2024-03-29T08:30:32.890 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-29T08:30:32.890 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": { 2024-03-29T08:30:32.890 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 2 2024-03-29T08:30:32.890 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:30:32.890 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": { 2024-03-29T08:30:32.890 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 2 2024-03-29T08:30:32.891 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:30:32.891 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": { 2024-03-29T08:30:32.891 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 6 2024-03-29T08:30:32.891 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:30:32.891 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": { 2024-03-29T08:30:32.891 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 17.2.7-789-g4213e17c (4213e17c4de67016ed14f55d22503f4923b3a901) quincy (stable)": 1, 2024-03-29T08:30:32.891 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 3 2024-03-29T08:30:32.891 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:30:32.891 INFO:teuthology.orchestra.run.smithi037.stdout: "overall": { 2024-03-29T08:30:32.891 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 17.2.7-789-g4213e17c (4213e17c4de67016ed14f55d22503f4923b3a901) quincy (stable)": 1, 2024-03-29T08:30:32.891 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 13 2024-03-29T08:30:32.891 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-29T08:30:32.891 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-29T08:30:32.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:32.892+0000 7f552c800700 1 -- 172.21.15.37:0/3318573429 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f550c06cc30 msgr2=0x7f550c06f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:32.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:32.892+0000 7f552c800700 1 --2- 172.21.15.37:0/3318573429 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f550c06cc30 0x7f550c06f0e0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f551800b020 tx=0x7f5518009e30 comp rx=0 tx=0).stop 2024-03-29T08:30:32.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:32.892+0000 7f552c800700 1 -- 172.21.15.37:0/3318573429 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f55241037a0 msgr2=0x7f5524116230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:32.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:32.892+0000 7f552c800700 1 --2- 172.21.15.37:0/3318573429 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f55241037a0 0x7f5524116230 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f5514002a50 tx=0x7f5514004600 comp rx=0 tx=0).stop 2024-03-29T08:30:32.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:32.892+0000 7f552c800700 1 -- 172.21.15.37:0/3318573429 shutdown_connections 2024-03-29T08:30:32.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:32.892+0000 7f552c800700 1 --2- 172.21.15.37:0/3318573429 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f550c06cc30 0x7f550c06f0e0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:32.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:32.892+0000 7f552c800700 1 --2- 172.21.15.37:0/3318573429 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5524102e00 0x7f5524115cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:32.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:32.892+0000 7f552c800700 1 --2- 172.21.15.37:0/3318573429 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f55241037a0 0x7f5524116230 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:32.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:32.892+0000 7f552c800700 1 -- 172.21.15.37:0/3318573429 >> 172.21.15.37:0/3318573429 conn(0x7f55240f89b0 msgr2=0x7f55240706d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:30:32.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:32.892+0000 7f552c800700 1 -- 172.21.15.37:0/3318573429 shutdown_connections 2024-03-29T08:30:32.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:32.892+0000 7f552c800700 1 -- 172.21.15.37:0/3318573429 wait complete. 2024-03-29T08:30:32.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:30:32 smithi037 ceph-mon[142325]: from='client.35092 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:30:33.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:33.029+0000 7f1a83a00700 1 -- 172.21.15.37:0/4220269056 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1a7c102e20 msgr2=0x7f1a7c1031f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:33.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:33.029+0000 7f1a83a00700 1 --2- 172.21.15.37:0/4220269056 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1a7c102e20 0x7f1a7c1031f0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f1a70004940 tx=0x7f1a70009e30 comp rx=0 tx=0).stop 2024-03-29T08:30:33.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:33.029+0000 7f1a83a00700 1 -- 172.21.15.37:0/4220269056 shutdown_connections 2024-03-29T08:30:33.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:33.029+0000 7f1a83a00700 1 --2- 172.21.15.37:0/4220269056 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1a7c1037c0 0x7f1a7c0fce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:33.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:33.029+0000 7f1a83a00700 1 --2- 172.21.15.37:0/4220269056 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1a7c102e20 0x7f1a7c1031f0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:33.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:33.029+0000 7f1a83a00700 1 -- 172.21.15.37:0/4220269056 >> 172.21.15.37:0/4220269056 conn(0x7f1a7c0f8970 msgr2=0x7f1a7c0fad80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:30:33.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:33.029+0000 7f1a83a00700 1 -- 172.21.15.37:0/4220269056 shutdown_connections 2024-03-29T08:30:33.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:33.029+0000 7f1a83a00700 1 -- 172.21.15.37:0/4220269056 wait complete. 2024-03-29T08:30:33.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:33.030+0000 7f1a83a00700 1 Processor -- start 2024-03-29T08:30:33.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:33.030+0000 7f1a83a00700 1 -- start start 2024-03-29T08:30:33.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:33.031+0000 7f1a83a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1a7c102e20 0x7f1a7c0702c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:33.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:33.031+0000 7f1a83a00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1a7c1037c0 0x7f1a7c070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:33.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:33.031+0000 7f1a83a00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a7c070d90 con 0x7f1a7c102e20 2024-03-29T08:30:33.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:33.031+0000 7f1a83a00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a7c070ed0 con 0x7f1a7c1037c0 2024-03-29T08:30:33.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:33.031+0000 7f1a81000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1a7c102e20 0x7f1a7c0702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:33.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:33.031+0000 7f1a81000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1a7c102e20 0x7f1a7c0702c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40932/0 (socket says 172.21.15.37:40932) 2024-03-29T08:30:33.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:33.031+0000 7f1a81000700 1 -- 172.21.15.37:0/3166526696 learned_addr learned my addr 172.21.15.37:0/3166526696 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:30:33.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:33.031+0000 7f1a81000700 1 -- 172.21.15.37:0/3166526696 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1a7c1037c0 msgr2=0x7f1a7c070800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:33.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:33.031+0000 7f1a81000700 1 --2- 172.21.15.37:0/3166526696 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1a7c1037c0 0x7f1a7c070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:33.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:33.031+0000 7f1a81000700 1 -- 172.21.15.37:0/3166526696 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a70005040 con 0x7f1a7c102e20 2024-03-29T08:30:33.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:33.032+0000 7f1a81000700 1 --2- 172.21.15.37:0/3166526696 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1a7c102e20 0x7f1a7c0702c0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f1a70009360 tx=0x7f1a70009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:33.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:33.032+0000 7f1a78c00700 1 -- 172.21.15.37:0/3166526696 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1a7000e040 con 0x7f1a7c102e20 2024-03-29T08:30:33.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:33.032+0000 7f1a78c00700 1 -- 172.21.15.37:0/3166526696 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f1a70013420 con 0x7f1a7c102e20 2024-03-29T08:30:33.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:33.032+0000 7f1a78c00700 1 -- 172.21.15.37:0/3166526696 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1a7001c5f0 con 0x7f1a7c102e20 2024-03-29T08:30:33.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:33.032+0000 7f1a83a00700 1 -- 172.21.15.37:0/3166526696 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a7c071120 con 0x7f1a7c102e20 2024-03-29T08:30:33.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:33.032+0000 7f1a83a00700 1 -- 172.21.15.37:0/3166526696 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a7c11a180 con 0x7f1a7c102e20 2024-03-29T08:30:33.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:33.033+0000 7f1a83a00700 1 -- 172.21.15.37:0/3166526696 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a7c0fe570 con 0x7f1a7c102e20 2024-03-29T08:30:33.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:33.034+0000 7f1a78c00700 1 -- 172.21.15.37:0/3166526696 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f1a70013590 con 0x7f1a7c102e20 2024-03-29T08:30:33.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:33.035+0000 7f1a78c00700 1 --2- 172.21.15.37:0/3166526696 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f1a6406cc30 0x7f1a6406f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:33.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:33.035+0000 7f1a78c00700 1 -- 172.21.15.37:0/3166526696 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f1a7008d340 con 0x7f1a7c102e20 2024-03-29T08:30:33.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:33.035+0000 7f1a7be00700 1 --2- 172.21.15.37:0/3166526696 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f1a6406cc30 0x7f1a6406f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:33.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:33.036+0000 7f1a7be00700 1 --2- 172.21.15.37:0/3166526696 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f1a6406cc30 0x7f1a6406f0e0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f1a68009b80 tx=0x7f1a68009d60 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:33.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:33.040+0000 7f1a78c00700 1 -- 172.21.15.37:0/3166526696 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f1a70057300 con 0x7f1a7c102e20 2024-03-29T08:30:33.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:30:32 smithi148 ceph-mon[106692]: from='client.35092 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:30:33.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:33.300+0000 7f1a83a00700 1 -- 172.21.15.37:0/3166526696 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f1a7c04bed0 con 0x7f1a7c102e20 2024-03-29T08:30:33.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:33.300+0000 7f1a78c00700 1 -- 172.21.15.37:0/3166526696 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 28 v28) v1 ==== 76+0+1841 (secure 0 0 0) 0x7f1a7005a920 con 0x7f1a7c102e20 2024-03-29T08:30:33.301 INFO:teuthology.orchestra.run.smithi037.stdout:e28 2024-03-29T08:30:33.301 INFO:teuthology.orchestra.run.smithi037.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-29T08:30:33.301 INFO:teuthology.orchestra.run.smithi037.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T08:30:33.301 INFO:teuthology.orchestra.run.smithi037.stdout:legacy client fscid: 1 2024-03-29T08:30:33.301 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:30:33.301 INFO:teuthology.orchestra.run.smithi037.stdout:Filesystem 'cephfs' (1) 2024-03-29T08:30:33.301 INFO:teuthology.orchestra.run.smithi037.stdout:fs_name cephfs 2024-03-29T08:30:33.301 INFO:teuthology.orchestra.run.smithi037.stdout:epoch 21 2024-03-29T08:30:33.301 INFO:teuthology.orchestra.run.smithi037.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-03-29T08:30:33.301 INFO:teuthology.orchestra.run.smithi037.stdout:created 2024-03-29T08:03:29.241269+0000 2024-03-29T08:30:33.301 INFO:teuthology.orchestra.run.smithi037.stdout:modified 2024-03-29T08:17:16.237238+0000 2024-03-29T08:30:33.302 INFO:teuthology.orchestra.run.smithi037.stdout:tableserver 0 2024-03-29T08:30:33.302 INFO:teuthology.orchestra.run.smithi037.stdout:root 0 2024-03-29T08:30:33.302 INFO:teuthology.orchestra.run.smithi037.stdout:session_timeout 60 2024-03-29T08:30:33.302 INFO:teuthology.orchestra.run.smithi037.stdout:session_autoclose 300 2024-03-29T08:30:33.302 INFO:teuthology.orchestra.run.smithi037.stdout:max_file_size 1099511627776 2024-03-29T08:30:33.302 INFO:teuthology.orchestra.run.smithi037.stdout:max_xattr_size 65536 2024-03-29T08:30:33.302 INFO:teuthology.orchestra.run.smithi037.stdout:required_client_features {} 2024-03-29T08:30:33.302 INFO:teuthology.orchestra.run.smithi037.stdout:last_failure 0 2024-03-29T08:30:33.302 INFO:teuthology.orchestra.run.smithi037.stdout:last_failure_osd_epoch 82 2024-03-29T08:30:33.302 INFO:teuthology.orchestra.run.smithi037.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T08:30:33.302 INFO:teuthology.orchestra.run.smithi037.stdout:max_mds 2 2024-03-29T08:30:33.302 INFO:teuthology.orchestra.run.smithi037.stdout:in 0,1 2024-03-29T08:30:33.302 INFO:teuthology.orchestra.run.smithi037.stdout:up {} 2024-03-29T08:30:33.302 INFO:teuthology.orchestra.run.smithi037.stdout:failed 0,1 2024-03-29T08:30:33.303 INFO:teuthology.orchestra.run.smithi037.stdout:damaged 2024-03-29T08:30:33.303 INFO:teuthology.orchestra.run.smithi037.stdout:stopped 2024-03-29T08:30:33.303 INFO:teuthology.orchestra.run.smithi037.stdout:data_pools [3] 2024-03-29T08:30:33.303 INFO:teuthology.orchestra.run.smithi037.stdout:metadata_pool 2 2024-03-29T08:30:33.303 INFO:teuthology.orchestra.run.smithi037.stdout:inline_data disabled 2024-03-29T08:30:33.303 INFO:teuthology.orchestra.run.smithi037.stdout:balancer 2024-03-29T08:30:33.303 INFO:teuthology.orchestra.run.smithi037.stdout:bal_rank_mask -1 2024-03-29T08:30:33.303 INFO:teuthology.orchestra.run.smithi037.stdout:standby_count_wanted 1 2024-03-29T08:30:33.303 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:30:33.303 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:30:33.303 INFO:teuthology.orchestra.run.smithi037.stdout:Standby daemons: 2024-03-29T08:30:33.303 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:30:33.303 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi148.qukpjh{-1:24319} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.148:6826/2641327912,v1:172.21.15.148:6827/2641327912] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:30:33.303 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi037.mmwgxe{-1:34440} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.37:6828/1248142005,v1:172.21.15.37:6829/1248142005] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:30:33.303 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi037.iifsmh{-1:34444} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.37:6826/2060882959,v1:172.21.15.37:6827/2060882959] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:30:33.304 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi148.sqteih{-1:44395} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.148:6824/4078652242,v1:172.21.15.148:6825/4078652242] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:30:33.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:33.303+0000 7f1a83a00700 1 -- 172.21.15.37:0/3166526696 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f1a6406cc30 msgr2=0x7f1a6406f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:33.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:33.303+0000 7f1a83a00700 1 --2- 172.21.15.37:0/3166526696 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f1a6406cc30 0x7f1a6406f0e0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f1a68009b80 tx=0x7f1a68009d60 comp rx=0 tx=0).stop 2024-03-29T08:30:33.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:33.303+0000 7f1a83a00700 1 -- 172.21.15.37:0/3166526696 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1a7c102e20 msgr2=0x7f1a7c0702c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:33.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:33.303+0000 7f1a83a00700 1 --2- 172.21.15.37:0/3166526696 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1a7c102e20 0x7f1a7c0702c0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f1a70009360 tx=0x7f1a70009e30 comp rx=0 tx=0).stop 2024-03-29T08:30:33.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:33.303+0000 7f1a83a00700 1 -- 172.21.15.37:0/3166526696 shutdown_connections 2024-03-29T08:30:33.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:33.303+0000 7f1a83a00700 1 --2- 172.21.15.37:0/3166526696 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f1a6406cc30 0x7f1a6406f0e0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:33.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:33.303+0000 7f1a83a00700 1 --2- 172.21.15.37:0/3166526696 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1a7c102e20 0x7f1a7c0702c0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:33.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:33.303+0000 7f1a83a00700 1 --2- 172.21.15.37:0/3166526696 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1a7c1037c0 0x7f1a7c070800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:33.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:33.303+0000 7f1a83a00700 1 -- 172.21.15.37:0/3166526696 >> 172.21.15.37:0/3166526696 conn(0x7f1a7c0f8970 msgr2=0x7f1a7c0f99c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:30:33.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:33.303+0000 7f1a83a00700 1 -- 172.21.15.37:0/3166526696 shutdown_connections 2024-03-29T08:30:33.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:33.303+0000 7f1a83a00700 1 -- 172.21.15.37:0/3166526696 wait complete. 2024-03-29T08:30:33.305 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 28 2024-03-29T08:30:33.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:33.442+0000 7f238cc00700 1 -- 172.21.15.37:0/3388377673 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2388071f90 msgr2=0x7f2388070360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:33.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:33.442+0000 7f238cc00700 1 --2- 172.21.15.37:0/3388377673 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2388071f90 0x7f2388070360 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f2378004970 tx=0x7f2378009e30 comp rx=0 tx=0).stop 2024-03-29T08:30:33.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:33.442+0000 7f238cc00700 1 -- 172.21.15.37:0/3388377673 shutdown_connections 2024-03-29T08:30:33.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:33.442+0000 7f238cc00700 1 --2- 172.21.15.37:0/3388377673 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f23880708a0 0x7f2388070d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:33.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:33.442+0000 7f238cc00700 1 --2- 172.21.15.37:0/3388377673 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2388071f90 0x7f2388070360 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:33.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:33.442+0000 7f238cc00700 1 -- 172.21.15.37:0/3388377673 >> 172.21.15.37:0/3388377673 conn(0x7f23880f89b0 msgr2=0x7f23880fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:30:33.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:33.442+0000 7f238cc00700 1 -- 172.21.15.37:0/3388377673 shutdown_connections 2024-03-29T08:30:33.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:33.442+0000 7f238cc00700 1 -- 172.21.15.37:0/3388377673 wait complete. 2024-03-29T08:30:33.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:33.443+0000 7f238cc00700 1 Processor -- start 2024-03-29T08:30:33.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:33.443+0000 7f238cc00700 1 -- start start 2024-03-29T08:30:33.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:33.443+0000 7f238cc00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f23880708a0 0x7f238811a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:33.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:33.443+0000 7f238cc00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2388071f90 0x7f238811a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:33.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:33.443+0000 7f238cc00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f238811ac50 con 0x7f2388071f90 2024-03-29T08:30:33.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:33.443+0000 7f238cc00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2388114020 con 0x7f23880708a0 2024-03-29T08:30:33.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:33.444+0000 7f2385400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2388071f90 0x7f238811a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:33.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:33.444+0000 7f2385400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2388071f90 0x7f238811a600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40940/0 (socket says 172.21.15.37:40940) 2024-03-29T08:30:33.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:33.444+0000 7f2385400700 1 -- 172.21.15.37:0/1191816440 learned_addr learned my addr 172.21.15.37:0/1191816440 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:30:33.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:33.444+0000 7f2385e00700 1 --2- 172.21.15.37:0/1191816440 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f23880708a0 0x7f238811a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:33.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:33.444+0000 7f2385400700 1 -- 172.21.15.37:0/1191816440 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f23880708a0 msgr2=0x7f238811a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:33.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:33.444+0000 7f2385400700 1 --2- 172.21.15.37:0/1191816440 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f23880708a0 0x7f238811a0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:33.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:33.444+0000 7f2385400700 1 -- 172.21.15.37:0/1191816440 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2378005040 con 0x7f2388071f90 2024-03-29T08:30:33.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:33.444+0000 7f2385400700 1 --2- 172.21.15.37:0/1191816440 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2388071f90 0x7f238811a600 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f2374002a50 tx=0x7f23740046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:33.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:33.445+0000 7f237e000700 1 -- 172.21.15.37:0/1191816440 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2374007cc0 con 0x7f2388071f90 2024-03-29T08:30:33.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:33.445+0000 7f237e000700 1 -- 172.21.15.37:0/1191816440 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f237400f040 con 0x7f2388071f90 2024-03-29T08:30:33.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:33.445+0000 7f237e000700 1 -- 172.21.15.37:0/1191816440 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f237400d9c0 con 0x7f2388071f90 2024-03-29T08:30:33.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:33.445+0000 7f238cc00700 1 -- 172.21.15.37:0/1191816440 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2388114300 con 0x7f2388071f90 2024-03-29T08:30:33.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:33.445+0000 7f238cc00700 1 -- 172.21.15.37:0/1191816440 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2388114840 con 0x7f2388071f90 2024-03-29T08:30:33.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:33.446+0000 7f238cc00700 1 -- 172.21.15.37:0/1191816440 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2388071aa0 con 0x7f2388071f90 2024-03-29T08:30:33.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:33.447+0000 7f237e000700 1 -- 172.21.15.37:0/1191816440 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f237400db20 con 0x7f2388071f90 2024-03-29T08:30:33.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:33.448+0000 7f237e000700 1 --2- 172.21.15.37:0/1191816440 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f236c06cc30 0x7f236c06f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:33.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:33.448+0000 7f237e000700 1 -- 172.21.15.37:0/1191816440 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f237408bd20 con 0x7f2388071f90 2024-03-29T08:30:33.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:33.448+0000 7f2385e00700 1 --2- 172.21.15.37:0/1191816440 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f236c06cc30 0x7f236c06f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:33.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:33.449+0000 7f2385e00700 1 --2- 172.21.15.37:0/1191816440 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f236c06cc30 0x7f236c06f0e0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f237800b370 tx=0x7f2378009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:33.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:33.452+0000 7f237e000700 1 -- 172.21.15.37:0/1191816440 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f2374059770 con 0x7f2388071f90 2024-03-29T08:30:33.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:33.687+0000 7f238cc00700 1 -- 172.21.15.37:0/1191816440 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f2388110450 con 0x7f236c06cc30 2024-03-29T08:30:33.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:33.688+0000 7f237e000700 1 -- 172.21.15.37:0/1191816440 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f2388110450 con 0x7f236c06cc30 2024-03-29T08:30:33.688 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-29T08:30:33.688 INFO:teuthology.orchestra.run.smithi037.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59", 2024-03-29T08:30:33.688 INFO:teuthology.orchestra.run.smithi037.stdout: "in_progress": true, 2024-03-29T08:30:33.688 INFO:teuthology.orchestra.run.smithi037.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-29T08:30:33.688 INFO:teuthology.orchestra.run.smithi037.stdout: "services_complete": [ 2024-03-29T08:30:33.689 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr", 2024-03-29T08:30:33.689 INFO:teuthology.orchestra.run.smithi037.stdout: "mon", 2024-03-29T08:30:33.689 INFO:teuthology.orchestra.run.smithi037.stdout: "crash", 2024-03-29T08:30:33.689 INFO:teuthology.orchestra.run.smithi037.stdout: "osd" 2024-03-29T08:30:33.689 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-29T08:30:33.689 INFO:teuthology.orchestra.run.smithi037.stdout: "progress": "15/21 daemons upgraded", 2024-03-29T08:30:33.689 INFO:teuthology.orchestra.run.smithi037.stdout: "message": "Currently upgrading mds daemons", 2024-03-29T08:30:33.689 INFO:teuthology.orchestra.run.smithi037.stdout: "is_paused": false 2024-03-29T08:30:33.689 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-29T08:30:33.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:33.690+0000 7f238cc00700 1 -- 172.21.15.37:0/1191816440 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f236c06cc30 msgr2=0x7f236c06f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:33.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:33.690+0000 7f238cc00700 1 --2- 172.21.15.37:0/1191816440 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f236c06cc30 0x7f236c06f0e0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f237800b370 tx=0x7f2378009e30 comp rx=0 tx=0).stop 2024-03-29T08:30:33.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:33.691+0000 7f238cc00700 1 -- 172.21.15.37:0/1191816440 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2388071f90 msgr2=0x7f238811a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:33.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:33.691+0000 7f238cc00700 1 --2- 172.21.15.37:0/1191816440 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2388071f90 0x7f238811a600 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f2374002a50 tx=0x7f23740046b0 comp rx=0 tx=0).stop 2024-03-29T08:30:33.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:33.691+0000 7f238cc00700 1 -- 172.21.15.37:0/1191816440 shutdown_connections 2024-03-29T08:30:33.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:33.691+0000 7f238cc00700 1 --2- 172.21.15.37:0/1191816440 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f236c06cc30 0x7f236c06f0e0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:33.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:33.691+0000 7f238cc00700 1 --2- 172.21.15.37:0/1191816440 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f23880708a0 0x7f238811a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:33.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:33.691+0000 7f238cc00700 1 --2- 172.21.15.37:0/1191816440 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2388071f90 0x7f238811a600 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:33.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:33.691+0000 7f238cc00700 1 -- 172.21.15.37:0/1191816440 >> 172.21.15.37:0/1191816440 conn(0x7f23880f89b0 msgr2=0x7f23880f9b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:30:33.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:33.691+0000 7f238cc00700 1 -- 172.21.15.37:0/1191816440 shutdown_connections 2024-03-29T08:30:33.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:33.691+0000 7f238cc00700 1 -- 172.21.15.37:0/1191816440 wait complete. 2024-03-29T08:30:33.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:33.828+0000 7f4d3da00700 1 -- 172.21.15.37:0/1908540808 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4d38102e00 msgr2=0x7f4d381031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:33.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:33.828+0000 7f4d3da00700 1 --2- 172.21.15.37:0/1908540808 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4d38102e00 0x7f4d381031d0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f4d28004940 tx=0x7f4d28009e30 comp rx=0 tx=0).stop 2024-03-29T08:30:33.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:33.828+0000 7f4d3da00700 1 -- 172.21.15.37:0/1908540808 shutdown_connections 2024-03-29T08:30:33.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:33.828+0000 7f4d3da00700 1 --2- 172.21.15.37:0/1908540808 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4d381037a0 0x7f4d380fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:33.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:33.828+0000 7f4d3da00700 1 --2- 172.21.15.37:0/1908540808 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4d38102e00 0x7f4d381031d0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:33.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:33.828+0000 7f4d3da00700 1 -- 172.21.15.37:0/1908540808 >> 172.21.15.37:0/1908540808 conn(0x7f4d380f89b0 msgr2=0x7f4d380fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:30:33.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:33.829+0000 7f4d3da00700 1 -- 172.21.15.37:0/1908540808 shutdown_connections 2024-03-29T08:30:33.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:33.829+0000 7f4d3da00700 1 -- 172.21.15.37:0/1908540808 wait complete. 2024-03-29T08:30:33.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:33.829+0000 7f4d3da00700 1 Processor -- start 2024-03-29T08:30:33.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:33.830+0000 7f4d3da00700 1 -- start start 2024-03-29T08:30:33.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:33.830+0000 7f4d3da00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4d38102e00 0x7f4d38115cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:33.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:33.830+0000 7f4d3da00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4d381037a0 0x7f4d38116210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:33.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:33.830+0000 7f4d3da00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d38116860 con 0x7f4d38102e00 2024-03-29T08:30:33.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:33.830+0000 7f4d3da00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d3810fc30 con 0x7f4d381037a0 2024-03-29T08:30:33.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:33.830+0000 7f4d36a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4d38102e00 0x7f4d38115cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:33.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:33.830+0000 7f4d36a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4d38102e00 0x7f4d38115cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40972/0 (socket says 172.21.15.37:40972) 2024-03-29T08:30:33.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:33.830+0000 7f4d36a00700 1 -- 172.21.15.37:0/1374386117 learned_addr learned my addr 172.21.15.37:0/1374386117 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:30:33.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:33.830+0000 7f4d36000700 1 --2- 172.21.15.37:0/1374386117 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4d381037a0 0x7f4d38116210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:33.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:33.831+0000 7f4d36a00700 1 -- 172.21.15.37:0/1374386117 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4d381037a0 msgr2=0x7f4d38116210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:33.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:33.831+0000 7f4d36a00700 1 --2- 172.21.15.37:0/1374386117 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4d381037a0 0x7f4d38116210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:33.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:33.831+0000 7f4d36a00700 1 -- 172.21.15.37:0/1374386117 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d28005040 con 0x7f4d38102e00 2024-03-29T08:30:33.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:33.831+0000 7f4d36000700 1 --2- 172.21.15.37:0/1374386117 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4d381037a0 0x7f4d38116210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:30:33.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:33.831+0000 7f4d36a00700 1 --2- 172.21.15.37:0/1374386117 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4d38102e00 0x7f4d38115cd0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f4d28009360 tx=0x7f4d28009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:33.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:33.831+0000 7f4d2ea00700 1 -- 172.21.15.37:0/1374386117 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4d2800e040 con 0x7f4d38102e00 2024-03-29T08:30:33.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:33.831+0000 7f4d2ea00700 1 -- 172.21.15.37:0/1374386117 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f4d28013420 con 0x7f4d38102e00 2024-03-29T08:30:33.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:33.831+0000 7f4d3da00700 1 -- 172.21.15.37:0/1374386117 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d3810feb0 con 0x7f4d38102e00 2024-03-29T08:30:33.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:33.831+0000 7f4d2ea00700 1 -- 172.21.15.37:0/1374386117 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4d2801c5f0 con 0x7f4d38102e00 2024-03-29T08:30:33.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:33.831+0000 7f4d3da00700 1 -- 172.21.15.37:0/1374386117 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d38110470 con 0x7f4d38102e00 2024-03-29T08:30:33.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:33.832+0000 7f4d3da00700 1 -- 172.21.15.37:0/1374386117 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d380fe550 con 0x7f4d38102e00 2024-03-29T08:30:33.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:33.833+0000 7f4d2ea00700 1 -- 172.21.15.37:0/1374386117 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f4d28013590 con 0x7f4d38102e00 2024-03-29T08:30:33.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:33.834+0000 7f4d2ea00700 1 --2- 172.21.15.37:0/1374386117 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f4d1c06cc30 0x7f4d1c06f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:33.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:33.834+0000 7f4d2ea00700 1 -- 172.21.15.37:0/1374386117 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f4d2808d240 con 0x7f4d38102e00 2024-03-29T08:30:33.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:33.835+0000 7f4d36000700 1 --2- 172.21.15.37:0/1374386117 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f4d1c06cc30 0x7f4d1c06f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:33.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:33.835+0000 7f4d36000700 1 --2- 172.21.15.37:0/1374386117 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f4d1c06cc30 0x7f4d1c06f0e0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f4d24009fd0 tx=0x7f4d24009ee0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:33.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:33.839+0000 7f4d2ea00700 1 -- 172.21.15.37:0/1374386117 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f4d2805ad10 con 0x7f4d38102e00 2024-03-29T08:30:33.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:30:33 smithi037 ceph-mon[142325]: pgmap v589: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:30:33.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:30:33 smithi037 ceph-mon[142325]: from='client.35096 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:30:33.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:30:33 smithi037 ceph-mon[142325]: from='client.35100 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:30:33.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:30:33 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/3318573429' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:30:33.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:30:33 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/3166526696' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T08:30:34.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:30:33 smithi148 ceph-mon[106692]: pgmap v589: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:30:34.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:30:33 smithi148 ceph-mon[106692]: from='client.35096 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:30:34.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:30:33 smithi148 ceph-mon[106692]: from='client.35100 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:30:34.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:30:33 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/3318573429' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:30:34.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:30:33 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/3166526696' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T08:30:34.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:34.138+0000 7f4d3da00700 1 -- 172.21.15.37:0/1374386117 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f4d3804bf00 con 0x7f4d38102e00 2024-03-29T08:30:34.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:34.139+0000 7f4d2ea00700 1 -- 172.21.15.37:0/1374386117 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7f4d2805a8a0 con 0x7f4d38102e00 2024-03-29T08:30:34.139 INFO:teuthology.orchestra.run.smithi037.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-03-29T08:30:34.139 INFO:teuthology.orchestra.run.smithi037.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-03-29T08:30:34.139 INFO:teuthology.orchestra.run.smithi037.stdout: fs cephfs is degraded 2024-03-29T08:30:34.139 INFO:teuthology.orchestra.run.smithi037.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-03-29T08:30:34.140 INFO:teuthology.orchestra.run.smithi037.stdout: fs cephfs is offline because no MDS is active for it. 2024-03-29T08:30:34.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:34.141+0000 7f4d3da00700 1 -- 172.21.15.37:0/1374386117 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f4d1c06cc30 msgr2=0x7f4d1c06f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:34.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:34.141+0000 7f4d3da00700 1 --2- 172.21.15.37:0/1374386117 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f4d1c06cc30 0x7f4d1c06f0e0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f4d24009fd0 tx=0x7f4d24009ee0 comp rx=0 tx=0).stop 2024-03-29T08:30:34.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:34.141+0000 7f4d3da00700 1 -- 172.21.15.37:0/1374386117 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4d38102e00 msgr2=0x7f4d38115cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:34.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:34.141+0000 7f4d3da00700 1 --2- 172.21.15.37:0/1374386117 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4d38102e00 0x7f4d38115cd0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f4d28009360 tx=0x7f4d28009e30 comp rx=0 tx=0).stop 2024-03-29T08:30:34.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:34.141+0000 7f4d3da00700 1 -- 172.21.15.37:0/1374386117 shutdown_connections 2024-03-29T08:30:34.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:34.142+0000 7f4d3da00700 1 --2- 172.21.15.37:0/1374386117 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f4d1c06cc30 0x7f4d1c06f0e0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:34.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:34.142+0000 7f4d3da00700 1 --2- 172.21.15.37:0/1374386117 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4d38102e00 0x7f4d38115cd0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:34.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:34.142+0000 7f4d3da00700 1 --2- 172.21.15.37:0/1374386117 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4d381037a0 0x7f4d38116210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:34.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:34.142+0000 7f4d3da00700 1 -- 172.21.15.37:0/1374386117 >> 172.21.15.37:0/1374386117 conn(0x7f4d380f89b0 msgr2=0x7f4d380f99e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:30:34.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:34.142+0000 7f4d3da00700 1 -- 172.21.15.37:0/1374386117 shutdown_connections 2024-03-29T08:30:34.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:30:34.142+0000 7f4d3da00700 1 -- 172.21.15.37:0/1374386117 wait complete. 2024-03-29T08:30:34.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:30:34 smithi037 ceph-mon[142325]: from='client.35112 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:30:34.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:30:34 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/1374386117' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T08:30:35.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:30:34 smithi148 ceph-mon[106692]: from='client.35112 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:30:35.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:30:34 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/1374386117' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T08:30:35.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:30:35 smithi037 ceph-mon[142325]: pgmap v590: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:30:36.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:30:35 smithi148 ceph-mon[106692]: pgmap v590: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:30:37.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:30:37 smithi037 ceph-mon[142325]: pgmap v591: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:30:38.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:30:37 smithi148 ceph-mon[106692]: pgmap v591: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:30:39.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:30:39 smithi037 ceph-mon[142325]: pgmap v592: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:30:39.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:30:39 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:30:40.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:30:39 smithi148 ceph-mon[106692]: pgmap v592: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:30:40.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:30:39 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:30:41.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:30:41 smithi037 ceph-mon[142325]: pgmap v593: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:30:42.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:30:41 smithi148 ceph-mon[106692]: pgmap v593: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:30:43.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:30:43 smithi037 ceph-mon[142325]: pgmap v594: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:30:44.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:30:43 smithi148 ceph-mon[106692]: pgmap v594: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:30:45.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:30:45 smithi037 ceph-mon[142325]: pgmap v595: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:30:46.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:30:45 smithi148 ceph-mon[106692]: pgmap v595: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:30:47.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:30:47 smithi037 ceph-mon[142325]: pgmap v596: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:30:48.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:30:47 smithi148 ceph-mon[106692]: pgmap v596: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:30:49.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:30:49 smithi037 ceph-mon[142325]: pgmap v597: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:30:50.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:30:49 smithi148 ceph-mon[106692]: pgmap v597: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:30:51.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:30:51 smithi037 ceph-mon[142325]: pgmap v598: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:30:52.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:30:51 smithi148 ceph-mon[106692]: pgmap v598: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:30:53.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:30:53 smithi037 ceph-mon[142325]: pgmap v599: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:30:54.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:30:53 smithi148 ceph-mon[106692]: pgmap v599: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:30:55.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:30:55 smithi037 ceph-mon[142325]: pgmap v600: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:30:55.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:30:55 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:30:55.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:30:55 smithi148 ceph-mon[106692]: pgmap v600: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:30:55.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:30:55 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:30:57.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:30:57 smithi037 ceph-mon[142325]: pgmap v601: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:30:57.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:30:57 smithi148 ceph-mon[106692]: pgmap v601: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:30:59.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:30:59 smithi037 ceph-mon[142325]: pgmap v602: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:30:59.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:30:59 smithi148 ceph-mon[106692]: pgmap v602: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:31:01.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:31:01 smithi037 ceph-mon[142325]: pgmap v603: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:31:01.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:31:01 smithi148 ceph-mon[106692]: pgmap v603: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:31:03.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:31:03 smithi037 ceph-mon[142325]: pgmap v604: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:31:03.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:31:03 smithi148 ceph-mon[106692]: pgmap v604: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:31:04.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:04.295+0000 7f26fd200700 1 -- 172.21.15.37:0/4171639236 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f26f81037a0 msgr2=0x7f26f80fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:31:04.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:04.295+0000 7f26fd200700 1 --2- 172.21.15.37:0/4171639236 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f26f81037a0 0x7f26f80fce10 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f26e4004940 tx=0x7f26e4009e30 comp rx=0 tx=0).stop 2024-03-29T08:31:04.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:04.295+0000 7f26fd200700 1 -- 172.21.15.37:0/4171639236 shutdown_connections 2024-03-29T08:31:04.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:04.295+0000 7f26fd200700 1 --2- 172.21.15.37:0/4171639236 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f26f81037a0 0x7f26f80fce10 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:31:04.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:04.295+0000 7f26fd200700 1 --2- 172.21.15.37:0/4171639236 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f26f8102e00 0x7f26f81031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:31:04.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:04.295+0000 7f26fd200700 1 -- 172.21.15.37:0/4171639236 >> 172.21.15.37:0/4171639236 conn(0x7f26f80f89b0 msgr2=0x7f26f80fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:31:04.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:04.295+0000 7f26fd200700 1 -- 172.21.15.37:0/4171639236 shutdown_connections 2024-03-29T08:31:04.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:04.295+0000 7f26fd200700 1 -- 172.21.15.37:0/4171639236 wait complete. 2024-03-29T08:31:04.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:04.296+0000 7f26fd200700 1 Processor -- start 2024-03-29T08:31:04.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:04.296+0000 7f26fd200700 1 -- start start 2024-03-29T08:31:04.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:04.297+0000 7f26fd200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f26f8102e00 0x7f26f8115d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:31:04.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:04.297+0000 7f26fd200700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f26f81037a0 0x7f26f8116260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:31:04.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:04.297+0000 7f26fd200700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26f81168b0 con 0x7f26f8102e00 2024-03-29T08:31:04.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:04.297+0000 7f26fd200700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26f810fc80 con 0x7f26f81037a0 2024-03-29T08:31:04.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:04.297+0000 7f26f5e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f26f8102e00 0x7f26f8115d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:31:04.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:04.297+0000 7f26f5e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f26f8102e00 0x7f26f8115d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45628/0 (socket says 172.21.15.37:45628) 2024-03-29T08:31:04.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:04.297+0000 7f26f5e00700 1 -- 172.21.15.37:0/3624015488 learned_addr learned my addr 172.21.15.37:0/3624015488 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:31:04.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:04.297+0000 7f26f5e00700 1 -- 172.21.15.37:0/3624015488 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f26f81037a0 msgr2=0x7f26f8116260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:31:04.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:04.297+0000 7f26f5e00700 1 --2- 172.21.15.37:0/3624015488 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f26f81037a0 0x7f26f8116260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:31:04.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:04.297+0000 7f26f5e00700 1 -- 172.21.15.37:0/3624015488 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26e4005040 con 0x7f26f8102e00 2024-03-29T08:31:04.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:04.297+0000 7f26f5e00700 1 --2- 172.21.15.37:0/3624015488 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f26f8102e00 0x7f26f8115d20 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f26e800c8c0 tx=0x7f26e80044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:31:04.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:04.298+0000 7f26ee000700 1 -- 172.21.15.37:0/3624015488 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f26e800a980 con 0x7f26f8102e00 2024-03-29T08:31:04.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:04.298+0000 7f26ee000700 1 -- 172.21.15.37:0/3624015488 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f26e800ce60 con 0x7f26f8102e00 2024-03-29T08:31:04.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:04.298+0000 7f26fd200700 1 -- 172.21.15.37:0/3624015488 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26f810ff60 con 0x7f26f8102e00 2024-03-29T08:31:04.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:04.298+0000 7f26ee000700 1 -- 172.21.15.37:0/3624015488 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f26e80113a0 con 0x7f26f8102e00 2024-03-29T08:31:04.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:04.298+0000 7f26fd200700 1 -- 172.21.15.37:0/3624015488 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26f8110520 con 0x7f26f8102e00 2024-03-29T08:31:04.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:04.299+0000 7f26fd200700 1 -- 172.21.15.37:0/3624015488 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26f80fe550 con 0x7f26f8102e00 2024-03-29T08:31:04.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:04.300+0000 7f26ee000700 1 -- 172.21.15.37:0/3624015488 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f26e8012400 con 0x7f26f8102e00 2024-03-29T08:31:04.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:04.301+0000 7f26ee000700 1 --2- 172.21.15.37:0/3624015488 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f26dc06cc30 0x7f26dc06f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:31:04.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:04.301+0000 7f26ee000700 1 -- 172.21.15.37:0/3624015488 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f26e808c0f0 con 0x7f26f8102e00 2024-03-29T08:31:04.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:04.301+0000 7f26f5400700 1 --2- 172.21.15.37:0/3624015488 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f26dc06cc30 0x7f26dc06f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:31:04.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:04.302+0000 7f26f5400700 1 --2- 172.21.15.37:0/3624015488 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f26dc06cc30 0x7f26dc06f0e0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f26e4004f90 tx=0x7f26e4009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:31:04.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:04.305+0000 7f26ee000700 1 -- 172.21.15.37:0/3624015488 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f26e8059bc0 con 0x7f26f8102e00 2024-03-29T08:31:04.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:04.540+0000 7f26fd200700 1 -- 172.21.15.37:0/3624015488 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f26f8109da0 con 0x7f26dc06cc30 2024-03-29T08:31:04.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:04.541+0000 7f26ee000700 1 -- 172.21.15.37:0/3624015488 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f26f8109da0 con 0x7f26dc06cc30 2024-03-29T08:31:04.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:04.544+0000 7f26fd200700 1 -- 172.21.15.37:0/3624015488 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f26dc06cc30 msgr2=0x7f26dc06f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:31:04.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:04.544+0000 7f26fd200700 1 --2- 172.21.15.37:0/3624015488 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f26dc06cc30 0x7f26dc06f0e0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f26e4004f90 tx=0x7f26e4009e30 comp rx=0 tx=0).stop 2024-03-29T08:31:04.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:04.544+0000 7f26fd200700 1 -- 172.21.15.37:0/3624015488 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f26f8102e00 msgr2=0x7f26f8115d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:31:04.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:04.544+0000 7f26fd200700 1 --2- 172.21.15.37:0/3624015488 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f26f8102e00 0x7f26f8115d20 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f26e800c8c0 tx=0x7f26e80044c0 comp rx=0 tx=0).stop 2024-03-29T08:31:04.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:04.544+0000 7f26fd200700 1 -- 172.21.15.37:0/3624015488 shutdown_connections 2024-03-29T08:31:04.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:04.544+0000 7f26fd200700 1 --2- 172.21.15.37:0/3624015488 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f26dc06cc30 0x7f26dc06f0e0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:31:04.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:04.544+0000 7f26fd200700 1 --2- 172.21.15.37:0/3624015488 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f26f8102e00 0x7f26f8115d20 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:31:04.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:04.544+0000 7f26fd200700 1 --2- 172.21.15.37:0/3624015488 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f26f81037a0 0x7f26f8116260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:31:04.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:04.544+0000 7f26fd200700 1 -- 172.21.15.37:0/3624015488 >> 172.21.15.37:0/3624015488 conn(0x7f26f80f89b0 msgr2=0x7f26f80f99f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:31:04.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:04.544+0000 7f26fd200700 1 -- 172.21.15.37:0/3624015488 shutdown_connections 2024-03-29T08:31:04.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:04.544+0000 7f26fd200700 1 -- 172.21.15.37:0/3624015488 wait complete. 2024-03-29T08:31:04.559 INFO:teuthology.orchestra.run.smithi037.stdout:true 2024-03-29T08:31:04.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:04.681+0000 7fee57e00700 1 -- 172.21.15.37:0/94024555 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fee58070360 msgr2=0x7fee58070730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:31:04.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:04.681+0000 7fee57e00700 1 --2- 172.21.15.37:0/94024555 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fee58070360 0x7fee58070730 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fee48004970 tx=0x7fee48009e30 comp rx=0 tx=0).stop 2024-03-29T08:31:04.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:04.681+0000 7fee57e00700 1 -- 172.21.15.37:0/94024555 shutdown_connections 2024-03-29T08:31:04.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:04.681+0000 7fee57e00700 1 --2- 172.21.15.37:0/94024555 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fee58070c70 0x7fee58108f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:31:04.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:04.681+0000 7fee57e00700 1 --2- 172.21.15.37:0/94024555 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fee58070360 0x7fee58070730 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:31:04.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:04.681+0000 7fee57e00700 1 -- 172.21.15.37:0/94024555 >> 172.21.15.37:0/94024555 conn(0x7fee580f8830 msgr2=0x7fee580fac40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:31:04.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:04.682+0000 7fee57e00700 1 -- 172.21.15.37:0/94024555 shutdown_connections 2024-03-29T08:31:04.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:04.682+0000 7fee57e00700 1 -- 172.21.15.37:0/94024555 wait complete. 2024-03-29T08:31:04.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:04.682+0000 7fee57e00700 1 Processor -- start 2024-03-29T08:31:04.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:04.683+0000 7fee57e00700 1 -- start start 2024-03-29T08:31:04.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:04.683+0000 7fee57e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fee58070360 0x7fee58115a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:31:04.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:04.683+0000 7fee57e00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fee58070c70 0x7fee58115fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:31:04.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:04.683+0000 7fee57e00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee58116620 con 0x7fee58070360 2024-03-29T08:31:04.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:04.683+0000 7fee57e00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee5810fa50 con 0x7fee58070c70 2024-03-29T08:31:04.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:04.683+0000 7fee56a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fee58070360 0x7fee58115a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:31:04.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:04.683+0000 7fee56a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fee58070360 0x7fee58115a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45650/0 (socket says 172.21.15.37:45650) 2024-03-29T08:31:04.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:04.683+0000 7fee56a00700 1 -- 172.21.15.37:0/3105450805 learned_addr learned my addr 172.21.15.37:0/3105450805 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:31:04.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:04.684+0000 7fee56a00700 1 -- 172.21.15.37:0/3105450805 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fee58070c70 msgr2=0x7fee58115fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:31:04.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:04.684+0000 7fee56a00700 1 --2- 172.21.15.37:0/3105450805 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fee58070c70 0x7fee58115fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:31:04.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:04.684+0000 7fee56a00700 1 -- 172.21.15.37:0/3105450805 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee48005040 con 0x7fee58070360 2024-03-29T08:31:04.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:04.684+0000 7fee56a00700 1 --2- 172.21.15.37:0/3105450805 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fee58070360 0x7fee58115a90 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fee48009360 tx=0x7fee48009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:31:04.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:04.684+0000 7fee4ea00700 1 -- 172.21.15.37:0/3105450805 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fee4800e040 con 0x7fee58070360 2024-03-29T08:31:04.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:04.684+0000 7fee4ea00700 1 -- 172.21.15.37:0/3105450805 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fee48004290 con 0x7fee58070360 2024-03-29T08:31:04.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:04.684+0000 7fee57e00700 1 -- 172.21.15.37:0/3105450805 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee5810fcd0 con 0x7fee58070360 2024-03-29T08:31:04.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:04.684+0000 7fee4ea00700 1 -- 172.21.15.37:0/3105450805 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fee4801c3f0 con 0x7fee58070360 2024-03-29T08:31:04.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:04.684+0000 7fee57e00700 1 -- 172.21.15.37:0/3105450805 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee58110210 con 0x7fee58070360 2024-03-29T08:31:04.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:04.686+0000 7fee57e00700 1 -- 172.21.15.37:0/3105450805 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee58071aa0 con 0x7fee58070360 2024-03-29T08:31:04.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:04.686+0000 7fee4ea00700 1 -- 172.21.15.37:0/3105450805 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fee48013420 con 0x7fee58070360 2024-03-29T08:31:04.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:04.687+0000 7fee4ea00700 1 --2- 172.21.15.37:0/3105450805 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fee3c06cc30 0x7fee3c06f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:31:04.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:04.687+0000 7fee4ea00700 1 -- 172.21.15.37:0/3105450805 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fee4808d360 con 0x7fee58070360 2024-03-29T08:31:04.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:04.687+0000 7fee56000700 1 --2- 172.21.15.37:0/3105450805 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fee3c06cc30 0x7fee3c06f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:31:04.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:04.688+0000 7fee56000700 1 --2- 172.21.15.37:0/3105450805 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fee3c06cc30 0x7fee3c06f0e0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fee58104940 tx=0x7fee44007cc0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:31:04.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:04.692+0000 7fee4ea00700 1 -- 172.21.15.37:0/3105450805 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fee4805aef0 con 0x7fee58070360 2024-03-29T08:31:04.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:04.924+0000 7fee57e00700 1 -- 172.21.15.37:0/3105450805 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fee5810bdf0 con 0x7fee3c06cc30 2024-03-29T08:31:04.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:04.926+0000 7fee4ea00700 1 -- 172.21.15.37:0/3105450805 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fee5810bdf0 con 0x7fee3c06cc30 2024-03-29T08:31:04.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:04.928+0000 7fee57e00700 1 -- 172.21.15.37:0/3105450805 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fee3c06cc30 msgr2=0x7fee3c06f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:31:04.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:04.928+0000 7fee57e00700 1 --2- 172.21.15.37:0/3105450805 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fee3c06cc30 0x7fee3c06f0e0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fee58104940 tx=0x7fee44007cc0 comp rx=0 tx=0).stop 2024-03-29T08:31:04.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:04.928+0000 7fee57e00700 1 -- 172.21.15.37:0/3105450805 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fee58070360 msgr2=0x7fee58115a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:31:04.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:04.928+0000 7fee57e00700 1 --2- 172.21.15.37:0/3105450805 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fee58070360 0x7fee58115a90 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fee48009360 tx=0x7fee48009e30 comp rx=0 tx=0).stop 2024-03-29T08:31:04.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:04.928+0000 7fee57e00700 1 -- 172.21.15.37:0/3105450805 shutdown_connections 2024-03-29T08:31:04.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:04.928+0000 7fee57e00700 1 --2- 172.21.15.37:0/3105450805 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fee3c06cc30 0x7fee3c06f0e0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:31:04.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:04.928+0000 7fee57e00700 1 --2- 172.21.15.37:0/3105450805 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fee58070360 0x7fee58115a90 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:31:04.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:04.928+0000 7fee57e00700 1 --2- 172.21.15.37:0/3105450805 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fee58070c70 0x7fee58115fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:31:04.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:04.928+0000 7fee57e00700 1 -- 172.21.15.37:0/3105450805 >> 172.21.15.37:0/3105450805 conn(0x7fee580f8830 msgr2=0x7fee580f9de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:31:04.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:04.928+0000 7fee57e00700 1 -- 172.21.15.37:0/3105450805 shutdown_connections 2024-03-29T08:31:04.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:04.928+0000 7fee57e00700 1 -- 172.21.15.37:0/3105450805 wait complete. 2024-03-29T08:31:05.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:05.066+0000 7f7f6d800700 1 -- 172.21.15.37:0/1371260497 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7f68103740 msgr2=0x7f7f680fcdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:31:05.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:05.066+0000 7f7f6d800700 1 --2- 172.21.15.37:0/1371260497 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7f68103740 0x7f7f680fcdb0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f7f54004940 tx=0x7f7f54009e30 comp rx=0 tx=0).stop 2024-03-29T08:31:05.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:05.066+0000 7f7f6d800700 1 -- 172.21.15.37:0/1371260497 shutdown_connections 2024-03-29T08:31:05.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:05.066+0000 7f7f6d800700 1 --2- 172.21.15.37:0/1371260497 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7f68103740 0x7f7f680fcdb0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:31:05.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:05.066+0000 7f7f6d800700 1 --2- 172.21.15.37:0/1371260497 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7f68102da0 0x7f7f68103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:31:05.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:05.066+0000 7f7f6d800700 1 -- 172.21.15.37:0/1371260497 >> 172.21.15.37:0/1371260497 conn(0x7f7f680f8950 msgr2=0x7f7f680fad60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:31:05.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:05.066+0000 7f7f6d800700 1 -- 172.21.15.37:0/1371260497 shutdown_connections 2024-03-29T08:31:05.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:05.066+0000 7f7f6d800700 1 -- 172.21.15.37:0/1371260497 wait complete. 2024-03-29T08:31:05.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:05.067+0000 7f7f6d800700 1 Processor -- start 2024-03-29T08:31:05.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:05.067+0000 7f7f6d800700 1 -- start start 2024-03-29T08:31:05.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:05.068+0000 7f7f6d800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7f68102da0 0x7f7f680702c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:31:05.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:05.068+0000 7f7f6d800700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7f68103740 0x7f7f68070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:31:05.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:05.068+0000 7f7f6d800700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f68070d90 con 0x7f7f68102da0 2024-03-29T08:31:05.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:05.068+0000 7f7f6d800700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f68070ed0 con 0x7f7f68103740 2024-03-29T08:31:05.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:05.068+0000 7f7f66a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7f68102da0 0x7f7f680702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:31:05.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:05.068+0000 7f7f66a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7f68102da0 0x7f7f680702c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45672/0 (socket says 172.21.15.37:45672) 2024-03-29T08:31:05.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:05.068+0000 7f7f66a00700 1 -- 172.21.15.37:0/2920082004 learned_addr learned my addr 172.21.15.37:0/2920082004 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:31:05.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:05.069+0000 7f7f66000700 1 --2- 172.21.15.37:0/2920082004 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7f68103740 0x7f7f68070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:31:05.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:05.069+0000 7f7f66a00700 1 -- 172.21.15.37:0/2920082004 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7f68103740 msgr2=0x7f7f68070800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:31:05.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:05.069+0000 7f7f66a00700 1 --2- 172.21.15.37:0/2920082004 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7f68103740 0x7f7f68070800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:31:05.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:05.069+0000 7f7f66a00700 1 -- 172.21.15.37:0/2920082004 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f54005040 con 0x7f7f68102da0 2024-03-29T08:31:05.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:05.069+0000 7f7f66a00700 1 --2- 172.21.15.37:0/2920082004 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7f68102da0 0x7f7f680702c0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f7f58002a50 tx=0x7f7f580046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:31:05.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:05.069+0000 7f7f5ea00700 1 -- 172.21.15.37:0/2920082004 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7f58007cc0 con 0x7f7f68102da0 2024-03-29T08:31:05.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:05.069+0000 7f7f5ea00700 1 -- 172.21.15.37:0/2920082004 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f7f5800f040 con 0x7f7f68102da0 2024-03-29T08:31:05.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:05.069+0000 7f7f5ea00700 1 -- 172.21.15.37:0/2920082004 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7f58009540 con 0x7f7f68102da0 2024-03-29T08:31:05.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:05.069+0000 7f7f6d800700 1 -- 172.21.15.37:0/2920082004 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f68119d20 con 0x7f7f68102da0 2024-03-29T08:31:05.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:05.069+0000 7f7f6d800700 1 -- 172.21.15.37:0/2920082004 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f6811a1a0 con 0x7f7f68102da0 2024-03-29T08:31:05.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:05.070+0000 7f7f6d800700 1 -- 172.21.15.37:0/2920082004 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f680fe4f0 con 0x7f7f68102da0 2024-03-29T08:31:05.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:05.071+0000 7f7f5ea00700 1 -- 172.21.15.37:0/2920082004 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f7f580096a0 con 0x7f7f68102da0 2024-03-29T08:31:05.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:05.072+0000 7f7f5ea00700 1 --2- 172.21.15.37:0/2920082004 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f7f4c06cc30 0x7f7f4c06f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:31:05.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:05.072+0000 7f7f5ea00700 1 -- 172.21.15.37:0/2920082004 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f7f5808be00 con 0x7f7f68102da0 2024-03-29T08:31:05.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:05.073+0000 7f7f66000700 1 --2- 172.21.15.37:0/2920082004 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f7f4c06cc30 0x7f7f4c06f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:31:05.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:05.073+0000 7f7f66000700 1 --2- 172.21.15.37:0/2920082004 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f7f4c06cc30 0x7f7f4c06f0e0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f7f5400b020 tx=0x7f7f54009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:31:05.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:05.077+0000 7f7f5ea00700 1 -- 172.21.15.37:0/2920082004 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f7f58055dc0 con 0x7f7f68102da0 2024-03-29T08:31:05.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:05.302+0000 7f7f6d800700 1 -- 172.21.15.37:0/2920082004 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f7f680719b0 con 0x7f7f4c06cc30 2024-03-29T08:31:05.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:05.312+0000 7f7f5ea00700 1 -- 172.21.15.37:0/2920082004 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f7f680719b0 con 0x7f7f4c06cc30 2024-03-29T08:31:05.313 INFO:teuthology.orchestra.run.smithi037.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-29T08:31:05.313 INFO:teuthology.orchestra.run.smithi037.stdout:alertmanager.smithi037 smithi037 *:9093,9094 running (20m) 13m ago 33m 14.9M - 0.25.0 c8568f914cd2 2f9f9746ed0f 2024-03-29T08:31:05.313 INFO:teuthology.orchestra.run.smithi037.stdout:crash.smithi037 smithi037 running (17m) 13m ago 34m 6647k - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 178d416a3186 2024-03-29T08:31:05.314 INFO:teuthology.orchestra.run.smithi037.stdout:crash.smithi148 smithi148 running (17m) 12m ago 32m 6656k - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 23666989d58f 2024-03-29T08:31:05.314 INFO:teuthology.orchestra.run.smithi037.stdout:grafana.smithi037 smithi037 *:3000 running (20m) 13m ago 33m 71.9M - 9.4.7 954c08fa6188 6b976fe49952 2024-03-29T08:31:05.314 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi037.iifsmh smithi037 running (13m) 13m ago 27m 10.8M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 2d97f562cc5b 2024-03-29T08:31:05.314 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi037.mmwgxe smithi037 running (13m) 13m ago 27m 10.9M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff ec37037f46e8 2024-03-29T08:31:05.314 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi148.qukpjh smithi148 running (27m) 12m ago 27m 15.3M - 17.2.7-789-g4213e17c cf22eb8683be 8e469782a273 2024-03-29T08:31:05.314 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi148.sqteih smithi148 running (12m) 12m ago 27m 10.7M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 44c15c77037a 2024-03-29T08:31:05.314 INFO:teuthology.orchestra.run.smithi037.stdout:mgr.smithi037.wjikii smithi037 *:8443,9283,8765 running (23m) 13m ago 35m 506M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff c5a1417b9417 2024-03-29T08:31:05.314 INFO:teuthology.orchestra.run.smithi037.stdout:mgr.smithi148.onhgvm smithi148 *:8443,9283,8765 running (22m) 12m ago 32m 450M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff fcfdbce062bd 2024-03-29T08:31:05.314 INFO:teuthology.orchestra.run.smithi037.stdout:mon.smithi037 smithi037 running (19m) 13m ago 35m 54.8M 2048M 18.2.2-1015-gd0e392d9 1cc2f45c17ff eea2060783af 2024-03-29T08:31:05.314 INFO:teuthology.orchestra.run.smithi037.stdout:mon.smithi148 smithi148 running (18m) 12m ago 32m 43.7M 2048M 18.2.2-1015-gd0e392d9 1cc2f45c17ff ad4b2c5748be 2024-03-29T08:31:05.314 INFO:teuthology.orchestra.run.smithi037.stdout:node-exporter.smithi037 smithi037 *:9100 running (21m) 13m ago 34m 7625k - 1.5.0 0da6a335fe13 26b8ae9cb878 2024-03-29T08:31:05.314 INFO:teuthology.orchestra.run.smithi037.stdout:node-exporter.smithi148 smithi148 *:9100 running (21m) 12m ago 32m 8166k - 1.5.0 0da6a335fe13 98414e4e64ed 2024-03-29T08:31:05.314 INFO:teuthology.orchestra.run.smithi037.stdout:osd.0 smithi037 running (17m) 13m ago 31m 104M 1619M 18.2.2-1015-gd0e392d9 1cc2f45c17ff c0c584f6245c 2024-03-29T08:31:05.315 INFO:teuthology.orchestra.run.smithi037.stdout:osd.1 smithi037 running (16m) 13m ago 31m 100M 1619M 18.2.2-1015-gd0e392d9 1cc2f45c17ff a4a2ff4cc83d 2024-03-29T08:31:05.315 INFO:teuthology.orchestra.run.smithi037.stdout:osd.2 smithi037 running (15m) 13m ago 30m 82.3M 1619M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 8b1bc71aa79e 2024-03-29T08:31:05.315 INFO:teuthology.orchestra.run.smithi037.stdout:osd.3 smithi148 running (15m) 12m ago 30m 105M 2642M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 1c656ab8ff86 2024-03-29T08:31:05.315 INFO:teuthology.orchestra.run.smithi037.stdout:osd.4 smithi148 running (14m) 12m ago 29m 97.3M 2642M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 966a9de11d22 2024-03-29T08:31:05.315 INFO:teuthology.orchestra.run.smithi037.stdout:osd.5 smithi148 running (14m) 12m ago 29m 85.8M 2642M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 474c4f9212ce 2024-03-29T08:31:05.315 INFO:teuthology.orchestra.run.smithi037.stdout:prometheus.smithi037 smithi037 *:9095 running (21m) 13m ago 33m 54.4M - 2.43.0 a07b618ecd1d 30c0223ec589 2024-03-29T08:31:05.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:05.314+0000 7f7f6d800700 1 -- 172.21.15.37:0/2920082004 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f7f4c06cc30 msgr2=0x7f7f4c06f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:31:05.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:05.314+0000 7f7f6d800700 1 --2- 172.21.15.37:0/2920082004 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f7f4c06cc30 0x7f7f4c06f0e0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f7f5400b020 tx=0x7f7f54009e30 comp rx=0 tx=0).stop 2024-03-29T08:31:05.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:05.314+0000 7f7f6d800700 1 -- 172.21.15.37:0/2920082004 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7f68102da0 msgr2=0x7f7f680702c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:31:05.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:05.314+0000 7f7f6d800700 1 --2- 172.21.15.37:0/2920082004 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7f68102da0 0x7f7f680702c0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f7f58002a50 tx=0x7f7f580046b0 comp rx=0 tx=0).stop 2024-03-29T08:31:05.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:05.315+0000 7f7f6d800700 1 -- 172.21.15.37:0/2920082004 shutdown_connections 2024-03-29T08:31:05.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:05.315+0000 7f7f6d800700 1 --2- 172.21.15.37:0/2920082004 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f7f4c06cc30 0x7f7f4c06f0e0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:31:05.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:05.315+0000 7f7f6d800700 1 --2- 172.21.15.37:0/2920082004 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7f68102da0 0x7f7f680702c0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:31:05.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:05.315+0000 7f7f6d800700 1 --2- 172.21.15.37:0/2920082004 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7f68103740 0x7f7f68070800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:31:05.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:05.315+0000 7f7f6d800700 1 -- 172.21.15.37:0/2920082004 >> 172.21.15.37:0/2920082004 conn(0x7f7f680f8950 msgr2=0x7f7f680f9990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:31:05.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:05.315+0000 7f7f6d800700 1 -- 172.21.15.37:0/2920082004 shutdown_connections 2024-03-29T08:31:05.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:05.315+0000 7f7f6d800700 1 -- 172.21.15.37:0/2920082004 wait complete. 2024-03-29T08:31:05.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:05.452+0000 7f6259600700 1 -- 172.21.15.37:0/2194191400 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f62540fbc80 msgr2=0x7f62540fc050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:31:05.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:05.452+0000 7f6259600700 1 --2- 172.21.15.37:0/2194191400 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f62540fbc80 0x7f62540fc050 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f6244004940 tx=0x7f6244009e30 comp rx=0 tx=0).stop 2024-03-29T08:31:05.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:05.452+0000 7f6259600700 1 -- 172.21.15.37:0/2194191400 shutdown_connections 2024-03-29T08:31:05.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:05.452+0000 7f6259600700 1 --2- 172.21.15.37:0/2194191400 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f62540fc620 0x7f6254100cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:31:05.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:05.452+0000 7f6259600700 1 --2- 172.21.15.37:0/2194191400 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f62540fbc80 0x7f62540fc050 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:31:05.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:05.452+0000 7f6259600700 1 -- 172.21.15.37:0/2194191400 >> 172.21.15.37:0/2194191400 conn(0x7f6254072bc0 msgr2=0x7f6254072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:31:05.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:05.452+0000 7f6259600700 1 -- 172.21.15.37:0/2194191400 shutdown_connections 2024-03-29T08:31:05.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:05.452+0000 7f6259600700 1 -- 172.21.15.37:0/2194191400 wait complete. 2024-03-29T08:31:05.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:05.453+0000 7f6259600700 1 Processor -- start 2024-03-29T08:31:05.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:05.453+0000 7f6259600700 1 -- start start 2024-03-29T08:31:05.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:05.453+0000 7f6259600700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f62540fbc80 0x7f625411a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:31:05.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:05.453+0000 7f6259600700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f62540fc620 0x7f625411a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:31:05.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:05.453+0000 7f6259600700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f625411aca0 con 0x7f62540fc620 2024-03-29T08:31:05.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:05.453+0000 7f6259600700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6254114070 con 0x7f62540fbc80 2024-03-29T08:31:05.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:05.454+0000 7f6252000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f62540fc620 0x7f625411a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:31:05.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:05.454+0000 7f6252000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f62540fc620 0x7f625411a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45694/0 (socket says 172.21.15.37:45694) 2024-03-29T08:31:05.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:05.454+0000 7f6252000700 1 -- 172.21.15.37:0/999296681 learned_addr learned my addr 172.21.15.37:0/999296681 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:31:05.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:05.454+0000 7f6252a00700 1 --2- 172.21.15.37:0/999296681 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f62540fbc80 0x7f625411a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:31:05.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:05.454+0000 7f6252000700 1 -- 172.21.15.37:0/999296681 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f62540fbc80 msgr2=0x7f625411a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:31:05.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:05.454+0000 7f6252000700 1 --2- 172.21.15.37:0/999296681 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f62540fbc80 0x7f625411a110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:31:05.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:05.454+0000 7f6252000700 1 -- 172.21.15.37:0/999296681 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6244005040 con 0x7f62540fc620 2024-03-29T08:31:05.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:05.454+0000 7f6252000700 1 --2- 172.21.15.37:0/999296681 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f62540fc620 0x7f625411a650 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f6240002a50 tx=0x7f62400046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:31:05.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:05.455+0000 7f624aa00700 1 -- 172.21.15.37:0/999296681 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6240007cc0 con 0x7f62540fc620 2024-03-29T08:31:05.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:05.455+0000 7f624aa00700 1 -- 172.21.15.37:0/999296681 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f624000f040 con 0x7f62540fc620 2024-03-29T08:31:05.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:05.455+0000 7f6259600700 1 -- 172.21.15.37:0/999296681 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6254114350 con 0x7f62540fc620 2024-03-29T08:31:05.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:05.455+0000 7f624aa00700 1 -- 172.21.15.37:0/999296681 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f624000d9c0 con 0x7f62540fc620 2024-03-29T08:31:05.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:05.455+0000 7f6259600700 1 -- 172.21.15.37:0/999296681 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6254114890 con 0x7f62540fc620 2024-03-29T08:31:05.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:05.456+0000 7f6259600700 1 -- 172.21.15.37:0/999296681 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6254065e30 con 0x7f62540fc620 2024-03-29T08:31:05.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:05.457+0000 7f624aa00700 1 -- 172.21.15.37:0/999296681 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f624000db20 con 0x7f62540fc620 2024-03-29T08:31:05.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:05.458+0000 7f624aa00700 1 --2- 172.21.15.37:0/999296681 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f623806cc20 0x7f623806f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:31:05.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:05.458+0000 7f624aa00700 1 -- 172.21.15.37:0/999296681 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f624008bd20 con 0x7f62540fc620 2024-03-29T08:31:05.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:05.458+0000 7f6252a00700 1 --2- 172.21.15.37:0/999296681 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f623806cc20 0x7f623806f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:31:05.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:05.459+0000 7f6252a00700 1 --2- 172.21.15.37:0/999296681 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f623806cc20 0x7f623806f0d0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f6244004f90 tx=0x7f6244009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:31:05.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:05.462+0000 7f624aa00700 1 -- 172.21.15.37:0/999296681 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f6240059770 con 0x7f62540fc620 2024-03-29T08:31:05.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:31:05 smithi037 ceph-mon[142325]: pgmap v605: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:31:05.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:31:05 smithi037 ceph-mon[142325]: from='client.35120 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:31:05.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:05.766+0000 7f6259600700 1 -- 172.21.15.37:0/999296681 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f625404bed0 con 0x7f62540fc620 2024-03-29T08:31:05.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:05.766+0000 7f624aa00700 1 -- 172.21.15.37:0/999296681 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+847 (secure 0 0 0) 0x7f6240059300 con 0x7f62540fc620 2024-03-29T08:31:05.768 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-29T08:31:05.768 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": { 2024-03-29T08:31:05.768 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 2 2024-03-29T08:31:05.768 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:31:05.768 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": { 2024-03-29T08:31:05.769 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 2 2024-03-29T08:31:05.769 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:31:05.769 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": { 2024-03-29T08:31:05.769 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 6 2024-03-29T08:31:05.769 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:31:05.769 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": { 2024-03-29T08:31:05.769 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 17.2.7-789-g4213e17c (4213e17c4de67016ed14f55d22503f4923b3a901) quincy (stable)": 1, 2024-03-29T08:31:05.769 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 3 2024-03-29T08:31:05.769 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:31:05.769 INFO:teuthology.orchestra.run.smithi037.stdout: "overall": { 2024-03-29T08:31:05.769 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 17.2.7-789-g4213e17c (4213e17c4de67016ed14f55d22503f4923b3a901) quincy (stable)": 1, 2024-03-29T08:31:05.769 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 13 2024-03-29T08:31:05.769 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-29T08:31:05.770 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-29T08:31:05.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:05.769+0000 7f6259600700 1 -- 172.21.15.37:0/999296681 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f623806cc20 msgr2=0x7f623806f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:31:05.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:05.769+0000 7f6259600700 1 --2- 172.21.15.37:0/999296681 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f623806cc20 0x7f623806f0d0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f6244004f90 tx=0x7f6244009e30 comp rx=0 tx=0).stop 2024-03-29T08:31:05.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:05.769+0000 7f6259600700 1 -- 172.21.15.37:0/999296681 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f62540fc620 msgr2=0x7f625411a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:31:05.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:05.769+0000 7f6259600700 1 --2- 172.21.15.37:0/999296681 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f62540fc620 0x7f625411a650 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f6240002a50 tx=0x7f62400046b0 comp rx=0 tx=0).stop 2024-03-29T08:31:05.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:05.769+0000 7f6259600700 1 -- 172.21.15.37:0/999296681 shutdown_connections 2024-03-29T08:31:05.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:05.769+0000 7f6259600700 1 --2- 172.21.15.37:0/999296681 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f623806cc20 0x7f623806f0d0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:31:05.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:05.769+0000 7f6259600700 1 --2- 172.21.15.37:0/999296681 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f62540fbc80 0x7f625411a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:31:05.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:05.769+0000 7f6259600700 1 --2- 172.21.15.37:0/999296681 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f62540fc620 0x7f625411a650 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:31:05.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:05.769+0000 7f6259600700 1 -- 172.21.15.37:0/999296681 >> 172.21.15.37:0/999296681 conn(0x7f6254072bc0 msgr2=0x7f625410aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:31:05.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:05.769+0000 7f6259600700 1 -- 172.21.15.37:0/999296681 shutdown_connections 2024-03-29T08:31:05.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:05.769+0000 7f6259600700 1 -- 172.21.15.37:0/999296681 wait complete. 2024-03-29T08:31:05.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:31:05 smithi148 ceph-mon[106692]: pgmap v605: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:31:05.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:31:05 smithi148 ceph-mon[106692]: from='client.35120 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:31:05.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:05.906+0000 7f792ee00700 1 -- 172.21.15.37:0/3059912528 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f79280708a0 msgr2=0x7f7928070d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:31:05.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:05.906+0000 7f792ee00700 1 --2- 172.21.15.37:0/3059912528 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f79280708a0 0x7f7928070d10 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f7914004970 tx=0x7f7914009e30 comp rx=0 tx=0).stop 2024-03-29T08:31:05.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:05.906+0000 7f792ee00700 1 -- 172.21.15.37:0/3059912528 shutdown_connections 2024-03-29T08:31:05.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:05.906+0000 7f792ee00700 1 --2- 172.21.15.37:0/3059912528 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f79280708a0 0x7f7928070d10 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:31:05.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:05.906+0000 7f792ee00700 1 --2- 172.21.15.37:0/3059912528 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7928071f90 0x7f7928070360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:31:05.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:05.906+0000 7f792ee00700 1 -- 172.21.15.37:0/3059912528 >> 172.21.15.37:0/3059912528 conn(0x7f79280f8930 msgr2=0x7f79280fad40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:31:05.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:05.906+0000 7f792ee00700 1 -- 172.21.15.37:0/3059912528 shutdown_connections 2024-03-29T08:31:05.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:05.906+0000 7f792ee00700 1 -- 172.21.15.37:0/3059912528 wait complete. 2024-03-29T08:31:05.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:05.907+0000 7f792ee00700 1 Processor -- start 2024-03-29T08:31:05.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:05.907+0000 7f792ee00700 1 -- start start 2024-03-29T08:31:05.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:05.908+0000 7f792ee00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f79280708a0 0x7f792811a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:31:05.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:05.908+0000 7f792ee00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7928071f90 0x7f792811a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:31:05.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:05.908+0000 7f792ee00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f792811ab90 con 0x7f7928071f90 2024-03-29T08:31:05.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:05.908+0000 7f792ee00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7928113f60 con 0x7f79280708a0 2024-03-29T08:31:05.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:05.908+0000 7f7927400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7928071f90 0x7f792811a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:31:05.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:05.908+0000 7f7927e00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f79280708a0 0x7f792811a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:31:05.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:05.908+0000 7f7927400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7928071f90 0x7f792811a540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45708/0 (socket says 172.21.15.37:45708) 2024-03-29T08:31:05.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:05.908+0000 7f7927400700 1 -- 172.21.15.37:0/2735848640 learned_addr learned my addr 172.21.15.37:0/2735848640 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:31:05.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:05.909+0000 7f7927400700 1 -- 172.21.15.37:0/2735848640 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f79280708a0 msgr2=0x7f792811a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:31:05.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:05.909+0000 7f7927400700 1 --2- 172.21.15.37:0/2735848640 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f79280708a0 0x7f792811a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:31:05.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:05.909+0000 7f7927400700 1 -- 172.21.15.37:0/2735848640 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7914005040 con 0x7f7928071f90 2024-03-29T08:31:05.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:05.909+0000 7f7927400700 1 --2- 172.21.15.37:0/2735848640 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7928071f90 0x7f792811a540 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f7914004f50 tx=0x7f7914009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:31:05.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:05.909+0000 7f791fe00700 1 -- 172.21.15.37:0/2735848640 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f791400e040 con 0x7f7928071f90 2024-03-29T08:31:05.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:05.909+0000 7f792ee00700 1 -- 172.21.15.37:0/2735848640 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79281141e0 con 0x7f7928071f90 2024-03-29T08:31:05.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:05.909+0000 7f791fe00700 1 -- 172.21.15.37:0/2735848640 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f7914013420 con 0x7f7928071f90 2024-03-29T08:31:05.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:05.909+0000 7f791fe00700 1 -- 172.21.15.37:0/2735848640 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f791401c630 con 0x7f7928071f90 2024-03-29T08:31:05.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:05.909+0000 7f792ee00700 1 -- 172.21.15.37:0/2735848640 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7928114720 con 0x7f7928071f90 2024-03-29T08:31:05.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:05.910+0000 7f792ee00700 1 -- 172.21.15.37:0/2735848640 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7928071aa0 con 0x7f7928071f90 2024-03-29T08:31:05.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:05.911+0000 7f791fe00700 1 -- 172.21.15.37:0/2735848640 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f7914013590 con 0x7f7928071f90 2024-03-29T08:31:05.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:05.912+0000 7f791fe00700 1 --2- 172.21.15.37:0/2735848640 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f790c06cc30 0x7f790c06f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:31:05.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:05.912+0000 7f791fe00700 1 -- 172.21.15.37:0/2735848640 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f791408d260 con 0x7f7928071f90 2024-03-29T08:31:05.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:05.912+0000 7f7927e00700 1 --2- 172.21.15.37:0/2735848640 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f790c06cc30 0x7f790c06f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:31:05.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:05.913+0000 7f7927e00700 1 --2- 172.21.15.37:0/2735848640 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f790c06cc30 0x7f790c06f0e0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f7918004f60 tx=0x7f791800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:31:05.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:05.917+0000 7f791fe00700 1 -- 172.21.15.37:0/2735848640 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f791405ade0 con 0x7f7928071f90 2024-03-29T08:31:06.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:06.175+0000 7f792ee00700 1 -- 172.21.15.37:0/2735848640 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f7928115680 con 0x7f7928071f90 2024-03-29T08:31:06.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:06.175+0000 7f791fe00700 1 -- 172.21.15.37:0/2735848640 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 28 v28) v1 ==== 76+0+1841 (secure 0 0 0) 0x7f791405a970 con 0x7f7928071f90 2024-03-29T08:31:06.176 INFO:teuthology.orchestra.run.smithi037.stdout:e28 2024-03-29T08:31:06.176 INFO:teuthology.orchestra.run.smithi037.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-29T08:31:06.176 INFO:teuthology.orchestra.run.smithi037.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T08:31:06.177 INFO:teuthology.orchestra.run.smithi037.stdout:legacy client fscid: 1 2024-03-29T08:31:06.177 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:31:06.177 INFO:teuthology.orchestra.run.smithi037.stdout:Filesystem 'cephfs' (1) 2024-03-29T08:31:06.177 INFO:teuthology.orchestra.run.smithi037.stdout:fs_name cephfs 2024-03-29T08:31:06.177 INFO:teuthology.orchestra.run.smithi037.stdout:epoch 21 2024-03-29T08:31:06.177 INFO:teuthology.orchestra.run.smithi037.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-03-29T08:31:06.177 INFO:teuthology.orchestra.run.smithi037.stdout:created 2024-03-29T08:03:29.241269+0000 2024-03-29T08:31:06.177 INFO:teuthology.orchestra.run.smithi037.stdout:modified 2024-03-29T08:17:16.237238+0000 2024-03-29T08:31:06.177 INFO:teuthology.orchestra.run.smithi037.stdout:tableserver 0 2024-03-29T08:31:06.177 INFO:teuthology.orchestra.run.smithi037.stdout:root 0 2024-03-29T08:31:06.177 INFO:teuthology.orchestra.run.smithi037.stdout:session_timeout 60 2024-03-29T08:31:06.177 INFO:teuthology.orchestra.run.smithi037.stdout:session_autoclose 300 2024-03-29T08:31:06.177 INFO:teuthology.orchestra.run.smithi037.stdout:max_file_size 1099511627776 2024-03-29T08:31:06.177 INFO:teuthology.orchestra.run.smithi037.stdout:max_xattr_size 65536 2024-03-29T08:31:06.177 INFO:teuthology.orchestra.run.smithi037.stdout:required_client_features {} 2024-03-29T08:31:06.178 INFO:teuthology.orchestra.run.smithi037.stdout:last_failure 0 2024-03-29T08:31:06.178 INFO:teuthology.orchestra.run.smithi037.stdout:last_failure_osd_epoch 82 2024-03-29T08:31:06.178 INFO:teuthology.orchestra.run.smithi037.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T08:31:06.178 INFO:teuthology.orchestra.run.smithi037.stdout:max_mds 2 2024-03-29T08:31:06.178 INFO:teuthology.orchestra.run.smithi037.stdout:in 0,1 2024-03-29T08:31:06.178 INFO:teuthology.orchestra.run.smithi037.stdout:up {} 2024-03-29T08:31:06.178 INFO:teuthology.orchestra.run.smithi037.stdout:failed 0,1 2024-03-29T08:31:06.178 INFO:teuthology.orchestra.run.smithi037.stdout:damaged 2024-03-29T08:31:06.178 INFO:teuthology.orchestra.run.smithi037.stdout:stopped 2024-03-29T08:31:06.178 INFO:teuthology.orchestra.run.smithi037.stdout:data_pools [3] 2024-03-29T08:31:06.178 INFO:teuthology.orchestra.run.smithi037.stdout:metadata_pool 2 2024-03-29T08:31:06.178 INFO:teuthology.orchestra.run.smithi037.stdout:inline_data disabled 2024-03-29T08:31:06.178 INFO:teuthology.orchestra.run.smithi037.stdout:balancer 2024-03-29T08:31:06.178 INFO:teuthology.orchestra.run.smithi037.stdout:bal_rank_mask -1 2024-03-29T08:31:06.178 INFO:teuthology.orchestra.run.smithi037.stdout:standby_count_wanted 1 2024-03-29T08:31:06.179 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:31:06.179 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:31:06.179 INFO:teuthology.orchestra.run.smithi037.stdout:Standby daemons: 2024-03-29T08:31:06.179 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:31:06.179 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi148.qukpjh{-1:24319} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.148:6826/2641327912,v1:172.21.15.148:6827/2641327912] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:31:06.179 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi037.mmwgxe{-1:34440} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.37:6828/1248142005,v1:172.21.15.37:6829/1248142005] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:31:06.179 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi037.iifsmh{-1:34444} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.37:6826/2060882959,v1:172.21.15.37:6827/2060882959] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:31:06.179 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi148.sqteih{-1:44395} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.148:6824/4078652242,v1:172.21.15.148:6825/4078652242] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:31:06.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:06.177+0000 7f792ee00700 1 -- 172.21.15.37:0/2735848640 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f790c06cc30 msgr2=0x7f790c06f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:31:06.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:06.177+0000 7f792ee00700 1 --2- 172.21.15.37:0/2735848640 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f790c06cc30 0x7f790c06f0e0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f7918004f60 tx=0x7f791800b040 comp rx=0 tx=0).stop 2024-03-29T08:31:06.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:06.177+0000 7f792ee00700 1 -- 172.21.15.37:0/2735848640 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7928071f90 msgr2=0x7f792811a540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:31:06.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:06.177+0000 7f792ee00700 1 --2- 172.21.15.37:0/2735848640 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7928071f90 0x7f792811a540 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f7914004f50 tx=0x7f7914009e30 comp rx=0 tx=0).stop 2024-03-29T08:31:06.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:06.178+0000 7f792ee00700 1 -- 172.21.15.37:0/2735848640 shutdown_connections 2024-03-29T08:31:06.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:06.178+0000 7f792ee00700 1 --2- 172.21.15.37:0/2735848640 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f790c06cc30 0x7f790c06f0e0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:31:06.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:06.178+0000 7f792ee00700 1 --2- 172.21.15.37:0/2735848640 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f79280708a0 0x7f792811a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:31:06.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:06.178+0000 7f792ee00700 1 --2- 172.21.15.37:0/2735848640 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7928071f90 0x7f792811a540 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:31:06.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:06.178+0000 7f792ee00700 1 -- 172.21.15.37:0/2735848640 >> 172.21.15.37:0/2735848640 conn(0x7f79280f8930 msgr2=0x7f79280f9ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:31:06.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:06.178+0000 7f792ee00700 1 -- 172.21.15.37:0/2735848640 shutdown_connections 2024-03-29T08:31:06.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:06.178+0000 7f792ee00700 1 -- 172.21.15.37:0/2735848640 wait complete. 2024-03-29T08:31:06.181 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 28 2024-03-29T08:31:06.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:06.315+0000 7fa5c0800700 1 -- 172.21.15.37:0/4107736148 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa5b8101d70 msgr2=0x7fa5b8102140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:31:06.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:06.315+0000 7fa5c0800700 1 --2- 172.21.15.37:0/4107736148 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa5b8101d70 0x7fa5b8102140 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fa5ac004940 tx=0x7fa5ac009e30 comp rx=0 tx=0).stop 2024-03-29T08:31:06.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:06.316+0000 7fa5c0800700 1 -- 172.21.15.37:0/4107736148 shutdown_connections 2024-03-29T08:31:06.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:06.316+0000 7fa5c0800700 1 --2- 172.21.15.37:0/4107736148 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa5b8102710 0x7fa5b80fbd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:31:06.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:06.316+0000 7fa5c0800700 1 --2- 172.21.15.37:0/4107736148 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa5b8101d70 0x7fa5b8102140 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:31:06.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:06.316+0000 7fa5c0800700 1 -- 172.21.15.37:0/4107736148 >> 172.21.15.37:0/4107736148 conn(0x7fa5b8072bc0 msgr2=0x7fa5b8072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:31:06.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:06.316+0000 7fa5c0800700 1 -- 172.21.15.37:0/4107736148 shutdown_connections 2024-03-29T08:31:06.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:06.316+0000 7fa5c0800700 1 -- 172.21.15.37:0/4107736148 wait complete. 2024-03-29T08:31:06.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:06.317+0000 7fa5c0800700 1 Processor -- start 2024-03-29T08:31:06.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:06.317+0000 7fa5c0800700 1 -- start start 2024-03-29T08:31:06.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:06.317+0000 7fa5c0800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa5b8101d70 0x7fa5b8113980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:31:06.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:06.317+0000 7fa5c0800700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa5b8102710 0x7fa5b8113ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:31:06.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:06.317+0000 7fa5c0800700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5b8114510 con 0x7fa5b8101d70 2024-03-29T08:31:06.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:06.317+0000 7fa5c0800700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5b810d940 con 0x7fa5b8102710 2024-03-29T08:31:06.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:06.317+0000 7fa5bde00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa5b8101d70 0x7fa5b8113980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:31:06.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:06.317+0000 7fa5bde00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa5b8101d70 0x7fa5b8113980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45730/0 (socket says 172.21.15.37:45730) 2024-03-29T08:31:06.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:06.317+0000 7fa5bde00700 1 -- 172.21.15.37:0/2722246616 learned_addr learned my addr 172.21.15.37:0/2722246616 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:31:06.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:06.318+0000 7fa5bd400700 1 --2- 172.21.15.37:0/2722246616 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa5b8102710 0x7fa5b8113ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:31:06.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:06.318+0000 7fa5bde00700 1 -- 172.21.15.37:0/2722246616 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa5b8102710 msgr2=0x7fa5b8113ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:31:06.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:06.318+0000 7fa5bde00700 1 --2- 172.21.15.37:0/2722246616 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa5b8102710 0x7fa5b8113ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:31:06.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:06.318+0000 7fa5bde00700 1 -- 172.21.15.37:0/2722246616 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5ac005040 con 0x7fa5b8101d70 2024-03-29T08:31:06.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:06.318+0000 7fa5bd400700 1 --2- 172.21.15.37:0/2722246616 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa5b8102710 0x7fa5b8113ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:31:06.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:06.318+0000 7fa5bde00700 1 --2- 172.21.15.37:0/2722246616 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa5b8101d70 0x7fa5b8113980 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fa5ac007f50 tx=0x7fa5ac009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:31:06.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:06.318+0000 7fa5b2000700 1 -- 172.21.15.37:0/2722246616 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa5ac021070 con 0x7fa5b8101d70 2024-03-29T08:31:06.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:06.318+0000 7fa5b2000700 1 -- 172.21.15.37:0/2722246616 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fa5ac013e00 con 0x7fa5b8101d70 2024-03-29T08:31:06.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:06.318+0000 7fa5c0800700 1 -- 172.21.15.37:0/2722246616 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5b810dbc0 con 0x7fa5b8101d70 2024-03-29T08:31:06.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:06.318+0000 7fa5b2000700 1 -- 172.21.15.37:0/2722246616 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa5ac00e070 con 0x7fa5b8101d70 2024-03-29T08:31:06.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:06.319+0000 7fa5c0800700 1 -- 172.21.15.37:0/2722246616 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5b810e180 con 0x7fa5b8101d70 2024-03-29T08:31:06.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:06.320+0000 7fa5c0800700 1 -- 172.21.15.37:0/2722246616 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5b80fd4c0 con 0x7fa5b8101d70 2024-03-29T08:31:06.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:06.321+0000 7fa5b2000700 1 -- 172.21.15.37:0/2722246616 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fa5ac013950 con 0x7fa5b8101d70 2024-03-29T08:31:06.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:06.321+0000 7fa5b2000700 1 --2- 172.21.15.37:0/2722246616 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fa5a006cc30 0x7fa5a006f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:31:06.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:06.322+0000 7fa5b2000700 1 -- 172.21.15.37:0/2722246616 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fa5ac08d350 con 0x7fa5b8101d70 2024-03-29T08:31:06.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:06.322+0000 7fa5bd400700 1 --2- 172.21.15.37:0/2722246616 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fa5a006cc30 0x7fa5a006f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:31:06.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:06.322+0000 7fa5bd400700 1 --2- 172.21.15.37:0/2722246616 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fa5a006cc30 0x7fa5a006f0e0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fa5a8004600 tx=0x7fa5a800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:31:06.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:06.326+0000 7fa5b2000700 1 -- 172.21.15.37:0/2722246616 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fa5ac05ada0 con 0x7fa5b8101d70 2024-03-29T08:31:06.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:06.561+0000 7fa5c0800700 1 -- 172.21.15.37:0/2722246616 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa5b804bf40 con 0x7fa5a006cc30 2024-03-29T08:31:06.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:06.563+0000 7fa5b2000700 1 -- 172.21.15.37:0/2722246616 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fa5b804bf40 con 0x7fa5a006cc30 2024-03-29T08:31:06.564 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-29T08:31:06.564 INFO:teuthology.orchestra.run.smithi037.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59", 2024-03-29T08:31:06.564 INFO:teuthology.orchestra.run.smithi037.stdout: "in_progress": true, 2024-03-29T08:31:06.564 INFO:teuthology.orchestra.run.smithi037.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-29T08:31:06.564 INFO:teuthology.orchestra.run.smithi037.stdout: "services_complete": [ 2024-03-29T08:31:06.564 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr", 2024-03-29T08:31:06.564 INFO:teuthology.orchestra.run.smithi037.stdout: "mon", 2024-03-29T08:31:06.564 INFO:teuthology.orchestra.run.smithi037.stdout: "crash", 2024-03-29T08:31:06.564 INFO:teuthology.orchestra.run.smithi037.stdout: "osd" 2024-03-29T08:31:06.564 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-29T08:31:06.564 INFO:teuthology.orchestra.run.smithi037.stdout: "progress": "15/21 daemons upgraded", 2024-03-29T08:31:06.564 INFO:teuthology.orchestra.run.smithi037.stdout: "message": "Currently upgrading mds daemons", 2024-03-29T08:31:06.565 INFO:teuthology.orchestra.run.smithi037.stdout: "is_paused": false 2024-03-29T08:31:06.565 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-29T08:31:06.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:06.565+0000 7fa5c0800700 1 -- 172.21.15.37:0/2722246616 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fa5a006cc30 msgr2=0x7fa5a006f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:31:06.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:06.565+0000 7fa5c0800700 1 --2- 172.21.15.37:0/2722246616 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fa5a006cc30 0x7fa5a006f0e0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fa5a8004600 tx=0x7fa5a800b040 comp rx=0 tx=0).stop 2024-03-29T08:31:06.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:06.565+0000 7fa5c0800700 1 -- 172.21.15.37:0/2722246616 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa5b8101d70 msgr2=0x7fa5b8113980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:31:06.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:06.565+0000 7fa5c0800700 1 --2- 172.21.15.37:0/2722246616 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa5b8101d70 0x7fa5b8113980 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fa5ac007f50 tx=0x7fa5ac009e30 comp rx=0 tx=0).stop 2024-03-29T08:31:06.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:06.565+0000 7fa5c0800700 1 -- 172.21.15.37:0/2722246616 shutdown_connections 2024-03-29T08:31:06.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:06.565+0000 7fa5c0800700 1 --2- 172.21.15.37:0/2722246616 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fa5a006cc30 0x7fa5a006f0e0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:31:06.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:06.565+0000 7fa5c0800700 1 --2- 172.21.15.37:0/2722246616 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa5b8101d70 0x7fa5b8113980 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:31:06.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:06.565+0000 7fa5c0800700 1 --2- 172.21.15.37:0/2722246616 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa5b8102710 0x7fa5b8113ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:31:06.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:06.565+0000 7fa5c0800700 1 -- 172.21.15.37:0/2722246616 >> 172.21.15.37:0/2722246616 conn(0x7fa5b8072bc0 msgr2=0x7fa5b8065940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:31:06.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:06.565+0000 7fa5c0800700 1 -- 172.21.15.37:0/2722246616 shutdown_connections 2024-03-29T08:31:06.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:06.565+0000 7fa5c0800700 1 -- 172.21.15.37:0/2722246616 wait complete. 2024-03-29T08:31:06.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:31:06 smithi037 ceph-mon[142325]: from='client.35124 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:31:06.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:31:06 smithi037 ceph-mon[142325]: from='client.35128 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:31:06.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:31:06 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/999296681' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:31:06.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:31:06 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/2735848640' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T08:31:06.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:06.703+0000 7fc5e4a00700 1 -- 172.21.15.37:0/3920241466 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc5e00fe850 msgr2=0x7fc5e0106f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:31:06.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:06.703+0000 7fc5e4a00700 1 --2- 172.21.15.37:0/3920241466 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc5e00fe850 0x7fc5e0106f20 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fc5cc004970 tx=0x7fc5cc009e30 comp rx=0 tx=0).stop 2024-03-29T08:31:06.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:06.703+0000 7fc5e4a00700 1 -- 172.21.15.37:0/3920241466 shutdown_connections 2024-03-29T08:31:06.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:06.703+0000 7fc5e4a00700 1 --2- 172.21.15.37:0/3920241466 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc5e00fe850 0x7fc5e0106f20 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:31:06.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:06.704+0000 7fc5e4a00700 1 --2- 172.21.15.37:0/3920241466 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc5e00fdeb0 0x7fc5e00fe280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:31:06.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:06.704+0000 7fc5e4a00700 1 -- 172.21.15.37:0/3920241466 >> 172.21.15.37:0/3920241466 conn(0x7fc5e0072bc0 msgr2=0x7fc5e0072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:31:06.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:06.704+0000 7fc5e4a00700 1 -- 172.21.15.37:0/3920241466 shutdown_connections 2024-03-29T08:31:06.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:06.704+0000 7fc5e4a00700 1 -- 172.21.15.37:0/3920241466 wait complete. 2024-03-29T08:31:06.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:06.704+0000 7fc5e4a00700 1 Processor -- start 2024-03-29T08:31:06.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:06.705+0000 7fc5e4a00700 1 -- start start 2024-03-29T08:31:06.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:06.705+0000 7fc5e4a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc5e00fdeb0 0x7fc5e0115c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:31:06.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:06.705+0000 7fc5e4a00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc5e00fe850 0x7fc5e0116140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:31:06.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:06.705+0000 7fc5e4a00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5e0116790 con 0x7fc5e00fdeb0 2024-03-29T08:31:06.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:06.705+0000 7fc5e4a00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5e010fb80 con 0x7fc5e00fe850 2024-03-29T08:31:06.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:06.705+0000 7fc5dde00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc5e00fdeb0 0x7fc5e0115c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:31:06.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:06.705+0000 7fc5dde00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc5e00fdeb0 0x7fc5e0115c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45744/0 (socket says 172.21.15.37:45744) 2024-03-29T08:31:06.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:06.705+0000 7fc5dde00700 1 -- 172.21.15.37:0/3720393947 learned_addr learned my addr 172.21.15.37:0/3720393947 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:31:06.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:06.705+0000 7fc5dd400700 1 --2- 172.21.15.37:0/3720393947 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc5e00fe850 0x7fc5e0116140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:31:06.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:06.706+0000 7fc5dde00700 1 -- 172.21.15.37:0/3720393947 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc5e00fe850 msgr2=0x7fc5e0116140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:31:06.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:06.706+0000 7fc5dde00700 1 --2- 172.21.15.37:0/3720393947 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc5e00fe850 0x7fc5e0116140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:31:06.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:06.706+0000 7fc5dde00700 1 -- 172.21.15.37:0/3720393947 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5cc005040 con 0x7fc5e00fdeb0 2024-03-29T08:31:06.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:06.706+0000 7fc5dd400700 1 --2- 172.21.15.37:0/3720393947 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc5e00fe850 0x7fc5e0116140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:31:06.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:06.706+0000 7fc5dde00700 1 --2- 172.21.15.37:0/3720393947 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc5e00fdeb0 0x7fc5e0115c00 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fc5d000c8c0 tx=0x7fc5d00044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:31:06.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:06.706+0000 7fc5d6000700 1 -- 172.21.15.37:0/3720393947 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc5d000a980 con 0x7fc5e00fdeb0 2024-03-29T08:31:06.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:06.706+0000 7fc5d6000700 1 -- 172.21.15.37:0/3720393947 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fc5d000ce60 con 0x7fc5e00fdeb0 2024-03-29T08:31:06.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:06.706+0000 7fc5e4a00700 1 -- 172.21.15.37:0/3720393947 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5e010fe60 con 0x7fc5e00fdeb0 2024-03-29T08:31:06.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:06.706+0000 7fc5d6000700 1 -- 172.21.15.37:0/3720393947 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc5d00113a0 con 0x7fc5e00fdeb0 2024-03-29T08:31:06.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:06.706+0000 7fc5e4a00700 1 -- 172.21.15.37:0/3720393947 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5e0110370 con 0x7fc5e00fdeb0 2024-03-29T08:31:06.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:06.708+0000 7fc5e4a00700 1 -- 172.21.15.37:0/3720393947 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5e01069f0 con 0x7fc5e00fdeb0 2024-03-29T08:31:06.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:06.709+0000 7fc5d6000700 1 -- 172.21.15.37:0/3720393947 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fc5d0012400 con 0x7fc5e00fdeb0 2024-03-29T08:31:06.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:06.710+0000 7fc5d6000700 1 --2- 172.21.15.37:0/3720393947 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fc5c406cc30 0x7fc5c406f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:31:06.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:06.710+0000 7fc5d6000700 1 -- 172.21.15.37:0/3720393947 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fc5d008c1d0 con 0x7fc5e00fdeb0 2024-03-29T08:31:06.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:06.710+0000 7fc5dd400700 1 --2- 172.21.15.37:0/3720393947 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fc5c406cc30 0x7fc5c406f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:31:06.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:06.711+0000 7fc5dd400700 1 --2- 172.21.15.37:0/3720393947 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fc5c406cc30 0x7fc5c406f0e0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fc5cc009fd0 tx=0x7fc5cc009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:31:06.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:06.715+0000 7fc5d6000700 1 -- 172.21.15.37:0/3720393947 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fc5d0059c20 con 0x7fc5e00fdeb0 2024-03-29T08:31:06.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:31:06 smithi148 ceph-mon[106692]: from='client.35124 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:31:06.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:31:06 smithi148 ceph-mon[106692]: from='client.35128 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:31:06.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:31:06 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/999296681' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:31:06.807 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:31:06 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/2735848640' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T08:31:07.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:07.017+0000 7fc5e4a00700 1 -- 172.21.15.37:0/3720393947 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fc5e0109e50 con 0x7fc5e00fdeb0 2024-03-29T08:31:07.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:07.017+0000 7fc5d6000700 1 -- 172.21.15.37:0/3720393947 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7fc5d00597b0 con 0x7fc5e00fdeb0 2024-03-29T08:31:07.019 INFO:teuthology.orchestra.run.smithi037.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-03-29T08:31:07.019 INFO:teuthology.orchestra.run.smithi037.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-03-29T08:31:07.019 INFO:teuthology.orchestra.run.smithi037.stdout: fs cephfs is degraded 2024-03-29T08:31:07.019 INFO:teuthology.orchestra.run.smithi037.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-03-29T08:31:07.019 INFO:teuthology.orchestra.run.smithi037.stdout: fs cephfs is offline because no MDS is active for it. 2024-03-29T08:31:07.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:07.020+0000 7fc5e4a00700 1 -- 172.21.15.37:0/3720393947 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fc5c406cc30 msgr2=0x7fc5c406f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:31:07.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:07.020+0000 7fc5e4a00700 1 --2- 172.21.15.37:0/3720393947 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fc5c406cc30 0x7fc5c406f0e0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fc5cc009fd0 tx=0x7fc5cc009e30 comp rx=0 tx=0).stop 2024-03-29T08:31:07.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:07.020+0000 7fc5e4a00700 1 -- 172.21.15.37:0/3720393947 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc5e00fdeb0 msgr2=0x7fc5e0115c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:31:07.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:07.020+0000 7fc5e4a00700 1 --2- 172.21.15.37:0/3720393947 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc5e00fdeb0 0x7fc5e0115c00 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fc5d000c8c0 tx=0x7fc5d00044c0 comp rx=0 tx=0).stop 2024-03-29T08:31:07.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:07.020+0000 7fc5e4a00700 1 -- 172.21.15.37:0/3720393947 shutdown_connections 2024-03-29T08:31:07.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:07.020+0000 7fc5e4a00700 1 --2- 172.21.15.37:0/3720393947 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fc5c406cc30 0x7fc5c406f0e0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:31:07.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:07.020+0000 7fc5e4a00700 1 --2- 172.21.15.37:0/3720393947 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc5e00fdeb0 0x7fc5e0115c00 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:31:07.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:07.020+0000 7fc5e4a00700 1 --2- 172.21.15.37:0/3720393947 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc5e00fe850 0x7fc5e0116140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:31:07.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:07.020+0000 7fc5e4a00700 1 -- 172.21.15.37:0/3720393947 >> 172.21.15.37:0/3720393947 conn(0x7fc5e0072bc0 msgr2=0x7fc5e00fb640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:31:07.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:07.020+0000 7fc5e4a00700 1 -- 172.21.15.37:0/3720393947 shutdown_connections 2024-03-29T08:31:07.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:07.020+0000 7fc5e4a00700 1 -- 172.21.15.37:0/3720393947 wait complete. 2024-03-29T08:31:07.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:31:07 smithi037 ceph-mon[142325]: pgmap v606: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:31:07.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:31:07 smithi037 ceph-mon[142325]: from='client.35140 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:31:07.678 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:31:07 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/3720393947' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T08:31:07.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:31:07 smithi148 ceph-mon[106692]: pgmap v606: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:31:07.807 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:31:07 smithi148 ceph-mon[106692]: from='client.35140 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:31:07.807 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:31:07 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/3720393947' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T08:31:09.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:31:09 smithi037 ceph-mon[142325]: pgmap v607: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:31:09.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:31:09 smithi148 ceph-mon[106692]: pgmap v607: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:31:10.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:31:10 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:31:10.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:31:10 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:31:11.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:31:11 smithi037 ceph-mon[142325]: pgmap v608: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:31:11.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:31:11 smithi148 ceph-mon[106692]: pgmap v608: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:31:13.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:31:13 smithi037 ceph-mon[142325]: pgmap v609: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:31:13.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:31:13 smithi148 ceph-mon[106692]: pgmap v609: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:31:15.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:31:15 smithi037 ceph-mon[142325]: pgmap v610: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:31:15.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:31:15 smithi148 ceph-mon[106692]: pgmap v610: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:31:17.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:31:17 smithi037 ceph-mon[142325]: pgmap v611: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:31:17.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:31:17 smithi148 ceph-mon[106692]: pgmap v611: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:31:19.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:31:19 smithi037 ceph-mon[142325]: pgmap v612: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:31:19.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:31:19 smithi148 ceph-mon[106692]: pgmap v612: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:31:21.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:31:21 smithi037 ceph-mon[142325]: pgmap v613: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:31:21.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:31:21 smithi148 ceph-mon[106692]: pgmap v613: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:31:23.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:31:23 smithi037 ceph-mon[142325]: pgmap v614: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:31:23.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:31:23 smithi148 ceph-mon[106692]: pgmap v614: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:31:25.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:31:25 smithi037 ceph-mon[142325]: pgmap v615: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:31:25.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:31:25 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:31:25.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:31:25 smithi148 ceph-mon[106692]: pgmap v615: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:31:25.807 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:31:25 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:31:27.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:31:27 smithi037 ceph-mon[142325]: pgmap v616: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:31:27.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:31:27 smithi148 ceph-mon[106692]: pgmap v616: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:31:29.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:31:29 smithi037 ceph-mon[142325]: pgmap v617: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:31:29.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:31:29 smithi148 ceph-mon[106692]: pgmap v617: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:31:31.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:31:31 smithi037 ceph-mon[142325]: pgmap v618: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:31:31.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:31:31 smithi148 ceph-mon[106692]: pgmap v618: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:31:33.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:31:33 smithi037 ceph-mon[142325]: pgmap v619: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:31:33.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:31:33 smithi148 ceph-mon[106692]: pgmap v619: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:31:35.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:31:35 smithi148 ceph-mon[106692]: pgmap v620: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:31:35.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:31:35 smithi037 ceph-mon[142325]: pgmap v620: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:31:37.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:37.175+0000 7f0437200700 1 -- 172.21.15.37:0/1815166282 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f04301037a0 msgr2=0x7f04300fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:31:37.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:37.175+0000 7f0437200700 1 --2- 172.21.15.37:0/1815166282 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f04301037a0 0x7f04300fce10 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f041c004940 tx=0x7f041c009e30 comp rx=0 tx=0).stop 2024-03-29T08:31:37.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:37.175+0000 7f0437200700 1 -- 172.21.15.37:0/1815166282 shutdown_connections 2024-03-29T08:31:37.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:37.175+0000 7f0437200700 1 --2- 172.21.15.37:0/1815166282 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f04301037a0 0x7f04300fce10 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:31:37.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:37.175+0000 7f0437200700 1 --2- 172.21.15.37:0/1815166282 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0430102e00 0x7f04301031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:31:37.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:37.175+0000 7f0437200700 1 -- 172.21.15.37:0/1815166282 >> 172.21.15.37:0/1815166282 conn(0x7f04300f89b0 msgr2=0x7f04300fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:31:37.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:37.175+0000 7f0437200700 1 -- 172.21.15.37:0/1815166282 shutdown_connections 2024-03-29T08:31:37.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:37.176+0000 7f0437200700 1 -- 172.21.15.37:0/1815166282 wait complete. 2024-03-29T08:31:37.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:37.176+0000 7f0437200700 1 Processor -- start 2024-03-29T08:31:37.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:37.176+0000 7f0437200700 1 -- start start 2024-03-29T08:31:37.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:37.177+0000 7f0437200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0430102e00 0x7f04300702c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:31:37.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:37.177+0000 7f0437200700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f04301037a0 0x7f0430070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:31:37.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:37.177+0000 7f0437200700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0430070d90 con 0x7f0430102e00 2024-03-29T08:31:37.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:37.177+0000 7f0437200700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0430070ed0 con 0x7f04301037a0 2024-03-29T08:31:37.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:37.177+0000 7f042fe00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0430102e00 0x7f04300702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:31:37.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:37.177+0000 7f042fe00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0430102e00 0x7f04300702c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51304/0 (socket says 172.21.15.37:51304) 2024-03-29T08:31:37.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:37.177+0000 7f042fe00700 1 -- 172.21.15.37:0/3672096904 learned_addr learned my addr 172.21.15.37:0/3672096904 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:31:37.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:37.177+0000 7f042f400700 1 --2- 172.21.15.37:0/3672096904 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f04301037a0 0x7f0430070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:31:37.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:37.178+0000 7f042fe00700 1 -- 172.21.15.37:0/3672096904 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f04301037a0 msgr2=0x7f0430070800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:31:37.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:37.178+0000 7f042fe00700 1 --2- 172.21.15.37:0/3672096904 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f04301037a0 0x7f0430070800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:31:37.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:37.178+0000 7f042fe00700 1 -- 172.21.15.37:0/3672096904 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f041c005040 con 0x7f0430102e00 2024-03-29T08:31:37.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:37.178+0000 7f042fe00700 1 --2- 172.21.15.37:0/3672096904 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0430102e00 0x7f04300702c0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f0420002a50 tx=0x7f04200046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:31:37.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:37.178+0000 7f0427e00700 1 -- 172.21.15.37:0/3672096904 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0420007cc0 con 0x7f0430102e00 2024-03-29T08:31:37.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:37.178+0000 7f0427e00700 1 -- 172.21.15.37:0/3672096904 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f042000f040 con 0x7f0430102e00 2024-03-29T08:31:37.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:37.178+0000 7f0427e00700 1 -- 172.21.15.37:0/3672096904 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f042000d9c0 con 0x7f0430102e00 2024-03-29T08:31:37.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:37.178+0000 7f0437200700 1 -- 172.21.15.37:0/3672096904 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0430119e00 con 0x7f0430102e00 2024-03-29T08:31:37.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:37.178+0000 7f0437200700 1 -- 172.21.15.37:0/3672096904 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f043011a220 con 0x7f0430102e00 2024-03-29T08:31:37.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:37.179+0000 7f0437200700 1 -- 172.21.15.37:0/3672096904 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04300fe550 con 0x7f0430102e00 2024-03-29T08:31:37.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:37.181+0000 7f0427e00700 1 -- 172.21.15.37:0/3672096904 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f042000db20 con 0x7f0430102e00 2024-03-29T08:31:37.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:37.182+0000 7f0427e00700 1 --2- 172.21.15.37:0/3672096904 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f041406cc30 0x7f041406f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:31:37.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:37.183+0000 7f0427e00700 1 -- 172.21.15.37:0/3672096904 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f042008be20 con 0x7f0430102e00 2024-03-29T08:31:37.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:37.183+0000 7f042f400700 1 --2- 172.21.15.37:0/3672096904 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f041406cc30 0x7f041406f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:31:37.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:37.184+0000 7f042f400700 1 --2- 172.21.15.37:0/3672096904 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f041406cc30 0x7f041406f0e0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f0430071c00 tx=0x7f041c009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:31:37.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:37.188+0000 7f0427e00700 1 -- 172.21.15.37:0/3672096904 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f0420055d60 con 0x7f0430102e00 2024-03-29T08:31:37.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:37.423+0000 7f0437200700 1 -- 172.21.15.37:0/3672096904 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f04300719b0 con 0x7f041406cc30 2024-03-29T08:31:37.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:37.424+0000 7f0427e00700 1 -- 172.21.15.37:0/3672096904 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f04300719b0 con 0x7f041406cc30 2024-03-29T08:31:37.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:37.426+0000 7f0437200700 1 -- 172.21.15.37:0/3672096904 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f041406cc30 msgr2=0x7f041406f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:31:37.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:37.426+0000 7f0437200700 1 --2- 172.21.15.37:0/3672096904 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f041406cc30 0x7f041406f0e0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f0430071c00 tx=0x7f041c009e30 comp rx=0 tx=0).stop 2024-03-29T08:31:37.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:37.426+0000 7f0437200700 1 -- 172.21.15.37:0/3672096904 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0430102e00 msgr2=0x7f04300702c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:31:37.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:37.426+0000 7f0437200700 1 --2- 172.21.15.37:0/3672096904 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0430102e00 0x7f04300702c0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f0420002a50 tx=0x7f04200046b0 comp rx=0 tx=0).stop 2024-03-29T08:31:37.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:37.426+0000 7f0437200700 1 -- 172.21.15.37:0/3672096904 shutdown_connections 2024-03-29T08:31:37.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:37.426+0000 7f0437200700 1 --2- 172.21.15.37:0/3672096904 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f041406cc30 0x7f041406f0e0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:31:37.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:37.426+0000 7f0437200700 1 --2- 172.21.15.37:0/3672096904 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0430102e00 0x7f04300702c0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:31:37.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:37.426+0000 7f0437200700 1 --2- 172.21.15.37:0/3672096904 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f04301037a0 0x7f0430070800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:31:37.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:37.426+0000 7f0437200700 1 -- 172.21.15.37:0/3672096904 >> 172.21.15.37:0/3672096904 conn(0x7f04300f89b0 msgr2=0x7f04300f9ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:31:37.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:37.426+0000 7f0437200700 1 -- 172.21.15.37:0/3672096904 shutdown_connections 2024-03-29T08:31:37.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:37.427+0000 7f0437200700 1 -- 172.21.15.37:0/3672096904 wait complete. 2024-03-29T08:31:37.441 INFO:teuthology.orchestra.run.smithi037.stdout:true 2024-03-29T08:31:37.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:37.564+0000 7f5029600700 1 -- 172.21.15.37:0/515776511 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f50240fe830 msgr2=0x7f5024106f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:31:37.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:37.564+0000 7f5029600700 1 --2- 172.21.15.37:0/515776511 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f50240fe830 0x7f5024106f00 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f5010004940 tx=0x7f5010009e30 comp rx=0 tx=0).stop 2024-03-29T08:31:37.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:37.564+0000 7f5029600700 1 -- 172.21.15.37:0/515776511 shutdown_connections 2024-03-29T08:31:37.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:37.564+0000 7f5029600700 1 --2- 172.21.15.37:0/515776511 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f50240fe830 0x7f5024106f00 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:31:37.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:37.564+0000 7f5029600700 1 --2- 172.21.15.37:0/515776511 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f50240fde90 0x7f50240fe260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:31:37.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:37.564+0000 7f5029600700 1 -- 172.21.15.37:0/515776511 >> 172.21.15.37:0/515776511 conn(0x7f5024072bc0 msgr2=0x7f5024072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:31:37.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:37.565+0000 7f5029600700 1 -- 172.21.15.37:0/515776511 shutdown_connections 2024-03-29T08:31:37.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:37.565+0000 7f5029600700 1 -- 172.21.15.37:0/515776511 wait complete. 2024-03-29T08:31:37.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:37.565+0000 7f5029600700 1 Processor -- start 2024-03-29T08:31:37.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:37.566+0000 7f5029600700 1 -- start start 2024-03-29T08:31:37.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:37.566+0000 7f5029600700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f50240fde90 0x7f5024115d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:31:37.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:37.566+0000 7f5029600700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f50240fe830 0x7f5024116270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:31:37.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:37.566+0000 7f5029600700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50241168c0 con 0x7f50240fde90 2024-03-29T08:31:37.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:37.566+0000 7f5029600700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f502410fc90 con 0x7f50240fe830 2024-03-29T08:31:37.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:37.566+0000 7f5022a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f50240fde90 0x7f5024115d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:31:37.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:37.566+0000 7f5022a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f50240fde90 0x7f5024115d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51328/0 (socket says 172.21.15.37:51328) 2024-03-29T08:31:37.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:37.566+0000 7f5022a00700 1 -- 172.21.15.37:0/4288006208 learned_addr learned my addr 172.21.15.37:0/4288006208 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:31:37.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:37.567+0000 7f5022000700 1 --2- 172.21.15.37:0/4288006208 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f50240fe830 0x7f5024116270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:31:37.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:37.567+0000 7f5022a00700 1 -- 172.21.15.37:0/4288006208 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f50240fe830 msgr2=0x7f5024116270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:31:37.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:37.567+0000 7f5022a00700 1 --2- 172.21.15.37:0/4288006208 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f50240fe830 0x7f5024116270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:31:37.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:37.567+0000 7f5022a00700 1 -- 172.21.15.37:0/4288006208 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5010005040 con 0x7f50240fde90 2024-03-29T08:31:37.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:37.567+0000 7f5022a00700 1 --2- 172.21.15.37:0/4288006208 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f50240fde90 0x7f5024115d30 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f5014002a50 tx=0x7f50140046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:31:37.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:37.567+0000 7f501aa00700 1 -- 172.21.15.37:0/4288006208 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5014007cc0 con 0x7f50240fde90 2024-03-29T08:31:37.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:37.567+0000 7f501aa00700 1 -- 172.21.15.37:0/4288006208 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f501400f040 con 0x7f50240fde90 2024-03-29T08:31:37.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:37.567+0000 7f5029600700 1 -- 172.21.15.37:0/4288006208 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f502410ff70 con 0x7f50240fde90 2024-03-29T08:31:37.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:37.567+0000 7f501aa00700 1 -- 172.21.15.37:0/4288006208 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5014009540 con 0x7f50240fde90 2024-03-29T08:31:37.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:37.567+0000 7f5029600700 1 -- 172.21.15.37:0/4288006208 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50241104b0 con 0x7f50240fde90 2024-03-29T08:31:37.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:37.569+0000 7f5029600700 1 -- 172.21.15.37:0/4288006208 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5024106a20 con 0x7f50240fde90 2024-03-29T08:31:37.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:37.569+0000 7f501aa00700 1 -- 172.21.15.37:0/4288006208 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f50140096a0 con 0x7f50240fde90 2024-03-29T08:31:37.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:37.570+0000 7f501aa00700 1 --2- 172.21.15.37:0/4288006208 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f500806cc30 0x7f500806f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:31:37.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:37.570+0000 7f501aa00700 1 -- 172.21.15.37:0/4288006208 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f501408bee0 con 0x7f50240fde90 2024-03-29T08:31:37.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:37.571+0000 7f5022000700 1 --2- 172.21.15.37:0/4288006208 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f500806cc30 0x7f500806f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:31:37.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:37.571+0000 7f5022000700 1 --2- 172.21.15.37:0/4288006208 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f500806cc30 0x7f500806f0e0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f50241115d0 tx=0x7f5010009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:31:37.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:37.575+0000 7f501aa00700 1 -- 172.21.15.37:0/4288006208 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f50140598b0 con 0x7f50240fde90 2024-03-29T08:31:37.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:31:37 smithi148 ceph-mon[106692]: pgmap v621: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:31:37.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:37.809+0000 7f5029600700 1 -- 172.21.15.37:0/4288006208 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5024109e40 con 0x7f500806cc30 2024-03-29T08:31:37.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:37.810+0000 7f501aa00700 1 -- 172.21.15.37:0/4288006208 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f5024109e40 con 0x7f500806cc30 2024-03-29T08:31:37.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:37.812+0000 7f5029600700 1 -- 172.21.15.37:0/4288006208 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f500806cc30 msgr2=0x7f500806f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:31:37.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:37.812+0000 7f5029600700 1 --2- 172.21.15.37:0/4288006208 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f500806cc30 0x7f500806f0e0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f50241115d0 tx=0x7f5010009e30 comp rx=0 tx=0).stop 2024-03-29T08:31:37.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:37.812+0000 7f5029600700 1 -- 172.21.15.37:0/4288006208 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f50240fde90 msgr2=0x7f5024115d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:31:37.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:37.812+0000 7f5029600700 1 --2- 172.21.15.37:0/4288006208 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f50240fde90 0x7f5024115d30 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f5014002a50 tx=0x7f50140046b0 comp rx=0 tx=0).stop 2024-03-29T08:31:37.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:37.812+0000 7f5029600700 1 -- 172.21.15.37:0/4288006208 shutdown_connections 2024-03-29T08:31:37.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:37.812+0000 7f5029600700 1 --2- 172.21.15.37:0/4288006208 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f500806cc30 0x7f500806f0e0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:31:37.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:37.812+0000 7f5029600700 1 --2- 172.21.15.37:0/4288006208 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f50240fde90 0x7f5024115d30 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:31:37.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:37.812+0000 7f5029600700 1 --2- 172.21.15.37:0/4288006208 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f50240fe830 0x7f5024116270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:31:37.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:37.812+0000 7f5029600700 1 -- 172.21.15.37:0/4288006208 >> 172.21.15.37:0/4288006208 conn(0x7f5024072bc0 msgr2=0x7f50240fb840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:31:37.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:37.813+0000 7f5029600700 1 -- 172.21.15.37:0/4288006208 shutdown_connections 2024-03-29T08:31:37.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:37.813+0000 7f5029600700 1 -- 172.21.15.37:0/4288006208 wait complete. 2024-03-29T08:31:37.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:31:37 smithi037 ceph-mon[142325]: pgmap v621: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:31:37.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:37.950+0000 7f9a3cc00700 1 -- 172.21.15.37:0/3050070167 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9a380fc590 msgr2=0x7f9a381092f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:31:37.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:37.950+0000 7f9a3cc00700 1 --2- 172.21.15.37:0/3050070167 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9a380fc590 0x7f9a381092f0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f9a24004940 tx=0x7f9a24009e30 comp rx=0 tx=0).stop 2024-03-29T08:31:37.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:37.950+0000 7f9a3cc00700 1 -- 172.21.15.37:0/3050070167 shutdown_connections 2024-03-29T08:31:37.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:37.950+0000 7f9a3cc00700 1 --2- 172.21.15.37:0/3050070167 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9a380fc590 0x7f9a381092f0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:31:37.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:37.950+0000 7f9a3cc00700 1 --2- 172.21.15.37:0/3050070167 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9a380fbc80 0x7f9a380fc050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:31:37.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:37.950+0000 7f9a3cc00700 1 -- 172.21.15.37:0/3050070167 >> 172.21.15.37:0/3050070167 conn(0x7f9a38072bc0 msgr2=0x7f9a38072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:31:37.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:37.950+0000 7f9a3cc00700 1 -- 172.21.15.37:0/3050070167 shutdown_connections 2024-03-29T08:31:37.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:37.950+0000 7f9a3cc00700 1 -- 172.21.15.37:0/3050070167 wait complete. 2024-03-29T08:31:37.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:37.951+0000 7f9a3cc00700 1 Processor -- start 2024-03-29T08:31:37.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:37.951+0000 7f9a3cc00700 1 -- start start 2024-03-29T08:31:37.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:37.952+0000 7f9a3cc00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9a380fbc80 0x7f9a38115c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:31:37.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:37.952+0000 7f9a3cc00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9a380fc590 0x7f9a381161b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:31:37.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:37.952+0000 7f9a3cc00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a38116800 con 0x7f9a380fbc80 2024-03-29T08:31:37.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:37.952+0000 7f9a3cc00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a3810fbd0 con 0x7f9a380fc590 2024-03-29T08:31:37.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:37.952+0000 7f9a35e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9a380fbc80 0x7f9a38115c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:31:37.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:37.952+0000 7f9a35e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9a380fbc80 0x7f9a38115c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51348/0 (socket says 172.21.15.37:51348) 2024-03-29T08:31:37.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:37.952+0000 7f9a35e00700 1 -- 172.21.15.37:0/4218208820 learned_addr learned my addr 172.21.15.37:0/4218208820 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:31:37.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:37.952+0000 7f9a35400700 1 --2- 172.21.15.37:0/4218208820 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9a380fc590 0x7f9a381161b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:31:37.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:37.952+0000 7f9a35e00700 1 -- 172.21.15.37:0/4218208820 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9a380fc590 msgr2=0x7f9a381161b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:31:37.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:37.952+0000 7f9a35e00700 1 --2- 172.21.15.37:0/4218208820 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9a380fc590 0x7f9a381161b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:31:37.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:37.952+0000 7f9a35e00700 1 -- 172.21.15.37:0/4218208820 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a24005040 con 0x7f9a380fbc80 2024-03-29T08:31:37.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:37.953+0000 7f9a35e00700 1 --2- 172.21.15.37:0/4218208820 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9a380fbc80 0x7f9a38115c70 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f9a28002a50 tx=0x7f9a280046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:31:37.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:37.953+0000 7f9a2e000700 1 -- 172.21.15.37:0/4218208820 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9a28007cc0 con 0x7f9a380fbc80 2024-03-29T08:31:37.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:37.953+0000 7f9a2e000700 1 -- 172.21.15.37:0/4218208820 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f9a2800f040 con 0x7f9a380fbc80 2024-03-29T08:31:37.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:37.953+0000 7f9a3cc00700 1 -- 172.21.15.37:0/4218208820 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a3810feb0 con 0x7f9a380fbc80 2024-03-29T08:31:37.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:37.953+0000 7f9a2e000700 1 -- 172.21.15.37:0/4218208820 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9a2800d9c0 con 0x7f9a380fbc80 2024-03-29T08:31:37.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:37.953+0000 7f9a3cc00700 1 -- 172.21.15.37:0/4218208820 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a38110470 con 0x7f9a380fbc80 2024-03-29T08:31:37.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:37.954+0000 7f9a3cc00700 1 -- 172.21.15.37:0/4218208820 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a38065e30 con 0x7f9a380fbc80 2024-03-29T08:31:37.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:37.955+0000 7f9a2e000700 1 -- 172.21.15.37:0/4218208820 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f9a2800db20 con 0x7f9a380fbc80 2024-03-29T08:31:37.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:37.956+0000 7f9a2e000700 1 --2- 172.21.15.37:0/4218208820 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f9a1c06cc30 0x7f9a1c06f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:31:37.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:37.956+0000 7f9a2e000700 1 -- 172.21.15.37:0/4218208820 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f9a2808bde0 con 0x7f9a380fbc80 2024-03-29T08:31:37.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:37.956+0000 7f9a35400700 1 --2- 172.21.15.37:0/4218208820 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f9a1c06cc30 0x7f9a1c06f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:31:37.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:37.957+0000 7f9a35400700 1 --2- 172.21.15.37:0/4218208820 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f9a1c06cc30 0x7f9a1c06f0e0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f9a2400b840 tx=0x7f9a24009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:31:37.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:37.961+0000 7f9a2e000700 1 -- 172.21.15.37:0/4218208820 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f9a280597b0 con 0x7f9a380fbc80 2024-03-29T08:31:38.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:38.182+0000 7f9a3cc00700 1 -- 172.21.15.37:0/4218208820 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f9a3804bed0 con 0x7f9a1c06cc30 2024-03-29T08:31:38.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:38.192+0000 7f9a2e000700 1 -- 172.21.15.37:0/4218208820 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f9a3804bed0 con 0x7f9a1c06cc30 2024-03-29T08:31:38.193 INFO:teuthology.orchestra.run.smithi037.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-29T08:31:38.193 INFO:teuthology.orchestra.run.smithi037.stdout:alertmanager.smithi037 smithi037 *:9093,9094 running (21m) 13m ago 34m 14.9M - 0.25.0 c8568f914cd2 2f9f9746ed0f 2024-03-29T08:31:38.193 INFO:teuthology.orchestra.run.smithi037.stdout:crash.smithi037 smithi037 running (18m) 13m ago 34m 6647k - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 178d416a3186 2024-03-29T08:31:38.193 INFO:teuthology.orchestra.run.smithi037.stdout:crash.smithi148 smithi148 running (18m) 13m ago 33m 6656k - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 23666989d58f 2024-03-29T08:31:38.193 INFO:teuthology.orchestra.run.smithi037.stdout:grafana.smithi037 smithi037 *:3000 running (21m) 13m ago 34m 71.9M - 9.4.7 954c08fa6188 6b976fe49952 2024-03-29T08:31:38.193 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi037.iifsmh smithi037 running (13m) 13m ago 27m 10.8M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 2d97f562cc5b 2024-03-29T08:31:38.193 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi037.mmwgxe smithi037 running (14m) 13m ago 28m 10.9M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff ec37037f46e8 2024-03-29T08:31:38.193 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi148.qukpjh smithi148 running (27m) 13m ago 27m 15.3M - 17.2.7-789-g4213e17c cf22eb8683be 8e469782a273 2024-03-29T08:31:38.193 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi148.sqteih smithi148 running (13m) 13m ago 28m 10.7M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 44c15c77037a 2024-03-29T08:31:38.193 INFO:teuthology.orchestra.run.smithi037.stdout:mgr.smithi037.wjikii smithi037 *:8443,9283,8765 running (23m) 13m ago 36m 506M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff c5a1417b9417 2024-03-29T08:31:38.193 INFO:teuthology.orchestra.run.smithi037.stdout:mgr.smithi148.onhgvm smithi148 *:8443,9283,8765 running (22m) 13m ago 33m 450M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff fcfdbce062bd 2024-03-29T08:31:38.193 INFO:teuthology.orchestra.run.smithi037.stdout:mon.smithi037 smithi037 running (19m) 13m ago 36m 54.8M 2048M 18.2.2-1015-gd0e392d9 1cc2f45c17ff eea2060783af 2024-03-29T08:31:38.193 INFO:teuthology.orchestra.run.smithi037.stdout:mon.smithi148 smithi148 running (19m) 13m ago 33m 43.7M 2048M 18.2.2-1015-gd0e392d9 1cc2f45c17ff ad4b2c5748be 2024-03-29T08:31:38.194 INFO:teuthology.orchestra.run.smithi037.stdout:node-exporter.smithi037 smithi037 *:9100 running (22m) 13m ago 34m 7625k - 1.5.0 0da6a335fe13 26b8ae9cb878 2024-03-29T08:31:38.194 INFO:teuthology.orchestra.run.smithi037.stdout:node-exporter.smithi148 smithi148 *:9100 running (22m) 13m ago 33m 8166k - 1.5.0 0da6a335fe13 98414e4e64ed 2024-03-29T08:31:38.194 INFO:teuthology.orchestra.run.smithi037.stdout:osd.0 smithi037 running (17m) 13m ago 32m 104M 1619M 18.2.2-1015-gd0e392d9 1cc2f45c17ff c0c584f6245c 2024-03-29T08:31:38.194 INFO:teuthology.orchestra.run.smithi037.stdout:osd.1 smithi037 running (17m) 13m ago 31m 100M 1619M 18.2.2-1015-gd0e392d9 1cc2f45c17ff a4a2ff4cc83d 2024-03-29T08:31:38.194 INFO:teuthology.orchestra.run.smithi037.stdout:osd.2 smithi037 running (16m) 13m ago 31m 82.3M 1619M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 8b1bc71aa79e 2024-03-29T08:31:38.194 INFO:teuthology.orchestra.run.smithi037.stdout:osd.3 smithi148 running (15m) 13m ago 30m 105M 2642M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 1c656ab8ff86 2024-03-29T08:31:38.194 INFO:teuthology.orchestra.run.smithi037.stdout:osd.4 smithi148 running (15m) 13m ago 30m 97.3M 2642M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 966a9de11d22 2024-03-29T08:31:38.194 INFO:teuthology.orchestra.run.smithi037.stdout:osd.5 smithi148 running (14m) 13m ago 29m 85.8M 2642M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 474c4f9212ce 2024-03-29T08:31:38.194 INFO:teuthology.orchestra.run.smithi037.stdout:prometheus.smithi037 smithi037 *:9095 running (21m) 13m ago 33m 54.4M - 2.43.0 a07b618ecd1d 30c0223ec589 2024-03-29T08:31:38.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:38.195+0000 7f9a3cc00700 1 -- 172.21.15.37:0/4218208820 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f9a1c06cc30 msgr2=0x7f9a1c06f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:31:38.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:38.195+0000 7f9a3cc00700 1 --2- 172.21.15.37:0/4218208820 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f9a1c06cc30 0x7f9a1c06f0e0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f9a2400b840 tx=0x7f9a24009e30 comp rx=0 tx=0).stop 2024-03-29T08:31:38.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:38.195+0000 7f9a3cc00700 1 -- 172.21.15.37:0/4218208820 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9a380fbc80 msgr2=0x7f9a38115c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:31:38.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:38.195+0000 7f9a3cc00700 1 --2- 172.21.15.37:0/4218208820 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9a380fbc80 0x7f9a38115c70 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f9a28002a50 tx=0x7f9a280046b0 comp rx=0 tx=0).stop 2024-03-29T08:31:38.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:38.195+0000 7f9a3cc00700 1 -- 172.21.15.37:0/4218208820 shutdown_connections 2024-03-29T08:31:38.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:38.195+0000 7f9a3cc00700 1 --2- 172.21.15.37:0/4218208820 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f9a1c06cc30 0x7f9a1c06f0e0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:31:38.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:38.195+0000 7f9a3cc00700 1 --2- 172.21.15.37:0/4218208820 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9a380fbc80 0x7f9a38115c70 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:31:38.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:38.195+0000 7f9a3cc00700 1 --2- 172.21.15.37:0/4218208820 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9a380fc590 0x7f9a381161b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:31:38.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:38.195+0000 7f9a3cc00700 1 -- 172.21.15.37:0/4218208820 >> 172.21.15.37:0/4218208820 conn(0x7f9a38072bc0 msgr2=0x7f9a38105bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:31:38.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:38.195+0000 7f9a3cc00700 1 -- 172.21.15.37:0/4218208820 shutdown_connections 2024-03-29T08:31:38.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:38.195+0000 7f9a3cc00700 1 -- 172.21.15.37:0/4218208820 wait complete. 2024-03-29T08:31:38.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:38.331+0000 7f0527a00700 1 -- 172.21.15.37:0/3228507628 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f05201037a0 msgr2=0x7f05200fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:31:38.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:38.331+0000 7f0527a00700 1 --2- 172.21.15.37:0/3228507628 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f05201037a0 0x7f05200fce10 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f050c004940 tx=0x7f050c009e30 comp rx=0 tx=0).stop 2024-03-29T08:31:38.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:38.331+0000 7f0527a00700 1 -- 172.21.15.37:0/3228507628 shutdown_connections 2024-03-29T08:31:38.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:38.331+0000 7f0527a00700 1 --2- 172.21.15.37:0/3228507628 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f05201037a0 0x7f05200fce10 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:31:38.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:38.331+0000 7f0527a00700 1 --2- 172.21.15.37:0/3228507628 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0520102e00 0x7f05201031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:31:38.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:38.331+0000 7f0527a00700 1 -- 172.21.15.37:0/3228507628 >> 172.21.15.37:0/3228507628 conn(0x7f05200f89b0 msgr2=0x7f05200fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:31:38.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:38.331+0000 7f0527a00700 1 -- 172.21.15.37:0/3228507628 shutdown_connections 2024-03-29T08:31:38.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:38.331+0000 7f0527a00700 1 -- 172.21.15.37:0/3228507628 wait complete. 2024-03-29T08:31:38.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:38.332+0000 7f0527a00700 1 Processor -- start 2024-03-29T08:31:38.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:38.332+0000 7f0527a00700 1 -- start start 2024-03-29T08:31:38.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:38.333+0000 7f0527a00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0520102e00 0x7f0520115c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:31:38.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:38.333+0000 7f0527a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f05201037a0 0x7f05201161c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:31:38.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:38.333+0000 7f0527a00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0520116810 con 0x7f05201037a0 2024-03-29T08:31:38.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:38.333+0000 7f0527a00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f052010fbe0 con 0x7f0520102e00 2024-03-29T08:31:38.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:38.333+0000 7f051fe00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f05201037a0 0x7f05201161c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:31:38.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:38.333+0000 7f051fe00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f05201037a0 0x7f05201161c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51360/0 (socket says 172.21.15.37:51360) 2024-03-29T08:31:38.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:38.333+0000 7f051fe00700 1 -- 172.21.15.37:0/2571067314 learned_addr learned my addr 172.21.15.37:0/2571067314 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:31:38.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:38.333+0000 7f0525000700 1 --2- 172.21.15.37:0/2571067314 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0520102e00 0x7f0520115c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:31:38.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:38.333+0000 7f051fe00700 1 -- 172.21.15.37:0/2571067314 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0520102e00 msgr2=0x7f0520115c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:31:38.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:38.333+0000 7f051fe00700 1 --2- 172.21.15.37:0/2571067314 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0520102e00 0x7f0520115c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:31:38.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:38.333+0000 7f051fe00700 1 -- 172.21.15.37:0/2571067314 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f050c005040 con 0x7f05201037a0 2024-03-29T08:31:38.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:38.334+0000 7f051fe00700 1 --2- 172.21.15.37:0/2571067314 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f05201037a0 0x7f05201161c0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f050c009fd0 tx=0x7f050c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:31:38.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:38.334+0000 7f051cc00700 1 -- 172.21.15.37:0/2571067314 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f050c00e030 con 0x7f05201037a0 2024-03-29T08:31:38.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:38.334+0000 7f0527a00700 1 -- 172.21.15.37:0/2571067314 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f052010fe60 con 0x7f05201037a0 2024-03-29T08:31:38.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:38.334+0000 7f051cc00700 1 -- 172.21.15.37:0/2571067314 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f050c0093b0 con 0x7f05201037a0 2024-03-29T08:31:38.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:38.334+0000 7f051cc00700 1 -- 172.21.15.37:0/2571067314 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f050c01cc80 con 0x7f05201037a0 2024-03-29T08:31:38.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:38.334+0000 7f0527a00700 1 -- 172.21.15.37:0/2571067314 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0520110420 con 0x7f05201037a0 2024-03-29T08:31:38.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:38.335+0000 7f0527a00700 1 -- 172.21.15.37:0/2571067314 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05200fe550 con 0x7f05201037a0 2024-03-29T08:31:38.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:38.336+0000 7f051cc00700 1 -- 172.21.15.37:0/2571067314 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f050c01c400 con 0x7f05201037a0 2024-03-29T08:31:38.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:38.337+0000 7f051cc00700 1 --2- 172.21.15.37:0/2571067314 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f050806cc30 0x7f050806f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:31:38.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:38.337+0000 7f051cc00700 1 -- 172.21.15.37:0/2571067314 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f050c08d8b0 con 0x7f05201037a0 2024-03-29T08:31:38.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:38.337+0000 7f0525000700 1 --2- 172.21.15.37:0/2571067314 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f050806cc30 0x7f050806f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:31:38.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:38.338+0000 7f0525000700 1 --2- 172.21.15.37:0/2571067314 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f050806cc30 0x7f050806f0e0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f0514009b80 tx=0x7f051400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:31:38.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:38.342+0000 7f051cc00700 1 -- 172.21.15.37:0/2571067314 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f050c05b300 con 0x7f05201037a0 2024-03-29T08:31:38.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:38.644+0000 7f0527a00700 1 -- 172.21.15.37:0/2571067314 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f052004bed0 con 0x7f05201037a0 2024-03-29T08:31:38.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:38.644+0000 7f051cc00700 1 -- 172.21.15.37:0/2571067314 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+847 (secure 0 0 0) 0x7f050c08be60 con 0x7f05201037a0 2024-03-29T08:31:38.645 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-29T08:31:38.645 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": { 2024-03-29T08:31:38.645 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 2 2024-03-29T08:31:38.645 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:31:38.645 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": { 2024-03-29T08:31:38.645 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 2 2024-03-29T08:31:38.645 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:31:38.645 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": { 2024-03-29T08:31:38.645 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 6 2024-03-29T08:31:38.645 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:31:38.645 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": { 2024-03-29T08:31:38.645 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 17.2.7-789-g4213e17c (4213e17c4de67016ed14f55d22503f4923b3a901) quincy (stable)": 1, 2024-03-29T08:31:38.645 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 3 2024-03-29T08:31:38.645 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:31:38.645 INFO:teuthology.orchestra.run.smithi037.stdout: "overall": { 2024-03-29T08:31:38.646 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 17.2.7-789-g4213e17c (4213e17c4de67016ed14f55d22503f4923b3a901) quincy (stable)": 1, 2024-03-29T08:31:38.646 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 13 2024-03-29T08:31:38.646 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-29T08:31:38.646 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-29T08:31:38.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:38.647+0000 7f0527a00700 1 -- 172.21.15.37:0/2571067314 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f050806cc30 msgr2=0x7f050806f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:31:38.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:38.647+0000 7f0527a00700 1 --2- 172.21.15.37:0/2571067314 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f050806cc30 0x7f050806f0e0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f0514009b80 tx=0x7f051400b040 comp rx=0 tx=0).stop 2024-03-29T08:31:38.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:38.647+0000 7f0527a00700 1 -- 172.21.15.37:0/2571067314 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f05201037a0 msgr2=0x7f05201161c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:31:38.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:38.647+0000 7f0527a00700 1 --2- 172.21.15.37:0/2571067314 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f05201037a0 0x7f05201161c0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f050c009fd0 tx=0x7f050c009e30 comp rx=0 tx=0).stop 2024-03-29T08:31:38.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:38.647+0000 7f0527a00700 1 -- 172.21.15.37:0/2571067314 shutdown_connections 2024-03-29T08:31:38.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:38.647+0000 7f0527a00700 1 --2- 172.21.15.37:0/2571067314 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f050806cc30 0x7f050806f0e0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:31:38.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:38.647+0000 7f0527a00700 1 --2- 172.21.15.37:0/2571067314 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0520102e00 0x7f0520115c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:31:38.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:38.647+0000 7f0527a00700 1 --2- 172.21.15.37:0/2571067314 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f05201037a0 0x7f05201161c0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:31:38.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:38.647+0000 7f0527a00700 1 -- 172.21.15.37:0/2571067314 >> 172.21.15.37:0/2571067314 conn(0x7f05200f89b0 msgr2=0x7f05200f99a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:31:38.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:38.647+0000 7f0527a00700 1 -- 172.21.15.37:0/2571067314 shutdown_connections 2024-03-29T08:31:38.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:38.647+0000 7f0527a00700 1 -- 172.21.15.37:0/2571067314 wait complete. 2024-03-29T08:31:38.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:38.785+0000 7f4d09800700 1 -- 172.21.15.37:0/1440773553 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4d04102e20 msgr2=0x7f4d041031f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:31:38.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:38.785+0000 7f4d09800700 1 --2- 172.21.15.37:0/1440773553 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4d04102e20 0x7f4d041031f0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f4cf4004940 tx=0x7f4cf4009e30 comp rx=0 tx=0).stop 2024-03-29T08:31:38.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:38.785+0000 7f4d09800700 1 -- 172.21.15.37:0/1440773553 shutdown_connections 2024-03-29T08:31:38.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:38.785+0000 7f4d09800700 1 --2- 172.21.15.37:0/1440773553 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4d041037c0 0x7f4d040fce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:31:38.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:38.785+0000 7f4d09800700 1 --2- 172.21.15.37:0/1440773553 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4d04102e20 0x7f4d041031f0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:31:38.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:38.785+0000 7f4d09800700 1 -- 172.21.15.37:0/1440773553 >> 172.21.15.37:0/1440773553 conn(0x7f4d040f8970 msgr2=0x7f4d040fad80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:31:38.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:38.785+0000 7f4d09800700 1 -- 172.21.15.37:0/1440773553 shutdown_connections 2024-03-29T08:31:38.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:38.785+0000 7f4d09800700 1 -- 172.21.15.37:0/1440773553 wait complete. 2024-03-29T08:31:38.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:38.786+0000 7f4d09800700 1 Processor -- start 2024-03-29T08:31:38.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:38.787+0000 7f4d09800700 1 -- start start 2024-03-29T08:31:38.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:38.787+0000 7f4d09800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4d04102e20 0x7f4d04115d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:31:38.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:38.787+0000 7f4d09800700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4d041037c0 0x7f4d04116270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:31:38.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:38.787+0000 7f4d09800700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d04116950 con 0x7f4d04102e20 2024-03-29T08:31:38.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:38.787+0000 7f4d09800700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d0410fc90 con 0x7f4d041037c0 2024-03-29T08:31:38.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:38.788+0000 7f4d02a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4d04102e20 0x7f4d04115d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:31:38.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:38.788+0000 7f4d02a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4d04102e20 0x7f4d04115d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51372/0 (socket says 172.21.15.37:51372) 2024-03-29T08:31:38.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:38.788+0000 7f4d02a00700 1 -- 172.21.15.37:0/2605836419 learned_addr learned my addr 172.21.15.37:0/2605836419 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:31:38.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:38.788+0000 7f4d02000700 1 --2- 172.21.15.37:0/2605836419 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4d041037c0 0x7f4d04116270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:31:38.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:38.788+0000 7f4d02a00700 1 -- 172.21.15.37:0/2605836419 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4d041037c0 msgr2=0x7f4d04116270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:31:38.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:38.788+0000 7f4d02a00700 1 --2- 172.21.15.37:0/2605836419 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4d041037c0 0x7f4d04116270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:31:38.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:38.788+0000 7f4d02a00700 1 -- 172.21.15.37:0/2605836419 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cf4005040 con 0x7f4d04102e20 2024-03-29T08:31:38.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:38.788+0000 7f4d02000700 1 --2- 172.21.15.37:0/2605836419 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4d041037c0 0x7f4d04116270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:31:38.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:38.788+0000 7f4d02a00700 1 --2- 172.21.15.37:0/2605836419 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4d04102e20 0x7f4d04115d30 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f4cf4009360 tx=0x7f4cf4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:31:38.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:38.789+0000 7f4cfaa00700 1 -- 172.21.15.37:0/2605836419 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4cf400e040 con 0x7f4d04102e20 2024-03-29T08:31:38.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:38.789+0000 7f4cfaa00700 1 -- 172.21.15.37:0/2605836419 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f4cf4013420 con 0x7f4d04102e20 2024-03-29T08:31:38.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:38.789+0000 7f4d09800700 1 -- 172.21.15.37:0/2605836419 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d0410feb0 con 0x7f4d04102e20 2024-03-29T08:31:38.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:38.789+0000 7f4cfaa00700 1 -- 172.21.15.37:0/2605836419 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4cf401c5f0 con 0x7f4d04102e20 2024-03-29T08:31:38.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:38.789+0000 7f4d09800700 1 -- 172.21.15.37:0/2605836419 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d04110470 con 0x7f4d04102e20 2024-03-29T08:31:38.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:38.790+0000 7f4d09800700 1 -- 172.21.15.37:0/2605836419 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d040fe570 con 0x7f4d04102e20 2024-03-29T08:31:38.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:38.791+0000 7f4cfaa00700 1 -- 172.21.15.37:0/2605836419 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f4cf4013590 con 0x7f4d04102e20 2024-03-29T08:31:38.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:38.792+0000 7f4cfaa00700 1 --2- 172.21.15.37:0/2605836419 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f4ce806cc30 0x7f4ce806f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:31:38.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:38.792+0000 7f4cfaa00700 1 -- 172.21.15.37:0/2605836419 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f4cf408d320 con 0x7f4d04102e20 2024-03-29T08:31:38.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:38.792+0000 7f4d02000700 1 --2- 172.21.15.37:0/2605836419 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f4ce806cc30 0x7f4ce806f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:31:38.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:38.792+0000 7f4d02000700 1 --2- 172.21.15.37:0/2605836419 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f4ce806cc30 0x7f4ce806f0e0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f4cf0006fd0 tx=0x7f4cf0008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:31:38.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:38.796+0000 7f4cfaa00700 1 -- 172.21.15.37:0/2605836419 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f4cf405ae20 con 0x7f4d04102e20 2024-03-29T08:31:38.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:31:38 smithi148 ceph-mon[106692]: from='client.35148 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:31:38.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:31:38 smithi037 ceph-mon[142325]: from='client.35148 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:31:39.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:39.054+0000 7f4d09800700 1 -- 172.21.15.37:0/2605836419 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f4d0404bed0 con 0x7f4d04102e20 2024-03-29T08:31:39.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:39.054+0000 7f4cfaa00700 1 -- 172.21.15.37:0/2605836419 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 28 v28) v1 ==== 76+0+1841 (secure 0 0 0) 0x7f4cf405a9b0 con 0x7f4d04102e20 2024-03-29T08:31:39.055 INFO:teuthology.orchestra.run.smithi037.stdout:e28 2024-03-29T08:31:39.055 INFO:teuthology.orchestra.run.smithi037.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-29T08:31:39.055 INFO:teuthology.orchestra.run.smithi037.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T08:31:39.055 INFO:teuthology.orchestra.run.smithi037.stdout:legacy client fscid: 1 2024-03-29T08:31:39.055 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:31:39.055 INFO:teuthology.orchestra.run.smithi037.stdout:Filesystem 'cephfs' (1) 2024-03-29T08:31:39.055 INFO:teuthology.orchestra.run.smithi037.stdout:fs_name cephfs 2024-03-29T08:31:39.055 INFO:teuthology.orchestra.run.smithi037.stdout:epoch 21 2024-03-29T08:31:39.055 INFO:teuthology.orchestra.run.smithi037.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-03-29T08:31:39.055 INFO:teuthology.orchestra.run.smithi037.stdout:created 2024-03-29T08:03:29.241269+0000 2024-03-29T08:31:39.055 INFO:teuthology.orchestra.run.smithi037.stdout:modified 2024-03-29T08:17:16.237238+0000 2024-03-29T08:31:39.055 INFO:teuthology.orchestra.run.smithi037.stdout:tableserver 0 2024-03-29T08:31:39.055 INFO:teuthology.orchestra.run.smithi037.stdout:root 0 2024-03-29T08:31:39.055 INFO:teuthology.orchestra.run.smithi037.stdout:session_timeout 60 2024-03-29T08:31:39.056 INFO:teuthology.orchestra.run.smithi037.stdout:session_autoclose 300 2024-03-29T08:31:39.056 INFO:teuthology.orchestra.run.smithi037.stdout:max_file_size 1099511627776 2024-03-29T08:31:39.056 INFO:teuthology.orchestra.run.smithi037.stdout:max_xattr_size 65536 2024-03-29T08:31:39.056 INFO:teuthology.orchestra.run.smithi037.stdout:required_client_features {} 2024-03-29T08:31:39.056 INFO:teuthology.orchestra.run.smithi037.stdout:last_failure 0 2024-03-29T08:31:39.056 INFO:teuthology.orchestra.run.smithi037.stdout:last_failure_osd_epoch 82 2024-03-29T08:31:39.056 INFO:teuthology.orchestra.run.smithi037.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T08:31:39.056 INFO:teuthology.orchestra.run.smithi037.stdout:max_mds 2 2024-03-29T08:31:39.056 INFO:teuthology.orchestra.run.smithi037.stdout:in 0,1 2024-03-29T08:31:39.056 INFO:teuthology.orchestra.run.smithi037.stdout:up {} 2024-03-29T08:31:39.056 INFO:teuthology.orchestra.run.smithi037.stdout:failed 0,1 2024-03-29T08:31:39.056 INFO:teuthology.orchestra.run.smithi037.stdout:damaged 2024-03-29T08:31:39.056 INFO:teuthology.orchestra.run.smithi037.stdout:stopped 2024-03-29T08:31:39.056 INFO:teuthology.orchestra.run.smithi037.stdout:data_pools [3] 2024-03-29T08:31:39.056 INFO:teuthology.orchestra.run.smithi037.stdout:metadata_pool 2 2024-03-29T08:31:39.056 INFO:teuthology.orchestra.run.smithi037.stdout:inline_data disabled 2024-03-29T08:31:39.057 INFO:teuthology.orchestra.run.smithi037.stdout:balancer 2024-03-29T08:31:39.057 INFO:teuthology.orchestra.run.smithi037.stdout:bal_rank_mask -1 2024-03-29T08:31:39.057 INFO:teuthology.orchestra.run.smithi037.stdout:standby_count_wanted 1 2024-03-29T08:31:39.057 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:31:39.057 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:31:39.057 INFO:teuthology.orchestra.run.smithi037.stdout:Standby daemons: 2024-03-29T08:31:39.057 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:31:39.057 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi148.qukpjh{-1:24319} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.148:6826/2641327912,v1:172.21.15.148:6827/2641327912] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:31:39.057 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi037.mmwgxe{-1:34440} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.37:6828/1248142005,v1:172.21.15.37:6829/1248142005] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:31:39.057 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi037.iifsmh{-1:34444} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.37:6826/2060882959,v1:172.21.15.37:6827/2060882959] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:31:39.057 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi148.sqteih{-1:44395} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.148:6824/4078652242,v1:172.21.15.148:6825/4078652242] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:31:39.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:39.056+0000 7f4d09800700 1 -- 172.21.15.37:0/2605836419 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f4ce806cc30 msgr2=0x7f4ce806f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:31:39.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:39.056+0000 7f4d09800700 1 --2- 172.21.15.37:0/2605836419 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f4ce806cc30 0x7f4ce806f0e0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f4cf0006fd0 tx=0x7f4cf0008040 comp rx=0 tx=0).stop 2024-03-29T08:31:39.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:39.056+0000 7f4d09800700 1 -- 172.21.15.37:0/2605836419 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4d04102e20 msgr2=0x7f4d04115d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:31:39.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:39.056+0000 7f4d09800700 1 --2- 172.21.15.37:0/2605836419 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4d04102e20 0x7f4d04115d30 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f4cf4009360 tx=0x7f4cf4009e30 comp rx=0 tx=0).stop 2024-03-29T08:31:39.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:39.057+0000 7f4d09800700 1 -- 172.21.15.37:0/2605836419 shutdown_connections 2024-03-29T08:31:39.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:39.057+0000 7f4d09800700 1 --2- 172.21.15.37:0/2605836419 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f4ce806cc30 0x7f4ce806f0e0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:31:39.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:39.057+0000 7f4d09800700 1 --2- 172.21.15.37:0/2605836419 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4d04102e20 0x7f4d04115d30 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:31:39.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:39.057+0000 7f4d09800700 1 --2- 172.21.15.37:0/2605836419 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4d041037c0 0x7f4d04116270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:31:39.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:39.057+0000 7f4d09800700 1 -- 172.21.15.37:0/2605836419 >> 172.21.15.37:0/2605836419 conn(0x7f4d040f8970 msgr2=0x7f4d04070480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:31:39.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:39.057+0000 7f4d09800700 1 -- 172.21.15.37:0/2605836419 shutdown_connections 2024-03-29T08:31:39.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:39.057+0000 7f4d09800700 1 -- 172.21.15.37:0/2605836419 wait complete. 2024-03-29T08:31:39.059 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 28 2024-03-29T08:31:39.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:39.194+0000 7f00d6600700 1 -- 172.21.15.37:0/2460791609 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f00d00fde90 msgr2=0x7f00d00fe260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:31:39.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:39.194+0000 7f00d6600700 1 --2- 172.21.15.37:0/2460791609 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f00d00fde90 0x7f00d00fe260 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f00c0004940 tx=0x7f00c0009e30 comp rx=0 tx=0).stop 2024-03-29T08:31:39.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:39.194+0000 7f00d6600700 1 -- 172.21.15.37:0/2460791609 shutdown_connections 2024-03-29T08:31:39.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:39.194+0000 7f00d6600700 1 --2- 172.21.15.37:0/2460791609 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f00d00fe830 0x7f00d0106f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:31:39.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:39.194+0000 7f00d6600700 1 --2- 172.21.15.37:0/2460791609 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f00d00fde90 0x7f00d00fe260 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:31:39.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:39.194+0000 7f00d6600700 1 -- 172.21.15.37:0/2460791609 >> 172.21.15.37:0/2460791609 conn(0x7f00d0072bc0 msgr2=0x7f00d0072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:31:39.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:39.194+0000 7f00d6600700 1 -- 172.21.15.37:0/2460791609 shutdown_connections 2024-03-29T08:31:39.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:39.194+0000 7f00d6600700 1 -- 172.21.15.37:0/2460791609 wait complete. 2024-03-29T08:31:39.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:39.195+0000 7f00d6600700 1 Processor -- start 2024-03-29T08:31:39.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:39.195+0000 7f00d6600700 1 -- start start 2024-03-29T08:31:39.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:39.196+0000 7f00d6600700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f00d00fde90 0x7f00d0115d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:31:39.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:39.196+0000 7f00d6600700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f00d00fe830 0x7f00d0116240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:31:39.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:39.196+0000 7f00d6600700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00d0116890 con 0x7f00d00fe830 2024-03-29T08:31:39.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:39.196+0000 7f00d6600700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00d010fc60 con 0x7f00d00fde90 2024-03-29T08:31:39.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:39.196+0000 7f00cea00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f00d00fe830 0x7f00d0116240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:31:39.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:39.196+0000 7f00cea00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f00d00fe830 0x7f00d0116240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51392/0 (socket says 172.21.15.37:51392) 2024-03-29T08:31:39.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:39.196+0000 7f00cea00700 1 -- 172.21.15.37:0/1182958420 learned_addr learned my addr 172.21.15.37:0/1182958420 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:31:39.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:39.196+0000 7f00cf400700 1 --2- 172.21.15.37:0/1182958420 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f00d00fde90 0x7f00d0115d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:31:39.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:39.197+0000 7f00cea00700 1 -- 172.21.15.37:0/1182958420 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f00d00fde90 msgr2=0x7f00d0115d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:31:39.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:39.197+0000 7f00cea00700 1 --2- 172.21.15.37:0/1182958420 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f00d00fde90 0x7f00d0115d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:31:39.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:39.197+0000 7f00cea00700 1 -- 172.21.15.37:0/1182958420 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00c0005040 con 0x7f00d00fe830 2024-03-29T08:31:39.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:39.197+0000 7f00cea00700 1 --2- 172.21.15.37:0/1182958420 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f00d00fe830 0x7f00d0116240 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f00bc002a50 tx=0x7f00bc0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:31:39.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:39.197+0000 7f00c7400700 1 -- 172.21.15.37:0/1182958420 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f00bc007cc0 con 0x7f00d00fe830 2024-03-29T08:31:39.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:39.197+0000 7f00c7400700 1 -- 172.21.15.37:0/1182958420 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f00bc00f040 con 0x7f00d00fe830 2024-03-29T08:31:39.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:39.197+0000 7f00d6600700 1 -- 172.21.15.37:0/1182958420 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00d010ff40 con 0x7f00d00fe830 2024-03-29T08:31:39.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:39.197+0000 7f00c7400700 1 -- 172.21.15.37:0/1182958420 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f00bc00d9c0 con 0x7f00d00fe830 2024-03-29T08:31:39.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:39.197+0000 7f00d6600700 1 -- 172.21.15.37:0/1182958420 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00d0110480 con 0x7f00d00fe830 2024-03-29T08:31:39.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:39.198+0000 7f00d6600700 1 -- 172.21.15.37:0/1182958420 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00d0106a20 con 0x7f00d00fe830 2024-03-29T08:31:39.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:39.199+0000 7f00c7400700 1 -- 172.21.15.37:0/1182958420 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f00bc00db20 con 0x7f00d00fe830 2024-03-29T08:31:39.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:39.200+0000 7f00c7400700 1 --2- 172.21.15.37:0/1182958420 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f00b406cc30 0x7f00b406f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:31:39.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:39.200+0000 7f00c7400700 1 -- 172.21.15.37:0/1182958420 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f00bc08bd20 con 0x7f00d00fe830 2024-03-29T08:31:39.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:39.201+0000 7f00cf400700 1 --2- 172.21.15.37:0/1182958420 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f00b406cc30 0x7f00b406f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:31:39.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:39.201+0000 7f00cf400700 1 --2- 172.21.15.37:0/1182958420 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f00b406cc30 0x7f00b406f0e0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f00c000b3b0 tx=0x7f00c0009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:31:39.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:39.205+0000 7f00c7400700 1 -- 172.21.15.37:0/1182958420 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f00bc059770 con 0x7f00d00fe830 2024-03-29T08:31:39.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:39.439+0000 7f00d6600700 1 -- 172.21.15.37:0/1182958420 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f00d01113c0 con 0x7f00b406cc30 2024-03-29T08:31:39.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:39.440+0000 7f00c7400700 1 -- 172.21.15.37:0/1182958420 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f00d01113c0 con 0x7f00b406cc30 2024-03-29T08:31:39.440 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-29T08:31:39.440 INFO:teuthology.orchestra.run.smithi037.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59", 2024-03-29T08:31:39.440 INFO:teuthology.orchestra.run.smithi037.stdout: "in_progress": true, 2024-03-29T08:31:39.440 INFO:teuthology.orchestra.run.smithi037.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-29T08:31:39.440 INFO:teuthology.orchestra.run.smithi037.stdout: "services_complete": [ 2024-03-29T08:31:39.440 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr", 2024-03-29T08:31:39.441 INFO:teuthology.orchestra.run.smithi037.stdout: "mon", 2024-03-29T08:31:39.441 INFO:teuthology.orchestra.run.smithi037.stdout: "crash", 2024-03-29T08:31:39.441 INFO:teuthology.orchestra.run.smithi037.stdout: "osd" 2024-03-29T08:31:39.441 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-29T08:31:39.441 INFO:teuthology.orchestra.run.smithi037.stdout: "progress": "15/21 daemons upgraded", 2024-03-29T08:31:39.441 INFO:teuthology.orchestra.run.smithi037.stdout: "message": "Currently upgrading mds daemons", 2024-03-29T08:31:39.441 INFO:teuthology.orchestra.run.smithi037.stdout: "is_paused": false 2024-03-29T08:31:39.441 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-29T08:31:39.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:39.442+0000 7f00d6600700 1 -- 172.21.15.37:0/1182958420 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f00b406cc30 msgr2=0x7f00b406f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:31:39.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:39.442+0000 7f00d6600700 1 --2- 172.21.15.37:0/1182958420 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f00b406cc30 0x7f00b406f0e0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f00c000b3b0 tx=0x7f00c0009e30 comp rx=0 tx=0).stop 2024-03-29T08:31:39.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:39.442+0000 7f00d6600700 1 -- 172.21.15.37:0/1182958420 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f00d00fe830 msgr2=0x7f00d0116240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:31:39.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:39.442+0000 7f00d6600700 1 --2- 172.21.15.37:0/1182958420 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f00d00fe830 0x7f00d0116240 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f00bc002a50 tx=0x7f00bc0046b0 comp rx=0 tx=0).stop 2024-03-29T08:31:39.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:39.442+0000 7f00d6600700 1 -- 172.21.15.37:0/1182958420 shutdown_connections 2024-03-29T08:31:39.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:39.442+0000 7f00d6600700 1 --2- 172.21.15.37:0/1182958420 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f00b406cc30 0x7f00b406f0e0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:31:39.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:39.442+0000 7f00d6600700 1 --2- 172.21.15.37:0/1182958420 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f00d00fde90 0x7f00d0115d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:31:39.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:39.442+0000 7f00d6600700 1 --2- 172.21.15.37:0/1182958420 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f00d00fe830 0x7f00d0116240 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:31:39.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:39.442+0000 7f00d6600700 1 -- 172.21.15.37:0/1182958420 >> 172.21.15.37:0/1182958420 conn(0x7f00d0072bc0 msgr2=0x7f00d00fd590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:31:39.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:39.443+0000 7f00d6600700 1 -- 172.21.15.37:0/1182958420 shutdown_connections 2024-03-29T08:31:39.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:39.443+0000 7f00d6600700 1 -- 172.21.15.37:0/1182958420 wait complete. 2024-03-29T08:31:39.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:39.580+0000 7f1866c00700 1 -- 172.21.15.37:0/815075390 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1860070c10 msgr2=0x7f1860109170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:31:39.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:39.580+0000 7f1866c00700 1 --2- 172.21.15.37:0/815075390 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1860070c10 0x7f1860109170 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f184c004970 tx=0x7f184c009e30 comp rx=0 tx=0).stop 2024-03-29T08:31:39.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:39.580+0000 7f1866c00700 1 -- 172.21.15.37:0/815075390 shutdown_connections 2024-03-29T08:31:39.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:39.580+0000 7f1866c00700 1 --2- 172.21.15.37:0/815075390 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1860070c10 0x7f1860109170 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:31:39.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:39.580+0000 7f1866c00700 1 --2- 172.21.15.37:0/815075390 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1860070300 0x7f18600706d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:31:39.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:39.580+0000 7f1866c00700 1 -- 172.21.15.37:0/815075390 >> 172.21.15.37:0/815075390 conn(0x7f18600f89b0 msgr2=0x7f18600fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:31:39.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:39.580+0000 7f1866c00700 1 -- 172.21.15.37:0/815075390 shutdown_connections 2024-03-29T08:31:39.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:39.580+0000 7f1866c00700 1 -- 172.21.15.37:0/815075390 wait complete. 2024-03-29T08:31:39.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:39.581+0000 7f1866c00700 1 Processor -- start 2024-03-29T08:31:39.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:39.581+0000 7f1866c00700 1 -- start start 2024-03-29T08:31:39.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:39.582+0000 7f1866c00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1860070300 0x7f1860115a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:31:39.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:39.582+0000 7f1866c00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1860070c10 0x7f1860115fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:31:39.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:39.582+0000 7f1866c00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18601166a0 con 0x7f1860070c10 2024-03-29T08:31:39.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:39.582+0000 7f1866c00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f186010fc30 con 0x7f1860070300 2024-03-29T08:31:39.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:39.582+0000 7f185f400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1860070c10 0x7f1860115fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:31:39.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:39.582+0000 7f185f400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1860070c10 0x7f1860115fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51418/0 (socket says 172.21.15.37:51418) 2024-03-29T08:31:39.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:39.582+0000 7f185f400700 1 -- 172.21.15.37:0/1924501301 learned_addr learned my addr 172.21.15.37:0/1924501301 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:31:39.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:39.582+0000 7f185fe00700 1 --2- 172.21.15.37:0/1924501301 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1860070300 0x7f1860115a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:31:39.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:39.582+0000 7f185f400700 1 -- 172.21.15.37:0/1924501301 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1860070300 msgr2=0x7f1860115a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:31:39.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:39.582+0000 7f185f400700 1 --2- 172.21.15.37:0/1924501301 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1860070300 0x7f1860115a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:31:39.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:39.582+0000 7f185f400700 1 -- 172.21.15.37:0/1924501301 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f184c005040 con 0x7f1860070c10 2024-03-29T08:31:39.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:39.582+0000 7f185fe00700 1 --2- 172.21.15.37:0/1924501301 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1860070300 0x7f1860115a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:31:39.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:39.583+0000 7f185f400700 1 --2- 172.21.15.37:0/1924501301 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1860070c10 0x7f1860115fc0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f184c009380 tx=0x7f184c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:31:39.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:39.583+0000 7f1857e00700 1 -- 172.21.15.37:0/1924501301 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f184c00e040 con 0x7f1860070c10 2024-03-29T08:31:39.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:39.583+0000 7f1857e00700 1 -- 172.21.15.37:0/1924501301 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f184c013420 con 0x7f1860070c10 2024-03-29T08:31:39.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:39.583+0000 7f1866c00700 1 -- 172.21.15.37:0/1924501301 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f186010feb0 con 0x7f1860070c10 2024-03-29T08:31:39.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:39.583+0000 7f1857e00700 1 -- 172.21.15.37:0/1924501301 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f184c01c630 con 0x7f1860070c10 2024-03-29T08:31:39.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:39.583+0000 7f1866c00700 1 -- 172.21.15.37:0/1924501301 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18601103f0 con 0x7f1860070c10 2024-03-29T08:31:39.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:39.584+0000 7f1866c00700 1 -- 172.21.15.37:0/1924501301 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1860071a40 con 0x7f1860070c10 2024-03-29T08:31:39.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:39.585+0000 7f1857e00700 1 -- 172.21.15.37:0/1924501301 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f184c0139e0 con 0x7f1860070c10 2024-03-29T08:31:39.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:39.586+0000 7f1857e00700 1 --2- 172.21.15.37:0/1924501301 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f184406cc20 0x7f184406f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:31:39.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:39.586+0000 7f1857e00700 1 -- 172.21.15.37:0/1924501301 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f184c08d1e0 con 0x7f1860070c10 2024-03-29T08:31:39.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:39.586+0000 7f185fe00700 1 --2- 172.21.15.37:0/1924501301 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f184406cc20 0x7f184406f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:31:39.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:39.587+0000 7f185fe00700 1 --2- 172.21.15.37:0/1924501301 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f184406cc20 0x7f184406f0d0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f1850009b10 tx=0x7f1850008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:31:39.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:39.591+0000 7f1857e00700 1 -- 172.21.15.37:0/1924501301 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f184c057220 con 0x7f1860070c10 2024-03-29T08:31:39.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:31:39 smithi148 ceph-mon[106692]: from='client.35152 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:31:39.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:31:39 smithi148 ceph-mon[106692]: pgmap v622: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:31:39.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:31:39 smithi148 ceph-mon[106692]: from='client.35156 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:31:39.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:31:39 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/2571067314' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:31:39.807 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:31:39 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/2605836419' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T08:31:39.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:39.896+0000 7f1866c00700 1 -- 172.21.15.37:0/1924501301 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f186004c710 con 0x7f1860070c10 2024-03-29T08:31:39.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:39.897+0000 7f1857e00700 1 -- 172.21.15.37:0/1924501301 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7f184c05a840 con 0x7f1860070c10 2024-03-29T08:31:39.897 INFO:teuthology.orchestra.run.smithi037.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-03-29T08:31:39.897 INFO:teuthology.orchestra.run.smithi037.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-03-29T08:31:39.897 INFO:teuthology.orchestra.run.smithi037.stdout: fs cephfs is degraded 2024-03-29T08:31:39.897 INFO:teuthology.orchestra.run.smithi037.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-03-29T08:31:39.897 INFO:teuthology.orchestra.run.smithi037.stdout: fs cephfs is offline because no MDS is active for it. 2024-03-29T08:31:39.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:39.899+0000 7f1866c00700 1 -- 172.21.15.37:0/1924501301 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f184406cc20 msgr2=0x7f184406f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:31:39.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:39.899+0000 7f1866c00700 1 --2- 172.21.15.37:0/1924501301 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f184406cc20 0x7f184406f0d0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f1850009b10 tx=0x7f1850008040 comp rx=0 tx=0).stop 2024-03-29T08:31:39.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:39.899+0000 7f1866c00700 1 -- 172.21.15.37:0/1924501301 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1860070c10 msgr2=0x7f1860115fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:31:39.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:39.899+0000 7f1866c00700 1 --2- 172.21.15.37:0/1924501301 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1860070c10 0x7f1860115fc0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f184c009380 tx=0x7f184c009e30 comp rx=0 tx=0).stop 2024-03-29T08:31:39.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:39.899+0000 7f1866c00700 1 -- 172.21.15.37:0/1924501301 shutdown_connections 2024-03-29T08:31:39.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:39.900+0000 7f1866c00700 1 --2- 172.21.15.37:0/1924501301 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f184406cc20 0x7f184406f0d0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:31:39.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:39.900+0000 7f1866c00700 1 --2- 172.21.15.37:0/1924501301 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1860070300 0x7f1860115a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:31:39.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:39.900+0000 7f1866c00700 1 --2- 172.21.15.37:0/1924501301 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1860070c10 0x7f1860115fc0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:31:39.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:39.900+0000 7f1866c00700 1 -- 172.21.15.37:0/1924501301 >> 172.21.15.37:0/1924501301 conn(0x7f18600f89b0 msgr2=0x7f18600f9ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:31:39.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:39.900+0000 7f1866c00700 1 -- 172.21.15.37:0/1924501301 shutdown_connections 2024-03-29T08:31:39.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:31:39.900+0000 7f1866c00700 1 -- 172.21.15.37:0/1924501301 wait complete. 2024-03-29T08:31:39.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:31:39 smithi037 ceph-mon[142325]: from='client.35152 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:31:39.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:31:39 smithi037 ceph-mon[142325]: pgmap v622: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:31:39.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:31:39 smithi037 ceph-mon[142325]: from='client.35156 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:31:39.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:31:39 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/2571067314' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:31:39.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:31:39 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/2605836419' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T08:31:40.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:31:40 smithi148 ceph-mon[106692]: from='client.35168 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:31:40.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:31:40 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:31:40.807 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:31:40 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/1924501301' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T08:31:40.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:31:40 smithi037 ceph-mon[142325]: from='client.35168 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:31:40.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:31:40 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:31:40.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:31:40 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/1924501301' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T08:31:41.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:31:41 smithi148 ceph-mon[106692]: pgmap v623: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:31:41.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:31:41 smithi037 ceph-mon[142325]: pgmap v623: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:31:43.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:31:43 smithi148 ceph-mon[106692]: pgmap v624: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:31:43.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:31:43 smithi037 ceph-mon[142325]: pgmap v624: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:31:45.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:31:45 smithi148 ceph-mon[106692]: pgmap v625: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:31:45.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:31:45 smithi037 ceph-mon[142325]: pgmap v625: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:31:47.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:31:47 smithi148 ceph-mon[106692]: pgmap v626: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:31:47.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:31:47 smithi037 ceph-mon[142325]: pgmap v626: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:31:49.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:31:49 smithi148 ceph-mon[106692]: pgmap v627: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:31:49.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:31:49 smithi037 ceph-mon[142325]: pgmap v627: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:31:51.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:31:51 smithi148 ceph-mon[106692]: pgmap v628: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:31:51.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:31:51 smithi037 ceph-mon[142325]: pgmap v628: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:31:53.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:31:53 smithi148 ceph-mon[106692]: pgmap v629: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:31:53.929 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:31:53 smithi037 ceph-mon[142325]: pgmap v629: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:31:55.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:31:55 smithi148 ceph-mon[106692]: pgmap v630: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:31:55.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:31:55 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:31:55.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:31:55 smithi037 ceph-mon[142325]: pgmap v630: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:31:55.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:31:55 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:31:57.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:31:57 smithi148 ceph-mon[106692]: pgmap v631: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:31:57.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:31:57 smithi037 ceph-mon[142325]: pgmap v631: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:31:59.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:31:59 smithi148 ceph-mon[106692]: pgmap v632: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:31:59.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:31:59 smithi037 ceph-mon[142325]: pgmap v632: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:32:01.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:32:01 smithi148 ceph-mon[106692]: pgmap v633: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:32:01.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:32:01 smithi037 ceph-mon[142325]: pgmap v633: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:32:03.634 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:32:03 smithi148 ceph-mon[106692]: pgmap v634: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:32:03.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:32:03 smithi037 ceph-mon[142325]: pgmap v634: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:32:05.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:32:05 smithi148 ceph-mon[106692]: pgmap v635: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:32:05.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:32:05 smithi037 ceph-mon[142325]: pgmap v635: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:32:07.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:32:07 smithi148 ceph-mon[106692]: pgmap v636: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:32:07.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:32:07 smithi037 ceph-mon[142325]: pgmap v636: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:32:09.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:32:09 smithi148 ceph-mon[106692]: pgmap v637: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:32:09.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:32:09 smithi037 ceph-mon[142325]: pgmap v637: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:32:10.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:10.050+0000 7fedf8a00700 1 -- 172.21.15.37:0/2891175789 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fedf4102e00 msgr2=0x7fedf41031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:32:10.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:10.050+0000 7fedf8a00700 1 --2- 172.21.15.37:0/2891175789 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fedf4102e00 0x7fedf41031d0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fede4004940 tx=0x7fede4009e30 comp rx=0 tx=0).stop 2024-03-29T08:32:10.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:10.050+0000 7fedf8a00700 1 -- 172.21.15.37:0/2891175789 shutdown_connections 2024-03-29T08:32:10.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:10.050+0000 7fedf8a00700 1 --2- 172.21.15.37:0/2891175789 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fedf41037a0 0x7fedf40fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:32:10.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:10.050+0000 7fedf8a00700 1 --2- 172.21.15.37:0/2891175789 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fedf4102e00 0x7fedf41031d0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:32:10.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:10.050+0000 7fedf8a00700 1 -- 172.21.15.37:0/2891175789 >> 172.21.15.37:0/2891175789 conn(0x7fedf40f89b0 msgr2=0x7fedf40fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:32:10.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:10.050+0000 7fedf8a00700 1 -- 172.21.15.37:0/2891175789 shutdown_connections 2024-03-29T08:32:10.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:10.050+0000 7fedf8a00700 1 -- 172.21.15.37:0/2891175789 wait complete. 2024-03-29T08:32:10.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:10.051+0000 7fedf8a00700 1 Processor -- start 2024-03-29T08:32:10.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:10.051+0000 7fedf8a00700 1 -- start start 2024-03-29T08:32:10.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:10.051+0000 7fedf8a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fedf4102e00 0x7fedf4115cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:32:10.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:10.051+0000 7fedf8a00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fedf41037a0 0x7fedf4116230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:32:10.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:10.051+0000 7fedf8a00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedf4116880 con 0x7fedf4102e00 2024-03-29T08:32:10.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:10.051+0000 7fedf8a00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedf410fc50 con 0x7fedf41037a0 2024-03-29T08:32:10.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:10.052+0000 7fedf1e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fedf4102e00 0x7fedf4115cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:32:10.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:10.052+0000 7fedf1e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fedf4102e00 0x7fedf4115cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39566/0 (socket says 172.21.15.37:39566) 2024-03-29T08:32:10.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:10.052+0000 7fedf1e00700 1 -- 172.21.15.37:0/4266985819 learned_addr learned my addr 172.21.15.37:0/4266985819 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:32:10.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:10.052+0000 7fedf1400700 1 --2- 172.21.15.37:0/4266985819 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fedf41037a0 0x7fedf4116230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:32:10.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:10.052+0000 7fedf1e00700 1 -- 172.21.15.37:0/4266985819 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fedf41037a0 msgr2=0x7fedf4116230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:32:10.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:10.052+0000 7fedf1e00700 1 --2- 172.21.15.37:0/4266985819 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fedf41037a0 0x7fedf4116230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:32:10.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:10.052+0000 7fedf1e00700 1 -- 172.21.15.37:0/4266985819 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fede4005040 con 0x7fedf4102e00 2024-03-29T08:32:10.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:10.052+0000 7fedf1e00700 1 --2- 172.21.15.37:0/4266985819 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fedf4102e00 0x7fedf4115cf0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fede400b3b0 tx=0x7fede4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:32:10.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:10.053+0000 7fedea000700 1 -- 172.21.15.37:0/4266985819 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fede400e040 con 0x7fedf4102e00 2024-03-29T08:32:10.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:10.053+0000 7fedea000700 1 -- 172.21.15.37:0/4266985819 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fede4013420 con 0x7fedf4102e00 2024-03-29T08:32:10.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:10.053+0000 7fedea000700 1 -- 172.21.15.37:0/4266985819 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fede401c630 con 0x7fedf4102e00 2024-03-29T08:32:10.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:10.053+0000 7fedf8a00700 1 -- 172.21.15.37:0/4266985819 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fedf410fed0 con 0x7fedf4102e00 2024-03-29T08:32:10.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:10.053+0000 7fedf8a00700 1 -- 172.21.15.37:0/4266985819 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedf4110490 con 0x7fedf4102e00 2024-03-29T08:32:10.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:10.054+0000 7fedf8a00700 1 -- 172.21.15.37:0/4266985819 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fedf40fe550 con 0x7fedf4102e00 2024-03-29T08:32:10.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:10.055+0000 7fedea000700 1 -- 172.21.15.37:0/4266985819 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fede4013970 con 0x7fedf4102e00 2024-03-29T08:32:10.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:10.056+0000 7fedea000700 1 --2- 172.21.15.37:0/4266985819 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fedd806cc20 0x7fedd806f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:32:10.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:10.056+0000 7fedea000700 1 -- 172.21.15.37:0/4266985819 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fede408d250 con 0x7fedf4102e00 2024-03-29T08:32:10.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:10.057+0000 7fedf1400700 1 --2- 172.21.15.37:0/4266985819 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fedd806cc20 0x7fedd806f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:32:10.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:10.057+0000 7fedf1400700 1 --2- 172.21.15.37:0/4266985819 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fedd806cc20 0x7fedd806f0d0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fedf4111590 tx=0x7fede0009ee0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:32:10.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:10.061+0000 7fedea000700 1 -- 172.21.15.37:0/4266985819 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fede405ad20 con 0x7fedf4102e00 2024-03-29T08:32:10.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:10.293+0000 7fedf8a00700 1 -- 172.21.15.37:0/4266985819 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fedf404bf40 con 0x7fedd806cc20 2024-03-29T08:32:10.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:10.294+0000 7fedea000700 1 -- 172.21.15.37:0/4266985819 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fedf404bf40 con 0x7fedd806cc20 2024-03-29T08:32:10.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:10.297+0000 7fedf8a00700 1 -- 172.21.15.37:0/4266985819 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fedd806cc20 msgr2=0x7fedd806f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:32:10.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:10.297+0000 7fedf8a00700 1 --2- 172.21.15.37:0/4266985819 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fedd806cc20 0x7fedd806f0d0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fedf4111590 tx=0x7fede0009ee0 comp rx=0 tx=0).stop 2024-03-29T08:32:10.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:10.297+0000 7fedf8a00700 1 -- 172.21.15.37:0/4266985819 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fedf4102e00 msgr2=0x7fedf4115cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:32:10.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:10.297+0000 7fedf8a00700 1 --2- 172.21.15.37:0/4266985819 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fedf4102e00 0x7fedf4115cf0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fede400b3b0 tx=0x7fede4009e30 comp rx=0 tx=0).stop 2024-03-29T08:32:10.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:10.297+0000 7fedf8a00700 1 -- 172.21.15.37:0/4266985819 shutdown_connections 2024-03-29T08:32:10.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:10.297+0000 7fedf8a00700 1 --2- 172.21.15.37:0/4266985819 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fedd806cc20 0x7fedd806f0d0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:32:10.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:10.297+0000 7fedf8a00700 1 --2- 172.21.15.37:0/4266985819 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fedf4102e00 0x7fedf4115cf0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:32:10.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:10.297+0000 7fedf8a00700 1 --2- 172.21.15.37:0/4266985819 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fedf41037a0 0x7fedf4116230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:32:10.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:10.297+0000 7fedf8a00700 1 -- 172.21.15.37:0/4266985819 >> 172.21.15.37:0/4266985819 conn(0x7fedf40f89b0 msgr2=0x7fedf40706d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:32:10.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:10.297+0000 7fedf8a00700 1 -- 172.21.15.37:0/4266985819 shutdown_connections 2024-03-29T08:32:10.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:10.297+0000 7fedf8a00700 1 -- 172.21.15.37:0/4266985819 wait complete. 2024-03-29T08:32:10.313 INFO:teuthology.orchestra.run.smithi037.stdout:true 2024-03-29T08:32:10.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:10.436+0000 7fcfde000700 1 -- 172.21.15.37:0/4240405470 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcfd80fde90 msgr2=0x7fcfd80fe260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:32:10.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:10.436+0000 7fcfde000700 1 --2- 172.21.15.37:0/4240405470 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcfd80fde90 0x7fcfd80fe260 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fcfc8004940 tx=0x7fcfc8009e30 comp rx=0 tx=0).stop 2024-03-29T08:32:10.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:10.436+0000 7fcfde000700 1 -- 172.21.15.37:0/4240405470 shutdown_connections 2024-03-29T08:32:10.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:10.436+0000 7fcfde000700 1 --2- 172.21.15.37:0/4240405470 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcfd80fe830 0x7fcfd8106f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:32:10.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:10.436+0000 7fcfde000700 1 --2- 172.21.15.37:0/4240405470 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcfd80fde90 0x7fcfd80fe260 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:32:10.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:10.436+0000 7fcfde000700 1 -- 172.21.15.37:0/4240405470 >> 172.21.15.37:0/4240405470 conn(0x7fcfd8072bc0 msgr2=0x7fcfd8072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:32:10.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:10.436+0000 7fcfde000700 1 -- 172.21.15.37:0/4240405470 shutdown_connections 2024-03-29T08:32:10.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:10.436+0000 7fcfde000700 1 -- 172.21.15.37:0/4240405470 wait complete. 2024-03-29T08:32:10.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:10.437+0000 7fcfde000700 1 Processor -- start 2024-03-29T08:32:10.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:10.437+0000 7fcfde000700 1 -- start start 2024-03-29T08:32:10.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:10.437+0000 7fcfde000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcfd80fde90 0x7fcfd8115c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:32:10.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:10.437+0000 7fcfde000700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcfd80fe830 0x7fcfd81161a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:32:10.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:10.437+0000 7fcfde000700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfd81167f0 con 0x7fcfd80fde90 2024-03-29T08:32:10.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:10.437+0000 7fcfde000700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfd810fbc0 con 0x7fcfd80fe830 2024-03-29T08:32:10.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:10.438+0000 7fcfd7400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcfd80fde90 0x7fcfd8115c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:32:10.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:10.438+0000 7fcfd7400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcfd80fde90 0x7fcfd8115c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39582/0 (socket says 172.21.15.37:39582) 2024-03-29T08:32:10.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:10.438+0000 7fcfd7400700 1 -- 172.21.15.37:0/231271212 learned_addr learned my addr 172.21.15.37:0/231271212 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:32:10.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:10.438+0000 7fcfd6a00700 1 --2- 172.21.15.37:0/231271212 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcfd80fe830 0x7fcfd81161a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:32:10.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:10.438+0000 7fcfd7400700 1 -- 172.21.15.37:0/231271212 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcfd80fe830 msgr2=0x7fcfd81161a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:32:10.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:10.438+0000 7fcfd7400700 1 --2- 172.21.15.37:0/231271212 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcfd80fe830 0x7fcfd81161a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:32:10.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:10.438+0000 7fcfd7400700 1 -- 172.21.15.37:0/231271212 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfc8005040 con 0x7fcfd80fde90 2024-03-29T08:32:10.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:10.438+0000 7fcfd6a00700 1 --2- 172.21.15.37:0/231271212 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcfd80fe830 0x7fcfd81161a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:32:10.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:10.438+0000 7fcfd7400700 1 --2- 172.21.15.37:0/231271212 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcfd80fde90 0x7fcfd8115c60 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fcfc8000c00 tx=0x7fcfc8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:32:10.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:10.439+0000 7fcfcf400700 1 -- 172.21.15.37:0/231271212 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcfc800e040 con 0x7fcfd80fde90 2024-03-29T08:32:10.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:10.439+0000 7fcfcf400700 1 -- 172.21.15.37:0/231271212 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fcfc8013420 con 0x7fcfd80fde90 2024-03-29T08:32:10.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:10.439+0000 7fcfcf400700 1 -- 172.21.15.37:0/231271212 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcfc801c630 con 0x7fcfd80fde90 2024-03-29T08:32:10.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:10.439+0000 7fcfde000700 1 -- 172.21.15.37:0/231271212 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfd810fe40 con 0x7fcfd80fde90 2024-03-29T08:32:10.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:10.439+0000 7fcfde000700 1 -- 172.21.15.37:0/231271212 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfd8110380 con 0x7fcfd80fde90 2024-03-29T08:32:10.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:10.440+0000 7fcfde000700 1 -- 172.21.15.37:0/231271212 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfd8106a20 con 0x7fcfd80fde90 2024-03-29T08:32:10.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:10.441+0000 7fcfcf400700 1 -- 172.21.15.37:0/231271212 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fcfc8013970 con 0x7fcfd80fde90 2024-03-29T08:32:10.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:10.442+0000 7fcfcf400700 1 --2- 172.21.15.37:0/231271212 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fcfbc06cc30 0x7fcfbc06f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:32:10.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:10.442+0000 7fcfcf400700 1 -- 172.21.15.37:0/231271212 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fcfc808d2e0 con 0x7fcfd80fde90 2024-03-29T08:32:10.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:10.443+0000 7fcfd6a00700 1 --2- 172.21.15.37:0/231271212 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fcfbc06cc30 0x7fcfbc06f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:32:10.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:10.443+0000 7fcfd6a00700 1 --2- 172.21.15.37:0/231271212 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fcfbc06cc30 0x7fcfbc06f0e0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fcfc4007fd0 tx=0x7fcfc4009f60 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:32:10.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:10.447+0000 7fcfcf400700 1 -- 172.21.15.37:0/231271212 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fcfc805ad30 con 0x7fcfd80fde90 2024-03-29T08:32:10.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:10.680+0000 7fcfde000700 1 -- 172.21.15.37:0/231271212 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fcfd8109dc0 con 0x7fcfbc06cc30 2024-03-29T08:32:10.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:10.681+0000 7fcfcf400700 1 -- 172.21.15.37:0/231271212 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fcfd8109dc0 con 0x7fcfbc06cc30 2024-03-29T08:32:10.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:10.684+0000 7fcfde000700 1 -- 172.21.15.37:0/231271212 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fcfbc06cc30 msgr2=0x7fcfbc06f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:32:10.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:10.684+0000 7fcfde000700 1 --2- 172.21.15.37:0/231271212 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fcfbc06cc30 0x7fcfbc06f0e0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fcfc4007fd0 tx=0x7fcfc4009f60 comp rx=0 tx=0).stop 2024-03-29T08:32:10.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:10.684+0000 7fcfde000700 1 -- 172.21.15.37:0/231271212 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcfd80fde90 msgr2=0x7fcfd8115c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:32:10.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:10.684+0000 7fcfde000700 1 --2- 172.21.15.37:0/231271212 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcfd80fde90 0x7fcfd8115c60 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fcfc8000c00 tx=0x7fcfc8009e30 comp rx=0 tx=0).stop 2024-03-29T08:32:10.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:10.684+0000 7fcfde000700 1 -- 172.21.15.37:0/231271212 shutdown_connections 2024-03-29T08:32:10.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:10.684+0000 7fcfde000700 1 --2- 172.21.15.37:0/231271212 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fcfbc06cc30 0x7fcfbc06f0e0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:32:10.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:10.684+0000 7fcfde000700 1 --2- 172.21.15.37:0/231271212 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcfd80fde90 0x7fcfd8115c60 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:32:10.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:10.684+0000 7fcfde000700 1 --2- 172.21.15.37:0/231271212 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcfd80fe830 0x7fcfd81161a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:32:10.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:10.684+0000 7fcfde000700 1 -- 172.21.15.37:0/231271212 >> 172.21.15.37:0/231271212 conn(0x7fcfd8072bc0 msgr2=0x7fcfd80fd570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:32:10.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:10.684+0000 7fcfde000700 1 -- 172.21.15.37:0/231271212 shutdown_connections 2024-03-29T08:32:10.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:10.684+0000 7fcfde000700 1 -- 172.21.15.37:0/231271212 wait complete. 2024-03-29T08:32:10.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:32:10 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:32:10.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:10.823+0000 7f7bdb400700 1 -- 172.21.15.37:0/1014172241 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7bd40fde90 msgr2=0x7f7bd40fe260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:32:10.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:10.823+0000 7f7bdb400700 1 --2- 172.21.15.37:0/1014172241 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7bd40fde90 0x7f7bd40fe260 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f7bc8004940 tx=0x7f7bc8009e30 comp rx=0 tx=0).stop 2024-03-29T08:32:10.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:10.823+0000 7f7bdb400700 1 -- 172.21.15.37:0/1014172241 shutdown_connections 2024-03-29T08:32:10.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:10.823+0000 7f7bdb400700 1 --2- 172.21.15.37:0/1014172241 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7bd40fe830 0x7f7bd4106f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:32:10.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:10.823+0000 7f7bdb400700 1 --2- 172.21.15.37:0/1014172241 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7bd40fde90 0x7f7bd40fe260 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:32:10.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:10.823+0000 7f7bdb400700 1 -- 172.21.15.37:0/1014172241 >> 172.21.15.37:0/1014172241 conn(0x7f7bd4072bc0 msgr2=0x7f7bd4072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:32:10.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:10.823+0000 7f7bdb400700 1 -- 172.21.15.37:0/1014172241 shutdown_connections 2024-03-29T08:32:10.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:10.823+0000 7f7bdb400700 1 -- 172.21.15.37:0/1014172241 wait complete. 2024-03-29T08:32:10.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:10.824+0000 7f7bdb400700 1 Processor -- start 2024-03-29T08:32:10.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:10.824+0000 7f7bdb400700 1 -- start start 2024-03-29T08:32:10.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:10.824+0000 7f7bdb400700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7bd40fde90 0x7f7bd4113a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:32:10.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:10.824+0000 7f7bdb400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7bd40fe830 0x7f7bd4113fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:32:10.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:10.824+0000 7f7bdb400700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bd4114600 con 0x7f7bd40fe830 2024-03-29T08:32:10.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:10.824+0000 7f7bdb400700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bd410d9f0 con 0x7f7bd40fde90 2024-03-29T08:32:10.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:10.825+0000 7f7bd3e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7bd40fe830 0x7f7bd4113fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:32:10.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:10.825+0000 7f7bd3e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7bd40fe830 0x7f7bd4113fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39610/0 (socket says 172.21.15.37:39610) 2024-03-29T08:32:10.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:10.825+0000 7f7bd3e00700 1 -- 172.21.15.37:0/2216512850 learned_addr learned my addr 172.21.15.37:0/2216512850 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:32:10.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:10.825+0000 7f7bd8a00700 1 --2- 172.21.15.37:0/2216512850 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7bd40fde90 0x7f7bd4113a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:32:10.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:10.825+0000 7f7bd3e00700 1 -- 172.21.15.37:0/2216512850 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7bd40fde90 msgr2=0x7f7bd4113a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:32:10.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:10.825+0000 7f7bd3e00700 1 --2- 172.21.15.37:0/2216512850 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7bd40fde90 0x7f7bd4113a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:32:10.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:10.825+0000 7f7bd3e00700 1 -- 172.21.15.37:0/2216512850 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bc8005040 con 0x7f7bd40fe830 2024-03-29T08:32:10.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:10.825+0000 7f7bd3e00700 1 --2- 172.21.15.37:0/2216512850 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7bd40fe830 0x7f7bd4113fb0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f7bc000eb40 tx=0x7f7bc000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:32:10.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:10.825+0000 7f7bd0c00700 1 -- 172.21.15.37:0/2216512850 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7bc0007cc0 con 0x7f7bd40fe830 2024-03-29T08:32:10.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:10.826+0000 7f7bd0c00700 1 -- 172.21.15.37:0/2216512850 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f7bc000f040 con 0x7f7bd40fe830 2024-03-29T08:32:10.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:10.826+0000 7f7bd0c00700 1 -- 172.21.15.37:0/2216512850 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7bc000c990 con 0x7f7bd40fe830 2024-03-29T08:32:10.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:10.826+0000 7f7bdb400700 1 -- 172.21.15.37:0/2216512850 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bd410dc70 con 0x7f7bd40fe830 2024-03-29T08:32:10.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:10.826+0000 7f7bdb400700 1 -- 172.21.15.37:0/2216512850 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bd410e1b0 con 0x7f7bd40fe830 2024-03-29T08:32:10.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:10.827+0000 7f7bdb400700 1 -- 172.21.15.37:0/2216512850 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7bd4065e30 con 0x7f7bd40fe830 2024-03-29T08:32:10.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:10.828+0000 7f7bd0c00700 1 -- 172.21.15.37:0/2216512850 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f7bc000caf0 con 0x7f7bd40fe830 2024-03-29T08:32:10.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:10.829+0000 7f7bd0c00700 1 --2- 172.21.15.37:0/2216512850 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f7bbc06cc30 0x7f7bbc06f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:32:10.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:10.829+0000 7f7bd0c00700 1 -- 172.21.15.37:0/2216512850 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f7bc008bc00 con 0x7f7bd40fe830 2024-03-29T08:32:10.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:10.829+0000 7f7bd8a00700 1 --2- 172.21.15.37:0/2216512850 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f7bbc06cc30 0x7f7bbc06f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:32:10.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:10.830+0000 7f7bd8a00700 1 --2- 172.21.15.37:0/2216512850 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f7bbc06cc30 0x7f7bbc06f0e0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f7bc8009fd0 tx=0x7f7bc8009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:32:10.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:10.833+0000 7f7bd0c00700 1 -- 172.21.15.37:0/2216512850 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f7bc0059700 con 0x7f7bd40fe830 2024-03-29T08:32:10.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:32:10 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:32:11.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:11.060+0000 7f7bdb400700 1 -- 172.21.15.37:0/2216512850 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f7bd4109dc0 con 0x7f7bbc06cc30 2024-03-29T08:32:11.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:11.070+0000 7f7bd0c00700 1 -- 172.21.15.37:0/2216512850 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f7bd4109dc0 con 0x7f7bbc06cc30 2024-03-29T08:32:11.072 INFO:teuthology.orchestra.run.smithi037.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-29T08:32:11.072 INFO:teuthology.orchestra.run.smithi037.stdout:alertmanager.smithi037 smithi037 *:9093,9094 running (22m) 14m ago 34m 14.9M - 0.25.0 c8568f914cd2 2f9f9746ed0f 2024-03-29T08:32:11.072 INFO:teuthology.orchestra.run.smithi037.stdout:crash.smithi037 smithi037 running (18m) 14m ago 35m 6647k - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 178d416a3186 2024-03-29T08:32:11.072 INFO:teuthology.orchestra.run.smithi037.stdout:crash.smithi148 smithi148 running (18m) 13m ago 34m 6656k - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 23666989d58f 2024-03-29T08:32:11.072 INFO:teuthology.orchestra.run.smithi037.stdout:grafana.smithi037 smithi037 *:3000 running (21m) 14m ago 34m 71.9M - 9.4.7 954c08fa6188 6b976fe49952 2024-03-29T08:32:11.072 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi037.iifsmh smithi037 running (14m) 14m ago 28m 10.8M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 2d97f562cc5b 2024-03-29T08:32:11.072 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi037.mmwgxe smithi037 running (14m) 14m ago 28m 10.9M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff ec37037f46e8 2024-03-29T08:32:11.072 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi148.qukpjh smithi148 running (28m) 13m ago 28m 15.3M - 17.2.7-789-g4213e17c cf22eb8683be 8e469782a273 2024-03-29T08:32:11.073 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi148.sqteih smithi148 running (14m) 13m ago 28m 10.7M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 44c15c77037a 2024-03-29T08:32:11.073 INFO:teuthology.orchestra.run.smithi037.stdout:mgr.smithi037.wjikii smithi037 *:8443,9283,8765 running (24m) 14m ago 36m 506M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff c5a1417b9417 2024-03-29T08:32:11.073 INFO:teuthology.orchestra.run.smithi037.stdout:mgr.smithi148.onhgvm smithi148 *:8443,9283,8765 running (23m) 13m ago 33m 450M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff fcfdbce062bd 2024-03-29T08:32:11.073 INFO:teuthology.orchestra.run.smithi037.stdout:mon.smithi037 smithi037 running (20m) 14m ago 36m 54.8M 2048M 18.2.2-1015-gd0e392d9 1cc2f45c17ff eea2060783af 2024-03-29T08:32:11.073 INFO:teuthology.orchestra.run.smithi037.stdout:mon.smithi148 smithi148 running (19m) 13m ago 33m 43.7M 2048M 18.2.2-1015-gd0e392d9 1cc2f45c17ff ad4b2c5748be 2024-03-29T08:32:11.073 INFO:teuthology.orchestra.run.smithi037.stdout:node-exporter.smithi037 smithi037 *:9100 running (22m) 14m ago 35m 7625k - 1.5.0 0da6a335fe13 26b8ae9cb878 2024-03-29T08:32:11.073 INFO:teuthology.orchestra.run.smithi037.stdout:node-exporter.smithi148 smithi148 *:9100 running (22m) 13m ago 33m 8166k - 1.5.0 0da6a335fe13 98414e4e64ed 2024-03-29T08:32:11.073 INFO:teuthology.orchestra.run.smithi037.stdout:osd.0 smithi037 running (18m) 14m ago 32m 104M 1619M 18.2.2-1015-gd0e392d9 1cc2f45c17ff c0c584f6245c 2024-03-29T08:32:11.073 INFO:teuthology.orchestra.run.smithi037.stdout:osd.1 smithi037 running (17m) 14m ago 32m 100M 1619M 18.2.2-1015-gd0e392d9 1cc2f45c17ff a4a2ff4cc83d 2024-03-29T08:32:11.073 INFO:teuthology.orchestra.run.smithi037.stdout:osd.2 smithi037 running (17m) 14m ago 31m 82.3M 1619M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 8b1bc71aa79e 2024-03-29T08:32:11.073 INFO:teuthology.orchestra.run.smithi037.stdout:osd.3 smithi148 running (16m) 13m ago 31m 105M 2642M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 1c656ab8ff86 2024-03-29T08:32:11.073 INFO:teuthology.orchestra.run.smithi037.stdout:osd.4 smithi148 running (15m) 13m ago 30m 97.3M 2642M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 966a9de11d22 2024-03-29T08:32:11.073 INFO:teuthology.orchestra.run.smithi037.stdout:osd.5 smithi148 running (15m) 13m ago 30m 85.8M 2642M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 474c4f9212ce 2024-03-29T08:32:11.074 INFO:teuthology.orchestra.run.smithi037.stdout:prometheus.smithi037 smithi037 *:9095 running (22m) 14m ago 34m 54.4M - 2.43.0 a07b618ecd1d 30c0223ec589 2024-03-29T08:32:11.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:11.073+0000 7f7bdb400700 1 -- 172.21.15.37:0/2216512850 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f7bbc06cc30 msgr2=0x7f7bbc06f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:32:11.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:11.073+0000 7f7bdb400700 1 --2- 172.21.15.37:0/2216512850 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f7bbc06cc30 0x7f7bbc06f0e0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f7bc8009fd0 tx=0x7f7bc8009e30 comp rx=0 tx=0).stop 2024-03-29T08:32:11.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:11.073+0000 7f7bdb400700 1 -- 172.21.15.37:0/2216512850 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7bd40fe830 msgr2=0x7f7bd4113fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:32:11.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:11.073+0000 7f7bdb400700 1 --2- 172.21.15.37:0/2216512850 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7bd40fe830 0x7f7bd4113fb0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f7bc000eb40 tx=0x7f7bc000b040 comp rx=0 tx=0).stop 2024-03-29T08:32:11.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:11.073+0000 7f7bdb400700 1 -- 172.21.15.37:0/2216512850 shutdown_connections 2024-03-29T08:32:11.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:11.073+0000 7f7bdb400700 1 --2- 172.21.15.37:0/2216512850 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f7bbc06cc30 0x7f7bbc06f0e0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:32:11.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:11.073+0000 7f7bdb400700 1 --2- 172.21.15.37:0/2216512850 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7bd40fde90 0x7f7bd4113a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:32:11.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:11.073+0000 7f7bdb400700 1 --2- 172.21.15.37:0/2216512850 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7bd40fe830 0x7f7bd4113fb0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:32:11.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:11.073+0000 7f7bdb400700 1 -- 172.21.15.37:0/2216512850 >> 172.21.15.37:0/2216512850 conn(0x7f7bd4072bc0 msgr2=0x7f7bd41037c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:32:11.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:11.073+0000 7f7bdb400700 1 -- 172.21.15.37:0/2216512850 shutdown_connections 2024-03-29T08:32:11.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:11.073+0000 7f7bdb400700 1 -- 172.21.15.37:0/2216512850 wait complete. 2024-03-29T08:32:11.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:11.210+0000 7fec1e400700 1 -- 172.21.15.37:0/2770550701 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fec181037a0 msgr2=0x7fec180fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:32:11.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:11.210+0000 7fec1e400700 1 --2- 172.21.15.37:0/2770550701 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fec181037a0 0x7fec180fce10 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fec04004940 tx=0x7fec04009e30 comp rx=0 tx=0).stop 2024-03-29T08:32:11.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:11.210+0000 7fec1e400700 1 -- 172.21.15.37:0/2770550701 shutdown_connections 2024-03-29T08:32:11.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:11.210+0000 7fec1e400700 1 --2- 172.21.15.37:0/2770550701 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fec181037a0 0x7fec180fce10 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:32:11.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:11.210+0000 7fec1e400700 1 --2- 172.21.15.37:0/2770550701 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fec18102e00 0x7fec181031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:32:11.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:11.210+0000 7fec1e400700 1 -- 172.21.15.37:0/2770550701 >> 172.21.15.37:0/2770550701 conn(0x7fec180f89b0 msgr2=0x7fec180fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:32:11.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:11.211+0000 7fec1e400700 1 -- 172.21.15.37:0/2770550701 shutdown_connections 2024-03-29T08:32:11.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:11.211+0000 7fec1e400700 1 -- 172.21.15.37:0/2770550701 wait complete. 2024-03-29T08:32:11.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:11.211+0000 7fec1e400700 1 Processor -- start 2024-03-29T08:32:11.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:11.212+0000 7fec1e400700 1 -- start start 2024-03-29T08:32:11.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:11.212+0000 7fec1e400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fec18102e00 0x7fec18115c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:32:11.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:11.212+0000 7fec1e400700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fec181037a0 0x7fec181161c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:32:11.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:11.212+0000 7fec1e400700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec18116810 con 0x7fec18102e00 2024-03-29T08:32:11.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:11.212+0000 7fec1e400700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec1810fbe0 con 0x7fec181037a0 2024-03-29T08:32:11.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:11.212+0000 7fec17400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fec18102e00 0x7fec18115c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:32:11.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:11.212+0000 7fec17400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fec18102e00 0x7fec18115c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39636/0 (socket says 172.21.15.37:39636) 2024-03-29T08:32:11.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:11.212+0000 7fec17400700 1 -- 172.21.15.37:0/194584145 learned_addr learned my addr 172.21.15.37:0/194584145 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:32:11.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:11.212+0000 7fec16a00700 1 --2- 172.21.15.37:0/194584145 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fec181037a0 0x7fec181161c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:32:11.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:11.213+0000 7fec17400700 1 -- 172.21.15.37:0/194584145 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fec181037a0 msgr2=0x7fec181161c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:32:11.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:11.213+0000 7fec17400700 1 --2- 172.21.15.37:0/194584145 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fec181037a0 0x7fec181161c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:32:11.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:11.213+0000 7fec17400700 1 -- 172.21.15.37:0/194584145 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec04005040 con 0x7fec18102e00 2024-03-29T08:32:11.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:11.213+0000 7fec16a00700 1 --2- 172.21.15.37:0/194584145 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fec181037a0 0x7fec181161c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:32:11.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:11.213+0000 7fec17400700 1 --2- 172.21.15.37:0/194584145 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fec18102e00 0x7fec18115c80 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fec0800eb40 tx=0x7fec0800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:32:11.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:11.213+0000 7fec0f400700 1 -- 172.21.15.37:0/194584145 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fec08007cc0 con 0x7fec18102e00 2024-03-29T08:32:11.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:11.213+0000 7fec0f400700 1 -- 172.21.15.37:0/194584145 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fec0800f040 con 0x7fec18102e00 2024-03-29T08:32:11.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:11.213+0000 7fec0f400700 1 -- 172.21.15.37:0/194584145 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fec08009570 con 0x7fec18102e00 2024-03-29T08:32:11.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:11.213+0000 7fec1e400700 1 -- 172.21.15.37:0/194584145 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec1810fe60 con 0x7fec18102e00 2024-03-29T08:32:11.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:11.213+0000 7fec1e400700 1 -- 172.21.15.37:0/194584145 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec181103d0 con 0x7fec18102e00 2024-03-29T08:32:11.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:11.214+0000 7fec1e400700 1 -- 172.21.15.37:0/194584145 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec180fe550 con 0x7fec18102e00 2024-03-29T08:32:11.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:11.215+0000 7fec0f400700 1 -- 172.21.15.37:0/194584145 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fec080096d0 con 0x7fec18102e00 2024-03-29T08:32:11.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:11.216+0000 7fec0f400700 1 --2- 172.21.15.37:0/194584145 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7febfc06cc20 0x7febfc06f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:32:11.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:11.216+0000 7fec0f400700 1 -- 172.21.15.37:0/194584145 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fec0808bda0 con 0x7fec18102e00 2024-03-29T08:32:11.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:11.217+0000 7fec16a00700 1 --2- 172.21.15.37:0/194584145 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7febfc06cc20 0x7febfc06f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:32:11.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:11.217+0000 7fec16a00700 1 --2- 172.21.15.37:0/194584145 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7febfc06cc20 0x7febfc06f0d0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fec04009fd0 tx=0x7fec04009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:32:11.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:11.221+0000 7fec0f400700 1 -- 172.21.15.37:0/194584145 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fec08059820 con 0x7fec18102e00 2024-03-29T08:32:11.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:11.520+0000 7fec1e400700 1 -- 172.21.15.37:0/194584145 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fec1804bed0 con 0x7fec18102e00 2024-03-29T08:32:11.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:11.520+0000 7fec0f400700 1 -- 172.21.15.37:0/194584145 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+847 (secure 0 0 0) 0x7fec080593b0 con 0x7fec18102e00 2024-03-29T08:32:11.521 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-29T08:32:11.521 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": { 2024-03-29T08:32:11.522 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 2 2024-03-29T08:32:11.522 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:32:11.522 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": { 2024-03-29T08:32:11.522 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 2 2024-03-29T08:32:11.522 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:32:11.522 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": { 2024-03-29T08:32:11.522 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 6 2024-03-29T08:32:11.522 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:32:11.522 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": { 2024-03-29T08:32:11.522 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 17.2.7-789-g4213e17c (4213e17c4de67016ed14f55d22503f4923b3a901) quincy (stable)": 1, 2024-03-29T08:32:11.522 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 3 2024-03-29T08:32:11.522 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:32:11.522 INFO:teuthology.orchestra.run.smithi037.stdout: "overall": { 2024-03-29T08:32:11.522 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 17.2.7-789-g4213e17c (4213e17c4de67016ed14f55d22503f4923b3a901) quincy (stable)": 1, 2024-03-29T08:32:11.523 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 13 2024-03-29T08:32:11.523 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-29T08:32:11.523 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-29T08:32:11.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:11.523+0000 7fec1e400700 1 -- 172.21.15.37:0/194584145 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7febfc06cc20 msgr2=0x7febfc06f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:32:11.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:11.523+0000 7fec1e400700 1 --2- 172.21.15.37:0/194584145 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7febfc06cc20 0x7febfc06f0d0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fec04009fd0 tx=0x7fec04009e30 comp rx=0 tx=0).stop 2024-03-29T08:32:11.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:11.523+0000 7fec1e400700 1 -- 172.21.15.37:0/194584145 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fec18102e00 msgr2=0x7fec18115c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:32:11.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:11.523+0000 7fec1e400700 1 --2- 172.21.15.37:0/194584145 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fec18102e00 0x7fec18115c80 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fec0800eb40 tx=0x7fec0800b040 comp rx=0 tx=0).stop 2024-03-29T08:32:11.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:11.523+0000 7fec1e400700 1 -- 172.21.15.37:0/194584145 shutdown_connections 2024-03-29T08:32:11.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:11.523+0000 7fec1e400700 1 --2- 172.21.15.37:0/194584145 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7febfc06cc20 0x7febfc06f0d0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:32:11.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:11.523+0000 7fec1e400700 1 --2- 172.21.15.37:0/194584145 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fec18102e00 0x7fec18115c80 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:32:11.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:11.523+0000 7fec1e400700 1 --2- 172.21.15.37:0/194584145 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fec181037a0 0x7fec181161c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:32:11.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:11.523+0000 7fec1e400700 1 -- 172.21.15.37:0/194584145 >> 172.21.15.37:0/194584145 conn(0x7fec180f89b0 msgr2=0x7fec180f9a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:32:11.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:11.523+0000 7fec1e400700 1 -- 172.21.15.37:0/194584145 shutdown_connections 2024-03-29T08:32:11.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:11.523+0000 7fec1e400700 1 -- 172.21.15.37:0/194584145 wait complete. 2024-03-29T08:32:11.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:11.661+0000 7f06bde00700 1 -- 172.21.15.37:0/68569780 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f06b8070c10 msgr2=0x7f06b8109110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:32:11.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:11.661+0000 7f06bde00700 1 --2- 172.21.15.37:0/68569780 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f06b8070c10 0x7f06b8109110 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f06a4004970 tx=0x7f06a4009e30 comp rx=0 tx=0).stop 2024-03-29T08:32:11.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:11.661+0000 7f06bde00700 1 -- 172.21.15.37:0/68569780 shutdown_connections 2024-03-29T08:32:11.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:11.661+0000 7f06bde00700 1 --2- 172.21.15.37:0/68569780 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f06b8070c10 0x7f06b8109110 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:32:11.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:11.661+0000 7f06bde00700 1 --2- 172.21.15.37:0/68569780 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f06b8070300 0x7f06b80706d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:32:11.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:11.661+0000 7f06bde00700 1 -- 172.21.15.37:0/68569780 >> 172.21.15.37:0/68569780 conn(0x7f06b80f8950 msgr2=0x7f06b80fad60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:32:11.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:11.661+0000 7f06bde00700 1 -- 172.21.15.37:0/68569780 shutdown_connections 2024-03-29T08:32:11.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:11.661+0000 7f06bde00700 1 -- 172.21.15.37:0/68569780 wait complete. 2024-03-29T08:32:11.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:11.662+0000 7f06bde00700 1 Processor -- start 2024-03-29T08:32:11.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:11.662+0000 7f06bde00700 1 -- start start 2024-03-29T08:32:11.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:11.663+0000 7f06bde00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f06b8070300 0x7f06b8115c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:32:11.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:11.663+0000 7f06bde00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f06b8070c10 0x7f06b8116180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:32:11.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:11.663+0000 7f06bde00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06b8116860 con 0x7f06b8070c10 2024-03-29T08:32:11.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:11.663+0000 7f06bde00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06b810fba0 con 0x7f06b8070300 2024-03-29T08:32:11.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:11.663+0000 7f06b6000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f06b8070c10 0x7f06b8116180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:32:11.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:11.663+0000 7f06b6000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f06b8070c10 0x7f06b8116180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39662/0 (socket says 172.21.15.37:39662) 2024-03-29T08:32:11.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:11.663+0000 7f06b6000700 1 -- 172.21.15.37:0/4196477743 learned_addr learned my addr 172.21.15.37:0/4196477743 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:32:11.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:11.663+0000 7f06b6a00700 1 --2- 172.21.15.37:0/4196477743 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f06b8070300 0x7f06b8115c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:32:11.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:11.663+0000 7f06b6000700 1 -- 172.21.15.37:0/4196477743 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f06b8070300 msgr2=0x7f06b8115c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:32:11.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:11.663+0000 7f06b6000700 1 --2- 172.21.15.37:0/4196477743 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f06b8070300 0x7f06b8115c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:32:11.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:11.663+0000 7f06b6000700 1 -- 172.21.15.37:0/4196477743 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06a4005040 con 0x7f06b8070c10 2024-03-29T08:32:11.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:11.663+0000 7f06b6a00700 1 --2- 172.21.15.37:0/4196477743 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f06b8070300 0x7f06b8115c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:32:11.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:11.664+0000 7f06b6000700 1 --2- 172.21.15.37:0/4196477743 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f06b8070c10 0x7f06b8116180 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f06a4004f50 tx=0x7f06a4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:32:11.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:11.664+0000 7f06aea00700 1 -- 172.21.15.37:0/4196477743 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f06a400e040 con 0x7f06b8070c10 2024-03-29T08:32:11.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:11.664+0000 7f06aea00700 1 -- 172.21.15.37:0/4196477743 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f06a4013420 con 0x7f06b8070c10 2024-03-29T08:32:11.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:11.664+0000 7f06bde00700 1 -- 172.21.15.37:0/4196477743 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06b810fdc0 con 0x7f06b8070c10 2024-03-29T08:32:11.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:11.664+0000 7f06aea00700 1 -- 172.21.15.37:0/4196477743 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f06a401c610 con 0x7f06b8070c10 2024-03-29T08:32:11.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:11.664+0000 7f06bde00700 1 -- 172.21.15.37:0/4196477743 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06b8110300 con 0x7f06b8070c10 2024-03-29T08:32:11.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:11.665+0000 7f06bde00700 1 -- 172.21.15.37:0/4196477743 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06b8071a40 con 0x7f06b8070c10 2024-03-29T08:32:11.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:11.666+0000 7f06aea00700 1 -- 172.21.15.37:0/4196477743 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f06a4013590 con 0x7f06b8070c10 2024-03-29T08:32:11.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:11.667+0000 7f06aea00700 1 --2- 172.21.15.37:0/4196477743 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f069c06cc30 0x7f069c06f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:32:11.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:11.667+0000 7f06aea00700 1 -- 172.21.15.37:0/4196477743 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f06a408d2b0 con 0x7f06b8070c10 2024-03-29T08:32:11.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:11.667+0000 7f06b6a00700 1 --2- 172.21.15.37:0/4196477743 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f069c06cc30 0x7f069c06f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:32:11.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:11.668+0000 7f06b6a00700 1 --2- 172.21.15.37:0/4196477743 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f069c06cc30 0x7f069c06f0e0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f06a8006fd0 tx=0x7f06a8008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:32:11.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:11.671+0000 7f06aea00700 1 -- 172.21.15.37:0/4196477743 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f06a405ac80 con 0x7f06b8070c10 2024-03-29T08:32:11.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:32:11 smithi148 ceph-mon[106692]: pgmap v638: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:32:11.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:32:11 smithi148 ceph-mon[106692]: from='client.35176 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:32:11.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:32:11 smithi148 ceph-mon[106692]: from='client.35180 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:32:11.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:32:11 smithi037 ceph-mon[142325]: pgmap v638: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:32:11.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:32:11 smithi037 ceph-mon[142325]: from='client.35176 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:32:11.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:32:11 smithi037 ceph-mon[142325]: from='client.35180 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:32:11.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:11.931+0000 7f06bde00700 1 -- 172.21.15.37:0/4196477743 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f06b810c060 con 0x7f06b8070c10 2024-03-29T08:32:11.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:11.931+0000 7f06aea00700 1 -- 172.21.15.37:0/4196477743 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 28 v28) v1 ==== 76+0+1841 (secure 0 0 0) 0x7f06a405a810 con 0x7f06b8070c10 2024-03-29T08:32:11.932 INFO:teuthology.orchestra.run.smithi037.stdout:e28 2024-03-29T08:32:11.932 INFO:teuthology.orchestra.run.smithi037.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-29T08:32:11.932 INFO:teuthology.orchestra.run.smithi037.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T08:32:11.932 INFO:teuthology.orchestra.run.smithi037.stdout:legacy client fscid: 1 2024-03-29T08:32:11.932 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:32:11.932 INFO:teuthology.orchestra.run.smithi037.stdout:Filesystem 'cephfs' (1) 2024-03-29T08:32:11.933 INFO:teuthology.orchestra.run.smithi037.stdout:fs_name cephfs 2024-03-29T08:32:11.933 INFO:teuthology.orchestra.run.smithi037.stdout:epoch 21 2024-03-29T08:32:11.933 INFO:teuthology.orchestra.run.smithi037.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-03-29T08:32:11.933 INFO:teuthology.orchestra.run.smithi037.stdout:created 2024-03-29T08:03:29.241269+0000 2024-03-29T08:32:11.933 INFO:teuthology.orchestra.run.smithi037.stdout:modified 2024-03-29T08:17:16.237238+0000 2024-03-29T08:32:11.933 INFO:teuthology.orchestra.run.smithi037.stdout:tableserver 0 2024-03-29T08:32:11.933 INFO:teuthology.orchestra.run.smithi037.stdout:root 0 2024-03-29T08:32:11.933 INFO:teuthology.orchestra.run.smithi037.stdout:session_timeout 60 2024-03-29T08:32:11.933 INFO:teuthology.orchestra.run.smithi037.stdout:session_autoclose 300 2024-03-29T08:32:11.933 INFO:teuthology.orchestra.run.smithi037.stdout:max_file_size 1099511627776 2024-03-29T08:32:11.933 INFO:teuthology.orchestra.run.smithi037.stdout:max_xattr_size 65536 2024-03-29T08:32:11.933 INFO:teuthology.orchestra.run.smithi037.stdout:required_client_features {} 2024-03-29T08:32:11.933 INFO:teuthology.orchestra.run.smithi037.stdout:last_failure 0 2024-03-29T08:32:11.933 INFO:teuthology.orchestra.run.smithi037.stdout:last_failure_osd_epoch 82 2024-03-29T08:32:11.933 INFO:teuthology.orchestra.run.smithi037.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T08:32:11.934 INFO:teuthology.orchestra.run.smithi037.stdout:max_mds 2 2024-03-29T08:32:11.934 INFO:teuthology.orchestra.run.smithi037.stdout:in 0,1 2024-03-29T08:32:11.934 INFO:teuthology.orchestra.run.smithi037.stdout:up {} 2024-03-29T08:32:11.934 INFO:teuthology.orchestra.run.smithi037.stdout:failed 0,1 2024-03-29T08:32:11.934 INFO:teuthology.orchestra.run.smithi037.stdout:damaged 2024-03-29T08:32:11.934 INFO:teuthology.orchestra.run.smithi037.stdout:stopped 2024-03-29T08:32:11.934 INFO:teuthology.orchestra.run.smithi037.stdout:data_pools [3] 2024-03-29T08:32:11.934 INFO:teuthology.orchestra.run.smithi037.stdout:metadata_pool 2 2024-03-29T08:32:11.934 INFO:teuthology.orchestra.run.smithi037.stdout:inline_data disabled 2024-03-29T08:32:11.934 INFO:teuthology.orchestra.run.smithi037.stdout:balancer 2024-03-29T08:32:11.934 INFO:teuthology.orchestra.run.smithi037.stdout:bal_rank_mask -1 2024-03-29T08:32:11.934 INFO:teuthology.orchestra.run.smithi037.stdout:standby_count_wanted 1 2024-03-29T08:32:11.934 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:32:11.934 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:32:11.934 INFO:teuthology.orchestra.run.smithi037.stdout:Standby daemons: 2024-03-29T08:32:11.935 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:32:11.935 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi148.qukpjh{-1:24319} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.148:6826/2641327912,v1:172.21.15.148:6827/2641327912] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:32:11.935 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi037.mmwgxe{-1:34440} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.37:6828/1248142005,v1:172.21.15.37:6829/1248142005] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:32:11.935 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi037.iifsmh{-1:34444} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.37:6826/2060882959,v1:172.21.15.37:6827/2060882959] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:32:11.935 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi148.sqteih{-1:44395} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.148:6824/4078652242,v1:172.21.15.148:6825/4078652242] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:32:11.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:11.933+0000 7f06bde00700 1 -- 172.21.15.37:0/4196477743 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f069c06cc30 msgr2=0x7f069c06f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:32:11.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:11.933+0000 7f06bde00700 1 --2- 172.21.15.37:0/4196477743 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f069c06cc30 0x7f069c06f0e0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f06a8006fd0 tx=0x7f06a8008040 comp rx=0 tx=0).stop 2024-03-29T08:32:11.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:11.934+0000 7f06bde00700 1 -- 172.21.15.37:0/4196477743 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f06b8070c10 msgr2=0x7f06b8116180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:32:11.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:11.934+0000 7f06bde00700 1 --2- 172.21.15.37:0/4196477743 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f06b8070c10 0x7f06b8116180 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f06a4004f50 tx=0x7f06a4009e30 comp rx=0 tx=0).stop 2024-03-29T08:32:11.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:11.934+0000 7f06bde00700 1 -- 172.21.15.37:0/4196477743 shutdown_connections 2024-03-29T08:32:11.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:11.934+0000 7f06bde00700 1 --2- 172.21.15.37:0/4196477743 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f069c06cc30 0x7f069c06f0e0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:32:11.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:11.934+0000 7f06bde00700 1 --2- 172.21.15.37:0/4196477743 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f06b8070300 0x7f06b8115c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:32:11.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:11.934+0000 7f06bde00700 1 --2- 172.21.15.37:0/4196477743 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f06b8070c10 0x7f06b8116180 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:32:11.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:11.934+0000 7f06bde00700 1 -- 172.21.15.37:0/4196477743 >> 172.21.15.37:0/4196477743 conn(0x7f06b80f8950 msgr2=0x7f06b80f9e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:32:11.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:11.934+0000 7f06bde00700 1 -- 172.21.15.37:0/4196477743 shutdown_connections 2024-03-29T08:32:11.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:11.934+0000 7f06bde00700 1 -- 172.21.15.37:0/4196477743 wait complete. 2024-03-29T08:32:11.936 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 28 2024-03-29T08:32:12.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:12.073+0000 7f6f6bc00700 1 -- 172.21.15.37:0/312473658 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6f64102e00 msgr2=0x7f6f641031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:32:12.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:12.073+0000 7f6f6bc00700 1 --2- 172.21.15.37:0/312473658 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6f64102e00 0x7f6f641031d0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f6f58004940 tx=0x7f6f58009e30 comp rx=0 tx=0).stop 2024-03-29T08:32:12.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:12.073+0000 7f6f6bc00700 1 -- 172.21.15.37:0/312473658 shutdown_connections 2024-03-29T08:32:12.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:12.073+0000 7f6f6bc00700 1 --2- 172.21.15.37:0/312473658 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6f641037a0 0x7f6f640fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:32:12.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:12.073+0000 7f6f6bc00700 1 --2- 172.21.15.37:0/312473658 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6f64102e00 0x7f6f641031d0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:32:12.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:12.073+0000 7f6f6bc00700 1 -- 172.21.15.37:0/312473658 >> 172.21.15.37:0/312473658 conn(0x7f6f640f89b0 msgr2=0x7f6f640fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:32:12.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:12.073+0000 7f6f6bc00700 1 -- 172.21.15.37:0/312473658 shutdown_connections 2024-03-29T08:32:12.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:12.073+0000 7f6f6bc00700 1 -- 172.21.15.37:0/312473658 wait complete. 2024-03-29T08:32:12.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:12.074+0000 7f6f6bc00700 1 Processor -- start 2024-03-29T08:32:12.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:12.074+0000 7f6f6bc00700 1 -- start start 2024-03-29T08:32:12.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:12.074+0000 7f6f6bc00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6f64102e00 0x7f6f64115cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:32:12.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:12.075+0000 7f6f6bc00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6f641037a0 0x7f6f64116230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:32:12.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:12.075+0000 7f6f6bc00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f64116880 con 0x7f6f64102e00 2024-03-29T08:32:12.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:12.075+0000 7f6f6bc00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f6410fc50 con 0x7f6f641037a0 2024-03-29T08:32:12.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:12.075+0000 7f6f69200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6f64102e00 0x7f6f64115cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:32:12.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:12.075+0000 7f6f69200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6f64102e00 0x7f6f64115cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39674/0 (socket says 172.21.15.37:39674) 2024-03-29T08:32:12.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:12.075+0000 7f6f69200700 1 -- 172.21.15.37:0/2593476139 learned_addr learned my addr 172.21.15.37:0/2593476139 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:32:12.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:12.075+0000 7f6f63e00700 1 --2- 172.21.15.37:0/2593476139 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6f641037a0 0x7f6f64116230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:32:12.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:12.075+0000 7f6f69200700 1 -- 172.21.15.37:0/2593476139 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6f641037a0 msgr2=0x7f6f64116230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:32:12.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:12.075+0000 7f6f69200700 1 --2- 172.21.15.37:0/2593476139 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6f641037a0 0x7f6f64116230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:32:12.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:12.075+0000 7f6f69200700 1 -- 172.21.15.37:0/2593476139 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f58005040 con 0x7f6f64102e00 2024-03-29T08:32:12.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:12.075+0000 7f6f69200700 1 --2- 172.21.15.37:0/2593476139 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6f64102e00 0x7f6f64115cf0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f6f58009390 tx=0x7f6f58009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:32:12.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:12.076+0000 7f6f60c00700 1 -- 172.21.15.37:0/2593476139 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6f5800e040 con 0x7f6f64102e00 2024-03-29T08:32:12.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:12.076+0000 7f6f60c00700 1 -- 172.21.15.37:0/2593476139 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f6f58013420 con 0x7f6f64102e00 2024-03-29T08:32:12.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:12.076+0000 7f6f6bc00700 1 -- 172.21.15.37:0/2593476139 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f6410fed0 con 0x7f6f64102e00 2024-03-29T08:32:12.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:12.076+0000 7f6f60c00700 1 -- 172.21.15.37:0/2593476139 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6f5801c630 con 0x7f6f64102e00 2024-03-29T08:32:12.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:12.076+0000 7f6f6bc00700 1 -- 172.21.15.37:0/2593476139 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f64110490 con 0x7f6f64102e00 2024-03-29T08:32:12.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:12.077+0000 7f6f6bc00700 1 -- 172.21.15.37:0/2593476139 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f640fe550 con 0x7f6f64102e00 2024-03-29T08:32:12.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:12.078+0000 7f6f60c00700 1 -- 172.21.15.37:0/2593476139 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f6f580139e0 con 0x7f6f64102e00 2024-03-29T08:32:12.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:12.079+0000 7f6f60c00700 1 --2- 172.21.15.37:0/2593476139 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f6f4c06cc30 0x7f6f4c06f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:32:12.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:12.079+0000 7f6f60c00700 1 -- 172.21.15.37:0/2593476139 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f6f5808d2e0 con 0x7f6f64102e00 2024-03-29T08:32:12.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:12.079+0000 7f6f63e00700 1 --2- 172.21.15.37:0/2593476139 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f6f4c06cc30 0x7f6f4c06f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:32:12.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:12.080+0000 7f6f63e00700 1 --2- 172.21.15.37:0/2593476139 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f6f4c06cc30 0x7f6f4c06f0e0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f6f64111590 tx=0x7f6f50009ee0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:32:12.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:12.084+0000 7f6f60c00700 1 -- 172.21.15.37:0/2593476139 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f6f5805ae60 con 0x7f6f64102e00 2024-03-29T08:32:12.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:12.320+0000 7f6f6bc00700 1 -- 172.21.15.37:0/2593476139 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6f6404bf40 con 0x7f6f4c06cc30 2024-03-29T08:32:12.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:12.321+0000 7f6f60c00700 1 -- 172.21.15.37:0/2593476139 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f6f6404bf40 con 0x7f6f4c06cc30 2024-03-29T08:32:12.322 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-29T08:32:12.322 INFO:teuthology.orchestra.run.smithi037.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59", 2024-03-29T08:32:12.322 INFO:teuthology.orchestra.run.smithi037.stdout: "in_progress": true, 2024-03-29T08:32:12.322 INFO:teuthology.orchestra.run.smithi037.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-29T08:32:12.322 INFO:teuthology.orchestra.run.smithi037.stdout: "services_complete": [ 2024-03-29T08:32:12.322 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr", 2024-03-29T08:32:12.322 INFO:teuthology.orchestra.run.smithi037.stdout: "mon", 2024-03-29T08:32:12.323 INFO:teuthology.orchestra.run.smithi037.stdout: "crash", 2024-03-29T08:32:12.323 INFO:teuthology.orchestra.run.smithi037.stdout: "osd" 2024-03-29T08:32:12.323 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-29T08:32:12.323 INFO:teuthology.orchestra.run.smithi037.stdout: "progress": "15/21 daemons upgraded", 2024-03-29T08:32:12.323 INFO:teuthology.orchestra.run.smithi037.stdout: "message": "Currently upgrading mds daemons", 2024-03-29T08:32:12.323 INFO:teuthology.orchestra.run.smithi037.stdout: "is_paused": false 2024-03-29T08:32:12.323 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-29T08:32:12.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:12.323+0000 7f6f6bc00700 1 -- 172.21.15.37:0/2593476139 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f6f4c06cc30 msgr2=0x7f6f4c06f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:32:12.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:12.323+0000 7f6f6bc00700 1 --2- 172.21.15.37:0/2593476139 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f6f4c06cc30 0x7f6f4c06f0e0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f6f64111590 tx=0x7f6f50009ee0 comp rx=0 tx=0).stop 2024-03-29T08:32:12.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:12.323+0000 7f6f6bc00700 1 -- 172.21.15.37:0/2593476139 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6f64102e00 msgr2=0x7f6f64115cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:32:12.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:12.323+0000 7f6f6bc00700 1 --2- 172.21.15.37:0/2593476139 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6f64102e00 0x7f6f64115cf0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f6f58009390 tx=0x7f6f58009e30 comp rx=0 tx=0).stop 2024-03-29T08:32:12.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:12.323+0000 7f6f6bc00700 1 -- 172.21.15.37:0/2593476139 shutdown_connections 2024-03-29T08:32:12.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:12.323+0000 7f6f6bc00700 1 --2- 172.21.15.37:0/2593476139 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f6f4c06cc30 0x7f6f4c06f0e0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:32:12.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:12.323+0000 7f6f6bc00700 1 --2- 172.21.15.37:0/2593476139 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6f64102e00 0x7f6f64115cf0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:32:12.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:12.323+0000 7f6f6bc00700 1 --2- 172.21.15.37:0/2593476139 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6f641037a0 0x7f6f64116230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:32:12.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:12.323+0000 7f6f6bc00700 1 -- 172.21.15.37:0/2593476139 >> 172.21.15.37:0/2593476139 conn(0x7f6f640f89b0 msgr2=0x7f6f640706d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:32:12.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:12.324+0000 7f6f6bc00700 1 -- 172.21.15.37:0/2593476139 shutdown_connections 2024-03-29T08:32:12.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:12.324+0000 7f6f6bc00700 1 -- 172.21.15.37:0/2593476139 wait complete. 2024-03-29T08:32:12.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:12.463+0000 7fed5d600700 1 -- 172.21.15.37:0/227256221 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed581037a0 msgr2=0x7fed580fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:32:12.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:12.463+0000 7fed5d600700 1 --2- 172.21.15.37:0/227256221 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed581037a0 0x7fed580fce10 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fed44004940 tx=0x7fed44009e30 comp rx=0 tx=0).stop 2024-03-29T08:32:12.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:12.463+0000 7fed5d600700 1 -- 172.21.15.37:0/227256221 shutdown_connections 2024-03-29T08:32:12.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:12.463+0000 7fed5d600700 1 --2- 172.21.15.37:0/227256221 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed581037a0 0x7fed580fce10 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:32:12.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:12.463+0000 7fed5d600700 1 --2- 172.21.15.37:0/227256221 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fed58102e00 0x7fed581031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:32:12.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:12.463+0000 7fed5d600700 1 -- 172.21.15.37:0/227256221 >> 172.21.15.37:0/227256221 conn(0x7fed580f89b0 msgr2=0x7fed580fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:32:12.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:12.463+0000 7fed5d600700 1 -- 172.21.15.37:0/227256221 shutdown_connections 2024-03-29T08:32:12.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:12.463+0000 7fed5d600700 1 -- 172.21.15.37:0/227256221 wait complete. 2024-03-29T08:32:12.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:12.464+0000 7fed5d600700 1 Processor -- start 2024-03-29T08:32:12.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:12.464+0000 7fed5d600700 1 -- start start 2024-03-29T08:32:12.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:12.464+0000 7fed5d600700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed58102e00 0x7fed58115d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:32:12.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:12.464+0000 7fed5d600700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fed581037a0 0x7fed58116240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:32:12.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:12.464+0000 7fed5d600700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed58116890 con 0x7fed58102e00 2024-03-29T08:32:12.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:12.464+0000 7fed5d600700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed5810fc60 con 0x7fed581037a0 2024-03-29T08:32:12.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:12.465+0000 7fed56a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed58102e00 0x7fed58115d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:32:12.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:12.465+0000 7fed56a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed58102e00 0x7fed58115d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39686/0 (socket says 172.21.15.37:39686) 2024-03-29T08:32:12.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:12.465+0000 7fed56a00700 1 -- 172.21.15.37:0/2720134841 learned_addr learned my addr 172.21.15.37:0/2720134841 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:32:12.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:12.465+0000 7fed56000700 1 --2- 172.21.15.37:0/2720134841 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fed581037a0 0x7fed58116240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:32:12.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:12.465+0000 7fed56a00700 1 -- 172.21.15.37:0/2720134841 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fed581037a0 msgr2=0x7fed58116240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:32:12.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:12.465+0000 7fed56a00700 1 --2- 172.21.15.37:0/2720134841 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fed581037a0 0x7fed58116240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:32:12.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:12.465+0000 7fed56a00700 1 -- 172.21.15.37:0/2720134841 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed44005040 con 0x7fed58102e00 2024-03-29T08:32:12.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:12.465+0000 7fed56a00700 1 --2- 172.21.15.37:0/2720134841 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed58102e00 0x7fed58115d00 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fed48002a50 tx=0x7fed480046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:32:12.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:12.466+0000 7fed4ea00700 1 -- 172.21.15.37:0/2720134841 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fed48007cc0 con 0x7fed58102e00 2024-03-29T08:32:12.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:12.466+0000 7fed4ea00700 1 -- 172.21.15.37:0/2720134841 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fed4800f040 con 0x7fed58102e00 2024-03-29T08:32:12.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:12.466+0000 7fed5d600700 1 -- 172.21.15.37:0/2720134841 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed5810ff40 con 0x7fed58102e00 2024-03-29T08:32:12.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:12.466+0000 7fed4ea00700 1 -- 172.21.15.37:0/2720134841 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fed48009540 con 0x7fed58102e00 2024-03-29T08:32:12.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:12.466+0000 7fed5d600700 1 -- 172.21.15.37:0/2720134841 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed58110500 con 0x7fed58102e00 2024-03-29T08:32:12.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:12.467+0000 7fed5d600700 1 -- 172.21.15.37:0/2720134841 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed580fe550 con 0x7fed58102e00 2024-03-29T08:32:12.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:12.468+0000 7fed4ea00700 1 -- 172.21.15.37:0/2720134841 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fed480096a0 con 0x7fed58102e00 2024-03-29T08:32:12.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:12.469+0000 7fed4ea00700 1 --2- 172.21.15.37:0/2720134841 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fed3c06cc30 0x7fed3c06f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:32:12.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:12.469+0000 7fed4ea00700 1 -- 172.21.15.37:0/2720134841 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fed4808bee0 con 0x7fed58102e00 2024-03-29T08:32:12.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:12.469+0000 7fed56000700 1 --2- 172.21.15.37:0/2720134841 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fed3c06cc30 0x7fed3c06f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:32:12.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:12.470+0000 7fed56000700 1 --2- 172.21.15.37:0/2720134841 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fed3c06cc30 0x7fed3c06f0e0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fed581115a0 tx=0x7fed44009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:32:12.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:12.473+0000 7fed4ea00700 1 -- 172.21.15.37:0/2720134841 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fed480598b0 con 0x7fed58102e00 2024-03-29T08:32:12.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:12.775+0000 7fed5d600700 1 -- 172.21.15.37:0/2720134841 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fed5804bed0 con 0x7fed58102e00 2024-03-29T08:32:12.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:12.775+0000 7fed4ea00700 1 -- 172.21.15.37:0/2720134841 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7fed48059440 con 0x7fed58102e00 2024-03-29T08:32:12.777 INFO:teuthology.orchestra.run.smithi037.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-03-29T08:32:12.777 INFO:teuthology.orchestra.run.smithi037.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-03-29T08:32:12.777 INFO:teuthology.orchestra.run.smithi037.stdout: fs cephfs is degraded 2024-03-29T08:32:12.777 INFO:teuthology.orchestra.run.smithi037.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-03-29T08:32:12.777 INFO:teuthology.orchestra.run.smithi037.stdout: fs cephfs is offline because no MDS is active for it. 2024-03-29T08:32:12.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:12.778+0000 7fed5d600700 1 -- 172.21.15.37:0/2720134841 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fed3c06cc30 msgr2=0x7fed3c06f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:32:12.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:12.778+0000 7fed5d600700 1 --2- 172.21.15.37:0/2720134841 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fed3c06cc30 0x7fed3c06f0e0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fed581115a0 tx=0x7fed44009e30 comp rx=0 tx=0).stop 2024-03-29T08:32:12.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:12.778+0000 7fed5d600700 1 -- 172.21.15.37:0/2720134841 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed58102e00 msgr2=0x7fed58115d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:32:12.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:12.778+0000 7fed5d600700 1 --2- 172.21.15.37:0/2720134841 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed58102e00 0x7fed58115d00 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fed48002a50 tx=0x7fed480046b0 comp rx=0 tx=0).stop 2024-03-29T08:32:12.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:12.778+0000 7fed5d600700 1 -- 172.21.15.37:0/2720134841 shutdown_connections 2024-03-29T08:32:12.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:12.778+0000 7fed5d600700 1 --2- 172.21.15.37:0/2720134841 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fed3c06cc30 0x7fed3c06f0e0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:32:12.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:12.778+0000 7fed5d600700 1 --2- 172.21.15.37:0/2720134841 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed58102e00 0x7fed58115d00 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:32:12.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:12.778+0000 7fed5d600700 1 --2- 172.21.15.37:0/2720134841 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fed581037a0 0x7fed58116240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:32:12.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:12.778+0000 7fed5d600700 1 -- 172.21.15.37:0/2720134841 >> 172.21.15.37:0/2720134841 conn(0x7fed580f89b0 msgr2=0x7fed580f9ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:32:12.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:12.778+0000 7fed5d600700 1 -- 172.21.15.37:0/2720134841 shutdown_connections 2024-03-29T08:32:12.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:12.778+0000 7fed5d600700 1 -- 172.21.15.37:0/2720134841 wait complete. 2024-03-29T08:32:12.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:32:12 smithi148 ceph-mon[106692]: from='client.35184 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:32:12.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:32:12 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/194584145' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:32:12.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:32:12 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/4196477743' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T08:32:12.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:32:12 smithi037 ceph-mon[142325]: from='client.35184 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:32:12.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:32:12 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/194584145' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:32:12.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:32:12 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/4196477743' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T08:32:13.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:32:13 smithi148 ceph-mon[106692]: pgmap v639: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:32:13.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:32:13 smithi148 ceph-mon[106692]: from='client.35196 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:32:13.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:32:13 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/2720134841' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T08:32:13.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:32:13 smithi037 ceph-mon[142325]: pgmap v639: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:32:13.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:32:13 smithi037 ceph-mon[142325]: from='client.35196 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:32:13.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:32:13 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/2720134841' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T08:32:15.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:32:15 smithi148 ceph-mon[106692]: pgmap v640: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:32:15.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:32:15 smithi037 ceph-mon[142325]: pgmap v640: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:32:17.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:32:17 smithi148 ceph-mon[106692]: pgmap v641: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:32:17.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:32:17 smithi037 ceph-mon[142325]: pgmap v641: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:32:19.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:32:19 smithi148 ceph-mon[106692]: pgmap v642: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:32:19.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:32:19 smithi037 ceph-mon[142325]: pgmap v642: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:32:21.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:32:21 smithi148 ceph-mon[106692]: pgmap v643: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:32:21.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:32:21 smithi037 ceph-mon[142325]: pgmap v643: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:32:23.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:32:23 smithi148 ceph-mon[106692]: pgmap v644: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:32:23.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:32:23 smithi037 ceph-mon[142325]: pgmap v644: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:32:25.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:32:25 smithi148 ceph-mon[106692]: pgmap v645: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:32:25.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:32:25 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:32:25.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:32:25 smithi037 ceph-mon[142325]: pgmap v645: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:32:25.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:32:25 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:32:27.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:32:27 smithi148 ceph-mon[106692]: pgmap v646: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:32:27.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:32:27 smithi037 ceph-mon[142325]: pgmap v646: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:32:29.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:32:29 smithi037 ceph-mon[142325]: pgmap v647: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:32:30.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:32:29 smithi148 ceph-mon[106692]: pgmap v647: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:32:31.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:32:31 smithi037 ceph-mon[142325]: pgmap v648: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:32:32.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:32:31 smithi148 ceph-mon[106692]: pgmap v648: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:32:33.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:32:33 smithi037 ceph-mon[142325]: pgmap v649: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:32:34.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:32:33 smithi148 ceph-mon[106692]: pgmap v649: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:32:35.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:32:35 smithi037 ceph-mon[142325]: pgmap v650: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:32:36.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:32:35 smithi148 ceph-mon[106692]: pgmap v650: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:32:37.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:32:37 smithi037 ceph-mon[142325]: pgmap v651: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:32:38.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:32:37 smithi148 ceph-mon[106692]: pgmap v651: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:32:39.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:32:39 smithi037 ceph-mon[142325]: pgmap v652: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:32:40.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:32:39 smithi148 ceph-mon[106692]: pgmap v652: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:32:40.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:32:40 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:32:41.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:32:40 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:32:41.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:32:41 smithi037 ceph-mon[142325]: pgmap v653: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:32:42.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:32:41 smithi148 ceph-mon[106692]: pgmap v653: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:32:42.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:42.936+0000 7fc177200700 1 -- 172.21.15.37:0/3052053168 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc1701037a0 msgr2=0x7fc1700fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:32:42.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:42.936+0000 7fc177200700 1 --2- 172.21.15.37:0/3052053168 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc1701037a0 0x7fc1700fce10 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fc15c004940 tx=0x7fc15c009e30 comp rx=0 tx=0).stop 2024-03-29T08:32:42.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:42.936+0000 7fc177200700 1 -- 172.21.15.37:0/3052053168 shutdown_connections 2024-03-29T08:32:42.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:42.936+0000 7fc177200700 1 --2- 172.21.15.37:0/3052053168 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc1701037a0 0x7fc1700fce10 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:32:42.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:42.936+0000 7fc177200700 1 --2- 172.21.15.37:0/3052053168 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc170102e00 0x7fc1701031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:32:42.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:42.936+0000 7fc177200700 1 -- 172.21.15.37:0/3052053168 >> 172.21.15.37:0/3052053168 conn(0x7fc1700f89b0 msgr2=0x7fc1700fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:32:42.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:42.936+0000 7fc177200700 1 -- 172.21.15.37:0/3052053168 shutdown_connections 2024-03-29T08:32:42.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:42.936+0000 7fc177200700 1 -- 172.21.15.37:0/3052053168 wait complete. 2024-03-29T08:32:42.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:42.937+0000 7fc177200700 1 Processor -- start 2024-03-29T08:32:42.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:42.937+0000 7fc177200700 1 -- start start 2024-03-29T08:32:42.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:42.937+0000 7fc177200700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc170102e00 0x7fc170115d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:32:42.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:42.937+0000 7fc177200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc1701037a0 0x7fc170116240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:32:42.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:42.937+0000 7fc177200700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc170116890 con 0x7fc1701037a0 2024-03-29T08:32:42.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:42.937+0000 7fc177200700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc17010fc60 con 0x7fc170102e00 2024-03-29T08:32:42.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:42.938+0000 7fc16f400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc1701037a0 0x7fc170116240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:32:42.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:42.938+0000 7fc16f400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc1701037a0 0x7fc170116240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53900/0 (socket says 172.21.15.37:53900) 2024-03-29T08:32:42.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:42.938+0000 7fc16f400700 1 -- 172.21.15.37:0/158429800 learned_addr learned my addr 172.21.15.37:0/158429800 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:32:42.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:42.938+0000 7fc16fe00700 1 --2- 172.21.15.37:0/158429800 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc170102e00 0x7fc170115d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:32:42.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:42.938+0000 7fc16f400700 1 -- 172.21.15.37:0/158429800 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc170102e00 msgr2=0x7fc170115d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:32:42.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:42.938+0000 7fc16f400700 1 --2- 172.21.15.37:0/158429800 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc170102e00 0x7fc170115d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:32:42.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:42.938+0000 7fc16f400700 1 -- 172.21.15.37:0/158429800 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc15c005040 con 0x7fc1701037a0 2024-03-29T08:32:42.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:42.938+0000 7fc16f400700 1 --2- 172.21.15.37:0/158429800 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc1701037a0 0x7fc170116240 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fc15c009390 tx=0x7fc15c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:32:42.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:42.939+0000 7fc167e00700 1 -- 172.21.15.37:0/158429800 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc15c00e040 con 0x7fc1701037a0 2024-03-29T08:32:42.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:42.939+0000 7fc167e00700 1 -- 172.21.15.37:0/158429800 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fc15c013420 con 0x7fc1701037a0 2024-03-29T08:32:42.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:42.939+0000 7fc167e00700 1 -- 172.21.15.37:0/158429800 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc15c01c630 con 0x7fc1701037a0 2024-03-29T08:32:42.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:42.939+0000 7fc177200700 1 -- 172.21.15.37:0/158429800 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc17010fee0 con 0x7fc1701037a0 2024-03-29T08:32:42.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:42.939+0000 7fc177200700 1 -- 172.21.15.37:0/158429800 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1701104a0 con 0x7fc1701037a0 2024-03-29T08:32:42.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:42.940+0000 7fc177200700 1 -- 172.21.15.37:0/158429800 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1700fe550 con 0x7fc1701037a0 2024-03-29T08:32:42.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:42.941+0000 7fc167e00700 1 -- 172.21.15.37:0/158429800 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fc15c013a00 con 0x7fc1701037a0 2024-03-29T08:32:42.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:42.942+0000 7fc167e00700 1 --2- 172.21.15.37:0/158429800 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fc15406cc30 0x7fc15406f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:32:42.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:42.943+0000 7fc167e00700 1 -- 172.21.15.37:0/158429800 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fc15c08d1b0 con 0x7fc1701037a0 2024-03-29T08:32:42.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:42.943+0000 7fc16fe00700 1 --2- 172.21.15.37:0/158429800 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fc15406cc30 0x7fc15406f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:32:42.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:42.943+0000 7fc16fe00700 1 --2- 172.21.15.37:0/158429800 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fc15406cc30 0x7fc15406f0e0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fc160007fd0 tx=0x7fc160009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:32:42.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:42.948+0000 7fc167e00700 1 -- 172.21.15.37:0/158429800 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fc15c05ac00 con 0x7fc1701037a0 2024-03-29T08:32:43.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:43.183+0000 7fc177200700 1 -- 172.21.15.37:0/158429800 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc170109da0 con 0x7fc15406cc30 2024-03-29T08:32:43.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:43.184+0000 7fc167e00700 1 -- 172.21.15.37:0/158429800 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fc170109da0 con 0x7fc15406cc30 2024-03-29T08:32:43.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:43.187+0000 7fc177200700 1 -- 172.21.15.37:0/158429800 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fc15406cc30 msgr2=0x7fc15406f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:32:43.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:43.187+0000 7fc177200700 1 --2- 172.21.15.37:0/158429800 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fc15406cc30 0x7fc15406f0e0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fc160007fd0 tx=0x7fc160009040 comp rx=0 tx=0).stop 2024-03-29T08:32:43.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:43.187+0000 7fc177200700 1 -- 172.21.15.37:0/158429800 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc1701037a0 msgr2=0x7fc170116240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:32:43.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:43.187+0000 7fc177200700 1 --2- 172.21.15.37:0/158429800 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc1701037a0 0x7fc170116240 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fc15c009390 tx=0x7fc15c009e30 comp rx=0 tx=0).stop 2024-03-29T08:32:43.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:43.187+0000 7fc177200700 1 -- 172.21.15.37:0/158429800 shutdown_connections 2024-03-29T08:32:43.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:43.187+0000 7fc177200700 1 --2- 172.21.15.37:0/158429800 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fc15406cc30 0x7fc15406f0e0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:32:43.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:43.187+0000 7fc177200700 1 --2- 172.21.15.37:0/158429800 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc170102e00 0x7fc170115d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:32:43.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:43.187+0000 7fc177200700 1 --2- 172.21.15.37:0/158429800 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc1701037a0 0x7fc170116240 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:32:43.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:43.187+0000 7fc177200700 1 -- 172.21.15.37:0/158429800 >> 172.21.15.37:0/158429800 conn(0x7fc1700f89b0 msgr2=0x7fc1700f9ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:32:43.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:43.187+0000 7fc177200700 1 -- 172.21.15.37:0/158429800 shutdown_connections 2024-03-29T08:32:43.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:43.187+0000 7fc177200700 1 -- 172.21.15.37:0/158429800 wait complete. 2024-03-29T08:32:43.202 INFO:teuthology.orchestra.run.smithi037.stdout:true 2024-03-29T08:32:43.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:43.325+0000 7f48b6600700 1 -- 172.21.15.37:0/1994524911 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f48b0071f90 msgr2=0x7f48b0070360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:32:43.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:43.325+0000 7f48b6600700 1 --2- 172.21.15.37:0/1994524911 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f48b0071f90 0x7f48b0070360 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f48a0004970 tx=0x7f48a0009e30 comp rx=0 tx=0).stop 2024-03-29T08:32:43.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:43.325+0000 7f48b6600700 1 -- 172.21.15.37:0/1994524911 shutdown_connections 2024-03-29T08:32:43.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:43.325+0000 7f48b6600700 1 --2- 172.21.15.37:0/1994524911 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f48b00708a0 0x7f48b0070d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:32:43.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:43.325+0000 7f48b6600700 1 --2- 172.21.15.37:0/1994524911 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f48b0071f90 0x7f48b0070360 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:32:43.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:43.325+0000 7f48b6600700 1 -- 172.21.15.37:0/1994524911 >> 172.21.15.37:0/1994524911 conn(0x7f48b00f89b0 msgr2=0x7f48b00fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:32:43.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:43.325+0000 7f48b6600700 1 -- 172.21.15.37:0/1994524911 shutdown_connections 2024-03-29T08:32:43.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:43.325+0000 7f48b6600700 1 -- 172.21.15.37:0/1994524911 wait complete. 2024-03-29T08:32:43.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:43.326+0000 7f48b6600700 1 Processor -- start 2024-03-29T08:32:43.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:43.326+0000 7f48b6600700 1 -- start start 2024-03-29T08:32:43.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:43.327+0000 7f48b6600700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f48b00708a0 0x7f48b011a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:32:43.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:43.327+0000 7f48b6600700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f48b0071f90 0x7f48b011a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:32:43.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:43.327+0000 7f48b6600700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48b011ac50 con 0x7f48b0071f90 2024-03-29T08:32:43.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:43.327+0000 7f48b6600700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48b0114020 con 0x7f48b00708a0 2024-03-29T08:32:43.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:43.327+0000 7f48aea00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f48b0071f90 0x7f48b011a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:32:43.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:43.327+0000 7f48aea00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f48b0071f90 0x7f48b011a600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53914/0 (socket says 172.21.15.37:53914) 2024-03-29T08:32:43.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:43.327+0000 7f48aea00700 1 -- 172.21.15.37:0/1006188524 learned_addr learned my addr 172.21.15.37:0/1006188524 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:32:43.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:43.327+0000 7f48af400700 1 --2- 172.21.15.37:0/1006188524 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f48b00708a0 0x7f48b011a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:32:43.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:43.327+0000 7f48aea00700 1 -- 172.21.15.37:0/1006188524 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f48b00708a0 msgr2=0x7f48b011a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:32:43.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:43.327+0000 7f48aea00700 1 --2- 172.21.15.37:0/1006188524 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f48b00708a0 0x7f48b011a0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:32:43.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:43.327+0000 7f48aea00700 1 -- 172.21.15.37:0/1006188524 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48a0005040 con 0x7f48b0071f90 2024-03-29T08:32:43.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:43.328+0000 7f48aea00700 1 --2- 172.21.15.37:0/1006188524 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f48b0071f90 0x7f48b011a600 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f489c00c8c0 tx=0x7f489c0044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:32:43.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:43.328+0000 7f48a7400700 1 -- 172.21.15.37:0/1006188524 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f489c00a980 con 0x7f48b0071f90 2024-03-29T08:32:43.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:43.328+0000 7f48a7400700 1 -- 172.21.15.37:0/1006188524 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f489c00ce60 con 0x7f48b0071f90 2024-03-29T08:32:43.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:43.328+0000 7f48b6600700 1 -- 172.21.15.37:0/1006188524 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48b0114300 con 0x7f48b0071f90 2024-03-29T08:32:43.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:43.328+0000 7f48a7400700 1 -- 172.21.15.37:0/1006188524 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f489c0113a0 con 0x7f48b0071f90 2024-03-29T08:32:43.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:43.328+0000 7f48b6600700 1 -- 172.21.15.37:0/1006188524 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48b0114840 con 0x7f48b0071f90 2024-03-29T08:32:43.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:43.329+0000 7f48b6600700 1 -- 172.21.15.37:0/1006188524 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48b0071aa0 con 0x7f48b0071f90 2024-03-29T08:32:43.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:43.330+0000 7f48a7400700 1 -- 172.21.15.37:0/1006188524 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f489c012400 con 0x7f48b0071f90 2024-03-29T08:32:43.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:43.331+0000 7f48a7400700 1 --2- 172.21.15.37:0/1006188524 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f489406cc30 0x7f489406f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:32:43.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:43.331+0000 7f48a7400700 1 -- 172.21.15.37:0/1006188524 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f489c08c1d0 con 0x7f48b0071f90 2024-03-29T08:32:43.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:43.331+0000 7f48af400700 1 --2- 172.21.15.37:0/1006188524 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f489406cc30 0x7f489406f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:32:43.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:43.332+0000 7f48af400700 1 --2- 172.21.15.37:0/1006188524 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f489406cc30 0x7f489406f0e0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f48a0009fd0 tx=0x7f48a0009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:32:43.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:43.335+0000 7f48a7400700 1 -- 172.21.15.37:0/1006188524 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f489c059c20 con 0x7f48b0071f90 2024-03-29T08:32:43.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:43.569+0000 7f48b6600700 1 -- 172.21.15.37:0/1006188524 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f48b01103e0 con 0x7f489406cc30 2024-03-29T08:32:43.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:43.570+0000 7f48a7400700 1 -- 172.21.15.37:0/1006188524 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f48b01103e0 con 0x7f489406cc30 2024-03-29T08:32:43.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:43.572+0000 7f48b6600700 1 -- 172.21.15.37:0/1006188524 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f489406cc30 msgr2=0x7f489406f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:32:43.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:43.572+0000 7f48b6600700 1 --2- 172.21.15.37:0/1006188524 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f489406cc30 0x7f489406f0e0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f48a0009fd0 tx=0x7f48a0009e30 comp rx=0 tx=0).stop 2024-03-29T08:32:43.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:43.572+0000 7f48b6600700 1 -- 172.21.15.37:0/1006188524 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f48b0071f90 msgr2=0x7f48b011a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:32:43.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:43.572+0000 7f48b6600700 1 --2- 172.21.15.37:0/1006188524 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f48b0071f90 0x7f48b011a600 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f489c00c8c0 tx=0x7f489c0044c0 comp rx=0 tx=0).stop 2024-03-29T08:32:43.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:43.572+0000 7f48b6600700 1 -- 172.21.15.37:0/1006188524 shutdown_connections 2024-03-29T08:32:43.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:43.572+0000 7f48b6600700 1 --2- 172.21.15.37:0/1006188524 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f489406cc30 0x7f489406f0e0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:32:43.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:43.572+0000 7f48b6600700 1 --2- 172.21.15.37:0/1006188524 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f48b00708a0 0x7f48b011a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:32:43.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:43.572+0000 7f48b6600700 1 --2- 172.21.15.37:0/1006188524 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f48b0071f90 0x7f48b011a600 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:32:43.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:43.572+0000 7f48b6600700 1 -- 172.21.15.37:0/1006188524 >> 172.21.15.37:0/1006188524 conn(0x7f48b00f89b0 msgr2=0x7f48b00f9b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:32:43.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:43.572+0000 7f48b6600700 1 -- 172.21.15.37:0/1006188524 shutdown_connections 2024-03-29T08:32:43.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:43.572+0000 7f48b6600700 1 -- 172.21.15.37:0/1006188524 wait complete. 2024-03-29T08:32:43.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:43.712+0000 7f20b6800700 1 -- 172.21.15.37:0/131518846 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f20b0070c10 msgr2=0x7f20b0108fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:32:43.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:43.712+0000 7f20b6800700 1 --2- 172.21.15.37:0/131518846 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f20b0070c10 0x7f20b0108fa0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f209c004970 tx=0x7f209c009e30 comp rx=0 tx=0).stop 2024-03-29T08:32:43.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:43.712+0000 7f20b6800700 1 -- 172.21.15.37:0/131518846 shutdown_connections 2024-03-29T08:32:43.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:43.712+0000 7f20b6800700 1 --2- 172.21.15.37:0/131518846 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f20b0070c10 0x7f20b0108fa0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:32:43.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:43.712+0000 7f20b6800700 1 --2- 172.21.15.37:0/131518846 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f20b0070300 0x7f20b00706d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:32:43.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:43.712+0000 7f20b6800700 1 -- 172.21.15.37:0/131518846 >> 172.21.15.37:0/131518846 conn(0x7f20b00f8760 msgr2=0x7f20b00fab70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:32:43.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:43.712+0000 7f20b6800700 1 -- 172.21.15.37:0/131518846 shutdown_connections 2024-03-29T08:32:43.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:43.712+0000 7f20b6800700 1 -- 172.21.15.37:0/131518846 wait complete. 2024-03-29T08:32:43.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:43.713+0000 7f20b6800700 1 Processor -- start 2024-03-29T08:32:43.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:43.713+0000 7f20b6800700 1 -- start start 2024-03-29T08:32:43.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:43.714+0000 7f20b6800700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f20b0070300 0x7f20b0115ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:32:43.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:43.714+0000 7f20b6800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f20b0070c10 0x7f20b0116010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:32:43.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:43.714+0000 7f20b6800700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20b0116660 con 0x7f20b0070c10 2024-03-29T08:32:43.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:43.714+0000 7f20b6800700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20b010fa50 con 0x7f20b0070300 2024-03-29T08:32:43.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:43.714+0000 7f20b4a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f20b0070c10 0x7f20b0116010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:32:43.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:43.714+0000 7f20b4a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f20b0070c10 0x7f20b0116010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53938/0 (socket says 172.21.15.37:53938) 2024-03-29T08:32:43.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:43.714+0000 7f20b4a00700 1 -- 172.21.15.37:0/2686381006 learned_addr learned my addr 172.21.15.37:0/2686381006 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:32:43.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:43.714+0000 7f20b5400700 1 --2- 172.21.15.37:0/2686381006 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f20b0070300 0x7f20b0115ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:32:43.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:43.714+0000 7f20b4a00700 1 -- 172.21.15.37:0/2686381006 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f20b0070300 msgr2=0x7f20b0115ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:32:43.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:43.714+0000 7f20b4a00700 1 --2- 172.21.15.37:0/2686381006 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f20b0070300 0x7f20b0115ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:32:43.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:43.714+0000 7f20b4a00700 1 -- 172.21.15.37:0/2686381006 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f209c005040 con 0x7f20b0070c10 2024-03-29T08:32:43.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:43.714+0000 7f20b5400700 1 --2- 172.21.15.37:0/2686381006 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f20b0070300 0x7f20b0115ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:32:43.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:43.715+0000 7f20b4a00700 1 --2- 172.21.15.37:0/2686381006 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f20b0070c10 0x7f20b0116010 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f209c004f50 tx=0x7f209c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:32:43.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:43.715+0000 7f20ad600700 1 -- 172.21.15.37:0/2686381006 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f209c00e040 con 0x7f20b0070c10 2024-03-29T08:32:43.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:43.715+0000 7f20ad600700 1 -- 172.21.15.37:0/2686381006 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f209c013420 con 0x7f20b0070c10 2024-03-29T08:32:43.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:43.715+0000 7f20ad600700 1 -- 172.21.15.37:0/2686381006 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f209c01c610 con 0x7f20b0070c10 2024-03-29T08:32:43.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:43.715+0000 7f20b6800700 1 -- 172.21.15.37:0/2686381006 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20b010fcd0 con 0x7f20b0070c10 2024-03-29T08:32:43.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:43.715+0000 7f20b6800700 1 -- 172.21.15.37:0/2686381006 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20b0110210 con 0x7f20b0070c10 2024-03-29T08:32:43.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:43.716+0000 7f20b6800700 1 -- 172.21.15.37:0/2686381006 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20b0071a40 con 0x7f20b0070c10 2024-03-29T08:32:43.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:43.717+0000 7f20ad600700 1 -- 172.21.15.37:0/2686381006 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f209c013590 con 0x7f20b0070c10 2024-03-29T08:32:43.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:43.718+0000 7f20ad600700 1 --2- 172.21.15.37:0/2686381006 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f209806cc20 0x7f209806f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:32:43.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:43.718+0000 7f20ad600700 1 -- 172.21.15.37:0/2686381006 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f209c08d260 con 0x7f20b0070c10 2024-03-29T08:32:43.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:43.719+0000 7f20b5400700 1 --2- 172.21.15.37:0/2686381006 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f209806cc20 0x7f209806f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:32:43.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:43.719+0000 7f20b5400700 1 --2- 172.21.15.37:0/2686381006 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f209806cc20 0x7f209806f0d0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f20a4007fd0 tx=0x7f20a4009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:32:43.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:43.723+0000 7f20ad600700 1 -- 172.21.15.37:0/2686381006 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f209c05acb0 con 0x7f20b0070c10 2024-03-29T08:32:43.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:32:43 smithi037 ceph-mon[142325]: pgmap v654: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:32:43.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:43.948+0000 7f20b6800700 1 -- 172.21.15.37:0/2686381006 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f20b010bdf0 con 0x7f209806cc20 2024-03-29T08:32:43.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:43.958+0000 7f20ad600700 1 -- 172.21.15.37:0/2686381006 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f20b010bdf0 con 0x7f209806cc20 2024-03-29T08:32:43.958 INFO:teuthology.orchestra.run.smithi037.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-29T08:32:43.958 INFO:teuthology.orchestra.run.smithi037.stdout:alertmanager.smithi037 smithi037 *:9093,9094 running (22m) 14m ago 35m 14.9M - 0.25.0 c8568f914cd2 2f9f9746ed0f 2024-03-29T08:32:43.958 INFO:teuthology.orchestra.run.smithi037.stdout:crash.smithi037 smithi037 running (19m) 14m ago 35m 6647k - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 178d416a3186 2024-03-29T08:32:43.958 INFO:teuthology.orchestra.run.smithi037.stdout:crash.smithi148 smithi148 running (19m) 14m ago 34m 6656k - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 23666989d58f 2024-03-29T08:32:43.958 INFO:teuthology.orchestra.run.smithi037.stdout:grafana.smithi037 smithi037 *:3000 running (22m) 14m ago 35m 71.9M - 9.4.7 954c08fa6188 6b976fe49952 2024-03-29T08:32:43.958 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi037.iifsmh smithi037 running (14m) 14m ago 29m 10.8M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 2d97f562cc5b 2024-03-29T08:32:43.958 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi037.mmwgxe smithi037 running (15m) 14m ago 29m 10.9M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff ec37037f46e8 2024-03-29T08:32:43.958 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi148.qukpjh smithi148 running (28m) 14m ago 28m 15.3M - 17.2.7-789-g4213e17c cf22eb8683be 8e469782a273 2024-03-29T08:32:43.959 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi148.sqteih smithi148 running (14m) 14m ago 29m 10.7M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 44c15c77037a 2024-03-29T08:32:43.959 INFO:teuthology.orchestra.run.smithi037.stdout:mgr.smithi037.wjikii smithi037 *:8443,9283,8765 running (24m) 14m ago 37m 506M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff c5a1417b9417 2024-03-29T08:32:43.959 INFO:teuthology.orchestra.run.smithi037.stdout:mgr.smithi148.onhgvm smithi148 *:8443,9283,8765 running (23m) 14m ago 34m 450M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff fcfdbce062bd 2024-03-29T08:32:43.959 INFO:teuthology.orchestra.run.smithi037.stdout:mon.smithi037 smithi037 running (20m) 14m ago 37m 54.8M 2048M 18.2.2-1015-gd0e392d9 1cc2f45c17ff eea2060783af 2024-03-29T08:32:43.959 INFO:teuthology.orchestra.run.smithi037.stdout:mon.smithi148 smithi148 running (20m) 14m ago 34m 43.7M 2048M 18.2.2-1015-gd0e392d9 1cc2f45c17ff ad4b2c5748be 2024-03-29T08:32:43.959 INFO:teuthology.orchestra.run.smithi037.stdout:node-exporter.smithi037 smithi037 *:9100 running (23m) 14m ago 35m 7625k - 1.5.0 0da6a335fe13 26b8ae9cb878 2024-03-29T08:32:43.959 INFO:teuthology.orchestra.run.smithi037.stdout:node-exporter.smithi148 smithi148 *:9100 running (23m) 14m ago 34m 8166k - 1.5.0 0da6a335fe13 98414e4e64ed 2024-03-29T08:32:43.959 INFO:teuthology.orchestra.run.smithi037.stdout:osd.0 smithi037 running (18m) 14m ago 33m 104M 1619M 18.2.2-1015-gd0e392d9 1cc2f45c17ff c0c584f6245c 2024-03-29T08:32:43.959 INFO:teuthology.orchestra.run.smithi037.stdout:osd.1 smithi037 running (18m) 14m ago 32m 100M 1619M 18.2.2-1015-gd0e392d9 1cc2f45c17ff a4a2ff4cc83d 2024-03-29T08:32:43.959 INFO:teuthology.orchestra.run.smithi037.stdout:osd.2 smithi037 running (17m) 14m ago 32m 82.3M 1619M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 8b1bc71aa79e 2024-03-29T08:32:43.959 INFO:teuthology.orchestra.run.smithi037.stdout:osd.3 smithi148 running (16m) 14m ago 31m 105M 2642M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 1c656ab8ff86 2024-03-29T08:32:43.959 INFO:teuthology.orchestra.run.smithi037.stdout:osd.4 smithi148 running (16m) 14m ago 31m 97.3M 2642M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 966a9de11d22 2024-03-29T08:32:43.959 INFO:teuthology.orchestra.run.smithi037.stdout:osd.5 smithi148 running (15m) 14m ago 30m 85.8M 2642M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 474c4f9212ce 2024-03-29T08:32:43.959 INFO:teuthology.orchestra.run.smithi037.stdout:prometheus.smithi037 smithi037 *:9095 running (22m) 14m ago 34m 54.4M - 2.43.0 a07b618ecd1d 30c0223ec589 2024-03-29T08:32:43.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:43.960+0000 7f20b6800700 1 -- 172.21.15.37:0/2686381006 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f209806cc20 msgr2=0x7f209806f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:32:43.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:43.960+0000 7f20b6800700 1 --2- 172.21.15.37:0/2686381006 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f209806cc20 0x7f209806f0d0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f20a4007fd0 tx=0x7f20a4009040 comp rx=0 tx=0).stop 2024-03-29T08:32:43.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:43.960+0000 7f20b6800700 1 -- 172.21.15.37:0/2686381006 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f20b0070c10 msgr2=0x7f20b0116010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:32:43.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:43.960+0000 7f20b6800700 1 --2- 172.21.15.37:0/2686381006 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f20b0070c10 0x7f20b0116010 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f209c004f50 tx=0x7f209c009e30 comp rx=0 tx=0).stop 2024-03-29T08:32:43.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:43.960+0000 7f20b6800700 1 -- 172.21.15.37:0/2686381006 shutdown_connections 2024-03-29T08:32:43.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:43.960+0000 7f20b6800700 1 --2- 172.21.15.37:0/2686381006 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f209806cc20 0x7f209806f0d0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:32:43.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:43.960+0000 7f20b6800700 1 --2- 172.21.15.37:0/2686381006 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f20b0070300 0x7f20b0115ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:32:43.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:43.960+0000 7f20b6800700 1 --2- 172.21.15.37:0/2686381006 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f20b0070c10 0x7f20b0116010 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:32:43.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:43.960+0000 7f20b6800700 1 -- 172.21.15.37:0/2686381006 >> 172.21.15.37:0/2686381006 conn(0x7f20b00f8760 msgr2=0x7f20b00f9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:32:43.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:43.960+0000 7f20b6800700 1 -- 172.21.15.37:0/2686381006 shutdown_connections 2024-03-29T08:32:43.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:43.960+0000 7f20b6800700 1 -- 172.21.15.37:0/2686381006 wait complete. 2024-03-29T08:32:44.059 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:32:43 smithi148 ceph-mon[106692]: pgmap v654: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:32:44.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:44.097+0000 7fa64da00700 1 -- 172.21.15.37:0/4031081039 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa6480fde90 msgr2=0x7fa6480fe260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:32:44.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:44.097+0000 7fa64da00700 1 --2- 172.21.15.37:0/4031081039 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa6480fde90 0x7fa6480fe260 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fa638004940 tx=0x7fa638009e30 comp rx=0 tx=0).stop 2024-03-29T08:32:44.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:44.097+0000 7fa64da00700 1 -- 172.21.15.37:0/4031081039 shutdown_connections 2024-03-29T08:32:44.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:44.097+0000 7fa64da00700 1 --2- 172.21.15.37:0/4031081039 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa6480fe830 0x7fa648106f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:32:44.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:44.097+0000 7fa64da00700 1 --2- 172.21.15.37:0/4031081039 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa6480fde90 0x7fa6480fe260 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:32:44.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:44.097+0000 7fa64da00700 1 -- 172.21.15.37:0/4031081039 >> 172.21.15.37:0/4031081039 conn(0x7fa648072bc0 msgr2=0x7fa648072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:32:44.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:44.098+0000 7fa64da00700 1 -- 172.21.15.37:0/4031081039 shutdown_connections 2024-03-29T08:32:44.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:44.098+0000 7fa64da00700 1 -- 172.21.15.37:0/4031081039 wait complete. 2024-03-29T08:32:44.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:44.098+0000 7fa64da00700 1 Processor -- start 2024-03-29T08:32:44.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:44.099+0000 7fa64da00700 1 -- start start 2024-03-29T08:32:44.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:44.099+0000 7fa64da00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa6480fde90 0x7fa648115c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:32:44.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:44.099+0000 7fa64da00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa6480fe830 0x7fa6481161c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:32:44.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:44.099+0000 7fa64da00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa648116810 con 0x7fa6480fde90 2024-03-29T08:32:44.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:44.099+0000 7fa64da00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa64810fbe0 con 0x7fa6480fe830 2024-03-29T08:32:44.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:44.099+0000 7fa646a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa6480fde90 0x7fa648115c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:32:44.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:44.099+0000 7fa646a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa6480fde90 0x7fa648115c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53950/0 (socket says 172.21.15.37:53950) 2024-03-29T08:32:44.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:44.099+0000 7fa646a00700 1 -- 172.21.15.37:0/1624944321 learned_addr learned my addr 172.21.15.37:0/1624944321 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:32:44.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:44.099+0000 7fa646000700 1 --2- 172.21.15.37:0/1624944321 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa6480fe830 0x7fa6481161c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:32:44.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:44.100+0000 7fa646a00700 1 -- 172.21.15.37:0/1624944321 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa6480fe830 msgr2=0x7fa6481161c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:32:44.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:44.100+0000 7fa646a00700 1 --2- 172.21.15.37:0/1624944321 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa6480fe830 0x7fa6481161c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:32:44.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:44.100+0000 7fa646a00700 1 -- 172.21.15.37:0/1624944321 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa638005040 con 0x7fa6480fde90 2024-03-29T08:32:44.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:44.100+0000 7fa646a00700 1 --2- 172.21.15.37:0/1624944321 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa6480fde90 0x7fa648115c80 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fa638000c00 tx=0x7fa638009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:32:44.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:44.100+0000 7fa63ea00700 1 -- 172.21.15.37:0/1624944321 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa638021070 con 0x7fa6480fde90 2024-03-29T08:32:44.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:44.100+0000 7fa63ea00700 1 -- 172.21.15.37:0/1624944321 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fa638013e00 con 0x7fa6480fde90 2024-03-29T08:32:44.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:44.100+0000 7fa63ea00700 1 -- 172.21.15.37:0/1624944321 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa63800e070 con 0x7fa6480fde90 2024-03-29T08:32:44.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:44.100+0000 7fa64da00700 1 -- 172.21.15.37:0/1624944321 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa64810fe60 con 0x7fa6480fde90 2024-03-29T08:32:44.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:44.100+0000 7fa64da00700 1 -- 172.21.15.37:0/1624944321 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6481103a0 con 0x7fa6480fde90 2024-03-29T08:32:44.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:44.101+0000 7fa64da00700 1 -- 172.21.15.37:0/1624944321 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa648106a20 con 0x7fa6480fde90 2024-03-29T08:32:44.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:44.102+0000 7fa63ea00700 1 -- 172.21.15.37:0/1624944321 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fa638013950 con 0x7fa6480fde90 2024-03-29T08:32:44.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:44.103+0000 7fa63ea00700 1 --2- 172.21.15.37:0/1624944321 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fa62c06cc30 0x7fa62c06f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:32:44.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:44.103+0000 7fa63ea00700 1 -- 172.21.15.37:0/1624944321 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fa63808d050 con 0x7fa6480fde90 2024-03-29T08:32:44.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:44.104+0000 7fa646000700 1 --2- 172.21.15.37:0/1624944321 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fa62c06cc30 0x7fa62c06f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:32:44.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:44.104+0000 7fa646000700 1 --2- 172.21.15.37:0/1624944321 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fa62c06cc30 0x7fa62c06f0e0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fa634004600 tx=0x7fa63400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:32:44.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:44.108+0000 7fa63ea00700 1 -- 172.21.15.37:0/1624944321 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fa63805ab20 con 0x7fa6480fde90 2024-03-29T08:32:44.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:44.408+0000 7fa64da00700 1 -- 172.21.15.37:0/1624944321 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fa648109dc0 con 0x7fa6480fde90 2024-03-29T08:32:44.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:44.409+0000 7fa63ea00700 1 -- 172.21.15.37:0/1624944321 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+847 (secure 0 0 0) 0x7fa63805a6b0 con 0x7fa6480fde90 2024-03-29T08:32:44.409 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-29T08:32:44.409 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": { 2024-03-29T08:32:44.409 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 2 2024-03-29T08:32:44.409 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:32:44.410 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": { 2024-03-29T08:32:44.410 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 2 2024-03-29T08:32:44.410 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:32:44.410 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": { 2024-03-29T08:32:44.410 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 6 2024-03-29T08:32:44.410 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:32:44.410 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": { 2024-03-29T08:32:44.410 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 17.2.7-789-g4213e17c (4213e17c4de67016ed14f55d22503f4923b3a901) quincy (stable)": 1, 2024-03-29T08:32:44.410 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 3 2024-03-29T08:32:44.410 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:32:44.410 INFO:teuthology.orchestra.run.smithi037.stdout: "overall": { 2024-03-29T08:32:44.410 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 17.2.7-789-g4213e17c (4213e17c4de67016ed14f55d22503f4923b3a901) quincy (stable)": 1, 2024-03-29T08:32:44.410 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 13 2024-03-29T08:32:44.411 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-29T08:32:44.411 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-29T08:32:44.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:44.411+0000 7fa64da00700 1 -- 172.21.15.37:0/1624944321 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fa62c06cc30 msgr2=0x7fa62c06f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:32:44.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:44.411+0000 7fa64da00700 1 --2- 172.21.15.37:0/1624944321 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fa62c06cc30 0x7fa62c06f0e0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fa634004600 tx=0x7fa63400b040 comp rx=0 tx=0).stop 2024-03-29T08:32:44.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:44.411+0000 7fa64da00700 1 -- 172.21.15.37:0/1624944321 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa6480fde90 msgr2=0x7fa648115c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:32:44.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:44.411+0000 7fa64da00700 1 --2- 172.21.15.37:0/1624944321 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa6480fde90 0x7fa648115c80 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fa638000c00 tx=0x7fa638009e30 comp rx=0 tx=0).stop 2024-03-29T08:32:44.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:44.411+0000 7fa64da00700 1 -- 172.21.15.37:0/1624944321 shutdown_connections 2024-03-29T08:32:44.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:44.411+0000 7fa64da00700 1 --2- 172.21.15.37:0/1624944321 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fa62c06cc30 0x7fa62c06f0e0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:32:44.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:44.411+0000 7fa64da00700 1 --2- 172.21.15.37:0/1624944321 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa6480fde90 0x7fa648115c80 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:32:44.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:44.411+0000 7fa64da00700 1 --2- 172.21.15.37:0/1624944321 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa6480fe830 0x7fa6481161c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:32:44.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:44.411+0000 7fa64da00700 1 -- 172.21.15.37:0/1624944321 >> 172.21.15.37:0/1624944321 conn(0x7fa648072bc0 msgr2=0x7fa6480fb7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:32:44.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:44.412+0000 7fa64da00700 1 -- 172.21.15.37:0/1624944321 shutdown_connections 2024-03-29T08:32:44.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:44.412+0000 7fa64da00700 1 -- 172.21.15.37:0/1624944321 wait complete. 2024-03-29T08:32:44.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:44.550+0000 7fb17f600700 1 -- 172.21.15.37:0/4045509387 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb1780fde90 msgr2=0x7fb1780fe260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:32:44.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:44.550+0000 7fb17f600700 1 --2- 172.21.15.37:0/4045509387 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb1780fde90 0x7fb1780fe260 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fb16c004940 tx=0x7fb16c009e30 comp rx=0 tx=0).stop 2024-03-29T08:32:44.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:44.550+0000 7fb17f600700 1 -- 172.21.15.37:0/4045509387 shutdown_connections 2024-03-29T08:32:44.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:44.550+0000 7fb17f600700 1 --2- 172.21.15.37:0/4045509387 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb1780fe830 0x7fb178106f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:32:44.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:44.550+0000 7fb17f600700 1 --2- 172.21.15.37:0/4045509387 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb1780fde90 0x7fb1780fe260 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:32:44.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:44.550+0000 7fb17f600700 1 -- 172.21.15.37:0/4045509387 >> 172.21.15.37:0/4045509387 conn(0x7fb178072bc0 msgr2=0x7fb178072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:32:44.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:44.550+0000 7fb17f600700 1 -- 172.21.15.37:0/4045509387 shutdown_connections 2024-03-29T08:32:44.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:44.550+0000 7fb17f600700 1 -- 172.21.15.37:0/4045509387 wait complete. 2024-03-29T08:32:44.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:44.551+0000 7fb17f600700 1 Processor -- start 2024-03-29T08:32:44.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:44.551+0000 7fb17f600700 1 -- start start 2024-03-29T08:32:44.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:44.551+0000 7fb17f600700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb1780fde90 0x7fb1780fbd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:32:44.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:44.551+0000 7fb17f600700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb1780fe830 0x7fb1780fc290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:32:44.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:44.551+0000 7fb17f600700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1780fc820 con 0x7fb1780fde90 2024-03-29T08:32:44.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:44.551+0000 7fb17f600700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1780fc960 con 0x7fb1780fe830 2024-03-29T08:32:44.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:44.551+0000 7fb17cc00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb1780fde90 0x7fb1780fbd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:32:44.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:44.552+0000 7fb17cc00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb1780fde90 0x7fb1780fbd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53960/0 (socket says 172.21.15.37:53960) 2024-03-29T08:32:44.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:44.552+0000 7fb17cc00700 1 -- 172.21.15.37:0/1685227226 learned_addr learned my addr 172.21.15.37:0/1685227226 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:32:44.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:44.552+0000 7fb177e00700 1 --2- 172.21.15.37:0/1685227226 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb1780fe830 0x7fb1780fc290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:32:44.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:44.552+0000 7fb17cc00700 1 -- 172.21.15.37:0/1685227226 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb1780fe830 msgr2=0x7fb1780fc290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:32:44.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:44.552+0000 7fb17cc00700 1 --2- 172.21.15.37:0/1685227226 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb1780fe830 0x7fb1780fc290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:32:44.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:44.552+0000 7fb17cc00700 1 -- 172.21.15.37:0/1685227226 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb16c005040 con 0x7fb1780fde90 2024-03-29T08:32:44.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:44.552+0000 7fb17cc00700 1 --2- 172.21.15.37:0/1685227226 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb1780fde90 0x7fb1780fbd50 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fb16c009360 tx=0x7fb16c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:32:44.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:44.552+0000 7fb174c00700 1 -- 172.21.15.37:0/1685227226 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb16c00e040 con 0x7fb1780fde90 2024-03-29T08:32:44.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:44.552+0000 7fb174c00700 1 -- 172.21.15.37:0/1685227226 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fb16c013420 con 0x7fb1780fde90 2024-03-29T08:32:44.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:44.552+0000 7fb17f600700 1 -- 172.21.15.37:0/1685227226 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1780fcbb0 con 0x7fb1780fde90 2024-03-29T08:32:44.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:44.553+0000 7fb174c00700 1 -- 172.21.15.37:0/1685227226 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb16c01c630 con 0x7fb1780fde90 2024-03-29T08:32:44.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:44.553+0000 7fb17f600700 1 -- 172.21.15.37:0/1685227226 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb17811a1a0 con 0x7fb1780fde90 2024-03-29T08:32:44.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:44.554+0000 7fb17f600700 1 -- 172.21.15.37:0/1685227226 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb178065e30 con 0x7fb1780fde90 2024-03-29T08:32:44.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:44.555+0000 7fb174c00700 1 -- 172.21.15.37:0/1685227226 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fb16c0139e0 con 0x7fb1780fde90 2024-03-29T08:32:44.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:44.555+0000 7fb174c00700 1 --2- 172.21.15.37:0/1685227226 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fb16006cc20 0x7fb16006f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:32:44.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:44.556+0000 7fb174c00700 1 -- 172.21.15.37:0/1685227226 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fb16c08d330 con 0x7fb1780fde90 2024-03-29T08:32:44.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:44.556+0000 7fb177e00700 1 --2- 172.21.15.37:0/1685227226 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fb16006cc20 0x7fb16006f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:32:44.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:44.556+0000 7fb177e00700 1 --2- 172.21.15.37:0/1685227226 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fb16006cc20 0x7fb16006f0d0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fb164007fd0 tx=0x7fb164009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:32:44.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:44.561+0000 7fb174c00700 1 -- 172.21.15.37:0/1685227226 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fb16c0572f0 con 0x7fb1780fde90 2024-03-29T08:32:44.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:44.819+0000 7fb17f600700 1 -- 172.21.15.37:0/1685227226 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fb1780fd440 con 0x7fb1780fde90 2024-03-29T08:32:44.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:44.820+0000 7fb174c00700 1 -- 172.21.15.37:0/1685227226 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 28 v28) v1 ==== 76+0+1841 (secure 0 0 0) 0x7fb16c05a910 con 0x7fb1780fde90 2024-03-29T08:32:44.820 INFO:teuthology.orchestra.run.smithi037.stdout:e28 2024-03-29T08:32:44.821 INFO:teuthology.orchestra.run.smithi037.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-29T08:32:44.821 INFO:teuthology.orchestra.run.smithi037.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T08:32:44.821 INFO:teuthology.orchestra.run.smithi037.stdout:legacy client fscid: 1 2024-03-29T08:32:44.821 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:32:44.821 INFO:teuthology.orchestra.run.smithi037.stdout:Filesystem 'cephfs' (1) 2024-03-29T08:32:44.822 INFO:teuthology.orchestra.run.smithi037.stdout:fs_name cephfs 2024-03-29T08:32:44.822 INFO:teuthology.orchestra.run.smithi037.stdout:epoch 21 2024-03-29T08:32:44.822 INFO:teuthology.orchestra.run.smithi037.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-03-29T08:32:44.822 INFO:teuthology.orchestra.run.smithi037.stdout:created 2024-03-29T08:03:29.241269+0000 2024-03-29T08:32:44.822 INFO:teuthology.orchestra.run.smithi037.stdout:modified 2024-03-29T08:17:16.237238+0000 2024-03-29T08:32:44.822 INFO:teuthology.orchestra.run.smithi037.stdout:tableserver 0 2024-03-29T08:32:44.822 INFO:teuthology.orchestra.run.smithi037.stdout:root 0 2024-03-29T08:32:44.822 INFO:teuthology.orchestra.run.smithi037.stdout:session_timeout 60 2024-03-29T08:32:44.822 INFO:teuthology.orchestra.run.smithi037.stdout:session_autoclose 300 2024-03-29T08:32:44.823 INFO:teuthology.orchestra.run.smithi037.stdout:max_file_size 1099511627776 2024-03-29T08:32:44.823 INFO:teuthology.orchestra.run.smithi037.stdout:max_xattr_size 65536 2024-03-29T08:32:44.823 INFO:teuthology.orchestra.run.smithi037.stdout:required_client_features {} 2024-03-29T08:32:44.823 INFO:teuthology.orchestra.run.smithi037.stdout:last_failure 0 2024-03-29T08:32:44.823 INFO:teuthology.orchestra.run.smithi037.stdout:last_failure_osd_epoch 82 2024-03-29T08:32:44.823 INFO:teuthology.orchestra.run.smithi037.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T08:32:44.823 INFO:teuthology.orchestra.run.smithi037.stdout:max_mds 2 2024-03-29T08:32:44.823 INFO:teuthology.orchestra.run.smithi037.stdout:in 0,1 2024-03-29T08:32:44.823 INFO:teuthology.orchestra.run.smithi037.stdout:up {} 2024-03-29T08:32:44.823 INFO:teuthology.orchestra.run.smithi037.stdout:failed 0,1 2024-03-29T08:32:44.824 INFO:teuthology.orchestra.run.smithi037.stdout:damaged 2024-03-29T08:32:44.824 INFO:teuthology.orchestra.run.smithi037.stdout:stopped 2024-03-29T08:32:44.824 INFO:teuthology.orchestra.run.smithi037.stdout:data_pools [3] 2024-03-29T08:32:44.824 INFO:teuthology.orchestra.run.smithi037.stdout:metadata_pool 2 2024-03-29T08:32:44.824 INFO:teuthology.orchestra.run.smithi037.stdout:inline_data disabled 2024-03-29T08:32:44.824 INFO:teuthology.orchestra.run.smithi037.stdout:balancer 2024-03-29T08:32:44.824 INFO:teuthology.orchestra.run.smithi037.stdout:bal_rank_mask -1 2024-03-29T08:32:44.824 INFO:teuthology.orchestra.run.smithi037.stdout:standby_count_wanted 1 2024-03-29T08:32:44.825 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:32:44.825 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:32:44.825 INFO:teuthology.orchestra.run.smithi037.stdout:Standby daemons: 2024-03-29T08:32:44.825 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:32:44.825 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi148.qukpjh{-1:24319} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.148:6826/2641327912,v1:172.21.15.148:6827/2641327912] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:32:44.825 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi037.mmwgxe{-1:34440} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.37:6828/1248142005,v1:172.21.15.37:6829/1248142005] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:32:44.825 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi037.iifsmh{-1:34444} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.37:6826/2060882959,v1:172.21.15.37:6827/2060882959] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:32:44.825 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi148.sqteih{-1:44395} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.148:6824/4078652242,v1:172.21.15.148:6825/4078652242] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:32:44.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:44.822+0000 7fb17f600700 1 -- 172.21.15.37:0/1685227226 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fb16006cc20 msgr2=0x7fb16006f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:32:44.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:44.822+0000 7fb17f600700 1 --2- 172.21.15.37:0/1685227226 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fb16006cc20 0x7fb16006f0d0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fb164007fd0 tx=0x7fb164009040 comp rx=0 tx=0).stop 2024-03-29T08:32:44.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:44.822+0000 7fb17f600700 1 -- 172.21.15.37:0/1685227226 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb1780fde90 msgr2=0x7fb1780fbd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:32:44.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:44.822+0000 7fb17f600700 1 --2- 172.21.15.37:0/1685227226 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb1780fde90 0x7fb1780fbd50 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fb16c009360 tx=0x7fb16c009e30 comp rx=0 tx=0).stop 2024-03-29T08:32:44.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:44.822+0000 7fb17f600700 1 -- 172.21.15.37:0/1685227226 shutdown_connections 2024-03-29T08:32:44.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:44.822+0000 7fb17f600700 1 --2- 172.21.15.37:0/1685227226 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fb16006cc20 0x7fb16006f0d0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:32:44.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:44.822+0000 7fb17f600700 1 --2- 172.21.15.37:0/1685227226 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb1780fde90 0x7fb1780fbd50 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:32:44.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:44.822+0000 7fb17f600700 1 --2- 172.21.15.37:0/1685227226 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb1780fe830 0x7fb1780fc290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:32:44.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:44.822+0000 7fb17f600700 1 -- 172.21.15.37:0/1685227226 >> 172.21.15.37:0/1685227226 conn(0x7fb178072bc0 msgr2=0x7fb1780ff3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:32:44.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:44.822+0000 7fb17f600700 1 -- 172.21.15.37:0/1685227226 shutdown_connections 2024-03-29T08:32:44.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:44.823+0000 7fb17f600700 1 -- 172.21.15.37:0/1685227226 wait complete. 2024-03-29T08:32:44.827 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 28 2024-03-29T08:32:44.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:32:44 smithi037 ceph-mon[142325]: from='client.35204 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:32:44.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:32:44 smithi037 ceph-mon[142325]: from='client.35208 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:32:44.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:32:44 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/1624944321' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:32:44.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:44.960+0000 7f7172800700 1 -- 172.21.15.37:0/1683652955 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f716c102e00 msgr2=0x7f716c1031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:32:44.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:44.960+0000 7f7172800700 1 --2- 172.21.15.37:0/1683652955 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f716c102e00 0x7f716c1031d0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f715c004940 tx=0x7f715c009e30 comp rx=0 tx=0).stop 2024-03-29T08:32:44.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:44.960+0000 7f7172800700 1 -- 172.21.15.37:0/1683652955 shutdown_connections 2024-03-29T08:32:44.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:44.960+0000 7f7172800700 1 --2- 172.21.15.37:0/1683652955 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f716c1037a0 0x7f716c0fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:32:44.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:44.960+0000 7f7172800700 1 --2- 172.21.15.37:0/1683652955 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f716c102e00 0x7f716c1031d0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:32:44.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:44.960+0000 7f7172800700 1 -- 172.21.15.37:0/1683652955 >> 172.21.15.37:0/1683652955 conn(0x7f716c0f89b0 msgr2=0x7f716c0fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:32:44.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:44.960+0000 7f7172800700 1 -- 172.21.15.37:0/1683652955 shutdown_connections 2024-03-29T08:32:44.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:44.960+0000 7f7172800700 1 -- 172.21.15.37:0/1683652955 wait complete. 2024-03-29T08:32:44.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:44.961+0000 7f7172800700 1 Processor -- start 2024-03-29T08:32:44.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:44.961+0000 7f7172800700 1 -- start start 2024-03-29T08:32:44.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:44.962+0000 7f7172800700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f716c102e00 0x7f716c0702c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:32:44.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:44.962+0000 7f7172800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f716c1037a0 0x7f716c070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:32:44.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:44.962+0000 7f7172800700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f716c070d90 con 0x7f716c1037a0 2024-03-29T08:32:44.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:44.962+0000 7f7172800700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f716c070ed0 con 0x7f716c102e00 2024-03-29T08:32:44.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:44.962+0000 7f716aa00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f716c1037a0 0x7f716c070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:32:44.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:44.962+0000 7f716aa00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f716c1037a0 0x7f716c070800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53982/0 (socket says 172.21.15.37:53982) 2024-03-29T08:32:44.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:44.962+0000 7f716b400700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f716c102e00 0x7f716c0702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:32:44.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:44.962+0000 7f716aa00700 1 -- 172.21.15.37:0/4185274410 learned_addr learned my addr 172.21.15.37:0/4185274410 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:32:44.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:44.963+0000 7f716aa00700 1 -- 172.21.15.37:0/4185274410 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f716c102e00 msgr2=0x7f716c0702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:32:44.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:44.963+0000 7f716aa00700 1 --2- 172.21.15.37:0/4185274410 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f716c102e00 0x7f716c0702c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:32:44.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:44.963+0000 7f716aa00700 1 -- 172.21.15.37:0/4185274410 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f715c005040 con 0x7f716c1037a0 2024-03-29T08:32:44.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:44.963+0000 7f716aa00700 1 --2- 172.21.15.37:0/4185274410 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f716c1037a0 0x7f716c070800 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f715800c890 tx=0x7f71580044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:32:44.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:44.963+0000 7f7163400700 1 -- 172.21.15.37:0/4185274410 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f715800a700 con 0x7f716c1037a0 2024-03-29T08:32:44.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:44.963+0000 7f7163400700 1 -- 172.21.15.37:0/4185274410 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f715800ad10 con 0x7f716c1037a0 2024-03-29T08:32:44.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:44.963+0000 7f7172800700 1 -- 172.21.15.37:0/4185274410 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f716c119e20 con 0x7f716c1037a0 2024-03-29T08:32:44.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:44.963+0000 7f7163400700 1 -- 172.21.15.37:0/4185274410 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7158012e70 con 0x7f716c1037a0 2024-03-29T08:32:44.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:44.963+0000 7f7172800700 1 -- 172.21.15.37:0/4185274410 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f716c11a2a0 con 0x7f716c1037a0 2024-03-29T08:32:44.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:44.964+0000 7f7172800700 1 -- 172.21.15.37:0/4185274410 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f716c0fe550 con 0x7f716c1037a0 2024-03-29T08:32:44.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:44.965+0000 7f7163400700 1 -- 172.21.15.37:0/4185274410 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f715800a860 con 0x7f716c1037a0 2024-03-29T08:32:44.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:44.966+0000 7f7163400700 1 --2- 172.21.15.37:0/4185274410 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f715006cc30 0x7f715006f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:32:44.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:44.966+0000 7f7163400700 1 -- 172.21.15.37:0/4185274410 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f715808c0a0 con 0x7f716c1037a0 2024-03-29T08:32:44.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:44.967+0000 7f716b400700 1 --2- 172.21.15.37:0/4185274410 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f715006cc30 0x7f715006f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:32:44.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:44.967+0000 7f716b400700 1 --2- 172.21.15.37:0/4185274410 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f715006cc30 0x7f715006f0e0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f715c009fd0 tx=0x7f715c009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:32:44.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:44.971+0000 7f7163400700 1 -- 172.21.15.37:0/4185274410 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f71580560e0 con 0x7f716c1037a0 2024-03-29T08:32:45.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:32:44 smithi148 ceph-mon[106692]: from='client.35204 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:32:45.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:32:44 smithi148 ceph-mon[106692]: from='client.35208 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:32:45.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:32:44 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/1624944321' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:32:45.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:45.201+0000 7f7172800700 1 -- 172.21.15.37:0/4185274410 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f716c04bed0 con 0x7f715006cc30 2024-03-29T08:32:45.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:45.203+0000 7f7163400700 1 -- 172.21.15.37:0/4185274410 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f716c04bed0 con 0x7f715006cc30 2024-03-29T08:32:45.203 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-29T08:32:45.203 INFO:teuthology.orchestra.run.smithi037.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59", 2024-03-29T08:32:45.204 INFO:teuthology.orchestra.run.smithi037.stdout: "in_progress": true, 2024-03-29T08:32:45.204 INFO:teuthology.orchestra.run.smithi037.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-29T08:32:45.204 INFO:teuthology.orchestra.run.smithi037.stdout: "services_complete": [ 2024-03-29T08:32:45.204 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr", 2024-03-29T08:32:45.204 INFO:teuthology.orchestra.run.smithi037.stdout: "mon", 2024-03-29T08:32:45.205 INFO:teuthology.orchestra.run.smithi037.stdout: "crash", 2024-03-29T08:32:45.205 INFO:teuthology.orchestra.run.smithi037.stdout: "osd" 2024-03-29T08:32:45.205 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-29T08:32:45.205 INFO:teuthology.orchestra.run.smithi037.stdout: "progress": "15/21 daemons upgraded", 2024-03-29T08:32:45.205 INFO:teuthology.orchestra.run.smithi037.stdout: "message": "Currently upgrading mds daemons", 2024-03-29T08:32:45.205 INFO:teuthology.orchestra.run.smithi037.stdout: "is_paused": false 2024-03-29T08:32:45.206 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-29T08:32:45.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:45.205+0000 7f7172800700 1 -- 172.21.15.37:0/4185274410 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f715006cc30 msgr2=0x7f715006f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:32:45.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:45.205+0000 7f7172800700 1 --2- 172.21.15.37:0/4185274410 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f715006cc30 0x7f715006f0e0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f715c009fd0 tx=0x7f715c009e30 comp rx=0 tx=0).stop 2024-03-29T08:32:45.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:45.205+0000 7f7172800700 1 -- 172.21.15.37:0/4185274410 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f716c1037a0 msgr2=0x7f716c070800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:32:45.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:45.205+0000 7f7172800700 1 --2- 172.21.15.37:0/4185274410 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f716c1037a0 0x7f716c070800 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f715800c890 tx=0x7f71580044c0 comp rx=0 tx=0).stop 2024-03-29T08:32:45.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:45.205+0000 7f7172800700 1 -- 172.21.15.37:0/4185274410 shutdown_connections 2024-03-29T08:32:45.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:45.205+0000 7f7172800700 1 --2- 172.21.15.37:0/4185274410 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f715006cc30 0x7f715006f0e0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:32:45.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:45.205+0000 7f7172800700 1 --2- 172.21.15.37:0/4185274410 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f716c102e00 0x7f716c0702c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:32:45.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:45.205+0000 7f7172800700 1 --2- 172.21.15.37:0/4185274410 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f716c1037a0 0x7f716c070800 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:32:45.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:45.205+0000 7f7172800700 1 -- 172.21.15.37:0/4185274410 >> 172.21.15.37:0/4185274410 conn(0x7f716c0f89b0 msgr2=0x7f716c0f9a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:32:45.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:45.205+0000 7f7172800700 1 -- 172.21.15.37:0/4185274410 shutdown_connections 2024-03-29T08:32:45.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:45.205+0000 7f7172800700 1 -- 172.21.15.37:0/4185274410 wait complete. 2024-03-29T08:32:45.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:45.342+0000 7fd7f0600700 1 -- 172.21.15.37:0/2950196856 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd7e81037a0 msgr2=0x7fd7e80fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:32:45.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:45.342+0000 7fd7f0600700 1 --2- 172.21.15.37:0/2950196856 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd7e81037a0 0x7fd7e80fce10 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fd7d4004940 tx=0x7fd7d4009e30 comp rx=0 tx=0).stop 2024-03-29T08:32:45.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:45.342+0000 7fd7f0600700 1 -- 172.21.15.37:0/2950196856 shutdown_connections 2024-03-29T08:32:45.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:45.342+0000 7fd7f0600700 1 --2- 172.21.15.37:0/2950196856 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd7e81037a0 0x7fd7e80fce10 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:32:45.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:45.342+0000 7fd7f0600700 1 --2- 172.21.15.37:0/2950196856 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd7e8102e00 0x7fd7e81031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:32:45.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:45.342+0000 7fd7f0600700 1 -- 172.21.15.37:0/2950196856 >> 172.21.15.37:0/2950196856 conn(0x7fd7e80f89b0 msgr2=0x7fd7e80fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:32:45.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:45.342+0000 7fd7f0600700 1 -- 172.21.15.37:0/2950196856 shutdown_connections 2024-03-29T08:32:45.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:45.342+0000 7fd7f0600700 1 -- 172.21.15.37:0/2950196856 wait complete. 2024-03-29T08:32:45.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:45.343+0000 7fd7f0600700 1 Processor -- start 2024-03-29T08:32:45.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:45.343+0000 7fd7f0600700 1 -- start start 2024-03-29T08:32:45.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:45.344+0000 7fd7f0600700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd7e8102e00 0x7fd7e8115aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:32:45.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:45.344+0000 7fd7f0600700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd7e81037a0 0x7fd7e8115fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:32:45.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:45.344+0000 7fd7f0600700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7e8116630 con 0x7fd7e8102e00 2024-03-29T08:32:45.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:45.344+0000 7fd7f0600700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7e810fc10 con 0x7fd7e81037a0 2024-03-29T08:32:45.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:45.344+0000 7fd7edc00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd7e8102e00 0x7fd7e8115aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:32:45.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:45.344+0000 7fd7edc00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd7e8102e00 0x7fd7e8115aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53996/0 (socket says 172.21.15.37:53996) 2024-03-29T08:32:45.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:45.344+0000 7fd7edc00700 1 -- 172.21.15.37:0/2284528648 learned_addr learned my addr 172.21.15.37:0/2284528648 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:32:45.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:45.344+0000 7fd7ed200700 1 --2- 172.21.15.37:0/2284528648 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd7e81037a0 0x7fd7e8115fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:32:45.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:45.344+0000 7fd7edc00700 1 -- 172.21.15.37:0/2284528648 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd7e81037a0 msgr2=0x7fd7e8115fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:32:45.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:45.344+0000 7fd7edc00700 1 --2- 172.21.15.37:0/2284528648 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd7e81037a0 0x7fd7e8115fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:32:45.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:45.344+0000 7fd7edc00700 1 -- 172.21.15.37:0/2284528648 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7d4005040 con 0x7fd7e8102e00 2024-03-29T08:32:45.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:45.345+0000 7fd7edc00700 1 --2- 172.21.15.37:0/2284528648 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd7e8102e00 0x7fd7e8115aa0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fd7dc002a50 tx=0x7fd7dc0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:32:45.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:45.345+0000 7fd7e5600700 1 -- 172.21.15.37:0/2284528648 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd7dc007cc0 con 0x7fd7e8102e00 2024-03-29T08:32:45.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:45.345+0000 7fd7e5600700 1 -- 172.21.15.37:0/2284528648 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd7dc00f040 con 0x7fd7e8102e00 2024-03-29T08:32:45.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:45.345+0000 7fd7f0600700 1 -- 172.21.15.37:0/2284528648 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7e810fef0 con 0x7fd7e8102e00 2024-03-29T08:32:45.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:45.345+0000 7fd7e5600700 1 -- 172.21.15.37:0/2284528648 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd7dc009540 con 0x7fd7e8102e00 2024-03-29T08:32:45.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:45.345+0000 7fd7f0600700 1 -- 172.21.15.37:0/2284528648 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7e81104b0 con 0x7fd7e8102e00 2024-03-29T08:32:45.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:45.346+0000 7fd7f0600700 1 -- 172.21.15.37:0/2284528648 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7e80fe550 con 0x7fd7e8102e00 2024-03-29T08:32:45.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:45.347+0000 7fd7e5600700 1 -- 172.21.15.37:0/2284528648 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fd7dc0096a0 con 0x7fd7e8102e00 2024-03-29T08:32:45.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:45.348+0000 7fd7e5600700 1 --2- 172.21.15.37:0/2284528648 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fd7d006cc20 0x7fd7d006f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:32:45.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:45.348+0000 7fd7e5600700 1 -- 172.21.15.37:0/2284528648 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fd7dc08bde0 con 0x7fd7e8102e00 2024-03-29T08:32:45.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:45.348+0000 7fd7ed200700 1 --2- 172.21.15.37:0/2284528648 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fd7d006cc20 0x7fd7d006f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:32:45.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:45.349+0000 7fd7ed200700 1 --2- 172.21.15.37:0/2284528648 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fd7d006cc20 0x7fd7d006f0d0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fd7e8111340 tx=0x7fd7d4009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:32:45.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:45.352+0000 7fd7e5600700 1 -- 172.21.15.37:0/2284528648 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fd7dc059830 con 0x7fd7e8102e00 2024-03-29T08:32:45.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:45.648+0000 7fd7f0600700 1 -- 172.21.15.37:0/2284528648 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fd7e81110f0 con 0x7fd7e8102e00 2024-03-29T08:32:45.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:45.648+0000 7fd7e5600700 1 -- 172.21.15.37:0/2284528648 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7fd7dc0593c0 con 0x7fd7e8102e00 2024-03-29T08:32:45.648 INFO:teuthology.orchestra.run.smithi037.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-03-29T08:32:45.649 INFO:teuthology.orchestra.run.smithi037.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-03-29T08:32:45.649 INFO:teuthology.orchestra.run.smithi037.stdout: fs cephfs is degraded 2024-03-29T08:32:45.649 INFO:teuthology.orchestra.run.smithi037.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-03-29T08:32:45.649 INFO:teuthology.orchestra.run.smithi037.stdout: fs cephfs is offline because no MDS is active for it. 2024-03-29T08:32:45.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:45.651+0000 7fd7f0600700 1 -- 172.21.15.37:0/2284528648 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fd7d006cc20 msgr2=0x7fd7d006f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:32:45.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:45.651+0000 7fd7f0600700 1 --2- 172.21.15.37:0/2284528648 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fd7d006cc20 0x7fd7d006f0d0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fd7e8111340 tx=0x7fd7d4009e30 comp rx=0 tx=0).stop 2024-03-29T08:32:45.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:45.651+0000 7fd7f0600700 1 -- 172.21.15.37:0/2284528648 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd7e8102e00 msgr2=0x7fd7e8115aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:32:45.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:45.651+0000 7fd7f0600700 1 --2- 172.21.15.37:0/2284528648 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd7e8102e00 0x7fd7e8115aa0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fd7dc002a50 tx=0x7fd7dc0046b0 comp rx=0 tx=0).stop 2024-03-29T08:32:45.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:45.651+0000 7fd7f0600700 1 -- 172.21.15.37:0/2284528648 shutdown_connections 2024-03-29T08:32:45.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:45.651+0000 7fd7f0600700 1 --2- 172.21.15.37:0/2284528648 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fd7d006cc20 0x7fd7d006f0d0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:32:45.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:45.651+0000 7fd7f0600700 1 --2- 172.21.15.37:0/2284528648 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd7e8102e00 0x7fd7e8115aa0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:32:45.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:45.651+0000 7fd7f0600700 1 --2- 172.21.15.37:0/2284528648 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd7e81037a0 0x7fd7e8115fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:32:45.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:45.651+0000 7fd7f0600700 1 -- 172.21.15.37:0/2284528648 >> 172.21.15.37:0/2284528648 conn(0x7fd7e80f89b0 msgr2=0x7fd7e80706b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:32:45.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:45.651+0000 7fd7f0600700 1 -- 172.21.15.37:0/2284528648 shutdown_connections 2024-03-29T08:32:45.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:32:45.651+0000 7fd7f0600700 1 -- 172.21.15.37:0/2284528648 wait complete. 2024-03-29T08:32:45.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:32:45 smithi037 ceph-mon[142325]: pgmap v655: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:32:45.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:32:45 smithi037 ceph-mon[142325]: from='client.35212 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:32:45.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:32:45 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/1685227226' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T08:32:46.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:32:45 smithi148 ceph-mon[106692]: pgmap v655: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:32:46.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:32:45 smithi148 ceph-mon[106692]: from='client.35212 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:32:46.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:32:45 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/1685227226' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T08:32:46.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:32:46 smithi037 ceph-mon[142325]: from='client.35224 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:32:46.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:32:46 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/2284528648' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T08:32:47.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:32:46 smithi148 ceph-mon[106692]: from='client.35224 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:32:47.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:32:46 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/2284528648' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T08:32:47.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:32:47 smithi037 ceph-mon[142325]: pgmap v656: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:32:48.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:32:47 smithi148 ceph-mon[106692]: pgmap v656: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:32:49.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:32:49 smithi037 ceph-mon[142325]: pgmap v657: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:32:50.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:32:49 smithi148 ceph-mon[106692]: pgmap v657: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:32:51.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:32:51 smithi037 ceph-mon[142325]: pgmap v658: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:32:52.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:32:51 smithi148 ceph-mon[106692]: pgmap v658: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:32:53.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:32:53 smithi037 ceph-mon[142325]: pgmap v659: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:32:54.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:32:53 smithi148 ceph-mon[106692]: pgmap v659: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:32:55.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:32:55 smithi037 ceph-mon[142325]: pgmap v660: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:32:55.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:32:55 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:32:55.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:32:55 smithi148 ceph-mon[106692]: pgmap v660: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:32:55.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:32:55 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:32:57.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:32:57 smithi037 ceph-mon[142325]: pgmap v661: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:32:57.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:32:57 smithi148 ceph-mon[106692]: pgmap v661: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:32:59.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:32:59 smithi037 ceph-mon[142325]: pgmap v662: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:32:59.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:32:59 smithi148 ceph-mon[106692]: pgmap v662: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:33:01.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:33:01 smithi037 ceph-mon[142325]: pgmap v663: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:33:01.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:33:01 smithi148 ceph-mon[106692]: pgmap v663: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:33:03.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:33:03 smithi037 ceph-mon[142325]: pgmap v664: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:33:03.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:33:03 smithi148 ceph-mon[106692]: pgmap v664: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:33:05.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:33:05 smithi037 ceph-mon[142325]: pgmap v665: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:33:05.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:33:05 smithi148 ceph-mon[106692]: pgmap v665: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:33:07.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:33:07 smithi037 ceph-mon[142325]: pgmap v666: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:33:07.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:33:07 smithi148 ceph-mon[106692]: pgmap v666: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:33:09.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:33:09 smithi037 ceph-mon[142325]: pgmap v667: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:33:09.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:33:09 smithi148 ceph-mon[106692]: pgmap v667: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:33:10.678 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:33:10 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:33:10.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:33:10 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:33:11.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:33:11 smithi037 ceph-mon[142325]: pgmap v668: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:33:11.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:33:11 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T08:33:11.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:33:11 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T08:33:11.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:33:11 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:33:11.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:33:11 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T08:33:11.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:33:11 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:33:11.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:33:11 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:33:11.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:33:11 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:33:11.678 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:33:11 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:33:11.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:33:11 smithi148 ceph-mon[106692]: pgmap v668: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:33:11.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:33:11 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T08:33:11.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:33:11 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T08:33:11.807 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:33:11 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:33:11.807 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:33:11 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T08:33:11.807 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:33:11 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:33:11.807 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:33:11 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:33:11.807 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:33:11 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:33:11.807 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:33:11 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:33:12.666 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:33:12 smithi148 ceph-mon[106692]: pgmap v669: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:33:12.666 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:33:12 smithi148 ceph-mon[106692]: Health check failed: failed to probe daemons or devices (CEPHADM_REFRESH_FAILED) 2024-03-29T08:33:12.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:33:12 smithi037 ceph-mon[142325]: pgmap v669: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:33:12.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:33:12 smithi037 ceph-mon[142325]: Health check failed: failed to probe daemons or devices (CEPHADM_REFRESH_FAILED) 2024-03-29T08:33:14.428 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:33:14 smithi037 ceph-mon[142325]: pgmap v670: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:33:14.428 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:33:14 smithi037 ceph-mon[142325]: Upgrade: Updating mds.cephfs.smithi148.qukpjh 2024-03-29T08:33:14.428 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:33:14 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:33:14.429 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:33:14 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi148.qukpjh", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-29T08:33:14.429 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:33:14 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T08:33:14.429 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:33:14 smithi037 ceph-mon[142325]: Deploying daemon mds.cephfs.smithi148.qukpjh on smithi148 2024-03-29T08:33:14.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:33:14 smithi148 ceph-mon[106692]: pgmap v670: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:33:14.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:33:14 smithi148 ceph-mon[106692]: Upgrade: Updating mds.cephfs.smithi148.qukpjh 2024-03-29T08:33:14.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:33:14 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:33:14.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:33:14 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi148.qukpjh", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-29T08:33:14.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:33:14 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T08:33:14.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:33:14 smithi148 ceph-mon[106692]: Deploying daemon mds.cephfs.smithi148.qukpjh on smithi148 2024-03-29T08:33:15.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:15.806+0000 7f4728800700 1 -- 172.21.15.37:0/2349154035 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4720102e00 msgr2=0x7f47201031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:33:15.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:15.806+0000 7f4728800700 1 --2- 172.21.15.37:0/2349154035 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4720102e00 0x7f47201031d0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f4714004940 tx=0x7f4714009e30 comp rx=0 tx=0).stop 2024-03-29T08:33:15.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:15.806+0000 7f4728800700 1 -- 172.21.15.37:0/2349154035 shutdown_connections 2024-03-29T08:33:15.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:15.806+0000 7f4728800700 1 --2- 172.21.15.37:0/2349154035 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f47201037a0 0x7f47200fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:33:15.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:15.806+0000 7f4728800700 1 --2- 172.21.15.37:0/2349154035 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4720102e00 0x7f47201031d0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:33:15.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:15.806+0000 7f4728800700 1 -- 172.21.15.37:0/2349154035 >> 172.21.15.37:0/2349154035 conn(0x7f47200f89b0 msgr2=0x7f47200fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:33:15.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:15.806+0000 7f4728800700 1 -- 172.21.15.37:0/2349154035 shutdown_connections 2024-03-29T08:33:15.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:15.807+0000 7f4728800700 1 -- 172.21.15.37:0/2349154035 wait complete. 2024-03-29T08:33:15.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:15.807+0000 7f4728800700 1 Processor -- start 2024-03-29T08:33:15.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:15.807+0000 7f4728800700 1 -- start start 2024-03-29T08:33:15.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:15.808+0000 7f4728800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4720102e00 0x7f4720115cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:33:15.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:15.808+0000 7f4728800700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f47201037a0 0x7f4720116230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:33:15.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:15.808+0000 7f4728800700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4720116880 con 0x7f4720102e00 2024-03-29T08:33:15.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:15.808+0000 7f4728800700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f472010fc50 con 0x7f47201037a0 2024-03-29T08:33:15.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:15.808+0000 7f4725e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4720102e00 0x7f4720115cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:33:15.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:15.808+0000 7f4725e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4720102e00 0x7f4720115cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40956/0 (socket says 172.21.15.37:40956) 2024-03-29T08:33:15.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:15.808+0000 7f4725e00700 1 -- 172.21.15.37:0/3638236356 learned_addr learned my addr 172.21.15.37:0/3638236356 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:33:15.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:15.808+0000 7f4725400700 1 --2- 172.21.15.37:0/3638236356 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f47201037a0 0x7f4720116230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:33:15.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:15.809+0000 7f4725e00700 1 -- 172.21.15.37:0/3638236356 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f47201037a0 msgr2=0x7f4720116230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:33:15.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:15.809+0000 7f4725e00700 1 --2- 172.21.15.37:0/3638236356 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f47201037a0 0x7f4720116230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:33:15.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:15.809+0000 7f4725e00700 1 -- 172.21.15.37:0/3638236356 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4714005040 con 0x7f4720102e00 2024-03-29T08:33:15.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:15.809+0000 7f4725e00700 1 --2- 172.21.15.37:0/3638236356 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4720102e00 0x7f4720115cf0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f4714009360 tx=0x7f4714009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:33:15.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:15.809+0000 7f471a000700 1 -- 172.21.15.37:0/3638236356 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f471400e040 con 0x7f4720102e00 2024-03-29T08:33:15.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:15.809+0000 7f471a000700 1 -- 172.21.15.37:0/3638236356 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f4714013420 con 0x7f4720102e00 2024-03-29T08:33:15.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:15.809+0000 7f4728800700 1 -- 172.21.15.37:0/3638236356 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f472010fed0 con 0x7f4720102e00 2024-03-29T08:33:15.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:15.809+0000 7f471a000700 1 -- 172.21.15.37:0/3638236356 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f471401c5d0 con 0x7f4720102e00 2024-03-29T08:33:15.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:15.809+0000 7f4728800700 1 -- 172.21.15.37:0/3638236356 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4720110490 con 0x7f4720102e00 2024-03-29T08:33:15.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:15.811+0000 7f4728800700 1 -- 172.21.15.37:0/3638236356 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47200fe550 con 0x7f4720102e00 2024-03-29T08:33:15.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:15.811+0000 7f471a000700 1 -- 172.21.15.37:0/3638236356 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f4714013590 con 0x7f4720102e00 2024-03-29T08:33:15.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:15.812+0000 7f471a000700 1 --2- 172.21.15.37:0/3638236356 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f470806cc30 0x7f470806f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:33:15.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:15.812+0000 7f471a000700 1 -- 172.21.15.37:0/3638236356 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f471408d250 con 0x7f4720102e00 2024-03-29T08:33:15.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:15.813+0000 7f4725400700 1 --2- 172.21.15.37:0/3638236356 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f470806cc30 0x7f470806f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:33:15.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:15.813+0000 7f4725400700 1 --2- 172.21.15.37:0/3638236356 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f470806cc30 0x7f470806f0e0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f4720111590 tx=0x7f471000b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:33:15.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:15.817+0000 7f471a000700 1 -- 172.21.15.37:0/3638236356 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f471405ac20 con 0x7f4720102e00 2024-03-29T08:33:16.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:16.052+0000 7f4728800700 1 -- 172.21.15.37:0/3638236356 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f472004bed0 con 0x7f470806cc30 2024-03-29T08:33:16.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:16.053+0000 7f471a000700 1 -- 172.21.15.37:0/3638236356 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f472004bed0 con 0x7f470806cc30 2024-03-29T08:33:16.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:16.056+0000 7f4728800700 1 -- 172.21.15.37:0/3638236356 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f470806cc30 msgr2=0x7f470806f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:33:16.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:16.056+0000 7f4728800700 1 --2- 172.21.15.37:0/3638236356 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f470806cc30 0x7f470806f0e0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f4720111590 tx=0x7f471000b040 comp rx=0 tx=0).stop 2024-03-29T08:33:16.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:16.056+0000 7f4728800700 1 -- 172.21.15.37:0/3638236356 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4720102e00 msgr2=0x7f4720115cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:33:16.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:16.056+0000 7f4728800700 1 --2- 172.21.15.37:0/3638236356 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4720102e00 0x7f4720115cf0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f4714009360 tx=0x7f4714009e30 comp rx=0 tx=0).stop 2024-03-29T08:33:16.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:16.056+0000 7f4728800700 1 -- 172.21.15.37:0/3638236356 shutdown_connections 2024-03-29T08:33:16.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:16.056+0000 7f4728800700 1 --2- 172.21.15.37:0/3638236356 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f470806cc30 0x7f470806f0e0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:33:16.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:16.056+0000 7f4728800700 1 --2- 172.21.15.37:0/3638236356 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4720102e00 0x7f4720115cf0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:33:16.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:16.056+0000 7f4728800700 1 --2- 172.21.15.37:0/3638236356 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f47201037a0 0x7f4720116230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:33:16.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:16.056+0000 7f4728800700 1 -- 172.21.15.37:0/3638236356 >> 172.21.15.37:0/3638236356 conn(0x7f47200f89b0 msgr2=0x7f47200706d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:33:16.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:16.056+0000 7f4728800700 1 -- 172.21.15.37:0/3638236356 shutdown_connections 2024-03-29T08:33:16.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:16.056+0000 7f4728800700 1 -- 172.21.15.37:0/3638236356 wait complete. 2024-03-29T08:33:16.071 INFO:teuthology.orchestra.run.smithi037.stdout:true 2024-03-29T08:33:16.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:16.195+0000 7fd425a00700 1 -- 172.21.15.37:0/3189804283 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd420071f90 msgr2=0x7fd420070360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:33:16.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:16.195+0000 7fd425a00700 1 --2- 172.21.15.37:0/3189804283 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd420071f90 0x7fd420070360 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fd410004970 tx=0x7fd410009e30 comp rx=0 tx=0).stop 2024-03-29T08:33:16.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:16.195+0000 7fd425a00700 1 -- 172.21.15.37:0/3189804283 shutdown_connections 2024-03-29T08:33:16.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:16.195+0000 7fd425a00700 1 --2- 172.21.15.37:0/3189804283 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd4200708a0 0x7fd420070d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:33:16.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:16.195+0000 7fd425a00700 1 --2- 172.21.15.37:0/3189804283 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd420071f90 0x7fd420070360 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:33:16.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:16.195+0000 7fd425a00700 1 -- 172.21.15.37:0/3189804283 >> 172.21.15.37:0/3189804283 conn(0x7fd4200f89b0 msgr2=0x7fd4200fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:33:16.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:16.196+0000 7fd425a00700 1 -- 172.21.15.37:0/3189804283 shutdown_connections 2024-03-29T08:33:16.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:16.196+0000 7fd425a00700 1 -- 172.21.15.37:0/3189804283 wait complete. 2024-03-29T08:33:16.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:16.196+0000 7fd425a00700 1 Processor -- start 2024-03-29T08:33:16.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:16.197+0000 7fd425a00700 1 -- start start 2024-03-29T08:33:16.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:16.197+0000 7fd425a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd4200708a0 0x7fd42011a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:33:16.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:16.197+0000 7fd425a00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd420071f90 0x7fd42011a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:33:16.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:16.197+0000 7fd425a00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd42011ac50 con 0x7fd4200708a0 2024-03-29T08:33:16.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:16.197+0000 7fd425a00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd420114020 con 0x7fd420071f90 2024-03-29T08:33:16.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:16.197+0000 7fd41ea00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd4200708a0 0x7fd42011a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:33:16.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:16.197+0000 7fd41ea00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd4200708a0 0x7fd42011a0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40976/0 (socket says 172.21.15.37:40976) 2024-03-29T08:33:16.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:16.197+0000 7fd41ea00700 1 -- 172.21.15.37:0/3753600969 learned_addr learned my addr 172.21.15.37:0/3753600969 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:33:16.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:16.197+0000 7fd41e000700 1 --2- 172.21.15.37:0/3753600969 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd420071f90 0x7fd42011a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:33:16.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:16.198+0000 7fd41ea00700 1 -- 172.21.15.37:0/3753600969 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd420071f90 msgr2=0x7fd42011a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:33:16.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:16.198+0000 7fd41ea00700 1 --2- 172.21.15.37:0/3753600969 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd420071f90 0x7fd42011a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:33:16.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:16.198+0000 7fd41ea00700 1 -- 172.21.15.37:0/3753600969 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd410005040 con 0x7fd4200708a0 2024-03-29T08:33:16.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:16.198+0000 7fd41ea00700 1 --2- 172.21.15.37:0/3753600969 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd4200708a0 0x7fd42011a0c0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fd410004f50 tx=0x7fd410009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:33:16.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:16.198+0000 7fd416a00700 1 -- 172.21.15.37:0/3753600969 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd41000e040 con 0x7fd4200708a0 2024-03-29T08:33:16.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:16.198+0000 7fd425a00700 1 -- 172.21.15.37:0/3753600969 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4201142a0 con 0x7fd4200708a0 2024-03-29T08:33:16.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:16.198+0000 7fd416a00700 1 -- 172.21.15.37:0/3753600969 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd410013420 con 0x7fd4200708a0 2024-03-29T08:33:16.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:16.198+0000 7fd416a00700 1 -- 172.21.15.37:0/3753600969 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd41001c630 con 0x7fd4200708a0 2024-03-29T08:33:16.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:16.198+0000 7fd425a00700 1 -- 172.21.15.37:0/3753600969 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4201147e0 con 0x7fd4200708a0 2024-03-29T08:33:16.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:16.200+0000 7fd425a00700 1 -- 172.21.15.37:0/3753600969 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd420071aa0 con 0x7fd4200708a0 2024-03-29T08:33:16.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:16.200+0000 7fd416a00700 1 -- 172.21.15.37:0/3753600969 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fd410013590 con 0x7fd4200708a0 2024-03-29T08:33:16.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:16.201+0000 7fd416a00700 1 --2- 172.21.15.37:0/3753600969 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fd40406cc30 0x7fd40406f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:33:16.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:16.201+0000 7fd416a00700 1 -- 172.21.15.37:0/3753600969 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fd41008d1a0 con 0x7fd4200708a0 2024-03-29T08:33:16.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:16.202+0000 7fd41e000700 1 --2- 172.21.15.37:0/3753600969 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fd40406cc30 0x7fd40406f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:33:16.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:16.202+0000 7fd41e000700 1 --2- 172.21.15.37:0/3753600969 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fd40406cc30 0x7fd40406f0e0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fd420115960 tx=0x7fd40c00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:33:16.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:16.206+0000 7fd416a00700 1 -- 172.21.15.37:0/3753600969 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fd41005ade0 con 0x7fd4200708a0 2024-03-29T08:33:16.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:16.442+0000 7fd425a00700 1 -- 172.21.15.37:0/3753600969 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd4201104a0 con 0x7fd40406cc30 2024-03-29T08:33:16.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:16.443+0000 7fd416a00700 1 -- 172.21.15.37:0/3753600969 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fd40c00f040 con 0x7fd40406cc30 2024-03-29T08:33:16.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:16.445+0000 7fd425a00700 1 -- 172.21.15.37:0/3753600969 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fd40406cc30 msgr2=0x7fd40406f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:33:16.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:16.445+0000 7fd425a00700 1 --2- 172.21.15.37:0/3753600969 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fd40406cc30 0x7fd40406f0e0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fd420115960 tx=0x7fd40c00b040 comp rx=0 tx=0).stop 2024-03-29T08:33:16.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:16.445+0000 7fd425a00700 1 -- 172.21.15.37:0/3753600969 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd4200708a0 msgr2=0x7fd42011a0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:33:16.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:16.445+0000 7fd425a00700 1 --2- 172.21.15.37:0/3753600969 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd4200708a0 0x7fd42011a0c0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fd410004f50 tx=0x7fd410009e30 comp rx=0 tx=0).stop 2024-03-29T08:33:16.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:16.445+0000 7fd425a00700 1 -- 172.21.15.37:0/3753600969 shutdown_connections 2024-03-29T08:33:16.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:16.445+0000 7fd425a00700 1 --2- 172.21.15.37:0/3753600969 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fd40406cc30 0x7fd40406f0e0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:33:16.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:16.445+0000 7fd425a00700 1 --2- 172.21.15.37:0/3753600969 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd4200708a0 0x7fd42011a0c0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:33:16.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:16.445+0000 7fd425a00700 1 --2- 172.21.15.37:0/3753600969 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd420071f90 0x7fd42011a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:33:16.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:16.445+0000 7fd425a00700 1 -- 172.21.15.37:0/3753600969 >> 172.21.15.37:0/3753600969 conn(0x7fd4200f89b0 msgr2=0x7fd4200f9ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:33:16.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:16.446+0000 7fd425a00700 1 -- 172.21.15.37:0/3753600969 shutdown_connections 2024-03-29T08:33:16.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:16.446+0000 7fd425a00700 1 -- 172.21.15.37:0/3753600969 wait complete. 2024-03-29T08:33:16.557 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:33:16 smithi148 ceph-mon[106692]: pgmap v671: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:33:16.557 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:33:16 smithi148 ceph-mon[106692]: osdmap e86: 6 total, 6 up, 6 in 2024-03-29T08:33:16.557 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:33:16 smithi148 ceph-mon[106692]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-03-29T08:33:16.557 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:33:16 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi148.qukpjh"}]: dispatch 2024-03-29T08:33:16.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:16.582+0000 7f02bc400700 1 -- 172.21.15.37:0/1802162930 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f02b40fde90 msgr2=0x7f02b40fe260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:33:16.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:16.582+0000 7f02bc400700 1 --2- 172.21.15.37:0/1802162930 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f02b40fde90 0x7f02b40fe260 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f02a8004940 tx=0x7f02a8009e30 comp rx=0 tx=0).stop 2024-03-29T08:33:16.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:16.582+0000 7f02bc400700 1 -- 172.21.15.37:0/1802162930 shutdown_connections 2024-03-29T08:33:16.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:16.583+0000 7f02bc400700 1 --2- 172.21.15.37:0/1802162930 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f02b40fe830 0x7f02b4106f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:33:16.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:16.583+0000 7f02bc400700 1 --2- 172.21.15.37:0/1802162930 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f02b40fde90 0x7f02b40fe260 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:33:16.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:16.583+0000 7f02bc400700 1 -- 172.21.15.37:0/1802162930 >> 172.21.15.37:0/1802162930 conn(0x7f02b4072bc0 msgr2=0x7f02b4072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:33:16.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:16.583+0000 7f02bc400700 1 -- 172.21.15.37:0/1802162930 shutdown_connections 2024-03-29T08:33:16.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:16.583+0000 7f02bc400700 1 -- 172.21.15.37:0/1802162930 wait complete. 2024-03-29T08:33:16.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:16.583+0000 7f02bc400700 1 Processor -- start 2024-03-29T08:33:16.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:16.584+0000 7f02bc400700 1 -- start start 2024-03-29T08:33:16.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:16.584+0000 7f02bc400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f02b40fde90 0x7f02b4115d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:33:16.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:16.584+0000 7f02bc400700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f02b40fe830 0x7f02b4116240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:33:16.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:16.584+0000 7f02bc400700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02b4116890 con 0x7f02b40fde90 2024-03-29T08:33:16.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:16.584+0000 7f02bc400700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02b410fc60 con 0x7f02b40fe830 2024-03-29T08:33:16.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:16.584+0000 7f02b9a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f02b40fde90 0x7f02b4115d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:33:16.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:16.584+0000 7f02b9a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f02b40fde90 0x7f02b4115d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40988/0 (socket says 172.21.15.37:40988) 2024-03-29T08:33:16.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:16.584+0000 7f02b9a00700 1 -- 172.21.15.37:0/2322198320 learned_addr learned my addr 172.21.15.37:0/2322198320 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:33:16.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:16.584+0000 7f02b9000700 1 --2- 172.21.15.37:0/2322198320 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f02b40fe830 0x7f02b4116240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:33:16.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:16.585+0000 7f02b9a00700 1 -- 172.21.15.37:0/2322198320 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f02b40fe830 msgr2=0x7f02b4116240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:33:16.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:16.585+0000 7f02b9a00700 1 --2- 172.21.15.37:0/2322198320 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f02b40fe830 0x7f02b4116240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:33:16.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:16.585+0000 7f02b9a00700 1 -- 172.21.15.37:0/2322198320 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02a8005040 con 0x7f02b40fde90 2024-03-29T08:33:16.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:16.585+0000 7f02b9a00700 1 --2- 172.21.15.37:0/2322198320 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f02b40fde90 0x7f02b4115d00 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f02a8000c00 tx=0x7f02a8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:33:16.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:16.585+0000 7f02b1600700 1 -- 172.21.15.37:0/2322198320 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f02a800e040 con 0x7f02b40fde90 2024-03-29T08:33:16.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:16.585+0000 7f02b1600700 1 -- 172.21.15.37:0/2322198320 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f02a800bed0 con 0x7f02b40fde90 2024-03-29T08:33:16.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:16.585+0000 7f02bc400700 1 -- 172.21.15.37:0/2322198320 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02b410fee0 con 0x7f02b40fde90 2024-03-29T08:33:16.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:16.585+0000 7f02b1600700 1 -- 172.21.15.37:0/2322198320 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f02a801c3f0 con 0x7f02b40fde90 2024-03-29T08:33:16.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:16.585+0000 7f02bc400700 1 -- 172.21.15.37:0/2322198320 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02b4110420 con 0x7f02b40fde90 2024-03-29T08:33:16.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:16.587+0000 7f02bc400700 1 -- 172.21.15.37:0/2322198320 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02b4106a20 con 0x7f02b40fde90 2024-03-29T08:33:16.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:16.587+0000 7f02b1600700 1 -- 172.21.15.37:0/2322198320 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f02a8013420 con 0x7f02b40fde90 2024-03-29T08:33:16.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:16.589+0000 7f02b1600700 1 --2- 172.21.15.37:0/2322198320 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f029c06cc30 0x7f029c06f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:33:16.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:16.589+0000 7f02b1600700 1 -- 172.21.15.37:0/2322198320 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f02a808d500 con 0x7f02b40fde90 2024-03-29T08:33:16.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:16.589+0000 7f02b9000700 1 --2- 172.21.15.37:0/2322198320 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f029c06cc30 0x7f029c06f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:33:16.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:16.590+0000 7f02b9000700 1 --2- 172.21.15.37:0/2322198320 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f029c06cc30 0x7f029c06f0e0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f02b41115a0 tx=0x7f02a000b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:33:16.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:16.593+0000 7f02b1600700 1 -- 172.21.15.37:0/2322198320 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f02a805af50 con 0x7f02b40fde90 2024-03-29T08:33:16.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:33:16 smithi037 ceph-mon[142325]: pgmap v671: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:33:16.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:33:16 smithi037 ceph-mon[142325]: osdmap e86: 6 total, 6 up, 6 in 2024-03-29T08:33:16.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:33:16 smithi037 ceph-mon[142325]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-03-29T08:33:16.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:33:16 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi148.qukpjh"}]: dispatch 2024-03-29T08:33:16.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:16.816+0000 7f02bc400700 1 -- 172.21.15.37:0/2322198320 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f02b4109e30 con 0x7f029c06cc30 2024-03-29T08:33:16.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:16.826+0000 7f02b1600700 1 -- 172.21.15.37:0/2322198320 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f02b4109e30 con 0x7f029c06cc30 2024-03-29T08:33:16.826 INFO:teuthology.orchestra.run.smithi037.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-29T08:33:16.826 INFO:teuthology.orchestra.run.smithi037.stdout:alertmanager.smithi037 smithi037 *:9093,9094 running (23m) 15m ago 36m 14.9M - 0.25.0 c8568f914cd2 2f9f9746ed0f 2024-03-29T08:33:16.826 INFO:teuthology.orchestra.run.smithi037.stdout:crash.smithi037 smithi037 running (20m) 15m ago 36m 6647k - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 178d416a3186 2024-03-29T08:33:16.826 INFO:teuthology.orchestra.run.smithi037.stdout:crash.smithi148 smithi148 running (19m) 15m ago 35m 6656k - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 23666989d58f 2024-03-29T08:33:16.826 INFO:teuthology.orchestra.run.smithi037.stdout:grafana.smithi037 smithi037 *:3000 running (22m) 15m ago 35m 71.9M - 9.4.7 954c08fa6188 6b976fe49952 2024-03-29T08:33:16.826 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi037.iifsmh smithi037 running (15m) 15m ago 29m 10.8M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 2d97f562cc5b 2024-03-29T08:33:16.826 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi037.mmwgxe smithi037 running (15m) 15m ago 29m 10.9M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff ec37037f46e8 2024-03-29T08:33:16.827 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi148.qukpjh smithi148 running (29m) 15m ago 29m 15.3M - 17.2.7-789-g4213e17c cf22eb8683be 8e469782a273 2024-03-29T08:33:16.827 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi148.sqteih smithi148 running (15m) 15m ago 29m 10.7M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 44c15c77037a 2024-03-29T08:33:16.827 INFO:teuthology.orchestra.run.smithi037.stdout:mgr.smithi037.wjikii smithi037 *:8443,9283,8765 running (25m) 15m ago 37m 506M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff c5a1417b9417 2024-03-29T08:33:16.827 INFO:teuthology.orchestra.run.smithi037.stdout:mgr.smithi148.onhgvm smithi148 *:8443,9283,8765 running (24m) 15m ago 34m 450M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff fcfdbce062bd 2024-03-29T08:33:16.827 INFO:teuthology.orchestra.run.smithi037.stdout:mon.smithi037 smithi037 running (21m) 15m ago 37m 54.8M 2048M 18.2.2-1015-gd0e392d9 1cc2f45c17ff eea2060783af 2024-03-29T08:33:16.827 INFO:teuthology.orchestra.run.smithi037.stdout:mon.smithi148 smithi148 running (20m) 15m ago 34m 43.7M 2048M 18.2.2-1015-gd0e392d9 1cc2f45c17ff ad4b2c5748be 2024-03-29T08:33:16.827 INFO:teuthology.orchestra.run.smithi037.stdout:node-exporter.smithi037 smithi037 *:9100 running (23m) 15m ago 36m 7625k - 1.5.0 0da6a335fe13 26b8ae9cb878 2024-03-29T08:33:16.827 INFO:teuthology.orchestra.run.smithi037.stdout:node-exporter.smithi148 smithi148 *:9100 running (23m) 15m ago 34m 8166k - 1.5.0 0da6a335fe13 98414e4e64ed 2024-03-29T08:33:16.827 INFO:teuthology.orchestra.run.smithi037.stdout:osd.0 smithi037 running (19m) 15m ago 33m 104M 1619M 18.2.2-1015-gd0e392d9 1cc2f45c17ff c0c584f6245c 2024-03-29T08:33:16.827 INFO:teuthology.orchestra.run.smithi037.stdout:osd.1 smithi037 running (18m) 15m ago 33m 100M 1619M 18.2.2-1015-gd0e392d9 1cc2f45c17ff a4a2ff4cc83d 2024-03-29T08:33:16.827 INFO:teuthology.orchestra.run.smithi037.stdout:osd.2 smithi037 running (18m) 15m ago 32m 82.3M 1619M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 8b1bc71aa79e 2024-03-29T08:33:16.827 INFO:teuthology.orchestra.run.smithi037.stdout:osd.3 smithi148 running (17m) 15m ago 32m 105M 2642M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 1c656ab8ff86 2024-03-29T08:33:16.827 INFO:teuthology.orchestra.run.smithi037.stdout:osd.4 smithi148 running (16m) 15m ago 31m 97.3M 2642M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 966a9de11d22 2024-03-29T08:33:16.827 INFO:teuthology.orchestra.run.smithi037.stdout:osd.5 smithi148 running (16m) 15m ago 31m 85.8M 2642M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 474c4f9212ce 2024-03-29T08:33:16.827 INFO:teuthology.orchestra.run.smithi037.stdout:prometheus.smithi037 smithi037 *:9095 running (23m) 15m ago 35m 54.4M - 2.43.0 a07b618ecd1d 30c0223ec589 2024-03-29T08:33:16.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:16.828+0000 7f02bc400700 1 -- 172.21.15.37:0/2322198320 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f029c06cc30 msgr2=0x7f029c06f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:33:16.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:16.828+0000 7f02bc400700 1 --2- 172.21.15.37:0/2322198320 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f029c06cc30 0x7f029c06f0e0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f02b41115a0 tx=0x7f02a000b040 comp rx=0 tx=0).stop 2024-03-29T08:33:16.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:16.828+0000 7f02bc400700 1 -- 172.21.15.37:0/2322198320 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f02b40fde90 msgr2=0x7f02b4115d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:33:16.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:16.828+0000 7f02bc400700 1 --2- 172.21.15.37:0/2322198320 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f02b40fde90 0x7f02b4115d00 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f02a8000c00 tx=0x7f02a8009e30 comp rx=0 tx=0).stop 2024-03-29T08:33:16.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:16.828+0000 7f02bc400700 1 -- 172.21.15.37:0/2322198320 shutdown_connections 2024-03-29T08:33:16.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:16.828+0000 7f02bc400700 1 --2- 172.21.15.37:0/2322198320 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f029c06cc30 0x7f029c06f0e0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:33:16.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:16.829+0000 7f02bc400700 1 --2- 172.21.15.37:0/2322198320 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f02b40fde90 0x7f02b4115d00 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:33:16.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:16.829+0000 7f02bc400700 1 --2- 172.21.15.37:0/2322198320 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f02b40fe830 0x7f02b4116240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:33:16.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:16.829+0000 7f02bc400700 1 -- 172.21.15.37:0/2322198320 >> 172.21.15.37:0/2322198320 conn(0x7f02b4072bc0 msgr2=0x7f02b40fd590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:33:16.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:16.829+0000 7f02bc400700 1 -- 172.21.15.37:0/2322198320 shutdown_connections 2024-03-29T08:33:16.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:16.829+0000 7f02bc400700 1 -- 172.21.15.37:0/2322198320 wait complete. 2024-03-29T08:33:16.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:16.965+0000 7fbc40400700 1 -- 172.21.15.37:0/395341571 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbc381037a0 msgr2=0x7fbc380fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:33:16.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:16.965+0000 7fbc40400700 1 --2- 172.21.15.37:0/395341571 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbc381037a0 0x7fbc380fce10 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fbc24004940 tx=0x7fbc24009e30 comp rx=0 tx=0).stop 2024-03-29T08:33:16.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:16.965+0000 7fbc40400700 1 -- 172.21.15.37:0/395341571 shutdown_connections 2024-03-29T08:33:16.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:16.965+0000 7fbc40400700 1 --2- 172.21.15.37:0/395341571 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbc381037a0 0x7fbc380fce10 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:33:16.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:16.965+0000 7fbc40400700 1 --2- 172.21.15.37:0/395341571 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbc38102e00 0x7fbc381031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:33:16.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:16.965+0000 7fbc40400700 1 -- 172.21.15.37:0/395341571 >> 172.21.15.37:0/395341571 conn(0x7fbc380f89b0 msgr2=0x7fbc380fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:33:16.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:16.965+0000 7fbc40400700 1 -- 172.21.15.37:0/395341571 shutdown_connections 2024-03-29T08:33:16.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:16.965+0000 7fbc40400700 1 -- 172.21.15.37:0/395341571 wait complete. 2024-03-29T08:33:16.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:16.966+0000 7fbc40400700 1 Processor -- start 2024-03-29T08:33:16.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:16.966+0000 7fbc40400700 1 -- start start 2024-03-29T08:33:16.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:16.966+0000 7fbc40400700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbc38102e00 0x7fbc38115d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:33:16.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:16.967+0000 7fbc40400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbc381037a0 0x7fbc38116240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:33:16.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:16.967+0000 7fbc40400700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc38116890 con 0x7fbc381037a0 2024-03-29T08:33:16.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:16.967+0000 7fbc40400700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc3810fc60 con 0x7fbc38102e00 2024-03-29T08:33:16.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:16.967+0000 7fbc3d000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbc381037a0 0x7fbc38116240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:33:16.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:16.967+0000 7fbc3d000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbc381037a0 0x7fbc38116240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41004/0 (socket says 172.21.15.37:41004) 2024-03-29T08:33:16.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:16.967+0000 7fbc3d000700 1 -- 172.21.15.37:0/665025048 learned_addr learned my addr 172.21.15.37:0/665025048 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:33:16.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:16.967+0000 7fbc3d000700 1 -- 172.21.15.37:0/665025048 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbc38102e00 msgr2=0x7fbc38115d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:33:16.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:16.967+0000 7fbc3d000700 1 --2- 172.21.15.37:0/665025048 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbc38102e00 0x7fbc38115d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:33:16.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:16.967+0000 7fbc3d000700 1 -- 172.21.15.37:0/665025048 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc24005040 con 0x7fbc381037a0 2024-03-29T08:33:16.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:16.967+0000 7fbc3d000700 1 --2- 172.21.15.37:0/665025048 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbc381037a0 0x7fbc38116240 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fbc24009390 tx=0x7fbc24009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:33:16.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:16.968+0000 7fbc35600700 1 -- 172.21.15.37:0/665025048 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbc2400e040 con 0x7fbc381037a0 2024-03-29T08:33:16.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:16.968+0000 7fbc35600700 1 -- 172.21.15.37:0/665025048 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fbc24013420 con 0x7fbc381037a0 2024-03-29T08:33:16.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:16.968+0000 7fbc35600700 1 -- 172.21.15.37:0/665025048 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbc2401c630 con 0x7fbc381037a0 2024-03-29T08:33:16.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:16.968+0000 7fbc40400700 1 -- 172.21.15.37:0/665025048 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc3810fee0 con 0x7fbc381037a0 2024-03-29T08:33:16.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:16.968+0000 7fbc40400700 1 -- 172.21.15.37:0/665025048 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc381104a0 con 0x7fbc381037a0 2024-03-29T08:33:16.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:16.969+0000 7fbc40400700 1 -- 172.21.15.37:0/665025048 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc380fe550 con 0x7fbc381037a0 2024-03-29T08:33:16.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:16.970+0000 7fbc35600700 1 -- 172.21.15.37:0/665025048 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fbc24013a00 con 0x7fbc381037a0 2024-03-29T08:33:16.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:16.971+0000 7fbc35600700 1 --2- 172.21.15.37:0/665025048 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fbc2006cc30 0x7fbc2006f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:33:16.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:16.971+0000 7fbc35600700 1 -- 172.21.15.37:0/665025048 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fbc2408d2c0 con 0x7fbc381037a0 2024-03-29T08:33:16.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:16.971+0000 7fbc3da00700 1 --2- 172.21.15.37:0/665025048 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fbc2006cc30 0x7fbc2006f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:33:16.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:16.972+0000 7fbc3da00700 1 --2- 172.21.15.37:0/665025048 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fbc2006cc30 0x7fbc2006f0e0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fbc2c004d00 tx=0x7fbc2c009ce0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:33:16.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:16.975+0000 7fbc35600700 1 -- 172.21.15.37:0/665025048 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fbc2405ad90 con 0x7fbc381037a0 2024-03-29T08:33:17.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:17.274+0000 7fbc40400700 1 -- 172.21.15.37:0/665025048 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fbc3804bed0 con 0x7fbc381037a0 2024-03-29T08:33:17.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:17.275+0000 7fbc35600700 1 -- 172.21.15.37:0/665025048 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7fbc2405a920 con 0x7fbc381037a0 2024-03-29T08:33:17.275 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-29T08:33:17.275 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": { 2024-03-29T08:33:17.275 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 2 2024-03-29T08:33:17.276 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:33:17.276 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": { 2024-03-29T08:33:17.276 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 2 2024-03-29T08:33:17.276 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:33:17.276 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": { 2024-03-29T08:33:17.276 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 6 2024-03-29T08:33:17.277 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:33:17.277 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": { 2024-03-29T08:33:17.277 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 3 2024-03-29T08:33:17.277 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:33:17.277 INFO:teuthology.orchestra.run.smithi037.stdout: "overall": { 2024-03-29T08:33:17.278 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 13 2024-03-29T08:33:17.278 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-29T08:33:17.278 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-29T08:33:17.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:17.277+0000 7fbc40400700 1 -- 172.21.15.37:0/665025048 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fbc2006cc30 msgr2=0x7fbc2006f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:33:17.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:17.277+0000 7fbc40400700 1 --2- 172.21.15.37:0/665025048 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fbc2006cc30 0x7fbc2006f0e0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fbc2c004d00 tx=0x7fbc2c009ce0 comp rx=0 tx=0).stop 2024-03-29T08:33:17.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:17.277+0000 7fbc40400700 1 -- 172.21.15.37:0/665025048 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbc381037a0 msgr2=0x7fbc38116240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:33:17.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:17.277+0000 7fbc40400700 1 --2- 172.21.15.37:0/665025048 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbc381037a0 0x7fbc38116240 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fbc24009390 tx=0x7fbc24009e30 comp rx=0 tx=0).stop 2024-03-29T08:33:17.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:17.277+0000 7fbc40400700 1 -- 172.21.15.37:0/665025048 shutdown_connections 2024-03-29T08:33:17.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:17.277+0000 7fbc40400700 1 --2- 172.21.15.37:0/665025048 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fbc2006cc30 0x7fbc2006f0e0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:33:17.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:17.277+0000 7fbc40400700 1 --2- 172.21.15.37:0/665025048 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbc38102e00 0x7fbc38115d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:33:17.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:17.277+0000 7fbc40400700 1 --2- 172.21.15.37:0/665025048 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbc381037a0 0x7fbc38116240 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:33:17.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:17.277+0000 7fbc40400700 1 -- 172.21.15.37:0/665025048 >> 172.21.15.37:0/665025048 conn(0x7fbc380f89b0 msgr2=0x7fbc380f9ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:33:17.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:17.277+0000 7fbc40400700 1 -- 172.21.15.37:0/665025048 shutdown_connections 2024-03-29T08:33:17.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:17.278+0000 7fbc40400700 1 -- 172.21.15.37:0/665025048 wait complete. 2024-03-29T08:33:17.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:17.414+0000 7fdb41200700 1 -- 172.21.15.37:0/336458843 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdb3c0fde90 msgr2=0x7fdb3c0fe260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:33:17.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:17.414+0000 7fdb41200700 1 --2- 172.21.15.37:0/336458843 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdb3c0fde90 0x7fdb3c0fe260 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fdb2c004940 tx=0x7fdb2c009e30 comp rx=0 tx=0).stop 2024-03-29T08:33:17.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:17.414+0000 7fdb41200700 1 -- 172.21.15.37:0/336458843 shutdown_connections 2024-03-29T08:33:17.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:17.414+0000 7fdb41200700 1 --2- 172.21.15.37:0/336458843 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdb3c0fe830 0x7fdb3c106f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:33:17.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:17.414+0000 7fdb41200700 1 --2- 172.21.15.37:0/336458843 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdb3c0fde90 0x7fdb3c0fe260 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:33:17.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:17.414+0000 7fdb41200700 1 -- 172.21.15.37:0/336458843 >> 172.21.15.37:0/336458843 conn(0x7fdb3c072bc0 msgr2=0x7fdb3c072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:33:17.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:17.414+0000 7fdb41200700 1 -- 172.21.15.37:0/336458843 shutdown_connections 2024-03-29T08:33:17.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:17.414+0000 7fdb41200700 1 -- 172.21.15.37:0/336458843 wait complete. 2024-03-29T08:33:17.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:17.415+0000 7fdb41200700 1 Processor -- start 2024-03-29T08:33:17.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:17.416+0000 7fdb41200700 1 -- start start 2024-03-29T08:33:17.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:17.416+0000 7fdb41200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdb3c0fde90 0x7fdb3c0fbd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:33:17.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:17.416+0000 7fdb41200700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdb3c0fe830 0x7fdb3c0fc290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:33:17.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:17.416+0000 7fdb41200700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb3c0fc820 con 0x7fdb3c0fde90 2024-03-29T08:33:17.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:17.416+0000 7fdb41200700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb3c0fc960 con 0x7fdb3c0fe830 2024-03-29T08:33:17.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:17.416+0000 7fdb39e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdb3c0fde90 0x7fdb3c0fbd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:33:17.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:17.416+0000 7fdb39e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdb3c0fde90 0x7fdb3c0fbd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41032/0 (socket says 172.21.15.37:41032) 2024-03-29T08:33:17.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:17.416+0000 7fdb39e00700 1 -- 172.21.15.37:0/4207290814 learned_addr learned my addr 172.21.15.37:0/4207290814 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:33:17.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:17.417+0000 7fdb39400700 1 --2- 172.21.15.37:0/4207290814 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdb3c0fe830 0x7fdb3c0fc290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:33:17.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:17.417+0000 7fdb39e00700 1 -- 172.21.15.37:0/4207290814 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdb3c0fe830 msgr2=0x7fdb3c0fc290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:33:17.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:17.417+0000 7fdb39e00700 1 --2- 172.21.15.37:0/4207290814 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdb3c0fe830 0x7fdb3c0fc290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:33:17.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:17.417+0000 7fdb39e00700 1 -- 172.21.15.37:0/4207290814 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb2c005040 con 0x7fdb3c0fde90 2024-03-29T08:33:17.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:17.417+0000 7fdb39e00700 1 --2- 172.21.15.37:0/4207290814 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdb3c0fde90 0x7fdb3c0fbd50 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fdb2c000c00 tx=0x7fdb2c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:33:17.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:17.417+0000 7fdb32000700 1 -- 172.21.15.37:0/4207290814 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdb2c00e040 con 0x7fdb3c0fde90 2024-03-29T08:33:17.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:17.417+0000 7fdb32000700 1 -- 172.21.15.37:0/4207290814 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fdb2c013420 con 0x7fdb3c0fde90 2024-03-29T08:33:17.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:17.417+0000 7fdb32000700 1 -- 172.21.15.37:0/4207290814 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdb2c01c630 con 0x7fdb3c0fde90 2024-03-29T08:33:17.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:17.417+0000 7fdb41200700 1 -- 172.21.15.37:0/4207290814 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb3c0fcbb0 con 0x7fdb3c0fde90 2024-03-29T08:33:17.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:17.418+0000 7fdb41200700 1 -- 172.21.15.37:0/4207290814 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb3c11a1a0 con 0x7fdb3c0fde90 2024-03-29T08:33:17.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:17.419+0000 7fdb41200700 1 -- 172.21.15.37:0/4207290814 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb3c065e30 con 0x7fdb3c0fde90 2024-03-29T08:33:17.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:17.420+0000 7fdb32000700 1 -- 172.21.15.37:0/4207290814 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fdb2c013970 con 0x7fdb3c0fde90 2024-03-29T08:33:17.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:17.420+0000 7fdb32000700 1 --2- 172.21.15.37:0/4207290814 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fdb2006cc30 0x7fdb2006f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:33:17.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:17.421+0000 7fdb32000700 1 -- 172.21.15.37:0/4207290814 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fdb2c08d230 con 0x7fdb3c0fde90 2024-03-29T08:33:17.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:17.421+0000 7fdb39400700 1 --2- 172.21.15.37:0/4207290814 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fdb2006cc30 0x7fdb2006f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:33:17.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:17.421+0000 7fdb39400700 1 --2- 172.21.15.37:0/4207290814 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fdb2006cc30 0x7fdb2006f0e0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fdb28007fd0 tx=0x7fdb28009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:33:17.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:17.425+0000 7fdb32000700 1 -- 172.21.15.37:0/4207290814 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fdb2c057270 con 0x7fdb3c0fde90 2024-03-29T08:33:17.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:33:17 smithi148 ceph-mon[106692]: from='client.35232 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:33:17.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:33:17 smithi148 ceph-mon[106692]: from='client.35236 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:33:17.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:33:17 smithi037 ceph-mon[142325]: from='client.35232 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:33:17.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:33:17 smithi037 ceph-mon[142325]: from='client.35236 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:33:17.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:17.682+0000 7fdb41200700 1 -- 172.21.15.37:0/4207290814 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fdb3c0fd440 con 0x7fdb3c0fde90 2024-03-29T08:33:17.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:17.683+0000 7fdb32000700 1 -- 172.21.15.37:0/4207290814 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 29 v29) v1 ==== 76+0+1663 (secure 0 0 0) 0x7fdb2c05a890 con 0x7fdb3c0fde90 2024-03-29T08:33:17.683 INFO:teuthology.orchestra.run.smithi037.stdout:e29 2024-03-29T08:33:17.683 INFO:teuthology.orchestra.run.smithi037.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-29T08:33:17.683 INFO:teuthology.orchestra.run.smithi037.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T08:33:17.683 INFO:teuthology.orchestra.run.smithi037.stdout:legacy client fscid: 1 2024-03-29T08:33:17.683 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:33:17.683 INFO:teuthology.orchestra.run.smithi037.stdout:Filesystem 'cephfs' (1) 2024-03-29T08:33:17.683 INFO:teuthology.orchestra.run.smithi037.stdout:fs_name cephfs 2024-03-29T08:33:17.683 INFO:teuthology.orchestra.run.smithi037.stdout:epoch 21 2024-03-29T08:33:17.683 INFO:teuthology.orchestra.run.smithi037.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-03-29T08:33:17.684 INFO:teuthology.orchestra.run.smithi037.stdout:created 2024-03-29T08:03:29.241269+0000 2024-03-29T08:33:17.684 INFO:teuthology.orchestra.run.smithi037.stdout:modified 2024-03-29T08:17:16.237238+0000 2024-03-29T08:33:17.684 INFO:teuthology.orchestra.run.smithi037.stdout:tableserver 0 2024-03-29T08:33:17.684 INFO:teuthology.orchestra.run.smithi037.stdout:root 0 2024-03-29T08:33:17.684 INFO:teuthology.orchestra.run.smithi037.stdout:session_timeout 60 2024-03-29T08:33:17.684 INFO:teuthology.orchestra.run.smithi037.stdout:session_autoclose 300 2024-03-29T08:33:17.684 INFO:teuthology.orchestra.run.smithi037.stdout:max_file_size 1099511627776 2024-03-29T08:33:17.684 INFO:teuthology.orchestra.run.smithi037.stdout:max_xattr_size 65536 2024-03-29T08:33:17.684 INFO:teuthology.orchestra.run.smithi037.stdout:required_client_features {} 2024-03-29T08:33:17.684 INFO:teuthology.orchestra.run.smithi037.stdout:last_failure 0 2024-03-29T08:33:17.684 INFO:teuthology.orchestra.run.smithi037.stdout:last_failure_osd_epoch 82 2024-03-29T08:33:17.685 INFO:teuthology.orchestra.run.smithi037.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T08:33:17.685 INFO:teuthology.orchestra.run.smithi037.stdout:max_mds 2 2024-03-29T08:33:17.685 INFO:teuthology.orchestra.run.smithi037.stdout:in 0,1 2024-03-29T08:33:17.685 INFO:teuthology.orchestra.run.smithi037.stdout:up {} 2024-03-29T08:33:17.685 INFO:teuthology.orchestra.run.smithi037.stdout:failed 0,1 2024-03-29T08:33:17.685 INFO:teuthology.orchestra.run.smithi037.stdout:damaged 2024-03-29T08:33:17.685 INFO:teuthology.orchestra.run.smithi037.stdout:stopped 2024-03-29T08:33:17.685 INFO:teuthology.orchestra.run.smithi037.stdout:data_pools [3] 2024-03-29T08:33:17.685 INFO:teuthology.orchestra.run.smithi037.stdout:metadata_pool 2 2024-03-29T08:33:17.685 INFO:teuthology.orchestra.run.smithi037.stdout:inline_data disabled 2024-03-29T08:33:17.685 INFO:teuthology.orchestra.run.smithi037.stdout:balancer 2024-03-29T08:33:17.685 INFO:teuthology.orchestra.run.smithi037.stdout:bal_rank_mask -1 2024-03-29T08:33:17.685 INFO:teuthology.orchestra.run.smithi037.stdout:standby_count_wanted 1 2024-03-29T08:33:17.685 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:33:17.685 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:33:17.685 INFO:teuthology.orchestra.run.smithi037.stdout:Standby daemons: 2024-03-29T08:33:17.685 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:33:17.686 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi037.mmwgxe{-1:34440} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.37:6828/1248142005,v1:172.21.15.37:6829/1248142005] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:33:17.686 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi037.iifsmh{-1:34444} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.37:6826/2060882959,v1:172.21.15.37:6827/2060882959] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:33:17.686 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi148.sqteih{-1:44395} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.148:6824/4078652242,v1:172.21.15.148:6825/4078652242] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:33:17.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:17.685+0000 7fdb41200700 1 -- 172.21.15.37:0/4207290814 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fdb2006cc30 msgr2=0x7fdb2006f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:33:17.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:17.685+0000 7fdb41200700 1 --2- 172.21.15.37:0/4207290814 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fdb2006cc30 0x7fdb2006f0e0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fdb28007fd0 tx=0x7fdb28009040 comp rx=0 tx=0).stop 2024-03-29T08:33:17.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:17.685+0000 7fdb41200700 1 -- 172.21.15.37:0/4207290814 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdb3c0fde90 msgr2=0x7fdb3c0fbd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:33:17.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:17.685+0000 7fdb41200700 1 --2- 172.21.15.37:0/4207290814 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdb3c0fde90 0x7fdb3c0fbd50 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fdb2c000c00 tx=0x7fdb2c009e30 comp rx=0 tx=0).stop 2024-03-29T08:33:17.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:17.685+0000 7fdb41200700 1 -- 172.21.15.37:0/4207290814 shutdown_connections 2024-03-29T08:33:17.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:17.685+0000 7fdb41200700 1 --2- 172.21.15.37:0/4207290814 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fdb2006cc30 0x7fdb2006f0e0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:33:17.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:17.685+0000 7fdb41200700 1 --2- 172.21.15.37:0/4207290814 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdb3c0fde90 0x7fdb3c0fbd50 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:33:17.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:17.685+0000 7fdb41200700 1 --2- 172.21.15.37:0/4207290814 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdb3c0fe830 0x7fdb3c0fc290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:33:17.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:17.685+0000 7fdb41200700 1 -- 172.21.15.37:0/4207290814 >> 172.21.15.37:0/4207290814 conn(0x7fdb3c072bc0 msgr2=0x7fdb3c0ff3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:33:17.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:17.685+0000 7fdb41200700 1 -- 172.21.15.37:0/4207290814 shutdown_connections 2024-03-29T08:33:17.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:17.686+0000 7fdb41200700 1 -- 172.21.15.37:0/4207290814 wait complete. 2024-03-29T08:33:17.687 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 29 2024-03-29T08:33:17.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:17.823+0000 7fc849000700 1 -- 172.21.15.37:0/2118620735 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc844102e00 msgr2=0x7fc8441031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:33:17.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:17.823+0000 7fc849000700 1 --2- 172.21.15.37:0/2118620735 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc844102e00 0x7fc8441031d0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fc834004940 tx=0x7fc834009e30 comp rx=0 tx=0).stop 2024-03-29T08:33:17.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:17.823+0000 7fc849000700 1 -- 172.21.15.37:0/2118620735 shutdown_connections 2024-03-29T08:33:17.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:17.823+0000 7fc849000700 1 --2- 172.21.15.37:0/2118620735 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc8441037a0 0x7fc8440fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:33:17.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:17.823+0000 7fc849000700 1 --2- 172.21.15.37:0/2118620735 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc844102e00 0x7fc8441031d0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:33:17.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:17.823+0000 7fc849000700 1 -- 172.21.15.37:0/2118620735 >> 172.21.15.37:0/2118620735 conn(0x7fc8440f89b0 msgr2=0x7fc8440fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:33:17.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:17.823+0000 7fc849000700 1 -- 172.21.15.37:0/2118620735 shutdown_connections 2024-03-29T08:33:17.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:17.823+0000 7fc849000700 1 -- 172.21.15.37:0/2118620735 wait complete. 2024-03-29T08:33:17.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:17.824+0000 7fc849000700 1 Processor -- start 2024-03-29T08:33:17.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:17.824+0000 7fc849000700 1 -- start start 2024-03-29T08:33:17.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:17.825+0000 7fc849000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc844102e00 0x7fc844115c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:33:17.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:17.825+0000 7fc849000700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc8441037a0 0x7fc8441161b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:33:17.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:17.825+0000 7fc849000700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc844116800 con 0x7fc844102e00 2024-03-29T08:33:17.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:17.825+0000 7fc849000700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc84410fbd0 con 0x7fc8441037a0 2024-03-29T08:33:17.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:17.825+0000 7fc841e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc844102e00 0x7fc844115c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:33:17.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:17.825+0000 7fc841e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc844102e00 0x7fc844115c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41052/0 (socket says 172.21.15.37:41052) 2024-03-29T08:33:17.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:17.825+0000 7fc841e00700 1 -- 172.21.15.37:0/3891105390 learned_addr learned my addr 172.21.15.37:0/3891105390 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:33:17.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:17.825+0000 7fc841400700 1 --2- 172.21.15.37:0/3891105390 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc8441037a0 0x7fc8441161b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:33:17.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:17.825+0000 7fc841e00700 1 -- 172.21.15.37:0/3891105390 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc8441037a0 msgr2=0x7fc8441161b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:33:17.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:17.825+0000 7fc841e00700 1 --2- 172.21.15.37:0/3891105390 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc8441037a0 0x7fc8441161b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:33:17.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:17.825+0000 7fc841e00700 1 -- 172.21.15.37:0/3891105390 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc834005040 con 0x7fc844102e00 2024-03-29T08:33:17.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:17.825+0000 7fc841400700 1 --2- 172.21.15.37:0/3891105390 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc8441037a0 0x7fc8441161b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:33:17.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:17.826+0000 7fc841e00700 1 --2- 172.21.15.37:0/3891105390 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc844102e00 0x7fc844115c70 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fc834007f50 tx=0x7fc834009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:33:17.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:17.826+0000 7fc83a000700 1 -- 172.21.15.37:0/3891105390 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc834021070 con 0x7fc844102e00 2024-03-29T08:33:17.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:17.826+0000 7fc83a000700 1 -- 172.21.15.37:0/3891105390 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fc834013e00 con 0x7fc844102e00 2024-03-29T08:33:17.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:17.826+0000 7fc83a000700 1 -- 172.21.15.37:0/3891105390 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc83400e070 con 0x7fc844102e00 2024-03-29T08:33:17.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:17.826+0000 7fc849000700 1 -- 172.21.15.37:0/3891105390 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc84410fe50 con 0x7fc844102e00 2024-03-29T08:33:17.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:17.826+0000 7fc849000700 1 -- 172.21.15.37:0/3891105390 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc844110410 con 0x7fc844102e00 2024-03-29T08:33:17.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:17.827+0000 7fc849000700 1 -- 172.21.15.37:0/3891105390 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8440fe550 con 0x7fc844102e00 2024-03-29T08:33:17.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:17.828+0000 7fc83a000700 1 -- 172.21.15.37:0/3891105390 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fc834013950 con 0x7fc844102e00 2024-03-29T08:33:17.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:17.829+0000 7fc83a000700 1 --2- 172.21.15.37:0/3891105390 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fc82806cc30 0x7fc82806f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:33:17.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:17.829+0000 7fc83a000700 1 -- 172.21.15.37:0/3891105390 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fc83408d190 con 0x7fc844102e00 2024-03-29T08:33:17.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:17.830+0000 7fc841400700 1 --2- 172.21.15.37:0/3891105390 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fc82806cc30 0x7fc82806f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:33:17.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:17.830+0000 7fc841400700 1 --2- 172.21.15.37:0/3891105390 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fc82806cc30 0x7fc82806f0e0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fc830004600 tx=0x7fc83000b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:33:17.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:17.834+0000 7fc83a000700 1 -- 172.21.15.37:0/3891105390 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fc83405ad20 con 0x7fc844102e00 2024-03-29T08:33:18.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:18.066+0000 7fc849000700 1 -- 172.21.15.37:0/3891105390 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc84404bf40 con 0x7fc82806cc30 2024-03-29T08:33:18.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:18.067+0000 7fc83a000700 1 -- 172.21.15.37:0/3891105390 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fc84404bf40 con 0x7fc82806cc30 2024-03-29T08:33:18.067 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-29T08:33:18.067 INFO:teuthology.orchestra.run.smithi037.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59", 2024-03-29T08:33:18.067 INFO:teuthology.orchestra.run.smithi037.stdout: "in_progress": true, 2024-03-29T08:33:18.067 INFO:teuthology.orchestra.run.smithi037.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-29T08:33:18.067 INFO:teuthology.orchestra.run.smithi037.stdout: "services_complete": [ 2024-03-29T08:33:18.067 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr", 2024-03-29T08:33:18.067 INFO:teuthology.orchestra.run.smithi037.stdout: "mon", 2024-03-29T08:33:18.067 INFO:teuthology.orchestra.run.smithi037.stdout: "crash", 2024-03-29T08:33:18.067 INFO:teuthology.orchestra.run.smithi037.stdout: "osd" 2024-03-29T08:33:18.067 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-29T08:33:18.067 INFO:teuthology.orchestra.run.smithi037.stdout: "progress": "15/21 daemons upgraded", 2024-03-29T08:33:18.067 INFO:teuthology.orchestra.run.smithi037.stdout: "message": "Currently upgrading mds daemons", 2024-03-29T08:33:18.068 INFO:teuthology.orchestra.run.smithi037.stdout: "is_paused": false 2024-03-29T08:33:18.068 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-29T08:33:18.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:18.069+0000 7fc849000700 1 -- 172.21.15.37:0/3891105390 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fc82806cc30 msgr2=0x7fc82806f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:33:18.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:18.069+0000 7fc849000700 1 --2- 172.21.15.37:0/3891105390 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fc82806cc30 0x7fc82806f0e0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fc830004600 tx=0x7fc83000b040 comp rx=0 tx=0).stop 2024-03-29T08:33:18.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:18.070+0000 7fc849000700 1 -- 172.21.15.37:0/3891105390 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc844102e00 msgr2=0x7fc844115c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:33:18.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:18.070+0000 7fc849000700 1 --2- 172.21.15.37:0/3891105390 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc844102e00 0x7fc844115c70 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fc834007f50 tx=0x7fc834009e30 comp rx=0 tx=0).stop 2024-03-29T08:33:18.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:18.070+0000 7fc849000700 1 -- 172.21.15.37:0/3891105390 shutdown_connections 2024-03-29T08:33:18.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:18.070+0000 7fc849000700 1 --2- 172.21.15.37:0/3891105390 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fc82806cc30 0x7fc82806f0e0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:33:18.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:18.070+0000 7fc849000700 1 --2- 172.21.15.37:0/3891105390 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc844102e00 0x7fc844115c70 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:33:18.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:18.070+0000 7fc849000700 1 --2- 172.21.15.37:0/3891105390 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc8441037a0 0x7fc8441161b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:33:18.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:18.070+0000 7fc849000700 1 -- 172.21.15.37:0/3891105390 >> 172.21.15.37:0/3891105390 conn(0x7fc8440f89b0 msgr2=0x7fc8440f9960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:33:18.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:18.070+0000 7fc849000700 1 -- 172.21.15.37:0/3891105390 shutdown_connections 2024-03-29T08:33:18.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:18.070+0000 7fc849000700 1 -- 172.21.15.37:0/3891105390 wait complete. 2024-03-29T08:33:18.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:18.206+0000 7f5c00c00700 1 -- 172.21.15.37:0/2951724116 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5bfc0fde90 msgr2=0x7f5bfc0fe260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:33:18.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:18.206+0000 7f5c00c00700 1 --2- 172.21.15.37:0/2951724116 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5bfc0fde90 0x7f5bfc0fe260 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f5bec004940 tx=0x7f5bec009e30 comp rx=0 tx=0).stop 2024-03-29T08:33:18.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:18.207+0000 7f5c00c00700 1 -- 172.21.15.37:0/2951724116 shutdown_connections 2024-03-29T08:33:18.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:18.207+0000 7f5c00c00700 1 --2- 172.21.15.37:0/2951724116 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5bfc0fe830 0x7f5bfc106f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:33:18.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:18.207+0000 7f5c00c00700 1 --2- 172.21.15.37:0/2951724116 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5bfc0fde90 0x7f5bfc0fe260 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:33:18.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:18.207+0000 7f5c00c00700 1 -- 172.21.15.37:0/2951724116 >> 172.21.15.37:0/2951724116 conn(0x7f5bfc072bc0 msgr2=0x7f5bfc072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:33:18.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:18.207+0000 7f5c00c00700 1 -- 172.21.15.37:0/2951724116 shutdown_connections 2024-03-29T08:33:18.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:18.207+0000 7f5c00c00700 1 -- 172.21.15.37:0/2951724116 wait complete. 2024-03-29T08:33:18.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:18.207+0000 7f5c00c00700 1 Processor -- start 2024-03-29T08:33:18.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:18.208+0000 7f5c00c00700 1 -- start start 2024-03-29T08:33:18.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:18.208+0000 7f5c00c00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5bfc0fde90 0x7f5bfc115d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:33:18.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:18.208+0000 7f5c00c00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5bfc0fe830 0x7f5bfc116240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:33:18.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:18.208+0000 7f5c00c00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bfc116890 con 0x7f5bfc0fe830 2024-03-29T08:33:18.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:18.208+0000 7f5c00c00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bfc10fc60 con 0x7f5bfc0fde90 2024-03-29T08:33:18.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:18.208+0000 7f5bf9400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5bfc0fe830 0x7f5bfc116240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:33:18.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:18.208+0000 7f5bf9400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5bfc0fe830 0x7f5bfc116240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41074/0 (socket says 172.21.15.37:41074) 2024-03-29T08:33:18.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:18.208+0000 7f5bf9400700 1 -- 172.21.15.37:0/3031801048 learned_addr learned my addr 172.21.15.37:0/3031801048 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:33:18.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:18.208+0000 7f5bf9e00700 1 --2- 172.21.15.37:0/3031801048 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5bfc0fde90 0x7f5bfc115d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:33:18.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:18.209+0000 7f5bf9400700 1 -- 172.21.15.37:0/3031801048 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5bfc0fde90 msgr2=0x7f5bfc115d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:33:18.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:18.209+0000 7f5bf9400700 1 --2- 172.21.15.37:0/3031801048 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5bfc0fde90 0x7f5bfc115d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:33:18.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:18.209+0000 7f5bf9400700 1 -- 172.21.15.37:0/3031801048 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bec005040 con 0x7f5bfc0fe830 2024-03-29T08:33:18.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:18.209+0000 7f5bf9400700 1 --2- 172.21.15.37:0/3031801048 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5bfc0fe830 0x7f5bfc116240 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f5be8002a50 tx=0x7f5be80046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:33:18.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:18.209+0000 7f5bf2000700 1 -- 172.21.15.37:0/3031801048 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5be8007cc0 con 0x7f5bfc0fe830 2024-03-29T08:33:18.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:18.209+0000 7f5bf2000700 1 -- 172.21.15.37:0/3031801048 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f5be800f040 con 0x7f5bfc0fe830 2024-03-29T08:33:18.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:18.209+0000 7f5c00c00700 1 -- 172.21.15.37:0/3031801048 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bfc10ff40 con 0x7f5bfc0fe830 2024-03-29T08:33:18.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:18.209+0000 7f5bf2000700 1 -- 172.21.15.37:0/3031801048 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5be800d9c0 con 0x7f5bfc0fe830 2024-03-29T08:33:18.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:18.209+0000 7f5c00c00700 1 -- 172.21.15.37:0/3031801048 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bfc110480 con 0x7f5bfc0fe830 2024-03-29T08:33:18.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:18.210+0000 7f5c00c00700 1 -- 172.21.15.37:0/3031801048 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5bfc106a20 con 0x7f5bfc0fe830 2024-03-29T08:33:18.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:18.212+0000 7f5bf2000700 1 -- 172.21.15.37:0/3031801048 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f5be800db20 con 0x7f5bfc0fe830 2024-03-29T08:33:18.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:18.212+0000 7f5bf2000700 1 --2- 172.21.15.37:0/3031801048 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f5be006cc30 0x7f5be006f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:33:18.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:18.212+0000 7f5bf2000700 1 -- 172.21.15.37:0/3031801048 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f5be808bd20 con 0x7f5bfc0fe830 2024-03-29T08:33:18.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:18.213+0000 7f5bf9e00700 1 --2- 172.21.15.37:0/3031801048 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f5be006cc30 0x7f5be006f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:33:18.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:18.213+0000 7f5bf9e00700 1 --2- 172.21.15.37:0/3031801048 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f5be006cc30 0x7f5be006f0e0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f5bec00b3b0 tx=0x7f5bec009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:33:18.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:18.217+0000 7f5bf2000700 1 -- 172.21.15.37:0/3031801048 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f5be8059770 con 0x7f5bfc0fe830 2024-03-29T08:33:18.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:33:18 smithi148 ceph-mon[106692]: from='client.35240 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:33:18.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:33:18 smithi148 ceph-mon[106692]: pgmap v673: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:33:18.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:33:18 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/665025048' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:33:18.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:33:18 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/4207290814' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T08:33:18.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:18.516+0000 7f5c00c00700 1 -- 172.21.15.37:0/3031801048 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f5bfc065e30 con 0x7f5bfc0fe830 2024-03-29T08:33:18.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:18.517+0000 7f5bf2000700 1 -- 172.21.15.37:0/3031801048 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+3583 (secure 0 0 0) 0x7f5be8059300 con 0x7f5bfc0fe830 2024-03-29T08:33:18.517 INFO:teuthology.orchestra.run.smithi037.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem is offline 2024-03-29T08:33:18.517 INFO:teuthology.orchestra.run.smithi037.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-03-29T08:33:18.517 INFO:teuthology.orchestra.run.smithi037.stdout: host smithi148 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/mon.smithi148/config 2024-03-29T08:33:18.517 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/podman: timeout after 895 seconds 2024-03-29T08:33:18.517 INFO:teuthology.orchestra.run.smithi037.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59 -e NODE_NAME=smithi148 -e CEPH_USE_RANDOM_NONCE=1 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4:/var/run/ceph:z -v /var/log/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4:/var/log/ceph:z -v /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpkx2rz2d4:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59 inventory --format=json-pretty --filter-for-batch 2024-03-29T08:33:18.517 INFO:teuthology.orchestra.run.smithi037.stdout:Traceback (most recent call last): 2024-03-29T08:33:18.517 INFO:teuthology.orchestra.run.smithi037.stdout: File "/usr/lib64/python3.6/runpy.py", line 193, in _run_module_as_main 2024-03-29T08:33:18.517 INFO:teuthology.orchestra.run.smithi037.stdout: "__main__", mod_spec) 2024-03-29T08:33:18.517 INFO:teuthology.orchestra.run.smithi037.stdout: File "/usr/lib64/python3.6/runpy.py", line 85, in _run_code 2024-03-29T08:33:18.517 INFO:teuthology.orchestra.run.smithi037.stdout: exec(code, run_globals) 2024-03-29T08:33:18.518 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 10889, in 2024-03-29T08:33:18.518 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 10877, in main 2024-03-29T08:33:18.518 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 2576, in _infer_config 2024-03-29T08:33:18.518 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 2492, in _infer_fsid 2024-03-29T08:33:18.518 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 2604, in _infer_image 2024-03-29T08:33:18.518 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 2479, in _validate_fsid 2024-03-29T08:33:18.518 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 7145, in command_ceph_volume 2024-03-29T08:33:18.518 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 2267, in call_throws 2024-03-29T08:33:18.518 INFO:teuthology.orchestra.run.smithi037.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59 -e NODE_NAME=smithi148 -e CEPH_USE_RANDOM_NONCE=1 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4:/var/run/ceph:z -v /var/log/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4:/var/log/ceph:z -v /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpkx2rz2d4:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59 inventory --format=json-pretty --filter-for-batch 2024-03-29T08:33:18.518 INFO:teuthology.orchestra.run.smithi037.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-03-29T08:33:18.518 INFO:teuthology.orchestra.run.smithi037.stdout: fs cephfs is degraded 2024-03-29T08:33:18.518 INFO:teuthology.orchestra.run.smithi037.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-03-29T08:33:18.518 INFO:teuthology.orchestra.run.smithi037.stdout: fs cephfs is offline because no MDS is active for it. 2024-03-29T08:33:18.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:18.519+0000 7f5c00c00700 1 -- 172.21.15.37:0/3031801048 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f5be006cc30 msgr2=0x7f5be006f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:33:18.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:18.519+0000 7f5c00c00700 1 --2- 172.21.15.37:0/3031801048 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f5be006cc30 0x7f5be006f0e0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f5bec00b3b0 tx=0x7f5bec009e30 comp rx=0 tx=0).stop 2024-03-29T08:33:18.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:18.519+0000 7f5c00c00700 1 -- 172.21.15.37:0/3031801048 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5bfc0fe830 msgr2=0x7f5bfc116240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:33:18.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:18.519+0000 7f5c00c00700 1 --2- 172.21.15.37:0/3031801048 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5bfc0fe830 0x7f5bfc116240 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f5be8002a50 tx=0x7f5be80046b0 comp rx=0 tx=0).stop 2024-03-29T08:33:18.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:18.519+0000 7f5c00c00700 1 -- 172.21.15.37:0/3031801048 shutdown_connections 2024-03-29T08:33:18.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:18.519+0000 7f5c00c00700 1 --2- 172.21.15.37:0/3031801048 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f5be006cc30 0x7f5be006f0e0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:33:18.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:18.519+0000 7f5c00c00700 1 --2- 172.21.15.37:0/3031801048 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5bfc0fde90 0x7f5bfc115d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:33:18.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:18.519+0000 7f5c00c00700 1 --2- 172.21.15.37:0/3031801048 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5bfc0fe830 0x7f5bfc116240 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:33:18.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:18.519+0000 7f5c00c00700 1 -- 172.21.15.37:0/3031801048 >> 172.21.15.37:0/3031801048 conn(0x7f5bfc072bc0 msgr2=0x7f5bfc0fd590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:33:18.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:18.519+0000 7f5c00c00700 1 -- 172.21.15.37:0/3031801048 shutdown_connections 2024-03-29T08:33:18.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:18.519+0000 7f5c00c00700 1 -- 172.21.15.37:0/3031801048 wait complete. 2024-03-29T08:33:18.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:33:18 smithi037 ceph-mon[142325]: from='client.35240 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:33:18.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:33:18 smithi037 ceph-mon[142325]: pgmap v673: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:33:18.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:33:18 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/665025048' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:33:18.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:33:18 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/4207290814' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T08:33:19.545 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:33:19 smithi148 ceph-mon[106692]: from='client.35252 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:33:19.546 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:33:19 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/3031801048' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T08:33:19.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:33:19 smithi037 ceph-mon[142325]: from='client.35252 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:33:19.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:33:19 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/3031801048' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T08:33:20.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:33:20 smithi037 ceph-mon[142325]: pgmap v674: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:33:20.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:33:20 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:33:20.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:33:20 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:33:20.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:33:20 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T08:33:20.928 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:33:20 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:33:20.928 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:33:20 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:33:20.928 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:33:20 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:33:20.928 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:33:20 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:33:20.928 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:33:20 smithi037 ceph-mon[142325]: mds.? [v2:172.21.15.148:6826/2394751105,v1:172.21.15.148:6827/2394751105] up:boot 2024-03-29T08:33:20.928 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:33:20 smithi037 ceph-mon[142325]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-03-29T08:33:20.928 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:33:20 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi148.qukpjh"}]: dispatch 2024-03-29T08:33:21.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:33:20 smithi148 ceph-mon[106692]: pgmap v674: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:33:21.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:33:20 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:33:21.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:33:20 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:33:21.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:33:20 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T08:33:21.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:33:20 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:33:21.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:33:20 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:33:21.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:33:20 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:33:21.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:33:20 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:33:21.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:33:20 smithi148 ceph-mon[106692]: mds.? [v2:172.21.15.148:6826/2394751105,v1:172.21.15.148:6827/2394751105] up:boot 2024-03-29T08:33:21.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:33:20 smithi148 ceph-mon[106692]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-03-29T08:33:21.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:33:20 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi148.qukpjh"}]: dispatch 2024-03-29T08:33:22.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:33:22 smithi037 ceph-mon[142325]: pgmap v675: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:33:23.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:33:22 smithi148 ceph-mon[106692]: pgmap v675: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:33:24.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:33:23 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:33:24.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:33:23 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:33:24.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:33:23 smithi037 ceph-mon[142325]: pgmap v676: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:33:24.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:33:23 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:33:24.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:33:23 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:33:24.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:33:23 smithi148 ceph-mon[106692]: pgmap v676: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:33:25.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:33:25 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:33:25.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:33:25 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:33:25.307 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:33:25 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:33:25.307 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:33:25 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:33:25.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:33:25 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:33:25.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:33:25 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:33:25.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:33:25 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:33:25.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:33:25 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:33:26.807 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:33:26 smithi148 ceph-mon[106692]: pgmap v677: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:33:26.807 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:33:26 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:33:26.807 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:33:26 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:33:26.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:33:26 smithi037 ceph-mon[142325]: pgmap v677: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:33:26.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:33:26 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:33:26.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:33:26 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:33:28.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:33:28 smithi148 ceph-mon[106692]: pgmap v678: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:33:28.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:33:28 smithi037 ceph-mon[142325]: pgmap v678: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:33:30.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:33:30 smithi148 ceph-mon[106692]: pgmap v679: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:33:30.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:33:30 smithi037 ceph-mon[142325]: pgmap v679: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:33:32.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:33:32 smithi148 ceph-mon[106692]: pgmap v680: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:33:32.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:33:32 smithi037 ceph-mon[142325]: pgmap v680: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:33:34.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:33:34 smithi148 ceph-mon[106692]: pgmap v681: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:33:34.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:33:34 smithi037 ceph-mon[142325]: pgmap v681: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:33:36.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:33:36 smithi148 ceph-mon[106692]: pgmap v682: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:33:36.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:33:36 smithi037 ceph-mon[142325]: pgmap v682: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:33:38.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:33:38 smithi148 ceph-mon[106692]: pgmap v683: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:33:38.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:33:38 smithi037 ceph-mon[142325]: pgmap v683: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:33:40.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:33:40 smithi148 ceph-mon[106692]: pgmap v684: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:33:40.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:33:40 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:33:40.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:33:40 smithi037 ceph-mon[142325]: pgmap v684: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:33:40.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:33:40 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:33:42.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:33:42 smithi148 ceph-mon[106692]: pgmap v685: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:33:42.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:33:42 smithi037 ceph-mon[142325]: pgmap v685: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:33:44.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:33:44 smithi148 ceph-mon[106692]: pgmap v686: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:33:44.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:33:44 smithi037 ceph-mon[142325]: pgmap v686: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:33:46.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:33:46 smithi148 ceph-mon[106692]: pgmap v687: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:33:46.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:33:46 smithi037 ceph-mon[142325]: pgmap v687: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:33:48.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:48.677+0000 7fcae0400700 1 -- 172.21.15.37:0/2195460894 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcad81037a0 msgr2=0x7fcad80fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:33:48.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:48.677+0000 7fcae0400700 1 --2- 172.21.15.37:0/2195460894 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcad81037a0 0x7fcad80fce10 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fcac4004940 tx=0x7fcac4009e30 comp rx=0 tx=0).stop 2024-03-29T08:33:48.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:48.677+0000 7fcae0400700 1 -- 172.21.15.37:0/2195460894 shutdown_connections 2024-03-29T08:33:48.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:48.677+0000 7fcae0400700 1 --2- 172.21.15.37:0/2195460894 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcad81037a0 0x7fcad80fce10 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:33:48.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:48.677+0000 7fcae0400700 1 --2- 172.21.15.37:0/2195460894 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcad8102e00 0x7fcad81031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:33:48.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:48.677+0000 7fcae0400700 1 -- 172.21.15.37:0/2195460894 >> 172.21.15.37:0/2195460894 conn(0x7fcad80f89b0 msgr2=0x7fcad80fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:33:48.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:48.677+0000 7fcae0400700 1 -- 172.21.15.37:0/2195460894 shutdown_connections 2024-03-29T08:33:48.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:48.677+0000 7fcae0400700 1 -- 172.21.15.37:0/2195460894 wait complete. 2024-03-29T08:33:48.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:48.678+0000 7fcae0400700 1 Processor -- start 2024-03-29T08:33:48.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:48.678+0000 7fcae0400700 1 -- start start 2024-03-29T08:33:48.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:48.678+0000 7fcae0400700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcad8102e00 0x7fcad8115c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:33:48.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:48.678+0000 7fcae0400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcad81037a0 0x7fcad81161a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:33:48.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:48.678+0000 7fcae0400700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcad81167f0 con 0x7fcad81037a0 2024-03-29T08:33:48.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:48.678+0000 7fcae0400700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcad810fbc0 con 0x7fcad8102e00 2024-03-29T08:33:48.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:48.679+0000 7fcadd000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcad81037a0 0x7fcad81161a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:33:48.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:48.679+0000 7fcadd000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcad81037a0 0x7fcad81161a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44596/0 (socket says 172.21.15.37:44596) 2024-03-29T08:33:48.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:48.679+0000 7fcadd000700 1 -- 172.21.15.37:0/3958640989 learned_addr learned my addr 172.21.15.37:0/3958640989 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:33:48.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:48.679+0000 7fcadda00700 1 --2- 172.21.15.37:0/3958640989 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcad8102e00 0x7fcad8115c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:33:48.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:48.679+0000 7fcadd000700 1 -- 172.21.15.37:0/3958640989 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcad8102e00 msgr2=0x7fcad8115c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:33:48.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:48.679+0000 7fcadd000700 1 --2- 172.21.15.37:0/3958640989 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcad8102e00 0x7fcad8115c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:33:48.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:48.679+0000 7fcadd000700 1 -- 172.21.15.37:0/3958640989 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcac4005040 con 0x7fcad81037a0 2024-03-29T08:33:48.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:48.679+0000 7fcadda00700 1 --2- 172.21.15.37:0/3958640989 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcad8102e00 0x7fcad8115c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:33:48.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:48.679+0000 7fcadd000700 1 --2- 172.21.15.37:0/3958640989 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcad81037a0 0x7fcad81161a0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fcac4009fd0 tx=0x7fcac4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:33:48.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:48.680+0000 7fcad5600700 1 -- 172.21.15.37:0/3958640989 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcac400e030 con 0x7fcad81037a0 2024-03-29T08:33:48.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:48.680+0000 7fcad5600700 1 -- 172.21.15.37:0/3958640989 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fcac4013e00 con 0x7fcad81037a0 2024-03-29T08:33:48.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:48.680+0000 7fcae0400700 1 -- 172.21.15.37:0/3958640989 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcad810fe40 con 0x7fcad81037a0 2024-03-29T08:33:48.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:48.680+0000 7fcad5600700 1 -- 172.21.15.37:0/3958640989 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcac401ccd0 con 0x7fcad81037a0 2024-03-29T08:33:48.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:48.680+0000 7fcae0400700 1 -- 172.21.15.37:0/3958640989 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcad8110400 con 0x7fcad81037a0 2024-03-29T08:33:48.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:48.681+0000 7fcae0400700 1 -- 172.21.15.37:0/3958640989 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcad80fe550 con 0x7fcad81037a0 2024-03-29T08:33:48.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:48.682+0000 7fcad5600700 1 -- 172.21.15.37:0/3958640989 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fcac401c500 con 0x7fcad81037a0 2024-03-29T08:33:48.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:48.683+0000 7fcad5600700 1 --2- 172.21.15.37:0/3958640989 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fcac006cc30 0x7fcac006f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:33:48.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:48.683+0000 7fcad5600700 1 -- 172.21.15.37:0/3958640989 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fcac408d980 con 0x7fcad81037a0 2024-03-29T08:33:48.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:48.683+0000 7fcadda00700 1 --2- 172.21.15.37:0/3958640989 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fcac006cc30 0x7fcac006f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:33:48.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:48.684+0000 7fcadda00700 1 --2- 172.21.15.37:0/3958640989 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fcac006cc30 0x7fcac006f0e0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fcacc009fd0 tx=0x7fcacc00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:33:48.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:48.688+0000 7fcad5600700 1 -- 172.21.15.37:0/3958640989 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fcac405b3d0 con 0x7fcad81037a0 2024-03-29T08:33:48.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:33:48 smithi148 ceph-mon[106692]: pgmap v688: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:33:48.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:48.922+0000 7fcae0400700 1 -- 172.21.15.37:0/3958640989 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fcad8109e10 con 0x7fcac006cc30 2024-03-29T08:33:48.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:48.923+0000 7fcad5600700 1 -- 172.21.15.37:0/3958640989 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fcad8109e10 con 0x7fcac006cc30 2024-03-29T08:33:48.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:48.925+0000 7fcae0400700 1 -- 172.21.15.37:0/3958640989 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fcac006cc30 msgr2=0x7fcac006f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:33:48.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:48.925+0000 7fcae0400700 1 --2- 172.21.15.37:0/3958640989 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fcac006cc30 0x7fcac006f0e0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fcacc009fd0 tx=0x7fcacc00b040 comp rx=0 tx=0).stop 2024-03-29T08:33:48.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:48.925+0000 7fcae0400700 1 -- 172.21.15.37:0/3958640989 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcad81037a0 msgr2=0x7fcad81161a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:33:48.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:48.925+0000 7fcae0400700 1 --2- 172.21.15.37:0/3958640989 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcad81037a0 0x7fcad81161a0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fcac4009fd0 tx=0x7fcac4009e30 comp rx=0 tx=0).stop 2024-03-29T08:33:48.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:48.925+0000 7fcae0400700 1 -- 172.21.15.37:0/3958640989 shutdown_connections 2024-03-29T08:33:48.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:48.925+0000 7fcae0400700 1 --2- 172.21.15.37:0/3958640989 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fcac006cc30 0x7fcac006f0e0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:33:48.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:48.925+0000 7fcae0400700 1 --2- 172.21.15.37:0/3958640989 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcad8102e00 0x7fcad8115c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:33:48.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:48.925+0000 7fcae0400700 1 --2- 172.21.15.37:0/3958640989 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcad81037a0 0x7fcad81161a0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:33:48.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:48.925+0000 7fcae0400700 1 -- 172.21.15.37:0/3958640989 >> 172.21.15.37:0/3958640989 conn(0x7fcad80f89b0 msgr2=0x7fcad80f99c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:33:48.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:48.926+0000 7fcae0400700 1 -- 172.21.15.37:0/3958640989 shutdown_connections 2024-03-29T08:33:48.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:48.926+0000 7fcae0400700 1 -- 172.21.15.37:0/3958640989 wait complete. 2024-03-29T08:33:48.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:33:48 smithi037 ceph-mon[142325]: pgmap v688: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:33:48.940 INFO:teuthology.orchestra.run.smithi037.stdout:true 2024-03-29T08:33:49.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:49.064+0000 7febc8000700 1 -- 172.21.15.37:0/1147078400 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7febc00fe830 msgr2=0x7febc0106f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:33:49.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:49.064+0000 7febc8000700 1 --2- 172.21.15.37:0/1147078400 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7febc00fe830 0x7febc0106f00 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7febac004940 tx=0x7febac009e30 comp rx=0 tx=0).stop 2024-03-29T08:33:49.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:49.064+0000 7febc8000700 1 -- 172.21.15.37:0/1147078400 shutdown_connections 2024-03-29T08:33:49.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:49.064+0000 7febc8000700 1 --2- 172.21.15.37:0/1147078400 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7febc00fe830 0x7febc0106f00 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:33:49.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:49.064+0000 7febc8000700 1 --2- 172.21.15.37:0/1147078400 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7febc00fde90 0x7febc00fe260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:33:49.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:49.064+0000 7febc8000700 1 -- 172.21.15.37:0/1147078400 >> 172.21.15.37:0/1147078400 conn(0x7febc0072bc0 msgr2=0x7febc0072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:33:49.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:49.064+0000 7febc8000700 1 -- 172.21.15.37:0/1147078400 shutdown_connections 2024-03-29T08:33:49.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:49.064+0000 7febc8000700 1 -- 172.21.15.37:0/1147078400 wait complete. 2024-03-29T08:33:49.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:49.065+0000 7febc8000700 1 Processor -- start 2024-03-29T08:33:49.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:49.065+0000 7febc8000700 1 -- start start 2024-03-29T08:33:49.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:49.066+0000 7febc8000700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7febc00fde90 0x7febc0113a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:33:49.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:49.066+0000 7febc8000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7febc00fe830 0x7febc0113fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:33:49.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:49.066+0000 7febc8000700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febc0114610 con 0x7febc00fe830 2024-03-29T08:33:49.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:49.066+0000 7febc8000700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febc010da00 con 0x7febc00fde90 2024-03-29T08:33:49.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:49.066+0000 7febc4c00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7febc00fe830 0x7febc0113fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:33:49.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:49.066+0000 7febc4c00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7febc00fe830 0x7febc0113fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44620/0 (socket says 172.21.15.37:44620) 2024-03-29T08:33:49.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:49.066+0000 7febc4c00700 1 -- 172.21.15.37:0/2008968466 learned_addr learned my addr 172.21.15.37:0/2008968466 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:33:49.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:49.066+0000 7febc5600700 1 --2- 172.21.15.37:0/2008968466 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7febc00fde90 0x7febc0113a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:33:49.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:49.066+0000 7febc4c00700 1 -- 172.21.15.37:0/2008968466 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7febc00fde90 msgr2=0x7febc0113a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:33:49.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:49.066+0000 7febc4c00700 1 --2- 172.21.15.37:0/2008968466 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7febc00fde90 0x7febc0113a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:33:49.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:49.066+0000 7febc4c00700 1 -- 172.21.15.37:0/2008968466 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febac005040 con 0x7febc00fe830 2024-03-29T08:33:49.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:49.067+0000 7febc4c00700 1 --2- 172.21.15.37:0/2008968466 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7febc00fe830 0x7febc0113fc0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7febac000c00 tx=0x7febac009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:33:49.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:49.067+0000 7febbd600700 1 -- 172.21.15.37:0/2008968466 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7febac021070 con 0x7febc00fe830 2024-03-29T08:33:49.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:49.067+0000 7febbd600700 1 -- 172.21.15.37:0/2008968466 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7febac013e00 con 0x7febc00fe830 2024-03-29T08:33:49.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:49.067+0000 7febbd600700 1 -- 172.21.15.37:0/2008968466 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7febac00e070 con 0x7febc00fe830 2024-03-29T08:33:49.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:49.067+0000 7febc8000700 1 -- 172.21.15.37:0/2008968466 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febc010dc80 con 0x7febc00fe830 2024-03-29T08:33:49.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:49.067+0000 7febc8000700 1 -- 172.21.15.37:0/2008968466 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febc010e170 con 0x7febc00fe830 2024-03-29T08:33:49.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:49.068+0000 7febc8000700 1 -- 172.21.15.37:0/2008968466 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febc0101980 con 0x7febc00fe830 2024-03-29T08:33:49.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:49.069+0000 7febbd600700 1 -- 172.21.15.37:0/2008968466 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7febac013950 con 0x7febc00fe830 2024-03-29T08:33:49.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:49.070+0000 7febbd600700 1 --2- 172.21.15.37:0/2008968466 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7feba806cc30 0x7feba806f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:33:49.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:49.070+0000 7febbd600700 1 -- 172.21.15.37:0/2008968466 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7febac08d0d0 con 0x7febc00fe830 2024-03-29T08:33:49.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:49.070+0000 7febc5600700 1 --2- 172.21.15.37:0/2008968466 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7feba806cc30 0x7feba806f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:33:49.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:49.071+0000 7febc5600700 1 --2- 172.21.15.37:0/2008968466 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7feba806cc30 0x7feba806f0e0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7febb4009b80 tx=0x7febb400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:33:49.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:49.075+0000 7febbd600700 1 -- 172.21.15.37:0/2008968466 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7febac05ab20 con 0x7febc00fe830 2024-03-29T08:33:49.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:49.304+0000 7febc8000700 1 -- 172.21.15.37:0/2008968466 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7febc010f0d0 con 0x7feba806cc30 2024-03-29T08:33:49.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:49.306+0000 7febbd600700 1 -- 172.21.15.37:0/2008968466 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7febc010f0d0 con 0x7feba806cc30 2024-03-29T08:33:49.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:49.308+0000 7febc8000700 1 -- 172.21.15.37:0/2008968466 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7feba806cc30 msgr2=0x7feba806f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:33:49.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:49.308+0000 7febc8000700 1 --2- 172.21.15.37:0/2008968466 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7feba806cc30 0x7feba806f0e0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7febb4009b80 tx=0x7febb400b040 comp rx=0 tx=0).stop 2024-03-29T08:33:49.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:49.308+0000 7febc8000700 1 -- 172.21.15.37:0/2008968466 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7febc00fe830 msgr2=0x7febc0113fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:33:49.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:49.308+0000 7febc8000700 1 --2- 172.21.15.37:0/2008968466 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7febc00fe830 0x7febc0113fc0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7febac000c00 tx=0x7febac009e30 comp rx=0 tx=0).stop 2024-03-29T08:33:49.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:49.308+0000 7febc8000700 1 -- 172.21.15.37:0/2008968466 shutdown_connections 2024-03-29T08:33:49.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:49.308+0000 7febc8000700 1 --2- 172.21.15.37:0/2008968466 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7feba806cc30 0x7feba806f0e0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:33:49.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:49.308+0000 7febc8000700 1 --2- 172.21.15.37:0/2008968466 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7febc00fde90 0x7febc0113a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:33:49.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:49.308+0000 7febc8000700 1 --2- 172.21.15.37:0/2008968466 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7febc00fe830 0x7febc0113fc0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:33:49.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:49.308+0000 7febc8000700 1 -- 172.21.15.37:0/2008968466 >> 172.21.15.37:0/2008968466 conn(0x7febc0072bc0 msgr2=0x7febc00ff3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:33:49.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:49.308+0000 7febc8000700 1 -- 172.21.15.37:0/2008968466 shutdown_connections 2024-03-29T08:33:49.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:49.308+0000 7febc8000700 1 -- 172.21.15.37:0/2008968466 wait complete. 2024-03-29T08:33:49.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:49.446+0000 7f6256800700 1 -- 172.21.15.37:0/2967652429 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6250070c70 msgr2=0x7f6250108f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:33:49.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:49.446+0000 7f6256800700 1 --2- 172.21.15.37:0/2967652429 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6250070c70 0x7f6250108f50 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f623c004970 tx=0x7f623c009e30 comp rx=0 tx=0).stop 2024-03-29T08:33:49.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:49.447+0000 7f6256800700 1 -- 172.21.15.37:0/2967652429 shutdown_connections 2024-03-29T08:33:49.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:49.447+0000 7f6256800700 1 --2- 172.21.15.37:0/2967652429 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6250070c70 0x7f6250108f50 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:33:49.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:49.447+0000 7f6256800700 1 --2- 172.21.15.37:0/2967652429 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6250070360 0x7f6250070730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:33:49.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:49.447+0000 7f6256800700 1 -- 172.21.15.37:0/2967652429 >> 172.21.15.37:0/2967652429 conn(0x7f62500f8830 msgr2=0x7f62500fac40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:33:49.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:49.447+0000 7f6256800700 1 -- 172.21.15.37:0/2967652429 shutdown_connections 2024-03-29T08:33:49.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:49.447+0000 7f6256800700 1 -- 172.21.15.37:0/2967652429 wait complete. 2024-03-29T08:33:49.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:49.448+0000 7f6256800700 1 Processor -- start 2024-03-29T08:33:49.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:49.448+0000 7f6256800700 1 -- start start 2024-03-29T08:33:49.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:49.448+0000 7f6256800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6250070360 0x7f6250115ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:33:49.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:49.448+0000 7f6256800700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6250070c70 0x7f6250116000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:33:49.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:49.448+0000 7f6256800700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6250116650 con 0x7f6250070360 2024-03-29T08:33:49.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:49.448+0000 7f6256800700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f625010fa40 con 0x7f6250070c70 2024-03-29T08:33:49.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:49.448+0000 7f6255400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6250070360 0x7f6250115ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:33:49.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:49.448+0000 7f6255400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6250070360 0x7f6250115ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44636/0 (socket says 172.21.15.37:44636) 2024-03-29T08:33:49.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:49.448+0000 7f6255400700 1 -- 172.21.15.37:0/2663390 learned_addr learned my addr 172.21.15.37:0/2663390 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:33:49.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:49.449+0000 7f6254a00700 1 --2- 172.21.15.37:0/2663390 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6250070c70 0x7f6250116000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:33:49.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:49.449+0000 7f6255400700 1 -- 172.21.15.37:0/2663390 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6250070c70 msgr2=0x7f6250116000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:33:49.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:49.449+0000 7f6255400700 1 --2- 172.21.15.37:0/2663390 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6250070c70 0x7f6250116000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:33:49.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:49.449+0000 7f6255400700 1 -- 172.21.15.37:0/2663390 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f623c005040 con 0x7f6250070360 2024-03-29T08:33:49.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:49.449+0000 7f6255400700 1 --2- 172.21.15.37:0/2663390 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6250070360 0x7f6250115ac0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f6244002a50 tx=0x7f62440046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:33:49.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:49.449+0000 7f624d600700 1 -- 172.21.15.37:0/2663390 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6244007cc0 con 0x7f6250070360 2024-03-29T08:33:49.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:49.449+0000 7f624d600700 1 -- 172.21.15.37:0/2663390 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f624400f040 con 0x7f6250070360 2024-03-29T08:33:49.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:49.449+0000 7f6256800700 1 -- 172.21.15.37:0/2663390 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f625010fd20 con 0x7f6250070360 2024-03-29T08:33:49.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:49.449+0000 7f624d600700 1 -- 172.21.15.37:0/2663390 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f624400d9c0 con 0x7f6250070360 2024-03-29T08:33:49.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:49.450+0000 7f6256800700 1 -- 172.21.15.37:0/2663390 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6250110230 con 0x7f6250070360 2024-03-29T08:33:49.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:49.451+0000 7f6256800700 1 -- 172.21.15.37:0/2663390 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6250071aa0 con 0x7f6250070360 2024-03-29T08:33:49.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:49.452+0000 7f624d600700 1 -- 172.21.15.37:0/2663390 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f624400db20 con 0x7f6250070360 2024-03-29T08:33:49.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:49.452+0000 7f624d600700 1 --2- 172.21.15.37:0/2663390 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f623806cc30 0x7f623806f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:33:49.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:49.453+0000 7f624d600700 1 -- 172.21.15.37:0/2663390 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f624408bd20 con 0x7f6250070360 2024-03-29T08:33:49.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:49.453+0000 7f6254a00700 1 --2- 172.21.15.37:0/2663390 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f623806cc30 0x7f623806f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:33:49.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:49.453+0000 7f6254a00700 1 --2- 172.21.15.37:0/2663390 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f623806cc30 0x7f623806f0e0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f623c009fd0 tx=0x7f623c009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:33:49.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:49.457+0000 7f624d600700 1 -- 172.21.15.37:0/2663390 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f6244059770 con 0x7f6250070360 2024-03-29T08:33:49.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:49.684+0000 7f6256800700 1 -- 172.21.15.37:0/2663390 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f625010bdf0 con 0x7f623806cc30 2024-03-29T08:33:49.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:49.695+0000 7f624d600700 1 -- 172.21.15.37:0/2663390 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f625010bdf0 con 0x7f623806cc30 2024-03-29T08:33:49.694 INFO:teuthology.orchestra.run.smithi037.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-29T08:33:49.695 INFO:teuthology.orchestra.run.smithi037.stdout:alertmanager.smithi037 smithi037 *:9093,9094 running (23m) 24s ago 36m 14.3M - 0.25.0 c8568f914cd2 2f9f9746ed0f 2024-03-29T08:33:49.695 INFO:teuthology.orchestra.run.smithi037.stdout:crash.smithi037 smithi037 running (20m) 24s ago 36m 6647k - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 178d416a3186 2024-03-29T08:33:49.695 INFO:teuthology.orchestra.run.smithi037.stdout:crash.smithi148 smithi148 running (20m) 26s ago 35m 6656k - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 23666989d58f 2024-03-29T08:33:49.695 INFO:teuthology.orchestra.run.smithi037.stdout:grafana.smithi037 smithi037 *:3000 running (23m) 24s ago 36m 74.5M - 9.4.7 954c08fa6188 6b976fe49952 2024-03-29T08:33:49.695 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi037.iifsmh smithi037 running (16m) 24s ago 30m 16.4M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 2d97f562cc5b 2024-03-29T08:33:49.695 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi037.mmwgxe smithi037 running (16m) 24s ago 30m 16.7M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff ec37037f46e8 2024-03-29T08:33:49.695 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi148.qukpjh smithi148 running (30s) 26s ago 30m 10.7M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 3257219bc046 2024-03-29T08:33:49.695 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi148.sqteih smithi148 running (15m) 26s ago 30m 16.3M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 44c15c77037a 2024-03-29T08:33:49.695 INFO:teuthology.orchestra.run.smithi037.stdout:mgr.smithi037.wjikii smithi037 *:8443,9283,8765 running (26m) 24s ago 38m 510M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff c5a1417b9417 2024-03-29T08:33:49.695 INFO:teuthology.orchestra.run.smithi037.stdout:mgr.smithi148.onhgvm smithi148 *:8443,9283,8765 running (24m) 26s ago 35m 456M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff fcfdbce062bd 2024-03-29T08:33:49.695 INFO:teuthology.orchestra.run.smithi037.stdout:mon.smithi037 smithi037 running (21m) 24s ago 38m 70.2M 2048M 18.2.2-1015-gd0e392d9 1cc2f45c17ff eea2060783af 2024-03-29T08:33:49.695 INFO:teuthology.orchestra.run.smithi037.stdout:mon.smithi148 smithi148 running (21m) 26s ago 35m 50.4M 2048M 18.2.2-1015-gd0e392d9 1cc2f45c17ff ad4b2c5748be 2024-03-29T08:33:49.695 INFO:teuthology.orchestra.run.smithi037.stdout:node-exporter.smithi037 smithi037 *:9100 running (24m) 24s ago 36m 8514k - 1.5.0 0da6a335fe13 26b8ae9cb878 2024-03-29T08:33:49.695 INFO:teuthology.orchestra.run.smithi037.stdout:node-exporter.smithi148 smithi148 *:9100 running (24m) 26s ago 35m 8464k - 1.5.0 0da6a335fe13 98414e4e64ed 2024-03-29T08:33:49.696 INFO:teuthology.orchestra.run.smithi037.stdout:osd.0 smithi037 running (20m) 24s ago 34m 108M 1619M 18.2.2-1015-gd0e392d9 1cc2f45c17ff c0c584f6245c 2024-03-29T08:33:49.696 INFO:teuthology.orchestra.run.smithi037.stdout:osd.1 smithi037 running (19m) 24s ago 33m 104M 1619M 18.2.2-1015-gd0e392d9 1cc2f45c17ff a4a2ff4cc83d 2024-03-29T08:33:49.696 INFO:teuthology.orchestra.run.smithi037.stdout:osd.2 smithi037 running (18m) 24s ago 33m 85.9M 1619M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 8b1bc71aa79e 2024-03-29T08:33:49.696 INFO:teuthology.orchestra.run.smithi037.stdout:osd.3 smithi148 running (18m) 26s ago 32m 109M 2642M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 1c656ab8ff86 2024-03-29T08:33:49.696 INFO:teuthology.orchestra.run.smithi037.stdout:osd.4 smithi148 running (17m) 26s ago 32m 101M 2642M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 966a9de11d22 2024-03-29T08:33:49.696 INFO:teuthology.orchestra.run.smithi037.stdout:osd.5 smithi148 running (16m) 26s ago 31m 89.4M 2642M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 474c4f9212ce 2024-03-29T08:33:49.696 INFO:teuthology.orchestra.run.smithi037.stdout:prometheus.smithi037 smithi037 *:9095 running (24m) 24s ago 36m 61.0M - 2.43.0 a07b618ecd1d 30c0223ec589 2024-03-29T08:33:49.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:49.697+0000 7f6256800700 1 -- 172.21.15.37:0/2663390 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f623806cc30 msgr2=0x7f623806f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:33:49.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:49.697+0000 7f6256800700 1 --2- 172.21.15.37:0/2663390 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f623806cc30 0x7f623806f0e0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f623c009fd0 tx=0x7f623c009e30 comp rx=0 tx=0).stop 2024-03-29T08:33:49.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:49.697+0000 7f6256800700 1 -- 172.21.15.37:0/2663390 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6250070360 msgr2=0x7f6250115ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:33:49.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:49.697+0000 7f6256800700 1 --2- 172.21.15.37:0/2663390 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6250070360 0x7f6250115ac0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f6244002a50 tx=0x7f62440046b0 comp rx=0 tx=0).stop 2024-03-29T08:33:49.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:49.697+0000 7f6256800700 1 -- 172.21.15.37:0/2663390 shutdown_connections 2024-03-29T08:33:49.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:49.697+0000 7f6256800700 1 --2- 172.21.15.37:0/2663390 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f623806cc30 0x7f623806f0e0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:33:49.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:49.697+0000 7f6256800700 1 --2- 172.21.15.37:0/2663390 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6250070360 0x7f6250115ac0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:33:49.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:49.697+0000 7f6256800700 1 --2- 172.21.15.37:0/2663390 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6250070c70 0x7f6250116000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:33:49.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:49.697+0000 7f6256800700 1 -- 172.21.15.37:0/2663390 >> 172.21.15.37:0/2663390 conn(0x7f62500f8830 msgr2=0x7f62500f9d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:33:49.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:49.697+0000 7f6256800700 1 -- 172.21.15.37:0/2663390 shutdown_connections 2024-03-29T08:33:49.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:49.697+0000 7f6256800700 1 -- 172.21.15.37:0/2663390 wait complete. 2024-03-29T08:33:49.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:49.834+0000 7f5155a00700 1 -- 172.21.15.37:0/1450719743 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5150102c40 msgr2=0x7f5150103010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:33:49.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:49.834+0000 7f5155a00700 1 --2- 172.21.15.37:0/1450719743 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5150102c40 0x7f5150103010 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f5140004940 tx=0x7f5140009e30 comp rx=0 tx=0).stop 2024-03-29T08:33:49.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:49.834+0000 7f5155a00700 1 -- 172.21.15.37:0/1450719743 shutdown_connections 2024-03-29T08:33:49.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:49.834+0000 7f5155a00700 1 --2- 172.21.15.37:0/1450719743 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f51501035e0 0x7f51500fcc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:33:49.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:49.834+0000 7f5155a00700 1 --2- 172.21.15.37:0/1450719743 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5150102c40 0x7f5150103010 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:33:49.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:49.834+0000 7f5155a00700 1 -- 172.21.15.37:0/1450719743 >> 172.21.15.37:0/1450719743 conn(0x7f51500f8830 msgr2=0x7f51500fac40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:33:49.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:49.834+0000 7f5155a00700 1 -- 172.21.15.37:0/1450719743 shutdown_connections 2024-03-29T08:33:49.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:49.834+0000 7f5155a00700 1 -- 172.21.15.37:0/1450719743 wait complete. 2024-03-29T08:33:49.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:49.835+0000 7f5155a00700 1 Processor -- start 2024-03-29T08:33:49.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:49.835+0000 7f5155a00700 1 -- start start 2024-03-29T08:33:49.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:49.836+0000 7f5155a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5150102c40 0x7f5150113870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:33:49.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:49.836+0000 7f5155a00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f51501035e0 0x7f5150113db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:33:49.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:49.836+0000 7f5155a00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5150114400 con 0x7f5150102c40 2024-03-29T08:33:49.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:49.836+0000 7f5155a00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f515010d7f0 con 0x7f51501035e0 2024-03-29T08:33:49.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:49.836+0000 7f514fe00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5150102c40 0x7f5150113870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:33:49.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:49.836+0000 7f514fe00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5150102c40 0x7f5150113870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44648/0 (socket says 172.21.15.37:44648) 2024-03-29T08:33:49.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:49.836+0000 7f514fe00700 1 -- 172.21.15.37:0/527714671 learned_addr learned my addr 172.21.15.37:0/527714671 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:33:49.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:49.836+0000 7f514f400700 1 --2- 172.21.15.37:0/527714671 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f51501035e0 0x7f5150113db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:33:49.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:49.836+0000 7f514fe00700 1 -- 172.21.15.37:0/527714671 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f51501035e0 msgr2=0x7f5150113db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:33:49.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:49.836+0000 7f514fe00700 1 --2- 172.21.15.37:0/527714671 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f51501035e0 0x7f5150113db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:33:49.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:49.836+0000 7f514fe00700 1 -- 172.21.15.37:0/527714671 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5140005040 con 0x7f5150102c40 2024-03-29T08:33:49.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:49.836+0000 7f514f400700 1 --2- 172.21.15.37:0/527714671 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f51501035e0 0x7f5150113db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:33:49.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:49.837+0000 7f514fe00700 1 --2- 172.21.15.37:0/527714671 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5150102c40 0x7f5150113870 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f5140008d80 tx=0x7f5140009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:33:49.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:49.837+0000 7f5147e00700 1 -- 172.21.15.37:0/527714671 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f514000e030 con 0x7f5150102c40 2024-03-29T08:33:49.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:49.837+0000 7f5147e00700 1 -- 172.21.15.37:0/527714671 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f5140013de0 con 0x7f5150102c40 2024-03-29T08:33:49.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:49.837+0000 7f5155a00700 1 -- 172.21.15.37:0/527714671 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f515010da70 con 0x7f5150102c40 2024-03-29T08:33:49.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:49.837+0000 7f5147e00700 1 -- 172.21.15.37:0/527714671 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f514001cd60 con 0x7f5150102c40 2024-03-29T08:33:49.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:49.837+0000 7f5155a00700 1 -- 172.21.15.37:0/527714671 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f515010e030 con 0x7f5150102c40 2024-03-29T08:33:49.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:49.838+0000 7f5155a00700 1 -- 172.21.15.37:0/527714671 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51500fe390 con 0x7f5150102c40 2024-03-29T08:33:49.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:49.839+0000 7f5147e00700 1 -- 172.21.15.37:0/527714671 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f514001c590 con 0x7f5150102c40 2024-03-29T08:33:49.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:49.840+0000 7f5147e00700 1 --2- 172.21.15.37:0/527714671 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f513406cc30 0x7f513406f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:33:49.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:49.841+0000 7f5147e00700 1 -- 172.21.15.37:0/527714671 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f514008d9a0 con 0x7f5150102c40 2024-03-29T08:33:49.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:49.841+0000 7f514f400700 1 --2- 172.21.15.37:0/527714671 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f513406cc30 0x7f513406f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:33:49.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:49.841+0000 7f514f400700 1 --2- 172.21.15.37:0/527714671 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f513406cc30 0x7f513406f0e0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f513c004600 tx=0x7f513c00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:33:49.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:49.845+0000 7f5147e00700 1 -- 172.21.15.37:0/527714671 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f514005b3f0 con 0x7f5150102c40 2024-03-29T08:33:50.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:50.145+0000 7f5155a00700 1 -- 172.21.15.37:0/527714671 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f515004bed0 con 0x7f5150102c40 2024-03-29T08:33:50.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:50.146+0000 7f5147e00700 1 -- 172.21.15.37:0/527714671 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7f514005af80 con 0x7f5150102c40 2024-03-29T08:33:50.146 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-29T08:33:50.146 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": { 2024-03-29T08:33:50.146 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 2 2024-03-29T08:33:50.146 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:33:50.146 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": { 2024-03-29T08:33:50.147 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 2 2024-03-29T08:33:50.147 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:33:50.147 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": { 2024-03-29T08:33:50.147 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 6 2024-03-29T08:33:50.147 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:33:50.147 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": { 2024-03-29T08:33:50.147 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 4 2024-03-29T08:33:50.147 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:33:50.147 INFO:teuthology.orchestra.run.smithi037.stdout: "overall": { 2024-03-29T08:33:50.147 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 14 2024-03-29T08:33:50.147 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-29T08:33:50.147 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-29T08:33:50.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:50.148+0000 7f5155a00700 1 -- 172.21.15.37:0/527714671 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f513406cc30 msgr2=0x7f513406f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:33:50.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:50.148+0000 7f5155a00700 1 --2- 172.21.15.37:0/527714671 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f513406cc30 0x7f513406f0e0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f513c004600 tx=0x7f513c00b040 comp rx=0 tx=0).stop 2024-03-29T08:33:50.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:50.148+0000 7f5155a00700 1 -- 172.21.15.37:0/527714671 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5150102c40 msgr2=0x7f5150113870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:33:50.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:50.148+0000 7f5155a00700 1 --2- 172.21.15.37:0/527714671 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5150102c40 0x7f5150113870 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f5140008d80 tx=0x7f5140009e30 comp rx=0 tx=0).stop 2024-03-29T08:33:50.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:50.148+0000 7f5155a00700 1 -- 172.21.15.37:0/527714671 shutdown_connections 2024-03-29T08:33:50.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:50.148+0000 7f5155a00700 1 --2- 172.21.15.37:0/527714671 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f513406cc30 0x7f513406f0e0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:33:50.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:50.148+0000 7f5155a00700 1 --2- 172.21.15.37:0/527714671 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5150102c40 0x7f5150113870 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:33:50.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:50.148+0000 7f5155a00700 1 --2- 172.21.15.37:0/527714671 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f51501035e0 0x7f5150113db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:33:50.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:50.148+0000 7f5155a00700 1 -- 172.21.15.37:0/527714671 >> 172.21.15.37:0/527714671 conn(0x7f51500f8830 msgr2=0x7f51500f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:33:50.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:50.149+0000 7f5155a00700 1 -- 172.21.15.37:0/527714671 shutdown_connections 2024-03-29T08:33:50.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:50.149+0000 7f5155a00700 1 -- 172.21.15.37:0/527714671 wait complete. 2024-03-29T08:33:50.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:50.285+0000 7f1eac800700 1 -- 172.21.15.37:0/1587481656 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ea4071f90 msgr2=0x7f1ea4070360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:33:50.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:50.285+0000 7f1eac800700 1 --2- 172.21.15.37:0/1587481656 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ea4071f90 0x7f1ea4070360 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f1e98004970 tx=0x7f1e98009e30 comp rx=0 tx=0).stop 2024-03-29T08:33:50.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:50.286+0000 7f1eac800700 1 -- 172.21.15.37:0/1587481656 shutdown_connections 2024-03-29T08:33:50.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:50.286+0000 7f1eac800700 1 --2- 172.21.15.37:0/1587481656 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1ea40708a0 0x7f1ea4070d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:33:50.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:50.286+0000 7f1eac800700 1 --2- 172.21.15.37:0/1587481656 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ea4071f90 0x7f1ea4070360 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:33:50.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:50.286+0000 7f1eac800700 1 -- 172.21.15.37:0/1587481656 >> 172.21.15.37:0/1587481656 conn(0x7f1ea40f89b0 msgr2=0x7f1ea40fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:33:50.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:50.286+0000 7f1eac800700 1 -- 172.21.15.37:0/1587481656 shutdown_connections 2024-03-29T08:33:50.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:50.286+0000 7f1eac800700 1 -- 172.21.15.37:0/1587481656 wait complete. 2024-03-29T08:33:50.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:50.286+0000 7f1eac800700 1 Processor -- start 2024-03-29T08:33:50.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:50.287+0000 7f1eac800700 1 -- start start 2024-03-29T08:33:50.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:50.287+0000 7f1eac800700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1ea40708a0 0x7f1ea411a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:33:50.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:50.287+0000 7f1eac800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ea4071f90 0x7f1ea411a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:33:50.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:50.287+0000 7f1eac800700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ea411ac50 con 0x7f1ea4071f90 2024-03-29T08:33:50.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:50.287+0000 7f1eac800700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ea4114020 con 0x7f1ea40708a0 2024-03-29T08:33:50.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:50.287+0000 7f1ea9400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ea4071f90 0x7f1ea411a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:33:50.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:50.287+0000 7f1ea9400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ea4071f90 0x7f1ea411a600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45662/0 (socket says 172.21.15.37:45662) 2024-03-29T08:33:50.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:50.287+0000 7f1ea9400700 1 -- 172.21.15.37:0/1346371256 learned_addr learned my addr 172.21.15.37:0/1346371256 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:33:50.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:50.287+0000 7f1ea9e00700 1 --2- 172.21.15.37:0/1346371256 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1ea40708a0 0x7f1ea411a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:33:50.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:50.288+0000 7f1ea9400700 1 -- 172.21.15.37:0/1346371256 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1ea40708a0 msgr2=0x7f1ea411a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:33:50.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:50.288+0000 7f1ea9400700 1 --2- 172.21.15.37:0/1346371256 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1ea40708a0 0x7f1ea411a0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:33:50.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:50.288+0000 7f1ea9400700 1 -- 172.21.15.37:0/1346371256 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e98005040 con 0x7f1ea4071f90 2024-03-29T08:33:50.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:50.288+0000 7f1ea9400700 1 --2- 172.21.15.37:0/1346371256 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ea4071f90 0x7f1ea411a600 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f1e94002a80 tx=0x7f1e940046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:33:50.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:50.288+0000 7f1e9e000700 1 -- 172.21.15.37:0/1346371256 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1e94007cc0 con 0x7f1ea4071f90 2024-03-29T08:33:50.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:50.288+0000 7f1e9e000700 1 -- 172.21.15.37:0/1346371256 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f1e9400f040 con 0x7f1ea4071f90 2024-03-29T08:33:50.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:50.288+0000 7f1eac800700 1 -- 172.21.15.37:0/1346371256 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ea4114300 con 0x7f1ea4071f90 2024-03-29T08:33:50.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:50.288+0000 7f1e9e000700 1 -- 172.21.15.37:0/1346371256 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1e9400da60 con 0x7f1ea4071f90 2024-03-29T08:33:50.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:50.288+0000 7f1eac800700 1 -- 172.21.15.37:0/1346371256 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ea4114840 con 0x7f1ea4071f90 2024-03-29T08:33:50.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:50.289+0000 7f1eac800700 1 -- 172.21.15.37:0/1346371256 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ea4071aa0 con 0x7f1ea4071f90 2024-03-29T08:33:50.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:50.290+0000 7f1e9e000700 1 -- 172.21.15.37:0/1346371256 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f1e9400dbc0 con 0x7f1ea4071f90 2024-03-29T08:33:50.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:50.291+0000 7f1e9e000700 1 --2- 172.21.15.37:0/1346371256 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f1e8c06cc30 0x7f1e8c06f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:33:50.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:50.292+0000 7f1e9e000700 1 -- 172.21.15.37:0/1346371256 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f1e9408bd20 con 0x7f1ea4071f90 2024-03-29T08:33:50.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:50.292+0000 7f1ea9e00700 1 --2- 172.21.15.37:0/1346371256 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f1e8c06cc30 0x7f1e8c06f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:33:50.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:50.292+0000 7f1ea9e00700 1 --2- 172.21.15.37:0/1346371256 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f1e8c06cc30 0x7f1e8c06f0e0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f1e9800b370 tx=0x7f1e98009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:33:50.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:50.296+0000 7f1e9e000700 1 -- 172.21.15.37:0/1346371256 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f1e940596f0 con 0x7f1ea4071f90 2024-03-29T08:33:50.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:50.555+0000 7f1eac800700 1 -- 172.21.15.37:0/1346371256 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f1ea405f5f0 con 0x7f1ea4071f90 2024-03-29T08:33:50.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:50.555+0000 7f1e9e000700 1 -- 172.21.15.37:0/1346371256 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 30 v30) v1 ==== 76+0+1841 (secure 0 0 0) 0x7f1e94059280 con 0x7f1ea4071f90 2024-03-29T08:33:50.555 INFO:teuthology.orchestra.run.smithi037.stdout:e30 2024-03-29T08:33:50.555 INFO:teuthology.orchestra.run.smithi037.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-29T08:33:50.556 INFO:teuthology.orchestra.run.smithi037.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T08:33:50.556 INFO:teuthology.orchestra.run.smithi037.stdout:legacy client fscid: 1 2024-03-29T08:33:50.556 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:33:50.556 INFO:teuthology.orchestra.run.smithi037.stdout:Filesystem 'cephfs' (1) 2024-03-29T08:33:50.556 INFO:teuthology.orchestra.run.smithi037.stdout:fs_name cephfs 2024-03-29T08:33:50.556 INFO:teuthology.orchestra.run.smithi037.stdout:epoch 21 2024-03-29T08:33:50.556 INFO:teuthology.orchestra.run.smithi037.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-03-29T08:33:50.556 INFO:teuthology.orchestra.run.smithi037.stdout:created 2024-03-29T08:03:29.241269+0000 2024-03-29T08:33:50.556 INFO:teuthology.orchestra.run.smithi037.stdout:modified 2024-03-29T08:17:16.237238+0000 2024-03-29T08:33:50.557 INFO:teuthology.orchestra.run.smithi037.stdout:tableserver 0 2024-03-29T08:33:50.557 INFO:teuthology.orchestra.run.smithi037.stdout:root 0 2024-03-29T08:33:50.557 INFO:teuthology.orchestra.run.smithi037.stdout:session_timeout 60 2024-03-29T08:33:50.557 INFO:teuthology.orchestra.run.smithi037.stdout:session_autoclose 300 2024-03-29T08:33:50.557 INFO:teuthology.orchestra.run.smithi037.stdout:max_file_size 1099511627776 2024-03-29T08:33:50.557 INFO:teuthology.orchestra.run.smithi037.stdout:max_xattr_size 65536 2024-03-29T08:33:50.557 INFO:teuthology.orchestra.run.smithi037.stdout:required_client_features {} 2024-03-29T08:33:50.558 INFO:teuthology.orchestra.run.smithi037.stdout:last_failure 0 2024-03-29T08:33:50.558 INFO:teuthology.orchestra.run.smithi037.stdout:last_failure_osd_epoch 82 2024-03-29T08:33:50.558 INFO:teuthology.orchestra.run.smithi037.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T08:33:50.558 INFO:teuthology.orchestra.run.smithi037.stdout:max_mds 2 2024-03-29T08:33:50.558 INFO:teuthology.orchestra.run.smithi037.stdout:in 0,1 2024-03-29T08:33:50.558 INFO:teuthology.orchestra.run.smithi037.stdout:up {} 2024-03-29T08:33:50.558 INFO:teuthology.orchestra.run.smithi037.stdout:failed 0,1 2024-03-29T08:33:50.558 INFO:teuthology.orchestra.run.smithi037.stdout:damaged 2024-03-29T08:33:50.558 INFO:teuthology.orchestra.run.smithi037.stdout:stopped 2024-03-29T08:33:50.559 INFO:teuthology.orchestra.run.smithi037.stdout:data_pools [3] 2024-03-29T08:33:50.559 INFO:teuthology.orchestra.run.smithi037.stdout:metadata_pool 2 2024-03-29T08:33:50.559 INFO:teuthology.orchestra.run.smithi037.stdout:inline_data disabled 2024-03-29T08:33:50.559 INFO:teuthology.orchestra.run.smithi037.stdout:balancer 2024-03-29T08:33:50.559 INFO:teuthology.orchestra.run.smithi037.stdout:bal_rank_mask -1 2024-03-29T08:33:50.559 INFO:teuthology.orchestra.run.smithi037.stdout:standby_count_wanted 1 2024-03-29T08:33:50.559 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:33:50.559 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:33:50.559 INFO:teuthology.orchestra.run.smithi037.stdout:Standby daemons: 2024-03-29T08:33:50.560 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:33:50.560 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi037.mmwgxe{-1:34440} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.37:6828/1248142005,v1:172.21.15.37:6829/1248142005] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:33:50.560 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi037.iifsmh{-1:34444} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.37:6826/2060882959,v1:172.21.15.37:6827/2060882959] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:33:50.560 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi148.sqteih{-1:44395} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.148:6824/4078652242,v1:172.21.15.148:6825/4078652242] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:33:50.560 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi148.qukpjh{-1:45007} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.148:6826/2394751105,v1:172.21.15.148:6827/2394751105] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:33:50.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:50.558+0000 7f1eac800700 1 -- 172.21.15.37:0/1346371256 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f1e8c06cc30 msgr2=0x7f1e8c06f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:33:50.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:50.558+0000 7f1eac800700 1 --2- 172.21.15.37:0/1346371256 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f1e8c06cc30 0x7f1e8c06f0e0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f1e9800b370 tx=0x7f1e98009e30 comp rx=0 tx=0).stop 2024-03-29T08:33:50.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:50.558+0000 7f1eac800700 1 -- 172.21.15.37:0/1346371256 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ea4071f90 msgr2=0x7f1ea411a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:33:50.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:50.558+0000 7f1eac800700 1 --2- 172.21.15.37:0/1346371256 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ea4071f90 0x7f1ea411a600 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f1e94002a80 tx=0x7f1e940046b0 comp rx=0 tx=0).stop 2024-03-29T08:33:50.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:50.558+0000 7f1eac800700 1 -- 172.21.15.37:0/1346371256 shutdown_connections 2024-03-29T08:33:50.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:50.558+0000 7f1eac800700 1 --2- 172.21.15.37:0/1346371256 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f1e8c06cc30 0x7f1e8c06f0e0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:33:50.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:50.558+0000 7f1eac800700 1 --2- 172.21.15.37:0/1346371256 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1ea40708a0 0x7f1ea411a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:33:50.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:50.558+0000 7f1eac800700 1 --2- 172.21.15.37:0/1346371256 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ea4071f90 0x7f1ea411a600 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:33:50.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:50.558+0000 7f1eac800700 1 -- 172.21.15.37:0/1346371256 >> 172.21.15.37:0/1346371256 conn(0x7f1ea40f89b0 msgr2=0x7f1ea40f9b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:33:50.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:50.558+0000 7f1eac800700 1 -- 172.21.15.37:0/1346371256 shutdown_connections 2024-03-29T08:33:50.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:50.558+0000 7f1eac800700 1 -- 172.21.15.37:0/1346371256 wait complete. 2024-03-29T08:33:50.562 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 30 2024-03-29T08:33:50.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:50.697+0000 7f4526200700 1 -- 172.21.15.37:0/185480374 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4520102e00 msgr2=0x7f45201031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:33:50.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:50.697+0000 7f4526200700 1 --2- 172.21.15.37:0/185480374 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4520102e00 0x7f45201031d0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f4510004940 tx=0x7f4510009e30 comp rx=0 tx=0).stop 2024-03-29T08:33:50.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:50.697+0000 7f4526200700 1 -- 172.21.15.37:0/185480374 shutdown_connections 2024-03-29T08:33:50.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:50.697+0000 7f4526200700 1 --2- 172.21.15.37:0/185480374 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f45201037a0 0x7f45200fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:33:50.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:50.697+0000 7f4526200700 1 --2- 172.21.15.37:0/185480374 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4520102e00 0x7f45201031d0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:33:50.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:50.697+0000 7f4526200700 1 -- 172.21.15.37:0/185480374 >> 172.21.15.37:0/185480374 conn(0x7f45200f89b0 msgr2=0x7f45200fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:33:50.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:50.697+0000 7f4526200700 1 -- 172.21.15.37:0/185480374 shutdown_connections 2024-03-29T08:33:50.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:50.697+0000 7f4526200700 1 -- 172.21.15.37:0/185480374 wait complete. 2024-03-29T08:33:50.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:50.698+0000 7f4526200700 1 Processor -- start 2024-03-29T08:33:50.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:50.698+0000 7f4526200700 1 -- start start 2024-03-29T08:33:50.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:50.699+0000 7f4526200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4520102e00 0x7f4520115c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:33:50.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:50.699+0000 7f4526200700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f45201037a0 0x7f4520116180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:33:50.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:50.699+0000 7f4526200700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45201167d0 con 0x7f4520102e00 2024-03-29T08:33:50.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:50.699+0000 7f4526200700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f452010fc50 con 0x7f45201037a0 2024-03-29T08:33:50.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:50.699+0000 7f451f400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4520102e00 0x7f4520115c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:33:50.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:50.699+0000 7f451f400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4520102e00 0x7f4520115c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45668/0 (socket says 172.21.15.37:45668) 2024-03-29T08:33:50.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:50.699+0000 7f451f400700 1 -- 172.21.15.37:0/1125671959 learned_addr learned my addr 172.21.15.37:0/1125671959 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:33:50.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:50.699+0000 7f451f400700 1 -- 172.21.15.37:0/1125671959 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f45201037a0 msgr2=0x7f4520116180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:33:50.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:50.699+0000 7f451f400700 1 --2- 172.21.15.37:0/1125671959 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f45201037a0 0x7f4520116180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:33:50.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:50.699+0000 7f451f400700 1 -- 172.21.15.37:0/1125671959 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4510005040 con 0x7f4520102e00 2024-03-29T08:33:50.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:50.700+0000 7f451f400700 1 --2- 172.21.15.37:0/1125671959 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4520102e00 0x7f4520115c40 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f4510009360 tx=0x7f4510009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:33:50.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:50.700+0000 7f4517400700 1 -- 172.21.15.37:0/1125671959 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f451000e040 con 0x7f4520102e00 2024-03-29T08:33:50.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:50.700+0000 7f4517400700 1 -- 172.21.15.37:0/1125671959 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f4510013420 con 0x7f4520102e00 2024-03-29T08:33:50.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:50.700+0000 7f4517400700 1 -- 172.21.15.37:0/1125671959 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f451001c5f0 con 0x7f4520102e00 2024-03-29T08:33:50.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:50.700+0000 7f4526200700 1 -- 172.21.15.37:0/1125671959 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f452010fed0 con 0x7f4520102e00 2024-03-29T08:33:50.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:50.700+0000 7f4526200700 1 -- 172.21.15.37:0/1125671959 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4520110490 con 0x7f4520102e00 2024-03-29T08:33:50.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:50.701+0000 7f4526200700 1 -- 172.21.15.37:0/1125671959 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45200fe550 con 0x7f4520102e00 2024-03-29T08:33:50.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:50.702+0000 7f4517400700 1 -- 172.21.15.37:0/1125671959 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f4510013590 con 0x7f4520102e00 2024-03-29T08:33:50.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:50.703+0000 7f4517400700 1 --2- 172.21.15.37:0/1125671959 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f450406cc30 0x7f450406f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:33:50.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:50.703+0000 7f4517400700 1 -- 172.21.15.37:0/1125671959 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f451008d250 con 0x7f4520102e00 2024-03-29T08:33:50.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:50.703+0000 7f451ea00700 1 --2- 172.21.15.37:0/1125671959 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f450406cc30 0x7f450406f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:33:50.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:50.704+0000 7f451ea00700 1 --2- 172.21.15.37:0/1125671959 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f450406cc30 0x7f450406f0e0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f4520071720 tx=0x7f450c009ce0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:33:50.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:50.708+0000 7f4517400700 1 -- 172.21.15.37:0/1125671959 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f4510057340 con 0x7f4520102e00 2024-03-29T08:33:50.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:33:50 smithi037 ceph-mon[142325]: from='client.35262 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:33:50.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:33:50 smithi037 ceph-mon[142325]: pgmap v689: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:33:50.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:33:50 smithi037 ceph-mon[142325]: from='client.35266 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:33:50.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:33:50 smithi037 ceph-mon[142325]: from='client.35270 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:33:50.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:33:50 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/527714671' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:33:50.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:50.940+0000 7f4526200700 1 -- 172.21.15.37:0/1125671959 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4520109dc0 con 0x7f450406cc30 2024-03-29T08:33:50.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:50.941+0000 7f4517400700 1 -- 172.21.15.37:0/1125671959 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f4520109dc0 con 0x7f450406cc30 2024-03-29T08:33:50.941 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-29T08:33:50.942 INFO:teuthology.orchestra.run.smithi037.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59", 2024-03-29T08:33:50.942 INFO:teuthology.orchestra.run.smithi037.stdout: "in_progress": true, 2024-03-29T08:33:50.942 INFO:teuthology.orchestra.run.smithi037.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-29T08:33:50.942 INFO:teuthology.orchestra.run.smithi037.stdout: "services_complete": [ 2024-03-29T08:33:50.942 INFO:teuthology.orchestra.run.smithi037.stdout: "crash", 2024-03-29T08:33:50.942 INFO:teuthology.orchestra.run.smithi037.stdout: "osd", 2024-03-29T08:33:50.942 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr", 2024-03-29T08:33:50.942 INFO:teuthology.orchestra.run.smithi037.stdout: "mds", 2024-03-29T08:33:50.942 INFO:teuthology.orchestra.run.smithi037.stdout: "mon" 2024-03-29T08:33:50.942 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-29T08:33:50.942 INFO:teuthology.orchestra.run.smithi037.stdout: "progress": "16/21 daemons upgraded", 2024-03-29T08:33:50.942 INFO:teuthology.orchestra.run.smithi037.stdout: "message": "Currently upgrading mds daemons", 2024-03-29T08:33:50.943 INFO:teuthology.orchestra.run.smithi037.stdout: "is_paused": false 2024-03-29T08:33:50.943 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-29T08:33:50.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:50.944+0000 7f4526200700 1 -- 172.21.15.37:0/1125671959 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f450406cc30 msgr2=0x7f450406f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:33:50.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:50.944+0000 7f4526200700 1 --2- 172.21.15.37:0/1125671959 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f450406cc30 0x7f450406f0e0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f4520071720 tx=0x7f450c009ce0 comp rx=0 tx=0).stop 2024-03-29T08:33:50.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:50.944+0000 7f4526200700 1 -- 172.21.15.37:0/1125671959 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4520102e00 msgr2=0x7f4520115c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:33:50.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:50.944+0000 7f4526200700 1 --2- 172.21.15.37:0/1125671959 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4520102e00 0x7f4520115c40 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f4510009360 tx=0x7f4510009e30 comp rx=0 tx=0).stop 2024-03-29T08:33:50.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:50.944+0000 7f4526200700 1 -- 172.21.15.37:0/1125671959 shutdown_connections 2024-03-29T08:33:50.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:50.944+0000 7f4526200700 1 --2- 172.21.15.37:0/1125671959 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f450406cc30 0x7f450406f0e0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:33:50.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:50.944+0000 7f4526200700 1 --2- 172.21.15.37:0/1125671959 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4520102e00 0x7f4520115c40 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:33:50.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:50.944+0000 7f4526200700 1 --2- 172.21.15.37:0/1125671959 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f45201037a0 0x7f4520116180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:33:50.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:50.944+0000 7f4526200700 1 -- 172.21.15.37:0/1125671959 >> 172.21.15.37:0/1125671959 conn(0x7f45200f89b0 msgr2=0x7f45200706d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:33:50.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:50.944+0000 7f4526200700 1 -- 172.21.15.37:0/1125671959 shutdown_connections 2024-03-29T08:33:50.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:50.944+0000 7f4526200700 1 -- 172.21.15.37:0/1125671959 wait complete. 2024-03-29T08:33:51.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:33:50 smithi148 ceph-mon[106692]: from='client.35262 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:33:51.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:33:50 smithi148 ceph-mon[106692]: pgmap v689: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:33:51.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:33:50 smithi148 ceph-mon[106692]: from='client.35266 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:33:51.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:33:50 smithi148 ceph-mon[106692]: from='client.35270 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:33:51.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:33:50 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/527714671' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:33:51.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:51.081+0000 7fe5f4c00700 1 -- 172.21.15.37:0/755318272 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe5f00fe830 msgr2=0x7fe5f0106f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:33:51.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:51.081+0000 7fe5f4c00700 1 --2- 172.21.15.37:0/755318272 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe5f00fe830 0x7fe5f0106f00 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fe5dc004940 tx=0x7fe5dc009e30 comp rx=0 tx=0).stop 2024-03-29T08:33:51.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:51.081+0000 7fe5f4c00700 1 -- 172.21.15.37:0/755318272 shutdown_connections 2024-03-29T08:33:51.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:51.081+0000 7fe5f4c00700 1 --2- 172.21.15.37:0/755318272 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe5f00fe830 0x7fe5f0106f00 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:33:51.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:51.081+0000 7fe5f4c00700 1 --2- 172.21.15.37:0/755318272 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe5f00fde90 0x7fe5f00fe260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:33:51.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:51.081+0000 7fe5f4c00700 1 -- 172.21.15.37:0/755318272 >> 172.21.15.37:0/755318272 conn(0x7fe5f0072bc0 msgr2=0x7fe5f0072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:33:51.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:51.081+0000 7fe5f4c00700 1 -- 172.21.15.37:0/755318272 shutdown_connections 2024-03-29T08:33:51.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:51.081+0000 7fe5f4c00700 1 -- 172.21.15.37:0/755318272 wait complete. 2024-03-29T08:33:51.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:51.082+0000 7fe5f4c00700 1 Processor -- start 2024-03-29T08:33:51.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:51.082+0000 7fe5f4c00700 1 -- start start 2024-03-29T08:33:51.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:51.082+0000 7fe5f4c00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe5f00fde90 0x7fe5f0115d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:33:51.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:51.082+0000 7fe5f4c00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe5f00fe830 0x7fe5f0116270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:33:51.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:51.082+0000 7fe5f4c00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5f01168c0 con 0x7fe5f00fe830 2024-03-29T08:33:51.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:51.082+0000 7fe5f4c00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5f010fc90 con 0x7fe5f00fde90 2024-03-29T08:33:51.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:51.083+0000 7fe5ed400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe5f00fe830 0x7fe5f0116270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:33:51.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:51.083+0000 7fe5ed400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe5f00fe830 0x7fe5f0116270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45688/0 (socket says 172.21.15.37:45688) 2024-03-29T08:33:51.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:51.083+0000 7fe5ed400700 1 -- 172.21.15.37:0/3690026583 learned_addr learned my addr 172.21.15.37:0/3690026583 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:33:51.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:51.083+0000 7fe5ede00700 1 --2- 172.21.15.37:0/3690026583 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe5f00fde90 0x7fe5f0115d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:33:51.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:51.083+0000 7fe5ed400700 1 -- 172.21.15.37:0/3690026583 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe5f00fde90 msgr2=0x7fe5f0115d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:33:51.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:51.083+0000 7fe5ed400700 1 --2- 172.21.15.37:0/3690026583 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe5f00fde90 0x7fe5f0115d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:33:51.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:51.083+0000 7fe5ed400700 1 -- 172.21.15.37:0/3690026583 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5dc005040 con 0x7fe5f00fe830 2024-03-29T08:33:51.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:51.083+0000 7fe5ed400700 1 --2- 172.21.15.37:0/3690026583 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe5f00fe830 0x7fe5f0116270 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fe5dc0093b0 tx=0x7fe5dc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:33:51.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:51.084+0000 7fe5e6000700 1 -- 172.21.15.37:0/3690026583 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe5dc00e040 con 0x7fe5f00fe830 2024-03-29T08:33:51.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:51.084+0000 7fe5e6000700 1 -- 172.21.15.37:0/3690026583 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fe5dc004290 con 0x7fe5f00fe830 2024-03-29T08:33:51.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:51.084+0000 7fe5f4c00700 1 -- 172.21.15.37:0/3690026583 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5f010ff10 con 0x7fe5f00fe830 2024-03-29T08:33:51.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:51.084+0000 7fe5e6000700 1 -- 172.21.15.37:0/3690026583 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe5dc01c3f0 con 0x7fe5f00fe830 2024-03-29T08:33:51.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:51.084+0000 7fe5f4c00700 1 -- 172.21.15.37:0/3690026583 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5f0110450 con 0x7fe5f00fe830 2024-03-29T08:33:51.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:51.085+0000 7fe5f4c00700 1 -- 172.21.15.37:0/3690026583 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5f0106a20 con 0x7fe5f00fe830 2024-03-29T08:33:51.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:51.086+0000 7fe5e6000700 1 -- 172.21.15.37:0/3690026583 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fe5dc013790 con 0x7fe5f00fe830 2024-03-29T08:33:51.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:51.087+0000 7fe5e6000700 1 --2- 172.21.15.37:0/3690026583 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fe5d406cc30 0x7fe5d406f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:33:51.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:51.087+0000 7fe5e6000700 1 -- 172.21.15.37:0/3690026583 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fe5dc08d380 con 0x7fe5f00fe830 2024-03-29T08:33:51.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:51.087+0000 7fe5ede00700 1 --2- 172.21.15.37:0/3690026583 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fe5d406cc30 0x7fe5d406f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:33:51.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:51.088+0000 7fe5ede00700 1 --2- 172.21.15.37:0/3690026583 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fe5d406cc30 0x7fe5d406f0e0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fe5e0007fd0 tx=0x7fe5e0009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:33:51.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:51.091+0000 7fe5e6000700 1 -- 172.21.15.37:0/3690026583 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fe5dc05add0 con 0x7fe5f00fe830 2024-03-29T08:33:51.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:51.389+0000 7fe5f4c00700 1 -- 172.21.15.37:0/3690026583 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fe5f0111380 con 0x7fe5f00fe830 2024-03-29T08:33:51.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:51.389+0000 7fe5e6000700 1 -- 172.21.15.37:0/3690026583 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+3583 (secure 0 0 0) 0x7fe5dc013420 con 0x7fe5f00fe830 2024-03-29T08:33:51.389 INFO:teuthology.orchestra.run.smithi037.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem is offline 2024-03-29T08:33:51.389 INFO:teuthology.orchestra.run.smithi037.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-03-29T08:33:51.389 INFO:teuthology.orchestra.run.smithi037.stdout: host smithi148 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/mon.smithi148/config 2024-03-29T08:33:51.389 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/podman: timeout after 895 seconds 2024-03-29T08:33:51.389 INFO:teuthology.orchestra.run.smithi037.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59 -e NODE_NAME=smithi148 -e CEPH_USE_RANDOM_NONCE=1 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4:/var/run/ceph:z -v /var/log/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4:/var/log/ceph:z -v /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpkx2rz2d4:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59 inventory --format=json-pretty --filter-for-batch 2024-03-29T08:33:51.389 INFO:teuthology.orchestra.run.smithi037.stdout:Traceback (most recent call last): 2024-03-29T08:33:51.389 INFO:teuthology.orchestra.run.smithi037.stdout: File "/usr/lib64/python3.6/runpy.py", line 193, in _run_module_as_main 2024-03-29T08:33:51.389 INFO:teuthology.orchestra.run.smithi037.stdout: "__main__", mod_spec) 2024-03-29T08:33:51.390 INFO:teuthology.orchestra.run.smithi037.stdout: File "/usr/lib64/python3.6/runpy.py", line 85, in _run_code 2024-03-29T08:33:51.390 INFO:teuthology.orchestra.run.smithi037.stdout: exec(code, run_globals) 2024-03-29T08:33:51.390 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 10889, in 2024-03-29T08:33:51.390 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 10877, in main 2024-03-29T08:33:51.390 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 2576, in _infer_config 2024-03-29T08:33:51.390 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 2492, in _infer_fsid 2024-03-29T08:33:51.390 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 2604, in _infer_image 2024-03-29T08:33:51.390 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 2479, in _validate_fsid 2024-03-29T08:33:51.390 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 7145, in command_ceph_volume 2024-03-29T08:33:51.390 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 2267, in call_throws 2024-03-29T08:33:51.390 INFO:teuthology.orchestra.run.smithi037.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59 -e NODE_NAME=smithi148 -e CEPH_USE_RANDOM_NONCE=1 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4:/var/run/ceph:z -v /var/log/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4:/var/log/ceph:z -v /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpkx2rz2d4:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59 inventory --format=json-pretty --filter-for-batch 2024-03-29T08:33:51.390 INFO:teuthology.orchestra.run.smithi037.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-03-29T08:33:51.390 INFO:teuthology.orchestra.run.smithi037.stdout: fs cephfs is degraded 2024-03-29T08:33:51.390 INFO:teuthology.orchestra.run.smithi037.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-03-29T08:33:51.390 INFO:teuthology.orchestra.run.smithi037.stdout: fs cephfs is offline because no MDS is active for it. 2024-03-29T08:33:51.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:51.391+0000 7fe5f4c00700 1 -- 172.21.15.37:0/3690026583 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fe5d406cc30 msgr2=0x7fe5d406f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:33:51.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:51.391+0000 7fe5f4c00700 1 --2- 172.21.15.37:0/3690026583 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fe5d406cc30 0x7fe5d406f0e0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fe5e0007fd0 tx=0x7fe5e0009040 comp rx=0 tx=0).stop 2024-03-29T08:33:51.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:51.391+0000 7fe5f4c00700 1 -- 172.21.15.37:0/3690026583 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe5f00fe830 msgr2=0x7fe5f0116270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:33:51.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:51.391+0000 7fe5f4c00700 1 --2- 172.21.15.37:0/3690026583 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe5f00fe830 0x7fe5f0116270 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fe5dc0093b0 tx=0x7fe5dc009e30 comp rx=0 tx=0).stop 2024-03-29T08:33:51.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:51.391+0000 7fe5f4c00700 1 -- 172.21.15.37:0/3690026583 shutdown_connections 2024-03-29T08:33:51.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:51.391+0000 7fe5f4c00700 1 --2- 172.21.15.37:0/3690026583 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fe5d406cc30 0x7fe5d406f0e0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:33:51.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:51.391+0000 7fe5f4c00700 1 --2- 172.21.15.37:0/3690026583 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe5f00fde90 0x7fe5f0115d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:33:51.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:51.391+0000 7fe5f4c00700 1 --2- 172.21.15.37:0/3690026583 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe5f00fe830 0x7fe5f0116270 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:33:51.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:51.391+0000 7fe5f4c00700 1 -- 172.21.15.37:0/3690026583 >> 172.21.15.37:0/3690026583 conn(0x7fe5f0072bc0 msgr2=0x7fe5f00fb840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:33:51.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:51.392+0000 7fe5f4c00700 1 -- 172.21.15.37:0/3690026583 shutdown_connections 2024-03-29T08:33:51.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:33:51.392+0000 7fe5f4c00700 1 -- 172.21.15.37:0/3690026583 wait complete. 2024-03-29T08:33:51.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:33:51 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/1346371256' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T08:33:51.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:33:51 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/3690026583' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T08:33:52.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:33:51 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/1346371256' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T08:33:52.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:33:51 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/3690026583' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T08:33:52.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:33:52 smithi037 ceph-mon[142325]: from='client.35282 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:33:52.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:33:52 smithi037 ceph-mon[142325]: pgmap v690: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:33:53.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:33:52 smithi148 ceph-mon[106692]: from='client.35282 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:33:53.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:33:52 smithi148 ceph-mon[106692]: pgmap v690: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:33:54.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:33:54 smithi037 ceph-mon[142325]: pgmap v691: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:33:55.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:33:54 smithi148 ceph-mon[106692]: pgmap v691: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:33:55.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:33:55 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:33:56.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:33:55 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:33:56.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:33:56 smithi037 ceph-mon[142325]: pgmap v692: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:33:57.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:33:56 smithi148 ceph-mon[106692]: pgmap v692: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:33:58.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:33:58 smithi037 ceph-mon[142325]: pgmap v693: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:33:59.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:33:58 smithi148 ceph-mon[106692]: pgmap v693: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:34:00.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:34:00 smithi037 ceph-mon[142325]: pgmap v694: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:34:01.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:34:00 smithi148 ceph-mon[106692]: pgmap v694: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:34:02.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:34:02 smithi037 ceph-mon[142325]: pgmap v695: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:34:03.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:34:02 smithi148 ceph-mon[106692]: pgmap v695: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:34:04.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:34:04 smithi037 ceph-mon[142325]: pgmap v696: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:34:05.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:34:04 smithi148 ceph-mon[106692]: pgmap v696: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:34:06.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:34:06 smithi148 ceph-mon[106692]: pgmap v697: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:34:06.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:34:06 smithi037 ceph-mon[142325]: pgmap v697: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:34:08.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:34:08 smithi037 ceph-mon[142325]: pgmap v698: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:34:09.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:34:08 smithi148 ceph-mon[106692]: pgmap v698: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:34:10.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:34:10 smithi037 ceph-mon[142325]: pgmap v699: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:34:10.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:34:10 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:34:10.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:34:10 smithi148 ceph-mon[106692]: pgmap v699: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:34:10.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:34:10 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:34:12.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:34:12 smithi037 ceph-mon[142325]: pgmap v700: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:34:12.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:34:12 smithi148 ceph-mon[106692]: pgmap v700: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:34:14.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:34:14 smithi037 ceph-mon[142325]: pgmap v701: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:34:14.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:34:14 smithi148 ceph-mon[106692]: pgmap v701: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:34:16.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:34:16 smithi037 ceph-mon[142325]: pgmap v702: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:34:16.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:34:16 smithi148 ceph-mon[106692]: pgmap v702: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:34:18.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:34:18 smithi037 ceph-mon[142325]: pgmap v703: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:34:18.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:34:18 smithi148 ceph-mon[106692]: pgmap v703: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:34:20.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:34:20 smithi037 ceph-mon[142325]: pgmap v704: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:34:20.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:34:20 smithi148 ceph-mon[106692]: pgmap v704: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:34:21.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:21.547+0000 7f3550200700 1 -- 172.21.15.37:0/4039031137 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3548102e00 msgr2=0x7f35481031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:34:21.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:21.547+0000 7f3550200700 1 --2- 172.21.15.37:0/4039031137 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3548102e00 0x7f35481031d0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f353c004940 tx=0x7f353c009e30 comp rx=0 tx=0).stop 2024-03-29T08:34:21.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:21.547+0000 7f3550200700 1 -- 172.21.15.37:0/4039031137 shutdown_connections 2024-03-29T08:34:21.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:21.547+0000 7f3550200700 1 --2- 172.21.15.37:0/4039031137 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f35481037a0 0x7f35480fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:34:21.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:21.547+0000 7f3550200700 1 --2- 172.21.15.37:0/4039031137 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3548102e00 0x7f35481031d0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:34:21.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:21.547+0000 7f3550200700 1 -- 172.21.15.37:0/4039031137 >> 172.21.15.37:0/4039031137 conn(0x7f35480f89b0 msgr2=0x7f35480fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:34:21.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:21.547+0000 7f3550200700 1 -- 172.21.15.37:0/4039031137 shutdown_connections 2024-03-29T08:34:21.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:21.547+0000 7f3550200700 1 -- 172.21.15.37:0/4039031137 wait complete. 2024-03-29T08:34:21.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:21.548+0000 7f3550200700 1 Processor -- start 2024-03-29T08:34:21.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:21.548+0000 7f3550200700 1 -- start start 2024-03-29T08:34:21.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:21.548+0000 7f3550200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3548102e00 0x7f3548115cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:34:21.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:21.548+0000 7f3550200700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f35481037a0 0x7f3548116230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:34:21.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:21.548+0000 7f3550200700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3548116880 con 0x7f3548102e00 2024-03-29T08:34:21.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:21.548+0000 7f3550200700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f354810fc50 con 0x7f35481037a0 2024-03-29T08:34:21.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:21.549+0000 7f354d800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3548102e00 0x7f3548115cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:34:21.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:21.549+0000 7f354d800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3548102e00 0x7f3548115cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39886/0 (socket says 172.21.15.37:39886) 2024-03-29T08:34:21.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:21.549+0000 7f354ce00700 1 --2- 172.21.15.37:0/1157431927 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f35481037a0 0x7f3548116230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:34:21.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:21.549+0000 7f354d800700 1 -- 172.21.15.37:0/1157431927 learned_addr learned my addr 172.21.15.37:0/1157431927 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:34:21.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:21.549+0000 7f354d800700 1 -- 172.21.15.37:0/1157431927 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f35481037a0 msgr2=0x7f3548116230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:34:21.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:21.549+0000 7f354d800700 1 --2- 172.21.15.37:0/1157431927 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f35481037a0 0x7f3548116230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:34:21.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:21.549+0000 7f354d800700 1 -- 172.21.15.37:0/1157431927 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f353c005040 con 0x7f3548102e00 2024-03-29T08:34:21.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:21.550+0000 7f354d800700 1 --2- 172.21.15.37:0/1157431927 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3548102e00 0x7f3548115cf0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f353c009360 tx=0x7f353c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:34:21.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:21.550+0000 7f3545600700 1 -- 172.21.15.37:0/1157431927 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f353c00e040 con 0x7f3548102e00 2024-03-29T08:34:21.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:21.550+0000 7f3545600700 1 -- 172.21.15.37:0/1157431927 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f353c013420 con 0x7f3548102e00 2024-03-29T08:34:21.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:21.550+0000 7f3550200700 1 -- 172.21.15.37:0/1157431927 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f354810fed0 con 0x7f3548102e00 2024-03-29T08:34:21.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:21.550+0000 7f3545600700 1 -- 172.21.15.37:0/1157431927 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f353c01c5d0 con 0x7f3548102e00 2024-03-29T08:34:21.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:21.550+0000 7f3550200700 1 -- 172.21.15.37:0/1157431927 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3548110490 con 0x7f3548102e00 2024-03-29T08:34:21.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:21.551+0000 7f3550200700 1 -- 172.21.15.37:0/1157431927 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35480fe550 con 0x7f3548102e00 2024-03-29T08:34:21.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:21.552+0000 7f3545600700 1 -- 172.21.15.37:0/1157431927 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f353c013590 con 0x7f3548102e00 2024-03-29T08:34:21.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:21.553+0000 7f3545600700 1 --2- 172.21.15.37:0/1157431927 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f353006cc30 0x7f353006f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:34:21.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:21.553+0000 7f3545600700 1 -- 172.21.15.37:0/1157431927 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f353c08d1f0 con 0x7f3548102e00 2024-03-29T08:34:21.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:21.554+0000 7f354ce00700 1 --2- 172.21.15.37:0/1157431927 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f353006cc30 0x7f353006f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:34:21.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:21.554+0000 7f354ce00700 1 --2- 172.21.15.37:0/1157431927 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f353006cc30 0x7f353006f0e0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f3548111590 tx=0x7f353400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:34:21.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:21.558+0000 7f3545600700 1 -- 172.21.15.37:0/1157431927 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f353c05ac40 con 0x7f3548102e00 2024-03-29T08:34:21.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:21.792+0000 7f3550200700 1 -- 172.21.15.37:0/1157431927 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f354804bed0 con 0x7f353006cc30 2024-03-29T08:34:21.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:21.794+0000 7f3545600700 1 -- 172.21.15.37:0/1157431927 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f354804bed0 con 0x7f353006cc30 2024-03-29T08:34:21.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:21.796+0000 7f3550200700 1 -- 172.21.15.37:0/1157431927 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f353006cc30 msgr2=0x7f353006f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:34:21.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:21.796+0000 7f3550200700 1 --2- 172.21.15.37:0/1157431927 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f353006cc30 0x7f353006f0e0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f3548111590 tx=0x7f353400b040 comp rx=0 tx=0).stop 2024-03-29T08:34:21.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:21.796+0000 7f3550200700 1 -- 172.21.15.37:0/1157431927 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3548102e00 msgr2=0x7f3548115cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:34:21.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:21.796+0000 7f3550200700 1 --2- 172.21.15.37:0/1157431927 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3548102e00 0x7f3548115cf0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f353c009360 tx=0x7f353c009e30 comp rx=0 tx=0).stop 2024-03-29T08:34:21.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:21.796+0000 7f3550200700 1 -- 172.21.15.37:0/1157431927 shutdown_connections 2024-03-29T08:34:21.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:21.796+0000 7f3550200700 1 --2- 172.21.15.37:0/1157431927 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f353006cc30 0x7f353006f0e0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:34:21.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:21.796+0000 7f3550200700 1 --2- 172.21.15.37:0/1157431927 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3548102e00 0x7f3548115cf0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:34:21.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:21.796+0000 7f3550200700 1 --2- 172.21.15.37:0/1157431927 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f35481037a0 0x7f3548116230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:34:21.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:21.796+0000 7f3550200700 1 -- 172.21.15.37:0/1157431927 >> 172.21.15.37:0/1157431927 conn(0x7f35480f89b0 msgr2=0x7f35480706d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:34:21.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:21.796+0000 7f3550200700 1 -- 172.21.15.37:0/1157431927 shutdown_connections 2024-03-29T08:34:21.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:21.796+0000 7f3550200700 1 -- 172.21.15.37:0/1157431927 wait complete. 2024-03-29T08:34:21.813 INFO:teuthology.orchestra.run.smithi037.stdout:true 2024-03-29T08:34:21.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:21.935+0000 7fcdbe200700 1 -- 172.21.15.37:0/2255268514 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcdb8102e00 msgr2=0x7fcdb81031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:34:21.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:21.935+0000 7fcdbe200700 1 --2- 172.21.15.37:0/2255268514 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcdb8102e00 0x7fcdb81031d0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fcda8004940 tx=0x7fcda8009e30 comp rx=0 tx=0).stop 2024-03-29T08:34:21.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:21.935+0000 7fcdbe200700 1 -- 172.21.15.37:0/2255268514 shutdown_connections 2024-03-29T08:34:21.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:21.935+0000 7fcdbe200700 1 --2- 172.21.15.37:0/2255268514 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcdb81037a0 0x7fcdb80fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:34:21.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:21.935+0000 7fcdbe200700 1 --2- 172.21.15.37:0/2255268514 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcdb8102e00 0x7fcdb81031d0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:34:21.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:21.935+0000 7fcdbe200700 1 -- 172.21.15.37:0/2255268514 >> 172.21.15.37:0/2255268514 conn(0x7fcdb80f89b0 msgr2=0x7fcdb80fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:34:21.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:21.935+0000 7fcdbe200700 1 -- 172.21.15.37:0/2255268514 shutdown_connections 2024-03-29T08:34:21.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:21.935+0000 7fcdbe200700 1 -- 172.21.15.37:0/2255268514 wait complete. 2024-03-29T08:34:21.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:21.936+0000 7fcdbe200700 1 Processor -- start 2024-03-29T08:34:21.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:21.936+0000 7fcdbe200700 1 -- start start 2024-03-29T08:34:21.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:21.937+0000 7fcdbe200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcdb8102e00 0x7fcdb8115d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:34:21.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:21.937+0000 7fcdbe200700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcdb81037a0 0x7fcdb8116250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:34:21.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:21.937+0000 7fcdbe200700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdb81168a0 con 0x7fcdb8102e00 2024-03-29T08:34:21.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:21.937+0000 7fcdbe200700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdb810fc70 con 0x7fcdb81037a0 2024-03-29T08:34:21.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:21.937+0000 7fcdb7400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcdb8102e00 0x7fcdb8115d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:34:21.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:21.937+0000 7fcdb7400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcdb8102e00 0x7fcdb8115d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39912/0 (socket says 172.21.15.37:39912) 2024-03-29T08:34:21.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:21.937+0000 7fcdb7400700 1 -- 172.21.15.37:0/1291132849 learned_addr learned my addr 172.21.15.37:0/1291132849 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:34:21.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:21.937+0000 7fcdb6a00700 1 --2- 172.21.15.37:0/1291132849 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcdb81037a0 0x7fcdb8116250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:34:21.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:21.937+0000 7fcdb7400700 1 -- 172.21.15.37:0/1291132849 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcdb81037a0 msgr2=0x7fcdb8116250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:34:21.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:21.937+0000 7fcdb7400700 1 --2- 172.21.15.37:0/1291132849 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcdb81037a0 0x7fcdb8116250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:34:21.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:21.937+0000 7fcdb7400700 1 -- 172.21.15.37:0/1291132849 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcda8005040 con 0x7fcdb8102e00 2024-03-29T08:34:21.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:21.938+0000 7fcdb7400700 1 --2- 172.21.15.37:0/1291132849 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcdb8102e00 0x7fcdb8115d10 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fcda8009390 tx=0x7fcda8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:34:21.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:21.938+0000 7fcdaf400700 1 -- 172.21.15.37:0/1291132849 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcda800e040 con 0x7fcdb8102e00 2024-03-29T08:34:21.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:21.938+0000 7fcdaf400700 1 -- 172.21.15.37:0/1291132849 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fcda8013420 con 0x7fcdb8102e00 2024-03-29T08:34:21.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:21.938+0000 7fcdbe200700 1 -- 172.21.15.37:0/1291132849 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdb810fef0 con 0x7fcdb8102e00 2024-03-29T08:34:21.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:21.938+0000 7fcdaf400700 1 -- 172.21.15.37:0/1291132849 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcda801c630 con 0x7fcdb8102e00 2024-03-29T08:34:21.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:21.938+0000 7fcdbe200700 1 -- 172.21.15.37:0/1291132849 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdb81104b0 con 0x7fcdb8102e00 2024-03-29T08:34:21.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:21.939+0000 7fcdbe200700 1 -- 172.21.15.37:0/1291132849 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcdb80fe550 con 0x7fcdb8102e00 2024-03-29T08:34:21.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:21.940+0000 7fcdaf400700 1 -- 172.21.15.37:0/1291132849 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fcda80139e0 con 0x7fcdb8102e00 2024-03-29T08:34:21.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:21.941+0000 7fcdaf400700 1 --2- 172.21.15.37:0/1291132849 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fcd9c06cc30 0x7fcd9c06f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:34:21.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:21.941+0000 7fcdaf400700 1 -- 172.21.15.37:0/1291132849 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fcda808d250 con 0x7fcdb8102e00 2024-03-29T08:34:21.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:21.941+0000 7fcdb6a00700 1 --2- 172.21.15.37:0/1291132849 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fcd9c06cc30 0x7fcd9c06f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:34:21.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:21.942+0000 7fcdb6a00700 1 --2- 172.21.15.37:0/1291132849 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fcd9c06cc30 0x7fcd9c06f0e0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fcdb81115b0 tx=0x7fcda4009ee0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:34:21.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:21.945+0000 7fcdaf400700 1 -- 172.21.15.37:0/1291132849 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fcda805add0 con 0x7fcdb8102e00 2024-03-29T08:34:22.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:22.177+0000 7fcdbe200700 1 -- 172.21.15.37:0/1291132849 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fcdb804bf40 con 0x7fcd9c06cc30 2024-03-29T08:34:22.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:22.178+0000 7fcdaf400700 1 -- 172.21.15.37:0/1291132849 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fcdb804bf40 con 0x7fcd9c06cc30 2024-03-29T08:34:22.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:22.181+0000 7fcdbe200700 1 -- 172.21.15.37:0/1291132849 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fcd9c06cc30 msgr2=0x7fcd9c06f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:34:22.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:22.181+0000 7fcdbe200700 1 --2- 172.21.15.37:0/1291132849 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fcd9c06cc30 0x7fcd9c06f0e0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fcdb81115b0 tx=0x7fcda4009ee0 comp rx=0 tx=0).stop 2024-03-29T08:34:22.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:22.181+0000 7fcdbe200700 1 -- 172.21.15.37:0/1291132849 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcdb8102e00 msgr2=0x7fcdb8115d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:34:22.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:22.181+0000 7fcdbe200700 1 --2- 172.21.15.37:0/1291132849 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcdb8102e00 0x7fcdb8115d10 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fcda8009390 tx=0x7fcda8009e30 comp rx=0 tx=0).stop 2024-03-29T08:34:22.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:22.181+0000 7fcdbe200700 1 -- 172.21.15.37:0/1291132849 shutdown_connections 2024-03-29T08:34:22.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:22.181+0000 7fcdbe200700 1 --2- 172.21.15.37:0/1291132849 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fcd9c06cc30 0x7fcd9c06f0e0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:34:22.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:22.181+0000 7fcdbe200700 1 --2- 172.21.15.37:0/1291132849 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcdb8102e00 0x7fcdb8115d10 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:34:22.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:22.181+0000 7fcdbe200700 1 --2- 172.21.15.37:0/1291132849 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcdb81037a0 0x7fcdb8116250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:34:22.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:22.181+0000 7fcdbe200700 1 -- 172.21.15.37:0/1291132849 >> 172.21.15.37:0/1291132849 conn(0x7fcdb80f89b0 msgr2=0x7fcdb80704f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:34:22.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:22.181+0000 7fcdbe200700 1 -- 172.21.15.37:0/1291132849 shutdown_connections 2024-03-29T08:34:22.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:22.181+0000 7fcdbe200700 1 -- 172.21.15.37:0/1291132849 wait complete. 2024-03-29T08:34:22.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:22.319+0000 7f9ad5a00700 1 -- 172.21.15.37:0/2646210589 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9ad00fde90 msgr2=0x7f9ad00fe260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:34:22.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:22.319+0000 7f9ad5a00700 1 --2- 172.21.15.37:0/2646210589 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9ad00fde90 0x7f9ad00fe260 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f9ac0004940 tx=0x7f9ac0009e30 comp rx=0 tx=0).stop 2024-03-29T08:34:22.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:22.319+0000 7f9ad5a00700 1 -- 172.21.15.37:0/2646210589 shutdown_connections 2024-03-29T08:34:22.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:22.319+0000 7f9ad5a00700 1 --2- 172.21.15.37:0/2646210589 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9ad00fe830 0x7f9ad0106f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:34:22.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:22.319+0000 7f9ad5a00700 1 --2- 172.21.15.37:0/2646210589 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9ad00fde90 0x7f9ad00fe260 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:34:22.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:22.319+0000 7f9ad5a00700 1 -- 172.21.15.37:0/2646210589 >> 172.21.15.37:0/2646210589 conn(0x7f9ad0072bc0 msgr2=0x7f9ad0072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:34:22.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:22.319+0000 7f9ad5a00700 1 -- 172.21.15.37:0/2646210589 shutdown_connections 2024-03-29T08:34:22.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:22.319+0000 7f9ad5a00700 1 -- 172.21.15.37:0/2646210589 wait complete. 2024-03-29T08:34:22.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:22.320+0000 7f9ad5a00700 1 Processor -- start 2024-03-29T08:34:22.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:22.320+0000 7f9ad5a00700 1 -- start start 2024-03-29T08:34:22.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:22.321+0000 7f9ad5a00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9ad00fde90 0x7f9ad0115c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:34:22.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:22.321+0000 7f9ad5a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9ad00fe830 0x7f9ad01161c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:34:22.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:22.321+0000 7f9ad5a00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ad0116810 con 0x7f9ad00fe830 2024-03-29T08:34:22.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:22.321+0000 7f9ad5a00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ad010fbe0 con 0x7f9ad00fde90 2024-03-29T08:34:22.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:22.321+0000 7f9ace000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9ad00fe830 0x7f9ad01161c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:34:22.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:22.321+0000 7f9ace000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9ad00fe830 0x7f9ad01161c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39932/0 (socket says 172.21.15.37:39932) 2024-03-29T08:34:22.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:22.321+0000 7f9ace000700 1 -- 172.21.15.37:0/1352892766 learned_addr learned my addr 172.21.15.37:0/1352892766 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:34:22.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:22.321+0000 7f9acea00700 1 --2- 172.21.15.37:0/1352892766 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9ad00fde90 0x7f9ad0115c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:34:22.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:22.321+0000 7f9ace000700 1 -- 172.21.15.37:0/1352892766 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9ad00fde90 msgr2=0x7f9ad0115c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:34:22.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:22.321+0000 7f9ace000700 1 --2- 172.21.15.37:0/1352892766 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9ad00fde90 0x7f9ad0115c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:34:22.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:22.321+0000 7f9ace000700 1 -- 172.21.15.37:0/1352892766 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ac0005040 con 0x7f9ad00fe830 2024-03-29T08:34:22.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:22.322+0000 7f9ace000700 1 --2- 172.21.15.37:0/1352892766 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9ad00fe830 0x7f9ad01161c0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f9abc002a80 tx=0x7f9abc0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:34:22.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:22.322+0000 7f9ac6a00700 1 -- 172.21.15.37:0/1352892766 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9abc007cc0 con 0x7f9ad00fe830 2024-03-29T08:34:22.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:22.322+0000 7f9ac6a00700 1 -- 172.21.15.37:0/1352892766 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f9abc00f040 con 0x7f9ad00fe830 2024-03-29T08:34:22.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:22.322+0000 7f9ac6a00700 1 -- 172.21.15.37:0/1352892766 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9abc009640 con 0x7f9ad00fe830 2024-03-29T08:34:22.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:22.322+0000 7f9ad5a00700 1 -- 172.21.15.37:0/1352892766 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ad010fec0 con 0x7f9ad00fe830 2024-03-29T08:34:22.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:22.322+0000 7f9ad5a00700 1 -- 172.21.15.37:0/1352892766 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ad0110400 con 0x7f9ad00fe830 2024-03-29T08:34:22.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:22.323+0000 7f9ad5a00700 1 -- 172.21.15.37:0/1352892766 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ad0106a20 con 0x7f9ad00fe830 2024-03-29T08:34:22.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:22.324+0000 7f9ac6a00700 1 -- 172.21.15.37:0/1352892766 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f9abc0097a0 con 0x7f9ad00fe830 2024-03-29T08:34:22.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:22.325+0000 7f9ac6a00700 1 --2- 172.21.15.37:0/1352892766 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f9ab406cc30 0x7f9ab406f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:34:22.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:22.325+0000 7f9ac6a00700 1 -- 172.21.15.37:0/1352892766 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f9abc08b7e0 con 0x7f9ad00fe830 2024-03-29T08:34:22.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:22.325+0000 7f9acea00700 1 --2- 172.21.15.37:0/1352892766 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f9ab406cc30 0x7f9ab406f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:34:22.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:22.326+0000 7f9acea00700 1 --2- 172.21.15.37:0/1352892766 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f9ab406cc30 0x7f9ab406f0e0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f9ac000b3b0 tx=0x7f9ac0009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:34:22.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:22.329+0000 7f9ac6a00700 1 -- 172.21.15.37:0/1352892766 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f9abc059230 con 0x7f9ad00fe830 2024-03-29T08:34:22.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:22.552+0000 7f9ad5a00700 1 -- 172.21.15.37:0/1352892766 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f9ad0109dc0 con 0x7f9ab406cc30 2024-03-29T08:34:22.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:22.562+0000 7f9ac6a00700 1 -- 172.21.15.37:0/1352892766 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f9ad0109dc0 con 0x7f9ab406cc30 2024-03-29T08:34:22.562 INFO:teuthology.orchestra.run.smithi037.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-29T08:34:22.562 INFO:teuthology.orchestra.run.smithi037.stdout:alertmanager.smithi037 smithi037 *:9093,9094 running (24m) 57s ago 37m 14.3M - 0.25.0 c8568f914cd2 2f9f9746ed0f 2024-03-29T08:34:22.562 INFO:teuthology.orchestra.run.smithi037.stdout:crash.smithi037 smithi037 running (21m) 57s ago 37m 6647k - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 178d416a3186 2024-03-29T08:34:22.562 INFO:teuthology.orchestra.run.smithi037.stdout:crash.smithi148 smithi148 running (21m) 59s ago 36m 6656k - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 23666989d58f 2024-03-29T08:34:22.562 INFO:teuthology.orchestra.run.smithi037.stdout:grafana.smithi037 smithi037 *:3000 running (23m) 57s ago 36m 74.5M - 9.4.7 954c08fa6188 6b976fe49952 2024-03-29T08:34:22.562 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi037.iifsmh smithi037 running (16m) 57s ago 30m 16.4M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 2d97f562cc5b 2024-03-29T08:34:22.562 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi037.mmwgxe smithi037 running (16m) 57s ago 30m 16.7M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff ec37037f46e8 2024-03-29T08:34:22.562 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi148.qukpjh smithi148 running (63s) 59s ago 30m 10.7M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 3257219bc046 2024-03-29T08:34:22.562 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi148.sqteih smithi148 running (16m) 59s ago 30m 16.3M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 44c15c77037a 2024-03-29T08:34:22.562 INFO:teuthology.orchestra.run.smithi037.stdout:mgr.smithi037.wjikii smithi037 *:8443,9283,8765 running (26m) 57s ago 38m 510M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff c5a1417b9417 2024-03-29T08:34:22.562 INFO:teuthology.orchestra.run.smithi037.stdout:mgr.smithi148.onhgvm smithi148 *:8443,9283,8765 running (25m) 59s ago 36m 456M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff fcfdbce062bd 2024-03-29T08:34:22.563 INFO:teuthology.orchestra.run.smithi037.stdout:mon.smithi037 smithi037 running (22m) 57s ago 38m 70.2M 2048M 18.2.2-1015-gd0e392d9 1cc2f45c17ff eea2060783af 2024-03-29T08:34:22.563 INFO:teuthology.orchestra.run.smithi037.stdout:mon.smithi148 smithi148 running (21m) 59s ago 35m 50.4M 2048M 18.2.2-1015-gd0e392d9 1cc2f45c17ff ad4b2c5748be 2024-03-29T08:34:22.563 INFO:teuthology.orchestra.run.smithi037.stdout:node-exporter.smithi037 smithi037 *:9100 running (25m) 57s ago 37m 8514k - 1.5.0 0da6a335fe13 26b8ae9cb878 2024-03-29T08:34:22.563 INFO:teuthology.orchestra.run.smithi037.stdout:node-exporter.smithi148 smithi148 *:9100 running (24m) 59s ago 36m 8464k - 1.5.0 0da6a335fe13 98414e4e64ed 2024-03-29T08:34:22.563 INFO:teuthology.orchestra.run.smithi037.stdout:osd.0 smithi037 running (20m) 57s ago 35m 108M 1619M 18.2.2-1015-gd0e392d9 1cc2f45c17ff c0c584f6245c 2024-03-29T08:34:22.563 INFO:teuthology.orchestra.run.smithi037.stdout:osd.1 smithi037 running (19m) 57s ago 34m 104M 1619M 18.2.2-1015-gd0e392d9 1cc2f45c17ff a4a2ff4cc83d 2024-03-29T08:34:22.563 INFO:teuthology.orchestra.run.smithi037.stdout:osd.2 smithi037 running (19m) 57s ago 33m 85.9M 1619M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 8b1bc71aa79e 2024-03-29T08:34:22.563 INFO:teuthology.orchestra.run.smithi037.stdout:osd.3 smithi148 running (18m) 59s ago 33m 109M 2642M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 1c656ab8ff86 2024-03-29T08:34:22.563 INFO:teuthology.orchestra.run.smithi037.stdout:osd.4 smithi148 running (17m) 59s ago 32m 101M 2642M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 966a9de11d22 2024-03-29T08:34:22.563 INFO:teuthology.orchestra.run.smithi037.stdout:osd.5 smithi148 running (17m) 59s ago 32m 89.4M 2642M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 474c4f9212ce 2024-03-29T08:34:22.563 INFO:teuthology.orchestra.run.smithi037.stdout:prometheus.smithi037 smithi037 *:9095 running (24m) 57s ago 36m 61.0M - 2.43.0 a07b618ecd1d 30c0223ec589 2024-03-29T08:34:22.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:22.564+0000 7f9ad5a00700 1 -- 172.21.15.37:0/1352892766 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f9ab406cc30 msgr2=0x7f9ab406f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:34:22.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:22.564+0000 7f9ad5a00700 1 --2- 172.21.15.37:0/1352892766 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f9ab406cc30 0x7f9ab406f0e0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f9ac000b3b0 tx=0x7f9ac0009e30 comp rx=0 tx=0).stop 2024-03-29T08:34:22.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:22.564+0000 7f9ad5a00700 1 -- 172.21.15.37:0/1352892766 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9ad00fe830 msgr2=0x7f9ad01161c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:34:22.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:22.564+0000 7f9ad5a00700 1 --2- 172.21.15.37:0/1352892766 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9ad00fe830 0x7f9ad01161c0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f9abc002a80 tx=0x7f9abc0046b0 comp rx=0 tx=0).stop 2024-03-29T08:34:22.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:22.565+0000 7f9ad5a00700 1 -- 172.21.15.37:0/1352892766 shutdown_connections 2024-03-29T08:34:22.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:22.565+0000 7f9ad5a00700 1 --2- 172.21.15.37:0/1352892766 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f9ab406cc30 0x7f9ab406f0e0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:34:22.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:22.565+0000 7f9ad5a00700 1 --2- 172.21.15.37:0/1352892766 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9ad00fde90 0x7f9ad0115c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:34:22.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:22.565+0000 7f9ad5a00700 1 --2- 172.21.15.37:0/1352892766 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9ad00fe830 0x7f9ad01161c0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:34:22.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:22.565+0000 7f9ad5a00700 1 -- 172.21.15.37:0/1352892766 >> 172.21.15.37:0/1352892766 conn(0x7f9ad0072bc0 msgr2=0x7f9ad00fb7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:34:22.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:22.565+0000 7f9ad5a00700 1 -- 172.21.15.37:0/1352892766 shutdown_connections 2024-03-29T08:34:22.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:22.565+0000 7f9ad5a00700 1 -- 172.21.15.37:0/1352892766 wait complete. 2024-03-29T08:34:22.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:34:22 smithi037 ceph-mon[142325]: pgmap v705: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:34:22.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:34:22 smithi037 ceph-mon[142325]: from='client.35290 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:34:22.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:22.701+0000 7ff44e200700 1 -- 172.21.15.37:0/4279684908 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff4481037a0 msgr2=0x7ff4480fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:34:22.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:22.701+0000 7ff44e200700 1 --2- 172.21.15.37:0/4279684908 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff4481037a0 0x7ff4480fce10 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7ff434004940 tx=0x7ff434009e30 comp rx=0 tx=0).stop 2024-03-29T08:34:22.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:22.701+0000 7ff44e200700 1 -- 172.21.15.37:0/4279684908 shutdown_connections 2024-03-29T08:34:22.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:22.701+0000 7ff44e200700 1 --2- 172.21.15.37:0/4279684908 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff4481037a0 0x7ff4480fce10 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:34:22.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:22.701+0000 7ff44e200700 1 --2- 172.21.15.37:0/4279684908 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff448102e00 0x7ff4481031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:34:22.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:22.701+0000 7ff44e200700 1 -- 172.21.15.37:0/4279684908 >> 172.21.15.37:0/4279684908 conn(0x7ff4480f89b0 msgr2=0x7ff4480fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:34:22.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:22.701+0000 7ff44e200700 1 -- 172.21.15.37:0/4279684908 shutdown_connections 2024-03-29T08:34:22.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:22.701+0000 7ff44e200700 1 -- 172.21.15.37:0/4279684908 wait complete. 2024-03-29T08:34:22.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:22.702+0000 7ff44e200700 1 Processor -- start 2024-03-29T08:34:22.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:22.702+0000 7ff44e200700 1 -- start start 2024-03-29T08:34:22.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:22.702+0000 7ff44e200700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff448102e00 0x7ff4480702c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:34:22.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:22.702+0000 7ff44e200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff4481037a0 0x7ff448070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:34:22.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:22.702+0000 7ff44e200700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff448070d90 con 0x7ff4481037a0 2024-03-29T08:34:22.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:22.702+0000 7ff44e200700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff448070ed0 con 0x7ff448102e00 2024-03-29T08:34:22.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:22.703+0000 7ff446a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff4481037a0 0x7ff448070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:34:22.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:22.703+0000 7ff446a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff4481037a0 0x7ff448070800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39962/0 (socket says 172.21.15.37:39962) 2024-03-29T08:34:22.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:22.703+0000 7ff446a00700 1 -- 172.21.15.37:0/3201688941 learned_addr learned my addr 172.21.15.37:0/3201688941 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:34:22.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:22.703+0000 7ff447400700 1 --2- 172.21.15.37:0/3201688941 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff448102e00 0x7ff4480702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:34:22.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:22.703+0000 7ff446a00700 1 -- 172.21.15.37:0/3201688941 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff448102e00 msgr2=0x7ff4480702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:34:22.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:22.703+0000 7ff446a00700 1 --2- 172.21.15.37:0/3201688941 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff448102e00 0x7ff4480702c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:34:22.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:22.703+0000 7ff446a00700 1 -- 172.21.15.37:0/3201688941 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff434005040 con 0x7ff4481037a0 2024-03-29T08:34:22.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:22.703+0000 7ff447400700 1 --2- 172.21.15.37:0/3201688941 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff448102e00 0x7ff4480702c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:34:22.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:22.703+0000 7ff446a00700 1 --2- 172.21.15.37:0/3201688941 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff4481037a0 0x7ff448070800 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7ff434009390 tx=0x7ff434009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:34:22.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:22.704+0000 7ff43f400700 1 -- 172.21.15.37:0/3201688941 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff43400e040 con 0x7ff4481037a0 2024-03-29T08:34:22.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:22.704+0000 7ff43f400700 1 -- 172.21.15.37:0/3201688941 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ff434013420 con 0x7ff4481037a0 2024-03-29T08:34:22.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:22.704+0000 7ff44e200700 1 -- 172.21.15.37:0/3201688941 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff448071120 con 0x7ff4481037a0 2024-03-29T08:34:22.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:22.704+0000 7ff43f400700 1 -- 172.21.15.37:0/3201688941 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff43401c630 con 0x7ff4481037a0 2024-03-29T08:34:22.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:22.704+0000 7ff44e200700 1 -- 172.21.15.37:0/3201688941 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff44811a230 con 0x7ff4481037a0 2024-03-29T08:34:22.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:22.705+0000 7ff44e200700 1 -- 172.21.15.37:0/3201688941 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4480fe550 con 0x7ff4481037a0 2024-03-29T08:34:22.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:22.706+0000 7ff43f400700 1 -- 172.21.15.37:0/3201688941 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7ff434013a00 con 0x7ff4481037a0 2024-03-29T08:34:22.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:22.707+0000 7ff43f400700 1 --2- 172.21.15.37:0/3201688941 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7ff42c06cc30 0x7ff42c06f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:34:22.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:22.707+0000 7ff43f400700 1 -- 172.21.15.37:0/3201688941 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7ff43408d2a0 con 0x7ff4481037a0 2024-03-29T08:34:22.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:22.707+0000 7ff447400700 1 --2- 172.21.15.37:0/3201688941 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7ff42c06cc30 0x7ff42c06f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:34:22.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:22.708+0000 7ff447400700 1 --2- 172.21.15.37:0/3201688941 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7ff42c06cc30 0x7ff42c06f0e0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7ff438009fd0 tx=0x7ff438009ee0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:34:22.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:22.711+0000 7ff43f400700 1 -- 172.21.15.37:0/3201688941 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7ff4340572e0 con 0x7ff4481037a0 2024-03-29T08:34:22.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:34:22 smithi148 ceph-mon[106692]: pgmap v705: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:34:22.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:34:22 smithi148 ceph-mon[106692]: from='client.35290 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:34:23.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:23.014+0000 7ff44e200700 1 -- 172.21.15.37:0/3201688941 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7ff448109dd0 con 0x7ff4481037a0 2024-03-29T08:34:23.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:23.014+0000 7ff43f400700 1 -- 172.21.15.37:0/3201688941 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7ff43405a900 con 0x7ff4481037a0 2024-03-29T08:34:23.014 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-29T08:34:23.014 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": { 2024-03-29T08:34:23.014 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 2 2024-03-29T08:34:23.014 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:34:23.014 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": { 2024-03-29T08:34:23.015 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 2 2024-03-29T08:34:23.015 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:34:23.015 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": { 2024-03-29T08:34:23.015 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 6 2024-03-29T08:34:23.015 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:34:23.015 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": { 2024-03-29T08:34:23.015 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 4 2024-03-29T08:34:23.015 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:34:23.015 INFO:teuthology.orchestra.run.smithi037.stdout: "overall": { 2024-03-29T08:34:23.015 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 14 2024-03-29T08:34:23.015 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-29T08:34:23.015 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-29T08:34:23.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:23.017+0000 7ff44e200700 1 -- 172.21.15.37:0/3201688941 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7ff42c06cc30 msgr2=0x7ff42c06f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:34:23.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:23.017+0000 7ff44e200700 1 --2- 172.21.15.37:0/3201688941 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7ff42c06cc30 0x7ff42c06f0e0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7ff438009fd0 tx=0x7ff438009ee0 comp rx=0 tx=0).stop 2024-03-29T08:34:23.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:23.017+0000 7ff44e200700 1 -- 172.21.15.37:0/3201688941 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff4481037a0 msgr2=0x7ff448070800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:34:23.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:23.017+0000 7ff44e200700 1 --2- 172.21.15.37:0/3201688941 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff4481037a0 0x7ff448070800 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7ff434009390 tx=0x7ff434009e30 comp rx=0 tx=0).stop 2024-03-29T08:34:23.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:23.017+0000 7ff44e200700 1 -- 172.21.15.37:0/3201688941 shutdown_connections 2024-03-29T08:34:23.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:23.017+0000 7ff44e200700 1 --2- 172.21.15.37:0/3201688941 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7ff42c06cc30 0x7ff42c06f0e0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:34:23.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:23.017+0000 7ff44e200700 1 --2- 172.21.15.37:0/3201688941 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff448102e00 0x7ff4480702c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:34:23.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:23.017+0000 7ff44e200700 1 --2- 172.21.15.37:0/3201688941 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff4481037a0 0x7ff448070800 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:34:23.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:23.017+0000 7ff44e200700 1 -- 172.21.15.37:0/3201688941 >> 172.21.15.37:0/3201688941 conn(0x7ff4480f89b0 msgr2=0x7ff4480f9ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:34:23.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:23.017+0000 7ff44e200700 1 -- 172.21.15.37:0/3201688941 shutdown_connections 2024-03-29T08:34:23.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:23.017+0000 7ff44e200700 1 -- 172.21.15.37:0/3201688941 wait complete. 2024-03-29T08:34:23.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:23.155+0000 7ff456800700 1 -- 172.21.15.37:0/2386639756 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff4500fe830 msgr2=0x7ff450106f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:34:23.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:23.155+0000 7ff456800700 1 --2- 172.21.15.37:0/2386639756 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff4500fe830 0x7ff450106f00 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7ff43c004940 tx=0x7ff43c009e30 comp rx=0 tx=0).stop 2024-03-29T08:34:23.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:23.155+0000 7ff456800700 1 -- 172.21.15.37:0/2386639756 shutdown_connections 2024-03-29T08:34:23.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:23.155+0000 7ff456800700 1 --2- 172.21.15.37:0/2386639756 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff4500fe830 0x7ff450106f00 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:34:23.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:23.155+0000 7ff456800700 1 --2- 172.21.15.37:0/2386639756 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff4500fde90 0x7ff4500fe260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:34:23.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:23.155+0000 7ff456800700 1 -- 172.21.15.37:0/2386639756 >> 172.21.15.37:0/2386639756 conn(0x7ff450072bc0 msgr2=0x7ff450072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:34:23.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:23.155+0000 7ff456800700 1 -- 172.21.15.37:0/2386639756 shutdown_connections 2024-03-29T08:34:23.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:23.155+0000 7ff456800700 1 -- 172.21.15.37:0/2386639756 wait complete. 2024-03-29T08:34:23.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:23.156+0000 7ff456800700 1 Processor -- start 2024-03-29T08:34:23.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:23.156+0000 7ff456800700 1 -- start start 2024-03-29T08:34:23.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:23.156+0000 7ff456800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff4500fde90 0x7ff450115c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:34:23.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:23.156+0000 7ff456800700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff4500fe830 0x7ff4501161c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:34:23.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:23.156+0000 7ff456800700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff450116810 con 0x7ff4500fde90 2024-03-29T08:34:23.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:23.156+0000 7ff456800700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff45010fc30 con 0x7ff4500fe830 2024-03-29T08:34:23.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:23.157+0000 7ff44f400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff4500fde90 0x7ff450115c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:34:23.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:23.157+0000 7ff44f400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff4500fde90 0x7ff450115c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39984/0 (socket says 172.21.15.37:39984) 2024-03-29T08:34:23.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:23.157+0000 7ff44f400700 1 -- 172.21.15.37:0/611270151 learned_addr learned my addr 172.21.15.37:0/611270151 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:34:23.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:23.157+0000 7ff44f400700 1 -- 172.21.15.37:0/611270151 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff4500fe830 msgr2=0x7ff4501161c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:34:23.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:23.157+0000 7ff44f400700 1 --2- 172.21.15.37:0/611270151 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff4500fe830 0x7ff4501161c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:34:23.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:23.157+0000 7ff44f400700 1 -- 172.21.15.37:0/611270151 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff43c005040 con 0x7ff4500fde90 2024-03-29T08:34:23.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:23.157+0000 7ff44f400700 1 --2- 172.21.15.37:0/611270151 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff4500fde90 0x7ff450115c80 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7ff440002a80 tx=0x7ff4400046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:34:23.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:23.158+0000 7ff447400700 1 -- 172.21.15.37:0/611270151 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff440007cc0 con 0x7ff4500fde90 2024-03-29T08:34:23.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:23.158+0000 7ff447400700 1 -- 172.21.15.37:0/611270151 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ff44000f040 con 0x7ff4500fde90 2024-03-29T08:34:23.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:23.158+0000 7ff456800700 1 -- 172.21.15.37:0/611270151 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff45010ff10 con 0x7ff4500fde90 2024-03-29T08:34:23.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:23.158+0000 7ff447400700 1 -- 172.21.15.37:0/611270151 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff440009640 con 0x7ff4500fde90 2024-03-29T08:34:23.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:23.158+0000 7ff456800700 1 -- 172.21.15.37:0/611270151 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff450110450 con 0x7ff4500fde90 2024-03-29T08:34:23.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:23.159+0000 7ff456800700 1 -- 172.21.15.37:0/611270151 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff450106a20 con 0x7ff4500fde90 2024-03-29T08:34:23.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:23.160+0000 7ff447400700 1 -- 172.21.15.37:0/611270151 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7ff4400097a0 con 0x7ff4500fde90 2024-03-29T08:34:23.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:23.161+0000 7ff447400700 1 --2- 172.21.15.37:0/611270151 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7ff43406cc30 0x7ff43406f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:34:23.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:23.161+0000 7ff447400700 1 -- 172.21.15.37:0/611270151 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7ff44008b760 con 0x7ff4500fde90 2024-03-29T08:34:23.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:23.161+0000 7ff44ea00700 1 --2- 172.21.15.37:0/611270151 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7ff43406cc30 0x7ff43406f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:34:23.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:23.162+0000 7ff44ea00700 1 --2- 172.21.15.37:0/611270151 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7ff43406cc30 0x7ff43406f0e0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7ff43c004f90 tx=0x7ff43c009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:34:23.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:23.165+0000 7ff447400700 1 -- 172.21.15.37:0/611270151 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7ff4400557a0 con 0x7ff4500fde90 2024-03-29T08:34:23.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:23.429+0000 7ff456800700 1 -- 172.21.15.37:0/611270151 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7ff4501112d0 con 0x7ff4500fde90 2024-03-29T08:34:23.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:23.430+0000 7ff447400700 1 -- 172.21.15.37:0/611270151 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 30 v30) v1 ==== 76+0+1841 (secure 0 0 0) 0x7ff440058dc0 con 0x7ff4500fde90 2024-03-29T08:34:23.430 INFO:teuthology.orchestra.run.smithi037.stdout:e30 2024-03-29T08:34:23.430 INFO:teuthology.orchestra.run.smithi037.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-29T08:34:23.430 INFO:teuthology.orchestra.run.smithi037.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T08:34:23.430 INFO:teuthology.orchestra.run.smithi037.stdout:legacy client fscid: 1 2024-03-29T08:34:23.430 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:34:23.430 INFO:teuthology.orchestra.run.smithi037.stdout:Filesystem 'cephfs' (1) 2024-03-29T08:34:23.430 INFO:teuthology.orchestra.run.smithi037.stdout:fs_name cephfs 2024-03-29T08:34:23.430 INFO:teuthology.orchestra.run.smithi037.stdout:epoch 21 2024-03-29T08:34:23.431 INFO:teuthology.orchestra.run.smithi037.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-03-29T08:34:23.431 INFO:teuthology.orchestra.run.smithi037.stdout:created 2024-03-29T08:03:29.241269+0000 2024-03-29T08:34:23.431 INFO:teuthology.orchestra.run.smithi037.stdout:modified 2024-03-29T08:17:16.237238+0000 2024-03-29T08:34:23.431 INFO:teuthology.orchestra.run.smithi037.stdout:tableserver 0 2024-03-29T08:34:23.431 INFO:teuthology.orchestra.run.smithi037.stdout:root 0 2024-03-29T08:34:23.431 INFO:teuthology.orchestra.run.smithi037.stdout:session_timeout 60 2024-03-29T08:34:23.431 INFO:teuthology.orchestra.run.smithi037.stdout:session_autoclose 300 2024-03-29T08:34:23.431 INFO:teuthology.orchestra.run.smithi037.stdout:max_file_size 1099511627776 2024-03-29T08:34:23.431 INFO:teuthology.orchestra.run.smithi037.stdout:max_xattr_size 65536 2024-03-29T08:34:23.431 INFO:teuthology.orchestra.run.smithi037.stdout:required_client_features {} 2024-03-29T08:34:23.431 INFO:teuthology.orchestra.run.smithi037.stdout:last_failure 0 2024-03-29T08:34:23.431 INFO:teuthology.orchestra.run.smithi037.stdout:last_failure_osd_epoch 82 2024-03-29T08:34:23.431 INFO:teuthology.orchestra.run.smithi037.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T08:34:23.431 INFO:teuthology.orchestra.run.smithi037.stdout:max_mds 2 2024-03-29T08:34:23.431 INFO:teuthology.orchestra.run.smithi037.stdout:in 0,1 2024-03-29T08:34:23.432 INFO:teuthology.orchestra.run.smithi037.stdout:up {} 2024-03-29T08:34:23.432 INFO:teuthology.orchestra.run.smithi037.stdout:failed 0,1 2024-03-29T08:34:23.432 INFO:teuthology.orchestra.run.smithi037.stdout:damaged 2024-03-29T08:34:23.432 INFO:teuthology.orchestra.run.smithi037.stdout:stopped 2024-03-29T08:34:23.432 INFO:teuthology.orchestra.run.smithi037.stdout:data_pools [3] 2024-03-29T08:34:23.432 INFO:teuthology.orchestra.run.smithi037.stdout:metadata_pool 2 2024-03-29T08:34:23.432 INFO:teuthology.orchestra.run.smithi037.stdout:inline_data disabled 2024-03-29T08:34:23.432 INFO:teuthology.orchestra.run.smithi037.stdout:balancer 2024-03-29T08:34:23.432 INFO:teuthology.orchestra.run.smithi037.stdout:bal_rank_mask -1 2024-03-29T08:34:23.432 INFO:teuthology.orchestra.run.smithi037.stdout:standby_count_wanted 1 2024-03-29T08:34:23.432 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:34:23.432 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:34:23.432 INFO:teuthology.orchestra.run.smithi037.stdout:Standby daemons: 2024-03-29T08:34:23.432 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:34:23.432 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi037.mmwgxe{-1:34440} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.37:6828/1248142005,v1:172.21.15.37:6829/1248142005] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:34:23.433 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi037.iifsmh{-1:34444} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.37:6826/2060882959,v1:172.21.15.37:6827/2060882959] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:34:23.433 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi148.sqteih{-1:44395} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.148:6824/4078652242,v1:172.21.15.148:6825/4078652242] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:34:23.433 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi148.qukpjh{-1:45007} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.148:6826/2394751105,v1:172.21.15.148:6827/2394751105] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:34:23.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:23.432+0000 7ff456800700 1 -- 172.21.15.37:0/611270151 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7ff43406cc30 msgr2=0x7ff43406f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:34:23.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:23.432+0000 7ff456800700 1 --2- 172.21.15.37:0/611270151 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7ff43406cc30 0x7ff43406f0e0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7ff43c004f90 tx=0x7ff43c009e30 comp rx=0 tx=0).stop 2024-03-29T08:34:23.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:23.432+0000 7ff456800700 1 -- 172.21.15.37:0/611270151 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff4500fde90 msgr2=0x7ff450115c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:34:23.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:23.432+0000 7ff456800700 1 --2- 172.21.15.37:0/611270151 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff4500fde90 0x7ff450115c80 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7ff440002a80 tx=0x7ff4400046b0 comp rx=0 tx=0).stop 2024-03-29T08:34:23.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:23.432+0000 7ff456800700 1 -- 172.21.15.37:0/611270151 shutdown_connections 2024-03-29T08:34:23.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:23.432+0000 7ff456800700 1 --2- 172.21.15.37:0/611270151 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7ff43406cc30 0x7ff43406f0e0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:34:23.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:23.432+0000 7ff456800700 1 --2- 172.21.15.37:0/611270151 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff4500fde90 0x7ff450115c80 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:34:23.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:23.432+0000 7ff456800700 1 --2- 172.21.15.37:0/611270151 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff4500fe830 0x7ff4501161c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:34:23.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:23.432+0000 7ff456800700 1 -- 172.21.15.37:0/611270151 >> 172.21.15.37:0/611270151 conn(0x7ff450072bc0 msgr2=0x7ff4500fd510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:34:23.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:23.432+0000 7ff456800700 1 -- 172.21.15.37:0/611270151 shutdown_connections 2024-03-29T08:34:23.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:23.432+0000 7ff456800700 1 -- 172.21.15.37:0/611270151 wait complete. 2024-03-29T08:34:23.434 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 30 2024-03-29T08:34:23.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:23.571+0000 7f4b56000700 1 -- 172.21.15.37:0/601134192 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4b500fe830 msgr2=0x7f4b50106f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:34:23.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:23.571+0000 7f4b56000700 1 --2- 172.21.15.37:0/601134192 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4b500fe830 0x7f4b50106f00 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f4b3c004940 tx=0x7f4b3c009e30 comp rx=0 tx=0).stop 2024-03-29T08:34:23.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:23.572+0000 7f4b56000700 1 -- 172.21.15.37:0/601134192 shutdown_connections 2024-03-29T08:34:23.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:23.572+0000 7f4b56000700 1 --2- 172.21.15.37:0/601134192 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4b500fe830 0x7f4b50106f00 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:34:23.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:23.572+0000 7f4b56000700 1 --2- 172.21.15.37:0/601134192 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4b500fde90 0x7f4b500fe260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:34:23.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:23.572+0000 7f4b56000700 1 -- 172.21.15.37:0/601134192 >> 172.21.15.37:0/601134192 conn(0x7f4b50072bc0 msgr2=0x7f4b50072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:34:23.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:23.572+0000 7f4b56000700 1 -- 172.21.15.37:0/601134192 shutdown_connections 2024-03-29T08:34:23.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:23.572+0000 7f4b56000700 1 -- 172.21.15.37:0/601134192 wait complete. 2024-03-29T08:34:23.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:23.573+0000 7f4b56000700 1 Processor -- start 2024-03-29T08:34:23.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:23.573+0000 7f4b56000700 1 -- start start 2024-03-29T08:34:23.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:23.573+0000 7f4b56000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4b500fde90 0x7f4b50115d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:34:23.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:23.573+0000 7f4b56000700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4b500fe830 0x7f4b50116250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:34:23.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:23.573+0000 7f4b56000700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b501168a0 con 0x7f4b500fde90 2024-03-29T08:34:23.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:23.573+0000 7f4b56000700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b5010fc70 con 0x7f4b500fe830 2024-03-29T08:34:23.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:23.573+0000 7f4b4f400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4b500fde90 0x7f4b50115d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:34:23.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:23.573+0000 7f4b4f400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4b500fde90 0x7f4b50115d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40010/0 (socket says 172.21.15.37:40010) 2024-03-29T08:34:23.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:23.573+0000 7f4b4f400700 1 -- 172.21.15.37:0/2732796955 learned_addr learned my addr 172.21.15.37:0/2732796955 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:34:23.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:23.574+0000 7f4b4ea00700 1 --2- 172.21.15.37:0/2732796955 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4b500fe830 0x7f4b50116250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:34:23.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:23.574+0000 7f4b4f400700 1 -- 172.21.15.37:0/2732796955 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4b500fe830 msgr2=0x7f4b50116250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:34:23.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:23.574+0000 7f4b4f400700 1 --2- 172.21.15.37:0/2732796955 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4b500fe830 0x7f4b50116250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:34:23.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:23.574+0000 7f4b4f400700 1 -- 172.21.15.37:0/2732796955 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b3c005040 con 0x7f4b500fde90 2024-03-29T08:34:23.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:23.574+0000 7f4b4f400700 1 --2- 172.21.15.37:0/2732796955 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4b500fde90 0x7f4b50115d10 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f4b40002a50 tx=0x7f4b400046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:34:23.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:23.574+0000 7f4b47400700 1 -- 172.21.15.37:0/2732796955 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4b40007cc0 con 0x7f4b500fde90 2024-03-29T08:34:23.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:23.574+0000 7f4b47400700 1 -- 172.21.15.37:0/2732796955 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f4b4000f040 con 0x7f4b500fde90 2024-03-29T08:34:23.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:23.574+0000 7f4b56000700 1 -- 172.21.15.37:0/2732796955 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b5010ff50 con 0x7f4b500fde90 2024-03-29T08:34:23.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:23.574+0000 7f4b47400700 1 -- 172.21.15.37:0/2732796955 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4b4000d9c0 con 0x7f4b500fde90 2024-03-29T08:34:23.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:23.575+0000 7f4b56000700 1 -- 172.21.15.37:0/2732796955 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b50110490 con 0x7f4b500fde90 2024-03-29T08:34:23.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:23.576+0000 7f4b56000700 1 -- 172.21.15.37:0/2732796955 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b50106a20 con 0x7f4b500fde90 2024-03-29T08:34:23.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:23.577+0000 7f4b47400700 1 -- 172.21.15.37:0/2732796955 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f4b4000db20 con 0x7f4b500fde90 2024-03-29T08:34:23.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:23.577+0000 7f4b47400700 1 --2- 172.21.15.37:0/2732796955 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f4b3406cc30 0x7f4b3406f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:34:23.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:23.577+0000 7f4b47400700 1 -- 172.21.15.37:0/2732796955 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f4b4008bde0 con 0x7f4b500fde90 2024-03-29T08:34:23.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:23.578+0000 7f4b4ea00700 1 --2- 172.21.15.37:0/2732796955 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f4b3406cc30 0x7f4b3406f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:34:23.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:23.578+0000 7f4b4ea00700 1 --2- 172.21.15.37:0/2732796955 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f4b3406cc30 0x7f4b3406f0e0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f4b501115b0 tx=0x7f4b3c009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:34:23.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:23.582+0000 7f4b47400700 1 -- 172.21.15.37:0/2732796955 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f4b400597b0 con 0x7f4b500fde90 2024-03-29T08:34:23.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:34:23 smithi037 ceph-mon[142325]: from='client.35294 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:34:23.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:34:23 smithi037 ceph-mon[142325]: from='client.35298 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:34:23.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:34:23 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/3201688941' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:34:23.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:34:23 smithi148 ceph-mon[106692]: from='client.35294 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:34:23.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:34:23 smithi148 ceph-mon[106692]: from='client.35298 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:34:23.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:34:23 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/3201688941' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:34:23.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:23.817+0000 7f4b56000700 1 -- 172.21.15.37:0/2732796955 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4b50109e40 con 0x7f4b3406cc30 2024-03-29T08:34:23.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:23.818+0000 7f4b47400700 1 -- 172.21.15.37:0/2732796955 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f4b50109e40 con 0x7f4b3406cc30 2024-03-29T08:34:23.818 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-29T08:34:23.818 INFO:teuthology.orchestra.run.smithi037.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59", 2024-03-29T08:34:23.818 INFO:teuthology.orchestra.run.smithi037.stdout: "in_progress": true, 2024-03-29T08:34:23.819 INFO:teuthology.orchestra.run.smithi037.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-29T08:34:23.819 INFO:teuthology.orchestra.run.smithi037.stdout: "services_complete": [ 2024-03-29T08:34:23.819 INFO:teuthology.orchestra.run.smithi037.stdout: "crash", 2024-03-29T08:34:23.819 INFO:teuthology.orchestra.run.smithi037.stdout: "osd", 2024-03-29T08:34:23.819 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr", 2024-03-29T08:34:23.819 INFO:teuthology.orchestra.run.smithi037.stdout: "mds", 2024-03-29T08:34:23.820 INFO:teuthology.orchestra.run.smithi037.stdout: "mon" 2024-03-29T08:34:23.820 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-29T08:34:23.820 INFO:teuthology.orchestra.run.smithi037.stdout: "progress": "16/21 daemons upgraded", 2024-03-29T08:34:23.820 INFO:teuthology.orchestra.run.smithi037.stdout: "message": "Currently upgrading mds daemons", 2024-03-29T08:34:23.820 INFO:teuthology.orchestra.run.smithi037.stdout: "is_paused": false 2024-03-29T08:34:23.820 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-29T08:34:23.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:23.820+0000 7f4b56000700 1 -- 172.21.15.37:0/2732796955 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f4b3406cc30 msgr2=0x7f4b3406f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:34:23.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:23.820+0000 7f4b56000700 1 --2- 172.21.15.37:0/2732796955 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f4b3406cc30 0x7f4b3406f0e0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f4b501115b0 tx=0x7f4b3c009e30 comp rx=0 tx=0).stop 2024-03-29T08:34:23.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:23.820+0000 7f4b56000700 1 -- 172.21.15.37:0/2732796955 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4b500fde90 msgr2=0x7f4b50115d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:34:23.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:23.820+0000 7f4b56000700 1 --2- 172.21.15.37:0/2732796955 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4b500fde90 0x7f4b50115d10 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f4b40002a50 tx=0x7f4b400046b0 comp rx=0 tx=0).stop 2024-03-29T08:34:23.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:23.820+0000 7f4b56000700 1 -- 172.21.15.37:0/2732796955 shutdown_connections 2024-03-29T08:34:23.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:23.820+0000 7f4b56000700 1 --2- 172.21.15.37:0/2732796955 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f4b3406cc30 0x7f4b3406f0e0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:34:23.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:23.820+0000 7f4b56000700 1 --2- 172.21.15.37:0/2732796955 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4b500fde90 0x7f4b50115d10 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:34:23.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:23.820+0000 7f4b56000700 1 --2- 172.21.15.37:0/2732796955 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4b500fe830 0x7f4b50116250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:34:23.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:23.820+0000 7f4b56000700 1 -- 172.21.15.37:0/2732796955 >> 172.21.15.37:0/2732796955 conn(0x7f4b50072bc0 msgr2=0x7f4b500fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:34:23.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:23.820+0000 7f4b56000700 1 -- 172.21.15.37:0/2732796955 shutdown_connections 2024-03-29T08:34:23.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:23.820+0000 7f4b56000700 1 -- 172.21.15.37:0/2732796955 wait complete. 2024-03-29T08:34:23.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:23.957+0000 7f6416a00700 1 -- 172.21.15.37:0/2084913143 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f64100fe830 msgr2=0x7f6410106f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:34:23.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:23.957+0000 7f6416a00700 1 --2- 172.21.15.37:0/2084913143 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f64100fe830 0x7f6410106f00 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f63fc004940 tx=0x7f63fc009e30 comp rx=0 tx=0).stop 2024-03-29T08:34:23.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:23.957+0000 7f6416a00700 1 -- 172.21.15.37:0/2084913143 shutdown_connections 2024-03-29T08:34:23.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:23.957+0000 7f6416a00700 1 --2- 172.21.15.37:0/2084913143 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f64100fe830 0x7f6410106f00 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:34:23.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:23.957+0000 7f6416a00700 1 --2- 172.21.15.37:0/2084913143 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f64100fde90 0x7f64100fe260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:34:23.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:23.957+0000 7f6416a00700 1 -- 172.21.15.37:0/2084913143 >> 172.21.15.37:0/2084913143 conn(0x7f6410072bc0 msgr2=0x7f6410072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:34:23.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:23.957+0000 7f6416a00700 1 -- 172.21.15.37:0/2084913143 shutdown_connections 2024-03-29T08:34:23.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:23.957+0000 7f6416a00700 1 -- 172.21.15.37:0/2084913143 wait complete. 2024-03-29T08:34:23.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:23.958+0000 7f6416a00700 1 Processor -- start 2024-03-29T08:34:23.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:23.958+0000 7f6416a00700 1 -- start start 2024-03-29T08:34:23.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:23.958+0000 7f6416a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f64100fde90 0x7f6410115c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:34:23.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:23.958+0000 7f6416a00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f64100fe830 0x7f64101161c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:34:23.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:23.958+0000 7f6416a00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6410116810 con 0x7f64100fde90 2024-03-29T08:34:23.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:23.958+0000 7f6416a00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f641010fc30 con 0x7f64100fe830 2024-03-29T08:34:23.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:23.959+0000 7f640fe00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f64100fde90 0x7f6410115c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:34:23.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:23.959+0000 7f640fe00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f64100fde90 0x7f6410115c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40032/0 (socket says 172.21.15.37:40032) 2024-03-29T08:34:23.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:23.959+0000 7f640fe00700 1 -- 172.21.15.37:0/1881060663 learned_addr learned my addr 172.21.15.37:0/1881060663 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:34:23.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:23.959+0000 7f640f400700 1 --2- 172.21.15.37:0/1881060663 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f64100fe830 0x7f64101161c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:34:23.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:23.959+0000 7f640fe00700 1 -- 172.21.15.37:0/1881060663 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f64100fe830 msgr2=0x7f64101161c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:34:23.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:23.959+0000 7f640fe00700 1 --2- 172.21.15.37:0/1881060663 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f64100fe830 0x7f64101161c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:34:23.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:23.959+0000 7f640fe00700 1 -- 172.21.15.37:0/1881060663 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63fc005040 con 0x7f64100fde90 2024-03-29T08:34:23.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:23.959+0000 7f640fe00700 1 --2- 172.21.15.37:0/1881060663 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f64100fde90 0x7f6410115c80 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f640000eb40 tx=0x7f640000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:34:23.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:23.959+0000 7f6407e00700 1 -- 172.21.15.37:0/1881060663 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6400007cc0 con 0x7f64100fde90 2024-03-29T08:34:23.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:23.959+0000 7f6407e00700 1 -- 172.21.15.37:0/1881060663 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f640000f040 con 0x7f64100fde90 2024-03-29T08:34:23.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:23.959+0000 7f6416a00700 1 -- 172.21.15.37:0/1881060663 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f641010feb0 con 0x7f64100fde90 2024-03-29T08:34:23.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:23.960+0000 7f6407e00700 1 -- 172.21.15.37:0/1881060663 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6400009570 con 0x7f64100fde90 2024-03-29T08:34:23.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:23.960+0000 7f6416a00700 1 -- 172.21.15.37:0/1881060663 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64101103f0 con 0x7f64100fde90 2024-03-29T08:34:23.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:23.961+0000 7f6416a00700 1 -- 172.21.15.37:0/1881060663 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6410106a20 con 0x7f64100fde90 2024-03-29T08:34:23.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:23.962+0000 7f6407e00700 1 -- 172.21.15.37:0/1881060663 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f64000096d0 con 0x7f64100fde90 2024-03-29T08:34:23.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:23.962+0000 7f6407e00700 1 --2- 172.21.15.37:0/1881060663 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f63f406cc30 0x7f63f406f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:34:23.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:23.963+0000 7f6407e00700 1 -- 172.21.15.37:0/1881060663 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f640008bda0 con 0x7f64100fde90 2024-03-29T08:34:23.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:23.963+0000 7f640f400700 1 --2- 172.21.15.37:0/1881060663 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f63f406cc30 0x7f63f406f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:34:23.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:23.963+0000 7f640f400700 1 --2- 172.21.15.37:0/1881060663 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f63f406cc30 0x7f63f406f0e0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f63fc00b0d0 tx=0x7f63fc009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:34:23.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:23.967+0000 7f6407e00700 1 -- 172.21.15.37:0/1881060663 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f6400055d90 con 0x7f64100fde90 2024-03-29T08:34:24.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:24.263+0000 7f6416a00700 1 -- 172.21.15.37:0/1881060663 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f64101112d0 con 0x7f64100fde90 2024-03-29T08:34:24.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:24.264+0000 7f6407e00700 1 -- 172.21.15.37:0/1881060663 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+3583 (secure 0 0 0) 0x7f6400019e10 con 0x7f64100fde90 2024-03-29T08:34:24.266 INFO:teuthology.orchestra.run.smithi037.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem is offline 2024-03-29T08:34:24.266 INFO:teuthology.orchestra.run.smithi037.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-03-29T08:34:24.266 INFO:teuthology.orchestra.run.smithi037.stdout: host smithi148 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/mon.smithi148/config 2024-03-29T08:34:24.267 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/podman: timeout after 895 seconds 2024-03-29T08:34:24.267 INFO:teuthology.orchestra.run.smithi037.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59 -e NODE_NAME=smithi148 -e CEPH_USE_RANDOM_NONCE=1 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4:/var/run/ceph:z -v /var/log/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4:/var/log/ceph:z -v /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpkx2rz2d4:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59 inventory --format=json-pretty --filter-for-batch 2024-03-29T08:34:24.267 INFO:teuthology.orchestra.run.smithi037.stdout:Traceback (most recent call last): 2024-03-29T08:34:24.267 INFO:teuthology.orchestra.run.smithi037.stdout: File "/usr/lib64/python3.6/runpy.py", line 193, in _run_module_as_main 2024-03-29T08:34:24.267 INFO:teuthology.orchestra.run.smithi037.stdout: "__main__", mod_spec) 2024-03-29T08:34:24.267 INFO:teuthology.orchestra.run.smithi037.stdout: File "/usr/lib64/python3.6/runpy.py", line 85, in _run_code 2024-03-29T08:34:24.267 INFO:teuthology.orchestra.run.smithi037.stdout: exec(code, run_globals) 2024-03-29T08:34:24.267 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 10889, in 2024-03-29T08:34:24.267 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 10877, in main 2024-03-29T08:34:24.267 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 2576, in _infer_config 2024-03-29T08:34:24.267 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 2492, in _infer_fsid 2024-03-29T08:34:24.267 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 2604, in _infer_image 2024-03-29T08:34:24.267 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 2479, in _validate_fsid 2024-03-29T08:34:24.267 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 7145, in command_ceph_volume 2024-03-29T08:34:24.268 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 2267, in call_throws 2024-03-29T08:34:24.268 INFO:teuthology.orchestra.run.smithi037.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59 -e NODE_NAME=smithi148 -e CEPH_USE_RANDOM_NONCE=1 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4:/var/run/ceph:z -v /var/log/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4:/var/log/ceph:z -v /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpkx2rz2d4:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59 inventory --format=json-pretty --filter-for-batch 2024-03-29T08:34:24.268 INFO:teuthology.orchestra.run.smithi037.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-03-29T08:34:24.268 INFO:teuthology.orchestra.run.smithi037.stdout: fs cephfs is degraded 2024-03-29T08:34:24.268 INFO:teuthology.orchestra.run.smithi037.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-03-29T08:34:24.268 INFO:teuthology.orchestra.run.smithi037.stdout: fs cephfs is offline because no MDS is active for it. 2024-03-29T08:34:24.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:24.266+0000 7f6416a00700 1 -- 172.21.15.37:0/1881060663 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f63f406cc30 msgr2=0x7f63f406f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:34:24.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:24.266+0000 7f6416a00700 1 --2- 172.21.15.37:0/1881060663 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f63f406cc30 0x7f63f406f0e0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f63fc00b0d0 tx=0x7f63fc009e30 comp rx=0 tx=0).stop 2024-03-29T08:34:24.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:24.266+0000 7f6416a00700 1 -- 172.21.15.37:0/1881060663 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f64100fde90 msgr2=0x7f6410115c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:34:24.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:24.266+0000 7f6416a00700 1 --2- 172.21.15.37:0/1881060663 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f64100fde90 0x7f6410115c80 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f640000eb40 tx=0x7f640000b040 comp rx=0 tx=0).stop 2024-03-29T08:34:24.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:24.266+0000 7f6416a00700 1 -- 172.21.15.37:0/1881060663 shutdown_connections 2024-03-29T08:34:24.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:24.266+0000 7f6416a00700 1 --2- 172.21.15.37:0/1881060663 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f63f406cc30 0x7f63f406f0e0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:34:24.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:24.266+0000 7f6416a00700 1 --2- 172.21.15.37:0/1881060663 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f64100fde90 0x7f6410115c80 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:34:24.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:24.266+0000 7f6416a00700 1 --2- 172.21.15.37:0/1881060663 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f64100fe830 0x7f64101161c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:34:24.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:24.266+0000 7f6416a00700 1 -- 172.21.15.37:0/1881060663 >> 172.21.15.37:0/1881060663 conn(0x7f6410072bc0 msgr2=0x7f64100fd510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:34:24.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:24.266+0000 7f6416a00700 1 -- 172.21.15.37:0/1881060663 shutdown_connections 2024-03-29T08:34:24.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:24.266+0000 7f6416a00700 1 -- 172.21.15.37:0/1881060663 wait complete. 2024-03-29T08:34:24.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:34:24 smithi037 ceph-mon[142325]: pgmap v706: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:34:24.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:34:24 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/611270151' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T08:34:24.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:34:24 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/1881060663' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T08:34:24.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:34:24 smithi148 ceph-mon[106692]: pgmap v706: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:34:24.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:34:24 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/611270151' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T08:34:24.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:34:24 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/1881060663' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T08:34:25.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:34:25 smithi148 ceph-mon[106692]: from='client.35310 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:34:25.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:34:25 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:34:25.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:34:25 smithi037 ceph-mon[142325]: from='client.35310 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:34:25.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:34:25 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:34:26.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:34:26 smithi148 ceph-mon[106692]: pgmap v707: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:34:26.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:34:26 smithi037 ceph-mon[142325]: pgmap v707: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:34:28.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:34:28 smithi148 ceph-mon[106692]: pgmap v708: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:34:28.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:34:28 smithi037 ceph-mon[142325]: pgmap v708: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:34:30.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:34:30 smithi148 ceph-mon[106692]: pgmap v709: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:34:30.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:34:30 smithi037 ceph-mon[142325]: pgmap v709: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:34:32.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:34:32 smithi148 ceph-mon[106692]: pgmap v710: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:34:32.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:34:32 smithi037 ceph-mon[142325]: pgmap v710: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:34:34.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:34:34 smithi148 ceph-mon[106692]: pgmap v711: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:34:34.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:34:34 smithi037 ceph-mon[142325]: pgmap v711: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:34:36.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:34:36 smithi148 ceph-mon[106692]: pgmap v712: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:34:36.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:34:36 smithi037 ceph-mon[142325]: pgmap v712: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:34:38.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:34:38 smithi148 ceph-mon[106692]: pgmap v713: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:34:38.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:34:38 smithi037 ceph-mon[142325]: pgmap v713: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:34:40.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:34:40 smithi148 ceph-mon[106692]: pgmap v714: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:34:40.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:34:40 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:34:40.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:34:40 smithi037 ceph-mon[142325]: pgmap v714: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:34:40.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:34:40 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:34:42.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:34:42 smithi148 ceph-mon[106692]: pgmap v715: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:34:42.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:34:42 smithi037 ceph-mon[142325]: pgmap v715: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:34:44.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:34:44 smithi148 ceph-mon[106692]: pgmap v716: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:34:44.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:34:44 smithi037 ceph-mon[142325]: pgmap v716: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:34:46.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:34:46 smithi148 ceph-mon[106692]: pgmap v717: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:34:46.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:34:46 smithi037 ceph-mon[142325]: pgmap v717: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:34:48.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:34:48 smithi148 ceph-mon[106692]: pgmap v718: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:34:48.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:34:48 smithi037 ceph-mon[142325]: pgmap v718: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:34:50.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:34:50 smithi148 ceph-mon[106692]: pgmap v719: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:34:50.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:34:50 smithi037 ceph-mon[142325]: pgmap v719: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:34:52.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:34:52 smithi148 ceph-mon[106692]: pgmap v720: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:34:52.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:34:52 smithi037 ceph-mon[142325]: pgmap v720: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:34:54.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:54.421+0000 7fb5edc00700 1 -- 172.21.15.37:0/2599940825 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb5e8102e00 msgr2=0x7fb5e81031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:34:54.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:54.421+0000 7fb5edc00700 1 --2- 172.21.15.37:0/2599940825 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb5e8102e00 0x7fb5e81031d0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fb5d8004940 tx=0x7fb5d8009e30 comp rx=0 tx=0).stop 2024-03-29T08:34:54.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:54.421+0000 7fb5edc00700 1 -- 172.21.15.37:0/2599940825 shutdown_connections 2024-03-29T08:34:54.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:54.421+0000 7fb5edc00700 1 --2- 172.21.15.37:0/2599940825 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb5e81037a0 0x7fb5e80fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:34:54.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:54.421+0000 7fb5edc00700 1 --2- 172.21.15.37:0/2599940825 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb5e8102e00 0x7fb5e81031d0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:34:54.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:54.421+0000 7fb5edc00700 1 -- 172.21.15.37:0/2599940825 >> 172.21.15.37:0/2599940825 conn(0x7fb5e80f89b0 msgr2=0x7fb5e80fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:34:54.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:54.421+0000 7fb5edc00700 1 -- 172.21.15.37:0/2599940825 shutdown_connections 2024-03-29T08:34:54.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:54.421+0000 7fb5edc00700 1 -- 172.21.15.37:0/2599940825 wait complete. 2024-03-29T08:34:54.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:54.422+0000 7fb5edc00700 1 Processor -- start 2024-03-29T08:34:54.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:54.422+0000 7fb5edc00700 1 -- start start 2024-03-29T08:34:54.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:54.423+0000 7fb5edc00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb5e8102e00 0x7fb5e8115cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:34:54.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:54.423+0000 7fb5edc00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb5e81037a0 0x7fb5e8116230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:34:54.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:54.423+0000 7fb5edc00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5e8116880 con 0x7fb5e8102e00 2024-03-29T08:34:54.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:54.423+0000 7fb5edc00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5e810fc50 con 0x7fb5e81037a0 2024-03-29T08:34:54.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:54.423+0000 7fb5e6a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb5e8102e00 0x7fb5e8115cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:34:54.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:54.423+0000 7fb5e6a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb5e8102e00 0x7fb5e8115cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39296/0 (socket says 172.21.15.37:39296) 2024-03-29T08:34:54.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:54.423+0000 7fb5e6a00700 1 -- 172.21.15.37:0/3066747672 learned_addr learned my addr 172.21.15.37:0/3066747672 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:34:54.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:54.423+0000 7fb5e6000700 1 --2- 172.21.15.37:0/3066747672 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb5e81037a0 0x7fb5e8116230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:34:54.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:54.423+0000 7fb5e6a00700 1 -- 172.21.15.37:0/3066747672 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb5e81037a0 msgr2=0x7fb5e8116230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:34:54.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:54.423+0000 7fb5e6a00700 1 --2- 172.21.15.37:0/3066747672 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb5e81037a0 0x7fb5e8116230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:34:54.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:54.423+0000 7fb5e6a00700 1 -- 172.21.15.37:0/3066747672 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5d8005040 con 0x7fb5e8102e00 2024-03-29T08:34:54.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:54.423+0000 7fb5e6000700 1 --2- 172.21.15.37:0/3066747672 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb5e81037a0 0x7fb5e8116230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:34:54.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:54.424+0000 7fb5e6a00700 1 --2- 172.21.15.37:0/3066747672 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb5e8102e00 0x7fb5e8115cf0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fb5d8009180 tx=0x7fb5d8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:34:54.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:54.424+0000 7fb5dea00700 1 -- 172.21.15.37:0/3066747672 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb5d800e040 con 0x7fb5e8102e00 2024-03-29T08:34:54.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:54.424+0000 7fb5dea00700 1 -- 172.21.15.37:0/3066747672 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fb5d8004290 con 0x7fb5e8102e00 2024-03-29T08:34:54.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:54.424+0000 7fb5edc00700 1 -- 172.21.15.37:0/3066747672 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5e810fed0 con 0x7fb5e8102e00 2024-03-29T08:34:54.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:54.424+0000 7fb5dea00700 1 -- 172.21.15.37:0/3066747672 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb5d801c3f0 con 0x7fb5e8102e00 2024-03-29T08:34:54.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:54.424+0000 7fb5edc00700 1 -- 172.21.15.37:0/3066747672 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5e8110490 con 0x7fb5e8102e00 2024-03-29T08:34:54.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:54.425+0000 7fb5edc00700 1 -- 172.21.15.37:0/3066747672 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5e80fe550 con 0x7fb5e8102e00 2024-03-29T08:34:54.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:54.427+0000 7fb5dea00700 1 -- 172.21.15.37:0/3066747672 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fb5d8013790 con 0x7fb5e8102e00 2024-03-29T08:34:54.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:54.428+0000 7fb5dea00700 1 --2- 172.21.15.37:0/3066747672 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fb5cc06cc20 0x7fb5cc06f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:34:54.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:54.428+0000 7fb5dea00700 1 -- 172.21.15.37:0/3066747672 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fb5d808d370 con 0x7fb5e8102e00 2024-03-29T08:34:54.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:54.428+0000 7fb5e6000700 1 --2- 172.21.15.37:0/3066747672 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fb5cc06cc20 0x7fb5cc06f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:34:54.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:54.429+0000 7fb5e6000700 1 --2- 172.21.15.37:0/3066747672 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fb5cc06cc20 0x7fb5cc06f0d0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fb5d4007fd0 tx=0x7fb5d4009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:34:54.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:54.433+0000 7fb5dea00700 1 -- 172.21.15.37:0/3066747672 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fb5d805adc0 con 0x7fb5e8102e00 2024-03-29T08:34:54.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:54.670+0000 7fb5edc00700 1 -- 172.21.15.37:0/3066747672 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb5e804bf20 con 0x7fb5cc06cc20 2024-03-29T08:34:54.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:54.671+0000 7fb5dea00700 1 -- 172.21.15.37:0/3066747672 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fb5e804bf20 con 0x7fb5cc06cc20 2024-03-29T08:34:54.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:54.674+0000 7fb5edc00700 1 -- 172.21.15.37:0/3066747672 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fb5cc06cc20 msgr2=0x7fb5cc06f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:34:54.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:54.674+0000 7fb5edc00700 1 --2- 172.21.15.37:0/3066747672 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fb5cc06cc20 0x7fb5cc06f0d0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fb5d4007fd0 tx=0x7fb5d4009040 comp rx=0 tx=0).stop 2024-03-29T08:34:54.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:54.674+0000 7fb5edc00700 1 -- 172.21.15.37:0/3066747672 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb5e8102e00 msgr2=0x7fb5e8115cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:34:54.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:54.674+0000 7fb5edc00700 1 --2- 172.21.15.37:0/3066747672 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb5e8102e00 0x7fb5e8115cf0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fb5d8009180 tx=0x7fb5d8009e30 comp rx=0 tx=0).stop 2024-03-29T08:34:54.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:54.674+0000 7fb5edc00700 1 -- 172.21.15.37:0/3066747672 shutdown_connections 2024-03-29T08:34:54.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:54.674+0000 7fb5edc00700 1 --2- 172.21.15.37:0/3066747672 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fb5cc06cc20 0x7fb5cc06f0d0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:34:54.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:54.674+0000 7fb5edc00700 1 --2- 172.21.15.37:0/3066747672 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb5e8102e00 0x7fb5e8115cf0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:34:54.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:54.674+0000 7fb5edc00700 1 --2- 172.21.15.37:0/3066747672 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb5e81037a0 0x7fb5e8116230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:34:54.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:54.674+0000 7fb5edc00700 1 -- 172.21.15.37:0/3066747672 >> 172.21.15.37:0/3066747672 conn(0x7fb5e80f89b0 msgr2=0x7fb5e80706d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:34:54.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:54.674+0000 7fb5edc00700 1 -- 172.21.15.37:0/3066747672 shutdown_connections 2024-03-29T08:34:54.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:54.674+0000 7fb5edc00700 1 -- 172.21.15.37:0/3066747672 wait complete. 2024-03-29T08:34:54.688 INFO:teuthology.orchestra.run.smithi037.stdout:true 2024-03-29T08:34:54.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:34:54 smithi148 ceph-mon[106692]: pgmap v721: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:34:54.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:54.813+0000 7efe7bc00700 1 -- 172.21.15.37:0/3515120153 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efe740fde90 msgr2=0x7efe740fe260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:34:54.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:54.813+0000 7efe7bc00700 1 --2- 172.21.15.37:0/3515120153 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efe740fde90 0x7efe740fe260 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7efe68004940 tx=0x7efe68009e30 comp rx=0 tx=0).stop 2024-03-29T08:34:54.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:54.814+0000 7efe7bc00700 1 -- 172.21.15.37:0/3515120153 shutdown_connections 2024-03-29T08:34:54.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:54.814+0000 7efe7bc00700 1 --2- 172.21.15.37:0/3515120153 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe740fe830 0x7efe74106f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:34:54.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:54.814+0000 7efe7bc00700 1 --2- 172.21.15.37:0/3515120153 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efe740fde90 0x7efe740fe260 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:34:54.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:54.814+0000 7efe7bc00700 1 -- 172.21.15.37:0/3515120153 >> 172.21.15.37:0/3515120153 conn(0x7efe74072bc0 msgr2=0x7efe74072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:34:54.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:54.814+0000 7efe7bc00700 1 -- 172.21.15.37:0/3515120153 shutdown_connections 2024-03-29T08:34:54.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:54.814+0000 7efe7bc00700 1 -- 172.21.15.37:0/3515120153 wait complete. 2024-03-29T08:34:54.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:54.814+0000 7efe7bc00700 1 Processor -- start 2024-03-29T08:34:54.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:54.815+0000 7efe7bc00700 1 -- start start 2024-03-29T08:34:54.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:54.815+0000 7efe7bc00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efe740fde90 0x7efe74115d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:34:54.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:54.815+0000 7efe7bc00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe740fe830 0x7efe74116250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:34:54.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:54.815+0000 7efe7bc00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe741168a0 con 0x7efe740fde90 2024-03-29T08:34:54.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:54.815+0000 7efe7bc00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe7410fc70 con 0x7efe740fe830 2024-03-29T08:34:54.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:54.815+0000 7efe79200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efe740fde90 0x7efe74115d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:34:54.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:54.815+0000 7efe79200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efe740fde90 0x7efe74115d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39318/0 (socket says 172.21.15.37:39318) 2024-03-29T08:34:54.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:54.815+0000 7efe79200700 1 -- 172.21.15.37:0/1539696208 learned_addr learned my addr 172.21.15.37:0/1539696208 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:34:54.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:54.816+0000 7efe73e00700 1 --2- 172.21.15.37:0/1539696208 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe740fe830 0x7efe74116250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:34:54.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:54.816+0000 7efe79200700 1 -- 172.21.15.37:0/1539696208 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe740fe830 msgr2=0x7efe74116250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:34:54.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:54.816+0000 7efe79200700 1 --2- 172.21.15.37:0/1539696208 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe740fe830 0x7efe74116250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:34:54.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:54.816+0000 7efe79200700 1 -- 172.21.15.37:0/1539696208 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe68005040 con 0x7efe740fde90 2024-03-29T08:34:54.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:54.816+0000 7efe73e00700 1 --2- 172.21.15.37:0/1539696208 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe740fe830 0x7efe74116250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:34:54.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:54.816+0000 7efe79200700 1 --2- 172.21.15.37:0/1539696208 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efe740fde90 0x7efe74115d10 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7efe68000c00 tx=0x7efe68009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:34:54.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:54.816+0000 7efe70c00700 1 -- 172.21.15.37:0/1539696208 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efe6800e040 con 0x7efe740fde90 2024-03-29T08:34:54.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:54.816+0000 7efe7bc00700 1 -- 172.21.15.37:0/1539696208 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe7410fef0 con 0x7efe740fde90 2024-03-29T08:34:54.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:54.816+0000 7efe70c00700 1 -- 172.21.15.37:0/1539696208 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7efe68013420 con 0x7efe740fde90 2024-03-29T08:34:54.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:54.816+0000 7efe70c00700 1 -- 172.21.15.37:0/1539696208 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efe6801c630 con 0x7efe740fde90 2024-03-29T08:34:54.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:54.816+0000 7efe7bc00700 1 -- 172.21.15.37:0/1539696208 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe74110430 con 0x7efe740fde90 2024-03-29T08:34:54.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:54.818+0000 7efe7bc00700 1 -- 172.21.15.37:0/1539696208 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe74106a20 con 0x7efe740fde90 2024-03-29T08:34:54.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:54.819+0000 7efe70c00700 1 -- 172.21.15.37:0/1539696208 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7efe68013970 con 0x7efe740fde90 2024-03-29T08:34:54.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:54.819+0000 7efe70c00700 1 --2- 172.21.15.37:0/1539696208 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7efe5c06cc30 0x7efe5c06f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:34:54.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:54.820+0000 7efe70c00700 1 -- 172.21.15.37:0/1539696208 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7efe6808d260 con 0x7efe740fde90 2024-03-29T08:34:54.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:54.820+0000 7efe73e00700 1 --2- 172.21.15.37:0/1539696208 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7efe5c06cc30 0x7efe5c06f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:34:54.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:54.820+0000 7efe73e00700 1 --2- 172.21.15.37:0/1539696208 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7efe5c06cc30 0x7efe5c06f0e0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7efe60009fd0 tx=0x7efe6000b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:34:54.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:54.824+0000 7efe70c00700 1 -- 172.21.15.37:0/1539696208 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7efe6805ad30 con 0x7efe740fde90 2024-03-29T08:34:54.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:34:54 smithi037 ceph-mon[142325]: pgmap v721: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:34:55.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:55.061+0000 7efe7bc00700 1 -- 172.21.15.37:0/1539696208 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7efe74109df0 con 0x7efe5c06cc30 2024-03-29T08:34:55.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:55.062+0000 7efe70c00700 1 -- 172.21.15.37:0/1539696208 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7efe74109df0 con 0x7efe5c06cc30 2024-03-29T08:34:55.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:55.064+0000 7efe7bc00700 1 -- 172.21.15.37:0/1539696208 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7efe5c06cc30 msgr2=0x7efe5c06f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:34:55.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:55.064+0000 7efe7bc00700 1 --2- 172.21.15.37:0/1539696208 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7efe5c06cc30 0x7efe5c06f0e0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7efe60009fd0 tx=0x7efe6000b040 comp rx=0 tx=0).stop 2024-03-29T08:34:55.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:55.064+0000 7efe7bc00700 1 -- 172.21.15.37:0/1539696208 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efe740fde90 msgr2=0x7efe74115d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:34:55.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:55.064+0000 7efe7bc00700 1 --2- 172.21.15.37:0/1539696208 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efe740fde90 0x7efe74115d10 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7efe68000c00 tx=0x7efe68009e30 comp rx=0 tx=0).stop 2024-03-29T08:34:55.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:55.064+0000 7efe7bc00700 1 -- 172.21.15.37:0/1539696208 shutdown_connections 2024-03-29T08:34:55.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:55.064+0000 7efe7bc00700 1 --2- 172.21.15.37:0/1539696208 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7efe5c06cc30 0x7efe5c06f0e0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:34:55.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:55.064+0000 7efe7bc00700 1 --2- 172.21.15.37:0/1539696208 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efe740fde90 0x7efe74115d10 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:34:55.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:55.064+0000 7efe7bc00700 1 --2- 172.21.15.37:0/1539696208 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe740fe830 0x7efe74116250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:34:55.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:55.064+0000 7efe7bc00700 1 -- 172.21.15.37:0/1539696208 >> 172.21.15.37:0/1539696208 conn(0x7efe74072bc0 msgr2=0x7efe740fd610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:34:55.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:55.065+0000 7efe7bc00700 1 -- 172.21.15.37:0/1539696208 shutdown_connections 2024-03-29T08:34:55.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:55.065+0000 7efe7bc00700 1 -- 172.21.15.37:0/1539696208 wait complete. 2024-03-29T08:34:55.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:55.203+0000 7fdde9400700 1 -- 172.21.15.37:0/4090752010 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdde4102e00 msgr2=0x7fdde41031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:34:55.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:55.203+0000 7fdde9400700 1 --2- 172.21.15.37:0/4090752010 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdde4102e00 0x7fdde41031d0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fddd4004940 tx=0x7fddd4009e30 comp rx=0 tx=0).stop 2024-03-29T08:34:55.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:55.203+0000 7fdde9400700 1 -- 172.21.15.37:0/4090752010 shutdown_connections 2024-03-29T08:34:55.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:55.203+0000 7fdde9400700 1 --2- 172.21.15.37:0/4090752010 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdde41037a0 0x7fdde40fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:34:55.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:55.203+0000 7fdde9400700 1 --2- 172.21.15.37:0/4090752010 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdde4102e00 0x7fdde41031d0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:34:55.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:55.203+0000 7fdde9400700 1 -- 172.21.15.37:0/4090752010 >> 172.21.15.37:0/4090752010 conn(0x7fdde40f89b0 msgr2=0x7fdde40fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:34:55.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:55.203+0000 7fdde9400700 1 -- 172.21.15.37:0/4090752010 shutdown_connections 2024-03-29T08:34:55.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:55.203+0000 7fdde9400700 1 -- 172.21.15.37:0/4090752010 wait complete. 2024-03-29T08:34:55.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:55.204+0000 7fdde9400700 1 Processor -- start 2024-03-29T08:34:55.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:55.204+0000 7fdde9400700 1 -- start start 2024-03-29T08:34:55.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:55.204+0000 7fdde9400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdde4102e00 0x7fdde40702c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:34:55.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:55.204+0000 7fdde9400700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdde41037a0 0x7fdde4070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:34:55.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:55.204+0000 7fdde9400700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdde4070d90 con 0x7fdde4102e00 2024-03-29T08:34:55.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:55.204+0000 7fdde9400700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdde4070ed0 con 0x7fdde41037a0 2024-03-29T08:34:55.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:55.204+0000 7fdde2800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdde4102e00 0x7fdde40702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:34:55.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:55.205+0000 7fdde2800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdde4102e00 0x7fdde40702c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39330/0 (socket says 172.21.15.37:39330) 2024-03-29T08:34:55.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:55.205+0000 7fdde2800700 1 -- 172.21.15.37:0/433684231 learned_addr learned my addr 172.21.15.37:0/433684231 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:34:55.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:55.205+0000 7fdde2800700 1 -- 172.21.15.37:0/433684231 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdde41037a0 msgr2=0x7fdde4070800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:34:55.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:55.205+0000 7fdddbe00700 1 --2- 172.21.15.37:0/433684231 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdde41037a0 0x7fdde4070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:34:55.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:55.205+0000 7fdde2800700 1 --2- 172.21.15.37:0/433684231 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdde41037a0 0x7fdde4070800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:34:55.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:55.205+0000 7fdde2800700 1 -- 172.21.15.37:0/433684231 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddd4005040 con 0x7fdde4102e00 2024-03-29T08:34:55.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:55.205+0000 7fdddbe00700 1 --2- 172.21.15.37:0/433684231 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdde41037a0 0x7fdde4070800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:34:55.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:55.205+0000 7fdde2800700 1 --2- 172.21.15.37:0/433684231 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdde4102e00 0x7fdde40702c0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fddd4009360 tx=0x7fddd4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:34:55.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:55.205+0000 7fdddaa00700 1 -- 172.21.15.37:0/433684231 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fddd400e040 con 0x7fdde4102e00 2024-03-29T08:34:55.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:55.206+0000 7fdddaa00700 1 -- 172.21.15.37:0/433684231 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fddd4013420 con 0x7fdde4102e00 2024-03-29T08:34:55.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:55.206+0000 7fdddaa00700 1 -- 172.21.15.37:0/433684231 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fddd401c5f0 con 0x7fdde4102e00 2024-03-29T08:34:55.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:55.206+0000 7fdde9400700 1 -- 172.21.15.37:0/433684231 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdde4071120 con 0x7fdde4102e00 2024-03-29T08:34:55.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:55.206+0000 7fdde9400700 1 -- 172.21.15.37:0/433684231 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdde411a250 con 0x7fdde4102e00 2024-03-29T08:34:55.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:55.207+0000 7fdde9400700 1 -- 172.21.15.37:0/433684231 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdde40fe550 con 0x7fdde4102e00 2024-03-29T08:34:55.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:55.208+0000 7fdddaa00700 1 -- 172.21.15.37:0/433684231 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fddd4013590 con 0x7fdde4102e00 2024-03-29T08:34:55.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:55.208+0000 7fdddaa00700 1 --2- 172.21.15.37:0/433684231 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fddc806cc30 0x7fddc806f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:34:55.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:55.209+0000 7fdddaa00700 1 -- 172.21.15.37:0/433684231 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fddd408d270 con 0x7fdde4102e00 2024-03-29T08:34:55.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:55.209+0000 7fdddbe00700 1 --2- 172.21.15.37:0/433684231 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fddc806cc30 0x7fddc806f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:34:55.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:55.209+0000 7fdddbe00700 1 --2- 172.21.15.37:0/433684231 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fddc806cc30 0x7fddc806f0e0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fddcc009fd0 tx=0x7fddcc00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:34:55.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:55.213+0000 7fdddaa00700 1 -- 172.21.15.37:0/433684231 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fddd40572b0 con 0x7fdde4102e00 2024-03-29T08:34:55.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:55.434+0000 7fdde9400700 1 -- 172.21.15.37:0/433684231 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fdde40719b0 con 0x7fddc806cc30 2024-03-29T08:34:55.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:55.444+0000 7fdddaa00700 1 -- 172.21.15.37:0/433684231 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7fdde40719b0 con 0x7fddc806cc30 2024-03-29T08:34:55.444 INFO:teuthology.orchestra.run.smithi037.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-29T08:34:55.444 INFO:teuthology.orchestra.run.smithi037.stdout:alertmanager.smithi037 smithi037 *:9093,9094 running (24m) 89s ago 37m 14.3M - 0.25.0 c8568f914cd2 2f9f9746ed0f 2024-03-29T08:34:55.444 INFO:teuthology.orchestra.run.smithi037.stdout:crash.smithi037 smithi037 running (21m) 89s ago 37m 6647k - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 178d416a3186 2024-03-29T08:34:55.444 INFO:teuthology.orchestra.run.smithi037.stdout:crash.smithi148 smithi148 running (21m) 92s ago 36m 6656k - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 23666989d58f 2024-03-29T08:34:55.444 INFO:teuthology.orchestra.run.smithi037.stdout:grafana.smithi037 smithi037 *:3000 running (24m) 89s ago 37m 74.5M - 9.4.7 954c08fa6188 6b976fe49952 2024-03-29T08:34:55.444 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi037.iifsmh smithi037 running (17m) 89s ago 31m 16.4M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 2d97f562cc5b 2024-03-29T08:34:55.444 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi037.mmwgxe smithi037 running (17m) 89s ago 31m 16.7M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff ec37037f46e8 2024-03-29T08:34:55.444 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi148.qukpjh smithi148 running (96s) 92s ago 31m 10.7M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 3257219bc046 2024-03-29T08:34:55.444 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi148.sqteih smithi148 running (16m) 92s ago 31m 16.3M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 44c15c77037a 2024-03-29T08:34:55.445 INFO:teuthology.orchestra.run.smithi037.stdout:mgr.smithi037.wjikii smithi037 *:8443,9283,8765 running (27m) 89s ago 39m 510M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff c5a1417b9417 2024-03-29T08:34:55.445 INFO:teuthology.orchestra.run.smithi037.stdout:mgr.smithi148.onhgvm smithi148 *:8443,9283,8765 running (25m) 92s ago 36m 456M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff fcfdbce062bd 2024-03-29T08:34:55.445 INFO:teuthology.orchestra.run.smithi037.stdout:mon.smithi037 smithi037 running (22m) 89s ago 39m 70.2M 2048M 18.2.2-1015-gd0e392d9 1cc2f45c17ff eea2060783af 2024-03-29T08:34:55.445 INFO:teuthology.orchestra.run.smithi037.stdout:mon.smithi148 smithi148 running (22m) 92s ago 36m 50.4M 2048M 18.2.2-1015-gd0e392d9 1cc2f45c17ff ad4b2c5748be 2024-03-29T08:34:55.445 INFO:teuthology.orchestra.run.smithi037.stdout:node-exporter.smithi037 smithi037 *:9100 running (25m) 89s ago 37m 8514k - 1.5.0 0da6a335fe13 26b8ae9cb878 2024-03-29T08:34:55.445 INFO:teuthology.orchestra.run.smithi037.stdout:node-exporter.smithi148 smithi148 *:9100 running (25m) 92s ago 36m 8464k - 1.5.0 0da6a335fe13 98414e4e64ed 2024-03-29T08:34:55.445 INFO:teuthology.orchestra.run.smithi037.stdout:osd.0 smithi037 running (21m) 89s ago 35m 108M 1619M 18.2.2-1015-gd0e392d9 1cc2f45c17ff c0c584f6245c 2024-03-29T08:34:55.445 INFO:teuthology.orchestra.run.smithi037.stdout:osd.1 smithi037 running (20m) 89s ago 35m 104M 1619M 18.2.2-1015-gd0e392d9 1cc2f45c17ff a4a2ff4cc83d 2024-03-29T08:34:55.445 INFO:teuthology.orchestra.run.smithi037.stdout:osd.2 smithi037 running (19m) 89s ago 34m 85.9M 1619M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 8b1bc71aa79e 2024-03-29T08:34:55.445 INFO:teuthology.orchestra.run.smithi037.stdout:osd.3 smithi148 running (19m) 92s ago 34m 109M 2642M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 1c656ab8ff86 2024-03-29T08:34:55.445 INFO:teuthology.orchestra.run.smithi037.stdout:osd.4 smithi148 running (18m) 92s ago 33m 101M 2642M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 966a9de11d22 2024-03-29T08:34:55.445 INFO:teuthology.orchestra.run.smithi037.stdout:osd.5 smithi148 running (17m) 92s ago 32m 89.4M 2642M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 474c4f9212ce 2024-03-29T08:34:55.445 INFO:teuthology.orchestra.run.smithi037.stdout:prometheus.smithi037 smithi037 *:9095 running (25m) 89s ago 37m 61.0M - 2.43.0 a07b618ecd1d 30c0223ec589 2024-03-29T08:34:55.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:55.446+0000 7fdde9400700 1 -- 172.21.15.37:0/433684231 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fddc806cc30 msgr2=0x7fddc806f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:34:55.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:55.446+0000 7fdde9400700 1 --2- 172.21.15.37:0/433684231 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fddc806cc30 0x7fddc806f0e0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fddcc009fd0 tx=0x7fddcc00b040 comp rx=0 tx=0).stop 2024-03-29T08:34:55.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:55.446+0000 7fdde9400700 1 -- 172.21.15.37:0/433684231 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdde4102e00 msgr2=0x7fdde40702c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:34:55.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:55.446+0000 7fdde9400700 1 --2- 172.21.15.37:0/433684231 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdde4102e00 0x7fdde40702c0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fddd4009360 tx=0x7fddd4009e30 comp rx=0 tx=0).stop 2024-03-29T08:34:55.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:55.446+0000 7fdde9400700 1 -- 172.21.15.37:0/433684231 shutdown_connections 2024-03-29T08:34:55.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:55.446+0000 7fdde9400700 1 --2- 172.21.15.37:0/433684231 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fddc806cc30 0x7fddc806f0e0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:34:55.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:55.446+0000 7fdde9400700 1 --2- 172.21.15.37:0/433684231 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdde4102e00 0x7fdde40702c0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:34:55.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:55.446+0000 7fdde9400700 1 --2- 172.21.15.37:0/433684231 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdde41037a0 0x7fdde4070800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:34:55.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:55.446+0000 7fdde9400700 1 -- 172.21.15.37:0/433684231 >> 172.21.15.37:0/433684231 conn(0x7fdde40f89b0 msgr2=0x7fdde40f9a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:34:55.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:55.446+0000 7fdde9400700 1 -- 172.21.15.37:0/433684231 shutdown_connections 2024-03-29T08:34:55.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:55.446+0000 7fdde9400700 1 -- 172.21.15.37:0/433684231 wait complete. 2024-03-29T08:34:55.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:55.583+0000 7f41eb000700 1 -- 172.21.15.37:0/1377680500 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f41e4070300 msgr2=0x7f41e40706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:34:55.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:55.583+0000 7f41eb000700 1 --2- 172.21.15.37:0/1377680500 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f41e4070300 0x7f41e40706d0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f41d4004970 tx=0x7f41d4009e30 comp rx=0 tx=0).stop 2024-03-29T08:34:55.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:55.584+0000 7f41eb000700 1 -- 172.21.15.37:0/1377680500 shutdown_connections 2024-03-29T08:34:55.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:55.584+0000 7f41eb000700 1 --2- 172.21.15.37:0/1377680500 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f41e4070c10 0x7f41e4109190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:34:55.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:55.584+0000 7f41eb000700 1 --2- 172.21.15.37:0/1377680500 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f41e4070300 0x7f41e40706d0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:34:55.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:55.584+0000 7f41eb000700 1 -- 172.21.15.37:0/1377680500 >> 172.21.15.37:0/1377680500 conn(0x7f41e40f8970 msgr2=0x7f41e40fad80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:34:55.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:55.584+0000 7f41eb000700 1 -- 172.21.15.37:0/1377680500 shutdown_connections 2024-03-29T08:34:55.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:55.584+0000 7f41eb000700 1 -- 172.21.15.37:0/1377680500 wait complete. 2024-03-29T08:34:55.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:55.585+0000 7f41eb000700 1 Processor -- start 2024-03-29T08:34:55.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:55.585+0000 7f41eb000700 1 -- start start 2024-03-29T08:34:55.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:55.585+0000 7f41eb000700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f41e4070300 0x7f41e4115d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:34:55.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:55.585+0000 7f41eb000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f41e4070c10 0x7f41e4116250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:34:55.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:55.585+0000 7f41eb000700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41e4116930 con 0x7f41e4070c10 2024-03-29T08:34:55.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:55.585+0000 7f41eb000700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41e410fcb0 con 0x7f41e4070300 2024-03-29T08:34:55.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:55.585+0000 7f41e3400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f41e4070c10 0x7f41e4116250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:34:55.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:55.585+0000 7f41e3400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f41e4070c10 0x7f41e4116250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39350/0 (socket says 172.21.15.37:39350) 2024-03-29T08:34:55.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:55.585+0000 7f41e3400700 1 -- 172.21.15.37:0/3758600277 learned_addr learned my addr 172.21.15.37:0/3758600277 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:34:55.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:55.586+0000 7f41e3e00700 1 --2- 172.21.15.37:0/3758600277 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f41e4070300 0x7f41e4115d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:34:55.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:55.586+0000 7f41e3400700 1 -- 172.21.15.37:0/3758600277 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f41e4070300 msgr2=0x7f41e4115d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:34:55.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:55.586+0000 7f41e3400700 1 --2- 172.21.15.37:0/3758600277 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f41e4070300 0x7f41e4115d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:34:55.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:55.586+0000 7f41e3400700 1 -- 172.21.15.37:0/3758600277 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41d4005040 con 0x7f41e4070c10 2024-03-29T08:34:55.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:55.586+0000 7f41e3400700 1 --2- 172.21.15.37:0/3758600277 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f41e4070c10 0x7f41e4116250 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f41d000b790 tx=0x7f41d0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:34:55.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:55.586+0000 7f41dbe00700 1 -- 172.21.15.37:0/3758600277 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f41d000f770 con 0x7f41e4070c10 2024-03-29T08:34:55.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:55.586+0000 7f41dbe00700 1 -- 172.21.15.37:0/3758600277 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f41d000fd80 con 0x7f41e4070c10 2024-03-29T08:34:55.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:55.586+0000 7f41dbe00700 1 -- 172.21.15.37:0/3758600277 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f41d000d310 con 0x7f41e4070c10 2024-03-29T08:34:55.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:55.586+0000 7f41eb000700 1 -- 172.21.15.37:0/3758600277 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41e410ff90 con 0x7f41e4070c10 2024-03-29T08:34:55.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:55.587+0000 7f41eb000700 1 -- 172.21.15.37:0/3758600277 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41e41104d0 con 0x7f41e4070c10 2024-03-29T08:34:55.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:55.588+0000 7f41eb000700 1 -- 172.21.15.37:0/3758600277 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41e4071a40 con 0x7f41e4070c10 2024-03-29T08:34:55.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:55.589+0000 7f41dbe00700 1 -- 172.21.15.37:0/3758600277 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f41d0010420 con 0x7f41e4070c10 2024-03-29T08:34:55.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:55.590+0000 7f41dbe00700 1 --2- 172.21.15.37:0/3758600277 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f41c806cc30 0x7f41c806f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:34:55.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:55.590+0000 7f41dbe00700 1 -- 172.21.15.37:0/3758600277 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f41d008bfe0 con 0x7f41e4070c10 2024-03-29T08:34:55.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:55.590+0000 7f41e3e00700 1 --2- 172.21.15.37:0/3758600277 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f41c806cc30 0x7f41c806f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:34:55.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:55.591+0000 7f41e3e00700 1 --2- 172.21.15.37:0/3758600277 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f41c806cc30 0x7f41c806f0e0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f41d4004ee0 tx=0x7f41d4009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:34:55.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:55.595+0000 7f41dbe00700 1 -- 172.21.15.37:0/3758600277 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f41d00bf9f0 con 0x7f41e4070c10 2024-03-29T08:34:55.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:34:55 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:34:55.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:34:55 smithi148 ceph-mon[106692]: from='client.35318 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:34:55.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:55.891+0000 7f41eb000700 1 -- 172.21.15.37:0/3758600277 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f41e405f5f0 con 0x7f41e4070c10 2024-03-29T08:34:55.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:55.892+0000 7f41dbe00700 1 -- 172.21.15.37:0/3758600277 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7f41d0056020 con 0x7f41e4070c10 2024-03-29T08:34:55.892 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-29T08:34:55.892 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": { 2024-03-29T08:34:55.892 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 2 2024-03-29T08:34:55.892 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:34:55.893 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": { 2024-03-29T08:34:55.893 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 2 2024-03-29T08:34:55.893 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:34:55.893 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": { 2024-03-29T08:34:55.893 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 6 2024-03-29T08:34:55.893 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:34:55.893 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": { 2024-03-29T08:34:55.893 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 4 2024-03-29T08:34:55.893 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:34:55.893 INFO:teuthology.orchestra.run.smithi037.stdout: "overall": { 2024-03-29T08:34:55.893 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 14 2024-03-29T08:34:55.893 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-29T08:34:55.893 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-29T08:34:55.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:55.894+0000 7f41eb000700 1 -- 172.21.15.37:0/3758600277 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f41c806cc30 msgr2=0x7f41c806f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:34:55.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:55.894+0000 7f41eb000700 1 --2- 172.21.15.37:0/3758600277 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f41c806cc30 0x7f41c806f0e0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f41d4004ee0 tx=0x7f41d4009e30 comp rx=0 tx=0).stop 2024-03-29T08:34:55.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:55.894+0000 7f41eb000700 1 -- 172.21.15.37:0/3758600277 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f41e4070c10 msgr2=0x7f41e4116250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:34:55.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:55.894+0000 7f41eb000700 1 --2- 172.21.15.37:0/3758600277 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f41e4070c10 0x7f41e4116250 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f41d000b790 tx=0x7f41d0009250 comp rx=0 tx=0).stop 2024-03-29T08:34:55.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:55.894+0000 7f41eb000700 1 -- 172.21.15.37:0/3758600277 shutdown_connections 2024-03-29T08:34:55.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:55.894+0000 7f41eb000700 1 --2- 172.21.15.37:0/3758600277 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f41c806cc30 0x7f41c806f0e0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:34:55.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:55.894+0000 7f41eb000700 1 --2- 172.21.15.37:0/3758600277 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f41e4070300 0x7f41e4115d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:34:55.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:55.894+0000 7f41eb000700 1 --2- 172.21.15.37:0/3758600277 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f41e4070c10 0x7f41e4116250 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:34:55.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:55.894+0000 7f41eb000700 1 -- 172.21.15.37:0/3758600277 >> 172.21.15.37:0/3758600277 conn(0x7f41e40f8970 msgr2=0x7f41e40f9ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:34:55.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:55.895+0000 7f41eb000700 1 -- 172.21.15.37:0/3758600277 shutdown_connections 2024-03-29T08:34:55.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:55.895+0000 7f41eb000700 1 -- 172.21.15.37:0/3758600277 wait complete. 2024-03-29T08:34:55.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:34:55 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:34:55.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:34:55 smithi037 ceph-mon[142325]: from='client.35318 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:34:56.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:56.032+0000 7f1f0de00700 1 -- 172.21.15.37:0/2222025314 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1f080fe830 msgr2=0x7f1f08106f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:34:56.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:56.032+0000 7f1f0de00700 1 --2- 172.21.15.37:0/2222025314 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1f080fe830 0x7f1f08106f00 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f1ef4004940 tx=0x7f1ef4009e30 comp rx=0 tx=0).stop 2024-03-29T08:34:56.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:56.032+0000 7f1f0de00700 1 -- 172.21.15.37:0/2222025314 shutdown_connections 2024-03-29T08:34:56.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:56.032+0000 7f1f0de00700 1 --2- 172.21.15.37:0/2222025314 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1f080fe830 0x7f1f08106f00 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:34:56.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:56.032+0000 7f1f0de00700 1 --2- 172.21.15.37:0/2222025314 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1f080fde90 0x7f1f080fe260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:34:56.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:56.032+0000 7f1f0de00700 1 -- 172.21.15.37:0/2222025314 >> 172.21.15.37:0/2222025314 conn(0x7f1f08072bc0 msgr2=0x7f1f08072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:34:56.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:56.033+0000 7f1f0de00700 1 -- 172.21.15.37:0/2222025314 shutdown_connections 2024-03-29T08:34:56.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:56.033+0000 7f1f0de00700 1 -- 172.21.15.37:0/2222025314 wait complete. 2024-03-29T08:34:56.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:56.033+0000 7f1f0de00700 1 Processor -- start 2024-03-29T08:34:56.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:56.034+0000 7f1f0de00700 1 -- start start 2024-03-29T08:34:56.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:56.034+0000 7f1f0de00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1f080fde90 0x7f1f08115d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:34:56.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:56.034+0000 7f1f0de00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1f080fe830 0x7f1f08116270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:34:56.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:56.034+0000 7f1f0de00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f081168c0 con 0x7f1f080fe830 2024-03-29T08:34:56.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:56.034+0000 7f1f0de00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f0810fc90 con 0x7f1f080fde90 2024-03-29T08:34:56.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:56.034+0000 7f1f06000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1f080fe830 0x7f1f08116270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:34:56.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:56.034+0000 7f1f06000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1f080fe830 0x7f1f08116270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39370/0 (socket says 172.21.15.37:39370) 2024-03-29T08:34:56.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:56.034+0000 7f1f06000700 1 -- 172.21.15.37:0/2241272656 learned_addr learned my addr 172.21.15.37:0/2241272656 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:34:56.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:56.034+0000 7f1f06a00700 1 --2- 172.21.15.37:0/2241272656 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1f080fde90 0x7f1f08115d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:34:56.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:56.035+0000 7f1f06000700 1 -- 172.21.15.37:0/2241272656 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1f080fde90 msgr2=0x7f1f08115d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:34:56.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:56.035+0000 7f1f06000700 1 --2- 172.21.15.37:0/2241272656 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1f080fde90 0x7f1f08115d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:34:56.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:56.035+0000 7f1f06000700 1 -- 172.21.15.37:0/2241272656 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ef4005040 con 0x7f1f080fe830 2024-03-29T08:34:56.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:56.035+0000 7f1f06000700 1 --2- 172.21.15.37:0/2241272656 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1f080fe830 0x7f1f08116270 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f1ef4000c00 tx=0x7f1ef4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:34:56.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:56.035+0000 7f1efea00700 1 -- 172.21.15.37:0/2241272656 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1ef400e040 con 0x7f1f080fe830 2024-03-29T08:34:56.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:56.035+0000 7f1efea00700 1 -- 172.21.15.37:0/2241272656 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f1ef4013420 con 0x7f1f080fe830 2024-03-29T08:34:56.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:56.035+0000 7f1efea00700 1 -- 172.21.15.37:0/2241272656 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1ef401c630 con 0x7f1f080fe830 2024-03-29T08:34:56.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:56.035+0000 7f1f0de00700 1 -- 172.21.15.37:0/2241272656 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f0810ff10 con 0x7f1f080fe830 2024-03-29T08:34:56.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:56.035+0000 7f1f0de00700 1 -- 172.21.15.37:0/2241272656 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f08110450 con 0x7f1f080fe830 2024-03-29T08:34:56.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:56.036+0000 7f1f0de00700 1 -- 172.21.15.37:0/2241272656 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f08106a20 con 0x7f1f080fe830 2024-03-29T08:34:56.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:56.037+0000 7f1efea00700 1 -- 172.21.15.37:0/2241272656 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f1ef4013970 con 0x7f1f080fe830 2024-03-29T08:34:56.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:56.038+0000 7f1efea00700 1 --2- 172.21.15.37:0/2241272656 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f1eec06cc30 0x7f1eec06f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:34:56.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:56.038+0000 7f1efea00700 1 -- 172.21.15.37:0/2241272656 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f1ef408d260 con 0x7f1f080fe830 2024-03-29T08:34:56.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:56.038+0000 7f1f06a00700 1 --2- 172.21.15.37:0/2241272656 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f1eec06cc30 0x7f1eec06f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:34:56.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:56.039+0000 7f1f06a00700 1 --2- 172.21.15.37:0/2241272656 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f1eec06cc30 0x7f1eec06f0e0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f1ef8007fd0 tx=0x7f1ef8009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:34:56.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:56.043+0000 7f1efea00700 1 -- 172.21.15.37:0/2241272656 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f1ef405ad30 con 0x7f1f080fe830 2024-03-29T08:34:56.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:56.300+0000 7f1f0de00700 1 -- 172.21.15.37:0/2241272656 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f1f08111380 con 0x7f1f080fe830 2024-03-29T08:34:56.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:56.300+0000 7f1efea00700 1 -- 172.21.15.37:0/2241272656 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 30 v30) v1 ==== 76+0+1841 (secure 0 0 0) 0x7f1ef405a8c0 con 0x7f1f080fe830 2024-03-29T08:34:56.300 INFO:teuthology.orchestra.run.smithi037.stdout:e30 2024-03-29T08:34:56.301 INFO:teuthology.orchestra.run.smithi037.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-29T08:34:56.301 INFO:teuthology.orchestra.run.smithi037.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T08:34:56.301 INFO:teuthology.orchestra.run.smithi037.stdout:legacy client fscid: 1 2024-03-29T08:34:56.301 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:34:56.301 INFO:teuthology.orchestra.run.smithi037.stdout:Filesystem 'cephfs' (1) 2024-03-29T08:34:56.301 INFO:teuthology.orchestra.run.smithi037.stdout:fs_name cephfs 2024-03-29T08:34:56.301 INFO:teuthology.orchestra.run.smithi037.stdout:epoch 21 2024-03-29T08:34:56.301 INFO:teuthology.orchestra.run.smithi037.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-03-29T08:34:56.301 INFO:teuthology.orchestra.run.smithi037.stdout:created 2024-03-29T08:03:29.241269+0000 2024-03-29T08:34:56.301 INFO:teuthology.orchestra.run.smithi037.stdout:modified 2024-03-29T08:17:16.237238+0000 2024-03-29T08:34:56.301 INFO:teuthology.orchestra.run.smithi037.stdout:tableserver 0 2024-03-29T08:34:56.301 INFO:teuthology.orchestra.run.smithi037.stdout:root 0 2024-03-29T08:34:56.301 INFO:teuthology.orchestra.run.smithi037.stdout:session_timeout 60 2024-03-29T08:34:56.301 INFO:teuthology.orchestra.run.smithi037.stdout:session_autoclose 300 2024-03-29T08:34:56.302 INFO:teuthology.orchestra.run.smithi037.stdout:max_file_size 1099511627776 2024-03-29T08:34:56.302 INFO:teuthology.orchestra.run.smithi037.stdout:max_xattr_size 65536 2024-03-29T08:34:56.302 INFO:teuthology.orchestra.run.smithi037.stdout:required_client_features {} 2024-03-29T08:34:56.302 INFO:teuthology.orchestra.run.smithi037.stdout:last_failure 0 2024-03-29T08:34:56.302 INFO:teuthology.orchestra.run.smithi037.stdout:last_failure_osd_epoch 82 2024-03-29T08:34:56.302 INFO:teuthology.orchestra.run.smithi037.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T08:34:56.302 INFO:teuthology.orchestra.run.smithi037.stdout:max_mds 2 2024-03-29T08:34:56.302 INFO:teuthology.orchestra.run.smithi037.stdout:in 0,1 2024-03-29T08:34:56.302 INFO:teuthology.orchestra.run.smithi037.stdout:up {} 2024-03-29T08:34:56.302 INFO:teuthology.orchestra.run.smithi037.stdout:failed 0,1 2024-03-29T08:34:56.302 INFO:teuthology.orchestra.run.smithi037.stdout:damaged 2024-03-29T08:34:56.302 INFO:teuthology.orchestra.run.smithi037.stdout:stopped 2024-03-29T08:34:56.302 INFO:teuthology.orchestra.run.smithi037.stdout:data_pools [3] 2024-03-29T08:34:56.302 INFO:teuthology.orchestra.run.smithi037.stdout:metadata_pool 2 2024-03-29T08:34:56.303 INFO:teuthology.orchestra.run.smithi037.stdout:inline_data disabled 2024-03-29T08:34:56.303 INFO:teuthology.orchestra.run.smithi037.stdout:balancer 2024-03-29T08:34:56.303 INFO:teuthology.orchestra.run.smithi037.stdout:bal_rank_mask -1 2024-03-29T08:34:56.303 INFO:teuthology.orchestra.run.smithi037.stdout:standby_count_wanted 1 2024-03-29T08:34:56.303 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:34:56.303 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:34:56.303 INFO:teuthology.orchestra.run.smithi037.stdout:Standby daemons: 2024-03-29T08:34:56.303 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:34:56.303 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi037.mmwgxe{-1:34440} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.37:6828/1248142005,v1:172.21.15.37:6829/1248142005] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:34:56.303 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi037.iifsmh{-1:34444} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.37:6826/2060882959,v1:172.21.15.37:6827/2060882959] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:34:56.303 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi148.sqteih{-1:44395} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.148:6824/4078652242,v1:172.21.15.148:6825/4078652242] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:34:56.303 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi148.qukpjh{-1:45007} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.148:6826/2394751105,v1:172.21.15.148:6827/2394751105] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:34:56.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:56.303+0000 7f1f0de00700 1 -- 172.21.15.37:0/2241272656 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f1eec06cc30 msgr2=0x7f1eec06f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:34:56.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:56.303+0000 7f1f0de00700 1 --2- 172.21.15.37:0/2241272656 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f1eec06cc30 0x7f1eec06f0e0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f1ef8007fd0 tx=0x7f1ef8009040 comp rx=0 tx=0).stop 2024-03-29T08:34:56.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:56.303+0000 7f1f0de00700 1 -- 172.21.15.37:0/2241272656 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1f080fe830 msgr2=0x7f1f08116270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:34:56.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:56.303+0000 7f1f0de00700 1 --2- 172.21.15.37:0/2241272656 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1f080fe830 0x7f1f08116270 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f1ef4000c00 tx=0x7f1ef4009e30 comp rx=0 tx=0).stop 2024-03-29T08:34:56.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:56.303+0000 7f1f0de00700 1 -- 172.21.15.37:0/2241272656 shutdown_connections 2024-03-29T08:34:56.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:56.303+0000 7f1f0de00700 1 --2- 172.21.15.37:0/2241272656 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f1eec06cc30 0x7f1eec06f0e0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:34:56.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:56.303+0000 7f1f0de00700 1 --2- 172.21.15.37:0/2241272656 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1f080fde90 0x7f1f08115d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:34:56.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:56.303+0000 7f1f0de00700 1 --2- 172.21.15.37:0/2241272656 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1f080fe830 0x7f1f08116270 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:34:56.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:56.303+0000 7f1f0de00700 1 -- 172.21.15.37:0/2241272656 >> 172.21.15.37:0/2241272656 conn(0x7f1f08072bc0 msgr2=0x7f1f080fb840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:34:56.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:56.303+0000 7f1f0de00700 1 -- 172.21.15.37:0/2241272656 shutdown_connections 2024-03-29T08:34:56.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:56.303+0000 7f1f0de00700 1 -- 172.21.15.37:0/2241272656 wait complete. 2024-03-29T08:34:56.305 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 30 2024-03-29T08:34:56.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:56.440+0000 7fe891c00700 1 -- 172.21.15.37:0/382245240 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe88c102e00 msgr2=0x7fe88c1031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:34:56.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:56.440+0000 7fe891c00700 1 --2- 172.21.15.37:0/382245240 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe88c102e00 0x7fe88c1031d0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fe87c004940 tx=0x7fe87c009e30 comp rx=0 tx=0).stop 2024-03-29T08:34:56.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:56.440+0000 7fe891c00700 1 -- 172.21.15.37:0/382245240 shutdown_connections 2024-03-29T08:34:56.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:56.440+0000 7fe891c00700 1 --2- 172.21.15.37:0/382245240 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe88c1037a0 0x7fe88c0fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:34:56.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:56.440+0000 7fe891c00700 1 --2- 172.21.15.37:0/382245240 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe88c102e00 0x7fe88c1031d0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:34:56.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:56.440+0000 7fe891c00700 1 -- 172.21.15.37:0/382245240 >> 172.21.15.37:0/382245240 conn(0x7fe88c0f89b0 msgr2=0x7fe88c0fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:34:56.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:56.441+0000 7fe891c00700 1 -- 172.21.15.37:0/382245240 shutdown_connections 2024-03-29T08:34:56.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:56.441+0000 7fe891c00700 1 -- 172.21.15.37:0/382245240 wait complete. 2024-03-29T08:34:56.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:56.441+0000 7fe891c00700 1 Processor -- start 2024-03-29T08:34:56.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:56.442+0000 7fe891c00700 1 -- start start 2024-03-29T08:34:56.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:56.442+0000 7fe891c00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe88c102e00 0x7fe88c0702c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:34:56.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:56.442+0000 7fe891c00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe88c1037a0 0x7fe88c070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:34:56.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:56.442+0000 7fe891c00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe88c070d90 con 0x7fe88c1037a0 2024-03-29T08:34:56.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:56.442+0000 7fe891c00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe88c070ed0 con 0x7fe88c102e00 2024-03-29T08:34:56.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:56.442+0000 7fe88a000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe88c1037a0 0x7fe88c070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:34:56.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:56.442+0000 7fe88a000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe88c1037a0 0x7fe88c070800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39380/0 (socket says 172.21.15.37:39380) 2024-03-29T08:34:56.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:56.442+0000 7fe88a000700 1 -- 172.21.15.37:0/431814356 learned_addr learned my addr 172.21.15.37:0/431814356 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:34:56.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:56.442+0000 7fe88aa00700 1 --2- 172.21.15.37:0/431814356 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe88c102e00 0x7fe88c0702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:34:56.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:56.443+0000 7fe88a000700 1 -- 172.21.15.37:0/431814356 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe88c102e00 msgr2=0x7fe88c0702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:34:56.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:56.443+0000 7fe88a000700 1 --2- 172.21.15.37:0/431814356 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe88c102e00 0x7fe88c0702c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:34:56.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:56.443+0000 7fe88a000700 1 -- 172.21.15.37:0/431814356 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe87c005040 con 0x7fe88c1037a0 2024-03-29T08:34:56.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:56.443+0000 7fe88a000700 1 --2- 172.21.15.37:0/431814356 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe88c1037a0 0x7fe88c070800 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fe878002a50 tx=0x7fe8780046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:34:56.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:56.443+0000 7fe882a00700 1 -- 172.21.15.37:0/431814356 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe878007cc0 con 0x7fe88c1037a0 2024-03-29T08:34:56.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:56.443+0000 7fe891c00700 1 -- 172.21.15.37:0/431814356 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe88c119d40 con 0x7fe88c1037a0 2024-03-29T08:34:56.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:56.443+0000 7fe882a00700 1 -- 172.21.15.37:0/431814356 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fe87800f040 con 0x7fe88c1037a0 2024-03-29T08:34:56.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:56.443+0000 7fe882a00700 1 -- 172.21.15.37:0/431814356 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe8780095d0 con 0x7fe88c1037a0 2024-03-29T08:34:56.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:56.443+0000 7fe891c00700 1 -- 172.21.15.37:0/431814356 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe88c11a1c0 con 0x7fe88c1037a0 2024-03-29T08:34:56.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:56.444+0000 7fe891c00700 1 -- 172.21.15.37:0/431814356 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe88c0fe550 con 0x7fe88c1037a0 2024-03-29T08:34:56.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:56.445+0000 7fe882a00700 1 -- 172.21.15.37:0/431814356 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fe878009730 con 0x7fe88c1037a0 2024-03-29T08:34:56.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:56.446+0000 7fe882a00700 1 --2- 172.21.15.37:0/431814356 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fe87006cc20 0x7fe87006f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:34:56.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:56.446+0000 7fe882a00700 1 -- 172.21.15.37:0/431814356 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fe87808bdb0 con 0x7fe88c1037a0 2024-03-29T08:34:56.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:56.447+0000 7fe88aa00700 1 --2- 172.21.15.37:0/431814356 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fe87006cc20 0x7fe87006f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:34:56.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:56.447+0000 7fe88aa00700 1 --2- 172.21.15.37:0/431814356 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fe87006cc20 0x7fe87006f0d0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fe87c004f90 tx=0x7fe87c009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:34:56.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:56.451+0000 7fe882a00700 1 -- 172.21.15.37:0/431814356 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fe878055cf0 con 0x7fe88c1037a0 2024-03-29T08:34:56.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:56.684+0000 7fe891c00700 1 -- 172.21.15.37:0/431814356 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe88c04bed0 con 0x7fe87006cc20 2024-03-29T08:34:56.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:56.685+0000 7fe882a00700 1 -- 172.21.15.37:0/431814356 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fe88c04bed0 con 0x7fe87006cc20 2024-03-29T08:34:56.685 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-29T08:34:56.685 INFO:teuthology.orchestra.run.smithi037.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59", 2024-03-29T08:34:56.685 INFO:teuthology.orchestra.run.smithi037.stdout: "in_progress": true, 2024-03-29T08:34:56.685 INFO:teuthology.orchestra.run.smithi037.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-29T08:34:56.685 INFO:teuthology.orchestra.run.smithi037.stdout: "services_complete": [ 2024-03-29T08:34:56.685 INFO:teuthology.orchestra.run.smithi037.stdout: "crash", 2024-03-29T08:34:56.685 INFO:teuthology.orchestra.run.smithi037.stdout: "osd", 2024-03-29T08:34:56.685 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr", 2024-03-29T08:34:56.685 INFO:teuthology.orchestra.run.smithi037.stdout: "mds", 2024-03-29T08:34:56.685 INFO:teuthology.orchestra.run.smithi037.stdout: "mon" 2024-03-29T08:34:56.685 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-29T08:34:56.686 INFO:teuthology.orchestra.run.smithi037.stdout: "progress": "16/21 daemons upgraded", 2024-03-29T08:34:56.686 INFO:teuthology.orchestra.run.smithi037.stdout: "message": "Currently upgrading mds daemons", 2024-03-29T08:34:56.686 INFO:teuthology.orchestra.run.smithi037.stdout: "is_paused": false 2024-03-29T08:34:56.686 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-29T08:34:56.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:56.687+0000 7fe891c00700 1 -- 172.21.15.37:0/431814356 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fe87006cc20 msgr2=0x7fe87006f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:34:56.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:56.687+0000 7fe891c00700 1 --2- 172.21.15.37:0/431814356 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fe87006cc20 0x7fe87006f0d0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fe87c004f90 tx=0x7fe87c009e30 comp rx=0 tx=0).stop 2024-03-29T08:34:56.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:56.687+0000 7fe891c00700 1 -- 172.21.15.37:0/431814356 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe88c1037a0 msgr2=0x7fe88c070800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:34:56.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:56.687+0000 7fe891c00700 1 --2- 172.21.15.37:0/431814356 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe88c1037a0 0x7fe88c070800 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fe878002a50 tx=0x7fe8780046b0 comp rx=0 tx=0).stop 2024-03-29T08:34:56.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:56.687+0000 7fe891c00700 1 -- 172.21.15.37:0/431814356 shutdown_connections 2024-03-29T08:34:56.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:56.687+0000 7fe891c00700 1 --2- 172.21.15.37:0/431814356 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fe87006cc20 0x7fe87006f0d0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:34:56.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:56.687+0000 7fe891c00700 1 --2- 172.21.15.37:0/431814356 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe88c102e00 0x7fe88c0702c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:34:56.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:56.687+0000 7fe891c00700 1 --2- 172.21.15.37:0/431814356 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe88c1037a0 0x7fe88c070800 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:34:56.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:56.687+0000 7fe891c00700 1 -- 172.21.15.37:0/431814356 >> 172.21.15.37:0/431814356 conn(0x7fe88c0f89b0 msgr2=0x7fe88c0f99f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:34:56.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:56.688+0000 7fe891c00700 1 -- 172.21.15.37:0/431814356 shutdown_connections 2024-03-29T08:34:56.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:56.688+0000 7fe891c00700 1 -- 172.21.15.37:0/431814356 wait complete. 2024-03-29T08:34:56.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:34:56 smithi148 ceph-mon[106692]: from='client.35322 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:34:56.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:34:56 smithi148 ceph-mon[106692]: pgmap v722: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:34:56.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:34:56 smithi148 ceph-mon[106692]: from='client.35326 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:34:56.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:34:56 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/3758600277' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:34:56.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:34:56 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/2241272656' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T08:34:56.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:56.825+0000 7f2c06200700 1 -- 172.21.15.37:0/3630285583 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2c00071f90 msgr2=0x7f2c00070360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:34:56.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:56.825+0000 7f2c06200700 1 --2- 172.21.15.37:0/3630285583 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2c00071f90 0x7f2c00070360 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f2bf0004970 tx=0x7f2bf0009e30 comp rx=0 tx=0).stop 2024-03-29T08:34:56.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:56.825+0000 7f2c06200700 1 -- 172.21.15.37:0/3630285583 shutdown_connections 2024-03-29T08:34:56.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:56.825+0000 7f2c06200700 1 --2- 172.21.15.37:0/3630285583 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2c000708a0 0x7f2c00070d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:34:56.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:56.825+0000 7f2c06200700 1 --2- 172.21.15.37:0/3630285583 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2c00071f90 0x7f2c00070360 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:34:56.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:56.825+0000 7f2c06200700 1 -- 172.21.15.37:0/3630285583 >> 172.21.15.37:0/3630285583 conn(0x7f2c000f8970 msgr2=0x7f2c000fad80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:34:56.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:56.825+0000 7f2c06200700 1 -- 172.21.15.37:0/3630285583 shutdown_connections 2024-03-29T08:34:56.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:56.825+0000 7f2c06200700 1 -- 172.21.15.37:0/3630285583 wait complete. 2024-03-29T08:34:56.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:56.826+0000 7f2c06200700 1 Processor -- start 2024-03-29T08:34:56.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:56.826+0000 7f2c06200700 1 -- start start 2024-03-29T08:34:56.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:56.826+0000 7f2c06200700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2c000708a0 0x7f2c0011a140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:34:56.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:56.826+0000 7f2c06200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2c00071f90 0x7f2c0011a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:34:56.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:56.826+0000 7f2c06200700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c0011acd0 con 0x7f2c00071f90 2024-03-29T08:34:56.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:56.826+0000 7f2c06200700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c001140a0 con 0x7f2c000708a0 2024-03-29T08:34:56.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:56.826+0000 7f2bfea00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2c00071f90 0x7f2c0011a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:34:56.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:56.827+0000 7f2bfea00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2c00071f90 0x7f2c0011a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39388/0 (socket says 172.21.15.37:39388) 2024-03-29T08:34:56.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:56.827+0000 7f2bfea00700 1 -- 172.21.15.37:0/2772488361 learned_addr learned my addr 172.21.15.37:0/2772488361 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:34:56.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:56.827+0000 7f2bff400700 1 --2- 172.21.15.37:0/2772488361 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2c000708a0 0x7f2c0011a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:34:56.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:56.827+0000 7f2bfea00700 1 -- 172.21.15.37:0/2772488361 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2c000708a0 msgr2=0x7f2c0011a140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:34:56.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:56.827+0000 7f2bfea00700 1 --2- 172.21.15.37:0/2772488361 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2c000708a0 0x7f2c0011a140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:34:56.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:56.827+0000 7f2bfea00700 1 -- 172.21.15.37:0/2772488361 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bf0005040 con 0x7f2c00071f90 2024-03-29T08:34:56.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:56.827+0000 7f2bfea00700 1 --2- 172.21.15.37:0/2772488361 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2c00071f90 0x7f2c0011a680 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f2bec002a50 tx=0x7f2bec0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:34:56.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:56.827+0000 7f2bf7400700 1 -- 172.21.15.37:0/2772488361 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2bec007cc0 con 0x7f2c00071f90 2024-03-29T08:34:56.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:56.827+0000 7f2bf7400700 1 -- 172.21.15.37:0/2772488361 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f2bec00f040 con 0x7f2c00071f90 2024-03-29T08:34:56.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:56.828+0000 7f2bf7400700 1 -- 172.21.15.37:0/2772488361 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2bec00d9c0 con 0x7f2c00071f90 2024-03-29T08:34:56.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:56.828+0000 7f2c06200700 1 -- 172.21.15.37:0/2772488361 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c00114380 con 0x7f2c00071f90 2024-03-29T08:34:56.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:56.828+0000 7f2c06200700 1 -- 172.21.15.37:0/2772488361 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c001148c0 con 0x7f2c00071f90 2024-03-29T08:34:56.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:56.829+0000 7f2c06200700 1 -- 172.21.15.37:0/2772488361 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c00071aa0 con 0x7f2c00071f90 2024-03-29T08:34:56.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:56.830+0000 7f2bf7400700 1 -- 172.21.15.37:0/2772488361 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f2bec00db20 con 0x7f2c00071f90 2024-03-29T08:34:56.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:56.831+0000 7f2bf7400700 1 --2- 172.21.15.37:0/2772488361 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f2be406cc30 0x7f2be406f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:34:56.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:56.831+0000 7f2bf7400700 1 -- 172.21.15.37:0/2772488361 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f2bec08bde0 con 0x7f2c00071f90 2024-03-29T08:34:56.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:56.831+0000 7f2bff400700 1 --2- 172.21.15.37:0/2772488361 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f2be406cc30 0x7f2be406f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:34:56.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:56.832+0000 7f2bff400700 1 --2- 172.21.15.37:0/2772488361 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f2be406cc30 0x7f2be406f0e0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f2bf000b370 tx=0x7f2bf0009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:34:56.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:56.835+0000 7f2bf7400700 1 -- 172.21.15.37:0/2772488361 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f2bec0597b0 con 0x7f2c00071f90 2024-03-29T08:34:56.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:34:56 smithi037 ceph-mon[142325]: from='client.35322 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:34:56.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:34:56 smithi037 ceph-mon[142325]: pgmap v722: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:34:56.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:34:56 smithi037 ceph-mon[142325]: from='client.35326 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:34:56.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:34:56 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/3758600277' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:34:56.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:34:56 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/2241272656' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T08:34:57.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:57.131+0000 7f2c06200700 1 -- 172.21.15.37:0/2772488361 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f2c0005f5f0 con 0x7f2c00071f90 2024-03-29T08:34:57.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:57.131+0000 7f2bf7400700 1 -- 172.21.15.37:0/2772488361 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+3583 (secure 0 0 0) 0x7f2bec059340 con 0x7f2c00071f90 2024-03-29T08:34:57.131 INFO:teuthology.orchestra.run.smithi037.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem is offline 2024-03-29T08:34:57.131 INFO:teuthology.orchestra.run.smithi037.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-03-29T08:34:57.131 INFO:teuthology.orchestra.run.smithi037.stdout: host smithi148 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/mon.smithi148/config 2024-03-29T08:34:57.131 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/podman: timeout after 895 seconds 2024-03-29T08:34:57.131 INFO:teuthology.orchestra.run.smithi037.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59 -e NODE_NAME=smithi148 -e CEPH_USE_RANDOM_NONCE=1 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4:/var/run/ceph:z -v /var/log/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4:/var/log/ceph:z -v /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpkx2rz2d4:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59 inventory --format=json-pretty --filter-for-batch 2024-03-29T08:34:57.131 INFO:teuthology.orchestra.run.smithi037.stdout:Traceback (most recent call last): 2024-03-29T08:34:57.131 INFO:teuthology.orchestra.run.smithi037.stdout: File "/usr/lib64/python3.6/runpy.py", line 193, in _run_module_as_main 2024-03-29T08:34:57.132 INFO:teuthology.orchestra.run.smithi037.stdout: "__main__", mod_spec) 2024-03-29T08:34:57.132 INFO:teuthology.orchestra.run.smithi037.stdout: File "/usr/lib64/python3.6/runpy.py", line 85, in _run_code 2024-03-29T08:34:57.132 INFO:teuthology.orchestra.run.smithi037.stdout: exec(code, run_globals) 2024-03-29T08:34:57.132 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 10889, in 2024-03-29T08:34:57.132 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 10877, in main 2024-03-29T08:34:57.132 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 2576, in _infer_config 2024-03-29T08:34:57.132 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 2492, in _infer_fsid 2024-03-29T08:34:57.132 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 2604, in _infer_image 2024-03-29T08:34:57.132 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 2479, in _validate_fsid 2024-03-29T08:34:57.132 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 7145, in command_ceph_volume 2024-03-29T08:34:57.132 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 2267, in call_throws 2024-03-29T08:34:57.132 INFO:teuthology.orchestra.run.smithi037.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59 -e NODE_NAME=smithi148 -e CEPH_USE_RANDOM_NONCE=1 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4:/var/run/ceph:z -v /var/log/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4:/var/log/ceph:z -v /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpkx2rz2d4:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59 inventory --format=json-pretty --filter-for-batch 2024-03-29T08:34:57.132 INFO:teuthology.orchestra.run.smithi037.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-03-29T08:34:57.132 INFO:teuthology.orchestra.run.smithi037.stdout: fs cephfs is degraded 2024-03-29T08:34:57.133 INFO:teuthology.orchestra.run.smithi037.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-03-29T08:34:57.133 INFO:teuthology.orchestra.run.smithi037.stdout: fs cephfs is offline because no MDS is active for it. 2024-03-29T08:34:57.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:57.133+0000 7f2c06200700 1 -- 172.21.15.37:0/2772488361 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f2be406cc30 msgr2=0x7f2be406f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:34:57.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:57.133+0000 7f2c06200700 1 --2- 172.21.15.37:0/2772488361 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f2be406cc30 0x7f2be406f0e0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f2bf000b370 tx=0x7f2bf0009e30 comp rx=0 tx=0).stop 2024-03-29T08:34:57.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:57.133+0000 7f2c06200700 1 -- 172.21.15.37:0/2772488361 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2c00071f90 msgr2=0x7f2c0011a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:34:57.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:57.133+0000 7f2c06200700 1 --2- 172.21.15.37:0/2772488361 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2c00071f90 0x7f2c0011a680 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f2bec002a50 tx=0x7f2bec0046b0 comp rx=0 tx=0).stop 2024-03-29T08:34:57.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:57.133+0000 7f2c06200700 1 -- 172.21.15.37:0/2772488361 shutdown_connections 2024-03-29T08:34:57.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:57.133+0000 7f2c06200700 1 --2- 172.21.15.37:0/2772488361 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f2be406cc30 0x7f2be406f0e0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:34:57.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:57.133+0000 7f2c06200700 1 --2- 172.21.15.37:0/2772488361 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2c000708a0 0x7f2c0011a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:34:57.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:57.133+0000 7f2c06200700 1 --2- 172.21.15.37:0/2772488361 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2c00071f90 0x7f2c0011a680 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:34:57.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:57.133+0000 7f2c06200700 1 -- 172.21.15.37:0/2772488361 >> 172.21.15.37:0/2772488361 conn(0x7f2c000f8970 msgr2=0x7f2c000f9b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:34:57.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:57.134+0000 7f2c06200700 1 -- 172.21.15.37:0/2772488361 shutdown_connections 2024-03-29T08:34:57.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:34:57.134+0000 7f2c06200700 1 -- 172.21.15.37:0/2772488361 wait complete. 2024-03-29T08:34:57.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:34:57 smithi148 ceph-mon[106692]: from='client.35338 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:34:57.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:34:57 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/2772488361' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T08:34:57.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:34:57 smithi037 ceph-mon[142325]: from='client.35338 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:34:57.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:34:57 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/2772488361' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T08:34:58.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:34:58 smithi148 ceph-mon[106692]: pgmap v723: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:34:58.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:34:58 smithi037 ceph-mon[142325]: pgmap v723: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:35:00.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:35:00 smithi148 ceph-mon[106692]: pgmap v724: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:35:00.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:35:00 smithi037 ceph-mon[142325]: pgmap v724: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:35:02.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:35:02 smithi148 ceph-mon[106692]: pgmap v725: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:35:02.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:35:02 smithi037 ceph-mon[142325]: pgmap v725: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:35:04.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:35:04 smithi148 ceph-mon[106692]: pgmap v726: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:35:04.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:35:04 smithi037 ceph-mon[142325]: pgmap v726: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:35:06.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:35:06 smithi148 ceph-mon[106692]: pgmap v727: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:35:06.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:35:06 smithi037 ceph-mon[142325]: pgmap v727: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:35:08.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:35:08 smithi148 ceph-mon[106692]: pgmap v728: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:35:08.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:35:08 smithi037 ceph-mon[142325]: pgmap v728: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:35:10.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:35:10 smithi148 ceph-mon[106692]: pgmap v729: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:35:10.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:35:10 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:35:10.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:35:10 smithi037 ceph-mon[142325]: pgmap v729: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:35:10.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:35:10 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:35:12.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:35:12 smithi148 ceph-mon[106692]: pgmap v730: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:35:12.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:35:12 smithi037 ceph-mon[142325]: pgmap v730: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:35:14.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:35:14 smithi148 ceph-mon[106692]: pgmap v731: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:35:14.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:35:14 smithi037 ceph-mon[142325]: pgmap v731: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:35:16.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:35:16 smithi148 ceph-mon[106692]: pgmap v732: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:35:16.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:35:16 smithi037 ceph-mon[142325]: pgmap v732: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:35:18.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:35:18 smithi148 ceph-mon[106692]: pgmap v733: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:35:18.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:35:18 smithi037 ceph-mon[142325]: pgmap v733: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:35:20.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:35:20 smithi148 ceph-mon[106692]: pgmap v734: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:35:20.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:35:20 smithi037 ceph-mon[142325]: pgmap v734: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:35:22.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:35:22 smithi148 ceph-mon[106692]: pgmap v735: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:35:22.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:35:22 smithi037 ceph-mon[142325]: pgmap v735: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:35:24.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:35:24 smithi037 ceph-mon[142325]: pgmap v736: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:35:25.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:35:24 smithi148 ceph-mon[106692]: pgmap v736: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:35:25.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:35:25 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:35:26.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:35:25 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:35:26.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:35:26 smithi037 ceph-mon[142325]: pgmap v737: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:35:27.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:35:26 smithi148 ceph-mon[106692]: pgmap v737: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:35:27.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:27.287+0000 7f74a8600700 1 -- 172.21.15.37:0/3829616898 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f74a01037a0 msgr2=0x7f74a00fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:35:27.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:27.287+0000 7f74a8600700 1 --2- 172.21.15.37:0/3829616898 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f74a01037a0 0x7f74a00fce10 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f748c004940 tx=0x7f748c009e30 comp rx=0 tx=0).stop 2024-03-29T08:35:27.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:27.287+0000 7f74a8600700 1 -- 172.21.15.37:0/3829616898 shutdown_connections 2024-03-29T08:35:27.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:27.287+0000 7f74a8600700 1 --2- 172.21.15.37:0/3829616898 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f74a01037a0 0x7f74a00fce10 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:35:27.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:27.287+0000 7f74a8600700 1 --2- 172.21.15.37:0/3829616898 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f74a0102e00 0x7f74a01031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:35:27.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:27.287+0000 7f74a8600700 1 -- 172.21.15.37:0/3829616898 >> 172.21.15.37:0/3829616898 conn(0x7f74a00f89b0 msgr2=0x7f74a00fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:35:27.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:27.287+0000 7f74a8600700 1 -- 172.21.15.37:0/3829616898 shutdown_connections 2024-03-29T08:35:27.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:27.287+0000 7f74a8600700 1 -- 172.21.15.37:0/3829616898 wait complete. 2024-03-29T08:35:27.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:27.288+0000 7f74a8600700 1 Processor -- start 2024-03-29T08:35:27.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:27.288+0000 7f74a8600700 1 -- start start 2024-03-29T08:35:27.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:27.288+0000 7f74a8600700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f74a0102e00 0x7f74a0115d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:35:27.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:27.288+0000 7f74a8600700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f74a01037a0 0x7f74a0116240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:35:27.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:27.288+0000 7f74a8600700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74a0116890 con 0x7f74a01037a0 2024-03-29T08:35:27.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:27.288+0000 7f74a8600700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74a010fc60 con 0x7f74a0102e00 2024-03-29T08:35:27.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:27.289+0000 7f74a5200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f74a01037a0 0x7f74a0116240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:35:27.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:27.289+0000 7f74a5200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f74a01037a0 0x7f74a0116240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51946/0 (socket says 172.21.15.37:51946) 2024-03-29T08:35:27.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:27.289+0000 7f74a5200700 1 -- 172.21.15.37:0/159799447 learned_addr learned my addr 172.21.15.37:0/159799447 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:35:27.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:27.289+0000 7f74a5c00700 1 --2- 172.21.15.37:0/159799447 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f74a0102e00 0x7f74a0115d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:35:27.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:27.289+0000 7f74a5200700 1 -- 172.21.15.37:0/159799447 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f74a0102e00 msgr2=0x7f74a0115d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:35:27.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:27.289+0000 7f74a5200700 1 --2- 172.21.15.37:0/159799447 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f74a0102e00 0x7f74a0115d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:35:27.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:27.289+0000 7f74a5200700 1 -- 172.21.15.37:0/159799447 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f748c005040 con 0x7f74a01037a0 2024-03-29T08:35:27.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:27.289+0000 7f74a5200700 1 --2- 172.21.15.37:0/159799447 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f74a01037a0 0x7f74a0116240 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f748c009390 tx=0x7f748c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:35:27.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:27.290+0000 7f749d600700 1 -- 172.21.15.37:0/159799447 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f748c00e040 con 0x7f74a01037a0 2024-03-29T08:35:27.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:27.290+0000 7f749d600700 1 -- 172.21.15.37:0/159799447 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f748c013420 con 0x7f74a01037a0 2024-03-29T08:35:27.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:27.290+0000 7f74a8600700 1 -- 172.21.15.37:0/159799447 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74a010fee0 con 0x7f74a01037a0 2024-03-29T08:35:27.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:27.290+0000 7f749d600700 1 -- 172.21.15.37:0/159799447 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f748c01c630 con 0x7f74a01037a0 2024-03-29T08:35:27.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:27.290+0000 7f74a8600700 1 -- 172.21.15.37:0/159799447 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74a01104a0 con 0x7f74a01037a0 2024-03-29T08:35:27.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:27.291+0000 7f74a8600700 1 -- 172.21.15.37:0/159799447 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74a00fe550 con 0x7f74a01037a0 2024-03-29T08:35:27.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:27.292+0000 7f749d600700 1 -- 172.21.15.37:0/159799447 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f748c013a00 con 0x7f74a01037a0 2024-03-29T08:35:27.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:27.293+0000 7f749d600700 1 --2- 172.21.15.37:0/159799447 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f748806cc30 0x7f748806f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:35:27.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:27.293+0000 7f749d600700 1 -- 172.21.15.37:0/159799447 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f748c08d020 con 0x7f74a01037a0 2024-03-29T08:35:27.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:27.293+0000 7f74a5c00700 1 --2- 172.21.15.37:0/159799447 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f748806cc30 0x7f748806f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:35:27.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:27.294+0000 7f74a5c00700 1 --2- 172.21.15.37:0/159799447 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f748806cc30 0x7f748806f0e0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f7494004f40 tx=0x7f749400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:35:27.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:27.298+0000 7f749d600700 1 -- 172.21.15.37:0/159799447 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f748c05abb0 con 0x7f74a01037a0 2024-03-29T08:35:27.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:27.536+0000 7f74a8600700 1 -- 172.21.15.37:0/159799447 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f74a0109e10 con 0x7f748806cc30 2024-03-29T08:35:27.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:27.538+0000 7f749d600700 1 -- 172.21.15.37:0/159799447 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f74a0109e10 con 0x7f748806cc30 2024-03-29T08:35:27.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:27.540+0000 7f74a8600700 1 -- 172.21.15.37:0/159799447 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f748806cc30 msgr2=0x7f748806f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:35:27.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:27.540+0000 7f74a8600700 1 --2- 172.21.15.37:0/159799447 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f748806cc30 0x7f748806f0e0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f7494004f40 tx=0x7f749400b040 comp rx=0 tx=0).stop 2024-03-29T08:35:27.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:27.540+0000 7f74a8600700 1 -- 172.21.15.37:0/159799447 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f74a01037a0 msgr2=0x7f74a0116240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:35:27.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:27.540+0000 7f74a8600700 1 --2- 172.21.15.37:0/159799447 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f74a01037a0 0x7f74a0116240 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f748c009390 tx=0x7f748c009e30 comp rx=0 tx=0).stop 2024-03-29T08:35:27.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:27.540+0000 7f74a8600700 1 -- 172.21.15.37:0/159799447 shutdown_connections 2024-03-29T08:35:27.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:27.540+0000 7f74a8600700 1 --2- 172.21.15.37:0/159799447 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f748806cc30 0x7f748806f0e0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:35:27.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:27.540+0000 7f74a8600700 1 --2- 172.21.15.37:0/159799447 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f74a0102e00 0x7f74a0115d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:35:27.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:27.540+0000 7f74a8600700 1 --2- 172.21.15.37:0/159799447 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f74a01037a0 0x7f74a0116240 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:35:27.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:27.540+0000 7f74a8600700 1 -- 172.21.15.37:0/159799447 >> 172.21.15.37:0/159799447 conn(0x7f74a00f89b0 msgr2=0x7f74a00f9ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:35:27.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:27.540+0000 7f74a8600700 1 -- 172.21.15.37:0/159799447 shutdown_connections 2024-03-29T08:35:27.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:27.540+0000 7f74a8600700 1 -- 172.21.15.37:0/159799447 wait complete. 2024-03-29T08:35:27.555 INFO:teuthology.orchestra.run.smithi037.stdout:true 2024-03-29T08:35:27.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:27.680+0000 7f8118200700 1 -- 172.21.15.37:0/3527803862 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8110102e00 msgr2=0x7f81101031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:35:27.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:27.680+0000 7f8118200700 1 --2- 172.21.15.37:0/3527803862 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8110102e00 0x7f81101031d0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f8104004940 tx=0x7f8104009e30 comp rx=0 tx=0).stop 2024-03-29T08:35:27.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:27.680+0000 7f8118200700 1 -- 172.21.15.37:0/3527803862 shutdown_connections 2024-03-29T08:35:27.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:27.680+0000 7f8118200700 1 --2- 172.21.15.37:0/3527803862 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f81101037a0 0x7f81100fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:35:27.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:27.680+0000 7f8118200700 1 --2- 172.21.15.37:0/3527803862 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8110102e00 0x7f81101031d0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:35:27.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:27.680+0000 7f8118200700 1 -- 172.21.15.37:0/3527803862 >> 172.21.15.37:0/3527803862 conn(0x7f81100f89b0 msgr2=0x7f81100fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:35:27.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:27.681+0000 7f8118200700 1 -- 172.21.15.37:0/3527803862 shutdown_connections 2024-03-29T08:35:27.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:27.681+0000 7f8118200700 1 -- 172.21.15.37:0/3527803862 wait complete. 2024-03-29T08:35:27.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:27.681+0000 7f8118200700 1 Processor -- start 2024-03-29T08:35:27.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:27.681+0000 7f8118200700 1 -- start start 2024-03-29T08:35:27.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:27.682+0000 7f8118200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8110102e00 0x7f8110115cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:35:27.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:27.682+0000 7f8118200700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f81101037a0 0x7f8110116210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:35:27.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:27.682+0000 7f8118200700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8110116860 con 0x7f8110102e00 2024-03-29T08:35:27.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:27.682+0000 7f8118200700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f811010fc30 con 0x7f81101037a0 2024-03-29T08:35:27.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:27.682+0000 7f8115800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8110102e00 0x7f8110115cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:35:27.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:27.682+0000 7f8115800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8110102e00 0x7f8110115cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51968/0 (socket says 172.21.15.37:51968) 2024-03-29T08:35:27.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:27.682+0000 7f8115800700 1 -- 172.21.15.37:0/544946754 learned_addr learned my addr 172.21.15.37:0/544946754 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:35:27.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:27.682+0000 7f8114e00700 1 --2- 172.21.15.37:0/544946754 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f81101037a0 0x7f8110116210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:35:27.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:27.682+0000 7f8115800700 1 -- 172.21.15.37:0/544946754 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f81101037a0 msgr2=0x7f8110116210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:35:27.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:27.682+0000 7f8115800700 1 --2- 172.21.15.37:0/544946754 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f81101037a0 0x7f8110116210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:35:27.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:27.682+0000 7f8115800700 1 -- 172.21.15.37:0/544946754 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8104005040 con 0x7f8110102e00 2024-03-29T08:35:27.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:27.682+0000 7f8114e00700 1 --2- 172.21.15.37:0/544946754 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f81101037a0 0x7f8110116210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:35:27.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:27.683+0000 7f8115800700 1 --2- 172.21.15.37:0/544946754 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8110102e00 0x7f8110115cd0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f8104009390 tx=0x7f8104009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:35:27.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:27.683+0000 7f810d600700 1 -- 172.21.15.37:0/544946754 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f810400e040 con 0x7f8110102e00 2024-03-29T08:35:27.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:27.683+0000 7f810d600700 1 -- 172.21.15.37:0/544946754 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f8104013420 con 0x7f8110102e00 2024-03-29T08:35:27.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:27.683+0000 7f810d600700 1 -- 172.21.15.37:0/544946754 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f810401c630 con 0x7f8110102e00 2024-03-29T08:35:27.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:27.683+0000 7f8118200700 1 -- 172.21.15.37:0/544946754 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f811010feb0 con 0x7f8110102e00 2024-03-29T08:35:27.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:27.683+0000 7f8118200700 1 -- 172.21.15.37:0/544946754 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8110110470 con 0x7f8110102e00 2024-03-29T08:35:27.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:27.684+0000 7f8118200700 1 -- 172.21.15.37:0/544946754 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81100fe550 con 0x7f8110102e00 2024-03-29T08:35:27.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:27.685+0000 7f810d600700 1 -- 172.21.15.37:0/544946754 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f81040139e0 con 0x7f8110102e00 2024-03-29T08:35:27.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:27.687+0000 7f810d600700 1 --2- 172.21.15.37:0/544946754 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f80f806cc30 0x7f80f806f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:35:27.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:27.687+0000 7f810d600700 1 -- 172.21.15.37:0/544946754 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f810408d250 con 0x7f8110102e00 2024-03-29T08:35:27.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:27.687+0000 7f8114e00700 1 --2- 172.21.15.37:0/544946754 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f80f806cc30 0x7f80f806f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:35:27.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:27.688+0000 7f8114e00700 1 --2- 172.21.15.37:0/544946754 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f80f806cc30 0x7f80f806f0e0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f8110111570 tx=0x7f80fc009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:35:27.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:27.691+0000 7f810d600700 1 -- 172.21.15.37:0/544946754 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f810405add0 con 0x7f8110102e00 2024-03-29T08:35:27.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:27.930+0000 7f8118200700 1 -- 172.21.15.37:0/544946754 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f811004bf40 con 0x7f80f806cc30 2024-03-29T08:35:27.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:27.931+0000 7f810d600700 1 -- 172.21.15.37:0/544946754 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f811004bf40 con 0x7f80f806cc30 2024-03-29T08:35:27.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:27.933+0000 7f8118200700 1 -- 172.21.15.37:0/544946754 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f80f806cc30 msgr2=0x7f80f806f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:35:27.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:27.934+0000 7f8118200700 1 --2- 172.21.15.37:0/544946754 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f80f806cc30 0x7f80f806f0e0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f8110111570 tx=0x7f80fc009040 comp rx=0 tx=0).stop 2024-03-29T08:35:27.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:27.934+0000 7f8118200700 1 -- 172.21.15.37:0/544946754 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8110102e00 msgr2=0x7f8110115cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:35:27.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:27.934+0000 7f8118200700 1 --2- 172.21.15.37:0/544946754 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8110102e00 0x7f8110115cd0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f8104009390 tx=0x7f8104009e30 comp rx=0 tx=0).stop 2024-03-29T08:35:27.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:27.934+0000 7f8118200700 1 -- 172.21.15.37:0/544946754 shutdown_connections 2024-03-29T08:35:27.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:27.934+0000 7f8118200700 1 --2- 172.21.15.37:0/544946754 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f80f806cc30 0x7f80f806f0e0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:35:27.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:27.934+0000 7f8118200700 1 --2- 172.21.15.37:0/544946754 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8110102e00 0x7f8110115cd0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:35:27.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:27.934+0000 7f8118200700 1 --2- 172.21.15.37:0/544946754 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f81101037a0 0x7f8110116210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:35:27.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:27.934+0000 7f8118200700 1 -- 172.21.15.37:0/544946754 >> 172.21.15.37:0/544946754 conn(0x7f81100f89b0 msgr2=0x7f81100f99e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:35:27.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:27.934+0000 7f8118200700 1 -- 172.21.15.37:0/544946754 shutdown_connections 2024-03-29T08:35:27.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:27.934+0000 7f8118200700 1 -- 172.21.15.37:0/544946754 wait complete. 2024-03-29T08:35:28.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:28.072+0000 7fefcb600700 1 -- 172.21.15.37:0/4137387623 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fefc4102e00 msgr2=0x7fefc41031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:35:28.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:28.072+0000 7fefcb600700 1 --2- 172.21.15.37:0/4137387623 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fefc4102e00 0x7fefc41031d0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fefb8004940 tx=0x7fefb8009e30 comp rx=0 tx=0).stop 2024-03-29T08:35:28.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:28.072+0000 7fefcb600700 1 -- 172.21.15.37:0/4137387623 shutdown_connections 2024-03-29T08:35:28.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:28.072+0000 7fefcb600700 1 --2- 172.21.15.37:0/4137387623 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fefc41037a0 0x7fefc40fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:35:28.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:28.072+0000 7fefcb600700 1 --2- 172.21.15.37:0/4137387623 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fefc4102e00 0x7fefc41031d0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:35:28.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:28.072+0000 7fefcb600700 1 -- 172.21.15.37:0/4137387623 >> 172.21.15.37:0/4137387623 conn(0x7fefc40f89b0 msgr2=0x7fefc40fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:35:28.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:28.072+0000 7fefcb600700 1 -- 172.21.15.37:0/4137387623 shutdown_connections 2024-03-29T08:35:28.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:28.072+0000 7fefcb600700 1 -- 172.21.15.37:0/4137387623 wait complete. 2024-03-29T08:35:28.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:28.073+0000 7fefcb600700 1 Processor -- start 2024-03-29T08:35:28.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:28.073+0000 7fefcb600700 1 -- start start 2024-03-29T08:35:28.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:28.074+0000 7fefcb600700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fefc4102e00 0x7fefc4115d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:35:28.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:28.074+0000 7fefcb600700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fefc41037a0 0x7fefc4116250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:35:28.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:28.074+0000 7fefcb600700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefc41168a0 con 0x7fefc41037a0 2024-03-29T08:35:28.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:28.074+0000 7fefcb600700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefc410fc70 con 0x7fefc4102e00 2024-03-29T08:35:28.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:28.074+0000 7fefc3e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fefc41037a0 0x7fefc4116250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:35:28.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:28.074+0000 7fefc3e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fefc41037a0 0x7fefc4116250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51986/0 (socket says 172.21.15.37:51986) 2024-03-29T08:35:28.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:28.074+0000 7fefc3e00700 1 -- 172.21.15.37:0/3411444930 learned_addr learned my addr 172.21.15.37:0/3411444930 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:35:28.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:28.074+0000 7fefc8c00700 1 --2- 172.21.15.37:0/3411444930 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fefc4102e00 0x7fefc4115d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:35:28.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:28.074+0000 7fefc3e00700 1 -- 172.21.15.37:0/3411444930 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fefc4102e00 msgr2=0x7fefc4115d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:35:28.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:28.074+0000 7fefc3e00700 1 --2- 172.21.15.37:0/3411444930 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fefc4102e00 0x7fefc4115d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:35:28.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:28.074+0000 7fefc3e00700 1 -- 172.21.15.37:0/3411444930 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefb8005040 con 0x7fefc41037a0 2024-03-29T08:35:28.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:28.075+0000 7fefc3e00700 1 --2- 172.21.15.37:0/3411444930 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fefc41037a0 0x7fefc4116250 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fefb000c890 tx=0x7fefb00044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:35:28.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:28.075+0000 7fefc0c00700 1 -- 172.21.15.37:0/3411444930 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fefb000a700 con 0x7fefc41037a0 2024-03-29T08:35:28.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:28.075+0000 7fefc0c00700 1 -- 172.21.15.37:0/3411444930 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fefb000ad10 con 0x7fefc41037a0 2024-03-29T08:35:28.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:28.075+0000 7fefcb600700 1 -- 172.21.15.37:0/3411444930 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefc410ff50 con 0x7fefc41037a0 2024-03-29T08:35:28.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:28.075+0000 7fefc0c00700 1 -- 172.21.15.37:0/3411444930 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fefb0012e70 con 0x7fefc41037a0 2024-03-29T08:35:28.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:28.075+0000 7fefcb600700 1 -- 172.21.15.37:0/3411444930 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefc4110510 con 0x7fefc41037a0 2024-03-29T08:35:28.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:28.076+0000 7fefcb600700 1 -- 172.21.15.37:0/3411444930 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fefc40fe550 con 0x7fefc41037a0 2024-03-29T08:35:28.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:28.077+0000 7fefc0c00700 1 -- 172.21.15.37:0/3411444930 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fefb000a860 con 0x7fefc41037a0 2024-03-29T08:35:28.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:28.078+0000 7fefc0c00700 1 --2- 172.21.15.37:0/3411444930 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fefac06cc30 0x7fefac06f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:35:28.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:28.078+0000 7fefc0c00700 1 -- 172.21.15.37:0/3411444930 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fefb008c050 con 0x7fefc41037a0 2024-03-29T08:35:28.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:28.079+0000 7fefc8c00700 1 --2- 172.21.15.37:0/3411444930 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fefac06cc30 0x7fefac06f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:35:28.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:28.079+0000 7fefc8c00700 1 --2- 172.21.15.37:0/3411444930 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fefac06cc30 0x7fefac06f0e0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fefb8009fd0 tx=0x7fefb8009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:35:28.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:28.082+0000 7fefc0c00700 1 -- 172.21.15.37:0/3411444930 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fefb0059b20 con 0x7fefc41037a0 2024-03-29T08:35:28.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:28.304+0000 7fefcb600700 1 -- 172.21.15.37:0/3411444930 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fefc4109e30 con 0x7fefac06cc30 2024-03-29T08:35:28.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:28.314+0000 7fefc0c00700 1 -- 172.21.15.37:0/3411444930 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7fefc4109e30 con 0x7fefac06cc30 2024-03-29T08:35:28.314 INFO:teuthology.orchestra.run.smithi037.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-29T08:35:28.314 INFO:teuthology.orchestra.run.smithi037.stdout:alertmanager.smithi037 smithi037 *:9093,9094 running (25m) 2m ago 38m 14.3M - 0.25.0 c8568f914cd2 2f9f9746ed0f 2024-03-29T08:35:28.314 INFO:teuthology.orchestra.run.smithi037.stdout:crash.smithi037 smithi037 running (22m) 2m ago 38m 6647k - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 178d416a3186 2024-03-29T08:35:28.314 INFO:teuthology.orchestra.run.smithi037.stdout:crash.smithi148 smithi148 running (22m) 2m ago 37m 6656k - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 23666989d58f 2024-03-29T08:35:28.314 INFO:teuthology.orchestra.run.smithi037.stdout:grafana.smithi037 smithi037 *:3000 running (24m) 2m ago 37m 74.5M - 9.4.7 954c08fa6188 6b976fe49952 2024-03-29T08:35:28.314 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi037.iifsmh smithi037 running (17m) 2m ago 31m 16.4M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 2d97f562cc5b 2024-03-29T08:35:28.314 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi037.mmwgxe smithi037 running (18m) 2m ago 31m 16.7M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff ec37037f46e8 2024-03-29T08:35:28.314 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi148.qukpjh smithi148 running (2m) 2m ago 31m 10.7M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 3257219bc046 2024-03-29T08:35:28.314 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi148.sqteih smithi148 running (17m) 2m ago 31m 16.3M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 44c15c77037a 2024-03-29T08:35:28.315 INFO:teuthology.orchestra.run.smithi037.stdout:mgr.smithi037.wjikii smithi037 *:8443,9283,8765 running (27m) 2m ago 39m 510M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff c5a1417b9417 2024-03-29T08:35:28.315 INFO:teuthology.orchestra.run.smithi037.stdout:mgr.smithi148.onhgvm smithi148 *:8443,9283,8765 running (26m) 2m ago 37m 456M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff fcfdbce062bd 2024-03-29T08:35:28.315 INFO:teuthology.orchestra.run.smithi037.stdout:mon.smithi037 smithi037 running (23m) 2m ago 40m 70.2M 2048M 18.2.2-1015-gd0e392d9 1cc2f45c17ff eea2060783af 2024-03-29T08:35:28.315 INFO:teuthology.orchestra.run.smithi037.stdout:mon.smithi148 smithi148 running (22m) 2m ago 37m 50.4M 2048M 18.2.2-1015-gd0e392d9 1cc2f45c17ff ad4b2c5748be 2024-03-29T08:35:28.315 INFO:teuthology.orchestra.run.smithi037.stdout:node-exporter.smithi037 smithi037 *:9100 running (26m) 2m ago 38m 8514k - 1.5.0 0da6a335fe13 26b8ae9cb878 2024-03-29T08:35:28.315 INFO:teuthology.orchestra.run.smithi037.stdout:node-exporter.smithi148 smithi148 *:9100 running (26m) 2m ago 37m 8464k - 1.5.0 0da6a335fe13 98414e4e64ed 2024-03-29T08:35:28.315 INFO:teuthology.orchestra.run.smithi037.stdout:osd.0 smithi037 running (21m) 2m ago 36m 108M 1619M 18.2.2-1015-gd0e392d9 1cc2f45c17ff c0c584f6245c 2024-03-29T08:35:28.315 INFO:teuthology.orchestra.run.smithi037.stdout:osd.1 smithi037 running (21m) 2m ago 35m 104M 1619M 18.2.2-1015-gd0e392d9 1cc2f45c17ff a4a2ff4cc83d 2024-03-29T08:35:28.315 INFO:teuthology.orchestra.run.smithi037.stdout:osd.2 smithi037 running (20m) 2m ago 35m 85.9M 1619M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 8b1bc71aa79e 2024-03-29T08:35:28.315 INFO:teuthology.orchestra.run.smithi037.stdout:osd.3 smithi148 running (19m) 2m ago 34m 109M 2642M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 1c656ab8ff86 2024-03-29T08:35:28.315 INFO:teuthology.orchestra.run.smithi037.stdout:osd.4 smithi148 running (19m) 2m ago 34m 101M 2642M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 966a9de11d22 2024-03-29T08:35:28.315 INFO:teuthology.orchestra.run.smithi037.stdout:osd.5 smithi148 running (18m) 2m ago 33m 89.4M 2642M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 474c4f9212ce 2024-03-29T08:35:28.315 INFO:teuthology.orchestra.run.smithi037.stdout:prometheus.smithi037 smithi037 *:9095 running (25m) 2m ago 37m 61.0M - 2.43.0 a07b618ecd1d 30c0223ec589 2024-03-29T08:35:28.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:28.316+0000 7fefcb600700 1 -- 172.21.15.37:0/3411444930 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fefac06cc30 msgr2=0x7fefac06f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:35:28.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:28.316+0000 7fefcb600700 1 --2- 172.21.15.37:0/3411444930 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fefac06cc30 0x7fefac06f0e0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fefb8009fd0 tx=0x7fefb8009e30 comp rx=0 tx=0).stop 2024-03-29T08:35:28.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:28.316+0000 7fefcb600700 1 -- 172.21.15.37:0/3411444930 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fefc41037a0 msgr2=0x7fefc4116250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:35:28.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:28.316+0000 7fefcb600700 1 --2- 172.21.15.37:0/3411444930 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fefc41037a0 0x7fefc4116250 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fefb000c890 tx=0x7fefb00044c0 comp rx=0 tx=0).stop 2024-03-29T08:35:28.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:28.316+0000 7fefcb600700 1 -- 172.21.15.37:0/3411444930 shutdown_connections 2024-03-29T08:35:28.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:28.316+0000 7fefcb600700 1 --2- 172.21.15.37:0/3411444930 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fefac06cc30 0x7fefac06f0e0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:35:28.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:28.316+0000 7fefcb600700 1 --2- 172.21.15.37:0/3411444930 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fefc4102e00 0x7fefc4115d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:35:28.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:28.316+0000 7fefcb600700 1 --2- 172.21.15.37:0/3411444930 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fefc41037a0 0x7fefc4116250 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:35:28.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:28.316+0000 7fefcb600700 1 -- 172.21.15.37:0/3411444930 >> 172.21.15.37:0/3411444930 conn(0x7fefc40f89b0 msgr2=0x7fefc40704f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:35:28.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:28.316+0000 7fefcb600700 1 -- 172.21.15.37:0/3411444930 shutdown_connections 2024-03-29T08:35:28.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:28.316+0000 7fefcb600700 1 -- 172.21.15.37:0/3411444930 wait complete. 2024-03-29T08:35:28.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:28.453+0000 7fab71c00700 1 -- 172.21.15.37:0/984187223 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fab6c1037a0 msgr2=0x7fab6c0fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:35:28.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:28.453+0000 7fab71c00700 1 --2- 172.21.15.37:0/984187223 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fab6c1037a0 0x7fab6c0fce10 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fab58004940 tx=0x7fab58009e30 comp rx=0 tx=0).stop 2024-03-29T08:35:28.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:28.453+0000 7fab71c00700 1 -- 172.21.15.37:0/984187223 shutdown_connections 2024-03-29T08:35:28.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:28.453+0000 7fab71c00700 1 --2- 172.21.15.37:0/984187223 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fab6c1037a0 0x7fab6c0fce10 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:35:28.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:28.453+0000 7fab71c00700 1 --2- 172.21.15.37:0/984187223 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fab6c102e00 0x7fab6c1031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:35:28.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:28.453+0000 7fab71c00700 1 -- 172.21.15.37:0/984187223 >> 172.21.15.37:0/984187223 conn(0x7fab6c0f89b0 msgr2=0x7fab6c0fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:35:28.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:28.454+0000 7fab71c00700 1 -- 172.21.15.37:0/984187223 shutdown_connections 2024-03-29T08:35:28.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:28.454+0000 7fab71c00700 1 -- 172.21.15.37:0/984187223 wait complete. 2024-03-29T08:35:28.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:28.454+0000 7fab71c00700 1 Processor -- start 2024-03-29T08:35:28.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:28.454+0000 7fab71c00700 1 -- start start 2024-03-29T08:35:28.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:28.455+0000 7fab71c00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fab6c102e00 0x7fab6c115d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:35:28.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:28.455+0000 7fab71c00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fab6c1037a0 0x7fab6c116240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:35:28.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:28.455+0000 7fab71c00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab6c116890 con 0x7fab6c102e00 2024-03-29T08:35:28.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:28.455+0000 7fab71c00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab6c10fc60 con 0x7fab6c1037a0 2024-03-29T08:35:28.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:28.455+0000 7fab6aa00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fab6c102e00 0x7fab6c115d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:35:28.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:28.455+0000 7fab6aa00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fab6c102e00 0x7fab6c115d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52000/0 (socket says 172.21.15.37:52000) 2024-03-29T08:35:28.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:28.455+0000 7fab6aa00700 1 -- 172.21.15.37:0/4154741160 learned_addr learned my addr 172.21.15.37:0/4154741160 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:35:28.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:28.456+0000 7fab6aa00700 1 -- 172.21.15.37:0/4154741160 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fab6c1037a0 msgr2=0x7fab6c116240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:35:28.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:28.456+0000 7fab6aa00700 1 --2- 172.21.15.37:0/4154741160 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fab6c1037a0 0x7fab6c116240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:35:28.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:28.456+0000 7fab6aa00700 1 -- 172.21.15.37:0/4154741160 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab58005040 con 0x7fab6c102e00 2024-03-29T08:35:28.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:28.456+0000 7fab6aa00700 1 --2- 172.21.15.37:0/4154741160 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fab6c102e00 0x7fab6c115d00 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fab5c002a50 tx=0x7fab5c0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:35:28.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:28.456+0000 7fab62a00700 1 -- 172.21.15.37:0/4154741160 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fab5c007cc0 con 0x7fab6c102e00 2024-03-29T08:35:28.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:28.456+0000 7fab62a00700 1 -- 172.21.15.37:0/4154741160 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fab5c00f040 con 0x7fab6c102e00 2024-03-29T08:35:28.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:28.456+0000 7fab71c00700 1 -- 172.21.15.37:0/4154741160 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab6c10ff40 con 0x7fab6c102e00 2024-03-29T08:35:28.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:28.456+0000 7fab62a00700 1 -- 172.21.15.37:0/4154741160 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fab5c009540 con 0x7fab6c102e00 2024-03-29T08:35:28.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:28.456+0000 7fab71c00700 1 -- 172.21.15.37:0/4154741160 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab6c110500 con 0x7fab6c102e00 2024-03-29T08:35:28.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:28.457+0000 7fab71c00700 1 -- 172.21.15.37:0/4154741160 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab6c0fe550 con 0x7fab6c102e00 2024-03-29T08:35:28.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:28.458+0000 7fab62a00700 1 -- 172.21.15.37:0/4154741160 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fab5c0096a0 con 0x7fab6c102e00 2024-03-29T08:35:28.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:28.459+0000 7fab62a00700 1 --2- 172.21.15.37:0/4154741160 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fab5006cc30 0x7fab5006f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:35:28.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:28.459+0000 7fab62a00700 1 -- 172.21.15.37:0/4154741160 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fab5c08be60 con 0x7fab6c102e00 2024-03-29T08:35:28.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:28.459+0000 7fab6a000700 1 --2- 172.21.15.37:0/4154741160 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fab5006cc30 0x7fab5006f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:35:28.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:28.460+0000 7fab6a000700 1 --2- 172.21.15.37:0/4154741160 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fab5006cc30 0x7fab5006f0e0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fab58004f90 tx=0x7fab58009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:35:28.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:28.464+0000 7fab62a00700 1 -- 172.21.15.37:0/4154741160 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fab5c0598b0 con 0x7fab6c102e00 2024-03-29T08:35:28.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:28.764+0000 7fab71c00700 1 -- 172.21.15.37:0/4154741160 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fab6c04bed0 con 0x7fab6c102e00 2024-03-29T08:35:28.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:28.764+0000 7fab62a00700 1 -- 172.21.15.37:0/4154741160 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7fab5c059440 con 0x7fab6c102e00 2024-03-29T08:35:28.764 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-29T08:35:28.764 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": { 2024-03-29T08:35:28.764 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 2 2024-03-29T08:35:28.764 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:35:28.765 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": { 2024-03-29T08:35:28.765 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 2 2024-03-29T08:35:28.765 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:35:28.765 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": { 2024-03-29T08:35:28.765 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 6 2024-03-29T08:35:28.765 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:35:28.765 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": { 2024-03-29T08:35:28.765 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 4 2024-03-29T08:35:28.765 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:35:28.765 INFO:teuthology.orchestra.run.smithi037.stdout: "overall": { 2024-03-29T08:35:28.765 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 14 2024-03-29T08:35:28.765 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-29T08:35:28.765 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-29T08:35:28.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:28.766+0000 7fab71c00700 1 -- 172.21.15.37:0/4154741160 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fab5006cc30 msgr2=0x7fab5006f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:35:28.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:28.766+0000 7fab71c00700 1 --2- 172.21.15.37:0/4154741160 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fab5006cc30 0x7fab5006f0e0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fab58004f90 tx=0x7fab58009e30 comp rx=0 tx=0).stop 2024-03-29T08:35:28.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:28.766+0000 7fab71c00700 1 -- 172.21.15.37:0/4154741160 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fab6c102e00 msgr2=0x7fab6c115d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:35:28.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:28.766+0000 7fab71c00700 1 --2- 172.21.15.37:0/4154741160 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fab6c102e00 0x7fab6c115d00 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fab5c002a50 tx=0x7fab5c0046b0 comp rx=0 tx=0).stop 2024-03-29T08:35:28.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:28.767+0000 7fab71c00700 1 -- 172.21.15.37:0/4154741160 shutdown_connections 2024-03-29T08:35:28.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:28.767+0000 7fab71c00700 1 --2- 172.21.15.37:0/4154741160 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fab5006cc30 0x7fab5006f0e0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:35:28.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:28.767+0000 7fab71c00700 1 --2- 172.21.15.37:0/4154741160 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fab6c102e00 0x7fab6c115d00 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:35:28.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:28.767+0000 7fab71c00700 1 --2- 172.21.15.37:0/4154741160 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fab6c1037a0 0x7fab6c116240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:35:28.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:28.767+0000 7fab71c00700 1 -- 172.21.15.37:0/4154741160 >> 172.21.15.37:0/4154741160 conn(0x7fab6c0f89b0 msgr2=0x7fab6c0f9ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:35:28.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:28.767+0000 7fab71c00700 1 -- 172.21.15.37:0/4154741160 shutdown_connections 2024-03-29T08:35:28.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:28.767+0000 7fab71c00700 1 -- 172.21.15.37:0/4154741160 wait complete. 2024-03-29T08:35:28.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:28.904+0000 7f928a600700 1 -- 172.21.15.37:0/3867521085 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9284070bf0 msgr2=0x7f9284108fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:35:28.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:28.904+0000 7f928a600700 1 --2- 172.21.15.37:0/3867521085 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9284070bf0 0x7f9284108fc0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f9270004940 tx=0x7f9270009e30 comp rx=0 tx=0).stop 2024-03-29T08:35:28.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:28.904+0000 7f928a600700 1 -- 172.21.15.37:0/3867521085 shutdown_connections 2024-03-29T08:35:28.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:28.904+0000 7f928a600700 1 --2- 172.21.15.37:0/3867521085 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9284070bf0 0x7f9284108fc0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:35:28.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:28.904+0000 7f928a600700 1 --2- 172.21.15.37:0/3867521085 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f92840702e0 0x7f92840706b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:35:28.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:28.904+0000 7f928a600700 1 -- 172.21.15.37:0/3867521085 >> 172.21.15.37:0/3867521085 conn(0x7f92840f87a0 msgr2=0x7f92840fabb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:35:28.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:28.904+0000 7f928a600700 1 -- 172.21.15.37:0/3867521085 shutdown_connections 2024-03-29T08:35:28.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:28.904+0000 7f928a600700 1 -- 172.21.15.37:0/3867521085 wait complete. 2024-03-29T08:35:28.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:28.905+0000 7f928a600700 1 Processor -- start 2024-03-29T08:35:28.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:28.905+0000 7f928a600700 1 -- start start 2024-03-29T08:35:28.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:28.906+0000 7f928a600700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f92840702e0 0x7f9284115b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:35:28.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:28.906+0000 7f928a600700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9284070bf0 0x7f9284116070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:35:28.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:28.906+0000 7f928a600700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92841166c0 con 0x7f9284070bf0 2024-03-29T08:35:28.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:28.906+0000 7f928a600700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f928410fab0 con 0x7f92840702e0 2024-03-29T08:35:28.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:28.906+0000 7f9283e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9284070bf0 0x7f9284116070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:35:28.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:28.906+0000 7f9283e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9284070bf0 0x7f9284116070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52006/0 (socket says 172.21.15.37:52006) 2024-03-29T08:35:28.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:28.906+0000 7f9283e00700 1 -- 172.21.15.37:0/3184002955 learned_addr learned my addr 172.21.15.37:0/3184002955 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:35:28.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:28.906+0000 7f9289200700 1 --2- 172.21.15.37:0/3184002955 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f92840702e0 0x7f9284115b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:35:28.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:28.906+0000 7f9283e00700 1 -- 172.21.15.37:0/3184002955 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f92840702e0 msgr2=0x7f9284115b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:35:28.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:28.906+0000 7f9283e00700 1 --2- 172.21.15.37:0/3184002955 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f92840702e0 0x7f9284115b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:35:28.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:28.906+0000 7f9283e00700 1 -- 172.21.15.37:0/3184002955 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9270005040 con 0x7f9284070bf0 2024-03-29T08:35:28.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:28.907+0000 7f9283e00700 1 --2- 172.21.15.37:0/3184002955 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9284070bf0 0x7f9284116070 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f9270009390 tx=0x7f9270009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:35:28.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:28.907+0000 7f9280c00700 1 -- 172.21.15.37:0/3184002955 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f927000e040 con 0x7f9284070bf0 2024-03-29T08:35:28.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:28.907+0000 7f9280c00700 1 -- 172.21.15.37:0/3184002955 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f9270013420 con 0x7f9284070bf0 2024-03-29T08:35:28.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:28.907+0000 7f9280c00700 1 -- 172.21.15.37:0/3184002955 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f927001c630 con 0x7f9284070bf0 2024-03-29T08:35:28.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:28.907+0000 7f928a600700 1 -- 172.21.15.37:0/3184002955 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f928410fd30 con 0x7f9284070bf0 2024-03-29T08:35:28.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:28.907+0000 7f928a600700 1 -- 172.21.15.37:0/3184002955 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9284110270 con 0x7f9284070bf0 2024-03-29T08:35:28.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:28.908+0000 7f928a600700 1 -- 172.21.15.37:0/3184002955 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9284071a20 con 0x7f9284070bf0 2024-03-29T08:35:28.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:28.909+0000 7f9280c00700 1 -- 172.21.15.37:0/3184002955 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f92700139e0 con 0x7f9284070bf0 2024-03-29T08:35:28.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:28.910+0000 7f9280c00700 1 --2- 172.21.15.37:0/3184002955 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f926c06cc30 0x7f926c06f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:35:28.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:28.910+0000 7f9280c00700 1 -- 172.21.15.37:0/3184002955 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f927008d340 con 0x7f9284070bf0 2024-03-29T08:35:28.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:28.910+0000 7f9289200700 1 --2- 172.21.15.37:0/3184002955 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f926c06cc30 0x7f926c06f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:35:28.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:28.911+0000 7f9289200700 1 --2- 172.21.15.37:0/3184002955 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f926c06cc30 0x7f926c06f0e0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f9278004f40 tx=0x7f927800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:35:28.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:28.915+0000 7f9280c00700 1 -- 172.21.15.37:0/3184002955 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f927005ae10 con 0x7f9284070bf0 2024-03-29T08:35:28.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:35:28 smithi037 ceph-mon[142325]: pgmap v738: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:35:28.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:35:28 smithi037 ceph-mon[142325]: from='client.35346 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:35:29.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:35:28 smithi148 ceph-mon[106692]: pgmap v738: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:35:29.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:35:28 smithi148 ceph-mon[106692]: from='client.35346 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:35:29.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:29.175+0000 7f928a600700 1 -- 172.21.15.37:0/3184002955 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f928405f5f0 con 0x7f9284070bf0 2024-03-29T08:35:29.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:29.176+0000 7f9280c00700 1 -- 172.21.15.37:0/3184002955 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 30 v30) v1 ==== 76+0+1841 (secure 0 0 0) 0x7f927005a9a0 con 0x7f9284070bf0 2024-03-29T08:35:29.176 INFO:teuthology.orchestra.run.smithi037.stdout:e30 2024-03-29T08:35:29.176 INFO:teuthology.orchestra.run.smithi037.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-29T08:35:29.176 INFO:teuthology.orchestra.run.smithi037.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T08:35:29.176 INFO:teuthology.orchestra.run.smithi037.stdout:legacy client fscid: 1 2024-03-29T08:35:29.176 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:35:29.176 INFO:teuthology.orchestra.run.smithi037.stdout:Filesystem 'cephfs' (1) 2024-03-29T08:35:29.176 INFO:teuthology.orchestra.run.smithi037.stdout:fs_name cephfs 2024-03-29T08:35:29.176 INFO:teuthology.orchestra.run.smithi037.stdout:epoch 21 2024-03-29T08:35:29.176 INFO:teuthology.orchestra.run.smithi037.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-03-29T08:35:29.177 INFO:teuthology.orchestra.run.smithi037.stdout:created 2024-03-29T08:03:29.241269+0000 2024-03-29T08:35:29.177 INFO:teuthology.orchestra.run.smithi037.stdout:modified 2024-03-29T08:17:16.237238+0000 2024-03-29T08:35:29.177 INFO:teuthology.orchestra.run.smithi037.stdout:tableserver 0 2024-03-29T08:35:29.177 INFO:teuthology.orchestra.run.smithi037.stdout:root 0 2024-03-29T08:35:29.177 INFO:teuthology.orchestra.run.smithi037.stdout:session_timeout 60 2024-03-29T08:35:29.177 INFO:teuthology.orchestra.run.smithi037.stdout:session_autoclose 300 2024-03-29T08:35:29.177 INFO:teuthology.orchestra.run.smithi037.stdout:max_file_size 1099511627776 2024-03-29T08:35:29.177 INFO:teuthology.orchestra.run.smithi037.stdout:max_xattr_size 65536 2024-03-29T08:35:29.177 INFO:teuthology.orchestra.run.smithi037.stdout:required_client_features {} 2024-03-29T08:35:29.177 INFO:teuthology.orchestra.run.smithi037.stdout:last_failure 0 2024-03-29T08:35:29.177 INFO:teuthology.orchestra.run.smithi037.stdout:last_failure_osd_epoch 82 2024-03-29T08:35:29.177 INFO:teuthology.orchestra.run.smithi037.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T08:35:29.177 INFO:teuthology.orchestra.run.smithi037.stdout:max_mds 2 2024-03-29T08:35:29.178 INFO:teuthology.orchestra.run.smithi037.stdout:in 0,1 2024-03-29T08:35:29.178 INFO:teuthology.orchestra.run.smithi037.stdout:up {} 2024-03-29T08:35:29.178 INFO:teuthology.orchestra.run.smithi037.stdout:failed 0,1 2024-03-29T08:35:29.178 INFO:teuthology.orchestra.run.smithi037.stdout:damaged 2024-03-29T08:35:29.178 INFO:teuthology.orchestra.run.smithi037.stdout:stopped 2024-03-29T08:35:29.178 INFO:teuthology.orchestra.run.smithi037.stdout:data_pools [3] 2024-03-29T08:35:29.178 INFO:teuthology.orchestra.run.smithi037.stdout:metadata_pool 2 2024-03-29T08:35:29.178 INFO:teuthology.orchestra.run.smithi037.stdout:inline_data disabled 2024-03-29T08:35:29.178 INFO:teuthology.orchestra.run.smithi037.stdout:balancer 2024-03-29T08:35:29.178 INFO:teuthology.orchestra.run.smithi037.stdout:bal_rank_mask -1 2024-03-29T08:35:29.178 INFO:teuthology.orchestra.run.smithi037.stdout:standby_count_wanted 1 2024-03-29T08:35:29.178 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:35:29.179 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:35:29.179 INFO:teuthology.orchestra.run.smithi037.stdout:Standby daemons: 2024-03-29T08:35:29.179 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:35:29.179 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi037.mmwgxe{-1:34440} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.37:6828/1248142005,v1:172.21.15.37:6829/1248142005] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:35:29.179 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi037.iifsmh{-1:34444} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.37:6826/2060882959,v1:172.21.15.37:6827/2060882959] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:35:29.179 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi148.sqteih{-1:44395} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.148:6824/4078652242,v1:172.21.15.148:6825/4078652242] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:35:29.179 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi148.qukpjh{-1:45007} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.148:6826/2394751105,v1:172.21.15.148:6827/2394751105] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:35:29.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:29.178+0000 7f928a600700 1 -- 172.21.15.37:0/3184002955 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f926c06cc30 msgr2=0x7f926c06f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:35:29.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:29.178+0000 7f928a600700 1 --2- 172.21.15.37:0/3184002955 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f926c06cc30 0x7f926c06f0e0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f9278004f40 tx=0x7f927800b040 comp rx=0 tx=0).stop 2024-03-29T08:35:29.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:29.178+0000 7f928a600700 1 -- 172.21.15.37:0/3184002955 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9284070bf0 msgr2=0x7f9284116070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:35:29.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:29.178+0000 7f928a600700 1 --2- 172.21.15.37:0/3184002955 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9284070bf0 0x7f9284116070 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f9270009390 tx=0x7f9270009e30 comp rx=0 tx=0).stop 2024-03-29T08:35:29.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:29.178+0000 7f928a600700 1 -- 172.21.15.37:0/3184002955 shutdown_connections 2024-03-29T08:35:29.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:29.178+0000 7f928a600700 1 --2- 172.21.15.37:0/3184002955 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f926c06cc30 0x7f926c06f0e0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:35:29.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:29.178+0000 7f928a600700 1 --2- 172.21.15.37:0/3184002955 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f92840702e0 0x7f9284115b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:35:29.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:29.178+0000 7f928a600700 1 --2- 172.21.15.37:0/3184002955 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9284070bf0 0x7f9284116070 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:35:29.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:29.178+0000 7f928a600700 1 -- 172.21.15.37:0/3184002955 >> 172.21.15.37:0/3184002955 conn(0x7f92840f87a0 msgr2=0x7f92840f9ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:35:29.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:29.179+0000 7f928a600700 1 -- 172.21.15.37:0/3184002955 shutdown_connections 2024-03-29T08:35:29.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:29.179+0000 7f928a600700 1 -- 172.21.15.37:0/3184002955 wait complete. 2024-03-29T08:35:29.181 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 30 2024-03-29T08:35:29.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:29.317+0000 7f4879000700 1 -- 172.21.15.37:0/4276510334 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f48740fe830 msgr2=0x7f4874106f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:35:29.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:29.317+0000 7f4879000700 1 --2- 172.21.15.37:0/4276510334 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f48740fe830 0x7f4874106f00 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f4860004940 tx=0x7f4860009e30 comp rx=0 tx=0).stop 2024-03-29T08:35:29.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:29.317+0000 7f4879000700 1 -- 172.21.15.37:0/4276510334 shutdown_connections 2024-03-29T08:35:29.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:29.317+0000 7f4879000700 1 --2- 172.21.15.37:0/4276510334 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f48740fe830 0x7f4874106f00 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:35:29.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:29.317+0000 7f4879000700 1 --2- 172.21.15.37:0/4276510334 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f48740fde90 0x7f48740fe260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:35:29.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:29.317+0000 7f4879000700 1 -- 172.21.15.37:0/4276510334 >> 172.21.15.37:0/4276510334 conn(0x7f4874072bc0 msgr2=0x7f4874072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:35:29.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:29.317+0000 7f4879000700 1 -- 172.21.15.37:0/4276510334 shutdown_connections 2024-03-29T08:35:29.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:29.317+0000 7f4879000700 1 -- 172.21.15.37:0/4276510334 wait complete. 2024-03-29T08:35:29.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:29.318+0000 7f4879000700 1 Processor -- start 2024-03-29T08:35:29.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:29.318+0000 7f4879000700 1 -- start start 2024-03-29T08:35:29.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:29.319+0000 7f4879000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f48740fde90 0x7f4874115c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:35:29.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:29.319+0000 7f4879000700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f48740fe830 0x7f48741161c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:35:29.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:29.319+0000 7f4879000700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4874116810 con 0x7f48740fde90 2024-03-29T08:35:29.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:29.319+0000 7f4879000700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f487410fc30 con 0x7f48740fe830 2024-03-29T08:35:29.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:29.319+0000 7f4871e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f48740fde90 0x7f4874115c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:35:29.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:29.319+0000 7f4871e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f48740fde90 0x7f4874115c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52026/0 (socket says 172.21.15.37:52026) 2024-03-29T08:35:29.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:29.319+0000 7f4871e00700 1 -- 172.21.15.37:0/3656679337 learned_addr learned my addr 172.21.15.37:0/3656679337 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:35:29.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:29.319+0000 7f4871400700 1 --2- 172.21.15.37:0/3656679337 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f48740fe830 0x7f48741161c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:35:29.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:29.319+0000 7f4871e00700 1 -- 172.21.15.37:0/3656679337 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f48740fe830 msgr2=0x7f48741161c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:35:29.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:29.319+0000 7f4871e00700 1 --2- 172.21.15.37:0/3656679337 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f48740fe830 0x7f48741161c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:35:29.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:29.319+0000 7f4871e00700 1 -- 172.21.15.37:0/3656679337 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4860005040 con 0x7f48740fde90 2024-03-29T08:35:29.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:29.320+0000 7f4871e00700 1 --2- 172.21.15.37:0/3656679337 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f48740fde90 0x7f4874115c80 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f4864002a80 tx=0x7f48640046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:35:29.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:29.320+0000 7f486a000700 1 -- 172.21.15.37:0/3656679337 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4864007cc0 con 0x7f48740fde90 2024-03-29T08:35:29.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:29.320+0000 7f486a000700 1 -- 172.21.15.37:0/3656679337 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f486400f040 con 0x7f48740fde90 2024-03-29T08:35:29.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:29.320+0000 7f486a000700 1 -- 172.21.15.37:0/3656679337 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f486400da60 con 0x7f48740fde90 2024-03-29T08:35:29.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:29.320+0000 7f4879000700 1 -- 172.21.15.37:0/3656679337 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f487410ff10 con 0x7f48740fde90 2024-03-29T08:35:29.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:29.320+0000 7f4879000700 1 -- 172.21.15.37:0/3656679337 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4874110450 con 0x7f48740fde90 2024-03-29T08:35:29.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:29.321+0000 7f4879000700 1 -- 172.21.15.37:0/3656679337 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4874106a20 con 0x7f48740fde90 2024-03-29T08:35:29.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:29.322+0000 7f486a000700 1 -- 172.21.15.37:0/3656679337 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f486400dbc0 con 0x7f48740fde90 2024-03-29T08:35:29.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:29.323+0000 7f486a000700 1 --2- 172.21.15.37:0/3656679337 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f485806cc30 0x7f485806f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:35:29.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:29.323+0000 7f486a000700 1 -- 172.21.15.37:0/3656679337 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f486408bbf0 con 0x7f48740fde90 2024-03-29T08:35:29.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:29.323+0000 7f4871400700 1 --2- 172.21.15.37:0/3656679337 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f485806cc30 0x7f485806f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:35:29.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:29.324+0000 7f4871400700 1 --2- 172.21.15.37:0/3656679337 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f485806cc30 0x7f485806f0e0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f4874111520 tx=0x7f4860009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:35:29.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:29.328+0000 7f486a000700 1 -- 172.21.15.37:0/3656679337 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f4864092080 con 0x7f48740fde90 2024-03-29T08:35:29.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:29.560+0000 7f4879000700 1 -- 172.21.15.37:0/3656679337 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4874111340 con 0x7f485806cc30 2024-03-29T08:35:29.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:29.561+0000 7f486a000700 1 -- 172.21.15.37:0/3656679337 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f4874111340 con 0x7f485806cc30 2024-03-29T08:35:29.561 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-29T08:35:29.561 INFO:teuthology.orchestra.run.smithi037.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59", 2024-03-29T08:35:29.561 INFO:teuthology.orchestra.run.smithi037.stdout: "in_progress": true, 2024-03-29T08:35:29.561 INFO:teuthology.orchestra.run.smithi037.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-29T08:35:29.561 INFO:teuthology.orchestra.run.smithi037.stdout: "services_complete": [ 2024-03-29T08:35:29.561 INFO:teuthology.orchestra.run.smithi037.stdout: "crash", 2024-03-29T08:35:29.561 INFO:teuthology.orchestra.run.smithi037.stdout: "osd", 2024-03-29T08:35:29.561 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr", 2024-03-29T08:35:29.561 INFO:teuthology.orchestra.run.smithi037.stdout: "mds", 2024-03-29T08:35:29.561 INFO:teuthology.orchestra.run.smithi037.stdout: "mon" 2024-03-29T08:35:29.561 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-29T08:35:29.561 INFO:teuthology.orchestra.run.smithi037.stdout: "progress": "16/21 daemons upgraded", 2024-03-29T08:35:29.561 INFO:teuthology.orchestra.run.smithi037.stdout: "message": "Currently upgrading mds daemons", 2024-03-29T08:35:29.561 INFO:teuthology.orchestra.run.smithi037.stdout: "is_paused": false 2024-03-29T08:35:29.562 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-29T08:35:29.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:29.563+0000 7f4879000700 1 -- 172.21.15.37:0/3656679337 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f485806cc30 msgr2=0x7f485806f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:35:29.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:29.563+0000 7f4879000700 1 --2- 172.21.15.37:0/3656679337 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f485806cc30 0x7f485806f0e0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f4874111520 tx=0x7f4860009e30 comp rx=0 tx=0).stop 2024-03-29T08:35:29.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:29.563+0000 7f4879000700 1 -- 172.21.15.37:0/3656679337 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f48740fde90 msgr2=0x7f4874115c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:35:29.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:29.563+0000 7f4879000700 1 --2- 172.21.15.37:0/3656679337 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f48740fde90 0x7f4874115c80 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f4864002a80 tx=0x7f48640046b0 comp rx=0 tx=0).stop 2024-03-29T08:35:29.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:29.563+0000 7f4879000700 1 -- 172.21.15.37:0/3656679337 shutdown_connections 2024-03-29T08:35:29.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:29.563+0000 7f4879000700 1 --2- 172.21.15.37:0/3656679337 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f485806cc30 0x7f485806f0e0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:35:29.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:29.563+0000 7f4879000700 1 --2- 172.21.15.37:0/3656679337 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f48740fde90 0x7f4874115c80 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:35:29.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:29.563+0000 7f4879000700 1 --2- 172.21.15.37:0/3656679337 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f48740fe830 0x7f48741161c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:35:29.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:29.563+0000 7f4879000700 1 -- 172.21.15.37:0/3656679337 >> 172.21.15.37:0/3656679337 conn(0x7f4874072bc0 msgr2=0x7f48740fd510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:35:29.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:29.564+0000 7f4879000700 1 -- 172.21.15.37:0/3656679337 shutdown_connections 2024-03-29T08:35:29.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:29.564+0000 7f4879000700 1 -- 172.21.15.37:0/3656679337 wait complete. 2024-03-29T08:35:29.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:29.702+0000 7f317a800700 1 -- 172.21.15.37:0/1523346342 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3174102c40 msgr2=0x7f3174103010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:35:29.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:29.702+0000 7f317a800700 1 --2- 172.21.15.37:0/1523346342 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3174102c40 0x7f3174103010 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f3168004940 tx=0x7f3168009e30 comp rx=0 tx=0).stop 2024-03-29T08:35:29.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:29.702+0000 7f317a800700 1 -- 172.21.15.37:0/1523346342 shutdown_connections 2024-03-29T08:35:29.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:29.702+0000 7f317a800700 1 --2- 172.21.15.37:0/1523346342 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f31741035e0 0x7f31740fcc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:35:29.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:29.702+0000 7f317a800700 1 --2- 172.21.15.37:0/1523346342 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3174102c40 0x7f3174103010 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:35:29.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:29.702+0000 7f317a800700 1 -- 172.21.15.37:0/1523346342 >> 172.21.15.37:0/1523346342 conn(0x7f31740f8830 msgr2=0x7f31740fac40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:35:29.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:29.702+0000 7f317a800700 1 -- 172.21.15.37:0/1523346342 shutdown_connections 2024-03-29T08:35:29.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:29.702+0000 7f317a800700 1 -- 172.21.15.37:0/1523346342 wait complete. 2024-03-29T08:35:29.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:29.703+0000 7f317a800700 1 Processor -- start 2024-03-29T08:35:29.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:29.703+0000 7f317a800700 1 -- start start 2024-03-29T08:35:29.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:29.703+0000 7f317a800700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3174102c40 0x7f31741138f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:35:29.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:29.703+0000 7f317a800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f31741035e0 0x7f3174113e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:35:29.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:29.703+0000 7f317a800700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3174114510 con 0x7f31741035e0 2024-03-29T08:35:29.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:29.703+0000 7f317a800700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f317410d870 con 0x7f3174102c40 2024-03-29T08:35:29.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:29.704+0000 7f3178a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f31741035e0 0x7f3174113e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:35:29.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:29.704+0000 7f3178a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f31741035e0 0x7f3174113e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52050/0 (socket says 172.21.15.37:52050) 2024-03-29T08:35:29.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:29.704+0000 7f3178a00700 1 -- 172.21.15.37:0/2780388554 learned_addr learned my addr 172.21.15.37:0/2780388554 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:35:29.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:29.704+0000 7f3179400700 1 --2- 172.21.15.37:0/2780388554 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3174102c40 0x7f31741138f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:35:29.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:29.704+0000 7f3178a00700 1 -- 172.21.15.37:0/2780388554 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3174102c40 msgr2=0x7f31741138f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:35:29.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:29.704+0000 7f3178a00700 1 --2- 172.21.15.37:0/2780388554 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3174102c40 0x7f31741138f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:35:29.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:29.704+0000 7f3178a00700 1 -- 172.21.15.37:0/2780388554 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3168005040 con 0x7f31741035e0 2024-03-29T08:35:29.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:29.704+0000 7f3178a00700 1 --2- 172.21.15.37:0/2780388554 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f31741035e0 0x7f3174113e30 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f316000b790 tx=0x7f3160009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:35:29.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:29.705+0000 7f3171600700 1 -- 172.21.15.37:0/2780388554 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f316000f6f0 con 0x7f31741035e0 2024-03-29T08:35:29.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:29.705+0000 7f3171600700 1 -- 172.21.15.37:0/2780388554 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f316000fd00 con 0x7f31741035e0 2024-03-29T08:35:29.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:29.705+0000 7f317a800700 1 -- 172.21.15.37:0/2780388554 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f317410db50 con 0x7f31741035e0 2024-03-29T08:35:29.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:29.705+0000 7f3171600700 1 -- 172.21.15.37:0/2780388554 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f31600103a0 con 0x7f31741035e0 2024-03-29T08:35:29.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:29.705+0000 7f317a800700 1 -- 172.21.15.37:0/2780388554 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f317410e110 con 0x7f31741035e0 2024-03-29T08:35:29.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:29.706+0000 7f317a800700 1 -- 172.21.15.37:0/2780388554 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31740fe390 con 0x7f31741035e0 2024-03-29T08:35:29.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:29.707+0000 7f3171600700 1 -- 172.21.15.37:0/2780388554 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f316000f850 con 0x7f31741035e0 2024-03-29T08:35:29.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:29.708+0000 7f3171600700 1 --2- 172.21.15.37:0/2780388554 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f315c06cc30 0x7f315c06f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:35:29.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:29.708+0000 7f3171600700 1 -- 172.21.15.37:0/2780388554 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f316008bf60 con 0x7f31741035e0 2024-03-29T08:35:29.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:29.709+0000 7f3179400700 1 --2- 172.21.15.37:0/2780388554 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f315c06cc30 0x7f315c06f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:35:29.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:29.709+0000 7f3179400700 1 --2- 172.21.15.37:0/2780388554 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f315c06cc30 0x7f315c06f0e0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f316800b020 tx=0x7f3168009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:35:29.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:29.713+0000 7f3171600700 1 -- 172.21.15.37:0/2780388554 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f3160059930 con 0x7f31741035e0 2024-03-29T08:35:29.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:35:29 smithi037 ceph-mon[142325]: from='client.35350 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:35:29.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:35:29 smithi037 ceph-mon[142325]: from='client.35354 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:35:29.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:35:29 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/4154741160' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:35:29.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:35:29 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/3184002955' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T08:35:30.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:30.013+0000 7f317a800700 1 -- 172.21.15.37:0/2780388554 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f317404bed0 con 0x7f31741035e0 2024-03-29T08:35:30.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:30.013+0000 7f3171600700 1 -- 172.21.15.37:0/2780388554 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+3583 (secure 0 0 0) 0x7f31600594c0 con 0x7f31741035e0 2024-03-29T08:35:30.013 INFO:teuthology.orchestra.run.smithi037.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem is offline 2024-03-29T08:35:30.013 INFO:teuthology.orchestra.run.smithi037.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-03-29T08:35:30.013 INFO:teuthology.orchestra.run.smithi037.stdout: host smithi148 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/mon.smithi148/config 2024-03-29T08:35:30.013 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/podman: timeout after 895 seconds 2024-03-29T08:35:30.014 INFO:teuthology.orchestra.run.smithi037.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59 -e NODE_NAME=smithi148 -e CEPH_USE_RANDOM_NONCE=1 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4:/var/run/ceph:z -v /var/log/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4:/var/log/ceph:z -v /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpkx2rz2d4:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59 inventory --format=json-pretty --filter-for-batch 2024-03-29T08:35:30.014 INFO:teuthology.orchestra.run.smithi037.stdout:Traceback (most recent call last): 2024-03-29T08:35:30.014 INFO:teuthology.orchestra.run.smithi037.stdout: File "/usr/lib64/python3.6/runpy.py", line 193, in _run_module_as_main 2024-03-29T08:35:30.014 INFO:teuthology.orchestra.run.smithi037.stdout: "__main__", mod_spec) 2024-03-29T08:35:30.014 INFO:teuthology.orchestra.run.smithi037.stdout: File "/usr/lib64/python3.6/runpy.py", line 85, in _run_code 2024-03-29T08:35:30.014 INFO:teuthology.orchestra.run.smithi037.stdout: exec(code, run_globals) 2024-03-29T08:35:30.014 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 10889, in 2024-03-29T08:35:30.014 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 10877, in main 2024-03-29T08:35:30.014 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 2576, in _infer_config 2024-03-29T08:35:30.014 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 2492, in _infer_fsid 2024-03-29T08:35:30.014 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 2604, in _infer_image 2024-03-29T08:35:30.014 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 2479, in _validate_fsid 2024-03-29T08:35:30.014 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 7145, in command_ceph_volume 2024-03-29T08:35:30.014 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 2267, in call_throws 2024-03-29T08:35:30.015 INFO:teuthology.orchestra.run.smithi037.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59 -e NODE_NAME=smithi148 -e CEPH_USE_RANDOM_NONCE=1 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4:/var/run/ceph:z -v /var/log/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4:/var/log/ceph:z -v /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpkx2rz2d4:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59 inventory --format=json-pretty --filter-for-batch 2024-03-29T08:35:30.015 INFO:teuthology.orchestra.run.smithi037.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-03-29T08:35:30.015 INFO:teuthology.orchestra.run.smithi037.stdout: fs cephfs is degraded 2024-03-29T08:35:30.015 INFO:teuthology.orchestra.run.smithi037.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-03-29T08:35:30.015 INFO:teuthology.orchestra.run.smithi037.stdout: fs cephfs is offline because no MDS is active for it. 2024-03-29T08:35:30.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:30.015+0000 7f317a800700 1 -- 172.21.15.37:0/2780388554 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f315c06cc30 msgr2=0x7f315c06f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:35:30.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:30.015+0000 7f317a800700 1 --2- 172.21.15.37:0/2780388554 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f315c06cc30 0x7f315c06f0e0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f316800b020 tx=0x7f3168009e30 comp rx=0 tx=0).stop 2024-03-29T08:35:30.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:30.015+0000 7f317a800700 1 -- 172.21.15.37:0/2780388554 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f31741035e0 msgr2=0x7f3174113e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:35:30.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:30.015+0000 7f317a800700 1 --2- 172.21.15.37:0/2780388554 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f31741035e0 0x7f3174113e30 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f316000b790 tx=0x7f3160009220 comp rx=0 tx=0).stop 2024-03-29T08:35:30.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:30.016+0000 7f317a800700 1 -- 172.21.15.37:0/2780388554 shutdown_connections 2024-03-29T08:35:30.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:30.016+0000 7f317a800700 1 --2- 172.21.15.37:0/2780388554 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f315c06cc30 0x7f315c06f0e0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:35:30.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:30.016+0000 7f317a800700 1 --2- 172.21.15.37:0/2780388554 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3174102c40 0x7f31741138f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:35:30.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:30.016+0000 7f317a800700 1 --2- 172.21.15.37:0/2780388554 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f31741035e0 0x7f3174113e30 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:35:30.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:30.016+0000 7f317a800700 1 -- 172.21.15.37:0/2780388554 >> 172.21.15.37:0/2780388554 conn(0x7f31740f8830 msgr2=0x7f31740f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:35:30.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:30.016+0000 7f317a800700 1 -- 172.21.15.37:0/2780388554 shutdown_connections 2024-03-29T08:35:30.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:35:30.016+0000 7f317a800700 1 -- 172.21.15.37:0/2780388554 wait complete. 2024-03-29T08:35:30.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:35:29 smithi148 ceph-mon[106692]: from='client.35350 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:35:30.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:35:29 smithi148 ceph-mon[106692]: from='client.35354 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:35:30.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:35:29 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/4154741160' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:35:30.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:35:29 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/3184002955' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T08:35:30.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:35:30 smithi037 ceph-mon[142325]: pgmap v739: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:35:30.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:35:30 smithi037 ceph-mon[142325]: from='client.35366 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:35:30.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:35:30 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/2780388554' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T08:35:31.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:35:30 smithi148 ceph-mon[106692]: pgmap v739: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:35:31.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:35:30 smithi148 ceph-mon[106692]: from='client.35366 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:35:31.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:35:30 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/2780388554' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T08:35:32.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:35:32 smithi037 ceph-mon[142325]: pgmap v740: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:35:33.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:35:32 smithi148 ceph-mon[106692]: pgmap v740: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:35:34.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:35:34 smithi037 ceph-mon[142325]: pgmap v741: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:35:35.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:35:34 smithi148 ceph-mon[106692]: pgmap v741: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:35:36.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:35:36 smithi037 ceph-mon[142325]: pgmap v742: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:35:37.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:35:36 smithi148 ceph-mon[106692]: pgmap v742: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:35:38.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:35:38 smithi037 ceph-mon[142325]: pgmap v743: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:35:39.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:35:38 smithi148 ceph-mon[106692]: pgmap v743: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:35:40.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:35:40 smithi037 ceph-mon[142325]: pgmap v744: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:35:40.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:35:40 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:35:41.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:35:40 smithi148 ceph-mon[106692]: pgmap v744: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:35:41.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:35:40 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:35:42.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:35:42 smithi037 ceph-mon[142325]: pgmap v745: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:35:43.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:35:42 smithi148 ceph-mon[106692]: pgmap v745: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:35:44.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:35:44 smithi037 ceph-mon[142325]: pgmap v746: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:35:45.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:35:44 smithi148 ceph-mon[106692]: pgmap v746: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:35:46.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:35:46 smithi037 ceph-mon[142325]: pgmap v747: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:35:47.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:35:46 smithi148 ceph-mon[106692]: pgmap v747: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:35:48.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:35:48 smithi037 ceph-mon[142325]: pgmap v748: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:35:49.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:35:48 smithi148 ceph-mon[106692]: pgmap v748: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:35:50.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:35:50 smithi037 ceph-mon[142325]: pgmap v749: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:35:51.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:35:50 smithi148 ceph-mon[106692]: pgmap v749: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:35:52.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:35:52 smithi037 ceph-mon[142325]: pgmap v750: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:35:53.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:35:52 smithi148 ceph-mon[106692]: pgmap v750: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:35:54.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:35:54 smithi037 ceph-mon[142325]: pgmap v751: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:35:54.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:35:54 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:35:55.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:35:54 smithi148 ceph-mon[106692]: pgmap v751: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:35:55.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:35:54 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:35:56.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:35:56 smithi037 ceph-mon[142325]: pgmap v752: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:35:57.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:35:56 smithi148 ceph-mon[106692]: pgmap v752: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:35:58.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:35:58 smithi037 ceph-mon[142325]: pgmap v753: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:35:59.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:35:58 smithi148 ceph-mon[106692]: pgmap v753: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:36:00.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:00.171+0000 7f6f26400700 1 -- 172.21.15.37:0/2453503325 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6f200fde90 msgr2=0x7f6f200fe260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:36:00.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:00.171+0000 7f6f26400700 1 --2- 172.21.15.37:0/2453503325 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6f200fde90 0x7f6f200fe260 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f6f10004940 tx=0x7f6f10009e30 comp rx=0 tx=0).stop 2024-03-29T08:36:00.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:00.171+0000 7f6f26400700 1 -- 172.21.15.37:0/2453503325 shutdown_connections 2024-03-29T08:36:00.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:00.171+0000 7f6f26400700 1 --2- 172.21.15.37:0/2453503325 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6f200fe830 0x7f6f20106f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:36:00.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:00.171+0000 7f6f26400700 1 --2- 172.21.15.37:0/2453503325 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6f200fde90 0x7f6f200fe260 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:36:00.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:00.171+0000 7f6f26400700 1 -- 172.21.15.37:0/2453503325 >> 172.21.15.37:0/2453503325 conn(0x7f6f20072bc0 msgr2=0x7f6f20072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:36:00.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:00.171+0000 7f6f26400700 1 -- 172.21.15.37:0/2453503325 shutdown_connections 2024-03-29T08:36:00.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:00.171+0000 7f6f26400700 1 -- 172.21.15.37:0/2453503325 wait complete. 2024-03-29T08:36:00.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:00.172+0000 7f6f26400700 1 Processor -- start 2024-03-29T08:36:00.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:00.172+0000 7f6f26400700 1 -- start start 2024-03-29T08:36:00.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:00.172+0000 7f6f26400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6f200fde90 0x7f6f200fbd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:36:00.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:00.172+0000 7f6f26400700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6f200fe830 0x7f6f200fc290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:36:00.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:00.172+0000 7f6f26400700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f200fc820 con 0x7f6f200fde90 2024-03-29T08:36:00.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:00.172+0000 7f6f26400700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f200fc960 con 0x7f6f200fe830 2024-03-29T08:36:00.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:00.173+0000 7f6f1f400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6f200fde90 0x7f6f200fbd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:36:00.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:00.173+0000 7f6f1f400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6f200fde90 0x7f6f200fbd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42592/0 (socket says 172.21.15.37:42592) 2024-03-29T08:36:00.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:00.173+0000 7f6f1f400700 1 -- 172.21.15.37:0/2274465418 learned_addr learned my addr 172.21.15.37:0/2274465418 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:36:00.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:00.173+0000 7f6f1ea00700 1 --2- 172.21.15.37:0/2274465418 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6f200fe830 0x7f6f200fc290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:36:00.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:00.173+0000 7f6f1f400700 1 -- 172.21.15.37:0/2274465418 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6f200fe830 msgr2=0x7f6f200fc290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:36:00.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:00.173+0000 7f6f1f400700 1 --2- 172.21.15.37:0/2274465418 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6f200fe830 0x7f6f200fc290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:36:00.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:00.173+0000 7f6f1f400700 1 -- 172.21.15.37:0/2274465418 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f10005040 con 0x7f6f200fde90 2024-03-29T08:36:00.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:00.173+0000 7f6f1f400700 1 --2- 172.21.15.37:0/2274465418 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6f200fde90 0x7f6f200fbd50 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f6f10000c00 tx=0x7f6f10009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:36:00.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:00.174+0000 7f6f17400700 1 -- 172.21.15.37:0/2274465418 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6f10021070 con 0x7f6f200fde90 2024-03-29T08:36:00.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:00.174+0000 7f6f17400700 1 -- 172.21.15.37:0/2274465418 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f6f10013e00 con 0x7f6f200fde90 2024-03-29T08:36:00.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:00.174+0000 7f6f17400700 1 -- 172.21.15.37:0/2274465418 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6f1000e070 con 0x7f6f200fde90 2024-03-29T08:36:00.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:00.174+0000 7f6f26400700 1 -- 172.21.15.37:0/2274465418 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f200fcbb0 con 0x7f6f200fde90 2024-03-29T08:36:00.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:00.174+0000 7f6f26400700 1 -- 172.21.15.37:0/2274465418 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f2011a130 con 0x7f6f200fde90 2024-03-29T08:36:00.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:00.175+0000 7f6f26400700 1 -- 172.21.15.37:0/2274465418 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f20065e30 con 0x7f6f200fde90 2024-03-29T08:36:00.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:00.176+0000 7f6f17400700 1 -- 172.21.15.37:0/2274465418 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f6f10013950 con 0x7f6f200fde90 2024-03-29T08:36:00.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:00.177+0000 7f6f17400700 1 --2- 172.21.15.37:0/2274465418 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f6f0406cc30 0x7f6f0406f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:36:00.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:00.177+0000 7f6f17400700 1 -- 172.21.15.37:0/2274465418 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f6f1008d050 con 0x7f6f200fde90 2024-03-29T08:36:00.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:00.177+0000 7f6f1ea00700 1 --2- 172.21.15.37:0/2274465418 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f6f0406cc30 0x7f6f0406f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:36:00.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:00.178+0000 7f6f1ea00700 1 --2- 172.21.15.37:0/2274465418 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f6f0406cc30 0x7f6f0406f0e0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f6f0c004600 tx=0x7f6f0c00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:36:00.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:00.182+0000 7f6f17400700 1 -- 172.21.15.37:0/2274465418 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f6f10057090 con 0x7f6f200fde90 2024-03-29T08:36:00.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:00.413+0000 7f6f26400700 1 -- 172.21.15.37:0/2274465418 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6f200fd440 con 0x7f6f0406cc30 2024-03-29T08:36:00.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:00.414+0000 7f6f17400700 1 -- 172.21.15.37:0/2274465418 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f6f200fd440 con 0x7f6f0406cc30 2024-03-29T08:36:00.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:00.416+0000 7f6f26400700 1 -- 172.21.15.37:0/2274465418 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f6f0406cc30 msgr2=0x7f6f0406f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:36:00.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:00.416+0000 7f6f26400700 1 --2- 172.21.15.37:0/2274465418 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f6f0406cc30 0x7f6f0406f0e0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f6f0c004600 tx=0x7f6f0c00b040 comp rx=0 tx=0).stop 2024-03-29T08:36:00.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:00.417+0000 7f6f26400700 1 -- 172.21.15.37:0/2274465418 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6f200fde90 msgr2=0x7f6f200fbd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:36:00.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:00.417+0000 7f6f26400700 1 --2- 172.21.15.37:0/2274465418 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6f200fde90 0x7f6f200fbd50 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f6f10000c00 tx=0x7f6f10009e30 comp rx=0 tx=0).stop 2024-03-29T08:36:00.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:00.417+0000 7f6f26400700 1 -- 172.21.15.37:0/2274465418 shutdown_connections 2024-03-29T08:36:00.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:00.417+0000 7f6f26400700 1 --2- 172.21.15.37:0/2274465418 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f6f0406cc30 0x7f6f0406f0e0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:36:00.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:00.417+0000 7f6f26400700 1 --2- 172.21.15.37:0/2274465418 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6f200fde90 0x7f6f200fbd50 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:36:00.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:00.417+0000 7f6f26400700 1 --2- 172.21.15.37:0/2274465418 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6f200fe830 0x7f6f200fc290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:36:00.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:00.417+0000 7f6f26400700 1 -- 172.21.15.37:0/2274465418 >> 172.21.15.37:0/2274465418 conn(0x7f6f20072bc0 msgr2=0x7f6f200ff3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:36:00.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:00.417+0000 7f6f26400700 1 -- 172.21.15.37:0/2274465418 shutdown_connections 2024-03-29T08:36:00.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:00.417+0000 7f6f26400700 1 -- 172.21.15.37:0/2274465418 wait complete. 2024-03-29T08:36:00.431 INFO:teuthology.orchestra.run.smithi037.stdout:true 2024-03-29T08:36:00.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:00.554+0000 7f710a000700 1 -- 172.21.15.37:0/204414399 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7104102e00 msgr2=0x7f71041031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:36:00.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:00.554+0000 7f710a000700 1 --2- 172.21.15.37:0/204414399 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7104102e00 0x7f71041031d0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f70f4004940 tx=0x7f70f4009e30 comp rx=0 tx=0).stop 2024-03-29T08:36:00.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:00.555+0000 7f710a000700 1 -- 172.21.15.37:0/204414399 shutdown_connections 2024-03-29T08:36:00.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:00.555+0000 7f710a000700 1 --2- 172.21.15.37:0/204414399 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f71041037a0 0x7f71040fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:36:00.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:00.555+0000 7f710a000700 1 --2- 172.21.15.37:0/204414399 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7104102e00 0x7f71041031d0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:36:00.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:00.555+0000 7f710a000700 1 -- 172.21.15.37:0/204414399 >> 172.21.15.37:0/204414399 conn(0x7f71040f89b0 msgr2=0x7f71040fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:36:00.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:00.555+0000 7f710a000700 1 -- 172.21.15.37:0/204414399 shutdown_connections 2024-03-29T08:36:00.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:00.555+0000 7f710a000700 1 -- 172.21.15.37:0/204414399 wait complete. 2024-03-29T08:36:00.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:00.555+0000 7f710a000700 1 Processor -- start 2024-03-29T08:36:00.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:00.556+0000 7f710a000700 1 -- start start 2024-03-29T08:36:00.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:00.556+0000 7f710a000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7104102e00 0x7f7104115cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:36:00.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:00.556+0000 7f710a000700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f71041037a0 0x7f7104116230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:36:00.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:00.556+0000 7f710a000700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7104116880 con 0x7f7104102e00 2024-03-29T08:36:00.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:00.556+0000 7f710a000700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f710410fc50 con 0x7f71041037a0 2024-03-29T08:36:00.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:00.556+0000 7f7103400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7104102e00 0x7f7104115cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:36:00.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:00.556+0000 7f7103400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7104102e00 0x7f7104115cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42616/0 (socket says 172.21.15.37:42616) 2024-03-29T08:36:00.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:00.556+0000 7f7103400700 1 -- 172.21.15.37:0/4283864577 learned_addr learned my addr 172.21.15.37:0/4283864577 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:36:00.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:00.557+0000 7f7102a00700 1 --2- 172.21.15.37:0/4283864577 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f71041037a0 0x7f7104116230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:36:00.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:00.557+0000 7f7103400700 1 -- 172.21.15.37:0/4283864577 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f71041037a0 msgr2=0x7f7104116230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:36:00.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:00.557+0000 7f7103400700 1 --2- 172.21.15.37:0/4283864577 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f71041037a0 0x7f7104116230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:36:00.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:00.557+0000 7f7103400700 1 -- 172.21.15.37:0/4283864577 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70f4005040 con 0x7f7104102e00 2024-03-29T08:36:00.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:00.557+0000 7f7102a00700 1 --2- 172.21.15.37:0/4283864577 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f71041037a0 0x7f7104116230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:36:00.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:00.557+0000 7f7103400700 1 --2- 172.21.15.37:0/4283864577 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7104102e00 0x7f7104115cf0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f70f4009360 tx=0x7f70f4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:36:00.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:00.557+0000 7f70fb400700 1 -- 172.21.15.37:0/4283864577 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f70f400e040 con 0x7f7104102e00 2024-03-29T08:36:00.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:00.557+0000 7f70fb400700 1 -- 172.21.15.37:0/4283864577 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f70f4013420 con 0x7f7104102e00 2024-03-29T08:36:00.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:00.557+0000 7f710a000700 1 -- 172.21.15.37:0/4283864577 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f710410fed0 con 0x7f7104102e00 2024-03-29T08:36:00.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:00.557+0000 7f70fb400700 1 -- 172.21.15.37:0/4283864577 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f70f401c5f0 con 0x7f7104102e00 2024-03-29T08:36:00.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:00.558+0000 7f710a000700 1 -- 172.21.15.37:0/4283864577 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7104110490 con 0x7f7104102e00 2024-03-29T08:36:00.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:00.559+0000 7f710a000700 1 -- 172.21.15.37:0/4283864577 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71040fe550 con 0x7f7104102e00 2024-03-29T08:36:00.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:00.560+0000 7f70fb400700 1 -- 172.21.15.37:0/4283864577 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f70f4013590 con 0x7f7104102e00 2024-03-29T08:36:00.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:00.560+0000 7f70fb400700 1 --2- 172.21.15.37:0/4283864577 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f70e806cc30 0x7f70e806f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:36:00.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:00.561+0000 7f70fb400700 1 -- 172.21.15.37:0/4283864577 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f70f408d250 con 0x7f7104102e00 2024-03-29T08:36:00.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:00.561+0000 7f7102a00700 1 --2- 172.21.15.37:0/4283864577 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f70e806cc30 0x7f70e806f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:36:00.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:00.561+0000 7f7102a00700 1 --2- 172.21.15.37:0/4283864577 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f70e806cc30 0x7f70e806f0e0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f70f0007fd0 tx=0x7f70f0009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:36:00.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:00.565+0000 7f70fb400700 1 -- 172.21.15.37:0/4283864577 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f70f405add0 con 0x7f7104102e00 2024-03-29T08:36:00.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:00.798+0000 7f710a000700 1 -- 172.21.15.37:0/4283864577 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f710404bed0 con 0x7f70e806cc30 2024-03-29T08:36:00.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:00.799+0000 7f70fb400700 1 -- 172.21.15.37:0/4283864577 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f710404bed0 con 0x7f70e806cc30 2024-03-29T08:36:00.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:00.802+0000 7f710a000700 1 -- 172.21.15.37:0/4283864577 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f70e806cc30 msgr2=0x7f70e806f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:36:00.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:00.802+0000 7f710a000700 1 --2- 172.21.15.37:0/4283864577 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f70e806cc30 0x7f70e806f0e0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f70f0007fd0 tx=0x7f70f0009040 comp rx=0 tx=0).stop 2024-03-29T08:36:00.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:00.802+0000 7f710a000700 1 -- 172.21.15.37:0/4283864577 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7104102e00 msgr2=0x7f7104115cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:36:00.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:00.802+0000 7f710a000700 1 --2- 172.21.15.37:0/4283864577 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7104102e00 0x7f7104115cf0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f70f4009360 tx=0x7f70f4009e30 comp rx=0 tx=0).stop 2024-03-29T08:36:00.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:00.802+0000 7f710a000700 1 -- 172.21.15.37:0/4283864577 shutdown_connections 2024-03-29T08:36:00.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:00.802+0000 7f710a000700 1 --2- 172.21.15.37:0/4283864577 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f70e806cc30 0x7f70e806f0e0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:36:00.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:00.802+0000 7f710a000700 1 --2- 172.21.15.37:0/4283864577 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7104102e00 0x7f7104115cf0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:36:00.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:00.802+0000 7f710a000700 1 --2- 172.21.15.37:0/4283864577 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f71041037a0 0x7f7104116230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:36:00.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:00.802+0000 7f710a000700 1 -- 172.21.15.37:0/4283864577 >> 172.21.15.37:0/4283864577 conn(0x7f71040f89b0 msgr2=0x7f71040706d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:36:00.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:00.802+0000 7f710a000700 1 -- 172.21.15.37:0/4283864577 shutdown_connections 2024-03-29T08:36:00.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:00.802+0000 7f710a000700 1 -- 172.21.15.37:0/4283864577 wait complete. 2024-03-29T08:36:00.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:36:00 smithi037 ceph-mon[142325]: pgmap v754: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:36:00.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:00.939+0000 7efe35000700 1 -- 172.21.15.37:0/861774047 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efe301059a0 msgr2=0x7efe300ff010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:36:00.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:00.939+0000 7efe35000700 1 --2- 172.21.15.37:0/861774047 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efe301059a0 0x7efe300ff010 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7efe1c004940 tx=0x7efe1c009e30 comp rx=0 tx=0).stop 2024-03-29T08:36:00.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:00.939+0000 7efe35000700 1 -- 172.21.15.37:0/861774047 shutdown_connections 2024-03-29T08:36:00.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:00.939+0000 7efe35000700 1 --2- 172.21.15.37:0/861774047 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efe301059a0 0x7efe300ff010 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:36:00.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:00.939+0000 7efe35000700 1 --2- 172.21.15.37:0/861774047 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe30105000 0x7efe301053d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:36:00.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:00.939+0000 7efe35000700 1 -- 172.21.15.37:0/861774047 >> 172.21.15.37:0/861774047 conn(0x7efe300fabd0 msgr2=0x7efe300fcfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:36:00.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:00.940+0000 7efe35000700 1 -- 172.21.15.37:0/861774047 shutdown_connections 2024-03-29T08:36:00.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:00.940+0000 7efe35000700 1 -- 172.21.15.37:0/861774047 wait complete. 2024-03-29T08:36:00.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:00.940+0000 7efe35000700 1 Processor -- start 2024-03-29T08:36:00.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:00.940+0000 7efe35000700 1 -- start start 2024-03-29T08:36:00.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:00.941+0000 7efe35000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efe30105000 0x7efe30115c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:36:00.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:00.941+0000 7efe35000700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe301059a0 0x7efe301161d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:36:00.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:00.941+0000 7efe35000700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe30116820 con 0x7efe30105000 2024-03-29T08:36:00.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:00.941+0000 7efe35000700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe3010fbf0 con 0x7efe301059a0 2024-03-29T08:36:00.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:00.941+0000 7efe2de00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efe30105000 0x7efe30115c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:36:00.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:00.941+0000 7efe2de00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efe30105000 0x7efe30115c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42622/0 (socket says 172.21.15.37:42622) 2024-03-29T08:36:00.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:00.941+0000 7efe2de00700 1 -- 172.21.15.37:0/2000471361 learned_addr learned my addr 172.21.15.37:0/2000471361 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:36:00.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:00.941+0000 7efe2d400700 1 --2- 172.21.15.37:0/2000471361 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe301059a0 0x7efe301161d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:36:00.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:00.942+0000 7efe2de00700 1 -- 172.21.15.37:0/2000471361 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe301059a0 msgr2=0x7efe301161d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:36:00.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:00.942+0000 7efe2de00700 1 --2- 172.21.15.37:0/2000471361 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe301059a0 0x7efe301161d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:36:00.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:00.942+0000 7efe2de00700 1 -- 172.21.15.37:0/2000471361 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe1c005040 con 0x7efe30105000 2024-03-29T08:36:00.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:00.942+0000 7efe2de00700 1 --2- 172.21.15.37:0/2000471361 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efe30105000 0x7efe30115c90 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7efe2000eb40 tx=0x7efe2000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:36:00.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:00.942+0000 7efe26000700 1 -- 172.21.15.37:0/2000471361 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efe20007cc0 con 0x7efe30105000 2024-03-29T08:36:00.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:00.942+0000 7efe26000700 1 -- 172.21.15.37:0/2000471361 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7efe2000f040 con 0x7efe30105000 2024-03-29T08:36:00.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:00.942+0000 7efe35000700 1 -- 172.21.15.37:0/2000471361 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe3010fe70 con 0x7efe30105000 2024-03-29T08:36:00.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:00.942+0000 7efe26000700 1 -- 172.21.15.37:0/2000471361 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efe2000c990 con 0x7efe30105000 2024-03-29T08:36:00.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:00.942+0000 7efe35000700 1 -- 172.21.15.37:0/2000471361 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe30110430 con 0x7efe30105000 2024-03-29T08:36:00.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:00.943+0000 7efe35000700 1 -- 172.21.15.37:0/2000471361 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe30102310 con 0x7efe30105000 2024-03-29T08:36:00.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:00.944+0000 7efe26000700 1 -- 172.21.15.37:0/2000471361 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7efe2000caf0 con 0x7efe30105000 2024-03-29T08:36:00.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:00.945+0000 7efe26000700 1 --2- 172.21.15.37:0/2000471361 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7efe1406cc30 0x7efe1406f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:36:00.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:00.945+0000 7efe26000700 1 -- 172.21.15.37:0/2000471361 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7efe2008bc00 con 0x7efe30105000 2024-03-29T08:36:00.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:00.945+0000 7efe2d400700 1 --2- 172.21.15.37:0/2000471361 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7efe1406cc30 0x7efe1406f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:36:00.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:00.946+0000 7efe2d400700 1 --2- 172.21.15.37:0/2000471361 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7efe1406cc30 0x7efe1406f0e0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7efe30111530 tx=0x7efe1c009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:36:00.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:00.950+0000 7efe26000700 1 -- 172.21.15.37:0/2000471361 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7efe20059700 con 0x7efe30105000 2024-03-29T08:36:01.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:36:00 smithi148 ceph-mon[106692]: pgmap v754: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:36:01.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:01.175+0000 7efe35000700 1 -- 172.21.15.37:0/2000471361 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7efe3010c010 con 0x7efe1406cc30 2024-03-29T08:36:01.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:01.185+0000 7efe26000700 1 -- 172.21.15.37:0/2000471361 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7efe3010c010 con 0x7efe1406cc30 2024-03-29T08:36:01.185 INFO:teuthology.orchestra.run.smithi037.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-29T08:36:01.185 INFO:teuthology.orchestra.run.smithi037.stdout:alertmanager.smithi037 smithi037 *:9093,9094 running (25m) 2m ago 38m 14.3M - 0.25.0 c8568f914cd2 2f9f9746ed0f 2024-03-29T08:36:01.185 INFO:teuthology.orchestra.run.smithi037.stdout:crash.smithi037 smithi037 running (22m) 2m ago 39m 6647k - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 178d416a3186 2024-03-29T08:36:01.185 INFO:teuthology.orchestra.run.smithi037.stdout:crash.smithi148 smithi148 running (22m) 2m ago 37m 6656k - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 23666989d58f 2024-03-29T08:36:01.185 INFO:teuthology.orchestra.run.smithi037.stdout:grafana.smithi037 smithi037 *:3000 running (25m) 2m ago 38m 74.5M - 9.4.7 954c08fa6188 6b976fe49952 2024-03-29T08:36:01.185 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi037.iifsmh smithi037 running (18m) 2m ago 32m 16.4M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 2d97f562cc5b 2024-03-29T08:36:01.185 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi037.mmwgxe smithi037 running (18m) 2m ago 32m 16.7M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff ec37037f46e8 2024-03-29T08:36:01.185 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi148.qukpjh smithi148 running (2m) 2m ago 32m 10.7M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 3257219bc046 2024-03-29T08:36:01.185 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi148.sqteih smithi148 running (17m) 2m ago 32m 16.3M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 44c15c77037a 2024-03-29T08:36:01.185 INFO:teuthology.orchestra.run.smithi037.stdout:mgr.smithi037.wjikii smithi037 *:8443,9283,8765 running (28m) 2m ago 40m 510M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff c5a1417b9417 2024-03-29T08:36:01.186 INFO:teuthology.orchestra.run.smithi037.stdout:mgr.smithi148.onhgvm smithi148 *:8443,9283,8765 running (27m) 2m ago 37m 456M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff fcfdbce062bd 2024-03-29T08:36:01.186 INFO:teuthology.orchestra.run.smithi037.stdout:mon.smithi037 smithi037 running (24m) 2m ago 40m 70.2M 2048M 18.2.2-1015-gd0e392d9 1cc2f45c17ff eea2060783af 2024-03-29T08:36:01.186 INFO:teuthology.orchestra.run.smithi037.stdout:mon.smithi148 smithi148 running (23m) 2m ago 37m 50.4M 2048M 18.2.2-1015-gd0e392d9 1cc2f45c17ff ad4b2c5748be 2024-03-29T08:36:01.186 INFO:teuthology.orchestra.run.smithi037.stdout:node-exporter.smithi037 smithi037 *:9100 running (26m) 2m ago 38m 8514k - 1.5.0 0da6a335fe13 26b8ae9cb878 2024-03-29T08:36:01.186 INFO:teuthology.orchestra.run.smithi037.stdout:node-exporter.smithi148 smithi148 *:9100 running (26m) 2m ago 37m 8464k - 1.5.0 0da6a335fe13 98414e4e64ed 2024-03-29T08:36:01.186 INFO:teuthology.orchestra.run.smithi037.stdout:osd.0 smithi037 running (22m) 2m ago 36m 108M 1619M 18.2.2-1015-gd0e392d9 1cc2f45c17ff c0c584f6245c 2024-03-29T08:36:01.186 INFO:teuthology.orchestra.run.smithi037.stdout:osd.1 smithi037 running (21m) 2m ago 36m 104M 1619M 18.2.2-1015-gd0e392d9 1cc2f45c17ff a4a2ff4cc83d 2024-03-29T08:36:01.186 INFO:teuthology.orchestra.run.smithi037.stdout:osd.2 smithi037 running (20m) 2m ago 35m 85.9M 1619M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 8b1bc71aa79e 2024-03-29T08:36:01.186 INFO:teuthology.orchestra.run.smithi037.stdout:osd.3 smithi148 running (20m) 2m ago 35m 109M 2642M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 1c656ab8ff86 2024-03-29T08:36:01.186 INFO:teuthology.orchestra.run.smithi037.stdout:osd.4 smithi148 running (19m) 2m ago 34m 101M 2642M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 966a9de11d22 2024-03-29T08:36:01.186 INFO:teuthology.orchestra.run.smithi037.stdout:osd.5 smithi148 running (18m) 2m ago 34m 89.4M 2642M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 474c4f9212ce 2024-03-29T08:36:01.186 INFO:teuthology.orchestra.run.smithi037.stdout:prometheus.smithi037 smithi037 *:9095 running (26m) 2m ago 38m 61.0M - 2.43.0 a07b618ecd1d 30c0223ec589 2024-03-29T08:36:01.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:01.187+0000 7efe35000700 1 -- 172.21.15.37:0/2000471361 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7efe1406cc30 msgr2=0x7efe1406f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:36:01.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:01.187+0000 7efe35000700 1 --2- 172.21.15.37:0/2000471361 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7efe1406cc30 0x7efe1406f0e0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7efe30111530 tx=0x7efe1c009e30 comp rx=0 tx=0).stop 2024-03-29T08:36:01.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:01.187+0000 7efe35000700 1 -- 172.21.15.37:0/2000471361 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efe30105000 msgr2=0x7efe30115c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:36:01.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:01.187+0000 7efe35000700 1 --2- 172.21.15.37:0/2000471361 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efe30105000 0x7efe30115c90 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7efe2000eb40 tx=0x7efe2000b040 comp rx=0 tx=0).stop 2024-03-29T08:36:01.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:01.187+0000 7efe35000700 1 -- 172.21.15.37:0/2000471361 shutdown_connections 2024-03-29T08:36:01.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:01.187+0000 7efe35000700 1 --2- 172.21.15.37:0/2000471361 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7efe1406cc30 0x7efe1406f0e0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:36:01.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:01.187+0000 7efe35000700 1 --2- 172.21.15.37:0/2000471361 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efe30105000 0x7efe30115c90 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:36:01.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:01.187+0000 7efe35000700 1 --2- 172.21.15.37:0/2000471361 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe301059a0 0x7efe301161d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:36:01.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:01.187+0000 7efe35000700 1 -- 172.21.15.37:0/2000471361 >> 172.21.15.37:0/2000471361 conn(0x7efe300fabd0 msgr2=0x7efe301095f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:36:01.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:01.187+0000 7efe35000700 1 -- 172.21.15.37:0/2000471361 shutdown_connections 2024-03-29T08:36:01.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:01.188+0000 7efe35000700 1 -- 172.21.15.37:0/2000471361 wait complete. 2024-03-29T08:36:01.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:01.325+0000 7f5055400700 1 -- 172.21.15.37:0/4156904554 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f50501037a0 msgr2=0x7f50500fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:36:01.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:01.325+0000 7f5055400700 1 --2- 172.21.15.37:0/4156904554 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f50501037a0 0x7f50500fce10 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f503c004940 tx=0x7f503c009e30 comp rx=0 tx=0).stop 2024-03-29T08:36:01.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:01.325+0000 7f5055400700 1 -- 172.21.15.37:0/4156904554 shutdown_connections 2024-03-29T08:36:01.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:01.325+0000 7f5055400700 1 --2- 172.21.15.37:0/4156904554 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f50501037a0 0x7f50500fce10 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:36:01.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:01.325+0000 7f5055400700 1 --2- 172.21.15.37:0/4156904554 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5050102e00 0x7f50501031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:36:01.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:01.325+0000 7f5055400700 1 -- 172.21.15.37:0/4156904554 >> 172.21.15.37:0/4156904554 conn(0x7f50500f89b0 msgr2=0x7f50500fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:36:01.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:01.325+0000 7f5055400700 1 -- 172.21.15.37:0/4156904554 shutdown_connections 2024-03-29T08:36:01.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:01.325+0000 7f5055400700 1 -- 172.21.15.37:0/4156904554 wait complete. 2024-03-29T08:36:01.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:01.326+0000 7f5055400700 1 Processor -- start 2024-03-29T08:36:01.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:01.326+0000 7f5055400700 1 -- start start 2024-03-29T08:36:01.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:01.327+0000 7f5055400700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5050102e00 0x7f5050115d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:36:01.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:01.327+0000 7f5055400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f50501037a0 0x7f5050116240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:36:01.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:01.327+0000 7f5055400700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5050116890 con 0x7f50501037a0 2024-03-29T08:36:01.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:01.327+0000 7f5055400700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f505010fc60 con 0x7f5050102e00 2024-03-29T08:36:01.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:01.327+0000 7f504de00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f50501037a0 0x7f5050116240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:36:01.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:01.327+0000 7f504de00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f50501037a0 0x7f5050116240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42636/0 (socket says 172.21.15.37:42636) 2024-03-29T08:36:01.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:01.327+0000 7f504de00700 1 -- 172.21.15.37:0/3848091058 learned_addr learned my addr 172.21.15.37:0/3848091058 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:36:01.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:01.327+0000 7f504de00700 1 -- 172.21.15.37:0/3848091058 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5050102e00 msgr2=0x7f5050115d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:36:01.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:01.327+0000 7f504de00700 1 --2- 172.21.15.37:0/3848091058 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5050102e00 0x7f5050115d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:36:01.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:01.327+0000 7f504de00700 1 -- 172.21.15.37:0/3848091058 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f503c005040 con 0x7f50501037a0 2024-03-29T08:36:01.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:01.328+0000 7f504de00700 1 --2- 172.21.15.37:0/3848091058 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f50501037a0 0x7f5050116240 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f503c0093b0 tx=0x7f503c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:36:01.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:01.328+0000 7f5046a00700 1 -- 172.21.15.37:0/3848091058 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f503c00e040 con 0x7f50501037a0 2024-03-29T08:36:01.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:01.328+0000 7f5055400700 1 -- 172.21.15.37:0/3848091058 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f505010fee0 con 0x7f50501037a0 2024-03-29T08:36:01.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:01.328+0000 7f5046a00700 1 -- 172.21.15.37:0/3848091058 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f503c013420 con 0x7f50501037a0 2024-03-29T08:36:01.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:01.328+0000 7f5046a00700 1 -- 172.21.15.37:0/3848091058 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f503c01c630 con 0x7f50501037a0 2024-03-29T08:36:01.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:01.328+0000 7f5055400700 1 -- 172.21.15.37:0/3848091058 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50501104a0 con 0x7f50501037a0 2024-03-29T08:36:01.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:01.329+0000 7f5055400700 1 -- 172.21.15.37:0/3848091058 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50500fe550 con 0x7f50501037a0 2024-03-29T08:36:01.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:01.330+0000 7f5046a00700 1 -- 172.21.15.37:0/3848091058 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f503c0139e0 con 0x7f50501037a0 2024-03-29T08:36:01.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:01.331+0000 7f5046a00700 1 --2- 172.21.15.37:0/3848091058 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f503406cc20 0x7f503406f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:36:01.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:01.331+0000 7f5046a00700 1 -- 172.21.15.37:0/3848091058 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f503c08d300 con 0x7f50501037a0 2024-03-29T08:36:01.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:01.331+0000 7f504e800700 1 --2- 172.21.15.37:0/3848091058 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f503406cc20 0x7f503406f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:36:01.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:01.332+0000 7f504e800700 1 --2- 172.21.15.37:0/3848091058 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f503406cc20 0x7f503406f0d0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f5040004d00 tx=0x7f5040009ce0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:36:01.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:01.336+0000 7f5046a00700 1 -- 172.21.15.37:0/3848091058 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f503c05ad50 con 0x7f50501037a0 2024-03-29T08:36:01.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:01.636+0000 7f5055400700 1 -- 172.21.15.37:0/3848091058 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f505004bed0 con 0x7f50501037a0 2024-03-29T08:36:01.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:01.636+0000 7f5046a00700 1 -- 172.21.15.37:0/3848091058 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7f503c05a8e0 con 0x7f50501037a0 2024-03-29T08:36:01.636 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-29T08:36:01.636 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": { 2024-03-29T08:36:01.636 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 2 2024-03-29T08:36:01.636 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:36:01.636 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": { 2024-03-29T08:36:01.637 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 2 2024-03-29T08:36:01.637 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:36:01.637 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": { 2024-03-29T08:36:01.637 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 6 2024-03-29T08:36:01.637 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:36:01.637 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": { 2024-03-29T08:36:01.637 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 4 2024-03-29T08:36:01.637 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:36:01.637 INFO:teuthology.orchestra.run.smithi037.stdout: "overall": { 2024-03-29T08:36:01.637 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 14 2024-03-29T08:36:01.637 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-29T08:36:01.637 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-29T08:36:01.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:01.639+0000 7f5055400700 1 -- 172.21.15.37:0/3848091058 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f503406cc20 msgr2=0x7f503406f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:36:01.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:01.639+0000 7f5055400700 1 --2- 172.21.15.37:0/3848091058 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f503406cc20 0x7f503406f0d0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f5040004d00 tx=0x7f5040009ce0 comp rx=0 tx=0).stop 2024-03-29T08:36:01.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:01.639+0000 7f5055400700 1 -- 172.21.15.37:0/3848091058 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f50501037a0 msgr2=0x7f5050116240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:36:01.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:01.639+0000 7f5055400700 1 --2- 172.21.15.37:0/3848091058 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f50501037a0 0x7f5050116240 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f503c0093b0 tx=0x7f503c009e30 comp rx=0 tx=0).stop 2024-03-29T08:36:01.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:01.639+0000 7f5055400700 1 -- 172.21.15.37:0/3848091058 shutdown_connections 2024-03-29T08:36:01.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:01.639+0000 7f5055400700 1 --2- 172.21.15.37:0/3848091058 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f503406cc20 0x7f503406f0d0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:36:01.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:01.639+0000 7f5055400700 1 --2- 172.21.15.37:0/3848091058 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5050102e00 0x7f5050115d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:36:01.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:01.639+0000 7f5055400700 1 --2- 172.21.15.37:0/3848091058 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f50501037a0 0x7f5050116240 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:36:01.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:01.639+0000 7f5055400700 1 -- 172.21.15.37:0/3848091058 >> 172.21.15.37:0/3848091058 conn(0x7f50500f89b0 msgr2=0x7f50500f9ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:36:01.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:01.639+0000 7f5055400700 1 -- 172.21.15.37:0/3848091058 shutdown_connections 2024-03-29T08:36:01.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:01.639+0000 7f5055400700 1 -- 172.21.15.37:0/3848091058 wait complete. 2024-03-29T08:36:01.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:01.776+0000 7f77aa200700 1 -- 172.21.15.37:0/705346965 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f77a4102e20 msgr2=0x7f77a41031f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:36:01.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:01.776+0000 7f77aa200700 1 --2- 172.21.15.37:0/705346965 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f77a4102e20 0x7f77a41031f0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f7794004940 tx=0x7f7794009e30 comp rx=0 tx=0).stop 2024-03-29T08:36:01.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:01.777+0000 7f77aa200700 1 -- 172.21.15.37:0/705346965 shutdown_connections 2024-03-29T08:36:01.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:01.777+0000 7f77aa200700 1 --2- 172.21.15.37:0/705346965 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f77a41037c0 0x7f77a40fce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:36:01.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:01.777+0000 7f77aa200700 1 --2- 172.21.15.37:0/705346965 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f77a4102e20 0x7f77a41031f0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:36:01.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:01.777+0000 7f77aa200700 1 -- 172.21.15.37:0/705346965 >> 172.21.15.37:0/705346965 conn(0x7f77a40f8970 msgr2=0x7f77a40fad80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:36:01.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:01.777+0000 7f77aa200700 1 -- 172.21.15.37:0/705346965 shutdown_connections 2024-03-29T08:36:01.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:01.777+0000 7f77aa200700 1 -- 172.21.15.37:0/705346965 wait complete. 2024-03-29T08:36:01.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:01.777+0000 7f77aa200700 1 Processor -- start 2024-03-29T08:36:01.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:01.778+0000 7f77aa200700 1 -- start start 2024-03-29T08:36:01.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:01.778+0000 7f77aa200700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f77a4102e20 0x7f77a4115d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:36:01.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:01.778+0000 7f77aa200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f77a41037c0 0x7f77a41162a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:36:01.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:01.778+0000 7f77aa200700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77a41168f0 con 0x7f77a41037c0 2024-03-29T08:36:01.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:01.778+0000 7f77aa200700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77a410fcc0 con 0x7f77a4102e20 2024-03-29T08:36:01.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:01.778+0000 7f77a2a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f77a41037c0 0x7f77a41162a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:36:01.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:01.778+0000 7f77a2a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f77a41037c0 0x7f77a41162a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42640/0 (socket says 172.21.15.37:42640) 2024-03-29T08:36:01.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:01.778+0000 7f77a2a00700 1 -- 172.21.15.37:0/460596368 learned_addr learned my addr 172.21.15.37:0/460596368 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:36:01.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:01.779+0000 7f77a3400700 1 --2- 172.21.15.37:0/460596368 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f77a4102e20 0x7f77a4115d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:36:01.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:01.779+0000 7f77a2a00700 1 -- 172.21.15.37:0/460596368 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f77a4102e20 msgr2=0x7f77a4115d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:36:01.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:01.779+0000 7f77a2a00700 1 --2- 172.21.15.37:0/460596368 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f77a4102e20 0x7f77a4115d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:36:01.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:01.779+0000 7f77a2a00700 1 -- 172.21.15.37:0/460596368 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7794005040 con 0x7f77a41037c0 2024-03-29T08:36:01.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:01.779+0000 7f77a2a00700 1 --2- 172.21.15.37:0/460596368 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f77a41037c0 0x7f77a41162a0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f7790002a50 tx=0x7f7790004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:36:01.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:01.779+0000 7f779b400700 1 -- 172.21.15.37:0/460596368 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7790007cc0 con 0x7f77a41037c0 2024-03-29T08:36:01.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:01.779+0000 7f77aa200700 1 -- 172.21.15.37:0/460596368 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77a410ffa0 con 0x7f77a41037c0 2024-03-29T08:36:01.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:01.779+0000 7f779b400700 1 -- 172.21.15.37:0/460596368 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f779000f040 con 0x7f77a41037c0 2024-03-29T08:36:01.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:01.779+0000 7f779b400700 1 -- 172.21.15.37:0/460596368 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f779000d920 con 0x7f77a41037c0 2024-03-29T08:36:01.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:01.780+0000 7f77aa200700 1 -- 172.21.15.37:0/460596368 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77a41104e0 con 0x7f77a41037c0 2024-03-29T08:36:01.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:01.781+0000 7f77aa200700 1 -- 172.21.15.37:0/460596368 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77a40fe570 con 0x7f77a41037c0 2024-03-29T08:36:01.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:01.781+0000 7f779b400700 1 -- 172.21.15.37:0/460596368 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f7790009680 con 0x7f77a41037c0 2024-03-29T08:36:01.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:01.782+0000 7f779b400700 1 --2- 172.21.15.37:0/460596368 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f778806cc30 0x7f778806f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:36:01.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:01.783+0000 7f779b400700 1 -- 172.21.15.37:0/460596368 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f779008be50 con 0x7f77a41037c0 2024-03-29T08:36:01.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:01.783+0000 7f77a3400700 1 --2- 172.21.15.37:0/460596368 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f778806cc30 0x7f778806f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:36:01.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:01.783+0000 7f77a3400700 1 --2- 172.21.15.37:0/460596368 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f778806cc30 0x7f778806f0e0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f7794009fd0 tx=0x7f7794009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:36:01.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:01.787+0000 7f779b400700 1 -- 172.21.15.37:0/460596368 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f7790059820 con 0x7f77a41037c0 2024-03-29T08:36:01.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:36:01 smithi037 ceph-mon[142325]: from='client.35374 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:36:01.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:36:01 smithi037 ceph-mon[142325]: from='client.35378 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:36:01.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:36:01 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/3848091058' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:36:02.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:02.044+0000 7f77aa200700 1 -- 172.21.15.37:0/460596368 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f77a404bed0 con 0x7f77a41037c0 2024-03-29T08:36:02.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:02.045+0000 7f779b400700 1 -- 172.21.15.37:0/460596368 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 30 v30) v1 ==== 76+0+1841 (secure 0 0 0) 0x7f77900593b0 con 0x7f77a41037c0 2024-03-29T08:36:02.045 INFO:teuthology.orchestra.run.smithi037.stdout:e30 2024-03-29T08:36:02.045 INFO:teuthology.orchestra.run.smithi037.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-29T08:36:02.045 INFO:teuthology.orchestra.run.smithi037.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T08:36:02.045 INFO:teuthology.orchestra.run.smithi037.stdout:legacy client fscid: 1 2024-03-29T08:36:02.045 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:36:02.045 INFO:teuthology.orchestra.run.smithi037.stdout:Filesystem 'cephfs' (1) 2024-03-29T08:36:02.045 INFO:teuthology.orchestra.run.smithi037.stdout:fs_name cephfs 2024-03-29T08:36:02.045 INFO:teuthology.orchestra.run.smithi037.stdout:epoch 21 2024-03-29T08:36:02.045 INFO:teuthology.orchestra.run.smithi037.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-03-29T08:36:02.045 INFO:teuthology.orchestra.run.smithi037.stdout:created 2024-03-29T08:03:29.241269+0000 2024-03-29T08:36:02.045 INFO:teuthology.orchestra.run.smithi037.stdout:modified 2024-03-29T08:17:16.237238+0000 2024-03-29T08:36:02.045 INFO:teuthology.orchestra.run.smithi037.stdout:tableserver 0 2024-03-29T08:36:02.045 INFO:teuthology.orchestra.run.smithi037.stdout:root 0 2024-03-29T08:36:02.045 INFO:teuthology.orchestra.run.smithi037.stdout:session_timeout 60 2024-03-29T08:36:02.046 INFO:teuthology.orchestra.run.smithi037.stdout:session_autoclose 300 2024-03-29T08:36:02.046 INFO:teuthology.orchestra.run.smithi037.stdout:max_file_size 1099511627776 2024-03-29T08:36:02.046 INFO:teuthology.orchestra.run.smithi037.stdout:max_xattr_size 65536 2024-03-29T08:36:02.046 INFO:teuthology.orchestra.run.smithi037.stdout:required_client_features {} 2024-03-29T08:36:02.046 INFO:teuthology.orchestra.run.smithi037.stdout:last_failure 0 2024-03-29T08:36:02.046 INFO:teuthology.orchestra.run.smithi037.stdout:last_failure_osd_epoch 82 2024-03-29T08:36:02.046 INFO:teuthology.orchestra.run.smithi037.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T08:36:02.046 INFO:teuthology.orchestra.run.smithi037.stdout:max_mds 2 2024-03-29T08:36:02.046 INFO:teuthology.orchestra.run.smithi037.stdout:in 0,1 2024-03-29T08:36:02.046 INFO:teuthology.orchestra.run.smithi037.stdout:up {} 2024-03-29T08:36:02.046 INFO:teuthology.orchestra.run.smithi037.stdout:failed 0,1 2024-03-29T08:36:02.046 INFO:teuthology.orchestra.run.smithi037.stdout:damaged 2024-03-29T08:36:02.046 INFO:teuthology.orchestra.run.smithi037.stdout:stopped 2024-03-29T08:36:02.046 INFO:teuthology.orchestra.run.smithi037.stdout:data_pools [3] 2024-03-29T08:36:02.046 INFO:teuthology.orchestra.run.smithi037.stdout:metadata_pool 2 2024-03-29T08:36:02.046 INFO:teuthology.orchestra.run.smithi037.stdout:inline_data disabled 2024-03-29T08:36:02.046 INFO:teuthology.orchestra.run.smithi037.stdout:balancer 2024-03-29T08:36:02.047 INFO:teuthology.orchestra.run.smithi037.stdout:bal_rank_mask -1 2024-03-29T08:36:02.047 INFO:teuthology.orchestra.run.smithi037.stdout:standby_count_wanted 1 2024-03-29T08:36:02.047 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:36:02.047 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:36:02.047 INFO:teuthology.orchestra.run.smithi037.stdout:Standby daemons: 2024-03-29T08:36:02.047 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:36:02.047 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi037.mmwgxe{-1:34440} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.37:6828/1248142005,v1:172.21.15.37:6829/1248142005] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:36:02.047 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi037.iifsmh{-1:34444} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.37:6826/2060882959,v1:172.21.15.37:6827/2060882959] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:36:02.047 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi148.sqteih{-1:44395} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.148:6824/4078652242,v1:172.21.15.148:6825/4078652242] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:36:02.047 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi148.qukpjh{-1:45007} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.148:6826/2394751105,v1:172.21.15.148:6827/2394751105] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:36:02.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:02.047+0000 7f77aa200700 1 -- 172.21.15.37:0/460596368 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f778806cc30 msgr2=0x7f778806f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:36:02.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:02.047+0000 7f77aa200700 1 --2- 172.21.15.37:0/460596368 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f778806cc30 0x7f778806f0e0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f7794009fd0 tx=0x7f7794009e30 comp rx=0 tx=0).stop 2024-03-29T08:36:02.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:02.047+0000 7f77aa200700 1 -- 172.21.15.37:0/460596368 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f77a41037c0 msgr2=0x7f77a41162a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:36:02.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:02.047+0000 7f77aa200700 1 --2- 172.21.15.37:0/460596368 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f77a41037c0 0x7f77a41162a0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f7790002a50 tx=0x7f7790004600 comp rx=0 tx=0).stop 2024-03-29T08:36:02.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:02.047+0000 7f77aa200700 1 -- 172.21.15.37:0/460596368 shutdown_connections 2024-03-29T08:36:02.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:02.047+0000 7f77aa200700 1 --2- 172.21.15.37:0/460596368 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f778806cc30 0x7f778806f0e0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:36:02.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:02.047+0000 7f77aa200700 1 --2- 172.21.15.37:0/460596368 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f77a4102e20 0x7f77a4115d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:36:02.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:02.047+0000 7f77aa200700 1 --2- 172.21.15.37:0/460596368 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f77a41037c0 0x7f77a41162a0 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:36:02.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:02.047+0000 7f77aa200700 1 -- 172.21.15.37:0/460596368 >> 172.21.15.37:0/460596368 conn(0x7f77a40f8970 msgr2=0x7f77a40f99d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:36:02.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:02.047+0000 7f77aa200700 1 -- 172.21.15.37:0/460596368 shutdown_connections 2024-03-29T08:36:02.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:02.048+0000 7f77aa200700 1 -- 172.21.15.37:0/460596368 wait complete. 2024-03-29T08:36:02.049 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 30 2024-03-29T08:36:02.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:36:01 smithi148 ceph-mon[106692]: from='client.35374 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:36:02.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:36:01 smithi148 ceph-mon[106692]: from='client.35378 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:36:02.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:36:01 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/3848091058' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:36:02.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:02.185+0000 7fe154c00700 1 -- 172.21.15.37:0/1749865456 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe1500fe850 msgr2=0x7fe150106f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:36:02.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:02.185+0000 7fe154c00700 1 --2- 172.21.15.37:0/1749865456 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe1500fe850 0x7fe150106f20 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fe13c004970 tx=0x7fe13c009e30 comp rx=0 tx=0).stop 2024-03-29T08:36:02.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:02.185+0000 7fe154c00700 1 -- 172.21.15.37:0/1749865456 shutdown_connections 2024-03-29T08:36:02.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:02.185+0000 7fe154c00700 1 --2- 172.21.15.37:0/1749865456 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe1500fe850 0x7fe150106f20 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:36:02.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:02.185+0000 7fe154c00700 1 --2- 172.21.15.37:0/1749865456 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe1500fdeb0 0x7fe1500fe280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:36:02.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:02.185+0000 7fe154c00700 1 -- 172.21.15.37:0/1749865456 >> 172.21.15.37:0/1749865456 conn(0x7fe150072bc0 msgr2=0x7fe150072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:36:02.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:02.185+0000 7fe154c00700 1 -- 172.21.15.37:0/1749865456 shutdown_connections 2024-03-29T08:36:02.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:02.185+0000 7fe154c00700 1 -- 172.21.15.37:0/1749865456 wait complete. 2024-03-29T08:36:02.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:02.186+0000 7fe154c00700 1 Processor -- start 2024-03-29T08:36:02.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:02.186+0000 7fe154c00700 1 -- start start 2024-03-29T08:36:02.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:02.187+0000 7fe154c00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe1500fdeb0 0x7fe150113a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:36:02.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:02.187+0000 7fe154c00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe1500fe850 0x7fe150113f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:36:02.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:02.187+0000 7fe154c00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1501145a0 con 0x7fe1500fdeb0 2024-03-29T08:36:02.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:02.187+0000 7fe154c00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe15010d990 con 0x7fe1500fe850 2024-03-29T08:36:02.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:02.187+0000 7fe14de00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe1500fdeb0 0x7fe150113a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:36:02.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:02.187+0000 7fe14de00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe1500fdeb0 0x7fe150113a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42658/0 (socket says 172.21.15.37:42658) 2024-03-29T08:36:02.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:02.187+0000 7fe14de00700 1 -- 172.21.15.37:0/836662821 learned_addr learned my addr 172.21.15.37:0/836662821 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:36:02.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:02.187+0000 7fe14d400700 1 --2- 172.21.15.37:0/836662821 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe1500fe850 0x7fe150113f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:36:02.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:02.188+0000 7fe14de00700 1 -- 172.21.15.37:0/836662821 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe1500fe850 msgr2=0x7fe150113f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:36:02.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:02.188+0000 7fe14de00700 1 --2- 172.21.15.37:0/836662821 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe1500fe850 0x7fe150113f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:36:02.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:02.188+0000 7fe14de00700 1 -- 172.21.15.37:0/836662821 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe13c005040 con 0x7fe1500fdeb0 2024-03-29T08:36:02.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:02.188+0000 7fe14d400700 1 --2- 172.21.15.37:0/836662821 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe1500fe850 0x7fe150113f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:36:02.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:02.188+0000 7fe14de00700 1 --2- 172.21.15.37:0/836662821 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe1500fdeb0 0x7fe150113a10 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fe14000eb40 tx=0x7fe14000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:36:02.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:02.188+0000 7fe146000700 1 -- 172.21.15.37:0/836662821 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe140007cc0 con 0x7fe1500fdeb0 2024-03-29T08:36:02.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:02.188+0000 7fe154c00700 1 -- 172.21.15.37:0/836662821 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe15010dc10 con 0x7fe1500fdeb0 2024-03-29T08:36:02.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:02.188+0000 7fe146000700 1 -- 172.21.15.37:0/836662821 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fe14000f040 con 0x7fe1500fdeb0 2024-03-29T08:36:02.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:02.188+0000 7fe146000700 1 -- 172.21.15.37:0/836662821 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe140009570 con 0x7fe1500fdeb0 2024-03-29T08:36:02.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:02.188+0000 7fe154c00700 1 -- 172.21.15.37:0/836662821 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe15010e150 con 0x7fe1500fdeb0 2024-03-29T08:36:02.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:02.189+0000 7fe154c00700 1 -- 172.21.15.37:0/836662821 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe150065e10 con 0x7fe1500fdeb0 2024-03-29T08:36:02.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:02.190+0000 7fe146000700 1 -- 172.21.15.37:0/836662821 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fe1400096d0 con 0x7fe1500fdeb0 2024-03-29T08:36:02.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:02.192+0000 7fe146000700 1 --2- 172.21.15.37:0/836662821 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fe13406cc30 0x7fe13406f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:36:02.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:02.192+0000 7fe146000700 1 -- 172.21.15.37:0/836662821 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fe14008bda0 con 0x7fe1500fdeb0 2024-03-29T08:36:02.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:02.192+0000 7fe14d400700 1 --2- 172.21.15.37:0/836662821 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fe13406cc30 0x7fe13406f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:36:02.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:02.193+0000 7fe14d400700 1 --2- 172.21.15.37:0/836662821 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fe13406cc30 0x7fe13406f0e0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fe13c005010 tx=0x7fe13c009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:36:02.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:02.196+0000 7fe146000700 1 -- 172.21.15.37:0/836662821 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fe140059820 con 0x7fe1500fdeb0 2024-03-29T08:36:02.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:02.429+0000 7fe154c00700 1 -- 172.21.15.37:0/836662821 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe150109e20 con 0x7fe13406cc30 2024-03-29T08:36:02.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:02.430+0000 7fe146000700 1 -- 172.21.15.37:0/836662821 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fe150109e20 con 0x7fe13406cc30 2024-03-29T08:36:02.430 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-29T08:36:02.430 INFO:teuthology.orchestra.run.smithi037.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59", 2024-03-29T08:36:02.430 INFO:teuthology.orchestra.run.smithi037.stdout: "in_progress": true, 2024-03-29T08:36:02.430 INFO:teuthology.orchestra.run.smithi037.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-29T08:36:02.430 INFO:teuthology.orchestra.run.smithi037.stdout: "services_complete": [ 2024-03-29T08:36:02.430 INFO:teuthology.orchestra.run.smithi037.stdout: "crash", 2024-03-29T08:36:02.431 INFO:teuthology.orchestra.run.smithi037.stdout: "osd", 2024-03-29T08:36:02.431 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr", 2024-03-29T08:36:02.431 INFO:teuthology.orchestra.run.smithi037.stdout: "mds", 2024-03-29T08:36:02.431 INFO:teuthology.orchestra.run.smithi037.stdout: "mon" 2024-03-29T08:36:02.431 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-29T08:36:02.431 INFO:teuthology.orchestra.run.smithi037.stdout: "progress": "16/21 daemons upgraded", 2024-03-29T08:36:02.431 INFO:teuthology.orchestra.run.smithi037.stdout: "message": "Currently upgrading mds daemons", 2024-03-29T08:36:02.431 INFO:teuthology.orchestra.run.smithi037.stdout: "is_paused": false 2024-03-29T08:36:02.431 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-29T08:36:02.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:02.433+0000 7fe154c00700 1 -- 172.21.15.37:0/836662821 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fe13406cc30 msgr2=0x7fe13406f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:36:02.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:02.433+0000 7fe154c00700 1 --2- 172.21.15.37:0/836662821 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fe13406cc30 0x7fe13406f0e0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fe13c005010 tx=0x7fe13c009e30 comp rx=0 tx=0).stop 2024-03-29T08:36:02.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:02.433+0000 7fe154c00700 1 -- 172.21.15.37:0/836662821 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe1500fdeb0 msgr2=0x7fe150113a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:36:02.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:02.433+0000 7fe154c00700 1 --2- 172.21.15.37:0/836662821 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe1500fdeb0 0x7fe150113a10 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fe14000eb40 tx=0x7fe14000b040 comp rx=0 tx=0).stop 2024-03-29T08:36:02.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:02.433+0000 7fe154c00700 1 -- 172.21.15.37:0/836662821 shutdown_connections 2024-03-29T08:36:02.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:02.433+0000 7fe154c00700 1 --2- 172.21.15.37:0/836662821 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fe13406cc30 0x7fe13406f0e0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:36:02.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:02.433+0000 7fe154c00700 1 --2- 172.21.15.37:0/836662821 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe1500fdeb0 0x7fe150113a10 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:36:02.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:02.433+0000 7fe154c00700 1 --2- 172.21.15.37:0/836662821 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe1500fe850 0x7fe150113f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:36:02.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:02.433+0000 7fe154c00700 1 -- 172.21.15.37:0/836662821 >> 172.21.15.37:0/836662821 conn(0x7fe150072bc0 msgr2=0x7fe150103780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:36:02.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:02.433+0000 7fe154c00700 1 -- 172.21.15.37:0/836662821 shutdown_connections 2024-03-29T08:36:02.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:02.433+0000 7fe154c00700 1 -- 172.21.15.37:0/836662821 wait complete. 2024-03-29T08:36:02.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:02.570+0000 7f67f3200700 1 -- 172.21.15.37:0/1062267071 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f67ec105a30 msgr2=0x7f67ec0ff0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:36:02.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:02.570+0000 7f67f3200700 1 --2- 172.21.15.37:0/1062267071 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f67ec105a30 0x7f67ec0ff0a0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f67d8004940 tx=0x7f67d8009e30 comp rx=0 tx=0).stop 2024-03-29T08:36:02.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:02.570+0000 7f67f3200700 1 -- 172.21.15.37:0/1062267071 shutdown_connections 2024-03-29T08:36:02.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:02.570+0000 7f67f3200700 1 --2- 172.21.15.37:0/1062267071 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f67ec105a30 0x7f67ec0ff0a0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:36:02.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:02.570+0000 7f67f3200700 1 --2- 172.21.15.37:0/1062267071 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f67ec105090 0x7f67ec105460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:36:02.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:02.570+0000 7f67f3200700 1 -- 172.21.15.37:0/1062267071 >> 172.21.15.37:0/1062267071 conn(0x7f67ec0fac00 msgr2=0x7f67ec0fd010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:36:02.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:02.570+0000 7f67f3200700 1 -- 172.21.15.37:0/1062267071 shutdown_connections 2024-03-29T08:36:02.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:02.571+0000 7f67f3200700 1 -- 172.21.15.37:0/1062267071 wait complete. 2024-03-29T08:36:02.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:02.571+0000 7f67f3200700 1 Processor -- start 2024-03-29T08:36:02.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:02.571+0000 7f67f3200700 1 -- start start 2024-03-29T08:36:02.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:02.572+0000 7f67f3200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f67ec105090 0x7f67ec115a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:36:02.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:02.572+0000 7f67f3200700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f67ec105a30 0x7f67ec115f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:36:02.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:02.572+0000 7f67f3200700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67ec1165a0 con 0x7f67ec105090 2024-03-29T08:36:02.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:02.572+0000 7f67f3200700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67ec10fba0 con 0x7f67ec105a30 2024-03-29T08:36:02.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:02.572+0000 7f67ebe00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f67ec105090 0x7f67ec115a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:36:02.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:02.572+0000 7f67ebe00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f67ec105090 0x7f67ec115a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42676/0 (socket says 172.21.15.37:42676) 2024-03-29T08:36:02.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:02.572+0000 7f67ebe00700 1 -- 172.21.15.37:0/2040425165 learned_addr learned my addr 172.21.15.37:0/2040425165 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:36:02.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:02.572+0000 7f67eb400700 1 --2- 172.21.15.37:0/2040425165 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f67ec105a30 0x7f67ec115f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:36:02.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:02.573+0000 7f67ebe00700 1 -- 172.21.15.37:0/2040425165 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f67ec105a30 msgr2=0x7f67ec115f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:36:02.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:02.573+0000 7f67ebe00700 1 --2- 172.21.15.37:0/2040425165 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f67ec105a30 0x7f67ec115f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:36:02.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:02.573+0000 7f67ebe00700 1 -- 172.21.15.37:0/2040425165 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67d8005040 con 0x7f67ec105090 2024-03-29T08:36:02.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:02.573+0000 7f67ebe00700 1 --2- 172.21.15.37:0/2040425165 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f67ec105090 0x7f67ec115a10 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f67dc00eb40 tx=0x7f67dc00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:36:02.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:02.573+0000 7f67e3e00700 1 -- 172.21.15.37:0/2040425165 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f67dc007cc0 con 0x7f67ec105090 2024-03-29T08:36:02.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:02.573+0000 7f67e3e00700 1 -- 172.21.15.37:0/2040425165 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f67dc00f040 con 0x7f67ec105090 2024-03-29T08:36:02.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:02.573+0000 7f67f3200700 1 -- 172.21.15.37:0/2040425165 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67ec10fe20 con 0x7f67ec105090 2024-03-29T08:36:02.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:02.573+0000 7f67e3e00700 1 -- 172.21.15.37:0/2040425165 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f67dc00c990 con 0x7f67ec105090 2024-03-29T08:36:02.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:02.573+0000 7f67f3200700 1 -- 172.21.15.37:0/2040425165 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67ec1103b0 con 0x7f67ec105090 2024-03-29T08:36:02.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:02.574+0000 7f67f3200700 1 -- 172.21.15.37:0/2040425165 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67ec1023a0 con 0x7f67ec105090 2024-03-29T08:36:02.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:02.575+0000 7f67e3e00700 1 -- 172.21.15.37:0/2040425165 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f67dc00cc00 con 0x7f67ec105090 2024-03-29T08:36:02.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:02.576+0000 7f67e3e00700 1 --2- 172.21.15.37:0/2040425165 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f67d006cc20 0x7f67d006f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:36:02.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:02.576+0000 7f67e3e00700 1 -- 172.21.15.37:0/2040425165 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f67dc08bcd0 con 0x7f67ec105090 2024-03-29T08:36:02.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:02.576+0000 7f67eb400700 1 --2- 172.21.15.37:0/2040425165 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f67d006cc20 0x7f67d006f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:36:02.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:02.577+0000 7f67eb400700 1 --2- 172.21.15.37:0/2040425165 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f67d006cc20 0x7f67d006f0d0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f67ec110e80 tx=0x7f67d8009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:36:02.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:02.581+0000 7f67e3e00700 1 -- 172.21.15.37:0/2040425165 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f67dc059750 con 0x7f67ec105090 2024-03-29T08:36:02.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:02.876+0000 7f67f3200700 1 -- 172.21.15.37:0/2040425165 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f67ec10c030 con 0x7f67ec105090 2024-03-29T08:36:02.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:02.876+0000 7f67e3e00700 1 -- 172.21.15.37:0/2040425165 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+3583 (secure 0 0 0) 0x7f67dc019e10 con 0x7f67ec105090 2024-03-29T08:36:02.876 INFO:teuthology.orchestra.run.smithi037.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem is offline 2024-03-29T08:36:02.876 INFO:teuthology.orchestra.run.smithi037.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-03-29T08:36:02.876 INFO:teuthology.orchestra.run.smithi037.stdout: host smithi148 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/mon.smithi148/config 2024-03-29T08:36:02.876 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/podman: timeout after 895 seconds 2024-03-29T08:36:02.876 INFO:teuthology.orchestra.run.smithi037.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59 -e NODE_NAME=smithi148 -e CEPH_USE_RANDOM_NONCE=1 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4:/var/run/ceph:z -v /var/log/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4:/var/log/ceph:z -v /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpkx2rz2d4:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59 inventory --format=json-pretty --filter-for-batch 2024-03-29T08:36:02.876 INFO:teuthology.orchestra.run.smithi037.stdout:Traceback (most recent call last): 2024-03-29T08:36:02.876 INFO:teuthology.orchestra.run.smithi037.stdout: File "/usr/lib64/python3.6/runpy.py", line 193, in _run_module_as_main 2024-03-29T08:36:02.876 INFO:teuthology.orchestra.run.smithi037.stdout: "__main__", mod_spec) 2024-03-29T08:36:02.877 INFO:teuthology.orchestra.run.smithi037.stdout: File "/usr/lib64/python3.6/runpy.py", line 85, in _run_code 2024-03-29T08:36:02.877 INFO:teuthology.orchestra.run.smithi037.stdout: exec(code, run_globals) 2024-03-29T08:36:02.877 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 10889, in 2024-03-29T08:36:02.877 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 10877, in main 2024-03-29T08:36:02.877 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 2576, in _infer_config 2024-03-29T08:36:02.877 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 2492, in _infer_fsid 2024-03-29T08:36:02.877 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 2604, in _infer_image 2024-03-29T08:36:02.877 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 2479, in _validate_fsid 2024-03-29T08:36:02.877 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 7145, in command_ceph_volume 2024-03-29T08:36:02.877 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 2267, in call_throws 2024-03-29T08:36:02.877 INFO:teuthology.orchestra.run.smithi037.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59 -e NODE_NAME=smithi148 -e CEPH_USE_RANDOM_NONCE=1 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4:/var/run/ceph:z -v /var/log/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4:/var/log/ceph:z -v /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpkx2rz2d4:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59 inventory --format=json-pretty --filter-for-batch 2024-03-29T08:36:02.877 INFO:teuthology.orchestra.run.smithi037.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-03-29T08:36:02.877 INFO:teuthology.orchestra.run.smithi037.stdout: fs cephfs is degraded 2024-03-29T08:36:02.877 INFO:teuthology.orchestra.run.smithi037.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-03-29T08:36:02.877 INFO:teuthology.orchestra.run.smithi037.stdout: fs cephfs is offline because no MDS is active for it. 2024-03-29T08:36:02.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:02.878+0000 7f67f3200700 1 -- 172.21.15.37:0/2040425165 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f67d006cc20 msgr2=0x7f67d006f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:36:02.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:02.878+0000 7f67f3200700 1 --2- 172.21.15.37:0/2040425165 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f67d006cc20 0x7f67d006f0d0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f67ec110e80 tx=0x7f67d8009e30 comp rx=0 tx=0).stop 2024-03-29T08:36:02.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:02.878+0000 7f67f3200700 1 -- 172.21.15.37:0/2040425165 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f67ec105090 msgr2=0x7f67ec115a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:36:02.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:02.878+0000 7f67f3200700 1 --2- 172.21.15.37:0/2040425165 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f67ec105090 0x7f67ec115a10 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f67dc00eb40 tx=0x7f67dc00b040 comp rx=0 tx=0).stop 2024-03-29T08:36:02.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:02.878+0000 7f67f3200700 1 -- 172.21.15.37:0/2040425165 shutdown_connections 2024-03-29T08:36:02.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:02.878+0000 7f67f3200700 1 --2- 172.21.15.37:0/2040425165 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f67d006cc20 0x7f67d006f0d0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:36:02.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:02.878+0000 7f67f3200700 1 --2- 172.21.15.37:0/2040425165 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f67ec105090 0x7f67ec115a10 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:36:02.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:02.878+0000 7f67f3200700 1 --2- 172.21.15.37:0/2040425165 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f67ec105a30 0x7f67ec115f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:36:02.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:02.878+0000 7f67f3200700 1 -- 172.21.15.37:0/2040425165 >> 172.21.15.37:0/2040425165 conn(0x7f67ec0fac00 msgr2=0x7f67ec0fc980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:36:02.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:02.879+0000 7f67f3200700 1 -- 172.21.15.37:0/2040425165 shutdown_connections 2024-03-29T08:36:02.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:02.879+0000 7f67f3200700 1 -- 172.21.15.37:0/2040425165 wait complete. 2024-03-29T08:36:02.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:36:02 smithi037 ceph-mon[142325]: pgmap v755: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:36:02.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:36:02 smithi037 ceph-mon[142325]: from='client.35382 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:36:02.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:36:02 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/460596368' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T08:36:03.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:36:02 smithi148 ceph-mon[106692]: pgmap v755: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:36:03.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:36:02 smithi148 ceph-mon[106692]: from='client.35382 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:36:03.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:36:02 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/460596368' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T08:36:03.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:36:03 smithi037 ceph-mon[142325]: from='client.35394 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:36:03.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:36:03 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/2040425165' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T08:36:04.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:36:03 smithi148 ceph-mon[106692]: from='client.35394 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:36:04.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:36:03 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/2040425165' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T08:36:04.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:36:04 smithi037 ceph-mon[142325]: pgmap v756: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:36:05.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:36:04 smithi148 ceph-mon[106692]: pgmap v756: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:36:06.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:36:06 smithi037 ceph-mon[142325]: pgmap v757: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:36:07.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:36:06 smithi148 ceph-mon[106692]: pgmap v757: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:36:08.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:36:08 smithi037 ceph-mon[142325]: pgmap v758: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:36:09.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:36:08 smithi148 ceph-mon[106692]: pgmap v758: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:36:10.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:36:10 smithi037 ceph-mon[142325]: pgmap v759: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:36:10.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:36:10 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:36:10.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:36:10 smithi148 ceph-mon[106692]: pgmap v759: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:36:10.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:36:10 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:36:12.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:36:12 smithi148 ceph-mon[106692]: pgmap v760: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:36:12.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:36:12 smithi037 ceph-mon[142325]: pgmap v760: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:36:14.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:36:14 smithi148 ceph-mon[106692]: pgmap v761: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:36:14.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:36:14 smithi037 ceph-mon[142325]: pgmap v761: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:36:16.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:36:16 smithi148 ceph-mon[106692]: pgmap v762: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:36:16.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:36:16 smithi037 ceph-mon[142325]: pgmap v762: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:36:18.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:36:18 smithi148 ceph-mon[106692]: pgmap v763: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:36:18.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:36:18 smithi037 ceph-mon[142325]: pgmap v763: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:36:20.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:36:20 smithi148 ceph-mon[106692]: pgmap v764: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:36:20.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:36:20 smithi037 ceph-mon[142325]: pgmap v764: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:36:22.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:36:22 smithi148 ceph-mon[106692]: pgmap v765: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:36:22.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:36:22 smithi037 ceph-mon[142325]: pgmap v765: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:36:24.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:36:24 smithi148 ceph-mon[106692]: pgmap v766: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:36:24.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:36:24 smithi037 ceph-mon[142325]: pgmap v766: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:36:25.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:36:25 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:36:25.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:36:25 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:36:26.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:36:26 smithi148 ceph-mon[106692]: pgmap v767: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:36:26.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:36:26 smithi037 ceph-mon[142325]: pgmap v767: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:36:28.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:36:28 smithi148 ceph-mon[106692]: pgmap v768: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:36:28.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:36:28 smithi037 ceph-mon[142325]: pgmap v768: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:36:30.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:36:30 smithi148 ceph-mon[106692]: pgmap v769: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:36:30.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:36:30 smithi037 ceph-mon[142325]: pgmap v769: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:36:32.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:36:32 smithi148 ceph-mon[106692]: pgmap v770: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:36:32.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:36:32 smithi037 ceph-mon[142325]: pgmap v770: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:36:33.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:33.032+0000 7f5d93600700 1 -- 172.21.15.37:0/1439089613 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5d8c0fe830 msgr2=0x7f5d8c106f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:36:33.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:33.032+0000 7f5d93600700 1 --2- 172.21.15.37:0/1439089613 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5d8c0fe830 0x7f5d8c106f00 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f5d78004940 tx=0x7f5d78009e30 comp rx=0 tx=0).stop 2024-03-29T08:36:33.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:33.033+0000 7f5d93600700 1 -- 172.21.15.37:0/1439089613 shutdown_connections 2024-03-29T08:36:33.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:33.033+0000 7f5d93600700 1 --2- 172.21.15.37:0/1439089613 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5d8c0fe830 0x7f5d8c106f00 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:36:33.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:33.033+0000 7f5d93600700 1 --2- 172.21.15.37:0/1439089613 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5d8c0fde90 0x7f5d8c0fe260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:36:33.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:33.033+0000 7f5d93600700 1 -- 172.21.15.37:0/1439089613 >> 172.21.15.37:0/1439089613 conn(0x7f5d8c072bc0 msgr2=0x7f5d8c072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:36:33.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:33.033+0000 7f5d93600700 1 -- 172.21.15.37:0/1439089613 shutdown_connections 2024-03-29T08:36:33.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:33.033+0000 7f5d93600700 1 -- 172.21.15.37:0/1439089613 wait complete. 2024-03-29T08:36:33.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:33.033+0000 7f5d93600700 1 Processor -- start 2024-03-29T08:36:33.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:33.034+0000 7f5d93600700 1 -- start start 2024-03-29T08:36:33.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:33.034+0000 7f5d93600700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5d8c0fde90 0x7f5d8c115d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:36:33.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:33.034+0000 7f5d93600700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5d8c0fe830 0x7f5d8c116250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:36:33.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:33.034+0000 7f5d93600700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d8c1168a0 con 0x7f5d8c0fde90 2024-03-29T08:36:33.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:33.034+0000 7f5d93600700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d8c10fc70 con 0x7f5d8c0fe830 2024-03-29T08:36:33.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:33.034+0000 7f5d90c00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5d8c0fde90 0x7f5d8c115d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:36:33.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:33.034+0000 7f5d90c00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5d8c0fde90 0x7f5d8c115d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34168/0 (socket says 172.21.15.37:34168) 2024-03-29T08:36:33.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:33.034+0000 7f5d90c00700 1 -- 172.21.15.37:0/1976466797 learned_addr learned my addr 172.21.15.37:0/1976466797 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:36:33.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:33.035+0000 7f5d8be00700 1 --2- 172.21.15.37:0/1976466797 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5d8c0fe830 0x7f5d8c116250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:36:33.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:33.035+0000 7f5d90c00700 1 -- 172.21.15.37:0/1976466797 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5d8c0fe830 msgr2=0x7f5d8c116250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:36:33.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:33.035+0000 7f5d90c00700 1 --2- 172.21.15.37:0/1976466797 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5d8c0fe830 0x7f5d8c116250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:36:33.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:33.035+0000 7f5d90c00700 1 -- 172.21.15.37:0/1976466797 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d78005040 con 0x7f5d8c0fde90 2024-03-29T08:36:33.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:33.035+0000 7f5d8be00700 1 --2- 172.21.15.37:0/1976466797 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5d8c0fe830 0x7f5d8c116250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:36:33.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:33.035+0000 7f5d90c00700 1 --2- 172.21.15.37:0/1976466797 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5d8c0fde90 0x7f5d8c115d10 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f5d80002a50 tx=0x7f5d800046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:36:33.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:33.035+0000 7f5d88c00700 1 -- 172.21.15.37:0/1976466797 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5d80007cc0 con 0x7f5d8c0fde90 2024-03-29T08:36:33.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:33.035+0000 7f5d88c00700 1 -- 172.21.15.37:0/1976466797 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f5d8000f040 con 0x7f5d8c0fde90 2024-03-29T08:36:33.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:33.035+0000 7f5d93600700 1 -- 172.21.15.37:0/1976466797 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d8c10ff50 con 0x7f5d8c0fde90 2024-03-29T08:36:33.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:33.035+0000 7f5d88c00700 1 -- 172.21.15.37:0/1976466797 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5d80009540 con 0x7f5d8c0fde90 2024-03-29T08:36:33.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:33.036+0000 7f5d93600700 1 -- 172.21.15.37:0/1976466797 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d8c110490 con 0x7f5d8c0fde90 2024-03-29T08:36:33.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:33.036+0000 7f5d93600700 1 -- 172.21.15.37:0/1976466797 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d8c106a20 con 0x7f5d8c0fde90 2024-03-29T08:36:33.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:33.038+0000 7f5d88c00700 1 -- 172.21.15.37:0/1976466797 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f5d800096a0 con 0x7f5d8c0fde90 2024-03-29T08:36:33.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:33.038+0000 7f5d88c00700 1 --2- 172.21.15.37:0/1976466797 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f5d7406cc30 0x7f5d7406f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:36:33.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:33.039+0000 7f5d88c00700 1 -- 172.21.15.37:0/1976466797 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f5d8008bee0 con 0x7f5d8c0fde90 2024-03-29T08:36:33.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:33.039+0000 7f5d8be00700 1 --2- 172.21.15.37:0/1976466797 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f5d7406cc30 0x7f5d7406f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:36:33.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:33.039+0000 7f5d8be00700 1 --2- 172.21.15.37:0/1976466797 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f5d7406cc30 0x7f5d7406f0e0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f5d8c1115b0 tx=0x7f5d78009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:36:33.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:33.044+0000 7f5d88c00700 1 -- 172.21.15.37:0/1976466797 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f5d800598b0 con 0x7f5d8c0fde90 2024-03-29T08:36:33.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:33.279+0000 7f5d93600700 1 -- 172.21.15.37:0/1976466797 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5d8c109e40 con 0x7f5d7406cc30 2024-03-29T08:36:33.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:33.280+0000 7f5d88c00700 1 -- 172.21.15.37:0/1976466797 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f5d8c109e40 con 0x7f5d7406cc30 2024-03-29T08:36:33.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:33.282+0000 7f5d93600700 1 -- 172.21.15.37:0/1976466797 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f5d7406cc30 msgr2=0x7f5d7406f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:36:33.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:33.282+0000 7f5d93600700 1 --2- 172.21.15.37:0/1976466797 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f5d7406cc30 0x7f5d7406f0e0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f5d8c1115b0 tx=0x7f5d78009e30 comp rx=0 tx=0).stop 2024-03-29T08:36:33.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:33.282+0000 7f5d93600700 1 -- 172.21.15.37:0/1976466797 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5d8c0fde90 msgr2=0x7f5d8c115d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:36:33.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:33.282+0000 7f5d93600700 1 --2- 172.21.15.37:0/1976466797 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5d8c0fde90 0x7f5d8c115d10 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f5d80002a50 tx=0x7f5d800046b0 comp rx=0 tx=0).stop 2024-03-29T08:36:33.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:33.282+0000 7f5d93600700 1 -- 172.21.15.37:0/1976466797 shutdown_connections 2024-03-29T08:36:33.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:33.282+0000 7f5d93600700 1 --2- 172.21.15.37:0/1976466797 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f5d7406cc30 0x7f5d7406f0e0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:36:33.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:33.282+0000 7f5d93600700 1 --2- 172.21.15.37:0/1976466797 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5d8c0fde90 0x7f5d8c115d10 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:36:33.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:33.282+0000 7f5d93600700 1 --2- 172.21.15.37:0/1976466797 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5d8c0fe830 0x7f5d8c116250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:36:33.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:33.282+0000 7f5d93600700 1 -- 172.21.15.37:0/1976466797 >> 172.21.15.37:0/1976466797 conn(0x7f5d8c072bc0 msgr2=0x7f5d8c0fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:36:33.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:33.283+0000 7f5d93600700 1 -- 172.21.15.37:0/1976466797 shutdown_connections 2024-03-29T08:36:33.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:33.283+0000 7f5d93600700 1 -- 172.21.15.37:0/1976466797 wait complete. 2024-03-29T08:36:33.298 INFO:teuthology.orchestra.run.smithi037.stdout:true 2024-03-29T08:36:33.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:33.422+0000 7f1f3a600700 1 -- 172.21.15.37:0/3722687128 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1f341037a0 msgr2=0x7f1f340fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:36:33.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:33.422+0000 7f1f3a600700 1 --2- 172.21.15.37:0/3722687128 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1f341037a0 0x7f1f340fce10 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f1f20004940 tx=0x7f1f20009e30 comp rx=0 tx=0).stop 2024-03-29T08:36:33.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:33.422+0000 7f1f3a600700 1 -- 172.21.15.37:0/3722687128 shutdown_connections 2024-03-29T08:36:33.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:33.422+0000 7f1f3a600700 1 --2- 172.21.15.37:0/3722687128 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1f341037a0 0x7f1f340fce10 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:36:33.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:33.422+0000 7f1f3a600700 1 --2- 172.21.15.37:0/3722687128 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1f34102e00 0x7f1f341031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:36:33.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:33.422+0000 7f1f3a600700 1 -- 172.21.15.37:0/3722687128 >> 172.21.15.37:0/3722687128 conn(0x7f1f340f89b0 msgr2=0x7f1f340fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:36:33.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:33.422+0000 7f1f3a600700 1 -- 172.21.15.37:0/3722687128 shutdown_connections 2024-03-29T08:36:33.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:33.422+0000 7f1f3a600700 1 -- 172.21.15.37:0/3722687128 wait complete. 2024-03-29T08:36:33.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:33.423+0000 7f1f3a600700 1 Processor -- start 2024-03-29T08:36:33.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:33.423+0000 7f1f3a600700 1 -- start start 2024-03-29T08:36:33.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:33.424+0000 7f1f3a600700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1f34102e00 0x7f1f34113b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:36:33.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:33.424+0000 7f1f3a600700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1f341037a0 0x7f1f34114040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:36:33.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:33.424+0000 7f1f3a600700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f34114690 con 0x7f1f341037a0 2024-03-29T08:36:33.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:33.424+0000 7f1f3a600700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f3410da80 con 0x7f1f34102e00 2024-03-29T08:36:33.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:33.424+0000 7f1f32a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1f341037a0 0x7f1f34114040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:36:33.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:33.424+0000 7f1f32a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1f341037a0 0x7f1f34114040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34192/0 (socket says 172.21.15.37:34192) 2024-03-29T08:36:33.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:33.424+0000 7f1f32a00700 1 -- 172.21.15.37:0/3637961312 learned_addr learned my addr 172.21.15.37:0/3637961312 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:36:33.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:33.425+0000 7f1f33400700 1 --2- 172.21.15.37:0/3637961312 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1f34102e00 0x7f1f34113b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:36:33.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:33.425+0000 7f1f32a00700 1 -- 172.21.15.37:0/3637961312 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1f34102e00 msgr2=0x7f1f34113b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:36:33.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:33.425+0000 7f1f32a00700 1 --2- 172.21.15.37:0/3637961312 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1f34102e00 0x7f1f34113b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:36:33.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:33.425+0000 7f1f32a00700 1 -- 172.21.15.37:0/3637961312 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f20005040 con 0x7f1f341037a0 2024-03-29T08:36:33.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:33.425+0000 7f1f33400700 1 --2- 172.21.15.37:0/3637961312 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1f34102e00 0x7f1f34113b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:36:33.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:33.425+0000 7f1f32a00700 1 --2- 172.21.15.37:0/3637961312 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1f341037a0 0x7f1f34114040 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f1f20009390 tx=0x7f1f20009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:36:33.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:33.425+0000 7f1f2b400700 1 -- 172.21.15.37:0/3637961312 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1f2000e040 con 0x7f1f341037a0 2024-03-29T08:36:33.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:33.425+0000 7f1f2b400700 1 -- 172.21.15.37:0/3637961312 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f1f20013420 con 0x7f1f341037a0 2024-03-29T08:36:33.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:33.425+0000 7f1f3a600700 1 -- 172.21.15.37:0/3637961312 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f3410dd00 con 0x7f1f341037a0 2024-03-29T08:36:33.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:33.425+0000 7f1f2b400700 1 -- 172.21.15.37:0/3637961312 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1f2001c630 con 0x7f1f341037a0 2024-03-29T08:36:33.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:33.425+0000 7f1f3a600700 1 -- 172.21.15.37:0/3637961312 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f3410e2c0 con 0x7f1f341037a0 2024-03-29T08:36:33.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:33.426+0000 7f1f3a600700 1 -- 172.21.15.37:0/3637961312 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f340fe550 con 0x7f1f341037a0 2024-03-29T08:36:33.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:33.427+0000 7f1f2b400700 1 -- 172.21.15.37:0/3637961312 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f1f200139e0 con 0x7f1f341037a0 2024-03-29T08:36:33.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:33.428+0000 7f1f2b400700 1 --2- 172.21.15.37:0/3637961312 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f1f1806cc30 0x7f1f1806f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:36:33.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:33.428+0000 7f1f2b400700 1 -- 172.21.15.37:0/3637961312 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f1f2008d2a0 con 0x7f1f341037a0 2024-03-29T08:36:33.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:33.429+0000 7f1f33400700 1 --2- 172.21.15.37:0/3637961312 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f1f1806cc30 0x7f1f1806f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:36:33.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:33.429+0000 7f1f33400700 1 --2- 172.21.15.37:0/3637961312 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f1f1806cc30 0x7f1f1806f0e0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f1f24007fd0 tx=0x7f1f24009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:36:33.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:33.433+0000 7f1f2b400700 1 -- 172.21.15.37:0/3637961312 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f1f2005ad70 con 0x7f1f341037a0 2024-03-29T08:36:33.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:33.666+0000 7f1f3a600700 1 -- 172.21.15.37:0/3637961312 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f1f34109da0 con 0x7f1f1806cc30 2024-03-29T08:36:33.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:33.668+0000 7f1f2b400700 1 -- 172.21.15.37:0/3637961312 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f1f34109da0 con 0x7f1f1806cc30 2024-03-29T08:36:33.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:33.670+0000 7f1f3a600700 1 -- 172.21.15.37:0/3637961312 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f1f1806cc30 msgr2=0x7f1f1806f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:36:33.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:33.670+0000 7f1f3a600700 1 --2- 172.21.15.37:0/3637961312 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f1f1806cc30 0x7f1f1806f0e0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f1f24007fd0 tx=0x7f1f24009040 comp rx=0 tx=0).stop 2024-03-29T08:36:33.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:33.670+0000 7f1f3a600700 1 -- 172.21.15.37:0/3637961312 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1f341037a0 msgr2=0x7f1f34114040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:36:33.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:33.670+0000 7f1f3a600700 1 --2- 172.21.15.37:0/3637961312 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1f341037a0 0x7f1f34114040 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f1f20009390 tx=0x7f1f20009e30 comp rx=0 tx=0).stop 2024-03-29T08:36:33.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:33.670+0000 7f1f3a600700 1 -- 172.21.15.37:0/3637961312 shutdown_connections 2024-03-29T08:36:33.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:33.670+0000 7f1f3a600700 1 --2- 172.21.15.37:0/3637961312 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f1f1806cc30 0x7f1f1806f0e0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:36:33.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:33.670+0000 7f1f3a600700 1 --2- 172.21.15.37:0/3637961312 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1f34102e00 0x7f1f34113b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:36:33.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:33.670+0000 7f1f3a600700 1 --2- 172.21.15.37:0/3637961312 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1f341037a0 0x7f1f34114040 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:36:33.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:33.670+0000 7f1f3a600700 1 -- 172.21.15.37:0/3637961312 >> 172.21.15.37:0/3637961312 conn(0x7f1f340f89b0 msgr2=0x7f1f340f9ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:36:33.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:33.670+0000 7f1f3a600700 1 -- 172.21.15.37:0/3637961312 shutdown_connections 2024-03-29T08:36:33.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:33.670+0000 7f1f3a600700 1 -- 172.21.15.37:0/3637961312 wait complete. 2024-03-29T08:36:33.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:33.809+0000 7f5321c00700 1 -- 172.21.15.37:0/3487037192 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f531c102e00 msgr2=0x7f531c1031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:36:33.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:33.809+0000 7f5321c00700 1 --2- 172.21.15.37:0/3487037192 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f531c102e00 0x7f531c1031d0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f530c004940 tx=0x7f530c009e30 comp rx=0 tx=0).stop 2024-03-29T08:36:33.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:33.809+0000 7f5321c00700 1 -- 172.21.15.37:0/3487037192 shutdown_connections 2024-03-29T08:36:33.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:33.809+0000 7f5321c00700 1 --2- 172.21.15.37:0/3487037192 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f531c1037a0 0x7f531c0fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:36:33.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:33.809+0000 7f5321c00700 1 --2- 172.21.15.37:0/3487037192 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f531c102e00 0x7f531c1031d0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:36:33.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:33.809+0000 7f5321c00700 1 -- 172.21.15.37:0/3487037192 >> 172.21.15.37:0/3487037192 conn(0x7f531c0f89b0 msgr2=0x7f531c0fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:36:33.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:33.809+0000 7f5321c00700 1 -- 172.21.15.37:0/3487037192 shutdown_connections 2024-03-29T08:36:33.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:33.809+0000 7f5321c00700 1 -- 172.21.15.37:0/3487037192 wait complete. 2024-03-29T08:36:33.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:33.810+0000 7f5321c00700 1 Processor -- start 2024-03-29T08:36:33.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:33.810+0000 7f5321c00700 1 -- start start 2024-03-29T08:36:33.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:33.810+0000 7f5321c00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f531c102e00 0x7f531c0702c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:36:33.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:33.810+0000 7f5321c00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f531c1037a0 0x7f531c070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:36:33.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:33.810+0000 7f5321c00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f531c070d90 con 0x7f531c102e00 2024-03-29T08:36:33.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:33.810+0000 7f5321c00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f531c070ed0 con 0x7f531c1037a0 2024-03-29T08:36:33.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:33.811+0000 7f531aa00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f531c102e00 0x7f531c0702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:36:33.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:33.811+0000 7f531aa00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f531c102e00 0x7f531c0702c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34196/0 (socket says 172.21.15.37:34196) 2024-03-29T08:36:33.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:33.811+0000 7f531aa00700 1 -- 172.21.15.37:0/1980798896 learned_addr learned my addr 172.21.15.37:0/1980798896 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:36:33.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:33.811+0000 7f531a000700 1 --2- 172.21.15.37:0/1980798896 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f531c1037a0 0x7f531c070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:36:33.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:33.811+0000 7f531aa00700 1 -- 172.21.15.37:0/1980798896 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f531c1037a0 msgr2=0x7f531c070800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:36:33.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:33.811+0000 7f531aa00700 1 --2- 172.21.15.37:0/1980798896 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f531c1037a0 0x7f531c070800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:36:33.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:33.811+0000 7f531aa00700 1 -- 172.21.15.37:0/1980798896 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f530c005040 con 0x7f531c102e00 2024-03-29T08:36:33.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:33.811+0000 7f531a000700 1 --2- 172.21.15.37:0/1980798896 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f531c1037a0 0x7f531c070800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:36:33.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:33.811+0000 7f531aa00700 1 --2- 172.21.15.37:0/1980798896 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f531c102e00 0x7f531c0702c0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f530c009360 tx=0x7f530c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:36:33.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:33.812+0000 7f5312a00700 1 -- 172.21.15.37:0/1980798896 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f530c00e040 con 0x7f531c102e00 2024-03-29T08:36:33.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:33.812+0000 7f5312a00700 1 -- 172.21.15.37:0/1980798896 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f530c013420 con 0x7f531c102e00 2024-03-29T08:36:33.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:33.812+0000 7f5312a00700 1 -- 172.21.15.37:0/1980798896 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f530c01c5f0 con 0x7f531c102e00 2024-03-29T08:36:33.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:33.812+0000 7f5321c00700 1 -- 172.21.15.37:0/1980798896 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f531c071120 con 0x7f531c102e00 2024-03-29T08:36:33.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:33.812+0000 7f5321c00700 1 -- 172.21.15.37:0/1980798896 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f531c11a250 con 0x7f531c102e00 2024-03-29T08:36:33.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:33.813+0000 7f5321c00700 1 -- 172.21.15.37:0/1980798896 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f531c0fe550 con 0x7f531c102e00 2024-03-29T08:36:33.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:33.814+0000 7f5312a00700 1 -- 172.21.15.37:0/1980798896 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f530c013590 con 0x7f531c102e00 2024-03-29T08:36:33.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:33.815+0000 7f5312a00700 1 --2- 172.21.15.37:0/1980798896 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f530006cc30 0x7f530006f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:36:33.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:33.815+0000 7f5312a00700 1 -- 172.21.15.37:0/1980798896 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f530c08d270 con 0x7f531c102e00 2024-03-29T08:36:33.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:33.816+0000 7f531a000700 1 --2- 172.21.15.37:0/1980798896 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f530006cc30 0x7f530006f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:36:33.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:33.816+0000 7f531a000700 1 --2- 172.21.15.37:0/1980798896 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f530006cc30 0x7f530006f0e0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f5308009fd0 tx=0x7f530800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:36:33.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:33.820+0000 7f5312a00700 1 -- 172.21.15.37:0/1980798896 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f530c0572b0 con 0x7f531c102e00 2024-03-29T08:36:34.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:34.045+0000 7f5321c00700 1 -- 172.21.15.37:0/1980798896 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f531c0719b0 con 0x7f530006cc30 2024-03-29T08:36:34.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:34.055+0000 7f5312a00700 1 -- 172.21.15.37:0/1980798896 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f531c0719b0 con 0x7f530006cc30 2024-03-29T08:36:34.055 INFO:teuthology.orchestra.run.smithi037.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-29T08:36:34.055 INFO:teuthology.orchestra.run.smithi037.stdout:alertmanager.smithi037 smithi037 *:9093,9094 running (26m) 3m ago 39m 14.3M - 0.25.0 c8568f914cd2 2f9f9746ed0f 2024-03-29T08:36:34.055 INFO:teuthology.orchestra.run.smithi037.stdout:crash.smithi037 smithi037 running (23m) 3m ago 39m 6647k - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 178d416a3186 2024-03-29T08:36:34.055 INFO:teuthology.orchestra.run.smithi037.stdout:crash.smithi148 smithi148 running (23m) 3m ago 38m 6656k - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 23666989d58f 2024-03-29T08:36:34.056 INFO:teuthology.orchestra.run.smithi037.stdout:grafana.smithi037 smithi037 *:3000 running (26m) 3m ago 39m 74.5M - 9.4.7 954c08fa6188 6b976fe49952 2024-03-29T08:36:34.056 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi037.iifsmh smithi037 running (18m) 3m ago 32m 16.4M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 2d97f562cc5b 2024-03-29T08:36:34.056 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi037.mmwgxe smithi037 running (19m) 3m ago 33m 16.7M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff ec37037f46e8 2024-03-29T08:36:34.056 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi148.qukpjh smithi148 running (3m) 3m ago 32m 10.7M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 3257219bc046 2024-03-29T08:36:34.056 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi148.sqteih smithi148 running (18m) 3m ago 32m 16.3M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 44c15c77037a 2024-03-29T08:36:34.056 INFO:teuthology.orchestra.run.smithi037.stdout:mgr.smithi037.wjikii smithi037 *:8443,9283,8765 running (28m) 3m ago 40m 510M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff c5a1417b9417 2024-03-29T08:36:34.056 INFO:teuthology.orchestra.run.smithi037.stdout:mgr.smithi148.onhgvm smithi148 *:8443,9283,8765 running (27m) 3m ago 38m 456M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff fcfdbce062bd 2024-03-29T08:36:34.056 INFO:teuthology.orchestra.run.smithi037.stdout:mon.smithi037 smithi037 running (24m) 3m ago 41m 70.2M 2048M 18.2.2-1015-gd0e392d9 1cc2f45c17ff eea2060783af 2024-03-29T08:36:34.056 INFO:teuthology.orchestra.run.smithi037.stdout:mon.smithi148 smithi148 running (24m) 3m ago 38m 50.4M 2048M 18.2.2-1015-gd0e392d9 1cc2f45c17ff ad4b2c5748be 2024-03-29T08:36:34.056 INFO:teuthology.orchestra.run.smithi037.stdout:node-exporter.smithi037 smithi037 *:9100 running (27m) 3m ago 39m 8514k - 1.5.0 0da6a335fe13 26b8ae9cb878 2024-03-29T08:36:34.056 INFO:teuthology.orchestra.run.smithi037.stdout:node-exporter.smithi148 smithi148 *:9100 running (27m) 3m ago 38m 8464k - 1.5.0 0da6a335fe13 98414e4e64ed 2024-03-29T08:36:34.056 INFO:teuthology.orchestra.run.smithi037.stdout:osd.0 smithi037 running (22m) 3m ago 37m 108M 1619M 18.2.2-1015-gd0e392d9 1cc2f45c17ff c0c584f6245c 2024-03-29T08:36:34.056 INFO:teuthology.orchestra.run.smithi037.stdout:osd.1 smithi037 running (22m) 3m ago 36m 104M 1619M 18.2.2-1015-gd0e392d9 1cc2f45c17ff a4a2ff4cc83d 2024-03-29T08:36:34.056 INFO:teuthology.orchestra.run.smithi037.stdout:osd.2 smithi037 running (21m) 3m ago 36m 85.9M 1619M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 8b1bc71aa79e 2024-03-29T08:36:34.056 INFO:teuthology.orchestra.run.smithi037.stdout:osd.3 smithi148 running (20m) 3m ago 35m 109M 2642M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 1c656ab8ff86 2024-03-29T08:36:34.057 INFO:teuthology.orchestra.run.smithi037.stdout:osd.4 smithi148 running (20m) 3m ago 35m 101M 2642M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 966a9de11d22 2024-03-29T08:36:34.057 INFO:teuthology.orchestra.run.smithi037.stdout:osd.5 smithi148 running (19m) 3m ago 34m 89.4M 2642M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 474c4f9212ce 2024-03-29T08:36:34.057 INFO:teuthology.orchestra.run.smithi037.stdout:prometheus.smithi037 smithi037 *:9095 running (26m) 3m ago 38m 61.0M - 2.43.0 a07b618ecd1d 30c0223ec589 2024-03-29T08:36:34.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:34.058+0000 7f5321c00700 1 -- 172.21.15.37:0/1980798896 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f530006cc30 msgr2=0x7f530006f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:36:34.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:34.058+0000 7f5321c00700 1 --2- 172.21.15.37:0/1980798896 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f530006cc30 0x7f530006f0e0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f5308009fd0 tx=0x7f530800b040 comp rx=0 tx=0).stop 2024-03-29T08:36:34.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:34.058+0000 7f5321c00700 1 -- 172.21.15.37:0/1980798896 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f531c102e00 msgr2=0x7f531c0702c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:36:34.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:34.058+0000 7f5321c00700 1 --2- 172.21.15.37:0/1980798896 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f531c102e00 0x7f531c0702c0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f530c009360 tx=0x7f530c009e30 comp rx=0 tx=0).stop 2024-03-29T08:36:34.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:34.058+0000 7f5321c00700 1 -- 172.21.15.37:0/1980798896 shutdown_connections 2024-03-29T08:36:34.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:34.058+0000 7f5321c00700 1 --2- 172.21.15.37:0/1980798896 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f530006cc30 0x7f530006f0e0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:36:34.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:34.058+0000 7f5321c00700 1 --2- 172.21.15.37:0/1980798896 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f531c102e00 0x7f531c0702c0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:36:34.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:34.058+0000 7f5321c00700 1 --2- 172.21.15.37:0/1980798896 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f531c1037a0 0x7f531c070800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:36:34.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:34.058+0000 7f5321c00700 1 -- 172.21.15.37:0/1980798896 >> 172.21.15.37:0/1980798896 conn(0x7f531c0f89b0 msgr2=0x7f531c0f9a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:36:34.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:34.058+0000 7f5321c00700 1 -- 172.21.15.37:0/1980798896 shutdown_connections 2024-03-29T08:36:34.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:34.058+0000 7f5321c00700 1 -- 172.21.15.37:0/1980798896 wait complete. 2024-03-29T08:36:34.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:34.195+0000 7f1da9e00700 1 -- 172.21.15.37:0/410898033 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1da41037a0 msgr2=0x7f1da40fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:36:34.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:34.195+0000 7f1da9e00700 1 --2- 172.21.15.37:0/410898033 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1da41037a0 0x7f1da40fce10 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f1d90004940 tx=0x7f1d90009e30 comp rx=0 tx=0).stop 2024-03-29T08:36:34.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:34.195+0000 7f1da9e00700 1 -- 172.21.15.37:0/410898033 shutdown_connections 2024-03-29T08:36:34.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:34.195+0000 7f1da9e00700 1 --2- 172.21.15.37:0/410898033 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1da41037a0 0x7f1da40fce10 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:36:34.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:34.195+0000 7f1da9e00700 1 --2- 172.21.15.37:0/410898033 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1da4102e00 0x7f1da41031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:36:34.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:34.195+0000 7f1da9e00700 1 -- 172.21.15.37:0/410898033 >> 172.21.15.37:0/410898033 conn(0x7f1da40f89b0 msgr2=0x7f1da40fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:36:34.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:34.195+0000 7f1da9e00700 1 -- 172.21.15.37:0/410898033 shutdown_connections 2024-03-29T08:36:34.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:34.195+0000 7f1da9e00700 1 -- 172.21.15.37:0/410898033 wait complete. 2024-03-29T08:36:34.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:34.196+0000 7f1da9e00700 1 Processor -- start 2024-03-29T08:36:34.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:34.196+0000 7f1da9e00700 1 -- start start 2024-03-29T08:36:34.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:34.196+0000 7f1da9e00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1da4102e00 0x7f1da4115cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:36:34.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:34.196+0000 7f1da9e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1da41037a0 0x7f1da4116230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:36:34.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:34.196+0000 7f1da9e00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1da4116880 con 0x7f1da41037a0 2024-03-29T08:36:34.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:34.196+0000 7f1da9e00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1da410fc50 con 0x7f1da4102e00 2024-03-29T08:36:34.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:34.197+0000 7f1da2000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1da41037a0 0x7f1da4116230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:36:34.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:34.197+0000 7f1da2000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1da41037a0 0x7f1da4116230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34216/0 (socket says 172.21.15.37:34216) 2024-03-29T08:36:34.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:34.197+0000 7f1da2000700 1 -- 172.21.15.37:0/1843969218 learned_addr learned my addr 172.21.15.37:0/1843969218 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:36:34.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:34.197+0000 7f1da2a00700 1 --2- 172.21.15.37:0/1843969218 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1da4102e00 0x7f1da4115cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:36:34.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:34.197+0000 7f1da2000700 1 -- 172.21.15.37:0/1843969218 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1da4102e00 msgr2=0x7f1da4115cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:36:34.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:34.197+0000 7f1da2000700 1 --2- 172.21.15.37:0/1843969218 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1da4102e00 0x7f1da4115cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:36:34.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:34.197+0000 7f1da2000700 1 -- 172.21.15.37:0/1843969218 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d90005040 con 0x7f1da41037a0 2024-03-29T08:36:34.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:34.197+0000 7f1da2a00700 1 --2- 172.21.15.37:0/1843969218 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1da4102e00 0x7f1da4115cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:36:34.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:34.197+0000 7f1da2000700 1 --2- 172.21.15.37:0/1843969218 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1da41037a0 0x7f1da4116230 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f1d90009390 tx=0x7f1d90009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:36:34.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:34.198+0000 7f1d9aa00700 1 -- 172.21.15.37:0/1843969218 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1d9000e040 con 0x7f1da41037a0 2024-03-29T08:36:34.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:34.198+0000 7f1d9aa00700 1 -- 172.21.15.37:0/1843969218 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f1d90013420 con 0x7f1da41037a0 2024-03-29T08:36:34.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:34.198+0000 7f1da9e00700 1 -- 172.21.15.37:0/1843969218 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1da410fed0 con 0x7f1da41037a0 2024-03-29T08:36:34.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:34.198+0000 7f1d9aa00700 1 -- 172.21.15.37:0/1843969218 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1d9001c630 con 0x7f1da41037a0 2024-03-29T08:36:34.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:34.198+0000 7f1da9e00700 1 -- 172.21.15.37:0/1843969218 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1da4110490 con 0x7f1da41037a0 2024-03-29T08:36:34.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:34.199+0000 7f1da9e00700 1 -- 172.21.15.37:0/1843969218 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1da40fe550 con 0x7f1da41037a0 2024-03-29T08:36:34.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:34.200+0000 7f1d9aa00700 1 -- 172.21.15.37:0/1843969218 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f1d90013a00 con 0x7f1da41037a0 2024-03-29T08:36:34.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:34.201+0000 7f1d9aa00700 1 --2- 172.21.15.37:0/1843969218 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f1d8806cc30 0x7f1d8806f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:36:34.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:34.201+0000 7f1d9aa00700 1 -- 172.21.15.37:0/1843969218 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f1d9008d2c0 con 0x7f1da41037a0 2024-03-29T08:36:34.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:34.201+0000 7f1da2a00700 1 --2- 172.21.15.37:0/1843969218 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f1d8806cc30 0x7f1d8806f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:36:34.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:34.202+0000 7f1da2a00700 1 --2- 172.21.15.37:0/1843969218 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f1d8806cc30 0x7f1d8806f0e0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f1d94009fd0 tx=0x7f1d94009ee0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:36:34.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:34.206+0000 7f1d9aa00700 1 -- 172.21.15.37:0/1843969218 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f1d9005ad90 con 0x7f1da41037a0 2024-03-29T08:36:34.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:34.503+0000 7f1da9e00700 1 -- 172.21.15.37:0/1843969218 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f1da404bed0 con 0x7f1da41037a0 2024-03-29T08:36:34.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:34.503+0000 7f1d9aa00700 1 -- 172.21.15.37:0/1843969218 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7f1d9005a920 con 0x7f1da41037a0 2024-03-29T08:36:34.503 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-29T08:36:34.503 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": { 2024-03-29T08:36:34.503 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 2 2024-03-29T08:36:34.503 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:36:34.504 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": { 2024-03-29T08:36:34.504 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 2 2024-03-29T08:36:34.504 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:36:34.504 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": { 2024-03-29T08:36:34.504 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 6 2024-03-29T08:36:34.504 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:36:34.504 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": { 2024-03-29T08:36:34.504 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 4 2024-03-29T08:36:34.504 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:36:34.504 INFO:teuthology.orchestra.run.smithi037.stdout: "overall": { 2024-03-29T08:36:34.504 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 14 2024-03-29T08:36:34.504 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-29T08:36:34.504 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-29T08:36:34.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:34.506+0000 7f1da9e00700 1 -- 172.21.15.37:0/1843969218 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f1d8806cc30 msgr2=0x7f1d8806f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:36:34.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:34.506+0000 7f1da9e00700 1 --2- 172.21.15.37:0/1843969218 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f1d8806cc30 0x7f1d8806f0e0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f1d94009fd0 tx=0x7f1d94009ee0 comp rx=0 tx=0).stop 2024-03-29T08:36:34.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:34.506+0000 7f1da9e00700 1 -- 172.21.15.37:0/1843969218 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1da41037a0 msgr2=0x7f1da4116230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:36:34.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:34.506+0000 7f1da9e00700 1 --2- 172.21.15.37:0/1843969218 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1da41037a0 0x7f1da4116230 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f1d90009390 tx=0x7f1d90009e30 comp rx=0 tx=0).stop 2024-03-29T08:36:34.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:34.506+0000 7f1da9e00700 1 -- 172.21.15.37:0/1843969218 shutdown_connections 2024-03-29T08:36:34.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:34.506+0000 7f1da9e00700 1 --2- 172.21.15.37:0/1843969218 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f1d8806cc30 0x7f1d8806f0e0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:36:34.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:34.506+0000 7f1da9e00700 1 --2- 172.21.15.37:0/1843969218 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1da4102e00 0x7f1da4115cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:36:34.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:34.506+0000 7f1da9e00700 1 --2- 172.21.15.37:0/1843969218 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1da41037a0 0x7f1da4116230 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:36:34.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:34.506+0000 7f1da9e00700 1 -- 172.21.15.37:0/1843969218 >> 172.21.15.37:0/1843969218 conn(0x7f1da40f89b0 msgr2=0x7f1da40f9a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:36:34.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:34.506+0000 7f1da9e00700 1 -- 172.21.15.37:0/1843969218 shutdown_connections 2024-03-29T08:36:34.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:34.506+0000 7f1da9e00700 1 -- 172.21.15.37:0/1843969218 wait complete. 2024-03-29T08:36:34.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:34.644+0000 7fb8c1400700 1 -- 172.21.15.37:0/2800461397 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb8bc1037a0 msgr2=0x7fb8bc0fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:36:34.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:34.644+0000 7fb8c1400700 1 --2- 172.21.15.37:0/2800461397 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb8bc1037a0 0x7fb8bc0fce10 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fb8a8004940 tx=0x7fb8a8009e30 comp rx=0 tx=0).stop 2024-03-29T08:36:34.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:34.644+0000 7fb8c1400700 1 -- 172.21.15.37:0/2800461397 shutdown_connections 2024-03-29T08:36:34.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:34.644+0000 7fb8c1400700 1 --2- 172.21.15.37:0/2800461397 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb8bc1037a0 0x7fb8bc0fce10 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:36:34.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:34.644+0000 7fb8c1400700 1 --2- 172.21.15.37:0/2800461397 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb8bc102e00 0x7fb8bc1031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:36:34.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:34.644+0000 7fb8c1400700 1 -- 172.21.15.37:0/2800461397 >> 172.21.15.37:0/2800461397 conn(0x7fb8bc0f89b0 msgr2=0x7fb8bc0fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:36:34.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:34.644+0000 7fb8c1400700 1 -- 172.21.15.37:0/2800461397 shutdown_connections 2024-03-29T08:36:34.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:34.644+0000 7fb8c1400700 1 -- 172.21.15.37:0/2800461397 wait complete. 2024-03-29T08:36:34.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:34.645+0000 7fb8c1400700 1 Processor -- start 2024-03-29T08:36:34.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:34.645+0000 7fb8c1400700 1 -- start start 2024-03-29T08:36:34.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:34.645+0000 7fb8c1400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb8bc102e00 0x7fb8bc115da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:36:34.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:34.645+0000 7fb8c1400700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb8bc1037a0 0x7fb8bc1162e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:36:34.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:34.645+0000 7fb8c1400700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8bc116930 con 0x7fb8bc102e00 2024-03-29T08:36:34.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:34.645+0000 7fb8c1400700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8bc10fd00 con 0x7fb8bc1037a0 2024-03-29T08:36:34.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:34.645+0000 7fb8ba800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb8bc102e00 0x7fb8bc115da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:36:34.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:34.646+0000 7fb8ba800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb8bc102e00 0x7fb8bc115da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34230/0 (socket says 172.21.15.37:34230) 2024-03-29T08:36:34.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:34.646+0000 7fb8ba800700 1 -- 172.21.15.37:0/4001361867 learned_addr learned my addr 172.21.15.37:0/4001361867 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:36:34.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:34.646+0000 7fb8b9e00700 1 --2- 172.21.15.37:0/4001361867 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb8bc1037a0 0x7fb8bc1162e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:36:34.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:34.646+0000 7fb8ba800700 1 -- 172.21.15.37:0/4001361867 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb8bc1037a0 msgr2=0x7fb8bc1162e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:36:34.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:34.646+0000 7fb8ba800700 1 --2- 172.21.15.37:0/4001361867 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb8bc1037a0 0x7fb8bc1162e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:36:34.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:34.646+0000 7fb8ba800700 1 -- 172.21.15.37:0/4001361867 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8a8005040 con 0x7fb8bc102e00 2024-03-29T08:36:34.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:34.646+0000 7fb8b9e00700 1 --2- 172.21.15.37:0/4001361867 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb8bc1037a0 0x7fb8bc1162e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:36:34.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:34.646+0000 7fb8ba800700 1 --2- 172.21.15.37:0/4001361867 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb8bc102e00 0x7fb8bc115da0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fb8ac00c8c0 tx=0x7fb8ac0044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:36:34.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:34.646+0000 7fb8b2a00700 1 -- 172.21.15.37:0/4001361867 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb8ac00a980 con 0x7fb8bc102e00 2024-03-29T08:36:34.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:34.646+0000 7fb8b2a00700 1 -- 172.21.15.37:0/4001361867 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fb8ac00ce60 con 0x7fb8bc102e00 2024-03-29T08:36:34.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:34.646+0000 7fb8c1400700 1 -- 172.21.15.37:0/4001361867 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8bc10ffe0 con 0x7fb8bc102e00 2024-03-29T08:36:34.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:34.646+0000 7fb8b2a00700 1 -- 172.21.15.37:0/4001361867 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb8ac0113a0 con 0x7fb8bc102e00 2024-03-29T08:36:34.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:34.647+0000 7fb8c1400700 1 -- 172.21.15.37:0/4001361867 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8bc110520 con 0x7fb8bc102e00 2024-03-29T08:36:34.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:34.648+0000 7fb8c1400700 1 -- 172.21.15.37:0/4001361867 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8bc0fe550 con 0x7fb8bc102e00 2024-03-29T08:36:34.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:34.649+0000 7fb8b2a00700 1 -- 172.21.15.37:0/4001361867 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fb8ac012400 con 0x7fb8bc102e00 2024-03-29T08:36:34.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:34.650+0000 7fb8b2a00700 1 --2- 172.21.15.37:0/4001361867 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fb8a006cc30 0x7fb8a006f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:36:34.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:34.650+0000 7fb8b2a00700 1 -- 172.21.15.37:0/4001361867 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fb8ac08c0f0 con 0x7fb8bc102e00 2024-03-29T08:36:34.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:34.650+0000 7fb8b9e00700 1 --2- 172.21.15.37:0/4001361867 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fb8a006cc30 0x7fb8a006f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:36:34.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:34.651+0000 7fb8b9e00700 1 --2- 172.21.15.37:0/4001361867 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fb8a006cc30 0x7fb8a006f0e0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fb8bc111640 tx=0x7fb8a8009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:36:34.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:34.655+0000 7fb8b2a00700 1 -- 172.21.15.37:0/4001361867 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fb8ac059bc0 con 0x7fb8bc102e00 2024-03-29T08:36:34.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:36:34 smithi148 ceph-mon[106692]: pgmap v771: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:36:34.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:36:34 smithi148 ceph-mon[106692]: from='client.35402 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:36:34.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:36:34 smithi148 ceph-mon[106692]: from='client.35406 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:36:34.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:34.912+0000 7fb8c1400700 1 -- 172.21.15.37:0/4001361867 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fb8bc04bed0 con 0x7fb8bc102e00 2024-03-29T08:36:34.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:34.912+0000 7fb8b2a00700 1 -- 172.21.15.37:0/4001361867 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 30 v30) v1 ==== 76+0+1841 (secure 0 0 0) 0x7fb8ac059750 con 0x7fb8bc102e00 2024-03-29T08:36:34.912 INFO:teuthology.orchestra.run.smithi037.stdout:e30 2024-03-29T08:36:34.912 INFO:teuthology.orchestra.run.smithi037.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-29T08:36:34.913 INFO:teuthology.orchestra.run.smithi037.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T08:36:34.913 INFO:teuthology.orchestra.run.smithi037.stdout:legacy client fscid: 1 2024-03-29T08:36:34.913 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:36:34.913 INFO:teuthology.orchestra.run.smithi037.stdout:Filesystem 'cephfs' (1) 2024-03-29T08:36:34.913 INFO:teuthology.orchestra.run.smithi037.stdout:fs_name cephfs 2024-03-29T08:36:34.913 INFO:teuthology.orchestra.run.smithi037.stdout:epoch 21 2024-03-29T08:36:34.913 INFO:teuthology.orchestra.run.smithi037.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-03-29T08:36:34.913 INFO:teuthology.orchestra.run.smithi037.stdout:created 2024-03-29T08:03:29.241269+0000 2024-03-29T08:36:34.913 INFO:teuthology.orchestra.run.smithi037.stdout:modified 2024-03-29T08:17:16.237238+0000 2024-03-29T08:36:34.913 INFO:teuthology.orchestra.run.smithi037.stdout:tableserver 0 2024-03-29T08:36:34.913 INFO:teuthology.orchestra.run.smithi037.stdout:root 0 2024-03-29T08:36:34.913 INFO:teuthology.orchestra.run.smithi037.stdout:session_timeout 60 2024-03-29T08:36:34.913 INFO:teuthology.orchestra.run.smithi037.stdout:session_autoclose 300 2024-03-29T08:36:34.914 INFO:teuthology.orchestra.run.smithi037.stdout:max_file_size 1099511627776 2024-03-29T08:36:34.914 INFO:teuthology.orchestra.run.smithi037.stdout:max_xattr_size 65536 2024-03-29T08:36:34.914 INFO:teuthology.orchestra.run.smithi037.stdout:required_client_features {} 2024-03-29T08:36:34.914 INFO:teuthology.orchestra.run.smithi037.stdout:last_failure 0 2024-03-29T08:36:34.914 INFO:teuthology.orchestra.run.smithi037.stdout:last_failure_osd_epoch 82 2024-03-29T08:36:34.914 INFO:teuthology.orchestra.run.smithi037.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T08:36:34.914 INFO:teuthology.orchestra.run.smithi037.stdout:max_mds 2 2024-03-29T08:36:34.914 INFO:teuthology.orchestra.run.smithi037.stdout:in 0,1 2024-03-29T08:36:34.914 INFO:teuthology.orchestra.run.smithi037.stdout:up {} 2024-03-29T08:36:34.914 INFO:teuthology.orchestra.run.smithi037.stdout:failed 0,1 2024-03-29T08:36:34.914 INFO:teuthology.orchestra.run.smithi037.stdout:damaged 2024-03-29T08:36:34.914 INFO:teuthology.orchestra.run.smithi037.stdout:stopped 2024-03-29T08:36:34.914 INFO:teuthology.orchestra.run.smithi037.stdout:data_pools [3] 2024-03-29T08:36:34.914 INFO:teuthology.orchestra.run.smithi037.stdout:metadata_pool 2 2024-03-29T08:36:34.914 INFO:teuthology.orchestra.run.smithi037.stdout:inline_data disabled 2024-03-29T08:36:34.915 INFO:teuthology.orchestra.run.smithi037.stdout:balancer 2024-03-29T08:36:34.915 INFO:teuthology.orchestra.run.smithi037.stdout:bal_rank_mask -1 2024-03-29T08:36:34.915 INFO:teuthology.orchestra.run.smithi037.stdout:standby_count_wanted 1 2024-03-29T08:36:34.915 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:36:34.915 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:36:34.915 INFO:teuthology.orchestra.run.smithi037.stdout:Standby daemons: 2024-03-29T08:36:34.915 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:36:34.915 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi037.mmwgxe{-1:34440} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.37:6828/1248142005,v1:172.21.15.37:6829/1248142005] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:36:34.915 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi037.iifsmh{-1:34444} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.37:6826/2060882959,v1:172.21.15.37:6827/2060882959] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:36:34.915 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi148.sqteih{-1:44395} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.148:6824/4078652242,v1:172.21.15.148:6825/4078652242] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:36:34.915 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi148.qukpjh{-1:45007} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.148:6826/2394751105,v1:172.21.15.148:6827/2394751105] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:36:34.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:34.915+0000 7fb8c1400700 1 -- 172.21.15.37:0/4001361867 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fb8a006cc30 msgr2=0x7fb8a006f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:36:34.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:34.915+0000 7fb8c1400700 1 --2- 172.21.15.37:0/4001361867 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fb8a006cc30 0x7fb8a006f0e0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fb8bc111640 tx=0x7fb8a8009e30 comp rx=0 tx=0).stop 2024-03-29T08:36:34.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:34.915+0000 7fb8c1400700 1 -- 172.21.15.37:0/4001361867 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb8bc102e00 msgr2=0x7fb8bc115da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:36:34.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:34.915+0000 7fb8c1400700 1 --2- 172.21.15.37:0/4001361867 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb8bc102e00 0x7fb8bc115da0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fb8ac00c8c0 tx=0x7fb8ac0044c0 comp rx=0 tx=0).stop 2024-03-29T08:36:34.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:34.915+0000 7fb8c1400700 1 -- 172.21.15.37:0/4001361867 shutdown_connections 2024-03-29T08:36:34.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:34.915+0000 7fb8c1400700 1 --2- 172.21.15.37:0/4001361867 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fb8a006cc30 0x7fb8a006f0e0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:36:34.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:34.915+0000 7fb8c1400700 1 --2- 172.21.15.37:0/4001361867 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb8bc102e00 0x7fb8bc115da0 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:36:34.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:34.915+0000 7fb8c1400700 1 --2- 172.21.15.37:0/4001361867 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb8bc1037a0 0x7fb8bc1162e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:36:34.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:34.915+0000 7fb8c1400700 1 -- 172.21.15.37:0/4001361867 >> 172.21.15.37:0/4001361867 conn(0x7fb8bc0f89b0 msgr2=0x7fb8bc0f9a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:36:34.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:34.915+0000 7fb8c1400700 1 -- 172.21.15.37:0/4001361867 shutdown_connections 2024-03-29T08:36:34.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:34.915+0000 7fb8c1400700 1 -- 172.21.15.37:0/4001361867 wait complete. 2024-03-29T08:36:34.917 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 30 2024-03-29T08:36:34.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:36:34 smithi037 ceph-mon[142325]: pgmap v771: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:36:34.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:36:34 smithi037 ceph-mon[142325]: from='client.35402 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:36:34.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:36:34 smithi037 ceph-mon[142325]: from='client.35406 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:36:35.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:35.053+0000 7fd828200700 1 -- 172.21.15.37:0/2621016495 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd8200708a0 msgr2=0x7fd820070d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:36:35.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:35.053+0000 7fd828200700 1 --2- 172.21.15.37:0/2621016495 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd8200708a0 0x7fd820070d10 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fd80c004970 tx=0x7fd80c009e30 comp rx=0 tx=0).stop 2024-03-29T08:36:35.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:35.053+0000 7fd828200700 1 -- 172.21.15.37:0/2621016495 shutdown_connections 2024-03-29T08:36:35.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:35.053+0000 7fd828200700 1 --2- 172.21.15.37:0/2621016495 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd8200708a0 0x7fd820070d10 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:36:35.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:35.053+0000 7fd828200700 1 --2- 172.21.15.37:0/2621016495 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd820071f90 0x7fd820070360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:36:35.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:35.053+0000 7fd828200700 1 -- 172.21.15.37:0/2621016495 >> 172.21.15.37:0/2621016495 conn(0x7fd8200f8970 msgr2=0x7fd8200fad80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:36:35.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:35.054+0000 7fd828200700 1 -- 172.21.15.37:0/2621016495 shutdown_connections 2024-03-29T08:36:35.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:35.054+0000 7fd828200700 1 -- 172.21.15.37:0/2621016495 wait complete. 2024-03-29T08:36:35.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:35.054+0000 7fd828200700 1 Processor -- start 2024-03-29T08:36:35.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:35.055+0000 7fd828200700 1 -- start start 2024-03-29T08:36:35.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:35.055+0000 7fd828200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd8200708a0 0x7fd82011a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:36:35.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:35.055+0000 7fd828200700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd820071f90 0x7fd82011a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:36:35.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:35.055+0000 7fd828200700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd82011ac40 con 0x7fd8200708a0 2024-03-29T08:36:35.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:35.055+0000 7fd828200700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd820114010 con 0x7fd820071f90 2024-03-29T08:36:35.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:35.055+0000 7fd825800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd8200708a0 0x7fd82011a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:36:35.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:35.055+0000 7fd825800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd8200708a0 0x7fd82011a0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34250/0 (socket says 172.21.15.37:34250) 2024-03-29T08:36:35.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:35.055+0000 7fd825800700 1 -- 172.21.15.37:0/1986861047 learned_addr learned my addr 172.21.15.37:0/1986861047 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:36:35.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:35.055+0000 7fd824e00700 1 --2- 172.21.15.37:0/1986861047 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd820071f90 0x7fd82011a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:36:35.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:35.056+0000 7fd825800700 1 -- 172.21.15.37:0/1986861047 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd820071f90 msgr2=0x7fd82011a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:36:35.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:35.056+0000 7fd825800700 1 --2- 172.21.15.37:0/1986861047 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd820071f90 0x7fd82011a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:36:35.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:35.056+0000 7fd825800700 1 -- 172.21.15.37:0/1986861047 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd80c005040 con 0x7fd8200708a0 2024-03-29T08:36:35.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:35.056+0000 7fd825800700 1 --2- 172.21.15.37:0/1986861047 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd8200708a0 0x7fd82011a0b0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fd814002a50 tx=0x7fd8140046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:36:35.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:35.056+0000 7fd81d600700 1 -- 172.21.15.37:0/1986861047 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd814007cc0 con 0x7fd8200708a0 2024-03-29T08:36:35.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:35.056+0000 7fd81d600700 1 -- 172.21.15.37:0/1986861047 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd81400f040 con 0x7fd8200708a0 2024-03-29T08:36:35.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:35.056+0000 7fd828200700 1 -- 172.21.15.37:0/1986861047 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8201142f0 con 0x7fd8200708a0 2024-03-29T08:36:35.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:35.056+0000 7fd81d600700 1 -- 172.21.15.37:0/1986861047 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd81400d9c0 con 0x7fd8200708a0 2024-03-29T08:36:35.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:35.056+0000 7fd828200700 1 -- 172.21.15.37:0/1986861047 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd820114800 con 0x7fd8200708a0 2024-03-29T08:36:35.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:35.058+0000 7fd828200700 1 -- 172.21.15.37:0/1986861047 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd820071aa0 con 0x7fd8200708a0 2024-03-29T08:36:35.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:35.059+0000 7fd81d600700 1 -- 172.21.15.37:0/1986861047 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fd81400db20 con 0x7fd8200708a0 2024-03-29T08:36:35.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:35.059+0000 7fd81d600700 1 --2- 172.21.15.37:0/1986861047 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fd80806cc30 0x7fd80806f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:36:35.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:35.060+0000 7fd81d600700 1 -- 172.21.15.37:0/1986861047 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fd81408bd60 con 0x7fd8200708a0 2024-03-29T08:36:35.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:35.060+0000 7fd824e00700 1 --2- 172.21.15.37:0/1986861047 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fd80806cc30 0x7fd80806f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:36:35.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:35.060+0000 7fd824e00700 1 --2- 172.21.15.37:0/1986861047 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fd80806cc30 0x7fd80806f0e0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fd80c00b370 tx=0x7fd80c009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:36:35.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:35.064+0000 7fd81d600700 1 -- 172.21.15.37:0/1986861047 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fd8140597b0 con 0x7fd8200708a0 2024-03-29T08:36:35.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:35.305+0000 7fd828200700 1 -- 172.21.15.37:0/1986861047 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd82004bed0 con 0x7fd80806cc30 2024-03-29T08:36:35.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:35.306+0000 7fd81d600700 1 -- 172.21.15.37:0/1986861047 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fd82004bed0 con 0x7fd80806cc30 2024-03-29T08:36:35.306 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-29T08:36:35.306 INFO:teuthology.orchestra.run.smithi037.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59", 2024-03-29T08:36:35.306 INFO:teuthology.orchestra.run.smithi037.stdout: "in_progress": true, 2024-03-29T08:36:35.306 INFO:teuthology.orchestra.run.smithi037.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-29T08:36:35.306 INFO:teuthology.orchestra.run.smithi037.stdout: "services_complete": [ 2024-03-29T08:36:35.306 INFO:teuthology.orchestra.run.smithi037.stdout: "crash", 2024-03-29T08:36:35.306 INFO:teuthology.orchestra.run.smithi037.stdout: "osd", 2024-03-29T08:36:35.306 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr", 2024-03-29T08:36:35.306 INFO:teuthology.orchestra.run.smithi037.stdout: "mds", 2024-03-29T08:36:35.306 INFO:teuthology.orchestra.run.smithi037.stdout: "mon" 2024-03-29T08:36:35.307 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-29T08:36:35.307 INFO:teuthology.orchestra.run.smithi037.stdout: "progress": "16/21 daemons upgraded", 2024-03-29T08:36:35.307 INFO:teuthology.orchestra.run.smithi037.stdout: "message": "Currently upgrading mds daemons", 2024-03-29T08:36:35.307 INFO:teuthology.orchestra.run.smithi037.stdout: "is_paused": false 2024-03-29T08:36:35.307 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-29T08:36:35.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:35.308+0000 7fd828200700 1 -- 172.21.15.37:0/1986861047 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fd80806cc30 msgr2=0x7fd80806f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:36:35.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:35.308+0000 7fd828200700 1 --2- 172.21.15.37:0/1986861047 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fd80806cc30 0x7fd80806f0e0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fd80c00b370 tx=0x7fd80c009e30 comp rx=0 tx=0).stop 2024-03-29T08:36:35.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:35.308+0000 7fd828200700 1 -- 172.21.15.37:0/1986861047 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd8200708a0 msgr2=0x7fd82011a0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:36:35.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:35.308+0000 7fd828200700 1 --2- 172.21.15.37:0/1986861047 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd8200708a0 0x7fd82011a0b0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fd814002a50 tx=0x7fd8140046b0 comp rx=0 tx=0).stop 2024-03-29T08:36:35.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:35.308+0000 7fd828200700 1 -- 172.21.15.37:0/1986861047 shutdown_connections 2024-03-29T08:36:35.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:35.308+0000 7fd828200700 1 --2- 172.21.15.37:0/1986861047 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fd80806cc30 0x7fd80806f0e0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:36:35.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:35.308+0000 7fd828200700 1 --2- 172.21.15.37:0/1986861047 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd8200708a0 0x7fd82011a0b0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:36:35.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:35.308+0000 7fd828200700 1 --2- 172.21.15.37:0/1986861047 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd820071f90 0x7fd82011a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:36:35.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:35.308+0000 7fd828200700 1 -- 172.21.15.37:0/1986861047 >> 172.21.15.37:0/1986861047 conn(0x7fd8200f8970 msgr2=0x7fd8200f9b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:36:35.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:35.308+0000 7fd828200700 1 -- 172.21.15.37:0/1986861047 shutdown_connections 2024-03-29T08:36:35.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:35.309+0000 7fd828200700 1 -- 172.21.15.37:0/1986861047 wait complete. 2024-03-29T08:36:35.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:35.446+0000 7f1a1c200700 1 -- 172.21.15.37:0/2799852891 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1a14102e00 msgr2=0x7f1a141031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:36:35.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:35.446+0000 7f1a1c200700 1 --2- 172.21.15.37:0/2799852891 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1a14102e00 0x7f1a141031d0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f1a08004940 tx=0x7f1a08009e30 comp rx=0 tx=0).stop 2024-03-29T08:36:35.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:35.446+0000 7f1a1c200700 1 -- 172.21.15.37:0/2799852891 shutdown_connections 2024-03-29T08:36:35.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:35.446+0000 7f1a1c200700 1 --2- 172.21.15.37:0/2799852891 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1a141037a0 0x7f1a140fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:36:35.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:35.446+0000 7f1a1c200700 1 --2- 172.21.15.37:0/2799852891 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1a14102e00 0x7f1a141031d0 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:36:35.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:35.446+0000 7f1a1c200700 1 -- 172.21.15.37:0/2799852891 >> 172.21.15.37:0/2799852891 conn(0x7f1a140f89b0 msgr2=0x7f1a140fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:36:35.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:35.446+0000 7f1a1c200700 1 -- 172.21.15.37:0/2799852891 shutdown_connections 2024-03-29T08:36:35.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:35.446+0000 7f1a1c200700 1 -- 172.21.15.37:0/2799852891 wait complete. 2024-03-29T08:36:35.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:35.447+0000 7f1a1c200700 1 Processor -- start 2024-03-29T08:36:35.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:35.448+0000 7f1a1c200700 1 -- start start 2024-03-29T08:36:35.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:35.448+0000 7f1a1c200700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1a14102e00 0x7f1a14113a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:36:35.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:35.448+0000 7f1a1c200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1a141037a0 0x7f1a14113f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:36:35.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:35.448+0000 7f1a1c200700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a141145d0 con 0x7f1a141037a0 2024-03-29T08:36:35.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:35.448+0000 7f1a1c200700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a1410d9c0 con 0x7f1a14102e00 2024-03-29T08:36:35.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:35.448+0000 7f1a18e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1a141037a0 0x7f1a14113f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:36:35.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:35.448+0000 7f1a18e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1a141037a0 0x7f1a14113f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34264/0 (socket says 172.21.15.37:34264) 2024-03-29T08:36:35.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:35.448+0000 7f1a18e00700 1 -- 172.21.15.37:0/2677261476 learned_addr learned my addr 172.21.15.37:0/2677261476 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:36:35.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:35.448+0000 7f1a19800700 1 --2- 172.21.15.37:0/2677261476 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1a14102e00 0x7f1a14113a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:36:35.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:35.449+0000 7f1a18e00700 1 -- 172.21.15.37:0/2677261476 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1a14102e00 msgr2=0x7f1a14113a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:36:35.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:35.449+0000 7f1a18e00700 1 --2- 172.21.15.37:0/2677261476 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1a14102e00 0x7f1a14113a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:36:35.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:35.449+0000 7f1a18e00700 1 -- 172.21.15.37:0/2677261476 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a08005040 con 0x7f1a141037a0 2024-03-29T08:36:35.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:35.449+0000 7f1a18e00700 1 --2- 172.21.15.37:0/2677261476 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1a141037a0 0x7f1a14113f80 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f1a0000eb40 tx=0x7f1a0000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:36:35.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:35.449+0000 7f1a11600700 1 -- 172.21.15.37:0/2677261476 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1a00007cc0 con 0x7f1a141037a0 2024-03-29T08:36:35.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:35.449+0000 7f1a11600700 1 -- 172.21.15.37:0/2677261476 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f1a0000f040 con 0x7f1a141037a0 2024-03-29T08:36:35.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:35.449+0000 7f1a1c200700 1 -- 172.21.15.37:0/2677261476 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a1410dc40 con 0x7f1a141037a0 2024-03-29T08:36:35.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:35.449+0000 7f1a11600700 1 -- 172.21.15.37:0/2677261476 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1a0000c8f0 con 0x7f1a141037a0 2024-03-29T08:36:35.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:35.450+0000 7f1a1c200700 1 -- 172.21.15.37:0/2677261476 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a1410e200 con 0x7f1a141037a0 2024-03-29T08:36:35.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:35.451+0000 7f1a1c200700 1 -- 172.21.15.37:0/2677261476 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a140fe550 con 0x7f1a141037a0 2024-03-29T08:36:35.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:35.452+0000 7f1a11600700 1 -- 172.21.15.37:0/2677261476 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f1a0000cb60 con 0x7f1a141037a0 2024-03-29T08:36:35.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:35.453+0000 7f1a11600700 1 --2- 172.21.15.37:0/2677261476 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f19fc06cc30 0x7f19fc06f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:36:35.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:35.453+0000 7f1a11600700 1 -- 172.21.15.37:0/2677261476 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f1a0008bb50 con 0x7f1a141037a0 2024-03-29T08:36:35.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:35.453+0000 7f1a19800700 1 --2- 172.21.15.37:0/2677261476 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f19fc06cc30 0x7f19fc06f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:36:35.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:35.454+0000 7f1a19800700 1 --2- 172.21.15.37:0/2677261476 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f19fc06cc30 0x7f19fc06f0e0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f1a0800b0d0 tx=0x7f1a08009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:36:35.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:35.457+0000 7f1a11600700 1 -- 172.21.15.37:0/2677261476 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f1a000595d0 con 0x7f1a141037a0 2024-03-29T08:36:35.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:35.762+0000 7f1a1c200700 1 -- 172.21.15.37:0/2677261476 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f1a14109dc0 con 0x7f1a141037a0 2024-03-29T08:36:35.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:35.762+0000 7f1a11600700 1 -- 172.21.15.37:0/2677261476 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+3583 (secure 0 0 0) 0x7f1a00019e10 con 0x7f1a141037a0 2024-03-29T08:36:35.762 INFO:teuthology.orchestra.run.smithi037.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem is offline 2024-03-29T08:36:35.762 INFO:teuthology.orchestra.run.smithi037.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-03-29T08:36:35.762 INFO:teuthology.orchestra.run.smithi037.stdout: host smithi148 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/mon.smithi148/config 2024-03-29T08:36:35.762 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/podman: timeout after 895 seconds 2024-03-29T08:36:35.762 INFO:teuthology.orchestra.run.smithi037.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59 -e NODE_NAME=smithi148 -e CEPH_USE_RANDOM_NONCE=1 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4:/var/run/ceph:z -v /var/log/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4:/var/log/ceph:z -v /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpkx2rz2d4:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59 inventory --format=json-pretty --filter-for-batch 2024-03-29T08:36:35.762 INFO:teuthology.orchestra.run.smithi037.stdout:Traceback (most recent call last): 2024-03-29T08:36:35.763 INFO:teuthology.orchestra.run.smithi037.stdout: File "/usr/lib64/python3.6/runpy.py", line 193, in _run_module_as_main 2024-03-29T08:36:35.763 INFO:teuthology.orchestra.run.smithi037.stdout: "__main__", mod_spec) 2024-03-29T08:36:35.763 INFO:teuthology.orchestra.run.smithi037.stdout: File "/usr/lib64/python3.6/runpy.py", line 85, in _run_code 2024-03-29T08:36:35.763 INFO:teuthology.orchestra.run.smithi037.stdout: exec(code, run_globals) 2024-03-29T08:36:35.763 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 10889, in 2024-03-29T08:36:35.763 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 10877, in main 2024-03-29T08:36:35.763 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 2576, in _infer_config 2024-03-29T08:36:35.763 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 2492, in _infer_fsid 2024-03-29T08:36:35.763 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 2604, in _infer_image 2024-03-29T08:36:35.763 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 2479, in _validate_fsid 2024-03-29T08:36:35.763 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 7145, in command_ceph_volume 2024-03-29T08:36:35.763 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 2267, in call_throws 2024-03-29T08:36:35.763 INFO:teuthology.orchestra.run.smithi037.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59 -e NODE_NAME=smithi148 -e CEPH_USE_RANDOM_NONCE=1 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4:/var/run/ceph:z -v /var/log/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4:/var/log/ceph:z -v /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpkx2rz2d4:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59 inventory --format=json-pretty --filter-for-batch 2024-03-29T08:36:35.763 INFO:teuthology.orchestra.run.smithi037.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-03-29T08:36:35.763 INFO:teuthology.orchestra.run.smithi037.stdout: fs cephfs is degraded 2024-03-29T08:36:35.764 INFO:teuthology.orchestra.run.smithi037.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-03-29T08:36:35.764 INFO:teuthology.orchestra.run.smithi037.stdout: fs cephfs is offline because no MDS is active for it. 2024-03-29T08:36:35.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:35.764+0000 7f1a1c200700 1 -- 172.21.15.37:0/2677261476 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f19fc06cc30 msgr2=0x7f19fc06f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:36:35.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:35.764+0000 7f1a1c200700 1 --2- 172.21.15.37:0/2677261476 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f19fc06cc30 0x7f19fc06f0e0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f1a0800b0d0 tx=0x7f1a08009e30 comp rx=0 tx=0).stop 2024-03-29T08:36:35.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:35.764+0000 7f1a1c200700 1 -- 172.21.15.37:0/2677261476 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1a141037a0 msgr2=0x7f1a14113f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:36:35.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:35.764+0000 7f1a1c200700 1 --2- 172.21.15.37:0/2677261476 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1a141037a0 0x7f1a14113f80 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f1a0000eb40 tx=0x7f1a0000b040 comp rx=0 tx=0).stop 2024-03-29T08:36:35.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:35.764+0000 7f1a1c200700 1 -- 172.21.15.37:0/2677261476 shutdown_connections 2024-03-29T08:36:35.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:35.764+0000 7f1a1c200700 1 --2- 172.21.15.37:0/2677261476 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f19fc06cc30 0x7f19fc06f0e0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:36:35.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:35.764+0000 7f1a1c200700 1 --2- 172.21.15.37:0/2677261476 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1a14102e00 0x7f1a14113a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:36:35.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:35.764+0000 7f1a1c200700 1 --2- 172.21.15.37:0/2677261476 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1a141037a0 0x7f1a14113f80 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:36:35.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:35.764+0000 7f1a1c200700 1 -- 172.21.15.37:0/2677261476 >> 172.21.15.37:0/2677261476 conn(0x7f1a140f89b0 msgr2=0x7f1a140f99f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:36:35.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:35.765+0000 7f1a1c200700 1 -- 172.21.15.37:0/2677261476 shutdown_connections 2024-03-29T08:36:35.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:36:35.765+0000 7f1a1c200700 1 -- 172.21.15.37:0/2677261476 wait complete. 2024-03-29T08:36:35.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:36:35 smithi148 ceph-mon[106692]: from='client.35410 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:36:35.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:36:35 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/1843969218' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:36:35.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:36:35 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/4001361867' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T08:36:35.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:36:35 smithi037 ceph-mon[142325]: from='client.35410 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:36:35.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:36:35 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/1843969218' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:36:35.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:36:35 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/4001361867' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T08:36:36.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:36:36 smithi148 ceph-mon[106692]: pgmap v772: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:36:36.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:36:36 smithi148 ceph-mon[106692]: from='client.35422 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:36:36.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:36:36 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/2677261476' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T08:36:36.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:36:36 smithi037 ceph-mon[142325]: pgmap v772: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:36:36.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:36:36 smithi037 ceph-mon[142325]: from='client.35422 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:36:36.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:36:36 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/2677261476' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T08:36:38.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:36:38 smithi148 ceph-mon[106692]: pgmap v773: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:36:38.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:36:38 smithi037 ceph-mon[142325]: pgmap v773: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:36:40.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:36:40 smithi148 ceph-mon[106692]: pgmap v774: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:36:40.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:36:40 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:36:40.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:36:40 smithi037 ceph-mon[142325]: pgmap v774: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:36:40.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:36:40 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:36:42.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:36:42 smithi148 ceph-mon[106692]: pgmap v775: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:36:42.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:36:42 smithi037 ceph-mon[142325]: pgmap v775: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:36:44.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:36:44 smithi148 ceph-mon[106692]: pgmap v776: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:36:44.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:36:44 smithi037 ceph-mon[142325]: pgmap v776: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:36:46.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:36:46 smithi148 ceph-mon[106692]: pgmap v777: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:36:46.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:36:46 smithi037 ceph-mon[142325]: pgmap v777: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:36:48.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:36:48 smithi148 ceph-mon[106692]: pgmap v778: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:36:48.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:36:48 smithi037 ceph-mon[142325]: pgmap v778: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:36:50.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:36:50 smithi148 ceph-mon[106692]: pgmap v779: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:36:50.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:36:50 smithi037 ceph-mon[142325]: pgmap v779: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:36:52.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:36:52 smithi148 ceph-mon[106692]: pgmap v780: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:36:52.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:36:52 smithi037 ceph-mon[142325]: pgmap v780: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:36:54.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:36:54 smithi148 ceph-mon[106692]: pgmap v781: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:36:54.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:36:54 smithi037 ceph-mon[142325]: pgmap v781: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:36:55.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:36:55 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:36:55.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:36:55 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:36:56.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:36:56 smithi148 ceph-mon[106692]: pgmap v782: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:36:56.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:36:56 smithi037 ceph-mon[142325]: pgmap v782: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:36:58.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:36:58 smithi148 ceph-mon[106692]: pgmap v783: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:36:58.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:36:58 smithi037 ceph-mon[142325]: pgmap v783: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:37:00.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:37:00 smithi148 ceph-mon[106692]: pgmap v784: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:37:00.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:37:00 smithi037 ceph-mon[142325]: pgmap v784: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:37:02.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:37:02 smithi148 ceph-mon[106692]: pgmap v785: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:37:02.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:37:02 smithi037 ceph-mon[142325]: pgmap v785: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:37:04.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:37:04 smithi148 ceph-mon[106692]: pgmap v786: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:37:04.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:37:04 smithi037 ceph-mon[142325]: pgmap v786: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:37:05.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:05.917+0000 7f6848200700 1 -- 172.21.15.37:0/3960543343 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f68401037a0 msgr2=0x7f68400fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:37:05.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:05.917+0000 7f6848200700 1 --2- 172.21.15.37:0/3960543343 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f68401037a0 0x7f68400fce10 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f682c004940 tx=0x7f682c009e30 comp rx=0 tx=0).stop 2024-03-29T08:37:05.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:05.917+0000 7f6848200700 1 -- 172.21.15.37:0/3960543343 shutdown_connections 2024-03-29T08:37:05.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:05.917+0000 7f6848200700 1 --2- 172.21.15.37:0/3960543343 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f68401037a0 0x7f68400fce10 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:37:05.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:05.917+0000 7f6848200700 1 --2- 172.21.15.37:0/3960543343 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6840102e00 0x7f68401031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:37:05.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:05.917+0000 7f6848200700 1 -- 172.21.15.37:0/3960543343 >> 172.21.15.37:0/3960543343 conn(0x7f68400f89b0 msgr2=0x7f68400fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:37:05.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:05.917+0000 7f6848200700 1 -- 172.21.15.37:0/3960543343 shutdown_connections 2024-03-29T08:37:05.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:05.917+0000 7f6848200700 1 -- 172.21.15.37:0/3960543343 wait complete. 2024-03-29T08:37:05.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:05.918+0000 7f6848200700 1 Processor -- start 2024-03-29T08:37:05.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:05.918+0000 7f6848200700 1 -- start start 2024-03-29T08:37:05.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:05.918+0000 7f6848200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6840102e00 0x7f68400702c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:37:05.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:05.918+0000 7f6848200700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f68401037a0 0x7f6840070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:37:05.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:05.918+0000 7f6848200700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6840070d90 con 0x7f6840102e00 2024-03-29T08:37:05.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:05.918+0000 7f6848200700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6840070ed0 con 0x7f68401037a0 2024-03-29T08:37:05.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:05.918+0000 7f6845800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6840102e00 0x7f68400702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:37:05.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:05.918+0000 7f6845800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6840102e00 0x7f68400702c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52674/0 (socket says 172.21.15.37:52674) 2024-03-29T08:37:05.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:05.919+0000 7f6845800700 1 -- 172.21.15.37:0/1048171803 learned_addr learned my addr 172.21.15.37:0/1048171803 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:37:05.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:05.919+0000 7f6844e00700 1 --2- 172.21.15.37:0/1048171803 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f68401037a0 0x7f6840070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:37:05.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:05.919+0000 7f6845800700 1 -- 172.21.15.37:0/1048171803 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f68401037a0 msgr2=0x7f6840070800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:37:05.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:05.919+0000 7f6845800700 1 --2- 172.21.15.37:0/1048171803 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f68401037a0 0x7f6840070800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:37:05.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:05.919+0000 7f6845800700 1 -- 172.21.15.37:0/1048171803 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f682c005040 con 0x7f6840102e00 2024-03-29T08:37:05.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:05.919+0000 7f6845800700 1 --2- 172.21.15.37:0/1048171803 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6840102e00 0x7f68400702c0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f6834002a50 tx=0x7f68340046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:37:05.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:05.919+0000 7f683d600700 1 -- 172.21.15.37:0/1048171803 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6834007cc0 con 0x7f6840102e00 2024-03-29T08:37:05.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:05.920+0000 7f683d600700 1 -- 172.21.15.37:0/1048171803 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f683400f040 con 0x7f6840102e00 2024-03-29T08:37:05.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:05.920+0000 7f6848200700 1 -- 172.21.15.37:0/1048171803 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6840119e00 con 0x7f6840102e00 2024-03-29T08:37:05.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:05.920+0000 7f683d600700 1 -- 172.21.15.37:0/1048171803 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6834009540 con 0x7f6840102e00 2024-03-29T08:37:05.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:05.920+0000 7f6848200700 1 -- 172.21.15.37:0/1048171803 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f684011a280 con 0x7f6840102e00 2024-03-29T08:37:05.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:05.921+0000 7f6848200700 1 -- 172.21.15.37:0/1048171803 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68400fe550 con 0x7f6840102e00 2024-03-29T08:37:05.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:05.922+0000 7f683d600700 1 -- 172.21.15.37:0/1048171803 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f68340096a0 con 0x7f6840102e00 2024-03-29T08:37:05.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:05.923+0000 7f683d600700 1 --2- 172.21.15.37:0/1048171803 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f682806cc30 0x7f682806f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:37:05.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:05.923+0000 7f683d600700 1 -- 172.21.15.37:0/1048171803 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f683408bed0 con 0x7f6840102e00 2024-03-29T08:37:05.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:05.923+0000 7f6844e00700 1 --2- 172.21.15.37:0/1048171803 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f682806cc30 0x7f682806f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:37:05.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:05.924+0000 7f6844e00700 1 --2- 172.21.15.37:0/1048171803 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f682806cc30 0x7f682806f0e0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f682c00b020 tx=0x7f682c009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:37:05.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:05.928+0000 7f683d600700 1 -- 172.21.15.37:0/1048171803 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f6834055e10 con 0x7f6840102e00 2024-03-29T08:37:06.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:06.157+0000 7f6848200700 1 -- 172.21.15.37:0/1048171803 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f68400719b0 con 0x7f682806cc30 2024-03-29T08:37:06.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:06.158+0000 7f683d600700 1 -- 172.21.15.37:0/1048171803 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f68400719b0 con 0x7f682806cc30 2024-03-29T08:37:06.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:06.161+0000 7f6848200700 1 -- 172.21.15.37:0/1048171803 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f682806cc30 msgr2=0x7f682806f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:37:06.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:06.161+0000 7f6848200700 1 --2- 172.21.15.37:0/1048171803 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f682806cc30 0x7f682806f0e0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f682c00b020 tx=0x7f682c009e30 comp rx=0 tx=0).stop 2024-03-29T08:37:06.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:06.161+0000 7f6848200700 1 -- 172.21.15.37:0/1048171803 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6840102e00 msgr2=0x7f68400702c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:37:06.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:06.161+0000 7f6848200700 1 --2- 172.21.15.37:0/1048171803 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6840102e00 0x7f68400702c0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f6834002a50 tx=0x7f68340046b0 comp rx=0 tx=0).stop 2024-03-29T08:37:06.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:06.161+0000 7f6848200700 1 -- 172.21.15.37:0/1048171803 shutdown_connections 2024-03-29T08:37:06.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:06.161+0000 7f6848200700 1 --2- 172.21.15.37:0/1048171803 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f682806cc30 0x7f682806f0e0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:37:06.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:06.161+0000 7f6848200700 1 --2- 172.21.15.37:0/1048171803 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6840102e00 0x7f68400702c0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:37:06.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:06.161+0000 7f6848200700 1 --2- 172.21.15.37:0/1048171803 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f68401037a0 0x7f6840070800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:37:06.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:06.161+0000 7f6848200700 1 -- 172.21.15.37:0/1048171803 >> 172.21.15.37:0/1048171803 conn(0x7f68400f89b0 msgr2=0x7f68400f9ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:37:06.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:06.161+0000 7f6848200700 1 -- 172.21.15.37:0/1048171803 shutdown_connections 2024-03-29T08:37:06.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:06.161+0000 7f6848200700 1 -- 172.21.15.37:0/1048171803 wait complete. 2024-03-29T08:37:06.177 INFO:teuthology.orchestra.run.smithi037.stdout:true 2024-03-29T08:37:06.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:06.300+0000 7f3991a00700 1 -- 172.21.15.37:0/1144214622 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f398c0fde90 msgr2=0x7f398c0fe260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:37:06.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:06.300+0000 7f3991a00700 1 --2- 172.21.15.37:0/1144214622 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f398c0fde90 0x7f398c0fe260 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f397c004940 tx=0x7f397c009e30 comp rx=0 tx=0).stop 2024-03-29T08:37:06.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:06.300+0000 7f3991a00700 1 -- 172.21.15.37:0/1144214622 shutdown_connections 2024-03-29T08:37:06.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:06.300+0000 7f3991a00700 1 --2- 172.21.15.37:0/1144214622 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f398c0fe830 0x7f398c106f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:37:06.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:06.300+0000 7f3991a00700 1 --2- 172.21.15.37:0/1144214622 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f398c0fde90 0x7f398c0fe260 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:37:06.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:06.300+0000 7f3991a00700 1 -- 172.21.15.37:0/1144214622 >> 172.21.15.37:0/1144214622 conn(0x7f398c072bc0 msgr2=0x7f398c072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:37:06.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:06.300+0000 7f3991a00700 1 -- 172.21.15.37:0/1144214622 shutdown_connections 2024-03-29T08:37:06.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:06.300+0000 7f3991a00700 1 -- 172.21.15.37:0/1144214622 wait complete. 2024-03-29T08:37:06.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:06.301+0000 7f3991a00700 1 Processor -- start 2024-03-29T08:37:06.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:06.302+0000 7f3991a00700 1 -- start start 2024-03-29T08:37:06.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:06.302+0000 7f3991a00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f398c0fde90 0x7f398c0fbd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:37:06.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:06.302+0000 7f3991a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f398c0fe830 0x7f398c0fc290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:37:06.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:06.302+0000 7f3991a00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f398c0fc820 con 0x7f398c0fe830 2024-03-29T08:37:06.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:06.302+0000 7f3991a00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f398c0fc960 con 0x7f398c0fde90 2024-03-29T08:37:06.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:06.302+0000 7f398a000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f398c0fe830 0x7f398c0fc290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:37:06.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:06.302+0000 7f398a000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f398c0fe830 0x7f398c0fc290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52700/0 (socket says 172.21.15.37:52700) 2024-03-29T08:37:06.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:06.303+0000 7f398a000700 1 -- 172.21.15.37:0/2874517557 learned_addr learned my addr 172.21.15.37:0/2874517557 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:37:06.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:06.303+0000 7f398aa00700 1 --2- 172.21.15.37:0/2874517557 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f398c0fde90 0x7f398c0fbd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:37:06.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:06.303+0000 7f398a000700 1 -- 172.21.15.37:0/2874517557 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f398c0fde90 msgr2=0x7f398c0fbd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:37:06.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:06.303+0000 7f398a000700 1 --2- 172.21.15.37:0/2874517557 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f398c0fde90 0x7f398c0fbd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:37:06.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:06.303+0000 7f398a000700 1 -- 172.21.15.37:0/2874517557 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f397c005040 con 0x7f398c0fe830 2024-03-29T08:37:06.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:06.303+0000 7f398a000700 1 --2- 172.21.15.37:0/2874517557 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f398c0fe830 0x7f398c0fc290 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f3978002a50 tx=0x7f39780046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:37:06.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:06.303+0000 7f3982a00700 1 -- 172.21.15.37:0/2874517557 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3978007cc0 con 0x7f398c0fe830 2024-03-29T08:37:06.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:06.303+0000 7f3982a00700 1 -- 172.21.15.37:0/2874517557 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f397800f040 con 0x7f398c0fe830 2024-03-29T08:37:06.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:06.303+0000 7f3991a00700 1 -- 172.21.15.37:0/2874517557 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f398c119df0 con 0x7f398c0fe830 2024-03-29T08:37:06.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:06.304+0000 7f3982a00700 1 -- 172.21.15.37:0/2874517557 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f397800d9c0 con 0x7f398c0fe830 2024-03-29T08:37:06.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:06.304+0000 7f3991a00700 1 -- 172.21.15.37:0/2874517557 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f398c11a1f0 con 0x7f398c0fe830 2024-03-29T08:37:06.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:06.305+0000 7f3991a00700 1 -- 172.21.15.37:0/2874517557 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f398c065e30 con 0x7f398c0fe830 2024-03-29T08:37:06.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:06.306+0000 7f3982a00700 1 -- 172.21.15.37:0/2874517557 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f397800db20 con 0x7f398c0fe830 2024-03-29T08:37:06.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:06.307+0000 7f3982a00700 1 --2- 172.21.15.37:0/2874517557 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f397006cc30 0x7f397006f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:37:06.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:06.307+0000 7f3982a00700 1 -- 172.21.15.37:0/2874517557 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f397808be20 con 0x7f398c0fe830 2024-03-29T08:37:06.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:06.307+0000 7f398aa00700 1 --2- 172.21.15.37:0/2874517557 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f397006cc30 0x7f397006f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:37:06.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:06.308+0000 7f398aa00700 1 --2- 172.21.15.37:0/2874517557 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f397006cc30 0x7f397006f0e0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f397c00b3b0 tx=0x7f397c009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:37:06.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:06.312+0000 7f3982a00700 1 -- 172.21.15.37:0/2874517557 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f3978055d60 con 0x7f398c0fe830 2024-03-29T08:37:06.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:06.546+0000 7f3991a00700 1 -- 172.21.15.37:0/2874517557 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f398c04bed0 con 0x7f397006cc30 2024-03-29T08:37:06.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:06.547+0000 7f3982a00700 1 -- 172.21.15.37:0/2874517557 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f398c04bed0 con 0x7f397006cc30 2024-03-29T08:37:06.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:06.550+0000 7f3991a00700 1 -- 172.21.15.37:0/2874517557 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f397006cc30 msgr2=0x7f397006f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:37:06.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:06.550+0000 7f3991a00700 1 --2- 172.21.15.37:0/2874517557 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f397006cc30 0x7f397006f0e0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f397c00b3b0 tx=0x7f397c009e30 comp rx=0 tx=0).stop 2024-03-29T08:37:06.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:06.550+0000 7f3991a00700 1 -- 172.21.15.37:0/2874517557 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f398c0fe830 msgr2=0x7f398c0fc290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:37:06.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:06.550+0000 7f3991a00700 1 --2- 172.21.15.37:0/2874517557 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f398c0fe830 0x7f398c0fc290 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f3978002a50 tx=0x7f39780046b0 comp rx=0 tx=0).stop 2024-03-29T08:37:06.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:06.550+0000 7f3991a00700 1 -- 172.21.15.37:0/2874517557 shutdown_connections 2024-03-29T08:37:06.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:06.550+0000 7f3991a00700 1 --2- 172.21.15.37:0/2874517557 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f397006cc30 0x7f397006f0e0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:37:06.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:06.550+0000 7f3991a00700 1 --2- 172.21.15.37:0/2874517557 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f398c0fde90 0x7f398c0fbd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:37:06.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:06.550+0000 7f3991a00700 1 --2- 172.21.15.37:0/2874517557 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f398c0fe830 0x7f398c0fc290 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:37:06.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:06.550+0000 7f3991a00700 1 -- 172.21.15.37:0/2874517557 >> 172.21.15.37:0/2874517557 conn(0x7f398c072bc0 msgr2=0x7f398c0ff3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:37:06.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:06.550+0000 7f3991a00700 1 -- 172.21.15.37:0/2874517557 shutdown_connections 2024-03-29T08:37:06.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:06.550+0000 7f3991a00700 1 -- 172.21.15.37:0/2874517557 wait complete. 2024-03-29T08:37:06.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:06.688+0000 7f561fc00700 1 -- 172.21.15.37:0/1943052595 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f56181037a0 msgr2=0x7f56180fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:37:06.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:06.688+0000 7f561fc00700 1 --2- 172.21.15.37:0/1943052595 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f56181037a0 0x7f56180fce10 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f5604004940 tx=0x7f5604009e30 comp rx=0 tx=0).stop 2024-03-29T08:37:06.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:06.689+0000 7f561fc00700 1 -- 172.21.15.37:0/1943052595 shutdown_connections 2024-03-29T08:37:06.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:06.689+0000 7f561fc00700 1 --2- 172.21.15.37:0/1943052595 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f56181037a0 0x7f56180fce10 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:37:06.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:06.689+0000 7f561fc00700 1 --2- 172.21.15.37:0/1943052595 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5618102e00 0x7f56181031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:37:06.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:06.689+0000 7f561fc00700 1 -- 172.21.15.37:0/1943052595 >> 172.21.15.37:0/1943052595 conn(0x7f56180f89b0 msgr2=0x7f56180fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:37:06.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:06.689+0000 7f561fc00700 1 -- 172.21.15.37:0/1943052595 shutdown_connections 2024-03-29T08:37:06.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:06.689+0000 7f561fc00700 1 -- 172.21.15.37:0/1943052595 wait complete. 2024-03-29T08:37:06.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:06.690+0000 7f561fc00700 1 Processor -- start 2024-03-29T08:37:06.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:06.690+0000 7f561fc00700 1 -- start start 2024-03-29T08:37:06.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:06.690+0000 7f561fc00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5618102e00 0x7f5618115da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:37:06.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:06.690+0000 7f561fc00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f56181037a0 0x7f56181162e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:37:06.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:06.690+0000 7f561fc00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5618116930 con 0x7f5618102e00 2024-03-29T08:37:06.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:06.690+0000 7f561fc00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f561810fd00 con 0x7f56181037a0 2024-03-29T08:37:06.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:06.690+0000 7f561d200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5618102e00 0x7f5618115da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:37:06.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:06.690+0000 7f561d200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5618102e00 0x7f5618115da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52704/0 (socket says 172.21.15.37:52704) 2024-03-29T08:37:06.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:06.690+0000 7f561d200700 1 -- 172.21.15.37:0/1188070592 learned_addr learned my addr 172.21.15.37:0/1188070592 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:37:06.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:06.690+0000 7f5617e00700 1 --2- 172.21.15.37:0/1188070592 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f56181037a0 0x7f56181162e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:37:06.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:06.691+0000 7f561d200700 1 -- 172.21.15.37:0/1188070592 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f56181037a0 msgr2=0x7f56181162e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:37:06.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:06.691+0000 7f561d200700 1 --2- 172.21.15.37:0/1188070592 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f56181037a0 0x7f56181162e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:37:06.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:06.691+0000 7f561d200700 1 -- 172.21.15.37:0/1188070592 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5604005040 con 0x7f5618102e00 2024-03-29T08:37:06.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:06.691+0000 7f561d200700 1 --2- 172.21.15.37:0/1188070592 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5618102e00 0x7f5618115da0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f560c002a50 tx=0x7f560c0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:37:06.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:06.691+0000 7f5614c00700 1 -- 172.21.15.37:0/1188070592 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f560c007cc0 con 0x7f5618102e00 2024-03-29T08:37:06.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:06.691+0000 7f5614c00700 1 -- 172.21.15.37:0/1188070592 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f560c00f040 con 0x7f5618102e00 2024-03-29T08:37:06.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:06.691+0000 7f561fc00700 1 -- 172.21.15.37:0/1188070592 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f561810ffe0 con 0x7f5618102e00 2024-03-29T08:37:06.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:06.691+0000 7f5614c00700 1 -- 172.21.15.37:0/1188070592 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f560c00d9c0 con 0x7f5618102e00 2024-03-29T08:37:06.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:06.692+0000 7f561fc00700 1 -- 172.21.15.37:0/1188070592 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56181104f0 con 0x7f5618102e00 2024-03-29T08:37:06.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:06.693+0000 7f561fc00700 1 -- 172.21.15.37:0/1188070592 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56180fe550 con 0x7f5618102e00 2024-03-29T08:37:06.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:06.694+0000 7f5614c00700 1 -- 172.21.15.37:0/1188070592 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f560c00db20 con 0x7f5618102e00 2024-03-29T08:37:06.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:06.694+0000 7f5614c00700 1 --2- 172.21.15.37:0/1188070592 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f560006cc20 0x7f560006f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:37:06.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:06.695+0000 7f5614c00700 1 -- 172.21.15.37:0/1188070592 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f560c08bde0 con 0x7f5618102e00 2024-03-29T08:37:06.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:06.695+0000 7f5617e00700 1 --2- 172.21.15.37:0/1188070592 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f560006cc20 0x7f560006f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:37:06.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:06.695+0000 7f5617e00700 1 --2- 172.21.15.37:0/1188070592 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f560006cc20 0x7f560006f0d0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f5604004890 tx=0x7f5604009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:37:06.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:06.699+0000 7f5614c00700 1 -- 172.21.15.37:0/1188070592 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f560c0597b0 con 0x7f5618102e00 2024-03-29T08:37:06.809 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:37:06 smithi148 ceph-mon[106692]: pgmap v787: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:37:06.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:06.924+0000 7f561fc00700 1 -- 172.21.15.37:0/1188070592 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f561804bf40 con 0x7f560006cc20 2024-03-29T08:37:06.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:37:06 smithi037 ceph-mon[142325]: pgmap v787: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:37:06.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:06.934+0000 7f5614c00700 1 -- 172.21.15.37:0/1188070592 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f561804bf40 con 0x7f560006cc20 2024-03-29T08:37:06.935 INFO:teuthology.orchestra.run.smithi037.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-29T08:37:06.935 INFO:teuthology.orchestra.run.smithi037.stdout:alertmanager.smithi037 smithi037 *:9093,9094 running (26m) 3m ago 39m 14.3M - 0.25.0 c8568f914cd2 2f9f9746ed0f 2024-03-29T08:37:06.935 INFO:teuthology.orchestra.run.smithi037.stdout:crash.smithi037 smithi037 running (23m) 3m ago 40m 6647k - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 178d416a3186 2024-03-29T08:37:06.935 INFO:teuthology.orchestra.run.smithi037.stdout:crash.smithi148 smithi148 running (23m) 3m ago 38m 6656k - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 23666989d58f 2024-03-29T08:37:06.935 INFO:teuthology.orchestra.run.smithi037.stdout:grafana.smithi037 smithi037 *:3000 running (26m) 3m ago 39m 74.5M - 9.4.7 954c08fa6188 6b976fe49952 2024-03-29T08:37:06.935 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi037.iifsmh smithi037 running (19m) 3m ago 33m 16.4M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 2d97f562cc5b 2024-03-29T08:37:06.935 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi037.mmwgxe smithi037 running (19m) 3m ago 33m 16.7M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff ec37037f46e8 2024-03-29T08:37:06.935 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi148.qukpjh smithi148 running (3m) 3m ago 33m 10.7M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 3257219bc046 2024-03-29T08:37:06.935 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi148.sqteih smithi148 running (18m) 3m ago 33m 16.3M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 44c15c77037a 2024-03-29T08:37:06.936 INFO:teuthology.orchestra.run.smithi037.stdout:mgr.smithi037.wjikii smithi037 *:8443,9283,8765 running (29m) 3m ago 41m 510M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff c5a1417b9417 2024-03-29T08:37:06.936 INFO:teuthology.orchestra.run.smithi037.stdout:mgr.smithi148.onhgvm smithi148 *:8443,9283,8765 running (28m) 3m ago 38m 456M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff fcfdbce062bd 2024-03-29T08:37:06.936 INFO:teuthology.orchestra.run.smithi037.stdout:mon.smithi037 smithi037 running (25m) 3m ago 41m 70.2M 2048M 18.2.2-1015-gd0e392d9 1cc2f45c17ff eea2060783af 2024-03-29T08:37:06.936 INFO:teuthology.orchestra.run.smithi037.stdout:mon.smithi148 smithi148 running (24m) 3m ago 38m 50.4M 2048M 18.2.2-1015-gd0e392d9 1cc2f45c17ff ad4b2c5748be 2024-03-29T08:37:06.936 INFO:teuthology.orchestra.run.smithi037.stdout:node-exporter.smithi037 smithi037 *:9100 running (27m) 3m ago 40m 8514k - 1.5.0 0da6a335fe13 26b8ae9cb878 2024-03-29T08:37:06.936 INFO:teuthology.orchestra.run.smithi037.stdout:node-exporter.smithi148 smithi148 *:9100 running (27m) 3m ago 38m 8464k - 1.5.0 0da6a335fe13 98414e4e64ed 2024-03-29T08:37:06.936 INFO:teuthology.orchestra.run.smithi037.stdout:osd.0 smithi037 running (23m) 3m ago 37m 108M 1619M 18.2.2-1015-gd0e392d9 1cc2f45c17ff c0c584f6245c 2024-03-29T08:37:06.936 INFO:teuthology.orchestra.run.smithi037.stdout:osd.1 smithi037 running (22m) 3m ago 37m 104M 1619M 18.2.2-1015-gd0e392d9 1cc2f45c17ff a4a2ff4cc83d 2024-03-29T08:37:06.936 INFO:teuthology.orchestra.run.smithi037.stdout:osd.2 smithi037 running (21m) 3m ago 36m 85.9M 1619M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 8b1bc71aa79e 2024-03-29T08:37:06.936 INFO:teuthology.orchestra.run.smithi037.stdout:osd.3 smithi148 running (21m) 3m ago 36m 109M 2642M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 1c656ab8ff86 2024-03-29T08:37:06.936 INFO:teuthology.orchestra.run.smithi037.stdout:osd.4 smithi148 running (20m) 3m ago 35m 101M 2642M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 966a9de11d22 2024-03-29T08:37:06.936 INFO:teuthology.orchestra.run.smithi037.stdout:osd.5 smithi148 running (20m) 3m ago 35m 89.4M 2642M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 474c4f9212ce 2024-03-29T08:37:06.936 INFO:teuthology.orchestra.run.smithi037.stdout:prometheus.smithi037 smithi037 *:9095 running (27m) 3m ago 39m 61.0M - 2.43.0 a07b618ecd1d 30c0223ec589 2024-03-29T08:37:06.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:06.937+0000 7f561fc00700 1 -- 172.21.15.37:0/1188070592 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f560006cc20 msgr2=0x7f560006f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:37:06.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:06.937+0000 7f561fc00700 1 --2- 172.21.15.37:0/1188070592 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f560006cc20 0x7f560006f0d0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f5604004890 tx=0x7f5604009e30 comp rx=0 tx=0).stop 2024-03-29T08:37:06.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:06.937+0000 7f561fc00700 1 -- 172.21.15.37:0/1188070592 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5618102e00 msgr2=0x7f5618115da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:37:06.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:06.937+0000 7f561fc00700 1 --2- 172.21.15.37:0/1188070592 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5618102e00 0x7f5618115da0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f560c002a50 tx=0x7f560c0046b0 comp rx=0 tx=0).stop 2024-03-29T08:37:06.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:06.937+0000 7f561fc00700 1 -- 172.21.15.37:0/1188070592 shutdown_connections 2024-03-29T08:37:06.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:06.937+0000 7f561fc00700 1 --2- 172.21.15.37:0/1188070592 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f560006cc20 0x7f560006f0d0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:37:06.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:06.937+0000 7f561fc00700 1 --2- 172.21.15.37:0/1188070592 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5618102e00 0x7f5618115da0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:37:06.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:06.937+0000 7f561fc00700 1 --2- 172.21.15.37:0/1188070592 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f56181037a0 0x7f56181162e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:37:06.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:06.937+0000 7f561fc00700 1 -- 172.21.15.37:0/1188070592 >> 172.21.15.37:0/1188070592 conn(0x7f56180f89b0 msgr2=0x7f56180f9a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:37:06.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:06.937+0000 7f561fc00700 1 -- 172.21.15.37:0/1188070592 shutdown_connections 2024-03-29T08:37:06.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:06.937+0000 7f561fc00700 1 -- 172.21.15.37:0/1188070592 wait complete. 2024-03-29T08:37:07.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:07.072+0000 7f18abc00700 1 -- 172.21.15.37:0/2783723239 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f18a4102e00 msgr2=0x7f18a41031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:37:07.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:07.072+0000 7f18abc00700 1 --2- 172.21.15.37:0/2783723239 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f18a4102e00 0x7f18a41031d0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f1898004940 tx=0x7f1898009e30 comp rx=0 tx=0).stop 2024-03-29T08:37:07.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:07.072+0000 7f18abc00700 1 -- 172.21.15.37:0/2783723239 shutdown_connections 2024-03-29T08:37:07.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:07.072+0000 7f18abc00700 1 --2- 172.21.15.37:0/2783723239 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f18a41037a0 0x7f18a40fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:37:07.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:07.072+0000 7f18abc00700 1 --2- 172.21.15.37:0/2783723239 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f18a4102e00 0x7f18a41031d0 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:37:07.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:07.072+0000 7f18abc00700 1 -- 172.21.15.37:0/2783723239 >> 172.21.15.37:0/2783723239 conn(0x7f18a40f89b0 msgr2=0x7f18a40fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:37:07.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:07.073+0000 7f18abc00700 1 -- 172.21.15.37:0/2783723239 shutdown_connections 2024-03-29T08:37:07.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:07.073+0000 7f18abc00700 1 -- 172.21.15.37:0/2783723239 wait complete. 2024-03-29T08:37:07.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:07.074+0000 7f18abc00700 1 Processor -- start 2024-03-29T08:37:07.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:07.074+0000 7f18abc00700 1 -- start start 2024-03-29T08:37:07.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:07.074+0000 7f18abc00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f18a4102e00 0x7f18a40702c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:37:07.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:07.074+0000 7f18abc00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f18a41037a0 0x7f18a4070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:37:07.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:07.074+0000 7f18abc00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18a4070d90 con 0x7f18a41037a0 2024-03-29T08:37:07.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:07.074+0000 7f18abc00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18a4070ed0 con 0x7f18a4102e00 2024-03-29T08:37:07.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:07.075+0000 7f18a3e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f18a41037a0 0x7f18a4070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:37:07.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:07.075+0000 7f18a3e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f18a41037a0 0x7f18a4070800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52720/0 (socket says 172.21.15.37:52720) 2024-03-29T08:37:07.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:07.075+0000 7f18a3e00700 1 -- 172.21.15.37:0/3949507226 learned_addr learned my addr 172.21.15.37:0/3949507226 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:37:07.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:07.075+0000 7f18a9200700 1 --2- 172.21.15.37:0/3949507226 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f18a4102e00 0x7f18a40702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:37:07.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:07.075+0000 7f18a3e00700 1 -- 172.21.15.37:0/3949507226 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f18a4102e00 msgr2=0x7f18a40702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:37:07.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:07.075+0000 7f18a3e00700 1 --2- 172.21.15.37:0/3949507226 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f18a4102e00 0x7f18a40702c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:37:07.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:07.075+0000 7f18a3e00700 1 -- 172.21.15.37:0/3949507226 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1898005040 con 0x7f18a41037a0 2024-03-29T08:37:07.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:07.075+0000 7f18a3e00700 1 --2- 172.21.15.37:0/3949507226 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f18a41037a0 0x7f18a4070800 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f189000eb40 tx=0x7f189000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:37:07.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:07.076+0000 7f18a0c00700 1 -- 172.21.15.37:0/3949507226 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1890007cc0 con 0x7f18a41037a0 2024-03-29T08:37:07.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:07.076+0000 7f18abc00700 1 -- 172.21.15.37:0/3949507226 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18a4071120 con 0x7f18a41037a0 2024-03-29T08:37:07.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:07.076+0000 7f18a0c00700 1 -- 172.21.15.37:0/3949507226 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f189000f040 con 0x7f18a41037a0 2024-03-29T08:37:07.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:07.076+0000 7f18abc00700 1 -- 172.21.15.37:0/3949507226 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18a411a170 con 0x7f18a41037a0 2024-03-29T08:37:07.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:07.076+0000 7f18a0c00700 1 -- 172.21.15.37:0/3949507226 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f189000c890 con 0x7f18a41037a0 2024-03-29T08:37:07.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:07.077+0000 7f18abc00700 1 -- 172.21.15.37:0/3949507226 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18a40fe550 con 0x7f18a41037a0 2024-03-29T08:37:07.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:07.078+0000 7f18a0c00700 1 -- 172.21.15.37:0/3949507226 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f18900090d0 con 0x7f18a41037a0 2024-03-29T08:37:07.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:07.079+0000 7f18a0c00700 1 --2- 172.21.15.37:0/3949507226 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f188c06cc30 0x7f188c06f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:37:07.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:07.079+0000 7f18a0c00700 1 -- 172.21.15.37:0/3949507226 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f189008bad0 con 0x7f18a41037a0 2024-03-29T08:37:07.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:07.079+0000 7f18a9200700 1 --2- 172.21.15.37:0/3949507226 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f188c06cc30 0x7f188c06f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:37:07.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:07.080+0000 7f18a9200700 1 --2- 172.21.15.37:0/3949507226 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f188c06cc30 0x7f188c06f0e0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f189800b0d0 tx=0x7f1898009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:37:07.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:07.083+0000 7f18a0c00700 1 -- 172.21.15.37:0/3949507226 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f1890055ac0 con 0x7f18a41037a0 2024-03-29T08:37:07.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:07.383+0000 7f18abc00700 1 -- 172.21.15.37:0/3949507226 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f18a4109e50 con 0x7f18a41037a0 2024-03-29T08:37:07.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:07.384+0000 7f18a0c00700 1 -- 172.21.15.37:0/3949507226 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7f18900590e0 con 0x7f18a41037a0 2024-03-29T08:37:07.385 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-29T08:37:07.385 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": { 2024-03-29T08:37:07.385 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 2 2024-03-29T08:37:07.385 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:37:07.385 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": { 2024-03-29T08:37:07.385 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 2 2024-03-29T08:37:07.385 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:37:07.386 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": { 2024-03-29T08:37:07.386 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 6 2024-03-29T08:37:07.386 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:37:07.386 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": { 2024-03-29T08:37:07.386 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 4 2024-03-29T08:37:07.386 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:37:07.386 INFO:teuthology.orchestra.run.smithi037.stdout: "overall": { 2024-03-29T08:37:07.386 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 14 2024-03-29T08:37:07.386 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-29T08:37:07.386 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-29T08:37:07.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:07.387+0000 7f18abc00700 1 -- 172.21.15.37:0/3949507226 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f188c06cc30 msgr2=0x7f188c06f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:37:07.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:07.387+0000 7f18abc00700 1 --2- 172.21.15.37:0/3949507226 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f188c06cc30 0x7f188c06f0e0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f189800b0d0 tx=0x7f1898009e30 comp rx=0 tx=0).stop 2024-03-29T08:37:07.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:07.387+0000 7f18abc00700 1 -- 172.21.15.37:0/3949507226 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f18a41037a0 msgr2=0x7f18a4070800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:37:07.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:07.387+0000 7f18abc00700 1 --2- 172.21.15.37:0/3949507226 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f18a41037a0 0x7f18a4070800 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f189000eb40 tx=0x7f189000b040 comp rx=0 tx=0).stop 2024-03-29T08:37:07.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:07.387+0000 7f18abc00700 1 -- 172.21.15.37:0/3949507226 shutdown_connections 2024-03-29T08:37:07.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:07.387+0000 7f18abc00700 1 --2- 172.21.15.37:0/3949507226 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f188c06cc30 0x7f188c06f0e0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:37:07.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:07.387+0000 7f18abc00700 1 --2- 172.21.15.37:0/3949507226 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f18a4102e00 0x7f18a40702c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:37:07.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:07.387+0000 7f18abc00700 1 --2- 172.21.15.37:0/3949507226 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f18a41037a0 0x7f18a4070800 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:37:07.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:07.387+0000 7f18abc00700 1 -- 172.21.15.37:0/3949507226 >> 172.21.15.37:0/3949507226 conn(0x7f18a40f89b0 msgr2=0x7f18a40f99f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:37:07.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:07.387+0000 7f18abc00700 1 -- 172.21.15.37:0/3949507226 shutdown_connections 2024-03-29T08:37:07.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:07.387+0000 7f18abc00700 1 -- 172.21.15.37:0/3949507226 wait complete. 2024-03-29T08:37:07.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:07.524+0000 7f25e6c00700 1 -- 172.21.15.37:0/3408174705 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f25e0105000 msgr2=0x7f25e01053d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:37:07.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:07.524+0000 7f25e6c00700 1 --2- 172.21.15.37:0/3408174705 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f25e0105000 0x7f25e01053d0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f25d0004940 tx=0x7f25d0009e30 comp rx=0 tx=0).stop 2024-03-29T08:37:07.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:07.524+0000 7f25e6c00700 1 -- 172.21.15.37:0/3408174705 shutdown_connections 2024-03-29T08:37:07.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:07.524+0000 7f25e6c00700 1 --2- 172.21.15.37:0/3408174705 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f25e01059a0 0x7f25e00ff010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:37:07.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:07.524+0000 7f25e6c00700 1 --2- 172.21.15.37:0/3408174705 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f25e0105000 0x7f25e01053d0 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:37:07.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:07.524+0000 7f25e6c00700 1 -- 172.21.15.37:0/3408174705 >> 172.21.15.37:0/3408174705 conn(0x7f25e00fabd0 msgr2=0x7f25e00fcfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:37:07.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:07.524+0000 7f25e6c00700 1 -- 172.21.15.37:0/3408174705 shutdown_connections 2024-03-29T08:37:07.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:07.524+0000 7f25e6c00700 1 -- 172.21.15.37:0/3408174705 wait complete. 2024-03-29T08:37:07.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:07.525+0000 7f25e6c00700 1 Processor -- start 2024-03-29T08:37:07.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:07.526+0000 7f25e6c00700 1 -- start start 2024-03-29T08:37:07.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:07.526+0000 7f25e6c00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f25e0105000 0x7f25e0115cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:37:07.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:07.526+0000 7f25e6c00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f25e01059a0 0x7f25e0116230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:37:07.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:07.526+0000 7f25e6c00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25e0116880 con 0x7f25e0105000 2024-03-29T08:37:07.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:07.526+0000 7f25e6c00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25e010fc50 con 0x7f25e01059a0 2024-03-29T08:37:07.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:07.526+0000 7f25dfe00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f25e0105000 0x7f25e0115cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:37:07.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:07.526+0000 7f25dfe00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f25e0105000 0x7f25e0115cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52730/0 (socket says 172.21.15.37:52730) 2024-03-29T08:37:07.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:07.526+0000 7f25dfe00700 1 -- 172.21.15.37:0/2438146260 learned_addr learned my addr 172.21.15.37:0/2438146260 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:37:07.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:07.527+0000 7f25df400700 1 --2- 172.21.15.37:0/2438146260 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f25e01059a0 0x7f25e0116230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:37:07.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:07.527+0000 7f25dfe00700 1 -- 172.21.15.37:0/2438146260 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f25e01059a0 msgr2=0x7f25e0116230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:37:07.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:07.527+0000 7f25dfe00700 1 --2- 172.21.15.37:0/2438146260 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f25e01059a0 0x7f25e0116230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:37:07.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:07.527+0000 7f25dfe00700 1 -- 172.21.15.37:0/2438146260 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25d0005040 con 0x7f25e0105000 2024-03-29T08:37:07.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:07.527+0000 7f25dfe00700 1 --2- 172.21.15.37:0/2438146260 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f25e0105000 0x7f25e0115cf0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f25d0009390 tx=0x7f25d0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:37:07.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:07.527+0000 7f25d7e00700 1 -- 172.21.15.37:0/2438146260 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f25d000e040 con 0x7f25e0105000 2024-03-29T08:37:07.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:07.527+0000 7f25d7e00700 1 -- 172.21.15.37:0/2438146260 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f25d0013420 con 0x7f25e0105000 2024-03-29T08:37:07.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:07.527+0000 7f25e6c00700 1 -- 172.21.15.37:0/2438146260 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25e010fed0 con 0x7f25e0105000 2024-03-29T08:37:07.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:07.527+0000 7f25d7e00700 1 -- 172.21.15.37:0/2438146260 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f25d001c630 con 0x7f25e0105000 2024-03-29T08:37:07.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:07.528+0000 7f25e6c00700 1 -- 172.21.15.37:0/2438146260 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25e0110490 con 0x7f25e0105000 2024-03-29T08:37:07.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:07.529+0000 7f25e6c00700 1 -- 172.21.15.37:0/2438146260 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25e0102310 con 0x7f25e0105000 2024-03-29T08:37:07.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:07.530+0000 7f25d7e00700 1 -- 172.21.15.37:0/2438146260 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f25d00139e0 con 0x7f25e0105000 2024-03-29T08:37:07.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:07.530+0000 7f25d7e00700 1 --2- 172.21.15.37:0/2438146260 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f25c406cc30 0x7f25c406f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:37:07.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:07.530+0000 7f25d7e00700 1 -- 172.21.15.37:0/2438146260 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f25d008d2a0 con 0x7f25e0105000 2024-03-29T08:37:07.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:07.531+0000 7f25df400700 1 --2- 172.21.15.37:0/2438146260 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f25c406cc30 0x7f25c406f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:37:07.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:07.531+0000 7f25df400700 1 --2- 172.21.15.37:0/2438146260 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f25c406cc30 0x7f25c406f0e0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f25e0111590 tx=0x7f25cc009ee0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:37:07.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:07.535+0000 7f25d7e00700 1 -- 172.21.15.37:0/2438146260 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f25d005ae20 con 0x7f25e0105000 2024-03-29T08:37:07.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:07.796+0000 7f25e6c00700 1 -- 172.21.15.37:0/2438146260 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f25e004bed0 con 0x7f25e0105000 2024-03-29T08:37:07.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:07.796+0000 7f25d7e00700 1 -- 172.21.15.37:0/2438146260 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 30 v30) v1 ==== 76+0+1841 (secure 0 0 0) 0x7f25d005a9b0 con 0x7f25e0105000 2024-03-29T08:37:07.797 INFO:teuthology.orchestra.run.smithi037.stdout:e30 2024-03-29T08:37:07.797 INFO:teuthology.orchestra.run.smithi037.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-29T08:37:07.797 INFO:teuthology.orchestra.run.smithi037.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T08:37:07.797 INFO:teuthology.orchestra.run.smithi037.stdout:legacy client fscid: 1 2024-03-29T08:37:07.798 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:37:07.798 INFO:teuthology.orchestra.run.smithi037.stdout:Filesystem 'cephfs' (1) 2024-03-29T08:37:07.798 INFO:teuthology.orchestra.run.smithi037.stdout:fs_name cephfs 2024-03-29T08:37:07.798 INFO:teuthology.orchestra.run.smithi037.stdout:epoch 21 2024-03-29T08:37:07.798 INFO:teuthology.orchestra.run.smithi037.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-03-29T08:37:07.798 INFO:teuthology.orchestra.run.smithi037.stdout:created 2024-03-29T08:03:29.241269+0000 2024-03-29T08:37:07.798 INFO:teuthology.orchestra.run.smithi037.stdout:modified 2024-03-29T08:17:16.237238+0000 2024-03-29T08:37:07.798 INFO:teuthology.orchestra.run.smithi037.stdout:tableserver 0 2024-03-29T08:37:07.798 INFO:teuthology.orchestra.run.smithi037.stdout:root 0 2024-03-29T08:37:07.798 INFO:teuthology.orchestra.run.smithi037.stdout:session_timeout 60 2024-03-29T08:37:07.798 INFO:teuthology.orchestra.run.smithi037.stdout:session_autoclose 300 2024-03-29T08:37:07.798 INFO:teuthology.orchestra.run.smithi037.stdout:max_file_size 1099511627776 2024-03-29T08:37:07.798 INFO:teuthology.orchestra.run.smithi037.stdout:max_xattr_size 65536 2024-03-29T08:37:07.798 INFO:teuthology.orchestra.run.smithi037.stdout:required_client_features {} 2024-03-29T08:37:07.799 INFO:teuthology.orchestra.run.smithi037.stdout:last_failure 0 2024-03-29T08:37:07.799 INFO:teuthology.orchestra.run.smithi037.stdout:last_failure_osd_epoch 82 2024-03-29T08:37:07.799 INFO:teuthology.orchestra.run.smithi037.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T08:37:07.799 INFO:teuthology.orchestra.run.smithi037.stdout:max_mds 2 2024-03-29T08:37:07.799 INFO:teuthology.orchestra.run.smithi037.stdout:in 0,1 2024-03-29T08:37:07.799 INFO:teuthology.orchestra.run.smithi037.stdout:up {} 2024-03-29T08:37:07.799 INFO:teuthology.orchestra.run.smithi037.stdout:failed 0,1 2024-03-29T08:37:07.799 INFO:teuthology.orchestra.run.smithi037.stdout:damaged 2024-03-29T08:37:07.799 INFO:teuthology.orchestra.run.smithi037.stdout:stopped 2024-03-29T08:37:07.799 INFO:teuthology.orchestra.run.smithi037.stdout:data_pools [3] 2024-03-29T08:37:07.799 INFO:teuthology.orchestra.run.smithi037.stdout:metadata_pool 2 2024-03-29T08:37:07.799 INFO:teuthology.orchestra.run.smithi037.stdout:inline_data disabled 2024-03-29T08:37:07.799 INFO:teuthology.orchestra.run.smithi037.stdout:balancer 2024-03-29T08:37:07.799 INFO:teuthology.orchestra.run.smithi037.stdout:bal_rank_mask -1 2024-03-29T08:37:07.799 INFO:teuthology.orchestra.run.smithi037.stdout:standby_count_wanted 1 2024-03-29T08:37:07.800 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:37:07.800 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:37:07.800 INFO:teuthology.orchestra.run.smithi037.stdout:Standby daemons: 2024-03-29T08:37:07.800 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:37:07.800 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi037.mmwgxe{-1:34440} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.37:6828/1248142005,v1:172.21.15.37:6829/1248142005] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:37:07.800 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi037.iifsmh{-1:34444} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.37:6826/2060882959,v1:172.21.15.37:6827/2060882959] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:37:07.800 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi148.sqteih{-1:44395} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.148:6824/4078652242,v1:172.21.15.148:6825/4078652242] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:37:07.800 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi148.qukpjh{-1:45007} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.148:6826/2394751105,v1:172.21.15.148:6827/2394751105] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:37:07.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:07.798+0000 7f25e6c00700 1 -- 172.21.15.37:0/2438146260 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f25c406cc30 msgr2=0x7f25c406f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:37:07.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:07.798+0000 7f25e6c00700 1 --2- 172.21.15.37:0/2438146260 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f25c406cc30 0x7f25c406f0e0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f25e0111590 tx=0x7f25cc009ee0 comp rx=0 tx=0).stop 2024-03-29T08:37:07.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:07.799+0000 7f25e6c00700 1 -- 172.21.15.37:0/2438146260 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f25e0105000 msgr2=0x7f25e0115cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:37:07.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:07.799+0000 7f25e6c00700 1 --2- 172.21.15.37:0/2438146260 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f25e0105000 0x7f25e0115cf0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f25d0009390 tx=0x7f25d0009e30 comp rx=0 tx=0).stop 2024-03-29T08:37:07.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:07.799+0000 7f25e6c00700 1 -- 172.21.15.37:0/2438146260 shutdown_connections 2024-03-29T08:37:07.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:07.799+0000 7f25e6c00700 1 --2- 172.21.15.37:0/2438146260 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f25c406cc30 0x7f25c406f0e0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:37:07.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:07.799+0000 7f25e6c00700 1 --2- 172.21.15.37:0/2438146260 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f25e0105000 0x7f25e0115cf0 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:37:07.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:07.799+0000 7f25e6c00700 1 --2- 172.21.15.37:0/2438146260 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f25e01059a0 0x7f25e0116230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:37:07.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:07.799+0000 7f25e6c00700 1 -- 172.21.15.37:0/2438146260 >> 172.21.15.37:0/2438146260 conn(0x7f25e00fabd0 msgr2=0x7f25e00fcf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:37:07.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:07.799+0000 7f25e6c00700 1 -- 172.21.15.37:0/2438146260 shutdown_connections 2024-03-29T08:37:07.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:07.799+0000 7f25e6c00700 1 -- 172.21.15.37:0/2438146260 wait complete. 2024-03-29T08:37:07.801 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 30 2024-03-29T08:37:07.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:37:07 smithi037 ceph-mon[142325]: from='client.35430 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:37:07.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:37:07 smithi037 ceph-mon[142325]: from='client.35434 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:37:07.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:37:07 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/3949507226' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:37:07.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:07.936+0000 7fb78bc00700 1 -- 172.21.15.37:0/662171196 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb784102e00 msgr2=0x7fb7841031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:37:07.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:07.936+0000 7fb78bc00700 1 --2- 172.21.15.37:0/662171196 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb784102e00 0x7fb7841031d0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fb778004940 tx=0x7fb778009e30 comp rx=0 tx=0).stop 2024-03-29T08:37:07.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:07.936+0000 7fb78bc00700 1 -- 172.21.15.37:0/662171196 shutdown_connections 2024-03-29T08:37:07.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:07.936+0000 7fb78bc00700 1 --2- 172.21.15.37:0/662171196 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb7841037a0 0x7fb7840fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:37:07.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:07.936+0000 7fb78bc00700 1 --2- 172.21.15.37:0/662171196 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb784102e00 0x7fb7841031d0 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:37:07.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:07.936+0000 7fb78bc00700 1 -- 172.21.15.37:0/662171196 >> 172.21.15.37:0/662171196 conn(0x7fb7840f89b0 msgr2=0x7fb7840fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:37:07.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:07.936+0000 7fb78bc00700 1 -- 172.21.15.37:0/662171196 shutdown_connections 2024-03-29T08:37:07.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:07.936+0000 7fb78bc00700 1 -- 172.21.15.37:0/662171196 wait complete. 2024-03-29T08:37:07.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:07.937+0000 7fb78bc00700 1 Processor -- start 2024-03-29T08:37:07.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:07.937+0000 7fb78bc00700 1 -- start start 2024-03-29T08:37:07.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:07.937+0000 7fb78bc00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb784102e00 0x7fb7840702c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:37:07.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:07.937+0000 7fb78bc00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb7841037a0 0x7fb784070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:37:07.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:07.937+0000 7fb78bc00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb784070d90 con 0x7fb784102e00 2024-03-29T08:37:07.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:07.937+0000 7fb78bc00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb784070ed0 con 0x7fb7841037a0 2024-03-29T08:37:07.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:07.938+0000 7fb789200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb784102e00 0x7fb7840702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:37:07.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:07.938+0000 7fb789200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb784102e00 0x7fb7840702c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52752/0 (socket says 172.21.15.37:52752) 2024-03-29T08:37:07.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:07.938+0000 7fb789200700 1 -- 172.21.15.37:0/742594004 learned_addr learned my addr 172.21.15.37:0/742594004 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:37:07.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:07.938+0000 7fb783e00700 1 --2- 172.21.15.37:0/742594004 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb7841037a0 0x7fb784070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:37:07.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:07.938+0000 7fb789200700 1 -- 172.21.15.37:0/742594004 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb7841037a0 msgr2=0x7fb784070800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:37:07.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:07.938+0000 7fb789200700 1 --2- 172.21.15.37:0/742594004 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb7841037a0 0x7fb784070800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:37:07.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:07.938+0000 7fb789200700 1 -- 172.21.15.37:0/742594004 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb778005040 con 0x7fb784102e00 2024-03-29T08:37:07.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:07.938+0000 7fb789200700 1 --2- 172.21.15.37:0/742594004 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb784102e00 0x7fb7840702c0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fb778009390 tx=0x7fb778009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:37:07.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:07.939+0000 7fb780c00700 1 -- 172.21.15.37:0/742594004 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb77800e040 con 0x7fb784102e00 2024-03-29T08:37:07.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:07.939+0000 7fb78bc00700 1 -- 172.21.15.37:0/742594004 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb784071120 con 0x7fb784102e00 2024-03-29T08:37:07.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:07.939+0000 7fb780c00700 1 -- 172.21.15.37:0/742594004 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fb778013420 con 0x7fb784102e00 2024-03-29T08:37:07.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:07.939+0000 7fb780c00700 1 -- 172.21.15.37:0/742594004 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb77801c630 con 0x7fb784102e00 2024-03-29T08:37:07.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:07.939+0000 7fb78bc00700 1 -- 172.21.15.37:0/742594004 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb78411a270 con 0x7fb784102e00 2024-03-29T08:37:07.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:07.940+0000 7fb78bc00700 1 -- 172.21.15.37:0/742594004 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7840fe550 con 0x7fb784102e00 2024-03-29T08:37:07.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:07.941+0000 7fb780c00700 1 -- 172.21.15.37:0/742594004 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fb7780139e0 con 0x7fb784102e00 2024-03-29T08:37:07.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:07.942+0000 7fb780c00700 1 --2- 172.21.15.37:0/742594004 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fb76c06cc30 0x7fb76c06f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:37:07.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:07.942+0000 7fb780c00700 1 -- 172.21.15.37:0/742594004 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fb77808d310 con 0x7fb784102e00 2024-03-29T08:37:07.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:07.942+0000 7fb783e00700 1 --2- 172.21.15.37:0/742594004 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fb76c06cc30 0x7fb76c06f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:37:07.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:07.942+0000 7fb783e00700 1 --2- 172.21.15.37:0/742594004 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fb76c06cc30 0x7fb76c06f0e0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fb770009fd0 tx=0x7fb770009f20 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:37:07.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:07.946+0000 7fb780c00700 1 -- 172.21.15.37:0/742594004 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fb7780572d0 con 0x7fb784102e00 2024-03-29T08:37:08.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:37:07 smithi148 ceph-mon[106692]: from='client.35430 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:37:08.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:37:07 smithi148 ceph-mon[106692]: from='client.35434 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:37:08.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:37:07 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/3949507226' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:37:08.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:08.176+0000 7fb78bc00700 1 -- 172.21.15.37:0/742594004 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb7840719b0 con 0x7fb76c06cc30 2024-03-29T08:37:08.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:08.178+0000 7fb780c00700 1 -- 172.21.15.37:0/742594004 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fb7840719b0 con 0x7fb76c06cc30 2024-03-29T08:37:08.178 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-29T08:37:08.178 INFO:teuthology.orchestra.run.smithi037.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59", 2024-03-29T08:37:08.179 INFO:teuthology.orchestra.run.smithi037.stdout: "in_progress": true, 2024-03-29T08:37:08.179 INFO:teuthology.orchestra.run.smithi037.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-29T08:37:08.179 INFO:teuthology.orchestra.run.smithi037.stdout: "services_complete": [ 2024-03-29T08:37:08.179 INFO:teuthology.orchestra.run.smithi037.stdout: "crash", 2024-03-29T08:37:08.179 INFO:teuthology.orchestra.run.smithi037.stdout: "osd", 2024-03-29T08:37:08.179 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr", 2024-03-29T08:37:08.179 INFO:teuthology.orchestra.run.smithi037.stdout: "mds", 2024-03-29T08:37:08.179 INFO:teuthology.orchestra.run.smithi037.stdout: "mon" 2024-03-29T08:37:08.179 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-29T08:37:08.179 INFO:teuthology.orchestra.run.smithi037.stdout: "progress": "16/21 daemons upgraded", 2024-03-29T08:37:08.179 INFO:teuthology.orchestra.run.smithi037.stdout: "message": "Currently upgrading mds daemons", 2024-03-29T08:37:08.179 INFO:teuthology.orchestra.run.smithi037.stdout: "is_paused": false 2024-03-29T08:37:08.179 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-29T08:37:08.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:08.180+0000 7fb78bc00700 1 -- 172.21.15.37:0/742594004 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fb76c06cc30 msgr2=0x7fb76c06f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:37:08.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:08.180+0000 7fb78bc00700 1 --2- 172.21.15.37:0/742594004 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fb76c06cc30 0x7fb76c06f0e0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fb770009fd0 tx=0x7fb770009f20 comp rx=0 tx=0).stop 2024-03-29T08:37:08.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:08.180+0000 7fb78bc00700 1 -- 172.21.15.37:0/742594004 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb784102e00 msgr2=0x7fb7840702c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:37:08.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:08.180+0000 7fb78bc00700 1 --2- 172.21.15.37:0/742594004 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb784102e00 0x7fb7840702c0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fb778009390 tx=0x7fb778009e30 comp rx=0 tx=0).stop 2024-03-29T08:37:08.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:08.180+0000 7fb78bc00700 1 -- 172.21.15.37:0/742594004 shutdown_connections 2024-03-29T08:37:08.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:08.180+0000 7fb78bc00700 1 --2- 172.21.15.37:0/742594004 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fb76c06cc30 0x7fb76c06f0e0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:37:08.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:08.180+0000 7fb78bc00700 1 --2- 172.21.15.37:0/742594004 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb784102e00 0x7fb7840702c0 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:37:08.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:08.180+0000 7fb78bc00700 1 --2- 172.21.15.37:0/742594004 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb7841037a0 0x7fb784070800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:37:08.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:08.180+0000 7fb78bc00700 1 -- 172.21.15.37:0/742594004 >> 172.21.15.37:0/742594004 conn(0x7fb7840f89b0 msgr2=0x7fb7840f9a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:37:08.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:08.180+0000 7fb78bc00700 1 -- 172.21.15.37:0/742594004 shutdown_connections 2024-03-29T08:37:08.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:08.180+0000 7fb78bc00700 1 -- 172.21.15.37:0/742594004 wait complete. 2024-03-29T08:37:08.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:08.317+0000 7fe837c00700 1 -- 172.21.15.37:0/879922581 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe8300fe850 msgr2=0x7fe830106f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:37:08.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:08.317+0000 7fe837c00700 1 --2- 172.21.15.37:0/879922581 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe8300fe850 0x7fe830106f20 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fe81c004970 tx=0x7fe81c009e30 comp rx=0 tx=0).stop 2024-03-29T08:37:08.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:08.317+0000 7fe837c00700 1 -- 172.21.15.37:0/879922581 shutdown_connections 2024-03-29T08:37:08.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:08.317+0000 7fe837c00700 1 --2- 172.21.15.37:0/879922581 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe8300fe850 0x7fe830106f20 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:37:08.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:08.317+0000 7fe837c00700 1 --2- 172.21.15.37:0/879922581 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe8300fdeb0 0x7fe8300fe280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:37:08.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:08.317+0000 7fe837c00700 1 -- 172.21.15.37:0/879922581 >> 172.21.15.37:0/879922581 conn(0x7fe830072bc0 msgr2=0x7fe830072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:37:08.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:08.317+0000 7fe837c00700 1 -- 172.21.15.37:0/879922581 shutdown_connections 2024-03-29T08:37:08.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:08.317+0000 7fe837c00700 1 -- 172.21.15.37:0/879922581 wait complete. 2024-03-29T08:37:08.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:08.318+0000 7fe837c00700 1 Processor -- start 2024-03-29T08:37:08.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:08.318+0000 7fe837c00700 1 -- start start 2024-03-29T08:37:08.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:08.318+0000 7fe837c00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe8300fdeb0 0x7fe830115c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:37:08.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:08.318+0000 7fe837c00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe8300fe850 0x7fe8301161c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:37:08.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:08.319+0000 7fe837c00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe830116810 con 0x7fe8300fdeb0 2024-03-29T08:37:08.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:08.319+0000 7fe837c00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe83010fc20 con 0x7fe8300fe850 2024-03-29T08:37:08.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:08.319+0000 7fe835200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe8300fdeb0 0x7fe830115c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:37:08.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:08.319+0000 7fe835200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe8300fdeb0 0x7fe830115c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52774/0 (socket says 172.21.15.37:52774) 2024-03-29T08:37:08.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:08.319+0000 7fe835200700 1 -- 172.21.15.37:0/2268546450 learned_addr learned my addr 172.21.15.37:0/2268546450 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:37:08.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:08.319+0000 7fe835200700 1 -- 172.21.15.37:0/2268546450 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe8300fe850 msgr2=0x7fe8301161c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:37:08.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:08.319+0000 7fe835200700 1 --2- 172.21.15.37:0/2268546450 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe8300fe850 0x7fe8301161c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:37:08.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:08.319+0000 7fe835200700 1 -- 172.21.15.37:0/2268546450 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe81c005040 con 0x7fe8300fdeb0 2024-03-29T08:37:08.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:08.319+0000 7fe835200700 1 --2- 172.21.15.37:0/2268546450 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe8300fdeb0 0x7fe830115c80 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fe824002a50 tx=0x7fe8240046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:37:08.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:08.320+0000 7fe82cc00700 1 -- 172.21.15.37:0/2268546450 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe824007cc0 con 0x7fe8300fdeb0 2024-03-29T08:37:08.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:08.320+0000 7fe82cc00700 1 -- 172.21.15.37:0/2268546450 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fe82400f040 con 0x7fe8300fdeb0 2024-03-29T08:37:08.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:08.320+0000 7fe837c00700 1 -- 172.21.15.37:0/2268546450 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe83010ff00 con 0x7fe8300fdeb0 2024-03-29T08:37:08.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:08.320+0000 7fe82cc00700 1 -- 172.21.15.37:0/2268546450 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe82400da70 con 0x7fe8300fdeb0 2024-03-29T08:37:08.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:08.320+0000 7fe837c00700 1 -- 172.21.15.37:0/2268546450 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe830110440 con 0x7fe8300fdeb0 2024-03-29T08:37:08.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:08.321+0000 7fe837c00700 1 -- 172.21.15.37:0/2268546450 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8301069f0 con 0x7fe8300fdeb0 2024-03-29T08:37:08.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:08.322+0000 7fe82cc00700 1 -- 172.21.15.37:0/2268546450 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fe82400dc10 con 0x7fe8300fdeb0 2024-03-29T08:37:08.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:08.323+0000 7fe82cc00700 1 --2- 172.21.15.37:0/2268546450 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fe81806cc20 0x7fe81806f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:37:08.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:08.323+0000 7fe82cc00700 1 -- 172.21.15.37:0/2268546450 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fe82408b740 con 0x7fe8300fdeb0 2024-03-29T08:37:08.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:08.323+0000 7fe82fe00700 1 --2- 172.21.15.37:0/2268546450 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fe81806cc20 0x7fe81806f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:37:08.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:08.324+0000 7fe82fe00700 1 --2- 172.21.15.37:0/2268546450 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fe81806cc20 0x7fe81806f0d0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fe81c009fd0 tx=0x7fe81c009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:37:08.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:08.328+0000 7fe82cc00700 1 -- 172.21.15.37:0/2268546450 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fe824055700 con 0x7fe8300fdeb0 2024-03-29T08:37:08.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:08.627+0000 7fe837c00700 1 -- 172.21.15.37:0/2268546450 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fe830109e40 con 0x7fe8300fdeb0 2024-03-29T08:37:08.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:08.627+0000 7fe82cc00700 1 -- 172.21.15.37:0/2268546450 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+3583 (secure 0 0 0) 0x7fe824017e10 con 0x7fe8300fdeb0 2024-03-29T08:37:08.628 INFO:teuthology.orchestra.run.smithi037.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem is offline 2024-03-29T08:37:08.628 INFO:teuthology.orchestra.run.smithi037.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-03-29T08:37:08.629 INFO:teuthology.orchestra.run.smithi037.stdout: host smithi148 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/mon.smithi148/config 2024-03-29T08:37:08.629 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/podman: timeout after 895 seconds 2024-03-29T08:37:08.629 INFO:teuthology.orchestra.run.smithi037.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59 -e NODE_NAME=smithi148 -e CEPH_USE_RANDOM_NONCE=1 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4:/var/run/ceph:z -v /var/log/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4:/var/log/ceph:z -v /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpkx2rz2d4:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59 inventory --format=json-pretty --filter-for-batch 2024-03-29T08:37:08.629 INFO:teuthology.orchestra.run.smithi037.stdout:Traceback (most recent call last): 2024-03-29T08:37:08.629 INFO:teuthology.orchestra.run.smithi037.stdout: File "/usr/lib64/python3.6/runpy.py", line 193, in _run_module_as_main 2024-03-29T08:37:08.629 INFO:teuthology.orchestra.run.smithi037.stdout: "__main__", mod_spec) 2024-03-29T08:37:08.629 INFO:teuthology.orchestra.run.smithi037.stdout: File "/usr/lib64/python3.6/runpy.py", line 85, in _run_code 2024-03-29T08:37:08.629 INFO:teuthology.orchestra.run.smithi037.stdout: exec(code, run_globals) 2024-03-29T08:37:08.629 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 10889, in 2024-03-29T08:37:08.629 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 10877, in main 2024-03-29T08:37:08.629 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 2576, in _infer_config 2024-03-29T08:37:08.629 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 2492, in _infer_fsid 2024-03-29T08:37:08.629 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 2604, in _infer_image 2024-03-29T08:37:08.629 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 2479, in _validate_fsid 2024-03-29T08:37:08.629 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 7145, in command_ceph_volume 2024-03-29T08:37:08.629 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 2267, in call_throws 2024-03-29T08:37:08.630 INFO:teuthology.orchestra.run.smithi037.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59 -e NODE_NAME=smithi148 -e CEPH_USE_RANDOM_NONCE=1 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4:/var/run/ceph:z -v /var/log/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4:/var/log/ceph:z -v /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpkx2rz2d4:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59 inventory --format=json-pretty --filter-for-batch 2024-03-29T08:37:08.630 INFO:teuthology.orchestra.run.smithi037.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-03-29T08:37:08.630 INFO:teuthology.orchestra.run.smithi037.stdout: fs cephfs is degraded 2024-03-29T08:37:08.630 INFO:teuthology.orchestra.run.smithi037.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-03-29T08:37:08.630 INFO:teuthology.orchestra.run.smithi037.stdout: fs cephfs is offline because no MDS is active for it. 2024-03-29T08:37:08.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:08.630+0000 7fe837c00700 1 -- 172.21.15.37:0/2268546450 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fe81806cc20 msgr2=0x7fe81806f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:37:08.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:08.630+0000 7fe837c00700 1 --2- 172.21.15.37:0/2268546450 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fe81806cc20 0x7fe81806f0d0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fe81c009fd0 tx=0x7fe81c009e30 comp rx=0 tx=0).stop 2024-03-29T08:37:08.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:08.630+0000 7fe837c00700 1 -- 172.21.15.37:0/2268546450 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe8300fdeb0 msgr2=0x7fe830115c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:37:08.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:08.630+0000 7fe837c00700 1 --2- 172.21.15.37:0/2268546450 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe8300fdeb0 0x7fe830115c80 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fe824002a50 tx=0x7fe8240046b0 comp rx=0 tx=0).stop 2024-03-29T08:37:08.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:08.630+0000 7fe837c00700 1 -- 172.21.15.37:0/2268546450 shutdown_connections 2024-03-29T08:37:08.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:08.630+0000 7fe837c00700 1 --2- 172.21.15.37:0/2268546450 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fe81806cc20 0x7fe81806f0d0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:37:08.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:08.630+0000 7fe837c00700 1 --2- 172.21.15.37:0/2268546450 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe8300fdeb0 0x7fe830115c80 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:37:08.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:08.630+0000 7fe837c00700 1 --2- 172.21.15.37:0/2268546450 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe8300fe850 0x7fe8301161c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:37:08.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:08.630+0000 7fe837c00700 1 -- 172.21.15.37:0/2268546450 >> 172.21.15.37:0/2268546450 conn(0x7fe830072bc0 msgr2=0x7fe8300fd4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:37:08.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:08.630+0000 7fe837c00700 1 -- 172.21.15.37:0/2268546450 shutdown_connections 2024-03-29T08:37:08.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:08.630+0000 7fe837c00700 1 -- 172.21.15.37:0/2268546450 wait complete. 2024-03-29T08:37:08.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:37:08 smithi037 ceph-mon[142325]: from='client.35438 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:37:08.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:37:08 smithi037 ceph-mon[142325]: pgmap v788: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:37:08.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:37:08 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/2438146260' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T08:37:09.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:37:08 smithi148 ceph-mon[106692]: from='client.35438 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:37:09.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:37:08 smithi148 ceph-mon[106692]: pgmap v788: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:37:09.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:37:08 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/2438146260' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T08:37:09.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:37:09 smithi037 ceph-mon[142325]: from='client.35450 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:37:09.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:37:09 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/2268546450' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T08:37:10.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:37:09 smithi148 ceph-mon[106692]: from='client.35450 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:37:10.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:37:09 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/2268546450' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T08:37:10.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:37:10 smithi037 ceph-mon[142325]: pgmap v789: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:37:10.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:37:10 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:37:11.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:37:10 smithi148 ceph-mon[106692]: pgmap v789: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:37:11.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:37:10 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:37:12.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:37:12 smithi037 ceph-mon[142325]: pgmap v790: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:37:13.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:37:12 smithi148 ceph-mon[106692]: pgmap v790: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:37:14.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:37:14 smithi037 ceph-mon[142325]: pgmap v791: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:37:15.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:37:14 smithi148 ceph-mon[106692]: pgmap v791: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:37:16.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:37:16 smithi037 ceph-mon[142325]: pgmap v792: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:37:17.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:37:16 smithi148 ceph-mon[106692]: pgmap v792: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:37:18.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:37:18 smithi037 ceph-mon[142325]: pgmap v793: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:37:19.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:37:18 smithi148 ceph-mon[106692]: pgmap v793: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:37:20.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:37:20 smithi037 ceph-mon[142325]: pgmap v794: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:37:21.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:37:20 smithi148 ceph-mon[106692]: pgmap v794: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:37:22.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:37:22 smithi037 ceph-mon[142325]: pgmap v795: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:37:23.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:37:22 smithi148 ceph-mon[106692]: pgmap v795: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:37:24.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:37:24 smithi037 ceph-mon[142325]: pgmap v796: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:37:25.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:37:24 smithi148 ceph-mon[106692]: pgmap v796: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:37:25.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:37:25 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:37:26.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:37:25 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:37:26.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:37:26 smithi037 ceph-mon[142325]: pgmap v797: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:37:27.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:37:26 smithi148 ceph-mon[106692]: pgmap v797: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:37:28.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:37:28 smithi037 ceph-mon[142325]: pgmap v798: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:37:29.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:37:28 smithi148 ceph-mon[106692]: pgmap v798: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:37:30.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:37:30 smithi037 ceph-mon[142325]: pgmap v799: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:37:31.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:37:30 smithi148 ceph-mon[106692]: pgmap v799: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:37:32.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:37:32 smithi037 ceph-mon[142325]: pgmap v800: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:37:33.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:37:32 smithi148 ceph-mon[106692]: pgmap v800: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:37:34.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:37:34 smithi037 ceph-mon[142325]: pgmap v801: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:37:35.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:37:34 smithi148 ceph-mon[106692]: pgmap v801: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:37:36.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:37:36 smithi037 ceph-mon[142325]: pgmap v802: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:37:37.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:37:36 smithi148 ceph-mon[106692]: pgmap v802: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:37:38.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:38.786+0000 7fe56ce00700 1 -- 172.21.15.37:0/1724756790 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe568070c10 msgr2=0x7fe568109190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:37:38.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:38.786+0000 7fe56ce00700 1 --2- 172.21.15.37:0/1724756790 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe568070c10 0x7fe568109190 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fe554004970 tx=0x7fe554009e30 comp rx=0 tx=0).stop 2024-03-29T08:37:38.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:38.786+0000 7fe56ce00700 1 -- 172.21.15.37:0/1724756790 shutdown_connections 2024-03-29T08:37:38.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:38.786+0000 7fe56ce00700 1 --2- 172.21.15.37:0/1724756790 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe568070c10 0x7fe568109190 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:37:38.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:38.786+0000 7fe56ce00700 1 --2- 172.21.15.37:0/1724756790 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe568070300 0x7fe5680706d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:37:38.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:38.786+0000 7fe56ce00700 1 -- 172.21.15.37:0/1724756790 >> 172.21.15.37:0/1724756790 conn(0x7fe5680f8970 msgr2=0x7fe5680fad80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:37:38.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:38.786+0000 7fe56ce00700 1 -- 172.21.15.37:0/1724756790 shutdown_connections 2024-03-29T08:37:38.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:38.786+0000 7fe56ce00700 1 -- 172.21.15.37:0/1724756790 wait complete. 2024-03-29T08:37:38.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:38.787+0000 7fe56ce00700 1 Processor -- start 2024-03-29T08:37:38.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:38.787+0000 7fe56ce00700 1 -- start start 2024-03-29T08:37:38.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:38.788+0000 7fe56ce00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe568070300 0x7fe568115cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:37:38.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:38.788+0000 7fe56ce00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe568070c10 0x7fe568116200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:37:38.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:38.788+0000 7fe56ce00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe568116850 con 0x7fe568070c10 2024-03-29T08:37:38.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:38.788+0000 7fe56ce00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe56810fc20 con 0x7fe568070300 2024-03-29T08:37:38.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:38.788+0000 7fe565400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe568070c10 0x7fe568116200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:37:38.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:38.788+0000 7fe565400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe568070c10 0x7fe568116200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49416/0 (socket says 172.21.15.37:49416) 2024-03-29T08:37:38.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:38.788+0000 7fe565400700 1 -- 172.21.15.37:0/987773852 learned_addr learned my addr 172.21.15.37:0/987773852 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:37:38.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:38.788+0000 7fe565e00700 1 --2- 172.21.15.37:0/987773852 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe568070300 0x7fe568115cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:37:38.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:38.788+0000 7fe565400700 1 -- 172.21.15.37:0/987773852 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe568070300 msgr2=0x7fe568115cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:37:38.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:38.788+0000 7fe565400700 1 --2- 172.21.15.37:0/987773852 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe568070300 0x7fe568115cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:37:38.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:38.788+0000 7fe565400700 1 -- 172.21.15.37:0/987773852 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe554005040 con 0x7fe568070c10 2024-03-29T08:37:38.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:38.789+0000 7fe565400700 1 --2- 172.21.15.37:0/987773852 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe568070c10 0x7fe568116200 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fe554004f50 tx=0x7fe554009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:37:38.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:38.789+0000 7fe55e000700 1 -- 172.21.15.37:0/987773852 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe55400e040 con 0x7fe568070c10 2024-03-29T08:37:38.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:38.789+0000 7fe55e000700 1 -- 172.21.15.37:0/987773852 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fe554013420 con 0x7fe568070c10 2024-03-29T08:37:38.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:38.789+0000 7fe55e000700 1 -- 172.21.15.37:0/987773852 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe55401c630 con 0x7fe568070c10 2024-03-29T08:37:38.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:38.789+0000 7fe56ce00700 1 -- 172.21.15.37:0/987773852 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe56810fea0 con 0x7fe568070c10 2024-03-29T08:37:38.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:38.789+0000 7fe56ce00700 1 -- 172.21.15.37:0/987773852 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5681103e0 con 0x7fe568070c10 2024-03-29T08:37:38.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:38.791+0000 7fe56ce00700 1 -- 172.21.15.37:0/987773852 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe568071a40 con 0x7fe568070c10 2024-03-29T08:37:38.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:38.791+0000 7fe55e000700 1 -- 172.21.15.37:0/987773852 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fe554013590 con 0x7fe568070c10 2024-03-29T08:37:38.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:38.792+0000 7fe55e000700 1 --2- 172.21.15.37:0/987773852 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fe54c06cc30 0x7fe54c06f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:37:38.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:38.793+0000 7fe55e000700 1 -- 172.21.15.37:0/987773852 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fe55408d1c0 con 0x7fe568070c10 2024-03-29T08:37:38.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:38.793+0000 7fe565e00700 1 --2- 172.21.15.37:0/987773852 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fe54c06cc30 0x7fe54c06f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:37:38.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:38.793+0000 7fe565e00700 1 --2- 172.21.15.37:0/987773852 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fe54c06cc30 0x7fe54c06f0e0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fe558004f40 tx=0x7fe55800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:37:38.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:38.798+0000 7fe55e000700 1 -- 172.21.15.37:0/987773852 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fe55405ac90 con 0x7fe568070c10 2024-03-29T08:37:38.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:37:38 smithi037 ceph-mon[142325]: pgmap v803: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:37:39.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:39.030+0000 7fe56ce00700 1 -- 172.21.15.37:0/987773852 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe56810c050 con 0x7fe54c06cc30 2024-03-29T08:37:39.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:39.031+0000 7fe55e000700 1 -- 172.21.15.37:0/987773852 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fe56810c050 con 0x7fe54c06cc30 2024-03-29T08:37:39.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:39.033+0000 7fe56ce00700 1 -- 172.21.15.37:0/987773852 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fe54c06cc30 msgr2=0x7fe54c06f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:37:39.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:39.033+0000 7fe56ce00700 1 --2- 172.21.15.37:0/987773852 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fe54c06cc30 0x7fe54c06f0e0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fe558004f40 tx=0x7fe55800b040 comp rx=0 tx=0).stop 2024-03-29T08:37:39.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:39.033+0000 7fe56ce00700 1 -- 172.21.15.37:0/987773852 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe568070c10 msgr2=0x7fe568116200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:37:39.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:39.033+0000 7fe56ce00700 1 --2- 172.21.15.37:0/987773852 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe568070c10 0x7fe568116200 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fe554004f50 tx=0x7fe554009e30 comp rx=0 tx=0).stop 2024-03-29T08:37:39.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:39.034+0000 7fe56ce00700 1 -- 172.21.15.37:0/987773852 shutdown_connections 2024-03-29T08:37:39.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:39.034+0000 7fe56ce00700 1 --2- 172.21.15.37:0/987773852 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fe54c06cc30 0x7fe54c06f0e0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:37:39.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:39.034+0000 7fe56ce00700 1 --2- 172.21.15.37:0/987773852 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe568070300 0x7fe568115cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:37:39.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:39.034+0000 7fe56ce00700 1 --2- 172.21.15.37:0/987773852 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe568070c10 0x7fe568116200 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:37:39.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:39.034+0000 7fe56ce00700 1 -- 172.21.15.37:0/987773852 >> 172.21.15.37:0/987773852 conn(0x7fe5680f8970 msgr2=0x7fe5680f9de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:37:39.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:39.034+0000 7fe56ce00700 1 -- 172.21.15.37:0/987773852 shutdown_connections 2024-03-29T08:37:39.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:39.034+0000 7fe56ce00700 1 -- 172.21.15.37:0/987773852 wait complete. 2024-03-29T08:37:39.048 INFO:teuthology.orchestra.run.smithi037.stdout:true 2024-03-29T08:37:39.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:37:38 smithi148 ceph-mon[106692]: pgmap v803: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:37:39.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:39.171+0000 7f5bf8e00700 1 -- 172.21.15.37:0/2447971882 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5bf4102e00 msgr2=0x7f5bf41031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:37:39.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:39.171+0000 7f5bf8e00700 1 --2- 172.21.15.37:0/2447971882 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5bf4102e00 0x7f5bf41031d0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f5be4004940 tx=0x7f5be4009e30 comp rx=0 tx=0).stop 2024-03-29T08:37:39.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:39.172+0000 7f5bf8e00700 1 -- 172.21.15.37:0/2447971882 shutdown_connections 2024-03-29T08:37:39.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:39.172+0000 7f5bf8e00700 1 --2- 172.21.15.37:0/2447971882 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5bf41037a0 0x7f5bf40fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:37:39.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:39.172+0000 7f5bf8e00700 1 --2- 172.21.15.37:0/2447971882 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5bf4102e00 0x7f5bf41031d0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:37:39.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:39.172+0000 7f5bf8e00700 1 -- 172.21.15.37:0/2447971882 >> 172.21.15.37:0/2447971882 conn(0x7f5bf40f89b0 msgr2=0x7f5bf40fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:37:39.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:39.172+0000 7f5bf8e00700 1 -- 172.21.15.37:0/2447971882 shutdown_connections 2024-03-29T08:37:39.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:39.172+0000 7f5bf8e00700 1 -- 172.21.15.37:0/2447971882 wait complete. 2024-03-29T08:37:39.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:39.173+0000 7f5bf8e00700 1 Processor -- start 2024-03-29T08:37:39.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:39.173+0000 7f5bf8e00700 1 -- start start 2024-03-29T08:37:39.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:39.174+0000 7f5bf8e00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5bf4102e00 0x7f5bf4113b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:37:39.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:39.174+0000 7f5bf8e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5bf41037a0 0x7f5bf4114060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:37:39.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:39.174+0000 7f5bf8e00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bf41146b0 con 0x7f5bf41037a0 2024-03-29T08:37:39.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:39.174+0000 7f5bf8e00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bf410daa0 con 0x7f5bf4102e00 2024-03-29T08:37:39.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:39.174+0000 7f5bf1400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5bf41037a0 0x7f5bf4114060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:37:39.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:39.174+0000 7f5bf1400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5bf41037a0 0x7f5bf4114060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49438/0 (socket says 172.21.15.37:49438) 2024-03-29T08:37:39.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:39.174+0000 7f5bf1400700 1 -- 172.21.15.37:0/2291233281 learned_addr learned my addr 172.21.15.37:0/2291233281 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:37:39.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:39.174+0000 7f5bf1e00700 1 --2- 172.21.15.37:0/2291233281 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5bf4102e00 0x7f5bf4113b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:37:39.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:39.174+0000 7f5bf1400700 1 -- 172.21.15.37:0/2291233281 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5bf4102e00 msgr2=0x7f5bf4113b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:37:39.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:39.174+0000 7f5bf1400700 1 --2- 172.21.15.37:0/2291233281 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5bf4102e00 0x7f5bf4113b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:37:39.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:39.174+0000 7f5bf1400700 1 -- 172.21.15.37:0/2291233281 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5be4005040 con 0x7f5bf41037a0 2024-03-29T08:37:39.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:39.175+0000 7f5bf1e00700 1 --2- 172.21.15.37:0/2291233281 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5bf4102e00 0x7f5bf4113b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:37:39.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:39.175+0000 7f5bf1400700 1 --2- 172.21.15.37:0/2291233281 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5bf41037a0 0x7f5bf4114060 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f5be0002a50 tx=0x7f5be0004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:37:39.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:39.175+0000 7f5bea000700 1 -- 172.21.15.37:0/2291233281 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5be0007cc0 con 0x7f5bf41037a0 2024-03-29T08:37:39.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:39.175+0000 7f5bea000700 1 -- 172.21.15.37:0/2291233281 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f5be000f040 con 0x7f5bf41037a0 2024-03-29T08:37:39.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:39.175+0000 7f5bf8e00700 1 -- 172.21.15.37:0/2291233281 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bf410dd80 con 0x7f5bf41037a0 2024-03-29T08:37:39.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:39.175+0000 7f5bea000700 1 -- 172.21.15.37:0/2291233281 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5be000d9a0 con 0x7f5bf41037a0 2024-03-29T08:37:39.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:39.175+0000 7f5bf8e00700 1 -- 172.21.15.37:0/2291233281 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bf410e340 con 0x7f5bf41037a0 2024-03-29T08:37:39.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:39.176+0000 7f5bf8e00700 1 -- 172.21.15.37:0/2291233281 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5bf40fe550 con 0x7f5bf41037a0 2024-03-29T08:37:39.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:39.177+0000 7f5bea000700 1 -- 172.21.15.37:0/2291233281 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f5be000dc10 con 0x7f5bf41037a0 2024-03-29T08:37:39.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:39.178+0000 7f5bea000700 1 --2- 172.21.15.37:0/2291233281 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f5bd806cc30 0x7f5bd806f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:37:39.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:39.178+0000 7f5bea000700 1 -- 172.21.15.37:0/2291233281 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f5be008be30 con 0x7f5bf41037a0 2024-03-29T08:37:39.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:39.178+0000 7f5bf1e00700 1 --2- 172.21.15.37:0/2291233281 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f5bd806cc30 0x7f5bd806f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:37:39.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:39.179+0000 7f5bf1e00700 1 --2- 172.21.15.37:0/2291233281 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f5bd806cc30 0x7f5bd806f0e0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f5be400b3b0 tx=0x7f5be4009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:37:39.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:39.183+0000 7f5bea000700 1 -- 172.21.15.37:0/2291233281 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f5be0059880 con 0x7f5bf41037a0 2024-03-29T08:37:39.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:39.414+0000 7f5bf8e00700 1 -- 172.21.15.37:0/2291233281 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5bf4109e60 con 0x7f5bd806cc30 2024-03-29T08:37:39.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:39.415+0000 7f5bea000700 1 -- 172.21.15.37:0/2291233281 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f5be4008e30 con 0x7f5bd806cc30 2024-03-29T08:37:39.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:39.418+0000 7f5bf8e00700 1 -- 172.21.15.37:0/2291233281 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f5bd806cc30 msgr2=0x7f5bd806f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:37:39.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:39.418+0000 7f5bf8e00700 1 --2- 172.21.15.37:0/2291233281 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f5bd806cc30 0x7f5bd806f0e0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f5be400b3b0 tx=0x7f5be4009e30 comp rx=0 tx=0).stop 2024-03-29T08:37:39.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:39.418+0000 7f5bf8e00700 1 -- 172.21.15.37:0/2291233281 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5bf41037a0 msgr2=0x7f5bf4114060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:37:39.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:39.418+0000 7f5bf8e00700 1 --2- 172.21.15.37:0/2291233281 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5bf41037a0 0x7f5bf4114060 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f5be0002a50 tx=0x7f5be0004600 comp rx=0 tx=0).stop 2024-03-29T08:37:39.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:39.418+0000 7f5bf8e00700 1 -- 172.21.15.37:0/2291233281 shutdown_connections 2024-03-29T08:37:39.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:39.418+0000 7f5bf8e00700 1 --2- 172.21.15.37:0/2291233281 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f5bd806cc30 0x7f5bd806f0e0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:37:39.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:39.418+0000 7f5bf8e00700 1 --2- 172.21.15.37:0/2291233281 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5bf4102e00 0x7f5bf4113b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:37:39.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:39.418+0000 7f5bf8e00700 1 --2- 172.21.15.37:0/2291233281 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5bf41037a0 0x7f5bf4114060 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:37:39.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:39.418+0000 7f5bf8e00700 1 -- 172.21.15.37:0/2291233281 >> 172.21.15.37:0/2291233281 conn(0x7f5bf40f89b0 msgr2=0x7f5bf40f9a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:37:39.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:39.418+0000 7f5bf8e00700 1 -- 172.21.15.37:0/2291233281 shutdown_connections 2024-03-29T08:37:39.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:39.418+0000 7f5bf8e00700 1 -- 172.21.15.37:0/2291233281 wait complete. 2024-03-29T08:37:39.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:39.555+0000 7f8271c00700 1 -- 172.21.15.37:0/3865524400 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f826c0fe830 msgr2=0x7f826c106f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:37:39.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:39.555+0000 7f8271c00700 1 --2- 172.21.15.37:0/3865524400 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f826c0fe830 0x7f826c106f00 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f8258004940 tx=0x7f8258009e30 comp rx=0 tx=0).stop 2024-03-29T08:37:39.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:39.555+0000 7f8271c00700 1 -- 172.21.15.37:0/3865524400 shutdown_connections 2024-03-29T08:37:39.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:39.555+0000 7f8271c00700 1 --2- 172.21.15.37:0/3865524400 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f826c0fe830 0x7f826c106f00 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:37:39.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:39.555+0000 7f8271c00700 1 --2- 172.21.15.37:0/3865524400 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f826c0fde90 0x7f826c0fe260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:37:39.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:39.555+0000 7f8271c00700 1 -- 172.21.15.37:0/3865524400 >> 172.21.15.37:0/3865524400 conn(0x7f826c072bc0 msgr2=0x7f826c072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:37:39.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:39.555+0000 7f8271c00700 1 -- 172.21.15.37:0/3865524400 shutdown_connections 2024-03-29T08:37:39.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:39.555+0000 7f8271c00700 1 -- 172.21.15.37:0/3865524400 wait complete. 2024-03-29T08:37:39.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:39.556+0000 7f8271c00700 1 Processor -- start 2024-03-29T08:37:39.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:39.556+0000 7f8271c00700 1 -- start start 2024-03-29T08:37:39.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:39.557+0000 7f8271c00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f826c0fde90 0x7f826c115d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:37:39.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:39.557+0000 7f8271c00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f826c0fe830 0x7f826c116250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:37:39.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:39.557+0000 7f8271c00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f826c1168a0 con 0x7f826c0fe830 2024-03-29T08:37:39.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:39.557+0000 7f8271c00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f826c10fc70 con 0x7f826c0fde90 2024-03-29T08:37:39.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:39.557+0000 7f826a000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f826c0fe830 0x7f826c116250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:37:39.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:39.557+0000 7f826a000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f826c0fe830 0x7f826c116250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49460/0 (socket says 172.21.15.37:49460) 2024-03-29T08:37:39.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:39.557+0000 7f826a000700 1 -- 172.21.15.37:0/3109441510 learned_addr learned my addr 172.21.15.37:0/3109441510 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:37:39.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:39.557+0000 7f826aa00700 1 --2- 172.21.15.37:0/3109441510 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f826c0fde90 0x7f826c115d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:37:39.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:39.558+0000 7f826a000700 1 -- 172.21.15.37:0/3109441510 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f826c0fde90 msgr2=0x7f826c115d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:37:39.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:39.558+0000 7f826a000700 1 --2- 172.21.15.37:0/3109441510 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f826c0fde90 0x7f826c115d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:37:39.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:39.558+0000 7f826a000700 1 -- 172.21.15.37:0/3109441510 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8258005040 con 0x7f826c0fe830 2024-03-29T08:37:39.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:39.558+0000 7f826a000700 1 --2- 172.21.15.37:0/3109441510 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f826c0fe830 0x7f826c116250 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f8258000c00 tx=0x7f8258009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:37:39.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:39.558+0000 7f8262a00700 1 -- 172.21.15.37:0/3109441510 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f825800e040 con 0x7f826c0fe830 2024-03-29T08:37:39.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:39.558+0000 7f8262a00700 1 -- 172.21.15.37:0/3109441510 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f825800bed0 con 0x7f826c0fe830 2024-03-29T08:37:39.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:39.558+0000 7f8271c00700 1 -- 172.21.15.37:0/3109441510 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f826c10fef0 con 0x7f826c0fe830 2024-03-29T08:37:39.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:39.558+0000 7f8262a00700 1 -- 172.21.15.37:0/3109441510 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f825801c3f0 con 0x7f826c0fe830 2024-03-29T08:37:39.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:39.558+0000 7f8271c00700 1 -- 172.21.15.37:0/3109441510 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f826c110430 con 0x7f826c0fe830 2024-03-29T08:37:39.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:39.559+0000 7f8271c00700 1 -- 172.21.15.37:0/3109441510 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f826c106a20 con 0x7f826c0fe830 2024-03-29T08:37:39.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:39.560+0000 7f8262a00700 1 -- 172.21.15.37:0/3109441510 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f8258013420 con 0x7f826c0fe830 2024-03-29T08:37:39.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:39.561+0000 7f8262a00700 1 --2- 172.21.15.37:0/3109441510 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f825006cc30 0x7f825006f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:37:39.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:39.561+0000 7f8262a00700 1 -- 172.21.15.37:0/3109441510 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f825808d400 con 0x7f826c0fe830 2024-03-29T08:37:39.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:39.561+0000 7f826aa00700 1 --2- 172.21.15.37:0/3109441510 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f825006cc30 0x7f825006f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:37:39.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:39.562+0000 7f826aa00700 1 --2- 172.21.15.37:0/3109441510 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f825006cc30 0x7f825006f0e0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f825c004f40 tx=0x7f825c00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:37:39.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:39.566+0000 7f8262a00700 1 -- 172.21.15.37:0/3109441510 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f825805aed0 con 0x7f826c0fe830 2024-03-29T08:37:39.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:39.789+0000 7f8271c00700 1 -- 172.21.15.37:0/3109441510 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f826c111400 con 0x7f825006cc30 2024-03-29T08:37:39.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:39.799+0000 7f8262a00700 1 -- 172.21.15.37:0/3109441510 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f826c111400 con 0x7f825006cc30 2024-03-29T08:37:39.799 INFO:teuthology.orchestra.run.smithi037.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-29T08:37:39.799 INFO:teuthology.orchestra.run.smithi037.stdout:alertmanager.smithi037 smithi037 *:9093,9094 running (27m) 4m ago 40m 14.3M - 0.25.0 c8568f914cd2 2f9f9746ed0f 2024-03-29T08:37:39.799 INFO:teuthology.orchestra.run.smithi037.stdout:crash.smithi037 smithi037 running (24m) 4m ago 40m 6647k - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 178d416a3186 2024-03-29T08:37:39.799 INFO:teuthology.orchestra.run.smithi037.stdout:crash.smithi148 smithi148 running (24m) 4m ago 39m 6656k - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 23666989d58f 2024-03-29T08:37:39.799 INFO:teuthology.orchestra.run.smithi037.stdout:grafana.smithi037 smithi037 *:3000 running (27m) 4m ago 40m 74.5M - 9.4.7 954c08fa6188 6b976fe49952 2024-03-29T08:37:39.799 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi037.iifsmh smithi037 running (19m) 4m ago 33m 16.4M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 2d97f562cc5b 2024-03-29T08:37:39.799 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi037.mmwgxe smithi037 running (20m) 4m ago 34m 16.7M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff ec37037f46e8 2024-03-29T08:37:39.799 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi148.qukpjh smithi148 running (4m) 4m ago 33m 10.7M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 3257219bc046 2024-03-29T08:37:39.799 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi148.sqteih smithi148 running (19m) 4m ago 34m 16.3M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 44c15c77037a 2024-03-29T08:37:39.799 INFO:teuthology.orchestra.run.smithi037.stdout:mgr.smithi037.wjikii smithi037 *:8443,9283,8765 running (29m) 4m ago 42m 510M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff c5a1417b9417 2024-03-29T08:37:39.799 INFO:teuthology.orchestra.run.smithi037.stdout:mgr.smithi148.onhgvm smithi148 *:8443,9283,8765 running (28m) 4m ago 39m 456M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff fcfdbce062bd 2024-03-29T08:37:39.800 INFO:teuthology.orchestra.run.smithi037.stdout:mon.smithi037 smithi037 running (25m) 4m ago 42m 70.2M 2048M 18.2.2-1015-gd0e392d9 1cc2f45c17ff eea2060783af 2024-03-29T08:37:39.800 INFO:teuthology.orchestra.run.smithi037.stdout:mon.smithi148 smithi148 running (25m) 4m ago 39m 50.4M 2048M 18.2.2-1015-gd0e392d9 1cc2f45c17ff ad4b2c5748be 2024-03-29T08:37:39.800 INFO:teuthology.orchestra.run.smithi037.stdout:node-exporter.smithi037 smithi037 *:9100 running (28m) 4m ago 40m 8514k - 1.5.0 0da6a335fe13 26b8ae9cb878 2024-03-29T08:37:39.800 INFO:teuthology.orchestra.run.smithi037.stdout:node-exporter.smithi148 smithi148 *:9100 running (28m) 4m ago 39m 8464k - 1.5.0 0da6a335fe13 98414e4e64ed 2024-03-29T08:37:39.800 INFO:teuthology.orchestra.run.smithi037.stdout:osd.0 smithi037 running (23m) 4m ago 38m 108M 1619M 18.2.2-1015-gd0e392d9 1cc2f45c17ff c0c584f6245c 2024-03-29T08:37:39.800 INFO:teuthology.orchestra.run.smithi037.stdout:osd.1 smithi037 running (23m) 4m ago 37m 104M 1619M 18.2.2-1015-gd0e392d9 1cc2f45c17ff a4a2ff4cc83d 2024-03-29T08:37:39.800 INFO:teuthology.orchestra.run.smithi037.stdout:osd.2 smithi037 running (22m) 4m ago 37m 85.9M 1619M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 8b1bc71aa79e 2024-03-29T08:37:39.800 INFO:teuthology.orchestra.run.smithi037.stdout:osd.3 smithi148 running (21m) 4m ago 36m 109M 2642M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 1c656ab8ff86 2024-03-29T08:37:39.800 INFO:teuthology.orchestra.run.smithi037.stdout:osd.4 smithi148 running (21m) 4m ago 36m 101M 2642M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 966a9de11d22 2024-03-29T08:37:39.800 INFO:teuthology.orchestra.run.smithi037.stdout:osd.5 smithi148 running (20m) 4m ago 35m 89.4M 2642M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 474c4f9212ce 2024-03-29T08:37:39.800 INFO:teuthology.orchestra.run.smithi037.stdout:prometheus.smithi037 smithi037 *:9095 running (27m) 4m ago 39m 61.0M - 2.43.0 a07b618ecd1d 30c0223ec589 2024-03-29T08:37:39.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:39.801+0000 7f8271c00700 1 -- 172.21.15.37:0/3109441510 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f825006cc30 msgr2=0x7f825006f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:37:39.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:39.801+0000 7f8271c00700 1 --2- 172.21.15.37:0/3109441510 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f825006cc30 0x7f825006f0e0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f825c004f40 tx=0x7f825c00b040 comp rx=0 tx=0).stop 2024-03-29T08:37:39.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:39.801+0000 7f8271c00700 1 -- 172.21.15.37:0/3109441510 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f826c0fe830 msgr2=0x7f826c116250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:37:39.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:39.801+0000 7f8271c00700 1 --2- 172.21.15.37:0/3109441510 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f826c0fe830 0x7f826c116250 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f8258000c00 tx=0x7f8258009e30 comp rx=0 tx=0).stop 2024-03-29T08:37:39.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:39.801+0000 7f8271c00700 1 -- 172.21.15.37:0/3109441510 shutdown_connections 2024-03-29T08:37:39.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:39.801+0000 7f8271c00700 1 --2- 172.21.15.37:0/3109441510 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f825006cc30 0x7f825006f0e0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:37:39.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:39.801+0000 7f8271c00700 1 --2- 172.21.15.37:0/3109441510 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f826c0fde90 0x7f826c115d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:37:39.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:39.801+0000 7f8271c00700 1 --2- 172.21.15.37:0/3109441510 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f826c0fe830 0x7f826c116250 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:37:39.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:39.801+0000 7f8271c00700 1 -- 172.21.15.37:0/3109441510 >> 172.21.15.37:0/3109441510 conn(0x7f826c072bc0 msgr2=0x7f826c0fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:37:39.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:39.801+0000 7f8271c00700 1 -- 172.21.15.37:0/3109441510 shutdown_connections 2024-03-29T08:37:39.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:39.801+0000 7f8271c00700 1 -- 172.21.15.37:0/3109441510 wait complete. 2024-03-29T08:37:39.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:39.938+0000 7f133fe00700 1 -- 172.21.15.37:0/1452675345 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f13380fdeb0 msgr2=0x7f13380fe280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:37:39.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:39.938+0000 7f133fe00700 1 --2- 172.21.15.37:0/1452675345 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f13380fdeb0 0x7f13380fe280 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f132c004970 tx=0x7f132c009e30 comp rx=0 tx=0).stop 2024-03-29T08:37:39.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:39.938+0000 7f133fe00700 1 -- 172.21.15.37:0/1452675345 shutdown_connections 2024-03-29T08:37:39.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:39.938+0000 7f133fe00700 1 --2- 172.21.15.37:0/1452675345 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f13380fe850 0x7f1338106f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:37:39.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:39.938+0000 7f133fe00700 1 --2- 172.21.15.37:0/1452675345 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f13380fdeb0 0x7f13380fe280 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:37:39.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:39.938+0000 7f133fe00700 1 -- 172.21.15.37:0/1452675345 >> 172.21.15.37:0/1452675345 conn(0x7f1338072bc0 msgr2=0x7f1338072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:37:39.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:39.938+0000 7f133fe00700 1 -- 172.21.15.37:0/1452675345 shutdown_connections 2024-03-29T08:37:39.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:39.938+0000 7f133fe00700 1 -- 172.21.15.37:0/1452675345 wait complete. 2024-03-29T08:37:39.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:39.939+0000 7f133fe00700 1 Processor -- start 2024-03-29T08:37:39.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:39.939+0000 7f133fe00700 1 -- start start 2024-03-29T08:37:39.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:39.939+0000 7f133fe00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f13380fdeb0 0x7f13380fbd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:37:39.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:39.939+0000 7f133fe00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f13380fe850 0x7f13380fc2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:37:39.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:39.939+0000 7f133fe00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13380fc840 con 0x7f13380fe850 2024-03-29T08:37:39.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:39.939+0000 7f133fe00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13380fc980 con 0x7f13380fdeb0 2024-03-29T08:37:39.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:39.940+0000 7f133ca00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f13380fe850 0x7f13380fc2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:37:39.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:39.940+0000 7f133ca00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f13380fe850 0x7f13380fc2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45510/0 (socket says 172.21.15.37:45510) 2024-03-29T08:37:39.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:39.940+0000 7f133ca00700 1 -- 172.21.15.37:0/1127496647 learned_addr learned my addr 172.21.15.37:0/1127496647 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:37:39.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:39.940+0000 7f133d400700 1 --2- 172.21.15.37:0/1127496647 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f13380fdeb0 0x7f13380fbd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:37:39.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:39.940+0000 7f133ca00700 1 -- 172.21.15.37:0/1127496647 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f13380fdeb0 msgr2=0x7f13380fbd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:37:39.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:39.940+0000 7f133ca00700 1 --2- 172.21.15.37:0/1127496647 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f13380fdeb0 0x7f13380fbd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:37:39.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:39.940+0000 7f133ca00700 1 -- 172.21.15.37:0/1127496647 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f132c005040 con 0x7f13380fe850 2024-03-29T08:37:39.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:39.940+0000 7f133d400700 1 --2- 172.21.15.37:0/1127496647 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f13380fdeb0 0x7f13380fbd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:37:39.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:39.940+0000 7f133ca00700 1 --2- 172.21.15.37:0/1127496647 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f13380fe850 0x7f13380fc2b0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f132400eb40 tx=0x7f132400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:37:39.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:39.941+0000 7f1335600700 1 -- 172.21.15.37:0/1127496647 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1324007cc0 con 0x7f13380fe850 2024-03-29T08:37:39.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:39.941+0000 7f1335600700 1 -- 172.21.15.37:0/1127496647 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f132400f040 con 0x7f13380fe850 2024-03-29T08:37:39.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:39.941+0000 7f1335600700 1 -- 172.21.15.37:0/1127496647 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1324009570 con 0x7f13380fe850 2024-03-29T08:37:39.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:39.941+0000 7f133fe00700 1 -- 172.21.15.37:0/1127496647 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1338119d70 con 0x7f13380fe850 2024-03-29T08:37:39.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:39.941+0000 7f133fe00700 1 -- 172.21.15.37:0/1127496647 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f133811a170 con 0x7f13380fe850 2024-03-29T08:37:39.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:39.942+0000 7f133fe00700 1 -- 172.21.15.37:0/1127496647 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1338065e10 con 0x7f13380fe850 2024-03-29T08:37:39.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:39.943+0000 7f1335600700 1 -- 172.21.15.37:0/1127496647 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f13240096d0 con 0x7f13380fe850 2024-03-29T08:37:39.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:39.944+0000 7f1335600700 1 --2- 172.21.15.37:0/1127496647 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f132006cc30 0x7f132006f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:37:39.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:39.944+0000 7f1335600700 1 -- 172.21.15.37:0/1127496647 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f132408bc20 con 0x7f13380fe850 2024-03-29T08:37:39.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:39.944+0000 7f133d400700 1 --2- 172.21.15.37:0/1127496647 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f132006cc30 0x7f132006f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:37:39.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:39.945+0000 7f133d400700 1 --2- 172.21.15.37:0/1127496647 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f132006cc30 0x7f132006f0e0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f132c009fd0 tx=0x7f132c009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:37:39.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:39.949+0000 7f1335600700 1 -- 172.21.15.37:0/1127496647 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f1324055d10 con 0x7f13380fe850 2024-03-29T08:37:40.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:40.249+0000 7f133fe00700 1 -- 172.21.15.37:0/1127496647 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f133804bed0 con 0x7f13380fe850 2024-03-29T08:37:40.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:40.249+0000 7f1335600700 1 -- 172.21.15.37:0/1127496647 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7f1324059330 con 0x7f13380fe850 2024-03-29T08:37:40.249 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-29T08:37:40.250 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": { 2024-03-29T08:37:40.250 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 2 2024-03-29T08:37:40.250 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:37:40.250 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": { 2024-03-29T08:37:40.250 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 2 2024-03-29T08:37:40.250 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:37:40.250 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": { 2024-03-29T08:37:40.250 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 6 2024-03-29T08:37:40.250 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:37:40.250 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": { 2024-03-29T08:37:40.250 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 4 2024-03-29T08:37:40.250 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:37:40.250 INFO:teuthology.orchestra.run.smithi037.stdout: "overall": { 2024-03-29T08:37:40.250 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 14 2024-03-29T08:37:40.251 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-29T08:37:40.251 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-29T08:37:40.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:40.252+0000 7f133fe00700 1 -- 172.21.15.37:0/1127496647 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f132006cc30 msgr2=0x7f132006f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:37:40.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:40.252+0000 7f133fe00700 1 --2- 172.21.15.37:0/1127496647 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f132006cc30 0x7f132006f0e0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f132c009fd0 tx=0x7f132c009e30 comp rx=0 tx=0).stop 2024-03-29T08:37:40.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:40.252+0000 7f133fe00700 1 -- 172.21.15.37:0/1127496647 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f13380fe850 msgr2=0x7f13380fc2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:37:40.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:40.252+0000 7f133fe00700 1 --2- 172.21.15.37:0/1127496647 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f13380fe850 0x7f13380fc2b0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f132400eb40 tx=0x7f132400b040 comp rx=0 tx=0).stop 2024-03-29T08:37:40.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:40.252+0000 7f133fe00700 1 -- 172.21.15.37:0/1127496647 shutdown_connections 2024-03-29T08:37:40.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:40.252+0000 7f133fe00700 1 --2- 172.21.15.37:0/1127496647 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f132006cc30 0x7f132006f0e0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:37:40.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:40.252+0000 7f133fe00700 1 --2- 172.21.15.37:0/1127496647 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f13380fdeb0 0x7f13380fbd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:37:40.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:40.252+0000 7f133fe00700 1 --2- 172.21.15.37:0/1127496647 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f13380fe850 0x7f13380fc2b0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:37:40.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:40.252+0000 7f133fe00700 1 -- 172.21.15.37:0/1127496647 >> 172.21.15.37:0/1127496647 conn(0x7f1338072bc0 msgr2=0x7f13380ff410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:37:40.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:40.252+0000 7f133fe00700 1 -- 172.21.15.37:0/1127496647 shutdown_connections 2024-03-29T08:37:40.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:40.252+0000 7f133fe00700 1 -- 172.21.15.37:0/1127496647 wait complete. 2024-03-29T08:37:40.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:40.389+0000 7fd5c6200700 1 -- 172.21.15.37:0/3764588640 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd5c00fde90 msgr2=0x7fd5c00fe260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:37:40.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:40.389+0000 7fd5c6200700 1 --2- 172.21.15.37:0/3764588640 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd5c00fde90 0x7fd5c00fe260 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fd5b0004940 tx=0x7fd5b0009e30 comp rx=0 tx=0).stop 2024-03-29T08:37:40.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:40.389+0000 7fd5c6200700 1 -- 172.21.15.37:0/3764588640 shutdown_connections 2024-03-29T08:37:40.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:40.389+0000 7fd5c6200700 1 --2- 172.21.15.37:0/3764588640 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd5c00fe830 0x7fd5c0106f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:37:40.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:40.389+0000 7fd5c6200700 1 --2- 172.21.15.37:0/3764588640 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd5c00fde90 0x7fd5c00fe260 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:37:40.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:40.389+0000 7fd5c6200700 1 -- 172.21.15.37:0/3764588640 >> 172.21.15.37:0/3764588640 conn(0x7fd5c0072bc0 msgr2=0x7fd5c0072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:37:40.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:40.389+0000 7fd5c6200700 1 -- 172.21.15.37:0/3764588640 shutdown_connections 2024-03-29T08:37:40.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:40.389+0000 7fd5c6200700 1 -- 172.21.15.37:0/3764588640 wait complete. 2024-03-29T08:37:40.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:40.390+0000 7fd5c6200700 1 Processor -- start 2024-03-29T08:37:40.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:40.390+0000 7fd5c6200700 1 -- start start 2024-03-29T08:37:40.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:40.390+0000 7fd5c6200700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd5c00fde90 0x7fd5c0115cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:37:40.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:40.390+0000 7fd5c6200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd5c00fe830 0x7fd5c0116200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:37:40.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:40.390+0000 7fd5c6200700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5c0116850 con 0x7fd5c00fe830 2024-03-29T08:37:40.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:40.391+0000 7fd5c6200700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5c010fcb0 con 0x7fd5c00fde90 2024-03-29T08:37:40.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:40.391+0000 7fd5bea00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd5c00fe830 0x7fd5c0116200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:37:40.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:40.391+0000 7fd5bea00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd5c00fe830 0x7fd5c0116200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45538/0 (socket says 172.21.15.37:45538) 2024-03-29T08:37:40.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:40.391+0000 7fd5bea00700 1 -- 172.21.15.37:0/2559152153 learned_addr learned my addr 172.21.15.37:0/2559152153 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:37:40.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:40.391+0000 7fd5bf400700 1 --2- 172.21.15.37:0/2559152153 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd5c00fde90 0x7fd5c0115cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:37:40.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:40.392+0000 7fd5bea00700 1 -- 172.21.15.37:0/2559152153 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd5c00fde90 msgr2=0x7fd5c0115cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:37:40.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:40.392+0000 7fd5bea00700 1 --2- 172.21.15.37:0/2559152153 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd5c00fde90 0x7fd5c0115cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:37:40.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:40.392+0000 7fd5bea00700 1 -- 172.21.15.37:0/2559152153 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5b0005040 con 0x7fd5c00fe830 2024-03-29T08:37:40.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:40.392+0000 7fd5bf400700 1 --2- 172.21.15.37:0/2559152153 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd5c00fde90 0x7fd5c0115cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T08:37:40.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:40.392+0000 7fd5bea00700 1 --2- 172.21.15.37:0/2559152153 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd5c00fe830 0x7fd5c0116200 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fd5ac002a50 tx=0x7fd5ac0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:37:40.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:40.393+0000 7fd5b7400700 1 -- 172.21.15.37:0/2559152153 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd5ac007cc0 con 0x7fd5c00fe830 2024-03-29T08:37:40.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:40.393+0000 7fd5c6200700 1 -- 172.21.15.37:0/2559152153 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5c010ff90 con 0x7fd5c00fe830 2024-03-29T08:37:40.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:40.393+0000 7fd5b7400700 1 -- 172.21.15.37:0/2559152153 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd5ac00f040 con 0x7fd5c00fe830 2024-03-29T08:37:40.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:40.393+0000 7fd5b7400700 1 -- 172.21.15.37:0/2559152153 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd5ac00d9c0 con 0x7fd5c00fe830 2024-03-29T08:37:40.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:40.393+0000 7fd5c6200700 1 -- 172.21.15.37:0/2559152153 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5c01104d0 con 0x7fd5c00fe830 2024-03-29T08:37:40.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:40.394+0000 7fd5c6200700 1 -- 172.21.15.37:0/2559152153 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5c0106a20 con 0x7fd5c00fe830 2024-03-29T08:37:40.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:40.395+0000 7fd5b7400700 1 -- 172.21.15.37:0/2559152153 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fd5ac00dc00 con 0x7fd5c00fe830 2024-03-29T08:37:40.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:40.396+0000 7fd5b7400700 1 --2- 172.21.15.37:0/2559152153 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fd5a406cc30 0x7fd5a406f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:37:40.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:40.396+0000 7fd5b7400700 1 -- 172.21.15.37:0/2559152153 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fd5ac08b7a0 con 0x7fd5c00fe830 2024-03-29T08:37:40.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:40.396+0000 7fd5bf400700 1 --2- 172.21.15.37:0/2559152153 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fd5a406cc30 0x7fd5a406f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:37:40.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:40.397+0000 7fd5bf400700 1 --2- 172.21.15.37:0/2559152153 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fd5a406cc30 0x7fd5a406f0e0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fd5b000b3b0 tx=0x7fd5b0009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:37:40.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:40.401+0000 7fd5b7400700 1 -- 172.21.15.37:0/2559152153 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fd5ac0556e0 con 0x7fd5c00fe830 2024-03-29T08:37:40.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:40.662+0000 7fd5c6200700 1 -- 172.21.15.37:0/2559152153 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fd5c005f650 con 0x7fd5c00fe830 2024-03-29T08:37:40.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:40.662+0000 7fd5b7400700 1 -- 172.21.15.37:0/2559152153 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 30 v30) v1 ==== 76+0+1841 (secure 0 0 0) 0x7fd5ac058d00 con 0x7fd5c00fe830 2024-03-29T08:37:40.663 INFO:teuthology.orchestra.run.smithi037.stdout:e30 2024-03-29T08:37:40.663 INFO:teuthology.orchestra.run.smithi037.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-29T08:37:40.663 INFO:teuthology.orchestra.run.smithi037.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T08:37:40.663 INFO:teuthology.orchestra.run.smithi037.stdout:legacy client fscid: 1 2024-03-29T08:37:40.663 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:37:40.663 INFO:teuthology.orchestra.run.smithi037.stdout:Filesystem 'cephfs' (1) 2024-03-29T08:37:40.663 INFO:teuthology.orchestra.run.smithi037.stdout:fs_name cephfs 2024-03-29T08:37:40.663 INFO:teuthology.orchestra.run.smithi037.stdout:epoch 21 2024-03-29T08:37:40.663 INFO:teuthology.orchestra.run.smithi037.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-03-29T08:37:40.663 INFO:teuthology.orchestra.run.smithi037.stdout:created 2024-03-29T08:03:29.241269+0000 2024-03-29T08:37:40.663 INFO:teuthology.orchestra.run.smithi037.stdout:modified 2024-03-29T08:17:16.237238+0000 2024-03-29T08:37:40.663 INFO:teuthology.orchestra.run.smithi037.stdout:tableserver 0 2024-03-29T08:37:40.663 INFO:teuthology.orchestra.run.smithi037.stdout:root 0 2024-03-29T08:37:40.664 INFO:teuthology.orchestra.run.smithi037.stdout:session_timeout 60 2024-03-29T08:37:40.664 INFO:teuthology.orchestra.run.smithi037.stdout:session_autoclose 300 2024-03-29T08:37:40.664 INFO:teuthology.orchestra.run.smithi037.stdout:max_file_size 1099511627776 2024-03-29T08:37:40.664 INFO:teuthology.orchestra.run.smithi037.stdout:max_xattr_size 65536 2024-03-29T08:37:40.664 INFO:teuthology.orchestra.run.smithi037.stdout:required_client_features {} 2024-03-29T08:37:40.664 INFO:teuthology.orchestra.run.smithi037.stdout:last_failure 0 2024-03-29T08:37:40.664 INFO:teuthology.orchestra.run.smithi037.stdout:last_failure_osd_epoch 82 2024-03-29T08:37:40.664 INFO:teuthology.orchestra.run.smithi037.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T08:37:40.664 INFO:teuthology.orchestra.run.smithi037.stdout:max_mds 2 2024-03-29T08:37:40.664 INFO:teuthology.orchestra.run.smithi037.stdout:in 0,1 2024-03-29T08:37:40.664 INFO:teuthology.orchestra.run.smithi037.stdout:up {} 2024-03-29T08:37:40.664 INFO:teuthology.orchestra.run.smithi037.stdout:failed 0,1 2024-03-29T08:37:40.664 INFO:teuthology.orchestra.run.smithi037.stdout:damaged 2024-03-29T08:37:40.664 INFO:teuthology.orchestra.run.smithi037.stdout:stopped 2024-03-29T08:37:40.664 INFO:teuthology.orchestra.run.smithi037.stdout:data_pools [3] 2024-03-29T08:37:40.665 INFO:teuthology.orchestra.run.smithi037.stdout:metadata_pool 2 2024-03-29T08:37:40.665 INFO:teuthology.orchestra.run.smithi037.stdout:inline_data disabled 2024-03-29T08:37:40.665 INFO:teuthology.orchestra.run.smithi037.stdout:balancer 2024-03-29T08:37:40.665 INFO:teuthology.orchestra.run.smithi037.stdout:bal_rank_mask -1 2024-03-29T08:37:40.665 INFO:teuthology.orchestra.run.smithi037.stdout:standby_count_wanted 1 2024-03-29T08:37:40.665 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:37:40.665 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:37:40.665 INFO:teuthology.orchestra.run.smithi037.stdout:Standby daemons: 2024-03-29T08:37:40.665 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:37:40.665 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi037.mmwgxe{-1:34440} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.37:6828/1248142005,v1:172.21.15.37:6829/1248142005] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:37:40.665 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi037.iifsmh{-1:34444} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.37:6826/2060882959,v1:172.21.15.37:6827/2060882959] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:37:40.665 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi148.sqteih{-1:44395} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.148:6824/4078652242,v1:172.21.15.148:6825/4078652242] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:37:40.665 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi148.qukpjh{-1:45007} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.148:6826/2394751105,v1:172.21.15.148:6827/2394751105] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:37:40.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:40.665+0000 7fd5c6200700 1 -- 172.21.15.37:0/2559152153 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fd5a406cc30 msgr2=0x7fd5a406f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:37:40.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:40.665+0000 7fd5c6200700 1 --2- 172.21.15.37:0/2559152153 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fd5a406cc30 0x7fd5a406f0e0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fd5b000b3b0 tx=0x7fd5b0009e30 comp rx=0 tx=0).stop 2024-03-29T08:37:40.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:40.665+0000 7fd5c6200700 1 -- 172.21.15.37:0/2559152153 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd5c00fe830 msgr2=0x7fd5c0116200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:37:40.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:40.665+0000 7fd5c6200700 1 --2- 172.21.15.37:0/2559152153 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd5c00fe830 0x7fd5c0116200 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fd5ac002a50 tx=0x7fd5ac0046b0 comp rx=0 tx=0).stop 2024-03-29T08:37:40.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:40.665+0000 7fd5c6200700 1 -- 172.21.15.37:0/2559152153 shutdown_connections 2024-03-29T08:37:40.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:40.665+0000 7fd5c6200700 1 --2- 172.21.15.37:0/2559152153 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fd5a406cc30 0x7fd5a406f0e0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:37:40.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:40.665+0000 7fd5c6200700 1 --2- 172.21.15.37:0/2559152153 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd5c00fde90 0x7fd5c0115cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:37:40.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:40.665+0000 7fd5c6200700 1 --2- 172.21.15.37:0/2559152153 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd5c00fe830 0x7fd5c0116200 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:37:40.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:40.665+0000 7fd5c6200700 1 -- 172.21.15.37:0/2559152153 >> 172.21.15.37:0/2559152153 conn(0x7fd5c0072bc0 msgr2=0x7fd5c00fb900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:37:40.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:40.665+0000 7fd5c6200700 1 -- 172.21.15.37:0/2559152153 shutdown_connections 2024-03-29T08:37:40.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:40.665+0000 7fd5c6200700 1 -- 172.21.15.37:0/2559152153 wait complete. 2024-03-29T08:37:40.667 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 30 2024-03-29T08:37:40.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:40.803+0000 7f2f1c400700 1 -- 172.21.15.37:0/1457079645 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2f141037a0 msgr2=0x7f2f140fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:37:40.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:40.803+0000 7f2f1c400700 1 --2- 172.21.15.37:0/1457079645 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2f141037a0 0x7f2f140fce10 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f2f00004940 tx=0x7f2f00009e30 comp rx=0 tx=0).stop 2024-03-29T08:37:40.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:40.803+0000 7f2f1c400700 1 -- 172.21.15.37:0/1457079645 shutdown_connections 2024-03-29T08:37:40.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:40.803+0000 7f2f1c400700 1 --2- 172.21.15.37:0/1457079645 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2f141037a0 0x7f2f140fce10 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:37:40.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:40.803+0000 7f2f1c400700 1 --2- 172.21.15.37:0/1457079645 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2f14102e00 0x7f2f141031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:37:40.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:40.803+0000 7f2f1c400700 1 -- 172.21.15.37:0/1457079645 >> 172.21.15.37:0/1457079645 conn(0x7f2f140f89b0 msgr2=0x7f2f140fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:37:40.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:40.803+0000 7f2f1c400700 1 -- 172.21.15.37:0/1457079645 shutdown_connections 2024-03-29T08:37:40.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:40.803+0000 7f2f1c400700 1 -- 172.21.15.37:0/1457079645 wait complete. 2024-03-29T08:37:40.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:40.804+0000 7f2f1c400700 1 Processor -- start 2024-03-29T08:37:40.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:40.804+0000 7f2f1c400700 1 -- start start 2024-03-29T08:37:40.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:40.804+0000 7f2f1c400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2f14102e00 0x7f2f14115cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:37:40.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:40.804+0000 7f2f1c400700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2f141037a0 0x7f2f14116230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:37:40.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:40.804+0000 7f2f1c400700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f14116880 con 0x7f2f14102e00 2024-03-29T08:37:40.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:40.804+0000 7f2f1c400700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f1410fc50 con 0x7f2f141037a0 2024-03-29T08:37:40.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:40.804+0000 7f2f19a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2f14102e00 0x7f2f14115cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:37:40.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:40.805+0000 7f2f19a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2f14102e00 0x7f2f14115cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45560/0 (socket says 172.21.15.37:45560) 2024-03-29T08:37:40.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:40.805+0000 7f2f19a00700 1 -- 172.21.15.37:0/2770189062 learned_addr learned my addr 172.21.15.37:0/2770189062 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:37:40.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:40.805+0000 7f2f19000700 1 --2- 172.21.15.37:0/2770189062 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2f141037a0 0x7f2f14116230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:37:40.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:40.805+0000 7f2f19a00700 1 -- 172.21.15.37:0/2770189062 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2f141037a0 msgr2=0x7f2f14116230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:37:40.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:40.805+0000 7f2f19a00700 1 --2- 172.21.15.37:0/2770189062 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2f141037a0 0x7f2f14116230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:37:40.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:40.805+0000 7f2f19a00700 1 -- 172.21.15.37:0/2770189062 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f00005040 con 0x7f2f14102e00 2024-03-29T08:37:40.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:37:40 smithi148 ceph-mon[106692]: from='client.35458 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:37:40.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:37:40 smithi148 ceph-mon[106692]: pgmap v804: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:37:40.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:37:40 smithi148 ceph-mon[106692]: from='client.35462 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:37:40.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:37:40 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:37:40.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:37:40 smithi148 ceph-mon[106692]: from='client.35466 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:37:40.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:37:40 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/1127496647' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:37:40.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:40.805+0000 7f2f19a00700 1 --2- 172.21.15.37:0/2770189062 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2f14102e00 0x7f2f14115cf0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f2f08002a50 tx=0x7f2f080046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:37:40.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:40.805+0000 7f2f11600700 1 -- 172.21.15.37:0/2770189062 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2f08007cc0 con 0x7f2f14102e00 2024-03-29T08:37:40.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:40.805+0000 7f2f11600700 1 -- 172.21.15.37:0/2770189062 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f2f0800f040 con 0x7f2f14102e00 2024-03-29T08:37:40.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:40.805+0000 7f2f1c400700 1 -- 172.21.15.37:0/2770189062 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f1410ff30 con 0x7f2f14102e00 2024-03-29T08:37:40.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:40.806+0000 7f2f11600700 1 -- 172.21.15.37:0/2770189062 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2f0800d9c0 con 0x7f2f14102e00 2024-03-29T08:37:40.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:40.806+0000 7f2f1c400700 1 -- 172.21.15.37:0/2770189062 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f141104f0 con 0x7f2f14102e00 2024-03-29T08:37:40.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:40.807+0000 7f2f1c400700 1 -- 172.21.15.37:0/2770189062 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f140fe550 con 0x7f2f14102e00 2024-03-29T08:37:40.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:40.808+0000 7f2f11600700 1 -- 172.21.15.37:0/2770189062 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f2f0800db20 con 0x7f2f14102e00 2024-03-29T08:37:40.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:40.809+0000 7f2f11600700 1 --2- 172.21.15.37:0/2770189062 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f2efc06cc30 0x7f2efc06f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:37:40.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:40.809+0000 7f2f11600700 1 -- 172.21.15.37:0/2770189062 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f2f0808bd20 con 0x7f2f14102e00 2024-03-29T08:37:40.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:40.809+0000 7f2f19000700 1 --2- 172.21.15.37:0/2770189062 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f2efc06cc30 0x7f2efc06f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:37:40.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:40.810+0000 7f2f19000700 1 --2- 172.21.15.37:0/2770189062 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f2efc06cc30 0x7f2efc06f0e0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f2f14111590 tx=0x7f2f00009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:37:40.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:40.813+0000 7f2f11600700 1 -- 172.21.15.37:0/2770189062 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f2f08059770 con 0x7f2f14102e00 2024-03-29T08:37:40.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:37:40 smithi037 ceph-mon[142325]: from='client.35458 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:37:40.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:37:40 smithi037 ceph-mon[142325]: pgmap v804: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:37:40.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:37:40 smithi037 ceph-mon[142325]: from='client.35462 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:37:40.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:37:40 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:37:40.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:37:40 smithi037 ceph-mon[142325]: from='client.35466 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:37:40.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:37:40 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/1127496647' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:37:41.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:41.045+0000 7f2f1c400700 1 -- 172.21.15.37:0/2770189062 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f2f1404bf40 con 0x7f2efc06cc30 2024-03-29T08:37:41.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:41.046+0000 7f2f11600700 1 -- 172.21.15.37:0/2770189062 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f2f1404bf40 con 0x7f2efc06cc30 2024-03-29T08:37:41.046 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-29T08:37:41.046 INFO:teuthology.orchestra.run.smithi037.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59", 2024-03-29T08:37:41.046 INFO:teuthology.orchestra.run.smithi037.stdout: "in_progress": true, 2024-03-29T08:37:41.046 INFO:teuthology.orchestra.run.smithi037.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-29T08:37:41.046 INFO:teuthology.orchestra.run.smithi037.stdout: "services_complete": [ 2024-03-29T08:37:41.047 INFO:teuthology.orchestra.run.smithi037.stdout: "crash", 2024-03-29T08:37:41.047 INFO:teuthology.orchestra.run.smithi037.stdout: "osd", 2024-03-29T08:37:41.047 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr", 2024-03-29T08:37:41.047 INFO:teuthology.orchestra.run.smithi037.stdout: "mds", 2024-03-29T08:37:41.047 INFO:teuthology.orchestra.run.smithi037.stdout: "mon" 2024-03-29T08:37:41.047 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-29T08:37:41.047 INFO:teuthology.orchestra.run.smithi037.stdout: "progress": "16/21 daemons upgraded", 2024-03-29T08:37:41.047 INFO:teuthology.orchestra.run.smithi037.stdout: "message": "Currently upgrading mds daemons", 2024-03-29T08:37:41.047 INFO:teuthology.orchestra.run.smithi037.stdout: "is_paused": false 2024-03-29T08:37:41.048 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-29T08:37:41.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:41.048+0000 7f2f1c400700 1 -- 172.21.15.37:0/2770189062 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f2efc06cc30 msgr2=0x7f2efc06f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:37:41.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:41.048+0000 7f2f1c400700 1 --2- 172.21.15.37:0/2770189062 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f2efc06cc30 0x7f2efc06f0e0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f2f14111590 tx=0x7f2f00009e30 comp rx=0 tx=0).stop 2024-03-29T08:37:41.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:41.048+0000 7f2f1c400700 1 -- 172.21.15.37:0/2770189062 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2f14102e00 msgr2=0x7f2f14115cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:37:41.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:41.048+0000 7f2f1c400700 1 --2- 172.21.15.37:0/2770189062 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2f14102e00 0x7f2f14115cf0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f2f08002a50 tx=0x7f2f080046b0 comp rx=0 tx=0).stop 2024-03-29T08:37:41.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:41.048+0000 7f2f1c400700 1 -- 172.21.15.37:0/2770189062 shutdown_connections 2024-03-29T08:37:41.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:41.048+0000 7f2f1c400700 1 --2- 172.21.15.37:0/2770189062 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f2efc06cc30 0x7f2efc06f0e0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:37:41.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:41.048+0000 7f2f1c400700 1 --2- 172.21.15.37:0/2770189062 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2f14102e00 0x7f2f14115cf0 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:37:41.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:41.048+0000 7f2f1c400700 1 --2- 172.21.15.37:0/2770189062 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2f141037a0 0x7f2f14116230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:37:41.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:41.048+0000 7f2f1c400700 1 -- 172.21.15.37:0/2770189062 >> 172.21.15.37:0/2770189062 conn(0x7f2f140f89b0 msgr2=0x7f2f140f9a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:37:41.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:41.049+0000 7f2f1c400700 1 -- 172.21.15.37:0/2770189062 shutdown_connections 2024-03-29T08:37:41.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:41.049+0000 7f2f1c400700 1 -- 172.21.15.37:0/2770189062 wait complete. 2024-03-29T08:37:41.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:41.186+0000 7f957bc00700 1 -- 172.21.15.37:0/2433160132 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9574102e00 msgr2=0x7f95741031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:37:41.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:41.186+0000 7f957bc00700 1 --2- 172.21.15.37:0/2433160132 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9574102e00 0x7f95741031d0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f9568004940 tx=0x7f9568009e30 comp rx=0 tx=0).stop 2024-03-29T08:37:41.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:41.186+0000 7f957bc00700 1 -- 172.21.15.37:0/2433160132 shutdown_connections 2024-03-29T08:37:41.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:41.186+0000 7f957bc00700 1 --2- 172.21.15.37:0/2433160132 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f95741037a0 0x7f95740fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:37:41.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:41.186+0000 7f957bc00700 1 --2- 172.21.15.37:0/2433160132 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9574102e00 0x7f95741031d0 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:37:41.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:41.186+0000 7f957bc00700 1 -- 172.21.15.37:0/2433160132 >> 172.21.15.37:0/2433160132 conn(0x7f95740f89b0 msgr2=0x7f95740fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:37:41.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:41.186+0000 7f957bc00700 1 -- 172.21.15.37:0/2433160132 shutdown_connections 2024-03-29T08:37:41.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:41.186+0000 7f957bc00700 1 -- 172.21.15.37:0/2433160132 wait complete. 2024-03-29T08:37:41.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:41.187+0000 7f957bc00700 1 Processor -- start 2024-03-29T08:37:41.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:41.187+0000 7f957bc00700 1 -- start start 2024-03-29T08:37:41.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:41.188+0000 7f957bc00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9574102e00 0x7f9574115cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:37:41.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:41.188+0000 7f957bc00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f95741037a0 0x7f9574116210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:37:41.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:41.188+0000 7f957bc00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9574116860 con 0x7f95741037a0 2024-03-29T08:37:41.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:41.188+0000 7f957bc00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f957410fc30 con 0x7f9574102e00 2024-03-29T08:37:41.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:41.188+0000 7f9573e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f95741037a0 0x7f9574116210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:37:41.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:41.188+0000 7f9573e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f95741037a0 0x7f9574116210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45588/0 (socket says 172.21.15.37:45588) 2024-03-29T08:37:41.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:41.188+0000 7f9573e00700 1 -- 172.21.15.37:0/4253647418 learned_addr learned my addr 172.21.15.37:0/4253647418 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:37:41.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:41.188+0000 7f9579200700 1 --2- 172.21.15.37:0/4253647418 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9574102e00 0x7f9574115cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:37:41.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:41.188+0000 7f9573e00700 1 -- 172.21.15.37:0/4253647418 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9574102e00 msgr2=0x7f9574115cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:37:41.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:41.188+0000 7f9573e00700 1 --2- 172.21.15.37:0/4253647418 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9574102e00 0x7f9574115cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:37:41.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:41.188+0000 7f9573e00700 1 -- 172.21.15.37:0/4253647418 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9568005040 con 0x7f95741037a0 2024-03-29T08:37:41.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:41.189+0000 7f9573e00700 1 --2- 172.21.15.37:0/4253647418 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f95741037a0 0x7f9574116210 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f956000c800 tx=0x7f9560004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:37:41.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:41.189+0000 7f9570c00700 1 -- 172.21.15.37:0/4253647418 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f95600106f0 con 0x7f95741037a0 2024-03-29T08:37:41.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:41.189+0000 7f9570c00700 1 -- 172.21.15.37:0/4253647418 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f9560010d00 con 0x7f95741037a0 2024-03-29T08:37:41.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:41.189+0000 7f957bc00700 1 -- 172.21.15.37:0/4253647418 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f957410ff10 con 0x7f95741037a0 2024-03-29T08:37:41.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:41.189+0000 7f9570c00700 1 -- 172.21.15.37:0/4253647418 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f95600113a0 con 0x7f95741037a0 2024-03-29T08:37:41.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:41.189+0000 7f957bc00700 1 -- 172.21.15.37:0/4253647418 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95741104d0 con 0x7f95741037a0 2024-03-29T08:37:41.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:41.190+0000 7f957bc00700 1 -- 172.21.15.37:0/4253647418 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95740fe550 con 0x7f95741037a0 2024-03-29T08:37:41.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:41.192+0000 7f9570c00700 1 -- 172.21.15.37:0/4253647418 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f9560012400 con 0x7f95741037a0 2024-03-29T08:37:41.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:41.193+0000 7f9570c00700 1 --2- 172.21.15.37:0/4253647418 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f955c06cc20 0x7f955c06f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:37:41.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:41.193+0000 7f9570c00700 1 -- 172.21.15.37:0/4253647418 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f956008bef0 con 0x7f95741037a0 2024-03-29T08:37:41.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:41.193+0000 7f9579200700 1 --2- 172.21.15.37:0/4253647418 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f955c06cc20 0x7f955c06f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:37:41.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:41.194+0000 7f9579200700 1 --2- 172.21.15.37:0/4253647418 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f955c06cc20 0x7f955c06f0d0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f9568004f90 tx=0x7f9568009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:37:41.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:41.197+0000 7f9570c00700 1 -- 172.21.15.37:0/4253647418 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f9560059a80 con 0x7f95741037a0 2024-03-29T08:37:41.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:41.499+0000 7f957bc00700 1 -- 172.21.15.37:0/4253647418 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f957405f5f0 con 0x7f95741037a0 2024-03-29T08:37:41.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:41.500+0000 7f9570c00700 1 -- 172.21.15.37:0/4253647418 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+3583 (secure 0 0 0) 0x7f9560059610 con 0x7f95741037a0 2024-03-29T08:37:41.500 INFO:teuthology.orchestra.run.smithi037.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem is offline 2024-03-29T08:37:41.500 INFO:teuthology.orchestra.run.smithi037.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-03-29T08:37:41.500 INFO:teuthology.orchestra.run.smithi037.stdout: host smithi148 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/mon.smithi148/config 2024-03-29T08:37:41.500 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/podman: timeout after 895 seconds 2024-03-29T08:37:41.500 INFO:teuthology.orchestra.run.smithi037.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59 -e NODE_NAME=smithi148 -e CEPH_USE_RANDOM_NONCE=1 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4:/var/run/ceph:z -v /var/log/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4:/var/log/ceph:z -v /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpkx2rz2d4:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59 inventory --format=json-pretty --filter-for-batch 2024-03-29T08:37:41.500 INFO:teuthology.orchestra.run.smithi037.stdout:Traceback (most recent call last): 2024-03-29T08:37:41.500 INFO:teuthology.orchestra.run.smithi037.stdout: File "/usr/lib64/python3.6/runpy.py", line 193, in _run_module_as_main 2024-03-29T08:37:41.500 INFO:teuthology.orchestra.run.smithi037.stdout: "__main__", mod_spec) 2024-03-29T08:37:41.500 INFO:teuthology.orchestra.run.smithi037.stdout: File "/usr/lib64/python3.6/runpy.py", line 85, in _run_code 2024-03-29T08:37:41.500 INFO:teuthology.orchestra.run.smithi037.stdout: exec(code, run_globals) 2024-03-29T08:37:41.500 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 10889, in 2024-03-29T08:37:41.500 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 10877, in main 2024-03-29T08:37:41.501 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 2576, in _infer_config 2024-03-29T08:37:41.501 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 2492, in _infer_fsid 2024-03-29T08:37:41.501 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 2604, in _infer_image 2024-03-29T08:37:41.501 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 2479, in _validate_fsid 2024-03-29T08:37:41.501 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 7145, in command_ceph_volume 2024-03-29T08:37:41.501 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 2267, in call_throws 2024-03-29T08:37:41.501 INFO:teuthology.orchestra.run.smithi037.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59 -e NODE_NAME=smithi148 -e CEPH_USE_RANDOM_NONCE=1 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4:/var/run/ceph:z -v /var/log/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4:/var/log/ceph:z -v /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpkx2rz2d4:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59 inventory --format=json-pretty --filter-for-batch 2024-03-29T08:37:41.501 INFO:teuthology.orchestra.run.smithi037.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-03-29T08:37:41.501 INFO:teuthology.orchestra.run.smithi037.stdout: fs cephfs is degraded 2024-03-29T08:37:41.501 INFO:teuthology.orchestra.run.smithi037.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-03-29T08:37:41.501 INFO:teuthology.orchestra.run.smithi037.stdout: fs cephfs is offline because no MDS is active for it. 2024-03-29T08:37:41.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:41.502+0000 7f957bc00700 1 -- 172.21.15.37:0/4253647418 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f955c06cc20 msgr2=0x7f955c06f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:37:41.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:41.502+0000 7f957bc00700 1 --2- 172.21.15.37:0/4253647418 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f955c06cc20 0x7f955c06f0d0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f9568004f90 tx=0x7f9568009e30 comp rx=0 tx=0).stop 2024-03-29T08:37:41.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:41.502+0000 7f957bc00700 1 -- 172.21.15.37:0/4253647418 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f95741037a0 msgr2=0x7f9574116210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:37:41.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:41.502+0000 7f957bc00700 1 --2- 172.21.15.37:0/4253647418 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f95741037a0 0x7f9574116210 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f956000c800 tx=0x7f9560004470 comp rx=0 tx=0).stop 2024-03-29T08:37:41.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:41.502+0000 7f957bc00700 1 -- 172.21.15.37:0/4253647418 shutdown_connections 2024-03-29T08:37:41.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:41.502+0000 7f957bc00700 1 --2- 172.21.15.37:0/4253647418 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f955c06cc20 0x7f955c06f0d0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:37:41.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:41.502+0000 7f957bc00700 1 --2- 172.21.15.37:0/4253647418 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9574102e00 0x7f9574115cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:37:41.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:41.502+0000 7f957bc00700 1 --2- 172.21.15.37:0/4253647418 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f95741037a0 0x7f9574116210 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:37:41.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:41.502+0000 7f957bc00700 1 -- 172.21.15.37:0/4253647418 >> 172.21.15.37:0/4253647418 conn(0x7f95740f89b0 msgr2=0x7f95740f99e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:37:41.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:41.502+0000 7f957bc00700 1 -- 172.21.15.37:0/4253647418 shutdown_connections 2024-03-29T08:37:41.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:37:41.502+0000 7f957bc00700 1 -- 172.21.15.37:0/4253647418 wait complete. 2024-03-29T08:37:41.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:37:41 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/2559152153' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T08:37:41.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:37:41 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/2559152153' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T08:37:42.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:37:42 smithi148 ceph-mon[106692]: from='client.35478 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:37:42.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:37:42 smithi148 ceph-mon[106692]: pgmap v805: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:37:42.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:37:42 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/4253647418' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T08:37:42.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:37:42 smithi037 ceph-mon[142325]: from='client.35478 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:37:42.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:37:42 smithi037 ceph-mon[142325]: pgmap v805: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:37:42.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:37:42 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/4253647418' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T08:37:44.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:37:44 smithi148 ceph-mon[106692]: pgmap v806: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:37:44.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:37:44 smithi037 ceph-mon[142325]: pgmap v806: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:37:46.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:37:46 smithi148 ceph-mon[106692]: pgmap v807: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:37:46.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:37:46 smithi037 ceph-mon[142325]: pgmap v807: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:37:48.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:37:48 smithi148 ceph-mon[106692]: pgmap v808: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:37:48.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:37:48 smithi037 ceph-mon[142325]: pgmap v808: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:37:50.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:37:50 smithi148 ceph-mon[106692]: pgmap v809: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:37:50.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:37:50 smithi037 ceph-mon[142325]: pgmap v809: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:37:52.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:37:52 smithi148 ceph-mon[106692]: pgmap v810: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:37:52.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:37:52 smithi037 ceph-mon[142325]: pgmap v810: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:37:54.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:37:54 smithi148 ceph-mon[106692]: pgmap v811: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:37:54.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:37:54 smithi037 ceph-mon[142325]: pgmap v811: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:37:55.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:37:55 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:37:55.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:37:55 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:37:56.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:37:56 smithi148 ceph-mon[106692]: pgmap v812: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:37:56.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:37:56 smithi037 ceph-mon[142325]: pgmap v812: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:37:58.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:37:58 smithi148 ceph-mon[106692]: pgmap v813: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:37:58.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:37:58 smithi037 ceph-mon[142325]: pgmap v813: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:38:00.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:38:00 smithi148 ceph-mon[106692]: pgmap v814: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:38:00.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:38:00 smithi037 ceph-mon[142325]: pgmap v814: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:38:02.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:38:02 smithi148 ceph-mon[106692]: pgmap v815: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:38:02.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:38:02 smithi037 ceph-mon[142325]: pgmap v815: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:38:04.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:38:04 smithi148 ceph-mon[106692]: pgmap v816: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:38:04.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:38:04 smithi037 ceph-mon[142325]: pgmap v816: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:38:06.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:38:06 smithi148 ceph-mon[106692]: pgmap v817: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:38:06.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:38:06 smithi037 ceph-mon[142325]: pgmap v817: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:38:08.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:38:08 smithi148 ceph-mon[106692]: pgmap v818: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:38:08.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:38:08 smithi037 ceph-mon[142325]: pgmap v818: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:38:10.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:38:10 smithi148 ceph-mon[106692]: pgmap v819: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:38:10.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:38:10 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:38:10.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:38:10 smithi037 ceph-mon[142325]: pgmap v819: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:38:10.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:38:10 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:38:11.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:11.654+0000 7f1610600700 1 -- 172.21.15.37:0/2222199447 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1608102e00 msgr2=0x7f16081031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:38:11.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:11.654+0000 7f1610600700 1 --2- 172.21.15.37:0/2222199447 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1608102e00 0x7f16081031d0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f15fc004940 tx=0x7f15fc009e30 comp rx=0 tx=0).stop 2024-03-29T08:38:11.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:11.654+0000 7f1610600700 1 -- 172.21.15.37:0/2222199447 shutdown_connections 2024-03-29T08:38:11.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:11.654+0000 7f1610600700 1 --2- 172.21.15.37:0/2222199447 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f16081037a0 0x7f16080fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:38:11.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:11.654+0000 7f1610600700 1 --2- 172.21.15.37:0/2222199447 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1608102e00 0x7f16081031d0 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:38:11.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:11.654+0000 7f1610600700 1 -- 172.21.15.37:0/2222199447 >> 172.21.15.37:0/2222199447 conn(0x7f16080f89b0 msgr2=0x7f16080fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:38:11.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:11.654+0000 7f1610600700 1 -- 172.21.15.37:0/2222199447 shutdown_connections 2024-03-29T08:38:11.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:11.654+0000 7f1610600700 1 -- 172.21.15.37:0/2222199447 wait complete. 2024-03-29T08:38:11.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:11.655+0000 7f1610600700 1 Processor -- start 2024-03-29T08:38:11.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:11.655+0000 7f1610600700 1 -- start start 2024-03-29T08:38:11.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:11.656+0000 7f1610600700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1608102e00 0x7f1608115c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:38:11.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:11.656+0000 7f1610600700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f16081037a0 0x7f16081161c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:38:11.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:11.656+0000 7f1610600700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1608116810 con 0x7f16081037a0 2024-03-29T08:38:11.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:11.656+0000 7f1610600700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f160810fbe0 con 0x7f1608102e00 2024-03-29T08:38:11.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:11.656+0000 7f160d200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f16081037a0 0x7f16081161c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:38:11.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:11.656+0000 7f160d200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f16081037a0 0x7f16081161c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39544/0 (socket says 172.21.15.37:39544) 2024-03-29T08:38:11.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:11.656+0000 7f160d200700 1 -- 172.21.15.37:0/3384448063 learned_addr learned my addr 172.21.15.37:0/3384448063 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:38:11.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:11.656+0000 7f160dc00700 1 --2- 172.21.15.37:0/3384448063 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1608102e00 0x7f1608115c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:38:11.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:11.657+0000 7f160d200700 1 -- 172.21.15.37:0/3384448063 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1608102e00 msgr2=0x7f1608115c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:38:11.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:11.657+0000 7f160d200700 1 --2- 172.21.15.37:0/3384448063 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1608102e00 0x7f1608115c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:38:11.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:11.657+0000 7f160d200700 1 -- 172.21.15.37:0/3384448063 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15fc005040 con 0x7f16081037a0 2024-03-29T08:38:11.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:11.657+0000 7f160d200700 1 --2- 172.21.15.37:0/3384448063 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f16081037a0 0x7f16081161c0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f15f4002a50 tx=0x7f15f40046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:38:11.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:11.657+0000 7f1605600700 1 -- 172.21.15.37:0/3384448063 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f15f4007cc0 con 0x7f16081037a0 2024-03-29T08:38:11.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:11.657+0000 7f1605600700 1 -- 172.21.15.37:0/3384448063 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f15f400f040 con 0x7f16081037a0 2024-03-29T08:38:11.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:11.657+0000 7f1610600700 1 -- 172.21.15.37:0/3384448063 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f160810fec0 con 0x7f16081037a0 2024-03-29T08:38:11.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:11.657+0000 7f1605600700 1 -- 172.21.15.37:0/3384448063 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f15f400da30 con 0x7f16081037a0 2024-03-29T08:38:11.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:11.657+0000 7f1610600700 1 -- 172.21.15.37:0/3384448063 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1608110480 con 0x7f16081037a0 2024-03-29T08:38:11.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:11.658+0000 7f1610600700 1 -- 172.21.15.37:0/3384448063 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16080fe550 con 0x7f16081037a0 2024-03-29T08:38:11.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:11.659+0000 7f1605600700 1 -- 172.21.15.37:0/3384448063 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f15f400db90 con 0x7f16081037a0 2024-03-29T08:38:11.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:11.661+0000 7f1605600700 1 --2- 172.21.15.37:0/3384448063 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f15f006cc30 0x7f15f006f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:38:11.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:11.661+0000 7f1605600700 1 -- 172.21.15.37:0/3384448063 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f15f408bd00 con 0x7f16081037a0 2024-03-29T08:38:11.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:11.661+0000 7f160dc00700 1 --2- 172.21.15.37:0/3384448063 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f15f006cc30 0x7f15f006f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:38:11.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:11.662+0000 7f160dc00700 1 --2- 172.21.15.37:0/3384448063 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f15f006cc30 0x7f15f006f0e0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f15fc009fd0 tx=0x7f15fc009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:38:11.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:11.666+0000 7f1605600700 1 -- 172.21.15.37:0/3384448063 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f15f40596d0 con 0x7f16081037a0 2024-03-29T08:38:11.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:11.900+0000 7f1610600700 1 -- 172.21.15.37:0/3384448063 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f1608109e30 con 0x7f15f006cc30 2024-03-29T08:38:11.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:11.902+0000 7f1605600700 1 -- 172.21.15.37:0/3384448063 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f1608109e30 con 0x7f15f006cc30 2024-03-29T08:38:11.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:11.904+0000 7f1610600700 1 -- 172.21.15.37:0/3384448063 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f15f006cc30 msgr2=0x7f15f006f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:38:11.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:11.904+0000 7f1610600700 1 --2- 172.21.15.37:0/3384448063 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f15f006cc30 0x7f15f006f0e0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f15fc009fd0 tx=0x7f15fc009e30 comp rx=0 tx=0).stop 2024-03-29T08:38:11.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:11.904+0000 7f1610600700 1 -- 172.21.15.37:0/3384448063 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f16081037a0 msgr2=0x7f16081161c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:38:11.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:11.904+0000 7f1610600700 1 --2- 172.21.15.37:0/3384448063 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f16081037a0 0x7f16081161c0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f15f4002a50 tx=0x7f15f40046b0 comp rx=0 tx=0).stop 2024-03-29T08:38:11.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:11.904+0000 7f1610600700 1 -- 172.21.15.37:0/3384448063 shutdown_connections 2024-03-29T08:38:11.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:11.904+0000 7f1610600700 1 --2- 172.21.15.37:0/3384448063 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f15f006cc30 0x7f15f006f0e0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:38:11.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:11.904+0000 7f1610600700 1 --2- 172.21.15.37:0/3384448063 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1608102e00 0x7f1608115c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:38:11.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:11.904+0000 7f1610600700 1 --2- 172.21.15.37:0/3384448063 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f16081037a0 0x7f16081161c0 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:38:11.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:11.904+0000 7f1610600700 1 -- 172.21.15.37:0/3384448063 >> 172.21.15.37:0/3384448063 conn(0x7f16080f89b0 msgr2=0x7f16080f9940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:38:11.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:11.904+0000 7f1610600700 1 -- 172.21.15.37:0/3384448063 shutdown_connections 2024-03-29T08:38:11.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:11.904+0000 7f1610600700 1 -- 172.21.15.37:0/3384448063 wait complete. 2024-03-29T08:38:11.920 INFO:teuthology.orchestra.run.smithi037.stdout:true 2024-03-29T08:38:12.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:12.043+0000 7f35d3200700 1 -- 172.21.15.37:0/3909625798 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f35cc070300 msgr2=0x7f35cc0706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:38:12.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:12.043+0000 7f35d3200700 1 --2- 172.21.15.37:0/3909625798 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f35cc070300 0x7f35cc0706d0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f35bc004970 tx=0x7f35bc009e30 comp rx=0 tx=0).stop 2024-03-29T08:38:12.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:12.043+0000 7f35d3200700 1 -- 172.21.15.37:0/3909625798 shutdown_connections 2024-03-29T08:38:12.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:12.043+0000 7f35d3200700 1 --2- 172.21.15.37:0/3909625798 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f35cc070c10 0x7f35cc109180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:38:12.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:12.043+0000 7f35d3200700 1 --2- 172.21.15.37:0/3909625798 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f35cc070300 0x7f35cc0706d0 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:38:12.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:12.043+0000 7f35d3200700 1 -- 172.21.15.37:0/3909625798 >> 172.21.15.37:0/3909625798 conn(0x7f35cc0f89a0 msgr2=0x7f35cc0fadb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:38:12.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:12.043+0000 7f35d3200700 1 -- 172.21.15.37:0/3909625798 shutdown_connections 2024-03-29T08:38:12.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:12.043+0000 7f35d3200700 1 -- 172.21.15.37:0/3909625798 wait complete. 2024-03-29T08:38:12.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:12.044+0000 7f35d3200700 1 Processor -- start 2024-03-29T08:38:12.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:12.044+0000 7f35d3200700 1 -- start start 2024-03-29T08:38:12.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:12.045+0000 7f35d3200700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f35cc070300 0x7f35cc115cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:38:12.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:12.045+0000 7f35d3200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f35cc070c10 0x7f35cc1161f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:38:12.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:12.045+0000 7f35d3200700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35cc116840 con 0x7f35cc070c10 2024-03-29T08:38:12.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:12.045+0000 7f35d3200700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35cc10fc10 con 0x7f35cc070300 2024-03-29T08:38:12.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:12.045+0000 7f35cb400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f35cc070c10 0x7f35cc1161f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:38:12.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:12.045+0000 7f35cb400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f35cc070c10 0x7f35cc1161f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39568/0 (socket says 172.21.15.37:39568) 2024-03-29T08:38:12.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:12.045+0000 7f35cb400700 1 -- 172.21.15.37:0/3158017807 learned_addr learned my addr 172.21.15.37:0/3158017807 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:38:12.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:12.045+0000 7f35cbe00700 1 --2- 172.21.15.37:0/3158017807 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f35cc070300 0x7f35cc115cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:38:12.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:12.045+0000 7f35cb400700 1 -- 172.21.15.37:0/3158017807 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f35cc070300 msgr2=0x7f35cc115cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:38:12.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:12.045+0000 7f35cb400700 1 --2- 172.21.15.37:0/3158017807 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f35cc070300 0x7f35cc115cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:38:12.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:12.045+0000 7f35cb400700 1 -- 172.21.15.37:0/3158017807 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35bc005040 con 0x7f35cc070c10 2024-03-29T08:38:12.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:12.046+0000 7f35cb400700 1 --2- 172.21.15.37:0/3158017807 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f35cc070c10 0x7f35cc1161f0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f35b800eb40 tx=0x7f35b800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:38:12.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:12.046+0000 7f35c3e00700 1 -- 172.21.15.37:0/3158017807 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f35b8007cc0 con 0x7f35cc070c10 2024-03-29T08:38:12.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:12.046+0000 7f35c3e00700 1 -- 172.21.15.37:0/3158017807 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f35b800f040 con 0x7f35cc070c10 2024-03-29T08:38:12.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:12.046+0000 7f35d3200700 1 -- 172.21.15.37:0/3158017807 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35cc10fe90 con 0x7f35cc070c10 2024-03-29T08:38:12.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:12.046+0000 7f35c3e00700 1 -- 172.21.15.37:0/3158017807 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f35b800c990 con 0x7f35cc070c10 2024-03-29T08:38:12.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:12.046+0000 7f35d3200700 1 -- 172.21.15.37:0/3158017807 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35cc1103d0 con 0x7f35cc070c10 2024-03-29T08:38:12.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:12.047+0000 7f35d3200700 1 -- 172.21.15.37:0/3158017807 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35cc103b90 con 0x7f35cc070c10 2024-03-29T08:38:12.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:12.048+0000 7f35c3e00700 1 -- 172.21.15.37:0/3158017807 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f35b800caf0 con 0x7f35cc070c10 2024-03-29T08:38:12.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:12.049+0000 7f35c3e00700 1 --2- 172.21.15.37:0/3158017807 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f35b006cc30 0x7f35b006f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:38:12.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:12.049+0000 7f35c3e00700 1 -- 172.21.15.37:0/3158017807 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f35b808bc00 con 0x7f35cc070c10 2024-03-29T08:38:12.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:12.050+0000 7f35cbe00700 1 --2- 172.21.15.37:0/3158017807 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f35b006cc30 0x7f35b006f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:38:12.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:12.050+0000 7f35cbe00700 1 --2- 172.21.15.37:0/3158017807 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f35b006cc30 0x7f35b006f0e0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f35bc009fd0 tx=0x7f35bc009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:38:12.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:12.054+0000 7f35c3e00700 1 -- 172.21.15.37:0/3158017807 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f35b8059700 con 0x7f35cc070c10 2024-03-29T08:38:12.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:12.286+0000 7f35d3200700 1 -- 172.21.15.37:0/3158017807 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f35cc111370 con 0x7f35b006cc30 2024-03-29T08:38:12.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:12.287+0000 7f35c3e00700 1 -- 172.21.15.37:0/3158017807 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f35cc111370 con 0x7f35b006cc30 2024-03-29T08:38:12.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:12.290+0000 7f35d3200700 1 -- 172.21.15.37:0/3158017807 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f35b006cc30 msgr2=0x7f35b006f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:38:12.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:12.290+0000 7f35d3200700 1 --2- 172.21.15.37:0/3158017807 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f35b006cc30 0x7f35b006f0e0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f35bc009fd0 tx=0x7f35bc009e30 comp rx=0 tx=0).stop 2024-03-29T08:38:12.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:12.290+0000 7f35d3200700 1 -- 172.21.15.37:0/3158017807 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f35cc070c10 msgr2=0x7f35cc1161f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:38:12.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:12.290+0000 7f35d3200700 1 --2- 172.21.15.37:0/3158017807 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f35cc070c10 0x7f35cc1161f0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f35b800eb40 tx=0x7f35b800b040 comp rx=0 tx=0).stop 2024-03-29T08:38:12.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:12.290+0000 7f35d3200700 1 -- 172.21.15.37:0/3158017807 shutdown_connections 2024-03-29T08:38:12.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:12.290+0000 7f35d3200700 1 --2- 172.21.15.37:0/3158017807 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f35b006cc30 0x7f35b006f0e0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:38:12.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:12.290+0000 7f35d3200700 1 --2- 172.21.15.37:0/3158017807 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f35cc070300 0x7f35cc115cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:38:12.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:12.290+0000 7f35d3200700 1 --2- 172.21.15.37:0/3158017807 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f35cc070c10 0x7f35cc1161f0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:38:12.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:12.290+0000 7f35d3200700 1 -- 172.21.15.37:0/3158017807 >> 172.21.15.37:0/3158017807 conn(0x7f35cc0f89a0 msgr2=0x7f35cc101660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:38:12.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:12.290+0000 7f35d3200700 1 -- 172.21.15.37:0/3158017807 shutdown_connections 2024-03-29T08:38:12.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:12.290+0000 7f35d3200700 1 -- 172.21.15.37:0/3158017807 wait complete. 2024-03-29T08:38:12.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:12.430+0000 7f5e36200700 1 -- 172.21.15.37:0/2577443168 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5e30102e00 msgr2=0x7f5e301031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:38:12.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:12.430+0000 7f5e36200700 1 --2- 172.21.15.37:0/2577443168 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5e30102e00 0x7f5e301031d0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f5e20004940 tx=0x7f5e20009e30 comp rx=0 tx=0).stop 2024-03-29T08:38:12.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:12.430+0000 7f5e36200700 1 -- 172.21.15.37:0/2577443168 shutdown_connections 2024-03-29T08:38:12.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:12.430+0000 7f5e36200700 1 --2- 172.21.15.37:0/2577443168 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5e301037a0 0x7f5e300fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:38:12.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:12.430+0000 7f5e36200700 1 --2- 172.21.15.37:0/2577443168 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5e30102e00 0x7f5e301031d0 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:38:12.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:12.430+0000 7f5e36200700 1 -- 172.21.15.37:0/2577443168 >> 172.21.15.37:0/2577443168 conn(0x7f5e300f89b0 msgr2=0x7f5e300fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:38:12.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:12.430+0000 7f5e36200700 1 -- 172.21.15.37:0/2577443168 shutdown_connections 2024-03-29T08:38:12.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:12.430+0000 7f5e36200700 1 -- 172.21.15.37:0/2577443168 wait complete. 2024-03-29T08:38:12.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:12.431+0000 7f5e36200700 1 Processor -- start 2024-03-29T08:38:12.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:12.431+0000 7f5e36200700 1 -- start start 2024-03-29T08:38:12.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:12.431+0000 7f5e36200700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5e30102e00 0x7f5e30115c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:38:12.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:12.431+0000 7f5e36200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5e301037a0 0x7f5e30116190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:38:12.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:12.431+0000 7f5e36200700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e301167e0 con 0x7f5e301037a0 2024-03-29T08:38:12.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:12.431+0000 7f5e36200700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e3010fbb0 con 0x7f5e30102e00 2024-03-29T08:38:12.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:12.432+0000 7f5e2ea00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5e301037a0 0x7f5e30116190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:38:12.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:12.432+0000 7f5e2ea00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5e301037a0 0x7f5e30116190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39596/0 (socket says 172.21.15.37:39596) 2024-03-29T08:38:12.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:12.432+0000 7f5e2ea00700 1 -- 172.21.15.37:0/2280514523 learned_addr learned my addr 172.21.15.37:0/2280514523 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:38:12.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:12.432+0000 7f5e2f400700 1 --2- 172.21.15.37:0/2280514523 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5e30102e00 0x7f5e30115c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:38:12.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:12.432+0000 7f5e2ea00700 1 -- 172.21.15.37:0/2280514523 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5e30102e00 msgr2=0x7f5e30115c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:38:12.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:12.432+0000 7f5e2ea00700 1 --2- 172.21.15.37:0/2280514523 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5e30102e00 0x7f5e30115c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:38:12.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:12.432+0000 7f5e2ea00700 1 -- 172.21.15.37:0/2280514523 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e20005040 con 0x7f5e301037a0 2024-03-29T08:38:12.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:12.432+0000 7f5e2ea00700 1 --2- 172.21.15.37:0/2280514523 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5e301037a0 0x7f5e30116190 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f5e1c00eb40 tx=0x7f5e1c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:38:12.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:12.432+0000 7f5e27400700 1 -- 172.21.15.37:0/2280514523 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5e1c007cc0 con 0x7f5e301037a0 2024-03-29T08:38:12.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:12.433+0000 7f5e27400700 1 -- 172.21.15.37:0/2280514523 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f5e1c00f040 con 0x7f5e301037a0 2024-03-29T08:38:12.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:12.433+0000 7f5e36200700 1 -- 172.21.15.37:0/2280514523 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e3010fe30 con 0x7f5e301037a0 2024-03-29T08:38:12.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:12.433+0000 7f5e27400700 1 -- 172.21.15.37:0/2280514523 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5e1c00c8f0 con 0x7f5e301037a0 2024-03-29T08:38:12.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:12.433+0000 7f5e36200700 1 -- 172.21.15.37:0/2280514523 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e301103f0 con 0x7f5e301037a0 2024-03-29T08:38:12.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:12.434+0000 7f5e36200700 1 -- 172.21.15.37:0/2280514523 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e300fe550 con 0x7f5e301037a0 2024-03-29T08:38:12.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:12.435+0000 7f5e27400700 1 -- 172.21.15.37:0/2280514523 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f5e1c00cb60 con 0x7f5e301037a0 2024-03-29T08:38:12.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:12.436+0000 7f5e27400700 1 --2- 172.21.15.37:0/2280514523 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f5e1406cc30 0x7f5e1406f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:38:12.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:12.436+0000 7f5e27400700 1 -- 172.21.15.37:0/2280514523 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f5e1c08bad0 con 0x7f5e301037a0 2024-03-29T08:38:12.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:12.436+0000 7f5e2f400700 1 --2- 172.21.15.37:0/2280514523 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f5e1406cc30 0x7f5e1406f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:38:12.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:12.437+0000 7f5e2f400700 1 --2- 172.21.15.37:0/2280514523 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f5e1406cc30 0x7f5e1406f0e0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f5e200045a0 tx=0x7f5e20009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:38:12.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:12.440+0000 7f5e27400700 1 -- 172.21.15.37:0/2280514523 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f5e1c0595d0 con 0x7f5e301037a0 2024-03-29T08:38:12.557 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:38:12 smithi148 ceph-mon[106692]: pgmap v820: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:38:12.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:12.664+0000 7f5e36200700 1 -- 172.21.15.37:0/2280514523 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f5e30109e30 con 0x7f5e1406cc30 2024-03-29T08:38:12.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:12.674+0000 7f5e27400700 1 -- 172.21.15.37:0/2280514523 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f5e30109e30 con 0x7f5e1406cc30 2024-03-29T08:38:12.677 INFO:teuthology.orchestra.run.smithi037.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-29T08:38:12.677 INFO:teuthology.orchestra.run.smithi037.stdout:alertmanager.smithi037 smithi037 *:9093,9094 running (28m) 4m ago 40m 14.3M - 0.25.0 c8568f914cd2 2f9f9746ed0f 2024-03-29T08:38:12.677 INFO:teuthology.orchestra.run.smithi037.stdout:crash.smithi037 smithi037 running (24m) 4m ago 41m 6647k - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 178d416a3186 2024-03-29T08:38:12.678 INFO:teuthology.orchestra.run.smithi037.stdout:crash.smithi148 smithi148 running (24m) 4m ago 40m 6656k - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 23666989d58f 2024-03-29T08:38:12.678 INFO:teuthology.orchestra.run.smithi037.stdout:grafana.smithi037 smithi037 *:3000 running (27m) 4m ago 40m 74.5M - 9.4.7 954c08fa6188 6b976fe49952 2024-03-29T08:38:12.678 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi037.iifsmh smithi037 running (20m) 4m ago 34m 16.4M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 2d97f562cc5b 2024-03-29T08:38:12.678 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi037.mmwgxe smithi037 running (20m) 4m ago 34m 16.7M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff ec37037f46e8 2024-03-29T08:38:12.678 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi148.qukpjh smithi148 running (4m) 4m ago 34m 10.7M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 3257219bc046 2024-03-29T08:38:12.678 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi148.sqteih smithi148 running (20m) 4m ago 34m 16.3M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 44c15c77037a 2024-03-29T08:38:12.678 INFO:teuthology.orchestra.run.smithi037.stdout:mgr.smithi037.wjikii smithi037 *:8443,9283,8765 running (30m) 4m ago 42m 510M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff c5a1417b9417 2024-03-29T08:38:12.678 INFO:teuthology.orchestra.run.smithi037.stdout:mgr.smithi148.onhgvm smithi148 *:8443,9283,8765 running (29m) 4m ago 39m 456M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff fcfdbce062bd 2024-03-29T08:38:12.678 INFO:teuthology.orchestra.run.smithi037.stdout:mon.smithi037 smithi037 running (26m) 4m ago 42m 70.2M 2048M 18.2.2-1015-gd0e392d9 1cc2f45c17ff eea2060783af 2024-03-29T08:38:12.678 INFO:teuthology.orchestra.run.smithi037.stdout:mon.smithi148 smithi148 running (25m) 4m ago 39m 50.4M 2048M 18.2.2-1015-gd0e392d9 1cc2f45c17ff ad4b2c5748be 2024-03-29T08:38:12.678 INFO:teuthology.orchestra.run.smithi037.stdout:node-exporter.smithi037 smithi037 *:9100 running (28m) 4m ago 41m 8514k - 1.5.0 0da6a335fe13 26b8ae9cb878 2024-03-29T08:38:12.679 INFO:teuthology.orchestra.run.smithi037.stdout:node-exporter.smithi148 smithi148 *:9100 running (28m) 4m ago 39m 8464k - 1.5.0 0da6a335fe13 98414e4e64ed 2024-03-29T08:38:12.679 INFO:teuthology.orchestra.run.smithi037.stdout:osd.0 smithi037 running (24m) 4m ago 38m 108M 1619M 18.2.2-1015-gd0e392d9 1cc2f45c17ff c0c584f6245c 2024-03-29T08:38:12.679 INFO:teuthology.orchestra.run.smithi037.stdout:osd.1 smithi037 running (23m) 4m ago 38m 104M 1619M 18.2.2-1015-gd0e392d9 1cc2f45c17ff a4a2ff4cc83d 2024-03-29T08:38:12.679 INFO:teuthology.orchestra.run.smithi037.stdout:osd.2 smithi037 running (23m) 4m ago 37m 85.9M 1619M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 8b1bc71aa79e 2024-03-29T08:38:12.679 INFO:teuthology.orchestra.run.smithi037.stdout:osd.3 smithi148 running (22m) 4m ago 37m 109M 2642M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 1c656ab8ff86 2024-03-29T08:38:12.679 INFO:teuthology.orchestra.run.smithi037.stdout:osd.4 smithi148 running (21m) 4m ago 36m 101M 2642M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 966a9de11d22 2024-03-29T08:38:12.679 INFO:teuthology.orchestra.run.smithi037.stdout:osd.5 smithi148 running (21m) 4m ago 36m 89.4M 2642M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 474c4f9212ce 2024-03-29T08:38:12.679 INFO:teuthology.orchestra.run.smithi037.stdout:prometheus.smithi037 smithi037 *:9095 running (28m) 4m ago 40m 61.0M - 2.43.0 a07b618ecd1d 30c0223ec589 2024-03-29T08:38:12.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:12.676+0000 7f5e36200700 1 -- 172.21.15.37:0/2280514523 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f5e1406cc30 msgr2=0x7f5e1406f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:38:12.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:12.676+0000 7f5e36200700 1 --2- 172.21.15.37:0/2280514523 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f5e1406cc30 0x7f5e1406f0e0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f5e200045a0 tx=0x7f5e20009e30 comp rx=0 tx=0).stop 2024-03-29T08:38:12.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:12.676+0000 7f5e36200700 1 -- 172.21.15.37:0/2280514523 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5e301037a0 msgr2=0x7f5e30116190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:38:12.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:12.676+0000 7f5e36200700 1 --2- 172.21.15.37:0/2280514523 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5e301037a0 0x7f5e30116190 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f5e1c00eb40 tx=0x7f5e1c00b040 comp rx=0 tx=0).stop 2024-03-29T08:38:12.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:12.677+0000 7f5e36200700 1 -- 172.21.15.37:0/2280514523 shutdown_connections 2024-03-29T08:38:12.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:12.677+0000 7f5e36200700 1 --2- 172.21.15.37:0/2280514523 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f5e1406cc30 0x7f5e1406f0e0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:38:12.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:12.677+0000 7f5e36200700 1 --2- 172.21.15.37:0/2280514523 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5e30102e00 0x7f5e30115c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:38:12.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:12.677+0000 7f5e36200700 1 --2- 172.21.15.37:0/2280514523 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5e301037a0 0x7f5e30116190 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:38:12.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:12.677+0000 7f5e36200700 1 -- 172.21.15.37:0/2280514523 >> 172.21.15.37:0/2280514523 conn(0x7f5e300f89b0 msgr2=0x7f5e300f9900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:38:12.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:12.677+0000 7f5e36200700 1 -- 172.21.15.37:0/2280514523 shutdown_connections 2024-03-29T08:38:12.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:12.677+0000 7f5e36200700 1 -- 172.21.15.37:0/2280514523 wait complete. 2024-03-29T08:38:12.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:12.812+0000 7fa564a00700 1 -- 172.21.15.37:0/4036038794 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa5601037a0 msgr2=0x7fa5600fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:38:12.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:12.812+0000 7fa564a00700 1 --2- 172.21.15.37:0/4036038794 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa5601037a0 0x7fa5600fce10 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fa54c004940 tx=0x7fa54c009e30 comp rx=0 tx=0).stop 2024-03-29T08:38:12.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:12.812+0000 7fa564a00700 1 -- 172.21.15.37:0/4036038794 shutdown_connections 2024-03-29T08:38:12.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:12.812+0000 7fa564a00700 1 --2- 172.21.15.37:0/4036038794 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa5601037a0 0x7fa5600fce10 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:38:12.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:12.812+0000 7fa564a00700 1 --2- 172.21.15.37:0/4036038794 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa560102e00 0x7fa5601031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:38:12.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:12.812+0000 7fa564a00700 1 -- 172.21.15.37:0/4036038794 >> 172.21.15.37:0/4036038794 conn(0x7fa5600f89b0 msgr2=0x7fa5600fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:38:12.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:12.812+0000 7fa564a00700 1 -- 172.21.15.37:0/4036038794 shutdown_connections 2024-03-29T08:38:12.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:12.812+0000 7fa564a00700 1 -- 172.21.15.37:0/4036038794 wait complete. 2024-03-29T08:38:12.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:12.813+0000 7fa564a00700 1 Processor -- start 2024-03-29T08:38:12.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:12.813+0000 7fa564a00700 1 -- start start 2024-03-29T08:38:12.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:12.813+0000 7fa564a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa560102e00 0x7fa5600702c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:38:12.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:12.813+0000 7fa564a00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa5601037a0 0x7fa560070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:38:12.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:12.813+0000 7fa564a00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa560070d90 con 0x7fa560102e00 2024-03-29T08:38:12.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:12.813+0000 7fa564a00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa560070ed0 con 0x7fa5601037a0 2024-03-29T08:38:12.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:12.814+0000 7fa55de00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa560102e00 0x7fa5600702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:38:12.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:12.814+0000 7fa55de00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa560102e00 0x7fa5600702c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39612/0 (socket says 172.21.15.37:39612) 2024-03-29T08:38:12.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:12.814+0000 7fa55de00700 1 -- 172.21.15.37:0/2191216721 learned_addr learned my addr 172.21.15.37:0/2191216721 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:38:12.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:12.814+0000 7fa55d400700 1 --2- 172.21.15.37:0/2191216721 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa5601037a0 0x7fa560070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:38:12.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:12.814+0000 7fa55de00700 1 -- 172.21.15.37:0/2191216721 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa5601037a0 msgr2=0x7fa560070800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:38:12.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:12.814+0000 7fa55de00700 1 --2- 172.21.15.37:0/2191216721 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa5601037a0 0x7fa560070800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:38:12.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:12.814+0000 7fa55de00700 1 -- 172.21.15.37:0/2191216721 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa54c005040 con 0x7fa560102e00 2024-03-29T08:38:12.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:12.814+0000 7fa55de00700 1 --2- 172.21.15.37:0/2191216721 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa560102e00 0x7fa5600702c0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fa550002a50 tx=0x7fa5500046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:38:12.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:12.815+0000 7fa556000700 1 -- 172.21.15.37:0/2191216721 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa550007cc0 con 0x7fa560102e00 2024-03-29T08:38:12.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:12.815+0000 7fa564a00700 1 -- 172.21.15.37:0/2191216721 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa560119e00 con 0x7fa560102e00 2024-03-29T08:38:12.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:12.815+0000 7fa556000700 1 -- 172.21.15.37:0/2191216721 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fa55000f040 con 0x7fa560102e00 2024-03-29T08:38:12.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:12.815+0000 7fa556000700 1 -- 172.21.15.37:0/2191216721 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa55000d9c0 con 0x7fa560102e00 2024-03-29T08:38:12.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:12.815+0000 7fa564a00700 1 -- 172.21.15.37:0/2191216721 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa56011a280 con 0x7fa560102e00 2024-03-29T08:38:12.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:12.816+0000 7fa564a00700 1 -- 172.21.15.37:0/2191216721 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5600fe550 con 0x7fa560102e00 2024-03-29T08:38:12.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:12.817+0000 7fa556000700 1 -- 172.21.15.37:0/2191216721 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fa55000db20 con 0x7fa560102e00 2024-03-29T08:38:12.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:12.818+0000 7fa556000700 1 --2- 172.21.15.37:0/2191216721 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fa54406cc30 0x7fa54406f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:38:12.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:12.818+0000 7fa556000700 1 -- 172.21.15.37:0/2191216721 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fa55008bd00 con 0x7fa560102e00 2024-03-29T08:38:12.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:12.818+0000 7fa55d400700 1 --2- 172.21.15.37:0/2191216721 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fa54406cc30 0x7fa54406f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:38:12.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:12.819+0000 7fa55d400700 1 --2- 172.21.15.37:0/2191216721 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fa54406cc30 0x7fa54406f0e0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fa560071c00 tx=0x7fa54c009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:38:12.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:12.823+0000 7fa556000700 1 -- 172.21.15.37:0/2191216721 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fa550055cc0 con 0x7fa560102e00 2024-03-29T08:38:12.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:38:12 smithi037 ceph-mon[142325]: pgmap v820: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:38:13.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:13.130+0000 7fa564a00700 1 -- 172.21.15.37:0/2191216721 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fa5600719b0 con 0x7fa560102e00 2024-03-29T08:38:13.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:13.131+0000 7fa556000700 1 -- 172.21.15.37:0/2191216721 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7fa5500592e0 con 0x7fa560102e00 2024-03-29T08:38:13.132 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-29T08:38:13.132 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": { 2024-03-29T08:38:13.132 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 2 2024-03-29T08:38:13.132 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:38:13.132 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": { 2024-03-29T08:38:13.132 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 2 2024-03-29T08:38:13.132 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:38:13.132 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": { 2024-03-29T08:38:13.132 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 6 2024-03-29T08:38:13.133 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:38:13.133 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": { 2024-03-29T08:38:13.133 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 4 2024-03-29T08:38:13.133 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:38:13.133 INFO:teuthology.orchestra.run.smithi037.stdout: "overall": { 2024-03-29T08:38:13.133 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 14 2024-03-29T08:38:13.133 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-29T08:38:13.133 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-29T08:38:13.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:13.133+0000 7fa564a00700 1 -- 172.21.15.37:0/2191216721 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fa54406cc30 msgr2=0x7fa54406f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:38:13.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:13.133+0000 7fa564a00700 1 --2- 172.21.15.37:0/2191216721 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fa54406cc30 0x7fa54406f0e0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fa560071c00 tx=0x7fa54c009e30 comp rx=0 tx=0).stop 2024-03-29T08:38:13.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:13.133+0000 7fa564a00700 1 -- 172.21.15.37:0/2191216721 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa560102e00 msgr2=0x7fa5600702c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:38:13.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:13.133+0000 7fa564a00700 1 --2- 172.21.15.37:0/2191216721 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa560102e00 0x7fa5600702c0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fa550002a50 tx=0x7fa5500046b0 comp rx=0 tx=0).stop 2024-03-29T08:38:13.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:13.133+0000 7fa564a00700 1 -- 172.21.15.37:0/2191216721 shutdown_connections 2024-03-29T08:38:13.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:13.133+0000 7fa564a00700 1 --2- 172.21.15.37:0/2191216721 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fa54406cc30 0x7fa54406f0e0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:38:13.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:13.133+0000 7fa564a00700 1 --2- 172.21.15.37:0/2191216721 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa560102e00 0x7fa5600702c0 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:38:13.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:13.133+0000 7fa564a00700 1 --2- 172.21.15.37:0/2191216721 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa5601037a0 0x7fa560070800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:38:13.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:13.133+0000 7fa564a00700 1 -- 172.21.15.37:0/2191216721 >> 172.21.15.37:0/2191216721 conn(0x7fa5600f89b0 msgr2=0x7fa5600f9ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:38:13.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:13.133+0000 7fa564a00700 1 -- 172.21.15.37:0/2191216721 shutdown_connections 2024-03-29T08:38:13.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:13.133+0000 7fa564a00700 1 -- 172.21.15.37:0/2191216721 wait complete. 2024-03-29T08:38:13.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:13.270+0000 7fb6da400700 1 -- 172.21.15.37:0/1756572988 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb6d4102c40 msgr2=0x7fb6d4103010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:38:13.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:13.270+0000 7fb6da400700 1 --2- 172.21.15.37:0/1756572988 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb6d4102c40 0x7fb6d4103010 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fb6c8004880 tx=0x7fb6c8009e30 comp rx=0 tx=0).stop 2024-03-29T08:38:13.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:13.270+0000 7fb6da400700 1 -- 172.21.15.37:0/1756572988 shutdown_connections 2024-03-29T08:38:13.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:13.270+0000 7fb6da400700 1 --2- 172.21.15.37:0/1756572988 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb6d41035e0 0x7fb6d40fcc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:38:13.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:13.270+0000 7fb6da400700 1 --2- 172.21.15.37:0/1756572988 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb6d4102c40 0x7fb6d4103010 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:38:13.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:13.270+0000 7fb6da400700 1 -- 172.21.15.37:0/1756572988 >> 172.21.15.37:0/1756572988 conn(0x7fb6d40f8830 msgr2=0x7fb6d40fac40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:38:13.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:13.271+0000 7fb6da400700 1 -- 172.21.15.37:0/1756572988 shutdown_connections 2024-03-29T08:38:13.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:13.271+0000 7fb6da400700 1 -- 172.21.15.37:0/1756572988 wait complete. 2024-03-29T08:38:13.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:13.271+0000 7fb6da400700 1 Processor -- start 2024-03-29T08:38:13.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:13.271+0000 7fb6da400700 1 -- start start 2024-03-29T08:38:13.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:13.272+0000 7fb6da400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb6d4102c40 0x7fb6d41138a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:38:13.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:13.272+0000 7fb6da400700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb6d41035e0 0x7fb6d4113de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:38:13.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:13.272+0000 7fb6da400700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6d4114430 con 0x7fb6d4102c40 2024-03-29T08:38:13.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:13.272+0000 7fb6da400700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6d410d820 con 0x7fb6d41035e0 2024-03-29T08:38:13.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:13.272+0000 7fb6d9000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb6d4102c40 0x7fb6d41138a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:38:13.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:13.272+0000 7fb6d9000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb6d4102c40 0x7fb6d41138a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39634/0 (socket says 172.21.15.37:39634) 2024-03-29T08:38:13.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:13.272+0000 7fb6d9000700 1 -- 172.21.15.37:0/574655855 learned_addr learned my addr 172.21.15.37:0/574655855 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:38:13.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:13.272+0000 7fb6d3e00700 1 --2- 172.21.15.37:0/574655855 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb6d41035e0 0x7fb6d4113de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:38:13.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:13.272+0000 7fb6d9000700 1 -- 172.21.15.37:0/574655855 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb6d41035e0 msgr2=0x7fb6d4113de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:38:13.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:13.272+0000 7fb6d9000700 1 --2- 172.21.15.37:0/574655855 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb6d41035e0 0x7fb6d4113de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:38:13.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:13.272+0000 7fb6d9000700 1 -- 172.21.15.37:0/574655855 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6c8005040 con 0x7fb6d4102c40 2024-03-29T08:38:13.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:13.273+0000 7fb6d9000700 1 --2- 172.21.15.37:0/574655855 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb6d4102c40 0x7fb6d41138a0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fb6c8000c00 tx=0x7fb6c8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:38:13.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:13.273+0000 7fb6d0c00700 1 -- 172.21.15.37:0/574655855 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb6c800e070 con 0x7fb6d4102c40 2024-03-29T08:38:13.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:13.273+0000 7fb6da400700 1 -- 172.21.15.37:0/574655855 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6d410daa0 con 0x7fb6d4102c40 2024-03-29T08:38:13.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:13.273+0000 7fb6d0c00700 1 -- 172.21.15.37:0/574655855 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fb6c8013dd0 con 0x7fb6d4102c40 2024-03-29T08:38:13.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:13.273+0000 7fb6d0c00700 1 -- 172.21.15.37:0/574655855 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb6c801ce00 con 0x7fb6d4102c40 2024-03-29T08:38:13.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:13.273+0000 7fb6da400700 1 -- 172.21.15.37:0/574655855 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6d410e000 con 0x7fb6d4102c40 2024-03-29T08:38:13.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:13.274+0000 7fb6da400700 1 -- 172.21.15.37:0/574655855 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6d40fe390 con 0x7fb6d4102c40 2024-03-29T08:38:13.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:13.275+0000 7fb6d0c00700 1 -- 172.21.15.37:0/574655855 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fb6c8013920 con 0x7fb6d4102c40 2024-03-29T08:38:13.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:13.277+0000 7fb6d0c00700 1 --2- 172.21.15.37:0/574655855 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fb6bc06cc30 0x7fb6bc06f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:38:13.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:13.277+0000 7fb6d0c00700 1 -- 172.21.15.37:0/574655855 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fb6c808d690 con 0x7fb6d4102c40 2024-03-29T08:38:13.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:13.277+0000 7fb6d3e00700 1 --2- 172.21.15.37:0/574655855 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fb6bc06cc30 0x7fb6bc06f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:38:13.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:13.277+0000 7fb6d3e00700 1 --2- 172.21.15.37:0/574655855 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fb6bc06cc30 0x7fb6bc06f0e0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fb6c00046c0 tx=0x7fb6c000b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:38:13.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:13.281+0000 7fb6d0c00700 1 -- 172.21.15.37:0/574655855 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fb6c805b0e0 con 0x7fb6d4102c40 2024-03-29T08:38:13.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:13.544+0000 7fb6da400700 1 -- 172.21.15.37:0/574655855 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fb6d4109bf0 con 0x7fb6d4102c40 2024-03-29T08:38:13.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:13.544+0000 7fb6d0c00700 1 -- 172.21.15.37:0/574655855 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 30 v30) v1 ==== 76+0+1841 (secure 0 0 0) 0x7fb6c805ac70 con 0x7fb6d4102c40 2024-03-29T08:38:13.545 INFO:teuthology.orchestra.run.smithi037.stdout:e30 2024-03-29T08:38:13.545 INFO:teuthology.orchestra.run.smithi037.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-29T08:38:13.545 INFO:teuthology.orchestra.run.smithi037.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T08:38:13.545 INFO:teuthology.orchestra.run.smithi037.stdout:legacy client fscid: 1 2024-03-29T08:38:13.545 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:38:13.546 INFO:teuthology.orchestra.run.smithi037.stdout:Filesystem 'cephfs' (1) 2024-03-29T08:38:13.546 INFO:teuthology.orchestra.run.smithi037.stdout:fs_name cephfs 2024-03-29T08:38:13.546 INFO:teuthology.orchestra.run.smithi037.stdout:epoch 21 2024-03-29T08:38:13.546 INFO:teuthology.orchestra.run.smithi037.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-03-29T08:38:13.546 INFO:teuthology.orchestra.run.smithi037.stdout:created 2024-03-29T08:03:29.241269+0000 2024-03-29T08:38:13.546 INFO:teuthology.orchestra.run.smithi037.stdout:modified 2024-03-29T08:17:16.237238+0000 2024-03-29T08:38:13.546 INFO:teuthology.orchestra.run.smithi037.stdout:tableserver 0 2024-03-29T08:38:13.546 INFO:teuthology.orchestra.run.smithi037.stdout:root 0 2024-03-29T08:38:13.546 INFO:teuthology.orchestra.run.smithi037.stdout:session_timeout 60 2024-03-29T08:38:13.546 INFO:teuthology.orchestra.run.smithi037.stdout:session_autoclose 300 2024-03-29T08:38:13.546 INFO:teuthology.orchestra.run.smithi037.stdout:max_file_size 1099511627776 2024-03-29T08:38:13.546 INFO:teuthology.orchestra.run.smithi037.stdout:max_xattr_size 65536 2024-03-29T08:38:13.546 INFO:teuthology.orchestra.run.smithi037.stdout:required_client_features {} 2024-03-29T08:38:13.546 INFO:teuthology.orchestra.run.smithi037.stdout:last_failure 0 2024-03-29T08:38:13.546 INFO:teuthology.orchestra.run.smithi037.stdout:last_failure_osd_epoch 82 2024-03-29T08:38:13.547 INFO:teuthology.orchestra.run.smithi037.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T08:38:13.547 INFO:teuthology.orchestra.run.smithi037.stdout:max_mds 2 2024-03-29T08:38:13.547 INFO:teuthology.orchestra.run.smithi037.stdout:in 0,1 2024-03-29T08:38:13.547 INFO:teuthology.orchestra.run.smithi037.stdout:up {} 2024-03-29T08:38:13.547 INFO:teuthology.orchestra.run.smithi037.stdout:failed 0,1 2024-03-29T08:38:13.547 INFO:teuthology.orchestra.run.smithi037.stdout:damaged 2024-03-29T08:38:13.547 INFO:teuthology.orchestra.run.smithi037.stdout:stopped 2024-03-29T08:38:13.547 INFO:teuthology.orchestra.run.smithi037.stdout:data_pools [3] 2024-03-29T08:38:13.547 INFO:teuthology.orchestra.run.smithi037.stdout:metadata_pool 2 2024-03-29T08:38:13.547 INFO:teuthology.orchestra.run.smithi037.stdout:inline_data disabled 2024-03-29T08:38:13.547 INFO:teuthology.orchestra.run.smithi037.stdout:balancer 2024-03-29T08:38:13.547 INFO:teuthology.orchestra.run.smithi037.stdout:bal_rank_mask -1 2024-03-29T08:38:13.547 INFO:teuthology.orchestra.run.smithi037.stdout:standby_count_wanted 1 2024-03-29T08:38:13.547 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:38:13.547 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:38:13.548 INFO:teuthology.orchestra.run.smithi037.stdout:Standby daemons: 2024-03-29T08:38:13.548 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:38:13.548 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi037.mmwgxe{-1:34440} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.37:6828/1248142005,v1:172.21.15.37:6829/1248142005] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:38:13.548 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi037.iifsmh{-1:34444} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.37:6826/2060882959,v1:172.21.15.37:6827/2060882959] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:38:13.548 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi148.sqteih{-1:44395} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.148:6824/4078652242,v1:172.21.15.148:6825/4078652242] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:38:13.548 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi148.qukpjh{-1:45007} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.148:6826/2394751105,v1:172.21.15.148:6827/2394751105] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:38:13.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:13.547+0000 7fb6da400700 1 -- 172.21.15.37:0/574655855 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fb6bc06cc30 msgr2=0x7fb6bc06f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:38:13.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:13.547+0000 7fb6da400700 1 --2- 172.21.15.37:0/574655855 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fb6bc06cc30 0x7fb6bc06f0e0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fb6c00046c0 tx=0x7fb6c000b040 comp rx=0 tx=0).stop 2024-03-29T08:38:13.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:13.547+0000 7fb6da400700 1 -- 172.21.15.37:0/574655855 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb6d4102c40 msgr2=0x7fb6d41138a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:38:13.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:13.547+0000 7fb6da400700 1 --2- 172.21.15.37:0/574655855 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb6d4102c40 0x7fb6d41138a0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fb6c8000c00 tx=0x7fb6c8009e30 comp rx=0 tx=0).stop 2024-03-29T08:38:13.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:13.547+0000 7fb6da400700 1 -- 172.21.15.37:0/574655855 shutdown_connections 2024-03-29T08:38:13.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:13.547+0000 7fb6da400700 1 --2- 172.21.15.37:0/574655855 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fb6bc06cc30 0x7fb6bc06f0e0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:38:13.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:13.547+0000 7fb6da400700 1 --2- 172.21.15.37:0/574655855 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb6d4102c40 0x7fb6d41138a0 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:38:13.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:13.547+0000 7fb6da400700 1 --2- 172.21.15.37:0/574655855 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb6d41035e0 0x7fb6d4113de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:38:13.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:13.547+0000 7fb6da400700 1 -- 172.21.15.37:0/574655855 >> 172.21.15.37:0/574655855 conn(0x7fb6d40f8830 msgr2=0x7fb6d40f98d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:38:13.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:13.547+0000 7fb6da400700 1 -- 172.21.15.37:0/574655855 shutdown_connections 2024-03-29T08:38:13.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:13.547+0000 7fb6da400700 1 -- 172.21.15.37:0/574655855 wait complete. 2024-03-29T08:38:13.550 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 30 2024-03-29T08:38:13.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:13.685+0000 7f3bdf400700 1 -- 172.21.15.37:0/1651696960 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3bd81037a0 msgr2=0x7f3bd80fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:38:13.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:13.685+0000 7f3bdf400700 1 --2- 172.21.15.37:0/1651696960 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3bd81037a0 0x7f3bd80fce10 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f3bc4004940 tx=0x7f3bc4009e30 comp rx=0 tx=0).stop 2024-03-29T08:38:13.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:13.685+0000 7f3bdf400700 1 -- 172.21.15.37:0/1651696960 shutdown_connections 2024-03-29T08:38:13.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:13.685+0000 7f3bdf400700 1 --2- 172.21.15.37:0/1651696960 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3bd81037a0 0x7f3bd80fce10 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:38:13.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:13.685+0000 7f3bdf400700 1 --2- 172.21.15.37:0/1651696960 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3bd8102e00 0x7f3bd81031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:38:13.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:13.685+0000 7f3bdf400700 1 -- 172.21.15.37:0/1651696960 >> 172.21.15.37:0/1651696960 conn(0x7f3bd80f89b0 msgr2=0x7f3bd80fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:38:13.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:13.685+0000 7f3bdf400700 1 -- 172.21.15.37:0/1651696960 shutdown_connections 2024-03-29T08:38:13.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:13.685+0000 7f3bdf400700 1 -- 172.21.15.37:0/1651696960 wait complete. 2024-03-29T08:38:13.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:13.686+0000 7f3bdf400700 1 Processor -- start 2024-03-29T08:38:13.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:13.686+0000 7f3bdf400700 1 -- start start 2024-03-29T08:38:13.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:13.687+0000 7f3bdf400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3bd8102e00 0x7f3bd8115d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:38:13.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:13.687+0000 7f3bdf400700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3bd81037a0 0x7f3bd8116260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:38:13.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:13.687+0000 7f3bdf400700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bd81168b0 con 0x7f3bd8102e00 2024-03-29T08:38:13.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:13.687+0000 7f3bdf400700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bd810fc80 con 0x7f3bd81037a0 2024-03-29T08:38:13.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:13.687+0000 7f3bdca00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3bd8102e00 0x7f3bd8115d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:38:13.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:13.687+0000 7f3bdca00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3bd8102e00 0x7f3bd8115d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39656/0 (socket says 172.21.15.37:39656) 2024-03-29T08:38:13.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:13.687+0000 7f3bdca00700 1 -- 172.21.15.37:0/778550758 learned_addr learned my addr 172.21.15.37:0/778550758 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:38:13.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:13.687+0000 7f3bd7e00700 1 --2- 172.21.15.37:0/778550758 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3bd81037a0 0x7f3bd8116260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:38:13.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:13.687+0000 7f3bdca00700 1 -- 172.21.15.37:0/778550758 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3bd81037a0 msgr2=0x7f3bd8116260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:38:13.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:13.687+0000 7f3bdca00700 1 --2- 172.21.15.37:0/778550758 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3bd81037a0 0x7f3bd8116260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:38:13.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:13.687+0000 7f3bdca00700 1 -- 172.21.15.37:0/778550758 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bc4005040 con 0x7f3bd8102e00 2024-03-29T08:38:13.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:13.688+0000 7f3bdca00700 1 --2- 172.21.15.37:0/778550758 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3bd8102e00 0x7f3bd8115d20 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f3bcc002a50 tx=0x7f3bcc0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:38:13.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:13.688+0000 7f3bd4c00700 1 -- 172.21.15.37:0/778550758 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3bcc007cc0 con 0x7f3bd8102e00 2024-03-29T08:38:13.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:13.688+0000 7f3bd4c00700 1 -- 172.21.15.37:0/778550758 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f3bcc00f040 con 0x7f3bd8102e00 2024-03-29T08:38:13.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:13.688+0000 7f3bdf400700 1 -- 172.21.15.37:0/778550758 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bd810ff60 con 0x7f3bd8102e00 2024-03-29T08:38:13.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:13.688+0000 7f3bd4c00700 1 -- 172.21.15.37:0/778550758 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3bcc009540 con 0x7f3bd8102e00 2024-03-29T08:38:13.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:13.688+0000 7f3bdf400700 1 -- 172.21.15.37:0/778550758 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bd8110520 con 0x7f3bd8102e00 2024-03-29T08:38:13.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:13.689+0000 7f3bdf400700 1 -- 172.21.15.37:0/778550758 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3bd80fe550 con 0x7f3bd8102e00 2024-03-29T08:38:13.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:13.690+0000 7f3bd4c00700 1 -- 172.21.15.37:0/778550758 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f3bcc0096a0 con 0x7f3bd8102e00 2024-03-29T08:38:13.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:13.691+0000 7f3bd4c00700 1 --2- 172.21.15.37:0/778550758 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f3bc006cc30 0x7f3bc006f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:38:13.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:13.691+0000 7f3bd4c00700 1 -- 172.21.15.37:0/778550758 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f3bcc08bee0 con 0x7f3bd8102e00 2024-03-29T08:38:13.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:13.691+0000 7f3bd7e00700 1 --2- 172.21.15.37:0/778550758 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f3bc006cc30 0x7f3bc006f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:38:13.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:13.692+0000 7f3bd7e00700 1 --2- 172.21.15.37:0/778550758 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f3bc006cc30 0x7f3bc006f0e0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f3bd81115c0 tx=0x7f3bc4009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:38:13.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:13.696+0000 7f3bd4c00700 1 -- 172.21.15.37:0/778550758 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f3bcc0598b0 con 0x7f3bd8102e00 2024-03-29T08:38:13.807 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:38:13 smithi148 ceph-mon[106692]: from='client.35486 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:38:13.808 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:38:13 smithi148 ceph-mon[106692]: from='client.35490 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:38:13.808 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:38:13 smithi148 ceph-mon[106692]: from='client.35494 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:38:13.808 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:38:13 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/2191216721' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:38:13.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:38:13 smithi037 ceph-mon[142325]: from='client.35486 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:38:13.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:38:13 smithi037 ceph-mon[142325]: from='client.35490 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:38:13.928 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:38:13 smithi037 ceph-mon[142325]: from='client.35494 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:38:13.928 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:38:13 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/2191216721' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:38:13.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:13.929+0000 7f3bdf400700 1 -- 172.21.15.37:0/778550758 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3bd804bf40 con 0x7f3bc006cc30 2024-03-29T08:38:13.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:13.931+0000 7f3bd4c00700 1 -- 172.21.15.37:0/778550758 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f3bd804bf40 con 0x7f3bc006cc30 2024-03-29T08:38:13.931 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-29T08:38:13.932 INFO:teuthology.orchestra.run.smithi037.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59", 2024-03-29T08:38:13.932 INFO:teuthology.orchestra.run.smithi037.stdout: "in_progress": true, 2024-03-29T08:38:13.932 INFO:teuthology.orchestra.run.smithi037.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-29T08:38:13.932 INFO:teuthology.orchestra.run.smithi037.stdout: "services_complete": [ 2024-03-29T08:38:13.932 INFO:teuthology.orchestra.run.smithi037.stdout: "crash", 2024-03-29T08:38:13.932 INFO:teuthology.orchestra.run.smithi037.stdout: "osd", 2024-03-29T08:38:13.932 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr", 2024-03-29T08:38:13.932 INFO:teuthology.orchestra.run.smithi037.stdout: "mds", 2024-03-29T08:38:13.932 INFO:teuthology.orchestra.run.smithi037.stdout: "mon" 2024-03-29T08:38:13.932 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-29T08:38:13.932 INFO:teuthology.orchestra.run.smithi037.stdout: "progress": "16/21 daemons upgraded", 2024-03-29T08:38:13.932 INFO:teuthology.orchestra.run.smithi037.stdout: "message": "Currently upgrading mds daemons", 2024-03-29T08:38:13.932 INFO:teuthology.orchestra.run.smithi037.stdout: "is_paused": false 2024-03-29T08:38:13.932 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-29T08:38:13.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:13.933+0000 7f3bdf400700 1 -- 172.21.15.37:0/778550758 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f3bc006cc30 msgr2=0x7f3bc006f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:38:13.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:13.933+0000 7f3bdf400700 1 --2- 172.21.15.37:0/778550758 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f3bc006cc30 0x7f3bc006f0e0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f3bd81115c0 tx=0x7f3bc4009e30 comp rx=0 tx=0).stop 2024-03-29T08:38:13.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:13.933+0000 7f3bdf400700 1 -- 172.21.15.37:0/778550758 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3bd8102e00 msgr2=0x7f3bd8115d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:38:13.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:13.933+0000 7f3bdf400700 1 --2- 172.21.15.37:0/778550758 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3bd8102e00 0x7f3bd8115d20 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f3bcc002a50 tx=0x7f3bcc0046b0 comp rx=0 tx=0).stop 2024-03-29T08:38:13.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:13.933+0000 7f3bdf400700 1 -- 172.21.15.37:0/778550758 shutdown_connections 2024-03-29T08:38:13.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:13.933+0000 7f3bdf400700 1 --2- 172.21.15.37:0/778550758 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f3bc006cc30 0x7f3bc006f0e0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:38:13.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:13.933+0000 7f3bdf400700 1 --2- 172.21.15.37:0/778550758 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3bd8102e00 0x7f3bd8115d20 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:38:13.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:13.933+0000 7f3bdf400700 1 --2- 172.21.15.37:0/778550758 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3bd81037a0 0x7f3bd8116260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:38:13.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:13.933+0000 7f3bdf400700 1 -- 172.21.15.37:0/778550758 >> 172.21.15.37:0/778550758 conn(0x7f3bd80f89b0 msgr2=0x7f3bd80f99f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:38:13.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:13.933+0000 7f3bdf400700 1 -- 172.21.15.37:0/778550758 shutdown_connections 2024-03-29T08:38:13.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:13.933+0000 7f3bdf400700 1 -- 172.21.15.37:0/778550758 wait complete. 2024-03-29T08:38:14.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:14.070+0000 7f2700400700 1 -- 172.21.15.37:0/157040392 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f26f81037a0 msgr2=0x7f26f80fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:38:14.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:14.070+0000 7f2700400700 1 --2- 172.21.15.37:0/157040392 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f26f81037a0 0x7f26f80fce10 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f26e4004940 tx=0x7f26e4009e30 comp rx=0 tx=0).stop 2024-03-29T08:38:14.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:14.070+0000 7f2700400700 1 -- 172.21.15.37:0/157040392 shutdown_connections 2024-03-29T08:38:14.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:14.070+0000 7f2700400700 1 --2- 172.21.15.37:0/157040392 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f26f81037a0 0x7f26f80fce10 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:38:14.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:14.070+0000 7f2700400700 1 --2- 172.21.15.37:0/157040392 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f26f8102e00 0x7f26f81031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:38:14.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:14.070+0000 7f2700400700 1 -- 172.21.15.37:0/157040392 >> 172.21.15.37:0/157040392 conn(0x7f26f80f89b0 msgr2=0x7f26f80fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:38:14.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:14.071+0000 7f2700400700 1 -- 172.21.15.37:0/157040392 shutdown_connections 2024-03-29T08:38:14.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:14.071+0000 7f2700400700 1 -- 172.21.15.37:0/157040392 wait complete. 2024-03-29T08:38:14.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:14.071+0000 7f2700400700 1 Processor -- start 2024-03-29T08:38:14.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:14.071+0000 7f2700400700 1 -- start start 2024-03-29T08:38:14.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:14.072+0000 7f2700400700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f26f8102e00 0x7f26f8115cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:38:14.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:14.072+0000 7f2700400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f26f81037a0 0x7f26f8116230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:38:14.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:14.072+0000 7f2700400700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26f8116880 con 0x7f26f81037a0 2024-03-29T08:38:14.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:14.072+0000 7f2700400700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26f810fc50 con 0x7f26f8102e00 2024-03-29T08:38:14.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:14.072+0000 7f26fd000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f26f81037a0 0x7f26f8116230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:38:14.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:14.072+0000 7f26fd000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f26f81037a0 0x7f26f8116230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39676/0 (socket says 172.21.15.37:39676) 2024-03-29T08:38:14.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:14.072+0000 7f26fd000700 1 -- 172.21.15.37:0/1480786103 learned_addr learned my addr 172.21.15.37:0/1480786103 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:38:14.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:14.073+0000 7f26fda00700 1 --2- 172.21.15.37:0/1480786103 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f26f8102e00 0x7f26f8115cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:38:14.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:14.073+0000 7f26fd000700 1 -- 172.21.15.37:0/1480786103 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f26f8102e00 msgr2=0x7f26f8115cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:38:14.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:14.073+0000 7f26fd000700 1 --2- 172.21.15.37:0/1480786103 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f26f8102e00 0x7f26f8115cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:38:14.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:14.073+0000 7f26fd000700 1 -- 172.21.15.37:0/1480786103 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26e4005040 con 0x7f26f81037a0 2024-03-29T08:38:14.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:14.073+0000 7f26fda00700 1 --2- 172.21.15.37:0/1480786103 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f26f8102e00 0x7f26f8115cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:38:14.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:14.073+0000 7f26fd000700 1 --2- 172.21.15.37:0/1480786103 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f26f81037a0 0x7f26f8116230 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f26e40093b0 tx=0x7f26e4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:38:14.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:14.073+0000 7f26f5600700 1 -- 172.21.15.37:0/1480786103 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f26e400e040 con 0x7f26f81037a0 2024-03-29T08:38:14.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:14.073+0000 7f26f5600700 1 -- 172.21.15.37:0/1480786103 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f26e4013420 con 0x7f26f81037a0 2024-03-29T08:38:14.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:14.073+0000 7f2700400700 1 -- 172.21.15.37:0/1480786103 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26f810fed0 con 0x7f26f81037a0 2024-03-29T08:38:14.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:14.073+0000 7f26f5600700 1 -- 172.21.15.37:0/1480786103 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f26e401c630 con 0x7f26f81037a0 2024-03-29T08:38:14.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:14.073+0000 7f2700400700 1 -- 172.21.15.37:0/1480786103 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26f8110490 con 0x7f26f81037a0 2024-03-29T08:38:14.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:14.074+0000 7f2700400700 1 -- 172.21.15.37:0/1480786103 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26f80fe550 con 0x7f26f81037a0 2024-03-29T08:38:14.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:14.075+0000 7f26f5600700 1 -- 172.21.15.37:0/1480786103 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f26e40139e0 con 0x7f26f81037a0 2024-03-29T08:38:14.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:14.076+0000 7f26f5600700 1 --2- 172.21.15.37:0/1480786103 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f26e006cc20 0x7f26e006f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:38:14.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:14.076+0000 7f26f5600700 1 -- 172.21.15.37:0/1480786103 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f26e408d300 con 0x7f26f81037a0 2024-03-29T08:38:14.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:14.076+0000 7f26fda00700 1 --2- 172.21.15.37:0/1480786103 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f26e006cc20 0x7f26e006f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:38:14.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:14.077+0000 7f26fda00700 1 --2- 172.21.15.37:0/1480786103 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f26e006cc20 0x7f26e006f0d0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f26ec009fd0 tx=0x7f26ec00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:38:14.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:14.081+0000 7f26f5600700 1 -- 172.21.15.37:0/1480786103 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f26e405ad50 con 0x7f26f81037a0 2024-03-29T08:38:14.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:14.383+0000 7f2700400700 1 -- 172.21.15.37:0/1480786103 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f26f8109da0 con 0x7f26f81037a0 2024-03-29T08:38:14.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:14.383+0000 7f26f5600700 1 -- 172.21.15.37:0/1480786103 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+3583 (secure 0 0 0) 0x7f26e405a8e0 con 0x7f26f81037a0 2024-03-29T08:38:14.384 INFO:teuthology.orchestra.run.smithi037.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem is offline 2024-03-29T08:38:14.384 INFO:teuthology.orchestra.run.smithi037.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-03-29T08:38:14.384 INFO:teuthology.orchestra.run.smithi037.stdout: host smithi148 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/mon.smithi148/config 2024-03-29T08:38:14.384 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/podman: timeout after 895 seconds 2024-03-29T08:38:14.384 INFO:teuthology.orchestra.run.smithi037.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59 -e NODE_NAME=smithi148 -e CEPH_USE_RANDOM_NONCE=1 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4:/var/run/ceph:z -v /var/log/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4:/var/log/ceph:z -v /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpkx2rz2d4:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59 inventory --format=json-pretty --filter-for-batch 2024-03-29T08:38:14.384 INFO:teuthology.orchestra.run.smithi037.stdout:Traceback (most recent call last): 2024-03-29T08:38:14.384 INFO:teuthology.orchestra.run.smithi037.stdout: File "/usr/lib64/python3.6/runpy.py", line 193, in _run_module_as_main 2024-03-29T08:38:14.384 INFO:teuthology.orchestra.run.smithi037.stdout: "__main__", mod_spec) 2024-03-29T08:38:14.385 INFO:teuthology.orchestra.run.smithi037.stdout: File "/usr/lib64/python3.6/runpy.py", line 85, in _run_code 2024-03-29T08:38:14.385 INFO:teuthology.orchestra.run.smithi037.stdout: exec(code, run_globals) 2024-03-29T08:38:14.385 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 10889, in 2024-03-29T08:38:14.385 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 10877, in main 2024-03-29T08:38:14.385 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 2576, in _infer_config 2024-03-29T08:38:14.385 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 2492, in _infer_fsid 2024-03-29T08:38:14.385 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 2604, in _infer_image 2024-03-29T08:38:14.385 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 2479, in _validate_fsid 2024-03-29T08:38:14.385 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 7145, in command_ceph_volume 2024-03-29T08:38:14.385 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 2267, in call_throws 2024-03-29T08:38:14.385 INFO:teuthology.orchestra.run.smithi037.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59 -e NODE_NAME=smithi148 -e CEPH_USE_RANDOM_NONCE=1 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4:/var/run/ceph:z -v /var/log/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4:/var/log/ceph:z -v /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpkx2rz2d4:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59 inventory --format=json-pretty --filter-for-batch 2024-03-29T08:38:14.385 INFO:teuthology.orchestra.run.smithi037.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-03-29T08:38:14.385 INFO:teuthology.orchestra.run.smithi037.stdout: fs cephfs is degraded 2024-03-29T08:38:14.385 INFO:teuthology.orchestra.run.smithi037.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-03-29T08:38:14.385 INFO:teuthology.orchestra.run.smithi037.stdout: fs cephfs is offline because no MDS is active for it. 2024-03-29T08:38:14.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:14.385+0000 7f2700400700 1 -- 172.21.15.37:0/1480786103 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f26e006cc20 msgr2=0x7f26e006f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:38:14.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:14.385+0000 7f2700400700 1 --2- 172.21.15.37:0/1480786103 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f26e006cc20 0x7f26e006f0d0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f26ec009fd0 tx=0x7f26ec00b040 comp rx=0 tx=0).stop 2024-03-29T08:38:14.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:14.385+0000 7f2700400700 1 -- 172.21.15.37:0/1480786103 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f26f81037a0 msgr2=0x7f26f8116230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:38:14.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:14.385+0000 7f2700400700 1 --2- 172.21.15.37:0/1480786103 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f26f81037a0 0x7f26f8116230 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f26e40093b0 tx=0x7f26e4009e30 comp rx=0 tx=0).stop 2024-03-29T08:38:14.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:14.385+0000 7f2700400700 1 -- 172.21.15.37:0/1480786103 shutdown_connections 2024-03-29T08:38:14.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:14.385+0000 7f2700400700 1 --2- 172.21.15.37:0/1480786103 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f26e006cc20 0x7f26e006f0d0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:38:14.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:14.385+0000 7f2700400700 1 --2- 172.21.15.37:0/1480786103 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f26f8102e00 0x7f26f8115cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:38:14.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:14.385+0000 7f2700400700 1 --2- 172.21.15.37:0/1480786103 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f26f81037a0 0x7f26f8116230 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:38:14.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:14.385+0000 7f2700400700 1 -- 172.21.15.37:0/1480786103 >> 172.21.15.37:0/1480786103 conn(0x7f26f80f89b0 msgr2=0x7f26f80f9a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:38:14.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:14.386+0000 7f2700400700 1 -- 172.21.15.37:0/1480786103 shutdown_connections 2024-03-29T08:38:14.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:14.386+0000 7f2700400700 1 -- 172.21.15.37:0/1480786103 wait complete. 2024-03-29T08:38:14.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:38:14 smithi148 ceph-mon[106692]: pgmap v821: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:38:14.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:38:14 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/574655855' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T08:38:14.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:38:14 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/1480786103' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T08:38:14.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:38:14 smithi037 ceph-mon[142325]: pgmap v821: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:38:14.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:38:14 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/574655855' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T08:38:14.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:38:14 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/1480786103' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T08:38:15.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:38:15 smithi148 ceph-mon[106692]: from='client.35506 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:38:15.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:38:15 smithi037 ceph-mon[142325]: from='client.35506 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:38:16.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:38:16 smithi148 ceph-mon[106692]: pgmap v822: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:38:16.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:38:16 smithi037 ceph-mon[142325]: pgmap v822: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:38:18.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:38:18 smithi148 ceph-mon[106692]: pgmap v823: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:38:18.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:38:18 smithi037 ceph-mon[142325]: pgmap v823: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:38:20.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:38:20 smithi148 ceph-mon[106692]: pgmap v824: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:38:20.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:38:20 smithi037 ceph-mon[142325]: pgmap v824: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:38:22.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:38:22 smithi148 ceph-mon[106692]: pgmap v825: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:38:22.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:38:22 smithi037 ceph-mon[142325]: pgmap v825: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:38:24.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:38:24 smithi148 ceph-mon[106692]: pgmap v826: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:38:24.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:38:24 smithi037 ceph-mon[142325]: pgmap v826: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:38:25.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:38:25 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:38:26.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:38:25 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:38:26.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:38:26 smithi037 ceph-mon[142325]: pgmap v827: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:38:27.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:38:26 smithi148 ceph-mon[106692]: pgmap v827: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:38:28.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:38:28 smithi037 ceph-mon[142325]: pgmap v828: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:38:29.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:38:28 smithi148 ceph-mon[106692]: pgmap v828: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:38:30.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:38:30 smithi037 ceph-mon[142325]: pgmap v829: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:38:31.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:38:30 smithi148 ceph-mon[106692]: pgmap v829: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:38:32.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:38:32 smithi037 ceph-mon[142325]: pgmap v830: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:38:33.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:38:32 smithi148 ceph-mon[106692]: pgmap v830: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:38:34.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:38:34 smithi037 ceph-mon[142325]: pgmap v831: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:38:35.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:38:34 smithi148 ceph-mon[106692]: pgmap v831: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:38:36.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:38:36 smithi037 ceph-mon[142325]: pgmap v832: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:38:37.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:38:36 smithi148 ceph-mon[106692]: pgmap v832: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:38:38.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:38:38 smithi037 ceph-mon[142325]: pgmap v833: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:38:39.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:38:38 smithi148 ceph-mon[106692]: pgmap v833: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:38:40.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:38:40 smithi037 ceph-mon[142325]: pgmap v834: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:38:40.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:38:40 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:38:41.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:38:40 smithi148 ceph-mon[106692]: pgmap v834: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:38:41.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:38:40 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:38:42.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:38:42 smithi037 ceph-mon[142325]: pgmap v835: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:38:43.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:38:42 smithi148 ceph-mon[106692]: pgmap v835: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:38:44.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:44.539+0000 7f732a200700 1 -- 172.21.15.37:0/2073590387 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f73240fe830 msgr2=0x7f7324106f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:38:44.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:44.539+0000 7f732a200700 1 --2- 172.21.15.37:0/2073590387 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f73240fe830 0x7f7324106f00 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f7310004940 tx=0x7f7310009e30 comp rx=0 tx=0).stop 2024-03-29T08:38:44.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:44.539+0000 7f732a200700 1 -- 172.21.15.37:0/2073590387 shutdown_connections 2024-03-29T08:38:44.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:44.539+0000 7f732a200700 1 --2- 172.21.15.37:0/2073590387 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f73240fe830 0x7f7324106f00 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:38:44.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:44.539+0000 7f732a200700 1 --2- 172.21.15.37:0/2073590387 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f73240fde90 0x7f73240fe260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:38:44.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:44.539+0000 7f732a200700 1 -- 172.21.15.37:0/2073590387 >> 172.21.15.37:0/2073590387 conn(0x7f7324072bc0 msgr2=0x7f7324072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:38:44.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:44.539+0000 7f732a200700 1 -- 172.21.15.37:0/2073590387 shutdown_connections 2024-03-29T08:38:44.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:44.539+0000 7f732a200700 1 -- 172.21.15.37:0/2073590387 wait complete. 2024-03-29T08:38:44.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:44.540+0000 7f732a200700 1 Processor -- start 2024-03-29T08:38:44.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:44.540+0000 7f732a200700 1 -- start start 2024-03-29T08:38:44.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:44.541+0000 7f732a200700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f73240fde90 0x7f7324115c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:38:44.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:44.541+0000 7f732a200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f73240fe830 0x7f73241161d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:38:44.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:44.541+0000 7f732a200700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7324116820 con 0x7f73240fe830 2024-03-29T08:38:44.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:44.541+0000 7f732a200700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f732410fbf0 con 0x7f73240fde90 2024-03-29T08:38:44.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:44.541+0000 7f7322a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f73240fe830 0x7f73241161d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:38:44.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:44.541+0000 7f7322a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f73240fe830 0x7f73241161d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35432/0 (socket says 172.21.15.37:35432) 2024-03-29T08:38:44.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:44.541+0000 7f7322a00700 1 -- 172.21.15.37:0/4157585707 learned_addr learned my addr 172.21.15.37:0/4157585707 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:38:44.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:44.541+0000 7f7323400700 1 --2- 172.21.15.37:0/4157585707 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f73240fde90 0x7f7324115c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:38:44.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:44.541+0000 7f7322a00700 1 -- 172.21.15.37:0/4157585707 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f73240fde90 msgr2=0x7f7324115c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:38:44.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:44.541+0000 7f7322a00700 1 --2- 172.21.15.37:0/4157585707 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f73240fde90 0x7f7324115c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:38:44.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:44.541+0000 7f7322a00700 1 -- 172.21.15.37:0/4157585707 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7310005040 con 0x7f73240fe830 2024-03-29T08:38:44.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:44.542+0000 7f7322a00700 1 --2- 172.21.15.37:0/4157585707 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f73240fe830 0x7f73241161d0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f7310000c00 tx=0x7f7310009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:38:44.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:44.542+0000 7f731b400700 1 -- 172.21.15.37:0/4157585707 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7310021070 con 0x7f73240fe830 2024-03-29T08:38:44.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:44.542+0000 7f731b400700 1 -- 172.21.15.37:0/4157585707 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f7310013e00 con 0x7f73240fe830 2024-03-29T08:38:44.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:44.542+0000 7f731b400700 1 -- 172.21.15.37:0/4157585707 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f731000e070 con 0x7f73240fe830 2024-03-29T08:38:44.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:44.542+0000 7f732a200700 1 -- 172.21.15.37:0/4157585707 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f732410fe70 con 0x7f73240fe830 2024-03-29T08:38:44.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:44.542+0000 7f732a200700 1 -- 172.21.15.37:0/4157585707 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73241103b0 con 0x7f73240fe830 2024-03-29T08:38:44.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:44.543+0000 7f732a200700 1 -- 172.21.15.37:0/4157585707 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7324106a20 con 0x7f73240fe830 2024-03-29T08:38:44.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:44.544+0000 7f731b400700 1 -- 172.21.15.37:0/4157585707 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f7310013950 con 0x7f73240fe830 2024-03-29T08:38:44.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:44.545+0000 7f731b400700 1 --2- 172.21.15.37:0/4157585707 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f730806cc30 0x7f730806f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:38:44.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:44.545+0000 7f731b400700 1 -- 172.21.15.37:0/4157585707 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f731008cff0 con 0x7f73240fe830 2024-03-29T08:38:44.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:44.546+0000 7f7323400700 1 --2- 172.21.15.37:0/4157585707 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f730806cc30 0x7f730806f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:38:44.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:44.546+0000 7f7323400700 1 --2- 172.21.15.37:0/4157585707 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f730806cc30 0x7f730806f0e0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f7314009b80 tx=0x7f731400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:38:44.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:44.550+0000 7f731b400700 1 -- 172.21.15.37:0/4157585707 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f73100c09f0 con 0x7f73240fe830 2024-03-29T08:38:44.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:44.780+0000 7f732a200700 1 -- 172.21.15.37:0/4157585707 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7324111350 con 0x7f730806cc30 2024-03-29T08:38:44.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:44.782+0000 7f731b400700 1 -- 172.21.15.37:0/4157585707 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f7324111350 con 0x7f730806cc30 2024-03-29T08:38:44.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:44.784+0000 7f732a200700 1 -- 172.21.15.37:0/4157585707 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f730806cc30 msgr2=0x7f730806f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:38:44.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:44.784+0000 7f732a200700 1 --2- 172.21.15.37:0/4157585707 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f730806cc30 0x7f730806f0e0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f7314009b80 tx=0x7f731400b040 comp rx=0 tx=0).stop 2024-03-29T08:38:44.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:44.784+0000 7f732a200700 1 -- 172.21.15.37:0/4157585707 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f73240fe830 msgr2=0x7f73241161d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:38:44.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:44.784+0000 7f732a200700 1 --2- 172.21.15.37:0/4157585707 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f73240fe830 0x7f73241161d0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f7310000c00 tx=0x7f7310009e30 comp rx=0 tx=0).stop 2024-03-29T08:38:44.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:44.784+0000 7f732a200700 1 -- 172.21.15.37:0/4157585707 shutdown_connections 2024-03-29T08:38:44.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:44.784+0000 7f732a200700 1 --2- 172.21.15.37:0/4157585707 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f730806cc30 0x7f730806f0e0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:38:44.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:44.784+0000 7f732a200700 1 --2- 172.21.15.37:0/4157585707 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f73240fde90 0x7f7324115c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:38:44.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:44.784+0000 7f732a200700 1 --2- 172.21.15.37:0/4157585707 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f73240fe830 0x7f73241161d0 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:38:44.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:44.784+0000 7f732a200700 1 -- 172.21.15.37:0/4157585707 >> 172.21.15.37:0/4157585707 conn(0x7f7324072bc0 msgr2=0x7f73240fb6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:38:44.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:44.784+0000 7f732a200700 1 -- 172.21.15.37:0/4157585707 shutdown_connections 2024-03-29T08:38:44.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:44.784+0000 7f732a200700 1 -- 172.21.15.37:0/4157585707 wait complete. 2024-03-29T08:38:44.798 INFO:teuthology.orchestra.run.smithi037.stdout:true 2024-03-29T08:38:44.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:44.922+0000 7f4430400700 1 -- 172.21.15.37:0/1655328185 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f44280fe830 msgr2=0x7f4428106f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:38:44.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:44.922+0000 7f4430400700 1 --2- 172.21.15.37:0/1655328185 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f44280fe830 0x7f4428106f00 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f4414004940 tx=0x7f4414009e30 comp rx=0 tx=0).stop 2024-03-29T08:38:44.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:44.922+0000 7f4430400700 1 -- 172.21.15.37:0/1655328185 shutdown_connections 2024-03-29T08:38:44.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:44.922+0000 7f4430400700 1 --2- 172.21.15.37:0/1655328185 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f44280fe830 0x7f4428106f00 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:38:44.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:44.922+0000 7f4430400700 1 --2- 172.21.15.37:0/1655328185 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f44280fde90 0x7f44280fe260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:38:44.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:44.922+0000 7f4430400700 1 -- 172.21.15.37:0/1655328185 >> 172.21.15.37:0/1655328185 conn(0x7f4428072bc0 msgr2=0x7f4428072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:38:44.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:44.922+0000 7f4430400700 1 -- 172.21.15.37:0/1655328185 shutdown_connections 2024-03-29T08:38:44.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:44.922+0000 7f4430400700 1 -- 172.21.15.37:0/1655328185 wait complete. 2024-03-29T08:38:44.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:44.923+0000 7f4430400700 1 Processor -- start 2024-03-29T08:38:44.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:44.923+0000 7f4430400700 1 -- start start 2024-03-29T08:38:44.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:44.924+0000 7f4430400700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f44280fde90 0x7f4428115d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:38:44.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:44.924+0000 7f4430400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f44280fe830 0x7f4428116280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:38:44.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:44.924+0000 7f4430400700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4428116960 con 0x7f44280fe830 2024-03-29T08:38:44.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:44.924+0000 7f4430400700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f442810fca0 con 0x7f44280fde90 2024-03-29T08:38:44.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:44.924+0000 7f442d000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f44280fe830 0x7f4428116280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:38:44.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:44.924+0000 7f442d000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f44280fe830 0x7f4428116280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35446/0 (socket says 172.21.15.37:35446) 2024-03-29T08:38:44.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:44.924+0000 7f442d000700 1 -- 172.21.15.37:0/2167651415 learned_addr learned my addr 172.21.15.37:0/2167651415 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:38:44.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:44.924+0000 7f442da00700 1 --2- 172.21.15.37:0/2167651415 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f44280fde90 0x7f4428115d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:38:44.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:44.924+0000 7f442d000700 1 -- 172.21.15.37:0/2167651415 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f44280fde90 msgr2=0x7f4428115d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:38:44.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:44.924+0000 7f442d000700 1 --2- 172.21.15.37:0/2167651415 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f44280fde90 0x7f4428115d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:38:44.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:44.924+0000 7f442d000700 1 -- 172.21.15.37:0/2167651415 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4414005040 con 0x7f44280fe830 2024-03-29T08:38:44.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:44.925+0000 7f442d000700 1 --2- 172.21.15.37:0/2167651415 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f44280fe830 0x7f4428116280 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f44140093b0 tx=0x7f4414009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:38:44.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:44.925+0000 7f4425600700 1 -- 172.21.15.37:0/2167651415 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f441400e040 con 0x7f44280fe830 2024-03-29T08:38:44.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:44.925+0000 7f4430400700 1 -- 172.21.15.37:0/2167651415 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f442810fec0 con 0x7f44280fe830 2024-03-29T08:38:44.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:44.925+0000 7f4425600700 1 -- 172.21.15.37:0/2167651415 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f4414004290 con 0x7f44280fe830 2024-03-29T08:38:44.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:44.925+0000 7f4425600700 1 -- 172.21.15.37:0/2167651415 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f441401c3f0 con 0x7f44280fe830 2024-03-29T08:38:44.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:44.925+0000 7f4430400700 1 -- 172.21.15.37:0/2167651415 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4428110400 con 0x7f44280fe830 2024-03-29T08:38:44.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:44.926+0000 7f4430400700 1 -- 172.21.15.37:0/2167651415 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4428106a20 con 0x7f44280fe830 2024-03-29T08:38:44.928 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:38:44 smithi037 ceph-mon[142325]: pgmap v836: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:38:44.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:44.927+0000 7f4425600700 1 -- 172.21.15.37:0/2167651415 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f4414013790 con 0x7f44280fe830 2024-03-29T08:38:44.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:44.928+0000 7f4425600700 1 --2- 172.21.15.37:0/2167651415 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f441006cc30 0x7f441006f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:38:44.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:44.928+0000 7f4425600700 1 -- 172.21.15.37:0/2167651415 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f441408d2a0 con 0x7f44280fe830 2024-03-29T08:38:44.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:44.928+0000 7f442da00700 1 --2- 172.21.15.37:0/2167651415 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f441006cc30 0x7f441006f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:38:44.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:44.929+0000 7f442da00700 1 --2- 172.21.15.37:0/2167651415 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f441006cc30 0x7f441006f0e0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f441c006fd0 tx=0x7f441c008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:38:44.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:44.933+0000 7f4425600700 1 -- 172.21.15.37:0/2167651415 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f441405ad70 con 0x7f44280fe830 2024-03-29T08:38:45.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:38:44 smithi148 ceph-mon[106692]: pgmap v836: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:38:45.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:45.164+0000 7f4430400700 1 -- 172.21.15.37:0/2167651415 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4428111430 con 0x7f441006cc30 2024-03-29T08:38:45.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:45.166+0000 7f4425600700 1 -- 172.21.15.37:0/2167651415 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f4428111430 con 0x7f441006cc30 2024-03-29T08:38:45.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:45.168+0000 7f4430400700 1 -- 172.21.15.37:0/2167651415 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f441006cc30 msgr2=0x7f441006f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:38:45.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:45.168+0000 7f4430400700 1 --2- 172.21.15.37:0/2167651415 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f441006cc30 0x7f441006f0e0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f441c006fd0 tx=0x7f441c008040 comp rx=0 tx=0).stop 2024-03-29T08:38:45.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:45.168+0000 7f4430400700 1 -- 172.21.15.37:0/2167651415 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f44280fe830 msgr2=0x7f4428116280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:38:45.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:45.168+0000 7f4430400700 1 --2- 172.21.15.37:0/2167651415 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f44280fe830 0x7f4428116280 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f44140093b0 tx=0x7f4414009e30 comp rx=0 tx=0).stop 2024-03-29T08:38:45.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:45.168+0000 7f4430400700 1 -- 172.21.15.37:0/2167651415 shutdown_connections 2024-03-29T08:38:45.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:45.168+0000 7f4430400700 1 --2- 172.21.15.37:0/2167651415 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f441006cc30 0x7f441006f0e0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:38:45.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:45.168+0000 7f4430400700 1 --2- 172.21.15.37:0/2167651415 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f44280fde90 0x7f4428115d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:38:45.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:45.168+0000 7f4430400700 1 --2- 172.21.15.37:0/2167651415 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f44280fe830 0x7f4428116280 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:38:45.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:45.168+0000 7f4430400700 1 -- 172.21.15.37:0/2167651415 >> 172.21.15.37:0/2167651415 conn(0x7f4428072bc0 msgr2=0x7f44280fb740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:38:45.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:45.168+0000 7f4430400700 1 -- 172.21.15.37:0/2167651415 shutdown_connections 2024-03-29T08:38:45.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:45.168+0000 7f4430400700 1 -- 172.21.15.37:0/2167651415 wait complete. 2024-03-29T08:38:45.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:45.307+0000 7f8e9b600700 1 -- 172.21.15.37:0/2621965079 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8e940fe830 msgr2=0x7f8e94106f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:38:45.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:45.307+0000 7f8e9b600700 1 --2- 172.21.15.37:0/2621965079 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8e940fe830 0x7f8e94106f00 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f8e80004940 tx=0x7f8e80009e30 comp rx=0 tx=0).stop 2024-03-29T08:38:45.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:45.307+0000 7f8e9b600700 1 -- 172.21.15.37:0/2621965079 shutdown_connections 2024-03-29T08:38:45.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:45.307+0000 7f8e9b600700 1 --2- 172.21.15.37:0/2621965079 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8e940fe830 0x7f8e94106f00 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:38:45.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:45.307+0000 7f8e9b600700 1 --2- 172.21.15.37:0/2621965079 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8e940fde90 0x7f8e940fe260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:38:45.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:45.307+0000 7f8e9b600700 1 -- 172.21.15.37:0/2621965079 >> 172.21.15.37:0/2621965079 conn(0x7f8e94072bc0 msgr2=0x7f8e94072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:38:45.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:45.307+0000 7f8e9b600700 1 -- 172.21.15.37:0/2621965079 shutdown_connections 2024-03-29T08:38:45.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:45.307+0000 7f8e9b600700 1 -- 172.21.15.37:0/2621965079 wait complete. 2024-03-29T08:38:45.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:45.308+0000 7f8e9b600700 1 Processor -- start 2024-03-29T08:38:45.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:45.308+0000 7f8e9b600700 1 -- start start 2024-03-29T08:38:45.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:45.308+0000 7f8e9b600700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8e940fde90 0x7f8e94113a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:38:45.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:45.308+0000 7f8e9b600700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8e940fe830 0x7f8e94113fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:38:45.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:45.308+0000 7f8e9b600700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e94114610 con 0x7f8e940fe830 2024-03-29T08:38:45.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:45.308+0000 7f8e9b600700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e9410da00 con 0x7f8e940fde90 2024-03-29T08:38:45.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:45.308+0000 7f8e93e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8e940fe830 0x7f8e94113fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:38:45.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:45.309+0000 7f8e93e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8e940fe830 0x7f8e94113fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35472/0 (socket says 172.21.15.37:35472) 2024-03-29T08:38:45.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:45.309+0000 7f8e93e00700 1 -- 172.21.15.37:0/1698037025 learned_addr learned my addr 172.21.15.37:0/1698037025 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:38:45.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:45.309+0000 7f8e98c00700 1 --2- 172.21.15.37:0/1698037025 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8e940fde90 0x7f8e94113a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:38:45.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:45.309+0000 7f8e93e00700 1 -- 172.21.15.37:0/1698037025 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8e940fde90 msgr2=0x7f8e94113a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:38:45.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:45.309+0000 7f8e93e00700 1 --2- 172.21.15.37:0/1698037025 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8e940fde90 0x7f8e94113a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:38:45.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:45.309+0000 7f8e93e00700 1 -- 172.21.15.37:0/1698037025 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e80005040 con 0x7f8e940fe830 2024-03-29T08:38:45.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:45.309+0000 7f8e93e00700 1 --2- 172.21.15.37:0/1698037025 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8e940fe830 0x7f8e94113fc0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f8e80000c00 tx=0x7f8e80009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:38:45.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:45.309+0000 7f8e90c00700 1 -- 172.21.15.37:0/1698037025 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8e8000e040 con 0x7f8e940fe830 2024-03-29T08:38:45.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:45.310+0000 7f8e90c00700 1 -- 172.21.15.37:0/1698037025 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f8e8000bed0 con 0x7f8e940fe830 2024-03-29T08:38:45.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:45.310+0000 7f8e9b600700 1 -- 172.21.15.37:0/1698037025 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e9410dc80 con 0x7f8e940fe830 2024-03-29T08:38:45.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:45.310+0000 7f8e90c00700 1 -- 172.21.15.37:0/1698037025 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8e8001c3f0 con 0x7f8e940fe830 2024-03-29T08:38:45.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:45.310+0000 7f8e9b600700 1 -- 172.21.15.37:0/1698037025 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e9410e1c0 con 0x7f8e940fe830 2024-03-29T08:38:45.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:45.311+0000 7f8e9b600700 1 -- 172.21.15.37:0/1698037025 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e94101980 con 0x7f8e940fe830 2024-03-29T08:38:45.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:45.312+0000 7f8e90c00700 1 -- 172.21.15.37:0/1698037025 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f8e80013420 con 0x7f8e940fe830 2024-03-29T08:38:45.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:45.313+0000 7f8e90c00700 1 --2- 172.21.15.37:0/1698037025 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f8e7c06cc30 0x7f8e7c06f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:38:45.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:45.313+0000 7f8e90c00700 1 -- 172.21.15.37:0/1698037025 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f8e8008d400 con 0x7f8e940fe830 2024-03-29T08:38:45.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:45.313+0000 7f8e98c00700 1 --2- 172.21.15.37:0/1698037025 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f8e7c06cc30 0x7f8e7c06f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:38:45.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:45.314+0000 7f8e98c00700 1 --2- 172.21.15.37:0/1698037025 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f8e7c06cc30 0x7f8e7c06f0e0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f8e88009fd0 tx=0x7f8e8800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:38:45.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:45.317+0000 7f8e90c00700 1 -- 172.21.15.37:0/1698037025 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f8e8005aed0 con 0x7f8e940fe830 2024-03-29T08:38:45.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:45.540+0000 7f8e9b600700 1 -- 172.21.15.37:0/1698037025 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f8e9410f0d0 con 0x7f8e7c06cc30 2024-03-29T08:38:45.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:45.550+0000 7f8e90c00700 1 -- 172.21.15.37:0/1698037025 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f8e9410f0d0 con 0x7f8e7c06cc30 2024-03-29T08:38:45.549 INFO:teuthology.orchestra.run.smithi037.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-29T08:38:45.550 INFO:teuthology.orchestra.run.smithi037.stdout:alertmanager.smithi037 smithi037 *:9093,9094 running (28m) 5m ago 41m 14.3M - 0.25.0 c8568f914cd2 2f9f9746ed0f 2024-03-29T08:38:45.550 INFO:teuthology.orchestra.run.smithi037.stdout:crash.smithi037 smithi037 running (25m) 5m ago 41m 6647k - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 178d416a3186 2024-03-29T08:38:45.550 INFO:teuthology.orchestra.run.smithi037.stdout:crash.smithi148 smithi148 running (25m) 5m ago 40m 6656k - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 23666989d58f 2024-03-29T08:38:45.550 INFO:teuthology.orchestra.run.smithi037.stdout:grafana.smithi037 smithi037 *:3000 running (28m) 5m ago 41m 74.5M - 9.4.7 954c08fa6188 6b976fe49952 2024-03-29T08:38:45.550 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi037.iifsmh smithi037 running (20m) 5m ago 35m 16.4M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 2d97f562cc5b 2024-03-29T08:38:45.550 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi037.mmwgxe smithi037 running (21m) 5m ago 35m 16.7M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff ec37037f46e8 2024-03-29T08:38:45.550 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi148.qukpjh smithi148 running (5m) 5m ago 35m 10.7M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 3257219bc046 2024-03-29T08:38:45.550 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi148.sqteih smithi148 running (20m) 5m ago 35m 16.3M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 44c15c77037a 2024-03-29T08:38:45.550 INFO:teuthology.orchestra.run.smithi037.stdout:mgr.smithi037.wjikii smithi037 *:8443,9283,8765 running (30m) 5m ago 43m 510M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff c5a1417b9417 2024-03-29T08:38:45.550 INFO:teuthology.orchestra.run.smithi037.stdout:mgr.smithi148.onhgvm smithi148 *:8443,9283,8765 running (29m) 5m ago 40m 456M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff fcfdbce062bd 2024-03-29T08:38:45.550 INFO:teuthology.orchestra.run.smithi037.stdout:mon.smithi037 smithi037 running (26m) 5m ago 43m 70.2M 2048M 18.2.2-1015-gd0e392d9 1cc2f45c17ff eea2060783af 2024-03-29T08:38:45.550 INFO:teuthology.orchestra.run.smithi037.stdout:mon.smithi148 smithi148 running (26m) 5m ago 40m 50.4M 2048M 18.2.2-1015-gd0e392d9 1cc2f45c17ff ad4b2c5748be 2024-03-29T08:38:45.550 INFO:teuthology.orchestra.run.smithi037.stdout:node-exporter.smithi037 smithi037 *:9100 running (29m) 5m ago 41m 8514k - 1.5.0 0da6a335fe13 26b8ae9cb878 2024-03-29T08:38:45.550 INFO:teuthology.orchestra.run.smithi037.stdout:node-exporter.smithi148 smithi148 *:9100 running (29m) 5m ago 40m 8464k - 1.5.0 0da6a335fe13 98414e4e64ed 2024-03-29T08:38:45.551 INFO:teuthology.orchestra.run.smithi037.stdout:osd.0 smithi037 running (24m) 5m ago 39m 108M 1619M 18.2.2-1015-gd0e392d9 1cc2f45c17ff c0c584f6245c 2024-03-29T08:38:45.551 INFO:teuthology.orchestra.run.smithi037.stdout:osd.1 smithi037 running (24m) 5m ago 38m 104M 1619M 18.2.2-1015-gd0e392d9 1cc2f45c17ff a4a2ff4cc83d 2024-03-29T08:38:45.551 INFO:teuthology.orchestra.run.smithi037.stdout:osd.2 smithi037 running (23m) 5m ago 38m 85.9M 1619M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 8b1bc71aa79e 2024-03-29T08:38:45.551 INFO:teuthology.orchestra.run.smithi037.stdout:osd.3 smithi148 running (22m) 5m ago 37m 109M 2642M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 1c656ab8ff86 2024-03-29T08:38:45.551 INFO:teuthology.orchestra.run.smithi037.stdout:osd.4 smithi148 running (22m) 5m ago 37m 101M 2642M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 966a9de11d22 2024-03-29T08:38:45.551 INFO:teuthology.orchestra.run.smithi037.stdout:osd.5 smithi148 running (21m) 5m ago 36m 89.4M 2642M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 474c4f9212ce 2024-03-29T08:38:45.551 INFO:teuthology.orchestra.run.smithi037.stdout:prometheus.smithi037 smithi037 *:9095 running (28m) 5m ago 40m 61.0M - 2.43.0 a07b618ecd1d 30c0223ec589 2024-03-29T08:38:45.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:45.552+0000 7f8e9b600700 1 -- 172.21.15.37:0/1698037025 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f8e7c06cc30 msgr2=0x7f8e7c06f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:38:45.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:45.552+0000 7f8e9b600700 1 --2- 172.21.15.37:0/1698037025 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f8e7c06cc30 0x7f8e7c06f0e0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f8e88009fd0 tx=0x7f8e8800b040 comp rx=0 tx=0).stop 2024-03-29T08:38:45.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:45.552+0000 7f8e9b600700 1 -- 172.21.15.37:0/1698037025 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8e940fe830 msgr2=0x7f8e94113fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:38:45.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:45.552+0000 7f8e9b600700 1 --2- 172.21.15.37:0/1698037025 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8e940fe830 0x7f8e94113fc0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f8e80000c00 tx=0x7f8e80009e30 comp rx=0 tx=0).stop 2024-03-29T08:38:45.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:45.552+0000 7f8e9b600700 1 -- 172.21.15.37:0/1698037025 shutdown_connections 2024-03-29T08:38:45.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:45.552+0000 7f8e9b600700 1 --2- 172.21.15.37:0/1698037025 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f8e7c06cc30 0x7f8e7c06f0e0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:38:45.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:45.552+0000 7f8e9b600700 1 --2- 172.21.15.37:0/1698037025 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8e940fde90 0x7f8e94113a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:38:45.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:45.552+0000 7f8e9b600700 1 --2- 172.21.15.37:0/1698037025 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8e940fe830 0x7f8e94113fc0 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:38:45.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:45.552+0000 7f8e9b600700 1 -- 172.21.15.37:0/1698037025 >> 172.21.15.37:0/1698037025 conn(0x7f8e94072bc0 msgr2=0x7f8e940ff3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:38:45.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:45.552+0000 7f8e9b600700 1 -- 172.21.15.37:0/1698037025 shutdown_connections 2024-03-29T08:38:45.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:45.552+0000 7f8e9b600700 1 -- 172.21.15.37:0/1698037025 wait complete. 2024-03-29T08:38:45.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:45.690+0000 7fad1ee00700 1 -- 172.21.15.37:0/4268003752 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fad181035e0 msgr2=0x7fad180fcc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:38:45.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:45.690+0000 7fad1ee00700 1 --2- 172.21.15.37:0/4268003752 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fad181035e0 0x7fad180fcc50 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fad04004940 tx=0x7fad04009e30 comp rx=0 tx=0).stop 2024-03-29T08:38:45.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:45.690+0000 7fad1ee00700 1 -- 172.21.15.37:0/4268003752 shutdown_connections 2024-03-29T08:38:45.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:45.690+0000 7fad1ee00700 1 --2- 172.21.15.37:0/4268003752 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fad181035e0 0x7fad180fcc50 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:38:45.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:45.690+0000 7fad1ee00700 1 --2- 172.21.15.37:0/4268003752 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fad18102c40 0x7fad18103010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:38:45.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:45.690+0000 7fad1ee00700 1 -- 172.21.15.37:0/4268003752 >> 172.21.15.37:0/4268003752 conn(0x7fad180f8830 msgr2=0x7fad180fac40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:38:45.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:45.690+0000 7fad1ee00700 1 -- 172.21.15.37:0/4268003752 shutdown_connections 2024-03-29T08:38:45.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:45.690+0000 7fad1ee00700 1 -- 172.21.15.37:0/4268003752 wait complete. 2024-03-29T08:38:45.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:45.691+0000 7fad1ee00700 1 Processor -- start 2024-03-29T08:38:45.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:45.691+0000 7fad1ee00700 1 -- start start 2024-03-29T08:38:45.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:45.691+0000 7fad1ee00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fad18102c40 0x7fad181138e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:38:45.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:45.691+0000 7fad1ee00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fad181035e0 0x7fad18113e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:38:45.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:45.691+0000 7fad1ee00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad18114470 con 0x7fad181035e0 2024-03-29T08:38:45.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:45.691+0000 7fad1ee00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad1810d860 con 0x7fad18102c40 2024-03-29T08:38:45.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:45.692+0000 7fad1d000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fad181035e0 0x7fad18113e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:38:45.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:45.692+0000 7fad1d000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fad181035e0 0x7fad18113e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35496/0 (socket says 172.21.15.37:35496) 2024-03-29T08:38:45.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:45.692+0000 7fad1d000700 1 -- 172.21.15.37:0/198227406 learned_addr learned my addr 172.21.15.37:0/198227406 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:38:45.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:45.692+0000 7fad1da00700 1 --2- 172.21.15.37:0/198227406 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fad18102c40 0x7fad181138e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:38:45.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:45.692+0000 7fad1d000700 1 -- 172.21.15.37:0/198227406 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fad18102c40 msgr2=0x7fad181138e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:38:45.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:45.692+0000 7fad1d000700 1 --2- 172.21.15.37:0/198227406 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fad18102c40 0x7fad181138e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:38:45.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:45.692+0000 7fad1d000700 1 -- 172.21.15.37:0/198227406 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad04005040 con 0x7fad181035e0 2024-03-29T08:38:45.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:45.693+0000 7fad1d000700 1 --2- 172.21.15.37:0/198227406 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fad181035e0 0x7fad18113e20 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fad04009390 tx=0x7fad04009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:38:45.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:45.693+0000 7fad15600700 1 -- 172.21.15.37:0/198227406 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fad0400e040 con 0x7fad181035e0 2024-03-29T08:38:45.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:45.693+0000 7fad15600700 1 -- 172.21.15.37:0/198227406 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fad04004290 con 0x7fad181035e0 2024-03-29T08:38:45.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:45.693+0000 7fad1ee00700 1 -- 172.21.15.37:0/198227406 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad1810dae0 con 0x7fad181035e0 2024-03-29T08:38:45.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:45.693+0000 7fad15600700 1 -- 172.21.15.37:0/198227406 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fad04013e70 con 0x7fad181035e0 2024-03-29T08:38:45.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:45.693+0000 7fad1ee00700 1 -- 172.21.15.37:0/198227406 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad1810e0a0 con 0x7fad181035e0 2024-03-29T08:38:45.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:45.694+0000 7fad1ee00700 1 -- 172.21.15.37:0/198227406 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad180fe390 con 0x7fad181035e0 2024-03-29T08:38:45.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:45.695+0000 7fad15600700 1 -- 172.21.15.37:0/198227406 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fad04013690 con 0x7fad181035e0 2024-03-29T08:38:45.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:45.696+0000 7fad15600700 1 --2- 172.21.15.37:0/198227406 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fad0006cbc0 0x7fad0006f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:38:45.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:45.696+0000 7fad15600700 1 -- 172.21.15.37:0/198227406 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fad0408d2c0 con 0x7fad181035e0 2024-03-29T08:38:45.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:45.697+0000 7fad1da00700 1 --2- 172.21.15.37:0/198227406 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fad0006cbc0 0x7fad0006f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:38:45.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:45.697+0000 7fad1da00700 1 --2- 172.21.15.37:0/198227406 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fad0006cbc0 0x7fad0006f070 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fad0c004f40 tx=0x7fad0c00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:38:45.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:45.701+0000 7fad15600700 1 -- 172.21.15.37:0/198227406 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fad0405adc0 con 0x7fad181035e0 2024-03-29T08:38:45.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:38:45 smithi037 ceph-mon[142325]: from='client.35514 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:38:46.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:46.002+0000 7fad1ee00700 1 -- 172.21.15.37:0/198227406 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fad18109bd0 con 0x7fad181035e0 2024-03-29T08:38:46.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:46.002+0000 7fad15600700 1 -- 172.21.15.37:0/198227406 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7fad04013420 con 0x7fad181035e0 2024-03-29T08:38:46.002 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-29T08:38:46.002 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": { 2024-03-29T08:38:46.002 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 2 2024-03-29T08:38:46.003 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:38:46.003 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": { 2024-03-29T08:38:46.003 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 2 2024-03-29T08:38:46.003 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:38:46.003 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": { 2024-03-29T08:38:46.003 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 6 2024-03-29T08:38:46.003 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:38:46.003 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": { 2024-03-29T08:38:46.003 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 4 2024-03-29T08:38:46.003 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:38:46.003 INFO:teuthology.orchestra.run.smithi037.stdout: "overall": { 2024-03-29T08:38:46.003 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 14 2024-03-29T08:38:46.003 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-29T08:38:46.003 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-29T08:38:46.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:46.005+0000 7fad1ee00700 1 -- 172.21.15.37:0/198227406 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fad0006cbc0 msgr2=0x7fad0006f070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:38:46.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:46.005+0000 7fad1ee00700 1 --2- 172.21.15.37:0/198227406 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fad0006cbc0 0x7fad0006f070 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fad0c004f40 tx=0x7fad0c00b040 comp rx=0 tx=0).stop 2024-03-29T08:38:46.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:46.005+0000 7fad1ee00700 1 -- 172.21.15.37:0/198227406 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fad181035e0 msgr2=0x7fad18113e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:38:46.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:46.005+0000 7fad1ee00700 1 --2- 172.21.15.37:0/198227406 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fad181035e0 0x7fad18113e20 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fad04009390 tx=0x7fad04009e30 comp rx=0 tx=0).stop 2024-03-29T08:38:46.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:46.005+0000 7fad1ee00700 1 -- 172.21.15.37:0/198227406 shutdown_connections 2024-03-29T08:38:46.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:46.005+0000 7fad1ee00700 1 --2- 172.21.15.37:0/198227406 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fad0006cbc0 0x7fad0006f070 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:38:46.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:46.005+0000 7fad1ee00700 1 --2- 172.21.15.37:0/198227406 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fad18102c40 0x7fad181138e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:38:46.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:46.005+0000 7fad1ee00700 1 --2- 172.21.15.37:0/198227406 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fad181035e0 0x7fad18113e20 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:38:46.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:46.005+0000 7fad1ee00700 1 -- 172.21.15.37:0/198227406 >> 172.21.15.37:0/198227406 conn(0x7fad180f8830 msgr2=0x7fad180f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:38:46.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:46.005+0000 7fad1ee00700 1 -- 172.21.15.37:0/198227406 shutdown_connections 2024-03-29T08:38:46.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:46.005+0000 7fad1ee00700 1 -- 172.21.15.37:0/198227406 wait complete. 2024-03-29T08:38:46.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:38:45 smithi148 ceph-mon[106692]: from='client.35514 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:38:46.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:46.142+0000 7feaedc00700 1 -- 172.21.15.37:0/3505055657 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feae8102c40 msgr2=0x7feae8103010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:38:46.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:46.142+0000 7feaedc00700 1 --2- 172.21.15.37:0/3505055657 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feae8102c40 0x7feae8103010 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fead8004940 tx=0x7fead8009e30 comp rx=0 tx=0).stop 2024-03-29T08:38:46.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:46.142+0000 7feaedc00700 1 -- 172.21.15.37:0/3505055657 shutdown_connections 2024-03-29T08:38:46.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:46.142+0000 7feaedc00700 1 --2- 172.21.15.37:0/3505055657 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feae81035e0 0x7feae80fcc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:38:46.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:46.142+0000 7feaedc00700 1 --2- 172.21.15.37:0/3505055657 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feae8102c40 0x7feae8103010 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:38:46.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:46.142+0000 7feaedc00700 1 -- 172.21.15.37:0/3505055657 >> 172.21.15.37:0/3505055657 conn(0x7feae80f8830 msgr2=0x7feae80fac40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:38:46.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:46.142+0000 7feaedc00700 1 -- 172.21.15.37:0/3505055657 shutdown_connections 2024-03-29T08:38:46.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:46.142+0000 7feaedc00700 1 -- 172.21.15.37:0/3505055657 wait complete. 2024-03-29T08:38:46.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:46.143+0000 7feaedc00700 1 Processor -- start 2024-03-29T08:38:46.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:46.143+0000 7feaedc00700 1 -- start start 2024-03-29T08:38:46.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:46.144+0000 7feaedc00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feae8102c40 0x7feae8115b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:38:46.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:46.144+0000 7feaedc00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feae81035e0 0x7feae8116070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:38:46.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:46.144+0000 7feaedc00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feae81166c0 con 0x7feae8102c40 2024-03-29T08:38:46.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:46.144+0000 7feaedc00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feae810fab0 con 0x7feae81035e0 2024-03-29T08:38:46.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:46.144+0000 7feae7e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feae8102c40 0x7feae8115b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:38:46.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:46.144+0000 7feae7e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feae8102c40 0x7feae8115b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35514/0 (socket says 172.21.15.37:35514) 2024-03-29T08:38:46.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:46.144+0000 7feae7e00700 1 -- 172.21.15.37:0/3714656259 learned_addr learned my addr 172.21.15.37:0/3714656259 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:38:46.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:46.144+0000 7feae7400700 1 --2- 172.21.15.37:0/3714656259 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feae81035e0 0x7feae8116070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:38:46.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:46.144+0000 7feae7e00700 1 -- 172.21.15.37:0/3714656259 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feae81035e0 msgr2=0x7feae8116070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:38:46.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:46.144+0000 7feae7e00700 1 --2- 172.21.15.37:0/3714656259 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feae81035e0 0x7feae8116070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:38:46.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:46.144+0000 7feae7e00700 1 -- 172.21.15.37:0/3714656259 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fead8005040 con 0x7feae8102c40 2024-03-29T08:38:46.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:46.144+0000 7feae7400700 1 --2- 172.21.15.37:0/3714656259 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feae81035e0 0x7feae8116070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:38:46.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:46.144+0000 7feae7e00700 1 --2- 172.21.15.37:0/3714656259 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feae8102c40 0x7feae8115b30 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fead8009360 tx=0x7fead8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:38:46.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:46.145+0000 7feadfe00700 1 -- 172.21.15.37:0/3714656259 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fead800e040 con 0x7feae8102c40 2024-03-29T08:38:46.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:46.145+0000 7feadfe00700 1 -- 172.21.15.37:0/3714656259 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fead8013420 con 0x7feae8102c40 2024-03-29T08:38:46.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:46.145+0000 7feaedc00700 1 -- 172.21.15.37:0/3714656259 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feae810fd30 con 0x7feae8102c40 2024-03-29T08:38:46.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:46.145+0000 7feadfe00700 1 -- 172.21.15.37:0/3714656259 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fead801c5f0 con 0x7feae8102c40 2024-03-29T08:38:46.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:46.145+0000 7feaedc00700 1 -- 172.21.15.37:0/3714656259 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feae81102f0 con 0x7feae8102c40 2024-03-29T08:38:46.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:46.146+0000 7feaedc00700 1 -- 172.21.15.37:0/3714656259 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feae80fe390 con 0x7feae8102c40 2024-03-29T08:38:46.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:46.147+0000 7feadfe00700 1 -- 172.21.15.37:0/3714656259 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fead8013590 con 0x7feae8102c40 2024-03-29T08:38:46.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:46.148+0000 7feadfe00700 1 --2- 172.21.15.37:0/3714656259 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7feacc06cc30 0x7feacc06f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:38:46.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:46.148+0000 7feadfe00700 1 -- 172.21.15.37:0/3714656259 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fead808d250 con 0x7feae8102c40 2024-03-29T08:38:46.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:46.148+0000 7feae7400700 1 --2- 172.21.15.37:0/3714656259 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7feacc06cc30 0x7feacc06f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:38:46.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:46.149+0000 7feae7400700 1 --2- 172.21.15.37:0/3714656259 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7feacc06cc30 0x7feacc06f0e0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fead4009fd0 tx=0x7fead4009ee0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:38:46.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:46.152+0000 7feadfe00700 1 -- 172.21.15.37:0/3714656259 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fead805add0 con 0x7feae8102c40 2024-03-29T08:38:46.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:46.412+0000 7feaedc00700 1 -- 172.21.15.37:0/3714656259 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7feae810fec0 con 0x7feae8102c40 2024-03-29T08:38:46.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:46.412+0000 7feadfe00700 1 -- 172.21.15.37:0/3714656259 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 30 v30) v1 ==== 76+0+1841 (secure 0 0 0) 0x7feae810fec0 con 0x7feae8102c40 2024-03-29T08:38:46.412 INFO:teuthology.orchestra.run.smithi037.stdout:e30 2024-03-29T08:38:46.412 INFO:teuthology.orchestra.run.smithi037.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-29T08:38:46.412 INFO:teuthology.orchestra.run.smithi037.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T08:38:46.413 INFO:teuthology.orchestra.run.smithi037.stdout:legacy client fscid: 1 2024-03-29T08:38:46.413 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:38:46.413 INFO:teuthology.orchestra.run.smithi037.stdout:Filesystem 'cephfs' (1) 2024-03-29T08:38:46.413 INFO:teuthology.orchestra.run.smithi037.stdout:fs_name cephfs 2024-03-29T08:38:46.413 INFO:teuthology.orchestra.run.smithi037.stdout:epoch 21 2024-03-29T08:38:46.413 INFO:teuthology.orchestra.run.smithi037.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-03-29T08:38:46.413 INFO:teuthology.orchestra.run.smithi037.stdout:created 2024-03-29T08:03:29.241269+0000 2024-03-29T08:38:46.413 INFO:teuthology.orchestra.run.smithi037.stdout:modified 2024-03-29T08:17:16.237238+0000 2024-03-29T08:38:46.413 INFO:teuthology.orchestra.run.smithi037.stdout:tableserver 0 2024-03-29T08:38:46.413 INFO:teuthology.orchestra.run.smithi037.stdout:root 0 2024-03-29T08:38:46.413 INFO:teuthology.orchestra.run.smithi037.stdout:session_timeout 60 2024-03-29T08:38:46.413 INFO:teuthology.orchestra.run.smithi037.stdout:session_autoclose 300 2024-03-29T08:38:46.413 INFO:teuthology.orchestra.run.smithi037.stdout:max_file_size 1099511627776 2024-03-29T08:38:46.413 INFO:teuthology.orchestra.run.smithi037.stdout:max_xattr_size 65536 2024-03-29T08:38:46.413 INFO:teuthology.orchestra.run.smithi037.stdout:required_client_features {} 2024-03-29T08:38:46.413 INFO:teuthology.orchestra.run.smithi037.stdout:last_failure 0 2024-03-29T08:38:46.414 INFO:teuthology.orchestra.run.smithi037.stdout:last_failure_osd_epoch 82 2024-03-29T08:38:46.414 INFO:teuthology.orchestra.run.smithi037.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T08:38:46.414 INFO:teuthology.orchestra.run.smithi037.stdout:max_mds 2 2024-03-29T08:38:46.414 INFO:teuthology.orchestra.run.smithi037.stdout:in 0,1 2024-03-29T08:38:46.414 INFO:teuthology.orchestra.run.smithi037.stdout:up {} 2024-03-29T08:38:46.414 INFO:teuthology.orchestra.run.smithi037.stdout:failed 0,1 2024-03-29T08:38:46.414 INFO:teuthology.orchestra.run.smithi037.stdout:damaged 2024-03-29T08:38:46.414 INFO:teuthology.orchestra.run.smithi037.stdout:stopped 2024-03-29T08:38:46.414 INFO:teuthology.orchestra.run.smithi037.stdout:data_pools [3] 2024-03-29T08:38:46.414 INFO:teuthology.orchestra.run.smithi037.stdout:metadata_pool 2 2024-03-29T08:38:46.414 INFO:teuthology.orchestra.run.smithi037.stdout:inline_data disabled 2024-03-29T08:38:46.414 INFO:teuthology.orchestra.run.smithi037.stdout:balancer 2024-03-29T08:38:46.414 INFO:teuthology.orchestra.run.smithi037.stdout:bal_rank_mask -1 2024-03-29T08:38:46.415 INFO:teuthology.orchestra.run.smithi037.stdout:standby_count_wanted 1 2024-03-29T08:38:46.415 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:38:46.415 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:38:46.415 INFO:teuthology.orchestra.run.smithi037.stdout:Standby daemons: 2024-03-29T08:38:46.415 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:38:46.415 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi037.mmwgxe{-1:34440} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.37:6828/1248142005,v1:172.21.15.37:6829/1248142005] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:38:46.415 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi037.iifsmh{-1:34444} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.37:6826/2060882959,v1:172.21.15.37:6827/2060882959] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:38:46.415 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi148.sqteih{-1:44395} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.148:6824/4078652242,v1:172.21.15.148:6825/4078652242] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:38:46.415 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi148.qukpjh{-1:45007} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.148:6826/2394751105,v1:172.21.15.148:6827/2394751105] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:38:46.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:46.414+0000 7feaedc00700 1 -- 172.21.15.37:0/3714656259 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7feacc06cc30 msgr2=0x7feacc06f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:38:46.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:46.414+0000 7feaedc00700 1 --2- 172.21.15.37:0/3714656259 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7feacc06cc30 0x7feacc06f0e0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fead4009fd0 tx=0x7fead4009ee0 comp rx=0 tx=0).stop 2024-03-29T08:38:46.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:46.414+0000 7feaedc00700 1 -- 172.21.15.37:0/3714656259 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feae8102c40 msgr2=0x7feae8115b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:38:46.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:46.414+0000 7feaedc00700 1 --2- 172.21.15.37:0/3714656259 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feae8102c40 0x7feae8115b30 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fead8009360 tx=0x7fead8009e30 comp rx=0 tx=0).stop 2024-03-29T08:38:46.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:46.415+0000 7feaedc00700 1 -- 172.21.15.37:0/3714656259 shutdown_connections 2024-03-29T08:38:46.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:46.415+0000 7feaedc00700 1 --2- 172.21.15.37:0/3714656259 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7feacc06cc30 0x7feacc06f0e0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:38:46.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:46.415+0000 7feaedc00700 1 --2- 172.21.15.37:0/3714656259 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feae8102c40 0x7feae8115b30 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:38:46.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:46.415+0000 7feaedc00700 1 --2- 172.21.15.37:0/3714656259 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feae81035e0 0x7feae8116070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:38:46.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:46.415+0000 7feaedc00700 1 -- 172.21.15.37:0/3714656259 >> 172.21.15.37:0/3714656259 conn(0x7feae80f8830 msgr2=0x7feae80706b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:38:46.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:46.415+0000 7feaedc00700 1 -- 172.21.15.37:0/3714656259 shutdown_connections 2024-03-29T08:38:46.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:46.415+0000 7feaedc00700 1 -- 172.21.15.37:0/3714656259 wait complete. 2024-03-29T08:38:46.417 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 30 2024-03-29T08:38:46.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:46.552+0000 7fb58a400700 1 -- 172.21.15.37:0/1087376159 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb5841037a0 msgr2=0x7fb5840fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:38:46.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:46.552+0000 7fb58a400700 1 --2- 172.21.15.37:0/1087376159 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb5841037a0 0x7fb5840fce10 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fb570004940 tx=0x7fb570009e30 comp rx=0 tx=0).stop 2024-03-29T08:38:46.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:46.552+0000 7fb58a400700 1 -- 172.21.15.37:0/1087376159 shutdown_connections 2024-03-29T08:38:46.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:46.552+0000 7fb58a400700 1 --2- 172.21.15.37:0/1087376159 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb5841037a0 0x7fb5840fce10 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:38:46.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:46.552+0000 7fb58a400700 1 --2- 172.21.15.37:0/1087376159 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb584102e00 0x7fb5841031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:38:46.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:46.552+0000 7fb58a400700 1 -- 172.21.15.37:0/1087376159 >> 172.21.15.37:0/1087376159 conn(0x7fb5840f89b0 msgr2=0x7fb5840fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:38:46.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:46.552+0000 7fb58a400700 1 -- 172.21.15.37:0/1087376159 shutdown_connections 2024-03-29T08:38:46.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:46.552+0000 7fb58a400700 1 -- 172.21.15.37:0/1087376159 wait complete. 2024-03-29T08:38:46.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:46.553+0000 7fb58a400700 1 Processor -- start 2024-03-29T08:38:46.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:46.553+0000 7fb58a400700 1 -- start start 2024-03-29T08:38:46.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:46.554+0000 7fb58a400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb584102e00 0x7fb584115d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:38:46.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:46.554+0000 7fb58a400700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb5841037a0 0x7fb584116240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:38:46.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:46.554+0000 7fb58a400700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb584116890 con 0x7fb584102e00 2024-03-29T08:38:46.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:46.554+0000 7fb58a400700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb58410fc60 con 0x7fb5841037a0 2024-03-29T08:38:46.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:46.554+0000 7fb583400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb584102e00 0x7fb584115d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:38:46.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:46.554+0000 7fb583400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb584102e00 0x7fb584115d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35526/0 (socket says 172.21.15.37:35526) 2024-03-29T08:38:46.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:46.554+0000 7fb583400700 1 -- 172.21.15.37:0/2459992974 learned_addr learned my addr 172.21.15.37:0/2459992974 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:38:46.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:46.554+0000 7fb582a00700 1 --2- 172.21.15.37:0/2459992974 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb5841037a0 0x7fb584116240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:38:46.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:46.555+0000 7fb583400700 1 -- 172.21.15.37:0/2459992974 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb5841037a0 msgr2=0x7fb584116240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:38:46.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:46.555+0000 7fb583400700 1 --2- 172.21.15.37:0/2459992974 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb5841037a0 0x7fb584116240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:38:46.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:46.555+0000 7fb583400700 1 -- 172.21.15.37:0/2459992974 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb570005040 con 0x7fb584102e00 2024-03-29T08:38:46.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:46.555+0000 7fb583400700 1 --2- 172.21.15.37:0/2459992974 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb584102e00 0x7fb584115d00 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fb574002a50 tx=0x7fb5740046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:38:46.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:46.555+0000 7fb57b400700 1 -- 172.21.15.37:0/2459992974 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb574007cc0 con 0x7fb584102e00 2024-03-29T08:38:46.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:46.555+0000 7fb57b400700 1 -- 172.21.15.37:0/2459992974 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fb57400f040 con 0x7fb584102e00 2024-03-29T08:38:46.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:46.555+0000 7fb57b400700 1 -- 172.21.15.37:0/2459992974 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb57400d9c0 con 0x7fb584102e00 2024-03-29T08:38:46.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:46.555+0000 7fb58a400700 1 -- 172.21.15.37:0/2459992974 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb58410ff40 con 0x7fb584102e00 2024-03-29T08:38:46.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:46.555+0000 7fb58a400700 1 -- 172.21.15.37:0/2459992974 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb584110500 con 0x7fb584102e00 2024-03-29T08:38:46.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:46.556+0000 7fb58a400700 1 -- 172.21.15.37:0/2459992974 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5840fe550 con 0x7fb584102e00 2024-03-29T08:38:46.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:46.557+0000 7fb57b400700 1 -- 172.21.15.37:0/2459992974 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fb57400db20 con 0x7fb584102e00 2024-03-29T08:38:46.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:46.558+0000 7fb57b400700 1 --2- 172.21.15.37:0/2459992974 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fb56806cc30 0x7fb56806f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:38:46.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:46.558+0000 7fb57b400700 1 -- 172.21.15.37:0/2459992974 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fb57408bde0 con 0x7fb584102e00 2024-03-29T08:38:46.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:46.559+0000 7fb582a00700 1 --2- 172.21.15.37:0/2459992974 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fb56806cc30 0x7fb56806f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:38:46.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:46.559+0000 7fb582a00700 1 --2- 172.21.15.37:0/2459992974 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fb56806cc30 0x7fb56806f0e0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fb5841115a0 tx=0x7fb570009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:38:46.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:46.563+0000 7fb57b400700 1 -- 172.21.15.37:0/2459992974 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fb5740597b0 con 0x7fb584102e00 2024-03-29T08:38:46.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:46.796+0000 7fb58a400700 1 -- 172.21.15.37:0/2459992974 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb58404bf40 con 0x7fb56806cc30 2024-03-29T08:38:46.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:46.797+0000 7fb57b400700 1 -- 172.21.15.37:0/2459992974 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fb58404bf40 con 0x7fb56806cc30 2024-03-29T08:38:46.797 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-29T08:38:46.797 INFO:teuthology.orchestra.run.smithi037.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59", 2024-03-29T08:38:46.797 INFO:teuthology.orchestra.run.smithi037.stdout: "in_progress": true, 2024-03-29T08:38:46.797 INFO:teuthology.orchestra.run.smithi037.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-29T08:38:46.797 INFO:teuthology.orchestra.run.smithi037.stdout: "services_complete": [ 2024-03-29T08:38:46.797 INFO:teuthology.orchestra.run.smithi037.stdout: "crash", 2024-03-29T08:38:46.797 INFO:teuthology.orchestra.run.smithi037.stdout: "osd", 2024-03-29T08:38:46.797 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr", 2024-03-29T08:38:46.797 INFO:teuthology.orchestra.run.smithi037.stdout: "mds", 2024-03-29T08:38:46.798 INFO:teuthology.orchestra.run.smithi037.stdout: "mon" 2024-03-29T08:38:46.798 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-29T08:38:46.798 INFO:teuthology.orchestra.run.smithi037.stdout: "progress": "16/21 daemons upgraded", 2024-03-29T08:38:46.798 INFO:teuthology.orchestra.run.smithi037.stdout: "message": "Currently upgrading mds daemons", 2024-03-29T08:38:46.798 INFO:teuthology.orchestra.run.smithi037.stdout: "is_paused": false 2024-03-29T08:38:46.798 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-29T08:38:46.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:46.799+0000 7fb58a400700 1 -- 172.21.15.37:0/2459992974 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fb56806cc30 msgr2=0x7fb56806f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:38:46.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:46.799+0000 7fb58a400700 1 --2- 172.21.15.37:0/2459992974 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fb56806cc30 0x7fb56806f0e0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fb5841115a0 tx=0x7fb570009e30 comp rx=0 tx=0).stop 2024-03-29T08:38:46.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:46.800+0000 7fb58a400700 1 -- 172.21.15.37:0/2459992974 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb584102e00 msgr2=0x7fb584115d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:38:46.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:46.800+0000 7fb58a400700 1 --2- 172.21.15.37:0/2459992974 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb584102e00 0x7fb584115d00 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fb574002a50 tx=0x7fb5740046b0 comp rx=0 tx=0).stop 2024-03-29T08:38:46.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:46.800+0000 7fb58a400700 1 -- 172.21.15.37:0/2459992974 shutdown_connections 2024-03-29T08:38:46.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:46.800+0000 7fb58a400700 1 --2- 172.21.15.37:0/2459992974 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fb56806cc30 0x7fb56806f0e0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:38:46.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:46.800+0000 7fb58a400700 1 --2- 172.21.15.37:0/2459992974 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb584102e00 0x7fb584115d00 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:38:46.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:46.800+0000 7fb58a400700 1 --2- 172.21.15.37:0/2459992974 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb5841037a0 0x7fb584116240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:38:46.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:46.800+0000 7fb58a400700 1 -- 172.21.15.37:0/2459992974 >> 172.21.15.37:0/2459992974 conn(0x7fb5840f89b0 msgr2=0x7fb5840f9ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:38:46.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:46.800+0000 7fb58a400700 1 -- 172.21.15.37:0/2459992974 shutdown_connections 2024-03-29T08:38:46.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:46.800+0000 7fb58a400700 1 -- 172.21.15.37:0/2459992974 wait complete. 2024-03-29T08:38:46.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:38:46 smithi037 ceph-mon[142325]: from='client.35518 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:38:46.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:38:46 smithi037 ceph-mon[142325]: pgmap v837: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:38:46.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:38:46 smithi037 ceph-mon[142325]: from='client.35522 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:38:46.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:38:46 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/198227406' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:38:46.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:38:46 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/3714656259' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T08:38:46.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:46.936+0000 7f30a1000700 1 -- 172.21.15.37:0/2320009940 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f309c102e00 msgr2=0x7f309c1031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:38:46.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:46.936+0000 7f30a1000700 1 --2- 172.21.15.37:0/2320009940 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f309c102e00 0x7f309c1031d0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f308c004940 tx=0x7f308c009e30 comp rx=0 tx=0).stop 2024-03-29T08:38:46.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:46.936+0000 7f30a1000700 1 -- 172.21.15.37:0/2320009940 shutdown_connections 2024-03-29T08:38:46.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:46.936+0000 7f30a1000700 1 --2- 172.21.15.37:0/2320009940 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f309c1037a0 0x7f309c0fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:38:46.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:46.936+0000 7f30a1000700 1 --2- 172.21.15.37:0/2320009940 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f309c102e00 0x7f309c1031d0 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:38:46.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:46.936+0000 7f30a1000700 1 -- 172.21.15.37:0/2320009940 >> 172.21.15.37:0/2320009940 conn(0x7f309c0f89b0 msgr2=0x7f309c0fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:38:46.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:46.937+0000 7f30a1000700 1 -- 172.21.15.37:0/2320009940 shutdown_connections 2024-03-29T08:38:46.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:46.937+0000 7f30a1000700 1 -- 172.21.15.37:0/2320009940 wait complete. 2024-03-29T08:38:46.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:46.937+0000 7f30a1000700 1 Processor -- start 2024-03-29T08:38:46.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:46.937+0000 7f30a1000700 1 -- start start 2024-03-29T08:38:46.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:46.938+0000 7f30a1000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f309c102e00 0x7f309c0702c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:38:46.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:46.938+0000 7f30a1000700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f309c1037a0 0x7f309c070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:38:46.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:46.938+0000 7f30a1000700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f309c070d90 con 0x7f309c102e00 2024-03-29T08:38:46.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:46.938+0000 7f30a1000700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f309c070ed0 con 0x7f309c1037a0 2024-03-29T08:38:46.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:46.938+0000 7f3099e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f309c102e00 0x7f309c0702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:38:46.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:46.938+0000 7f3099e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f309c102e00 0x7f309c0702c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35548/0 (socket says 172.21.15.37:35548) 2024-03-29T08:38:46.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:46.938+0000 7f3099e00700 1 -- 172.21.15.37:0/3086015600 learned_addr learned my addr 172.21.15.37:0/3086015600 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:38:46.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:46.938+0000 7f3099400700 1 --2- 172.21.15.37:0/3086015600 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f309c1037a0 0x7f309c070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:38:46.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:46.938+0000 7f3099e00700 1 -- 172.21.15.37:0/3086015600 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f309c1037a0 msgr2=0x7f309c070800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:38:46.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:46.938+0000 7f3099e00700 1 --2- 172.21.15.37:0/3086015600 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f309c1037a0 0x7f309c070800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:38:46.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:46.938+0000 7f3099e00700 1 -- 172.21.15.37:0/3086015600 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f308c005040 con 0x7f309c102e00 2024-03-29T08:38:46.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:46.939+0000 7f3099e00700 1 --2- 172.21.15.37:0/3086015600 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f309c102e00 0x7f309c0702c0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f308c009180 tx=0x7f308c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:38:46.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:46.939+0000 7f3092000700 1 -- 172.21.15.37:0/3086015600 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f308c00e040 con 0x7f309c102e00 2024-03-29T08:38:46.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:46.939+0000 7f3092000700 1 -- 172.21.15.37:0/3086015600 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f308c004290 con 0x7f309c102e00 2024-03-29T08:38:46.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:46.939+0000 7f30a1000700 1 -- 172.21.15.37:0/3086015600 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f309c071120 con 0x7f309c102e00 2024-03-29T08:38:46.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:46.939+0000 7f3092000700 1 -- 172.21.15.37:0/3086015600 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f308c01c3f0 con 0x7f309c102e00 2024-03-29T08:38:46.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:46.939+0000 7f30a1000700 1 -- 172.21.15.37:0/3086015600 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f309c11a270 con 0x7f309c102e00 2024-03-29T08:38:46.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:46.940+0000 7f30a1000700 1 -- 172.21.15.37:0/3086015600 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f309c0fe550 con 0x7f309c102e00 2024-03-29T08:38:46.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:46.942+0000 7f3092000700 1 -- 172.21.15.37:0/3086015600 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f308c013790 con 0x7f309c102e00 2024-03-29T08:38:46.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:46.943+0000 7f3092000700 1 --2- 172.21.15.37:0/3086015600 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f308006cc20 0x7f308006f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:38:46.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:46.943+0000 7f3092000700 1 -- 172.21.15.37:0/3086015600 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f308c08d370 con 0x7f309c102e00 2024-03-29T08:38:46.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:46.943+0000 7f3099400700 1 --2- 172.21.15.37:0/3086015600 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f308006cc20 0x7f308006f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:38:46.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:46.943+0000 7f3099400700 1 --2- 172.21.15.37:0/3086015600 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f308006cc20 0x7f308006f0d0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f3088009fd0 tx=0x7f3088009f20 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:38:46.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:46.947+0000 7f3092000700 1 -- 172.21.15.37:0/3086015600 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f308c057330 con 0x7f309c102e00 2024-03-29T08:38:47.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:38:46 smithi148 ceph-mon[106692]: from='client.35518 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:38:47.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:38:46 smithi148 ceph-mon[106692]: pgmap v837: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:38:47.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:38:46 smithi148 ceph-mon[106692]: from='client.35522 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:38:47.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:38:46 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/198227406' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:38:47.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:38:46 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/3714656259' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T08:38:47.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:47.245+0000 7f30a1000700 1 -- 172.21.15.37:0/3086015600 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f309c05f5f0 con 0x7f309c102e00 2024-03-29T08:38:47.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:47.246+0000 7f3092000700 1 -- 172.21.15.37:0/3086015600 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+3583 (secure 0 0 0) 0x7f308c013420 con 0x7f309c102e00 2024-03-29T08:38:47.246 INFO:teuthology.orchestra.run.smithi037.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem is offline 2024-03-29T08:38:47.246 INFO:teuthology.orchestra.run.smithi037.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-03-29T08:38:47.246 INFO:teuthology.orchestra.run.smithi037.stdout: host smithi148 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/mon.smithi148/config 2024-03-29T08:38:47.246 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/podman: timeout after 895 seconds 2024-03-29T08:38:47.246 INFO:teuthology.orchestra.run.smithi037.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59 -e NODE_NAME=smithi148 -e CEPH_USE_RANDOM_NONCE=1 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4:/var/run/ceph:z -v /var/log/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4:/var/log/ceph:z -v /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpkx2rz2d4:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59 inventory --format=json-pretty --filter-for-batch 2024-03-29T08:38:47.246 INFO:teuthology.orchestra.run.smithi037.stdout:Traceback (most recent call last): 2024-03-29T08:38:47.246 INFO:teuthology.orchestra.run.smithi037.stdout: File "/usr/lib64/python3.6/runpy.py", line 193, in _run_module_as_main 2024-03-29T08:38:47.246 INFO:teuthology.orchestra.run.smithi037.stdout: "__main__", mod_spec) 2024-03-29T08:38:47.247 INFO:teuthology.orchestra.run.smithi037.stdout: File "/usr/lib64/python3.6/runpy.py", line 85, in _run_code 2024-03-29T08:38:47.247 INFO:teuthology.orchestra.run.smithi037.stdout: exec(code, run_globals) 2024-03-29T08:38:47.247 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 10889, in 2024-03-29T08:38:47.247 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 10877, in main 2024-03-29T08:38:47.247 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 2576, in _infer_config 2024-03-29T08:38:47.247 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 2492, in _infer_fsid 2024-03-29T08:38:47.247 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 2604, in _infer_image 2024-03-29T08:38:47.247 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 2479, in _validate_fsid 2024-03-29T08:38:47.247 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 7145, in command_ceph_volume 2024-03-29T08:38:47.247 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 2267, in call_throws 2024-03-29T08:38:47.247 INFO:teuthology.orchestra.run.smithi037.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59 -e NODE_NAME=smithi148 -e CEPH_USE_RANDOM_NONCE=1 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4:/var/run/ceph:z -v /var/log/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4:/var/log/ceph:z -v /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpkx2rz2d4:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59 inventory --format=json-pretty --filter-for-batch 2024-03-29T08:38:47.247 INFO:teuthology.orchestra.run.smithi037.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-03-29T08:38:47.248 INFO:teuthology.orchestra.run.smithi037.stdout: fs cephfs is degraded 2024-03-29T08:38:47.248 INFO:teuthology.orchestra.run.smithi037.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-03-29T08:38:47.248 INFO:teuthology.orchestra.run.smithi037.stdout: fs cephfs is offline because no MDS is active for it. 2024-03-29T08:38:47.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:47.248+0000 7f30a1000700 1 -- 172.21.15.37:0/3086015600 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f308006cc20 msgr2=0x7f308006f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:38:47.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:47.248+0000 7f30a1000700 1 --2- 172.21.15.37:0/3086015600 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f308006cc20 0x7f308006f0d0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f3088009fd0 tx=0x7f3088009f20 comp rx=0 tx=0).stop 2024-03-29T08:38:47.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:47.248+0000 7f30a1000700 1 -- 172.21.15.37:0/3086015600 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f309c102e00 msgr2=0x7f309c0702c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:38:47.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:47.248+0000 7f30a1000700 1 --2- 172.21.15.37:0/3086015600 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f309c102e00 0x7f309c0702c0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f308c009180 tx=0x7f308c009e30 comp rx=0 tx=0).stop 2024-03-29T08:38:47.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:47.248+0000 7f30a1000700 1 -- 172.21.15.37:0/3086015600 shutdown_connections 2024-03-29T08:38:47.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:47.248+0000 7f30a1000700 1 --2- 172.21.15.37:0/3086015600 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f308006cc20 0x7f308006f0d0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:38:47.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:47.248+0000 7f30a1000700 1 --2- 172.21.15.37:0/3086015600 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f309c102e00 0x7f309c0702c0 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:38:47.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:47.248+0000 7f30a1000700 1 --2- 172.21.15.37:0/3086015600 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f309c1037a0 0x7f309c070800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:38:47.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:47.248+0000 7f30a1000700 1 -- 172.21.15.37:0/3086015600 >> 172.21.15.37:0/3086015600 conn(0x7f309c0f89b0 msgr2=0x7f309c0f9a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:38:47.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:47.248+0000 7f30a1000700 1 -- 172.21.15.37:0/3086015600 shutdown_connections 2024-03-29T08:38:47.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:38:47.248+0000 7f30a1000700 1 -- 172.21.15.37:0/3086015600 wait complete. 2024-03-29T08:38:47.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:38:47 smithi037 ceph-mon[142325]: from='client.35534 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:38:47.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:38:47 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/3086015600' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T08:38:48.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:38:47 smithi148 ceph-mon[106692]: from='client.35534 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:38:48.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:38:47 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/3086015600' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T08:38:48.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:38:48 smithi037 ceph-mon[142325]: pgmap v838: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:38:49.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:38:48 smithi148 ceph-mon[106692]: pgmap v838: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:38:50.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:38:50 smithi037 ceph-mon[142325]: pgmap v839: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:38:51.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:38:50 smithi148 ceph-mon[106692]: pgmap v839: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:38:52.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:38:52 smithi037 ceph-mon[142325]: pgmap v840: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:38:53.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:38:52 smithi148 ceph-mon[106692]: pgmap v840: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:38:54.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:38:54 smithi037 ceph-mon[142325]: pgmap v841: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:38:54.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:38:54 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:38:55.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:38:54 smithi148 ceph-mon[106692]: pgmap v841: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:38:55.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:38:54 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:38:56.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:38:56 smithi037 ceph-mon[142325]: pgmap v842: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:38:57.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:38:56 smithi148 ceph-mon[106692]: pgmap v842: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:38:58.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:38:58 smithi037 ceph-mon[142325]: pgmap v843: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:38:59.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:38:58 smithi148 ceph-mon[106692]: pgmap v843: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:39:00.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:39:00 smithi037 ceph-mon[142325]: pgmap v844: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:39:01.058 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:39:00 smithi148 ceph-mon[106692]: pgmap v844: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:39:02.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:39:02 smithi037 ceph-mon[142325]: pgmap v845: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:39:03.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:39:02 smithi148 ceph-mon[106692]: pgmap v845: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:39:04.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:39:04 smithi037 ceph-mon[142325]: pgmap v846: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:39:05.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:39:04 smithi148 ceph-mon[106692]: pgmap v846: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:39:06.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:39:06 smithi037 ceph-mon[142325]: pgmap v847: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:39:07.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:39:06 smithi148 ceph-mon[106692]: pgmap v847: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:39:08.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:39:08 smithi037 ceph-mon[142325]: pgmap v848: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:39:09.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:39:08 smithi148 ceph-mon[106692]: pgmap v848: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:39:10.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:39:10 smithi148 ceph-mon[106692]: pgmap v849: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:39:10.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:39:10 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:39:10.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:39:10 smithi037 ceph-mon[142325]: pgmap v849: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:39:10.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:39:10 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:39:12.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:39:12 smithi148 ceph-mon[106692]: pgmap v850: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:39:12.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:39:12 smithi037 ceph-mon[142325]: pgmap v850: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:39:14.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:39:14 smithi148 ceph-mon[106692]: pgmap v851: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:39:14.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:39:14 smithi037 ceph-mon[142325]: pgmap v851: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:39:16.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:39:16 smithi148 ceph-mon[106692]: pgmap v852: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:39:16.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:39:16 smithi037 ceph-mon[142325]: pgmap v852: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:39:17.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:17.402+0000 7f4bfd000700 1 -- 172.21.15.37:0/3882464383 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4bf8102e00 msgr2=0x7f4bf81031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:39:17.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:17.402+0000 7f4bfd000700 1 --2- 172.21.15.37:0/3882464383 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4bf8102e00 0x7f4bf81031d0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f4be8004940 tx=0x7f4be8009e30 comp rx=0 tx=0).stop 2024-03-29T08:39:17.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:17.402+0000 7f4bfd000700 1 -- 172.21.15.37:0/3882464383 shutdown_connections 2024-03-29T08:39:17.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:17.402+0000 7f4bfd000700 1 --2- 172.21.15.37:0/3882464383 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4bf81037a0 0x7f4bf80fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:39:17.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:17.402+0000 7f4bfd000700 1 --2- 172.21.15.37:0/3882464383 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4bf8102e00 0x7f4bf81031d0 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:39:17.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:17.402+0000 7f4bfd000700 1 -- 172.21.15.37:0/3882464383 >> 172.21.15.37:0/3882464383 conn(0x7f4bf80f89b0 msgr2=0x7f4bf80fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:39:17.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:17.402+0000 7f4bfd000700 1 -- 172.21.15.37:0/3882464383 shutdown_connections 2024-03-29T08:39:17.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:17.402+0000 7f4bfd000700 1 -- 172.21.15.37:0/3882464383 wait complete. 2024-03-29T08:39:17.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:17.403+0000 7f4bfd000700 1 Processor -- start 2024-03-29T08:39:17.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:17.403+0000 7f4bfd000700 1 -- start start 2024-03-29T08:39:17.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:17.404+0000 7f4bfd000700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4bf8102e00 0x7f4bf80702c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:39:17.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:17.404+0000 7f4bfd000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4bf81037a0 0x7f4bf8070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:39:17.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:17.404+0000 7f4bfd000700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bf8070d90 con 0x7f4bf81037a0 2024-03-29T08:39:17.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:17.404+0000 7f4bfd000700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bf8070ed0 con 0x7f4bf8102e00 2024-03-29T08:39:17.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:17.404+0000 7f4bf5400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4bf81037a0 0x7f4bf8070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:39:17.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:17.404+0000 7f4bf5400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4bf81037a0 0x7f4bf8070800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54928/0 (socket says 172.21.15.37:54928) 2024-03-29T08:39:17.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:17.404+0000 7f4bf5400700 1 -- 172.21.15.37:0/2061996663 learned_addr learned my addr 172.21.15.37:0/2061996663 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:39:17.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:17.404+0000 7f4bf5e00700 1 --2- 172.21.15.37:0/2061996663 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4bf8102e00 0x7f4bf80702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:39:17.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:17.404+0000 7f4bf5400700 1 -- 172.21.15.37:0/2061996663 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4bf8102e00 msgr2=0x7f4bf80702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:39:17.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:17.404+0000 7f4bf5400700 1 --2- 172.21.15.37:0/2061996663 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4bf8102e00 0x7f4bf80702c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:39:17.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:17.404+0000 7f4bf5400700 1 -- 172.21.15.37:0/2061996663 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4be8005040 con 0x7f4bf81037a0 2024-03-29T08:39:17.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:17.405+0000 7f4bf5400700 1 --2- 172.21.15.37:0/2061996663 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4bf81037a0 0x7f4bf8070800 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f4be4002a50 tx=0x7f4be40046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:39:17.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:17.405+0000 7f4bee000700 1 -- 172.21.15.37:0/2061996663 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4be4007cc0 con 0x7f4bf81037a0 2024-03-29T08:39:17.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:17.405+0000 7f4bee000700 1 -- 172.21.15.37:0/2061996663 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f4be400f040 con 0x7f4bf81037a0 2024-03-29T08:39:17.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:17.405+0000 7f4bfd000700 1 -- 172.21.15.37:0/2061996663 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bf8119d40 con 0x7f4bf81037a0 2024-03-29T08:39:17.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:17.405+0000 7f4bee000700 1 -- 172.21.15.37:0/2061996663 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4be400da30 con 0x7f4bf81037a0 2024-03-29T08:39:17.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:17.405+0000 7f4bfd000700 1 -- 172.21.15.37:0/2061996663 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bf811a1c0 con 0x7f4bf81037a0 2024-03-29T08:39:17.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:17.406+0000 7f4bfd000700 1 -- 172.21.15.37:0/2061996663 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4bf80fe550 con 0x7f4bf81037a0 2024-03-29T08:39:17.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:17.407+0000 7f4bee000700 1 -- 172.21.15.37:0/2061996663 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f4be400db90 con 0x7f4bf81037a0 2024-03-29T08:39:17.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:17.408+0000 7f4bee000700 1 --2- 172.21.15.37:0/2061996663 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f4bdc06cc30 0x7f4bdc06f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:39:17.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:17.408+0000 7f4bee000700 1 -- 172.21.15.37:0/2061996663 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f4be408bcf0 con 0x7f4bf81037a0 2024-03-29T08:39:17.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:17.409+0000 7f4bf5e00700 1 --2- 172.21.15.37:0/2061996663 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f4bdc06cc30 0x7f4bdc06f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:39:17.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:17.409+0000 7f4bf5e00700 1 --2- 172.21.15.37:0/2061996663 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f4bdc06cc30 0x7f4bdc06f0e0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f4be8009fd0 tx=0x7f4be8009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:39:17.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:17.413+0000 7f4bee000700 1 -- 172.21.15.37:0/2061996663 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f4be4055c30 con 0x7f4bf81037a0 2024-03-29T08:39:17.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:17.646+0000 7f4bfd000700 1 -- 172.21.15.37:0/2061996663 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4bf804bed0 con 0x7f4bdc06cc30 2024-03-29T08:39:17.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:17.647+0000 7f4bee000700 1 -- 172.21.15.37:0/2061996663 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f4bf804bed0 con 0x7f4bdc06cc30 2024-03-29T08:39:17.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:17.649+0000 7f4bfd000700 1 -- 172.21.15.37:0/2061996663 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f4bdc06cc30 msgr2=0x7f4bdc06f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:39:17.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:17.649+0000 7f4bfd000700 1 --2- 172.21.15.37:0/2061996663 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f4bdc06cc30 0x7f4bdc06f0e0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f4be8009fd0 tx=0x7f4be8009e30 comp rx=0 tx=0).stop 2024-03-29T08:39:17.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:17.649+0000 7f4bfd000700 1 -- 172.21.15.37:0/2061996663 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4bf81037a0 msgr2=0x7f4bf8070800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:39:17.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:17.649+0000 7f4bfd000700 1 --2- 172.21.15.37:0/2061996663 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4bf81037a0 0x7f4bf8070800 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f4be4002a50 tx=0x7f4be40046b0 comp rx=0 tx=0).stop 2024-03-29T08:39:17.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:17.649+0000 7f4bfd000700 1 -- 172.21.15.37:0/2061996663 shutdown_connections 2024-03-29T08:39:17.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:17.649+0000 7f4bfd000700 1 --2- 172.21.15.37:0/2061996663 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f4bdc06cc30 0x7f4bdc06f0e0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:39:17.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:17.649+0000 7f4bfd000700 1 --2- 172.21.15.37:0/2061996663 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4bf8102e00 0x7f4bf80702c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:39:17.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:17.649+0000 7f4bfd000700 1 --2- 172.21.15.37:0/2061996663 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4bf81037a0 0x7f4bf8070800 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:39:17.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:17.649+0000 7f4bfd000700 1 -- 172.21.15.37:0/2061996663 >> 172.21.15.37:0/2061996663 conn(0x7f4bf80f89b0 msgr2=0x7f4bf80f99f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:39:17.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:17.650+0000 7f4bfd000700 1 -- 172.21.15.37:0/2061996663 shutdown_connections 2024-03-29T08:39:17.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:17.650+0000 7f4bfd000700 1 -- 172.21.15.37:0/2061996663 wait complete. 2024-03-29T08:39:17.664 INFO:teuthology.orchestra.run.smithi037.stdout:true 2024-03-29T08:39:17.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:17.788+0000 7ff2df200700 1 -- 172.21.15.37:0/2323626676 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff2e0102c40 msgr2=0x7ff2e0103010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:39:17.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:17.788+0000 7ff2df200700 1 --2- 172.21.15.37:0/2323626676 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff2e0102c40 0x7ff2e0103010 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7ff2d0004940 tx=0x7ff2d0009e30 comp rx=0 tx=0).stop 2024-03-29T08:39:17.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:17.788+0000 7ff2df200700 1 -- 172.21.15.37:0/2323626676 shutdown_connections 2024-03-29T08:39:17.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:17.788+0000 7ff2df200700 1 --2- 172.21.15.37:0/2323626676 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff2e01035e0 0x7ff2e00fcc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:39:17.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:17.788+0000 7ff2df200700 1 --2- 172.21.15.37:0/2323626676 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff2e0102c40 0x7ff2e0103010 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:39:17.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:17.788+0000 7ff2df200700 1 -- 172.21.15.37:0/2323626676 >> 172.21.15.37:0/2323626676 conn(0x7ff2e00f8830 msgr2=0x7ff2e00fac40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:39:17.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:17.788+0000 7ff2df200700 1 -- 172.21.15.37:0/2323626676 shutdown_connections 2024-03-29T08:39:17.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:17.788+0000 7ff2df200700 1 -- 172.21.15.37:0/2323626676 wait complete. 2024-03-29T08:39:17.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:17.789+0000 7ff2df200700 1 Processor -- start 2024-03-29T08:39:17.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:17.789+0000 7ff2df200700 1 -- start start 2024-03-29T08:39:17.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:17.790+0000 7ff2df200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff2e0102c40 0x7ff2e0115ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:39:17.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:17.790+0000 7ff2df200700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff2e01035e0 0x7ff2e0115ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:39:17.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:17.790+0000 7ff2df200700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2e0116640 con 0x7ff2e0102c40 2024-03-29T08:39:17.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:17.790+0000 7ff2df200700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2e010fa30 con 0x7ff2e01035e0 2024-03-29T08:39:17.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:17.790+0000 7ff2dde00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff2e0102c40 0x7ff2e0115ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:39:17.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:17.790+0000 7ff2dde00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff2e0102c40 0x7ff2e0115ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54950/0 (socket says 172.21.15.37:54950) 2024-03-29T08:39:17.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:17.790+0000 7ff2dde00700 1 -- 172.21.15.37:0/1816501131 learned_addr learned my addr 172.21.15.37:0/1816501131 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:39:17.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:17.790+0000 7ff2dd400700 1 --2- 172.21.15.37:0/1816501131 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff2e01035e0 0x7ff2e0115ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:39:17.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:17.790+0000 7ff2dde00700 1 -- 172.21.15.37:0/1816501131 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff2e01035e0 msgr2=0x7ff2e0115ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:39:17.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:17.790+0000 7ff2dde00700 1 --2- 172.21.15.37:0/1816501131 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff2e01035e0 0x7ff2e0115ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:39:17.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:17.790+0000 7ff2dde00700 1 -- 172.21.15.37:0/1816501131 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2d0005040 con 0x7ff2e0102c40 2024-03-29T08:39:17.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:17.790+0000 7ff2dd400700 1 --2- 172.21.15.37:0/1816501131 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff2e01035e0 0x7ff2e0115ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:39:17.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:17.791+0000 7ff2dde00700 1 --2- 172.21.15.37:0/1816501131 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff2e0102c40 0x7ff2e0115ab0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7ff2d0009390 tx=0x7ff2d0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:39:17.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:17.791+0000 7ff2d6000700 1 -- 172.21.15.37:0/1816501131 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff2d000e040 con 0x7ff2e0102c40 2024-03-29T08:39:17.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:17.791+0000 7ff2d6000700 1 -- 172.21.15.37:0/1816501131 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ff2d0013420 con 0x7ff2e0102c40 2024-03-29T08:39:17.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:17.791+0000 7ff2df200700 1 -- 172.21.15.37:0/1816501131 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2e010fcb0 con 0x7ff2e0102c40 2024-03-29T08:39:17.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:17.791+0000 7ff2d6000700 1 -- 172.21.15.37:0/1816501131 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff2d001c630 con 0x7ff2e0102c40 2024-03-29T08:39:17.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:17.791+0000 7ff2df200700 1 -- 172.21.15.37:0/1816501131 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2e0110270 con 0x7ff2e0102c40 2024-03-29T08:39:17.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:17.792+0000 7ff2df200700 1 -- 172.21.15.37:0/1816501131 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2e00fe390 con 0x7ff2e0102c40 2024-03-29T08:39:17.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:17.793+0000 7ff2d6000700 1 -- 172.21.15.37:0/1816501131 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7ff2d00139e0 con 0x7ff2e0102c40 2024-03-29T08:39:17.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:17.794+0000 7ff2d6000700 1 --2- 172.21.15.37:0/1816501131 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7ff2c406cc30 0x7ff2c406f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:39:17.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:17.794+0000 7ff2d6000700 1 -- 172.21.15.37:0/1816501131 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7ff2d008d250 con 0x7ff2e0102c40 2024-03-29T08:39:17.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:17.794+0000 7ff2dd400700 1 --2- 172.21.15.37:0/1816501131 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7ff2c406cc30 0x7ff2c406f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:39:17.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:17.795+0000 7ff2dd400700 1 --2- 172.21.15.37:0/1816501131 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7ff2c406cc30 0x7ff2c406f0e0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7ff2cc009b80 tx=0x7ff2cc00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:39:17.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:17.798+0000 7ff2d6000700 1 -- 172.21.15.37:0/1816501131 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7ff2d005add0 con 0x7ff2e0102c40 2024-03-29T08:39:18.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:18.031+0000 7ff2df200700 1 -- 172.21.15.37:0/1816501131 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff2e0109bf0 con 0x7ff2c406cc30 2024-03-29T08:39:18.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:18.032+0000 7ff2d6000700 1 -- 172.21.15.37:0/1816501131 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7ff2e0109bf0 con 0x7ff2c406cc30 2024-03-29T08:39:18.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:18.035+0000 7ff2df200700 1 -- 172.21.15.37:0/1816501131 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7ff2c406cc30 msgr2=0x7ff2c406f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:39:18.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:18.035+0000 7ff2df200700 1 --2- 172.21.15.37:0/1816501131 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7ff2c406cc30 0x7ff2c406f0e0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7ff2cc009b80 tx=0x7ff2cc00b040 comp rx=0 tx=0).stop 2024-03-29T08:39:18.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:18.035+0000 7ff2df200700 1 -- 172.21.15.37:0/1816501131 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff2e0102c40 msgr2=0x7ff2e0115ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:39:18.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:18.035+0000 7ff2df200700 1 --2- 172.21.15.37:0/1816501131 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff2e0102c40 0x7ff2e0115ab0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7ff2d0009390 tx=0x7ff2d0009e30 comp rx=0 tx=0).stop 2024-03-29T08:39:18.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:18.035+0000 7ff2df200700 1 -- 172.21.15.37:0/1816501131 shutdown_connections 2024-03-29T08:39:18.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:18.035+0000 7ff2df200700 1 --2- 172.21.15.37:0/1816501131 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7ff2c406cc30 0x7ff2c406f0e0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:39:18.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:18.035+0000 7ff2df200700 1 --2- 172.21.15.37:0/1816501131 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff2e0102c40 0x7ff2e0115ab0 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:39:18.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:18.035+0000 7ff2df200700 1 --2- 172.21.15.37:0/1816501131 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff2e01035e0 0x7ff2e0115ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:39:18.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:18.035+0000 7ff2df200700 1 -- 172.21.15.37:0/1816501131 >> 172.21.15.37:0/1816501131 conn(0x7ff2e00f8830 msgr2=0x7ff2e00f97b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:39:18.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:18.035+0000 7ff2df200700 1 -- 172.21.15.37:0/1816501131 shutdown_connections 2024-03-29T08:39:18.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:18.035+0000 7ff2df200700 1 -- 172.21.15.37:0/1816501131 wait complete. 2024-03-29T08:39:18.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:18.171+0000 7f60b5c00700 1 -- 172.21.15.37:0/95568382 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f60b0065630 msgr2=0x7f60b0065a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:39:18.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:18.171+0000 7f60b5c00700 1 --2- 172.21.15.37:0/95568382 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f60b0065630 0x7f60b0065a80 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f609c004940 tx=0x7f609c009e30 comp rx=0 tx=0).stop 2024-03-29T08:39:18.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:18.171+0000 7f60b5c00700 1 -- 172.21.15.37:0/95568382 shutdown_connections 2024-03-29T08:39:18.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:18.171+0000 7f60b5c00700 1 --2- 172.21.15.37:0/95568382 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f60b0065630 0x7f60b0065a80 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:39:18.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:18.171+0000 7f60b5c00700 1 --2- 172.21.15.37:0/95568382 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f60b0102ca0 0x7f60b0103070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:39:18.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:18.171+0000 7f60b5c00700 1 -- 172.21.15.37:0/95568382 >> 172.21.15.37:0/95568382 conn(0x7f60b0071b70 msgr2=0x7f60b0071f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:39:18.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:18.171+0000 7f60b5c00700 1 -- 172.21.15.37:0/95568382 shutdown_connections 2024-03-29T08:39:18.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:18.171+0000 7f60b5c00700 1 -- 172.21.15.37:0/95568382 wait complete. 2024-03-29T08:39:18.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:18.172+0000 7f60b5c00700 1 Processor -- start 2024-03-29T08:39:18.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:18.172+0000 7f60b5c00700 1 -- start start 2024-03-29T08:39:18.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:18.173+0000 7f60b5c00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f60b0065630 0x7f60b0115aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:39:18.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:18.173+0000 7f60b5c00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f60b0102ca0 0x7f60b0115fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:39:18.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:18.173+0000 7f60b5c00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60b0116630 con 0x7f60b0065630 2024-03-29T08:39:18.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:18.173+0000 7f60b5c00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60b010fa70 con 0x7f60b0102ca0 2024-03-29T08:39:18.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:18.173+0000 7f60afe00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f60b0065630 0x7f60b0115aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:39:18.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:18.173+0000 7f60afe00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f60b0065630 0x7f60b0115aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54962/0 (socket says 172.21.15.37:54962) 2024-03-29T08:39:18.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:18.173+0000 7f60afe00700 1 -- 172.21.15.37:0/1205935278 learned_addr learned my addr 172.21.15.37:0/1205935278 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:39:18.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:18.173+0000 7f60af400700 1 --2- 172.21.15.37:0/1205935278 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f60b0102ca0 0x7f60b0115fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:39:18.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:18.173+0000 7f60afe00700 1 -- 172.21.15.37:0/1205935278 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f60b0102ca0 msgr2=0x7f60b0115fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:39:18.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:18.173+0000 7f60afe00700 1 --2- 172.21.15.37:0/1205935278 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f60b0102ca0 0x7f60b0115fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:39:18.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:18.174+0000 7f60afe00700 1 -- 172.21.15.37:0/1205935278 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f609c005040 con 0x7f60b0065630 2024-03-29T08:39:18.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:18.174+0000 7f60afe00700 1 --2- 172.21.15.37:0/1205935278 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f60b0065630 0x7f60b0115aa0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f60a0002a80 tx=0x7f60a00046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:39:18.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:18.174+0000 7f60a7e00700 1 -- 172.21.15.37:0/1205935278 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f60a0007cc0 con 0x7f60b0065630 2024-03-29T08:39:18.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:18.174+0000 7f60a7e00700 1 -- 172.21.15.37:0/1205935278 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f60a000f040 con 0x7f60b0065630 2024-03-29T08:39:18.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:18.174+0000 7f60a7e00700 1 -- 172.21.15.37:0/1205935278 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f60a000da60 con 0x7f60b0065630 2024-03-29T08:39:18.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:18.174+0000 7f60b5c00700 1 -- 172.21.15.37:0/1205935278 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60b010fd50 con 0x7f60b0065630 2024-03-29T08:39:18.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:18.174+0000 7f60b5c00700 1 -- 172.21.15.37:0/1205935278 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60b0110290 con 0x7f60b0065630 2024-03-29T08:39:18.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:18.175+0000 7f60b5c00700 1 -- 172.21.15.37:0/1205935278 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60b0106840 con 0x7f60b0065630 2024-03-29T08:39:18.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:18.176+0000 7f60a7e00700 1 -- 172.21.15.37:0/1205935278 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f60a000dbc0 con 0x7f60b0065630 2024-03-29T08:39:18.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:18.177+0000 7f60a7e00700 1 --2- 172.21.15.37:0/1205935278 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f609406cc30 0x7f609406f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:39:18.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:18.177+0000 7f60a7e00700 1 -- 172.21.15.37:0/1205935278 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f60a008bc70 con 0x7f60b0065630 2024-03-29T08:39:18.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:18.177+0000 7f60af400700 1 --2- 172.21.15.37:0/1205935278 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f609406cc30 0x7f609406f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:39:18.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:18.178+0000 7f60af400700 1 --2- 172.21.15.37:0/1205935278 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f609406cc30 0x7f609406f0e0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f60b0110be0 tx=0x7f609c009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:39:18.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:18.181+0000 7f60a7e00700 1 -- 172.21.15.37:0/1205935278 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f60a0092080 con 0x7f60b0065630 2024-03-29T08:39:18.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:18.406+0000 7f60b5c00700 1 -- 172.21.15.37:0/1205935278 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f60b01110f0 con 0x7f609406cc30 2024-03-29T08:39:18.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:18.416+0000 7f60a7e00700 1 -- 172.21.15.37:0/1205935278 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f60b01110f0 con 0x7f609406cc30 2024-03-29T08:39:18.416 INFO:teuthology.orchestra.run.smithi037.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-29T08:39:18.416 INFO:teuthology.orchestra.run.smithi037.stdout:alertmanager.smithi037 smithi037 *:9093,9094 running (29m) 5m ago 42m 14.3M - 0.25.0 c8568f914cd2 2f9f9746ed0f 2024-03-29T08:39:18.416 INFO:teuthology.orchestra.run.smithi037.stdout:crash.smithi037 smithi037 running (26m) 5m ago 42m 6647k - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 178d416a3186 2024-03-29T08:39:18.416 INFO:teuthology.orchestra.run.smithi037.stdout:crash.smithi148 smithi148 running (25m) 5m ago 41m 6656k - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 23666989d58f 2024-03-29T08:39:18.416 INFO:teuthology.orchestra.run.smithi037.stdout:grafana.smithi037 smithi037 *:3000 running (28m) 5m ago 41m 74.5M - 9.4.7 954c08fa6188 6b976fe49952 2024-03-29T08:39:18.416 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi037.iifsmh smithi037 running (21m) 5m ago 35m 16.4M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 2d97f562cc5b 2024-03-29T08:39:18.416 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi037.mmwgxe smithi037 running (21m) 5m ago 35m 16.7M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff ec37037f46e8 2024-03-29T08:39:18.416 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi148.qukpjh smithi148 running (5m) 5m ago 35m 10.7M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 3257219bc046 2024-03-29T08:39:18.416 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi148.sqteih smithi148 running (21m) 5m ago 35m 16.3M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 44c15c77037a 2024-03-29T08:39:18.417 INFO:teuthology.orchestra.run.smithi037.stdout:mgr.smithi037.wjikii smithi037 *:8443,9283,8765 running (31m) 5m ago 43m 510M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff c5a1417b9417 2024-03-29T08:39:18.417 INFO:teuthology.orchestra.run.smithi037.stdout:mgr.smithi148.onhgvm smithi148 *:8443,9283,8765 running (30m) 5m ago 40m 456M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff fcfdbce062bd 2024-03-29T08:39:18.417 INFO:teuthology.orchestra.run.smithi037.stdout:mon.smithi037 smithi037 running (27m) 5m ago 43m 70.2M 2048M 18.2.2-1015-gd0e392d9 1cc2f45c17ff eea2060783af 2024-03-29T08:39:18.417 INFO:teuthology.orchestra.run.smithi037.stdout:mon.smithi148 smithi148 running (26m) 5m ago 40m 50.4M 2048M 18.2.2-1015-gd0e392d9 1cc2f45c17ff ad4b2c5748be 2024-03-29T08:39:18.417 INFO:teuthology.orchestra.run.smithi037.stdout:node-exporter.smithi037 smithi037 *:9100 running (29m) 5m ago 42m 8514k - 1.5.0 0da6a335fe13 26b8ae9cb878 2024-03-29T08:39:18.417 INFO:teuthology.orchestra.run.smithi037.stdout:node-exporter.smithi148 smithi148 *:9100 running (29m) 5m ago 41m 8464k - 1.5.0 0da6a335fe13 98414e4e64ed 2024-03-29T08:39:18.417 INFO:teuthology.orchestra.run.smithi037.stdout:osd.0 smithi037 running (25m) 5m ago 40m 108M 1619M 18.2.2-1015-gd0e392d9 1cc2f45c17ff c0c584f6245c 2024-03-29T08:39:18.417 INFO:teuthology.orchestra.run.smithi037.stdout:osd.1 smithi037 running (24m) 5m ago 39m 104M 1619M 18.2.2-1015-gd0e392d9 1cc2f45c17ff a4a2ff4cc83d 2024-03-29T08:39:18.417 INFO:teuthology.orchestra.run.smithi037.stdout:osd.2 smithi037 running (24m) 5m ago 38m 85.9M 1619M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 8b1bc71aa79e 2024-03-29T08:39:18.417 INFO:teuthology.orchestra.run.smithi037.stdout:osd.3 smithi148 running (23m) 5m ago 38m 109M 2642M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 1c656ab8ff86 2024-03-29T08:39:18.417 INFO:teuthology.orchestra.run.smithi037.stdout:osd.4 smithi148 running (22m) 5m ago 37m 101M 2642M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 966a9de11d22 2024-03-29T08:39:18.417 INFO:teuthology.orchestra.run.smithi037.stdout:osd.5 smithi148 running (22m) 5m ago 37m 89.4M 2642M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 474c4f9212ce 2024-03-29T08:39:18.417 INFO:teuthology.orchestra.run.smithi037.stdout:prometheus.smithi037 smithi037 *:9095 running (29m) 5m ago 41m 61.0M - 2.43.0 a07b618ecd1d 30c0223ec589 2024-03-29T08:39:18.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:18.418+0000 7f60b5c00700 1 -- 172.21.15.37:0/1205935278 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f609406cc30 msgr2=0x7f609406f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:39:18.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:18.418+0000 7f60b5c00700 1 --2- 172.21.15.37:0/1205935278 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f609406cc30 0x7f609406f0e0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f60b0110be0 tx=0x7f609c009e30 comp rx=0 tx=0).stop 2024-03-29T08:39:18.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:18.418+0000 7f60b5c00700 1 -- 172.21.15.37:0/1205935278 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f60b0065630 msgr2=0x7f60b0115aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:39:18.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:18.418+0000 7f60b5c00700 1 --2- 172.21.15.37:0/1205935278 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f60b0065630 0x7f60b0115aa0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f60a0002a80 tx=0x7f60a00046b0 comp rx=0 tx=0).stop 2024-03-29T08:39:18.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:18.419+0000 7f60b5c00700 1 -- 172.21.15.37:0/1205935278 shutdown_connections 2024-03-29T08:39:18.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:18.419+0000 7f60b5c00700 1 --2- 172.21.15.37:0/1205935278 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f609406cc30 0x7f609406f0e0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:39:18.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:18.419+0000 7f60b5c00700 1 --2- 172.21.15.37:0/1205935278 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f60b0065630 0x7f60b0115aa0 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:39:18.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:18.419+0000 7f60b5c00700 1 --2- 172.21.15.37:0/1205935278 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f60b0102ca0 0x7f60b0115fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:39:18.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:18.419+0000 7f60b5c00700 1 -- 172.21.15.37:0/1205935278 >> 172.21.15.37:0/1205935278 conn(0x7f60b0071b70 msgr2=0x7f60b01039c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:39:18.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:18.419+0000 7f60b5c00700 1 -- 172.21.15.37:0/1205935278 shutdown_connections 2024-03-29T08:39:18.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:18.419+0000 7f60b5c00700 1 -- 172.21.15.37:0/1205935278 wait complete. 2024-03-29T08:39:18.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:18.555+0000 7fe0a7e00700 1 -- 172.21.15.37:0/835732049 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe0a01037a0 msgr2=0x7fe0a00fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:39:18.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:18.555+0000 7fe0a7e00700 1 --2- 172.21.15.37:0/835732049 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe0a01037a0 0x7fe0a00fce10 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fe08c004940 tx=0x7fe08c009e30 comp rx=0 tx=0).stop 2024-03-29T08:39:18.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:18.555+0000 7fe0a7e00700 1 -- 172.21.15.37:0/835732049 shutdown_connections 2024-03-29T08:39:18.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:18.555+0000 7fe0a7e00700 1 --2- 172.21.15.37:0/835732049 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe0a01037a0 0x7fe0a00fce10 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:39:18.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:18.555+0000 7fe0a7e00700 1 --2- 172.21.15.37:0/835732049 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe0a0102e00 0x7fe0a01031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:39:18.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:18.555+0000 7fe0a7e00700 1 -- 172.21.15.37:0/835732049 >> 172.21.15.37:0/835732049 conn(0x7fe0a00f89b0 msgr2=0x7fe0a00fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:39:18.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:18.555+0000 7fe0a7e00700 1 -- 172.21.15.37:0/835732049 shutdown_connections 2024-03-29T08:39:18.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:18.555+0000 7fe0a7e00700 1 -- 172.21.15.37:0/835732049 wait complete. 2024-03-29T08:39:18.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:18.556+0000 7fe0a7e00700 1 Processor -- start 2024-03-29T08:39:18.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:18.556+0000 7fe0a7e00700 1 -- start start 2024-03-29T08:39:18.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:18.556+0000 7fe0a7e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe0a0102e00 0x7fe0a00702c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:39:18.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:18.556+0000 7fe0a7e00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe0a01037a0 0x7fe0a0070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:39:18.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:18.556+0000 7fe0a7e00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0a0070d90 con 0x7fe0a0102e00 2024-03-29T08:39:18.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:18.556+0000 7fe0a7e00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0a0070ed0 con 0x7fe0a01037a0 2024-03-29T08:39:18.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:18.557+0000 7fe0a5400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe0a0102e00 0x7fe0a00702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:39:18.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:18.557+0000 7fe0a5400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe0a0102e00 0x7fe0a00702c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54982/0 (socket says 172.21.15.37:54982) 2024-03-29T08:39:18.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:18.557+0000 7fe0a5400700 1 -- 172.21.15.37:0/2178621741 learned_addr learned my addr 172.21.15.37:0/2178621741 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:39:18.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:18.557+0000 7fe0a5400700 1 -- 172.21.15.37:0/2178621741 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe0a01037a0 msgr2=0x7fe0a0070800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:39:18.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:18.557+0000 7fe0a5400700 1 --2- 172.21.15.37:0/2178621741 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe0a01037a0 0x7fe0a0070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:39:18.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:18.557+0000 7fe0a5400700 1 -- 172.21.15.37:0/2178621741 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe08c005040 con 0x7fe0a0102e00 2024-03-29T08:39:18.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:18.557+0000 7fe0a5400700 1 --2- 172.21.15.37:0/2178621741 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe0a0102e00 0x7fe0a00702c0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fe094002a50 tx=0x7fe0940046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:39:18.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:18.558+0000 7fe09d600700 1 -- 172.21.15.37:0/2178621741 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe094007cc0 con 0x7fe0a0102e00 2024-03-29T08:39:18.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:18.558+0000 7fe09d600700 1 -- 172.21.15.37:0/2178621741 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fe09400f040 con 0x7fe0a0102e00 2024-03-29T08:39:18.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:18.558+0000 7fe0a7e00700 1 -- 172.21.15.37:0/2178621741 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0a0119e00 con 0x7fe0a0102e00 2024-03-29T08:39:18.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:18.558+0000 7fe09d600700 1 -- 172.21.15.37:0/2178621741 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe094009540 con 0x7fe0a0102e00 2024-03-29T08:39:18.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:18.558+0000 7fe0a7e00700 1 -- 172.21.15.37:0/2178621741 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0a011a280 con 0x7fe0a0102e00 2024-03-29T08:39:18.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:18.559+0000 7fe0a7e00700 1 -- 172.21.15.37:0/2178621741 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0a00fe550 con 0x7fe0a0102e00 2024-03-29T08:39:18.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:18.560+0000 7fe09d600700 1 -- 172.21.15.37:0/2178621741 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fe0940096a0 con 0x7fe0a0102e00 2024-03-29T08:39:18.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:18.560+0000 7fe09d600700 1 --2- 172.21.15.37:0/2178621741 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fe08806cc30 0x7fe08806f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:39:18.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:18.561+0000 7fe09d600700 1 -- 172.21.15.37:0/2178621741 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fe09408bed0 con 0x7fe0a0102e00 2024-03-29T08:39:18.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:18.561+0000 7fe0a4a00700 1 --2- 172.21.15.37:0/2178621741 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fe08806cc30 0x7fe08806f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:39:18.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:18.561+0000 7fe0a4a00700 1 --2- 172.21.15.37:0/2178621741 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fe08806cc30 0x7fe08806f0e0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fe08c004f90 tx=0x7fe08c009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:39:18.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:18.565+0000 7fe09d600700 1 -- 172.21.15.37:0/2178621741 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fe094055e10 con 0x7fe0a0102e00 2024-03-29T08:39:18.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:39:18 smithi148 ceph-mon[106692]: pgmap v853: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:39:18.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:39:18 smithi148 ceph-mon[106692]: from='client.35542 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:39:18.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:18.864+0000 7fe0a7e00700 1 -- 172.21.15.37:0/2178621741 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fe0a004bed0 con 0x7fe0a0102e00 2024-03-29T08:39:18.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:18.865+0000 7fe09d600700 1 -- 172.21.15.37:0/2178621741 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7fe094059430 con 0x7fe0a0102e00 2024-03-29T08:39:18.865 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-29T08:39:18.865 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": { 2024-03-29T08:39:18.865 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 2 2024-03-29T08:39:18.866 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:39:18.866 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": { 2024-03-29T08:39:18.866 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 2 2024-03-29T08:39:18.866 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:39:18.866 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": { 2024-03-29T08:39:18.866 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 6 2024-03-29T08:39:18.867 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:39:18.867 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": { 2024-03-29T08:39:18.867 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 4 2024-03-29T08:39:18.867 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:39:18.867 INFO:teuthology.orchestra.run.smithi037.stdout: "overall": { 2024-03-29T08:39:18.867 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 14 2024-03-29T08:39:18.868 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-29T08:39:18.868 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-29T08:39:18.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:18.868+0000 7fe0a7e00700 1 -- 172.21.15.37:0/2178621741 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fe08806cc30 msgr2=0x7fe08806f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:39:18.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:18.868+0000 7fe0a7e00700 1 --2- 172.21.15.37:0/2178621741 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fe08806cc30 0x7fe08806f0e0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fe08c004f90 tx=0x7fe08c009e30 comp rx=0 tx=0).stop 2024-03-29T08:39:18.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:18.868+0000 7fe0a7e00700 1 -- 172.21.15.37:0/2178621741 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe0a0102e00 msgr2=0x7fe0a00702c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:39:18.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:18.868+0000 7fe0a7e00700 1 --2- 172.21.15.37:0/2178621741 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe0a0102e00 0x7fe0a00702c0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fe094002a50 tx=0x7fe0940046b0 comp rx=0 tx=0).stop 2024-03-29T08:39:18.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:18.868+0000 7fe0a7e00700 1 -- 172.21.15.37:0/2178621741 shutdown_connections 2024-03-29T08:39:18.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:18.868+0000 7fe0a7e00700 1 --2- 172.21.15.37:0/2178621741 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fe08806cc30 0x7fe08806f0e0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:39:18.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:18.868+0000 7fe0a7e00700 1 --2- 172.21.15.37:0/2178621741 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe0a0102e00 0x7fe0a00702c0 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:39:18.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:18.868+0000 7fe0a7e00700 1 --2- 172.21.15.37:0/2178621741 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe0a01037a0 0x7fe0a0070800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:39:18.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:18.868+0000 7fe0a7e00700 1 -- 172.21.15.37:0/2178621741 >> 172.21.15.37:0/2178621741 conn(0x7fe0a00f89b0 msgr2=0x7fe0a00f9ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:39:18.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:18.868+0000 7fe0a7e00700 1 -- 172.21.15.37:0/2178621741 shutdown_connections 2024-03-29T08:39:18.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:18.868+0000 7fe0a7e00700 1 -- 172.21.15.37:0/2178621741 wait complete. 2024-03-29T08:39:18.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:39:18 smithi037 ceph-mon[142325]: pgmap v853: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:39:18.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:39:18 smithi037 ceph-mon[142325]: from='client.35542 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:39:19.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:19.008+0000 7f728b400700 1 -- 172.21.15.37:0/3093719214 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7284102e00 msgr2=0x7f72841031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:39:19.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:19.008+0000 7f728b400700 1 --2- 172.21.15.37:0/3093719214 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7284102e00 0x7f72841031d0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f7278004940 tx=0x7f7278009e30 comp rx=0 tx=0).stop 2024-03-29T08:39:19.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:19.008+0000 7f728b400700 1 -- 172.21.15.37:0/3093719214 shutdown_connections 2024-03-29T08:39:19.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:19.008+0000 7f728b400700 1 --2- 172.21.15.37:0/3093719214 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f72841037a0 0x7f72840fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:39:19.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:19.008+0000 7f728b400700 1 --2- 172.21.15.37:0/3093719214 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7284102e00 0x7f72841031d0 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:39:19.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:19.008+0000 7f728b400700 1 -- 172.21.15.37:0/3093719214 >> 172.21.15.37:0/3093719214 conn(0x7f72840f89b0 msgr2=0x7f72840fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:39:19.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:19.008+0000 7f728b400700 1 -- 172.21.15.37:0/3093719214 shutdown_connections 2024-03-29T08:39:19.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:19.008+0000 7f728b400700 1 -- 172.21.15.37:0/3093719214 wait complete. 2024-03-29T08:39:19.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:19.009+0000 7f728b400700 1 Processor -- start 2024-03-29T08:39:19.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:19.009+0000 7f728b400700 1 -- start start 2024-03-29T08:39:19.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:19.010+0000 7f728b400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7284102e00 0x7f7284115cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:39:19.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:19.010+0000 7f728b400700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f72841037a0 0x7f7284116230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:39:19.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:19.010+0000 7f728b400700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7284116880 con 0x7f7284102e00 2024-03-29T08:39:19.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:19.010+0000 7f728b400700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f728410fc50 con 0x7f72841037a0 2024-03-29T08:39:19.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:19.010+0000 7f7288a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7284102e00 0x7f7284115cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:39:19.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:19.010+0000 7f7288a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7284102e00 0x7f7284115cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55000/0 (socket says 172.21.15.37:55000) 2024-03-29T08:39:19.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:19.010+0000 7f7288a00700 1 -- 172.21.15.37:0/2499190816 learned_addr learned my addr 172.21.15.37:0/2499190816 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:39:19.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:19.010+0000 7f7283e00700 1 --2- 172.21.15.37:0/2499190816 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f72841037a0 0x7f7284116230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:39:19.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:19.010+0000 7f7288a00700 1 -- 172.21.15.37:0/2499190816 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f72841037a0 msgr2=0x7f7284116230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:39:19.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:19.010+0000 7f7288a00700 1 --2- 172.21.15.37:0/2499190816 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f72841037a0 0x7f7284116230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:39:19.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:19.010+0000 7f7288a00700 1 -- 172.21.15.37:0/2499190816 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7278005040 con 0x7f7284102e00 2024-03-29T08:39:19.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:19.010+0000 7f7283e00700 1 --2- 172.21.15.37:0/2499190816 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f72841037a0 0x7f7284116230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:39:19.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:19.011+0000 7f7288a00700 1 --2- 172.21.15.37:0/2499190816 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7284102e00 0x7f7284115cf0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f7278009390 tx=0x7f7278009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:39:19.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:19.011+0000 7f7280c00700 1 -- 172.21.15.37:0/2499190816 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f727800e040 con 0x7f7284102e00 2024-03-29T08:39:19.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:19.011+0000 7f7280c00700 1 -- 172.21.15.37:0/2499190816 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f7278013420 con 0x7f7284102e00 2024-03-29T08:39:19.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:19.011+0000 7f728b400700 1 -- 172.21.15.37:0/2499190816 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f728410fed0 con 0x7f7284102e00 2024-03-29T08:39:19.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:19.011+0000 7f7280c00700 1 -- 172.21.15.37:0/2499190816 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f727801c630 con 0x7f7284102e00 2024-03-29T08:39:19.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:19.011+0000 7f728b400700 1 -- 172.21.15.37:0/2499190816 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7284110490 con 0x7f7284102e00 2024-03-29T08:39:19.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:19.012+0000 7f728b400700 1 -- 172.21.15.37:0/2499190816 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72840fe550 con 0x7f7284102e00 2024-03-29T08:39:19.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:19.013+0000 7f7280c00700 1 -- 172.21.15.37:0/2499190816 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f72780139e0 con 0x7f7284102e00 2024-03-29T08:39:19.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:19.014+0000 7f7280c00700 1 --2- 172.21.15.37:0/2499190816 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f726c06cc30 0x7f726c06f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:39:19.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:19.014+0000 7f7280c00700 1 -- 172.21.15.37:0/2499190816 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f727808d320 con 0x7f7284102e00 2024-03-29T08:39:19.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:19.015+0000 7f7283e00700 1 --2- 172.21.15.37:0/2499190816 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f726c06cc30 0x7f726c06f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:39:19.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:19.015+0000 7f7283e00700 1 --2- 172.21.15.37:0/2499190816 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f726c06cc30 0x7f726c06f0e0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f7270009fd0 tx=0x7f727000b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:39:19.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:19.019+0000 7f7280c00700 1 -- 172.21.15.37:0/2499190816 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f727805ae20 con 0x7f7284102e00 2024-03-29T08:39:19.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:19.274+0000 7f728b400700 1 -- 172.21.15.37:0/2499190816 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f728404bed0 con 0x7f7284102e00 2024-03-29T08:39:19.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:19.275+0000 7f7280c00700 1 -- 172.21.15.37:0/2499190816 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 30 v30) v1 ==== 76+0+1841 (secure 0 0 0) 0x7f727805a9b0 con 0x7f7284102e00 2024-03-29T08:39:19.275 INFO:teuthology.orchestra.run.smithi037.stdout:e30 2024-03-29T08:39:19.275 INFO:teuthology.orchestra.run.smithi037.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-29T08:39:19.275 INFO:teuthology.orchestra.run.smithi037.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T08:39:19.275 INFO:teuthology.orchestra.run.smithi037.stdout:legacy client fscid: 1 2024-03-29T08:39:19.275 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:39:19.275 INFO:teuthology.orchestra.run.smithi037.stdout:Filesystem 'cephfs' (1) 2024-03-29T08:39:19.275 INFO:teuthology.orchestra.run.smithi037.stdout:fs_name cephfs 2024-03-29T08:39:19.275 INFO:teuthology.orchestra.run.smithi037.stdout:epoch 21 2024-03-29T08:39:19.275 INFO:teuthology.orchestra.run.smithi037.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-03-29T08:39:19.275 INFO:teuthology.orchestra.run.smithi037.stdout:created 2024-03-29T08:03:29.241269+0000 2024-03-29T08:39:19.276 INFO:teuthology.orchestra.run.smithi037.stdout:modified 2024-03-29T08:17:16.237238+0000 2024-03-29T08:39:19.276 INFO:teuthology.orchestra.run.smithi037.stdout:tableserver 0 2024-03-29T08:39:19.276 INFO:teuthology.orchestra.run.smithi037.stdout:root 0 2024-03-29T08:39:19.276 INFO:teuthology.orchestra.run.smithi037.stdout:session_timeout 60 2024-03-29T08:39:19.276 INFO:teuthology.orchestra.run.smithi037.stdout:session_autoclose 300 2024-03-29T08:39:19.276 INFO:teuthology.orchestra.run.smithi037.stdout:max_file_size 1099511627776 2024-03-29T08:39:19.276 INFO:teuthology.orchestra.run.smithi037.stdout:max_xattr_size 65536 2024-03-29T08:39:19.276 INFO:teuthology.orchestra.run.smithi037.stdout:required_client_features {} 2024-03-29T08:39:19.276 INFO:teuthology.orchestra.run.smithi037.stdout:last_failure 0 2024-03-29T08:39:19.276 INFO:teuthology.orchestra.run.smithi037.stdout:last_failure_osd_epoch 82 2024-03-29T08:39:19.276 INFO:teuthology.orchestra.run.smithi037.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T08:39:19.276 INFO:teuthology.orchestra.run.smithi037.stdout:max_mds 2 2024-03-29T08:39:19.276 INFO:teuthology.orchestra.run.smithi037.stdout:in 0,1 2024-03-29T08:39:19.276 INFO:teuthology.orchestra.run.smithi037.stdout:up {} 2024-03-29T08:39:19.277 INFO:teuthology.orchestra.run.smithi037.stdout:failed 0,1 2024-03-29T08:39:19.277 INFO:teuthology.orchestra.run.smithi037.stdout:damaged 2024-03-29T08:39:19.277 INFO:teuthology.orchestra.run.smithi037.stdout:stopped 2024-03-29T08:39:19.277 INFO:teuthology.orchestra.run.smithi037.stdout:data_pools [3] 2024-03-29T08:39:19.277 INFO:teuthology.orchestra.run.smithi037.stdout:metadata_pool 2 2024-03-29T08:39:19.277 INFO:teuthology.orchestra.run.smithi037.stdout:inline_data disabled 2024-03-29T08:39:19.277 INFO:teuthology.orchestra.run.smithi037.stdout:balancer 2024-03-29T08:39:19.277 INFO:teuthology.orchestra.run.smithi037.stdout:bal_rank_mask -1 2024-03-29T08:39:19.277 INFO:teuthology.orchestra.run.smithi037.stdout:standby_count_wanted 1 2024-03-29T08:39:19.277 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:39:19.277 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:39:19.277 INFO:teuthology.orchestra.run.smithi037.stdout:Standby daemons: 2024-03-29T08:39:19.277 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:39:19.277 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi037.mmwgxe{-1:34440} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.37:6828/1248142005,v1:172.21.15.37:6829/1248142005] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:39:19.277 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi037.iifsmh{-1:34444} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.37:6826/2060882959,v1:172.21.15.37:6827/2060882959] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:39:19.278 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi148.sqteih{-1:44395} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.148:6824/4078652242,v1:172.21.15.148:6825/4078652242] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:39:19.278 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi148.qukpjh{-1:45007} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.148:6826/2394751105,v1:172.21.15.148:6827/2394751105] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:39:19.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:19.277+0000 7f728b400700 1 -- 172.21.15.37:0/2499190816 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f726c06cc30 msgr2=0x7f726c06f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:39:19.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:19.277+0000 7f728b400700 1 --2- 172.21.15.37:0/2499190816 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f726c06cc30 0x7f726c06f0e0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f7270009fd0 tx=0x7f727000b040 comp rx=0 tx=0).stop 2024-03-29T08:39:19.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:19.277+0000 7f728b400700 1 -- 172.21.15.37:0/2499190816 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7284102e00 msgr2=0x7f7284115cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:39:19.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:19.277+0000 7f728b400700 1 --2- 172.21.15.37:0/2499190816 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7284102e00 0x7f7284115cf0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f7278009390 tx=0x7f7278009e30 comp rx=0 tx=0).stop 2024-03-29T08:39:19.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:19.278+0000 7f728b400700 1 -- 172.21.15.37:0/2499190816 shutdown_connections 2024-03-29T08:39:19.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:19.278+0000 7f728b400700 1 --2- 172.21.15.37:0/2499190816 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f726c06cc30 0x7f726c06f0e0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:39:19.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:19.278+0000 7f728b400700 1 --2- 172.21.15.37:0/2499190816 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7284102e00 0x7f7284115cf0 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:39:19.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:19.278+0000 7f728b400700 1 --2- 172.21.15.37:0/2499190816 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f72841037a0 0x7f7284116230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:39:19.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:19.278+0000 7f728b400700 1 -- 172.21.15.37:0/2499190816 >> 172.21.15.37:0/2499190816 conn(0x7f72840f89b0 msgr2=0x7f72840706d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:39:19.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:19.278+0000 7f728b400700 1 -- 172.21.15.37:0/2499190816 shutdown_connections 2024-03-29T08:39:19.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:19.278+0000 7f728b400700 1 -- 172.21.15.37:0/2499190816 wait complete. 2024-03-29T08:39:19.279 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 30 2024-03-29T08:39:19.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:19.417+0000 7f8a9e800700 1 -- 172.21.15.37:0/505982114 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8a98071f90 msgr2=0x7f8a98070360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:39:19.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:19.417+0000 7f8a9e800700 1 --2- 172.21.15.37:0/505982114 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8a98071f90 0x7f8a98070360 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f8a88004970 tx=0x7f8a88009e30 comp rx=0 tx=0).stop 2024-03-29T08:39:19.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:19.417+0000 7f8a9e800700 1 -- 172.21.15.37:0/505982114 shutdown_connections 2024-03-29T08:39:19.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:19.417+0000 7f8a9e800700 1 --2- 172.21.15.37:0/505982114 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8a980708a0 0x7f8a98070d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:39:19.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:19.417+0000 7f8a9e800700 1 --2- 172.21.15.37:0/505982114 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8a98071f90 0x7f8a98070360 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:39:19.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:19.417+0000 7f8a9e800700 1 -- 172.21.15.37:0/505982114 >> 172.21.15.37:0/505982114 conn(0x7f8a980f89b0 msgr2=0x7f8a980fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:39:19.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:19.417+0000 7f8a9e800700 1 -- 172.21.15.37:0/505982114 shutdown_connections 2024-03-29T08:39:19.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:19.417+0000 7f8a9e800700 1 -- 172.21.15.37:0/505982114 wait complete. 2024-03-29T08:39:19.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:19.418+0000 7f8a9e800700 1 Processor -- start 2024-03-29T08:39:19.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:19.418+0000 7f8a9e800700 1 -- start start 2024-03-29T08:39:19.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:19.418+0000 7f8a9e800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8a980708a0 0x7f8a9811a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:39:19.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:19.418+0000 7f8a9e800700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8a98071f90 0x7f8a9811a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:39:19.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:19.418+0000 7f8a9e800700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a9811ac50 con 0x7f8a980708a0 2024-03-29T08:39:19.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:19.418+0000 7f8a9e800700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a98114020 con 0x7f8a98071f90 2024-03-29T08:39:19.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:19.419+0000 7f8a97400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8a980708a0 0x7f8a9811a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:39:19.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:19.419+0000 7f8a97400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8a980708a0 0x7f8a9811a0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55030/0 (socket says 172.21.15.37:55030) 2024-03-29T08:39:19.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:19.419+0000 7f8a97400700 1 -- 172.21.15.37:0/3992485311 learned_addr learned my addr 172.21.15.37:0/3992485311 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:39:19.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:19.419+0000 7f8a96a00700 1 --2- 172.21.15.37:0/3992485311 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8a98071f90 0x7f8a9811a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:39:19.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:19.419+0000 7f8a97400700 1 -- 172.21.15.37:0/3992485311 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8a98071f90 msgr2=0x7f8a9811a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:39:19.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:19.419+0000 7f8a97400700 1 --2- 172.21.15.37:0/3992485311 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8a98071f90 0x7f8a9811a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:39:19.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:19.419+0000 7f8a97400700 1 -- 172.21.15.37:0/3992485311 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a88005040 con 0x7f8a980708a0 2024-03-29T08:39:19.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:19.419+0000 7f8a96a00700 1 --2- 172.21.15.37:0/3992485311 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8a98071f90 0x7f8a9811a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:39:19.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:19.419+0000 7f8a97400700 1 --2- 172.21.15.37:0/3992485311 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8a980708a0 0x7f8a9811a0c0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f8a88009360 tx=0x7f8a88009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:39:19.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:19.419+0000 7f8a8f400700 1 -- 172.21.15.37:0/3992485311 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8a8800e040 con 0x7f8a980708a0 2024-03-29T08:39:19.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:19.420+0000 7f8a8f400700 1 -- 172.21.15.37:0/3992485311 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f8a88013420 con 0x7f8a980708a0 2024-03-29T08:39:19.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:19.420+0000 7f8a9e800700 1 -- 172.21.15.37:0/3992485311 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a981142a0 con 0x7f8a980708a0 2024-03-29T08:39:19.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:19.420+0000 7f8a8f400700 1 -- 172.21.15.37:0/3992485311 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8a8801c630 con 0x7f8a980708a0 2024-03-29T08:39:19.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:19.420+0000 7f8a9e800700 1 -- 172.21.15.37:0/3992485311 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a981147e0 con 0x7f8a980708a0 2024-03-29T08:39:19.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:19.421+0000 7f8a9e800700 1 -- 172.21.15.37:0/3992485311 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a98071aa0 con 0x7f8a980708a0 2024-03-29T08:39:19.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:19.422+0000 7f8a8f400700 1 -- 172.21.15.37:0/3992485311 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f8a880139e0 con 0x7f8a980708a0 2024-03-29T08:39:19.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:19.423+0000 7f8a8f400700 1 --2- 172.21.15.37:0/3992485311 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f8a7c06cc30 0x7f8a7c06f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:39:19.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:19.423+0000 7f8a8f400700 1 -- 172.21.15.37:0/3992485311 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f8a8808d2e0 con 0x7f8a980708a0 2024-03-29T08:39:19.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:19.423+0000 7f8a96a00700 1 --2- 172.21.15.37:0/3992485311 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f8a7c06cc30 0x7f8a7c06f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:39:19.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:19.424+0000 7f8a96a00700 1 --2- 172.21.15.37:0/3992485311 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f8a7c06cc30 0x7f8a7c06f0e0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f8a84009fd0 tx=0x7f8a84009ee0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:39:19.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:19.427+0000 7f8a8f400700 1 -- 172.21.15.37:0/3992485311 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f8a8805adb0 con 0x7f8a980708a0 2024-03-29T08:39:19.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:19.660+0000 7f8a9e800700 1 -- 172.21.15.37:0/3992485311 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f8a9804bed0 con 0x7f8a7c06cc30 2024-03-29T08:39:19.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:19.661+0000 7f8a8f400700 1 -- 172.21.15.37:0/3992485311 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f8a9804bed0 con 0x7f8a7c06cc30 2024-03-29T08:39:19.661 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-29T08:39:19.661 INFO:teuthology.orchestra.run.smithi037.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59", 2024-03-29T08:39:19.661 INFO:teuthology.orchestra.run.smithi037.stdout: "in_progress": true, 2024-03-29T08:39:19.661 INFO:teuthology.orchestra.run.smithi037.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-29T08:39:19.661 INFO:teuthology.orchestra.run.smithi037.stdout: "services_complete": [ 2024-03-29T08:39:19.661 INFO:teuthology.orchestra.run.smithi037.stdout: "crash", 2024-03-29T08:39:19.661 INFO:teuthology.orchestra.run.smithi037.stdout: "osd", 2024-03-29T08:39:19.661 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr", 2024-03-29T08:39:19.661 INFO:teuthology.orchestra.run.smithi037.stdout: "mds", 2024-03-29T08:39:19.661 INFO:teuthology.orchestra.run.smithi037.stdout: "mon" 2024-03-29T08:39:19.661 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-29T08:39:19.662 INFO:teuthology.orchestra.run.smithi037.stdout: "progress": "16/21 daemons upgraded", 2024-03-29T08:39:19.662 INFO:teuthology.orchestra.run.smithi037.stdout: "message": "Currently upgrading mds daemons", 2024-03-29T08:39:19.662 INFO:teuthology.orchestra.run.smithi037.stdout: "is_paused": false 2024-03-29T08:39:19.662 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-29T08:39:19.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:19.664+0000 7f8a9e800700 1 -- 172.21.15.37:0/3992485311 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f8a7c06cc30 msgr2=0x7f8a7c06f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:39:19.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:19.664+0000 7f8a9e800700 1 --2- 172.21.15.37:0/3992485311 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f8a7c06cc30 0x7f8a7c06f0e0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f8a84009fd0 tx=0x7f8a84009ee0 comp rx=0 tx=0).stop 2024-03-29T08:39:19.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:19.664+0000 7f8a9e800700 1 -- 172.21.15.37:0/3992485311 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8a980708a0 msgr2=0x7f8a9811a0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:39:19.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:19.664+0000 7f8a9e800700 1 --2- 172.21.15.37:0/3992485311 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8a980708a0 0x7f8a9811a0c0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f8a88009360 tx=0x7f8a88009e30 comp rx=0 tx=0).stop 2024-03-29T08:39:19.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:19.664+0000 7f8a9e800700 1 -- 172.21.15.37:0/3992485311 shutdown_connections 2024-03-29T08:39:19.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:19.664+0000 7f8a9e800700 1 --2- 172.21.15.37:0/3992485311 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f8a7c06cc30 0x7f8a7c06f0e0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:39:19.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:19.664+0000 7f8a9e800700 1 --2- 172.21.15.37:0/3992485311 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8a980708a0 0x7f8a9811a0c0 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:39:19.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:19.664+0000 7f8a9e800700 1 --2- 172.21.15.37:0/3992485311 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8a98071f90 0x7f8a9811a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:39:19.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:19.664+0000 7f8a9e800700 1 -- 172.21.15.37:0/3992485311 >> 172.21.15.37:0/3992485311 conn(0x7f8a980f89b0 msgr2=0x7f8a980f9b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:39:19.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:19.664+0000 7f8a9e800700 1 -- 172.21.15.37:0/3992485311 shutdown_connections 2024-03-29T08:39:19.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:19.664+0000 7f8a9e800700 1 -- 172.21.15.37:0/3992485311 wait complete. 2024-03-29T08:39:19.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:19.801+0000 7efe32e00700 1 -- 172.21.15.37:0/315485420 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efe2c102e00 msgr2=0x7efe2c1031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:39:19.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:19.801+0000 7efe32e00700 1 --2- 172.21.15.37:0/315485420 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efe2c102e00 0x7efe2c1031d0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7efe1c004940 tx=0x7efe1c009e30 comp rx=0 tx=0).stop 2024-03-29T08:39:19.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:19.801+0000 7efe32e00700 1 -- 172.21.15.37:0/315485420 shutdown_connections 2024-03-29T08:39:19.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:19.801+0000 7efe32e00700 1 --2- 172.21.15.37:0/315485420 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe2c1037a0 0x7efe2c0fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:39:19.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:19.801+0000 7efe32e00700 1 --2- 172.21.15.37:0/315485420 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efe2c102e00 0x7efe2c1031d0 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:39:19.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:19.801+0000 7efe32e00700 1 -- 172.21.15.37:0/315485420 >> 172.21.15.37:0/315485420 conn(0x7efe2c0f89b0 msgr2=0x7efe2c0fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:39:19.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:19.801+0000 7efe32e00700 1 -- 172.21.15.37:0/315485420 shutdown_connections 2024-03-29T08:39:19.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:19.801+0000 7efe32e00700 1 -- 172.21.15.37:0/315485420 wait complete. 2024-03-29T08:39:19.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:19.802+0000 7efe32e00700 1 Processor -- start 2024-03-29T08:39:19.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:19.802+0000 7efe32e00700 1 -- start start 2024-03-29T08:39:19.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:19.803+0000 7efe32e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efe2c102e00 0x7efe2c10af40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:39:19.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:19.803+0000 7efe32e00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe2c1037a0 0x7efe2c10b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:39:19.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:19.803+0000 7efe32e00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe2c10f0d0 con 0x7efe2c102e00 2024-03-29T08:39:19.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:19.803+0000 7efe32e00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe2c10b9c0 con 0x7efe2c1037a0 2024-03-29T08:39:19.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:19.803+0000 7efe2be00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efe2c102e00 0x7efe2c10af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:39:19.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:19.803+0000 7efe2be00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efe2c102e00 0x7efe2c10af40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55044/0 (socket says 172.21.15.37:55044) 2024-03-29T08:39:19.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:19.803+0000 7efe2be00700 1 -- 172.21.15.37:0/3969539492 learned_addr learned my addr 172.21.15.37:0/3969539492 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:39:19.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:19.803+0000 7efe2b400700 1 --2- 172.21.15.37:0/3969539492 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe2c1037a0 0x7efe2c10b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:39:19.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:19.803+0000 7efe2be00700 1 -- 172.21.15.37:0/3969539492 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe2c1037a0 msgr2=0x7efe2c10b480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:39:19.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:19.803+0000 7efe2be00700 1 --2- 172.21.15.37:0/3969539492 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe2c1037a0 0x7efe2c10b480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:39:19.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:19.803+0000 7efe2be00700 1 -- 172.21.15.37:0/3969539492 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe1c005040 con 0x7efe2c102e00 2024-03-29T08:39:19.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:19.804+0000 7efe2be00700 1 --2- 172.21.15.37:0/3969539492 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efe2c102e00 0x7efe2c10af40 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7efe1c009360 tx=0x7efe1c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:39:19.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:19.804+0000 7efe23e00700 1 -- 172.21.15.37:0/3969539492 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efe1c00e040 con 0x7efe2c102e00 2024-03-29T08:39:19.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:19.804+0000 7efe23e00700 1 -- 172.21.15.37:0/3969539492 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7efe1c013420 con 0x7efe2c102e00 2024-03-29T08:39:19.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:19.804+0000 7efe32e00700 1 -- 172.21.15.37:0/3969539492 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe2c10bc40 con 0x7efe2c102e00 2024-03-29T08:39:19.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:19.804+0000 7efe23e00700 1 -- 172.21.15.37:0/3969539492 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efe1c01c5f0 con 0x7efe2c102e00 2024-03-29T08:39:19.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:19.804+0000 7efe32e00700 1 -- 172.21.15.37:0/3969539492 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe2c11e640 con 0x7efe2c102e00 2024-03-29T08:39:19.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:19.805+0000 7efe32e00700 1 -- 172.21.15.37:0/3969539492 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe2c0fe550 con 0x7efe2c102e00 2024-03-29T08:39:19.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:19.806+0000 7efe23e00700 1 -- 172.21.15.37:0/3969539492 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7efe1c013590 con 0x7efe2c102e00 2024-03-29T08:39:19.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:39:19 smithi148 ceph-mon[106692]: from='client.35546 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:39:19.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:39:19 smithi148 ceph-mon[106692]: from='client.35550 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:39:19.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:39:19 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/2178621741' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:39:19.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:39:19 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/2499190816' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T08:39:19.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:19.807+0000 7efe23e00700 1 --2- 172.21.15.37:0/3969539492 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7efe1006cc30 0x7efe1006f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:39:19.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:19.807+0000 7efe23e00700 1 -- 172.21.15.37:0/3969539492 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7efe1c08d270 con 0x7efe2c102e00 2024-03-29T08:39:19.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:19.807+0000 7efe2b400700 1 --2- 172.21.15.37:0/3969539492 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7efe1006cc30 0x7efe1006f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:39:19.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:19.808+0000 7efe2b400700 1 --2- 172.21.15.37:0/3969539492 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7efe1006cc30 0x7efe1006f0e0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7efe2c10c880 tx=0x7efe18009ee0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:39:19.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:19.812+0000 7efe23e00700 1 -- 172.21.15.37:0/3969539492 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7efe1c0572b0 con 0x7efe2c102e00 2024-03-29T08:39:19.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:39:19 smithi037 ceph-mon[142325]: from='client.35546 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:39:19.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:39:19 smithi037 ceph-mon[142325]: from='client.35550 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:39:19.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:39:19 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/2178621741' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:39:19.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:39:19 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/2499190816' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T08:39:20.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:20.109+0000 7efe32e00700 1 -- 172.21.15.37:0/3969539492 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7efe2c10c630 con 0x7efe2c102e00 2024-03-29T08:39:20.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:20.109+0000 7efe23e00700 1 -- 172.21.15.37:0/3969539492 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+3583 (secure 0 0 0) 0x7efe1c05a8d0 con 0x7efe2c102e00 2024-03-29T08:39:20.109 INFO:teuthology.orchestra.run.smithi037.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem is offline 2024-03-29T08:39:20.109 INFO:teuthology.orchestra.run.smithi037.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-03-29T08:39:20.109 INFO:teuthology.orchestra.run.smithi037.stdout: host smithi148 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/mon.smithi148/config 2024-03-29T08:39:20.109 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/podman: timeout after 895 seconds 2024-03-29T08:39:20.109 INFO:teuthology.orchestra.run.smithi037.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59 -e NODE_NAME=smithi148 -e CEPH_USE_RANDOM_NONCE=1 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4:/var/run/ceph:z -v /var/log/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4:/var/log/ceph:z -v /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpkx2rz2d4:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59 inventory --format=json-pretty --filter-for-batch 2024-03-29T08:39:20.110 INFO:teuthology.orchestra.run.smithi037.stdout:Traceback (most recent call last): 2024-03-29T08:39:20.110 INFO:teuthology.orchestra.run.smithi037.stdout: File "/usr/lib64/python3.6/runpy.py", line 193, in _run_module_as_main 2024-03-29T08:39:20.110 INFO:teuthology.orchestra.run.smithi037.stdout: "__main__", mod_spec) 2024-03-29T08:39:20.110 INFO:teuthology.orchestra.run.smithi037.stdout: File "/usr/lib64/python3.6/runpy.py", line 85, in _run_code 2024-03-29T08:39:20.110 INFO:teuthology.orchestra.run.smithi037.stdout: exec(code, run_globals) 2024-03-29T08:39:20.110 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 10889, in 2024-03-29T08:39:20.110 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 10877, in main 2024-03-29T08:39:20.110 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 2576, in _infer_config 2024-03-29T08:39:20.110 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 2492, in _infer_fsid 2024-03-29T08:39:20.110 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 2604, in _infer_image 2024-03-29T08:39:20.110 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 2479, in _validate_fsid 2024-03-29T08:39:20.110 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 7145, in command_ceph_volume 2024-03-29T08:39:20.110 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 2267, in call_throws 2024-03-29T08:39:20.110 INFO:teuthology.orchestra.run.smithi037.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59 -e NODE_NAME=smithi148 -e CEPH_USE_RANDOM_NONCE=1 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4:/var/run/ceph:z -v /var/log/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4:/var/log/ceph:z -v /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpkx2rz2d4:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59 inventory --format=json-pretty --filter-for-batch 2024-03-29T08:39:20.111 INFO:teuthology.orchestra.run.smithi037.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-03-29T08:39:20.111 INFO:teuthology.orchestra.run.smithi037.stdout: fs cephfs is degraded 2024-03-29T08:39:20.111 INFO:teuthology.orchestra.run.smithi037.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-03-29T08:39:20.111 INFO:teuthology.orchestra.run.smithi037.stdout: fs cephfs is offline because no MDS is active for it. 2024-03-29T08:39:20.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:20.111+0000 7efe32e00700 1 -- 172.21.15.37:0/3969539492 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7efe1006cc30 msgr2=0x7efe1006f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:39:20.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:20.111+0000 7efe32e00700 1 --2- 172.21.15.37:0/3969539492 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7efe1006cc30 0x7efe1006f0e0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7efe2c10c880 tx=0x7efe18009ee0 comp rx=0 tx=0).stop 2024-03-29T08:39:20.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:20.111+0000 7efe32e00700 1 -- 172.21.15.37:0/3969539492 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efe2c102e00 msgr2=0x7efe2c10af40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:39:20.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:20.111+0000 7efe32e00700 1 --2- 172.21.15.37:0/3969539492 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efe2c102e00 0x7efe2c10af40 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7efe1c009360 tx=0x7efe1c009e30 comp rx=0 tx=0).stop 2024-03-29T08:39:20.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:20.112+0000 7efe32e00700 1 -- 172.21.15.37:0/3969539492 shutdown_connections 2024-03-29T08:39:20.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:20.112+0000 7efe32e00700 1 --2- 172.21.15.37:0/3969539492 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7efe1006cc30 0x7efe1006f0e0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:39:20.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:20.112+0000 7efe32e00700 1 --2- 172.21.15.37:0/3969539492 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efe2c102e00 0x7efe2c10af40 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:39:20.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:20.112+0000 7efe32e00700 1 --2- 172.21.15.37:0/3969539492 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe2c1037a0 0x7efe2c10b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:39:20.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:20.112+0000 7efe32e00700 1 -- 172.21.15.37:0/3969539492 >> 172.21.15.37:0/3969539492 conn(0x7efe2c0f89b0 msgr2=0x7efe2c0f99e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:39:20.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:20.112+0000 7efe32e00700 1 -- 172.21.15.37:0/3969539492 shutdown_connections 2024-03-29T08:39:20.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:20.112+0000 7efe32e00700 1 -- 172.21.15.37:0/3969539492 wait complete. 2024-03-29T08:39:20.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:39:20 smithi148 ceph-mon[106692]: pgmap v854: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:39:20.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:39:20 smithi148 ceph-mon[106692]: from='client.35562 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:39:20.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:39:20 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/3969539492' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T08:39:20.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:39:20 smithi037 ceph-mon[142325]: pgmap v854: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:39:20.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:39:20 smithi037 ceph-mon[142325]: from='client.35562 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:39:20.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:39:20 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/3969539492' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T08:39:22.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:39:22 smithi148 ceph-mon[106692]: pgmap v855: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:39:22.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:39:22 smithi037 ceph-mon[142325]: pgmap v855: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:39:24.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:39:24 smithi148 ceph-mon[106692]: pgmap v856: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:39:24.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:39:24 smithi037 ceph-mon[142325]: pgmap v856: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:39:25.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:39:25 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:39:25.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:39:25 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:39:26.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:39:26 smithi148 ceph-mon[106692]: pgmap v857: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:39:26.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:39:26 smithi037 ceph-mon[142325]: pgmap v857: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:39:28.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:39:28 smithi148 ceph-mon[106692]: pgmap v858: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:39:28.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:39:28 smithi037 ceph-mon[142325]: pgmap v858: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:39:30.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:39:30 smithi148 ceph-mon[106692]: pgmap v859: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:39:30.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:39:30 smithi037 ceph-mon[142325]: pgmap v859: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:39:32.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:39:32 smithi148 ceph-mon[106692]: pgmap v860: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:39:32.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:39:32 smithi037 ceph-mon[142325]: pgmap v860: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:39:34.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:39:34 smithi148 ceph-mon[106692]: pgmap v861: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:39:34.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:39:34 smithi037 ceph-mon[142325]: pgmap v861: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:39:36.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:39:36 smithi148 ceph-mon[106692]: pgmap v862: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:39:36.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:39:36 smithi037 ceph-mon[142325]: pgmap v862: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:39:38.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:39:38 smithi148 ceph-mon[106692]: pgmap v863: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:39:38.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:39:38 smithi037 ceph-mon[142325]: pgmap v863: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:39:40.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:39:40 smithi148 ceph-mon[106692]: pgmap v864: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:39:40.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:39:40 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:39:40.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:39:40 smithi037 ceph-mon[142325]: pgmap v864: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:39:40.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:39:40 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:39:42.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:39:42 smithi148 ceph-mon[106692]: pgmap v865: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:39:42.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:39:42 smithi037 ceph-mon[142325]: pgmap v865: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:39:44.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:39:44 smithi148 ceph-mon[106692]: pgmap v866: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:39:44.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:39:44 smithi037 ceph-mon[142325]: pgmap v866: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:39:46.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:39:46 smithi037 ceph-mon[142325]: pgmap v867: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:39:47.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:39:46 smithi148 ceph-mon[106692]: pgmap v867: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:39:48.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:39:48 smithi037 ceph-mon[142325]: pgmap v868: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:39:49.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:39:48 smithi148 ceph-mon[106692]: pgmap v868: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:39:50.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:50.266+0000 7f20e4600700 1 -- 172.21.15.37:0/2449066105 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f20dc1037a0 msgr2=0x7f20dc0fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:39:50.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:50.266+0000 7f20e4600700 1 --2- 172.21.15.37:0/2449066105 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f20dc1037a0 0x7f20dc0fce10 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f20c8004940 tx=0x7f20c8009e30 comp rx=0 tx=0).stop 2024-03-29T08:39:50.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:50.266+0000 7f20e4600700 1 -- 172.21.15.37:0/2449066105 shutdown_connections 2024-03-29T08:39:50.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:50.266+0000 7f20e4600700 1 --2- 172.21.15.37:0/2449066105 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f20dc1037a0 0x7f20dc0fce10 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:39:50.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:50.266+0000 7f20e4600700 1 --2- 172.21.15.37:0/2449066105 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f20dc102e00 0x7f20dc1031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:39:50.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:50.266+0000 7f20e4600700 1 -- 172.21.15.37:0/2449066105 >> 172.21.15.37:0/2449066105 conn(0x7f20dc0f89b0 msgr2=0x7f20dc0fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:39:50.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:50.266+0000 7f20e4600700 1 -- 172.21.15.37:0/2449066105 shutdown_connections 2024-03-29T08:39:50.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:50.266+0000 7f20e4600700 1 -- 172.21.15.37:0/2449066105 wait complete. 2024-03-29T08:39:50.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:50.267+0000 7f20e4600700 1 Processor -- start 2024-03-29T08:39:50.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:50.267+0000 7f20e4600700 1 -- start start 2024-03-29T08:39:50.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:50.268+0000 7f20e4600700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f20dc102e00 0x7f20dc113b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:39:50.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:50.268+0000 7f20e4600700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f20dc1037a0 0x7f20dc114040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:39:50.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:50.268+0000 7f20e4600700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20dc114690 con 0x7f20dc102e00 2024-03-29T08:39:50.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:50.268+0000 7f20e4600700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20dc10da80 con 0x7f20dc1037a0 2024-03-29T08:39:50.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:50.268+0000 7f20e1c00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f20dc102e00 0x7f20dc113b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:39:50.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:50.268+0000 7f20e1c00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f20dc102e00 0x7f20dc113b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44348/0 (socket says 172.21.15.37:44348) 2024-03-29T08:39:50.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:50.268+0000 7f20e1c00700 1 -- 172.21.15.37:0/2268221241 learned_addr learned my addr 172.21.15.37:0/2268221241 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:39:50.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:50.268+0000 7f20e1c00700 1 -- 172.21.15.37:0/2268221241 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f20dc1037a0 msgr2=0x7f20dc114040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:39:50.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:50.268+0000 7f20e1200700 1 --2- 172.21.15.37:0/2268221241 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f20dc1037a0 0x7f20dc114040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:39:50.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:50.268+0000 7f20e1c00700 1 --2- 172.21.15.37:0/2268221241 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f20dc1037a0 0x7f20dc114040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:39:50.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:50.268+0000 7f20e1c00700 1 -- 172.21.15.37:0/2268221241 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20c8005040 con 0x7f20dc102e00 2024-03-29T08:39:50.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:50.268+0000 7f20e1200700 1 --2- 172.21.15.37:0/2268221241 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f20dc1037a0 0x7f20dc114040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:39:50.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:50.269+0000 7f20e1c00700 1 --2- 172.21.15.37:0/2268221241 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f20dc102e00 0x7f20dc113b00 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f20d000c8c0 tx=0x7f20d00044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:39:50.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:50.269+0000 7f20d9600700 1 -- 172.21.15.37:0/2268221241 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f20d000a980 con 0x7f20dc102e00 2024-03-29T08:39:50.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:50.269+0000 7f20d9600700 1 -- 172.21.15.37:0/2268221241 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f20d000ce60 con 0x7f20dc102e00 2024-03-29T08:39:50.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:50.269+0000 7f20d9600700 1 -- 172.21.15.37:0/2268221241 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f20d00113a0 con 0x7f20dc102e00 2024-03-29T08:39:50.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:50.269+0000 7f20e4600700 1 -- 172.21.15.37:0/2268221241 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20dc10dd60 con 0x7f20dc102e00 2024-03-29T08:39:50.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:50.269+0000 7f20e4600700 1 -- 172.21.15.37:0/2268221241 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20dc10e320 con 0x7f20dc102e00 2024-03-29T08:39:50.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:50.270+0000 7f20e4600700 1 -- 172.21.15.37:0/2268221241 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20dc0fe550 con 0x7f20dc102e00 2024-03-29T08:39:50.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:50.271+0000 7f20d9600700 1 -- 172.21.15.37:0/2268221241 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f20d0012400 con 0x7f20dc102e00 2024-03-29T08:39:50.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:50.273+0000 7f20d9600700 1 --2- 172.21.15.37:0/2268221241 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f20c406cc30 0x7f20c406f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:39:50.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:50.273+0000 7f20d9600700 1 -- 172.21.15.37:0/2268221241 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f20d008c0f0 con 0x7f20dc102e00 2024-03-29T08:39:50.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:50.273+0000 7f20e1200700 1 --2- 172.21.15.37:0/2268221241 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f20c406cc30 0x7f20c406f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:39:50.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:50.274+0000 7f20e1200700 1 --2- 172.21.15.37:0/2268221241 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f20c406cc30 0x7f20c406f0e0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f20c8009fd0 tx=0x7f20c8009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:39:50.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:50.278+0000 7f20d9600700 1 -- 172.21.15.37:0/2268221241 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f20d0059bc0 con 0x7f20dc102e00 2024-03-29T08:39:50.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:50.511+0000 7f20e4600700 1 -- 172.21.15.37:0/2268221241 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f20dc109da0 con 0x7f20c406cc30 2024-03-29T08:39:50.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:50.512+0000 7f20d9600700 1 -- 172.21.15.37:0/2268221241 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f20dc109da0 con 0x7f20c406cc30 2024-03-29T08:39:50.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:50.515+0000 7f20e4600700 1 -- 172.21.15.37:0/2268221241 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f20c406cc30 msgr2=0x7f20c406f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:39:50.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:50.515+0000 7f20e4600700 1 --2- 172.21.15.37:0/2268221241 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f20c406cc30 0x7f20c406f0e0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f20c8009fd0 tx=0x7f20c8009e30 comp rx=0 tx=0).stop 2024-03-29T08:39:50.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:50.515+0000 7f20e4600700 1 -- 172.21.15.37:0/2268221241 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f20dc102e00 msgr2=0x7f20dc113b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:39:50.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:50.515+0000 7f20e4600700 1 --2- 172.21.15.37:0/2268221241 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f20dc102e00 0x7f20dc113b00 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f20d000c8c0 tx=0x7f20d00044c0 comp rx=0 tx=0).stop 2024-03-29T08:39:50.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:50.515+0000 7f20e4600700 1 -- 172.21.15.37:0/2268221241 shutdown_connections 2024-03-29T08:39:50.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:50.515+0000 7f20e4600700 1 --2- 172.21.15.37:0/2268221241 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f20c406cc30 0x7f20c406f0e0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:39:50.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:50.515+0000 7f20e4600700 1 --2- 172.21.15.37:0/2268221241 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f20dc102e00 0x7f20dc113b00 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:39:50.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:50.515+0000 7f20e4600700 1 --2- 172.21.15.37:0/2268221241 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f20dc1037a0 0x7f20dc114040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:39:50.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:50.515+0000 7f20e4600700 1 -- 172.21.15.37:0/2268221241 >> 172.21.15.37:0/2268221241 conn(0x7f20dc0f89b0 msgr2=0x7f20dc0f9ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:39:50.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:50.515+0000 7f20e4600700 1 -- 172.21.15.37:0/2268221241 shutdown_connections 2024-03-29T08:39:50.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:50.515+0000 7f20e4600700 1 -- 172.21.15.37:0/2268221241 wait complete. 2024-03-29T08:39:50.530 INFO:teuthology.orchestra.run.smithi037.stdout:true 2024-03-29T08:39:50.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:50.654+0000 7eff1a600700 1 -- 172.21.15.37:0/2745941420 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7eff14070360 msgr2=0x7eff14070730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:39:50.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:50.654+0000 7eff1a600700 1 --2- 172.21.15.37:0/2745941420 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7eff14070360 0x7eff14070730 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7eff04004970 tx=0x7eff04009e30 comp rx=0 tx=0).stop 2024-03-29T08:39:50.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:50.654+0000 7eff1a600700 1 -- 172.21.15.37:0/2745941420 shutdown_connections 2024-03-29T08:39:50.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:50.654+0000 7eff1a600700 1 --2- 172.21.15.37:0/2745941420 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7eff14070c70 0x7eff14109110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:39:50.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:50.654+0000 7eff1a600700 1 --2- 172.21.15.37:0/2745941420 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7eff14070360 0x7eff14070730 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:39:50.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:50.654+0000 7eff1a600700 1 -- 172.21.15.37:0/2745941420 >> 172.21.15.37:0/2745941420 conn(0x7eff140f89b0 msgr2=0x7eff140fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:39:50.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:50.654+0000 7eff1a600700 1 -- 172.21.15.37:0/2745941420 shutdown_connections 2024-03-29T08:39:50.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:50.654+0000 7eff1a600700 1 -- 172.21.15.37:0/2745941420 wait complete. 2024-03-29T08:39:50.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:50.655+0000 7eff1a600700 1 Processor -- start 2024-03-29T08:39:50.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:50.655+0000 7eff1a600700 1 -- start start 2024-03-29T08:39:50.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:50.656+0000 7eff1a600700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7eff14070360 0x7eff14115c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:39:50.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:50.656+0000 7eff1a600700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7eff14070c70 0x7eff141161a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:39:50.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:50.656+0000 7eff1a600700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff141167f0 con 0x7eff14070360 2024-03-29T08:39:50.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:50.656+0000 7eff1a600700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff1410fc00 con 0x7eff14070c70 2024-03-29T08:39:50.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:50.656+0000 7eff13400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7eff14070360 0x7eff14115c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:39:50.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:50.656+0000 7eff13400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7eff14070360 0x7eff14115c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44368/0 (socket says 172.21.15.37:44368) 2024-03-29T08:39:50.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:50.656+0000 7eff13400700 1 -- 172.21.15.37:0/2359117745 learned_addr learned my addr 172.21.15.37:0/2359117745 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:39:50.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:50.657+0000 7eff12a00700 1 --2- 172.21.15.37:0/2359117745 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7eff14070c70 0x7eff141161a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:39:50.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:50.657+0000 7eff13400700 1 -- 172.21.15.37:0/2359117745 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7eff14070c70 msgr2=0x7eff141161a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:39:50.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:50.657+0000 7eff13400700 1 --2- 172.21.15.37:0/2359117745 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7eff14070c70 0x7eff141161a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:39:50.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:50.657+0000 7eff13400700 1 -- 172.21.15.37:0/2359117745 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff04005040 con 0x7eff14070360 2024-03-29T08:39:50.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:50.657+0000 7eff13400700 1 --2- 172.21.15.37:0/2359117745 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7eff14070360 0x7eff14115c60 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7eff04005010 tx=0x7eff04009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:39:50.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:50.657+0000 7eff0b400700 1 -- 172.21.15.37:0/2359117745 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7eff0400e040 con 0x7eff14070360 2024-03-29T08:39:50.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:50.657+0000 7eff1a600700 1 -- 172.21.15.37:0/2359117745 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff1410fe80 con 0x7eff14070360 2024-03-29T08:39:50.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:50.657+0000 7eff0b400700 1 -- 172.21.15.37:0/2359117745 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7eff04004290 con 0x7eff14070360 2024-03-29T08:39:50.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:50.657+0000 7eff1a600700 1 -- 172.21.15.37:0/2359117745 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff141103c0 con 0x7eff14070360 2024-03-29T08:39:50.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:50.658+0000 7eff0b400700 1 -- 172.21.15.37:0/2359117745 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7eff04013e30 con 0x7eff14070360 2024-03-29T08:39:50.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:50.658+0000 7eff1a600700 1 -- 172.21.15.37:0/2359117745 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff14071aa0 con 0x7eff14070360 2024-03-29T08:39:50.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:50.659+0000 7eff0b400700 1 -- 172.21.15.37:0/2359117745 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7eff04013620 con 0x7eff14070360 2024-03-29T08:39:50.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:50.660+0000 7eff0b400700 1 --2- 172.21.15.37:0/2359117745 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7efef806cc30 0x7efef806f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:39:50.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:50.660+0000 7eff0b400700 1 -- 172.21.15.37:0/2359117745 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7eff0408d370 con 0x7eff14070360 2024-03-29T08:39:50.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:50.661+0000 7eff12a00700 1 --2- 172.21.15.37:0/2359117745 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7efef806cc30 0x7efef806f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:39:50.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:50.661+0000 7eff12a00700 1 --2- 172.21.15.37:0/2359117745 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7efef806cc30 0x7efef806f0e0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7eff00004600 tx=0x7eff0000b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:39:50.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:50.665+0000 7eff0b400700 1 -- 172.21.15.37:0/2359117745 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7eff04057330 con 0x7eff14070360 2024-03-29T08:39:50.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:50.897+0000 7eff1a600700 1 -- 172.21.15.37:0/2359117745 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7eff1410c070 con 0x7efef806cc30 2024-03-29T08:39:50.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:50.899+0000 7eff0b400700 1 -- 172.21.15.37:0/2359117745 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7eff0000f040 con 0x7efef806cc30 2024-03-29T08:39:50.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:50.901+0000 7eff1a600700 1 -- 172.21.15.37:0/2359117745 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7efef806cc30 msgr2=0x7efef806f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:39:50.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:50.901+0000 7eff1a600700 1 --2- 172.21.15.37:0/2359117745 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7efef806cc30 0x7efef806f0e0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7eff00004600 tx=0x7eff0000b040 comp rx=0 tx=0).stop 2024-03-29T08:39:50.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:50.901+0000 7eff1a600700 1 -- 172.21.15.37:0/2359117745 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7eff14070360 msgr2=0x7eff14115c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:39:50.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:50.901+0000 7eff1a600700 1 --2- 172.21.15.37:0/2359117745 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7eff14070360 0x7eff14115c60 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7eff04005010 tx=0x7eff04009e30 comp rx=0 tx=0).stop 2024-03-29T08:39:50.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:50.901+0000 7eff1a600700 1 -- 172.21.15.37:0/2359117745 shutdown_connections 2024-03-29T08:39:50.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:50.901+0000 7eff1a600700 1 --2- 172.21.15.37:0/2359117745 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7efef806cc30 0x7efef806f0e0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:39:50.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:50.901+0000 7eff1a600700 1 --2- 172.21.15.37:0/2359117745 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7eff14070360 0x7eff14115c60 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:39:50.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:50.901+0000 7eff1a600700 1 --2- 172.21.15.37:0/2359117745 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7eff14070c70 0x7eff141161a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:39:50.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:50.901+0000 7eff1a600700 1 -- 172.21.15.37:0/2359117745 >> 172.21.15.37:0/2359117745 conn(0x7eff140f89b0 msgr2=0x7eff140f9f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:39:50.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:50.901+0000 7eff1a600700 1 -- 172.21.15.37:0/2359117745 shutdown_connections 2024-03-29T08:39:50.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:50.901+0000 7eff1a600700 1 -- 172.21.15.37:0/2359117745 wait complete. 2024-03-29T08:39:50.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:39:50 smithi037 ceph-mon[142325]: pgmap v869: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:39:51.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:51.038+0000 7f10cac00700 1 -- 172.21.15.37:0/1090771563 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f10c4102c40 msgr2=0x7f10c4103010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:39:51.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:51.038+0000 7f10cac00700 1 --2- 172.21.15.37:0/1090771563 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f10c4102c40 0x7f10c4103010 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f10b8004940 tx=0x7f10b8009e30 comp rx=0 tx=0).stop 2024-03-29T08:39:51.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:51.038+0000 7f10cac00700 1 -- 172.21.15.37:0/1090771563 shutdown_connections 2024-03-29T08:39:51.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:51.038+0000 7f10cac00700 1 --2- 172.21.15.37:0/1090771563 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f10c41035e0 0x7f10c40fcc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:39:51.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:51.038+0000 7f10cac00700 1 --2- 172.21.15.37:0/1090771563 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f10c4102c40 0x7f10c4103010 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:39:51.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:51.038+0000 7f10cac00700 1 -- 172.21.15.37:0/1090771563 >> 172.21.15.37:0/1090771563 conn(0x7f10c40f8830 msgr2=0x7f10c40fac40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:39:51.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:51.038+0000 7f10cac00700 1 -- 172.21.15.37:0/1090771563 shutdown_connections 2024-03-29T08:39:51.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:51.038+0000 7f10cac00700 1 -- 172.21.15.37:0/1090771563 wait complete. 2024-03-29T08:39:51.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:51.040+0000 7f10cac00700 1 Processor -- start 2024-03-29T08:39:51.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:51.040+0000 7f10cac00700 1 -- start start 2024-03-29T08:39:51.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:51.040+0000 7f10cac00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f10c4102c40 0x7f10c40702c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:39:51.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:51.040+0000 7f10cac00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f10c41035e0 0x7f10c4070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:39:51.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:51.040+0000 7f10cac00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10c4070d90 con 0x7f10c41035e0 2024-03-29T08:39:51.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:51.040+0000 7f10cac00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10c4070ed0 con 0x7f10c4102c40 2024-03-29T08:39:51.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:51.041+0000 7f10c8e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f10c41035e0 0x7f10c4070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:39:51.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:51.041+0000 7f10c8e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f10c41035e0 0x7f10c4070800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44382/0 (socket says 172.21.15.37:44382) 2024-03-29T08:39:51.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:51.041+0000 7f10c8e00700 1 -- 172.21.15.37:0/1649617601 learned_addr learned my addr 172.21.15.37:0/1649617601 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:39:51.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:51.041+0000 7f10c9800700 1 --2- 172.21.15.37:0/1649617601 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f10c4102c40 0x7f10c40702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:39:51.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:51.041+0000 7f10c8e00700 1 -- 172.21.15.37:0/1649617601 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f10c4102c40 msgr2=0x7f10c40702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:39:51.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:51.041+0000 7f10c8e00700 1 --2- 172.21.15.37:0/1649617601 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f10c4102c40 0x7f10c40702c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:39:51.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:51.041+0000 7f10c8e00700 1 -- 172.21.15.37:0/1649617601 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10b8005040 con 0x7f10c41035e0 2024-03-29T08:39:51.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:51.041+0000 7f10c9800700 1 --2- 172.21.15.37:0/1649617601 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f10c4102c40 0x7f10c40702c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:39:51.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:51.041+0000 7f10c8e00700 1 --2- 172.21.15.37:0/1649617601 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f10c41035e0 0x7f10c4070800 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f10b0002a80 tx=0x7f10b0004630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:39:51.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:51.042+0000 7f10c1600700 1 -- 172.21.15.37:0/1649617601 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f10b0007cc0 con 0x7f10c41035e0 2024-03-29T08:39:51.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:51.042+0000 7f10cac00700 1 -- 172.21.15.37:0/1649617601 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10c4119c70 con 0x7f10c41035e0 2024-03-29T08:39:51.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:51.042+0000 7f10c1600700 1 -- 172.21.15.37:0/1649617601 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f10b000f040 con 0x7f10c41035e0 2024-03-29T08:39:51.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:51.042+0000 7f10c1600700 1 -- 172.21.15.37:0/1649617601 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f10b000da40 con 0x7f10c41035e0 2024-03-29T08:39:51.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:51.042+0000 7f10cac00700 1 -- 172.21.15.37:0/1649617601 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10c411a0f0 con 0x7f10c41035e0 2024-03-29T08:39:51.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:51.043+0000 7f10cac00700 1 -- 172.21.15.37:0/1649617601 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10c40fe390 con 0x7f10c41035e0 2024-03-29T08:39:51.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:51.044+0000 7f10c1600700 1 -- 172.21.15.37:0/1649617601 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f10b000dbe0 con 0x7f10c41035e0 2024-03-29T08:39:51.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:51.045+0000 7f10c1600700 1 --2- 172.21.15.37:0/1649617601 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f10ac06cc30 0x7f10ac06f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:39:51.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:51.045+0000 7f10c1600700 1 -- 172.21.15.37:0/1649617601 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f10b008b7c0 con 0x7f10c41035e0 2024-03-29T08:39:51.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:51.045+0000 7f10c9800700 1 --2- 172.21.15.37:0/1649617601 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f10ac06cc30 0x7f10ac06f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:39:51.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:51.046+0000 7f10c9800700 1 --2- 172.21.15.37:0/1649617601 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f10ac06cc30 0x7f10ac06f0e0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f10b800b840 tx=0x7f10b8009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:39:51.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:51.049+0000 7f10c1600700 1 -- 172.21.15.37:0/1649617601 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f10b0055700 con 0x7f10c41035e0 2024-03-29T08:39:51.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:39:50 smithi148 ceph-mon[106692]: pgmap v869: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:39:51.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:51.272+0000 7f10cac00700 1 -- 172.21.15.37:0/1649617601 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f10c404bed0 con 0x7f10ac06cc30 2024-03-29T08:39:51.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:51.282+0000 7f10c1600700 1 -- 172.21.15.37:0/1649617601 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f10c404bed0 con 0x7f10ac06cc30 2024-03-29T08:39:51.282 INFO:teuthology.orchestra.run.smithi037.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-29T08:39:51.282 INFO:teuthology.orchestra.run.smithi037.stdout:alertmanager.smithi037 smithi037 *:9093,9094 running (29m) 6m ago 42m 14.3M - 0.25.0 c8568f914cd2 2f9f9746ed0f 2024-03-29T08:39:51.282 INFO:teuthology.orchestra.run.smithi037.stdout:crash.smithi037 smithi037 running (26m) 6m ago 42m 6647k - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 178d416a3186 2024-03-29T08:39:51.282 INFO:teuthology.orchestra.run.smithi037.stdout:crash.smithi148 smithi148 running (26m) 6m ago 41m 6656k - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 23666989d58f 2024-03-29T08:39:51.282 INFO:teuthology.orchestra.run.smithi037.stdout:grafana.smithi037 smithi037 *:3000 running (29m) 6m ago 42m 74.5M - 9.4.7 954c08fa6188 6b976fe49952 2024-03-29T08:39:51.282 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi037.iifsmh smithi037 running (22m) 6m ago 36m 16.4M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 2d97f562cc5b 2024-03-29T08:39:51.282 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi037.mmwgxe smithi037 running (22m) 6m ago 36m 16.7M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff ec37037f46e8 2024-03-29T08:39:51.282 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi148.qukpjh smithi148 running (6m) 6m ago 36m 10.7M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 3257219bc046 2024-03-29T08:39:51.282 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi148.sqteih smithi148 running (21m) 6m ago 36m 16.3M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 44c15c77037a 2024-03-29T08:39:51.282 INFO:teuthology.orchestra.run.smithi037.stdout:mgr.smithi037.wjikii smithi037 *:8443,9283,8765 running (32m) 6m ago 44m 510M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff c5a1417b9417 2024-03-29T08:39:51.283 INFO:teuthology.orchestra.run.smithi037.stdout:mgr.smithi148.onhgvm smithi148 *:8443,9283,8765 running (30m) 6m ago 41m 456M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff fcfdbce062bd 2024-03-29T08:39:51.283 INFO:teuthology.orchestra.run.smithi037.stdout:mon.smithi037 smithi037 running (27m) 6m ago 44m 70.2M 2048M 18.2.2-1015-gd0e392d9 1cc2f45c17ff eea2060783af 2024-03-29T08:39:51.283 INFO:teuthology.orchestra.run.smithi037.stdout:mon.smithi148 smithi148 running (27m) 6m ago 41m 50.4M 2048M 18.2.2-1015-gd0e392d9 1cc2f45c17ff ad4b2c5748be 2024-03-29T08:39:51.283 INFO:teuthology.orchestra.run.smithi037.stdout:node-exporter.smithi037 smithi037 *:9100 running (30m) 6m ago 42m 8514k - 1.5.0 0da6a335fe13 26b8ae9cb878 2024-03-29T08:39:51.283 INFO:teuthology.orchestra.run.smithi037.stdout:node-exporter.smithi148 smithi148 *:9100 running (30m) 6m ago 41m 8464k - 1.5.0 0da6a335fe13 98414e4e64ed 2024-03-29T08:39:51.283 INFO:teuthology.orchestra.run.smithi037.stdout:osd.0 smithi037 running (26m) 6m ago 40m 108M 1619M 18.2.2-1015-gd0e392d9 1cc2f45c17ff c0c584f6245c 2024-03-29T08:39:51.283 INFO:teuthology.orchestra.run.smithi037.stdout:osd.1 smithi037 running (25m) 6m ago 40m 104M 1619M 18.2.2-1015-gd0e392d9 1cc2f45c17ff a4a2ff4cc83d 2024-03-29T08:39:51.283 INFO:teuthology.orchestra.run.smithi037.stdout:osd.2 smithi037 running (24m) 6m ago 39m 85.9M 1619M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 8b1bc71aa79e 2024-03-29T08:39:51.283 INFO:teuthology.orchestra.run.smithi037.stdout:osd.3 smithi148 running (24m) 6m ago 38m 109M 2642M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 1c656ab8ff86 2024-03-29T08:39:51.283 INFO:teuthology.orchestra.run.smithi037.stdout:osd.4 smithi148 running (23m) 6m ago 38m 101M 2642M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 966a9de11d22 2024-03-29T08:39:51.283 INFO:teuthology.orchestra.run.smithi037.stdout:osd.5 smithi148 running (22m) 6m ago 37m 89.4M 2642M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 474c4f9212ce 2024-03-29T08:39:51.283 INFO:teuthology.orchestra.run.smithi037.stdout:prometheus.smithi037 smithi037 *:9095 running (30m) 6m ago 42m 61.0M - 2.43.0 a07b618ecd1d 30c0223ec589 2024-03-29T08:39:51.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:51.284+0000 7f10cac00700 1 -- 172.21.15.37:0/1649617601 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f10ac06cc30 msgr2=0x7f10ac06f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:39:51.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:51.284+0000 7f10cac00700 1 --2- 172.21.15.37:0/1649617601 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f10ac06cc30 0x7f10ac06f0e0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f10b800b840 tx=0x7f10b8009e30 comp rx=0 tx=0).stop 2024-03-29T08:39:51.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:51.284+0000 7f10cac00700 1 -- 172.21.15.37:0/1649617601 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f10c41035e0 msgr2=0x7f10c4070800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:39:51.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:51.284+0000 7f10cac00700 1 --2- 172.21.15.37:0/1649617601 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f10c41035e0 0x7f10c4070800 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f10b0002a80 tx=0x7f10b0004630 comp rx=0 tx=0).stop 2024-03-29T08:39:51.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:51.284+0000 7f10cac00700 1 -- 172.21.15.37:0/1649617601 shutdown_connections 2024-03-29T08:39:51.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:51.284+0000 7f10cac00700 1 --2- 172.21.15.37:0/1649617601 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f10ac06cc30 0x7f10ac06f0e0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:39:51.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:51.284+0000 7f10cac00700 1 --2- 172.21.15.37:0/1649617601 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f10c4102c40 0x7f10c40702c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:39:51.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:51.284+0000 7f10cac00700 1 --2- 172.21.15.37:0/1649617601 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f10c41035e0 0x7f10c4070800 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:39:51.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:51.284+0000 7f10cac00700 1 -- 172.21.15.37:0/1649617601 >> 172.21.15.37:0/1649617601 conn(0x7f10c40f8830 msgr2=0x7f10c40f98d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:39:51.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:51.285+0000 7f10cac00700 1 -- 172.21.15.37:0/1649617601 shutdown_connections 2024-03-29T08:39:51.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:51.285+0000 7f10cac00700 1 -- 172.21.15.37:0/1649617601 wait complete. 2024-03-29T08:39:51.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:51.421+0000 7f54ef000700 1 -- 172.21.15.37:0/2020933283 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f54e8102e00 msgr2=0x7f54e81031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:39:51.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:51.421+0000 7f54ef000700 1 --2- 172.21.15.37:0/2020933283 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f54e8102e00 0x7f54e81031d0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f54d8004940 tx=0x7f54d8009e30 comp rx=0 tx=0).stop 2024-03-29T08:39:51.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:51.421+0000 7f54ef000700 1 -- 172.21.15.37:0/2020933283 shutdown_connections 2024-03-29T08:39:51.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:51.421+0000 7f54ef000700 1 --2- 172.21.15.37:0/2020933283 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54e81037a0 0x7f54e80fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:39:51.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:51.421+0000 7f54ef000700 1 --2- 172.21.15.37:0/2020933283 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f54e8102e00 0x7f54e81031d0 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:39:51.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:51.421+0000 7f54ef000700 1 -- 172.21.15.37:0/2020933283 >> 172.21.15.37:0/2020933283 conn(0x7f54e80f89b0 msgr2=0x7f54e80fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:39:51.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:51.421+0000 7f54ef000700 1 -- 172.21.15.37:0/2020933283 shutdown_connections 2024-03-29T08:39:51.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:51.421+0000 7f54ef000700 1 -- 172.21.15.37:0/2020933283 wait complete. 2024-03-29T08:39:51.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:51.422+0000 7f54ef000700 1 Processor -- start 2024-03-29T08:39:51.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:51.422+0000 7f54ef000700 1 -- start start 2024-03-29T08:39:51.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:51.423+0000 7f54ef000700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54e8102e00 0x7f54e8113b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:39:51.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:51.423+0000 7f54ef000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f54e81037a0 0x7f54e8114060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:39:51.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:51.423+0000 7f54ef000700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54e81146b0 con 0x7f54e81037a0 2024-03-29T08:39:51.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:51.423+0000 7f54ef000700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54e810daa0 con 0x7f54e8102e00 2024-03-29T08:39:51.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:51.423+0000 7f54e7400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f54e81037a0 0x7f54e8114060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:39:51.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:51.423+0000 7f54e7400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f54e81037a0 0x7f54e8114060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44398/0 (socket says 172.21.15.37:44398) 2024-03-29T08:39:51.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:51.423+0000 7f54e7400700 1 -- 172.21.15.37:0/1937565690 learned_addr learned my addr 172.21.15.37:0/1937565690 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:39:51.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:51.423+0000 7f54e7e00700 1 --2- 172.21.15.37:0/1937565690 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54e8102e00 0x7f54e8113b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:39:51.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:51.423+0000 7f54e7400700 1 -- 172.21.15.37:0/1937565690 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54e8102e00 msgr2=0x7f54e8113b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:39:51.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:51.423+0000 7f54e7400700 1 --2- 172.21.15.37:0/1937565690 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54e8102e00 0x7f54e8113b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:39:51.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:51.423+0000 7f54e7400700 1 -- 172.21.15.37:0/1937565690 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54d8005040 con 0x7f54e81037a0 2024-03-29T08:39:51.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:51.424+0000 7f54e7400700 1 --2- 172.21.15.37:0/1937565690 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f54e81037a0 0x7f54e8114060 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f54d4002a50 tx=0x7f54d4004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:39:51.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:51.424+0000 7f54dfe00700 1 -- 172.21.15.37:0/1937565690 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f54d4007cc0 con 0x7f54e81037a0 2024-03-29T08:39:51.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:51.424+0000 7f54dfe00700 1 -- 172.21.15.37:0/1937565690 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f54d400f040 con 0x7f54e81037a0 2024-03-29T08:39:51.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:51.424+0000 7f54ef000700 1 -- 172.21.15.37:0/1937565690 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54e810dd80 con 0x7f54e81037a0 2024-03-29T08:39:51.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:51.424+0000 7f54dfe00700 1 -- 172.21.15.37:0/1937565690 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f54d400d920 con 0x7f54e81037a0 2024-03-29T08:39:51.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:51.424+0000 7f54ef000700 1 -- 172.21.15.37:0/1937565690 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54e810e340 con 0x7f54e81037a0 2024-03-29T08:39:51.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:51.425+0000 7f54ef000700 1 -- 172.21.15.37:0/1937565690 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54e80fe550 con 0x7f54e81037a0 2024-03-29T08:39:51.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:51.427+0000 7f54dfe00700 1 -- 172.21.15.37:0/1937565690 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f54d4009680 con 0x7f54e81037a0 2024-03-29T08:39:51.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:51.427+0000 7f54dfe00700 1 --2- 172.21.15.37:0/1937565690 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f54cc06cc30 0x7f54cc06f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:39:51.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:51.428+0000 7f54dfe00700 1 -- 172.21.15.37:0/1937565690 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f54d408bd10 con 0x7f54e81037a0 2024-03-29T08:39:51.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:51.428+0000 7f54e7e00700 1 --2- 172.21.15.37:0/1937565690 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f54cc06cc30 0x7f54cc06f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:39:51.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:51.428+0000 7f54e7e00700 1 --2- 172.21.15.37:0/1937565690 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f54cc06cc30 0x7f54cc06f0e0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f54d8009fd0 tx=0x7f54d8009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:39:51.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:51.432+0000 7f54dfe00700 1 -- 172.21.15.37:0/1937565690 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f54d4059760 con 0x7f54e81037a0 2024-03-29T08:39:51.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:51.731+0000 7f54ef000700 1 -- 172.21.15.37:0/1937565690 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f54e8109dc0 con 0x7f54e81037a0 2024-03-29T08:39:51.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:51.731+0000 7f54dfe00700 1 -- 172.21.15.37:0/1937565690 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7f54d40592f0 con 0x7f54e81037a0 2024-03-29T08:39:51.731 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-29T08:39:51.731 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": { 2024-03-29T08:39:51.731 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 2 2024-03-29T08:39:51.731 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:39:51.731 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": { 2024-03-29T08:39:51.731 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 2 2024-03-29T08:39:51.732 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:39:51.732 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": { 2024-03-29T08:39:51.732 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 6 2024-03-29T08:39:51.732 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:39:51.732 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": { 2024-03-29T08:39:51.732 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 4 2024-03-29T08:39:51.732 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:39:51.732 INFO:teuthology.orchestra.run.smithi037.stdout: "overall": { 2024-03-29T08:39:51.732 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 14 2024-03-29T08:39:51.732 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-29T08:39:51.732 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-29T08:39:51.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:51.734+0000 7f54ef000700 1 -- 172.21.15.37:0/1937565690 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f54cc06cc30 msgr2=0x7f54cc06f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:39:51.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:51.734+0000 7f54ef000700 1 --2- 172.21.15.37:0/1937565690 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f54cc06cc30 0x7f54cc06f0e0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f54d8009fd0 tx=0x7f54d8009e30 comp rx=0 tx=0).stop 2024-03-29T08:39:51.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:51.734+0000 7f54ef000700 1 -- 172.21.15.37:0/1937565690 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f54e81037a0 msgr2=0x7f54e8114060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:39:51.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:51.734+0000 7f54ef000700 1 --2- 172.21.15.37:0/1937565690 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f54e81037a0 0x7f54e8114060 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f54d4002a50 tx=0x7f54d4004600 comp rx=0 tx=0).stop 2024-03-29T08:39:51.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:51.734+0000 7f54ef000700 1 -- 172.21.15.37:0/1937565690 shutdown_connections 2024-03-29T08:39:51.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:51.734+0000 7f54ef000700 1 --2- 172.21.15.37:0/1937565690 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f54cc06cc30 0x7f54cc06f0e0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:39:51.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:51.734+0000 7f54ef000700 1 --2- 172.21.15.37:0/1937565690 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54e8102e00 0x7f54e8113b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:39:51.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:51.734+0000 7f54ef000700 1 --2- 172.21.15.37:0/1937565690 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f54e81037a0 0x7f54e8114060 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:39:51.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:51.734+0000 7f54ef000700 1 -- 172.21.15.37:0/1937565690 >> 172.21.15.37:0/1937565690 conn(0x7f54e80f89b0 msgr2=0x7f54e80f9a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:39:51.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:51.734+0000 7f54ef000700 1 -- 172.21.15.37:0/1937565690 shutdown_connections 2024-03-29T08:39:51.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:51.734+0000 7f54ef000700 1 -- 172.21.15.37:0/1937565690 wait complete. 2024-03-29T08:39:51.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:51.872+0000 7fe1e0e00700 1 -- 172.21.15.37:0/2742596706 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe1dc1037a0 msgr2=0x7fe1dc0fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:39:51.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:51.872+0000 7fe1e0e00700 1 --2- 172.21.15.37:0/2742596706 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe1dc1037a0 0x7fe1dc0fce10 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fe1c8004940 tx=0x7fe1c8009e30 comp rx=0 tx=0).stop 2024-03-29T08:39:51.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:51.875+0000 7fe1e0e00700 1 -- 172.21.15.37:0/2742596706 shutdown_connections 2024-03-29T08:39:51.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:51.875+0000 7fe1e0e00700 1 --2- 172.21.15.37:0/2742596706 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe1dc1037a0 0x7fe1dc0fce10 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:39:51.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:51.875+0000 7fe1e0e00700 1 --2- 172.21.15.37:0/2742596706 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe1dc102e00 0x7fe1dc1031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:39:51.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:51.875+0000 7fe1e0e00700 1 -- 172.21.15.37:0/2742596706 >> 172.21.15.37:0/2742596706 conn(0x7fe1dc0f89b0 msgr2=0x7fe1dc0fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:39:51.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:51.875+0000 7fe1e0e00700 1 -- 172.21.15.37:0/2742596706 shutdown_connections 2024-03-29T08:39:51.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:51.875+0000 7fe1e0e00700 1 -- 172.21.15.37:0/2742596706 wait complete. 2024-03-29T08:39:51.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:51.876+0000 7fe1e0e00700 1 Processor -- start 2024-03-29T08:39:51.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:51.876+0000 7fe1e0e00700 1 -- start start 2024-03-29T08:39:51.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:51.877+0000 7fe1e0e00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe1dc102e00 0x7fe1dc0702c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:39:51.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:51.877+0000 7fe1e0e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe1dc1037a0 0x7fe1dc070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:39:51.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:51.877+0000 7fe1e0e00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1dc070d90 con 0x7fe1dc1037a0 2024-03-29T08:39:51.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:51.877+0000 7fe1e0e00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1dc070ed0 con 0x7fe1dc102e00 2024-03-29T08:39:51.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:51.877+0000 7fe1d9400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe1dc1037a0 0x7fe1dc070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:39:51.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:51.877+0000 7fe1d9400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe1dc1037a0 0x7fe1dc070800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44426/0 (socket says 172.21.15.37:44426) 2024-03-29T08:39:51.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:51.877+0000 7fe1d9400700 1 -- 172.21.15.37:0/699651883 learned_addr learned my addr 172.21.15.37:0/699651883 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:39:51.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:51.878+0000 7fe1d9e00700 1 --2- 172.21.15.37:0/699651883 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe1dc102e00 0x7fe1dc0702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:39:51.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:51.878+0000 7fe1d9400700 1 -- 172.21.15.37:0/699651883 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe1dc102e00 msgr2=0x7fe1dc0702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:39:51.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:51.878+0000 7fe1d9400700 1 --2- 172.21.15.37:0/699651883 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe1dc102e00 0x7fe1dc0702c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:39:51.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:51.878+0000 7fe1d9400700 1 -- 172.21.15.37:0/699651883 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1c8005040 con 0x7fe1dc1037a0 2024-03-29T08:39:51.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:51.878+0000 7fe1d9400700 1 --2- 172.21.15.37:0/699651883 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe1dc1037a0 0x7fe1dc070800 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fe1c8008d50 tx=0x7fe1c8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:39:51.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:51.878+0000 7fe1d2000700 1 -- 172.21.15.37:0/699651883 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe1c800e030 con 0x7fe1dc1037a0 2024-03-29T08:39:51.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:51.879+0000 7fe1d2000700 1 -- 172.21.15.37:0/699651883 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fe1c80093b0 con 0x7fe1dc1037a0 2024-03-29T08:39:51.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:51.879+0000 7fe1e0e00700 1 -- 172.21.15.37:0/699651883 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1dc071120 con 0x7fe1dc1037a0 2024-03-29T08:39:51.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:51.879+0000 7fe1d2000700 1 -- 172.21.15.37:0/699651883 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe1c801cbb0 con 0x7fe1dc1037a0 2024-03-29T08:39:51.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:51.879+0000 7fe1e0e00700 1 -- 172.21.15.37:0/699651883 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1dc11a1a0 con 0x7fe1dc1037a0 2024-03-29T08:39:51.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:51.880+0000 7fe1e0e00700 1 -- 172.21.15.37:0/699651883 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1dc0fe550 con 0x7fe1dc1037a0 2024-03-29T08:39:51.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:51.881+0000 7fe1d2000700 1 -- 172.21.15.37:0/699651883 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fe1c8013930 con 0x7fe1dc1037a0 2024-03-29T08:39:51.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:51.882+0000 7fe1d2000700 1 --2- 172.21.15.37:0/699651883 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fe1c006cc30 0x7fe1c006f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:39:51.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:51.882+0000 7fe1d2000700 1 -- 172.21.15.37:0/699651883 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fe1c808d7a0 con 0x7fe1dc1037a0 2024-03-29T08:39:51.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:51.882+0000 7fe1d9e00700 1 --2- 172.21.15.37:0/699651883 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fe1c006cc30 0x7fe1c006f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:39:51.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:51.883+0000 7fe1d9e00700 1 --2- 172.21.15.37:0/699651883 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fe1c006cc30 0x7fe1c006f0e0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fe1cc009b80 tx=0x7fe1cc00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:39:51.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:51.886+0000 7fe1d2000700 1 -- 172.21.15.37:0/699651883 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fe1c8057760 con 0x7fe1dc1037a0 2024-03-29T08:39:51.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:39:51 smithi037 ceph-mon[142325]: from='client.35570 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:39:52.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:39:51 smithi148 ceph-mon[106692]: from='client.35570 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:39:52.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:52.145+0000 7fe1e0e00700 1 -- 172.21.15.37:0/699651883 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fe1dc05f5f0 con 0x7fe1dc1037a0 2024-03-29T08:39:52.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:52.145+0000 7fe1d2000700 1 -- 172.21.15.37:0/699651883 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 30 v30) v1 ==== 76+0+1841 (secure 0 0 0) 0x7fe1c8061020 con 0x7fe1dc1037a0 2024-03-29T08:39:52.145 INFO:teuthology.orchestra.run.smithi037.stdout:e30 2024-03-29T08:39:52.145 INFO:teuthology.orchestra.run.smithi037.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-29T08:39:52.145 INFO:teuthology.orchestra.run.smithi037.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T08:39:52.146 INFO:teuthology.orchestra.run.smithi037.stdout:legacy client fscid: 1 2024-03-29T08:39:52.146 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:39:52.146 INFO:teuthology.orchestra.run.smithi037.stdout:Filesystem 'cephfs' (1) 2024-03-29T08:39:52.146 INFO:teuthology.orchestra.run.smithi037.stdout:fs_name cephfs 2024-03-29T08:39:52.146 INFO:teuthology.orchestra.run.smithi037.stdout:epoch 21 2024-03-29T08:39:52.146 INFO:teuthology.orchestra.run.smithi037.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-03-29T08:39:52.146 INFO:teuthology.orchestra.run.smithi037.stdout:created 2024-03-29T08:03:29.241269+0000 2024-03-29T08:39:52.146 INFO:teuthology.orchestra.run.smithi037.stdout:modified 2024-03-29T08:17:16.237238+0000 2024-03-29T08:39:52.146 INFO:teuthology.orchestra.run.smithi037.stdout:tableserver 0 2024-03-29T08:39:52.146 INFO:teuthology.orchestra.run.smithi037.stdout:root 0 2024-03-29T08:39:52.146 INFO:teuthology.orchestra.run.smithi037.stdout:session_timeout 60 2024-03-29T08:39:52.146 INFO:teuthology.orchestra.run.smithi037.stdout:session_autoclose 300 2024-03-29T08:39:52.146 INFO:teuthology.orchestra.run.smithi037.stdout:max_file_size 1099511627776 2024-03-29T08:39:52.146 INFO:teuthology.orchestra.run.smithi037.stdout:max_xattr_size 65536 2024-03-29T08:39:52.147 INFO:teuthology.orchestra.run.smithi037.stdout:required_client_features {} 2024-03-29T08:39:52.147 INFO:teuthology.orchestra.run.smithi037.stdout:last_failure 0 2024-03-29T08:39:52.147 INFO:teuthology.orchestra.run.smithi037.stdout:last_failure_osd_epoch 82 2024-03-29T08:39:52.147 INFO:teuthology.orchestra.run.smithi037.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T08:39:52.147 INFO:teuthology.orchestra.run.smithi037.stdout:max_mds 2 2024-03-29T08:39:52.147 INFO:teuthology.orchestra.run.smithi037.stdout:in 0,1 2024-03-29T08:39:52.147 INFO:teuthology.orchestra.run.smithi037.stdout:up {} 2024-03-29T08:39:52.147 INFO:teuthology.orchestra.run.smithi037.stdout:failed 0,1 2024-03-29T08:39:52.147 INFO:teuthology.orchestra.run.smithi037.stdout:damaged 2024-03-29T08:39:52.147 INFO:teuthology.orchestra.run.smithi037.stdout:stopped 2024-03-29T08:39:52.147 INFO:teuthology.orchestra.run.smithi037.stdout:data_pools [3] 2024-03-29T08:39:52.147 INFO:teuthology.orchestra.run.smithi037.stdout:metadata_pool 2 2024-03-29T08:39:52.147 INFO:teuthology.orchestra.run.smithi037.stdout:inline_data disabled 2024-03-29T08:39:52.147 INFO:teuthology.orchestra.run.smithi037.stdout:balancer 2024-03-29T08:39:52.147 INFO:teuthology.orchestra.run.smithi037.stdout:bal_rank_mask -1 2024-03-29T08:39:52.148 INFO:teuthology.orchestra.run.smithi037.stdout:standby_count_wanted 1 2024-03-29T08:39:52.148 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:39:52.148 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:39:52.148 INFO:teuthology.orchestra.run.smithi037.stdout:Standby daemons: 2024-03-29T08:39:52.148 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:39:52.148 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi037.mmwgxe{-1:34440} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.37:6828/1248142005,v1:172.21.15.37:6829/1248142005] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:39:52.148 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi037.iifsmh{-1:34444} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.37:6826/2060882959,v1:172.21.15.37:6827/2060882959] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:39:52.148 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi148.sqteih{-1:44395} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.148:6824/4078652242,v1:172.21.15.148:6825/4078652242] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:39:52.148 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi148.qukpjh{-1:45007} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.148:6826/2394751105,v1:172.21.15.148:6827/2394751105] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:39:52.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:52.148+0000 7fe1e0e00700 1 -- 172.21.15.37:0/699651883 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fe1c006cc30 msgr2=0x7fe1c006f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:39:52.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:52.148+0000 7fe1e0e00700 1 --2- 172.21.15.37:0/699651883 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fe1c006cc30 0x7fe1c006f0e0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fe1cc009b80 tx=0x7fe1cc00b040 comp rx=0 tx=0).stop 2024-03-29T08:39:52.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:52.148+0000 7fe1e0e00700 1 -- 172.21.15.37:0/699651883 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe1dc1037a0 msgr2=0x7fe1dc070800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:39:52.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:52.148+0000 7fe1e0e00700 1 --2- 172.21.15.37:0/699651883 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe1dc1037a0 0x7fe1dc070800 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fe1c8008d50 tx=0x7fe1c8009e30 comp rx=0 tx=0).stop 2024-03-29T08:39:52.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:52.148+0000 7fe1e0e00700 1 -- 172.21.15.37:0/699651883 shutdown_connections 2024-03-29T08:39:52.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:52.148+0000 7fe1e0e00700 1 --2- 172.21.15.37:0/699651883 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fe1c006cc30 0x7fe1c006f0e0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:39:52.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:52.148+0000 7fe1e0e00700 1 --2- 172.21.15.37:0/699651883 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe1dc102e00 0x7fe1dc0702c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:39:52.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:52.148+0000 7fe1e0e00700 1 --2- 172.21.15.37:0/699651883 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe1dc1037a0 0x7fe1dc070800 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:39:52.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:52.148+0000 7fe1e0e00700 1 -- 172.21.15.37:0/699651883 >> 172.21.15.37:0/699651883 conn(0x7fe1dc0f89b0 msgr2=0x7fe1dc0f9a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:39:52.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:52.148+0000 7fe1e0e00700 1 -- 172.21.15.37:0/699651883 shutdown_connections 2024-03-29T08:39:52.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:52.148+0000 7fe1e0e00700 1 -- 172.21.15.37:0/699651883 wait complete. 2024-03-29T08:39:52.150 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 30 2024-03-29T08:39:52.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:52.286+0000 7f423ae00700 1 -- 172.21.15.37:0/3374845487 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f42340fdeb0 msgr2=0x7f42340fe280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:39:52.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:52.286+0000 7f423ae00700 1 --2- 172.21.15.37:0/3374845487 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f42340fdeb0 0x7f42340fe280 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f4224004970 tx=0x7f4224009e30 comp rx=0 tx=0).stop 2024-03-29T08:39:52.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:52.287+0000 7f423ae00700 1 -- 172.21.15.37:0/3374845487 shutdown_connections 2024-03-29T08:39:52.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:52.287+0000 7f423ae00700 1 --2- 172.21.15.37:0/3374845487 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f42340fe850 0x7f4234106f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:39:52.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:52.287+0000 7f423ae00700 1 --2- 172.21.15.37:0/3374845487 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f42340fdeb0 0x7f42340fe280 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:39:52.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:52.287+0000 7f423ae00700 1 -- 172.21.15.37:0/3374845487 >> 172.21.15.37:0/3374845487 conn(0x7f4234072bc0 msgr2=0x7f4234072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:39:52.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:52.287+0000 7f423ae00700 1 -- 172.21.15.37:0/3374845487 shutdown_connections 2024-03-29T08:39:52.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:52.287+0000 7f423ae00700 1 -- 172.21.15.37:0/3374845487 wait complete. 2024-03-29T08:39:52.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:52.287+0000 7f423ae00700 1 Processor -- start 2024-03-29T08:39:52.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:52.288+0000 7f423ae00700 1 -- start start 2024-03-29T08:39:52.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:52.288+0000 7f423ae00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f42340fdeb0 0x7f4234113a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:39:52.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:52.288+0000 7f423ae00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f42340fe850 0x7f4234113f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:39:52.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:52.288+0000 7f423ae00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42341145b0 con 0x7f42340fe850 2024-03-29T08:39:52.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:52.288+0000 7f423ae00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f423410d9f0 con 0x7f42340fdeb0 2024-03-29T08:39:52.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:52.288+0000 7f4233400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f42340fe850 0x7f4234113f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:39:52.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:52.288+0000 7f4233400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f42340fe850 0x7f4234113f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44446/0 (socket says 172.21.15.37:44446) 2024-03-29T08:39:52.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:52.288+0000 7f4233400700 1 -- 172.21.15.37:0/2687426061 learned_addr learned my addr 172.21.15.37:0/2687426061 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:39:52.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:52.289+0000 7f4233e00700 1 --2- 172.21.15.37:0/2687426061 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f42340fdeb0 0x7f4234113a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:39:52.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:52.289+0000 7f4233400700 1 -- 172.21.15.37:0/2687426061 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f42340fdeb0 msgr2=0x7f4234113a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:39:52.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:52.289+0000 7f4233400700 1 --2- 172.21.15.37:0/2687426061 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f42340fdeb0 0x7f4234113a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:39:52.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:52.289+0000 7f4233400700 1 -- 172.21.15.37:0/2687426061 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4224005040 con 0x7f42340fe850 2024-03-29T08:39:52.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:52.289+0000 7f4233e00700 1 --2- 172.21.15.37:0/2687426061 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f42340fdeb0 0x7f4234113a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:39:52.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:52.289+0000 7f4233400700 1 --2- 172.21.15.37:0/2687426061 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f42340fe850 0x7f4234113f60 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f4220002a50 tx=0x7f42200046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:39:52.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:52.289+0000 7f422be00700 1 -- 172.21.15.37:0/2687426061 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4220007cc0 con 0x7f42340fe850 2024-03-29T08:39:52.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:52.289+0000 7f422be00700 1 -- 172.21.15.37:0/2687426061 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f422000f040 con 0x7f42340fe850 2024-03-29T08:39:52.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:52.289+0000 7f422be00700 1 -- 172.21.15.37:0/2687426061 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4220009540 con 0x7f42340fe850 2024-03-29T08:39:52.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:52.289+0000 7f423ae00700 1 -- 172.21.15.37:0/2687426061 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f423410dcd0 con 0x7f42340fe850 2024-03-29T08:39:52.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:52.290+0000 7f423ae00700 1 -- 172.21.15.37:0/2687426061 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f423410e210 con 0x7f42340fe850 2024-03-29T08:39:52.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:52.291+0000 7f423ae00700 1 -- 172.21.15.37:0/2687426061 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4234065e10 con 0x7f42340fe850 2024-03-29T08:39:52.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:52.292+0000 7f422be00700 1 -- 172.21.15.37:0/2687426061 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f42200096a0 con 0x7f42340fe850 2024-03-29T08:39:52.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:52.293+0000 7f422be00700 1 --2- 172.21.15.37:0/2687426061 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f421806cc30 0x7f421806f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:39:52.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:52.293+0000 7f422be00700 1 -- 172.21.15.37:0/2687426061 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f422008bd60 con 0x7f42340fe850 2024-03-29T08:39:52.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:52.293+0000 7f4233e00700 1 --2- 172.21.15.37:0/2687426061 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f421806cc30 0x7f421806f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:39:52.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:52.294+0000 7f4233e00700 1 --2- 172.21.15.37:0/2687426061 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f421806cc30 0x7f421806f0e0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f4224009fd0 tx=0x7f4224009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:39:52.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:52.297+0000 7f422be00700 1 -- 172.21.15.37:0/2687426061 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f4220059830 con 0x7f42340fe850 2024-03-29T08:39:52.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:52.530+0000 7f423ae00700 1 -- 172.21.15.37:0/2687426061 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f42341136f0 con 0x7f421806cc30 2024-03-29T08:39:52.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:52.531+0000 7f422be00700 1 -- 172.21.15.37:0/2687426061 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f42341136f0 con 0x7f421806cc30 2024-03-29T08:39:52.531 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-29T08:39:52.531 INFO:teuthology.orchestra.run.smithi037.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59", 2024-03-29T08:39:52.531 INFO:teuthology.orchestra.run.smithi037.stdout: "in_progress": true, 2024-03-29T08:39:52.531 INFO:teuthology.orchestra.run.smithi037.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-29T08:39:52.531 INFO:teuthology.orchestra.run.smithi037.stdout: "services_complete": [ 2024-03-29T08:39:52.531 INFO:teuthology.orchestra.run.smithi037.stdout: "crash", 2024-03-29T08:39:52.531 INFO:teuthology.orchestra.run.smithi037.stdout: "osd", 2024-03-29T08:39:52.531 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr", 2024-03-29T08:39:52.531 INFO:teuthology.orchestra.run.smithi037.stdout: "mds", 2024-03-29T08:39:52.531 INFO:teuthology.orchestra.run.smithi037.stdout: "mon" 2024-03-29T08:39:52.531 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-29T08:39:52.531 INFO:teuthology.orchestra.run.smithi037.stdout: "progress": "16/21 daemons upgraded", 2024-03-29T08:39:52.531 INFO:teuthology.orchestra.run.smithi037.stdout: "message": "Currently upgrading mds daemons", 2024-03-29T08:39:52.531 INFO:teuthology.orchestra.run.smithi037.stdout: "is_paused": false 2024-03-29T08:39:52.532 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-29T08:39:52.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:52.533+0000 7f423ae00700 1 -- 172.21.15.37:0/2687426061 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f421806cc30 msgr2=0x7f421806f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:39:52.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:52.533+0000 7f423ae00700 1 --2- 172.21.15.37:0/2687426061 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f421806cc30 0x7f421806f0e0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f4224009fd0 tx=0x7f4224009e30 comp rx=0 tx=0).stop 2024-03-29T08:39:52.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:52.533+0000 7f423ae00700 1 -- 172.21.15.37:0/2687426061 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f42340fe850 msgr2=0x7f4234113f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:39:52.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:52.533+0000 7f423ae00700 1 --2- 172.21.15.37:0/2687426061 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f42340fe850 0x7f4234113f60 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f4220002a50 tx=0x7f42200046b0 comp rx=0 tx=0).stop 2024-03-29T08:39:52.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:52.533+0000 7f423ae00700 1 -- 172.21.15.37:0/2687426061 shutdown_connections 2024-03-29T08:39:52.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:52.533+0000 7f423ae00700 1 --2- 172.21.15.37:0/2687426061 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f421806cc30 0x7f421806f0e0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:39:52.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:52.533+0000 7f423ae00700 1 --2- 172.21.15.37:0/2687426061 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f42340fdeb0 0x7f4234113a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:39:52.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:52.533+0000 7f423ae00700 1 --2- 172.21.15.37:0/2687426061 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f42340fe850 0x7f4234113f60 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:39:52.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:52.533+0000 7f423ae00700 1 -- 172.21.15.37:0/2687426061 >> 172.21.15.37:0/2687426061 conn(0x7f4234072bc0 msgr2=0x7f42340ff410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:39:52.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:52.534+0000 7f423ae00700 1 -- 172.21.15.37:0/2687426061 shutdown_connections 2024-03-29T08:39:52.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:52.534+0000 7f423ae00700 1 -- 172.21.15.37:0/2687426061 wait complete. 2024-03-29T08:39:52.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:52.672+0000 7ff77f400700 1 -- 172.21.15.37:0/2605528120 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff778102e00 msgr2=0x7ff7781031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:39:52.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:52.672+0000 7ff77f400700 1 --2- 172.21.15.37:0/2605528120 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff778102e00 0x7ff7781031d0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7ff76c004940 tx=0x7ff76c009e30 comp rx=0 tx=0).stop 2024-03-29T08:39:52.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:52.672+0000 7ff77f400700 1 -- 172.21.15.37:0/2605528120 shutdown_connections 2024-03-29T08:39:52.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:52.672+0000 7ff77f400700 1 --2- 172.21.15.37:0/2605528120 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff7781037a0 0x7ff7780fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:39:52.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:52.672+0000 7ff77f400700 1 --2- 172.21.15.37:0/2605528120 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff778102e00 0x7ff7781031d0 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:39:52.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:52.672+0000 7ff77f400700 1 -- 172.21.15.37:0/2605528120 >> 172.21.15.37:0/2605528120 conn(0x7ff7780f89b0 msgr2=0x7ff7780fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:39:52.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:52.672+0000 7ff77f400700 1 -- 172.21.15.37:0/2605528120 shutdown_connections 2024-03-29T08:39:52.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:52.672+0000 7ff77f400700 1 -- 172.21.15.37:0/2605528120 wait complete. 2024-03-29T08:39:52.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:52.673+0000 7ff77f400700 1 Processor -- start 2024-03-29T08:39:52.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:52.673+0000 7ff77f400700 1 -- start start 2024-03-29T08:39:52.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:52.674+0000 7ff77f400700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff778102e00 0x7ff7780702c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:39:52.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:52.674+0000 7ff77f400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff7781037a0 0x7ff778070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:39:52.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:52.674+0000 7ff77f400700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff778070d90 con 0x7ff7781037a0 2024-03-29T08:39:52.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:52.674+0000 7ff77f400700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff778070ed0 con 0x7ff778102e00 2024-03-29T08:39:52.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:52.674+0000 7ff777e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff7781037a0 0x7ff778070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:39:52.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:52.674+0000 7ff777e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff7781037a0 0x7ff778070800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44474/0 (socket says 172.21.15.37:44474) 2024-03-29T08:39:52.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:52.674+0000 7ff777e00700 1 -- 172.21.15.37:0/3389583701 learned_addr learned my addr 172.21.15.37:0/3389583701 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:39:52.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:52.674+0000 7ff77ca00700 1 --2- 172.21.15.37:0/3389583701 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff778102e00 0x7ff7780702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:39:52.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:52.674+0000 7ff777e00700 1 -- 172.21.15.37:0/3389583701 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff778102e00 msgr2=0x7ff7780702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:39:52.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:52.674+0000 7ff777e00700 1 --2- 172.21.15.37:0/3389583701 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff778102e00 0x7ff7780702c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:39:52.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:52.674+0000 7ff777e00700 1 -- 172.21.15.37:0/3389583701 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff76c005040 con 0x7ff7781037a0 2024-03-29T08:39:52.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:52.675+0000 7ff777e00700 1 --2- 172.21.15.37:0/3389583701 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff7781037a0 0x7ff778070800 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7ff764002a50 tx=0x7ff764004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:39:52.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:52.675+0000 7ff774c00700 1 -- 172.21.15.37:0/3389583701 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff764007cc0 con 0x7ff7781037a0 2024-03-29T08:39:52.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:52.675+0000 7ff774c00700 1 -- 172.21.15.37:0/3389583701 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ff76400f040 con 0x7ff7781037a0 2024-03-29T08:39:52.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:52.675+0000 7ff774c00700 1 -- 172.21.15.37:0/3389583701 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff76400d990 con 0x7ff7781037a0 2024-03-29T08:39:52.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:52.675+0000 7ff77f400700 1 -- 172.21.15.37:0/3389583701 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff778119e20 con 0x7ff7781037a0 2024-03-29T08:39:52.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:52.675+0000 7ff77f400700 1 -- 172.21.15.37:0/3389583701 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff77811a2a0 con 0x7ff7781037a0 2024-03-29T08:39:52.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:52.676+0000 7ff77f400700 1 -- 172.21.15.37:0/3389583701 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7780fe550 con 0x7ff7781037a0 2024-03-29T08:39:52.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:52.677+0000 7ff774c00700 1 -- 172.21.15.37:0/3389583701 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7ff764009710 con 0x7ff7781037a0 2024-03-29T08:39:52.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:52.679+0000 7ff774c00700 1 --2- 172.21.15.37:0/3389583701 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7ff76006cc30 0x7ff76006f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:39:52.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:52.679+0000 7ff774c00700 1 -- 172.21.15.37:0/3389583701 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7ff76408bdf0 con 0x7ff7781037a0 2024-03-29T08:39:52.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:52.679+0000 7ff77ca00700 1 --2- 172.21.15.37:0/3389583701 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7ff76006cc30 0x7ff76006f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:39:52.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:52.679+0000 7ff77ca00700 1 --2- 172.21.15.37:0/3389583701 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7ff76006cc30 0x7ff76006f0e0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7ff76c00b3b0 tx=0x7ff76c009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:39:52.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:52.683+0000 7ff774c00700 1 -- 172.21.15.37:0/3389583701 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7ff764059840 con 0x7ff7781037a0 2024-03-29T08:39:52.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:39:52 smithi037 ceph-mon[142325]: from='client.35574 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:39:52.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:39:52 smithi037 ceph-mon[142325]: pgmap v870: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:39:52.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:39:52 smithi037 ceph-mon[142325]: from='client.35578 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:39:52.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:39:52 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/1937565690' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:39:52.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:39:52 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/699651883' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T08:39:52.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:52.979+0000 7ff77f400700 1 -- 172.21.15.37:0/3389583701 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7ff77805f5f0 con 0x7ff7781037a0 2024-03-29T08:39:52.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:52.980+0000 7ff774c00700 1 -- 172.21.15.37:0/3389583701 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+3583 (secure 0 0 0) 0x7ff77805f5f0 con 0x7ff7781037a0 2024-03-29T08:39:52.980 INFO:teuthology.orchestra.run.smithi037.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem is offline 2024-03-29T08:39:52.980 INFO:teuthology.orchestra.run.smithi037.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-03-29T08:39:52.980 INFO:teuthology.orchestra.run.smithi037.stdout: host smithi148 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/mon.smithi148/config 2024-03-29T08:39:52.980 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/podman: timeout after 895 seconds 2024-03-29T08:39:52.980 INFO:teuthology.orchestra.run.smithi037.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59 -e NODE_NAME=smithi148 -e CEPH_USE_RANDOM_NONCE=1 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4:/var/run/ceph:z -v /var/log/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4:/var/log/ceph:z -v /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpkx2rz2d4:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59 inventory --format=json-pretty --filter-for-batch 2024-03-29T08:39:52.980 INFO:teuthology.orchestra.run.smithi037.stdout:Traceback (most recent call last): 2024-03-29T08:39:52.980 INFO:teuthology.orchestra.run.smithi037.stdout: File "/usr/lib64/python3.6/runpy.py", line 193, in _run_module_as_main 2024-03-29T08:39:52.980 INFO:teuthology.orchestra.run.smithi037.stdout: "__main__", mod_spec) 2024-03-29T08:39:52.980 INFO:teuthology.orchestra.run.smithi037.stdout: File "/usr/lib64/python3.6/runpy.py", line 85, in _run_code 2024-03-29T08:39:52.980 INFO:teuthology.orchestra.run.smithi037.stdout: exec(code, run_globals) 2024-03-29T08:39:52.980 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 10889, in 2024-03-29T08:39:52.980 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 10877, in main 2024-03-29T08:39:52.981 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 2576, in _infer_config 2024-03-29T08:39:52.981 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 2492, in _infer_fsid 2024-03-29T08:39:52.981 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 2604, in _infer_image 2024-03-29T08:39:52.981 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 2479, in _validate_fsid 2024-03-29T08:39:52.981 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 7145, in command_ceph_volume 2024-03-29T08:39:52.981 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 2267, in call_throws 2024-03-29T08:39:52.981 INFO:teuthology.orchestra.run.smithi037.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59 -e NODE_NAME=smithi148 -e CEPH_USE_RANDOM_NONCE=1 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4:/var/run/ceph:z -v /var/log/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4:/var/log/ceph:z -v /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpkx2rz2d4:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59 inventory --format=json-pretty --filter-for-batch 2024-03-29T08:39:52.981 INFO:teuthology.orchestra.run.smithi037.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-03-29T08:39:52.981 INFO:teuthology.orchestra.run.smithi037.stdout: fs cephfs is degraded 2024-03-29T08:39:52.981 INFO:teuthology.orchestra.run.smithi037.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-03-29T08:39:52.981 INFO:teuthology.orchestra.run.smithi037.stdout: fs cephfs is offline because no MDS is active for it. 2024-03-29T08:39:52.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:52.982+0000 7ff77f400700 1 -- 172.21.15.37:0/3389583701 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7ff76006cc30 msgr2=0x7ff76006f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:39:52.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:52.982+0000 7ff77f400700 1 --2- 172.21.15.37:0/3389583701 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7ff76006cc30 0x7ff76006f0e0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7ff76c00b3b0 tx=0x7ff76c009e30 comp rx=0 tx=0).stop 2024-03-29T08:39:52.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:52.982+0000 7ff77f400700 1 -- 172.21.15.37:0/3389583701 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff7781037a0 msgr2=0x7ff778070800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:39:52.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:52.982+0000 7ff77f400700 1 --2- 172.21.15.37:0/3389583701 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff7781037a0 0x7ff778070800 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7ff764002a50 tx=0x7ff764004600 comp rx=0 tx=0).stop 2024-03-29T08:39:52.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:52.982+0000 7ff77f400700 1 -- 172.21.15.37:0/3389583701 shutdown_connections 2024-03-29T08:39:52.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:52.982+0000 7ff77f400700 1 --2- 172.21.15.37:0/3389583701 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7ff76006cc30 0x7ff76006f0e0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:39:52.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:52.982+0000 7ff77f400700 1 --2- 172.21.15.37:0/3389583701 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff778102e00 0x7ff7780702c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:39:52.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:52.982+0000 7ff77f400700 1 --2- 172.21.15.37:0/3389583701 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff7781037a0 0x7ff778070800 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:39:52.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:52.982+0000 7ff77f400700 1 -- 172.21.15.37:0/3389583701 >> 172.21.15.37:0/3389583701 conn(0x7ff7780f89b0 msgr2=0x7ff7780f9a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:39:52.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:52.983+0000 7ff77f400700 1 -- 172.21.15.37:0/3389583701 shutdown_connections 2024-03-29T08:39:52.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:39:52.983+0000 7ff77f400700 1 -- 172.21.15.37:0/3389583701 wait complete. 2024-03-29T08:39:53.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:39:52 smithi148 ceph-mon[106692]: from='client.35574 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:39:53.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:39:52 smithi148 ceph-mon[106692]: pgmap v870: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:39:53.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:39:52 smithi148 ceph-mon[106692]: from='client.35578 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:39:53.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:39:52 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/1937565690' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:39:53.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:39:52 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/699651883' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T08:39:53.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:39:53 smithi037 ceph-mon[142325]: from='client.35590 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:39:53.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:39:53 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/3389583701' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T08:39:54.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:39:53 smithi148 ceph-mon[106692]: from='client.35590 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:39:54.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:39:53 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/3389583701' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T08:39:54.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:39:54 smithi037 ceph-mon[142325]: pgmap v871: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:39:55.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:39:54 smithi148 ceph-mon[106692]: pgmap v871: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:39:55.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:39:55 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:39:56.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:39:55 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:39:56.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:39:56 smithi037 ceph-mon[142325]: pgmap v872: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:39:57.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:39:56 smithi148 ceph-mon[106692]: pgmap v872: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:39:58.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:39:58 smithi037 ceph-mon[142325]: pgmap v873: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:39:59.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:39:58 smithi148 ceph-mon[106692]: pgmap v873: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:40:00.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:40:00 smithi037 ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4-mon-smithi037[142302]: 2024-03-29T08:39:59.999+0000 7f2c60400700 -1 log_channel(cluster) log [ERR] : Health detail: HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem is offline 2024-03-29T08:40:00.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:40:00 smithi037 ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4-mon-smithi037[142302]: 2024-03-29T08:39:59.999+0000 7f2c60400700 -1 log_channel(cluster) log [ERR] : [WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-03-29T08:40:00.428 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:40:00 smithi037 ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4-mon-smithi037[142302]: 2024-03-29T08:39:59.999+0000 7f2c60400700 -1 log_channel(cluster) log [ERR] : host smithi148 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/mon.smithi148/config 2024-03-29T08:40:00.428 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:40:00 smithi037 ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4-mon-smithi037[142302]: 2024-03-29T08:39:59.999+0000 7f2c60400700 -1 log_channel(cluster) log [ERR] : /usr/bin/podman: timeout after 895 seconds 2024-03-29T08:40:00.428 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:40:00 smithi037 ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4-mon-smithi037[142302]: 2024-03-29T08:39:59.999+0000 7f2c60400700 -1 log_channel(cluster) log [ERR] : Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59 -e NODE_NAME=smithi148 -e CEPH_USE_RANDOM_NONCE=1 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4:/var/run/ceph:z -v /var/log/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4:/var/log/ceph:z -v /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpkx2rz2d4:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59 inventory --format=json-pretty --filter-for-batch 2024-03-29T08:40:00.428 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:40:00 smithi037 ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4-mon-smithi037[142302]: 2024-03-29T08:39:59.999+0000 7f2c60400700 -1 log_channel(cluster) log [ERR] : Traceback (most recent call last): 2024-03-29T08:40:00.428 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:40:00 smithi037 ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4-mon-smithi037[142302]: 2024-03-29T08:39:59.999+0000 7f2c60400700 -1 log_channel(cluster) log [ERR] : File "/usr/lib64/python3.6/runpy.py", line 193, in _run_module_as_main 2024-03-29T08:40:00.428 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:40:00 smithi037 ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4-mon-smithi037[142302]: 2024-03-29T08:39:59.999+0000 7f2c60400700 -1 log_channel(cluster) log [ERR] : "__main__", mod_spec) 2024-03-29T08:40:00.428 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:40:00 smithi037 ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4-mon-smithi037[142302]: 2024-03-29T08:39:59.999+0000 7f2c60400700 -1 log_channel(cluster) log [ERR] : File "/usr/lib64/python3.6/runpy.py", line 85, in _run_code 2024-03-29T08:40:00.428 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:40:00 smithi037 ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4-mon-smithi037[142302]: 2024-03-29T08:39:59.999+0000 7f2c60400700 -1 log_channel(cluster) log [ERR] : exec(code, run_globals) 2024-03-29T08:40:00.428 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:40:00 smithi037 ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4-mon-smithi037[142302]: 2024-03-29T08:39:59.999+0000 7f2c60400700 -1 log_channel(cluster) log [ERR] : File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 10889, in 2024-03-29T08:40:00.428 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:40:00 smithi037 ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4-mon-smithi037[142302]: 2024-03-29T08:39:59.999+0000 7f2c60400700 -1 log_channel(cluster) log [ERR] : File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 10877, in main 2024-03-29T08:40:00.428 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:40:00 smithi037 ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4-mon-smithi037[142302]: 2024-03-29T08:39:59.999+0000 7f2c60400700 -1 log_channel(cluster) log [ERR] : File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 2576, in _infer_config 2024-03-29T08:40:00.428 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:40:00 smithi037 ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4-mon-smithi037[142302]: 2024-03-29T08:39:59.999+0000 7f2c60400700 -1 log_channel(cluster) log [ERR] : File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 2492, in _infer_fsid 2024-03-29T08:40:00.428 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:40:00 smithi037 ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4-mon-smithi037[142302]: 2024-03-29T08:39:59.999+0000 7f2c60400700 -1 log_channel(cluster) log [ERR] : File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 2604, in _infer_image 2024-03-29T08:40:00.429 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:40:00 smithi037 ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4-mon-smithi037[142302]: 2024-03-29T08:39:59.999+0000 7f2c60400700 -1 log_channel(cluster) log [ERR] : File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 2479, in _validate_fsid 2024-03-29T08:40:00.429 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:40:00 smithi037 ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4-mon-smithi037[142302]: 2024-03-29T08:39:59.999+0000 7f2c60400700 -1 log_channel(cluster) log [ERR] : File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 7145, in command_ceph_volume 2024-03-29T08:40:00.429 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:40:00 smithi037 ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4-mon-smithi037[142302]: 2024-03-29T08:39:59.999+0000 7f2c60400700 -1 log_channel(cluster) log [ERR] : File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 2267, in call_throws 2024-03-29T08:40:00.429 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:40:00 smithi037 ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4-mon-smithi037[142302]: 2024-03-29T08:39:59.999+0000 7f2c60400700 -1 log_channel(cluster) log [ERR] : RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59 -e NODE_NAME=smithi148 -e CEPH_USE_RANDOM_NONCE=1 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4:/var/run/ceph:z -v /var/log/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4:/var/log/ceph:z -v /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpkx2rz2d4:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59 inventory --format=json-pretty --filter-for-batch 2024-03-29T08:40:00.429 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:40:00 smithi037 ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4-mon-smithi037[142302]: 2024-03-29T08:39:59.999+0000 7f2c60400700 -1 log_channel(cluster) log [ERR] : [WRN] FS_DEGRADED: 1 filesystem is degraded 2024-03-29T08:40:00.429 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:40:00 smithi037 ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4-mon-smithi037[142302]: 2024-03-29T08:39:59.999+0000 7f2c60400700 -1 log_channel(cluster) log [ERR] : fs cephfs is degraded 2024-03-29T08:40:00.429 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:40:00 smithi037 ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4-mon-smithi037[142302]: 2024-03-29T08:39:59.999+0000 7f2c60400700 -1 log_channel(cluster) log [ERR] : [ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-03-29T08:40:00.429 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:40:00 smithi037 ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4-mon-smithi037[142302]: 2024-03-29T08:39:59.999+0000 7f2c60400700 -1 log_channel(cluster) log [ERR] : fs cephfs is offline because no MDS is active for it. 2024-03-29T08:40:00.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:40:00 smithi037 ceph-mon[142325]: pgmap v874: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:40:00.928 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:40:00 smithi037 ceph-mon[142325]: Health detail: HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem is offline 2024-03-29T08:40:00.928 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:40:00 smithi037 ceph-mon[142325]: [WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-03-29T08:40:00.928 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:40:00 smithi037 ceph-mon[142325]: host smithi148 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/mon.smithi148/config 2024-03-29T08:40:00.928 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:40:00 smithi037 ceph-mon[142325]: /usr/bin/podman: timeout after 895 seconds 2024-03-29T08:40:00.928 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:40:00 smithi037 ceph-mon[142325]: Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59 -e NODE_NAME=smithi148 -e CEPH_USE_RANDOM_NONCE=1 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4:/var/run/ceph:z -v /var/log/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4:/var/log/ceph:z -v /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpkx2rz2d4:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59 inventory --format=json-pretty --filter-for-batch 2024-03-29T08:40:00.928 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:40:00 smithi037 ceph-mon[142325]: Traceback (most recent call last): 2024-03-29T08:40:00.928 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:40:00 smithi037 ceph-mon[142325]: File "/usr/lib64/python3.6/runpy.py", line 193, in _run_module_as_main 2024-03-29T08:40:00.928 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:40:00 smithi037 ceph-mon[142325]: "__main__", mod_spec) 2024-03-29T08:40:00.928 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:40:00 smithi037 ceph-mon[142325]: File "/usr/lib64/python3.6/runpy.py", line 85, in _run_code 2024-03-29T08:40:00.928 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:40:00 smithi037 ceph-mon[142325]: exec(code, run_globals) 2024-03-29T08:40:00.928 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:40:00 smithi037 ceph-mon[142325]: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 10889, in 2024-03-29T08:40:00.928 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:40:00 smithi037 ceph-mon[142325]: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 10877, in main 2024-03-29T08:40:00.929 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:40:00 smithi037 ceph-mon[142325]: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 2576, in _infer_config 2024-03-29T08:40:00.929 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:40:00 smithi037 ceph-mon[142325]: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 2492, in _infer_fsid 2024-03-29T08:40:00.929 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:40:00 smithi037 ceph-mon[142325]: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 2604, in _infer_image 2024-03-29T08:40:00.929 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:40:00 smithi037 ceph-mon[142325]: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 2479, in _validate_fsid 2024-03-29T08:40:00.929 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:40:00 smithi037 ceph-mon[142325]: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 7145, in command_ceph_volume 2024-03-29T08:40:00.929 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:40:00 smithi037 ceph-mon[142325]: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 2267, in call_throws 2024-03-29T08:40:00.929 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:40:00 smithi037 ceph-mon[142325]: RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59 -e NODE_NAME=smithi148 -e CEPH_USE_RANDOM_NONCE=1 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4:/var/run/ceph:z -v /var/log/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4:/var/log/ceph:z -v /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpkx2rz2d4:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59 inventory --format=json-pretty --filter-for-batch 2024-03-29T08:40:00.929 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:40:00 smithi037 ceph-mon[142325]: [WRN] FS_DEGRADED: 1 filesystem is degraded 2024-03-29T08:40:00.929 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:40:00 smithi037 ceph-mon[142325]: fs cephfs is degraded 2024-03-29T08:40:00.929 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:40:00 smithi037 ceph-mon[142325]: [ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-03-29T08:40:00.929 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:40:00 smithi037 ceph-mon[142325]: fs cephfs is offline because no MDS is active for it. 2024-03-29T08:40:01.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:40:00 smithi148 ceph-mon[106692]: pgmap v874: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:40:01.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:40:00 smithi148 ceph-mon[106692]: Health detail: HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem is offline 2024-03-29T08:40:01.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:40:00 smithi148 ceph-mon[106692]: [WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-03-29T08:40:01.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:40:00 smithi148 ceph-mon[106692]: host smithi148 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/mon.smithi148/config 2024-03-29T08:40:01.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:40:00 smithi148 ceph-mon[106692]: /usr/bin/podman: timeout after 895 seconds 2024-03-29T08:40:01.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:40:00 smithi148 ceph-mon[106692]: Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59 -e NODE_NAME=smithi148 -e CEPH_USE_RANDOM_NONCE=1 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4:/var/run/ceph:z -v /var/log/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4:/var/log/ceph:z -v /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpkx2rz2d4:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59 inventory --format=json-pretty --filter-for-batch 2024-03-29T08:40:01.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:40:00 smithi148 ceph-mon[106692]: Traceback (most recent call last): 2024-03-29T08:40:01.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:40:00 smithi148 ceph-mon[106692]: File "/usr/lib64/python3.6/runpy.py", line 193, in _run_module_as_main 2024-03-29T08:40:01.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:40:00 smithi148 ceph-mon[106692]: "__main__", mod_spec) 2024-03-29T08:40:01.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:40:00 smithi148 ceph-mon[106692]: File "/usr/lib64/python3.6/runpy.py", line 85, in _run_code 2024-03-29T08:40:01.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:40:00 smithi148 ceph-mon[106692]: exec(code, run_globals) 2024-03-29T08:40:01.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:40:00 smithi148 ceph-mon[106692]: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 10889, in 2024-03-29T08:40:01.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:40:00 smithi148 ceph-mon[106692]: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 10877, in main 2024-03-29T08:40:01.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:40:00 smithi148 ceph-mon[106692]: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 2576, in _infer_config 2024-03-29T08:40:01.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:40:00 smithi148 ceph-mon[106692]: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 2492, in _infer_fsid 2024-03-29T08:40:01.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:40:00 smithi148 ceph-mon[106692]: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 2604, in _infer_image 2024-03-29T08:40:01.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:40:00 smithi148 ceph-mon[106692]: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 2479, in _validate_fsid 2024-03-29T08:40:01.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:40:00 smithi148 ceph-mon[106692]: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 7145, in command_ceph_volume 2024-03-29T08:40:01.058 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:40:00 smithi148 ceph-mon[106692]: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 2267, in call_throws 2024-03-29T08:40:01.058 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:40:00 smithi148 ceph-mon[106692]: RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59 -e NODE_NAME=smithi148 -e CEPH_USE_RANDOM_NONCE=1 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4:/var/run/ceph:z -v /var/log/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4:/var/log/ceph:z -v /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpkx2rz2d4:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59 inventory --format=json-pretty --filter-for-batch 2024-03-29T08:40:01.058 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:40:00 smithi148 ceph-mon[106692]: [WRN] FS_DEGRADED: 1 filesystem is degraded 2024-03-29T08:40:01.058 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:40:00 smithi148 ceph-mon[106692]: fs cephfs is degraded 2024-03-29T08:40:01.058 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:40:00 smithi148 ceph-mon[106692]: [ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-03-29T08:40:01.058 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:40:00 smithi148 ceph-mon[106692]: fs cephfs is offline because no MDS is active for it. 2024-03-29T08:40:02.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:40:02 smithi037 ceph-mon[142325]: pgmap v875: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:40:03.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:40:02 smithi148 ceph-mon[106692]: pgmap v875: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:40:04.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:40:04 smithi037 ceph-mon[142325]: pgmap v876: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:40:05.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:40:04 smithi148 ceph-mon[106692]: pgmap v876: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:40:06.929 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:40:06 smithi037 ceph-mon[142325]: pgmap v877: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:40:07.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:40:06 smithi148 ceph-mon[106692]: pgmap v877: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:40:08.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:40:08 smithi037 ceph-mon[142325]: pgmap v878: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:40:09.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:40:08 smithi148 ceph-mon[106692]: pgmap v878: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:40:10.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:40:10 smithi148 ceph-mon[106692]: pgmap v879: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:40:10.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:40:10 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:40:10.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:40:10 smithi037 ceph-mon[142325]: pgmap v879: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:40:10.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:40:10 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:40:12.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:40:12 smithi148 ceph-mon[106692]: pgmap v880: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:40:12.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:40:12 smithi037 ceph-mon[142325]: pgmap v880: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:40:14.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:40:14 smithi148 ceph-mon[106692]: pgmap v881: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:40:14.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:40:14 smithi037 ceph-mon[142325]: pgmap v881: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:40:16.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:40:16 smithi148 ceph-mon[106692]: pgmap v882: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:40:16.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:40:16 smithi037 ceph-mon[142325]: pgmap v882: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:40:18.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:40:18 smithi148 ceph-mon[106692]: pgmap v883: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:40:18.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:40:18 smithi037 ceph-mon[142325]: pgmap v883: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:40:20.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:40:20 smithi148 ceph-mon[106692]: pgmap v884: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:40:20.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:40:20 smithi037 ceph-mon[142325]: pgmap v884: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:40:22.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:40:22 smithi148 ceph-mon[106692]: pgmap v885: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:40:22.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:40:22 smithi037 ceph-mon[142325]: pgmap v885: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:40:23.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:23.137+0000 7f547c200700 1 -- 172.21.15.37:0/2713432282 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f54741037a0 msgr2=0x7f54740fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:40:23.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:23.137+0000 7f547c200700 1 --2- 172.21.15.37:0/2713432282 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f54741037a0 0x7f54740fce10 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f5460004940 tx=0x7f5460009e30 comp rx=0 tx=0).stop 2024-03-29T08:40:23.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:23.137+0000 7f547c200700 1 -- 172.21.15.37:0/2713432282 shutdown_connections 2024-03-29T08:40:23.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:23.137+0000 7f547c200700 1 --2- 172.21.15.37:0/2713432282 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f54741037a0 0x7f54740fce10 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:40:23.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:23.137+0000 7f547c200700 1 --2- 172.21.15.37:0/2713432282 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5474102e00 0x7f54741031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:40:23.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:23.137+0000 7f547c200700 1 -- 172.21.15.37:0/2713432282 >> 172.21.15.37:0/2713432282 conn(0x7f54740f89b0 msgr2=0x7f54740fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:40:23.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:23.137+0000 7f547c200700 1 -- 172.21.15.37:0/2713432282 shutdown_connections 2024-03-29T08:40:23.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:23.137+0000 7f547c200700 1 -- 172.21.15.37:0/2713432282 wait complete. 2024-03-29T08:40:23.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:23.138+0000 7f547c200700 1 Processor -- start 2024-03-29T08:40:23.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:23.138+0000 7f547c200700 1 -- start start 2024-03-29T08:40:23.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:23.139+0000 7f547c200700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5474102e00 0x7f5474113b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:40:23.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:23.139+0000 7f547c200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f54741037a0 0x7f5474114040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:40:23.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:23.139+0000 7f547c200700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5474114690 con 0x7f54741037a0 2024-03-29T08:40:23.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:23.139+0000 7f547c200700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f547410da80 con 0x7f5474102e00 2024-03-29T08:40:23.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:23.139+0000 7f5478e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f54741037a0 0x7f5474114040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:40:23.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:23.139+0000 7f5478e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f54741037a0 0x7f5474114040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58770/0 (socket says 172.21.15.37:58770) 2024-03-29T08:40:23.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:23.139+0000 7f5478e00700 1 -- 172.21.15.37:0/2542198905 learned_addr learned my addr 172.21.15.37:0/2542198905 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:40:23.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:23.139+0000 7f5479800700 1 --2- 172.21.15.37:0/2542198905 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5474102e00 0x7f5474113b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:40:23.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:23.139+0000 7f5478e00700 1 -- 172.21.15.37:0/2542198905 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5474102e00 msgr2=0x7f5474113b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:40:23.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:23.139+0000 7f5478e00700 1 --2- 172.21.15.37:0/2542198905 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5474102e00 0x7f5474113b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:40:23.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:23.139+0000 7f5478e00700 1 -- 172.21.15.37:0/2542198905 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5460005040 con 0x7f54741037a0 2024-03-29T08:40:23.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:23.139+0000 7f5479800700 1 --2- 172.21.15.37:0/2542198905 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5474102e00 0x7f5474113b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:40:23.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:23.140+0000 7f5478e00700 1 --2- 172.21.15.37:0/2542198905 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f54741037a0 0x7f5474114040 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f5460009390 tx=0x7f5460009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:40:23.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:23.140+0000 7f5471600700 1 -- 172.21.15.37:0/2542198905 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f546000e040 con 0x7f54741037a0 2024-03-29T08:40:23.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:23.140+0000 7f5471600700 1 -- 172.21.15.37:0/2542198905 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f5460013420 con 0x7f54741037a0 2024-03-29T08:40:23.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:23.140+0000 7f547c200700 1 -- 172.21.15.37:0/2542198905 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f547410dd00 con 0x7f54741037a0 2024-03-29T08:40:23.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:23.140+0000 7f5471600700 1 -- 172.21.15.37:0/2542198905 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f546001c630 con 0x7f54741037a0 2024-03-29T08:40:23.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:23.140+0000 7f547c200700 1 -- 172.21.15.37:0/2542198905 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f547410e2c0 con 0x7f54741037a0 2024-03-29T08:40:23.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:23.141+0000 7f547c200700 1 -- 172.21.15.37:0/2542198905 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54740fe550 con 0x7f54741037a0 2024-03-29T08:40:23.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:23.142+0000 7f5471600700 1 -- 172.21.15.37:0/2542198905 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f54600139e0 con 0x7f54741037a0 2024-03-29T08:40:23.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:23.143+0000 7f5471600700 1 --2- 172.21.15.37:0/2542198905 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f545c06cc30 0x7f545c06f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:40:23.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:23.144+0000 7f5471600700 1 -- 172.21.15.37:0/2542198905 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f546008d2a0 con 0x7f54741037a0 2024-03-29T08:40:23.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:23.144+0000 7f5479800700 1 --2- 172.21.15.37:0/2542198905 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f545c06cc30 0x7f545c06f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:40:23.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:23.145+0000 7f5479800700 1 --2- 172.21.15.37:0/2542198905 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f545c06cc30 0x7f545c06f0e0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f5468009fd0 tx=0x7f5468009ee0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:40:23.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:23.148+0000 7f5471600700 1 -- 172.21.15.37:0/2542198905 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f546005ad70 con 0x7f54741037a0 2024-03-29T08:40:23.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:23.381+0000 7f547c200700 1 -- 172.21.15.37:0/2542198905 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5474109da0 con 0x7f545c06cc30 2024-03-29T08:40:23.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:23.382+0000 7f5471600700 1 -- 172.21.15.37:0/2542198905 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f5474109da0 con 0x7f545c06cc30 2024-03-29T08:40:23.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:23.385+0000 7f547c200700 1 -- 172.21.15.37:0/2542198905 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f545c06cc30 msgr2=0x7f545c06f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:40:23.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:23.385+0000 7f547c200700 1 --2- 172.21.15.37:0/2542198905 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f545c06cc30 0x7f545c06f0e0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f5468009fd0 tx=0x7f5468009ee0 comp rx=0 tx=0).stop 2024-03-29T08:40:23.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:23.385+0000 7f547c200700 1 -- 172.21.15.37:0/2542198905 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f54741037a0 msgr2=0x7f5474114040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:40:23.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:23.385+0000 7f547c200700 1 --2- 172.21.15.37:0/2542198905 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f54741037a0 0x7f5474114040 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f5460009390 tx=0x7f5460009e30 comp rx=0 tx=0).stop 2024-03-29T08:40:23.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:23.385+0000 7f547c200700 1 -- 172.21.15.37:0/2542198905 shutdown_connections 2024-03-29T08:40:23.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:23.385+0000 7f547c200700 1 --2- 172.21.15.37:0/2542198905 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f545c06cc30 0x7f545c06f0e0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:40:23.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:23.385+0000 7f547c200700 1 --2- 172.21.15.37:0/2542198905 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5474102e00 0x7f5474113b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:40:23.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:23.385+0000 7f547c200700 1 --2- 172.21.15.37:0/2542198905 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f54741037a0 0x7f5474114040 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:40:23.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:23.385+0000 7f547c200700 1 -- 172.21.15.37:0/2542198905 >> 172.21.15.37:0/2542198905 conn(0x7f54740f89b0 msgr2=0x7f54740f9ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:40:23.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:23.385+0000 7f547c200700 1 -- 172.21.15.37:0/2542198905 shutdown_connections 2024-03-29T08:40:23.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:23.385+0000 7f547c200700 1 -- 172.21.15.37:0/2542198905 wait complete. 2024-03-29T08:40:23.399 INFO:teuthology.orchestra.run.smithi037.stdout:true 2024-03-29T08:40:23.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:23.524+0000 7f30cdc00700 1 -- 172.21.15.37:0/1648897287 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f30c8102e00 msgr2=0x7f30c81031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:40:23.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:23.524+0000 7f30cdc00700 1 --2- 172.21.15.37:0/1648897287 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f30c8102e00 0x7f30c81031d0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f30b8004940 tx=0x7f30b8009e30 comp rx=0 tx=0).stop 2024-03-29T08:40:23.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:23.524+0000 7f30cdc00700 1 -- 172.21.15.37:0/1648897287 shutdown_connections 2024-03-29T08:40:23.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:23.524+0000 7f30cdc00700 1 --2- 172.21.15.37:0/1648897287 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f30c81037a0 0x7f30c80fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:40:23.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:23.524+0000 7f30cdc00700 1 --2- 172.21.15.37:0/1648897287 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f30c8102e00 0x7f30c81031d0 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:40:23.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:23.524+0000 7f30cdc00700 1 -- 172.21.15.37:0/1648897287 >> 172.21.15.37:0/1648897287 conn(0x7f30c80f89b0 msgr2=0x7f30c80fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:40:23.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:23.524+0000 7f30cdc00700 1 -- 172.21.15.37:0/1648897287 shutdown_connections 2024-03-29T08:40:23.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:23.524+0000 7f30cdc00700 1 -- 172.21.15.37:0/1648897287 wait complete. 2024-03-29T08:40:23.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:23.525+0000 7f30cdc00700 1 Processor -- start 2024-03-29T08:40:23.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:23.525+0000 7f30cdc00700 1 -- start start 2024-03-29T08:40:23.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:23.526+0000 7f30cdc00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f30c8102e00 0x7f30c8115cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:40:23.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:23.526+0000 7f30cdc00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f30c81037a0 0x7f30c8116230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:40:23.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:23.526+0000 7f30cdc00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30c8116880 con 0x7f30c81037a0 2024-03-29T08:40:23.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:23.526+0000 7f30cdc00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30c810fc50 con 0x7f30c8102e00 2024-03-29T08:40:23.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:23.526+0000 7f30c6000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f30c81037a0 0x7f30c8116230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:40:23.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:23.526+0000 7f30c6000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f30c81037a0 0x7f30c8116230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58800/0 (socket says 172.21.15.37:58800) 2024-03-29T08:40:23.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:23.526+0000 7f30c6000700 1 -- 172.21.15.37:0/2522879480 learned_addr learned my addr 172.21.15.37:0/2522879480 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:40:23.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:23.526+0000 7f30c6a00700 1 --2- 172.21.15.37:0/2522879480 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f30c8102e00 0x7f30c8115cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:40:23.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:23.526+0000 7f30c6000700 1 -- 172.21.15.37:0/2522879480 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f30c8102e00 msgr2=0x7f30c8115cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:40:23.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:23.526+0000 7f30c6000700 1 --2- 172.21.15.37:0/2522879480 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f30c8102e00 0x7f30c8115cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:40:23.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:23.526+0000 7f30c6000700 1 -- 172.21.15.37:0/2522879480 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30b8005040 con 0x7f30c81037a0 2024-03-29T08:40:23.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:23.527+0000 7f30c6000700 1 --2- 172.21.15.37:0/2522879480 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f30c81037a0 0x7f30c8116230 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f30b4002a50 tx=0x7f30b4004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:40:23.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:23.527+0000 7f30bea00700 1 -- 172.21.15.37:0/2522879480 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f30b4007cc0 con 0x7f30c81037a0 2024-03-29T08:40:23.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:23.527+0000 7f30cdc00700 1 -- 172.21.15.37:0/2522879480 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30c810ff30 con 0x7f30c81037a0 2024-03-29T08:40:23.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:23.527+0000 7f30bea00700 1 -- 172.21.15.37:0/2522879480 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f30b400f040 con 0x7f30c81037a0 2024-03-29T08:40:23.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:23.527+0000 7f30bea00700 1 -- 172.21.15.37:0/2522879480 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f30b400d920 con 0x7f30c81037a0 2024-03-29T08:40:23.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:23.527+0000 7f30cdc00700 1 -- 172.21.15.37:0/2522879480 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30c81104f0 con 0x7f30c81037a0 2024-03-29T08:40:23.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:23.528+0000 7f30cdc00700 1 -- 172.21.15.37:0/2522879480 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30c80fe550 con 0x7f30c81037a0 2024-03-29T08:40:23.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:23.529+0000 7f30bea00700 1 -- 172.21.15.37:0/2522879480 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f30b4009680 con 0x7f30c81037a0 2024-03-29T08:40:23.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:23.530+0000 7f30bea00700 1 --2- 172.21.15.37:0/2522879480 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f30ac06cc30 0x7f30ac06f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:40:23.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:23.530+0000 7f30bea00700 1 -- 172.21.15.37:0/2522879480 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f30b408bd10 con 0x7f30c81037a0 2024-03-29T08:40:23.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:23.530+0000 7f30c6a00700 1 --2- 172.21.15.37:0/2522879480 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f30ac06cc30 0x7f30ac06f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:40:23.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:23.531+0000 7f30c6a00700 1 --2- 172.21.15.37:0/2522879480 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f30ac06cc30 0x7f30ac06f0e0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f30b800b840 tx=0x7f30b8009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:40:23.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:23.535+0000 7f30bea00700 1 -- 172.21.15.37:0/2522879480 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f30b4059760 con 0x7f30c81037a0 2024-03-29T08:40:23.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:23.769+0000 7f30cdc00700 1 -- 172.21.15.37:0/2522879480 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f30c8109e30 con 0x7f30ac06cc30 2024-03-29T08:40:23.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:23.770+0000 7f30bea00700 1 -- 172.21.15.37:0/2522879480 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f30c8109e30 con 0x7f30ac06cc30 2024-03-29T08:40:23.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:23.772+0000 7f30cdc00700 1 -- 172.21.15.37:0/2522879480 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f30ac06cc30 msgr2=0x7f30ac06f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:40:23.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:23.772+0000 7f30cdc00700 1 --2- 172.21.15.37:0/2522879480 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f30ac06cc30 0x7f30ac06f0e0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f30b800b840 tx=0x7f30b8009e30 comp rx=0 tx=0).stop 2024-03-29T08:40:23.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:23.773+0000 7f30cdc00700 1 -- 172.21.15.37:0/2522879480 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f30c81037a0 msgr2=0x7f30c8116230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:40:23.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:23.773+0000 7f30cdc00700 1 --2- 172.21.15.37:0/2522879480 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f30c81037a0 0x7f30c8116230 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f30b4002a50 tx=0x7f30b4004600 comp rx=0 tx=0).stop 2024-03-29T08:40:23.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:23.773+0000 7f30cdc00700 1 -- 172.21.15.37:0/2522879480 shutdown_connections 2024-03-29T08:40:23.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:23.773+0000 7f30cdc00700 1 --2- 172.21.15.37:0/2522879480 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f30ac06cc30 0x7f30ac06f0e0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:40:23.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:23.773+0000 7f30cdc00700 1 --2- 172.21.15.37:0/2522879480 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f30c8102e00 0x7f30c8115cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:40:23.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:23.773+0000 7f30cdc00700 1 --2- 172.21.15.37:0/2522879480 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f30c81037a0 0x7f30c8116230 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:40:23.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:23.773+0000 7f30cdc00700 1 -- 172.21.15.37:0/2522879480 >> 172.21.15.37:0/2522879480 conn(0x7f30c80f89b0 msgr2=0x7f30c80706d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:40:23.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:23.773+0000 7f30cdc00700 1 -- 172.21.15.37:0/2522879480 shutdown_connections 2024-03-29T08:40:23.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:23.773+0000 7f30cdc00700 1 -- 172.21.15.37:0/2522879480 wait complete. 2024-03-29T08:40:23.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:23.910+0000 7f8fe4200700 1 -- 172.21.15.37:0/168834139 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8fdc102e00 msgr2=0x7f8fdc1031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:40:23.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:23.910+0000 7f8fe4200700 1 --2- 172.21.15.37:0/168834139 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8fdc102e00 0x7f8fdc1031d0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f8fd0004940 tx=0x7f8fd0009e30 comp rx=0 tx=0).stop 2024-03-29T08:40:23.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:23.910+0000 7f8fe4200700 1 -- 172.21.15.37:0/168834139 shutdown_connections 2024-03-29T08:40:23.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:23.910+0000 7f8fe4200700 1 --2- 172.21.15.37:0/168834139 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8fdc1037a0 0x7f8fdc0fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:40:23.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:23.910+0000 7f8fe4200700 1 --2- 172.21.15.37:0/168834139 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8fdc102e00 0x7f8fdc1031d0 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:40:23.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:23.910+0000 7f8fe4200700 1 -- 172.21.15.37:0/168834139 >> 172.21.15.37:0/168834139 conn(0x7f8fdc0f89b0 msgr2=0x7f8fdc0fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:40:23.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:23.910+0000 7f8fe4200700 1 -- 172.21.15.37:0/168834139 shutdown_connections 2024-03-29T08:40:23.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:23.910+0000 7f8fe4200700 1 -- 172.21.15.37:0/168834139 wait complete. 2024-03-29T08:40:23.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:23.911+0000 7f8fe4200700 1 Processor -- start 2024-03-29T08:40:23.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:23.911+0000 7f8fe4200700 1 -- start start 2024-03-29T08:40:23.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:23.912+0000 7f8fe4200700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8fdc102e00 0x7f8fdc1138b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:40:23.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:23.912+0000 7f8fe4200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8fdc1037a0 0x7f8fdc113df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:40:23.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:23.912+0000 7f8fe4200700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fdc114440 con 0x7f8fdc1037a0 2024-03-29T08:40:23.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:23.912+0000 7f8fe4200700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fdc10da40 con 0x7f8fdc102e00 2024-03-29T08:40:23.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:23.912+0000 7f8fe0e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8fdc1037a0 0x7f8fdc113df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:40:23.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:23.912+0000 7f8fe0e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8fdc1037a0 0x7f8fdc113df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58814/0 (socket says 172.21.15.37:58814) 2024-03-29T08:40:23.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:23.912+0000 7f8fe0e00700 1 -- 172.21.15.37:0/1042400120 learned_addr learned my addr 172.21.15.37:0/1042400120 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:40:23.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:23.912+0000 7f8fe1800700 1 --2- 172.21.15.37:0/1042400120 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8fdc102e00 0x7f8fdc1138b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:40:23.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:23.912+0000 7f8fe0e00700 1 -- 172.21.15.37:0/1042400120 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8fdc102e00 msgr2=0x7f8fdc1138b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:40:23.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:23.912+0000 7f8fe0e00700 1 --2- 172.21.15.37:0/1042400120 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8fdc102e00 0x7f8fdc1138b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:40:23.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:23.912+0000 7f8fe0e00700 1 -- 172.21.15.37:0/1042400120 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fd0005040 con 0x7f8fdc1037a0 2024-03-29T08:40:23.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:23.913+0000 7f8fe0e00700 1 --2- 172.21.15.37:0/1042400120 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8fdc1037a0 0x7f8fdc113df0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f8fc8002a50 tx=0x7f8fc8004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:40:23.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:23.913+0000 7f8fd9600700 1 -- 172.21.15.37:0/1042400120 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8fc8007cc0 con 0x7f8fdc1037a0 2024-03-29T08:40:23.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:23.913+0000 7f8fd9600700 1 -- 172.21.15.37:0/1042400120 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f8fc800f040 con 0x7f8fdc1037a0 2024-03-29T08:40:23.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:23.913+0000 7f8fe4200700 1 -- 172.21.15.37:0/1042400120 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fdc10dd20 con 0x7f8fdc1037a0 2024-03-29T08:40:23.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:23.913+0000 7f8fd9600700 1 -- 172.21.15.37:0/1042400120 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8fc800d920 con 0x7f8fdc1037a0 2024-03-29T08:40:23.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:23.913+0000 7f8fe4200700 1 -- 172.21.15.37:0/1042400120 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fdc10e2e0 con 0x7f8fdc1037a0 2024-03-29T08:40:23.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:23.914+0000 7f8fe4200700 1 -- 172.21.15.37:0/1042400120 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fdc0fe550 con 0x7f8fdc1037a0 2024-03-29T08:40:23.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:23.915+0000 7f8fd9600700 1 -- 172.21.15.37:0/1042400120 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f8fc8009680 con 0x7f8fdc1037a0 2024-03-29T08:40:23.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:23.916+0000 7f8fd9600700 1 --2- 172.21.15.37:0/1042400120 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f8fc406cc20 0x7f8fc406f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:40:23.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:23.917+0000 7f8fd9600700 1 -- 172.21.15.37:0/1042400120 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f8fc808bdd0 con 0x7f8fdc1037a0 2024-03-29T08:40:23.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:23.917+0000 7f8fe1800700 1 --2- 172.21.15.37:0/1042400120 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f8fc406cc20 0x7f8fc406f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:40:23.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:23.917+0000 7f8fe1800700 1 --2- 172.21.15.37:0/1042400120 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f8fc406cc20 0x7f8fc406f0d0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f8fd0009fd0 tx=0x7f8fd0009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:40:23.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:23.921+0000 7f8fd9600700 1 -- 172.21.15.37:0/1042400120 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f8fc8059820 con 0x7f8fdc1037a0 2024-03-29T08:40:24.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:24.142+0000 7f8fe4200700 1 -- 172.21.15.37:0/1042400120 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f8fdc109dc0 con 0x7f8fc406cc20 2024-03-29T08:40:24.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:24.152+0000 7f8fd9600700 1 -- 172.21.15.37:0/1042400120 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f8fdc109dc0 con 0x7f8fc406cc20 2024-03-29T08:40:24.152 INFO:teuthology.orchestra.run.smithi037.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-29T08:40:24.152 INFO:teuthology.orchestra.run.smithi037.stdout:alertmanager.smithi037 smithi037 *:9093,9094 running (30m) 6m ago 43m 14.3M - 0.25.0 c8568f914cd2 2f9f9746ed0f 2024-03-29T08:40:24.153 INFO:teuthology.orchestra.run.smithi037.stdout:crash.smithi037 smithi037 running (27m) 6m ago 43m 6647k - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 178d416a3186 2024-03-29T08:40:24.153 INFO:teuthology.orchestra.run.smithi037.stdout:crash.smithi148 smithi148 running (27m) 7m ago 42m 6656k - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 23666989d58f 2024-03-29T08:40:24.153 INFO:teuthology.orchestra.run.smithi037.stdout:grafana.smithi037 smithi037 *:3000 running (29m) 6m ago 42m 74.5M - 9.4.7 954c08fa6188 6b976fe49952 2024-03-29T08:40:24.153 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi037.iifsmh smithi037 running (22m) 6m ago 36m 16.4M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 2d97f562cc5b 2024-03-29T08:40:24.153 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi037.mmwgxe smithi037 running (22m) 6m ago 36m 16.7M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff ec37037f46e8 2024-03-29T08:40:24.153 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi148.qukpjh smithi148 running (7m) 7m ago 36m 10.7M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 3257219bc046 2024-03-29T08:40:24.153 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi148.sqteih smithi148 running (22m) 7m ago 36m 16.3M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 44c15c77037a 2024-03-29T08:40:24.153 INFO:teuthology.orchestra.run.smithi037.stdout:mgr.smithi037.wjikii smithi037 *:8443,9283,8765 running (32m) 6m ago 44m 510M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff c5a1417b9417 2024-03-29T08:40:24.153 INFO:teuthology.orchestra.run.smithi037.stdout:mgr.smithi148.onhgvm smithi148 *:8443,9283,8765 running (31m) 7m ago 42m 456M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff fcfdbce062bd 2024-03-29T08:40:24.153 INFO:teuthology.orchestra.run.smithi037.stdout:mon.smithi037 smithi037 running (28m) 6m ago 45m 70.2M 2048M 18.2.2-1015-gd0e392d9 1cc2f45c17ff eea2060783af 2024-03-29T08:40:24.153 INFO:teuthology.orchestra.run.smithi037.stdout:mon.smithi148 smithi148 running (27m) 7m ago 41m 50.4M 2048M 18.2.2-1015-gd0e392d9 1cc2f45c17ff ad4b2c5748be 2024-03-29T08:40:24.153 INFO:teuthology.orchestra.run.smithi037.stdout:node-exporter.smithi037 smithi037 *:9100 running (31m) 6m ago 43m 8514k - 1.5.0 0da6a335fe13 26b8ae9cb878 2024-03-29T08:40:24.153 INFO:teuthology.orchestra.run.smithi037.stdout:node-exporter.smithi148 smithi148 *:9100 running (30m) 7m ago 42m 8464k - 1.5.0 0da6a335fe13 98414e4e64ed 2024-03-29T08:40:24.153 INFO:teuthology.orchestra.run.smithi037.stdout:osd.0 smithi037 running (26m) 6m ago 41m 108M 1619M 18.2.2-1015-gd0e392d9 1cc2f45c17ff c0c584f6245c 2024-03-29T08:40:24.153 INFO:teuthology.orchestra.run.smithi037.stdout:osd.1 smithi037 running (25m) 6m ago 40m 104M 1619M 18.2.2-1015-gd0e392d9 1cc2f45c17ff a4a2ff4cc83d 2024-03-29T08:40:24.154 INFO:teuthology.orchestra.run.smithi037.stdout:osd.2 smithi037 running (25m) 6m ago 40m 85.9M 1619M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 8b1bc71aa79e 2024-03-29T08:40:24.154 INFO:teuthology.orchestra.run.smithi037.stdout:osd.3 smithi148 running (24m) 7m ago 39m 109M 2642M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 1c656ab8ff86 2024-03-29T08:40:24.154 INFO:teuthology.orchestra.run.smithi037.stdout:osd.4 smithi148 running (23m) 7m ago 38m 101M 2642M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 966a9de11d22 2024-03-29T08:40:24.154 INFO:teuthology.orchestra.run.smithi037.stdout:osd.5 smithi148 running (23m) 7m ago 38m 89.4M 2642M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 474c4f9212ce 2024-03-29T08:40:24.154 INFO:teuthology.orchestra.run.smithi037.stdout:prometheus.smithi037 smithi037 *:9095 running (30m) 6m ago 42m 61.0M - 2.43.0 a07b618ecd1d 30c0223ec589 2024-03-29T08:40:24.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:24.155+0000 7f8fe4200700 1 -- 172.21.15.37:0/1042400120 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f8fc406cc20 msgr2=0x7f8fc406f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:40:24.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:24.155+0000 7f8fe4200700 1 --2- 172.21.15.37:0/1042400120 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f8fc406cc20 0x7f8fc406f0d0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f8fd0009fd0 tx=0x7f8fd0009e30 comp rx=0 tx=0).stop 2024-03-29T08:40:24.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:24.155+0000 7f8fe4200700 1 -- 172.21.15.37:0/1042400120 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8fdc1037a0 msgr2=0x7f8fdc113df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:40:24.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:24.155+0000 7f8fe4200700 1 --2- 172.21.15.37:0/1042400120 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8fdc1037a0 0x7f8fdc113df0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f8fc8002a50 tx=0x7f8fc8004600 comp rx=0 tx=0).stop 2024-03-29T08:40:24.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:24.155+0000 7f8fe4200700 1 -- 172.21.15.37:0/1042400120 shutdown_connections 2024-03-29T08:40:24.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:24.155+0000 7f8fe4200700 1 --2- 172.21.15.37:0/1042400120 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f8fc406cc20 0x7f8fc406f0d0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:40:24.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:24.155+0000 7f8fe4200700 1 --2- 172.21.15.37:0/1042400120 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8fdc102e00 0x7f8fdc1138b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:40:24.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:24.155+0000 7f8fe4200700 1 --2- 172.21.15.37:0/1042400120 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8fdc1037a0 0x7f8fdc113df0 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:40:24.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:24.155+0000 7f8fe4200700 1 -- 172.21.15.37:0/1042400120 >> 172.21.15.37:0/1042400120 conn(0x7f8fdc0f89b0 msgr2=0x7f8fdc0f9a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:40:24.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:24.155+0000 7f8fe4200700 1 -- 172.21.15.37:0/1042400120 shutdown_connections 2024-03-29T08:40:24.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:24.155+0000 7f8fe4200700 1 -- 172.21.15.37:0/1042400120 wait complete. 2024-03-29T08:40:24.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:24.291+0000 7f1bf4a00700 1 -- 172.21.15.37:0/2059580163 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1bf0070360 msgr2=0x7f1bf0070730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:40:24.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:24.291+0000 7f1bf4a00700 1 --2- 172.21.15.37:0/2059580163 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1bf0070360 0x7f1bf0070730 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f1be0004970 tx=0x7f1be0009e30 comp rx=0 tx=0).stop 2024-03-29T08:40:24.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:24.291+0000 7f1bf4a00700 1 -- 172.21.15.37:0/2059580163 shutdown_connections 2024-03-29T08:40:24.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:24.291+0000 7f1bf4a00700 1 --2- 172.21.15.37:0/2059580163 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1bf0070c70 0x7f1bf0108f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:40:24.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:24.291+0000 7f1bf4a00700 1 --2- 172.21.15.37:0/2059580163 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1bf0070360 0x7f1bf0070730 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:40:24.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:24.291+0000 7f1bf4a00700 1 -- 172.21.15.37:0/2059580163 >> 172.21.15.37:0/2059580163 conn(0x7f1bf00f8830 msgr2=0x7f1bf00fac40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:40:24.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:24.291+0000 7f1bf4a00700 1 -- 172.21.15.37:0/2059580163 shutdown_connections 2024-03-29T08:40:24.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:24.291+0000 7f1bf4a00700 1 -- 172.21.15.37:0/2059580163 wait complete. 2024-03-29T08:40:24.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:24.292+0000 7f1bf4a00700 1 Processor -- start 2024-03-29T08:40:24.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:24.292+0000 7f1bf4a00700 1 -- start start 2024-03-29T08:40:24.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:24.293+0000 7f1bf4a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1bf0070360 0x7f1bf0115af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:40:24.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:24.293+0000 7f1bf4a00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1bf0070c70 0x7f1bf0116030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:40:24.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:24.293+0000 7f1bf4a00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bf0116680 con 0x7f1bf0070360 2024-03-29T08:40:24.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:24.293+0000 7f1bf4a00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bf010fa70 con 0x7f1bf0070c70 2024-03-29T08:40:24.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:24.293+0000 7f1bee800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1bf0070360 0x7f1bf0115af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:40:24.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:24.293+0000 7f1bee800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1bf0070360 0x7f1bf0115af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58832/0 (socket says 172.21.15.37:58832) 2024-03-29T08:40:24.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:24.293+0000 7f1bee800700 1 -- 172.21.15.37:0/736034585 learned_addr learned my addr 172.21.15.37:0/736034585 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:40:24.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:24.293+0000 7f1bede00700 1 --2- 172.21.15.37:0/736034585 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1bf0070c70 0x7f1bf0116030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:40:24.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:24.293+0000 7f1bee800700 1 -- 172.21.15.37:0/736034585 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1bf0070c70 msgr2=0x7f1bf0116030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:40:24.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:24.293+0000 7f1bee800700 1 --2- 172.21.15.37:0/736034585 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1bf0070c70 0x7f1bf0116030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:40:24.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:24.293+0000 7f1bee800700 1 -- 172.21.15.37:0/736034585 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1be0005040 con 0x7f1bf0070360 2024-03-29T08:40:24.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:24.294+0000 7f1bee800700 1 --2- 172.21.15.37:0/736034585 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1bf0070360 0x7f1bf0115af0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f1be0004f50 tx=0x7f1be0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:40:24.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:24.294+0000 7f1be6a00700 1 -- 172.21.15.37:0/736034585 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1be000e040 con 0x7f1bf0070360 2024-03-29T08:40:24.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:24.294+0000 7f1be6a00700 1 -- 172.21.15.37:0/736034585 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f1be0013420 con 0x7f1bf0070360 2024-03-29T08:40:24.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:24.294+0000 7f1bf4a00700 1 -- 172.21.15.37:0/736034585 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1bf010fcf0 con 0x7f1bf0070360 2024-03-29T08:40:24.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:24.294+0000 7f1be6a00700 1 -- 172.21.15.37:0/736034585 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1be001c630 con 0x7f1bf0070360 2024-03-29T08:40:24.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:24.294+0000 7f1bf4a00700 1 -- 172.21.15.37:0/736034585 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1bf0110230 con 0x7f1bf0070360 2024-03-29T08:40:24.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:24.296+0000 7f1bf4a00700 1 -- 172.21.15.37:0/736034585 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1bf0071aa0 con 0x7f1bf0070360 2024-03-29T08:40:24.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:24.296+0000 7f1be6a00700 1 -- 172.21.15.37:0/736034585 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f1be0013590 con 0x7f1bf0070360 2024-03-29T08:40:24.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:24.297+0000 7f1be6a00700 1 --2- 172.21.15.37:0/736034585 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f1bd406cc30 0x7f1bd406f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:40:24.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:24.298+0000 7f1be6a00700 1 -- 172.21.15.37:0/736034585 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f1be008d280 con 0x7f1bf0070360 2024-03-29T08:40:24.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:24.298+0000 7f1bede00700 1 --2- 172.21.15.37:0/736034585 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f1bd406cc30 0x7f1bd406f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:40:24.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:24.298+0000 7f1bede00700 1 --2- 172.21.15.37:0/736034585 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f1bd406cc30 0x7f1bd406f0e0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f1bf0110c30 tx=0x7f1bdc00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:40:24.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:24.302+0000 7f1be6a00700 1 -- 172.21.15.37:0/736034585 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f1be005acd0 con 0x7f1bf0070360 2024-03-29T08:40:24.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:24.602+0000 7f1bf4a00700 1 -- 172.21.15.37:0/736034585 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f1bf010bdf0 con 0x7f1bf0070360 2024-03-29T08:40:24.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:24.602+0000 7f1be6a00700 1 -- 172.21.15.37:0/736034585 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7f1be005a860 con 0x7f1bf0070360 2024-03-29T08:40:24.602 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-29T08:40:24.602 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": { 2024-03-29T08:40:24.602 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 2 2024-03-29T08:40:24.602 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:40:24.602 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": { 2024-03-29T08:40:24.602 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 2 2024-03-29T08:40:24.602 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:40:24.603 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": { 2024-03-29T08:40:24.603 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 6 2024-03-29T08:40:24.603 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:40:24.603 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": { 2024-03-29T08:40:24.603 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 4 2024-03-29T08:40:24.603 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:40:24.603 INFO:teuthology.orchestra.run.smithi037.stdout: "overall": { 2024-03-29T08:40:24.603 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 14 2024-03-29T08:40:24.603 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-29T08:40:24.603 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-29T08:40:24.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:24.604+0000 7f1bf4a00700 1 -- 172.21.15.37:0/736034585 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f1bd406cc30 msgr2=0x7f1bd406f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:40:24.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:24.604+0000 7f1bf4a00700 1 --2- 172.21.15.37:0/736034585 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f1bd406cc30 0x7f1bd406f0e0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f1bf0110c30 tx=0x7f1bdc00b040 comp rx=0 tx=0).stop 2024-03-29T08:40:24.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:24.604+0000 7f1bf4a00700 1 -- 172.21.15.37:0/736034585 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1bf0070360 msgr2=0x7f1bf0115af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:40:24.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:24.604+0000 7f1bf4a00700 1 --2- 172.21.15.37:0/736034585 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1bf0070360 0x7f1bf0115af0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f1be0004f50 tx=0x7f1be0009e30 comp rx=0 tx=0).stop 2024-03-29T08:40:24.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:24.605+0000 7f1bf4a00700 1 -- 172.21.15.37:0/736034585 shutdown_connections 2024-03-29T08:40:24.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:24.605+0000 7f1bf4a00700 1 --2- 172.21.15.37:0/736034585 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f1bd406cc30 0x7f1bd406f0e0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:40:24.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:24.605+0000 7f1bf4a00700 1 --2- 172.21.15.37:0/736034585 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1bf0070360 0x7f1bf0115af0 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:40:24.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:24.605+0000 7f1bf4a00700 1 --2- 172.21.15.37:0/736034585 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1bf0070c70 0x7f1bf0116030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:40:24.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:24.605+0000 7f1bf4a00700 1 -- 172.21.15.37:0/736034585 >> 172.21.15.37:0/736034585 conn(0x7f1bf00f8830 msgr2=0x7f1bf00f9de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:40:24.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:24.605+0000 7f1bf4a00700 1 -- 172.21.15.37:0/736034585 shutdown_connections 2024-03-29T08:40:24.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:24.605+0000 7f1bf4a00700 1 -- 172.21.15.37:0/736034585 wait complete. 2024-03-29T08:40:24.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:24.744+0000 7fbd14400700 1 -- 172.21.15.37:0/3380891582 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbd0c0fe830 msgr2=0x7fbd0c106f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:40:24.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:24.744+0000 7fbd14400700 1 --2- 172.21.15.37:0/3380891582 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbd0c0fe830 0x7fbd0c106f00 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fbcf8004940 tx=0x7fbcf8009e30 comp rx=0 tx=0).stop 2024-03-29T08:40:24.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:24.744+0000 7fbd14400700 1 -- 172.21.15.37:0/3380891582 shutdown_connections 2024-03-29T08:40:24.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:24.744+0000 7fbd14400700 1 --2- 172.21.15.37:0/3380891582 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbd0c0fe830 0x7fbd0c106f00 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:40:24.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:24.744+0000 7fbd14400700 1 --2- 172.21.15.37:0/3380891582 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbd0c0fde90 0x7fbd0c0fe260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:40:24.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:24.744+0000 7fbd14400700 1 -- 172.21.15.37:0/3380891582 >> 172.21.15.37:0/3380891582 conn(0x7fbd0c072bc0 msgr2=0x7fbd0c072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:40:24.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:24.744+0000 7fbd14400700 1 -- 172.21.15.37:0/3380891582 shutdown_connections 2024-03-29T08:40:24.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:24.745+0000 7fbd14400700 1 -- 172.21.15.37:0/3380891582 wait complete. 2024-03-29T08:40:24.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:24.745+0000 7fbd14400700 1 Processor -- start 2024-03-29T08:40:24.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:24.745+0000 7fbd14400700 1 -- start start 2024-03-29T08:40:24.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:24.746+0000 7fbd14400700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbd0c0fde90 0x7fbd0c115d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:40:24.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:24.746+0000 7fbd14400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbd0c0fe830 0x7fbd0c116270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:40:24.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:24.746+0000 7fbd14400700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd0c1168c0 con 0x7fbd0c0fe830 2024-03-29T08:40:24.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:24.746+0000 7fbd14400700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd0c10fc90 con 0x7fbd0c0fde90 2024-03-29T08:40:24.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:24.746+0000 7fbd11000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbd0c0fe830 0x7fbd0c116270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:40:24.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:24.746+0000 7fbd11a00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbd0c0fde90 0x7fbd0c115d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:40:24.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:24.746+0000 7fbd11000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbd0c0fe830 0x7fbd0c116270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58864/0 (socket says 172.21.15.37:58864) 2024-03-29T08:40:24.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:24.746+0000 7fbd11000700 1 -- 172.21.15.37:0/1868440670 learned_addr learned my addr 172.21.15.37:0/1868440670 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:40:24.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:24.747+0000 7fbd11000700 1 -- 172.21.15.37:0/1868440670 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbd0c0fde90 msgr2=0x7fbd0c115d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:40:24.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:24.747+0000 7fbd11000700 1 --2- 172.21.15.37:0/1868440670 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbd0c0fde90 0x7fbd0c115d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:40:24.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:24.747+0000 7fbd11000700 1 -- 172.21.15.37:0/1868440670 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcf8005040 con 0x7fbd0c0fe830 2024-03-29T08:40:24.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:24.747+0000 7fbd11000700 1 --2- 172.21.15.37:0/1868440670 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbd0c0fe830 0x7fbd0c116270 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fbcf80093b0 tx=0x7fbcf8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:40:24.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:24.747+0000 7fbd09600700 1 -- 172.21.15.37:0/1868440670 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbcf800e040 con 0x7fbd0c0fe830 2024-03-29T08:40:24.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:24.747+0000 7fbd09600700 1 -- 172.21.15.37:0/1868440670 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fbcf800bed0 con 0x7fbd0c0fe830 2024-03-29T08:40:24.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:24.747+0000 7fbd14400700 1 -- 172.21.15.37:0/1868440670 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd0c10ff10 con 0x7fbd0c0fe830 2024-03-29T08:40:24.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:24.747+0000 7fbd09600700 1 -- 172.21.15.37:0/1868440670 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbcf801c3f0 con 0x7fbd0c0fe830 2024-03-29T08:40:24.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:24.747+0000 7fbd14400700 1 -- 172.21.15.37:0/1868440670 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd0c110450 con 0x7fbd0c0fe830 2024-03-29T08:40:24.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:24.748+0000 7fbd14400700 1 -- 172.21.15.37:0/1868440670 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd0c106a20 con 0x7fbd0c0fe830 2024-03-29T08:40:24.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:24.749+0000 7fbd09600700 1 -- 172.21.15.37:0/1868440670 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fbcf8013790 con 0x7fbd0c0fe830 2024-03-29T08:40:24.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:24.750+0000 7fbd09600700 1 --2- 172.21.15.37:0/1868440670 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fbcf406cc30 0x7fbcf406f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:40:24.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:24.750+0000 7fbd09600700 1 -- 172.21.15.37:0/1868440670 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fbcf808d3d0 con 0x7fbd0c0fe830 2024-03-29T08:40:24.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:24.751+0000 7fbd11a00700 1 --2- 172.21.15.37:0/1868440670 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fbcf406cc30 0x7fbcf406f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:40:24.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:24.751+0000 7fbd11a00700 1 --2- 172.21.15.37:0/1868440670 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fbcf406cc30 0x7fbcf406f0e0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fbd00007fd0 tx=0x7fbd00009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:40:24.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:24.755+0000 7fbd09600700 1 -- 172.21.15.37:0/1868440670 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fbcf805ae20 con 0x7fbd0c0fe830 2024-03-29T08:40:24.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:40:24 smithi148 ceph-mon[106692]: pgmap v886: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:40:24.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:40:24 smithi148 ceph-mon[106692]: from='client.35598 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:40:24.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:40:24 smithi148 ceph-mon[106692]: from='client.35602 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:40:24.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:40:24 smithi037 ceph-mon[142325]: pgmap v886: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:40:24.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:40:24 smithi037 ceph-mon[142325]: from='client.35598 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:40:24.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:40:24 smithi037 ceph-mon[142325]: from='client.35602 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:40:25.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:25.014+0000 7fbd14400700 1 -- 172.21.15.37:0/1868440670 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fbd0c065e30 con 0x7fbd0c0fe830 2024-03-29T08:40:25.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:25.014+0000 7fbd09600700 1 -- 172.21.15.37:0/1868440670 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 30 v30) v1 ==== 76+0+1841 (secure 0 0 0) 0x7fbcf8013420 con 0x7fbd0c0fe830 2024-03-29T08:40:25.014 INFO:teuthology.orchestra.run.smithi037.stdout:e30 2024-03-29T08:40:25.014 INFO:teuthology.orchestra.run.smithi037.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-29T08:40:25.014 INFO:teuthology.orchestra.run.smithi037.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T08:40:25.014 INFO:teuthology.orchestra.run.smithi037.stdout:legacy client fscid: 1 2024-03-29T08:40:25.014 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:40:25.015 INFO:teuthology.orchestra.run.smithi037.stdout:Filesystem 'cephfs' (1) 2024-03-29T08:40:25.015 INFO:teuthology.orchestra.run.smithi037.stdout:fs_name cephfs 2024-03-29T08:40:25.015 INFO:teuthology.orchestra.run.smithi037.stdout:epoch 21 2024-03-29T08:40:25.015 INFO:teuthology.orchestra.run.smithi037.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-03-29T08:40:25.015 INFO:teuthology.orchestra.run.smithi037.stdout:created 2024-03-29T08:03:29.241269+0000 2024-03-29T08:40:25.015 INFO:teuthology.orchestra.run.smithi037.stdout:modified 2024-03-29T08:17:16.237238+0000 2024-03-29T08:40:25.015 INFO:teuthology.orchestra.run.smithi037.stdout:tableserver 0 2024-03-29T08:40:25.015 INFO:teuthology.orchestra.run.smithi037.stdout:root 0 2024-03-29T08:40:25.015 INFO:teuthology.orchestra.run.smithi037.stdout:session_timeout 60 2024-03-29T08:40:25.015 INFO:teuthology.orchestra.run.smithi037.stdout:session_autoclose 300 2024-03-29T08:40:25.015 INFO:teuthology.orchestra.run.smithi037.stdout:max_file_size 1099511627776 2024-03-29T08:40:25.015 INFO:teuthology.orchestra.run.smithi037.stdout:max_xattr_size 65536 2024-03-29T08:40:25.015 INFO:teuthology.orchestra.run.smithi037.stdout:required_client_features {} 2024-03-29T08:40:25.015 INFO:teuthology.orchestra.run.smithi037.stdout:last_failure 0 2024-03-29T08:40:25.015 INFO:teuthology.orchestra.run.smithi037.stdout:last_failure_osd_epoch 82 2024-03-29T08:40:25.015 INFO:teuthology.orchestra.run.smithi037.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T08:40:25.016 INFO:teuthology.orchestra.run.smithi037.stdout:max_mds 2 2024-03-29T08:40:25.016 INFO:teuthology.orchestra.run.smithi037.stdout:in 0,1 2024-03-29T08:40:25.016 INFO:teuthology.orchestra.run.smithi037.stdout:up {} 2024-03-29T08:40:25.016 INFO:teuthology.orchestra.run.smithi037.stdout:failed 0,1 2024-03-29T08:40:25.016 INFO:teuthology.orchestra.run.smithi037.stdout:damaged 2024-03-29T08:40:25.016 INFO:teuthology.orchestra.run.smithi037.stdout:stopped 2024-03-29T08:40:25.016 INFO:teuthology.orchestra.run.smithi037.stdout:data_pools [3] 2024-03-29T08:40:25.016 INFO:teuthology.orchestra.run.smithi037.stdout:metadata_pool 2 2024-03-29T08:40:25.016 INFO:teuthology.orchestra.run.smithi037.stdout:inline_data disabled 2024-03-29T08:40:25.016 INFO:teuthology.orchestra.run.smithi037.stdout:balancer 2024-03-29T08:40:25.016 INFO:teuthology.orchestra.run.smithi037.stdout:bal_rank_mask -1 2024-03-29T08:40:25.016 INFO:teuthology.orchestra.run.smithi037.stdout:standby_count_wanted 1 2024-03-29T08:40:25.016 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:40:25.016 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:40:25.016 INFO:teuthology.orchestra.run.smithi037.stdout:Standby daemons: 2024-03-29T08:40:25.016 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:40:25.017 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi037.mmwgxe{-1:34440} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.37:6828/1248142005,v1:172.21.15.37:6829/1248142005] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:40:25.017 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi037.iifsmh{-1:34444} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.37:6826/2060882959,v1:172.21.15.37:6827/2060882959] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:40:25.017 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi148.sqteih{-1:44395} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.148:6824/4078652242,v1:172.21.15.148:6825/4078652242] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:40:25.017 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi148.qukpjh{-1:45007} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.148:6826/2394751105,v1:172.21.15.148:6827/2394751105] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:40:25.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:25.017+0000 7fbd14400700 1 -- 172.21.15.37:0/1868440670 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fbcf406cc30 msgr2=0x7fbcf406f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:40:25.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:25.017+0000 7fbd14400700 1 --2- 172.21.15.37:0/1868440670 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fbcf406cc30 0x7fbcf406f0e0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fbd00007fd0 tx=0x7fbd00009040 comp rx=0 tx=0).stop 2024-03-29T08:40:25.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:25.017+0000 7fbd14400700 1 -- 172.21.15.37:0/1868440670 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbd0c0fe830 msgr2=0x7fbd0c116270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:40:25.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:25.017+0000 7fbd14400700 1 --2- 172.21.15.37:0/1868440670 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbd0c0fe830 0x7fbd0c116270 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fbcf80093b0 tx=0x7fbcf8009e30 comp rx=0 tx=0).stop 2024-03-29T08:40:25.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:25.017+0000 7fbd14400700 1 -- 172.21.15.37:0/1868440670 shutdown_connections 2024-03-29T08:40:25.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:25.017+0000 7fbd14400700 1 --2- 172.21.15.37:0/1868440670 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fbcf406cc30 0x7fbcf406f0e0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:40:25.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:25.017+0000 7fbd14400700 1 --2- 172.21.15.37:0/1868440670 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbd0c0fde90 0x7fbd0c115d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:40:25.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:25.017+0000 7fbd14400700 1 --2- 172.21.15.37:0/1868440670 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbd0c0fe830 0x7fbd0c116270 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:40:25.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:25.017+0000 7fbd14400700 1 -- 172.21.15.37:0/1868440670 >> 172.21.15.37:0/1868440670 conn(0x7fbd0c072bc0 msgr2=0x7fbd0c0fb840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:40:25.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:25.017+0000 7fbd14400700 1 -- 172.21.15.37:0/1868440670 shutdown_connections 2024-03-29T08:40:25.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:25.017+0000 7fbd14400700 1 -- 172.21.15.37:0/1868440670 wait complete. 2024-03-29T08:40:25.019 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 30 2024-03-29T08:40:25.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:25.156+0000 7f8e7e000700 1 -- 172.21.15.37:0/2954729995 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8e78104e30 msgr2=0x7f8e78105200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:40:25.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:25.156+0000 7f8e7e000700 1 --2- 172.21.15.37:0/2954729995 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8e78104e30 0x7f8e78105200 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f8e6c004940 tx=0x7f8e6c009e30 comp rx=0 tx=0).stop 2024-03-29T08:40:25.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:25.157+0000 7f8e7e000700 1 -- 172.21.15.37:0/2954729995 shutdown_connections 2024-03-29T08:40:25.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:25.157+0000 7f8e7e000700 1 --2- 172.21.15.37:0/2954729995 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8e781057d0 0x7f8e780fee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:40:25.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:25.157+0000 7f8e7e000700 1 --2- 172.21.15.37:0/2954729995 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8e78104e30 0x7f8e78105200 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:40:25.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:25.157+0000 7f8e7e000700 1 -- 172.21.15.37:0/2954729995 >> 172.21.15.37:0/2954729995 conn(0x7f8e780fab10 msgr2=0x7f8e780fcef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:40:25.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:25.157+0000 7f8e7e000700 1 -- 172.21.15.37:0/2954729995 shutdown_connections 2024-03-29T08:40:25.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:25.157+0000 7f8e7e000700 1 -- 172.21.15.37:0/2954729995 wait complete. 2024-03-29T08:40:25.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:25.158+0000 7f8e7e000700 1 Processor -- start 2024-03-29T08:40:25.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:25.158+0000 7f8e7e000700 1 -- start start 2024-03-29T08:40:25.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:25.159+0000 7f8e7e000700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8e78104e30 0x7f8e78115a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:40:25.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:25.159+0000 7f8e7e000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8e781057d0 0x7f8e78115f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:40:25.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:25.159+0000 7f8e7e000700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e781165c0 con 0x7f8e781057d0 2024-03-29T08:40:25.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:25.159+0000 7f8e7e000700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e7810f9b0 con 0x7f8e78104e30 2024-03-29T08:40:25.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:25.159+0000 7f8e77e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8e781057d0 0x7f8e78115f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:40:25.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:25.159+0000 7f8e77e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8e781057d0 0x7f8e78115f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58894/0 (socket says 172.21.15.37:58894) 2024-03-29T08:40:25.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:25.159+0000 7f8e77e00700 1 -- 172.21.15.37:0/2872628724 learned_addr learned my addr 172.21.15.37:0/2872628724 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:40:25.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:25.159+0000 7f8e7cc00700 1 --2- 172.21.15.37:0/2872628724 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8e78104e30 0x7f8e78115a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:40:25.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:25.159+0000 7f8e77e00700 1 -- 172.21.15.37:0/2872628724 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8e78104e30 msgr2=0x7f8e78115a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:40:25.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:25.159+0000 7f8e77e00700 1 --2- 172.21.15.37:0/2872628724 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8e78104e30 0x7f8e78115a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:40:25.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:25.159+0000 7f8e77e00700 1 -- 172.21.15.37:0/2872628724 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e6c005040 con 0x7f8e781057d0 2024-03-29T08:40:25.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:25.160+0000 7f8e77e00700 1 --2- 172.21.15.37:0/2872628724 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8e781057d0 0x7f8e78115f70 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f8e64002a50 tx=0x7f8e640046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:40:25.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:25.160+0000 7f8e74c00700 1 -- 172.21.15.37:0/2872628724 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8e64007cc0 con 0x7f8e781057d0 2024-03-29T08:40:25.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:25.160+0000 7f8e7e000700 1 -- 172.21.15.37:0/2872628724 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e7810fc90 con 0x7f8e781057d0 2024-03-29T08:40:25.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:25.160+0000 7f8e74c00700 1 -- 172.21.15.37:0/2872628724 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f8e6400f040 con 0x7f8e781057d0 2024-03-29T08:40:25.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:25.160+0000 7f8e74c00700 1 -- 172.21.15.37:0/2872628724 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8e64009590 con 0x7f8e781057d0 2024-03-29T08:40:25.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:25.160+0000 7f8e7e000700 1 -- 172.21.15.37:0/2872628724 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e78110250 con 0x7f8e781057d0 2024-03-29T08:40:25.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:25.162+0000 7f8e7e000700 1 -- 172.21.15.37:0/2872628724 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e78100580 con 0x7f8e781057d0 2024-03-29T08:40:25.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:25.162+0000 7f8e74c00700 1 -- 172.21.15.37:0/2872628724 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f8e640096f0 con 0x7f8e781057d0 2024-03-29T08:40:25.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:25.163+0000 7f8e74c00700 1 --2- 172.21.15.37:0/2872628724 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f8e6006cc20 0x7f8e6006f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:40:25.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:25.163+0000 7f8e74c00700 1 -- 172.21.15.37:0/2872628724 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f8e6408bfd0 con 0x7f8e781057d0 2024-03-29T08:40:25.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:25.164+0000 7f8e7cc00700 1 --2- 172.21.15.37:0/2872628724 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f8e6006cc20 0x7f8e6006f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:40:25.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:25.164+0000 7f8e7cc00700 1 --2- 172.21.15.37:0/2872628724 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f8e6006cc20 0x7f8e6006f0d0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f8e6c004f90 tx=0x7f8e6c009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:40:25.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:25.168+0000 7f8e74c00700 1 -- 172.21.15.37:0/2872628724 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f8e640bf9f0 con 0x7f8e781057d0 2024-03-29T08:40:25.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:25.416+0000 7f8e7e000700 1 -- 172.21.15.37:0/2872628724 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f8e7810bdf0 con 0x7f8e6006cc20 2024-03-29T08:40:25.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:25.417+0000 7f8e74c00700 1 -- 172.21.15.37:0/2872628724 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f8e7810bdf0 con 0x7f8e6006cc20 2024-03-29T08:40:25.417 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-29T08:40:25.417 INFO:teuthology.orchestra.run.smithi037.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59", 2024-03-29T08:40:25.417 INFO:teuthology.orchestra.run.smithi037.stdout: "in_progress": true, 2024-03-29T08:40:25.417 INFO:teuthology.orchestra.run.smithi037.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-29T08:40:25.417 INFO:teuthology.orchestra.run.smithi037.stdout: "services_complete": [ 2024-03-29T08:40:25.417 INFO:teuthology.orchestra.run.smithi037.stdout: "crash", 2024-03-29T08:40:25.417 INFO:teuthology.orchestra.run.smithi037.stdout: "osd", 2024-03-29T08:40:25.417 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr", 2024-03-29T08:40:25.417 INFO:teuthology.orchestra.run.smithi037.stdout: "mds", 2024-03-29T08:40:25.417 INFO:teuthology.orchestra.run.smithi037.stdout: "mon" 2024-03-29T08:40:25.418 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-29T08:40:25.418 INFO:teuthology.orchestra.run.smithi037.stdout: "progress": "16/21 daemons upgraded", 2024-03-29T08:40:25.418 INFO:teuthology.orchestra.run.smithi037.stdout: "message": "Currently upgrading mds daemons", 2024-03-29T08:40:25.418 INFO:teuthology.orchestra.run.smithi037.stdout: "is_paused": false 2024-03-29T08:40:25.418 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-29T08:40:25.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:25.419+0000 7f8e7e000700 1 -- 172.21.15.37:0/2872628724 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f8e6006cc20 msgr2=0x7f8e6006f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:40:25.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:25.419+0000 7f8e7e000700 1 --2- 172.21.15.37:0/2872628724 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f8e6006cc20 0x7f8e6006f0d0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f8e6c004f90 tx=0x7f8e6c009e30 comp rx=0 tx=0).stop 2024-03-29T08:40:25.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:25.419+0000 7f8e7e000700 1 -- 172.21.15.37:0/2872628724 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8e781057d0 msgr2=0x7f8e78115f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:40:25.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:25.419+0000 7f8e7e000700 1 --2- 172.21.15.37:0/2872628724 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8e781057d0 0x7f8e78115f70 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f8e64002a50 tx=0x7f8e640046b0 comp rx=0 tx=0).stop 2024-03-29T08:40:25.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:25.419+0000 7f8e7e000700 1 -- 172.21.15.37:0/2872628724 shutdown_connections 2024-03-29T08:40:25.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:25.419+0000 7f8e7e000700 1 --2- 172.21.15.37:0/2872628724 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f8e6006cc20 0x7f8e6006f0d0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:40:25.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:25.419+0000 7f8e7e000700 1 --2- 172.21.15.37:0/2872628724 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8e78104e30 0x7f8e78115a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:40:25.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:25.419+0000 7f8e7e000700 1 --2- 172.21.15.37:0/2872628724 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8e781057d0 0x7f8e78115f70 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:40:25.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:25.419+0000 7f8e7e000700 1 -- 172.21.15.37:0/2872628724 >> 172.21.15.37:0/2872628724 conn(0x7f8e780fab10 msgr2=0x7f8e780fc9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:40:25.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:25.420+0000 7f8e7e000700 1 -- 172.21.15.37:0/2872628724 shutdown_connections 2024-03-29T08:40:25.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:25.420+0000 7f8e7e000700 1 -- 172.21.15.37:0/2872628724 wait complete. 2024-03-29T08:40:25.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:25.557+0000 7fed7cc00700 1 -- 172.21.15.37:0/1339844920 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed78102e00 msgr2=0x7fed781031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:40:25.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:25.557+0000 7fed7cc00700 1 --2- 172.21.15.37:0/1339844920 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed78102e00 0x7fed781031d0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fed68004940 tx=0x7fed68009e30 comp rx=0 tx=0).stop 2024-03-29T08:40:25.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:25.558+0000 7fed7cc00700 1 -- 172.21.15.37:0/1339844920 shutdown_connections 2024-03-29T08:40:25.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:25.558+0000 7fed7cc00700 1 --2- 172.21.15.37:0/1339844920 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fed781037a0 0x7fed780fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:40:25.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:25.558+0000 7fed7cc00700 1 --2- 172.21.15.37:0/1339844920 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed78102e00 0x7fed781031d0 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:40:25.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:25.558+0000 7fed7cc00700 1 -- 172.21.15.37:0/1339844920 >> 172.21.15.37:0/1339844920 conn(0x7fed780f89b0 msgr2=0x7fed780fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:40:25.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:25.558+0000 7fed7cc00700 1 -- 172.21.15.37:0/1339844920 shutdown_connections 2024-03-29T08:40:25.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:25.558+0000 7fed7cc00700 1 -- 172.21.15.37:0/1339844920 wait complete. 2024-03-29T08:40:25.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:25.558+0000 7fed7cc00700 1 Processor -- start 2024-03-29T08:40:25.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:25.559+0000 7fed7cc00700 1 -- start start 2024-03-29T08:40:25.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:25.559+0000 7fed7cc00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed78102e00 0x7fed78115cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:40:25.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:25.559+0000 7fed7cc00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fed781037a0 0x7fed78116230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:40:25.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:25.559+0000 7fed7cc00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed78116880 con 0x7fed78102e00 2024-03-29T08:40:25.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:25.559+0000 7fed7cc00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed7810fc50 con 0x7fed781037a0 2024-03-29T08:40:25.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:25.559+0000 7fed75e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed78102e00 0x7fed78115cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:40:25.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:25.559+0000 7fed75e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed78102e00 0x7fed78115cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58918/0 (socket says 172.21.15.37:58918) 2024-03-29T08:40:25.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:25.559+0000 7fed75e00700 1 -- 172.21.15.37:0/1433511102 learned_addr learned my addr 172.21.15.37:0/1433511102 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:40:25.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:25.560+0000 7fed75400700 1 --2- 172.21.15.37:0/1433511102 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fed781037a0 0x7fed78116230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:40:25.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:25.560+0000 7fed75e00700 1 -- 172.21.15.37:0/1433511102 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fed781037a0 msgr2=0x7fed78116230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:40:25.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:25.560+0000 7fed75e00700 1 --2- 172.21.15.37:0/1433511102 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fed781037a0 0x7fed78116230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:40:25.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:25.560+0000 7fed75e00700 1 -- 172.21.15.37:0/1433511102 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed68005040 con 0x7fed78102e00 2024-03-29T08:40:25.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:25.560+0000 7fed75400700 1 --2- 172.21.15.37:0/1433511102 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fed781037a0 0x7fed78116230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:40:25.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:25.560+0000 7fed75e00700 1 --2- 172.21.15.37:0/1433511102 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed78102e00 0x7fed78115cf0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fed68009360 tx=0x7fed68009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:40:25.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:25.560+0000 7fed6e000700 1 -- 172.21.15.37:0/1433511102 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fed6800e040 con 0x7fed78102e00 2024-03-29T08:40:25.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:25.560+0000 7fed6e000700 1 -- 172.21.15.37:0/1433511102 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fed68013420 con 0x7fed78102e00 2024-03-29T08:40:25.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:25.560+0000 7fed6e000700 1 -- 172.21.15.37:0/1433511102 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fed6801c5f0 con 0x7fed78102e00 2024-03-29T08:40:25.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:25.560+0000 7fed7cc00700 1 -- 172.21.15.37:0/1433511102 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed7810fed0 con 0x7fed78102e00 2024-03-29T08:40:25.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:25.560+0000 7fed7cc00700 1 -- 172.21.15.37:0/1433511102 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed78110490 con 0x7fed78102e00 2024-03-29T08:40:25.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:25.561+0000 7fed7cc00700 1 -- 172.21.15.37:0/1433511102 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed780fe550 con 0x7fed78102e00 2024-03-29T08:40:25.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:25.562+0000 7fed6e000700 1 -- 172.21.15.37:0/1433511102 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fed68013590 con 0x7fed78102e00 2024-03-29T08:40:25.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:25.563+0000 7fed6e000700 1 --2- 172.21.15.37:0/1433511102 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fed5c06cc30 0x7fed5c06f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:40:25.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:25.563+0000 7fed6e000700 1 -- 172.21.15.37:0/1433511102 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fed6808d240 con 0x7fed78102e00 2024-03-29T08:40:25.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:25.564+0000 7fed75400700 1 --2- 172.21.15.37:0/1433511102 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fed5c06cc30 0x7fed5c06f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:40:25.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:25.564+0000 7fed75400700 1 --2- 172.21.15.37:0/1433511102 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fed5c06cc30 0x7fed5c06f0e0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fed64007fd0 tx=0x7fed64009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:40:25.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:25.568+0000 7fed6e000700 1 -- 172.21.15.37:0/1433511102 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fed6805ad10 con 0x7fed78102e00 2024-03-29T08:40:25.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:40:25 smithi148 ceph-mon[106692]: from='client.35606 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:40:25.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:40:25 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/736034585' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:40:25.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:40:25 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:40:25.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:40:25 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/1868440670' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T08:40:25.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:25.870+0000 7fed7cc00700 1 -- 172.21.15.37:0/1433511102 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fed7804bed0 con 0x7fed78102e00 2024-03-29T08:40:25.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:25.870+0000 7fed6e000700 1 -- 172.21.15.37:0/1433511102 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+3583 (secure 0 0 0) 0x7fed6805a8a0 con 0x7fed78102e00 2024-03-29T08:40:25.870 INFO:teuthology.orchestra.run.smithi037.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem is offline 2024-03-29T08:40:25.870 INFO:teuthology.orchestra.run.smithi037.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-03-29T08:40:25.870 INFO:teuthology.orchestra.run.smithi037.stdout: host smithi148 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/mon.smithi148/config 2024-03-29T08:40:25.870 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/podman: timeout after 895 seconds 2024-03-29T08:40:25.870 INFO:teuthology.orchestra.run.smithi037.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59 -e NODE_NAME=smithi148 -e CEPH_USE_RANDOM_NONCE=1 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4:/var/run/ceph:z -v /var/log/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4:/var/log/ceph:z -v /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpkx2rz2d4:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59 inventory --format=json-pretty --filter-for-batch 2024-03-29T08:40:25.871 INFO:teuthology.orchestra.run.smithi037.stdout:Traceback (most recent call last): 2024-03-29T08:40:25.871 INFO:teuthology.orchestra.run.smithi037.stdout: File "/usr/lib64/python3.6/runpy.py", line 193, in _run_module_as_main 2024-03-29T08:40:25.871 INFO:teuthology.orchestra.run.smithi037.stdout: "__main__", mod_spec) 2024-03-29T08:40:25.871 INFO:teuthology.orchestra.run.smithi037.stdout: File "/usr/lib64/python3.6/runpy.py", line 85, in _run_code 2024-03-29T08:40:25.871 INFO:teuthology.orchestra.run.smithi037.stdout: exec(code, run_globals) 2024-03-29T08:40:25.871 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 10889, in 2024-03-29T08:40:25.871 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 10877, in main 2024-03-29T08:40:25.871 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 2576, in _infer_config 2024-03-29T08:40:25.871 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 2492, in _infer_fsid 2024-03-29T08:40:25.871 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 2604, in _infer_image 2024-03-29T08:40:25.871 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 2479, in _validate_fsid 2024-03-29T08:40:25.871 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 7145, in command_ceph_volume 2024-03-29T08:40:25.871 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 2267, in call_throws 2024-03-29T08:40:25.871 INFO:teuthology.orchestra.run.smithi037.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59 -e NODE_NAME=smithi148 -e CEPH_USE_RANDOM_NONCE=1 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4:/var/run/ceph:z -v /var/log/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4:/var/log/ceph:z -v /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpkx2rz2d4:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59 inventory --format=json-pretty --filter-for-batch 2024-03-29T08:40:25.871 INFO:teuthology.orchestra.run.smithi037.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-03-29T08:40:25.872 INFO:teuthology.orchestra.run.smithi037.stdout: fs cephfs is degraded 2024-03-29T08:40:25.872 INFO:teuthology.orchestra.run.smithi037.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-03-29T08:40:25.872 INFO:teuthology.orchestra.run.smithi037.stdout: fs cephfs is offline because no MDS is active for it. 2024-03-29T08:40:25.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:25.872+0000 7fed7cc00700 1 -- 172.21.15.37:0/1433511102 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fed5c06cc30 msgr2=0x7fed5c06f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:40:25.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:25.872+0000 7fed7cc00700 1 --2- 172.21.15.37:0/1433511102 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fed5c06cc30 0x7fed5c06f0e0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fed64007fd0 tx=0x7fed64009040 comp rx=0 tx=0).stop 2024-03-29T08:40:25.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:25.872+0000 7fed7cc00700 1 -- 172.21.15.37:0/1433511102 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed78102e00 msgr2=0x7fed78115cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:40:25.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:25.872+0000 7fed7cc00700 1 --2- 172.21.15.37:0/1433511102 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed78102e00 0x7fed78115cf0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fed68009360 tx=0x7fed68009e30 comp rx=0 tx=0).stop 2024-03-29T08:40:25.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:25.873+0000 7fed7cc00700 1 -- 172.21.15.37:0/1433511102 shutdown_connections 2024-03-29T08:40:25.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:25.873+0000 7fed7cc00700 1 --2- 172.21.15.37:0/1433511102 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fed5c06cc30 0x7fed5c06f0e0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:40:25.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:25.873+0000 7fed7cc00700 1 --2- 172.21.15.37:0/1433511102 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed78102e00 0x7fed78115cf0 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:40:25.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:25.873+0000 7fed7cc00700 1 --2- 172.21.15.37:0/1433511102 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fed781037a0 0x7fed78116230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:40:25.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:25.873+0000 7fed7cc00700 1 -- 172.21.15.37:0/1433511102 >> 172.21.15.37:0/1433511102 conn(0x7fed780f89b0 msgr2=0x7fed780706d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:40:25.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:25.873+0000 7fed7cc00700 1 -- 172.21.15.37:0/1433511102 shutdown_connections 2024-03-29T08:40:25.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:25.873+0000 7fed7cc00700 1 -- 172.21.15.37:0/1433511102 wait complete. 2024-03-29T08:40:25.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:40:25 smithi037 ceph-mon[142325]: from='client.35606 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:40:25.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:40:25 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/736034585' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:40:25.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:40:25 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:40:25.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:40:25 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/1868440670' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T08:40:26.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:40:26 smithi148 ceph-mon[106692]: pgmap v887: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:40:26.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:40:26 smithi148 ceph-mon[106692]: from='client.35618 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:40:26.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:40:26 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/1433511102' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T08:40:26.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:40:26 smithi037 ceph-mon[142325]: pgmap v887: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:40:26.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:40:26 smithi037 ceph-mon[142325]: from='client.35618 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:40:26.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:40:26 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/1433511102' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T08:40:28.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:40:28 smithi148 ceph-mon[106692]: pgmap v888: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:40:28.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:40:28 smithi037 ceph-mon[142325]: pgmap v888: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:40:30.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:40:30 smithi148 ceph-mon[106692]: pgmap v889: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:40:30.822 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:40:30 smithi037 ceph-mon[142325]: pgmap v889: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:40:32.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:40:32 smithi148 ceph-mon[106692]: pgmap v890: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:40:32.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:40:32 smithi037 ceph-mon[142325]: pgmap v890: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:40:34.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:40:34 smithi148 ceph-mon[106692]: pgmap v891: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:40:34.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:40:34 smithi037 ceph-mon[142325]: pgmap v891: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:40:36.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:40:36 smithi148 ceph-mon[106692]: pgmap v892: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:40:36.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:40:36 smithi037 ceph-mon[142325]: pgmap v892: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:40:38.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:40:38 smithi148 ceph-mon[106692]: pgmap v893: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:40:38.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:40:38 smithi037 ceph-mon[142325]: pgmap v893: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:40:40.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:40:40 smithi148 ceph-mon[106692]: pgmap v894: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:40:40.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:40:40 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:40:40.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:40:40 smithi037 ceph-mon[142325]: pgmap v894: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:40:40.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:40:40 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:40:42.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:40:42 smithi037 ceph-mon[142325]: pgmap v895: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:40:43.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:40:42 smithi148 ceph-mon[106692]: pgmap v895: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:40:44.928 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:40:44 smithi037 ceph-mon[142325]: pgmap v896: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:40:45.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:40:44 smithi148 ceph-mon[106692]: pgmap v896: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:40:46.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:40:46 smithi037 ceph-mon[142325]: pgmap v897: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:40:47.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:40:46 smithi148 ceph-mon[106692]: pgmap v897: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:40:48.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:40:48 smithi037 ceph-mon[142325]: pgmap v898: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:40:49.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:40:48 smithi148 ceph-mon[106692]: pgmap v898: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:40:50.929 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:40:50 smithi037 ceph-mon[142325]: pgmap v899: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:40:51.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:40:50 smithi148 ceph-mon[106692]: pgmap v899: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:40:52.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:40:52 smithi037 ceph-mon[142325]: pgmap v900: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:40:53.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:40:52 smithi148 ceph-mon[106692]: pgmap v900: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:40:54.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:40:54 smithi037 ceph-mon[142325]: pgmap v901: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:40:55.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:40:54 smithi148 ceph-mon[106692]: pgmap v901: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:40:55.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:40:55 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:40:56.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:56.027+0000 7f3603000700 1 -- 172.21.15.37:0/3763056431 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f35fc1037a0 msgr2=0x7f35fc0fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:40:56.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:56.027+0000 7f3603000700 1 --2- 172.21.15.37:0/3763056431 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f35fc1037a0 0x7f35fc0fce10 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f35e8004940 tx=0x7f35e8009e30 comp rx=0 tx=0).stop 2024-03-29T08:40:56.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:56.027+0000 7f3603000700 1 -- 172.21.15.37:0/3763056431 shutdown_connections 2024-03-29T08:40:56.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:56.027+0000 7f3603000700 1 --2- 172.21.15.37:0/3763056431 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f35fc1037a0 0x7f35fc0fce10 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:40:56.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:56.027+0000 7f3603000700 1 --2- 172.21.15.37:0/3763056431 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f35fc102e00 0x7f35fc1031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:40:56.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:56.027+0000 7f3603000700 1 -- 172.21.15.37:0/3763056431 >> 172.21.15.37:0/3763056431 conn(0x7f35fc0f89b0 msgr2=0x7f35fc0fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:40:56.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:56.027+0000 7f3603000700 1 -- 172.21.15.37:0/3763056431 shutdown_connections 2024-03-29T08:40:56.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:56.027+0000 7f3603000700 1 -- 172.21.15.37:0/3763056431 wait complete. 2024-03-29T08:40:56.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:56.028+0000 7f3603000700 1 Processor -- start 2024-03-29T08:40:56.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:56.028+0000 7f3603000700 1 -- start start 2024-03-29T08:40:56.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:56.028+0000 7f3603000700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f35fc102e00 0x7f35fc115c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:40:56.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:56.028+0000 7f3603000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f35fc1037a0 0x7f35fc116190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:40:56.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:56.028+0000 7f3603000700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35fc1167e0 con 0x7f35fc1037a0 2024-03-29T08:40:56.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:56.028+0000 7f3603000700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35fc10fbb0 con 0x7f35fc102e00 2024-03-29T08:40:56.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:56.029+0000 7f35fb400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f35fc1037a0 0x7f35fc116190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:40:56.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:56.029+0000 7f35fb400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f35fc1037a0 0x7f35fc116190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38232/0 (socket says 172.21.15.37:38232) 2024-03-29T08:40:56.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:56.029+0000 7f35fb400700 1 -- 172.21.15.37:0/963608227 learned_addr learned my addr 172.21.15.37:0/963608227 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:40:56.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:56.029+0000 7f35fbe00700 1 --2- 172.21.15.37:0/963608227 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f35fc102e00 0x7f35fc115c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:40:56.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:56.029+0000 7f35fb400700 1 -- 172.21.15.37:0/963608227 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f35fc102e00 msgr2=0x7f35fc115c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:40:56.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:56.029+0000 7f35fb400700 1 --2- 172.21.15.37:0/963608227 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f35fc102e00 0x7f35fc115c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:40:56.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:56.029+0000 7f35fb400700 1 -- 172.21.15.37:0/963608227 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35e8005040 con 0x7f35fc1037a0 2024-03-29T08:40:56.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:56.030+0000 7f35fb400700 1 --2- 172.21.15.37:0/963608227 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f35fc1037a0 0x7f35fc116190 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f35e8009fd0 tx=0x7f35e8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:40:56.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:56.030+0000 7f35f3e00700 1 -- 172.21.15.37:0/963608227 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f35e800e030 con 0x7f35fc1037a0 2024-03-29T08:40:56.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:56.030+0000 7f35f3e00700 1 -- 172.21.15.37:0/963608227 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f35e80093b0 con 0x7f35fc1037a0 2024-03-29T08:40:56.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:56.030+0000 7f3603000700 1 -- 172.21.15.37:0/963608227 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35fc10fe30 con 0x7f35fc1037a0 2024-03-29T08:40:56.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:56.030+0000 7f35f3e00700 1 -- 172.21.15.37:0/963608227 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f35e801cc80 con 0x7f35fc1037a0 2024-03-29T08:40:56.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:56.030+0000 7f3603000700 1 -- 172.21.15.37:0/963608227 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35fc1103f0 con 0x7f35fc1037a0 2024-03-29T08:40:56.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:56.031+0000 7f3603000700 1 -- 172.21.15.37:0/963608227 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35fc0fe550 con 0x7f35fc1037a0 2024-03-29T08:40:56.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:56.032+0000 7f35f3e00700 1 -- 172.21.15.37:0/963608227 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f35e801c400 con 0x7f35fc1037a0 2024-03-29T08:40:56.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:56.033+0000 7f35f3e00700 1 --2- 172.21.15.37:0/963608227 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f35e006cc30 0x7f35e006f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:40:56.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:56.033+0000 7f35f3e00700 1 -- 172.21.15.37:0/963608227 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f35e808d890 con 0x7f35fc1037a0 2024-03-29T08:40:56.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:56.033+0000 7f35fbe00700 1 --2- 172.21.15.37:0/963608227 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f35e006cc30 0x7f35e006f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:40:56.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:56.034+0000 7f35fbe00700 1 --2- 172.21.15.37:0/963608227 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f35e006cc30 0x7f35e006f0e0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f35ec009b80 tx=0x7f35ec00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:40:56.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:56.038+0000 7f35f3e00700 1 -- 172.21.15.37:0/963608227 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f35e805b2e0 con 0x7f35fc1037a0 2024-03-29T08:40:56.055 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:40:55 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:40:56.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:56.271+0000 7f3603000700 1 -- 172.21.15.37:0/963608227 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f35fc071160 con 0x7f35e006cc30 2024-03-29T08:40:56.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:56.273+0000 7f35f3e00700 1 -- 172.21.15.37:0/963608227 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f35fc071160 con 0x7f35e006cc30 2024-03-29T08:40:56.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:56.275+0000 7f3603000700 1 -- 172.21.15.37:0/963608227 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f35e006cc30 msgr2=0x7f35e006f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:40:56.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:56.275+0000 7f3603000700 1 --2- 172.21.15.37:0/963608227 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f35e006cc30 0x7f35e006f0e0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f35ec009b80 tx=0x7f35ec00b040 comp rx=0 tx=0).stop 2024-03-29T08:40:56.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:56.275+0000 7f3603000700 1 -- 172.21.15.37:0/963608227 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f35fc1037a0 msgr2=0x7f35fc116190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:40:56.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:56.275+0000 7f3603000700 1 --2- 172.21.15.37:0/963608227 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f35fc1037a0 0x7f35fc116190 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f35e8009fd0 tx=0x7f35e8009e30 comp rx=0 tx=0).stop 2024-03-29T08:40:56.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:56.275+0000 7f3603000700 1 -- 172.21.15.37:0/963608227 shutdown_connections 2024-03-29T08:40:56.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:56.275+0000 7f3603000700 1 --2- 172.21.15.37:0/963608227 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f35e006cc30 0x7f35e006f0e0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:40:56.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:56.275+0000 7f3603000700 1 --2- 172.21.15.37:0/963608227 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f35fc102e00 0x7f35fc115c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:40:56.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:56.275+0000 7f3603000700 1 --2- 172.21.15.37:0/963608227 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f35fc1037a0 0x7f35fc116190 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:40:56.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:56.275+0000 7f3603000700 1 -- 172.21.15.37:0/963608227 >> 172.21.15.37:0/963608227 conn(0x7f35fc0f89b0 msgr2=0x7f35fc1009c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:40:56.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:56.275+0000 7f3603000700 1 -- 172.21.15.37:0/963608227 shutdown_connections 2024-03-29T08:40:56.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:56.275+0000 7f3603000700 1 -- 172.21.15.37:0/963608227 wait complete. 2024-03-29T08:40:56.290 INFO:teuthology.orchestra.run.smithi037.stdout:true 2024-03-29T08:40:56.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:56.415+0000 7fa4ea200700 1 -- 172.21.15.37:0/304518969 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa4e40fde90 msgr2=0x7fa4e40fe260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:40:56.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:56.415+0000 7fa4ea200700 1 --2- 172.21.15.37:0/304518969 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa4e40fde90 0x7fa4e40fe260 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fa4d4004940 tx=0x7fa4d4009e30 comp rx=0 tx=0).stop 2024-03-29T08:40:56.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:56.415+0000 7fa4ea200700 1 -- 172.21.15.37:0/304518969 shutdown_connections 2024-03-29T08:40:56.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:56.415+0000 7fa4ea200700 1 --2- 172.21.15.37:0/304518969 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa4e40fe830 0x7fa4e4106f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:40:56.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:56.415+0000 7fa4ea200700 1 --2- 172.21.15.37:0/304518969 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa4e40fde90 0x7fa4e40fe260 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:40:56.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:56.415+0000 7fa4ea200700 1 -- 172.21.15.37:0/304518969 >> 172.21.15.37:0/304518969 conn(0x7fa4e4072bc0 msgr2=0x7fa4e4072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:40:56.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:56.415+0000 7fa4ea200700 1 -- 172.21.15.37:0/304518969 shutdown_connections 2024-03-29T08:40:56.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:56.415+0000 7fa4ea200700 1 -- 172.21.15.37:0/304518969 wait complete. 2024-03-29T08:40:56.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:56.416+0000 7fa4ea200700 1 Processor -- start 2024-03-29T08:40:56.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:56.416+0000 7fa4ea200700 1 -- start start 2024-03-29T08:40:56.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:56.416+0000 7fa4ea200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa4e40fde90 0x7fa4e4113a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:40:56.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:56.416+0000 7fa4ea200700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa4e40fe830 0x7fa4e4113fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:40:56.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:56.416+0000 7fa4ea200700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4e4114600 con 0x7fa4e40fde90 2024-03-29T08:40:56.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:56.416+0000 7fa4ea200700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4e410d9f0 con 0x7fa4e40fe830 2024-03-29T08:40:56.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:56.416+0000 7fa4e3400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa4e40fde90 0x7fa4e4113a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:40:56.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:56.416+0000 7fa4e3400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa4e40fde90 0x7fa4e4113a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38244/0 (socket says 172.21.15.37:38244) 2024-03-29T08:40:56.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:56.416+0000 7fa4e3400700 1 -- 172.21.15.37:0/2377843852 learned_addr learned my addr 172.21.15.37:0/2377843852 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:40:56.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:56.417+0000 7fa4e2a00700 1 --2- 172.21.15.37:0/2377843852 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa4e40fe830 0x7fa4e4113fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:40:56.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:56.417+0000 7fa4e3400700 1 -- 172.21.15.37:0/2377843852 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa4e40fe830 msgr2=0x7fa4e4113fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:40:56.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:56.417+0000 7fa4e3400700 1 --2- 172.21.15.37:0/2377843852 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa4e40fe830 0x7fa4e4113fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:40:56.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:56.417+0000 7fa4e3400700 1 -- 172.21.15.37:0/2377843852 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4d4005040 con 0x7fa4e40fde90 2024-03-29T08:40:56.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:56.417+0000 7fa4e2a00700 1 --2- 172.21.15.37:0/2377843852 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa4e40fe830 0x7fa4e4113fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:40:56.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:56.417+0000 7fa4e3400700 1 --2- 172.21.15.37:0/2377843852 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa4e40fde90 0x7fa4e4113a70 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fa4d4000c00 tx=0x7fa4d4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:40:56.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:56.417+0000 7fa4db400700 1 -- 172.21.15.37:0/2377843852 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa4d4021070 con 0x7fa4e40fde90 2024-03-29T08:40:56.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:56.417+0000 7fa4db400700 1 -- 172.21.15.37:0/2377843852 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fa4d4013e00 con 0x7fa4e40fde90 2024-03-29T08:40:56.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:56.417+0000 7fa4db400700 1 -- 172.21.15.37:0/2377843852 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa4d400e070 con 0x7fa4e40fde90 2024-03-29T08:40:56.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:56.417+0000 7fa4ea200700 1 -- 172.21.15.37:0/2377843852 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4e410dc70 con 0x7fa4e40fde90 2024-03-29T08:40:56.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:56.418+0000 7fa4ea200700 1 -- 172.21.15.37:0/2377843852 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4e410e1b0 con 0x7fa4e40fde90 2024-03-29T08:40:56.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:56.419+0000 7fa4ea200700 1 -- 172.21.15.37:0/2377843852 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4e4065e30 con 0x7fa4e40fde90 2024-03-29T08:40:56.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:56.420+0000 7fa4db400700 1 -- 172.21.15.37:0/2377843852 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fa4d4013950 con 0x7fa4e40fde90 2024-03-29T08:40:56.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:56.420+0000 7fa4db400700 1 --2- 172.21.15.37:0/2377843852 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fa4c806cc30 0x7fa4c806f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:40:56.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:56.421+0000 7fa4db400700 1 -- 172.21.15.37:0/2377843852 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fa4d408d120 con 0x7fa4e40fde90 2024-03-29T08:40:56.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:56.421+0000 7fa4e2a00700 1 --2- 172.21.15.37:0/2377843852 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fa4c806cc30 0x7fa4c806f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:40:56.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:56.421+0000 7fa4e2a00700 1 --2- 172.21.15.37:0/2377843852 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fa4c806cc30 0x7fa4c806f0e0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fa4d0004600 tx=0x7fa4d000b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:40:56.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:56.425+0000 7fa4db400700 1 -- 172.21.15.37:0/2377843852 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fa4d405ab70 con 0x7fa4e40fde90 2024-03-29T08:40:56.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:56.656+0000 7fa4ea200700 1 -- 172.21.15.37:0/2377843852 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa4e4109dc0 con 0x7fa4c806cc30 2024-03-29T08:40:56.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:56.657+0000 7fa4db400700 1 -- 172.21.15.37:0/2377843852 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fa4e4109dc0 con 0x7fa4c806cc30 2024-03-29T08:40:56.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:56.659+0000 7fa4ea200700 1 -- 172.21.15.37:0/2377843852 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fa4c806cc30 msgr2=0x7fa4c806f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:40:56.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:56.659+0000 7fa4ea200700 1 --2- 172.21.15.37:0/2377843852 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fa4c806cc30 0x7fa4c806f0e0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fa4d0004600 tx=0x7fa4d000b040 comp rx=0 tx=0).stop 2024-03-29T08:40:56.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:56.660+0000 7fa4ea200700 1 -- 172.21.15.37:0/2377843852 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa4e40fde90 msgr2=0x7fa4e4113a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:40:56.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:56.660+0000 7fa4ea200700 1 --2- 172.21.15.37:0/2377843852 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa4e40fde90 0x7fa4e4113a70 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fa4d4000c00 tx=0x7fa4d4009e30 comp rx=0 tx=0).stop 2024-03-29T08:40:56.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:56.660+0000 7fa4ea200700 1 -- 172.21.15.37:0/2377843852 shutdown_connections 2024-03-29T08:40:56.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:56.660+0000 7fa4ea200700 1 --2- 172.21.15.37:0/2377843852 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fa4c806cc30 0x7fa4c806f0e0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:40:56.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:56.660+0000 7fa4ea200700 1 --2- 172.21.15.37:0/2377843852 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa4e40fde90 0x7fa4e4113a70 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:40:56.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:56.660+0000 7fa4ea200700 1 --2- 172.21.15.37:0/2377843852 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa4e40fe830 0x7fa4e4113fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:40:56.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:56.660+0000 7fa4ea200700 1 -- 172.21.15.37:0/2377843852 >> 172.21.15.37:0/2377843852 conn(0x7fa4e4072bc0 msgr2=0x7fa4e41037c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:40:56.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:56.660+0000 7fa4ea200700 1 -- 172.21.15.37:0/2377843852 shutdown_connections 2024-03-29T08:40:56.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:56.660+0000 7fa4ea200700 1 -- 172.21.15.37:0/2377843852 wait complete. 2024-03-29T08:40:56.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:56.797+0000 7f6817000700 1 -- 172.21.15.37:0/2745614668 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6810102e00 msgr2=0x7f68101031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:40:56.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:56.797+0000 7f6817000700 1 --2- 172.21.15.37:0/2745614668 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6810102e00 0x7f68101031d0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f6800004940 tx=0x7f6800009e30 comp rx=0 tx=0).stop 2024-03-29T08:40:56.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:56.797+0000 7f6817000700 1 -- 172.21.15.37:0/2745614668 shutdown_connections 2024-03-29T08:40:56.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:56.797+0000 7f6817000700 1 --2- 172.21.15.37:0/2745614668 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f68101037a0 0x7f68100fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:40:56.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:56.797+0000 7f6817000700 1 --2- 172.21.15.37:0/2745614668 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6810102e00 0x7f68101031d0 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:40:56.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:56.797+0000 7f6817000700 1 -- 172.21.15.37:0/2745614668 >> 172.21.15.37:0/2745614668 conn(0x7f68100f89b0 msgr2=0x7f68100fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:40:56.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:56.797+0000 7f6817000700 1 -- 172.21.15.37:0/2745614668 shutdown_connections 2024-03-29T08:40:56.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:56.797+0000 7f6817000700 1 -- 172.21.15.37:0/2745614668 wait complete. 2024-03-29T08:40:56.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:56.798+0000 7f6817000700 1 Processor -- start 2024-03-29T08:40:56.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:56.798+0000 7f6817000700 1 -- start start 2024-03-29T08:40:56.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:56.798+0000 7f6817000700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6810102e00 0x7f6810113b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:40:56.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:56.798+0000 7f6817000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f68101037a0 0x7f6810114060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:40:56.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:56.798+0000 7f6817000700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68101146b0 con 0x7f68101037a0 2024-03-29T08:40:56.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:56.798+0000 7f6817000700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f681010daa0 con 0x7f6810102e00 2024-03-29T08:40:56.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:56.798+0000 7f680f400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f68101037a0 0x7f6810114060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:40:56.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:56.798+0000 7f680f400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f68101037a0 0x7f6810114060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38250/0 (socket says 172.21.15.37:38250) 2024-03-29T08:40:56.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:56.798+0000 7f680f400700 1 -- 172.21.15.37:0/2848290342 learned_addr learned my addr 172.21.15.37:0/2848290342 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:40:56.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:56.799+0000 7f680fe00700 1 --2- 172.21.15.37:0/2848290342 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6810102e00 0x7f6810113b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:40:56.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:56.799+0000 7f680f400700 1 -- 172.21.15.37:0/2848290342 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6810102e00 msgr2=0x7f6810113b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:40:56.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:56.799+0000 7f680f400700 1 --2- 172.21.15.37:0/2848290342 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6810102e00 0x7f6810113b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:40:56.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:56.799+0000 7f680f400700 1 -- 172.21.15.37:0/2848290342 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6800005040 con 0x7f68101037a0 2024-03-29T08:40:56.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:56.799+0000 7f680f400700 1 --2- 172.21.15.37:0/2848290342 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f68101037a0 0x7f6810114060 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f67fc002a50 tx=0x7f67fc004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:40:56.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:56.799+0000 7f6807e00700 1 -- 172.21.15.37:0/2848290342 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f67fc007cc0 con 0x7f68101037a0 2024-03-29T08:40:56.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:56.799+0000 7f6807e00700 1 -- 172.21.15.37:0/2848290342 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f67fc00f040 con 0x7f68101037a0 2024-03-29T08:40:56.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:56.799+0000 7f6817000700 1 -- 172.21.15.37:0/2848290342 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f681010dd80 con 0x7f68101037a0 2024-03-29T08:40:56.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:56.799+0000 7f6807e00700 1 -- 172.21.15.37:0/2848290342 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f67fc00d920 con 0x7f68101037a0 2024-03-29T08:40:56.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:56.800+0000 7f6817000700 1 -- 172.21.15.37:0/2848290342 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f681010e340 con 0x7f68101037a0 2024-03-29T08:40:56.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:56.801+0000 7f6817000700 1 -- 172.21.15.37:0/2848290342 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68100fe550 con 0x7f68101037a0 2024-03-29T08:40:56.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:56.802+0000 7f6807e00700 1 -- 172.21.15.37:0/2848290342 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f67fc009680 con 0x7f68101037a0 2024-03-29T08:40:56.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:56.803+0000 7f6807e00700 1 --2- 172.21.15.37:0/2848290342 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f67f406cc30 0x7f67f406f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:40:56.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:56.803+0000 7f6807e00700 1 -- 172.21.15.37:0/2848290342 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f67fc08bd10 con 0x7f68101037a0 2024-03-29T08:40:56.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:56.804+0000 7f680fe00700 1 --2- 172.21.15.37:0/2848290342 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f67f406cc30 0x7f67f406f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:40:56.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:56.804+0000 7f680fe00700 1 --2- 172.21.15.37:0/2848290342 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f67f406cc30 0x7f67f406f0e0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f6800009fd0 tx=0x7f6800009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:40:56.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:56.808+0000 7f6807e00700 1 -- 172.21.15.37:0/2848290342 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f67fc059760 con 0x7f68101037a0 2024-03-29T08:40:56.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:40:56 smithi037 ceph-mon[142325]: pgmap v902: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:40:57.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:57.031+0000 7f6817000700 1 -- 172.21.15.37:0/2848290342 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f6810109dc0 con 0x7f67f406cc30 2024-03-29T08:40:57.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:57.041+0000 7f6807e00700 1 -- 172.21.15.37:0/2848290342 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f6810109dc0 con 0x7f67f406cc30 2024-03-29T08:40:57.041 INFO:teuthology.orchestra.run.smithi037.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-29T08:40:57.041 INFO:teuthology.orchestra.run.smithi037.stdout:alertmanager.smithi037 smithi037 *:9093,9094 running (30m) 7m ago 43m 14.3M - 0.25.0 c8568f914cd2 2f9f9746ed0f 2024-03-29T08:40:57.042 INFO:teuthology.orchestra.run.smithi037.stdout:crash.smithi037 smithi037 running (27m) 7m ago 44m 6647k - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 178d416a3186 2024-03-29T08:40:57.042 INFO:teuthology.orchestra.run.smithi037.stdout:crash.smithi148 smithi148 running (27m) 7m ago 42m 6656k - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 23666989d58f 2024-03-29T08:40:57.042 INFO:teuthology.orchestra.run.smithi037.stdout:grafana.smithi037 smithi037 *:3000 running (30m) 7m ago 43m 74.5M - 9.4.7 954c08fa6188 6b976fe49952 2024-03-29T08:40:57.042 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi037.iifsmh smithi037 running (23m) 7m ago 37m 16.4M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 2d97f562cc5b 2024-03-29T08:40:57.042 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi037.mmwgxe smithi037 running (23m) 7m ago 37m 16.7M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff ec37037f46e8 2024-03-29T08:40:57.042 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi148.qukpjh smithi148 running (7m) 7m ago 37m 10.7M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 3257219bc046 2024-03-29T08:40:57.042 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi148.sqteih smithi148 running (22m) 7m ago 37m 16.3M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 44c15c77037a 2024-03-29T08:40:57.042 INFO:teuthology.orchestra.run.smithi037.stdout:mgr.smithi037.wjikii smithi037 *:8443,9283,8765 running (33m) 7m ago 45m 510M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff c5a1417b9417 2024-03-29T08:40:57.042 INFO:teuthology.orchestra.run.smithi037.stdout:mgr.smithi148.onhgvm smithi148 *:8443,9283,8765 running (31m) 7m ago 42m 456M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff fcfdbce062bd 2024-03-29T08:40:57.042 INFO:teuthology.orchestra.run.smithi037.stdout:mon.smithi037 smithi037 running (28m) 7m ago 45m 70.2M 2048M 18.2.2-1015-gd0e392d9 1cc2f45c17ff eea2060783af 2024-03-29T08:40:57.042 INFO:teuthology.orchestra.run.smithi037.stdout:mon.smithi148 smithi148 running (28m) 7m ago 42m 50.4M 2048M 18.2.2-1015-gd0e392d9 1cc2f45c17ff ad4b2c5748be 2024-03-29T08:40:57.042 INFO:teuthology.orchestra.run.smithi037.stdout:node-exporter.smithi037 smithi037 *:9100 running (31m) 7m ago 43m 8514k - 1.5.0 0da6a335fe13 26b8ae9cb878 2024-03-29T08:40:57.043 INFO:teuthology.orchestra.run.smithi037.stdout:node-exporter.smithi148 smithi148 *:9100 running (31m) 7m ago 42m 8464k - 1.5.0 0da6a335fe13 98414e4e64ed 2024-03-29T08:40:57.043 INFO:teuthology.orchestra.run.smithi037.stdout:osd.0 smithi037 running (27m) 7m ago 41m 108M 1619M 18.2.2-1015-gd0e392d9 1cc2f45c17ff c0c584f6245c 2024-03-29T08:40:57.043 INFO:teuthology.orchestra.run.smithi037.stdout:osd.1 smithi037 running (26m) 7m ago 41m 104M 1619M 18.2.2-1015-gd0e392d9 1cc2f45c17ff a4a2ff4cc83d 2024-03-29T08:40:57.043 INFO:teuthology.orchestra.run.smithi037.stdout:osd.2 smithi037 running (25m) 7m ago 40m 85.9M 1619M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 8b1bc71aa79e 2024-03-29T08:40:57.043 INFO:teuthology.orchestra.run.smithi037.stdout:osd.3 smithi148 running (25m) 7m ago 40m 109M 2642M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 1c656ab8ff86 2024-03-29T08:40:57.043 INFO:teuthology.orchestra.run.smithi037.stdout:osd.4 smithi148 running (24m) 7m ago 39m 101M 2642M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 966a9de11d22 2024-03-29T08:40:57.043 INFO:teuthology.orchestra.run.smithi037.stdout:osd.5 smithi148 running (23m) 7m ago 38m 89.4M 2642M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 474c4f9212ce 2024-03-29T08:40:57.043 INFO:teuthology.orchestra.run.smithi037.stdout:prometheus.smithi037 smithi037 *:9095 running (31m) 7m ago 43m 61.0M - 2.43.0 a07b618ecd1d 30c0223ec589 2024-03-29T08:40:57.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:57.044+0000 7f6817000700 1 -- 172.21.15.37:0/2848290342 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f67f406cc30 msgr2=0x7f67f406f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:40:57.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:57.044+0000 7f6817000700 1 --2- 172.21.15.37:0/2848290342 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f67f406cc30 0x7f67f406f0e0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f6800009fd0 tx=0x7f6800009e30 comp rx=0 tx=0).stop 2024-03-29T08:40:57.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:57.044+0000 7f6817000700 1 -- 172.21.15.37:0/2848290342 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f68101037a0 msgr2=0x7f6810114060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:40:57.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:57.044+0000 7f6817000700 1 --2- 172.21.15.37:0/2848290342 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f68101037a0 0x7f6810114060 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f67fc002a50 tx=0x7f67fc004600 comp rx=0 tx=0).stop 2024-03-29T08:40:57.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:57.044+0000 7f6817000700 1 -- 172.21.15.37:0/2848290342 shutdown_connections 2024-03-29T08:40:57.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:57.044+0000 7f6817000700 1 --2- 172.21.15.37:0/2848290342 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f67f406cc30 0x7f67f406f0e0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:40:57.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:57.044+0000 7f6817000700 1 --2- 172.21.15.37:0/2848290342 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6810102e00 0x7f6810113b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:40:57.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:57.044+0000 7f6817000700 1 --2- 172.21.15.37:0/2848290342 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f68101037a0 0x7f6810114060 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:40:57.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:57.044+0000 7f6817000700 1 -- 172.21.15.37:0/2848290342 >> 172.21.15.37:0/2848290342 conn(0x7f68100f89b0 msgr2=0x7f68100f9a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:40:57.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:57.044+0000 7f6817000700 1 -- 172.21.15.37:0/2848290342 shutdown_connections 2024-03-29T08:40:57.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:57.044+0000 7f6817000700 1 -- 172.21.15.37:0/2848290342 wait complete. 2024-03-29T08:40:57.055 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:40:56 smithi148 ceph-mon[106692]: pgmap v902: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:40:57.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:57.181+0000 7f08be000700 1 -- 172.21.15.37:0/1743999626 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f08b8070300 msgr2=0x7f08b80706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:40:57.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:57.181+0000 7f08be000700 1 --2- 172.21.15.37:0/1743999626 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f08b8070300 0x7f08b80706d0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f08a8004970 tx=0x7f08a8009e30 comp rx=0 tx=0).stop 2024-03-29T08:40:57.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:57.181+0000 7f08be000700 1 -- 172.21.15.37:0/1743999626 shutdown_connections 2024-03-29T08:40:57.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:57.181+0000 7f08be000700 1 --2- 172.21.15.37:0/1743999626 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f08b8070c10 0x7f08b8109190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:40:57.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:57.181+0000 7f08be000700 1 --2- 172.21.15.37:0/1743999626 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f08b8070300 0x7f08b80706d0 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:40:57.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:57.181+0000 7f08be000700 1 -- 172.21.15.37:0/1743999626 >> 172.21.15.37:0/1743999626 conn(0x7f08b80f8970 msgr2=0x7f08b80fad80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:40:57.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:57.181+0000 7f08be000700 1 -- 172.21.15.37:0/1743999626 shutdown_connections 2024-03-29T08:40:57.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:57.181+0000 7f08be000700 1 -- 172.21.15.37:0/1743999626 wait complete. 2024-03-29T08:40:57.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:57.182+0000 7f08be000700 1 Processor -- start 2024-03-29T08:40:57.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:57.182+0000 7f08be000700 1 -- start start 2024-03-29T08:40:57.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:57.183+0000 7f08be000700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f08b8070300 0x7f08b8115c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:40:57.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:57.183+0000 7f08be000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f08b8070c10 0x7f08b81161d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:40:57.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:57.183+0000 7f08be000700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08b8116820 con 0x7f08b8070c10 2024-03-29T08:40:57.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:57.183+0000 7f08be000700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08b810fc30 con 0x7f08b8070300 2024-03-29T08:40:57.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:57.183+0000 7f08b6a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f08b8070c10 0x7f08b81161d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:40:57.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:57.183+0000 7f08b6a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f08b8070c10 0x7f08b81161d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38272/0 (socket says 172.21.15.37:38272) 2024-03-29T08:40:57.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:57.183+0000 7f08b6a00700 1 -- 172.21.15.37:0/2147674579 learned_addr learned my addr 172.21.15.37:0/2147674579 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:40:57.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:57.183+0000 7f08b7400700 1 --2- 172.21.15.37:0/2147674579 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f08b8070300 0x7f08b8115c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:40:57.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:57.183+0000 7f08b6a00700 1 -- 172.21.15.37:0/2147674579 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f08b8070300 msgr2=0x7f08b8115c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:40:57.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:57.183+0000 7f08b6a00700 1 --2- 172.21.15.37:0/2147674579 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f08b8070300 0x7f08b8115c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:40:57.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:57.183+0000 7f08b6a00700 1 -- 172.21.15.37:0/2147674579 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08a8005040 con 0x7f08b8070c10 2024-03-29T08:40:57.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:57.184+0000 7f08b6a00700 1 --2- 172.21.15.37:0/2147674579 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f08b8070c10 0x7f08b81161d0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f08a4002a50 tx=0x7f08a40046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:40:57.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:57.184+0000 7f08af400700 1 -- 172.21.15.37:0/2147674579 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f08a4007cc0 con 0x7f08b8070c10 2024-03-29T08:40:57.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:57.184+0000 7f08af400700 1 -- 172.21.15.37:0/2147674579 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f08a400f040 con 0x7f08b8070c10 2024-03-29T08:40:57.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:57.184+0000 7f08be000700 1 -- 172.21.15.37:0/2147674579 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08b810ff10 con 0x7f08b8070c10 2024-03-29T08:40:57.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:57.184+0000 7f08af400700 1 -- 172.21.15.37:0/2147674579 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f08a400d9c0 con 0x7f08b8070c10 2024-03-29T08:40:57.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:57.184+0000 7f08be000700 1 -- 172.21.15.37:0/2147674579 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08b8110450 con 0x7f08b8070c10 2024-03-29T08:40:57.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:57.185+0000 7f08be000700 1 -- 172.21.15.37:0/2147674579 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08b8071a40 con 0x7f08b8070c10 2024-03-29T08:40:57.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:57.186+0000 7f08af400700 1 -- 172.21.15.37:0/2147674579 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f08a400db20 con 0x7f08b8070c10 2024-03-29T08:40:57.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:57.187+0000 7f08af400700 1 --2- 172.21.15.37:0/2147674579 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f089c06cc30 0x7f089c06f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:40:57.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:57.188+0000 7f08af400700 1 -- 172.21.15.37:0/2147674579 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f08a408bde0 con 0x7f08b8070c10 2024-03-29T08:40:57.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:57.188+0000 7f08b7400700 1 --2- 172.21.15.37:0/2147674579 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f089c06cc30 0x7f089c06f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:40:57.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:57.188+0000 7f08b7400700 1 --2- 172.21.15.37:0/2147674579 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f089c06cc30 0x7f089c06f0e0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f08a8009fd0 tx=0x7f08a8009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:40:57.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:57.192+0000 7f08af400700 1 -- 172.21.15.37:0/2147674579 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f08a4055d20 con 0x7f08b8070c10 2024-03-29T08:40:57.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:57.493+0000 7f08be000700 1 -- 172.21.15.37:0/2147674579 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f08b810bfd0 con 0x7f08b8070c10 2024-03-29T08:40:57.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:57.493+0000 7f08af400700 1 -- 172.21.15.37:0/2147674579 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7f08a4059340 con 0x7f08b8070c10 2024-03-29T08:40:57.493 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-29T08:40:57.493 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": { 2024-03-29T08:40:57.493 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 2 2024-03-29T08:40:57.493 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:40:57.493 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": { 2024-03-29T08:40:57.493 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 2 2024-03-29T08:40:57.494 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:40:57.494 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": { 2024-03-29T08:40:57.494 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 6 2024-03-29T08:40:57.494 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:40:57.494 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": { 2024-03-29T08:40:57.494 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 4 2024-03-29T08:40:57.494 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:40:57.494 INFO:teuthology.orchestra.run.smithi037.stdout: "overall": { 2024-03-29T08:40:57.494 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 14 2024-03-29T08:40:57.494 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-29T08:40:57.494 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-29T08:40:57.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:57.496+0000 7f08be000700 1 -- 172.21.15.37:0/2147674579 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f089c06cc30 msgr2=0x7f089c06f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:40:57.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:57.496+0000 7f08be000700 1 --2- 172.21.15.37:0/2147674579 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f089c06cc30 0x7f089c06f0e0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f08a8009fd0 tx=0x7f08a8009e30 comp rx=0 tx=0).stop 2024-03-29T08:40:57.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:57.496+0000 7f08be000700 1 -- 172.21.15.37:0/2147674579 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f08b8070c10 msgr2=0x7f08b81161d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:40:57.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:57.496+0000 7f08be000700 1 --2- 172.21.15.37:0/2147674579 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f08b8070c10 0x7f08b81161d0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f08a4002a50 tx=0x7f08a40046b0 comp rx=0 tx=0).stop 2024-03-29T08:40:57.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:57.496+0000 7f08be000700 1 -- 172.21.15.37:0/2147674579 shutdown_connections 2024-03-29T08:40:57.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:57.496+0000 7f08be000700 1 --2- 172.21.15.37:0/2147674579 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f089c06cc30 0x7f089c06f0e0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:40:57.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:57.496+0000 7f08be000700 1 --2- 172.21.15.37:0/2147674579 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f08b8070300 0x7f08b8115c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:40:57.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:57.496+0000 7f08be000700 1 --2- 172.21.15.37:0/2147674579 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f08b8070c10 0x7f08b81161d0 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:40:57.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:57.496+0000 7f08be000700 1 -- 172.21.15.37:0/2147674579 >> 172.21.15.37:0/2147674579 conn(0x7f08b80f8970 msgr2=0x7f08b80f9ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:40:57.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:57.496+0000 7f08be000700 1 -- 172.21.15.37:0/2147674579 shutdown_connections 2024-03-29T08:40:57.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:57.496+0000 7f08be000700 1 -- 172.21.15.37:0/2147674579 wait complete. 2024-03-29T08:40:57.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:57.633+0000 7fac5b000700 1 -- 172.21.15.37:0/1957711326 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fac540fdeb0 msgr2=0x7fac540fe280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:40:57.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:57.633+0000 7fac5b000700 1 --2- 172.21.15.37:0/1957711326 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fac540fdeb0 0x7fac540fe280 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fac44004970 tx=0x7fac44009e30 comp rx=0 tx=0).stop 2024-03-29T08:40:57.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:57.634+0000 7fac5b000700 1 -- 172.21.15.37:0/1957711326 shutdown_connections 2024-03-29T08:40:57.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:57.634+0000 7fac5b000700 1 --2- 172.21.15.37:0/1957711326 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fac540fe850 0x7fac54106f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:40:57.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:57.634+0000 7fac5b000700 1 --2- 172.21.15.37:0/1957711326 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fac540fdeb0 0x7fac540fe280 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:40:57.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:57.634+0000 7fac5b000700 1 -- 172.21.15.37:0/1957711326 >> 172.21.15.37:0/1957711326 conn(0x7fac54072bc0 msgr2=0x7fac54072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:40:57.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:57.634+0000 7fac5b000700 1 -- 172.21.15.37:0/1957711326 shutdown_connections 2024-03-29T08:40:57.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:57.634+0000 7fac5b000700 1 -- 172.21.15.37:0/1957711326 wait complete. 2024-03-29T08:40:57.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:57.635+0000 7fac5b000700 1 Processor -- start 2024-03-29T08:40:57.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:57.635+0000 7fac5b000700 1 -- start start 2024-03-29T08:40:57.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:57.636+0000 7fac5b000700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fac540fdeb0 0x7fac54113a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:40:57.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:57.636+0000 7fac5b000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fac540fe850 0x7fac54113f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:40:57.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:57.636+0000 7fac5b000700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac541145b0 con 0x7fac540fe850 2024-03-29T08:40:57.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:57.636+0000 7fac5b000700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac5410d9f0 con 0x7fac540fdeb0 2024-03-29T08:40:57.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:57.636+0000 7fac53400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fac540fe850 0x7fac54113f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:40:57.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:57.636+0000 7fac53400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fac540fe850 0x7fac54113f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38284/0 (socket says 172.21.15.37:38284) 2024-03-29T08:40:57.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:57.636+0000 7fac53400700 1 -- 172.21.15.37:0/3169390027 learned_addr learned my addr 172.21.15.37:0/3169390027 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:40:57.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:57.636+0000 7fac53e00700 1 --2- 172.21.15.37:0/3169390027 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fac540fdeb0 0x7fac54113a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:40:57.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:57.636+0000 7fac53400700 1 -- 172.21.15.37:0/3169390027 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fac540fdeb0 msgr2=0x7fac54113a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:40:57.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:57.636+0000 7fac53400700 1 --2- 172.21.15.37:0/3169390027 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fac540fdeb0 0x7fac54113a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:40:57.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:57.636+0000 7fac53400700 1 -- 172.21.15.37:0/3169390027 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac44005040 con 0x7fac540fe850 2024-03-29T08:40:57.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:57.637+0000 7fac53400700 1 --2- 172.21.15.37:0/3169390027 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fac540fe850 0x7fac54113f60 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fac40002a80 tx=0x7fac400046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:40:57.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:57.637+0000 7fac4be00700 1 -- 172.21.15.37:0/3169390027 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fac40007cc0 con 0x7fac540fe850 2024-03-29T08:40:57.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:57.637+0000 7fac4be00700 1 -- 172.21.15.37:0/3169390027 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fac4000f040 con 0x7fac540fe850 2024-03-29T08:40:57.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:57.637+0000 7fac4be00700 1 -- 172.21.15.37:0/3169390027 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fac40009640 con 0x7fac540fe850 2024-03-29T08:40:57.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:57.637+0000 7fac5b000700 1 -- 172.21.15.37:0/3169390027 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac5410dcd0 con 0x7fac540fe850 2024-03-29T08:40:57.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:57.637+0000 7fac5b000700 1 -- 172.21.15.37:0/3169390027 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac5410e210 con 0x7fac540fe850 2024-03-29T08:40:57.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:57.638+0000 7fac5b000700 1 -- 172.21.15.37:0/3169390027 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac54065e10 con 0x7fac540fe850 2024-03-29T08:40:57.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:57.639+0000 7fac4be00700 1 -- 172.21.15.37:0/3169390027 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fac400097a0 con 0x7fac540fe850 2024-03-29T08:40:57.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:57.640+0000 7fac4be00700 1 --2- 172.21.15.37:0/3169390027 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fac3806cc30 0x7fac3806f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:40:57.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:57.640+0000 7fac4be00700 1 -- 172.21.15.37:0/3169390027 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fac4008b7e0 con 0x7fac540fe850 2024-03-29T08:40:57.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:57.641+0000 7fac53e00700 1 --2- 172.21.15.37:0/3169390027 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fac3806cc30 0x7fac3806f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:40:57.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:57.641+0000 7fac53e00700 1 --2- 172.21.15.37:0/3169390027 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fac3806cc30 0x7fac3806f0e0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fac44004ee0 tx=0x7fac44009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:40:57.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:57.645+0000 7fac4be00700 1 -- 172.21.15.37:0/3169390027 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fac40059230 con 0x7fac540fe850 2024-03-29T08:40:57.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:57.902+0000 7fac5b000700 1 -- 172.21.15.37:0/3169390027 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fac5405f5f0 con 0x7fac540fe850 2024-03-29T08:40:57.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:57.903+0000 7fac4be00700 1 -- 172.21.15.37:0/3169390027 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 30 v30) v1 ==== 76+0+1841 (secure 0 0 0) 0x7fac40058dc0 con 0x7fac540fe850 2024-03-29T08:40:57.903 INFO:teuthology.orchestra.run.smithi037.stdout:e30 2024-03-29T08:40:57.903 INFO:teuthology.orchestra.run.smithi037.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-29T08:40:57.903 INFO:teuthology.orchestra.run.smithi037.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T08:40:57.903 INFO:teuthology.orchestra.run.smithi037.stdout:legacy client fscid: 1 2024-03-29T08:40:57.903 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:40:57.903 INFO:teuthology.orchestra.run.smithi037.stdout:Filesystem 'cephfs' (1) 2024-03-29T08:40:57.903 INFO:teuthology.orchestra.run.smithi037.stdout:fs_name cephfs 2024-03-29T08:40:57.903 INFO:teuthology.orchestra.run.smithi037.stdout:epoch 21 2024-03-29T08:40:57.903 INFO:teuthology.orchestra.run.smithi037.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-03-29T08:40:57.903 INFO:teuthology.orchestra.run.smithi037.stdout:created 2024-03-29T08:03:29.241269+0000 2024-03-29T08:40:57.903 INFO:teuthology.orchestra.run.smithi037.stdout:modified 2024-03-29T08:17:16.237238+0000 2024-03-29T08:40:57.903 INFO:teuthology.orchestra.run.smithi037.stdout:tableserver 0 2024-03-29T08:40:57.903 INFO:teuthology.orchestra.run.smithi037.stdout:root 0 2024-03-29T08:40:57.904 INFO:teuthology.orchestra.run.smithi037.stdout:session_timeout 60 2024-03-29T08:40:57.904 INFO:teuthology.orchestra.run.smithi037.stdout:session_autoclose 300 2024-03-29T08:40:57.904 INFO:teuthology.orchestra.run.smithi037.stdout:max_file_size 1099511627776 2024-03-29T08:40:57.904 INFO:teuthology.orchestra.run.smithi037.stdout:max_xattr_size 65536 2024-03-29T08:40:57.904 INFO:teuthology.orchestra.run.smithi037.stdout:required_client_features {} 2024-03-29T08:40:57.904 INFO:teuthology.orchestra.run.smithi037.stdout:last_failure 0 2024-03-29T08:40:57.904 INFO:teuthology.orchestra.run.smithi037.stdout:last_failure_osd_epoch 82 2024-03-29T08:40:57.904 INFO:teuthology.orchestra.run.smithi037.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T08:40:57.904 INFO:teuthology.orchestra.run.smithi037.stdout:max_mds 2 2024-03-29T08:40:57.904 INFO:teuthology.orchestra.run.smithi037.stdout:in 0,1 2024-03-29T08:40:57.904 INFO:teuthology.orchestra.run.smithi037.stdout:up {} 2024-03-29T08:40:57.904 INFO:teuthology.orchestra.run.smithi037.stdout:failed 0,1 2024-03-29T08:40:57.904 INFO:teuthology.orchestra.run.smithi037.stdout:damaged 2024-03-29T08:40:57.904 INFO:teuthology.orchestra.run.smithi037.stdout:stopped 2024-03-29T08:40:57.904 INFO:teuthology.orchestra.run.smithi037.stdout:data_pools [3] 2024-03-29T08:40:57.905 INFO:teuthology.orchestra.run.smithi037.stdout:metadata_pool 2 2024-03-29T08:40:57.905 INFO:teuthology.orchestra.run.smithi037.stdout:inline_data disabled 2024-03-29T08:40:57.905 INFO:teuthology.orchestra.run.smithi037.stdout:balancer 2024-03-29T08:40:57.905 INFO:teuthology.orchestra.run.smithi037.stdout:bal_rank_mask -1 2024-03-29T08:40:57.905 INFO:teuthology.orchestra.run.smithi037.stdout:standby_count_wanted 1 2024-03-29T08:40:57.905 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:40:57.905 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:40:57.905 INFO:teuthology.orchestra.run.smithi037.stdout:Standby daemons: 2024-03-29T08:40:57.905 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:40:57.905 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi037.mmwgxe{-1:34440} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.37:6828/1248142005,v1:172.21.15.37:6829/1248142005] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:40:57.905 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi037.iifsmh{-1:34444} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.37:6826/2060882959,v1:172.21.15.37:6827/2060882959] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:40:57.905 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi148.sqteih{-1:44395} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.148:6824/4078652242,v1:172.21.15.148:6825/4078652242] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:40:57.905 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi148.qukpjh{-1:45007} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.148:6826/2394751105,v1:172.21.15.148:6827/2394751105] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:40:57.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:57.905+0000 7fac5b000700 1 -- 172.21.15.37:0/3169390027 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fac3806cc30 msgr2=0x7fac3806f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:40:57.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:57.905+0000 7fac5b000700 1 --2- 172.21.15.37:0/3169390027 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fac3806cc30 0x7fac3806f0e0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fac44004ee0 tx=0x7fac44009e30 comp rx=0 tx=0).stop 2024-03-29T08:40:57.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:57.905+0000 7fac5b000700 1 -- 172.21.15.37:0/3169390027 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fac540fe850 msgr2=0x7fac54113f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:40:57.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:57.905+0000 7fac5b000700 1 --2- 172.21.15.37:0/3169390027 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fac540fe850 0x7fac54113f60 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fac40002a80 tx=0x7fac400046b0 comp rx=0 tx=0).stop 2024-03-29T08:40:57.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:57.905+0000 7fac5b000700 1 -- 172.21.15.37:0/3169390027 shutdown_connections 2024-03-29T08:40:57.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:57.905+0000 7fac5b000700 1 --2- 172.21.15.37:0/3169390027 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fac3806cc30 0x7fac3806f0e0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:40:57.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:57.905+0000 7fac5b000700 1 --2- 172.21.15.37:0/3169390027 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fac540fdeb0 0x7fac54113a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:40:57.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:57.905+0000 7fac5b000700 1 --2- 172.21.15.37:0/3169390027 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fac540fe850 0x7fac54113f60 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:40:57.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:57.905+0000 7fac5b000700 1 -- 172.21.15.37:0/3169390027 >> 172.21.15.37:0/3169390027 conn(0x7fac54072bc0 msgr2=0x7fac540ff410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:40:57.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:57.905+0000 7fac5b000700 1 -- 172.21.15.37:0/3169390027 shutdown_connections 2024-03-29T08:40:57.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:57.905+0000 7fac5b000700 1 -- 172.21.15.37:0/3169390027 wait complete. 2024-03-29T08:40:57.907 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 30 2024-03-29T08:40:57.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:40:57 smithi037 ceph-mon[142325]: from='client.35626 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:40:57.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:40:57 smithi037 ceph-mon[142325]: from='client.35630 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:40:57.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:40:57 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/2147674579' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:40:58.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:58.044+0000 7f7f7fc00700 1 -- 172.21.15.37:0/2598837933 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7f780fde90 msgr2=0x7f7f780fe260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:40:58.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:58.044+0000 7f7f7fc00700 1 --2- 172.21.15.37:0/2598837933 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7f780fde90 0x7f7f780fe260 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f7f6c004940 tx=0x7f7f6c009e30 comp rx=0 tx=0).stop 2024-03-29T08:40:58.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:58.044+0000 7f7f7fc00700 1 -- 172.21.15.37:0/2598837933 shutdown_connections 2024-03-29T08:40:58.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:58.044+0000 7f7f7fc00700 1 --2- 172.21.15.37:0/2598837933 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7f780fe830 0x7f7f78106f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:40:58.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:58.044+0000 7f7f7fc00700 1 --2- 172.21.15.37:0/2598837933 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7f780fde90 0x7f7f780fe260 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:40:58.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:58.044+0000 7f7f7fc00700 1 -- 172.21.15.37:0/2598837933 >> 172.21.15.37:0/2598837933 conn(0x7f7f78072bc0 msgr2=0x7f7f78072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:40:58.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:58.044+0000 7f7f7fc00700 1 -- 172.21.15.37:0/2598837933 shutdown_connections 2024-03-29T08:40:58.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:58.044+0000 7f7f7fc00700 1 -- 172.21.15.37:0/2598837933 wait complete. 2024-03-29T08:40:58.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:58.045+0000 7f7f7fc00700 1 Processor -- start 2024-03-29T08:40:58.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:58.045+0000 7f7f7fc00700 1 -- start start 2024-03-29T08:40:58.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:58.045+0000 7f7f7fc00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7f780fde90 0x7f7f78113af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:40:58.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:58.045+0000 7f7f7fc00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7f780fe830 0x7f7f78114030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:40:58.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:58.045+0000 7f7f7fc00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f78114680 con 0x7f7f780fde90 2024-03-29T08:40:58.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:58.045+0000 7f7f7fc00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f7810da70 con 0x7f7f780fe830 2024-03-29T08:40:58.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:58.046+0000 7f7f7d200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7f780fde90 0x7f7f78113af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:40:58.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:58.046+0000 7f7f7d200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7f780fde90 0x7f7f78113af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38300/0 (socket says 172.21.15.37:38300) 2024-03-29T08:40:58.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:58.046+0000 7f7f7d200700 1 -- 172.21.15.37:0/3596429286 learned_addr learned my addr 172.21.15.37:0/3596429286 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:40:58.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:58.046+0000 7f7f77e00700 1 --2- 172.21.15.37:0/3596429286 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7f780fe830 0x7f7f78114030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:40:58.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:58.046+0000 7f7f7d200700 1 -- 172.21.15.37:0/3596429286 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7f780fe830 msgr2=0x7f7f78114030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:40:58.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:58.046+0000 7f7f7d200700 1 --2- 172.21.15.37:0/3596429286 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7f780fe830 0x7f7f78114030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:40:58.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:58.046+0000 7f7f7d200700 1 -- 172.21.15.37:0/3596429286 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f6c005040 con 0x7f7f780fde90 2024-03-29T08:40:58.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:58.046+0000 7f7f77e00700 1 --2- 172.21.15.37:0/3596429286 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7f780fe830 0x7f7f78114030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:40:58.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:58.046+0000 7f7f7d200700 1 --2- 172.21.15.37:0/3596429286 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7f780fde90 0x7f7f78113af0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f7f6c000c00 tx=0x7f7f6c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:40:58.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:58.047+0000 7f7f74c00700 1 -- 172.21.15.37:0/3596429286 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7f6c00e040 con 0x7f7f780fde90 2024-03-29T08:40:58.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:58.047+0000 7f7f74c00700 1 -- 172.21.15.37:0/3596429286 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f7f6c013420 con 0x7f7f780fde90 2024-03-29T08:40:58.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:58.047+0000 7f7f7fc00700 1 -- 172.21.15.37:0/3596429286 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f7810dcf0 con 0x7f7f780fde90 2024-03-29T08:40:58.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:58.047+0000 7f7f74c00700 1 -- 172.21.15.37:0/3596429286 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7f6c01c630 con 0x7f7f780fde90 2024-03-29T08:40:58.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:58.047+0000 7f7f7fc00700 1 -- 172.21.15.37:0/3596429286 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f7810e230 con 0x7f7f780fde90 2024-03-29T08:40:58.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:58.048+0000 7f7f7fc00700 1 -- 172.21.15.37:0/3596429286 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f781019b0 con 0x7f7f780fde90 2024-03-29T08:40:58.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:58.049+0000 7f7f74c00700 1 -- 172.21.15.37:0/3596429286 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f7f6c013970 con 0x7f7f780fde90 2024-03-29T08:40:58.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:58.050+0000 7f7f74c00700 1 --2- 172.21.15.37:0/3596429286 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f7f6006cc30 0x7f7f6006f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:40:58.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:58.050+0000 7f7f74c00700 1 -- 172.21.15.37:0/3596429286 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f7f6c08d2e0 con 0x7f7f780fde90 2024-03-29T08:40:58.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:58.050+0000 7f7f77e00700 1 --2- 172.21.15.37:0/3596429286 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f7f6006cc30 0x7f7f6006f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:40:58.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:58.051+0000 7f7f77e00700 1 --2- 172.21.15.37:0/3596429286 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f7f6006cc30 0x7f7f6006f0e0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f7f64007fd0 tx=0x7f7f64009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:40:58.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:58.054+0000 7f7f74c00700 1 -- 172.21.15.37:0/3596429286 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f7f6c05ad30 con 0x7f7f780fde90 2024-03-29T08:40:58.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:40:57 smithi148 ceph-mon[106692]: from='client.35626 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:40:58.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:40:57 smithi148 ceph-mon[106692]: from='client.35630 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:40:58.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:40:57 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/2147674579' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:40:58.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:58.285+0000 7f7f7fc00700 1 -- 172.21.15.37:0/3596429286 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7f7810f140 con 0x7f7f6006cc30 2024-03-29T08:40:58.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:58.287+0000 7f7f74c00700 1 -- 172.21.15.37:0/3596429286 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f7f7810f140 con 0x7f7f6006cc30 2024-03-29T08:40:58.287 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-29T08:40:58.287 INFO:teuthology.orchestra.run.smithi037.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59", 2024-03-29T08:40:58.287 INFO:teuthology.orchestra.run.smithi037.stdout: "in_progress": true, 2024-03-29T08:40:58.287 INFO:teuthology.orchestra.run.smithi037.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-29T08:40:58.287 INFO:teuthology.orchestra.run.smithi037.stdout: "services_complete": [ 2024-03-29T08:40:58.287 INFO:teuthology.orchestra.run.smithi037.stdout: "crash", 2024-03-29T08:40:58.287 INFO:teuthology.orchestra.run.smithi037.stdout: "osd", 2024-03-29T08:40:58.287 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr", 2024-03-29T08:40:58.287 INFO:teuthology.orchestra.run.smithi037.stdout: "mds", 2024-03-29T08:40:58.287 INFO:teuthology.orchestra.run.smithi037.stdout: "mon" 2024-03-29T08:40:58.287 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-29T08:40:58.287 INFO:teuthology.orchestra.run.smithi037.stdout: "progress": "16/21 daemons upgraded", 2024-03-29T08:40:58.287 INFO:teuthology.orchestra.run.smithi037.stdout: "message": "Currently upgrading mds daemons", 2024-03-29T08:40:58.287 INFO:teuthology.orchestra.run.smithi037.stdout: "is_paused": false 2024-03-29T08:40:58.287 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-29T08:40:58.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:58.289+0000 7f7f7fc00700 1 -- 172.21.15.37:0/3596429286 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f7f6006cc30 msgr2=0x7f7f6006f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:40:58.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:58.289+0000 7f7f7fc00700 1 --2- 172.21.15.37:0/3596429286 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f7f6006cc30 0x7f7f6006f0e0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f7f64007fd0 tx=0x7f7f64009040 comp rx=0 tx=0).stop 2024-03-29T08:40:58.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:58.289+0000 7f7f7fc00700 1 -- 172.21.15.37:0/3596429286 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7f780fde90 msgr2=0x7f7f78113af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:40:58.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:58.289+0000 7f7f7fc00700 1 --2- 172.21.15.37:0/3596429286 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7f780fde90 0x7f7f78113af0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f7f6c000c00 tx=0x7f7f6c009e30 comp rx=0 tx=0).stop 2024-03-29T08:40:58.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:58.289+0000 7f7f7fc00700 1 -- 172.21.15.37:0/3596429286 shutdown_connections 2024-03-29T08:40:58.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:58.289+0000 7f7f7fc00700 1 --2- 172.21.15.37:0/3596429286 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f7f6006cc30 0x7f7f6006f0e0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:40:58.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:58.289+0000 7f7f7fc00700 1 --2- 172.21.15.37:0/3596429286 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7f780fde90 0x7f7f78113af0 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:40:58.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:58.289+0000 7f7f7fc00700 1 --2- 172.21.15.37:0/3596429286 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7f780fe830 0x7f7f78114030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:40:58.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:58.289+0000 7f7f7fc00700 1 -- 172.21.15.37:0/3596429286 >> 172.21.15.37:0/3596429286 conn(0x7f7f78072bc0 msgr2=0x7f7f780ff3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:40:58.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:58.289+0000 7f7f7fc00700 1 -- 172.21.15.37:0/3596429286 shutdown_connections 2024-03-29T08:40:58.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:58.289+0000 7f7f7fc00700 1 -- 172.21.15.37:0/3596429286 wait complete. 2024-03-29T08:40:58.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:58.426+0000 7f4268c00700 1 -- 172.21.15.37:0/1073728378 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f42641037a0 msgr2=0x7f42640fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:40:58.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:58.426+0000 7f4268c00700 1 --2- 172.21.15.37:0/1073728378 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f42641037a0 0x7f42640fce10 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f4250004940 tx=0x7f4250009e30 comp rx=0 tx=0).stop 2024-03-29T08:40:58.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:58.426+0000 7f4268c00700 1 -- 172.21.15.37:0/1073728378 shutdown_connections 2024-03-29T08:40:58.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:58.426+0000 7f4268c00700 1 --2- 172.21.15.37:0/1073728378 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f42641037a0 0x7f42640fce10 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:40:58.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:58.426+0000 7f4268c00700 1 --2- 172.21.15.37:0/1073728378 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4264102e00 0x7f42641031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:40:58.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:58.426+0000 7f4268c00700 1 -- 172.21.15.37:0/1073728378 >> 172.21.15.37:0/1073728378 conn(0x7f42640f89b0 msgr2=0x7f42640fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:40:58.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:58.426+0000 7f4268c00700 1 -- 172.21.15.37:0/1073728378 shutdown_connections 2024-03-29T08:40:58.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:58.426+0000 7f4268c00700 1 -- 172.21.15.37:0/1073728378 wait complete. 2024-03-29T08:40:58.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:58.427+0000 7f4268c00700 1 Processor -- start 2024-03-29T08:40:58.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:58.427+0000 7f4268c00700 1 -- start start 2024-03-29T08:40:58.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:58.428+0000 7f4268c00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4264102e00 0x7f4264113b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:40:58.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:58.428+0000 7f4268c00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f42641037a0 0x7f4264114040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:40:58.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:58.428+0000 7f4268c00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4264114690 con 0x7f4264102e00 2024-03-29T08:40:58.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:58.428+0000 7f4268c00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f426410da80 con 0x7f42641037a0 2024-03-29T08:40:58.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:58.428+0000 7f4261e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4264102e00 0x7f4264113b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:40:58.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:58.428+0000 7f4261e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4264102e00 0x7f4264113b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38328/0 (socket says 172.21.15.37:38328) 2024-03-29T08:40:58.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:58.428+0000 7f4261e00700 1 -- 172.21.15.37:0/923774276 learned_addr learned my addr 172.21.15.37:0/923774276 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:40:58.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:58.428+0000 7f4261400700 1 --2- 172.21.15.37:0/923774276 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f42641037a0 0x7f4264114040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:40:58.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:58.428+0000 7f4261e00700 1 -- 172.21.15.37:0/923774276 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f42641037a0 msgr2=0x7f4264114040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:40:58.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:58.428+0000 7f4261e00700 1 --2- 172.21.15.37:0/923774276 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f42641037a0 0x7f4264114040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:40:58.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:58.428+0000 7f4261e00700 1 -- 172.21.15.37:0/923774276 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4250005040 con 0x7f4264102e00 2024-03-29T08:40:58.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:58.429+0000 7f4261e00700 1 --2- 172.21.15.37:0/923774276 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4264102e00 0x7f4264113b00 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f4254002a50 tx=0x7f42540046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:40:58.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:58.429+0000 7f425a000700 1 -- 172.21.15.37:0/923774276 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4254007cc0 con 0x7f4264102e00 2024-03-29T08:40:58.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:58.429+0000 7f425a000700 1 -- 172.21.15.37:0/923774276 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f425400f040 con 0x7f4264102e00 2024-03-29T08:40:58.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:58.429+0000 7f4268c00700 1 -- 172.21.15.37:0/923774276 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f426410dd60 con 0x7f4264102e00 2024-03-29T08:40:58.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:58.429+0000 7f425a000700 1 -- 172.21.15.37:0/923774276 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f425400d9c0 con 0x7f4264102e00 2024-03-29T08:40:58.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:58.429+0000 7f4268c00700 1 -- 172.21.15.37:0/923774276 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f426410e2f0 con 0x7f4264102e00 2024-03-29T08:40:58.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:58.430+0000 7f4268c00700 1 -- 172.21.15.37:0/923774276 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42640fe550 con 0x7f4264102e00 2024-03-29T08:40:58.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:58.431+0000 7f425a000700 1 -- 172.21.15.37:0/923774276 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f425400db20 con 0x7f4264102e00 2024-03-29T08:40:58.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:58.432+0000 7f425a000700 1 --2- 172.21.15.37:0/923774276 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f424806cc20 0x7f424806f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:40:58.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:58.432+0000 7f425a000700 1 -- 172.21.15.37:0/923774276 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f425408bb40 con 0x7f4264102e00 2024-03-29T08:40:58.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:58.432+0000 7f4261400700 1 --2- 172.21.15.37:0/923774276 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f424806cc20 0x7f424806f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:40:58.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:58.433+0000 7f4261400700 1 --2- 172.21.15.37:0/923774276 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f424806cc20 0x7f424806f0d0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f4250004890 tx=0x7f4250009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:40:58.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:58.437+0000 7f425a000700 1 -- 172.21.15.37:0/923774276 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f4254059590 con 0x7f4264102e00 2024-03-29T08:40:58.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:58.736+0000 7f4268c00700 1 -- 172.21.15.37:0/923774276 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f4264109dd0 con 0x7f4264102e00 2024-03-29T08:40:58.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:58.736+0000 7f425a000700 1 -- 172.21.15.37:0/923774276 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+3583 (secure 0 0 0) 0x7f4254059120 con 0x7f4264102e00 2024-03-29T08:40:58.736 INFO:teuthology.orchestra.run.smithi037.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem is offline 2024-03-29T08:40:58.736 INFO:teuthology.orchestra.run.smithi037.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-03-29T08:40:58.736 INFO:teuthology.orchestra.run.smithi037.stdout: host smithi148 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/mon.smithi148/config 2024-03-29T08:40:58.736 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/podman: timeout after 895 seconds 2024-03-29T08:40:58.737 INFO:teuthology.orchestra.run.smithi037.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59 -e NODE_NAME=smithi148 -e CEPH_USE_RANDOM_NONCE=1 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4:/var/run/ceph:z -v /var/log/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4:/var/log/ceph:z -v /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpkx2rz2d4:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59 inventory --format=json-pretty --filter-for-batch 2024-03-29T08:40:58.737 INFO:teuthology.orchestra.run.smithi037.stdout:Traceback (most recent call last): 2024-03-29T08:40:58.737 INFO:teuthology.orchestra.run.smithi037.stdout: File "/usr/lib64/python3.6/runpy.py", line 193, in _run_module_as_main 2024-03-29T08:40:58.737 INFO:teuthology.orchestra.run.smithi037.stdout: "__main__", mod_spec) 2024-03-29T08:40:58.737 INFO:teuthology.orchestra.run.smithi037.stdout: File "/usr/lib64/python3.6/runpy.py", line 85, in _run_code 2024-03-29T08:40:58.737 INFO:teuthology.orchestra.run.smithi037.stdout: exec(code, run_globals) 2024-03-29T08:40:58.737 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 10889, in 2024-03-29T08:40:58.737 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 10877, in main 2024-03-29T08:40:58.737 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 2576, in _infer_config 2024-03-29T08:40:58.737 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 2492, in _infer_fsid 2024-03-29T08:40:58.737 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 2604, in _infer_image 2024-03-29T08:40:58.737 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 2479, in _validate_fsid 2024-03-29T08:40:58.737 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 7145, in command_ceph_volume 2024-03-29T08:40:58.737 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 2267, in call_throws 2024-03-29T08:40:58.737 INFO:teuthology.orchestra.run.smithi037.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59 -e NODE_NAME=smithi148 -e CEPH_USE_RANDOM_NONCE=1 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4:/var/run/ceph:z -v /var/log/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4:/var/log/ceph:z -v /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpkx2rz2d4:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59 inventory --format=json-pretty --filter-for-batch 2024-03-29T08:40:58.738 INFO:teuthology.orchestra.run.smithi037.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-03-29T08:40:58.738 INFO:teuthology.orchestra.run.smithi037.stdout: fs cephfs is degraded 2024-03-29T08:40:58.738 INFO:teuthology.orchestra.run.smithi037.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-03-29T08:40:58.738 INFO:teuthology.orchestra.run.smithi037.stdout: fs cephfs is offline because no MDS is active for it. 2024-03-29T08:40:58.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:58.738+0000 7f4268c00700 1 -- 172.21.15.37:0/923774276 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f424806cc20 msgr2=0x7f424806f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:40:58.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:58.739+0000 7f4268c00700 1 --2- 172.21.15.37:0/923774276 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f424806cc20 0x7f424806f0d0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f4250004890 tx=0x7f4250009e30 comp rx=0 tx=0).stop 2024-03-29T08:40:58.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:58.739+0000 7f4268c00700 1 -- 172.21.15.37:0/923774276 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4264102e00 msgr2=0x7f4264113b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:40:58.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:58.739+0000 7f4268c00700 1 --2- 172.21.15.37:0/923774276 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4264102e00 0x7f4264113b00 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f4254002a50 tx=0x7f42540046b0 comp rx=0 tx=0).stop 2024-03-29T08:40:58.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:58.739+0000 7f4268c00700 1 -- 172.21.15.37:0/923774276 shutdown_connections 2024-03-29T08:40:58.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:58.739+0000 7f4268c00700 1 --2- 172.21.15.37:0/923774276 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f424806cc20 0x7f424806f0d0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:40:58.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:58.739+0000 7f4268c00700 1 --2- 172.21.15.37:0/923774276 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4264102e00 0x7f4264113b00 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:40:58.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:58.739+0000 7f4268c00700 1 --2- 172.21.15.37:0/923774276 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f42641037a0 0x7f4264114040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:40:58.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:58.739+0000 7f4268c00700 1 -- 172.21.15.37:0/923774276 >> 172.21.15.37:0/923774276 conn(0x7f42640f89b0 msgr2=0x7f42640f9ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:40:58.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:58.739+0000 7f4268c00700 1 -- 172.21.15.37:0/923774276 shutdown_connections 2024-03-29T08:40:58.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:40:58.739+0000 7f4268c00700 1 -- 172.21.15.37:0/923774276 wait complete. 2024-03-29T08:40:58.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:40:58 smithi037 ceph-mon[142325]: from='client.35634 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:40:58.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:40:58 smithi037 ceph-mon[142325]: pgmap v903: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:40:58.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:40:58 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/3169390027' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T08:40:59.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:40:58 smithi148 ceph-mon[106692]: from='client.35634 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:40:59.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:40:58 smithi148 ceph-mon[106692]: pgmap v903: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:40:59.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:40:58 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/3169390027' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T08:40:59.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:40:59 smithi037 ceph-mon[142325]: from='client.35646 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:40:59.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:40:59 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/923774276' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T08:41:00.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:40:59 smithi148 ceph-mon[106692]: from='client.35646 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:41:00.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:40:59 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/923774276' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T08:41:00.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:41:00 smithi037 ceph-mon[142325]: pgmap v904: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:41:01.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:41:00 smithi148 ceph-mon[106692]: pgmap v904: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:41:02.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:41:02 smithi037 ceph-mon[142325]: pgmap v905: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:41:03.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:41:02 smithi148 ceph-mon[106692]: pgmap v905: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:41:04.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:41:04 smithi037 ceph-mon[142325]: pgmap v906: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:41:05.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:41:04 smithi148 ceph-mon[106692]: pgmap v906: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:41:06.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:41:06 smithi037 ceph-mon[142325]: pgmap v907: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:41:07.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:41:06 smithi148 ceph-mon[106692]: pgmap v907: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:41:08.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:41:08 smithi037 ceph-mon[142325]: pgmap v908: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:41:09.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:41:08 smithi148 ceph-mon[106692]: pgmap v908: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:41:10.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:41:10 smithi148 ceph-mon[106692]: pgmap v909: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:41:10.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:41:10 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:41:10.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:41:10 smithi037 ceph-mon[142325]: pgmap v909: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:41:10.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:41:10 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:41:12.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:41:12 smithi148 ceph-mon[106692]: pgmap v910: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:41:12.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:41:12 smithi037 ceph-mon[142325]: pgmap v910: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:41:14.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:41:14 smithi148 ceph-mon[106692]: pgmap v911: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:41:14.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:41:14 smithi037 ceph-mon[142325]: pgmap v911: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:41:16.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:41:16 smithi148 ceph-mon[106692]: pgmap v912: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:41:16.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:41:16 smithi037 ceph-mon[142325]: pgmap v912: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:41:18.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:41:18 smithi148 ceph-mon[106692]: pgmap v913: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:41:18.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:41:18 smithi037 ceph-mon[142325]: pgmap v913: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:41:20.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:41:20 smithi148 ceph-mon[106692]: pgmap v914: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:41:20.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:41:20 smithi037 ceph-mon[142325]: pgmap v914: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:41:22.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:41:22 smithi148 ceph-mon[106692]: pgmap v915: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:41:22.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:41:22 smithi037 ceph-mon[142325]: pgmap v915: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:41:24.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:41:24 smithi148 ceph-mon[106692]: pgmap v916: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:41:24.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:41:24 smithi037 ceph-mon[142325]: pgmap v916: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:41:25.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:41:25 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:41:25.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:41:25 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:41:26.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:41:26 smithi148 ceph-mon[106692]: pgmap v917: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:41:26.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:41:26 smithi037 ceph-mon[142325]: pgmap v917: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:41:28.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:41:28 smithi148 ceph-mon[106692]: pgmap v918: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:41:28.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:28.892+0000 7f01e4a00700 1 -- 172.21.15.37:0/3494894270 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f01e0102e00 msgr2=0x7f01e01031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:41:28.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:28.892+0000 7f01e4a00700 1 --2- 172.21.15.37:0/3494894270 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f01e0102e00 0x7f01e01031d0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f01d0004940 tx=0x7f01d0009e30 comp rx=0 tx=0).stop 2024-03-29T08:41:28.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:28.892+0000 7f01e4a00700 1 -- 172.21.15.37:0/3494894270 shutdown_connections 2024-03-29T08:41:28.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:28.892+0000 7f01e4a00700 1 --2- 172.21.15.37:0/3494894270 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f01e01037a0 0x7f01e00fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:41:28.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:28.892+0000 7f01e4a00700 1 --2- 172.21.15.37:0/3494894270 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f01e0102e00 0x7f01e01031d0 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:41:28.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:28.892+0000 7f01e4a00700 1 -- 172.21.15.37:0/3494894270 >> 172.21.15.37:0/3494894270 conn(0x7f01e00f89b0 msgr2=0x7f01e00fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:41:28.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:28.892+0000 7f01e4a00700 1 -- 172.21.15.37:0/3494894270 shutdown_connections 2024-03-29T08:41:28.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:28.892+0000 7f01e4a00700 1 -- 172.21.15.37:0/3494894270 wait complete. 2024-03-29T08:41:28.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:28.893+0000 7f01e4a00700 1 Processor -- start 2024-03-29T08:41:28.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:28.893+0000 7f01e4a00700 1 -- start start 2024-03-29T08:41:28.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:28.894+0000 7f01e4a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f01e0102e00 0x7f01e0115cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:41:28.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:28.894+0000 7f01e4a00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f01e01037a0 0x7f01e0116230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:41:28.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:28.894+0000 7f01e4a00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01e0116880 con 0x7f01e0102e00 2024-03-29T08:41:28.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:28.894+0000 7f01e4a00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01e010fc50 con 0x7f01e01037a0 2024-03-29T08:41:28.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:28.894+0000 7f01dde00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f01e0102e00 0x7f01e0115cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:41:28.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:28.894+0000 7f01dde00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f01e0102e00 0x7f01e0115cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44622/0 (socket says 172.21.15.37:44622) 2024-03-29T08:41:28.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:28.894+0000 7f01dde00700 1 -- 172.21.15.37:0/3552173069 learned_addr learned my addr 172.21.15.37:0/3552173069 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:41:28.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:28.894+0000 7f01dd400700 1 --2- 172.21.15.37:0/3552173069 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f01e01037a0 0x7f01e0116230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:41:28.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:28.895+0000 7f01dde00700 1 -- 172.21.15.37:0/3552173069 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f01e01037a0 msgr2=0x7f01e0116230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:41:28.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:28.895+0000 7f01dde00700 1 --2- 172.21.15.37:0/3552173069 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f01e01037a0 0x7f01e0116230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:41:28.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:28.895+0000 7f01dde00700 1 -- 172.21.15.37:0/3552173069 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01d0005040 con 0x7f01e0102e00 2024-03-29T08:41:28.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:28.895+0000 7f01dde00700 1 --2- 172.21.15.37:0/3552173069 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f01e0102e00 0x7f01e0115cf0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f01d0009390 tx=0x7f01d0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:41:28.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:28.895+0000 7f01d6000700 1 -- 172.21.15.37:0/3552173069 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f01d000e040 con 0x7f01e0102e00 2024-03-29T08:41:28.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:28.895+0000 7f01d6000700 1 -- 172.21.15.37:0/3552173069 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f01d0013420 con 0x7f01e0102e00 2024-03-29T08:41:28.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:28.895+0000 7f01d6000700 1 -- 172.21.15.37:0/3552173069 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f01d001c630 con 0x7f01e0102e00 2024-03-29T08:41:28.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:28.895+0000 7f01e4a00700 1 -- 172.21.15.37:0/3552173069 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01e010fed0 con 0x7f01e0102e00 2024-03-29T08:41:28.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:28.895+0000 7f01e4a00700 1 -- 172.21.15.37:0/3552173069 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01e0110490 con 0x7f01e0102e00 2024-03-29T08:41:28.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:28.896+0000 7f01e4a00700 1 -- 172.21.15.37:0/3552173069 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01e00fe550 con 0x7f01e0102e00 2024-03-29T08:41:28.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:28.897+0000 7f01d6000700 1 -- 172.21.15.37:0/3552173069 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f01d00139e0 con 0x7f01e0102e00 2024-03-29T08:41:28.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:28.898+0000 7f01d6000700 1 --2- 172.21.15.37:0/3552173069 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f01c406cc30 0x7f01c406f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:41:28.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:28.898+0000 7f01d6000700 1 -- 172.21.15.37:0/3552173069 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f01d008d250 con 0x7f01e0102e00 2024-03-29T08:41:28.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:28.899+0000 7f01dd400700 1 --2- 172.21.15.37:0/3552173069 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f01c406cc30 0x7f01c406f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:41:28.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:28.899+0000 7f01dd400700 1 --2- 172.21.15.37:0/3552173069 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f01c406cc30 0x7f01c406f0e0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f01e0111590 tx=0x7f01cc009ee0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:41:28.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:28.903+0000 7f01d6000700 1 -- 172.21.15.37:0/3552173069 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f01d005add0 con 0x7f01e0102e00 2024-03-29T08:41:28.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:41:28 smithi037 ceph-mon[142325]: pgmap v918: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:41:29.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:29.134+0000 7f01e4a00700 1 -- 172.21.15.37:0/3552173069 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f01e004bf40 con 0x7f01c406cc30 2024-03-29T08:41:29.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:29.136+0000 7f01d6000700 1 -- 172.21.15.37:0/3552173069 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f01e004bf40 con 0x7f01c406cc30 2024-03-29T08:41:29.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:29.138+0000 7f01e4a00700 1 -- 172.21.15.37:0/3552173069 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f01c406cc30 msgr2=0x7f01c406f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:41:29.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:29.138+0000 7f01e4a00700 1 --2- 172.21.15.37:0/3552173069 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f01c406cc30 0x7f01c406f0e0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f01e0111590 tx=0x7f01cc009ee0 comp rx=0 tx=0).stop 2024-03-29T08:41:29.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:29.138+0000 7f01e4a00700 1 -- 172.21.15.37:0/3552173069 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f01e0102e00 msgr2=0x7f01e0115cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:41:29.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:29.138+0000 7f01e4a00700 1 --2- 172.21.15.37:0/3552173069 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f01e0102e00 0x7f01e0115cf0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f01d0009390 tx=0x7f01d0009e30 comp rx=0 tx=0).stop 2024-03-29T08:41:29.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:29.138+0000 7f01e4a00700 1 -- 172.21.15.37:0/3552173069 shutdown_connections 2024-03-29T08:41:29.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:29.138+0000 7f01e4a00700 1 --2- 172.21.15.37:0/3552173069 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f01c406cc30 0x7f01c406f0e0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:41:29.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:29.138+0000 7f01e4a00700 1 --2- 172.21.15.37:0/3552173069 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f01e0102e00 0x7f01e0115cf0 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:41:29.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:29.138+0000 7f01e4a00700 1 --2- 172.21.15.37:0/3552173069 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f01e01037a0 0x7f01e0116230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:41:29.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:29.138+0000 7f01e4a00700 1 -- 172.21.15.37:0/3552173069 >> 172.21.15.37:0/3552173069 conn(0x7f01e00f89b0 msgr2=0x7f01e00706d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:41:29.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:29.138+0000 7f01e4a00700 1 -- 172.21.15.37:0/3552173069 shutdown_connections 2024-03-29T08:41:29.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:29.138+0000 7f01e4a00700 1 -- 172.21.15.37:0/3552173069 wait complete. 2024-03-29T08:41:29.152 INFO:teuthology.orchestra.run.smithi037.stdout:true 2024-03-29T08:41:29.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:29.279+0000 7fec7bc00700 1 -- 172.21.15.37:0/1121601075 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fec74102e00 msgr2=0x7fec741031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:41:29.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:29.279+0000 7fec7bc00700 1 --2- 172.21.15.37:0/1121601075 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fec74102e00 0x7fec741031d0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fec68004940 tx=0x7fec68009e30 comp rx=0 tx=0).stop 2024-03-29T08:41:29.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:29.279+0000 7fec7bc00700 1 -- 172.21.15.37:0/1121601075 shutdown_connections 2024-03-29T08:41:29.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:29.279+0000 7fec7bc00700 1 --2- 172.21.15.37:0/1121601075 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fec741037a0 0x7fec740fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:41:29.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:29.279+0000 7fec7bc00700 1 --2- 172.21.15.37:0/1121601075 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fec74102e00 0x7fec741031d0 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:41:29.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:29.279+0000 7fec7bc00700 1 -- 172.21.15.37:0/1121601075 >> 172.21.15.37:0/1121601075 conn(0x7fec740f89b0 msgr2=0x7fec740fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:41:29.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:29.279+0000 7fec7bc00700 1 -- 172.21.15.37:0/1121601075 shutdown_connections 2024-03-29T08:41:29.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:29.279+0000 7fec7bc00700 1 -- 172.21.15.37:0/1121601075 wait complete. 2024-03-29T08:41:29.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:29.280+0000 7fec7bc00700 1 Processor -- start 2024-03-29T08:41:29.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:29.280+0000 7fec7bc00700 1 -- start start 2024-03-29T08:41:29.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:29.281+0000 7fec7bc00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fec74102e00 0x7fec74115d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:41:29.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:29.281+0000 7fec7bc00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fec741037a0 0x7fec74116250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:41:29.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:29.281+0000 7fec7bc00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec741168a0 con 0x7fec741037a0 2024-03-29T08:41:29.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:29.281+0000 7fec7bc00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec7410fc70 con 0x7fec74102e00 2024-03-29T08:41:29.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:29.281+0000 7fec73e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fec741037a0 0x7fec74116250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:41:29.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:29.281+0000 7fec73e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fec741037a0 0x7fec74116250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44646/0 (socket says 172.21.15.37:44646) 2024-03-29T08:41:29.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:29.281+0000 7fec73e00700 1 -- 172.21.15.37:0/1292010541 learned_addr learned my addr 172.21.15.37:0/1292010541 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:41:29.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:29.281+0000 7fec79200700 1 --2- 172.21.15.37:0/1292010541 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fec74102e00 0x7fec74115d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:41:29.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:29.281+0000 7fec73e00700 1 -- 172.21.15.37:0/1292010541 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fec74102e00 msgr2=0x7fec74115d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:41:29.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:29.281+0000 7fec73e00700 1 --2- 172.21.15.37:0/1292010541 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fec74102e00 0x7fec74115d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:41:29.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:29.281+0000 7fec73e00700 1 -- 172.21.15.37:0/1292010541 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec68005040 con 0x7fec741037a0 2024-03-29T08:41:29.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:29.281+0000 7fec79200700 1 --2- 172.21.15.37:0/1292010541 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fec74102e00 0x7fec74115d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:41:29.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:29.282+0000 7fec73e00700 1 --2- 172.21.15.37:0/1292010541 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fec741037a0 0x7fec74116250 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fec60002a50 tx=0x7fec60004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:41:29.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:29.282+0000 7fec70c00700 1 -- 172.21.15.37:0/1292010541 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fec60007cc0 con 0x7fec741037a0 2024-03-29T08:41:29.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:29.282+0000 7fec70c00700 1 -- 172.21.15.37:0/1292010541 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fec6000f040 con 0x7fec741037a0 2024-03-29T08:41:29.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:29.282+0000 7fec7bc00700 1 -- 172.21.15.37:0/1292010541 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec7410ff50 con 0x7fec741037a0 2024-03-29T08:41:29.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:29.282+0000 7fec70c00700 1 -- 172.21.15.37:0/1292010541 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fec6000d9a0 con 0x7fec741037a0 2024-03-29T08:41:29.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:29.282+0000 7fec7bc00700 1 -- 172.21.15.37:0/1292010541 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec74110510 con 0x7fec741037a0 2024-03-29T08:41:29.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:29.283+0000 7fec7bc00700 1 -- 172.21.15.37:0/1292010541 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec740fe550 con 0x7fec741037a0 2024-03-29T08:41:29.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:29.284+0000 7fec70c00700 1 -- 172.21.15.37:0/1292010541 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fec6000dc60 con 0x7fec741037a0 2024-03-29T08:41:29.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:29.285+0000 7fec70c00700 1 --2- 172.21.15.37:0/1292010541 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fec5c06cc30 0x7fec5c06f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:41:29.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:29.286+0000 7fec70c00700 1 -- 172.21.15.37:0/1292010541 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fec6008bef0 con 0x7fec741037a0 2024-03-29T08:41:29.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:29.286+0000 7fec79200700 1 --2- 172.21.15.37:0/1292010541 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fec5c06cc30 0x7fec5c06f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:41:29.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:29.286+0000 7fec79200700 1 --2- 172.21.15.37:0/1292010541 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fec5c06cc30 0x7fec5c06f0e0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fec6800b840 tx=0x7fec68009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:41:29.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:29.289+0000 7fec70c00700 1 -- 172.21.15.37:0/1292010541 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fec600598c0 con 0x7fec741037a0 2024-03-29T08:41:29.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:29.521+0000 7fec7bc00700 1 -- 172.21.15.37:0/1292010541 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fec74109e30 con 0x7fec5c06cc30 2024-03-29T08:41:29.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:29.522+0000 7fec70c00700 1 -- 172.21.15.37:0/1292010541 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fec74109e30 con 0x7fec5c06cc30 2024-03-29T08:41:29.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:29.524+0000 7fec7bc00700 1 -- 172.21.15.37:0/1292010541 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fec5c06cc30 msgr2=0x7fec5c06f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:41:29.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:29.524+0000 7fec7bc00700 1 --2- 172.21.15.37:0/1292010541 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fec5c06cc30 0x7fec5c06f0e0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fec6800b840 tx=0x7fec68009e30 comp rx=0 tx=0).stop 2024-03-29T08:41:29.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:29.524+0000 7fec7bc00700 1 -- 172.21.15.37:0/1292010541 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fec741037a0 msgr2=0x7fec74116250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:41:29.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:29.524+0000 7fec7bc00700 1 --2- 172.21.15.37:0/1292010541 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fec741037a0 0x7fec74116250 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fec60002a50 tx=0x7fec60004600 comp rx=0 tx=0).stop 2024-03-29T08:41:29.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:29.524+0000 7fec7bc00700 1 -- 172.21.15.37:0/1292010541 shutdown_connections 2024-03-29T08:41:29.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:29.524+0000 7fec7bc00700 1 --2- 172.21.15.37:0/1292010541 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fec5c06cc30 0x7fec5c06f0e0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:41:29.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:29.524+0000 7fec7bc00700 1 --2- 172.21.15.37:0/1292010541 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fec74102e00 0x7fec74115d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:41:29.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:29.524+0000 7fec7bc00700 1 --2- 172.21.15.37:0/1292010541 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fec741037a0 0x7fec74116250 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:41:29.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:29.524+0000 7fec7bc00700 1 -- 172.21.15.37:0/1292010541 >> 172.21.15.37:0/1292010541 conn(0x7fec740f89b0 msgr2=0x7fec740704f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:41:29.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:29.525+0000 7fec7bc00700 1 -- 172.21.15.37:0/1292010541 shutdown_connections 2024-03-29T08:41:29.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:29.525+0000 7fec7bc00700 1 -- 172.21.15.37:0/1292010541 wait complete. 2024-03-29T08:41:29.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:29.662+0000 7fe632200700 1 -- 172.21.15.37:0/3124881573 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe62c0fde90 msgr2=0x7fe62c0fe260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:41:29.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:29.662+0000 7fe632200700 1 --2- 172.21.15.37:0/3124881573 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe62c0fde90 0x7fe62c0fe260 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fe61c004940 tx=0x7fe61c009e30 comp rx=0 tx=0).stop 2024-03-29T08:41:29.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:29.662+0000 7fe632200700 1 -- 172.21.15.37:0/3124881573 shutdown_connections 2024-03-29T08:41:29.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:29.662+0000 7fe632200700 1 --2- 172.21.15.37:0/3124881573 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe62c0fe830 0x7fe62c106f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:41:29.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:29.662+0000 7fe632200700 1 --2- 172.21.15.37:0/3124881573 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe62c0fde90 0x7fe62c0fe260 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:41:29.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:29.662+0000 7fe632200700 1 -- 172.21.15.37:0/3124881573 >> 172.21.15.37:0/3124881573 conn(0x7fe62c072bc0 msgr2=0x7fe62c072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:41:29.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:29.663+0000 7fe632200700 1 -- 172.21.15.37:0/3124881573 shutdown_connections 2024-03-29T08:41:29.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:29.663+0000 7fe632200700 1 -- 172.21.15.37:0/3124881573 wait complete. 2024-03-29T08:41:29.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:29.663+0000 7fe632200700 1 Processor -- start 2024-03-29T08:41:29.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:29.664+0000 7fe632200700 1 -- start start 2024-03-29T08:41:29.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:29.664+0000 7fe632200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe62c0fde90 0x7fe62c115d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:41:29.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:29.664+0000 7fe632200700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe62c0fe830 0x7fe62c116240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:41:29.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:29.664+0000 7fe632200700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe62c116890 con 0x7fe62c0fde90 2024-03-29T08:41:29.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:29.664+0000 7fe632200700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe62c10fc60 con 0x7fe62c0fe830 2024-03-29T08:41:29.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:29.664+0000 7fe62b400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe62c0fde90 0x7fe62c115d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:41:29.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:29.664+0000 7fe62b400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe62c0fde90 0x7fe62c115d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44668/0 (socket says 172.21.15.37:44668) 2024-03-29T08:41:29.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:29.664+0000 7fe62b400700 1 -- 172.21.15.37:0/1119206808 learned_addr learned my addr 172.21.15.37:0/1119206808 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:41:29.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:29.664+0000 7fe62aa00700 1 --2- 172.21.15.37:0/1119206808 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe62c0fe830 0x7fe62c116240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:41:29.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:29.665+0000 7fe62b400700 1 -- 172.21.15.37:0/1119206808 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe62c0fe830 msgr2=0x7fe62c116240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:41:29.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:29.665+0000 7fe62b400700 1 --2- 172.21.15.37:0/1119206808 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe62c0fe830 0x7fe62c116240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:41:29.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:29.665+0000 7fe62b400700 1 -- 172.21.15.37:0/1119206808 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe61c005040 con 0x7fe62c0fde90 2024-03-29T08:41:29.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:29.665+0000 7fe62b400700 1 --2- 172.21.15.37:0/1119206808 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe62c0fde90 0x7fe62c115d00 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fe61c000c00 tx=0x7fe61c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:41:29.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:29.665+0000 7fe623400700 1 -- 172.21.15.37:0/1119206808 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe61c00e040 con 0x7fe62c0fde90 2024-03-29T08:41:29.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:29.665+0000 7fe623400700 1 -- 172.21.15.37:0/1119206808 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fe61c00bed0 con 0x7fe62c0fde90 2024-03-29T08:41:29.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:29.665+0000 7fe632200700 1 -- 172.21.15.37:0/1119206808 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe62c10fee0 con 0x7fe62c0fde90 2024-03-29T08:41:29.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:29.665+0000 7fe623400700 1 -- 172.21.15.37:0/1119206808 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe61c01c3f0 con 0x7fe62c0fde90 2024-03-29T08:41:29.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:29.665+0000 7fe632200700 1 -- 172.21.15.37:0/1119206808 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe62c110420 con 0x7fe62c0fde90 2024-03-29T08:41:29.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:29.667+0000 7fe632200700 1 -- 172.21.15.37:0/1119206808 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe62c106a20 con 0x7fe62c0fde90 2024-03-29T08:41:29.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:29.668+0000 7fe623400700 1 -- 172.21.15.37:0/1119206808 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fe61c013420 con 0x7fe62c0fde90 2024-03-29T08:41:29.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:29.668+0000 7fe623400700 1 --2- 172.21.15.37:0/1119206808 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fe61006cc30 0x7fe61006f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:41:29.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:29.669+0000 7fe623400700 1 -- 172.21.15.37:0/1119206808 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fe61c08d440 con 0x7fe62c0fde90 2024-03-29T08:41:29.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:29.669+0000 7fe62aa00700 1 --2- 172.21.15.37:0/1119206808 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fe61006cc30 0x7fe61006f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:41:29.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:29.669+0000 7fe62aa00700 1 --2- 172.21.15.37:0/1119206808 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fe61006cc30 0x7fe61006f0e0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fe62c1115a0 tx=0x7fe61800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:41:29.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:29.673+0000 7fe623400700 1 -- 172.21.15.37:0/1119206808 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fe61c05af10 con 0x7fe62c0fde90 2024-03-29T08:41:29.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:29.897+0000 7fe632200700 1 -- 172.21.15.37:0/1119206808 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fe62c109e30 con 0x7fe61006cc30 2024-03-29T08:41:29.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:29.907+0000 7fe623400700 1 -- 172.21.15.37:0/1119206808 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7fe62c109e30 con 0x7fe61006cc30 2024-03-29T08:41:29.907 INFO:teuthology.orchestra.run.smithi037.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-29T08:41:29.907 INFO:teuthology.orchestra.run.smithi037.stdout:alertmanager.smithi037 smithi037 *:9093,9094 running (31m) 8m ago 44m 14.3M - 0.25.0 c8568f914cd2 2f9f9746ed0f 2024-03-29T08:41:29.907 INFO:teuthology.orchestra.run.smithi037.stdout:crash.smithi037 smithi037 running (28m) 8m ago 44m 6647k - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 178d416a3186 2024-03-29T08:41:29.907 INFO:teuthology.orchestra.run.smithi037.stdout:crash.smithi148 smithi148 running (28m) 8m ago 43m 6656k - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 23666989d58f 2024-03-29T08:41:29.907 INFO:teuthology.orchestra.run.smithi037.stdout:grafana.smithi037 smithi037 *:3000 running (31m) 8m ago 43m 74.5M - 9.4.7 954c08fa6188 6b976fe49952 2024-03-29T08:41:29.908 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi037.iifsmh smithi037 running (23m) 8m ago 37m 16.4M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 2d97f562cc5b 2024-03-29T08:41:29.908 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi037.mmwgxe smithi037 running (24m) 8m ago 37m 16.7M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff ec37037f46e8 2024-03-29T08:41:29.908 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi148.qukpjh smithi148 running (8m) 8m ago 37m 10.7M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 3257219bc046 2024-03-29T08:41:29.908 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi148.sqteih smithi148 running (23m) 8m ago 37m 16.3M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 44c15c77037a 2024-03-29T08:41:29.908 INFO:teuthology.orchestra.run.smithi037.stdout:mgr.smithi037.wjikii smithi037 *:8443,9283,8765 running (33m) 8m ago 45m 510M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff c5a1417b9417 2024-03-29T08:41:29.908 INFO:teuthology.orchestra.run.smithi037.stdout:mgr.smithi148.onhgvm smithi148 *:8443,9283,8765 running (32m) 8m ago 43m 456M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff fcfdbce062bd 2024-03-29T08:41:29.908 INFO:teuthology.orchestra.run.smithi037.stdout:mon.smithi037 smithi037 running (29m) 8m ago 46m 70.2M 2048M 18.2.2-1015-gd0e392d9 1cc2f45c17ff eea2060783af 2024-03-29T08:41:29.908 INFO:teuthology.orchestra.run.smithi037.stdout:mon.smithi148 smithi148 running (29m) 8m ago 43m 50.4M 2048M 18.2.2-1015-gd0e392d9 1cc2f45c17ff ad4b2c5748be 2024-03-29T08:41:29.908 INFO:teuthology.orchestra.run.smithi037.stdout:node-exporter.smithi037 smithi037 *:9100 running (32m) 8m ago 44m 8514k - 1.5.0 0da6a335fe13 26b8ae9cb878 2024-03-29T08:41:29.908 INFO:teuthology.orchestra.run.smithi037.stdout:node-exporter.smithi148 smithi148 *:9100 running (32m) 8m ago 43m 8464k - 1.5.0 0da6a335fe13 98414e4e64ed 2024-03-29T08:41:29.908 INFO:teuthology.orchestra.run.smithi037.stdout:osd.0 smithi037 running (27m) 8m ago 42m 108M 1619M 18.2.2-1015-gd0e392d9 1cc2f45c17ff c0c584f6245c 2024-03-29T08:41:29.908 INFO:teuthology.orchestra.run.smithi037.stdout:osd.1 smithi037 running (27m) 8m ago 41m 104M 1619M 18.2.2-1015-gd0e392d9 1cc2f45c17ff a4a2ff4cc83d 2024-03-29T08:41:29.908 INFO:teuthology.orchestra.run.smithi037.stdout:osd.2 smithi037 running (26m) 8m ago 41m 85.9M 1619M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 8b1bc71aa79e 2024-03-29T08:41:29.908 INFO:teuthology.orchestra.run.smithi037.stdout:osd.3 smithi148 running (25m) 8m ago 40m 109M 2642M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 1c656ab8ff86 2024-03-29T08:41:29.909 INFO:teuthology.orchestra.run.smithi037.stdout:osd.4 smithi148 running (25m) 8m ago 40m 101M 2642M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 966a9de11d22 2024-03-29T08:41:29.909 INFO:teuthology.orchestra.run.smithi037.stdout:osd.5 smithi148 running (24m) 8m ago 39m 89.4M 2642M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 474c4f9212ce 2024-03-29T08:41:29.909 INFO:teuthology.orchestra.run.smithi037.stdout:prometheus.smithi037 smithi037 *:9095 running (31m) 8m ago 43m 61.0M - 2.43.0 a07b618ecd1d 30c0223ec589 2024-03-29T08:41:29.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:29.909+0000 7fe632200700 1 -- 172.21.15.37:0/1119206808 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fe61006cc30 msgr2=0x7fe61006f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:41:29.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:29.909+0000 7fe632200700 1 --2- 172.21.15.37:0/1119206808 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fe61006cc30 0x7fe61006f0e0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fe62c1115a0 tx=0x7fe61800b040 comp rx=0 tx=0).stop 2024-03-29T08:41:29.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:29.909+0000 7fe632200700 1 -- 172.21.15.37:0/1119206808 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe62c0fde90 msgr2=0x7fe62c115d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:41:29.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:29.909+0000 7fe632200700 1 --2- 172.21.15.37:0/1119206808 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe62c0fde90 0x7fe62c115d00 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fe61c000c00 tx=0x7fe61c009e30 comp rx=0 tx=0).stop 2024-03-29T08:41:29.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:29.909+0000 7fe632200700 1 -- 172.21.15.37:0/1119206808 shutdown_connections 2024-03-29T08:41:29.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:29.909+0000 7fe632200700 1 --2- 172.21.15.37:0/1119206808 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fe61006cc30 0x7fe61006f0e0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:41:29.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:29.909+0000 7fe632200700 1 --2- 172.21.15.37:0/1119206808 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe62c0fde90 0x7fe62c115d00 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:41:29.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:29.909+0000 7fe632200700 1 --2- 172.21.15.37:0/1119206808 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe62c0fe830 0x7fe62c116240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:41:29.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:29.909+0000 7fe632200700 1 -- 172.21.15.37:0/1119206808 >> 172.21.15.37:0/1119206808 conn(0x7fe62c072bc0 msgr2=0x7fe62c0fd590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:41:29.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:29.909+0000 7fe632200700 1 -- 172.21.15.37:0/1119206808 shutdown_connections 2024-03-29T08:41:29.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:29.909+0000 7fe632200700 1 -- 172.21.15.37:0/1119206808 wait complete. 2024-03-29T08:41:30.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:30.044+0000 7fb578a00700 1 -- 172.21.15.37:0/2756141056 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb574070c70 msgr2=0x7fb574109110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:41:30.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:30.044+0000 7fb578a00700 1 --2- 172.21.15.37:0/2756141056 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb574070c70 0x7fb574109110 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fb560004970 tx=0x7fb560009e30 comp rx=0 tx=0).stop 2024-03-29T08:41:30.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:30.045+0000 7fb578a00700 1 -- 172.21.15.37:0/2756141056 shutdown_connections 2024-03-29T08:41:30.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:30.045+0000 7fb578a00700 1 --2- 172.21.15.37:0/2756141056 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb574070c70 0x7fb574109110 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:41:30.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:30.045+0000 7fb578a00700 1 --2- 172.21.15.37:0/2756141056 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb574070360 0x7fb574070730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:41:30.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:30.045+0000 7fb578a00700 1 -- 172.21.15.37:0/2756141056 >> 172.21.15.37:0/2756141056 conn(0x7fb5740f89b0 msgr2=0x7fb5740fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:41:30.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:30.045+0000 7fb578a00700 1 -- 172.21.15.37:0/2756141056 shutdown_connections 2024-03-29T08:41:30.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:30.045+0000 7fb578a00700 1 -- 172.21.15.37:0/2756141056 wait complete. 2024-03-29T08:41:30.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:30.046+0000 7fb578a00700 1 Processor -- start 2024-03-29T08:41:30.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:30.046+0000 7fb578a00700 1 -- start start 2024-03-29T08:41:30.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:30.046+0000 7fb578a00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb574070360 0x7fb574115ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:41:30.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:30.046+0000 7fb578a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb574070c70 0x7fb5741161e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:41:30.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:30.046+0000 7fb578a00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb574116830 con 0x7fb574070c70 2024-03-29T08:41:30.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:30.046+0000 7fb578a00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb57410fc00 con 0x7fb574070360 2024-03-29T08:41:30.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:30.046+0000 7fb571400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb574070c70 0x7fb5741161e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:41:30.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:30.046+0000 7fb571400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb574070c70 0x7fb5741161e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47742/0 (socket says 172.21.15.37:47742) 2024-03-29T08:41:30.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:30.046+0000 7fb571400700 1 -- 172.21.15.37:0/1189456595 learned_addr learned my addr 172.21.15.37:0/1189456595 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:41:30.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:30.047+0000 7fb571400700 1 -- 172.21.15.37:0/1189456595 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb574070360 msgr2=0x7fb574115ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:41:30.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:30.047+0000 7fb571e00700 1 --2- 172.21.15.37:0/1189456595 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb574070360 0x7fb574115ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:41:30.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:30.047+0000 7fb571400700 1 --2- 172.21.15.37:0/1189456595 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb574070360 0x7fb574115ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:41:30.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:30.047+0000 7fb571400700 1 -- 172.21.15.37:0/1189456595 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb560005040 con 0x7fb574070c70 2024-03-29T08:41:30.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:30.047+0000 7fb571e00700 1 --2- 172.21.15.37:0/1189456595 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb574070360 0x7fb574115ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:41:30.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:30.047+0000 7fb571400700 1 --2- 172.21.15.37:0/1189456595 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb574070c70 0x7fb5741161e0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fb560004f50 tx=0x7fb560009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:41:30.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:30.047+0000 7fb56a000700 1 -- 172.21.15.37:0/1189456595 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb56000e040 con 0x7fb574070c70 2024-03-29T08:41:30.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:30.047+0000 7fb56a000700 1 -- 172.21.15.37:0/1189456595 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fb560013420 con 0x7fb574070c70 2024-03-29T08:41:30.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:30.047+0000 7fb578a00700 1 -- 172.21.15.37:0/1189456595 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb57410fe80 con 0x7fb574070c70 2024-03-29T08:41:30.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:30.047+0000 7fb56a000700 1 -- 172.21.15.37:0/1189456595 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb56001c610 con 0x7fb574070c70 2024-03-29T08:41:30.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:30.048+0000 7fb578a00700 1 -- 172.21.15.37:0/1189456595 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5741103c0 con 0x7fb574070c70 2024-03-29T08:41:30.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:30.049+0000 7fb578a00700 1 -- 172.21.15.37:0/1189456595 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb574103bd0 con 0x7fb574070c70 2024-03-29T08:41:30.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:30.050+0000 7fb56a000700 1 -- 172.21.15.37:0/1189456595 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fb560013590 con 0x7fb574070c70 2024-03-29T08:41:30.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:30.050+0000 7fb56a000700 1 --2- 172.21.15.37:0/1189456595 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fb55806cc30 0x7fb55806f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:41:30.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:30.051+0000 7fb56a000700 1 -- 172.21.15.37:0/1189456595 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fb56008d260 con 0x7fb574070c70 2024-03-29T08:41:30.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:30.051+0000 7fb571e00700 1 --2- 172.21.15.37:0/1189456595 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fb55806cc30 0x7fb55806f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:41:30.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:30.051+0000 7fb571e00700 1 --2- 172.21.15.37:0/1189456595 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fb55806cc30 0x7fb55806f0e0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fb564009fd0 tx=0x7fb56400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:41:30.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:30.055+0000 7fb56a000700 1 -- 172.21.15.37:0/1189456595 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fb56005acb0 con 0x7fb574070c70 2024-03-29T08:41:30.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:30.354+0000 7fb578a00700 1 -- 172.21.15.37:0/1189456595 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fb5741112f0 con 0x7fb574070c70 2024-03-29T08:41:30.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:30.354+0000 7fb56a000700 1 -- 172.21.15.37:0/1189456595 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7fb56005a840 con 0x7fb574070c70 2024-03-29T08:41:30.354 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-29T08:41:30.354 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": { 2024-03-29T08:41:30.354 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 2 2024-03-29T08:41:30.354 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:41:30.354 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": { 2024-03-29T08:41:30.354 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 2 2024-03-29T08:41:30.354 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:41:30.354 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": { 2024-03-29T08:41:30.355 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 6 2024-03-29T08:41:30.355 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:41:30.355 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": { 2024-03-29T08:41:30.355 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 4 2024-03-29T08:41:30.355 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:41:30.355 INFO:teuthology.orchestra.run.smithi037.stdout: "overall": { 2024-03-29T08:41:30.355 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 14 2024-03-29T08:41:30.355 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-29T08:41:30.355 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-29T08:41:30.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:30.356+0000 7fb578a00700 1 -- 172.21.15.37:0/1189456595 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fb55806cc30 msgr2=0x7fb55806f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:41:30.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:30.356+0000 7fb578a00700 1 --2- 172.21.15.37:0/1189456595 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fb55806cc30 0x7fb55806f0e0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fb564009fd0 tx=0x7fb56400b040 comp rx=0 tx=0).stop 2024-03-29T08:41:30.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:30.356+0000 7fb578a00700 1 -- 172.21.15.37:0/1189456595 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb574070c70 msgr2=0x7fb5741161e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:41:30.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:30.356+0000 7fb578a00700 1 --2- 172.21.15.37:0/1189456595 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb574070c70 0x7fb5741161e0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fb560004f50 tx=0x7fb560009e30 comp rx=0 tx=0).stop 2024-03-29T08:41:30.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:30.357+0000 7fb578a00700 1 -- 172.21.15.37:0/1189456595 shutdown_connections 2024-03-29T08:41:30.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:30.357+0000 7fb578a00700 1 --2- 172.21.15.37:0/1189456595 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fb55806cc30 0x7fb55806f0e0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:41:30.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:30.357+0000 7fb578a00700 1 --2- 172.21.15.37:0/1189456595 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb574070360 0x7fb574115ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:41:30.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:30.357+0000 7fb578a00700 1 --2- 172.21.15.37:0/1189456595 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb574070c70 0x7fb5741161e0 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:41:30.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:30.357+0000 7fb578a00700 1 -- 172.21.15.37:0/1189456595 >> 172.21.15.37:0/1189456595 conn(0x7fb5740f89b0 msgr2=0x7fb5741015f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:41:30.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:30.357+0000 7fb578a00700 1 -- 172.21.15.37:0/1189456595 shutdown_connections 2024-03-29T08:41:30.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:30.357+0000 7fb578a00700 1 -- 172.21.15.37:0/1189456595 wait complete. 2024-03-29T08:41:30.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:30.493+0000 7f9390000700 1 -- 172.21.15.37:0/1485798486 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f93880fde90 msgr2=0x7f93880fe260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:41:30.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:30.493+0000 7f9390000700 1 --2- 172.21.15.37:0/1485798486 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f93880fde90 0x7f93880fe260 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f937c004940 tx=0x7f937c009e30 comp rx=0 tx=0).stop 2024-03-29T08:41:30.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:30.494+0000 7f9390000700 1 -- 172.21.15.37:0/1485798486 shutdown_connections 2024-03-29T08:41:30.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:30.494+0000 7f9390000700 1 --2- 172.21.15.37:0/1485798486 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f93880fe830 0x7f9388106f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:41:30.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:30.494+0000 7f9390000700 1 --2- 172.21.15.37:0/1485798486 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f93880fde90 0x7f93880fe260 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:41:30.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:30.494+0000 7f9390000700 1 -- 172.21.15.37:0/1485798486 >> 172.21.15.37:0/1485798486 conn(0x7f9388072bc0 msgr2=0x7f9388072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:41:30.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:30.494+0000 7f9390000700 1 -- 172.21.15.37:0/1485798486 shutdown_connections 2024-03-29T08:41:30.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:30.494+0000 7f9390000700 1 -- 172.21.15.37:0/1485798486 wait complete. 2024-03-29T08:41:30.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:30.495+0000 7f9390000700 1 Processor -- start 2024-03-29T08:41:30.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:30.495+0000 7f9390000700 1 -- start start 2024-03-29T08:41:30.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:30.495+0000 7f9390000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f93880fde90 0x7f9388115d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:41:30.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:30.495+0000 7f9390000700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f93880fe830 0x7f9388116260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:41:30.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:30.495+0000 7f9390000700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93881168b0 con 0x7f93880fde90 2024-03-29T08:41:30.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:30.495+0000 7f9390000700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f938810fc80 con 0x7f93880fe830 2024-03-29T08:41:30.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:30.495+0000 7f938d600700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f93880fde90 0x7f9388115d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:41:30.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:30.496+0000 7f938d600700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f93880fde90 0x7f9388115d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47756/0 (socket says 172.21.15.37:47756) 2024-03-29T08:41:30.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:30.496+0000 7f938d600700 1 -- 172.21.15.37:0/2999696942 learned_addr learned my addr 172.21.15.37:0/2999696942 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:41:30.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:30.496+0000 7f938cc00700 1 --2- 172.21.15.37:0/2999696942 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f93880fe830 0x7f9388116260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:41:30.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:30.496+0000 7f938d600700 1 -- 172.21.15.37:0/2999696942 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f93880fe830 msgr2=0x7f9388116260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:41:30.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:30.496+0000 7f938d600700 1 --2- 172.21.15.37:0/2999696942 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f93880fe830 0x7f9388116260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:41:30.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:30.496+0000 7f938d600700 1 -- 172.21.15.37:0/2999696942 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f937c005040 con 0x7f93880fde90 2024-03-29T08:41:30.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:30.496+0000 7f938d600700 1 --2- 172.21.15.37:0/2999696942 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f93880fde90 0x7f9388115d20 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f937c000c00 tx=0x7f937c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:41:30.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:30.496+0000 7f9385600700 1 -- 172.21.15.37:0/2999696942 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f937c00e040 con 0x7f93880fde90 2024-03-29T08:41:30.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:30.496+0000 7f9385600700 1 -- 172.21.15.37:0/2999696942 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f937c013420 con 0x7f93880fde90 2024-03-29T08:41:30.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:30.496+0000 7f9390000700 1 -- 172.21.15.37:0/2999696942 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f938810ff00 con 0x7f93880fde90 2024-03-29T08:41:30.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:30.496+0000 7f9385600700 1 -- 172.21.15.37:0/2999696942 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f937c01c630 con 0x7f93880fde90 2024-03-29T08:41:30.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:30.497+0000 7f9390000700 1 -- 172.21.15.37:0/2999696942 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9388110440 con 0x7f93880fde90 2024-03-29T08:41:30.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:30.498+0000 7f9390000700 1 -- 172.21.15.37:0/2999696942 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9388106a20 con 0x7f93880fde90 2024-03-29T08:41:30.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:30.499+0000 7f9385600700 1 -- 172.21.15.37:0/2999696942 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f937c013970 con 0x7f93880fde90 2024-03-29T08:41:30.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:30.499+0000 7f9385600700 1 --2- 172.21.15.37:0/2999696942 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f937006cc30 0x7f937006f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:41:30.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:30.500+0000 7f9385600700 1 -- 172.21.15.37:0/2999696942 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f937c08d2e0 con 0x7f93880fde90 2024-03-29T08:41:30.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:30.500+0000 7f938cc00700 1 --2- 172.21.15.37:0/2999696942 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f937006cc30 0x7f937006f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:41:30.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:30.500+0000 7f938cc00700 1 --2- 172.21.15.37:0/2999696942 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f937006cc30 0x7f937006f0e0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f9374007fd0 tx=0x7f9374009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:41:30.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:30.504+0000 7f9385600700 1 -- 172.21.15.37:0/2999696942 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f937c05ad30 con 0x7f93880fde90 2024-03-29T08:41:30.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:30.765+0000 7f9390000700 1 -- 172.21.15.37:0/2999696942 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f9388109dc0 con 0x7f93880fde90 2024-03-29T08:41:30.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:30.765+0000 7f9385600700 1 -- 172.21.15.37:0/2999696942 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 30 v30) v1 ==== 76+0+1841 (secure 0 0 0) 0x7f937c05a8c0 con 0x7f93880fde90 2024-03-29T08:41:30.765 INFO:teuthology.orchestra.run.smithi037.stdout:e30 2024-03-29T08:41:30.766 INFO:teuthology.orchestra.run.smithi037.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-29T08:41:30.766 INFO:teuthology.orchestra.run.smithi037.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T08:41:30.766 INFO:teuthology.orchestra.run.smithi037.stdout:legacy client fscid: 1 2024-03-29T08:41:30.766 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:41:30.766 INFO:teuthology.orchestra.run.smithi037.stdout:Filesystem 'cephfs' (1) 2024-03-29T08:41:30.766 INFO:teuthology.orchestra.run.smithi037.stdout:fs_name cephfs 2024-03-29T08:41:30.766 INFO:teuthology.orchestra.run.smithi037.stdout:epoch 21 2024-03-29T08:41:30.766 INFO:teuthology.orchestra.run.smithi037.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-03-29T08:41:30.766 INFO:teuthology.orchestra.run.smithi037.stdout:created 2024-03-29T08:03:29.241269+0000 2024-03-29T08:41:30.766 INFO:teuthology.orchestra.run.smithi037.stdout:modified 2024-03-29T08:17:16.237238+0000 2024-03-29T08:41:30.767 INFO:teuthology.orchestra.run.smithi037.stdout:tableserver 0 2024-03-29T08:41:30.767 INFO:teuthology.orchestra.run.smithi037.stdout:root 0 2024-03-29T08:41:30.767 INFO:teuthology.orchestra.run.smithi037.stdout:session_timeout 60 2024-03-29T08:41:30.767 INFO:teuthology.orchestra.run.smithi037.stdout:session_autoclose 300 2024-03-29T08:41:30.767 INFO:teuthology.orchestra.run.smithi037.stdout:max_file_size 1099511627776 2024-03-29T08:41:30.767 INFO:teuthology.orchestra.run.smithi037.stdout:max_xattr_size 65536 2024-03-29T08:41:30.767 INFO:teuthology.orchestra.run.smithi037.stdout:required_client_features {} 2024-03-29T08:41:30.767 INFO:teuthology.orchestra.run.smithi037.stdout:last_failure 0 2024-03-29T08:41:30.767 INFO:teuthology.orchestra.run.smithi037.stdout:last_failure_osd_epoch 82 2024-03-29T08:41:30.768 INFO:teuthology.orchestra.run.smithi037.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T08:41:30.768 INFO:teuthology.orchestra.run.smithi037.stdout:max_mds 2 2024-03-29T08:41:30.768 INFO:teuthology.orchestra.run.smithi037.stdout:in 0,1 2024-03-29T08:41:30.768 INFO:teuthology.orchestra.run.smithi037.stdout:up {} 2024-03-29T08:41:30.768 INFO:teuthology.orchestra.run.smithi037.stdout:failed 0,1 2024-03-29T08:41:30.768 INFO:teuthology.orchestra.run.smithi037.stdout:damaged 2024-03-29T08:41:30.768 INFO:teuthology.orchestra.run.smithi037.stdout:stopped 2024-03-29T08:41:30.768 INFO:teuthology.orchestra.run.smithi037.stdout:data_pools [3] 2024-03-29T08:41:30.768 INFO:teuthology.orchestra.run.smithi037.stdout:metadata_pool 2 2024-03-29T08:41:30.768 INFO:teuthology.orchestra.run.smithi037.stdout:inline_data disabled 2024-03-29T08:41:30.769 INFO:teuthology.orchestra.run.smithi037.stdout:balancer 2024-03-29T08:41:30.769 INFO:teuthology.orchestra.run.smithi037.stdout:bal_rank_mask -1 2024-03-29T08:41:30.769 INFO:teuthology.orchestra.run.smithi037.stdout:standby_count_wanted 1 2024-03-29T08:41:30.769 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:41:30.769 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:41:30.769 INFO:teuthology.orchestra.run.smithi037.stdout:Standby daemons: 2024-03-29T08:41:30.769 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:41:30.769 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi037.mmwgxe{-1:34440} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.37:6828/1248142005,v1:172.21.15.37:6829/1248142005] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:41:30.769 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi037.iifsmh{-1:34444} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.37:6826/2060882959,v1:172.21.15.37:6827/2060882959] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:41:30.769 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi148.sqteih{-1:44395} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.148:6824/4078652242,v1:172.21.15.148:6825/4078652242] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:41:30.770 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi148.qukpjh{-1:45007} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.148:6826/2394751105,v1:172.21.15.148:6827/2394751105] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:41:30.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:30.768+0000 7f9390000700 1 -- 172.21.15.37:0/2999696942 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f937006cc30 msgr2=0x7f937006f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:41:30.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:30.768+0000 7f9390000700 1 --2- 172.21.15.37:0/2999696942 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f937006cc30 0x7f937006f0e0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f9374007fd0 tx=0x7f9374009040 comp rx=0 tx=0).stop 2024-03-29T08:41:30.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:30.768+0000 7f9390000700 1 -- 172.21.15.37:0/2999696942 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f93880fde90 msgr2=0x7f9388115d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:41:30.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:30.768+0000 7f9390000700 1 --2- 172.21.15.37:0/2999696942 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f93880fde90 0x7f9388115d20 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f937c000c00 tx=0x7f937c009e30 comp rx=0 tx=0).stop 2024-03-29T08:41:30.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:30.768+0000 7f9390000700 1 -- 172.21.15.37:0/2999696942 shutdown_connections 2024-03-29T08:41:30.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:30.768+0000 7f9390000700 1 --2- 172.21.15.37:0/2999696942 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f937006cc30 0x7f937006f0e0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:41:30.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:30.768+0000 7f9390000700 1 --2- 172.21.15.37:0/2999696942 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f93880fde90 0x7f9388115d20 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:41:30.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:30.768+0000 7f9390000700 1 --2- 172.21.15.37:0/2999696942 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f93880fe830 0x7f9388116260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:41:30.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:30.768+0000 7f9390000700 1 -- 172.21.15.37:0/2999696942 >> 172.21.15.37:0/2999696942 conn(0x7f9388072bc0 msgr2=0x7f93880fb8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:41:30.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:30.768+0000 7f9390000700 1 -- 172.21.15.37:0/2999696942 shutdown_connections 2024-03-29T08:41:30.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:30.768+0000 7f9390000700 1 -- 172.21.15.37:0/2999696942 wait complete. 2024-03-29T08:41:30.773 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 30 2024-03-29T08:41:30.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:41:30 smithi148 ceph-mon[106692]: from='client.35654 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:41:30.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:41:30 smithi148 ceph-mon[106692]: pgmap v919: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:41:30.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:41:30 smithi148 ceph-mon[106692]: from='client.35658 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:41:30.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:41:30 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/1189456595' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:41:30.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:30.908+0000 7ff030a00700 1 -- 172.21.15.37:0/131558599 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff02c0fde90 msgr2=0x7ff02c0fe260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:41:30.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:30.908+0000 7ff030a00700 1 --2- 172.21.15.37:0/131558599 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff02c0fde90 0x7ff02c0fe260 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7ff01c004940 tx=0x7ff01c009e30 comp rx=0 tx=0).stop 2024-03-29T08:41:30.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:30.908+0000 7ff030a00700 1 -- 172.21.15.37:0/131558599 shutdown_connections 2024-03-29T08:41:30.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:30.908+0000 7ff030a00700 1 --2- 172.21.15.37:0/131558599 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff02c0fe830 0x7ff02c106f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:41:30.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:30.908+0000 7ff030a00700 1 --2- 172.21.15.37:0/131558599 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff02c0fde90 0x7ff02c0fe260 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:41:30.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:30.908+0000 7ff030a00700 1 -- 172.21.15.37:0/131558599 >> 172.21.15.37:0/131558599 conn(0x7ff02c072bc0 msgr2=0x7ff02c072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:41:30.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:30.908+0000 7ff030a00700 1 -- 172.21.15.37:0/131558599 shutdown_connections 2024-03-29T08:41:30.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:30.908+0000 7ff030a00700 1 -- 172.21.15.37:0/131558599 wait complete. 2024-03-29T08:41:30.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:30.909+0000 7ff030a00700 1 Processor -- start 2024-03-29T08:41:30.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:30.909+0000 7ff030a00700 1 -- start start 2024-03-29T08:41:30.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:30.909+0000 7ff030a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff02c0fde90 0x7ff02c115d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:41:30.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:30.909+0000 7ff030a00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff02c0fe830 0x7ff02c116260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:41:30.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:30.909+0000 7ff030a00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff02c1168b0 con 0x7ff02c0fde90 2024-03-29T08:41:30.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:30.909+0000 7ff030a00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff02c10fc80 con 0x7ff02c0fe830 2024-03-29T08:41:30.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:30.910+0000 7ff029e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff02c0fde90 0x7ff02c115d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:41:30.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:30.910+0000 7ff029e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff02c0fde90 0x7ff02c115d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47780/0 (socket says 172.21.15.37:47780) 2024-03-29T08:41:30.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:30.910+0000 7ff029e00700 1 -- 172.21.15.37:0/158294315 learned_addr learned my addr 172.21.15.37:0/158294315 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:41:30.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:30.910+0000 7ff029400700 1 --2- 172.21.15.37:0/158294315 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff02c0fe830 0x7ff02c116260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:41:30.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:30.910+0000 7ff029e00700 1 -- 172.21.15.37:0/158294315 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff02c0fe830 msgr2=0x7ff02c116260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:41:30.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:30.910+0000 7ff029e00700 1 --2- 172.21.15.37:0/158294315 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff02c0fe830 0x7ff02c116260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:41:30.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:30.910+0000 7ff029e00700 1 -- 172.21.15.37:0/158294315 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff01c005040 con 0x7ff02c0fde90 2024-03-29T08:41:30.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:30.910+0000 7ff029e00700 1 --2- 172.21.15.37:0/158294315 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff02c0fde90 0x7ff02c115d20 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7ff01c000c00 tx=0x7ff01c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:41:30.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:30.911+0000 7ff022000700 1 -- 172.21.15.37:0/158294315 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff01c00e040 con 0x7ff02c0fde90 2024-03-29T08:41:30.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:30.911+0000 7ff030a00700 1 -- 172.21.15.37:0/158294315 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff02c10ff00 con 0x7ff02c0fde90 2024-03-29T08:41:30.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:30.911+0000 7ff022000700 1 -- 172.21.15.37:0/158294315 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ff01c013420 con 0x7ff02c0fde90 2024-03-29T08:41:30.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:30.911+0000 7ff030a00700 1 -- 172.21.15.37:0/158294315 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff02c110440 con 0x7ff02c0fde90 2024-03-29T08:41:30.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:30.911+0000 7ff022000700 1 -- 172.21.15.37:0/158294315 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff01c0255e0 con 0x7ff02c0fde90 2024-03-29T08:41:30.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:30.912+0000 7ff030a00700 1 -- 172.21.15.37:0/158294315 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff02c106a20 con 0x7ff02c0fde90 2024-03-29T08:41:30.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:30.913+0000 7ff022000700 1 -- 172.21.15.37:0/158294315 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7ff01c013970 con 0x7ff02c0fde90 2024-03-29T08:41:30.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:30.914+0000 7ff022000700 1 --2- 172.21.15.37:0/158294315 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7ff01006cc30 0x7ff01006f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:41:30.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:30.915+0000 7ff022000700 1 -- 172.21.15.37:0/158294315 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7ff01c08d2d0 con 0x7ff02c0fde90 2024-03-29T08:41:30.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:30.915+0000 7ff029400700 1 --2- 172.21.15.37:0/158294315 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7ff01006cc30 0x7ff01006f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:41:30.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:30.915+0000 7ff029400700 1 --2- 172.21.15.37:0/158294315 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7ff01006cc30 0x7ff01006f0e0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7ff018007fd0 tx=0x7ff018009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:41:30.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:30.919+0000 7ff022000700 1 -- 172.21.15.37:0/158294315 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7ff01c05add0 con 0x7ff02c0fde90 2024-03-29T08:41:30.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:41:30 smithi037 ceph-mon[142325]: from='client.35654 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:41:30.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:41:30 smithi037 ceph-mon[142325]: pgmap v919: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:41:30.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:41:30 smithi037 ceph-mon[142325]: from='client.35658 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:41:30.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:41:30 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/1189456595' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:41:31.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:31.149+0000 7ff030a00700 1 -- 172.21.15.37:0/158294315 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff02c109dc0 con 0x7ff01006cc30 2024-03-29T08:41:31.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:31.151+0000 7ff022000700 1 -- 172.21.15.37:0/158294315 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7ff02c109dc0 con 0x7ff01006cc30 2024-03-29T08:41:31.150 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-29T08:41:31.151 INFO:teuthology.orchestra.run.smithi037.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59", 2024-03-29T08:41:31.151 INFO:teuthology.orchestra.run.smithi037.stdout: "in_progress": true, 2024-03-29T08:41:31.151 INFO:teuthology.orchestra.run.smithi037.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-29T08:41:31.151 INFO:teuthology.orchestra.run.smithi037.stdout: "services_complete": [ 2024-03-29T08:41:31.151 INFO:teuthology.orchestra.run.smithi037.stdout: "crash", 2024-03-29T08:41:31.151 INFO:teuthology.orchestra.run.smithi037.stdout: "osd", 2024-03-29T08:41:31.151 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr", 2024-03-29T08:41:31.151 INFO:teuthology.orchestra.run.smithi037.stdout: "mds", 2024-03-29T08:41:31.151 INFO:teuthology.orchestra.run.smithi037.stdout: "mon" 2024-03-29T08:41:31.151 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-29T08:41:31.152 INFO:teuthology.orchestra.run.smithi037.stdout: "progress": "16/21 daemons upgraded", 2024-03-29T08:41:31.152 INFO:teuthology.orchestra.run.smithi037.stdout: "message": "Currently upgrading mds daemons", 2024-03-29T08:41:31.152 INFO:teuthology.orchestra.run.smithi037.stdout: "is_paused": false 2024-03-29T08:41:31.152 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-29T08:41:31.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:31.153+0000 7ff030a00700 1 -- 172.21.15.37:0/158294315 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7ff01006cc30 msgr2=0x7ff01006f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:41:31.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:31.153+0000 7ff030a00700 1 --2- 172.21.15.37:0/158294315 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7ff01006cc30 0x7ff01006f0e0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7ff018007fd0 tx=0x7ff018009040 comp rx=0 tx=0).stop 2024-03-29T08:41:31.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:31.153+0000 7ff030a00700 1 -- 172.21.15.37:0/158294315 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff02c0fde90 msgr2=0x7ff02c115d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:41:31.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:31.153+0000 7ff030a00700 1 --2- 172.21.15.37:0/158294315 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff02c0fde90 0x7ff02c115d20 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7ff01c000c00 tx=0x7ff01c009e30 comp rx=0 tx=0).stop 2024-03-29T08:41:31.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:31.154+0000 7ff030a00700 1 -- 172.21.15.37:0/158294315 shutdown_connections 2024-03-29T08:41:31.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:31.154+0000 7ff030a00700 1 --2- 172.21.15.37:0/158294315 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7ff01006cc30 0x7ff01006f0e0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:41:31.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:31.154+0000 7ff030a00700 1 --2- 172.21.15.37:0/158294315 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff02c0fde90 0x7ff02c115d20 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:41:31.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:31.154+0000 7ff030a00700 1 --2- 172.21.15.37:0/158294315 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff02c0fe830 0x7ff02c116260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:41:31.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:31.154+0000 7ff030a00700 1 -- 172.21.15.37:0/158294315 >> 172.21.15.37:0/158294315 conn(0x7ff02c072bc0 msgr2=0x7ff02c0fb8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:41:31.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:31.154+0000 7ff030a00700 1 -- 172.21.15.37:0/158294315 shutdown_connections 2024-03-29T08:41:31.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:31.154+0000 7ff030a00700 1 -- 172.21.15.37:0/158294315 wait complete. 2024-03-29T08:41:31.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:31.292+0000 7f9506400700 1 -- 172.21.15.37:0/1753635738 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f95001037c0 msgr2=0x7f95000fce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:41:31.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:31.292+0000 7f9506400700 1 --2- 172.21.15.37:0/1753635738 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f95001037c0 0x7f95000fce30 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f94ec004940 tx=0x7f94ec009e30 comp rx=0 tx=0).stop 2024-03-29T08:41:31.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:31.292+0000 7f9506400700 1 -- 172.21.15.37:0/1753635738 shutdown_connections 2024-03-29T08:41:31.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:31.292+0000 7f9506400700 1 --2- 172.21.15.37:0/1753635738 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f95001037c0 0x7f95000fce30 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:41:31.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:31.292+0000 7f9506400700 1 --2- 172.21.15.37:0/1753635738 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9500102e20 0x7f95001031f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:41:31.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:31.292+0000 7f9506400700 1 -- 172.21.15.37:0/1753635738 >> 172.21.15.37:0/1753635738 conn(0x7f95000f8970 msgr2=0x7f95000fad80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:41:31.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:31.292+0000 7f9506400700 1 -- 172.21.15.37:0/1753635738 shutdown_connections 2024-03-29T08:41:31.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:31.292+0000 7f9506400700 1 -- 172.21.15.37:0/1753635738 wait complete. 2024-03-29T08:41:31.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:31.293+0000 7f9506400700 1 Processor -- start 2024-03-29T08:41:31.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:31.293+0000 7f9506400700 1 -- start start 2024-03-29T08:41:31.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:31.293+0000 7f9506400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9500102e20 0x7f9500115cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:41:31.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:31.293+0000 7f9506400700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f95001037c0 0x7f9500116210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:41:31.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:31.293+0000 7f9506400700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9500116860 con 0x7f9500102e20 2024-03-29T08:41:31.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:31.293+0000 7f9506400700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f950010fc30 con 0x7f95001037c0 2024-03-29T08:41:31.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:31.294+0000 7f94ff400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9500102e20 0x7f9500115cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:41:31.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:31.294+0000 7f94ff400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9500102e20 0x7f9500115cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47794/0 (socket says 172.21.15.37:47794) 2024-03-29T08:41:31.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:31.294+0000 7f94ff400700 1 -- 172.21.15.37:0/3469458513 learned_addr learned my addr 172.21.15.37:0/3469458513 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:41:31.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:31.294+0000 7f94fea00700 1 --2- 172.21.15.37:0/3469458513 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f95001037c0 0x7f9500116210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:41:31.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:31.294+0000 7f94ff400700 1 -- 172.21.15.37:0/3469458513 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f95001037c0 msgr2=0x7f9500116210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:41:31.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:31.294+0000 7f94ff400700 1 --2- 172.21.15.37:0/3469458513 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f95001037c0 0x7f9500116210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:41:31.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:31.294+0000 7f94ff400700 1 -- 172.21.15.37:0/3469458513 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94ec005040 con 0x7f9500102e20 2024-03-29T08:41:31.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:31.294+0000 7f94fea00700 1 --2- 172.21.15.37:0/3469458513 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f95001037c0 0x7f9500116210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:41:31.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:31.294+0000 7f94ff400700 1 --2- 172.21.15.37:0/3469458513 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9500102e20 0x7f9500115cd0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f94f0002a50 tx=0x7f94f00046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:41:31.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:31.295+0000 7f94f7400700 1 -- 172.21.15.37:0/3469458513 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f94f0007cc0 con 0x7f9500102e20 2024-03-29T08:41:31.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:31.295+0000 7f94f7400700 1 -- 172.21.15.37:0/3469458513 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f94f000f040 con 0x7f9500102e20 2024-03-29T08:41:31.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:31.295+0000 7f9506400700 1 -- 172.21.15.37:0/3469458513 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f950010ff10 con 0x7f9500102e20 2024-03-29T08:41:31.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:31.295+0000 7f94f7400700 1 -- 172.21.15.37:0/3469458513 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f94f0009540 con 0x7f9500102e20 2024-03-29T08:41:31.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:31.295+0000 7f9506400700 1 -- 172.21.15.37:0/3469458513 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95001104d0 con 0x7f9500102e20 2024-03-29T08:41:31.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:31.296+0000 7f9506400700 1 -- 172.21.15.37:0/3469458513 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95000fe570 con 0x7f9500102e20 2024-03-29T08:41:31.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:31.297+0000 7f94f7400700 1 -- 172.21.15.37:0/3469458513 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f94f00096a0 con 0x7f9500102e20 2024-03-29T08:41:31.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:31.298+0000 7f94f7400700 1 --2- 172.21.15.37:0/3469458513 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f94e406cc30 0x7f94e406f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:41:31.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:31.298+0000 7f94f7400700 1 -- 172.21.15.37:0/3469458513 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f94f008bde0 con 0x7f9500102e20 2024-03-29T08:41:31.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:31.298+0000 7f94fea00700 1 --2- 172.21.15.37:0/3469458513 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f94e406cc30 0x7f94e406f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:41:31.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:31.299+0000 7f94fea00700 1 --2- 172.21.15.37:0/3469458513 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f94e406cc30 0x7f94e406f0e0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f94ec00b3b0 tx=0x7f94ec009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:41:31.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:31.303+0000 7f94f7400700 1 -- 172.21.15.37:0/3469458513 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f94f0059830 con 0x7f9500102e20 2024-03-29T08:41:31.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:31.603+0000 7f9506400700 1 -- 172.21.15.37:0/3469458513 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f950004bed0 con 0x7f9500102e20 2024-03-29T08:41:31.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:31.603+0000 7f94f7400700 1 -- 172.21.15.37:0/3469458513 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+3583 (secure 0 0 0) 0x7f94f00593c0 con 0x7f9500102e20 2024-03-29T08:41:31.603 INFO:teuthology.orchestra.run.smithi037.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem is offline 2024-03-29T08:41:31.603 INFO:teuthology.orchestra.run.smithi037.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-03-29T08:41:31.604 INFO:teuthology.orchestra.run.smithi037.stdout: host smithi148 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/mon.smithi148/config 2024-03-29T08:41:31.604 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/podman: timeout after 895 seconds 2024-03-29T08:41:31.604 INFO:teuthology.orchestra.run.smithi037.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59 -e NODE_NAME=smithi148 -e CEPH_USE_RANDOM_NONCE=1 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4:/var/run/ceph:z -v /var/log/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4:/var/log/ceph:z -v /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpkx2rz2d4:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59 inventory --format=json-pretty --filter-for-batch 2024-03-29T08:41:31.604 INFO:teuthology.orchestra.run.smithi037.stdout:Traceback (most recent call last): 2024-03-29T08:41:31.604 INFO:teuthology.orchestra.run.smithi037.stdout: File "/usr/lib64/python3.6/runpy.py", line 193, in _run_module_as_main 2024-03-29T08:41:31.604 INFO:teuthology.orchestra.run.smithi037.stdout: "__main__", mod_spec) 2024-03-29T08:41:31.604 INFO:teuthology.orchestra.run.smithi037.stdout: File "/usr/lib64/python3.6/runpy.py", line 85, in _run_code 2024-03-29T08:41:31.604 INFO:teuthology.orchestra.run.smithi037.stdout: exec(code, run_globals) 2024-03-29T08:41:31.604 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 10889, in 2024-03-29T08:41:31.604 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 10877, in main 2024-03-29T08:41:31.604 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 2576, in _infer_config 2024-03-29T08:41:31.604 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 2492, in _infer_fsid 2024-03-29T08:41:31.604 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 2604, in _infer_image 2024-03-29T08:41:31.604 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 2479, in _validate_fsid 2024-03-29T08:41:31.604 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 7145, in command_ceph_volume 2024-03-29T08:41:31.605 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 2267, in call_throws 2024-03-29T08:41:31.605 INFO:teuthology.orchestra.run.smithi037.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59 -e NODE_NAME=smithi148 -e CEPH_USE_RANDOM_NONCE=1 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4:/var/run/ceph:z -v /var/log/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4:/var/log/ceph:z -v /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpkx2rz2d4:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59 inventory --format=json-pretty --filter-for-batch 2024-03-29T08:41:31.605 INFO:teuthology.orchestra.run.smithi037.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-03-29T08:41:31.605 INFO:teuthology.orchestra.run.smithi037.stdout: fs cephfs is degraded 2024-03-29T08:41:31.605 INFO:teuthology.orchestra.run.smithi037.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-03-29T08:41:31.605 INFO:teuthology.orchestra.run.smithi037.stdout: fs cephfs is offline because no MDS is active for it. 2024-03-29T08:41:31.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:31.606+0000 7f9506400700 1 -- 172.21.15.37:0/3469458513 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f94e406cc30 msgr2=0x7f94e406f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:41:31.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:31.606+0000 7f9506400700 1 --2- 172.21.15.37:0/3469458513 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f94e406cc30 0x7f94e406f0e0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f94ec00b3b0 tx=0x7f94ec009e30 comp rx=0 tx=0).stop 2024-03-29T08:41:31.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:31.606+0000 7f9506400700 1 -- 172.21.15.37:0/3469458513 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9500102e20 msgr2=0x7f9500115cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:41:31.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:31.606+0000 7f9506400700 1 --2- 172.21.15.37:0/3469458513 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9500102e20 0x7f9500115cd0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f94f0002a50 tx=0x7f94f00046b0 comp rx=0 tx=0).stop 2024-03-29T08:41:31.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:31.606+0000 7f9506400700 1 -- 172.21.15.37:0/3469458513 shutdown_connections 2024-03-29T08:41:31.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:31.606+0000 7f9506400700 1 --2- 172.21.15.37:0/3469458513 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f94e406cc30 0x7f94e406f0e0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:41:31.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:31.606+0000 7f9506400700 1 --2- 172.21.15.37:0/3469458513 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9500102e20 0x7f9500115cd0 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:41:31.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:31.606+0000 7f9506400700 1 --2- 172.21.15.37:0/3469458513 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f95001037c0 0x7f9500116210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:41:31.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:31.606+0000 7f9506400700 1 -- 172.21.15.37:0/3469458513 >> 172.21.15.37:0/3469458513 conn(0x7f95000f8970 msgr2=0x7f95000f9960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:41:31.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:31.606+0000 7f9506400700 1 -- 172.21.15.37:0/3469458513 shutdown_connections 2024-03-29T08:41:31.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:41:31.606+0000 7f9506400700 1 -- 172.21.15.37:0/3469458513 wait complete. 2024-03-29T08:41:31.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:41:31 smithi037 ceph-mon[142325]: from='client.35662 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:41:31.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:41:31 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/2999696942' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T08:41:32.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:41:31 smithi148 ceph-mon[106692]: from='client.35662 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:41:32.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:41:31 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/2999696942' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T08:41:32.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:41:32 smithi037 ceph-mon[142325]: from='client.35674 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:41:32.928 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:41:32 smithi037 ceph-mon[142325]: pgmap v920: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:41:32.928 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:41:32 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/3469458513' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T08:41:33.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:41:32 smithi148 ceph-mon[106692]: from='client.35674 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:41:33.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:41:32 smithi148 ceph-mon[106692]: pgmap v920: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:41:33.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:41:32 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/3469458513' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T08:41:34.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:41:34 smithi037 ceph-mon[142325]: pgmap v921: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:41:35.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:41:34 smithi148 ceph-mon[106692]: pgmap v921: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:41:36.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:41:36 smithi037 ceph-mon[142325]: pgmap v922: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:41:37.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:41:36 smithi148 ceph-mon[106692]: pgmap v922: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:41:38.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:41:38 smithi037 ceph-mon[142325]: pgmap v923: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:41:39.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:41:38 smithi148 ceph-mon[106692]: pgmap v923: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:41:40.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:41:40 smithi037 ceph-mon[142325]: pgmap v924: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:41:40.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:41:40 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:41:41.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:41:40 smithi148 ceph-mon[106692]: pgmap v924: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:41:41.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:41:40 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:41:42.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:41:42 smithi037 ceph-mon[142325]: pgmap v925: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:41:43.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:41:42 smithi148 ceph-mon[106692]: pgmap v925: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:41:44.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:41:44 smithi037 ceph-mon[142325]: pgmap v926: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:41:45.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:41:44 smithi148 ceph-mon[106692]: pgmap v926: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:41:46.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:41:46 smithi037 ceph-mon[142325]: pgmap v927: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:41:47.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:41:46 smithi148 ceph-mon[106692]: pgmap v927: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:41:48.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:41:48 smithi037 ceph-mon[142325]: pgmap v928: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:41:49.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:41:48 smithi148 ceph-mon[106692]: pgmap v928: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:41:50.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:41:50 smithi037 ceph-mon[142325]: pgmap v929: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:41:51.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:41:50 smithi148 ceph-mon[106692]: pgmap v929: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:41:52.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:41:52 smithi037 ceph-mon[142325]: pgmap v930: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:41:53.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:41:52 smithi148 ceph-mon[106692]: pgmap v930: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:41:54.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:41:54 smithi037 ceph-mon[142325]: pgmap v931: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:41:55.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:41:54 smithi148 ceph-mon[106692]: pgmap v931: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:41:55.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:41:55 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:41:56.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:41:55 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:41:56.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:41:56 smithi037 ceph-mon[142325]: pgmap v932: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:41:57.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:41:56 smithi148 ceph-mon[106692]: pgmap v932: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:41:58.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:41:58 smithi037 ceph-mon[142325]: pgmap v933: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:41:59.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:41:58 smithi148 ceph-mon[106692]: pgmap v933: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:42:00.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:42:00 smithi037 ceph-mon[142325]: pgmap v934: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:42:01.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:42:00 smithi148 ceph-mon[106692]: pgmap v934: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:42:01.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:01.757+0000 7f4cc8400700 1 -- 172.21.15.37:0/1946221123 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4cc0102e00 msgr2=0x7f4cc01031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:42:01.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:01.757+0000 7f4cc8400700 1 --2- 172.21.15.37:0/1946221123 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4cc0102e00 0x7f4cc01031d0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f4cb4004940 tx=0x7f4cb4009e30 comp rx=0 tx=0).stop 2024-03-29T08:42:01.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:01.757+0000 7f4cc8400700 1 -- 172.21.15.37:0/1946221123 shutdown_connections 2024-03-29T08:42:01.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:01.757+0000 7f4cc8400700 1 --2- 172.21.15.37:0/1946221123 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4cc01037a0 0x7f4cc00fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:42:01.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:01.757+0000 7f4cc8400700 1 --2- 172.21.15.37:0/1946221123 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4cc0102e00 0x7f4cc01031d0 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:42:01.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:01.757+0000 7f4cc8400700 1 -- 172.21.15.37:0/1946221123 >> 172.21.15.37:0/1946221123 conn(0x7f4cc00f89b0 msgr2=0x7f4cc00fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:42:01.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:01.757+0000 7f4cc8400700 1 -- 172.21.15.37:0/1946221123 shutdown_connections 2024-03-29T08:42:01.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:01.757+0000 7f4cc8400700 1 -- 172.21.15.37:0/1946221123 wait complete. 2024-03-29T08:42:01.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:01.758+0000 7f4cc8400700 1 Processor -- start 2024-03-29T08:42:01.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:01.758+0000 7f4cc8400700 1 -- start start 2024-03-29T08:42:01.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:01.758+0000 7f4cc8400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4cc0102e00 0x7f4cc010af40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:42:01.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:01.758+0000 7f4cc8400700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4cc01037a0 0x7f4cc010b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:42:01.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:01.758+0000 7f4cc8400700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cc010f0d0 con 0x7f4cc0102e00 2024-03-29T08:42:01.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:01.758+0000 7f4cc8400700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cc010b9c0 con 0x7f4cc01037a0 2024-03-29T08:42:01.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:01.759+0000 7f4cc5a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4cc0102e00 0x7f4cc010af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:42:01.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:01.759+0000 7f4cc5a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4cc0102e00 0x7f4cc010af40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57204/0 (socket says 172.21.15.37:57204) 2024-03-29T08:42:01.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:01.759+0000 7f4cc5a00700 1 -- 172.21.15.37:0/501394455 learned_addr learned my addr 172.21.15.37:0/501394455 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:42:01.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:01.759+0000 7f4cc5000700 1 --2- 172.21.15.37:0/501394455 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4cc01037a0 0x7f4cc010b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:42:01.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:01.759+0000 7f4cc5a00700 1 -- 172.21.15.37:0/501394455 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4cc01037a0 msgr2=0x7f4cc010b480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:42:01.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:01.759+0000 7f4cc5a00700 1 --2- 172.21.15.37:0/501394455 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4cc01037a0 0x7f4cc010b480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:42:01.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:01.759+0000 7f4cc5a00700 1 -- 172.21.15.37:0/501394455 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cb4005040 con 0x7f4cc0102e00 2024-03-29T08:42:01.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:01.759+0000 7f4cc5a00700 1 --2- 172.21.15.37:0/501394455 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4cc0102e00 0x7f4cc010af40 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f4cb4009180 tx=0x7f4cb4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:42:01.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:01.759+0000 7f4cbd600700 1 -- 172.21.15.37:0/501394455 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4cb400e040 con 0x7f4cc0102e00 2024-03-29T08:42:01.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:01.759+0000 7f4cbd600700 1 -- 172.21.15.37:0/501394455 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f4cb4004290 con 0x7f4cc0102e00 2024-03-29T08:42:01.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:01.759+0000 7f4cc8400700 1 -- 172.21.15.37:0/501394455 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4cc010bc40 con 0x7f4cc0102e00 2024-03-29T08:42:01.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:01.760+0000 7f4cbd600700 1 -- 172.21.15.37:0/501394455 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4cb401c3f0 con 0x7f4cc0102e00 2024-03-29T08:42:01.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:01.760+0000 7f4cc8400700 1 -- 172.21.15.37:0/501394455 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4cc011e640 con 0x7f4cc0102e00 2024-03-29T08:42:01.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:01.761+0000 7f4cc8400700 1 -- 172.21.15.37:0/501394455 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4cc00fe550 con 0x7f4cc0102e00 2024-03-29T08:42:01.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:01.762+0000 7f4cbd600700 1 -- 172.21.15.37:0/501394455 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f4cb4013790 con 0x7f4cc0102e00 2024-03-29T08:42:01.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:01.762+0000 7f4cbd600700 1 --2- 172.21.15.37:0/501394455 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f4ca806cc20 0x7f4ca806f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:42:01.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:01.763+0000 7f4cbd600700 1 -- 172.21.15.37:0/501394455 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f4cb408d2a0 con 0x7f4cc0102e00 2024-03-29T08:42:01.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:01.763+0000 7f4cc5000700 1 --2- 172.21.15.37:0/501394455 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f4ca806cc20 0x7f4ca806f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:42:01.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:01.763+0000 7f4cc5000700 1 --2- 172.21.15.37:0/501394455 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f4ca806cc20 0x7f4ca806f0d0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f4cc010c880 tx=0x7f4cac009ee0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:42:01.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:01.767+0000 7f4cbd600700 1 -- 172.21.15.37:0/501394455 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f4cb40572e0 con 0x7f4cc0102e00 2024-03-29T08:42:01.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:01.998+0000 7f4cc8400700 1 -- 172.21.15.37:0/501394455 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4cc010c630 con 0x7f4ca806cc20 2024-03-29T08:42:01.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:01.999+0000 7f4cbd600700 1 -- 172.21.15.37:0/501394455 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f4cc010c630 con 0x7f4ca806cc20 2024-03-29T08:42:02.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:02.002+0000 7f4cc8400700 1 -- 172.21.15.37:0/501394455 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f4ca806cc20 msgr2=0x7f4ca806f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:42:02.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:02.002+0000 7f4cc8400700 1 --2- 172.21.15.37:0/501394455 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f4ca806cc20 0x7f4ca806f0d0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f4cc010c880 tx=0x7f4cac009ee0 comp rx=0 tx=0).stop 2024-03-29T08:42:02.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:02.002+0000 7f4cc8400700 1 -- 172.21.15.37:0/501394455 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4cc0102e00 msgr2=0x7f4cc010af40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:42:02.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:02.002+0000 7f4cc8400700 1 --2- 172.21.15.37:0/501394455 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4cc0102e00 0x7f4cc010af40 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f4cb4009180 tx=0x7f4cb4009e30 comp rx=0 tx=0).stop 2024-03-29T08:42:02.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:02.002+0000 7f4cc8400700 1 -- 172.21.15.37:0/501394455 shutdown_connections 2024-03-29T08:42:02.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:02.002+0000 7f4cc8400700 1 --2- 172.21.15.37:0/501394455 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f4ca806cc20 0x7f4ca806f0d0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:42:02.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:02.002+0000 7f4cc8400700 1 --2- 172.21.15.37:0/501394455 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4cc0102e00 0x7f4cc010af40 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:42:02.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:02.002+0000 7f4cc8400700 1 --2- 172.21.15.37:0/501394455 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4cc01037a0 0x7f4cc010b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:42:02.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:02.002+0000 7f4cc8400700 1 -- 172.21.15.37:0/501394455 >> 172.21.15.37:0/501394455 conn(0x7f4cc00f89b0 msgr2=0x7f4cc00f9980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:42:02.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:02.002+0000 7f4cc8400700 1 -- 172.21.15.37:0/501394455 shutdown_connections 2024-03-29T08:42:02.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:02.002+0000 7f4cc8400700 1 -- 172.21.15.37:0/501394455 wait complete. 2024-03-29T08:42:02.016 INFO:teuthology.orchestra.run.smithi037.stdout:true 2024-03-29T08:42:02.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:02.139+0000 7f38fe400700 1 -- 172.21.15.37:0/3979680952 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f38f81037a0 msgr2=0x7f38f80fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:42:02.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:02.139+0000 7f38fe400700 1 --2- 172.21.15.37:0/3979680952 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f38f81037a0 0x7f38f80fce10 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f38e4004940 tx=0x7f38e4009e30 comp rx=0 tx=0).stop 2024-03-29T08:42:02.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:02.140+0000 7f38fe400700 1 -- 172.21.15.37:0/3979680952 shutdown_connections 2024-03-29T08:42:02.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:02.140+0000 7f38fe400700 1 --2- 172.21.15.37:0/3979680952 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f38f81037a0 0x7f38f80fce10 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:42:02.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:02.140+0000 7f38fe400700 1 --2- 172.21.15.37:0/3979680952 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f38f8102e00 0x7f38f81031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:42:02.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:02.140+0000 7f38fe400700 1 -- 172.21.15.37:0/3979680952 >> 172.21.15.37:0/3979680952 conn(0x7f38f80f89b0 msgr2=0x7f38f80fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:42:02.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:02.140+0000 7f38fe400700 1 -- 172.21.15.37:0/3979680952 shutdown_connections 2024-03-29T08:42:02.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:02.140+0000 7f38fe400700 1 -- 172.21.15.37:0/3979680952 wait complete. 2024-03-29T08:42:02.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:02.141+0000 7f38fe400700 1 Processor -- start 2024-03-29T08:42:02.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:02.141+0000 7f38fe400700 1 -- start start 2024-03-29T08:42:02.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:02.142+0000 7f38fe400700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f38f8102e00 0x7f38f8113b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:42:02.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:02.142+0000 7f38fe400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f38f81037a0 0x7f38f8114040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:42:02.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:02.142+0000 7f38fe400700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38f8114690 con 0x7f38f81037a0 2024-03-29T08:42:02.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:02.142+0000 7f38fe400700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38f810da80 con 0x7f38f8102e00 2024-03-29T08:42:02.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:02.142+0000 7f38f6a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f38f81037a0 0x7f38f8114040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:42:02.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:02.142+0000 7f38f6a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f38f81037a0 0x7f38f8114040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57216/0 (socket says 172.21.15.37:57216) 2024-03-29T08:42:02.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:02.142+0000 7f38f6a00700 1 -- 172.21.15.37:0/2033613926 learned_addr learned my addr 172.21.15.37:0/2033613926 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:42:02.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:02.142+0000 7f38f7400700 1 --2- 172.21.15.37:0/2033613926 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f38f8102e00 0x7f38f8113b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:42:02.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:02.142+0000 7f38f6a00700 1 -- 172.21.15.37:0/2033613926 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f38f8102e00 msgr2=0x7f38f8113b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:42:02.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:02.142+0000 7f38f6a00700 1 --2- 172.21.15.37:0/2033613926 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f38f8102e00 0x7f38f8113b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:42:02.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:02.142+0000 7f38f6a00700 1 -- 172.21.15.37:0/2033613926 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38e4005040 con 0x7f38f81037a0 2024-03-29T08:42:02.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:02.143+0000 7f38f6a00700 1 --2- 172.21.15.37:0/2033613926 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f38f81037a0 0x7f38f8114040 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f38e4009390 tx=0x7f38e4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:42:02.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:02.143+0000 7f38ef400700 1 -- 172.21.15.37:0/2033613926 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f38e400e040 con 0x7f38f81037a0 2024-03-29T08:42:02.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:02.143+0000 7f38ef400700 1 -- 172.21.15.37:0/2033613926 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f38e4013420 con 0x7f38f81037a0 2024-03-29T08:42:02.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:02.143+0000 7f38fe400700 1 -- 172.21.15.37:0/2033613926 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38f810dd00 con 0x7f38f81037a0 2024-03-29T08:42:02.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:02.143+0000 7f38ef400700 1 -- 172.21.15.37:0/2033613926 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f38e401c630 con 0x7f38f81037a0 2024-03-29T08:42:02.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:02.143+0000 7f38fe400700 1 -- 172.21.15.37:0/2033613926 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38f810e2c0 con 0x7f38f81037a0 2024-03-29T08:42:02.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:02.144+0000 7f38fe400700 1 -- 172.21.15.37:0/2033613926 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38f80fe550 con 0x7f38f81037a0 2024-03-29T08:42:02.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:02.145+0000 7f38ef400700 1 -- 172.21.15.37:0/2033613926 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f38e4013a00 con 0x7f38f81037a0 2024-03-29T08:42:02.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:02.146+0000 7f38ef400700 1 --2- 172.21.15.37:0/2033613926 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f38dc06cc30 0x7f38dc06f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:42:02.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:02.146+0000 7f38ef400700 1 -- 172.21.15.37:0/2033613926 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f38e408d390 con 0x7f38f81037a0 2024-03-29T08:42:02.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:02.146+0000 7f38f7400700 1 --2- 172.21.15.37:0/2033613926 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f38dc06cc30 0x7f38dc06f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:42:02.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:02.147+0000 7f38f7400700 1 --2- 172.21.15.37:0/2033613926 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f38dc06cc30 0x7f38dc06f0e0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f38e8004d60 tx=0x7f38e800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:42:02.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:02.151+0000 7f38ef400700 1 -- 172.21.15.37:0/2033613926 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f38e405ade0 con 0x7f38f81037a0 2024-03-29T08:42:02.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:02.383+0000 7f38fe400700 1 -- 172.21.15.37:0/2033613926 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f38f8109da0 con 0x7f38dc06cc30 2024-03-29T08:42:02.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:02.384+0000 7f38ef400700 1 -- 172.21.15.37:0/2033613926 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f38f8109da0 con 0x7f38dc06cc30 2024-03-29T08:42:02.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:02.386+0000 7f38fe400700 1 -- 172.21.15.37:0/2033613926 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f38dc06cc30 msgr2=0x7f38dc06f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:42:02.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:02.386+0000 7f38fe400700 1 --2- 172.21.15.37:0/2033613926 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f38dc06cc30 0x7f38dc06f0e0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f38e8004d60 tx=0x7f38e800b040 comp rx=0 tx=0).stop 2024-03-29T08:42:02.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:02.386+0000 7f38fe400700 1 -- 172.21.15.37:0/2033613926 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f38f81037a0 msgr2=0x7f38f8114040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:42:02.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:02.386+0000 7f38fe400700 1 --2- 172.21.15.37:0/2033613926 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f38f81037a0 0x7f38f8114040 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f38e4009390 tx=0x7f38e4009e30 comp rx=0 tx=0).stop 2024-03-29T08:42:02.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:02.386+0000 7f38fe400700 1 -- 172.21.15.37:0/2033613926 shutdown_connections 2024-03-29T08:42:02.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:02.386+0000 7f38fe400700 1 --2- 172.21.15.37:0/2033613926 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f38dc06cc30 0x7f38dc06f0e0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:42:02.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:02.386+0000 7f38fe400700 1 --2- 172.21.15.37:0/2033613926 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f38f8102e00 0x7f38f8113b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:42:02.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:02.386+0000 7f38fe400700 1 --2- 172.21.15.37:0/2033613926 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f38f81037a0 0x7f38f8114040 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:42:02.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:02.386+0000 7f38fe400700 1 -- 172.21.15.37:0/2033613926 >> 172.21.15.37:0/2033613926 conn(0x7f38f80f89b0 msgr2=0x7f38f80f9ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:42:02.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:02.386+0000 7f38fe400700 1 -- 172.21.15.37:0/2033613926 shutdown_connections 2024-03-29T08:42:02.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:02.387+0000 7f38fe400700 1 -- 172.21.15.37:0/2033613926 wait complete. 2024-03-29T08:42:02.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:02.525+0000 7fa0c5800700 1 -- 172.21.15.37:0/2103152667 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa0c01037a0 msgr2=0x7fa0c00fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:42:02.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:02.525+0000 7fa0c5800700 1 --2- 172.21.15.37:0/2103152667 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa0c01037a0 0x7fa0c00fce10 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fa0ac004940 tx=0x7fa0ac009e30 comp rx=0 tx=0).stop 2024-03-29T08:42:02.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:02.525+0000 7fa0c5800700 1 -- 172.21.15.37:0/2103152667 shutdown_connections 2024-03-29T08:42:02.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:02.525+0000 7fa0c5800700 1 --2- 172.21.15.37:0/2103152667 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa0c01037a0 0x7fa0c00fce10 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:42:02.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:02.525+0000 7fa0c5800700 1 --2- 172.21.15.37:0/2103152667 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa0c0102e00 0x7fa0c01031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:42:02.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:02.525+0000 7fa0c5800700 1 -- 172.21.15.37:0/2103152667 >> 172.21.15.37:0/2103152667 conn(0x7fa0c00f89b0 msgr2=0x7fa0c00fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:42:02.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:02.525+0000 7fa0c5800700 1 -- 172.21.15.37:0/2103152667 shutdown_connections 2024-03-29T08:42:02.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:02.525+0000 7fa0c5800700 1 -- 172.21.15.37:0/2103152667 wait complete. 2024-03-29T08:42:02.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:02.526+0000 7fa0c5800700 1 Processor -- start 2024-03-29T08:42:02.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:02.526+0000 7fa0c5800700 1 -- start start 2024-03-29T08:42:02.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:02.527+0000 7fa0c5800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa0c0102e00 0x7fa0c0115cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:42:02.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:02.527+0000 7fa0c5800700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa0c01037a0 0x7fa0c0116230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:42:02.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:02.527+0000 7fa0c5800700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0c0116880 con 0x7fa0c0102e00 2024-03-29T08:42:02.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:02.527+0000 7fa0c5800700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0c010fc50 con 0x7fa0c01037a0 2024-03-29T08:42:02.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:02.527+0000 7fa0bea00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa0c0102e00 0x7fa0c0115cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:42:02.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:02.527+0000 7fa0bea00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa0c0102e00 0x7fa0c0115cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57234/0 (socket says 172.21.15.37:57234) 2024-03-29T08:42:02.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:02.527+0000 7fa0bea00700 1 -- 172.21.15.37:0/2662447206 learned_addr learned my addr 172.21.15.37:0/2662447206 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:42:02.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:02.527+0000 7fa0be000700 1 --2- 172.21.15.37:0/2662447206 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa0c01037a0 0x7fa0c0116230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:42:02.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:02.527+0000 7fa0bea00700 1 -- 172.21.15.37:0/2662447206 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa0c01037a0 msgr2=0x7fa0c0116230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:42:02.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:02.527+0000 7fa0bea00700 1 --2- 172.21.15.37:0/2662447206 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa0c01037a0 0x7fa0c0116230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:42:02.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:02.527+0000 7fa0bea00700 1 -- 172.21.15.37:0/2662447206 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0ac005040 con 0x7fa0c0102e00 2024-03-29T08:42:02.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:02.528+0000 7fa0bea00700 1 --2- 172.21.15.37:0/2662447206 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa0c0102e00 0x7fa0c0115cf0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fa0b0002a50 tx=0x7fa0b00046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:42:02.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:02.528+0000 7fa0b6a00700 1 -- 172.21.15.37:0/2662447206 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa0b0007cc0 con 0x7fa0c0102e00 2024-03-29T08:42:02.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:02.528+0000 7fa0b6a00700 1 -- 172.21.15.37:0/2662447206 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fa0b000f040 con 0x7fa0c0102e00 2024-03-29T08:42:02.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:02.528+0000 7fa0c5800700 1 -- 172.21.15.37:0/2662447206 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0c010ff30 con 0x7fa0c0102e00 2024-03-29T08:42:02.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:02.528+0000 7fa0b6a00700 1 -- 172.21.15.37:0/2662447206 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa0b000d9c0 con 0x7fa0c0102e00 2024-03-29T08:42:02.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:02.528+0000 7fa0c5800700 1 -- 172.21.15.37:0/2662447206 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0c01104f0 con 0x7fa0c0102e00 2024-03-29T08:42:02.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:02.529+0000 7fa0c5800700 1 -- 172.21.15.37:0/2662447206 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0c00fe550 con 0x7fa0c0102e00 2024-03-29T08:42:02.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:02.530+0000 7fa0b6a00700 1 -- 172.21.15.37:0/2662447206 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fa0b000db20 con 0x7fa0c0102e00 2024-03-29T08:42:02.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:02.531+0000 7fa0b6a00700 1 --2- 172.21.15.37:0/2662447206 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fa0a406cc30 0x7fa0a406f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:42:02.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:02.531+0000 7fa0b6a00700 1 -- 172.21.15.37:0/2662447206 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fa0b008bde0 con 0x7fa0c0102e00 2024-03-29T08:42:02.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:02.531+0000 7fa0be000700 1 --2- 172.21.15.37:0/2662447206 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fa0a406cc30 0x7fa0a406f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:42:02.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:02.532+0000 7fa0be000700 1 --2- 172.21.15.37:0/2662447206 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fa0a406cc30 0x7fa0a406f0e0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fa0c0111590 tx=0x7fa0ac009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:42:02.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:02.536+0000 7fa0b6a00700 1 -- 172.21.15.37:0/2662447206 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fa0b00597b0 con 0x7fa0c0102e00 2024-03-29T08:42:02.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:02.759+0000 7fa0c5800700 1 -- 172.21.15.37:0/2662447206 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fa0c004bf40 con 0x7fa0a406cc30 2024-03-29T08:42:02.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:02.769+0000 7fa0b6a00700 1 -- 172.21.15.37:0/2662447206 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7fa0c004bf40 con 0x7fa0a406cc30 2024-03-29T08:42:02.769 INFO:teuthology.orchestra.run.smithi037.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-29T08:42:02.769 INFO:teuthology.orchestra.run.smithi037.stdout:alertmanager.smithi037 smithi037 *:9093,9094 running (31m) 8m ago 44m 14.3M - 0.25.0 c8568f914cd2 2f9f9746ed0f 2024-03-29T08:42:02.769 INFO:teuthology.orchestra.run.smithi037.stdout:crash.smithi037 smithi037 running (28m) 8m ago 45m 6647k - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 178d416a3186 2024-03-29T08:42:02.769 INFO:teuthology.orchestra.run.smithi037.stdout:crash.smithi148 smithi148 running (28m) 8m ago 43m 6656k - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 23666989d58f 2024-03-29T08:42:02.769 INFO:teuthology.orchestra.run.smithi037.stdout:grafana.smithi037 smithi037 *:3000 running (31m) 8m ago 44m 74.5M - 9.4.7 954c08fa6188 6b976fe49952 2024-03-29T08:42:02.769 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi037.iifsmh smithi037 running (24m) 8m ago 38m 16.4M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 2d97f562cc5b 2024-03-29T08:42:02.770 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi037.mmwgxe smithi037 running (24m) 8m ago 38m 16.7M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff ec37037f46e8 2024-03-29T08:42:02.770 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi148.qukpjh smithi148 running (8m) 8m ago 38m 10.7M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 3257219bc046 2024-03-29T08:42:02.770 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi148.sqteih smithi148 running (23m) 8m ago 38m 16.3M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 44c15c77037a 2024-03-29T08:42:02.770 INFO:teuthology.orchestra.run.smithi037.stdout:mgr.smithi037.wjikii smithi037 *:8443,9283,8765 running (34m) 8m ago 46m 510M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff c5a1417b9417 2024-03-29T08:42:02.770 INFO:teuthology.orchestra.run.smithi037.stdout:mgr.smithi148.onhgvm smithi148 *:8443,9283,8765 running (33m) 8m ago 43m 456M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff fcfdbce062bd 2024-03-29T08:42:02.770 INFO:teuthology.orchestra.run.smithi037.stdout:mon.smithi037 smithi037 running (30m) 8m ago 46m 70.2M 2048M 18.2.2-1015-gd0e392d9 1cc2f45c17ff eea2060783af 2024-03-29T08:42:02.770 INFO:teuthology.orchestra.run.smithi037.stdout:mon.smithi148 smithi148 running (29m) 8m ago 43m 50.4M 2048M 18.2.2-1015-gd0e392d9 1cc2f45c17ff ad4b2c5748be 2024-03-29T08:42:02.770 INFO:teuthology.orchestra.run.smithi037.stdout:node-exporter.smithi037 smithi037 *:9100 running (32m) 8m ago 45m 8514k - 1.5.0 0da6a335fe13 26b8ae9cb878 2024-03-29T08:42:02.770 INFO:teuthology.orchestra.run.smithi037.stdout:node-exporter.smithi148 smithi148 *:9100 running (32m) 8m ago 43m 8464k - 1.5.0 0da6a335fe13 98414e4e64ed 2024-03-29T08:42:02.770 INFO:teuthology.orchestra.run.smithi037.stdout:osd.0 smithi037 running (28m) 8m ago 42m 108M 1619M 18.2.2-1015-gd0e392d9 1cc2f45c17ff c0c584f6245c 2024-03-29T08:42:02.770 INFO:teuthology.orchestra.run.smithi037.stdout:osd.1 smithi037 running (27m) 8m ago 42m 104M 1619M 18.2.2-1015-gd0e392d9 1cc2f45c17ff a4a2ff4cc83d 2024-03-29T08:42:02.771 INFO:teuthology.orchestra.run.smithi037.stdout:osd.2 smithi037 running (26m) 8m ago 41m 85.9M 1619M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 8b1bc71aa79e 2024-03-29T08:42:02.771 INFO:teuthology.orchestra.run.smithi037.stdout:osd.3 smithi148 running (26m) 8m ago 41m 109M 2642M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 1c656ab8ff86 2024-03-29T08:42:02.771 INFO:teuthology.orchestra.run.smithi037.stdout:osd.4 smithi148 running (25m) 8m ago 40m 101M 2642M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 966a9de11d22 2024-03-29T08:42:02.771 INFO:teuthology.orchestra.run.smithi037.stdout:osd.5 smithi148 running (24m) 8m ago 40m 89.4M 2642M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 474c4f9212ce 2024-03-29T08:42:02.771 INFO:teuthology.orchestra.run.smithi037.stdout:prometheus.smithi037 smithi037 *:9095 running (32m) 8m ago 44m 61.0M - 2.43.0 a07b618ecd1d 30c0223ec589 2024-03-29T08:42:02.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:02.771+0000 7fa0c5800700 1 -- 172.21.15.37:0/2662447206 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fa0a406cc30 msgr2=0x7fa0a406f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:42:02.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:02.771+0000 7fa0c5800700 1 --2- 172.21.15.37:0/2662447206 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fa0a406cc30 0x7fa0a406f0e0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fa0c0111590 tx=0x7fa0ac009e30 comp rx=0 tx=0).stop 2024-03-29T08:42:02.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:02.771+0000 7fa0c5800700 1 -- 172.21.15.37:0/2662447206 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa0c0102e00 msgr2=0x7fa0c0115cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:42:02.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:02.771+0000 7fa0c5800700 1 --2- 172.21.15.37:0/2662447206 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa0c0102e00 0x7fa0c0115cf0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fa0b0002a50 tx=0x7fa0b00046b0 comp rx=0 tx=0).stop 2024-03-29T08:42:02.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:02.771+0000 7fa0c5800700 1 -- 172.21.15.37:0/2662447206 shutdown_connections 2024-03-29T08:42:02.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:02.771+0000 7fa0c5800700 1 --2- 172.21.15.37:0/2662447206 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fa0a406cc30 0x7fa0a406f0e0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:42:02.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:02.771+0000 7fa0c5800700 1 --2- 172.21.15.37:0/2662447206 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa0c0102e00 0x7fa0c0115cf0 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:42:02.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:02.771+0000 7fa0c5800700 1 --2- 172.21.15.37:0/2662447206 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa0c01037a0 0x7fa0c0116230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:42:02.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:02.771+0000 7fa0c5800700 1 -- 172.21.15.37:0/2662447206 >> 172.21.15.37:0/2662447206 conn(0x7fa0c00f89b0 msgr2=0x7fa0c00f9a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:42:02.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:02.772+0000 7fa0c5800700 1 -- 172.21.15.37:0/2662447206 shutdown_connections 2024-03-29T08:42:02.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:02.772+0000 7fa0c5800700 1 -- 172.21.15.37:0/2662447206 wait complete. 2024-03-29T08:42:02.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:02.907+0000 7f6d7ec00700 1 -- 172.21.15.37:0/3927052639 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6d781037a0 msgr2=0x7f6d780fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:42:02.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:02.907+0000 7f6d7ec00700 1 --2- 172.21.15.37:0/3927052639 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6d781037a0 0x7f6d780fce10 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f6d64004940 tx=0x7f6d64009e30 comp rx=0 tx=0).stop 2024-03-29T08:42:02.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:02.907+0000 7f6d7ec00700 1 -- 172.21.15.37:0/3927052639 shutdown_connections 2024-03-29T08:42:02.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:02.907+0000 7f6d7ec00700 1 --2- 172.21.15.37:0/3927052639 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6d781037a0 0x7f6d780fce10 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:42:02.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:02.907+0000 7f6d7ec00700 1 --2- 172.21.15.37:0/3927052639 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6d78102e00 0x7f6d781031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:42:02.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:02.907+0000 7f6d7ec00700 1 -- 172.21.15.37:0/3927052639 >> 172.21.15.37:0/3927052639 conn(0x7f6d780f89b0 msgr2=0x7f6d780fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:42:02.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:02.907+0000 7f6d7ec00700 1 -- 172.21.15.37:0/3927052639 shutdown_connections 2024-03-29T08:42:02.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:02.907+0000 7f6d7ec00700 1 -- 172.21.15.37:0/3927052639 wait complete. 2024-03-29T08:42:02.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:02.908+0000 7f6d7ec00700 1 Processor -- start 2024-03-29T08:42:02.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:02.908+0000 7f6d7ec00700 1 -- start start 2024-03-29T08:42:02.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:02.909+0000 7f6d7ec00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6d78102e00 0x7f6d780702c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:42:02.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:02.909+0000 7f6d7ec00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6d781037a0 0x7f6d78070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:42:02.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:02.909+0000 7f6d7ec00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d78070d90 con 0x7f6d78102e00 2024-03-29T08:42:02.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:02.909+0000 7f6d7ec00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d78070ed0 con 0x7f6d781037a0 2024-03-29T08:42:02.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:02.909+0000 7f6d77e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6d78102e00 0x7f6d780702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:42:02.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:02.909+0000 7f6d77e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6d78102e00 0x7f6d780702c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57252/0 (socket says 172.21.15.37:57252) 2024-03-29T08:42:02.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:02.909+0000 7f6d77e00700 1 -- 172.21.15.37:0/2422195924 learned_addr learned my addr 172.21.15.37:0/2422195924 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:42:02.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:02.910+0000 7f6d77e00700 1 -- 172.21.15.37:0/2422195924 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6d781037a0 msgr2=0x7f6d78070800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:42:02.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:02.910+0000 7f6d77e00700 1 --2- 172.21.15.37:0/2422195924 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6d781037a0 0x7f6d78070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:42:02.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:02.910+0000 7f6d77e00700 1 -- 172.21.15.37:0/2422195924 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d64005040 con 0x7f6d78102e00 2024-03-29T08:42:02.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:02.910+0000 7f6d77e00700 1 --2- 172.21.15.37:0/2422195924 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6d78102e00 0x7f6d780702c0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f6d6800c8c0 tx=0x7f6d680044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:42:02.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:02.910+0000 7f6d6fe00700 1 -- 172.21.15.37:0/2422195924 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6d6800a980 con 0x7f6d78102e00 2024-03-29T08:42:02.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:02.910+0000 7f6d6fe00700 1 -- 172.21.15.37:0/2422195924 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f6d6800ce60 con 0x7f6d78102e00 2024-03-29T08:42:02.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:02.910+0000 7f6d7ec00700 1 -- 172.21.15.37:0/2422195924 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d78119e00 con 0x7f6d78102e00 2024-03-29T08:42:02.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:02.910+0000 7f6d6fe00700 1 -- 172.21.15.37:0/2422195924 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6d680113a0 con 0x7f6d78102e00 2024-03-29T08:42:02.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:02.910+0000 7f6d7ec00700 1 -- 172.21.15.37:0/2422195924 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d7811a280 con 0x7f6d78102e00 2024-03-29T08:42:02.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:02.911+0000 7f6d7ec00700 1 -- 172.21.15.37:0/2422195924 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d780fe550 con 0x7f6d78102e00 2024-03-29T08:42:02.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:02.912+0000 7f6d6fe00700 1 -- 172.21.15.37:0/2422195924 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f6d68012400 con 0x7f6d78102e00 2024-03-29T08:42:02.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:02.913+0000 7f6d6fe00700 1 --2- 172.21.15.37:0/2422195924 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f6d5c06cc30 0x7f6d5c06f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:42:02.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:02.914+0000 7f6d6fe00700 1 -- 172.21.15.37:0/2422195924 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f6d6808c110 con 0x7f6d78102e00 2024-03-29T08:42:02.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:02.914+0000 7f6d77400700 1 --2- 172.21.15.37:0/2422195924 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f6d5c06cc30 0x7f6d5c06f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:42:02.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:02.914+0000 7f6d77400700 1 --2- 172.21.15.37:0/2422195924 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f6d5c06cc30 0x7f6d5c06f0e0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f6d64004f90 tx=0x7f6d64009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:42:02.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:02.918+0000 7f6d6fe00700 1 -- 172.21.15.37:0/2422195924 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f6d68056150 con 0x7f6d78102e00 2024-03-29T08:42:02.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:42:02 smithi037 ceph-mon[142325]: pgmap v935: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:42:03.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:42:02 smithi148 ceph-mon[106692]: pgmap v935: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:42:03.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:03.214+0000 7f6d7ec00700 1 -- 172.21.15.37:0/2422195924 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f6d7804bed0 con 0x7f6d78102e00 2024-03-29T08:42:03.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:03.215+0000 7f6d6fe00700 1 -- 172.21.15.37:0/2422195924 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7f6d68059770 con 0x7f6d78102e00 2024-03-29T08:42:03.215 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-29T08:42:03.215 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": { 2024-03-29T08:42:03.215 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 2 2024-03-29T08:42:03.215 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:42:03.215 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": { 2024-03-29T08:42:03.215 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 2 2024-03-29T08:42:03.215 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:42:03.215 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": { 2024-03-29T08:42:03.215 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 6 2024-03-29T08:42:03.215 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:42:03.215 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": { 2024-03-29T08:42:03.215 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 4 2024-03-29T08:42:03.215 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:42:03.216 INFO:teuthology.orchestra.run.smithi037.stdout: "overall": { 2024-03-29T08:42:03.216 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 14 2024-03-29T08:42:03.216 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-29T08:42:03.216 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-29T08:42:03.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:03.217+0000 7f6d7ec00700 1 -- 172.21.15.37:0/2422195924 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f6d5c06cc30 msgr2=0x7f6d5c06f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:42:03.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:03.217+0000 7f6d7ec00700 1 --2- 172.21.15.37:0/2422195924 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f6d5c06cc30 0x7f6d5c06f0e0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f6d64004f90 tx=0x7f6d64009e30 comp rx=0 tx=0).stop 2024-03-29T08:42:03.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:03.217+0000 7f6d7ec00700 1 -- 172.21.15.37:0/2422195924 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6d78102e00 msgr2=0x7f6d780702c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:42:03.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:03.217+0000 7f6d7ec00700 1 --2- 172.21.15.37:0/2422195924 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6d78102e00 0x7f6d780702c0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f6d6800c8c0 tx=0x7f6d680044c0 comp rx=0 tx=0).stop 2024-03-29T08:42:03.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:03.217+0000 7f6d7ec00700 1 -- 172.21.15.37:0/2422195924 shutdown_connections 2024-03-29T08:42:03.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:03.217+0000 7f6d7ec00700 1 --2- 172.21.15.37:0/2422195924 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f6d5c06cc30 0x7f6d5c06f0e0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:42:03.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:03.217+0000 7f6d7ec00700 1 --2- 172.21.15.37:0/2422195924 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6d78102e00 0x7f6d780702c0 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:42:03.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:03.217+0000 7f6d7ec00700 1 --2- 172.21.15.37:0/2422195924 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6d781037a0 0x7f6d78070800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:42:03.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:03.217+0000 7f6d7ec00700 1 -- 172.21.15.37:0/2422195924 >> 172.21.15.37:0/2422195924 conn(0x7f6d780f89b0 msgr2=0x7f6d780f9ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:42:03.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:03.217+0000 7f6d7ec00700 1 -- 172.21.15.37:0/2422195924 shutdown_connections 2024-03-29T08:42:03.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:03.217+0000 7f6d7ec00700 1 -- 172.21.15.37:0/2422195924 wait complete. 2024-03-29T08:42:03.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:03.354+0000 7f58ed600700 1 -- 172.21.15.37:0/3389084785 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f58e80fde90 msgr2=0x7f58e80fe260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:42:03.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:03.354+0000 7f58ed600700 1 --2- 172.21.15.37:0/3389084785 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f58e80fde90 0x7f58e80fe260 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f58d8004940 tx=0x7f58d8009e30 comp rx=0 tx=0).stop 2024-03-29T08:42:03.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:03.354+0000 7f58ed600700 1 -- 172.21.15.37:0/3389084785 shutdown_connections 2024-03-29T08:42:03.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:03.354+0000 7f58ed600700 1 --2- 172.21.15.37:0/3389084785 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f58e80fe830 0x7f58e8106f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:42:03.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:03.354+0000 7f58ed600700 1 --2- 172.21.15.37:0/3389084785 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f58e80fde90 0x7f58e80fe260 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:42:03.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:03.354+0000 7f58ed600700 1 -- 172.21.15.37:0/3389084785 >> 172.21.15.37:0/3389084785 conn(0x7f58e8072bc0 msgr2=0x7f58e8072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:42:03.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:03.354+0000 7f58ed600700 1 -- 172.21.15.37:0/3389084785 shutdown_connections 2024-03-29T08:42:03.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:03.354+0000 7f58ed600700 1 -- 172.21.15.37:0/3389084785 wait complete. 2024-03-29T08:42:03.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:03.355+0000 7f58ed600700 1 Processor -- start 2024-03-29T08:42:03.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:03.355+0000 7f58ed600700 1 -- start start 2024-03-29T08:42:03.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:03.355+0000 7f58ed600700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f58e80fde90 0x7f58e8113a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:42:03.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:03.355+0000 7f58ed600700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f58e80fe830 0x7f58e8113fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:42:03.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:03.355+0000 7f58ed600700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58e8114600 con 0x7f58e80fde90 2024-03-29T08:42:03.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:03.355+0000 7f58ed600700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58e810d9f0 con 0x7f58e80fe830 2024-03-29T08:42:03.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:03.355+0000 7f58e6a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f58e80fde90 0x7f58e8113a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:42:03.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:03.355+0000 7f58e6a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f58e80fde90 0x7f58e8113a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57276/0 (socket says 172.21.15.37:57276) 2024-03-29T08:42:03.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:03.355+0000 7f58e6a00700 1 -- 172.21.15.37:0/3568610519 learned_addr learned my addr 172.21.15.37:0/3568610519 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:42:03.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:03.356+0000 7f58e6000700 1 --2- 172.21.15.37:0/3568610519 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f58e80fe830 0x7f58e8113fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:42:03.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:03.356+0000 7f58e6a00700 1 -- 172.21.15.37:0/3568610519 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f58e80fe830 msgr2=0x7f58e8113fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:42:03.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:03.356+0000 7f58e6a00700 1 --2- 172.21.15.37:0/3568610519 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f58e80fe830 0x7f58e8113fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:42:03.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:03.356+0000 7f58e6a00700 1 -- 172.21.15.37:0/3568610519 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58d8005040 con 0x7f58e80fde90 2024-03-29T08:42:03.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:03.356+0000 7f58e6a00700 1 --2- 172.21.15.37:0/3568610519 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f58e80fde90 0x7f58e8113a70 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f58d8000c00 tx=0x7f58d8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:42:03.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:03.356+0000 7f58dea00700 1 -- 172.21.15.37:0/3568610519 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f58d800e040 con 0x7f58e80fde90 2024-03-29T08:42:03.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:03.356+0000 7f58dea00700 1 -- 172.21.15.37:0/3568610519 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f58d800bed0 con 0x7f58e80fde90 2024-03-29T08:42:03.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:03.356+0000 7f58dea00700 1 -- 172.21.15.37:0/3568610519 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f58d801c3f0 con 0x7f58e80fde90 2024-03-29T08:42:03.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:03.356+0000 7f58ed600700 1 -- 172.21.15.37:0/3568610519 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58e810dc70 con 0x7f58e80fde90 2024-03-29T08:42:03.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:03.357+0000 7f58ed600700 1 -- 172.21.15.37:0/3568610519 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58e810e1b0 con 0x7f58e80fde90 2024-03-29T08:42:03.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:03.358+0000 7f58ed600700 1 -- 172.21.15.37:0/3568610519 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58e8065e30 con 0x7f58e80fde90 2024-03-29T08:42:03.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:03.359+0000 7f58dea00700 1 -- 172.21.15.37:0/3568610519 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f58d8013420 con 0x7f58e80fde90 2024-03-29T08:42:03.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:03.359+0000 7f58dea00700 1 --2- 172.21.15.37:0/3568610519 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f58cc06cc30 0x7f58cc06f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:42:03.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:03.360+0000 7f58dea00700 1 -- 172.21.15.37:0/3568610519 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f58d808d440 con 0x7f58e80fde90 2024-03-29T08:42:03.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:03.360+0000 7f58e6000700 1 --2- 172.21.15.37:0/3568610519 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f58cc06cc30 0x7f58cc06f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:42:03.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:03.360+0000 7f58e6000700 1 --2- 172.21.15.37:0/3568610519 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f58cc06cc30 0x7f58cc06f0e0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f58e810ebb0 tx=0x7f58d400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:42:03.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:03.364+0000 7f58dea00700 1 -- 172.21.15.37:0/3568610519 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f58d805af10 con 0x7f58e80fde90 2024-03-29T08:42:03.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:03.622+0000 7f58ed600700 1 -- 172.21.15.37:0/3568610519 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f58e8109e10 con 0x7f58e80fde90 2024-03-29T08:42:03.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:03.623+0000 7f58dea00700 1 -- 172.21.15.37:0/3568610519 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 30 v30) v1 ==== 76+0+1841 (secure 0 0 0) 0x7f58d805aaa0 con 0x7f58e80fde90 2024-03-29T08:42:03.622 INFO:teuthology.orchestra.run.smithi037.stdout:e30 2024-03-29T08:42:03.623 INFO:teuthology.orchestra.run.smithi037.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-29T08:42:03.623 INFO:teuthology.orchestra.run.smithi037.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T08:42:03.623 INFO:teuthology.orchestra.run.smithi037.stdout:legacy client fscid: 1 2024-03-29T08:42:03.623 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:42:03.623 INFO:teuthology.orchestra.run.smithi037.stdout:Filesystem 'cephfs' (1) 2024-03-29T08:42:03.623 INFO:teuthology.orchestra.run.smithi037.stdout:fs_name cephfs 2024-03-29T08:42:03.623 INFO:teuthology.orchestra.run.smithi037.stdout:epoch 21 2024-03-29T08:42:03.623 INFO:teuthology.orchestra.run.smithi037.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-03-29T08:42:03.623 INFO:teuthology.orchestra.run.smithi037.stdout:created 2024-03-29T08:03:29.241269+0000 2024-03-29T08:42:03.623 INFO:teuthology.orchestra.run.smithi037.stdout:modified 2024-03-29T08:17:16.237238+0000 2024-03-29T08:42:03.623 INFO:teuthology.orchestra.run.smithi037.stdout:tableserver 0 2024-03-29T08:42:03.623 INFO:teuthology.orchestra.run.smithi037.stdout:root 0 2024-03-29T08:42:03.623 INFO:teuthology.orchestra.run.smithi037.stdout:session_timeout 60 2024-03-29T08:42:03.623 INFO:teuthology.orchestra.run.smithi037.stdout:session_autoclose 300 2024-03-29T08:42:03.624 INFO:teuthology.orchestra.run.smithi037.stdout:max_file_size 1099511627776 2024-03-29T08:42:03.624 INFO:teuthology.orchestra.run.smithi037.stdout:max_xattr_size 65536 2024-03-29T08:42:03.624 INFO:teuthology.orchestra.run.smithi037.stdout:required_client_features {} 2024-03-29T08:42:03.624 INFO:teuthology.orchestra.run.smithi037.stdout:last_failure 0 2024-03-29T08:42:03.624 INFO:teuthology.orchestra.run.smithi037.stdout:last_failure_osd_epoch 82 2024-03-29T08:42:03.624 INFO:teuthology.orchestra.run.smithi037.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T08:42:03.624 INFO:teuthology.orchestra.run.smithi037.stdout:max_mds 2 2024-03-29T08:42:03.624 INFO:teuthology.orchestra.run.smithi037.stdout:in 0,1 2024-03-29T08:42:03.624 INFO:teuthology.orchestra.run.smithi037.stdout:up {} 2024-03-29T08:42:03.624 INFO:teuthology.orchestra.run.smithi037.stdout:failed 0,1 2024-03-29T08:42:03.624 INFO:teuthology.orchestra.run.smithi037.stdout:damaged 2024-03-29T08:42:03.624 INFO:teuthology.orchestra.run.smithi037.stdout:stopped 2024-03-29T08:42:03.624 INFO:teuthology.orchestra.run.smithi037.stdout:data_pools [3] 2024-03-29T08:42:03.624 INFO:teuthology.orchestra.run.smithi037.stdout:metadata_pool 2 2024-03-29T08:42:03.624 INFO:teuthology.orchestra.run.smithi037.stdout:inline_data disabled 2024-03-29T08:42:03.625 INFO:teuthology.orchestra.run.smithi037.stdout:balancer 2024-03-29T08:42:03.625 INFO:teuthology.orchestra.run.smithi037.stdout:bal_rank_mask -1 2024-03-29T08:42:03.625 INFO:teuthology.orchestra.run.smithi037.stdout:standby_count_wanted 1 2024-03-29T08:42:03.625 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:42:03.625 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:42:03.625 INFO:teuthology.orchestra.run.smithi037.stdout:Standby daemons: 2024-03-29T08:42:03.625 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:42:03.625 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi037.mmwgxe{-1:34440} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.37:6828/1248142005,v1:172.21.15.37:6829/1248142005] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:42:03.625 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi037.iifsmh{-1:34444} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.37:6826/2060882959,v1:172.21.15.37:6827/2060882959] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:42:03.625 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi148.sqteih{-1:44395} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.148:6824/4078652242,v1:172.21.15.148:6825/4078652242] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:42:03.625 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi148.qukpjh{-1:45007} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.148:6826/2394751105,v1:172.21.15.148:6827/2394751105] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:42:03.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:03.625+0000 7f58ed600700 1 -- 172.21.15.37:0/3568610519 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f58cc06cc30 msgr2=0x7f58cc06f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:42:03.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:03.625+0000 7f58ed600700 1 --2- 172.21.15.37:0/3568610519 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f58cc06cc30 0x7f58cc06f0e0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f58e810ebb0 tx=0x7f58d400b040 comp rx=0 tx=0).stop 2024-03-29T08:42:03.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:03.625+0000 7f58ed600700 1 -- 172.21.15.37:0/3568610519 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f58e80fde90 msgr2=0x7f58e8113a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:42:03.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:03.625+0000 7f58ed600700 1 --2- 172.21.15.37:0/3568610519 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f58e80fde90 0x7f58e8113a70 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f58d8000c00 tx=0x7f58d8009e30 comp rx=0 tx=0).stop 2024-03-29T08:42:03.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:03.625+0000 7f58ed600700 1 -- 172.21.15.37:0/3568610519 shutdown_connections 2024-03-29T08:42:03.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:03.625+0000 7f58ed600700 1 --2- 172.21.15.37:0/3568610519 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f58cc06cc30 0x7f58cc06f0e0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:42:03.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:03.625+0000 7f58ed600700 1 --2- 172.21.15.37:0/3568610519 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f58e80fde90 0x7f58e8113a70 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:42:03.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:03.625+0000 7f58ed600700 1 --2- 172.21.15.37:0/3568610519 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f58e80fe830 0x7f58e8113fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:42:03.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:03.625+0000 7f58ed600700 1 -- 172.21.15.37:0/3568610519 >> 172.21.15.37:0/3568610519 conn(0x7f58e8072bc0 msgr2=0x7f58e81037c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:42:03.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:03.625+0000 7f58ed600700 1 -- 172.21.15.37:0/3568610519 shutdown_connections 2024-03-29T08:42:03.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:03.625+0000 7f58ed600700 1 -- 172.21.15.37:0/3568610519 wait complete. 2024-03-29T08:42:03.627 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 30 2024-03-29T08:42:03.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:03.764+0000 7fc9df000700 1 -- 172.21.15.37:0/3051387752 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc9d8102e00 msgr2=0x7fc9d81031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:42:03.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:03.764+0000 7fc9df000700 1 --2- 172.21.15.37:0/3051387752 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc9d8102e00 0x7fc9d81031d0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fc9c8004940 tx=0x7fc9c8009e30 comp rx=0 tx=0).stop 2024-03-29T08:42:03.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:03.764+0000 7fc9df000700 1 -- 172.21.15.37:0/3051387752 shutdown_connections 2024-03-29T08:42:03.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:03.764+0000 7fc9df000700 1 --2- 172.21.15.37:0/3051387752 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc9d81037a0 0x7fc9d80fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:42:03.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:03.764+0000 7fc9df000700 1 --2- 172.21.15.37:0/3051387752 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc9d8102e00 0x7fc9d81031d0 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:42:03.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:03.764+0000 7fc9df000700 1 -- 172.21.15.37:0/3051387752 >> 172.21.15.37:0/3051387752 conn(0x7fc9d80f89b0 msgr2=0x7fc9d80fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:42:03.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:03.764+0000 7fc9df000700 1 -- 172.21.15.37:0/3051387752 shutdown_connections 2024-03-29T08:42:03.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:03.764+0000 7fc9df000700 1 -- 172.21.15.37:0/3051387752 wait complete. 2024-03-29T08:42:03.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:03.765+0000 7fc9df000700 1 Processor -- start 2024-03-29T08:42:03.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:03.765+0000 7fc9df000700 1 -- start start 2024-03-29T08:42:03.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:03.766+0000 7fc9df000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc9d8102e00 0x7fc9d81138b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:42:03.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:03.766+0000 7fc9df000700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc9d81037a0 0x7fc9d8113df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:42:03.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:03.766+0000 7fc9df000700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9d8114440 con 0x7fc9d8102e00 2024-03-29T08:42:03.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:03.766+0000 7fc9df000700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9d810da40 con 0x7fc9d81037a0 2024-03-29T08:42:03.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:03.766+0000 7fc9d7e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc9d8102e00 0x7fc9d81138b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:42:03.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:03.766+0000 7fc9d7e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc9d8102e00 0x7fc9d81138b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57304/0 (socket says 172.21.15.37:57304) 2024-03-29T08:42:03.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:03.766+0000 7fc9d7e00700 1 -- 172.21.15.37:0/435296855 learned_addr learned my addr 172.21.15.37:0/435296855 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:42:03.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:03.766+0000 7fc9d7400700 1 --2- 172.21.15.37:0/435296855 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc9d81037a0 0x7fc9d8113df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:42:03.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:03.766+0000 7fc9d7e00700 1 -- 172.21.15.37:0/435296855 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc9d81037a0 msgr2=0x7fc9d8113df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:42:03.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:03.766+0000 7fc9d7e00700 1 --2- 172.21.15.37:0/435296855 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc9d81037a0 0x7fc9d8113df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:42:03.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:03.766+0000 7fc9d7e00700 1 -- 172.21.15.37:0/435296855 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9c8005040 con 0x7fc9d8102e00 2024-03-29T08:42:03.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:03.766+0000 7fc9d7e00700 1 --2- 172.21.15.37:0/435296855 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc9d8102e00 0x7fc9d81138b0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fc9c8009390 tx=0x7fc9c8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:42:03.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:03.767+0000 7fc9cfe00700 1 -- 172.21.15.37:0/435296855 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc9c800e040 con 0x7fc9d8102e00 2024-03-29T08:42:03.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:03.767+0000 7fc9cfe00700 1 -- 172.21.15.37:0/435296855 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fc9c8013420 con 0x7fc9d8102e00 2024-03-29T08:42:03.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:03.767+0000 7fc9cfe00700 1 -- 172.21.15.37:0/435296855 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc9c801c630 con 0x7fc9d8102e00 2024-03-29T08:42:03.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:03.767+0000 7fc9df000700 1 -- 172.21.15.37:0/435296855 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9d810dcc0 con 0x7fc9d8102e00 2024-03-29T08:42:03.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:03.767+0000 7fc9df000700 1 -- 172.21.15.37:0/435296855 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9d810e230 con 0x7fc9d8102e00 2024-03-29T08:42:03.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:03.768+0000 7fc9df000700 1 -- 172.21.15.37:0/435296855 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9d80fe550 con 0x7fc9d8102e00 2024-03-29T08:42:03.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:03.769+0000 7fc9cfe00700 1 -- 172.21.15.37:0/435296855 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fc9c80139e0 con 0x7fc9d8102e00 2024-03-29T08:42:03.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:03.770+0000 7fc9cfe00700 1 --2- 172.21.15.37:0/435296855 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fc9bc06cc20 0x7fc9bc06f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:42:03.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:03.770+0000 7fc9cfe00700 1 -- 172.21.15.37:0/435296855 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fc9c808d320 con 0x7fc9d8102e00 2024-03-29T08:42:03.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:03.770+0000 7fc9d7400700 1 --2- 172.21.15.37:0/435296855 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fc9bc06cc20 0x7fc9bc06f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:42:03.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:03.771+0000 7fc9d7400700 1 --2- 172.21.15.37:0/435296855 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fc9bc06cc20 0x7fc9bc06f0d0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fc9d810ec00 tx=0x7fc9c4009ee0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:42:03.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:03.775+0000 7fc9cfe00700 1 -- 172.21.15.37:0/435296855 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fc9c805ae20 con 0x7fc9d8102e00 2024-03-29T08:42:03.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:42:03 smithi037 ceph-mon[142325]: from='client.35682 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:42:03.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:42:03 smithi037 ceph-mon[142325]: from='client.35686 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:42:03.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:42:03 smithi037 ceph-mon[142325]: from='client.35690 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:42:03.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:42:03 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/2422195924' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:42:03.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:42:03 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/3568610519' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T08:42:04.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:04.017+0000 7fc9df000700 1 -- 172.21.15.37:0/435296855 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc9d810e4a0 con 0x7fc9bc06cc20 2024-03-29T08:42:04.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:04.018+0000 7fc9cfe00700 1 -- 172.21.15.37:0/435296855 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fc9d810e4a0 con 0x7fc9bc06cc20 2024-03-29T08:42:04.018 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-29T08:42:04.018 INFO:teuthology.orchestra.run.smithi037.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59", 2024-03-29T08:42:04.018 INFO:teuthology.orchestra.run.smithi037.stdout: "in_progress": true, 2024-03-29T08:42:04.018 INFO:teuthology.orchestra.run.smithi037.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-29T08:42:04.018 INFO:teuthology.orchestra.run.smithi037.stdout: "services_complete": [ 2024-03-29T08:42:04.018 INFO:teuthology.orchestra.run.smithi037.stdout: "crash", 2024-03-29T08:42:04.018 INFO:teuthology.orchestra.run.smithi037.stdout: "osd", 2024-03-29T08:42:04.018 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr", 2024-03-29T08:42:04.019 INFO:teuthology.orchestra.run.smithi037.stdout: "mds", 2024-03-29T08:42:04.019 INFO:teuthology.orchestra.run.smithi037.stdout: "mon" 2024-03-29T08:42:04.019 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-29T08:42:04.019 INFO:teuthology.orchestra.run.smithi037.stdout: "progress": "16/21 daemons upgraded", 2024-03-29T08:42:04.019 INFO:teuthology.orchestra.run.smithi037.stdout: "message": "Currently upgrading mds daemons", 2024-03-29T08:42:04.019 INFO:teuthology.orchestra.run.smithi037.stdout: "is_paused": false 2024-03-29T08:42:04.019 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-29T08:42:04.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:04.020+0000 7fc9df000700 1 -- 172.21.15.37:0/435296855 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fc9bc06cc20 msgr2=0x7fc9bc06f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:42:04.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:04.020+0000 7fc9df000700 1 --2- 172.21.15.37:0/435296855 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fc9bc06cc20 0x7fc9bc06f0d0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fc9d810ec00 tx=0x7fc9c4009ee0 comp rx=0 tx=0).stop 2024-03-29T08:42:04.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:04.020+0000 7fc9df000700 1 -- 172.21.15.37:0/435296855 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc9d8102e00 msgr2=0x7fc9d81138b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:42:04.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:04.020+0000 7fc9df000700 1 --2- 172.21.15.37:0/435296855 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc9d8102e00 0x7fc9d81138b0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fc9c8009390 tx=0x7fc9c8009e30 comp rx=0 tx=0).stop 2024-03-29T08:42:04.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:04.020+0000 7fc9df000700 1 -- 172.21.15.37:0/435296855 shutdown_connections 2024-03-29T08:42:04.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:04.020+0000 7fc9df000700 1 --2- 172.21.15.37:0/435296855 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fc9bc06cc20 0x7fc9bc06f0d0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:42:04.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:04.020+0000 7fc9df000700 1 --2- 172.21.15.37:0/435296855 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc9d8102e00 0x7fc9d81138b0 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:42:04.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:04.020+0000 7fc9df000700 1 --2- 172.21.15.37:0/435296855 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc9d81037a0 0x7fc9d8113df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:42:04.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:04.021+0000 7fc9df000700 1 -- 172.21.15.37:0/435296855 >> 172.21.15.37:0/435296855 conn(0x7fc9d80f89b0 msgr2=0x7fc9d80f9a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:42:04.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:04.021+0000 7fc9df000700 1 -- 172.21.15.37:0/435296855 shutdown_connections 2024-03-29T08:42:04.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:04.021+0000 7fc9df000700 1 -- 172.21.15.37:0/435296855 wait complete. 2024-03-29T08:42:04.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:42:03 smithi148 ceph-mon[106692]: from='client.35682 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:42:04.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:42:03 smithi148 ceph-mon[106692]: from='client.35686 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:42:04.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:42:03 smithi148 ceph-mon[106692]: from='client.35690 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:42:04.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:42:03 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/2422195924' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:42:04.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:42:03 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/3568610519' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T08:42:04.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:04.157+0000 7f4e13e00700 1 -- 172.21.15.37:0/410065931 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4e14102ca0 msgr2=0x7f4e14103070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:42:04.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:04.157+0000 7f4e13e00700 1 --2- 172.21.15.37:0/410065931 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4e14102ca0 0x7f4e14103070 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f4e04004940 tx=0x7f4e04009e30 comp rx=0 tx=0).stop 2024-03-29T08:42:04.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:04.157+0000 7f4e13e00700 1 -- 172.21.15.37:0/410065931 shutdown_connections 2024-03-29T08:42:04.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:04.157+0000 7f4e13e00700 1 --2- 172.21.15.37:0/410065931 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4e14065630 0x7f4e14065a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:42:04.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:04.157+0000 7f4e13e00700 1 --2- 172.21.15.37:0/410065931 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4e14102ca0 0x7f4e14103070 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:42:04.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:04.157+0000 7f4e13e00700 1 -- 172.21.15.37:0/410065931 >> 172.21.15.37:0/410065931 conn(0x7f4e14071b70 msgr2=0x7f4e14071f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:42:04.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:04.157+0000 7f4e13e00700 1 -- 172.21.15.37:0/410065931 shutdown_connections 2024-03-29T08:42:04.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:04.157+0000 7f4e13e00700 1 -- 172.21.15.37:0/410065931 wait complete. 2024-03-29T08:42:04.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:04.158+0000 7f4e13e00700 1 Processor -- start 2024-03-29T08:42:04.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:04.158+0000 7f4e13e00700 1 -- start start 2024-03-29T08:42:04.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:04.159+0000 7f4e13e00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4e14065630 0x7f4e140fbc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:42:04.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:04.159+0000 7f4e13e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4e14102ca0 0x7f4e140fc160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:42:04.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:04.159+0000 7f4e13e00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e140fc6f0 con 0x7f4e14102ca0 2024-03-29T08:42:04.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:04.159+0000 7f4e13e00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e140fc830 con 0x7f4e14065630 2024-03-29T08:42:04.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:04.159+0000 7f4e12000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4e14102ca0 0x7f4e140fc160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:42:04.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:04.159+0000 7f4e12000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4e14102ca0 0x7f4e140fc160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57332/0 (socket says 172.21.15.37:57332) 2024-03-29T08:42:04.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:04.159+0000 7f4e12000700 1 -- 172.21.15.37:0/2344176137 learned_addr learned my addr 172.21.15.37:0/2344176137 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:42:04.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:04.159+0000 7f4e12a00700 1 --2- 172.21.15.37:0/2344176137 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4e14065630 0x7f4e140fbc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:42:04.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:04.159+0000 7f4e12000700 1 -- 172.21.15.37:0/2344176137 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4e14065630 msgr2=0x7f4e140fbc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:42:04.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:04.159+0000 7f4e12000700 1 --2- 172.21.15.37:0/2344176137 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4e14065630 0x7f4e140fbc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:42:04.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:04.159+0000 7f4e12000700 1 -- 172.21.15.37:0/2344176137 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e04005040 con 0x7f4e14102ca0 2024-03-29T08:42:04.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:04.160+0000 7f4e12000700 1 --2- 172.21.15.37:0/2344176137 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4e14102ca0 0x7f4e140fc160 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f4e00002a50 tx=0x7f4e000046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:42:04.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:04.160+0000 7f4e0aa00700 1 -- 172.21.15.37:0/2344176137 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4e00007cc0 con 0x7f4e14102ca0 2024-03-29T08:42:04.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:04.160+0000 7f4e13e00700 1 -- 172.21.15.37:0/2344176137 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e14119ba0 con 0x7f4e14102ca0 2024-03-29T08:42:04.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:04.160+0000 7f4e0aa00700 1 -- 172.21.15.37:0/2344176137 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f4e0000f040 con 0x7f4e14102ca0 2024-03-29T08:42:04.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:04.160+0000 7f4e0aa00700 1 -- 172.21.15.37:0/2344176137 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4e0000d9c0 con 0x7f4e14102ca0 2024-03-29T08:42:04.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:04.160+0000 7f4e13e00700 1 -- 172.21.15.37:0/2344176137 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e1411a020 con 0x7f4e14102ca0 2024-03-29T08:42:04.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:04.161+0000 7f4e13e00700 1 -- 172.21.15.37:0/2344176137 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e140fe380 con 0x7f4e14102ca0 2024-03-29T08:42:04.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:04.162+0000 7f4e0aa00700 1 -- 172.21.15.37:0/2344176137 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f4e0000db20 con 0x7f4e14102ca0 2024-03-29T08:42:04.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:04.163+0000 7f4e0aa00700 1 --2- 172.21.15.37:0/2344176137 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f4df806cc20 0x7f4df806f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:42:04.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:04.163+0000 7f4e0aa00700 1 -- 172.21.15.37:0/2344176137 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f4e0008bde0 con 0x7f4e14102ca0 2024-03-29T08:42:04.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:04.164+0000 7f4e12a00700 1 --2- 172.21.15.37:0/2344176137 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f4df806cc20 0x7f4df806f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:42:04.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:04.164+0000 7f4e12a00700 1 --2- 172.21.15.37:0/2344176137 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f4df806cc20 0x7f4df806f0d0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f4e04004f90 tx=0x7f4e04009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:42:04.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:04.168+0000 7f4e0aa00700 1 -- 172.21.15.37:0/2344176137 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f4e00055d20 con 0x7f4e14102ca0 2024-03-29T08:42:04.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:04.472+0000 7f4e13e00700 1 -- 172.21.15.37:0/2344176137 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f4e1405f5f0 con 0x7f4e14102ca0 2024-03-29T08:42:04.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:04.472+0000 7f4e0aa00700 1 -- 172.21.15.37:0/2344176137 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+3583 (secure 0 0 0) 0x7f4e00059340 con 0x7f4e14102ca0 2024-03-29T08:42:04.472 INFO:teuthology.orchestra.run.smithi037.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem is offline 2024-03-29T08:42:04.472 INFO:teuthology.orchestra.run.smithi037.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-03-29T08:42:04.472 INFO:teuthology.orchestra.run.smithi037.stdout: host smithi148 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/mon.smithi148/config 2024-03-29T08:42:04.472 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/podman: timeout after 895 seconds 2024-03-29T08:42:04.473 INFO:teuthology.orchestra.run.smithi037.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59 -e NODE_NAME=smithi148 -e CEPH_USE_RANDOM_NONCE=1 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4:/var/run/ceph:z -v /var/log/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4:/var/log/ceph:z -v /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpkx2rz2d4:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59 inventory --format=json-pretty --filter-for-batch 2024-03-29T08:42:04.473 INFO:teuthology.orchestra.run.smithi037.stdout:Traceback (most recent call last): 2024-03-29T08:42:04.473 INFO:teuthology.orchestra.run.smithi037.stdout: File "/usr/lib64/python3.6/runpy.py", line 193, in _run_module_as_main 2024-03-29T08:42:04.473 INFO:teuthology.orchestra.run.smithi037.stdout: "__main__", mod_spec) 2024-03-29T08:42:04.473 INFO:teuthology.orchestra.run.smithi037.stdout: File "/usr/lib64/python3.6/runpy.py", line 85, in _run_code 2024-03-29T08:42:04.473 INFO:teuthology.orchestra.run.smithi037.stdout: exec(code, run_globals) 2024-03-29T08:42:04.473 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 10889, in 2024-03-29T08:42:04.473 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 10877, in main 2024-03-29T08:42:04.473 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 2576, in _infer_config 2024-03-29T08:42:04.473 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 2492, in _infer_fsid 2024-03-29T08:42:04.473 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 2604, in _infer_image 2024-03-29T08:42:04.473 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 2479, in _validate_fsid 2024-03-29T08:42:04.473 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 7145, in command_ceph_volume 2024-03-29T08:42:04.473 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 2267, in call_throws 2024-03-29T08:42:04.473 INFO:teuthology.orchestra.run.smithi037.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59 -e NODE_NAME=smithi148 -e CEPH_USE_RANDOM_NONCE=1 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4:/var/run/ceph:z -v /var/log/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4:/var/log/ceph:z -v /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpkx2rz2d4:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59 inventory --format=json-pretty --filter-for-batch 2024-03-29T08:42:04.474 INFO:teuthology.orchestra.run.smithi037.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-03-29T08:42:04.474 INFO:teuthology.orchestra.run.smithi037.stdout: fs cephfs is degraded 2024-03-29T08:42:04.474 INFO:teuthology.orchestra.run.smithi037.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-03-29T08:42:04.474 INFO:teuthology.orchestra.run.smithi037.stdout: fs cephfs is offline because no MDS is active for it. 2024-03-29T08:42:04.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:04.475+0000 7f4e13e00700 1 -- 172.21.15.37:0/2344176137 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f4df806cc20 msgr2=0x7f4df806f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:42:04.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:04.475+0000 7f4e13e00700 1 --2- 172.21.15.37:0/2344176137 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f4df806cc20 0x7f4df806f0d0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f4e04004f90 tx=0x7f4e04009e30 comp rx=0 tx=0).stop 2024-03-29T08:42:04.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:04.475+0000 7f4e13e00700 1 -- 172.21.15.37:0/2344176137 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4e14102ca0 msgr2=0x7f4e140fc160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:42:04.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:04.475+0000 7f4e13e00700 1 --2- 172.21.15.37:0/2344176137 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4e14102ca0 0x7f4e140fc160 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f4e00002a50 tx=0x7f4e000046b0 comp rx=0 tx=0).stop 2024-03-29T08:42:04.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:04.475+0000 7f4e13e00700 1 -- 172.21.15.37:0/2344176137 shutdown_connections 2024-03-29T08:42:04.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:04.475+0000 7f4e13e00700 1 --2- 172.21.15.37:0/2344176137 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f4df806cc20 0x7f4df806f0d0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:42:04.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:04.475+0000 7f4e13e00700 1 --2- 172.21.15.37:0/2344176137 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4e14065630 0x7f4e140fbc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:42:04.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:04.475+0000 7f4e13e00700 1 --2- 172.21.15.37:0/2344176137 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4e14102ca0 0x7f4e140fc160 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:42:04.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:04.475+0000 7f4e13e00700 1 -- 172.21.15.37:0/2344176137 >> 172.21.15.37:0/2344176137 conn(0x7f4e14071b70 msgr2=0x7f4e141059d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:42:04.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:04.475+0000 7f4e13e00700 1 -- 172.21.15.37:0/2344176137 shutdown_connections 2024-03-29T08:42:04.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:04.475+0000 7f4e13e00700 1 -- 172.21.15.37:0/2344176137 wait complete. 2024-03-29T08:42:04.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:42:04 smithi037 ceph-mon[142325]: pgmap v936: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:42:04.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:42:04 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/2344176137' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T08:42:05.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:42:04 smithi148 ceph-mon[106692]: pgmap v936: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:42:05.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:42:04 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/2344176137' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T08:42:05.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:42:05 smithi037 ceph-mon[142325]: from='client.35702 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:42:06.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:42:05 smithi148 ceph-mon[106692]: from='client.35702 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:42:06.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:42:06 smithi037 ceph-mon[142325]: pgmap v937: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:42:07.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:42:06 smithi148 ceph-mon[106692]: pgmap v937: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:42:08.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:42:08 smithi037 ceph-mon[142325]: pgmap v938: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:42:09.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:42:08 smithi148 ceph-mon[106692]: pgmap v938: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:42:10.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:42:10 smithi148 ceph-mon[106692]: pgmap v939: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:42:10.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:42:10 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:42:10.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:42:10 smithi037 ceph-mon[142325]: pgmap v939: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:42:10.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:42:10 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:42:12.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:42:12 smithi148 ceph-mon[106692]: pgmap v940: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:42:12.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:42:12 smithi037 ceph-mon[142325]: pgmap v940: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:42:14.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:42:14 smithi148 ceph-mon[106692]: pgmap v941: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:42:14.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:42:14 smithi037 ceph-mon[142325]: pgmap v941: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:42:16.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:42:16 smithi148 ceph-mon[106692]: pgmap v942: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:42:16.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:42:16 smithi037 ceph-mon[142325]: pgmap v942: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:42:18.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:42:18 smithi148 ceph-mon[106692]: pgmap v943: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:42:18.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:42:18 smithi037 ceph-mon[142325]: pgmap v943: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:42:20.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:42:20 smithi148 ceph-mon[106692]: pgmap v944: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:42:20.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:42:20 smithi037 ceph-mon[142325]: pgmap v944: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:42:22.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:42:22 smithi148 ceph-mon[106692]: pgmap v945: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:42:22.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:42:22 smithi037 ceph-mon[142325]: pgmap v945: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:42:24.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:42:24 smithi148 ceph-mon[106692]: pgmap v946: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:42:24.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:42:24 smithi037 ceph-mon[142325]: pgmap v946: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:42:25.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:42:25 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:42:26.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:42:25 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:42:26.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:42:26 smithi037 ceph-mon[142325]: pgmap v947: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:42:27.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:42:26 smithi148 ceph-mon[106692]: pgmap v947: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:42:28.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:42:28 smithi037 ceph-mon[142325]: pgmap v948: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:42:29.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:42:28 smithi148 ceph-mon[106692]: pgmap v948: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:42:30.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:42:30 smithi037 ceph-mon[142325]: pgmap v949: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:42:31.058 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:42:30 smithi148 ceph-mon[106692]: pgmap v949: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:42:32.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:42:32 smithi037 ceph-mon[142325]: pgmap v950: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:42:33.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:42:32 smithi148 ceph-mon[106692]: pgmap v950: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:42:34.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:34.631+0000 7f14cda00700 1 -- 172.21.15.37:0/2545820143 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f14c8070360 msgr2=0x7f14c8070730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:42:34.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:34.631+0000 7f14cda00700 1 --2- 172.21.15.37:0/2545820143 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f14c8070360 0x7f14c8070730 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f14b8004970 tx=0x7f14b8009e30 comp rx=0 tx=0).stop 2024-03-29T08:42:34.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:34.631+0000 7f14cda00700 1 -- 172.21.15.37:0/2545820143 shutdown_connections 2024-03-29T08:42:34.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:34.631+0000 7f14cda00700 1 --2- 172.21.15.37:0/2545820143 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f14c8070c70 0x7f14c8109110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:42:34.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:34.631+0000 7f14cda00700 1 --2- 172.21.15.37:0/2545820143 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f14c8070360 0x7f14c8070730 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:42:34.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:34.631+0000 7f14cda00700 1 -- 172.21.15.37:0/2545820143 >> 172.21.15.37:0/2545820143 conn(0x7f14c80f89b0 msgr2=0x7f14c80fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:42:34.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:34.631+0000 7f14cda00700 1 -- 172.21.15.37:0/2545820143 shutdown_connections 2024-03-29T08:42:34.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:34.631+0000 7f14cda00700 1 -- 172.21.15.37:0/2545820143 wait complete. 2024-03-29T08:42:34.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:34.632+0000 7f14cda00700 1 Processor -- start 2024-03-29T08:42:34.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:34.632+0000 7f14cda00700 1 -- start start 2024-03-29T08:42:34.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:34.633+0000 7f14cda00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f14c8070360 0x7f14c8115c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:42:34.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:34.633+0000 7f14cda00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f14c8070c70 0x7f14c81161c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:42:34.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:34.633+0000 7f14cda00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14c8116810 con 0x7f14c8070c70 2024-03-29T08:42:34.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:34.633+0000 7f14cda00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14c810fbe0 con 0x7f14c8070360 2024-03-29T08:42:34.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:34.633+0000 7f14c6000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f14c8070c70 0x7f14c81161c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:42:34.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:34.633+0000 7f14c6000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f14c8070c70 0x7f14c81161c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42042/0 (socket says 172.21.15.37:42042) 2024-03-29T08:42:34.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:34.633+0000 7f14c6000700 1 -- 172.21.15.37:0/1865065134 learned_addr learned my addr 172.21.15.37:0/1865065134 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:42:34.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:34.633+0000 7f14c6a00700 1 --2- 172.21.15.37:0/1865065134 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f14c8070360 0x7f14c8115c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:42:34.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:34.633+0000 7f14c6000700 1 -- 172.21.15.37:0/1865065134 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f14c8070360 msgr2=0x7f14c8115c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:42:34.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:34.633+0000 7f14c6000700 1 --2- 172.21.15.37:0/1865065134 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f14c8070360 0x7f14c8115c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:42:34.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:34.633+0000 7f14c6000700 1 -- 172.21.15.37:0/1865065134 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14b8005040 con 0x7f14c8070c70 2024-03-29T08:42:34.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:34.634+0000 7f14c6000700 1 --2- 172.21.15.37:0/1865065134 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f14c8070c70 0x7f14c81161c0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f14b4002a80 tx=0x7f14b40046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:42:34.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:34.634+0000 7f14bea00700 1 -- 172.21.15.37:0/1865065134 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f14b4007cc0 con 0x7f14c8070c70 2024-03-29T08:42:34.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:34.634+0000 7f14bea00700 1 -- 172.21.15.37:0/1865065134 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f14b400f040 con 0x7f14c8070c70 2024-03-29T08:42:34.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:34.634+0000 7f14cda00700 1 -- 172.21.15.37:0/1865065134 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14c810fec0 con 0x7f14c8070c70 2024-03-29T08:42:34.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:34.634+0000 7f14bea00700 1 -- 172.21.15.37:0/1865065134 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f14b400da60 con 0x7f14c8070c70 2024-03-29T08:42:34.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:34.634+0000 7f14cda00700 1 -- 172.21.15.37:0/1865065134 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14c8110400 con 0x7f14c8070c70 2024-03-29T08:42:34.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:34.635+0000 7f14cda00700 1 -- 172.21.15.37:0/1865065134 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14c8103b90 con 0x7f14c8070c70 2024-03-29T08:42:34.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:34.636+0000 7f14bea00700 1 -- 172.21.15.37:0/1865065134 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f14b400dbc0 con 0x7f14c8070c70 2024-03-29T08:42:34.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:34.637+0000 7f14bea00700 1 --2- 172.21.15.37:0/1865065134 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f14ac06cc20 0x7f14ac06f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:42:34.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:34.637+0000 7f14bea00700 1 -- 172.21.15.37:0/1865065134 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f14b408bc70 con 0x7f14c8070c70 2024-03-29T08:42:34.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:34.637+0000 7f14c6a00700 1 --2- 172.21.15.37:0/1865065134 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f14ac06cc20 0x7f14ac06f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:42:34.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:34.638+0000 7f14c6a00700 1 --2- 172.21.15.37:0/1865065134 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f14ac06cc20 0x7f14ac06f0d0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f14b8004f50 tx=0x7f14b8009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:42:34.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:34.642+0000 7f14bea00700 1 -- 172.21.15.37:0/1865065134 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f14b4092080 con 0x7f14c8070c70 2024-03-29T08:42:34.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:34.872+0000 7f14cda00700 1 -- 172.21.15.37:0/1865065134 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f14c8111340 con 0x7f14ac06cc20 2024-03-29T08:42:34.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:34.873+0000 7f14bea00700 1 -- 172.21.15.37:0/1865065134 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f14c8111340 con 0x7f14ac06cc20 2024-03-29T08:42:34.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:34.875+0000 7f14cda00700 1 -- 172.21.15.37:0/1865065134 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f14ac06cc20 msgr2=0x7f14ac06f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:42:34.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:34.875+0000 7f14cda00700 1 --2- 172.21.15.37:0/1865065134 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f14ac06cc20 0x7f14ac06f0d0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f14b8004f50 tx=0x7f14b8009e30 comp rx=0 tx=0).stop 2024-03-29T08:42:34.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:34.875+0000 7f14cda00700 1 -- 172.21.15.37:0/1865065134 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f14c8070c70 msgr2=0x7f14c81161c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:42:34.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:34.875+0000 7f14cda00700 1 --2- 172.21.15.37:0/1865065134 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f14c8070c70 0x7f14c81161c0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f14b4002a80 tx=0x7f14b40046b0 comp rx=0 tx=0).stop 2024-03-29T08:42:34.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:34.875+0000 7f14cda00700 1 -- 172.21.15.37:0/1865065134 shutdown_connections 2024-03-29T08:42:34.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:34.875+0000 7f14cda00700 1 --2- 172.21.15.37:0/1865065134 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f14ac06cc20 0x7f14ac06f0d0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:42:34.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:34.875+0000 7f14cda00700 1 --2- 172.21.15.37:0/1865065134 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f14c8070360 0x7f14c8115c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:42:34.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:34.875+0000 7f14cda00700 1 --2- 172.21.15.37:0/1865065134 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f14c8070c70 0x7f14c81161c0 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:42:34.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:34.875+0000 7f14cda00700 1 -- 172.21.15.37:0/1865065134 >> 172.21.15.37:0/1865065134 conn(0x7f14c80f89b0 msgr2=0x7f14c81015f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:42:34.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:34.876+0000 7f14cda00700 1 -- 172.21.15.37:0/1865065134 shutdown_connections 2024-03-29T08:42:34.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:34.876+0000 7f14cda00700 1 -- 172.21.15.37:0/1865065134 wait complete. 2024-03-29T08:42:34.889 INFO:teuthology.orchestra.run.smithi037.stdout:true 2024-03-29T08:42:34.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:42:34 smithi037 ceph-mon[142325]: pgmap v951: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:42:35.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:35.013+0000 7fb1a2000700 1 -- 172.21.15.37:0/2801297979 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb19c1037a0 msgr2=0x7fb19c0fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:42:35.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:35.013+0000 7fb1a2000700 1 --2- 172.21.15.37:0/2801297979 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb19c1037a0 0x7fb19c0fce10 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7fb188004940 tx=0x7fb188009e30 comp rx=0 tx=0).stop 2024-03-29T08:42:35.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:35.013+0000 7fb1a2000700 1 -- 172.21.15.37:0/2801297979 shutdown_connections 2024-03-29T08:42:35.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:35.013+0000 7fb1a2000700 1 --2- 172.21.15.37:0/2801297979 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb19c1037a0 0x7fb19c0fce10 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:42:35.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:35.013+0000 7fb1a2000700 1 --2- 172.21.15.37:0/2801297979 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb19c102e00 0x7fb19c1031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:42:35.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:35.013+0000 7fb1a2000700 1 -- 172.21.15.37:0/2801297979 >> 172.21.15.37:0/2801297979 conn(0x7fb19c0f89b0 msgr2=0x7fb19c0fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:42:35.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:35.014+0000 7fb1a2000700 1 -- 172.21.15.37:0/2801297979 shutdown_connections 2024-03-29T08:42:35.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:35.014+0000 7fb1a2000700 1 -- 172.21.15.37:0/2801297979 wait complete. 2024-03-29T08:42:35.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:35.014+0000 7fb1a2000700 1 Processor -- start 2024-03-29T08:42:35.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:35.015+0000 7fb1a2000700 1 -- start start 2024-03-29T08:42:35.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:35.015+0000 7fb1a2000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb19c102e00 0x7fb19c115d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:42:35.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:35.015+0000 7fb1a2000700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb19c1037a0 0x7fb19c116240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:42:35.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:35.015+0000 7fb1a2000700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb19c116890 con 0x7fb19c102e00 2024-03-29T08:42:35.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:35.015+0000 7fb1a2000700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb19c10fc60 con 0x7fb19c1037a0 2024-03-29T08:42:35.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:35.015+0000 7fb19b400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb19c102e00 0x7fb19c115d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:42:35.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:35.015+0000 7fb19b400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb19c102e00 0x7fb19c115d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42050/0 (socket says 172.21.15.37:42050) 2024-03-29T08:42:35.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:35.015+0000 7fb19b400700 1 -- 172.21.15.37:0/3548739480 learned_addr learned my addr 172.21.15.37:0/3548739480 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:42:35.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:35.015+0000 7fb19aa00700 1 --2- 172.21.15.37:0/3548739480 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb19c1037a0 0x7fb19c116240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:42:35.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:35.016+0000 7fb19b400700 1 -- 172.21.15.37:0/3548739480 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb19c1037a0 msgr2=0x7fb19c116240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:42:35.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:35.016+0000 7fb19b400700 1 --2- 172.21.15.37:0/3548739480 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb19c1037a0 0x7fb19c116240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:42:35.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:35.016+0000 7fb19b400700 1 -- 172.21.15.37:0/3548739480 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb188005040 con 0x7fb19c102e00 2024-03-29T08:42:35.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:35.016+0000 7fb19b400700 1 --2- 172.21.15.37:0/3548739480 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb19c102e00 0x7fb19c115d00 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fb18c002a50 tx=0x7fb18c0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:42:35.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:35.016+0000 7fb193400700 1 -- 172.21.15.37:0/3548739480 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb18c007cc0 con 0x7fb19c102e00 2024-03-29T08:42:35.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:35.016+0000 7fb1a2000700 1 -- 172.21.15.37:0/3548739480 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb19c10ff40 con 0x7fb19c102e00 2024-03-29T08:42:35.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:35.016+0000 7fb193400700 1 -- 172.21.15.37:0/3548739480 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fb18c00f040 con 0x7fb19c102e00 2024-03-29T08:42:35.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:35.016+0000 7fb193400700 1 -- 172.21.15.37:0/3548739480 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb18c00d9c0 con 0x7fb19c102e00 2024-03-29T08:42:35.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:35.016+0000 7fb1a2000700 1 -- 172.21.15.37:0/3548739480 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb19c110500 con 0x7fb19c102e00 2024-03-29T08:42:35.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:35.017+0000 7fb1a2000700 1 -- 172.21.15.37:0/3548739480 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb19c0fe550 con 0x7fb19c102e00 2024-03-29T08:42:35.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:35.018+0000 7fb193400700 1 -- 172.21.15.37:0/3548739480 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fb18c00db20 con 0x7fb19c102e00 2024-03-29T08:42:35.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:35.019+0000 7fb193400700 1 --2- 172.21.15.37:0/3548739480 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fb18006cc30 0x7fb18006f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:42:35.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:35.019+0000 7fb193400700 1 -- 172.21.15.37:0/3548739480 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fb18c08bd20 con 0x7fb19c102e00 2024-03-29T08:42:35.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:35.019+0000 7fb19aa00700 1 --2- 172.21.15.37:0/3548739480 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fb18006cc30 0x7fb18006f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:42:35.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:35.020+0000 7fb19aa00700 1 --2- 172.21.15.37:0/3548739480 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fb18006cc30 0x7fb18006f0e0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fb19c1115a0 tx=0x7fb188009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:42:35.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:35.024+0000 7fb193400700 1 -- 172.21.15.37:0/3548739480 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fb18c059770 con 0x7fb19c102e00 2024-03-29T08:42:35.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:42:34 smithi148 ceph-mon[106692]: pgmap v951: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:42:35.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:35.257+0000 7fb1a2000700 1 -- 172.21.15.37:0/3548739480 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb19c04bf40 con 0x7fb18006cc30 2024-03-29T08:42:35.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:35.259+0000 7fb193400700 1 -- 172.21.15.37:0/3548739480 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fb19c04bf40 con 0x7fb18006cc30 2024-03-29T08:42:35.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:35.261+0000 7fb1a2000700 1 -- 172.21.15.37:0/3548739480 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fb18006cc30 msgr2=0x7fb18006f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:42:35.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:35.261+0000 7fb1a2000700 1 --2- 172.21.15.37:0/3548739480 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fb18006cc30 0x7fb18006f0e0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fb19c1115a0 tx=0x7fb188009e30 comp rx=0 tx=0).stop 2024-03-29T08:42:35.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:35.261+0000 7fb1a2000700 1 -- 172.21.15.37:0/3548739480 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb19c102e00 msgr2=0x7fb19c115d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:42:35.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:35.261+0000 7fb1a2000700 1 --2- 172.21.15.37:0/3548739480 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb19c102e00 0x7fb19c115d00 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fb18c002a50 tx=0x7fb18c0046b0 comp rx=0 tx=0).stop 2024-03-29T08:42:35.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:35.261+0000 7fb1a2000700 1 -- 172.21.15.37:0/3548739480 shutdown_connections 2024-03-29T08:42:35.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:35.261+0000 7fb1a2000700 1 --2- 172.21.15.37:0/3548739480 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fb18006cc30 0x7fb18006f0e0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:42:35.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:35.261+0000 7fb1a2000700 1 --2- 172.21.15.37:0/3548739480 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb19c102e00 0x7fb19c115d00 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:42:35.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:35.261+0000 7fb1a2000700 1 --2- 172.21.15.37:0/3548739480 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb19c1037a0 0x7fb19c116240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:42:35.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:35.261+0000 7fb1a2000700 1 -- 172.21.15.37:0/3548739480 >> 172.21.15.37:0/3548739480 conn(0x7fb19c0f89b0 msgr2=0x7fb19c0f9ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:42:35.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:35.261+0000 7fb1a2000700 1 -- 172.21.15.37:0/3548739480 shutdown_connections 2024-03-29T08:42:35.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:35.261+0000 7fb1a2000700 1 -- 172.21.15.37:0/3548739480 wait complete. 2024-03-29T08:42:35.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:35.398+0000 7f091ca00700 1 -- 172.21.15.37:0/876249273 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0918102c40 msgr2=0x7f0918103010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:42:35.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:35.398+0000 7f091ca00700 1 --2- 172.21.15.37:0/876249273 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0918102c40 0x7f0918103010 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f0908004940 tx=0x7f0908009e30 comp rx=0 tx=0).stop 2024-03-29T08:42:35.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:35.398+0000 7f091ca00700 1 -- 172.21.15.37:0/876249273 shutdown_connections 2024-03-29T08:42:35.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:35.398+0000 7f091ca00700 1 --2- 172.21.15.37:0/876249273 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f09181035e0 0x7f09180fcc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:42:35.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:35.398+0000 7f091ca00700 1 --2- 172.21.15.37:0/876249273 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0918102c40 0x7f0918103010 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:42:35.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:35.398+0000 7f091ca00700 1 -- 172.21.15.37:0/876249273 >> 172.21.15.37:0/876249273 conn(0x7f09180f8830 msgr2=0x7f09180fac40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:42:35.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:35.398+0000 7f091ca00700 1 -- 172.21.15.37:0/876249273 shutdown_connections 2024-03-29T08:42:35.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:35.398+0000 7f091ca00700 1 -- 172.21.15.37:0/876249273 wait complete. 2024-03-29T08:42:35.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:35.399+0000 7f091ca00700 1 Processor -- start 2024-03-29T08:42:35.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:35.399+0000 7f091ca00700 1 -- start start 2024-03-29T08:42:35.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:35.400+0000 7f091ca00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0918102c40 0x7f0918115a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:42:35.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:35.400+0000 7f091ca00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f09181035e0 0x7f0918115fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:42:35.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:35.400+0000 7f091ca00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0918116610 con 0x7f0918102c40 2024-03-29T08:42:35.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:35.400+0000 7f091ca00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f091810fa00 con 0x7f09181035e0 2024-03-29T08:42:35.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:35.400+0000 7f0917400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0918102c40 0x7f0918115a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:42:35.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:35.400+0000 7f0917400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0918102c40 0x7f0918115a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42068/0 (socket says 172.21.15.37:42068) 2024-03-29T08:42:35.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:35.400+0000 7f0917400700 1 -- 172.21.15.37:0/3109300192 learned_addr learned my addr 172.21.15.37:0/3109300192 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:42:35.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:35.400+0000 7f0916a00700 1 --2- 172.21.15.37:0/3109300192 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f09181035e0 0x7f0918115fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:42:35.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:35.400+0000 7f0917400700 1 -- 172.21.15.37:0/3109300192 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f09181035e0 msgr2=0x7f0918115fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:42:35.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:35.400+0000 7f0917400700 1 --2- 172.21.15.37:0/3109300192 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f09181035e0 0x7f0918115fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:42:35.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:35.400+0000 7f0917400700 1 -- 172.21.15.37:0/3109300192 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0908005040 con 0x7f0918102c40 2024-03-29T08:42:35.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:35.401+0000 7f0917400700 1 --2- 172.21.15.37:0/3109300192 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0918102c40 0x7f0918115a80 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f0908009390 tx=0x7f0908009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:42:35.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:35.401+0000 7f090f400700 1 -- 172.21.15.37:0/3109300192 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f090800e040 con 0x7f0918102c40 2024-03-29T08:42:35.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:35.401+0000 7f090f400700 1 -- 172.21.15.37:0/3109300192 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f0908013420 con 0x7f0918102c40 2024-03-29T08:42:35.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:35.401+0000 7f090f400700 1 -- 172.21.15.37:0/3109300192 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f090801c630 con 0x7f0918102c40 2024-03-29T08:42:35.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:35.401+0000 7f091ca00700 1 -- 172.21.15.37:0/3109300192 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f091810fc80 con 0x7f0918102c40 2024-03-29T08:42:35.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:35.401+0000 7f091ca00700 1 -- 172.21.15.37:0/3109300192 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0918110240 con 0x7f0918102c40 2024-03-29T08:42:35.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:35.402+0000 7f091ca00700 1 -- 172.21.15.37:0/3109300192 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09180fe390 con 0x7f0918102c40 2024-03-29T08:42:35.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:35.403+0000 7f090f400700 1 -- 172.21.15.37:0/3109300192 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f09080139e0 con 0x7f0918102c40 2024-03-29T08:42:35.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:35.404+0000 7f090f400700 1 --2- 172.21.15.37:0/3109300192 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f08fc06cc30 0x7f08fc06f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:42:35.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:35.404+0000 7f090f400700 1 -- 172.21.15.37:0/3109300192 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f090808d2f0 con 0x7f0918102c40 2024-03-29T08:42:35.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:35.404+0000 7f0916a00700 1 --2- 172.21.15.37:0/3109300192 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f08fc06cc30 0x7f08fc06f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:42:35.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:35.405+0000 7f0916a00700 1 --2- 172.21.15.37:0/3109300192 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f08fc06cc30 0x7f08fc06f0e0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f0918110bc0 tx=0x7f090400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:42:35.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:35.408+0000 7f090f400700 1 -- 172.21.15.37:0/3109300192 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f090805adc0 con 0x7f0918102c40 2024-03-29T08:42:35.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:35.630+0000 7f091ca00700 1 -- 172.21.15.37:0/3109300192 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f0918109c40 con 0x7f08fc06cc30 2024-03-29T08:42:35.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:35.640+0000 7f090f400700 1 -- 172.21.15.37:0/3109300192 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f0918109c40 con 0x7f08fc06cc30 2024-03-29T08:42:35.640 INFO:teuthology.orchestra.run.smithi037.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-29T08:42:35.640 INFO:teuthology.orchestra.run.smithi037.stdout:alertmanager.smithi037 smithi037 *:9093,9094 running (32m) 9m ago 45m 14.3M - 0.25.0 c8568f914cd2 2f9f9746ed0f 2024-03-29T08:42:35.640 INFO:teuthology.orchestra.run.smithi037.stdout:crash.smithi037 smithi037 running (29m) 9m ago 45m 6647k - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 178d416a3186 2024-03-29T08:42:35.640 INFO:teuthology.orchestra.run.smithi037.stdout:crash.smithi148 smithi148 running (29m) 9m ago 44m 6656k - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 23666989d58f 2024-03-29T08:42:35.640 INFO:teuthology.orchestra.run.smithi037.stdout:grafana.smithi037 smithi037 *:3000 running (32m) 9m ago 45m 74.5M - 9.4.7 954c08fa6188 6b976fe49952 2024-03-29T08:42:35.640 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi037.iifsmh smithi037 running (24m) 9m ago 38m 16.4M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 2d97f562cc5b 2024-03-29T08:42:35.640 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi037.mmwgxe smithi037 running (25m) 9m ago 39m 16.7M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff ec37037f46e8 2024-03-29T08:42:35.640 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi148.qukpjh smithi148 running (9m) 9m ago 38m 10.7M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 3257219bc046 2024-03-29T08:42:35.640 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi148.sqteih smithi148 running (24m) 9m ago 38m 16.3M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 44c15c77037a 2024-03-29T08:42:35.640 INFO:teuthology.orchestra.run.smithi037.stdout:mgr.smithi037.wjikii smithi037 *:8443,9283,8765 running (34m) 9m ago 47m 510M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff c5a1417b9417 2024-03-29T08:42:35.640 INFO:teuthology.orchestra.run.smithi037.stdout:mgr.smithi148.onhgvm smithi148 *:8443,9283,8765 running (33m) 9m ago 44m 456M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff fcfdbce062bd 2024-03-29T08:42:35.641 INFO:teuthology.orchestra.run.smithi037.stdout:mon.smithi037 smithi037 running (30m) 9m ago 47m 70.2M 2048M 18.2.2-1015-gd0e392d9 1cc2f45c17ff eea2060783af 2024-03-29T08:42:35.641 INFO:teuthology.orchestra.run.smithi037.stdout:mon.smithi148 smithi148 running (30m) 9m ago 44m 50.4M 2048M 18.2.2-1015-gd0e392d9 1cc2f45c17ff ad4b2c5748be 2024-03-29T08:42:35.641 INFO:teuthology.orchestra.run.smithi037.stdout:node-exporter.smithi037 smithi037 *:9100 running (33m) 9m ago 45m 8514k - 1.5.0 0da6a335fe13 26b8ae9cb878 2024-03-29T08:42:35.641 INFO:teuthology.orchestra.run.smithi037.stdout:node-exporter.smithi148 smithi148 *:9100 running (33m) 9m ago 44m 8464k - 1.5.0 0da6a335fe13 98414e4e64ed 2024-03-29T08:42:35.641 INFO:teuthology.orchestra.run.smithi037.stdout:osd.0 smithi037 running (28m) 9m ago 43m 108M 1619M 18.2.2-1015-gd0e392d9 1cc2f45c17ff c0c584f6245c 2024-03-29T08:42:35.641 INFO:teuthology.orchestra.run.smithi037.stdout:osd.1 smithi037 running (28m) 9m ago 42m 104M 1619M 18.2.2-1015-gd0e392d9 1cc2f45c17ff a4a2ff4cc83d 2024-03-29T08:42:35.641 INFO:teuthology.orchestra.run.smithi037.stdout:osd.2 smithi037 running (27m) 9m ago 42m 85.9M 1619M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 8b1bc71aa79e 2024-03-29T08:42:35.641 INFO:teuthology.orchestra.run.smithi037.stdout:osd.3 smithi148 running (26m) 9m ago 41m 109M 2642M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 1c656ab8ff86 2024-03-29T08:42:35.641 INFO:teuthology.orchestra.run.smithi037.stdout:osd.4 smithi148 running (26m) 9m ago 41m 101M 2642M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 966a9de11d22 2024-03-29T08:42:35.641 INFO:teuthology.orchestra.run.smithi037.stdout:osd.5 smithi148 running (25m) 9m ago 40m 89.4M 2642M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 474c4f9212ce 2024-03-29T08:42:35.641 INFO:teuthology.orchestra.run.smithi037.stdout:prometheus.smithi037 smithi037 *:9095 running (32m) 9m ago 44m 61.0M - 2.43.0 a07b618ecd1d 30c0223ec589 2024-03-29T08:42:35.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:35.642+0000 7f091ca00700 1 -- 172.21.15.37:0/3109300192 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f08fc06cc30 msgr2=0x7f08fc06f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:42:35.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:35.642+0000 7f091ca00700 1 --2- 172.21.15.37:0/3109300192 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f08fc06cc30 0x7f08fc06f0e0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f0918110bc0 tx=0x7f090400b040 comp rx=0 tx=0).stop 2024-03-29T08:42:35.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:35.642+0000 7f091ca00700 1 -- 172.21.15.37:0/3109300192 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0918102c40 msgr2=0x7f0918115a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:42:35.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:35.642+0000 7f091ca00700 1 --2- 172.21.15.37:0/3109300192 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0918102c40 0x7f0918115a80 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f0908009390 tx=0x7f0908009e30 comp rx=0 tx=0).stop 2024-03-29T08:42:35.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:35.642+0000 7f091ca00700 1 -- 172.21.15.37:0/3109300192 shutdown_connections 2024-03-29T08:42:35.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:35.642+0000 7f091ca00700 1 --2- 172.21.15.37:0/3109300192 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f08fc06cc30 0x7f08fc06f0e0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:42:35.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:35.642+0000 7f091ca00700 1 --2- 172.21.15.37:0/3109300192 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0918102c40 0x7f0918115a80 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:42:35.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:35.642+0000 7f091ca00700 1 --2- 172.21.15.37:0/3109300192 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f09181035e0 0x7f0918115fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:42:35.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:35.642+0000 7f091ca00700 1 -- 172.21.15.37:0/3109300192 >> 172.21.15.37:0/3109300192 conn(0x7f09180f8830 msgr2=0x7f09180f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:42:35.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:35.642+0000 7f091ca00700 1 -- 172.21.15.37:0/3109300192 shutdown_connections 2024-03-29T08:42:35.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:35.642+0000 7f091ca00700 1 -- 172.21.15.37:0/3109300192 wait complete. 2024-03-29T08:42:35.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:35.779+0000 7f43ea400700 1 -- 172.21.15.37:0/453114404 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f43e4070c10 msgr2=0x7f43e4109190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:42:35.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:35.779+0000 7f43ea400700 1 --2- 172.21.15.37:0/453114404 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f43e4070c10 0x7f43e4109190 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f43d0004970 tx=0x7f43d0009e30 comp rx=0 tx=0).stop 2024-03-29T08:42:35.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:35.779+0000 7f43ea400700 1 -- 172.21.15.37:0/453114404 shutdown_connections 2024-03-29T08:42:35.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:35.779+0000 7f43ea400700 1 --2- 172.21.15.37:0/453114404 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f43e4070c10 0x7f43e4109190 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:42:35.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:35.779+0000 7f43ea400700 1 --2- 172.21.15.37:0/453114404 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f43e4070300 0x7f43e40706d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:42:35.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:35.779+0000 7f43ea400700 1 -- 172.21.15.37:0/453114404 >> 172.21.15.37:0/453114404 conn(0x7f43e40f8970 msgr2=0x7f43e40fad80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:42:35.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:35.779+0000 7f43ea400700 1 -- 172.21.15.37:0/453114404 shutdown_connections 2024-03-29T08:42:35.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:35.779+0000 7f43ea400700 1 -- 172.21.15.37:0/453114404 wait complete. 2024-03-29T08:42:35.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:35.780+0000 7f43ea400700 1 Processor -- start 2024-03-29T08:42:35.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:35.780+0000 7f43ea400700 1 -- start start 2024-03-29T08:42:35.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:35.780+0000 7f43ea400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f43e4070300 0x7f43e4115cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:42:35.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:35.781+0000 7f43ea400700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f43e4070c10 0x7f43e4116200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:42:35.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:35.781+0000 7f43ea400700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43e4116850 con 0x7f43e4070300 2024-03-29T08:42:35.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:35.781+0000 7f43ea400700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43e410fc20 con 0x7f43e4070c10 2024-03-29T08:42:35.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:35.781+0000 7f43e3400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f43e4070300 0x7f43e4115cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:42:35.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:35.781+0000 7f43e3400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f43e4070300 0x7f43e4115cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42086/0 (socket says 172.21.15.37:42086) 2024-03-29T08:42:35.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:35.781+0000 7f43e3400700 1 -- 172.21.15.37:0/1486055507 learned_addr learned my addr 172.21.15.37:0/1486055507 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:42:35.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:35.781+0000 7f43e3400700 1 -- 172.21.15.37:0/1486055507 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f43e4070c10 msgr2=0x7f43e4116200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:42:35.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:35.781+0000 7f43e3400700 1 --2- 172.21.15.37:0/1486055507 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f43e4070c10 0x7f43e4116200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:42:35.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:35.781+0000 7f43e3400700 1 -- 172.21.15.37:0/1486055507 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43d0005040 con 0x7f43e4070300 2024-03-29T08:42:35.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:35.781+0000 7f43e3400700 1 --2- 172.21.15.37:0/1486055507 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f43e4070300 0x7f43e4115cc0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f43d400c8c0 tx=0x7f43d40044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:42:35.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:35.782+0000 7f43db400700 1 -- 172.21.15.37:0/1486055507 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f43d400a980 con 0x7f43e4070300 2024-03-29T08:42:35.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:35.782+0000 7f43db400700 1 -- 172.21.15.37:0/1486055507 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f43d400ce60 con 0x7f43e4070300 2024-03-29T08:42:35.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:35.782+0000 7f43ea400700 1 -- 172.21.15.37:0/1486055507 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43e410ff00 con 0x7f43e4070300 2024-03-29T08:42:35.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:35.782+0000 7f43db400700 1 -- 172.21.15.37:0/1486055507 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f43d40113a0 con 0x7f43e4070300 2024-03-29T08:42:35.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:35.782+0000 7f43ea400700 1 -- 172.21.15.37:0/1486055507 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43e4110440 con 0x7f43e4070300 2024-03-29T08:42:35.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:35.783+0000 7f43ea400700 1 -- 172.21.15.37:0/1486055507 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43e4071a40 con 0x7f43e4070300 2024-03-29T08:42:35.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:35.784+0000 7f43db400700 1 -- 172.21.15.37:0/1486055507 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f43d4012400 con 0x7f43e4070300 2024-03-29T08:42:35.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:35.785+0000 7f43db400700 1 --2- 172.21.15.37:0/1486055507 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f43c806cc30 0x7f43c806f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:42:35.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:35.785+0000 7f43db400700 1 -- 172.21.15.37:0/1486055507 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f43d408c0f0 con 0x7f43e4070300 2024-03-29T08:42:35.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:35.785+0000 7f43e2a00700 1 --2- 172.21.15.37:0/1486055507 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f43c806cc30 0x7f43c806f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:42:35.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:35.786+0000 7f43e2a00700 1 --2- 172.21.15.37:0/1486055507 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f43c806cc30 0x7f43c806f0e0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f43d0000c00 tx=0x7f43d0009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:42:35.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:35.790+0000 7f43db400700 1 -- 172.21.15.37:0/1486055507 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f43d4059bc0 con 0x7f43e4070300 2024-03-29T08:42:35.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:42:35 smithi037 ceph-mon[142325]: from='client.35710 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:42:36.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:42:35 smithi148 ceph-mon[106692]: from='client.35710 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:42:36.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:36.091+0000 7f43ea400700 1 -- 172.21.15.37:0/1486055507 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f43e404bed0 con 0x7f43e4070300 2024-03-29T08:42:36.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:36.091+0000 7f43db400700 1 -- 172.21.15.37:0/1486055507 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7f43d4059750 con 0x7f43e4070300 2024-03-29T08:42:36.091 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-29T08:42:36.091 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": { 2024-03-29T08:42:36.091 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 2 2024-03-29T08:42:36.091 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:42:36.091 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": { 2024-03-29T08:42:36.091 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 2 2024-03-29T08:42:36.091 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:42:36.091 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": { 2024-03-29T08:42:36.092 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 6 2024-03-29T08:42:36.092 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:42:36.092 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": { 2024-03-29T08:42:36.092 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 4 2024-03-29T08:42:36.092 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:42:36.092 INFO:teuthology.orchestra.run.smithi037.stdout: "overall": { 2024-03-29T08:42:36.092 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 14 2024-03-29T08:42:36.092 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-29T08:42:36.092 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-29T08:42:36.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:36.093+0000 7f43ea400700 1 -- 172.21.15.37:0/1486055507 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f43c806cc30 msgr2=0x7f43c806f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:42:36.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:36.093+0000 7f43ea400700 1 --2- 172.21.15.37:0/1486055507 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f43c806cc30 0x7f43c806f0e0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f43d0000c00 tx=0x7f43d0009e30 comp rx=0 tx=0).stop 2024-03-29T08:42:36.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:36.093+0000 7f43ea400700 1 -- 172.21.15.37:0/1486055507 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f43e4070300 msgr2=0x7f43e4115cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:42:36.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:36.093+0000 7f43ea400700 1 --2- 172.21.15.37:0/1486055507 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f43e4070300 0x7f43e4115cc0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f43d400c8c0 tx=0x7f43d40044c0 comp rx=0 tx=0).stop 2024-03-29T08:42:36.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:36.094+0000 7f43ea400700 1 -- 172.21.15.37:0/1486055507 shutdown_connections 2024-03-29T08:42:36.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:36.094+0000 7f43ea400700 1 --2- 172.21.15.37:0/1486055507 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f43c806cc30 0x7f43c806f0e0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:42:36.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:36.094+0000 7f43ea400700 1 --2- 172.21.15.37:0/1486055507 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f43e4070300 0x7f43e4115cc0 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:42:36.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:36.094+0000 7f43ea400700 1 --2- 172.21.15.37:0/1486055507 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f43e4070c10 0x7f43e4116200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:42:36.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:36.094+0000 7f43ea400700 1 -- 172.21.15.37:0/1486055507 >> 172.21.15.37:0/1486055507 conn(0x7f43e40f8970 msgr2=0x7f43e40f9de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:42:36.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:36.094+0000 7f43ea400700 1 -- 172.21.15.37:0/1486055507 shutdown_connections 2024-03-29T08:42:36.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:36.094+0000 7f43ea400700 1 -- 172.21.15.37:0/1486055507 wait complete. 2024-03-29T08:42:36.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:36.230+0000 7f1858800700 1 -- 172.21.15.37:0/643314410 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f18501037a0 msgr2=0x7f18500fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:42:36.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:36.230+0000 7f1858800700 1 --2- 172.21.15.37:0/643314410 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f18501037a0 0x7f18500fce10 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f1840004940 tx=0x7f1840009e30 comp rx=0 tx=0).stop 2024-03-29T08:42:36.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:36.230+0000 7f1858800700 1 -- 172.21.15.37:0/643314410 shutdown_connections 2024-03-29T08:42:36.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:36.230+0000 7f1858800700 1 --2- 172.21.15.37:0/643314410 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f18501037a0 0x7f18500fce10 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:42:36.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:36.230+0000 7f1858800700 1 --2- 172.21.15.37:0/643314410 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1850102e00 0x7f18501031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:42:36.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:36.230+0000 7f1858800700 1 -- 172.21.15.37:0/643314410 >> 172.21.15.37:0/643314410 conn(0x7f18500f89b0 msgr2=0x7f18500fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:42:36.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:36.230+0000 7f1858800700 1 -- 172.21.15.37:0/643314410 shutdown_connections 2024-03-29T08:42:36.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:36.230+0000 7f1858800700 1 -- 172.21.15.37:0/643314410 wait complete. 2024-03-29T08:42:36.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:36.231+0000 7f1858800700 1 Processor -- start 2024-03-29T08:42:36.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:36.231+0000 7f1858800700 1 -- start start 2024-03-29T08:42:36.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:36.232+0000 7f1858800700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1850102e00 0x7f1850115da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:42:36.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:36.232+0000 7f1858800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f18501037a0 0x7f18501162e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:42:36.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:36.232+0000 7f1858800700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1850116930 con 0x7f18501037a0 2024-03-29T08:42:36.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:36.232+0000 7f1858800700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f185010fd00 con 0x7f1850102e00 2024-03-29T08:42:36.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:36.232+0000 7f1855400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f18501037a0 0x7f18501162e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:42:36.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:36.232+0000 7f1855400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f18501037a0 0x7f18501162e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42106/0 (socket says 172.21.15.37:42106) 2024-03-29T08:42:36.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:36.232+0000 7f1855400700 1 -- 172.21.15.37:0/1089044316 learned_addr learned my addr 172.21.15.37:0/1089044316 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:42:36.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:36.232+0000 7f1855e00700 1 --2- 172.21.15.37:0/1089044316 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1850102e00 0x7f1850115da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:42:36.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:36.232+0000 7f1855400700 1 -- 172.21.15.37:0/1089044316 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1850102e00 msgr2=0x7f1850115da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:42:36.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:36.232+0000 7f1855400700 1 --2- 172.21.15.37:0/1089044316 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1850102e00 0x7f1850115da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:42:36.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:36.232+0000 7f1855400700 1 -- 172.21.15.37:0/1089044316 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1840005040 con 0x7f18501037a0 2024-03-29T08:42:36.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:36.233+0000 7f1855400700 1 --2- 172.21.15.37:0/1089044316 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f18501037a0 0x7f18501162e0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f1840009390 tx=0x7f1840009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:42:36.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:36.233+0000 7f184a000700 1 -- 172.21.15.37:0/1089044316 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f184000e040 con 0x7f18501037a0 2024-03-29T08:42:36.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:36.233+0000 7f1858800700 1 -- 172.21.15.37:0/1089044316 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f185010ff80 con 0x7f18501037a0 2024-03-29T08:42:36.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:36.233+0000 7f184a000700 1 -- 172.21.15.37:0/1089044316 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f1840013420 con 0x7f18501037a0 2024-03-29T08:42:36.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:36.233+0000 7f184a000700 1 -- 172.21.15.37:0/1089044316 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f184001c630 con 0x7f18501037a0 2024-03-29T08:42:36.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:36.233+0000 7f1858800700 1 -- 172.21.15.37:0/1089044316 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18501104c0 con 0x7f18501037a0 2024-03-29T08:42:36.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:36.234+0000 7f1858800700 1 -- 172.21.15.37:0/1089044316 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18500fe550 con 0x7f18501037a0 2024-03-29T08:42:36.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:36.235+0000 7f184a000700 1 -- 172.21.15.37:0/1089044316 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f1840013a00 con 0x7f18501037a0 2024-03-29T08:42:36.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:36.236+0000 7f184a000700 1 --2- 172.21.15.37:0/1089044316 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f183806cc30 0x7f183806f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:42:36.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:36.236+0000 7f184a000700 1 -- 172.21.15.37:0/1089044316 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f184008d1e0 con 0x7f18501037a0 2024-03-29T08:42:36.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:36.236+0000 7f1855e00700 1 --2- 172.21.15.37:0/1089044316 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f183806cc30 0x7f183806f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:42:36.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:36.237+0000 7f1855e00700 1 --2- 172.21.15.37:0/1089044316 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f183806cc30 0x7f183806f0e0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f1844004f40 tx=0x7f184400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:42:36.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:36.240+0000 7f184a000700 1 -- 172.21.15.37:0/1089044316 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f184005ac30 con 0x7f18501037a0 2024-03-29T08:42:36.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:36.505+0000 7f1858800700 1 -- 172.21.15.37:0/1089044316 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f185004bed0 con 0x7f18501037a0 2024-03-29T08:42:36.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:36.505+0000 7f184a000700 1 -- 172.21.15.37:0/1089044316 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 30 v30) v1 ==== 76+0+1841 (secure 0 0 0) 0x7f184005a7c0 con 0x7f18501037a0 2024-03-29T08:42:36.505 INFO:teuthology.orchestra.run.smithi037.stdout:e30 2024-03-29T08:42:36.505 INFO:teuthology.orchestra.run.smithi037.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-29T08:42:36.505 INFO:teuthology.orchestra.run.smithi037.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T08:42:36.505 INFO:teuthology.orchestra.run.smithi037.stdout:legacy client fscid: 1 2024-03-29T08:42:36.505 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:42:36.505 INFO:teuthology.orchestra.run.smithi037.stdout:Filesystem 'cephfs' (1) 2024-03-29T08:42:36.505 INFO:teuthology.orchestra.run.smithi037.stdout:fs_name cephfs 2024-03-29T08:42:36.505 INFO:teuthology.orchestra.run.smithi037.stdout:epoch 21 2024-03-29T08:42:36.505 INFO:teuthology.orchestra.run.smithi037.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-03-29T08:42:36.506 INFO:teuthology.orchestra.run.smithi037.stdout:created 2024-03-29T08:03:29.241269+0000 2024-03-29T08:42:36.506 INFO:teuthology.orchestra.run.smithi037.stdout:modified 2024-03-29T08:17:16.237238+0000 2024-03-29T08:42:36.506 INFO:teuthology.orchestra.run.smithi037.stdout:tableserver 0 2024-03-29T08:42:36.506 INFO:teuthology.orchestra.run.smithi037.stdout:root 0 2024-03-29T08:42:36.506 INFO:teuthology.orchestra.run.smithi037.stdout:session_timeout 60 2024-03-29T08:42:36.506 INFO:teuthology.orchestra.run.smithi037.stdout:session_autoclose 300 2024-03-29T08:42:36.506 INFO:teuthology.orchestra.run.smithi037.stdout:max_file_size 1099511627776 2024-03-29T08:42:36.506 INFO:teuthology.orchestra.run.smithi037.stdout:max_xattr_size 65536 2024-03-29T08:42:36.506 INFO:teuthology.orchestra.run.smithi037.stdout:required_client_features {} 2024-03-29T08:42:36.506 INFO:teuthology.orchestra.run.smithi037.stdout:last_failure 0 2024-03-29T08:42:36.506 INFO:teuthology.orchestra.run.smithi037.stdout:last_failure_osd_epoch 82 2024-03-29T08:42:36.506 INFO:teuthology.orchestra.run.smithi037.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T08:42:36.506 INFO:teuthology.orchestra.run.smithi037.stdout:max_mds 2 2024-03-29T08:42:36.506 INFO:teuthology.orchestra.run.smithi037.stdout:in 0,1 2024-03-29T08:42:36.506 INFO:teuthology.orchestra.run.smithi037.stdout:up {} 2024-03-29T08:42:36.507 INFO:teuthology.orchestra.run.smithi037.stdout:failed 0,1 2024-03-29T08:42:36.507 INFO:teuthology.orchestra.run.smithi037.stdout:damaged 2024-03-29T08:42:36.507 INFO:teuthology.orchestra.run.smithi037.stdout:stopped 2024-03-29T08:42:36.507 INFO:teuthology.orchestra.run.smithi037.stdout:data_pools [3] 2024-03-29T08:42:36.507 INFO:teuthology.orchestra.run.smithi037.stdout:metadata_pool 2 2024-03-29T08:42:36.507 INFO:teuthology.orchestra.run.smithi037.stdout:inline_data disabled 2024-03-29T08:42:36.507 INFO:teuthology.orchestra.run.smithi037.stdout:balancer 2024-03-29T08:42:36.507 INFO:teuthology.orchestra.run.smithi037.stdout:bal_rank_mask -1 2024-03-29T08:42:36.507 INFO:teuthology.orchestra.run.smithi037.stdout:standby_count_wanted 1 2024-03-29T08:42:36.507 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:42:36.507 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:42:36.507 INFO:teuthology.orchestra.run.smithi037.stdout:Standby daemons: 2024-03-29T08:42:36.507 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:42:36.507 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi037.mmwgxe{-1:34440} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.37:6828/1248142005,v1:172.21.15.37:6829/1248142005] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:42:36.507 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi037.iifsmh{-1:34444} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.37:6826/2060882959,v1:172.21.15.37:6827/2060882959] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:42:36.508 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi148.sqteih{-1:44395} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.148:6824/4078652242,v1:172.21.15.148:6825/4078652242] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:42:36.508 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi148.qukpjh{-1:45007} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.148:6826/2394751105,v1:172.21.15.148:6827/2394751105] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:42:36.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:36.507+0000 7f1858800700 1 -- 172.21.15.37:0/1089044316 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f183806cc30 msgr2=0x7f183806f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:42:36.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:36.507+0000 7f1858800700 1 --2- 172.21.15.37:0/1089044316 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f183806cc30 0x7f183806f0e0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f1844004f40 tx=0x7f184400b040 comp rx=0 tx=0).stop 2024-03-29T08:42:36.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:36.507+0000 7f1858800700 1 -- 172.21.15.37:0/1089044316 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f18501037a0 msgr2=0x7f18501162e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:42:36.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:36.507+0000 7f1858800700 1 --2- 172.21.15.37:0/1089044316 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f18501037a0 0x7f18501162e0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f1840009390 tx=0x7f1840009e30 comp rx=0 tx=0).stop 2024-03-29T08:42:36.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:36.508+0000 7f1858800700 1 -- 172.21.15.37:0/1089044316 shutdown_connections 2024-03-29T08:42:36.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:36.508+0000 7f1858800700 1 --2- 172.21.15.37:0/1089044316 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f183806cc30 0x7f183806f0e0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:42:36.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:36.508+0000 7f1858800700 1 --2- 172.21.15.37:0/1089044316 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1850102e00 0x7f1850115da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:42:36.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:36.508+0000 7f1858800700 1 --2- 172.21.15.37:0/1089044316 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f18501037a0 0x7f18501162e0 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:42:36.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:36.508+0000 7f1858800700 1 -- 172.21.15.37:0/1089044316 >> 172.21.15.37:0/1089044316 conn(0x7f18500f89b0 msgr2=0x7f18500f9a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:42:36.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:36.508+0000 7f1858800700 1 -- 172.21.15.37:0/1089044316 shutdown_connections 2024-03-29T08:42:36.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:36.508+0000 7f1858800700 1 -- 172.21.15.37:0/1089044316 wait complete. 2024-03-29T08:42:36.509 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 30 2024-03-29T08:42:36.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:36.646+0000 7f5e3d000700 1 -- 172.21.15.37:0/1291711247 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5e380fe830 msgr2=0x7f5e38106f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:42:36.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:36.646+0000 7f5e3d000700 1 --2- 172.21.15.37:0/1291711247 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5e380fe830 0x7f5e38106f00 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f5e24004940 tx=0x7f5e24009e30 comp rx=0 tx=0).stop 2024-03-29T08:42:36.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:36.646+0000 7f5e3d000700 1 -- 172.21.15.37:0/1291711247 shutdown_connections 2024-03-29T08:42:36.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:36.646+0000 7f5e3d000700 1 --2- 172.21.15.37:0/1291711247 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5e380fe830 0x7f5e38106f00 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:42:36.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:36.646+0000 7f5e3d000700 1 --2- 172.21.15.37:0/1291711247 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5e380fde90 0x7f5e380fe260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:42:36.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:36.646+0000 7f5e3d000700 1 -- 172.21.15.37:0/1291711247 >> 172.21.15.37:0/1291711247 conn(0x7f5e38072bc0 msgr2=0x7f5e38072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:42:36.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:36.646+0000 7f5e3d000700 1 -- 172.21.15.37:0/1291711247 shutdown_connections 2024-03-29T08:42:36.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:36.646+0000 7f5e3d000700 1 -- 172.21.15.37:0/1291711247 wait complete. 2024-03-29T08:42:36.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:36.647+0000 7f5e3d000700 1 Processor -- start 2024-03-29T08:42:36.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:36.647+0000 7f5e3d000700 1 -- start start 2024-03-29T08:42:36.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:36.648+0000 7f5e3d000700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5e380fde90 0x7f5e38115d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:42:36.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:36.648+0000 7f5e3d000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5e380fe830 0x7f5e38116250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:42:36.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:36.648+0000 7f5e3d000700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e381168a0 con 0x7f5e380fe830 2024-03-29T08:42:36.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:36.648+0000 7f5e3d000700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e3810fc70 con 0x7f5e380fde90 2024-03-29T08:42:36.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:36.648+0000 7f5e35400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5e380fe830 0x7f5e38116250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:42:36.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:36.648+0000 7f5e35400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5e380fe830 0x7f5e38116250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42132/0 (socket says 172.21.15.37:42132) 2024-03-29T08:42:36.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:36.648+0000 7f5e35400700 1 -- 172.21.15.37:0/299953716 learned_addr learned my addr 172.21.15.37:0/299953716 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:42:36.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:36.648+0000 7f5e35e00700 1 --2- 172.21.15.37:0/299953716 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5e380fde90 0x7f5e38115d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:42:36.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:36.648+0000 7f5e35400700 1 -- 172.21.15.37:0/299953716 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5e380fde90 msgr2=0x7f5e38115d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:42:36.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:36.648+0000 7f5e35400700 1 --2- 172.21.15.37:0/299953716 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5e380fde90 0x7f5e38115d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:42:36.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:36.648+0000 7f5e35400700 1 -- 172.21.15.37:0/299953716 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e24005040 con 0x7f5e380fe830 2024-03-29T08:42:36.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:36.648+0000 7f5e35400700 1 --2- 172.21.15.37:0/299953716 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5e380fe830 0x7f5e38116250 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f5e240093b0 tx=0x7f5e24009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:42:36.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:36.649+0000 7f5e2e000700 1 -- 172.21.15.37:0/299953716 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5e2400e040 con 0x7f5e380fe830 2024-03-29T08:42:36.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:36.649+0000 7f5e2e000700 1 -- 172.21.15.37:0/299953716 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f5e24004290 con 0x7f5e380fe830 2024-03-29T08:42:36.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:36.649+0000 7f5e3d000700 1 -- 172.21.15.37:0/299953716 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e3810fef0 con 0x7f5e380fe830 2024-03-29T08:42:36.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:36.649+0000 7f5e2e000700 1 -- 172.21.15.37:0/299953716 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5e2401c3f0 con 0x7f5e380fe830 2024-03-29T08:42:36.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:36.649+0000 7f5e3d000700 1 -- 172.21.15.37:0/299953716 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e38110430 con 0x7f5e380fe830 2024-03-29T08:42:36.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:36.650+0000 7f5e3d000700 1 -- 172.21.15.37:0/299953716 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e38106a20 con 0x7f5e380fe830 2024-03-29T08:42:36.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:36.651+0000 7f5e2e000700 1 -- 172.21.15.37:0/299953716 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f5e24013790 con 0x7f5e380fe830 2024-03-29T08:42:36.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:36.652+0000 7f5e2e000700 1 --2- 172.21.15.37:0/299953716 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f5e1c06cc30 0x7f5e1c06f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:42:36.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:36.652+0000 7f5e2e000700 1 -- 172.21.15.37:0/299953716 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f5e2408d380 con 0x7f5e380fe830 2024-03-29T08:42:36.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:36.652+0000 7f5e35e00700 1 --2- 172.21.15.37:0/299953716 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f5e1c06cc30 0x7f5e1c06f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:42:36.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:36.653+0000 7f5e35e00700 1 --2- 172.21.15.37:0/299953716 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f5e1c06cc30 0x7f5e1c06f0e0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f5e28004990 tx=0x7f5e28009ee0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:42:36.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:36.656+0000 7f5e2e000700 1 -- 172.21.15.37:0/299953716 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f5e2405add0 con 0x7f5e380fe830 2024-03-29T08:42:36.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:36.889+0000 7f5e3d000700 1 -- 172.21.15.37:0/299953716 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5e381113d0 con 0x7f5e1c06cc30 2024-03-29T08:42:36.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:36.891+0000 7f5e2e000700 1 -- 172.21.15.37:0/299953716 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f5e381113d0 con 0x7f5e1c06cc30 2024-03-29T08:42:36.891 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-29T08:42:36.891 INFO:teuthology.orchestra.run.smithi037.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59", 2024-03-29T08:42:36.891 INFO:teuthology.orchestra.run.smithi037.stdout: "in_progress": true, 2024-03-29T08:42:36.891 INFO:teuthology.orchestra.run.smithi037.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-29T08:42:36.891 INFO:teuthology.orchestra.run.smithi037.stdout: "services_complete": [ 2024-03-29T08:42:36.891 INFO:teuthology.orchestra.run.smithi037.stdout: "crash", 2024-03-29T08:42:36.891 INFO:teuthology.orchestra.run.smithi037.stdout: "osd", 2024-03-29T08:42:36.891 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr", 2024-03-29T08:42:36.891 INFO:teuthology.orchestra.run.smithi037.stdout: "mds", 2024-03-29T08:42:36.891 INFO:teuthology.orchestra.run.smithi037.stdout: "mon" 2024-03-29T08:42:36.891 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-29T08:42:36.891 INFO:teuthology.orchestra.run.smithi037.stdout: "progress": "16/21 daemons upgraded", 2024-03-29T08:42:36.892 INFO:teuthology.orchestra.run.smithi037.stdout: "message": "Currently upgrading mds daemons", 2024-03-29T08:42:36.892 INFO:teuthology.orchestra.run.smithi037.stdout: "is_paused": false 2024-03-29T08:42:36.892 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-29T08:42:36.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:36.893+0000 7f5e3d000700 1 -- 172.21.15.37:0/299953716 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f5e1c06cc30 msgr2=0x7f5e1c06f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:42:36.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:36.893+0000 7f5e3d000700 1 --2- 172.21.15.37:0/299953716 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f5e1c06cc30 0x7f5e1c06f0e0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f5e28004990 tx=0x7f5e28009ee0 comp rx=0 tx=0).stop 2024-03-29T08:42:36.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:36.893+0000 7f5e3d000700 1 -- 172.21.15.37:0/299953716 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5e380fe830 msgr2=0x7f5e38116250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:42:36.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:36.893+0000 7f5e3d000700 1 --2- 172.21.15.37:0/299953716 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5e380fe830 0x7f5e38116250 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f5e240093b0 tx=0x7f5e24009e30 comp rx=0 tx=0).stop 2024-03-29T08:42:36.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:36.893+0000 7f5e3d000700 1 -- 172.21.15.37:0/299953716 shutdown_connections 2024-03-29T08:42:36.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:36.893+0000 7f5e3d000700 1 --2- 172.21.15.37:0/299953716 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f5e1c06cc30 0x7f5e1c06f0e0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:42:36.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:36.893+0000 7f5e3d000700 1 --2- 172.21.15.37:0/299953716 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5e380fde90 0x7f5e38115d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:42:36.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:36.893+0000 7f5e3d000700 1 --2- 172.21.15.37:0/299953716 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5e380fe830 0x7f5e38116250 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:42:36.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:36.893+0000 7f5e3d000700 1 -- 172.21.15.37:0/299953716 >> 172.21.15.37:0/299953716 conn(0x7f5e38072bc0 msgr2=0x7f5e380fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:42:36.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:36.893+0000 7f5e3d000700 1 -- 172.21.15.37:0/299953716 shutdown_connections 2024-03-29T08:42:36.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:36.893+0000 7f5e3d000700 1 -- 172.21.15.37:0/299953716 wait complete. 2024-03-29T08:42:36.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:42:36 smithi037 ceph-mon[142325]: pgmap v952: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:42:36.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:42:36 smithi037 ceph-mon[142325]: from='client.35714 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:42:36.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:42:36 smithi037 ceph-mon[142325]: from='client.35718 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:42:36.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:42:36 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/1486055507' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:42:36.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:42:36 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/1089044316' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T08:42:37.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:37.031+0000 7fb4af400700 1 -- 172.21.15.37:0/3788846565 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb4a8102e00 msgr2=0x7fb4a81031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:42:37.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:37.031+0000 7fb4af400700 1 --2- 172.21.15.37:0/3788846565 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb4a8102e00 0x7fb4a81031d0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7fb49c004940 tx=0x7fb49c009e30 comp rx=0 tx=0).stop 2024-03-29T08:42:37.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:37.031+0000 7fb4af400700 1 -- 172.21.15.37:0/3788846565 shutdown_connections 2024-03-29T08:42:37.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:37.031+0000 7fb4af400700 1 --2- 172.21.15.37:0/3788846565 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb4a81037a0 0x7fb4a80fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:42:37.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:37.031+0000 7fb4af400700 1 --2- 172.21.15.37:0/3788846565 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb4a8102e00 0x7fb4a81031d0 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:42:37.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:37.031+0000 7fb4af400700 1 -- 172.21.15.37:0/3788846565 >> 172.21.15.37:0/3788846565 conn(0x7fb4a80f89b0 msgr2=0x7fb4a80fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:42:37.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:37.031+0000 7fb4af400700 1 -- 172.21.15.37:0/3788846565 shutdown_connections 2024-03-29T08:42:37.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:37.031+0000 7fb4af400700 1 -- 172.21.15.37:0/3788846565 wait complete. 2024-03-29T08:42:37.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:37.032+0000 7fb4af400700 1 Processor -- start 2024-03-29T08:42:37.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:37.032+0000 7fb4af400700 1 -- start start 2024-03-29T08:42:37.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:37.032+0000 7fb4af400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb4a8102e00 0x7fb4a8115cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:42:37.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:37.032+0000 7fb4af400700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb4a81037a0 0x7fb4a8116210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:42:37.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:37.032+0000 7fb4af400700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4a8116860 con 0x7fb4a8102e00 2024-03-29T08:42:37.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:37.032+0000 7fb4af400700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4a810fc30 con 0x7fb4a81037a0 2024-03-29T08:42:37.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:37.033+0000 7fb4aca00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb4a8102e00 0x7fb4a8115cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:42:37.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:37.033+0000 7fb4aca00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb4a8102e00 0x7fb4a8115cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42154/0 (socket says 172.21.15.37:42154) 2024-03-29T08:42:37.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:37.033+0000 7fb4aca00700 1 -- 172.21.15.37:0/2491703965 learned_addr learned my addr 172.21.15.37:0/2491703965 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:42:37.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:37.033+0000 7fb4a7e00700 1 --2- 172.21.15.37:0/2491703965 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb4a81037a0 0x7fb4a8116210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:42:37.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:37.033+0000 7fb4aca00700 1 -- 172.21.15.37:0/2491703965 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb4a81037a0 msgr2=0x7fb4a8116210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:42:37.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:37.033+0000 7fb4aca00700 1 --2- 172.21.15.37:0/2491703965 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb4a81037a0 0x7fb4a8116210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:42:37.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:37.033+0000 7fb4aca00700 1 -- 172.21.15.37:0/2491703965 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb49c005040 con 0x7fb4a8102e00 2024-03-29T08:42:37.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:37.033+0000 7fb4aca00700 1 --2- 172.21.15.37:0/2491703965 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb4a8102e00 0x7fb4a8115cd0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7fb49c009180 tx=0x7fb49c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:42:37.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:37.034+0000 7fb4a4c00700 1 -- 172.21.15.37:0/2491703965 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb49c00e040 con 0x7fb4a8102e00 2024-03-29T08:42:37.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:37.034+0000 7fb4a4c00700 1 -- 172.21.15.37:0/2491703965 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fb49c004290 con 0x7fb4a8102e00 2024-03-29T08:42:37.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:37.034+0000 7fb4af400700 1 -- 172.21.15.37:0/2491703965 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4a810feb0 con 0x7fb4a8102e00 2024-03-29T08:42:37.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:37.034+0000 7fb4a4c00700 1 -- 172.21.15.37:0/2491703965 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb49c013e60 con 0x7fb4a8102e00 2024-03-29T08:42:37.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:37.034+0000 7fb4af400700 1 -- 172.21.15.37:0/2491703965 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4a8110470 con 0x7fb4a8102e00 2024-03-29T08:42:37.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:37.035+0000 7fb4af400700 1 -- 172.21.15.37:0/2491703965 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4a80fe550 con 0x7fb4a8102e00 2024-03-29T08:42:37.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:37.036+0000 7fb4a4c00700 1 -- 172.21.15.37:0/2491703965 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fb49c0136b0 con 0x7fb4a8102e00 2024-03-29T08:42:37.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:37.037+0000 7fb4a4c00700 1 --2- 172.21.15.37:0/2491703965 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fb49006cbb0 0x7fb49006f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:42:37.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:37.037+0000 7fb4a4c00700 1 -- 172.21.15.37:0/2491703965 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fb49c08d310 con 0x7fb4a8102e00 2024-03-29T08:42:37.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:37.037+0000 7fb4a7e00700 1 --2- 172.21.15.37:0/2491703965 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fb49006cbb0 0x7fb49006f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:42:37.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:37.038+0000 7fb4a7e00700 1 --2- 172.21.15.37:0/2491703965 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fb49006cbb0 0x7fb49006f060 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fb4a8111570 tx=0x7fb49400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:42:37.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:37.041+0000 7fb4a4c00700 1 -- 172.21.15.37:0/2491703965 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fb49c05ae10 con 0x7fb4a8102e00 2024-03-29T08:42:37.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:42:36 smithi148 ceph-mon[106692]: pgmap v952: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:42:37.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:42:36 smithi148 ceph-mon[106692]: from='client.35714 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:42:37.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:42:36 smithi148 ceph-mon[106692]: from='client.35718 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:42:37.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:42:36 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/1486055507' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:42:37.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:42:36 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/1089044316' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T08:42:37.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:37.342+0000 7fb4af400700 1 -- 172.21.15.37:0/2491703965 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fb4a804bf00 con 0x7fb4a8102e00 2024-03-29T08:42:37.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:37.343+0000 7fb4a4c00700 1 -- 172.21.15.37:0/2491703965 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+3583 (secure 0 0 0) 0x7fb49c05a9a0 con 0x7fb4a8102e00 2024-03-29T08:42:37.342 INFO:teuthology.orchestra.run.smithi037.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem is offline 2024-03-29T08:42:37.343 INFO:teuthology.orchestra.run.smithi037.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-03-29T08:42:37.343 INFO:teuthology.orchestra.run.smithi037.stdout: host smithi148 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/mon.smithi148/config 2024-03-29T08:42:37.343 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/podman: timeout after 895 seconds 2024-03-29T08:42:37.343 INFO:teuthology.orchestra.run.smithi037.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59 -e NODE_NAME=smithi148 -e CEPH_USE_RANDOM_NONCE=1 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4:/var/run/ceph:z -v /var/log/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4:/var/log/ceph:z -v /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpkx2rz2d4:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59 inventory --format=json-pretty --filter-for-batch 2024-03-29T08:42:37.343 INFO:teuthology.orchestra.run.smithi037.stdout:Traceback (most recent call last): 2024-03-29T08:42:37.343 INFO:teuthology.orchestra.run.smithi037.stdout: File "/usr/lib64/python3.6/runpy.py", line 193, in _run_module_as_main 2024-03-29T08:42:37.343 INFO:teuthology.orchestra.run.smithi037.stdout: "__main__", mod_spec) 2024-03-29T08:42:37.343 INFO:teuthology.orchestra.run.smithi037.stdout: File "/usr/lib64/python3.6/runpy.py", line 85, in _run_code 2024-03-29T08:42:37.343 INFO:teuthology.orchestra.run.smithi037.stdout: exec(code, run_globals) 2024-03-29T08:42:37.343 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 10889, in 2024-03-29T08:42:37.343 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 10877, in main 2024-03-29T08:42:37.343 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 2576, in _infer_config 2024-03-29T08:42:37.343 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 2492, in _infer_fsid 2024-03-29T08:42:37.344 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 2604, in _infer_image 2024-03-29T08:42:37.344 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 2479, in _validate_fsid 2024-03-29T08:42:37.344 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 7145, in command_ceph_volume 2024-03-29T08:42:37.344 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 2267, in call_throws 2024-03-29T08:42:37.344 INFO:teuthology.orchestra.run.smithi037.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59 -e NODE_NAME=smithi148 -e CEPH_USE_RANDOM_NONCE=1 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4:/var/run/ceph:z -v /var/log/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4:/var/log/ceph:z -v /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpkx2rz2d4:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59 inventory --format=json-pretty --filter-for-batch 2024-03-29T08:42:37.344 INFO:teuthology.orchestra.run.smithi037.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-03-29T08:42:37.344 INFO:teuthology.orchestra.run.smithi037.stdout: fs cephfs is degraded 2024-03-29T08:42:37.344 INFO:teuthology.orchestra.run.smithi037.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-03-29T08:42:37.344 INFO:teuthology.orchestra.run.smithi037.stdout: fs cephfs is offline because no MDS is active for it. 2024-03-29T08:42:37.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:37.345+0000 7fb4af400700 1 -- 172.21.15.37:0/2491703965 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fb49006cbb0 msgr2=0x7fb49006f060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:42:37.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:37.345+0000 7fb4af400700 1 --2- 172.21.15.37:0/2491703965 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fb49006cbb0 0x7fb49006f060 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fb4a8111570 tx=0x7fb49400b040 comp rx=0 tx=0).stop 2024-03-29T08:42:37.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:37.345+0000 7fb4af400700 1 -- 172.21.15.37:0/2491703965 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb4a8102e00 msgr2=0x7fb4a8115cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:42:37.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:37.345+0000 7fb4af400700 1 --2- 172.21.15.37:0/2491703965 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb4a8102e00 0x7fb4a8115cd0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7fb49c009180 tx=0x7fb49c009e30 comp rx=0 tx=0).stop 2024-03-29T08:42:37.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:37.346+0000 7fb4af400700 1 -- 172.21.15.37:0/2491703965 shutdown_connections 2024-03-29T08:42:37.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:37.346+0000 7fb4af400700 1 --2- 172.21.15.37:0/2491703965 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fb49006cbb0 0x7fb49006f060 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:42:37.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:37.346+0000 7fb4af400700 1 --2- 172.21.15.37:0/2491703965 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb4a8102e00 0x7fb4a8115cd0 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:42:37.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:37.346+0000 7fb4af400700 1 --2- 172.21.15.37:0/2491703965 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb4a81037a0 0x7fb4a8116210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:42:37.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:37.346+0000 7fb4af400700 1 -- 172.21.15.37:0/2491703965 >> 172.21.15.37:0/2491703965 conn(0x7fb4a80f89b0 msgr2=0x7fb4a80f99e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:42:37.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:37.346+0000 7fb4af400700 1 -- 172.21.15.37:0/2491703965 shutdown_connections 2024-03-29T08:42:37.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:42:37.346+0000 7fb4af400700 1 -- 172.21.15.37:0/2491703965 wait complete. 2024-03-29T08:42:37.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:42:37 smithi037 ceph-mon[142325]: from='client.35730 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:42:37.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:42:37 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/2491703965' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T08:42:38.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:42:37 smithi148 ceph-mon[106692]: from='client.35730 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:42:38.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:42:37 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/2491703965' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T08:42:38.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:42:38 smithi037 ceph-mon[142325]: pgmap v953: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:42:39.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:42:38 smithi148 ceph-mon[106692]: pgmap v953: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:42:40.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:42:40 smithi037 ceph-mon[142325]: pgmap v954: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:42:40.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:42:40 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:42:41.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:42:40 smithi148 ceph-mon[106692]: pgmap v954: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:42:41.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:42:40 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:42:42.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:42:42 smithi037 ceph-mon[142325]: pgmap v955: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:42:43.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:42:42 smithi148 ceph-mon[106692]: pgmap v955: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:42:44.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:42:44 smithi037 ceph-mon[142325]: pgmap v956: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:42:45.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:42:44 smithi148 ceph-mon[106692]: pgmap v956: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:42:46.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:42:46 smithi037 ceph-mon[142325]: pgmap v957: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:42:47.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:42:46 smithi148 ceph-mon[106692]: pgmap v957: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:42:48.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:42:48 smithi037 ceph-mon[142325]: pgmap v958: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:42:49.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:42:48 smithi148 ceph-mon[106692]: pgmap v958: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:42:50.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:42:50 smithi037 ceph-mon[142325]: pgmap v959: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:42:51.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:42:50 smithi148 ceph-mon[106692]: pgmap v959: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:42:52.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:42:52 smithi037 ceph-mon[142325]: pgmap v960: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:42:53.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:42:52 smithi148 ceph-mon[106692]: pgmap v960: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:42:54.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:42:54 smithi037 ceph-mon[142325]: pgmap v961: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:42:54.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:42:54 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:42:55.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:42:54 smithi148 ceph-mon[106692]: pgmap v961: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:42:55.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:42:54 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:42:56.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:42:56 smithi037 ceph-mon[142325]: pgmap v962: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:42:57.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:42:56 smithi148 ceph-mon[106692]: pgmap v962: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:42:58.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:42:58 smithi037 ceph-mon[142325]: pgmap v963: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:42:59.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:42:58 smithi148 ceph-mon[106692]: pgmap v963: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:43:00.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:43:00 smithi037 ceph-mon[142325]: pgmap v964: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:43:01.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:43:00 smithi148 ceph-mon[106692]: pgmap v964: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:43:02.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:43:02 smithi037 ceph-mon[142325]: pgmap v965: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:43:03.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:43:02 smithi148 ceph-mon[106692]: pgmap v965: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:43:04.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:43:04 smithi037 ceph-mon[142325]: pgmap v966: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:43:05.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:43:04 smithi148 ceph-mon[106692]: pgmap v966: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:43:06.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:43:06 smithi037 ceph-mon[142325]: pgmap v967: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:43:07.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:43:06 smithi148 ceph-mon[106692]: pgmap v967: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:43:07.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:07.499+0000 7fdce4a00700 1 -- 172.21.15.37:0/3150996109 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdce0102e00 msgr2=0x7fdce01031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:43:07.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:07.499+0000 7fdce4a00700 1 --2- 172.21.15.37:0/3150996109 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdce0102e00 0x7fdce01031d0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7fdcd0004940 tx=0x7fdcd0009e30 comp rx=0 tx=0).stop 2024-03-29T08:43:07.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:07.499+0000 7fdce4a00700 1 -- 172.21.15.37:0/3150996109 shutdown_connections 2024-03-29T08:43:07.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:07.499+0000 7fdce4a00700 1 --2- 172.21.15.37:0/3150996109 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdce01037a0 0x7fdce00fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:43:07.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:07.499+0000 7fdce4a00700 1 --2- 172.21.15.37:0/3150996109 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdce0102e00 0x7fdce01031d0 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:43:07.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:07.499+0000 7fdce4a00700 1 -- 172.21.15.37:0/3150996109 >> 172.21.15.37:0/3150996109 conn(0x7fdce00f89b0 msgr2=0x7fdce00fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:43:07.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:07.499+0000 7fdce4a00700 1 -- 172.21.15.37:0/3150996109 shutdown_connections 2024-03-29T08:43:07.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:07.499+0000 7fdce4a00700 1 -- 172.21.15.37:0/3150996109 wait complete. 2024-03-29T08:43:07.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:07.500+0000 7fdce4a00700 1 Processor -- start 2024-03-29T08:43:07.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:07.500+0000 7fdce4a00700 1 -- start start 2024-03-29T08:43:07.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:07.500+0000 7fdce4a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdce0102e00 0x7fdce0115c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:43:07.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:07.501+0000 7fdce4a00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdce01037a0 0x7fdce0116190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:43:07.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:07.501+0000 7fdce4a00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdce01167e0 con 0x7fdce0102e00 2024-03-29T08:43:07.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:07.501+0000 7fdce4a00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdce010fbb0 con 0x7fdce01037a0 2024-03-29T08:43:07.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:07.501+0000 7fdcdde00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdce0102e00 0x7fdce0115c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:43:07.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:07.501+0000 7fdcdde00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdce0102e00 0x7fdce0115c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56584/0 (socket says 172.21.15.37:56584) 2024-03-29T08:43:07.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:07.501+0000 7fdcdde00700 1 -- 172.21.15.37:0/1856676774 learned_addr learned my addr 172.21.15.37:0/1856676774 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:43:07.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:07.501+0000 7fdcdd400700 1 --2- 172.21.15.37:0/1856676774 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdce01037a0 0x7fdce0116190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:43:07.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:07.501+0000 7fdcdde00700 1 -- 172.21.15.37:0/1856676774 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdce01037a0 msgr2=0x7fdce0116190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:43:07.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:07.501+0000 7fdcdde00700 1 --2- 172.21.15.37:0/1856676774 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdce01037a0 0x7fdce0116190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:43:07.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:07.501+0000 7fdcdde00700 1 -- 172.21.15.37:0/1856676774 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcd0005040 con 0x7fdce0102e00 2024-03-29T08:43:07.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:07.501+0000 7fdcdd400700 1 --2- 172.21.15.37:0/1856676774 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdce01037a0 0x7fdce0116190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:43:07.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:07.502+0000 7fdcdde00700 1 --2- 172.21.15.37:0/1856676774 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdce0102e00 0x7fdce0115c50 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fdcd0007f50 tx=0x7fdcd0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:43:07.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:07.502+0000 7fdcd6000700 1 -- 172.21.15.37:0/1856676774 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdcd0021070 con 0x7fdce0102e00 2024-03-29T08:43:07.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:07.502+0000 7fdcd6000700 1 -- 172.21.15.37:0/1856676774 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fdcd0013e00 con 0x7fdce0102e00 2024-03-29T08:43:07.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:07.502+0000 7fdce4a00700 1 -- 172.21.15.37:0/1856676774 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdce010fe30 con 0x7fdce0102e00 2024-03-29T08:43:07.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:07.502+0000 7fdcd6000700 1 -- 172.21.15.37:0/1856676774 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdcd000e070 con 0x7fdce0102e00 2024-03-29T08:43:07.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:07.502+0000 7fdce4a00700 1 -- 172.21.15.37:0/1856676774 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdce01103f0 con 0x7fdce0102e00 2024-03-29T08:43:07.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:07.503+0000 7fdce4a00700 1 -- 172.21.15.37:0/1856676774 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdce00fe550 con 0x7fdce0102e00 2024-03-29T08:43:07.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:07.504+0000 7fdcd6000700 1 -- 172.21.15.37:0/1856676774 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fdcd0013950 con 0x7fdce0102e00 2024-03-29T08:43:07.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:07.506+0000 7fdcd6000700 1 --2- 172.21.15.37:0/1856676774 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fdcc406cc30 0x7fdcc406f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:43:07.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:07.506+0000 7fdcd6000700 1 -- 172.21.15.37:0/1856676774 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fdcd008d330 con 0x7fdce0102e00 2024-03-29T08:43:07.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:07.506+0000 7fdcdd400700 1 --2- 172.21.15.37:0/1856676774 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fdcc406cc30 0x7fdcc406f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:43:07.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:07.506+0000 7fdcdd400700 1 --2- 172.21.15.37:0/1856676774 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fdcc406cc30 0x7fdcc406f0e0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fdccc004600 tx=0x7fdccc00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:43:07.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:07.511+0000 7fdcd6000700 1 -- 172.21.15.37:0/1856676774 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fdcd005ad80 con 0x7fdce0102e00 2024-03-29T08:43:07.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:07.745+0000 7fdce4a00700 1 -- 172.21.15.37:0/1856676774 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fdce01112a0 con 0x7fdcc406cc30 2024-03-29T08:43:07.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:07.746+0000 7fdcd6000700 1 -- 172.21.15.37:0/1856676774 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fdce01112a0 con 0x7fdcc406cc30 2024-03-29T08:43:07.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:07.748+0000 7fdce4a00700 1 -- 172.21.15.37:0/1856676774 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fdcc406cc30 msgr2=0x7fdcc406f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:43:07.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:07.748+0000 7fdce4a00700 1 --2- 172.21.15.37:0/1856676774 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fdcc406cc30 0x7fdcc406f0e0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fdccc004600 tx=0x7fdccc00b040 comp rx=0 tx=0).stop 2024-03-29T08:43:07.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:07.748+0000 7fdce4a00700 1 -- 172.21.15.37:0/1856676774 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdce0102e00 msgr2=0x7fdce0115c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:43:07.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:07.748+0000 7fdce4a00700 1 --2- 172.21.15.37:0/1856676774 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdce0102e00 0x7fdce0115c50 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fdcd0007f50 tx=0x7fdcd0009e30 comp rx=0 tx=0).stop 2024-03-29T08:43:07.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:07.748+0000 7fdce4a00700 1 -- 172.21.15.37:0/1856676774 shutdown_connections 2024-03-29T08:43:07.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:07.748+0000 7fdce4a00700 1 --2- 172.21.15.37:0/1856676774 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fdcc406cc30 0x7fdcc406f0e0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:43:07.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:07.748+0000 7fdce4a00700 1 --2- 172.21.15.37:0/1856676774 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdce0102e00 0x7fdce0115c50 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:43:07.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:07.748+0000 7fdce4a00700 1 --2- 172.21.15.37:0/1856676774 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdce01037a0 0x7fdce0116190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:43:07.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:07.748+0000 7fdce4a00700 1 -- 172.21.15.37:0/1856676774 >> 172.21.15.37:0/1856676774 conn(0x7fdce00f89b0 msgr2=0x7fdce00f9900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:43:07.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:07.749+0000 7fdce4a00700 1 -- 172.21.15.37:0/1856676774 shutdown_connections 2024-03-29T08:43:07.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:07.749+0000 7fdce4a00700 1 -- 172.21.15.37:0/1856676774 wait complete. 2024-03-29T08:43:07.764 INFO:teuthology.orchestra.run.smithi037.stdout:true 2024-03-29T08:43:07.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:07.887+0000 7f37caa00700 1 -- 172.21.15.37:0/3472887850 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f37c4102e00 msgr2=0x7f37c41031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:43:07.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:07.887+0000 7f37caa00700 1 --2- 172.21.15.37:0/3472887850 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f37c4102e00 0x7f37c41031d0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f37b4004940 tx=0x7f37b4009e30 comp rx=0 tx=0).stop 2024-03-29T08:43:07.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:07.887+0000 7f37caa00700 1 -- 172.21.15.37:0/3472887850 shutdown_connections 2024-03-29T08:43:07.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:07.887+0000 7f37caa00700 1 --2- 172.21.15.37:0/3472887850 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f37c41037a0 0x7f37c40fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:43:07.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:07.887+0000 7f37caa00700 1 --2- 172.21.15.37:0/3472887850 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f37c4102e00 0x7f37c41031d0 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:43:07.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:07.887+0000 7f37caa00700 1 -- 172.21.15.37:0/3472887850 >> 172.21.15.37:0/3472887850 conn(0x7f37c40f89b0 msgr2=0x7f37c40fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:43:07.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:07.887+0000 7f37caa00700 1 -- 172.21.15.37:0/3472887850 shutdown_connections 2024-03-29T08:43:07.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:07.887+0000 7f37caa00700 1 -- 172.21.15.37:0/3472887850 wait complete. 2024-03-29T08:43:07.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:07.888+0000 7f37caa00700 1 Processor -- start 2024-03-29T08:43:07.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:07.888+0000 7f37caa00700 1 -- start start 2024-03-29T08:43:07.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:07.889+0000 7f37caa00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f37c4102e00 0x7f37c4115cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:43:07.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:07.889+0000 7f37caa00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f37c41037a0 0x7f37c4116230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:43:07.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:07.889+0000 7f37caa00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37c4116880 con 0x7f37c41037a0 2024-03-29T08:43:07.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:07.889+0000 7f37caa00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37c410fc50 con 0x7f37c4102e00 2024-03-29T08:43:07.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:07.889+0000 7f37c3400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f37c41037a0 0x7f37c4116230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:43:07.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:07.889+0000 7f37c3400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f37c41037a0 0x7f37c4116230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56604/0 (socket says 172.21.15.37:56604) 2024-03-29T08:43:07.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:07.889+0000 7f37c3400700 1 -- 172.21.15.37:0/1100341412 learned_addr learned my addr 172.21.15.37:0/1100341412 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:43:07.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:07.889+0000 7f37c3e00700 1 --2- 172.21.15.37:0/1100341412 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f37c4102e00 0x7f37c4115cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:43:07.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:07.889+0000 7f37c3400700 1 -- 172.21.15.37:0/1100341412 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f37c4102e00 msgr2=0x7f37c4115cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:43:07.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:07.889+0000 7f37c3400700 1 --2- 172.21.15.37:0/1100341412 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f37c4102e00 0x7f37c4115cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:43:07.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:07.889+0000 7f37c3400700 1 -- 172.21.15.37:0/1100341412 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37b4005040 con 0x7f37c41037a0 2024-03-29T08:43:07.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:07.889+0000 7f37c3e00700 1 --2- 172.21.15.37:0/1100341412 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f37c4102e00 0x7f37c4115cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:43:07.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:07.890+0000 7f37c3400700 1 --2- 172.21.15.37:0/1100341412 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f37c41037a0 0x7f37c4116230 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f37b0002a50 tx=0x7f37b0004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:43:07.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:07.890+0000 7f37bbe00700 1 -- 172.21.15.37:0/1100341412 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f37b0007cc0 con 0x7f37c41037a0 2024-03-29T08:43:07.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:07.890+0000 7f37caa00700 1 -- 172.21.15.37:0/1100341412 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37c410ff30 con 0x7f37c41037a0 2024-03-29T08:43:07.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:07.890+0000 7f37bbe00700 1 -- 172.21.15.37:0/1100341412 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f37b000f040 con 0x7f37c41037a0 2024-03-29T08:43:07.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:07.890+0000 7f37bbe00700 1 -- 172.21.15.37:0/1100341412 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f37b000d9a0 con 0x7f37c41037a0 2024-03-29T08:43:07.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:07.890+0000 7f37caa00700 1 -- 172.21.15.37:0/1100341412 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37c41104f0 con 0x7f37c41037a0 2024-03-29T08:43:07.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:07.891+0000 7f37caa00700 1 -- 172.21.15.37:0/1100341412 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37c40fe550 con 0x7f37c41037a0 2024-03-29T08:43:07.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:07.893+0000 7f37bbe00700 1 -- 172.21.15.37:0/1100341412 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f37b000dc60 con 0x7f37c41037a0 2024-03-29T08:43:07.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:07.893+0000 7f37bbe00700 1 --2- 172.21.15.37:0/1100341412 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f37a806cc30 0x7f37a806f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:43:07.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:07.894+0000 7f37bbe00700 1 -- 172.21.15.37:0/1100341412 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f37b008be70 con 0x7f37c41037a0 2024-03-29T08:43:07.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:07.894+0000 7f37c3e00700 1 --2- 172.21.15.37:0/1100341412 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f37a806cc30 0x7f37a806f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:43:07.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:07.894+0000 7f37c3e00700 1 --2- 172.21.15.37:0/1100341412 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f37a806cc30 0x7f37a806f0e0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f37b4000c00 tx=0x7f37b4009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:43:07.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:07.898+0000 7f37bbe00700 1 -- 172.21.15.37:0/1100341412 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f37b00598c0 con 0x7f37c41037a0 2024-03-29T08:43:08.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:08.131+0000 7f37caa00700 1 -- 172.21.15.37:0/1100341412 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f37c4109e30 con 0x7f37a806cc30 2024-03-29T08:43:08.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:08.132+0000 7f37bbe00700 1 -- 172.21.15.37:0/1100341412 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f37c4109e30 con 0x7f37a806cc30 2024-03-29T08:43:08.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:08.135+0000 7f37caa00700 1 -- 172.21.15.37:0/1100341412 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f37a806cc30 msgr2=0x7f37a806f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:43:08.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:08.135+0000 7f37caa00700 1 --2- 172.21.15.37:0/1100341412 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f37a806cc30 0x7f37a806f0e0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f37b4000c00 tx=0x7f37b4009e30 comp rx=0 tx=0).stop 2024-03-29T08:43:08.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:08.135+0000 7f37caa00700 1 -- 172.21.15.37:0/1100341412 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f37c41037a0 msgr2=0x7f37c4116230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:43:08.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:08.135+0000 7f37caa00700 1 --2- 172.21.15.37:0/1100341412 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f37c41037a0 0x7f37c4116230 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f37b0002a50 tx=0x7f37b0004600 comp rx=0 tx=0).stop 2024-03-29T08:43:08.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:08.135+0000 7f37caa00700 1 -- 172.21.15.37:0/1100341412 shutdown_connections 2024-03-29T08:43:08.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:08.135+0000 7f37caa00700 1 --2- 172.21.15.37:0/1100341412 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f37a806cc30 0x7f37a806f0e0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:43:08.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:08.135+0000 7f37caa00700 1 --2- 172.21.15.37:0/1100341412 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f37c4102e00 0x7f37c4115cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:43:08.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:08.135+0000 7f37caa00700 1 --2- 172.21.15.37:0/1100341412 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f37c41037a0 0x7f37c4116230 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:43:08.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:08.135+0000 7f37caa00700 1 -- 172.21.15.37:0/1100341412 >> 172.21.15.37:0/1100341412 conn(0x7f37c40f89b0 msgr2=0x7f37c40706d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:43:08.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:08.135+0000 7f37caa00700 1 -- 172.21.15.37:0/1100341412 shutdown_connections 2024-03-29T08:43:08.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:08.135+0000 7f37caa00700 1 -- 172.21.15.37:0/1100341412 wait complete. 2024-03-29T08:43:08.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:08.273+0000 7fb048a00700 1 -- 172.21.15.37:0/1642898538 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb044102ca0 msgr2=0x7fb044103070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:43:08.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:08.273+0000 7fb048a00700 1 --2- 172.21.15.37:0/1642898538 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb044102ca0 0x7fb044103070 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fb034004940 tx=0x7fb034009e30 comp rx=0 tx=0).stop 2024-03-29T08:43:08.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:08.273+0000 7fb048a00700 1 -- 172.21.15.37:0/1642898538 shutdown_connections 2024-03-29T08:43:08.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:08.273+0000 7fb048a00700 1 --2- 172.21.15.37:0/1642898538 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb044065630 0x7fb044065a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:43:08.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:08.273+0000 7fb048a00700 1 --2- 172.21.15.37:0/1642898538 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb044102ca0 0x7fb044103070 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:43:08.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:08.273+0000 7fb048a00700 1 -- 172.21.15.37:0/1642898538 >> 172.21.15.37:0/1642898538 conn(0x7fb044071b70 msgr2=0x7fb044071f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:43:08.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:08.273+0000 7fb048a00700 1 -- 172.21.15.37:0/1642898538 shutdown_connections 2024-03-29T08:43:08.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:08.273+0000 7fb048a00700 1 -- 172.21.15.37:0/1642898538 wait complete. 2024-03-29T08:43:08.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:08.274+0000 7fb048a00700 1 Processor -- start 2024-03-29T08:43:08.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:08.274+0000 7fb048a00700 1 -- start start 2024-03-29T08:43:08.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:08.275+0000 7fb048a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb044065630 0x7fb044113890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:43:08.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:08.275+0000 7fb048a00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb044102ca0 0x7fb044113dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:43:08.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:08.275+0000 7fb048a00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0441144b0 con 0x7fb044065630 2024-03-29T08:43:08.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:08.275+0000 7fb048a00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb04410d810 con 0x7fb044102ca0 2024-03-29T08:43:08.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:08.275+0000 7fb042800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb044065630 0x7fb044113890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:43:08.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:08.275+0000 7fb042800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb044065630 0x7fb044113890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56624/0 (socket says 172.21.15.37:56624) 2024-03-29T08:43:08.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:08.275+0000 7fb042800700 1 -- 172.21.15.37:0/4020997944 learned_addr learned my addr 172.21.15.37:0/4020997944 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:43:08.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:08.275+0000 7fb041e00700 1 --2- 172.21.15.37:0/4020997944 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb044102ca0 0x7fb044113dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:43:08.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:08.275+0000 7fb042800700 1 -- 172.21.15.37:0/4020997944 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb044102ca0 msgr2=0x7fb044113dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:43:08.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:08.275+0000 7fb042800700 1 --2- 172.21.15.37:0/4020997944 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb044102ca0 0x7fb044113dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:43:08.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:08.275+0000 7fb042800700 1 -- 172.21.15.37:0/4020997944 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb034005040 con 0x7fb044065630 2024-03-29T08:43:08.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:08.275+0000 7fb041e00700 1 --2- 172.21.15.37:0/4020997944 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb044102ca0 0x7fb044113dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:43:08.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:08.276+0000 7fb042800700 1 --2- 172.21.15.37:0/4020997944 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb044065630 0x7fb044113890 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fb034009360 tx=0x7fb034009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:43:08.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:08.276+0000 7fb03aa00700 1 -- 172.21.15.37:0/4020997944 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb03400e040 con 0x7fb044065630 2024-03-29T08:43:08.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:08.276+0000 7fb03aa00700 1 -- 172.21.15.37:0/4020997944 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fb034013420 con 0x7fb044065630 2024-03-29T08:43:08.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:08.276+0000 7fb03aa00700 1 -- 172.21.15.37:0/4020997944 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb03401c5f0 con 0x7fb044065630 2024-03-29T08:43:08.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:08.276+0000 7fb048a00700 1 -- 172.21.15.37:0/4020997944 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb04410da90 con 0x7fb044065630 2024-03-29T08:43:08.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:08.276+0000 7fb048a00700 1 -- 172.21.15.37:0/4020997944 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb04410e050 con 0x7fb044065630 2024-03-29T08:43:08.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:08.277+0000 7fb048a00700 1 -- 172.21.15.37:0/4020997944 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0440fe380 con 0x7fb044065630 2024-03-29T08:43:08.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:08.278+0000 7fb03aa00700 1 -- 172.21.15.37:0/4020997944 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fb034013590 con 0x7fb044065630 2024-03-29T08:43:08.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:08.279+0000 7fb03aa00700 1 --2- 172.21.15.37:0/4020997944 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fb02806cc30 0x7fb02806f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:43:08.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:08.279+0000 7fb03aa00700 1 -- 172.21.15.37:0/4020997944 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fb03408d250 con 0x7fb044065630 2024-03-29T08:43:08.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:08.279+0000 7fb041e00700 1 --2- 172.21.15.37:0/4020997944 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fb02806cc30 0x7fb02806f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:43:08.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:08.280+0000 7fb041e00700 1 --2- 172.21.15.37:0/4020997944 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fb02806cc30 0x7fb02806f0e0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fb030006fd0 tx=0x7fb030008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:43:08.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:08.284+0000 7fb03aa00700 1 -- 172.21.15.37:0/4020997944 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fb03405add0 con 0x7fb044065630 2024-03-29T08:43:08.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:08.509+0000 7fb048a00700 1 -- 172.21.15.37:0/4020997944 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fb044109c90 con 0x7fb02806cc30 2024-03-29T08:43:08.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:08.519+0000 7fb03aa00700 1 -- 172.21.15.37:0/4020997944 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7fb044109c90 con 0x7fb02806cc30 2024-03-29T08:43:08.520 INFO:teuthology.orchestra.run.smithi037.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-29T08:43:08.520 INFO:teuthology.orchestra.run.smithi037.stdout:alertmanager.smithi037 smithi037 *:9093,9094 running (32m) 9m ago 45m 14.3M - 0.25.0 c8568f914cd2 2f9f9746ed0f 2024-03-29T08:43:08.520 INFO:teuthology.orchestra.run.smithi037.stdout:crash.smithi037 smithi037 running (29m) 9m ago 46m 6647k - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 178d416a3186 2024-03-29T08:43:08.520 INFO:teuthology.orchestra.run.smithi037.stdout:crash.smithi148 smithi148 running (29m) 9m ago 44m 6656k - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 23666989d58f 2024-03-29T08:43:08.520 INFO:teuthology.orchestra.run.smithi037.stdout:grafana.smithi037 smithi037 *:3000 running (32m) 9m ago 45m 74.5M - 9.4.7 954c08fa6188 6b976fe49952 2024-03-29T08:43:08.520 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi037.iifsmh smithi037 running (25m) 9m ago 39m 16.4M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 2d97f562cc5b 2024-03-29T08:43:08.520 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi037.mmwgxe smithi037 running (25m) 9m ago 39m 16.7M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff ec37037f46e8 2024-03-29T08:43:08.521 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi148.qukpjh smithi148 running (9m) 9m ago 39m 10.7M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 3257219bc046 2024-03-29T08:43:08.521 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi148.sqteih smithi148 running (25m) 9m ago 39m 16.3M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 44c15c77037a 2024-03-29T08:43:08.521 INFO:teuthology.orchestra.run.smithi037.stdout:mgr.smithi037.wjikii smithi037 *:8443,9283,8765 running (35m) 9m ago 47m 510M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff c5a1417b9417 2024-03-29T08:43:08.521 INFO:teuthology.orchestra.run.smithi037.stdout:mgr.smithi148.onhgvm smithi148 *:8443,9283,8765 running (34m) 9m ago 44m 456M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff fcfdbce062bd 2024-03-29T08:43:08.521 INFO:teuthology.orchestra.run.smithi037.stdout:mon.smithi037 smithi037 running (31m) 9m ago 47m 70.2M 2048M 18.2.2-1015-gd0e392d9 1cc2f45c17ff eea2060783af 2024-03-29T08:43:08.521 INFO:teuthology.orchestra.run.smithi037.stdout:mon.smithi148 smithi148 running (30m) 9m ago 44m 50.4M 2048M 18.2.2-1015-gd0e392d9 1cc2f45c17ff ad4b2c5748be 2024-03-29T08:43:08.521 INFO:teuthology.orchestra.run.smithi037.stdout:node-exporter.smithi037 smithi037 *:9100 running (33m) 9m ago 46m 8514k - 1.5.0 0da6a335fe13 26b8ae9cb878 2024-03-29T08:43:08.521 INFO:teuthology.orchestra.run.smithi037.stdout:node-exporter.smithi148 smithi148 *:9100 running (33m) 9m ago 44m 8464k - 1.5.0 0da6a335fe13 98414e4e64ed 2024-03-29T08:43:08.521 INFO:teuthology.orchestra.run.smithi037.stdout:osd.0 smithi037 running (29m) 9m ago 43m 108M 1619M 18.2.2-1015-gd0e392d9 1cc2f45c17ff c0c584f6245c 2024-03-29T08:43:08.521 INFO:teuthology.orchestra.run.smithi037.stdout:osd.1 smithi037 running (28m) 9m ago 43m 104M 1619M 18.2.2-1015-gd0e392d9 1cc2f45c17ff a4a2ff4cc83d 2024-03-29T08:43:08.521 INFO:teuthology.orchestra.run.smithi037.stdout:osd.2 smithi037 running (28m) 9m ago 42m 85.9M 1619M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 8b1bc71aa79e 2024-03-29T08:43:08.521 INFO:teuthology.orchestra.run.smithi037.stdout:osd.3 smithi148 running (27m) 9m ago 42m 109M 2642M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 1c656ab8ff86 2024-03-29T08:43:08.521 INFO:teuthology.orchestra.run.smithi037.stdout:osd.4 smithi148 running (26m) 9m ago 41m 101M 2642M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 966a9de11d22 2024-03-29T08:43:08.521 INFO:teuthology.orchestra.run.smithi037.stdout:osd.5 smithi148 running (26m) 9m ago 41m 89.4M 2642M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 474c4f9212ce 2024-03-29T08:43:08.521 INFO:teuthology.orchestra.run.smithi037.stdout:prometheus.smithi037 smithi037 *:9095 running (33m) 9m ago 45m 61.0M - 2.43.0 a07b618ecd1d 30c0223ec589 2024-03-29T08:43:08.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:08.521+0000 7fb048a00700 1 -- 172.21.15.37:0/4020997944 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fb02806cc30 msgr2=0x7fb02806f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:43:08.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:08.521+0000 7fb048a00700 1 --2- 172.21.15.37:0/4020997944 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fb02806cc30 0x7fb02806f0e0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fb030006fd0 tx=0x7fb030008040 comp rx=0 tx=0).stop 2024-03-29T08:43:08.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:08.521+0000 7fb048a00700 1 -- 172.21.15.37:0/4020997944 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb044065630 msgr2=0x7fb044113890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:43:08.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:08.521+0000 7fb048a00700 1 --2- 172.21.15.37:0/4020997944 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb044065630 0x7fb044113890 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fb034009360 tx=0x7fb034009e30 comp rx=0 tx=0).stop 2024-03-29T08:43:08.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:08.522+0000 7fb048a00700 1 -- 172.21.15.37:0/4020997944 shutdown_connections 2024-03-29T08:43:08.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:08.522+0000 7fb048a00700 1 --2- 172.21.15.37:0/4020997944 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fb02806cc30 0x7fb02806f0e0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:43:08.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:08.522+0000 7fb048a00700 1 --2- 172.21.15.37:0/4020997944 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb044065630 0x7fb044113890 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:43:08.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:08.522+0000 7fb048a00700 1 --2- 172.21.15.37:0/4020997944 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb044102ca0 0x7fb044113dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:43:08.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:08.522+0000 7fb048a00700 1 -- 172.21.15.37:0/4020997944 >> 172.21.15.37:0/4020997944 conn(0x7fb044071b70 msgr2=0x7fb0441059d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:43:08.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:08.522+0000 7fb048a00700 1 -- 172.21.15.37:0/4020997944 shutdown_connections 2024-03-29T08:43:08.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:08.522+0000 7fb048a00700 1 -- 172.21.15.37:0/4020997944 wait complete. 2024-03-29T08:43:08.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:08.657+0000 7fbb42600700 1 -- 172.21.15.37:0/1777649824 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbb3c102e00 msgr2=0x7fbb3c1031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:43:08.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:08.657+0000 7fbb42600700 1 --2- 172.21.15.37:0/1777649824 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbb3c102e00 0x7fbb3c1031d0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fbb2c004940 tx=0x7fbb2c009e30 comp rx=0 tx=0).stop 2024-03-29T08:43:08.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:08.658+0000 7fbb42600700 1 -- 172.21.15.37:0/1777649824 shutdown_connections 2024-03-29T08:43:08.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:08.658+0000 7fbb42600700 1 --2- 172.21.15.37:0/1777649824 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb3c1037a0 0x7fbb3c0fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:43:08.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:08.658+0000 7fbb42600700 1 --2- 172.21.15.37:0/1777649824 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbb3c102e00 0x7fbb3c1031d0 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:43:08.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:08.658+0000 7fbb42600700 1 -- 172.21.15.37:0/1777649824 >> 172.21.15.37:0/1777649824 conn(0x7fbb3c0f89b0 msgr2=0x7fbb3c0fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:43:08.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:08.658+0000 7fbb42600700 1 -- 172.21.15.37:0/1777649824 shutdown_connections 2024-03-29T08:43:08.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:08.658+0000 7fbb42600700 1 -- 172.21.15.37:0/1777649824 wait complete. 2024-03-29T08:43:08.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:08.659+0000 7fbb42600700 1 Processor -- start 2024-03-29T08:43:08.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:08.659+0000 7fbb42600700 1 -- start start 2024-03-29T08:43:08.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:08.659+0000 7fbb42600700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbb3c102e00 0x7fbb3c115cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:43:08.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:08.659+0000 7fbb42600700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb3c1037a0 0x7fbb3c116210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:43:08.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:08.659+0000 7fbb42600700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb3c116860 con 0x7fbb3c102e00 2024-03-29T08:43:08.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:08.659+0000 7fbb42600700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb3c10fc30 con 0x7fbb3c1037a0 2024-03-29T08:43:08.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:08.660+0000 7fbb3b400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbb3c102e00 0x7fbb3c115cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:43:08.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:08.660+0000 7fbb3b400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbb3c102e00 0x7fbb3c115cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56648/0 (socket says 172.21.15.37:56648) 2024-03-29T08:43:08.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:08.660+0000 7fbb3b400700 1 -- 172.21.15.37:0/1555265336 learned_addr learned my addr 172.21.15.37:0/1555265336 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:43:08.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:08.660+0000 7fbb3aa00700 1 --2- 172.21.15.37:0/1555265336 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb3c1037a0 0x7fbb3c116210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:43:08.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:08.660+0000 7fbb3b400700 1 -- 172.21.15.37:0/1555265336 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb3c1037a0 msgr2=0x7fbb3c116210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:43:08.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:08.660+0000 7fbb3b400700 1 --2- 172.21.15.37:0/1555265336 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb3c1037a0 0x7fbb3c116210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:43:08.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:08.660+0000 7fbb3b400700 1 -- 172.21.15.37:0/1555265336 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb2c005040 con 0x7fbb3c102e00 2024-03-29T08:43:08.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:08.660+0000 7fbb3b400700 1 --2- 172.21.15.37:0/1555265336 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbb3c102e00 0x7fbb3c115cd0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fbb2c009390 tx=0x7fbb2c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:43:08.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:08.660+0000 7fbb33400700 1 -- 172.21.15.37:0/1555265336 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbb2c00e040 con 0x7fbb3c102e00 2024-03-29T08:43:08.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:08.661+0000 7fbb33400700 1 -- 172.21.15.37:0/1555265336 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fbb2c013420 con 0x7fbb3c102e00 2024-03-29T08:43:08.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:08.661+0000 7fbb42600700 1 -- 172.21.15.37:0/1555265336 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb3c10feb0 con 0x7fbb3c102e00 2024-03-29T08:43:08.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:08.661+0000 7fbb33400700 1 -- 172.21.15.37:0/1555265336 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbb2c01c630 con 0x7fbb3c102e00 2024-03-29T08:43:08.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:08.661+0000 7fbb42600700 1 -- 172.21.15.37:0/1555265336 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb3c110470 con 0x7fbb3c102e00 2024-03-29T08:43:08.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:08.662+0000 7fbb42600700 1 -- 172.21.15.37:0/1555265336 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb3c0fe550 con 0x7fbb3c102e00 2024-03-29T08:43:08.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:08.663+0000 7fbb33400700 1 -- 172.21.15.37:0/1555265336 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fbb2c0139e0 con 0x7fbb3c102e00 2024-03-29T08:43:08.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:08.664+0000 7fbb33400700 1 --2- 172.21.15.37:0/1555265336 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fbb2006cc30 0x7fbb2006f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:43:08.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:08.664+0000 7fbb33400700 1 -- 172.21.15.37:0/1555265336 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fbb2c08d320 con 0x7fbb3c102e00 2024-03-29T08:43:08.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:08.664+0000 7fbb3aa00700 1 --2- 172.21.15.37:0/1555265336 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fbb2006cc30 0x7fbb2006f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:43:08.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:08.665+0000 7fbb3aa00700 1 --2- 172.21.15.37:0/1555265336 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fbb2006cc30 0x7fbb2006f0e0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fbb3c111570 tx=0x7fbb28009ee0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:43:08.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:08.668+0000 7fbb33400700 1 -- 172.21.15.37:0/1555265336 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fbb2c05ae20 con 0x7fbb3c102e00 2024-03-29T08:43:08.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:43:08 smithi037 ceph-mon[142325]: pgmap v968: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:43:08.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:43:08 smithi037 ceph-mon[142325]: from='client.35738 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:43:08.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:08.974+0000 7fbb42600700 1 -- 172.21.15.37:0/1555265336 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fbb3c04bed0 con 0x7fbb3c102e00 2024-03-29T08:43:08.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:08.975+0000 7fbb33400700 1 -- 172.21.15.37:0/1555265336 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7fbb2c05a9b0 con 0x7fbb3c102e00 2024-03-29T08:43:08.976 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-29T08:43:08.976 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": { 2024-03-29T08:43:08.976 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 2 2024-03-29T08:43:08.976 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:43:08.976 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": { 2024-03-29T08:43:08.976 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 2 2024-03-29T08:43:08.976 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:43:08.976 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": { 2024-03-29T08:43:08.976 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 6 2024-03-29T08:43:08.976 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:43:08.976 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": { 2024-03-29T08:43:08.977 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 4 2024-03-29T08:43:08.977 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:43:08.977 INFO:teuthology.orchestra.run.smithi037.stdout: "overall": { 2024-03-29T08:43:08.977 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 14 2024-03-29T08:43:08.977 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-29T08:43:08.977 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-29T08:43:08.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:08.977+0000 7fbb42600700 1 -- 172.21.15.37:0/1555265336 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fbb2006cc30 msgr2=0x7fbb2006f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:43:08.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:08.977+0000 7fbb42600700 1 --2- 172.21.15.37:0/1555265336 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fbb2006cc30 0x7fbb2006f0e0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fbb3c111570 tx=0x7fbb28009ee0 comp rx=0 tx=0).stop 2024-03-29T08:43:08.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:08.977+0000 7fbb42600700 1 -- 172.21.15.37:0/1555265336 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbb3c102e00 msgr2=0x7fbb3c115cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:43:08.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:08.977+0000 7fbb42600700 1 --2- 172.21.15.37:0/1555265336 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbb3c102e00 0x7fbb3c115cd0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fbb2c009390 tx=0x7fbb2c009e30 comp rx=0 tx=0).stop 2024-03-29T08:43:08.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:08.977+0000 7fbb42600700 1 -- 172.21.15.37:0/1555265336 shutdown_connections 2024-03-29T08:43:08.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:08.977+0000 7fbb42600700 1 --2- 172.21.15.37:0/1555265336 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fbb2006cc30 0x7fbb2006f0e0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:43:08.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:08.977+0000 7fbb42600700 1 --2- 172.21.15.37:0/1555265336 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbb3c102e00 0x7fbb3c115cd0 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:43:08.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:08.977+0000 7fbb42600700 1 --2- 172.21.15.37:0/1555265336 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb3c1037a0 0x7fbb3c116210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:43:08.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:08.977+0000 7fbb42600700 1 -- 172.21.15.37:0/1555265336 >> 172.21.15.37:0/1555265336 conn(0x7fbb3c0f89b0 msgr2=0x7fbb3c0f99e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:43:08.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:08.977+0000 7fbb42600700 1 -- 172.21.15.37:0/1555265336 shutdown_connections 2024-03-29T08:43:08.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:08.977+0000 7fbb42600700 1 -- 172.21.15.37:0/1555265336 wait complete. 2024-03-29T08:43:09.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:43:08 smithi148 ceph-mon[106692]: pgmap v968: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:43:09.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:43:08 smithi148 ceph-mon[106692]: from='client.35738 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:43:09.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:09.113+0000 7fcb49200700 1 -- 172.21.15.37:0/2460020879 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcb44102e00 msgr2=0x7fcb441031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:43:09.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:09.113+0000 7fcb49200700 1 --2- 172.21.15.37:0/2460020879 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcb44102e00 0x7fcb441031d0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fcb34004940 tx=0x7fcb34009e30 comp rx=0 tx=0).stop 2024-03-29T08:43:09.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:09.114+0000 7fcb49200700 1 -- 172.21.15.37:0/2460020879 shutdown_connections 2024-03-29T08:43:09.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:09.114+0000 7fcb49200700 1 --2- 172.21.15.37:0/2460020879 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcb441037a0 0x7fcb440fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:43:09.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:09.114+0000 7fcb49200700 1 --2- 172.21.15.37:0/2460020879 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcb44102e00 0x7fcb441031d0 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:43:09.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:09.114+0000 7fcb49200700 1 -- 172.21.15.37:0/2460020879 >> 172.21.15.37:0/2460020879 conn(0x7fcb440f89b0 msgr2=0x7fcb440fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:43:09.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:09.114+0000 7fcb49200700 1 -- 172.21.15.37:0/2460020879 shutdown_connections 2024-03-29T08:43:09.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:09.114+0000 7fcb49200700 1 -- 172.21.15.37:0/2460020879 wait complete. 2024-03-29T08:43:09.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:09.114+0000 7fcb49200700 1 Processor -- start 2024-03-29T08:43:09.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:09.115+0000 7fcb49200700 1 -- start start 2024-03-29T08:43:09.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:09.115+0000 7fcb49200700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcb44102e00 0x7fcb44113aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:43:09.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:09.115+0000 7fcb49200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcb441037a0 0x7fcb44113fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:43:09.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:09.115+0000 7fcb49200700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb441146c0 con 0x7fcb441037a0 2024-03-29T08:43:09.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:09.115+0000 7fcb49200700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb4410da20 con 0x7fcb44102e00 2024-03-29T08:43:09.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:09.115+0000 7fcb41400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcb441037a0 0x7fcb44113fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:43:09.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:09.115+0000 7fcb41e00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcb44102e00 0x7fcb44113aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:43:09.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:09.115+0000 7fcb41400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcb441037a0 0x7fcb44113fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56668/0 (socket says 172.21.15.37:56668) 2024-03-29T08:43:09.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:09.115+0000 7fcb41400700 1 -- 172.21.15.37:0/1275748300 learned_addr learned my addr 172.21.15.37:0/1275748300 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:43:09.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:09.116+0000 7fcb41400700 1 -- 172.21.15.37:0/1275748300 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcb44102e00 msgr2=0x7fcb44113aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:43:09.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:09.116+0000 7fcb41400700 1 --2- 172.21.15.37:0/1275748300 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcb44102e00 0x7fcb44113aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:43:09.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:09.116+0000 7fcb41400700 1 -- 172.21.15.37:0/1275748300 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb34005040 con 0x7fcb441037a0 2024-03-29T08:43:09.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:09.116+0000 7fcb41400700 1 --2- 172.21.15.37:0/1275748300 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcb441037a0 0x7fcb44113fe0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fcb3000b790 tx=0x7fcb30009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:43:09.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:09.116+0000 7fcb3a000700 1 -- 172.21.15.37:0/1275748300 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcb3000f6f0 con 0x7fcb441037a0 2024-03-29T08:43:09.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:09.116+0000 7fcb3a000700 1 -- 172.21.15.37:0/1275748300 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fcb3000fd00 con 0x7fcb441037a0 2024-03-29T08:43:09.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:09.116+0000 7fcb49200700 1 -- 172.21.15.37:0/1275748300 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb4410dd00 con 0x7fcb441037a0 2024-03-29T08:43:09.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:09.116+0000 7fcb3a000700 1 -- 172.21.15.37:0/1275748300 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcb300103a0 con 0x7fcb441037a0 2024-03-29T08:43:09.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:09.117+0000 7fcb49200700 1 -- 172.21.15.37:0/1275748300 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb4410e2c0 con 0x7fcb441037a0 2024-03-29T08:43:09.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:09.117+0000 7fcb49200700 1 -- 172.21.15.37:0/1275748300 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb440fe550 con 0x7fcb441037a0 2024-03-29T08:43:09.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:09.119+0000 7fcb3a000700 1 -- 172.21.15.37:0/1275748300 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fcb3000f850 con 0x7fcb441037a0 2024-03-29T08:43:09.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:09.119+0000 7fcb3a000700 1 --2- 172.21.15.37:0/1275748300 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fcb2806cc30 0x7fcb2806f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:43:09.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:09.120+0000 7fcb3a000700 1 -- 172.21.15.37:0/1275748300 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fcb3008bec0 con 0x7fcb441037a0 2024-03-29T08:43:09.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:09.120+0000 7fcb41e00700 1 --2- 172.21.15.37:0/1275748300 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fcb2806cc30 0x7fcb2806f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:43:09.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:09.120+0000 7fcb41e00700 1 --2- 172.21.15.37:0/1275748300 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fcb2806cc30 0x7fcb2806f0e0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fcb34009fd0 tx=0x7fcb34009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:43:09.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:09.124+0000 7fcb3a000700 1 -- 172.21.15.37:0/1275748300 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fcb30059910 con 0x7fcb441037a0 2024-03-29T08:43:09.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:09.380+0000 7fcb49200700 1 -- 172.21.15.37:0/1275748300 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fcb44109e60 con 0x7fcb441037a0 2024-03-29T08:43:09.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:09.381+0000 7fcb3a000700 1 -- 172.21.15.37:0/1275748300 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 30 v30) v1 ==== 76+0+1841 (secure 0 0 0) 0x7fcb300594a0 con 0x7fcb441037a0 2024-03-29T08:43:09.382 INFO:teuthology.orchestra.run.smithi037.stdout:e30 2024-03-29T08:43:09.382 INFO:teuthology.orchestra.run.smithi037.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-29T08:43:09.382 INFO:teuthology.orchestra.run.smithi037.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T08:43:09.382 INFO:teuthology.orchestra.run.smithi037.stdout:legacy client fscid: 1 2024-03-29T08:43:09.382 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:43:09.382 INFO:teuthology.orchestra.run.smithi037.stdout:Filesystem 'cephfs' (1) 2024-03-29T08:43:09.382 INFO:teuthology.orchestra.run.smithi037.stdout:fs_name cephfs 2024-03-29T08:43:09.382 INFO:teuthology.orchestra.run.smithi037.stdout:epoch 21 2024-03-29T08:43:09.382 INFO:teuthology.orchestra.run.smithi037.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-03-29T08:43:09.382 INFO:teuthology.orchestra.run.smithi037.stdout:created 2024-03-29T08:03:29.241269+0000 2024-03-29T08:43:09.382 INFO:teuthology.orchestra.run.smithi037.stdout:modified 2024-03-29T08:17:16.237238+0000 2024-03-29T08:43:09.383 INFO:teuthology.orchestra.run.smithi037.stdout:tableserver 0 2024-03-29T08:43:09.383 INFO:teuthology.orchestra.run.smithi037.stdout:root 0 2024-03-29T08:43:09.383 INFO:teuthology.orchestra.run.smithi037.stdout:session_timeout 60 2024-03-29T08:43:09.383 INFO:teuthology.orchestra.run.smithi037.stdout:session_autoclose 300 2024-03-29T08:43:09.383 INFO:teuthology.orchestra.run.smithi037.stdout:max_file_size 1099511627776 2024-03-29T08:43:09.383 INFO:teuthology.orchestra.run.smithi037.stdout:max_xattr_size 65536 2024-03-29T08:43:09.383 INFO:teuthology.orchestra.run.smithi037.stdout:required_client_features {} 2024-03-29T08:43:09.383 INFO:teuthology.orchestra.run.smithi037.stdout:last_failure 0 2024-03-29T08:43:09.383 INFO:teuthology.orchestra.run.smithi037.stdout:last_failure_osd_epoch 82 2024-03-29T08:43:09.383 INFO:teuthology.orchestra.run.smithi037.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T08:43:09.383 INFO:teuthology.orchestra.run.smithi037.stdout:max_mds 2 2024-03-29T08:43:09.383 INFO:teuthology.orchestra.run.smithi037.stdout:in 0,1 2024-03-29T08:43:09.383 INFO:teuthology.orchestra.run.smithi037.stdout:up {} 2024-03-29T08:43:09.383 INFO:teuthology.orchestra.run.smithi037.stdout:failed 0,1 2024-03-29T08:43:09.384 INFO:teuthology.orchestra.run.smithi037.stdout:damaged 2024-03-29T08:43:09.384 INFO:teuthology.orchestra.run.smithi037.stdout:stopped 2024-03-29T08:43:09.384 INFO:teuthology.orchestra.run.smithi037.stdout:data_pools [3] 2024-03-29T08:43:09.384 INFO:teuthology.orchestra.run.smithi037.stdout:metadata_pool 2 2024-03-29T08:43:09.384 INFO:teuthology.orchestra.run.smithi037.stdout:inline_data disabled 2024-03-29T08:43:09.384 INFO:teuthology.orchestra.run.smithi037.stdout:balancer 2024-03-29T08:43:09.384 INFO:teuthology.orchestra.run.smithi037.stdout:bal_rank_mask -1 2024-03-29T08:43:09.384 INFO:teuthology.orchestra.run.smithi037.stdout:standby_count_wanted 1 2024-03-29T08:43:09.384 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:43:09.384 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:43:09.384 INFO:teuthology.orchestra.run.smithi037.stdout:Standby daemons: 2024-03-29T08:43:09.384 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:43:09.384 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi037.mmwgxe{-1:34440} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.37:6828/1248142005,v1:172.21.15.37:6829/1248142005] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:43:09.384 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi037.iifsmh{-1:34444} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.37:6826/2060882959,v1:172.21.15.37:6827/2060882959] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:43:09.384 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi148.sqteih{-1:44395} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.148:6824/4078652242,v1:172.21.15.148:6825/4078652242] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:43:09.385 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi148.qukpjh{-1:45007} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.148:6826/2394751105,v1:172.21.15.148:6827/2394751105] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:43:09.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:09.383+0000 7fcb49200700 1 -- 172.21.15.37:0/1275748300 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fcb2806cc30 msgr2=0x7fcb2806f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:43:09.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:09.383+0000 7fcb49200700 1 --2- 172.21.15.37:0/1275748300 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fcb2806cc30 0x7fcb2806f0e0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fcb34009fd0 tx=0x7fcb34009e30 comp rx=0 tx=0).stop 2024-03-29T08:43:09.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:09.383+0000 7fcb49200700 1 -- 172.21.15.37:0/1275748300 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcb441037a0 msgr2=0x7fcb44113fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:43:09.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:09.383+0000 7fcb49200700 1 --2- 172.21.15.37:0/1275748300 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcb441037a0 0x7fcb44113fe0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fcb3000b790 tx=0x7fcb30009220 comp rx=0 tx=0).stop 2024-03-29T08:43:09.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:09.383+0000 7fcb49200700 1 -- 172.21.15.37:0/1275748300 shutdown_connections 2024-03-29T08:43:09.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:09.383+0000 7fcb49200700 1 --2- 172.21.15.37:0/1275748300 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fcb2806cc30 0x7fcb2806f0e0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:43:09.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:09.383+0000 7fcb49200700 1 --2- 172.21.15.37:0/1275748300 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcb44102e00 0x7fcb44113aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:43:09.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:09.383+0000 7fcb49200700 1 --2- 172.21.15.37:0/1275748300 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcb441037a0 0x7fcb44113fe0 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:43:09.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:09.383+0000 7fcb49200700 1 -- 172.21.15.37:0/1275748300 >> 172.21.15.37:0/1275748300 conn(0x7fcb440f89b0 msgr2=0x7fcb440f9a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:43:09.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:09.383+0000 7fcb49200700 1 -- 172.21.15.37:0/1275748300 shutdown_connections 2024-03-29T08:43:09.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:09.384+0000 7fcb49200700 1 -- 172.21.15.37:0/1275748300 wait complete. 2024-03-29T08:43:09.386 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 30 2024-03-29T08:43:09.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:09.521+0000 7f850fe00700 1 -- 172.21.15.37:0/4109540951 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8508102e00 msgr2=0x7f85081031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:43:09.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:09.521+0000 7f850fe00700 1 --2- 172.21.15.37:0/4109540951 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8508102e00 0x7f85081031d0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f84fc004940 tx=0x7f84fc009e30 comp rx=0 tx=0).stop 2024-03-29T08:43:09.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:09.521+0000 7f850fe00700 1 -- 172.21.15.37:0/4109540951 shutdown_connections 2024-03-29T08:43:09.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:09.521+0000 7f850fe00700 1 --2- 172.21.15.37:0/4109540951 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f85081037a0 0x7f85080fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:43:09.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:09.521+0000 7f850fe00700 1 --2- 172.21.15.37:0/4109540951 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8508102e00 0x7f85081031d0 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:43:09.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:09.521+0000 7f850fe00700 1 -- 172.21.15.37:0/4109540951 >> 172.21.15.37:0/4109540951 conn(0x7f85080f89b0 msgr2=0x7f85080fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:43:09.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:09.521+0000 7f850fe00700 1 -- 172.21.15.37:0/4109540951 shutdown_connections 2024-03-29T08:43:09.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:09.521+0000 7f850fe00700 1 -- 172.21.15.37:0/4109540951 wait complete. 2024-03-29T08:43:09.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:09.522+0000 7f850fe00700 1 Processor -- start 2024-03-29T08:43:09.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:09.522+0000 7f850fe00700 1 -- start start 2024-03-29T08:43:09.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:09.522+0000 7f850fe00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8508102e00 0x7f8508115cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:43:09.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:09.523+0000 7f850fe00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f85081037a0 0x7f8508116210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:43:09.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:09.523+0000 7f850fe00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8508116860 con 0x7f8508102e00 2024-03-29T08:43:09.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:09.523+0000 7f850fe00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f850810fc30 con 0x7f85081037a0 2024-03-29T08:43:09.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:09.523+0000 7f850d400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8508102e00 0x7f8508115cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:43:09.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:09.523+0000 7f850d400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8508102e00 0x7f8508115cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56686/0 (socket says 172.21.15.37:56686) 2024-03-29T08:43:09.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:09.523+0000 7f850d400700 1 -- 172.21.15.37:0/139159098 learned_addr learned my addr 172.21.15.37:0/139159098 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:43:09.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:09.523+0000 7f850ca00700 1 --2- 172.21.15.37:0/139159098 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f85081037a0 0x7f8508116210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:43:09.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:09.523+0000 7f850d400700 1 -- 172.21.15.37:0/139159098 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f85081037a0 msgr2=0x7f8508116210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:43:09.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:09.523+0000 7f850d400700 1 --2- 172.21.15.37:0/139159098 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f85081037a0 0x7f8508116210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:43:09.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:09.523+0000 7f850d400700 1 -- 172.21.15.37:0/139159098 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84fc005040 con 0x7f8508102e00 2024-03-29T08:43:09.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:09.523+0000 7f850ca00700 1 --2- 172.21.15.37:0/139159098 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f85081037a0 0x7f8508116210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:43:09.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:09.523+0000 7f850d400700 1 --2- 172.21.15.37:0/139159098 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8508102e00 0x7f8508115cd0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f84fc009390 tx=0x7f84fc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:43:09.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:09.524+0000 7f8505600700 1 -- 172.21.15.37:0/139159098 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f84fc00e040 con 0x7f8508102e00 2024-03-29T08:43:09.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:09.524+0000 7f850fe00700 1 -- 172.21.15.37:0/139159098 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f850810feb0 con 0x7f8508102e00 2024-03-29T08:43:09.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:09.524+0000 7f8505600700 1 -- 172.21.15.37:0/139159098 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f84fc013420 con 0x7f8508102e00 2024-03-29T08:43:09.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:09.524+0000 7f850fe00700 1 -- 172.21.15.37:0/139159098 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8508110470 con 0x7f8508102e00 2024-03-29T08:43:09.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:09.524+0000 7f8505600700 1 -- 172.21.15.37:0/139159098 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f84fc0255e0 con 0x7f8508102e00 2024-03-29T08:43:09.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:09.525+0000 7f850fe00700 1 -- 172.21.15.37:0/139159098 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85080fe550 con 0x7f8508102e00 2024-03-29T08:43:09.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:09.526+0000 7f8505600700 1 -- 172.21.15.37:0/139159098 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f84fc0139e0 con 0x7f8508102e00 2024-03-29T08:43:09.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:09.527+0000 7f8505600700 1 --2- 172.21.15.37:0/139159098 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f84f006cc30 0x7f84f006f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:43:09.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:09.527+0000 7f8505600700 1 -- 172.21.15.37:0/139159098 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f84fc08d300 con 0x7f8508102e00 2024-03-29T08:43:09.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:09.527+0000 7f850ca00700 1 --2- 172.21.15.37:0/139159098 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f84f006cc30 0x7f84f006f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:43:09.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:09.528+0000 7f850ca00700 1 --2- 172.21.15.37:0/139159098 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f84f006cc30 0x7f84f006f0e0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f84f4009fd0 tx=0x7f84f4009ee0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:43:09.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:09.532+0000 7f8505600700 1 -- 172.21.15.37:0/139159098 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f84fc05ad50 con 0x7f8508102e00 2024-03-29T08:43:09.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:09.771+0000 7f850fe00700 1 -- 172.21.15.37:0/139159098 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f850804bf20 con 0x7f84f006cc30 2024-03-29T08:43:09.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:09.772+0000 7f8505600700 1 -- 172.21.15.37:0/139159098 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f850804bf20 con 0x7f84f006cc30 2024-03-29T08:43:09.773 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-29T08:43:09.773 INFO:teuthology.orchestra.run.smithi037.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59", 2024-03-29T08:43:09.773 INFO:teuthology.orchestra.run.smithi037.stdout: "in_progress": true, 2024-03-29T08:43:09.773 INFO:teuthology.orchestra.run.smithi037.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-29T08:43:09.773 INFO:teuthology.orchestra.run.smithi037.stdout: "services_complete": [ 2024-03-29T08:43:09.773 INFO:teuthology.orchestra.run.smithi037.stdout: "crash", 2024-03-29T08:43:09.774 INFO:teuthology.orchestra.run.smithi037.stdout: "osd", 2024-03-29T08:43:09.774 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr", 2024-03-29T08:43:09.774 INFO:teuthology.orchestra.run.smithi037.stdout: "mds", 2024-03-29T08:43:09.774 INFO:teuthology.orchestra.run.smithi037.stdout: "mon" 2024-03-29T08:43:09.774 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-29T08:43:09.774 INFO:teuthology.orchestra.run.smithi037.stdout: "progress": "16/21 daemons upgraded", 2024-03-29T08:43:09.774 INFO:teuthology.orchestra.run.smithi037.stdout: "message": "Currently upgrading mds daemons", 2024-03-29T08:43:09.774 INFO:teuthology.orchestra.run.smithi037.stdout: "is_paused": false 2024-03-29T08:43:09.774 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-29T08:43:09.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:09.775+0000 7f850fe00700 1 -- 172.21.15.37:0/139159098 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f84f006cc30 msgr2=0x7f84f006f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:43:09.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:09.775+0000 7f850fe00700 1 --2- 172.21.15.37:0/139159098 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f84f006cc30 0x7f84f006f0e0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f84f4009fd0 tx=0x7f84f4009ee0 comp rx=0 tx=0).stop 2024-03-29T08:43:09.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:09.775+0000 7f850fe00700 1 -- 172.21.15.37:0/139159098 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8508102e00 msgr2=0x7f8508115cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:43:09.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:09.775+0000 7f850fe00700 1 --2- 172.21.15.37:0/139159098 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8508102e00 0x7f8508115cd0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f84fc009390 tx=0x7f84fc009e30 comp rx=0 tx=0).stop 2024-03-29T08:43:09.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:09.775+0000 7f850fe00700 1 -- 172.21.15.37:0/139159098 shutdown_connections 2024-03-29T08:43:09.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:09.775+0000 7f850fe00700 1 --2- 172.21.15.37:0/139159098 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f84f006cc30 0x7f84f006f0e0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:43:09.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:09.775+0000 7f850fe00700 1 --2- 172.21.15.37:0/139159098 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8508102e00 0x7f8508115cd0 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:43:09.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:09.775+0000 7f850fe00700 1 --2- 172.21.15.37:0/139159098 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f85081037a0 0x7f8508116210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:43:09.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:09.775+0000 7f850fe00700 1 -- 172.21.15.37:0/139159098 >> 172.21.15.37:0/139159098 conn(0x7f85080f89b0 msgr2=0x7f85080f99e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:43:09.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:09.775+0000 7f850fe00700 1 -- 172.21.15.37:0/139159098 shutdown_connections 2024-03-29T08:43:09.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:09.775+0000 7f850fe00700 1 -- 172.21.15.37:0/139159098 wait complete. 2024-03-29T08:43:09.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:09.913+0000 7fe43d800700 1 -- 172.21.15.37:0/3416412629 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe4381037c0 msgr2=0x7fe4380fce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:43:09.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:09.913+0000 7fe43d800700 1 --2- 172.21.15.37:0/3416412629 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe4381037c0 0x7fe4380fce30 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fe424004940 tx=0x7fe424009e30 comp rx=0 tx=0).stop 2024-03-29T08:43:09.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:09.913+0000 7fe43d800700 1 -- 172.21.15.37:0/3416412629 shutdown_connections 2024-03-29T08:43:09.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:09.913+0000 7fe43d800700 1 --2- 172.21.15.37:0/3416412629 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe4381037c0 0x7fe4380fce30 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:43:09.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:09.913+0000 7fe43d800700 1 --2- 172.21.15.37:0/3416412629 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe438102e20 0x7fe4381031f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:43:09.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:09.913+0000 7fe43d800700 1 -- 172.21.15.37:0/3416412629 >> 172.21.15.37:0/3416412629 conn(0x7fe4380f89d0 msgr2=0x7fe4380fade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:43:09.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:09.913+0000 7fe43d800700 1 -- 172.21.15.37:0/3416412629 shutdown_connections 2024-03-29T08:43:09.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:09.913+0000 7fe43d800700 1 -- 172.21.15.37:0/3416412629 wait complete. 2024-03-29T08:43:09.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:09.914+0000 7fe43d800700 1 Processor -- start 2024-03-29T08:43:09.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:09.914+0000 7fe43d800700 1 -- start start 2024-03-29T08:43:09.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:09.915+0000 7fe43d800700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe438102e20 0x7fe438115cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:43:09.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:09.915+0000 7fe43d800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe4381037c0 0x7fe438116210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:43:09.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:09.915+0000 7fe43d800700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe438116860 con 0x7fe4381037c0 2024-03-29T08:43:09.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:09.915+0000 7fe43d800700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe43810fc30 con 0x7fe438102e20 2024-03-29T08:43:09.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:09.915+0000 7fe436000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe4381037c0 0x7fe438116210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:43:09.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:09.915+0000 7fe436000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe4381037c0 0x7fe438116210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56712/0 (socket says 172.21.15.37:56712) 2024-03-29T08:43:09.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:09.915+0000 7fe436000700 1 -- 172.21.15.37:0/1520957470 learned_addr learned my addr 172.21.15.37:0/1520957470 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:43:09.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:09.915+0000 7fe436a00700 1 --2- 172.21.15.37:0/1520957470 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe438102e20 0x7fe438115cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:43:09.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:09.915+0000 7fe436000700 1 -- 172.21.15.37:0/1520957470 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe438102e20 msgr2=0x7fe438115cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:43:09.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:09.915+0000 7fe436000700 1 --2- 172.21.15.37:0/1520957470 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe438102e20 0x7fe438115cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:43:09.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:09.916+0000 7fe436000700 1 -- 172.21.15.37:0/1520957470 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe424005040 con 0x7fe4381037c0 2024-03-29T08:43:09.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:09.916+0000 7fe436000700 1 --2- 172.21.15.37:0/1520957470 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe4381037c0 0x7fe438116210 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7fe424009390 tx=0x7fe424009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:43:09.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:09.916+0000 7fe42ea00700 1 -- 172.21.15.37:0/1520957470 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe42400e040 con 0x7fe4381037c0 2024-03-29T08:43:09.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:09.916+0000 7fe42ea00700 1 -- 172.21.15.37:0/1520957470 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fe424013420 con 0x7fe4381037c0 2024-03-29T08:43:09.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:09.916+0000 7fe42ea00700 1 -- 172.21.15.37:0/1520957470 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe42401c630 con 0x7fe4381037c0 2024-03-29T08:43:09.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:09.916+0000 7fe43d800700 1 -- 172.21.15.37:0/1520957470 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe43810feb0 con 0x7fe4381037c0 2024-03-29T08:43:09.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:09.916+0000 7fe43d800700 1 -- 172.21.15.37:0/1520957470 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe438110470 con 0x7fe4381037c0 2024-03-29T08:43:09.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:09.917+0000 7fe43d800700 1 -- 172.21.15.37:0/1520957470 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4380fe570 con 0x7fe4381037c0 2024-03-29T08:43:09.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:09.919+0000 7fe42ea00700 1 -- 172.21.15.37:0/1520957470 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fe424013a00 con 0x7fe4381037c0 2024-03-29T08:43:09.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:09.919+0000 7fe42ea00700 1 --2- 172.21.15.37:0/1520957470 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fe41c06cc30 0x7fe41c06f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:43:09.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:09.920+0000 7fe42ea00700 1 -- 172.21.15.37:0/1520957470 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fe42408dc60 con 0x7fe4381037c0 2024-03-29T08:43:09.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:09.920+0000 7fe436a00700 1 --2- 172.21.15.37:0/1520957470 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fe41c06cc30 0x7fe41c06f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:43:09.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:09.920+0000 7fe436a00700 1 --2- 172.21.15.37:0/1520957470 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fe41c06cc30 0x7fe41c06f0e0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fe428004f20 tx=0x7fe42800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:43:09.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:09.924+0000 7fe42ea00700 1 -- 172.21.15.37:0/1520957470 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fe42405b730 con 0x7fe4381037c0 2024-03-29T08:43:09.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:43:09 smithi037 ceph-mon[142325]: from='client.35742 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:43:09.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:43:09 smithi037 ceph-mon[142325]: from='client.35746 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:43:09.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:43:09 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/1555265336' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:43:09.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:43:09 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/1275748300' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T08:43:09.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:43:09 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:43:10.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:43:09 smithi148 ceph-mon[106692]: from='client.35742 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:43:10.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:43:09 smithi148 ceph-mon[106692]: from='client.35746 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:43:10.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:43:09 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/1555265336' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:43:10.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:43:09 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/1275748300' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T08:43:10.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:43:09 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:43:10.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:10.223+0000 7fe43d800700 1 -- 172.21.15.37:0/1520957470 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fe43805f5f0 con 0x7fe4381037c0 2024-03-29T08:43:10.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:10.223+0000 7fe42ea00700 1 -- 172.21.15.37:0/1520957470 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+3583 (secure 0 0 0) 0x7fe42405b2c0 con 0x7fe4381037c0 2024-03-29T08:43:10.224 INFO:teuthology.orchestra.run.smithi037.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem is offline 2024-03-29T08:43:10.224 INFO:teuthology.orchestra.run.smithi037.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-03-29T08:43:10.224 INFO:teuthology.orchestra.run.smithi037.stdout: host smithi148 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/mon.smithi148/config 2024-03-29T08:43:10.224 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/podman: timeout after 895 seconds 2024-03-29T08:43:10.224 INFO:teuthology.orchestra.run.smithi037.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59 -e NODE_NAME=smithi148 -e CEPH_USE_RANDOM_NONCE=1 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4:/var/run/ceph:z -v /var/log/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4:/var/log/ceph:z -v /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpkx2rz2d4:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59 inventory --format=json-pretty --filter-for-batch 2024-03-29T08:43:10.224 INFO:teuthology.orchestra.run.smithi037.stdout:Traceback (most recent call last): 2024-03-29T08:43:10.224 INFO:teuthology.orchestra.run.smithi037.stdout: File "/usr/lib64/python3.6/runpy.py", line 193, in _run_module_as_main 2024-03-29T08:43:10.225 INFO:teuthology.orchestra.run.smithi037.stdout: "__main__", mod_spec) 2024-03-29T08:43:10.225 INFO:teuthology.orchestra.run.smithi037.stdout: File "/usr/lib64/python3.6/runpy.py", line 85, in _run_code 2024-03-29T08:43:10.225 INFO:teuthology.orchestra.run.smithi037.stdout: exec(code, run_globals) 2024-03-29T08:43:10.225 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 10889, in 2024-03-29T08:43:10.225 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 10877, in main 2024-03-29T08:43:10.225 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 2576, in _infer_config 2024-03-29T08:43:10.225 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 2492, in _infer_fsid 2024-03-29T08:43:10.225 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 2604, in _infer_image 2024-03-29T08:43:10.225 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 2479, in _validate_fsid 2024-03-29T08:43:10.225 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 7145, in command_ceph_volume 2024-03-29T08:43:10.225 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 2267, in call_throws 2024-03-29T08:43:10.225 INFO:teuthology.orchestra.run.smithi037.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59 -e NODE_NAME=smithi148 -e CEPH_USE_RANDOM_NONCE=1 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4:/var/run/ceph:z -v /var/log/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4:/var/log/ceph:z -v /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpkx2rz2d4:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59 inventory --format=json-pretty --filter-for-batch 2024-03-29T08:43:10.225 INFO:teuthology.orchestra.run.smithi037.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-03-29T08:43:10.225 INFO:teuthology.orchestra.run.smithi037.stdout: fs cephfs is degraded 2024-03-29T08:43:10.225 INFO:teuthology.orchestra.run.smithi037.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-03-29T08:43:10.225 INFO:teuthology.orchestra.run.smithi037.stdout: fs cephfs is offline because no MDS is active for it. 2024-03-29T08:43:10.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:10.225+0000 7fe43d800700 1 -- 172.21.15.37:0/1520957470 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fe41c06cc30 msgr2=0x7fe41c06f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:43:10.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:10.225+0000 7fe43d800700 1 --2- 172.21.15.37:0/1520957470 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fe41c06cc30 0x7fe41c06f0e0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fe428004f20 tx=0x7fe42800b040 comp rx=0 tx=0).stop 2024-03-29T08:43:10.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:10.225+0000 7fe43d800700 1 -- 172.21.15.37:0/1520957470 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe4381037c0 msgr2=0x7fe438116210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:43:10.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:10.225+0000 7fe43d800700 1 --2- 172.21.15.37:0/1520957470 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe4381037c0 0x7fe438116210 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7fe424009390 tx=0x7fe424009e30 comp rx=0 tx=0).stop 2024-03-29T08:43:10.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:10.226+0000 7fe43d800700 1 -- 172.21.15.37:0/1520957470 shutdown_connections 2024-03-29T08:43:10.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:10.226+0000 7fe43d800700 1 --2- 172.21.15.37:0/1520957470 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fe41c06cc30 0x7fe41c06f0e0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:43:10.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:10.226+0000 7fe43d800700 1 --2- 172.21.15.37:0/1520957470 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe438102e20 0x7fe438115cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:43:10.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:10.226+0000 7fe43d800700 1 --2- 172.21.15.37:0/1520957470 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe4381037c0 0x7fe438116210 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:43:10.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:10.226+0000 7fe43d800700 1 -- 172.21.15.37:0/1520957470 >> 172.21.15.37:0/1520957470 conn(0x7fe4380f89d0 msgr2=0x7fe4380706a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:43:10.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:10.226+0000 7fe43d800700 1 -- 172.21.15.37:0/1520957470 shutdown_connections 2024-03-29T08:43:10.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:10.226+0000 7fe43d800700 1 -- 172.21.15.37:0/1520957470 wait complete. 2024-03-29T08:43:10.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:43:10 smithi037 ceph-mon[142325]: pgmap v969: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:43:10.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:43:10 smithi037 ceph-mon[142325]: from='client.35758 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:43:10.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:43:10 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/1520957470' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T08:43:11.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:43:10 smithi148 ceph-mon[106692]: pgmap v969: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:43:11.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:43:10 smithi148 ceph-mon[106692]: from='client.35758 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:43:11.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:43:10 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/1520957470' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T08:43:12.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:43:12 smithi037 ceph-mon[142325]: pgmap v970: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:43:13.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:43:12 smithi148 ceph-mon[106692]: pgmap v970: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:43:14.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:43:14 smithi037 ceph-mon[142325]: pgmap v971: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:43:15.055 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:43:14 smithi148 ceph-mon[106692]: pgmap v971: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:43:16.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:43:16 smithi037 ceph-mon[142325]: pgmap v972: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:43:17.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:43:16 smithi148 ceph-mon[106692]: pgmap v972: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:43:18.930 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:43:18 smithi037 ceph-mon[142325]: pgmap v973: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:43:19.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:43:18 smithi148 ceph-mon[106692]: pgmap v973: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:43:21.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:43:20 smithi148 ceph-mon[106692]: pgmap v974: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:43:21.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:43:20 smithi037 ceph-mon[142325]: pgmap v974: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:43:23.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:43:22 smithi148 ceph-mon[106692]: pgmap v975: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:43:23.149 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:43:22 smithi037 ceph-mon[142325]: pgmap v975: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:43:25.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:43:24 smithi148 ceph-mon[106692]: pgmap v976: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:43:25.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:43:24 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:43:25.176 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:43:24 smithi037 ceph-mon[142325]: pgmap v976: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:43:25.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:43:24 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:43:27.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:43:26 smithi148 ceph-mon[106692]: pgmap v977: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:43:27.178 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:43:26 smithi037 ceph-mon[142325]: pgmap v977: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:43:29.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:43:28 smithi148 ceph-mon[106692]: pgmap v978: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:43:29.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:43:28 smithi037 ceph-mon[142325]: pgmap v978: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:43:31.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:43:30 smithi148 ceph-mon[106692]: pgmap v979: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:43:31.176 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:43:30 smithi037 ceph-mon[142325]: pgmap v979: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:43:33.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:43:32 smithi148 ceph-mon[106692]: pgmap v980: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:43:33.149 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:43:32 smithi037 ceph-mon[142325]: pgmap v980: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:43:35.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:43:34 smithi148 ceph-mon[106692]: pgmap v981: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:43:35.176 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:43:34 smithi037 ceph-mon[142325]: pgmap v981: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:43:37.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:43:36 smithi148 ceph-mon[106692]: pgmap v982: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:43:37.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:43:36 smithi037 ceph-mon[142325]: pgmap v982: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:43:39.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:43:38 smithi148 ceph-mon[106692]: pgmap v983: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:43:39.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:43:38 smithi037 ceph-mon[142325]: pgmap v983: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:43:40.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:40.381+0000 7f641ec00700 1 -- 172.21.15.37:0/1682043315 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6418102e00 msgr2=0x7f64181031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:43:40.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:40.381+0000 7f641ec00700 1 --2- 172.21.15.37:0/1682043315 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6418102e00 0x7f64181031d0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f6408004940 tx=0x7f6408009e30 comp rx=0 tx=0).stop 2024-03-29T08:43:40.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:40.381+0000 7f641ec00700 1 -- 172.21.15.37:0/1682043315 shutdown_connections 2024-03-29T08:43:40.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:40.381+0000 7f641ec00700 1 --2- 172.21.15.37:0/1682043315 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f64181037a0 0x7f64180fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:43:40.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:40.381+0000 7f641ec00700 1 --2- 172.21.15.37:0/1682043315 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6418102e00 0x7f64181031d0 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:43:40.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:40.381+0000 7f641ec00700 1 -- 172.21.15.37:0/1682043315 >> 172.21.15.37:0/1682043315 conn(0x7f64180f89b0 msgr2=0x7f64180fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:43:40.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:40.381+0000 7f641ec00700 1 -- 172.21.15.37:0/1682043315 shutdown_connections 2024-03-29T08:43:40.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:40.381+0000 7f641ec00700 1 -- 172.21.15.37:0/1682043315 wait complete. 2024-03-29T08:43:40.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:40.382+0000 7f641ec00700 1 Processor -- start 2024-03-29T08:43:40.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:40.382+0000 7f641ec00700 1 -- start start 2024-03-29T08:43:40.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:40.383+0000 7f641ec00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6418102e00 0x7f6418115cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:43:40.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:40.383+0000 7f641ec00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f64181037a0 0x7f6418116210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:43:40.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:40.383+0000 7f641ec00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6418116860 con 0x7f64181037a0 2024-03-29T08:43:40.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:40.383+0000 7f641ec00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f641810fc30 con 0x7f6418102e00 2024-03-29T08:43:40.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:40.383+0000 7f6417400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f64181037a0 0x7f6418116210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:43:40.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:40.383+0000 7f6417400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f64181037a0 0x7f6418116210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41378/0 (socket says 172.21.15.37:41378) 2024-03-29T08:43:40.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:40.383+0000 7f6417400700 1 -- 172.21.15.37:0/854918085 learned_addr learned my addr 172.21.15.37:0/854918085 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:43:40.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:40.383+0000 7f6417e00700 1 --2- 172.21.15.37:0/854918085 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6418102e00 0x7f6418115cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:43:40.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:40.384+0000 7f6417400700 1 -- 172.21.15.37:0/854918085 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6418102e00 msgr2=0x7f6418115cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:43:40.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:40.384+0000 7f6417400700 1 --2- 172.21.15.37:0/854918085 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6418102e00 0x7f6418115cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:43:40.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:40.384+0000 7f6417400700 1 -- 172.21.15.37:0/854918085 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6408005040 con 0x7f64181037a0 2024-03-29T08:43:40.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:40.384+0000 7f6417400700 1 --2- 172.21.15.37:0/854918085 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f64181037a0 0x7f6418116210 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f6404002a50 tx=0x7f6404004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:43:40.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:40.384+0000 7f640fe00700 1 -- 172.21.15.37:0/854918085 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6404007cc0 con 0x7f64181037a0 2024-03-29T08:43:40.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:40.384+0000 7f640fe00700 1 -- 172.21.15.37:0/854918085 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f640400f040 con 0x7f64181037a0 2024-03-29T08:43:40.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:40.384+0000 7f640fe00700 1 -- 172.21.15.37:0/854918085 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f640400d920 con 0x7f64181037a0 2024-03-29T08:43:40.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:40.384+0000 7f641ec00700 1 -- 172.21.15.37:0/854918085 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f641810ff10 con 0x7f64181037a0 2024-03-29T08:43:40.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:40.384+0000 7f641ec00700 1 -- 172.21.15.37:0/854918085 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64181104d0 con 0x7f64181037a0 2024-03-29T08:43:40.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:40.385+0000 7f641ec00700 1 -- 172.21.15.37:0/854918085 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64180fe550 con 0x7f64181037a0 2024-03-29T08:43:40.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:40.387+0000 7f640fe00700 1 -- 172.21.15.37:0/854918085 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f6404009680 con 0x7f64181037a0 2024-03-29T08:43:40.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:40.388+0000 7f640fe00700 1 --2- 172.21.15.37:0/854918085 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f63fc06cc20 0x7f63fc06f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:43:40.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:40.388+0000 7f640fe00700 1 -- 172.21.15.37:0/854918085 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f640408bd10 con 0x7f64181037a0 2024-03-29T08:43:40.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:40.388+0000 7f6417e00700 1 --2- 172.21.15.37:0/854918085 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f63fc06cc20 0x7f63fc06f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:43:40.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:40.388+0000 7f6417e00700 1 --2- 172.21.15.37:0/854918085 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f63fc06cc20 0x7f63fc06f0d0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f6408004f90 tx=0x7f6408009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:43:40.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:40.392+0000 7f640fe00700 1 -- 172.21.15.37:0/854918085 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f6404059760 con 0x7f64181037a0 2024-03-29T08:43:40.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:40.630+0000 7f641ec00700 1 -- 172.21.15.37:0/854918085 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6418109e30 con 0x7f63fc06cc20 2024-03-29T08:43:40.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:40.631+0000 7f640fe00700 1 -- 172.21.15.37:0/854918085 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f6418109e30 con 0x7f63fc06cc20 2024-03-29T08:43:40.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:40.633+0000 7f641ec00700 1 -- 172.21.15.37:0/854918085 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f63fc06cc20 msgr2=0x7f63fc06f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:43:40.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:40.633+0000 7f641ec00700 1 --2- 172.21.15.37:0/854918085 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f63fc06cc20 0x7f63fc06f0d0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f6408004f90 tx=0x7f6408009e30 comp rx=0 tx=0).stop 2024-03-29T08:43:40.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:40.633+0000 7f641ec00700 1 -- 172.21.15.37:0/854918085 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f64181037a0 msgr2=0x7f6418116210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:43:40.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:40.633+0000 7f641ec00700 1 --2- 172.21.15.37:0/854918085 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f64181037a0 0x7f6418116210 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f6404002a50 tx=0x7f6404004600 comp rx=0 tx=0).stop 2024-03-29T08:43:40.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:40.633+0000 7f641ec00700 1 -- 172.21.15.37:0/854918085 shutdown_connections 2024-03-29T08:43:40.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:40.633+0000 7f641ec00700 1 --2- 172.21.15.37:0/854918085 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f63fc06cc20 0x7f63fc06f0d0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:43:40.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:40.633+0000 7f641ec00700 1 --2- 172.21.15.37:0/854918085 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6418102e00 0x7f6418115cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:43:40.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:40.633+0000 7f641ec00700 1 --2- 172.21.15.37:0/854918085 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f64181037a0 0x7f6418116210 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:43:40.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:40.633+0000 7f641ec00700 1 -- 172.21.15.37:0/854918085 >> 172.21.15.37:0/854918085 conn(0x7f64180f89b0 msgr2=0x7f64180f9980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:43:40.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:40.633+0000 7f641ec00700 1 -- 172.21.15.37:0/854918085 shutdown_connections 2024-03-29T08:43:40.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:40.634+0000 7f641ec00700 1 -- 172.21.15.37:0/854918085 wait complete. 2024-03-29T08:43:40.648 INFO:teuthology.orchestra.run.smithi037.stdout:true 2024-03-29T08:43:40.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:40.773+0000 7fa3f0a00700 1 -- 172.21.15.37:0/2269683257 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa3ec071f90 msgr2=0x7fa3ec070360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:43:40.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:40.773+0000 7fa3f0a00700 1 --2- 172.21.15.37:0/2269683257 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa3ec071f90 0x7fa3ec070360 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7fa3dc004970 tx=0x7fa3dc009e30 comp rx=0 tx=0).stop 2024-03-29T08:43:40.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:40.773+0000 7fa3f0a00700 1 -- 172.21.15.37:0/2269683257 shutdown_connections 2024-03-29T08:43:40.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:40.773+0000 7fa3f0a00700 1 --2- 172.21.15.37:0/2269683257 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa3ec0708a0 0x7fa3ec070d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:43:40.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:40.773+0000 7fa3f0a00700 1 --2- 172.21.15.37:0/2269683257 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa3ec071f90 0x7fa3ec070360 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:43:40.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:40.773+0000 7fa3f0a00700 1 -- 172.21.15.37:0/2269683257 >> 172.21.15.37:0/2269683257 conn(0x7fa3ec0f89b0 msgr2=0x7fa3ec0fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:43:40.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:40.773+0000 7fa3f0a00700 1 -- 172.21.15.37:0/2269683257 shutdown_connections 2024-03-29T08:43:40.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:40.773+0000 7fa3f0a00700 1 -- 172.21.15.37:0/2269683257 wait complete. 2024-03-29T08:43:40.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:40.774+0000 7fa3f0a00700 1 Processor -- start 2024-03-29T08:43:40.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:40.774+0000 7fa3f0a00700 1 -- start start 2024-03-29T08:43:40.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:40.774+0000 7fa3f0a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa3ec0708a0 0x7fa3ec11a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:43:40.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:40.774+0000 7fa3f0a00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa3ec071f90 0x7fa3ec11a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:43:40.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:40.774+0000 7fa3f0a00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3ec11ac50 con 0x7fa3ec0708a0 2024-03-29T08:43:40.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:40.774+0000 7fa3f0a00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3ec114020 con 0x7fa3ec071f90 2024-03-29T08:43:40.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:40.774+0000 7fa3e9e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa3ec0708a0 0x7fa3ec11a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:43:40.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:40.775+0000 7fa3e9e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa3ec0708a0 0x7fa3ec11a0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41400/0 (socket says 172.21.15.37:41400) 2024-03-29T08:43:40.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:40.775+0000 7fa3e9e00700 1 -- 172.21.15.37:0/2426880261 learned_addr learned my addr 172.21.15.37:0/2426880261 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:43:40.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:40.775+0000 7fa3e9400700 1 --2- 172.21.15.37:0/2426880261 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa3ec071f90 0x7fa3ec11a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:43:40.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:40.775+0000 7fa3e9e00700 1 -- 172.21.15.37:0/2426880261 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa3ec071f90 msgr2=0x7fa3ec11a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:43:40.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:40.775+0000 7fa3e9e00700 1 --2- 172.21.15.37:0/2426880261 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa3ec071f90 0x7fa3ec11a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:43:40.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:40.775+0000 7fa3e9e00700 1 -- 172.21.15.37:0/2426880261 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3dc005040 con 0x7fa3ec0708a0 2024-03-29T08:43:40.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:40.775+0000 7fa3e9e00700 1 --2- 172.21.15.37:0/2426880261 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa3ec0708a0 0x7fa3ec11a0c0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7fa3dc005010 tx=0x7fa3dc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:43:40.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:40.775+0000 7fa3e2000700 1 -- 172.21.15.37:0/2426880261 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa3dc00e040 con 0x7fa3ec0708a0 2024-03-29T08:43:40.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:40.775+0000 7fa3e2000700 1 -- 172.21.15.37:0/2426880261 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fa3dc009930 con 0x7fa3ec0708a0 2024-03-29T08:43:40.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:40.775+0000 7fa3e2000700 1 -- 172.21.15.37:0/2426880261 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa3dc01c3f0 con 0x7fa3ec0708a0 2024-03-29T08:43:40.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:40.775+0000 7fa3f0a00700 1 -- 172.21.15.37:0/2426880261 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3ec1142a0 con 0x7fa3ec0708a0 2024-03-29T08:43:40.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:40.776+0000 7fa3f0a00700 1 -- 172.21.15.37:0/2426880261 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3ec1147e0 con 0x7fa3ec0708a0 2024-03-29T08:43:40.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:40.777+0000 7fa3f0a00700 1 -- 172.21.15.37:0/2426880261 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3ec071aa0 con 0x7fa3ec0708a0 2024-03-29T08:43:40.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:40.778+0000 7fa3e2000700 1 -- 172.21.15.37:0/2426880261 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fa3dc013420 con 0x7fa3ec0708a0 2024-03-29T08:43:40.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:40.778+0000 7fa3e2000700 1 --2- 172.21.15.37:0/2426880261 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fa3d006cc30 0x7fa3d006f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:43:40.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:40.779+0000 7fa3e2000700 1 -- 172.21.15.37:0/2426880261 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fa3dc08d270 con 0x7fa3ec0708a0 2024-03-29T08:43:40.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:40.779+0000 7fa3e9400700 1 --2- 172.21.15.37:0/2426880261 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fa3d006cc30 0x7fa3d006f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:43:40.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:40.779+0000 7fa3e9400700 1 --2- 172.21.15.37:0/2426880261 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fa3d006cc30 0x7fa3d006f0e0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fa3d8004600 tx=0x7fa3d800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:43:40.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:40.783+0000 7fa3e2000700 1 -- 172.21.15.37:0/2426880261 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fa3dc05ad40 con 0x7fa3ec0708a0 2024-03-29T08:43:40.805 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:43:40 smithi148 ceph-mon[106692]: pgmap v984: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:43:40.805 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:43:40 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:43:40.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:43:40 smithi037 ceph-mon[142325]: pgmap v984: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:43:40.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:43:40 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:43:41.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:41.018+0000 7fa3f0a00700 1 -- 172.21.15.37:0/2426880261 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa3ec04bf40 con 0x7fa3d006cc30 2024-03-29T08:43:41.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:41.019+0000 7fa3e2000700 1 -- 172.21.15.37:0/2426880261 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fa3ec04bf40 con 0x7fa3d006cc30 2024-03-29T08:43:41.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:41.022+0000 7fa3f0a00700 1 -- 172.21.15.37:0/2426880261 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fa3d006cc30 msgr2=0x7fa3d006f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:43:41.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:41.022+0000 7fa3f0a00700 1 --2- 172.21.15.37:0/2426880261 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fa3d006cc30 0x7fa3d006f0e0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fa3d8004600 tx=0x7fa3d800b040 comp rx=0 tx=0).stop 2024-03-29T08:43:41.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:41.022+0000 7fa3f0a00700 1 -- 172.21.15.37:0/2426880261 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa3ec0708a0 msgr2=0x7fa3ec11a0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:43:41.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:41.022+0000 7fa3f0a00700 1 --2- 172.21.15.37:0/2426880261 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa3ec0708a0 0x7fa3ec11a0c0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7fa3dc005010 tx=0x7fa3dc009e30 comp rx=0 tx=0).stop 2024-03-29T08:43:41.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:41.022+0000 7fa3f0a00700 1 -- 172.21.15.37:0/2426880261 shutdown_connections 2024-03-29T08:43:41.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:41.022+0000 7fa3f0a00700 1 --2- 172.21.15.37:0/2426880261 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fa3d006cc30 0x7fa3d006f0e0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:43:41.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:41.022+0000 7fa3f0a00700 1 --2- 172.21.15.37:0/2426880261 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa3ec0708a0 0x7fa3ec11a0c0 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:43:41.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:41.022+0000 7fa3f0a00700 1 --2- 172.21.15.37:0/2426880261 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa3ec071f90 0x7fa3ec11a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:43:41.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:41.022+0000 7fa3f0a00700 1 -- 172.21.15.37:0/2426880261 >> 172.21.15.37:0/2426880261 conn(0x7fa3ec0f89b0 msgr2=0x7fa3ec0f9b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:43:41.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:41.022+0000 7fa3f0a00700 1 -- 172.21.15.37:0/2426880261 shutdown_connections 2024-03-29T08:43:41.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:41.022+0000 7fa3f0a00700 1 -- 172.21.15.37:0/2426880261 wait complete. 2024-03-29T08:43:41.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:41.158+0000 7fe553400700 1 -- 172.21.15.37:0/1306253306 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe54c0fde90 msgr2=0x7fe54c0fe260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:43:41.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:41.158+0000 7fe553400700 1 --2- 172.21.15.37:0/1306253306 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe54c0fde90 0x7fe54c0fe260 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7fe540004940 tx=0x7fe540009e30 comp rx=0 tx=0).stop 2024-03-29T08:43:41.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:41.158+0000 7fe553400700 1 -- 172.21.15.37:0/1306253306 shutdown_connections 2024-03-29T08:43:41.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:41.158+0000 7fe553400700 1 --2- 172.21.15.37:0/1306253306 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe54c0fe830 0x7fe54c106f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:43:41.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:41.158+0000 7fe553400700 1 --2- 172.21.15.37:0/1306253306 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe54c0fde90 0x7fe54c0fe260 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:43:41.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:41.158+0000 7fe553400700 1 -- 172.21.15.37:0/1306253306 >> 172.21.15.37:0/1306253306 conn(0x7fe54c072bc0 msgr2=0x7fe54c072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:43:41.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:41.158+0000 7fe553400700 1 -- 172.21.15.37:0/1306253306 shutdown_connections 2024-03-29T08:43:41.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:41.158+0000 7fe553400700 1 -- 172.21.15.37:0/1306253306 wait complete. 2024-03-29T08:43:41.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:41.159+0000 7fe553400700 1 Processor -- start 2024-03-29T08:43:41.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:41.159+0000 7fe553400700 1 -- start start 2024-03-29T08:43:41.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:41.160+0000 7fe553400700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe54c0fde90 0x7fe54c115c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:43:41.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:41.160+0000 7fe553400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe54c0fe830 0x7fe54c1161c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:43:41.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:41.160+0000 7fe553400700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe54c116810 con 0x7fe54c0fe830 2024-03-29T08:43:41.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:41.160+0000 7fe553400700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe54c10fbe0 con 0x7fe54c0fde90 2024-03-29T08:43:41.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:41.160+0000 7fe54be00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe54c0fe830 0x7fe54c1161c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:43:41.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:41.160+0000 7fe54be00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe54c0fe830 0x7fe54c1161c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41416/0 (socket says 172.21.15.37:41416) 2024-03-29T08:43:41.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:41.160+0000 7fe54be00700 1 -- 172.21.15.37:0/4016783458 learned_addr learned my addr 172.21.15.37:0/4016783458 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:43:41.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:41.160+0000 7fe550a00700 1 --2- 172.21.15.37:0/4016783458 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe54c0fde90 0x7fe54c115c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:43:41.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:41.160+0000 7fe54be00700 1 -- 172.21.15.37:0/4016783458 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe54c0fde90 msgr2=0x7fe54c115c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:43:41.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:41.160+0000 7fe54be00700 1 --2- 172.21.15.37:0/4016783458 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe54c0fde90 0x7fe54c115c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:43:41.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:41.160+0000 7fe54be00700 1 -- 172.21.15.37:0/4016783458 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe540005040 con 0x7fe54c0fe830 2024-03-29T08:43:41.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:41.161+0000 7fe54be00700 1 --2- 172.21.15.37:0/4016783458 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe54c0fe830 0x7fe54c1161c0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fe538002a80 tx=0x7fe5380046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:43:41.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:41.161+0000 7fe548c00700 1 -- 172.21.15.37:0/4016783458 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe538007cc0 con 0x7fe54c0fe830 2024-03-29T08:43:41.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:41.161+0000 7fe553400700 1 -- 172.21.15.37:0/4016783458 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe54c10fec0 con 0x7fe54c0fe830 2024-03-29T08:43:41.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:41.161+0000 7fe548c00700 1 -- 172.21.15.37:0/4016783458 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fe53800f040 con 0x7fe54c0fe830 2024-03-29T08:43:41.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:41.161+0000 7fe548c00700 1 -- 172.21.15.37:0/4016783458 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe538009640 con 0x7fe54c0fe830 2024-03-29T08:43:41.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:41.161+0000 7fe553400700 1 -- 172.21.15.37:0/4016783458 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe54c110400 con 0x7fe54c0fe830 2024-03-29T08:43:41.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:41.162+0000 7fe553400700 1 -- 172.21.15.37:0/4016783458 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe54c106a20 con 0x7fe54c0fe830 2024-03-29T08:43:41.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:41.163+0000 7fe548c00700 1 -- 172.21.15.37:0/4016783458 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fe5380097a0 con 0x7fe54c0fe830 2024-03-29T08:43:41.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:41.164+0000 7fe548c00700 1 --2- 172.21.15.37:0/4016783458 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fe53406cc30 0x7fe53406f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:43:41.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:41.164+0000 7fe548c00700 1 -- 172.21.15.37:0/4016783458 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fe53808b8f0 con 0x7fe54c0fe830 2024-03-29T08:43:41.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:41.165+0000 7fe550a00700 1 --2- 172.21.15.37:0/4016783458 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fe53406cc30 0x7fe53406f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:43:41.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:41.165+0000 7fe550a00700 1 --2- 172.21.15.37:0/4016783458 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fe53406cc30 0x7fe53406f0e0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fe54000b3b0 tx=0x7fe540009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:43:41.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:41.168+0000 7fe548c00700 1 -- 172.21.15.37:0/4016783458 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fe5380592c0 con 0x7fe54c0fe830 2024-03-29T08:43:41.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:41.392+0000 7fe553400700 1 -- 172.21.15.37:0/4016783458 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fe54c111340 con 0x7fe53406cc30 2024-03-29T08:43:41.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:41.402+0000 7fe548c00700 1 -- 172.21.15.37:0/4016783458 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7fe54c111340 con 0x7fe53406cc30 2024-03-29T08:43:41.402 INFO:teuthology.orchestra.run.smithi037.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-29T08:43:41.402 INFO:teuthology.orchestra.run.smithi037.stdout:alertmanager.smithi037 smithi037 *:9093,9094 running (33m) 10m ago 46m 14.3M - 0.25.0 c8568f914cd2 2f9f9746ed0f 2024-03-29T08:43:41.402 INFO:teuthology.orchestra.run.smithi037.stdout:crash.smithi037 smithi037 running (30m) 10m ago 46m 6647k - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 178d416a3186 2024-03-29T08:43:41.402 INFO:teuthology.orchestra.run.smithi037.stdout:crash.smithi148 smithi148 running (30m) 10m ago 45m 6656k - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 23666989d58f 2024-03-29T08:43:41.402 INFO:teuthology.orchestra.run.smithi037.stdout:grafana.smithi037 smithi037 *:3000 running (33m) 10m ago 46m 74.5M - 9.4.7 954c08fa6188 6b976fe49952 2024-03-29T08:43:41.402 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi037.iifsmh smithi037 running (25m) 10m ago 39m 16.4M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 2d97f562cc5b 2024-03-29T08:43:41.402 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi037.mmwgxe smithi037 running (26m) 10m ago 40m 16.7M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff ec37037f46e8 2024-03-29T08:43:41.402 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi148.qukpjh smithi148 running (10m) 10m ago 39m 10.7M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 3257219bc046 2024-03-29T08:43:41.402 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi148.sqteih smithi148 running (25m) 10m ago 40m 16.3M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 44c15c77037a 2024-03-29T08:43:41.402 INFO:teuthology.orchestra.run.smithi037.stdout:mgr.smithi037.wjikii smithi037 *:8443,9283,8765 running (35m) 10m ago 48m 510M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff c5a1417b9417 2024-03-29T08:43:41.402 INFO:teuthology.orchestra.run.smithi037.stdout:mgr.smithi148.onhgvm smithi148 *:8443,9283,8765 running (34m) 10m ago 45m 456M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff fcfdbce062bd 2024-03-29T08:43:41.403 INFO:teuthology.orchestra.run.smithi037.stdout:mon.smithi037 smithi037 running (31m) 10m ago 48m 70.2M 2048M 18.2.2-1015-gd0e392d9 1cc2f45c17ff eea2060783af 2024-03-29T08:43:41.403 INFO:teuthology.orchestra.run.smithi037.stdout:mon.smithi148 smithi148 running (31m) 10m ago 45m 50.4M 2048M 18.2.2-1015-gd0e392d9 1cc2f45c17ff ad4b2c5748be 2024-03-29T08:43:41.403 INFO:teuthology.orchestra.run.smithi037.stdout:node-exporter.smithi037 smithi037 *:9100 running (34m) 10m ago 46m 8514k - 1.5.0 0da6a335fe13 26b8ae9cb878 2024-03-29T08:43:41.403 INFO:teuthology.orchestra.run.smithi037.stdout:node-exporter.smithi148 smithi148 *:9100 running (34m) 10m ago 45m 8464k - 1.5.0 0da6a335fe13 98414e4e64ed 2024-03-29T08:43:41.403 INFO:teuthology.orchestra.run.smithi037.stdout:osd.0 smithi037 running (29m) 10m ago 44m 108M 1619M 18.2.2-1015-gd0e392d9 1cc2f45c17ff c0c584f6245c 2024-03-29T08:43:41.403 INFO:teuthology.orchestra.run.smithi037.stdout:osd.1 smithi037 running (29m) 10m ago 43m 104M 1619M 18.2.2-1015-gd0e392d9 1cc2f45c17ff a4a2ff4cc83d 2024-03-29T08:43:41.403 INFO:teuthology.orchestra.run.smithi037.stdout:osd.2 smithi037 running (28m) 10m ago 43m 85.9M 1619M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 8b1bc71aa79e 2024-03-29T08:43:41.403 INFO:teuthology.orchestra.run.smithi037.stdout:osd.3 smithi148 running (27m) 10m ago 42m 109M 2642M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 1c656ab8ff86 2024-03-29T08:43:41.403 INFO:teuthology.orchestra.run.smithi037.stdout:osd.4 smithi148 running (27m) 10m ago 42m 101M 2642M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 966a9de11d22 2024-03-29T08:43:41.403 INFO:teuthology.orchestra.run.smithi037.stdout:osd.5 smithi148 running (26m) 10m ago 41m 89.4M 2642M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 474c4f9212ce 2024-03-29T08:43:41.403 INFO:teuthology.orchestra.run.smithi037.stdout:prometheus.smithi037 smithi037 *:9095 running (33m) 10m ago 45m 61.0M - 2.43.0 a07b618ecd1d 30c0223ec589 2024-03-29T08:43:41.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:41.404+0000 7fe553400700 1 -- 172.21.15.37:0/4016783458 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fe53406cc30 msgr2=0x7fe53406f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:43:41.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:41.404+0000 7fe553400700 1 --2- 172.21.15.37:0/4016783458 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fe53406cc30 0x7fe53406f0e0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fe54000b3b0 tx=0x7fe540009e30 comp rx=0 tx=0).stop 2024-03-29T08:43:41.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:41.404+0000 7fe553400700 1 -- 172.21.15.37:0/4016783458 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe54c0fe830 msgr2=0x7fe54c1161c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:43:41.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:41.404+0000 7fe553400700 1 --2- 172.21.15.37:0/4016783458 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe54c0fe830 0x7fe54c1161c0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fe538002a80 tx=0x7fe5380046b0 comp rx=0 tx=0).stop 2024-03-29T08:43:41.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:41.404+0000 7fe553400700 1 -- 172.21.15.37:0/4016783458 shutdown_connections 2024-03-29T08:43:41.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:41.404+0000 7fe553400700 1 --2- 172.21.15.37:0/4016783458 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fe53406cc30 0x7fe53406f0e0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:43:41.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:41.404+0000 7fe553400700 1 --2- 172.21.15.37:0/4016783458 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe54c0fde90 0x7fe54c115c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:43:41.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:41.404+0000 7fe553400700 1 --2- 172.21.15.37:0/4016783458 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe54c0fe830 0x7fe54c1161c0 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:43:41.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:41.404+0000 7fe553400700 1 -- 172.21.15.37:0/4016783458 >> 172.21.15.37:0/4016783458 conn(0x7fe54c072bc0 msgr2=0x7fe54c0fb7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:43:41.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:41.405+0000 7fe553400700 1 -- 172.21.15.37:0/4016783458 shutdown_connections 2024-03-29T08:43:41.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:41.405+0000 7fe553400700 1 -- 172.21.15.37:0/4016783458 wait complete. 2024-03-29T08:43:41.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:41.540+0000 7fd270200700 1 -- 172.21.15.37:0/3546269692 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd268102e00 msgr2=0x7fd2681031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:43:41.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:41.540+0000 7fd270200700 1 --2- 172.21.15.37:0/3546269692 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd268102e00 0x7fd2681031d0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7fd25c004940 tx=0x7fd25c009e30 comp rx=0 tx=0).stop 2024-03-29T08:43:41.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:41.540+0000 7fd270200700 1 -- 172.21.15.37:0/3546269692 shutdown_connections 2024-03-29T08:43:41.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:41.540+0000 7fd270200700 1 --2- 172.21.15.37:0/3546269692 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd2681037a0 0x7fd2680fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:43:41.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:41.540+0000 7fd270200700 1 --2- 172.21.15.37:0/3546269692 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd268102e00 0x7fd2681031d0 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:43:41.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:41.540+0000 7fd270200700 1 -- 172.21.15.37:0/3546269692 >> 172.21.15.37:0/3546269692 conn(0x7fd2680f89b0 msgr2=0x7fd2680fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:43:41.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:41.540+0000 7fd270200700 1 -- 172.21.15.37:0/3546269692 shutdown_connections 2024-03-29T08:43:41.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:41.540+0000 7fd270200700 1 -- 172.21.15.37:0/3546269692 wait complete. 2024-03-29T08:43:41.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:41.541+0000 7fd270200700 1 Processor -- start 2024-03-29T08:43:41.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:41.541+0000 7fd270200700 1 -- start start 2024-03-29T08:43:41.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:41.541+0000 7fd270200700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd268102e00 0x7fd268115d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:43:41.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:41.541+0000 7fd270200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd2681037a0 0x7fd268116250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:43:41.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:41.541+0000 7fd270200700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2681168a0 con 0x7fd2681037a0 2024-03-29T08:43:41.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:41.541+0000 7fd270200700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd26810fc70 con 0x7fd268102e00 2024-03-29T08:43:41.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:41.541+0000 7fd26ce00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd2681037a0 0x7fd268116250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:43:41.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:41.542+0000 7fd26ce00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd2681037a0 0x7fd268116250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41440/0 (socket says 172.21.15.37:41440) 2024-03-29T08:43:41.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:41.542+0000 7fd26ce00700 1 -- 172.21.15.37:0/3571708384 learned_addr learned my addr 172.21.15.37:0/3571708384 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:43:41.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:41.542+0000 7fd26d800700 1 --2- 172.21.15.37:0/3571708384 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd268102e00 0x7fd268115d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:43:41.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:41.542+0000 7fd26ce00700 1 -- 172.21.15.37:0/3571708384 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd268102e00 msgr2=0x7fd268115d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:43:41.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:41.542+0000 7fd26ce00700 1 --2- 172.21.15.37:0/3571708384 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd268102e00 0x7fd268115d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:43:41.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:41.542+0000 7fd26ce00700 1 -- 172.21.15.37:0/3571708384 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd25c005040 con 0x7fd2681037a0 2024-03-29T08:43:41.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:41.542+0000 7fd26ce00700 1 --2- 172.21.15.37:0/3571708384 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd2681037a0 0x7fd268116250 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fd25400c890 tx=0x7fd2540044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:43:41.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:41.542+0000 7fd265600700 1 -- 172.21.15.37:0/3571708384 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd25400a700 con 0x7fd2681037a0 2024-03-29T08:43:41.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:41.542+0000 7fd265600700 1 -- 172.21.15.37:0/3571708384 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd25400ad10 con 0x7fd2681037a0 2024-03-29T08:43:41.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:41.542+0000 7fd270200700 1 -- 172.21.15.37:0/3571708384 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd26810ff50 con 0x7fd2681037a0 2024-03-29T08:43:41.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:41.543+0000 7fd265600700 1 -- 172.21.15.37:0/3571708384 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd254012e70 con 0x7fd2681037a0 2024-03-29T08:43:41.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:41.543+0000 7fd270200700 1 -- 172.21.15.37:0/3571708384 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd268110510 con 0x7fd2681037a0 2024-03-29T08:43:41.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:41.544+0000 7fd270200700 1 -- 172.21.15.37:0/3571708384 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2680fe550 con 0x7fd2681037a0 2024-03-29T08:43:41.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:41.545+0000 7fd265600700 1 -- 172.21.15.37:0/3571708384 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fd25400a860 con 0x7fd2681037a0 2024-03-29T08:43:41.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:41.546+0000 7fd265600700 1 --2- 172.21.15.37:0/3571708384 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fd25006cc30 0x7fd25006f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:43:41.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:41.546+0000 7fd265600700 1 -- 172.21.15.37:0/3571708384 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fd25408c050 con 0x7fd2681037a0 2024-03-29T08:43:41.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:41.546+0000 7fd26d800700 1 --2- 172.21.15.37:0/3571708384 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fd25006cc30 0x7fd25006f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:43:41.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:41.547+0000 7fd26d800700 1 --2- 172.21.15.37:0/3571708384 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fd25006cc30 0x7fd25006f0e0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fd25c00b840 tx=0x7fd25c009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:43:41.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:41.550+0000 7fd265600700 1 -- 172.21.15.37:0/3571708384 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fd254059b20 con 0x7fd2681037a0 2024-03-29T08:43:41.805 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:43:41 smithi148 ceph-mon[106692]: from='client.35766 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:43:41.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:41.849+0000 7fd270200700 1 -- 172.21.15.37:0/3571708384 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fd26804bed0 con 0x7fd2681037a0 2024-03-29T08:43:41.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:41.849+0000 7fd265600700 1 -- 172.21.15.37:0/3571708384 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7fd2540596b0 con 0x7fd2681037a0 2024-03-29T08:43:41.849 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-29T08:43:41.849 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": { 2024-03-29T08:43:41.849 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 2 2024-03-29T08:43:41.849 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:43:41.849 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": { 2024-03-29T08:43:41.849 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 2 2024-03-29T08:43:41.849 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:43:41.849 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": { 2024-03-29T08:43:41.850 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 6 2024-03-29T08:43:41.850 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:43:41.850 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": { 2024-03-29T08:43:41.850 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 4 2024-03-29T08:43:41.850 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:43:41.850 INFO:teuthology.orchestra.run.smithi037.stdout: "overall": { 2024-03-29T08:43:41.850 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 14 2024-03-29T08:43:41.850 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-29T08:43:41.850 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-29T08:43:41.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:41.852+0000 7fd270200700 1 -- 172.21.15.37:0/3571708384 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fd25006cc30 msgr2=0x7fd25006f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:43:41.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:41.852+0000 7fd270200700 1 --2- 172.21.15.37:0/3571708384 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fd25006cc30 0x7fd25006f0e0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fd25c00b840 tx=0x7fd25c009e30 comp rx=0 tx=0).stop 2024-03-29T08:43:41.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:41.852+0000 7fd270200700 1 -- 172.21.15.37:0/3571708384 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd2681037a0 msgr2=0x7fd268116250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:43:41.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:41.852+0000 7fd270200700 1 --2- 172.21.15.37:0/3571708384 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd2681037a0 0x7fd268116250 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fd25400c890 tx=0x7fd2540044c0 comp rx=0 tx=0).stop 2024-03-29T08:43:41.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:41.852+0000 7fd270200700 1 -- 172.21.15.37:0/3571708384 shutdown_connections 2024-03-29T08:43:41.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:41.852+0000 7fd270200700 1 --2- 172.21.15.37:0/3571708384 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fd25006cc30 0x7fd25006f0e0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:43:41.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:41.852+0000 7fd270200700 1 --2- 172.21.15.37:0/3571708384 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd268102e00 0x7fd268115d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:43:41.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:41.852+0000 7fd270200700 1 --2- 172.21.15.37:0/3571708384 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd2681037a0 0x7fd268116250 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:43:41.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:41.852+0000 7fd270200700 1 -- 172.21.15.37:0/3571708384 >> 172.21.15.37:0/3571708384 conn(0x7fd2680f89b0 msgr2=0x7fd2680704f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:43:41.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:41.852+0000 7fd270200700 1 -- 172.21.15.37:0/3571708384 shutdown_connections 2024-03-29T08:43:41.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:41.852+0000 7fd270200700 1 -- 172.21.15.37:0/3571708384 wait complete. 2024-03-29T08:43:41.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:43:41 smithi037 ceph-mon[142325]: from='client.35766 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:43:41.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:41.989+0000 7f809d600700 1 -- 172.21.15.37:0/2737184395 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f80980fe820 msgr2=0x7f8098106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:43:41.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:41.989+0000 7f809d600700 1 --2- 172.21.15.37:0/2737184395 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f80980fe820 0x7f8098106e60 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f8084004940 tx=0x7f8084009e30 comp rx=0 tx=0).stop 2024-03-29T08:43:41.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:41.989+0000 7f809d600700 1 -- 172.21.15.37:0/2737184395 shutdown_connections 2024-03-29T08:43:41.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:41.989+0000 7f809d600700 1 --2- 172.21.15.37:0/2737184395 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f80980fe820 0x7f8098106e60 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:43:41.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:41.989+0000 7f809d600700 1 --2- 172.21.15.37:0/2737184395 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f80980fde80 0x7f80980fe250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:43:41.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:41.989+0000 7f809d600700 1 -- 172.21.15.37:0/2737184395 >> 172.21.15.37:0/2737184395 conn(0x7f80980f78f0 msgr2=0x7f80980f9d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:43:41.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:41.990+0000 7f809d600700 1 -- 172.21.15.37:0/2737184395 shutdown_connections 2024-03-29T08:43:41.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:41.990+0000 7f809d600700 1 -- 172.21.15.37:0/2737184395 wait complete. 2024-03-29T08:43:41.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:41.990+0000 7f809d600700 1 Processor -- start 2024-03-29T08:43:41.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:41.991+0000 7f809d600700 1 -- start start 2024-03-29T08:43:41.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:41.991+0000 7f809d600700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f80980fde80 0x7f80981138d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:43:41.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:41.991+0000 7f809d600700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f80980fe820 0x7f8098113e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:43:41.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:41.991+0000 7f809d600700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8098114460 con 0x7f80980fde80 2024-03-29T08:43:41.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:41.991+0000 7f809d600700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f809810da60 con 0x7f80980fe820 2024-03-29T08:43:41.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:41.991+0000 7f8096a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f80980fde80 0x7f80981138d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:43:41.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:41.991+0000 7f8096a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f80980fde80 0x7f80981138d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41452/0 (socket says 172.21.15.37:41452) 2024-03-29T08:43:41.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:41.991+0000 7f8096a00700 1 -- 172.21.15.37:0/2485841523 learned_addr learned my addr 172.21.15.37:0/2485841523 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:43:41.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:41.992+0000 7f8096a00700 1 -- 172.21.15.37:0/2485841523 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f80980fe820 msgr2=0x7f8098113e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:43:41.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:41.992+0000 7f8096a00700 1 --2- 172.21.15.37:0/2485841523 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f80980fe820 0x7f8098113e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:43:41.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:41.992+0000 7f8096a00700 1 -- 172.21.15.37:0/2485841523 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8084005040 con 0x7f80980fde80 2024-03-29T08:43:41.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:41.992+0000 7f8096a00700 1 --2- 172.21.15.37:0/2485841523 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f80980fde80 0x7f80981138d0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f8088002a50 tx=0x7f80880046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:43:41.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:41.992+0000 7f808ea00700 1 -- 172.21.15.37:0/2485841523 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8088007cc0 con 0x7f80980fde80 2024-03-29T08:43:41.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:41.992+0000 7f808ea00700 1 -- 172.21.15.37:0/2485841523 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f808800f040 con 0x7f80980fde80 2024-03-29T08:43:41.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:41.992+0000 7f808ea00700 1 -- 172.21.15.37:0/2485841523 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8088009540 con 0x7f80980fde80 2024-03-29T08:43:41.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:41.992+0000 7f809d600700 1 -- 172.21.15.37:0/2485841523 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f809810dd40 con 0x7f80980fde80 2024-03-29T08:43:41.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:41.992+0000 7f809d600700 1 -- 172.21.15.37:0/2485841523 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f809810e230 con 0x7f80980fde80 2024-03-29T08:43:41.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:41.993+0000 7f809d600700 1 -- 172.21.15.37:0/2485841523 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8098066af0 con 0x7f80980fde80 2024-03-29T08:43:41.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:41.994+0000 7f808ea00700 1 -- 172.21.15.37:0/2485841523 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f80880096a0 con 0x7f80980fde80 2024-03-29T08:43:41.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:41.995+0000 7f808ea00700 1 --2- 172.21.15.37:0/2485841523 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f807c06cc20 0x7f807c06f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:43:41.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:41.995+0000 7f808ea00700 1 -- 172.21.15.37:0/2485841523 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f808808bee0 con 0x7f80980fde80 2024-03-29T08:43:41.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:41.996+0000 7f8096000700 1 --2- 172.21.15.37:0/2485841523 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f807c06cc20 0x7f807c06f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:43:41.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:41.996+0000 7f8096000700 1 --2- 172.21.15.37:0/2485841523 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f807c06cc20 0x7f807c06f0d0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f8084004f90 tx=0x7f8084009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:43:41.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:42.000+0000 7f808ea00700 1 -- 172.21.15.37:0/2485841523 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f80880598b0 con 0x7f80980fde80 2024-03-29T08:43:42.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:42.256+0000 7f809d600700 1 -- 172.21.15.37:0/2485841523 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f80981016e0 con 0x7f80980fde80 2024-03-29T08:43:42.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:42.257+0000 7f808ea00700 1 -- 172.21.15.37:0/2485841523 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 30 v30) v1 ==== 76+0+1841 (secure 0 0 0) 0x7f8088059440 con 0x7f80980fde80 2024-03-29T08:43:42.257 INFO:teuthology.orchestra.run.smithi037.stdout:e30 2024-03-29T08:43:42.257 INFO:teuthology.orchestra.run.smithi037.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-29T08:43:42.257 INFO:teuthology.orchestra.run.smithi037.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T08:43:42.257 INFO:teuthology.orchestra.run.smithi037.stdout:legacy client fscid: 1 2024-03-29T08:43:42.257 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:43:42.257 INFO:teuthology.orchestra.run.smithi037.stdout:Filesystem 'cephfs' (1) 2024-03-29T08:43:42.257 INFO:teuthology.orchestra.run.smithi037.stdout:fs_name cephfs 2024-03-29T08:43:42.257 INFO:teuthology.orchestra.run.smithi037.stdout:epoch 21 2024-03-29T08:43:42.257 INFO:teuthology.orchestra.run.smithi037.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-03-29T08:43:42.257 INFO:teuthology.orchestra.run.smithi037.stdout:created 2024-03-29T08:03:29.241269+0000 2024-03-29T08:43:42.257 INFO:teuthology.orchestra.run.smithi037.stdout:modified 2024-03-29T08:17:16.237238+0000 2024-03-29T08:43:42.258 INFO:teuthology.orchestra.run.smithi037.stdout:tableserver 0 2024-03-29T08:43:42.258 INFO:teuthology.orchestra.run.smithi037.stdout:root 0 2024-03-29T08:43:42.258 INFO:teuthology.orchestra.run.smithi037.stdout:session_timeout 60 2024-03-29T08:43:42.258 INFO:teuthology.orchestra.run.smithi037.stdout:session_autoclose 300 2024-03-29T08:43:42.258 INFO:teuthology.orchestra.run.smithi037.stdout:max_file_size 1099511627776 2024-03-29T08:43:42.258 INFO:teuthology.orchestra.run.smithi037.stdout:max_xattr_size 65536 2024-03-29T08:43:42.258 INFO:teuthology.orchestra.run.smithi037.stdout:required_client_features {} 2024-03-29T08:43:42.259 INFO:teuthology.orchestra.run.smithi037.stdout:last_failure 0 2024-03-29T08:43:42.259 INFO:teuthology.orchestra.run.smithi037.stdout:last_failure_osd_epoch 82 2024-03-29T08:43:42.259 INFO:teuthology.orchestra.run.smithi037.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T08:43:42.259 INFO:teuthology.orchestra.run.smithi037.stdout:max_mds 2 2024-03-29T08:43:42.259 INFO:teuthology.orchestra.run.smithi037.stdout:in 0,1 2024-03-29T08:43:42.259 INFO:teuthology.orchestra.run.smithi037.stdout:up {} 2024-03-29T08:43:42.259 INFO:teuthology.orchestra.run.smithi037.stdout:failed 0,1 2024-03-29T08:43:42.259 INFO:teuthology.orchestra.run.smithi037.stdout:damaged 2024-03-29T08:43:42.259 INFO:teuthology.orchestra.run.smithi037.stdout:stopped 2024-03-29T08:43:42.259 INFO:teuthology.orchestra.run.smithi037.stdout:data_pools [3] 2024-03-29T08:43:42.259 INFO:teuthology.orchestra.run.smithi037.stdout:metadata_pool 2 2024-03-29T08:43:42.260 INFO:teuthology.orchestra.run.smithi037.stdout:inline_data disabled 2024-03-29T08:43:42.260 INFO:teuthology.orchestra.run.smithi037.stdout:balancer 2024-03-29T08:43:42.260 INFO:teuthology.orchestra.run.smithi037.stdout:bal_rank_mask -1 2024-03-29T08:43:42.260 INFO:teuthology.orchestra.run.smithi037.stdout:standby_count_wanted 1 2024-03-29T08:43:42.260 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:43:42.260 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:43:42.260 INFO:teuthology.orchestra.run.smithi037.stdout:Standby daemons: 2024-03-29T08:43:42.260 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:43:42.260 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi037.mmwgxe{-1:34440} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.37:6828/1248142005,v1:172.21.15.37:6829/1248142005] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:43:42.260 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi037.iifsmh{-1:34444} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.37:6826/2060882959,v1:172.21.15.37:6827/2060882959] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:43:42.260 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi148.sqteih{-1:44395} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.148:6824/4078652242,v1:172.21.15.148:6825/4078652242] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:43:42.260 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi148.qukpjh{-1:45007} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.148:6826/2394751105,v1:172.21.15.148:6827/2394751105] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:43:42.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:42.259+0000 7f809d600700 1 -- 172.21.15.37:0/2485841523 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f807c06cc20 msgr2=0x7f807c06f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:43:42.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:42.259+0000 7f809d600700 1 --2- 172.21.15.37:0/2485841523 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f807c06cc20 0x7f807c06f0d0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f8084004f90 tx=0x7f8084009e30 comp rx=0 tx=0).stop 2024-03-29T08:43:42.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:42.259+0000 7f809d600700 1 -- 172.21.15.37:0/2485841523 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f80980fde80 msgr2=0x7f80981138d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:43:42.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:42.259+0000 7f809d600700 1 --2- 172.21.15.37:0/2485841523 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f80980fde80 0x7f80981138d0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f8088002a50 tx=0x7f80880046b0 comp rx=0 tx=0).stop 2024-03-29T08:43:42.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:42.259+0000 7f809d600700 1 -- 172.21.15.37:0/2485841523 shutdown_connections 2024-03-29T08:43:42.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:42.259+0000 7f809d600700 1 --2- 172.21.15.37:0/2485841523 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f807c06cc20 0x7f807c06f0d0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:43:42.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:42.259+0000 7f809d600700 1 --2- 172.21.15.37:0/2485841523 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f80980fde80 0x7f80981138d0 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:43:42.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:42.259+0000 7f809d600700 1 --2- 172.21.15.37:0/2485841523 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f80980fe820 0x7f8098113e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:43:42.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:42.259+0000 7f809d600700 1 -- 172.21.15.37:0/2485841523 >> 172.21.15.37:0/2485841523 conn(0x7f80980f78f0 msgr2=0x7f80980ff2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:43:42.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:42.260+0000 7f809d600700 1 -- 172.21.15.37:0/2485841523 shutdown_connections 2024-03-29T08:43:42.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:42.260+0000 7f809d600700 1 -- 172.21.15.37:0/2485841523 wait complete. 2024-03-29T08:43:42.262 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 30 2024-03-29T08:43:42.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:42.398+0000 7f6df3e00700 1 -- 172.21.15.37:0/889370233 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6df4105800 msgr2=0x7f6df40fee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:43:42.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:42.398+0000 7f6df3e00700 1 --2- 172.21.15.37:0/889370233 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6df4105800 0x7f6df40fee70 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f6de0004940 tx=0x7f6de0009e30 comp rx=0 tx=0).stop 2024-03-29T08:43:42.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:42.398+0000 7f6df3e00700 1 -- 172.21.15.37:0/889370233 shutdown_connections 2024-03-29T08:43:42.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:42.398+0000 7f6df3e00700 1 --2- 172.21.15.37:0/889370233 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6df4105800 0x7f6df40fee70 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:43:42.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:42.398+0000 7f6df3e00700 1 --2- 172.21.15.37:0/889370233 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6df4104e60 0x7f6df4105230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:43:42.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:42.398+0000 7f6df3e00700 1 -- 172.21.15.37:0/889370233 >> 172.21.15.37:0/889370233 conn(0x7f6df40fab10 msgr2=0x7f6df40fcef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:43:42.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:42.399+0000 7f6df3e00700 1 -- 172.21.15.37:0/889370233 shutdown_connections 2024-03-29T08:43:42.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:42.399+0000 7f6df3e00700 1 -- 172.21.15.37:0/889370233 wait complete. 2024-03-29T08:43:42.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:42.400+0000 7f6df3e00700 1 Processor -- start 2024-03-29T08:43:42.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:42.400+0000 7f6df3e00700 1 -- start start 2024-03-29T08:43:42.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:42.400+0000 7f6df3e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6df4104e60 0x7f6df4115ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:43:42.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:42.400+0000 7f6df3e00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6df4105800 0x7f6df4116020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:43:42.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:42.401+0000 7f6df3e00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6df4116670 con 0x7f6df4104e60 2024-03-29T08:43:42.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:42.401+0000 7f6df3e00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6df410fa60 con 0x7f6df4105800 2024-03-29T08:43:42.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:42.401+0000 7f6df2a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6df4104e60 0x7f6df4115ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:43:42.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:42.401+0000 7f6df2a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6df4104e60 0x7f6df4115ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41472/0 (socket says 172.21.15.37:41472) 2024-03-29T08:43:42.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:42.401+0000 7f6df2a00700 1 -- 172.21.15.37:0/1361005533 learned_addr learned my addr 172.21.15.37:0/1361005533 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:43:42.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:42.401+0000 7f6df2000700 1 --2- 172.21.15.37:0/1361005533 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6df4105800 0x7f6df4116020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:43:42.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:42.401+0000 7f6df2a00700 1 -- 172.21.15.37:0/1361005533 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6df4105800 msgr2=0x7f6df4116020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:43:42.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:42.401+0000 7f6df2a00700 1 --2- 172.21.15.37:0/1361005533 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6df4105800 0x7f6df4116020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:43:42.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:42.401+0000 7f6df2a00700 1 -- 172.21.15.37:0/1361005533 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6de0005040 con 0x7f6df4104e60 2024-03-29T08:43:42.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:42.401+0000 7f6df2a00700 1 --2- 172.21.15.37:0/1361005533 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6df4104e60 0x7f6df4115ae0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f6de4002a50 tx=0x7f6de40046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:43:42.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:42.402+0000 7f6deaa00700 1 -- 172.21.15.37:0/1361005533 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6de4007cc0 con 0x7f6df4104e60 2024-03-29T08:43:42.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:42.402+0000 7f6deaa00700 1 -- 172.21.15.37:0/1361005533 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f6de400f040 con 0x7f6df4104e60 2024-03-29T08:43:42.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:42.402+0000 7f6df3e00700 1 -- 172.21.15.37:0/1361005533 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6df410fd40 con 0x7f6df4104e60 2024-03-29T08:43:42.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:42.402+0000 7f6deaa00700 1 -- 172.21.15.37:0/1361005533 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6de4009540 con 0x7f6df4104e60 2024-03-29T08:43:42.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:42.402+0000 7f6df3e00700 1 -- 172.21.15.37:0/1361005533 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6df4110300 con 0x7f6df4104e60 2024-03-29T08:43:42.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:42.403+0000 7f6df3e00700 1 -- 172.21.15.37:0/1361005533 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6df41005b0 con 0x7f6df4104e60 2024-03-29T08:43:42.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:42.404+0000 7f6deaa00700 1 -- 172.21.15.37:0/1361005533 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f6de40096a0 con 0x7f6df4104e60 2024-03-29T08:43:42.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:42.405+0000 7f6deaa00700 1 --2- 172.21.15.37:0/1361005533 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f6dd806cc20 0x7f6dd806f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:43:42.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:42.405+0000 7f6deaa00700 1 -- 172.21.15.37:0/1361005533 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f6de408bee0 con 0x7f6df4104e60 2024-03-29T08:43:42.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:42.405+0000 7f6df2000700 1 --2- 172.21.15.37:0/1361005533 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f6dd806cc20 0x7f6dd806f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:43:42.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:42.406+0000 7f6df2000700 1 --2- 172.21.15.37:0/1361005533 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f6dd806cc20 0x7f6dd806f0d0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f6df4110c20 tx=0x7f6de0009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:43:42.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:42.409+0000 7f6deaa00700 1 -- 172.21.15.37:0/1361005533 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f6de40598b0 con 0x7f6df4104e60 2024-03-29T08:43:42.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:42.642+0000 7f6df3e00700 1 -- 172.21.15.37:0/1361005533 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6df410bea0 con 0x7f6dd806cc20 2024-03-29T08:43:42.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:42.644+0000 7f6deaa00700 1 -- 172.21.15.37:0/1361005533 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f6de0008d60 con 0x7f6dd806cc20 2024-03-29T08:43:42.643 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-29T08:43:42.643 INFO:teuthology.orchestra.run.smithi037.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59", 2024-03-29T08:43:42.643 INFO:teuthology.orchestra.run.smithi037.stdout: "in_progress": true, 2024-03-29T08:43:42.643 INFO:teuthology.orchestra.run.smithi037.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-29T08:43:42.644 INFO:teuthology.orchestra.run.smithi037.stdout: "services_complete": [ 2024-03-29T08:43:42.644 INFO:teuthology.orchestra.run.smithi037.stdout: "crash", 2024-03-29T08:43:42.644 INFO:teuthology.orchestra.run.smithi037.stdout: "osd", 2024-03-29T08:43:42.644 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr", 2024-03-29T08:43:42.644 INFO:teuthology.orchestra.run.smithi037.stdout: "mds", 2024-03-29T08:43:42.644 INFO:teuthology.orchestra.run.smithi037.stdout: "mon" 2024-03-29T08:43:42.644 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-29T08:43:42.644 INFO:teuthology.orchestra.run.smithi037.stdout: "progress": "16/21 daemons upgraded", 2024-03-29T08:43:42.644 INFO:teuthology.orchestra.run.smithi037.stdout: "message": "Currently upgrading mds daemons", 2024-03-29T08:43:42.644 INFO:teuthology.orchestra.run.smithi037.stdout: "is_paused": false 2024-03-29T08:43:42.644 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-29T08:43:42.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:42.646+0000 7f6df3e00700 1 -- 172.21.15.37:0/1361005533 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f6dd806cc20 msgr2=0x7f6dd806f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:43:42.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:42.646+0000 7f6df3e00700 1 --2- 172.21.15.37:0/1361005533 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f6dd806cc20 0x7f6dd806f0d0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f6df4110c20 tx=0x7f6de0009e30 comp rx=0 tx=0).stop 2024-03-29T08:43:42.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:42.646+0000 7f6df3e00700 1 -- 172.21.15.37:0/1361005533 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6df4104e60 msgr2=0x7f6df4115ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:43:42.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:42.646+0000 7f6df3e00700 1 --2- 172.21.15.37:0/1361005533 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6df4104e60 0x7f6df4115ae0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f6de4002a50 tx=0x7f6de40046b0 comp rx=0 tx=0).stop 2024-03-29T08:43:42.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:42.646+0000 7f6df3e00700 1 -- 172.21.15.37:0/1361005533 shutdown_connections 2024-03-29T08:43:42.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:42.646+0000 7f6df3e00700 1 --2- 172.21.15.37:0/1361005533 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f6dd806cc20 0x7f6dd806f0d0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:43:42.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:42.646+0000 7f6df3e00700 1 --2- 172.21.15.37:0/1361005533 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6df4104e60 0x7f6df4115ae0 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:43:42.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:42.646+0000 7f6df3e00700 1 --2- 172.21.15.37:0/1361005533 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6df4105800 0x7f6df4116020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:43:42.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:42.646+0000 7f6df3e00700 1 -- 172.21.15.37:0/1361005533 >> 172.21.15.37:0/1361005533 conn(0x7f6df40fab10 msgr2=0x7f6df40fca50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:43:42.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:42.646+0000 7f6df3e00700 1 -- 172.21.15.37:0/1361005533 shutdown_connections 2024-03-29T08:43:42.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:42.646+0000 7f6df3e00700 1 -- 172.21.15.37:0/1361005533 wait complete. 2024-03-29T08:43:42.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:42.784+0000 7f4200a00700 1 -- 172.21.15.37:0/675869220 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f41fc1037a0 msgr2=0x7f41fc0fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:43:42.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:42.784+0000 7f4200a00700 1 --2- 172.21.15.37:0/675869220 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f41fc1037a0 0x7f41fc0fce10 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f41e8004940 tx=0x7f41e8009e30 comp rx=0 tx=0).stop 2024-03-29T08:43:42.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:42.784+0000 7f4200a00700 1 -- 172.21.15.37:0/675869220 shutdown_connections 2024-03-29T08:43:42.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:42.784+0000 7f4200a00700 1 --2- 172.21.15.37:0/675869220 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f41fc1037a0 0x7f41fc0fce10 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:43:42.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:42.784+0000 7f4200a00700 1 --2- 172.21.15.37:0/675869220 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f41fc102e00 0x7f41fc1031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:43:42.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:42.784+0000 7f4200a00700 1 -- 172.21.15.37:0/675869220 >> 172.21.15.37:0/675869220 conn(0x7f41fc0f89b0 msgr2=0x7f41fc0fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:43:42.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:42.784+0000 7f4200a00700 1 -- 172.21.15.37:0/675869220 shutdown_connections 2024-03-29T08:43:42.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:42.784+0000 7f4200a00700 1 -- 172.21.15.37:0/675869220 wait complete. 2024-03-29T08:43:42.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:42.785+0000 7f4200a00700 1 Processor -- start 2024-03-29T08:43:42.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:42.785+0000 7f4200a00700 1 -- start start 2024-03-29T08:43:42.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:42.785+0000 7f4200a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f41fc102e00 0x7f41fc115da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:43:42.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:42.786+0000 7f4200a00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f41fc1037a0 0x7f41fc1162e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:43:42.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:42.786+0000 7f4200a00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41fc116930 con 0x7f41fc102e00 2024-03-29T08:43:42.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:42.786+0000 7f4200a00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41fc10fd00 con 0x7f41fc1037a0 2024-03-29T08:43:42.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:42.786+0000 7f41f9e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f41fc102e00 0x7f41fc115da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:43:42.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:42.786+0000 7f41f9e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f41fc102e00 0x7f41fc115da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41502/0 (socket says 172.21.15.37:41502) 2024-03-29T08:43:42.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:42.786+0000 7f41f9e00700 1 -- 172.21.15.37:0/1969106374 learned_addr learned my addr 172.21.15.37:0/1969106374 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:43:42.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:42.786+0000 7f41f9400700 1 --2- 172.21.15.37:0/1969106374 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f41fc1037a0 0x7f41fc1162e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:43:42.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:42.786+0000 7f41f9e00700 1 -- 172.21.15.37:0/1969106374 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f41fc1037a0 msgr2=0x7f41fc1162e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:43:42.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:42.786+0000 7f41f9e00700 1 --2- 172.21.15.37:0/1969106374 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f41fc1037a0 0x7f41fc1162e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:43:42.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:42.786+0000 7f41f9e00700 1 -- 172.21.15.37:0/1969106374 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41e8005040 con 0x7f41fc102e00 2024-03-29T08:43:42.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:42.786+0000 7f41f9e00700 1 --2- 172.21.15.37:0/1969106374 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f41fc102e00 0x7f41fc115da0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f41ec00c830 tx=0x7f41ec004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:43:42.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:42.787+0000 7f41f2000700 1 -- 172.21.15.37:0/1969106374 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f41ec00a760 con 0x7f41fc102e00 2024-03-29T08:43:42.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:42.787+0000 7f41f2000700 1 -- 172.21.15.37:0/1969106374 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f41ec00ad70 con 0x7f41fc102e00 2024-03-29T08:43:42.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:42.787+0000 7f4200a00700 1 -- 172.21.15.37:0/1969106374 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41fc10ffe0 con 0x7f41fc102e00 2024-03-29T08:43:42.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:42.787+0000 7f41f2000700 1 -- 172.21.15.37:0/1969106374 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f41ec0113a0 con 0x7f41fc102e00 2024-03-29T08:43:42.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:42.787+0000 7f4200a00700 1 -- 172.21.15.37:0/1969106374 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41fc110520 con 0x7f41fc102e00 2024-03-29T08:43:42.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:42.788+0000 7f4200a00700 1 -- 172.21.15.37:0/1969106374 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41fc0fe550 con 0x7f41fc102e00 2024-03-29T08:43:42.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:42.789+0000 7f41f2000700 1 -- 172.21.15.37:0/1969106374 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f41ec012400 con 0x7f41fc102e00 2024-03-29T08:43:42.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:42.790+0000 7f41f2000700 1 --2- 172.21.15.37:0/1969106374 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f41e006cc30 0x7f41e006f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:43:42.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:42.790+0000 7f41f2000700 1 -- 172.21.15.37:0/1969106374 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f41ec08c120 con 0x7f41fc102e00 2024-03-29T08:43:42.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:42.791+0000 7f41f9400700 1 --2- 172.21.15.37:0/1969106374 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f41e006cc30 0x7f41e006f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:43:42.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:42.791+0000 7f41f9400700 1 --2- 172.21.15.37:0/1969106374 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f41e006cc30 0x7f41e006f0e0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f41e800b020 tx=0x7f41e8009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:43:42.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:42.794+0000 7f41f2000700 1 -- 172.21.15.37:0/1969106374 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f41ec059b70 con 0x7f41fc102e00 2024-03-29T08:43:42.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:43:42 smithi037 ceph-mon[142325]: from='client.35770 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:43:42.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:43:42 smithi037 ceph-mon[142325]: pgmap v985: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:43:42.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:43:42 smithi037 ceph-mon[142325]: from='client.35774 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:43:42.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:43:42 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/3571708384' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:43:42.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:43:42 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/2485841523' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T08:43:43.058 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:43:42 smithi148 ceph-mon[106692]: from='client.35770 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:43:43.058 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:43:42 smithi148 ceph-mon[106692]: pgmap v985: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:43:43.059 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:43:42 smithi148 ceph-mon[106692]: from='client.35774 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:43:43.059 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:43:42 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/3571708384' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:43:43.059 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:43:42 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/2485841523' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T08:43:43.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:43.094+0000 7f4200a00700 1 -- 172.21.15.37:0/1969106374 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f41fc05f5f0 con 0x7f41fc102e00 2024-03-29T08:43:43.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:43.095+0000 7f41f2000700 1 -- 172.21.15.37:0/1969106374 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+3583 (secure 0 0 0) 0x7f41ec059700 con 0x7f41fc102e00 2024-03-29T08:43:43.095 INFO:teuthology.orchestra.run.smithi037.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem is offline 2024-03-29T08:43:43.095 INFO:teuthology.orchestra.run.smithi037.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-03-29T08:43:43.095 INFO:teuthology.orchestra.run.smithi037.stdout: host smithi148 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/mon.smithi148/config 2024-03-29T08:43:43.095 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/podman: timeout after 895 seconds 2024-03-29T08:43:43.095 INFO:teuthology.orchestra.run.smithi037.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59 -e NODE_NAME=smithi148 -e CEPH_USE_RANDOM_NONCE=1 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4:/var/run/ceph:z -v /var/log/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4:/var/log/ceph:z -v /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpkx2rz2d4:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59 inventory --format=json-pretty --filter-for-batch 2024-03-29T08:43:43.095 INFO:teuthology.orchestra.run.smithi037.stdout:Traceback (most recent call last): 2024-03-29T08:43:43.095 INFO:teuthology.orchestra.run.smithi037.stdout: File "/usr/lib64/python3.6/runpy.py", line 193, in _run_module_as_main 2024-03-29T08:43:43.095 INFO:teuthology.orchestra.run.smithi037.stdout: "__main__", mod_spec) 2024-03-29T08:43:43.095 INFO:teuthology.orchestra.run.smithi037.stdout: File "/usr/lib64/python3.6/runpy.py", line 85, in _run_code 2024-03-29T08:43:43.095 INFO:teuthology.orchestra.run.smithi037.stdout: exec(code, run_globals) 2024-03-29T08:43:43.095 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 10889, in 2024-03-29T08:43:43.095 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 10877, in main 2024-03-29T08:43:43.095 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 2576, in _infer_config 2024-03-29T08:43:43.096 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 2492, in _infer_fsid 2024-03-29T08:43:43.096 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 2604, in _infer_image 2024-03-29T08:43:43.096 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 2479, in _validate_fsid 2024-03-29T08:43:43.096 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 7145, in command_ceph_volume 2024-03-29T08:43:43.096 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 2267, in call_throws 2024-03-29T08:43:43.096 INFO:teuthology.orchestra.run.smithi037.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59 -e NODE_NAME=smithi148 -e CEPH_USE_RANDOM_NONCE=1 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4:/var/run/ceph:z -v /var/log/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4:/var/log/ceph:z -v /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpkx2rz2d4:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59 inventory --format=json-pretty --filter-for-batch 2024-03-29T08:43:43.096 INFO:teuthology.orchestra.run.smithi037.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-03-29T08:43:43.096 INFO:teuthology.orchestra.run.smithi037.stdout: fs cephfs is degraded 2024-03-29T08:43:43.096 INFO:teuthology.orchestra.run.smithi037.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-03-29T08:43:43.096 INFO:teuthology.orchestra.run.smithi037.stdout: fs cephfs is offline because no MDS is active for it. 2024-03-29T08:43:43.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:43.097+0000 7f4200a00700 1 -- 172.21.15.37:0/1969106374 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f41e006cc30 msgr2=0x7f41e006f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:43:43.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:43.097+0000 7f4200a00700 1 --2- 172.21.15.37:0/1969106374 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f41e006cc30 0x7f41e006f0e0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f41e800b020 tx=0x7f41e8009e30 comp rx=0 tx=0).stop 2024-03-29T08:43:43.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:43.097+0000 7f4200a00700 1 -- 172.21.15.37:0/1969106374 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f41fc102e00 msgr2=0x7f41fc115da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:43:43.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:43.097+0000 7f4200a00700 1 --2- 172.21.15.37:0/1969106374 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f41fc102e00 0x7f41fc115da0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f41ec00c830 tx=0x7f41ec004470 comp rx=0 tx=0).stop 2024-03-29T08:43:43.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:43.097+0000 7f4200a00700 1 -- 172.21.15.37:0/1969106374 shutdown_connections 2024-03-29T08:43:43.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:43.098+0000 7f4200a00700 1 --2- 172.21.15.37:0/1969106374 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f41e006cc30 0x7f41e006f0e0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:43:43.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:43.098+0000 7f4200a00700 1 --2- 172.21.15.37:0/1969106374 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f41fc102e00 0x7f41fc115da0 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:43:43.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:43.098+0000 7f4200a00700 1 --2- 172.21.15.37:0/1969106374 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f41fc1037a0 0x7f41fc1162e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:43:43.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:43.098+0000 7f4200a00700 1 -- 172.21.15.37:0/1969106374 >> 172.21.15.37:0/1969106374 conn(0x7f41fc0f89b0 msgr2=0x7f41fc0f9a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:43:43.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:43.098+0000 7f4200a00700 1 -- 172.21.15.37:0/1969106374 shutdown_connections 2024-03-29T08:43:43.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:43:43.098+0000 7f4200a00700 1 -- 172.21.15.37:0/1969106374 wait complete. 2024-03-29T08:43:43.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:43:43 smithi037 ceph-mon[142325]: from='client.35786 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:43:43.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:43:43 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/1969106374' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T08:43:44.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:43:43 smithi148 ceph-mon[106692]: from='client.35786 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:43:44.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:43:43 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/1969106374' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T08:43:44.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:43:44 smithi037 ceph-mon[142325]: pgmap v986: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:43:45.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:43:44 smithi148 ceph-mon[106692]: pgmap v986: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:43:46.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:43:46 smithi037 ceph-mon[142325]: pgmap v987: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:43:47.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:43:46 smithi148 ceph-mon[106692]: pgmap v987: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:43:48.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:43:48 smithi037 ceph-mon[142325]: pgmap v988: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:43:49.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:43:48 smithi148 ceph-mon[106692]: pgmap v988: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:43:50.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:43:50 smithi037 ceph-mon[142325]: pgmap v989: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:43:51.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:43:50 smithi148 ceph-mon[106692]: pgmap v989: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:43:52.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:43:52 smithi037 ceph-mon[142325]: pgmap v990: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:43:53.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:43:52 smithi148 ceph-mon[106692]: pgmap v990: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:43:54.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:43:54 smithi037 ceph-mon[142325]: pgmap v991: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:43:55.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:43:54 smithi148 ceph-mon[106692]: pgmap v991: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:43:55.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:43:55 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:43:56.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:43:55 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:43:56.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:43:56 smithi037 ceph-mon[142325]: pgmap v992: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:43:57.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:43:56 smithi148 ceph-mon[106692]: pgmap v992: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:43:58.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:43:58 smithi037 ceph-mon[142325]: pgmap v993: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:43:59.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:43:58 smithi148 ceph-mon[106692]: pgmap v993: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:44:00.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:44:00 smithi037 ceph-mon[142325]: pgmap v994: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:44:01.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:44:00 smithi148 ceph-mon[106692]: pgmap v994: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:44:02.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:44:02 smithi037 ceph-mon[142325]: pgmap v995: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:44:03.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:44:02 smithi148 ceph-mon[106692]: pgmap v995: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:44:04.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:44:04 smithi037 ceph-mon[142325]: pgmap v996: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:44:05.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:44:04 smithi148 ceph-mon[106692]: pgmap v996: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:44:06.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:44:06 smithi037 ceph-mon[142325]: pgmap v997: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:44:07.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:44:06 smithi148 ceph-mon[106692]: pgmap v997: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:44:08.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:44:08 smithi037 ceph-mon[142325]: pgmap v998: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:44:09.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:44:08 smithi148 ceph-mon[106692]: pgmap v998: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:44:10.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:44:10 smithi148 ceph-mon[106692]: pgmap v999: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:44:10.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:44:10 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:44:10.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:44:10 smithi037 ceph-mon[142325]: pgmap v999: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:44:10.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:44:10 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:44:12.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:44:12 smithi037 ceph-mon[142325]: pgmap v1000: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:44:13.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:44:12 smithi148 ceph-mon[106692]: pgmap v1000: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:44:13.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:13.252+0000 7ff14e200700 1 -- 172.21.15.37:0/678319319 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff148102e00 msgr2=0x7ff1481031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:44:13.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:13.252+0000 7ff14e200700 1 --2- 172.21.15.37:0/678319319 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff148102e00 0x7ff1481031d0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7ff138004940 tx=0x7ff138009e30 comp rx=0 tx=0).stop 2024-03-29T08:44:13.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:13.252+0000 7ff14e200700 1 -- 172.21.15.37:0/678319319 shutdown_connections 2024-03-29T08:44:13.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:13.252+0000 7ff14e200700 1 --2- 172.21.15.37:0/678319319 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff1481037a0 0x7ff1480fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:44:13.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:13.252+0000 7ff14e200700 1 --2- 172.21.15.37:0/678319319 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff148102e00 0x7ff1481031d0 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:44:13.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:13.252+0000 7ff14e200700 1 -- 172.21.15.37:0/678319319 >> 172.21.15.37:0/678319319 conn(0x7ff1480f89b0 msgr2=0x7ff1480fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:44:13.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:13.252+0000 7ff14e200700 1 -- 172.21.15.37:0/678319319 shutdown_connections 2024-03-29T08:44:13.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:13.252+0000 7ff14e200700 1 -- 172.21.15.37:0/678319319 wait complete. 2024-03-29T08:44:13.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:13.254+0000 7ff14e200700 1 Processor -- start 2024-03-29T08:44:13.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:13.254+0000 7ff14e200700 1 -- start start 2024-03-29T08:44:13.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:13.254+0000 7ff14e200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff148102e00 0x7ff148113a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:44:13.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:13.254+0000 7ff14e200700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff1481037a0 0x7ff148113f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:44:13.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:13.254+0000 7ff14e200700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1481145d0 con 0x7ff148102e00 2024-03-29T08:44:13.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:13.254+0000 7ff14e200700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff14810d9c0 con 0x7ff1481037a0 2024-03-29T08:44:13.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:13.255+0000 7ff147400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff148102e00 0x7ff148113a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:44:13.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:13.255+0000 7ff147400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff148102e00 0x7ff148113a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41996/0 (socket says 172.21.15.37:41996) 2024-03-29T08:44:13.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:13.255+0000 7ff147400700 1 -- 172.21.15.37:0/681384878 learned_addr learned my addr 172.21.15.37:0/681384878 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:44:13.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:13.255+0000 7ff146a00700 1 --2- 172.21.15.37:0/681384878 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff1481037a0 0x7ff148113f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:44:13.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:13.255+0000 7ff147400700 1 -- 172.21.15.37:0/681384878 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff1481037a0 msgr2=0x7ff148113f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:44:13.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:13.255+0000 7ff147400700 1 --2- 172.21.15.37:0/681384878 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff1481037a0 0x7ff148113f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:44:13.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:13.255+0000 7ff147400700 1 -- 172.21.15.37:0/681384878 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff138005040 con 0x7ff148102e00 2024-03-29T08:44:13.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:13.255+0000 7ff147400700 1 --2- 172.21.15.37:0/681384878 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff148102e00 0x7ff148113a40 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7ff138007f50 tx=0x7ff138009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:44:13.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:13.256+0000 7ff13f400700 1 -- 172.21.15.37:0/681384878 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff138021070 con 0x7ff148102e00 2024-03-29T08:44:13.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:13.256+0000 7ff13f400700 1 -- 172.21.15.37:0/681384878 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ff1380093b0 con 0x7ff148102e00 2024-03-29T08:44:13.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:13.256+0000 7ff13f400700 1 -- 172.21.15.37:0/681384878 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff13800e070 con 0x7ff148102e00 2024-03-29T08:44:13.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:13.256+0000 7ff14e200700 1 -- 172.21.15.37:0/681384878 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff14810dc40 con 0x7ff148102e00 2024-03-29T08:44:13.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:13.256+0000 7ff14e200700 1 -- 172.21.15.37:0/681384878 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff14810e1a0 con 0x7ff148102e00 2024-03-29T08:44:13.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:13.257+0000 7ff14e200700 1 -- 172.21.15.37:0/681384878 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1480fe550 con 0x7ff148102e00 2024-03-29T08:44:13.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:13.258+0000 7ff13f400700 1 -- 172.21.15.37:0/681384878 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7ff13801c400 con 0x7ff148102e00 2024-03-29T08:44:13.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:13.259+0000 7ff13f400700 1 --2- 172.21.15.37:0/681384878 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7ff12c06cc30 0x7ff12c06f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:44:13.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:13.259+0000 7ff13f400700 1 -- 172.21.15.37:0/681384878 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7ff13808d220 con 0x7ff148102e00 2024-03-29T08:44:13.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:13.259+0000 7ff146a00700 1 --2- 172.21.15.37:0/681384878 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7ff12c06cc30 0x7ff12c06f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:44:13.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:13.260+0000 7ff146a00700 1 --2- 172.21.15.37:0/681384878 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7ff12c06cc30 0x7ff12c06f0e0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7ff14810eb80 tx=0x7ff13400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:44:13.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:13.264+0000 7ff13f400700 1 -- 172.21.15.37:0/681384878 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7ff13805ac70 con 0x7ff148102e00 2024-03-29T08:44:13.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:13.495+0000 7ff14e200700 1 -- 172.21.15.37:0/681384878 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff14804bed0 con 0x7ff12c06cc30 2024-03-29T08:44:13.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:13.497+0000 7ff13f400700 1 -- 172.21.15.37:0/681384878 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7ff14804bed0 con 0x7ff12c06cc30 2024-03-29T08:44:13.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:13.499+0000 7ff14e200700 1 -- 172.21.15.37:0/681384878 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7ff12c06cc30 msgr2=0x7ff12c06f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:44:13.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:13.499+0000 7ff14e200700 1 --2- 172.21.15.37:0/681384878 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7ff12c06cc30 0x7ff12c06f0e0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7ff14810eb80 tx=0x7ff13400b040 comp rx=0 tx=0).stop 2024-03-29T08:44:13.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:13.499+0000 7ff14e200700 1 -- 172.21.15.37:0/681384878 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff148102e00 msgr2=0x7ff148113a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:44:13.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:13.499+0000 7ff14e200700 1 --2- 172.21.15.37:0/681384878 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff148102e00 0x7ff148113a40 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7ff138007f50 tx=0x7ff138009e30 comp rx=0 tx=0).stop 2024-03-29T08:44:13.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:13.499+0000 7ff14e200700 1 -- 172.21.15.37:0/681384878 shutdown_connections 2024-03-29T08:44:13.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:13.499+0000 7ff14e200700 1 --2- 172.21.15.37:0/681384878 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7ff12c06cc30 0x7ff12c06f0e0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:44:13.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:13.499+0000 7ff14e200700 1 --2- 172.21.15.37:0/681384878 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff148102e00 0x7ff148113a40 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:44:13.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:13.499+0000 7ff14e200700 1 --2- 172.21.15.37:0/681384878 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff1481037a0 0x7ff148113f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:44:13.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:13.499+0000 7ff14e200700 1 -- 172.21.15.37:0/681384878 >> 172.21.15.37:0/681384878 conn(0x7ff1480f89b0 msgr2=0x7ff1480f99f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:44:13.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:13.500+0000 7ff14e200700 1 -- 172.21.15.37:0/681384878 shutdown_connections 2024-03-29T08:44:13.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:13.500+0000 7ff14e200700 1 -- 172.21.15.37:0/681384878 wait complete. 2024-03-29T08:44:13.515 INFO:teuthology.orchestra.run.smithi037.stdout:true 2024-03-29T08:44:13.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:13.639+0000 7f3d08c00700 1 -- 172.21.15.37:0/4050921485 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3d04102e00 msgr2=0x7f3d041031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:44:13.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:13.639+0000 7f3d08c00700 1 --2- 172.21.15.37:0/4050921485 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3d04102e00 0x7f3d041031d0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f3cf4004940 tx=0x7f3cf4009e30 comp rx=0 tx=0).stop 2024-03-29T08:44:13.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:13.639+0000 7f3d08c00700 1 -- 172.21.15.37:0/4050921485 shutdown_connections 2024-03-29T08:44:13.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:13.639+0000 7f3d08c00700 1 --2- 172.21.15.37:0/4050921485 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3d041037a0 0x7f3d040fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:44:13.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:13.639+0000 7f3d08c00700 1 --2- 172.21.15.37:0/4050921485 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3d04102e00 0x7f3d041031d0 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:44:13.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:13.639+0000 7f3d08c00700 1 -- 172.21.15.37:0/4050921485 >> 172.21.15.37:0/4050921485 conn(0x7f3d040f89b0 msgr2=0x7f3d040fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:44:13.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:13.639+0000 7f3d08c00700 1 -- 172.21.15.37:0/4050921485 shutdown_connections 2024-03-29T08:44:13.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:13.639+0000 7f3d08c00700 1 -- 172.21.15.37:0/4050921485 wait complete. 2024-03-29T08:44:13.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:13.640+0000 7f3d08c00700 1 Processor -- start 2024-03-29T08:44:13.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:13.640+0000 7f3d08c00700 1 -- start start 2024-03-29T08:44:13.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:13.640+0000 7f3d08c00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3d04102e00 0x7f3d04113a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:44:13.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:13.640+0000 7f3d08c00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3d041037a0 0x7f3d04113f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:44:13.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:13.640+0000 7f3d08c00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d041145d0 con 0x7f3d041037a0 2024-03-29T08:44:13.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:13.640+0000 7f3d08c00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d0410d9c0 con 0x7f3d04102e00 2024-03-29T08:44:13.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:13.641+0000 7f3d01400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3d041037a0 0x7f3d04113f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:44:13.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:13.641+0000 7f3d01400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3d041037a0 0x7f3d04113f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42020/0 (socket says 172.21.15.37:42020) 2024-03-29T08:44:13.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:13.641+0000 7f3d01400700 1 -- 172.21.15.37:0/1274298539 learned_addr learned my addr 172.21.15.37:0/1274298539 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:44:13.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:13.641+0000 7f3d01e00700 1 --2- 172.21.15.37:0/1274298539 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3d04102e00 0x7f3d04113a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:44:13.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:13.641+0000 7f3d01400700 1 -- 172.21.15.37:0/1274298539 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3d04102e00 msgr2=0x7f3d04113a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:44:13.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:13.641+0000 7f3d01400700 1 --2- 172.21.15.37:0/1274298539 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3d04102e00 0x7f3d04113a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:44:13.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:13.641+0000 7f3d01400700 1 -- 172.21.15.37:0/1274298539 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cf4005040 con 0x7f3d041037a0 2024-03-29T08:44:13.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:13.641+0000 7f3d01400700 1 --2- 172.21.15.37:0/1274298539 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3d041037a0 0x7f3d04113f80 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f3cf000eb40 tx=0x7f3cf000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:44:13.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:13.642+0000 7f3cfa000700 1 -- 172.21.15.37:0/1274298539 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3cf0007cc0 con 0x7f3d041037a0 2024-03-29T08:44:13.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:13.642+0000 7f3cfa000700 1 -- 172.21.15.37:0/1274298539 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f3cf000f040 con 0x7f3d041037a0 2024-03-29T08:44:13.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:13.642+0000 7f3d08c00700 1 -- 172.21.15.37:0/1274298539 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d0410dc40 con 0x7f3d041037a0 2024-03-29T08:44:13.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:13.642+0000 7f3cfa000700 1 -- 172.21.15.37:0/1274298539 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3cf0009570 con 0x7f3d041037a0 2024-03-29T08:44:13.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:13.642+0000 7f3d08c00700 1 -- 172.21.15.37:0/1274298539 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d0410e200 con 0x7f3d041037a0 2024-03-29T08:44:13.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:13.643+0000 7f3d08c00700 1 -- 172.21.15.37:0/1274298539 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d040fe550 con 0x7f3d041037a0 2024-03-29T08:44:13.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:13.644+0000 7f3cfa000700 1 -- 172.21.15.37:0/1274298539 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f3cf00096d0 con 0x7f3d041037a0 2024-03-29T08:44:13.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:13.645+0000 7f3cfa000700 1 --2- 172.21.15.37:0/1274298539 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f3ce806cc30 0x7f3ce806f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:44:13.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:13.645+0000 7f3cfa000700 1 -- 172.21.15.37:0/1274298539 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f3cf008bb90 con 0x7f3d041037a0 2024-03-29T08:44:13.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:13.645+0000 7f3d01e00700 1 --2- 172.21.15.37:0/1274298539 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f3ce806cc30 0x7f3ce806f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:44:13.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:13.646+0000 7f3d01e00700 1 --2- 172.21.15.37:0/1274298539 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f3ce806cc30 0x7f3ce806f0e0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f3cf400b0d0 tx=0x7f3cf4009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:44:13.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:13.650+0000 7f3cfa000700 1 -- 172.21.15.37:0/1274298539 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f3cf0059690 con 0x7f3d041037a0 2024-03-29T08:44:13.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:13.884+0000 7f3d08c00700 1 -- 172.21.15.37:0/1274298539 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3d04109dc0 con 0x7f3ce806cc30 2024-03-29T08:44:13.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:13.885+0000 7f3cfa000700 1 -- 172.21.15.37:0/1274298539 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f3d04109dc0 con 0x7f3ce806cc30 2024-03-29T08:44:13.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:13.887+0000 7f3d08c00700 1 -- 172.21.15.37:0/1274298539 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f3ce806cc30 msgr2=0x7f3ce806f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:44:13.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:13.887+0000 7f3d08c00700 1 --2- 172.21.15.37:0/1274298539 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f3ce806cc30 0x7f3ce806f0e0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f3cf400b0d0 tx=0x7f3cf4009e30 comp rx=0 tx=0).stop 2024-03-29T08:44:13.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:13.887+0000 7f3d08c00700 1 -- 172.21.15.37:0/1274298539 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3d041037a0 msgr2=0x7f3d04113f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:44:13.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:13.887+0000 7f3d08c00700 1 --2- 172.21.15.37:0/1274298539 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3d041037a0 0x7f3d04113f80 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f3cf000eb40 tx=0x7f3cf000b040 comp rx=0 tx=0).stop 2024-03-29T08:44:13.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:13.887+0000 7f3d08c00700 1 -- 172.21.15.37:0/1274298539 shutdown_connections 2024-03-29T08:44:13.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:13.887+0000 7f3d08c00700 1 --2- 172.21.15.37:0/1274298539 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f3ce806cc30 0x7f3ce806f0e0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:44:13.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:13.887+0000 7f3d08c00700 1 --2- 172.21.15.37:0/1274298539 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3d04102e00 0x7f3d04113a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:44:13.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:13.887+0000 7f3d08c00700 1 --2- 172.21.15.37:0/1274298539 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3d041037a0 0x7f3d04113f80 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:44:13.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:13.887+0000 7f3d08c00700 1 -- 172.21.15.37:0/1274298539 >> 172.21.15.37:0/1274298539 conn(0x7f3d040f89b0 msgr2=0x7f3d040f99f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:44:13.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:13.888+0000 7f3d08c00700 1 -- 172.21.15.37:0/1274298539 shutdown_connections 2024-03-29T08:44:13.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:13.888+0000 7f3d08c00700 1 -- 172.21.15.37:0/1274298539 wait complete. 2024-03-29T08:44:14.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:14.025+0000 7faa94a00700 1 -- 172.21.15.37:0/798136741 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faa900fde90 msgr2=0x7faa900fe260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:44:14.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:14.025+0000 7faa94a00700 1 --2- 172.21.15.37:0/798136741 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faa900fde90 0x7faa900fe260 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7faa80004940 tx=0x7faa80009e30 comp rx=0 tx=0).stop 2024-03-29T08:44:14.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:14.025+0000 7faa94a00700 1 -- 172.21.15.37:0/798136741 shutdown_connections 2024-03-29T08:44:14.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:14.025+0000 7faa94a00700 1 --2- 172.21.15.37:0/798136741 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faa900fe830 0x7faa90106f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:44:14.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:14.025+0000 7faa94a00700 1 --2- 172.21.15.37:0/798136741 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faa900fde90 0x7faa900fe260 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:44:14.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:14.025+0000 7faa94a00700 1 -- 172.21.15.37:0/798136741 >> 172.21.15.37:0/798136741 conn(0x7faa90072bc0 msgr2=0x7faa90072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:44:14.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:14.025+0000 7faa94a00700 1 -- 172.21.15.37:0/798136741 shutdown_connections 2024-03-29T08:44:14.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:14.026+0000 7faa94a00700 1 -- 172.21.15.37:0/798136741 wait complete. 2024-03-29T08:44:14.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:14.026+0000 7faa94a00700 1 Processor -- start 2024-03-29T08:44:14.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:14.026+0000 7faa94a00700 1 -- start start 2024-03-29T08:44:14.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:14.027+0000 7faa94a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faa900fde90 0x7faa90115c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:44:14.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:14.027+0000 7faa94a00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faa900fe830 0x7faa901161c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:44:14.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:14.027+0000 7faa94a00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa90116810 con 0x7faa900fde90 2024-03-29T08:44:14.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:14.027+0000 7faa94a00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa9010fbe0 con 0x7faa900fe830 2024-03-29T08:44:14.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:14.027+0000 7faa8de00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faa900fde90 0x7faa90115c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:44:14.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:14.027+0000 7faa8de00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faa900fde90 0x7faa90115c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42044/0 (socket says 172.21.15.37:42044) 2024-03-29T08:44:14.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:14.027+0000 7faa8de00700 1 -- 172.21.15.37:0/3352162227 learned_addr learned my addr 172.21.15.37:0/3352162227 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:44:14.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:14.027+0000 7faa8d400700 1 --2- 172.21.15.37:0/3352162227 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faa900fe830 0x7faa901161c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:44:14.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:14.027+0000 7faa8de00700 1 -- 172.21.15.37:0/3352162227 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faa900fe830 msgr2=0x7faa901161c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:44:14.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:14.027+0000 7faa8de00700 1 --2- 172.21.15.37:0/3352162227 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faa900fe830 0x7faa901161c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:44:14.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:14.027+0000 7faa8de00700 1 -- 172.21.15.37:0/3352162227 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa80005040 con 0x7faa900fde90 2024-03-29T08:44:14.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:14.028+0000 7faa8de00700 1 --2- 172.21.15.37:0/3352162227 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faa900fde90 0x7faa90115c80 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7faa80000c00 tx=0x7faa80009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:44:14.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:14.028+0000 7faa86000700 1 -- 172.21.15.37:0/3352162227 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faa8000e040 con 0x7faa900fde90 2024-03-29T08:44:14.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:14.028+0000 7faa86000700 1 -- 172.21.15.37:0/3352162227 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7faa8000bed0 con 0x7faa900fde90 2024-03-29T08:44:14.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:14.028+0000 7faa94a00700 1 -- 172.21.15.37:0/3352162227 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa9010fe60 con 0x7faa900fde90 2024-03-29T08:44:14.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:14.028+0000 7faa86000700 1 -- 172.21.15.37:0/3352162227 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faa8001c3f0 con 0x7faa900fde90 2024-03-29T08:44:14.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:14.028+0000 7faa94a00700 1 -- 172.21.15.37:0/3352162227 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa901103a0 con 0x7faa900fde90 2024-03-29T08:44:14.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:14.029+0000 7faa94a00700 1 -- 172.21.15.37:0/3352162227 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa90106a20 con 0x7faa900fde90 2024-03-29T08:44:14.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:14.030+0000 7faa86000700 1 -- 172.21.15.37:0/3352162227 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7faa80013420 con 0x7faa900fde90 2024-03-29T08:44:14.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:14.031+0000 7faa86000700 1 --2- 172.21.15.37:0/3352162227 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7faa7406cc30 0x7faa7406f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:44:14.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:14.031+0000 7faa86000700 1 -- 172.21.15.37:0/3352162227 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7faa8008d500 con 0x7faa900fde90 2024-03-29T08:44:14.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:14.031+0000 7faa8d400700 1 --2- 172.21.15.37:0/3352162227 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7faa7406cc30 0x7faa7406f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:44:14.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:14.032+0000 7faa8d400700 1 --2- 172.21.15.37:0/3352162227 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7faa7406cc30 0x7faa7406f0e0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7faa90111520 tx=0x7faa7c00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:44:14.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:14.035+0000 7faa86000700 1 -- 172.21.15.37:0/3352162227 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7faa8005af50 con 0x7faa900fde90 2024-03-29T08:44:14.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:14.257+0000 7faa94a00700 1 -- 172.21.15.37:0/3352162227 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7faa90109e30 con 0x7faa7406cc30 2024-03-29T08:44:14.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:14.267+0000 7faa86000700 1 -- 172.21.15.37:0/3352162227 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7faa90109e30 con 0x7faa7406cc30 2024-03-29T08:44:14.268 INFO:teuthology.orchestra.run.smithi037.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-29T08:44:14.268 INFO:teuthology.orchestra.run.smithi037.stdout:alertmanager.smithi037 smithi037 *:9093,9094 running (34m) 10m ago 47m 14.3M - 0.25.0 c8568f914cd2 2f9f9746ed0f 2024-03-29T08:44:14.268 INFO:teuthology.orchestra.run.smithi037.stdout:crash.smithi037 smithi037 running (30m) 10m ago 47m 6647k - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 178d416a3186 2024-03-29T08:44:14.268 INFO:teuthology.orchestra.run.smithi037.stdout:crash.smithi148 smithi148 running (30m) 10m ago 46m 6656k - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 23666989d58f 2024-03-29T08:44:14.269 INFO:teuthology.orchestra.run.smithi037.stdout:grafana.smithi037 smithi037 *:3000 running (33m) 10m ago 46m 74.5M - 9.4.7 954c08fa6188 6b976fe49952 2024-03-29T08:44:14.269 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi037.iifsmh smithi037 running (26m) 10m ago 40m 16.4M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 2d97f562cc5b 2024-03-29T08:44:14.269 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi037.mmwgxe smithi037 running (26m) 10m ago 40m 16.7M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff ec37037f46e8 2024-03-29T08:44:14.269 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi148.qukpjh smithi148 running (10m) 10m ago 40m 10.7M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 3257219bc046 2024-03-29T08:44:14.269 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi148.sqteih smithi148 running (26m) 10m ago 40m 16.3M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 44c15c77037a 2024-03-29T08:44:14.269 INFO:teuthology.orchestra.run.smithi037.stdout:mgr.smithi037.wjikii smithi037 *:8443,9283,8765 running (36m) 10m ago 48m 510M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff c5a1417b9417 2024-03-29T08:44:14.269 INFO:teuthology.orchestra.run.smithi037.stdout:mgr.smithi148.onhgvm smithi148 *:8443,9283,8765 running (35m) 10m ago 45m 456M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff fcfdbce062bd 2024-03-29T08:44:14.269 INFO:teuthology.orchestra.run.smithi037.stdout:mon.smithi037 smithi037 running (32m) 10m ago 48m 70.2M 2048M 18.2.2-1015-gd0e392d9 1cc2f45c17ff eea2060783af 2024-03-29T08:44:14.269 INFO:teuthology.orchestra.run.smithi037.stdout:mon.smithi148 smithi148 running (31m) 10m ago 45m 50.4M 2048M 18.2.2-1015-gd0e392d9 1cc2f45c17ff ad4b2c5748be 2024-03-29T08:44:14.269 INFO:teuthology.orchestra.run.smithi037.stdout:node-exporter.smithi037 smithi037 *:9100 running (34m) 10m ago 47m 8514k - 1.5.0 0da6a335fe13 26b8ae9cb878 2024-03-29T08:44:14.269 INFO:teuthology.orchestra.run.smithi037.stdout:node-exporter.smithi148 smithi148 *:9100 running (34m) 10m ago 45m 8464k - 1.5.0 0da6a335fe13 98414e4e64ed 2024-03-29T08:44:14.269 INFO:teuthology.orchestra.run.smithi037.stdout:osd.0 smithi037 running (30m) 10m ago 44m 108M 1619M 18.2.2-1015-gd0e392d9 1cc2f45c17ff c0c584f6245c 2024-03-29T08:44:14.269 INFO:teuthology.orchestra.run.smithi037.stdout:osd.1 smithi037 running (29m) 10m ago 44m 104M 1619M 18.2.2-1015-gd0e392d9 1cc2f45c17ff a4a2ff4cc83d 2024-03-29T08:44:14.270 INFO:teuthology.orchestra.run.smithi037.stdout:osd.2 smithi037 running (29m) 10m ago 43m 85.9M 1619M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 8b1bc71aa79e 2024-03-29T08:44:14.270 INFO:teuthology.orchestra.run.smithi037.stdout:osd.3 smithi148 running (28m) 10m ago 43m 109M 2642M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 1c656ab8ff86 2024-03-29T08:44:14.270 INFO:teuthology.orchestra.run.smithi037.stdout:osd.4 smithi148 running (27m) 10m ago 42m 101M 2642M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 966a9de11d22 2024-03-29T08:44:14.270 INFO:teuthology.orchestra.run.smithi037.stdout:osd.5 smithi148 running (27m) 10m ago 42m 89.4M 2642M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 474c4f9212ce 2024-03-29T08:44:14.270 INFO:teuthology.orchestra.run.smithi037.stdout:prometheus.smithi037 smithi037 *:9095 running (34m) 10m ago 46m 61.0M - 2.43.0 a07b618ecd1d 30c0223ec589 2024-03-29T08:44:14.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:14.270+0000 7faa94a00700 1 -- 172.21.15.37:0/3352162227 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7faa7406cc30 msgr2=0x7faa7406f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:44:14.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:14.270+0000 7faa94a00700 1 --2- 172.21.15.37:0/3352162227 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7faa7406cc30 0x7faa7406f0e0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7faa90111520 tx=0x7faa7c00b040 comp rx=0 tx=0).stop 2024-03-29T08:44:14.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:14.270+0000 7faa94a00700 1 -- 172.21.15.37:0/3352162227 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faa900fde90 msgr2=0x7faa90115c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:44:14.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:14.270+0000 7faa94a00700 1 --2- 172.21.15.37:0/3352162227 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faa900fde90 0x7faa90115c80 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7faa80000c00 tx=0x7faa80009e30 comp rx=0 tx=0).stop 2024-03-29T08:44:14.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:14.270+0000 7faa94a00700 1 -- 172.21.15.37:0/3352162227 shutdown_connections 2024-03-29T08:44:14.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:14.270+0000 7faa94a00700 1 --2- 172.21.15.37:0/3352162227 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7faa7406cc30 0x7faa7406f0e0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:44:14.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:14.270+0000 7faa94a00700 1 --2- 172.21.15.37:0/3352162227 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faa900fde90 0x7faa90115c80 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:44:14.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:14.270+0000 7faa94a00700 1 --2- 172.21.15.37:0/3352162227 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faa900fe830 0x7faa901161c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:44:14.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:14.270+0000 7faa94a00700 1 -- 172.21.15.37:0/3352162227 >> 172.21.15.37:0/3352162227 conn(0x7faa90072bc0 msgr2=0x7faa900fb7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:44:14.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:14.270+0000 7faa94a00700 1 -- 172.21.15.37:0/3352162227 shutdown_connections 2024-03-29T08:44:14.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:14.270+0000 7faa94a00700 1 -- 172.21.15.37:0/3352162227 wait complete. 2024-03-29T08:44:14.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:14.406+0000 7f38be000700 1 -- 172.21.15.37:0/1482567209 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f38b81037a0 msgr2=0x7f38b80fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:44:14.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:14.406+0000 7f38be000700 1 --2- 172.21.15.37:0/1482567209 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f38b81037a0 0x7f38b80fce10 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f38a4004940 tx=0x7f38a4009e30 comp rx=0 tx=0).stop 2024-03-29T08:44:14.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:14.406+0000 7f38be000700 1 -- 172.21.15.37:0/1482567209 shutdown_connections 2024-03-29T08:44:14.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:14.406+0000 7f38be000700 1 --2- 172.21.15.37:0/1482567209 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f38b81037a0 0x7f38b80fce10 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:44:14.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:14.406+0000 7f38be000700 1 --2- 172.21.15.37:0/1482567209 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f38b8102e00 0x7f38b81031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:44:14.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:14.406+0000 7f38be000700 1 -- 172.21.15.37:0/1482567209 >> 172.21.15.37:0/1482567209 conn(0x7f38b80f89b0 msgr2=0x7f38b80fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:44:14.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:14.406+0000 7f38be000700 1 -- 172.21.15.37:0/1482567209 shutdown_connections 2024-03-29T08:44:14.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:14.406+0000 7f38be000700 1 -- 172.21.15.37:0/1482567209 wait complete. 2024-03-29T08:44:14.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:14.407+0000 7f38be000700 1 Processor -- start 2024-03-29T08:44:14.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:14.407+0000 7f38be000700 1 -- start start 2024-03-29T08:44:14.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:14.407+0000 7f38be000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f38b8102e00 0x7f38b80702c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:44:14.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:14.407+0000 7f38be000700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f38b81037a0 0x7f38b8070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:44:14.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:14.407+0000 7f38be000700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38b8070d90 con 0x7f38b8102e00 2024-03-29T08:44:14.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:14.407+0000 7f38be000700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38b8070ed0 con 0x7f38b81037a0 2024-03-29T08:44:14.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:14.408+0000 7f38b7400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f38b8102e00 0x7f38b80702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:44:14.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:14.408+0000 7f38b7400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f38b8102e00 0x7f38b80702c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42064/0 (socket says 172.21.15.37:42064) 2024-03-29T08:44:14.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:14.408+0000 7f38b7400700 1 -- 172.21.15.37:0/2311085279 learned_addr learned my addr 172.21.15.37:0/2311085279 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:44:14.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:14.408+0000 7f38b6a00700 1 --2- 172.21.15.37:0/2311085279 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f38b81037a0 0x7f38b8070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:44:14.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:14.408+0000 7f38b7400700 1 -- 172.21.15.37:0/2311085279 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f38b81037a0 msgr2=0x7f38b8070800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:44:14.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:14.408+0000 7f38b7400700 1 --2- 172.21.15.37:0/2311085279 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f38b81037a0 0x7f38b8070800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:44:14.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:14.408+0000 7f38b7400700 1 -- 172.21.15.37:0/2311085279 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38a4005040 con 0x7f38b8102e00 2024-03-29T08:44:14.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:14.408+0000 7f38b6a00700 1 --2- 172.21.15.37:0/2311085279 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f38b81037a0 0x7f38b8070800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:44:14.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:14.408+0000 7f38b7400700 1 --2- 172.21.15.37:0/2311085279 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f38b8102e00 0x7f38b80702c0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f38a8002a50 tx=0x7f38a80046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:44:14.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:14.409+0000 7f38af400700 1 -- 172.21.15.37:0/2311085279 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f38a8007cc0 con 0x7f38b8102e00 2024-03-29T08:44:14.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:14.409+0000 7f38af400700 1 -- 172.21.15.37:0/2311085279 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f38a800f040 con 0x7f38b8102e00 2024-03-29T08:44:14.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:14.409+0000 7f38af400700 1 -- 172.21.15.37:0/2311085279 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f38a8009540 con 0x7f38b8102e00 2024-03-29T08:44:14.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:14.409+0000 7f38be000700 1 -- 172.21.15.37:0/2311085279 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38b8119e00 con 0x7f38b8102e00 2024-03-29T08:44:14.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:14.409+0000 7f38be000700 1 -- 172.21.15.37:0/2311085279 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38b811a280 con 0x7f38b8102e00 2024-03-29T08:44:14.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:14.410+0000 7f38be000700 1 -- 172.21.15.37:0/2311085279 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38b80fe550 con 0x7f38b8102e00 2024-03-29T08:44:14.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:14.411+0000 7f38af400700 1 -- 172.21.15.37:0/2311085279 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f38a80096a0 con 0x7f38b8102e00 2024-03-29T08:44:14.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:14.412+0000 7f38af400700 1 --2- 172.21.15.37:0/2311085279 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f389c06cc30 0x7f389c06f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:44:14.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:14.412+0000 7f38af400700 1 -- 172.21.15.37:0/2311085279 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f38a808be00 con 0x7f38b8102e00 2024-03-29T08:44:14.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:14.413+0000 7f38b6a00700 1 --2- 172.21.15.37:0/2311085279 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f389c06cc30 0x7f389c06f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:44:14.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:14.413+0000 7f38b6a00700 1 --2- 172.21.15.37:0/2311085279 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f389c06cc30 0x7f389c06f0e0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f38a4009fd0 tx=0x7f38a4009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:44:14.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:14.417+0000 7f38af400700 1 -- 172.21.15.37:0/2311085279 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f38a8055dc0 con 0x7f38b8102e00 2024-03-29T08:44:14.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:14.716+0000 7f38be000700 1 -- 172.21.15.37:0/2311085279 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f38b80719b0 con 0x7f38b8102e00 2024-03-29T08:44:14.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:14.717+0000 7f38af400700 1 -- 172.21.15.37:0/2311085279 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7f38a80593e0 con 0x7f38b8102e00 2024-03-29T08:44:14.718 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-29T08:44:14.718 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": { 2024-03-29T08:44:14.718 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 2 2024-03-29T08:44:14.718 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:44:14.718 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": { 2024-03-29T08:44:14.718 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 2 2024-03-29T08:44:14.718 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:44:14.718 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": { 2024-03-29T08:44:14.718 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 6 2024-03-29T08:44:14.718 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:44:14.718 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": { 2024-03-29T08:44:14.718 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 4 2024-03-29T08:44:14.719 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:44:14.719 INFO:teuthology.orchestra.run.smithi037.stdout: "overall": { 2024-03-29T08:44:14.719 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 14 2024-03-29T08:44:14.719 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-29T08:44:14.719 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-29T08:44:14.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:14.720+0000 7f38be000700 1 -- 172.21.15.37:0/2311085279 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f389c06cc30 msgr2=0x7f389c06f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:44:14.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:14.720+0000 7f38be000700 1 --2- 172.21.15.37:0/2311085279 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f389c06cc30 0x7f389c06f0e0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f38a4009fd0 tx=0x7f38a4009e30 comp rx=0 tx=0).stop 2024-03-29T08:44:14.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:14.720+0000 7f38be000700 1 -- 172.21.15.37:0/2311085279 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f38b8102e00 msgr2=0x7f38b80702c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:44:14.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:14.720+0000 7f38be000700 1 --2- 172.21.15.37:0/2311085279 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f38b8102e00 0x7f38b80702c0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f38a8002a50 tx=0x7f38a80046b0 comp rx=0 tx=0).stop 2024-03-29T08:44:14.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:14.720+0000 7f38be000700 1 -- 172.21.15.37:0/2311085279 shutdown_connections 2024-03-29T08:44:14.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:14.720+0000 7f38be000700 1 --2- 172.21.15.37:0/2311085279 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f389c06cc30 0x7f389c06f0e0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:44:14.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:14.720+0000 7f38be000700 1 --2- 172.21.15.37:0/2311085279 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f38b8102e00 0x7f38b80702c0 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:44:14.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:14.720+0000 7f38be000700 1 --2- 172.21.15.37:0/2311085279 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f38b81037a0 0x7f38b8070800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:44:14.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:14.720+0000 7f38be000700 1 -- 172.21.15.37:0/2311085279 >> 172.21.15.37:0/2311085279 conn(0x7f38b80f89b0 msgr2=0x7f38b80f9ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:44:14.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:14.720+0000 7f38be000700 1 -- 172.21.15.37:0/2311085279 shutdown_connections 2024-03-29T08:44:14.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:14.720+0000 7f38be000700 1 -- 172.21.15.37:0/2311085279 wait complete. 2024-03-29T08:44:14.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:14.857+0000 7fb392000700 1 -- 172.21.15.37:0/2998835246 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb38c0fe830 msgr2=0x7fb38c106f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:44:14.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:14.857+0000 7fb392000700 1 --2- 172.21.15.37:0/2998835246 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb38c0fe830 0x7fb38c106f00 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fb378004940 tx=0x7fb378009e30 comp rx=0 tx=0).stop 2024-03-29T08:44:14.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:14.857+0000 7fb392000700 1 -- 172.21.15.37:0/2998835246 shutdown_connections 2024-03-29T08:44:14.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:14.857+0000 7fb392000700 1 --2- 172.21.15.37:0/2998835246 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb38c0fe830 0x7fb38c106f00 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:44:14.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:14.857+0000 7fb392000700 1 --2- 172.21.15.37:0/2998835246 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb38c0fde90 0x7fb38c0fe260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:44:14.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:14.857+0000 7fb392000700 1 -- 172.21.15.37:0/2998835246 >> 172.21.15.37:0/2998835246 conn(0x7fb38c072bc0 msgr2=0x7fb38c072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:44:14.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:14.857+0000 7fb392000700 1 -- 172.21.15.37:0/2998835246 shutdown_connections 2024-03-29T08:44:14.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:14.857+0000 7fb392000700 1 -- 172.21.15.37:0/2998835246 wait complete. 2024-03-29T08:44:14.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:14.858+0000 7fb392000700 1 Processor -- start 2024-03-29T08:44:14.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:14.858+0000 7fb392000700 1 -- start start 2024-03-29T08:44:14.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:14.858+0000 7fb392000700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb38c0fde90 0x7fb38c0fbd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:44:14.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:14.859+0000 7fb392000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb38c0fe830 0x7fb38c0fc290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:44:14.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:14.859+0000 7fb392000700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb38c0fc820 con 0x7fb38c0fe830 2024-03-29T08:44:14.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:14.859+0000 7fb392000700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb38c0fc960 con 0x7fb38c0fde90 2024-03-29T08:44:14.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:14.859+0000 7fb38aa00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb38c0fe830 0x7fb38c0fc290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:44:14.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:14.859+0000 7fb38aa00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb38c0fe830 0x7fb38c0fc290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42090/0 (socket says 172.21.15.37:42090) 2024-03-29T08:44:14.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:14.859+0000 7fb38aa00700 1 -- 172.21.15.37:0/469889078 learned_addr learned my addr 172.21.15.37:0/469889078 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:44:14.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:14.859+0000 7fb38b400700 1 --2- 172.21.15.37:0/469889078 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb38c0fde90 0x7fb38c0fbd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:44:14.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:14.859+0000 7fb38aa00700 1 -- 172.21.15.37:0/469889078 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb38c0fde90 msgr2=0x7fb38c0fbd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:44:14.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:14.859+0000 7fb38aa00700 1 --2- 172.21.15.37:0/469889078 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb38c0fde90 0x7fb38c0fbd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:44:14.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:14.859+0000 7fb38aa00700 1 -- 172.21.15.37:0/469889078 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb378005040 con 0x7fb38c0fe830 2024-03-29T08:44:14.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:14.859+0000 7fb38b400700 1 --2- 172.21.15.37:0/469889078 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb38c0fde90 0x7fb38c0fbd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:44:14.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:14.859+0000 7fb38aa00700 1 --2- 172.21.15.37:0/469889078 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb38c0fe830 0x7fb38c0fc290 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fb378000c00 tx=0x7fb378009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:44:14.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:14.860+0000 7fb383400700 1 -- 172.21.15.37:0/469889078 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb378021070 con 0x7fb38c0fe830 2024-03-29T08:44:14.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:14.860+0000 7fb383400700 1 -- 172.21.15.37:0/469889078 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fb378013e00 con 0x7fb38c0fe830 2024-03-29T08:44:14.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:14.860+0000 7fb392000700 1 -- 172.21.15.37:0/469889078 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb38c0fcbb0 con 0x7fb38c0fe830 2024-03-29T08:44:14.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:14.860+0000 7fb383400700 1 -- 172.21.15.37:0/469889078 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb37800e070 con 0x7fb38c0fe830 2024-03-29T08:44:14.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:14.860+0000 7fb392000700 1 -- 172.21.15.37:0/469889078 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb38c11a130 con 0x7fb38c0fe830 2024-03-29T08:44:14.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:14.861+0000 7fb392000700 1 -- 172.21.15.37:0/469889078 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb38c065e30 con 0x7fb38c0fe830 2024-03-29T08:44:14.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:14.862+0000 7fb383400700 1 -- 172.21.15.37:0/469889078 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fb378013950 con 0x7fb38c0fe830 2024-03-29T08:44:14.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:14.863+0000 7fb383400700 1 --2- 172.21.15.37:0/469889078 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fb37006cc30 0x7fb37006f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:44:14.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:14.863+0000 7fb383400700 1 -- 172.21.15.37:0/469889078 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fb37808d050 con 0x7fb38c0fe830 2024-03-29T08:44:14.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:14.863+0000 7fb38b400700 1 --2- 172.21.15.37:0/469889078 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fb37006cc30 0x7fb37006f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:44:14.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:14.864+0000 7fb38b400700 1 --2- 172.21.15.37:0/469889078 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fb37006cc30 0x7fb37006f0e0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fb37c004600 tx=0x7fb37c00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:44:14.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:14.868+0000 7fb383400700 1 -- 172.21.15.37:0/469889078 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fb378057090 con 0x7fb38c0fe830 2024-03-29T08:44:14.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:44:14 smithi037 ceph-mon[142325]: pgmap v1001: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:44:14.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:44:14 smithi037 ceph-mon[142325]: from='client.35794 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:44:14.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:44:14 smithi037 ceph-mon[142325]: from='client.35798 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:44:15.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:44:14 smithi148 ceph-mon[106692]: pgmap v1001: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:44:15.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:44:14 smithi148 ceph-mon[106692]: from='client.35794 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:44:15.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:44:14 smithi148 ceph-mon[106692]: from='client.35798 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:44:15.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:15.125+0000 7fb392000700 1 -- 172.21.15.37:0/469889078 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fb38c05f5f0 con 0x7fb38c0fe830 2024-03-29T08:44:15.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:15.126+0000 7fb383400700 1 -- 172.21.15.37:0/469889078 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 30 v30) v1 ==== 76+0+1841 (secure 0 0 0) 0x7fb37805a6b0 con 0x7fb38c0fe830 2024-03-29T08:44:15.127 INFO:teuthology.orchestra.run.smithi037.stdout:e30 2024-03-29T08:44:15.127 INFO:teuthology.orchestra.run.smithi037.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-29T08:44:15.127 INFO:teuthology.orchestra.run.smithi037.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T08:44:15.127 INFO:teuthology.orchestra.run.smithi037.stdout:legacy client fscid: 1 2024-03-29T08:44:15.127 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:44:15.127 INFO:teuthology.orchestra.run.smithi037.stdout:Filesystem 'cephfs' (1) 2024-03-29T08:44:15.127 INFO:teuthology.orchestra.run.smithi037.stdout:fs_name cephfs 2024-03-29T08:44:15.127 INFO:teuthology.orchestra.run.smithi037.stdout:epoch 21 2024-03-29T08:44:15.127 INFO:teuthology.orchestra.run.smithi037.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-03-29T08:44:15.127 INFO:teuthology.orchestra.run.smithi037.stdout:created 2024-03-29T08:03:29.241269+0000 2024-03-29T08:44:15.127 INFO:teuthology.orchestra.run.smithi037.stdout:modified 2024-03-29T08:17:16.237238+0000 2024-03-29T08:44:15.127 INFO:teuthology.orchestra.run.smithi037.stdout:tableserver 0 2024-03-29T08:44:15.127 INFO:teuthology.orchestra.run.smithi037.stdout:root 0 2024-03-29T08:44:15.128 INFO:teuthology.orchestra.run.smithi037.stdout:session_timeout 60 2024-03-29T08:44:15.128 INFO:teuthology.orchestra.run.smithi037.stdout:session_autoclose 300 2024-03-29T08:44:15.128 INFO:teuthology.orchestra.run.smithi037.stdout:max_file_size 1099511627776 2024-03-29T08:44:15.128 INFO:teuthology.orchestra.run.smithi037.stdout:max_xattr_size 65536 2024-03-29T08:44:15.128 INFO:teuthology.orchestra.run.smithi037.stdout:required_client_features {} 2024-03-29T08:44:15.128 INFO:teuthology.orchestra.run.smithi037.stdout:last_failure 0 2024-03-29T08:44:15.128 INFO:teuthology.orchestra.run.smithi037.stdout:last_failure_osd_epoch 82 2024-03-29T08:44:15.128 INFO:teuthology.orchestra.run.smithi037.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T08:44:15.128 INFO:teuthology.orchestra.run.smithi037.stdout:max_mds 2 2024-03-29T08:44:15.128 INFO:teuthology.orchestra.run.smithi037.stdout:in 0,1 2024-03-29T08:44:15.128 INFO:teuthology.orchestra.run.smithi037.stdout:up {} 2024-03-29T08:44:15.128 INFO:teuthology.orchestra.run.smithi037.stdout:failed 0,1 2024-03-29T08:44:15.128 INFO:teuthology.orchestra.run.smithi037.stdout:damaged 2024-03-29T08:44:15.128 INFO:teuthology.orchestra.run.smithi037.stdout:stopped 2024-03-29T08:44:15.128 INFO:teuthology.orchestra.run.smithi037.stdout:data_pools [3] 2024-03-29T08:44:15.128 INFO:teuthology.orchestra.run.smithi037.stdout:metadata_pool 2 2024-03-29T08:44:15.129 INFO:teuthology.orchestra.run.smithi037.stdout:inline_data disabled 2024-03-29T08:44:15.129 INFO:teuthology.orchestra.run.smithi037.stdout:balancer 2024-03-29T08:44:15.129 INFO:teuthology.orchestra.run.smithi037.stdout:bal_rank_mask -1 2024-03-29T08:44:15.129 INFO:teuthology.orchestra.run.smithi037.stdout:standby_count_wanted 1 2024-03-29T08:44:15.129 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:44:15.129 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:44:15.129 INFO:teuthology.orchestra.run.smithi037.stdout:Standby daemons: 2024-03-29T08:44:15.129 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:44:15.129 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi037.mmwgxe{-1:34440} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.37:6828/1248142005,v1:172.21.15.37:6829/1248142005] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:44:15.129 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi037.iifsmh{-1:34444} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.37:6826/2060882959,v1:172.21.15.37:6827/2060882959] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:44:15.129 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi148.sqteih{-1:44395} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.148:6824/4078652242,v1:172.21.15.148:6825/4078652242] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:44:15.129 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi148.qukpjh{-1:45007} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.148:6826/2394751105,v1:172.21.15.148:6827/2394751105] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:44:15.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:15.128+0000 7fb392000700 1 -- 172.21.15.37:0/469889078 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fb37006cc30 msgr2=0x7fb37006f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:44:15.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:15.128+0000 7fb392000700 1 --2- 172.21.15.37:0/469889078 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fb37006cc30 0x7fb37006f0e0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fb37c004600 tx=0x7fb37c00b040 comp rx=0 tx=0).stop 2024-03-29T08:44:15.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:15.128+0000 7fb392000700 1 -- 172.21.15.37:0/469889078 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb38c0fe830 msgr2=0x7fb38c0fc290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:44:15.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:15.128+0000 7fb392000700 1 --2- 172.21.15.37:0/469889078 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb38c0fe830 0x7fb38c0fc290 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fb378000c00 tx=0x7fb378009e30 comp rx=0 tx=0).stop 2024-03-29T08:44:15.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:15.128+0000 7fb392000700 1 -- 172.21.15.37:0/469889078 shutdown_connections 2024-03-29T08:44:15.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:15.128+0000 7fb392000700 1 --2- 172.21.15.37:0/469889078 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fb37006cc30 0x7fb37006f0e0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:44:15.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:15.128+0000 7fb392000700 1 --2- 172.21.15.37:0/469889078 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb38c0fde90 0x7fb38c0fbd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:44:15.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:15.128+0000 7fb392000700 1 --2- 172.21.15.37:0/469889078 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb38c0fe830 0x7fb38c0fc290 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:44:15.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:15.128+0000 7fb392000700 1 -- 172.21.15.37:0/469889078 >> 172.21.15.37:0/469889078 conn(0x7fb38c072bc0 msgr2=0x7fb38c0ff3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:44:15.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:15.128+0000 7fb392000700 1 -- 172.21.15.37:0/469889078 shutdown_connections 2024-03-29T08:44:15.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:15.129+0000 7fb392000700 1 -- 172.21.15.37:0/469889078 wait complete. 2024-03-29T08:44:15.131 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 30 2024-03-29T08:44:15.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:15.267+0000 7fdbe3e00700 1 -- 172.21.15.37:0/1610022157 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdbe4070c10 msgr2=0x7fdbe4108fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:44:15.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:15.267+0000 7fdbe3e00700 1 --2- 172.21.15.37:0/1610022157 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdbe4070c10 0x7fdbe4108fa0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fdbd0004970 tx=0x7fdbd0009e30 comp rx=0 tx=0).stop 2024-03-29T08:44:15.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:15.267+0000 7fdbe3e00700 1 -- 172.21.15.37:0/1610022157 shutdown_connections 2024-03-29T08:44:15.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:15.267+0000 7fdbe3e00700 1 --2- 172.21.15.37:0/1610022157 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdbe4070c10 0x7fdbe4108fa0 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:44:15.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:15.267+0000 7fdbe3e00700 1 --2- 172.21.15.37:0/1610022157 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdbe4070300 0x7fdbe40706d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:44:15.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:15.267+0000 7fdbe3e00700 1 -- 172.21.15.37:0/1610022157 >> 172.21.15.37:0/1610022157 conn(0x7fdbe40f87a0 msgr2=0x7fdbe40fabb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:44:15.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:15.267+0000 7fdbe3e00700 1 -- 172.21.15.37:0/1610022157 shutdown_connections 2024-03-29T08:44:15.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:15.268+0000 7fdbe3e00700 1 -- 172.21.15.37:0/1610022157 wait complete. 2024-03-29T08:44:15.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:15.268+0000 7fdbe3e00700 1 Processor -- start 2024-03-29T08:44:15.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:15.268+0000 7fdbe3e00700 1 -- start start 2024-03-29T08:44:15.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:15.269+0000 7fdbe3e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdbe4070300 0x7fdbe4115a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:44:15.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:15.269+0000 7fdbe3e00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdbe4070c10 0x7fdbe4115fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:44:15.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:15.269+0000 7fdbe3e00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbe4116610 con 0x7fdbe4070300 2024-03-29T08:44:15.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:15.269+0000 7fdbe3e00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbe410fa00 con 0x7fdbe4070c10 2024-03-29T08:44:15.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:15.269+0000 7fdbe2a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdbe4070300 0x7fdbe4115a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:44:15.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:15.269+0000 7fdbe2a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdbe4070300 0x7fdbe4115a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42110/0 (socket says 172.21.15.37:42110) 2024-03-29T08:44:15.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:15.269+0000 7fdbe2a00700 1 -- 172.21.15.37:0/2859911949 learned_addr learned my addr 172.21.15.37:0/2859911949 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:44:15.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:15.269+0000 7fdbe2000700 1 --2- 172.21.15.37:0/2859911949 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdbe4070c10 0x7fdbe4115fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:44:15.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:15.270+0000 7fdbe2a00700 1 -- 172.21.15.37:0/2859911949 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdbe4070c10 msgr2=0x7fdbe4115fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:44:15.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:15.270+0000 7fdbe2a00700 1 --2- 172.21.15.37:0/2859911949 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdbe4070c10 0x7fdbe4115fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:44:15.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:15.270+0000 7fdbe2a00700 1 -- 172.21.15.37:0/2859911949 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbd0005040 con 0x7fdbe4070300 2024-03-29T08:44:15.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:15.270+0000 7fdbe2a00700 1 --2- 172.21.15.37:0/2859911949 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdbe4070300 0x7fdbe4115a80 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7fdbd4002a50 tx=0x7fdbd40046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:44:15.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:15.270+0000 7fdbdaa00700 1 -- 172.21.15.37:0/2859911949 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdbd4007cc0 con 0x7fdbe4070300 2024-03-29T08:44:15.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:15.270+0000 7fdbdaa00700 1 -- 172.21.15.37:0/2859911949 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fdbd400f040 con 0x7fdbe4070300 2024-03-29T08:44:15.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:15.270+0000 7fdbdaa00700 1 -- 172.21.15.37:0/2859911949 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdbd400d9c0 con 0x7fdbe4070300 2024-03-29T08:44:15.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:15.270+0000 7fdbe3e00700 1 -- 172.21.15.37:0/2859911949 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbe410fce0 con 0x7fdbe4070300 2024-03-29T08:44:15.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:15.270+0000 7fdbe3e00700 1 -- 172.21.15.37:0/2859911949 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbe41101f0 con 0x7fdbe4070300 2024-03-29T08:44:15.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:15.271+0000 7fdbe3e00700 1 -- 172.21.15.37:0/2859911949 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdbe4071a40 con 0x7fdbe4070300 2024-03-29T08:44:15.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:15.272+0000 7fdbdaa00700 1 -- 172.21.15.37:0/2859911949 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fdbd400db20 con 0x7fdbe4070300 2024-03-29T08:44:15.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:15.273+0000 7fdbdaa00700 1 --2- 172.21.15.37:0/2859911949 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fdbc806cc30 0x7fdbc806f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:44:15.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:15.273+0000 7fdbdaa00700 1 -- 172.21.15.37:0/2859911949 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fdbd408bd20 con 0x7fdbe4070300 2024-03-29T08:44:15.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:15.273+0000 7fdbe2000700 1 --2- 172.21.15.37:0/2859911949 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fdbc806cc30 0x7fdbc806f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:44:15.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:15.274+0000 7fdbe2000700 1 --2- 172.21.15.37:0/2859911949 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fdbc806cc30 0x7fdbc806f0e0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fdbd000b370 tx=0x7fdbd0009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:44:15.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:15.278+0000 7fdbdaa00700 1 -- 172.21.15.37:0/2859911949 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fdbd4059770 con 0x7fdbe4070300 2024-03-29T08:44:15.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:15.509+0000 7fdbe3e00700 1 -- 172.21.15.37:0/2859911949 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fdbe410be60 con 0x7fdbc806cc30 2024-03-29T08:44:15.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:15.511+0000 7fdbdaa00700 1 -- 172.21.15.37:0/2859911949 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fdbe410be60 con 0x7fdbc806cc30 2024-03-29T08:44:15.512 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-29T08:44:15.512 INFO:teuthology.orchestra.run.smithi037.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59", 2024-03-29T08:44:15.512 INFO:teuthology.orchestra.run.smithi037.stdout: "in_progress": true, 2024-03-29T08:44:15.512 INFO:teuthology.orchestra.run.smithi037.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-29T08:44:15.512 INFO:teuthology.orchestra.run.smithi037.stdout: "services_complete": [ 2024-03-29T08:44:15.512 INFO:teuthology.orchestra.run.smithi037.stdout: "crash", 2024-03-29T08:44:15.512 INFO:teuthology.orchestra.run.smithi037.stdout: "osd", 2024-03-29T08:44:15.512 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr", 2024-03-29T08:44:15.512 INFO:teuthology.orchestra.run.smithi037.stdout: "mds", 2024-03-29T08:44:15.512 INFO:teuthology.orchestra.run.smithi037.stdout: "mon" 2024-03-29T08:44:15.512 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-29T08:44:15.512 INFO:teuthology.orchestra.run.smithi037.stdout: "progress": "16/21 daemons upgraded", 2024-03-29T08:44:15.512 INFO:teuthology.orchestra.run.smithi037.stdout: "message": "Currently upgrading mds daemons", 2024-03-29T08:44:15.512 INFO:teuthology.orchestra.run.smithi037.stdout: "is_paused": false 2024-03-29T08:44:15.513 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-29T08:44:15.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:15.513+0000 7fdbe3e00700 1 -- 172.21.15.37:0/2859911949 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fdbc806cc30 msgr2=0x7fdbc806f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:44:15.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:15.513+0000 7fdbe3e00700 1 --2- 172.21.15.37:0/2859911949 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fdbc806cc30 0x7fdbc806f0e0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fdbd000b370 tx=0x7fdbd0009e30 comp rx=0 tx=0).stop 2024-03-29T08:44:15.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:15.513+0000 7fdbe3e00700 1 -- 172.21.15.37:0/2859911949 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdbe4070300 msgr2=0x7fdbe4115a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:44:15.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:15.513+0000 7fdbe3e00700 1 --2- 172.21.15.37:0/2859911949 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdbe4070300 0x7fdbe4115a80 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7fdbd4002a50 tx=0x7fdbd40046b0 comp rx=0 tx=0).stop 2024-03-29T08:44:15.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:15.513+0000 7fdbe3e00700 1 -- 172.21.15.37:0/2859911949 shutdown_connections 2024-03-29T08:44:15.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:15.513+0000 7fdbe3e00700 1 --2- 172.21.15.37:0/2859911949 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fdbc806cc30 0x7fdbc806f0e0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:44:15.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:15.513+0000 7fdbe3e00700 1 --2- 172.21.15.37:0/2859911949 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdbe4070300 0x7fdbe4115a80 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:44:15.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:15.513+0000 7fdbe3e00700 1 --2- 172.21.15.37:0/2859911949 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdbe4070c10 0x7fdbe4115fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:44:15.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:15.513+0000 7fdbe3e00700 1 -- 172.21.15.37:0/2859911949 >> 172.21.15.37:0/2859911949 conn(0x7fdbe40f87a0 msgr2=0x7fdbe40f9cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:44:15.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:15.513+0000 7fdbe3e00700 1 -- 172.21.15.37:0/2859911949 shutdown_connections 2024-03-29T08:44:15.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:15.513+0000 7fdbe3e00700 1 -- 172.21.15.37:0/2859911949 wait complete. 2024-03-29T08:44:15.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:15.651+0000 7ff3fe200700 1 -- 172.21.15.37:0/2191129688 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff3f8070c70 msgr2=0x7ff3f8109110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:44:15.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:15.651+0000 7ff3fe200700 1 --2- 172.21.15.37:0/2191129688 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff3f8070c70 0x7ff3f8109110 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7ff3e4004970 tx=0x7ff3e4009e30 comp rx=0 tx=0).stop 2024-03-29T08:44:15.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:15.651+0000 7ff3fe200700 1 -- 172.21.15.37:0/2191129688 shutdown_connections 2024-03-29T08:44:15.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:15.651+0000 7ff3fe200700 1 --2- 172.21.15.37:0/2191129688 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff3f8070c70 0x7ff3f8109110 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:44:15.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:15.651+0000 7ff3fe200700 1 --2- 172.21.15.37:0/2191129688 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff3f8070360 0x7ff3f8070730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:44:15.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:15.651+0000 7ff3fe200700 1 -- 172.21.15.37:0/2191129688 >> 172.21.15.37:0/2191129688 conn(0x7ff3f80f89b0 msgr2=0x7ff3f80fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:44:15.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:15.651+0000 7ff3fe200700 1 -- 172.21.15.37:0/2191129688 shutdown_connections 2024-03-29T08:44:15.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:15.651+0000 7ff3fe200700 1 -- 172.21.15.37:0/2191129688 wait complete. 2024-03-29T08:44:15.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:15.652+0000 7ff3fe200700 1 Processor -- start 2024-03-29T08:44:15.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:15.652+0000 7ff3fe200700 1 -- start start 2024-03-29T08:44:15.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:15.653+0000 7ff3fe200700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff3f8070360 0x7ff3f8115d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:44:15.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:15.653+0000 7ff3fe200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff3f8070c70 0x7ff3f8116250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:44:15.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:15.653+0000 7ff3fe200700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3f8116930 con 0x7ff3f8070c70 2024-03-29T08:44:15.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:15.653+0000 7ff3fe200700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3f810fc70 con 0x7ff3f8070360 2024-03-29T08:44:15.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:15.653+0000 7ff3f6a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff3f8070c70 0x7ff3f8116250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:44:15.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:15.653+0000 7ff3f6a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff3f8070c70 0x7ff3f8116250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42138/0 (socket says 172.21.15.37:42138) 2024-03-29T08:44:15.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:15.653+0000 7ff3f6a00700 1 -- 172.21.15.37:0/3817539957 learned_addr learned my addr 172.21.15.37:0/3817539957 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:44:15.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:15.653+0000 7ff3f7400700 1 --2- 172.21.15.37:0/3817539957 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff3f8070360 0x7ff3f8115d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:44:15.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:15.654+0000 7ff3f6a00700 1 -- 172.21.15.37:0/3817539957 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff3f8070360 msgr2=0x7ff3f8115d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:44:15.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:15.654+0000 7ff3f6a00700 1 --2- 172.21.15.37:0/3817539957 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff3f8070360 0x7ff3f8115d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:44:15.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:15.654+0000 7ff3f6a00700 1 -- 172.21.15.37:0/3817539957 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3e4005040 con 0x7ff3f8070c70 2024-03-29T08:44:15.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:15.654+0000 7ff3f7400700 1 --2- 172.21.15.37:0/3817539957 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff3f8070360 0x7ff3f8115d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:44:15.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:15.654+0000 7ff3f6a00700 1 --2- 172.21.15.37:0/3817539957 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff3f8070c70 0x7ff3f8116250 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7ff3e4004f50 tx=0x7ff3e4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:44:15.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:15.654+0000 7ff3ef400700 1 -- 172.21.15.37:0/3817539957 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff3e400e040 con 0x7ff3f8070c70 2024-03-29T08:44:15.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:15.654+0000 7ff3ef400700 1 -- 172.21.15.37:0/3817539957 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ff3e4013420 con 0x7ff3f8070c70 2024-03-29T08:44:15.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:15.654+0000 7ff3fe200700 1 -- 172.21.15.37:0/3817539957 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3f810fe90 con 0x7ff3f8070c70 2024-03-29T08:44:15.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:15.654+0000 7ff3ef400700 1 -- 172.21.15.37:0/3817539957 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff3e401c610 con 0x7ff3f8070c70 2024-03-29T08:44:15.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:15.654+0000 7ff3fe200700 1 -- 172.21.15.37:0/3817539957 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3f81103d0 con 0x7ff3f8070c70 2024-03-29T08:44:15.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:15.655+0000 7ff3fe200700 1 -- 172.21.15.37:0/3817539957 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3f8071aa0 con 0x7ff3f8070c70 2024-03-29T08:44:15.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:15.657+0000 7ff3ef400700 1 -- 172.21.15.37:0/3817539957 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7ff3e4013590 con 0x7ff3f8070c70 2024-03-29T08:44:15.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:15.657+0000 7ff3ef400700 1 --2- 172.21.15.37:0/3817539957 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7ff3dc06cc30 0x7ff3dc06f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:44:15.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:15.658+0000 7ff3ef400700 1 -- 172.21.15.37:0/3817539957 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7ff3e408dcd0 con 0x7ff3f8070c70 2024-03-29T08:44:15.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:15.658+0000 7ff3f7400700 1 --2- 172.21.15.37:0/3817539957 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7ff3dc06cc30 0x7ff3dc06f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:44:15.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:15.658+0000 7ff3f7400700 1 --2- 172.21.15.37:0/3817539957 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7ff3dc06cc30 0x7ff3dc06f0e0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7ff3e8009b10 tx=0x7ff3e8008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:44:15.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:15.661+0000 7ff3ef400700 1 -- 172.21.15.37:0/3817539957 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7ff3e405b7a0 con 0x7ff3f8070c70 2024-03-29T08:44:15.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:44:15 smithi037 ceph-mon[142325]: from='client.35802 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:44:15.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:44:15 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/2311085279' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:44:15.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:44:15 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/469889078' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T08:44:15.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:15.963+0000 7ff3fe200700 1 -- 172.21.15.37:0/3817539957 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7ff3f810c0c0 con 0x7ff3f8070c70 2024-03-29T08:44:15.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:15.964+0000 7ff3ef400700 1 -- 172.21.15.37:0/3817539957 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+3583 (secure 0 0 0) 0x7ff3e405b330 con 0x7ff3f8070c70 2024-03-29T08:44:15.964 INFO:teuthology.orchestra.run.smithi037.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem is offline 2024-03-29T08:44:15.965 INFO:teuthology.orchestra.run.smithi037.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-03-29T08:44:15.965 INFO:teuthology.orchestra.run.smithi037.stdout: host smithi148 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/mon.smithi148/config 2024-03-29T08:44:15.965 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/podman: timeout after 895 seconds 2024-03-29T08:44:15.965 INFO:teuthology.orchestra.run.smithi037.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59 -e NODE_NAME=smithi148 -e CEPH_USE_RANDOM_NONCE=1 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4:/var/run/ceph:z -v /var/log/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4:/var/log/ceph:z -v /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpkx2rz2d4:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59 inventory --format=json-pretty --filter-for-batch 2024-03-29T08:44:15.965 INFO:teuthology.orchestra.run.smithi037.stdout:Traceback (most recent call last): 2024-03-29T08:44:15.965 INFO:teuthology.orchestra.run.smithi037.stdout: File "/usr/lib64/python3.6/runpy.py", line 193, in _run_module_as_main 2024-03-29T08:44:15.965 INFO:teuthology.orchestra.run.smithi037.stdout: "__main__", mod_spec) 2024-03-29T08:44:15.965 INFO:teuthology.orchestra.run.smithi037.stdout: File "/usr/lib64/python3.6/runpy.py", line 85, in _run_code 2024-03-29T08:44:15.965 INFO:teuthology.orchestra.run.smithi037.stdout: exec(code, run_globals) 2024-03-29T08:44:15.965 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 10889, in 2024-03-29T08:44:15.965 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 10877, in main 2024-03-29T08:44:15.965 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 2576, in _infer_config 2024-03-29T08:44:15.965 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 2492, in _infer_fsid 2024-03-29T08:44:15.965 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 2604, in _infer_image 2024-03-29T08:44:15.966 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 2479, in _validate_fsid 2024-03-29T08:44:15.966 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 7145, in command_ceph_volume 2024-03-29T08:44:15.966 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 2267, in call_throws 2024-03-29T08:44:15.966 INFO:teuthology.orchestra.run.smithi037.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59 -e NODE_NAME=smithi148 -e CEPH_USE_RANDOM_NONCE=1 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4:/var/run/ceph:z -v /var/log/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4:/var/log/ceph:z -v /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpkx2rz2d4:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59 inventory --format=json-pretty --filter-for-batch 2024-03-29T08:44:15.966 INFO:teuthology.orchestra.run.smithi037.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-03-29T08:44:15.966 INFO:teuthology.orchestra.run.smithi037.stdout: fs cephfs is degraded 2024-03-29T08:44:15.966 INFO:teuthology.orchestra.run.smithi037.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-03-29T08:44:15.966 INFO:teuthology.orchestra.run.smithi037.stdout: fs cephfs is offline because no MDS is active for it. 2024-03-29T08:44:15.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:15.966+0000 7ff3fe200700 1 -- 172.21.15.37:0/3817539957 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7ff3dc06cc30 msgr2=0x7ff3dc06f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:44:15.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:15.966+0000 7ff3fe200700 1 --2- 172.21.15.37:0/3817539957 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7ff3dc06cc30 0x7ff3dc06f0e0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7ff3e8009b10 tx=0x7ff3e8008040 comp rx=0 tx=0).stop 2024-03-29T08:44:15.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:15.966+0000 7ff3fe200700 1 -- 172.21.15.37:0/3817539957 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff3f8070c70 msgr2=0x7ff3f8116250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:44:15.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:15.966+0000 7ff3fe200700 1 --2- 172.21.15.37:0/3817539957 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff3f8070c70 0x7ff3f8116250 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7ff3e4004f50 tx=0x7ff3e4009e30 comp rx=0 tx=0).stop 2024-03-29T08:44:15.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:15.966+0000 7ff3fe200700 1 -- 172.21.15.37:0/3817539957 shutdown_connections 2024-03-29T08:44:15.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:15.966+0000 7ff3fe200700 1 --2- 172.21.15.37:0/3817539957 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7ff3dc06cc30 0x7ff3dc06f0e0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:44:15.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:15.966+0000 7ff3fe200700 1 --2- 172.21.15.37:0/3817539957 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff3f8070360 0x7ff3f8115d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:44:15.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:15.966+0000 7ff3fe200700 1 --2- 172.21.15.37:0/3817539957 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff3f8070c70 0x7ff3f8116250 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:44:15.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:15.966+0000 7ff3fe200700 1 -- 172.21.15.37:0/3817539957 >> 172.21.15.37:0/3817539957 conn(0x7ff3f80f89b0 msgr2=0x7ff3f80f9eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:44:15.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:15.966+0000 7ff3fe200700 1 -- 172.21.15.37:0/3817539957 shutdown_connections 2024-03-29T08:44:15.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:15.966+0000 7ff3fe200700 1 -- 172.21.15.37:0/3817539957 wait complete. 2024-03-29T08:44:16.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:44:15 smithi148 ceph-mon[106692]: from='client.35802 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:44:16.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:44:15 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/2311085279' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:44:16.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:44:15 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/469889078' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T08:44:16.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:44:16 smithi037 ceph-mon[142325]: pgmap v1002: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:44:16.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:44:16 smithi037 ceph-mon[142325]: from='client.35814 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:44:16.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:44:16 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/3817539957' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T08:44:17.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:44:16 smithi148 ceph-mon[106692]: pgmap v1002: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:44:17.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:44:16 smithi148 ceph-mon[106692]: from='client.35814 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:44:17.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:44:16 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/3817539957' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T08:44:18.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:44:18 smithi037 ceph-mon[142325]: pgmap v1003: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:44:19.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:44:18 smithi148 ceph-mon[106692]: pgmap v1003: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:44:20.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:44:20 smithi037 ceph-mon[142325]: pgmap v1004: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:44:21.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:44:20 smithi148 ceph-mon[106692]: pgmap v1004: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:44:22.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:44:22 smithi037 ceph-mon[142325]: pgmap v1005: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:44:23.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:44:22 smithi148 ceph-mon[106692]: pgmap v1005: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:44:24.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:44:24 smithi037 ceph-mon[142325]: pgmap v1006: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:44:25.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:44:24 smithi148 ceph-mon[106692]: pgmap v1006: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:44:25.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:44:25 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:44:26.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:44:25 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:44:26.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:44:26 smithi037 ceph-mon[142325]: pgmap v1007: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:44:27.055 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:44:26 smithi148 ceph-mon[106692]: pgmap v1007: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:44:28.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:44:28 smithi037 ceph-mon[142325]: pgmap v1008: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:44:29.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:44:28 smithi148 ceph-mon[106692]: pgmap v1008: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:44:30.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:44:30 smithi037 ceph-mon[142325]: pgmap v1009: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:44:31.059 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:44:30 smithi148 ceph-mon[106692]: pgmap v1009: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:44:32.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:44:32 smithi037 ceph-mon[142325]: pgmap v1010: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:44:33.055 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:44:32 smithi148 ceph-mon[106692]: pgmap v1010: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:44:34.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:44:34 smithi037 ceph-mon[142325]: pgmap v1011: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:44:35.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:44:34 smithi148 ceph-mon[106692]: pgmap v1011: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:44:36.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:44:36 smithi037 ceph-mon[142325]: pgmap v1012: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:44:37.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:44:36 smithi148 ceph-mon[106692]: pgmap v1012: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:44:38.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:44:38 smithi037 ceph-mon[142325]: pgmap v1013: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:44:39.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:44:38 smithi148 ceph-mon[106692]: pgmap v1013: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:44:40.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:44:40 smithi037 ceph-mon[142325]: pgmap v1014: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:44:40.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:44:40 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:44:41.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:44:40 smithi148 ceph-mon[106692]: pgmap v1014: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:44:41.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:44:40 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:44:42.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:44:42 smithi037 ceph-mon[142325]: pgmap v1015: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:44:43.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:44:42 smithi148 ceph-mon[106692]: pgmap v1015: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:44:44.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:44:44 smithi037 ceph-mon[142325]: pgmap v1016: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:44:45.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:44:44 smithi148 ceph-mon[106692]: pgmap v1016: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:44:46.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:46.122+0000 7f11e9a00700 1 -- 172.21.15.37:0/3077872180 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f11e4102e00 msgr2=0x7f11e41031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:44:46.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:46.122+0000 7f11e9a00700 1 --2- 172.21.15.37:0/3077872180 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f11e4102e00 0x7f11e41031d0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f11d4004940 tx=0x7f11d4009e30 comp rx=0 tx=0).stop 2024-03-29T08:44:46.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:46.122+0000 7f11e9a00700 1 -- 172.21.15.37:0/3077872180 shutdown_connections 2024-03-29T08:44:46.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:46.122+0000 7f11e9a00700 1 --2- 172.21.15.37:0/3077872180 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f11e41037a0 0x7f11e40fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:44:46.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:46.122+0000 7f11e9a00700 1 --2- 172.21.15.37:0/3077872180 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f11e4102e00 0x7f11e41031d0 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:44:46.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:46.122+0000 7f11e9a00700 1 -- 172.21.15.37:0/3077872180 >> 172.21.15.37:0/3077872180 conn(0x7f11e40f89b0 msgr2=0x7f11e40fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:44:46.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:46.122+0000 7f11e9a00700 1 -- 172.21.15.37:0/3077872180 shutdown_connections 2024-03-29T08:44:46.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:46.122+0000 7f11e9a00700 1 -- 172.21.15.37:0/3077872180 wait complete. 2024-03-29T08:44:46.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:46.123+0000 7f11e9a00700 1 Processor -- start 2024-03-29T08:44:46.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:46.123+0000 7f11e9a00700 1 -- start start 2024-03-29T08:44:46.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:46.124+0000 7f11e9a00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f11e4102e00 0x7f11e4115cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:44:46.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:46.124+0000 7f11e9a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f11e41037a0 0x7f11e4116210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:44:46.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:46.124+0000 7f11e9a00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11e4116860 con 0x7f11e41037a0 2024-03-29T08:44:46.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:46.124+0000 7f11e9a00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11e410fc30 con 0x7f11e4102e00 2024-03-29T08:44:46.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:46.124+0000 7f11e2000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f11e41037a0 0x7f11e4116210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:44:46.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:46.124+0000 7f11e2000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f11e41037a0 0x7f11e4116210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59308/0 (socket says 172.21.15.37:59308) 2024-03-29T08:44:46.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:46.124+0000 7f11e2000700 1 -- 172.21.15.37:0/4149734618 learned_addr learned my addr 172.21.15.37:0/4149734618 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:44:46.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:46.124+0000 7f11e2a00700 1 --2- 172.21.15.37:0/4149734618 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f11e4102e00 0x7f11e4115cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:44:46.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:46.124+0000 7f11e2000700 1 -- 172.21.15.37:0/4149734618 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f11e4102e00 msgr2=0x7f11e4115cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:44:46.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:46.124+0000 7f11e2000700 1 --2- 172.21.15.37:0/4149734618 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f11e4102e00 0x7f11e4115cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:44:46.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:46.124+0000 7f11e2000700 1 -- 172.21.15.37:0/4149734618 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11d4005040 con 0x7f11e41037a0 2024-03-29T08:44:46.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:46.124+0000 7f11e2a00700 1 --2- 172.21.15.37:0/4149734618 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f11e4102e00 0x7f11e4115cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:44:46.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:46.125+0000 7f11e2000700 1 --2- 172.21.15.37:0/4149734618 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f11e41037a0 0x7f11e4116210 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f11d0002a50 tx=0x7f11d0004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:44:46.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:46.125+0000 7f11daa00700 1 -- 172.21.15.37:0/4149734618 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f11d0007cc0 con 0x7f11e41037a0 2024-03-29T08:44:46.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:46.125+0000 7f11daa00700 1 -- 172.21.15.37:0/4149734618 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f11d000f040 con 0x7f11e41037a0 2024-03-29T08:44:46.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:46.125+0000 7f11e9a00700 1 -- 172.21.15.37:0/4149734618 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11e410ff10 con 0x7f11e41037a0 2024-03-29T08:44:46.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:46.125+0000 7f11e9a00700 1 -- 172.21.15.37:0/4149734618 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11e41104d0 con 0x7f11e41037a0 2024-03-29T08:44:46.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:46.125+0000 7f11daa00700 1 -- 172.21.15.37:0/4149734618 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f11d000d9a0 con 0x7f11e41037a0 2024-03-29T08:44:46.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:46.126+0000 7f11e9a00700 1 -- 172.21.15.37:0/4149734618 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11c40052b0 con 0x7f11e41037a0 2024-03-29T08:44:46.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:46.127+0000 7f11daa00700 1 -- 172.21.15.37:0/4149734618 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f11d000dc10 con 0x7f11e41037a0 2024-03-29T08:44:46.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:46.128+0000 7f11daa00700 1 --2- 172.21.15.37:0/4149734618 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f11c806cc30 0x7f11c806f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:44:46.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:46.128+0000 7f11daa00700 1 -- 172.21.15.37:0/4149734618 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f11d008bd80 con 0x7f11e41037a0 2024-03-29T08:44:46.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:46.128+0000 7f11e2a00700 1 --2- 172.21.15.37:0/4149734618 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f11c806cc30 0x7f11c806f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:44:46.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:46.129+0000 7f11e2a00700 1 --2- 172.21.15.37:0/4149734618 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f11c806cc30 0x7f11c806f0e0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f11d4000c00 tx=0x7f11d4009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:44:46.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:46.133+0000 7f11daa00700 1 -- 172.21.15.37:0/4149734618 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f11d00597d0 con 0x7f11e41037a0 2024-03-29T08:44:46.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:46.365+0000 7f11e9a00700 1 -- 172.21.15.37:0/4149734618 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f11c4000bc0 con 0x7f11c806cc30 2024-03-29T08:44:46.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:46.367+0000 7f11daa00700 1 -- 172.21.15.37:0/4149734618 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f11c4000bc0 con 0x7f11c806cc30 2024-03-29T08:44:46.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:46.369+0000 7f11e9a00700 1 -- 172.21.15.37:0/4149734618 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f11c806cc30 msgr2=0x7f11c806f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:44:46.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:46.369+0000 7f11e9a00700 1 --2- 172.21.15.37:0/4149734618 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f11c806cc30 0x7f11c806f0e0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f11d4000c00 tx=0x7f11d4009e30 comp rx=0 tx=0).stop 2024-03-29T08:44:46.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:46.369+0000 7f11e9a00700 1 -- 172.21.15.37:0/4149734618 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f11e41037a0 msgr2=0x7f11e4116210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:44:46.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:46.369+0000 7f11e9a00700 1 --2- 172.21.15.37:0/4149734618 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f11e41037a0 0x7f11e4116210 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f11d0002a50 tx=0x7f11d0004600 comp rx=0 tx=0).stop 2024-03-29T08:44:46.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:46.369+0000 7f11e9a00700 1 -- 172.21.15.37:0/4149734618 shutdown_connections 2024-03-29T08:44:46.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:46.369+0000 7f11e9a00700 1 --2- 172.21.15.37:0/4149734618 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f11c806cc30 0x7f11c806f0e0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:44:46.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:46.369+0000 7f11e9a00700 1 --2- 172.21.15.37:0/4149734618 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f11e4102e00 0x7f11e4115cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:44:46.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:46.369+0000 7f11e9a00700 1 --2- 172.21.15.37:0/4149734618 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f11e41037a0 0x7f11e4116210 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:44:46.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:46.369+0000 7f11e9a00700 1 -- 172.21.15.37:0/4149734618 >> 172.21.15.37:0/4149734618 conn(0x7f11e40f89b0 msgr2=0x7f11e40f9980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:44:46.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:46.369+0000 7f11e9a00700 1 -- 172.21.15.37:0/4149734618 shutdown_connections 2024-03-29T08:44:46.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:46.369+0000 7f11e9a00700 1 -- 172.21.15.37:0/4149734618 wait complete. 2024-03-29T08:44:46.384 INFO:teuthology.orchestra.run.smithi037.stdout:true 2024-03-29T08:44:46.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:46.508+0000 7f209fa00700 1 -- 172.21.15.37:0/2663472182 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2098105090 msgr2=0x7f2098105460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:44:46.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:46.508+0000 7f209fa00700 1 --2- 172.21.15.37:0/2663472182 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2098105090 0x7f2098105460 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f208c009ce0 tx=0x7f208c004d10 comp rx=0 tx=0).stop 2024-03-29T08:44:46.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:46.508+0000 7f209fa00700 1 -- 172.21.15.37:0/2663472182 shutdown_connections 2024-03-29T08:44:46.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:46.508+0000 7f209fa00700 1 --2- 172.21.15.37:0/2663472182 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2098105a30 0x7f20980ff0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:44:46.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:46.508+0000 7f209fa00700 1 --2- 172.21.15.37:0/2663472182 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2098105090 0x7f2098105460 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:44:46.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:46.508+0000 7f209fa00700 1 -- 172.21.15.37:0/2663472182 >> 172.21.15.37:0/2663472182 conn(0x7f20980fac00 msgr2=0x7f20980fd010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:44:46.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:46.509+0000 7f209fa00700 1 -- 172.21.15.37:0/2663472182 shutdown_connections 2024-03-29T08:44:46.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:46.509+0000 7f209fa00700 1 -- 172.21.15.37:0/2663472182 wait complete. 2024-03-29T08:44:46.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:46.509+0000 7f209fa00700 1 Processor -- start 2024-03-29T08:44:46.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:46.510+0000 7f209fa00700 1 -- start start 2024-03-29T08:44:46.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:46.510+0000 7f209fa00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2098105090 0x7f2098115c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:44:46.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:46.510+0000 7f209fa00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2098105a30 0x7f2098116190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:44:46.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:46.510+0000 7f209fa00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20981167e0 con 0x7f2098105090 2024-03-29T08:44:46.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:46.510+0000 7f209fa00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f209810fdc0 con 0x7f2098105a30 2024-03-29T08:44:46.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:46.510+0000 7f209d000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2098105090 0x7f2098115c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:44:46.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:46.510+0000 7f209d000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2098105090 0x7f2098115c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59326/0 (socket says 172.21.15.37:59326) 2024-03-29T08:44:46.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:46.510+0000 7f209d000700 1 -- 172.21.15.37:0/2870742895 learned_addr learned my addr 172.21.15.37:0/2870742895 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:44:46.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:46.510+0000 7f209d000700 1 -- 172.21.15.37:0/2870742895 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2098105a30 msgr2=0x7f2098116190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:44:46.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:46.511+0000 7f209d000700 1 --2- 172.21.15.37:0/2870742895 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2098105a30 0x7f2098116190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:44:46.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:46.511+0000 7f209d000700 1 -- 172.21.15.37:0/2870742895 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f208c009940 con 0x7f2098105090 2024-03-29T08:44:46.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:46.511+0000 7f209d000700 1 --2- 172.21.15.37:0/2870742895 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2098105090 0x7f2098115c50 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f208c005010 tx=0x7f208c004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:44:46.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:46.511+0000 7f2094c00700 1 -- 172.21.15.37:0/2870742895 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f208c00e040 con 0x7f2098105090 2024-03-29T08:44:46.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:46.511+0000 7f2094c00700 1 -- 172.21.15.37:0/2870742895 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f208c008da0 con 0x7f2098105090 2024-03-29T08:44:46.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:46.511+0000 7f2094c00700 1 -- 172.21.15.37:0/2870742895 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f208c01c640 con 0x7f2098105090 2024-03-29T08:44:46.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:46.511+0000 7f209fa00700 1 -- 172.21.15.37:0/2870742895 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2098110040 con 0x7f2098105090 2024-03-29T08:44:46.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:46.511+0000 7f209fa00700 1 -- 172.21.15.37:0/2870742895 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2098110600 con 0x7f2098105090 2024-03-29T08:44:46.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:46.512+0000 7f209fa00700 1 -- 172.21.15.37:0/2870742895 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20981023a0 con 0x7f2098105090 2024-03-29T08:44:46.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:46.513+0000 7f2094c00700 1 -- 172.21.15.37:0/2870742895 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f208c012410 con 0x7f2098105090 2024-03-29T08:44:46.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:46.514+0000 7f2094c00700 1 --2- 172.21.15.37:0/2870742895 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f208006cc20 0x7f208006f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:44:46.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:46.514+0000 7f2094c00700 1 -- 172.21.15.37:0/2870742895 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f208c08db30 con 0x7f2098105090 2024-03-29T08:44:46.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:46.514+0000 7f2097e00700 1 --2- 172.21.15.37:0/2870742895 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f208006cc20 0x7f208006f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:44:46.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:46.515+0000 7f2097e00700 1 --2- 172.21.15.37:0/2870742895 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f208006cc20 0x7f208006f0d0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f20981114f0 tx=0x7f2084009ce0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:44:46.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:46.519+0000 7f2094c00700 1 -- 172.21.15.37:0/2870742895 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f208c05b580 con 0x7f2098105090 2024-03-29T08:44:46.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:46.753+0000 7f209fa00700 1 -- 172.21.15.37:0/2870742895 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f2098111310 con 0x7f208006cc20 2024-03-29T08:44:46.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:46.755+0000 7f2094c00700 1 -- 172.21.15.37:0/2870742895 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f2098111310 con 0x7f208006cc20 2024-03-29T08:44:46.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:46.757+0000 7f209fa00700 1 -- 172.21.15.37:0/2870742895 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f208006cc20 msgr2=0x7f208006f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:44:46.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:46.757+0000 7f209fa00700 1 --2- 172.21.15.37:0/2870742895 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f208006cc20 0x7f208006f0d0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f20981114f0 tx=0x7f2084009ce0 comp rx=0 tx=0).stop 2024-03-29T08:44:46.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:46.757+0000 7f209fa00700 1 -- 172.21.15.37:0/2870742895 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2098105090 msgr2=0x7f2098115c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:44:46.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:46.757+0000 7f209fa00700 1 --2- 172.21.15.37:0/2870742895 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2098105090 0x7f2098115c50 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f208c005010 tx=0x7f208c004d10 comp rx=0 tx=0).stop 2024-03-29T08:44:46.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:46.757+0000 7f209fa00700 1 -- 172.21.15.37:0/2870742895 shutdown_connections 2024-03-29T08:44:46.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:46.757+0000 7f209fa00700 1 --2- 172.21.15.37:0/2870742895 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f208006cc20 0x7f208006f0d0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:44:46.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:46.757+0000 7f209fa00700 1 --2- 172.21.15.37:0/2870742895 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2098105090 0x7f2098115c50 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:44:46.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:46.757+0000 7f209fa00700 1 --2- 172.21.15.37:0/2870742895 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2098105a30 0x7f2098116190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:44:46.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:46.757+0000 7f209fa00700 1 -- 172.21.15.37:0/2870742895 >> 172.21.15.37:0/2870742895 conn(0x7f20980fac00 msgr2=0x7f20980fcac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:44:46.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:46.757+0000 7f209fa00700 1 -- 172.21.15.37:0/2870742895 shutdown_connections 2024-03-29T08:44:46.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:46.757+0000 7f209fa00700 1 -- 172.21.15.37:0/2870742895 wait complete. 2024-03-29T08:44:46.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:46.893+0000 7f222c800700 1 -- 172.21.15.37:0/2224629400 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f22241037a0 msgr2=0x7f22240fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:44:46.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:46.893+0000 7f222c800700 1 --2- 172.21.15.37:0/2224629400 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f22241037a0 0x7f22240fce10 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f2214004940 tx=0x7f2214009e30 comp rx=0 tx=0).stop 2024-03-29T08:44:46.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:46.893+0000 7f222c800700 1 -- 172.21.15.37:0/2224629400 shutdown_connections 2024-03-29T08:44:46.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:46.893+0000 7f222c800700 1 --2- 172.21.15.37:0/2224629400 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f22241037a0 0x7f22240fce10 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:44:46.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:46.893+0000 7f222c800700 1 --2- 172.21.15.37:0/2224629400 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2224102e00 0x7f22241031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:44:46.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:46.893+0000 7f222c800700 1 -- 172.21.15.37:0/2224629400 >> 172.21.15.37:0/2224629400 conn(0x7f22240f89b0 msgr2=0x7f22240fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:44:46.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:46.893+0000 7f222c800700 1 -- 172.21.15.37:0/2224629400 shutdown_connections 2024-03-29T08:44:46.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:46.893+0000 7f222c800700 1 -- 172.21.15.37:0/2224629400 wait complete. 2024-03-29T08:44:46.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:46.894+0000 7f222c800700 1 Processor -- start 2024-03-29T08:44:46.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:46.894+0000 7f222c800700 1 -- start start 2024-03-29T08:44:46.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:46.895+0000 7f222c800700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2224102e00 0x7f2224115af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:44:46.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:46.895+0000 7f222c800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f22241037a0 0x7f2224116030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:44:46.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:46.895+0000 7f222c800700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2224116680 con 0x7f22241037a0 2024-03-29T08:44:46.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:46.895+0000 7f222c800700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f222410fc60 con 0x7f2224102e00 2024-03-29T08:44:46.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:46.895+0000 7f2229400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f22241037a0 0x7f2224116030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:44:46.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:46.895+0000 7f2229400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f22241037a0 0x7f2224116030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59344/0 (socket says 172.21.15.37:59344) 2024-03-29T08:44:46.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:46.895+0000 7f2229400700 1 -- 172.21.15.37:0/1468530754 learned_addr learned my addr 172.21.15.37:0/1468530754 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:44:46.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:46.895+0000 7f2229e00700 1 --2- 172.21.15.37:0/1468530754 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2224102e00 0x7f2224115af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:44:46.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:46.895+0000 7f2229400700 1 -- 172.21.15.37:0/1468530754 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2224102e00 msgr2=0x7f2224115af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:44:46.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:46.895+0000 7f2229400700 1 --2- 172.21.15.37:0/1468530754 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2224102e00 0x7f2224115af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:44:46.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:46.895+0000 7f2229400700 1 -- 172.21.15.37:0/1468530754 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2214005040 con 0x7f22241037a0 2024-03-29T08:44:46.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:46.895+0000 7f2229e00700 1 --2- 172.21.15.37:0/1468530754 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2224102e00 0x7f2224115af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:44:46.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:46.896+0000 7f2229400700 1 --2- 172.21.15.37:0/1468530754 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f22241037a0 0x7f2224116030 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f2214009390 tx=0x7f2214009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:44:46.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:46.896+0000 7f221e000700 1 -- 172.21.15.37:0/1468530754 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f221400e040 con 0x7f22241037a0 2024-03-29T08:44:46.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:46.896+0000 7f221e000700 1 -- 172.21.15.37:0/1468530754 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f2214013420 con 0x7f22241037a0 2024-03-29T08:44:46.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:46.896+0000 7f222c800700 1 -- 172.21.15.37:0/1468530754 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f222410fee0 con 0x7f22241037a0 2024-03-29T08:44:46.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:46.896+0000 7f221e000700 1 -- 172.21.15.37:0/1468530754 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f221401c630 con 0x7f22241037a0 2024-03-29T08:44:46.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:46.896+0000 7f222c800700 1 -- 172.21.15.37:0/1468530754 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2224110420 con 0x7f22241037a0 2024-03-29T08:44:46.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:46.897+0000 7f222c800700 1 -- 172.21.15.37:0/1468530754 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22240fe550 con 0x7f22241037a0 2024-03-29T08:44:46.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:46.898+0000 7f221e000700 1 -- 172.21.15.37:0/1468530754 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f2214013a00 con 0x7f22241037a0 2024-03-29T08:44:46.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:46.899+0000 7f221e000700 1 --2- 172.21.15.37:0/1468530754 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f220c06cc20 0x7f220c06f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:44:46.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:46.899+0000 7f221e000700 1 -- 172.21.15.37:0/1468530754 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f221408d2c0 con 0x7f22241037a0 2024-03-29T08:44:46.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:46.899+0000 7f2229e00700 1 --2- 172.21.15.37:0/1468530754 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f220c06cc20 0x7f220c06f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:44:46.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:46.900+0000 7f2229e00700 1 --2- 172.21.15.37:0/1468530754 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f220c06cc20 0x7f220c06f0d0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f2218009fd0 tx=0x7f2218009ee0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:44:46.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:46.903+0000 7f221e000700 1 -- 172.21.15.37:0/1468530754 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f221405ad90 con 0x7f22241037a0 2024-03-29T08:44:46.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:44:46 smithi037 ceph-mon[142325]: pgmap v1017: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:44:47.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:44:46 smithi148 ceph-mon[106692]: pgmap v1017: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:44:47.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:47.126+0000 7f222c800700 1 -- 172.21.15.37:0/1468530754 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f22241111e0 con 0x7f220c06cc20 2024-03-29T08:44:47.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:47.136+0000 7f221e000700 1 -- 172.21.15.37:0/1468530754 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f22241111e0 con 0x7f220c06cc20 2024-03-29T08:44:47.136 INFO:teuthology.orchestra.run.smithi037.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-29T08:44:47.136 INFO:teuthology.orchestra.run.smithi037.stdout:alertmanager.smithi037 smithi037 *:9093,9094 running (34m) 11m ago 47m 14.3M - 0.25.0 c8568f914cd2 2f9f9746ed0f 2024-03-29T08:44:47.136 INFO:teuthology.orchestra.run.smithi037.stdout:crash.smithi037 smithi037 running (31m) 11m ago 47m 6647k - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 178d416a3186 2024-03-29T08:44:47.136 INFO:teuthology.orchestra.run.smithi037.stdout:crash.smithi148 smithi148 running (31m) 11m ago 46m 6656k - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 23666989d58f 2024-03-29T08:44:47.136 INFO:teuthology.orchestra.run.smithi037.stdout:grafana.smithi037 smithi037 *:3000 running (34m) 11m ago 47m 74.5M - 9.4.7 954c08fa6188 6b976fe49952 2024-03-29T08:44:47.136 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi037.iifsmh smithi037 running (27m) 11m ago 41m 16.4M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 2d97f562cc5b 2024-03-29T08:44:47.136 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi037.mmwgxe smithi037 running (27m) 11m ago 41m 16.7M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff ec37037f46e8 2024-03-29T08:44:47.136 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi148.qukpjh smithi148 running (11m) 11m ago 41m 10.7M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 3257219bc046 2024-03-29T08:44:47.136 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi148.sqteih smithi148 running (26m) 11m ago 41m 16.3M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 44c15c77037a 2024-03-29T08:44:47.136 INFO:teuthology.orchestra.run.smithi037.stdout:mgr.smithi037.wjikii smithi037 *:8443,9283,8765 running (36m) 11m ago 49m 510M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff c5a1417b9417 2024-03-29T08:44:47.136 INFO:teuthology.orchestra.run.smithi037.stdout:mgr.smithi148.onhgvm smithi148 *:8443,9283,8765 running (35m) 11m ago 46m 456M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff fcfdbce062bd 2024-03-29T08:44:47.137 INFO:teuthology.orchestra.run.smithi037.stdout:mon.smithi037 smithi037 running (32m) 11m ago 49m 70.2M 2048M 18.2.2-1015-gd0e392d9 1cc2f45c17ff eea2060783af 2024-03-29T08:44:47.137 INFO:teuthology.orchestra.run.smithi037.stdout:mon.smithi148 smithi148 running (32m) 11m ago 46m 50.4M 2048M 18.2.2-1015-gd0e392d9 1cc2f45c17ff ad4b2c5748be 2024-03-29T08:44:47.137 INFO:teuthology.orchestra.run.smithi037.stdout:node-exporter.smithi037 smithi037 *:9100 running (35m) 11m ago 47m 8514k - 1.5.0 0da6a335fe13 26b8ae9cb878 2024-03-29T08:44:47.137 INFO:teuthology.orchestra.run.smithi037.stdout:node-exporter.smithi148 smithi148 *:9100 running (35m) 11m ago 46m 8464k - 1.5.0 0da6a335fe13 98414e4e64ed 2024-03-29T08:44:47.137 INFO:teuthology.orchestra.run.smithi037.stdout:osd.0 smithi037 running (31m) 11m ago 45m 108M 1619M 18.2.2-1015-gd0e392d9 1cc2f45c17ff c0c584f6245c 2024-03-29T08:44:47.137 INFO:teuthology.orchestra.run.smithi037.stdout:osd.1 smithi037 running (30m) 11m ago 44m 104M 1619M 18.2.2-1015-gd0e392d9 1cc2f45c17ff a4a2ff4cc83d 2024-03-29T08:44:47.137 INFO:teuthology.orchestra.run.smithi037.stdout:osd.2 smithi037 running (29m) 11m ago 44m 85.9M 1619M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 8b1bc71aa79e 2024-03-29T08:44:47.137 INFO:teuthology.orchestra.run.smithi037.stdout:osd.3 smithi148 running (28m) 11m ago 43m 109M 2642M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 1c656ab8ff86 2024-03-29T08:44:47.137 INFO:teuthology.orchestra.run.smithi037.stdout:osd.4 smithi148 running (28m) 11m ago 43m 101M 2642M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 966a9de11d22 2024-03-29T08:44:47.137 INFO:teuthology.orchestra.run.smithi037.stdout:osd.5 smithi148 running (27m) 11m ago 42m 89.4M 2642M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 474c4f9212ce 2024-03-29T08:44:47.137 INFO:teuthology.orchestra.run.smithi037.stdout:prometheus.smithi037 smithi037 *:9095 running (34m) 11m ago 47m 61.0M - 2.43.0 a07b618ecd1d 30c0223ec589 2024-03-29T08:44:47.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:47.138+0000 7f222c800700 1 -- 172.21.15.37:0/1468530754 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f220c06cc20 msgr2=0x7f220c06f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:44:47.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:47.138+0000 7f222c800700 1 --2- 172.21.15.37:0/1468530754 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f220c06cc20 0x7f220c06f0d0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f2218009fd0 tx=0x7f2218009ee0 comp rx=0 tx=0).stop 2024-03-29T08:44:47.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:47.138+0000 7f222c800700 1 -- 172.21.15.37:0/1468530754 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f22241037a0 msgr2=0x7f2224116030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:44:47.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:47.138+0000 7f222c800700 1 --2- 172.21.15.37:0/1468530754 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f22241037a0 0x7f2224116030 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f2214009390 tx=0x7f2214009e30 comp rx=0 tx=0).stop 2024-03-29T08:44:47.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:47.138+0000 7f222c800700 1 -- 172.21.15.37:0/1468530754 shutdown_connections 2024-03-29T08:44:47.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:47.138+0000 7f222c800700 1 --2- 172.21.15.37:0/1468530754 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f220c06cc20 0x7f220c06f0d0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:44:47.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:47.138+0000 7f222c800700 1 --2- 172.21.15.37:0/1468530754 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2224102e00 0x7f2224115af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:44:47.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:47.138+0000 7f222c800700 1 --2- 172.21.15.37:0/1468530754 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f22241037a0 0x7f2224116030 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:44:47.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:47.139+0000 7f222c800700 1 -- 172.21.15.37:0/1468530754 >> 172.21.15.37:0/1468530754 conn(0x7f22240f89b0 msgr2=0x7f22240f99c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:44:47.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:47.139+0000 7f222c800700 1 -- 172.21.15.37:0/1468530754 shutdown_connections 2024-03-29T08:44:47.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:47.139+0000 7f222c800700 1 -- 172.21.15.37:0/1468530754 wait complete. 2024-03-29T08:44:47.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:47.274+0000 7fbb74c00700 1 -- 172.21.15.37:0/1431492012 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbb700fe850 msgr2=0x7fbb70106f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:44:47.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:47.274+0000 7fbb74c00700 1 --2- 172.21.15.37:0/1431492012 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbb700fe850 0x7fbb70106f20 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7fbb5c004970 tx=0x7fbb5c009e30 comp rx=0 tx=0).stop 2024-03-29T08:44:47.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:47.274+0000 7fbb74c00700 1 -- 172.21.15.37:0/1431492012 shutdown_connections 2024-03-29T08:44:47.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:47.274+0000 7fbb74c00700 1 --2- 172.21.15.37:0/1431492012 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbb700fe850 0x7fbb70106f20 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:44:47.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:47.274+0000 7fbb74c00700 1 --2- 172.21.15.37:0/1431492012 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb700fdeb0 0x7fbb700fe280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:44:47.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:47.274+0000 7fbb74c00700 1 -- 172.21.15.37:0/1431492012 >> 172.21.15.37:0/1431492012 conn(0x7fbb70072bc0 msgr2=0x7fbb70072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:44:47.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:47.274+0000 7fbb74c00700 1 -- 172.21.15.37:0/1431492012 shutdown_connections 2024-03-29T08:44:47.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:47.274+0000 7fbb74c00700 1 -- 172.21.15.37:0/1431492012 wait complete. 2024-03-29T08:44:47.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:47.275+0000 7fbb74c00700 1 Processor -- start 2024-03-29T08:44:47.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:47.275+0000 7fbb74c00700 1 -- start start 2024-03-29T08:44:47.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:47.275+0000 7fbb74c00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbb700fdeb0 0x7fbb70113a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:44:47.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:47.276+0000 7fbb74c00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb700fe850 0x7fbb70113f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:44:47.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:47.276+0000 7fbb74c00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb701145a0 con 0x7fbb700fdeb0 2024-03-29T08:44:47.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:47.276+0000 7fbb74c00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb7010d990 con 0x7fbb700fe850 2024-03-29T08:44:47.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:47.276+0000 7fbb6de00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbb700fdeb0 0x7fbb70113a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:44:47.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:47.276+0000 7fbb6de00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbb700fdeb0 0x7fbb70113a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59356/0 (socket says 172.21.15.37:59356) 2024-03-29T08:44:47.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:47.276+0000 7fbb6de00700 1 -- 172.21.15.37:0/3083068681 learned_addr learned my addr 172.21.15.37:0/3083068681 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:44:47.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:47.276+0000 7fbb6de00700 1 -- 172.21.15.37:0/3083068681 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb700fe850 msgr2=0x7fbb70113f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:44:47.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:47.276+0000 7fbb6de00700 1 --2- 172.21.15.37:0/3083068681 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb700fe850 0x7fbb70113f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:44:47.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:47.276+0000 7fbb6de00700 1 -- 172.21.15.37:0/3083068681 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb5c005040 con 0x7fbb700fdeb0 2024-03-29T08:44:47.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:47.276+0000 7fbb6de00700 1 --2- 172.21.15.37:0/3083068681 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbb700fdeb0 0x7fbb70113a10 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fbb6000eb40 tx=0x7fbb6000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:44:47.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:47.277+0000 7fbb66000700 1 -- 172.21.15.37:0/3083068681 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbb60007cc0 con 0x7fbb700fdeb0 2024-03-29T08:44:47.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:47.277+0000 7fbb66000700 1 -- 172.21.15.37:0/3083068681 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fbb6000f040 con 0x7fbb700fdeb0 2024-03-29T08:44:47.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:47.277+0000 7fbb74c00700 1 -- 172.21.15.37:0/3083068681 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb7010dc10 con 0x7fbb700fdeb0 2024-03-29T08:44:47.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:47.277+0000 7fbb66000700 1 -- 172.21.15.37:0/3083068681 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbb60009570 con 0x7fbb700fdeb0 2024-03-29T08:44:47.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:47.277+0000 7fbb74c00700 1 -- 172.21.15.37:0/3083068681 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb7010e150 con 0x7fbb700fdeb0 2024-03-29T08:44:47.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:47.278+0000 7fbb74c00700 1 -- 172.21.15.37:0/3083068681 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb70065e10 con 0x7fbb700fdeb0 2024-03-29T08:44:47.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:47.279+0000 7fbb66000700 1 -- 172.21.15.37:0/3083068681 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fbb600096d0 con 0x7fbb700fdeb0 2024-03-29T08:44:47.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:47.280+0000 7fbb66000700 1 --2- 172.21.15.37:0/3083068681 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fbb5406cc30 0x7fbb5406f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:44:47.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:47.280+0000 7fbb66000700 1 -- 172.21.15.37:0/3083068681 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fbb6008bda0 con 0x7fbb700fdeb0 2024-03-29T08:44:47.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:47.280+0000 7fbb6d400700 1 --2- 172.21.15.37:0/3083068681 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fbb5406cc30 0x7fbb5406f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:44:47.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:47.281+0000 7fbb6d400700 1 --2- 172.21.15.37:0/3083068681 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fbb5406cc30 0x7fbb5406f0e0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fbb5c00b370 tx=0x7fbb5c009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:44:47.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:47.285+0000 7fbb66000700 1 -- 172.21.15.37:0/3083068681 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fbb60059820 con 0x7fbb700fdeb0 2024-03-29T08:44:47.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:47.585+0000 7fbb74c00700 1 -- 172.21.15.37:0/3083068681 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fbb7004bed0 con 0x7fbb700fdeb0 2024-03-29T08:44:47.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:47.586+0000 7fbb66000700 1 -- 172.21.15.37:0/3083068681 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7fbb600593b0 con 0x7fbb700fdeb0 2024-03-29T08:44:47.586 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-29T08:44:47.586 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": { 2024-03-29T08:44:47.586 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 2 2024-03-29T08:44:47.586 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:44:47.586 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": { 2024-03-29T08:44:47.586 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 2 2024-03-29T08:44:47.586 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:44:47.586 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": { 2024-03-29T08:44:47.586 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 6 2024-03-29T08:44:47.586 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:44:47.586 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": { 2024-03-29T08:44:47.586 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 4 2024-03-29T08:44:47.586 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:44:47.587 INFO:teuthology.orchestra.run.smithi037.stdout: "overall": { 2024-03-29T08:44:47.587 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 14 2024-03-29T08:44:47.587 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-29T08:44:47.587 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-29T08:44:47.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:47.588+0000 7fbb74c00700 1 -- 172.21.15.37:0/3083068681 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fbb5406cc30 msgr2=0x7fbb5406f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:44:47.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:47.588+0000 7fbb74c00700 1 --2- 172.21.15.37:0/3083068681 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fbb5406cc30 0x7fbb5406f0e0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fbb5c00b370 tx=0x7fbb5c009e30 comp rx=0 tx=0).stop 2024-03-29T08:44:47.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:47.588+0000 7fbb74c00700 1 -- 172.21.15.37:0/3083068681 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbb700fdeb0 msgr2=0x7fbb70113a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:44:47.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:47.588+0000 7fbb74c00700 1 --2- 172.21.15.37:0/3083068681 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbb700fdeb0 0x7fbb70113a10 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fbb6000eb40 tx=0x7fbb6000b040 comp rx=0 tx=0).stop 2024-03-29T08:44:47.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:47.588+0000 7fbb74c00700 1 -- 172.21.15.37:0/3083068681 shutdown_connections 2024-03-29T08:44:47.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:47.588+0000 7fbb74c00700 1 --2- 172.21.15.37:0/3083068681 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fbb5406cc30 0x7fbb5406f0e0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:44:47.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:47.588+0000 7fbb74c00700 1 --2- 172.21.15.37:0/3083068681 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbb700fdeb0 0x7fbb70113a10 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:44:47.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:47.588+0000 7fbb74c00700 1 --2- 172.21.15.37:0/3083068681 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb700fe850 0x7fbb70113f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:44:47.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:47.588+0000 7fbb74c00700 1 -- 172.21.15.37:0/3083068681 >> 172.21.15.37:0/3083068681 conn(0x7fbb70072bc0 msgr2=0x7fbb70103780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:44:47.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:47.588+0000 7fbb74c00700 1 -- 172.21.15.37:0/3083068681 shutdown_connections 2024-03-29T08:44:47.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:47.588+0000 7fbb74c00700 1 -- 172.21.15.37:0/3083068681 wait complete. 2024-03-29T08:44:47.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:47.726+0000 7f89aa000700 1 -- 172.21.15.37:0/626295536 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f89a41037a0 msgr2=0x7f89a40fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:44:47.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:47.726+0000 7f89aa000700 1 --2- 172.21.15.37:0/626295536 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f89a41037a0 0x7f89a40fce10 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f8990004940 tx=0x7f8990009e30 comp rx=0 tx=0).stop 2024-03-29T08:44:47.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:47.726+0000 7f89aa000700 1 -- 172.21.15.37:0/626295536 shutdown_connections 2024-03-29T08:44:47.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:47.726+0000 7f89aa000700 1 --2- 172.21.15.37:0/626295536 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f89a41037a0 0x7f89a40fce10 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:44:47.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:47.726+0000 7f89aa000700 1 --2- 172.21.15.37:0/626295536 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f89a4102e00 0x7f89a41031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:44:47.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:47.726+0000 7f89aa000700 1 -- 172.21.15.37:0/626295536 >> 172.21.15.37:0/626295536 conn(0x7f89a40f89b0 msgr2=0x7f89a40fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:44:47.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:47.726+0000 7f89aa000700 1 -- 172.21.15.37:0/626295536 shutdown_connections 2024-03-29T08:44:47.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:47.726+0000 7f89aa000700 1 -- 172.21.15.37:0/626295536 wait complete. 2024-03-29T08:44:47.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:47.727+0000 7f89aa000700 1 Processor -- start 2024-03-29T08:44:47.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:47.727+0000 7f89aa000700 1 -- start start 2024-03-29T08:44:47.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:47.728+0000 7f89aa000700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f89a4102e00 0x7f89a4115cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:44:47.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:47.728+0000 7f89aa000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f89a41037a0 0x7f89a4116230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:44:47.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:47.728+0000 7f89aa000700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89a4116880 con 0x7f89a41037a0 2024-03-29T08:44:47.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:47.728+0000 7f89aa000700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89a410fc50 con 0x7f89a4102e00 2024-03-29T08:44:47.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:47.728+0000 7f89a2a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f89a41037a0 0x7f89a4116230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:44:47.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:47.728+0000 7f89a2a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f89a41037a0 0x7f89a4116230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59376/0 (socket says 172.21.15.37:59376) 2024-03-29T08:44:47.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:47.728+0000 7f89a2a00700 1 -- 172.21.15.37:0/1482013946 learned_addr learned my addr 172.21.15.37:0/1482013946 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:44:47.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:47.728+0000 7f89a3400700 1 --2- 172.21.15.37:0/1482013946 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f89a4102e00 0x7f89a4115cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:44:47.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:47.728+0000 7f89a2a00700 1 -- 172.21.15.37:0/1482013946 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f89a4102e00 msgr2=0x7f89a4115cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:44:47.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:47.728+0000 7f89a2a00700 1 --2- 172.21.15.37:0/1482013946 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f89a4102e00 0x7f89a4115cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:44:47.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:47.728+0000 7f89a2a00700 1 -- 172.21.15.37:0/1482013946 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8990005040 con 0x7f89a41037a0 2024-03-29T08:44:47.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:47.728+0000 7f89a3400700 1 --2- 172.21.15.37:0/1482013946 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f89a4102e00 0x7f89a4115cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:44:47.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:47.729+0000 7f89a2a00700 1 --2- 172.21.15.37:0/1482013946 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f89a41037a0 0x7f89a4116230 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f8990009390 tx=0x7f8990009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:44:47.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:47.729+0000 7f899b400700 1 -- 172.21.15.37:0/1482013946 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f899000e040 con 0x7f89a41037a0 2024-03-29T08:44:47.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:47.729+0000 7f899b400700 1 -- 172.21.15.37:0/1482013946 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f8990013420 con 0x7f89a41037a0 2024-03-29T08:44:47.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:47.729+0000 7f899b400700 1 -- 172.21.15.37:0/1482013946 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f899001c630 con 0x7f89a41037a0 2024-03-29T08:44:47.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:47.729+0000 7f89aa000700 1 -- 172.21.15.37:0/1482013946 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89a410fed0 con 0x7f89a41037a0 2024-03-29T08:44:47.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:47.729+0000 7f89aa000700 1 -- 172.21.15.37:0/1482013946 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89a4110490 con 0x7f89a41037a0 2024-03-29T08:44:47.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:47.730+0000 7f89aa000700 1 -- 172.21.15.37:0/1482013946 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89a40fe550 con 0x7f89a41037a0 2024-03-29T08:44:47.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:47.731+0000 7f899b400700 1 -- 172.21.15.37:0/1482013946 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f8990013a00 con 0x7f89a41037a0 2024-03-29T08:44:47.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:47.732+0000 7f899b400700 1 --2- 172.21.15.37:0/1482013946 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f898806cc30 0x7f898806f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:44:47.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:47.732+0000 7f899b400700 1 -- 172.21.15.37:0/1482013946 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f899008d2c0 con 0x7f89a41037a0 2024-03-29T08:44:47.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:47.732+0000 7f89a3400700 1 --2- 172.21.15.37:0/1482013946 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f898806cc30 0x7f898806f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:44:47.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:47.733+0000 7f89a3400700 1 --2- 172.21.15.37:0/1482013946 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f898806cc30 0x7f898806f0e0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f8994009fd0 tx=0x7f8994009ee0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:44:47.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:47.737+0000 7f899b400700 1 -- 172.21.15.37:0/1482013946 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f899005ad90 con 0x7f89a41037a0 2024-03-29T08:44:47.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:44:47 smithi037 ceph-mon[142325]: from='client.35822 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:44:47.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:44:47 smithi037 ceph-mon[142325]: from='client.35826 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:44:47.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:47.998+0000 7f89aa000700 1 -- 172.21.15.37:0/1482013946 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f89a404bed0 con 0x7f89a41037a0 2024-03-29T08:44:47.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:47.999+0000 7f899b400700 1 -- 172.21.15.37:0/1482013946 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 30 v30) v1 ==== 76+0+1841 (secure 0 0 0) 0x7f899005a920 con 0x7f89a41037a0 2024-03-29T08:44:47.999 INFO:teuthology.orchestra.run.smithi037.stdout:e30 2024-03-29T08:44:47.999 INFO:teuthology.orchestra.run.smithi037.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-29T08:44:47.999 INFO:teuthology.orchestra.run.smithi037.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T08:44:47.999 INFO:teuthology.orchestra.run.smithi037.stdout:legacy client fscid: 1 2024-03-29T08:44:47.999 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:44:47.999 INFO:teuthology.orchestra.run.smithi037.stdout:Filesystem 'cephfs' (1) 2024-03-29T08:44:47.999 INFO:teuthology.orchestra.run.smithi037.stdout:fs_name cephfs 2024-03-29T08:44:47.999 INFO:teuthology.orchestra.run.smithi037.stdout:epoch 21 2024-03-29T08:44:47.999 INFO:teuthology.orchestra.run.smithi037.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-03-29T08:44:47.999 INFO:teuthology.orchestra.run.smithi037.stdout:created 2024-03-29T08:03:29.241269+0000 2024-03-29T08:44:47.999 INFO:teuthology.orchestra.run.smithi037.stdout:modified 2024-03-29T08:17:16.237238+0000 2024-03-29T08:44:48.000 INFO:teuthology.orchestra.run.smithi037.stdout:tableserver 0 2024-03-29T08:44:48.000 INFO:teuthology.orchestra.run.smithi037.stdout:root 0 2024-03-29T08:44:48.000 INFO:teuthology.orchestra.run.smithi037.stdout:session_timeout 60 2024-03-29T08:44:48.000 INFO:teuthology.orchestra.run.smithi037.stdout:session_autoclose 300 2024-03-29T08:44:48.000 INFO:teuthology.orchestra.run.smithi037.stdout:max_file_size 1099511627776 2024-03-29T08:44:48.000 INFO:teuthology.orchestra.run.smithi037.stdout:max_xattr_size 65536 2024-03-29T08:44:48.000 INFO:teuthology.orchestra.run.smithi037.stdout:required_client_features {} 2024-03-29T08:44:48.000 INFO:teuthology.orchestra.run.smithi037.stdout:last_failure 0 2024-03-29T08:44:48.000 INFO:teuthology.orchestra.run.smithi037.stdout:last_failure_osd_epoch 82 2024-03-29T08:44:48.000 INFO:teuthology.orchestra.run.smithi037.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T08:44:48.000 INFO:teuthology.orchestra.run.smithi037.stdout:max_mds 2 2024-03-29T08:44:48.000 INFO:teuthology.orchestra.run.smithi037.stdout:in 0,1 2024-03-29T08:44:48.000 INFO:teuthology.orchestra.run.smithi037.stdout:up {} 2024-03-29T08:44:48.000 INFO:teuthology.orchestra.run.smithi037.stdout:failed 0,1 2024-03-29T08:44:48.000 INFO:teuthology.orchestra.run.smithi037.stdout:damaged 2024-03-29T08:44:48.000 INFO:teuthology.orchestra.run.smithi037.stdout:stopped 2024-03-29T08:44:48.000 INFO:teuthology.orchestra.run.smithi037.stdout:data_pools [3] 2024-03-29T08:44:48.001 INFO:teuthology.orchestra.run.smithi037.stdout:metadata_pool 2 2024-03-29T08:44:48.001 INFO:teuthology.orchestra.run.smithi037.stdout:inline_data disabled 2024-03-29T08:44:48.001 INFO:teuthology.orchestra.run.smithi037.stdout:balancer 2024-03-29T08:44:48.001 INFO:teuthology.orchestra.run.smithi037.stdout:bal_rank_mask -1 2024-03-29T08:44:48.001 INFO:teuthology.orchestra.run.smithi037.stdout:standby_count_wanted 1 2024-03-29T08:44:48.001 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:44:48.001 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:44:48.001 INFO:teuthology.orchestra.run.smithi037.stdout:Standby daemons: 2024-03-29T08:44:48.001 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:44:48.001 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi037.mmwgxe{-1:34440} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.37:6828/1248142005,v1:172.21.15.37:6829/1248142005] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:44:48.001 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi037.iifsmh{-1:34444} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.37:6826/2060882959,v1:172.21.15.37:6827/2060882959] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:44:48.001 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi148.sqteih{-1:44395} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.148:6824/4078652242,v1:172.21.15.148:6825/4078652242] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:44:48.001 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi148.qukpjh{-1:45007} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.148:6826/2394751105,v1:172.21.15.148:6827/2394751105] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:44:48.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:48.001+0000 7f89aa000700 1 -- 172.21.15.37:0/1482013946 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f898806cc30 msgr2=0x7f898806f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:44:48.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:48.001+0000 7f89aa000700 1 --2- 172.21.15.37:0/1482013946 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f898806cc30 0x7f898806f0e0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f8994009fd0 tx=0x7f8994009ee0 comp rx=0 tx=0).stop 2024-03-29T08:44:48.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:48.001+0000 7f89aa000700 1 -- 172.21.15.37:0/1482013946 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f89a41037a0 msgr2=0x7f89a4116230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:44:48.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:48.001+0000 7f89aa000700 1 --2- 172.21.15.37:0/1482013946 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f89a41037a0 0x7f89a4116230 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f8990009390 tx=0x7f8990009e30 comp rx=0 tx=0).stop 2024-03-29T08:44:48.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:48.001+0000 7f89aa000700 1 -- 172.21.15.37:0/1482013946 shutdown_connections 2024-03-29T08:44:48.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:48.001+0000 7f89aa000700 1 --2- 172.21.15.37:0/1482013946 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f898806cc30 0x7f898806f0e0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:44:48.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:48.001+0000 7f89aa000700 1 --2- 172.21.15.37:0/1482013946 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f89a4102e00 0x7f89a4115cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:44:48.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:48.001+0000 7f89aa000700 1 --2- 172.21.15.37:0/1482013946 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f89a41037a0 0x7f89a4116230 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:44:48.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:48.001+0000 7f89aa000700 1 -- 172.21.15.37:0/1482013946 >> 172.21.15.37:0/1482013946 conn(0x7f89a40f89b0 msgr2=0x7f89a40f9a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:44:48.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:48.002+0000 7f89aa000700 1 -- 172.21.15.37:0/1482013946 shutdown_connections 2024-03-29T08:44:48.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:48.002+0000 7f89aa000700 1 -- 172.21.15.37:0/1482013946 wait complete. 2024-03-29T08:44:48.003 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 30 2024-03-29T08:44:48.055 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:44:47 smithi148 ceph-mon[106692]: from='client.35822 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:44:48.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:44:47 smithi148 ceph-mon[106692]: from='client.35826 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:44:48.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:48.140+0000 7ff939c00700 1 -- 172.21.15.37:0/1494801552 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff9340fe830 msgr2=0x7ff934106f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:44:48.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:48.140+0000 7ff939c00700 1 --2- 172.21.15.37:0/1494801552 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff9340fe830 0x7ff934106f00 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7ff920004940 tx=0x7ff920009e30 comp rx=0 tx=0).stop 2024-03-29T08:44:48.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:48.140+0000 7ff939c00700 1 -- 172.21.15.37:0/1494801552 shutdown_connections 2024-03-29T08:44:48.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:48.140+0000 7ff939c00700 1 --2- 172.21.15.37:0/1494801552 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff9340fe830 0x7ff934106f00 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:44:48.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:48.140+0000 7ff939c00700 1 --2- 172.21.15.37:0/1494801552 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff9340fde90 0x7ff9340fe260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:44:48.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:48.140+0000 7ff939c00700 1 -- 172.21.15.37:0/1494801552 >> 172.21.15.37:0/1494801552 conn(0x7ff934072bc0 msgr2=0x7ff934072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:44:48.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:48.140+0000 7ff939c00700 1 -- 172.21.15.37:0/1494801552 shutdown_connections 2024-03-29T08:44:48.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:48.140+0000 7ff939c00700 1 -- 172.21.15.37:0/1494801552 wait complete. 2024-03-29T08:44:48.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:48.141+0000 7ff939c00700 1 Processor -- start 2024-03-29T08:44:48.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:48.141+0000 7ff939c00700 1 -- start start 2024-03-29T08:44:48.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:48.141+0000 7ff939c00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff9340fde90 0x7ff934115d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:44:48.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:48.141+0000 7ff939c00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff9340fe830 0x7ff934116250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:44:48.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:48.141+0000 7ff939c00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9341168a0 con 0x7ff9340fde90 2024-03-29T08:44:48.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:48.141+0000 7ff939c00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff93410fc70 con 0x7ff9340fe830 2024-03-29T08:44:48.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:48.142+0000 7ff932a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff9340fde90 0x7ff934115d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:44:48.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:48.142+0000 7ff932a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff9340fde90 0x7ff934115d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59402/0 (socket says 172.21.15.37:59402) 2024-03-29T08:44:48.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:48.142+0000 7ff932a00700 1 -- 172.21.15.37:0/2170656386 learned_addr learned my addr 172.21.15.37:0/2170656386 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:44:48.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:48.142+0000 7ff932000700 1 --2- 172.21.15.37:0/2170656386 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff9340fe830 0x7ff934116250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:44:48.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:48.142+0000 7ff932a00700 1 -- 172.21.15.37:0/2170656386 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff9340fe830 msgr2=0x7ff934116250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:44:48.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:48.142+0000 7ff932a00700 1 --2- 172.21.15.37:0/2170656386 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff9340fe830 0x7ff934116250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:44:48.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:48.142+0000 7ff932a00700 1 -- 172.21.15.37:0/2170656386 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff920005040 con 0x7ff9340fde90 2024-03-29T08:44:48.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:48.142+0000 7ff932000700 1 --2- 172.21.15.37:0/2170656386 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff9340fe830 0x7ff934116250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:44:48.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:48.142+0000 7ff932a00700 1 --2- 172.21.15.37:0/2170656386 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff9340fde90 0x7ff934115d10 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7ff92400c8c0 tx=0x7ff9240044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:44:48.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:48.143+0000 7ff92aa00700 1 -- 172.21.15.37:0/2170656386 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff92400a980 con 0x7ff9340fde90 2024-03-29T08:44:48.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:48.143+0000 7ff92aa00700 1 -- 172.21.15.37:0/2170656386 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ff92400ce60 con 0x7ff9340fde90 2024-03-29T08:44:48.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:48.143+0000 7ff92aa00700 1 -- 172.21.15.37:0/2170656386 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff9240113a0 con 0x7ff9340fde90 2024-03-29T08:44:48.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:48.143+0000 7ff939c00700 1 -- 172.21.15.37:0/2170656386 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff93410ff50 con 0x7ff9340fde90 2024-03-29T08:44:48.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:48.143+0000 7ff939c00700 1 -- 172.21.15.37:0/2170656386 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff934110490 con 0x7ff9340fde90 2024-03-29T08:44:48.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:48.144+0000 7ff939c00700 1 -- 172.21.15.37:0/2170656386 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff934106a20 con 0x7ff9340fde90 2024-03-29T08:44:48.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:48.145+0000 7ff92aa00700 1 -- 172.21.15.37:0/2170656386 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7ff924012400 con 0x7ff9340fde90 2024-03-29T08:44:48.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:48.146+0000 7ff92aa00700 1 --2- 172.21.15.37:0/2170656386 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7ff91806cc30 0x7ff91806f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:44:48.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:48.146+0000 7ff92aa00700 1 -- 172.21.15.37:0/2170656386 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7ff92408c0f0 con 0x7ff9340fde90 2024-03-29T08:44:48.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:48.146+0000 7ff932000700 1 --2- 172.21.15.37:0/2170656386 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7ff91806cc30 0x7ff91806f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:44:48.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:48.147+0000 7ff932000700 1 --2- 172.21.15.37:0/2170656386 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7ff91806cc30 0x7ff91806f0e0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7ff92000b3b0 tx=0x7ff920009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:44:48.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:48.150+0000 7ff92aa00700 1 -- 172.21.15.37:0/2170656386 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7ff924059bc0 con 0x7ff9340fde90 2024-03-29T08:44:48.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:48.382+0000 7ff939c00700 1 -- 172.21.15.37:0/2170656386 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff934109dd0 con 0x7ff91806cc30 2024-03-29T08:44:48.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:48.383+0000 7ff92aa00700 1 -- 172.21.15.37:0/2170656386 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7ff934109dd0 con 0x7ff91806cc30 2024-03-29T08:44:48.383 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-29T08:44:48.383 INFO:teuthology.orchestra.run.smithi037.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59", 2024-03-29T08:44:48.383 INFO:teuthology.orchestra.run.smithi037.stdout: "in_progress": true, 2024-03-29T08:44:48.383 INFO:teuthology.orchestra.run.smithi037.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-29T08:44:48.383 INFO:teuthology.orchestra.run.smithi037.stdout: "services_complete": [ 2024-03-29T08:44:48.383 INFO:teuthology.orchestra.run.smithi037.stdout: "crash", 2024-03-29T08:44:48.383 INFO:teuthology.orchestra.run.smithi037.stdout: "osd", 2024-03-29T08:44:48.383 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr", 2024-03-29T08:44:48.384 INFO:teuthology.orchestra.run.smithi037.stdout: "mds", 2024-03-29T08:44:48.384 INFO:teuthology.orchestra.run.smithi037.stdout: "mon" 2024-03-29T08:44:48.384 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-29T08:44:48.384 INFO:teuthology.orchestra.run.smithi037.stdout: "progress": "16/21 daemons upgraded", 2024-03-29T08:44:48.384 INFO:teuthology.orchestra.run.smithi037.stdout: "message": "Currently upgrading mds daemons", 2024-03-29T08:44:48.384 INFO:teuthology.orchestra.run.smithi037.stdout: "is_paused": false 2024-03-29T08:44:48.384 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-29T08:44:48.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:48.386+0000 7ff939c00700 1 -- 172.21.15.37:0/2170656386 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7ff91806cc30 msgr2=0x7ff91806f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:44:48.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:48.386+0000 7ff939c00700 1 --2- 172.21.15.37:0/2170656386 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7ff91806cc30 0x7ff91806f0e0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7ff92000b3b0 tx=0x7ff920009e30 comp rx=0 tx=0).stop 2024-03-29T08:44:48.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:48.386+0000 7ff939c00700 1 -- 172.21.15.37:0/2170656386 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff9340fde90 msgr2=0x7ff934115d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:44:48.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:48.386+0000 7ff939c00700 1 --2- 172.21.15.37:0/2170656386 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff9340fde90 0x7ff934115d10 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7ff92400c8c0 tx=0x7ff9240044c0 comp rx=0 tx=0).stop 2024-03-29T08:44:48.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:48.386+0000 7ff939c00700 1 -- 172.21.15.37:0/2170656386 shutdown_connections 2024-03-29T08:44:48.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:48.386+0000 7ff939c00700 1 --2- 172.21.15.37:0/2170656386 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7ff91806cc30 0x7ff91806f0e0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:44:48.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:48.386+0000 7ff939c00700 1 --2- 172.21.15.37:0/2170656386 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff9340fde90 0x7ff934115d10 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:44:48.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:48.386+0000 7ff939c00700 1 --2- 172.21.15.37:0/2170656386 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff9340fe830 0x7ff934116250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:44:48.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:48.386+0000 7ff939c00700 1 -- 172.21.15.37:0/2170656386 >> 172.21.15.37:0/2170656386 conn(0x7ff934072bc0 msgr2=0x7ff9340fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:44:48.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:48.386+0000 7ff939c00700 1 -- 172.21.15.37:0/2170656386 shutdown_connections 2024-03-29T08:44:48.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:48.386+0000 7ff939c00700 1 -- 172.21.15.37:0/2170656386 wait complete. 2024-03-29T08:44:48.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:48.523+0000 7f8553600700 1 -- 172.21.15.37:0/742658532 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f854c102e00 msgr2=0x7f854c1031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:44:48.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:48.523+0000 7f8553600700 1 --2- 172.21.15.37:0/742658532 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f854c102e00 0x7f854c1031d0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f8540004940 tx=0x7f8540009e30 comp rx=0 tx=0).stop 2024-03-29T08:44:48.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:48.523+0000 7f8553600700 1 -- 172.21.15.37:0/742658532 shutdown_connections 2024-03-29T08:44:48.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:48.523+0000 7f8553600700 1 --2- 172.21.15.37:0/742658532 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f854c1037a0 0x7f854c0fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:44:48.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:48.523+0000 7f8553600700 1 --2- 172.21.15.37:0/742658532 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f854c102e00 0x7f854c1031d0 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:44:48.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:48.523+0000 7f8553600700 1 -- 172.21.15.37:0/742658532 >> 172.21.15.37:0/742658532 conn(0x7f854c0f89b0 msgr2=0x7f854c0fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:44:48.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:48.523+0000 7f8553600700 1 -- 172.21.15.37:0/742658532 shutdown_connections 2024-03-29T08:44:48.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:48.523+0000 7f8553600700 1 -- 172.21.15.37:0/742658532 wait complete. 2024-03-29T08:44:48.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:48.524+0000 7f8553600700 1 Processor -- start 2024-03-29T08:44:48.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:48.524+0000 7f8553600700 1 -- start start 2024-03-29T08:44:48.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:48.524+0000 7f8553600700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f854c102e00 0x7f854c115c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:44:48.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:48.524+0000 7f8553600700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f854c1037a0 0x7f854c1161b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:44:48.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:48.524+0000 7f8553600700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f854c116800 con 0x7f854c102e00 2024-03-29T08:44:48.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:48.524+0000 7f8553600700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f854c10fbd0 con 0x7f854c1037a0 2024-03-29T08:44:48.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:48.524+0000 7f8550c00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f854c102e00 0x7f854c115c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:44:48.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:48.525+0000 7f8550c00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f854c102e00 0x7f854c115c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59420/0 (socket says 172.21.15.37:59420) 2024-03-29T08:44:48.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:48.525+0000 7f8550c00700 1 -- 172.21.15.37:0/1282359284 learned_addr learned my addr 172.21.15.37:0/1282359284 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:44:48.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:48.525+0000 7f854be00700 1 --2- 172.21.15.37:0/1282359284 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f854c1037a0 0x7f854c1161b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:44:48.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:48.525+0000 7f8550c00700 1 -- 172.21.15.37:0/1282359284 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f854c1037a0 msgr2=0x7f854c1161b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:44:48.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:48.525+0000 7f8550c00700 1 --2- 172.21.15.37:0/1282359284 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f854c1037a0 0x7f854c1161b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:44:48.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:48.525+0000 7f8550c00700 1 -- 172.21.15.37:0/1282359284 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8540005040 con 0x7f854c102e00 2024-03-29T08:44:48.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:48.525+0000 7f8550c00700 1 --2- 172.21.15.37:0/1282359284 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f854c102e00 0x7f854c115c70 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f8540007f50 tx=0x7f8540009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:44:48.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:48.525+0000 7f8548c00700 1 -- 172.21.15.37:0/1282359284 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8540021070 con 0x7f854c102e00 2024-03-29T08:44:48.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:48.526+0000 7f8548c00700 1 -- 172.21.15.37:0/1282359284 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f85400093b0 con 0x7f854c102e00 2024-03-29T08:44:48.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:48.526+0000 7f8548c00700 1 -- 172.21.15.37:0/1282359284 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f854000e070 con 0x7f854c102e00 2024-03-29T08:44:48.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:48.526+0000 7f8553600700 1 -- 172.21.15.37:0/1282359284 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f854c10fe50 con 0x7f854c102e00 2024-03-29T08:44:48.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:48.526+0000 7f8553600700 1 -- 172.21.15.37:0/1282359284 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f854c1103b0 con 0x7f854c102e00 2024-03-29T08:44:48.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:48.527+0000 7f8553600700 1 -- 172.21.15.37:0/1282359284 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f854c0fe550 con 0x7f854c102e00 2024-03-29T08:44:48.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:48.528+0000 7f8548c00700 1 -- 172.21.15.37:0/1282359284 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f854001c400 con 0x7f854c102e00 2024-03-29T08:44:48.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:48.529+0000 7f8548c00700 1 --2- 172.21.15.37:0/1282359284 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f853406cc30 0x7f853406f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:44:48.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:48.529+0000 7f8548c00700 1 -- 172.21.15.37:0/1282359284 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f854008d120 con 0x7f854c102e00 2024-03-29T08:44:48.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:48.529+0000 7f854be00700 1 --2- 172.21.15.37:0/1282359284 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f853406cc30 0x7f853406f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:44:48.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:48.529+0000 7f854be00700 1 --2- 172.21.15.37:0/1282359284 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f853406cc30 0x7f853406f0e0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f854c111510 tx=0x7f853800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:44:48.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:48.533+0000 7f8548c00700 1 -- 172.21.15.37:0/1282359284 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f854005abf0 con 0x7f854c102e00 2024-03-29T08:44:48.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:48.831+0000 7f8553600700 1 -- 172.21.15.37:0/1282359284 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f854c04bed0 con 0x7f854c102e00 2024-03-29T08:44:48.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:48.831+0000 7f8548c00700 1 -- 172.21.15.37:0/1282359284 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+3583 (secure 0 0 0) 0x7f854005a780 con 0x7f854c102e00 2024-03-29T08:44:48.831 INFO:teuthology.orchestra.run.smithi037.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem is offline 2024-03-29T08:44:48.831 INFO:teuthology.orchestra.run.smithi037.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-03-29T08:44:48.831 INFO:teuthology.orchestra.run.smithi037.stdout: host smithi148 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/mon.smithi148/config 2024-03-29T08:44:48.832 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/podman: timeout after 895 seconds 2024-03-29T08:44:48.832 INFO:teuthology.orchestra.run.smithi037.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59 -e NODE_NAME=smithi148 -e CEPH_USE_RANDOM_NONCE=1 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4:/var/run/ceph:z -v /var/log/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4:/var/log/ceph:z -v /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpkx2rz2d4:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59 inventory --format=json-pretty --filter-for-batch 2024-03-29T08:44:48.832 INFO:teuthology.orchestra.run.smithi037.stdout:Traceback (most recent call last): 2024-03-29T08:44:48.832 INFO:teuthology.orchestra.run.smithi037.stdout: File "/usr/lib64/python3.6/runpy.py", line 193, in _run_module_as_main 2024-03-29T08:44:48.832 INFO:teuthology.orchestra.run.smithi037.stdout: "__main__", mod_spec) 2024-03-29T08:44:48.832 INFO:teuthology.orchestra.run.smithi037.stdout: File "/usr/lib64/python3.6/runpy.py", line 85, in _run_code 2024-03-29T08:44:48.832 INFO:teuthology.orchestra.run.smithi037.stdout: exec(code, run_globals) 2024-03-29T08:44:48.832 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 10889, in 2024-03-29T08:44:48.832 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 10877, in main 2024-03-29T08:44:48.832 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 2576, in _infer_config 2024-03-29T08:44:48.832 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 2492, in _infer_fsid 2024-03-29T08:44:48.832 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 2604, in _infer_image 2024-03-29T08:44:48.832 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 2479, in _validate_fsid 2024-03-29T08:44:48.832 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 7145, in command_ceph_volume 2024-03-29T08:44:48.833 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 2267, in call_throws 2024-03-29T08:44:48.833 INFO:teuthology.orchestra.run.smithi037.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59 -e NODE_NAME=smithi148 -e CEPH_USE_RANDOM_NONCE=1 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4:/var/run/ceph:z -v /var/log/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4:/var/log/ceph:z -v /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpkx2rz2d4:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59 inventory --format=json-pretty --filter-for-batch 2024-03-29T08:44:48.833 INFO:teuthology.orchestra.run.smithi037.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-03-29T08:44:48.833 INFO:teuthology.orchestra.run.smithi037.stdout: fs cephfs is degraded 2024-03-29T08:44:48.833 INFO:teuthology.orchestra.run.smithi037.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-03-29T08:44:48.833 INFO:teuthology.orchestra.run.smithi037.stdout: fs cephfs is offline because no MDS is active for it. 2024-03-29T08:44:48.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:48.834+0000 7f8553600700 1 -- 172.21.15.37:0/1282359284 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f853406cc30 msgr2=0x7f853406f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:44:48.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:48.834+0000 7f8553600700 1 --2- 172.21.15.37:0/1282359284 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f853406cc30 0x7f853406f0e0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f854c111510 tx=0x7f853800b040 comp rx=0 tx=0).stop 2024-03-29T08:44:48.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:48.834+0000 7f8553600700 1 -- 172.21.15.37:0/1282359284 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f854c102e00 msgr2=0x7f854c115c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:44:48.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:48.834+0000 7f8553600700 1 --2- 172.21.15.37:0/1282359284 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f854c102e00 0x7f854c115c70 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f8540007f50 tx=0x7f8540009e30 comp rx=0 tx=0).stop 2024-03-29T08:44:48.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:48.834+0000 7f8553600700 1 -- 172.21.15.37:0/1282359284 shutdown_connections 2024-03-29T08:44:48.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:48.834+0000 7f8553600700 1 --2- 172.21.15.37:0/1282359284 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f853406cc30 0x7f853406f0e0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:44:48.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:48.834+0000 7f8553600700 1 --2- 172.21.15.37:0/1282359284 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f854c102e00 0x7f854c115c70 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:44:48.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:48.834+0000 7f8553600700 1 --2- 172.21.15.37:0/1282359284 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f854c1037a0 0x7f854c1161b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:44:48.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:48.834+0000 7f8553600700 1 -- 172.21.15.37:0/1282359284 >> 172.21.15.37:0/1282359284 conn(0x7f854c0f89b0 msgr2=0x7f854c0f9960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:44:48.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:48.834+0000 7f8553600700 1 -- 172.21.15.37:0/1282359284 shutdown_connections 2024-03-29T08:44:48.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:44:48.834+0000 7f8553600700 1 -- 172.21.15.37:0/1282359284 wait complete. 2024-03-29T08:44:48.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:44:48 smithi037 ceph-mon[142325]: from='client.35830 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:44:48.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:44:48 smithi037 ceph-mon[142325]: pgmap v1018: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:44:48.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:44:48 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/3083068681' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:44:48.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:44:48 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/1482013946' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T08:44:49.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:44:48 smithi148 ceph-mon[106692]: from='client.35830 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:44:49.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:44:48 smithi148 ceph-mon[106692]: pgmap v1018: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:44:49.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:44:48 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/3083068681' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:44:49.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:44:48 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/1482013946' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T08:44:49.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:44:49 smithi037 ceph-mon[142325]: from='client.35842 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:44:49.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:44:49 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/1282359284' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T08:44:50.055 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:44:49 smithi148 ceph-mon[106692]: from='client.35842 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:44:50.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:44:49 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/1282359284' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T08:44:50.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:44:50 smithi037 ceph-mon[142325]: pgmap v1019: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:44:51.055 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:44:50 smithi148 ceph-mon[106692]: pgmap v1019: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:44:52.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:44:52 smithi037 ceph-mon[142325]: pgmap v1020: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:44:53.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:44:52 smithi148 ceph-mon[106692]: pgmap v1020: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:44:54.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:44:54 smithi037 ceph-mon[142325]: pgmap v1021: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:44:55.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:44:54 smithi148 ceph-mon[106692]: pgmap v1021: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:44:55.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:44:55 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:44:56.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:44:55 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:44:56.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:44:56 smithi037 ceph-mon[142325]: pgmap v1022: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:44:57.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:44:56 smithi148 ceph-mon[106692]: pgmap v1022: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:44:58.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:44:58 smithi037 ceph-mon[142325]: pgmap v1023: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:44:59.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:44:58 smithi148 ceph-mon[106692]: pgmap v1023: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:45:00.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:45:00 smithi037 ceph-mon[142325]: pgmap v1024: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:45:01.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:45:00 smithi148 ceph-mon[106692]: pgmap v1024: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:45:02.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:45:02 smithi037 ceph-mon[142325]: pgmap v1025: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:45:03.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:45:02 smithi148 ceph-mon[106692]: pgmap v1025: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:45:04.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:45:04 smithi037 ceph-mon[142325]: pgmap v1026: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:45:05.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:45:04 smithi148 ceph-mon[106692]: pgmap v1026: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:45:06.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:45:06 smithi037 ceph-mon[142325]: pgmap v1027: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:45:07.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:45:06 smithi148 ceph-mon[106692]: pgmap v1027: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:45:08.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:45:08 smithi037 ceph-mon[142325]: pgmap v1028: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:45:09.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:45:08 smithi148 ceph-mon[106692]: pgmap v1028: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:45:10.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:45:10 smithi037 ceph-mon[142325]: pgmap v1029: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:45:10.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:45:10 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:45:11.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:45:10 smithi148 ceph-mon[106692]: pgmap v1029: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:45:11.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:45:10 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:45:12.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:45:12 smithi037 ceph-mon[142325]: pgmap v1030: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:45:13.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:45:12 smithi148 ceph-mon[106692]: pgmap v1030: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:45:14.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:45:14 smithi037 ceph-mon[142325]: pgmap v1031: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:45:15.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:45:14 smithi148 ceph-mon[106692]: pgmap v1031: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:45:16.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:45:16 smithi037 ceph-mon[142325]: pgmap v1032: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:45:17.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:45:16 smithi148 ceph-mon[106692]: pgmap v1032: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:45:18.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:45:18 smithi037 ceph-mon[142325]: pgmap v1033: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:45:18.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:18.988+0000 7f7a98400700 1 -- 172.21.15.37:0/3132017807 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7a901037a0 msgr2=0x7f7a900fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:45:18.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:18.988+0000 7f7a98400700 1 --2- 172.21.15.37:0/3132017807 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7a901037a0 0x7f7a900fce10 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f7a7c004940 tx=0x7f7a7c009e30 comp rx=0 tx=0).stop 2024-03-29T08:45:18.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:18.988+0000 7f7a98400700 1 -- 172.21.15.37:0/3132017807 shutdown_connections 2024-03-29T08:45:18.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:18.988+0000 7f7a98400700 1 --2- 172.21.15.37:0/3132017807 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7a901037a0 0x7f7a900fce10 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:45:18.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:18.988+0000 7f7a98400700 1 --2- 172.21.15.37:0/3132017807 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7a90102e00 0x7f7a901031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:45:18.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:18.988+0000 7f7a98400700 1 -- 172.21.15.37:0/3132017807 >> 172.21.15.37:0/3132017807 conn(0x7f7a900f89b0 msgr2=0x7f7a900fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:45:18.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:18.988+0000 7f7a98400700 1 -- 172.21.15.37:0/3132017807 shutdown_connections 2024-03-29T08:45:18.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:18.988+0000 7f7a98400700 1 -- 172.21.15.37:0/3132017807 wait complete. 2024-03-29T08:45:18.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:18.989+0000 7f7a98400700 1 Processor -- start 2024-03-29T08:45:18.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:18.989+0000 7f7a98400700 1 -- start start 2024-03-29T08:45:18.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:18.990+0000 7f7a98400700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7a90102e00 0x7f7a90115cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:45:18.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:18.990+0000 7f7a98400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7a901037a0 0x7f7a90116230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:45:18.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:18.990+0000 7f7a98400700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a90116880 con 0x7f7a901037a0 2024-03-29T08:45:18.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:18.990+0000 7f7a98400700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a9010fc50 con 0x7f7a90102e00 2024-03-29T08:45:18.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:18.990+0000 7f7a95000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7a901037a0 0x7f7a90116230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:45:18.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:18.990+0000 7f7a95000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7a901037a0 0x7f7a90116230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49350/0 (socket says 172.21.15.37:49350) 2024-03-29T08:45:18.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:18.990+0000 7f7a95000700 1 -- 172.21.15.37:0/3933631567 learned_addr learned my addr 172.21.15.37:0/3933631567 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:45:18.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:18.990+0000 7f7a95a00700 1 --2- 172.21.15.37:0/3933631567 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7a90102e00 0x7f7a90115cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:45:18.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:18.990+0000 7f7a95000700 1 -- 172.21.15.37:0/3933631567 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7a90102e00 msgr2=0x7f7a90115cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:45:18.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:18.990+0000 7f7a95000700 1 --2- 172.21.15.37:0/3933631567 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7a90102e00 0x7f7a90115cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:45:18.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:18.990+0000 7f7a95000700 1 -- 172.21.15.37:0/3933631567 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a7c005040 con 0x7f7a901037a0 2024-03-29T08:45:18.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:18.991+0000 7f7a95000700 1 --2- 172.21.15.37:0/3933631567 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7a901037a0 0x7f7a90116230 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f7a7c009390 tx=0x7f7a7c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:45:18.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:18.991+0000 7f7a8d600700 1 -- 172.21.15.37:0/3933631567 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7a7c00e040 con 0x7f7a901037a0 2024-03-29T08:45:18.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:18.991+0000 7f7a8d600700 1 -- 172.21.15.37:0/3933631567 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f7a7c004290 con 0x7f7a901037a0 2024-03-29T08:45:18.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:18.991+0000 7f7a98400700 1 -- 172.21.15.37:0/3933631567 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a9010fed0 con 0x7f7a901037a0 2024-03-29T08:45:18.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:18.991+0000 7f7a8d600700 1 -- 172.21.15.37:0/3933631567 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7a7c013e70 con 0x7f7a901037a0 2024-03-29T08:45:18.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:18.991+0000 7f7a98400700 1 -- 172.21.15.37:0/3933631567 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a90110490 con 0x7f7a901037a0 2024-03-29T08:45:18.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:18.992+0000 7f7a98400700 1 -- 172.21.15.37:0/3933631567 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a900fe550 con 0x7f7a901037a0 2024-03-29T08:45:18.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:18.993+0000 7f7a8d600700 1 -- 172.21.15.37:0/3933631567 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f7a7c013690 con 0x7f7a901037a0 2024-03-29T08:45:18.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:18.994+0000 7f7a8d600700 1 --2- 172.21.15.37:0/3933631567 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f7a7806cbc0 0x7f7a7806f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:45:18.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:18.994+0000 7f7a8d600700 1 -- 172.21.15.37:0/3933631567 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f7a7c08d2b0 con 0x7f7a901037a0 2024-03-29T08:45:18.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:18.994+0000 7f7a95a00700 1 --2- 172.21.15.37:0/3933631567 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f7a7806cbc0 0x7f7a7806f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:45:18.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:18.995+0000 7f7a95a00700 1 --2- 172.21.15.37:0/3933631567 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f7a7806cbc0 0x7f7a7806f070 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f7a84004f40 tx=0x7f7a8400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:45:18.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:18.999+0000 7f7a8d600700 1 -- 172.21.15.37:0/3933631567 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f7a7c05adb0 con 0x7f7a901037a0 2024-03-29T08:45:19.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:45:18 smithi148 ceph-mon[106692]: pgmap v1033: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:45:19.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:19.229+0000 7f7a98400700 1 -- 172.21.15.37:0/3933631567 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7a90109e10 con 0x7f7a7806cbc0 2024-03-29T08:45:19.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:19.230+0000 7f7a8d600700 1 -- 172.21.15.37:0/3933631567 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f7a90109e10 con 0x7f7a7806cbc0 2024-03-29T08:45:19.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:19.233+0000 7f7a98400700 1 -- 172.21.15.37:0/3933631567 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f7a7806cbc0 msgr2=0x7f7a7806f070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:45:19.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:19.233+0000 7f7a98400700 1 --2- 172.21.15.37:0/3933631567 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f7a7806cbc0 0x7f7a7806f070 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f7a84004f40 tx=0x7f7a8400b040 comp rx=0 tx=0).stop 2024-03-29T08:45:19.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:19.233+0000 7f7a98400700 1 -- 172.21.15.37:0/3933631567 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7a901037a0 msgr2=0x7f7a90116230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:45:19.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:19.233+0000 7f7a98400700 1 --2- 172.21.15.37:0/3933631567 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7a901037a0 0x7f7a90116230 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f7a7c009390 tx=0x7f7a7c009e30 comp rx=0 tx=0).stop 2024-03-29T08:45:19.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:19.233+0000 7f7a98400700 1 -- 172.21.15.37:0/3933631567 shutdown_connections 2024-03-29T08:45:19.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:19.233+0000 7f7a98400700 1 --2- 172.21.15.37:0/3933631567 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f7a7806cbc0 0x7f7a7806f070 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:45:19.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:19.233+0000 7f7a98400700 1 --2- 172.21.15.37:0/3933631567 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7a90102e00 0x7f7a90115cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:45:19.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:19.233+0000 7f7a98400700 1 --2- 172.21.15.37:0/3933631567 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7a901037a0 0x7f7a90116230 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:45:19.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:19.233+0000 7f7a98400700 1 -- 172.21.15.37:0/3933631567 >> 172.21.15.37:0/3933631567 conn(0x7f7a900f89b0 msgr2=0x7f7a900f9a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:45:19.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:19.233+0000 7f7a98400700 1 -- 172.21.15.37:0/3933631567 shutdown_connections 2024-03-29T08:45:19.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:19.233+0000 7f7a98400700 1 -- 172.21.15.37:0/3933631567 wait complete. 2024-03-29T08:45:19.248 INFO:teuthology.orchestra.run.smithi037.stdout:true 2024-03-29T08:45:19.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:19.372+0000 7f80e9800700 1 -- 172.21.15.37:0/3845885462 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f80e4070c10 msgr2=0x7f80e4109190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:45:19.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:19.372+0000 7f80e9800700 1 --2- 172.21.15.37:0/3845885462 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f80e4070c10 0x7f80e4109190 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f80d0004970 tx=0x7f80d0009e30 comp rx=0 tx=0).stop 2024-03-29T08:45:19.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:19.372+0000 7f80e9800700 1 -- 172.21.15.37:0/3845885462 shutdown_connections 2024-03-29T08:45:19.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:19.372+0000 7f80e9800700 1 --2- 172.21.15.37:0/3845885462 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f80e4070c10 0x7f80e4109190 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:45:19.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:19.372+0000 7f80e9800700 1 --2- 172.21.15.37:0/3845885462 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f80e4070300 0x7f80e40706d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:45:19.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:19.372+0000 7f80e9800700 1 -- 172.21.15.37:0/3845885462 >> 172.21.15.37:0/3845885462 conn(0x7f80e40f8970 msgr2=0x7f80e40fad80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:45:19.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:19.372+0000 7f80e9800700 1 -- 172.21.15.37:0/3845885462 shutdown_connections 2024-03-29T08:45:19.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:19.372+0000 7f80e9800700 1 -- 172.21.15.37:0/3845885462 wait complete. 2024-03-29T08:45:19.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:19.373+0000 7f80e9800700 1 Processor -- start 2024-03-29T08:45:19.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:19.373+0000 7f80e9800700 1 -- start start 2024-03-29T08:45:19.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:19.374+0000 7f80e9800700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f80e4070300 0x7f80e4115cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:45:19.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:19.374+0000 7f80e9800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f80e4070c10 0x7f80e4116200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:45:19.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:19.374+0000 7f80e9800700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80e4116850 con 0x7f80e4070c10 2024-03-29T08:45:19.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:19.374+0000 7f80e9800700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80e410fc20 con 0x7f80e4070300 2024-03-29T08:45:19.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:19.374+0000 7f80e2000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f80e4070c10 0x7f80e4116200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:45:19.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:19.374+0000 7f80e2000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f80e4070c10 0x7f80e4116200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49362/0 (socket says 172.21.15.37:49362) 2024-03-29T08:45:19.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:19.374+0000 7f80e2000700 1 -- 172.21.15.37:0/656323369 learned_addr learned my addr 172.21.15.37:0/656323369 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:45:19.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:19.374+0000 7f80e2a00700 1 --2- 172.21.15.37:0/656323369 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f80e4070300 0x7f80e4115cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:45:19.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:19.375+0000 7f80e2000700 1 -- 172.21.15.37:0/656323369 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f80e4070300 msgr2=0x7f80e4115cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:45:19.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:19.375+0000 7f80e2000700 1 --2- 172.21.15.37:0/656323369 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f80e4070300 0x7f80e4115cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:45:19.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:19.375+0000 7f80e2000700 1 -- 172.21.15.37:0/656323369 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80d0005040 con 0x7f80e4070c10 2024-03-29T08:45:19.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:19.375+0000 7f80e2000700 1 --2- 172.21.15.37:0/656323369 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f80e4070c10 0x7f80e4116200 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f80d0009360 tx=0x7f80d0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:45:19.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:19.375+0000 7f80daa00700 1 -- 172.21.15.37:0/656323369 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f80d000e040 con 0x7f80e4070c10 2024-03-29T08:45:19.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:19.375+0000 7f80daa00700 1 -- 172.21.15.37:0/656323369 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f80d0004290 con 0x7f80e4070c10 2024-03-29T08:45:19.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:19.375+0000 7f80e9800700 1 -- 172.21.15.37:0/656323369 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80e410fea0 con 0x7f80e4070c10 2024-03-29T08:45:19.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:19.375+0000 7f80daa00700 1 -- 172.21.15.37:0/656323369 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f80d0013d90 con 0x7f80e4070c10 2024-03-29T08:45:19.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:19.375+0000 7f80e9800700 1 -- 172.21.15.37:0/656323369 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80e41103e0 con 0x7f80e4070c10 2024-03-29T08:45:19.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:19.376+0000 7f80e9800700 1 -- 172.21.15.37:0/656323369 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80e4071a40 con 0x7f80e4070c10 2024-03-29T08:45:19.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:19.377+0000 7f80daa00700 1 -- 172.21.15.37:0/656323369 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f80d000ba40 con 0x7f80e4070c10 2024-03-29T08:45:19.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:19.378+0000 7f80daa00700 1 --2- 172.21.15.37:0/656323369 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f80c806cc30 0x7f80c806f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:45:19.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:19.378+0000 7f80daa00700 1 -- 172.21.15.37:0/656323369 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f80d008d2f0 con 0x7f80e4070c10 2024-03-29T08:45:19.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:19.378+0000 7f80e2a00700 1 --2- 172.21.15.37:0/656323369 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f80c806cc30 0x7f80c806f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:45:19.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:19.379+0000 7f80e2a00700 1 --2- 172.21.15.37:0/656323369 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f80c806cc30 0x7f80c806f0e0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f80d4007fd0 tx=0x7f80d4009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:45:19.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:19.383+0000 7f80daa00700 1 -- 172.21.15.37:0/656323369 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f80d005ad40 con 0x7f80e4070c10 2024-03-29T08:45:19.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:19.614+0000 7f80e9800700 1 -- 172.21.15.37:0/656323369 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f80e410c050 con 0x7f80c806cc30 2024-03-29T08:45:19.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:19.615+0000 7f80daa00700 1 -- 172.21.15.37:0/656323369 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f80e410c050 con 0x7f80c806cc30 2024-03-29T08:45:19.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:19.618+0000 7f80e9800700 1 -- 172.21.15.37:0/656323369 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f80c806cc30 msgr2=0x7f80c806f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:45:19.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:19.618+0000 7f80e9800700 1 --2- 172.21.15.37:0/656323369 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f80c806cc30 0x7f80c806f0e0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f80d4007fd0 tx=0x7f80d4009040 comp rx=0 tx=0).stop 2024-03-29T08:45:19.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:19.618+0000 7f80e9800700 1 -- 172.21.15.37:0/656323369 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f80e4070c10 msgr2=0x7f80e4116200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:45:19.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:19.618+0000 7f80e9800700 1 --2- 172.21.15.37:0/656323369 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f80e4070c10 0x7f80e4116200 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f80d0009360 tx=0x7f80d0009e30 comp rx=0 tx=0).stop 2024-03-29T08:45:19.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:19.618+0000 7f80e9800700 1 -- 172.21.15.37:0/656323369 shutdown_connections 2024-03-29T08:45:19.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:19.618+0000 7f80e9800700 1 --2- 172.21.15.37:0/656323369 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f80c806cc30 0x7f80c806f0e0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:45:19.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:19.618+0000 7f80e9800700 1 --2- 172.21.15.37:0/656323369 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f80e4070300 0x7f80e4115cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:45:19.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:19.618+0000 7f80e9800700 1 --2- 172.21.15.37:0/656323369 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f80e4070c10 0x7f80e4116200 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:45:19.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:19.618+0000 7f80e9800700 1 -- 172.21.15.37:0/656323369 >> 172.21.15.37:0/656323369 conn(0x7f80e40f8970 msgr2=0x7f80e40f9de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:45:19.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:19.618+0000 7f80e9800700 1 -- 172.21.15.37:0/656323369 shutdown_connections 2024-03-29T08:45:19.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:19.618+0000 7f80e9800700 1 -- 172.21.15.37:0/656323369 wait complete. 2024-03-29T08:45:19.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:19.755+0000 7fb151000700 1 -- 172.21.15.37:0/3055870457 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb14c070c10 msgr2=0x7fb14c109180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:45:19.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:19.755+0000 7fb151000700 1 --2- 172.21.15.37:0/3055870457 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb14c070c10 0x7fb14c109180 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fb138004970 tx=0x7fb138009e30 comp rx=0 tx=0).stop 2024-03-29T08:45:19.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:19.755+0000 7fb151000700 1 -- 172.21.15.37:0/3055870457 shutdown_connections 2024-03-29T08:45:19.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:19.756+0000 7fb151000700 1 --2- 172.21.15.37:0/3055870457 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb14c070c10 0x7fb14c109180 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:45:19.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:19.756+0000 7fb151000700 1 --2- 172.21.15.37:0/3055870457 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb14c070300 0x7fb14c0706d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:45:19.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:19.756+0000 7fb151000700 1 -- 172.21.15.37:0/3055870457 >> 172.21.15.37:0/3055870457 conn(0x7fb14c0f89a0 msgr2=0x7fb14c0fadb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:45:19.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:19.756+0000 7fb151000700 1 -- 172.21.15.37:0/3055870457 shutdown_connections 2024-03-29T08:45:19.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:19.756+0000 7fb151000700 1 -- 172.21.15.37:0/3055870457 wait complete. 2024-03-29T08:45:19.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:19.756+0000 7fb151000700 1 Processor -- start 2024-03-29T08:45:19.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:19.757+0000 7fb151000700 1 -- start start 2024-03-29T08:45:19.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:19.757+0000 7fb151000700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb14c070300 0x7fb14c115c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:45:19.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:19.757+0000 7fb151000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb14c070c10 0x7fb14c1161b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:45:19.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:19.757+0000 7fb151000700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb14c116800 con 0x7fb14c070c10 2024-03-29T08:45:19.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:19.757+0000 7fb151000700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb14c10fbd0 con 0x7fb14c070300 2024-03-29T08:45:19.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:19.757+0000 7fb149400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb14c070c10 0x7fb14c1161b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:45:19.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:19.757+0000 7fb149400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb14c070c10 0x7fb14c1161b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49384/0 (socket says 172.21.15.37:49384) 2024-03-29T08:45:19.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:19.757+0000 7fb149400700 1 -- 172.21.15.37:0/1581745808 learned_addr learned my addr 172.21.15.37:0/1581745808 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:45:19.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:19.757+0000 7fb149e00700 1 --2- 172.21.15.37:0/1581745808 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb14c070300 0x7fb14c115c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:45:19.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:19.758+0000 7fb149400700 1 -- 172.21.15.37:0/1581745808 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb14c070300 msgr2=0x7fb14c115c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:45:19.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:19.758+0000 7fb149400700 1 --2- 172.21.15.37:0/1581745808 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb14c070300 0x7fb14c115c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:45:19.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:19.758+0000 7fb149400700 1 -- 172.21.15.37:0/1581745808 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb138005040 con 0x7fb14c070c10 2024-03-29T08:45:19.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:19.758+0000 7fb149400700 1 --2- 172.21.15.37:0/1581745808 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb14c070c10 0x7fb14c1161b0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fb138004f50 tx=0x7fb138009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:45:19.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:19.758+0000 7fb142000700 1 -- 172.21.15.37:0/1581745808 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb13800e040 con 0x7fb14c070c10 2024-03-29T08:45:19.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:19.758+0000 7fb142000700 1 -- 172.21.15.37:0/1581745808 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fb138013420 con 0x7fb14c070c10 2024-03-29T08:45:19.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:19.758+0000 7fb151000700 1 -- 172.21.15.37:0/1581745808 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb14c10fe50 con 0x7fb14c070c10 2024-03-29T08:45:19.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:19.758+0000 7fb142000700 1 -- 172.21.15.37:0/1581745808 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb13801c630 con 0x7fb14c070c10 2024-03-29T08:45:19.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:19.759+0000 7fb151000700 1 -- 172.21.15.37:0/1581745808 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb14c110390 con 0x7fb14c070c10 2024-03-29T08:45:19.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:19.760+0000 7fb151000700 1 -- 172.21.15.37:0/1581745808 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb14c071a40 con 0x7fb14c070c10 2024-03-29T08:45:19.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:19.761+0000 7fb142000700 1 -- 172.21.15.37:0/1581745808 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fb138013590 con 0x7fb14c070c10 2024-03-29T08:45:19.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:19.762+0000 7fb142000700 1 --2- 172.21.15.37:0/1581745808 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fb13006cc30 0x7fb13006f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:45:19.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:19.762+0000 7fb142000700 1 -- 172.21.15.37:0/1581745808 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fb13808d1c0 con 0x7fb14c070c10 2024-03-29T08:45:19.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:19.762+0000 7fb149e00700 1 --2- 172.21.15.37:0/1581745808 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fb13006cc30 0x7fb13006f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:45:19.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:19.763+0000 7fb149e00700 1 --2- 172.21.15.37:0/1581745808 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fb13006cc30 0x7fb13006f0e0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fb13c004f40 tx=0x7fb13c00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:45:19.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:19.767+0000 7fb142000700 1 -- 172.21.15.37:0/1581745808 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fb13805ac90 con 0x7fb14c070c10 2024-03-29T08:45:19.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:19.993+0000 7fb151000700 1 -- 172.21.15.37:0/1581745808 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fb14c10bfd0 con 0x7fb13006cc30 2024-03-29T08:45:20.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:20.005+0000 7fb142000700 1 -- 172.21.15.37:0/1581745808 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7fb14c10bfd0 con 0x7fb13006cc30 2024-03-29T08:45:20.005 INFO:teuthology.orchestra.run.smithi037.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-29T08:45:20.005 INFO:teuthology.orchestra.run.smithi037.stdout:alertmanager.smithi037 smithi037 *:9093,9094 running (35m) 11m ago 48m 14.3M - 0.25.0 c8568f914cd2 2f9f9746ed0f 2024-03-29T08:45:20.005 INFO:teuthology.orchestra.run.smithi037.stdout:crash.smithi037 smithi037 running (32m) 11m ago 48m 6647k - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 178d416a3186 2024-03-29T08:45:20.005 INFO:teuthology.orchestra.run.smithi037.stdout:crash.smithi148 smithi148 running (31m) 11m ago 47m 6656k - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 23666989d58f 2024-03-29T08:45:20.005 INFO:teuthology.orchestra.run.smithi037.stdout:grafana.smithi037 smithi037 *:3000 running (34m) 11m ago 47m 74.5M - 9.4.7 954c08fa6188 6b976fe49952 2024-03-29T08:45:20.006 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi037.iifsmh smithi037 running (27m) 11m ago 41m 16.4M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 2d97f562cc5b 2024-03-29T08:45:20.006 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi037.mmwgxe smithi037 running (27m) 11m ago 41m 16.7M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff ec37037f46e8 2024-03-29T08:45:20.006 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi148.qukpjh smithi148 running (12m) 11m ago 41m 10.7M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 3257219bc046 2024-03-29T08:45:20.006 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi148.sqteih smithi148 running (27m) 11m ago 41m 16.3M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 44c15c77037a 2024-03-29T08:45:20.006 INFO:teuthology.orchestra.run.smithi037.stdout:mgr.smithi037.wjikii smithi037 *:8443,9283,8765 running (37m) 11m ago 49m 510M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff c5a1417b9417 2024-03-29T08:45:20.006 INFO:teuthology.orchestra.run.smithi037.stdout:mgr.smithi148.onhgvm smithi148 *:8443,9283,8765 running (36m) 11m ago 46m 456M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff fcfdbce062bd 2024-03-29T08:45:20.006 INFO:teuthology.orchestra.run.smithi037.stdout:mon.smithi037 smithi037 running (33m) 11m ago 49m 70.2M 2048M 18.2.2-1015-gd0e392d9 1cc2f45c17ff eea2060783af 2024-03-29T08:45:20.006 INFO:teuthology.orchestra.run.smithi037.stdout:mon.smithi148 smithi148 running (32m) 11m ago 46m 50.4M 2048M 18.2.2-1015-gd0e392d9 1cc2f45c17ff ad4b2c5748be 2024-03-29T08:45:20.006 INFO:teuthology.orchestra.run.smithi037.stdout:node-exporter.smithi037 smithi037 *:9100 running (35m) 11m ago 48m 8514k - 1.5.0 0da6a335fe13 26b8ae9cb878 2024-03-29T08:45:20.006 INFO:teuthology.orchestra.run.smithi037.stdout:node-exporter.smithi148 smithi148 *:9100 running (35m) 11m ago 47m 8464k - 1.5.0 0da6a335fe13 98414e4e64ed 2024-03-29T08:45:20.006 INFO:teuthology.orchestra.run.smithi037.stdout:osd.0 smithi037 running (31m) 11m ago 46m 108M 1619M 18.2.2-1015-gd0e392d9 1cc2f45c17ff c0c584f6245c 2024-03-29T08:45:20.006 INFO:teuthology.orchestra.run.smithi037.stdout:osd.1 smithi037 running (30m) 11m ago 45m 104M 1619M 18.2.2-1015-gd0e392d9 1cc2f45c17ff a4a2ff4cc83d 2024-03-29T08:45:20.006 INFO:teuthology.orchestra.run.smithi037.stdout:osd.2 smithi037 running (30m) 11m ago 44m 85.9M 1619M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 8b1bc71aa79e 2024-03-29T08:45:20.006 INFO:teuthology.orchestra.run.smithi037.stdout:osd.3 smithi148 running (29m) 11m ago 44m 109M 2642M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 1c656ab8ff86 2024-03-29T08:45:20.007 INFO:teuthology.orchestra.run.smithi037.stdout:osd.4 smithi148 running (28m) 11m ago 43m 101M 2642M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 966a9de11d22 2024-03-29T08:45:20.007 INFO:teuthology.orchestra.run.smithi037.stdout:osd.5 smithi148 running (28m) 11m ago 43m 89.4M 2642M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 474c4f9212ce 2024-03-29T08:45:20.007 INFO:teuthology.orchestra.run.smithi037.stdout:prometheus.smithi037 smithi037 *:9095 running (35m) 11m ago 47m 61.0M - 2.43.0 a07b618ecd1d 30c0223ec589 2024-03-29T08:45:20.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:20.007+0000 7fb151000700 1 -- 172.21.15.37:0/1581745808 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fb13006cc30 msgr2=0x7fb13006f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:45:20.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:20.007+0000 7fb151000700 1 --2- 172.21.15.37:0/1581745808 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fb13006cc30 0x7fb13006f0e0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fb13c004f40 tx=0x7fb13c00b040 comp rx=0 tx=0).stop 2024-03-29T08:45:20.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:20.007+0000 7fb151000700 1 -- 172.21.15.37:0/1581745808 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb14c070c10 msgr2=0x7fb14c1161b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:45:20.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:20.007+0000 7fb151000700 1 --2- 172.21.15.37:0/1581745808 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb14c070c10 0x7fb14c1161b0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fb138004f50 tx=0x7fb138009e30 comp rx=0 tx=0).stop 2024-03-29T08:45:20.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:20.008+0000 7fb151000700 1 -- 172.21.15.37:0/1581745808 shutdown_connections 2024-03-29T08:45:20.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:20.008+0000 7fb151000700 1 --2- 172.21.15.37:0/1581745808 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fb13006cc30 0x7fb13006f0e0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:45:20.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:20.008+0000 7fb151000700 1 --2- 172.21.15.37:0/1581745808 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb14c070300 0x7fb14c115c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:45:20.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:20.008+0000 7fb151000700 1 --2- 172.21.15.37:0/1581745808 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb14c070c10 0x7fb14c1161b0 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:45:20.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:20.008+0000 7fb151000700 1 -- 172.21.15.37:0/1581745808 >> 172.21.15.37:0/1581745808 conn(0x7fb14c0f89a0 msgr2=0x7fb14c0f9f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:45:20.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:20.008+0000 7fb151000700 1 -- 172.21.15.37:0/1581745808 shutdown_connections 2024-03-29T08:45:20.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:20.008+0000 7fb151000700 1 -- 172.21.15.37:0/1581745808 wait complete. 2024-03-29T08:45:20.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:20.144+0000 7f85c8400700 1 -- 172.21.15.37:0/1533445957 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f85c00fde90 msgr2=0x7f85c00fe260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:45:20.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:20.144+0000 7f85c8400700 1 --2- 172.21.15.37:0/1533445957 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f85c00fde90 0x7f85c00fe260 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f85b4004940 tx=0x7f85b4009e30 comp rx=0 tx=0).stop 2024-03-29T08:45:20.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:20.144+0000 7f85c8400700 1 -- 172.21.15.37:0/1533445957 shutdown_connections 2024-03-29T08:45:20.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:20.144+0000 7f85c8400700 1 --2- 172.21.15.37:0/1533445957 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f85c00fe830 0x7f85c0106f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:45:20.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:20.144+0000 7f85c8400700 1 --2- 172.21.15.37:0/1533445957 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f85c00fde90 0x7f85c00fe260 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:45:20.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:20.144+0000 7f85c8400700 1 -- 172.21.15.37:0/1533445957 >> 172.21.15.37:0/1533445957 conn(0x7f85c0072bc0 msgr2=0x7f85c0072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:45:20.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:20.144+0000 7f85c8400700 1 -- 172.21.15.37:0/1533445957 shutdown_connections 2024-03-29T08:45:20.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:20.144+0000 7f85c8400700 1 -- 172.21.15.37:0/1533445957 wait complete. 2024-03-29T08:45:20.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:20.145+0000 7f85c8400700 1 Processor -- start 2024-03-29T08:45:20.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:20.145+0000 7f85c8400700 1 -- start start 2024-03-29T08:45:20.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:20.146+0000 7f85c8400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f85c00fde90 0x7f85c0115d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:45:20.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:20.146+0000 7f85c8400700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f85c00fe830 0x7f85c0116260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:45:20.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:20.146+0000 7f85c8400700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85c01168b0 con 0x7f85c00fde90 2024-03-29T08:45:20.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:20.146+0000 7f85c8400700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85c010fc80 con 0x7f85c00fe830 2024-03-29T08:45:20.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:20.146+0000 7f85c5a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f85c00fde90 0x7f85c0115d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:45:20.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:20.146+0000 7f85c5a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f85c00fde90 0x7f85c0115d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37044/0 (socket says 172.21.15.37:37044) 2024-03-29T08:45:20.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:20.146+0000 7f85c5a00700 1 -- 172.21.15.37:0/1876952542 learned_addr learned my addr 172.21.15.37:0/1876952542 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:45:20.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:20.146+0000 7f85c5000700 1 --2- 172.21.15.37:0/1876952542 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f85c00fe830 0x7f85c0116260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:45:20.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:20.146+0000 7f85c5a00700 1 -- 172.21.15.37:0/1876952542 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f85c00fe830 msgr2=0x7f85c0116260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:45:20.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:20.146+0000 7f85c5a00700 1 --2- 172.21.15.37:0/1876952542 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f85c00fe830 0x7f85c0116260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:45:20.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:20.146+0000 7f85c5a00700 1 -- 172.21.15.37:0/1876952542 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85b4005040 con 0x7f85c00fde90 2024-03-29T08:45:20.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:20.147+0000 7f85c5a00700 1 --2- 172.21.15.37:0/1876952542 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f85c00fde90 0x7f85c0115d20 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f85b40093b0 tx=0x7f85b4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:45:20.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:20.147+0000 7f85bd600700 1 -- 172.21.15.37:0/1876952542 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f85b400e040 con 0x7f85c00fde90 2024-03-29T08:45:20.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:20.147+0000 7f85bd600700 1 -- 172.21.15.37:0/1876952542 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f85b400bed0 con 0x7f85c00fde90 2024-03-29T08:45:20.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:20.147+0000 7f85bd600700 1 -- 172.21.15.37:0/1876952542 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f85b401c3f0 con 0x7f85c00fde90 2024-03-29T08:45:20.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:20.147+0000 7f85c8400700 1 -- 172.21.15.37:0/1876952542 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85c010ff00 con 0x7f85c00fde90 2024-03-29T08:45:20.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:20.147+0000 7f85c8400700 1 -- 172.21.15.37:0/1876952542 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85c0110440 con 0x7f85c00fde90 2024-03-29T08:45:20.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:20.148+0000 7f85c8400700 1 -- 172.21.15.37:0/1876952542 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85c0106a20 con 0x7f85c00fde90 2024-03-29T08:45:20.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:20.149+0000 7f85bd600700 1 -- 172.21.15.37:0/1876952542 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f85b4013790 con 0x7f85c00fde90 2024-03-29T08:45:20.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:20.150+0000 7f85bd600700 1 --2- 172.21.15.37:0/1876952542 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f85a806cc30 0x7f85a806f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:45:20.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:20.150+0000 7f85bd600700 1 -- 172.21.15.37:0/1876952542 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f85b408d200 con 0x7f85c00fde90 2024-03-29T08:45:20.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:20.150+0000 7f85c5000700 1 --2- 172.21.15.37:0/1876952542 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f85a806cc30 0x7f85a806f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:45:20.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:20.151+0000 7f85c5000700 1 --2- 172.21.15.37:0/1876952542 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f85a806cc30 0x7f85a806f0e0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f85c01115c0 tx=0x7f85ac00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:45:20.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:20.155+0000 7f85bd600700 1 -- 172.21.15.37:0/1876952542 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f85b405ad90 con 0x7f85c00fde90 2024-03-29T08:45:20.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:20.455+0000 7f85c8400700 1 -- 172.21.15.37:0/1876952542 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f85c004bed0 con 0x7f85c00fde90 2024-03-29T08:45:20.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:20.456+0000 7f85bd600700 1 -- 172.21.15.37:0/1876952542 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7f85b4013420 con 0x7f85c00fde90 2024-03-29T08:45:20.455 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-29T08:45:20.455 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": { 2024-03-29T08:45:20.456 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 2 2024-03-29T08:45:20.456 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:45:20.456 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": { 2024-03-29T08:45:20.456 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 2 2024-03-29T08:45:20.456 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:45:20.456 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": { 2024-03-29T08:45:20.456 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 6 2024-03-29T08:45:20.456 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:45:20.456 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": { 2024-03-29T08:45:20.456 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 4 2024-03-29T08:45:20.456 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:45:20.456 INFO:teuthology.orchestra.run.smithi037.stdout: "overall": { 2024-03-29T08:45:20.456 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 14 2024-03-29T08:45:20.456 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-29T08:45:20.456 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-29T08:45:20.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:20.458+0000 7f85c8400700 1 -- 172.21.15.37:0/1876952542 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f85a806cc30 msgr2=0x7f85a806f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:45:20.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:20.458+0000 7f85c8400700 1 --2- 172.21.15.37:0/1876952542 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f85a806cc30 0x7f85a806f0e0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f85c01115c0 tx=0x7f85ac00b040 comp rx=0 tx=0).stop 2024-03-29T08:45:20.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:20.458+0000 7f85c8400700 1 -- 172.21.15.37:0/1876952542 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f85c00fde90 msgr2=0x7f85c0115d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:45:20.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:20.458+0000 7f85c8400700 1 --2- 172.21.15.37:0/1876952542 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f85c00fde90 0x7f85c0115d20 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f85b40093b0 tx=0x7f85b4009e30 comp rx=0 tx=0).stop 2024-03-29T08:45:20.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:20.458+0000 7f85c8400700 1 -- 172.21.15.37:0/1876952542 shutdown_connections 2024-03-29T08:45:20.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:20.458+0000 7f85c8400700 1 --2- 172.21.15.37:0/1876952542 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f85a806cc30 0x7f85a806f0e0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:45:20.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:20.458+0000 7f85c8400700 1 --2- 172.21.15.37:0/1876952542 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f85c00fde90 0x7f85c0115d20 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:45:20.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:20.458+0000 7f85c8400700 1 --2- 172.21.15.37:0/1876952542 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f85c00fe830 0x7f85c0116260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:45:20.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:20.458+0000 7f85c8400700 1 -- 172.21.15.37:0/1876952542 >> 172.21.15.37:0/1876952542 conn(0x7f85c0072bc0 msgr2=0x7f85c00fb8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:45:20.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:20.458+0000 7f85c8400700 1 -- 172.21.15.37:0/1876952542 shutdown_connections 2024-03-29T08:45:20.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:20.458+0000 7f85c8400700 1 -- 172.21.15.37:0/1876952542 wait complete. 2024-03-29T08:45:20.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:20.595+0000 7fb951400700 1 -- 172.21.15.37:0/2049055151 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb94c1037a0 msgr2=0x7fb94c0fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:45:20.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:20.595+0000 7fb951400700 1 --2- 172.21.15.37:0/2049055151 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb94c1037a0 0x7fb94c0fce10 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7fb938004940 tx=0x7fb938009e30 comp rx=0 tx=0).stop 2024-03-29T08:45:20.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:20.595+0000 7fb951400700 1 -- 172.21.15.37:0/2049055151 shutdown_connections 2024-03-29T08:45:20.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:20.595+0000 7fb951400700 1 --2- 172.21.15.37:0/2049055151 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb94c1037a0 0x7fb94c0fce10 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:45:20.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:20.595+0000 7fb951400700 1 --2- 172.21.15.37:0/2049055151 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb94c102e00 0x7fb94c1031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:45:20.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:20.595+0000 7fb951400700 1 -- 172.21.15.37:0/2049055151 >> 172.21.15.37:0/2049055151 conn(0x7fb94c0f89b0 msgr2=0x7fb94c0fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:45:20.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:20.595+0000 7fb951400700 1 -- 172.21.15.37:0/2049055151 shutdown_connections 2024-03-29T08:45:20.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:20.595+0000 7fb951400700 1 -- 172.21.15.37:0/2049055151 wait complete. 2024-03-29T08:45:20.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:20.596+0000 7fb951400700 1 Processor -- start 2024-03-29T08:45:20.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:20.596+0000 7fb951400700 1 -- start start 2024-03-29T08:45:20.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:20.596+0000 7fb951400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb94c102e00 0x7fb94c115c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:45:20.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:20.597+0000 7fb951400700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb94c1037a0 0x7fb94c1161c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:45:20.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:20.597+0000 7fb951400700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb94c116810 con 0x7fb94c102e00 2024-03-29T08:45:20.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:20.597+0000 7fb951400700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb94c10fbe0 con 0x7fb94c1037a0 2024-03-29T08:45:20.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:20.597+0000 7fb94a800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb94c102e00 0x7fb94c115c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:45:20.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:20.597+0000 7fb94a800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb94c102e00 0x7fb94c115c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37062/0 (socket says 172.21.15.37:37062) 2024-03-29T08:45:20.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:20.597+0000 7fb94a800700 1 -- 172.21.15.37:0/1084787492 learned_addr learned my addr 172.21.15.37:0/1084787492 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:45:20.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:20.597+0000 7fb94a800700 1 -- 172.21.15.37:0/1084787492 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb94c1037a0 msgr2=0x7fb94c1161c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:45:20.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:20.597+0000 7fb94a800700 1 --2- 172.21.15.37:0/1084787492 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb94c1037a0 0x7fb94c1161c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:45:20.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:20.597+0000 7fb94a800700 1 -- 172.21.15.37:0/1084787492 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb938005040 con 0x7fb94c102e00 2024-03-29T08:45:20.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:20.597+0000 7fb94a800700 1 --2- 172.21.15.37:0/1084787492 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb94c102e00 0x7fb94c115c80 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7fb93c00eb40 tx=0x7fb93c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:45:20.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:20.598+0000 7fb942a00700 1 -- 172.21.15.37:0/1084787492 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb93c007cc0 con 0x7fb94c102e00 2024-03-29T08:45:20.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:20.598+0000 7fb942a00700 1 -- 172.21.15.37:0/1084787492 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fb93c00f040 con 0x7fb94c102e00 2024-03-29T08:45:20.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:20.598+0000 7fb942a00700 1 -- 172.21.15.37:0/1084787492 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb93c009570 con 0x7fb94c102e00 2024-03-29T08:45:20.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:20.598+0000 7fb951400700 1 -- 172.21.15.37:0/1084787492 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb94c10fe60 con 0x7fb94c102e00 2024-03-29T08:45:20.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:20.598+0000 7fb951400700 1 -- 172.21.15.37:0/1084787492 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb94c110420 con 0x7fb94c102e00 2024-03-29T08:45:20.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:20.599+0000 7fb951400700 1 -- 172.21.15.37:0/1084787492 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb94c0fe550 con 0x7fb94c102e00 2024-03-29T08:45:20.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:20.600+0000 7fb942a00700 1 -- 172.21.15.37:0/1084787492 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fb93c0096d0 con 0x7fb94c102e00 2024-03-29T08:45:20.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:20.601+0000 7fb942a00700 1 --2- 172.21.15.37:0/1084787492 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fb93006cc30 0x7fb93006f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:45:20.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:20.601+0000 7fb942a00700 1 -- 172.21.15.37:0/1084787492 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fb93c08bcb0 con 0x7fb94c102e00 2024-03-29T08:45:20.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:20.601+0000 7fb949e00700 1 --2- 172.21.15.37:0/1084787492 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fb93006cc30 0x7fb93006f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:45:20.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:20.602+0000 7fb949e00700 1 --2- 172.21.15.37:0/1084787492 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fb93006cc30 0x7fb93006f0e0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fb938000c00 tx=0x7fb938009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:45:20.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:20.606+0000 7fb942a00700 1 -- 172.21.15.37:0/1084787492 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fb93c0597b0 con 0x7fb94c102e00 2024-03-29T08:45:20.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:20.866+0000 7fb951400700 1 -- 172.21.15.37:0/1084787492 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fb94c04bed0 con 0x7fb94c102e00 2024-03-29T08:45:20.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:20.866+0000 7fb942a00700 1 -- 172.21.15.37:0/1084787492 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 30 v30) v1 ==== 76+0+1841 (secure 0 0 0) 0x7fb93c019750 con 0x7fb94c102e00 2024-03-29T08:45:20.866 INFO:teuthology.orchestra.run.smithi037.stdout:e30 2024-03-29T08:45:20.866 INFO:teuthology.orchestra.run.smithi037.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-29T08:45:20.866 INFO:teuthology.orchestra.run.smithi037.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T08:45:20.866 INFO:teuthology.orchestra.run.smithi037.stdout:legacy client fscid: 1 2024-03-29T08:45:20.866 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:45:20.866 INFO:teuthology.orchestra.run.smithi037.stdout:Filesystem 'cephfs' (1) 2024-03-29T08:45:20.867 INFO:teuthology.orchestra.run.smithi037.stdout:fs_name cephfs 2024-03-29T08:45:20.867 INFO:teuthology.orchestra.run.smithi037.stdout:epoch 21 2024-03-29T08:45:20.867 INFO:teuthology.orchestra.run.smithi037.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-03-29T08:45:20.867 INFO:teuthology.orchestra.run.smithi037.stdout:created 2024-03-29T08:03:29.241269+0000 2024-03-29T08:45:20.867 INFO:teuthology.orchestra.run.smithi037.stdout:modified 2024-03-29T08:17:16.237238+0000 2024-03-29T08:45:20.867 INFO:teuthology.orchestra.run.smithi037.stdout:tableserver 0 2024-03-29T08:45:20.867 INFO:teuthology.orchestra.run.smithi037.stdout:root 0 2024-03-29T08:45:20.867 INFO:teuthology.orchestra.run.smithi037.stdout:session_timeout 60 2024-03-29T08:45:20.867 INFO:teuthology.orchestra.run.smithi037.stdout:session_autoclose 300 2024-03-29T08:45:20.867 INFO:teuthology.orchestra.run.smithi037.stdout:max_file_size 1099511627776 2024-03-29T08:45:20.867 INFO:teuthology.orchestra.run.smithi037.stdout:max_xattr_size 65536 2024-03-29T08:45:20.867 INFO:teuthology.orchestra.run.smithi037.stdout:required_client_features {} 2024-03-29T08:45:20.867 INFO:teuthology.orchestra.run.smithi037.stdout:last_failure 0 2024-03-29T08:45:20.867 INFO:teuthology.orchestra.run.smithi037.stdout:last_failure_osd_epoch 82 2024-03-29T08:45:20.867 INFO:teuthology.orchestra.run.smithi037.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T08:45:20.867 INFO:teuthology.orchestra.run.smithi037.stdout:max_mds 2 2024-03-29T08:45:20.867 INFO:teuthology.orchestra.run.smithi037.stdout:in 0,1 2024-03-29T08:45:20.868 INFO:teuthology.orchestra.run.smithi037.stdout:up {} 2024-03-29T08:45:20.868 INFO:teuthology.orchestra.run.smithi037.stdout:failed 0,1 2024-03-29T08:45:20.868 INFO:teuthology.orchestra.run.smithi037.stdout:damaged 2024-03-29T08:45:20.868 INFO:teuthology.orchestra.run.smithi037.stdout:stopped 2024-03-29T08:45:20.868 INFO:teuthology.orchestra.run.smithi037.stdout:data_pools [3] 2024-03-29T08:45:20.868 INFO:teuthology.orchestra.run.smithi037.stdout:metadata_pool 2 2024-03-29T08:45:20.868 INFO:teuthology.orchestra.run.smithi037.stdout:inline_data disabled 2024-03-29T08:45:20.868 INFO:teuthology.orchestra.run.smithi037.stdout:balancer 2024-03-29T08:45:20.868 INFO:teuthology.orchestra.run.smithi037.stdout:bal_rank_mask -1 2024-03-29T08:45:20.868 INFO:teuthology.orchestra.run.smithi037.stdout:standby_count_wanted 1 2024-03-29T08:45:20.868 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:45:20.868 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:45:20.868 INFO:teuthology.orchestra.run.smithi037.stdout:Standby daemons: 2024-03-29T08:45:20.868 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:45:20.868 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi037.mmwgxe{-1:34440} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.37:6828/1248142005,v1:172.21.15.37:6829/1248142005] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:45:20.868 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi037.iifsmh{-1:34444} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.37:6826/2060882959,v1:172.21.15.37:6827/2060882959] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:45:20.868 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi148.sqteih{-1:44395} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.148:6824/4078652242,v1:172.21.15.148:6825/4078652242] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:45:20.869 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi148.qukpjh{-1:45007} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.148:6826/2394751105,v1:172.21.15.148:6827/2394751105] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:45:20.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:20.868+0000 7fb951400700 1 -- 172.21.15.37:0/1084787492 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fb93006cc30 msgr2=0x7fb93006f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:45:20.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:20.868+0000 7fb951400700 1 --2- 172.21.15.37:0/1084787492 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fb93006cc30 0x7fb93006f0e0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fb938000c00 tx=0x7fb938009e30 comp rx=0 tx=0).stop 2024-03-29T08:45:20.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:20.868+0000 7fb951400700 1 -- 172.21.15.37:0/1084787492 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb94c102e00 msgr2=0x7fb94c115c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:45:20.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:20.868+0000 7fb951400700 1 --2- 172.21.15.37:0/1084787492 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb94c102e00 0x7fb94c115c80 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7fb93c00eb40 tx=0x7fb93c00b040 comp rx=0 tx=0).stop 2024-03-29T08:45:20.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:20.869+0000 7fb951400700 1 -- 172.21.15.37:0/1084787492 shutdown_connections 2024-03-29T08:45:20.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:20.869+0000 7fb951400700 1 --2- 172.21.15.37:0/1084787492 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fb93006cc30 0x7fb93006f0e0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:45:20.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:20.869+0000 7fb951400700 1 --2- 172.21.15.37:0/1084787492 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb94c102e00 0x7fb94c115c80 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:45:20.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:20.869+0000 7fb951400700 1 --2- 172.21.15.37:0/1084787492 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb94c1037a0 0x7fb94c1161c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:45:20.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:20.869+0000 7fb951400700 1 -- 172.21.15.37:0/1084787492 >> 172.21.15.37:0/1084787492 conn(0x7fb94c0f89b0 msgr2=0x7fb94c0f99a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:45:20.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:20.869+0000 7fb951400700 1 -- 172.21.15.37:0/1084787492 shutdown_connections 2024-03-29T08:45:20.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:20.869+0000 7fb951400700 1 -- 172.21.15.37:0/1084787492 wait complete. 2024-03-29T08:45:20.870 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 30 2024-03-29T08:45:20.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:45:20 smithi037 ceph-mon[142325]: from='client.35850 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:45:20.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:45:20 smithi037 ceph-mon[142325]: pgmap v1034: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:45:20.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:45:20 smithi037 ceph-mon[142325]: from='client.35854 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:45:20.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:45:20 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/1876952542' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:45:21.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:21.007+0000 7fa279c00700 1 -- 172.21.15.37:0/1725253799 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa2741059b0 msgr2=0x7fa2740ff020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:45:21.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:21.007+0000 7fa279c00700 1 --2- 172.21.15.37:0/1725253799 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa2741059b0 0x7fa2740ff020 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7fa260004940 tx=0x7fa260009e30 comp rx=0 tx=0).stop 2024-03-29T08:45:21.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:21.007+0000 7fa279c00700 1 -- 172.21.15.37:0/1725253799 shutdown_connections 2024-03-29T08:45:21.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:21.007+0000 7fa279c00700 1 --2- 172.21.15.37:0/1725253799 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa2741059b0 0x7fa2740ff020 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:45:21.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:21.007+0000 7fa279c00700 1 --2- 172.21.15.37:0/1725253799 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa274105010 0x7fa2741053e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:45:21.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:21.007+0000 7fa279c00700 1 -- 172.21.15.37:0/1725253799 >> 172.21.15.37:0/1725253799 conn(0x7fa2740faba0 msgr2=0x7fa2740fcfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:45:21.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:21.007+0000 7fa279c00700 1 -- 172.21.15.37:0/1725253799 shutdown_connections 2024-03-29T08:45:21.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:21.007+0000 7fa279c00700 1 -- 172.21.15.37:0/1725253799 wait complete. 2024-03-29T08:45:21.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:21.008+0000 7fa279c00700 1 Processor -- start 2024-03-29T08:45:21.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:21.008+0000 7fa279c00700 1 -- start start 2024-03-29T08:45:21.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:21.008+0000 7fa279c00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa274105010 0x7fa274115cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:45:21.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:21.008+0000 7fa279c00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa2741059b0 0x7fa2741161f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:45:21.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:21.008+0000 7fa279c00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa274116840 con 0x7fa274105010 2024-03-29T08:45:21.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:21.008+0000 7fa279c00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa27410fc60 con 0x7fa2741059b0 2024-03-29T08:45:21.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:21.008+0000 7fa272a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa274105010 0x7fa274115cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:45:21.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:21.008+0000 7fa272a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa274105010 0x7fa274115cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37082/0 (socket says 172.21.15.37:37082) 2024-03-29T08:45:21.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:21.008+0000 7fa272a00700 1 -- 172.21.15.37:0/1849310258 learned_addr learned my addr 172.21.15.37:0/1849310258 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:45:21.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:21.009+0000 7fa272a00700 1 -- 172.21.15.37:0/1849310258 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa2741059b0 msgr2=0x7fa2741161f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:45:21.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:21.009+0000 7fa272a00700 1 --2- 172.21.15.37:0/1849310258 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa2741059b0 0x7fa2741161f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:45:21.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:21.009+0000 7fa272a00700 1 -- 172.21.15.37:0/1849310258 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa260005040 con 0x7fa274105010 2024-03-29T08:45:21.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:21.009+0000 7fa272a00700 1 --2- 172.21.15.37:0/1849310258 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa274105010 0x7fa274115cb0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7fa26400c8c0 tx=0x7fa2640044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:45:21.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:21.009+0000 7fa26aa00700 1 -- 172.21.15.37:0/1849310258 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa26400a980 con 0x7fa274105010 2024-03-29T08:45:21.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:21.009+0000 7fa26aa00700 1 -- 172.21.15.37:0/1849310258 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fa26400ce60 con 0x7fa274105010 2024-03-29T08:45:21.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:21.009+0000 7fa26aa00700 1 -- 172.21.15.37:0/1849310258 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa2640113a0 con 0x7fa274105010 2024-03-29T08:45:21.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:21.009+0000 7fa279c00700 1 -- 172.21.15.37:0/1849310258 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa27410ff40 con 0x7fa274105010 2024-03-29T08:45:21.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:21.010+0000 7fa279c00700 1 -- 172.21.15.37:0/1849310258 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa274110500 con 0x7fa274105010 2024-03-29T08:45:21.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:21.011+0000 7fa279c00700 1 -- 172.21.15.37:0/1849310258 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa274102320 con 0x7fa274105010 2024-03-29T08:45:21.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:21.011+0000 7fa26aa00700 1 -- 172.21.15.37:0/1849310258 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fa264012400 con 0x7fa274105010 2024-03-29T08:45:21.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:21.012+0000 7fa26aa00700 1 --2- 172.21.15.37:0/1849310258 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fa25806cc30 0x7fa25806f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:45:21.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:21.012+0000 7fa26aa00700 1 -- 172.21.15.37:0/1849310258 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fa26408c0f0 con 0x7fa274105010 2024-03-29T08:45:21.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:21.013+0000 7fa272000700 1 --2- 172.21.15.37:0/1849310258 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fa25806cc30 0x7fa25806f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:45:21.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:21.013+0000 7fa272000700 1 --2- 172.21.15.37:0/1849310258 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fa25806cc30 0x7fa25806f0e0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fa260004f90 tx=0x7fa260009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:45:21.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:21.017+0000 7fa26aa00700 1 -- 172.21.15.37:0/1849310258 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fa264059bc0 con 0x7fa274105010 2024-03-29T08:45:21.055 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:45:20 smithi148 ceph-mon[106692]: from='client.35850 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:45:21.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:45:20 smithi148 ceph-mon[106692]: pgmap v1034: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:45:21.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:45:20 smithi148 ceph-mon[106692]: from='client.35854 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:45:21.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:45:20 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/1876952542' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:45:21.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:21.249+0000 7fa279c00700 1 -- 172.21.15.37:0/1849310258 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa27410bfb0 con 0x7fa25806cc30 2024-03-29T08:45:21.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:21.250+0000 7fa26aa00700 1 -- 172.21.15.37:0/1849310258 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fa27410bfb0 con 0x7fa25806cc30 2024-03-29T08:45:21.250 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-29T08:45:21.250 INFO:teuthology.orchestra.run.smithi037.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59", 2024-03-29T08:45:21.250 INFO:teuthology.orchestra.run.smithi037.stdout: "in_progress": true, 2024-03-29T08:45:21.250 INFO:teuthology.orchestra.run.smithi037.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-29T08:45:21.250 INFO:teuthology.orchestra.run.smithi037.stdout: "services_complete": [ 2024-03-29T08:45:21.250 INFO:teuthology.orchestra.run.smithi037.stdout: "crash", 2024-03-29T08:45:21.250 INFO:teuthology.orchestra.run.smithi037.stdout: "osd", 2024-03-29T08:45:21.250 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr", 2024-03-29T08:45:21.250 INFO:teuthology.orchestra.run.smithi037.stdout: "mds", 2024-03-29T08:45:21.250 INFO:teuthology.orchestra.run.smithi037.stdout: "mon" 2024-03-29T08:45:21.250 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-29T08:45:21.250 INFO:teuthology.orchestra.run.smithi037.stdout: "progress": "16/21 daemons upgraded", 2024-03-29T08:45:21.250 INFO:teuthology.orchestra.run.smithi037.stdout: "message": "Currently upgrading mds daemons", 2024-03-29T08:45:21.250 INFO:teuthology.orchestra.run.smithi037.stdout: "is_paused": false 2024-03-29T08:45:21.251 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-29T08:45:21.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:21.252+0000 7fa279c00700 1 -- 172.21.15.37:0/1849310258 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fa25806cc30 msgr2=0x7fa25806f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:45:21.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:21.252+0000 7fa279c00700 1 --2- 172.21.15.37:0/1849310258 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fa25806cc30 0x7fa25806f0e0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fa260004f90 tx=0x7fa260009e30 comp rx=0 tx=0).stop 2024-03-29T08:45:21.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:21.252+0000 7fa279c00700 1 -- 172.21.15.37:0/1849310258 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa274105010 msgr2=0x7fa274115cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:45:21.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:21.252+0000 7fa279c00700 1 --2- 172.21.15.37:0/1849310258 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa274105010 0x7fa274115cb0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7fa26400c8c0 tx=0x7fa2640044c0 comp rx=0 tx=0).stop 2024-03-29T08:45:21.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:21.252+0000 7fa279c00700 1 -- 172.21.15.37:0/1849310258 shutdown_connections 2024-03-29T08:45:21.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:21.252+0000 7fa279c00700 1 --2- 172.21.15.37:0/1849310258 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fa25806cc30 0x7fa25806f0e0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:45:21.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:21.252+0000 7fa279c00700 1 --2- 172.21.15.37:0/1849310258 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa274105010 0x7fa274115cb0 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:45:21.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:21.252+0000 7fa279c00700 1 --2- 172.21.15.37:0/1849310258 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa2741059b0 0x7fa2741161f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:45:21.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:21.252+0000 7fa279c00700 1 -- 172.21.15.37:0/1849310258 >> 172.21.15.37:0/1849310258 conn(0x7fa2740faba0 msgr2=0x7fa274108640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:45:21.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:21.253+0000 7fa279c00700 1 -- 172.21.15.37:0/1849310258 shutdown_connections 2024-03-29T08:45:21.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:21.253+0000 7fa279c00700 1 -- 172.21.15.37:0/1849310258 wait complete. 2024-03-29T08:45:21.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:21.388+0000 7f6972e00700 1 -- 172.21.15.37:0/849868529 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f696c102c40 msgr2=0x7f696c103010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:45:21.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:21.388+0000 7f6972e00700 1 --2- 172.21.15.37:0/849868529 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f696c102c40 0x7f696c103010 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f6960004940 tx=0x7f6960009e30 comp rx=0 tx=0).stop 2024-03-29T08:45:21.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:21.388+0000 7f6972e00700 1 -- 172.21.15.37:0/849868529 shutdown_connections 2024-03-29T08:45:21.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:21.388+0000 7f6972e00700 1 --2- 172.21.15.37:0/849868529 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f696c1035e0 0x7f696c0fcc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:45:21.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:21.388+0000 7f6972e00700 1 --2- 172.21.15.37:0/849868529 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f696c102c40 0x7f696c103010 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:45:21.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:21.388+0000 7f6972e00700 1 -- 172.21.15.37:0/849868529 >> 172.21.15.37:0/849868529 conn(0x7f696c0f8830 msgr2=0x7f696c0fac40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:45:21.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:21.388+0000 7f6972e00700 1 -- 172.21.15.37:0/849868529 shutdown_connections 2024-03-29T08:45:21.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:21.388+0000 7f6972e00700 1 -- 172.21.15.37:0/849868529 wait complete. 2024-03-29T08:45:21.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:21.389+0000 7f6972e00700 1 Processor -- start 2024-03-29T08:45:21.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:21.389+0000 7f6972e00700 1 -- start start 2024-03-29T08:45:21.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:21.390+0000 7f6972e00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f696c102c40 0x7f696c0702c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:45:21.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:21.390+0000 7f6972e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f696c1035e0 0x7f696c070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:45:21.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:21.390+0000 7f6972e00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f696c070d90 con 0x7f696c1035e0 2024-03-29T08:45:21.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:21.390+0000 7f6972e00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f696c070ed0 con 0x7f696c102c40 2024-03-29T08:45:21.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:21.390+0000 7f6971000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f696c1035e0 0x7f696c070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:45:21.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:21.390+0000 7f6971000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f696c1035e0 0x7f696c070800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37100/0 (socket says 172.21.15.37:37100) 2024-03-29T08:45:21.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:21.390+0000 7f6971000700 1 -- 172.21.15.37:0/214725677 learned_addr learned my addr 172.21.15.37:0/214725677 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:45:21.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:21.391+0000 7f6971a00700 1 --2- 172.21.15.37:0/214725677 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f696c102c40 0x7f696c0702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:45:21.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:21.391+0000 7f6971000700 1 -- 172.21.15.37:0/214725677 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f696c102c40 msgr2=0x7f696c0702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:45:21.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:21.391+0000 7f6971000700 1 --2- 172.21.15.37:0/214725677 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f696c102c40 0x7f696c0702c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:45:21.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:21.391+0000 7f6971000700 1 -- 172.21.15.37:0/214725677 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6960005040 con 0x7f696c1035e0 2024-03-29T08:45:21.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:21.391+0000 7f6971a00700 1 --2- 172.21.15.37:0/214725677 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f696c102c40 0x7f696c0702c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:45:21.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:21.391+0000 7f6971000700 1 --2- 172.21.15.37:0/214725677 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f696c1035e0 0x7f696c070800 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f695800c890 tx=0x7f69580044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:45:21.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:21.391+0000 7f6969600700 1 -- 172.21.15.37:0/214725677 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f695800a7d0 con 0x7f696c1035e0 2024-03-29T08:45:21.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:21.391+0000 7f6969600700 1 -- 172.21.15.37:0/214725677 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f695800ade0 con 0x7f696c1035e0 2024-03-29T08:45:21.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:21.391+0000 7f6972e00700 1 -- 172.21.15.37:0/214725677 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f696c119c50 con 0x7f696c1035e0 2024-03-29T08:45:21.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:21.391+0000 7f6969600700 1 -- 172.21.15.37:0/214725677 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f69580113a0 con 0x7f696c1035e0 2024-03-29T08:45:21.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:21.392+0000 7f6972e00700 1 -- 172.21.15.37:0/214725677 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f696c11a0d0 con 0x7f696c1035e0 2024-03-29T08:45:21.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:21.392+0000 7f6972e00700 1 -- 172.21.15.37:0/214725677 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f696c0fe390 con 0x7f696c1035e0 2024-03-29T08:45:21.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:21.394+0000 7f6969600700 1 -- 172.21.15.37:0/214725677 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f6958012400 con 0x7f696c1035e0 2024-03-29T08:45:21.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:21.395+0000 7f6969600700 1 --2- 172.21.15.37:0/214725677 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f695406cc30 0x7f695406f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:45:21.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:21.395+0000 7f6969600700 1 -- 172.21.15.37:0/214725677 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f695808c8f0 con 0x7f696c1035e0 2024-03-29T08:45:21.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:21.395+0000 7f6971a00700 1 --2- 172.21.15.37:0/214725677 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f695406cc30 0x7f695406f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:45:21.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:21.396+0000 7f6971a00700 1 --2- 172.21.15.37:0/214725677 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f695406cc30 0x7f695406f0e0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f696000b840 tx=0x7f6960009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:45:21.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:21.398+0000 7f6969600700 1 -- 172.21.15.37:0/214725677 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f6958056930 con 0x7f696c1035e0 2024-03-29T08:45:21.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:21.698+0000 7f6972e00700 1 -- 172.21.15.37:0/214725677 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f696c05f5f0 con 0x7f696c1035e0 2024-03-29T08:45:21.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:21.698+0000 7f6969600700 1 -- 172.21.15.37:0/214725677 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+3583 (secure 0 0 0) 0x7f6958059f50 con 0x7f696c1035e0 2024-03-29T08:45:21.699 INFO:teuthology.orchestra.run.smithi037.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem is offline 2024-03-29T08:45:21.700 INFO:teuthology.orchestra.run.smithi037.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-03-29T08:45:21.700 INFO:teuthology.orchestra.run.smithi037.stdout: host smithi148 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/mon.smithi148/config 2024-03-29T08:45:21.700 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/podman: timeout after 895 seconds 2024-03-29T08:45:21.700 INFO:teuthology.orchestra.run.smithi037.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59 -e NODE_NAME=smithi148 -e CEPH_USE_RANDOM_NONCE=1 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4:/var/run/ceph:z -v /var/log/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4:/var/log/ceph:z -v /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpkx2rz2d4:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59 inventory --format=json-pretty --filter-for-batch 2024-03-29T08:45:21.700 INFO:teuthology.orchestra.run.smithi037.stdout:Traceback (most recent call last): 2024-03-29T08:45:21.700 INFO:teuthology.orchestra.run.smithi037.stdout: File "/usr/lib64/python3.6/runpy.py", line 193, in _run_module_as_main 2024-03-29T08:45:21.700 INFO:teuthology.orchestra.run.smithi037.stdout: "__main__", mod_spec) 2024-03-29T08:45:21.700 INFO:teuthology.orchestra.run.smithi037.stdout: File "/usr/lib64/python3.6/runpy.py", line 85, in _run_code 2024-03-29T08:45:21.700 INFO:teuthology.orchestra.run.smithi037.stdout: exec(code, run_globals) 2024-03-29T08:45:21.701 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 10889, in 2024-03-29T08:45:21.701 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 10877, in main 2024-03-29T08:45:21.701 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 2576, in _infer_config 2024-03-29T08:45:21.701 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 2492, in _infer_fsid 2024-03-29T08:45:21.701 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 2604, in _infer_image 2024-03-29T08:45:21.701 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 2479, in _validate_fsid 2024-03-29T08:45:21.701 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 7145, in command_ceph_volume 2024-03-29T08:45:21.701 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 2267, in call_throws 2024-03-29T08:45:21.701 INFO:teuthology.orchestra.run.smithi037.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59 -e NODE_NAME=smithi148 -e CEPH_USE_RANDOM_NONCE=1 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4:/var/run/ceph:z -v /var/log/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4:/var/log/ceph:z -v /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpkx2rz2d4:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59 inventory --format=json-pretty --filter-for-batch 2024-03-29T08:45:21.701 INFO:teuthology.orchestra.run.smithi037.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-03-29T08:45:21.701 INFO:teuthology.orchestra.run.smithi037.stdout: fs cephfs is degraded 2024-03-29T08:45:21.701 INFO:teuthology.orchestra.run.smithi037.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-03-29T08:45:21.701 INFO:teuthology.orchestra.run.smithi037.stdout: fs cephfs is offline because no MDS is active for it. 2024-03-29T08:45:21.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:21.701+0000 7f6972e00700 1 -- 172.21.15.37:0/214725677 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f695406cc30 msgr2=0x7f695406f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:45:21.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:21.701+0000 7f6972e00700 1 --2- 172.21.15.37:0/214725677 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f695406cc30 0x7f695406f0e0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f696000b840 tx=0x7f6960009e30 comp rx=0 tx=0).stop 2024-03-29T08:45:21.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:21.701+0000 7f6972e00700 1 -- 172.21.15.37:0/214725677 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f696c1035e0 msgr2=0x7f696c070800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:45:21.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:21.701+0000 7f6972e00700 1 --2- 172.21.15.37:0/214725677 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f696c1035e0 0x7f696c070800 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f695800c890 tx=0x7f69580044c0 comp rx=0 tx=0).stop 2024-03-29T08:45:21.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:21.701+0000 7f6972e00700 1 -- 172.21.15.37:0/214725677 shutdown_connections 2024-03-29T08:45:21.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:21.701+0000 7f6972e00700 1 --2- 172.21.15.37:0/214725677 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f695406cc30 0x7f695406f0e0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:45:21.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:21.701+0000 7f6972e00700 1 --2- 172.21.15.37:0/214725677 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f696c102c40 0x7f696c0702c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:45:21.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:21.701+0000 7f6972e00700 1 --2- 172.21.15.37:0/214725677 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f696c1035e0 0x7f696c070800 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:45:21.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:21.701+0000 7f6972e00700 1 -- 172.21.15.37:0/214725677 >> 172.21.15.37:0/214725677 conn(0x7f696c0f8830 msgr2=0x7f696c0f98d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:45:21.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:21.701+0000 7f6972e00700 1 -- 172.21.15.37:0/214725677 shutdown_connections 2024-03-29T08:45:21.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:21.701+0000 7f6972e00700 1 -- 172.21.15.37:0/214725677 wait complete. 2024-03-29T08:45:21.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:45:21 smithi037 ceph-mon[142325]: from='client.35858 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:45:21.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:45:21 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/1084787492' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T08:45:22.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:45:21 smithi148 ceph-mon[106692]: from='client.35858 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:45:22.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:45:21 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/1084787492' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T08:45:22.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:45:22 smithi037 ceph-mon[142325]: from='client.35870 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:45:22.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:45:22 smithi037 ceph-mon[142325]: pgmap v1035: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:45:22.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:45:22 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/214725677' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T08:45:23.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:45:22 smithi148 ceph-mon[106692]: from='client.35870 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:45:23.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:45:22 smithi148 ceph-mon[106692]: pgmap v1035: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:45:23.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:45:22 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/214725677' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T08:45:24.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:45:24 smithi037 ceph-mon[142325]: pgmap v1036: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:45:25.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:45:24 smithi148 ceph-mon[106692]: pgmap v1036: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:45:25.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:45:25 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:45:26.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:45:25 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:45:26.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:45:26 smithi037 ceph-mon[142325]: pgmap v1037: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:45:27.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:45:26 smithi148 ceph-mon[106692]: pgmap v1037: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:45:28.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:45:28 smithi037 ceph-mon[142325]: pgmap v1038: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:45:29.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:45:28 smithi148 ceph-mon[106692]: pgmap v1038: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:45:30.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:45:30 smithi037 ceph-mon[142325]: pgmap v1039: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:45:31.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:45:30 smithi148 ceph-mon[106692]: pgmap v1039: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:45:32.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:45:32 smithi037 ceph-mon[142325]: pgmap v1040: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:45:33.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:45:32 smithi148 ceph-mon[106692]: pgmap v1040: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:45:34.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:45:34 smithi037 ceph-mon[142325]: pgmap v1041: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:45:35.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:45:34 smithi148 ceph-mon[106692]: pgmap v1041: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:45:36.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:45:36 smithi037 ceph-mon[142325]: pgmap v1042: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:45:37.055 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:45:36 smithi148 ceph-mon[106692]: pgmap v1042: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:45:38.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:45:38 smithi037 ceph-mon[142325]: pgmap v1043: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:45:39.055 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:45:38 smithi148 ceph-mon[106692]: pgmap v1043: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:45:40.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:45:40 smithi037 ceph-mon[142325]: pgmap v1044: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:45:40.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:45:40 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:45:41.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:45:40 smithi148 ceph-mon[106692]: pgmap v1044: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:45:41.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:45:40 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:45:42.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:45:42 smithi037 ceph-mon[142325]: pgmap v1045: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:45:43.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:45:42 smithi148 ceph-mon[106692]: pgmap v1045: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:45:44.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:45:44 smithi037 ceph-mon[142325]: pgmap v1046: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:45:45.055 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:45:44 smithi148 ceph-mon[106692]: pgmap v1046: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:45:46.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:45:46 smithi037 ceph-mon[142325]: pgmap v1047: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:45:47.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:45:46 smithi148 ceph-mon[106692]: pgmap v1047: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:45:48.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:45:48 smithi037 ceph-mon[142325]: pgmap v1048: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:45:49.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:45:48 smithi148 ceph-mon[106692]: pgmap v1048: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:45:50.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:45:50 smithi037 ceph-mon[142325]: pgmap v1049: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:45:51.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:45:50 smithi148 ceph-mon[106692]: pgmap v1049: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:45:51.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:51.857+0000 7f827f200700 1 -- 172.21.15.37:0/2223396675 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f82801035e0 msgr2=0x7f82800fcc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:45:51.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:51.857+0000 7f827f200700 1 --2- 172.21.15.37:0/2223396675 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f82801035e0 0x7f82800fcc50 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f826c004940 tx=0x7f826c009e30 comp rx=0 tx=0).stop 2024-03-29T08:45:51.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:51.857+0000 7f827f200700 1 -- 172.21.15.37:0/2223396675 shutdown_connections 2024-03-29T08:45:51.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:51.857+0000 7f827f200700 1 --2- 172.21.15.37:0/2223396675 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f82801035e0 0x7f82800fcc50 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:45:51.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:51.857+0000 7f827f200700 1 --2- 172.21.15.37:0/2223396675 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8280102c40 0x7f8280103010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:45:51.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:51.857+0000 7f827f200700 1 -- 172.21.15.37:0/2223396675 >> 172.21.15.37:0/2223396675 conn(0x7f82800f8830 msgr2=0x7f82800fac40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:45:51.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:51.857+0000 7f827f200700 1 -- 172.21.15.37:0/2223396675 shutdown_connections 2024-03-29T08:45:51.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:51.857+0000 7f827f200700 1 -- 172.21.15.37:0/2223396675 wait complete. 2024-03-29T08:45:51.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:51.858+0000 7f827f200700 1 Processor -- start 2024-03-29T08:45:51.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:51.858+0000 7f827f200700 1 -- start start 2024-03-29T08:45:51.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:51.858+0000 7f827f200700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8280102c40 0x7f8280115ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:45:51.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:51.858+0000 7f827f200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f82801035e0 0x7f8280116020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:45:51.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:51.858+0000 7f827f200700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8280116670 con 0x7f82801035e0 2024-03-29T08:45:51.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:51.858+0000 7f827f200700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f828010fa60 con 0x7f8280102c40 2024-03-29T08:45:51.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:51.859+0000 7f827d400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f82801035e0 0x7f8280116020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:45:51.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:51.859+0000 7f827d400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f82801035e0 0x7f8280116020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36456/0 (socket says 172.21.15.37:36456) 2024-03-29T08:45:51.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:51.859+0000 7f827d400700 1 -- 172.21.15.37:0/912009169 learned_addr learned my addr 172.21.15.37:0/912009169 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:45:51.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:51.859+0000 7f827de00700 1 --2- 172.21.15.37:0/912009169 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8280102c40 0x7f8280115ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:45:51.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:51.859+0000 7f827d400700 1 -- 172.21.15.37:0/912009169 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8280102c40 msgr2=0x7f8280115ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:45:51.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:51.859+0000 7f827d400700 1 --2- 172.21.15.37:0/912009169 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8280102c40 0x7f8280115ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:45:51.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:51.859+0000 7f827d400700 1 -- 172.21.15.37:0/912009169 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f826c005040 con 0x7f82801035e0 2024-03-29T08:45:51.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:51.859+0000 7f827d400700 1 --2- 172.21.15.37:0/912009169 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f82801035e0 0x7f8280116020 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f826c009390 tx=0x7f826c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:45:51.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:51.860+0000 7f8276000700 1 -- 172.21.15.37:0/912009169 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f826c00e040 con 0x7f82801035e0 2024-03-29T08:45:51.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:51.860+0000 7f8276000700 1 -- 172.21.15.37:0/912009169 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f826c013420 con 0x7f82801035e0 2024-03-29T08:45:51.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:51.860+0000 7f8276000700 1 -- 172.21.15.37:0/912009169 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f826c01c630 con 0x7f82801035e0 2024-03-29T08:45:51.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:51.860+0000 7f827f200700 1 -- 172.21.15.37:0/912009169 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f828010fce0 con 0x7f82801035e0 2024-03-29T08:45:51.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:51.860+0000 7f827f200700 1 -- 172.21.15.37:0/912009169 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82801102a0 con 0x7f82801035e0 2024-03-29T08:45:51.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:51.861+0000 7f827f200700 1 -- 172.21.15.37:0/912009169 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82800fe390 con 0x7f82801035e0 2024-03-29T08:45:51.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:51.862+0000 7f8276000700 1 -- 172.21.15.37:0/912009169 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f826c013a00 con 0x7f82801035e0 2024-03-29T08:45:51.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:51.863+0000 7f8276000700 1 --2- 172.21.15.37:0/912009169 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f826406cc30 0x7f826406f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:45:51.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:51.863+0000 7f8276000700 1 -- 172.21.15.37:0/912009169 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f826c08d2d0 con 0x7f82801035e0 2024-03-29T08:45:51.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:51.863+0000 7f827de00700 1 --2- 172.21.15.37:0/912009169 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f826406cc30 0x7f826406f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:45:51.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:51.864+0000 7f827de00700 1 --2- 172.21.15.37:0/912009169 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f826406cc30 0x7f826406f0e0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f8270007fd0 tx=0x7f8270009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:45:51.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:51.868+0000 7f8276000700 1 -- 172.21.15.37:0/912009169 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f826c05ada0 con 0x7f82801035e0 2024-03-29T08:45:52.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:52.103+0000 7f827f200700 1 -- 172.21.15.37:0/912009169 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f8280109bd0 con 0x7f826406cc30 2024-03-29T08:45:52.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:52.105+0000 7f8276000700 1 -- 172.21.15.37:0/912009169 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f8280109bd0 con 0x7f826406cc30 2024-03-29T08:45:52.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:52.107+0000 7f827f200700 1 -- 172.21.15.37:0/912009169 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f826406cc30 msgr2=0x7f826406f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:45:52.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:52.107+0000 7f827f200700 1 --2- 172.21.15.37:0/912009169 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f826406cc30 0x7f826406f0e0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f8270007fd0 tx=0x7f8270009040 comp rx=0 tx=0).stop 2024-03-29T08:45:52.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:52.107+0000 7f827f200700 1 -- 172.21.15.37:0/912009169 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f82801035e0 msgr2=0x7f8280116020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:45:52.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:52.107+0000 7f827f200700 1 --2- 172.21.15.37:0/912009169 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f82801035e0 0x7f8280116020 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f826c009390 tx=0x7f826c009e30 comp rx=0 tx=0).stop 2024-03-29T08:45:52.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:52.107+0000 7f827f200700 1 -- 172.21.15.37:0/912009169 shutdown_connections 2024-03-29T08:45:52.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:52.107+0000 7f827f200700 1 --2- 172.21.15.37:0/912009169 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f826406cc30 0x7f826406f0e0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:45:52.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:52.107+0000 7f827f200700 1 --2- 172.21.15.37:0/912009169 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8280102c40 0x7f8280115ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:45:52.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:52.107+0000 7f827f200700 1 --2- 172.21.15.37:0/912009169 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f82801035e0 0x7f8280116020 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:45:52.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:52.107+0000 7f827f200700 1 -- 172.21.15.37:0/912009169 >> 172.21.15.37:0/912009169 conn(0x7f82800f8830 msgr2=0x7f82800f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:45:52.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:52.107+0000 7f827f200700 1 -- 172.21.15.37:0/912009169 shutdown_connections 2024-03-29T08:45:52.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:52.107+0000 7f827f200700 1 -- 172.21.15.37:0/912009169 wait complete. 2024-03-29T08:45:52.121 INFO:teuthology.orchestra.run.smithi037.stdout:true 2024-03-29T08:45:52.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:52.245+0000 7f801f400700 1 -- 172.21.15.37:0/67794760 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f80180fde90 msgr2=0x7f80180fe260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:45:52.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:52.245+0000 7f801f400700 1 --2- 172.21.15.37:0/67794760 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f80180fde90 0x7f80180fe260 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f800c004940 tx=0x7f800c009e30 comp rx=0 tx=0).stop 2024-03-29T08:45:52.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:52.245+0000 7f801f400700 1 -- 172.21.15.37:0/67794760 shutdown_connections 2024-03-29T08:45:52.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:52.245+0000 7f801f400700 1 --2- 172.21.15.37:0/67794760 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f80180fe830 0x7f8018106f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:45:52.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:52.245+0000 7f801f400700 1 --2- 172.21.15.37:0/67794760 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f80180fde90 0x7f80180fe260 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:45:52.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:52.245+0000 7f801f400700 1 -- 172.21.15.37:0/67794760 >> 172.21.15.37:0/67794760 conn(0x7f8018072bc0 msgr2=0x7f8018072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:45:52.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:52.245+0000 7f801f400700 1 -- 172.21.15.37:0/67794760 shutdown_connections 2024-03-29T08:45:52.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:52.245+0000 7f801f400700 1 -- 172.21.15.37:0/67794760 wait complete. 2024-03-29T08:45:52.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:52.246+0000 7f801f400700 1 Processor -- start 2024-03-29T08:45:52.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:52.246+0000 7f801f400700 1 -- start start 2024-03-29T08:45:52.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:52.247+0000 7f801f400700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f80180fde90 0x7f8018115d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:45:52.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:52.247+0000 7f801f400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f80180fe830 0x7f8018116240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:45:52.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:52.247+0000 7f801f400700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8018116890 con 0x7f80180fe830 2024-03-29T08:45:52.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:52.247+0000 7f801f400700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f801810fc60 con 0x7f80180fde90 2024-03-29T08:45:52.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:52.247+0000 7f8017e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f80180fe830 0x7f8018116240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:45:52.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:52.247+0000 7f8017e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f80180fe830 0x7f8018116240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36472/0 (socket says 172.21.15.37:36472) 2024-03-29T08:45:52.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:52.247+0000 7f8017e00700 1 -- 172.21.15.37:0/4146117819 learned_addr learned my addr 172.21.15.37:0/4146117819 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:45:52.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:52.247+0000 7f801ca00700 1 --2- 172.21.15.37:0/4146117819 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f80180fde90 0x7f8018115d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:45:52.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:52.248+0000 7f8017e00700 1 -- 172.21.15.37:0/4146117819 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f80180fde90 msgr2=0x7f8018115d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:45:52.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:52.248+0000 7f8017e00700 1 --2- 172.21.15.37:0/4146117819 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f80180fde90 0x7f8018115d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:45:52.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:52.248+0000 7f8017e00700 1 -- 172.21.15.37:0/4146117819 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f800c005040 con 0x7f80180fe830 2024-03-29T08:45:52.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:52.248+0000 7f801ca00700 1 --2- 172.21.15.37:0/4146117819 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f80180fde90 0x7f8018115d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:45:52.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:52.248+0000 7f8017e00700 1 --2- 172.21.15.37:0/4146117819 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f80180fe830 0x7f8018116240 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f8004002a50 tx=0x7f80040046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:45:52.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:52.248+0000 7f8014c00700 1 -- 172.21.15.37:0/4146117819 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8004007cc0 con 0x7f80180fe830 2024-03-29T08:45:52.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:52.248+0000 7f8014c00700 1 -- 172.21.15.37:0/4146117819 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f800400f040 con 0x7f80180fe830 2024-03-29T08:45:52.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:52.248+0000 7f8014c00700 1 -- 172.21.15.37:0/4146117819 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8004009540 con 0x7f80180fe830 2024-03-29T08:45:52.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:52.248+0000 7f801f400700 1 -- 172.21.15.37:0/4146117819 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f801810ff40 con 0x7f80180fe830 2024-03-29T08:45:52.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:52.248+0000 7f801f400700 1 -- 172.21.15.37:0/4146117819 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8018110480 con 0x7f80180fe830 2024-03-29T08:45:52.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:52.249+0000 7f801f400700 1 -- 172.21.15.37:0/4146117819 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8018106a20 con 0x7f80180fe830 2024-03-29T08:45:52.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:52.251+0000 7f8014c00700 1 -- 172.21.15.37:0/4146117819 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f80040096c0 con 0x7f80180fe830 2024-03-29T08:45:52.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:52.252+0000 7f8014c00700 1 --2- 172.21.15.37:0/4146117819 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f800006cc30 0x7f800006f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:45:52.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:52.252+0000 7f8014c00700 1 -- 172.21.15.37:0/4146117819 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f800408bde0 con 0x7f80180fe830 2024-03-29T08:45:52.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:52.252+0000 7f801ca00700 1 --2- 172.21.15.37:0/4146117819 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f800006cc30 0x7f800006f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:45:52.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:52.253+0000 7f801ca00700 1 --2- 172.21.15.37:0/4146117819 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f800006cc30 0x7f800006f0e0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f800c00b3b0 tx=0x7f800c009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:45:52.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:52.257+0000 7f8014c00700 1 -- 172.21.15.37:0/4146117819 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f8004059830 con 0x7f80180fe830 2024-03-29T08:45:52.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:52.492+0000 7f801f400700 1 -- 172.21.15.37:0/4146117819 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f80181113c0 con 0x7f800006cc30 2024-03-29T08:45:52.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:52.494+0000 7f8014c00700 1 -- 172.21.15.37:0/4146117819 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f80181113c0 con 0x7f800006cc30 2024-03-29T08:45:52.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:52.496+0000 7f801f400700 1 -- 172.21.15.37:0/4146117819 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f800006cc30 msgr2=0x7f800006f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:45:52.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:52.496+0000 7f801f400700 1 --2- 172.21.15.37:0/4146117819 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f800006cc30 0x7f800006f0e0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f800c00b3b0 tx=0x7f800c009e30 comp rx=0 tx=0).stop 2024-03-29T08:45:52.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:52.496+0000 7f801f400700 1 -- 172.21.15.37:0/4146117819 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f80180fe830 msgr2=0x7f8018116240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:45:52.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:52.496+0000 7f801f400700 1 --2- 172.21.15.37:0/4146117819 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f80180fe830 0x7f8018116240 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f8004002a50 tx=0x7f80040046b0 comp rx=0 tx=0).stop 2024-03-29T08:45:52.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:52.496+0000 7f801f400700 1 -- 172.21.15.37:0/4146117819 shutdown_connections 2024-03-29T08:45:52.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:52.496+0000 7f801f400700 1 --2- 172.21.15.37:0/4146117819 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f800006cc30 0x7f800006f0e0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:45:52.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:52.496+0000 7f801f400700 1 --2- 172.21.15.37:0/4146117819 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f80180fde90 0x7f8018115d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:45:52.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:52.496+0000 7f801f400700 1 --2- 172.21.15.37:0/4146117819 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f80180fe830 0x7f8018116240 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:45:52.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:52.496+0000 7f801f400700 1 -- 172.21.15.37:0/4146117819 >> 172.21.15.37:0/4146117819 conn(0x7f8018072bc0 msgr2=0x7f80180fd590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:45:52.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:52.496+0000 7f801f400700 1 -- 172.21.15.37:0/4146117819 shutdown_connections 2024-03-29T08:45:52.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:52.496+0000 7f801f400700 1 -- 172.21.15.37:0/4146117819 wait complete. 2024-03-29T08:45:52.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:52.635+0000 7f131e000700 1 -- 172.21.15.37:0/1430624074 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f13181037a0 msgr2=0x7f13180fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:45:52.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:52.635+0000 7f131e000700 1 --2- 172.21.15.37:0/1430624074 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f13181037a0 0x7f13180fce10 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f1304004940 tx=0x7f1304009e30 comp rx=0 tx=0).stop 2024-03-29T08:45:52.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:52.635+0000 7f131e000700 1 -- 172.21.15.37:0/1430624074 shutdown_connections 2024-03-29T08:45:52.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:52.635+0000 7f131e000700 1 --2- 172.21.15.37:0/1430624074 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f13181037a0 0x7f13180fce10 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:45:52.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:52.635+0000 7f131e000700 1 --2- 172.21.15.37:0/1430624074 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1318102e00 0x7f13181031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:45:52.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:52.635+0000 7f131e000700 1 -- 172.21.15.37:0/1430624074 >> 172.21.15.37:0/1430624074 conn(0x7f13180f89b0 msgr2=0x7f13180fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:45:52.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:52.635+0000 7f131e000700 1 -- 172.21.15.37:0/1430624074 shutdown_connections 2024-03-29T08:45:52.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:52.635+0000 7f131e000700 1 -- 172.21.15.37:0/1430624074 wait complete. 2024-03-29T08:45:52.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:52.636+0000 7f131e000700 1 Processor -- start 2024-03-29T08:45:52.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:52.636+0000 7f131e000700 1 -- start start 2024-03-29T08:45:52.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:52.636+0000 7f131e000700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1318102e00 0x7f1318115cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:45:52.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:52.636+0000 7f131e000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f13181037a0 0x7f1318116230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:45:52.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:52.636+0000 7f131e000700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1318116880 con 0x7f13181037a0 2024-03-29T08:45:52.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:52.636+0000 7f131e000700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f131810fc50 con 0x7f1318102e00 2024-03-29T08:45:52.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:52.637+0000 7f1316a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f13181037a0 0x7f1318116230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:45:52.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:52.637+0000 7f1316a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f13181037a0 0x7f1318116230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36482/0 (socket says 172.21.15.37:36482) 2024-03-29T08:45:52.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:52.637+0000 7f1316a00700 1 -- 172.21.15.37:0/1320247769 learned_addr learned my addr 172.21.15.37:0/1320247769 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:45:52.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:52.637+0000 7f1317400700 1 --2- 172.21.15.37:0/1320247769 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1318102e00 0x7f1318115cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:45:52.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:52.637+0000 7f1316a00700 1 -- 172.21.15.37:0/1320247769 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1318102e00 msgr2=0x7f1318115cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:45:52.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:52.637+0000 7f1316a00700 1 --2- 172.21.15.37:0/1320247769 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1318102e00 0x7f1318115cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:45:52.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:52.637+0000 7f1316a00700 1 -- 172.21.15.37:0/1320247769 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1304005040 con 0x7f13181037a0 2024-03-29T08:45:52.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:52.637+0000 7f1316a00700 1 --2- 172.21.15.37:0/1320247769 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f13181037a0 0x7f1318116230 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f1304009390 tx=0x7f1304009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:45:52.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:52.637+0000 7f130f400700 1 -- 172.21.15.37:0/1320247769 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f130400e040 con 0x7f13181037a0 2024-03-29T08:45:52.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:52.637+0000 7f130f400700 1 -- 172.21.15.37:0/1320247769 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f1304013420 con 0x7f13181037a0 2024-03-29T08:45:52.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:52.637+0000 7f130f400700 1 -- 172.21.15.37:0/1320247769 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f130401c630 con 0x7f13181037a0 2024-03-29T08:45:52.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:52.637+0000 7f131e000700 1 -- 172.21.15.37:0/1320247769 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f131810fed0 con 0x7f13181037a0 2024-03-29T08:45:52.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:52.638+0000 7f131e000700 1 -- 172.21.15.37:0/1320247769 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1318110490 con 0x7f13181037a0 2024-03-29T08:45:52.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:52.639+0000 7f131e000700 1 -- 172.21.15.37:0/1320247769 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13180fe550 con 0x7f13181037a0 2024-03-29T08:45:52.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:52.640+0000 7f130f400700 1 -- 172.21.15.37:0/1320247769 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f1304013a00 con 0x7f13181037a0 2024-03-29T08:45:52.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:52.641+0000 7f130f400700 1 --2- 172.21.15.37:0/1320247769 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f12fc06cc30 0x7f12fc06f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:45:52.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:52.641+0000 7f130f400700 1 -- 172.21.15.37:0/1320247769 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f130408d2c0 con 0x7f13181037a0 2024-03-29T08:45:52.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:52.641+0000 7f1317400700 1 --2- 172.21.15.37:0/1320247769 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f12fc06cc30 0x7f12fc06f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:45:52.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:52.642+0000 7f1317400700 1 --2- 172.21.15.37:0/1320247769 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f12fc06cc30 0x7f12fc06f0e0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f1308004990 tx=0x7f1308009ee0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:45:52.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:52.646+0000 7f130f400700 1 -- 172.21.15.37:0/1320247769 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f130405ad90 con 0x7f13181037a0 2024-03-29T08:45:52.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:52.868+0000 7f131e000700 1 -- 172.21.15.37:0/1320247769 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f1318109e10 con 0x7f12fc06cc30 2024-03-29T08:45:52.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:52.878+0000 7f130f400700 1 -- 172.21.15.37:0/1320247769 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f1318109e10 con 0x7f12fc06cc30 2024-03-29T08:45:52.877 INFO:teuthology.orchestra.run.smithi037.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-29T08:45:52.878 INFO:teuthology.orchestra.run.smithi037.stdout:alertmanager.smithi037 smithi037 *:9093,9094 running (35m) 12m ago 48m 14.3M - 0.25.0 c8568f914cd2 2f9f9746ed0f 2024-03-29T08:45:52.878 INFO:teuthology.orchestra.run.smithi037.stdout:crash.smithi037 smithi037 running (32m) 12m ago 48m 6647k - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 178d416a3186 2024-03-29T08:45:52.878 INFO:teuthology.orchestra.run.smithi037.stdout:crash.smithi148 smithi148 running (32m) 12m ago 47m 6656k - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 23666989d58f 2024-03-29T08:45:52.878 INFO:teuthology.orchestra.run.smithi037.stdout:grafana.smithi037 smithi037 *:3000 running (35m) 12m ago 48m 74.5M - 9.4.7 954c08fa6188 6b976fe49952 2024-03-29T08:45:52.878 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi037.iifsmh smithi037 running (28m) 12m ago 42m 16.4M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 2d97f562cc5b 2024-03-29T08:45:52.878 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi037.mmwgxe smithi037 running (28m) 12m ago 42m 16.7M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff ec37037f46e8 2024-03-29T08:45:52.878 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi148.qukpjh smithi148 running (12m) 12m ago 42m 10.7M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 3257219bc046 2024-03-29T08:45:52.878 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi148.sqteih smithi148 running (27m) 12m ago 42m 16.3M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 44c15c77037a 2024-03-29T08:45:52.878 INFO:teuthology.orchestra.run.smithi037.stdout:mgr.smithi037.wjikii smithi037 *:8443,9283,8765 running (38m) 12m ago 50m 510M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff c5a1417b9417 2024-03-29T08:45:52.878 INFO:teuthology.orchestra.run.smithi037.stdout:mgr.smithi148.onhgvm smithi148 *:8443,9283,8765 running (36m) 12m ago 47m 456M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff fcfdbce062bd 2024-03-29T08:45:52.878 INFO:teuthology.orchestra.run.smithi037.stdout:mon.smithi037 smithi037 running (33m) 12m ago 50m 70.2M 2048M 18.2.2-1015-gd0e392d9 1cc2f45c17ff eea2060783af 2024-03-29T08:45:52.878 INFO:teuthology.orchestra.run.smithi037.stdout:mon.smithi148 smithi148 running (33m) 12m ago 47m 50.4M 2048M 18.2.2-1015-gd0e392d9 1cc2f45c17ff ad4b2c5748be 2024-03-29T08:45:52.878 INFO:teuthology.orchestra.run.smithi037.stdout:node-exporter.smithi037 smithi037 *:9100 running (36m) 12m ago 48m 8514k - 1.5.0 0da6a335fe13 26b8ae9cb878 2024-03-29T08:45:52.878 INFO:teuthology.orchestra.run.smithi037.stdout:node-exporter.smithi148 smithi148 *:9100 running (36m) 12m ago 47m 8464k - 1.5.0 0da6a335fe13 98414e4e64ed 2024-03-29T08:45:52.879 INFO:teuthology.orchestra.run.smithi037.stdout:osd.0 smithi037 running (32m) 12m ago 46m 108M 1619M 18.2.2-1015-gd0e392d9 1cc2f45c17ff c0c584f6245c 2024-03-29T08:45:52.879 INFO:teuthology.orchestra.run.smithi037.stdout:osd.1 smithi037 running (31m) 12m ago 46m 104M 1619M 18.2.2-1015-gd0e392d9 1cc2f45c17ff a4a2ff4cc83d 2024-03-29T08:45:52.879 INFO:teuthology.orchestra.run.smithi037.stdout:osd.2 smithi037 running (30m) 12m ago 45m 85.9M 1619M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 8b1bc71aa79e 2024-03-29T08:45:52.879 INFO:teuthology.orchestra.run.smithi037.stdout:osd.3 smithi148 running (30m) 12m ago 44m 109M 2642M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 1c656ab8ff86 2024-03-29T08:45:52.879 INFO:teuthology.orchestra.run.smithi037.stdout:osd.4 smithi148 running (29m) 12m ago 44m 101M 2642M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 966a9de11d22 2024-03-29T08:45:52.879 INFO:teuthology.orchestra.run.smithi037.stdout:osd.5 smithi148 running (28m) 12m ago 43m 89.4M 2642M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 474c4f9212ce 2024-03-29T08:45:52.879 INFO:teuthology.orchestra.run.smithi037.stdout:prometheus.smithi037 smithi037 *:9095 running (36m) 12m ago 48m 61.0M - 2.43.0 a07b618ecd1d 30c0223ec589 2024-03-29T08:45:52.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:52.880+0000 7f131e000700 1 -- 172.21.15.37:0/1320247769 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f12fc06cc30 msgr2=0x7f12fc06f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:45:52.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:52.880+0000 7f131e000700 1 --2- 172.21.15.37:0/1320247769 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f12fc06cc30 0x7f12fc06f0e0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f1308004990 tx=0x7f1308009ee0 comp rx=0 tx=0).stop 2024-03-29T08:45:52.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:52.880+0000 7f131e000700 1 -- 172.21.15.37:0/1320247769 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f13181037a0 msgr2=0x7f1318116230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:45:52.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:52.880+0000 7f131e000700 1 --2- 172.21.15.37:0/1320247769 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f13181037a0 0x7f1318116230 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f1304009390 tx=0x7f1304009e30 comp rx=0 tx=0).stop 2024-03-29T08:45:52.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:52.880+0000 7f131e000700 1 -- 172.21.15.37:0/1320247769 shutdown_connections 2024-03-29T08:45:52.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:52.880+0000 7f131e000700 1 --2- 172.21.15.37:0/1320247769 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f12fc06cc30 0x7f12fc06f0e0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:45:52.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:52.880+0000 7f131e000700 1 --2- 172.21.15.37:0/1320247769 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1318102e00 0x7f1318115cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:45:52.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:52.880+0000 7f131e000700 1 --2- 172.21.15.37:0/1320247769 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f13181037a0 0x7f1318116230 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:45:52.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:52.880+0000 7f131e000700 1 -- 172.21.15.37:0/1320247769 >> 172.21.15.37:0/1320247769 conn(0x7f13180f89b0 msgr2=0x7f13180f9a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:45:52.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:52.880+0000 7f131e000700 1 -- 172.21.15.37:0/1320247769 shutdown_connections 2024-03-29T08:45:52.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:52.880+0000 7f131e000700 1 -- 172.21.15.37:0/1320247769 wait complete. 2024-03-29T08:45:52.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:45:52 smithi037 ceph-mon[142325]: pgmap v1050: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:45:53.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:53.016+0000 7f37fa200700 1 -- 172.21.15.37:0/80128512 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f37f4070300 msgr2=0x7f37f40706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:45:53.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:53.016+0000 7f37fa200700 1 --2- 172.21.15.37:0/80128512 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f37f4070300 0x7f37f40706d0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f37e8004970 tx=0x7f37e8009e30 comp rx=0 tx=0).stop 2024-03-29T08:45:53.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:53.016+0000 7f37fa200700 1 -- 172.21.15.37:0/80128512 shutdown_connections 2024-03-29T08:45:53.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:53.016+0000 7f37fa200700 1 --2- 172.21.15.37:0/80128512 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f37f4070c10 0x7f37f4108fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:45:53.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:53.016+0000 7f37fa200700 1 --2- 172.21.15.37:0/80128512 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f37f4070300 0x7f37f40706d0 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:45:53.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:53.016+0000 7f37fa200700 1 -- 172.21.15.37:0/80128512 >> 172.21.15.37:0/80128512 conn(0x7f37f40f87e0 msgr2=0x7f37f40fabf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:45:53.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:53.016+0000 7f37fa200700 1 -- 172.21.15.37:0/80128512 shutdown_connections 2024-03-29T08:45:53.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:53.016+0000 7f37fa200700 1 -- 172.21.15.37:0/80128512 wait complete. 2024-03-29T08:45:53.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:53.017+0000 7f37fa200700 1 Processor -- start 2024-03-29T08:45:53.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:53.017+0000 7f37fa200700 1 -- start start 2024-03-29T08:45:53.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:53.018+0000 7f37fa200700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f37f4070300 0x7f37f4115b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:45:53.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:53.018+0000 7f37fa200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f37f4070c10 0x7f37f4116090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:45:53.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:53.018+0000 7f37fa200700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37f41166e0 con 0x7f37f4070c10 2024-03-29T08:45:53.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:53.018+0000 7f37fa200700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37f410fad0 con 0x7f37f4070300 2024-03-29T08:45:53.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:53.018+0000 7f37f3e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f37f4070c10 0x7f37f4116090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:45:53.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:53.018+0000 7f37f3e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f37f4070c10 0x7f37f4116090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36510/0 (socket says 172.21.15.37:36510) 2024-03-29T08:45:53.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:53.018+0000 7f37f3e00700 1 -- 172.21.15.37:0/652618989 learned_addr learned my addr 172.21.15.37:0/652618989 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:45:53.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:53.018+0000 7f37f8e00700 1 --2- 172.21.15.37:0/652618989 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f37f4070300 0x7f37f4115b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:45:53.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:53.018+0000 7f37f3e00700 1 -- 172.21.15.37:0/652618989 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f37f4070300 msgr2=0x7f37f4115b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:45:53.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:53.018+0000 7f37f3e00700 1 --2- 172.21.15.37:0/652618989 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f37f4070300 0x7f37f4115b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:45:53.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:53.018+0000 7f37f3e00700 1 -- 172.21.15.37:0/652618989 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37e8005040 con 0x7f37f4070c10 2024-03-29T08:45:53.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:53.018+0000 7f37f8e00700 1 --2- 172.21.15.37:0/652618989 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f37f4070300 0x7f37f4115b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:45:53.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:53.019+0000 7f37f3e00700 1 --2- 172.21.15.37:0/652618989 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f37f4070c10 0x7f37f4116090 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f37e000eb40 tx=0x7f37e000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:45:53.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:53.019+0000 7f37f0c00700 1 -- 172.21.15.37:0/652618989 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f37e0007cc0 con 0x7f37f4070c10 2024-03-29T08:45:53.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:53.019+0000 7f37f0c00700 1 -- 172.21.15.37:0/652618989 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f37e000f040 con 0x7f37f4070c10 2024-03-29T08:45:53.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:53.019+0000 7f37f0c00700 1 -- 172.21.15.37:0/652618989 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f37e0009570 con 0x7f37f4070c10 2024-03-29T08:45:53.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:53.019+0000 7f37fa200700 1 -- 172.21.15.37:0/652618989 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37f410fd50 con 0x7f37f4070c10 2024-03-29T08:45:53.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:53.019+0000 7f37fa200700 1 -- 172.21.15.37:0/652618989 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37f4110290 con 0x7f37f4070c10 2024-03-29T08:45:53.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:53.020+0000 7f37fa200700 1 -- 172.21.15.37:0/652618989 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37f4071a40 con 0x7f37f4070c10 2024-03-29T08:45:53.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:53.021+0000 7f37f0c00700 1 -- 172.21.15.37:0/652618989 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f37e00096d0 con 0x7f37f4070c10 2024-03-29T08:45:53.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:53.022+0000 7f37f0c00700 1 --2- 172.21.15.37:0/652618989 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f37dc06cc30 0x7f37dc06f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:45:53.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:53.022+0000 7f37f0c00700 1 -- 172.21.15.37:0/652618989 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f37e008bcb0 con 0x7f37f4070c10 2024-03-29T08:45:53.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:53.023+0000 7f37f8e00700 1 --2- 172.21.15.37:0/652618989 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f37dc06cc30 0x7f37dc06f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:45:53.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:53.023+0000 7f37f8e00700 1 --2- 172.21.15.37:0/652618989 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f37dc06cc30 0x7f37dc06f0e0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f37e8004560 tx=0x7f37e8009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:45:53.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:53.026+0000 7f37f0c00700 1 -- 172.21.15.37:0/652618989 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f37e00597b0 con 0x7f37f4070c10 2024-03-29T08:45:53.055 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:45:52 smithi148 ceph-mon[106692]: pgmap v1050: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:45:53.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:53.331+0000 7f37fa200700 1 -- 172.21.15.37:0/652618989 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f37f410be00 con 0x7f37f4070c10 2024-03-29T08:45:53.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:53.331+0000 7f37f0c00700 1 -- 172.21.15.37:0/652618989 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7f37e0059340 con 0x7f37f4070c10 2024-03-29T08:45:53.331 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-29T08:45:53.331 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": { 2024-03-29T08:45:53.331 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 2 2024-03-29T08:45:53.331 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:45:53.331 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": { 2024-03-29T08:45:53.331 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 2 2024-03-29T08:45:53.331 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:45:53.331 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": { 2024-03-29T08:45:53.331 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 6 2024-03-29T08:45:53.332 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:45:53.332 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": { 2024-03-29T08:45:53.332 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 4 2024-03-29T08:45:53.332 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:45:53.332 INFO:teuthology.orchestra.run.smithi037.stdout: "overall": { 2024-03-29T08:45:53.332 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 14 2024-03-29T08:45:53.332 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-29T08:45:53.332 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-29T08:45:53.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:53.333+0000 7f37fa200700 1 -- 172.21.15.37:0/652618989 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f37dc06cc30 msgr2=0x7f37dc06f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:45:53.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:53.333+0000 7f37fa200700 1 --2- 172.21.15.37:0/652618989 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f37dc06cc30 0x7f37dc06f0e0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f37e8004560 tx=0x7f37e8009e30 comp rx=0 tx=0).stop 2024-03-29T08:45:53.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:53.333+0000 7f37fa200700 1 -- 172.21.15.37:0/652618989 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f37f4070c10 msgr2=0x7f37f4116090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:45:53.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:53.333+0000 7f37fa200700 1 --2- 172.21.15.37:0/652618989 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f37f4070c10 0x7f37f4116090 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f37e000eb40 tx=0x7f37e000b040 comp rx=0 tx=0).stop 2024-03-29T08:45:53.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:53.334+0000 7f37fa200700 1 -- 172.21.15.37:0/652618989 shutdown_connections 2024-03-29T08:45:53.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:53.334+0000 7f37fa200700 1 --2- 172.21.15.37:0/652618989 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f37dc06cc30 0x7f37dc06f0e0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:45:53.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:53.334+0000 7f37fa200700 1 --2- 172.21.15.37:0/652618989 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f37f4070300 0x7f37f4115b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:45:53.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:53.334+0000 7f37fa200700 1 --2- 172.21.15.37:0/652618989 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f37f4070c10 0x7f37f4116090 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:45:53.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:53.334+0000 7f37fa200700 1 -- 172.21.15.37:0/652618989 >> 172.21.15.37:0/652618989 conn(0x7f37f40f87e0 msgr2=0x7f37f40f9d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:45:53.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:53.334+0000 7f37fa200700 1 -- 172.21.15.37:0/652618989 shutdown_connections 2024-03-29T08:45:53.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:53.334+0000 7f37fa200700 1 -- 172.21.15.37:0/652618989 wait complete. 2024-03-29T08:45:53.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:53.470+0000 7fdd8a000700 1 -- 172.21.15.37:0/2784306250 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdd84102e00 msgr2=0x7fdd841031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:45:53.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:53.470+0000 7fdd8a000700 1 --2- 172.21.15.37:0/2784306250 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdd84102e00 0x7fdd841031d0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7fdd74004940 tx=0x7fdd74009e30 comp rx=0 tx=0).stop 2024-03-29T08:45:53.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:53.470+0000 7fdd8a000700 1 -- 172.21.15.37:0/2784306250 shutdown_connections 2024-03-29T08:45:53.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:53.470+0000 7fdd8a000700 1 --2- 172.21.15.37:0/2784306250 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdd841037a0 0x7fdd840fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:45:53.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:53.470+0000 7fdd8a000700 1 --2- 172.21.15.37:0/2784306250 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdd84102e00 0x7fdd841031d0 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:45:53.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:53.470+0000 7fdd8a000700 1 -- 172.21.15.37:0/2784306250 >> 172.21.15.37:0/2784306250 conn(0x7fdd840f89b0 msgr2=0x7fdd840fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:45:53.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:53.470+0000 7fdd8a000700 1 -- 172.21.15.37:0/2784306250 shutdown_connections 2024-03-29T08:45:53.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:53.470+0000 7fdd8a000700 1 -- 172.21.15.37:0/2784306250 wait complete. 2024-03-29T08:45:53.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:53.471+0000 7fdd8a000700 1 Processor -- start 2024-03-29T08:45:53.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:53.471+0000 7fdd8a000700 1 -- start start 2024-03-29T08:45:53.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:53.471+0000 7fdd8a000700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdd84102e00 0x7fdd84113b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:45:53.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:53.471+0000 7fdd8a000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdd841037a0 0x7fdd84114060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:45:53.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:53.472+0000 7fdd8a000700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd841146b0 con 0x7fdd841037a0 2024-03-29T08:45:53.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:53.472+0000 7fdd8a000700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd8410daa0 con 0x7fdd84102e00 2024-03-29T08:45:53.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:53.472+0000 7fdd82a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdd841037a0 0x7fdd84114060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:45:53.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:53.472+0000 7fdd82a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdd841037a0 0x7fdd84114060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36524/0 (socket says 172.21.15.37:36524) 2024-03-29T08:45:53.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:53.472+0000 7fdd82a00700 1 -- 172.21.15.37:0/1689672143 learned_addr learned my addr 172.21.15.37:0/1689672143 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:45:53.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:53.472+0000 7fdd83400700 1 --2- 172.21.15.37:0/1689672143 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdd84102e00 0x7fdd84113b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:45:53.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:53.472+0000 7fdd82a00700 1 -- 172.21.15.37:0/1689672143 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdd84102e00 msgr2=0x7fdd84113b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:45:53.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:53.472+0000 7fdd82a00700 1 --2- 172.21.15.37:0/1689672143 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdd84102e00 0x7fdd84113b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:45:53.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:53.472+0000 7fdd82a00700 1 -- 172.21.15.37:0/1689672143 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd74005040 con 0x7fdd841037a0 2024-03-29T08:45:53.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:53.472+0000 7fdd83400700 1 --2- 172.21.15.37:0/1689672143 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdd84102e00 0x7fdd84113b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:45:53.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:53.472+0000 7fdd82a00700 1 --2- 172.21.15.37:0/1689672143 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdd841037a0 0x7fdd84114060 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7fdd70002a50 tx=0x7fdd70004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:45:53.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:53.473+0000 7fdd7b400700 1 -- 172.21.15.37:0/1689672143 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdd70007cc0 con 0x7fdd841037a0 2024-03-29T08:45:53.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:53.473+0000 7fdd7b400700 1 -- 172.21.15.37:0/1689672143 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fdd7000f040 con 0x7fdd841037a0 2024-03-29T08:45:53.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:53.473+0000 7fdd7b400700 1 -- 172.21.15.37:0/1689672143 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdd7000d9a0 con 0x7fdd841037a0 2024-03-29T08:45:53.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:53.473+0000 7fdd8a000700 1 -- 172.21.15.37:0/1689672143 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd8410dd80 con 0x7fdd841037a0 2024-03-29T08:45:53.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:53.473+0000 7fdd8a000700 1 -- 172.21.15.37:0/1689672143 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd8410e340 con 0x7fdd841037a0 2024-03-29T08:45:53.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:53.474+0000 7fdd8a000700 1 -- 172.21.15.37:0/1689672143 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd840fe550 con 0x7fdd841037a0 2024-03-29T08:45:53.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:53.475+0000 7fdd7b400700 1 -- 172.21.15.37:0/1689672143 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fdd7000dc60 con 0x7fdd841037a0 2024-03-29T08:45:53.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:53.476+0000 7fdd7b400700 1 --2- 172.21.15.37:0/1689672143 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fdd6806cc30 0x7fdd6806f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:45:53.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:53.476+0000 7fdd7b400700 1 -- 172.21.15.37:0/1689672143 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fdd7008bef0 con 0x7fdd841037a0 2024-03-29T08:45:53.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:53.476+0000 7fdd83400700 1 --2- 172.21.15.37:0/1689672143 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fdd6806cc30 0x7fdd6806f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:45:53.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:53.477+0000 7fdd83400700 1 --2- 172.21.15.37:0/1689672143 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fdd6806cc30 0x7fdd6806f0e0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fdd74000c00 tx=0x7fdd74009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:45:53.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:53.480+0000 7fdd7b400700 1 -- 172.21.15.37:0/1689672143 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fdd700598c0 con 0x7fdd841037a0 2024-03-29T08:45:53.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:53.742+0000 7fdd8a000700 1 -- 172.21.15.37:0/1689672143 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fdd84109dc0 con 0x7fdd841037a0 2024-03-29T08:45:53.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:53.742+0000 7fdd7b400700 1 -- 172.21.15.37:0/1689672143 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 30 v30) v1 ==== 76+0+1841 (secure 0 0 0) 0x7fdd70059450 con 0x7fdd841037a0 2024-03-29T08:45:53.742 INFO:teuthology.orchestra.run.smithi037.stdout:e30 2024-03-29T08:45:53.742 INFO:teuthology.orchestra.run.smithi037.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-29T08:45:53.742 INFO:teuthology.orchestra.run.smithi037.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T08:45:53.743 INFO:teuthology.orchestra.run.smithi037.stdout:legacy client fscid: 1 2024-03-29T08:45:53.743 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:45:53.743 INFO:teuthology.orchestra.run.smithi037.stdout:Filesystem 'cephfs' (1) 2024-03-29T08:45:53.743 INFO:teuthology.orchestra.run.smithi037.stdout:fs_name cephfs 2024-03-29T08:45:53.743 INFO:teuthology.orchestra.run.smithi037.stdout:epoch 21 2024-03-29T08:45:53.743 INFO:teuthology.orchestra.run.smithi037.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-03-29T08:45:53.743 INFO:teuthology.orchestra.run.smithi037.stdout:created 2024-03-29T08:03:29.241269+0000 2024-03-29T08:45:53.743 INFO:teuthology.orchestra.run.smithi037.stdout:modified 2024-03-29T08:17:16.237238+0000 2024-03-29T08:45:53.743 INFO:teuthology.orchestra.run.smithi037.stdout:tableserver 0 2024-03-29T08:45:53.743 INFO:teuthology.orchestra.run.smithi037.stdout:root 0 2024-03-29T08:45:53.743 INFO:teuthology.orchestra.run.smithi037.stdout:session_timeout 60 2024-03-29T08:45:53.743 INFO:teuthology.orchestra.run.smithi037.stdout:session_autoclose 300 2024-03-29T08:45:53.743 INFO:teuthology.orchestra.run.smithi037.stdout:max_file_size 1099511627776 2024-03-29T08:45:53.743 INFO:teuthology.orchestra.run.smithi037.stdout:max_xattr_size 65536 2024-03-29T08:45:53.744 INFO:teuthology.orchestra.run.smithi037.stdout:required_client_features {} 2024-03-29T08:45:53.744 INFO:teuthology.orchestra.run.smithi037.stdout:last_failure 0 2024-03-29T08:45:53.744 INFO:teuthology.orchestra.run.smithi037.stdout:last_failure_osd_epoch 82 2024-03-29T08:45:53.744 INFO:teuthology.orchestra.run.smithi037.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T08:45:53.744 INFO:teuthology.orchestra.run.smithi037.stdout:max_mds 2 2024-03-29T08:45:53.744 INFO:teuthology.orchestra.run.smithi037.stdout:in 0,1 2024-03-29T08:45:53.744 INFO:teuthology.orchestra.run.smithi037.stdout:up {} 2024-03-29T08:45:53.744 INFO:teuthology.orchestra.run.smithi037.stdout:failed 0,1 2024-03-29T08:45:53.744 INFO:teuthology.orchestra.run.smithi037.stdout:damaged 2024-03-29T08:45:53.744 INFO:teuthology.orchestra.run.smithi037.stdout:stopped 2024-03-29T08:45:53.744 INFO:teuthology.orchestra.run.smithi037.stdout:data_pools [3] 2024-03-29T08:45:53.744 INFO:teuthology.orchestra.run.smithi037.stdout:metadata_pool 2 2024-03-29T08:45:53.744 INFO:teuthology.orchestra.run.smithi037.stdout:inline_data disabled 2024-03-29T08:45:53.744 INFO:teuthology.orchestra.run.smithi037.stdout:balancer 2024-03-29T08:45:53.744 INFO:teuthology.orchestra.run.smithi037.stdout:bal_rank_mask -1 2024-03-29T08:45:53.745 INFO:teuthology.orchestra.run.smithi037.stdout:standby_count_wanted 1 2024-03-29T08:45:53.745 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:45:53.745 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:45:53.745 INFO:teuthology.orchestra.run.smithi037.stdout:Standby daemons: 2024-03-29T08:45:53.745 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:45:53.745 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi037.mmwgxe{-1:34440} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.37:6828/1248142005,v1:172.21.15.37:6829/1248142005] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:45:53.745 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi037.iifsmh{-1:34444} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.37:6826/2060882959,v1:172.21.15.37:6827/2060882959] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:45:53.745 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi148.sqteih{-1:44395} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.148:6824/4078652242,v1:172.21.15.148:6825/4078652242] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:45:53.745 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi148.qukpjh{-1:45007} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.148:6826/2394751105,v1:172.21.15.148:6827/2394751105] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:45:53.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:53.745+0000 7fdd8a000700 1 -- 172.21.15.37:0/1689672143 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fdd6806cc30 msgr2=0x7fdd6806f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:45:53.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:53.745+0000 7fdd8a000700 1 --2- 172.21.15.37:0/1689672143 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fdd6806cc30 0x7fdd6806f0e0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fdd74000c00 tx=0x7fdd74009e30 comp rx=0 tx=0).stop 2024-03-29T08:45:53.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:53.745+0000 7fdd8a000700 1 -- 172.21.15.37:0/1689672143 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdd841037a0 msgr2=0x7fdd84114060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:45:53.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:53.745+0000 7fdd8a000700 1 --2- 172.21.15.37:0/1689672143 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdd841037a0 0x7fdd84114060 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7fdd70002a50 tx=0x7fdd70004600 comp rx=0 tx=0).stop 2024-03-29T08:45:53.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:53.745+0000 7fdd8a000700 1 -- 172.21.15.37:0/1689672143 shutdown_connections 2024-03-29T08:45:53.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:53.745+0000 7fdd8a000700 1 --2- 172.21.15.37:0/1689672143 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fdd6806cc30 0x7fdd6806f0e0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:45:53.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:53.745+0000 7fdd8a000700 1 --2- 172.21.15.37:0/1689672143 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdd84102e00 0x7fdd84113b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:45:53.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:53.745+0000 7fdd8a000700 1 --2- 172.21.15.37:0/1689672143 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdd841037a0 0x7fdd84114060 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:45:53.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:53.745+0000 7fdd8a000700 1 -- 172.21.15.37:0/1689672143 >> 172.21.15.37:0/1689672143 conn(0x7fdd840f89b0 msgr2=0x7fdd840f9a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:45:53.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:53.745+0000 7fdd8a000700 1 -- 172.21.15.37:0/1689672143 shutdown_connections 2024-03-29T08:45:53.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:53.745+0000 7fdd8a000700 1 -- 172.21.15.37:0/1689672143 wait complete. 2024-03-29T08:45:53.747 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 30 2024-03-29T08:45:53.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:53.882+0000 7f33c7a00700 1 -- 172.21.15.37:0/2657670922 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f33c0102de0 msgr2=0x7f33c01031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:45:53.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:53.882+0000 7f33c7a00700 1 --2- 172.21.15.37:0/2657670922 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f33c0102de0 0x7f33c01031b0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f33b4004940 tx=0x7f33b4009e30 comp rx=0 tx=0).stop 2024-03-29T08:45:53.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:53.883+0000 7f33c7a00700 1 -- 172.21.15.37:0/2657670922 shutdown_connections 2024-03-29T08:45:53.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:53.883+0000 7f33c7a00700 1 --2- 172.21.15.37:0/2657670922 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f33c00656a0 0x7f33c0065af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:45:53.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:53.883+0000 7f33c7a00700 1 --2- 172.21.15.37:0/2657670922 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f33c0102de0 0x7f33c01031b0 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:45:53.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:53.883+0000 7f33c7a00700 1 -- 172.21.15.37:0/2657670922 >> 172.21.15.37:0/2657670922 conn(0x7f33c0072bc0 msgr2=0x7f33c0072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:45:53.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:53.883+0000 7f33c7a00700 1 -- 172.21.15.37:0/2657670922 shutdown_connections 2024-03-29T08:45:53.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:53.883+0000 7f33c7a00700 1 -- 172.21.15.37:0/2657670922 wait complete. 2024-03-29T08:45:53.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:53.883+0000 7f33c7a00700 1 Processor -- start 2024-03-29T08:45:53.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:53.884+0000 7f33c7a00700 1 -- start start 2024-03-29T08:45:53.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:53.884+0000 7f33c7a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f33c00656a0 0x7f33c0115bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:45:53.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:53.884+0000 7f33c7a00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f33c0102de0 0x7f33c0116130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:45:53.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:53.884+0000 7f33c7a00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33c0116780 con 0x7f33c00656a0 2024-03-29T08:45:53.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:53.884+0000 7f33c7a00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33c010fb90 con 0x7f33c0102de0 2024-03-29T08:45:53.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:53.884+0000 7f33c5000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f33c00656a0 0x7f33c0115bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:45:53.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:53.884+0000 7f33c5000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f33c00656a0 0x7f33c0115bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36556/0 (socket says 172.21.15.37:36556) 2024-03-29T08:45:53.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:53.884+0000 7f33c5000700 1 -- 172.21.15.37:0/1562737844 learned_addr learned my addr 172.21.15.37:0/1562737844 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:45:53.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:53.884+0000 7f33bfe00700 1 --2- 172.21.15.37:0/1562737844 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f33c0102de0 0x7f33c0116130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:45:53.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:53.885+0000 7f33c5000700 1 -- 172.21.15.37:0/1562737844 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f33c0102de0 msgr2=0x7f33c0116130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:45:53.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:53.885+0000 7f33c5000700 1 --2- 172.21.15.37:0/1562737844 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f33c0102de0 0x7f33c0116130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:45:53.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:53.885+0000 7f33c5000700 1 -- 172.21.15.37:0/1562737844 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33b4005040 con 0x7f33c00656a0 2024-03-29T08:45:53.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:53.885+0000 7f33c5000700 1 --2- 172.21.15.37:0/1562737844 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f33c00656a0 0x7f33c0115bf0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f33b4009390 tx=0x7f33b4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:45:53.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:53.885+0000 7f33bcc00700 1 -- 172.21.15.37:0/1562737844 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f33b400e040 con 0x7f33c00656a0 2024-03-29T08:45:53.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:53.885+0000 7f33bcc00700 1 -- 172.21.15.37:0/1562737844 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f33b4013420 con 0x7f33c00656a0 2024-03-29T08:45:53.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:53.885+0000 7f33c7a00700 1 -- 172.21.15.37:0/1562737844 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33c010fe10 con 0x7f33c00656a0 2024-03-29T08:45:53.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:53.885+0000 7f33bcc00700 1 -- 172.21.15.37:0/1562737844 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f33b401c630 con 0x7f33c00656a0 2024-03-29T08:45:53.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:53.885+0000 7f33c7a00700 1 -- 172.21.15.37:0/1562737844 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33c01103d0 con 0x7f33c00656a0 2024-03-29T08:45:53.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:53.886+0000 7f33c7a00700 1 -- 172.21.15.37:0/1562737844 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33c0106980 con 0x7f33c00656a0 2024-03-29T08:45:53.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:53.887+0000 7f33bcc00700 1 -- 172.21.15.37:0/1562737844 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f33b40139e0 con 0x7f33c00656a0 2024-03-29T08:45:53.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:53.888+0000 7f33bcc00700 1 --2- 172.21.15.37:0/1562737844 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f33a806cc30 0x7f33a806f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:45:53.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:53.888+0000 7f33bcc00700 1 -- 172.21.15.37:0/1562737844 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f33b408d240 con 0x7f33c00656a0 2024-03-29T08:45:53.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:53.889+0000 7f33bfe00700 1 --2- 172.21.15.37:0/1562737844 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f33a806cc30 0x7f33a806f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:45:53.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:53.889+0000 7f33bfe00700 1 --2- 172.21.15.37:0/1562737844 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f33a806cc30 0x7f33a806f0e0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f33ac007fd0 tx=0x7f33ac009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:45:53.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:53.893+0000 7f33bcc00700 1 -- 172.21.15.37:0/1562737844 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f33b4057330 con 0x7f33c00656a0 2024-03-29T08:45:53.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:45:53 smithi037 ceph-mon[142325]: from='client.35878 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:45:53.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:45:53 smithi037 ceph-mon[142325]: from='client.35882 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:45:53.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:45:53 smithi037 ceph-mon[142325]: from='client.35886 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:45:53.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:45:53 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/652618989' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:45:54.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:45:53 smithi148 ceph-mon[106692]: from='client.35878 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:45:54.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:45:53 smithi148 ceph-mon[106692]: from='client.35882 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:45:54.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:45:53 smithi148 ceph-mon[106692]: from='client.35886 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:45:54.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:45:53 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/652618989' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:45:54.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:54.126+0000 7f33c7a00700 1 -- 172.21.15.37:0/1562737844 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f33c0109cf0 con 0x7f33a806cc30 2024-03-29T08:45:54.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:54.127+0000 7f33bcc00700 1 -- 172.21.15.37:0/1562737844 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f33c0109cf0 con 0x7f33a806cc30 2024-03-29T08:45:54.127 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-29T08:45:54.127 INFO:teuthology.orchestra.run.smithi037.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59", 2024-03-29T08:45:54.127 INFO:teuthology.orchestra.run.smithi037.stdout: "in_progress": true, 2024-03-29T08:45:54.127 INFO:teuthology.orchestra.run.smithi037.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-29T08:45:54.127 INFO:teuthology.orchestra.run.smithi037.stdout: "services_complete": [ 2024-03-29T08:45:54.127 INFO:teuthology.orchestra.run.smithi037.stdout: "crash", 2024-03-29T08:45:54.127 INFO:teuthology.orchestra.run.smithi037.stdout: "osd", 2024-03-29T08:45:54.127 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr", 2024-03-29T08:45:54.127 INFO:teuthology.orchestra.run.smithi037.stdout: "mds", 2024-03-29T08:45:54.127 INFO:teuthology.orchestra.run.smithi037.stdout: "mon" 2024-03-29T08:45:54.127 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-29T08:45:54.127 INFO:teuthology.orchestra.run.smithi037.stdout: "progress": "16/21 daemons upgraded", 2024-03-29T08:45:54.127 INFO:teuthology.orchestra.run.smithi037.stdout: "message": "Currently upgrading mds daemons", 2024-03-29T08:45:54.127 INFO:teuthology.orchestra.run.smithi037.stdout: "is_paused": false 2024-03-29T08:45:54.127 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-29T08:45:54.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:54.130+0000 7f33c7a00700 1 -- 172.21.15.37:0/1562737844 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f33a806cc30 msgr2=0x7f33a806f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:45:54.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:54.130+0000 7f33c7a00700 1 --2- 172.21.15.37:0/1562737844 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f33a806cc30 0x7f33a806f0e0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f33ac007fd0 tx=0x7f33ac009040 comp rx=0 tx=0).stop 2024-03-29T08:45:54.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:54.130+0000 7f33c7a00700 1 -- 172.21.15.37:0/1562737844 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f33c00656a0 msgr2=0x7f33c0115bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:45:54.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:54.130+0000 7f33c7a00700 1 --2- 172.21.15.37:0/1562737844 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f33c00656a0 0x7f33c0115bf0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f33b4009390 tx=0x7f33b4009e30 comp rx=0 tx=0).stop 2024-03-29T08:45:54.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:54.130+0000 7f33c7a00700 1 -- 172.21.15.37:0/1562737844 shutdown_connections 2024-03-29T08:45:54.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:54.130+0000 7f33c7a00700 1 --2- 172.21.15.37:0/1562737844 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f33a806cc30 0x7f33a806f0e0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:45:54.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:54.130+0000 7f33c7a00700 1 --2- 172.21.15.37:0/1562737844 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f33c00656a0 0x7f33c0115bf0 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:45:54.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:54.130+0000 7f33c7a00700 1 --2- 172.21.15.37:0/1562737844 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f33c0102de0 0x7f33c0116130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:45:54.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:54.130+0000 7f33c7a00700 1 -- 172.21.15.37:0/1562737844 >> 172.21.15.37:0/1562737844 conn(0x7f33c0072bc0 msgr2=0x7f33c0103b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:45:54.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:54.130+0000 7f33c7a00700 1 -- 172.21.15.37:0/1562737844 shutdown_connections 2024-03-29T08:45:54.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:54.130+0000 7f33c7a00700 1 -- 172.21.15.37:0/1562737844 wait complete. 2024-03-29T08:45:54.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:54.267+0000 7fd5bc200700 1 -- 172.21.15.37:0/2049612479 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd5b40fde90 msgr2=0x7fd5b40fe260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:45:54.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:54.267+0000 7fd5bc200700 1 --2- 172.21.15.37:0/2049612479 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd5b40fde90 0x7fd5b40fe260 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7fd5a8004940 tx=0x7fd5a8009e30 comp rx=0 tx=0).stop 2024-03-29T08:45:54.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:54.267+0000 7fd5bc200700 1 -- 172.21.15.37:0/2049612479 shutdown_connections 2024-03-29T08:45:54.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:54.267+0000 7fd5bc200700 1 --2- 172.21.15.37:0/2049612479 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd5b40fe830 0x7fd5b4106f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:45:54.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:54.267+0000 7fd5bc200700 1 --2- 172.21.15.37:0/2049612479 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd5b40fde90 0x7fd5b40fe260 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:45:54.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:54.267+0000 7fd5bc200700 1 -- 172.21.15.37:0/2049612479 >> 172.21.15.37:0/2049612479 conn(0x7fd5b4072bc0 msgr2=0x7fd5b4072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:45:54.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:54.267+0000 7fd5bc200700 1 -- 172.21.15.37:0/2049612479 shutdown_connections 2024-03-29T08:45:54.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:54.267+0000 7fd5bc200700 1 -- 172.21.15.37:0/2049612479 wait complete. 2024-03-29T08:45:54.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:54.268+0000 7fd5bc200700 1 Processor -- start 2024-03-29T08:45:54.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:54.268+0000 7fd5bc200700 1 -- start start 2024-03-29T08:45:54.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:54.268+0000 7fd5bc200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd5b40fde90 0x7fd5b4115d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:45:54.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:54.268+0000 7fd5bc200700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd5b40fe830 0x7fd5b4116240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:45:54.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:54.268+0000 7fd5bc200700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5b4116890 con 0x7fd5b40fde90 2024-03-29T08:45:54.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:54.269+0000 7fd5bc200700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5b410fc60 con 0x7fd5b40fe830 2024-03-29T08:45:54.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:54.269+0000 7fd5b9800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd5b40fde90 0x7fd5b4115d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:45:54.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:54.269+0000 7fd5b9800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd5b40fde90 0x7fd5b4115d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36574/0 (socket says 172.21.15.37:36574) 2024-03-29T08:45:54.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:54.269+0000 7fd5b9800700 1 -- 172.21.15.37:0/506110232 learned_addr learned my addr 172.21.15.37:0/506110232 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:45:54.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:54.269+0000 7fd5b8e00700 1 --2- 172.21.15.37:0/506110232 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd5b40fe830 0x7fd5b4116240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:45:54.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:54.269+0000 7fd5b9800700 1 -- 172.21.15.37:0/506110232 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd5b40fe830 msgr2=0x7fd5b4116240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:45:54.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:54.269+0000 7fd5b9800700 1 --2- 172.21.15.37:0/506110232 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd5b40fe830 0x7fd5b4116240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:45:54.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:54.269+0000 7fd5b9800700 1 -- 172.21.15.37:0/506110232 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5a8005040 con 0x7fd5b40fde90 2024-03-29T08:45:54.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:54.269+0000 7fd5b8e00700 1 --2- 172.21.15.37:0/506110232 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd5b40fe830 0x7fd5b4116240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:45:54.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:54.269+0000 7fd5b9800700 1 --2- 172.21.15.37:0/506110232 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd5b40fde90 0x7fd5b4115d00 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7fd5a80093b0 tx=0x7fd5a8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:45:54.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:54.270+0000 7fd5b1600700 1 -- 172.21.15.37:0/506110232 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd5a800e040 con 0x7fd5b40fde90 2024-03-29T08:45:54.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:54.270+0000 7fd5b1600700 1 -- 172.21.15.37:0/506110232 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd5a8004290 con 0x7fd5b40fde90 2024-03-29T08:45:54.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:54.270+0000 7fd5bc200700 1 -- 172.21.15.37:0/506110232 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5b410fee0 con 0x7fd5b40fde90 2024-03-29T08:45:54.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:54.270+0000 7fd5b1600700 1 -- 172.21.15.37:0/506110232 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd5a801c3f0 con 0x7fd5b40fde90 2024-03-29T08:45:54.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:54.270+0000 7fd5bc200700 1 -- 172.21.15.37:0/506110232 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5b4110420 con 0x7fd5b40fde90 2024-03-29T08:45:54.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:54.271+0000 7fd5bc200700 1 -- 172.21.15.37:0/506110232 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5b4106a20 con 0x7fd5b40fde90 2024-03-29T08:45:54.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:54.272+0000 7fd5b1600700 1 -- 172.21.15.37:0/506110232 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fd5a8013790 con 0x7fd5b40fde90 2024-03-29T08:45:54.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:54.273+0000 7fd5b1600700 1 --2- 172.21.15.37:0/506110232 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fd59c06cc30 0x7fd59c06f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:45:54.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:54.273+0000 7fd5b1600700 1 -- 172.21.15.37:0/506110232 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fd5a808d380 con 0x7fd5b40fde90 2024-03-29T08:45:54.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:54.273+0000 7fd5b8e00700 1 --2- 172.21.15.37:0/506110232 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fd59c06cc30 0x7fd59c06f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:45:54.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:54.274+0000 7fd5b8e00700 1 --2- 172.21.15.37:0/506110232 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fd59c06cc30 0x7fd59c06f0e0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fd5a0007fd0 tx=0x7fd5a0009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:45:54.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:54.278+0000 7fd5b1600700 1 -- 172.21.15.37:0/506110232 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fd5a805add0 con 0x7fd5b40fde90 2024-03-29T08:45:54.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:54.576+0000 7fd5bc200700 1 -- 172.21.15.37:0/506110232 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fd5b404bed0 con 0x7fd5b40fde90 2024-03-29T08:45:54.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:54.576+0000 7fd5b1600700 1 -- 172.21.15.37:0/506110232 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+3583 (secure 0 0 0) 0x7fd5a8013420 con 0x7fd5b40fde90 2024-03-29T08:45:54.576 INFO:teuthology.orchestra.run.smithi037.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem is offline 2024-03-29T08:45:54.576 INFO:teuthology.orchestra.run.smithi037.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-03-29T08:45:54.576 INFO:teuthology.orchestra.run.smithi037.stdout: host smithi148 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/mon.smithi148/config 2024-03-29T08:45:54.577 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/podman: timeout after 895 seconds 2024-03-29T08:45:54.577 INFO:teuthology.orchestra.run.smithi037.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59 -e NODE_NAME=smithi148 -e CEPH_USE_RANDOM_NONCE=1 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4:/var/run/ceph:z -v /var/log/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4:/var/log/ceph:z -v /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpkx2rz2d4:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59 inventory --format=json-pretty --filter-for-batch 2024-03-29T08:45:54.577 INFO:teuthology.orchestra.run.smithi037.stdout:Traceback (most recent call last): 2024-03-29T08:45:54.577 INFO:teuthology.orchestra.run.smithi037.stdout: File "/usr/lib64/python3.6/runpy.py", line 193, in _run_module_as_main 2024-03-29T08:45:54.577 INFO:teuthology.orchestra.run.smithi037.stdout: "__main__", mod_spec) 2024-03-29T08:45:54.577 INFO:teuthology.orchestra.run.smithi037.stdout: File "/usr/lib64/python3.6/runpy.py", line 85, in _run_code 2024-03-29T08:45:54.577 INFO:teuthology.orchestra.run.smithi037.stdout: exec(code, run_globals) 2024-03-29T08:45:54.577 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 10889, in 2024-03-29T08:45:54.577 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 10877, in main 2024-03-29T08:45:54.577 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 2576, in _infer_config 2024-03-29T08:45:54.577 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 2492, in _infer_fsid 2024-03-29T08:45:54.577 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 2604, in _infer_image 2024-03-29T08:45:54.577 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 2479, in _validate_fsid 2024-03-29T08:45:54.577 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 7145, in command_ceph_volume 2024-03-29T08:45:54.577 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 2267, in call_throws 2024-03-29T08:45:54.578 INFO:teuthology.orchestra.run.smithi037.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59 -e NODE_NAME=smithi148 -e CEPH_USE_RANDOM_NONCE=1 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4:/var/run/ceph:z -v /var/log/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4:/var/log/ceph:z -v /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpkx2rz2d4:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59 inventory --format=json-pretty --filter-for-batch 2024-03-29T08:45:54.578 INFO:teuthology.orchestra.run.smithi037.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-03-29T08:45:54.578 INFO:teuthology.orchestra.run.smithi037.stdout: fs cephfs is degraded 2024-03-29T08:45:54.578 INFO:teuthology.orchestra.run.smithi037.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-03-29T08:45:54.578 INFO:teuthology.orchestra.run.smithi037.stdout: fs cephfs is offline because no MDS is active for it. 2024-03-29T08:45:54.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:54.580+0000 7fd5bc200700 1 -- 172.21.15.37:0/506110232 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fd59c06cc30 msgr2=0x7fd59c06f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:45:54.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:54.580+0000 7fd5bc200700 1 --2- 172.21.15.37:0/506110232 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fd59c06cc30 0x7fd59c06f0e0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fd5a0007fd0 tx=0x7fd5a0009040 comp rx=0 tx=0).stop 2024-03-29T08:45:54.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:54.580+0000 7fd5bc200700 1 -- 172.21.15.37:0/506110232 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd5b40fde90 msgr2=0x7fd5b4115d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:45:54.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:54.580+0000 7fd5bc200700 1 --2- 172.21.15.37:0/506110232 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd5b40fde90 0x7fd5b4115d00 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7fd5a80093b0 tx=0x7fd5a8009e30 comp rx=0 tx=0).stop 2024-03-29T08:45:54.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:54.580+0000 7fd5bc200700 1 -- 172.21.15.37:0/506110232 shutdown_connections 2024-03-29T08:45:54.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:54.580+0000 7fd5bc200700 1 --2- 172.21.15.37:0/506110232 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fd59c06cc30 0x7fd59c06f0e0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:45:54.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:54.580+0000 7fd5bc200700 1 --2- 172.21.15.37:0/506110232 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd5b40fde90 0x7fd5b4115d00 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:45:54.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:54.580+0000 7fd5bc200700 1 --2- 172.21.15.37:0/506110232 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd5b40fe830 0x7fd5b4116240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:45:54.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:54.580+0000 7fd5bc200700 1 -- 172.21.15.37:0/506110232 >> 172.21.15.37:0/506110232 conn(0x7fd5b4072bc0 msgr2=0x7fd5b40fd590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:45:54.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:54.580+0000 7fd5bc200700 1 -- 172.21.15.37:0/506110232 shutdown_connections 2024-03-29T08:45:54.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:45:54.580+0000 7fd5bc200700 1 -- 172.21.15.37:0/506110232 wait complete. 2024-03-29T08:45:54.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:45:54 smithi037 ceph-mon[142325]: pgmap v1051: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:45:54.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:45:54 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/1689672143' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T08:45:54.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:45:54 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/506110232' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T08:45:55.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:45:54 smithi148 ceph-mon[106692]: pgmap v1051: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:45:55.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:45:54 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/1689672143' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T08:45:55.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:45:54 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/506110232' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T08:45:55.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:45:55 smithi037 ceph-mon[142325]: from='client.35898 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:45:55.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:45:55 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:45:56.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:45:55 smithi148 ceph-mon[106692]: from='client.35898 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:45:56.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:45:55 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:45:56.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:45:56 smithi037 ceph-mon[142325]: pgmap v1052: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:45:57.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:45:56 smithi148 ceph-mon[106692]: pgmap v1052: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:45:58.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:45:58 smithi037 ceph-mon[142325]: pgmap v1053: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:45:59.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:45:58 smithi148 ceph-mon[106692]: pgmap v1053: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:46:00.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:46:00 smithi037 ceph-mon[142325]: pgmap v1054: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:46:01.055 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:46:00 smithi148 ceph-mon[106692]: pgmap v1054: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:46:02.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:46:02 smithi037 ceph-mon[142325]: pgmap v1055: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:46:03.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:46:02 smithi148 ceph-mon[106692]: pgmap v1055: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:46:04.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:46:04 smithi037 ceph-mon[142325]: pgmap v1056: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:46:05.055 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:46:04 smithi148 ceph-mon[106692]: pgmap v1056: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:46:06.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:46:06 smithi037 ceph-mon[142325]: pgmap v1057: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:46:07.055 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:46:06 smithi148 ceph-mon[106692]: pgmap v1057: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:46:08.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:46:08 smithi037 ceph-mon[142325]: pgmap v1058: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:46:09.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:46:08 smithi148 ceph-mon[106692]: pgmap v1058: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:46:10.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:46:10 smithi037 ceph-mon[142325]: pgmap v1059: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:46:10.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:46:10 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:46:11.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:46:10 smithi148 ceph-mon[106692]: pgmap v1059: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:46:11.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:46:10 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:46:12.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:46:12 smithi037 ceph-mon[142325]: pgmap v1060: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:46:13.055 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:46:12 smithi148 ceph-mon[106692]: pgmap v1060: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:46:14.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:46:14 smithi037 ceph-mon[142325]: pgmap v1061: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:46:15.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:46:14 smithi148 ceph-mon[106692]: pgmap v1061: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:46:16.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:46:16 smithi037 ceph-mon[142325]: pgmap v1062: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:46:17.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:46:16 smithi148 ceph-mon[106692]: pgmap v1062: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:46:18.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:46:18 smithi037 ceph-mon[142325]: pgmap v1063: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:46:19.055 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:46:18 smithi148 ceph-mon[106692]: pgmap v1063: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:46:20.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:46:20 smithi037 ceph-mon[142325]: pgmap v1064: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:46:21.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:46:20 smithi148 ceph-mon[106692]: pgmap v1064: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:46:22.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:46:22 smithi037 ceph-mon[142325]: pgmap v1065: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:46:23.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:46:22 smithi148 ceph-mon[106692]: pgmap v1065: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:46:24.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:24.735+0000 7f5d87600700 1 -- 172.21.15.37:0/452334307 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5d800fde90 msgr2=0x7f5d800fe260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:46:24.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:24.735+0000 7f5d87600700 1 --2- 172.21.15.37:0/452334307 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5d800fde90 0x7f5d800fe260 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f5d74004940 tx=0x7f5d74009e30 comp rx=0 tx=0).stop 2024-03-29T08:46:24.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:24.735+0000 7f5d87600700 1 -- 172.21.15.37:0/452334307 shutdown_connections 2024-03-29T08:46:24.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:24.735+0000 7f5d87600700 1 --2- 172.21.15.37:0/452334307 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5d800fe830 0x7f5d80106f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:46:24.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:24.735+0000 7f5d87600700 1 --2- 172.21.15.37:0/452334307 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5d800fde90 0x7f5d800fe260 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:46:24.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:24.735+0000 7f5d87600700 1 -- 172.21.15.37:0/452334307 >> 172.21.15.37:0/452334307 conn(0x7f5d80072bc0 msgr2=0x7f5d80072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:46:24.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:24.735+0000 7f5d87600700 1 -- 172.21.15.37:0/452334307 shutdown_connections 2024-03-29T08:46:24.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:24.735+0000 7f5d87600700 1 -- 172.21.15.37:0/452334307 wait complete. 2024-03-29T08:46:24.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:24.736+0000 7f5d87600700 1 Processor -- start 2024-03-29T08:46:24.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:24.736+0000 7f5d87600700 1 -- start start 2024-03-29T08:46:24.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:24.737+0000 7f5d87600700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5d800fde90 0x7f5d80115c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:46:24.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:24.737+0000 7f5d87600700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5d800fe830 0x7f5d801161c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:46:24.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:24.737+0000 7f5d87600700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d80116810 con 0x7f5d800fde90 2024-03-29T08:46:24.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:24.737+0000 7f5d87600700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d8010fbe0 con 0x7f5d800fe830 2024-03-29T08:46:24.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:24.737+0000 7f5d84c00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5d800fde90 0x7f5d80115c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:46:24.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:24.737+0000 7f5d84c00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5d800fde90 0x7f5d80115c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37882/0 (socket says 172.21.15.37:37882) 2024-03-29T08:46:24.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:24.737+0000 7f5d84c00700 1 -- 172.21.15.37:0/3922198356 learned_addr learned my addr 172.21.15.37:0/3922198356 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:46:24.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:24.737+0000 7f5d7fe00700 1 --2- 172.21.15.37:0/3922198356 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5d800fe830 0x7f5d801161c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:46:24.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:24.737+0000 7f5d84c00700 1 -- 172.21.15.37:0/3922198356 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5d800fe830 msgr2=0x7f5d801161c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:46:24.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:24.737+0000 7f5d84c00700 1 --2- 172.21.15.37:0/3922198356 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5d800fe830 0x7f5d801161c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:46:24.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:24.738+0000 7f5d84c00700 1 -- 172.21.15.37:0/3922198356 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d74005040 con 0x7f5d800fde90 2024-03-29T08:46:24.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:24.738+0000 7f5d84c00700 1 --2- 172.21.15.37:0/3922198356 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5d800fde90 0x7f5d80115c80 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f5d74000c00 tx=0x7f5d74009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:46:24.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:24.738+0000 7f5d7cc00700 1 -- 172.21.15.37:0/3922198356 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5d74021070 con 0x7f5d800fde90 2024-03-29T08:46:24.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:24.738+0000 7f5d7cc00700 1 -- 172.21.15.37:0/3922198356 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f5d74013e00 con 0x7f5d800fde90 2024-03-29T08:46:24.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:24.738+0000 7f5d7cc00700 1 -- 172.21.15.37:0/3922198356 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5d7400e070 con 0x7f5d800fde90 2024-03-29T08:46:24.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:24.738+0000 7f5d87600700 1 -- 172.21.15.37:0/3922198356 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d8010fe60 con 0x7f5d800fde90 2024-03-29T08:46:24.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:24.738+0000 7f5d87600700 1 -- 172.21.15.37:0/3922198356 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d801103a0 con 0x7f5d800fde90 2024-03-29T08:46:24.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:24.739+0000 7f5d87600700 1 -- 172.21.15.37:0/3922198356 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d80106a20 con 0x7f5d800fde90 2024-03-29T08:46:24.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:24.740+0000 7f5d7cc00700 1 -- 172.21.15.37:0/3922198356 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f5d74013950 con 0x7f5d800fde90 2024-03-29T08:46:24.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:24.741+0000 7f5d7cc00700 1 --2- 172.21.15.37:0/3922198356 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f5d6806cc30 0x7f5d6806f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:46:24.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:24.741+0000 7f5d7cc00700 1 -- 172.21.15.37:0/3922198356 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f5d7408cff0 con 0x7f5d800fde90 2024-03-29T08:46:24.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:24.742+0000 7f5d7fe00700 1 --2- 172.21.15.37:0/3922198356 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f5d6806cc30 0x7f5d6806f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:46:24.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:24.742+0000 7f5d7fe00700 1 --2- 172.21.15.37:0/3922198356 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f5d6806cc30 0x7f5d6806f0e0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f5d80111520 tx=0x7f5d6c00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:46:24.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:24.746+0000 7f5d7cc00700 1 -- 172.21.15.37:0/3922198356 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f5d740c09f0 con 0x7f5d800fde90 2024-03-29T08:46:24.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:46:24 smithi037 ceph-mon[142325]: pgmap v1066: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:46:24.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:24.978+0000 7f5d87600700 1 -- 172.21.15.37:0/3922198356 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5d80109dc0 con 0x7f5d6806cc30 2024-03-29T08:46:24.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:24.979+0000 7f5d7cc00700 1 -- 172.21.15.37:0/3922198356 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f5d80109dc0 con 0x7f5d6806cc30 2024-03-29T08:46:24.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:24.982+0000 7f5d87600700 1 -- 172.21.15.37:0/3922198356 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f5d6806cc30 msgr2=0x7f5d6806f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:46:24.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:24.982+0000 7f5d87600700 1 --2- 172.21.15.37:0/3922198356 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f5d6806cc30 0x7f5d6806f0e0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f5d80111520 tx=0x7f5d6c00b040 comp rx=0 tx=0).stop 2024-03-29T08:46:24.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:24.982+0000 7f5d87600700 1 -- 172.21.15.37:0/3922198356 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5d800fde90 msgr2=0x7f5d80115c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:46:24.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:24.982+0000 7f5d87600700 1 --2- 172.21.15.37:0/3922198356 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5d800fde90 0x7f5d80115c80 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f5d74000c00 tx=0x7f5d74009e30 comp rx=0 tx=0).stop 2024-03-29T08:46:24.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:24.982+0000 7f5d87600700 1 -- 172.21.15.37:0/3922198356 shutdown_connections 2024-03-29T08:46:24.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:24.982+0000 7f5d87600700 1 --2- 172.21.15.37:0/3922198356 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f5d6806cc30 0x7f5d6806f0e0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:46:24.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:24.982+0000 7f5d87600700 1 --2- 172.21.15.37:0/3922198356 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5d800fde90 0x7f5d80115c80 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:46:24.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:24.982+0000 7f5d87600700 1 --2- 172.21.15.37:0/3922198356 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5d800fe830 0x7f5d801161c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:46:24.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:24.982+0000 7f5d87600700 1 -- 172.21.15.37:0/3922198356 >> 172.21.15.37:0/3922198356 conn(0x7f5d80072bc0 msgr2=0x7f5d800fb7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:46:24.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:24.982+0000 7f5d87600700 1 -- 172.21.15.37:0/3922198356 shutdown_connections 2024-03-29T08:46:24.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:24.982+0000 7f5d87600700 1 -- 172.21.15.37:0/3922198356 wait complete. 2024-03-29T08:46:24.997 INFO:teuthology.orchestra.run.smithi037.stdout:true 2024-03-29T08:46:25.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:46:24 smithi148 ceph-mon[106692]: pgmap v1066: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:46:25.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:25.120+0000 7fee9ec00700 1 -- 172.21.15.37:0/2204278446 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fee981035e0 msgr2=0x7fee980fcc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:46:25.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:25.120+0000 7fee9ec00700 1 --2- 172.21.15.37:0/2204278446 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fee981035e0 0x7fee980fcc50 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7fee84004940 tx=0x7fee84009e30 comp rx=0 tx=0).stop 2024-03-29T08:46:25.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:25.120+0000 7fee9ec00700 1 -- 172.21.15.37:0/2204278446 shutdown_connections 2024-03-29T08:46:25.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:25.120+0000 7fee9ec00700 1 --2- 172.21.15.37:0/2204278446 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fee981035e0 0x7fee980fcc50 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:46:25.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:25.120+0000 7fee9ec00700 1 --2- 172.21.15.37:0/2204278446 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fee98102c40 0x7fee98103010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:46:25.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:25.120+0000 7fee9ec00700 1 -- 172.21.15.37:0/2204278446 >> 172.21.15.37:0/2204278446 conn(0x7fee980f8830 msgr2=0x7fee980fac40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:46:25.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:25.120+0000 7fee9ec00700 1 -- 172.21.15.37:0/2204278446 shutdown_connections 2024-03-29T08:46:25.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:25.120+0000 7fee9ec00700 1 -- 172.21.15.37:0/2204278446 wait complete. 2024-03-29T08:46:25.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:25.121+0000 7fee9ec00700 1 Processor -- start 2024-03-29T08:46:25.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:25.121+0000 7fee9ec00700 1 -- start start 2024-03-29T08:46:25.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:25.122+0000 7fee9ec00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fee98102c40 0x7fee98115af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:46:25.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:25.122+0000 7fee9ec00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fee981035e0 0x7fee98116030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:46:25.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:25.122+0000 7fee9ec00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee98116680 con 0x7fee98102c40 2024-03-29T08:46:25.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:25.122+0000 7fee9ec00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee9810fa70 con 0x7fee981035e0 2024-03-29T08:46:25.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:25.122+0000 7fee9d800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fee98102c40 0x7fee98115af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:46:25.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:25.122+0000 7fee9d800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fee98102c40 0x7fee98115af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37908/0 (socket says 172.21.15.37:37908) 2024-03-29T08:46:25.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:25.122+0000 7fee9d800700 1 -- 172.21.15.37:0/4002405334 learned_addr learned my addr 172.21.15.37:0/4002405334 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:46:25.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:25.122+0000 7fee9ce00700 1 --2- 172.21.15.37:0/4002405334 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fee981035e0 0x7fee98116030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:46:25.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:25.122+0000 7fee9d800700 1 -- 172.21.15.37:0/4002405334 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fee981035e0 msgr2=0x7fee98116030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:46:25.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:25.122+0000 7fee9d800700 1 --2- 172.21.15.37:0/4002405334 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fee981035e0 0x7fee98116030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:46:25.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:25.122+0000 7fee9d800700 1 -- 172.21.15.37:0/4002405334 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee84005040 con 0x7fee98102c40 2024-03-29T08:46:25.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:25.123+0000 7fee9d800700 1 --2- 172.21.15.37:0/4002405334 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fee98102c40 0x7fee98115af0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7fee8c002a50 tx=0x7fee8c0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:46:25.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:25.123+0000 7fee95600700 1 -- 172.21.15.37:0/4002405334 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fee8c007cc0 con 0x7fee98102c40 2024-03-29T08:46:25.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:25.123+0000 7fee95600700 1 -- 172.21.15.37:0/4002405334 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fee8c00f040 con 0x7fee98102c40 2024-03-29T08:46:25.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:25.123+0000 7fee95600700 1 -- 172.21.15.37:0/4002405334 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fee8c009540 con 0x7fee98102c40 2024-03-29T08:46:25.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:25.123+0000 7fee9ec00700 1 -- 172.21.15.37:0/4002405334 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee9810fd50 con 0x7fee98102c40 2024-03-29T08:46:25.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:25.123+0000 7fee9ec00700 1 -- 172.21.15.37:0/4002405334 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee98110310 con 0x7fee98102c40 2024-03-29T08:46:25.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:25.124+0000 7fee9ec00700 1 -- 172.21.15.37:0/4002405334 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee980fe390 con 0x7fee98102c40 2024-03-29T08:46:25.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:25.125+0000 7fee95600700 1 -- 172.21.15.37:0/4002405334 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fee8c0096a0 con 0x7fee98102c40 2024-03-29T08:46:25.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:25.127+0000 7fee95600700 1 --2- 172.21.15.37:0/4002405334 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fee8006cc30 0x7fee8006f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:46:25.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:25.127+0000 7fee95600700 1 -- 172.21.15.37:0/4002405334 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fee8c08bde0 con 0x7fee98102c40 2024-03-29T08:46:25.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:25.127+0000 7fee9ce00700 1 --2- 172.21.15.37:0/4002405334 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fee8006cc30 0x7fee8006f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:46:25.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:25.127+0000 7fee9ce00700 1 --2- 172.21.15.37:0/4002405334 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fee8006cc30 0x7fee8006f0e0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fee98110c30 tx=0x7fee84009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:46:25.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:25.131+0000 7fee95600700 1 -- 172.21.15.37:0/4002405334 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fee8c059830 con 0x7fee98102c40 2024-03-29T08:46:25.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:25.367+0000 7fee9ec00700 1 -- 172.21.15.37:0/4002405334 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fee98109bd0 con 0x7fee8006cc30 2024-03-29T08:46:25.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:25.369+0000 7fee95600700 1 -- 172.21.15.37:0/4002405334 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fee98109bd0 con 0x7fee8006cc30 2024-03-29T08:46:25.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:25.371+0000 7fee9ec00700 1 -- 172.21.15.37:0/4002405334 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fee8006cc30 msgr2=0x7fee8006f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:46:25.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:25.371+0000 7fee9ec00700 1 --2- 172.21.15.37:0/4002405334 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fee8006cc30 0x7fee8006f0e0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fee98110c30 tx=0x7fee84009e30 comp rx=0 tx=0).stop 2024-03-29T08:46:25.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:25.371+0000 7fee9ec00700 1 -- 172.21.15.37:0/4002405334 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fee98102c40 msgr2=0x7fee98115af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:46:25.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:25.371+0000 7fee9ec00700 1 --2- 172.21.15.37:0/4002405334 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fee98102c40 0x7fee98115af0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7fee8c002a50 tx=0x7fee8c0046b0 comp rx=0 tx=0).stop 2024-03-29T08:46:25.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:25.371+0000 7fee9ec00700 1 -- 172.21.15.37:0/4002405334 shutdown_connections 2024-03-29T08:46:25.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:25.371+0000 7fee9ec00700 1 --2- 172.21.15.37:0/4002405334 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fee8006cc30 0x7fee8006f0e0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:46:25.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:25.371+0000 7fee9ec00700 1 --2- 172.21.15.37:0/4002405334 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fee98102c40 0x7fee98115af0 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:46:25.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:25.371+0000 7fee9ec00700 1 --2- 172.21.15.37:0/4002405334 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fee981035e0 0x7fee98116030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:46:25.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:25.371+0000 7fee9ec00700 1 -- 172.21.15.37:0/4002405334 >> 172.21.15.37:0/4002405334 conn(0x7fee980f8830 msgr2=0x7fee980f98b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:46:25.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:25.371+0000 7fee9ec00700 1 -- 172.21.15.37:0/4002405334 shutdown_connections 2024-03-29T08:46:25.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:25.371+0000 7fee9ec00700 1 -- 172.21.15.37:0/4002405334 wait complete. 2024-03-29T08:46:25.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:25.508+0000 7f0c40000700 1 -- 172.21.15.37:0/3308356461 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0c380fde90 msgr2=0x7f0c380fe260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:46:25.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:25.508+0000 7f0c40000700 1 --2- 172.21.15.37:0/3308356461 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0c380fde90 0x7f0c380fe260 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f0c2c004940 tx=0x7f0c2c009e30 comp rx=0 tx=0).stop 2024-03-29T08:46:25.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:25.508+0000 7f0c40000700 1 -- 172.21.15.37:0/3308356461 shutdown_connections 2024-03-29T08:46:25.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:25.508+0000 7f0c40000700 1 --2- 172.21.15.37:0/3308356461 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0c380fe830 0x7f0c38106f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:46:25.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:25.508+0000 7f0c40000700 1 --2- 172.21.15.37:0/3308356461 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0c380fde90 0x7f0c380fe260 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:46:25.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:25.508+0000 7f0c40000700 1 -- 172.21.15.37:0/3308356461 >> 172.21.15.37:0/3308356461 conn(0x7f0c38072bc0 msgr2=0x7f0c38072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:46:25.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:25.509+0000 7f0c40000700 1 -- 172.21.15.37:0/3308356461 shutdown_connections 2024-03-29T08:46:25.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:25.509+0000 7f0c40000700 1 -- 172.21.15.37:0/3308356461 wait complete. 2024-03-29T08:46:25.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:25.509+0000 7f0c40000700 1 Processor -- start 2024-03-29T08:46:25.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:25.510+0000 7f0c40000700 1 -- start start 2024-03-29T08:46:25.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:25.510+0000 7f0c40000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0c380fde90 0x7f0c380fef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:46:25.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:25.510+0000 7f0c40000700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0c380fe830 0x7f0c380ff4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:46:25.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:25.510+0000 7f0c40000700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c38103100 con 0x7f0c380fde90 2024-03-29T08:46:25.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:25.510+0000 7f0c40000700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c380ff9f0 con 0x7f0c380fe830 2024-03-29T08:46:25.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:25.510+0000 7f0c3d600700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0c380fde90 0x7f0c380fef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:46:25.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:25.510+0000 7f0c3d600700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0c380fde90 0x7f0c380fef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37934/0 (socket says 172.21.15.37:37934) 2024-03-29T08:46:25.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:25.510+0000 7f0c3d600700 1 -- 172.21.15.37:0/2825755726 learned_addr learned my addr 172.21.15.37:0/2825755726 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:46:25.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:25.511+0000 7f0c3cc00700 1 --2- 172.21.15.37:0/2825755726 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0c380fe830 0x7f0c380ff4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:46:25.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:25.511+0000 7f0c3d600700 1 -- 172.21.15.37:0/2825755726 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0c380fe830 msgr2=0x7f0c380ff4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:46:25.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:25.511+0000 7f0c3d600700 1 --2- 172.21.15.37:0/2825755726 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0c380fe830 0x7f0c380ff4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:46:25.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:25.511+0000 7f0c3d600700 1 -- 172.21.15.37:0/2825755726 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c2c005040 con 0x7f0c380fde90 2024-03-29T08:46:25.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:25.511+0000 7f0c3cc00700 1 --2- 172.21.15.37:0/2825755726 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0c380fe830 0x7f0c380ff4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:46:25.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:25.511+0000 7f0c3d600700 1 --2- 172.21.15.37:0/2825755726 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0c380fde90 0x7f0c380fef70 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f0c2c000c00 tx=0x7f0c2c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:46:25.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:25.511+0000 7f0c35600700 1 -- 172.21.15.37:0/2825755726 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0c2c00e040 con 0x7f0c380fde90 2024-03-29T08:46:25.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:25.511+0000 7f0c35600700 1 -- 172.21.15.37:0/2825755726 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f0c2c013420 con 0x7f0c380fde90 2024-03-29T08:46:25.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:25.511+0000 7f0c40000700 1 -- 172.21.15.37:0/2825755726 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c380ffc70 con 0x7f0c380fde90 2024-03-29T08:46:25.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:25.511+0000 7f0c35600700 1 -- 172.21.15.37:0/2825755726 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0c2c01c630 con 0x7f0c380fde90 2024-03-29T08:46:25.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:25.511+0000 7f0c40000700 1 -- 172.21.15.37:0/2825755726 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c3811e550 con 0x7f0c380fde90 2024-03-29T08:46:25.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:25.512+0000 7f0c40000700 1 -- 172.21.15.37:0/2825755726 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c38106a20 con 0x7f0c380fde90 2024-03-29T08:46:25.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:25.514+0000 7f0c35600700 1 -- 172.21.15.37:0/2825755726 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f0c2c013970 con 0x7f0c380fde90 2024-03-29T08:46:25.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:25.514+0000 7f0c35600700 1 --2- 172.21.15.37:0/2825755726 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f0c2006cc30 0x7f0c2006f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:46:25.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:25.515+0000 7f0c35600700 1 -- 172.21.15.37:0/2825755726 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f0c2c08d2e0 con 0x7f0c380fde90 2024-03-29T08:46:25.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:25.515+0000 7f0c3cc00700 1 --2- 172.21.15.37:0/2825755726 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f0c2006cc30 0x7f0c2006f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:46:25.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:25.515+0000 7f0c3cc00700 1 --2- 172.21.15.37:0/2825755726 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f0c2006cc30 0x7f0c2006f0e0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f0c24009b80 tx=0x7f0c2400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:46:25.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:25.519+0000 7f0c35600700 1 -- 172.21.15.37:0/2825755726 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f0c2c0572a0 con 0x7f0c380fde90 2024-03-29T08:46:25.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:25.740+0000 7f0c40000700 1 -- 172.21.15.37:0/2825755726 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f0c3804bed0 con 0x7f0c2006cc30 2024-03-29T08:46:25.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:25.751+0000 7f0c35600700 1 -- 172.21.15.37:0/2825755726 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f0c3804bed0 con 0x7f0c2006cc30 2024-03-29T08:46:25.750 INFO:teuthology.orchestra.run.smithi037.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-29T08:46:25.751 INFO:teuthology.orchestra.run.smithi037.stdout:alertmanager.smithi037 smithi037 *:9093,9094 running (36m) 13m ago 49m 14.3M - 0.25.0 c8568f914cd2 2f9f9746ed0f 2024-03-29T08:46:25.751 INFO:teuthology.orchestra.run.smithi037.stdout:crash.smithi037 smithi037 running (33m) 13m ago 49m 6647k - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 178d416a3186 2024-03-29T08:46:25.751 INFO:teuthology.orchestra.run.smithi037.stdout:crash.smithi148 smithi148 running (33m) 13m ago 48m 6656k - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 23666989d58f 2024-03-29T08:46:25.751 INFO:teuthology.orchestra.run.smithi037.stdout:grafana.smithi037 smithi037 *:3000 running (35m) 13m ago 48m 74.5M - 9.4.7 954c08fa6188 6b976fe49952 2024-03-29T08:46:25.751 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi037.iifsmh smithi037 running (28m) 13m ago 42m 16.4M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 2d97f562cc5b 2024-03-29T08:46:25.751 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi037.mmwgxe smithi037 running (28m) 13m ago 42m 16.7M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff ec37037f46e8 2024-03-29T08:46:25.751 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi148.qukpjh smithi148 running (13m) 13m ago 42m 10.7M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 3257219bc046 2024-03-29T08:46:25.751 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi148.sqteih smithi148 running (28m) 13m ago 42m 16.3M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 44c15c77037a 2024-03-29T08:46:25.751 INFO:teuthology.orchestra.run.smithi037.stdout:mgr.smithi037.wjikii smithi037 *:8443,9283,8765 running (38m) 13m ago 50m 510M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff c5a1417b9417 2024-03-29T08:46:25.751 INFO:teuthology.orchestra.run.smithi037.stdout:mgr.smithi148.onhgvm smithi148 *:8443,9283,8765 running (37m) 13m ago 48m 456M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff fcfdbce062bd 2024-03-29T08:46:25.751 INFO:teuthology.orchestra.run.smithi037.stdout:mon.smithi037 smithi037 running (34m) 13m ago 51m 70.2M 2048M 18.2.2-1015-gd0e392d9 1cc2f45c17ff eea2060783af 2024-03-29T08:46:25.751 INFO:teuthology.orchestra.run.smithi037.stdout:mon.smithi148 smithi148 running (33m) 13m ago 47m 50.4M 2048M 18.2.2-1015-gd0e392d9 1cc2f45c17ff ad4b2c5748be 2024-03-29T08:46:25.751 INFO:teuthology.orchestra.run.smithi037.stdout:node-exporter.smithi037 smithi037 *:9100 running (37m) 13m ago 49m 8514k - 1.5.0 0da6a335fe13 26b8ae9cb878 2024-03-29T08:46:25.752 INFO:teuthology.orchestra.run.smithi037.stdout:node-exporter.smithi148 smithi148 *:9100 running (36m) 13m ago 48m 8464k - 1.5.0 0da6a335fe13 98414e4e64ed 2024-03-29T08:46:25.752 INFO:teuthology.orchestra.run.smithi037.stdout:osd.0 smithi037 running (32m) 13m ago 47m 108M 1619M 18.2.2-1015-gd0e392d9 1cc2f45c17ff c0c584f6245c 2024-03-29T08:46:25.752 INFO:teuthology.orchestra.run.smithi037.stdout:osd.1 smithi037 running (31m) 13m ago 46m 104M 1619M 18.2.2-1015-gd0e392d9 1cc2f45c17ff a4a2ff4cc83d 2024-03-29T08:46:25.752 INFO:teuthology.orchestra.run.smithi037.stdout:osd.2 smithi037 running (31m) 13m ago 46m 85.9M 1619M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 8b1bc71aa79e 2024-03-29T08:46:25.752 INFO:teuthology.orchestra.run.smithi037.stdout:osd.3 smithi148 running (30m) 13m ago 45m 109M 2642M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 1c656ab8ff86 2024-03-29T08:46:25.752 INFO:teuthology.orchestra.run.smithi037.stdout:osd.4 smithi148 running (29m) 13m ago 44m 101M 2642M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 966a9de11d22 2024-03-29T08:46:25.752 INFO:teuthology.orchestra.run.smithi037.stdout:osd.5 smithi148 running (29m) 13m ago 44m 89.4M 2642M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 474c4f9212ce 2024-03-29T08:46:25.752 INFO:teuthology.orchestra.run.smithi037.stdout:prometheus.smithi037 smithi037 *:9095 running (36m) 13m ago 48m 61.0M - 2.43.0 a07b618ecd1d 30c0223ec589 2024-03-29T08:46:25.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:25.753+0000 7f0c40000700 1 -- 172.21.15.37:0/2825755726 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f0c2006cc30 msgr2=0x7f0c2006f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:46:25.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:25.753+0000 7f0c40000700 1 --2- 172.21.15.37:0/2825755726 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f0c2006cc30 0x7f0c2006f0e0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f0c24009b80 tx=0x7f0c2400b040 comp rx=0 tx=0).stop 2024-03-29T08:46:25.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:25.753+0000 7f0c40000700 1 -- 172.21.15.37:0/2825755726 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0c380fde90 msgr2=0x7f0c380fef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:46:25.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:25.753+0000 7f0c40000700 1 --2- 172.21.15.37:0/2825755726 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0c380fde90 0x7f0c380fef70 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f0c2c000c00 tx=0x7f0c2c009e30 comp rx=0 tx=0).stop 2024-03-29T08:46:25.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:25.753+0000 7f0c40000700 1 -- 172.21.15.37:0/2825755726 shutdown_connections 2024-03-29T08:46:25.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:25.753+0000 7f0c40000700 1 --2- 172.21.15.37:0/2825755726 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f0c2006cc30 0x7f0c2006f0e0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:46:25.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:25.753+0000 7f0c40000700 1 --2- 172.21.15.37:0/2825755726 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0c380fde90 0x7f0c380fef70 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:46:25.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:25.753+0000 7f0c40000700 1 --2- 172.21.15.37:0/2825755726 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0c380fe830 0x7f0c380ff4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:46:25.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:25.753+0000 7f0c40000700 1 -- 172.21.15.37:0/2825755726 >> 172.21.15.37:0/2825755726 conn(0x7f0c38072bc0 msgr2=0x7f0c380fd570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:46:25.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:25.754+0000 7f0c40000700 1 -- 172.21.15.37:0/2825755726 shutdown_connections 2024-03-29T08:46:25.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:25.754+0000 7f0c40000700 1 -- 172.21.15.37:0/2825755726 wait complete. 2024-03-29T08:46:25.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:25.890+0000 7facf2000700 1 -- 172.21.15.37:0/1512005729 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7facec102e00 msgr2=0x7facec1031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:46:25.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:25.890+0000 7facf2000700 1 --2- 172.21.15.37:0/1512005729 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7facec102e00 0x7facec1031d0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7facdc004940 tx=0x7facdc009e30 comp rx=0 tx=0).stop 2024-03-29T08:46:25.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:25.890+0000 7facf2000700 1 -- 172.21.15.37:0/1512005729 shutdown_connections 2024-03-29T08:46:25.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:25.890+0000 7facf2000700 1 --2- 172.21.15.37:0/1512005729 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7facec1037a0 0x7facec0fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:46:25.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:25.890+0000 7facf2000700 1 --2- 172.21.15.37:0/1512005729 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7facec102e00 0x7facec1031d0 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:46:25.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:25.890+0000 7facf2000700 1 -- 172.21.15.37:0/1512005729 >> 172.21.15.37:0/1512005729 conn(0x7facec0f89b0 msgr2=0x7facec0fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:46:25.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:25.890+0000 7facf2000700 1 -- 172.21.15.37:0/1512005729 shutdown_connections 2024-03-29T08:46:25.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:25.890+0000 7facf2000700 1 -- 172.21.15.37:0/1512005729 wait complete. 2024-03-29T08:46:25.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:25.891+0000 7facf2000700 1 Processor -- start 2024-03-29T08:46:25.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:25.891+0000 7facf2000700 1 -- start start 2024-03-29T08:46:25.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:25.892+0000 7facf2000700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7facec102e00 0x7facec115cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:46:25.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:25.892+0000 7facf2000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7facec1037a0 0x7facec116230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:46:25.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:25.892+0000 7facf2000700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facec116880 con 0x7facec1037a0 2024-03-29T08:46:25.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:25.892+0000 7facf2000700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facec10fc50 con 0x7facec102e00 2024-03-29T08:46:25.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:25.892+0000 7faceaa00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7facec1037a0 0x7facec116230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:46:25.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:25.892+0000 7faceaa00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7facec1037a0 0x7facec116230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37944/0 (socket says 172.21.15.37:37944) 2024-03-29T08:46:25.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:25.892+0000 7faceaa00700 1 -- 172.21.15.37:0/3147626645 learned_addr learned my addr 172.21.15.37:0/3147626645 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:46:25.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:25.892+0000 7faceb400700 1 --2- 172.21.15.37:0/3147626645 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7facec102e00 0x7facec115cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:46:25.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:25.892+0000 7faceaa00700 1 -- 172.21.15.37:0/3147626645 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7facec102e00 msgr2=0x7facec115cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:46:25.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:25.892+0000 7faceaa00700 1 --2- 172.21.15.37:0/3147626645 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7facec102e00 0x7facec115cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:46:25.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:25.892+0000 7faceaa00700 1 -- 172.21.15.37:0/3147626645 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facdc005040 con 0x7facec1037a0 2024-03-29T08:46:25.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:25.893+0000 7faceaa00700 1 --2- 172.21.15.37:0/3147626645 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7facec1037a0 0x7facec116230 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7facd800c890 tx=0x7facd80044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:46:25.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:25.893+0000 7face3400700 1 -- 172.21.15.37:0/3147626645 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7facd800a700 con 0x7facec1037a0 2024-03-29T08:46:25.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:25.893+0000 7facf2000700 1 -- 172.21.15.37:0/3147626645 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7facec10ff30 con 0x7facec1037a0 2024-03-29T08:46:25.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:25.893+0000 7face3400700 1 -- 172.21.15.37:0/3147626645 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7facd800ad10 con 0x7facec1037a0 2024-03-29T08:46:25.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:25.893+0000 7face3400700 1 -- 172.21.15.37:0/3147626645 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7facd8012e70 con 0x7facec1037a0 2024-03-29T08:46:25.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:25.893+0000 7facf2000700 1 -- 172.21.15.37:0/3147626645 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7facec1104f0 con 0x7facec1037a0 2024-03-29T08:46:25.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:25.894+0000 7facf2000700 1 -- 172.21.15.37:0/3147626645 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7facec0fe550 con 0x7facec1037a0 2024-03-29T08:46:25.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:25.895+0000 7face3400700 1 -- 172.21.15.37:0/3147626645 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7facd800a860 con 0x7facec1037a0 2024-03-29T08:46:25.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:25.896+0000 7face3400700 1 --2- 172.21.15.37:0/3147626645 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7facd006cc30 0x7facd006f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:46:25.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:25.896+0000 7face3400700 1 -- 172.21.15.37:0/3147626645 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7facd808c120 con 0x7facec1037a0 2024-03-29T08:46:25.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:25.896+0000 7faceb400700 1 --2- 172.21.15.37:0/3147626645 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7facd006cc30 0x7facd006f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:46:25.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:25.897+0000 7faceb400700 1 --2- 172.21.15.37:0/3147626645 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7facd006cc30 0x7facd006f0e0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7facdc00b840 tx=0x7facdc009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:46:25.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:25.901+0000 7face3400700 1 -- 172.21.15.37:0/3147626645 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7facd8059b70 con 0x7facec1037a0 2024-03-29T08:46:25.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:46:25 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:46:26.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:46:25 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:46:26.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:26.208+0000 7facf2000700 1 -- 172.21.15.37:0/3147626645 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7facec04bed0 con 0x7facec1037a0 2024-03-29T08:46:26.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:26.208+0000 7face3400700 1 -- 172.21.15.37:0/3147626645 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7facd8059700 con 0x7facec1037a0 2024-03-29T08:46:26.208 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-29T08:46:26.208 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": { 2024-03-29T08:46:26.208 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 2 2024-03-29T08:46:26.208 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:46:26.208 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": { 2024-03-29T08:46:26.208 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 2 2024-03-29T08:46:26.208 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:46:26.208 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": { 2024-03-29T08:46:26.209 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 6 2024-03-29T08:46:26.209 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:46:26.209 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": { 2024-03-29T08:46:26.209 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 4 2024-03-29T08:46:26.209 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:46:26.209 INFO:teuthology.orchestra.run.smithi037.stdout: "overall": { 2024-03-29T08:46:26.209 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 14 2024-03-29T08:46:26.209 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-29T08:46:26.209 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-29T08:46:26.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:26.210+0000 7facf2000700 1 -- 172.21.15.37:0/3147626645 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7facd006cc30 msgr2=0x7facd006f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:46:26.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:26.210+0000 7facf2000700 1 --2- 172.21.15.37:0/3147626645 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7facd006cc30 0x7facd006f0e0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7facdc00b840 tx=0x7facdc009e30 comp rx=0 tx=0).stop 2024-03-29T08:46:26.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:26.210+0000 7facf2000700 1 -- 172.21.15.37:0/3147626645 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7facec1037a0 msgr2=0x7facec116230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:46:26.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:26.210+0000 7facf2000700 1 --2- 172.21.15.37:0/3147626645 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7facec1037a0 0x7facec116230 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7facd800c890 tx=0x7facd80044c0 comp rx=0 tx=0).stop 2024-03-29T08:46:26.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:26.211+0000 7facf2000700 1 -- 172.21.15.37:0/3147626645 shutdown_connections 2024-03-29T08:46:26.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:26.211+0000 7facf2000700 1 --2- 172.21.15.37:0/3147626645 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7facd006cc30 0x7facd006f0e0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:46:26.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:26.211+0000 7facf2000700 1 --2- 172.21.15.37:0/3147626645 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7facec102e00 0x7facec115cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:46:26.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:26.211+0000 7facf2000700 1 --2- 172.21.15.37:0/3147626645 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7facec1037a0 0x7facec116230 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:46:26.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:26.211+0000 7facf2000700 1 -- 172.21.15.37:0/3147626645 >> 172.21.15.37:0/3147626645 conn(0x7facec0f89b0 msgr2=0x7facec0706d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:46:26.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:26.211+0000 7facf2000700 1 -- 172.21.15.37:0/3147626645 shutdown_connections 2024-03-29T08:46:26.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:26.211+0000 7facf2000700 1 -- 172.21.15.37:0/3147626645 wait complete. 2024-03-29T08:46:26.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:26.347+0000 7f051ce00700 1 -- 172.21.15.37:0/3104846281 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0518102e00 msgr2=0x7f05181031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:46:26.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:26.347+0000 7f051ce00700 1 --2- 172.21.15.37:0/3104846281 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0518102e00 0x7f05181031d0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f0508004940 tx=0x7f0508009e30 comp rx=0 tx=0).stop 2024-03-29T08:46:26.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:26.347+0000 7f051ce00700 1 -- 172.21.15.37:0/3104846281 shutdown_connections 2024-03-29T08:46:26.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:26.347+0000 7f051ce00700 1 --2- 172.21.15.37:0/3104846281 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f05181037a0 0x7f05180fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:46:26.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:26.347+0000 7f051ce00700 1 --2- 172.21.15.37:0/3104846281 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0518102e00 0x7f05181031d0 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:46:26.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:26.347+0000 7f051ce00700 1 -- 172.21.15.37:0/3104846281 >> 172.21.15.37:0/3104846281 conn(0x7f05180f89b0 msgr2=0x7f05180fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:46:26.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:26.347+0000 7f051ce00700 1 -- 172.21.15.37:0/3104846281 shutdown_connections 2024-03-29T08:46:26.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:26.347+0000 7f051ce00700 1 -- 172.21.15.37:0/3104846281 wait complete. 2024-03-29T08:46:26.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:26.348+0000 7f051ce00700 1 Processor -- start 2024-03-29T08:46:26.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:26.348+0000 7f051ce00700 1 -- start start 2024-03-29T08:46:26.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:26.349+0000 7f051ce00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0518102e00 0x7f0518115cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:46:26.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:26.349+0000 7f051ce00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f05181037a0 0x7f0518116230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:46:26.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:26.349+0000 7f051ce00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0518116880 con 0x7f05181037a0 2024-03-29T08:46:26.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:26.349+0000 7f051ce00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f051810fc50 con 0x7f0518102e00 2024-03-29T08:46:26.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:26.349+0000 7f0515400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f05181037a0 0x7f0518116230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:46:26.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:26.349+0000 7f0515400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f05181037a0 0x7f0518116230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37964/0 (socket says 172.21.15.37:37964) 2024-03-29T08:46:26.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:26.349+0000 7f0515400700 1 -- 172.21.15.37:0/1841205360 learned_addr learned my addr 172.21.15.37:0/1841205360 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:46:26.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:26.349+0000 7f0515e00700 1 --2- 172.21.15.37:0/1841205360 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0518102e00 0x7f0518115cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:46:26.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:26.350+0000 7f0515400700 1 -- 172.21.15.37:0/1841205360 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0518102e00 msgr2=0x7f0518115cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:46:26.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:26.350+0000 7f0515400700 1 --2- 172.21.15.37:0/1841205360 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0518102e00 0x7f0518115cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:46:26.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:26.350+0000 7f0515400700 1 -- 172.21.15.37:0/1841205360 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0508005040 con 0x7f05181037a0 2024-03-29T08:46:26.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:26.350+0000 7f0515400700 1 --2- 172.21.15.37:0/1841205360 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f05181037a0 0x7f0518116230 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f0504002a50 tx=0x7f0504004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:46:26.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:26.350+0000 7f050e000700 1 -- 172.21.15.37:0/1841205360 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0504007cc0 con 0x7f05181037a0 2024-03-29T08:46:26.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:26.350+0000 7f050e000700 1 -- 172.21.15.37:0/1841205360 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f050400f040 con 0x7f05181037a0 2024-03-29T08:46:26.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:26.350+0000 7f051ce00700 1 -- 172.21.15.37:0/1841205360 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f051810ff30 con 0x7f05181037a0 2024-03-29T08:46:26.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:26.350+0000 7f050e000700 1 -- 172.21.15.37:0/1841205360 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f050400d920 con 0x7f05181037a0 2024-03-29T08:46:26.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:26.350+0000 7f051ce00700 1 -- 172.21.15.37:0/1841205360 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05181104f0 con 0x7f05181037a0 2024-03-29T08:46:26.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:26.351+0000 7f051ce00700 1 -- 172.21.15.37:0/1841205360 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05180fe550 con 0x7f05181037a0 2024-03-29T08:46:26.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:26.352+0000 7f050e000700 1 -- 172.21.15.37:0/1841205360 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f0504009680 con 0x7f05181037a0 2024-03-29T08:46:26.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:26.353+0000 7f050e000700 1 --2- 172.21.15.37:0/1841205360 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f04fc06cc30 0x7f04fc06f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:46:26.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:26.353+0000 7f050e000700 1 -- 172.21.15.37:0/1841205360 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f050408bdb0 con 0x7f05181037a0 2024-03-29T08:46:26.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:26.353+0000 7f0515e00700 1 --2- 172.21.15.37:0/1841205360 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f04fc06cc30 0x7f04fc06f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:46:26.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:26.354+0000 7f0515e00700 1 --2- 172.21.15.37:0/1841205360 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f04fc06cc30 0x7f04fc06f0e0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f0508009fd0 tx=0x7f0508009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:46:26.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:26.358+0000 7f050e000700 1 -- 172.21.15.37:0/1841205360 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f0504059780 con 0x7f05181037a0 2024-03-29T08:46:26.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:26.612+0000 7f051ce00700 1 -- 172.21.15.37:0/1841205360 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f0518109df0 con 0x7f05181037a0 2024-03-29T08:46:26.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:26.612+0000 7f050e000700 1 -- 172.21.15.37:0/1841205360 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 30 v30) v1 ==== 76+0+1841 (secure 0 0 0) 0x7f0504059310 con 0x7f05181037a0 2024-03-29T08:46:26.612 INFO:teuthology.orchestra.run.smithi037.stdout:e30 2024-03-29T08:46:26.612 INFO:teuthology.orchestra.run.smithi037.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-29T08:46:26.612 INFO:teuthology.orchestra.run.smithi037.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T08:46:26.612 INFO:teuthology.orchestra.run.smithi037.stdout:legacy client fscid: 1 2024-03-29T08:46:26.612 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:46:26.612 INFO:teuthology.orchestra.run.smithi037.stdout:Filesystem 'cephfs' (1) 2024-03-29T08:46:26.612 INFO:teuthology.orchestra.run.smithi037.stdout:fs_name cephfs 2024-03-29T08:46:26.613 INFO:teuthology.orchestra.run.smithi037.stdout:epoch 21 2024-03-29T08:46:26.613 INFO:teuthology.orchestra.run.smithi037.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-03-29T08:46:26.613 INFO:teuthology.orchestra.run.smithi037.stdout:created 2024-03-29T08:03:29.241269+0000 2024-03-29T08:46:26.613 INFO:teuthology.orchestra.run.smithi037.stdout:modified 2024-03-29T08:17:16.237238+0000 2024-03-29T08:46:26.613 INFO:teuthology.orchestra.run.smithi037.stdout:tableserver 0 2024-03-29T08:46:26.613 INFO:teuthology.orchestra.run.smithi037.stdout:root 0 2024-03-29T08:46:26.613 INFO:teuthology.orchestra.run.smithi037.stdout:session_timeout 60 2024-03-29T08:46:26.613 INFO:teuthology.orchestra.run.smithi037.stdout:session_autoclose 300 2024-03-29T08:46:26.613 INFO:teuthology.orchestra.run.smithi037.stdout:max_file_size 1099511627776 2024-03-29T08:46:26.613 INFO:teuthology.orchestra.run.smithi037.stdout:max_xattr_size 65536 2024-03-29T08:46:26.613 INFO:teuthology.orchestra.run.smithi037.stdout:required_client_features {} 2024-03-29T08:46:26.613 INFO:teuthology.orchestra.run.smithi037.stdout:last_failure 0 2024-03-29T08:46:26.613 INFO:teuthology.orchestra.run.smithi037.stdout:last_failure_osd_epoch 82 2024-03-29T08:46:26.613 INFO:teuthology.orchestra.run.smithi037.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T08:46:26.613 INFO:teuthology.orchestra.run.smithi037.stdout:max_mds 2 2024-03-29T08:46:26.614 INFO:teuthology.orchestra.run.smithi037.stdout:in 0,1 2024-03-29T08:46:26.614 INFO:teuthology.orchestra.run.smithi037.stdout:up {} 2024-03-29T08:46:26.614 INFO:teuthology.orchestra.run.smithi037.stdout:failed 0,1 2024-03-29T08:46:26.614 INFO:teuthology.orchestra.run.smithi037.stdout:damaged 2024-03-29T08:46:26.614 INFO:teuthology.orchestra.run.smithi037.stdout:stopped 2024-03-29T08:46:26.614 INFO:teuthology.orchestra.run.smithi037.stdout:data_pools [3] 2024-03-29T08:46:26.614 INFO:teuthology.orchestra.run.smithi037.stdout:metadata_pool 2 2024-03-29T08:46:26.614 INFO:teuthology.orchestra.run.smithi037.stdout:inline_data disabled 2024-03-29T08:46:26.614 INFO:teuthology.orchestra.run.smithi037.stdout:balancer 2024-03-29T08:46:26.614 INFO:teuthology.orchestra.run.smithi037.stdout:bal_rank_mask -1 2024-03-29T08:46:26.614 INFO:teuthology.orchestra.run.smithi037.stdout:standby_count_wanted 1 2024-03-29T08:46:26.614 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:46:26.614 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:46:26.614 INFO:teuthology.orchestra.run.smithi037.stdout:Standby daemons: 2024-03-29T08:46:26.615 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:46:26.615 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi037.mmwgxe{-1:34440} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.37:6828/1248142005,v1:172.21.15.37:6829/1248142005] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:46:26.615 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi037.iifsmh{-1:34444} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.37:6826/2060882959,v1:172.21.15.37:6827/2060882959] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:46:26.615 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi148.sqteih{-1:44395} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.148:6824/4078652242,v1:172.21.15.148:6825/4078652242] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:46:26.615 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi148.qukpjh{-1:45007} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.148:6826/2394751105,v1:172.21.15.148:6827/2394751105] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:46:26.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:26.614+0000 7f051ce00700 1 -- 172.21.15.37:0/1841205360 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f04fc06cc30 msgr2=0x7f04fc06f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:46:26.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:26.614+0000 7f051ce00700 1 --2- 172.21.15.37:0/1841205360 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f04fc06cc30 0x7f04fc06f0e0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f0508009fd0 tx=0x7f0508009e30 comp rx=0 tx=0).stop 2024-03-29T08:46:26.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:26.614+0000 7f051ce00700 1 -- 172.21.15.37:0/1841205360 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f05181037a0 msgr2=0x7f0518116230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:46:26.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:26.614+0000 7f051ce00700 1 --2- 172.21.15.37:0/1841205360 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f05181037a0 0x7f0518116230 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f0504002a50 tx=0x7f0504004600 comp rx=0 tx=0).stop 2024-03-29T08:46:26.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:26.614+0000 7f051ce00700 1 -- 172.21.15.37:0/1841205360 shutdown_connections 2024-03-29T08:46:26.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:26.614+0000 7f051ce00700 1 --2- 172.21.15.37:0/1841205360 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f04fc06cc30 0x7f04fc06f0e0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:46:26.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:26.614+0000 7f051ce00700 1 --2- 172.21.15.37:0/1841205360 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0518102e00 0x7f0518115cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:46:26.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:26.614+0000 7f051ce00700 1 --2- 172.21.15.37:0/1841205360 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f05181037a0 0x7f0518116230 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:46:26.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:26.614+0000 7f051ce00700 1 -- 172.21.15.37:0/1841205360 >> 172.21.15.37:0/1841205360 conn(0x7f05180f89b0 msgr2=0x7f05180706d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:46:26.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:26.615+0000 7f051ce00700 1 -- 172.21.15.37:0/1841205360 shutdown_connections 2024-03-29T08:46:26.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:26.615+0000 7f051ce00700 1 -- 172.21.15.37:0/1841205360 wait complete. 2024-03-29T08:46:26.616 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 30 2024-03-29T08:46:26.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:26.753+0000 7f266e400700 1 -- 172.21.15.37:0/2820791957 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2668102e00 msgr2=0x7f26681031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:46:26.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:26.753+0000 7f266e400700 1 --2- 172.21.15.37:0/2820791957 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2668102e00 0x7f26681031d0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f2658004940 tx=0x7f2658009e30 comp rx=0 tx=0).stop 2024-03-29T08:46:26.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:26.753+0000 7f266e400700 1 -- 172.21.15.37:0/2820791957 shutdown_connections 2024-03-29T08:46:26.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:26.753+0000 7f266e400700 1 --2- 172.21.15.37:0/2820791957 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f26681037a0 0x7f26680fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:46:26.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:26.753+0000 7f266e400700 1 --2- 172.21.15.37:0/2820791957 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2668102e00 0x7f26681031d0 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:46:26.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:26.753+0000 7f266e400700 1 -- 172.21.15.37:0/2820791957 >> 172.21.15.37:0/2820791957 conn(0x7f26680f89b0 msgr2=0x7f26680fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:46:26.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:26.753+0000 7f266e400700 1 -- 172.21.15.37:0/2820791957 shutdown_connections 2024-03-29T08:46:26.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:26.753+0000 7f266e400700 1 -- 172.21.15.37:0/2820791957 wait complete. 2024-03-29T08:46:26.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:26.754+0000 7f266e400700 1 Processor -- start 2024-03-29T08:46:26.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:26.754+0000 7f266e400700 1 -- start start 2024-03-29T08:46:26.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:26.754+0000 7f266e400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2668102e00 0x7f2668115cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:46:26.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:26.754+0000 7f266e400700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f26681037a0 0x7f2668116230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:46:26.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:26.754+0000 7f266e400700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2668116880 con 0x7f2668102e00 2024-03-29T08:46:26.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:26.754+0000 7f266e400700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f266810fc50 con 0x7f26681037a0 2024-03-29T08:46:26.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:26.755+0000 7f2667400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2668102e00 0x7f2668115cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:46:26.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:26.755+0000 7f2667400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2668102e00 0x7f2668115cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37972/0 (socket says 172.21.15.37:37972) 2024-03-29T08:46:26.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:26.755+0000 7f2667400700 1 -- 172.21.15.37:0/3836726730 learned_addr learned my addr 172.21.15.37:0/3836726730 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:46:26.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:26.755+0000 7f2666a00700 1 --2- 172.21.15.37:0/3836726730 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f26681037a0 0x7f2668116230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:46:26.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:26.755+0000 7f2667400700 1 -- 172.21.15.37:0/3836726730 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f26681037a0 msgr2=0x7f2668116230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:46:26.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:26.755+0000 7f2667400700 1 --2- 172.21.15.37:0/3836726730 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f26681037a0 0x7f2668116230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:46:26.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:26.755+0000 7f2667400700 1 -- 172.21.15.37:0/3836726730 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2658005040 con 0x7f2668102e00 2024-03-29T08:46:26.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:26.755+0000 7f2667400700 1 --2- 172.21.15.37:0/3836726730 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2668102e00 0x7f2668115cf0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f2658009360 tx=0x7f2658009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:46:26.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:26.756+0000 7f265f400700 1 -- 172.21.15.37:0/3836726730 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f265800e040 con 0x7f2668102e00 2024-03-29T08:46:26.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:26.756+0000 7f266e400700 1 -- 172.21.15.37:0/3836726730 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f266810fed0 con 0x7f2668102e00 2024-03-29T08:46:26.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:26.756+0000 7f265f400700 1 -- 172.21.15.37:0/3836726730 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f2658013420 con 0x7f2668102e00 2024-03-29T08:46:26.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:26.756+0000 7f265f400700 1 -- 172.21.15.37:0/3836726730 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f265801c5f0 con 0x7f2668102e00 2024-03-29T08:46:26.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:26.756+0000 7f266e400700 1 -- 172.21.15.37:0/3836726730 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2668110490 con 0x7f2668102e00 2024-03-29T08:46:26.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:26.757+0000 7f266e400700 1 -- 172.21.15.37:0/3836726730 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26680fe550 con 0x7f2668102e00 2024-03-29T08:46:26.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:26.758+0000 7f265f400700 1 -- 172.21.15.37:0/3836726730 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f2658013590 con 0x7f2668102e00 2024-03-29T08:46:26.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:26.759+0000 7f265f400700 1 --2- 172.21.15.37:0/3836726730 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f264c06cc30 0x7f264c06f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:46:26.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:26.759+0000 7f265f400700 1 -- 172.21.15.37:0/3836726730 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f265808d250 con 0x7f2668102e00 2024-03-29T08:46:26.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:26.759+0000 7f2666a00700 1 --2- 172.21.15.37:0/3836726730 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f264c06cc30 0x7f264c06f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:46:26.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:26.760+0000 7f2666a00700 1 --2- 172.21.15.37:0/3836726730 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f264c06cc30 0x7f264c06f0e0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f2668111590 tx=0x7f2654009ee0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:46:26.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:26.763+0000 7f265f400700 1 -- 172.21.15.37:0/3836726730 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f265805add0 con 0x7f2668102e00 2024-03-29T08:46:26.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:46:26 smithi037 ceph-mon[142325]: from='client.35906 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:46:26.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:46:26 smithi037 ceph-mon[142325]: pgmap v1067: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:46:26.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:46:26 smithi037 ceph-mon[142325]: from='client.35910 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:46:26.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:46:26 smithi037 ceph-mon[142325]: from='client.35914 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:46:26.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:46:26 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/3147626645' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:46:26.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:46:26 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/1841205360' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T08:46:26.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:26.999+0000 7f266e400700 1 -- 172.21.15.37:0/3836726730 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f266804bed0 con 0x7f264c06cc30 2024-03-29T08:46:27.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:27.001+0000 7f265f400700 1 -- 172.21.15.37:0/3836726730 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f266804bed0 con 0x7f264c06cc30 2024-03-29T08:46:27.000 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-29T08:46:27.000 INFO:teuthology.orchestra.run.smithi037.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59", 2024-03-29T08:46:27.000 INFO:teuthology.orchestra.run.smithi037.stdout: "in_progress": true, 2024-03-29T08:46:27.000 INFO:teuthology.orchestra.run.smithi037.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-29T08:46:27.000 INFO:teuthology.orchestra.run.smithi037.stdout: "services_complete": [ 2024-03-29T08:46:27.001 INFO:teuthology.orchestra.run.smithi037.stdout: "crash", 2024-03-29T08:46:27.001 INFO:teuthology.orchestra.run.smithi037.stdout: "osd", 2024-03-29T08:46:27.001 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr", 2024-03-29T08:46:27.001 INFO:teuthology.orchestra.run.smithi037.stdout: "mds", 2024-03-29T08:46:27.001 INFO:teuthology.orchestra.run.smithi037.stdout: "mon" 2024-03-29T08:46:27.001 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-29T08:46:27.001 INFO:teuthology.orchestra.run.smithi037.stdout: "progress": "16/21 daemons upgraded", 2024-03-29T08:46:27.001 INFO:teuthology.orchestra.run.smithi037.stdout: "message": "Currently upgrading mds daemons", 2024-03-29T08:46:27.001 INFO:teuthology.orchestra.run.smithi037.stdout: "is_paused": false 2024-03-29T08:46:27.001 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-29T08:46:27.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:27.003+0000 7f266e400700 1 -- 172.21.15.37:0/3836726730 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f264c06cc30 msgr2=0x7f264c06f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:46:27.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:27.003+0000 7f266e400700 1 --2- 172.21.15.37:0/3836726730 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f264c06cc30 0x7f264c06f0e0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f2668111590 tx=0x7f2654009ee0 comp rx=0 tx=0).stop 2024-03-29T08:46:27.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:27.003+0000 7f266e400700 1 -- 172.21.15.37:0/3836726730 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2668102e00 msgr2=0x7f2668115cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:46:27.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:27.003+0000 7f266e400700 1 --2- 172.21.15.37:0/3836726730 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2668102e00 0x7f2668115cf0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f2658009360 tx=0x7f2658009e30 comp rx=0 tx=0).stop 2024-03-29T08:46:27.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:27.003+0000 7f266e400700 1 -- 172.21.15.37:0/3836726730 shutdown_connections 2024-03-29T08:46:27.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:27.003+0000 7f266e400700 1 --2- 172.21.15.37:0/3836726730 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f264c06cc30 0x7f264c06f0e0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:46:27.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:27.003+0000 7f266e400700 1 --2- 172.21.15.37:0/3836726730 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2668102e00 0x7f2668115cf0 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:46:27.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:27.003+0000 7f266e400700 1 --2- 172.21.15.37:0/3836726730 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f26681037a0 0x7f2668116230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:46:27.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:27.003+0000 7f266e400700 1 -- 172.21.15.37:0/3836726730 >> 172.21.15.37:0/3836726730 conn(0x7f26680f89b0 msgr2=0x7f26680706d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:46:27.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:27.003+0000 7f266e400700 1 -- 172.21.15.37:0/3836726730 shutdown_connections 2024-03-29T08:46:27.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:27.003+0000 7f266e400700 1 -- 172.21.15.37:0/3836726730 wait complete. 2024-03-29T08:46:27.055 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:46:26 smithi148 ceph-mon[106692]: from='client.35906 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:46:27.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:46:26 smithi148 ceph-mon[106692]: pgmap v1067: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:46:27.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:46:26 smithi148 ceph-mon[106692]: from='client.35910 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:46:27.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:46:26 smithi148 ceph-mon[106692]: from='client.35914 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:46:27.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:46:26 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/3147626645' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:46:27.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:46:26 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/1841205360' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T08:46:27.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:27.139+0000 7fe2a1c00700 1 -- 172.21.15.37:0/2830596089 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe29c0fde90 msgr2=0x7fe29c0fe260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:46:27.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:27.139+0000 7fe2a1c00700 1 --2- 172.21.15.37:0/2830596089 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe29c0fde90 0x7fe29c0fe260 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7fe28c004940 tx=0x7fe28c009e30 comp rx=0 tx=0).stop 2024-03-29T08:46:27.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:27.139+0000 7fe2a1c00700 1 -- 172.21.15.37:0/2830596089 shutdown_connections 2024-03-29T08:46:27.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:27.139+0000 7fe2a1c00700 1 --2- 172.21.15.37:0/2830596089 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe29c0fe830 0x7fe29c106f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:46:27.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:27.139+0000 7fe2a1c00700 1 --2- 172.21.15.37:0/2830596089 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe29c0fde90 0x7fe29c0fe260 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:46:27.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:27.139+0000 7fe2a1c00700 1 -- 172.21.15.37:0/2830596089 >> 172.21.15.37:0/2830596089 conn(0x7fe29c072bc0 msgr2=0x7fe29c072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:46:27.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:27.139+0000 7fe2a1c00700 1 -- 172.21.15.37:0/2830596089 shutdown_connections 2024-03-29T08:46:27.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:27.139+0000 7fe2a1c00700 1 -- 172.21.15.37:0/2830596089 wait complete. 2024-03-29T08:46:27.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:27.141+0000 7fe2a1c00700 1 Processor -- start 2024-03-29T08:46:27.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:27.141+0000 7fe2a1c00700 1 -- start start 2024-03-29T08:46:27.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:27.141+0000 7fe2a1c00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe29c0fde90 0x7fe29c115d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:46:27.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:27.141+0000 7fe2a1c00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe29c0fe830 0x7fe29c116260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:46:27.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:27.141+0000 7fe2a1c00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe29c1168b0 con 0x7fe29c0fe830 2024-03-29T08:46:27.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:27.141+0000 7fe2a1c00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe29c10fc80 con 0x7fe29c0fde90 2024-03-29T08:46:27.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:27.142+0000 7fe29a000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe29c0fe830 0x7fe29c116260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:46:27.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:27.142+0000 7fe29a000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe29c0fe830 0x7fe29c116260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37986/0 (socket says 172.21.15.37:37986) 2024-03-29T08:46:27.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:27.142+0000 7fe29a000700 1 -- 172.21.15.37:0/1740373710 learned_addr learned my addr 172.21.15.37:0/1740373710 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:46:27.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:27.142+0000 7fe29aa00700 1 --2- 172.21.15.37:0/1740373710 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe29c0fde90 0x7fe29c115d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:46:27.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:27.142+0000 7fe29a000700 1 -- 172.21.15.37:0/1740373710 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe29c0fde90 msgr2=0x7fe29c115d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:46:27.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:27.142+0000 7fe29a000700 1 --2- 172.21.15.37:0/1740373710 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe29c0fde90 0x7fe29c115d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:46:27.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:27.142+0000 7fe29a000700 1 -- 172.21.15.37:0/1740373710 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe28c005040 con 0x7fe29c0fe830 2024-03-29T08:46:27.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:27.142+0000 7fe29a000700 1 --2- 172.21.15.37:0/1740373710 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe29c0fe830 0x7fe29c116260 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7fe288002a50 tx=0x7fe2880046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:46:27.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:27.143+0000 7fe292a00700 1 -- 172.21.15.37:0/1740373710 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe288007cc0 con 0x7fe29c0fe830 2024-03-29T08:46:27.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:27.143+0000 7fe2a1c00700 1 -- 172.21.15.37:0/1740373710 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe29c10ff60 con 0x7fe29c0fe830 2024-03-29T08:46:27.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:27.143+0000 7fe292a00700 1 -- 172.21.15.37:0/1740373710 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fe28800f040 con 0x7fe29c0fe830 2024-03-29T08:46:27.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:27.143+0000 7fe292a00700 1 -- 172.21.15.37:0/1740373710 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe28800d9c0 con 0x7fe29c0fe830 2024-03-29T08:46:27.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:27.143+0000 7fe2a1c00700 1 -- 172.21.15.37:0/1740373710 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe29c1104a0 con 0x7fe29c0fe830 2024-03-29T08:46:27.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:27.144+0000 7fe2a1c00700 1 -- 172.21.15.37:0/1740373710 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe29c106a20 con 0x7fe29c0fe830 2024-03-29T08:46:27.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:27.145+0000 7fe292a00700 1 -- 172.21.15.37:0/1740373710 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fe28800db20 con 0x7fe29c0fe830 2024-03-29T08:46:27.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:27.146+0000 7fe292a00700 1 --2- 172.21.15.37:0/1740373710 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fe28006cc30 0x7fe28006f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:46:27.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:27.146+0000 7fe292a00700 1 -- 172.21.15.37:0/1740373710 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fe28808bd20 con 0x7fe29c0fe830 2024-03-29T08:46:27.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:27.146+0000 7fe29aa00700 1 --2- 172.21.15.37:0/1740373710 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fe28006cc30 0x7fe28006f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:46:27.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:27.147+0000 7fe29aa00700 1 --2- 172.21.15.37:0/1740373710 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fe28006cc30 0x7fe28006f0e0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fe28c00b3b0 tx=0x7fe28c009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:46:27.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:27.150+0000 7fe292a00700 1 -- 172.21.15.37:0/1740373710 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fe288059770 con 0x7fe29c0fe830 2024-03-29T08:46:27.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:27.450+0000 7fe2a1c00700 1 -- 172.21.15.37:0/1740373710 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fe29c065e30 con 0x7fe29c0fe830 2024-03-29T08:46:27.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:27.450+0000 7fe292a00700 1 -- 172.21.15.37:0/1740373710 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+3583 (secure 0 0 0) 0x7fe288059300 con 0x7fe29c0fe830 2024-03-29T08:46:27.450 INFO:teuthology.orchestra.run.smithi037.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem is offline 2024-03-29T08:46:27.450 INFO:teuthology.orchestra.run.smithi037.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-03-29T08:46:27.450 INFO:teuthology.orchestra.run.smithi037.stdout: host smithi148 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/mon.smithi148/config 2024-03-29T08:46:27.451 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/podman: timeout after 895 seconds 2024-03-29T08:46:27.451 INFO:teuthology.orchestra.run.smithi037.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59 -e NODE_NAME=smithi148 -e CEPH_USE_RANDOM_NONCE=1 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4:/var/run/ceph:z -v /var/log/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4:/var/log/ceph:z -v /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpkx2rz2d4:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59 inventory --format=json-pretty --filter-for-batch 2024-03-29T08:46:27.451 INFO:teuthology.orchestra.run.smithi037.stdout:Traceback (most recent call last): 2024-03-29T08:46:27.451 INFO:teuthology.orchestra.run.smithi037.stdout: File "/usr/lib64/python3.6/runpy.py", line 193, in _run_module_as_main 2024-03-29T08:46:27.451 INFO:teuthology.orchestra.run.smithi037.stdout: "__main__", mod_spec) 2024-03-29T08:46:27.451 INFO:teuthology.orchestra.run.smithi037.stdout: File "/usr/lib64/python3.6/runpy.py", line 85, in _run_code 2024-03-29T08:46:27.451 INFO:teuthology.orchestra.run.smithi037.stdout: exec(code, run_globals) 2024-03-29T08:46:27.451 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 10889, in 2024-03-29T08:46:27.451 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 10877, in main 2024-03-29T08:46:27.451 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 2576, in _infer_config 2024-03-29T08:46:27.451 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 2492, in _infer_fsid 2024-03-29T08:46:27.451 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 2604, in _infer_image 2024-03-29T08:46:27.451 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 2479, in _validate_fsid 2024-03-29T08:46:27.451 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 7145, in command_ceph_volume 2024-03-29T08:46:27.451 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 2267, in call_throws 2024-03-29T08:46:27.451 INFO:teuthology.orchestra.run.smithi037.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59 -e NODE_NAME=smithi148 -e CEPH_USE_RANDOM_NONCE=1 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4:/var/run/ceph:z -v /var/log/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4:/var/log/ceph:z -v /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpkx2rz2d4:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59 inventory --format=json-pretty --filter-for-batch 2024-03-29T08:46:27.452 INFO:teuthology.orchestra.run.smithi037.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-03-29T08:46:27.452 INFO:teuthology.orchestra.run.smithi037.stdout: fs cephfs is degraded 2024-03-29T08:46:27.452 INFO:teuthology.orchestra.run.smithi037.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-03-29T08:46:27.452 INFO:teuthology.orchestra.run.smithi037.stdout: fs cephfs is offline because no MDS is active for it. 2024-03-29T08:46:27.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:27.452+0000 7fe2a1c00700 1 -- 172.21.15.37:0/1740373710 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fe28006cc30 msgr2=0x7fe28006f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:46:27.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:27.452+0000 7fe2a1c00700 1 --2- 172.21.15.37:0/1740373710 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fe28006cc30 0x7fe28006f0e0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fe28c00b3b0 tx=0x7fe28c009e30 comp rx=0 tx=0).stop 2024-03-29T08:46:27.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:27.452+0000 7fe2a1c00700 1 -- 172.21.15.37:0/1740373710 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe29c0fe830 msgr2=0x7fe29c116260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:46:27.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:27.452+0000 7fe2a1c00700 1 --2- 172.21.15.37:0/1740373710 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe29c0fe830 0x7fe29c116260 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7fe288002a50 tx=0x7fe2880046b0 comp rx=0 tx=0).stop 2024-03-29T08:46:27.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:27.452+0000 7fe2a1c00700 1 -- 172.21.15.37:0/1740373710 shutdown_connections 2024-03-29T08:46:27.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:27.452+0000 7fe2a1c00700 1 --2- 172.21.15.37:0/1740373710 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fe28006cc30 0x7fe28006f0e0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:46:27.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:27.452+0000 7fe2a1c00700 1 --2- 172.21.15.37:0/1740373710 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe29c0fde90 0x7fe29c115d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:46:27.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:27.452+0000 7fe2a1c00700 1 --2- 172.21.15.37:0/1740373710 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe29c0fe830 0x7fe29c116260 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:46:27.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:27.453+0000 7fe2a1c00700 1 -- 172.21.15.37:0/1740373710 >> 172.21.15.37:0/1740373710 conn(0x7fe29c072bc0 msgr2=0x7fe29c0fb8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:46:27.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:27.453+0000 7fe2a1c00700 1 -- 172.21.15.37:0/1740373710 shutdown_connections 2024-03-29T08:46:27.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:27.453+0000 7fe2a1c00700 1 -- 172.21.15.37:0/1740373710 wait complete. 2024-03-29T08:46:27.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:46:27 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/1740373710' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T08:46:28.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:46:27 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/1740373710' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T08:46:28.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:46:28 smithi037 ceph-mon[142325]: from='client.35926 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:46:28.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:46:28 smithi037 ceph-mon[142325]: pgmap v1068: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:46:29.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:46:28 smithi148 ceph-mon[106692]: from='client.35926 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:46:29.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:46:28 smithi148 ceph-mon[106692]: pgmap v1068: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:46:30.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:46:30 smithi037 ceph-mon[142325]: pgmap v1069: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:46:31.055 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:46:30 smithi148 ceph-mon[106692]: pgmap v1069: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:46:32.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:46:32 smithi037 ceph-mon[142325]: pgmap v1070: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:46:33.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:46:32 smithi148 ceph-mon[106692]: pgmap v1070: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:46:34.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:46:34 smithi037 ceph-mon[142325]: pgmap v1071: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:46:35.055 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:46:34 smithi148 ceph-mon[106692]: pgmap v1071: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:46:36.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:46:36 smithi037 ceph-mon[142325]: pgmap v1072: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:46:37.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:46:36 smithi148 ceph-mon[106692]: pgmap v1072: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:46:38.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:46:38 smithi037 ceph-mon[142325]: pgmap v1073: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:46:39.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:46:38 smithi148 ceph-mon[106692]: pgmap v1073: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:46:40.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:46:40 smithi037 ceph-mon[142325]: pgmap v1074: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:46:40.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:46:40 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:46:41.055 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:46:40 smithi148 ceph-mon[106692]: pgmap v1074: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:46:41.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:46:40 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:46:42.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:46:42 smithi037 ceph-mon[142325]: pgmap v1075: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:46:43.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:46:42 smithi148 ceph-mon[106692]: pgmap v1075: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:46:44.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:46:44 smithi037 ceph-mon[142325]: pgmap v1076: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:46:45.055 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:46:44 smithi148 ceph-mon[106692]: pgmap v1076: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:46:46.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:46:46 smithi037 ceph-mon[142325]: pgmap v1077: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:46:47.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:46:46 smithi148 ceph-mon[106692]: pgmap v1077: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:46:48.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:46:48 smithi037 ceph-mon[142325]: pgmap v1078: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:46:49.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:46:48 smithi148 ceph-mon[106692]: pgmap v1078: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:46:50.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:46:50 smithi037 ceph-mon[142325]: pgmap v1079: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:46:51.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:46:50 smithi148 ceph-mon[106692]: pgmap v1079: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:46:52.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:46:52 smithi037 ceph-mon[142325]: pgmap v1080: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:46:53.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:46:52 smithi148 ceph-mon[106692]: pgmap v1080: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:46:54.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:46:54 smithi037 ceph-mon[142325]: pgmap v1081: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:46:55.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:46:54 smithi148 ceph-mon[106692]: pgmap v1081: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:46:55.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:46:55 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:46:56.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:46:55 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:46:56.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:46:56 smithi037 ceph-mon[142325]: pgmap v1082: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:46:57.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:46:56 smithi148 ceph-mon[106692]: pgmap v1082: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:46:57.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:57.606+0000 7f3aa4a00700 1 -- 172.21.15.37:0/1778035999 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3aa0070c70 msgr2=0x7f3aa0109090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:46:57.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:57.606+0000 7f3aa4a00700 1 --2- 172.21.15.37:0/1778035999 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3aa0070c70 0x7f3aa0109090 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f3a8c004970 tx=0x7f3a8c009e30 comp rx=0 tx=0).stop 2024-03-29T08:46:57.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:57.606+0000 7f3aa4a00700 1 -- 172.21.15.37:0/1778035999 shutdown_connections 2024-03-29T08:46:57.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:57.606+0000 7f3aa4a00700 1 --2- 172.21.15.37:0/1778035999 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3aa0070c70 0x7f3aa0109090 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:46:57.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:57.606+0000 7f3aa4a00700 1 --2- 172.21.15.37:0/1778035999 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3aa0070360 0x7f3aa0070730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:46:57.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:57.606+0000 7f3aa4a00700 1 -- 172.21.15.37:0/1778035999 >> 172.21.15.37:0/1778035999 conn(0x7f3aa00f8930 msgr2=0x7f3aa00fad40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:46:57.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:57.606+0000 7f3aa4a00700 1 -- 172.21.15.37:0/1778035999 shutdown_connections 2024-03-29T08:46:57.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:57.606+0000 7f3aa4a00700 1 -- 172.21.15.37:0/1778035999 wait complete. 2024-03-29T08:46:57.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:57.607+0000 7f3aa4a00700 1 Processor -- start 2024-03-29T08:46:57.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:57.607+0000 7f3aa4a00700 1 -- start start 2024-03-29T08:46:57.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:57.607+0000 7f3aa4a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3aa0070360 0x7f3aa0115ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:46:57.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:57.607+0000 7f3aa4a00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3aa0070c70 0x7f3aa01160e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:46:57.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:57.607+0000 7f3aa4a00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3aa0116730 con 0x7f3aa0070360 2024-03-29T08:46:57.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:57.607+0000 7f3aa4a00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3aa010fb40 con 0x7f3aa0070c70 2024-03-29T08:46:57.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:57.608+0000 7f3a9de00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3aa0070360 0x7f3aa0115ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:46:57.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:57.608+0000 7f3a9de00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3aa0070360 0x7f3aa0115ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53518/0 (socket says 172.21.15.37:53518) 2024-03-29T08:46:57.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:57.608+0000 7f3a9de00700 1 -- 172.21.15.37:0/1260796753 learned_addr learned my addr 172.21.15.37:0/1260796753 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:46:57.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:57.608+0000 7f3a9d400700 1 --2- 172.21.15.37:0/1260796753 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3aa0070c70 0x7f3aa01160e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:46:57.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:57.608+0000 7f3a9de00700 1 -- 172.21.15.37:0/1260796753 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3aa0070c70 msgr2=0x7f3aa01160e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:46:57.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:57.608+0000 7f3a9de00700 1 --2- 172.21.15.37:0/1260796753 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3aa0070c70 0x7f3aa01160e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:46:57.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:57.608+0000 7f3a9de00700 1 -- 172.21.15.37:0/1260796753 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a8c005040 con 0x7f3aa0070360 2024-03-29T08:46:57.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:57.608+0000 7f3a9d400700 1 --2- 172.21.15.37:0/1260796753 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3aa0070c70 0x7f3aa01160e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:46:57.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:57.608+0000 7f3a9de00700 1 --2- 172.21.15.37:0/1260796753 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3aa0070360 0x7f3aa0115ba0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f3a9000c830 tx=0x7f3a90004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:46:57.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:57.609+0000 7f3a96000700 1 -- 172.21.15.37:0/1260796753 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3a9000a760 con 0x7f3aa0070360 2024-03-29T08:46:57.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:57.609+0000 7f3a96000700 1 -- 172.21.15.37:0/1260796753 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f3a9000ad70 con 0x7f3aa0070360 2024-03-29T08:46:57.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:57.609+0000 7f3aa4a00700 1 -- 172.21.15.37:0/1260796753 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3aa010fe20 con 0x7f3aa0070360 2024-03-29T08:46:57.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:57.609+0000 7f3a96000700 1 -- 172.21.15.37:0/1260796753 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3a900113a0 con 0x7f3aa0070360 2024-03-29T08:46:57.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:57.609+0000 7f3aa4a00700 1 -- 172.21.15.37:0/1260796753 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3aa0110360 con 0x7f3aa0070360 2024-03-29T08:46:57.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:57.610+0000 7f3aa4a00700 1 -- 172.21.15.37:0/1260796753 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3aa0071aa0 con 0x7f3aa0070360 2024-03-29T08:46:57.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:57.611+0000 7f3a96000700 1 -- 172.21.15.37:0/1260796753 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f3a90012400 con 0x7f3aa0070360 2024-03-29T08:46:57.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:57.612+0000 7f3a96000700 1 --2- 172.21.15.37:0/1260796753 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f3a8406cc30 0x7f3a8406f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:46:57.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:57.612+0000 7f3a96000700 1 -- 172.21.15.37:0/1260796753 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f3a9008c120 con 0x7f3aa0070360 2024-03-29T08:46:57.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:57.612+0000 7f3a9d400700 1 --2- 172.21.15.37:0/1260796753 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f3a8406cc30 0x7f3a8406f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:46:57.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:57.613+0000 7f3a9d400700 1 --2- 172.21.15.37:0/1260796753 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f3a8406cc30 0x7f3a8406f0e0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f3aa0111440 tx=0x7f3a8c009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:46:57.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:57.617+0000 7f3a96000700 1 -- 172.21.15.37:0/1260796753 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f3a900560e0 con 0x7f3aa0070360 2024-03-29T08:46:57.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:57.850+0000 7f3aa4a00700 1 -- 172.21.15.37:0/1260796753 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3aa004bed0 con 0x7f3a8406cc30 2024-03-29T08:46:57.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:57.851+0000 7f3a96000700 1 -- 172.21.15.37:0/1260796753 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f3aa004bed0 con 0x7f3a8406cc30 2024-03-29T08:46:57.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:57.853+0000 7f3aa4a00700 1 -- 172.21.15.37:0/1260796753 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f3a8406cc30 msgr2=0x7f3a8406f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:46:57.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:57.853+0000 7f3aa4a00700 1 --2- 172.21.15.37:0/1260796753 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f3a8406cc30 0x7f3a8406f0e0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f3aa0111440 tx=0x7f3a8c009e30 comp rx=0 tx=0).stop 2024-03-29T08:46:57.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:57.853+0000 7f3aa4a00700 1 -- 172.21.15.37:0/1260796753 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3aa0070360 msgr2=0x7f3aa0115ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:46:57.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:57.853+0000 7f3aa4a00700 1 --2- 172.21.15.37:0/1260796753 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3aa0070360 0x7f3aa0115ba0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f3a9000c830 tx=0x7f3a90004470 comp rx=0 tx=0).stop 2024-03-29T08:46:57.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:57.853+0000 7f3aa4a00700 1 -- 172.21.15.37:0/1260796753 shutdown_connections 2024-03-29T08:46:57.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:57.853+0000 7f3aa4a00700 1 --2- 172.21.15.37:0/1260796753 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f3a8406cc30 0x7f3a8406f0e0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:46:57.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:57.853+0000 7f3aa4a00700 1 --2- 172.21.15.37:0/1260796753 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3aa0070360 0x7f3aa0115ba0 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:46:57.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:57.853+0000 7f3aa4a00700 1 --2- 172.21.15.37:0/1260796753 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3aa0070c70 0x7f3aa01160e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:46:57.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:57.853+0000 7f3aa4a00700 1 -- 172.21.15.37:0/1260796753 >> 172.21.15.37:0/1260796753 conn(0x7f3aa00f8930 msgr2=0x7f3aa00f9e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:46:57.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:57.854+0000 7f3aa4a00700 1 -- 172.21.15.37:0/1260796753 shutdown_connections 2024-03-29T08:46:57.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:57.854+0000 7f3aa4a00700 1 -- 172.21.15.37:0/1260796753 wait complete. 2024-03-29T08:46:57.868 INFO:teuthology.orchestra.run.smithi037.stdout:true 2024-03-29T08:46:57.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:57.991+0000 7f8213400700 1 -- 172.21.15.37:0/3894455177 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f820c0fe850 msgr2=0x7f820c106f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:46:57.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:57.991+0000 7f8213400700 1 --2- 172.21.15.37:0/3894455177 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f820c0fe850 0x7f820c106f20 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f81f8004970 tx=0x7f81f8009e30 comp rx=0 tx=0).stop 2024-03-29T08:46:57.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:57.991+0000 7f8213400700 1 -- 172.21.15.37:0/3894455177 shutdown_connections 2024-03-29T08:46:57.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:57.991+0000 7f8213400700 1 --2- 172.21.15.37:0/3894455177 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f820c0fe850 0x7f820c106f20 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:46:57.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:57.991+0000 7f8213400700 1 --2- 172.21.15.37:0/3894455177 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f820c0fdeb0 0x7f820c0fe280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:46:57.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:57.991+0000 7f8213400700 1 -- 172.21.15.37:0/3894455177 >> 172.21.15.37:0/3894455177 conn(0x7f820c072bc0 msgr2=0x7f820c072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:46:57.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:57.992+0000 7f8213400700 1 -- 172.21.15.37:0/3894455177 shutdown_connections 2024-03-29T08:46:57.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:57.992+0000 7f8213400700 1 -- 172.21.15.37:0/3894455177 wait complete. 2024-03-29T08:46:57.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:57.992+0000 7f8213400700 1 Processor -- start 2024-03-29T08:46:57.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:57.993+0000 7f8213400700 1 -- start start 2024-03-29T08:46:57.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:57.993+0000 7f8213400700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f820c0fdeb0 0x7f820c115c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:46:57.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:57.993+0000 7f8213400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f820c0fe850 0x7f820c116140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:46:57.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:57.993+0000 7f8213400700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f820c116790 con 0x7f820c0fe850 2024-03-29T08:46:57.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:57.993+0000 7f8213400700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f820c10fb80 con 0x7f820c0fdeb0 2024-03-29T08:46:57.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:57.993+0000 7f820be00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f820c0fe850 0x7f820c116140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:46:57.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:57.993+0000 7f820be00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f820c0fe850 0x7f820c116140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53536/0 (socket says 172.21.15.37:53536) 2024-03-29T08:46:57.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:57.993+0000 7f820be00700 1 -- 172.21.15.37:0/1602790699 learned_addr learned my addr 172.21.15.37:0/1602790699 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:46:57.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:57.993+0000 7f8210a00700 1 --2- 172.21.15.37:0/1602790699 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f820c0fdeb0 0x7f820c115c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:46:57.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:57.994+0000 7f820be00700 1 -- 172.21.15.37:0/1602790699 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f820c0fdeb0 msgr2=0x7f820c115c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:46:57.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:57.994+0000 7f820be00700 1 --2- 172.21.15.37:0/1602790699 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f820c0fdeb0 0x7f820c115c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:46:57.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:57.994+0000 7f820be00700 1 -- 172.21.15.37:0/1602790699 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81f8005040 con 0x7f820c0fe850 2024-03-29T08:46:57.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:57.994+0000 7f820be00700 1 --2- 172.21.15.37:0/1602790699 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f820c0fe850 0x7f820c116140 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f81f8004f50 tx=0x7f81f8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:46:57.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:57.994+0000 7f8208c00700 1 -- 172.21.15.37:0/1602790699 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f81f800e040 con 0x7f820c0fe850 2024-03-29T08:46:57.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:57.994+0000 7f8208c00700 1 -- 172.21.15.37:0/1602790699 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f81f8013420 con 0x7f820c0fe850 2024-03-29T08:46:57.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:57.994+0000 7f8208c00700 1 -- 172.21.15.37:0/1602790699 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f81f801c630 con 0x7f820c0fe850 2024-03-29T08:46:57.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:57.994+0000 7f8213400700 1 -- 172.21.15.37:0/1602790699 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f820c10fe00 con 0x7f820c0fe850 2024-03-29T08:46:57.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:57.995+0000 7f8213400700 1 -- 172.21.15.37:0/1602790699 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f820c110340 con 0x7f820c0fe850 2024-03-29T08:46:57.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:57.995+0000 7f8213400700 1 -- 172.21.15.37:0/1602790699 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f820c1069f0 con 0x7f820c0fe850 2024-03-29T08:46:57.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:57.996+0000 7f8208c00700 1 -- 172.21.15.37:0/1602790699 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f81f8013590 con 0x7f820c0fe850 2024-03-29T08:46:57.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:57.998+0000 7f8208c00700 1 --2- 172.21.15.37:0/1602790699 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f81f406cc30 0x7f81f406f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:46:57.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:57.998+0000 7f8208c00700 1 -- 172.21.15.37:0/1602790699 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f81f808d190 con 0x7f820c0fe850 2024-03-29T08:46:57.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:57.998+0000 7f8210a00700 1 --2- 172.21.15.37:0/1602790699 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f81f406cc30 0x7f81f406f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:46:57.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:57.998+0000 7f8210a00700 1 --2- 172.21.15.37:0/1602790699 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f81f406cc30 0x7f81f406f0e0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f8200009fd0 tx=0x7f820000b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:46:58.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:58.002+0000 7f8208c00700 1 -- 172.21.15.37:0/1602790699 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f81f805ac60 con 0x7f820c0fe850 2024-03-29T08:46:58.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:58.232+0000 7f8213400700 1 -- 172.21.15.37:0/1602790699 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f820c1111e0 con 0x7f81f406cc30 2024-03-29T08:46:58.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:58.233+0000 7f8208c00700 1 -- 172.21.15.37:0/1602790699 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f820c1111e0 con 0x7f81f406cc30 2024-03-29T08:46:58.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:58.236+0000 7f8213400700 1 -- 172.21.15.37:0/1602790699 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f81f406cc30 msgr2=0x7f81f406f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:46:58.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:58.236+0000 7f8213400700 1 --2- 172.21.15.37:0/1602790699 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f81f406cc30 0x7f81f406f0e0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f8200009fd0 tx=0x7f820000b040 comp rx=0 tx=0).stop 2024-03-29T08:46:58.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:58.236+0000 7f8213400700 1 -- 172.21.15.37:0/1602790699 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f820c0fe850 msgr2=0x7f820c116140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:46:58.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:58.236+0000 7f8213400700 1 --2- 172.21.15.37:0/1602790699 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f820c0fe850 0x7f820c116140 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f81f8004f50 tx=0x7f81f8009e30 comp rx=0 tx=0).stop 2024-03-29T08:46:58.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:58.236+0000 7f8213400700 1 -- 172.21.15.37:0/1602790699 shutdown_connections 2024-03-29T08:46:58.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:58.236+0000 7f8213400700 1 --2- 172.21.15.37:0/1602790699 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f81f406cc30 0x7f81f406f0e0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:46:58.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:58.236+0000 7f8213400700 1 --2- 172.21.15.37:0/1602790699 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f820c0fdeb0 0x7f820c115c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:46:58.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:58.236+0000 7f8213400700 1 --2- 172.21.15.37:0/1602790699 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f820c0fe850 0x7f820c116140 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:46:58.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:58.236+0000 7f8213400700 1 -- 172.21.15.37:0/1602790699 >> 172.21.15.37:0/1602790699 conn(0x7f820c072bc0 msgr2=0x7f820c0fb640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:46:58.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:58.236+0000 7f8213400700 1 -- 172.21.15.37:0/1602790699 shutdown_connections 2024-03-29T08:46:58.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:58.236+0000 7f8213400700 1 -- 172.21.15.37:0/1602790699 wait complete. 2024-03-29T08:46:58.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:58.374+0000 7f38d7c00700 1 -- 172.21.15.37:0/530117866 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f38d0102e00 msgr2=0x7f38d01031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:46:58.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:58.374+0000 7f38d7c00700 1 --2- 172.21.15.37:0/530117866 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f38d0102e00 0x7f38d01031d0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f38c4004940 tx=0x7f38c4009e30 comp rx=0 tx=0).stop 2024-03-29T08:46:58.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:58.374+0000 7f38d7c00700 1 -- 172.21.15.37:0/530117866 shutdown_connections 2024-03-29T08:46:58.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:58.374+0000 7f38d7c00700 1 --2- 172.21.15.37:0/530117866 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f38d01037a0 0x7f38d00fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:46:58.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:58.374+0000 7f38d7c00700 1 --2- 172.21.15.37:0/530117866 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f38d0102e00 0x7f38d01031d0 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:46:58.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:58.374+0000 7f38d7c00700 1 -- 172.21.15.37:0/530117866 >> 172.21.15.37:0/530117866 conn(0x7f38d00f89b0 msgr2=0x7f38d00fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:46:58.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:58.374+0000 7f38d7c00700 1 -- 172.21.15.37:0/530117866 shutdown_connections 2024-03-29T08:46:58.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:58.374+0000 7f38d7c00700 1 -- 172.21.15.37:0/530117866 wait complete. 2024-03-29T08:46:58.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:58.375+0000 7f38d7c00700 1 Processor -- start 2024-03-29T08:46:58.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:58.375+0000 7f38d7c00700 1 -- start start 2024-03-29T08:46:58.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:58.375+0000 7f38d7c00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f38d0102e00 0x7f38d0115cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:46:58.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:58.375+0000 7f38d7c00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f38d01037a0 0x7f38d0116230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:46:58.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:58.375+0000 7f38d7c00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38d0116880 con 0x7f38d0102e00 2024-03-29T08:46:58.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:58.375+0000 7f38d7c00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38d010fc50 con 0x7f38d01037a0 2024-03-29T08:46:58.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:58.376+0000 7f38d5200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f38d0102e00 0x7f38d0115cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:46:58.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:58.376+0000 7f38d5200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f38d0102e00 0x7f38d0115cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53564/0 (socket says 172.21.15.37:53564) 2024-03-29T08:46:58.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:58.376+0000 7f38d5200700 1 -- 172.21.15.37:0/921405373 learned_addr learned my addr 172.21.15.37:0/921405373 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:46:58.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:58.376+0000 7f38cfe00700 1 --2- 172.21.15.37:0/921405373 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f38d01037a0 0x7f38d0116230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:46:58.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:58.376+0000 7f38d5200700 1 -- 172.21.15.37:0/921405373 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f38d01037a0 msgr2=0x7f38d0116230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:46:58.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:58.376+0000 7f38d5200700 1 --2- 172.21.15.37:0/921405373 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f38d01037a0 0x7f38d0116230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:46:58.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:58.376+0000 7f38d5200700 1 -- 172.21.15.37:0/921405373 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38c4005040 con 0x7f38d0102e00 2024-03-29T08:46:58.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:58.376+0000 7f38cfe00700 1 --2- 172.21.15.37:0/921405373 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f38d01037a0 0x7f38d0116230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:46:58.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:58.376+0000 7f38d5200700 1 --2- 172.21.15.37:0/921405373 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f38d0102e00 0x7f38d0115cf0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f38c400b3b0 tx=0x7f38c4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:46:58.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:58.376+0000 7f38ccc00700 1 -- 172.21.15.37:0/921405373 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f38c400e040 con 0x7f38d0102e00 2024-03-29T08:46:58.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:58.377+0000 7f38d7c00700 1 -- 172.21.15.37:0/921405373 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38d010fed0 con 0x7f38d0102e00 2024-03-29T08:46:58.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:58.377+0000 7f38ccc00700 1 -- 172.21.15.37:0/921405373 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f38c4013420 con 0x7f38d0102e00 2024-03-29T08:46:58.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:58.377+0000 7f38d7c00700 1 -- 172.21.15.37:0/921405373 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38d0110490 con 0x7f38d0102e00 2024-03-29T08:46:58.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:58.377+0000 7f38ccc00700 1 -- 172.21.15.37:0/921405373 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f38c40255e0 con 0x7f38d0102e00 2024-03-29T08:46:58.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:58.378+0000 7f38d7c00700 1 -- 172.21.15.37:0/921405373 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38d00fe550 con 0x7f38d0102e00 2024-03-29T08:46:58.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:58.379+0000 7f38ccc00700 1 -- 172.21.15.37:0/921405373 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f38c4013970 con 0x7f38d0102e00 2024-03-29T08:46:58.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:58.380+0000 7f38ccc00700 1 --2- 172.21.15.37:0/921405373 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f38b806cc20 0x7f38b806f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:46:58.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:58.380+0000 7f38ccc00700 1 -- 172.21.15.37:0/921405373 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f38c408d2d0 con 0x7f38d0102e00 2024-03-29T08:46:58.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:58.380+0000 7f38cfe00700 1 --2- 172.21.15.37:0/921405373 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f38b806cc20 0x7f38b806f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:46:58.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:58.380+0000 7f38cfe00700 1 --2- 172.21.15.37:0/921405373 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f38b806cc20 0x7f38b806f0d0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f38bc009fd0 tx=0x7f38bc009ee0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:46:58.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:58.384+0000 7f38ccc00700 1 -- 172.21.15.37:0/921405373 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f38c405ad20 con 0x7f38d0102e00 2024-03-29T08:46:58.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:58.617+0000 7f38d7c00700 1 -- 172.21.15.37:0/921405373 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f38d004bed0 con 0x7f38b806cc20 2024-03-29T08:46:58.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:58.627+0000 7f38ccc00700 1 -- 172.21.15.37:0/921405373 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f38d004bed0 con 0x7f38b806cc20 2024-03-29T08:46:58.627 INFO:teuthology.orchestra.run.smithi037.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-29T08:46:58.627 INFO:teuthology.orchestra.run.smithi037.stdout:alertmanager.smithi037 smithi037 *:9093,9094 running (36m) 13m ago 49m 14.3M - 0.25.0 c8568f914cd2 2f9f9746ed0f 2024-03-29T08:46:58.627 INFO:teuthology.orchestra.run.smithi037.stdout:crash.smithi037 smithi037 running (33m) 13m ago 50m 6647k - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 178d416a3186 2024-03-29T08:46:58.627 INFO:teuthology.orchestra.run.smithi037.stdout:crash.smithi148 smithi148 running (33m) 13m ago 48m 6656k - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 23666989d58f 2024-03-29T08:46:58.627 INFO:teuthology.orchestra.run.smithi037.stdout:grafana.smithi037 smithi037 *:3000 running (36m) 13m ago 49m 74.5M - 9.4.7 954c08fa6188 6b976fe49952 2024-03-29T08:46:58.627 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi037.iifsmh smithi037 running (29m) 13m ago 43m 16.4M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 2d97f562cc5b 2024-03-29T08:46:58.627 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi037.mmwgxe smithi037 running (29m) 13m ago 43m 16.7M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff ec37037f46e8 2024-03-29T08:46:58.627 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi148.qukpjh smithi148 running (13m) 13m ago 43m 10.7M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 3257219bc046 2024-03-29T08:46:58.627 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi148.sqteih smithi148 running (28m) 13m ago 43m 16.3M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 44c15c77037a 2024-03-29T08:46:58.628 INFO:teuthology.orchestra.run.smithi037.stdout:mgr.smithi037.wjikii smithi037 *:8443,9283,8765 running (39m) 13m ago 51m 510M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff c5a1417b9417 2024-03-29T08:46:58.628 INFO:teuthology.orchestra.run.smithi037.stdout:mgr.smithi148.onhgvm smithi148 *:8443,9283,8765 running (37m) 13m ago 48m 456M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff fcfdbce062bd 2024-03-29T08:46:58.628 INFO:teuthology.orchestra.run.smithi037.stdout:mon.smithi037 smithi037 running (35m) 13m ago 51m 70.2M 2048M 18.2.2-1015-gd0e392d9 1cc2f45c17ff eea2060783af 2024-03-29T08:46:58.628 INFO:teuthology.orchestra.run.smithi037.stdout:mon.smithi148 smithi148 running (34m) 13m ago 48m 50.4M 2048M 18.2.2-1015-gd0e392d9 1cc2f45c17ff ad4b2c5748be 2024-03-29T08:46:58.628 INFO:teuthology.orchestra.run.smithi037.stdout:node-exporter.smithi037 smithi037 *:9100 running (37m) 13m ago 49m 8514k - 1.5.0 0da6a335fe13 26b8ae9cb878 2024-03-29T08:46:58.628 INFO:teuthology.orchestra.run.smithi037.stdout:node-exporter.smithi148 smithi148 *:9100 running (37m) 13m ago 48m 8464k - 1.5.0 0da6a335fe13 98414e4e64ed 2024-03-29T08:46:58.628 INFO:teuthology.orchestra.run.smithi037.stdout:osd.0 smithi037 running (33m) 13m ago 47m 108M 1619M 18.2.2-1015-gd0e392d9 1cc2f45c17ff c0c584f6245c 2024-03-29T08:46:58.628 INFO:teuthology.orchestra.run.smithi037.stdout:osd.1 smithi037 running (32m) 13m ago 47m 104M 1619M 18.2.2-1015-gd0e392d9 1cc2f45c17ff a4a2ff4cc83d 2024-03-29T08:46:58.628 INFO:teuthology.orchestra.run.smithi037.stdout:osd.2 smithi037 running (31m) 13m ago 46m 85.9M 1619M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 8b1bc71aa79e 2024-03-29T08:46:58.628 INFO:teuthology.orchestra.run.smithi037.stdout:osd.3 smithi148 running (31m) 13m ago 46m 109M 2642M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 1c656ab8ff86 2024-03-29T08:46:58.628 INFO:teuthology.orchestra.run.smithi037.stdout:osd.4 smithi148 running (30m) 13m ago 45m 101M 2642M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 966a9de11d22 2024-03-29T08:46:58.628 INFO:teuthology.orchestra.run.smithi037.stdout:osd.5 smithi148 running (29m) 13m ago 45m 89.4M 2642M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 474c4f9212ce 2024-03-29T08:46:58.628 INFO:teuthology.orchestra.run.smithi037.stdout:prometheus.smithi037 smithi037 *:9095 running (37m) 13m ago 49m 61.0M - 2.43.0 a07b618ecd1d 30c0223ec589 2024-03-29T08:46:58.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:58.630+0000 7f38d7c00700 1 -- 172.21.15.37:0/921405373 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f38b806cc20 msgr2=0x7f38b806f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:46:58.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:58.630+0000 7f38d7c00700 1 --2- 172.21.15.37:0/921405373 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f38b806cc20 0x7f38b806f0d0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f38bc009fd0 tx=0x7f38bc009ee0 comp rx=0 tx=0).stop 2024-03-29T08:46:58.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:58.630+0000 7f38d7c00700 1 -- 172.21.15.37:0/921405373 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f38d0102e00 msgr2=0x7f38d0115cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:46:58.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:58.630+0000 7f38d7c00700 1 --2- 172.21.15.37:0/921405373 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f38d0102e00 0x7f38d0115cf0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f38c400b3b0 tx=0x7f38c4009e30 comp rx=0 tx=0).stop 2024-03-29T08:46:58.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:58.630+0000 7f38d7c00700 1 -- 172.21.15.37:0/921405373 shutdown_connections 2024-03-29T08:46:58.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:58.630+0000 7f38d7c00700 1 --2- 172.21.15.37:0/921405373 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f38b806cc20 0x7f38b806f0d0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:46:58.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:58.630+0000 7f38d7c00700 1 --2- 172.21.15.37:0/921405373 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f38d0102e00 0x7f38d0115cf0 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:46:58.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:58.630+0000 7f38d7c00700 1 --2- 172.21.15.37:0/921405373 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f38d01037a0 0x7f38d0116230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:46:58.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:58.630+0000 7f38d7c00700 1 -- 172.21.15.37:0/921405373 >> 172.21.15.37:0/921405373 conn(0x7f38d00f89b0 msgr2=0x7f38d00706d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:46:58.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:58.630+0000 7f38d7c00700 1 -- 172.21.15.37:0/921405373 shutdown_connections 2024-03-29T08:46:58.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:58.630+0000 7f38d7c00700 1 -- 172.21.15.37:0/921405373 wait complete. 2024-03-29T08:46:58.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:58.767+0000 7f3ba3400700 1 -- 172.21.15.37:0/2653568030 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3b9c1037a0 msgr2=0x7f3b9c0fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:46:58.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:58.767+0000 7f3ba3400700 1 --2- 172.21.15.37:0/2653568030 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3b9c1037a0 0x7f3b9c0fce10 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f3b88004940 tx=0x7f3b88009e30 comp rx=0 tx=0).stop 2024-03-29T08:46:58.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:58.767+0000 7f3ba3400700 1 -- 172.21.15.37:0/2653568030 shutdown_connections 2024-03-29T08:46:58.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:58.767+0000 7f3ba3400700 1 --2- 172.21.15.37:0/2653568030 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3b9c1037a0 0x7f3b9c0fce10 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:46:58.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:58.767+0000 7f3ba3400700 1 --2- 172.21.15.37:0/2653568030 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3b9c102e00 0x7f3b9c1031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:46:58.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:58.767+0000 7f3ba3400700 1 -- 172.21.15.37:0/2653568030 >> 172.21.15.37:0/2653568030 conn(0x7f3b9c0f89b0 msgr2=0x7f3b9c0fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:46:58.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:58.767+0000 7f3ba3400700 1 -- 172.21.15.37:0/2653568030 shutdown_connections 2024-03-29T08:46:58.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:58.767+0000 7f3ba3400700 1 -- 172.21.15.37:0/2653568030 wait complete. 2024-03-29T08:46:58.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:58.768+0000 7f3ba3400700 1 Processor -- start 2024-03-29T08:46:58.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:58.768+0000 7f3ba3400700 1 -- start start 2024-03-29T08:46:58.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:58.768+0000 7f3ba3400700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3b9c102e00 0x7f3b9c113b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:46:58.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:58.768+0000 7f3ba3400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3b9c1037a0 0x7f3b9c114040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:46:58.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:58.768+0000 7f3ba3400700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b9c114690 con 0x7f3b9c1037a0 2024-03-29T08:46:58.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:58.768+0000 7f3ba3400700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b9c10da80 con 0x7f3b9c102e00 2024-03-29T08:46:58.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:58.768+0000 7f3b9be00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3b9c1037a0 0x7f3b9c114040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:46:58.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:58.768+0000 7f3b9be00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3b9c1037a0 0x7f3b9c114040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53570/0 (socket says 172.21.15.37:53570) 2024-03-29T08:46:58.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:58.768+0000 7f3b9be00700 1 -- 172.21.15.37:0/16693753 learned_addr learned my addr 172.21.15.37:0/16693753 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:46:58.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:58.769+0000 7f3ba0a00700 1 --2- 172.21.15.37:0/16693753 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3b9c102e00 0x7f3b9c113b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:46:58.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:58.769+0000 7f3b9be00700 1 -- 172.21.15.37:0/16693753 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3b9c102e00 msgr2=0x7f3b9c113b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:46:58.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:58.769+0000 7f3b9be00700 1 --2- 172.21.15.37:0/16693753 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3b9c102e00 0x7f3b9c113b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:46:58.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:58.769+0000 7f3b9be00700 1 -- 172.21.15.37:0/16693753 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b88005040 con 0x7f3b9c1037a0 2024-03-29T08:46:58.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:58.769+0000 7f3b9be00700 1 --2- 172.21.15.37:0/16693753 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3b9c1037a0 0x7f3b9c114040 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f3b88009390 tx=0x7f3b88009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:46:58.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:58.769+0000 7f3b98c00700 1 -- 172.21.15.37:0/16693753 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3b8800e040 con 0x7f3b9c1037a0 2024-03-29T08:46:58.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:58.769+0000 7f3b98c00700 1 -- 172.21.15.37:0/16693753 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f3b88013420 con 0x7f3b9c1037a0 2024-03-29T08:46:58.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:58.769+0000 7f3b98c00700 1 -- 172.21.15.37:0/16693753 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3b8801c630 con 0x7f3b9c1037a0 2024-03-29T08:46:58.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:58.769+0000 7f3ba3400700 1 -- 172.21.15.37:0/16693753 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b9c10dd00 con 0x7f3b9c1037a0 2024-03-29T08:46:58.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:58.770+0000 7f3ba3400700 1 -- 172.21.15.37:0/16693753 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b9c10e2c0 con 0x7f3b9c1037a0 2024-03-29T08:46:58.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:58.770+0000 7f3ba3400700 1 -- 172.21.15.37:0/16693753 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b9c0fe550 con 0x7f3b9c1037a0 2024-03-29T08:46:58.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:58.772+0000 7f3b98c00700 1 -- 172.21.15.37:0/16693753 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f3b880139e0 con 0x7f3b9c1037a0 2024-03-29T08:46:58.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:58.774+0000 7f3b98c00700 1 --2- 172.21.15.37:0/16693753 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f3b8406cc30 0x7f3b8406f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:46:58.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:58.774+0000 7f3b98c00700 1 -- 172.21.15.37:0/16693753 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f3b8808d360 con 0x7f3b9c1037a0 2024-03-29T08:46:58.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:58.774+0000 7f3ba0a00700 1 --2- 172.21.15.37:0/16693753 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f3b8406cc30 0x7f3b8406f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:46:58.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:58.775+0000 7f3ba0a00700 1 --2- 172.21.15.37:0/16693753 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f3b8406cc30 0x7f3b8406f0e0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f3b90007fd0 tx=0x7f3b90009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:46:58.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:58.778+0000 7f3b98c00700 1 -- 172.21.15.37:0/16693753 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f3b8805adb0 con 0x7f3b9c1037a0 2024-03-29T08:46:58.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:46:58 smithi037 ceph-mon[142325]: pgmap v1083: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:46:58.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:46:58 smithi037 ceph-mon[142325]: from='client.35934 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:46:59.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:46:58 smithi148 ceph-mon[106692]: pgmap v1083: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:46:59.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:46:58 smithi148 ceph-mon[106692]: from='client.35934 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:46:59.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:59.080+0000 7f3ba3400700 1 -- 172.21.15.37:0/16693753 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f3b9c109da0 con 0x7f3b9c1037a0 2024-03-29T08:46:59.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:59.081+0000 7f3b98c00700 1 -- 172.21.15.37:0/16693753 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7f3b8805a940 con 0x7f3b9c1037a0 2024-03-29T08:46:59.080 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-29T08:46:59.080 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": { 2024-03-29T08:46:59.080 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 2 2024-03-29T08:46:59.080 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:46:59.080 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": { 2024-03-29T08:46:59.081 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 2 2024-03-29T08:46:59.081 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:46:59.081 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": { 2024-03-29T08:46:59.081 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 6 2024-03-29T08:46:59.081 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:46:59.081 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": { 2024-03-29T08:46:59.081 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 4 2024-03-29T08:46:59.081 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:46:59.081 INFO:teuthology.orchestra.run.smithi037.stdout: "overall": { 2024-03-29T08:46:59.081 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 14 2024-03-29T08:46:59.081 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-29T08:46:59.081 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-29T08:46:59.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:59.083+0000 7f3ba3400700 1 -- 172.21.15.37:0/16693753 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f3b8406cc30 msgr2=0x7f3b8406f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:46:59.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:59.083+0000 7f3ba3400700 1 --2- 172.21.15.37:0/16693753 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f3b8406cc30 0x7f3b8406f0e0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f3b90007fd0 tx=0x7f3b90009040 comp rx=0 tx=0).stop 2024-03-29T08:46:59.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:59.083+0000 7f3ba3400700 1 -- 172.21.15.37:0/16693753 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3b9c1037a0 msgr2=0x7f3b9c114040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:46:59.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:59.083+0000 7f3ba3400700 1 --2- 172.21.15.37:0/16693753 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3b9c1037a0 0x7f3b9c114040 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f3b88009390 tx=0x7f3b88009e30 comp rx=0 tx=0).stop 2024-03-29T08:46:59.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:59.083+0000 7f3ba3400700 1 -- 172.21.15.37:0/16693753 shutdown_connections 2024-03-29T08:46:59.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:59.083+0000 7f3ba3400700 1 --2- 172.21.15.37:0/16693753 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f3b8406cc30 0x7f3b8406f0e0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:46:59.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:59.083+0000 7f3ba3400700 1 --2- 172.21.15.37:0/16693753 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3b9c102e00 0x7f3b9c113b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:46:59.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:59.083+0000 7f3ba3400700 1 --2- 172.21.15.37:0/16693753 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3b9c1037a0 0x7f3b9c114040 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:46:59.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:59.083+0000 7f3ba3400700 1 -- 172.21.15.37:0/16693753 >> 172.21.15.37:0/16693753 conn(0x7f3b9c0f89b0 msgr2=0x7f3b9c0f9ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:46:59.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:59.083+0000 7f3ba3400700 1 -- 172.21.15.37:0/16693753 shutdown_connections 2024-03-29T08:46:59.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:59.083+0000 7f3ba3400700 1 -- 172.21.15.37:0/16693753 wait complete. 2024-03-29T08:46:59.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:59.220+0000 7fcbb3e00700 1 -- 172.21.15.37:0/3592618492 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcbb4102ca0 msgr2=0x7fcbb4103070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:46:59.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:59.220+0000 7fcbb3e00700 1 --2- 172.21.15.37:0/3592618492 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcbb4102ca0 0x7fcbb4103070 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7fcba4004940 tx=0x7fcba4009e30 comp rx=0 tx=0).stop 2024-03-29T08:46:59.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:59.220+0000 7fcbb3e00700 1 -- 172.21.15.37:0/3592618492 shutdown_connections 2024-03-29T08:46:59.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:59.220+0000 7fcbb3e00700 1 --2- 172.21.15.37:0/3592618492 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcbb4065630 0x7fcbb4065a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:46:59.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:59.220+0000 7fcbb3e00700 1 --2- 172.21.15.37:0/3592618492 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcbb4102ca0 0x7fcbb4103070 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:46:59.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:59.220+0000 7fcbb3e00700 1 -- 172.21.15.37:0/3592618492 >> 172.21.15.37:0/3592618492 conn(0x7fcbb4071b70 msgr2=0x7fcbb4071f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:46:59.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:59.220+0000 7fcbb3e00700 1 -- 172.21.15.37:0/3592618492 shutdown_connections 2024-03-29T08:46:59.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:59.220+0000 7fcbb3e00700 1 -- 172.21.15.37:0/3592618492 wait complete. 2024-03-29T08:46:59.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:59.221+0000 7fcbb3e00700 1 Processor -- start 2024-03-29T08:46:59.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:59.221+0000 7fcbb3e00700 1 -- start start 2024-03-29T08:46:59.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:59.222+0000 7fcbb3e00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcbb4065630 0x7fcbb4113830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:46:59.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:59.222+0000 7fcbb3e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcbb4102ca0 0x7fcbb4113d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:46:59.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:59.222+0000 7fcbb3e00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbb41143c0 con 0x7fcbb4102ca0 2024-03-29T08:46:59.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:59.222+0000 7fcbb3e00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbb410d7b0 con 0x7fcbb4065630 2024-03-29T08:46:59.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:59.222+0000 7fcbb2000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcbb4102ca0 0x7fcbb4113d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:46:59.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:59.222+0000 7fcbb2000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcbb4102ca0 0x7fcbb4113d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53588/0 (socket says 172.21.15.37:53588) 2024-03-29T08:46:59.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:59.222+0000 7fcbb2000700 1 -- 172.21.15.37:0/223833381 learned_addr learned my addr 172.21.15.37:0/223833381 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:46:59.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:59.222+0000 7fcbb2a00700 1 --2- 172.21.15.37:0/223833381 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcbb4065630 0x7fcbb4113830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:46:59.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:59.223+0000 7fcbb2000700 1 -- 172.21.15.37:0/223833381 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcbb4065630 msgr2=0x7fcbb4113830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:46:59.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:59.223+0000 7fcbb2000700 1 --2- 172.21.15.37:0/223833381 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcbb4065630 0x7fcbb4113830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:46:59.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:59.223+0000 7fcbb2000700 1 -- 172.21.15.37:0/223833381 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcba4005040 con 0x7fcbb4102ca0 2024-03-29T08:46:59.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:59.223+0000 7fcbb2000700 1 --2- 172.21.15.37:0/223833381 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcbb4102ca0 0x7fcbb4113d70 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7fcba000eb70 tx=0x7fcba000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:46:59.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:59.223+0000 7fcbaaa00700 1 -- 172.21.15.37:0/223833381 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcba0007cc0 con 0x7fcbb4102ca0 2024-03-29T08:46:59.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:59.223+0000 7fcbaaa00700 1 -- 172.21.15.37:0/223833381 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fcba000f040 con 0x7fcbb4102ca0 2024-03-29T08:46:59.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:59.223+0000 7fcbaaa00700 1 -- 172.21.15.37:0/223833381 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcba000c9f0 con 0x7fcbb4102ca0 2024-03-29T08:46:59.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:59.223+0000 7fcbb3e00700 1 -- 172.21.15.37:0/223833381 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbb410da30 con 0x7fcbb4102ca0 2024-03-29T08:46:59.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:59.223+0000 7fcbb3e00700 1 -- 172.21.15.37:0/223833381 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbb410dff0 con 0x7fcbb4102ca0 2024-03-29T08:46:59.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:59.224+0000 7fcbb3e00700 1 -- 172.21.15.37:0/223833381 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcbb40fe380 con 0x7fcbb4102ca0 2024-03-29T08:46:59.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:59.225+0000 7fcbaaa00700 1 -- 172.21.15.37:0/223833381 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fcba000cb50 con 0x7fcbb4102ca0 2024-03-29T08:46:59.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:59.226+0000 7fcbaaa00700 1 --2- 172.21.15.37:0/223833381 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fcb9806cc30 0x7fcb9806f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:46:59.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:59.226+0000 7fcbaaa00700 1 -- 172.21.15.37:0/223833381 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fcba008bbe0 con 0x7fcbb4102ca0 2024-03-29T08:46:59.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:59.227+0000 7fcbb2a00700 1 --2- 172.21.15.37:0/223833381 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fcb9806cc30 0x7fcb9806f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:46:59.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:59.227+0000 7fcbb2a00700 1 --2- 172.21.15.37:0/223833381 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fcb9806cc30 0x7fcb9806f0e0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fcba40045a0 tx=0x7fcba4009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:46:59.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:59.231+0000 7fcbaaa00700 1 -- 172.21.15.37:0/223833381 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fcba0059630 con 0x7fcbb4102ca0 2024-03-29T08:46:59.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:59.488+0000 7fcbb3e00700 1 -- 172.21.15.37:0/223833381 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fcbb4109bc0 con 0x7fcbb4102ca0 2024-03-29T08:46:59.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:59.488+0000 7fcbaaa00700 1 -- 172.21.15.37:0/223833381 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 30 v30) v1 ==== 76+0+1841 (secure 0 0 0) 0x7fcba00591c0 con 0x7fcbb4102ca0 2024-03-29T08:46:59.490 INFO:teuthology.orchestra.run.smithi037.stdout:e30 2024-03-29T08:46:59.490 INFO:teuthology.orchestra.run.smithi037.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-29T08:46:59.490 INFO:teuthology.orchestra.run.smithi037.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T08:46:59.490 INFO:teuthology.orchestra.run.smithi037.stdout:legacy client fscid: 1 2024-03-29T08:46:59.491 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:46:59.491 INFO:teuthology.orchestra.run.smithi037.stdout:Filesystem 'cephfs' (1) 2024-03-29T08:46:59.491 INFO:teuthology.orchestra.run.smithi037.stdout:fs_name cephfs 2024-03-29T08:46:59.491 INFO:teuthology.orchestra.run.smithi037.stdout:epoch 21 2024-03-29T08:46:59.491 INFO:teuthology.orchestra.run.smithi037.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-03-29T08:46:59.491 INFO:teuthology.orchestra.run.smithi037.stdout:created 2024-03-29T08:03:29.241269+0000 2024-03-29T08:46:59.491 INFO:teuthology.orchestra.run.smithi037.stdout:modified 2024-03-29T08:17:16.237238+0000 2024-03-29T08:46:59.491 INFO:teuthology.orchestra.run.smithi037.stdout:tableserver 0 2024-03-29T08:46:59.491 INFO:teuthology.orchestra.run.smithi037.stdout:root 0 2024-03-29T08:46:59.491 INFO:teuthology.orchestra.run.smithi037.stdout:session_timeout 60 2024-03-29T08:46:59.491 INFO:teuthology.orchestra.run.smithi037.stdout:session_autoclose 300 2024-03-29T08:46:59.491 INFO:teuthology.orchestra.run.smithi037.stdout:max_file_size 1099511627776 2024-03-29T08:46:59.491 INFO:teuthology.orchestra.run.smithi037.stdout:max_xattr_size 65536 2024-03-29T08:46:59.491 INFO:teuthology.orchestra.run.smithi037.stdout:required_client_features {} 2024-03-29T08:46:59.491 INFO:teuthology.orchestra.run.smithi037.stdout:last_failure 0 2024-03-29T08:46:59.492 INFO:teuthology.orchestra.run.smithi037.stdout:last_failure_osd_epoch 82 2024-03-29T08:46:59.492 INFO:teuthology.orchestra.run.smithi037.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T08:46:59.492 INFO:teuthology.orchestra.run.smithi037.stdout:max_mds 2 2024-03-29T08:46:59.492 INFO:teuthology.orchestra.run.smithi037.stdout:in 0,1 2024-03-29T08:46:59.492 INFO:teuthology.orchestra.run.smithi037.stdout:up {} 2024-03-29T08:46:59.492 INFO:teuthology.orchestra.run.smithi037.stdout:failed 0,1 2024-03-29T08:46:59.492 INFO:teuthology.orchestra.run.smithi037.stdout:damaged 2024-03-29T08:46:59.492 INFO:teuthology.orchestra.run.smithi037.stdout:stopped 2024-03-29T08:46:59.492 INFO:teuthology.orchestra.run.smithi037.stdout:data_pools [3] 2024-03-29T08:46:59.492 INFO:teuthology.orchestra.run.smithi037.stdout:metadata_pool 2 2024-03-29T08:46:59.492 INFO:teuthology.orchestra.run.smithi037.stdout:inline_data disabled 2024-03-29T08:46:59.492 INFO:teuthology.orchestra.run.smithi037.stdout:balancer 2024-03-29T08:46:59.492 INFO:teuthology.orchestra.run.smithi037.stdout:bal_rank_mask -1 2024-03-29T08:46:59.492 INFO:teuthology.orchestra.run.smithi037.stdout:standby_count_wanted 1 2024-03-29T08:46:59.492 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:46:59.492 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:46:59.492 INFO:teuthology.orchestra.run.smithi037.stdout:Standby daemons: 2024-03-29T08:46:59.493 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:46:59.493 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi037.mmwgxe{-1:34440} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.37:6828/1248142005,v1:172.21.15.37:6829/1248142005] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:46:59.493 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi037.iifsmh{-1:34444} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.37:6826/2060882959,v1:172.21.15.37:6827/2060882959] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:46:59.493 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi148.sqteih{-1:44395} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.148:6824/4078652242,v1:172.21.15.148:6825/4078652242] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:46:59.493 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi148.qukpjh{-1:45007} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.148:6826/2394751105,v1:172.21.15.148:6827/2394751105] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:46:59.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:59.490+0000 7fcbb3e00700 1 -- 172.21.15.37:0/223833381 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fcb9806cc30 msgr2=0x7fcb9806f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:46:59.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:59.491+0000 7fcbb3e00700 1 --2- 172.21.15.37:0/223833381 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fcb9806cc30 0x7fcb9806f0e0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fcba40045a0 tx=0x7fcba4009e30 comp rx=0 tx=0).stop 2024-03-29T08:46:59.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:59.491+0000 7fcbb3e00700 1 -- 172.21.15.37:0/223833381 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcbb4102ca0 msgr2=0x7fcbb4113d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:46:59.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:59.491+0000 7fcbb3e00700 1 --2- 172.21.15.37:0/223833381 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcbb4102ca0 0x7fcbb4113d70 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7fcba000eb70 tx=0x7fcba000b040 comp rx=0 tx=0).stop 2024-03-29T08:46:59.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:59.491+0000 7fcbb3e00700 1 -- 172.21.15.37:0/223833381 shutdown_connections 2024-03-29T08:46:59.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:59.491+0000 7fcbb3e00700 1 --2- 172.21.15.37:0/223833381 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fcb9806cc30 0x7fcb9806f0e0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:46:59.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:59.491+0000 7fcbb3e00700 1 --2- 172.21.15.37:0/223833381 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcbb4065630 0x7fcbb4113830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:46:59.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:59.491+0000 7fcbb3e00700 1 --2- 172.21.15.37:0/223833381 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcbb4102ca0 0x7fcbb4113d70 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:46:59.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:59.491+0000 7fcbb3e00700 1 -- 172.21.15.37:0/223833381 >> 172.21.15.37:0/223833381 conn(0x7fcbb4071b70 msgr2=0x7fcbb41059d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:46:59.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:59.491+0000 7fcbb3e00700 1 -- 172.21.15.37:0/223833381 shutdown_connections 2024-03-29T08:46:59.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:59.491+0000 7fcbb3e00700 1 -- 172.21.15.37:0/223833381 wait complete. 2024-03-29T08:46:59.494 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 30 2024-03-29T08:46:59.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:59.630+0000 7faa99c00700 1 -- 172.21.15.37:0/4210629199 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faa94102e00 msgr2=0x7faa941031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:46:59.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:59.630+0000 7faa99c00700 1 --2- 172.21.15.37:0/4210629199 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faa94102e00 0x7faa941031d0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7faa84004940 tx=0x7faa84009e30 comp rx=0 tx=0).stop 2024-03-29T08:46:59.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:59.630+0000 7faa99c00700 1 -- 172.21.15.37:0/4210629199 shutdown_connections 2024-03-29T08:46:59.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:59.630+0000 7faa99c00700 1 --2- 172.21.15.37:0/4210629199 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faa941037a0 0x7faa940fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:46:59.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:59.630+0000 7faa99c00700 1 --2- 172.21.15.37:0/4210629199 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faa94102e00 0x7faa941031d0 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:46:59.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:59.630+0000 7faa99c00700 1 -- 172.21.15.37:0/4210629199 >> 172.21.15.37:0/4210629199 conn(0x7faa940f89b0 msgr2=0x7faa940fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:46:59.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:59.630+0000 7faa99c00700 1 -- 172.21.15.37:0/4210629199 shutdown_connections 2024-03-29T08:46:59.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:59.630+0000 7faa99c00700 1 -- 172.21.15.37:0/4210629199 wait complete. 2024-03-29T08:46:59.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:59.631+0000 7faa99c00700 1 Processor -- start 2024-03-29T08:46:59.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:59.631+0000 7faa99c00700 1 -- start start 2024-03-29T08:46:59.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:59.632+0000 7faa99c00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faa94102e00 0x7faa94115cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:46:59.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:59.632+0000 7faa99c00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faa941037a0 0x7faa94116230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:46:59.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:59.632+0000 7faa99c00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa94116880 con 0x7faa941037a0 2024-03-29T08:46:59.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:59.632+0000 7faa99c00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa9410fc50 con 0x7faa94102e00 2024-03-29T08:46:59.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:59.632+0000 7faa92000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faa941037a0 0x7faa94116230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:46:59.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:59.632+0000 7faa92000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faa941037a0 0x7faa94116230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53600/0 (socket says 172.21.15.37:53600) 2024-03-29T08:46:59.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:59.632+0000 7faa92000700 1 -- 172.21.15.37:0/569023949 learned_addr learned my addr 172.21.15.37:0/569023949 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:46:59.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:59.632+0000 7faa92a00700 1 --2- 172.21.15.37:0/569023949 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faa94102e00 0x7faa94115cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:46:59.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:59.632+0000 7faa92000700 1 -- 172.21.15.37:0/569023949 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faa94102e00 msgr2=0x7faa94115cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:46:59.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:59.632+0000 7faa92000700 1 --2- 172.21.15.37:0/569023949 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faa94102e00 0x7faa94115cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:46:59.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:59.632+0000 7faa92000700 1 -- 172.21.15.37:0/569023949 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa84005040 con 0x7faa941037a0 2024-03-29T08:46:59.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:59.633+0000 7faa92000700 1 --2- 172.21.15.37:0/569023949 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faa941037a0 0x7faa94116230 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7faa80002a50 tx=0x7faa80004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:46:59.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:59.633+0000 7faa8aa00700 1 -- 172.21.15.37:0/569023949 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faa80007cc0 con 0x7faa941037a0 2024-03-29T08:46:59.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:59.633+0000 7faa99c00700 1 -- 172.21.15.37:0/569023949 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa9410ff30 con 0x7faa941037a0 2024-03-29T08:46:59.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:59.633+0000 7faa8aa00700 1 -- 172.21.15.37:0/569023949 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7faa8000f040 con 0x7faa941037a0 2024-03-29T08:46:59.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:59.633+0000 7faa8aa00700 1 -- 172.21.15.37:0/569023949 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faa8000d920 con 0x7faa941037a0 2024-03-29T08:46:59.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:59.633+0000 7faa99c00700 1 -- 172.21.15.37:0/569023949 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa941104f0 con 0x7faa941037a0 2024-03-29T08:46:59.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:59.634+0000 7faa99c00700 1 -- 172.21.15.37:0/569023949 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa940fe550 con 0x7faa941037a0 2024-03-29T08:46:59.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:59.635+0000 7faa8aa00700 1 -- 172.21.15.37:0/569023949 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7faa80009680 con 0x7faa941037a0 2024-03-29T08:46:59.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:59.636+0000 7faa8aa00700 1 --2- 172.21.15.37:0/569023949 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7faa7806cc30 0x7faa7806f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:46:59.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:59.636+0000 7faa8aa00700 1 -- 172.21.15.37:0/569023949 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7faa8008bd30 con 0x7faa941037a0 2024-03-29T08:46:59.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:59.637+0000 7faa92a00700 1 --2- 172.21.15.37:0/569023949 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7faa7806cc30 0x7faa7806f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:46:59.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:59.637+0000 7faa92a00700 1 --2- 172.21.15.37:0/569023949 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7faa7806cc30 0x7faa7806f0e0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7faa84000c00 tx=0x7faa84009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:46:59.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:59.642+0000 7faa8aa00700 1 -- 172.21.15.37:0/569023949 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7faa80059780 con 0x7faa941037a0 2024-03-29T08:46:59.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:59.877+0000 7faa99c00700 1 -- 172.21.15.37:0/569023949 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7faa94109e30 con 0x7faa7806cc30 2024-03-29T08:46:59.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:59.878+0000 7faa8aa00700 1 -- 172.21.15.37:0/569023949 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7faa94109e30 con 0x7faa7806cc30 2024-03-29T08:46:59.878 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-29T08:46:59.878 INFO:teuthology.orchestra.run.smithi037.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59", 2024-03-29T08:46:59.879 INFO:teuthology.orchestra.run.smithi037.stdout: "in_progress": true, 2024-03-29T08:46:59.879 INFO:teuthology.orchestra.run.smithi037.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-29T08:46:59.879 INFO:teuthology.orchestra.run.smithi037.stdout: "services_complete": [ 2024-03-29T08:46:59.879 INFO:teuthology.orchestra.run.smithi037.stdout: "crash", 2024-03-29T08:46:59.879 INFO:teuthology.orchestra.run.smithi037.stdout: "osd", 2024-03-29T08:46:59.880 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr", 2024-03-29T08:46:59.880 INFO:teuthology.orchestra.run.smithi037.stdout: "mds", 2024-03-29T08:46:59.880 INFO:teuthology.orchestra.run.smithi037.stdout: "mon" 2024-03-29T08:46:59.880 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-29T08:46:59.880 INFO:teuthology.orchestra.run.smithi037.stdout: "progress": "16/21 daemons upgraded", 2024-03-29T08:46:59.880 INFO:teuthology.orchestra.run.smithi037.stdout: "message": "Currently upgrading mds daemons", 2024-03-29T08:46:59.881 INFO:teuthology.orchestra.run.smithi037.stdout: "is_paused": false 2024-03-29T08:46:59.881 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-29T08:46:59.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:59.881+0000 7faa99c00700 1 -- 172.21.15.37:0/569023949 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7faa7806cc30 msgr2=0x7faa7806f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:46:59.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:59.881+0000 7faa99c00700 1 --2- 172.21.15.37:0/569023949 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7faa7806cc30 0x7faa7806f0e0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7faa84000c00 tx=0x7faa84009e30 comp rx=0 tx=0).stop 2024-03-29T08:46:59.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:59.881+0000 7faa99c00700 1 -- 172.21.15.37:0/569023949 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faa941037a0 msgr2=0x7faa94116230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:46:59.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:59.881+0000 7faa99c00700 1 --2- 172.21.15.37:0/569023949 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faa941037a0 0x7faa94116230 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7faa80002a50 tx=0x7faa80004600 comp rx=0 tx=0).stop 2024-03-29T08:46:59.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:59.881+0000 7faa99c00700 1 -- 172.21.15.37:0/569023949 shutdown_connections 2024-03-29T08:46:59.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:59.881+0000 7faa99c00700 1 --2- 172.21.15.37:0/569023949 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7faa7806cc30 0x7faa7806f0e0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:46:59.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:59.881+0000 7faa99c00700 1 --2- 172.21.15.37:0/569023949 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faa94102e00 0x7faa94115cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:46:59.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:59.881+0000 7faa99c00700 1 --2- 172.21.15.37:0/569023949 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faa941037a0 0x7faa94116230 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:46:59.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:59.881+0000 7faa99c00700 1 -- 172.21.15.37:0/569023949 >> 172.21.15.37:0/569023949 conn(0x7faa940f89b0 msgr2=0x7faa940706d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:46:59.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:59.881+0000 7faa99c00700 1 -- 172.21.15.37:0/569023949 shutdown_connections 2024-03-29T08:46:59.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:46:59.881+0000 7faa99c00700 1 -- 172.21.15.37:0/569023949 wait complete. 2024-03-29T08:46:59.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:46:59 smithi037 ceph-mon[142325]: from='client.35938 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:46:59.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:46:59 smithi037 ceph-mon[142325]: from='client.35942 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:46:59.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:46:59 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/16693753' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:46:59.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:46:59 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/223833381' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T08:47:00.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:00.020+0000 7fac5e400700 1 -- 172.21.15.37:0/3877462557 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fac58070c10 msgr2=0x7fac58108fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:47:00.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:00.020+0000 7fac5e400700 1 --2- 172.21.15.37:0/3877462557 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fac58070c10 0x7fac58108fa0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7fac44004970 tx=0x7fac44009e30 comp rx=0 tx=0).stop 2024-03-29T08:47:00.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:00.020+0000 7fac5e400700 1 -- 172.21.15.37:0/3877462557 shutdown_connections 2024-03-29T08:47:00.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:00.020+0000 7fac5e400700 1 --2- 172.21.15.37:0/3877462557 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fac58070c10 0x7fac58108fa0 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:47:00.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:00.020+0000 7fac5e400700 1 --2- 172.21.15.37:0/3877462557 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fac58070300 0x7fac580706d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:47:00.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:00.020+0000 7fac5e400700 1 -- 172.21.15.37:0/3877462557 >> 172.21.15.37:0/3877462557 conn(0x7fac580f8760 msgr2=0x7fac580fab70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:47:00.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:00.020+0000 7fac5e400700 1 -- 172.21.15.37:0/3877462557 shutdown_connections 2024-03-29T08:47:00.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:00.021+0000 7fac5e400700 1 -- 172.21.15.37:0/3877462557 wait complete. 2024-03-29T08:47:00.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:00.021+0000 7fac5e400700 1 Processor -- start 2024-03-29T08:47:00.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:00.021+0000 7fac5e400700 1 -- start start 2024-03-29T08:47:00.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:00.022+0000 7fac5e400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fac58070300 0x7fac58115ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:47:00.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:00.022+0000 7fac5e400700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fac58070c10 0x7fac58116010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:47:00.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:00.022+0000 7fac5e400700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac58116660 con 0x7fac58070300 2024-03-29T08:47:00.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:00.022+0000 7fac5e400700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac5810fa50 con 0x7fac58070c10 2024-03-29T08:47:00.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:00.022+0000 7fac5d000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fac58070300 0x7fac58115ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:47:00.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:00.022+0000 7fac5d000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fac58070300 0x7fac58115ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54096/0 (socket says 172.21.15.37:54096) 2024-03-29T08:47:00.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:00.022+0000 7fac5d000700 1 -- 172.21.15.37:0/364968605 learned_addr learned my addr 172.21.15.37:0/364968605 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:47:00.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:00.022+0000 7fac57e00700 1 --2- 172.21.15.37:0/364968605 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fac58070c10 0x7fac58116010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:47:00.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:00.022+0000 7fac5d000700 1 -- 172.21.15.37:0/364968605 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fac58070c10 msgr2=0x7fac58116010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:47:00.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:00.022+0000 7fac5d000700 1 --2- 172.21.15.37:0/364968605 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fac58070c10 0x7fac58116010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:47:00.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:00.022+0000 7fac5d000700 1 -- 172.21.15.37:0/364968605 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac44005040 con 0x7fac58070300 2024-03-29T08:47:00.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:00.023+0000 7fac5d000700 1 --2- 172.21.15.37:0/364968605 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fac58070300 0x7fac58115ad0 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7fac4c00eb40 tx=0x7fac4c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:47:00.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:00.023+0000 7fac54c00700 1 -- 172.21.15.37:0/364968605 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fac4c007cc0 con 0x7fac58070300 2024-03-29T08:47:00.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:00.023+0000 7fac54c00700 1 -- 172.21.15.37:0/364968605 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fac4c00f040 con 0x7fac58070300 2024-03-29T08:47:00.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:00.023+0000 7fac5e400700 1 -- 172.21.15.37:0/364968605 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac5810fcd0 con 0x7fac58070300 2024-03-29T08:47:00.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:00.023+0000 7fac54c00700 1 -- 172.21.15.37:0/364968605 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fac4c00c990 con 0x7fac58070300 2024-03-29T08:47:00.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:00.023+0000 7fac5e400700 1 -- 172.21.15.37:0/364968605 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac581101e0 con 0x7fac58070300 2024-03-29T08:47:00.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:00.024+0000 7fac5e400700 1 -- 172.21.15.37:0/364968605 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac58071a40 con 0x7fac58070300 2024-03-29T08:47:00.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:00.025+0000 7fac54c00700 1 -- 172.21.15.37:0/364968605 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fac4c00caf0 con 0x7fac58070300 2024-03-29T08:47:00.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:00.026+0000 7fac54c00700 1 --2- 172.21.15.37:0/364968605 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fac4006cc20 0x7fac4006f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:47:00.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:00.026+0000 7fac54c00700 1 -- 172.21.15.37:0/364968605 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fac4c08bc00 con 0x7fac58070300 2024-03-29T08:47:00.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:00.027+0000 7fac57e00700 1 --2- 172.21.15.37:0/364968605 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fac4006cc20 0x7fac4006f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:47:00.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:00.027+0000 7fac57e00700 1 --2- 172.21.15.37:0/364968605 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fac4006cc20 0x7fac4006f0d0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fac58110d30 tx=0x7fac44009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:47:00.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:00.031+0000 7fac54c00700 1 -- 172.21.15.37:0/364968605 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fac4c059700 con 0x7fac58070300 2024-03-29T08:47:00.055 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:46:59 smithi148 ceph-mon[106692]: from='client.35938 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:47:00.055 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:46:59 smithi148 ceph-mon[106692]: from='client.35942 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:47:00.055 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:46:59 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/16693753' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:47:00.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:46:59 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/223833381' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T08:47:00.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:00.330+0000 7fac5e400700 1 -- 172.21.15.37:0/364968605 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fac5810be80 con 0x7fac58070300 2024-03-29T08:47:00.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:00.330+0000 7fac54c00700 1 -- 172.21.15.37:0/364968605 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+3583 (secure 0 0 0) 0x7fac4c019e10 con 0x7fac58070300 2024-03-29T08:47:00.331 INFO:teuthology.orchestra.run.smithi037.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem is offline 2024-03-29T08:47:00.331 INFO:teuthology.orchestra.run.smithi037.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-03-29T08:47:00.331 INFO:teuthology.orchestra.run.smithi037.stdout: host smithi148 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/mon.smithi148/config 2024-03-29T08:47:00.331 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/podman: timeout after 895 seconds 2024-03-29T08:47:00.331 INFO:teuthology.orchestra.run.smithi037.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59 -e NODE_NAME=smithi148 -e CEPH_USE_RANDOM_NONCE=1 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4:/var/run/ceph:z -v /var/log/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4:/var/log/ceph:z -v /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpkx2rz2d4:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59 inventory --format=json-pretty --filter-for-batch 2024-03-29T08:47:00.331 INFO:teuthology.orchestra.run.smithi037.stdout:Traceback (most recent call last): 2024-03-29T08:47:00.331 INFO:teuthology.orchestra.run.smithi037.stdout: File "/usr/lib64/python3.6/runpy.py", line 193, in _run_module_as_main 2024-03-29T08:47:00.331 INFO:teuthology.orchestra.run.smithi037.stdout: "__main__", mod_spec) 2024-03-29T08:47:00.332 INFO:teuthology.orchestra.run.smithi037.stdout: File "/usr/lib64/python3.6/runpy.py", line 85, in _run_code 2024-03-29T08:47:00.332 INFO:teuthology.orchestra.run.smithi037.stdout: exec(code, run_globals) 2024-03-29T08:47:00.332 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 10889, in 2024-03-29T08:47:00.332 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 10877, in main 2024-03-29T08:47:00.332 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 2576, in _infer_config 2024-03-29T08:47:00.332 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 2492, in _infer_fsid 2024-03-29T08:47:00.332 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 2604, in _infer_image 2024-03-29T08:47:00.332 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 2479, in _validate_fsid 2024-03-29T08:47:00.332 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 7145, in command_ceph_volume 2024-03-29T08:47:00.332 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 2267, in call_throws 2024-03-29T08:47:00.332 INFO:teuthology.orchestra.run.smithi037.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59 -e NODE_NAME=smithi148 -e CEPH_USE_RANDOM_NONCE=1 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4:/var/run/ceph:z -v /var/log/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4:/var/log/ceph:z -v /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpkx2rz2d4:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59 inventory --format=json-pretty --filter-for-batch 2024-03-29T08:47:00.332 INFO:teuthology.orchestra.run.smithi037.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-03-29T08:47:00.332 INFO:teuthology.orchestra.run.smithi037.stdout: fs cephfs is degraded 2024-03-29T08:47:00.332 INFO:teuthology.orchestra.run.smithi037.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-03-29T08:47:00.332 INFO:teuthology.orchestra.run.smithi037.stdout: fs cephfs is offline because no MDS is active for it. 2024-03-29T08:47:00.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:00.333+0000 7fac5e400700 1 -- 172.21.15.37:0/364968605 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fac4006cc20 msgr2=0x7fac4006f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:47:00.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:00.333+0000 7fac5e400700 1 --2- 172.21.15.37:0/364968605 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fac4006cc20 0x7fac4006f0d0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fac58110d30 tx=0x7fac44009e30 comp rx=0 tx=0).stop 2024-03-29T08:47:00.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:00.333+0000 7fac5e400700 1 -- 172.21.15.37:0/364968605 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fac58070300 msgr2=0x7fac58115ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:47:00.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:00.333+0000 7fac5e400700 1 --2- 172.21.15.37:0/364968605 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fac58070300 0x7fac58115ad0 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7fac4c00eb40 tx=0x7fac4c00b040 comp rx=0 tx=0).stop 2024-03-29T08:47:00.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:00.333+0000 7fac5e400700 1 -- 172.21.15.37:0/364968605 shutdown_connections 2024-03-29T08:47:00.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:00.333+0000 7fac5e400700 1 --2- 172.21.15.37:0/364968605 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fac4006cc20 0x7fac4006f0d0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:47:00.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:00.333+0000 7fac5e400700 1 --2- 172.21.15.37:0/364968605 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fac58070300 0x7fac58115ad0 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:47:00.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:00.333+0000 7fac5e400700 1 --2- 172.21.15.37:0/364968605 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fac58070c10 0x7fac58116010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:47:00.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:00.333+0000 7fac5e400700 1 -- 172.21.15.37:0/364968605 >> 172.21.15.37:0/364968605 conn(0x7fac580f8760 msgr2=0x7fac580f9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:47:00.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:00.333+0000 7fac5e400700 1 -- 172.21.15.37:0/364968605 shutdown_connections 2024-03-29T08:47:00.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:00.333+0000 7fac5e400700 1 -- 172.21.15.37:0/364968605 wait complete. 2024-03-29T08:47:00.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:47:00 smithi037 ceph-mon[142325]: pgmap v1084: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:47:00.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:47:00 smithi037 ceph-mon[142325]: from='client.35954 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:47:00.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:47:00 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/364968605' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T08:47:01.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:47:00 smithi148 ceph-mon[106692]: pgmap v1084: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:47:01.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:47:00 smithi148 ceph-mon[106692]: from='client.35954 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:47:01.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:47:00 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/364968605' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T08:47:02.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:47:02 smithi037 ceph-mon[142325]: pgmap v1085: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:47:03.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:47:02 smithi148 ceph-mon[106692]: pgmap v1085: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:47:04.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:47:04 smithi037 ceph-mon[142325]: pgmap v1086: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:47:05.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:47:04 smithi148 ceph-mon[106692]: pgmap v1086: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:47:06.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:47:06 smithi037 ceph-mon[142325]: pgmap v1087: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:47:07.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:47:06 smithi148 ceph-mon[106692]: pgmap v1087: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:47:08.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:47:08 smithi037 ceph-mon[142325]: pgmap v1088: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:47:09.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:47:08 smithi148 ceph-mon[106692]: pgmap v1088: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:47:10.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:47:10 smithi037 ceph-mon[142325]: pgmap v1089: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:47:10.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:47:10 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:47:11.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:47:10 smithi148 ceph-mon[106692]: pgmap v1089: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:47:11.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:47:10 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:47:12.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:47:12 smithi037 ceph-mon[142325]: pgmap v1090: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:47:13.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:47:12 smithi148 ceph-mon[106692]: pgmap v1090: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:47:14.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:47:14 smithi037 ceph-mon[142325]: pgmap v1091: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:47:15.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:47:14 smithi148 ceph-mon[106692]: pgmap v1091: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:47:16.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:47:16 smithi037 ceph-mon[142325]: pgmap v1092: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:47:17.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:47:16 smithi148 ceph-mon[106692]: pgmap v1092: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:47:18.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:47:18 smithi037 ceph-mon[142325]: pgmap v1093: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:47:19.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:47:18 smithi148 ceph-mon[106692]: pgmap v1093: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:47:20.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:47:20 smithi037 ceph-mon[142325]: pgmap v1094: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:47:21.055 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:47:20 smithi148 ceph-mon[106692]: pgmap v1094: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:47:22.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:47:22 smithi037 ceph-mon[142325]: pgmap v1095: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:47:23.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:47:22 smithi148 ceph-mon[106692]: pgmap v1095: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:47:24.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:47:24 smithi037 ceph-mon[142325]: pgmap v1096: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:47:24.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:47:24 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:47:25.055 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:47:24 smithi148 ceph-mon[106692]: pgmap v1096: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:47:25.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:47:24 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:47:26.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:47:26 smithi037 ceph-mon[142325]: pgmap v1097: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:47:27.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:47:26 smithi148 ceph-mon[106692]: pgmap v1097: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:47:28.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:47:28 smithi037 ceph-mon[142325]: pgmap v1098: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:47:29.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:47:28 smithi148 ceph-mon[106692]: pgmap v1098: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:47:30.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:30.487+0000 7f8845a00700 1 -- 172.21.15.37:0/335126940 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f88401037a0 msgr2=0x7f88400fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:47:30.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:30.487+0000 7f8845a00700 1 --2- 172.21.15.37:0/335126940 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f88401037a0 0x7f88400fce10 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f882c004940 tx=0x7f882c009e30 comp rx=0 tx=0).stop 2024-03-29T08:47:30.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:30.487+0000 7f8845a00700 1 -- 172.21.15.37:0/335126940 shutdown_connections 2024-03-29T08:47:30.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:30.487+0000 7f8845a00700 1 --2- 172.21.15.37:0/335126940 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f88401037a0 0x7f88400fce10 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:47:30.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:30.487+0000 7f8845a00700 1 --2- 172.21.15.37:0/335126940 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8840102e00 0x7f88401031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:47:30.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:30.487+0000 7f8845a00700 1 -- 172.21.15.37:0/335126940 >> 172.21.15.37:0/335126940 conn(0x7f88400f89b0 msgr2=0x7f88400fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:47:30.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:30.487+0000 7f8845a00700 1 -- 172.21.15.37:0/335126940 shutdown_connections 2024-03-29T08:47:30.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:30.488+0000 7f8845a00700 1 -- 172.21.15.37:0/335126940 wait complete. 2024-03-29T08:47:30.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:30.489+0000 7f8845a00700 1 Processor -- start 2024-03-29T08:47:30.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:30.489+0000 7f8845a00700 1 -- start start 2024-03-29T08:47:30.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:30.489+0000 7f8845a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8840102e00 0x7f8840113b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:47:30.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:30.489+0000 7f8845a00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f88401037a0 0x7f8840114040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:47:30.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:30.489+0000 7f8845a00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8840114690 con 0x7f8840102e00 2024-03-29T08:47:30.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:30.489+0000 7f8845a00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f884010da80 con 0x7f88401037a0 2024-03-29T08:47:30.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:30.490+0000 7f883ea00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8840102e00 0x7f8840113b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:47:30.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:30.490+0000 7f883ea00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8840102e00 0x7f8840113b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53520/0 (socket says 172.21.15.37:53520) 2024-03-29T08:47:30.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:30.490+0000 7f883ea00700 1 -- 172.21.15.37:0/1223562912 learned_addr learned my addr 172.21.15.37:0/1223562912 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:47:30.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:30.490+0000 7f883e000700 1 --2- 172.21.15.37:0/1223562912 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f88401037a0 0x7f8840114040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:47:30.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:30.490+0000 7f883ea00700 1 -- 172.21.15.37:0/1223562912 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f88401037a0 msgr2=0x7f8840114040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:47:30.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:30.490+0000 7f883ea00700 1 --2- 172.21.15.37:0/1223562912 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f88401037a0 0x7f8840114040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:47:30.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:30.490+0000 7f883ea00700 1 -- 172.21.15.37:0/1223562912 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f882c005040 con 0x7f8840102e00 2024-03-29T08:47:30.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:30.490+0000 7f883e000700 1 --2- 172.21.15.37:0/1223562912 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f88401037a0 0x7f8840114040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:47:30.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:30.490+0000 7f883ea00700 1 --2- 172.21.15.37:0/1223562912 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8840102e00 0x7f8840113b00 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f8830002a50 tx=0x7f88300046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:47:30.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:30.491+0000 7f8836a00700 1 -- 172.21.15.37:0/1223562912 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8830007cc0 con 0x7f8840102e00 2024-03-29T08:47:30.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:30.491+0000 7f8836a00700 1 -- 172.21.15.37:0/1223562912 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f883000f040 con 0x7f8840102e00 2024-03-29T08:47:30.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:30.491+0000 7f8845a00700 1 -- 172.21.15.37:0/1223562912 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f884010dd60 con 0x7f8840102e00 2024-03-29T08:47:30.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:30.491+0000 7f8836a00700 1 -- 172.21.15.37:0/1223562912 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8830009540 con 0x7f8840102e00 2024-03-29T08:47:30.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:30.491+0000 7f8845a00700 1 -- 172.21.15.37:0/1223562912 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f884010e320 con 0x7f8840102e00 2024-03-29T08:47:30.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:30.492+0000 7f8845a00700 1 -- 172.21.15.37:0/1223562912 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88400fe550 con 0x7f8840102e00 2024-03-29T08:47:30.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:30.493+0000 7f8836a00700 1 -- 172.21.15.37:0/1223562912 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f88300096a0 con 0x7f8840102e00 2024-03-29T08:47:30.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:30.494+0000 7f8836a00700 1 --2- 172.21.15.37:0/1223562912 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f882406cc30 0x7f882406f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:47:30.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:30.494+0000 7f8836a00700 1 -- 172.21.15.37:0/1223562912 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f883008bee0 con 0x7f8840102e00 2024-03-29T08:47:30.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:30.494+0000 7f883e000700 1 --2- 172.21.15.37:0/1223562912 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f882406cc30 0x7f882406f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:47:30.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:30.495+0000 7f883e000700 1 --2- 172.21.15.37:0/1223562912 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f882406cc30 0x7f882406f0e0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f884010ec40 tx=0x7f882c009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:47:30.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:30.499+0000 7f8836a00700 1 -- 172.21.15.37:0/1223562912 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f88300598b0 con 0x7f8840102e00 2024-03-29T08:47:30.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:30.734+0000 7f8845a00700 1 -- 172.21.15.37:0/1223562912 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f8840109da0 con 0x7f882406cc30 2024-03-29T08:47:30.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:30.736+0000 7f8836a00700 1 -- 172.21.15.37:0/1223562912 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f8840109da0 con 0x7f882406cc30 2024-03-29T08:47:30.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:30.738+0000 7f8845a00700 1 -- 172.21.15.37:0/1223562912 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f882406cc30 msgr2=0x7f882406f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:47:30.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:30.738+0000 7f8845a00700 1 --2- 172.21.15.37:0/1223562912 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f882406cc30 0x7f882406f0e0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f884010ec40 tx=0x7f882c009e30 comp rx=0 tx=0).stop 2024-03-29T08:47:30.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:30.738+0000 7f8845a00700 1 -- 172.21.15.37:0/1223562912 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8840102e00 msgr2=0x7f8840113b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:47:30.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:30.738+0000 7f8845a00700 1 --2- 172.21.15.37:0/1223562912 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8840102e00 0x7f8840113b00 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f8830002a50 tx=0x7f88300046b0 comp rx=0 tx=0).stop 2024-03-29T08:47:30.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:30.738+0000 7f8845a00700 1 -- 172.21.15.37:0/1223562912 shutdown_connections 2024-03-29T08:47:30.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:30.738+0000 7f8845a00700 1 --2- 172.21.15.37:0/1223562912 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f882406cc30 0x7f882406f0e0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:47:30.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:30.738+0000 7f8845a00700 1 --2- 172.21.15.37:0/1223562912 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8840102e00 0x7f8840113b00 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:47:30.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:30.738+0000 7f8845a00700 1 --2- 172.21.15.37:0/1223562912 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f88401037a0 0x7f8840114040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:47:30.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:30.738+0000 7f8845a00700 1 -- 172.21.15.37:0/1223562912 >> 172.21.15.37:0/1223562912 conn(0x7f88400f89b0 msgr2=0x7f88400f9ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:47:30.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:30.739+0000 7f8845a00700 1 -- 172.21.15.37:0/1223562912 shutdown_connections 2024-03-29T08:47:30.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:30.739+0000 7f8845a00700 1 -- 172.21.15.37:0/1223562912 wait complete. 2024-03-29T08:47:30.753 INFO:teuthology.orchestra.run.smithi037.stdout:true 2024-03-29T08:47:30.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:30.884+0000 7f9a7ac00700 1 -- 172.21.15.37:0/3850240953 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9a740fe830 msgr2=0x7f9a74106f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:47:30.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:30.884+0000 7f9a7ac00700 1 --2- 172.21.15.37:0/3850240953 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9a740fe830 0x7f9a74106f00 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f9a60004940 tx=0x7f9a60009e30 comp rx=0 tx=0).stop 2024-03-29T08:47:30.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:30.884+0000 7f9a7ac00700 1 -- 172.21.15.37:0/3850240953 shutdown_connections 2024-03-29T08:47:30.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:30.884+0000 7f9a7ac00700 1 --2- 172.21.15.37:0/3850240953 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9a740fe830 0x7f9a74106f00 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:47:30.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:30.884+0000 7f9a7ac00700 1 --2- 172.21.15.37:0/3850240953 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9a740fde90 0x7f9a740fe260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:47:30.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:30.884+0000 7f9a7ac00700 1 -- 172.21.15.37:0/3850240953 >> 172.21.15.37:0/3850240953 conn(0x7f9a74072bc0 msgr2=0x7f9a74072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:47:30.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:30.884+0000 7f9a7ac00700 1 -- 172.21.15.37:0/3850240953 shutdown_connections 2024-03-29T08:47:30.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:30.884+0000 7f9a7ac00700 1 -- 172.21.15.37:0/3850240953 wait complete. 2024-03-29T08:47:30.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:30.885+0000 7f9a7ac00700 1 Processor -- start 2024-03-29T08:47:30.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:30.885+0000 7f9a7ac00700 1 -- start start 2024-03-29T08:47:30.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:30.885+0000 7f9a7ac00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9a740fde90 0x7f9a74115d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:47:30.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:30.885+0000 7f9a7ac00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9a740fe830 0x7f9a74116250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:47:30.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:30.885+0000 7f9a7ac00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a741168a0 con 0x7f9a740fde90 2024-03-29T08:47:30.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:30.885+0000 7f9a7ac00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a7410fc70 con 0x7f9a740fe830 2024-03-29T08:47:30.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:30.885+0000 7f9a73e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9a740fde90 0x7f9a74115d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:47:30.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:30.885+0000 7f9a73e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9a740fde90 0x7f9a74115d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53526/0 (socket says 172.21.15.37:53526) 2024-03-29T08:47:30.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:30.885+0000 7f9a73e00700 1 -- 172.21.15.37:0/615135635 learned_addr learned my addr 172.21.15.37:0/615135635 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:47:30.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:30.886+0000 7f9a73400700 1 --2- 172.21.15.37:0/615135635 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9a740fe830 0x7f9a74116250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:47:30.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:30.886+0000 7f9a73e00700 1 -- 172.21.15.37:0/615135635 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9a740fe830 msgr2=0x7f9a74116250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:47:30.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:30.886+0000 7f9a73e00700 1 --2- 172.21.15.37:0/615135635 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9a740fe830 0x7f9a74116250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:47:30.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:30.886+0000 7f9a73e00700 1 -- 172.21.15.37:0/615135635 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a60005040 con 0x7f9a740fde90 2024-03-29T08:47:30.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:30.886+0000 7f9a73e00700 1 --2- 172.21.15.37:0/615135635 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9a740fde90 0x7f9a74115d10 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f9a64002a50 tx=0x7f9a640046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:47:30.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:30.886+0000 7f9a6be00700 1 -- 172.21.15.37:0/615135635 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9a64007cc0 con 0x7f9a740fde90 2024-03-29T08:47:30.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:30.886+0000 7f9a6be00700 1 -- 172.21.15.37:0/615135635 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f9a6400f040 con 0x7f9a740fde90 2024-03-29T08:47:30.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:30.886+0000 7f9a7ac00700 1 -- 172.21.15.37:0/615135635 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a7410ff50 con 0x7f9a740fde90 2024-03-29T08:47:30.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:30.886+0000 7f9a6be00700 1 -- 172.21.15.37:0/615135635 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9a64009540 con 0x7f9a740fde90 2024-03-29T08:47:30.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:30.887+0000 7f9a7ac00700 1 -- 172.21.15.37:0/615135635 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a74110490 con 0x7f9a740fde90 2024-03-29T08:47:30.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:30.888+0000 7f9a7ac00700 1 -- 172.21.15.37:0/615135635 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a74106a20 con 0x7f9a740fde90 2024-03-29T08:47:30.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:30.888+0000 7f9a6be00700 1 -- 172.21.15.37:0/615135635 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f9a640096a0 con 0x7f9a740fde90 2024-03-29T08:47:30.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:30.889+0000 7f9a6be00700 1 --2- 172.21.15.37:0/615135635 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f9a5806cc30 0x7f9a5806f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:47:30.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:30.889+0000 7f9a6be00700 1 -- 172.21.15.37:0/615135635 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f9a6408bee0 con 0x7f9a740fde90 2024-03-29T08:47:30.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:30.890+0000 7f9a73400700 1 --2- 172.21.15.37:0/615135635 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f9a5806cc30 0x7f9a5806f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:47:30.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:30.890+0000 7f9a73400700 1 --2- 172.21.15.37:0/615135635 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f9a5806cc30 0x7f9a5806f0e0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f9a741115b0 tx=0x7f9a60009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:47:30.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:30.894+0000 7f9a6be00700 1 -- 172.21.15.37:0/615135635 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f9a640598b0 con 0x7f9a740fde90 2024-03-29T08:47:30.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:47:30 smithi037 ceph-mon[142325]: pgmap v1099: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:47:31.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:47:30 smithi148 ceph-mon[106692]: pgmap v1099: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:47:31.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:31.124+0000 7f9a7ac00700 1 -- 172.21.15.37:0/615135635 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f9a74109e40 con 0x7f9a5806cc30 2024-03-29T08:47:31.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:31.126+0000 7f9a6be00700 1 -- 172.21.15.37:0/615135635 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f9a74109e40 con 0x7f9a5806cc30 2024-03-29T08:47:31.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:31.128+0000 7f9a7ac00700 1 -- 172.21.15.37:0/615135635 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f9a5806cc30 msgr2=0x7f9a5806f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:47:31.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:31.128+0000 7f9a7ac00700 1 --2- 172.21.15.37:0/615135635 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f9a5806cc30 0x7f9a5806f0e0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f9a741115b0 tx=0x7f9a60009e30 comp rx=0 tx=0).stop 2024-03-29T08:47:31.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:31.128+0000 7f9a7ac00700 1 -- 172.21.15.37:0/615135635 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9a740fde90 msgr2=0x7f9a74115d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:47:31.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:31.128+0000 7f9a7ac00700 1 --2- 172.21.15.37:0/615135635 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9a740fde90 0x7f9a74115d10 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f9a64002a50 tx=0x7f9a640046b0 comp rx=0 tx=0).stop 2024-03-29T08:47:31.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:31.128+0000 7f9a7ac00700 1 -- 172.21.15.37:0/615135635 shutdown_connections 2024-03-29T08:47:31.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:31.128+0000 7f9a7ac00700 1 --2- 172.21.15.37:0/615135635 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f9a5806cc30 0x7f9a5806f0e0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:47:31.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:31.128+0000 7f9a7ac00700 1 --2- 172.21.15.37:0/615135635 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9a740fde90 0x7f9a74115d10 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:47:31.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:31.128+0000 7f9a7ac00700 1 --2- 172.21.15.37:0/615135635 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9a740fe830 0x7f9a74116250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:47:31.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:31.128+0000 7f9a7ac00700 1 -- 172.21.15.37:0/615135635 >> 172.21.15.37:0/615135635 conn(0x7f9a74072bc0 msgr2=0x7f9a740fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:47:31.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:31.128+0000 7f9a7ac00700 1 -- 172.21.15.37:0/615135635 shutdown_connections 2024-03-29T08:47:31.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:31.128+0000 7f9a7ac00700 1 -- 172.21.15.37:0/615135635 wait complete. 2024-03-29T08:47:31.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:31.266+0000 7f892d600700 1 -- 172.21.15.37:0/3436330908 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f89280fde90 msgr2=0x7f89280fe260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:47:31.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:31.266+0000 7f892d600700 1 --2- 172.21.15.37:0/3436330908 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f89280fde90 0x7f89280fe260 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f8918004940 tx=0x7f8918009e30 comp rx=0 tx=0).stop 2024-03-29T08:47:31.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:31.266+0000 7f892d600700 1 -- 172.21.15.37:0/3436330908 shutdown_connections 2024-03-29T08:47:31.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:31.266+0000 7f892d600700 1 --2- 172.21.15.37:0/3436330908 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f89280fe830 0x7f8928106f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:47:31.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:31.266+0000 7f892d600700 1 --2- 172.21.15.37:0/3436330908 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f89280fde90 0x7f89280fe260 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:47:31.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:31.266+0000 7f892d600700 1 -- 172.21.15.37:0/3436330908 >> 172.21.15.37:0/3436330908 conn(0x7f8928072bc0 msgr2=0x7f8928072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:47:31.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:31.266+0000 7f892d600700 1 -- 172.21.15.37:0/3436330908 shutdown_connections 2024-03-29T08:47:31.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:31.266+0000 7f892d600700 1 -- 172.21.15.37:0/3436330908 wait complete. 2024-03-29T08:47:31.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:31.268+0000 7f892d600700 1 Processor -- start 2024-03-29T08:47:31.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:31.268+0000 7f892d600700 1 -- start start 2024-03-29T08:47:31.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:31.268+0000 7f892d600700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f89280fde90 0x7f89280fbd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:47:31.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:31.268+0000 7f892d600700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f89280fe830 0x7f89280fc290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:47:31.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:31.268+0000 7f892d600700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89280fc820 con 0x7f89280fe830 2024-03-29T08:47:31.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:31.268+0000 7f892d600700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89280fc960 con 0x7f89280fde90 2024-03-29T08:47:31.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:31.269+0000 7f8926000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f89280fe830 0x7f89280fc290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:47:31.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:31.269+0000 7f8926000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f89280fe830 0x7f89280fc290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53550/0 (socket says 172.21.15.37:53550) 2024-03-29T08:47:31.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:31.269+0000 7f8926000700 1 -- 172.21.15.37:0/1280829125 learned_addr learned my addr 172.21.15.37:0/1280829125 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:47:31.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:31.269+0000 7f8926a00700 1 --2- 172.21.15.37:0/1280829125 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f89280fde90 0x7f89280fbd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:47:31.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:31.269+0000 7f8926000700 1 -- 172.21.15.37:0/1280829125 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f89280fde90 msgr2=0x7f89280fbd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:47:31.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:31.269+0000 7f8926000700 1 --2- 172.21.15.37:0/1280829125 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f89280fde90 0x7f89280fbd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:47:31.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:31.269+0000 7f8926000700 1 -- 172.21.15.37:0/1280829125 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8918005040 con 0x7f89280fe830 2024-03-29T08:47:31.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:31.269+0000 7f8926000700 1 --2- 172.21.15.37:0/1280829125 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f89280fe830 0x7f89280fc290 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f8914002a50 tx=0x7f89140046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:47:31.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:31.270+0000 7f891ea00700 1 -- 172.21.15.37:0/1280829125 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8914007cc0 con 0x7f89280fe830 2024-03-29T08:47:31.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:31.270+0000 7f891ea00700 1 -- 172.21.15.37:0/1280829125 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f891400f040 con 0x7f89280fe830 2024-03-29T08:47:31.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:31.270+0000 7f892d600700 1 -- 172.21.15.37:0/1280829125 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8928119df0 con 0x7f89280fe830 2024-03-29T08:47:31.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:31.270+0000 7f891ea00700 1 -- 172.21.15.37:0/1280829125 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f891400d9c0 con 0x7f89280fe830 2024-03-29T08:47:31.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:31.270+0000 7f892d600700 1 -- 172.21.15.37:0/1280829125 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f892811a1f0 con 0x7f89280fe830 2024-03-29T08:47:31.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:31.271+0000 7f892d600700 1 -- 172.21.15.37:0/1280829125 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8928065e30 con 0x7f89280fe830 2024-03-29T08:47:31.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:31.272+0000 7f891ea00700 1 -- 172.21.15.37:0/1280829125 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f891400db20 con 0x7f89280fe830 2024-03-29T08:47:31.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:31.273+0000 7f891ea00700 1 --2- 172.21.15.37:0/1280829125 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f890c06cc30 0x7f890c06f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:47:31.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:31.273+0000 7f891ea00700 1 -- 172.21.15.37:0/1280829125 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f891408bd00 con 0x7f89280fe830 2024-03-29T08:47:31.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:31.273+0000 7f8926a00700 1 --2- 172.21.15.37:0/1280829125 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f890c06cc30 0x7f890c06f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:47:31.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:31.273+0000 7f8926a00700 1 --2- 172.21.15.37:0/1280829125 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f890c06cc30 0x7f890c06f0e0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f891800b3b0 tx=0x7f8918009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:47:31.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:31.277+0000 7f891ea00700 1 -- 172.21.15.37:0/1280829125 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f8914055cc0 con 0x7f89280fe830 2024-03-29T08:47:31.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:31.501+0000 7f892d600700 1 -- 172.21.15.37:0/1280829125 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f892804bed0 con 0x7f890c06cc30 2024-03-29T08:47:31.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:31.511+0000 7f891ea00700 1 -- 172.21.15.37:0/1280829125 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f892804bed0 con 0x7f890c06cc30 2024-03-29T08:47:31.511 INFO:teuthology.orchestra.run.smithi037.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-29T08:47:31.511 INFO:teuthology.orchestra.run.smithi037.stdout:alertmanager.smithi037 smithi037 *:9093,9094 running (37m) 14m ago 50m 14.3M - 0.25.0 c8568f914cd2 2f9f9746ed0f 2024-03-29T08:47:31.511 INFO:teuthology.orchestra.run.smithi037.stdout:crash.smithi037 smithi037 running (34m) 14m ago 50m 6647k - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 178d416a3186 2024-03-29T08:47:31.511 INFO:teuthology.orchestra.run.smithi037.stdout:crash.smithi148 smithi148 running (34m) 14m ago 49m 6656k - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 23666989d58f 2024-03-29T08:47:31.511 INFO:teuthology.orchestra.run.smithi037.stdout:grafana.smithi037 smithi037 *:3000 running (37m) 14m ago 49m 74.5M - 9.4.7 954c08fa6188 6b976fe49952 2024-03-29T08:47:31.511 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi037.iifsmh smithi037 running (29m) 14m ago 43m 16.4M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 2d97f562cc5b 2024-03-29T08:47:31.511 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi037.mmwgxe smithi037 running (30m) 14m ago 43m 16.7M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff ec37037f46e8 2024-03-29T08:47:31.511 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi148.qukpjh smithi148 running (14m) 14m ago 43m 10.7M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 3257219bc046 2024-03-29T08:47:31.511 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi148.sqteih smithi148 running (29m) 14m ago 43m 16.3M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 44c15c77037a 2024-03-29T08:47:31.511 INFO:teuthology.orchestra.run.smithi037.stdout:mgr.smithi037.wjikii smithi037 *:8443,9283,8765 running (39m) 14m ago 51m 510M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff c5a1417b9417 2024-03-29T08:47:31.511 INFO:teuthology.orchestra.run.smithi037.stdout:mgr.smithi148.onhgvm smithi148 *:8443,9283,8765 running (38m) 14m ago 49m 456M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff fcfdbce062bd 2024-03-29T08:47:31.512 INFO:teuthology.orchestra.run.smithi037.stdout:mon.smithi037 smithi037 running (35m) 14m ago 52m 70.2M 2048M 18.2.2-1015-gd0e392d9 1cc2f45c17ff eea2060783af 2024-03-29T08:47:31.512 INFO:teuthology.orchestra.run.smithi037.stdout:mon.smithi148 smithi148 running (35m) 14m ago 49m 50.4M 2048M 18.2.2-1015-gd0e392d9 1cc2f45c17ff ad4b2c5748be 2024-03-29T08:47:31.512 INFO:teuthology.orchestra.run.smithi037.stdout:node-exporter.smithi037 smithi037 *:9100 running (38m) 14m ago 50m 8514k - 1.5.0 0da6a335fe13 26b8ae9cb878 2024-03-29T08:47:31.512 INFO:teuthology.orchestra.run.smithi037.stdout:node-exporter.smithi148 smithi148 *:9100 running (38m) 14m ago 49m 8464k - 1.5.0 0da6a335fe13 98414e4e64ed 2024-03-29T08:47:31.512 INFO:teuthology.orchestra.run.smithi037.stdout:osd.0 smithi037 running (33m) 14m ago 48m 108M 1619M 18.2.2-1015-gd0e392d9 1cc2f45c17ff c0c584f6245c 2024-03-29T08:47:31.512 INFO:teuthology.orchestra.run.smithi037.stdout:osd.1 smithi037 running (33m) 14m ago 47m 104M 1619M 18.2.2-1015-gd0e392d9 1cc2f45c17ff a4a2ff4cc83d 2024-03-29T08:47:31.512 INFO:teuthology.orchestra.run.smithi037.stdout:osd.2 smithi037 running (32m) 14m ago 47m 85.9M 1619M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 8b1bc71aa79e 2024-03-29T08:47:31.512 INFO:teuthology.orchestra.run.smithi037.stdout:osd.3 smithi148 running (31m) 14m ago 46m 109M 2642M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 1c656ab8ff86 2024-03-29T08:47:31.512 INFO:teuthology.orchestra.run.smithi037.stdout:osd.4 smithi148 running (31m) 14m ago 46m 101M 2642M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 966a9de11d22 2024-03-29T08:47:31.512 INFO:teuthology.orchestra.run.smithi037.stdout:osd.5 smithi148 running (30m) 14m ago 45m 89.4M 2642M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 474c4f9212ce 2024-03-29T08:47:31.512 INFO:teuthology.orchestra.run.smithi037.stdout:prometheus.smithi037 smithi037 *:9095 running (37m) 14m ago 49m 61.0M - 2.43.0 a07b618ecd1d 30c0223ec589 2024-03-29T08:47:31.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:31.513+0000 7f892d600700 1 -- 172.21.15.37:0/1280829125 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f890c06cc30 msgr2=0x7f890c06f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:47:31.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:31.513+0000 7f892d600700 1 --2- 172.21.15.37:0/1280829125 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f890c06cc30 0x7f890c06f0e0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f891800b3b0 tx=0x7f8918009e30 comp rx=0 tx=0).stop 2024-03-29T08:47:31.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:31.513+0000 7f892d600700 1 -- 172.21.15.37:0/1280829125 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f89280fe830 msgr2=0x7f89280fc290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:47:31.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:31.513+0000 7f892d600700 1 --2- 172.21.15.37:0/1280829125 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f89280fe830 0x7f89280fc290 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f8914002a50 tx=0x7f89140046b0 comp rx=0 tx=0).stop 2024-03-29T08:47:31.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:31.513+0000 7f892d600700 1 -- 172.21.15.37:0/1280829125 shutdown_connections 2024-03-29T08:47:31.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:31.513+0000 7f892d600700 1 --2- 172.21.15.37:0/1280829125 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f890c06cc30 0x7f890c06f0e0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:47:31.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:31.513+0000 7f892d600700 1 --2- 172.21.15.37:0/1280829125 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f89280fde90 0x7f89280fbd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:47:31.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:31.513+0000 7f892d600700 1 --2- 172.21.15.37:0/1280829125 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f89280fe830 0x7f89280fc290 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:47:31.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:31.513+0000 7f892d600700 1 -- 172.21.15.37:0/1280829125 >> 172.21.15.37:0/1280829125 conn(0x7f8928072bc0 msgr2=0x7f89280ff3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:47:31.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:31.514+0000 7f892d600700 1 -- 172.21.15.37:0/1280829125 shutdown_connections 2024-03-29T08:47:31.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:31.514+0000 7f892d600700 1 -- 172.21.15.37:0/1280829125 wait complete. 2024-03-29T08:47:31.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:31.650+0000 7fd859c00700 1 -- 172.21.15.37:0/53115188 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd854070c70 msgr2=0x7fd854109110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:47:31.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:31.650+0000 7fd859c00700 1 --2- 172.21.15.37:0/53115188 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd854070c70 0x7fd854109110 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7fd840004970 tx=0x7fd840009e30 comp rx=0 tx=0).stop 2024-03-29T08:47:31.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:31.650+0000 7fd859c00700 1 -- 172.21.15.37:0/53115188 shutdown_connections 2024-03-29T08:47:31.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:31.650+0000 7fd859c00700 1 --2- 172.21.15.37:0/53115188 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd854070c70 0x7fd854109110 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:47:31.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:31.650+0000 7fd859c00700 1 --2- 172.21.15.37:0/53115188 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd854070360 0x7fd854070730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:47:31.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:31.650+0000 7fd859c00700 1 -- 172.21.15.37:0/53115188 >> 172.21.15.37:0/53115188 conn(0x7fd8540f89b0 msgr2=0x7fd8540fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:47:31.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:31.650+0000 7fd859c00700 1 -- 172.21.15.37:0/53115188 shutdown_connections 2024-03-29T08:47:31.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:31.650+0000 7fd859c00700 1 -- 172.21.15.37:0/53115188 wait complete. 2024-03-29T08:47:31.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:31.651+0000 7fd859c00700 1 Processor -- start 2024-03-29T08:47:31.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:31.651+0000 7fd859c00700 1 -- start start 2024-03-29T08:47:31.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:31.651+0000 7fd859c00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd854070360 0x7fd854115ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:47:31.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:31.651+0000 7fd859c00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd854070c70 0x7fd8541161e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:47:31.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:31.651+0000 7fd859c00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd854116830 con 0x7fd854070c70 2024-03-29T08:47:31.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:31.651+0000 7fd859c00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd85410fc00 con 0x7fd854070360 2024-03-29T08:47:31.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:31.652+0000 7fd852000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd854070c70 0x7fd8541161e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:47:31.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:31.652+0000 7fd852000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd854070c70 0x7fd8541161e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53574/0 (socket says 172.21.15.37:53574) 2024-03-29T08:47:31.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:31.652+0000 7fd852000700 1 -- 172.21.15.37:0/4169858908 learned_addr learned my addr 172.21.15.37:0/4169858908 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:47:31.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:31.652+0000 7fd852a00700 1 --2- 172.21.15.37:0/4169858908 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd854070360 0x7fd854115ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:47:31.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:31.652+0000 7fd852000700 1 -- 172.21.15.37:0/4169858908 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd854070360 msgr2=0x7fd854115ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:47:31.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:31.652+0000 7fd852000700 1 --2- 172.21.15.37:0/4169858908 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd854070360 0x7fd854115ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:47:31.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:31.652+0000 7fd852000700 1 -- 172.21.15.37:0/4169858908 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd840005040 con 0x7fd854070c70 2024-03-29T08:47:31.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:31.652+0000 7fd852000700 1 --2- 172.21.15.37:0/4169858908 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd854070c70 0x7fd8541161e0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7fd840005010 tx=0x7fd840009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:47:31.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:31.652+0000 7fd84aa00700 1 -- 172.21.15.37:0/4169858908 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd84000e040 con 0x7fd854070c70 2024-03-29T08:47:31.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:31.652+0000 7fd84aa00700 1 -- 172.21.15.37:0/4169858908 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd840004290 con 0x7fd854070c70 2024-03-29T08:47:31.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:31.653+0000 7fd84aa00700 1 -- 172.21.15.37:0/4169858908 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd840013e40 con 0x7fd854070c70 2024-03-29T08:47:31.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:31.653+0000 7fd859c00700 1 -- 172.21.15.37:0/4169858908 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd85410fe80 con 0x7fd854070c70 2024-03-29T08:47:31.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:31.653+0000 7fd859c00700 1 -- 172.21.15.37:0/4169858908 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8541103c0 con 0x7fd854070c70 2024-03-29T08:47:31.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:31.654+0000 7fd859c00700 1 -- 172.21.15.37:0/4169858908 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd854103bd0 con 0x7fd854070c70 2024-03-29T08:47:31.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:31.655+0000 7fd84aa00700 1 -- 172.21.15.37:0/4169858908 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fd840013620 con 0x7fd854070c70 2024-03-29T08:47:31.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:31.655+0000 7fd84aa00700 1 --2- 172.21.15.37:0/4169858908 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fd83806cc30 0x7fd83806f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:47:31.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:31.656+0000 7fd84aa00700 1 -- 172.21.15.37:0/4169858908 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fd84008d290 con 0x7fd854070c70 2024-03-29T08:47:31.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:31.656+0000 7fd852a00700 1 --2- 172.21.15.37:0/4169858908 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fd83806cc30 0x7fd83806f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:47:31.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:31.656+0000 7fd852a00700 1 --2- 172.21.15.37:0/4169858908 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fd83806cc30 0x7fd83806f0e0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fd844004600 tx=0x7fd84400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:47:31.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:31.660+0000 7fd84aa00700 1 -- 172.21.15.37:0/4169858908 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fd84005ad60 con 0x7fd854070c70 2024-03-29T08:47:31.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:47:31 smithi037 ceph-mon[142325]: from='client.35962 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:47:31.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:31.964+0000 7fd859c00700 1 -- 172.21.15.37:0/4169858908 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fd8541112f0 con 0x7fd854070c70 2024-03-29T08:47:31.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:31.964+0000 7fd84aa00700 1 -- 172.21.15.37:0/4169858908 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7fd84005a8f0 con 0x7fd854070c70 2024-03-29T08:47:31.964 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-29T08:47:31.964 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": { 2024-03-29T08:47:31.964 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 2 2024-03-29T08:47:31.964 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:47:31.964 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": { 2024-03-29T08:47:31.964 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 2 2024-03-29T08:47:31.965 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:47:31.965 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": { 2024-03-29T08:47:31.965 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 6 2024-03-29T08:47:31.965 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:47:31.965 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": { 2024-03-29T08:47:31.965 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 4 2024-03-29T08:47:31.965 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:47:31.965 INFO:teuthology.orchestra.run.smithi037.stdout: "overall": { 2024-03-29T08:47:31.965 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 14 2024-03-29T08:47:31.965 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-29T08:47:31.965 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-29T08:47:31.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:31.967+0000 7fd859c00700 1 -- 172.21.15.37:0/4169858908 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fd83806cc30 msgr2=0x7fd83806f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:47:31.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:31.967+0000 7fd859c00700 1 --2- 172.21.15.37:0/4169858908 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fd83806cc30 0x7fd83806f0e0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fd844004600 tx=0x7fd84400b040 comp rx=0 tx=0).stop 2024-03-29T08:47:31.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:31.967+0000 7fd859c00700 1 -- 172.21.15.37:0/4169858908 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd854070c70 msgr2=0x7fd8541161e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:47:31.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:31.967+0000 7fd859c00700 1 --2- 172.21.15.37:0/4169858908 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd854070c70 0x7fd8541161e0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7fd840005010 tx=0x7fd840009e30 comp rx=0 tx=0).stop 2024-03-29T08:47:31.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:31.967+0000 7fd859c00700 1 -- 172.21.15.37:0/4169858908 shutdown_connections 2024-03-29T08:47:31.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:31.967+0000 7fd859c00700 1 --2- 172.21.15.37:0/4169858908 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fd83806cc30 0x7fd83806f0e0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:47:31.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:31.967+0000 7fd859c00700 1 --2- 172.21.15.37:0/4169858908 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd854070360 0x7fd854115ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:47:31.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:31.967+0000 7fd859c00700 1 --2- 172.21.15.37:0/4169858908 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd854070c70 0x7fd8541161e0 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:47:31.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:31.967+0000 7fd859c00700 1 -- 172.21.15.37:0/4169858908 >> 172.21.15.37:0/4169858908 conn(0x7fd8540f89b0 msgr2=0x7fd8541015f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:47:31.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:31.967+0000 7fd859c00700 1 -- 172.21.15.37:0/4169858908 shutdown_connections 2024-03-29T08:47:31.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:31.967+0000 7fd859c00700 1 -- 172.21.15.37:0/4169858908 wait complete. 2024-03-29T08:47:32.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:47:31 smithi148 ceph-mon[106692]: from='client.35962 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:47:32.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:32.104+0000 7fd0cbe00700 1 -- 172.21.15.37:0/1367636040 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd0c4102e00 msgr2=0x7fd0c41031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:47:32.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:32.104+0000 7fd0cbe00700 1 --2- 172.21.15.37:0/1367636040 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd0c4102e00 0x7fd0c41031d0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7fd0b8004940 tx=0x7fd0b8009e30 comp rx=0 tx=0).stop 2024-03-29T08:47:32.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:32.104+0000 7fd0cbe00700 1 -- 172.21.15.37:0/1367636040 shutdown_connections 2024-03-29T08:47:32.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:32.104+0000 7fd0cbe00700 1 --2- 172.21.15.37:0/1367636040 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd0c41037a0 0x7fd0c40fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:47:32.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:32.104+0000 7fd0cbe00700 1 --2- 172.21.15.37:0/1367636040 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd0c4102e00 0x7fd0c41031d0 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:47:32.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:32.104+0000 7fd0cbe00700 1 -- 172.21.15.37:0/1367636040 >> 172.21.15.37:0/1367636040 conn(0x7fd0c40f89b0 msgr2=0x7fd0c40fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:47:32.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:32.104+0000 7fd0cbe00700 1 -- 172.21.15.37:0/1367636040 shutdown_connections 2024-03-29T08:47:32.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:32.104+0000 7fd0cbe00700 1 -- 172.21.15.37:0/1367636040 wait complete. 2024-03-29T08:47:32.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:32.105+0000 7fd0cbe00700 1 Processor -- start 2024-03-29T08:47:32.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:32.105+0000 7fd0cbe00700 1 -- start start 2024-03-29T08:47:32.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:32.105+0000 7fd0cbe00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd0c4102e00 0x7fd0c4115c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:47:32.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:32.105+0000 7fd0cbe00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd0c41037a0 0x7fd0c41161b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:47:32.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:32.105+0000 7fd0cbe00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0c4116800 con 0x7fd0c41037a0 2024-03-29T08:47:32.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:32.105+0000 7fd0cbe00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0c410fbd0 con 0x7fd0c4102e00 2024-03-29T08:47:32.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:32.106+0000 7fd0c8a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd0c41037a0 0x7fd0c41161b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:47:32.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:32.106+0000 7fd0c8a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd0c41037a0 0x7fd0c41161b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53594/0 (socket says 172.21.15.37:53594) 2024-03-29T08:47:32.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:32.106+0000 7fd0c8a00700 1 -- 172.21.15.37:0/2333774207 learned_addr learned my addr 172.21.15.37:0/2333774207 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:47:32.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:32.106+0000 7fd0c9400700 1 --2- 172.21.15.37:0/2333774207 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd0c4102e00 0x7fd0c4115c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:47:32.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:32.106+0000 7fd0c8a00700 1 -- 172.21.15.37:0/2333774207 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd0c4102e00 msgr2=0x7fd0c4115c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:47:32.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:32.106+0000 7fd0c8a00700 1 --2- 172.21.15.37:0/2333774207 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd0c4102e00 0x7fd0c4115c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:47:32.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:32.106+0000 7fd0c8a00700 1 -- 172.21.15.37:0/2333774207 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0b8005040 con 0x7fd0c41037a0 2024-03-29T08:47:32.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:32.106+0000 7fd0c8a00700 1 --2- 172.21.15.37:0/2333774207 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd0c41037a0 0x7fd0c41161b0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7fd0b000eb40 tx=0x7fd0b000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:47:32.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:32.106+0000 7fd0c1600700 1 -- 172.21.15.37:0/2333774207 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd0b0007cc0 con 0x7fd0c41037a0 2024-03-29T08:47:32.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:32.106+0000 7fd0c1600700 1 -- 172.21.15.37:0/2333774207 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd0b000f040 con 0x7fd0c41037a0 2024-03-29T08:47:32.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:32.106+0000 7fd0cbe00700 1 -- 172.21.15.37:0/2333774207 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0c410fe50 con 0x7fd0c41037a0 2024-03-29T08:47:32.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:32.107+0000 7fd0c1600700 1 -- 172.21.15.37:0/2333774207 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd0b000c8f0 con 0x7fd0c41037a0 2024-03-29T08:47:32.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:32.107+0000 7fd0cbe00700 1 -- 172.21.15.37:0/2333774207 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0c4110410 con 0x7fd0c41037a0 2024-03-29T08:47:32.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:32.108+0000 7fd0cbe00700 1 -- 172.21.15.37:0/2333774207 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0c40fe550 con 0x7fd0c41037a0 2024-03-29T08:47:32.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:32.109+0000 7fd0c1600700 1 -- 172.21.15.37:0/2333774207 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fd0b00090d0 con 0x7fd0c41037a0 2024-03-29T08:47:32.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:32.110+0000 7fd0c1600700 1 --2- 172.21.15.37:0/2333774207 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fd0ac06cc30 0x7fd0ac06f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:47:32.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:32.110+0000 7fd0c1600700 1 -- 172.21.15.37:0/2333774207 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fd0b008ba50 con 0x7fd0c41037a0 2024-03-29T08:47:32.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:32.110+0000 7fd0c9400700 1 --2- 172.21.15.37:0/2333774207 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fd0ac06cc30 0x7fd0ac06f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:47:32.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:32.111+0000 7fd0c9400700 1 --2- 172.21.15.37:0/2333774207 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fd0ac06cc30 0x7fd0ac06f0e0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fd0b80045a0 tx=0x7fd0b8009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:47:32.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:32.114+0000 7fd0c1600700 1 -- 172.21.15.37:0/2333774207 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fd0b0059550 con 0x7fd0c41037a0 2024-03-29T08:47:32.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:32.376+0000 7fd0cbe00700 1 -- 172.21.15.37:0/2333774207 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fd0c404bed0 con 0x7fd0c41037a0 2024-03-29T08:47:32.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:32.376+0000 7fd0c1600700 1 -- 172.21.15.37:0/2333774207 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 30 v30) v1 ==== 76+0+1841 (secure 0 0 0) 0x7fd0b00590e0 con 0x7fd0c41037a0 2024-03-29T08:47:32.376 INFO:teuthology.orchestra.run.smithi037.stdout:e30 2024-03-29T08:47:32.377 INFO:teuthology.orchestra.run.smithi037.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-29T08:47:32.377 INFO:teuthology.orchestra.run.smithi037.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T08:47:32.377 INFO:teuthology.orchestra.run.smithi037.stdout:legacy client fscid: 1 2024-03-29T08:47:32.377 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:47:32.377 INFO:teuthology.orchestra.run.smithi037.stdout:Filesystem 'cephfs' (1) 2024-03-29T08:47:32.377 INFO:teuthology.orchestra.run.smithi037.stdout:fs_name cephfs 2024-03-29T08:47:32.377 INFO:teuthology.orchestra.run.smithi037.stdout:epoch 21 2024-03-29T08:47:32.377 INFO:teuthology.orchestra.run.smithi037.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-03-29T08:47:32.377 INFO:teuthology.orchestra.run.smithi037.stdout:created 2024-03-29T08:03:29.241269+0000 2024-03-29T08:47:32.377 INFO:teuthology.orchestra.run.smithi037.stdout:modified 2024-03-29T08:17:16.237238+0000 2024-03-29T08:47:32.377 INFO:teuthology.orchestra.run.smithi037.stdout:tableserver 0 2024-03-29T08:47:32.377 INFO:teuthology.orchestra.run.smithi037.stdout:root 0 2024-03-29T08:47:32.377 INFO:teuthology.orchestra.run.smithi037.stdout:session_timeout 60 2024-03-29T08:47:32.377 INFO:teuthology.orchestra.run.smithi037.stdout:session_autoclose 300 2024-03-29T08:47:32.377 INFO:teuthology.orchestra.run.smithi037.stdout:max_file_size 1099511627776 2024-03-29T08:47:32.377 INFO:teuthology.orchestra.run.smithi037.stdout:max_xattr_size 65536 2024-03-29T08:47:32.378 INFO:teuthology.orchestra.run.smithi037.stdout:required_client_features {} 2024-03-29T08:47:32.378 INFO:teuthology.orchestra.run.smithi037.stdout:last_failure 0 2024-03-29T08:47:32.378 INFO:teuthology.orchestra.run.smithi037.stdout:last_failure_osd_epoch 82 2024-03-29T08:47:32.378 INFO:teuthology.orchestra.run.smithi037.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T08:47:32.378 INFO:teuthology.orchestra.run.smithi037.stdout:max_mds 2 2024-03-29T08:47:32.378 INFO:teuthology.orchestra.run.smithi037.stdout:in 0,1 2024-03-29T08:47:32.378 INFO:teuthology.orchestra.run.smithi037.stdout:up {} 2024-03-29T08:47:32.378 INFO:teuthology.orchestra.run.smithi037.stdout:failed 0,1 2024-03-29T08:47:32.378 INFO:teuthology.orchestra.run.smithi037.stdout:damaged 2024-03-29T08:47:32.378 INFO:teuthology.orchestra.run.smithi037.stdout:stopped 2024-03-29T08:47:32.378 INFO:teuthology.orchestra.run.smithi037.stdout:data_pools [3] 2024-03-29T08:47:32.378 INFO:teuthology.orchestra.run.smithi037.stdout:metadata_pool 2 2024-03-29T08:47:32.379 INFO:teuthology.orchestra.run.smithi037.stdout:inline_data disabled 2024-03-29T08:47:32.379 INFO:teuthology.orchestra.run.smithi037.stdout:balancer 2024-03-29T08:47:32.379 INFO:teuthology.orchestra.run.smithi037.stdout:bal_rank_mask -1 2024-03-29T08:47:32.379 INFO:teuthology.orchestra.run.smithi037.stdout:standby_count_wanted 1 2024-03-29T08:47:32.379 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:47:32.379 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:47:32.379 INFO:teuthology.orchestra.run.smithi037.stdout:Standby daemons: 2024-03-29T08:47:32.379 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:47:32.379 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi037.mmwgxe{-1:34440} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.37:6828/1248142005,v1:172.21.15.37:6829/1248142005] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:47:32.379 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi037.iifsmh{-1:34444} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.37:6826/2060882959,v1:172.21.15.37:6827/2060882959] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:47:32.379 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi148.sqteih{-1:44395} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.148:6824/4078652242,v1:172.21.15.148:6825/4078652242] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:47:32.379 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi148.qukpjh{-1:45007} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.148:6826/2394751105,v1:172.21.15.148:6827/2394751105] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:47:32.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:32.379+0000 7fd0cbe00700 1 -- 172.21.15.37:0/2333774207 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fd0ac06cc30 msgr2=0x7fd0ac06f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:47:32.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:32.379+0000 7fd0cbe00700 1 --2- 172.21.15.37:0/2333774207 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fd0ac06cc30 0x7fd0ac06f0e0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fd0b80045a0 tx=0x7fd0b8009e30 comp rx=0 tx=0).stop 2024-03-29T08:47:32.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:32.379+0000 7fd0cbe00700 1 -- 172.21.15.37:0/2333774207 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd0c41037a0 msgr2=0x7fd0c41161b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:47:32.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:32.379+0000 7fd0cbe00700 1 --2- 172.21.15.37:0/2333774207 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd0c41037a0 0x7fd0c41161b0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7fd0b000eb40 tx=0x7fd0b000b040 comp rx=0 tx=0).stop 2024-03-29T08:47:32.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:32.379+0000 7fd0cbe00700 1 -- 172.21.15.37:0/2333774207 shutdown_connections 2024-03-29T08:47:32.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:32.379+0000 7fd0cbe00700 1 --2- 172.21.15.37:0/2333774207 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fd0ac06cc30 0x7fd0ac06f0e0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:47:32.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:32.379+0000 7fd0cbe00700 1 --2- 172.21.15.37:0/2333774207 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd0c4102e00 0x7fd0c4115c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:47:32.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:32.379+0000 7fd0cbe00700 1 --2- 172.21.15.37:0/2333774207 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd0c41037a0 0x7fd0c41161b0 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:47:32.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:32.379+0000 7fd0cbe00700 1 -- 172.21.15.37:0/2333774207 >> 172.21.15.37:0/2333774207 conn(0x7fd0c40f89b0 msgr2=0x7fd0c40f9960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:47:32.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:32.379+0000 7fd0cbe00700 1 -- 172.21.15.37:0/2333774207 shutdown_connections 2024-03-29T08:47:32.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:32.379+0000 7fd0cbe00700 1 -- 172.21.15.37:0/2333774207 wait complete. 2024-03-29T08:47:32.381 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 30 2024-03-29T08:47:32.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:32.519+0000 7fa6a8800700 1 -- 172.21.15.37:0/754559096 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa6a01037a0 msgr2=0x7fa6a00fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:47:32.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:32.519+0000 7fa6a8800700 1 --2- 172.21.15.37:0/754559096 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa6a01037a0 0x7fa6a00fce10 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7fa690004940 tx=0x7fa690009e30 comp rx=0 tx=0).stop 2024-03-29T08:47:32.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:32.520+0000 7fa6a8800700 1 -- 172.21.15.37:0/754559096 shutdown_connections 2024-03-29T08:47:32.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:32.520+0000 7fa6a8800700 1 --2- 172.21.15.37:0/754559096 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa6a01037a0 0x7fa6a00fce10 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:47:32.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:32.520+0000 7fa6a8800700 1 --2- 172.21.15.37:0/754559096 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa6a0102e00 0x7fa6a01031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:47:32.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:32.520+0000 7fa6a8800700 1 -- 172.21.15.37:0/754559096 >> 172.21.15.37:0/754559096 conn(0x7fa6a00f89b0 msgr2=0x7fa6a00fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:47:32.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:32.520+0000 7fa6a8800700 1 -- 172.21.15.37:0/754559096 shutdown_connections 2024-03-29T08:47:32.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:32.520+0000 7fa6a8800700 1 -- 172.21.15.37:0/754559096 wait complete. 2024-03-29T08:47:32.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:32.520+0000 7fa6a8800700 1 Processor -- start 2024-03-29T08:47:32.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:32.521+0000 7fa6a8800700 1 -- start start 2024-03-29T08:47:32.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:32.521+0000 7fa6a8800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa6a0102e00 0x7fa6a0115d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:47:32.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:32.521+0000 7fa6a8800700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa6a01037a0 0x7fa6a0116240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:47:32.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:32.521+0000 7fa6a8800700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6a0116890 con 0x7fa6a0102e00 2024-03-29T08:47:32.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:32.521+0000 7fa6a8800700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6a010fc60 con 0x7fa6a01037a0 2024-03-29T08:47:32.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:32.521+0000 7fa6a5e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa6a0102e00 0x7fa6a0115d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:47:32.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:32.521+0000 7fa6a5e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa6a0102e00 0x7fa6a0115d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53616/0 (socket says 172.21.15.37:53616) 2024-03-29T08:47:32.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:32.521+0000 7fa6a5e00700 1 -- 172.21.15.37:0/3348003797 learned_addr learned my addr 172.21.15.37:0/3348003797 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:47:32.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:32.522+0000 7fa6a5400700 1 --2- 172.21.15.37:0/3348003797 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa6a01037a0 0x7fa6a0116240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:47:32.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:32.522+0000 7fa6a5e00700 1 -- 172.21.15.37:0/3348003797 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa6a01037a0 msgr2=0x7fa6a0116240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:47:32.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:32.522+0000 7fa6a5e00700 1 --2- 172.21.15.37:0/3348003797 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa6a01037a0 0x7fa6a0116240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:47:32.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:32.522+0000 7fa6a5e00700 1 -- 172.21.15.37:0/3348003797 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa690005040 con 0x7fa6a0102e00 2024-03-29T08:47:32.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:32.522+0000 7fa6a5e00700 1 --2- 172.21.15.37:0/3348003797 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa6a0102e00 0x7fa6a0115d00 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7fa69400c8c0 tx=0x7fa6940044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:47:32.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:32.522+0000 7fa69a000700 1 -- 172.21.15.37:0/3348003797 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa69400a980 con 0x7fa6a0102e00 2024-03-29T08:47:32.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:32.522+0000 7fa69a000700 1 -- 172.21.15.37:0/3348003797 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fa69400ce60 con 0x7fa6a0102e00 2024-03-29T08:47:32.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:32.522+0000 7fa6a8800700 1 -- 172.21.15.37:0/3348003797 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6a010ff40 con 0x7fa6a0102e00 2024-03-29T08:47:32.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:32.522+0000 7fa69a000700 1 -- 172.21.15.37:0/3348003797 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa6940113a0 con 0x7fa6a0102e00 2024-03-29T08:47:32.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:32.522+0000 7fa6a8800700 1 -- 172.21.15.37:0/3348003797 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6a0110500 con 0x7fa6a0102e00 2024-03-29T08:47:32.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:32.523+0000 7fa6a8800700 1 -- 172.21.15.37:0/3348003797 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6a00fe550 con 0x7fa6a0102e00 2024-03-29T08:47:32.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:32.524+0000 7fa69a000700 1 -- 172.21.15.37:0/3348003797 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fa694012400 con 0x7fa6a0102e00 2024-03-29T08:47:32.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:32.525+0000 7fa69a000700 1 --2- 172.21.15.37:0/3348003797 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fa68806cc30 0x7fa68806f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:47:32.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:32.525+0000 7fa69a000700 1 -- 172.21.15.37:0/3348003797 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fa69408c0f0 con 0x7fa6a0102e00 2024-03-29T08:47:32.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:32.526+0000 7fa6a5400700 1 --2- 172.21.15.37:0/3348003797 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fa68806cc30 0x7fa68806f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:47:32.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:32.526+0000 7fa6a5400700 1 --2- 172.21.15.37:0/3348003797 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fa68806cc30 0x7fa68806f0e0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fa6a01115a0 tx=0x7fa690009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:47:32.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:32.530+0000 7fa69a000700 1 -- 172.21.15.37:0/3348003797 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fa694059bc0 con 0x7fa6a0102e00 2024-03-29T08:47:32.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:32.768+0000 7fa6a8800700 1 -- 172.21.15.37:0/3348003797 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa6a004bf40 con 0x7fa68806cc30 2024-03-29T08:47:32.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:32.769+0000 7fa69a000700 1 -- 172.21.15.37:0/3348003797 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fa6a004bf40 con 0x7fa68806cc30 2024-03-29T08:47:32.768 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-29T08:47:32.769 INFO:teuthology.orchestra.run.smithi037.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59", 2024-03-29T08:47:32.769 INFO:teuthology.orchestra.run.smithi037.stdout: "in_progress": true, 2024-03-29T08:47:32.769 INFO:teuthology.orchestra.run.smithi037.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-29T08:47:32.769 INFO:teuthology.orchestra.run.smithi037.stdout: "services_complete": [ 2024-03-29T08:47:32.769 INFO:teuthology.orchestra.run.smithi037.stdout: "crash", 2024-03-29T08:47:32.769 INFO:teuthology.orchestra.run.smithi037.stdout: "osd", 2024-03-29T08:47:32.769 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr", 2024-03-29T08:47:32.769 INFO:teuthology.orchestra.run.smithi037.stdout: "mds", 2024-03-29T08:47:32.769 INFO:teuthology.orchestra.run.smithi037.stdout: "mon" 2024-03-29T08:47:32.769 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-29T08:47:32.769 INFO:teuthology.orchestra.run.smithi037.stdout: "progress": "16/21 daemons upgraded", 2024-03-29T08:47:32.769 INFO:teuthology.orchestra.run.smithi037.stdout: "message": "Currently upgrading mds daemons", 2024-03-29T08:47:32.769 INFO:teuthology.orchestra.run.smithi037.stdout: "is_paused": false 2024-03-29T08:47:32.770 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-29T08:47:32.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:32.771+0000 7fa6a8800700 1 -- 172.21.15.37:0/3348003797 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fa68806cc30 msgr2=0x7fa68806f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:47:32.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:32.771+0000 7fa6a8800700 1 --2- 172.21.15.37:0/3348003797 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fa68806cc30 0x7fa68806f0e0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fa6a01115a0 tx=0x7fa690009e30 comp rx=0 tx=0).stop 2024-03-29T08:47:32.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:32.771+0000 7fa6a8800700 1 -- 172.21.15.37:0/3348003797 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa6a0102e00 msgr2=0x7fa6a0115d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:47:32.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:32.771+0000 7fa6a8800700 1 --2- 172.21.15.37:0/3348003797 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa6a0102e00 0x7fa6a0115d00 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7fa69400c8c0 tx=0x7fa6940044c0 comp rx=0 tx=0).stop 2024-03-29T08:47:32.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:32.771+0000 7fa6a8800700 1 -- 172.21.15.37:0/3348003797 shutdown_connections 2024-03-29T08:47:32.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:32.771+0000 7fa6a8800700 1 --2- 172.21.15.37:0/3348003797 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fa68806cc30 0x7fa68806f0e0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:47:32.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:32.771+0000 7fa6a8800700 1 --2- 172.21.15.37:0/3348003797 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa6a0102e00 0x7fa6a0115d00 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:47:32.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:32.771+0000 7fa6a8800700 1 --2- 172.21.15.37:0/3348003797 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa6a01037a0 0x7fa6a0116240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:47:32.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:32.771+0000 7fa6a8800700 1 -- 172.21.15.37:0/3348003797 >> 172.21.15.37:0/3348003797 conn(0x7fa6a00f89b0 msgr2=0x7fa6a00f9ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:47:32.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:32.771+0000 7fa6a8800700 1 -- 172.21.15.37:0/3348003797 shutdown_connections 2024-03-29T08:47:32.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:32.772+0000 7fa6a8800700 1 -- 172.21.15.37:0/3348003797 wait complete. 2024-03-29T08:47:32.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:32.908+0000 7f23f1e00700 1 -- 172.21.15.37:0/2820247177 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f23ec0fde90 msgr2=0x7f23ec0fe260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:47:32.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:32.908+0000 7f23f1e00700 1 --2- 172.21.15.37:0/2820247177 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f23ec0fde90 0x7f23ec0fe260 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f23dc004940 tx=0x7f23dc009e30 comp rx=0 tx=0).stop 2024-03-29T08:47:32.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:32.908+0000 7f23f1e00700 1 -- 172.21.15.37:0/2820247177 shutdown_connections 2024-03-29T08:47:32.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:32.908+0000 7f23f1e00700 1 --2- 172.21.15.37:0/2820247177 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f23ec0fe830 0x7f23ec106f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:47:32.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:32.908+0000 7f23f1e00700 1 --2- 172.21.15.37:0/2820247177 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f23ec0fde90 0x7f23ec0fe260 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:47:32.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:32.908+0000 7f23f1e00700 1 -- 172.21.15.37:0/2820247177 >> 172.21.15.37:0/2820247177 conn(0x7f23ec072bc0 msgr2=0x7f23ec072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:47:32.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:32.908+0000 7f23f1e00700 1 -- 172.21.15.37:0/2820247177 shutdown_connections 2024-03-29T08:47:32.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:32.908+0000 7f23f1e00700 1 -- 172.21.15.37:0/2820247177 wait complete. 2024-03-29T08:47:32.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:32.909+0000 7f23f1e00700 1 Processor -- start 2024-03-29T08:47:32.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:32.909+0000 7f23f1e00700 1 -- start start 2024-03-29T08:47:32.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:32.910+0000 7f23f1e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f23ec0fde90 0x7f23ec115d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:47:32.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:32.910+0000 7f23f1e00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f23ec0fe830 0x7f23ec116260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:47:32.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:32.910+0000 7f23f1e00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23ec1168b0 con 0x7f23ec0fde90 2024-03-29T08:47:32.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:32.910+0000 7f23f1e00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23ec10fc80 con 0x7f23ec0fe830 2024-03-29T08:47:32.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:32.910+0000 7f23eaa00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f23ec0fde90 0x7f23ec115d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:47:32.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:32.910+0000 7f23eaa00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f23ec0fde90 0x7f23ec115d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53636/0 (socket says 172.21.15.37:53636) 2024-03-29T08:47:32.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:32.910+0000 7f23eaa00700 1 -- 172.21.15.37:0/1100881881 learned_addr learned my addr 172.21.15.37:0/1100881881 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:47:32.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:32.910+0000 7f23ea000700 1 --2- 172.21.15.37:0/1100881881 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f23ec0fe830 0x7f23ec116260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:47:32.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:32.910+0000 7f23eaa00700 1 -- 172.21.15.37:0/1100881881 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f23ec0fe830 msgr2=0x7f23ec116260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:47:32.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:32.910+0000 7f23eaa00700 1 --2- 172.21.15.37:0/1100881881 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f23ec0fe830 0x7f23ec116260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:47:32.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:32.910+0000 7f23eaa00700 1 -- 172.21.15.37:0/1100881881 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23dc005040 con 0x7f23ec0fde90 2024-03-29T08:47:32.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:32.911+0000 7f23eaa00700 1 --2- 172.21.15.37:0/1100881881 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f23ec0fde90 0x7f23ec115d20 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f23dc009360 tx=0x7f23dc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:47:32.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:32.911+0000 7f23e2a00700 1 -- 172.21.15.37:0/1100881881 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f23dc00e040 con 0x7f23ec0fde90 2024-03-29T08:47:32.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:32.911+0000 7f23e2a00700 1 -- 172.21.15.37:0/1100881881 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f23dc00bed0 con 0x7f23ec0fde90 2024-03-29T08:47:32.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:32.911+0000 7f23f1e00700 1 -- 172.21.15.37:0/1100881881 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23ec10ff00 con 0x7f23ec0fde90 2024-03-29T08:47:32.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:32.911+0000 7f23e2a00700 1 -- 172.21.15.37:0/1100881881 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f23dc01c3f0 con 0x7f23ec0fde90 2024-03-29T08:47:32.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:32.911+0000 7f23f1e00700 1 -- 172.21.15.37:0/1100881881 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23ec110440 con 0x7f23ec0fde90 2024-03-29T08:47:32.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:32.912+0000 7f23f1e00700 1 -- 172.21.15.37:0/1100881881 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23ec106a20 con 0x7f23ec0fde90 2024-03-29T08:47:32.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:32.913+0000 7f23e2a00700 1 -- 172.21.15.37:0/1100881881 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f23dc013790 con 0x7f23ec0fde90 2024-03-29T08:47:32.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:32.914+0000 7f23e2a00700 1 --2- 172.21.15.37:0/1100881881 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f23d006cc20 0x7f23d006f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:47:32.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:32.914+0000 7f23e2a00700 1 -- 172.21.15.37:0/1100881881 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f23dc08d480 con 0x7f23ec0fde90 2024-03-29T08:47:32.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:32.915+0000 7f23ea000700 1 --2- 172.21.15.37:0/1100881881 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f23d006cc20 0x7f23d006f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:47:32.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:32.915+0000 7f23ea000700 1 --2- 172.21.15.37:0/1100881881 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f23d006cc20 0x7f23d006f0d0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f23ec1115c0 tx=0x7f23d800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:47:32.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:32.919+0000 7f23e2a00700 1 -- 172.21.15.37:0/1100881881 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f23dc05ae50 con 0x7f23ec0fde90 2024-03-29T08:47:32.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:47:32 smithi037 ceph-mon[142325]: from='client.35966 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:47:32.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:47:32 smithi037 ceph-mon[142325]: pgmap v1100: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:47:32.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:47:32 smithi037 ceph-mon[142325]: from='client.35970 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:47:32.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:47:32 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/4169858908' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:47:32.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:47:32 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/2333774207' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T08:47:33.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:47:32 smithi148 ceph-mon[106692]: from='client.35966 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:47:33.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:47:32 smithi148 ceph-mon[106692]: pgmap v1100: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:47:33.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:47:32 smithi148 ceph-mon[106692]: from='client.35970 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:47:33.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:47:32 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/4169858908' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:47:33.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:47:32 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/2333774207' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T08:47:33.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:33.223+0000 7f23f1e00700 1 -- 172.21.15.37:0/1100881881 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f23ec04bed0 con 0x7f23ec0fde90 2024-03-29T08:47:33.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:33.223+0000 7f23e2a00700 1 -- 172.21.15.37:0/1100881881 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+3583 (secure 0 0 0) 0x7f23dc013420 con 0x7f23ec0fde90 2024-03-29T08:47:33.223 INFO:teuthology.orchestra.run.smithi037.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem is offline 2024-03-29T08:47:33.223 INFO:teuthology.orchestra.run.smithi037.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-03-29T08:47:33.223 INFO:teuthology.orchestra.run.smithi037.stdout: host smithi148 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/mon.smithi148/config 2024-03-29T08:47:33.223 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/podman: timeout after 895 seconds 2024-03-29T08:47:33.223 INFO:teuthology.orchestra.run.smithi037.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59 -e NODE_NAME=smithi148 -e CEPH_USE_RANDOM_NONCE=1 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4:/var/run/ceph:z -v /var/log/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4:/var/log/ceph:z -v /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpkx2rz2d4:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59 inventory --format=json-pretty --filter-for-batch 2024-03-29T08:47:33.223 INFO:teuthology.orchestra.run.smithi037.stdout:Traceback (most recent call last): 2024-03-29T08:47:33.223 INFO:teuthology.orchestra.run.smithi037.stdout: File "/usr/lib64/python3.6/runpy.py", line 193, in _run_module_as_main 2024-03-29T08:47:33.223 INFO:teuthology.orchestra.run.smithi037.stdout: "__main__", mod_spec) 2024-03-29T08:47:33.224 INFO:teuthology.orchestra.run.smithi037.stdout: File "/usr/lib64/python3.6/runpy.py", line 85, in _run_code 2024-03-29T08:47:33.224 INFO:teuthology.orchestra.run.smithi037.stdout: exec(code, run_globals) 2024-03-29T08:47:33.224 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 10889, in 2024-03-29T08:47:33.224 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 10877, in main 2024-03-29T08:47:33.224 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 2576, in _infer_config 2024-03-29T08:47:33.224 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 2492, in _infer_fsid 2024-03-29T08:47:33.224 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 2604, in _infer_image 2024-03-29T08:47:33.224 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 2479, in _validate_fsid 2024-03-29T08:47:33.224 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 7145, in command_ceph_volume 2024-03-29T08:47:33.224 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 2267, in call_throws 2024-03-29T08:47:33.224 INFO:teuthology.orchestra.run.smithi037.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59 -e NODE_NAME=smithi148 -e CEPH_USE_RANDOM_NONCE=1 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4:/var/run/ceph:z -v /var/log/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4:/var/log/ceph:z -v /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpkx2rz2d4:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59 inventory --format=json-pretty --filter-for-batch 2024-03-29T08:47:33.224 INFO:teuthology.orchestra.run.smithi037.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-03-29T08:47:33.224 INFO:teuthology.orchestra.run.smithi037.stdout: fs cephfs is degraded 2024-03-29T08:47:33.224 INFO:teuthology.orchestra.run.smithi037.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-03-29T08:47:33.224 INFO:teuthology.orchestra.run.smithi037.stdout: fs cephfs is offline because no MDS is active for it. 2024-03-29T08:47:33.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:33.225+0000 7f23f1e00700 1 -- 172.21.15.37:0/1100881881 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f23d006cc20 msgr2=0x7f23d006f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:47:33.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:33.226+0000 7f23f1e00700 1 --2- 172.21.15.37:0/1100881881 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f23d006cc20 0x7f23d006f0d0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f23ec1115c0 tx=0x7f23d800b040 comp rx=0 tx=0).stop 2024-03-29T08:47:33.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:33.226+0000 7f23f1e00700 1 -- 172.21.15.37:0/1100881881 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f23ec0fde90 msgr2=0x7f23ec115d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:47:33.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:33.226+0000 7f23f1e00700 1 --2- 172.21.15.37:0/1100881881 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f23ec0fde90 0x7f23ec115d20 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f23dc009360 tx=0x7f23dc009e30 comp rx=0 tx=0).stop 2024-03-29T08:47:33.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:33.226+0000 7f23f1e00700 1 -- 172.21.15.37:0/1100881881 shutdown_connections 2024-03-29T08:47:33.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:33.226+0000 7f23f1e00700 1 --2- 172.21.15.37:0/1100881881 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f23d006cc20 0x7f23d006f0d0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:47:33.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:33.226+0000 7f23f1e00700 1 --2- 172.21.15.37:0/1100881881 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f23ec0fde90 0x7f23ec115d20 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:47:33.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:33.226+0000 7f23f1e00700 1 --2- 172.21.15.37:0/1100881881 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f23ec0fe830 0x7f23ec116260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:47:33.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:33.226+0000 7f23f1e00700 1 -- 172.21.15.37:0/1100881881 >> 172.21.15.37:0/1100881881 conn(0x7f23ec072bc0 msgr2=0x7f23ec0fb8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:47:33.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:33.226+0000 7f23f1e00700 1 -- 172.21.15.37:0/1100881881 shutdown_connections 2024-03-29T08:47:33.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:47:33.226+0000 7f23f1e00700 1 -- 172.21.15.37:0/1100881881 wait complete. 2024-03-29T08:47:33.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:47:33 smithi037 ceph-mon[142325]: from='client.35982 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:47:33.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:47:33 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/1100881881' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T08:47:34.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:47:33 smithi148 ceph-mon[106692]: from='client.35982 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:47:34.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:47:33 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/1100881881' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T08:47:34.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:47:34 smithi037 ceph-mon[142325]: pgmap v1101: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:47:35.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:47:34 smithi148 ceph-mon[106692]: pgmap v1101: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:47:36.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:47:36 smithi037 ceph-mon[142325]: pgmap v1102: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:47:37.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:47:36 smithi148 ceph-mon[106692]: pgmap v1102: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:47:38.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:47:38 smithi037 ceph-mon[142325]: pgmap v1103: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:47:39.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:47:38 smithi148 ceph-mon[106692]: pgmap v1103: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:47:40.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:47:40 smithi037 ceph-mon[142325]: pgmap v1104: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:47:40.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:47:40 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:47:41.055 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:47:40 smithi148 ceph-mon[106692]: pgmap v1104: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:47:41.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:47:40 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:47:42.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:47:42 smithi037 ceph-mon[142325]: pgmap v1105: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:47:43.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:47:42 smithi148 ceph-mon[106692]: pgmap v1105: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:47:44.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:47:44 smithi037 ceph-mon[142325]: pgmap v1106: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:47:45.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:47:44 smithi148 ceph-mon[106692]: pgmap v1106: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:47:46.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:47:46 smithi037 ceph-mon[142325]: pgmap v1107: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:47:47.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:47:46 smithi148 ceph-mon[106692]: pgmap v1107: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:47:48.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:47:48 smithi037 ceph-mon[142325]: pgmap v1108: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:47:49.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:47:48 smithi148 ceph-mon[106692]: pgmap v1108: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:47:50.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:47:50 smithi037 ceph-mon[142325]: pgmap v1109: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:47:51.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:47:50 smithi148 ceph-mon[106692]: pgmap v1109: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:47:52.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:47:52 smithi037 ceph-mon[142325]: pgmap v1110: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:47:53.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:47:52 smithi148 ceph-mon[106692]: pgmap v1110: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:47:54.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:47:54 smithi037 ceph-mon[142325]: pgmap v1111: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:47:54.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:47:54 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:47:55.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:47:54 smithi148 ceph-mon[106692]: pgmap v1111: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:47:55.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:47:54 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:47:56.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:47:56 smithi037 ceph-mon[142325]: pgmap v1112: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:47:57.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:47:56 smithi148 ceph-mon[106692]: pgmap v1112: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:47:58.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:47:58 smithi037 ceph-mon[142325]: pgmap v1113: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:47:59.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:47:58 smithi148 ceph-mon[106692]: pgmap v1113: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:48:00.928 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:48:00 smithi037 ceph-mon[142325]: pgmap v1114: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:48:01.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:48:00 smithi148 ceph-mon[106692]: pgmap v1114: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:48:02.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:48:02 smithi037 ceph-mon[142325]: pgmap v1115: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:48:03.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:48:02 smithi148 ceph-mon[106692]: pgmap v1115: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:48:03.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:03.378+0000 7f1160800700 1 -- 172.21.15.37:0/1712849877 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1158070c70 msgr2=0x7f1158109110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:48:03.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:03.378+0000 7f1160800700 1 --2- 172.21.15.37:0/1712849877 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1158070c70 0x7f1158109110 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f1148004970 tx=0x7f1148009e30 comp rx=0 tx=0).stop 2024-03-29T08:48:03.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:03.378+0000 7f1160800700 1 -- 172.21.15.37:0/1712849877 shutdown_connections 2024-03-29T08:48:03.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:03.378+0000 7f1160800700 1 --2- 172.21.15.37:0/1712849877 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1158070c70 0x7f1158109110 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:48:03.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:03.378+0000 7f1160800700 1 --2- 172.21.15.37:0/1712849877 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1158070360 0x7f1158070730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:48:03.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:03.378+0000 7f1160800700 1 -- 172.21.15.37:0/1712849877 >> 172.21.15.37:0/1712849877 conn(0x7f11580f89b0 msgr2=0x7f11580fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:48:03.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:03.379+0000 7f1160800700 1 -- 172.21.15.37:0/1712849877 shutdown_connections 2024-03-29T08:48:03.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:03.379+0000 7f1160800700 1 -- 172.21.15.37:0/1712849877 wait complete. 2024-03-29T08:48:03.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:03.379+0000 7f1160800700 1 Processor -- start 2024-03-29T08:48:03.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:03.380+0000 7f1160800700 1 -- start start 2024-03-29T08:48:03.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:03.380+0000 7f1160800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1158070360 0x7f1158115ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:48:03.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:03.380+0000 7f1160800700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1158070c70 0x7f11581161e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:48:03.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:03.380+0000 7f1160800700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1158116830 con 0x7f1158070360 2024-03-29T08:48:03.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:03.380+0000 7f1160800700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f115810fc00 con 0x7f1158070c70 2024-03-29T08:48:03.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:03.380+0000 7f115de00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1158070360 0x7f1158115ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:48:03.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:03.380+0000 7f115de00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1158070360 0x7f1158115ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40760/0 (socket says 172.21.15.37:40760) 2024-03-29T08:48:03.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:03.380+0000 7f115de00700 1 -- 172.21.15.37:0/771889046 learned_addr learned my addr 172.21.15.37:0/771889046 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:48:03.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:03.380+0000 7f115d400700 1 --2- 172.21.15.37:0/771889046 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1158070c70 0x7f11581161e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:48:03.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:03.381+0000 7f115de00700 1 -- 172.21.15.37:0/771889046 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1158070c70 msgr2=0x7f11581161e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:48:03.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:03.381+0000 7f115de00700 1 --2- 172.21.15.37:0/771889046 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1158070c70 0x7f11581161e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:48:03.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:03.381+0000 7f115de00700 1 -- 172.21.15.37:0/771889046 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1148005040 con 0x7f1158070360 2024-03-29T08:48:03.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:03.381+0000 7f115d400700 1 --2- 172.21.15.37:0/771889046 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1158070c70 0x7f11581161e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:48:03.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:03.381+0000 7f115de00700 1 --2- 172.21.15.37:0/771889046 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1158070360 0x7f1158115ca0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f114c00c8c0 tx=0x7f114c0044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:48:03.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:03.381+0000 7f1152000700 1 -- 172.21.15.37:0/771889046 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f114c00a980 con 0x7f1158070360 2024-03-29T08:48:03.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:03.381+0000 7f1152000700 1 -- 172.21.15.37:0/771889046 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f114c00ce60 con 0x7f1158070360 2024-03-29T08:48:03.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:03.381+0000 7f1152000700 1 -- 172.21.15.37:0/771889046 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f114c0113a0 con 0x7f1158070360 2024-03-29T08:48:03.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:03.381+0000 7f1160800700 1 -- 172.21.15.37:0/771889046 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f115810fee0 con 0x7f1158070360 2024-03-29T08:48:03.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:03.381+0000 7f1160800700 1 -- 172.21.15.37:0/771889046 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1158110420 con 0x7f1158070360 2024-03-29T08:48:03.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:03.382+0000 7f1160800700 1 -- 172.21.15.37:0/771889046 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1158103bd0 con 0x7f1158070360 2024-03-29T08:48:03.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:03.384+0000 7f1152000700 1 -- 172.21.15.37:0/771889046 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f114c012400 con 0x7f1158070360 2024-03-29T08:48:03.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:03.385+0000 7f1152000700 1 --2- 172.21.15.37:0/771889046 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f114006cc30 0x7f114006f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:48:03.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:03.385+0000 7f1152000700 1 -- 172.21.15.37:0/771889046 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f114c08c0f0 con 0x7f1158070360 2024-03-29T08:48:03.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:03.385+0000 7f115d400700 1 --2- 172.21.15.37:0/771889046 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f114006cc30 0x7f114006f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:48:03.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:03.386+0000 7f115d400700 1 --2- 172.21.15.37:0/771889046 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f114006cc30 0x7f114006f0e0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f1158111540 tx=0x7f1148009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:48:03.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:03.390+0000 7f1152000700 1 -- 172.21.15.37:0/771889046 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f114c059bc0 con 0x7f1158070360 2024-03-29T08:48:03.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:03.626+0000 7f1160800700 1 -- 172.21.15.37:0/771889046 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f115810c020 con 0x7f114006cc30 2024-03-29T08:48:03.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:03.627+0000 7f1152000700 1 -- 172.21.15.37:0/771889046 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f115810c020 con 0x7f114006cc30 2024-03-29T08:48:03.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:03.630+0000 7f1160800700 1 -- 172.21.15.37:0/771889046 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f114006cc30 msgr2=0x7f114006f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:48:03.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:03.630+0000 7f1160800700 1 --2- 172.21.15.37:0/771889046 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f114006cc30 0x7f114006f0e0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f1158111540 tx=0x7f1148009e30 comp rx=0 tx=0).stop 2024-03-29T08:48:03.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:03.630+0000 7f1160800700 1 -- 172.21.15.37:0/771889046 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1158070360 msgr2=0x7f1158115ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:48:03.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:03.630+0000 7f1160800700 1 --2- 172.21.15.37:0/771889046 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1158070360 0x7f1158115ca0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f114c00c8c0 tx=0x7f114c0044c0 comp rx=0 tx=0).stop 2024-03-29T08:48:03.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:03.630+0000 7f1160800700 1 -- 172.21.15.37:0/771889046 shutdown_connections 2024-03-29T08:48:03.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:03.630+0000 7f1160800700 1 --2- 172.21.15.37:0/771889046 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f114006cc30 0x7f114006f0e0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:48:03.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:03.630+0000 7f1160800700 1 --2- 172.21.15.37:0/771889046 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1158070360 0x7f1158115ca0 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:48:03.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:03.630+0000 7f1160800700 1 --2- 172.21.15.37:0/771889046 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1158070c70 0x7f11581161e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:48:03.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:03.630+0000 7f1160800700 1 -- 172.21.15.37:0/771889046 >> 172.21.15.37:0/771889046 conn(0x7f11580f89b0 msgr2=0x7f11581015f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:48:03.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:03.630+0000 7f1160800700 1 -- 172.21.15.37:0/771889046 shutdown_connections 2024-03-29T08:48:03.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:03.630+0000 7f1160800700 1 -- 172.21.15.37:0/771889046 wait complete. 2024-03-29T08:48:03.644 INFO:teuthology.orchestra.run.smithi037.stdout:true 2024-03-29T08:48:03.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:03.770+0000 7f5d99400700 1 -- 172.21.15.37:0/2140206217 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5d941035e0 msgr2=0x7f5d940fcc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:48:03.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:03.770+0000 7f5d99400700 1 --2- 172.21.15.37:0/2140206217 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5d941035e0 0x7f5d940fcc50 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f5d80004940 tx=0x7f5d80009e30 comp rx=0 tx=0).stop 2024-03-29T08:48:03.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:03.770+0000 7f5d99400700 1 -- 172.21.15.37:0/2140206217 shutdown_connections 2024-03-29T08:48:03.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:03.770+0000 7f5d99400700 1 --2- 172.21.15.37:0/2140206217 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5d941035e0 0x7f5d940fcc50 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:48:03.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:03.770+0000 7f5d99400700 1 --2- 172.21.15.37:0/2140206217 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5d94102c40 0x7f5d94103010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:48:03.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:03.770+0000 7f5d99400700 1 -- 172.21.15.37:0/2140206217 >> 172.21.15.37:0/2140206217 conn(0x7f5d940f8830 msgr2=0x7f5d940fac40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:48:03.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:03.771+0000 7f5d99400700 1 -- 172.21.15.37:0/2140206217 shutdown_connections 2024-03-29T08:48:03.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:03.771+0000 7f5d99400700 1 -- 172.21.15.37:0/2140206217 wait complete. 2024-03-29T08:48:03.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:03.771+0000 7f5d99400700 1 Processor -- start 2024-03-29T08:48:03.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:03.772+0000 7f5d99400700 1 -- start start 2024-03-29T08:48:03.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:03.772+0000 7f5d99400700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5d94102c40 0x7f5d94115af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:48:03.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:03.772+0000 7f5d99400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5d941035e0 0x7f5d94116030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:48:03.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:03.772+0000 7f5d99400700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d94116680 con 0x7f5d941035e0 2024-03-29T08:48:03.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:03.772+0000 7f5d99400700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d9410fa70 con 0x7f5d94102c40 2024-03-29T08:48:03.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:03.772+0000 7f5d93400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5d941035e0 0x7f5d94116030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:48:03.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:03.772+0000 7f5d93400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5d941035e0 0x7f5d94116030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40776/0 (socket says 172.21.15.37:40776) 2024-03-29T08:48:03.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:03.772+0000 7f5d93400700 1 -- 172.21.15.37:0/3969493943 learned_addr learned my addr 172.21.15.37:0/3969493943 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:48:03.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:03.773+0000 7f5d93e00700 1 --2- 172.21.15.37:0/3969493943 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5d94102c40 0x7f5d94115af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:48:03.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:03.773+0000 7f5d93400700 1 -- 172.21.15.37:0/3969493943 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5d94102c40 msgr2=0x7f5d94115af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:48:03.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:03.773+0000 7f5d93400700 1 --2- 172.21.15.37:0/3969493943 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5d94102c40 0x7f5d94115af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:48:03.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:03.773+0000 7f5d93400700 1 -- 172.21.15.37:0/3969493943 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d80005040 con 0x7f5d941035e0 2024-03-29T08:48:03.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:03.773+0000 7f5d93400700 1 --2- 172.21.15.37:0/3969493943 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5d941035e0 0x7f5d94116030 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f5d80009390 tx=0x7f5d80009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:48:03.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:03.773+0000 7f5d8be00700 1 -- 172.21.15.37:0/3969493943 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5d8000e040 con 0x7f5d941035e0 2024-03-29T08:48:03.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:03.773+0000 7f5d8be00700 1 -- 172.21.15.37:0/3969493943 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f5d80013420 con 0x7f5d941035e0 2024-03-29T08:48:03.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:03.773+0000 7f5d8be00700 1 -- 172.21.15.37:0/3969493943 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5d8001c630 con 0x7f5d941035e0 2024-03-29T08:48:03.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:03.773+0000 7f5d99400700 1 -- 172.21.15.37:0/3969493943 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d9410fcf0 con 0x7f5d941035e0 2024-03-29T08:48:03.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:03.773+0000 7f5d99400700 1 -- 172.21.15.37:0/3969493943 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d941102b0 con 0x7f5d941035e0 2024-03-29T08:48:03.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:03.774+0000 7f5d99400700 1 -- 172.21.15.37:0/3969493943 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d940fe390 con 0x7f5d941035e0 2024-03-29T08:48:03.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:03.775+0000 7f5d8be00700 1 -- 172.21.15.37:0/3969493943 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f5d80013a00 con 0x7f5d941035e0 2024-03-29T08:48:03.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:03.776+0000 7f5d8be00700 1 --2- 172.21.15.37:0/3969493943 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f5d7806cc30 0x7f5d7806f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:48:03.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:03.776+0000 7f5d8be00700 1 -- 172.21.15.37:0/3969493943 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f5d8008d2d0 con 0x7f5d941035e0 2024-03-29T08:48:03.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:03.777+0000 7f5d93e00700 1 --2- 172.21.15.37:0/3969493943 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f5d7806cc30 0x7f5d7806f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:48:03.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:03.777+0000 7f5d93e00700 1 --2- 172.21.15.37:0/3969493943 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f5d7806cc30 0x7f5d7806f0e0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f5d84007fd0 tx=0x7f5d84009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:48:03.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:03.781+0000 7f5d8be00700 1 -- 172.21.15.37:0/3969493943 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f5d8005ada0 con 0x7f5d941035e0 2024-03-29T08:48:04.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:04.013+0000 7f5d99400700 1 -- 172.21.15.37:0/3969493943 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5d94109bd0 con 0x7f5d7806cc30 2024-03-29T08:48:04.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:04.014+0000 7f5d8be00700 1 -- 172.21.15.37:0/3969493943 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f5d94109bd0 con 0x7f5d7806cc30 2024-03-29T08:48:04.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:04.016+0000 7f5d99400700 1 -- 172.21.15.37:0/3969493943 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f5d7806cc30 msgr2=0x7f5d7806f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:48:04.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:04.016+0000 7f5d99400700 1 --2- 172.21.15.37:0/3969493943 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f5d7806cc30 0x7f5d7806f0e0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f5d84007fd0 tx=0x7f5d84009040 comp rx=0 tx=0).stop 2024-03-29T08:48:04.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:04.016+0000 7f5d99400700 1 -- 172.21.15.37:0/3969493943 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5d941035e0 msgr2=0x7f5d94116030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:48:04.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:04.016+0000 7f5d99400700 1 --2- 172.21.15.37:0/3969493943 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5d941035e0 0x7f5d94116030 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f5d80009390 tx=0x7f5d80009e30 comp rx=0 tx=0).stop 2024-03-29T08:48:04.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:04.016+0000 7f5d99400700 1 -- 172.21.15.37:0/3969493943 shutdown_connections 2024-03-29T08:48:04.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:04.017+0000 7f5d99400700 1 --2- 172.21.15.37:0/3969493943 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f5d7806cc30 0x7f5d7806f0e0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:48:04.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:04.017+0000 7f5d99400700 1 --2- 172.21.15.37:0/3969493943 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5d94102c40 0x7f5d94115af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:48:04.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:04.017+0000 7f5d99400700 1 --2- 172.21.15.37:0/3969493943 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5d941035e0 0x7f5d94116030 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:48:04.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:04.017+0000 7f5d99400700 1 -- 172.21.15.37:0/3969493943 >> 172.21.15.37:0/3969493943 conn(0x7f5d940f8830 msgr2=0x7f5d940f9870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:48:04.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:04.017+0000 7f5d99400700 1 -- 172.21.15.37:0/3969493943 shutdown_connections 2024-03-29T08:48:04.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:04.017+0000 7f5d99400700 1 -- 172.21.15.37:0/3969493943 wait complete. 2024-03-29T08:48:04.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:04.153+0000 7fa6a2200700 1 -- 172.21.15.37:0/2697992540 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa69c1035e0 msgr2=0x7fa69c0fcc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:48:04.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:04.153+0000 7fa6a2200700 1 --2- 172.21.15.37:0/2697992540 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa69c1035e0 0x7fa69c0fcc50 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7fa688004940 tx=0x7fa688009e30 comp rx=0 tx=0).stop 2024-03-29T08:48:04.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:04.153+0000 7fa6a2200700 1 -- 172.21.15.37:0/2697992540 shutdown_connections 2024-03-29T08:48:04.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:04.153+0000 7fa6a2200700 1 --2- 172.21.15.37:0/2697992540 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa69c1035e0 0x7fa69c0fcc50 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:48:04.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:04.153+0000 7fa6a2200700 1 --2- 172.21.15.37:0/2697992540 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa69c102c40 0x7fa69c103010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:48:04.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:04.153+0000 7fa6a2200700 1 -- 172.21.15.37:0/2697992540 >> 172.21.15.37:0/2697992540 conn(0x7fa69c0f8830 msgr2=0x7fa69c0fac40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:48:04.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:04.153+0000 7fa6a2200700 1 -- 172.21.15.37:0/2697992540 shutdown_connections 2024-03-29T08:48:04.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:04.154+0000 7fa6a2200700 1 -- 172.21.15.37:0/2697992540 wait complete. 2024-03-29T08:48:04.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:04.154+0000 7fa6a2200700 1 Processor -- start 2024-03-29T08:48:04.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:04.154+0000 7fa6a2200700 1 -- start start 2024-03-29T08:48:04.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:04.155+0000 7fa6a2200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa69c102c40 0x7fa69c115a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:48:04.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:04.155+0000 7fa6a2200700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa69c1035e0 0x7fa69c115fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:48:04.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:04.155+0000 7fa6a2200700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa69c116610 con 0x7fa69c102c40 2024-03-29T08:48:04.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:04.155+0000 7fa6a2200700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa69c10fa00 con 0x7fa69c1035e0 2024-03-29T08:48:04.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:04.155+0000 7fa6a0e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa69c102c40 0x7fa69c115a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:48:04.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:04.155+0000 7fa6a0e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa69c102c40 0x7fa69c115a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40786/0 (socket says 172.21.15.37:40786) 2024-03-29T08:48:04.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:04.155+0000 7fa6a0e00700 1 -- 172.21.15.37:0/284361714 learned_addr learned my addr 172.21.15.37:0/284361714 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:48:04.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:04.155+0000 7fa69be00700 1 --2- 172.21.15.37:0/284361714 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa69c1035e0 0x7fa69c115fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:48:04.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:04.156+0000 7fa6a0e00700 1 -- 172.21.15.37:0/284361714 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa69c1035e0 msgr2=0x7fa69c115fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:48:04.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:04.156+0000 7fa6a0e00700 1 --2- 172.21.15.37:0/284361714 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa69c1035e0 0x7fa69c115fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:48:04.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:04.156+0000 7fa6a0e00700 1 -- 172.21.15.37:0/284361714 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa688005040 con 0x7fa69c102c40 2024-03-29T08:48:04.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:04.156+0000 7fa6a0e00700 1 --2- 172.21.15.37:0/284361714 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa69c102c40 0x7fa69c115a80 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7fa690002a80 tx=0x7fa6900046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:48:04.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:04.156+0000 7fa698c00700 1 -- 172.21.15.37:0/284361714 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa690007cc0 con 0x7fa69c102c40 2024-03-29T08:48:04.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:04.156+0000 7fa698c00700 1 -- 172.21.15.37:0/284361714 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fa69000f040 con 0x7fa69c102c40 2024-03-29T08:48:04.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:04.156+0000 7fa698c00700 1 -- 172.21.15.37:0/284361714 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa69000da60 con 0x7fa69c102c40 2024-03-29T08:48:04.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:04.156+0000 7fa6a2200700 1 -- 172.21.15.37:0/284361714 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa69c10fce0 con 0x7fa69c102c40 2024-03-29T08:48:04.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:04.156+0000 7fa6a2200700 1 -- 172.21.15.37:0/284361714 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa69c110270 con 0x7fa69c102c40 2024-03-29T08:48:04.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:04.157+0000 7fa6a2200700 1 -- 172.21.15.37:0/284361714 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa69c0fe390 con 0x7fa69c102c40 2024-03-29T08:48:04.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:04.158+0000 7fa698c00700 1 -- 172.21.15.37:0/284361714 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fa69000dbc0 con 0x7fa69c102c40 2024-03-29T08:48:04.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:04.159+0000 7fa698c00700 1 --2- 172.21.15.37:0/284361714 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fa68406cc30 0x7fa68406f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:48:04.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:04.159+0000 7fa698c00700 1 -- 172.21.15.37:0/284361714 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fa69008bc70 con 0x7fa69c102c40 2024-03-29T08:48:04.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:04.159+0000 7fa69be00700 1 --2- 172.21.15.37:0/284361714 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fa68406cc30 0x7fa68406f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:48:04.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:04.160+0000 7fa69be00700 1 --2- 172.21.15.37:0/284361714 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fa68406cc30 0x7fa68406f0e0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fa69c110ce0 tx=0x7fa688009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:48:04.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:04.164+0000 7fa698c00700 1 -- 172.21.15.37:0/284361714 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fa690092080 con 0x7fa69c102c40 2024-03-29T08:48:04.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:04.386+0000 7fa6a2200700 1 -- 172.21.15.37:0/284361714 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fa69c109bd0 con 0x7fa68406cc30 2024-03-29T08:48:04.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:04.396+0000 7fa698c00700 1 -- 172.21.15.37:0/284361714 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7fa69c109bd0 con 0x7fa68406cc30 2024-03-29T08:48:04.396 INFO:teuthology.orchestra.run.smithi037.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-29T08:48:04.396 INFO:teuthology.orchestra.run.smithi037.stdout:alertmanager.smithi037 smithi037 *:9093,9094 running (37m) 14m ago 50m 14.3M - 0.25.0 c8568f914cd2 2f9f9746ed0f 2024-03-29T08:48:04.396 INFO:teuthology.orchestra.run.smithi037.stdout:crash.smithi037 smithi037 running (34m) 14m ago 51m 6647k - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 178d416a3186 2024-03-29T08:48:04.396 INFO:teuthology.orchestra.run.smithi037.stdout:crash.smithi148 smithi148 running (34m) 14m ago 49m 6656k - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 23666989d58f 2024-03-29T08:48:04.396 INFO:teuthology.orchestra.run.smithi037.stdout:grafana.smithi037 smithi037 *:3000 running (37m) 14m ago 50m 74.5M - 9.4.7 954c08fa6188 6b976fe49952 2024-03-29T08:48:04.396 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi037.iifsmh smithi037 running (30m) 14m ago 44m 16.4M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 2d97f562cc5b 2024-03-29T08:48:04.396 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi037.mmwgxe smithi037 running (30m) 14m ago 44m 16.7M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff ec37037f46e8 2024-03-29T08:48:04.396 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi148.qukpjh smithi148 running (14m) 14m ago 44m 10.7M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 3257219bc046 2024-03-29T08:48:04.397 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi148.sqteih smithi148 running (29m) 14m ago 44m 16.3M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 44c15c77037a 2024-03-29T08:48:04.397 INFO:teuthology.orchestra.run.smithi037.stdout:mgr.smithi037.wjikii smithi037 *:8443,9283,8765 running (40m) 14m ago 52m 510M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff c5a1417b9417 2024-03-29T08:48:04.397 INFO:teuthology.orchestra.run.smithi037.stdout:mgr.smithi148.onhgvm smithi148 *:8443,9283,8765 running (39m) 14m ago 49m 456M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff fcfdbce062bd 2024-03-29T08:48:04.397 INFO:teuthology.orchestra.run.smithi037.stdout:mon.smithi037 smithi037 running (36m) 14m ago 52m 70.2M 2048M 18.2.2-1015-gd0e392d9 1cc2f45c17ff eea2060783af 2024-03-29T08:48:04.397 INFO:teuthology.orchestra.run.smithi037.stdout:mon.smithi148 smithi148 running (35m) 14m ago 49m 50.4M 2048M 18.2.2-1015-gd0e392d9 1cc2f45c17ff ad4b2c5748be 2024-03-29T08:48:04.397 INFO:teuthology.orchestra.run.smithi037.stdout:node-exporter.smithi037 smithi037 *:9100 running (38m) 14m ago 51m 8514k - 1.5.0 0da6a335fe13 26b8ae9cb878 2024-03-29T08:48:04.397 INFO:teuthology.orchestra.run.smithi037.stdout:node-exporter.smithi148 smithi148 *:9100 running (38m) 14m ago 49m 8464k - 1.5.0 0da6a335fe13 98414e4e64ed 2024-03-29T08:48:04.397 INFO:teuthology.orchestra.run.smithi037.stdout:osd.0 smithi037 running (34m) 14m ago 48m 108M 1619M 18.2.2-1015-gd0e392d9 1cc2f45c17ff c0c584f6245c 2024-03-29T08:48:04.397 INFO:teuthology.orchestra.run.smithi037.stdout:osd.1 smithi037 running (33m) 14m ago 48m 104M 1619M 18.2.2-1015-gd0e392d9 1cc2f45c17ff a4a2ff4cc83d 2024-03-29T08:48:04.397 INFO:teuthology.orchestra.run.smithi037.stdout:osd.2 smithi037 running (32m) 14m ago 47m 85.9M 1619M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 8b1bc71aa79e 2024-03-29T08:48:04.397 INFO:teuthology.orchestra.run.smithi037.stdout:osd.3 smithi148 running (32m) 14m ago 47m 109M 2642M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 1c656ab8ff86 2024-03-29T08:48:04.397 INFO:teuthology.orchestra.run.smithi037.stdout:osd.4 smithi148 running (31m) 14m ago 46m 101M 2642M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 966a9de11d22 2024-03-29T08:48:04.397 INFO:teuthology.orchestra.run.smithi037.stdout:osd.5 smithi148 running (31m) 14m ago 46m 89.4M 2642M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 474c4f9212ce 2024-03-29T08:48:04.397 INFO:teuthology.orchestra.run.smithi037.stdout:prometheus.smithi037 smithi037 *:9095 running (38m) 14m ago 50m 61.0M - 2.43.0 a07b618ecd1d 30c0223ec589 2024-03-29T08:48:04.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:04.398+0000 7fa6a2200700 1 -- 172.21.15.37:0/284361714 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fa68406cc30 msgr2=0x7fa68406f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:48:04.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:04.398+0000 7fa6a2200700 1 --2- 172.21.15.37:0/284361714 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fa68406cc30 0x7fa68406f0e0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fa69c110ce0 tx=0x7fa688009e30 comp rx=0 tx=0).stop 2024-03-29T08:48:04.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:04.398+0000 7fa6a2200700 1 -- 172.21.15.37:0/284361714 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa69c102c40 msgr2=0x7fa69c115a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:48:04.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:04.398+0000 7fa6a2200700 1 --2- 172.21.15.37:0/284361714 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa69c102c40 0x7fa69c115a80 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7fa690002a80 tx=0x7fa6900046b0 comp rx=0 tx=0).stop 2024-03-29T08:48:04.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:04.398+0000 7fa6a2200700 1 -- 172.21.15.37:0/284361714 shutdown_connections 2024-03-29T08:48:04.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:04.398+0000 7fa6a2200700 1 --2- 172.21.15.37:0/284361714 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fa68406cc30 0x7fa68406f0e0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:48:04.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:04.399+0000 7fa6a2200700 1 --2- 172.21.15.37:0/284361714 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa69c102c40 0x7fa69c115a80 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:48:04.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:04.399+0000 7fa6a2200700 1 --2- 172.21.15.37:0/284361714 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa69c1035e0 0x7fa69c115fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:48:04.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:04.399+0000 7fa6a2200700 1 -- 172.21.15.37:0/284361714 >> 172.21.15.37:0/284361714 conn(0x7fa69c0f8830 msgr2=0x7fa69c0f9830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:48:04.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:04.399+0000 7fa6a2200700 1 -- 172.21.15.37:0/284361714 shutdown_connections 2024-03-29T08:48:04.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:04.399+0000 7fa6a2200700 1 -- 172.21.15.37:0/284361714 wait complete. 2024-03-29T08:48:04.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:04.536+0000 7fe5ed000700 1 -- 172.21.15.37:0/3255079833 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe5e81037a0 msgr2=0x7fe5e80fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:48:04.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:04.536+0000 7fe5ed000700 1 --2- 172.21.15.37:0/3255079833 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe5e81037a0 0x7fe5e80fce10 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7fe5d4004940 tx=0x7fe5d4009e30 comp rx=0 tx=0).stop 2024-03-29T08:48:04.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:04.536+0000 7fe5ed000700 1 -- 172.21.15.37:0/3255079833 shutdown_connections 2024-03-29T08:48:04.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:04.536+0000 7fe5ed000700 1 --2- 172.21.15.37:0/3255079833 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe5e81037a0 0x7fe5e80fce10 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:48:04.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:04.536+0000 7fe5ed000700 1 --2- 172.21.15.37:0/3255079833 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe5e8102e00 0x7fe5e81031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:48:04.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:04.536+0000 7fe5ed000700 1 -- 172.21.15.37:0/3255079833 >> 172.21.15.37:0/3255079833 conn(0x7fe5e80f89b0 msgr2=0x7fe5e80fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:48:04.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:04.536+0000 7fe5ed000700 1 -- 172.21.15.37:0/3255079833 shutdown_connections 2024-03-29T08:48:04.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:04.536+0000 7fe5ed000700 1 -- 172.21.15.37:0/3255079833 wait complete. 2024-03-29T08:48:04.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:04.537+0000 7fe5ed000700 1 Processor -- start 2024-03-29T08:48:04.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:04.537+0000 7fe5ed000700 1 -- start start 2024-03-29T08:48:04.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:04.537+0000 7fe5ed000700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe5e8102e00 0x7fe5e8115d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:48:04.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:04.537+0000 7fe5ed000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe5e81037a0 0x7fe5e8116240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:48:04.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:04.537+0000 7fe5ed000700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5e8116890 con 0x7fe5e81037a0 2024-03-29T08:48:04.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:04.537+0000 7fe5ed000700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5e810fc60 con 0x7fe5e8102e00 2024-03-29T08:48:04.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:04.538+0000 7fe5e5400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe5e81037a0 0x7fe5e8116240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:48:04.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:04.538+0000 7fe5e5400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe5e81037a0 0x7fe5e8116240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40804/0 (socket says 172.21.15.37:40804) 2024-03-29T08:48:04.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:04.538+0000 7fe5e5400700 1 -- 172.21.15.37:0/1989564571 learned_addr learned my addr 172.21.15.37:0/1989564571 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:48:04.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:04.538+0000 7fe5e5400700 1 -- 172.21.15.37:0/1989564571 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe5e8102e00 msgr2=0x7fe5e8115d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:48:04.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:04.538+0000 7fe5e5400700 1 --2- 172.21.15.37:0/1989564571 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe5e8102e00 0x7fe5e8115d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:48:04.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:04.538+0000 7fe5e5400700 1 -- 172.21.15.37:0/1989564571 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5d4005040 con 0x7fe5e81037a0 2024-03-29T08:48:04.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:04.538+0000 7fe5e5400700 1 --2- 172.21.15.37:0/1989564571 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe5e81037a0 0x7fe5e8116240 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7fe5d4009390 tx=0x7fe5d4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:48:04.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:04.539+0000 7fe5de000700 1 -- 172.21.15.37:0/1989564571 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe5d400e040 con 0x7fe5e81037a0 2024-03-29T08:48:04.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:04.539+0000 7fe5de000700 1 -- 172.21.15.37:0/1989564571 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fe5d4013420 con 0x7fe5e81037a0 2024-03-29T08:48:04.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:04.539+0000 7fe5de000700 1 -- 172.21.15.37:0/1989564571 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe5d401c630 con 0x7fe5e81037a0 2024-03-29T08:48:04.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:04.539+0000 7fe5ed000700 1 -- 172.21.15.37:0/1989564571 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5e810fee0 con 0x7fe5e81037a0 2024-03-29T08:48:04.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:04.539+0000 7fe5ed000700 1 -- 172.21.15.37:0/1989564571 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5e8110450 con 0x7fe5e81037a0 2024-03-29T08:48:04.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:04.540+0000 7fe5ed000700 1 -- 172.21.15.37:0/1989564571 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5e80fe550 con 0x7fe5e81037a0 2024-03-29T08:48:04.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:04.541+0000 7fe5de000700 1 -- 172.21.15.37:0/1989564571 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fe5d4013a00 con 0x7fe5e81037a0 2024-03-29T08:48:04.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:04.542+0000 7fe5de000700 1 --2- 172.21.15.37:0/1989564571 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fe5cc06cc30 0x7fe5cc06f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:48:04.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:04.542+0000 7fe5de000700 1 -- 172.21.15.37:0/1989564571 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fe5d408d310 con 0x7fe5e81037a0 2024-03-29T08:48:04.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:04.542+0000 7fe5e5e00700 1 --2- 172.21.15.37:0/1989564571 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fe5cc06cc30 0x7fe5cc06f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:48:04.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:04.543+0000 7fe5e5e00700 1 --2- 172.21.15.37:0/1989564571 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fe5cc06cc30 0x7fe5cc06f0e0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fe5d8004d00 tx=0x7fe5d8009ce0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:48:04.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:04.546+0000 7fe5de000700 1 -- 172.21.15.37:0/1989564571 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fe5d405ade0 con 0x7fe5e81037a0 2024-03-29T08:48:04.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:04.851+0000 7fe5ed000700 1 -- 172.21.15.37:0/1989564571 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fe5e804bed0 con 0x7fe5e81037a0 2024-03-29T08:48:04.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:04.852+0000 7fe5de000700 1 -- 172.21.15.37:0/1989564571 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7fe5d405a970 con 0x7fe5e81037a0 2024-03-29T08:48:04.851 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-29T08:48:04.851 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": { 2024-03-29T08:48:04.852 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 2 2024-03-29T08:48:04.852 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:48:04.852 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": { 2024-03-29T08:48:04.852 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 2 2024-03-29T08:48:04.852 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:48:04.852 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": { 2024-03-29T08:48:04.852 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 6 2024-03-29T08:48:04.852 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:48:04.852 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": { 2024-03-29T08:48:04.852 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 4 2024-03-29T08:48:04.852 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:48:04.852 INFO:teuthology.orchestra.run.smithi037.stdout: "overall": { 2024-03-29T08:48:04.852 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 14 2024-03-29T08:48:04.852 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-29T08:48:04.853 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-29T08:48:04.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:04.854+0000 7fe5ed000700 1 -- 172.21.15.37:0/1989564571 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fe5cc06cc30 msgr2=0x7fe5cc06f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:48:04.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:04.854+0000 7fe5ed000700 1 --2- 172.21.15.37:0/1989564571 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fe5cc06cc30 0x7fe5cc06f0e0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fe5d8004d00 tx=0x7fe5d8009ce0 comp rx=0 tx=0).stop 2024-03-29T08:48:04.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:04.854+0000 7fe5ed000700 1 -- 172.21.15.37:0/1989564571 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe5e81037a0 msgr2=0x7fe5e8116240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:48:04.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:04.854+0000 7fe5ed000700 1 --2- 172.21.15.37:0/1989564571 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe5e81037a0 0x7fe5e8116240 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7fe5d4009390 tx=0x7fe5d4009e30 comp rx=0 tx=0).stop 2024-03-29T08:48:04.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:04.854+0000 7fe5ed000700 1 -- 172.21.15.37:0/1989564571 shutdown_connections 2024-03-29T08:48:04.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:04.854+0000 7fe5ed000700 1 --2- 172.21.15.37:0/1989564571 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fe5cc06cc30 0x7fe5cc06f0e0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:48:04.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:04.854+0000 7fe5ed000700 1 --2- 172.21.15.37:0/1989564571 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe5e8102e00 0x7fe5e8115d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:48:04.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:04.854+0000 7fe5ed000700 1 --2- 172.21.15.37:0/1989564571 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe5e81037a0 0x7fe5e8116240 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:48:04.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:04.854+0000 7fe5ed000700 1 -- 172.21.15.37:0/1989564571 >> 172.21.15.37:0/1989564571 conn(0x7fe5e80f89b0 msgr2=0x7fe5e80f9ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:48:04.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:04.854+0000 7fe5ed000700 1 -- 172.21.15.37:0/1989564571 shutdown_connections 2024-03-29T08:48:04.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:04.854+0000 7fe5ed000700 1 -- 172.21.15.37:0/1989564571 wait complete. 2024-03-29T08:48:04.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:48:04 smithi037 ceph-mon[142325]: pgmap v1116: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:48:04.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:48:04 smithi037 ceph-mon[142325]: from='client.35990 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:48:04.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:04.991+0000 7fbdc8400700 1 -- 172.21.15.37:0/1822455709 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbdc0102e00 msgr2=0x7fbdc01031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:48:04.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:04.991+0000 7fbdc8400700 1 --2- 172.21.15.37:0/1822455709 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbdc0102e00 0x7fbdc01031d0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7fbdb4004940 tx=0x7fbdb4009e30 comp rx=0 tx=0).stop 2024-03-29T08:48:04.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:04.991+0000 7fbdc8400700 1 -- 172.21.15.37:0/1822455709 shutdown_connections 2024-03-29T08:48:04.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:04.991+0000 7fbdc8400700 1 --2- 172.21.15.37:0/1822455709 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbdc01037a0 0x7fbdc00fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:48:04.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:04.991+0000 7fbdc8400700 1 --2- 172.21.15.37:0/1822455709 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbdc0102e00 0x7fbdc01031d0 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:48:04.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:04.991+0000 7fbdc8400700 1 -- 172.21.15.37:0/1822455709 >> 172.21.15.37:0/1822455709 conn(0x7fbdc00f89b0 msgr2=0x7fbdc00fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:48:04.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:04.991+0000 7fbdc8400700 1 -- 172.21.15.37:0/1822455709 shutdown_connections 2024-03-29T08:48:04.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:04.991+0000 7fbdc8400700 1 -- 172.21.15.37:0/1822455709 wait complete. 2024-03-29T08:48:04.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:04.992+0000 7fbdc8400700 1 Processor -- start 2024-03-29T08:48:04.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:04.992+0000 7fbdc8400700 1 -- start start 2024-03-29T08:48:04.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:04.992+0000 7fbdc8400700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbdc0102e00 0x7fbdc00702c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:48:04.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:04.992+0000 7fbdc8400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbdc01037a0 0x7fbdc0070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:48:04.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:04.992+0000 7fbdc8400700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdc0070d90 con 0x7fbdc01037a0 2024-03-29T08:48:04.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:04.992+0000 7fbdc8400700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdc0070ed0 con 0x7fbdc0102e00 2024-03-29T08:48:04.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:04.993+0000 7fbdc5000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbdc01037a0 0x7fbdc0070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:48:04.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:04.993+0000 7fbdc5000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbdc01037a0 0x7fbdc0070800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40820/0 (socket says 172.21.15.37:40820) 2024-03-29T08:48:04.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:04.993+0000 7fbdc5000700 1 -- 172.21.15.37:0/3003164887 learned_addr learned my addr 172.21.15.37:0/3003164887 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:48:04.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:04.993+0000 7fbdc5a00700 1 --2- 172.21.15.37:0/3003164887 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbdc0102e00 0x7fbdc00702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:48:04.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:04.993+0000 7fbdc5000700 1 -- 172.21.15.37:0/3003164887 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbdc0102e00 msgr2=0x7fbdc00702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:48:04.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:04.993+0000 7fbdc5000700 1 --2- 172.21.15.37:0/3003164887 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbdc0102e00 0x7fbdc00702c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:48:04.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:04.993+0000 7fbdc5000700 1 -- 172.21.15.37:0/3003164887 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdb4005040 con 0x7fbdc01037a0 2024-03-29T08:48:04.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:04.993+0000 7fbdc5a00700 1 --2- 172.21.15.37:0/3003164887 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbdc0102e00 0x7fbdc00702c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:48:04.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:04.993+0000 7fbdc5000700 1 --2- 172.21.15.37:0/3003164887 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbdc01037a0 0x7fbdc0070800 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7fbdac002a50 tx=0x7fbdac004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:48:04.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:04.993+0000 7fbdbd600700 1 -- 172.21.15.37:0/3003164887 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbdac007cc0 con 0x7fbdc01037a0 2024-03-29T08:48:04.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:04.994+0000 7fbdbd600700 1 -- 172.21.15.37:0/3003164887 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fbdac00f040 con 0x7fbdc01037a0 2024-03-29T08:48:04.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:04.994+0000 7fbdbd600700 1 -- 172.21.15.37:0/3003164887 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbdac00d9a0 con 0x7fbdc01037a0 2024-03-29T08:48:04.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:04.994+0000 7fbdc8400700 1 -- 172.21.15.37:0/3003164887 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbdc0119e20 con 0x7fbdc01037a0 2024-03-29T08:48:04.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:04.994+0000 7fbdc8400700 1 -- 172.21.15.37:0/3003164887 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbdc011a2a0 con 0x7fbdc01037a0 2024-03-29T08:48:04.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:04.995+0000 7fbdc8400700 1 -- 172.21.15.37:0/3003164887 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbdc00fe550 con 0x7fbdc01037a0 2024-03-29T08:48:04.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:04.996+0000 7fbdbd600700 1 -- 172.21.15.37:0/3003164887 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fbdac00dc10 con 0x7fbdc01037a0 2024-03-29T08:48:04.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:04.997+0000 7fbdbd600700 1 --2- 172.21.15.37:0/3003164887 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fbda806cc30 0x7fbda806f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:48:04.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:04.997+0000 7fbdbd600700 1 -- 172.21.15.37:0/3003164887 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fbdac08be20 con 0x7fbdc01037a0 2024-03-29T08:48:04.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:04.997+0000 7fbdc5a00700 1 --2- 172.21.15.37:0/3003164887 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fbda806cc30 0x7fbda806f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:48:04.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:04.998+0000 7fbdc5a00700 1 --2- 172.21.15.37:0/3003164887 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fbda806cc30 0x7fbda806f0e0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fbdb400b840 tx=0x7fbdb4009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:48:05.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:05.001+0000 7fbdbd600700 1 -- 172.21.15.37:0/3003164887 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fbdac055de0 con 0x7fbdc01037a0 2024-03-29T08:48:05.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:48:04 smithi148 ceph-mon[106692]: pgmap v1116: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:48:05.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:48:04 smithi148 ceph-mon[106692]: from='client.35990 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:48:05.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:05.257+0000 7fbdc8400700 1 -- 172.21.15.37:0/3003164887 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fbdc004bed0 con 0x7fbdc01037a0 2024-03-29T08:48:05.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:05.258+0000 7fbdbd600700 1 -- 172.21.15.37:0/3003164887 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 30 v30) v1 ==== 76+0+1841 (secure 0 0 0) 0x7fbdac059400 con 0x7fbdc01037a0 2024-03-29T08:48:05.258 INFO:teuthology.orchestra.run.smithi037.stdout:e30 2024-03-29T08:48:05.258 INFO:teuthology.orchestra.run.smithi037.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-29T08:48:05.258 INFO:teuthology.orchestra.run.smithi037.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T08:48:05.258 INFO:teuthology.orchestra.run.smithi037.stdout:legacy client fscid: 1 2024-03-29T08:48:05.258 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:48:05.258 INFO:teuthology.orchestra.run.smithi037.stdout:Filesystem 'cephfs' (1) 2024-03-29T08:48:05.258 INFO:teuthology.orchestra.run.smithi037.stdout:fs_name cephfs 2024-03-29T08:48:05.258 INFO:teuthology.orchestra.run.smithi037.stdout:epoch 21 2024-03-29T08:48:05.258 INFO:teuthology.orchestra.run.smithi037.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-03-29T08:48:05.259 INFO:teuthology.orchestra.run.smithi037.stdout:created 2024-03-29T08:03:29.241269+0000 2024-03-29T08:48:05.259 INFO:teuthology.orchestra.run.smithi037.stdout:modified 2024-03-29T08:17:16.237238+0000 2024-03-29T08:48:05.259 INFO:teuthology.orchestra.run.smithi037.stdout:tableserver 0 2024-03-29T08:48:05.259 INFO:teuthology.orchestra.run.smithi037.stdout:root 0 2024-03-29T08:48:05.259 INFO:teuthology.orchestra.run.smithi037.stdout:session_timeout 60 2024-03-29T08:48:05.259 INFO:teuthology.orchestra.run.smithi037.stdout:session_autoclose 300 2024-03-29T08:48:05.259 INFO:teuthology.orchestra.run.smithi037.stdout:max_file_size 1099511627776 2024-03-29T08:48:05.259 INFO:teuthology.orchestra.run.smithi037.stdout:max_xattr_size 65536 2024-03-29T08:48:05.259 INFO:teuthology.orchestra.run.smithi037.stdout:required_client_features {} 2024-03-29T08:48:05.259 INFO:teuthology.orchestra.run.smithi037.stdout:last_failure 0 2024-03-29T08:48:05.259 INFO:teuthology.orchestra.run.smithi037.stdout:last_failure_osd_epoch 82 2024-03-29T08:48:05.259 INFO:teuthology.orchestra.run.smithi037.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T08:48:05.259 INFO:teuthology.orchestra.run.smithi037.stdout:max_mds 2 2024-03-29T08:48:05.259 INFO:teuthology.orchestra.run.smithi037.stdout:in 0,1 2024-03-29T08:48:05.259 INFO:teuthology.orchestra.run.smithi037.stdout:up {} 2024-03-29T08:48:05.260 INFO:teuthology.orchestra.run.smithi037.stdout:failed 0,1 2024-03-29T08:48:05.260 INFO:teuthology.orchestra.run.smithi037.stdout:damaged 2024-03-29T08:48:05.260 INFO:teuthology.orchestra.run.smithi037.stdout:stopped 2024-03-29T08:48:05.260 INFO:teuthology.orchestra.run.smithi037.stdout:data_pools [3] 2024-03-29T08:48:05.260 INFO:teuthology.orchestra.run.smithi037.stdout:metadata_pool 2 2024-03-29T08:48:05.260 INFO:teuthology.orchestra.run.smithi037.stdout:inline_data disabled 2024-03-29T08:48:05.260 INFO:teuthology.orchestra.run.smithi037.stdout:balancer 2024-03-29T08:48:05.260 INFO:teuthology.orchestra.run.smithi037.stdout:bal_rank_mask -1 2024-03-29T08:48:05.260 INFO:teuthology.orchestra.run.smithi037.stdout:standby_count_wanted 1 2024-03-29T08:48:05.260 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:48:05.260 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:48:05.260 INFO:teuthology.orchestra.run.smithi037.stdout:Standby daemons: 2024-03-29T08:48:05.260 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:48:05.260 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi037.mmwgxe{-1:34440} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.37:6828/1248142005,v1:172.21.15.37:6829/1248142005] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:48:05.260 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi037.iifsmh{-1:34444} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.37:6826/2060882959,v1:172.21.15.37:6827/2060882959] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:48:05.261 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi148.sqteih{-1:44395} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.148:6824/4078652242,v1:172.21.15.148:6825/4078652242] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:48:05.261 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi148.qukpjh{-1:45007} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.148:6826/2394751105,v1:172.21.15.148:6827/2394751105] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:48:05.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:05.260+0000 7fbdc8400700 1 -- 172.21.15.37:0/3003164887 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fbda806cc30 msgr2=0x7fbda806f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:48:05.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:05.260+0000 7fbdc8400700 1 --2- 172.21.15.37:0/3003164887 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fbda806cc30 0x7fbda806f0e0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fbdb400b840 tx=0x7fbdb4009e30 comp rx=0 tx=0).stop 2024-03-29T08:48:05.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:05.260+0000 7fbdc8400700 1 -- 172.21.15.37:0/3003164887 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbdc01037a0 msgr2=0x7fbdc0070800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:48:05.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:05.260+0000 7fbdc8400700 1 --2- 172.21.15.37:0/3003164887 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbdc01037a0 0x7fbdc0070800 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7fbdac002a50 tx=0x7fbdac004600 comp rx=0 tx=0).stop 2024-03-29T08:48:05.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:05.260+0000 7fbdc8400700 1 -- 172.21.15.37:0/3003164887 shutdown_connections 2024-03-29T08:48:05.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:05.260+0000 7fbdc8400700 1 --2- 172.21.15.37:0/3003164887 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fbda806cc30 0x7fbda806f0e0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:48:05.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:05.260+0000 7fbdc8400700 1 --2- 172.21.15.37:0/3003164887 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbdc0102e00 0x7fbdc00702c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:48:05.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:05.260+0000 7fbdc8400700 1 --2- 172.21.15.37:0/3003164887 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbdc01037a0 0x7fbdc0070800 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:48:05.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:05.260+0000 7fbdc8400700 1 -- 172.21.15.37:0/3003164887 >> 172.21.15.37:0/3003164887 conn(0x7fbdc00f89b0 msgr2=0x7fbdc00f9a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:48:05.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:05.260+0000 7fbdc8400700 1 -- 172.21.15.37:0/3003164887 shutdown_connections 2024-03-29T08:48:05.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:05.261+0000 7fbdc8400700 1 -- 172.21.15.37:0/3003164887 wait complete. 2024-03-29T08:48:05.262 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 30 2024-03-29T08:48:05.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:05.398+0000 7fdf53e00700 1 -- 172.21.15.37:0/2358175588 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdf541035d0 msgr2=0x7fdf540fcc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:48:05.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:05.398+0000 7fdf53e00700 1 --2- 172.21.15.37:0/2358175588 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdf541035d0 0x7fdf540fcc40 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7fdf40004940 tx=0x7fdf40009e30 comp rx=0 tx=0).stop 2024-03-29T08:48:05.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:05.398+0000 7fdf53e00700 1 -- 172.21.15.37:0/2358175588 shutdown_connections 2024-03-29T08:48:05.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:05.398+0000 7fdf53e00700 1 --2- 172.21.15.37:0/2358175588 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdf541035d0 0x7fdf540fcc40 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:48:05.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:05.398+0000 7fdf53e00700 1 --2- 172.21.15.37:0/2358175588 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdf54102c30 0x7fdf54103000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:48:05.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:05.398+0000 7fdf53e00700 1 -- 172.21.15.37:0/2358175588 >> 172.21.15.37:0/2358175588 conn(0x7fdf540f87a0 msgr2=0x7fdf540fabb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:48:05.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:05.399+0000 7fdf53e00700 1 -- 172.21.15.37:0/2358175588 shutdown_connections 2024-03-29T08:48:05.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:05.399+0000 7fdf53e00700 1 -- 172.21.15.37:0/2358175588 wait complete. 2024-03-29T08:48:05.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:05.399+0000 7fdf53e00700 1 Processor -- start 2024-03-29T08:48:05.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:05.400+0000 7fdf53e00700 1 -- start start 2024-03-29T08:48:05.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:05.400+0000 7fdf53e00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdf54102c30 0x7fdf54115ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:48:05.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:05.400+0000 7fdf53e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdf541035d0 0x7fdf54116010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:48:05.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:05.400+0000 7fdf53e00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf54116660 con 0x7fdf541035d0 2024-03-29T08:48:05.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:05.400+0000 7fdf53e00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf5410fa50 con 0x7fdf54102c30 2024-03-29T08:48:05.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:05.400+0000 7fdf52000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdf541035d0 0x7fdf54116010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:48:05.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:05.400+0000 7fdf52000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdf541035d0 0x7fdf54116010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40826/0 (socket says 172.21.15.37:40826) 2024-03-29T08:48:05.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:05.400+0000 7fdf52000700 1 -- 172.21.15.37:0/3021828858 learned_addr learned my addr 172.21.15.37:0/3021828858 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:48:05.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:05.401+0000 7fdf52a00700 1 --2- 172.21.15.37:0/3021828858 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdf54102c30 0x7fdf54115ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:48:05.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:05.401+0000 7fdf52000700 1 -- 172.21.15.37:0/3021828858 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdf54102c30 msgr2=0x7fdf54115ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:48:05.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:05.401+0000 7fdf52000700 1 --2- 172.21.15.37:0/3021828858 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdf54102c30 0x7fdf54115ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:48:05.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:05.401+0000 7fdf52000700 1 -- 172.21.15.37:0/3021828858 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf40005040 con 0x7fdf541035d0 2024-03-29T08:48:05.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:05.401+0000 7fdf52a00700 1 --2- 172.21.15.37:0/3021828858 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdf54102c30 0x7fdf54115ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:48:05.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:05.401+0000 7fdf52000700 1 --2- 172.21.15.37:0/3021828858 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdf541035d0 0x7fdf54116010 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7fdf40009390 tx=0x7fdf40009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:48:05.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:05.401+0000 7fdf4aa00700 1 -- 172.21.15.37:0/3021828858 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdf4000e040 con 0x7fdf541035d0 2024-03-29T08:48:05.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:05.401+0000 7fdf4aa00700 1 -- 172.21.15.37:0/3021828858 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fdf40013420 con 0x7fdf541035d0 2024-03-29T08:48:05.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:05.401+0000 7fdf53e00700 1 -- 172.21.15.37:0/3021828858 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf5410fcd0 con 0x7fdf541035d0 2024-03-29T08:48:05.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:05.401+0000 7fdf4aa00700 1 -- 172.21.15.37:0/3021828858 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdf4001c630 con 0x7fdf541035d0 2024-03-29T08:48:05.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:05.401+0000 7fdf53e00700 1 -- 172.21.15.37:0/3021828858 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf54110290 con 0x7fdf541035d0 2024-03-29T08:48:05.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:05.402+0000 7fdf53e00700 1 -- 172.21.15.37:0/3021828858 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf540fe380 con 0x7fdf541035d0 2024-03-29T08:48:05.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:05.403+0000 7fdf4aa00700 1 -- 172.21.15.37:0/3021828858 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fdf40013a00 con 0x7fdf541035d0 2024-03-29T08:48:05.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:05.405+0000 7fdf4aa00700 1 --2- 172.21.15.37:0/3021828858 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fdf3806cc30 0x7fdf3806f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:48:05.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:05.405+0000 7fdf4aa00700 1 -- 172.21.15.37:0/3021828858 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fdf4008d390 con 0x7fdf541035d0 2024-03-29T08:48:05.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:05.405+0000 7fdf52a00700 1 --2- 172.21.15.37:0/3021828858 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fdf3806cc30 0x7fdf3806f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:48:05.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:05.406+0000 7fdf52a00700 1 --2- 172.21.15.37:0/3021828858 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fdf3806cc30 0x7fdf3806f0e0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fdf44007fd0 tx=0x7fdf44009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:48:05.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:05.410+0000 7fdf4aa00700 1 -- 172.21.15.37:0/3021828858 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fdf4005ade0 con 0x7fdf541035d0 2024-03-29T08:48:05.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:05.643+0000 7fdf53e00700 1 -- 172.21.15.37:0/3021828858 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fdf54109bc0 con 0x7fdf3806cc30 2024-03-29T08:48:05.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:05.644+0000 7fdf4aa00700 1 -- 172.21.15.37:0/3021828858 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fdf54109bc0 con 0x7fdf3806cc30 2024-03-29T08:48:05.644 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-29T08:48:05.644 INFO:teuthology.orchestra.run.smithi037.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59", 2024-03-29T08:48:05.644 INFO:teuthology.orchestra.run.smithi037.stdout: "in_progress": true, 2024-03-29T08:48:05.644 INFO:teuthology.orchestra.run.smithi037.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-29T08:48:05.644 INFO:teuthology.orchestra.run.smithi037.stdout: "services_complete": [ 2024-03-29T08:48:05.644 INFO:teuthology.orchestra.run.smithi037.stdout: "crash", 2024-03-29T08:48:05.644 INFO:teuthology.orchestra.run.smithi037.stdout: "osd", 2024-03-29T08:48:05.644 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr", 2024-03-29T08:48:05.644 INFO:teuthology.orchestra.run.smithi037.stdout: "mds", 2024-03-29T08:48:05.644 INFO:teuthology.orchestra.run.smithi037.stdout: "mon" 2024-03-29T08:48:05.645 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-03-29T08:48:05.645 INFO:teuthology.orchestra.run.smithi037.stdout: "progress": "16/21 daemons upgraded", 2024-03-29T08:48:05.645 INFO:teuthology.orchestra.run.smithi037.stdout: "message": "Currently upgrading mds daemons", 2024-03-29T08:48:05.645 INFO:teuthology.orchestra.run.smithi037.stdout: "is_paused": false 2024-03-29T08:48:05.645 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-29T08:48:05.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:05.646+0000 7fdf53e00700 1 -- 172.21.15.37:0/3021828858 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fdf3806cc30 msgr2=0x7fdf3806f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:48:05.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:05.646+0000 7fdf53e00700 1 --2- 172.21.15.37:0/3021828858 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fdf3806cc30 0x7fdf3806f0e0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fdf44007fd0 tx=0x7fdf44009040 comp rx=0 tx=0).stop 2024-03-29T08:48:05.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:05.646+0000 7fdf53e00700 1 -- 172.21.15.37:0/3021828858 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdf541035d0 msgr2=0x7fdf54116010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:48:05.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:05.646+0000 7fdf53e00700 1 --2- 172.21.15.37:0/3021828858 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdf541035d0 0x7fdf54116010 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7fdf40009390 tx=0x7fdf40009e30 comp rx=0 tx=0).stop 2024-03-29T08:48:05.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:05.647+0000 7fdf53e00700 1 -- 172.21.15.37:0/3021828858 shutdown_connections 2024-03-29T08:48:05.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:05.647+0000 7fdf53e00700 1 --2- 172.21.15.37:0/3021828858 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fdf3806cc30 0x7fdf3806f0e0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:48:05.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:05.647+0000 7fdf53e00700 1 --2- 172.21.15.37:0/3021828858 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdf54102c30 0x7fdf54115ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:48:05.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:05.647+0000 7fdf53e00700 1 --2- 172.21.15.37:0/3021828858 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdf541035d0 0x7fdf54116010 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:48:05.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:05.647+0000 7fdf53e00700 1 -- 172.21.15.37:0/3021828858 >> 172.21.15.37:0/3021828858 conn(0x7fdf540f87a0 msgr2=0x7fdf540f9770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:48:05.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:05.647+0000 7fdf53e00700 1 -- 172.21.15.37:0/3021828858 shutdown_connections 2024-03-29T08:48:05.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:05.647+0000 7fdf53e00700 1 -- 172.21.15.37:0/3021828858 wait complete. 2024-03-29T08:48:05.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:05.783+0000 7fd91d200700 1 -- 172.21.15.37:0/2546111288 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd9181037a0 msgr2=0x7fd9180fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:48:05.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:05.783+0000 7fd91d200700 1 --2- 172.21.15.37:0/2546111288 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd9181037a0 0x7fd9180fce10 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7fd904004940 tx=0x7fd904009e30 comp rx=0 tx=0).stop 2024-03-29T08:48:05.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:05.784+0000 7fd91d200700 1 -- 172.21.15.37:0/2546111288 shutdown_connections 2024-03-29T08:48:05.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:05.784+0000 7fd91d200700 1 --2- 172.21.15.37:0/2546111288 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd9181037a0 0x7fd9180fce10 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:48:05.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:05.784+0000 7fd91d200700 1 --2- 172.21.15.37:0/2546111288 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd918102e00 0x7fd9181031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:48:05.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:05.784+0000 7fd91d200700 1 -- 172.21.15.37:0/2546111288 >> 172.21.15.37:0/2546111288 conn(0x7fd9180f89b0 msgr2=0x7fd9180fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:48:05.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:05.784+0000 7fd91d200700 1 -- 172.21.15.37:0/2546111288 shutdown_connections 2024-03-29T08:48:05.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:05.784+0000 7fd91d200700 1 -- 172.21.15.37:0/2546111288 wait complete. 2024-03-29T08:48:05.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:05.784+0000 7fd91d200700 1 Processor -- start 2024-03-29T08:48:05.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:05.785+0000 7fd91d200700 1 -- start start 2024-03-29T08:48:05.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:05.785+0000 7fd91d200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd918102e00 0x7fd918115da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:48:05.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:05.785+0000 7fd91d200700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd9181037a0 0x7fd9181162e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:48:05.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:05.785+0000 7fd91d200700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd918116930 con 0x7fd918102e00 2024-03-29T08:48:05.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:05.785+0000 7fd91d200700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd91810fd00 con 0x7fd9181037a0 2024-03-29T08:48:05.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:05.785+0000 7fd915e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd918102e00 0x7fd918115da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:48:05.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:05.785+0000 7fd915e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd918102e00 0x7fd918115da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40846/0 (socket says 172.21.15.37:40846) 2024-03-29T08:48:05.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:05.785+0000 7fd915e00700 1 -- 172.21.15.37:0/1770415125 learned_addr learned my addr 172.21.15.37:0/1770415125 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:48:05.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:05.786+0000 7fd915400700 1 --2- 172.21.15.37:0/1770415125 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd9181037a0 0x7fd9181162e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:48:05.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:05.786+0000 7fd915e00700 1 -- 172.21.15.37:0/1770415125 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd9181037a0 msgr2=0x7fd9181162e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:48:05.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:05.786+0000 7fd915e00700 1 --2- 172.21.15.37:0/1770415125 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd9181037a0 0x7fd9181162e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:48:05.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:05.786+0000 7fd915e00700 1 -- 172.21.15.37:0/1770415125 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd904005040 con 0x7fd918102e00 2024-03-29T08:48:05.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:05.786+0000 7fd915400700 1 --2- 172.21.15.37:0/1770415125 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd9181037a0 0x7fd9181162e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:48:05.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:05.786+0000 7fd915e00700 1 --2- 172.21.15.37:0/1770415125 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd918102e00 0x7fd918115da0 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7fd908002a50 tx=0x7fd9080046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:48:05.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:05.786+0000 7fd90e000700 1 -- 172.21.15.37:0/1770415125 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd908007cc0 con 0x7fd918102e00 2024-03-29T08:48:05.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:05.786+0000 7fd91d200700 1 -- 172.21.15.37:0/1770415125 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd91810ffe0 con 0x7fd918102e00 2024-03-29T08:48:05.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:05.786+0000 7fd90e000700 1 -- 172.21.15.37:0/1770415125 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd90800f040 con 0x7fd918102e00 2024-03-29T08:48:05.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:05.786+0000 7fd90e000700 1 -- 172.21.15.37:0/1770415125 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd908009540 con 0x7fd918102e00 2024-03-29T08:48:05.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:05.786+0000 7fd91d200700 1 -- 172.21.15.37:0/1770415125 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd918110520 con 0x7fd918102e00 2024-03-29T08:48:05.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:05.787+0000 7fd91d200700 1 -- 172.21.15.37:0/1770415125 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9180fe550 con 0x7fd918102e00 2024-03-29T08:48:05.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:05.788+0000 7fd90e000700 1 -- 172.21.15.37:0/1770415125 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fd9080096a0 con 0x7fd918102e00 2024-03-29T08:48:05.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:05.789+0000 7fd90e000700 1 --2- 172.21.15.37:0/1770415125 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fd8fc06cc30 0x7fd8fc06f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:48:05.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:05.789+0000 7fd90e000700 1 -- 172.21.15.37:0/1770415125 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fd90808bde0 con 0x7fd918102e00 2024-03-29T08:48:05.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:05.790+0000 7fd915400700 1 --2- 172.21.15.37:0/1770415125 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fd8fc06cc30 0x7fd8fc06f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:48:05.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:05.790+0000 7fd915400700 1 --2- 172.21.15.37:0/1770415125 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fd8fc06cc30 0x7fd8fc06f0e0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fd918111640 tx=0x7fd904009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:48:05.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:05.794+0000 7fd90e000700 1 -- 172.21.15.37:0/1770415125 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fd908059830 con 0x7fd918102e00 2024-03-29T08:48:05.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:48:05 smithi037 ceph-mon[142325]: from='client.35994 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:48:05.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:48:05 smithi037 ceph-mon[142325]: from='client.35998 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:48:05.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:48:05 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/1989564571' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:48:05.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:48:05 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/3003164887' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T08:48:06.055 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:48:05 smithi148 ceph-mon[106692]: from='client.35994 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:48:06.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:48:05 smithi148 ceph-mon[106692]: from='client.35998 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:48:06.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:48:05 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/1989564571' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:48:06.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:48:05 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/3003164887' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T08:48:06.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:06.089+0000 7fd91d200700 1 -- 172.21.15.37:0/1770415125 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fd91804bed0 con 0x7fd918102e00 2024-03-29T08:48:06.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:06.089+0000 7fd90e000700 1 -- 172.21.15.37:0/1770415125 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+3583 (secure 0 0 0) 0x7fd9080593c0 con 0x7fd918102e00 2024-03-29T08:48:06.089 INFO:teuthology.orchestra.run.smithi037.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem is offline 2024-03-29T08:48:06.089 INFO:teuthology.orchestra.run.smithi037.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-03-29T08:48:06.089 INFO:teuthology.orchestra.run.smithi037.stdout: host smithi148 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/mon.smithi148/config 2024-03-29T08:48:06.089 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/podman: timeout after 895 seconds 2024-03-29T08:48:06.089 INFO:teuthology.orchestra.run.smithi037.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59 -e NODE_NAME=smithi148 -e CEPH_USE_RANDOM_NONCE=1 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4:/var/run/ceph:z -v /var/log/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4:/var/log/ceph:z -v /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpkx2rz2d4:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59 inventory --format=json-pretty --filter-for-batch 2024-03-29T08:48:06.089 INFO:teuthology.orchestra.run.smithi037.stdout:Traceback (most recent call last): 2024-03-29T08:48:06.090 INFO:teuthology.orchestra.run.smithi037.stdout: File "/usr/lib64/python3.6/runpy.py", line 193, in _run_module_as_main 2024-03-29T08:48:06.090 INFO:teuthology.orchestra.run.smithi037.stdout: "__main__", mod_spec) 2024-03-29T08:48:06.090 INFO:teuthology.orchestra.run.smithi037.stdout: File "/usr/lib64/python3.6/runpy.py", line 85, in _run_code 2024-03-29T08:48:06.090 INFO:teuthology.orchestra.run.smithi037.stdout: exec(code, run_globals) 2024-03-29T08:48:06.090 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 10889, in 2024-03-29T08:48:06.090 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 10877, in main 2024-03-29T08:48:06.090 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 2576, in _infer_config 2024-03-29T08:48:06.090 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 2492, in _infer_fsid 2024-03-29T08:48:06.090 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 2604, in _infer_image 2024-03-29T08:48:06.090 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 2479, in _validate_fsid 2024-03-29T08:48:06.090 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 7145, in command_ceph_volume 2024-03-29T08:48:06.090 INFO:teuthology.orchestra.run.smithi037.stdout: File "/tmp/tmp0civ05gu.cephadm.build/__main__.py", line 2267, in call_throws 2024-03-29T08:48:06.090 INFO:teuthology.orchestra.run.smithi037.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59 -e NODE_NAME=smithi148 -e CEPH_USE_RANDOM_NONCE=1 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4:/var/run/ceph:z -v /var/log/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4:/var/log/ceph:z -v /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpkx2rz2d4:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:fa04870884a9d2857293361c500ea875659d8588479e2e2770f1385107d4ff59 inventory --format=json-pretty --filter-for-batch 2024-03-29T08:48:06.090 INFO:teuthology.orchestra.run.smithi037.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-03-29T08:48:06.091 INFO:teuthology.orchestra.run.smithi037.stdout: fs cephfs is degraded 2024-03-29T08:48:06.091 INFO:teuthology.orchestra.run.smithi037.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-03-29T08:48:06.091 INFO:teuthology.orchestra.run.smithi037.stdout: fs cephfs is offline because no MDS is active for it. 2024-03-29T08:48:06.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:06.092+0000 7fd91d200700 1 -- 172.21.15.37:0/1770415125 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fd8fc06cc30 msgr2=0x7fd8fc06f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:48:06.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:06.092+0000 7fd91d200700 1 --2- 172.21.15.37:0/1770415125 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fd8fc06cc30 0x7fd8fc06f0e0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fd918111640 tx=0x7fd904009e30 comp rx=0 tx=0).stop 2024-03-29T08:48:06.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:06.092+0000 7fd91d200700 1 -- 172.21.15.37:0/1770415125 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd918102e00 msgr2=0x7fd918115da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:48:06.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:06.092+0000 7fd91d200700 1 --2- 172.21.15.37:0/1770415125 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd918102e00 0x7fd918115da0 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7fd908002a50 tx=0x7fd9080046b0 comp rx=0 tx=0).stop 2024-03-29T08:48:06.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:06.092+0000 7fd91d200700 1 -- 172.21.15.37:0/1770415125 shutdown_connections 2024-03-29T08:48:06.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:06.092+0000 7fd91d200700 1 --2- 172.21.15.37:0/1770415125 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fd8fc06cc30 0x7fd8fc06f0e0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:48:06.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:06.092+0000 7fd91d200700 1 --2- 172.21.15.37:0/1770415125 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd918102e00 0x7fd918115da0 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:48:06.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:06.092+0000 7fd91d200700 1 --2- 172.21.15.37:0/1770415125 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd9181037a0 0x7fd9181162e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:48:06.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:06.092+0000 7fd91d200700 1 -- 172.21.15.37:0/1770415125 >> 172.21.15.37:0/1770415125 conn(0x7fd9180f89b0 msgr2=0x7fd9180f9a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:48:06.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:06.092+0000 7fd91d200700 1 -- 172.21.15.37:0/1770415125 shutdown_connections 2024-03-29T08:48:06.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:06.092+0000 7fd91d200700 1 -- 172.21.15.37:0/1770415125 wait complete. 2024-03-29T08:48:06.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:48:06 smithi037 ceph-mon[142325]: pgmap v1117: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:48:06.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:48:06 smithi037 ceph-mon[142325]: from='client.36010 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:48:06.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:48:06 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/1770415125' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T08:48:07.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:48:06 smithi148 ceph-mon[106692]: pgmap v1117: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:48:07.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:48:06 smithi148 ceph-mon[106692]: from='client.36010 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:48:07.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:48:06 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/1770415125' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T08:48:08.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:48:08 smithi037 ceph-mon[142325]: pgmap v1118: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:48:09.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:48:08 smithi148 ceph-mon[106692]: pgmap v1118: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:48:10.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:48:10 smithi037 ceph-mon[142325]: pgmap v1119: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:48:10.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:48:10 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:48:11.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:48:10 smithi148 ceph-mon[106692]: pgmap v1119: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:48:11.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:48:10 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:48:12.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:48:12 smithi037 ceph-mon[142325]: pgmap v1120: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:48:13.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:48:12 smithi148 ceph-mon[106692]: pgmap v1120: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:48:14.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:48:14 smithi037 ceph-mon[142325]: pgmap v1121: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:48:15.055 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:48:14 smithi148 ceph-mon[106692]: pgmap v1121: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:48:16.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:48:16 smithi037 ceph-mon[142325]: pgmap v1122: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:48:17.055 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:48:16 smithi148 ceph-mon[106692]: pgmap v1122: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:48:18.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:48:18 smithi037 ceph-mon[142325]: pgmap v1123: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:48:19.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:48:18 smithi148 ceph-mon[106692]: pgmap v1123: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:48:20.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:48:20 smithi037 ceph-mon[142325]: pgmap v1124: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:48:21.055 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:48:20 smithi148 ceph-mon[106692]: pgmap v1124: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:48:21.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:48:21 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T08:48:21.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:48:21 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T08:48:21.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:48:21 smithi037 ceph-mon[142325]: pgmap v1125: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:48:21.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:48:21 smithi037 ceph-mon[142325]: pgmap v1126: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:48:21.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:48:21 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:48:21.928 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:48:21 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T08:48:21.928 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:48:21 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:48:21.928 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:48:21 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:48:21.928 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:48:21 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:48:21.928 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:48:21 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:48:21.928 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:48:21 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:48:21.928 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:48:21 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:48:21.928 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:48:21 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi037.iifsmh"}]: dispatch 2024-03-29T08:48:21.928 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:48:21 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi037.iifsmh"}]': finished 2024-03-29T08:48:21.928 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:48:21 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi037.mmwgxe"}]: dispatch 2024-03-29T08:48:21.928 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:48:21 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi037.mmwgxe"}]': finished 2024-03-29T08:48:21.928 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:48:21 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi148.qukpjh"}]: dispatch 2024-03-29T08:48:21.929 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:48:21 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi148.qukpjh"}]': finished 2024-03-29T08:48:21.929 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:48:21 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi148.sqteih"}]: dispatch 2024-03-29T08:48:21.929 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:48:21 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi148.sqteih"}]': finished 2024-03-29T08:48:21.929 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:48:21 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]: dispatch 2024-03-29T08:48:22.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:48:21 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T08:48:22.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:48:21 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T08:48:22.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:48:21 smithi148 ceph-mon[106692]: pgmap v1125: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:48:22.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:48:21 smithi148 ceph-mon[106692]: pgmap v1126: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:48:22.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:48:21 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:48:22.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:48:21 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T08:48:22.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:48:21 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:48:22.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:48:21 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:48:22.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:48:21 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:48:22.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:48:21 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:48:22.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:48:21 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:48:22.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:48:21 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:48:22.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:48:21 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi037.iifsmh"}]: dispatch 2024-03-29T08:48:22.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:48:21 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi037.iifsmh"}]': finished 2024-03-29T08:48:22.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:48:21 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi037.mmwgxe"}]: dispatch 2024-03-29T08:48:22.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:48:21 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi037.mmwgxe"}]': finished 2024-03-29T08:48:22.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:48:21 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi148.qukpjh"}]: dispatch 2024-03-29T08:48:22.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:48:21 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi148.qukpjh"}]': finished 2024-03-29T08:48:22.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:48:21 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi148.sqteih"}]: dispatch 2024-03-29T08:48:22.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:48:21 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi148.sqteih"}]': finished 2024-03-29T08:48:22.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:48:21 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]: dispatch 2024-03-29T08:48:22.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:48:22 smithi037 ceph-mon[142325]: Upgrade: Setting container_image for all mds 2024-03-29T08:48:22.928 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:48:22 smithi037 ceph-mon[142325]: Upgrade: Setting filesystem cephfs Joinable 2024-03-29T08:48:22.928 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:48:22 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]': finished 2024-03-29T08:48:22.928 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:48:22 smithi037 ceph-mon[142325]: Standby daemon mds.cephfs.smithi037.mmwgxe assigned to filesystem cephfs as rank 0 2024-03-29T08:48:22.928 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:48:22 smithi037 ceph-mon[142325]: Standby daemon mds.cephfs.smithi037.iifsmh assigned to filesystem cephfs as rank 1 2024-03-29T08:48:22.928 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:48:22 smithi037 ceph-mon[142325]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-29T08:48:22.928 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:48:22 smithi037 ceph-mon[142325]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-03-29T08:48:22.928 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:48:22 smithi037 ceph-mon[142325]: fsmap cephfs:2/2 {0=cephfs.smithi037.mmwgxe=up:replay,1=cephfs.smithi037.iifsmh=up:replay} 2 up:standby 2024-03-29T08:48:22.928 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:48:22 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:48:22.928 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:48:22 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:48:22.928 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:48:22 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:48:22.928 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:48:22 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:48:22.928 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:48:22 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:48:22.928 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:48:22 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:48:22.929 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:48:22 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:48:22.929 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:48:22 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:48:22.929 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:48:22 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:48:22.929 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:48:22 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:48:22.929 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:48:22 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:48:22.929 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:48:22 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:48:22.929 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:48:22 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:48:22.929 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:48:22 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:48:22.929 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:48:22 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:48:22.929 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:48:22 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:48:22.929 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:48:22 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:48:22.929 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:48:22 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:48:22.929 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:48:22 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:48:22.930 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:48:22 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-03-29T08:48:22.930 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:48:22 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-03-29T08:48:22.930 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:48:22 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-29T08:48:22.930 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:48:22 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-03-29T08:48:22.930 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:48:22 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-03-29T08:48:22.930 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:48:22 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]': finished 2024-03-29T08:48:22.930 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:48:22 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-03-29T08:48:22.930 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:48:22 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd"}]': finished 2024-03-29T08:48:22.930 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:48:22 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-03-29T08:48:22.930 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:48:22 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds"}]': finished 2024-03-29T08:48:22.930 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:48:22 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-03-29T08:48:22.930 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:48:22 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-03-29T08:48:22.930 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:48:22 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-03-29T08:48:22.930 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:48:22 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-03-29T08:48:22.931 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:48:22 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-29T08:48:22.931 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:48:22 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2024-03-29T08:48:22.931 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:48:22 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]': finished 2024-03-29T08:48:22.931 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:48:22 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-03-29T08:48:22.931 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:48:22 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-03-29T08:48:22.931 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:48:22 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-03-29T08:48:22.931 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:48:22 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-03-29T08:48:22.931 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:48:22 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2024-03-29T08:48:22.931 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:48:22 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2024-03-29T08:48:22.931 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:48:22 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-29T08:48:22.931 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:48:22 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-29T08:48:22.931 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:48:22 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-29T08:48:22.931 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:48:22 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-29T08:48:22.931 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:48:22 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-29T08:48:22.932 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:48:22 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-29T08:48:22.932 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:48:22 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config rm", "name": "mon_mds_skip_sanity", "who": "mon"}]: dispatch 2024-03-29T08:48:22.932 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:48:22 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd='[{"prefix": "config rm", "name": "mon_mds_skip_sanity", "who": "mon"}]': finished 2024-03-29T08:48:22.932 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:48:22 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-03-29T08:48:22.932 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:48:22 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-03-29T08:48:22.932 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:48:22 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T08:48:23.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:48:22 smithi148 ceph-mon[106692]: Upgrade: Setting container_image for all mds 2024-03-29T08:48:23.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:48:22 smithi148 ceph-mon[106692]: Upgrade: Setting filesystem cephfs Joinable 2024-03-29T08:48:23.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:48:22 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]': finished 2024-03-29T08:48:23.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:48:22 smithi148 ceph-mon[106692]: Standby daemon mds.cephfs.smithi037.mmwgxe assigned to filesystem cephfs as rank 0 2024-03-29T08:48:23.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:48:22 smithi148 ceph-mon[106692]: Standby daemon mds.cephfs.smithi037.iifsmh assigned to filesystem cephfs as rank 1 2024-03-29T08:48:23.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:48:22 smithi148 ceph-mon[106692]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-29T08:48:23.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:48:22 smithi148 ceph-mon[106692]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-03-29T08:48:23.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:48:22 smithi148 ceph-mon[106692]: fsmap cephfs:2/2 {0=cephfs.smithi037.mmwgxe=up:replay,1=cephfs.smithi037.iifsmh=up:replay} 2 up:standby 2024-03-29T08:48:23.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:48:22 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:48:23.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:48:22 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:48:23.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:48:22 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:48:23.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:48:22 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:48:23.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:48:22 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:48:23.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:48:22 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:48:23.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:48:22 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:48:23.057 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:48:22 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:48:23.058 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:48:22 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:48:23.058 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:48:22 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:48:23.058 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:48:22 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:48:23.058 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:48:22 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:48:23.058 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:48:22 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:48:23.058 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:48:22 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:48:23.058 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:48:22 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:48:23.058 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:48:22 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:48:23.058 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:48:22 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:48:23.058 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:48:22 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:48:23.058 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:48:22 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:48:23.058 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:48:22 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-03-29T08:48:23.058 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:48:22 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-03-29T08:48:23.058 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:48:22 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-29T08:48:23.059 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:48:22 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-03-29T08:48:23.059 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:48:22 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-03-29T08:48:23.059 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:48:22 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]': finished 2024-03-29T08:48:23.059 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:48:22 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-03-29T08:48:23.059 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:48:22 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd"}]': finished 2024-03-29T08:48:23.059 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:48:22 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-03-29T08:48:23.059 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:48:22 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds"}]': finished 2024-03-29T08:48:23.059 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:48:22 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-03-29T08:48:23.059 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:48:22 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-03-29T08:48:23.059 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:48:22 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-03-29T08:48:23.059 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:48:22 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-03-29T08:48:23.059 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:48:22 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-29T08:48:23.059 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:48:22 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2024-03-29T08:48:23.059 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:48:22 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]': finished 2024-03-29T08:48:23.059 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:48:22 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-03-29T08:48:23.059 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:48:22 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-03-29T08:48:23.060 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:48:22 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-03-29T08:48:23.060 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:48:22 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-03-29T08:48:23.060 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:48:22 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2024-03-29T08:48:23.060 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:48:22 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2024-03-29T08:48:23.060 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:48:22 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-29T08:48:23.060 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:48:22 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-29T08:48:23.060 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:48:22 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-29T08:48:23.060 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:48:22 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-29T08:48:23.060 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:48:22 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-29T08:48:23.060 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:48:22 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-29T08:48:23.060 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:48:22 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config rm", "name": "mon_mds_skip_sanity", "who": "mon"}]: dispatch 2024-03-29T08:48:23.060 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:48:22 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd='[{"prefix": "config rm", "name": "mon_mds_skip_sanity", "who": "mon"}]': finished 2024-03-29T08:48:23.060 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:48:22 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-03-29T08:48:23.060 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:48:22 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-03-29T08:48:23.060 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:48:22 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T08:48:24.162 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:48:23 smithi037 ceph-mon[142325]: Upgrade: Setting container_image for all rgw 2024-03-29T08:48:24.162 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:48:23 smithi037 ceph-mon[142325]: Upgrade: Setting container_image for all rbd-mirror 2024-03-29T08:48:24.162 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:48:23 smithi037 ceph-mon[142325]: Upgrade: Setting container_image for all ceph-exporter 2024-03-29T08:48:24.162 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:48:23 smithi037 ceph-mon[142325]: Upgrade: Setting container_image for all iscsi 2024-03-29T08:48:24.162 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:48:23 smithi037 ceph-mon[142325]: Upgrade: Setting container_image for all nfs 2024-03-29T08:48:24.162 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:48:23 smithi037 ceph-mon[142325]: Upgrade: Setting container_image for all nvmeof 2024-03-29T08:48:24.162 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:48:23 smithi037 ceph-mon[142325]: Upgrade: Finalizing container_image settings 2024-03-29T08:48:24.162 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:48:23 smithi037 ceph-mon[142325]: Upgrade: Complete! 2024-03-29T08:48:24.163 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:48:23 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:48:24.163 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:48:23 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:48:24.163 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:48:23 smithi037 ceph-mon[142325]: pgmap v1127: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:48:24.163 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:48:23 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:48:24.163 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:48:23 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:48:24.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:48:23 smithi148 ceph-mon[106692]: Upgrade: Setting container_image for all rgw 2024-03-29T08:48:24.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:48:23 smithi148 ceph-mon[106692]: Upgrade: Setting container_image for all rbd-mirror 2024-03-29T08:48:24.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:48:23 smithi148 ceph-mon[106692]: Upgrade: Setting container_image for all ceph-exporter 2024-03-29T08:48:24.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:48:23 smithi148 ceph-mon[106692]: Upgrade: Setting container_image for all iscsi 2024-03-29T08:48:24.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:48:23 smithi148 ceph-mon[106692]: Upgrade: Setting container_image for all nfs 2024-03-29T08:48:24.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:48:23 smithi148 ceph-mon[106692]: Upgrade: Setting container_image for all nvmeof 2024-03-29T08:48:24.307 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:48:23 smithi148 ceph-mon[106692]: Upgrade: Finalizing container_image settings 2024-03-29T08:48:24.307 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:48:23 smithi148 ceph-mon[106692]: Upgrade: Complete! 2024-03-29T08:48:24.307 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:48:23 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:48:24.307 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:48:23 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:48:24.307 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:48:23 smithi148 ceph-mon[106692]: pgmap v1127: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:48:24.307 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:48:23 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:48:24.307 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:48:23 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:48:25.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:48:25 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:48:25.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:48:25 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:48:25.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:48:25 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:48:25.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:48:25 smithi037 ceph-mon[142325]: pgmap v1128: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail; 5.9 MiB/s rd, 3 op/s 2024-03-29T08:48:26.055 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:48:25 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:48:26.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:48:25 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:48:26.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:48:25 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:48:26.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:48:25 smithi148 ceph-mon[106692]: pgmap v1128: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail; 5.9 MiB/s rd, 3 op/s 2024-03-29T08:48:27.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:48:26 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:48:27.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:48:26 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:48:27.176 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:48:26 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:48:27.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:48:26 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:48:27.997 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:48:27 smithi037 ceph-mon[142325]: pgmap v1129: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail; 18 MiB/s rd, 7 op/s 2024-03-29T08:48:28.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:48:27 smithi148 ceph-mon[106692]: pgmap v1129: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail; 18 MiB/s rd, 7 op/s 2024-03-29T08:48:30.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:48:29 smithi148 ceph-mon[106692]: pgmap v1130: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail; 22 MiB/s rd, 7 op/s 2024-03-29T08:48:30.426 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:48:29 smithi037 ceph-mon[142325]: pgmap v1130: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail; 22 MiB/s rd, 7 op/s 2024-03-29T08:48:31.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:48:30 smithi148 ceph-mon[106692]: mds.? [v2:172.21.15.37:6826/2060882959,v1:172.21.15.37:6827/2060882959] up:resolve 2024-03-29T08:48:31.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:48:30 smithi148 ceph-mon[106692]: mds.? [v2:172.21.15.37:6828/1248142005,v1:172.21.15.37:6829/1248142005] up:resolve 2024-03-29T08:48:31.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:48:30 smithi148 ceph-mon[106692]: fsmap cephfs:2/2 {0=cephfs.smithi037.mmwgxe=up:resolve,1=cephfs.smithi037.iifsmh=up:resolve} 2 up:standby 2024-03-29T08:48:31.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:48:30 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:48:31.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:48:30 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:48:31.426 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:48:30 smithi037 ceph-mon[142325]: mds.? [v2:172.21.15.37:6826/2060882959,v1:172.21.15.37:6827/2060882959] up:resolve 2024-03-29T08:48:31.426 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:48:30 smithi037 ceph-mon[142325]: mds.? [v2:172.21.15.37:6828/1248142005,v1:172.21.15.37:6829/1248142005] up:resolve 2024-03-29T08:48:31.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:48:30 smithi037 ceph-mon[142325]: fsmap cephfs:2/2 {0=cephfs.smithi037.mmwgxe=up:resolve,1=cephfs.smithi037.iifsmh=up:resolve} 2 up:standby 2024-03-29T08:48:31.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:48:30 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:48:31.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:48:30 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:48:32.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:48:31 smithi148 ceph-mon[106692]: pgmap v1131: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail; 25 MiB/s rd, 8 op/s 2024-03-29T08:48:32.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:48:31 smithi148 ceph-mon[106692]: reconnect by client.14536 192.168.0.1:0/2061834695 after 0.00199997 2024-03-29T08:48:32.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:48:31 smithi148 ceph-mon[106692]: reconnect by client.24373 192.168.0.1:0/985505949 after 0.00199997 2024-03-29T08:48:32.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:48:31 smithi148 ceph-mon[106692]: reconnect by client.14536 192.168.0.1:0/2061834695 after 0.00199997 2024-03-29T08:48:32.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:48:31 smithi148 ceph-mon[106692]: reconnect by client.24373 192.168.0.1:0/985505949 after 0.00199997 2024-03-29T08:48:32.307 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:48:31 smithi148 ceph-mon[106692]: mds.? [v2:172.21.15.37:6828/1248142005,v1:172.21.15.37:6829/1248142005] up:reconnect 2024-03-29T08:48:32.307 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:48:31 smithi148 ceph-mon[106692]: mds.? [v2:172.21.15.37:6826/2060882959,v1:172.21.15.37:6827/2060882959] up:reconnect 2024-03-29T08:48:32.307 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:48:31 smithi148 ceph-mon[106692]: fsmap cephfs:2/2 {0=cephfs.smithi037.mmwgxe=up:reconnect,1=cephfs.smithi037.iifsmh=up:reconnect} 2 up:standby 2024-03-29T08:48:32.307 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:48:31 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:48:32.307 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:48:31 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:48:32.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:48:31 smithi037 ceph-mon[142325]: pgmap v1131: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail; 25 MiB/s rd, 8 op/s 2024-03-29T08:48:32.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:48:31 smithi037 ceph-mon[142325]: reconnect by client.14536 192.168.0.1:0/2061834695 after 0.00199997 2024-03-29T08:48:32.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:48:31 smithi037 ceph-mon[142325]: reconnect by client.24373 192.168.0.1:0/985505949 after 0.00199997 2024-03-29T08:48:32.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:48:31 smithi037 ceph-mon[142325]: reconnect by client.14536 192.168.0.1:0/2061834695 after 0.00199997 2024-03-29T08:48:32.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:48:31 smithi037 ceph-mon[142325]: reconnect by client.24373 192.168.0.1:0/985505949 after 0.00199997 2024-03-29T08:48:32.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:48:31 smithi037 ceph-mon[142325]: mds.? [v2:172.21.15.37:6828/1248142005,v1:172.21.15.37:6829/1248142005] up:reconnect 2024-03-29T08:48:32.428 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:48:31 smithi037 ceph-mon[142325]: mds.? [v2:172.21.15.37:6826/2060882959,v1:172.21.15.37:6827/2060882959] up:reconnect 2024-03-29T08:48:32.428 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:48:31 smithi037 ceph-mon[142325]: fsmap cephfs:2/2 {0=cephfs.smithi037.mmwgxe=up:reconnect,1=cephfs.smithi037.iifsmh=up:reconnect} 2 up:standby 2024-03-29T08:48:32.428 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:48:31 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:48:32.428 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:48:31 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:48:33.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:48:33 smithi148 ceph-mon[106692]: mds.? [v2:172.21.15.37:6828/1248142005,v1:172.21.15.37:6829/1248142005] up:rejoin 2024-03-29T08:48:33.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:48:33 smithi148 ceph-mon[106692]: mds.? [v2:172.21.15.37:6826/2060882959,v1:172.21.15.37:6827/2060882959] up:rejoin 2024-03-29T08:48:33.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:48:33 smithi148 ceph-mon[106692]: fsmap cephfs:2/2 {0=cephfs.smithi037.mmwgxe=up:rejoin,1=cephfs.smithi037.iifsmh=up:rejoin} 2 up:standby 2024-03-29T08:48:33.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:48:33 smithi148 ceph-mon[106692]: daemon mds.cephfs.smithi037.mmwgxe is now active in filesystem cephfs as rank 0 2024-03-29T08:48:33.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:48:33 smithi148 ceph-mon[106692]: daemon mds.cephfs.smithi037.iifsmh is now active in filesystem cephfs as rank 1 2024-03-29T08:48:33.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:48:33 smithi148 ceph-mon[106692]: pgmap v1132: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail; 21 MiB/s rd, 7 op/s 2024-03-29T08:48:33.426 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:48:33 smithi037 ceph-mon[142325]: mds.? [v2:172.21.15.37:6828/1248142005,v1:172.21.15.37:6829/1248142005] up:rejoin 2024-03-29T08:48:33.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:48:33 smithi037 ceph-mon[142325]: mds.? [v2:172.21.15.37:6826/2060882959,v1:172.21.15.37:6827/2060882959] up:rejoin 2024-03-29T08:48:33.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:48:33 smithi037 ceph-mon[142325]: fsmap cephfs:2/2 {0=cephfs.smithi037.mmwgxe=up:rejoin,1=cephfs.smithi037.iifsmh=up:rejoin} 2 up:standby 2024-03-29T08:48:33.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:48:33 smithi037 ceph-mon[142325]: daemon mds.cephfs.smithi037.mmwgxe is now active in filesystem cephfs as rank 0 2024-03-29T08:48:33.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:48:33 smithi037 ceph-mon[142325]: daemon mds.cephfs.smithi037.iifsmh is now active in filesystem cephfs as rank 1 2024-03-29T08:48:33.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:48:33 smithi037 ceph-mon[142325]: pgmap v1132: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail; 21 MiB/s rd, 7 op/s 2024-03-29T08:48:34.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:48:34 smithi148 ceph-mon[106692]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-29T08:48:34.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:48:34 smithi148 ceph-mon[106692]: mds.? [v2:172.21.15.37:6828/1248142005,v1:172.21.15.37:6829/1248142005] up:active 2024-03-29T08:48:34.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:48:34 smithi148 ceph-mon[106692]: mds.? [v2:172.21.15.37:6826/2060882959,v1:172.21.15.37:6827/2060882959] up:active 2024-03-29T08:48:34.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:48:34 smithi148 ceph-mon[106692]: fsmap cephfs:2 {0=cephfs.smithi037.mmwgxe=up:active,1=cephfs.smithi037.iifsmh=up:active} 2 up:standby 2024-03-29T08:48:34.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:48:34 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:48:34.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:48:34 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:48:34.426 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:48:34 smithi037 ceph-mon[142325]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-29T08:48:34.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:48:34 smithi037 ceph-mon[142325]: mds.? [v2:172.21.15.37:6828/1248142005,v1:172.21.15.37:6829/1248142005] up:active 2024-03-29T08:48:34.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:48:34 smithi037 ceph-mon[142325]: mds.? [v2:172.21.15.37:6826/2060882959,v1:172.21.15.37:6827/2060882959] up:active 2024-03-29T08:48:34.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:48:34 smithi037 ceph-mon[142325]: fsmap cephfs:2 {0=cephfs.smithi037.mmwgxe=up:active,1=cephfs.smithi037.iifsmh=up:active} 2 up:standby 2024-03-29T08:48:34.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:48:34 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:48:34.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:48:34 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:48:35.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:48:35 smithi037 ceph-mon[142325]: pgmap v1133: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail; 21 MiB/s rd, 426 B/s wr, 10 op/s 2024-03-29T08:48:36.055 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:48:35 smithi148 ceph-mon[106692]: pgmap v1133: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail; 21 MiB/s rd, 426 B/s wr, 10 op/s 2024-03-29T08:48:36.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:36.251+0000 7f1307200700 1 -- 172.21.15.37:0/687215651 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f13001037a0 msgr2=0x7f13000fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:48:36.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:36.251+0000 7f1307200700 1 --2- 172.21.15.37:0/687215651 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f13001037a0 0x7f13000fce10 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f12ec004940 tx=0x7f12ec009e30 comp rx=0 tx=0).stop 2024-03-29T08:48:36.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:36.251+0000 7f1307200700 1 -- 172.21.15.37:0/687215651 shutdown_connections 2024-03-29T08:48:36.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:36.251+0000 7f1307200700 1 --2- 172.21.15.37:0/687215651 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f13001037a0 0x7f13000fce10 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:48:36.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:36.251+0000 7f1307200700 1 --2- 172.21.15.37:0/687215651 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1300102e00 0x7f13001031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:48:36.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:36.251+0000 7f1307200700 1 -- 172.21.15.37:0/687215651 >> 172.21.15.37:0/687215651 conn(0x7f13000f89b0 msgr2=0x7f13000fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:48:36.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:36.251+0000 7f1307200700 1 -- 172.21.15.37:0/687215651 shutdown_connections 2024-03-29T08:48:36.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:36.251+0000 7f1307200700 1 -- 172.21.15.37:0/687215651 wait complete. 2024-03-29T08:48:36.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:36.252+0000 7f1307200700 1 Processor -- start 2024-03-29T08:48:36.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:36.252+0000 7f1307200700 1 -- start start 2024-03-29T08:48:36.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:36.253+0000 7f1307200700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1300102e00 0x7f13000702c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:48:36.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:36.253+0000 7f1307200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f13001037a0 0x7f1300070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:48:36.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:36.253+0000 7f1307200700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1300070d90 con 0x7f13001037a0 2024-03-29T08:48:36.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:36.253+0000 7f1307200700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1300070ed0 con 0x7f1300102e00 2024-03-29T08:48:36.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:36.253+0000 7f12ff400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f13001037a0 0x7f1300070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:48:36.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:36.253+0000 7f12ff400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f13001037a0 0x7f1300070800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60958/0 (socket says 172.21.15.37:60958) 2024-03-29T08:48:36.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:36.253+0000 7f12ff400700 1 -- 172.21.15.37:0/1890030133 learned_addr learned my addr 172.21.15.37:0/1890030133 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:48:36.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:36.254+0000 7f12ffe00700 1 --2- 172.21.15.37:0/1890030133 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1300102e00 0x7f13000702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:48:36.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:36.254+0000 7f12ff400700 1 -- 172.21.15.37:0/1890030133 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1300102e00 msgr2=0x7f13000702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:48:36.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:36.254+0000 7f12ff400700 1 --2- 172.21.15.37:0/1890030133 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1300102e00 0x7f13000702c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:48:36.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:36.254+0000 7f12ff400700 1 -- 172.21.15.37:0/1890030133 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12ec005040 con 0x7f13001037a0 2024-03-29T08:48:36.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:36.254+0000 7f12ff400700 1 --2- 172.21.15.37:0/1890030133 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f13001037a0 0x7f1300070800 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f12ec0093b0 tx=0x7f12ec009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:48:36.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:36.254+0000 7f12f7e00700 1 -- 172.21.15.37:0/1890030133 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f12ec00e040 con 0x7f13001037a0 2024-03-29T08:48:36.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:36.254+0000 7f12f7e00700 1 -- 172.21.15.37:0/1890030133 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f12ec013420 con 0x7f13001037a0 2024-03-29T08:48:36.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:36.254+0000 7f1307200700 1 -- 172.21.15.37:0/1890030133 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1300071120 con 0x7f13001037a0 2024-03-29T08:48:36.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:36.254+0000 7f12f7e00700 1 -- 172.21.15.37:0/1890030133 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f12ec01c630 con 0x7f13001037a0 2024-03-29T08:48:36.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:36.254+0000 7f1307200700 1 -- 172.21.15.37:0/1890030133 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f130011a230 con 0x7f13001037a0 2024-03-29T08:48:36.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:36.256+0000 7f1307200700 1 -- 172.21.15.37:0/1890030133 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13000fe550 con 0x7f13001037a0 2024-03-29T08:48:36.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:36.257+0000 7f12f7e00700 1 -- 172.21.15.37:0/1890030133 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f12ec0139e0 con 0x7f13001037a0 2024-03-29T08:48:36.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:36.257+0000 7f12f7e00700 1 --2- 172.21.15.37:0/1890030133 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f12e406cc20 0x7f12e406f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:48:36.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:36.258+0000 7f12f7e00700 1 -- 172.21.15.37:0/1890030133 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f12ec08d320 con 0x7f13001037a0 2024-03-29T08:48:36.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:36.258+0000 7f12ffe00700 1 --2- 172.21.15.37:0/1890030133 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f12e406cc20 0x7f12e406f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:48:36.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:36.258+0000 7f12ffe00700 1 --2- 172.21.15.37:0/1890030133 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f12e406cc20 0x7f12e406f0d0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f12f0004990 tx=0x7f12f0009ee0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:48:36.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:36.262+0000 7f12f7e00700 1 -- 172.21.15.37:0/1890030133 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f12ec057390 con 0x7f13001037a0 2024-03-29T08:48:36.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:36.522+0000 7f1307200700 1 -- 172.21.15.37:0/1890030133 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f1300109e40 con 0x7f12e406cc20 2024-03-29T08:48:36.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:36.523+0000 7f12f7e00700 1 -- 172.21.15.37:0/1890030133 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+175 (secure 0 0 0) 0x7f12f0004e60 con 0x7f12e406cc20 2024-03-29T08:48:36.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:36.525+0000 7f1307200700 1 -- 172.21.15.37:0/1890030133 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f12e406cc20 msgr2=0x7f12e406f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:48:36.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:36.525+0000 7f1307200700 1 --2- 172.21.15.37:0/1890030133 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f12e406cc20 0x7f12e406f0d0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f12f0004990 tx=0x7f12f0009ee0 comp rx=0 tx=0).stop 2024-03-29T08:48:36.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:36.525+0000 7f1307200700 1 -- 172.21.15.37:0/1890030133 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f13001037a0 msgr2=0x7f1300070800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:48:36.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:36.525+0000 7f1307200700 1 --2- 172.21.15.37:0/1890030133 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f13001037a0 0x7f1300070800 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f12ec0093b0 tx=0x7f12ec009e30 comp rx=0 tx=0).stop 2024-03-29T08:48:36.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:36.526+0000 7f1307200700 1 -- 172.21.15.37:0/1890030133 shutdown_connections 2024-03-29T08:48:36.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:36.526+0000 7f1307200700 1 --2- 172.21.15.37:0/1890030133 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f12e406cc20 0x7f12e406f0d0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:48:36.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:36.526+0000 7f1307200700 1 --2- 172.21.15.37:0/1890030133 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1300102e00 0x7f13000702c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:48:36.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:36.526+0000 7f1307200700 1 --2- 172.21.15.37:0/1890030133 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f13001037a0 0x7f1300070800 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:48:36.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:36.526+0000 7f1307200700 1 -- 172.21.15.37:0/1890030133 >> 172.21.15.37:0/1890030133 conn(0x7f13000f89b0 msgr2=0x7f13000f9ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:48:36.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:36.526+0000 7f1307200700 1 -- 172.21.15.37:0/1890030133 shutdown_connections 2024-03-29T08:48:36.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:36.526+0000 7f1307200700 1 -- 172.21.15.37:0/1890030133 wait complete. 2024-03-29T08:48:38.297 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:48:37 smithi037 ceph-mon[142325]: from='client.36018 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:48:38.297 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:48:37 smithi037 ceph-mon[142325]: pgmap v1134: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail; 17 MiB/s rd, 597 B/s wr, 11 op/s 2024-03-29T08:48:38.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:48:37 smithi148 ceph-mon[106692]: from='client.36018 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:48:38.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:48:37 smithi148 ceph-mon[106692]: pgmap v1134: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail; 17 MiB/s rd, 597 B/s wr, 11 op/s 2024-03-29T08:48:38.704 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 99b492c2-eda1-11ee-b647-cb9ed24678a4 -e sha1=d0e392d9aee635f2b65768c460c7a151ead7a5e2 -- bash -c 'ceph orch ps' 2024-03-29T08:48:39.277 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/mon.smithi037/config 2024-03-29T08:48:40.011 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:48:39 smithi037 ceph-mon[142325]: pgmap v1135: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail; 7.0 MiB/s rd, 4.5 KiB/s wr, 10 op/s 2024-03-29T08:48:40.011 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:48:39 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:48:40.011 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:48:39 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:48:40.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:48:39 smithi148 ceph-mon[106692]: pgmap v1135: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail; 7.0 MiB/s rd, 4.5 KiB/s wr, 10 op/s 2024-03-29T08:48:40.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:48:39 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:48:40.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:48:39 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:48:41.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:48:41 smithi037 ceph-mon[142325]: pgmap v1136: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail; 4.1 MiB/s rd, 8.7 KiB/s wr, 11 op/s 2024-03-29T08:48:41.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:48:41 smithi148 ceph-mon[106692]: pgmap v1136: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail; 4.1 MiB/s rd, 8.7 KiB/s wr, 11 op/s 2024-03-29T08:48:41.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:41.697+0000 7fd84ea00700 1 -- 172.21.15.37:0/2825865226 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd8481035e0 msgr2=0x7fd8480fcc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:48:41.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:41.697+0000 7fd84ea00700 1 --2- 172.21.15.37:0/2825865226 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd8481035e0 0x7fd8480fcc50 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7fd834004940 tx=0x7fd834009e30 comp rx=0 tx=0).stop 2024-03-29T08:48:41.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:41.697+0000 7fd84ea00700 1 -- 172.21.15.37:0/2825865226 shutdown_connections 2024-03-29T08:48:41.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:41.697+0000 7fd84ea00700 1 --2- 172.21.15.37:0/2825865226 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd8481035e0 0x7fd8480fcc50 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:48:41.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:41.697+0000 7fd84ea00700 1 --2- 172.21.15.37:0/2825865226 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd848102c40 0x7fd848103010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:48:41.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:41.697+0000 7fd84ea00700 1 -- 172.21.15.37:0/2825865226 >> 172.21.15.37:0/2825865226 conn(0x7fd8480f8830 msgr2=0x7fd8480fac40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:48:41.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:41.698+0000 7fd84ea00700 1 -- 172.21.15.37:0/2825865226 shutdown_connections 2024-03-29T08:48:41.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:41.698+0000 7fd84ea00700 1 -- 172.21.15.37:0/2825865226 wait complete. 2024-03-29T08:48:41.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:41.698+0000 7fd84ea00700 1 Processor -- start 2024-03-29T08:48:41.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:41.699+0000 7fd84ea00700 1 -- start start 2024-03-29T08:48:41.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:41.699+0000 7fd84ea00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd848102c40 0x7fd8480702c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:48:41.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:41.699+0000 7fd84ea00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd8481035e0 0x7fd848070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:48:41.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:41.699+0000 7fd84ea00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8480720b0 con 0x7fd848102c40 2024-03-29T08:48:41.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:41.699+0000 7fd84ea00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd848072220 con 0x7fd8481035e0 2024-03-29T08:48:41.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:41.699+0000 7fd84d600700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd848102c40 0x7fd8480702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:48:41.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:41.699+0000 7fd84d600700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd848102c40 0x7fd8480702c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45614/0 (socket says 172.21.15.37:45614) 2024-03-29T08:48:41.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:41.699+0000 7fd84d600700 1 -- 172.21.15.37:0/654240391 learned_addr learned my addr 172.21.15.37:0/654240391 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:48:41.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:41.699+0000 7fd84cc00700 1 --2- 172.21.15.37:0/654240391 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd8481035e0 0x7fd848070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:48:41.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:41.700+0000 7fd84d600700 1 -- 172.21.15.37:0/654240391 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd8481035e0 msgr2=0x7fd848070800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:48:41.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:41.700+0000 7fd84d600700 1 --2- 172.21.15.37:0/654240391 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd8481035e0 0x7fd848070800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:48:41.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:41.700+0000 7fd84d600700 1 -- 172.21.15.37:0/654240391 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd834005040 con 0x7fd848102c40 2024-03-29T08:48:41.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:41.700+0000 7fd84d600700 1 --2- 172.21.15.37:0/654240391 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd848102c40 0x7fd8480702c0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7fd83c002a50 tx=0x7fd83c0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:48:41.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:41.700+0000 7fd845600700 1 -- 172.21.15.37:0/654240391 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd83c007cc0 con 0x7fd848102c40 2024-03-29T08:48:41.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:41.700+0000 7fd845600700 1 -- 172.21.15.37:0/654240391 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd83c00f040 con 0x7fd848102c40 2024-03-29T08:48:41.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:41.700+0000 7fd84ea00700 1 -- 172.21.15.37:0/654240391 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd848070e60 con 0x7fd848102c40 2024-03-29T08:48:41.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:41.700+0000 7fd845600700 1 -- 172.21.15.37:0/654240391 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd83c00d9c0 con 0x7fd848102c40 2024-03-29T08:48:41.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:41.701+0000 7fd84ea00700 1 -- 172.21.15.37:0/654240391 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd84811e570 con 0x7fd848102c40 2024-03-29T08:48:41.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:41.702+0000 7fd84ea00700 1 -- 172.21.15.37:0/654240391 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8480fe390 con 0x7fd848102c40 2024-03-29T08:48:41.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:41.702+0000 7fd845600700 1 -- 172.21.15.37:0/654240391 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fd83c00db20 con 0x7fd848102c40 2024-03-29T08:48:41.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:41.703+0000 7fd845600700 1 --2- 172.21.15.37:0/654240391 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fd83006cc30 0x7fd83006f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:48:41.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:41.703+0000 7fd845600700 1 -- 172.21.15.37:0/654240391 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fd83c08bde0 con 0x7fd848102c40 2024-03-29T08:48:41.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:41.703+0000 7fd84cc00700 1 --2- 172.21.15.37:0/654240391 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fd83006cc30 0x7fd83006f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:48:41.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:41.704+0000 7fd84cc00700 1 --2- 172.21.15.37:0/654240391 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fd83006cc30 0x7fd83006f0e0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fd848071c00 tx=0x7fd834009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:48:41.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:41.708+0000 7fd845600700 1 -- 172.21.15.37:0/654240391 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fd83c0597b0 con 0x7fd848102c40 2024-03-29T08:48:41.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:41.932+0000 7fd84ea00700 1 -- 172.21.15.37:0/654240391 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fd8480719b0 con 0x7fd83006cc30 2024-03-29T08:48:41.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:41.943+0000 7fd845600700 1 -- 172.21.15.37:0/654240391 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7fd8480719b0 con 0x7fd83006cc30 2024-03-29T08:48:41.942 INFO:teuthology.orchestra.run.smithi037.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-29T08:48:41.943 INFO:teuthology.orchestra.run.smithi037.stdout:alertmanager.smithi037 smithi037 *:9093,9094 running (38m) 11s ago 51m 14.5M - 0.25.0 c8568f914cd2 2f9f9746ed0f 2024-03-29T08:48:41.943 INFO:teuthology.orchestra.run.smithi037.stdout:crash.smithi037 smithi037 running (35m) 11s ago 51m 6647k - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 178d416a3186 2024-03-29T08:48:41.943 INFO:teuthology.orchestra.run.smithi037.stdout:crash.smithi148 smithi148 running (35m) 16s ago 50m 6656k - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 23666989d58f 2024-03-29T08:48:41.943 INFO:teuthology.orchestra.run.smithi037.stdout:grafana.smithi037 smithi037 *:3000 running (38m) 11s ago 51m 73.3M - 9.4.7 954c08fa6188 6b976fe49952 2024-03-29T08:48:41.943 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi037.iifsmh smithi037 running (30m) 11s ago 44m 96.6M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 2d97f562cc5b 2024-03-29T08:48:41.943 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi037.mmwgxe smithi037 running (31m) 11s ago 45m 86.5M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff ec37037f46e8 2024-03-29T08:48:41.943 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi148.qukpjh smithi148 running (15m) 16s ago 44m 16.4M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 3257219bc046 2024-03-29T08:48:41.943 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi148.sqteih smithi148 running (30m) 16s ago 45m 21.5M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 44c15c77037a 2024-03-29T08:48:41.943 INFO:teuthology.orchestra.run.smithi037.stdout:mgr.smithi037.wjikii smithi037 *:8443,9283,8765 running (40m) 11s ago 53m 513M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff c5a1417b9417 2024-03-29T08:48:41.943 INFO:teuthology.orchestra.run.smithi037.stdout:mgr.smithi148.onhgvm smithi148 *:8443,9283,8765 running (39m) 16s ago 50m 457M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff fcfdbce062bd 2024-03-29T08:48:41.943 INFO:teuthology.orchestra.run.smithi037.stdout:mon.smithi037 smithi037 running (36m) 11s ago 53m 85.2M 2048M 18.2.2-1015-gd0e392d9 1cc2f45c17ff eea2060783af 2024-03-29T08:48:41.943 INFO:teuthology.orchestra.run.smithi037.stdout:mon.smithi148 smithi148 running (36m) 16s ago 50m 56.8M 2048M 18.2.2-1015-gd0e392d9 1cc2f45c17ff ad4b2c5748be 2024-03-29T08:48:41.943 INFO:teuthology.orchestra.run.smithi037.stdout:node-exporter.smithi037 smithi037 *:9100 running (39m) 11s ago 51m 8280k - 1.5.0 0da6a335fe13 26b8ae9cb878 2024-03-29T08:48:41.943 INFO:teuthology.orchestra.run.smithi037.stdout:node-exporter.smithi148 smithi148 *:9100 running (39m) 16s ago 50m 8753k - 1.5.0 0da6a335fe13 98414e4e64ed 2024-03-29T08:48:41.944 INFO:teuthology.orchestra.run.smithi037.stdout:osd.0 smithi037 running (34m) 11s ago 49m 112M 1619M 18.2.2-1015-gd0e392d9 1cc2f45c17ff c0c584f6245c 2024-03-29T08:48:41.944 INFO:teuthology.orchestra.run.smithi037.stdout:osd.1 smithi037 running (34m) 11s ago 48m 109M 1619M 18.2.2-1015-gd0e392d9 1cc2f45c17ff a4a2ff4cc83d 2024-03-29T08:48:41.944 INFO:teuthology.orchestra.run.smithi037.stdout:osd.2 smithi037 running (33m) 11s ago 48m 88.0M 1619M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 8b1bc71aa79e 2024-03-29T08:48:41.944 INFO:teuthology.orchestra.run.smithi037.stdout:osd.3 smithi148 running (32m) 16s ago 47m 114M 2642M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 1c656ab8ff86 2024-03-29T08:48:41.944 INFO:teuthology.orchestra.run.smithi037.stdout:osd.4 smithi148 running (32m) 16s ago 47m 103M 2642M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 966a9de11d22 2024-03-29T08:48:41.944 INFO:teuthology.orchestra.run.smithi037.stdout:osd.5 smithi148 running (31m) 16s ago 46m 95.6M 2642M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 474c4f9212ce 2024-03-29T08:48:41.944 INFO:teuthology.orchestra.run.smithi037.stdout:prometheus.smithi037 smithi037 *:9095 running (38m) 11s ago 50m 68.6M - 2.43.0 a07b618ecd1d 30c0223ec589 2024-03-29T08:48:41.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:41.945+0000 7fd84ea00700 1 -- 172.21.15.37:0/654240391 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fd83006cc30 msgr2=0x7fd83006f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:48:41.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:41.945+0000 7fd84ea00700 1 --2- 172.21.15.37:0/654240391 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fd83006cc30 0x7fd83006f0e0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fd848071c00 tx=0x7fd834009e30 comp rx=0 tx=0).stop 2024-03-29T08:48:41.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:41.945+0000 7fd84ea00700 1 -- 172.21.15.37:0/654240391 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd848102c40 msgr2=0x7fd8480702c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:48:41.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:41.945+0000 7fd84ea00700 1 --2- 172.21.15.37:0/654240391 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd848102c40 0x7fd8480702c0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7fd83c002a50 tx=0x7fd83c0046b0 comp rx=0 tx=0).stop 2024-03-29T08:48:41.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:41.945+0000 7fd84ea00700 1 -- 172.21.15.37:0/654240391 shutdown_connections 2024-03-29T08:48:41.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:41.945+0000 7fd84ea00700 1 --2- 172.21.15.37:0/654240391 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fd83006cc30 0x7fd83006f0e0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:48:41.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:41.945+0000 7fd84ea00700 1 --2- 172.21.15.37:0/654240391 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd848102c40 0x7fd8480702c0 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:48:41.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:41.945+0000 7fd84ea00700 1 --2- 172.21.15.37:0/654240391 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd8481035e0 0x7fd848070800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:48:41.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:41.945+0000 7fd84ea00700 1 -- 172.21.15.37:0/654240391 >> 172.21.15.37:0/654240391 conn(0x7fd8480f8830 msgr2=0x7fd8480f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:48:41.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:41.945+0000 7fd84ea00700 1 -- 172.21.15.37:0/654240391 shutdown_connections 2024-03-29T08:48:41.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:41.945+0000 7fd84ea00700 1 -- 172.21.15.37:0/654240391 wait complete. 2024-03-29T08:48:42.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:48:42 smithi148 ceph-mon[106692]: from='client.36022 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:48:42.426 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:48:42 smithi037 ceph-mon[142325]: from='client.36022 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:48:42.857 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 99b492c2-eda1-11ee-b647-cb9ed24678a4 -e sha1=d0e392d9aee635f2b65768c460c7a151ead7a5e2 -- bash -c 'ceph orch upgrade status' 2024-03-29T08:48:43.299 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/mon.smithi037/config 2024-03-29T08:48:43.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:48:43 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:48:43.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:48:43 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:48:43.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:48:43 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T08:48:43.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:48:43 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T08:48:43.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:48:43 smithi037 ceph-mon[142325]: pgmap v1137: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail; 370 KiB/s rd, 9.3 KiB/s wr, 11 op/s 2024-03-29T08:48:43.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:48:43 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:48:43.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:48:43 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T08:48:43.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:48:43 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T08:48:43.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:48:43 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T08:48:43.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:48:43 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:48:43.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:48:43 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:48:43.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:48:43 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:48:43.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:48:43 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T08:48:43.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:48:43 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T08:48:43.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:48:43 smithi148 ceph-mon[106692]: pgmap v1137: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail; 370 KiB/s rd, 9.3 KiB/s wr, 11 op/s 2024-03-29T08:48:43.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:48:43 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:48:43.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:48:43 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T08:48:43.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:48:43 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T08:48:43.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:48:43 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T08:48:43.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:48:43 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:48:44.426 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:48:44 smithi037 ceph-mon[142325]: Health check cleared: CEPHADM_REFRESH_FAILED (was: failed to probe daemons or devices) 2024-03-29T08:48:44.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:48:44 smithi037 ceph-mon[142325]: Cluster is now healthy 2024-03-29T08:48:44.555 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:48:44 smithi148 ceph-mon[106692]: Health check cleared: CEPHADM_REFRESH_FAILED (was: failed to probe daemons or devices) 2024-03-29T08:48:44.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:48:44 smithi148 ceph-mon[106692]: Cluster is now healthy 2024-03-29T08:48:45.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:45.279+0000 7f648f200700 1 -- 172.21.15.37:0/2574942193 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f64881037a0 msgr2=0x7f64880fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:48:45.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:45.279+0000 7f648f200700 1 --2- 172.21.15.37:0/2574942193 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f64881037a0 0x7f64880fce10 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f6474004940 tx=0x7f6474009e30 comp rx=0 tx=0).stop 2024-03-29T08:48:45.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:45.280+0000 7f648f200700 1 -- 172.21.15.37:0/2574942193 shutdown_connections 2024-03-29T08:48:45.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:45.280+0000 7f648f200700 1 --2- 172.21.15.37:0/2574942193 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f64881037a0 0x7f64880fce10 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:48:45.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:45.280+0000 7f648f200700 1 --2- 172.21.15.37:0/2574942193 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6488102e00 0x7f64881031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:48:45.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:45.280+0000 7f648f200700 1 -- 172.21.15.37:0/2574942193 >> 172.21.15.37:0/2574942193 conn(0x7f64880f89b0 msgr2=0x7f64880fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:48:45.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:45.280+0000 7f648f200700 1 -- 172.21.15.37:0/2574942193 shutdown_connections 2024-03-29T08:48:45.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:45.280+0000 7f648f200700 1 -- 172.21.15.37:0/2574942193 wait complete. 2024-03-29T08:48:45.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:45.281+0000 7f648f200700 1 Processor -- start 2024-03-29T08:48:45.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:45.281+0000 7f648f200700 1 -- start start 2024-03-29T08:48:45.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:45.281+0000 7f648f200700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6488102e00 0x7f6488115d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:48:45.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:45.281+0000 7f648f200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f64881037a0 0x7f6488116240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:48:45.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:45.281+0000 7f648f200700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6488116890 con 0x7f64881037a0 2024-03-29T08:48:45.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:45.281+0000 7f648f200700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f648810fc60 con 0x7f6488102e00 2024-03-29T08:48:45.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:45.281+0000 7f6487400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f64881037a0 0x7f6488116240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:48:45.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:45.281+0000 7f6487400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f64881037a0 0x7f6488116240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45632/0 (socket says 172.21.15.37:45632) 2024-03-29T08:48:45.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:45.281+0000 7f6487400700 1 -- 172.21.15.37:0/1855726072 learned_addr learned my addr 172.21.15.37:0/1855726072 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:48:45.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:45.282+0000 7f6487e00700 1 --2- 172.21.15.37:0/1855726072 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6488102e00 0x7f6488115d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:48:45.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:45.282+0000 7f6487400700 1 -- 172.21.15.37:0/1855726072 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6488102e00 msgr2=0x7f6488115d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:48:45.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:45.282+0000 7f6487400700 1 --2- 172.21.15.37:0/1855726072 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6488102e00 0x7f6488115d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:48:45.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:45.282+0000 7f6487400700 1 -- 172.21.15.37:0/1855726072 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6474005040 con 0x7f64881037a0 2024-03-29T08:48:45.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:45.282+0000 7f6487e00700 1 --2- 172.21.15.37:0/1855726072 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6488102e00 0x7f6488115d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:48:45.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:45.282+0000 7f6487400700 1 --2- 172.21.15.37:0/1855726072 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f64881037a0 0x7f6488116240 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f6474009390 tx=0x7f6474009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:48:45.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:45.282+0000 7f647fe00700 1 -- 172.21.15.37:0/1855726072 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f647400e040 con 0x7f64881037a0 2024-03-29T08:48:45.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:45.282+0000 7f647fe00700 1 -- 172.21.15.37:0/1855726072 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f6474013420 con 0x7f64881037a0 2024-03-29T08:48:45.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:45.282+0000 7f647fe00700 1 -- 172.21.15.37:0/1855726072 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f647401c630 con 0x7f64881037a0 2024-03-29T08:48:45.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:45.282+0000 7f648f200700 1 -- 172.21.15.37:0/1855726072 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f648810fee0 con 0x7f64881037a0 2024-03-29T08:48:45.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:45.283+0000 7f648f200700 1 -- 172.21.15.37:0/1855726072 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64881104a0 con 0x7f64881037a0 2024-03-29T08:48:45.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:45.284+0000 7f648f200700 1 -- 172.21.15.37:0/1855726072 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64880fe550 con 0x7f64881037a0 2024-03-29T08:48:45.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:45.285+0000 7f647fe00700 1 -- 172.21.15.37:0/1855726072 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f6474013a00 con 0x7f64881037a0 2024-03-29T08:48:45.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:45.286+0000 7f647fe00700 1 --2- 172.21.15.37:0/1855726072 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f646c06cc30 0x7f646c06f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:48:45.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:45.286+0000 7f647fe00700 1 -- 172.21.15.37:0/1855726072 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f647408d2c0 con 0x7f64881037a0 2024-03-29T08:48:45.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:45.286+0000 7f6487e00700 1 --2- 172.21.15.37:0/1855726072 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f646c06cc30 0x7f646c06f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:48:45.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:45.287+0000 7f6487e00700 1 --2- 172.21.15.37:0/1855726072 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f646c06cc30 0x7f646c06f0e0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f6478007fd0 tx=0x7f6478009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:48:45.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:45.291+0000 7f647fe00700 1 -- 172.21.15.37:0/1855726072 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f647405ad90 con 0x7f64881037a0 2024-03-29T08:48:45.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:45.522+0000 7f648f200700 1 -- 172.21.15.37:0/1855726072 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6488109da0 con 0x7f646c06cc30 2024-03-29T08:48:45.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:45.523+0000 7f647fe00700 1 -- 172.21.15.37:0/1855726072 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+175 (secure 0 0 0) 0x7f6488109da0 con 0x7f646c06cc30 2024-03-29T08:48:45.523 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-29T08:48:45.523 INFO:teuthology.orchestra.run.smithi037.stdout: "target_image": null, 2024-03-29T08:48:45.523 INFO:teuthology.orchestra.run.smithi037.stdout: "in_progress": false, 2024-03-29T08:48:45.523 INFO:teuthology.orchestra.run.smithi037.stdout: "which": "", 2024-03-29T08:48:45.523 INFO:teuthology.orchestra.run.smithi037.stdout: "services_complete": [], 2024-03-29T08:48:45.523 INFO:teuthology.orchestra.run.smithi037.stdout: "progress": null, 2024-03-29T08:48:45.523 INFO:teuthology.orchestra.run.smithi037.stdout: "message": "", 2024-03-29T08:48:45.523 INFO:teuthology.orchestra.run.smithi037.stdout: "is_paused": false 2024-03-29T08:48:45.523 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-29T08:48:45.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:45.526+0000 7f648f200700 1 -- 172.21.15.37:0/1855726072 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f646c06cc30 msgr2=0x7f646c06f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:48:45.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:45.526+0000 7f648f200700 1 --2- 172.21.15.37:0/1855726072 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f646c06cc30 0x7f646c06f0e0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f6478007fd0 tx=0x7f6478009040 comp rx=0 tx=0).stop 2024-03-29T08:48:45.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:45.526+0000 7f648f200700 1 -- 172.21.15.37:0/1855726072 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f64881037a0 msgr2=0x7f6488116240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:48:45.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:45.526+0000 7f648f200700 1 --2- 172.21.15.37:0/1855726072 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f64881037a0 0x7f6488116240 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f6474009390 tx=0x7f6474009e30 comp rx=0 tx=0).stop 2024-03-29T08:48:45.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:45.526+0000 7f648f200700 1 -- 172.21.15.37:0/1855726072 shutdown_connections 2024-03-29T08:48:45.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:45.526+0000 7f648f200700 1 --2- 172.21.15.37:0/1855726072 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f646c06cc30 0x7f646c06f0e0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:48:45.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:45.526+0000 7f648f200700 1 --2- 172.21.15.37:0/1855726072 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6488102e00 0x7f6488115d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:48:45.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:45.526+0000 7f648f200700 1 --2- 172.21.15.37:0/1855726072 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f64881037a0 0x7f6488116240 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:48:45.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:45.526+0000 7f648f200700 1 -- 172.21.15.37:0/1855726072 >> 172.21.15.37:0/1855726072 conn(0x7f64880f89b0 msgr2=0x7f64880f9ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:48:45.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:45.526+0000 7f648f200700 1 -- 172.21.15.37:0/1855726072 shutdown_connections 2024-03-29T08:48:45.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:45.526+0000 7f648f200700 1 -- 172.21.15.37:0/1855726072 wait complete. 2024-03-29T08:48:45.828 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:48:45 smithi037 ceph-mon[142325]: pgmap v1138: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail; 6.7 KiB/s rd, 9.3 KiB/s wr, 11 op/s 2024-03-29T08:48:46.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:48:45 smithi148 ceph-mon[106692]: pgmap v1138: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail; 6.7 KiB/s rd, 9.3 KiB/s wr, 11 op/s 2024-03-29T08:48:46.146 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 99b492c2-eda1-11ee-b647-cb9ed24678a4 -e sha1=d0e392d9aee635f2b65768c460c7a151ead7a5e2 -- bash -c 'ceph health detail' 2024-03-29T08:48:46.583 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/mon.smithi037/config 2024-03-29T08:48:46.907 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:48:46 smithi037 ceph-mon[142325]: from='client.36026 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:48:47.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:48:46 smithi148 ceph-mon[106692]: from='client.36026 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:48:47.676 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:48:47 smithi037 ceph-mon[142325]: pgmap v1139: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail; 3.2 KiB/s rd, 8.9 KiB/s wr, 7 op/s 2024-03-29T08:48:48.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:48:47 smithi148 ceph-mon[106692]: pgmap v1139: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail; 3.2 KiB/s rd, 8.9 KiB/s wr, 7 op/s 2024-03-29T08:48:48.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:48.465+0000 7fa977800700 1 -- 172.21.15.37:0/576941000 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa9701037a0 msgr2=0x7fa9700fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:48:48.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:48.465+0000 7fa977800700 1 --2- 172.21.15.37:0/576941000 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa9701037a0 0x7fa9700fce10 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7fa95c004940 tx=0x7fa95c009e30 comp rx=0 tx=0).stop 2024-03-29T08:48:48.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:48.465+0000 7fa977800700 1 -- 172.21.15.37:0/576941000 shutdown_connections 2024-03-29T08:48:48.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:48.465+0000 7fa977800700 1 --2- 172.21.15.37:0/576941000 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa9701037a0 0x7fa9700fce10 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:48:48.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:48.465+0000 7fa977800700 1 --2- 172.21.15.37:0/576941000 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa970102e00 0x7fa9701031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:48:48.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:48.465+0000 7fa977800700 1 -- 172.21.15.37:0/576941000 >> 172.21.15.37:0/576941000 conn(0x7fa9700f89b0 msgr2=0x7fa9700fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:48:48.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:48.465+0000 7fa977800700 1 -- 172.21.15.37:0/576941000 shutdown_connections 2024-03-29T08:48:48.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:48.465+0000 7fa977800700 1 -- 172.21.15.37:0/576941000 wait complete. 2024-03-29T08:48:48.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:48.466+0000 7fa977800700 1 Processor -- start 2024-03-29T08:48:48.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:48.466+0000 7fa977800700 1 -- start start 2024-03-29T08:48:48.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:48.467+0000 7fa977800700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa970102e00 0x7fa9700702c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:48:48.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:48.467+0000 7fa977800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa9701037a0 0x7fa970070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:48:48.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:48.467+0000 7fa977800700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa970070e20 con 0x7fa9701037a0 2024-03-29T08:48:48.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:48.467+0000 7fa977800700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa970070f60 con 0x7fa970102e00 2024-03-29T08:48:48.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:48.467+0000 7fa96fe00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa9701037a0 0x7fa970070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:48:48.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:48.467+0000 7fa96fe00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa9701037a0 0x7fa970070800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45648/0 (socket says 172.21.15.37:45648) 2024-03-29T08:48:48.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:48.467+0000 7fa96fe00700 1 -- 172.21.15.37:0/793829100 learned_addr learned my addr 172.21.15.37:0/793829100 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:48:48.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:48.467+0000 7fa974e00700 1 --2- 172.21.15.37:0/793829100 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa970102e00 0x7fa9700702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:48:48.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:48.468+0000 7fa96fe00700 1 -- 172.21.15.37:0/793829100 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa970102e00 msgr2=0x7fa9700702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:48:48.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:48.468+0000 7fa96fe00700 1 --2- 172.21.15.37:0/793829100 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa970102e00 0x7fa9700702c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:48:48.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:48.468+0000 7fa96fe00700 1 -- 172.21.15.37:0/793829100 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa95c005040 con 0x7fa9701037a0 2024-03-29T08:48:48.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:48.468+0000 7fa96fe00700 1 --2- 172.21.15.37:0/793829100 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa9701037a0 0x7fa970070800 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7fa95c009390 tx=0x7fa95c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:48:48.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:48.468+0000 7fa96cc00700 1 -- 172.21.15.37:0/793829100 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa95c00e040 con 0x7fa9701037a0 2024-03-29T08:48:48.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:48.468+0000 7fa96cc00700 1 -- 172.21.15.37:0/793829100 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fa95c013420 con 0x7fa9701037a0 2024-03-29T08:48:48.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:48.468+0000 7fa96cc00700 1 -- 172.21.15.37:0/793829100 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa95c01c630 con 0x7fa9701037a0 2024-03-29T08:48:48.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:48.468+0000 7fa977800700 1 -- 172.21.15.37:0/793829100 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa970119de0 con 0x7fa9701037a0 2024-03-29T08:48:48.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:48.469+0000 7fa977800700 1 -- 172.21.15.37:0/793829100 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa97011a2c0 con 0x7fa9701037a0 2024-03-29T08:48:48.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:48.470+0000 7fa977800700 1 -- 172.21.15.37:0/793829100 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9700fe550 con 0x7fa9701037a0 2024-03-29T08:48:48.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:48.471+0000 7fa96cc00700 1 -- 172.21.15.37:0/793829100 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fa95c013a00 con 0x7fa9701037a0 2024-03-29T08:48:48.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:48.472+0000 7fa96cc00700 1 --2- 172.21.15.37:0/793829100 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fa95806cc30 0x7fa95806f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:48:48.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:48.472+0000 7fa96cc00700 1 -- 172.21.15.37:0/793829100 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fa95c08d120 con 0x7fa9701037a0 2024-03-29T08:48:48.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:48.472+0000 7fa974e00700 1 --2- 172.21.15.37:0/793829100 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fa95806cc30 0x7fa95806f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:48:48.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:48.473+0000 7fa974e00700 1 --2- 172.21.15.37:0/793829100 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fa95806cc30 0x7fa95806f0e0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fa964009bf0 tx=0x7fa9640091a0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:48:48.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:48.476+0000 7fa96cc00700 1 -- 172.21.15.37:0/793829100 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fa95c057160 con 0x7fa9701037a0 2024-03-29T08:48:48.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:48.781+0000 7fa977800700 1 -- 172.21.15.37:0/793829100 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fa97004bed0 con 0x7fa9701037a0 2024-03-29T08:48:48.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:48.781+0000 7fa96cc00700 1 -- 172.21.15.37:0/793829100 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7fa95c05a780 con 0x7fa9701037a0 2024-03-29T08:48:48.781 INFO:teuthology.orchestra.run.smithi037.stdout:HEALTH_OK 2024-03-29T08:48:48.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:48.784+0000 7fa977800700 1 -- 172.21.15.37:0/793829100 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fa95806cc30 msgr2=0x7fa95806f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:48:48.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:48.784+0000 7fa977800700 1 --2- 172.21.15.37:0/793829100 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fa95806cc30 0x7fa95806f0e0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fa964009bf0 tx=0x7fa9640091a0 comp rx=0 tx=0).stop 2024-03-29T08:48:48.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:48.784+0000 7fa977800700 1 -- 172.21.15.37:0/793829100 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa9701037a0 msgr2=0x7fa970070800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:48:48.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:48.784+0000 7fa977800700 1 --2- 172.21.15.37:0/793829100 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa9701037a0 0x7fa970070800 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7fa95c009390 tx=0x7fa95c009e30 comp rx=0 tx=0).stop 2024-03-29T08:48:48.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:48.784+0000 7fa977800700 1 -- 172.21.15.37:0/793829100 shutdown_connections 2024-03-29T08:48:48.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:48.784+0000 7fa977800700 1 --2- 172.21.15.37:0/793829100 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fa95806cc30 0x7fa95806f0e0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:48:48.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:48.784+0000 7fa977800700 1 --2- 172.21.15.37:0/793829100 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa970102e00 0x7fa9700702c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:48:48.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:48.784+0000 7fa977800700 1 --2- 172.21.15.37:0/793829100 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa9701037a0 0x7fa970070800 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:48:48.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:48.784+0000 7fa977800700 1 -- 172.21.15.37:0/793829100 >> 172.21.15.37:0/793829100 conn(0x7fa9700f89b0 msgr2=0x7fa9700f99c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:48:48.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:48.784+0000 7fa977800700 1 -- 172.21.15.37:0/793829100 shutdown_connections 2024-03-29T08:48:48.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:48.784+0000 7fa977800700 1 -- 172.21.15.37:0/793829100 wait complete. 2024-03-29T08:48:49.461 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 99b492c2-eda1-11ee-b647-cb9ed24678a4 -e sha1=d0e392d9aee635f2b65768c460c7a151ead7a5e2 -- bash -c 'ceph versions' 2024-03-29T08:48:49.899 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/mon.smithi037/config 2024-03-29T08:48:49.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:48:49 smithi037 ceph-mon[142325]: pgmap v1140: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail; 8.7 KiB/s wr, 4 op/s 2024-03-29T08:48:49.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:48:49 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/793829100' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T08:48:50.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:48:49 smithi148 ceph-mon[106692]: pgmap v1140: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail; 8.7 KiB/s wr, 4 op/s 2024-03-29T08:48:50.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:48:49 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/793829100' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-29T08:48:51.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:51.754+0000 7fed6f800700 1 -- 172.21.15.37:0/163802101 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed68102e00 msgr2=0x7fed681031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:48:51.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:51.754+0000 7fed6f800700 1 --2- 172.21.15.37:0/163802101 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed68102e00 0x7fed681031d0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7fed5c004940 tx=0x7fed5c009e30 comp rx=0 tx=0).stop 2024-03-29T08:48:51.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:51.755+0000 7fed6f800700 1 -- 172.21.15.37:0/163802101 shutdown_connections 2024-03-29T08:48:51.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:51.755+0000 7fed6f800700 1 --2- 172.21.15.37:0/163802101 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fed681037a0 0x7fed680fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:48:51.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:51.755+0000 7fed6f800700 1 --2- 172.21.15.37:0/163802101 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed68102e00 0x7fed681031d0 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:48:51.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:51.755+0000 7fed6f800700 1 -- 172.21.15.37:0/163802101 >> 172.21.15.37:0/163802101 conn(0x7fed680f89b0 msgr2=0x7fed680fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:48:51.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:51.755+0000 7fed6f800700 1 -- 172.21.15.37:0/163802101 shutdown_connections 2024-03-29T08:48:51.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:51.755+0000 7fed6f800700 1 -- 172.21.15.37:0/163802101 wait complete. 2024-03-29T08:48:51.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:51.755+0000 7fed6f800700 1 Processor -- start 2024-03-29T08:48:51.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:51.756+0000 7fed6f800700 1 -- start start 2024-03-29T08:48:51.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:51.756+0000 7fed6f800700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fed68102e00 0x7fed68115cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:48:51.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:51.756+0000 7fed6f800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed681037a0 0x7fed68116230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:48:51.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:51.756+0000 7fed6f800700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed68116880 con 0x7fed681037a0 2024-03-29T08:48:51.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:51.756+0000 7fed6f800700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed6810fc50 con 0x7fed68102e00 2024-03-29T08:48:51.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:51.756+0000 7fed67e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed681037a0 0x7fed68116230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:48:51.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:51.756+0000 7fed67e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed681037a0 0x7fed68116230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:32920/0 (socket says 172.21.15.37:32920) 2024-03-29T08:48:51.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:51.756+0000 7fed67e00700 1 -- 172.21.15.37:0/1174078820 learned_addr learned my addr 172.21.15.37:0/1174078820 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:48:51.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:51.756+0000 7fed6ce00700 1 --2- 172.21.15.37:0/1174078820 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fed68102e00 0x7fed68115cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:48:51.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:51.757+0000 7fed67e00700 1 -- 172.21.15.37:0/1174078820 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fed68102e00 msgr2=0x7fed68115cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:48:51.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:51.757+0000 7fed67e00700 1 --2- 172.21.15.37:0/1174078820 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fed68102e00 0x7fed68115cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:48:51.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:51.757+0000 7fed67e00700 1 -- 172.21.15.37:0/1174078820 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed5c005040 con 0x7fed681037a0 2024-03-29T08:48:51.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:51.757+0000 7fed67e00700 1 --2- 172.21.15.37:0/1174078820 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed681037a0 0x7fed68116230 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7fed54002a50 tx=0x7fed54004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:48:51.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:51.757+0000 7fed64c00700 1 -- 172.21.15.37:0/1174078820 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fed54007cc0 con 0x7fed681037a0 2024-03-29T08:48:51.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:51.757+0000 7fed64c00700 1 -- 172.21.15.37:0/1174078820 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fed5400f040 con 0x7fed681037a0 2024-03-29T08:48:51.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:51.757+0000 7fed6f800700 1 -- 172.21.15.37:0/1174078820 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed6810ff30 con 0x7fed681037a0 2024-03-29T08:48:51.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:51.757+0000 7fed64c00700 1 -- 172.21.15.37:0/1174078820 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fed5400d920 con 0x7fed681037a0 2024-03-29T08:48:51.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:51.757+0000 7fed6f800700 1 -- 172.21.15.37:0/1174078820 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed681104f0 con 0x7fed681037a0 2024-03-29T08:48:51.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:51.759+0000 7fed6f800700 1 -- 172.21.15.37:0/1174078820 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed680fe550 con 0x7fed681037a0 2024-03-29T08:48:51.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:51.759+0000 7fed64c00700 1 -- 172.21.15.37:0/1174078820 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fed54009680 con 0x7fed681037a0 2024-03-29T08:48:51.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:51.761+0000 7fed64c00700 1 --2- 172.21.15.37:0/1174078820 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fed5006cc20 0x7fed5006f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:48:51.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:51.761+0000 7fed64c00700 1 -- 172.21.15.37:0/1174078820 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fed5408bdb0 con 0x7fed681037a0 2024-03-29T08:48:51.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:51.761+0000 7fed6ce00700 1 --2- 172.21.15.37:0/1174078820 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fed5006cc20 0x7fed5006f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:48:51.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:51.761+0000 7fed6ce00700 1 --2- 172.21.15.37:0/1174078820 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fed5006cc20 0x7fed5006f0d0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fed5c004f90 tx=0x7fed5c009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:48:51.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:51.765+0000 7fed64c00700 1 -- 172.21.15.37:0/1174078820 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fed54059780 con 0x7fed681037a0 2024-03-29T08:48:51.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:48:51 smithi037 ceph-mon[142325]: pgmap v1141: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail; 4.5 KiB/s wr, 2 op/s 2024-03-29T08:48:52.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:48:51 smithi148 ceph-mon[106692]: pgmap v1141: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail; 4.5 KiB/s wr, 2 op/s 2024-03-29T08:48:52.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:52.069+0000 7fed6f800700 1 -- 172.21.15.37:0/1174078820 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fed6804bed0 con 0x7fed681037a0 2024-03-29T08:48:52.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:52.070+0000 7fed64c00700 1 -- 172.21.15.37:0/1174078820 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7fed54059310 con 0x7fed681037a0 2024-03-29T08:48:52.069 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-29T08:48:52.070 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": { 2024-03-29T08:48:52.070 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 2 2024-03-29T08:48:52.070 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:48:52.070 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": { 2024-03-29T08:48:52.070 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 2 2024-03-29T08:48:52.070 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:48:52.070 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": { 2024-03-29T08:48:52.070 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 6 2024-03-29T08:48:52.070 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:48:52.071 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": { 2024-03-29T08:48:52.071 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 4 2024-03-29T08:48:52.071 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:48:52.071 INFO:teuthology.orchestra.run.smithi037.stdout: "overall": { 2024-03-29T08:48:52.071 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 14 2024-03-29T08:48:52.071 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-29T08:48:52.071 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-29T08:48:52.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:52.072+0000 7fed6f800700 1 -- 172.21.15.37:0/1174078820 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fed5006cc20 msgr2=0x7fed5006f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:48:52.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:52.072+0000 7fed6f800700 1 --2- 172.21.15.37:0/1174078820 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fed5006cc20 0x7fed5006f0d0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fed5c004f90 tx=0x7fed5c009e30 comp rx=0 tx=0).stop 2024-03-29T08:48:52.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:52.072+0000 7fed6f800700 1 -- 172.21.15.37:0/1174078820 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed681037a0 msgr2=0x7fed68116230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:48:52.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:52.072+0000 7fed6f800700 1 --2- 172.21.15.37:0/1174078820 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed681037a0 0x7fed68116230 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7fed54002a50 tx=0x7fed54004600 comp rx=0 tx=0).stop 2024-03-29T08:48:52.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:52.072+0000 7fed6f800700 1 -- 172.21.15.37:0/1174078820 shutdown_connections 2024-03-29T08:48:52.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:52.072+0000 7fed6f800700 1 --2- 172.21.15.37:0/1174078820 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fed5006cc20 0x7fed5006f0d0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:48:52.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:52.072+0000 7fed6f800700 1 --2- 172.21.15.37:0/1174078820 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fed68102e00 0x7fed68115cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:48:52.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:52.072+0000 7fed6f800700 1 --2- 172.21.15.37:0/1174078820 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed681037a0 0x7fed68116230 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:48:52.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:52.072+0000 7fed6f800700 1 -- 172.21.15.37:0/1174078820 >> 172.21.15.37:0/1174078820 conn(0x7fed680f89b0 msgr2=0x7fed680706d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:48:52.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:52.072+0000 7fed6f800700 1 -- 172.21.15.37:0/1174078820 shutdown_connections 2024-03-29T08:48:52.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:48:52.072+0000 7fed6f800700 1 -- 172.21.15.37:0/1174078820 wait complete. 2024-03-29T08:48:52.763 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:48:52 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/1174078820' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:48:52.788 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 99b492c2-eda1-11ee-b647-cb9ed24678a4 -e sha1=d0e392d9aee635f2b65768c460c7a151ead7a5e2 -- bash -c 'echo "wait for servicemap items w/ changing names to refresh"' 2024-03-29T08:48:53.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:48:52 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/1174078820' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:48:53.226 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/mon.smithi037/config 2024-03-29T08:48:53.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:48:53 smithi037 ceph-mon[142325]: pgmap v1142: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:48:54.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:48:53 smithi148 ceph-mon[106692]: pgmap v1142: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:48:55.162 INFO:teuthology.orchestra.run.smithi037.stdout:wait for servicemap items w/ changing names to refresh 2024-03-29T08:48:55.763 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:48:55 smithi037 ceph-mon[142325]: pgmap v1143: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:48:55.763 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:48:55 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:48:55.763 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:48:55 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:48:55.788 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 99b492c2-eda1-11ee-b647-cb9ed24678a4 -e sha1=d0e392d9aee635f2b65768c460c7a151ead7a5e2 -- bash -c 'sleep 60' 2024-03-29T08:48:56.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:48:55 smithi148 ceph-mon[106692]: pgmap v1143: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:48:56.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:48:55 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:48:56.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:48:55 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:48:56.227 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/mon.smithi037/config 2024-03-29T08:48:57.847 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:48:57 smithi037 ceph-mon[142325]: pgmap v1144: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:48:58.055 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:48:57 smithi148 ceph-mon[106692]: pgmap v1144: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:48:59.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:48:59 smithi037 ceph-mon[142325]: pgmap v1145: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:49:00.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:48:59 smithi148 ceph-mon[106692]: pgmap v1145: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:49:01.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:49:01 smithi037 ceph-mon[142325]: pgmap v1146: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:49:02.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:49:01 smithi148 ceph-mon[106692]: pgmap v1146: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:49:03.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:49:03 smithi037 ceph-mon[142325]: pgmap v1147: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:49:04.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:49:03 smithi148 ceph-mon[106692]: pgmap v1147: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:49:05.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:49:05 smithi037 ceph-mon[142325]: pgmap v1148: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:49:06.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:49:05 smithi148 ceph-mon[106692]: pgmap v1148: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:49:07.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:49:07 smithi037 ceph-mon[142325]: pgmap v1149: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:49:08.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:49:07 smithi148 ceph-mon[106692]: pgmap v1149: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:49:09.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:49:09 smithi037 ceph-mon[142325]: pgmap v1150: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:49:09.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:49:09 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:49:10.055 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:49:09 smithi148 ceph-mon[106692]: pgmap v1150: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:49:10.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:49:09 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:49:11.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:49:11 smithi037 ceph-mon[142325]: pgmap v1151: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:49:12.055 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:49:11 smithi148 ceph-mon[106692]: pgmap v1151: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:49:13.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:49:13 smithi037 ceph-mon[142325]: pgmap v1152: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:49:14.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:49:13 smithi148 ceph-mon[106692]: pgmap v1152: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:49:15.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:49:15 smithi037 ceph-mon[142325]: pgmap v1153: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:49:16.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:49:15 smithi148 ceph-mon[106692]: pgmap v1153: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:49:17.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:49:17 smithi037 ceph-mon[142325]: pgmap v1154: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:49:18.055 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:49:17 smithi148 ceph-mon[106692]: pgmap v1154: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:49:20.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:49:19 smithi148 ceph-mon[106692]: pgmap v1155: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:49:20.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:49:19 smithi037 ceph-mon[142325]: pgmap v1155: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:49:22.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:49:21 smithi148 ceph-mon[106692]: pgmap v1156: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:49:22.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:49:21 smithi037 ceph-mon[142325]: pgmap v1156: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:49:24.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:49:23 smithi148 ceph-mon[106692]: pgmap v1157: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:49:24.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:49:23 smithi037 ceph-mon[142325]: pgmap v1157: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:49:25.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:49:24 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:49:25.176 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:49:24 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:49:26.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:49:25 smithi148 ceph-mon[106692]: pgmap v1158: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:49:26.176 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:49:25 smithi037 ceph-mon[142325]: pgmap v1158: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:49:28.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:49:27 smithi148 ceph-mon[106692]: pgmap v1159: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:49:28.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:49:27 smithi037 ceph-mon[142325]: pgmap v1159: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:49:30.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:49:29 smithi148 ceph-mon[106692]: pgmap v1160: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:49:30.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:49:29 smithi037 ceph-mon[142325]: pgmap v1160: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:49:32.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:49:31 smithi148 ceph-mon[106692]: pgmap v1161: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:49:32.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:49:31 smithi037 ceph-mon[142325]: pgmap v1161: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:49:34.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:49:33 smithi148 ceph-mon[106692]: pgmap v1162: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:49:34.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:49:33 smithi037 ceph-mon[142325]: pgmap v1162: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:49:36.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:49:35 smithi148 ceph-mon[106692]: pgmap v1163: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:49:36.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:49:35 smithi037 ceph-mon[142325]: pgmap v1163: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:49:38.055 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:49:37 smithi148 ceph-mon[106692]: pgmap v1164: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:49:38.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:49:37 smithi037 ceph-mon[142325]: pgmap v1164: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:49:40.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:49:39 smithi148 ceph-mon[106692]: pgmap v1165: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:49:40.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:49:39 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:49:40.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:49:39 smithi037 ceph-mon[142325]: pgmap v1165: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:49:40.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:49:39 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:49:42.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:49:41 smithi148 ceph-mon[106692]: pgmap v1166: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:49:42.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:49:41 smithi037 ceph-mon[142325]: pgmap v1166: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:49:43.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:49:42 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T08:49:43.149 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:49:42 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T08:49:44.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:49:43 smithi148 ceph-mon[106692]: pgmap v1167: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:49:44.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:49:43 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T08:49:44.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:49:43 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T08:49:44.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:49:43 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:49:44.176 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:49:43 smithi037 ceph-mon[142325]: pgmap v1167: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:49:44.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:49:43 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T08:49:44.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:49:43 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T08:49:44.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:49:43 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:49:46.055 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:49:45 smithi148 ceph-mon[106692]: pgmap v1168: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:49:46.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:49:45 smithi037 ceph-mon[142325]: pgmap v1168: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:49:48.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:49:47 smithi148 ceph-mon[106692]: pgmap v1169: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:49:48.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:49:47 smithi037 ceph-mon[142325]: pgmap v1169: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:49:50.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:49:49 smithi148 ceph-mon[106692]: pgmap v1170: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:49:50.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:49:49 smithi037 ceph-mon[142325]: pgmap v1170: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:49:52.055 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:49:51 smithi148 ceph-mon[106692]: pgmap v1171: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:49:52.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:49:51 smithi037 ceph-mon[142325]: pgmap v1171: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:49:54.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:49:53 smithi148 ceph-mon[106692]: pgmap v1172: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:49:54.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:49:53 smithi037 ceph-mon[142325]: pgmap v1172: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:49:55.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:49:54 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:49:55.176 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:49:54 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:49:56.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:49:55 smithi148 ceph-mon[106692]: pgmap v1173: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:49:56.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:49:55 smithi037 ceph-mon[142325]: pgmap v1173: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:49:58.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:49:57 smithi148 ceph-mon[106692]: pgmap v1174: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:49:58.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:49:57 smithi037 ceph-mon[142325]: pgmap v1174: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:49:58.743 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 99b492c2-eda1-11ee-b647-cb9ed24678a4 -e sha1=d0e392d9aee635f2b65768c460c7a151ead7a5e2 -- bash -c 'ceph orch ps' 2024-03-29T08:49:59.187 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/mon.smithi037/config 2024-03-29T08:50:00.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:49:59 smithi148 ceph-mon[106692]: pgmap v1175: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:50:00.076 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:49:59 smithi037 ceph-mon[142325]: pgmap v1175: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:50:01.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:50:00 smithi148 ceph-mon[106692]: overall HEALTH_OK 2024-03-29T08:50:01.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:50:00 smithi037 ceph-mon[142325]: overall HEALTH_OK 2024-03-29T08:50:01.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:01.209+0000 7f3e41400700 1 -- 172.21.15.37:0/3301780114 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3e3c070300 msgr2=0x7f3e3c0706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:50:01.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:01.209+0000 7f3e41400700 1 --2- 172.21.15.37:0/3301780114 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3e3c070300 0x7f3e3c0706d0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f3e2c004970 tx=0x7f3e2c009e30 comp rx=0 tx=0).stop 2024-03-29T08:50:01.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:01.209+0000 7f3e41400700 1 -- 172.21.15.37:0/3301780114 shutdown_connections 2024-03-29T08:50:01.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:01.209+0000 7f3e41400700 1 --2- 172.21.15.37:0/3301780114 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3e3c070c10 0x7f3e3c109190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:01.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:01.209+0000 7f3e41400700 1 --2- 172.21.15.37:0/3301780114 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3e3c070300 0x7f3e3c0706d0 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:01.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:01.209+0000 7f3e41400700 1 -- 172.21.15.37:0/3301780114 >> 172.21.15.37:0/3301780114 conn(0x7f3e3c0f8970 msgr2=0x7f3e3c0fad80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:50:01.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:01.209+0000 7f3e41400700 1 -- 172.21.15.37:0/3301780114 shutdown_connections 2024-03-29T08:50:01.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:01.209+0000 7f3e41400700 1 -- 172.21.15.37:0/3301780114 wait complete. 2024-03-29T08:50:01.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:01.210+0000 7f3e41400700 1 Processor -- start 2024-03-29T08:50:01.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:01.210+0000 7f3e41400700 1 -- start start 2024-03-29T08:50:01.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:01.210+0000 7f3e41400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3e3c070300 0x7f3e3c115c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:50:01.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:01.210+0000 7f3e41400700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3e3c070c10 0x7f3e3c1161d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:50:01.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:01.211+0000 7f3e41400700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e3c116820 con 0x7f3e3c070300 2024-03-29T08:50:01.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:01.211+0000 7f3e41400700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e3c10fc30 con 0x7f3e3c070c10 2024-03-29T08:50:01.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:01.211+0000 7f3e3a800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3e3c070300 0x7f3e3c115c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:50:01.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:01.211+0000 7f3e3a800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3e3c070300 0x7f3e3c115c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54182/0 (socket says 172.21.15.37:54182) 2024-03-29T08:50:01.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:01.211+0000 7f3e3a800700 1 -- 172.21.15.37:0/2907566926 learned_addr learned my addr 172.21.15.37:0/2907566926 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:50:01.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:01.211+0000 7f3e39e00700 1 --2- 172.21.15.37:0/2907566926 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3e3c070c10 0x7f3e3c1161d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:50:01.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:01.211+0000 7f3e3a800700 1 -- 172.21.15.37:0/2907566926 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3e3c070c10 msgr2=0x7f3e3c1161d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:50:01.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:01.211+0000 7f3e3a800700 1 --2- 172.21.15.37:0/2907566926 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3e3c070c10 0x7f3e3c1161d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:01.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:01.211+0000 7f3e3a800700 1 -- 172.21.15.37:0/2907566926 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e2c005040 con 0x7f3e3c070300 2024-03-29T08:50:01.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:01.211+0000 7f3e39e00700 1 --2- 172.21.15.37:0/2907566926 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3e3c070c10 0x7f3e3c1161d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:50:01.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:01.211+0000 7f3e3a800700 1 --2- 172.21.15.37:0/2907566926 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3e3c070300 0x7f3e3c115c90 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f3e2c004f50 tx=0x7f3e2c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:50:01.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:01.212+0000 7f3e32a00700 1 -- 172.21.15.37:0/2907566926 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3e2c00e040 con 0x7f3e3c070300 2024-03-29T08:50:01.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:01.212+0000 7f3e32a00700 1 -- 172.21.15.37:0/2907566926 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f3e2c013420 con 0x7f3e3c070300 2024-03-29T08:50:01.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:01.212+0000 7f3e32a00700 1 -- 172.21.15.37:0/2907566926 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3e2c01c610 con 0x7f3e3c070300 2024-03-29T08:50:01.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:01.212+0000 7f3e41400700 1 -- 172.21.15.37:0/2907566926 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e3c10feb0 con 0x7f3e3c070300 2024-03-29T08:50:01.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:01.212+0000 7f3e41400700 1 -- 172.21.15.37:0/2907566926 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e3c1103f0 con 0x7f3e3c070300 2024-03-29T08:50:01.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:01.213+0000 7f3e41400700 1 -- 172.21.15.37:0/2907566926 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e3c071a40 con 0x7f3e3c070300 2024-03-29T08:50:01.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:01.214+0000 7f3e32a00700 1 -- 172.21.15.37:0/2907566926 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f3e2c013590 con 0x7f3e3c070300 2024-03-29T08:50:01.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:01.215+0000 7f3e32a00700 1 --2- 172.21.15.37:0/2907566926 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f3e2006cc30 0x7f3e2006f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:50:01.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:01.216+0000 7f3e32a00700 1 -- 172.21.15.37:0/2907566926 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f3e2c08d260 con 0x7f3e3c070300 2024-03-29T08:50:01.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:01.216+0000 7f3e39e00700 1 --2- 172.21.15.37:0/2907566926 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f3e2006cc30 0x7f3e2006f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:50:01.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:01.216+0000 7f3e39e00700 1 --2- 172.21.15.37:0/2907566926 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f3e2006cc30 0x7f3e2006f0e0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f3e28009fd0 tx=0x7f3e2800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:50:01.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:01.220+0000 7f3e32a00700 1 -- 172.21.15.37:0/2907566926 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f3e2c057220 con 0x7f3e3c070300 2024-03-29T08:50:01.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:01.444+0000 7f3e41400700 1 -- 172.21.15.37:0/2907566926 --> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f3e3c04bf20 con 0x7f3e2006cc30 2024-03-29T08:50:01.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:01.455+0000 7f3e32a00700 1 -- 172.21.15.37:0/2907566926 <== mgr.34104 v2:172.21.15.37:6800/794131733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f3e3c04bf20 con 0x7f3e2006cc30 2024-03-29T08:50:01.454 INFO:teuthology.orchestra.run.smithi037.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-29T08:50:01.454 INFO:teuthology.orchestra.run.smithi037.stdout:alertmanager.smithi037 smithi037 *:9093,9094 running (39m) 91s ago 52m 14.5M - 0.25.0 c8568f914cd2 2f9f9746ed0f 2024-03-29T08:50:01.455 INFO:teuthology.orchestra.run.smithi037.stdout:crash.smithi037 smithi037 running (36m) 91s ago 53m 6647k - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 178d416a3186 2024-03-29T08:50:01.455 INFO:teuthology.orchestra.run.smithi037.stdout:crash.smithi148 smithi148 running (36m) 95s ago 51m 6656k - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 23666989d58f 2024-03-29T08:50:01.455 INFO:teuthology.orchestra.run.smithi037.stdout:grafana.smithi037 smithi037 *:3000 running (39m) 91s ago 52m 73.3M - 9.4.7 954c08fa6188 6b976fe49952 2024-03-29T08:50:01.455 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi037.iifsmh smithi037 running (32m) 91s ago 46m 96.6M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 2d97f562cc5b 2024-03-29T08:50:01.455 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi037.mmwgxe smithi037 running (32m) 91s ago 46m 86.5M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff ec37037f46e8 2024-03-29T08:50:01.455 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi148.qukpjh smithi148 running (16m) 95s ago 46m 16.4M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 3257219bc046 2024-03-29T08:50:01.455 INFO:teuthology.orchestra.run.smithi037.stdout:mds.cephfs.smithi148.sqteih smithi148 running (31m) 95s ago 46m 21.5M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff 44c15c77037a 2024-03-29T08:50:01.455 INFO:teuthology.orchestra.run.smithi037.stdout:mgr.smithi037.wjikii smithi037 *:8443,9283,8765 running (42m) 91s ago 54m 513M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff c5a1417b9417 2024-03-29T08:50:01.455 INFO:teuthology.orchestra.run.smithi037.stdout:mgr.smithi148.onhgvm smithi148 *:8443,9283,8765 running (41m) 95s ago 51m 457M - 18.2.2-1015-gd0e392d9 1cc2f45c17ff fcfdbce062bd 2024-03-29T08:50:01.455 INFO:teuthology.orchestra.run.smithi037.stdout:mon.smithi037 smithi037 running (38m) 91s ago 54m 85.2M 2048M 18.2.2-1015-gd0e392d9 1cc2f45c17ff eea2060783af 2024-03-29T08:50:01.455 INFO:teuthology.orchestra.run.smithi037.stdout:mon.smithi148 smithi148 running (37m) 95s ago 51m 56.8M 2048M 18.2.2-1015-gd0e392d9 1cc2f45c17ff ad4b2c5748be 2024-03-29T08:50:01.455 INFO:teuthology.orchestra.run.smithi037.stdout:node-exporter.smithi037 smithi037 *:9100 running (40m) 91s ago 53m 8280k - 1.5.0 0da6a335fe13 26b8ae9cb878 2024-03-29T08:50:01.455 INFO:teuthology.orchestra.run.smithi037.stdout:node-exporter.smithi148 smithi148 *:9100 running (40m) 95s ago 51m 8753k - 1.5.0 0da6a335fe13 98414e4e64ed 2024-03-29T08:50:01.455 INFO:teuthology.orchestra.run.smithi037.stdout:osd.0 smithi037 running (36m) 91s ago 50m 112M 1619M 18.2.2-1015-gd0e392d9 1cc2f45c17ff c0c584f6245c 2024-03-29T08:50:01.456 INFO:teuthology.orchestra.run.smithi037.stdout:osd.1 smithi037 running (35m) 91s ago 50m 109M 1619M 18.2.2-1015-gd0e392d9 1cc2f45c17ff a4a2ff4cc83d 2024-03-29T08:50:01.456 INFO:teuthology.orchestra.run.smithi037.stdout:osd.2 smithi037 running (34m) 91s ago 49m 88.0M 1619M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 8b1bc71aa79e 2024-03-29T08:50:01.456 INFO:teuthology.orchestra.run.smithi037.stdout:osd.3 smithi148 running (34m) 95s ago 49m 114M 2642M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 1c656ab8ff86 2024-03-29T08:50:01.456 INFO:teuthology.orchestra.run.smithi037.stdout:osd.4 smithi148 running (33m) 95s ago 48m 103M 2642M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 966a9de11d22 2024-03-29T08:50:01.456 INFO:teuthology.orchestra.run.smithi037.stdout:osd.5 smithi148 running (32m) 95s ago 48m 95.6M 2642M 18.2.2-1015-gd0e392d9 1cc2f45c17ff 474c4f9212ce 2024-03-29T08:50:01.456 INFO:teuthology.orchestra.run.smithi037.stdout:prometheus.smithi037 smithi037 *:9095 running (40m) 91s ago 52m 68.6M - 2.43.0 a07b618ecd1d 30c0223ec589 2024-03-29T08:50:01.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:01.457+0000 7f3e41400700 1 -- 172.21.15.37:0/2907566926 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f3e2006cc30 msgr2=0x7f3e2006f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:50:01.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:01.457+0000 7f3e41400700 1 --2- 172.21.15.37:0/2907566926 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f3e2006cc30 0x7f3e2006f0e0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f3e28009fd0 tx=0x7f3e2800b040 comp rx=0 tx=0).stop 2024-03-29T08:50:01.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:01.457+0000 7f3e41400700 1 -- 172.21.15.37:0/2907566926 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3e3c070300 msgr2=0x7f3e3c115c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:50:01.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:01.457+0000 7f3e41400700 1 --2- 172.21.15.37:0/2907566926 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3e3c070300 0x7f3e3c115c90 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f3e2c004f50 tx=0x7f3e2c009e30 comp rx=0 tx=0).stop 2024-03-29T08:50:01.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:01.457+0000 7f3e41400700 1 -- 172.21.15.37:0/2907566926 shutdown_connections 2024-03-29T08:50:01.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:01.457+0000 7f3e41400700 1 --2- 172.21.15.37:0/2907566926 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f3e2006cc30 0x7f3e2006f0e0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:01.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:01.457+0000 7f3e41400700 1 --2- 172.21.15.37:0/2907566926 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3e3c070300 0x7f3e3c115c90 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:01.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:01.457+0000 7f3e41400700 1 --2- 172.21.15.37:0/2907566926 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3e3c070c10 0x7f3e3c1161d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:01.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:01.457+0000 7f3e41400700 1 -- 172.21.15.37:0/2907566926 >> 172.21.15.37:0/2907566926 conn(0x7f3e3c0f8970 msgr2=0x7f3e3c0f9ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:50:01.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:01.457+0000 7f3e41400700 1 -- 172.21.15.37:0/2907566926 shutdown_connections 2024-03-29T08:50:01.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:01.457+0000 7f3e41400700 1 -- 172.21.15.37:0/2907566926 wait complete. 2024-03-29T08:50:02.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:50:01 smithi148 ceph-mon[106692]: pgmap v1176: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:50:02.079 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:50:01 smithi037 ceph-mon[142325]: pgmap v1176: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:50:02.105 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 99b492c2-eda1-11ee-b647-cb9ed24678a4 -e sha1=d0e392d9aee635f2b65768c460c7a151ead7a5e2 -- bash -c 'ceph versions' 2024-03-29T08:50:02.542 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/mon.smithi037/config 2024-03-29T08:50:03.044 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:50:02 smithi037 ceph-mon[142325]: from='client.36038 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:50:03.055 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:50:02 smithi148 ceph-mon[106692]: from='client.36038 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-29T08:50:04.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:50:03 smithi148 ceph-mon[106692]: pgmap v1177: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:50:04.176 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:50:03 smithi037 ceph-mon[142325]: pgmap v1177: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:50:04.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:04.531+0000 7f01be000700 1 -- 172.21.15.37:0/3553253773 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f01b8102e00 msgr2=0x7f01b81031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:50:04.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:04.531+0000 7f01be000700 1 --2- 172.21.15.37:0/3553253773 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f01b8102e00 0x7f01b81031d0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f01a8004940 tx=0x7f01a8009e30 comp rx=0 tx=0).stop 2024-03-29T08:50:04.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:04.531+0000 7f01be000700 1 -- 172.21.15.37:0/3553253773 shutdown_connections 2024-03-29T08:50:04.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:04.531+0000 7f01be000700 1 --2- 172.21.15.37:0/3553253773 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f01b81037a0 0x7f01b80fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:04.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:04.531+0000 7f01be000700 1 --2- 172.21.15.37:0/3553253773 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f01b8102e00 0x7f01b81031d0 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:04.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:04.531+0000 7f01be000700 1 -- 172.21.15.37:0/3553253773 >> 172.21.15.37:0/3553253773 conn(0x7f01b80f89b0 msgr2=0x7f01b80fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:50:04.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:04.532+0000 7f01be000700 1 -- 172.21.15.37:0/3553253773 shutdown_connections 2024-03-29T08:50:04.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:04.532+0000 7f01be000700 1 -- 172.21.15.37:0/3553253773 wait complete. 2024-03-29T08:50:04.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:04.532+0000 7f01be000700 1 Processor -- start 2024-03-29T08:50:04.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:04.533+0000 7f01be000700 1 -- start start 2024-03-29T08:50:04.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:04.533+0000 7f01be000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f01b8102e00 0x7f01b8115cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:50:04.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:04.533+0000 7f01be000700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f01b81037a0 0x7f01b8116230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:50:04.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:04.533+0000 7f01be000700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01b8116880 con 0x7f01b8102e00 2024-03-29T08:50:04.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:04.533+0000 7f01be000700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01b810fc50 con 0x7f01b81037a0 2024-03-29T08:50:04.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:04.533+0000 7f01b7400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f01b8102e00 0x7f01b8115cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:50:04.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:04.533+0000 7f01b7400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f01b8102e00 0x7f01b8115cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54202/0 (socket says 172.21.15.37:54202) 2024-03-29T08:50:04.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:04.533+0000 7f01b7400700 1 -- 172.21.15.37:0/4009943233 learned_addr learned my addr 172.21.15.37:0/4009943233 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:50:04.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:04.534+0000 7f01b6a00700 1 --2- 172.21.15.37:0/4009943233 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f01b81037a0 0x7f01b8116230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:50:04.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:04.534+0000 7f01b7400700 1 -- 172.21.15.37:0/4009943233 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f01b81037a0 msgr2=0x7f01b8116230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:50:04.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:04.534+0000 7f01b7400700 1 --2- 172.21.15.37:0/4009943233 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f01b81037a0 0x7f01b8116230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:04.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:04.534+0000 7f01b7400700 1 -- 172.21.15.37:0/4009943233 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01a8005040 con 0x7f01b8102e00 2024-03-29T08:50:04.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:04.534+0000 7f01b6a00700 1 --2- 172.21.15.37:0/4009943233 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f01b81037a0 0x7f01b8116230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:50:04.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:04.534+0000 7f01b7400700 1 --2- 172.21.15.37:0/4009943233 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f01b8102e00 0x7f01b8115cf0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f01a8009390 tx=0x7f01a8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:50:04.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:04.534+0000 7f01af400700 1 -- 172.21.15.37:0/4009943233 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f01a800e040 con 0x7f01b8102e00 2024-03-29T08:50:04.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:04.534+0000 7f01af400700 1 -- 172.21.15.37:0/4009943233 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f01a8013420 con 0x7f01b8102e00 2024-03-29T08:50:04.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:04.534+0000 7f01be000700 1 -- 172.21.15.37:0/4009943233 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01b810fed0 con 0x7f01b8102e00 2024-03-29T08:50:04.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:04.534+0000 7f01af400700 1 -- 172.21.15.37:0/4009943233 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f01a801c630 con 0x7f01b8102e00 2024-03-29T08:50:04.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:04.535+0000 7f01be000700 1 -- 172.21.15.37:0/4009943233 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01b8110490 con 0x7f01b8102e00 2024-03-29T08:50:04.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:04.535+0000 7f01be000700 1 -- 172.21.15.37:0/4009943233 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01b80fe550 con 0x7f01b8102e00 2024-03-29T08:50:04.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:04.537+0000 7f01af400700 1 -- 172.21.15.37:0/4009943233 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f01a80139e0 con 0x7f01b8102e00 2024-03-29T08:50:04.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:04.537+0000 7f01af400700 1 --2- 172.21.15.37:0/4009943233 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f019c06cc30 0x7f019c06f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:50:04.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:04.537+0000 7f01af400700 1 -- 172.21.15.37:0/4009943233 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f01a808d320 con 0x7f01b8102e00 2024-03-29T08:50:04.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:04.538+0000 7f01b6a00700 1 --2- 172.21.15.37:0/4009943233 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f019c06cc30 0x7f019c06f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:50:04.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:04.538+0000 7f01b6a00700 1 --2- 172.21.15.37:0/4009943233 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f019c06cc30 0x7f019c06f0e0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f01a4007fd0 tx=0x7f01a4009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:50:04.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:04.542+0000 7f01af400700 1 -- 172.21.15.37:0/4009943233 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f01a805ae20 con 0x7f01b8102e00 2024-03-29T08:50:04.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:04.841+0000 7f01be000700 1 -- 172.21.15.37:0/4009943233 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f01b804bed0 con 0x7f01b8102e00 2024-03-29T08:50:04.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:04.842+0000 7f01af400700 1 -- 172.21.15.37:0/4009943233 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7f01a805a9b0 con 0x7f01b8102e00 2024-03-29T08:50:04.841 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-03-29T08:50:04.842 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": { 2024-03-29T08:50:04.842 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 2 2024-03-29T08:50:04.842 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:50:04.842 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": { 2024-03-29T08:50:04.842 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 2 2024-03-29T08:50:04.842 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:50:04.842 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": { 2024-03-29T08:50:04.842 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 6 2024-03-29T08:50:04.842 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:50:04.842 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": { 2024-03-29T08:50:04.842 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 4 2024-03-29T08:50:04.842 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-03-29T08:50:04.842 INFO:teuthology.orchestra.run.smithi037.stdout: "overall": { 2024-03-29T08:50:04.842 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)": 14 2024-03-29T08:50:04.843 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-03-29T08:50:04.843 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-03-29T08:50:04.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:04.845+0000 7f01be000700 1 -- 172.21.15.37:0/4009943233 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f019c06cc30 msgr2=0x7f019c06f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:50:04.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:04.845+0000 7f01be000700 1 --2- 172.21.15.37:0/4009943233 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f019c06cc30 0x7f019c06f0e0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f01a4007fd0 tx=0x7f01a4009040 comp rx=0 tx=0).stop 2024-03-29T08:50:04.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:04.845+0000 7f01be000700 1 -- 172.21.15.37:0/4009943233 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f01b8102e00 msgr2=0x7f01b8115cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:50:04.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:04.845+0000 7f01be000700 1 --2- 172.21.15.37:0/4009943233 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f01b8102e00 0x7f01b8115cf0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f01a8009390 tx=0x7f01a8009e30 comp rx=0 tx=0).stop 2024-03-29T08:50:04.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:04.845+0000 7f01be000700 1 -- 172.21.15.37:0/4009943233 shutdown_connections 2024-03-29T08:50:04.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:04.845+0000 7f01be000700 1 --2- 172.21.15.37:0/4009943233 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f019c06cc30 0x7f019c06f0e0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:04.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:04.845+0000 7f01be000700 1 --2- 172.21.15.37:0/4009943233 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f01b8102e00 0x7f01b8115cf0 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:04.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:04.845+0000 7f01be000700 1 --2- 172.21.15.37:0/4009943233 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f01b81037a0 0x7f01b8116230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:04.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:04.845+0000 7f01be000700 1 -- 172.21.15.37:0/4009943233 >> 172.21.15.37:0/4009943233 conn(0x7f01b80f89b0 msgr2=0x7f01b80706d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:50:04.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:04.845+0000 7f01be000700 1 -- 172.21.15.37:0/4009943233 shutdown_connections 2024-03-29T08:50:04.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:04.845+0000 7f01be000700 1 -- 172.21.15.37:0/4009943233 wait complete. 2024-03-29T08:50:05.549 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 99b492c2-eda1-11ee-b647-cb9ed24678a4 -e sha1=d0e392d9aee635f2b65768c460c7a151ead7a5e2 -- bash -c 'ceph versions | jq -e '"'"'.overall | length == 1'"'"'' 2024-03-29T08:50:05.987 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/mon.smithi037/config 2024-03-29T08:50:06.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:50:05 smithi148 ceph-mon[106692]: pgmap v1178: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:50:06.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:50:05 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/4009943233' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:50:06.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:50:05 smithi037 ceph-mon[142325]: pgmap v1178: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:50:06.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:50:05 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/4009943233' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:50:07.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:07.909+0000 7f6b64600700 1 -- 172.21.15.37:0/3188421920 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6b5c102e00 msgr2=0x7f6b5c1031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:50:07.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:07.909+0000 7f6b64600700 1 --2- 172.21.15.37:0/3188421920 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6b5c102e00 0x7f6b5c1031d0 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f6b50004940 tx=0x7f6b50009e30 comp rx=0 tx=0).stop 2024-03-29T08:50:07.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:07.909+0000 7f6b64600700 1 -- 172.21.15.37:0/3188421920 shutdown_connections 2024-03-29T08:50:07.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:07.909+0000 7f6b64600700 1 --2- 172.21.15.37:0/3188421920 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6b5c1037a0 0x7f6b5c0fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:07.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:07.909+0000 7f6b64600700 1 --2- 172.21.15.37:0/3188421920 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6b5c102e00 0x7f6b5c1031d0 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:07.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:07.909+0000 7f6b64600700 1 -- 172.21.15.37:0/3188421920 >> 172.21.15.37:0/3188421920 conn(0x7f6b5c0f89b0 msgr2=0x7f6b5c0fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:50:07.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:07.910+0000 7f6b64600700 1 -- 172.21.15.37:0/3188421920 shutdown_connections 2024-03-29T08:50:07.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:07.910+0000 7f6b64600700 1 -- 172.21.15.37:0/3188421920 wait complete. 2024-03-29T08:50:07.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:07.910+0000 7f6b64600700 1 Processor -- start 2024-03-29T08:50:07.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:07.910+0000 7f6b64600700 1 -- start start 2024-03-29T08:50:07.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:07.911+0000 7f6b64600700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6b5c102e00 0x7f6b5c115cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:50:07.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:07.911+0000 7f6b64600700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6b5c1037a0 0x7f6b5c116230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:50:07.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:07.911+0000 7f6b64600700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b5c116880 con 0x7f6b5c1037a0 2024-03-29T08:50:07.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:07.911+0000 7f6b64600700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b5c10fc50 con 0x7f6b5c102e00 2024-03-29T08:50:07.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:07.911+0000 7f6b61200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6b5c1037a0 0x7f6b5c116230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:50:07.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:07.911+0000 7f6b61200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6b5c1037a0 0x7f6b5c116230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54212/0 (socket says 172.21.15.37:54212) 2024-03-29T08:50:07.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:07.911+0000 7f6b61200700 1 -- 172.21.15.37:0/3911054407 learned_addr learned my addr 172.21.15.37:0/3911054407 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:50:07.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:07.911+0000 7f6b61c00700 1 --2- 172.21.15.37:0/3911054407 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6b5c102e00 0x7f6b5c115cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:50:07.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:07.912+0000 7f6b61200700 1 -- 172.21.15.37:0/3911054407 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6b5c102e00 msgr2=0x7f6b5c115cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:50:07.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:07.912+0000 7f6b61200700 1 --2- 172.21.15.37:0/3911054407 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6b5c102e00 0x7f6b5c115cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:07.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:07.912+0000 7f6b61200700 1 -- 172.21.15.37:0/3911054407 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b50005040 con 0x7f6b5c1037a0 2024-03-29T08:50:07.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:07.912+0000 7f6b61200700 1 --2- 172.21.15.37:0/3911054407 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6b5c1037a0 0x7f6b5c116230 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f6b4800c890 tx=0x7f6b480044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:50:07.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:07.912+0000 7f6b59600700 1 -- 172.21.15.37:0/3911054407 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6b4800a700 con 0x7f6b5c1037a0 2024-03-29T08:50:07.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:07.912+0000 7f6b59600700 1 -- 172.21.15.37:0/3911054407 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f6b4800ad10 con 0x7f6b5c1037a0 2024-03-29T08:50:07.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:07.912+0000 7f6b64600700 1 -- 172.21.15.37:0/3911054407 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b5c10ff30 con 0x7f6b5c1037a0 2024-03-29T08:50:07.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:07.912+0000 7f6b59600700 1 -- 172.21.15.37:0/3911054407 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6b48012e70 con 0x7f6b5c1037a0 2024-03-29T08:50:07.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:07.912+0000 7f6b64600700 1 -- 172.21.15.37:0/3911054407 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b5c1104f0 con 0x7f6b5c1037a0 2024-03-29T08:50:07.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:07.913+0000 7f6b64600700 1 -- 172.21.15.37:0/3911054407 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b5c0fe550 con 0x7f6b5c1037a0 2024-03-29T08:50:07.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:07.914+0000 7f6b59600700 1 -- 172.21.15.37:0/3911054407 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f6b4800a860 con 0x7f6b5c1037a0 2024-03-29T08:50:07.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:07.916+0000 7f6b59600700 1 --2- 172.21.15.37:0/3911054407 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f6b4406cc30 0x7f6b4406f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:50:07.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:07.916+0000 7f6b59600700 1 -- 172.21.15.37:0/3911054407 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f6b4808c120 con 0x7f6b5c1037a0 2024-03-29T08:50:07.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:07.916+0000 7f6b61c00700 1 --2- 172.21.15.37:0/3911054407 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f6b4406cc30 0x7f6b4406f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:50:07.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:07.917+0000 7f6b61c00700 1 --2- 172.21.15.37:0/3911054407 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f6b4406cc30 0x7f6b4406f0e0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f6b50009fd0 tx=0x7f6b50009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:50:07.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:07.920+0000 7f6b59600700 1 -- 172.21.15.37:0/3911054407 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f6b48059b70 con 0x7f6b5c1037a0 2024-03-29T08:50:07.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:50:07 smithi037 ceph-mon[142325]: pgmap v1179: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:50:08.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:50:07 smithi148 ceph-mon[106692]: pgmap v1179: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:50:08.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:08.220+0000 7f6b64600700 1 -- 172.21.15.37:0/3911054407 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f6b5c04bed0 con 0x7f6b5c1037a0 2024-03-29T08:50:08.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:08.220+0000 7f6b59600700 1 -- 172.21.15.37:0/3911054407 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7f6b48059700 con 0x7f6b5c1037a0 2024-03-29T08:50:08.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:08.223+0000 7f6b64600700 1 -- 172.21.15.37:0/3911054407 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f6b4406cc30 msgr2=0x7f6b4406f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:50:08.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:08.223+0000 7f6b64600700 1 --2- 172.21.15.37:0/3911054407 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f6b4406cc30 0x7f6b4406f0e0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f6b50009fd0 tx=0x7f6b50009e30 comp rx=0 tx=0).stop 2024-03-29T08:50:08.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:08.223+0000 7f6b64600700 1 -- 172.21.15.37:0/3911054407 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6b5c1037a0 msgr2=0x7f6b5c116230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:50:08.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:08.223+0000 7f6b64600700 1 --2- 172.21.15.37:0/3911054407 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6b5c1037a0 0x7f6b5c116230 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f6b4800c890 tx=0x7f6b480044c0 comp rx=0 tx=0).stop 2024-03-29T08:50:08.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:08.223+0000 7f6b64600700 1 -- 172.21.15.37:0/3911054407 shutdown_connections 2024-03-29T08:50:08.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:08.223+0000 7f6b64600700 1 --2- 172.21.15.37:0/3911054407 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f6b4406cc30 0x7f6b4406f0e0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:08.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:08.223+0000 7f6b64600700 1 --2- 172.21.15.37:0/3911054407 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6b5c102e00 0x7f6b5c115cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:08.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:08.223+0000 7f6b64600700 1 --2- 172.21.15.37:0/3911054407 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6b5c1037a0 0x7f6b5c116230 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:08.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:08.223+0000 7f6b64600700 1 -- 172.21.15.37:0/3911054407 >> 172.21.15.37:0/3911054407 conn(0x7f6b5c0f89b0 msgr2=0x7f6b5c0706d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:50:08.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:08.223+0000 7f6b64600700 1 -- 172.21.15.37:0/3911054407 shutdown_connections 2024-03-29T08:50:08.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:08.223+0000 7f6b64600700 1 -- 172.21.15.37:0/3911054407 wait complete. 2024-03-29T08:50:08.237 INFO:teuthology.orchestra.run.smithi037.stdout:true 2024-03-29T08:50:08.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:50:08 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/3911054407' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:50:08.956 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 99b492c2-eda1-11ee-b647-cb9ed24678a4 -e sha1=d0e392d9aee635f2b65768c460c7a151ead7a5e2 -- bash -c 'ceph versions | jq -e '"'"'.overall | keys'"'"' | grep $sha1' 2024-03-29T08:50:09.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:50:08 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/3911054407' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:50:09.396 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/mon.smithi037/config 2024-03-29T08:50:09.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:50:09 smithi037 ceph-mon[142325]: pgmap v1180: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:50:09.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:50:09 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:50:10.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:50:09 smithi148 ceph-mon[106692]: pgmap v1180: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:50:10.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:50:09 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:50:11.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:11.328+0000 7ff306600700 1 -- 172.21.15.37:0/2588549014 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff3000fde90 msgr2=0x7ff3000fe260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:50:11.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:11.328+0000 7ff306600700 1 --2- 172.21.15.37:0/2588549014 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff3000fde90 0x7ff3000fe260 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7ff2f0004940 tx=0x7ff2f0009e30 comp rx=0 tx=0).stop 2024-03-29T08:50:11.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:11.328+0000 7ff306600700 1 -- 172.21.15.37:0/2588549014 shutdown_connections 2024-03-29T08:50:11.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:11.328+0000 7ff306600700 1 --2- 172.21.15.37:0/2588549014 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff3000fe830 0x7ff300106f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:11.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:11.328+0000 7ff306600700 1 --2- 172.21.15.37:0/2588549014 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff3000fde90 0x7ff3000fe260 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:11.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:11.328+0000 7ff306600700 1 -- 172.21.15.37:0/2588549014 >> 172.21.15.37:0/2588549014 conn(0x7ff300072bc0 msgr2=0x7ff300072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:50:11.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:11.328+0000 7ff306600700 1 -- 172.21.15.37:0/2588549014 shutdown_connections 2024-03-29T08:50:11.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:11.328+0000 7ff306600700 1 -- 172.21.15.37:0/2588549014 wait complete. 2024-03-29T08:50:11.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:11.329+0000 7ff306600700 1 Processor -- start 2024-03-29T08:50:11.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:11.329+0000 7ff306600700 1 -- start start 2024-03-29T08:50:11.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:11.329+0000 7ff306600700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff3000fde90 0x7ff3000fbd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:50:11.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:11.329+0000 7ff306600700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff3000fe830 0x7ff3000fc290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:50:11.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:11.329+0000 7ff306600700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3000fc820 con 0x7ff3000fe830 2024-03-29T08:50:11.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:11.329+0000 7ff306600700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3000fc960 con 0x7ff3000fde90 2024-03-29T08:50:11.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:11.330+0000 7ff2fea00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff3000fe830 0x7ff3000fc290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:50:11.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:11.330+0000 7ff2fea00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff3000fe830 0x7ff3000fc290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60168/0 (socket says 172.21.15.37:60168) 2024-03-29T08:50:11.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:11.330+0000 7ff2fea00700 1 -- 172.21.15.37:0/3021564788 learned_addr learned my addr 172.21.15.37:0/3021564788 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:50:11.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:11.330+0000 7ff2ff400700 1 --2- 172.21.15.37:0/3021564788 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff3000fde90 0x7ff3000fbd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:50:11.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:11.330+0000 7ff2fea00700 1 -- 172.21.15.37:0/3021564788 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff3000fde90 msgr2=0x7ff3000fbd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:50:11.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:11.330+0000 7ff2fea00700 1 --2- 172.21.15.37:0/3021564788 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff3000fde90 0x7ff3000fbd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:11.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:11.330+0000 7ff2fea00700 1 -- 172.21.15.37:0/3021564788 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2f0005040 con 0x7ff3000fe830 2024-03-29T08:50:11.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:11.330+0000 7ff2fea00700 1 --2- 172.21.15.37:0/3021564788 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff3000fe830 0x7ff3000fc290 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7ff2ec002a50 tx=0x7ff2ec0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:50:11.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:11.331+0000 7ff2f7400700 1 -- 172.21.15.37:0/3021564788 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff2ec007cc0 con 0x7ff3000fe830 2024-03-29T08:50:11.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:11.331+0000 7ff2f7400700 1 -- 172.21.15.37:0/3021564788 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ff2ec00f040 con 0x7ff3000fe830 2024-03-29T08:50:11.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:11.331+0000 7ff306600700 1 -- 172.21.15.37:0/3021564788 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff300119df0 con 0x7ff3000fe830 2024-03-29T08:50:11.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:11.331+0000 7ff2f7400700 1 -- 172.21.15.37:0/3021564788 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff2ec00d9c0 con 0x7ff3000fe830 2024-03-29T08:50:11.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:11.331+0000 7ff306600700 1 -- 172.21.15.37:0/3021564788 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff30011a1f0 con 0x7ff3000fe830 2024-03-29T08:50:11.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:11.332+0000 7ff306600700 1 -- 172.21.15.37:0/3021564788 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff300065e30 con 0x7ff3000fe830 2024-03-29T08:50:11.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:11.333+0000 7ff2f7400700 1 -- 172.21.15.37:0/3021564788 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7ff2ec00db20 con 0x7ff3000fe830 2024-03-29T08:50:11.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:11.334+0000 7ff2f7400700 1 --2- 172.21.15.37:0/3021564788 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7ff2e406cc30 0x7ff2e406f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:50:11.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:11.334+0000 7ff2f7400700 1 -- 172.21.15.37:0/3021564788 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7ff2ec08be20 con 0x7ff3000fe830 2024-03-29T08:50:11.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:11.334+0000 7ff2ff400700 1 --2- 172.21.15.37:0/3021564788 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7ff2e406cc30 0x7ff2e406f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:50:11.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:11.335+0000 7ff2ff400700 1 --2- 172.21.15.37:0/3021564788 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7ff2e406cc30 0x7ff2e406f0e0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7ff2f000b3b0 tx=0x7ff2f0009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:50:11.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:11.339+0000 7ff2f7400700 1 -- 172.21.15.37:0/3021564788 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7ff2ec055d60 con 0x7ff3000fe830 2024-03-29T08:50:11.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:11.645+0000 7ff306600700 1 -- 172.21.15.37:0/3021564788 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7ff30004bed0 con 0x7ff3000fe830 2024-03-29T08:50:11.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:11.645+0000 7ff2f7400700 1 -- 172.21.15.37:0/3021564788 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7ff2ec059380 con 0x7ff3000fe830 2024-03-29T08:50:11.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:11.648+0000 7ff306600700 1 -- 172.21.15.37:0/3021564788 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7ff2e406cc30 msgr2=0x7ff2e406f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:50:11.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:11.648+0000 7ff306600700 1 --2- 172.21.15.37:0/3021564788 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7ff2e406cc30 0x7ff2e406f0e0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7ff2f000b3b0 tx=0x7ff2f0009e30 comp rx=0 tx=0).stop 2024-03-29T08:50:11.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:11.648+0000 7ff306600700 1 -- 172.21.15.37:0/3021564788 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff3000fe830 msgr2=0x7ff3000fc290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:50:11.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:11.648+0000 7ff306600700 1 --2- 172.21.15.37:0/3021564788 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff3000fe830 0x7ff3000fc290 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7ff2ec002a50 tx=0x7ff2ec0046b0 comp rx=0 tx=0).stop 2024-03-29T08:50:11.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:11.648+0000 7ff306600700 1 -- 172.21.15.37:0/3021564788 shutdown_connections 2024-03-29T08:50:11.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:11.648+0000 7ff306600700 1 --2- 172.21.15.37:0/3021564788 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7ff2e406cc30 0x7ff2e406f0e0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:11.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:11.648+0000 7ff306600700 1 --2- 172.21.15.37:0/3021564788 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff3000fde90 0x7ff3000fbd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:11.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:11.648+0000 7ff306600700 1 --2- 172.21.15.37:0/3021564788 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff3000fe830 0x7ff3000fc290 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:11.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:11.648+0000 7ff306600700 1 -- 172.21.15.37:0/3021564788 >> 172.21.15.37:0/3021564788 conn(0x7ff300072bc0 msgr2=0x7ff3000ff3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:50:11.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:11.648+0000 7ff306600700 1 -- 172.21.15.37:0/3021564788 shutdown_connections 2024-03-29T08:50:11.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:11.648+0000 7ff306600700 1 -- 172.21.15.37:0/3021564788 wait complete. 2024-03-29T08:50:11.664 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)" 2024-03-29T08:50:11.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:50:11 smithi037 ceph-mon[142325]: pgmap v1181: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:50:11.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:50:11 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/3021564788' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:50:12.305 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:50:11 smithi148 ceph-mon[106692]: pgmap v1181: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:50:12.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:50:11 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/3021564788' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-29T08:50:12.340 DEBUG:teuthology.parallel:result is None 2024-03-29T08:50:12.340 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-29T08:50:12.354 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi037.front.sepia.ceph.com 2024-03-29T08:50:12.354 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 99b492c2-eda1-11ee-b647-cb9ed24678a4 -- bash -c 'ceph fs dump' 2024-03-29T08:50:12.793 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/mon.smithi037/config 2024-03-29T08:50:14.131 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:50:13 smithi037 ceph-mon[142325]: pgmap v1182: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:50:14.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:50:13 smithi148 ceph-mon[106692]: pgmap v1182: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:50:14.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:14.835+0000 7f3068400700 1 -- 172.21.15.37:0/2413200938 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3060102e00 msgr2=0x7f30601031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:50:14.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:14.835+0000 7f3068400700 1 --2- 172.21.15.37:0/2413200938 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3060102e00 0x7f30601031d0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f3054004940 tx=0x7f3054009e30 comp rx=0 tx=0).stop 2024-03-29T08:50:14.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:14.835+0000 7f3068400700 1 -- 172.21.15.37:0/2413200938 shutdown_connections 2024-03-29T08:50:14.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:14.835+0000 7f3068400700 1 --2- 172.21.15.37:0/2413200938 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f30601037a0 0x7f30600fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:14.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:14.835+0000 7f3068400700 1 --2- 172.21.15.37:0/2413200938 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3060102e00 0x7f30601031d0 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:14.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:14.835+0000 7f3068400700 1 -- 172.21.15.37:0/2413200938 >> 172.21.15.37:0/2413200938 conn(0x7f30600f89b0 msgr2=0x7f30600fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:50:14.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:14.835+0000 7f3068400700 1 -- 172.21.15.37:0/2413200938 shutdown_connections 2024-03-29T08:50:14.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:14.835+0000 7f3068400700 1 -- 172.21.15.37:0/2413200938 wait complete. 2024-03-29T08:50:14.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:14.836+0000 7f3068400700 1 Processor -- start 2024-03-29T08:50:14.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:14.837+0000 7f3068400700 1 -- start start 2024-03-29T08:50:14.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:14.837+0000 7f3068400700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3060102e00 0x7f3060113b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:50:14.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:14.837+0000 7f3068400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f30601037a0 0x7f3060114060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:50:14.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:14.837+0000 7f3068400700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30601146b0 con 0x7f30601037a0 2024-03-29T08:50:14.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:14.837+0000 7f3068400700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f306010daa0 con 0x7f3060102e00 2024-03-29T08:50:14.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:14.837+0000 7f3065000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f30601037a0 0x7f3060114060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:50:14.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:14.837+0000 7f3065000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f30601037a0 0x7f3060114060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60192/0 (socket says 172.21.15.37:60192) 2024-03-29T08:50:14.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:14.838+0000 7f3065000700 1 -- 172.21.15.37:0/3435918286 learned_addr learned my addr 172.21.15.37:0/3435918286 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:50:14.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:14.838+0000 7f3065a00700 1 --2- 172.21.15.37:0/3435918286 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3060102e00 0x7f3060113b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:50:14.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:14.838+0000 7f3065000700 1 -- 172.21.15.37:0/3435918286 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3060102e00 msgr2=0x7f3060113b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:50:14.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:14.838+0000 7f3065000700 1 --2- 172.21.15.37:0/3435918286 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3060102e00 0x7f3060113b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:14.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:14.838+0000 7f3065000700 1 -- 172.21.15.37:0/3435918286 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3054005040 con 0x7f30601037a0 2024-03-29T08:50:14.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:14.838+0000 7f3065a00700 1 --2- 172.21.15.37:0/3435918286 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3060102e00 0x7f3060113b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:50:14.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:14.838+0000 7f3065000700 1 --2- 172.21.15.37:0/3435918286 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f30601037a0 0x7f3060114060 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f304c002a50 tx=0x7f304c004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:50:14.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:14.838+0000 7f305d600700 1 -- 172.21.15.37:0/3435918286 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f304c007cc0 con 0x7f30601037a0 2024-03-29T08:50:14.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:14.839+0000 7f3068400700 1 -- 172.21.15.37:0/3435918286 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f306010dd80 con 0x7f30601037a0 2024-03-29T08:50:14.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:14.839+0000 7f305d600700 1 -- 172.21.15.37:0/3435918286 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f304c00f040 con 0x7f30601037a0 2024-03-29T08:50:14.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:14.839+0000 7f305d600700 1 -- 172.21.15.37:0/3435918286 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f304c00d990 con 0x7f30601037a0 2024-03-29T08:50:14.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:14.839+0000 7f3068400700 1 -- 172.21.15.37:0/3435918286 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f306010e340 con 0x7f30601037a0 2024-03-29T08:50:14.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:14.840+0000 7f3068400700 1 -- 172.21.15.37:0/3435918286 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30600fe550 con 0x7f30601037a0 2024-03-29T08:50:14.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:14.841+0000 7f305d600700 1 -- 172.21.15.37:0/3435918286 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f304c009710 con 0x7f30601037a0 2024-03-29T08:50:14.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:14.842+0000 7f305d600700 1 --2- 172.21.15.37:0/3435918286 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f304806cc30 0x7f304806f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:50:14.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:14.842+0000 7f305d600700 1 -- 172.21.15.37:0/3435918286 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f304c08bda0 con 0x7f30601037a0 2024-03-29T08:50:14.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:14.842+0000 7f3065a00700 1 --2- 172.21.15.37:0/3435918286 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f304806cc30 0x7f304806f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:50:14.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:14.843+0000 7f3065a00700 1 --2- 172.21.15.37:0/3435918286 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f304806cc30 0x7f304806f0e0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f305400b3b0 tx=0x7f3054009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:50:14.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:14.847+0000 7f305d600700 1 -- 172.21.15.37:0/3435918286 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f304c0595e0 con 0x7f30601037a0 2024-03-29T08:50:15.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:15.107+0000 7f3068400700 1 -- 172.21.15.37:0/3435918286 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f3060109dc0 con 0x7f30601037a0 2024-03-29T08:50:15.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:15.108+0000 7f305d600700 1 -- 172.21.15.37:0/3435918286 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 36 v36) v1 ==== 76+0+1862 (secure 0 0 0) 0x7f304c059400 con 0x7f30601037a0 2024-03-29T08:50:15.109 INFO:teuthology.orchestra.run.smithi037.stdout:e36 2024-03-29T08:50:15.109 INFO:teuthology.orchestra.run.smithi037.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-29T08:50:15.109 INFO:teuthology.orchestra.run.smithi037.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T08:50:15.109 INFO:teuthology.orchestra.run.smithi037.stdout:legacy client fscid: 1 2024-03-29T08:50:15.109 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:50:15.109 INFO:teuthology.orchestra.run.smithi037.stdout:Filesystem 'cephfs' (1) 2024-03-29T08:50:15.109 INFO:teuthology.orchestra.run.smithi037.stdout:fs_name cephfs 2024-03-29T08:50:15.109 INFO:teuthology.orchestra.run.smithi037.stdout:epoch 36 2024-03-29T08:50:15.109 INFO:teuthology.orchestra.run.smithi037.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-03-29T08:50:15.109 INFO:teuthology.orchestra.run.smithi037.stdout:created 2024-03-29T08:03:29.241269+0000 2024-03-29T08:50:15.109 INFO:teuthology.orchestra.run.smithi037.stdout:modified 2024-03-29T08:48:32.995435+0000 2024-03-29T08:50:15.109 INFO:teuthology.orchestra.run.smithi037.stdout:tableserver 0 2024-03-29T08:50:15.110 INFO:teuthology.orchestra.run.smithi037.stdout:root 0 2024-03-29T08:50:15.110 INFO:teuthology.orchestra.run.smithi037.stdout:session_timeout 60 2024-03-29T08:50:15.110 INFO:teuthology.orchestra.run.smithi037.stdout:session_autoclose 300 2024-03-29T08:50:15.110 INFO:teuthology.orchestra.run.smithi037.stdout:max_file_size 1099511627776 2024-03-29T08:50:15.110 INFO:teuthology.orchestra.run.smithi037.stdout:max_xattr_size 65536 2024-03-29T08:50:15.110 INFO:teuthology.orchestra.run.smithi037.stdout:required_client_features {} 2024-03-29T08:50:15.110 INFO:teuthology.orchestra.run.smithi037.stdout:last_failure 0 2024-03-29T08:50:15.110 INFO:teuthology.orchestra.run.smithi037.stdout:last_failure_osd_epoch 82 2024-03-29T08:50:15.110 INFO:teuthology.orchestra.run.smithi037.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-29T08:50:15.110 INFO:teuthology.orchestra.run.smithi037.stdout:max_mds 2 2024-03-29T08:50:15.110 INFO:teuthology.orchestra.run.smithi037.stdout:in 0,1 2024-03-29T08:50:15.110 INFO:teuthology.orchestra.run.smithi037.stdout:up {0=34440,1=34444} 2024-03-29T08:50:15.110 INFO:teuthology.orchestra.run.smithi037.stdout:failed 2024-03-29T08:50:15.111 INFO:teuthology.orchestra.run.smithi037.stdout:damaged 2024-03-29T08:50:15.111 INFO:teuthology.orchestra.run.smithi037.stdout:stopped 2024-03-29T08:50:15.111 INFO:teuthology.orchestra.run.smithi037.stdout:data_pools [3] 2024-03-29T08:50:15.111 INFO:teuthology.orchestra.run.smithi037.stdout:metadata_pool 2 2024-03-29T08:50:15.111 INFO:teuthology.orchestra.run.smithi037.stdout:inline_data disabled 2024-03-29T08:50:15.111 INFO:teuthology.orchestra.run.smithi037.stdout:balancer 2024-03-29T08:50:15.111 INFO:teuthology.orchestra.run.smithi037.stdout:bal_rank_mask -1 2024-03-29T08:50:15.111 INFO:teuthology.orchestra.run.smithi037.stdout:standby_count_wanted 1 2024-03-29T08:50:15.111 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi037.mmwgxe{0:34440} state up:active seq 470 join_fscid=1 addr [v2:172.21.15.37:6828/1248142005,v1:172.21.15.37:6829/1248142005] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:50:15.111 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi037.iifsmh{1:34444} state up:active seq 465 join_fscid=1 addr [v2:172.21.15.37:6826/2060882959,v1:172.21.15.37:6827/2060882959] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:50:15.111 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:50:15.111 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:50:15.111 INFO:teuthology.orchestra.run.smithi037.stdout:Standby daemons: 2024-03-29T08:50:15.112 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:50:15.112 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi148.sqteih{-1:44395} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.148:6824/4078652242,v1:172.21.15.148:6825/4078652242] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:50:15.112 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.cephfs.smithi148.qukpjh{-1:45007} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.148:6826/2394751105,v1:172.21.15.148:6827/2394751105] compat {c=[1],r=[1],i=[7ff]}] 2024-03-29T08:50:15.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:15.110+0000 7f3068400700 1 -- 172.21.15.37:0/3435918286 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f304806cc30 msgr2=0x7f304806f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:50:15.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:15.110+0000 7f3068400700 1 --2- 172.21.15.37:0/3435918286 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f304806cc30 0x7f304806f0e0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f305400b3b0 tx=0x7f3054009e30 comp rx=0 tx=0).stop 2024-03-29T08:50:15.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:15.110+0000 7f3068400700 1 -- 172.21.15.37:0/3435918286 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f30601037a0 msgr2=0x7f3060114060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:50:15.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:15.110+0000 7f3068400700 1 --2- 172.21.15.37:0/3435918286 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f30601037a0 0x7f3060114060 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f304c002a50 tx=0x7f304c004600 comp rx=0 tx=0).stop 2024-03-29T08:50:15.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:15.110+0000 7f3068400700 1 -- 172.21.15.37:0/3435918286 shutdown_connections 2024-03-29T08:50:15.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:15.110+0000 7f3068400700 1 --2- 172.21.15.37:0/3435918286 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f304806cc30 0x7f304806f0e0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:15.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:15.110+0000 7f3068400700 1 --2- 172.21.15.37:0/3435918286 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3060102e00 0x7f3060113b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:15.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:15.110+0000 7f3068400700 1 --2- 172.21.15.37:0/3435918286 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f30601037a0 0x7f3060114060 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:15.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:15.110+0000 7f3068400700 1 -- 172.21.15.37:0/3435918286 >> 172.21.15.37:0/3435918286 conn(0x7f30600f89b0 msgr2=0x7f30600f9a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:50:15.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:15.110+0000 7f3068400700 1 -- 172.21.15.37:0/3435918286 shutdown_connections 2024-03-29T08:50:15.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:15.110+0000 7f3068400700 1 -- 172.21.15.37:0/3435918286 wait complete. 2024-03-29T08:50:15.113 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 36 2024-03-29T08:50:15.900 INFO:teuthology.run_tasks:Running task fs.post_upgrade_checks... 2024-03-29T08:50:15.912 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T08:50:16.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:16.109+0000 7f0628e00700 1 -- 172.21.15.37:0/1777804069 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0624102d00 msgr2=0x7f06241030d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:50:16.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:16.109+0000 7f0628e00700 1 --2- 172.21.15.37:0/1777804069 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0624102d00 0x7f06241030d0 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f0614004940 tx=0x7f0614009e30 comp rx=0 tx=0).stop 2024-03-29T08:50:16.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:16.109+0000 7f0628e00700 1 -- 172.21.15.37:0/1777804069 shutdown_connections 2024-03-29T08:50:16.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:16.109+0000 7f0628e00700 1 --2- 172.21.15.37:0/1777804069 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f06241036a0 0x7f06240fcd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:16.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:16.109+0000 7f0628e00700 1 --2- 172.21.15.37:0/1777804069 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0624102d00 0x7f06241030d0 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:16.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:16.109+0000 7f0628e00700 1 -- 172.21.15.37:0/1777804069 >> 172.21.15.37:0/1777804069 conn(0x7f06240f8850 msgr2=0x7f06240fac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:50:16.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:16.109+0000 7f0628e00700 1 -- 172.21.15.37:0/1777804069 shutdown_connections 2024-03-29T08:50:16.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:16.109+0000 7f0628e00700 1 -- 172.21.15.37:0/1777804069 wait complete. 2024-03-29T08:50:16.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:16.110+0000 7f0628e00700 1 Processor -- start 2024-03-29T08:50:16.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:16.110+0000 7f0628e00700 1 -- start start 2024-03-29T08:50:16.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:16.110+0000 7f0628e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0624102d00 0x7f0624115990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:50:16.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:16.110+0000 7f0628e00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f06241036a0 0x7f0624115ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:50:16.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:16.110+0000 7f0628e00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0624116520 con 0x7f0624102d00 2024-03-29T08:50:16.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:16.110+0000 7f0628e00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f062410fb40 con 0x7f06241036a0 2024-03-29T08:50:16.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:16.111+0000 7f0621e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0624102d00 0x7f0624115990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:50:16.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:16.111+0000 7f0621e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0624102d00 0x7f0624115990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60206/0 (socket says 172.21.15.37:60206) 2024-03-29T08:50:16.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:16.111+0000 7f0621e00700 1 -- 172.21.15.37:0/144201015 learned_addr learned my addr 172.21.15.37:0/144201015 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:50:16.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:16.111+0000 7f0621400700 1 --2- 172.21.15.37:0/144201015 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f06241036a0 0x7f0624115ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:50:16.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:16.111+0000 7f0621e00700 1 -- 172.21.15.37:0/144201015 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f06241036a0 msgr2=0x7f0624115ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:50:16.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:16.111+0000 7f0621e00700 1 --2- 172.21.15.37:0/144201015 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f06241036a0 0x7f0624115ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:16.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:16.111+0000 7f0621e00700 1 -- 172.21.15.37:0/144201015 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0614005040 con 0x7f0624102d00 2024-03-29T08:50:16.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:16.111+0000 7f0621400700 1 --2- 172.21.15.37:0/144201015 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f06241036a0 0x7f0624115ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:50:16.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:16.112+0000 7f0621e00700 1 --2- 172.21.15.37:0/144201015 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0624102d00 0x7f0624115990 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f0614009390 tx=0x7f0614009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:50:16.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:16.112+0000 7f061a000700 1 -- 172.21.15.37:0/144201015 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f061400e040 con 0x7f0624102d00 2024-03-29T08:50:16.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:16.112+0000 7f0628e00700 1 -- 172.21.15.37:0/144201015 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f062410fdc0 con 0x7f0624102d00 2024-03-29T08:50:16.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:16.112+0000 7f061a000700 1 -- 172.21.15.37:0/144201015 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f0614013420 con 0x7f0624102d00 2024-03-29T08:50:16.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:16.112+0000 7f061a000700 1 -- 172.21.15.37:0/144201015 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f061401c630 con 0x7f0624102d00 2024-03-29T08:50:16.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:16.112+0000 7f0628e00700 1 -- 172.21.15.37:0/144201015 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0624110380 con 0x7f0624102d00 2024-03-29T08:50:16.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:16.113+0000 7f0628e00700 1 -- 172.21.15.37:0/144201015 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06240fe450 con 0x7f0624102d00 2024-03-29T08:50:16.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:16.114+0000 7f061a000700 1 -- 172.21.15.37:0/144201015 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f06140139e0 con 0x7f0624102d00 2024-03-29T08:50:16.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:16.115+0000 7f061a000700 1 --2- 172.21.15.37:0/144201015 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f060806cc20 0x7f060806f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:50:16.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:16.115+0000 7f061a000700 1 -- 172.21.15.37:0/144201015 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f061408d320 con 0x7f0624102d00 2024-03-29T08:50:16.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:16.115+0000 7f0621400700 1 --2- 172.21.15.37:0/144201015 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f060806cc20 0x7f060806f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:50:16.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:16.116+0000 7f0621400700 1 --2- 172.21.15.37:0/144201015 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f060806cc20 0x7f060806f0d0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f0610009fd0 tx=0x7f0610009ee0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:50:16.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:16.119+0000 7f061a000700 1 -- 172.21.15.37:0/144201015 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f0614057390 con 0x7f0624102d00 2024-03-29T08:50:16.176 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:50:15 smithi037 ceph-mon[142325]: pgmap v1183: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:50:16.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:50:15 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/3435918286' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T08:50:16.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:50:15 smithi148 ceph-mon[106692]: pgmap v1183: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:50:16.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:50:15 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/3435918286' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-29T08:50:16.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:16.338+0000 7f0628e00700 1 -- 172.21.15.37:0/144201015 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0624110fe0 con 0x7f0624102d00 2024-03-29T08:50:16.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:16.340+0000 7f061a000700 1 -- 172.21.15.37:0/144201015 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+16549 (secure 0 0 0) 0x7f061405a9b0 con 0x7f0624102d00 2024-03-29T08:50:16.341 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:50:16.341 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":86,"fsid":"99b492c2-eda1-11ee-b647-cb9ed24678a4","created":"2024-03-29T07:55:23.673934+0000","modified":"2024-03-29T08:33:15.225558+0000","last_up_change":"2024-03-29T08:17:12.117876+0000","last_in_change":"2024-03-29T08:01:43.066485+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":20,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T08:00:32.022416+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs.cephfs.meta","create_time":"2024-03-29T08:03:28.176923+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs.cephfs.data","create_time":"2024-03-29T08:03:28.754148+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dba18fb6-b27d-4258-b769-dc89f4fef44d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":50,"up_from":54,"up_thru":79,"down_at":51,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":2026381241},{"type":"v1","addr":"172.21.15.37:6803","nonce":2026381241}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":2026381241},{"type":"v1","addr":"172.21.15.37:6805","nonce":2026381241}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":2026381241},{"type":"v1","addr":"172.21.15.37:6809","nonce":2026381241}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":2026381241},{"type":"v1","addr":"172.21.15.37:6807","nonce":2026381241}]},"public_addr":"172.21.15.37:6803/2026381241","cluster_addr":"172.21.15.37:6805/2026381241","heartbeat_back_addr":"172.21.15.37:6809/2026381241","heartbeat_front_addr":"172.21.15.37:6807/2026381241","state":["exists","up"]},{"osd":1,"uuid":"2876dd3f-aefb-4298-8fe4-562245e1103e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":55,"up_from":59,"up_thru":79,"down_at":56,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":2942536992},{"type":"v1","addr":"172.21.15.37:6811","nonce":2942536992}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":2942536992},{"type":"v1","addr":"172.21.15.37:6813","nonce":2942536992}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2942536992},{"type":"v1","addr":"172.21.15.37:6817","nonce":2942536992}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":2942536992},{"type":"v1","addr":"172.21.15.37:6815","nonce":2942536992}]},"public_addr":"172.21.15.37:6811/2942536992","cluster_addr":"172.21.15.37:6813/2942536992","heartbeat_back_addr":"172.21.15.37:6817/2942536992","heartbeat_front_addr":"172.21.15.37:6815/2942536992","state":["exists","up"]},{"osd":2,"uuid":"bba0f95c-f4e3-404e-adf7-b90353a96af4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":19,"last_clean_end":60,"up_from":64,"up_thru":79,"down_at":61,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":4140655393},{"type":"v1","addr":"172.21.15.37:6819","nonce":4140655393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":4140655393},{"type":"v1","addr":"172.21.15.37:6821","nonce":4140655393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":4140655393},{"type":"v1","addr":"172.21.15.37:6825","nonce":4140655393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":4140655393},{"type":"v1","addr":"172.21.15.37:6823","nonce":4140655393}]},"public_addr":"172.21.15.37:6819/4140655393","cluster_addr":"172.21.15.37:6821/4140655393","heartbeat_back_addr":"172.21.15.37:6825/4140655393","heartbeat_front_addr":"172.21.15.37:6823/4140655393","state":["exists","up"]},{"osd":3,"uuid":"8da1c140-ada5-4879-b5d9-1ed81b0c9aef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":27,"last_clean_end":65,"up_from":69,"up_thru":79,"down_at":66,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":227480876},{"type":"v1","addr":"172.21.15.148:6801","nonce":227480876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":227480876},{"type":"v1","addr":"172.21.15.148:6803","nonce":227480876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":227480876},{"type":"v1","addr":"172.21.15.148:6807","nonce":227480876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":227480876},{"type":"v1","addr":"172.21.15.148:6805","nonce":227480876}]},"public_addr":"172.21.15.148:6801/227480876","cluster_addr":"172.21.15.148:6803/227480876","heartbeat_back_addr":"172.21.15.148:6807/227480876","heartbeat_front_addr":"172.21.15.148:6805/227480876","state":["exists","up"]},{"osd":4,"uuid":"9af70e66-bbca-48c7-bad1-7ac78270cf16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":35,"last_clean_end":70,"up_from":74,"up_thru":79,"down_at":71,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":572347061},{"type":"v1","addr":"172.21.15.148:6809","nonce":572347061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":572347061},{"type":"v1","addr":"172.21.15.148:6811","nonce":572347061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":572347061},{"type":"v1","addr":"172.21.15.148:6815","nonce":572347061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":572347061},{"type":"v1","addr":"172.21.15.148:6813","nonce":572347061}]},"public_addr":"172.21.15.148:6809/572347061","cluster_addr":"172.21.15.148:6811/572347061","heartbeat_back_addr":"172.21.15.148:6815/572347061","heartbeat_front_addr":"172.21.15.148:6813/572347061","state":["exists","up"]},{"osd":5,"uuid":"a66944e4-38bd-4403-b45f-7de01ddfb7a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":40,"last_clean_end":75,"up_from":79,"up_thru":79,"down_at":76,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":2577126633},{"type":"v1","addr":"172.21.15.148:6817","nonce":2577126633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":2577126633},{"type":"v1","addr":"172.21.15.148:6819","nonce":2577126633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":2577126633},{"type":"v1","addr":"172.21.15.148:6823","nonce":2577126633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":2577126633},{"type":"v1","addr":"172.21.15.148:6821","nonce":2577126633}]},"public_addr":"172.21.15.148:6817/2577126633","cluster_addr":"172.21.15.148:6819/2577126633","heartbeat_back_addr":"172.21.15.148:6823/2577126633","heartbeat_front_addr":"172.21.15.148:6821/2577126633","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T07:59:23.110900+0000","dead_epoch":50},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T07:59:55.540549+0000","dead_epoch":55},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:00:28.082290+0000","dead_epoch":60},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:00:59.496893+0000","dead_epoch":65},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:01:31.307445+0000","dead_epoch":70},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:02:03.191177+0000","dead_epoch":75}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:0/3846762095":"2024-03-30T07:57:55.402909+0000","172.21.15.37:6800/2746946658":"2024-03-30T07:56:36.633439+0000","172.21.15.37:0/1909774742":"2024-03-30T07:55:59.989528+0000","172.21.15.37:0/1421768676":"2024-03-30T07:57:55.402909+0000","172.21.15.37:0/1395183736":"2024-03-30T08:06:01.871937+0000","172.21.15.37:6801/1802836043":"2024-03-30T08:06:01.871937+0000","172.21.15.37:0/368538117":"2024-03-30T07:57:55.402909+0000","172.21.15.148:0/855865016":"2024-03-30T08:08:06.449195+0000","172.21.15.37:0/2242434702":"2024-03-30T08:06:01.871937+0000","172.21.15.148:0/1612512235":"2024-03-30T08:08:06.449195+0000","172.21.15.148:6828/1134897184":"2024-03-30T08:08:06.449195+0000","172.21.15.37:0/1131673968":"2024-03-30T07:56:36.633439+0000","172.21.15.37:0/959705723":"2024-03-30T07:56:36.633439+0000","172.21.15.37:0/3667241947":"2024-03-30T08:12:09.488085+0000","172.21.15.37:6827/1559618909":"2024-03-30T08:17:16.236377+0000","172.21.15.37:0/3644857958":"2024-03-30T07:55:59.989528+0000","172.21.15.37:6828/2341530495":"2024-03-30T08:17:16.236404+0000","172.21.15.37:6800/1802836043":"2024-03-30T08:06:01.871937+0000","172.21.15.37:6801/2762339582":"2024-03-30T07:55:59.989528+0000","172.21.15.148:6829/1134897184":"2024-03-30T08:08:06.449195+0000","172.21.15.148:0/1484235408":"2024-03-30T08:08:06.449195+0000","172.21.15.37:6801/2746946658":"2024-03-30T07:56:36.633439+0000","172.21.15.148:0/4266419604":"2024-03-30T08:08:06.449195+0000","172.21.15.37:6800/4245458578":"2024-03-30T07:57:55.402909+0000","172.21.15.37:6829/2341530495":"2024-03-30T08:17:16.236404+0000","172.21.15.148:6824/2050993679":"2024-03-30T08:03:46.637260+0000","172.21.15.37:6801/4245458578":"2024-03-30T07:57:55.402909+0000","172.21.15.37:0/3116346321":"2024-03-30T07:55:59.989528+0000","172.21.15.37:0/2483635161":"2024-03-30T07:56:36.633439+0000","172.21.15.37:6800/4188329516":"2024-03-30T08:12:09.488085+0000","172.21.15.37:6800/2762339582":"2024-03-30T07:55:59.989528+0000","172.21.15.37:0/2522921436":"2024-03-30T08:06:01.871937+0000","172.21.15.37:0/968152496":"2024-03-30T08:12:09.488085+0000","172.21.15.37:0/68050196":"2024-03-30T08:12:09.488085+0000","172.21.15.37:0/2701457924":"2024-03-30T08:06:01.871937+0000","172.21.15.37:6801/4188329516":"2024-03-30T08:12:09.488085+0000","172.21.15.37:6826/1559618909":"2024-03-30T08:17:16.236377+0000","172.21.15.148:6825/2050993679":"2024-03-30T08:03:46.637260+0000","172.21.15.37:0/331291813":"2024-03-30T08:12:09.488085+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T08:50:16.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:16.343+0000 7f0628e00700 1 -- 172.21.15.37:0/144201015 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f060806cc20 msgr2=0x7f060806f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:50:16.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:16.343+0000 7f0628e00700 1 --2- 172.21.15.37:0/144201015 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f060806cc20 0x7f060806f0d0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f0610009fd0 tx=0x7f0610009ee0 comp rx=0 tx=0).stop 2024-03-29T08:50:16.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:16.343+0000 7f0628e00700 1 -- 172.21.15.37:0/144201015 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0624102d00 msgr2=0x7f0624115990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:50:16.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:16.343+0000 7f0628e00700 1 --2- 172.21.15.37:0/144201015 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0624102d00 0x7f0624115990 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f0614009390 tx=0x7f0614009e30 comp rx=0 tx=0).stop 2024-03-29T08:50:16.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:16.343+0000 7f0628e00700 1 -- 172.21.15.37:0/144201015 shutdown_connections 2024-03-29T08:50:16.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:16.343+0000 7f0628e00700 1 --2- 172.21.15.37:0/144201015 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f060806cc20 0x7f060806f0d0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:16.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:16.343+0000 7f0628e00700 1 --2- 172.21.15.37:0/144201015 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0624102d00 0x7f0624115990 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:16.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:16.343+0000 7f0628e00700 1 --2- 172.21.15.37:0/144201015 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f06241036a0 0x7f0624115ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:16.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:16.343+0000 7f0628e00700 1 -- 172.21.15.37:0/144201015 >> 172.21.15.37:0/144201015 conn(0x7f06240f8850 msgr2=0x7f06240f98a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:50:16.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:16.343+0000 7f0628e00700 1 -- 172.21.15.37:0/144201015 shutdown_connections 2024-03-29T08:50:16.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:16.343+0000 7f0628e00700 1 -- 172.21.15.37:0/144201015 wait complete. 2024-03-29T08:50:16.361 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-29T08:00:32.022416+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 6, 'score_stable': 6, 'optimal_score': 0.5, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs.cephfs.meta', 'create_time': '2024-03-29T08:03:28.176923+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.690000057220459, 'score_stable': 1.690000057220459, 'optimal_score': 1, 'raw_score_acting': 1.690000057220459, 'raw_score_stable': 1.690000057220459, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs.cephfs.data', 'create_time': '2024-03-29T08:03:28.754148+0000', 'flags': 131073, 'flags_names': 'hashpspool,bulk', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '46', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-29T08:50:16.362 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-29T08:50:16.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:16.557+0000 7f4df2400700 1 -- 172.21.15.37:0/3082174278 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4dec103670 msgr2=0x7f4dec0fcce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:50:16.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:16.557+0000 7f4df2400700 1 --2- 172.21.15.37:0/3082174278 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4dec103670 0x7f4dec0fcce0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f4dd8004940 tx=0x7f4dd8009e30 comp rx=0 tx=0).stop 2024-03-29T08:50:16.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:16.557+0000 7f4df2400700 1 -- 172.21.15.37:0/3082174278 shutdown_connections 2024-03-29T08:50:16.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:16.557+0000 7f4df2400700 1 --2- 172.21.15.37:0/3082174278 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4dec103670 0x7f4dec0fcce0 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:16.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:16.557+0000 7f4df2400700 1 --2- 172.21.15.37:0/3082174278 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4dec102cd0 0x7f4dec1030a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:16.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:16.557+0000 7f4df2400700 1 -- 172.21.15.37:0/3082174278 >> 172.21.15.37:0/3082174278 conn(0x7f4dec0f8820 msgr2=0x7f4dec0fac30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:50:16.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:16.557+0000 7f4df2400700 1 -- 172.21.15.37:0/3082174278 shutdown_connections 2024-03-29T08:50:16.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:16.557+0000 7f4df2400700 1 -- 172.21.15.37:0/3082174278 wait complete. 2024-03-29T08:50:16.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:16.558+0000 7f4df2400700 1 Processor -- start 2024-03-29T08:50:16.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:16.558+0000 7f4df2400700 1 -- start start 2024-03-29T08:50:16.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:16.558+0000 7f4df2400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4dec102cd0 0x7f4dec1136e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:50:16.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:16.558+0000 7f4df2400700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4dec103670 0x7f4dec113c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:50:16.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:16.558+0000 7f4df2400700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dec114270 con 0x7f4dec102cd0 2024-03-29T08:50:16.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:16.558+0000 7f4df2400700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dec10d890 con 0x7f4dec103670 2024-03-29T08:50:16.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:16.559+0000 7f4deb400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4dec102cd0 0x7f4dec1136e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:50:16.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:16.559+0000 7f4deb400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4dec102cd0 0x7f4dec1136e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60226/0 (socket says 172.21.15.37:60226) 2024-03-29T08:50:16.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:16.559+0000 7f4deb400700 1 -- 172.21.15.37:0/118871989 learned_addr learned my addr 172.21.15.37:0/118871989 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:50:16.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:16.559+0000 7f4deaa00700 1 --2- 172.21.15.37:0/118871989 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4dec103670 0x7f4dec113c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:50:16.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:16.559+0000 7f4deb400700 1 -- 172.21.15.37:0/118871989 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4dec103670 msgr2=0x7f4dec113c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:50:16.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:16.559+0000 7f4deb400700 1 --2- 172.21.15.37:0/118871989 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4dec103670 0x7f4dec113c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:16.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:16.559+0000 7f4deb400700 1 -- 172.21.15.37:0/118871989 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4dd8005040 con 0x7f4dec102cd0 2024-03-29T08:50:16.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:16.559+0000 7f4deb400700 1 --2- 172.21.15.37:0/118871989 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4dec102cd0 0x7f4dec1136e0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f4ddc00eb40 tx=0x7f4ddc00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:50:16.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:16.559+0000 7f4de3400700 1 -- 172.21.15.37:0/118871989 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4ddc007cc0 con 0x7f4dec102cd0 2024-03-29T08:50:16.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:16.559+0000 7f4df2400700 1 -- 172.21.15.37:0/118871989 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4dec10db10 con 0x7f4dec102cd0 2024-03-29T08:50:16.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:16.559+0000 7f4de3400700 1 -- 172.21.15.37:0/118871989 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f4ddc00f040 con 0x7f4dec102cd0 2024-03-29T08:50:16.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:16.559+0000 7f4de3400700 1 -- 172.21.15.37:0/118871989 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4ddc009570 con 0x7f4dec102cd0 2024-03-29T08:50:16.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:16.560+0000 7f4df2400700 1 -- 172.21.15.37:0/118871989 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4dec10e0d0 con 0x7f4dec102cd0 2024-03-29T08:50:16.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:16.561+0000 7f4df2400700 1 -- 172.21.15.37:0/118871989 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4dec0fe420 con 0x7f4dec102cd0 2024-03-29T08:50:16.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:16.561+0000 7f4de3400700 1 -- 172.21.15.37:0/118871989 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f4ddc0096d0 con 0x7f4dec102cd0 2024-03-29T08:50:16.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:16.562+0000 7f4de3400700 1 --2- 172.21.15.37:0/118871989 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f4dd006cc20 0x7f4dd006f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:50:16.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:16.562+0000 7f4de3400700 1 -- 172.21.15.37:0/118871989 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f4ddc08bda0 con 0x7f4dec102cd0 2024-03-29T08:50:16.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:16.563+0000 7f4deaa00700 1 --2- 172.21.15.37:0/118871989 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f4dd006cc20 0x7f4dd006f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:50:16.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:16.563+0000 7f4deaa00700 1 --2- 172.21.15.37:0/118871989 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f4dd006cc20 0x7f4dd006f0d0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f4dd8009fd0 tx=0x7f4dd8009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:50:16.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:16.567+0000 7f4de3400700 1 -- 172.21.15.37:0/118871989 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f4ddc059820 con 0x7f4dec102cd0 2024-03-29T08:50:16.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:16.782+0000 7f4df2400700 1 -- 172.21.15.37:0/118871989 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4dec10ed70 con 0x7f4dec102cd0 2024-03-29T08:50:16.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:16.782+0000 7f4de3400700 1 -- 172.21.15.37:0/118871989 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v86) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4ddc0593b0 con 0x7f4dec102cd0 2024-03-29T08:50:16.783 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 1 2024-03-29T08:50:16.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:16.785+0000 7f4df2400700 1 -- 172.21.15.37:0/118871989 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f4dd006cc20 msgr2=0x7f4dd006f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:50:16.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:16.785+0000 7f4df2400700 1 --2- 172.21.15.37:0/118871989 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f4dd006cc20 0x7f4dd006f0d0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f4dd8009fd0 tx=0x7f4dd8009e30 comp rx=0 tx=0).stop 2024-03-29T08:50:16.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:16.785+0000 7f4df2400700 1 -- 172.21.15.37:0/118871989 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4dec102cd0 msgr2=0x7f4dec1136e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:50:16.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:16.785+0000 7f4df2400700 1 --2- 172.21.15.37:0/118871989 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4dec102cd0 0x7f4dec1136e0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f4ddc00eb40 tx=0x7f4ddc00b040 comp rx=0 tx=0).stop 2024-03-29T08:50:16.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:16.785+0000 7f4df2400700 1 -- 172.21.15.37:0/118871989 shutdown_connections 2024-03-29T08:50:16.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:16.785+0000 7f4df2400700 1 --2- 172.21.15.37:0/118871989 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f4dd006cc20 0x7f4dd006f0d0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:16.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:16.785+0000 7f4df2400700 1 --2- 172.21.15.37:0/118871989 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4dec102cd0 0x7f4dec1136e0 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:16.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:16.785+0000 7f4df2400700 1 --2- 172.21.15.37:0/118871989 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4dec103670 0x7f4dec113c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:16.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:16.785+0000 7f4df2400700 1 -- 172.21.15.37:0/118871989 >> 172.21.15.37:0/118871989 conn(0x7f4dec0f8820 msgr2=0x7f4dec0f9890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:50:16.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:16.785+0000 7f4df2400700 1 -- 172.21.15.37:0/118871989 shutdown_connections 2024-03-29T08:50:16.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:16.785+0000 7f4df2400700 1 -- 172.21.15.37:0/118871989 wait complete. 2024-03-29T08:50:16.803 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.meta pg_num 2024-03-29T08:50:17.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:17.000+0000 7f6d37c00700 1 -- 172.21.15.37:0/2142531942 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6d30102d00 msgr2=0x7f6d301030d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:50:17.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:17.000+0000 7f6d37c00700 1 --2- 172.21.15.37:0/2142531942 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6d30102d00 0x7f6d301030d0 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f6d24004940 tx=0x7f6d24009e30 comp rx=0 tx=0).stop 2024-03-29T08:50:17.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:17.000+0000 7f6d37c00700 1 -- 172.21.15.37:0/2142531942 shutdown_connections 2024-03-29T08:50:17.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:17.000+0000 7f6d37c00700 1 --2- 172.21.15.37:0/2142531942 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6d301036a0 0x7f6d300fcd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:17.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:17.000+0000 7f6d37c00700 1 --2- 172.21.15.37:0/2142531942 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6d30102d00 0x7f6d301030d0 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:17.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:17.000+0000 7f6d37c00700 1 -- 172.21.15.37:0/2142531942 >> 172.21.15.37:0/2142531942 conn(0x7f6d300f8810 msgr2=0x7f6d300fac20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:50:17.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:17.000+0000 7f6d37c00700 1 -- 172.21.15.37:0/2142531942 shutdown_connections 2024-03-29T08:50:17.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:17.000+0000 7f6d37c00700 1 -- 172.21.15.37:0/2142531942 wait complete. 2024-03-29T08:50:17.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:17.001+0000 7f6d37c00700 1 Processor -- start 2024-03-29T08:50:17.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:17.001+0000 7f6d37c00700 1 -- start start 2024-03-29T08:50:17.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:17.001+0000 7f6d37c00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6d30102d00 0x7f6d30115970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:50:17.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:17.001+0000 7f6d37c00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6d301036a0 0x7f6d30115eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:50:17.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:17.001+0000 7f6d37c00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d30116590 con 0x7f6d301036a0 2024-03-29T08:50:17.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:17.001+0000 7f6d37c00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d3010fb00 con 0x7f6d30102d00 2024-03-29T08:50:17.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:17.002+0000 7f6d2fe00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6d301036a0 0x7f6d30115eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:50:17.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:17.002+0000 7f6d2fe00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6d301036a0 0x7f6d30115eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60252/0 (socket says 172.21.15.37:60252) 2024-03-29T08:50:17.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:17.002+0000 7f6d2fe00700 1 -- 172.21.15.37:0/2273805062 learned_addr learned my addr 172.21.15.37:0/2273805062 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:50:17.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:17.002+0000 7f6d35200700 1 --2- 172.21.15.37:0/2273805062 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6d30102d00 0x7f6d30115970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:50:17.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:17.002+0000 7f6d2fe00700 1 -- 172.21.15.37:0/2273805062 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6d30102d00 msgr2=0x7f6d30115970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:50:17.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:17.002+0000 7f6d2fe00700 1 --2- 172.21.15.37:0/2273805062 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6d30102d00 0x7f6d30115970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:17.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:17.002+0000 7f6d2fe00700 1 -- 172.21.15.37:0/2273805062 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d24005040 con 0x7f6d301036a0 2024-03-29T08:50:17.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:17.002+0000 7f6d2fe00700 1 --2- 172.21.15.37:0/2273805062 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6d301036a0 0x7f6d30115eb0 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f6d1c00b790 tx=0x7f6d1c009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:50:17.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:17.003+0000 7f6d2cc00700 1 -- 172.21.15.37:0/2273805062 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6d1c00f6f0 con 0x7f6d301036a0 2024-03-29T08:50:17.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:17.003+0000 7f6d2cc00700 1 -- 172.21.15.37:0/2273805062 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f6d1c00fd00 con 0x7f6d301036a0 2024-03-29T08:50:17.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:17.003+0000 7f6d37c00700 1 -- 172.21.15.37:0/2273805062 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d3010fde0 con 0x7f6d301036a0 2024-03-29T08:50:17.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:17.003+0000 7f6d2cc00700 1 -- 172.21.15.37:0/2273805062 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6d1c0103a0 con 0x7f6d301036a0 2024-03-29T08:50:17.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:17.003+0000 7f6d37c00700 1 -- 172.21.15.37:0/2273805062 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d301103a0 con 0x7f6d301036a0 2024-03-29T08:50:17.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:17.004+0000 7f6d37c00700 1 -- 172.21.15.37:0/2273805062 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d300fe450 con 0x7f6d301036a0 2024-03-29T08:50:17.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:17.005+0000 7f6d2cc00700 1 -- 172.21.15.37:0/2273805062 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f6d1c00f850 con 0x7f6d301036a0 2024-03-29T08:50:17.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:17.006+0000 7f6d2cc00700 1 --2- 172.21.15.37:0/2273805062 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f6d1806cc20 0x7f6d1806f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:50:17.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:17.006+0000 7f6d2cc00700 1 -- 172.21.15.37:0/2273805062 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f6d1c08bf60 con 0x7f6d301036a0 2024-03-29T08:50:17.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:17.006+0000 7f6d35200700 1 --2- 172.21.15.37:0/2273805062 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f6d1806cc20 0x7f6d1806f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:50:17.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:17.007+0000 7f6d35200700 1 --2- 172.21.15.37:0/2273805062 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f6d1806cc20 0x7f6d1806f0d0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f6d2400b020 tx=0x7f6d24009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:50:17.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:17.010+0000 7f6d2cc00700 1 -- 172.21.15.37:0/2273805062 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f6d1c059930 con 0x7f6d301036a0 2024-03-29T08:50:17.176 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:50:16 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/144201015' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-29T08:50:17.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:50:16 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/118871989' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-29T08:50:17.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:17.226+0000 7f6d37c00700 1 -- 172.21.15.37:0/2273805062 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"} v 0) v1 -- 0x7f6d30064970 con 0x7f6d301036a0 2024-03-29T08:50:17.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:17.226+0000 7f6d2cc00700 1 -- 172.21.15.37:0/2273805062 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]=0 v86) v1 ==== 107+0+11 (secure 0 0 0) 0x7f6d1c0594c0 con 0x7f6d301036a0 2024-03-29T08:50:17.227 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 32 2024-03-29T08:50:17.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:17.229+0000 7f6d37c00700 1 -- 172.21.15.37:0/2273805062 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f6d1806cc20 msgr2=0x7f6d1806f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:50:17.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:17.229+0000 7f6d37c00700 1 --2- 172.21.15.37:0/2273805062 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f6d1806cc20 0x7f6d1806f0d0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f6d2400b020 tx=0x7f6d24009e30 comp rx=0 tx=0).stop 2024-03-29T08:50:17.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:17.229+0000 7f6d37c00700 1 -- 172.21.15.37:0/2273805062 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6d301036a0 msgr2=0x7f6d30115eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:50:17.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:17.229+0000 7f6d37c00700 1 --2- 172.21.15.37:0/2273805062 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6d301036a0 0x7f6d30115eb0 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f6d1c00b790 tx=0x7f6d1c009220 comp rx=0 tx=0).stop 2024-03-29T08:50:17.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:17.229+0000 7f6d37c00700 1 -- 172.21.15.37:0/2273805062 shutdown_connections 2024-03-29T08:50:17.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:17.229+0000 7f6d37c00700 1 --2- 172.21.15.37:0/2273805062 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f6d1806cc20 0x7f6d1806f0d0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:17.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:17.229+0000 7f6d37c00700 1 --2- 172.21.15.37:0/2273805062 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6d30102d00 0x7f6d30115970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:17.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:17.229+0000 7f6d37c00700 1 --2- 172.21.15.37:0/2273805062 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6d301036a0 0x7f6d30115eb0 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:17.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:17.229+0000 7f6d37c00700 1 -- 172.21.15.37:0/2273805062 >> 172.21.15.37:0/2273805062 conn(0x7f6d300f8810 msgr2=0x7f6d30071af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:50:17.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:17.229+0000 7f6d37c00700 1 -- 172.21.15.37:0/2273805062 shutdown_connections 2024-03-29T08:50:17.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:17.229+0000 7f6d37c00700 1 -- 172.21.15.37:0/2273805062 wait complete. 2024-03-29T08:50:17.249 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.data pg_num 2024-03-29T08:50:17.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:50:16 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/144201015' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-29T08:50:17.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:50:16 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/118871989' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-29T08:50:17.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:17.446+0000 7fd31f200700 1 -- 172.21.15.37:0/2677659121 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd318102cd0 msgr2=0x7fd3181030a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:50:17.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:17.446+0000 7fd31f200700 1 --2- 172.21.15.37:0/2677659121 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd318102cd0 0x7fd3181030a0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7fd308004940 tx=0x7fd308009e30 comp rx=0 tx=0).stop 2024-03-29T08:50:17.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:17.446+0000 7fd31f200700 1 -- 172.21.15.37:0/2677659121 shutdown_connections 2024-03-29T08:50:17.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:17.446+0000 7fd31f200700 1 --2- 172.21.15.37:0/2677659121 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd318103670 0x7fd3180fcce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:17.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:17.446+0000 7fd31f200700 1 --2- 172.21.15.37:0/2677659121 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd318102cd0 0x7fd3181030a0 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:17.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:17.446+0000 7fd31f200700 1 -- 172.21.15.37:0/2677659121 >> 172.21.15.37:0/2677659121 conn(0x7fd3180f8820 msgr2=0x7fd3180fac30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:50:17.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:17.446+0000 7fd31f200700 1 -- 172.21.15.37:0/2677659121 shutdown_connections 2024-03-29T08:50:17.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:17.446+0000 7fd31f200700 1 -- 172.21.15.37:0/2677659121 wait complete. 2024-03-29T08:50:17.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:17.447+0000 7fd31f200700 1 Processor -- start 2024-03-29T08:50:17.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:17.448+0000 7fd31f200700 1 -- start start 2024-03-29T08:50:17.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:17.448+0000 7fd31f200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd318102cd0 0x7fd3180702c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:50:17.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:17.448+0000 7fd31f200700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd318103670 0x7fd318070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:50:17.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:17.448+0000 7fd31f200700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd318072210 con 0x7fd318102cd0 2024-03-29T08:50:17.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:17.448+0000 7fd31f200700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd318070d40 con 0x7fd318103670 2024-03-29T08:50:17.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:17.449+0000 7fd317e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd318102cd0 0x7fd3180702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:50:17.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:17.449+0000 7fd317e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd318102cd0 0x7fd3180702c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60270/0 (socket says 172.21.15.37:60270) 2024-03-29T08:50:17.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:17.449+0000 7fd317e00700 1 -- 172.21.15.37:0/1111526003 learned_addr learned my addr 172.21.15.37:0/1111526003 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:50:17.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:17.449+0000 7fd317400700 1 --2- 172.21.15.37:0/1111526003 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd318103670 0x7fd318070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:50:17.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:17.449+0000 7fd317e00700 1 -- 172.21.15.37:0/1111526003 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd318103670 msgr2=0x7fd318070800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:50:17.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:17.449+0000 7fd317e00700 1 --2- 172.21.15.37:0/1111526003 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd318103670 0x7fd318070800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:17.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:17.449+0000 7fd317e00700 1 -- 172.21.15.37:0/1111526003 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd308005040 con 0x7fd318102cd0 2024-03-29T08:50:17.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:17.449+0000 7fd317e00700 1 --2- 172.21.15.37:0/1111526003 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd318102cd0 0x7fd3180702c0 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7fd308009fd0 tx=0x7fd308009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:50:17.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:17.449+0000 7fd30fe00700 1 -- 172.21.15.37:0/1111526003 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd308021070 con 0x7fd318102cd0 2024-03-29T08:50:17.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:17.449+0000 7fd30fe00700 1 -- 172.21.15.37:0/1111526003 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd308013e00 con 0x7fd318102cd0 2024-03-29T08:50:17.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:17.449+0000 7fd31f200700 1 -- 172.21.15.37:0/1111526003 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd318070fc0 con 0x7fd318102cd0 2024-03-29T08:50:17.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:17.450+0000 7fd30fe00700 1 -- 172.21.15.37:0/1111526003 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd30800e070 con 0x7fd318102cd0 2024-03-29T08:50:17.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:17.450+0000 7fd31f200700 1 -- 172.21.15.37:0/1111526003 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd318119e90 con 0x7fd318102cd0 2024-03-29T08:50:17.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:17.451+0000 7fd31f200700 1 -- 172.21.15.37:0/1111526003 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3180fe420 con 0x7fd318102cd0 2024-03-29T08:50:17.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:17.452+0000 7fd30fe00700 1 -- 172.21.15.37:0/1111526003 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fd30801c590 con 0x7fd318102cd0 2024-03-29T08:50:17.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:17.453+0000 7fd30fe00700 1 --2- 172.21.15.37:0/1111526003 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fd2fc06cc20 0x7fd2fc06f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:50:17.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:17.453+0000 7fd30fe00700 1 -- 172.21.15.37:0/1111526003 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fd30808d1f0 con 0x7fd318102cd0 2024-03-29T08:50:17.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:17.453+0000 7fd317400700 1 --2- 172.21.15.37:0/1111526003 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fd2fc06cc20 0x7fd2fc06f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:50:17.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:17.454+0000 7fd317400700 1 --2- 172.21.15.37:0/1111526003 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fd2fc06cc20 0x7fd2fc06f0d0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fd304004600 tx=0x7fd30400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:50:17.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:17.457+0000 7fd30fe00700 1 -- 172.21.15.37:0/1111526003 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fd308057230 con 0x7fd318102cd0 2024-03-29T08:50:17.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:17.673+0000 7fd31f200700 1 -- 172.21.15.37:0/1111526003 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"} v 0) v1 -- 0x7fd318064970 con 0x7fd318102cd0 2024-03-29T08:50:17.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:17.674+0000 7fd30fe00700 1 -- 172.21.15.37:0/1111526003 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]=0 v86) v1 ==== 107+0+11 (secure 0 0 0) 0x7fd30805a850 con 0x7fd318102cd0 2024-03-29T08:50:17.675 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 32 2024-03-29T08:50:17.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:17.676+0000 7fd31f200700 1 -- 172.21.15.37:0/1111526003 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fd2fc06cc20 msgr2=0x7fd2fc06f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:50:17.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:17.676+0000 7fd31f200700 1 --2- 172.21.15.37:0/1111526003 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fd2fc06cc20 0x7fd2fc06f0d0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fd304004600 tx=0x7fd30400b040 comp rx=0 tx=0).stop 2024-03-29T08:50:17.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:17.676+0000 7fd31f200700 1 -- 172.21.15.37:0/1111526003 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd318102cd0 msgr2=0x7fd3180702c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:50:17.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:17.676+0000 7fd31f200700 1 --2- 172.21.15.37:0/1111526003 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd318102cd0 0x7fd3180702c0 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7fd308009fd0 tx=0x7fd308009e30 comp rx=0 tx=0).stop 2024-03-29T08:50:17.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:17.677+0000 7fd31f200700 1 -- 172.21.15.37:0/1111526003 shutdown_connections 2024-03-29T08:50:17.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:17.677+0000 7fd31f200700 1 --2- 172.21.15.37:0/1111526003 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fd2fc06cc20 0x7fd2fc06f0d0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:17.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:17.677+0000 7fd31f200700 1 --2- 172.21.15.37:0/1111526003 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd318102cd0 0x7fd3180702c0 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:17.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:17.677+0000 7fd31f200700 1 --2- 172.21.15.37:0/1111526003 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd318103670 0x7fd318070800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:17.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:17.677+0000 7fd31f200700 1 -- 172.21.15.37:0/1111526003 >> 172.21.15.37:0/1111526003 conn(0x7fd3180f8820 msgr2=0x7fd3180f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:50:17.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:17.677+0000 7fd31f200700 1 -- 172.21.15.37:0/1111526003 shutdown_connections 2024-03-29T08:50:17.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:17.677+0000 7fd31f200700 1 -- 172.21.15.37:0/1111526003 wait complete. 2024-03-29T08:50:17.695 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-29T08:50:17.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:17.889+0000 7ffb7b600700 1 -- 172.21.15.37:0/1921834126 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffb74070c70 msgr2=0x7ffb74109010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:50:17.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:17.889+0000 7ffb7b600700 1 --2- 172.21.15.37:0/1921834126 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffb74070c70 0x7ffb74109010 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7ffb60004970 tx=0x7ffb60009e30 comp rx=0 tx=0).stop 2024-03-29T08:50:17.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:17.889+0000 7ffb7b600700 1 -- 172.21.15.37:0/1921834126 shutdown_connections 2024-03-29T08:50:17.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:17.889+0000 7ffb7b600700 1 --2- 172.21.15.37:0/1921834126 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffb74070c70 0x7ffb74109010 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:17.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:17.889+0000 7ffb7b600700 1 --2- 172.21.15.37:0/1921834126 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffb74070360 0x7ffb74070730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:17.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:17.889+0000 7ffb7b600700 1 -- 172.21.15.37:0/1921834126 >> 172.21.15.37:0/1921834126 conn(0x7ffb740f8830 msgr2=0x7ffb740fac40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:50:17.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:17.889+0000 7ffb7b600700 1 -- 172.21.15.37:0/1921834126 shutdown_connections 2024-03-29T08:50:17.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:17.889+0000 7ffb7b600700 1 -- 172.21.15.37:0/1921834126 wait complete. 2024-03-29T08:50:17.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:17.890+0000 7ffb7b600700 1 Processor -- start 2024-03-29T08:50:17.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:17.890+0000 7ffb7b600700 1 -- start start 2024-03-29T08:50:17.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:17.891+0000 7ffb7b600700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffb74070360 0x7ffb74115970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:50:17.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:17.891+0000 7ffb7b600700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffb74070c70 0x7ffb74115eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:50:17.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:17.891+0000 7ffb7b600700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb74116500 con 0x7ffb74070360 2024-03-29T08:50:17.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:17.891+0000 7ffb7b600700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb7410fb00 con 0x7ffb74070c70 2024-03-29T08:50:17.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:17.891+0000 7ffb78c00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffb74070360 0x7ffb74115970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:50:17.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:17.891+0000 7ffb78c00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffb74070360 0x7ffb74115970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60294/0 (socket says 172.21.15.37:60294) 2024-03-29T08:50:17.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:17.891+0000 7ffb78c00700 1 -- 172.21.15.37:0/1390625059 learned_addr learned my addr 172.21.15.37:0/1390625059 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:50:17.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:17.891+0000 7ffb73e00700 1 --2- 172.21.15.37:0/1390625059 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffb74070c70 0x7ffb74115eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:50:17.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:17.892+0000 7ffb78c00700 1 -- 172.21.15.37:0/1390625059 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffb74070c70 msgr2=0x7ffb74115eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:50:17.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:17.892+0000 7ffb78c00700 1 --2- 172.21.15.37:0/1390625059 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffb74070c70 0x7ffb74115eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:17.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:17.892+0000 7ffb78c00700 1 -- 172.21.15.37:0/1390625059 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb60005040 con 0x7ffb74070360 2024-03-29T08:50:17.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:17.892+0000 7ffb78c00700 1 --2- 172.21.15.37:0/1390625059 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffb74070360 0x7ffb74115970 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7ffb68002a50 tx=0x7ffb680046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:50:17.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:17.892+0000 7ffb70c00700 1 -- 172.21.15.37:0/1390625059 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffb68007cc0 con 0x7ffb74070360 2024-03-29T08:50:17.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:17.892+0000 7ffb70c00700 1 -- 172.21.15.37:0/1390625059 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ffb6800f040 con 0x7ffb74070360 2024-03-29T08:50:17.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:17.892+0000 7ffb7b600700 1 -- 172.21.15.37:0/1390625059 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb7410fde0 con 0x7ffb74070360 2024-03-29T08:50:17.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:17.892+0000 7ffb70c00700 1 -- 172.21.15.37:0/1390625059 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffb6800da70 con 0x7ffb74070360 2024-03-29T08:50:17.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:17.892+0000 7ffb7b600700 1 -- 172.21.15.37:0/1390625059 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb741102f0 con 0x7ffb74070360 2024-03-29T08:50:17.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:17.893+0000 7ffb7b600700 1 -- 172.21.15.37:0/1390625059 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb74071aa0 con 0x7ffb74070360 2024-03-29T08:50:17.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:17.894+0000 7ffb70c00700 1 -- 172.21.15.37:0/1390625059 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7ffb6800dc10 con 0x7ffb74070360 2024-03-29T08:50:17.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:17.895+0000 7ffb70c00700 1 --2- 172.21.15.37:0/1390625059 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7ffb5c06cc20 0x7ffb5c06f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:50:17.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:17.895+0000 7ffb70c00700 1 -- 172.21.15.37:0/1390625059 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7ffb6808b810 con 0x7ffb74070360 2024-03-29T08:50:17.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:17.895+0000 7ffb73e00700 1 --2- 172.21.15.37:0/1390625059 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7ffb5c06cc20 0x7ffb5c06f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:50:17.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:17.896+0000 7ffb73e00700 1 --2- 172.21.15.37:0/1390625059 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7ffb5c06cc20 0x7ffb5c06f0d0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7ffb74110de0 tx=0x7ffb60009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:50:17.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:17.900+0000 7ffb70c00700 1 -- 172.21.15.37:0/1390625059 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7ffb680591e0 con 0x7ffb74070360 2024-03-29T08:50:18.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:18.164+0000 7ffb7b600700 1 -- 172.21.15.37:0/1390625059 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ffb7410bf20 con 0x7ffb74070360 2024-03-29T08:50:18.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:18.165+0000 7ffb70c00700 1 -- 172.21.15.37:0/1390625059 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) v1 ==== 94+0+4851 (secure 0 0 0) 0x7ffb68058d70 con 0x7ffb74070360 2024-03-29T08:50:18.166 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:50:18.166 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":36,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44395,"name":"cephfs.smithi148.sqteih","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6825/4078652242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":4078652242},{"type":"v1","addr":"172.21.15.148:6825","nonce":4078652242}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28},{"gid":45007,"name":"cephfs.smithi148.qukpjh","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6827/2394751105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2394751105},{"type":"v1","addr":"172.21.15.148:6827","nonce":2394751105}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-29T08:03:29.241269+0000","modified":"2024-03-29T08:48:32.995435+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":82,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":34440,"mds_1":34444},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34440":{"gid":34440,"name":"cephfs.smithi037.mmwgxe","rank":0,"incarnation":32,"state":"up:active","state_seq":470,"addr":"172.21.15.37:6829/1248142005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1248142005},{"type":"v1","addr":"172.21.15.37:6829","nonce":1248142005}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_34444":{"gid":34444,"name":"cephfs.smithi037.iifsmh","rank":1,"incarnation":32,"state":"up:active","state_seq":465,"addr":"172.21.15.37:6827/2060882959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":2060882959},{"type":"v1","addr":"172.21.15.37:6827","nonce":2060882959}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-03-29T08:50:18.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:18.167+0000 7ffb7b600700 1 -- 172.21.15.37:0/1390625059 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7ffb5c06cc20 msgr2=0x7ffb5c06f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:50:18.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:18.167+0000 7ffb7b600700 1 --2- 172.21.15.37:0/1390625059 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7ffb5c06cc20 0x7ffb5c06f0d0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7ffb74110de0 tx=0x7ffb60009e30 comp rx=0 tx=0).stop 2024-03-29T08:50:18.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:18.167+0000 7ffb7b600700 1 -- 172.21.15.37:0/1390625059 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffb74070360 msgr2=0x7ffb74115970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:50:18.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:18.167+0000 7ffb7b600700 1 --2- 172.21.15.37:0/1390625059 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffb74070360 0x7ffb74115970 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7ffb68002a50 tx=0x7ffb680046b0 comp rx=0 tx=0).stop 2024-03-29T08:50:18.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:18.167+0000 7ffb7b600700 1 -- 172.21.15.37:0/1390625059 shutdown_connections 2024-03-29T08:50:18.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:18.167+0000 7ffb7b600700 1 --2- 172.21.15.37:0/1390625059 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7ffb5c06cc20 0x7ffb5c06f0d0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:18.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:18.167+0000 7ffb7b600700 1 --2- 172.21.15.37:0/1390625059 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffb74070360 0x7ffb74115970 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:18.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:18.167+0000 7ffb7b600700 1 --2- 172.21.15.37:0/1390625059 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffb74070c70 0x7ffb74115eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:18.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:18.167+0000 7ffb7b600700 1 -- 172.21.15.37:0/1390625059 >> 172.21.15.37:0/1390625059 conn(0x7ffb740f8830 msgr2=0x7ffb740f9d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:50:18.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:18.168+0000 7ffb7b600700 1 -- 172.21.15.37:0/1390625059 shutdown_connections 2024-03-29T08:50:18.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:18.168+0000 7ffb7b600700 1 -- 172.21.15.37:0/1390625059 wait complete. 2024-03-29T08:50:18.169 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 36 2024-03-29T08:50:18.176 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:50:17 smithi037 ceph-mon[142325]: pgmap v1184: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:50:18.176 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:50:17 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/2273805062' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-03-29T08:50:18.176 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:50:17 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/1111526003' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-03-29T08:50:18.185 DEBUG:tasks.fs:checking fs fscid=1,name=cephfs state = {'epoch': 16, 'max_mds': 2, 'flags': 18} 2024-03-29T08:50:18.186 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 17 2024-03-29T08:50:18.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:50:17 smithi148 ceph-mon[106692]: pgmap v1184: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:50:18.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:50:17 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/2273805062' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-03-29T08:50:18.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:50:17 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/1111526003' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-03-29T08:50:18.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:18.379+0000 7fb687000700 1 -- 172.21.15.37:0/2301584395 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb6801036a0 msgr2=0x7fb6800fcd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:50:18.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:18.379+0000 7fb687000700 1 --2- 172.21.15.37:0/2301584395 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb6801036a0 0x7fb6800fcd10 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7fb66c004940 tx=0x7fb66c009e30 comp rx=0 tx=0).stop 2024-03-29T08:50:18.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:18.380+0000 7fb687000700 1 -- 172.21.15.37:0/2301584395 shutdown_connections 2024-03-29T08:50:18.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:18.380+0000 7fb687000700 1 --2- 172.21.15.37:0/2301584395 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb6801036a0 0x7fb6800fcd10 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:18.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:18.380+0000 7fb687000700 1 --2- 172.21.15.37:0/2301584395 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb680102d00 0x7fb6801030d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:18.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:18.380+0000 7fb687000700 1 -- 172.21.15.37:0/2301584395 >> 172.21.15.37:0/2301584395 conn(0x7fb6800f8810 msgr2=0x7fb6800fac20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:50:18.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:18.380+0000 7fb687000700 1 -- 172.21.15.37:0/2301584395 shutdown_connections 2024-03-29T08:50:18.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:18.380+0000 7fb687000700 1 -- 172.21.15.37:0/2301584395 wait complete. 2024-03-29T08:50:18.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:18.380+0000 7fb687000700 1 Processor -- start 2024-03-29T08:50:18.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:18.381+0000 7fb687000700 1 -- start start 2024-03-29T08:50:18.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:18.381+0000 7fb687000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb680102d00 0x7fb6800702c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:50:18.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:18.381+0000 7fb687000700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb6801036a0 0x7fb680070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:50:18.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:18.381+0000 7fb687000700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb680072260 con 0x7fb680102d00 2024-03-29T08:50:18.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:18.381+0000 7fb687000700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb680070d40 con 0x7fb6801036a0 2024-03-29T08:50:18.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:18.381+0000 7fb67fe00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb680102d00 0x7fb6800702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:50:18.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:18.381+0000 7fb67fe00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb680102d00 0x7fb6800702c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60322/0 (socket says 172.21.15.37:60322) 2024-03-29T08:50:18.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:18.381+0000 7fb67fe00700 1 -- 172.21.15.37:0/67381776 learned_addr learned my addr 172.21.15.37:0/67381776 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:50:18.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:18.381+0000 7fb67f400700 1 --2- 172.21.15.37:0/67381776 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb6801036a0 0x7fb680070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:50:18.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:18.382+0000 7fb67fe00700 1 -- 172.21.15.37:0/67381776 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb6801036a0 msgr2=0x7fb680070800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:50:18.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:18.382+0000 7fb67fe00700 1 --2- 172.21.15.37:0/67381776 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb6801036a0 0x7fb680070800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:18.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:18.382+0000 7fb67fe00700 1 -- 172.21.15.37:0/67381776 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb66c005040 con 0x7fb680102d00 2024-03-29T08:50:18.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:18.382+0000 7fb67fe00700 1 --2- 172.21.15.37:0/67381776 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb680102d00 0x7fb6800702c0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7fb67000eb40 tx=0x7fb67000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:50:18.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:18.382+0000 7fb677e00700 1 -- 172.21.15.37:0/67381776 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb670007cc0 con 0x7fb680102d00 2024-03-29T08:50:18.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:18.382+0000 7fb677e00700 1 -- 172.21.15.37:0/67381776 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fb67000f040 con 0x7fb680102d00 2024-03-29T08:50:18.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:18.382+0000 7fb687000700 1 -- 172.21.15.37:0/67381776 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb680070f90 con 0x7fb680102d00 2024-03-29T08:50:18.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:18.382+0000 7fb677e00700 1 -- 172.21.15.37:0/67381776 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb670009570 con 0x7fb680102d00 2024-03-29T08:50:18.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:18.382+0000 7fb687000700 1 -- 172.21.15.37:0/67381776 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb680119e50 con 0x7fb680102d00 2024-03-29T08:50:18.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:18.383+0000 7fb687000700 1 -- 172.21.15.37:0/67381776 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6800fe450 con 0x7fb680102d00 2024-03-29T08:50:18.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:18.384+0000 7fb677e00700 1 -- 172.21.15.37:0/67381776 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fb6700096d0 con 0x7fb680102d00 2024-03-29T08:50:18.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:18.385+0000 7fb677e00700 1 --2- 172.21.15.37:0/67381776 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fb66406cc20 0x7fb66406f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:50:18.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:18.385+0000 7fb677e00700 1 -- 172.21.15.37:0/67381776 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fb67008bc80 con 0x7fb680102d00 2024-03-29T08:50:18.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:18.385+0000 7fb67f400700 1 --2- 172.21.15.37:0/67381776 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fb66406cc20 0x7fb66406f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:50:18.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:18.386+0000 7fb67f400700 1 --2- 172.21.15.37:0/67381776 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fb66406cc20 0x7fb66406f0d0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fb66c009fd0 tx=0x7fb66c009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:50:18.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:18.390+0000 7fb677e00700 1 -- 172.21.15.37:0/67381776 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fb670055cf0 con 0x7fb680102d00 2024-03-29T08:50:18.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:18.652+0000 7fb687000700 1 -- 172.21.15.37:0/67381776 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 17, "format": "json"} v 0) v1 -- 0x7fb680064970 con 0x7fb680102d00 2024-03-29T08:50:18.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:18.653+0000 7fb677e00700 1 -- 172.21.15.37:0/67381776 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 17, "format": "json"}]=0 dumped fsmap epoch 17 v36) v1 ==== 107+0+4847 (secure 0 0 0) 0x7fb670059310 con 0x7fb680102d00 2024-03-29T08:50:18.653 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:50:18.653 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24319,"name":"cephfs.smithi148.qukpjh","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.148:6827/2641327912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2641327912},{"type":"v1","addr":"172.21.15.148:6827","nonce":2641327912}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15},{"gid":24327,"name":"cephfs.smithi148.sqteih","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6825/3252351726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":3252351726},{"type":"v1","addr":"172.21.15.148:6825","nonce":3252351726}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-29T08:03:29.241269+0000","modified":"2024-03-29T08:06:08.902666+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":47,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":14470,"mds_1":14482},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14470":{"gid":14470,"name":"cephfs.smithi037.mmwgxe","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.37:6827/1559618909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1559618909},{"type":"v1","addr":"172.21.15.37:6827","nonce":1559618909}]},"join_fscid":1,"export_targets":[1],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14482":{"gid":14482,"name":"cephfs.smithi037.iifsmh","rank":1,"incarnation":12,"state":"up:active","state_seq":6,"addr":"172.21.15.37:6829/2341530495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":2341530495},{"type":"v1","addr":"172.21.15.37:6829","nonce":2341530495}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-03-29T08:50:18.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:18.655+0000 7fb687000700 1 -- 172.21.15.37:0/67381776 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fb66406cc20 msgr2=0x7fb66406f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:50:18.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:18.655+0000 7fb687000700 1 --2- 172.21.15.37:0/67381776 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fb66406cc20 0x7fb66406f0d0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fb66c009fd0 tx=0x7fb66c009e30 comp rx=0 tx=0).stop 2024-03-29T08:50:18.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:18.655+0000 7fb687000700 1 -- 172.21.15.37:0/67381776 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb680102d00 msgr2=0x7fb6800702c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:50:18.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:18.655+0000 7fb687000700 1 --2- 172.21.15.37:0/67381776 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb680102d00 0x7fb6800702c0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7fb67000eb40 tx=0x7fb67000b040 comp rx=0 tx=0).stop 2024-03-29T08:50:18.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:18.655+0000 7fb687000700 1 -- 172.21.15.37:0/67381776 shutdown_connections 2024-03-29T08:50:18.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:18.655+0000 7fb687000700 1 --2- 172.21.15.37:0/67381776 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fb66406cc20 0x7fb66406f0d0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:18.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:18.655+0000 7fb687000700 1 --2- 172.21.15.37:0/67381776 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb680102d00 0x7fb6800702c0 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:18.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:18.655+0000 7fb687000700 1 --2- 172.21.15.37:0/67381776 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb6801036a0 0x7fb680070800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:18.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:18.655+0000 7fb687000700 1 -- 172.21.15.37:0/67381776 >> 172.21.15.37:0/67381776 conn(0x7fb6800f8810 msgr2=0x7fb6800f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:50:18.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:18.655+0000 7fb687000700 1 -- 172.21.15.37:0/67381776 shutdown_connections 2024-03-29T08:50:18.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:18.655+0000 7fb687000700 1 -- 172.21.15.37:0/67381776 wait complete. 2024-03-29T08:50:18.657 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 17 2024-03-29T08:50:18.674 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 18 2024-03-29T08:50:18.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:18.869+0000 7f88d4c00700 1 -- 172.21.15.37:0/4178188502 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f88d0102d00 msgr2=0x7f88d01030d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:50:18.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:18.869+0000 7f88d4c00700 1 --2- 172.21.15.37:0/4178188502 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f88d0102d00 0x7f88d01030d0 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f88c0004940 tx=0x7f88c0009e30 comp rx=0 tx=0).stop 2024-03-29T08:50:18.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:18.869+0000 7f88d4c00700 1 -- 172.21.15.37:0/4178188502 shutdown_connections 2024-03-29T08:50:18.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:18.869+0000 7f88d4c00700 1 --2- 172.21.15.37:0/4178188502 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f88d01036a0 0x7f88d00fcd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:18.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:18.869+0000 7f88d4c00700 1 --2- 172.21.15.37:0/4178188502 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f88d0102d00 0x7f88d01030d0 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:18.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:18.869+0000 7f88d4c00700 1 -- 172.21.15.37:0/4178188502 >> 172.21.15.37:0/4178188502 conn(0x7f88d00f8810 msgr2=0x7f88d00fac20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:50:18.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:18.869+0000 7f88d4c00700 1 -- 172.21.15.37:0/4178188502 shutdown_connections 2024-03-29T08:50:18.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:18.869+0000 7f88d4c00700 1 -- 172.21.15.37:0/4178188502 wait complete. 2024-03-29T08:50:18.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:18.870+0000 7f88d4c00700 1 Processor -- start 2024-03-29T08:50:18.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:18.870+0000 7f88d4c00700 1 -- start start 2024-03-29T08:50:18.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:18.871+0000 7f88d4c00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f88d0102d00 0x7f88d00702c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:50:18.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:18.871+0000 7f88d4c00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f88d01036a0 0x7f88d0070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:50:18.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:18.871+0000 7f88d4c00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88d0072210 con 0x7f88d0102d00 2024-03-29T08:50:18.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:18.871+0000 7f88d4c00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88d0070d40 con 0x7f88d01036a0 2024-03-29T08:50:18.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:18.871+0000 7f88cde00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f88d0102d00 0x7f88d00702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:50:18.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:18.871+0000 7f88cde00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f88d0102d00 0x7f88d00702c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60342/0 (socket says 172.21.15.37:60342) 2024-03-29T08:50:18.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:18.871+0000 7f88cde00700 1 -- 172.21.15.37:0/334165809 learned_addr learned my addr 172.21.15.37:0/334165809 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:50:18.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:18.872+0000 7f88cde00700 1 -- 172.21.15.37:0/334165809 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f88d01036a0 msgr2=0x7f88d0070800 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-29T08:50:18.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:18.872+0000 7f88cde00700 1 --2- 172.21.15.37:0/334165809 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f88d01036a0 0x7f88d0070800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:18.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:18.872+0000 7f88cde00700 1 -- 172.21.15.37:0/334165809 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88c0005040 con 0x7f88d0102d00 2024-03-29T08:50:18.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:18.873+0000 7f88cde00700 1 --2- 172.21.15.37:0/334165809 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f88d0102d00 0x7f88d00702c0 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f88c0009390 tx=0x7f88c0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:50:18.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:18.875+0000 7f88c6000700 1 -- 172.21.15.37:0/334165809 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f88c000e040 con 0x7f88d0102d00 2024-03-29T08:50:18.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:18.875+0000 7f88c6000700 1 -- 172.21.15.37:0/334165809 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f88c001fe40 con 0x7f88d0102d00 2024-03-29T08:50:18.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:18.875+0000 7f88c6000700 1 -- 172.21.15.37:0/334165809 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f88c0013bc0 con 0x7f88d0102d00 2024-03-29T08:50:18.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:18.875+0000 7f88d4c00700 1 -- 172.21.15.37:0/334165809 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88d0070fc0 con 0x7f88d0102d00 2024-03-29T08:50:18.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:18.876+0000 7f88d4c00700 1 -- 172.21.15.37:0/334165809 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88d010da50 con 0x7f88d0102d00 2024-03-29T08:50:18.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:18.877+0000 7f88d4c00700 1 -- 172.21.15.37:0/334165809 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88d00fe450 con 0x7f88d0102d00 2024-03-29T08:50:18.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:18.879+0000 7f88c6000700 1 -- 172.21.15.37:0/334165809 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f88c0013420 con 0x7f88d0102d00 2024-03-29T08:50:18.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:18.880+0000 7f88c6000700 1 --2- 172.21.15.37:0/334165809 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f88b406cce0 0x7f88b406f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:50:18.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:18.880+0000 7f88c6000700 1 -- 172.21.15.37:0/334165809 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f88c0095700 con 0x7f88d0102d00 2024-03-29T08:50:18.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:18.880+0000 7f88cd400700 1 --2- 172.21.15.37:0/334165809 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f88b406cce0 0x7f88b406f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:50:18.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:18.881+0000 7f88cd400700 1 --2- 172.21.15.37:0/334165809 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f88b406cce0 0x7f88b406f190 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f88d00719f0 tx=0x7f88bc009ce0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:50:18.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:18.884+0000 7f88c6000700 1 -- 172.21.15.37:0/334165809 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f88c005f800 con 0x7f88d0102d00 2024-03-29T08:50:18.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:50:18 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/1390625059' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-29T08:50:18.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:50:18 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/67381776' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 17, "format": "json"}]: dispatch 2024-03-29T08:50:19.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:19.152+0000 7f88d4c00700 1 -- 172.21.15.37:0/334165809 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 18, "format": "json"} v 0) v1 -- 0x7f88d010dc90 con 0x7f88d0102d00 2024-03-29T08:50:19.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:19.153+0000 7f88c6000700 1 -- 172.21.15.37:0/334165809 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 18, "format": "json"}]=0 dumped fsmap epoch 18 v36) v1 ==== 107+0+4848 (secure 0 0 0) 0x7f88c0062e20 con 0x7f88d0102d00 2024-03-29T08:50:19.154 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:50:19.154 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":18,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24319,"name":"cephfs.smithi148.qukpjh","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.148:6827/2641327912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2641327912},{"type":"v1","addr":"172.21.15.148:6827","nonce":2641327912}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15},{"gid":24327,"name":"cephfs.smithi148.sqteih","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6825/3252351726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":3252351726},{"type":"v1","addr":"172.21.15.148:6825","nonce":3252351726}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-29T08:03:29.241269+0000","modified":"2024-03-29T08:07:17.340199+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":47,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":14470,"mds_1":14482},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14470":{"gid":14470,"name":"cephfs.smithi037.mmwgxe","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.37:6827/1559618909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1559618909},{"type":"v1","addr":"172.21.15.37:6827","nonce":1559618909}]},"join_fscid":1,"export_targets":[1],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14482":{"gid":14482,"name":"cephfs.smithi037.iifsmh","rank":1,"incarnation":12,"state":"up:active","state_seq":6,"addr":"172.21.15.37:6829/2341530495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":2341530495},{"type":"v1","addr":"172.21.15.37:6829","nonce":2341530495}]},"join_fscid":1,"export_targets":[0],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-03-29T08:50:19.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:19.156+0000 7f88d4c00700 1 -- 172.21.15.37:0/334165809 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f88b406cce0 msgr2=0x7f88b406f190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:50:19.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:19.156+0000 7f88d4c00700 1 --2- 172.21.15.37:0/334165809 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f88b406cce0 0x7f88b406f190 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f88d00719f0 tx=0x7f88bc009ce0 comp rx=0 tx=0).stop 2024-03-29T08:50:19.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:19.156+0000 7f88d4c00700 1 -- 172.21.15.37:0/334165809 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f88d0102d00 msgr2=0x7f88d00702c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:50:19.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:19.156+0000 7f88d4c00700 1 --2- 172.21.15.37:0/334165809 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f88d0102d00 0x7f88d00702c0 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f88c0009390 tx=0x7f88c0009e30 comp rx=0 tx=0).stop 2024-03-29T08:50:19.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:19.156+0000 7f88d4c00700 1 -- 172.21.15.37:0/334165809 shutdown_connections 2024-03-29T08:50:19.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:19.156+0000 7f88d4c00700 1 --2- 172.21.15.37:0/334165809 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f88b406cce0 0x7f88b406f190 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:19.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:19.156+0000 7f88d4c00700 1 --2- 172.21.15.37:0/334165809 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f88d0102d00 0x7f88d00702c0 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:19.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:19.156+0000 7f88d4c00700 1 --2- 172.21.15.37:0/334165809 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f88d01036a0 0x7f88d0070800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:19.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:19.156+0000 7f88d4c00700 1 -- 172.21.15.37:0/334165809 >> 172.21.15.37:0/334165809 conn(0x7f88d00f8810 msgr2=0x7f88d00f98b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:50:19.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:19.156+0000 7f88d4c00700 1 -- 172.21.15.37:0/334165809 shutdown_connections 2024-03-29T08:50:19.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:19.156+0000 7f88d4c00700 1 -- 172.21.15.37:0/334165809 wait complete. 2024-03-29T08:50:19.158 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 18 2024-03-29T08:50:19.175 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 19 2024-03-29T08:50:19.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:50:18 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/1390625059' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-29T08:50:19.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:50:18 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/67381776' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 17, "format": "json"}]: dispatch 2024-03-29T08:50:19.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:19.368+0000 7f4e46800700 1 -- 172.21.15.37:0/2873193415 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4e400fe510 msgr2=0x7f4e40106be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:50:19.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:19.368+0000 7f4e46800700 1 --2- 172.21.15.37:0/2873193415 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4e400fe510 0x7f4e40106be0 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f4e2c004940 tx=0x7f4e2c009e30 comp rx=0 tx=0).stop 2024-03-29T08:50:19.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:19.368+0000 7f4e46800700 1 -- 172.21.15.37:0/2873193415 shutdown_connections 2024-03-29T08:50:19.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:19.368+0000 7f4e46800700 1 --2- 172.21.15.37:0/2873193415 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4e400fe510 0x7f4e40106be0 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:19.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:19.368+0000 7f4e46800700 1 --2- 172.21.15.37:0/2873193415 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4e400fdb70 0x7f4e400fdf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:19.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:19.368+0000 7f4e46800700 1 -- 172.21.15.37:0/2873193415 >> 172.21.15.37:0/2873193415 conn(0x7f4e400728c0 msgr2=0x7f4e40072cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:50:19.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:19.368+0000 7f4e46800700 1 -- 172.21.15.37:0/2873193415 shutdown_connections 2024-03-29T08:50:19.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:19.368+0000 7f4e46800700 1 -- 172.21.15.37:0/2873193415 wait complete. 2024-03-29T08:50:19.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:19.369+0000 7f4e46800700 1 Processor -- start 2024-03-29T08:50:19.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:19.369+0000 7f4e46800700 1 -- start start 2024-03-29T08:50:19.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:19.370+0000 7f4e46800700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4e400fdb70 0x7f4e400fba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:50:19.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:19.370+0000 7f4e46800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4e400fe510 0x7f4e400fbf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:50:19.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:19.370+0000 7f4e46800700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e400fd980 con 0x7f4e400fe510 2024-03-29T08:50:19.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:19.370+0000 7f4e46800700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e400fc4b0 con 0x7f4e400fdb70 2024-03-29T08:50:19.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:19.370+0000 7f4e44a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4e400fe510 0x7f4e400fbf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:50:19.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:19.370+0000 7f4e44a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4e400fe510 0x7f4e400fbf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60358/0 (socket says 172.21.15.37:60358) 2024-03-29T08:50:19.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:19.370+0000 7f4e44a00700 1 -- 172.21.15.37:0/1676079980 learned_addr learned my addr 172.21.15.37:0/1676079980 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:50:19.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:19.370+0000 7f4e45400700 1 --2- 172.21.15.37:0/1676079980 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4e400fdb70 0x7f4e400fba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:50:19.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:19.370+0000 7f4e44a00700 1 -- 172.21.15.37:0/1676079980 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4e400fdb70 msgr2=0x7f4e400fba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:50:19.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:19.370+0000 7f4e44a00700 1 --2- 172.21.15.37:0/1676079980 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4e400fdb70 0x7f4e400fba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:19.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:19.370+0000 7f4e44a00700 1 -- 172.21.15.37:0/1676079980 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e2c005040 con 0x7f4e400fe510 2024-03-29T08:50:19.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:19.370+0000 7f4e44a00700 1 --2- 172.21.15.37:0/1676079980 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4e400fe510 0x7f4e400fbf70 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f4e2c004190 tx=0x7f4e2c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:50:19.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:19.371+0000 7f4e3d600700 1 -- 172.21.15.37:0/1676079980 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4e2c021070 con 0x7f4e400fe510 2024-03-29T08:50:19.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:19.371+0000 7f4e3d600700 1 -- 172.21.15.37:0/1676079980 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f4e2c004290 con 0x7f4e400fe510 2024-03-29T08:50:19.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:19.371+0000 7f4e46800700 1 -- 172.21.15.37:0/1676079980 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e400fc730 con 0x7f4e400fe510 2024-03-29T08:50:19.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:19.371+0000 7f4e3d600700 1 -- 172.21.15.37:0/1676079980 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4e2c00e070 con 0x7f4e400fe510 2024-03-29T08:50:19.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:19.371+0000 7f4e46800700 1 -- 172.21.15.37:0/1676079980 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e40119cd0 con 0x7f4e400fe510 2024-03-29T08:50:19.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:19.372+0000 7f4e46800700 1 -- 172.21.15.37:0/1676079980 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e400655e0 con 0x7f4e400fe510 2024-03-29T08:50:19.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:19.373+0000 7f4e3d600700 1 -- 172.21.15.37:0/1676079980 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f4e2c01c500 con 0x7f4e400fe510 2024-03-29T08:50:19.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:19.374+0000 7f4e3d600700 1 --2- 172.21.15.37:0/1676079980 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f4e2806cc30 0x7f4e2806f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:50:19.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:19.374+0000 7f4e3d600700 1 -- 172.21.15.37:0/1676079980 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f4e2c08d150 con 0x7f4e400fe510 2024-03-29T08:50:19.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:19.374+0000 7f4e45400700 1 --2- 172.21.15.37:0/1676079980 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f4e2806cc30 0x7f4e2806f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:50:19.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:19.374+0000 7f4e45400700 1 --2- 172.21.15.37:0/1676079980 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f4e2806cc30 0x7f4e2806f0e0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f4e34004600 tx=0x7f4e3400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:50:19.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:19.378+0000 7f4e3d600700 1 -- 172.21.15.37:0/1676079980 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f4e2c057190 con 0x7f4e400fe510 2024-03-29T08:50:19.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:19.641+0000 7f4e46800700 1 -- 172.21.15.37:0/1676079980 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 19, "format": "json"} v 0) v1 -- 0x7f4e4004c710 con 0x7f4e400fe510 2024-03-29T08:50:19.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:19.642+0000 7f4e3d600700 1 -- 172.21.15.37:0/1676079980 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 19, "format": "json"}]=0 dumped fsmap epoch 19 v36) v1 ==== 107+0+4847 (secure 0 0 0) 0x7f4e2c05a7b0 con 0x7f4e400fe510 2024-03-29T08:50:19.643 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:50:19.643 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":19,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24319,"name":"cephfs.smithi148.qukpjh","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.148:6827/2641327912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2641327912},{"type":"v1","addr":"172.21.15.148:6827","nonce":2641327912}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15},{"gid":24327,"name":"cephfs.smithi148.sqteih","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6825/3252351726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":3252351726},{"type":"v1","addr":"172.21.15.148:6825","nonce":3252351726}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":19,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-29T08:03:29.241269+0000","modified":"2024-03-29T08:11:04.110925+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":47,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":14470,"mds_1":14482},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14470":{"gid":14470,"name":"cephfs.smithi037.mmwgxe","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.37:6827/1559618909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1559618909},{"type":"v1","addr":"172.21.15.37:6827","nonce":1559618909}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14482":{"gid":14482,"name":"cephfs.smithi037.iifsmh","rank":1,"incarnation":12,"state":"up:active","state_seq":6,"addr":"172.21.15.37:6829/2341530495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":2341530495},{"type":"v1","addr":"172.21.15.37:6829","nonce":2341530495}]},"join_fscid":1,"export_targets":[0],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-03-29T08:50:19.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:19.644+0000 7f4e46800700 1 -- 172.21.15.37:0/1676079980 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f4e2806cc30 msgr2=0x7f4e2806f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:50:19.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:19.644+0000 7f4e46800700 1 --2- 172.21.15.37:0/1676079980 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f4e2806cc30 0x7f4e2806f0e0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f4e34004600 tx=0x7f4e3400b040 comp rx=0 tx=0).stop 2024-03-29T08:50:19.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:19.644+0000 7f4e46800700 1 -- 172.21.15.37:0/1676079980 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4e400fe510 msgr2=0x7f4e400fbf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:50:19.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:19.644+0000 7f4e46800700 1 --2- 172.21.15.37:0/1676079980 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4e400fe510 0x7f4e400fbf70 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f4e2c004190 tx=0x7f4e2c009e30 comp rx=0 tx=0).stop 2024-03-29T08:50:19.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:19.644+0000 7f4e46800700 1 -- 172.21.15.37:0/1676079980 shutdown_connections 2024-03-29T08:50:19.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:19.644+0000 7f4e46800700 1 --2- 172.21.15.37:0/1676079980 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f4e2806cc30 0x7f4e2806f0e0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:19.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:19.644+0000 7f4e46800700 1 --2- 172.21.15.37:0/1676079980 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4e400fdb70 0x7f4e400fba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:19.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:19.644+0000 7f4e46800700 1 --2- 172.21.15.37:0/1676079980 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4e400fe510 0x7f4e400fbf70 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:19.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:19.644+0000 7f4e46800700 1 -- 172.21.15.37:0/1676079980 >> 172.21.15.37:0/1676079980 conn(0x7f4e400728c0 msgr2=0x7f4e400fa380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:50:19.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:19.644+0000 7f4e46800700 1 -- 172.21.15.37:0/1676079980 shutdown_connections 2024-03-29T08:50:19.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:19.645+0000 7f4e46800700 1 -- 172.21.15.37:0/1676079980 wait complete. 2024-03-29T08:50:19.646 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 19 2024-03-29T08:50:19.664 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 20 2024-03-29T08:50:19.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:19.860+0000 7fd480a00700 1 -- 172.21.15.37:0/1356844335 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd47c103670 msgr2=0x7fd47c0fcce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:50:19.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:19.860+0000 7fd480a00700 1 --2- 172.21.15.37:0/1356844335 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd47c103670 0x7fd47c0fcce0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7fd468004940 tx=0x7fd468009e30 comp rx=0 tx=0).stop 2024-03-29T08:50:19.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:19.860+0000 7fd480a00700 1 -- 172.21.15.37:0/1356844335 shutdown_connections 2024-03-29T08:50:19.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:19.860+0000 7fd480a00700 1 --2- 172.21.15.37:0/1356844335 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd47c103670 0x7fd47c0fcce0 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:19.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:19.860+0000 7fd480a00700 1 --2- 172.21.15.37:0/1356844335 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd47c102cd0 0x7fd47c1030a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:19.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:19.860+0000 7fd480a00700 1 -- 172.21.15.37:0/1356844335 >> 172.21.15.37:0/1356844335 conn(0x7fd47c0f8820 msgr2=0x7fd47c0fac30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:50:19.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:19.860+0000 7fd480a00700 1 -- 172.21.15.37:0/1356844335 shutdown_connections 2024-03-29T08:50:19.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:19.860+0000 7fd480a00700 1 -- 172.21.15.37:0/1356844335 wait complete. 2024-03-29T08:50:19.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:19.861+0000 7fd480a00700 1 Processor -- start 2024-03-29T08:50:19.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:19.861+0000 7fd480a00700 1 -- start start 2024-03-29T08:50:19.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:19.861+0000 7fd480a00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd47c102cd0 0x7fd47c115a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:50:19.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:19.861+0000 7fd480a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd47c103670 0x7fd47c115f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:50:19.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:19.861+0000 7fd480a00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd47c1165a0 con 0x7fd47c103670 2024-03-29T08:50:19.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:19.861+0000 7fd480a00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd47c10fba0 con 0x7fd47c102cd0 2024-03-29T08:50:19.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:19.861+0000 7fd479400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd47c103670 0x7fd47c115f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:50:19.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:19.861+0000 7fd479400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd47c103670 0x7fd47c115f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60378/0 (socket says 172.21.15.37:60378) 2024-03-29T08:50:19.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:19.861+0000 7fd479400700 1 -- 172.21.15.37:0/2920851316 learned_addr learned my addr 172.21.15.37:0/2920851316 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:50:19.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:19.862+0000 7fd479e00700 1 --2- 172.21.15.37:0/2920851316 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd47c102cd0 0x7fd47c115a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:50:19.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:19.862+0000 7fd479400700 1 -- 172.21.15.37:0/2920851316 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd47c102cd0 msgr2=0x7fd47c115a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:50:19.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:19.862+0000 7fd479400700 1 --2- 172.21.15.37:0/2920851316 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd47c102cd0 0x7fd47c115a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:19.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:19.862+0000 7fd479400700 1 -- 172.21.15.37:0/2920851316 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd468005040 con 0x7fd47c103670 2024-03-29T08:50:19.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:19.862+0000 7fd479e00700 1 --2- 172.21.15.37:0/2920851316 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd47c102cd0 0x7fd47c115a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:50:19.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:19.862+0000 7fd479400700 1 --2- 172.21.15.37:0/2920851316 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd47c103670 0x7fd47c115f50 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7fd468009390 tx=0x7fd468009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:50:19.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:19.862+0000 7fd472000700 1 -- 172.21.15.37:0/2920851316 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd46800e040 con 0x7fd47c103670 2024-03-29T08:50:19.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:19.862+0000 7fd472000700 1 -- 172.21.15.37:0/2920851316 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd468013420 con 0x7fd47c103670 2024-03-29T08:50:19.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:19.862+0000 7fd480a00700 1 -- 172.21.15.37:0/2920851316 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd47c10fe20 con 0x7fd47c103670 2024-03-29T08:50:19.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:19.862+0000 7fd472000700 1 -- 172.21.15.37:0/2920851316 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd46801c630 con 0x7fd47c103670 2024-03-29T08:50:19.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:19.863+0000 7fd480a00700 1 -- 172.21.15.37:0/2920851316 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd47c110360 con 0x7fd47c103670 2024-03-29T08:50:19.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:19.864+0000 7fd480a00700 1 -- 172.21.15.37:0/2920851316 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd47c0fe420 con 0x7fd47c103670 2024-03-29T08:50:19.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:19.864+0000 7fd472000700 1 -- 172.21.15.37:0/2920851316 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fd468013a00 con 0x7fd47c103670 2024-03-29T08:50:19.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:19.865+0000 7fd472000700 1 --2- 172.21.15.37:0/2920851316 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fd46006cc20 0x7fd46006f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:50:19.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:19.865+0000 7fd472000700 1 -- 172.21.15.37:0/2920851316 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fd46808d390 con 0x7fd47c103670 2024-03-29T08:50:19.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:19.866+0000 7fd479e00700 1 --2- 172.21.15.37:0/2920851316 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fd46006cc20 0x7fd46006f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:50:19.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:19.866+0000 7fd479e00700 1 --2- 172.21.15.37:0/2920851316 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fd46006cc20 0x7fd46006f0d0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fd46c009fd0 tx=0x7fd46c00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:50:19.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:19.870+0000 7fd472000700 1 -- 172.21.15.37:0/2920851316 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fd46805ade0 con 0x7fd47c103670 2024-03-29T08:50:19.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:50:19 smithi037 ceph-mon[142325]: pgmap v1185: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:50:19.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:50:19 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/334165809' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 18, "format": "json"}]: dispatch 2024-03-29T08:50:19.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:50:19 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/1676079980' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 19, "format": "json"}]: dispatch 2024-03-29T08:50:20.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:20.134+0000 7fd480a00700 1 -- 172.21.15.37:0/2920851316 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 20, "format": "json"} v 0) v1 -- 0x7fd47c109c70 con 0x7fd47c103670 2024-03-29T08:50:20.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:20.135+0000 7fd472000700 1 -- 172.21.15.37:0/2920851316 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 20, "format": "json"}]=0 dumped fsmap epoch 20 v36) v1 ==== 107+0+4846 (secure 0 0 0) 0x7fd46805a970 con 0x7fd47c103670 2024-03-29T08:50:20.136 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:50:20.136 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24319,"name":"cephfs.smithi148.qukpjh","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.148:6827/2641327912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2641327912},{"type":"v1","addr":"172.21.15.148:6827","nonce":2641327912}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15},{"gid":24327,"name":"cephfs.smithi148.sqteih","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6825/3252351726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":3252351726},{"type":"v1","addr":"172.21.15.148:6825","nonce":3252351726}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":20,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-29T08:03:29.241269+0000","modified":"2024-03-29T08:11:27.181622+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":47,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":14470,"mds_1":14482},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14470":{"gid":14470,"name":"cephfs.smithi037.mmwgxe","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.37:6827/1559618909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1559618909},{"type":"v1","addr":"172.21.15.37:6827","nonce":1559618909}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14482":{"gid":14482,"name":"cephfs.smithi037.iifsmh","rank":1,"incarnation":12,"state":"up:active","state_seq":6,"addr":"172.21.15.37:6829/2341530495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":2341530495},{"type":"v1","addr":"172.21.15.37:6829","nonce":2341530495}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-03-29T08:50:20.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:20.138+0000 7fd480a00700 1 -- 172.21.15.37:0/2920851316 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fd46006cc20 msgr2=0x7fd46006f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:50:20.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:20.138+0000 7fd480a00700 1 --2- 172.21.15.37:0/2920851316 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fd46006cc20 0x7fd46006f0d0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fd46c009fd0 tx=0x7fd46c00b040 comp rx=0 tx=0).stop 2024-03-29T08:50:20.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:20.138+0000 7fd480a00700 1 -- 172.21.15.37:0/2920851316 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd47c103670 msgr2=0x7fd47c115f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:50:20.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:20.138+0000 7fd480a00700 1 --2- 172.21.15.37:0/2920851316 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd47c103670 0x7fd47c115f50 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7fd468009390 tx=0x7fd468009e30 comp rx=0 tx=0).stop 2024-03-29T08:50:20.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:20.138+0000 7fd480a00700 1 -- 172.21.15.37:0/2920851316 shutdown_connections 2024-03-29T08:50:20.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:20.138+0000 7fd480a00700 1 --2- 172.21.15.37:0/2920851316 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fd46006cc20 0x7fd46006f0d0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:20.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:20.138+0000 7fd480a00700 1 --2- 172.21.15.37:0/2920851316 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd47c102cd0 0x7fd47c115a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:20.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:20.138+0000 7fd480a00700 1 --2- 172.21.15.37:0/2920851316 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd47c103670 0x7fd47c115f50 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:20.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:20.138+0000 7fd480a00700 1 -- 172.21.15.37:0/2920851316 >> 172.21.15.37:0/2920851316 conn(0x7fd47c0f8820 msgr2=0x7fd47c0f9880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:50:20.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:20.138+0000 7fd480a00700 1 -- 172.21.15.37:0/2920851316 shutdown_connections 2024-03-29T08:50:20.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:20.138+0000 7fd480a00700 1 -- 172.21.15.37:0/2920851316 wait complete. 2024-03-29T08:50:20.141 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 20 2024-03-29T08:50:20.157 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 21 2024-03-29T08:50:20.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:50:19 smithi148 ceph-mon[106692]: pgmap v1185: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:50:20.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:50:19 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/334165809' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 18, "format": "json"}]: dispatch 2024-03-29T08:50:20.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:50:19 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/1676079980' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 19, "format": "json"}]: dispatch 2024-03-29T08:50:20.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:20.351+0000 7efe1c000700 1 -- 172.21.15.37:0/3446838196 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efe14103670 msgr2=0x7efe140fcce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:50:20.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:20.351+0000 7efe1c000700 1 --2- 172.21.15.37:0/3446838196 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efe14103670 0x7efe140fcce0 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7efe00004940 tx=0x7efe00009e30 comp rx=0 tx=0).stop 2024-03-29T08:50:20.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:20.351+0000 7efe1c000700 1 -- 172.21.15.37:0/3446838196 shutdown_connections 2024-03-29T08:50:20.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:20.351+0000 7efe1c000700 1 --2- 172.21.15.37:0/3446838196 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efe14103670 0x7efe140fcce0 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:20.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:20.351+0000 7efe1c000700 1 --2- 172.21.15.37:0/3446838196 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe14102cd0 0x7efe141030a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:20.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:20.351+0000 7efe1c000700 1 -- 172.21.15.37:0/3446838196 >> 172.21.15.37:0/3446838196 conn(0x7efe140f8820 msgr2=0x7efe140fac30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:50:20.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:20.351+0000 7efe1c000700 1 -- 172.21.15.37:0/3446838196 shutdown_connections 2024-03-29T08:50:20.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:20.351+0000 7efe1c000700 1 -- 172.21.15.37:0/3446838196 wait complete. 2024-03-29T08:50:20.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:20.352+0000 7efe1c000700 1 Processor -- start 2024-03-29T08:50:20.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:20.352+0000 7efe1c000700 1 -- start start 2024-03-29T08:50:20.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:20.352+0000 7efe1c000700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe14102cd0 0x7efe141136e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:50:20.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:20.352+0000 7efe1c000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efe14103670 0x7efe14113c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:50:20.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:20.352+0000 7efe1c000700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe14114270 con 0x7efe14103670 2024-03-29T08:50:20.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:20.352+0000 7efe1c000700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe1410d890 con 0x7efe14102cd0 2024-03-29T08:50:20.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:20.352+0000 7efe18c00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efe14103670 0x7efe14113c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:50:20.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:20.352+0000 7efe18c00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efe14103670 0x7efe14113c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52664/0 (socket says 172.21.15.37:52664) 2024-03-29T08:50:20.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:20.352+0000 7efe18c00700 1 -- 172.21.15.37:0/3335132164 learned_addr learned my addr 172.21.15.37:0/3335132164 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:50:20.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:20.353+0000 7efe19600700 1 --2- 172.21.15.37:0/3335132164 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe14102cd0 0x7efe141136e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:50:20.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:20.353+0000 7efe18c00700 1 -- 172.21.15.37:0/3335132164 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe14102cd0 msgr2=0x7efe141136e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:50:20.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:20.353+0000 7efe18c00700 1 --2- 172.21.15.37:0/3335132164 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe14102cd0 0x7efe141136e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:20.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:20.353+0000 7efe18c00700 1 -- 172.21.15.37:0/3335132164 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe00005040 con 0x7efe14103670 2024-03-29T08:50:20.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:20.353+0000 7efe18c00700 1 --2- 172.21.15.37:0/3335132164 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efe14103670 0x7efe14113c20 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7efe00008d50 tx=0x7efe00009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:50:20.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:20.353+0000 7efe11600700 1 -- 172.21.15.37:0/3335132164 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efe0000e030 con 0x7efe14103670 2024-03-29T08:50:20.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:20.353+0000 7efe11600700 1 -- 172.21.15.37:0/3335132164 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7efe00013de0 con 0x7efe14103670 2024-03-29T08:50:20.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:20.353+0000 7efe1c000700 1 -- 172.21.15.37:0/3335132164 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe1410db10 con 0x7efe14103670 2024-03-29T08:50:20.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:20.353+0000 7efe11600700 1 -- 172.21.15.37:0/3335132164 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efe0001cd90 con 0x7efe14103670 2024-03-29T08:50:20.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:20.353+0000 7efe1c000700 1 -- 172.21.15.37:0/3335132164 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe1410e0d0 con 0x7efe14103670 2024-03-29T08:50:20.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:20.354+0000 7efe1c000700 1 -- 172.21.15.37:0/3335132164 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe140fe420 con 0x7efe14103670 2024-03-29T08:50:20.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:20.355+0000 7efe11600700 1 -- 172.21.15.37:0/3335132164 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7efe00013930 con 0x7efe14103670 2024-03-29T08:50:20.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:20.357+0000 7efe11600700 1 --2- 172.21.15.37:0/3335132164 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7efdfc06cc20 0x7efdfc06f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:50:20.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:20.357+0000 7efe11600700 1 -- 172.21.15.37:0/3335132164 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7efe0008d8c0 con 0x7efe14103670 2024-03-29T08:50:20.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:20.357+0000 7efe19600700 1 --2- 172.21.15.37:0/3335132164 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7efdfc06cc20 0x7efdfc06f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:50:20.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:20.358+0000 7efe19600700 1 --2- 172.21.15.37:0/3335132164 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7efdfc06cc20 0x7efdfc06f0d0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7efe08004600 tx=0x7efe0800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:50:20.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:20.361+0000 7efe11600700 1 -- 172.21.15.37:0/3335132164 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7efe0005b390 con 0x7efe14103670 2024-03-29T08:50:20.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:20.624+0000 7efe1c000700 1 -- 172.21.15.37:0/3335132164 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 21, "format": "json"} v 0) v1 -- 0x7efe14109c70 con 0x7efe14103670 2024-03-29T08:50:20.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:20.624+0000 7efe11600700 1 -- 172.21.15.37:0/3335132164 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 21, "format": "json"}]=0 dumped fsmap epoch 21 v36) v1 ==== 107+0+3267 (secure 0 0 0) 0x7efe0005af20 con 0x7efe14103670 2024-03-29T08:50:20.625 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:50:20.625 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24319,"name":"cephfs.smithi148.qukpjh","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.148:6827/2641327912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2641327912},{"type":"v1","addr":"172.21.15.148:6827","nonce":2641327912}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15},{"gid":24327,"name":"cephfs.smithi148.sqteih","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6825/3252351726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":3252351726},{"type":"v1","addr":"172.21.15.148:6825","nonce":3252351726}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":21,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-29T08:03:29.241269+0000","modified":"2024-03-29T08:17:16.237238+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":82,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-03-29T08:50:20.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:20.627+0000 7efe1c000700 1 -- 172.21.15.37:0/3335132164 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7efdfc06cc20 msgr2=0x7efdfc06f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:50:20.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:20.627+0000 7efe1c000700 1 --2- 172.21.15.37:0/3335132164 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7efdfc06cc20 0x7efdfc06f0d0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7efe08004600 tx=0x7efe0800b040 comp rx=0 tx=0).stop 2024-03-29T08:50:20.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:20.627+0000 7efe1c000700 1 -- 172.21.15.37:0/3335132164 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efe14103670 msgr2=0x7efe14113c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:50:20.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:20.627+0000 7efe1c000700 1 --2- 172.21.15.37:0/3335132164 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efe14103670 0x7efe14113c20 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7efe00008d50 tx=0x7efe00009e30 comp rx=0 tx=0).stop 2024-03-29T08:50:20.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:20.627+0000 7efe1c000700 1 -- 172.21.15.37:0/3335132164 shutdown_connections 2024-03-29T08:50:20.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:20.627+0000 7efe1c000700 1 --2- 172.21.15.37:0/3335132164 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7efdfc06cc20 0x7efdfc06f0d0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:20.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:20.627+0000 7efe1c000700 1 --2- 172.21.15.37:0/3335132164 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe14102cd0 0x7efe141136e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:20.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:20.627+0000 7efe1c000700 1 --2- 172.21.15.37:0/3335132164 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efe14103670 0x7efe14113c20 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:20.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:20.627+0000 7efe1c000700 1 -- 172.21.15.37:0/3335132164 >> 172.21.15.37:0/3335132164 conn(0x7efe140f8820 msgr2=0x7efe140f9890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:50:20.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:20.627+0000 7efe1c000700 1 -- 172.21.15.37:0/3335132164 shutdown_connections 2024-03-29T08:50:20.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:20.627+0000 7efe1c000700 1 -- 172.21.15.37:0/3335132164 wait complete. 2024-03-29T08:50:20.628 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 21 2024-03-29T08:50:20.645 DEBUG:tasks.fs:max_mds not reduced in epoch 21 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-03-29T08:50:20.645 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 22 2024-03-29T08:50:20.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:20.839+0000 7f088e400700 1 -- 172.21.15.37:0/1989489129 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0888070300 msgr2=0x7f08880706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:50:20.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:20.839+0000 7f088e400700 1 --2- 172.21.15.37:0/1989489129 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0888070300 0x7f08880706d0 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f0878004970 tx=0x7f0878009e30 comp rx=0 tx=0).stop 2024-03-29T08:50:20.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:20.839+0000 7f088e400700 1 -- 172.21.15.37:0/1989489129 shutdown_connections 2024-03-29T08:50:20.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:20.839+0000 7f088e400700 1 --2- 172.21.15.37:0/1989489129 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0888070c10 0x7f0888109050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:20.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:20.839+0000 7f088e400700 1 --2- 172.21.15.37:0/1989489129 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0888070300 0x7f08880706d0 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:20.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:20.839+0000 7f088e400700 1 -- 172.21.15.37:0/1989489129 >> 172.21.15.37:0/1989489129 conn(0x7f08880f8830 msgr2=0x7f08880fac40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:50:20.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:20.839+0000 7f088e400700 1 -- 172.21.15.37:0/1989489129 shutdown_connections 2024-03-29T08:50:20.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:20.839+0000 7f088e400700 1 -- 172.21.15.37:0/1989489129 wait complete. 2024-03-29T08:50:20.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:20.840+0000 7f088e400700 1 Processor -- start 2024-03-29T08:50:20.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:20.840+0000 7f088e400700 1 -- start start 2024-03-29T08:50:20.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:20.840+0000 7f088e400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0888070300 0x7f0888115980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:50:20.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:20.840+0000 7f088e400700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0888070c10 0x7f0888115ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:50:20.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:20.840+0000 7f088e400700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0888116510 con 0x7f0888070300 2024-03-29T08:50:20.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:20.840+0000 7f088e400700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f088810fb10 con 0x7f0888070c10 2024-03-29T08:50:20.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:20.841+0000 7f0887400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0888070300 0x7f0888115980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:50:20.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:20.841+0000 7f0887400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0888070300 0x7f0888115980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52690/0 (socket says 172.21.15.37:52690) 2024-03-29T08:50:20.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:20.841+0000 7f0887400700 1 -- 172.21.15.37:0/2265047764 learned_addr learned my addr 172.21.15.37:0/2265047764 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:50:20.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:20.841+0000 7f0886a00700 1 --2- 172.21.15.37:0/2265047764 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0888070c10 0x7f0888115ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:50:20.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:20.841+0000 7f0887400700 1 -- 172.21.15.37:0/2265047764 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0888070c10 msgr2=0x7f0888115ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:50:20.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:20.841+0000 7f0887400700 1 --2- 172.21.15.37:0/2265047764 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0888070c10 0x7f0888115ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:20.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:20.841+0000 7f0887400700 1 -- 172.21.15.37:0/2265047764 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0878005040 con 0x7f0888070300 2024-03-29T08:50:20.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:20.841+0000 7f0887400700 1 --2- 172.21.15.37:0/2265047764 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0888070300 0x7f0888115980 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f0878009360 tx=0x7f0878009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:50:20.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:20.841+0000 7f087f400700 1 -- 172.21.15.37:0/2265047764 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f087800e040 con 0x7f0888070300 2024-03-29T08:50:20.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:20.842+0000 7f088e400700 1 -- 172.21.15.37:0/2265047764 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f088810fd90 con 0x7f0888070300 2024-03-29T08:50:20.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:20.842+0000 7f087f400700 1 -- 172.21.15.37:0/2265047764 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f0878004290 con 0x7f0888070300 2024-03-29T08:50:20.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:20.842+0000 7f087f400700 1 -- 172.21.15.37:0/2265047764 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f087801c3f0 con 0x7f0888070300 2024-03-29T08:50:20.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:20.842+0000 7f088e400700 1 -- 172.21.15.37:0/2265047764 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08881102d0 con 0x7f0888070300 2024-03-29T08:50:20.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:20.843+0000 7f088e400700 1 -- 172.21.15.37:0/2265047764 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0888071a40 con 0x7f0888070300 2024-03-29T08:50:20.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:20.844+0000 7f087f400700 1 -- 172.21.15.37:0/2265047764 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f0878013420 con 0x7f0888070300 2024-03-29T08:50:20.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:20.845+0000 7f087f400700 1 --2- 172.21.15.37:0/2265047764 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f086c06cc20 0x7f086c06f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:50:20.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:20.845+0000 7f087f400700 1 -- 172.21.15.37:0/2265047764 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f087808d450 con 0x7f0888070300 2024-03-29T08:50:20.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:20.845+0000 7f0886a00700 1 --2- 172.21.15.37:0/2265047764 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f086c06cc20 0x7f086c06f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:50:20.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:20.846+0000 7f0886a00700 1 --2- 172.21.15.37:0/2265047764 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f086c06cc20 0x7f086c06f0d0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f0874007fd0 tx=0x7f0874009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:50:20.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:20.849+0000 7f087f400700 1 -- 172.21.15.37:0/2265047764 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f087805afd0 con 0x7f0888070300 2024-03-29T08:50:20.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:50:20 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/2920851316' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 20, "format": "json"}]: dispatch 2024-03-29T08:50:20.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:50:20 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/3335132164' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 21, "format": "json"}]: dispatch 2024-03-29T08:50:21.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:21.113+0000 7f088e400700 1 -- 172.21.15.37:0/2265047764 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 22, "format": "json"} v 0) v1 -- 0x7f088810be90 con 0x7f0888070300 2024-03-29T08:50:21.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:21.113+0000 7f087f400700 1 -- 172.21.15.37:0/2265047764 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 22, "format": "json"}]=0 dumped fsmap epoch 22 v36) v1 ==== 107+0+4825 (secure 0 0 0) 0x7f087805ab60 con 0x7f0888070300 2024-03-29T08:50:21.115 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:50:21.115 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24319,"name":"cephfs.smithi148.qukpjh","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.148:6827/2641327912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2641327912},{"type":"v1","addr":"172.21.15.148:6827","nonce":2641327912}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15},{"gid":24327,"name":"cephfs.smithi148.sqteih","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6825/3252351726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":3252351726},{"type":"v1","addr":"172.21.15.148:6825","nonce":3252351726}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13},{"gid":34406,"name":"cephfs.smithi037.iifsmh","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6827/3877453059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":3877453059},{"type":"v1","addr":"172.21.15.37:6827","nonce":3877453059}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":22},{"gid":44335,"name":"cephfs.smithi037.mmwgxe","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6829/2521422502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":2521422502},{"type":"v1","addr":"172.21.15.37:6829","nonce":2521422502}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":21,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-29T08:03:29.241269+0000","modified":"2024-03-29T08:17:16.237238+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":82,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-03-29T08:50:21.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:21.116+0000 7f088e400700 1 -- 172.21.15.37:0/2265047764 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f086c06cc20 msgr2=0x7f086c06f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:50:21.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:21.116+0000 7f088e400700 1 --2- 172.21.15.37:0/2265047764 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f086c06cc20 0x7f086c06f0d0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f0874007fd0 tx=0x7f0874009040 comp rx=0 tx=0).stop 2024-03-29T08:50:21.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:21.116+0000 7f088e400700 1 -- 172.21.15.37:0/2265047764 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0888070300 msgr2=0x7f0888115980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:50:21.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:21.116+0000 7f088e400700 1 --2- 172.21.15.37:0/2265047764 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0888070300 0x7f0888115980 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f0878009360 tx=0x7f0878009e30 comp rx=0 tx=0).stop 2024-03-29T08:50:21.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:21.116+0000 7f088e400700 1 -- 172.21.15.37:0/2265047764 shutdown_connections 2024-03-29T08:50:21.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:21.116+0000 7f088e400700 1 --2- 172.21.15.37:0/2265047764 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f086c06cc20 0x7f086c06f0d0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:21.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:21.116+0000 7f088e400700 1 --2- 172.21.15.37:0/2265047764 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0888070300 0x7f0888115980 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:21.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:21.116+0000 7f088e400700 1 --2- 172.21.15.37:0/2265047764 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0888070c10 0x7f0888115ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:21.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:21.116+0000 7f088e400700 1 -- 172.21.15.37:0/2265047764 >> 172.21.15.37:0/2265047764 conn(0x7f08880f8830 msgr2=0x7f08880f9d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:50:21.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:21.116+0000 7f088e400700 1 -- 172.21.15.37:0/2265047764 shutdown_connections 2024-03-29T08:50:21.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:21.116+0000 7f088e400700 1 -- 172.21.15.37:0/2265047764 wait complete. 2024-03-29T08:50:21.118 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 22 2024-03-29T08:50:21.134 DEBUG:tasks.fs:max_mds not reduced in epoch 22 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-03-29T08:50:21.135 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 23 2024-03-29T08:50:21.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:50:20 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/2920851316' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 20, "format": "json"}]: dispatch 2024-03-29T08:50:21.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:50:20 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/3335132164' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 21, "format": "json"}]: dispatch 2024-03-29T08:50:21.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:21.329+0000 7f3515000700 1 -- 172.21.15.37:0/1406883572 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3510070360 msgr2=0x7f3510070730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:50:21.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:21.329+0000 7f3515000700 1 --2- 172.21.15.37:0/1406883572 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3510070360 0x7f3510070730 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f3500004970 tx=0x7f3500009e30 comp rx=0 tx=0).stop 2024-03-29T08:50:21.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:21.329+0000 7f3515000700 1 -- 172.21.15.37:0/1406883572 shutdown_connections 2024-03-29T08:50:21.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:21.329+0000 7f3515000700 1 --2- 172.21.15.37:0/1406883572 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3510070c70 0x7f3510108ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:21.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:21.329+0000 7f3515000700 1 --2- 172.21.15.37:0/1406883572 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3510070360 0x7f3510070730 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:21.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:21.329+0000 7f3515000700 1 -- 172.21.15.37:0/1406883572 >> 172.21.15.37:0/1406883572 conn(0x7f35100f8830 msgr2=0x7f35100fac40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:50:21.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:21.329+0000 7f3515000700 1 -- 172.21.15.37:0/1406883572 shutdown_connections 2024-03-29T08:50:21.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:21.329+0000 7f3515000700 1 -- 172.21.15.37:0/1406883572 wait complete. 2024-03-29T08:50:21.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:21.330+0000 7f3515000700 1 Processor -- start 2024-03-29T08:50:21.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:21.330+0000 7f3515000700 1 -- start start 2024-03-29T08:50:21.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:21.330+0000 7f3515000700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3510070360 0x7f3510115950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:50:21.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:21.330+0000 7f3515000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3510070c70 0x7f3510115e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:50:21.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:21.330+0000 7f3515000700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35101164e0 con 0x7f3510070c70 2024-03-29T08:50:21.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:21.330+0000 7f3515000700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f351010fae0 con 0x7f3510070360 2024-03-29T08:50:21.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:21.331+0000 7f350d400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3510070c70 0x7f3510115e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:50:21.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:21.331+0000 7f350d400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3510070c70 0x7f3510115e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52712/0 (socket says 172.21.15.37:52712) 2024-03-29T08:50:21.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:21.331+0000 7f350d400700 1 -- 172.21.15.37:0/1909306330 learned_addr learned my addr 172.21.15.37:0/1909306330 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:50:21.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:21.331+0000 7f350de00700 1 --2- 172.21.15.37:0/1909306330 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3510070360 0x7f3510115950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:50:21.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:21.331+0000 7f350d400700 1 -- 172.21.15.37:0/1909306330 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3510070360 msgr2=0x7f3510115950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:50:21.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:21.331+0000 7f350d400700 1 --2- 172.21.15.37:0/1909306330 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3510070360 0x7f3510115950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:21.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:21.331+0000 7f350d400700 1 -- 172.21.15.37:0/1909306330 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3500005040 con 0x7f3510070c70 2024-03-29T08:50:21.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:21.331+0000 7f350d400700 1 --2- 172.21.15.37:0/1909306330 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3510070c70 0x7f3510115e90 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f34fc002a50 tx=0x7f34fc0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:50:21.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:21.332+0000 7f3506000700 1 -- 172.21.15.37:0/1909306330 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f34fc007cc0 con 0x7f3510070c70 2024-03-29T08:50:21.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:21.332+0000 7f3506000700 1 -- 172.21.15.37:0/1909306330 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f34fc00f040 con 0x7f3510070c70 2024-03-29T08:50:21.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:21.332+0000 7f3515000700 1 -- 172.21.15.37:0/1909306330 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f351010fdc0 con 0x7f3510070c70 2024-03-29T08:50:21.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:21.332+0000 7f3506000700 1 -- 172.21.15.37:0/1909306330 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f34fc009540 con 0x7f3510070c70 2024-03-29T08:50:21.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:21.332+0000 7f3515000700 1 -- 172.21.15.37:0/1909306330 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3510110300 con 0x7f3510070c70 2024-03-29T08:50:21.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:21.333+0000 7f3515000700 1 -- 172.21.15.37:0/1909306330 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3510071aa0 con 0x7f3510070c70 2024-03-29T08:50:21.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:21.334+0000 7f3506000700 1 -- 172.21.15.37:0/1909306330 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f34fc0096a0 con 0x7f3510070c70 2024-03-29T08:50:21.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:21.335+0000 7f3506000700 1 --2- 172.21.15.37:0/1909306330 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f34f406cc20 0x7f34f406f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:50:21.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:21.335+0000 7f3506000700 1 -- 172.21.15.37:0/1909306330 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f34fc08be60 con 0x7f3510070c70 2024-03-29T08:50:21.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:21.335+0000 7f350de00700 1 --2- 172.21.15.37:0/1909306330 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f34f406cc20 0x7f34f406f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:50:21.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:21.336+0000 7f350de00700 1 --2- 172.21.15.37:0/1909306330 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f34f406cc20 0x7f34f406f0d0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f3500009fd0 tx=0x7f3500009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:50:21.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:21.340+0000 7f3506000700 1 -- 172.21.15.37:0/1909306330 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f34fc0598b0 con 0x7f3510070c70 2024-03-29T08:50:21.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:21.607+0000 7f3515000700 1 -- 172.21.15.37:0/1909306330 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 23, "format": "json"} v 0) v1 -- 0x7f3510064970 con 0x7f3510070c70 2024-03-29T08:50:21.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:21.608+0000 7f3506000700 1 -- 172.21.15.37:0/1909306330 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 23, "format": "json"}]=0 dumped fsmap epoch 23 v36) v1 ==== 107+0+4046 (secure 0 0 0) 0x7f34fc059440 con 0x7f3510070c70 2024-03-29T08:50:21.608 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:50:21.608 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24319,"name":"cephfs.smithi148.qukpjh","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.148:6827/2641327912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2641327912},{"type":"v1","addr":"172.21.15.148:6827","nonce":2641327912}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15},{"gid":24327,"name":"cephfs.smithi148.sqteih","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6825/3252351726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":3252351726},{"type":"v1","addr":"172.21.15.148:6825","nonce":3252351726}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13},{"gid":34406,"name":"cephfs.smithi037.iifsmh","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6827/3877453059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":3877453059},{"type":"v1","addr":"172.21.15.37:6827","nonce":3877453059}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":21,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-29T08:03:29.241269+0000","modified":"2024-03-29T08:17:16.237238+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":82,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-03-29T08:50:21.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:21.610+0000 7f3515000700 1 -- 172.21.15.37:0/1909306330 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f34f406cc20 msgr2=0x7f34f406f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:50:21.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:21.610+0000 7f3515000700 1 --2- 172.21.15.37:0/1909306330 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f34f406cc20 0x7f34f406f0d0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f3500009fd0 tx=0x7f3500009e30 comp rx=0 tx=0).stop 2024-03-29T08:50:21.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:21.610+0000 7f3515000700 1 -- 172.21.15.37:0/1909306330 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3510070c70 msgr2=0x7f3510115e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:50:21.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:21.610+0000 7f3515000700 1 --2- 172.21.15.37:0/1909306330 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3510070c70 0x7f3510115e90 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f34fc002a50 tx=0x7f34fc0046b0 comp rx=0 tx=0).stop 2024-03-29T08:50:21.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:21.610+0000 7f3515000700 1 -- 172.21.15.37:0/1909306330 shutdown_connections 2024-03-29T08:50:21.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:21.610+0000 7f3515000700 1 --2- 172.21.15.37:0/1909306330 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f34f406cc20 0x7f34f406f0d0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:21.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:21.610+0000 7f3515000700 1 --2- 172.21.15.37:0/1909306330 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3510070360 0x7f3510115950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:21.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:21.610+0000 7f3515000700 1 --2- 172.21.15.37:0/1909306330 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3510070c70 0x7f3510115e90 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:21.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:21.610+0000 7f3515000700 1 -- 172.21.15.37:0/1909306330 >> 172.21.15.37:0/1909306330 conn(0x7f35100f8830 msgr2=0x7f35100f9dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:50:21.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:21.610+0000 7f3515000700 1 -- 172.21.15.37:0/1909306330 shutdown_connections 2024-03-29T08:50:21.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:21.610+0000 7f3515000700 1 -- 172.21.15.37:0/1909306330 wait complete. 2024-03-29T08:50:21.612 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 23 2024-03-29T08:50:21.629 DEBUG:tasks.fs:max_mds not reduced in epoch 23 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-03-29T08:50:21.630 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 24 2024-03-29T08:50:21.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:21.822+0000 7fe0fe000700 1 -- 172.21.15.37:0/4111637090 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe0f8102cd0 msgr2=0x7fe0f81030a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:50:21.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:21.822+0000 7fe0fe000700 1 --2- 172.21.15.37:0/4111637090 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe0f8102cd0 0x7fe0f81030a0 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7fe0e8004940 tx=0x7fe0e8009e30 comp rx=0 tx=0).stop 2024-03-29T08:50:21.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:21.822+0000 7fe0fe000700 1 -- 172.21.15.37:0/4111637090 shutdown_connections 2024-03-29T08:50:21.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:21.822+0000 7fe0fe000700 1 --2- 172.21.15.37:0/4111637090 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe0f8103670 0x7fe0f80fcce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:21.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:21.823+0000 7fe0fe000700 1 --2- 172.21.15.37:0/4111637090 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe0f8102cd0 0x7fe0f81030a0 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:21.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:21.823+0000 7fe0fe000700 1 -- 172.21.15.37:0/4111637090 >> 172.21.15.37:0/4111637090 conn(0x7fe0f80f8820 msgr2=0x7fe0f80fac30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:50:21.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:21.823+0000 7fe0fe000700 1 -- 172.21.15.37:0/4111637090 shutdown_connections 2024-03-29T08:50:21.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:21.823+0000 7fe0fe000700 1 -- 172.21.15.37:0/4111637090 wait complete. 2024-03-29T08:50:21.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:21.823+0000 7fe0fe000700 1 Processor -- start 2024-03-29T08:50:21.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:21.823+0000 7fe0fe000700 1 -- start start 2024-03-29T08:50:21.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:21.824+0000 7fe0fe000700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe0f8102cd0 0x7fe0f81159f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:50:21.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:21.824+0000 7fe0fe000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe0f8103670 0x7fe0f8115f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:50:21.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:21.824+0000 7fe0fe000700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0f81164e0 con 0x7fe0f8103670 2024-03-29T08:50:21.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:21.824+0000 7fe0fe000700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0f810fb30 con 0x7fe0f8102cd0 2024-03-29T08:50:21.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:21.824+0000 7fe0f6a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe0f8103670 0x7fe0f8115f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:50:21.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:21.824+0000 7fe0f7400700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe0f8102cd0 0x7fe0f81159f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:50:21.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:21.824+0000 7fe0f6a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe0f8103670 0x7fe0f8115f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52726/0 (socket says 172.21.15.37:52726) 2024-03-29T08:50:21.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:21.824+0000 7fe0f6a00700 1 -- 172.21.15.37:0/1761518515 learned_addr learned my addr 172.21.15.37:0/1761518515 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:50:21.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:21.825+0000 7fe0f6a00700 1 -- 172.21.15.37:0/1761518515 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe0f8102cd0 msgr2=0x7fe0f81159f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:50:21.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:21.825+0000 7fe0f6a00700 1 --2- 172.21.15.37:0/1761518515 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe0f8102cd0 0x7fe0f81159f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:21.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:21.825+0000 7fe0f6a00700 1 -- 172.21.15.37:0/1761518515 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0e8005040 con 0x7fe0f8103670 2024-03-29T08:50:21.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:21.825+0000 7fe0f6a00700 1 --2- 172.21.15.37:0/1761518515 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe0f8103670 0x7fe0f8115f30 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7fe0e4002a50 tx=0x7fe0e4004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:50:21.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:21.825+0000 7fe0ef400700 1 -- 172.21.15.37:0/1761518515 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe0e4007cc0 con 0x7fe0f8103670 2024-03-29T08:50:21.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:21.825+0000 7fe0ef400700 1 -- 172.21.15.37:0/1761518515 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fe0e400f040 con 0x7fe0f8103670 2024-03-29T08:50:21.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:21.825+0000 7fe0fe000700 1 -- 172.21.15.37:0/1761518515 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0f810fe10 con 0x7fe0f8103670 2024-03-29T08:50:21.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:21.825+0000 7fe0ef400700 1 -- 172.21.15.37:0/1761518515 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe0e400d920 con 0x7fe0f8103670 2024-03-29T08:50:21.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:21.825+0000 7fe0fe000700 1 -- 172.21.15.37:0/1761518515 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0f81103d0 con 0x7fe0f8103670 2024-03-29T08:50:21.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:21.826+0000 7fe0fe000700 1 -- 172.21.15.37:0/1761518515 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0f80fe420 con 0x7fe0f8103670 2024-03-29T08:50:21.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:21.827+0000 7fe0ef400700 1 -- 172.21.15.37:0/1761518515 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fe0e4009680 con 0x7fe0f8103670 2024-03-29T08:50:21.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:21.828+0000 7fe0ef400700 1 --2- 172.21.15.37:0/1761518515 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fe0dc06cc20 0x7fe0dc06f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:50:21.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:21.828+0000 7fe0ef400700 1 -- 172.21.15.37:0/1761518515 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fe0e408bdb0 con 0x7fe0f8103670 2024-03-29T08:50:21.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:21.828+0000 7fe0f7400700 1 --2- 172.21.15.37:0/1761518515 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fe0dc06cc20 0x7fe0dc06f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:50:21.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:21.829+0000 7fe0f7400700 1 --2- 172.21.15.37:0/1761518515 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fe0dc06cc20 0x7fe0dc06f0d0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fe0e8009fd0 tx=0x7fe0e8009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:50:21.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:21.833+0000 7fe0ef400700 1 -- 172.21.15.37:0/1761518515 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fe0e4055cf0 con 0x7fe0f8103670 2024-03-29T08:50:21.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:50:21 smithi037 ceph-mon[142325]: pgmap v1186: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:50:21.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:50:21 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/2265047764' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 22, "format": "json"}]: dispatch 2024-03-29T08:50:21.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:50:21 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/1909306330' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 23, "format": "json"}]: dispatch 2024-03-29T08:50:22.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:22.099+0000 7fe0fe000700 1 -- 172.21.15.37:0/1761518515 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 24, "format": "json"} v 0) v1 -- 0x7fe0f8109c90 con 0x7fe0f8103670 2024-03-29T08:50:22.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:22.100+0000 7fe0ef400700 1 -- 172.21.15.37:0/1761518515 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 24, "format": "json"}]=0 dumped fsmap epoch 24 v36) v1 ==== 107+0+4825 (secure 0 0 0) 0x7fe0e4059310 con 0x7fe0f8103670 2024-03-29T08:50:22.100 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:50:22.100 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24319,"name":"cephfs.smithi148.qukpjh","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.148:6827/2641327912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2641327912},{"type":"v1","addr":"172.21.15.148:6827","nonce":2641327912}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15},{"gid":24327,"name":"cephfs.smithi148.sqteih","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6825/3252351726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":3252351726},{"type":"v1","addr":"172.21.15.148:6825","nonce":3252351726}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13},{"gid":34406,"name":"cephfs.smithi037.iifsmh","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6827/3877453059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":3877453059},{"type":"v1","addr":"172.21.15.37:6827","nonce":3877453059}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":22},{"gid":34440,"name":"cephfs.smithi037.mmwgxe","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6829/1248142005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1248142005},{"type":"v1","addr":"172.21.15.37:6829","nonce":1248142005}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":24}],"filesystems":[{"mdsmap":{"epoch":21,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-29T08:03:29.241269+0000","modified":"2024-03-29T08:17:16.237238+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":82,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-03-29T08:50:22.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:22.102+0000 7fe0fe000700 1 -- 172.21.15.37:0/1761518515 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fe0dc06cc20 msgr2=0x7fe0dc06f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:50:22.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:22.102+0000 7fe0fe000700 1 --2- 172.21.15.37:0/1761518515 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fe0dc06cc20 0x7fe0dc06f0d0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fe0e8009fd0 tx=0x7fe0e8009e30 comp rx=0 tx=0).stop 2024-03-29T08:50:22.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:22.102+0000 7fe0fe000700 1 -- 172.21.15.37:0/1761518515 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe0f8103670 msgr2=0x7fe0f8115f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:50:22.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:22.102+0000 7fe0fe000700 1 --2- 172.21.15.37:0/1761518515 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe0f8103670 0x7fe0f8115f30 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7fe0e4002a50 tx=0x7fe0e4004600 comp rx=0 tx=0).stop 2024-03-29T08:50:22.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:22.102+0000 7fe0fe000700 1 -- 172.21.15.37:0/1761518515 shutdown_connections 2024-03-29T08:50:22.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:22.102+0000 7fe0fe000700 1 --2- 172.21.15.37:0/1761518515 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fe0dc06cc20 0x7fe0dc06f0d0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:22.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:22.102+0000 7fe0fe000700 1 --2- 172.21.15.37:0/1761518515 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe0f8102cd0 0x7fe0f81159f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:22.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:22.102+0000 7fe0fe000700 1 --2- 172.21.15.37:0/1761518515 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe0f8103670 0x7fe0f8115f30 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:22.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:22.102+0000 7fe0fe000700 1 -- 172.21.15.37:0/1761518515 >> 172.21.15.37:0/1761518515 conn(0x7fe0f80f8820 msgr2=0x7fe0f80f98f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:50:22.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:22.102+0000 7fe0fe000700 1 -- 172.21.15.37:0/1761518515 shutdown_connections 2024-03-29T08:50:22.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:22.102+0000 7fe0fe000700 1 -- 172.21.15.37:0/1761518515 wait complete. 2024-03-29T08:50:22.104 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 24 2024-03-29T08:50:22.120 DEBUG:tasks.fs:max_mds not reduced in epoch 24 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-03-29T08:50:22.120 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 25 2024-03-29T08:50:22.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:50:21 smithi148 ceph-mon[106692]: pgmap v1186: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:50:22.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:50:21 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/2265047764' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 22, "format": "json"}]: dispatch 2024-03-29T08:50:22.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:50:21 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/1909306330' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 23, "format": "json"}]: dispatch 2024-03-29T08:50:22.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:22.314+0000 7ff0ca400700 1 -- 172.21.15.37:0/1683199566 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff0c4102cd0 msgr2=0x7ff0c41030a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:50:22.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:22.314+0000 7ff0ca400700 1 --2- 172.21.15.37:0/1683199566 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff0c4102cd0 0x7ff0c41030a0 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7ff0b4004940 tx=0x7ff0b4009e30 comp rx=0 tx=0).stop 2024-03-29T08:50:22.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:22.314+0000 7ff0ca400700 1 -- 172.21.15.37:0/1683199566 shutdown_connections 2024-03-29T08:50:22.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:22.314+0000 7ff0ca400700 1 --2- 172.21.15.37:0/1683199566 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff0c4103670 0x7ff0c40fcce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:22.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:22.314+0000 7ff0ca400700 1 --2- 172.21.15.37:0/1683199566 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff0c4102cd0 0x7ff0c41030a0 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:22.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:22.314+0000 7ff0ca400700 1 -- 172.21.15.37:0/1683199566 >> 172.21.15.37:0/1683199566 conn(0x7ff0c40f8820 msgr2=0x7ff0c40fac30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:50:22.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:22.314+0000 7ff0ca400700 1 -- 172.21.15.37:0/1683199566 shutdown_connections 2024-03-29T08:50:22.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:22.314+0000 7ff0ca400700 1 -- 172.21.15.37:0/1683199566 wait complete. 2024-03-29T08:50:22.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:22.315+0000 7ff0ca400700 1 Processor -- start 2024-03-29T08:50:22.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:22.315+0000 7ff0ca400700 1 -- start start 2024-03-29T08:50:22.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:22.315+0000 7ff0ca400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff0c4102cd0 0x7ff0c41137b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:50:22.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:22.315+0000 7ff0ca400700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff0c4103670 0x7ff0c4113cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:50:22.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:22.315+0000 7ff0ca400700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0c41143d0 con 0x7ff0c4102cd0 2024-03-29T08:50:22.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:22.315+0000 7ff0ca400700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0c410d940 con 0x7ff0c4103670 2024-03-29T08:50:22.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:22.316+0000 7ff0c3400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff0c4102cd0 0x7ff0c41137b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:50:22.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:22.316+0000 7ff0c3400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff0c4102cd0 0x7ff0c41137b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52732/0 (socket says 172.21.15.37:52732) 2024-03-29T08:50:22.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:22.316+0000 7ff0c3400700 1 -- 172.21.15.37:0/2403365201 learned_addr learned my addr 172.21.15.37:0/2403365201 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:50:22.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:22.316+0000 7ff0c3400700 1 -- 172.21.15.37:0/2403365201 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff0c4103670 msgr2=0x7ff0c4113cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:50:22.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:22.316+0000 7ff0c3400700 1 --2- 172.21.15.37:0/2403365201 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff0c4103670 0x7ff0c4113cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:22.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:22.316+0000 7ff0c3400700 1 -- 172.21.15.37:0/2403365201 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0b4005040 con 0x7ff0c4102cd0 2024-03-29T08:50:22.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:22.316+0000 7ff0c3400700 1 --2- 172.21.15.37:0/2403365201 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff0c4102cd0 0x7ff0c41137b0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7ff0b4009360 tx=0x7ff0b4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:50:22.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:22.316+0000 7ff0bb400700 1 -- 172.21.15.37:0/2403365201 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff0b400e040 con 0x7ff0c4102cd0 2024-03-29T08:50:22.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:22.316+0000 7ff0bb400700 1 -- 172.21.15.37:0/2403365201 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ff0b4013420 con 0x7ff0c4102cd0 2024-03-29T08:50:22.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:22.316+0000 7ff0ca400700 1 -- 172.21.15.37:0/2403365201 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0c410dbc0 con 0x7ff0c4102cd0 2024-03-29T08:50:22.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:22.316+0000 7ff0bb400700 1 -- 172.21.15.37:0/2403365201 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff0b401c5f0 con 0x7ff0c4102cd0 2024-03-29T08:50:22.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:22.317+0000 7ff0ca400700 1 -- 172.21.15.37:0/2403365201 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0c410e180 con 0x7ff0c4102cd0 2024-03-29T08:50:22.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:22.318+0000 7ff0ca400700 1 -- 172.21.15.37:0/2403365201 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0c40fe420 con 0x7ff0c4102cd0 2024-03-29T08:50:22.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:22.319+0000 7ff0bb400700 1 -- 172.21.15.37:0/2403365201 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7ff0b4013590 con 0x7ff0c4102cd0 2024-03-29T08:50:22.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:22.319+0000 7ff0bb400700 1 --2- 172.21.15.37:0/2403365201 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7ff0a806cc20 0x7ff0a806f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:50:22.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:22.319+0000 7ff0bb400700 1 -- 172.21.15.37:0/2403365201 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7ff0b408d180 con 0x7ff0c4102cd0 2024-03-29T08:50:22.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:22.320+0000 7ff0bbe00700 1 --2- 172.21.15.37:0/2403365201 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7ff0a806cc20 0x7ff0a806f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:50:22.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:22.320+0000 7ff0bbe00700 1 --2- 172.21.15.37:0/2403365201 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7ff0a806cc20 0x7ff0a806f0d0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7ff0ac006fd0 tx=0x7ff0ac006db0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:50:22.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:22.324+0000 7ff0bb400700 1 -- 172.21.15.37:0/2403365201 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7ff0b405ad10 con 0x7ff0c4102cd0 2024-03-29T08:50:22.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:22.591+0000 7ff0ca400700 1 -- 172.21.15.37:0/2403365201 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 25, "format": "json"} v 0) v1 -- 0x7ff0c4109d60 con 0x7ff0c4102cd0 2024-03-29T08:50:22.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:22.591+0000 7ff0bb400700 1 -- 172.21.15.37:0/2403365201 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 25, "format": "json"}]=0 dumped fsmap epoch 25 v36) v1 ==== 107+0+4046 (secure 0 0 0) 0x7ff0b405a8a0 con 0x7ff0c4102cd0 2024-03-29T08:50:22.592 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:50:22.592 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24319,"name":"cephfs.smithi148.qukpjh","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.148:6827/2641327912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2641327912},{"type":"v1","addr":"172.21.15.148:6827","nonce":2641327912}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15},{"gid":24327,"name":"cephfs.smithi148.sqteih","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6825/3252351726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":3252351726},{"type":"v1","addr":"172.21.15.148:6825","nonce":3252351726}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13},{"gid":34440,"name":"cephfs.smithi037.mmwgxe","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6829/1248142005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1248142005},{"type":"v1","addr":"172.21.15.37:6829","nonce":1248142005}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":24}],"filesystems":[{"mdsmap":{"epoch":21,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-29T08:03:29.241269+0000","modified":"2024-03-29T08:17:16.237238+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":82,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-03-29T08:50:22.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:22.594+0000 7ff0ca400700 1 -- 172.21.15.37:0/2403365201 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7ff0a806cc20 msgr2=0x7ff0a806f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:50:22.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:22.594+0000 7ff0ca400700 1 --2- 172.21.15.37:0/2403365201 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7ff0a806cc20 0x7ff0a806f0d0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7ff0ac006fd0 tx=0x7ff0ac006db0 comp rx=0 tx=0).stop 2024-03-29T08:50:22.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:22.594+0000 7ff0ca400700 1 -- 172.21.15.37:0/2403365201 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff0c4102cd0 msgr2=0x7ff0c41137b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:50:22.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:22.594+0000 7ff0ca400700 1 --2- 172.21.15.37:0/2403365201 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff0c4102cd0 0x7ff0c41137b0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7ff0b4009360 tx=0x7ff0b4009e30 comp rx=0 tx=0).stop 2024-03-29T08:50:22.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:22.594+0000 7ff0ca400700 1 -- 172.21.15.37:0/2403365201 shutdown_connections 2024-03-29T08:50:22.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:22.594+0000 7ff0ca400700 1 --2- 172.21.15.37:0/2403365201 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7ff0a806cc20 0x7ff0a806f0d0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:22.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:22.594+0000 7ff0ca400700 1 --2- 172.21.15.37:0/2403365201 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff0c4102cd0 0x7ff0c41137b0 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:22.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:22.594+0000 7ff0ca400700 1 --2- 172.21.15.37:0/2403365201 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff0c4103670 0x7ff0c4113cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:22.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:22.594+0000 7ff0ca400700 1 -- 172.21.15.37:0/2403365201 >> 172.21.15.37:0/2403365201 conn(0x7ff0c40f8820 msgr2=0x7ff0c40f98a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:50:22.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:22.594+0000 7ff0ca400700 1 -- 172.21.15.37:0/2403365201 shutdown_connections 2024-03-29T08:50:22.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:22.594+0000 7ff0ca400700 1 -- 172.21.15.37:0/2403365201 wait complete. 2024-03-29T08:50:22.596 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 25 2024-03-29T08:50:22.612 DEBUG:tasks.fs:max_mds not reduced in epoch 25 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-03-29T08:50:22.612 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 26 2024-03-29T08:50:22.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:22.807+0000 7fdd08200700 1 -- 172.21.15.37:0/1431000794 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdd00103670 msgr2=0x7fdd000fcce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:50:22.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:22.807+0000 7fdd08200700 1 --2- 172.21.15.37:0/1431000794 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdd00103670 0x7fdd000fcce0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7fdcec004940 tx=0x7fdcec009e30 comp rx=0 tx=0).stop 2024-03-29T08:50:22.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:22.807+0000 7fdd08200700 1 -- 172.21.15.37:0/1431000794 shutdown_connections 2024-03-29T08:50:22.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:22.807+0000 7fdd08200700 1 --2- 172.21.15.37:0/1431000794 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdd00103670 0x7fdd000fcce0 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:22.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:22.807+0000 7fdd08200700 1 --2- 172.21.15.37:0/1431000794 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdd00102cd0 0x7fdd001030a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:22.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:22.807+0000 7fdd08200700 1 -- 172.21.15.37:0/1431000794 >> 172.21.15.37:0/1431000794 conn(0x7fdd000f8820 msgr2=0x7fdd000fac30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:50:22.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:22.807+0000 7fdd08200700 1 -- 172.21.15.37:0/1431000794 shutdown_connections 2024-03-29T08:50:22.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:22.807+0000 7fdd08200700 1 -- 172.21.15.37:0/1431000794 wait complete. 2024-03-29T08:50:22.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:22.808+0000 7fdd08200700 1 Processor -- start 2024-03-29T08:50:22.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:22.808+0000 7fdd08200700 1 -- start start 2024-03-29T08:50:22.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:22.808+0000 7fdd08200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdd00102cd0 0x7fdd000702c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:50:22.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:22.808+0000 7fdd08200700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdd00103670 0x7fdd00070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:50:22.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:22.808+0000 7fdd08200700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd00072210 con 0x7fdd00102cd0 2024-03-29T08:50:22.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:22.808+0000 7fdd08200700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd00070d40 con 0x7fdd00103670 2024-03-29T08:50:22.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:22.808+0000 7fdd05800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdd00102cd0 0x7fdd000702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:50:22.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:22.809+0000 7fdd05800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdd00102cd0 0x7fdd000702c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52758/0 (socket says 172.21.15.37:52758) 2024-03-29T08:50:22.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:22.809+0000 7fdd05800700 1 -- 172.21.15.37:0/3174166462 learned_addr learned my addr 172.21.15.37:0/3174166462 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:50:22.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:22.809+0000 7fdd04e00700 1 --2- 172.21.15.37:0/3174166462 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdd00103670 0x7fdd00070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:50:22.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:22.809+0000 7fdd05800700 1 -- 172.21.15.37:0/3174166462 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdd00103670 msgr2=0x7fdd00070800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:50:22.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:22.809+0000 7fdd05800700 1 --2- 172.21.15.37:0/3174166462 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdd00103670 0x7fdd00070800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:22.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:22.809+0000 7fdd05800700 1 -- 172.21.15.37:0/3174166462 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcec005040 con 0x7fdd00102cd0 2024-03-29T08:50:22.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:22.809+0000 7fdd05800700 1 --2- 172.21.15.37:0/3174166462 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdd00102cd0 0x7fdd000702c0 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7fdcf400c830 tx=0x7fdcf4004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:50:22.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:22.809+0000 7fdcfd600700 1 -- 172.21.15.37:0/3174166462 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdcf400a760 con 0x7fdd00102cd0 2024-03-29T08:50:22.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:22.809+0000 7fdcfd600700 1 -- 172.21.15.37:0/3174166462 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fdcf400ad70 con 0x7fdd00102cd0 2024-03-29T08:50:22.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:22.809+0000 7fdd08200700 1 -- 172.21.15.37:0/3174166462 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd00071020 con 0x7fdd00102cd0 2024-03-29T08:50:22.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:22.809+0000 7fdcfd600700 1 -- 172.21.15.37:0/3174166462 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdcf40113a0 con 0x7fdd00102cd0 2024-03-29T08:50:22.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:22.810+0000 7fdd08200700 1 -- 172.21.15.37:0/3174166462 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd00119f30 con 0x7fdd00102cd0 2024-03-29T08:50:22.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:22.810+0000 7fdd08200700 1 -- 172.21.15.37:0/3174166462 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd000fe420 con 0x7fdd00102cd0 2024-03-29T08:50:22.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:22.812+0000 7fdcfd600700 1 -- 172.21.15.37:0/3174166462 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fdcf4012400 con 0x7fdd00102cd0 2024-03-29T08:50:22.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:22.812+0000 7fdcfd600700 1 --2- 172.21.15.37:0/3174166462 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fdce806cc20 0x7fdce806f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:50:22.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:22.813+0000 7fdcfd600700 1 -- 172.21.15.37:0/3174166462 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fdcf408c120 con 0x7fdd00102cd0 2024-03-29T08:50:22.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:22.813+0000 7fdd04e00700 1 --2- 172.21.15.37:0/3174166462 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fdce806cc20 0x7fdce806f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:50:22.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:22.813+0000 7fdd04e00700 1 --2- 172.21.15.37:0/3174166462 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fdce806cc20 0x7fdce806f0d0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fdcec00b020 tx=0x7fdcec009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:50:22.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:22.817+0000 7fdcfd600700 1 -- 172.21.15.37:0/3174166462 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fdcf40560e0 con 0x7fdd00102cd0 2024-03-29T08:50:23.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:23.080+0000 7fdd08200700 1 -- 172.21.15.37:0/3174166462 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 26, "format": "json"} v 0) v1 -- 0x7fdd0011a180 con 0x7fdd00102cd0 2024-03-29T08:50:23.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:23.081+0000 7fdcfd600700 1 -- 172.21.15.37:0/3174166462 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 26, "format": "json"}]=0 dumped fsmap epoch 26 v36) v1 ==== 107+0+4825 (secure 0 0 0) 0x7fdcf4059700 con 0x7fdd00102cd0 2024-03-29T08:50:23.081 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:50:23.081 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24319,"name":"cephfs.smithi148.qukpjh","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.148:6827/2641327912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2641327912},{"type":"v1","addr":"172.21.15.148:6827","nonce":2641327912}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15},{"gid":24327,"name":"cephfs.smithi148.sqteih","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6825/3252351726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":3252351726},{"type":"v1","addr":"172.21.15.148:6825","nonce":3252351726}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13},{"gid":34440,"name":"cephfs.smithi037.mmwgxe","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6829/1248142005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1248142005},{"type":"v1","addr":"172.21.15.37:6829","nonce":1248142005}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":24},{"gid":34444,"name":"cephfs.smithi037.iifsmh","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6827/2060882959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":2060882959},{"type":"v1","addr":"172.21.15.37:6827","nonce":2060882959}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":21,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-29T08:03:29.241269+0000","modified":"2024-03-29T08:17:16.237238+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":82,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-03-29T08:50:23.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:23.083+0000 7fdd08200700 1 -- 172.21.15.37:0/3174166462 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fdce806cc20 msgr2=0x7fdce806f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:50:23.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:23.083+0000 7fdd08200700 1 --2- 172.21.15.37:0/3174166462 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fdce806cc20 0x7fdce806f0d0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fdcec00b020 tx=0x7fdcec009e30 comp rx=0 tx=0).stop 2024-03-29T08:50:23.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:23.083+0000 7fdd08200700 1 -- 172.21.15.37:0/3174166462 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdd00102cd0 msgr2=0x7fdd000702c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:50:23.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:23.083+0000 7fdd08200700 1 --2- 172.21.15.37:0/3174166462 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdd00102cd0 0x7fdd000702c0 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7fdcf400c830 tx=0x7fdcf4004470 comp rx=0 tx=0).stop 2024-03-29T08:50:23.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:23.083+0000 7fdd08200700 1 -- 172.21.15.37:0/3174166462 shutdown_connections 2024-03-29T08:50:23.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:23.083+0000 7fdd08200700 1 --2- 172.21.15.37:0/3174166462 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fdce806cc20 0x7fdce806f0d0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:23.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:23.083+0000 7fdd08200700 1 --2- 172.21.15.37:0/3174166462 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdd00102cd0 0x7fdd000702c0 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:23.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:23.083+0000 7fdd08200700 1 --2- 172.21.15.37:0/3174166462 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdd00103670 0x7fdd00070800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:23.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:23.083+0000 7fdd08200700 1 -- 172.21.15.37:0/3174166462 >> 172.21.15.37:0/3174166462 conn(0x7fdd000f8820 msgr2=0x7fdd000f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:50:23.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:23.084+0000 7fdd08200700 1 -- 172.21.15.37:0/3174166462 shutdown_connections 2024-03-29T08:50:23.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:23.084+0000 7fdd08200700 1 -- 172.21.15.37:0/3174166462 wait complete. 2024-03-29T08:50:23.085 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 26 2024-03-29T08:50:23.102 DEBUG:tasks.fs:max_mds not reduced in epoch 26 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-03-29T08:50:23.102 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 27 2024-03-29T08:50:23.150 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:50:22 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/1761518515' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 24, "format": "json"}]: dispatch 2024-03-29T08:50:23.150 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:50:22 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/2403365201' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 25, "format": "json"}]: dispatch 2024-03-29T08:50:23.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:23.298+0000 7fe11f200700 1 -- 172.21.15.37:0/1977645664 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe118102cd0 msgr2=0x7fe1181030a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:50:23.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:23.298+0000 7fe11f200700 1 --2- 172.21.15.37:0/1977645664 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe118102cd0 0x7fe1181030a0 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7fe108004940 tx=0x7fe108009e30 comp rx=0 tx=0).stop 2024-03-29T08:50:23.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:23.298+0000 7fe11f200700 1 -- 172.21.15.37:0/1977645664 shutdown_connections 2024-03-29T08:50:23.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:23.298+0000 7fe11f200700 1 --2- 172.21.15.37:0/1977645664 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe118103670 0x7fe1180fcce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:23.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:23.298+0000 7fe11f200700 1 --2- 172.21.15.37:0/1977645664 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe118102cd0 0x7fe1181030a0 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:23.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:23.298+0000 7fe11f200700 1 -- 172.21.15.37:0/1977645664 >> 172.21.15.37:0/1977645664 conn(0x7fe1180f8820 msgr2=0x7fe1180fac30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:50:23.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:23.298+0000 7fe11f200700 1 -- 172.21.15.37:0/1977645664 shutdown_connections 2024-03-29T08:50:23.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:23.298+0000 7fe11f200700 1 -- 172.21.15.37:0/1977645664 wait complete. 2024-03-29T08:50:23.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:23.299+0000 7fe11f200700 1 Processor -- start 2024-03-29T08:50:23.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:23.299+0000 7fe11f200700 1 -- start start 2024-03-29T08:50:23.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:23.300+0000 7fe11f200700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe118102cd0 0x7fe1180702c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:50:23.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:23.300+0000 7fe11f200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe118103670 0x7fe118070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:50:23.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:23.300+0000 7fe11f200700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe118072210 con 0x7fe118103670 2024-03-29T08:50:23.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:23.300+0000 7fe11f200700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe118070d40 con 0x7fe118102cd0 2024-03-29T08:50:23.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:23.300+0000 7fe117400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe118103670 0x7fe118070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:50:23.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:23.300+0000 7fe117400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe118103670 0x7fe118070800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52786/0 (socket says 172.21.15.37:52786) 2024-03-29T08:50:23.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:23.300+0000 7fe117400700 1 -- 172.21.15.37:0/3962307418 learned_addr learned my addr 172.21.15.37:0/3962307418 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:50:23.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:23.300+0000 7fe117e00700 1 --2- 172.21.15.37:0/3962307418 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe118102cd0 0x7fe1180702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:50:23.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:23.301+0000 7fe117400700 1 -- 172.21.15.37:0/3962307418 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe118102cd0 msgr2=0x7fe1180702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:50:23.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:23.301+0000 7fe117400700 1 --2- 172.21.15.37:0/3962307418 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe118102cd0 0x7fe1180702c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:23.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:23.301+0000 7fe117400700 1 -- 172.21.15.37:0/3962307418 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe108005040 con 0x7fe118103670 2024-03-29T08:50:23.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:23.301+0000 7fe117400700 1 --2- 172.21.15.37:0/3962307418 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe118103670 0x7fe118070800 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7fe104002a50 tx=0x7fe104004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:50:23.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:23.301+0000 7fe10fe00700 1 -- 172.21.15.37:0/3962307418 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe104007cc0 con 0x7fe118103670 2024-03-29T08:50:23.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:23.301+0000 7fe10fe00700 1 -- 172.21.15.37:0/3962307418 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fe10400f040 con 0x7fe118103670 2024-03-29T08:50:23.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:23.301+0000 7fe11f200700 1 -- 172.21.15.37:0/3962307418 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe118071020 con 0x7fe118103670 2024-03-29T08:50:23.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:23.301+0000 7fe10fe00700 1 -- 172.21.15.37:0/3962307418 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe10400d920 con 0x7fe118103670 2024-03-29T08:50:23.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:23.301+0000 7fe11f200700 1 -- 172.21.15.37:0/3962307418 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe118119f70 con 0x7fe118103670 2024-03-29T08:50:23.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:23.302+0000 7fe11f200700 1 -- 172.21.15.37:0/3962307418 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1180fe420 con 0x7fe118103670 2024-03-29T08:50:23.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:23.304+0000 7fe10fe00700 1 -- 172.21.15.37:0/3962307418 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fe104009680 con 0x7fe118103670 2024-03-29T08:50:23.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:23.304+0000 7fe10fe00700 1 --2- 172.21.15.37:0/3962307418 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fe0fc06cc20 0x7fe0fc06f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:50:23.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:23.305+0000 7fe10fe00700 1 -- 172.21.15.37:0/3962307418 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fe10408bd30 con 0x7fe118103670 2024-03-29T08:50:23.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:23.305+0000 7fe117e00700 1 --2- 172.21.15.37:0/3962307418 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fe0fc06cc20 0x7fe0fc06f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:50:23.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:50:22 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/1761518515' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 24, "format": "json"}]: dispatch 2024-03-29T08:50:23.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:50:22 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/2403365201' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 25, "format": "json"}]: dispatch 2024-03-29T08:50:23.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:23.305+0000 7fe117e00700 1 --2- 172.21.15.37:0/3962307418 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fe0fc06cc20 0x7fe0fc06f0d0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fe108009fd0 tx=0x7fe108009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:50:23.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:23.309+0000 7fe10fe00700 1 -- 172.21.15.37:0/3962307418 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fe104055cf0 con 0x7fe118103670 2024-03-29T08:50:23.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:23.575+0000 7fe11f200700 1 -- 172.21.15.37:0/3962307418 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 27, "format": "json"} v 0) v1 -- 0x7fe11811a160 con 0x7fe118103670 2024-03-29T08:50:23.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:23.576+0000 7fe10fe00700 1 -- 172.21.15.37:0/3962307418 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 27, "format": "json"}]=0 dumped fsmap epoch 27 v36) v1 ==== 107+0+4043 (secure 0 0 0) 0x7fe104059310 con 0x7fe118103670 2024-03-29T08:50:23.577 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:50:23.577 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24319,"name":"cephfs.smithi148.qukpjh","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.148:6827/2641327912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2641327912},{"type":"v1","addr":"172.21.15.148:6827","nonce":2641327912}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15},{"gid":34440,"name":"cephfs.smithi037.mmwgxe","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6829/1248142005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1248142005},{"type":"v1","addr":"172.21.15.37:6829","nonce":1248142005}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":24},{"gid":34444,"name":"cephfs.smithi037.iifsmh","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6827/2060882959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":2060882959},{"type":"v1","addr":"172.21.15.37:6827","nonce":2060882959}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":21,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-29T08:03:29.241269+0000","modified":"2024-03-29T08:17:16.237238+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":82,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-03-29T08:50:23.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:23.578+0000 7fe11f200700 1 -- 172.21.15.37:0/3962307418 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fe0fc06cc20 msgr2=0x7fe0fc06f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:50:23.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:23.578+0000 7fe11f200700 1 --2- 172.21.15.37:0/3962307418 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fe0fc06cc20 0x7fe0fc06f0d0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fe108009fd0 tx=0x7fe108009e30 comp rx=0 tx=0).stop 2024-03-29T08:50:23.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:23.578+0000 7fe11f200700 1 -- 172.21.15.37:0/3962307418 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe118103670 msgr2=0x7fe118070800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:50:23.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:23.578+0000 7fe11f200700 1 --2- 172.21.15.37:0/3962307418 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe118103670 0x7fe118070800 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7fe104002a50 tx=0x7fe104004600 comp rx=0 tx=0).stop 2024-03-29T08:50:23.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:23.578+0000 7fe11f200700 1 -- 172.21.15.37:0/3962307418 shutdown_connections 2024-03-29T08:50:23.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:23.578+0000 7fe11f200700 1 --2- 172.21.15.37:0/3962307418 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fe0fc06cc20 0x7fe0fc06f0d0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:23.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:23.578+0000 7fe11f200700 1 --2- 172.21.15.37:0/3962307418 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe118102cd0 0x7fe1180702c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:23.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:23.578+0000 7fe11f200700 1 --2- 172.21.15.37:0/3962307418 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe118103670 0x7fe118070800 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:23.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:23.578+0000 7fe11f200700 1 -- 172.21.15.37:0/3962307418 >> 172.21.15.37:0/3962307418 conn(0x7fe1180f8820 msgr2=0x7fe1180f98f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:50:23.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:23.578+0000 7fe11f200700 1 -- 172.21.15.37:0/3962307418 shutdown_connections 2024-03-29T08:50:23.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:23.579+0000 7fe11f200700 1 -- 172.21.15.37:0/3962307418 wait complete. 2024-03-29T08:50:23.582 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 27 2024-03-29T08:50:23.598 DEBUG:tasks.fs:max_mds not reduced in epoch 27 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-03-29T08:50:23.598 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 28 2024-03-29T08:50:23.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:23.793+0000 7f877e400700 1 -- 172.21.15.37:0/502942001 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8778103670 msgr2=0x7f87780fcce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:50:23.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:23.793+0000 7f877e400700 1 --2- 172.21.15.37:0/502942001 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8778103670 0x7f87780fcce0 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f8764004940 tx=0x7f8764009e30 comp rx=0 tx=0).stop 2024-03-29T08:50:23.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:23.793+0000 7f877e400700 1 -- 172.21.15.37:0/502942001 shutdown_connections 2024-03-29T08:50:23.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:23.793+0000 7f877e400700 1 --2- 172.21.15.37:0/502942001 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8778103670 0x7f87780fcce0 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:23.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:23.793+0000 7f877e400700 1 --2- 172.21.15.37:0/502942001 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8778102cd0 0x7f87781030a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:23.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:23.793+0000 7f877e400700 1 -- 172.21.15.37:0/502942001 >> 172.21.15.37:0/502942001 conn(0x7f87780f8820 msgr2=0x7f87780fac30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:50:23.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:23.793+0000 7f877e400700 1 -- 172.21.15.37:0/502942001 shutdown_connections 2024-03-29T08:50:23.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:23.793+0000 7f877e400700 1 -- 172.21.15.37:0/502942001 wait complete. 2024-03-29T08:50:23.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:23.794+0000 7f877e400700 1 Processor -- start 2024-03-29T08:50:23.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:23.794+0000 7f877e400700 1 -- start start 2024-03-29T08:50:23.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:23.794+0000 7f877e400700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8778102cd0 0x7f87780702c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:50:23.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:23.794+0000 7f877e400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8778103670 0x7f8778070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:50:23.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:23.794+0000 7f877e400700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8778072210 con 0x7f8778103670 2024-03-29T08:50:23.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:23.794+0000 7f877e400700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8778070d40 con 0x7f8778102cd0 2024-03-29T08:50:23.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:23.795+0000 7f8776a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8778103670 0x7f8778070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:50:23.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:23.795+0000 7f8776a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8778103670 0x7f8778070800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52802/0 (socket says 172.21.15.37:52802) 2024-03-29T08:50:23.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:23.795+0000 7f8776a00700 1 -- 172.21.15.37:0/4102921764 learned_addr learned my addr 172.21.15.37:0/4102921764 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:50:23.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:23.795+0000 7f8777400700 1 --2- 172.21.15.37:0/4102921764 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8778102cd0 0x7f87780702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:50:23.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:23.795+0000 7f8776a00700 1 -- 172.21.15.37:0/4102921764 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8778102cd0 msgr2=0x7f87780702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:50:23.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:23.795+0000 7f8776a00700 1 --2- 172.21.15.37:0/4102921764 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8778102cd0 0x7f87780702c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:23.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:23.795+0000 7f8776a00700 1 -- 172.21.15.37:0/4102921764 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8764005040 con 0x7f8778103670 2024-03-29T08:50:23.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:23.795+0000 7f8776a00700 1 --2- 172.21.15.37:0/4102921764 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8778103670 0x7f8778070800 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f8764009390 tx=0x7f8764009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:50:23.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:23.795+0000 7f876f400700 1 -- 172.21.15.37:0/4102921764 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f876400e040 con 0x7f8778103670 2024-03-29T08:50:23.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:23.796+0000 7f876f400700 1 -- 172.21.15.37:0/4102921764 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f8764013420 con 0x7f8778103670 2024-03-29T08:50:23.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:23.796+0000 7f876f400700 1 -- 172.21.15.37:0/4102921764 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f876401c630 con 0x7f8778103670 2024-03-29T08:50:23.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:23.796+0000 7f877e400700 1 -- 172.21.15.37:0/4102921764 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8778070fc0 con 0x7f8778103670 2024-03-29T08:50:23.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:23.796+0000 7f877e400700 1 -- 172.21.15.37:0/4102921764 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8778119ed0 con 0x7f8778103670 2024-03-29T08:50:23.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:23.797+0000 7f877e400700 1 -- 172.21.15.37:0/4102921764 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87780fe420 con 0x7f8778103670 2024-03-29T08:50:23.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:23.798+0000 7f876f400700 1 -- 172.21.15.37:0/4102921764 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f8764013a00 con 0x7f8778103670 2024-03-29T08:50:23.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:23.798+0000 7f876f400700 1 --2- 172.21.15.37:0/4102921764 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f875c06cc20 0x7f875c06f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:50:23.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:23.799+0000 7f876f400700 1 -- 172.21.15.37:0/4102921764 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f876408d0f0 con 0x7f8778103670 2024-03-29T08:50:23.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:23.799+0000 7f8777400700 1 --2- 172.21.15.37:0/4102921764 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f875c06cc20 0x7f875c06f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:50:23.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:23.799+0000 7f8777400700 1 --2- 172.21.15.37:0/4102921764 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f875c06cc20 0x7f875c06f0d0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f8768004f40 tx=0x7f876800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:50:23.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:23.803+0000 7f876f400700 1 -- 172.21.15.37:0/4102921764 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f8764057130 con 0x7f8778103670 2024-03-29T08:50:24.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:24.066+0000 7f877e400700 1 -- 172.21.15.37:0/4102921764 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 28, "format": "json"} v 0) v1 -- 0x7f8778109c70 con 0x7f8778103670 2024-03-29T08:50:24.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:24.067+0000 7f876f400700 1 -- 172.21.15.37:0/4102921764 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 28, "format": "json"}]=0 dumped fsmap epoch 28 v36) v1 ==== 107+0+4825 (secure 0 0 0) 0x7f876405a750 con 0x7f8778103670 2024-03-29T08:50:24.068 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:50:24.068 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24319,"name":"cephfs.smithi148.qukpjh","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.148:6827/2641327912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2641327912},{"type":"v1","addr":"172.21.15.148:6827","nonce":2641327912}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15},{"gid":34440,"name":"cephfs.smithi037.mmwgxe","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6829/1248142005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1248142005},{"type":"v1","addr":"172.21.15.37:6829","nonce":1248142005}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":24},{"gid":34444,"name":"cephfs.smithi037.iifsmh","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6827/2060882959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":2060882959},{"type":"v1","addr":"172.21.15.37:6827","nonce":2060882959}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":44395,"name":"cephfs.smithi148.sqteih","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6825/4078652242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":4078652242},{"type":"v1","addr":"172.21.15.148:6825","nonce":4078652242}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":21,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-29T08:03:29.241269+0000","modified":"2024-03-29T08:17:16.237238+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":82,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-03-29T08:50:24.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:24.069+0000 7f877e400700 1 -- 172.21.15.37:0/4102921764 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f875c06cc20 msgr2=0x7f875c06f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:50:24.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:24.069+0000 7f877e400700 1 --2- 172.21.15.37:0/4102921764 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f875c06cc20 0x7f875c06f0d0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f8768004f40 tx=0x7f876800b040 comp rx=0 tx=0).stop 2024-03-29T08:50:24.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:24.069+0000 7f877e400700 1 -- 172.21.15.37:0/4102921764 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8778103670 msgr2=0x7f8778070800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:50:24.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:24.069+0000 7f877e400700 1 --2- 172.21.15.37:0/4102921764 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8778103670 0x7f8778070800 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f8764009390 tx=0x7f8764009e30 comp rx=0 tx=0).stop 2024-03-29T08:50:24.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:24.069+0000 7f877e400700 1 -- 172.21.15.37:0/4102921764 shutdown_connections 2024-03-29T08:50:24.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:24.069+0000 7f877e400700 1 --2- 172.21.15.37:0/4102921764 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f875c06cc20 0x7f875c06f0d0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:24.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:24.069+0000 7f877e400700 1 --2- 172.21.15.37:0/4102921764 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8778102cd0 0x7f87780702c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:24.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:24.069+0000 7f877e400700 1 --2- 172.21.15.37:0/4102921764 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8778103670 0x7f8778070800 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:24.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:24.069+0000 7f877e400700 1 -- 172.21.15.37:0/4102921764 >> 172.21.15.37:0/4102921764 conn(0x7f87780f8820 msgr2=0x7f87780f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:50:24.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:24.069+0000 7f877e400700 1 -- 172.21.15.37:0/4102921764 shutdown_connections 2024-03-29T08:50:24.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:24.069+0000 7f877e400700 1 -- 172.21.15.37:0/4102921764 wait complete. 2024-03-29T08:50:24.073 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 28 2024-03-29T08:50:24.088 DEBUG:tasks.fs:max_mds not reduced in epoch 28 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-03-29T08:50:24.088 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 29 2024-03-29T08:50:24.148 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:50:23 smithi037 ceph-mon[142325]: pgmap v1187: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:50:24.148 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:50:23 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/3174166462' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 26, "format": "json"}]: dispatch 2024-03-29T08:50:24.148 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:50:23 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/3962307418' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 27, "format": "json"}]: dispatch 2024-03-29T08:50:24.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:24.282+0000 7f121c200700 1 -- 172.21.15.37:0/4201887808 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1214103670 msgr2=0x7f12140fcce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:50:24.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:24.282+0000 7f121c200700 1 --2- 172.21.15.37:0/4201887808 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1214103670 0x7f12140fcce0 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f1200004940 tx=0x7f1200009e30 comp rx=0 tx=0).stop 2024-03-29T08:50:24.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:24.282+0000 7f121c200700 1 -- 172.21.15.37:0/4201887808 shutdown_connections 2024-03-29T08:50:24.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:24.282+0000 7f121c200700 1 --2- 172.21.15.37:0/4201887808 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1214103670 0x7f12140fcce0 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:24.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:24.282+0000 7f121c200700 1 --2- 172.21.15.37:0/4201887808 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1214102cd0 0x7f12141030a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:24.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:24.282+0000 7f121c200700 1 -- 172.21.15.37:0/4201887808 >> 172.21.15.37:0/4201887808 conn(0x7f12140f8820 msgr2=0x7f12140fac30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:50:24.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:24.282+0000 7f121c200700 1 -- 172.21.15.37:0/4201887808 shutdown_connections 2024-03-29T08:50:24.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:24.282+0000 7f121c200700 1 -- 172.21.15.37:0/4201887808 wait complete. 2024-03-29T08:50:24.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:24.283+0000 7f121c200700 1 Processor -- start 2024-03-29T08:50:24.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:24.283+0000 7f121c200700 1 -- start start 2024-03-29T08:50:24.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:24.284+0000 7f121c200700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1214102cd0 0x7f1214115910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:50:24.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:24.284+0000 7f121c200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1214103670 0x7f1214115e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:50:24.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:24.284+0000 7f121c200700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12141164a0 con 0x7f1214103670 2024-03-29T08:50:24.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:24.284+0000 7f121c200700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f121410faa0 con 0x7f1214102cd0 2024-03-29T08:50:24.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:24.284+0000 7f1218e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1214103670 0x7f1214115e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:50:24.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:24.284+0000 7f1218e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1214103670 0x7f1214115e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52820/0 (socket says 172.21.15.37:52820) 2024-03-29T08:50:24.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:24.284+0000 7f1218e00700 1 -- 172.21.15.37:0/1459351230 learned_addr learned my addr 172.21.15.37:0/1459351230 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:50:24.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:24.284+0000 7f1219800700 1 --2- 172.21.15.37:0/1459351230 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1214102cd0 0x7f1214115910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:50:24.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:24.284+0000 7f1218e00700 1 -- 172.21.15.37:0/1459351230 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1214102cd0 msgr2=0x7f1214115910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:50:24.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:24.284+0000 7f1218e00700 1 --2- 172.21.15.37:0/1459351230 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1214102cd0 0x7f1214115910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:24.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:24.284+0000 7f1218e00700 1 -- 172.21.15.37:0/1459351230 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1200005040 con 0x7f1214103670 2024-03-29T08:50:24.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:24.284+0000 7f1219800700 1 --2- 172.21.15.37:0/1459351230 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1214102cd0 0x7f1214115910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:50:24.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:24.284+0000 7f1218e00700 1 --2- 172.21.15.37:0/1459351230 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1214103670 0x7f1214115e50 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f1200008d50 tx=0x7f1200009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:50:24.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:24.285+0000 7f1211600700 1 -- 172.21.15.37:0/1459351230 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f120000e030 con 0x7f1214103670 2024-03-29T08:50:24.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:24.285+0000 7f1211600700 1 -- 172.21.15.37:0/1459351230 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f1200013de0 con 0x7f1214103670 2024-03-29T08:50:24.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:24.285+0000 7f1211600700 1 -- 172.21.15.37:0/1459351230 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f120001cd90 con 0x7f1214103670 2024-03-29T08:50:24.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:24.285+0000 7f121c200700 1 -- 172.21.15.37:0/1459351230 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f121410fd20 con 0x7f1214103670 2024-03-29T08:50:24.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:24.285+0000 7f121c200700 1 -- 172.21.15.37:0/1459351230 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12141102e0 con 0x7f1214103670 2024-03-29T08:50:24.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:24.286+0000 7f121c200700 1 -- 172.21.15.37:0/1459351230 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12140fe420 con 0x7f1214103670 2024-03-29T08:50:24.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:24.287+0000 7f1211600700 1 -- 172.21.15.37:0/1459351230 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f1200013930 con 0x7f1214103670 2024-03-29T08:50:24.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:24.288+0000 7f1211600700 1 --2- 172.21.15.37:0/1459351230 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f11fc06cc20 0x7f11fc06f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:50:24.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:24.288+0000 7f1211600700 1 -- 172.21.15.37:0/1459351230 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f120008d800 con 0x7f1214103670 2024-03-29T08:50:24.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:24.288+0000 7f1219800700 1 --2- 172.21.15.37:0/1459351230 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f11fc06cc20 0x7f11fc06f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:50:24.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:24.289+0000 7f1219800700 1 --2- 172.21.15.37:0/1459351230 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f11fc06cc20 0x7f11fc06f0d0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f1208004600 tx=0x7f120800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:50:24.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:24.292+0000 7f1211600700 1 -- 172.21.15.37:0/1459351230 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f120005b390 con 0x7f1214103670 2024-03-29T08:50:24.305 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:50:23 smithi148 ceph-mon[106692]: pgmap v1187: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:50:24.305 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:50:23 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/3174166462' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 26, "format": "json"}]: dispatch 2024-03-29T08:50:24.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:50:23 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/3962307418' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 27, "format": "json"}]: dispatch 2024-03-29T08:50:24.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:24.558+0000 7f121c200700 1 -- 172.21.15.37:0/1459351230 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 29, "format": "json"} v 0) v1 -- 0x7f1214109c70 con 0x7f1214103670 2024-03-29T08:50:24.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:24.559+0000 7f1211600700 1 -- 172.21.15.37:0/1459351230 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 29, "format": "json"}]=0 dumped fsmap epoch 29 v36) v1 ==== 107+0+4043 (secure 0 0 0) 0x7f120005af20 con 0x7f1214103670 2024-03-29T08:50:24.560 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:50:24.560 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34440,"name":"cephfs.smithi037.mmwgxe","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6829/1248142005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1248142005},{"type":"v1","addr":"172.21.15.37:6829","nonce":1248142005}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":24},{"gid":34444,"name":"cephfs.smithi037.iifsmh","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6827/2060882959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":2060882959},{"type":"v1","addr":"172.21.15.37:6827","nonce":2060882959}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":44395,"name":"cephfs.smithi148.sqteih","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6825/4078652242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":4078652242},{"type":"v1","addr":"172.21.15.148:6825","nonce":4078652242}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":21,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-29T08:03:29.241269+0000","modified":"2024-03-29T08:17:16.237238+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":82,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-03-29T08:50:24.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:24.561+0000 7f121c200700 1 -- 172.21.15.37:0/1459351230 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f11fc06cc20 msgr2=0x7f11fc06f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:50:24.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:24.561+0000 7f121c200700 1 --2- 172.21.15.37:0/1459351230 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f11fc06cc20 0x7f11fc06f0d0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f1208004600 tx=0x7f120800b040 comp rx=0 tx=0).stop 2024-03-29T08:50:24.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:24.561+0000 7f121c200700 1 -- 172.21.15.37:0/1459351230 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1214103670 msgr2=0x7f1214115e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:50:24.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:24.561+0000 7f121c200700 1 --2- 172.21.15.37:0/1459351230 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1214103670 0x7f1214115e50 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f1200008d50 tx=0x7f1200009e30 comp rx=0 tx=0).stop 2024-03-29T08:50:24.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:24.562+0000 7f121c200700 1 -- 172.21.15.37:0/1459351230 shutdown_connections 2024-03-29T08:50:24.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:24.562+0000 7f121c200700 1 --2- 172.21.15.37:0/1459351230 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f11fc06cc20 0x7f11fc06f0d0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:24.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:24.562+0000 7f121c200700 1 --2- 172.21.15.37:0/1459351230 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1214102cd0 0x7f1214115910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:24.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:24.562+0000 7f121c200700 1 --2- 172.21.15.37:0/1459351230 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1214103670 0x7f1214115e50 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:24.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:24.562+0000 7f121c200700 1 -- 172.21.15.37:0/1459351230 >> 172.21.15.37:0/1459351230 conn(0x7f12140f8820 msgr2=0x7f12140f9800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:50:24.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:24.562+0000 7f121c200700 1 -- 172.21.15.37:0/1459351230 shutdown_connections 2024-03-29T08:50:24.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:24.562+0000 7f121c200700 1 -- 172.21.15.37:0/1459351230 wait complete. 2024-03-29T08:50:24.564 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 29 2024-03-29T08:50:24.580 DEBUG:tasks.fs:max_mds not reduced in epoch 29 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-03-29T08:50:24.581 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 30 2024-03-29T08:50:24.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:24.775+0000 7efd3a000700 1 -- 172.21.15.37:0/3040615068 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efd34103670 msgr2=0x7efd340fcce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:50:24.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:24.775+0000 7efd3a000700 1 --2- 172.21.15.37:0/3040615068 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efd34103670 0x7efd340fcce0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7efd20004940 tx=0x7efd20009e30 comp rx=0 tx=0).stop 2024-03-29T08:50:24.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:24.775+0000 7efd3a000700 1 -- 172.21.15.37:0/3040615068 shutdown_connections 2024-03-29T08:50:24.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:24.775+0000 7efd3a000700 1 --2- 172.21.15.37:0/3040615068 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efd34103670 0x7efd340fcce0 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:24.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:24.775+0000 7efd3a000700 1 --2- 172.21.15.37:0/3040615068 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efd34102cd0 0x7efd341030a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:24.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:24.775+0000 7efd3a000700 1 -- 172.21.15.37:0/3040615068 >> 172.21.15.37:0/3040615068 conn(0x7efd340f8820 msgr2=0x7efd340fac30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:50:24.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:24.775+0000 7efd3a000700 1 -- 172.21.15.37:0/3040615068 shutdown_connections 2024-03-29T08:50:24.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:24.775+0000 7efd3a000700 1 -- 172.21.15.37:0/3040615068 wait complete. 2024-03-29T08:50:24.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:24.776+0000 7efd3a000700 1 Processor -- start 2024-03-29T08:50:24.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:24.776+0000 7efd3a000700 1 -- start start 2024-03-29T08:50:24.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:24.776+0000 7efd3a000700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efd34102cd0 0x7efd340702c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:50:24.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:24.776+0000 7efd3a000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efd34103670 0x7efd34070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:50:24.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:24.776+0000 7efd3a000700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd34072210 con 0x7efd34103670 2024-03-29T08:50:24.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:24.776+0000 7efd3a000700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd34070d40 con 0x7efd34102cd0 2024-03-29T08:50:24.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:24.777+0000 7efd32a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efd34103670 0x7efd34070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:50:24.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:24.777+0000 7efd32a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efd34103670 0x7efd34070800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52838/0 (socket says 172.21.15.37:52838) 2024-03-29T08:50:24.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:24.777+0000 7efd32a00700 1 -- 172.21.15.37:0/2324372503 learned_addr learned my addr 172.21.15.37:0/2324372503 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:50:24.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:24.777+0000 7efd33400700 1 --2- 172.21.15.37:0/2324372503 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efd34102cd0 0x7efd340702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:50:24.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:24.777+0000 7efd32a00700 1 -- 172.21.15.37:0/2324372503 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efd34102cd0 msgr2=0x7efd340702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:50:24.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:24.777+0000 7efd32a00700 1 --2- 172.21.15.37:0/2324372503 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efd34102cd0 0x7efd340702c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:24.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:24.777+0000 7efd32a00700 1 -- 172.21.15.37:0/2324372503 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd20005040 con 0x7efd34103670 2024-03-29T08:50:24.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:24.777+0000 7efd32a00700 1 --2- 172.21.15.37:0/2324372503 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efd34103670 0x7efd34070800 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7efd20009fd0 tx=0x7efd20009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:50:24.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:24.778+0000 7efd2b400700 1 -- 172.21.15.37:0/2324372503 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efd2000e030 con 0x7efd34103670 2024-03-29T08:50:24.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:24.778+0000 7efd2b400700 1 -- 172.21.15.37:0/2324372503 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7efd20013e00 con 0x7efd34103670 2024-03-29T08:50:24.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:24.778+0000 7efd3a000700 1 -- 172.21.15.37:0/2324372503 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd34070fc0 con 0x7efd34103670 2024-03-29T08:50:24.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:24.778+0000 7efd2b400700 1 -- 172.21.15.37:0/2324372503 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efd2001ccd0 con 0x7efd34103670 2024-03-29T08:50:24.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:24.778+0000 7efd3a000700 1 -- 172.21.15.37:0/2324372503 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd34119e70 con 0x7efd34103670 2024-03-29T08:50:24.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:24.779+0000 7efd3a000700 1 -- 172.21.15.37:0/2324372503 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd340fe420 con 0x7efd34103670 2024-03-29T08:50:24.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:24.781+0000 7efd2b400700 1 -- 172.21.15.37:0/2324372503 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7efd2001c500 con 0x7efd34103670 2024-03-29T08:50:24.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:24.781+0000 7efd2b400700 1 --2- 172.21.15.37:0/2324372503 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7efd1806cc20 0x7efd1806f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:50:24.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:24.781+0000 7efd2b400700 1 -- 172.21.15.37:0/2324372503 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7efd2008d890 con 0x7efd34103670 2024-03-29T08:50:24.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:24.781+0000 7efd33400700 1 --2- 172.21.15.37:0/2324372503 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7efd1806cc20 0x7efd1806f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:50:24.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:24.781+0000 7efd33400700 1 --2- 172.21.15.37:0/2324372503 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7efd1806cc20 0x7efd1806f0d0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7efd24009fd0 tx=0x7efd2400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:50:24.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:24.786+0000 7efd2b400700 1 -- 172.21.15.37:0/2324372503 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7efd200578d0 con 0x7efd34103670 2024-03-29T08:50:25.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:25.048+0000 7efd3a000700 1 -- 172.21.15.37:0/2324372503 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 30, "format": "json"} v 0) v1 -- 0x7efd3411a0c0 con 0x7efd34103670 2024-03-29T08:50:25.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:25.049+0000 7efd2b400700 1 -- 172.21.15.37:0/2324372503 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 30, "format": "json"}]=0 dumped fsmap epoch 30 v36) v1 ==== 107+0+4825 (secure 0 0 0) 0x7efd2005aef0 con 0x7efd34103670 2024-03-29T08:50:25.049 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:50:25.050 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34440,"name":"cephfs.smithi037.mmwgxe","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6829/1248142005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1248142005},{"type":"v1","addr":"172.21.15.37:6829","nonce":1248142005}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":24},{"gid":34444,"name":"cephfs.smithi037.iifsmh","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6827/2060882959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":2060882959},{"type":"v1","addr":"172.21.15.37:6827","nonce":2060882959}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":44395,"name":"cephfs.smithi148.sqteih","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6825/4078652242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":4078652242},{"type":"v1","addr":"172.21.15.148:6825","nonce":4078652242}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28},{"gid":45007,"name":"cephfs.smithi148.qukpjh","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6827/2394751105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2394751105},{"type":"v1","addr":"172.21.15.148:6827","nonce":2394751105}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":21,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-29T08:03:29.241269+0000","modified":"2024-03-29T08:17:16.237238+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":82,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-03-29T08:50:25.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:25.051+0000 7efd3a000700 1 -- 172.21.15.37:0/2324372503 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7efd1806cc20 msgr2=0x7efd1806f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:50:25.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:25.051+0000 7efd3a000700 1 --2- 172.21.15.37:0/2324372503 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7efd1806cc20 0x7efd1806f0d0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7efd24009fd0 tx=0x7efd2400b040 comp rx=0 tx=0).stop 2024-03-29T08:50:25.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:25.051+0000 7efd3a000700 1 -- 172.21.15.37:0/2324372503 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efd34103670 msgr2=0x7efd34070800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:50:25.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:25.051+0000 7efd3a000700 1 --2- 172.21.15.37:0/2324372503 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efd34103670 0x7efd34070800 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7efd20009fd0 tx=0x7efd20009e30 comp rx=0 tx=0).stop 2024-03-29T08:50:25.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:25.051+0000 7efd3a000700 1 -- 172.21.15.37:0/2324372503 shutdown_connections 2024-03-29T08:50:25.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:25.051+0000 7efd3a000700 1 --2- 172.21.15.37:0/2324372503 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7efd1806cc20 0x7efd1806f0d0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:25.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:25.051+0000 7efd3a000700 1 --2- 172.21.15.37:0/2324372503 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efd34102cd0 0x7efd340702c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:25.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:25.051+0000 7efd3a000700 1 --2- 172.21.15.37:0/2324372503 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efd34103670 0x7efd34070800 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:25.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:25.051+0000 7efd3a000700 1 -- 172.21.15.37:0/2324372503 >> 172.21.15.37:0/2324372503 conn(0x7efd340f8820 msgr2=0x7efd340f9890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:50:25.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:25.051+0000 7efd3a000700 1 -- 172.21.15.37:0/2324372503 shutdown_connections 2024-03-29T08:50:25.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:25.051+0000 7efd3a000700 1 -- 172.21.15.37:0/2324372503 wait complete. 2024-03-29T08:50:25.053 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 30 2024-03-29T08:50:25.069 DEBUG:tasks.fs:max_mds not reduced in epoch 30 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-03-29T08:50:25.070 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 31 2024-03-29T08:50:25.129 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:50:24 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/4102921764' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 28, "format": "json"}]: dispatch 2024-03-29T08:50:25.129 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:50:24 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/1459351230' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 29, "format": "json"}]: dispatch 2024-03-29T08:50:25.129 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:50:24 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:50:25.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:25.266+0000 7f9e14c00700 1 -- 172.21.15.37:0/663862367 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9e10103670 msgr2=0x7f9e100fcce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:50:25.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:25.266+0000 7f9e14c00700 1 --2- 172.21.15.37:0/663862367 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9e10103670 0x7f9e100fcce0 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f9dfc004940 tx=0x7f9dfc009e30 comp rx=0 tx=0).stop 2024-03-29T08:50:25.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:25.266+0000 7f9e14c00700 1 -- 172.21.15.37:0/663862367 shutdown_connections 2024-03-29T08:50:25.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:25.266+0000 7f9e14c00700 1 --2- 172.21.15.37:0/663862367 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9e10103670 0x7f9e100fcce0 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:25.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:25.266+0000 7f9e14c00700 1 --2- 172.21.15.37:0/663862367 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9e10102cd0 0x7f9e101030a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:25.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:25.266+0000 7f9e14c00700 1 -- 172.21.15.37:0/663862367 >> 172.21.15.37:0/663862367 conn(0x7f9e100f8840 msgr2=0x7f9e100fac50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:50:25.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:25.266+0000 7f9e14c00700 1 -- 172.21.15.37:0/663862367 shutdown_connections 2024-03-29T08:50:25.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:25.266+0000 7f9e14c00700 1 -- 172.21.15.37:0/663862367 wait complete. 2024-03-29T08:50:25.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:25.267+0000 7f9e14c00700 1 Processor -- start 2024-03-29T08:50:25.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:25.267+0000 7f9e14c00700 1 -- start start 2024-03-29T08:50:25.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:25.268+0000 7f9e14c00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9e10102cd0 0x7f9e101159a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:50:25.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:25.268+0000 7f9e14c00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9e10103670 0x7f9e10115ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:50:25.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:25.268+0000 7f9e14c00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e10116530 con 0x7f9e10103670 2024-03-29T08:50:25.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:25.268+0000 7f9e14c00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e1010fb30 con 0x7f9e10102cd0 2024-03-29T08:50:25.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:25.268+0000 7f9e0d400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9e10103670 0x7f9e10115ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:50:25.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:25.268+0000 7f9e0d400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9e10103670 0x7f9e10115ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52854/0 (socket says 172.21.15.37:52854) 2024-03-29T08:50:25.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:25.268+0000 7f9e0de00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9e10102cd0 0x7f9e101159a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:50:25.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:25.268+0000 7f9e0d400700 1 -- 172.21.15.37:0/4101145079 learned_addr learned my addr 172.21.15.37:0/4101145079 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:50:25.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:25.268+0000 7f9e0d400700 1 -- 172.21.15.37:0/4101145079 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9e10102cd0 msgr2=0x7f9e101159a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:50:25.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:25.268+0000 7f9e0d400700 1 --2- 172.21.15.37:0/4101145079 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9e10102cd0 0x7f9e101159a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:25.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:25.268+0000 7f9e0d400700 1 -- 172.21.15.37:0/4101145079 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dfc005040 con 0x7f9e10103670 2024-03-29T08:50:25.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:25.269+0000 7f9e0d400700 1 --2- 172.21.15.37:0/4101145079 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9e10103670 0x7f9e10115ee0 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f9dfc009390 tx=0x7f9dfc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:50:25.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:25.269+0000 7f9e06000700 1 -- 172.21.15.37:0/4101145079 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9dfc00e040 con 0x7f9e10103670 2024-03-29T08:50:25.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:25.269+0000 7f9e06000700 1 -- 172.21.15.37:0/4101145079 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f9dfc004290 con 0x7f9e10103670 2024-03-29T08:50:25.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:25.269+0000 7f9e14c00700 1 -- 172.21.15.37:0/4101145079 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e1010fdb0 con 0x7f9e10103670 2024-03-29T08:50:25.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:25.269+0000 7f9e06000700 1 -- 172.21.15.37:0/4101145079 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9dfc013e70 con 0x7f9e10103670 2024-03-29T08:50:25.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:25.269+0000 7f9e14c00700 1 -- 172.21.15.37:0/4101145079 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e10110370 con 0x7f9e10103670 2024-03-29T08:50:25.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:25.270+0000 7f9e14c00700 1 -- 172.21.15.37:0/4101145079 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e100fe420 con 0x7f9e10103670 2024-03-29T08:50:25.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:25.271+0000 7f9e06000700 1 -- 172.21.15.37:0/4101145079 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f9dfc013690 con 0x7f9e10103670 2024-03-29T08:50:25.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:25.272+0000 7f9e06000700 1 --2- 172.21.15.37:0/4101145079 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f9df406cc20 0x7f9df406f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:50:25.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:25.272+0000 7f9e06000700 1 -- 172.21.15.37:0/4101145079 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f9dfc08d1a0 con 0x7f9e10103670 2024-03-29T08:50:25.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:25.272+0000 7f9e0de00700 1 --2- 172.21.15.37:0/4101145079 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f9df406cc20 0x7f9df406f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:50:25.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:25.273+0000 7f9e0de00700 1 --2- 172.21.15.37:0/4101145079 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f9df406cc20 0x7f9df406f0d0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f9e00004f40 tx=0x7f9e0000b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:50:25.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:25.276+0000 7f9e06000700 1 -- 172.21.15.37:0/4101145079 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f9dfc05ad20 con 0x7f9e10103670 2024-03-29T08:50:25.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:50:24 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/4102921764' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 28, "format": "json"}]: dispatch 2024-03-29T08:50:25.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:50:24 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/1459351230' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 29, "format": "json"}]: dispatch 2024-03-29T08:50:25.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:50:24 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:50:25.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:25.543+0000 7f9e14c00700 1 -- 172.21.15.37:0/4101145079 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 31, "format": "json"} v 0) v1 -- 0x7f9e10064dc0 con 0x7f9e10103670 2024-03-29T08:50:25.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:25.544+0000 7f9e06000700 1 -- 172.21.15.37:0/4101145079 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 31, "format": "json"}]=0 dumped fsmap epoch 31 v36) v1 ==== 107+0+4824 (secure 0 0 0) 0x7f9dfc013420 con 0x7f9e10103670 2024-03-29T08:50:25.544 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:50:25.544 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34440,"name":"cephfs.smithi037.mmwgxe","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6829/1248142005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1248142005},{"type":"v1","addr":"172.21.15.37:6829","nonce":1248142005}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":24},{"gid":34444,"name":"cephfs.smithi037.iifsmh","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6827/2060882959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":2060882959},{"type":"v1","addr":"172.21.15.37:6827","nonce":2060882959}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":44395,"name":"cephfs.smithi148.sqteih","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6825/4078652242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":4078652242},{"type":"v1","addr":"172.21.15.148:6825","nonce":4078652242}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28},{"gid":45007,"name":"cephfs.smithi148.qukpjh","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6827/2394751105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2394751105},{"type":"v1","addr":"172.21.15.148:6827","nonce":2394751105}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":31,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-29T08:03:29.241269+0000","modified":"2024-03-29T08:48:22.193246+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":82,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-03-29T08:50:25.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:25.546+0000 7f9e14c00700 1 -- 172.21.15.37:0/4101145079 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f9df406cc20 msgr2=0x7f9df406f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:50:25.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:25.546+0000 7f9e14c00700 1 --2- 172.21.15.37:0/4101145079 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f9df406cc20 0x7f9df406f0d0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f9e00004f40 tx=0x7f9e0000b040 comp rx=0 tx=0).stop 2024-03-29T08:50:25.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:25.546+0000 7f9e14c00700 1 -- 172.21.15.37:0/4101145079 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9e10103670 msgr2=0x7f9e10115ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:50:25.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:25.546+0000 7f9e14c00700 1 --2- 172.21.15.37:0/4101145079 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9e10103670 0x7f9e10115ee0 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f9dfc009390 tx=0x7f9dfc009e30 comp rx=0 tx=0).stop 2024-03-29T08:50:25.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:25.546+0000 7f9e14c00700 1 -- 172.21.15.37:0/4101145079 shutdown_connections 2024-03-29T08:50:25.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:25.546+0000 7f9e14c00700 1 --2- 172.21.15.37:0/4101145079 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f9df406cc20 0x7f9df406f0d0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:25.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:25.546+0000 7f9e14c00700 1 --2- 172.21.15.37:0/4101145079 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9e10102cd0 0x7f9e101159a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:25.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:25.546+0000 7f9e14c00700 1 --2- 172.21.15.37:0/4101145079 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9e10103670 0x7f9e10115ee0 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:25.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:25.546+0000 7f9e14c00700 1 -- 172.21.15.37:0/4101145079 >> 172.21.15.37:0/4101145079 conn(0x7f9e100f8840 msgr2=0x7f9e100f9940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:50:25.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:25.546+0000 7f9e14c00700 1 -- 172.21.15.37:0/4101145079 shutdown_connections 2024-03-29T08:50:25.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:25.546+0000 7f9e14c00700 1 -- 172.21.15.37:0/4101145079 wait complete. 2024-03-29T08:50:25.548 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 31 2024-03-29T08:50:25.565 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 32 2024-03-29T08:50:25.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:25.759+0000 7f3224400700 1 -- 172.21.15.37:0/1552828145 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f321c102ce0 msgr2=0x7f321c1030b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:50:25.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:25.759+0000 7f3224400700 1 --2- 172.21.15.37:0/1552828145 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f321c102ce0 0x7f321c1030b0 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f3210004940 tx=0x7f3210009e30 comp rx=0 tx=0).stop 2024-03-29T08:50:25.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:25.759+0000 7f3224400700 1 -- 172.21.15.37:0/1552828145 shutdown_connections 2024-03-29T08:50:25.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:25.759+0000 7f3224400700 1 --2- 172.21.15.37:0/1552828145 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f321c103680 0x7f321c0fccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:25.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:25.759+0000 7f3224400700 1 --2- 172.21.15.37:0/1552828145 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f321c102ce0 0x7f321c1030b0 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:25.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:25.759+0000 7f3224400700 1 -- 172.21.15.37:0/1552828145 >> 172.21.15.37:0/1552828145 conn(0x7f321c0f8830 msgr2=0x7f321c0fac40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:50:25.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:25.759+0000 7f3224400700 1 -- 172.21.15.37:0/1552828145 shutdown_connections 2024-03-29T08:50:25.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:25.759+0000 7f3224400700 1 -- 172.21.15.37:0/1552828145 wait complete. 2024-03-29T08:50:25.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:25.760+0000 7f3224400700 1 Processor -- start 2024-03-29T08:50:25.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:25.760+0000 7f3224400700 1 -- start start 2024-03-29T08:50:25.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:25.760+0000 7f3224400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f321c102ce0 0x7f321c115950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:50:25.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:25.760+0000 7f3224400700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f321c103680 0x7f321c115e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:50:25.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:25.760+0000 7f3224400700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f321c1164e0 con 0x7f321c102ce0 2024-03-29T08:50:25.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:25.760+0000 7f3224400700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f321c10fae0 con 0x7f321c103680 2024-03-29T08:50:25.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:25.761+0000 7f3221a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f321c102ce0 0x7f321c115950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:50:25.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:25.761+0000 7f3221a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f321c102ce0 0x7f321c115950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52880/0 (socket says 172.21.15.37:52880) 2024-03-29T08:50:25.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:25.761+0000 7f3221a00700 1 -- 172.21.15.37:0/2802530878 learned_addr learned my addr 172.21.15.37:0/2802530878 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:50:25.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:25.761+0000 7f3221000700 1 --2- 172.21.15.37:0/2802530878 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f321c103680 0x7f321c115e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:50:25.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:25.761+0000 7f3221a00700 1 -- 172.21.15.37:0/2802530878 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f321c103680 msgr2=0x7f321c115e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:50:25.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:25.761+0000 7f3221a00700 1 --2- 172.21.15.37:0/2802530878 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f321c103680 0x7f321c115e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:25.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:25.761+0000 7f3221a00700 1 -- 172.21.15.37:0/2802530878 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3210005040 con 0x7f321c102ce0 2024-03-29T08:50:25.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:25.762+0000 7f3221a00700 1 --2- 172.21.15.37:0/2802530878 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f321c102ce0 0x7f321c115950 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f3210007f50 tx=0x7f3210009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:50:25.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:25.762+0000 7f3219600700 1 -- 172.21.15.37:0/2802530878 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3210021070 con 0x7f321c102ce0 2024-03-29T08:50:25.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:25.762+0000 7f3219600700 1 -- 172.21.15.37:0/2802530878 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f32100093b0 con 0x7f321c102ce0 2024-03-29T08:50:25.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:25.762+0000 7f3224400700 1 -- 172.21.15.37:0/2802530878 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f321c10fd60 con 0x7f321c102ce0 2024-03-29T08:50:25.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:25.762+0000 7f3219600700 1 -- 172.21.15.37:0/2802530878 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f321000e070 con 0x7f321c102ce0 2024-03-29T08:50:25.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:25.762+0000 7f3224400700 1 -- 172.21.15.37:0/2802530878 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f321c1102c0 con 0x7f321c102ce0 2024-03-29T08:50:25.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:25.763+0000 7f3224400700 1 -- 172.21.15.37:0/2802530878 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f321c0fe430 con 0x7f321c102ce0 2024-03-29T08:50:25.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:25.764+0000 7f3219600700 1 -- 172.21.15.37:0/2802530878 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f321001c400 con 0x7f321c102ce0 2024-03-29T08:50:25.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:25.765+0000 7f3219600700 1 --2- 172.21.15.37:0/2802530878 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f320406cc20 0x7f320406f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:50:25.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:25.765+0000 7f3219600700 1 -- 172.21.15.37:0/2802530878 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f321008d120 con 0x7f321c102ce0 2024-03-29T08:50:25.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:25.765+0000 7f3221000700 1 --2- 172.21.15.37:0/2802530878 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f320406cc20 0x7f320406f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:50:25.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:25.766+0000 7f3221000700 1 --2- 172.21.15.37:0/2802530878 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f320406cc20 0x7f320406f0d0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f321c110ca0 tx=0x7f320800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:50:25.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:25.770+0000 7f3219600700 1 -- 172.21.15.37:0/2802530878 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f321005abf0 con 0x7f321c102ce0 2024-03-29T08:50:26.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:26.037+0000 7f3224400700 1 -- 172.21.15.37:0/2802530878 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 32, "format": "json"} v 0) v1 -- 0x7f321c109d30 con 0x7f321c102ce0 2024-03-29T08:50:26.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:26.037+0000 7f3219600700 1 -- 172.21.15.37:0/2802530878 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 32, "format": "json"}]=0 dumped fsmap epoch 32 v36) v1 ==== 107+0+4847 (secure 0 0 0) 0x7f321005a780 con 0x7f321c102ce0 2024-03-29T08:50:26.037 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:50:26.037 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44395,"name":"cephfs.smithi148.sqteih","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6825/4078652242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":4078652242},{"type":"v1","addr":"172.21.15.148:6825","nonce":4078652242}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28},{"gid":45007,"name":"cephfs.smithi148.qukpjh","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6827/2394751105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2394751105},{"type":"v1","addr":"172.21.15.148:6827","nonce":2394751105}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-29T08:03:29.241269+0000","modified":"2024-03-29T08:48:22.199874+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":82,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":34440,"mds_1":34444},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34440":{"gid":34440,"name":"cephfs.smithi037.mmwgxe","rank":0,"incarnation":32,"state":"up:replay","state_seq":1,"addr":"172.21.15.37:6829/1248142005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1248142005},{"type":"v1","addr":"172.21.15.37:6829","nonce":1248142005}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_34444":{"gid":34444,"name":"cephfs.smithi037.iifsmh","rank":1,"incarnation":32,"state":"up:replay","state_seq":1,"addr":"172.21.15.37:6827/2060882959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":2060882959},{"type":"v1","addr":"172.21.15.37:6827","nonce":2060882959}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-03-29T08:50:26.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:26.040+0000 7f3224400700 1 -- 172.21.15.37:0/2802530878 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f320406cc20 msgr2=0x7f320406f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:50:26.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:26.040+0000 7f3224400700 1 --2- 172.21.15.37:0/2802530878 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f320406cc20 0x7f320406f0d0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f321c110ca0 tx=0x7f320800b040 comp rx=0 tx=0).stop 2024-03-29T08:50:26.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:26.040+0000 7f3224400700 1 -- 172.21.15.37:0/2802530878 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f321c102ce0 msgr2=0x7f321c115950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:50:26.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:26.040+0000 7f3224400700 1 --2- 172.21.15.37:0/2802530878 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f321c102ce0 0x7f321c115950 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f3210007f50 tx=0x7f3210009e30 comp rx=0 tx=0).stop 2024-03-29T08:50:26.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:26.040+0000 7f3224400700 1 -- 172.21.15.37:0/2802530878 shutdown_connections 2024-03-29T08:50:26.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:26.040+0000 7f3224400700 1 --2- 172.21.15.37:0/2802530878 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f320406cc20 0x7f320406f0d0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:26.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:26.040+0000 7f3224400700 1 --2- 172.21.15.37:0/2802530878 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f321c102ce0 0x7f321c115950 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:26.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:26.040+0000 7f3224400700 1 --2- 172.21.15.37:0/2802530878 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f321c103680 0x7f321c115e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:26.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:26.040+0000 7f3224400700 1 -- 172.21.15.37:0/2802530878 >> 172.21.15.37:0/2802530878 conn(0x7f321c0f8830 msgr2=0x7f321c0f9880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:50:26.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:26.040+0000 7f3224400700 1 -- 172.21.15.37:0/2802530878 shutdown_connections 2024-03-29T08:50:26.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:26.040+0000 7f3224400700 1 -- 172.21.15.37:0/2802530878 wait complete. 2024-03-29T08:50:26.041 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 32 2024-03-29T08:50:26.058 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 33 2024-03-29T08:50:26.176 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:50:25 smithi037 ceph-mon[142325]: pgmap v1188: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:50:26.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:50:25 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/2324372503' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 30, "format": "json"}]: dispatch 2024-03-29T08:50:26.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:50:25 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/4101145079' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 31, "format": "json"}]: dispatch 2024-03-29T08:50:26.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:26.252+0000 7f3360200700 1 -- 172.21.15.37:0/2197095012 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3358103670 msgr2=0x7f33580fcce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:50:26.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:26.252+0000 7f3360200700 1 --2- 172.21.15.37:0/2197095012 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3358103670 0x7f33580fcce0 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f3344004940 tx=0x7f3344009e30 comp rx=0 tx=0).stop 2024-03-29T08:50:26.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:26.252+0000 7f3360200700 1 -- 172.21.15.37:0/2197095012 shutdown_connections 2024-03-29T08:50:26.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:26.252+0000 7f3360200700 1 --2- 172.21.15.37:0/2197095012 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3358103670 0x7f33580fcce0 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:26.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:26.252+0000 7f3360200700 1 --2- 172.21.15.37:0/2197095012 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3358102cd0 0x7f33581030a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:26.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:26.252+0000 7f3360200700 1 -- 172.21.15.37:0/2197095012 >> 172.21.15.37:0/2197095012 conn(0x7f33580f8820 msgr2=0x7f33580fac30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:50:26.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:26.253+0000 7f3360200700 1 -- 172.21.15.37:0/2197095012 shutdown_connections 2024-03-29T08:50:26.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:26.253+0000 7f3360200700 1 -- 172.21.15.37:0/2197095012 wait complete. 2024-03-29T08:50:26.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:26.253+0000 7f3360200700 1 Processor -- start 2024-03-29T08:50:26.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:26.253+0000 7f3360200700 1 -- start start 2024-03-29T08:50:26.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:26.254+0000 7f3360200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3358102cd0 0x7f3358113780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:50:26.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:26.254+0000 7f3360200700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3358103670 0x7f3358113cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:50:26.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:26.254+0000 7f3360200700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3358114310 con 0x7f3358102cd0 2024-03-29T08:50:26.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:26.254+0000 7f3360200700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f335810d910 con 0x7f3358103670 2024-03-29T08:50:26.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:26.254+0000 7f335d800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3358102cd0 0x7f3358113780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:50:26.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:26.254+0000 7f335d800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3358102cd0 0x7f3358113780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52902/0 (socket says 172.21.15.37:52902) 2024-03-29T08:50:26.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:26.254+0000 7f335d800700 1 -- 172.21.15.37:0/4083930788 learned_addr learned my addr 172.21.15.37:0/4083930788 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:50:26.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:26.254+0000 7f335ce00700 1 --2- 172.21.15.37:0/4083930788 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3358103670 0x7f3358113cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:50:26.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:26.255+0000 7f335d800700 1 -- 172.21.15.37:0/4083930788 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3358103670 msgr2=0x7f3358113cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:50:26.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:26.255+0000 7f335d800700 1 --2- 172.21.15.37:0/4083930788 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3358103670 0x7f3358113cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:26.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:26.255+0000 7f335d800700 1 -- 172.21.15.37:0/4083930788 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3344005040 con 0x7f3358102cd0 2024-03-29T08:50:26.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:26.255+0000 7f335d800700 1 --2- 172.21.15.37:0/4083930788 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3358102cd0 0x7f3358113780 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f334c00c8c0 tx=0x7f334c0044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:50:26.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:26.255+0000 7f3355600700 1 -- 172.21.15.37:0/4083930788 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f334c00a760 con 0x7f3358102cd0 2024-03-29T08:50:26.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:26.255+0000 7f3355600700 1 -- 172.21.15.37:0/4083930788 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f334c00ad70 con 0x7f3358102cd0 2024-03-29T08:50:26.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:26.255+0000 7f3360200700 1 -- 172.21.15.37:0/4083930788 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f335810dbf0 con 0x7f3358102cd0 2024-03-29T08:50:26.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:26.255+0000 7f3355600700 1 -- 172.21.15.37:0/4083930788 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f334c0113a0 con 0x7f3358102cd0 2024-03-29T08:50:26.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:26.255+0000 7f3360200700 1 -- 172.21.15.37:0/4083930788 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f335810e180 con 0x7f3358102cd0 2024-03-29T08:50:26.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:26.256+0000 7f3360200700 1 -- 172.21.15.37:0/4083930788 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33580fe420 con 0x7f3358102cd0 2024-03-29T08:50:26.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:26.257+0000 7f3355600700 1 -- 172.21.15.37:0/4083930788 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f334c012400 con 0x7f3358102cd0 2024-03-29T08:50:26.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:26.258+0000 7f3355600700 1 --2- 172.21.15.37:0/4083930788 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f334006cc20 0x7f334006f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:50:26.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:26.258+0000 7f3355600700 1 -- 172.21.15.37:0/4083930788 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f334c08bf60 con 0x7f3358102cd0 2024-03-29T08:50:26.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:26.258+0000 7f335ce00700 1 --2- 172.21.15.37:0/4083930788 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f334006cc20 0x7f334006f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:50:26.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:26.259+0000 7f335ce00700 1 --2- 172.21.15.37:0/4083930788 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f334006cc20 0x7f334006f0d0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f3344004890 tx=0x7f3344009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:50:26.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:26.263+0000 7f3355600700 1 -- 172.21.15.37:0/4083930788 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f334c0bf9f0 con 0x7f3358102cd0 2024-03-29T08:50:26.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:50:25 smithi148 ceph-mon[106692]: pgmap v1188: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:50:26.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:50:25 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/2324372503' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 30, "format": "json"}]: dispatch 2024-03-29T08:50:26.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:50:25 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/4101145079' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 31, "format": "json"}]: dispatch 2024-03-29T08:50:26.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:26.526+0000 7f3360200700 1 -- 172.21.15.37:0/4083930788 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 33, "format": "json"} v 0) v1 -- 0x7f3358109c70 con 0x7f3358102cd0 2024-03-29T08:50:26.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:26.527+0000 7f3355600700 1 -- 172.21.15.37:0/4083930788 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 33, "format": "json"}]=0 dumped fsmap epoch 33 v36) v1 ==== 107+0+4853 (secure 0 0 0) 0x7f334c059af0 con 0x7f3358102cd0 2024-03-29T08:50:26.527 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:50:26.527 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44395,"name":"cephfs.smithi148.sqteih","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6825/4078652242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":4078652242},{"type":"v1","addr":"172.21.15.148:6825","nonce":4078652242}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28},{"gid":45007,"name":"cephfs.smithi148.qukpjh","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6827/2394751105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2394751105},{"type":"v1","addr":"172.21.15.148:6827","nonce":2394751105}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-29T08:03:29.241269+0000","modified":"2024-03-29T08:48:29.969216+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":82,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":34440,"mds_1":34444},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34440":{"gid":34440,"name":"cephfs.smithi037.mmwgxe","rank":0,"incarnation":32,"state":"up:resolve","state_seq":467,"addr":"172.21.15.37:6829/1248142005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1248142005},{"type":"v1","addr":"172.21.15.37:6829","nonce":1248142005}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_34444":{"gid":34444,"name":"cephfs.smithi037.iifsmh","rank":1,"incarnation":32,"state":"up:resolve","state_seq":462,"addr":"172.21.15.37:6827/2060882959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":2060882959},{"type":"v1","addr":"172.21.15.37:6827","nonce":2060882959}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-03-29T08:50:26.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:26.530+0000 7f3360200700 1 -- 172.21.15.37:0/4083930788 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f334006cc20 msgr2=0x7f334006f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:50:26.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:26.530+0000 7f3360200700 1 --2- 172.21.15.37:0/4083930788 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f334006cc20 0x7f334006f0d0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f3344004890 tx=0x7f3344009e30 comp rx=0 tx=0).stop 2024-03-29T08:50:26.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:26.530+0000 7f3360200700 1 -- 172.21.15.37:0/4083930788 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3358102cd0 msgr2=0x7f3358113780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:50:26.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:26.530+0000 7f3360200700 1 --2- 172.21.15.37:0/4083930788 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3358102cd0 0x7f3358113780 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f334c00c8c0 tx=0x7f334c0044c0 comp rx=0 tx=0).stop 2024-03-29T08:50:26.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:26.530+0000 7f3360200700 1 -- 172.21.15.37:0/4083930788 shutdown_connections 2024-03-29T08:50:26.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:26.530+0000 7f3360200700 1 --2- 172.21.15.37:0/4083930788 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f334006cc20 0x7f334006f0d0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:26.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:26.530+0000 7f3360200700 1 --2- 172.21.15.37:0/4083930788 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3358102cd0 0x7f3358113780 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:26.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:26.530+0000 7f3360200700 1 --2- 172.21.15.37:0/4083930788 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3358103670 0x7f3358113cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:26.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:26.530+0000 7f3360200700 1 -- 172.21.15.37:0/4083930788 >> 172.21.15.37:0/4083930788 conn(0x7f33580f8820 msgr2=0x7f33580f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:50:26.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:26.530+0000 7f3360200700 1 -- 172.21.15.37:0/4083930788 shutdown_connections 2024-03-29T08:50:26.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:26.530+0000 7f3360200700 1 -- 172.21.15.37:0/4083930788 wait complete. 2024-03-29T08:50:26.530 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 33 2024-03-29T08:50:26.547 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 34 2024-03-29T08:50:26.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:26.742+0000 7f96aa600700 1 -- 172.21.15.37:0/3608581057 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f96a4103670 msgr2=0x7f96a40fcce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:50:26.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:26.742+0000 7f96aa600700 1 --2- 172.21.15.37:0/3608581057 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f96a4103670 0x7f96a40fcce0 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f9690004940 tx=0x7f9690009e30 comp rx=0 tx=0).stop 2024-03-29T08:50:26.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:26.742+0000 7f96aa600700 1 -- 172.21.15.37:0/3608581057 shutdown_connections 2024-03-29T08:50:26.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:26.742+0000 7f96aa600700 1 --2- 172.21.15.37:0/3608581057 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f96a4103670 0x7f96a40fcce0 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:26.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:26.742+0000 7f96aa600700 1 --2- 172.21.15.37:0/3608581057 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f96a4102cd0 0x7f96a41030a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:26.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:26.742+0000 7f96aa600700 1 -- 172.21.15.37:0/3608581057 >> 172.21.15.37:0/3608581057 conn(0x7f96a40f8820 msgr2=0x7f96a40fac30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:50:26.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:26.742+0000 7f96aa600700 1 -- 172.21.15.37:0/3608581057 shutdown_connections 2024-03-29T08:50:26.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:26.742+0000 7f96aa600700 1 -- 172.21.15.37:0/3608581057 wait complete. 2024-03-29T08:50:26.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:26.743+0000 7f96aa600700 1 Processor -- start 2024-03-29T08:50:26.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:26.743+0000 7f96aa600700 1 -- start start 2024-03-29T08:50:26.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:26.744+0000 7f96aa600700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f96a4102cd0 0x7f96a4115a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:50:26.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:26.744+0000 7f96aa600700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f96a4103670 0x7f96a4115f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:50:26.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:26.744+0000 7f96aa600700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96a41165a0 con 0x7f96a4103670 2024-03-29T08:50:26.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:26.744+0000 7f96aa600700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96a410fba0 con 0x7f96a4102cd0 2024-03-29T08:50:26.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:26.744+0000 7f96a2a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f96a4103670 0x7f96a4115f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:50:26.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:26.744+0000 7f96a2a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f96a4103670 0x7f96a4115f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52910/0 (socket says 172.21.15.37:52910) 2024-03-29T08:50:26.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:26.744+0000 7f96a2a00700 1 -- 172.21.15.37:0/325759127 learned_addr learned my addr 172.21.15.37:0/325759127 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:50:26.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:26.744+0000 7f96a3400700 1 --2- 172.21.15.37:0/325759127 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f96a4102cd0 0x7f96a4115a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:50:26.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:26.744+0000 7f96a2a00700 1 -- 172.21.15.37:0/325759127 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f96a4102cd0 msgr2=0x7f96a4115a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:50:26.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:26.744+0000 7f96a2a00700 1 --2- 172.21.15.37:0/325759127 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f96a4102cd0 0x7f96a4115a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:26.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:26.744+0000 7f96a2a00700 1 -- 172.21.15.37:0/325759127 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9690005040 con 0x7f96a4103670 2024-03-29T08:50:26.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:26.744+0000 7f96a3400700 1 --2- 172.21.15.37:0/325759127 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f96a4102cd0 0x7f96a4115a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:50:26.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:26.744+0000 7f96a2a00700 1 --2- 172.21.15.37:0/325759127 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f96a4103670 0x7f96a4115f50 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f96900093b0 tx=0x7f9690009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:50:26.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:26.745+0000 7f969b400700 1 -- 172.21.15.37:0/325759127 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f969000e040 con 0x7f96a4103670 2024-03-29T08:50:26.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:26.745+0000 7f969b400700 1 -- 172.21.15.37:0/325759127 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f9690013420 con 0x7f96a4103670 2024-03-29T08:50:26.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:26.745+0000 7f96aa600700 1 -- 172.21.15.37:0/325759127 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96a410fe20 con 0x7f96a4103670 2024-03-29T08:50:26.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:26.745+0000 7f969b400700 1 -- 172.21.15.37:0/325759127 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f969001c630 con 0x7f96a4103670 2024-03-29T08:50:26.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:26.745+0000 7f96aa600700 1 -- 172.21.15.37:0/325759127 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96a4110360 con 0x7f96a4103670 2024-03-29T08:50:26.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:26.746+0000 7f96aa600700 1 -- 172.21.15.37:0/325759127 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96a40fe420 con 0x7f96a4103670 2024-03-29T08:50:26.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:26.747+0000 7f969b400700 1 -- 172.21.15.37:0/325759127 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f96900139e0 con 0x7f96a4103670 2024-03-29T08:50:26.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:26.748+0000 7f969b400700 1 --2- 172.21.15.37:0/325759127 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f968806cc20 0x7f968806f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:50:26.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:26.748+0000 7f969b400700 1 -- 172.21.15.37:0/325759127 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f969008d300 con 0x7f96a4103670 2024-03-29T08:50:26.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:26.748+0000 7f96a3400700 1 --2- 172.21.15.37:0/325759127 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f968806cc20 0x7f968806f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:50:26.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:26.749+0000 7f96a3400700 1 --2- 172.21.15.37:0/325759127 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f968806cc20 0x7f968806f0d0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f9694009fd0 tx=0x7f969400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:50:26.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:26.752+0000 7f969b400700 1 -- 172.21.15.37:0/325759127 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f969005ae00 con 0x7f96a4103670 2024-03-29T08:50:27.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:27.019+0000 7f96aa600700 1 -- 172.21.15.37:0/325759127 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 34, "format": "json"} v 0) v1 -- 0x7f96a4109c70 con 0x7f96a4103670 2024-03-29T08:50:27.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:27.019+0000 7f969b400700 1 -- 172.21.15.37:0/325759127 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 34, "format": "json"}]=0 dumped fsmap epoch 34 v36) v1 ==== 107+0+4857 (secure 0 0 0) 0x7f969005a990 con 0x7f96a4103670 2024-03-29T08:50:27.019 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:50:27.019 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44395,"name":"cephfs.smithi148.sqteih","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6825/4078652242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":4078652242},{"type":"v1","addr":"172.21.15.148:6825","nonce":4078652242}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28},{"gid":45007,"name":"cephfs.smithi148.qukpjh","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6827/2394751105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2394751105},{"type":"v1","addr":"172.21.15.148:6827","nonce":2394751105}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-29T08:03:29.241269+0000","modified":"2024-03-29T08:48:30.989522+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":82,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":34440,"mds_1":34444},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34440":{"gid":34440,"name":"cephfs.smithi037.mmwgxe","rank":0,"incarnation":32,"state":"up:reconnect","state_seq":468,"addr":"172.21.15.37:6829/1248142005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1248142005},{"type":"v1","addr":"172.21.15.37:6829","nonce":1248142005}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_34444":{"gid":34444,"name":"cephfs.smithi037.iifsmh","rank":1,"incarnation":32,"state":"up:reconnect","state_seq":463,"addr":"172.21.15.37:6827/2060882959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":2060882959},{"type":"v1","addr":"172.21.15.37:6827","nonce":2060882959}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-03-29T08:50:27.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:27.022+0000 7f96aa600700 1 -- 172.21.15.37:0/325759127 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f968806cc20 msgr2=0x7f968806f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:50:27.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:27.022+0000 7f96aa600700 1 --2- 172.21.15.37:0/325759127 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f968806cc20 0x7f968806f0d0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f9694009fd0 tx=0x7f969400b040 comp rx=0 tx=0).stop 2024-03-29T08:50:27.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:27.022+0000 7f96aa600700 1 -- 172.21.15.37:0/325759127 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f96a4103670 msgr2=0x7f96a4115f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:50:27.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:27.022+0000 7f96aa600700 1 --2- 172.21.15.37:0/325759127 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f96a4103670 0x7f96a4115f50 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f96900093b0 tx=0x7f9690009e30 comp rx=0 tx=0).stop 2024-03-29T08:50:27.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:27.022+0000 7f96aa600700 1 -- 172.21.15.37:0/325759127 shutdown_connections 2024-03-29T08:50:27.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:27.022+0000 7f96aa600700 1 --2- 172.21.15.37:0/325759127 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f968806cc20 0x7f968806f0d0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:27.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:27.022+0000 7f96aa600700 1 --2- 172.21.15.37:0/325759127 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f96a4102cd0 0x7f96a4115a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:27.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:27.022+0000 7f96aa600700 1 --2- 172.21.15.37:0/325759127 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f96a4103670 0x7f96a4115f50 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:27.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:27.022+0000 7f96aa600700 1 -- 172.21.15.37:0/325759127 >> 172.21.15.37:0/325759127 conn(0x7f96a40f8820 msgr2=0x7f96a40f9880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:50:27.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:27.022+0000 7f96aa600700 1 -- 172.21.15.37:0/325759127 shutdown_connections 2024-03-29T08:50:27.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:27.022+0000 7f96aa600700 1 -- 172.21.15.37:0/325759127 wait complete. 2024-03-29T08:50:27.023 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 34 2024-03-29T08:50:27.040 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 35 2024-03-29T08:50:27.176 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:50:26 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/2802530878' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 32, "format": "json"}]: dispatch 2024-03-29T08:50:27.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:50:26 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/4083930788' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 33, "format": "json"}]: dispatch 2024-03-29T08:50:27.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:27.236+0000 7fe33c400700 1 -- 172.21.15.37:0/4147260762 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe334102cd0 msgr2=0x7fe3341030a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:50:27.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:27.236+0000 7fe33c400700 1 --2- 172.21.15.37:0/4147260762 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe334102cd0 0x7fe3341030a0 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7fe328004940 tx=0x7fe328009e30 comp rx=0 tx=0).stop 2024-03-29T08:50:27.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:27.236+0000 7fe33c400700 1 -- 172.21.15.37:0/4147260762 shutdown_connections 2024-03-29T08:50:27.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:27.236+0000 7fe33c400700 1 --2- 172.21.15.37:0/4147260762 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe334103670 0x7fe3340fcce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:27.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:27.236+0000 7fe33c400700 1 --2- 172.21.15.37:0/4147260762 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe334102cd0 0x7fe3341030a0 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:27.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:27.236+0000 7fe33c400700 1 -- 172.21.15.37:0/4147260762 >> 172.21.15.37:0/4147260762 conn(0x7fe3340f8820 msgr2=0x7fe3340fac30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:50:27.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:27.236+0000 7fe33c400700 1 -- 172.21.15.37:0/4147260762 shutdown_connections 2024-03-29T08:50:27.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:27.236+0000 7fe33c400700 1 -- 172.21.15.37:0/4147260762 wait complete. 2024-03-29T08:50:27.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:27.237+0000 7fe33c400700 1 Processor -- start 2024-03-29T08:50:27.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:27.237+0000 7fe33c400700 1 -- start start 2024-03-29T08:50:27.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:27.237+0000 7fe33c400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe334102cd0 0x7fe3340702c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:50:27.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:27.237+0000 7fe33c400700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe334103670 0x7fe334070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:50:27.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:27.237+0000 7fe33c400700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe334072210 con 0x7fe334102cd0 2024-03-29T08:50:27.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:27.237+0000 7fe33c400700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe334070d40 con 0x7fe334103670 2024-03-29T08:50:27.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:27.237+0000 7fe339a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe334102cd0 0x7fe3340702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:50:27.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:27.237+0000 7fe339a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe334102cd0 0x7fe3340702c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52930/0 (socket says 172.21.15.37:52930) 2024-03-29T08:50:27.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:27.237+0000 7fe339a00700 1 -- 172.21.15.37:0/2690767894 learned_addr learned my addr 172.21.15.37:0/2690767894 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:50:27.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:27.238+0000 7fe339a00700 1 -- 172.21.15.37:0/2690767894 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe334103670 msgr2=0x7fe334070800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:50:27.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:27.238+0000 7fe339a00700 1 --2- 172.21.15.37:0/2690767894 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe334103670 0x7fe334070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:27.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:27.238+0000 7fe339a00700 1 -- 172.21.15.37:0/2690767894 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe328005040 con 0x7fe334102cd0 2024-03-29T08:50:27.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:27.238+0000 7fe339a00700 1 --2- 172.21.15.37:0/2690767894 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe334102cd0 0x7fe3340702c0 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7fe328009360 tx=0x7fe328009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:50:27.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:27.238+0000 7fe331600700 1 -- 172.21.15.37:0/2690767894 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe32800e040 con 0x7fe334102cd0 2024-03-29T08:50:27.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:27.238+0000 7fe331600700 1 -- 172.21.15.37:0/2690767894 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fe328013420 con 0x7fe334102cd0 2024-03-29T08:50:27.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:27.238+0000 7fe331600700 1 -- 172.21.15.37:0/2690767894 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe32801c5f0 con 0x7fe334102cd0 2024-03-29T08:50:27.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:27.238+0000 7fe33c400700 1 -- 172.21.15.37:0/2690767894 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe334070fc0 con 0x7fe334102cd0 2024-03-29T08:50:27.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:27.238+0000 7fe33c400700 1 -- 172.21.15.37:0/2690767894 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe334119f10 con 0x7fe334102cd0 2024-03-29T08:50:27.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:27.239+0000 7fe33c400700 1 -- 172.21.15.37:0/2690767894 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3340fe420 con 0x7fe334102cd0 2024-03-29T08:50:27.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:27.240+0000 7fe331600700 1 -- 172.21.15.37:0/2690767894 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fe328013590 con 0x7fe334102cd0 2024-03-29T08:50:27.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:27.241+0000 7fe331600700 1 --2- 172.21.15.37:0/2690767894 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fe31c06cc20 0x7fe31c06f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:50:27.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:27.241+0000 7fe331600700 1 -- 172.21.15.37:0/2690767894 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fe32808d260 con 0x7fe334102cd0 2024-03-29T08:50:27.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:27.241+0000 7fe339000700 1 --2- 172.21.15.37:0/2690767894 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fe31c06cc20 0x7fe31c06f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:50:27.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:27.242+0000 7fe339000700 1 --2- 172.21.15.37:0/2690767894 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fe31c06cc20 0x7fe31c06f0d0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fe3340719f0 tx=0x7fe320009ce0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:50:27.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:27.246+0000 7fe331600700 1 -- 172.21.15.37:0/2690767894 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fe3280572a0 con 0x7fe334102cd0 2024-03-29T08:50:27.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:50:26 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/2802530878' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 32, "format": "json"}]: dispatch 2024-03-29T08:50:27.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:50:26 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/4083930788' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 33, "format": "json"}]: dispatch 2024-03-29T08:50:27.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:27.509+0000 7fe33c400700 1 -- 172.21.15.37:0/2690767894 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 35, "format": "json"} v 0) v1 -- 0x7fe33411a160 con 0x7fe334102cd0 2024-03-29T08:50:27.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:27.510+0000 7fe331600700 1 -- 172.21.15.37:0/2690767894 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 35, "format": "json"}]=0 dumped fsmap epoch 35 v36) v1 ==== 107+0+4851 (secure 0 0 0) 0x7fe32805a8c0 con 0x7fe334102cd0 2024-03-29T08:50:27.510 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:50:27.510 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":35,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44395,"name":"cephfs.smithi148.sqteih","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6825/4078652242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":4078652242},{"type":"v1","addr":"172.21.15.148:6825","nonce":4078652242}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28},{"gid":45007,"name":"cephfs.smithi148.qukpjh","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6827/2394751105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2394751105},{"type":"v1","addr":"172.21.15.148:6827","nonce":2394751105}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-29T08:03:29.241269+0000","modified":"2024-03-29T08:48:31.993239+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":82,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":34440,"mds_1":34444},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34440":{"gid":34440,"name":"cephfs.smithi037.mmwgxe","rank":0,"incarnation":32,"state":"up:rejoin","state_seq":469,"addr":"172.21.15.37:6829/1248142005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1248142005},{"type":"v1","addr":"172.21.15.37:6829","nonce":1248142005}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_34444":{"gid":34444,"name":"cephfs.smithi037.iifsmh","rank":1,"incarnation":32,"state":"up:rejoin","state_seq":464,"addr":"172.21.15.37:6827/2060882959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":2060882959},{"type":"v1","addr":"172.21.15.37:6827","nonce":2060882959}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-03-29T08:50:27.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:27.513+0000 7fe33c400700 1 -- 172.21.15.37:0/2690767894 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fe31c06cc20 msgr2=0x7fe31c06f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:50:27.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:27.513+0000 7fe33c400700 1 --2- 172.21.15.37:0/2690767894 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fe31c06cc20 0x7fe31c06f0d0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fe3340719f0 tx=0x7fe320009ce0 comp rx=0 tx=0).stop 2024-03-29T08:50:27.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:27.513+0000 7fe33c400700 1 -- 172.21.15.37:0/2690767894 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe334102cd0 msgr2=0x7fe3340702c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:50:27.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:27.513+0000 7fe33c400700 1 --2- 172.21.15.37:0/2690767894 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe334102cd0 0x7fe3340702c0 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7fe328009360 tx=0x7fe328009e30 comp rx=0 tx=0).stop 2024-03-29T08:50:27.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:27.513+0000 7fe33c400700 1 -- 172.21.15.37:0/2690767894 shutdown_connections 2024-03-29T08:50:27.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:27.513+0000 7fe33c400700 1 --2- 172.21.15.37:0/2690767894 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fe31c06cc20 0x7fe31c06f0d0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:27.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:27.513+0000 7fe33c400700 1 --2- 172.21.15.37:0/2690767894 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe334102cd0 0x7fe3340702c0 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:27.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:27.513+0000 7fe33c400700 1 --2- 172.21.15.37:0/2690767894 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe334103670 0x7fe334070800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:27.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:27.513+0000 7fe33c400700 1 -- 172.21.15.37:0/2690767894 >> 172.21.15.37:0/2690767894 conn(0x7fe3340f8820 msgr2=0x7fe3340f98f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:50:27.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:27.513+0000 7fe33c400700 1 -- 172.21.15.37:0/2690767894 shutdown_connections 2024-03-29T08:50:27.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:27.513+0000 7fe33c400700 1 -- 172.21.15.37:0/2690767894 wait complete. 2024-03-29T08:50:27.513 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 35 2024-03-29T08:50:27.530 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-03-29T08:50:27.547 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-03-29T08:50:27.547 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-03-29T08:50:27.547 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T08:50:27.593 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-03-29T08:50:27.593 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T08:50:27.680 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T08:50:27.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:27.913+0000 7f6f29000700 1 -- 172.21.15.37:0/590706474 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6f24070c70 msgr2=0x7f6f24109040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:50:27.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:27.913+0000 7f6f29000700 1 --2- 172.21.15.37:0/590706474 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6f24070c70 0x7f6f24109040 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f6f10004970 tx=0x7f6f10009e30 comp rx=0 tx=0).stop 2024-03-29T08:50:27.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:27.913+0000 7f6f29000700 1 -- 172.21.15.37:0/590706474 shutdown_connections 2024-03-29T08:50:27.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:27.913+0000 7f6f29000700 1 --2- 172.21.15.37:0/590706474 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6f24070c70 0x7f6f24109040 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:27.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:27.913+0000 7f6f29000700 1 --2- 172.21.15.37:0/590706474 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6f24070360 0x7f6f24070730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:27.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:27.913+0000 7f6f29000700 1 -- 172.21.15.37:0/590706474 >> 172.21.15.37:0/590706474 conn(0x7f6f240f8840 msgr2=0x7f6f240fac50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:50:27.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:27.914+0000 7f6f29000700 1 -- 172.21.15.37:0/590706474 shutdown_connections 2024-03-29T08:50:27.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:27.914+0000 7f6f29000700 1 -- 172.21.15.37:0/590706474 wait complete. 2024-03-29T08:50:27.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:27.914+0000 7f6f29000700 1 Processor -- start 2024-03-29T08:50:27.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:27.914+0000 7f6f29000700 1 -- start start 2024-03-29T08:50:27.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:27.915+0000 7f6f29000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6f24070360 0x7f6f24115950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:50:27.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:27.915+0000 7f6f29000700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6f24070c70 0x7f6f24115e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:50:27.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:27.915+0000 7f6f29000700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f241164e0 con 0x7f6f24070360 2024-03-29T08:50:27.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:27.915+0000 7f6f29000700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f2410fae0 con 0x7f6f24070c70 2024-03-29T08:50:27.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:27.915+0000 7f6f21e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6f24070360 0x7f6f24115950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:50:27.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:27.915+0000 7f6f21e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6f24070360 0x7f6f24115950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52940/0 (socket says 172.21.15.37:52940) 2024-03-29T08:50:27.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:27.915+0000 7f6f21e00700 1 -- 172.21.15.37:0/210993223 learned_addr learned my addr 172.21.15.37:0/210993223 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:50:27.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:27.915+0000 7f6f21400700 1 --2- 172.21.15.37:0/210993223 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6f24070c70 0x7f6f24115e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:50:27.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:27.915+0000 7f6f21e00700 1 -- 172.21.15.37:0/210993223 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6f24070c70 msgr2=0x7f6f24115e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:50:27.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:27.915+0000 7f6f21e00700 1 --2- 172.21.15.37:0/210993223 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6f24070c70 0x7f6f24115e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:27.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:27.915+0000 7f6f21e00700 1 -- 172.21.15.37:0/210993223 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f10005040 con 0x7f6f24070360 2024-03-29T08:50:27.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:27.915+0000 7f6f21400700 1 --2- 172.21.15.37:0/210993223 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6f24070c70 0x7f6f24115e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:50:27.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:27.916+0000 7f6f21e00700 1 --2- 172.21.15.37:0/210993223 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6f24070360 0x7f6f24115950 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f6f14002a50 tx=0x7f6f140046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:50:27.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:27.916+0000 7f6f1a000700 1 -- 172.21.15.37:0/210993223 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6f14007cc0 con 0x7f6f24070360 2024-03-29T08:50:27.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:27.916+0000 7f6f29000700 1 -- 172.21.15.37:0/210993223 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f2410fdc0 con 0x7f6f24070360 2024-03-29T08:50:27.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:27.916+0000 7f6f1a000700 1 -- 172.21.15.37:0/210993223 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f6f1400f040 con 0x7f6f24070360 2024-03-29T08:50:27.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:27.916+0000 7f6f1a000700 1 -- 172.21.15.37:0/210993223 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6f14009540 con 0x7f6f24070360 2024-03-29T08:50:27.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:27.916+0000 7f6f29000700 1 -- 172.21.15.37:0/210993223 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f241102d0 con 0x7f6f24070360 2024-03-29T08:50:27.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:27.917+0000 7f6f29000700 1 -- 172.21.15.37:0/210993223 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f24071aa0 con 0x7f6f24070360 2024-03-29T08:50:27.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:27.918+0000 7f6f1a000700 1 -- 172.21.15.37:0/210993223 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f6f140096a0 con 0x7f6f24070360 2024-03-29T08:50:27.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:27.919+0000 7f6f1a000700 1 --2- 172.21.15.37:0/210993223 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f6f0806cc20 0x7f6f0806f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:50:27.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:27.919+0000 7f6f1a000700 1 -- 172.21.15.37:0/210993223 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f6f1408bee0 con 0x7f6f24070360 2024-03-29T08:50:27.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:27.919+0000 7f6f21400700 1 --2- 172.21.15.37:0/210993223 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f6f0806cc20 0x7f6f0806f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:50:27.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:27.920+0000 7f6f21400700 1 --2- 172.21.15.37:0/210993223 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f6f0806cc20 0x7f6f0806f0d0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f6f24110dc0 tx=0x7f6f10009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:50:27.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:27.923+0000 7f6f1a000700 1 -- 172.21.15.37:0/210993223 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f6f140598b0 con 0x7f6f24070360 2024-03-29T08:50:28.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:28.138+0000 7f6f29000700 1 -- 172.21.15.37:0/210993223 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6f2410bee0 con 0x7f6f24070360 2024-03-29T08:50:28.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:28.139+0000 7f6f1a000700 1 -- 172.21.15.37:0/210993223 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+16549 (secure 0 0 0) 0x7f6f14059440 con 0x7f6f24070360 2024-03-29T08:50:28.139 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:50:28.139 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":86,"fsid":"99b492c2-eda1-11ee-b647-cb9ed24678a4","created":"2024-03-29T07:55:23.673934+0000","modified":"2024-03-29T08:33:15.225558+0000","last_up_change":"2024-03-29T08:17:12.117876+0000","last_in_change":"2024-03-29T08:01:43.066485+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":20,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T08:00:32.022416+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs.cephfs.meta","create_time":"2024-03-29T08:03:28.176923+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs.cephfs.data","create_time":"2024-03-29T08:03:28.754148+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dba18fb6-b27d-4258-b769-dc89f4fef44d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":50,"up_from":54,"up_thru":79,"down_at":51,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":2026381241},{"type":"v1","addr":"172.21.15.37:6803","nonce":2026381241}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":2026381241},{"type":"v1","addr":"172.21.15.37:6805","nonce":2026381241}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":2026381241},{"type":"v1","addr":"172.21.15.37:6809","nonce":2026381241}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":2026381241},{"type":"v1","addr":"172.21.15.37:6807","nonce":2026381241}]},"public_addr":"172.21.15.37:6803/2026381241","cluster_addr":"172.21.15.37:6805/2026381241","heartbeat_back_addr":"172.21.15.37:6809/2026381241","heartbeat_front_addr":"172.21.15.37:6807/2026381241","state":["exists","up"]},{"osd":1,"uuid":"2876dd3f-aefb-4298-8fe4-562245e1103e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":55,"up_from":59,"up_thru":79,"down_at":56,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":2942536992},{"type":"v1","addr":"172.21.15.37:6811","nonce":2942536992}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":2942536992},{"type":"v1","addr":"172.21.15.37:6813","nonce":2942536992}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2942536992},{"type":"v1","addr":"172.21.15.37:6817","nonce":2942536992}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":2942536992},{"type":"v1","addr":"172.21.15.37:6815","nonce":2942536992}]},"public_addr":"172.21.15.37:6811/2942536992","cluster_addr":"172.21.15.37:6813/2942536992","heartbeat_back_addr":"172.21.15.37:6817/2942536992","heartbeat_front_addr":"172.21.15.37:6815/2942536992","state":["exists","up"]},{"osd":2,"uuid":"bba0f95c-f4e3-404e-adf7-b90353a96af4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":19,"last_clean_end":60,"up_from":64,"up_thru":79,"down_at":61,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":4140655393},{"type":"v1","addr":"172.21.15.37:6819","nonce":4140655393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":4140655393},{"type":"v1","addr":"172.21.15.37:6821","nonce":4140655393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":4140655393},{"type":"v1","addr":"172.21.15.37:6825","nonce":4140655393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":4140655393},{"type":"v1","addr":"172.21.15.37:6823","nonce":4140655393}]},"public_addr":"172.21.15.37:6819/4140655393","cluster_addr":"172.21.15.37:6821/4140655393","heartbeat_back_addr":"172.21.15.37:6825/4140655393","heartbeat_front_addr":"172.21.15.37:6823/4140655393","state":["exists","up"]},{"osd":3,"uuid":"8da1c140-ada5-4879-b5d9-1ed81b0c9aef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":27,"last_clean_end":65,"up_from":69,"up_thru":79,"down_at":66,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":227480876},{"type":"v1","addr":"172.21.15.148:6801","nonce":227480876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":227480876},{"type":"v1","addr":"172.21.15.148:6803","nonce":227480876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":227480876},{"type":"v1","addr":"172.21.15.148:6807","nonce":227480876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":227480876},{"type":"v1","addr":"172.21.15.148:6805","nonce":227480876}]},"public_addr":"172.21.15.148:6801/227480876","cluster_addr":"172.21.15.148:6803/227480876","heartbeat_back_addr":"172.21.15.148:6807/227480876","heartbeat_front_addr":"172.21.15.148:6805/227480876","state":["exists","up"]},{"osd":4,"uuid":"9af70e66-bbca-48c7-bad1-7ac78270cf16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":35,"last_clean_end":70,"up_from":74,"up_thru":79,"down_at":71,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":572347061},{"type":"v1","addr":"172.21.15.148:6809","nonce":572347061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":572347061},{"type":"v1","addr":"172.21.15.148:6811","nonce":572347061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":572347061},{"type":"v1","addr":"172.21.15.148:6815","nonce":572347061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":572347061},{"type":"v1","addr":"172.21.15.148:6813","nonce":572347061}]},"public_addr":"172.21.15.148:6809/572347061","cluster_addr":"172.21.15.148:6811/572347061","heartbeat_back_addr":"172.21.15.148:6815/572347061","heartbeat_front_addr":"172.21.15.148:6813/572347061","state":["exists","up"]},{"osd":5,"uuid":"a66944e4-38bd-4403-b45f-7de01ddfb7a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":40,"last_clean_end":75,"up_from":79,"up_thru":79,"down_at":76,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":2577126633},{"type":"v1","addr":"172.21.15.148:6817","nonce":2577126633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":2577126633},{"type":"v1","addr":"172.21.15.148:6819","nonce":2577126633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":2577126633},{"type":"v1","addr":"172.21.15.148:6823","nonce":2577126633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":2577126633},{"type":"v1","addr":"172.21.15.148:6821","nonce":2577126633}]},"public_addr":"172.21.15.148:6817/2577126633","cluster_addr":"172.21.15.148:6819/2577126633","heartbeat_back_addr":"172.21.15.148:6823/2577126633","heartbeat_front_addr":"172.21.15.148:6821/2577126633","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T07:59:23.110900+0000","dead_epoch":50},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T07:59:55.540549+0000","dead_epoch":55},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:00:28.082290+0000","dead_epoch":60},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:00:59.496893+0000","dead_epoch":65},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:01:31.307445+0000","dead_epoch":70},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:02:03.191177+0000","dead_epoch":75}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:0/3846762095":"2024-03-30T07:57:55.402909+0000","172.21.15.37:6800/2746946658":"2024-03-30T07:56:36.633439+0000","172.21.15.37:0/1909774742":"2024-03-30T07:55:59.989528+0000","172.21.15.37:0/1421768676":"2024-03-30T07:57:55.402909+0000","172.21.15.37:0/1395183736":"2024-03-30T08:06:01.871937+0000","172.21.15.37:6801/1802836043":"2024-03-30T08:06:01.871937+0000","172.21.15.37:0/368538117":"2024-03-30T07:57:55.402909+0000","172.21.15.148:0/855865016":"2024-03-30T08:08:06.449195+0000","172.21.15.37:0/2242434702":"2024-03-30T08:06:01.871937+0000","172.21.15.148:0/1612512235":"2024-03-30T08:08:06.449195+0000","172.21.15.148:6828/1134897184":"2024-03-30T08:08:06.449195+0000","172.21.15.37:0/1131673968":"2024-03-30T07:56:36.633439+0000","172.21.15.37:0/959705723":"2024-03-30T07:56:36.633439+0000","172.21.15.37:0/3667241947":"2024-03-30T08:12:09.488085+0000","172.21.15.37:6827/1559618909":"2024-03-30T08:17:16.236377+0000","172.21.15.37:0/3644857958":"2024-03-30T07:55:59.989528+0000","172.21.15.37:6828/2341530495":"2024-03-30T08:17:16.236404+0000","172.21.15.37:6800/1802836043":"2024-03-30T08:06:01.871937+0000","172.21.15.37:6801/2762339582":"2024-03-30T07:55:59.989528+0000","172.21.15.148:6829/1134897184":"2024-03-30T08:08:06.449195+0000","172.21.15.148:0/1484235408":"2024-03-30T08:08:06.449195+0000","172.21.15.37:6801/2746946658":"2024-03-30T07:56:36.633439+0000","172.21.15.148:0/4266419604":"2024-03-30T08:08:06.449195+0000","172.21.15.37:6800/4245458578":"2024-03-30T07:57:55.402909+0000","172.21.15.37:6829/2341530495":"2024-03-30T08:17:16.236404+0000","172.21.15.148:6824/2050993679":"2024-03-30T08:03:46.637260+0000","172.21.15.37:6801/4245458578":"2024-03-30T07:57:55.402909+0000","172.21.15.37:0/3116346321":"2024-03-30T07:55:59.989528+0000","172.21.15.37:0/2483635161":"2024-03-30T07:56:36.633439+0000","172.21.15.37:6800/4188329516":"2024-03-30T08:12:09.488085+0000","172.21.15.37:6800/2762339582":"2024-03-30T07:55:59.989528+0000","172.21.15.37:0/2522921436":"2024-03-30T08:06:01.871937+0000","172.21.15.37:0/968152496":"2024-03-30T08:12:09.488085+0000","172.21.15.37:0/68050196":"2024-03-30T08:12:09.488085+0000","172.21.15.37:0/2701457924":"2024-03-30T08:06:01.871937+0000","172.21.15.37:6801/4188329516":"2024-03-30T08:12:09.488085+0000","172.21.15.37:6826/1559618909":"2024-03-30T08:17:16.236377+0000","172.21.15.148:6825/2050993679":"2024-03-30T08:03:46.637260+0000","172.21.15.37:0/331291813":"2024-03-30T08:12:09.488085+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T08:50:28.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:28.141+0000 7f6f29000700 1 -- 172.21.15.37:0/210993223 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f6f0806cc20 msgr2=0x7f6f0806f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:50:28.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:28.141+0000 7f6f29000700 1 --2- 172.21.15.37:0/210993223 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f6f0806cc20 0x7f6f0806f0d0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f6f24110dc0 tx=0x7f6f10009e30 comp rx=0 tx=0).stop 2024-03-29T08:50:28.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:28.141+0000 7f6f29000700 1 -- 172.21.15.37:0/210993223 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6f24070360 msgr2=0x7f6f24115950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:50:28.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:28.141+0000 7f6f29000700 1 --2- 172.21.15.37:0/210993223 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6f24070360 0x7f6f24115950 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f6f14002a50 tx=0x7f6f140046b0 comp rx=0 tx=0).stop 2024-03-29T08:50:28.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:28.142+0000 7f6f29000700 1 -- 172.21.15.37:0/210993223 shutdown_connections 2024-03-29T08:50:28.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:28.142+0000 7f6f29000700 1 --2- 172.21.15.37:0/210993223 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f6f0806cc20 0x7f6f0806f0d0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:28.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:28.142+0000 7f6f29000700 1 --2- 172.21.15.37:0/210993223 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6f24070360 0x7f6f24115950 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:28.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:28.142+0000 7f6f29000700 1 --2- 172.21.15.37:0/210993223 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6f24070c70 0x7f6f24115e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:28.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:28.142+0000 7f6f29000700 1 -- 172.21.15.37:0/210993223 >> 172.21.15.37:0/210993223 conn(0x7f6f240f8840 msgr2=0x7f6f240f9ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:50:28.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:28.142+0000 7f6f29000700 1 -- 172.21.15.37:0/210993223 shutdown_connections 2024-03-29T08:50:28.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:28.142+0000 7f6f29000700 1 -- 172.21.15.37:0/210993223 wait complete. 2024-03-29T08:50:28.157 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:50:27 smithi037 ceph-mon[142325]: pgmap v1189: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:50:28.157 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:50:27 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/325759127' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 34, "format": "json"}]: dispatch 2024-03-29T08:50:28.157 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:50:27 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/2690767894' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 35, "format": "json"}]: dispatch 2024-03-29T08:50:28.159 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-29T08:00:32.022416+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 6, 'score_stable': 6, 'optimal_score': 0.5, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs.cephfs.meta', 'create_time': '2024-03-29T08:03:28.176923+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.690000057220459, 'score_stable': 1.690000057220459, 'optimal_score': 1, 'raw_score_acting': 1.690000057220459, 'raw_score_stable': 1.690000057220459, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs.cephfs.data', 'create_time': '2024-03-29T08:03:28.754148+0000', 'flags': 131073, 'flags_names': 'hashpspool,bulk', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '46', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-29T08:50:28.160 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-29T08:50:28.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:50:27 smithi148 ceph-mon[106692]: pgmap v1189: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:50:28.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:50:27 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/325759127' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 34, "format": "json"}]: dispatch 2024-03-29T08:50:28.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:50:27 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/2690767894' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 35, "format": "json"}]: dispatch 2024-03-29T08:50:28.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:28.356+0000 7f72ad400700 1 -- 172.21.15.37:0/2672869466 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f72a8103680 msgr2=0x7f72a80fccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:50:28.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:28.356+0000 7f72ad400700 1 --2- 172.21.15.37:0/2672869466 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f72a8103680 0x7f72a80fccf0 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f7290004940 tx=0x7f7290009e30 comp rx=0 tx=0).stop 2024-03-29T08:50:28.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:28.356+0000 7f72ad400700 1 -- 172.21.15.37:0/2672869466 shutdown_connections 2024-03-29T08:50:28.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:28.356+0000 7f72ad400700 1 --2- 172.21.15.37:0/2672869466 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f72a8103680 0x7f72a80fccf0 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:28.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:28.356+0000 7f72ad400700 1 --2- 172.21.15.37:0/2672869466 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f72a8102ce0 0x7f72a81030b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:28.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:28.356+0000 7f72ad400700 1 -- 172.21.15.37:0/2672869466 >> 172.21.15.37:0/2672869466 conn(0x7f72a80f8830 msgr2=0x7f72a80fac40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:50:28.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:28.356+0000 7f72ad400700 1 -- 172.21.15.37:0/2672869466 shutdown_connections 2024-03-29T08:50:28.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:28.356+0000 7f72ad400700 1 -- 172.21.15.37:0/2672869466 wait complete. 2024-03-29T08:50:28.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:28.357+0000 7f72ad400700 1 Processor -- start 2024-03-29T08:50:28.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:28.358+0000 7f72ad400700 1 -- start start 2024-03-29T08:50:28.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:28.358+0000 7f72ad400700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f72a8102ce0 0x7f72a81137b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:50:28.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:28.358+0000 7f72ad400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f72a8103680 0x7f72a8113cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:50:28.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:28.358+0000 7f72ad400700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72a8114340 con 0x7f72a8103680 2024-03-29T08:50:28.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:28.358+0000 7f72ad400700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72a810d940 con 0x7f72a8102ce0 2024-03-29T08:50:28.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:28.358+0000 7f729fe00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f72a8103680 0x7f72a8113cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:50:28.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:28.359+0000 7f729fe00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f72a8103680 0x7f72a8113cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52962/0 (socket says 172.21.15.37:52962) 2024-03-29T08:50:28.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:28.359+0000 7f729fe00700 1 -- 172.21.15.37:0/3191468208 learned_addr learned my addr 172.21.15.37:0/3191468208 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:50:28.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:28.359+0000 7f729fe00700 1 -- 172.21.15.37:0/3191468208 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f72a8102ce0 msgr2=0x7f72a81137b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:50:28.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:28.359+0000 7f729fe00700 1 --2- 172.21.15.37:0/3191468208 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f72a8102ce0 0x7f72a81137b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:28.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:28.359+0000 7f729fe00700 1 -- 172.21.15.37:0/3191468208 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7290005040 con 0x7f72a8103680 2024-03-29T08:50:28.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:28.359+0000 7f729fe00700 1 --2- 172.21.15.37:0/3191468208 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f72a8103680 0x7f72a8113cf0 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f7290009390 tx=0x7f7290009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:50:28.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:28.359+0000 7f729ea00700 1 -- 172.21.15.37:0/3191468208 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f729000e040 con 0x7f72a8103680 2024-03-29T08:50:28.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:28.359+0000 7f729ea00700 1 -- 172.21.15.37:0/3191468208 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f7290013420 con 0x7f72a8103680 2024-03-29T08:50:28.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:28.359+0000 7f72ad400700 1 -- 172.21.15.37:0/3191468208 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72a810dbc0 con 0x7f72a8103680 2024-03-29T08:50:28.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:28.359+0000 7f729ea00700 1 -- 172.21.15.37:0/3191468208 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f729001c630 con 0x7f72a8103680 2024-03-29T08:50:28.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:28.359+0000 7f72ad400700 1 -- 172.21.15.37:0/3191468208 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72a810e180 con 0x7f72a8103680 2024-03-29T08:50:28.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:28.361+0000 7f72ad400700 1 -- 172.21.15.37:0/3191468208 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72a80fe430 con 0x7f72a8103680 2024-03-29T08:50:28.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:28.362+0000 7f729ea00700 1 -- 172.21.15.37:0/3191468208 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f72900139e0 con 0x7f72a8103680 2024-03-29T08:50:28.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:28.362+0000 7f729ea00700 1 --2- 172.21.15.37:0/3191468208 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f728c06cc20 0x7f728c06f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:50:28.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:28.363+0000 7f729ea00700 1 -- 172.21.15.37:0/3191468208 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f729008d1e0 con 0x7f72a8103680 2024-03-29T08:50:28.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:28.363+0000 7f72a6800700 1 --2- 172.21.15.37:0/3191468208 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f728c06cc20 0x7f728c06f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:50:28.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:28.363+0000 7f72a6800700 1 --2- 172.21.15.37:0/3191468208 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f728c06cc20 0x7f728c06f0d0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f7298004b70 tx=0x7f7298007cc0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:50:28.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:28.367+0000 7f729ea00700 1 -- 172.21.15.37:0/3191468208 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f729005ad70 con 0x7f72a8103680 2024-03-29T08:50:28.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:28.584+0000 7f72ad400700 1 -- 172.21.15.37:0/3191468208 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f72a8109c80 con 0x7f72a8103680 2024-03-29T08:50:28.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:28.584+0000 7f729ea00700 1 -- 172.21.15.37:0/3191468208 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v86) v1 ==== 93+0+10 (secure 0 0 0) 0x7f729005a900 con 0x7f72a8103680 2024-03-29T08:50:28.584 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 1 2024-03-29T08:50:28.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:28.587+0000 7f72ad400700 1 -- 172.21.15.37:0/3191468208 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f728c06cc20 msgr2=0x7f728c06f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:50:28.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:28.587+0000 7f72ad400700 1 --2- 172.21.15.37:0/3191468208 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f728c06cc20 0x7f728c06f0d0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f7298004b70 tx=0x7f7298007cc0 comp rx=0 tx=0).stop 2024-03-29T08:50:28.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:28.587+0000 7f72ad400700 1 -- 172.21.15.37:0/3191468208 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f72a8103680 msgr2=0x7f72a8113cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:50:28.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:28.587+0000 7f72ad400700 1 --2- 172.21.15.37:0/3191468208 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f72a8103680 0x7f72a8113cf0 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f7290009390 tx=0x7f7290009e30 comp rx=0 tx=0).stop 2024-03-29T08:50:28.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:28.587+0000 7f72ad400700 1 -- 172.21.15.37:0/3191468208 shutdown_connections 2024-03-29T08:50:28.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:28.587+0000 7f72ad400700 1 --2- 172.21.15.37:0/3191468208 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f728c06cc20 0x7f728c06f0d0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:28.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:28.587+0000 7f72ad400700 1 --2- 172.21.15.37:0/3191468208 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f72a8102ce0 0x7f72a81137b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:28.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:28.587+0000 7f72ad400700 1 --2- 172.21.15.37:0/3191468208 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f72a8103680 0x7f72a8113cf0 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:28.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:28.587+0000 7f72ad400700 1 -- 172.21.15.37:0/3191468208 >> 172.21.15.37:0/3191468208 conn(0x7f72a80f8830 msgr2=0x7f72a80f99b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:50:28.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:28.587+0000 7f72ad400700 1 -- 172.21.15.37:0/3191468208 shutdown_connections 2024-03-29T08:50:28.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:28.587+0000 7f72ad400700 1 -- 172.21.15.37:0/3191468208 wait complete. 2024-03-29T08:50:28.604 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.meta pg_num 2024-03-29T08:50:28.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:28.802+0000 7feeb8800700 1 -- 172.21.15.37:0/1507670924 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feeb0103670 msgr2=0x7feeb00fcce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:50:28.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:28.802+0000 7feeb8800700 1 --2- 172.21.15.37:0/1507670924 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feeb0103670 0x7feeb00fcce0 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7feea0004940 tx=0x7feea0009e30 comp rx=0 tx=0).stop 2024-03-29T08:50:28.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:28.802+0000 7feeb8800700 1 -- 172.21.15.37:0/1507670924 shutdown_connections 2024-03-29T08:50:28.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:28.802+0000 7feeb8800700 1 --2- 172.21.15.37:0/1507670924 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feeb0103670 0x7feeb00fcce0 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:28.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:28.802+0000 7feeb8800700 1 --2- 172.21.15.37:0/1507670924 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feeb0102cd0 0x7feeb01030a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:28.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:28.802+0000 7feeb8800700 1 -- 172.21.15.37:0/1507670924 >> 172.21.15.37:0/1507670924 conn(0x7feeb00f8820 msgr2=0x7feeb00fac30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:50:28.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:28.802+0000 7feeb8800700 1 -- 172.21.15.37:0/1507670924 shutdown_connections 2024-03-29T08:50:28.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:28.802+0000 7feeb8800700 1 -- 172.21.15.37:0/1507670924 wait complete. 2024-03-29T08:50:28.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:28.803+0000 7feeb8800700 1 Processor -- start 2024-03-29T08:50:28.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:28.803+0000 7feeb8800700 1 -- start start 2024-03-29T08:50:28.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:28.804+0000 7feeb8800700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feeb0102cd0 0x7feeb0113780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:50:28.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:28.804+0000 7feeb8800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feeb0103670 0x7feeb0113cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:50:28.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:28.804+0000 7feeb8800700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feeb0114310 con 0x7feeb0103670 2024-03-29T08:50:28.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:28.804+0000 7feeb8800700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feeb010d910 con 0x7feeb0102cd0 2024-03-29T08:50:28.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:28.804+0000 7feeb5400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feeb0103670 0x7feeb0113cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:50:28.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:28.804+0000 7feeb5400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feeb0103670 0x7feeb0113cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52986/0 (socket says 172.21.15.37:52986) 2024-03-29T08:50:28.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:28.804+0000 7feeb5400700 1 -- 172.21.15.37:0/3682137623 learned_addr learned my addr 172.21.15.37:0/3682137623 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:50:28.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:28.804+0000 7feeb5e00700 1 --2- 172.21.15.37:0/3682137623 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feeb0102cd0 0x7feeb0113780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:50:28.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:28.805+0000 7feeb5400700 1 -- 172.21.15.37:0/3682137623 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feeb0102cd0 msgr2=0x7feeb0113780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:50:28.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:28.805+0000 7feeb5400700 1 --2- 172.21.15.37:0/3682137623 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feeb0102cd0 0x7feeb0113780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:28.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:28.805+0000 7feeb5400700 1 -- 172.21.15.37:0/3682137623 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feea0005040 con 0x7feeb0103670 2024-03-29T08:50:28.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:28.805+0000 7feeb5e00700 1 --2- 172.21.15.37:0/3682137623 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feeb0102cd0 0x7feeb0113780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:50:28.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:28.805+0000 7feeb5400700 1 --2- 172.21.15.37:0/3682137623 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feeb0103670 0x7feeb0113cc0 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7feea0009390 tx=0x7feea0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:50:28.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:28.805+0000 7feeaa000700 1 -- 172.21.15.37:0/3682137623 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feea000e040 con 0x7feeb0103670 2024-03-29T08:50:28.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:28.805+0000 7feeaa000700 1 -- 172.21.15.37:0/3682137623 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7feea0013420 con 0x7feeb0103670 2024-03-29T08:50:28.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:28.805+0000 7feeb8800700 1 -- 172.21.15.37:0/3682137623 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feeb010db90 con 0x7feeb0103670 2024-03-29T08:50:28.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:28.805+0000 7feeaa000700 1 -- 172.21.15.37:0/3682137623 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feea001c630 con 0x7feeb0103670 2024-03-29T08:50:28.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:28.805+0000 7feeb8800700 1 -- 172.21.15.37:0/3682137623 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feeb010e150 con 0x7feeb0103670 2024-03-29T08:50:28.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:28.807+0000 7feeb8800700 1 -- 172.21.15.37:0/3682137623 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feeb00fe420 con 0x7feeb0103670 2024-03-29T08:50:28.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:28.807+0000 7feeaa000700 1 -- 172.21.15.37:0/3682137623 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7feea0013a00 con 0x7feeb0103670 2024-03-29T08:50:28.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:28.808+0000 7feeaa000700 1 --2- 172.21.15.37:0/3682137623 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fee9806cc20 0x7fee9806f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:50:28.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:28.808+0000 7feeaa000700 1 -- 172.21.15.37:0/3682137623 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7feea008d310 con 0x7feeb0103670 2024-03-29T08:50:28.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:28.809+0000 7feeb5e00700 1 --2- 172.21.15.37:0/3682137623 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fee9806cc20 0x7fee9806f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:50:28.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:28.809+0000 7feeb5e00700 1 --2- 172.21.15.37:0/3682137623 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fee9806cc20 0x7fee9806f0d0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7feea4009fd0 tx=0x7feea4009ee0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:50:28.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:28.813+0000 7feeaa000700 1 -- 172.21.15.37:0/3682137623 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7feea005ade0 con 0x7feeb0103670 2024-03-29T08:50:29.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:29.031+0000 7feeb8800700 1 -- 172.21.15.37:0/3682137623 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"} v 0) v1 -- 0x7feeb010edd0 con 0x7feeb0103670 2024-03-29T08:50:29.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:29.032+0000 7feeaa000700 1 -- 172.21.15.37:0/3682137623 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]=0 v86) v1 ==== 107+0+11 (secure 0 0 0) 0x7feea005a970 con 0x7feeb0103670 2024-03-29T08:50:29.032 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 32 2024-03-29T08:50:29.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:29.034+0000 7feeb8800700 1 -- 172.21.15.37:0/3682137623 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fee9806cc20 msgr2=0x7fee9806f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:50:29.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:29.034+0000 7feeb8800700 1 --2- 172.21.15.37:0/3682137623 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fee9806cc20 0x7fee9806f0d0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7feea4009fd0 tx=0x7feea4009ee0 comp rx=0 tx=0).stop 2024-03-29T08:50:29.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:29.034+0000 7feeb8800700 1 -- 172.21.15.37:0/3682137623 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feeb0103670 msgr2=0x7feeb0113cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:50:29.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:29.034+0000 7feeb8800700 1 --2- 172.21.15.37:0/3682137623 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feeb0103670 0x7feeb0113cc0 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7feea0009390 tx=0x7feea0009e30 comp rx=0 tx=0).stop 2024-03-29T08:50:29.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:29.034+0000 7feeb8800700 1 -- 172.21.15.37:0/3682137623 shutdown_connections 2024-03-29T08:50:29.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:29.034+0000 7feeb8800700 1 --2- 172.21.15.37:0/3682137623 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fee9806cc20 0x7fee9806f0d0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:29.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:29.034+0000 7feeb8800700 1 --2- 172.21.15.37:0/3682137623 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feeb0102cd0 0x7feeb0113780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:29.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:29.034+0000 7feeb8800700 1 --2- 172.21.15.37:0/3682137623 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feeb0103670 0x7feeb0113cc0 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:29.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:29.034+0000 7feeb8800700 1 -- 172.21.15.37:0/3682137623 >> 172.21.15.37:0/3682137623 conn(0x7feeb00f8820 msgr2=0x7feeb00f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:50:29.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:29.034+0000 7feeb8800700 1 -- 172.21.15.37:0/3682137623 shutdown_connections 2024-03-29T08:50:29.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:29.034+0000 7feeb8800700 1 -- 172.21.15.37:0/3682137623 wait complete. 2024-03-29T08:50:29.052 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.data pg_num 2024-03-29T08:50:29.176 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:50:28 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/210993223' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-29T08:50:29.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:50:28 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/3191468208' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-29T08:50:29.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:29.249+0000 7f1cc8c00700 1 -- 172.21.15.37:0/1220237451 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1cc4070360 msgr2=0x7f1cc4070730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:50:29.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:29.249+0000 7f1cc8c00700 1 --2- 172.21.15.37:0/1220237451 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1cc4070360 0x7f1cc4070730 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f1cb4004970 tx=0x7f1cb4009e30 comp rx=0 tx=0).stop 2024-03-29T08:50:29.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:29.249+0000 7f1cc8c00700 1 -- 172.21.15.37:0/1220237451 shutdown_connections 2024-03-29T08:50:29.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:29.249+0000 7f1cc8c00700 1 --2- 172.21.15.37:0/1220237451 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1cc4070c70 0x7f1cc4109010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:29.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:29.249+0000 7f1cc8c00700 1 --2- 172.21.15.37:0/1220237451 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1cc4070360 0x7f1cc4070730 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:29.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:29.249+0000 7f1cc8c00700 1 -- 172.21.15.37:0/1220237451 >> 172.21.15.37:0/1220237451 conn(0x7f1cc40f8810 msgr2=0x7f1cc40fac20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:50:29.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:29.249+0000 7f1cc8c00700 1 -- 172.21.15.37:0/1220237451 shutdown_connections 2024-03-29T08:50:29.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:29.249+0000 7f1cc8c00700 1 -- 172.21.15.37:0/1220237451 wait complete. 2024-03-29T08:50:29.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:29.250+0000 7f1cc8c00700 1 Processor -- start 2024-03-29T08:50:29.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:29.251+0000 7f1cc8c00700 1 -- start start 2024-03-29T08:50:29.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:29.251+0000 7f1cc8c00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1cc4070360 0x7f1cc4115950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:50:29.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:29.251+0000 7f1cc8c00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1cc4070c70 0x7f1cc4115e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:50:29.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:29.251+0000 7f1cc8c00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cc41164e0 con 0x7f1cc4070c70 2024-03-29T08:50:29.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:29.251+0000 7f1cc8c00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cc410fae0 con 0x7f1cc4070360 2024-03-29T08:50:29.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:29.252+0000 7f1cc1400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1cc4070c70 0x7f1cc4115e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:50:29.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:29.252+0000 7f1cc1400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1cc4070c70 0x7f1cc4115e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53010/0 (socket says 172.21.15.37:53010) 2024-03-29T08:50:29.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:29.252+0000 7f1cc1400700 1 -- 172.21.15.37:0/12117242 learned_addr learned my addr 172.21.15.37:0/12117242 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:50:29.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:29.252+0000 7f1cc1e00700 1 --2- 172.21.15.37:0/12117242 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1cc4070360 0x7f1cc4115950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:50:29.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:29.252+0000 7f1cc1400700 1 -- 172.21.15.37:0/12117242 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1cc4070360 msgr2=0x7f1cc4115950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:50:29.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:29.252+0000 7f1cc1400700 1 --2- 172.21.15.37:0/12117242 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1cc4070360 0x7f1cc4115950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:29.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:29.252+0000 7f1cc1400700 1 -- 172.21.15.37:0/12117242 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cb4005040 con 0x7f1cc4070c70 2024-03-29T08:50:29.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:29.252+0000 7f1cc1400700 1 --2- 172.21.15.37:0/12117242 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1cc4070c70 0x7f1cc4115e90 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f1cb000eb40 tx=0x7f1cb000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:50:29.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:29.252+0000 7f1cba000700 1 -- 172.21.15.37:0/12117242 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1cb0007cc0 con 0x7f1cc4070c70 2024-03-29T08:50:29.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:29.253+0000 7f1cba000700 1 -- 172.21.15.37:0/12117242 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f1cb000f040 con 0x7f1cc4070c70 2024-03-29T08:50:29.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:29.253+0000 7f1cc8c00700 1 -- 172.21.15.37:0/12117242 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cc410fd60 con 0x7f1cc4070c70 2024-03-29T08:50:29.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:29.253+0000 7f1cba000700 1 -- 172.21.15.37:0/12117242 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1cb000c990 con 0x7f1cc4070c70 2024-03-29T08:50:29.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:29.253+0000 7f1cc8c00700 1 -- 172.21.15.37:0/12117242 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cc41102a0 con 0x7f1cc4070c70 2024-03-29T08:50:29.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:29.254+0000 7f1cc8c00700 1 -- 172.21.15.37:0/12117242 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1cc4071aa0 con 0x7f1cc4070c70 2024-03-29T08:50:29.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:29.255+0000 7f1cba000700 1 -- 172.21.15.37:0/12117242 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f1cb000cc00 con 0x7f1cc4070c70 2024-03-29T08:50:29.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:29.256+0000 7f1cba000700 1 --2- 172.21.15.37:0/12117242 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f1ca806cc20 0x7f1ca806f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:50:29.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:29.256+0000 7f1cba000700 1 -- 172.21.15.37:0/12117242 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f1cb008bcd0 con 0x7f1cc4070c70 2024-03-29T08:50:29.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:29.256+0000 7f1cc1e00700 1 --2- 172.21.15.37:0/12117242 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f1ca806cc20 0x7f1ca806f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:50:29.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:29.256+0000 7f1cc1e00700 1 --2- 172.21.15.37:0/12117242 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f1ca806cc20 0x7f1ca806f0d0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f1cb4009fd0 tx=0x7f1cb4009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:50:29.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:29.260+0000 7f1cba000700 1 -- 172.21.15.37:0/12117242 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f1cb0059750 con 0x7f1cc4070c70 2024-03-29T08:50:29.305 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:50:28 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/210993223' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-29T08:50:29.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:50:28 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/3191468208' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-29T08:50:29.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:29.477+0000 7f1cc8c00700 1 -- 172.21.15.37:0/12117242 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"} v 0) v1 -- 0x7f1cc410beb0 con 0x7f1cc4070c70 2024-03-29T08:50:29.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:29.478+0000 7f1cba000700 1 -- 172.21.15.37:0/12117242 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]=0 v86) v1 ==== 107+0+11 (secure 0 0 0) 0x7f1cb00592e0 con 0x7f1cc4070c70 2024-03-29T08:50:29.478 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 32 2024-03-29T08:50:29.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:29.480+0000 7f1cc8c00700 1 -- 172.21.15.37:0/12117242 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f1ca806cc20 msgr2=0x7f1ca806f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:50:29.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:29.480+0000 7f1cc8c00700 1 --2- 172.21.15.37:0/12117242 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f1ca806cc20 0x7f1ca806f0d0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f1cb4009fd0 tx=0x7f1cb4009e30 comp rx=0 tx=0).stop 2024-03-29T08:50:29.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:29.480+0000 7f1cc8c00700 1 -- 172.21.15.37:0/12117242 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1cc4070c70 msgr2=0x7f1cc4115e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:50:29.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:29.480+0000 7f1cc8c00700 1 --2- 172.21.15.37:0/12117242 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1cc4070c70 0x7f1cc4115e90 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f1cb000eb40 tx=0x7f1cb000b040 comp rx=0 tx=0).stop 2024-03-29T08:50:29.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:29.480+0000 7f1cc8c00700 1 -- 172.21.15.37:0/12117242 shutdown_connections 2024-03-29T08:50:29.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:29.480+0000 7f1cc8c00700 1 --2- 172.21.15.37:0/12117242 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f1ca806cc20 0x7f1ca806f0d0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:29.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:29.480+0000 7f1cc8c00700 1 --2- 172.21.15.37:0/12117242 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1cc4070360 0x7f1cc4115950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:29.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:29.480+0000 7f1cc8c00700 1 --2- 172.21.15.37:0/12117242 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1cc4070c70 0x7f1cc4115e90 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:29.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:29.480+0000 7f1cc8c00700 1 -- 172.21.15.37:0/12117242 >> 172.21.15.37:0/12117242 conn(0x7f1cc40f8810 msgr2=0x7f1cc40f9d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:50:29.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:29.481+0000 7f1cc8c00700 1 -- 172.21.15.37:0/12117242 shutdown_connections 2024-03-29T08:50:29.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:29.481+0000 7f1cc8c00700 1 -- 172.21.15.37:0/12117242 wait complete. 2024-03-29T08:50:29.498 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-29T08:50:29.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:29.693+0000 7f15df600700 1 -- 172.21.15.37:0/1039056779 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15d8103670 msgr2=0x7f15d80fcce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:50:29.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:29.693+0000 7f15df600700 1 --2- 172.21.15.37:0/1039056779 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15d8103670 0x7f15d80fcce0 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f15c4004940 tx=0x7f15c4009e30 comp rx=0 tx=0).stop 2024-03-29T08:50:29.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:29.693+0000 7f15df600700 1 -- 172.21.15.37:0/1039056779 shutdown_connections 2024-03-29T08:50:29.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:29.693+0000 7f15df600700 1 --2- 172.21.15.37:0/1039056779 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15d8103670 0x7f15d80fcce0 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:29.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:29.693+0000 7f15df600700 1 --2- 172.21.15.37:0/1039056779 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f15d8102cd0 0x7f15d81030a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:29.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:29.693+0000 7f15df600700 1 -- 172.21.15.37:0/1039056779 >> 172.21.15.37:0/1039056779 conn(0x7f15d80f8820 msgr2=0x7f15d80fac30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:50:29.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:29.693+0000 7f15df600700 1 -- 172.21.15.37:0/1039056779 shutdown_connections 2024-03-29T08:50:29.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:29.693+0000 7f15df600700 1 -- 172.21.15.37:0/1039056779 wait complete. 2024-03-29T08:50:29.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:29.694+0000 7f15df600700 1 Processor -- start 2024-03-29T08:50:29.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:29.694+0000 7f15df600700 1 -- start start 2024-03-29T08:50:29.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:29.695+0000 7f15df600700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f15d8102cd0 0x7f15d8115970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:50:29.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:29.695+0000 7f15df600700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15d8103670 0x7f15d8115eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:50:29.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:29.695+0000 7f15df600700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15d8116500 con 0x7f15d8103670 2024-03-29T08:50:29.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:29.695+0000 7f15df600700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15d810fb00 con 0x7f15d8102cd0 2024-03-29T08:50:29.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:29.695+0000 7f15d7e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15d8103670 0x7f15d8115eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:50:29.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:29.695+0000 7f15d7e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15d8103670 0x7f15d8115eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53024/0 (socket says 172.21.15.37:53024) 2024-03-29T08:50:29.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:29.695+0000 7f15d7e00700 1 -- 172.21.15.37:0/1115882152 learned_addr learned my addr 172.21.15.37:0/1115882152 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:50:29.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:29.695+0000 7f15dcc00700 1 --2- 172.21.15.37:0/1115882152 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f15d8102cd0 0x7f15d8115970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:50:29.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:29.696+0000 7f15d7e00700 1 -- 172.21.15.37:0/1115882152 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f15d8102cd0 msgr2=0x7f15d8115970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:50:29.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:29.696+0000 7f15d7e00700 1 --2- 172.21.15.37:0/1115882152 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f15d8102cd0 0x7f15d8115970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:29.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:29.696+0000 7f15d7e00700 1 -- 172.21.15.37:0/1115882152 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15c4005040 con 0x7f15d8103670 2024-03-29T08:50:29.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:29.696+0000 7f15d7e00700 1 --2- 172.21.15.37:0/1115882152 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15d8103670 0x7f15d8115eb0 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f15c4009390 tx=0x7f15c4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:50:29.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:29.696+0000 7f15d4c00700 1 -- 172.21.15.37:0/1115882152 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f15c400e040 con 0x7f15d8103670 2024-03-29T08:50:29.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:29.696+0000 7f15df600700 1 -- 172.21.15.37:0/1115882152 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15d810fd80 con 0x7f15d8103670 2024-03-29T08:50:29.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:29.696+0000 7f15d4c00700 1 -- 172.21.15.37:0/1115882152 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f15c4013420 con 0x7f15d8103670 2024-03-29T08:50:29.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:29.696+0000 7f15d4c00700 1 -- 172.21.15.37:0/1115882152 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f15c401c630 con 0x7f15d8103670 2024-03-29T08:50:29.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:29.696+0000 7f15df600700 1 -- 172.21.15.37:0/1115882152 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15d8110340 con 0x7f15d8103670 2024-03-29T08:50:29.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:29.697+0000 7f15df600700 1 -- 172.21.15.37:0/1115882152 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15d80fe420 con 0x7f15d8103670 2024-03-29T08:50:29.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:29.698+0000 7f15d4c00700 1 -- 172.21.15.37:0/1115882152 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f15c4013a00 con 0x7f15d8103670 2024-03-29T08:50:29.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:29.699+0000 7f15d4c00700 1 --2- 172.21.15.37:0/1115882152 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f15c006cc20 0x7f15c006f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:50:29.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:29.699+0000 7f15d4c00700 1 -- 172.21.15.37:0/1115882152 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f15c408d120 con 0x7f15d8103670 2024-03-29T08:50:29.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:29.700+0000 7f15dcc00700 1 --2- 172.21.15.37:0/1115882152 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f15c006cc20 0x7f15c006f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:50:29.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:29.700+0000 7f15dcc00700 1 --2- 172.21.15.37:0/1115882152 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f15c006cc20 0x7f15c006f0d0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f15cc004f40 tx=0x7f15cc00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:50:29.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:29.704+0000 7f15d4c00700 1 -- 172.21.15.37:0/1115882152 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f15c405abf0 con 0x7f15d8103670 2024-03-29T08:50:29.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:29.939+0000 7f15df600700 1 -- 172.21.15.37:0/1115882152 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f15d8109c70 con 0x7f15d8103670 2024-03-29T08:50:29.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:29.939+0000 7f15d4c00700 1 -- 172.21.15.37:0/1115882152 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 39 entries v86) v1 ==== 81+0+2317 (secure 0 0 0) 0x7f15c405a780 con 0x7f15d8103670 2024-03-29T08:50:29.939 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:0/3846762095 2024-03-30T07:57:55.402909+0000 2024-03-29T08:50:29.939 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:6800/2746946658 2024-03-30T07:56:36.633439+0000 2024-03-29T08:50:29.939 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:0/1909774742 2024-03-30T07:55:59.989528+0000 2024-03-29T08:50:29.939 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:0/1421768676 2024-03-30T07:57:55.402909+0000 2024-03-29T08:50:29.940 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:0/1395183736 2024-03-30T08:06:01.871937+0000 2024-03-29T08:50:29.940 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:6801/1802836043 2024-03-30T08:06:01.871937+0000 2024-03-29T08:50:29.940 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:0/368538117 2024-03-30T07:57:55.402909+0000 2024-03-29T08:50:29.940 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.148:0/855865016 2024-03-30T08:08:06.449195+0000 2024-03-29T08:50:29.940 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:0/2242434702 2024-03-30T08:06:01.871937+0000 2024-03-29T08:50:29.940 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.148:0/1612512235 2024-03-30T08:08:06.449195+0000 2024-03-29T08:50:29.940 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.148:6828/1134897184 2024-03-30T08:08:06.449195+0000 2024-03-29T08:50:29.940 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:0/1131673968 2024-03-30T07:56:36.633439+0000 2024-03-29T08:50:29.940 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:0/959705723 2024-03-30T07:56:36.633439+0000 2024-03-29T08:50:29.940 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:0/3667241947 2024-03-30T08:12:09.488085+0000 2024-03-29T08:50:29.940 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:6827/1559618909 2024-03-30T08:17:16.236377+0000 2024-03-29T08:50:29.940 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:0/3644857958 2024-03-30T07:55:59.989528+0000 2024-03-29T08:50:29.940 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:6828/2341530495 2024-03-30T08:17:16.236404+0000 2024-03-29T08:50:29.940 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:6800/1802836043 2024-03-30T08:06:01.871937+0000 2024-03-29T08:50:29.941 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:6801/2762339582 2024-03-30T07:55:59.989528+0000 2024-03-29T08:50:29.941 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.148:6829/1134897184 2024-03-30T08:08:06.449195+0000 2024-03-29T08:50:29.941 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.148:0/1484235408 2024-03-30T08:08:06.449195+0000 2024-03-29T08:50:29.941 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:6801/2746946658 2024-03-30T07:56:36.633439+0000 2024-03-29T08:50:29.941 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.148:0/4266419604 2024-03-30T08:08:06.449195+0000 2024-03-29T08:50:29.941 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:6800/4245458578 2024-03-30T07:57:55.402909+0000 2024-03-29T08:50:29.941 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:6829/2341530495 2024-03-30T08:17:16.236404+0000 2024-03-29T08:50:29.941 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.148:6824/2050993679 2024-03-30T08:03:46.637260+0000 2024-03-29T08:50:29.941 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:6801/4245458578 2024-03-30T07:57:55.402909+0000 2024-03-29T08:50:29.941 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:0/3116346321 2024-03-30T07:55:59.989528+0000 2024-03-29T08:50:29.941 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:0/2483635161 2024-03-30T07:56:36.633439+0000 2024-03-29T08:50:29.941 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:6800/4188329516 2024-03-30T08:12:09.488085+0000 2024-03-29T08:50:29.941 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:6800/2762339582 2024-03-30T07:55:59.989528+0000 2024-03-29T08:50:29.941 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:0/2522921436 2024-03-30T08:06:01.871937+0000 2024-03-29T08:50:29.941 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:0/968152496 2024-03-30T08:12:09.488085+0000 2024-03-29T08:50:29.942 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:0/68050196 2024-03-30T08:12:09.488085+0000 2024-03-29T08:50:29.942 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:0/2701457924 2024-03-30T08:06:01.871937+0000 2024-03-29T08:50:29.942 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:6801/4188329516 2024-03-30T08:12:09.488085+0000 2024-03-29T08:50:29.942 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:6826/1559618909 2024-03-30T08:17:16.236377+0000 2024-03-29T08:50:29.942 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.148:6825/2050993679 2024-03-30T08:03:46.637260+0000 2024-03-29T08:50:29.942 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:0/331291813 2024-03-30T08:12:09.488085+0000 2024-03-29T08:50:29.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:29.942+0000 7f15df600700 1 -- 172.21.15.37:0/1115882152 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f15c006cc20 msgr2=0x7f15c006f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:50:29.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:29.942+0000 7f15df600700 1 --2- 172.21.15.37:0/1115882152 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f15c006cc20 0x7f15c006f0d0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f15cc004f40 tx=0x7f15cc00b040 comp rx=0 tx=0).stop 2024-03-29T08:50:29.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:29.942+0000 7f15df600700 1 -- 172.21.15.37:0/1115882152 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15d8103670 msgr2=0x7f15d8115eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:50:29.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:29.942+0000 7f15df600700 1 --2- 172.21.15.37:0/1115882152 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15d8103670 0x7f15d8115eb0 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f15c4009390 tx=0x7f15c4009e30 comp rx=0 tx=0).stop 2024-03-29T08:50:29.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:29.942+0000 7f15df600700 1 -- 172.21.15.37:0/1115882152 shutdown_connections 2024-03-29T08:50:29.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:29.942+0000 7f15df600700 1 --2- 172.21.15.37:0/1115882152 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f15c006cc20 0x7f15c006f0d0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:29.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:29.942+0000 7f15df600700 1 --2- 172.21.15.37:0/1115882152 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f15d8102cd0 0x7f15d8115970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:29.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:29.942+0000 7f15df600700 1 --2- 172.21.15.37:0/1115882152 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15d8103670 0x7f15d8115eb0 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:29.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:29.942+0000 7f15df600700 1 -- 172.21.15.37:0/1115882152 >> 172.21.15.37:0/1115882152 conn(0x7f15d80f8820 msgr2=0x7f15d80f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:50:29.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:29.942+0000 7f15df600700 1 -- 172.21.15.37:0/1115882152 shutdown_connections 2024-03-29T08:50:29.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:29.942+0000 7f15df600700 1 -- 172.21.15.37:0/1115882152 wait complete. 2024-03-29T08:50:29.944 INFO:teuthology.orchestra.run.smithi037.stderr:listed 39 entries 2024-03-29T08:50:29.960 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-03-29T08:50:29.960 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T08:50:30.006 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T08:50:30.176 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:50:29 smithi037 ceph-mon[142325]: pgmap v1190: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:50:30.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:50:29 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/3682137623' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-03-29T08:50:30.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:50:29 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/12117242' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-03-29T08:50:30.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:30.241+0000 7f475f400700 1 -- 172.21.15.37:0/3848016041 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4758102d80 msgr2=0x7f4758103150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:50:30.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:30.241+0000 7f475f400700 1 --2- 172.21.15.37:0/3848016041 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4758102d80 0x7f4758103150 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f474c004940 tx=0x7f474c009e30 comp rx=0 tx=0).stop 2024-03-29T08:50:30.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:30.241+0000 7f475f400700 1 -- 172.21.15.37:0/3848016041 shutdown_connections 2024-03-29T08:50:30.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:30.241+0000 7f475f400700 1 --2- 172.21.15.37:0/3848016041 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4758064e40 0x7f47580652b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:30.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:30.241+0000 7f475f400700 1 --2- 172.21.15.37:0/3848016041 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4758102d80 0x7f4758103150 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:30.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:30.241+0000 7f475f400700 1 -- 172.21.15.37:0/3848016041 >> 172.21.15.37:0/3848016041 conn(0x7f4758071c80 msgr2=0x7f4758072080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:50:30.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:30.241+0000 7f475f400700 1 -- 172.21.15.37:0/3848016041 shutdown_connections 2024-03-29T08:50:30.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:30.241+0000 7f475f400700 1 -- 172.21.15.37:0/3848016041 wait complete. 2024-03-29T08:50:30.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:30.242+0000 7f475f400700 1 Processor -- start 2024-03-29T08:50:30.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:30.242+0000 7f475f400700 1 -- start start 2024-03-29T08:50:30.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:30.243+0000 7f475f400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4758064e40 0x7f4758113790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:50:30.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:30.243+0000 7f475f400700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4758102d80 0x7f4758113cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:50:30.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:30.243+0000 7f475f400700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4758114320 con 0x7f4758064e40 2024-03-29T08:50:30.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:30.243+0000 7f475f400700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f475810d960 con 0x7f4758102d80 2024-03-29T08:50:30.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:30.243+0000 7f475ca00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4758064e40 0x7f4758113790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:50:30.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:30.243+0000 7f475ca00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4758064e40 0x7f4758113790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49254/0 (socket says 172.21.15.37:49254) 2024-03-29T08:50:30.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:30.243+0000 7f475ca00700 1 -- 172.21.15.37:0/2967341439 learned_addr learned my addr 172.21.15.37:0/2967341439 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:50:30.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:30.243+0000 7f4757e00700 1 --2- 172.21.15.37:0/2967341439 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4758102d80 0x7f4758113cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:50:30.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:30.243+0000 7f475ca00700 1 -- 172.21.15.37:0/2967341439 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4758102d80 msgr2=0x7f4758113cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:50:30.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:30.243+0000 7f475ca00700 1 --2- 172.21.15.37:0/2967341439 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4758102d80 0x7f4758113cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:30.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:30.243+0000 7f475ca00700 1 -- 172.21.15.37:0/2967341439 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f474c005040 con 0x7f4758064e40 2024-03-29T08:50:30.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:30.244+0000 7f475ca00700 1 --2- 172.21.15.37:0/2967341439 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4758064e40 0x7f4758113790 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f474c009390 tx=0x7f474c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:50:30.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:30.244+0000 7f4754c00700 1 -- 172.21.15.37:0/2967341439 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f474c00e040 con 0x7f4758064e40 2024-03-29T08:50:30.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:30.244+0000 7f4754c00700 1 -- 172.21.15.37:0/2967341439 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f474c013420 con 0x7f4758064e40 2024-03-29T08:50:30.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:30.244+0000 7f475f400700 1 -- 172.21.15.37:0/2967341439 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f475810dbe0 con 0x7f4758064e40 2024-03-29T08:50:30.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:30.244+0000 7f4754c00700 1 -- 172.21.15.37:0/2967341439 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f474c01c630 con 0x7f4758064e40 2024-03-29T08:50:30.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:30.244+0000 7f475f400700 1 -- 172.21.15.37:0/2967341439 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f475810e120 con 0x7f4758064e40 2024-03-29T08:50:30.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:30.245+0000 7f475f400700 1 -- 172.21.15.37:0/2967341439 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47580fe4e0 con 0x7f4758064e40 2024-03-29T08:50:30.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:30.246+0000 7f4754c00700 1 -- 172.21.15.37:0/2967341439 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f474c0139e0 con 0x7f4758064e40 2024-03-29T08:50:30.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:30.247+0000 7f4754c00700 1 --2- 172.21.15.37:0/2967341439 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f474006cc20 0x7f474006f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:50:30.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:30.247+0000 7f4754c00700 1 -- 172.21.15.37:0/2967341439 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f474c08d2f0 con 0x7f4758064e40 2024-03-29T08:50:30.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:30.247+0000 7f4757e00700 1 --2- 172.21.15.37:0/2967341439 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f474006cc20 0x7f474006f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:50:30.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:30.248+0000 7f4757e00700 1 --2- 172.21.15.37:0/2967341439 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f474006cc20 0x7f474006f0d0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f4758107ab0 tx=0x7f4744009ee0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:50:30.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:30.252+0000 7f4754c00700 1 -- 172.21.15.37:0/2967341439 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f474c05ae70 con 0x7f4758064e40 2024-03-29T08:50:30.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:50:29 smithi148 ceph-mon[106692]: pgmap v1190: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:50:30.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:50:29 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/3682137623' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-03-29T08:50:30.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:50:29 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/12117242' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-03-29T08:50:30.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:30.470+0000 7f475f400700 1 -- 172.21.15.37:0/2967341439 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4758064970 con 0x7f4758064e40 2024-03-29T08:50:30.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:30.472+0000 7f4754c00700 1 -- 172.21.15.37:0/2967341439 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+16549 (secure 0 0 0) 0x7f474c05aa00 con 0x7f4758064e40 2024-03-29T08:50:30.472 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-03-29T08:50:30.472 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":86,"fsid":"99b492c2-eda1-11ee-b647-cb9ed24678a4","created":"2024-03-29T07:55:23.673934+0000","modified":"2024-03-29T08:33:15.225558+0000","last_up_change":"2024-03-29T08:17:12.117876+0000","last_in_change":"2024-03-29T08:01:43.066485+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":20,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T08:00:32.022416+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs.cephfs.meta","create_time":"2024-03-29T08:03:28.176923+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs.cephfs.data","create_time":"2024-03-29T08:03:28.754148+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dba18fb6-b27d-4258-b769-dc89f4fef44d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":50,"up_from":54,"up_thru":79,"down_at":51,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":2026381241},{"type":"v1","addr":"172.21.15.37:6803","nonce":2026381241}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":2026381241},{"type":"v1","addr":"172.21.15.37:6805","nonce":2026381241}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":2026381241},{"type":"v1","addr":"172.21.15.37:6809","nonce":2026381241}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":2026381241},{"type":"v1","addr":"172.21.15.37:6807","nonce":2026381241}]},"public_addr":"172.21.15.37:6803/2026381241","cluster_addr":"172.21.15.37:6805/2026381241","heartbeat_back_addr":"172.21.15.37:6809/2026381241","heartbeat_front_addr":"172.21.15.37:6807/2026381241","state":["exists","up"]},{"osd":1,"uuid":"2876dd3f-aefb-4298-8fe4-562245e1103e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":55,"up_from":59,"up_thru":79,"down_at":56,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":2942536992},{"type":"v1","addr":"172.21.15.37:6811","nonce":2942536992}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":2942536992},{"type":"v1","addr":"172.21.15.37:6813","nonce":2942536992}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2942536992},{"type":"v1","addr":"172.21.15.37:6817","nonce":2942536992}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":2942536992},{"type":"v1","addr":"172.21.15.37:6815","nonce":2942536992}]},"public_addr":"172.21.15.37:6811/2942536992","cluster_addr":"172.21.15.37:6813/2942536992","heartbeat_back_addr":"172.21.15.37:6817/2942536992","heartbeat_front_addr":"172.21.15.37:6815/2942536992","state":["exists","up"]},{"osd":2,"uuid":"bba0f95c-f4e3-404e-adf7-b90353a96af4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":19,"last_clean_end":60,"up_from":64,"up_thru":79,"down_at":61,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":4140655393},{"type":"v1","addr":"172.21.15.37:6819","nonce":4140655393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":4140655393},{"type":"v1","addr":"172.21.15.37:6821","nonce":4140655393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":4140655393},{"type":"v1","addr":"172.21.15.37:6825","nonce":4140655393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":4140655393},{"type":"v1","addr":"172.21.15.37:6823","nonce":4140655393}]},"public_addr":"172.21.15.37:6819/4140655393","cluster_addr":"172.21.15.37:6821/4140655393","heartbeat_back_addr":"172.21.15.37:6825/4140655393","heartbeat_front_addr":"172.21.15.37:6823/4140655393","state":["exists","up"]},{"osd":3,"uuid":"8da1c140-ada5-4879-b5d9-1ed81b0c9aef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":27,"last_clean_end":65,"up_from":69,"up_thru":79,"down_at":66,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":227480876},{"type":"v1","addr":"172.21.15.148:6801","nonce":227480876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":227480876},{"type":"v1","addr":"172.21.15.148:6803","nonce":227480876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":227480876},{"type":"v1","addr":"172.21.15.148:6807","nonce":227480876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":227480876},{"type":"v1","addr":"172.21.15.148:6805","nonce":227480876}]},"public_addr":"172.21.15.148:6801/227480876","cluster_addr":"172.21.15.148:6803/227480876","heartbeat_back_addr":"172.21.15.148:6807/227480876","heartbeat_front_addr":"172.21.15.148:6805/227480876","state":["exists","up"]},{"osd":4,"uuid":"9af70e66-bbca-48c7-bad1-7ac78270cf16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":35,"last_clean_end":70,"up_from":74,"up_thru":79,"down_at":71,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":572347061},{"type":"v1","addr":"172.21.15.148:6809","nonce":572347061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":572347061},{"type":"v1","addr":"172.21.15.148:6811","nonce":572347061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":572347061},{"type":"v1","addr":"172.21.15.148:6815","nonce":572347061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":572347061},{"type":"v1","addr":"172.21.15.148:6813","nonce":572347061}]},"public_addr":"172.21.15.148:6809/572347061","cluster_addr":"172.21.15.148:6811/572347061","heartbeat_back_addr":"172.21.15.148:6815/572347061","heartbeat_front_addr":"172.21.15.148:6813/572347061","state":["exists","up"]},{"osd":5,"uuid":"a66944e4-38bd-4403-b45f-7de01ddfb7a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":40,"last_clean_end":75,"up_from":79,"up_thru":79,"down_at":76,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":2577126633},{"type":"v1","addr":"172.21.15.148:6817","nonce":2577126633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":2577126633},{"type":"v1","addr":"172.21.15.148:6819","nonce":2577126633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":2577126633},{"type":"v1","addr":"172.21.15.148:6823","nonce":2577126633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":2577126633},{"type":"v1","addr":"172.21.15.148:6821","nonce":2577126633}]},"public_addr":"172.21.15.148:6817/2577126633","cluster_addr":"172.21.15.148:6819/2577126633","heartbeat_back_addr":"172.21.15.148:6823/2577126633","heartbeat_front_addr":"172.21.15.148:6821/2577126633","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T07:59:23.110900+0000","dead_epoch":50},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T07:59:55.540549+0000","dead_epoch":55},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:00:28.082290+0000","dead_epoch":60},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:00:59.496893+0000","dead_epoch":65},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:01:31.307445+0000","dead_epoch":70},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:02:03.191177+0000","dead_epoch":75}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:0/3846762095":"2024-03-30T07:57:55.402909+0000","172.21.15.37:6800/2746946658":"2024-03-30T07:56:36.633439+0000","172.21.15.37:0/1909774742":"2024-03-30T07:55:59.989528+0000","172.21.15.37:0/1421768676":"2024-03-30T07:57:55.402909+0000","172.21.15.37:0/1395183736":"2024-03-30T08:06:01.871937+0000","172.21.15.37:6801/1802836043":"2024-03-30T08:06:01.871937+0000","172.21.15.37:0/368538117":"2024-03-30T07:57:55.402909+0000","172.21.15.148:0/855865016":"2024-03-30T08:08:06.449195+0000","172.21.15.37:0/2242434702":"2024-03-30T08:06:01.871937+0000","172.21.15.148:0/1612512235":"2024-03-30T08:08:06.449195+0000","172.21.15.148:6828/1134897184":"2024-03-30T08:08:06.449195+0000","172.21.15.37:0/1131673968":"2024-03-30T07:56:36.633439+0000","172.21.15.37:0/959705723":"2024-03-30T07:56:36.633439+0000","172.21.15.37:0/3667241947":"2024-03-30T08:12:09.488085+0000","172.21.15.37:6827/1559618909":"2024-03-30T08:17:16.236377+0000","172.21.15.37:0/3644857958":"2024-03-30T07:55:59.989528+0000","172.21.15.37:6828/2341530495":"2024-03-30T08:17:16.236404+0000","172.21.15.37:6800/1802836043":"2024-03-30T08:06:01.871937+0000","172.21.15.37:6801/2762339582":"2024-03-30T07:55:59.989528+0000","172.21.15.148:6829/1134897184":"2024-03-30T08:08:06.449195+0000","172.21.15.148:0/1484235408":"2024-03-30T08:08:06.449195+0000","172.21.15.37:6801/2746946658":"2024-03-30T07:56:36.633439+0000","172.21.15.148:0/4266419604":"2024-03-30T08:08:06.449195+0000","172.21.15.37:6800/4245458578":"2024-03-30T07:57:55.402909+0000","172.21.15.37:6829/2341530495":"2024-03-30T08:17:16.236404+0000","172.21.15.148:6824/2050993679":"2024-03-30T08:03:46.637260+0000","172.21.15.37:6801/4245458578":"2024-03-30T07:57:55.402909+0000","172.21.15.37:0/3116346321":"2024-03-30T07:55:59.989528+0000","172.21.15.37:0/2483635161":"2024-03-30T07:56:36.633439+0000","172.21.15.37:6800/4188329516":"2024-03-30T08:12:09.488085+0000","172.21.15.37:6800/2762339582":"2024-03-30T07:55:59.989528+0000","172.21.15.37:0/2522921436":"2024-03-30T08:06:01.871937+0000","172.21.15.37:0/968152496":"2024-03-30T08:12:09.488085+0000","172.21.15.37:0/68050196":"2024-03-30T08:12:09.488085+0000","172.21.15.37:0/2701457924":"2024-03-30T08:06:01.871937+0000","172.21.15.37:6801/4188329516":"2024-03-30T08:12:09.488085+0000","172.21.15.37:6826/1559618909":"2024-03-30T08:17:16.236377+0000","172.21.15.148:6825/2050993679":"2024-03-30T08:03:46.637260+0000","172.21.15.37:0/331291813":"2024-03-30T08:12:09.488085+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T08:50:30.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:30.474+0000 7f475f400700 1 -- 172.21.15.37:0/2967341439 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f474006cc20 msgr2=0x7f474006f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:50:30.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:30.474+0000 7f475f400700 1 --2- 172.21.15.37:0/2967341439 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f474006cc20 0x7f474006f0d0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f4758107ab0 tx=0x7f4744009ee0 comp rx=0 tx=0).stop 2024-03-29T08:50:30.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:30.474+0000 7f475f400700 1 -- 172.21.15.37:0/2967341439 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4758064e40 msgr2=0x7f4758113790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:50:30.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:30.474+0000 7f475f400700 1 --2- 172.21.15.37:0/2967341439 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4758064e40 0x7f4758113790 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f474c009390 tx=0x7f474c009e30 comp rx=0 tx=0).stop 2024-03-29T08:50:30.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:30.474+0000 7f475f400700 1 -- 172.21.15.37:0/2967341439 shutdown_connections 2024-03-29T08:50:30.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:30.474+0000 7f475f400700 1 --2- 172.21.15.37:0/2967341439 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f474006cc20 0x7f474006f0d0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:30.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:30.474+0000 7f475f400700 1 --2- 172.21.15.37:0/2967341439 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4758064e40 0x7f4758113790 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:30.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:30.474+0000 7f475f400700 1 --2- 172.21.15.37:0/2967341439 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4758102d80 0x7f4758113cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:30.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:30.474+0000 7f475f400700 1 -- 172.21.15.37:0/2967341439 >> 172.21.15.37:0/2967341439 conn(0x7f4758071c80 msgr2=0x7f47580fab70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:50:30.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:30.474+0000 7f475f400700 1 -- 172.21.15.37:0/2967341439 shutdown_connections 2024-03-29T08:50:30.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:30.475+0000 7f475f400700 1 -- 172.21.15.37:0/2967341439 wait complete. 2024-03-29T08:50:30.491 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-29T08:00:32.022416+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 6, 'score_stable': 6, 'optimal_score': 0.5, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs.cephfs.meta', 'create_time': '2024-03-29T08:03:28.176923+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.690000057220459, 'score_stable': 1.690000057220459, 'optimal_score': 1, 'raw_score_acting': 1.690000057220459, 'raw_score_stable': 1.690000057220459, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs.cephfs.data', 'create_time': '2024-03-29T08:03:28.754148+0000', 'flags': 131073, 'flags_names': 'hashpspool,bulk', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '46', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-29T08:50:30.492 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-29T08:50:30.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:30.689+0000 7fa066200700 1 -- 172.21.15.37:0/2371382465 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa060070c70 msgr2=0x7fa060108ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:50:30.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:30.689+0000 7fa066200700 1 --2- 172.21.15.37:0/2371382465 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa060070c70 0x7fa060108ff0 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7fa04c004970 tx=0x7fa04c009e30 comp rx=0 tx=0).stop 2024-03-29T08:50:30.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:30.689+0000 7fa066200700 1 -- 172.21.15.37:0/2371382465 shutdown_connections 2024-03-29T08:50:30.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:30.689+0000 7fa066200700 1 --2- 172.21.15.37:0/2371382465 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa060070c70 0x7fa060108ff0 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:30.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:30.689+0000 7fa066200700 1 --2- 172.21.15.37:0/2371382465 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa060070360 0x7fa060070730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:30.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:30.689+0000 7fa066200700 1 -- 172.21.15.37:0/2371382465 >> 172.21.15.37:0/2371382465 conn(0x7fa0600f8830 msgr2=0x7fa0600fac40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:50:30.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:30.689+0000 7fa066200700 1 -- 172.21.15.37:0/2371382465 shutdown_connections 2024-03-29T08:50:30.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:30.689+0000 7fa066200700 1 -- 172.21.15.37:0/2371382465 wait complete. 2024-03-29T08:50:30.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:30.690+0000 7fa066200700 1 Processor -- start 2024-03-29T08:50:30.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:30.690+0000 7fa066200700 1 -- start start 2024-03-29T08:50:30.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:30.690+0000 7fa066200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa060070360 0x7fa060115a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:50:30.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:30.690+0000 7fa066200700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa060070c70 0x7fa060115f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:50:30.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:30.690+0000 7fa066200700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa060116590 con 0x7fa060070360 2024-03-29T08:50:30.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:30.690+0000 7fa066200700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa06010fb90 con 0x7fa060070c70 2024-03-29T08:50:30.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:30.691+0000 7fa05f400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa060070360 0x7fa060115a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:50:30.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:30.691+0000 7fa05f400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa060070360 0x7fa060115a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49272/0 (socket says 172.21.15.37:49272) 2024-03-29T08:50:30.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:30.691+0000 7fa05f400700 1 -- 172.21.15.37:0/3724169391 learned_addr learned my addr 172.21.15.37:0/3724169391 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:50:30.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:30.691+0000 7fa05f400700 1 -- 172.21.15.37:0/3724169391 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa060070c70 msgr2=0x7fa060115f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:50:30.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:30.691+0000 7fa05f400700 1 --2- 172.21.15.37:0/3724169391 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa060070c70 0x7fa060115f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:30.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:30.691+0000 7fa05f400700 1 -- 172.21.15.37:0/3724169391 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa04c005040 con 0x7fa060070360 2024-03-29T08:50:30.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:30.691+0000 7fa05f400700 1 --2- 172.21.15.37:0/3724169391 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa060070360 0x7fa060115a00 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7fa05000c8c0 tx=0x7fa0500044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:50:30.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:30.691+0000 7fa057400700 1 -- 172.21.15.37:0/3724169391 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa05000a980 con 0x7fa060070360 2024-03-29T08:50:30.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:30.691+0000 7fa057400700 1 -- 172.21.15.37:0/3724169391 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fa05000ce60 con 0x7fa060070360 2024-03-29T08:50:30.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:30.692+0000 7fa057400700 1 -- 172.21.15.37:0/3724169391 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa0500113a0 con 0x7fa060070360 2024-03-29T08:50:30.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:30.692+0000 7fa066200700 1 -- 172.21.15.37:0/3724169391 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa06010fe70 con 0x7fa060070360 2024-03-29T08:50:30.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:30.692+0000 7fa066200700 1 -- 172.21.15.37:0/3724169391 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0601103b0 con 0x7fa060070360 2024-03-29T08:50:30.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:30.692+0000 7fa066200700 1 -- 172.21.15.37:0/3724169391 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa060071aa0 con 0x7fa060070360 2024-03-29T08:50:30.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:30.694+0000 7fa057400700 1 -- 172.21.15.37:0/3724169391 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fa050012400 con 0x7fa060070360 2024-03-29T08:50:30.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:30.694+0000 7fa057400700 1 --2- 172.21.15.37:0/3724169391 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fa04406cc20 0x7fa04406f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:50:30.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:30.695+0000 7fa057400700 1 -- 172.21.15.37:0/3724169391 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fa05008c1d0 con 0x7fa060070360 2024-03-29T08:50:30.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:30.695+0000 7fa05ea00700 1 --2- 172.21.15.37:0/3724169391 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fa04406cc20 0x7fa04406f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:50:30.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:30.695+0000 7fa05ea00700 1 --2- 172.21.15.37:0/3724169391 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fa04406cc20 0x7fa04406f0d0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fa04c000c00 tx=0x7fa04c009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:50:30.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:30.699+0000 7fa057400700 1 -- 172.21.15.37:0/3724169391 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fa050059c20 con 0x7fa060070360 2024-03-29T08:50:30.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:30.922+0000 7fa066200700 1 -- 172.21.15.37:0/3724169391 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa06010be90 con 0x7fa060070360 2024-03-29T08:50:30.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:30.922+0000 7fa057400700 1 -- 172.21.15.37:0/3724169391 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v86) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa0500597b0 con 0x7fa060070360 2024-03-29T08:50:30.922 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 1 2024-03-29T08:50:30.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:30.924+0000 7fa066200700 1 -- 172.21.15.37:0/3724169391 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fa04406cc20 msgr2=0x7fa04406f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:50:30.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:30.924+0000 7fa066200700 1 --2- 172.21.15.37:0/3724169391 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fa04406cc20 0x7fa04406f0d0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fa04c000c00 tx=0x7fa04c009e30 comp rx=0 tx=0).stop 2024-03-29T08:50:30.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:30.924+0000 7fa066200700 1 -- 172.21.15.37:0/3724169391 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa060070360 msgr2=0x7fa060115a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:50:30.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:30.924+0000 7fa066200700 1 --2- 172.21.15.37:0/3724169391 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa060070360 0x7fa060115a00 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7fa05000c8c0 tx=0x7fa0500044c0 comp rx=0 tx=0).stop 2024-03-29T08:50:30.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:30.924+0000 7fa066200700 1 -- 172.21.15.37:0/3724169391 shutdown_connections 2024-03-29T08:50:30.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:30.924+0000 7fa066200700 1 --2- 172.21.15.37:0/3724169391 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fa04406cc20 0x7fa04406f0d0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:30.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:30.924+0000 7fa066200700 1 --2- 172.21.15.37:0/3724169391 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa060070360 0x7fa060115a00 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:30.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:30.924+0000 7fa066200700 1 --2- 172.21.15.37:0/3724169391 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa060070c70 0x7fa060115f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:30.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:30.925+0000 7fa066200700 1 -- 172.21.15.37:0/3724169391 >> 172.21.15.37:0/3724169391 conn(0x7fa0600f8830 msgr2=0x7fa0600f9ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:50:30.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:30.925+0000 7fa066200700 1 -- 172.21.15.37:0/3724169391 shutdown_connections 2024-03-29T08:50:30.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:30.925+0000 7fa066200700 1 -- 172.21.15.37:0/3724169391 wait complete. 2024-03-29T08:50:30.941 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.meta pg_num 2024-03-29T08:50:30.945 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:50:30 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/1115882152' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-29T08:50:30.946 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:50:30 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/2967341439' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-29T08:50:31.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:31.137+0000 7fed25c00700 1 -- 172.21.15.37:0/1385433043 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed20070360 msgr2=0x7fed20070730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:50:31.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:31.137+0000 7fed25c00700 1 --2- 172.21.15.37:0/1385433043 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed20070360 0x7fed20070730 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7fed10004970 tx=0x7fed10009e30 comp rx=0 tx=0).stop 2024-03-29T08:50:31.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:31.137+0000 7fed25c00700 1 -- 172.21.15.37:0/1385433043 shutdown_connections 2024-03-29T08:50:31.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:31.137+0000 7fed25c00700 1 --2- 172.21.15.37:0/1385433043 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fed20070c70 0x7fed20108fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:31.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:31.137+0000 7fed25c00700 1 --2- 172.21.15.37:0/1385433043 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed20070360 0x7fed20070730 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:31.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:31.137+0000 7fed25c00700 1 -- 172.21.15.37:0/1385433043 >> 172.21.15.37:0/1385433043 conn(0x7fed200f8820 msgr2=0x7fed200fac30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:50:31.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:31.137+0000 7fed25c00700 1 -- 172.21.15.37:0/1385433043 shutdown_connections 2024-03-29T08:50:31.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:31.137+0000 7fed25c00700 1 -- 172.21.15.37:0/1385433043 wait complete. 2024-03-29T08:50:31.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:31.138+0000 7fed25c00700 1 Processor -- start 2024-03-29T08:50:31.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:31.138+0000 7fed25c00700 1 -- start start 2024-03-29T08:50:31.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:31.138+0000 7fed25c00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed20070360 0x7fed20115970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:50:31.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:31.138+0000 7fed25c00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fed20070c70 0x7fed20115eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:50:31.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:31.138+0000 7fed25c00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed20116500 con 0x7fed20070360 2024-03-29T08:50:31.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:31.138+0000 7fed25c00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed2010fb00 con 0x7fed20070c70 2024-03-29T08:50:31.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:31.139+0000 7fed1ea00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed20070360 0x7fed20115970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:50:31.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:31.139+0000 7fed1ea00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed20070360 0x7fed20115970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49290/0 (socket says 172.21.15.37:49290) 2024-03-29T08:50:31.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:31.139+0000 7fed1ea00700 1 -- 172.21.15.37:0/123767465 learned_addr learned my addr 172.21.15.37:0/123767465 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:50:31.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:31.139+0000 7fed1e000700 1 --2- 172.21.15.37:0/123767465 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fed20070c70 0x7fed20115eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:50:31.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:31.139+0000 7fed1ea00700 1 -- 172.21.15.37:0/123767465 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fed20070c70 msgr2=0x7fed20115eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:50:31.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:31.139+0000 7fed1ea00700 1 --2- 172.21.15.37:0/123767465 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fed20070c70 0x7fed20115eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:31.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:31.139+0000 7fed1ea00700 1 -- 172.21.15.37:0/123767465 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed10005040 con 0x7fed20070360 2024-03-29T08:50:31.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:31.139+0000 7fed1e000700 1 --2- 172.21.15.37:0/123767465 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fed20070c70 0x7fed20115eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:50:31.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:31.139+0000 7fed1ea00700 1 --2- 172.21.15.37:0/123767465 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed20070360 0x7fed20115970 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7fed10004f50 tx=0x7fed10009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:50:31.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:31.140+0000 7fed16a00700 1 -- 172.21.15.37:0/123767465 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fed1000e040 con 0x7fed20070360 2024-03-29T08:50:31.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:31.140+0000 7fed16a00700 1 -- 172.21.15.37:0/123767465 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fed10013420 con 0x7fed20070360 2024-03-29T08:50:31.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:31.140+0000 7fed25c00700 1 -- 172.21.15.37:0/123767465 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed2010fd80 con 0x7fed20070360 2024-03-29T08:50:31.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:31.140+0000 7fed16a00700 1 -- 172.21.15.37:0/123767465 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fed1001c610 con 0x7fed20070360 2024-03-29T08:50:31.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:31.140+0000 7fed25c00700 1 -- 172.21.15.37:0/123767465 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed201102c0 con 0x7fed20070360 2024-03-29T08:50:31.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:31.141+0000 7fed25c00700 1 -- 172.21.15.37:0/123767465 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed20071aa0 con 0x7fed20070360 2024-03-29T08:50:31.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:31.142+0000 7fed16a00700 1 -- 172.21.15.37:0/123767465 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fed10013590 con 0x7fed20070360 2024-03-29T08:50:31.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:31.142+0000 7fed16a00700 1 --2- 172.21.15.37:0/123767465 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fed0406cc20 0x7fed0406f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:50:31.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:31.143+0000 7fed16a00700 1 -- 172.21.15.37:0/123767465 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fed1008d260 con 0x7fed20070360 2024-03-29T08:50:31.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:31.143+0000 7fed1e000700 1 --2- 172.21.15.37:0/123767465 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fed0406cc20 0x7fed0406f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:50:31.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:31.143+0000 7fed1e000700 1 --2- 172.21.15.37:0/123767465 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fed0406cc20 0x7fed0406f0d0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fed0c007fd0 tx=0x7fed0c009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:50:31.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:31.147+0000 7fed16a00700 1 -- 172.21.15.37:0/123767465 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fed1005acb0 con 0x7fed20070360 2024-03-29T08:50:31.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:50:30 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/1115882152' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-29T08:50:31.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:50:30 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/2967341439' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-29T08:50:31.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:31.367+0000 7fed25c00700 1 -- 172.21.15.37:0/123767465 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"} v 0) v1 -- 0x7fed2010be80 con 0x7fed20070360 2024-03-29T08:50:31.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:31.367+0000 7fed16a00700 1 -- 172.21.15.37:0/123767465 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]=0 v86) v1 ==== 107+0+11 (secure 0 0 0) 0x7fed1005a840 con 0x7fed20070360 2024-03-29T08:50:31.367 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 32 2024-03-29T08:50:31.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:31.370+0000 7fed25c00700 1 -- 172.21.15.37:0/123767465 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fed0406cc20 msgr2=0x7fed0406f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:50:31.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:31.370+0000 7fed25c00700 1 --2- 172.21.15.37:0/123767465 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fed0406cc20 0x7fed0406f0d0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fed0c007fd0 tx=0x7fed0c009040 comp rx=0 tx=0).stop 2024-03-29T08:50:31.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:31.370+0000 7fed25c00700 1 -- 172.21.15.37:0/123767465 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed20070360 msgr2=0x7fed20115970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:50:31.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:31.370+0000 7fed25c00700 1 --2- 172.21.15.37:0/123767465 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed20070360 0x7fed20115970 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7fed10004f50 tx=0x7fed10009e30 comp rx=0 tx=0).stop 2024-03-29T08:50:31.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:31.370+0000 7fed25c00700 1 -- 172.21.15.37:0/123767465 shutdown_connections 2024-03-29T08:50:31.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:31.370+0000 7fed25c00700 1 --2- 172.21.15.37:0/123767465 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fed0406cc20 0x7fed0406f0d0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:31.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:31.370+0000 7fed25c00700 1 --2- 172.21.15.37:0/123767465 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed20070360 0x7fed20115970 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:31.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:31.370+0000 7fed25c00700 1 --2- 172.21.15.37:0/123767465 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fed20070c70 0x7fed20115eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:31.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:31.370+0000 7fed25c00700 1 -- 172.21.15.37:0/123767465 >> 172.21.15.37:0/123767465 conn(0x7fed200f8820 msgr2=0x7fed200f9d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:50:31.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:31.370+0000 7fed25c00700 1 -- 172.21.15.37:0/123767465 shutdown_connections 2024-03-29T08:50:31.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:31.370+0000 7fed25c00700 1 -- 172.21.15.37:0/123767465 wait complete. 2024-03-29T08:50:31.387 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.data pg_num 2024-03-29T08:50:31.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:31.624+0000 7fd928e00700 1 -- 172.21.15.37:0/1410505979 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd924070360 msgr2=0x7fd924070730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:50:31.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:31.624+0000 7fd928e00700 1 --2- 172.21.15.37:0/1410505979 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd924070360 0x7fd924070730 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7fd914004970 tx=0x7fd914009e30 comp rx=0 tx=0).stop 2024-03-29T08:50:31.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:31.624+0000 7fd928e00700 1 -- 172.21.15.37:0/1410505979 shutdown_connections 2024-03-29T08:50:31.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:31.624+0000 7fd928e00700 1 --2- 172.21.15.37:0/1410505979 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd924070c70 0x7fd924108fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:31.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:31.624+0000 7fd928e00700 1 --2- 172.21.15.37:0/1410505979 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd924070360 0x7fd924070730 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:31.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:31.624+0000 7fd928e00700 1 -- 172.21.15.37:0/1410505979 >> 172.21.15.37:0/1410505979 conn(0x7fd9240f8820 msgr2=0x7fd9240fac30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:50:31.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:31.624+0000 7fd928e00700 1 -- 172.21.15.37:0/1410505979 shutdown_connections 2024-03-29T08:50:31.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:31.624+0000 7fd928e00700 1 -- 172.21.15.37:0/1410505979 wait complete. 2024-03-29T08:50:31.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:31.625+0000 7fd928e00700 1 Processor -- start 2024-03-29T08:50:31.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:31.625+0000 7fd928e00700 1 -- start start 2024-03-29T08:50:31.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:31.626+0000 7fd928e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd924070360 0x7fd924115970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:50:31.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:31.626+0000 7fd928e00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd924070c70 0x7fd924115eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:50:31.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:31.626+0000 7fd928e00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd924116500 con 0x7fd924070360 2024-03-29T08:50:31.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:31.626+0000 7fd928e00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd92410fb00 con 0x7fd924070c70 2024-03-29T08:50:31.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:31.626+0000 7fd921e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd924070360 0x7fd924115970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:50:31.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:31.626+0000 7fd921e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd924070360 0x7fd924115970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49306/0 (socket says 172.21.15.37:49306) 2024-03-29T08:50:31.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:31.626+0000 7fd921e00700 1 -- 172.21.15.37:0/304601793 learned_addr learned my addr 172.21.15.37:0/304601793 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:50:31.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:31.626+0000 7fd921400700 1 --2- 172.21.15.37:0/304601793 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd924070c70 0x7fd924115eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:50:31.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:31.627+0000 7fd921e00700 1 -- 172.21.15.37:0/304601793 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd924070c70 msgr2=0x7fd924115eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:50:31.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:31.627+0000 7fd921e00700 1 --2- 172.21.15.37:0/304601793 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd924070c70 0x7fd924115eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:31.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:31.627+0000 7fd921e00700 1 -- 172.21.15.37:0/304601793 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd914005040 con 0x7fd924070360 2024-03-29T08:50:31.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:31.627+0000 7fd921400700 1 --2- 172.21.15.37:0/304601793 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd924070c70 0x7fd924115eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:50:31.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:31.627+0000 7fd921e00700 1 --2- 172.21.15.37:0/304601793 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd924070360 0x7fd924115970 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7fd914004f50 tx=0x7fd914009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:50:31.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:31.627+0000 7fd91a000700 1 -- 172.21.15.37:0/304601793 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd91400e040 con 0x7fd924070360 2024-03-29T08:50:31.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:31.627+0000 7fd91a000700 1 -- 172.21.15.37:0/304601793 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd914013420 con 0x7fd924070360 2024-03-29T08:50:31.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:31.627+0000 7fd91a000700 1 -- 172.21.15.37:0/304601793 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd91401c610 con 0x7fd924070360 2024-03-29T08:50:31.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:31.627+0000 7fd928e00700 1 -- 172.21.15.37:0/304601793 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd92410fd80 con 0x7fd924070360 2024-03-29T08:50:31.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:31.627+0000 7fd928e00700 1 -- 172.21.15.37:0/304601793 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd924110270 con 0x7fd924070360 2024-03-29T08:50:31.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:31.628+0000 7fd928e00700 1 -- 172.21.15.37:0/304601793 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd924071aa0 con 0x7fd924070360 2024-03-29T08:50:31.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:31.629+0000 7fd91a000700 1 -- 172.21.15.37:0/304601793 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fd914013590 con 0x7fd924070360 2024-03-29T08:50:31.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:31.630+0000 7fd91a000700 1 --2- 172.21.15.37:0/304601793 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fd90806cc20 0x7fd90806f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:50:31.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:31.630+0000 7fd91a000700 1 -- 172.21.15.37:0/304601793 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fd91408d310 con 0x7fd924070360 2024-03-29T08:50:31.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:31.630+0000 7fd921400700 1 --2- 172.21.15.37:0/304601793 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fd90806cc20 0x7fd90806f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:50:31.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:31.631+0000 7fd921400700 1 --2- 172.21.15.37:0/304601793 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fd90806cc20 0x7fd90806f0d0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fd910009fd0 tx=0x7fd91000b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:50:31.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:31.635+0000 7fd91a000700 1 -- 172.21.15.37:0/304601793 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fd91405ace0 con 0x7fd924070360 2024-03-29T08:50:31.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:31.851+0000 7fd928e00700 1 -- 172.21.15.37:0/304601793 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"} v 0) v1 -- 0x7fd924110ff0 con 0x7fd924070360 2024-03-29T08:50:31.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:31.852+0000 7fd91a000700 1 -- 172.21.15.37:0/304601793 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]=0 v86) v1 ==== 107+0+11 (secure 0 0 0) 0x7fd91405a870 con 0x7fd924070360 2024-03-29T08:50:31.851 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 32 2024-03-29T08:50:31.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:31.854+0000 7fd928e00700 1 -- 172.21.15.37:0/304601793 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fd90806cc20 msgr2=0x7fd90806f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:50:31.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:31.854+0000 7fd928e00700 1 --2- 172.21.15.37:0/304601793 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fd90806cc20 0x7fd90806f0d0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fd910009fd0 tx=0x7fd91000b040 comp rx=0 tx=0).stop 2024-03-29T08:50:31.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:31.854+0000 7fd928e00700 1 -- 172.21.15.37:0/304601793 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd924070360 msgr2=0x7fd924115970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:50:31.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:31.854+0000 7fd928e00700 1 --2- 172.21.15.37:0/304601793 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd924070360 0x7fd924115970 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7fd914004f50 tx=0x7fd914009e30 comp rx=0 tx=0).stop 2024-03-29T08:50:31.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:31.854+0000 7fd928e00700 1 -- 172.21.15.37:0/304601793 shutdown_connections 2024-03-29T08:50:31.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:31.854+0000 7fd928e00700 1 --2- 172.21.15.37:0/304601793 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7fd90806cc20 0x7fd90806f0d0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:31.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:31.854+0000 7fd928e00700 1 --2- 172.21.15.37:0/304601793 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd924070360 0x7fd924115970 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:31.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:31.854+0000 7fd928e00700 1 --2- 172.21.15.37:0/304601793 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd924070c70 0x7fd924115eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:31.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:31.854+0000 7fd928e00700 1 -- 172.21.15.37:0/304601793 >> 172.21.15.37:0/304601793 conn(0x7fd9240f8820 msgr2=0x7fd9240f9d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:50:31.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:31.854+0000 7fd928e00700 1 -- 172.21.15.37:0/304601793 shutdown_connections 2024-03-29T08:50:31.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:31.854+0000 7fd928e00700 1 -- 172.21.15.37:0/304601793 wait complete. 2024-03-29T08:50:31.872 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-29T08:50:32.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:32.067+0000 7f4fdea00700 1 -- 172.21.15.37:0/2740799753 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4fd8102cd0 msgr2=0x7f4fd81030a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:50:32.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:32.067+0000 7f4fdea00700 1 --2- 172.21.15.37:0/2740799753 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4fd8102cd0 0x7f4fd81030a0 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f4fc8004940 tx=0x7f4fc8009e30 comp rx=0 tx=0).stop 2024-03-29T08:50:32.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:32.067+0000 7f4fdea00700 1 -- 172.21.15.37:0/2740799753 shutdown_connections 2024-03-29T08:50:32.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:32.067+0000 7f4fdea00700 1 --2- 172.21.15.37:0/2740799753 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4fd8103670 0x7f4fd80fcce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:32.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:32.067+0000 7f4fdea00700 1 --2- 172.21.15.37:0/2740799753 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4fd8102cd0 0x7f4fd81030a0 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:32.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:32.067+0000 7f4fdea00700 1 -- 172.21.15.37:0/2740799753 >> 172.21.15.37:0/2740799753 conn(0x7f4fd80f8820 msgr2=0x7f4fd80fac30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:50:32.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:32.067+0000 7f4fdea00700 1 -- 172.21.15.37:0/2740799753 shutdown_connections 2024-03-29T08:50:32.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:32.067+0000 7f4fdea00700 1 -- 172.21.15.37:0/2740799753 wait complete. 2024-03-29T08:50:32.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:32.068+0000 7f4fdea00700 1 Processor -- start 2024-03-29T08:50:32.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:32.068+0000 7f4fdea00700 1 -- start start 2024-03-29T08:50:32.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:32.069+0000 7f4fdea00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4fd8102cd0 0x7f4fd8115a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:50:32.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:32.069+0000 7f4fdea00700 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4fd8103670 0x7f4fd8115f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:50:32.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:32.069+0000 7f4fdea00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fd81165b0 con 0x7f4fd8102cd0 2024-03-29T08:50:32.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:32.069+0000 7f4fdea00700 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fd810fb90 con 0x7f4fd8103670 2024-03-29T08:50:32.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:32.069+0000 7f4fd7e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4fd8102cd0 0x7f4fd8115a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:50:32.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:32.069+0000 7f4fd7e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4fd8102cd0 0x7f4fd8115a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49312/0 (socket says 172.21.15.37:49312) 2024-03-29T08:50:32.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:32.069+0000 7f4fd7e00700 1 -- 172.21.15.37:0/843509921 learned_addr learned my addr 172.21.15.37:0/843509921 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-03-29T08:50:32.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:32.069+0000 7f4fd7400700 1 --2- 172.21.15.37:0/843509921 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4fd8103670 0x7f4fd8115f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:50:32.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:32.069+0000 7f4fd7e00700 1 -- 172.21.15.37:0/843509921 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4fd8103670 msgr2=0x7f4fd8115f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:50:32.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:32.069+0000 7f4fd7e00700 1 --2- 172.21.15.37:0/843509921 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4fd8103670 0x7f4fd8115f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:32.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:32.069+0000 7f4fd7e00700 1 -- 172.21.15.37:0/843509921 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fc8005040 con 0x7f4fd8102cd0 2024-03-29T08:50:32.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:32.070+0000 7f4fd7e00700 1 --2- 172.21.15.37:0/843509921 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4fd8102cd0 0x7f4fd8115a20 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f4fc8009360 tx=0x7f4fc8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:50:32.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:32.070+0000 7f4fcfe00700 1 -- 172.21.15.37:0/843509921 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4fc800e040 con 0x7f4fd8102cd0 2024-03-29T08:50:32.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:32.070+0000 7f4fcfe00700 1 -- 172.21.15.37:0/843509921 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f4fc8013420 con 0x7f4fd8102cd0 2024-03-29T08:50:32.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:32.070+0000 7f4fcfe00700 1 -- 172.21.15.37:0/843509921 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4fc801c5d0 con 0x7f4fd8102cd0 2024-03-29T08:50:32.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:32.070+0000 7f4fdea00700 1 -- 172.21.15.37:0/843509921 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fd810fe10 con 0x7f4fd8102cd0 2024-03-29T08:50:32.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:32.070+0000 7f4fdea00700 1 -- 172.21.15.37:0/843509921 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fd81103d0 con 0x7f4fd8102cd0 2024-03-29T08:50:32.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:32.071+0000 7f4fdea00700 1 -- 172.21.15.37:0/843509921 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fd80fe420 con 0x7f4fd8102cd0 2024-03-29T08:50:32.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:32.072+0000 7f4fcfe00700 1 -- 172.21.15.37:0/843509921 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f4fc8013590 con 0x7f4fd8102cd0 2024-03-29T08:50:32.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:32.073+0000 7f4fcfe00700 1 --2- 172.21.15.37:0/843509921 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f4fbc06cc20 0x7f4fbc06f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:50:32.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:32.074+0000 7f4fcfe00700 1 -- 172.21.15.37:0/843509921 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f4fc808d1f0 con 0x7f4fd8102cd0 2024-03-29T08:50:32.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:32.074+0000 7f4fd7400700 1 --2- 172.21.15.37:0/843509921 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f4fbc06cc20 0x7f4fbc06f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:50:32.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:32.074+0000 7f4fd7400700 1 --2- 172.21.15.37:0/843509921 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f4fbc06cc20 0x7f4fbc06f0d0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f4fd81112c0 tx=0x7f4fc4009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:50:32.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:32.078+0000 7f4fcfe00700 1 -- 172.21.15.37:0/843509921 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f4fc805ac40 con 0x7f4fd8102cd0 2024-03-29T08:50:32.176 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:50:31 smithi037 ceph-mon[142325]: pgmap v1191: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:50:32.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:50:31 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/3724169391' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-29T08:50:32.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:50:31 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/123767465' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-03-29T08:50:32.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:50:31 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/304601793' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-03-29T08:50:32.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:50:31 smithi148 ceph-mon[106692]: pgmap v1191: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:50:32.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:50:31 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/3724169391' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-29T08:50:32.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:50:31 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/123767465' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-03-29T08:50:32.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:50:31 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/304601793' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-03-29T08:50:32.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:32.315+0000 7f4fdea00700 1 -- 172.21.15.37:0/843509921 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f4fd8064970 con 0x7f4fd8102cd0 2024-03-29T08:50:32.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:32.315+0000 7f4fcfe00700 1 -- 172.21.15.37:0/843509921 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 39 entries v86) v1 ==== 81+0+2317 (secure 0 0 0) 0x7f4fc805a7d0 con 0x7f4fd8102cd0 2024-03-29T08:50:32.315 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:0/3846762095 2024-03-30T07:57:55.402909+0000 2024-03-29T08:50:32.315 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:6800/2746946658 2024-03-30T07:56:36.633439+0000 2024-03-29T08:50:32.315 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:0/1909774742 2024-03-30T07:55:59.989528+0000 2024-03-29T08:50:32.315 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:0/1421768676 2024-03-30T07:57:55.402909+0000 2024-03-29T08:50:32.315 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:0/1395183736 2024-03-30T08:06:01.871937+0000 2024-03-29T08:50:32.315 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:6801/1802836043 2024-03-30T08:06:01.871937+0000 2024-03-29T08:50:32.315 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:0/368538117 2024-03-30T07:57:55.402909+0000 2024-03-29T08:50:32.315 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.148:0/855865016 2024-03-30T08:08:06.449195+0000 2024-03-29T08:50:32.316 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:0/2242434702 2024-03-30T08:06:01.871937+0000 2024-03-29T08:50:32.316 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.148:0/1612512235 2024-03-30T08:08:06.449195+0000 2024-03-29T08:50:32.316 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.148:6828/1134897184 2024-03-30T08:08:06.449195+0000 2024-03-29T08:50:32.316 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:0/1131673968 2024-03-30T07:56:36.633439+0000 2024-03-29T08:50:32.316 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:0/959705723 2024-03-30T07:56:36.633439+0000 2024-03-29T08:50:32.316 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:0/3667241947 2024-03-30T08:12:09.488085+0000 2024-03-29T08:50:32.316 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:6827/1559618909 2024-03-30T08:17:16.236377+0000 2024-03-29T08:50:32.316 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:0/3644857958 2024-03-30T07:55:59.989528+0000 2024-03-29T08:50:32.316 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:6828/2341530495 2024-03-30T08:17:16.236404+0000 2024-03-29T08:50:32.316 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:6800/1802836043 2024-03-30T08:06:01.871937+0000 2024-03-29T08:50:32.316 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:6801/2762339582 2024-03-30T07:55:59.989528+0000 2024-03-29T08:50:32.316 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.148:6829/1134897184 2024-03-30T08:08:06.449195+0000 2024-03-29T08:50:32.316 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.148:0/1484235408 2024-03-30T08:08:06.449195+0000 2024-03-29T08:50:32.316 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:6801/2746946658 2024-03-30T07:56:36.633439+0000 2024-03-29T08:50:32.316 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.148:0/4266419604 2024-03-30T08:08:06.449195+0000 2024-03-29T08:50:32.316 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:6800/4245458578 2024-03-30T07:57:55.402909+0000 2024-03-29T08:50:32.317 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:6829/2341530495 2024-03-30T08:17:16.236404+0000 2024-03-29T08:50:32.317 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.148:6824/2050993679 2024-03-30T08:03:46.637260+0000 2024-03-29T08:50:32.317 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:6801/4245458578 2024-03-30T07:57:55.402909+0000 2024-03-29T08:50:32.317 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:0/3116346321 2024-03-30T07:55:59.989528+0000 2024-03-29T08:50:32.317 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:0/2483635161 2024-03-30T07:56:36.633439+0000 2024-03-29T08:50:32.317 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:6800/4188329516 2024-03-30T08:12:09.488085+0000 2024-03-29T08:50:32.317 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:6800/2762339582 2024-03-30T07:55:59.989528+0000 2024-03-29T08:50:32.317 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:0/2522921436 2024-03-30T08:06:01.871937+0000 2024-03-29T08:50:32.317 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:0/968152496 2024-03-30T08:12:09.488085+0000 2024-03-29T08:50:32.317 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:0/68050196 2024-03-30T08:12:09.488085+0000 2024-03-29T08:50:32.317 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:0/2701457924 2024-03-30T08:06:01.871937+0000 2024-03-29T08:50:32.317 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:6801/4188329516 2024-03-30T08:12:09.488085+0000 2024-03-29T08:50:32.317 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:6826/1559618909 2024-03-30T08:17:16.236377+0000 2024-03-29T08:50:32.317 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.148:6825/2050993679 2024-03-30T08:03:46.637260+0000 2024-03-29T08:50:32.318 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:0/331291813 2024-03-30T08:12:09.488085+0000 2024-03-29T08:50:32.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:32.318+0000 7f4fdea00700 1 -- 172.21.15.37:0/843509921 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f4fbc06cc20 msgr2=0x7f4fbc06f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:50:32.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:32.318+0000 7f4fdea00700 1 --2- 172.21.15.37:0/843509921 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f4fbc06cc20 0x7f4fbc06f0d0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f4fd81112c0 tx=0x7f4fc4009040 comp rx=0 tx=0).stop 2024-03-29T08:50:32.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:32.318+0000 7f4fdea00700 1 -- 172.21.15.37:0/843509921 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4fd8102cd0 msgr2=0x7f4fd8115a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:50:32.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:32.318+0000 7f4fdea00700 1 --2- 172.21.15.37:0/843509921 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4fd8102cd0 0x7f4fd8115a20 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f4fc8009360 tx=0x7f4fc8009e30 comp rx=0 tx=0).stop 2024-03-29T08:50:32.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:32.318+0000 7f4fdea00700 1 -- 172.21.15.37:0/843509921 shutdown_connections 2024-03-29T08:50:32.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:32.318+0000 7f4fdea00700 1 --2- 172.21.15.37:0/843509921 >> [v2:172.21.15.37:6800/794131733,v1:172.21.15.37:6801/794131733] conn(0x7f4fbc06cc20 0x7f4fbc06f0d0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:32.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:32.318+0000 7f4fdea00700 1 --2- 172.21.15.37:0/843509921 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4fd8102cd0 0x7f4fd8115a20 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:32.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:32.318+0000 7f4fdea00700 1 --2- 172.21.15.37:0/843509921 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4fd8103670 0x7f4fd8115f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:50:32.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:32.318+0000 7f4fdea00700 1 -- 172.21.15.37:0/843509921 >> 172.21.15.37:0/843509921 conn(0x7f4fd80f8820 msgr2=0x7f4fd80f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:50:32.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:32.318+0000 7f4fdea00700 1 -- 172.21.15.37:0/843509921 shutdown_connections 2024-03-29T08:50:32.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-03-29T08:50:32.318+0000 7f4fdea00700 1 -- 172.21.15.37:0/843509921 wait complete. 2024-03-29T08:50:32.319 INFO:teuthology.orchestra.run.smithi037.stderr:listed 39 entries 2024-03-29T08:50:32.335 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi037.front.sepia.ceph.com... 2024-03-29T08:50:32.336 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:50:32.336 DEBUG:teuthology.orchestra.run.smithi037:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-03-29T08:50:32.404 INFO:teuthology.orchestra.run:waiting for 300 2024-03-29T08:50:33.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:50:32 smithi037 ceph-mon[142325]: from='client.? 172.21.15.37:0/843509921' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-29T08:50:33.305 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:50:32 smithi148 ceph-mon[106692]: from='client.? 172.21.15.37:0/843509921' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-29T08:50:34.176 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:50:33 smithi037 ceph-mon[142325]: pgmap v1192: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:50:34.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:50:33 smithi148 ceph-mon[106692]: pgmap v1192: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:50:36.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:50:35 smithi037 ceph-mon[142325]: pgmap v1193: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:50:36.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:50:35 smithi148 ceph-mon[106692]: pgmap v1193: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:50:38.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:50:37 smithi037 ceph-mon[142325]: pgmap v1194: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:50:38.305 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:50:37 smithi148 ceph-mon[106692]: pgmap v1194: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:50:40.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:50:39 smithi037 ceph-mon[142325]: pgmap v1195: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:50:40.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:50:39 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:50:40.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:50:39 smithi148 ceph-mon[106692]: pgmap v1195: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:50:40.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:50:39 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:50:42.176 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:50:41 smithi037 ceph-mon[142325]: pgmap v1196: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:50:42.305 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:50:41 smithi148 ceph-mon[106692]: pgmap v1196: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:50:44.305 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:50:43 smithi148 ceph-mon[106692]: pgmap v1197: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:50:44.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:50:43 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T08:50:44.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:50:43 smithi037 ceph-mon[142325]: pgmap v1197: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:50:44.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:50:43 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T08:50:45.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:50:44 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T08:50:45.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:50:44 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T08:50:45.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:50:44 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:50:45.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:50:44 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T08:50:45.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:50:44 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T08:50:45.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:50:44 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:50:46.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:50:45 smithi148 ceph-mon[106692]: pgmap v1198: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:50:46.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:50:45 smithi037 ceph-mon[142325]: pgmap v1198: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:50:48.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:50:47 smithi148 ceph-mon[106692]: pgmap v1199: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:50:48.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:50:47 smithi037 ceph-mon[142325]: pgmap v1199: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:50:50.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:50:49 smithi148 ceph-mon[106692]: pgmap v1200: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:50:50.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:50:49 smithi037 ceph-mon[142325]: pgmap v1200: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:50:52.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:50:51 smithi148 ceph-mon[106692]: pgmap v1201: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:50:52.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:50:51 smithi037 ceph-mon[142325]: pgmap v1201: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:50:54.305 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:50:53 smithi148 ceph-mon[106692]: pgmap v1202: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:50:54.426 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:50:53 smithi037 ceph-mon[142325]: pgmap v1202: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:50:55.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:50:54 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:50:55.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:50:54 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:50:56.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:50:55 smithi148 ceph-mon[106692]: pgmap v1203: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:50:56.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:50:55 smithi037 ceph-mon[142325]: pgmap v1203: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:50:58.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:50:57 smithi148 ceph-mon[106692]: pgmap v1204: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:50:58.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:50:57 smithi037 ceph-mon[142325]: pgmap v1204: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:51:00.305 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:50:59 smithi148 ceph-mon[106692]: pgmap v1205: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:51:00.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:50:59 smithi037 ceph-mon[142325]: pgmap v1205: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:51:02.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:51:01 smithi148 ceph-mon[106692]: pgmap v1206: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:51:02.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:51:01 smithi037 ceph-mon[142325]: pgmap v1206: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:51:04.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:51:03 smithi148 ceph-mon[106692]: pgmap v1207: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:51:04.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:51:03 smithi037 ceph-mon[142325]: pgmap v1207: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:51:06.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:51:05 smithi148 ceph-mon[106692]: pgmap v1208: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:51:06.426 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:51:05 smithi037 ceph-mon[142325]: pgmap v1208: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:51:08.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:51:07 smithi148 ceph-mon[106692]: pgmap v1209: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:51:08.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:51:07 smithi037 ceph-mon[142325]: pgmap v1209: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:51:10.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:51:09 smithi148 ceph-mon[106692]: pgmap v1210: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:51:10.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:51:09 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:51:10.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:51:09 smithi037 ceph-mon[142325]: pgmap v1210: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:51:10.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:51:09 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:51:12.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:51:11 smithi148 ceph-mon[106692]: pgmap v1211: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:51:12.426 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:51:11 smithi037 ceph-mon[142325]: pgmap v1211: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:51:14.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:51:14 smithi148 ceph-mon[106692]: pgmap v1212: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:51:14.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:51:14 smithi037 ceph-mon[142325]: pgmap v1212: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:51:15.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:51:15 smithi037 ceph-mon[142325]: pgmap v1213: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:51:16.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:51:15 smithi148 ceph-mon[106692]: pgmap v1213: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:51:17.926 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:51:17 smithi037 ceph-mon[142325]: pgmap v1214: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:51:18.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:51:17 smithi148 ceph-mon[106692]: pgmap v1214: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:51:19.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:51:19 smithi037 ceph-mon[142325]: pgmap v1215: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:51:20.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:51:19 smithi148 ceph-mon[106692]: pgmap v1215: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:51:22.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:51:21 smithi148 ceph-mon[106692]: pgmap v1216: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:51:22.176 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:51:21 smithi037 ceph-mon[142325]: pgmap v1216: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:51:24.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:51:23 smithi148 ceph-mon[106692]: pgmap v1217: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:51:24.176 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:51:23 smithi037 ceph-mon[142325]: pgmap v1217: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:51:25.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:51:24 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:51:25.176 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:51:24 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:51:26.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:51:25 smithi148 ceph-mon[106692]: pgmap v1218: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:51:26.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:51:25 smithi037 ceph-mon[142325]: pgmap v1218: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:51:28.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:51:27 smithi148 ceph-mon[106692]: pgmap v1219: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:51:28.176 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:51:27 smithi037 ceph-mon[142325]: pgmap v1219: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:51:30.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:51:29 smithi148 ceph-mon[106692]: pgmap v1220: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:51:30.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:51:29 smithi037 ceph-mon[142325]: pgmap v1220: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:51:32.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:51:31 smithi148 ceph-mon[106692]: pgmap v1221: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:51:32.176 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:51:31 smithi037 ceph-mon[142325]: pgmap v1221: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:51:34.058 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:51:33 smithi148 ceph-mon[106692]: pgmap v1222: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:51:34.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:51:33 smithi037 ceph-mon[142325]: pgmap v1222: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:51:36.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:51:35 smithi148 ceph-mon[106692]: pgmap v1223: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:51:36.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:51:35 smithi037 ceph-mon[142325]: pgmap v1223: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:51:38.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:51:37 smithi148 ceph-mon[106692]: pgmap v1224: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:51:38.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:51:37 smithi037 ceph-mon[142325]: pgmap v1224: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:51:40.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:51:39 smithi148 ceph-mon[106692]: pgmap v1225: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:51:40.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:51:39 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:51:40.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:51:39 smithi037 ceph-mon[142325]: pgmap v1225: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:51:40.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:51:39 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:51:42.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:51:41 smithi148 ceph-mon[106692]: pgmap v1226: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:51:42.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:51:41 smithi037 ceph-mon[142325]: pgmap v1226: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:51:44.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:51:43 smithi148 ceph-mon[106692]: pgmap v1227: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:51:44.176 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:51:43 smithi037 ceph-mon[142325]: pgmap v1227: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:51:46.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:51:45 smithi148 ceph-mon[106692]: pgmap v1228: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:51:46.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:51:45 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T08:51:46.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:51:45 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T08:51:46.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:51:45 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T08:51:46.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:51:45 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:51:46.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:51:45 smithi037 ceph-mon[142325]: pgmap v1228: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:51:46.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:51:45 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T08:51:46.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:51:45 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T08:51:46.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:51:45 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T08:51:46.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:51:45 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:51:48.055 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:51:47 smithi148 ceph-mon[106692]: pgmap v1229: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:51:48.176 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:51:47 smithi037 ceph-mon[142325]: pgmap v1229: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:51:50.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:51:49 smithi148 ceph-mon[106692]: pgmap v1230: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:51:50.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:51:49 smithi037 ceph-mon[142325]: pgmap v1230: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:51:52.055 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:51:51 smithi148 ceph-mon[106692]: pgmap v1231: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:51:52.176 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:51:51 smithi037 ceph-mon[142325]: pgmap v1231: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:51:54.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:51:53 smithi148 ceph-mon[106692]: pgmap v1232: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:51:54.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:51:53 smithi037 ceph-mon[142325]: pgmap v1232: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:51:55.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:51:54 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:51:55.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:51:54 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:51:56.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:51:55 smithi148 ceph-mon[106692]: pgmap v1233: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:51:56.176 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:51:55 smithi037 ceph-mon[142325]: pgmap v1233: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:51:58.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:51:57 smithi148 ceph-mon[106692]: pgmap v1234: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:51:58.176 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:51:57 smithi037 ceph-mon[142325]: pgmap v1234: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:52:00.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:51:59 smithi148 ceph-mon[106692]: pgmap v1235: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:52:00.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:51:59 smithi037 ceph-mon[142325]: pgmap v1235: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:52:02.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:52:01 smithi148 ceph-mon[106692]: pgmap v1236: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:52:02.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:52:01 smithi037 ceph-mon[142325]: pgmap v1236: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:52:04.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:52:03 smithi148 ceph-mon[106692]: pgmap v1237: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:52:04.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:52:03 smithi037 ceph-mon[142325]: pgmap v1237: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:52:06.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:52:05 smithi148 ceph-mon[106692]: pgmap v1238: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:52:06.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:52:05 smithi037 ceph-mon[142325]: pgmap v1238: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:52:08.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:52:07 smithi148 ceph-mon[106692]: pgmap v1239: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:52:08.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:52:07 smithi037 ceph-mon[142325]: pgmap v1239: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:52:10.055 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:52:09 smithi148 ceph-mon[106692]: pgmap v1240: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:52:10.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:52:09 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:52:10.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:52:09 smithi037 ceph-mon[142325]: pgmap v1240: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:52:10.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:52:09 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:52:12.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:52:11 smithi148 ceph-mon[106692]: pgmap v1241: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:52:12.176 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:52:11 smithi037 ceph-mon[142325]: pgmap v1241: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:52:14.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:52:13 smithi148 ceph-mon[106692]: pgmap v1242: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:52:14.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:52:13 smithi037 ceph-mon[142325]: pgmap v1242: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:52:16.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:52:15 smithi148 ceph-mon[106692]: pgmap v1243: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:52:16.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:52:15 smithi037 ceph-mon[142325]: pgmap v1243: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:52:18.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:52:17 smithi037 ceph-mon[142325]: pgmap v1244: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:52:18.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:52:17 smithi148 ceph-mon[106692]: pgmap v1244: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:52:20.176 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:52:19 smithi037 ceph-mon[142325]: pgmap v1245: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:52:20.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:52:19 smithi148 ceph-mon[106692]: pgmap v1245: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:52:22.176 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:52:21 smithi037 ceph-mon[142325]: pgmap v1246: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:52:22.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:52:21 smithi148 ceph-mon[106692]: pgmap v1246: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:52:24.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:52:23 smithi037 ceph-mon[142325]: pgmap v1247: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:52:24.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:52:23 smithi148 ceph-mon[106692]: pgmap v1247: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:52:25.176 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:52:24 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:52:25.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:52:24 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:52:26.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:52:25 smithi037 ceph-mon[142325]: pgmap v1248: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:52:26.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:52:25 smithi148 ceph-mon[106692]: pgmap v1248: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:52:28.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:52:27 smithi037 ceph-mon[142325]: pgmap v1249: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:52:28.305 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:52:27 smithi148 ceph-mon[106692]: pgmap v1249: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:52:30.176 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:52:29 smithi037 ceph-mon[142325]: pgmap v1250: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:52:30.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:52:29 smithi148 ceph-mon[106692]: pgmap v1250: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:52:32.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:52:31 smithi037 ceph-mon[142325]: pgmap v1251: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:52:32.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:52:31 smithi148 ceph-mon[106692]: pgmap v1251: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:52:34.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:52:33 smithi037 ceph-mon[142325]: pgmap v1252: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:52:34.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:52:33 smithi148 ceph-mon[106692]: pgmap v1252: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:52:36.176 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:52:35 smithi037 ceph-mon[142325]: pgmap v1253: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:52:36.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:52:35 smithi148 ceph-mon[106692]: pgmap v1253: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:52:38.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:52:37 smithi037 ceph-mon[142325]: pgmap v1254: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:52:38.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:52:37 smithi148 ceph-mon[106692]: pgmap v1254: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:52:40.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:52:39 smithi037 ceph-mon[142325]: pgmap v1255: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:52:40.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:52:39 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:52:40.305 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:52:39 smithi148 ceph-mon[106692]: pgmap v1255: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:52:40.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:52:39 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:52:42.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:52:41 smithi037 ceph-mon[142325]: pgmap v1256: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:52:42.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:52:41 smithi148 ceph-mon[106692]: pgmap v1256: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:52:44.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:52:43 smithi037 ceph-mon[142325]: pgmap v1257: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:52:44.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:52:43 smithi148 ceph-mon[106692]: pgmap v1257: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:52:46.176 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:52:45 smithi037 ceph-mon[142325]: pgmap v1258: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:52:46.305 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:52:45 smithi148 ceph-mon[106692]: pgmap v1258: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:52:47.176 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:52:46 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T08:52:47.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:52:46 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T08:52:47.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:52:46 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T08:52:47.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:52:46 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:52:47.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:52:46 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T08:52:47.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:52:46 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T08:52:47.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:52:46 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T08:52:47.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:52:46 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:52:48.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:52:47 smithi037 ceph-mon[142325]: pgmap v1259: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:52:48.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:52:47 smithi148 ceph-mon[106692]: pgmap v1259: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:52:50.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:52:49 smithi037 ceph-mon[142325]: pgmap v1260: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:52:50.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:52:49 smithi148 ceph-mon[106692]: pgmap v1260: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:52:52.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:52:51 smithi037 ceph-mon[142325]: pgmap v1261: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:52:52.305 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:52:51 smithi148 ceph-mon[106692]: pgmap v1261: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:52:54.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:52:53 smithi037 ceph-mon[142325]: pgmap v1262: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:52:54.305 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:52:53 smithi148 ceph-mon[106692]: pgmap v1262: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:52:55.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:52:54 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:52:55.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:52:54 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:52:56.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:52:55 smithi037 ceph-mon[142325]: pgmap v1263: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:52:56.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:52:55 smithi148 ceph-mon[106692]: pgmap v1263: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:52:58.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:52:57 smithi037 ceph-mon[142325]: pgmap v1264: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:52:58.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:52:57 smithi148 ceph-mon[106692]: pgmap v1264: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:53:00.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:52:59 smithi037 ceph-mon[142325]: pgmap v1265: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:53:00.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:52:59 smithi148 ceph-mon[106692]: pgmap v1265: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:53:02.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:53:01 smithi037 ceph-mon[142325]: pgmap v1266: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:53:02.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:53:01 smithi148 ceph-mon[106692]: pgmap v1266: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:53:04.176 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:53:03 smithi037 ceph-mon[142325]: pgmap v1267: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:53:04.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:53:03 smithi148 ceph-mon[106692]: pgmap v1267: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:53:06.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:53:05 smithi037 ceph-mon[142325]: pgmap v1268: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:53:06.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:53:05 smithi148 ceph-mon[106692]: pgmap v1268: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:53:08.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:53:07 smithi037 ceph-mon[142325]: pgmap v1269: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:53:08.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:53:07 smithi148 ceph-mon[106692]: pgmap v1269: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:53:10.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:53:09 smithi037 ceph-mon[142325]: pgmap v1270: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:53:10.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:53:09 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:53:10.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:53:09 smithi148 ceph-mon[106692]: pgmap v1270: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:53:10.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:53:09 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:53:12.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:53:11 smithi148 ceph-mon[106692]: pgmap v1271: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:53:12.426 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:53:11 smithi037 ceph-mon[142325]: pgmap v1271: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:53:14.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:53:13 smithi148 ceph-mon[106692]: pgmap v1272: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:53:14.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:53:13 smithi037 ceph-mon[142325]: pgmap v1272: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:53:16.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:53:15 smithi148 ceph-mon[106692]: pgmap v1273: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:53:16.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:53:15 smithi037 ceph-mon[142325]: pgmap v1273: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:53:18.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:53:17 smithi148 ceph-mon[106692]: pgmap v1274: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:53:18.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:53:17 smithi037 ceph-mon[142325]: pgmap v1274: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:53:20.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:53:19 smithi148 ceph-mon[106692]: pgmap v1275: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:53:20.426 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:53:19 smithi037 ceph-mon[142325]: pgmap v1275: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:53:22.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:53:21 smithi148 ceph-mon[106692]: pgmap v1276: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:53:22.426 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:53:21 smithi037 ceph-mon[142325]: pgmap v1276: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:53:24.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:53:23 smithi148 ceph-mon[106692]: pgmap v1277: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:53:24.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:53:23 smithi037 ceph-mon[142325]: pgmap v1277: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:53:25.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:53:24 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:53:25.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:53:24 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:53:26.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:53:25 smithi148 ceph-mon[106692]: pgmap v1278: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:53:26.426 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:53:25 smithi037 ceph-mon[142325]: pgmap v1278: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:53:28.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:53:27 smithi148 ceph-mon[106692]: pgmap v1279: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:53:28.426 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:53:27 smithi037 ceph-mon[142325]: pgmap v1279: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:53:30.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:53:29 smithi148 ceph-mon[106692]: pgmap v1280: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:53:30.426 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:53:29 smithi037 ceph-mon[142325]: pgmap v1280: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:53:32.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:53:31 smithi148 ceph-mon[106692]: pgmap v1281: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:53:32.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:53:31 smithi037 ceph-mon[142325]: pgmap v1281: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:53:34.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:53:33 smithi148 ceph-mon[106692]: pgmap v1282: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:53:34.426 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:53:33 smithi037 ceph-mon[142325]: pgmap v1282: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:53:36.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:53:35 smithi148 ceph-mon[106692]: pgmap v1283: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:53:36.426 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:53:35 smithi037 ceph-mon[142325]: pgmap v1283: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:53:38.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:53:37 smithi148 ceph-mon[106692]: pgmap v1284: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:53:38.426 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:53:37 smithi037 ceph-mon[142325]: pgmap v1284: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:53:40.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:53:39 smithi148 ceph-mon[106692]: pgmap v1285: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:53:40.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:53:39 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:53:40.426 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:53:39 smithi037 ceph-mon[142325]: pgmap v1285: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:53:40.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:53:39 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:53:42.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:53:41 smithi148 ceph-mon[106692]: pgmap v1286: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:53:42.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:53:41 smithi037 ceph-mon[142325]: pgmap v1286: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:53:44.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:53:44 smithi148 ceph-mon[106692]: pgmap v1287: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:53:44.426 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:53:44 smithi037 ceph-mon[142325]: pgmap v1287: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:53:46.305 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:53:46 smithi148 ceph-mon[106692]: pgmap v1288: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:53:46.426 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:53:46 smithi037 ceph-mon[142325]: pgmap v1288: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:53:47.305 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:53:47 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T08:53:47.426 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:53:47 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T08:53:48.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:53:48 smithi148 ceph-mon[106692]: pgmap v1289: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:53:48.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:53:48 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T08:53:48.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:53:48 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T08:53:48.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:53:48 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:53:48.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:53:48 smithi037 ceph-mon[142325]: pgmap v1289: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:53:48.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:53:48 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T08:53:48.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:53:48 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T08:53:48.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:53:48 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:53:50.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:53:50 smithi148 ceph-mon[106692]: pgmap v1290: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:53:50.426 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:53:50 smithi037 ceph-mon[142325]: pgmap v1290: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:53:52.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:53:52 smithi148 ceph-mon[106692]: pgmap v1291: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:53:52.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:53:52 smithi037 ceph-mon[142325]: pgmap v1291: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:53:54.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:53:54 smithi148 ceph-mon[106692]: pgmap v1292: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:53:54.426 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:53:54 smithi037 ceph-mon[142325]: pgmap v1292: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:53:55.305 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:53:55 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:53:55.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:53:55 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:53:56.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:53:56 smithi148 ceph-mon[106692]: pgmap v1293: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:53:56.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:53:56 smithi037 ceph-mon[142325]: pgmap v1293: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:53:57.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:53:57 smithi037 ceph-mon[142325]: pgmap v1294: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:53:57.556 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:53:57 smithi148 ceph-mon[106692]: pgmap v1294: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:53:59.677 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:53:59 smithi037 ceph-mon[142325]: pgmap v1295: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:53:59.805 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:53:59 smithi148 ceph-mon[106692]: pgmap v1295: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:54:02.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:54:01 smithi148 ceph-mon[106692]: pgmap v1296: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:54:02.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:54:01 smithi037 ceph-mon[142325]: pgmap v1296: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:54:04.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:54:03 smithi148 ceph-mon[106692]: pgmap v1297: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:54:04.176 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:54:03 smithi037 ceph-mon[142325]: pgmap v1297: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:54:06.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:54:05 smithi148 ceph-mon[106692]: pgmap v1298: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:54:06.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:54:05 smithi037 ceph-mon[142325]: pgmap v1298: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:54:08.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:54:07 smithi148 ceph-mon[106692]: pgmap v1299: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:54:08.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:54:07 smithi037 ceph-mon[142325]: pgmap v1299: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:54:10.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:54:09 smithi148 ceph-mon[106692]: pgmap v1300: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:54:10.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:54:09 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:54:10.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:54:09 smithi037 ceph-mon[142325]: pgmap v1300: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:54:10.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:54:09 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:54:12.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:54:11 smithi148 ceph-mon[106692]: pgmap v1301: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:54:12.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:54:11 smithi037 ceph-mon[142325]: pgmap v1301: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:54:14.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:54:13 smithi148 ceph-mon[106692]: pgmap v1302: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:54:14.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:54:13 smithi037 ceph-mon[142325]: pgmap v1302: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:54:16.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:54:15 smithi148 ceph-mon[106692]: pgmap v1303: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:54:16.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:54:15 smithi037 ceph-mon[142325]: pgmap v1303: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:54:18.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:54:17 smithi148 ceph-mon[106692]: pgmap v1304: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:54:18.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:54:17 smithi037 ceph-mon[142325]: pgmap v1304: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:54:20.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:54:19 smithi148 ceph-mon[106692]: pgmap v1305: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:54:20.176 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:54:19 smithi037 ceph-mon[142325]: pgmap v1305: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:54:22.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:54:21 smithi148 ceph-mon[106692]: pgmap v1306: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:54:22.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:54:21 smithi037 ceph-mon[142325]: pgmap v1306: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:54:24.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:54:23 smithi148 ceph-mon[106692]: pgmap v1307: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:54:24.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:54:23 smithi037 ceph-mon[142325]: pgmap v1307: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:54:25.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:54:24 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:54:25.176 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:54:24 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:54:26.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:54:25 smithi148 ceph-mon[106692]: pgmap v1308: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:54:26.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:54:25 smithi037 ceph-mon[142325]: pgmap v1308: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:54:28.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:54:27 smithi148 ceph-mon[106692]: pgmap v1309: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:54:28.176 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:54:27 smithi037 ceph-mon[142325]: pgmap v1309: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:54:30.055 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:54:29 smithi148 ceph-mon[106692]: pgmap v1310: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:54:30.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:54:29 smithi037 ceph-mon[142325]: pgmap v1310: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:54:32.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:54:31 smithi148 ceph-mon[106692]: pgmap v1311: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:54:32.176 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:54:31 smithi037 ceph-mon[142325]: pgmap v1311: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:54:34.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:54:33 smithi148 ceph-mon[106692]: pgmap v1312: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:54:34.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:54:33 smithi037 ceph-mon[142325]: pgmap v1312: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:54:36.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:54:35 smithi148 ceph-mon[106692]: pgmap v1313: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:54:36.176 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:54:35 smithi037 ceph-mon[142325]: pgmap v1313: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:54:38.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:54:37 smithi148 ceph-mon[106692]: pgmap v1314: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:54:38.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:54:37 smithi037 ceph-mon[142325]: pgmap v1314: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:54:40.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:54:39 smithi148 ceph-mon[106692]: pgmap v1315: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:54:40.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:54:39 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:54:40.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:54:39 smithi037 ceph-mon[142325]: pgmap v1315: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:54:40.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:54:39 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:54:42.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:54:41 smithi148 ceph-mon[106692]: pgmap v1316: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:54:42.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:54:41 smithi037 ceph-mon[142325]: pgmap v1316: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:54:44.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:54:43 smithi148 ceph-mon[106692]: pgmap v1317: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:54:44.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:54:43 smithi037 ceph-mon[142325]: pgmap v1317: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:54:46.056 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:54:45 smithi148 ceph-mon[106692]: pgmap v1318: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:54:46.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:54:45 smithi037 ceph-mon[142325]: pgmap v1318: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:54:48.176 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:54:47 smithi037 ceph-mon[142325]: pgmap v1319: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:54:48.305 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:54:47 smithi148 ceph-mon[106692]: pgmap v1319: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:54:49.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:54:48 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T08:54:49.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:54:48 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T08:54:49.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:54:48 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T08:54:49.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:54:48 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:54:49.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:54:48 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-29T08:54:49.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:54:48 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-29T08:54:49.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:54:48 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-29T08:54:49.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:54:48 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' 2024-03-29T08:54:50.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:54:49 smithi037 ceph-mon[142325]: pgmap v1320: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:54:50.305 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:54:49 smithi148 ceph-mon[106692]: pgmap v1320: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:54:52.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:54:51 smithi037 ceph-mon[142325]: pgmap v1321: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:54:52.305 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:54:51 smithi148 ceph-mon[106692]: pgmap v1321: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:54:54.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:54:53 smithi037 ceph-mon[142325]: pgmap v1322: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:54:54.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:54:53 smithi148 ceph-mon[106692]: pgmap v1322: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:54:55.176 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:54:54 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:54:55.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:54:54 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:54:56.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:54:55 smithi037 ceph-mon[142325]: pgmap v1323: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:54:56.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:54:55 smithi148 ceph-mon[106692]: pgmap v1323: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:54:58.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:54:57 smithi037 ceph-mon[142325]: pgmap v1324: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:54:58.305 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:54:57 smithi148 ceph-mon[106692]: pgmap v1324: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:55:00.176 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:54:59 smithi037 ceph-mon[142325]: pgmap v1325: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:55:00.305 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:54:59 smithi148 ceph-mon[106692]: pgmap v1325: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:55:02.176 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:55:01 smithi037 ceph-mon[142325]: pgmap v1326: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:55:02.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:55:01 smithi148 ceph-mon[106692]: pgmap v1326: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:55:04.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:55:03 smithi037 ceph-mon[142325]: pgmap v1327: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:55:04.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:55:03 smithi148 ceph-mon[106692]: pgmap v1327: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:55:06.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:55:05 smithi037 ceph-mon[142325]: pgmap v1328: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:55:06.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:55:05 smithi148 ceph-mon[106692]: pgmap v1328: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:55:08.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:55:07 smithi037 ceph-mon[142325]: pgmap v1329: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:55:08.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:55:07 smithi148 ceph-mon[106692]: pgmap v1329: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:55:10.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:55:09 smithi037 ceph-mon[142325]: pgmap v1330: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:55:10.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:55:09 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:55:10.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:55:09 smithi148 ceph-mon[106692]: pgmap v1330: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:55:10.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:55:09 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:55:12.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:55:11 smithi037 ceph-mon[142325]: pgmap v1331: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:55:12.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:55:11 smithi148 ceph-mon[106692]: pgmap v1331: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:55:14.176 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:55:13 smithi037 ceph-mon[142325]: pgmap v1332: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:55:14.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:55:13 smithi148 ceph-mon[106692]: pgmap v1332: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:55:16.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:55:15 smithi037 ceph-mon[142325]: pgmap v1333: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:55:16.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:55:15 smithi148 ceph-mon[106692]: pgmap v1333: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:55:18.178 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:55:17 smithi037 ceph-mon[142325]: pgmap v1334: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:55:18.305 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:55:17 smithi148 ceph-mon[106692]: pgmap v1334: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:55:20.176 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:55:19 smithi037 ceph-mon[142325]: pgmap v1335: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:55:20.305 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:55:19 smithi148 ceph-mon[106692]: pgmap v1335: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:55:22.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:55:21 smithi037 ceph-mon[142325]: pgmap v1336: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:55:22.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:55:21 smithi148 ceph-mon[106692]: pgmap v1336: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:55:24.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:55:23 smithi037 ceph-mon[142325]: pgmap v1337: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:55:24.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:55:23 smithi148 ceph-mon[106692]: pgmap v1337: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:55:25.176 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:55:24 smithi037 ceph-mon[142325]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:55:25.305 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:55:24 smithi148 ceph-mon[106692]: from='mgr.34104 172.21.15.37:0/2413616988' entity='mgr.smithi037.wjikii' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-29T08:55:26.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:55:25 smithi037 ceph-mon[142325]: pgmap v1338: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:55:26.308 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:55:25 smithi148 ceph-mon[106692]: pgmap v1338: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:55:28.177 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:55:27 smithi037 ceph-mon[142325]: pgmap v1339: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:55:28.305 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:55:27 smithi148 ceph-mon[106692]: pgmap v1339: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:55:30.176 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:55:29 smithi037 ceph-mon[142325]: pgmap v1340: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:55:30.305 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:55:29 smithi148 ceph-mon[106692]: pgmap v1340: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:55:31.482 ERROR:tasks.cephfs.fuse_mount:process failed to terminate after unmount. This probably indicates a bug within ceph-fuse. 2024-03-29T08:55:31.482 ERROR:teuthology.run_tasks:Manager failed: ceph-fuse Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_d0e392d9aee635f2b65768c460c7a151ead7a5e2/qa/tasks/ceph_fuse.py", line 183, in task mount.umount_wait() File "/home/teuthworker/src/git.ceph.com_ceph-c_d0e392d9aee635f2b65768c460c7a151ead7a5e2/qa/tasks/cephfs/fuse_mount.py", line 400, in umount_wait run.wait([self.fuse_daemon], timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 473, in wait check_time() File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds 2024-03-29T08:55:31.484 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-03-29T08:55:31.505 INFO:tasks.cephadm:Teardown begin 2024-03-29T08:55:31.505 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_ceph-c_d0e392d9aee635f2b65768c460c7a151ead7a5e2/qa/tasks/cephadm.py", line 1750, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_d0e392d9aee635f2b65768c460c7a151ead7a5e2/qa/tasks/ceph_fuse.py", line 183, in task mount.umount_wait() File "/home/teuthworker/src/git.ceph.com_ceph-c_d0e392d9aee635f2b65768c460c7a151ead7a5e2/qa/tasks/cephfs/fuse_mount.py", line 400, in umount_wait run.wait([self.fuse_daemon], timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 473, in wait check_time() File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds 2024-03-29T08:55:31.506 DEBUG:teuthology.orchestra.run.smithi037:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-29T08:55:31.591 DEBUG:teuthology.orchestra.run.smithi148:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-29T08:55:31.672 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-03-29T08:55:31.672 DEBUG:teuthology.orchestra.run.smithi037:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-03-29T08:55:31.719 DEBUG:teuthology.orchestra.run.smithi148:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-03-29T08:55:31.766 INFO:tasks.cephadm:Stopping all daemons... 2024-03-29T08:55:31.766 INFO:tasks.cephadm.mon.smithi037:Stopping mon.smithi037... 2024-03-29T08:55:31.767 DEBUG:teuthology.orchestra.run.smithi037:> sudo systemctl stop ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4@mon.smithi037 2024-03-29T08:55:32.159 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:55:31 smithi037 systemd[1]: Stopping Ceph mon.smithi037 for 99b492c2-eda1-11ee-b647-cb9ed24678a4... 2024-03-29T08:55:32.160 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:55:31 smithi037 ceph-mon[142325]: pgmap v1341: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:55:32.306 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:55:31 smithi148 ceph-mon[106692]: pgmap v1341: 65 pgs: 65 active+clean; 259 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-03-29T08:55:32.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:55:32 smithi037 ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4-mon-smithi037[142302]: 2024-03-29T08:55:32.159+0000 7f2c64000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi037 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-29T08:55:32.427 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:55:32 smithi037 ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4-mon-smithi037[142302]: 2024-03-29T08:55:32.159+0000 7f2c64000700 -1 mon.smithi037@0(leader) e3 *** Got Signal Terminated *** 2024-03-29T08:55:32.927 INFO:journalctl@ceph.mon.smithi037.smithi037.stdout:Mar 29 08:55:32 smithi037 bash[208738]: ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4-mon-smithi037 2024-03-29T08:55:33.231 DEBUG:teuthology.orchestra.run.smithi037:> sudo pkill -f 'journalctl -f -n 0 -u ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4@mon.smithi037.service' 2024-03-29T08:55:33.350 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-29T08:55:33.350 INFO:tasks.cephadm.mon.smithi037:Stopped mon.smithi037 2024-03-29T08:55:33.350 INFO:tasks.cephadm.mon.smithi148:Stopping mon.smithi148... 2024-03-29T08:55:33.351 DEBUG:teuthology.orchestra.run.smithi148:> sudo systemctl stop ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4@mon.smithi148 2024-03-29T08:55:33.806 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:55:33 smithi148 systemd[1]: Stopping Ceph mon.smithi148 for 99b492c2-eda1-11ee-b647-cb9ed24678a4... 2024-03-29T08:55:34.242 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:55:33 smithi148 ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4-mon-smithi148[106669]: 2024-03-29T08:55:33.831+0000 7fea44600700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi148 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-29T08:55:34.242 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:55:33 smithi148 ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4-mon-smithi148[106669]: 2024-03-29T08:55:33.831+0000 7fea44600700 -1 mon.smithi148@1(peon) e3 *** Got Signal Terminated *** 2024-03-29T08:55:34.555 INFO:journalctl@ceph.mon.smithi148.smithi148.stdout:Mar 29 08:55:34 smithi148 bash[152750]: ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4-mon-smithi148 2024-03-29T08:55:34.874 DEBUG:teuthology.orchestra.run.smithi148:> sudo pkill -f 'journalctl -f -n 0 -u ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4@mon.smithi148.service' 2024-03-29T08:55:34.971 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-29T08:55:34.971 INFO:tasks.cephadm.mon.smithi148:Stopped mon.smithi148 2024-03-29T08:55:34.971 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-03-29T08:55:34.972 DEBUG:teuthology.orchestra.run.smithi037:> sudo systemctl stop ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4@osd.0 2024-03-29T08:55:35.397 INFO:journalctl@ceph.osd.0.smithi037.stdout:Mar 29 08:55:35 smithi037 systemd[1]: Stopping Ceph osd.0 for 99b492c2-eda1-11ee-b647-cb9ed24678a4... 2024-03-29T08:55:35.677 INFO:journalctl@ceph.osd.0.smithi037.stdout:Mar 29 08:55:35 smithi037 ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4-osd-0[157269]: 2024-03-29T08:55:35.396+0000 7feb98600700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-29T08:55:35.677 INFO:journalctl@ceph.osd.0.smithi037.stdout:Mar 29 08:55:35 smithi037 ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4-osd-0[157269]: 2024-03-29T08:55:35.396+0000 7feb98600700 -1 osd.0 86 *** Got signal Terminated *** 2024-03-29T08:55:35.677 INFO:journalctl@ceph.osd.0.smithi037.stdout:Mar 29 08:55:35 smithi037 ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4-osd-0[157269]: 2024-03-29T08:55:35.396+0000 7feb98600700 -1 osd.0 86 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-29T08:55:41.677 INFO:journalctl@ceph.osd.0.smithi037.stdout:Mar 29 08:55:41 smithi037 bash[208897]: ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4-osd-0 2024-03-29T08:55:43.585 DEBUG:teuthology.orchestra.run.smithi037:> sudo pkill -f 'journalctl -f -n 0 -u ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4@osd.0.service' 2024-03-29T08:55:43.674 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-29T08:55:43.674 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-03-29T08:55:43.675 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-03-29T08:55:43.675 DEBUG:teuthology.orchestra.run.smithi037:> sudo systemctl stop ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4@osd.1 2024-03-29T08:55:43.926 INFO:journalctl@ceph.osd.1.smithi037.stdout:Mar 29 08:55:43 smithi037 systemd[1]: Stopping Ceph osd.1 for 99b492c2-eda1-11ee-b647-cb9ed24678a4... 2024-03-29T08:55:44.427 INFO:journalctl@ceph.osd.1.smithi037.stdout:Mar 29 08:55:44 smithi037 ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4-osd-1[164576]: 2024-03-29T08:55:44.166+0000 7f1ace600700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-29T08:55:44.427 INFO:journalctl@ceph.osd.1.smithi037.stdout:Mar 29 08:55:44 smithi037 ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4-osd-1[164576]: 2024-03-29T08:55:44.166+0000 7f1ace600700 -1 osd.1 86 *** Got signal Terminated *** 2024-03-29T08:55:44.427 INFO:journalctl@ceph.osd.1.smithi037.stdout:Mar 29 08:55:44 smithi037 ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4-osd-1[164576]: 2024-03-29T08:55:44.166+0000 7f1ace600700 -1 osd.1 86 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-29T08:55:50.177 INFO:journalctl@ceph.osd.1.smithi037.stdout:Mar 29 08:55:50 smithi037 bash[209168]: ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4-osd-1 2024-03-29T08:55:52.326 DEBUG:teuthology.orchestra.run.smithi037:> sudo pkill -f 'journalctl -f -n 0 -u ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4@osd.1.service' 2024-03-29T08:55:52.418 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-29T08:55:52.418 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-03-29T08:55:52.418 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-03-29T08:55:52.418 DEBUG:teuthology.orchestra.run.smithi037:> sudo systemctl stop ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4@osd.2 2024-03-29T08:55:52.676 INFO:journalctl@ceph.osd.2.smithi037.stdout:Mar 29 08:55:52 smithi037 systemd[1]: Stopping Ceph osd.2 for 99b492c2-eda1-11ee-b647-cb9ed24678a4... 2024-03-29T08:55:53.149 INFO:journalctl@ceph.osd.2.smithi037.stdout:Mar 29 08:55:52 smithi037 ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4-osd-2[171147]: 2024-03-29T08:55:52.802+0000 7f3221a00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-29T08:55:53.149 INFO:journalctl@ceph.osd.2.smithi037.stdout:Mar 29 08:55:52 smithi037 ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4-osd-2[171147]: 2024-03-29T08:55:52.802+0000 7f3221a00700 -1 osd.2 86 *** Got signal Terminated *** 2024-03-29T08:55:53.149 INFO:journalctl@ceph.osd.2.smithi037.stdout:Mar 29 08:55:52 smithi037 ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4-osd-2[171147]: 2024-03-29T08:55:52.802+0000 7f3221a00700 -1 osd.2 86 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-29T08:55:59.176 INFO:journalctl@ceph.osd.2.smithi037.stdout:Mar 29 08:55:58 smithi037 bash[209431]: ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4-osd-2 2024-03-29T08:56:01.257 DEBUG:teuthology.orchestra.run.smithi037:> sudo pkill -f 'journalctl -f -n 0 -u ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4@osd.2.service' 2024-03-29T08:56:01.345 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-29T08:56:01.345 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-03-29T08:56:01.345 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-03-29T08:56:01.346 DEBUG:teuthology.orchestra.run.smithi148:> sudo systemctl stop ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4@osd.3 2024-03-29T08:56:01.800 INFO:journalctl@ceph.osd.3.smithi148.stdout:Mar 29 08:56:01 smithi148 systemd[1]: Stopping Ceph osd.3 for 99b492c2-eda1-11ee-b647-cb9ed24678a4... 2024-03-29T08:56:02.056 INFO:journalctl@ceph.osd.5.smithi148.stdout:Mar 29 08:56:01 smithi148 ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4-osd-5[131123]: 2024-03-29T08:56:01.811+0000 7f6806200700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-29T08:55:35.869119+0000 front 2024-03-29T08:55:35.869168+0000 (oldest deadline 2024-03-29T08:56:01.768930+0000) 2024-03-29T08:56:02.056 INFO:journalctl@ceph.osd.3.smithi148.stdout:Mar 29 08:56:01 smithi148 ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4-osd-3[118708]: 2024-03-29T08:56:01.799+0000 7f9ac6a00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-29T08:56:02.056 INFO:journalctl@ceph.osd.3.smithi148.stdout:Mar 29 08:56:01 smithi148 ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4-osd-3[118708]: 2024-03-29T08:56:01.799+0000 7f9ac6a00700 -1 osd.3 86 *** Got signal Terminated *** 2024-03-29T08:56:02.056 INFO:journalctl@ceph.osd.3.smithi148.stdout:Mar 29 08:56:01 smithi148 ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4-osd-3[118708]: 2024-03-29T08:56:01.799+0000 7f9ac6a00700 -1 osd.3 86 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-29T08:56:03.306 INFO:journalctl@ceph.osd.5.smithi148.stdout:Mar 29 08:56:02 smithi148 ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4-osd-5[131123]: 2024-03-29T08:56:02.813+0000 7f6806200700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-29T08:55:35.869119+0000 front 2024-03-29T08:55:35.869168+0000 (oldest deadline 2024-03-29T08:56:01.768930+0000) 2024-03-29T08:56:03.306 INFO:journalctl@ceph.osd.3.smithi148.stdout:Mar 29 08:56:02 smithi148 ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4-osd-3[118708]: 2024-03-29T08:56:02.968+0000 7f9abdc00700 -1 osd.3 86 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-29T08:55:39.452285+0000 front 2024-03-29T08:55:39.452349+0000 (oldest deadline 2024-03-29T08:56:02.352282+0000) 2024-03-29T08:56:03.806 INFO:journalctl@ceph.osd.4.smithi148.stdout:Mar 29 08:56:03 smithi148 ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4-osd-4[124929]: 2024-03-29T08:56:03.374+0000 7f2807200700 -1 osd.4 86 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-29T08:55:39.154360+0000 front 2024-03-29T08:55:39.154350+0000 (oldest deadline 2024-03-29T08:56:02.654097+0000) 2024-03-29T08:56:04.305 INFO:journalctl@ceph.osd.5.smithi148.stdout:Mar 29 08:56:03 smithi148 ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4-osd-5[131123]: 2024-03-29T08:56:03.856+0000 7f6806200700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-29T08:55:35.869119+0000 front 2024-03-29T08:55:35.869168+0000 (oldest deadline 2024-03-29T08:56:01.768930+0000) 2024-03-29T08:56:04.306 INFO:journalctl@ceph.osd.3.smithi148.stdout:Mar 29 08:56:03 smithi148 ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4-osd-3[118708]: 2024-03-29T08:56:03.962+0000 7f9abdc00700 -1 osd.3 86 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-29T08:55:39.452285+0000 front 2024-03-29T08:55:39.452349+0000 (oldest deadline 2024-03-29T08:56:02.352282+0000) 2024-03-29T08:56:04.806 INFO:journalctl@ceph.osd.4.smithi148.stdout:Mar 29 08:56:04 smithi148 ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4-osd-4[124929]: 2024-03-29T08:56:04.389+0000 7f2807200700 -1 osd.4 86 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-29T08:55:39.154360+0000 front 2024-03-29T08:55:39.154350+0000 (oldest deadline 2024-03-29T08:56:02.654097+0000) 2024-03-29T08:56:05.305 INFO:journalctl@ceph.osd.5.smithi148.stdout:Mar 29 08:56:04 smithi148 ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4-osd-5[131123]: 2024-03-29T08:56:04.878+0000 7f6806200700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-29T08:55:35.869119+0000 front 2024-03-29T08:55:35.869168+0000 (oldest deadline 2024-03-29T08:56:01.768930+0000) 2024-03-29T08:56:05.306 INFO:journalctl@ceph.osd.3.smithi148.stdout:Mar 29 08:56:04 smithi148 ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4-osd-3[118708]: 2024-03-29T08:56:04.916+0000 7f9abdc00700 -1 osd.3 86 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-29T08:55:39.452285+0000 front 2024-03-29T08:55:39.452349+0000 (oldest deadline 2024-03-29T08:56:02.352282+0000) 2024-03-29T08:56:05.805 INFO:journalctl@ceph.osd.4.smithi148.stdout:Mar 29 08:56:05 smithi148 ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4-osd-4[124929]: 2024-03-29T08:56:05.427+0000 7f2807200700 -1 osd.4 86 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-29T08:55:39.154360+0000 front 2024-03-29T08:55:39.154350+0000 (oldest deadline 2024-03-29T08:56:02.654097+0000) 2024-03-29T08:56:06.306 INFO:journalctl@ceph.osd.5.smithi148.stdout:Mar 29 08:56:05 smithi148 ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4-osd-5[131123]: 2024-03-29T08:56:05.858+0000 7f6806200700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-29T08:55:35.869119+0000 front 2024-03-29T08:55:35.869168+0000 (oldest deadline 2024-03-29T08:56:01.768930+0000) 2024-03-29T08:56:06.306 INFO:journalctl@ceph.osd.3.smithi148.stdout:Mar 29 08:56:05 smithi148 ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4-osd-3[118708]: 2024-03-29T08:56:05.890+0000 7f9abdc00700 -1 osd.3 86 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-29T08:55:39.452285+0000 front 2024-03-29T08:55:39.452349+0000 (oldest deadline 2024-03-29T08:56:02.352282+0000) 2024-03-29T08:56:06.806 INFO:journalctl@ceph.osd.4.smithi148.stdout:Mar 29 08:56:06 smithi148 ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4-osd-4[124929]: 2024-03-29T08:56:06.438+0000 7f2807200700 -1 osd.4 86 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-29T08:55:39.154360+0000 front 2024-03-29T08:55:39.154350+0000 (oldest deadline 2024-03-29T08:56:02.654097+0000) 2024-03-29T08:56:07.307 INFO:journalctl@ceph.osd.5.smithi148.stdout:Mar 29 08:56:06 smithi148 ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4-osd-5[131123]: 2024-03-29T08:56:06.816+0000 7f6806200700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-29T08:55:35.869119+0000 front 2024-03-29T08:55:35.869168+0000 (oldest deadline 2024-03-29T08:56:01.768930+0000) 2024-03-29T08:56:07.663 INFO:journalctl@ceph.osd.3.smithi148.stdout:Mar 29 08:56:07 smithi148 bash[152896]: ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4-osd-3 2024-03-29T08:56:07.663 INFO:journalctl@ceph.osd.4.smithi148.stdout:Mar 29 08:56:07 smithi148 ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4-osd-4[124929]: 2024-03-29T08:56:07.413+0000 7f2807200700 -1 osd.4 86 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-29T08:55:39.154360+0000 front 2024-03-29T08:55:39.154350+0000 (oldest deadline 2024-03-29T08:56:02.654097+0000) 2024-03-29T08:56:08.306 INFO:journalctl@ceph.osd.5.smithi148.stdout:Mar 29 08:56:07 smithi148 ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4-osd-5[131123]: 2024-03-29T08:56:07.834+0000 7f6806200700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-29T08:55:35.869119+0000 front 2024-03-29T08:55:35.869168+0000 (oldest deadline 2024-03-29T08:56:01.768930+0000) 2024-03-29T08:56:08.806 INFO:journalctl@ceph.osd.4.smithi148.stdout:Mar 29 08:56:08 smithi148 ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4-osd-4[124929]: 2024-03-29T08:56:08.363+0000 7f2807200700 -1 osd.4 86 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-29T08:55:39.154360+0000 front 2024-03-29T08:55:39.154350+0000 (oldest deadline 2024-03-29T08:56:02.654097+0000) 2024-03-29T08:56:09.072 INFO:journalctl@ceph.osd.5.smithi148.stdout:Mar 29 08:56:08 smithi148 ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4-osd-5[131123]: 2024-03-29T08:56:08.819+0000 7f6806200700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-29T08:55:35.869119+0000 front 2024-03-29T08:55:35.869168+0000 (oldest deadline 2024-03-29T08:56:01.768930+0000) 2024-03-29T08:56:09.771 INFO:journalctl@ceph.osd.4.smithi148.stdout:Mar 29 08:56:09 smithi148 ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4-osd-4[124929]: 2024-03-29T08:56:09.404+0000 7f2807200700 -1 osd.4 86 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-29T08:55:39.154360+0000 front 2024-03-29T08:55:39.154350+0000 (oldest deadline 2024-03-29T08:56:02.654097+0000) 2024-03-29T08:56:09.798 DEBUG:teuthology.orchestra.run.smithi148:> sudo pkill -f 'journalctl -f -n 0 -u ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4@osd.3.service' 2024-03-29T08:56:09.885 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-29T08:56:09.885 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-03-29T08:56:09.885 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-03-29T08:56:09.885 DEBUG:teuthology.orchestra.run.smithi148:> sudo systemctl stop ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4@osd.4 2024-03-29T08:56:10.056 INFO:journalctl@ceph.osd.4.smithi148.stdout:Mar 29 08:56:09 smithi148 systemd[1]: Stopping Ceph osd.4 for 99b492c2-eda1-11ee-b647-cb9ed24678a4... 2024-03-29T08:56:10.056 INFO:journalctl@ceph.osd.5.smithi148.stdout:Mar 29 08:56:09 smithi148 ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4-osd-5[131123]: 2024-03-29T08:56:09.835+0000 7f6806200700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-29T08:55:35.869119+0000 front 2024-03-29T08:55:35.869168+0000 (oldest deadline 2024-03-29T08:56:01.768930+0000) 2024-03-29T08:56:10.556 INFO:journalctl@ceph.osd.4.smithi148.stdout:Mar 29 08:56:10 smithi148 ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4-osd-4[124929]: 2024-03-29T08:56:10.277+0000 7f2810000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-29T08:56:10.556 INFO:journalctl@ceph.osd.4.smithi148.stdout:Mar 29 08:56:10 smithi148 ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4-osd-4[124929]: 2024-03-29T08:56:10.277+0000 7f2810000700 -1 osd.4 86 *** Got signal Terminated *** 2024-03-29T08:56:10.556 INFO:journalctl@ceph.osd.4.smithi148.stdout:Mar 29 08:56:10 smithi148 ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4-osd-4[124929]: 2024-03-29T08:56:10.277+0000 7f2810000700 -1 osd.4 86 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-29T08:56:10.556 INFO:journalctl@ceph.osd.4.smithi148.stdout:Mar 29 08:56:10 smithi148 ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4-osd-4[124929]: 2024-03-29T08:56:10.366+0000 7f2807200700 -1 osd.4 86 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-29T08:55:39.154360+0000 front 2024-03-29T08:55:39.154350+0000 (oldest deadline 2024-03-29T08:56:02.654097+0000) 2024-03-29T08:56:11.305 INFO:journalctl@ceph.osd.5.smithi148.stdout:Mar 29 08:56:10 smithi148 ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4-osd-5[131123]: 2024-03-29T08:56:10.856+0000 7f6806200700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-29T08:55:35.869119+0000 front 2024-03-29T08:55:35.869168+0000 (oldest deadline 2024-03-29T08:56:01.768930+0000) 2024-03-29T08:56:11.806 INFO:journalctl@ceph.osd.4.smithi148.stdout:Mar 29 08:56:11 smithi148 ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4-osd-4[124929]: 2024-03-29T08:56:11.330+0000 7f2807200700 -1 osd.4 86 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-29T08:55:39.154360+0000 front 2024-03-29T08:55:39.154350+0000 (oldest deadline 2024-03-29T08:56:02.654097+0000) 2024-03-29T08:56:12.306 INFO:journalctl@ceph.osd.5.smithi148.stdout:Mar 29 08:56:11 smithi148 ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4-osd-5[131123]: 2024-03-29T08:56:11.831+0000 7f6806200700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-29T08:55:35.869119+0000 front 2024-03-29T08:55:35.869168+0000 (oldest deadline 2024-03-29T08:56:01.768930+0000) 2024-03-29T08:56:12.306 INFO:journalctl@ceph.osd.5.smithi148.stdout:Mar 29 08:56:11 smithi148 ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4-osd-5[131123]: 2024-03-29T08:56:11.831+0000 7f6806200700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.37:6814 osd.1 since back 2024-03-29T08:55:45.869882+0000 front 2024-03-29T08:55:45.869913+0000 (oldest deadline 2024-03-29T08:56:11.769789+0000) 2024-03-29T08:56:12.805 INFO:journalctl@ceph.osd.4.smithi148.stdout:Mar 29 08:56:12 smithi148 ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4-osd-4[124929]: 2024-03-29T08:56:12.347+0000 7f2807200700 -1 osd.4 86 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-29T08:55:39.154360+0000 front 2024-03-29T08:55:39.154350+0000 (oldest deadline 2024-03-29T08:56:02.654097+0000) 2024-03-29T08:56:12.806 INFO:journalctl@ceph.osd.4.smithi148.stdout:Mar 29 08:56:12 smithi148 ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4-osd-4[124929]: 2024-03-29T08:56:12.347+0000 7f2807200700 -1 osd.4 86 heartbeat_check: no reply from 172.21.15.37:6814 osd.1 since back 2024-03-29T08:55:49.055560+0000 front 2024-03-29T08:55:49.055545+0000 (oldest deadline 2024-03-29T08:56:11.955396+0000) 2024-03-29T08:56:13.301 INFO:journalctl@ceph.osd.5.smithi148.stdout:Mar 29 08:56:12 smithi148 ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4-osd-5[131123]: 2024-03-29T08:56:12.852+0000 7f6806200700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-29T08:55:35.869119+0000 front 2024-03-29T08:55:35.869168+0000 (oldest deadline 2024-03-29T08:56:01.768930+0000) 2024-03-29T08:56:13.302 INFO:journalctl@ceph.osd.5.smithi148.stdout:Mar 29 08:56:12 smithi148 ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4-osd-5[131123]: 2024-03-29T08:56:12.852+0000 7f6806200700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.37:6814 osd.1 since back 2024-03-29T08:55:45.869882+0000 front 2024-03-29T08:55:45.869913+0000 (oldest deadline 2024-03-29T08:56:11.769789+0000) 2024-03-29T08:56:13.555 INFO:journalctl@ceph.osd.4.smithi148.stdout:Mar 29 08:56:13 smithi148 ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4-osd-4[124929]: 2024-03-29T08:56:13.302+0000 7f2807200700 -1 osd.4 86 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-29T08:55:39.154360+0000 front 2024-03-29T08:55:39.154350+0000 (oldest deadline 2024-03-29T08:56:02.654097+0000) 2024-03-29T08:56:13.556 INFO:journalctl@ceph.osd.4.smithi148.stdout:Mar 29 08:56:13 smithi148 ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4-osd-4[124929]: 2024-03-29T08:56:13.302+0000 7f2807200700 -1 osd.4 86 heartbeat_check: no reply from 172.21.15.37:6814 osd.1 since back 2024-03-29T08:55:49.055560+0000 front 2024-03-29T08:55:49.055545+0000 (oldest deadline 2024-03-29T08:56:11.955396+0000) 2024-03-29T08:56:14.288 INFO:journalctl@ceph.osd.5.smithi148.stdout:Mar 29 08:56:13 smithi148 ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4-osd-5[131123]: 2024-03-29T08:56:13.862+0000 7f6806200700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-29T08:55:35.869119+0000 front 2024-03-29T08:55:35.869168+0000 (oldest deadline 2024-03-29T08:56:01.768930+0000) 2024-03-29T08:56:14.288 INFO:journalctl@ceph.osd.5.smithi148.stdout:Mar 29 08:56:13 smithi148 ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4-osd-5[131123]: 2024-03-29T08:56:13.862+0000 7f6806200700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.37:6814 osd.1 since back 2024-03-29T08:55:45.869882+0000 front 2024-03-29T08:55:45.869913+0000 (oldest deadline 2024-03-29T08:56:11.769789+0000) 2024-03-29T08:56:14.555 INFO:journalctl@ceph.osd.4.smithi148.stdout:Mar 29 08:56:14 smithi148 ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4-osd-4[124929]: 2024-03-29T08:56:14.288+0000 7f2807200700 -1 osd.4 86 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-29T08:55:39.154360+0000 front 2024-03-29T08:55:39.154350+0000 (oldest deadline 2024-03-29T08:56:02.654097+0000) 2024-03-29T08:56:14.556 INFO:journalctl@ceph.osd.4.smithi148.stdout:Mar 29 08:56:14 smithi148 ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4-osd-4[124929]: 2024-03-29T08:56:14.288+0000 7f2807200700 -1 osd.4 86 heartbeat_check: no reply from 172.21.15.37:6814 osd.1 since back 2024-03-29T08:55:49.055560+0000 front 2024-03-29T08:55:49.055545+0000 (oldest deadline 2024-03-29T08:56:11.955396+0000) 2024-03-29T08:56:15.306 INFO:journalctl@ceph.osd.5.smithi148.stdout:Mar 29 08:56:14 smithi148 ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4-osd-5[131123]: 2024-03-29T08:56:14.860+0000 7f6806200700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-29T08:55:35.869119+0000 front 2024-03-29T08:55:35.869168+0000 (oldest deadline 2024-03-29T08:56:01.768930+0000) 2024-03-29T08:56:15.306 INFO:journalctl@ceph.osd.5.smithi148.stdout:Mar 29 08:56:14 smithi148 ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4-osd-5[131123]: 2024-03-29T08:56:14.860+0000 7f6806200700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.37:6814 osd.1 since back 2024-03-29T08:55:45.869882+0000 front 2024-03-29T08:55:45.869913+0000 (oldest deadline 2024-03-29T08:56:11.769789+0000) 2024-03-29T08:56:16.104 INFO:journalctl@ceph.osd.5.smithi148.stdout:Mar 29 08:56:15 smithi148 ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4-osd-5[131123]: 2024-03-29T08:56:15.852+0000 7f6806200700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-29T08:55:35.869119+0000 front 2024-03-29T08:55:35.869168+0000 (oldest deadline 2024-03-29T08:56:01.768930+0000) 2024-03-29T08:56:16.104 INFO:journalctl@ceph.osd.5.smithi148.stdout:Mar 29 08:56:15 smithi148 ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4-osd-5[131123]: 2024-03-29T08:56:15.852+0000 7f6806200700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.37:6814 osd.1 since back 2024-03-29T08:55:45.869882+0000 front 2024-03-29T08:55:45.869913+0000 (oldest deadline 2024-03-29T08:56:11.769789+0000) 2024-03-29T08:56:16.555 INFO:journalctl@ceph.osd.4.smithi148.stdout:Mar 29 08:56:16 smithi148 bash[153160]: ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4-osd-4 2024-03-29T08:56:17.306 INFO:journalctl@ceph.osd.5.smithi148.stdout:Mar 29 08:56:16 smithi148 ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4-osd-5[131123]: 2024-03-29T08:56:16.822+0000 7f6806200700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-29T08:55:35.869119+0000 front 2024-03-29T08:55:35.869168+0000 (oldest deadline 2024-03-29T08:56:01.768930+0000) 2024-03-29T08:56:17.306 INFO:journalctl@ceph.osd.5.smithi148.stdout:Mar 29 08:56:16 smithi148 ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4-osd-5[131123]: 2024-03-29T08:56:16.822+0000 7f6806200700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.37:6814 osd.1 since back 2024-03-29T08:55:45.869882+0000 front 2024-03-29T08:55:45.869913+0000 (oldest deadline 2024-03-29T08:56:11.769789+0000) 2024-03-29T08:56:18.048 INFO:journalctl@ceph.osd.5.smithi148.stdout:Mar 29 08:56:17 smithi148 ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4-osd-5[131123]: 2024-03-29T08:56:17.868+0000 7f6806200700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-29T08:55:35.869119+0000 front 2024-03-29T08:55:35.869168+0000 (oldest deadline 2024-03-29T08:56:01.768930+0000) 2024-03-29T08:56:18.048 INFO:journalctl@ceph.osd.5.smithi148.stdout:Mar 29 08:56:17 smithi148 ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4-osd-5[131123]: 2024-03-29T08:56:17.868+0000 7f6806200700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.37:6814 osd.1 since back 2024-03-29T08:55:45.869882+0000 front 2024-03-29T08:55:45.869913+0000 (oldest deadline 2024-03-29T08:56:11.769789+0000) 2024-03-29T08:56:18.071 DEBUG:teuthology.orchestra.run.smithi148:> sudo pkill -f 'journalctl -f -n 0 -u ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4@osd.4.service' 2024-03-29T08:56:18.158 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-29T08:56:18.158 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-03-29T08:56:18.158 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-03-29T08:56:18.159 DEBUG:teuthology.orchestra.run.smithi148:> sudo systemctl stop ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4@osd.5 2024-03-29T08:56:18.529 INFO:journalctl@ceph.osd.5.smithi148.stdout:Mar 29 08:56:18 smithi148 systemd[1]: Stopping Ceph osd.5 for 99b492c2-eda1-11ee-b647-cb9ed24678a4... 2024-03-29T08:56:18.805 INFO:journalctl@ceph.osd.5.smithi148.stdout:Mar 29 08:56:18 smithi148 ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4-osd-5[131123]: 2024-03-29T08:56:18.529+0000 7f680a000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-29T08:56:18.806 INFO:journalctl@ceph.osd.5.smithi148.stdout:Mar 29 08:56:18 smithi148 ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4-osd-5[131123]: 2024-03-29T08:56:18.529+0000 7f680a000700 -1 osd.5 86 *** Got signal Terminated *** 2024-03-29T08:56:18.806 INFO:journalctl@ceph.osd.5.smithi148.stdout:Mar 29 08:56:18 smithi148 ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4-osd-5[131123]: 2024-03-29T08:56:18.529+0000 7f680a000700 -1 osd.5 86 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-29T08:56:19.306 INFO:journalctl@ceph.osd.5.smithi148.stdout:Mar 29 08:56:18 smithi148 ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4-osd-5[131123]: 2024-03-29T08:56:18.883+0000 7f6806200700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-29T08:55:35.869119+0000 front 2024-03-29T08:55:35.869168+0000 (oldest deadline 2024-03-29T08:56:01.768930+0000) 2024-03-29T08:56:19.306 INFO:journalctl@ceph.osd.5.smithi148.stdout:Mar 29 08:56:18 smithi148 ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4-osd-5[131123]: 2024-03-29T08:56:18.883+0000 7f6806200700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.37:6814 osd.1 since back 2024-03-29T08:55:45.869882+0000 front 2024-03-29T08:55:45.869913+0000 (oldest deadline 2024-03-29T08:56:11.769789+0000) 2024-03-29T08:56:20.305 INFO:journalctl@ceph.osd.5.smithi148.stdout:Mar 29 08:56:19 smithi148 ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4-osd-5[131123]: 2024-03-29T08:56:19.908+0000 7f6806200700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-29T08:55:35.869119+0000 front 2024-03-29T08:55:35.869168+0000 (oldest deadline 2024-03-29T08:56:01.768930+0000) 2024-03-29T08:56:20.306 INFO:journalctl@ceph.osd.5.smithi148.stdout:Mar 29 08:56:19 smithi148 ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4-osd-5[131123]: 2024-03-29T08:56:19.908+0000 7f6806200700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.37:6814 osd.1 since back 2024-03-29T08:55:45.869882+0000 front 2024-03-29T08:55:45.869913+0000 (oldest deadline 2024-03-29T08:56:11.769789+0000) 2024-03-29T08:56:21.306 INFO:journalctl@ceph.osd.5.smithi148.stdout:Mar 29 08:56:20 smithi148 ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4-osd-5[131123]: 2024-03-29T08:56:20.939+0000 7f6806200700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-29T08:55:35.869119+0000 front 2024-03-29T08:55:35.869168+0000 (oldest deadline 2024-03-29T08:56:01.768930+0000) 2024-03-29T08:56:21.306 INFO:journalctl@ceph.osd.5.smithi148.stdout:Mar 29 08:56:20 smithi148 ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4-osd-5[131123]: 2024-03-29T08:56:20.939+0000 7f6806200700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.37:6814 osd.1 since back 2024-03-29T08:55:45.869882+0000 front 2024-03-29T08:55:45.869913+0000 (oldest deadline 2024-03-29T08:56:11.769789+0000) 2024-03-29T08:56:22.305 INFO:journalctl@ceph.osd.5.smithi148.stdout:Mar 29 08:56:21 smithi148 ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4-osd-5[131123]: 2024-03-29T08:56:21.950+0000 7f6806200700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-29T08:55:35.869119+0000 front 2024-03-29T08:55:35.869168+0000 (oldest deadline 2024-03-29T08:56:01.768930+0000) 2024-03-29T08:56:22.306 INFO:journalctl@ceph.osd.5.smithi148.stdout:Mar 29 08:56:21 smithi148 ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4-osd-5[131123]: 2024-03-29T08:56:21.950+0000 7f6806200700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.37:6814 osd.1 since back 2024-03-29T08:55:45.869882+0000 front 2024-03-29T08:55:45.869913+0000 (oldest deadline 2024-03-29T08:56:11.769789+0000) 2024-03-29T08:56:22.306 INFO:journalctl@ceph.osd.5.smithi148.stdout:Mar 29 08:56:21 smithi148 ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4-osd-5[131123]: 2024-03-29T08:56:21.950+0000 7f6806200700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.37:6822 osd.2 since back 2024-03-29T08:55:55.870866+0000 front 2024-03-29T08:55:55.870847+0000 (oldest deadline 2024-03-29T08:56:21.170623+0000) 2024-03-29T08:56:23.306 INFO:journalctl@ceph.osd.5.smithi148.stdout:Mar 29 08:56:22 smithi148 ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4-osd-5[131123]: 2024-03-29T08:56:22.912+0000 7f6806200700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-03-29T08:55:35.869119+0000 front 2024-03-29T08:55:35.869168+0000 (oldest deadline 2024-03-29T08:56:01.768930+0000) 2024-03-29T08:56:23.306 INFO:journalctl@ceph.osd.5.smithi148.stdout:Mar 29 08:56:22 smithi148 ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4-osd-5[131123]: 2024-03-29T08:56:22.912+0000 7f6806200700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.37:6814 osd.1 since back 2024-03-29T08:55:45.869882+0000 front 2024-03-29T08:55:45.869913+0000 (oldest deadline 2024-03-29T08:56:11.769789+0000) 2024-03-29T08:56:23.306 INFO:journalctl@ceph.osd.5.smithi148.stdout:Mar 29 08:56:22 smithi148 ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4-osd-5[131123]: 2024-03-29T08:56:22.912+0000 7f6806200700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.37:6822 osd.2 since back 2024-03-29T08:55:55.870866+0000 front 2024-03-29T08:55:55.870847+0000 (oldest deadline 2024-03-29T08:56:21.170623+0000) 2024-03-29T08:56:24.556 INFO:journalctl@ceph.osd.5.smithi148.stdout:Mar 29 08:56:24 smithi148 bash[153424]: ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4-osd-5 2024-03-29T08:56:26.231 DEBUG:teuthology.orchestra.run.smithi148:> sudo pkill -f 'journalctl -f -n 0 -u ceph-99b492c2-eda1-11ee-b647-cb9ed24678a4@osd.5.service' 2024-03-29T08:56:26.318 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-29T08:56:26.319 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-03-29T08:56:26.319 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 99b492c2-eda1-11ee-b647-cb9ed24678a4 --force --keep-logs 2024-03-29T08:56:30.190 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi037.stderr:ceph-fuse[95187]: fuse finished with error 0 and tester_r 0 2024-03-29T08:56:50.825 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 99b492c2-eda1-11ee-b647-cb9ed24678a4 --force --keep-logs 2024-03-29T08:57:05.336 DEBUG:teuthology.orchestra.run.smithi037:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-29T08:57:05.415 DEBUG:teuthology.orchestra.run.smithi148:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-29T08:57:05.485 INFO:tasks.cephadm:Archiving crash dumps... 2024-03-29T08:57:05.487 DEBUG:teuthology.misc:Transferring archived files from smithi037:/var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/crash to /home/teuthworker/archive/vshankar-2024-03-28_11:21:59-fs-wip-vshankar-testing-20240326.105515-reef-testing-default-smithi/7628087/remote/smithi037/crash 2024-03-29T08:57:05.488 DEBUG:teuthology.orchestra.run.smithi037:> sudo tar c -f - -C /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/crash -- . 2024-03-29T08:57:05.556 INFO:teuthology.orchestra.run.smithi037.stderr:tar: /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/crash: Cannot open: No such file or directory 2024-03-29T08:57:05.557 INFO:teuthology.orchestra.run.smithi037.stderr:tar: Error is not recoverable: exiting now 2024-03-29T08:57:05.559 DEBUG:teuthology.misc:Transferring archived files from smithi148:/var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/crash to /home/teuthworker/archive/vshankar-2024-03-28_11:21:59-fs-wip-vshankar-testing-20240326.105515-reef-testing-default-smithi/7628087/remote/smithi148/crash 2024-03-29T08:57:05.561 DEBUG:teuthology.orchestra.run.smithi148:> sudo tar c -f - -C /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/crash -- . 2024-03-29T08:57:05.625 INFO:teuthology.orchestra.run.smithi148.stderr:tar: /var/lib/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/crash: Cannot open: No such file or directory 2024-03-29T08:57:05.625 INFO:teuthology.orchestra.run.smithi148.stderr:tar: Error is not recoverable: exiting now 2024-03-29T08:57:05.628 INFO:tasks.cephadm:Checking cluster log for badness... 2024-03-29T08:57:05.628 DEBUG:teuthology.orchestra.run.smithi037:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v OSD_DOWN | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | head -n 1 2024-03-29T08:57:05.706 INFO:teuthology.orchestra.run.smithi037.stderr:grep: /var/log/ceph/99b492c2-eda1-11ee-b647-cb9ed24678a4/ceph.log: No such file or directory 2024-03-29T08:57:05.711 INFO:tasks.cephadm:Compressing logs... 2024-03-29T08:57:05.711 DEBUG:teuthology.orchestra.run.smithi037:> sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-29T08:57:05.715 DEBUG:teuthology.orchestra.run.smithi148:> sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-29T08:57:05.778 INFO:teuthology.orchestra.run.smithi037.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-03-29T08:57:05.780 INFO:teuthology.orchestra.run.smithi148.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-03-29T08:57:23.615 INFO:teuthology.orchestra.run.smithi148.stderr:gzip: /var/log/ceph/ceph-client.1.log: file size changed while zipping 2024-03-29T08:59:52.833 INFO:tasks.cephadm:Archiving logs... 2024-03-29T08:59:52.834 DEBUG:teuthology.misc:Transferring archived files from smithi037:/var/log/ceph to /home/teuthworker/archive/vshankar-2024-03-28_11:21:59-fs-wip-vshankar-testing-20240326.105515-reef-testing-default-smithi/7628087/remote/smithi037/log 2024-03-29T08:59:52.835 DEBUG:teuthology.orchestra.run.smithi037:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-29T09:00:10.638 DEBUG:teuthology.misc:Transferring archived files from smithi148:/var/log/ceph to /home/teuthworker/archive/vshankar-2024-03-28_11:21:59-fs-wip-vshankar-testing-20240326.105515-reef-testing-default-smithi/7628087/remote/smithi148/log 2024-03-29T09:00:10.640 DEBUG:teuthology.orchestra.run.smithi148:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-29T09:00:16.211 INFO:tasks.cephadm:Removing cluster... 2024-03-29T09:00:16.211 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 99b492c2-eda1-11ee-b647-cb9ed24678a4 --force 2024-03-29T09:00:17.351 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 99b492c2-eda1-11ee-b647-cb9ed24678a4 --force 2024-03-29T09:00:18.149 INFO:tasks.cephadm:Removing cephadm ... 2024-03-29T09:00:18.149 DEBUG:teuthology.orchestra.run.smithi037:> rm -rf /home/ubuntu/cephtest/cephadm 2024-03-29T09:00:18.200 DEBUG:teuthology.orchestra.run.smithi148:> rm -rf /home/ubuntu/cephtest/cephadm 2024-03-29T09:00:18.246 INFO:tasks.cephadm:Teardown complete 2024-03-29T09:00:18.246 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-03-29T09:00:18.258 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/task/install/__init__.py", line 626, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_d0e392d9aee635f2b65768c460c7a151ead7a5e2/qa/tasks/ceph_fuse.py", line 183, in task mount.umount_wait() File "/home/teuthworker/src/git.ceph.com_ceph-c_d0e392d9aee635f2b65768c460c7a151ead7a5e2/qa/tasks/cephfs/fuse_mount.py", line 400, in umount_wait run.wait([self.fuse_daemon], timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 473, in wait check_time() File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds 2024-03-29T09:00:18.259 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-03-29T09:00:18.259 DEBUG:teuthology.orchestra.run.smithi037:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-29T09:00:18.264 DEBUG:teuthology.orchestra.run.smithi148:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-29T09:00:18.401 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-03-29T09:00:18.412 INFO:teuthology.task.clock:Checking final clock skew... 2024-03-29T09:00:18.412 DEBUG:teuthology.orchestra.run.smithi037:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-29T09:00:18.416 DEBUG:teuthology.orchestra.run.smithi148:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-29T09:00:18.456 INFO:teuthology.orchestra.run.smithi037.stderr:bash: ntpq: command not found 2024-03-29T09:00:18.459 INFO:teuthology.orchestra.run.smithi148.stderr:bash: ntpq: command not found 2024-03-29T09:00:18.464 INFO:teuthology.orchestra.run.smithi037.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-29T09:00:18.464 INFO:teuthology.orchestra.run.smithi037.stdout:=============================================================================== 2024-03-29T09:00:18.464 INFO:teuthology.orchestra.run.smithi037.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 28 +390us[ +390us] +/- 94ms 2024-03-29T09:00:18.464 INFO:teuthology.orchestra.run.smithi037.stdout:^* hv02.front.sepia.ceph.com 3 8 377 221 -546us[ -555us] +/- 62ms 2024-03-29T09:00:18.465 INFO:teuthology.orchestra.run.smithi037.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 27 +315us[ +315us] +/- 78ms 2024-03-29T09:00:18.465 INFO:teuthology.orchestra.run.smithi037.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-03-29T09:00:18.467 INFO:teuthology.orchestra.run.smithi148.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-29T09:00:18.467 INFO:teuthology.orchestra.run.smithi148.stdout:=============================================================================== 2024-03-29T09:00:18.467 INFO:teuthology.orchestra.run.smithi148.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 28 +439us[ +436us] +/- 94ms 2024-03-29T09:00:18.467 INFO:teuthology.orchestra.run.smithi148.stdout:^* hv02.front.sepia.ceph.com 3 7 377 26 -529us[ -532us] +/- 65ms 2024-03-29T09:00:18.467 INFO:teuthology.orchestra.run.smithi148.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 27 +328us[ +325us] +/- 78ms 2024-03-29T09:00:18.467 INFO:teuthology.orchestra.run.smithi148.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-03-29T09:00:18.468 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-03-29T09:00:18.480 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-03-29T09:00:18.481 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-03-29T09:00:18.492 DEBUG:teuthology.orchestra.run.smithi037:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-29T09:00:18.647 DEBUG:teuthology.orchestra.run.smithi148:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-29T09:00:18.781 DEBUG:teuthology.orchestra.run.smithi037:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-03-29T09:00:18.853 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-29T09:00:18.854 DEBUG:teuthology.orchestra.run.smithi148:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-03-29T09:00:18.926 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-29T09:00:18.926 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-03-29T09:00:18.938 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-03-29T09:00:18.950 INFO:teuthology.task.internal:Duration was 4608.983819 seconds 2024-03-29T09:00:18.950 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-03-29T09:00:18.965 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-03-29T09:00:18.965 DEBUG:teuthology.orchestra.run.smithi037:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-29T09:00:18.970 DEBUG:teuthology.orchestra.run.smithi148:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-29T09:00:19.079 INFO:teuthology.orchestra.run.smithi037.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-29T09:00:19.083 INFO:teuthology.orchestra.run.smithi148.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-29T09:00:19.337 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-03-29T09:00:19.337 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi037.front.sepia.ceph.com 2024-03-29T09:00:19.337 DEBUG:teuthology.orchestra.run.smithi037:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-29T09:00:19.396 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi148.front.sepia.ceph.com 2024-03-29T09:00:19.396 DEBUG:teuthology.orchestra.run.smithi148:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-29T09:00:19.455 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-03-29T09:00:19.456 DEBUG:teuthology.orchestra.run.smithi037:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-29T09:00:19.460 DEBUG:teuthology.orchestra.run.smithi148:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-29T09:00:19.569 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-03-29T09:00:19.570 DEBUG:teuthology.orchestra.run.smithi037:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-29T09:00:19.574 DEBUG:teuthology.orchestra.run.smithi148:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-29T09:00:19.775 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-03-29T09:00:19.788 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-03-29T09:00:19.788 DEBUG:teuthology.orchestra.run.smithi037:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-29T09:00:19.856 DEBUG:teuthology.orchestra.run.smithi148:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-29T09:00:19.922 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-03-29T09:00:19.935 DEBUG:teuthology.orchestra.run.smithi037:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-29T09:00:19.939 DEBUG:teuthology.orchestra.run.smithi148:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-29T09:00:20.002 INFO:teuthology.orchestra.run.smithi037.stdout:kernel.core_pattern = core 2024-03-29T09:00:20.005 INFO:teuthology.orchestra.run.smithi148.stdout:kernel.core_pattern = core 2024-03-29T09:00:20.045 DEBUG:teuthology.orchestra.run.smithi037:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-29T09:00:20.087 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-29T09:00:20.087 DEBUG:teuthology.orchestra.run.smithi148:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-29T09:00:20.130 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-29T09:00:20.130 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-03-29T09:00:20.143 INFO:teuthology.task.internal:Transferring archived files... 2024-03-29T09:00:20.144 DEBUG:teuthology.misc:Transferring archived files from smithi037:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/vshankar-2024-03-28_11:21:59-fs-wip-vshankar-testing-20240326.105515-reef-testing-default-smithi/7628087/remote/smithi037 2024-03-29T09:00:20.144 DEBUG:teuthology.orchestra.run.smithi037:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-29T09:00:20.232 DEBUG:teuthology.misc:Transferring archived files from smithi148:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/vshankar-2024-03-28_11:21:59-fs-wip-vshankar-testing-20240326.105515-reef-testing-default-smithi/7628087/remote/smithi148 2024-03-29T09:00:20.232 DEBUG:teuthology.orchestra.run.smithi148:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-29T09:00:20.315 INFO:teuthology.task.internal:Removing archive directory... 2024-03-29T09:00:20.315 DEBUG:teuthology.orchestra.run.smithi037:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-29T09:00:20.320 DEBUG:teuthology.orchestra.run.smithi148:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-29T09:00:20.364 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-03-29T09:00:20.378 INFO:teuthology.task.internal:Not uploading archives. 2024-03-29T09:00:20.378 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-03-29T09:00:20.390 INFO:teuthology.task.internal:Tidying up after the test... 2024-03-29T09:00:20.390 DEBUG:teuthology.orchestra.run.smithi037:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-29T09:00:20.405 DEBUG:teuthology.orchestra.run.smithi148:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-29T09:00:20.445 INFO:teuthology.orchestra.run.smithi037.stdout: 524301 4 drwxr-xr-x 3 ubuntu ubuntu 4096 Mar 29 09:00 /home/ubuntu/cephtest 2024-03-29T09:00:20.446 INFO:teuthology.orchestra.run.smithi037.stdout: 917757 4 d--------- 2 ubuntu ubuntu 4096 Mar 29 08:04 /home/ubuntu/cephtest/mnt.0 2024-03-29T09:00:20.446 INFO:teuthology.orchestra.run.smithi037.stderr:find: ‘/home/ubuntu/cephtest/mnt.0’: Permission denied 2024-03-29T09:00:20.447 INFO:teuthology.orchestra.run.smithi037.stderr:rmdir: failed to remove '/home/ubuntu/cephtest': Directory not empty 2024-03-29T09:00:20.448 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-29T09:00:20.448 ERROR:teuthology.run_tasks:Manager failed: internal.base Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/task/internal/__init__.py", line 48, in base yield File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_d0e392d9aee635f2b65768c460c7a151ead7a5e2/qa/tasks/ceph_fuse.py", line 183, in task mount.umount_wait() File "/home/teuthworker/src/git.ceph.com_ceph-c_d0e392d9aee635f2b65768c460c7a151ead7a5e2/qa/tasks/cephfs/fuse_mount.py", line 400, in umount_wait run.wait([self.fuse_daemon], timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 473, in wait check_time() File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/task/internal/__init__.py", line 53, in base run.wait( File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi037 with status 1: 'find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest' 2024-03-29T09:00:20.449 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-03-29T09:00:20.464 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-03-29T09:00:20.502 DEBUG:teuthology.run_tasks:Exception was not quenched, exiting: MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds 2024-03-29T09:00:20.502 INFO:teuthology.nuke:Checking targets against current locks 2024-03-29T09:00:20.529 DEBUG:teuthology.nuke:shortname: smithi037 2024-03-29T09:00:20.530 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-29T09:00:20.552 DEBUG:teuthology.nuke:shortname: smithi148 2024-03-29T09:00:20.552 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-29T09:00:20.556 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi037.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-03-28_11:21:59-fs-wip-vshankar-testing-20240326.105515-reef-testing-default-smithi/7628087', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-29 07:33:38.158020', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBG40xk/CPmL/al8W0cAbek/cWvcY1E1HDpyPXO/7dLFpI017G89+aK9jHEmCulUy1Vc1IwZ6wm57F+V5ShTIQwk='} 2024-03-29T09:00:20.581 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi148.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-03-28_11:21:59-fs-wip-vshankar-testing-20240326.105515-reef-testing-default-smithi/7628087', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-29 07:33:38.159370', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAtrgGk5u5xlk1Xeo4rxeod054uNJvPknXxGcaxLta9d8QZ6W3dYLKJuqZ01Wj4+0brVQe6hw1zwCHoWBZR4FW8='} 2024-03-29T09:00:20.605 INFO:teuthology.orchestra.console.smithi037:Power off 2024-03-29T09:00:20.605 DEBUG:teuthology.orchestra.console.smithi037:pexpect command: ipmitool -H smithi037.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-29T09:00:20.629 DEBUG:teuthology.orchestra.console.smithi037:power off output: Chassis Power Control: Down/Off 2024-03-29T09:00:20.629 DEBUG:teuthology.orchestra.console.smithi037:pexpect command: ipmitool -H smithi037.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-29T09:00:20.647 INFO:teuthology.orchestra.console.smithi148:Power off 2024-03-29T09:00:20.647 DEBUG:teuthology.orchestra.console.smithi148:pexpect command: ipmitool -H smithi148.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-29T09:00:20.658 DEBUG:teuthology.orchestra.console.smithi037:check power output: Chassis Power is on 2024-03-29T09:00:20.670 DEBUG:teuthology.orchestra.console.smithi148:power off output: Chassis Power Control: Down/Off 2024-03-29T09:00:20.670 DEBUG:teuthology.orchestra.console.smithi148:pexpect command: ipmitool -H smithi148.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-29T09:00:20.697 DEBUG:teuthology.orchestra.console.smithi148:check power output: Chassis Power is on 2024-03-29T09:00:24.660 DEBUG:teuthology.orchestra.console.smithi037:pexpect command: ipmitool -H smithi037.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-29T09:00:24.698 DEBUG:teuthology.orchestra.console.smithi148:pexpect command: ipmitool -H smithi148.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-29T09:00:24.771 DEBUG:teuthology.orchestra.console.smithi037:check power output: Chassis Power is on 2024-03-29T09:00:24.809 DEBUG:teuthology.orchestra.console.smithi148:check power output: Chassis Power is on 2024-03-29T09:00:28.775 DEBUG:teuthology.orchestra.console.smithi037:pexpect command: ipmitool -H smithi037.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-29T09:00:28.810 DEBUG:teuthology.orchestra.console.smithi148:pexpect command: ipmitool -H smithi148.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-29T09:00:28.886 DEBUG:teuthology.orchestra.console.smithi037:check power output: Chassis Power is on 2024-03-29T09:00:28.920 DEBUG:teuthology.orchestra.console.smithi148:check power output: Chassis Power is off 2024-03-29T09:00:29.020 INFO:teuthology.orchestra.console.smithi148:Power off completed 2024-03-29T09:00:32.890 DEBUG:teuthology.orchestra.console.smithi037:pexpect command: ipmitool -H smithi037.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-29T09:00:33.001 DEBUG:teuthology.orchestra.console.smithi037:check power output: Chassis Power is off 2024-03-29T09:00:33.102 INFO:teuthology.orchestra.console.smithi037:Power off completed 2024-03-29T09:00:33.206 INFO:teuthology.run:Summary data: description: fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} duration: 4608.9838190078735 failure_reason: reached maximum tries (51) after waiting for 300 seconds flavor: default host.a-kernel-sha1: 1df0c7e8a2354c42d738e5454ba83ea7332deac2 host.b-kernel-sha1: 1df0c7e8a2354c42d738e5454ba83ea7332deac2 owner: scheduled_vshankar@teuthology status: fail success: false 2024-03-29T09:00:33.207 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-29T09:00:33.465 INFO:teuthology.run:FAIL